From 5a180060562b16591b3049fde615aea28292460d Mon Sep 17 00:00:00 2001 From: Pelanyo Kamara Date: Wed, 3 Nov 2021 19:12:14 +0000 Subject: [PATCH 001/399] Update README.md (#99) Update README.md to highlight CPP code snippet. --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 89e560d3..7424e2e0 100644 --- a/README.md +++ b/README.md @@ -32,7 +32,7 @@ cmake --build . --target Luau.Repl.CLI Luau.Analyze.CLI --config RelWithDebInfo To integrate Luau into your CMake application projects, at the minimum you'll need to depend on `Luau.Compiler` and `Luau.VM` projects. From there you need to create a new Luau state (using Lua 5.x API such as `lua_newstate`), compile source to bytecode and load it into the VM like this: -``` +```cpp std::string bytecode = Luau::compile(source); // needs Luau/Compiler.h include if (luau_load(L, chunkname, bytecode.data(), bytecode.size()) == 0) return 1; /* return chunk main function */ From 05a41d5d54adf6277cc363dac7425e895fc38310 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Wed, 3 Nov 2021 12:13:42 -0700 Subject: [PATCH 002/399] Remove team restriction from RFC process documentation Fixes #98 --- rfcs/README.md | 2 -- 1 file changed, 2 deletions(-) diff --git a/rfcs/README.md b/rfcs/README.md index 4ebaeafe..f6c4c145 100644 --- a/rfcs/README.md +++ b/rfcs/README.md @@ -29,8 +29,6 @@ To open an RFC, a Pull Request must be opened which creates a new Markdown file **Please make sure to add `rfc` label to PRs *before* creating them!** This makes sure that our automatic notifications work correctly. -> Note: we currently don't accept community contributions for RFCs, although this will likely change in the future. - Every open RFC will be open for at least two calendar weeks. This is to make sure that there is sufficient time to review the proposal and raise concerns or suggest improvements. The discussion points should be reflected on the PR comments; when discussion happens outside of the comment stream, the points salient to the RFC should be summarized as a followup. When the initial comment period expires, the RFC can be merged if there's consensus that the change is important and that the details of the syntax/semantics presented are workable. The decision to merge the RFC is made by the Luau team. From e34586b847f2fc6fe45b0859cac408c8bb9ca258 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Wed, 3 Nov 2021 12:30:07 -0700 Subject: [PATCH 003/399] Update sandbox.md Soften the language around Rust --- docs/_pages/sandbox.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/_pages/sandbox.md b/docs/_pages/sandbox.md index e5d5e4d3..8457cd11 100644 --- a/docs/_pages/sandbox.md +++ b/docs/_pages/sandbox.md @@ -8,7 +8,7 @@ Luau is safe to embed. Broadly speaking, this means that even in the face of unt This safety is achieved through a combination of removing features from the standard library that are unsafe, adding features to the VM that make it possible to implement sandboxing and isolation, and making sure the implementation is safe from memory safety issues using fuzzing. -Of course, since the entire stack is implemented in C++, the sandboxing isn't formally proven - in theory, compiler or the standard library can have exploitable vulnerabilities. In practice these are usually found and fixed quickly. While implementing the stack in a safer language such as Rust would make it easier to provide these guarantees, to our knowledge (based on existing code) this would make it impossible to reach the level of performance required. +Of course, since the entire stack is implemented in C++, the sandboxing isn't formally proven - in theory, compiler or the standard library can have exploitable vulnerabilities. In practice these are usually found and fixed quickly. While implementing the stack in a safer language such as Rust would make it easier to provide these guarantees, to our knowledge (based on prior art) this would make it difficult to reach the level of performance required. ## Library From 2f7e1a2395e9fd97f0dcfaf320234309e34751a5 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Wed, 3 Nov 2021 12:30:30 -0700 Subject: [PATCH 004/399] Update CI scripts to ignore .md changes (#100) This reduces the load on GHA infra --- .github/workflows/build.yml | 5 +++++ .github/workflows/release.yml | 1 + 2 files changed, 6 insertions(+) diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml index a1c9db5b..cdee2f6c 100644 --- a/.github/workflows/build.yml +++ b/.github/workflows/build.yml @@ -4,6 +4,11 @@ on: push: branches: - 'master' + paths-ignore: + - 'docs/**' + - 'papers/**' + - 'rfcs/**' + - '*.md' pull_request: paths-ignore: - 'docs/**' diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index 8cfc27f6..2fd8d3d5 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -8,6 +8,7 @@ on: - 'docs/**' - 'papers/**' - 'rfcs/**' + - '*.md' jobs: build: From eed3c8c38fabee8d830a74a03748d4a87b66340b Mon Sep 17 00:00:00 2001 From: Amber's Careware <93293456+ambers-careware@users.noreply.github.com> Date: Wed, 3 Nov 2021 14:15:51 -0600 Subject: [PATCH 005/399] Update config-luaurc.md (#104) When running `luau-analyze` with a .luarc that has a "mode" key, it outputs the following: > .luaurc: Unknown key mode I'm assuming it was named "mode" at first and was re-named "languageMode" later on? --- rfcs/config-luaurc.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/rfcs/config-luaurc.md b/rfcs/config-luaurc.md index 0086a002..170142a6 100644 --- a/rfcs/config-luaurc.md +++ b/rfcs/config-luaurc.md @@ -29,20 +29,20 @@ Example of a valid .luaurc file: ```json5 { - "mode": "nonstrict", + "languageMode": "nonstrict", "lint": { "*": true, "LocalUnused": false }, "lintErrors": true, "globals": ["expect"] // TestEZ } ``` -Note that in absence of a configuration file, we will use default settings: mode will be set to nonstrict, a set of lint warnings is going to be enabled by default (this proposal doesn't detail that set - that will be subject to a different proposal), type checking issues are going to be treated as errors, lint issues are going to be treated as warnings. +Note that in absence of a configuration file, we will use default settings: languageMode will be set to nonstrict, a set of lint warnings is going to be enabled by default (this proposal doesn't detail that set - that will be subject to a different proposal), type checking issues are going to be treated as errors, lint issues are going to be treated as warnings. ## Design -- compatibility Today we support .robloxrc files; this proposal will keep parsing legacy specification of configuration for compatibility: -- Top-level `"language"` key can refer to an object that has `"mode"` key that also defines language mode +- Top-level `"language"` key can refer to an object that has `"languageMode"` key that also defines language mode - Top-level `"lint"` object values can refer to a string `"disabled"`/`"enabled"`/`"fatal"` instead of a boolean as a value. These keys are only going to be supported for compatibility and only when the file name is .robloxrc (which is only going to be parsed by internal Roblox command line tools but this proposal mentions it for completeness). From 06e79462d0cbbab8b6030939e334ea8ffdd2bc2a Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Wed, 3 Nov 2021 14:27:42 -0700 Subject: [PATCH 006/399] Update index.md Remove incorrect text about Luau being Roblox only. --- docs/index.md | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/docs/index.md b/docs/index.md index d5a78c4f..d77a9a3e 100644 --- a/docs/index.md +++ b/docs/index.md @@ -61,7 +61,7 @@ feature_row3: - title: Performance excerpt: > - In addition to a completely custom front end that implements parsing, linting and type checking, Luau runtime features new bytecode, interpreter and compiler that are heavily tuned for performance. Luau currently does not implement Just-In-Time compilation, but its interpreter is often competitive with LuaJIT interpreter on a wide set of benchmarks. We continue to optimize the runtime and rewrite portions of it to be even more efficient, including plans for a new garbage collector and further library optimizations, as well as an eventual JIT/AOT option. While our overall goal is to minimize the amount of time programmers spend tuning performance, some details about the performance characteristics are [provided for inquisitive minds](performance). + In addition to a completely custom front end that implements parsing, linting and type checking, Luau runtime features new bytecode, interpreter and compiler that are heavily tuned for performance. Luau currently does not implement Just-In-Time compilation, but its interpreter can be competitive with LuaJIT interpreter depending on the program. We continue to optimize the runtime and rewrite portions of it to be even more efficient. While our overall goal is to minimize the amount of time programmers spend tuning performance, some details about the performance characteristics are [provided for inquisitive minds](performance). url: /performance btn_label: Learn More btn_class: "btn--primary" @@ -69,7 +69,10 @@ feature_row3: - title: Libraries excerpt: > - As a language, Luau is a full superset of Lua 5.1. As far as standard library is concerned, some functions had to be removed from the builtin libraries, and some functions had to be added. Additionally, Luau is currently only runnable from the context of the Roblox engine, which exposes a large API surface [documented on Roblox developer portal](https://developer.roblox.com/en-us/api-reference). + As a language, Luau is a full superset of Lua 5.1. As far as standard library is concerned, some functions had to be removed from the builtin libraries, and some functions had to be added. When Luau is embedded into an application, the scripts normally get access to extra library features that are application-specific. + url: /library + btn_label: Learn More + btn_class: "btn--primary" --- From 3f0eb426c9461b40ac1c16879bc6ea6b925d3709 Mon Sep 17 00:00:00 2001 From: Peter van Dijk Date: Wed, 3 Nov 2021 22:37:15 +0100 Subject: [PATCH 007/399] cmake --target takes one argument, not two (#105) Adjust README so that we can work with CMake versions before 3.15 --- README.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index 7424e2e0..116f92a3 100644 --- a/README.md +++ b/README.md @@ -27,7 +27,8 @@ To build Luau tools or tests yourself, you can use CMake on all platforms, or al ```sh mkdir cmake && cd cmake cmake .. -DCMAKE_BUILD_TYPE=RelWithDebInfo -cmake --build . --target Luau.Repl.CLI Luau.Analyze.CLI --config RelWithDebInfo +cmake --build . --target Luau.Repl.CLI --config RelWithDebInfo +cmake --build . --target Luau.Analyze.CLI --config RelWithDebInfo ``` To integrate Luau into your CMake application projects, at the minimum you'll need to depend on `Luau.Compiler` and `Luau.VM` projects. From there you need to create a new Luau state (using Lua 5.x API such as `lua_newstate`), compile source to bytecode and load it into the VM like this: From 733ae0498d083707f48b263e637dd37bbe712e94 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Wed, 3 Nov 2021 19:10:49 -0700 Subject: [PATCH 008/399] Update index.md Cleanup text and remove extra buttons and redundant links --- docs/index.md | 34 +++++----------------------------- 1 file changed, 5 insertions(+), 29 deletions(-) diff --git a/docs/index.md b/docs/index.md index d77a9a3e..d7a344f8 100644 --- a/docs/index.md +++ b/docs/index.md @@ -6,10 +6,7 @@ permalink: / header: overlay_color: #000 overlay_filter: 0.8 - overlay_image: /assets/images/luau-header.png - actions: - - label: Learn More - url: /why + overlay_image: /assets/images/luau-header.png excerpt: > Lua*u* (lowercase *u*, /ˈlu.aʊ/) is a fast, small, safe, gradually typed embeddable scripting language derived from Lua. @@ -18,26 +15,17 @@ feature_row1: - title: Motivation excerpt: > - Around 2006, [Roblox](https://www.roblox.com) started using Lua 5.1 as a scripting language for games. Over the years we ended up substantially evolving the implementation and the language; to support growing sophistication of games on the Roblox platform, growing team sizes and large internal teams writing a lot of code for application/editor (1+MLOC as of 2020), we had to invest in performance, ease of use and language tooling, and introduce a gradual type system to the language. - url: /why - btn_label: Learn More - btn_class: "btn--primary" + Around 2006, [Roblox](https://www.roblox.com) started using Lua 5.1 as a scripting language for games. Over the years we ended up substantially evolving the implementation and the language; to support growing sophistication of games on the Roblox platform, growing team sizes and large internal teams writing a lot of code for application/editor (1+MLOC as of 2020), we had to invest in performance, ease of use and language tooling, and introduce a gradual type system to the language. [More...](/why) - title: Sandboxing excerpt: > - Luau limits the set of standard libraries exposed to the users and implements extra sandboxing features to be able to run unprivileged code (written by our game developers) side by side with privileged code (written by us). This results in an execution environment that is different from what is commonplace in Lua. - url: /sandbox - btn_label: Learn More - btn_class: "btn--primary" + Luau limits the set of standard libraries exposed to the users and implements extra sandboxing features to be able to run unprivileged code (written by our game developers) side by side with privileged code (written by us). This results in an execution environment that is different from what is commonplace in Lua. [More...](/sandbox) - title: Compatibility excerpt: > - Whenever possible, Luau aims to be backwards-compatible with Lua 5.1 and at the same time to incorporate features from later revisions of Lua. However, Luau is not a full superset of later versions of Lua - we do not agree with some design decisions made by the Lua authors, and have different use cases and constraints. All post-5.1 Lua features, along with their support status in Luau, [are documented here](compatibility). - url: /compatibility - btn_label: Learn More - btn_class: "btn--primary" + Whenever possible, Luau aims to be backwards-compatible with Lua 5.1 and at the same time to incorporate features from later revisions of Lua. However, Luau is not a full superset of later versions of Lua - we do not always agree with Lua design decisions, and have different use cases and constraints. All post-5.1 Lua features, along with their support status in Luau, [are documented here](compatibility). feature_row2: - @@ -45,34 +33,22 @@ feature_row2: image_path: /assets/images/example.png excerpt: > Luau is syntactically backwards-compatible with Lua 5.1 (code that is valid Lua 5.1 is also valid Luau); however, we have extended the language with a set of syntactical features that make the language more familiar and ergonomic. The syntax [is described here](syntax). - url: /syntax - btn:label: Learn More - btn_class: "btn--primary" feature_row3: - title: Analysis excerpt: > To make it easier to write correct code, Luau comes with a set of analysis tools that can surface common mistakes. These consist of a linter and a type checker, colloquially known as script analysis, and can be used from [Roblox Studio](https://developer.roblox.com/en-us/articles/The-Script-Analysis-Tool). The linting passes are [described here](lint), and the type checking user guide can [be found here](typecheck). - url: /typecheck - btn_label: Learn More - btn_class: "btn--primary" - title: Performance excerpt: > In addition to a completely custom front end that implements parsing, linting and type checking, Luau runtime features new bytecode, interpreter and compiler that are heavily tuned for performance. Luau currently does not implement Just-In-Time compilation, but its interpreter can be competitive with LuaJIT interpreter depending on the program. We continue to optimize the runtime and rewrite portions of it to be even more efficient. While our overall goal is to minimize the amount of time programmers spend tuning performance, some details about the performance characteristics are [provided for inquisitive minds](performance). - url: /performance - btn_label: Learn More - btn_class: "btn--primary" - title: Libraries excerpt: > - As a language, Luau is a full superset of Lua 5.1. As far as standard library is concerned, some functions had to be removed from the builtin libraries, and some functions had to be added. When Luau is embedded into an application, the scripts normally get access to extra library features that are application-specific. - url: /library - btn_label: Learn More - btn_class: "btn--primary" + As a language, Luau is a full superset of Lua 5.1. As far as standard library is concerned, some functions had to be removed from the builtin libraries, and some functions had to be added; refer to [full documentation](/library) for details. When Luau is embedded into an application, the scripts normally get access to extra library features that are application-specific. --- From e3f8c25e9e45ce8357bad3457804a6dd438dbc4c Mon Sep 17 00:00:00 2001 From: Grant Hernandez Date: Wed, 3 Nov 2021 21:17:07 -0700 Subject: [PATCH 009/399] Fix CLI abort when non lua string passed to error (#114) --- CLI/Repl.cpp | 24 ++++++++++++++++++++++-- 1 file changed, 22 insertions(+), 2 deletions(-) diff --git a/CLI/Repl.cpp b/CLI/Repl.cpp index 6baa21ea..f57a5009 100644 --- a/CLI/Repl.cpp +++ b/CLI/Repl.cpp @@ -169,7 +169,17 @@ static std::string runCode(lua_State* L, const std::string& source) } else { - std::string error = (status == LUA_YIELD) ? "thread yielded unexpectedly" : lua_tostring(T, -1); + std::string error; + + if (status == LUA_YIELD) + { + error = "thread yielded unexpectedly"; + } + else if (const char* str = lua_tostring(L, -1)) + { + error = str; + } + error += "\nstack backtrace:\n"; error += lua_debugtrace(T); @@ -322,7 +332,17 @@ static bool runFile(const char* name, lua_State* GL) } else { - std::string error = (status == LUA_YIELD) ? "thread yielded unexpectedly" : lua_tostring(L, -1); + std::string error; + + if (status == LUA_YIELD) + { + error = "thread yielded unexpectedly"; + } + else if (const char* str = lua_tostring(L, -1)) + { + error = str; + } + error += "\nstacktrace:\n"; error += lua_debugtrace(L); From 344d37f0b113ba1a473c3807739bd2e53e786dc1 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Valts=20Liepi=C5=86=C5=A1?= Date: Thu, 4 Nov 2021 06:23:20 +0200 Subject: [PATCH 010/399] Fixes incorrect MismatchedCount error message when returning (#103) --- Analysis/src/Error.cpp | 21 ++++++++++----------- Analysis/src/Unifier.cpp | 6 +++--- tests/TypeInfer.test.cpp | 21 ++++++++++++++++++++- 3 files changed, 33 insertions(+), 15 deletions(-) diff --git a/Analysis/src/Error.cpp b/Analysis/src/Error.cpp index 680bcf3f..a622f8a5 100644 --- a/Analysis/src/Error.cpp +++ b/Analysis/src/Error.cpp @@ -112,21 +112,20 @@ struct ErrorConverter std::string operator()(const Luau::CountMismatch& e) const { + const std::string expectedS = e.expected == 1 ? "" : "s"; + const std::string actualS = e.actual == 1 ? "" : "s"; + const std::string actualVerb = e.actual == 1 ? "is" : "are"; + switch (e.context) { case CountMismatch::Return: - { - const std::string expectedS = e.expected == 1 ? "" : "s"; - const std::string actualS = e.actual == 1 ? "is" : "are"; - return "Expected to return " + std::to_string(e.expected) + " value" + expectedS + ", but " + std::to_string(e.actual) + " " + actualS + - " returned here"; - } + return "Expected to return " + std::to_string(e.expected) + " value" + expectedS + ", but " + + std::to_string(e.actual) + " " + actualVerb + " returned here"; case CountMismatch::Result: - if (e.expected > e.actual) - return "Function returns " + std::to_string(e.expected) + " values but there are only " + std::to_string(e.expected) + - " values to unpack them into."; - else - return "Function only returns " + std::to_string(e.expected) + " values. " + std::to_string(e.actual) + " are required here"; + // It is alright if right hand side produces more values than the + // left hand side accepts. In this context consider only the opposite case. + return "Function only returns " + std::to_string(e.expected) + " value" + expectedS + ". " + + std::to_string(e.actual) + " are required here"; case CountMismatch::Arg: return "Argument count mismatch. Function " + wrongNumberOfArgsString(e.expected, e.actual); } diff --git a/Analysis/src/Unifier.cpp b/Analysis/src/Unifier.cpp index 89c3f80c..07aee38e 100644 --- a/Analysis/src/Unifier.cpp +++ b/Analysis/src/Unifier.cpp @@ -626,11 +626,11 @@ void Unifier::tryUnify_(TypePackId superTp, TypePackId subTp, bool isFunctionCal } // This is a bit weird because we don't actually know expected vs actual. We just know - // subtype vs supertype. If we are checking a return value, we swap these to produce - // the expected error message. + // subtype vs supertype. If we are checking the values returned by a function, we swap + // these to produce the expected error message. size_t expectedSize = size(superTp); size_t actualSize = size(subTp); - if (ctx == CountMismatch::Result || ctx == CountMismatch::Return) + if (ctx == CountMismatch::Result) std::swap(expectedSize, actualSize); errors.push_back(TypeError{location, CountMismatch{expectedSize, actualSize, ctx}}); diff --git a/tests/TypeInfer.test.cpp b/tests/TypeInfer.test.cpp index 37333b19..d8dc0fbd 100644 --- a/tests/TypeInfer.test.cpp +++ b/tests/TypeInfer.test.cpp @@ -3251,7 +3251,24 @@ TEST_CASE_FIXTURE(Fixture, "too_many_return_values") CountMismatch* acm = get(result.errors[0]); REQUIRE(acm); - CHECK(acm->context == CountMismatch::Result); + CHECK_EQ(acm->context, CountMismatch::Result); + CHECK_EQ(acm->expected, 1); + CHECK_EQ(acm->actual, 2); +} + +TEST_CASE_FIXTURE(Fixture, "ignored_return_values") +{ + CheckResult result = check(R"( + --!strict + + function f() + return 55, "" + end + + local a = f() + )"); + + LUAU_REQUIRE_ERROR_COUNT(0, result); } TEST_CASE_FIXTURE(Fixture, "function_does_not_return_enough_values") @@ -3269,6 +3286,8 @@ TEST_CASE_FIXTURE(Fixture, "function_does_not_return_enough_values") CountMismatch* acm = get(result.errors[0]); REQUIRE(acm); CHECK_EQ(acm->context, CountMismatch::Return); + CHECK_EQ(acm->expected, 2); + CHECK_EQ(acm->actual, 1); } TEST_CASE_FIXTURE(Fixture, "typecheck_unary_minus") From 1f0084daa35ede85e394e0fefe132ff467e18867 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Wed, 3 Nov 2021 21:35:25 -0700 Subject: [PATCH 011/399] Mark RFCs that were implemented as such. --- rfcs/config-luaurc.md | 2 ++ rfcs/function-table-freeze.md | 2 ++ rfcs/generic-functions.md | 2 ++ rfcs/recursive-type-restriction.md | 6 ++++++ rfcs/syntax-if-expression.md | 2 ++ rfcs/unsealed-table-assign-optional-property.md | 4 +++- 6 files changed, 17 insertions(+), 1 deletion(-) diff --git a/rfcs/config-luaurc.md b/rfcs/config-luaurc.md index 170142a6..b321b94b 100644 --- a/rfcs/config-luaurc.md +++ b/rfcs/config-luaurc.md @@ -1,5 +1,7 @@ # Configure analysis via .luaurc +**Status**: Implemented + ## Summary Introduces a way to configure type checker and linter using JSON-like .luaurc files diff --git a/rfcs/function-table-freeze.md b/rfcs/function-table-freeze.md index a99da64f..47de0f9e 100644 --- a/rfcs/function-table-freeze.md +++ b/rfcs/function-table-freeze.md @@ -2,6 +2,8 @@ > Note: this RFC was adapted from an internal proposal that predates RFC process +**Status**: Implemented + ## Summary Add `table.freeze` which allows to make a table read-only in a shallow way. diff --git a/rfcs/generic-functions.md b/rfcs/generic-functions.md index 1e9add96..59ec2d4a 100644 --- a/rfcs/generic-functions.md +++ b/rfcs/generic-functions.md @@ -1,5 +1,7 @@ # Generic functions +**Status**: Implemented + ## Summary Extend the syntax and semantics of functions to support explicit generic functions, which can bind type parameters as well as data parameters. diff --git a/rfcs/recursive-type-restriction.md b/rfcs/recursive-type-restriction.md index eb9b2124..6f69d43a 100644 --- a/rfcs/recursive-type-restriction.md +++ b/rfcs/recursive-type-restriction.md @@ -1,5 +1,11 @@ # Recursive type restriction +**Status**: Implemented + +## Summary + +Restrict generic type aliases to only be able to refer to the exact same instantiation of the generic that's being declared. + ## Motivation Luau supports recursive type aliases, but with an important restriction: diff --git a/rfcs/syntax-if-expression.md b/rfcs/syntax-if-expression.md index c900409e..9fc85560 100644 --- a/rfcs/syntax-if-expression.md +++ b/rfcs/syntax-if-expression.md @@ -2,6 +2,8 @@ > Note: this RFC was adapted from an internal proposal that predates RFC process +**Status**: Implemented + ## Summary Introduce a form of ternary conditional using `if cond then value else alternative` syntax. diff --git a/rfcs/unsealed-table-assign-optional-property.md b/rfcs/unsealed-table-assign-optional-property.md index 78f674aa..ed037b14 100644 --- a/rfcs/unsealed-table-assign-optional-property.md +++ b/rfcs/unsealed-table-assign-optional-property.md @@ -1,5 +1,7 @@ # Unsealed table assignment creates an optional property +## Summary + In Luau, tables have a state, which can, among others, be "unsealed". An unsealed table is one that we are still constructing. Currently assigning a table literal to an unsealed table does not introduce new @@ -53,4 +55,4 @@ and so needs access to an allocator. ## Alternatives Rather than introducing optional properties, we could introduce an indexer. For example we could infer the type of -`t` as `{ u: { [string]: number } }`. \ No newline at end of file +`t` as `{ u: { [string]: number } }`. From 3462a08d4a8e961fa951deebc47723fd5c716845 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Wed, 3 Nov 2021 22:08:23 -0700 Subject: [PATCH 012/399] Update CONTRIBUTING.md Add a section on performance. --- CONTRIBUTING.md | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index b48066ee..6eee61d3 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -47,6 +47,12 @@ You can run the tests yourself using `make test` or using `cmake` to build `Luau When making code changes please try to make sure they are covered by an existing test or add a new test accordingly. +## Performance + +One of the central feature of Luau is performance; our runtime in particular is heavily optimized for high performance and low memory consumption, and code is generally carefully tuned to result in close to optimal assembly for x64 and AArch64 architectures. The analysis code is not optimized to the same level of detail, but performance is still very important to make sure that we can support interactive IDE features. + +As such, it's important to make sure that the changes, including bug fixes, improve or at least do not regress performance. For VM this can be validated by running `bench.py` script from `bench` folder on two binaries built in Release mode, before and after the changes, although note that our benchmark coverage is not complete and in some cases additional performance testing will be necessary to determine if the change can be merged. + ## Feature flags For large bug fixes or features that apply to the Luau components and not just the CLI tools, we may ask that you introduce a feature flag to gate your changes. The feature flags use `LUAU_FASTFLAG` macro family defined in `Luau/Common.h` and allow us to ensure that the change can be safely shipped, enabled, and rolled back on the Roblox platform when the change makes it into our production codebase. The tests run the code with flags in their default state and enabled state as well to ensure correctness. From 57a42c4cb9b787825a187b610b221bd6a7830257 Mon Sep 17 00:00:00 2001 From: Tommy Vadakumchery Date: Thu, 4 Nov 2021 09:00:41 -0500 Subject: [PATCH 013/399] Fix HATRA '21 README.md formatting (#118) A misplaced backtick was causing the last two codeblocks from displaying correctly. --- papers/hatra21/README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/papers/hatra21/README.md b/papers/hatra21/README.md index 2123f26e..3105bb5b 100644 --- a/papers/hatra21/README.md +++ b/papers/hatra21/README.md @@ -27,10 +27,10 @@ sudo tlmgr install preprint sudo tlmgr install libertine sudo tlmgr install inconsolata sudo tlmgr install newtx -sudo tlmgr install latexmk` +sudo tlmgr install latexmk sudo tlmgr install montserrat sudo tlmgr install ly1 -`` +``` ## Building the paper From 278e848cc2ded5c12b80678205ebfa2e44145217 Mon Sep 17 00:00:00 2001 From: Josh Soref <2119212+jsoref@users.noreply.github.com> Date: Thu, 4 Nov 2021 10:50:46 -0400 Subject: [PATCH 014/399] Spelling (#119) Fixed various spelling errors. Co-authored-by: Josh Soref --- Analysis/include/Luau/TypeVar.h | 2 +- Analysis/src/Autocomplete.cpp | 2 +- Analysis/src/RequireTracer.cpp | 2 +- Analysis/src/TypeInfer.cpp | 6 +++--- Analysis/src/Unifier.cpp | 2 +- Ast/src/Parser.cpp | 2 +- Compiler/include/Luau/Bytecode.h | 4 ++-- Compiler/src/Compiler.cpp | 14 +++++++------- README.md | 2 +- VM/include/lualib.h | 2 +- VM/src/laux.cpp | 2 +- VM/src/lcorolib.cpp | 2 +- VM/src/ldo.cpp | 2 +- VM/src/ldo.h | 2 +- VM/src/lfunc.cpp | 2 +- VM/src/lgc.cpp | 8 ++++---- VM/src/lgc.h | 2 +- bench/tests/deltablue.lua | 8 ++++---- bench/tests/shootout/scimark.lua | 2 +- docs/_pages/library.md | 8 ++++---- docs/_pages/performance.md | 2 +- rfcs/function-debug-info.md | 2 +- rfcs/function-table-freeze.md | 2 +- rfcs/syntax-continue-statement.md | 2 +- rfcs/syntax-if-expression.md | 4 ++-- tests/TypeInfer.builtins.test.cpp | 2 +- tests/TypeInfer.provisional.test.cpp | 2 +- tests/TypeInfer.test.cpp | 14 +++++++------- tests/TypeInfer.unionTypes.test.cpp | 4 ++-- tests/conformance/closure.lua | 2 +- tests/conformance/coroutine.lua | 2 +- tests/conformance/gc.lua | 2 +- tests/conformance/locals.lua | 2 +- tests/conformance/math.lua | 2 +- tests/conformance/pm.lua | 4 ++-- 35 files changed, 62 insertions(+), 62 deletions(-) diff --git a/Analysis/include/Luau/TypeVar.h b/Analysis/include/Luau/TypeVar.h index 90a28b20..2e028df3 100644 --- a/Analysis/include/Luau/TypeVar.h +++ b/Analysis/include/Luau/TypeVar.h @@ -389,7 +389,7 @@ const std::string* getName(TypeId type); // Checks whether a union contains all types of another union. bool isSubset(const UnionTypeVar& super, const UnionTypeVar& sub); -// Checks if a type conains generic type binders +// Checks if a type contains generic type binders bool isGeneric(const TypeId ty); // Checks if a type may be instantiated to one containing generic type binders diff --git a/Analysis/src/Autocomplete.cpp b/Analysis/src/Autocomplete.cpp index dce92a0c..1cfa90dc 100644 --- a/Analysis/src/Autocomplete.cpp +++ b/Analysis/src/Autocomplete.cpp @@ -1461,7 +1461,7 @@ static AutocompleteResult autocomplete(const SourceModule& sourceModule, const M result.erase(std::string(stringKey->value.data, stringKey->value.size)); } - // If we know for sure that a key is being written, do not offer general epxression suggestions + // If we know for sure that a key is being written, do not offer general expression suggestions if (!key) autocompleteExpression(sourceModule, *module, typeChecker, typeArena, finder.ancestry, position, result); diff --git a/Analysis/src/RequireTracer.cpp b/Analysis/src/RequireTracer.cpp index 5b3997e2..c9e15cba 100644 --- a/Analysis/src/RequireTracer.cpp +++ b/Analysis/src/RequireTracer.cpp @@ -169,7 +169,7 @@ struct RequireTracer : AstVisitor result.exprs[call] = fileResolver->concat(*rootName, v); - // 'WaitForChild' can be used on modules that are not awailable at the typecheck time, but will be awailable at runtime + // 'WaitForChild' can be used on modules that are not available at the typecheck time, but will be available at runtime // If we fail to find such module, we will not report an UnknownRequire error if (FFlag::LuauTraceRequireLookupChild && indexName->index == "WaitForChild") result.optional[call] = true; diff --git a/Analysis/src/TypeInfer.cpp b/Analysis/src/TypeInfer.cpp index 2216881b..5c5217e7 100644 --- a/Analysis/src/TypeInfer.cpp +++ b/Analysis/src/TypeInfer.cpp @@ -412,7 +412,7 @@ void TypeChecker::checkBlock(const ScopePtr& scope, const AstStatBlock& block) // ``` // These both call each other, so `f` will be ordered before `g`, so the call to `g` // is typechecked before `g` has had its body checked. For this reason, there's three - // types for each functuion: before its body is checked, during checking its body, + // types for each function: before its body is checked, during checking its body, // and after its body is checked. // // We currently treat the before-type and the during-type as the same, @@ -1076,7 +1076,7 @@ void TypeChecker::check(const ScopePtr& scope, TypeId ty, const ScopePtr& funSco checkFunctionBody(funScope, ty, *function.func); // If in nonstrict mode and allowing redefinition of global function, restore the previous definition type - // in case this function has a differing signature. The signature discrepency will be caught in checkBlock. + // in case this function has a differing signature. The signature discrepancy will be caught in checkBlock. if (previouslyDefined) globalBindings[name] = oldBinding; else @@ -3150,7 +3150,7 @@ void TypeChecker::checkArgumentList( const ScopePtr& scope, Unifier& state, TypePackId argPack, TypePackId paramPack, const std::vector& argLocations) { /* Important terminology refresher: - * A function requires paramaters. + * A function requires parameters. * To call a function, you supply arguments. */ TypePackIterator argIter = begin(argPack); diff --git a/Analysis/src/Unifier.cpp b/Analysis/src/Unifier.cpp index 07aee38e..96795999 100644 --- a/Analysis/src/Unifier.cpp +++ b/Analysis/src/Unifier.cpp @@ -1025,7 +1025,7 @@ void Unifier::tryUnifySealedTables(TypeId left, TypeId right, bool isIntersectio // If the superTy/left is an immediate part of an intersection type, do not do extra-property check. // Otherwise, we would falsely generate an extra-property-error for 's' in this code: // local a: {n: number} & {s: string} = {n=1, s=""} - // When checking agaist the table '{n: number}'. + // When checking against the table '{n: number}'. if (!isIntersection && lt->state != TableState::Unsealed && !lt->indexer) { // Check for extra properties in the subTy diff --git a/Ast/src/Parser.cpp b/Ast/src/Parser.cpp index 6672efe8..9794a037 100644 --- a/Ast/src/Parser.cpp +++ b/Ast/src/Parser.cpp @@ -1535,7 +1535,7 @@ AstType* Parser::parseSimpleTypeAnnotation() { Location location = lexer.current().location; - // For a missing type annoation, capture 'space' between last token and the next one + // For a missing type annotation, capture 'space' between last token and the next one location = Location(lexer.previousLocation().end, lexer.current().location.begin); return reportTypeAnnotationError(location, {}, /*isMissing*/ true, "Expected type, got %s", lexer.current().toString().c_str()); diff --git a/Compiler/include/Luau/Bytecode.h b/Compiler/include/Luau/Bytecode.h index 07be2e74..4b03ed1c 100644 --- a/Compiler/include/Luau/Bytecode.h +++ b/Compiler/include/Luau/Bytecode.h @@ -208,14 +208,14 @@ enum LuauOpcode LOP_MODK, LOP_POWK, - // AND, OR: perform `and` or `or` operation (selecting first or second register based on whether the first one is truthful) and put the result into target register + // AND, OR: perform `and` or `or` operation (selecting first or second register based on whether the first one is truthy) and put the result into target register // A: target register // B: source register 1 // C: source register 2 LOP_AND, LOP_OR, - // ANDK, ORK: perform `and` or `or` operation (selecting source register or constant based on whether the source register is truthful) and put the result into target register + // ANDK, ORK: perform `and` or `or` operation (selecting source register or constant based on whether the source register is truthy) and put the result into target register // A: target register // B: source register // C: constant table index (0..255) diff --git a/Compiler/src/Compiler.cpp b/Compiler/src/Compiler.cpp index 022eccb7..21564248 100644 --- a/Compiler/src/Compiler.cpp +++ b/Compiler/src/Compiler.cpp @@ -712,9 +712,9 @@ struct Compiler } // compile expr to target temp register - // if the expr (or not expr if onlyTruth is false) is truthful, jump via skipJump - // if the expr (or not expr if onlyTruth is false) is falseful, fall through (target isn't guaranteed to be updated in this case) - // if target is omitted, then the jump behavior is the same - skipJump or fallthrough depending on the truthfulness of the expression + // if the expr (or not expr if onlyTruth is false) is truthy, jump via skipJump + // if the expr (or not expr if onlyTruth is false) is falsy, fall through (target isn't guaranteed to be updated in this case) + // if target is omitted, then the jump behavior is the same - skipJump or fallthrough depending on the truthiness of the expression void compileConditionValue(AstExpr* node, const uint8_t* target, std::vector& skipJump, bool onlyTruth) { // Optimization: we don't need to compute constant values @@ -722,7 +722,7 @@ struct Compiler if (cv && cv->type != Constant::Type_Unknown) { - // note that we only need to compute the value if it's truthful; otherwise we cal fall through + // note that we only need to compute the value if it's truthy; otherwise we cal fall through if (cv->isTruthful() == onlyTruth) { if (target) @@ -741,7 +741,7 @@ struct Compiler case AstExprBinary::And: case AstExprBinary::Or: { - // disambiguation: there's 4 cases (we only need truthful or falseful results based on onlyTruth) + // disambiguation: there's 4 cases (we only need truthy or falsy results based on onlyTruth) // onlyTruth = 1: a and b transforms to a ? b : dontcare // onlyTruth = 1: a or b transforms to a ? a : a // onlyTruth = 0: a and b transforms to !a ? a : b @@ -785,8 +785,8 @@ struct Compiler if (target) { // since target is a temp register, we'll initialize it to 1, and then jump if the comparison is true - // if the comparison is false, we'll fallthrough and target will still be 1 but target has unspecified value for falseful results - // when we only care about falseful values instead of truthful values, the process is the same but with flipped conditionals + // if the comparison is false, we'll fallthrough and target will still be 1 but target has unspecified value for falsy results + // when we only care about falsy values instead of truthy values, the process is the same but with flipped conditionals bytecode.emitABC(LOP_LOADB, *target, onlyTruth ? 1 : 0, 0); } diff --git a/README.md b/README.md index 116f92a3..81b30337 100644 --- a/README.md +++ b/README.md @@ -45,7 +45,7 @@ To gain advantage of many performance improvements it's highly recommended to us # Testing -Luau has an internal test suite; in CMake builds it is split into two targets, `Luau.UnitTest` (for bytecode compiler and type checker/linter tests) and `Luau.Conformance` (for VM tests). The unit tests are written in C++, whereas the conformance tests are largerly written in Luau (see `tests/conformance`). +Luau has an internal test suite; in CMake builds it is split into two targets, `Luau.UnitTest` (for bytecode compiler and type checker/linter tests) and `Luau.Conformance` (for VM tests). The unit tests are written in C++, whereas the conformance tests are largely written in Luau (see `tests/conformance`). Makefile builds combine both into a single target and can be ran via `make test`. diff --git a/VM/include/lualib.h b/VM/include/lualib.h index 7a09ae9f..30cffaff 100644 --- a/VM/include/lualib.h +++ b/VM/include/lualib.h @@ -76,7 +76,7 @@ struct luaL_Buffer char buffer[LUA_BUFFERSIZE]; }; -// when internal buffer storage is exhaused, a mutable string value 'storage' will be placed on the stack +// when internal buffer storage is exhausted, a mutable string value 'storage' will be placed on the stack // in general, functions expect the mutable string buffer to be placed on top of the stack (top-1) // with the exception of luaL_addvalue that expects the value at the top and string buffer further away (top-2) // functions that accept a 'boxloc' support string buffer placement at any location in the stack diff --git a/VM/src/laux.cpp b/VM/src/laux.cpp index e37618f7..2a684ee4 100644 --- a/VM/src/laux.cpp +++ b/VM/src/laux.cpp @@ -313,7 +313,7 @@ static size_t getnextbuffersize(lua_State* L, size_t currentsize, size_t desired { size_t newsize = currentsize + currentsize / 2; - // check for size oveflow + // check for size overflow if (SIZE_MAX - desiredsize < currentsize) luaL_error(L, "buffer too large"); diff --git a/VM/src/lcorolib.cpp b/VM/src/lcorolib.cpp index c0b50b96..a2515ba0 100644 --- a/VM/src/lcorolib.cpp +++ b/VM/src/lcorolib.cpp @@ -25,7 +25,7 @@ static int costatus(lua_State* L, lua_State* co) return CO_SUS; if (co->status == LUA_BREAK) return CO_NOR; - if (co->status != 0) /* some error occured */ + if (co->status != 0) /* some error occurred */ return CO_DEAD; if (co->ci != co->base_ci) /* does it have frames? */ return CO_NOR; diff --git a/VM/src/ldo.cpp b/VM/src/ldo.cpp index ee4962a5..77366714 100644 --- a/VM/src/ldo.cpp +++ b/VM/src/ldo.cpp @@ -532,7 +532,7 @@ int luaD_pcall(lua_State* L, Pfunc func, void* u, ptrdiff_t old_top, ptrdiff_t e status = LUA_ERRERR; } - // an error occured, check if we have a protected error callback + // an error occurred, check if we have a protected error callback if (L->global->cb.debugprotectederror) { L->global->cb.debugprotectederror(L); diff --git a/VM/src/ldo.h b/VM/src/ldo.h index 4fe1c341..72807f0f 100644 --- a/VM/src/ldo.h +++ b/VM/src/ldo.h @@ -37,7 +37,7 @@ /* results from luaD_precall */ #define PCRLUA 0 /* initiated a call to a Lua function */ #define PCRC 1 /* did a call to a C function */ -#define PCRYIELD 2 /* C funtion yielded */ +#define PCRYIELD 2 /* C function yielded */ /* type of protected functions, to be ran by `runprotected' */ typedef void (*Pfunc)(lua_State* L, void* ud); diff --git a/VM/src/lfunc.cpp b/VM/src/lfunc.cpp index 0b543026..64878569 100644 --- a/VM/src/lfunc.cpp +++ b/VM/src/lfunc.cpp @@ -76,7 +76,7 @@ UpVal* luaF_findupval(lua_State* L, StkId level) if (p->v == level) { /* found a corresponding upvalue? */ if (isdead(g, obj2gco(p))) /* is it dead? */ - changewhite(obj2gco(p)); /* ressurect it */ + changewhite(obj2gco(p)); /* resurrect it */ return p; } pp = &p->next; diff --git a/VM/src/lgc.cpp b/VM/src/lgc.cpp index 9b040fb5..85af403c 100644 --- a/VM/src/lgc.cpp +++ b/VM/src/lgc.cpp @@ -884,7 +884,7 @@ void luaC_step(lua_State* L, bool assist) g->gcstats.currcycle.explicitwork += lim; int lastgcstate = g->gcstate; - double lastttimestamp = lua_clock(); + double lasttimestamp = lua_clock(); // always perform at least one single step do @@ -899,14 +899,14 @@ void luaC_step(lua_State* L, bool assist) double now = lua_clock(); - recordGcStateTime(g, lastgcstate, now - lastttimestamp, assist); + recordGcStateTime(g, lastgcstate, now - lasttimestamp, assist); - lastttimestamp = now; + lasttimestamp = now; lastgcstate = g->gcstate; } } while (lim > 0 && g->gcstate != GCSpause); - recordGcStateTime(g, lastgcstate, lua_clock() - lastttimestamp, assist); + recordGcStateTime(g, lastgcstate, lua_clock() - lasttimestamp, assist); // at the end of the last cycle if (g->gcstate == GCSpause) diff --git a/VM/src/lgc.h b/VM/src/lgc.h index dc780bba..6fddd663 100644 --- a/VM/src/lgc.h +++ b/VM/src/lgc.h @@ -28,7 +28,7 @@ LUAU_FASTFLAG(LuauGcFullSkipInactiveThreads) #define keepinvariant(g) ((g)->gcstate == GCSpropagate || (g)->gcstate == GCSpropagateagain) /* -** some userful bit tricks +** some useful bit tricks */ #define resetbits(x, m) ((x) &= cast_to(uint8_t, ~(m))) #define setbits(x, m) ((x) |= (m)) diff --git a/bench/tests/deltablue.lua b/bench/tests/deltablue.lua index ecf246d3..ad18c233 100644 --- a/bench/tests/deltablue.lua +++ b/bench/tests/deltablue.lua @@ -21,7 +21,7 @@ local bench = script and require(script.Parent.bench_support) or require("bench_ -- This implementation of the DeltaBlue benchmark is derived -- from the Smalltalk implementation by John Maloney and Mario -- Wolczko. Some parts have been translated directly, whereas --- others have been modified more aggresively to make it feel +-- others have been modified more aggressively to make it feel -- more like a JavaScript program. @@ -222,7 +222,7 @@ end -- -- Normal constraints are not input constraints. An input constraint -- is one that depends on external state, such as the mouse, the --- keybord, a clock, or some arbitraty piece of imperative code. +-- keyboard, a clock, or some arbitrary piece of imperative code. -- function Constraint:isInput () return false @@ -614,7 +614,7 @@ end -- -- Attempt to satisfy the given constraint and, if successful, --- incrementally update the dataflow graph. Details: If satifying +-- incrementally update the dataflow graph. Details: If satisfying -- the constraint is successful, it may override a weaker constraint -- on its output. The algorithm attempts to resatisfy that -- constraint using some other method. This process is repeated @@ -837,7 +837,7 @@ end -- In case 1, the added constraint is stronger than the stay -- constraint and values must propagate down the entire length of the -- chain. In case 2, the added constraint is weaker than the stay --- constraint so it cannot be accomodated. The cost in this case is, +-- constraint so it cannot be accommodated. The cost in this case is, -- of course, very low. Typical situations lie somewhere between these -- two extremes. -- diff --git a/bench/tests/shootout/scimark.lua b/bench/tests/shootout/scimark.lua index 41d97bb8..ad0557b1 100644 --- a/bench/tests/shootout/scimark.lua +++ b/bench/tests/shootout/scimark.lua @@ -30,7 +30,7 @@ ------------------------------------------------------------------------------ ------------------------------------------------------------------------------ --- Modificatin to be compatible with Lua 5.3 +-- Modification to be compatible with Lua 5.3 ------------------------------------------------------------------------------ local bench = script and require(script.Parent.bench_support) or require("bench_support") diff --git a/docs/_pages/library.md b/docs/_pages/library.md index 0c04e24a..0a6e3ec9 100644 --- a/docs/_pages/library.md +++ b/docs/_pages/library.md @@ -18,7 +18,7 @@ While most library functions are provided as part of a library like `table`, a f function assert(value: T, message: string?): T ``` -`assert` checks if the value is truthful; if it's not (which means it's `false` or `nil`), it raises an error. The error message can be customized with an optional parameter. +`assert` checks if the value is truthy; if it's not (which means it's `false` or `nil`), it raises an error. The error message can be customized with an optional parameter. Upon success the function returns the `condition` argument. ``` @@ -146,7 +146,7 @@ Returns the triple (generator, state, nil) that can be used to traverse the tabl function pcall(f: function, args: ...any): (boolean, ...any) ``` -Calls function `f` with parameters `args`. If the function suceeds, returns `true` followed by all return values of `f`. If the function raises an error, returns `false` followed by the error object. +Calls function `f` with parameters `args`. If the function succeeds, returns `true` followed by all return values of `f`. If the function raises an error, returns `false` followed by the error object. Note that `f` can yield, which results in the entire coroutine yielding as well. ``` @@ -205,7 +205,7 @@ Rounds `n` upwards to the next integer boundary. function math.cosh(n: number): number ``` -Returns the hyberbolic cosine of `n`. +Returns the hyperbolic cosine of `n`. ``` function math.cos(n: number): number @@ -792,4 +792,4 @@ function debug.traceback(co: thread, msg: string?, level: number?): string function debug.traceback(msg: string?, level: number?): string ``` -Produces a stringifed callstack of the given thread, or the current thread, starting with level `level`. If `msg` is specified, then the resulting callstack includes the string before the callstack output, separated with a newline. The format of the callstack is human-readable and subject to change. +Produces a stringified callstack of the given thread, or the current thread, starting with level `level`. If `msg` is specified, then the resulting callstack includes the string before the callstack output, separated with a newline. The format of the callstack is human-readable and subject to change. diff --git a/docs/_pages/performance.md b/docs/_pages/performance.md index 3f77b1a3..f54c5ab8 100644 --- a/docs/_pages/performance.md +++ b/docs/_pages/performance.md @@ -88,7 +88,7 @@ The mechanism works by directly invoking a highly specialized and optimized impl As a result, builtin calls are very fast in Luau - they are still slightly slower than core instructions such as arithmetic operations, but only slightly so. The set of fastcall builtins is slowly expanding over time and as of this writing contains `assert`, `type`, `typeof`, `rawget`/`rawset`/`rawequal`, all functions from `math` and `bit32`, and some functions from `string` and `table` library. -> Note: The partial specialization mechanism is cute in that for `assert`, it only specializes on truthful conditions; hopefully performance of `assert(false)` isn't crucial for most code! +> Note: The partial specialization mechanism is cute in that for `assert`, it only specializes on truthy conditions; hopefully performance of `assert(false)` isn't crucial for most code! ## Optimized table iteration diff --git a/rfcs/function-debug-info.md b/rfcs/function-debug-info.md index 1362dc91..5f486db4 100644 --- a/rfcs/function-debug-info.md +++ b/rfcs/function-debug-info.md @@ -14,7 +14,7 @@ Today Luau provides only one method to get the callstack, `debug.traceback`. Thi There are a few cases where this can be inconvenient: -- Sometimes it is useful to pass the resulting call stack to some system expecting a structured input, e.g. for crash aggreggation +- Sometimes it is useful to pass the resulting call stack to some system expecting a structured input, e.g. for crash aggregation - Sometimes it is useful to use the information about the caller for logging or filtering purposes; in these cases using just the script name can be useful, and getting script name out of the traceback is slow and imprecise Additionally, in some cases instead of getting the information (such as script or function name) from the callstack, it can be useful to get it from a function object for diagnostic purposes. For example, maybe you want to call a callback and if it doesn't return expected results, display a user-friendly error message that contains the function name & script location - these aren't possible today at all. diff --git a/rfcs/function-table-freeze.md b/rfcs/function-table-freeze.md index 47de0f9e..ca819882 100644 --- a/rfcs/function-table-freeze.md +++ b/rfcs/function-table-freeze.md @@ -44,7 +44,7 @@ To limit the use of `table.freeze` to cases when table contents can be freely ma Exposing the internal "readonly" feature may have an impact on interoperability between scripts - for example, it becomes possible to freeze some tables that scripts may be expecting to have write access to from other scripts. Since we don't provide a way to unfreeze tables and freezing a table with a locked metatable fails, in theory the impact should not be any worse than allowing to change a metatable, but the full extents are unclear. -There may be existing code in the VM that allows changing frozen tables in ways that are benign to the current sanboxing code, but expose a "gap" in the implementation that becomes significant with this feature; thus we would need to audit all table writes when implementing this. +There may be existing code in the VM that allows changing frozen tables in ways that are benign to the current sandboxing code, but expose a "gap" in the implementation that becomes significant with this feature; thus we would need to audit all table writes when implementing this. ## Alternatives diff --git a/rfcs/syntax-continue-statement.md b/rfcs/syntax-continue-statement.md index 4adde09e..94e2009a 100644 --- a/rfcs/syntax-continue-statement.md +++ b/rfcs/syntax-continue-statement.md @@ -56,7 +56,7 @@ end These rules are simple to implement. In any Lua parser there is already a point where you have to disambiguate an identifier that starts an assignment statement (`foo = 5`) from an identifier that starts a function call (`foo(5)`). It's one of the few, if not the only, place in the Lua grammar where single token lookahead is not sufficient to parse Lua, because you could have `foo.bar(5)` or `foo.bar=5` or `foo.bar(5)[6] = 7`. -Because of this, we need to parse the entire left hand side of an assignment statement (primaryexpr in Lua's BNF) and then check if it was a function call; if not, we'd expect it to be an assignment statement. +Because of this, we need to parse the entire left hand side of an assignment statement (primaryexp in Lua's BNF) and then check if it was a function call; if not, we'd expect it to be an assignment statement. Alternatively in this specific case we could parse "continue", parse the next token, and if it's one of the exclusion list above, roll the parser state back and re-parse the non-continue statement. Our lexer currently doesn't support rollbacks but it's also an easy strategy that other implementations might employ for `continue` specifically. diff --git a/rfcs/syntax-if-expression.md b/rfcs/syntax-if-expression.md index 9fc85560..a435fd55 100644 --- a/rfcs/syntax-if-expression.md +++ b/rfcs/syntax-if-expression.md @@ -12,7 +12,7 @@ Introduce a form of ternary conditional using `if cond then value else alternati Luau does not have a first-class ternary operator; when a ternary operator is needed, it is usually emulated with `and/or` expression, such as `cond and value or alternative`. -This expression evaluates to `value` if `cond` and `value` are truthful, and `alternative` otherwise. In particular it means that when `value` is `false` or `nil`, the result of the entire expression is `alternative` even when `cond` is truthful - which doesn't match the expected ternary logic and is a frequent source of subtle errors. +This expression evaluates to `value` if `cond` and `value` are truthy, and `alternative` otherwise. In particular it means that when `value` is `false` or `nil`, the result of the entire expression is `alternative` even when `cond` is truthy - which doesn't match the expected ternary logic and is a frequent source of subtle errors. Instead of `and/or`, `if/else` statement can be used but since that requires a separate mutable variable, this option isn't ergonomic. An immediately invoked function expression is also unergonomic and results in performance issues at runtime. @@ -22,7 +22,7 @@ To solve these problems, we propose introducing a first-class ternary conditiona Concretely, the `if-then-else` expression must match `if then else `; it can also contain an arbitrary number of `elseif` clauses, like `if then elseif then else `. Note that in either case, `else` is mandatory. -The result of the expression is the then-expression when condition is truthful (not `nil` or `false`) and else-expression otherwise. Only one of the two possible resulting expressions is evaluated. +The result of the expression is the then-expression when condition is truthy (not `nil` or `false`) and else-expression otherwise. Only one of the two possible resulting expressions is evaluated. Example: diff --git a/tests/TypeInfer.builtins.test.cpp b/tests/TypeInfer.builtins.test.cpp index e8974776..17e32e9f 100644 --- a/tests/TypeInfer.builtins.test.cpp +++ b/tests/TypeInfer.builtins.test.cpp @@ -359,7 +359,7 @@ TEST_CASE_FIXTURE(Fixture, "table_insert_correctly_infers_type_of_array_2_args_o CHECK_EQ(typeChecker.stringType, requireType("s")); } -TEST_CASE_FIXTURE(Fixture, "table_insert_corrrectly_infers_type_of_array_3_args_overload") +TEST_CASE_FIXTURE(Fixture, "table_insert_correctly_infers_type_of_array_3_args_overload") { CheckResult result = check(R"( local t = {} diff --git a/tests/TypeInfer.provisional.test.cpp b/tests/TypeInfer.provisional.test.cpp index 46496fdb..8108e7f3 100644 --- a/tests/TypeInfer.provisional.test.cpp +++ b/tests/TypeInfer.provisional.test.cpp @@ -14,7 +14,7 @@ using namespace Luau; TEST_SUITE_BEGIN("ProvisionalTests"); -// These tests check for behavior that differes from the final behavior we'd +// These tests check for behavior that differs from the final behavior we'd // like to have. They serve to document the current state of the typechecker. // When making future improvements, its very likely these tests will break and // will need to be replaced. diff --git a/tests/TypeInfer.test.cpp b/tests/TypeInfer.test.cpp index d8dc0fbd..338698e7 100644 --- a/tests/TypeInfer.test.cpp +++ b/tests/TypeInfer.test.cpp @@ -405,7 +405,7 @@ TEST_CASE_FIXTURE(Fixture, "for_in_loop_error_on_factory_not_returning_the_right for p in primes2() do print(p) end -- mismatch in argument types, prime_iter takes {}, number, we are given {}, string - for p in primes3() do print(p) end -- no errror + for p in primes3() do print(p) end -- no error )"); LUAU_REQUIRE_ERROR_COUNT(2, result); @@ -2582,7 +2582,7 @@ TEST_CASE_FIXTURE(Fixture, "toposort_doesnt_break_mutual_recursion") --!strict local x = nil function f() g() end - -- make sure print(x) doen't get toposorted here, breaking the mutual block + -- make sure print(x) doesn't get toposorted here, breaking the mutual block function g() x = f end print(x) )"); @@ -3062,7 +3062,7 @@ TEST_CASE_FIXTURE(Fixture, "correctly_scope_locals_while") CHECK_EQ(us->name, "a"); } -TEST_CASE_FIXTURE(Fixture, "ipairs_produces_integral_indeces") +TEST_CASE_FIXTURE(Fixture, "ipairs_produces_integral_indices") { CheckResult result = check(R"( local key @@ -4885,7 +4885,7 @@ f(function(a) return a.x + a.y end) LUAU_REQUIRE_NO_ERRORS(result); - // An optional funciton is accepted, but since we already provide a function, nil can be ignored + // An optional function is accepted, but since we already provide a function, nil can be ignored result = check(R"( type Table = { x: number, y: number } local function f(a: ((Table) -> number)?) if a then return a({x = 1, y = 2}) else return 0 end end @@ -4913,7 +4913,7 @@ f(function(a: number, b, c) return c and a + b or b - a end) LUAU_REQUIRE_NO_ERRORS(result); - // Anonymous function has a varyadic pack + // Anonymous function has a variadic pack result = check(R"( type Table = { x: number, y: number } local function f(a: (Table) -> number) return a({x = 1, y = 2}) end @@ -4932,7 +4932,7 @@ f(function(a, b, c, ...) return a + b end) LUAU_REQUIRE_ERRORS(result); CHECK_EQ("Type '(number, number, a) -> number' could not be converted into '(number, number) -> number'", toString(result.errors[0])); - // Infer from varyadic packs into elements + // Infer from variadic packs into elements result = check(R"( function f(a: (...number) -> number) return a(1, 2) end f(function(a, b) return a + b end) @@ -4940,7 +4940,7 @@ f(function(a, b) return a + b end) LUAU_REQUIRE_NO_ERRORS(result); - // Infer from varyadic packs into varyadic packs + // Infer from variadic packs into variadic packs result = check(R"( type Table = { x: number, y: number } function f(a: (...Table) -> number) return a({x = 1, y = 2}, {x = 3, y = 4}) end diff --git a/tests/TypeInfer.unionTypes.test.cpp b/tests/TypeInfer.unionTypes.test.cpp index ae4d836b..456dbadc 100644 --- a/tests/TypeInfer.unionTypes.test.cpp +++ b/tests/TypeInfer.unionTypes.test.cpp @@ -52,7 +52,7 @@ TEST_CASE_FIXTURE(Fixture, "allow_more_specific_assign") LUAU_REQUIRE_NO_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "disallow_less_specifc_assign") +TEST_CASE_FIXTURE(Fixture, "disallow_less_specific_assign") { CheckResult result = check(R"( local a:number = 10 @@ -63,7 +63,7 @@ TEST_CASE_FIXTURE(Fixture, "disallow_less_specifc_assign") LUAU_REQUIRE_ERROR_COUNT(1, result); } -TEST_CASE_FIXTURE(Fixture, "disallow_less_specifc_assign2") +TEST_CASE_FIXTURE(Fixture, "disallow_less_specific_assign2") { CheckResult result = check(R"( local a:number? = 10 diff --git a/tests/conformance/closure.lua b/tests/conformance/closure.lua index 79f8d9c2..aac42c56 100644 --- a/tests/conformance/closure.lua +++ b/tests/conformance/closure.lua @@ -319,7 +319,7 @@ end assert(a == 5^4) --- access to locals of collected corroutines +-- access to locals of collected coroutines local C = {}; setmetatable(C, {__mode = "kv"}) local x = coroutine.wrap (function () local a = 10 diff --git a/tests/conformance/coroutine.lua b/tests/conformance/coroutine.lua index 73c3833d..75329642 100644 --- a/tests/conformance/coroutine.lua +++ b/tests/conformance/coroutine.lua @@ -185,7 +185,7 @@ end assert(a == 5^4) --- access to locals of collected corroutines +-- access to locals of collected coroutines local C = {}; setmetatable(C, {__mode = "kv"}) local x = coroutine.wrap (function () local a = 10 diff --git a/tests/conformance/gc.lua b/tests/conformance/gc.lua index fd4b4de1..4263dfda 100644 --- a/tests/conformance/gc.lua +++ b/tests/conformance/gc.lua @@ -277,7 +277,7 @@ do assert(getmetatable(o) == tt) -- create new objects during GC local a = 'xuxu'..(10+3)..'joao', {} - ___Glob = o -- ressurect object! + ___Glob = o -- resurrect object! newproxy(o) -- creates a new one with same metatable print(">>> closing state " .. "<<<\n") end diff --git a/tests/conformance/locals.lua b/tests/conformance/locals.lua index cbe5f92d..2d8d004b 100644 --- a/tests/conformance/locals.lua +++ b/tests/conformance/locals.lua @@ -117,7 +117,7 @@ if rawget(_G, "querytab") then local t = querytab(a) for k,_ in pairs(a) do a[k] = nil end - collectgarbage() -- restore GC and collect dead fiels in `a' + collectgarbage() -- restore GC and collect dead fields in `a' for i=0,t-1 do local k = querytab(a, i) assert(k == nil or type(k) == 'number' or k == 'alo') diff --git a/tests/conformance/math.lua b/tests/conformance/math.lua index 5e8b9398..d5bca44f 100644 --- a/tests/conformance/math.lua +++ b/tests/conformance/math.lua @@ -172,7 +172,7 @@ end a = nil --- testing implicit convertions +-- testing implicit conversions local a,b = '10', '20' assert(a*b == 200 and a+b == 30 and a-b == -10 and a/b == 0.5 and -b == -20) diff --git a/tests/conformance/pm.lua b/tests/conformance/pm.lua index 9a113964..263759ac 100644 --- a/tests/conformance/pm.lua +++ b/tests/conformance/pm.lua @@ -21,9 +21,9 @@ a,b = string.find('alo', '') assert(a == 1 and b == 0) a,b = string.find('a\0o a\0o a\0o', 'a', 1) -- first position assert(a == 1 and b == 1) -a,b = string.find('a\0o a\0o a\0o', 'a\0o', 2) -- starts in the midle +a,b = string.find('a\0o a\0o a\0o', 'a\0o', 2) -- starts in the middle assert(a == 5 and b == 7) -a,b = string.find('a\0o a\0o a\0o', 'a\0o', 9) -- starts in the midle +a,b = string.find('a\0o a\0o a\0o', 'a\0o', 9) -- starts in the middle assert(a == 9 and b == 11) a,b = string.find('a\0a\0a\0a\0\0ab', '\0ab', 2); -- finds at the end assert(a == 9 and b == 11); From 98fa75e91145c22a6e351f53be638dd2b5430ea5 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Thu, 4 Nov 2021 10:09:25 -0700 Subject: [PATCH 015/399] Fix fallout from #114: T is the right thread to use --- CLI/Repl.cpp | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/CLI/Repl.cpp b/CLI/Repl.cpp index f57a5009..4968d080 100644 --- a/CLI/Repl.cpp +++ b/CLI/Repl.cpp @@ -175,7 +175,7 @@ static std::string runCode(lua_State* L, const std::string& source) { error = "thread yielded unexpectedly"; } - else if (const char* str = lua_tostring(L, -1)) + else if (const char* str = lua_tostring(T, -1)) { error = str; } From e0c4f33217aa3d76d0e5d8b587f95a1f81d0f78e Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Thu, 4 Nov 2021 17:22:04 -0700 Subject: [PATCH 016/399] Add chess-profile.lua --- docs/assets/images/chess-profile.svg | 1356 ++++++++++++++++++++++++++ 1 file changed, 1356 insertions(+) create mode 100644 docs/assets/images/chess-profile.svg diff --git a/docs/assets/images/chess-profile.svg b/docs/assets/images/chess-profile.svg new file mode 100644 index 00000000..742dc6f2 --- /dev/null +++ b/docs/assets/images/chess-profile.svg @@ -0,0 +1,1356 @@ + + + + + + + + + + + + + +Flame Graph +Reset Zoom +Search +ic + + + + + + +
Function: [:0] (31,070 usec, 100.0%); self: 0 usec
+ + + + + + +chess.lua:3 +
Function: [chess.lua:3] (31,070 usec, 100.0%); self: 770 usec
+ + + + + +test +chess.lua:510 +
Function: test [chess.lua:510] (30,300 usec, 97.5%); self: 0 usec
+ +test +test + + +moveList +chess.lua:453 +
Function: moveList [chess.lua:453] (30,300 usec, 97.5%); self: 0 usec
+ +moveList +moveList + + +pmoves +chess.lua:310 +
Function: pmoves [chess.lua:310] (500 usec, 1.6%); self: 0 usec
+ + +pmoves + + +illegalyChecked +chess.lua:476 +
Function: illegalyChecked [chess.lua:476] (28,700 usec, 92.4%); self: 300 usec
+ +illegalyChecked +illegalyChecked + + +applyMove +chess.lua:490 +
Function: applyMove [chess.lua:490] (1,100 usec, 3.5%); self: 200 usec
+ +app.. +applyMove + + +generate +chess.lua:319 +
Function: generate [chess.lua:319] (500 usec, 1.6%); self: 0 usec
+ + +generate + + +pmoves +chess.lua:310 +
Function: pmoves [chess.lua:310] (27,400 usec, 88.2%); self: 100 usec
+ +pmoves +pmoves + + +band +chess.lua:125 +
Function: band [chess.lua:125] (200 usec, 0.6%); self: 0 usec
+ + +band + + +set +chess.lua:195 +
Function: set [chess.lua:195] (300 usec, 1.0%); self: 0 usec
+ + +set + + +ctz +chess.lua:141 +
Function: ctz [chess.lua:141] (400 usec, 1.3%); self: 400 usec
+ + +ctz + + +empty +chess.lua:137 +
Function: empty [chess.lua:137] (100 usec, 0.3%); self: 100 usec
+ + +empty + + +updateCache +chess.lua:283 +
Function: updateCache [chess.lua:283] (600 usec, 1.9%); self: 0 usec
+ +u.. +updateCache + + +set +chess.lua:195 +
Function: set [chess.lua:195] (100 usec, 0.3%); self: 0 usec
+ + +set + + +index +chess.lua:274 +
Function: index [chess.lua:274] (100 usec, 0.3%); self: 100 usec
+ + +index + + +new +chess.lua:228 +
Function: new [chess.lua:228] (100 usec, 0.3%); self: 0 usec
+ + +new + + +move +chess.lua:109 +
Function: move [chess.lua:109] (100 usec, 0.3%); self: 0 usec
+ + +move + + +isolate +chess.lua:304 +
Function: isolate [chess.lua:304] (400 usec, 1.3%); self: 0 usec
+ + +isolate + + +generate +chess.lua:319 +
Function: generate [chess.lua:319] (27,300 usec, 87.9%); self: 2,700 usec
+ +generate +generate + + +from +chess.lua:75 +
Function: from [chess.lua:75] (200 usec, 0.6%); self: 200 usec
+ + +from + + +from +chess.lua:75 +
Function: from [chess.lua:75] (300 usec, 1.0%); self: 200 usec
+ + +from + + +bor +chess.lua:129 +
Function: bor [chess.lua:129] (600 usec, 1.9%); self: 200 usec
+ +bor +bor + + +from +chess.lua:75 +
Function: from [chess.lua:75] (100 usec, 0.3%); self: 100 usec
+ + +from + + +GC +
Function: GC [GC:0] (100 usec, 0.3%); self: 100 usec
+ + +GC + + +right +chess.lua:101 +
Function: right [chess.lua:101] (100 usec, 0.3%); self: 0 usec
+ + +right + + +map +chess.lua:295 +
Function: map [chess.lua:295] (400 usec, 1.3%); self: 100 usec
+ + +map + + +move +chess.lua:109 +
Function: move [chess.lua:109] (4,400 usec, 14.2%); self: 800 usec
+ +move +move + + +band +chess.lua:125 +
Function: band [chess.lua:125] (1,900 usec, 6.1%); self: 400 usec
+ +band +band + + +isolate +chess.lua:304 +
Function: isolate [chess.lua:304] (13,000 usec, 41.8%); self: 0 usec
+ +isolate +isolate + + +index +chess.lua:274 +
Function: index [chess.lua:274] (700 usec, 2.3%); self: 200 usec
+ +i.. +index + + +down +chess.lua:97 +
Function: down [chess.lua:97] (100 usec, 0.3%); self: 0 usec
+ + +down + + +left +chess.lua:105 +
Function: left [chess.lua:105] (1,000 usec, 3.2%); self: 100 usec
+ +left +left + + +up +chess.lua:93 +
Function: up [chess.lua:93] (600 usec, 1.9%); self: 100 usec
+ +up +up + + +right +chess.lua:101 +
Function: right [chess.lua:101] (800 usec, 2.6%); self: 100 usec
+ +ri.. +right + + +GC +
Function: GC [GC:0] (500 usec, 1.6%); self: 500 usec
+ + +GC + + +bor +chess.lua:129 +
Function: bor [chess.lua:129] (900 usec, 2.9%); self: 400 usec
+ +bor +bor + + +empty +chess.lua:137 +
Function: empty [chess.lua:137] (400 usec, 1.3%); self: 400 usec
+ + +empty + + +some +chess.lua:207 +
Function: some [chess.lua:207] (300 usec, 1.0%); self: 200 usec
+ + +some + + +GC +
Function: GC [GC:0] (100 usec, 0.3%); self: 100 usec
+ + +GC + + +from +chess.lua:75 +
Function: from [chess.lua:75] (400 usec, 1.3%); self: 100 usec
+ + +from + + +lshift +chess.lua:164 +
Function: lshift [chess.lua:164] (100 usec, 0.3%); self: 0 usec
+ + +lshift + + +updateCache +chess.lua:283 +
Function: updateCache [chess.lua:283] (100 usec, 0.3%); self: 100 usec
+ + +updateCache + + + +chess.lua:305 +
Function: [chess.lua:305] (200 usec, 0.6%); self: 0 usec
+ + + + + +right +chess.lua:101 +
Function: right [chess.lua:101] (1,800 usec, 5.8%); self: 300 usec
+ +right +right + + +up +chess.lua:93 +
Function: up [chess.lua:93] (400 usec, 1.3%); self: 200 usec
+ + +up + + +left +chess.lua:105 +
Function: left [chess.lua:105] (1,100 usec, 3.5%); self: 100 usec
+ +left +left + + +down +chess.lua:97 +
Function: down [chess.lua:97] (300 usec, 1.0%); self: 200 usec
+ + +down + + +from +chess.lua:75 +
Function: from [chess.lua:75] (1,500 usec, 4.8%); self: 700 usec
+ +from +from + + +map +chess.lua:295 +
Function: map [chess.lua:295] (13,000 usec, 41.8%); self: 300 usec
+ +map +map + + +index +chess.lua:187 +
Function: index [chess.lua:187] (500 usec, 1.6%); self: 500 usec
+ + +index + + +rshift +chess.lua:176 +
Function: rshift [chess.lua:176] (100 usec, 0.3%); self: 100 usec
+ + +rshift + + +rshift +chess.lua:176 +
Function: rshift [chess.lua:176] (300 usec, 1.0%); self: 100 usec
+ + +rshift + + +inverse +chess.lua:133 +
Function: inverse [chess.lua:133] (400 usec, 1.3%); self: 100 usec
+ + +inverse + + +band +chess.lua:125 +
Function: band [chess.lua:125] (200 usec, 0.6%); self: 100 usec
+ + +band + + +lshift +chess.lua:164 +
Function: lshift [chess.lua:164] (500 usec, 1.6%); self: 200 usec
+ + +lshift + + +band +chess.lua:125 +
Function: band [chess.lua:125] (300 usec, 1.0%); self: 200 usec
+ + +band + + +inverse +chess.lua:133 +
Function: inverse [chess.lua:133] (300 usec, 1.0%); self: 0 usec
+ + +inverse + + +lshift +chess.lua:164 +
Function: lshift [chess.lua:164] (100 usec, 0.3%); self: 100 usec
+ + +lshift + + +from +chess.lua:75 +
Function: from [chess.lua:75] (500 usec, 1.6%); self: 300 usec
+ + +from + + +set +chess.lua:195 +
Function: set [chess.lua:195] (100 usec, 0.3%); self: 0 usec
+ + +set + + +GC +
Function: GC [GC:0] (300 usec, 1.0%); self: 300 usec
+ + +GC + + +from +chess.lua:75 +
Function: from [chess.lua:75] (100 usec, 0.3%); self: 0 usec
+ + +from + + +isolate +chess.lua:203 +
Function: isolate [chess.lua:203] (200 usec, 0.6%); self: 0 usec
+ + +isolate + + +inverse +chess.lua:133 +
Function: inverse [chess.lua:133] (200 usec, 0.6%); self: 200 usec
+ + +inverse + + +lshift +chess.lua:164 +
Function: lshift [chess.lua:164] (900 usec, 2.9%); self: 300 usec
+ +ls.. +lshift + + +band +chess.lua:125 +
Function: band [chess.lua:125] (400 usec, 1.3%); self: 300 usec
+ + +band + + +lshift +chess.lua:164 +
Function: lshift [chess.lua:164] (200 usec, 0.6%); self: 100 usec
+ + +lshift + + +rshift +chess.lua:176 +
Function: rshift [chess.lua:176] (400 usec, 1.3%); self: 100 usec
+ + +rshift + + +inverse +chess.lua:133 +
Function: inverse [chess.lua:133] (500 usec, 1.6%); self: 0 usec
+ + +inverse + + +band +chess.lua:125 +
Function: band [chess.lua:125] (100 usec, 0.3%); self: 0 usec
+ + +band + + +rshift +chess.lua:176 +
Function: rshift [chess.lua:176] (100 usec, 0.3%); self: 0 usec
+ + +rshift + + +GC +
Function: GC [GC:0] (800 usec, 2.6%); self: 800 usec
+ +GC +GC + + +updateCache +chess.lua:283 +
Function: updateCache [chess.lua:283] (3,800 usec, 12.2%); self: 500 usec
+ +updateCache +updateCache + + + +chess.lua:305 +
Function: [chess.lua:305] (7,700 usec, 24.8%); self: 300 usec
+ + + + + +new +chess.lua:228 +
Function: new [chess.lua:228] (1,200 usec, 3.9%); self: 1,000 usec
+ +new +new + + +from +chess.lua:75 +
Function: from [chess.lua:75] (200 usec, 0.6%); self: 100 usec
+ + +from + + +from +chess.lua:75 +
Function: from [chess.lua:75] (300 usec, 1.0%); self: 200 usec
+ + +from + + +from +chess.lua:75 +
Function: from [chess.lua:75] (100 usec, 0.3%); self: 0 usec
+ + +from + + +from +chess.lua:75 +
Function: from [chess.lua:75] (300 usec, 1.0%); self: 200 usec
+ + +from + + +from +chess.lua:75 +
Function: from [chess.lua:75] (100 usec, 0.3%); self: 0 usec
+ + +from + + +from +chess.lua:75 +
Function: from [chess.lua:75] (300 usec, 1.0%); self: 200 usec
+ + +from + + +GC +
Function: GC [GC:0] (200 usec, 0.6%); self: 200 usec
+ + +GC + + +from +chess.lua:75 +
Function: from [chess.lua:75] (100 usec, 0.3%); self: 0 usec
+ + +from + + +GC +
Function: GC [GC:0] (100 usec, 0.3%); self: 100 usec
+ + +GC + + +some +chess.lua:207 +
Function: some [chess.lua:207] (100 usec, 0.3%); self: 0 usec
+ + +some + + +band +chess.lua:125 +
Function: band [chess.lua:125] (100 usec, 0.3%); self: 0 usec
+ + +band + + +from +chess.lua:75 +
Function: from [chess.lua:75] (600 usec, 1.9%); self: 200 usec
+ +f.. +from + + +from +chess.lua:75 +
Function: from [chess.lua:75] (100 usec, 0.3%); self: 0 usec
+ + +from + + +from +chess.lua:75 +
Function: from [chess.lua:75] (100 usec, 0.3%); self: 100 usec
+ + +from + + +from +chess.lua:75 +
Function: from [chess.lua:75] (300 usec, 1.0%); self: 100 usec
+ + +from + + +from +chess.lua:75 +
Function: from [chess.lua:75] (500 usec, 1.6%); self: 100 usec
+ + +from + + +from +chess.lua:75 +
Function: from [chess.lua:75] (100 usec, 0.3%); self: 0 usec
+ + +from + + +from +chess.lua:75 +
Function: from [chess.lua:75] (100 usec, 0.3%); self: 0 usec
+ + +from + + +bor +chess.lua:129 +
Function: bor [chess.lua:129] (2,900 usec, 9.3%); self: 900 usec
+ +bor +bor + + +inverse +chess.lua:133 +
Function: inverse [chess.lua:133] (400 usec, 1.3%); self: 100 usec
+ + +inverse + + +isolate +chess.lua:203 +
Function: isolate [chess.lua:203] (7,400 usec, 23.8%); self: 700 usec
+ +isolate +isolate + + +GC +
Function: GC [GC:0] (200 usec, 0.6%); self: 200 usec
+ + +GC + + +GC +
Function: GC [GC:0] (100 usec, 0.3%); self: 100 usec
+ + +GC + + +GC +
Function: GC [GC:0] (100 usec, 0.3%); self: 100 usec
+ + +GC + + +GC +
Function: GC [GC:0] (100 usec, 0.3%); self: 100 usec
+ + +GC + + +GC +
Function: GC [GC:0] (100 usec, 0.3%); self: 100 usec
+ + +GC + + +GC +
Function: GC [GC:0] (100 usec, 0.3%); self: 100 usec
+ + +GC + + +GC +
Function: GC [GC:0] (100 usec, 0.3%); self: 100 usec
+ + +GC + + +GC +
Function: GC [GC:0] (100 usec, 0.3%); self: 100 usec
+ + +GC + + +set +chess.lua:195 +
Function: set [chess.lua:195] (100 usec, 0.3%); self: 0 usec
+ + +set + + +from +chess.lua:75 +
Function: from [chess.lua:75] (100 usec, 0.3%); self: 0 usec
+ + +from + + +GC +
Function: GC [GC:0] (400 usec, 1.3%); self: 400 usec
+ + +GC + + +GC +
Function: GC [GC:0] (100 usec, 0.3%); self: 100 usec
+ + +GC + + +GC +
Function: GC [GC:0] (200 usec, 0.6%); self: 200 usec
+ + +GC + + +GC +
Function: GC [GC:0] (400 usec, 1.3%); self: 400 usec
+ + +GC + + +GC +
Function: GC [GC:0] (100 usec, 0.3%); self: 100 usec
+ + +GC + + +GC +
Function: GC [GC:0] (100 usec, 0.3%); self: 100 usec
+ + +GC + + +from +chess.lua:75 +
Function: from [chess.lua:75] (2,000 usec, 6.4%); self: 1,100 usec
+ +from +from + + +from +chess.lua:75 +
Function: from [chess.lua:75] (300 usec, 1.0%); self: 0 usec
+ + +from + + +some +chess.lua:207 +
Function: some [chess.lua:207] (4,100 usec, 13.2%); self: 1,000 usec
+ +some +some + + +band +chess.lua:125 +
Function: band [chess.lua:125] (2,600 usec, 8.4%); self: 900 usec
+ +band +band + + +from +chess.lua:75 +
Function: from [chess.lua:75] (100 usec, 0.3%); self: 100 usec
+ + +from + + +GC +
Function: GC [GC:0] (100 usec, 0.3%); self: 100 usec
+ + +GC + + +GC +
Function: GC [GC:0] (900 usec, 2.9%); self: 900 usec
+ +GC +GC + + +GC +
Function: GC [GC:0] (300 usec, 1.0%); self: 300 usec
+ + +GC + + +set +chess.lua:195 +
Function: set [chess.lua:195] (3,100 usec, 10.0%); self: 1,500 usec
+ +set +set + + +from +chess.lua:75 +
Function: from [chess.lua:75] (1,700 usec, 5.5%); self: 800 usec
+ +from +from + + +from +chess.lua:75 +
Function: from [chess.lua:75] (1,600 usec, 5.1%); self: 1,000 usec
+ +from +from + + +GC +
Function: GC [GC:0] (900 usec, 2.9%); self: 900 usec
+ +GC +GC + + +GC +
Function: GC [GC:0] (600 usec, 1.9%); self: 600 usec
+ +GC +GC + + + + From 7c76a5a70a3ad05cbc5000676e250d3df52690f0 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Thu, 4 Nov 2021 17:27:56 -0700 Subject: [PATCH 017/399] Create profile.md Add profiler documentation. --- docs/_pages/profile.md | 61 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 61 insertions(+) create mode 100644 docs/_pages/profile.md diff --git a/docs/_pages/profile.md b/docs/_pages/profile.md new file mode 100644 index 00000000..76f36059 --- /dev/null +++ b/docs/_pages/profile.md @@ -0,0 +1,61 @@ +--- +permalink: /profile +title: Profiling +toc: true +--- + +One of main goals of Luau is to enable high performance code. To help with that goal, we are relentlessly optimizing the compiler and runtime - but ultimately, performance of their +code is in developers' hands, and is a combination of good algorithm design and implementation that adheres to the strengths of the language. To help write efficient code, Luau +provides a built-in profiler that samples the execution of the program and outputs a profiler dump that can be converted to an interactive flamegraph. + +To run the profiler, make sure you have an optimized build of the intepreter (otherwise profiling results are going to be very skewed) and run it with `--profile` argument: + +``` +$ luau --profile tests/chess.lua +OK 8902 rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - 0 1 +OK 2039 r3k2r/p1ppqpb1/bn2pnp1/3PN3/1p2P3/2N2Q1p/PPPBBPPP/R3K2R w KQkq - 0 0 +OK 2812 8/2p5/3p4/KP5r/1R3p1k/8/4P1P1/8 w - - 0 0 +OK 9467 r3k2r/Pppp1ppp/1b3nbN/nP6/BBP1P3/q4N2/Pp1P2PP/R2Q1RK1 w kq - 0 1 +OK 1486 rnbq1k1r/pp1Pbppp/2p5/8/2B5/8/PPP1NnPP/RNBQK2R w KQ - 1 8 +OK 2079 r4rk1/1pp1qppp/p1np1n2/2b1p1B1/2B1P1b1/P1NP1N2/1PP1QPPP/R4RK1 w - - 0 10 +Profiler dump written to profile.out (total runtime 2.034 seconds, 20344 samples, 374 stacks) +GC: 0.378 seconds (18.58%), mark 46.80%, remark 3.33%, atomic 1.93%, sweepstring 6.77%, sweep 41.16% +``` + +The resulting `profile.out` file can be converted to an SVG file by running `perfgraph.py` script that is part of Luau repository: + +``` +$ python tools/perfgraph.py profile.out >profile.svg +``` + +This produces an SVG file that can be opened in a browser (the image below is clickable): + +[![profile.svg](/assets/images/chess-profile.svg)](/assets/images/chess-profile.svg) + +In a flame graph visualization, the individual bars represent function calls, the width represents how much of the total program runtime they execute, and the deepness of the +red color represents how expensive they are based on their speed and how many times they are called: Dark Red bars are often the best targets for optimization, followed by +Bright Red, Dark Orange, Light Orange, Dark Yellow, Light Yellow. This is a fantastic visualization technique that allows you to hone in on the specific bottlenecks affecting +your program performance, optimize those exact bottlenecks, and then re-generate the profile data and visualizer, and look for the next set of true bottlenecks (if any). + +Hovering your mouse cursor over individual sections will display detailed function information in the status bar and in a tooltip. If you want to Search for a specific named +function, use the Search field in the upper right, or press Ctrl+F. + +Notice that some of the bars in the screenshot don't have any text. In some cases, there isn't enough room in the size of the bar to display the name. +You can hover your mouse over those bars to see the name and source location of the function in the tool tip, or double-click to zoom in on that part of the flame graph. + +Some tooltips will have a source location for the function you're hovering over, but no name. Those are anonymous functions, or functions that were not declared in a way that +allows Luau compiler to track the name. To fill in more names, you may want to make these changes to your code: + +`local myFunc = function() --[[ work ]] end` -> `local function myFunc() --[[ work ]] end` + +Even without these changes, you can hover over a given bar with no visible name and see it's source location. + +As any sampling profiler, this profiler relies on gathering enough information for the resulting output to be statistically meaningful. It may miss short functions if they +aren't called often enough. By default the profiler runs at 10 kHz, this can be customized by passing a different parameter to `--profile=`. Note that higher +frequencies result in higher profiling overhead and longer program execution, potentially skewing the results. + +This profiler doesn't track leaf C functions and instead attributes the time spent there to calling Luau functions. As a result, when thinking about why a given function is +slow, consider not just the work it does immediately but also the library functions it calls. + +This profiler tracks time consumed by Luau thread stacks; when a thread calls another thread via `coroutine.resume`, the time spent is not attributed to the parent thread that's +waiting for resume results. This limitation will be removed in the future in the future. From dc509b9849889de166dac3f296531d330d00f003 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Thu, 4 Nov 2021 17:28:24 -0700 Subject: [PATCH 018/399] Update navigation.yml Add profiling page to nav bar --- docs/_data/navigation.yml | 2 ++ 1 file changed, 2 insertions(+) diff --git a/docs/_data/navigation.yml b/docs/_data/navigation.yml index 77c45f12..ad2e55fd 100644 --- a/docs/_data/navigation.yml +++ b/docs/_data/navigation.yml @@ -21,5 +21,7 @@ pages: url: /compatibility - title: Typechecking url: /typecheck + - title: Profiling + url: /profile - title: Library url: /library From adacdcdf4eaa9d08e3721f568b8ff57512705a74 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Thu, 4 Nov 2021 18:07:34 -0700 Subject: [PATCH 019/399] Update profile.md Remove incorrect sentence copied from the incorrect internal documentation :) --- docs/_pages/profile.md | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/docs/_pages/profile.md b/docs/_pages/profile.md index 76f36059..edb30287 100644 --- a/docs/_pages/profile.md +++ b/docs/_pages/profile.md @@ -32,9 +32,7 @@ This produces an SVG file that can be opened in a browser (the image below is cl [![profile.svg](/assets/images/chess-profile.svg)](/assets/images/chess-profile.svg) -In a flame graph visualization, the individual bars represent function calls, the width represents how much of the total program runtime they execute, and the deepness of the -red color represents how expensive they are based on their speed and how many times they are called: Dark Red bars are often the best targets for optimization, followed by -Bright Red, Dark Orange, Light Orange, Dark Yellow, Light Yellow. This is a fantastic visualization technique that allows you to hone in on the specific bottlenecks affecting +In a flame graph visualization, the individual bars represent function calls, the width represents how much of the total program runtime they execute, and the nesting matches the call stack encountered during program execution. This is a fantastic visualization technique that allows you to hone in on the specific bottlenecks affecting your program performance, optimize those exact bottlenecks, and then re-generate the profile data and visualizer, and look for the next set of true bottlenecks (if any). Hovering your mouse cursor over individual sections will display detailed function information in the status bar and in a tooltip. If you want to Search for a specific named From 08c66ef2e17f685e2a2b3195909ac28816feb19a Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Thu, 4 Nov 2021 19:07:18 -0700 Subject: [PATCH 020/399] Sync to upstream/release/502 Changes: - Support for time tracing for analysis/compiler (not currently exposed through CLI) - Support for type pack arguments in type aliases (#83) - Basic support for require(path) in luau-analyze - Add a lint warning for table.move with 0 index as part of TableOperation lint - Remove last STL dependency from Luau.VM - Minor VS2022 performance tuning Co-authored-by: Rodactor --- .gitignore | 7 + Analysis/include/Luau/BuiltinDefinitions.h | 3 +- Analysis/include/Luau/Error.h | 1 + Analysis/include/Luau/FileResolver.h | 58 +- Analysis/include/Luau/Module.h | 10 +- Analysis/include/Luau/ModuleResolver.h | 6 - Analysis/include/Luau/RequireTracer.h | 5 +- Analysis/include/Luau/Scope.h | 67 ++ Analysis/include/Luau/Substitution.h | 14 +- Analysis/include/Luau/TypeInfer.h | 56 +- Analysis/include/Luau/TypePack.h | 3 +- Analysis/include/Luau/TypeVar.h | 22 +- Analysis/include/Luau/Unifier.h | 18 +- Analysis/src/AstQuery.cpp | 17 +- Analysis/src/Autocomplete.cpp | 49 +- Analysis/src/BuiltinDefinitions.cpp | 101 +-- Analysis/src/EmbeddedBuiltinDefinitions.cpp | 21 - Analysis/src/Error.cpp | 108 ++- Analysis/src/Frontend.cpp | 51 +- Analysis/src/IostreamHelpers.cpp | 18 +- Analysis/src/JsonEncoder.cpp | 32 +- Analysis/src/Linter.cpp | 19 +- Analysis/src/Module.cpp | 44 +- Analysis/src/RequireTracer.cpp | 215 ++++- Analysis/src/Scope.cpp | 123 +++ Analysis/src/Substitution.cpp | 34 +- Analysis/src/ToString.cpp | 131 ++- Analysis/src/Transpiler.cpp | 38 +- Analysis/src/TypeAttach.cpp | 124 ++- Analysis/src/TypeInfer.cpp | 577 ++++++------ Analysis/src/TypePack.cpp | 13 + Analysis/src/TypeUtils.cpp | 18 +- Analysis/src/TypeVar.cpp | 36 +- Analysis/src/Unifier.cpp | 500 +++++++++-- Ast/include/Luau/Ast.h | 33 +- Ast/include/Luau/DenseHash.h | 5 +- Ast/include/Luau/Parser.h | 8 +- Ast/include/Luau/TimeTrace.h | 223 +++++ Ast/src/Ast.cpp | 37 +- Ast/src/Parser.cpp | 147 ++- Ast/src/TimeTrace.cpp | 248 ++++++ CLI/Analyze.cpp | 18 +- Compiler/src/Compiler.cpp | 10 + Sources.cmake | 5 + VM/src/ldo.cpp | 12 +- VM/src/lgc.cpp | 191 +++- VM/src/ltablib.cpp | 22 - VM/src/lvmexecute.cpp | 12 +- VM/src/lvmload.cpp | 34 +- bench/tests/deltablue.lua | 934 -------------------- tests/Autocomplete.test.cpp | 853 +++++++++--------- tests/Fixture.cpp | 49 + tests/Fixture.h | 2 + tests/Frontend.test.cpp | 31 +- tests/Linter.test.cpp | 9 +- tests/Module.test.cpp | 1 + tests/NonstrictMode.test.cpp | 1 + tests/Parser.test.cpp | 15 + tests/RequireTracer.test.cpp | 68 +- tests/ToString.test.cpp | 3 +- tests/TypeInfer.aliases.test.cpp | 557 ++++++++++++ tests/TypeInfer.provisional.test.cpp | 36 +- tests/TypeInfer.refinements.test.cpp | 72 +- tests/TypeInfer.tables.test.cpp | 230 +++-- tests/TypeInfer.test.cpp | 563 +----------- tests/TypeInfer.tryUnify.test.cpp | 1 + tests/TypeInfer.typePacks.cpp | 366 ++++++++ tests/TypeInfer.unionTypes.test.cpp | 16 +- tests/TypeVar.test.cpp | 1 + tools/tracegraph.py | 95 ++ 70 files changed, 4485 insertions(+), 2962 deletions(-) create mode 100644 .gitignore create mode 100644 Analysis/include/Luau/Scope.h create mode 100644 Analysis/src/Scope.cpp create mode 100644 Ast/include/Luau/TimeTrace.h create mode 100644 Ast/src/TimeTrace.cpp delete mode 100644 bench/tests/deltablue.lua create mode 100644 tests/TypeInfer.aliases.test.cpp create mode 100644 tools/tracegraph.py diff --git a/.gitignore b/.gitignore new file mode 100644 index 00000000..0b2422ce --- /dev/null +++ b/.gitignore @@ -0,0 +1,7 @@ +^build/ +^coverage/ +^fuzz/luau.pb.* +^crash-* +^default.prof* +^fuzz-* +^luau$ diff --git a/Analysis/include/Luau/BuiltinDefinitions.h b/Analysis/include/Luau/BuiltinDefinitions.h index 8f17fff6..57a1907a 100644 --- a/Analysis/include/Luau/BuiltinDefinitions.h +++ b/Analysis/include/Luau/BuiltinDefinitions.h @@ -1,7 +1,8 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #pragma once -#include "TypeInfer.h" +#include "Luau/Scope.h" +#include "Luau/TypeInfer.h" namespace Luau { diff --git a/Analysis/include/Luau/Error.h b/Analysis/include/Luau/Error.h index 946bc928..ac6f13e9 100644 --- a/Analysis/include/Luau/Error.h +++ b/Analysis/include/Luau/Error.h @@ -120,6 +120,7 @@ struct IncorrectGenericParameterCount Name name; TypeFun typeFun; size_t actualParameters; + size_t actualPackParameters; bool operator==(const IncorrectGenericParameterCount& rhs) const; }; diff --git a/Analysis/include/Luau/FileResolver.h b/Analysis/include/Luau/FileResolver.h index 71f9464b..a05ec5e9 100644 --- a/Analysis/include/Luau/FileResolver.h +++ b/Analysis/include/Luau/FileResolver.h @@ -25,51 +25,39 @@ struct SourceCode Type type; }; +struct ModuleInfo +{ + ModuleName name; + bool optional = false; +}; + struct FileResolver { virtual ~FileResolver() {} - /** Fetch the source code associated with the provided ModuleName. - * - * FIXME: This requires a string copy! - * - * @returns The actual Lua code on success. - * @returns std::nullopt if no such file exists. When this occurs, type inference will report an UnknownRequire error. - */ virtual std::optional readSource(const ModuleName& name) = 0; - /** Does the module exist? - * - * Saves a string copy over reading the source and throwing it away. - */ - virtual bool moduleExists(const ModuleName& name) const = 0; + virtual std::optional resolveModule(const ModuleInfo* context, AstExpr* expr) + { + return std::nullopt; + } - virtual std::optional fromAstFragment(AstExpr* expr) const = 0; - - /** Given a valid module name and a string of arbitrary data, figure out the concatenation. - */ - virtual ModuleName concat(const ModuleName& lhs, std::string_view rhs) const = 0; - - /** Goes "up" a level in the hierarchy that the ModuleName represents. - * - * For instances, this is analogous to someInstance.Parent; for paths, this is equivalent to removing the last - * element of the path. Other ModuleName representations may have other ways of doing this. - * - * @returns The parent ModuleName, if one exists. - * @returns std::nullopt if there is no parent for this module name. - */ - virtual std::optional getParentModuleName(const ModuleName& name) const = 0; - - virtual std::optional getHumanReadableModuleName_(const ModuleName& name) const + virtual std::string getHumanReadableModuleName(const ModuleName& name) const { return name; } - virtual std::optional getEnvironmentForModule(const ModuleName& name) const = 0; + virtual std::optional getEnvironmentForModule(const ModuleName& name) const + { + return std::nullopt; + } - /** LanguageService only: - * std::optional fromInstance(Instance* inst) - */ + // DEPRECATED APIS + // These are going to be removed with LuauNewRequireTracer + virtual bool moduleExists(const ModuleName& name) const = 0; + virtual std::optional fromAstFragment(AstExpr* expr) const = 0; + virtual ModuleName concat(const ModuleName& lhs, std::string_view rhs) const = 0; + virtual std::optional getParentModuleName(const ModuleName& name) const = 0; }; struct NullFileResolver : FileResolver @@ -94,10 +82,6 @@ struct NullFileResolver : FileResolver { return std::nullopt; } - std::optional getEnvironmentForModule(const ModuleName& name) const override - { - return std::nullopt; - } }; } // namespace Luau diff --git a/Analysis/include/Luau/Module.h b/Analysis/include/Luau/Module.h index 413b68f4..d0844835 100644 --- a/Analysis/include/Luau/Module.h +++ b/Analysis/include/Luau/Module.h @@ -90,10 +90,12 @@ struct Module TypeArena internalTypes; std::vector> scopes; // never empty - std::unordered_map astTypes; - std::unordered_map astExpectedTypes; - std::unordered_map astOriginalCallTypes; - std::unordered_map astOverloadResolvedTypes; + + DenseHashMap astTypes{nullptr}; + DenseHashMap astExpectedTypes{nullptr}; + DenseHashMap astOriginalCallTypes{nullptr}; + DenseHashMap astOverloadResolvedTypes{nullptr}; + std::unordered_map declaredGlobals; ErrorVec errors; Mode mode; diff --git a/Analysis/include/Luau/ModuleResolver.h b/Analysis/include/Luau/ModuleResolver.h index a394a21b..d892ccd7 100644 --- a/Analysis/include/Luau/ModuleResolver.h +++ b/Analysis/include/Luau/ModuleResolver.h @@ -15,12 +15,6 @@ struct Module; using ModulePtr = std::shared_ptr; -struct ModuleInfo -{ - ModuleName name; - bool optional = false; -}; - struct ModuleResolver { virtual ~ModuleResolver() {} diff --git a/Analysis/include/Luau/RequireTracer.h b/Analysis/include/Luau/RequireTracer.h index e9778876..c25545f5 100644 --- a/Analysis/include/Luau/RequireTracer.h +++ b/Analysis/include/Luau/RequireTracer.h @@ -17,12 +17,11 @@ struct AstLocal; struct RequireTraceResult { - DenseHashMap exprs{0}; - DenseHashMap optional{0}; + DenseHashMap exprs{nullptr}; std::vector> requires; }; -RequireTraceResult traceRequires(FileResolver* fileResolver, AstStatBlock* root, ModuleName currentModuleName); +RequireTraceResult traceRequires(FileResolver* fileResolver, AstStatBlock* root, const ModuleName& currentModuleName); } // namespace Luau diff --git a/Analysis/include/Luau/Scope.h b/Analysis/include/Luau/Scope.h new file mode 100644 index 00000000..45338409 --- /dev/null +++ b/Analysis/include/Luau/Scope.h @@ -0,0 +1,67 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +#pragma once + +#include "Luau/Location.h" +#include "Luau/TypeVar.h" + +#include +#include +#include + +namespace Luau +{ + +struct Scope; + +using ScopePtr = std::shared_ptr; + +struct Binding +{ + TypeId typeId; + Location location; + bool deprecated = false; + std::string deprecatedSuggestion; + std::optional documentationSymbol; +}; + +struct Scope +{ + explicit Scope(TypePackId returnType); // root scope + explicit Scope(const ScopePtr& parent, int subLevel = 0); // child scope. Parent must not be nullptr. + + const ScopePtr parent; // null for the root + std::unordered_map bindings; + TypePackId returnType; + bool breakOk = false; + std::optional varargPack; + + TypeLevel level; + + std::unordered_map exportedTypeBindings; + std::unordered_map privateTypeBindings; + std::unordered_map typeAliasLocations; + + std::unordered_map> importedTypeBindings; + + std::optional lookup(const Symbol& name); + + std::optional lookupType(const Name& name); + std::optional lookupImportedType(const Name& moduleAlias, const Name& name); + + std::unordered_map privateTypePackBindings; + std::optional lookupPack(const Name& name); + + // WARNING: This function linearly scans for a string key of equal value! It is thus O(n**2) + std::optional linearSearchForBinding(const std::string& name, bool traverseScopeChain = true); + + RefinementMap refinements; + + // For mutually recursive type aliases, it's important that + // they use the same types for the same names. + // For instance, in `type Tree { data: T, children: Forest } type Forest = {Tree}` + // we need that the generic type `T` in both cases is the same, so we use a cache. + std::unordered_map typeAliasTypeParameters; + std::unordered_map typeAliasTypePackParameters; +}; + +} // namespace Luau diff --git a/Analysis/include/Luau/Substitution.h b/Analysis/include/Luau/Substitution.h index 6ac868f7..80a14e8f 100644 --- a/Analysis/include/Luau/Substitution.h +++ b/Analysis/include/Luau/Substitution.h @@ -52,8 +52,6 @@ // `T`, and the type of `f` are in the same SCC, which is why `f` gets // replaced. -LUAU_FASTFLAG(DebugLuauTrackOwningArena) - namespace Luau { @@ -188,20 +186,12 @@ struct Substitution : FindDirty template TypeId addType(const T& tv) { - TypeId allocated = currentModule->internalTypes.typeVars.allocate(tv); - if (FFlag::DebugLuauTrackOwningArena) - asMutable(allocated)->owningArena = ¤tModule->internalTypes; - - return allocated; + return currentModule->internalTypes.addType(tv); } template TypePackId addTypePack(const T& tp) { - TypePackId allocated = currentModule->internalTypes.typePacks.allocate(tp); - if (FFlag::DebugLuauTrackOwningArena) - asMutable(allocated)->owningArena = ¤tModule->internalTypes; - - return allocated; + return currentModule->internalTypes.addTypePack(TypePackVar{tp}); } }; diff --git a/Analysis/include/Luau/TypeInfer.h b/Analysis/include/Luau/TypeInfer.h index ec2a1a26..d701eb24 100644 --- a/Analysis/include/Luau/TypeInfer.h +++ b/Analysis/include/Luau/TypeInfer.h @@ -86,7 +86,10 @@ struct ApplyTypeFunction : Substitution { TypeLevel level; bool encounteredForwardedType; - std::unordered_map arguments; + std::unordered_map typeArguments; + std::unordered_map typePackArguments; + bool ignoreChildren(TypeId ty) override; + bool ignoreChildren(TypePackId tp) override; bool isDirty(TypeId ty) override; bool isDirty(TypePackId tp) override; TypeId clean(TypeId ty) override; @@ -328,7 +331,8 @@ private: TypeId resolveType(const ScopePtr& scope, const AstType& annotation, bool canBeGeneric = false); TypePackId resolveTypePack(const ScopePtr& scope, const AstTypeList& types); TypePackId resolveTypePack(const ScopePtr& scope, const AstTypePack& annotation); - TypeId instantiateTypeFun(const ScopePtr& scope, const TypeFun& tf, const std::vector& typeParams, const Location& location); + TypeId instantiateTypeFun(const ScopePtr& scope, const TypeFun& tf, const std::vector& typeParams, + const std::vector& typePackParams, const Location& location); // Note: `scope` must be a fresh scope. std::pair, std::vector> createGenericTypes( @@ -398,54 +402,6 @@ private: int recursionCount = 0; }; -struct Binding -{ - TypeId typeId; - Location location; - bool deprecated = false; - std::string deprecatedSuggestion; - std::optional documentationSymbol; -}; - -struct Scope -{ - explicit Scope(TypePackId returnType); // root scope - explicit Scope(const ScopePtr& parent, int subLevel = 0); // child scope. Parent must not be nullptr. - - const ScopePtr parent; // null for the root - std::unordered_map bindings; - TypePackId returnType; - bool breakOk = false; - std::optional varargPack; - - TypeLevel level; - - std::unordered_map exportedTypeBindings; - std::unordered_map privateTypeBindings; - std::unordered_map typeAliasLocations; - - std::unordered_map> importedTypeBindings; - - std::optional lookup(const Symbol& name); - - std::optional lookupType(const Name& name); - std::optional lookupImportedType(const Name& moduleAlias, const Name& name); - - std::unordered_map privateTypePackBindings; - std::optional lookupPack(const Name& name); - - // WARNING: This function linearly scans for a string key of equal value! It is thus O(n**2) - std::optional linearSearchForBinding(const std::string& name, bool traverseScopeChain = true); - - RefinementMap refinements; - - // For mutually recursive type aliases, it's important that - // they use the same types for the same names. - // For instance, in `type Tree { data: T, children: Forest } type Forest = {Tree}` - // we need that the generic type `T` in both cases is the same, so we use a cache. - std::unordered_map typeAliasParameters; -}; - // Unit test hook void setPrintLine(void (*pl)(const std::string& s)); void resetPrintLine(); diff --git a/Analysis/include/Luau/TypePack.h b/Analysis/include/Luau/TypePack.h index 0d0adce7..d987d46c 100644 --- a/Analysis/include/Luau/TypePack.h +++ b/Analysis/include/Luau/TypePack.h @@ -117,7 +117,8 @@ bool areEqual(SeenSet& seen, const TypePackVar& lhs, const TypePackVar& rhs); TypePackId follow(TypePackId tp); -size_t size(const TypePackId tp); +size_t size(TypePackId tp); +bool finite(TypePackId tp); size_t size(const TypePack& tp); std::optional first(TypePackId tp); diff --git a/Analysis/include/Luau/TypeVar.h b/Analysis/include/Luau/TypeVar.h index 90a28b20..d4e4e491 100644 --- a/Analysis/include/Luau/TypeVar.h +++ b/Analysis/include/Luau/TypeVar.h @@ -228,6 +228,7 @@ struct TableTypeVar std::map methodDefinitionLocations; std::vector instantiatedTypeParams; + std::vector instantiatedTypePackParams; ModuleName definitionModuleName; std::optional boundTo; @@ -284,8 +285,9 @@ struct ClassTypeVar struct TypeFun { - /// These should all be generic + // These should all be generic std::vector typeParams; + std::vector typePackParams; /** The underlying type. * @@ -293,6 +295,20 @@ struct TypeFun * You must first use TypeChecker::instantiateTypeFun to turn it into a real type. */ TypeId type; + + TypeFun() = default; + TypeFun(std::vector typeParams, TypeId type) + : typeParams(std::move(typeParams)) + , type(type) + { + } + + TypeFun(std::vector typeParams, std::vector typePackParams, TypeId type) + : typeParams(std::move(typeParams)) + , typePackParams(std::move(typePackParams)) + , type(type) + { + } }; // Anything! All static checking is off. @@ -524,8 +540,4 @@ UnionTypeVarIterator end(const UnionTypeVar* utv); using TypeIdPredicate = std::function(TypeId)>; std::vector filterMap(TypeId type, TypeIdPredicate predicate); -// TEMP: Clip this prototype with FFlag::LuauStringMetatable -std::optional> magicFunctionFormat( - struct TypeChecker& typechecker, const std::shared_ptr& scope, const AstExprCall& expr, ExprResult exprResult); - } // namespace Luau diff --git a/Analysis/include/Luau/Unifier.h b/Analysis/include/Luau/Unifier.h index 0ddc3cc0..522914b2 100644 --- a/Analysis/include/Luau/Unifier.h +++ b/Analysis/include/Luau/Unifier.h @@ -36,12 +36,17 @@ struct Unifier Variance variance = Covariant; CountMismatch::Context ctx = CountMismatch::Arg; - std::shared_ptr counters; + UnifierCounters* counters; + UnifierCounters countersData; + + std::shared_ptr counters_DEPRECATED; + InternalErrorReporter* iceHandler; Unifier(TypeArena* types, Mode mode, ScopePtr globalScope, const Location& location, Variance variance, InternalErrorReporter* iceHandler); Unifier(TypeArena* types, Mode mode, ScopePtr globalScope, const std::vector>& seen, const Location& location, - Variance variance, InternalErrorReporter* iceHandler, const std::shared_ptr& counters = nullptr); + Variance variance, InternalErrorReporter* iceHandler, const std::shared_ptr& counters_DEPRECATED = nullptr, + UnifierCounters* counters = nullptr); // Test whether the two type vars unify. Never commits the result. ErrorVec canUnify(TypeId superTy, TypeId subTy); @@ -58,11 +63,13 @@ private: void tryUnifyPrimitives(TypeId superTy, TypeId subTy); void tryUnifyFunctions(TypeId superTy, TypeId subTy, bool isFunctionCall = false); void tryUnifyTables(TypeId left, TypeId right, bool isIntersection = false); + void DEPRECATED_tryUnifyTables(TypeId left, TypeId right, bool isIntersection = false); void tryUnifyFreeTable(TypeId free, TypeId other); void tryUnifySealedTables(TypeId left, TypeId right, bool isIntersection); void tryUnifyWithMetatable(TypeId metatable, TypeId other, bool reversed); void tryUnifyWithClass(TypeId superTy, TypeId subTy, bool reversed); void tryUnify(const TableIndexer& superIndexer, const TableIndexer& subIndexer); + TypeId deeplyOptional(TypeId ty, std::unordered_map seen = {}); public: void tryUnify(TypePackId superTy, TypePackId subTy, bool isFunctionCall = false); @@ -80,9 +87,9 @@ private: public: // Report an "infinite type error" if the type "needle" already occurs within "haystack" void occursCheck(TypeId needle, TypeId haystack); - void occursCheck(std::unordered_set& seen, TypeId needle, TypeId haystack); + void occursCheck(std::unordered_set& seen_DEPRECATED, DenseHashSet& seen, TypeId needle, TypeId haystack); void occursCheck(TypePackId needle, TypePackId haystack); - void occursCheck(std::unordered_set& seen, TypePackId needle, TypePackId haystack); + void occursCheck(std::unordered_set& seen_DEPRECATED, DenseHashSet& seen, TypePackId needle, TypePackId haystack); Unifier makeChildUnifier(); @@ -93,6 +100,9 @@ private: [[noreturn]] void ice(const std::string& message, const Location& location); [[noreturn]] void ice(const std::string& message); + + DenseHashSet tempSeenTy{nullptr}; + DenseHashSet tempSeenTp{nullptr}; }; } // namespace Luau diff --git a/Analysis/src/AstQuery.cpp b/Analysis/src/AstQuery.cpp index d3de1754..0aed34c0 100644 --- a/Analysis/src/AstQuery.cpp +++ b/Analysis/src/AstQuery.cpp @@ -2,6 +2,7 @@ #include "Luau/AstQuery.h" #include "Luau/Module.h" +#include "Luau/Scope.h" #include "Luau/TypeInfer.h" #include "Luau/TypeVar.h" #include "Luau/ToString.h" @@ -143,8 +144,8 @@ std::optional findTypeAtPosition(const Module& module, const SourceModul { if (auto expr = findExprAtPosition(sourceModule, pos)) { - if (auto it = module.astTypes.find(expr); it != module.astTypes.end()) - return it->second; + if (auto it = module.astTypes.find(expr)) + return *it; } return std::nullopt; @@ -154,8 +155,8 @@ std::optional findExpectedTypeAtPosition(const Module& module, const Sou { if (auto expr = findExprAtPosition(sourceModule, pos)) { - if (auto it = module.astExpectedTypes.find(expr); it != module.astExpectedTypes.end()) - return it->second; + if (auto it = module.astExpectedTypes.find(expr)) + return *it; } return std::nullopt; @@ -322,9 +323,9 @@ std::optional getDocumentationSymbolAtPosition(const Source TypeId matchingOverload = nullptr; if (parentExpr && parentExpr->is()) { - if (auto it = module.astOverloadResolvedTypes.find(parentExpr); it != module.astOverloadResolvedTypes.end()) + if (auto it = module.astOverloadResolvedTypes.find(parentExpr)) { - matchingOverload = it->second; + matchingOverload = *it; } } @@ -345,9 +346,9 @@ std::optional getDocumentationSymbolAtPosition(const Source { if (AstExprIndexName* indexName = targetExpr->as()) { - if (auto it = module.astTypes.find(indexName->expr); it != module.astTypes.end()) + if (auto it = module.astTypes.find(indexName->expr)) { - TypeId parentTy = follow(it->second); + TypeId parentTy = follow(*it); if (const TableTypeVar* ttv = get(parentTy)) { if (auto propIt = ttv->props.find(indexName->index.value); propIt != ttv->props.end()) diff --git a/Analysis/src/Autocomplete.cpp b/Analysis/src/Autocomplete.cpp index dce92a0c..235abf36 100644 --- a/Analysis/src/Autocomplete.cpp +++ b/Analysis/src/Autocomplete.cpp @@ -210,10 +210,10 @@ static TypeCorrectKind checkTypeCorrectKind(const Module& module, TypeArena* typ return TypeCorrectKind::None; auto it = module.astExpectedTypes.find(expr); - if (it == module.astExpectedTypes.end()) + if (!it) return TypeCorrectKind::None; - TypeId expectedType = follow(it->second); + TypeId expectedType = follow(*it); if (canUnify(expectedType, ty)) return TypeCorrectKind::Correct; @@ -682,10 +682,10 @@ static std::optional functionIsExpectedAt(const Module& module, AstNode* n return std::nullopt; auto it = module.astExpectedTypes.find(expr); - if (it == module.astExpectedTypes.end()) + if (!it) return std::nullopt; - TypeId expectedType = follow(it->second); + TypeId expectedType = follow(*it); if (const FunctionTypeVar* ftv = get(expectedType)) return true; @@ -784,9 +784,9 @@ AutocompleteEntryMap autocompleteTypeNames(const Module& module, Position positi if (AstExprCall* exprCall = expr->as()) { - if (auto it = module.astTypes.find(exprCall->func); it != module.astTypes.end()) + if (auto it = module.astTypes.find(exprCall->func)) { - if (const FunctionTypeVar* ftv = get(follow(it->second))) + if (const FunctionTypeVar* ftv = get(follow(*it))) { if (auto ty = tryGetTypePackTypeAt(ftv->retType, tailPos)) inferredType = *ty; @@ -798,8 +798,8 @@ AutocompleteEntryMap autocompleteTypeNames(const Module& module, Position positi if (tailPos != 0) break; - if (auto it = module.astTypes.find(expr); it != module.astTypes.end()) - inferredType = it->second; + if (auto it = module.astTypes.find(expr)) + inferredType = *it; } if (inferredType) @@ -815,10 +815,10 @@ AutocompleteEntryMap autocompleteTypeNames(const Module& module, Position positi auto tryGetExpectedFunctionType = [](const Module& module, AstExpr* expr) -> const FunctionTypeVar* { auto it = module.astExpectedTypes.find(expr); - if (it == module.astExpectedTypes.end()) + if (!it) return nullptr; - TypeId ty = follow(it->second); + TypeId ty = follow(*it); if (const FunctionTypeVar* ftv = get(ty)) return ftv; @@ -1129,9 +1129,8 @@ static void autocompleteExpression(const SourceModule& sourceModule, const Modul if (node->is()) { - auto it = module.astTypes.find(node->asExpr()); - if (it != module.astTypes.end()) - autocompleteProps(module, typeArena, it->second, PropIndexType::Point, ancestry, result); + if (auto it = module.astTypes.find(node->asExpr())) + autocompleteProps(module, typeArena, *it, PropIndexType::Point, ancestry, result); } else if (FFlag::LuauIfElseExpressionAnalysisSupport && autocompleteIfElseExpression(node, ancestry, position, result)) return; @@ -1203,13 +1202,13 @@ static std::optional getMethodContainingClass(const ModuleP return std::nullopt; } - auto parentIter = module->astTypes.find(parentExpr); - if (parentIter == module->astTypes.end()) + auto parentIt = module->astTypes.find(parentExpr); + if (!parentIt) { return std::nullopt; } - Luau::TypeId parentType = Luau::follow(parentIter->second); + Luau::TypeId parentType = Luau::follow(*parentIt); if (auto parentClass = Luau::get(parentType)) { @@ -1250,8 +1249,8 @@ static std::optional autocompleteStringParams(const Source return std::nullopt; } - auto iter = module->astTypes.find(candidate->func); - if (iter == module->astTypes.end()) + auto it = module->astTypes.find(candidate->func); + if (!it) { return std::nullopt; } @@ -1267,7 +1266,7 @@ static std::optional autocompleteStringParams(const Source return std::nullopt; }; - auto followedId = Luau::follow(iter->second); + auto followedId = Luau::follow(*it); if (auto functionType = Luau::get(followedId)) { return performCallback(functionType); @@ -1316,10 +1315,10 @@ static AutocompleteResult autocomplete(const SourceModule& sourceModule, const M if (auto indexName = node->as()) { auto it = module->astTypes.find(indexName->expr); - if (it == module->astTypes.end()) + if (!it) return {}; - TypeId ty = follow(it->second); + TypeId ty = follow(*it); PropIndexType indexType = indexName->op == ':' ? PropIndexType::Colon : PropIndexType::Point; if (isString(ty)) @@ -1447,9 +1446,9 @@ static AutocompleteResult autocomplete(const SourceModule& sourceModule, const M // If item doesn't have a key, maybe the value is actually the key if (key ? key == node : node->is() && value == node) { - if (auto it = module->astExpectedTypes.find(exprTable); it != module->astExpectedTypes.end()) + if (auto it = module->astExpectedTypes.find(exprTable)) { - auto result = autocompleteProps(*module, typeArena, it->second, PropIndexType::Key, finder.ancestry); + auto result = autocompleteProps(*module, typeArena, *it, PropIndexType::Key, finder.ancestry); // Remove keys that are already completed for (const auto& item : exprTable->items) @@ -1485,9 +1484,9 @@ static AutocompleteResult autocomplete(const SourceModule& sourceModule, const M { if (auto idxExpr = finder.ancestry.at(finder.ancestry.size() - 2)->as()) { - if (auto it = module->astTypes.find(idxExpr->expr); it != module->astTypes.end()) + if (auto it = module->astTypes.find(idxExpr->expr)) { - return {autocompleteProps(*module, typeArena, follow(it->second), PropIndexType::Point, finder.ancestry), finder.ancestry}; + return {autocompleteProps(*module, typeArena, follow(*it), PropIndexType::Point, finder.ancestry), finder.ancestry}; } } } diff --git a/Analysis/src/BuiltinDefinitions.cpp b/Analysis/src/BuiltinDefinitions.cpp index 68ad5ac9..3b0c2163 100644 --- a/Analysis/src/BuiltinDefinitions.cpp +++ b/Analysis/src/BuiltinDefinitions.cpp @@ -11,7 +11,7 @@ LUAU_FASTFLAG(LuauParseGenericFunctions) LUAU_FASTFLAG(LuauGenericFunctions) LUAU_FASTFLAG(LuauRankNTypes) -LUAU_FASTFLAG(LuauStringMetatable) +LUAU_FASTFLAG(LuauNewRequireTrace) /** FIXME: Many of these type definitions are not quite completely accurate. * @@ -218,7 +218,6 @@ void registerBuiltinTypes(TypeChecker& typeChecker) TypePackId anyTypePack = typeChecker.anyTypePack; TypePackId numberVariadicList = arena.addTypePack(TypePackVar{VariadicTypePack{numberType}}); - TypePackId stringVariadicList = arena.addTypePack(TypePackVar{VariadicTypePack{stringType}}); TypePackId listOfAtLeastOneNumber = arena.addTypePack(TypePack{{numberType}, numberVariadicList}); TypeId listOfAtLeastOneNumberToNumberType = arena.addType(FunctionTypeVar{ @@ -255,85 +254,18 @@ void registerBuiltinTypes(TypeChecker& typeChecker) TypeId genericV = arena.addType(GenericTypeVar{"V"}); TypeId mapOfKtoV = arena.addType(TableTypeVar{{}, TableIndexer(genericK, genericV), typeChecker.globalScope->level}); - if (FFlag::LuauStringMetatable) + std::optional stringMetatableTy = getMetatable(singletonTypes.stringType); + LUAU_ASSERT(stringMetatableTy); + const TableTypeVar* stringMetatableTable = get(follow(*stringMetatableTy)); + LUAU_ASSERT(stringMetatableTable); + + auto it = stringMetatableTable->props.find("__index"); + LUAU_ASSERT(it != stringMetatableTable->props.end()); + + addGlobalBinding(typeChecker, "string", it->second.type, "@luau"); + + if (!FFlag::LuauParseGenericFunctions || !FFlag::LuauGenericFunctions) { - std::optional stringMetatableTy = getMetatable(singletonTypes.stringType); - LUAU_ASSERT(stringMetatableTy); - const TableTypeVar* stringMetatableTable = get(follow(*stringMetatableTy)); - LUAU_ASSERT(stringMetatableTable); - - auto it = stringMetatableTable->props.find("__index"); - LUAU_ASSERT(it != stringMetatableTable->props.end()); - - TypeId stringLib = it->second.type; - addGlobalBinding(typeChecker, "string", stringLib, "@luau"); - } - - if (FFlag::LuauParseGenericFunctions && FFlag::LuauGenericFunctions) - { - if (!FFlag::LuauStringMetatable) - { - TypeId stringLibTy = getGlobalBinding(typeChecker, "string"); - TableTypeVar* stringLib = getMutable(stringLibTy); - TypeId replArgType = makeUnion( - arena, {stringType, - arena.addType(TableTypeVar({}, TableIndexer(stringType, stringType), typeChecker.globalScope->level, TableState::Generic)), - makeFunction(arena, std::nullopt, {stringType}, {stringType})}); - TypeId gsubFunc = makeFunction(arena, stringType, {stringType, replArgType, optionalNumber}, {stringType, numberType}); - - stringLib->props["gsub"] = makeProperty(gsubFunc, "@luau/global/string.gsub"); - } - } - else - { - if (!FFlag::LuauStringMetatable) - { - TypeId stringToStringType = makeFunction(arena, std::nullopt, {stringType}, {stringType}); - - TypeId gmatchFunc = makeFunction(arena, stringType, {stringType}, {arena.addType(FunctionTypeVar{emptyPack, stringVariadicList})}); - - TypeId replArgType = makeUnion( - arena, {stringType, - arena.addType(TableTypeVar({}, TableIndexer(stringType, stringType), typeChecker.globalScope->level, TableState::Generic)), - makeFunction(arena, std::nullopt, {stringType}, {stringType})}); - TypeId gsubFunc = makeFunction(arena, stringType, {stringType, replArgType, optionalNumber}, {stringType, numberType}); - - TypeId formatFn = arena.addType(FunctionTypeVar{arena.addTypePack(TypePack{{stringType}, anyTypePack}), oneStringPack}); - - TableTypeVar::Props stringLib = { - // FIXME string.byte "can" return a pack of numbers, but only if 2nd or 3rd arguments were supplied - {"byte", {makeFunction(arena, stringType, {optionalNumber, optionalNumber}, {optionalNumber})}}, - // FIXME char takes a variadic pack of numbers - {"char", {makeFunction(arena, std::nullopt, {numberType, optionalNumber, optionalNumber, optionalNumber}, {stringType})}}, - {"find", {makeFunction(arena, stringType, {stringType, optionalNumber, optionalBoolean}, {optionalNumber, optionalNumber})}}, - {"format", {formatFn}}, // FIXME - {"gmatch", {gmatchFunc}}, - {"gsub", {gsubFunc}}, - {"len", {makeFunction(arena, stringType, {}, {numberType})}}, - {"lower", {stringToStringType}}, - {"match", {makeFunction(arena, stringType, {stringType, optionalNumber}, {optionalString})}}, - {"rep", {makeFunction(arena, stringType, {numberType}, {stringType})}}, - {"reverse", {stringToStringType}}, - {"sub", {makeFunction(arena, stringType, {numberType, optionalNumber}, {stringType})}}, - {"upper", {stringToStringType}}, - {"split", {makeFunction(arena, stringType, {stringType, optionalString}, - {arena.addType(TableTypeVar{{}, TableIndexer{numberType, stringType}, typeChecker.globalScope->level})})}}, - {"pack", {arena.addType(FunctionTypeVar{ - arena.addTypePack(TypePack{{stringType}, anyTypePack}), - oneStringPack, - })}}, - {"packsize", {makeFunction(arena, stringType, {}, {numberType})}}, - {"unpack", {arena.addType(FunctionTypeVar{ - arena.addTypePack(TypePack{{stringType, stringType, optionalNumber}}), - anyTypePack, - })}}, - }; - - assignPropDocumentationSymbols(stringLib, "@luau/global/string"); - addGlobalBinding(typeChecker, "string", - arena.addType(TableTypeVar{stringLib, std::nullopt, typeChecker.globalScope->level, TableState::Sealed}), "@luau"); - } - TableTypeVar::Props debugLib{ {"info", {makeIntersection(arena, { @@ -601,9 +533,6 @@ void registerBuiltinTypes(TypeChecker& typeChecker) auto tableLib = getMutable(getGlobalBinding(typeChecker, "table")); attachMagicFunction(tableLib->props["pack"].type, magicFunctionPack); - auto stringLib = getMutable(getGlobalBinding(typeChecker, "string")); - attachMagicFunction(stringLib->props["format"].type, magicFunctionFormat); - attachMagicFunction(getGlobalBinding(typeChecker, "require"), magicFunctionRequire); } @@ -791,11 +720,11 @@ static std::optional> magicFunctionRequire( return std::nullopt; } - AstExpr* require = expr.args.data[0]; - - if (!checkRequirePath(typechecker, require)) + if (!checkRequirePath(typechecker, expr.args.data[0])) return std::nullopt; + const AstExpr* require = FFlag::LuauNewRequireTrace ? &expr : expr.args.data[0]; + if (auto moduleInfo = typechecker.resolver->resolveModuleInfo(typechecker.currentModuleName, *require)) return ExprResult{arena.addTypePack({typechecker.checkRequire(scope, *moduleInfo, expr.location)})}; diff --git a/Analysis/src/EmbeddedBuiltinDefinitions.cpp b/Analysis/src/EmbeddedBuiltinDefinitions.cpp index 61a63f06..1e91561a 100644 --- a/Analysis/src/EmbeddedBuiltinDefinitions.cpp +++ b/Analysis/src/EmbeddedBuiltinDefinitions.cpp @@ -206,27 +206,6 @@ std::string getBuiltinDefinitionSource() graphemes: (string, number?, number?) -> (() -> (number, number)), } - declare string: { - byte: (string, number?, number?) -> ...number, - char: (number, ...number) -> string, - find: (string, string, number?, boolean?) -> (number?, number?), - -- `string.format` has a magic function attached that will provide more type information for literal format strings. - format: (string, A...) -> string, - gmatch: (string, string) -> () -> (...string), - -- gsub is defined in C++ because we don't have syntax for describing a generic table. - len: (string) -> number, - lower: (string) -> string, - match: (string, string, number?) -> string?, - rep: (string, number) -> string, - reverse: (string) -> string, - sub: (string, number, number?) -> string, - upper: (string) -> string, - split: (string, string, string?) -> {string}, - pack: (string, A...) -> string, - packsize: (string) -> number, - unpack: (string, string, number?) -> R..., - } - -- Cannot use `typeof` here because it will produce a polytype when we expect a monotype. declare function unpack(tab: {V}, i: number?, j: number?): ...V )"; diff --git a/Analysis/src/Error.cpp b/Analysis/src/Error.cpp index 680bcf3f..92fbffc8 100644 --- a/Analysis/src/Error.cpp +++ b/Analysis/src/Error.cpp @@ -7,9 +7,9 @@ #include -LUAU_FASTFLAG(LuauFasterStringifier) +LUAU_FASTFLAG(LuauTypeAliasPacks) -static std::string wrongNumberOfArgsString(size_t expectedCount, size_t actualCount, bool isTypeArgs = false) +static std::string wrongNumberOfArgsString_DEPRECATED(size_t expectedCount, size_t actualCount, bool isTypeArgs = false) { std::string s = "expects " + std::to_string(expectedCount) + " "; @@ -41,6 +41,52 @@ static std::string wrongNumberOfArgsString(size_t expectedCount, size_t actualCo return s; } +static std::string wrongNumberOfArgsString(size_t expectedCount, size_t actualCount, const char* argPrefix = nullptr, bool isVariadic = false) +{ + std::string s; + + if (FFlag::LuauTypeAliasPacks) + { + s = "expects "; + + if (isVariadic) + s += "at least "; + + s += std::to_string(expectedCount) + " "; + } + else + { + s = "expects " + std::to_string(expectedCount) + " "; + } + + if (argPrefix) + s += std::string(argPrefix) + " "; + + s += "argument"; + if (expectedCount != 1) + s += "s"; + + s += ", but "; + + if (actualCount == 0) + { + s += "none"; + } + else + { + if (actualCount < expectedCount) + s += "only "; + + s += std::to_string(actualCount); + } + + s += (actualCount == 1) ? " is" : " are"; + + s += " specified"; + + return s; +} + namespace Luau { @@ -128,7 +174,10 @@ struct ErrorConverter else return "Function only returns " + std::to_string(e.expected) + " values. " + std::to_string(e.actual) + " are required here"; case CountMismatch::Arg: - return "Argument count mismatch. Function " + wrongNumberOfArgsString(e.expected, e.actual); + if (FFlag::LuauTypeAliasPacks) + return "Argument count mismatch. Function " + wrongNumberOfArgsString(e.expected, e.actual); + else + return "Argument count mismatch. Function " + wrongNumberOfArgsString_DEPRECATED(e.expected, e.actual); } LUAU_ASSERT(!"Unknown context"); @@ -160,13 +209,16 @@ struct ErrorConverter std::string operator()(const Luau::UnknownRequire& e) const { - return "Unknown require: " + e.modulePath; + if (e.modulePath.empty()) + return "Unknown require: unsupported path"; + else + return "Unknown require: " + e.modulePath; } std::string operator()(const Luau::IncorrectGenericParameterCount& e) const { std::string name = e.name; - if (!e.typeFun.typeParams.empty()) + if (!e.typeFun.typeParams.empty() || (FFlag::LuauTypeAliasPacks && !e.typeFun.typePackParams.empty())) { name += "<"; bool first = true; @@ -179,10 +231,37 @@ struct ErrorConverter name += toString(t); } + + if (FFlag::LuauTypeAliasPacks) + { + for (TypePackId t : e.typeFun.typePackParams) + { + if (first) + first = false; + else + name += ", "; + + name += toString(t); + } + } + name += ">"; } - return "Generic type '" + name + "' " + wrongNumberOfArgsString(e.typeFun.typeParams.size(), e.actualParameters, /*isTypeArgs*/ true); + if (FFlag::LuauTypeAliasPacks) + { + if (e.typeFun.typeParams.size() != e.actualParameters) + return "Generic type '" + name + "' " + + wrongNumberOfArgsString(e.typeFun.typeParams.size(), e.actualParameters, "type", !e.typeFun.typePackParams.empty()); + + return "Generic type '" + name + "' " + + wrongNumberOfArgsString(e.typeFun.typePackParams.size(), e.actualPackParameters, "type pack", /*isVariadic*/ false); + } + else + { + return "Generic type '" + name + "' " + + wrongNumberOfArgsString_DEPRECATED(e.typeFun.typeParams.size(), e.actualParameters, /*isTypeArgs*/ true); + } } std::string operator()(const Luau::SyntaxError& e) const @@ -471,9 +550,26 @@ bool IncorrectGenericParameterCount::operator==(const IncorrectGenericParameterC if (typeFun.typeParams.size() != rhs.typeFun.typeParams.size()) return false; + if (FFlag::LuauTypeAliasPacks) + { + if (typeFun.typePackParams.size() != rhs.typeFun.typePackParams.size()) + return false; + } + for (size_t i = 0; i < typeFun.typeParams.size(); ++i) + { if (typeFun.typeParams[i] != rhs.typeFun.typeParams[i]) return false; + } + + if (FFlag::LuauTypeAliasPacks) + { + for (size_t i = 0; i < typeFun.typePackParams.size(); ++i) + { + if (typeFun.typePackParams[i] != rhs.typeFun.typePackParams[i]) + return false; + } + } return true; } diff --git a/Analysis/src/Frontend.cpp b/Analysis/src/Frontend.cpp index 4d385ec1..b2529840 100644 --- a/Analysis/src/Frontend.cpp +++ b/Analysis/src/Frontend.cpp @@ -1,9 +1,12 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #include "Luau/Frontend.h" +#include "Luau/Common.h" #include "Luau/Config.h" #include "Luau/FileResolver.h" +#include "Luau/Scope.h" #include "Luau/StringUtils.h" +#include "Luau/TimeTrace.h" #include "Luau/TypeInfer.h" #include "Luau/Variant.h" #include "Luau/Common.h" @@ -19,6 +22,7 @@ LUAU_FASTFLAGVARIABLE(LuauSecondTypecheckKnowsTheDataModel, false) LUAU_FASTFLAGVARIABLE(LuauResolveModuleNameWithoutACurrentModule, false) LUAU_FASTFLAG(LuauTraceRequireLookupChild) LUAU_FASTFLAGVARIABLE(LuauPersistDefinitionFileTypes, false) +LUAU_FASTFLAG(LuauNewRequireTrace) namespace Luau { @@ -69,6 +73,8 @@ static void generateDocumentationSymbols(TypeId ty, const std::string& rootName) LoadDefinitionFileResult loadDefinitionFile(TypeChecker& typeChecker, ScopePtr targetScope, std::string_view source, const std::string& packageName) { + LUAU_TIMETRACE_SCOPE("loadDefinitionFile", "Frontend"); + Luau::Allocator allocator; Luau::AstNameTable names(allocator); @@ -350,6 +356,9 @@ FrontendModuleResolver::FrontendModuleResolver(Frontend* frontend) CheckResult Frontend::check(const ModuleName& name) { + LUAU_TIMETRACE_SCOPE("Frontend::check", "Frontend"); + LUAU_TIMETRACE_ARGUMENT("name", name.c_str()); + CheckResult checkResult; auto it = sourceNodes.find(name); @@ -479,6 +488,9 @@ CheckResult Frontend::check(const ModuleName& name) bool Frontend::parseGraph(std::vector& buildQueue, CheckResult& checkResult, const ModuleName& root) { + LUAU_TIMETRACE_SCOPE("Frontend::parseGraph", "Frontend"); + LUAU_TIMETRACE_ARGUMENT("root", root.c_str()); + // https://en.wikipedia.org/wiki/Topological_sorting#Depth-first_search enum Mark { @@ -597,6 +609,9 @@ ScopePtr Frontend::getModuleEnvironment(const SourceModule& module, const Config LintResult Frontend::lint(const ModuleName& name, std::optional enabledLintWarnings) { + LUAU_TIMETRACE_SCOPE("Frontend::lint", "Frontend"); + LUAU_TIMETRACE_ARGUMENT("name", name.c_str()); + CheckResult checkResult; auto [_sourceNode, sourceModule] = getSourceNode(checkResult, name); @@ -608,6 +623,8 @@ LintResult Frontend::lint(const ModuleName& name, std::optional Frontend::lintFragment(std::string_view source, std::optional enabledLintWarnings) { + LUAU_TIMETRACE_SCOPE("Frontend::lintFragment", "Frontend"); + const Config& config = configResolver->getConfig(""); SourceModule sourceModule = parse(ModuleName{}, source, config.parseOptions); @@ -627,6 +644,9 @@ std::pair Frontend::lintFragment(std::string_view sour CheckResult Frontend::check(const SourceModule& module) { + LUAU_TIMETRACE_SCOPE("Frontend::check", "Frontend"); + LUAU_TIMETRACE_ARGUMENT("module", module.name.c_str()); + const Config& config = configResolver->getConfig(module.name); Mode mode = module.mode.value_or(config.mode); @@ -648,6 +668,9 @@ CheckResult Frontend::check(const SourceModule& module) LintResult Frontend::lint(const SourceModule& module, std::optional enabledLintWarnings) { + LUAU_TIMETRACE_SCOPE("Frontend::lint", "Frontend"); + LUAU_TIMETRACE_ARGUMENT("module", module.name.c_str()); + const Config& config = configResolver->getConfig(module.name); LintOptions options = enabledLintWarnings.value_or(config.enabledLint); @@ -746,6 +769,9 @@ const SourceModule* Frontend::getSourceModule(const ModuleName& moduleName) cons // Read AST into sourceModules if necessary. Trace require()s. Report parse errors. std::pair Frontend::getSourceNode(CheckResult& checkResult, const ModuleName& name) { + LUAU_TIMETRACE_SCOPE("Frontend::getSourceNode", "Frontend"); + LUAU_TIMETRACE_ARGUMENT("name", name.c_str()); + auto it = sourceNodes.find(name); if (it != sourceNodes.end() && !it->second.dirty) { @@ -815,6 +841,9 @@ std::pair Frontend::getSourceNode(CheckResult& check */ SourceModule Frontend::parse(const ModuleName& name, std::string_view src, const ParseOptions& parseOptions) { + LUAU_TIMETRACE_SCOPE("Frontend::parse", "Frontend"); + LUAU_TIMETRACE_ARGUMENT("name", name.c_str()); + SourceModule sourceModule; double timestamp = getTimestamp(); @@ -864,20 +893,11 @@ std::optional FrontendModuleResolver::resolveModuleInfo(const Module const auto& exprs = it->second.exprs; - const ModuleName* relativeName = exprs.find(&pathExpr); - if (!relativeName || relativeName->empty()) + const ModuleInfo* info = exprs.find(&pathExpr); + if (!info || (!FFlag::LuauNewRequireTrace && info->name.empty())) return std::nullopt; - if (FFlag::LuauTraceRequireLookupChild) - { - const bool* optional = it->second.optional.find(&pathExpr); - - return {{*relativeName, optional ? *optional : false}}; - } - else - { - return {{*relativeName, false}}; - } + return *info; } const ModulePtr FrontendModuleResolver::getModule(const ModuleName& moduleName) const @@ -891,12 +911,15 @@ const ModulePtr FrontendModuleResolver::getModule(const ModuleName& moduleName) bool FrontendModuleResolver::moduleExists(const ModuleName& moduleName) const { - return frontend->fileResolver->moduleExists(moduleName); + if (FFlag::LuauNewRequireTrace) + return frontend->sourceNodes.count(moduleName) != 0; + else + return frontend->fileResolver->moduleExists(moduleName); } std::string FrontendModuleResolver::getHumanReadableModuleName(const ModuleName& moduleName) const { - return frontend->fileResolver->getHumanReadableModuleName_(moduleName).value_or(moduleName); + return frontend->fileResolver->getHumanReadableModuleName(moduleName); } ScopePtr Frontend::addEnvironment(const std::string& environmentName) diff --git a/Analysis/src/IostreamHelpers.cpp b/Analysis/src/IostreamHelpers.cpp index 84e9b77f..3b267121 100644 --- a/Analysis/src/IostreamHelpers.cpp +++ b/Analysis/src/IostreamHelpers.cpp @@ -2,6 +2,8 @@ #include "Luau/IostreamHelpers.h" #include "Luau/ToString.h" +LUAU_FASTFLAG(LuauTypeAliasPacks) + namespace Luau { @@ -92,7 +94,7 @@ std::ostream& operator<<(std::ostream& stream, const IncorrectGenericParameterCo { stream << "IncorrectGenericParameterCount { name = " << error.name; - if (!error.typeFun.typeParams.empty()) + if (!error.typeFun.typeParams.empty() || (FFlag::LuauTypeAliasPacks && !error.typeFun.typePackParams.empty())) { stream << "<"; bool first = true; @@ -105,6 +107,20 @@ std::ostream& operator<<(std::ostream& stream, const IncorrectGenericParameterCo stream << toString(t); } + + if (FFlag::LuauTypeAliasPacks) + { + for (TypePackId t : error.typeFun.typePackParams) + { + if (first) + first = false; + else + stream << ", "; + + stream << toString(t); + } + } + stream << ">"; } diff --git a/Analysis/src/JsonEncoder.cpp b/Analysis/src/JsonEncoder.cpp index a1018297..064accba 100644 --- a/Analysis/src/JsonEncoder.cpp +++ b/Analysis/src/JsonEncoder.cpp @@ -3,6 +3,9 @@ #include "Luau/Ast.h" #include "Luau/StringUtils.h" +#include "Luau/Common.h" + +LUAU_FASTFLAG(LuauTypeAliasPacks) namespace Luau { @@ -612,6 +615,12 @@ struct AstJsonEncoder : public AstVisitor writeNode(node, "AstStatTypeAlias", [&]() { PROP(name); PROP(generics); + + if (FFlag::LuauTypeAliasPacks) + { + PROP(genericPacks); + } + PROP(type); PROP(exported); }); @@ -664,13 +673,21 @@ struct AstJsonEncoder : public AstVisitor }); } + void write(struct AstTypeOrPack node) + { + if (node.type) + write(node.type); + else + write(node.typePack); + } + void write(class AstTypeReference* node) { writeNode(node, "AstTypeReference", [&]() { if (node->hasPrefix) PROP(prefix); PROP(name); - PROP(generics); + PROP(parameters); }); } @@ -734,6 +751,13 @@ struct AstJsonEncoder : public AstVisitor }); } + void write(class AstTypePackExplicit* node) + { + writeNode(node, "AstTypePackExplicit", [&]() { + PROP(typeList); + }); + } + void write(class AstTypePackVariadic* node) { writeNode(node, "AstTypePackVariadic", [&]() { @@ -1018,6 +1042,12 @@ struct AstJsonEncoder : public AstVisitor return false; } + bool visit(class AstTypePackExplicit* node) override + { + write(node); + return false; + } + bool visit(class AstTypePackVariadic* node) override { write(node); diff --git a/Analysis/src/Linter.cpp b/Analysis/src/Linter.cpp index f97f6a4a..bff947a5 100644 --- a/Analysis/src/Linter.cpp +++ b/Analysis/src/Linter.cpp @@ -3,6 +3,7 @@ #include "Luau/AstQuery.h" #include "Luau/Module.h" +#include "Luau/Scope.h" #include "Luau/TypeInfer.h" #include "Luau/StringUtils.h" #include "Luau/Common.h" @@ -12,6 +13,7 @@ #include LUAU_FASTFLAGVARIABLE(LuauLinterUnknownTypeVectorAware, false) +LUAU_FASTFLAGVARIABLE(LuauLinterTableMoveZero, false) namespace Luau { @@ -85,10 +87,10 @@ struct LintContext return std::nullopt; auto it = module->astTypes.find(expr); - if (it == module->astTypes.end()) + if (!it) return std::nullopt; - return it->second; + return *it; } }; @@ -2144,6 +2146,19 @@ private: "wrap it in parentheses to silence"); } + if (FFlag::LuauLinterTableMoveZero && func->index == "move" && node->args.size >= 4) + { + // table.move(t, 0, _, _) + if (isConstant(args[1], 0.0)) + emitWarning(*context, LintWarning::Code_TableOperations, args[1]->location, + "table.move uses index 0 but arrays are 1-based; did you mean 1 instead?"); + + // table.move(t, _, _, 0) + else if (isConstant(args[3], 0.0)) + emitWarning(*context, LintWarning::Code_TableOperations, args[3]->location, + "table.move uses index 0 but arrays are 1-based; did you mean 1 instead?"); + } + return true; } diff --git a/Analysis/src/Module.cpp b/Analysis/src/Module.cpp index f1d975fe..df6be767 100644 --- a/Analysis/src/Module.cpp +++ b/Analysis/src/Module.cpp @@ -1,6 +1,7 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #include "Luau/Module.h" +#include "Luau/Scope.h" #include "Luau/TypeInfer.h" #include "Luau/TypePack.h" #include "Luau/TypeVar.h" @@ -13,6 +14,7 @@ LUAU_FASTFLAGVARIABLE(DebugLuauFreezeArena, false) LUAU_FASTFLAGVARIABLE(DebugLuauTrackOwningArena, false) LUAU_FASTFLAG(LuauSecondTypecheckKnowsTheDataModel) LUAU_FASTFLAG(LuauCaptureBrokenCommentSpans) +LUAU_FASTFLAG(LuauTypeAliasPacks) namespace Luau { @@ -188,7 +190,7 @@ struct TypePackCloner template void defaultClone(const T& t) { - TypePackId cloned = dest.typePacks.allocate(t); + TypePackId cloned = dest.addTypePack(TypePackVar{t}); seenTypePacks[typePackId] = cloned; } @@ -197,7 +199,7 @@ struct TypePackCloner if (encounteredFreeType) *encounteredFreeType = true; - seenTypePacks[typePackId] = dest.typePacks.allocate(TypePackVar{Unifiable::Error{}}); + seenTypePacks[typePackId] = dest.addTypePack(TypePackVar{Unifiable::Error{}}); } void operator()(const Unifiable::Generic& t) @@ -219,13 +221,13 @@ struct TypePackCloner void operator()(const VariadicTypePack& t) { - TypePackId cloned = dest.typePacks.allocate(VariadicTypePack{clone(t.ty, dest, seenTypes, seenTypePacks, encounteredFreeType)}); + TypePackId cloned = dest.addTypePack(TypePackVar{VariadicTypePack{clone(t.ty, dest, seenTypes, seenTypePacks, encounteredFreeType)}}); seenTypePacks[typePackId] = cloned; } void operator()(const TypePack& t) { - TypePackId cloned = dest.typePacks.allocate(TypePack{}); + TypePackId cloned = dest.addTypePack(TypePack{}); TypePack* destTp = getMutable(cloned); LUAU_ASSERT(destTp != nullptr); seenTypePacks[typePackId] = cloned; @@ -241,7 +243,7 @@ struct TypePackCloner template void TypeCloner::defaultClone(const T& t) { - TypeId cloned = dest.typeVars.allocate(t); + TypeId cloned = dest.addType(t); seenTypes[typeId] = cloned; } @@ -250,7 +252,7 @@ void TypeCloner::operator()(const Unifiable::Free& t) if (encounteredFreeType) *encounteredFreeType = true; - seenTypes[typeId] = dest.typeVars.allocate(ErrorTypeVar{}); + seenTypes[typeId] = dest.addType(ErrorTypeVar{}); } void TypeCloner::operator()(const Unifiable::Generic& t) @@ -275,7 +277,7 @@ void TypeCloner::operator()(const PrimitiveTypeVar& t) void TypeCloner::operator()(const FunctionTypeVar& t) { - TypeId result = dest.typeVars.allocate(FunctionTypeVar{TypeLevel{0, 0}, {}, {}, nullptr, nullptr, t.definition, t.hasSelf}); + TypeId result = dest.addType(FunctionTypeVar{TypeLevel{0, 0}, {}, {}, nullptr, nullptr, t.definition, t.hasSelf}); FunctionTypeVar* ftv = getMutable(result); LUAU_ASSERT(ftv != nullptr); @@ -297,7 +299,7 @@ void TypeCloner::operator()(const FunctionTypeVar& t) void TypeCloner::operator()(const TableTypeVar& t) { - TypeId result = dest.typeVars.allocate(TableTypeVar{}); + TypeId result = dest.addType(TableTypeVar{}); TableTypeVar* ttv = getMutable(result); LUAU_ASSERT(ttv != nullptr); @@ -323,7 +325,13 @@ void TypeCloner::operator()(const TableTypeVar& t) ttv->boundTo = clone(*t.boundTo, dest, seenTypes, seenTypePacks, encounteredFreeType); for (TypeId& arg : ttv->instantiatedTypeParams) - arg = (clone(arg, dest, seenTypes, seenTypePacks, encounteredFreeType)); + arg = clone(arg, dest, seenTypes, seenTypePacks, encounteredFreeType); + + if (FFlag::LuauTypeAliasPacks) + { + for (TypePackId& arg : ttv->instantiatedTypePackParams) + arg = clone(arg, dest, seenTypes, seenTypePacks, encounteredFreeType); + } if (ttv->state == TableState::Free) { @@ -343,7 +351,7 @@ void TypeCloner::operator()(const TableTypeVar& t) void TypeCloner::operator()(const MetatableTypeVar& t) { - TypeId result = dest.typeVars.allocate(MetatableTypeVar{}); + TypeId result = dest.addType(MetatableTypeVar{}); MetatableTypeVar* mtv = getMutable(result); seenTypes[typeId] = result; @@ -353,7 +361,7 @@ void TypeCloner::operator()(const MetatableTypeVar& t) void TypeCloner::operator()(const ClassTypeVar& t) { - TypeId result = dest.typeVars.allocate(ClassTypeVar{t.name, {}, std::nullopt, std::nullopt, t.tags, t.userData}); + TypeId result = dest.addType(ClassTypeVar{t.name, {}, std::nullopt, std::nullopt, t.tags, t.userData}); ClassTypeVar* ctv = getMutable(result); seenTypes[typeId] = result; @@ -378,7 +386,7 @@ void TypeCloner::operator()(const AnyTypeVar& t) void TypeCloner::operator()(const UnionTypeVar& t) { - TypeId result = dest.typeVars.allocate(UnionTypeVar{}); + TypeId result = dest.addType(UnionTypeVar{}); seenTypes[typeId] = result; UnionTypeVar* option = getMutable(result); @@ -390,7 +398,7 @@ void TypeCloner::operator()(const UnionTypeVar& t) void TypeCloner::operator()(const IntersectionTypeVar& t) { - TypeId result = dest.typeVars.allocate(IntersectionTypeVar{}); + TypeId result = dest.addType(IntersectionTypeVar{}); seenTypes[typeId] = result; IntersectionTypeVar* option = getMutable(result); @@ -451,8 +459,14 @@ TypeId clone(TypeId typeId, TypeArena& dest, SeenTypes& seenTypes, SeenTypePacks TypeFun clone(const TypeFun& typeFun, TypeArena& dest, SeenTypes& seenTypes, SeenTypePacks& seenTypePacks, bool* encounteredFreeType) { TypeFun result; - for (TypeId param : typeFun.typeParams) - result.typeParams.push_back(clone(param, dest, seenTypes, seenTypePacks, encounteredFreeType)); + for (TypeId ty : typeFun.typeParams) + result.typeParams.push_back(clone(ty, dest, seenTypes, seenTypePacks, encounteredFreeType)); + + if (FFlag::LuauTypeAliasPacks) + { + for (TypePackId tp : typeFun.typePackParams) + result.typePackParams.push_back(clone(tp, dest, seenTypes, seenTypePacks, encounteredFreeType)); + } result.type = clone(typeFun.type, dest, seenTypes, seenTypePacks, encounteredFreeType); diff --git a/Analysis/src/RequireTracer.cpp b/Analysis/src/RequireTracer.cpp index 5b3997e2..ad4d5ef4 100644 --- a/Analysis/src/RequireTracer.cpp +++ b/Analysis/src/RequireTracer.cpp @@ -5,6 +5,7 @@ #include "Luau/Module.h" LUAU_FASTFLAGVARIABLE(LuauTraceRequireLookupChild, false) +LUAU_FASTFLAGVARIABLE(LuauNewRequireTrace, false) namespace Luau { @@ -12,17 +13,18 @@ namespace Luau namespace { -struct RequireTracer : AstVisitor +struct RequireTracerOld : AstVisitor { - explicit RequireTracer(FileResolver* fileResolver, ModuleName currentModuleName) + explicit RequireTracerOld(FileResolver* fileResolver, const ModuleName& currentModuleName) : fileResolver(fileResolver) - , currentModuleName(std::move(currentModuleName)) + , currentModuleName(currentModuleName) { + LUAU_ASSERT(!FFlag::LuauNewRequireTrace); } FileResolver* const fileResolver; ModuleName currentModuleName; - DenseHashMap locals{0}; + DenseHashMap locals{nullptr}; RequireTraceResult result; std::optional fromAstFragment(AstExpr* expr) @@ -50,9 +52,9 @@ struct RequireTracer : AstVisitor AstExpr* expr = stat->values.data[i]; expr->visit(this); - const ModuleName* name = result.exprs.find(expr); - if (name) - locals[local] = *name; + const ModuleInfo* info = result.exprs.find(expr); + if (info) + locals[local] = info->name; } } @@ -63,7 +65,7 @@ struct RequireTracer : AstVisitor { std::optional name = fromAstFragment(global); if (name) - result.exprs[global] = *name; + result.exprs[global] = {*name}; return false; } @@ -72,7 +74,7 @@ struct RequireTracer : AstVisitor { const ModuleName* name = locals.find(local->local); if (name) - result.exprs[local] = *name; + result.exprs[local] = {*name}; return false; } @@ -81,16 +83,16 @@ struct RequireTracer : AstVisitor { indexName->expr->visit(this); - const ModuleName* name = result.exprs.find(indexName->expr); - if (name) + const ModuleInfo* info = result.exprs.find(indexName->expr); + if (info) { if (indexName->index == "parent" || indexName->index == "Parent") { - if (auto parent = fileResolver->getParentModuleName(*name)) - result.exprs[indexName] = *parent; + if (auto parent = fileResolver->getParentModuleName(info->name)) + result.exprs[indexName] = {*parent}; } else - result.exprs[indexName] = fileResolver->concat(*name, indexName->index.value); + result.exprs[indexName] = {fileResolver->concat(info->name, indexName->index.value)}; } return false; @@ -100,11 +102,11 @@ struct RequireTracer : AstVisitor { indexExpr->expr->visit(this); - const ModuleName* name = result.exprs.find(indexExpr->expr); + const ModuleInfo* info = result.exprs.find(indexExpr->expr); const AstExprConstantString* str = indexExpr->index->as(); - if (name && str) + if (info && str) { - result.exprs[indexExpr] = fileResolver->concat(*name, std::string_view(str->value.data, str->value.size)); + result.exprs[indexExpr] = {fileResolver->concat(info->name, std::string_view(str->value.data, str->value.size))}; } indexExpr->index->visit(this); @@ -129,8 +131,8 @@ struct RequireTracer : AstVisitor AstExprGlobal* globalName = call->func->as(); if (globalName && globalName->name == "require" && call->args.size >= 1) { - if (const ModuleName* moduleName = result.exprs.find(call->args.data[0])) - result.requires.push_back({*moduleName, call->location}); + if (const ModuleInfo* moduleInfo = result.exprs.find(call->args.data[0])) + result.requires.push_back({moduleInfo->name, call->location}); return false; } @@ -143,8 +145,8 @@ struct RequireTracer : AstVisitor if (FFlag::LuauTraceRequireLookupChild && !rootName) { - if (const ModuleName* moduleName = result.exprs.find(indexName->expr)) - rootName = *moduleName; + if (const ModuleInfo* moduleInfo = result.exprs.find(indexName->expr)) + rootName = moduleInfo->name; } if (!rootName) @@ -167,24 +169,183 @@ struct RequireTracer : AstVisitor if (v.end() != std::find(v.begin(), v.end(), '/')) return false; - result.exprs[call] = fileResolver->concat(*rootName, v); + result.exprs[call] = {fileResolver->concat(*rootName, v)}; // 'WaitForChild' can be used on modules that are not awailable at the typecheck time, but will be awailable at runtime // If we fail to find such module, we will not report an UnknownRequire error if (FFlag::LuauTraceRequireLookupChild && indexName->index == "WaitForChild") - result.optional[call] = true; + result.exprs[call].optional = true; return false; } }; +struct RequireTracer : AstVisitor +{ + RequireTracer(RequireTraceResult& result, FileResolver * fileResolver, const ModuleName& currentModuleName) + : result(result) + , fileResolver(fileResolver) + , currentModuleName(currentModuleName) + , locals(nullptr) + { + LUAU_ASSERT(FFlag::LuauNewRequireTrace); + } + + bool visit(AstExprTypeAssertion* expr) override + { + // suppress `require() :: any` + return false; + } + + bool visit(AstExprCall* expr) override + { + AstExprGlobal* global = expr->func->as(); + + if (global && global->name == "require" && expr->args.size >= 1) + requires.push_back(expr); + + return true; + } + + bool visit(AstStatLocal* stat) override + { + for (size_t i = 0; i < stat->vars.size && i < stat->values.size; ++i) + { + AstLocal* local = stat->vars.data[i]; + AstExpr* expr = stat->values.data[i]; + + // track initializing expression to be able to trace modules through locals + locals[local] = expr; + } + + return true; + } + + bool visit(AstStatAssign* stat) override + { + for (size_t i = 0; i < stat->vars.size; ++i) + { + // locals that are assigned don't have a known expression + if (AstExprLocal* expr = stat->vars.data[i]->as()) + locals[expr->local] = nullptr; + } + + return true; + } + + bool visit(AstType* node) override + { + // allow resolving require inside `typeof` annotations + return true; + } + + AstExpr* getDependent(AstExpr* node) + { + if (AstExprLocal* expr = node->as()) + return locals[expr->local]; + else if (AstExprIndexName* expr = node->as()) + return expr->expr; + else if (AstExprIndexExpr* expr = node->as()) + return expr->expr; + else if (AstExprCall* expr = node->as(); expr && expr->self) + return expr->func->as()->expr; + else + return nullptr; + } + + void process() + { + ModuleInfo moduleContext{currentModuleName}; + + // seed worklist with require arguments + work.reserve(requires.size()); + + for (AstExprCall* require: requires) + work.push_back(require->args.data[0]); + + // push all dependent expressions to the work stack; note that the vector is modified during traversal + for (size_t i = 0; i < work.size(); ++i) + if (AstExpr* dep = getDependent(work[i])) + work.push_back(dep); + + // resolve all expressions to a module info + for (size_t i = work.size(); i > 0; --i) + { + AstExpr* expr = work[i - 1]; + + // when multiple expressions depend on the same one we push it to work queue multiple times + if (result.exprs.contains(expr)) + continue; + + std::optional info; + + if (AstExpr* dep = getDependent(expr)) + { + const ModuleInfo* context = result.exprs.find(dep); + + // locals just inherit their dependent context, no resolution required + if (expr->is()) + info = context ? std::optional(*context) : std::nullopt; + else + info = fileResolver->resolveModule(context, expr); + } + else + { + info = fileResolver->resolveModule(&moduleContext, expr); + } + + if (info) + result.exprs[expr] = std::move(*info); + } + + // resolve all requires according to their argument + result.requires.reserve(requires.size()); + + for (AstExprCall* require : requires) + { + AstExpr* arg = require->args.data[0]; + + if (const ModuleInfo* info = result.exprs.find(arg)) + { + result.requires.push_back({info->name, require->location}); + + ModuleInfo infoCopy = *info; // copy *info out since next line invalidates info! + result.exprs[require] = std::move(infoCopy); + } + else + { + result.exprs[require] = {}; // mark require as unresolved + } + } + } + + RequireTraceResult& result; + FileResolver* fileResolver; + ModuleName currentModuleName; + + DenseHashMap locals; + std::vector work; + std::vector requires; +}; + } // anonymous namespace -RequireTraceResult traceRequires(FileResolver* fileResolver, AstStatBlock* root, ModuleName currentModuleName) +RequireTraceResult traceRequires(FileResolver* fileResolver, AstStatBlock* root, const ModuleName& currentModuleName) { - RequireTracer tracer{fileResolver, std::move(currentModuleName)}; - root->visit(&tracer); - return tracer.result; + if (FFlag::LuauNewRequireTrace) + { + RequireTraceResult result; + RequireTracer tracer{result, fileResolver, currentModuleName}; + root->visit(&tracer); + tracer.process(); + return result; + } + else + { + RequireTracerOld tracer{fileResolver, currentModuleName}; + root->visit(&tracer); + return tracer.result; + } } } // namespace Luau diff --git a/Analysis/src/Scope.cpp b/Analysis/src/Scope.cpp new file mode 100644 index 00000000..c30db9c2 --- /dev/null +++ b/Analysis/src/Scope.cpp @@ -0,0 +1,123 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details + +#include "Luau/Scope.h" + +namespace Luau +{ + +Scope::Scope(TypePackId returnType) + : parent(nullptr) + , returnType(returnType) + , level(TypeLevel()) +{ +} + +Scope::Scope(const ScopePtr& parent, int subLevel) + : parent(parent) + , returnType(parent->returnType) + , level(parent->level.incr()) +{ + level.subLevel = subLevel; +} + +std::optional Scope::lookup(const Symbol& name) +{ + Scope* scope = this; + + while (scope) + { + auto it = scope->bindings.find(name); + if (it != scope->bindings.end()) + return it->second.typeId; + + scope = scope->parent.get(); + } + + return std::nullopt; +} + +std::optional Scope::lookupType(const Name& name) +{ + const Scope* scope = this; + while (true) + { + auto it = scope->exportedTypeBindings.find(name); + if (it != scope->exportedTypeBindings.end()) + return it->second; + + it = scope->privateTypeBindings.find(name); + if (it != scope->privateTypeBindings.end()) + return it->second; + + if (scope->parent) + scope = scope->parent.get(); + else + return std::nullopt; + } +} + +std::optional Scope::lookupImportedType(const Name& moduleAlias, const Name& name) +{ + const Scope* scope = this; + while (scope) + { + auto it = scope->importedTypeBindings.find(moduleAlias); + if (it == scope->importedTypeBindings.end()) + { + scope = scope->parent.get(); + continue; + } + + auto it2 = it->second.find(name); + if (it2 == it->second.end()) + { + scope = scope->parent.get(); + continue; + } + + return it2->second; + } + + return std::nullopt; +} + +std::optional Scope::lookupPack(const Name& name) +{ + const Scope* scope = this; + while (true) + { + auto it = scope->privateTypePackBindings.find(name); + if (it != scope->privateTypePackBindings.end()) + return it->second; + + if (scope->parent) + scope = scope->parent.get(); + else + return std::nullopt; + } +} + +std::optional Scope::linearSearchForBinding(const std::string& name, bool traverseScopeChain) +{ + Scope* scope = this; + + while (scope) + { + for (const auto& [n, binding] : scope->bindings) + { + if (n.local && n.local->name == name.c_str()) + return binding; + else if (n.global.value && n.global == name.c_str()) + return binding; + } + + scope = scope->parent.get(); + + if (!traverseScopeChain) + break; + } + + return std::nullopt; +} + +} // namespace Luau diff --git a/Analysis/src/Substitution.cpp b/Analysis/src/Substitution.cpp index 7223998a..d861eb3d 100644 --- a/Analysis/src/Substitution.cpp +++ b/Analysis/src/Substitution.cpp @@ -6,9 +6,11 @@ #include #include -LUAU_FASTINTVARIABLE(LuauTarjanChildLimit, 0) +LUAU_FASTINTVARIABLE(LuauTarjanChildLimit, 1000) +LUAU_FASTFLAGVARIABLE(LuauSubstitutionDontReplaceIgnoredTypes, false) LUAU_FASTFLAG(LuauSecondTypecheckKnowsTheDataModel) LUAU_FASTFLAG(LuauRankNTypes) +LUAU_FASTFLAG(LuauTypeAliasPacks) namespace Luau { @@ -35,8 +37,15 @@ void Tarjan::visitChildren(TypeId ty, int index) visitChild(ttv->indexer->indexType); visitChild(ttv->indexer->indexResultType); } + for (TypeId itp : ttv->instantiatedTypeParams) visitChild(itp); + + if (FFlag::LuauTypeAliasPacks) + { + for (TypePackId itp : ttv->instantiatedTypePackParams) + visitChild(itp); + } } else if (const MetatableTypeVar* mtv = get(ty)) { @@ -332,9 +341,11 @@ std::optional Substitution::substitute(TypeId ty) return std::nullopt; for (auto [oldTy, newTy] : newTypes) - replaceChildren(newTy); + if (!FFlag::LuauSubstitutionDontReplaceIgnoredTypes || !ignoreChildren(oldTy)) + replaceChildren(newTy); for (auto [oldTp, newTp] : newPacks) - replaceChildren(newTp); + if (!FFlag::LuauSubstitutionDontReplaceIgnoredTypes || !ignoreChildren(oldTp)) + replaceChildren(newTp); TypeId newTy = replace(ty); return newTy; } @@ -350,9 +361,11 @@ std::optional Substitution::substitute(TypePackId tp) return std::nullopt; for (auto [oldTy, newTy] : newTypes) - replaceChildren(newTy); + if (!FFlag::LuauSubstitutionDontReplaceIgnoredTypes || !ignoreChildren(oldTy)) + replaceChildren(newTy); for (auto [oldTp, newTp] : newPacks) - replaceChildren(newTp); + if (!FFlag::LuauSubstitutionDontReplaceIgnoredTypes || !ignoreChildren(oldTp)) + replaceChildren(newTp); TypePackId newTp = replace(tp); return newTp; } @@ -382,6 +395,10 @@ TypeId Substitution::clone(TypeId ty) clone.name = ttv->name; clone.syntheticName = ttv->syntheticName; clone.instantiatedTypeParams = ttv->instantiatedTypeParams; + + if (FFlag::LuauTypeAliasPacks) + clone.instantiatedTypePackParams = ttv->instantiatedTypePackParams; + if (FFlag::LuauSecondTypecheckKnowsTheDataModel) clone.tags = ttv->tags; result = addType(std::move(clone)); @@ -487,8 +504,15 @@ void Substitution::replaceChildren(TypeId ty) ttv->indexer->indexType = replace(ttv->indexer->indexType); ttv->indexer->indexResultType = replace(ttv->indexer->indexResultType); } + for (TypeId& itp : ttv->instantiatedTypeParams) itp = replace(itp); + + if (FFlag::LuauTypeAliasPacks) + { + for (TypePackId& itp : ttv->instantiatedTypePackParams) + itp = replace(itp); + } } else if (MetatableTypeVar* mtv = getMutable(ty)) { diff --git a/Analysis/src/ToString.cpp b/Analysis/src/ToString.cpp index 9d2f47ba..5651af7e 100644 --- a/Analysis/src/ToString.cpp +++ b/Analysis/src/ToString.cpp @@ -1,6 +1,7 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #include "Luau/ToString.h" +#include "Luau/Scope.h" #include "Luau/TypeInfer.h" #include "Luau/TypePack.h" #include "Luau/TypeVar.h" @@ -9,10 +10,10 @@ #include #include -LUAU_FASTFLAG(LuauToStringFollowsBoundTo) LUAU_FASTFLAG(LuauExtraNilRecovery) LUAU_FASTFLAG(LuauOccursCheckOkWithRecursiveFunctions) LUAU_FASTFLAGVARIABLE(LuauInstantiatedTypeParamRecursion, false) +LUAU_FASTFLAG(LuauTypeAliasPacks) namespace Luau { @@ -59,6 +60,13 @@ struct FindCyclicTypes { for (TypeId itp : ttv.instantiatedTypeParams) visitTypeVar(itp, *this, seen); + + if (FFlag::LuauTypeAliasPacks) + { + for (TypePackId itp : ttv.instantiatedTypePackParams) + visitTypeVar(itp, *this, seen); + } + return exhaustive; } @@ -258,23 +266,60 @@ struct TypeVarStringifier void stringify(TypePackId tp); void stringify(TypePackId tpid, const std::vector>& names); - void stringify(const std::vector& types) + void stringify(const std::vector& types, const std::vector& typePacks) { - if (types.size() == 0) + if (types.size() == 0 && (!FFlag::LuauTypeAliasPacks || typePacks.size() == 0)) return; - if (types.size()) + if (types.size() || (FFlag::LuauTypeAliasPacks && typePacks.size())) state.emit("<"); - for (size_t i = 0; i < types.size(); ++i) + if (FFlag::LuauTypeAliasPacks) { - if (i > 0) - state.emit(", "); + bool first = true; - stringify(types[i]); + for (TypeId ty : types) + { + if (!first) + state.emit(", "); + first = false; + + stringify(ty); + } + + bool singleTp = typePacks.size() == 1; + + for (TypePackId tp : typePacks) + { + if (isEmpty(tp) && singleTp) + continue; + + if (!first) + state.emit(", "); + else + first = false; + + if (!singleTp) + state.emit("("); + + stringify(tp); + + if (!singleTp) + state.emit(")"); + } + } + else + { + for (size_t i = 0; i < types.size(); ++i) + { + if (i > 0) + state.emit(", "); + + stringify(types[i]); + } } - if (types.size()) + if (types.size() || (FFlag::LuauTypeAliasPacks && typePacks.size())) state.emit(">"); } @@ -388,7 +433,7 @@ struct TypeVarStringifier void operator()(TypeId, const TableTypeVar& ttv) { - if (FFlag::LuauToStringFollowsBoundTo && ttv.boundTo) + if (ttv.boundTo) return stringify(*ttv.boundTo); if (!state.exhaustive) @@ -411,14 +456,14 @@ struct TypeVarStringifier } state.emit(*ttv.name); - stringify(ttv.instantiatedTypeParams); + stringify(ttv.instantiatedTypeParams, ttv.instantiatedTypePackParams); return; } if (ttv.syntheticName) { state.result.invalid = true; state.emit(*ttv.syntheticName); - stringify(ttv.instantiatedTypeParams); + stringify(ttv.instantiatedTypeParams, ttv.instantiatedTypePackParams); return; } } @@ -900,13 +945,26 @@ ToStringResult toStringDetailed(TypeId ty, const ToStringOptions& opts) result.name += ttv->name ? *ttv->name : *ttv->syntheticName; - if (ttv->instantiatedTypeParams.empty()) + if (ttv->instantiatedTypeParams.empty() && (!FFlag::LuauTypeAliasPacks || ttv->instantiatedTypePackParams.empty())) return result; std::vector params; for (TypeId tp : ttv->instantiatedTypeParams) params.push_back(toString(tp)); + if (FFlag::LuauTypeAliasPacks) + { + // Doesn't preserve grouping of multiple type packs + // But this is under a parent block of code that is being removed later + for (TypePackId tp : ttv->instantiatedTypePackParams) + { + std::string content = toString(tp); + + if (!content.empty()) + params.push_back(std::move(content)); + } + } + result.name += "<" + join(params, ", ") + ">"; return result; } @@ -950,30 +1008,37 @@ ToStringResult toStringDetailed(TypeId ty, const ToStringOptions& opts) result.name += ttv->name ? *ttv->name : *ttv->syntheticName; - if (ttv->instantiatedTypeParams.empty()) - return result; - - result.name += "<"; - - bool first = true; - for (TypeId ty : ttv->instantiatedTypeParams) + if (FFlag::LuauTypeAliasPacks) { - if (!first) - result.name += ", "; - else - first = false; - - tvs.stringify(ty); - } - - if (opts.maxTypeLength > 0 && result.name.length() > opts.maxTypeLength) - { - result.truncated = true; - result.name += "... "; + tvs.stringify(ttv->instantiatedTypeParams, ttv->instantiatedTypePackParams); } else { - result.name += ">"; + if (ttv->instantiatedTypeParams.empty() && (!FFlag::LuauTypeAliasPacks || ttv->instantiatedTypePackParams.empty())) + return result; + + result.name += "<"; + + bool first = true; + for (TypeId ty : ttv->instantiatedTypeParams) + { + if (!first) + result.name += ", "; + else + first = false; + + tvs.stringify(ty); + } + + if (opts.maxTypeLength > 0 && result.name.length() > opts.maxTypeLength) + { + result.truncated = true; + result.name += "... "; + } + else + { + result.name += ">"; + } } return result; diff --git a/Analysis/src/Transpiler.cpp b/Analysis/src/Transpiler.cpp index 462c70ff..1b83ccdc 100644 --- a/Analysis/src/Transpiler.cpp +++ b/Analysis/src/Transpiler.cpp @@ -11,6 +11,7 @@ #include LUAU_FASTFLAG(LuauGenericFunctions) +LUAU_FASTFLAG(LuauTypeAliasPacks) namespace { @@ -280,10 +281,19 @@ struct Printer void visualizeTypePackAnnotation(const AstTypePack& annotation) { - if (const AstTypePackVariadic* variadic = annotation.as()) + if (const AstTypePackVariadic* variadicTp = annotation.as()) { writer.symbol("..."); - visualizeTypeAnnotation(*variadic->variadicType); + visualizeTypeAnnotation(*variadicTp->variadicType); + } + else if (const AstTypePackGeneric* genericTp = annotation.as()) + { + writer.symbol(genericTp->genericName.value); + writer.symbol("..."); + } + else if (const AstTypePackExplicit* explicitTp = annotation.as()) + { + visualizeTypeList(explicitTp->typeList, true); } else { @@ -807,7 +817,7 @@ struct Printer writer.keyword("type"); writer.identifier(a->name.value); - if (a->generics.size > 0) + if (a->generics.size > 0 || (FFlag::LuauTypeAliasPacks && a->genericPacks.size > 0)) { writer.symbol("<"); CommaSeparatorInserter comma(writer); @@ -817,6 +827,17 @@ struct Printer comma(); writer.identifier(o.value); } + + if (FFlag::LuauTypeAliasPacks) + { + for (auto o : a->genericPacks) + { + comma(); + writer.identifier(o.value); + writer.symbol("..."); + } + } + writer.symbol(">"); } writer.maybeSpace(a->type->location.begin, 2); @@ -960,15 +981,20 @@ struct Printer if (const auto& a = typeAnnotation.as()) { writer.write(a->name.value); - if (a->generics.size > 0) + if (a->parameters.size > 0) { CommaSeparatorInserter comma(writer); writer.symbol("<"); - for (auto o : a->generics) + for (auto o : a->parameters) { comma(); - visualizeTypeAnnotation(*o); + + if (o.type) + visualizeTypeAnnotation(*o.type); + else + visualizeTypePackAnnotation(*o.typePack); } + writer.symbol(">"); } } diff --git a/Analysis/src/TypeAttach.cpp b/Analysis/src/TypeAttach.cpp index 17c57c84..266c1986 100644 --- a/Analysis/src/TypeAttach.cpp +++ b/Analysis/src/TypeAttach.cpp @@ -5,6 +5,7 @@ #include "Luau/Module.h" #include "Luau/Parser.h" #include "Luau/RecursionCounter.h" +#include "Luau/Scope.h" #include "Luau/TypeInfer.h" #include "Luau/TypePack.h" #include "Luau/TypeVar.h" @@ -12,6 +13,7 @@ #include LUAU_FASTFLAG(LuauGenericFunctions) +LUAU_FASTFLAG(LuauTypeAliasPacks) static char* allocateString(Luau::Allocator& allocator, std::string_view contents) { @@ -33,7 +35,6 @@ static char* allocateString(Luau::Allocator& allocator, const char* format, Data namespace Luau { - class TypeRehydrationVisitor { mutable std::map seen; @@ -57,6 +58,8 @@ public: { } + AstTypePack* rehydrate(TypePackId tp) const; + AstType* operator()(const PrimitiveTypeVar& ptv) const { switch (ptv.type) @@ -85,16 +88,24 @@ public: if (ttv.name && options.bannedNames.find(*ttv.name) == options.bannedNames.end()) { - AstArray generics; - generics.size = ttv.instantiatedTypeParams.size(); - generics.data = static_cast(allocator->allocate(sizeof(AstType*) * generics.size)); + AstArray parameters; + parameters.size = ttv.instantiatedTypeParams.size(); + parameters.data = static_cast(allocator->allocate(sizeof(AstTypeOrPack) * parameters.size)); for (size_t i = 0; i < ttv.instantiatedTypeParams.size(); ++i) { - generics.data[i] = Luau::visit(*this, ttv.instantiatedTypeParams[i]->ty); + parameters.data[i] = {Luau::visit(*this, ttv.instantiatedTypeParams[i]->ty), {}}; } - return allocator->alloc(Location(), std::nullopt, AstName(ttv.name->c_str()), generics); + if (FFlag::LuauTypeAliasPacks) + { + for (size_t i = 0; i < ttv.instantiatedTypePackParams.size(); ++i) + { + parameters.data[i] = {{}, rehydrate(ttv.instantiatedTypePackParams[i])}; + } + } + + return allocator->alloc(Location(), std::nullopt, AstName(ttv.name->c_str()), parameters.size != 0, parameters); } if (hasSeen(&ttv)) @@ -222,10 +233,17 @@ public: AstTypePack* argTailAnnotation = nullptr; if (argTail) { - TypePackId tail = *argTail; - if (const VariadicTypePack* vtp = get(tail)) + if (FFlag::LuauTypeAliasPacks) { - argTailAnnotation = allocator->alloc(Location(), Luau::visit(*this, vtp->ty->ty)); + argTailAnnotation = rehydrate(*argTail); + } + else + { + TypePackId tail = *argTail; + if (const VariadicTypePack* vtp = get(tail)) + { + argTailAnnotation = allocator->alloc(Location(), Luau::visit(*this, vtp->ty->ty)); + } } } @@ -255,10 +273,17 @@ public: AstTypePack* retTailAnnotation = nullptr; if (retTail) { - TypePackId tail = *retTail; - if (const VariadicTypePack* vtp = get(tail)) + if (FFlag::LuauTypeAliasPacks) { - retTailAnnotation = allocator->alloc(Location(), Luau::visit(*this, vtp->ty->ty)); + retTailAnnotation = rehydrate(*retTail); + } + else + { + TypePackId tail = *retTail; + if (const VariadicTypePack* vtp = get(tail)) + { + retTailAnnotation = allocator->alloc(Location(), Luau::visit(*this, vtp->ty->ty)); + } } } @@ -313,6 +338,68 @@ private: const TypeRehydrationOptions& options; }; +class TypePackRehydrationVisitor +{ +public: + TypePackRehydrationVisitor(Allocator* allocator, const TypeRehydrationVisitor& typeVisitor) + : allocator(allocator) + , typeVisitor(typeVisitor) + { + } + + AstTypePack* operator()(const BoundTypePack& btp) const + { + return Luau::visit(*this, btp.boundTo->ty); + } + + AstTypePack* operator()(const TypePack& tp) const + { + AstArray head; + head.size = tp.head.size(); + head.data = static_cast(allocator->allocate(sizeof(AstType*) * tp.head.size())); + + for (size_t i = 0; i < tp.head.size(); i++) + head.data[i] = Luau::visit(typeVisitor, tp.head[i]->ty); + + AstTypePack* tail = nullptr; + + if (tp.tail) + tail = Luau::visit(*this, (*tp.tail)->ty); + + return allocator->alloc(Location(), AstTypeList{head, tail}); + } + + AstTypePack* operator()(const VariadicTypePack& vtp) const + { + return allocator->alloc(Location(), Luau::visit(typeVisitor, vtp.ty->ty)); + } + + AstTypePack* operator()(const GenericTypePack& gtp) const + { + return allocator->alloc(Location(), AstName(gtp.name.c_str())); + } + + AstTypePack* operator()(const FreeTypePack& gtp) const + { + return allocator->alloc(Location(), AstName("free")); + } + + AstTypePack* operator()(const Unifiable::Error&) const + { + return allocator->alloc(Location(), AstName("Unifiable")); + } + +private: + Allocator* allocator; + const TypeRehydrationVisitor& typeVisitor; +}; + +AstTypePack* TypeRehydrationVisitor::rehydrate(TypePackId tp) const +{ + TypePackRehydrationVisitor tprv(allocator, *this); + return Luau::visit(tprv, tp->ty); +} + class TypeAttacher : public AstVisitor { public: @@ -406,9 +493,16 @@ public: if (tail) { - TypePackId tailPack = *tail; - if (const VariadicTypePack* vtp = get(tailPack)) - variadicAnnotation = allocator->alloc(Location(), typeAst(vtp->ty)); + if (FFlag::LuauTypeAliasPacks) + { + variadicAnnotation = TypeRehydrationVisitor(allocator).rehydrate(*tail); + } + else + { + TypePackId tailPack = *tail; + if (const VariadicTypePack* vtp = get(tailPack)) + variadicAnnotation = allocator->alloc(Location(), typeAst(vtp->ty)); + } } fn->returnAnnotation = AstTypeList{typeAstPack(ret), variadicAnnotation}; diff --git a/Analysis/src/TypeInfer.cpp b/Analysis/src/TypeInfer.cpp index 2216881b..3a1fdfff 100644 --- a/Analysis/src/TypeInfer.cpp +++ b/Analysis/src/TypeInfer.cpp @@ -5,21 +5,22 @@ #include "Luau/ModuleResolver.h" #include "Luau/Parser.h" #include "Luau/RecursionCounter.h" +#include "Luau/Scope.h" #include "Luau/Substitution.h" #include "Luau/TopoSortStatements.h" #include "Luau/ToString.h" #include "Luau/TypePack.h" #include "Luau/TypeUtils.h" #include "Luau/TypeVar.h" +#include "Luau/TimeTrace.h" #include #include LUAU_FASTFLAGVARIABLE(DebugLuauMagicTypes, false) -LUAU_FASTINTVARIABLE(LuauTypeInferRecursionLimit, 0) -LUAU_FASTINTVARIABLE(LuauTypeInferTypePackLoopLimit, 0) +LUAU_FASTINTVARIABLE(LuauTypeInferRecursionLimit, 500) +LUAU_FASTINTVARIABLE(LuauTypeInferTypePackLoopLimit, 5000) LUAU_FASTINTVARIABLE(LuauCheckRecursionLimit, 500) -LUAU_FASTFLAGVARIABLE(LuauIndexTablesWithIndexers, false) LUAU_FASTFLAGVARIABLE(LuauGenericFunctions, false) LUAU_FASTFLAGVARIABLE(LuauGenericVariadicsUnification, false) LUAU_FASTFLAG(LuauKnowsTheDataModel3) @@ -27,14 +28,11 @@ LUAU_FASTFLAG(LuauSecondTypecheckKnowsTheDataModel) LUAU_FASTFLAGVARIABLE(LuauClassPropertyAccessAsString, false) LUAU_FASTFLAGVARIABLE(LuauEqConstraint, false) LUAU_FASTFLAGVARIABLE(LuauWeakEqConstraint, false) // Eventually removed as false. -LUAU_FASTFLAGVARIABLE(LuauImprovedTypeGuardPredicate2, false) LUAU_FASTFLAG(LuauTraceRequireLookupChild) -LUAU_FASTFLAG(DebugLuauTrackOwningArena) LUAU_FASTFLAGVARIABLE(LuauCloneCorrectlyBeforeMutatingTableType, false) LUAU_FASTFLAGVARIABLE(LuauStoreMatchingOverloadFnType, false) LUAU_FASTFLAGVARIABLE(LuauRankNTypes, false) LUAU_FASTFLAGVARIABLE(LuauOrPredicate, false) -LUAU_FASTFLAGVARIABLE(LuauFixTableTypeAliasClone, false) LUAU_FASTFLAGVARIABLE(LuauExtraNilRecovery, false) LUAU_FASTFLAGVARIABLE(LuauMissingUnionPropertyError, false) LUAU_FASTFLAGVARIABLE(LuauInferReturnAssertAssign, false) @@ -45,6 +43,10 @@ LUAU_FASTFLAGVARIABLE(LuauSlightlyMoreFlexibleBinaryPredicates, false) LUAU_FASTFLAGVARIABLE(LuauInferFunctionArgsFix, false) LUAU_FASTFLAGVARIABLE(LuauFollowInTypeFunApply, false) LUAU_FASTFLAGVARIABLE(LuauIfElseExpressionAnalysisSupport, false) +LUAU_FASTFLAGVARIABLE(LuauStrictRequire, false) +LUAU_FASTFLAG(LuauSubstitutionDontReplaceIgnoredTypes) +LUAU_FASTFLAG(LuauNewRequireTrace) +LUAU_FASTFLAG(LuauTypeAliasPacks) namespace Luau { @@ -216,9 +218,8 @@ TypeChecker::TypeChecker(ModuleResolver* resolver, InternalErrorReporter* iceHan , nilType(singletonTypes.nilType) , numberType(singletonTypes.numberType) , stringType(singletonTypes.stringType) - , booleanType( - FFlag::LuauImprovedTypeGuardPredicate2 ? singletonTypes.booleanType : globalTypes.addType(PrimitiveTypeVar(PrimitiveTypeVar::Boolean))) - , threadType(FFlag::LuauImprovedTypeGuardPredicate2 ? singletonTypes.threadType : globalTypes.addType(PrimitiveTypeVar(PrimitiveTypeVar::Thread))) + , booleanType(singletonTypes.booleanType) + , threadType(singletonTypes.threadType) , anyType(singletonTypes.anyType) , errorType(singletonTypes.errorType) , optionalNumberType(globalTypes.addType(UnionTypeVar{{numberType, nilType}})) @@ -237,6 +238,9 @@ TypeChecker::TypeChecker(ModuleResolver* resolver, InternalErrorReporter* iceHan ModulePtr TypeChecker::check(const SourceModule& module, Mode mode, std::optional environmentScope) { + LUAU_TIMETRACE_SCOPE("TypeChecker::check", "TypeChecker"); + LUAU_TIMETRACE_ARGUMENT("module", module.name.c_str()); + currentModule.reset(new Module()); currentModule->type = module.type; @@ -1177,44 +1181,61 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatTypeAlias& typealias { Location location = scope->typeAliasLocations[name]; reportError(TypeError{typealias.location, DuplicateTypeDefinition{name, location}}); - bindingsMap[name] = TypeFun{binding->typeParams, errorType}; + + if (FFlag::LuauTypeAliasPacks) + bindingsMap[name] = TypeFun{binding->typeParams, binding->typePackParams, errorType}; + else + bindingsMap[name] = TypeFun{binding->typeParams, errorType}; } else { ScopePtr aliasScope = childScope(scope, typealias.location); - std::vector generics; - for (AstName generic : typealias.generics) + if (FFlag::LuauTypeAliasPacks) { - Name n = generic.value; + auto [generics, genericPacks] = createGenericTypes(aliasScope, typealias, typealias.generics, typealias.genericPacks); - // These generics are the only thing that will ever be added to aliasScope, so we can be certain that - // a collision can only occur when two generic typevars have the same name. - if (aliasScope->privateTypeBindings.end() != aliasScope->privateTypeBindings.find(n)) - { - // TODO(jhuelsman): report the exact span of the generic type parameter whose name is a duplicate. - reportError(TypeError{typealias.location, DuplicateGenericParameter{n}}); - } - - TypeId g; - if (FFlag::LuauRecursiveTypeParameterRestriction) - { - TypeId& cached = scope->typeAliasParameters[n]; - if (!cached) - cached = addType(GenericTypeVar{aliasScope->level, n}); - g = cached; - } - else - g = addType(GenericTypeVar{aliasScope->level, n}); - generics.push_back(g); - aliasScope->privateTypeBindings[n] = TypeFun{{}, g}; + TypeId ty = (FFlag::LuauRankNTypes ? freshType(aliasScope) : DEPRECATED_freshType(scope, true)); + FreeTypeVar* ftv = getMutable(ty); + LUAU_ASSERT(ftv); + ftv->forwardedTypeAlias = true; + bindingsMap[name] = {std::move(generics), std::move(genericPacks), ty}; } + else + { + std::vector generics; + for (AstName generic : typealias.generics) + { + Name n = generic.value; - TypeId ty = (FFlag::LuauRankNTypes ? freshType(aliasScope) : DEPRECATED_freshType(scope, true)); - FreeTypeVar* ftv = getMutable(ty); - LUAU_ASSERT(ftv); - ftv->forwardedTypeAlias = true; - bindingsMap[name] = {std::move(generics), ty}; + // These generics are the only thing that will ever be added to aliasScope, so we can be certain that + // a collision can only occur when two generic typevars have the same name. + if (aliasScope->privateTypeBindings.end() != aliasScope->privateTypeBindings.find(n)) + { + // TODO(jhuelsman): report the exact span of the generic type parameter whose name is a duplicate. + reportError(TypeError{typealias.location, DuplicateGenericParameter{n}}); + } + + TypeId g; + if (FFlag::LuauRecursiveTypeParameterRestriction) + { + TypeId& cached = scope->typeAliasTypeParameters[n]; + if (!cached) + cached = addType(GenericTypeVar{aliasScope->level, n}); + g = cached; + } + else + g = addType(GenericTypeVar{aliasScope->level, n}); + generics.push_back(g); + aliasScope->privateTypeBindings[n] = TypeFun{{}, g}; + } + + TypeId ty = (FFlag::LuauRankNTypes ? freshType(aliasScope) : DEPRECATED_freshType(scope, true)); + FreeTypeVar* ftv = getMutable(ty); + LUAU_ASSERT(ftv); + ftv->forwardedTypeAlias = true; + bindingsMap[name] = {std::move(generics), ty}; + } } } else @@ -1231,6 +1252,16 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatTypeAlias& typealias aliasScope->privateTypeBindings[generic->name] = TypeFun{{}, ty}; } + if (FFlag::LuauTypeAliasPacks) + { + for (TypePackId tp : binding->typePackParams) + { + auto generic = get(tp); + LUAU_ASSERT(generic); + aliasScope->privateTypePackBindings[generic->name] = tp; + } + } + TypeId ty = (FFlag::LuauRankNTypes ? resolveType(aliasScope, *typealias.type) : resolveType(aliasScope, *typealias.type, true)); if (auto ttv = getMutable(follow(ty))) { @@ -1238,7 +1269,8 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatTypeAlias& typealias if (ttv->name) { // Copy can be skipped if this is an identical alias - if (!FFlag::LuauFixTableTypeAliasClone || ttv->name != name || ttv->instantiatedTypeParams != binding->typeParams) + if (ttv->name != name || ttv->instantiatedTypeParams != binding->typeParams || + (FFlag::LuauTypeAliasPacks && ttv->instantiatedTypePackParams != binding->typePackParams)) { // This is a shallow clone, original recursive links to self are not updated TableTypeVar clone = TableTypeVar{ttv->props, ttv->indexer, ttv->level, ttv->state}; @@ -1249,6 +1281,9 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatTypeAlias& typealias clone.name = name; clone.instantiatedTypeParams = binding->typeParams; + if (FFlag::LuauTypeAliasPacks) + clone.instantiatedTypePackParams = binding->typePackParams; + ty = addType(std::move(clone)); } } @@ -1256,6 +1291,9 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatTypeAlias& typealias { ttv->name = name; ttv->instantiatedTypeParams = binding->typeParams; + + if (FFlag::LuauTypeAliasPacks) + ttv->instantiatedTypePackParams = binding->typePackParams; } } else if (auto mtv = getMutable(follow(ty))) @@ -1280,7 +1318,7 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatDeclareClass& declar } // We don't have generic classes, so this assertion _should_ never be hit. - LUAU_ASSERT(lookupType->typeParams.size() == 0); + LUAU_ASSERT(lookupType->typeParams.size() == 0 && (!FFlag::LuauTypeAliasPacks || lookupType->typePackParams.size() == 0)); superTy = lookupType->type; if (FFlag::LuauAddMissingFollow) @@ -1465,7 +1503,8 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExpr& if (FFlag::LuauStoreMatchingOverloadFnType) { - currentModule->astTypes.try_emplace(&expr, result.type); + if (!currentModule->astTypes.find(&expr)) + currentModule->astTypes[&expr] = result.type; } else { @@ -2193,7 +2232,7 @@ TypeId TypeChecker::checkRelationalOperation( * have a better, more descriptive error teed up. */ Unifier state = mkUnifier(expr.location); - if (!FFlag::LuauEqConstraint || !isEquality) + if (!isEquality) state.tryUnify(lhsType, rhsType); bool needsMetamethod = !isEquality; @@ -2262,7 +2301,7 @@ TypeId TypeChecker::checkRelationalOperation( } } - if (get(FFlag::LuauAddMissingFollow ? follow(lhsType) : lhsType) && (!FFlag::LuauEqConstraint || !isEquality)) + if (get(FFlag::LuauAddMissingFollow ? follow(lhsType) : lhsType) && !isEquality) { auto name = getIdentifierOfBaseVar(expr.left); reportError(expr.location, CannotInferBinaryOperation{expr.op, name, CannotInferBinaryOperation::Comparison}); @@ -2276,18 +2315,6 @@ TypeId TypeChecker::checkRelationalOperation( return errorType; } - if (!FFlag::LuauEqConstraint) - { - if (isEquality) - { - ErrorVec errVec = tryUnify(rhsType, lhsType, expr.location); - if (!state.errors.empty() && !errVec.empty()) - reportError(expr.location, TypeMismatch{lhsType, rhsType}); - } - else - reportErrors(state.errors); - } - return booleanType; } @@ -2443,7 +2470,7 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprBi TypeId result = checkBinaryOperation(innerScope, expr, lhs.type, rhs.type, lhs.predicates); return {result, {OrPredicate{std::move(lhs.predicates), std::move(rhs.predicates)}}}; } - else if (FFlag::LuauEqConstraint && (expr.op == AstExprBinary::CompareEq || expr.op == AstExprBinary::CompareNe)) + else if (expr.op == AstExprBinary::CompareEq || expr.op == AstExprBinary::CompareNe) { if (auto predicate = tryGetTypeGuardPredicate(expr)) return {booleanType, {std::move(*predicate)}}; @@ -2466,14 +2493,6 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprBi } else { - // Once we have EqPredicate, we should break this else branch into its' own branch. - // For now, fall through is intentional. - if (expr.op == AstExprBinary::CompareEq || expr.op == AstExprBinary::CompareNe) - { - if (auto predicate = tryGetTypeGuardPredicate(expr)) - return {booleanType, {std::move(*predicate)}}; - } - ExprResult lhs = checkExpr(scope, *expr.left); ExprResult rhs = checkExpr(scope, *expr.right); @@ -2755,12 +2774,6 @@ std::pair TypeChecker::checkLValueBinding(const ScopePtr& scope exprTable->indexer = TableIndexer{anyIfNonstrict(indexType), anyIfNonstrict(resultType)}; return std::pair(resultType, nullptr); } - else if (FFlag::LuauIndexTablesWithIndexers) - { - // We allow t[x] where x:string for tables without an indexer - unify(indexType, stringType, expr.location); - return std::pair(anyType, nullptr); - } else { TypeId resultType = freshType(scope); @@ -3076,6 +3089,13 @@ static Location getEndLocation(const AstExprFunction& function) void TypeChecker::checkFunctionBody(const ScopePtr& scope, TypeId ty, const AstExprFunction& function) { + LUAU_TIMETRACE_SCOPE("TypeChecker::checkFunctionBody", "TypeChecker"); + + if (function.debugname.value) + LUAU_TIMETRACE_ARGUMENT("name", function.debugname.value); + else + LUAU_TIMETRACE_ARGUMENT("line", std::to_string(function.location.begin.line).c_str()); + if (FunctionTypeVar* funTy = getMutable(ty)) { check(scope, *function.body); @@ -3885,6 +3905,20 @@ std::optional TypeChecker::matchRequire(const AstExprCall& call) TypeId TypeChecker::checkRequire(const ScopePtr& scope, const ModuleInfo& moduleInfo, const Location& location) { + LUAU_TIMETRACE_SCOPE("TypeChecker::checkRequire", "TypeChecker"); + LUAU_TIMETRACE_ARGUMENT("moduleInfo", moduleInfo.name.c_str()); + + if (FFlag::LuauNewRequireTrace && moduleInfo.name.empty()) + { + if (FFlag::LuauStrictRequire && currentModule->mode == Mode::Strict) + { + reportError(TypeError{location, UnknownRequire{}}); + return errorType; + } + + return anyType; + } + ModulePtr module = resolver->getModule(moduleInfo.name); if (!module) { @@ -4472,7 +4506,7 @@ TypeId TypeChecker::freshType(const ScopePtr& scope) TypeId TypeChecker::freshType(TypeLevel level) { - return currentModule->internalTypes.typeVars.allocate(TypeVar(FreeTypeVar(level))); + return currentModule->internalTypes.addType(TypeVar(FreeTypeVar(level))); } TypeId TypeChecker::DEPRECATED_freshType(const ScopePtr& scope, bool canBeGeneric) @@ -4482,11 +4516,7 @@ TypeId TypeChecker::DEPRECATED_freshType(const ScopePtr& scope, bool canBeGeneri TypeId TypeChecker::DEPRECATED_freshType(TypeLevel level, bool canBeGeneric) { - TypeId allocated = currentModule->internalTypes.typeVars.allocate(TypeVar(FreeTypeVar(level, canBeGeneric))); - if (FFlag::DebugLuauTrackOwningArena) - asMutable(allocated)->owningArena = ¤tModule->internalTypes; - - return allocated; + return currentModule->internalTypes.addType(TypeVar(FreeTypeVar(level, canBeGeneric))); } std::optional TypeChecker::filterMap(TypeId type, TypeIdPredicate predicate) @@ -4506,20 +4536,12 @@ TypeId TypeChecker::addType(const UnionTypeVar& utv) TypeId TypeChecker::addTV(TypeVar&& tv) { - TypeId allocated = currentModule->internalTypes.typeVars.allocate(std::move(tv)); - if (FFlag::DebugLuauTrackOwningArena) - asMutable(allocated)->owningArena = ¤tModule->internalTypes; - - return allocated; + return currentModule->internalTypes.addType(std::move(tv)); } TypePackId TypeChecker::addTypePack(TypePackVar&& tv) { - TypePackId allocated = currentModule->internalTypes.typePacks.allocate(std::move(tv)); - if (FFlag::DebugLuauTrackOwningArena) - asMutable(allocated)->owningArena = ¤tModule->internalTypes; - - return allocated; + return currentModule->internalTypes.addTypePack(std::move(tv)); } TypePackId TypeChecker::addTypePack(TypePack&& tp) @@ -4578,7 +4600,7 @@ TypeId TypeChecker::resolveType(const ScopePtr& scope, const AstType& annotation else if (FFlag::DebugLuauMagicTypes && lit->name == "_luau_print") { - if (lit->generics.size != 1) + if (lit->parameters.size != 1 || !lit->parameters.data[0].type) { reportError(TypeError{annotation.location, GenericError{"_luau_print requires one generic parameter"}}); return addType(ErrorTypeVar{}); @@ -4588,7 +4610,7 @@ TypeId TypeChecker::resolveType(const ScopePtr& scope, const AstType& annotation opts.exhaustive = true; opts.maxTableLength = 0; - TypeId param = resolveType(scope, *lit->generics.data[0]); + TypeId param = resolveType(scope, *lit->parameters.data[0].type); luauPrintLine(format("_luau_print\t%s\t|\t%s", toString(param, opts).c_str(), toString(lit->location).c_str())); return param; } @@ -4614,18 +4636,86 @@ TypeId TypeChecker::resolveType(const ScopePtr& scope, const AstType& annotation return addType(ErrorTypeVar{}); } - if (lit->generics.size == 0 && tf->typeParams.empty()) - return tf->type; - else if (lit->generics.size != tf->typeParams.size()) + if (lit->parameters.size == 0 && tf->typeParams.empty() && (!FFlag::LuauTypeAliasPacks || tf->typePackParams.empty())) { - reportError(TypeError{annotation.location, IncorrectGenericParameterCount{lit->name.value, *tf, lit->generics.size}}); + return tf->type; + } + else if (!FFlag::LuauTypeAliasPacks && lit->parameters.size != tf->typeParams.size()) + { + reportError(TypeError{annotation.location, IncorrectGenericParameterCount{lit->name.value, *tf, lit->parameters.size, 0}}); return addType(ErrorTypeVar{}); } + else if (FFlag::LuauTypeAliasPacks) + { + if (!lit->hasParameterList && !tf->typePackParams.empty()) + { + reportError(TypeError{annotation.location, GenericError{"Type parameter list is required"}}); + return addType(ErrorTypeVar{}); + } + + std::vector typeParams; + std::vector extraTypes; + std::vector typePackParams; + + for (size_t i = 0; i < lit->parameters.size; ++i) + { + if (AstType* type = lit->parameters.data[i].type) + { + TypeId ty = resolveType(scope, *type); + + if (typeParams.size() < tf->typeParams.size() || tf->typePackParams.empty()) + typeParams.push_back(ty); + else if (typePackParams.empty()) + extraTypes.push_back(ty); + else + reportError(TypeError{annotation.location, GenericError{"Type parameters must come before type pack parameters"}}); + } + else if (AstTypePack* typePack = lit->parameters.data[i].typePack) + { + TypePackId tp = resolveTypePack(scope, *typePack); + + // If we have collected an implicit type pack, materialize it + if (typePackParams.empty() && !extraTypes.empty()) + typePackParams.push_back(addTypePack(extraTypes)); + + // If we need more regular types, we can use single element type packs to fill those in + if (typeParams.size() < tf->typeParams.size() && size(tp) == 1 && finite(tp) && first(tp)) + typeParams.push_back(*first(tp)); + else + typePackParams.push_back(tp); + } + } + + // If we still haven't meterialized an implicit type pack, do it now + if (typePackParams.empty() && !extraTypes.empty()) + typePackParams.push_back(addTypePack(extraTypes)); + + // If we didn't combine regular types into a type pack and we're still one type pack short, provide an empty type pack + if (extraTypes.empty() && typePackParams.size() + 1 == tf->typePackParams.size()) + typePackParams.push_back(addTypePack({})); + + if (typeParams.size() != tf->typeParams.size() || typePackParams.size() != tf->typePackParams.size()) + { + reportError( + TypeError{annotation.location, IncorrectGenericParameterCount{lit->name.value, *tf, typeParams.size(), typePackParams.size()}}); + return addType(ErrorTypeVar{}); + } + + if (FFlag::LuauRecursiveTypeParameterRestriction && typeParams == tf->typeParams && typePackParams == tf->typePackParams) + { + // If the generic parameters and the type arguments are the same, we are about to + // perform an identity substitution, which we can just short-circuit. + return tf->type; + } + + return instantiateTypeFun(scope, *tf, typeParams, typePackParams, annotation.location); + } else { std::vector typeParams; - for (AstType* paramAnnot : lit->generics) - typeParams.push_back(resolveType(scope, *paramAnnot)); + + for (const auto& param : lit->parameters) + typeParams.push_back(resolveType(scope, *param.type)); if (FFlag::LuauRecursiveTypeParameterRestriction && typeParams == tf->typeParams) { @@ -4634,7 +4724,7 @@ TypeId TypeChecker::resolveType(const ScopePtr& scope, const AstType& annotation return tf->type; } - return instantiateTypeFun(scope, *tf, typeParams, annotation.location); + return instantiateTypeFun(scope, *tf, typeParams, {}, annotation.location); } } else if (const auto& table = annotation.as()) @@ -4765,6 +4855,18 @@ TypePackId TypeChecker::resolveTypePack(const ScopePtr& scope, const AstTypePack return *genericTy; } + else if (const AstTypePackExplicit* explicitTp = annotation.as()) + { + std::vector types; + + for (auto type : explicitTp->typeList.types) + types.push_back(resolveType(scope, *type)); + + if (auto tailType = explicitTp->typeList.tailType) + return addTypePack(types, resolveTypePack(scope, *tailType)); + + return addTypePack(types); + } else { ice("Unknown AstTypePack kind"); @@ -4799,12 +4901,28 @@ bool ApplyTypeFunction::isDirty(TypePackId tp) return false; } +bool ApplyTypeFunction::ignoreChildren(TypeId ty) +{ + if (FFlag::LuauSubstitutionDontReplaceIgnoredTypes && get(ty)) + return true; + else + return false; +} + +bool ApplyTypeFunction::ignoreChildren(TypePackId tp) +{ + if (FFlag::LuauSubstitutionDontReplaceIgnoredTypes && get(tp)) + return true; + else + return false; +} + TypeId ApplyTypeFunction::clean(TypeId ty) { // Really this should just replace the arguments, // but for bug-compatibility with existing code, we replace // all generics by free type variables. - TypeId& arg = arguments[ty]; + TypeId& arg = typeArguments[ty]; if (arg) return arg; else @@ -4816,17 +4934,37 @@ TypePackId ApplyTypeFunction::clean(TypePackId tp) // Really this should just replace the arguments, // but for bug-compatibility with existing code, we replace // all generics by free type variables. - return addTypePack(FreeTypePack{level}); + if (FFlag::LuauTypeAliasPacks) + { + TypePackId& arg = typePackArguments[tp]; + if (arg) + return arg; + else + return addTypePack(FreeTypePack{level}); + } + else + { + return addTypePack(FreeTypePack{level}); + } } -TypeId TypeChecker::instantiateTypeFun(const ScopePtr& scope, const TypeFun& tf, const std::vector& typeParams, const Location& location) +TypeId TypeChecker::instantiateTypeFun(const ScopePtr& scope, const TypeFun& tf, const std::vector& typeParams, + const std::vector& typePackParams, const Location& location) { - if (tf.typeParams.empty()) + if (tf.typeParams.empty() && (!FFlag::LuauTypeAliasPacks || tf.typePackParams.empty())) return tf.type; - applyTypeFunction.arguments.clear(); + applyTypeFunction.typeArguments.clear(); for (size_t i = 0; i < tf.typeParams.size(); ++i) - applyTypeFunction.arguments[tf.typeParams[i]] = typeParams[i]; + applyTypeFunction.typeArguments[tf.typeParams[i]] = typeParams[i]; + + if (FFlag::LuauTypeAliasPacks) + { + applyTypeFunction.typePackArguments.clear(); + for (size_t i = 0; i < tf.typePackParams.size(); ++i) + applyTypeFunction.typePackArguments[tf.typePackParams[i]] = typePackParams[i]; + } + applyTypeFunction.currentModule = currentModule; applyTypeFunction.level = scope->level; applyTypeFunction.encounteredForwardedType = false; @@ -4875,6 +5013,9 @@ TypeId TypeChecker::instantiateTypeFun(const ScopePtr& scope, const TypeFun& tf, if (ttv) { ttv->instantiatedTypeParams = typeParams; + + if (FFlag::LuauTypeAliasPacks) + ttv->instantiatedTypePackParams = typePackParams; } } else @@ -4890,6 +5031,9 @@ TypeId TypeChecker::instantiateTypeFun(const ScopePtr& scope, const TypeFun& tf, } ttv->instantiatedTypeParams = typeParams; + + if (FFlag::LuauTypeAliasPacks) + ttv->instantiatedTypePackParams = typePackParams; } } @@ -4899,6 +5043,8 @@ TypeId TypeChecker::instantiateTypeFun(const ScopePtr& scope, const TypeFun& tf, std::pair, std::vector> TypeChecker::createGenericTypes( const ScopePtr& scope, const AstNode& node, const AstArray& genericNames, const AstArray& genericPackNames) { + LUAU_ASSERT(scope->parent); + std::vector generics; for (const AstName& generic : genericNames) { @@ -4912,7 +5058,19 @@ std::pair, std::vector> TypeChecker::createGener reportError(TypeError{node.location, DuplicateGenericParameter{n}}); } - TypeId g = addType(Unifiable::Generic{scope->level, n}); + TypeId g; + if (FFlag::LuauRecursiveTypeParameterRestriction && FFlag::LuauTypeAliasPacks) + { + TypeId& cached = scope->parent->typeAliasTypeParameters[n]; + if (!cached) + cached = addType(GenericTypeVar{scope->level, n}); + g = cached; + } + else + { + g = addType(Unifiable::Generic{scope->level, n}); + } + generics.push_back(g); scope->privateTypeBindings[n] = TypeFun{{}, g}; } @@ -4930,7 +5088,19 @@ std::pair, std::vector> TypeChecker::createGener reportError(TypeError{node.location, DuplicateGenericParameter{n}}); } - TypePackId g = addTypePack(TypePackVar{Unifiable::Generic{scope->level, n}}); + TypePackId g; + if (FFlag::LuauRecursiveTypeParameterRestriction && FFlag::LuauTypeAliasPacks) + { + TypePackId& cached = scope->parent->typeAliasTypePackParameters[n]; + if (!cached) + cached = addTypePack(TypePackVar{Unifiable::Generic{scope->level, n}}); + g = cached; + } + else + { + g = addTypePack(TypePackVar{Unifiable::Generic{scope->level, n}}); + } + genericPacks.push_back(g); scope->privateTypePackBindings[n] = g; } @@ -5013,13 +5183,8 @@ void TypeChecker::resolve(const Predicate& predicate, ErrorVec& errVec, Refineme else if (auto isaP = get(predicate)) resolve(*isaP, errVec, refis, scope, sense); else if (auto typeguardP = get(predicate)) - { - if (FFlag::LuauImprovedTypeGuardPredicate2) - resolve(*typeguardP, errVec, refis, scope, sense); - else - DEPRECATED_resolve(*typeguardP, errVec, refis, scope, sense); - } - else if (auto eqP = get(predicate); eqP && FFlag::LuauEqConstraint) + resolve(*typeguardP, errVec, refis, scope, sense); + else if (auto eqP = get(predicate)) resolve(*eqP, errVec, refis, scope, sense); else ice("Unhandled predicate kind"); @@ -5145,7 +5310,7 @@ void TypeChecker::resolve(const IsAPredicate& isaP, ErrorVec& errVec, Refinement return isaP.ty; } } - else if (FFlag::LuauImprovedTypeGuardPredicate2) + else { auto lctv = get(option); auto rctv = get(isaP.ty); @@ -5159,19 +5324,6 @@ void TypeChecker::resolve(const IsAPredicate& isaP, ErrorVec& errVec, Refinement if (canUnify(option, isaP.ty, isaP.location).empty() == sense) return isaP.ty; } - else - { - auto lctv = get(option); - auto rctv = get(isaP.ty); - - if (lctv && rctv) - { - if (isSubclass(lctv, rctv) == sense) - return option; - else if (isSubclass(rctv, lctv) == sense) - return isaP.ty; - } - } return std::nullopt; }; @@ -5266,7 +5418,7 @@ void TypeChecker::resolve(const TypeGuardPredicate& typeguardP, ErrorVec& errVec return fail(UnknownSymbol{typeguardP.kind, UnknownSymbol::Type}); auto typeFun = globalScope->lookupType(typeguardP.kind); - if (!typeFun || !typeFun->typeParams.empty()) + if (!typeFun || !typeFun->typeParams.empty() || (FFlag::LuauTypeAliasPacks && !typeFun->typePackParams.empty())) return fail(UnknownSymbol{typeguardP.kind, UnknownSymbol::Type}); TypeId type = follow(typeFun->type); @@ -5292,7 +5444,8 @@ void TypeChecker::DEPRECATED_resolve(const TypeGuardPredicate& typeguardP, Error "userdata", // no op. Requires special handling. }; - if (auto typeFun = globalScope->lookupType(typeguardP.kind); typeFun && typeFun->typeParams.empty()) + if (auto typeFun = globalScope->lookupType(typeguardP.kind); + typeFun && typeFun->typeParams.empty() && (!FFlag::LuauTypeAliasPacks || typeFun->typePackParams.empty())) { if (auto it = std::find(primitives.begin(), primitives.end(), typeguardP.kind); it != primitives.end()) addRefinement(refis, typeguardP.lvalue, typeFun->type); @@ -5319,38 +5472,41 @@ void TypeChecker::resolve(const EqPredicate& eqP, ErrorVec& errVec, RefinementMa return; } - std::optional ty = resolveLValue(refis, scope, eqP.lvalue); - if (!ty) - return; - - std::vector lhs = options(*ty); - std::vector rhs = options(eqP.type); - - if (sense && std::any_of(lhs.begin(), lhs.end(), isUndecidable)) + if (FFlag::LuauEqConstraint) { - addRefinement(refis, eqP.lvalue, eqP.type); - return; - } - else if (sense && std::any_of(rhs.begin(), rhs.end(), isUndecidable)) - return; // Optimization: the other side has unknown types, so there's probably an overlap. Refining is no-op here. + std::optional ty = resolveLValue(refis, scope, eqP.lvalue); + if (!ty) + return; - std::unordered_set set; - for (TypeId left : lhs) - { - for (TypeId right : rhs) + std::vector lhs = options(*ty); + std::vector rhs = options(eqP.type); + + if (sense && std::any_of(lhs.begin(), lhs.end(), isUndecidable)) { - // When singleton types arrive, `isNil` here probably should be replaced with `isLiteral`. - if (canUnify(left, right, eqP.location).empty() == sense || (!sense && !isNil(left))) - set.insert(left); + addRefinement(refis, eqP.lvalue, eqP.type); + return; } + else if (sense && std::any_of(rhs.begin(), rhs.end(), isUndecidable)) + return; // Optimization: the other side has unknown types, so there's probably an overlap. Refining is no-op here. + + std::unordered_set set; + for (TypeId left : lhs) + { + for (TypeId right : rhs) + { + // When singleton types arrive, `isNil` here probably should be replaced with `isLiteral`. + if (canUnify(left, right, eqP.location).empty() == sense || (!sense && !isNil(left))) + set.insert(left); + } + } + + if (set.empty()) + return; + + std::vector viable(set.begin(), set.end()); + TypeId result = viable.size() == 1 ? viable[0] : addType(UnionTypeVar{std::move(viable)}); + addRefinement(refis, eqP.lvalue, result); } - - if (set.empty()) - return; - - std::vector viable(set.begin(), set.end()); - TypeId result = viable.size() == 1 ? viable[0] : addType(UnionTypeVar{std::move(viable)}); - addRefinement(refis, eqP.lvalue, result); } bool TypeChecker::isNonstrictMode() const @@ -5379,119 +5535,4 @@ std::vector> TypeChecker::getScopes() const return currentModule->scopes; } -Scope::Scope(TypePackId returnType) - : parent(nullptr) - , returnType(returnType) - , level(TypeLevel()) -{ -} - -Scope::Scope(const ScopePtr& parent, int subLevel) - : parent(parent) - , returnType(parent->returnType) - , level(parent->level.incr()) -{ - level.subLevel = subLevel; -} - -std::optional Scope::lookup(const Symbol& name) -{ - Scope* scope = this; - - while (scope) - { - auto it = scope->bindings.find(name); - if (it != scope->bindings.end()) - return it->second.typeId; - - scope = scope->parent.get(); - } - - return std::nullopt; -} - -std::optional Scope::lookupType(const Name& name) -{ - const Scope* scope = this; - while (true) - { - auto it = scope->exportedTypeBindings.find(name); - if (it != scope->exportedTypeBindings.end()) - return it->second; - - it = scope->privateTypeBindings.find(name); - if (it != scope->privateTypeBindings.end()) - return it->second; - - if (scope->parent) - scope = scope->parent.get(); - else - return std::nullopt; - } -} - -std::optional Scope::lookupImportedType(const Name& moduleAlias, const Name& name) -{ - const Scope* scope = this; - while (scope) - { - auto it = scope->importedTypeBindings.find(moduleAlias); - if (it == scope->importedTypeBindings.end()) - { - scope = scope->parent.get(); - continue; - } - - auto it2 = it->second.find(name); - if (it2 == it->second.end()) - { - scope = scope->parent.get(); - continue; - } - - return it2->second; - } - - return std::nullopt; -} - -std::optional Scope::lookupPack(const Name& name) -{ - const Scope* scope = this; - while (true) - { - auto it = scope->privateTypePackBindings.find(name); - if (it != scope->privateTypePackBindings.end()) - return it->second; - - if (scope->parent) - scope = scope->parent.get(); - else - return std::nullopt; - } -} - -std::optional Scope::linearSearchForBinding(const std::string& name, bool traverseScopeChain) -{ - Scope* scope = this; - - while (scope) - { - for (const auto& [n, binding] : scope->bindings) - { - if (n.local && n.local->name == name.c_str()) - return binding; - else if (n.global.value && n.global == name.c_str()) - return binding; - } - - scope = scope->parent.get(); - - if (!traverseScopeChain) - break; - } - - return std::nullopt; -} - } // namespace Luau diff --git a/Analysis/src/TypePack.cpp b/Analysis/src/TypePack.cpp index 5970f304..68a16ef0 100644 --- a/Analysis/src/TypePack.cpp +++ b/Analysis/src/TypePack.cpp @@ -209,6 +209,19 @@ size_t size(TypePackId tp) return 0; } +bool finite(TypePackId tp) +{ + tp = follow(tp); + + if (auto pack = get(tp)) + return pack->tail ? finite(*pack->tail) : true; + + if (auto pack = get(tp)) + return false; + + return true; +} + size_t size(const TypePack& tp) { size_t result = tp.head.size(); diff --git a/Analysis/src/TypeUtils.cpp b/Analysis/src/TypeUtils.cpp index b9f50978..0d9d91e0 100644 --- a/Analysis/src/TypeUtils.cpp +++ b/Analysis/src/TypeUtils.cpp @@ -1,11 +1,10 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #include "Luau/TypeUtils.h" +#include "Luau/Scope.h" #include "Luau/ToString.h" #include "Luau/TypeInfer.h" -LUAU_FASTFLAG(LuauStringMetatable) - namespace Luau { @@ -13,21 +12,6 @@ std::optional findMetatableEntry(ErrorVec& errors, const ScopePtr& globa { type = follow(type); - if (!FFlag::LuauStringMetatable) - { - if (const PrimitiveTypeVar* primType = get(type)) - { - if (primType->type != PrimitiveTypeVar::String || "__index" != entry) - return std::nullopt; - - auto it = globalScope->bindings.find(AstName{"string"}); - if (it != globalScope->bindings.end()) - return it->second.typeId; - else - return std::nullopt; - } - } - std::optional metatable = getMetatable(type); if (!metatable) return std::nullopt; diff --git a/Analysis/src/TypeVar.cpp b/Analysis/src/TypeVar.cpp index 111f4f53..e963fc74 100644 --- a/Analysis/src/TypeVar.cpp +++ b/Analysis/src/TypeVar.cpp @@ -19,11 +19,9 @@ LUAU_FASTINTVARIABLE(LuauTypeMaximumStringifierLength, 500) LUAU_FASTINTVARIABLE(LuauTableTypeMaximumStringifierLength, 0) -LUAU_FASTFLAG(LuauImprovedTypeGuardPredicate2) -LUAU_FASTFLAGVARIABLE(LuauToStringFollowsBoundTo, false) LUAU_FASTFLAG(LuauRankNTypes) -LUAU_FASTFLAGVARIABLE(LuauStringMetatable, false) LUAU_FASTFLAG(LuauTypeGuardPeelsAwaySubclasses) +LUAU_FASTFLAG(LuauTypeAliasPacks) namespace Luau { @@ -193,27 +191,11 @@ bool isOptional(TypeId ty) bool isTableIntersection(TypeId ty) { - if (FFlag::LuauImprovedTypeGuardPredicate2) - { - if (!get(follow(ty))) - return false; - - std::vector parts = flattenIntersection(ty); - return std::all_of(parts.begin(), parts.end(), getTableType); - } - else - { - if (const IntersectionTypeVar* itv = get(ty)) - { - for (TypeId part : itv->parts) - { - if (getTableType(follow(part))) - return true; - } - } - + if (!get(follow(ty))) return false; - } + + std::vector parts = flattenIntersection(ty); + return std::all_of(parts.begin(), parts.end(), getTableType); } bool isOverloadedFunction(TypeId ty) @@ -236,7 +218,7 @@ std::optional getMetatable(TypeId type) else if (const ClassTypeVar* classType = get(type)) return classType->metatable; else if (const PrimitiveTypeVar* primitiveType = get(type); - FFlag::LuauStringMetatable && primitiveType && primitiveType->metatable) + primitiveType && primitiveType->metatable) { LUAU_ASSERT(primitiveType->type == PrimitiveTypeVar::String); return primitiveType->metatable; @@ -871,6 +853,12 @@ void StateDot::visitChildren(TypeId ty, int index) } for (TypeId itp : ttv->instantiatedTypeParams) visitChild(itp, index, "typeParam"); + + if (FFlag::LuauTypeAliasPacks) + { + for (TypePackId itp : ttv->instantiatedTypePackParams) + visitChild(itp, index, "typePackParam"); + } } else if (const MetatableTypeVar* mtv = get(ty)) { diff --git a/Analysis/src/Unifier.cpp b/Analysis/src/Unifier.cpp index 89c3f80c..117cbc28 100644 --- a/Analysis/src/Unifier.cpp +++ b/Analysis/src/Unifier.cpp @@ -3,23 +3,25 @@ #include "Luau/Common.h" #include "Luau/RecursionCounter.h" +#include "Luau/Scope.h" #include "Luau/TypePack.h" #include "Luau/TypeUtils.h" +#include "Luau/TimeTrace.h" #include LUAU_FASTINT(LuauTypeInferRecursionLimit); LUAU_FASTINT(LuauTypeInferTypePackLoopLimit); -LUAU_FASTINTVARIABLE(LuauTypeInferIterationLimit, 0); -LUAU_FASTFLAGVARIABLE(LuauLogTableTypeVarBoundTo, false) +LUAU_FASTINTVARIABLE(LuauTypeInferIterationLimit, 2000); LUAU_FASTFLAG(LuauGenericFunctions) +LUAU_FASTFLAGVARIABLE(LuauTableSubtypingVariance, false); LUAU_FASTFLAGVARIABLE(LuauDontMutatePersistentFunctions, false) LUAU_FASTFLAG(LuauRankNTypes) -LUAU_FASTFLAG(LuauStringMetatable) LUAU_FASTFLAGVARIABLE(LuauUnionHeuristic, false) LUAU_FASTFLAGVARIABLE(LuauTableUnificationEarlyTest, false) LUAU_FASTFLAGVARIABLE(LuauSealedTableUnifyOptionalFix, false) LUAU_FASTFLAGVARIABLE(LuauOccursCheckOkWithRecursiveFunctions, false) +LUAU_FASTFLAGVARIABLE(LuauTypecheckOpts, false) namespace Luau { @@ -43,21 +45,23 @@ Unifier::Unifier(TypeArena* types, Mode mode, ScopePtr globalScope, const Locati , globalScope(std::move(globalScope)) , location(location) , variance(variance) - , counters(std::make_shared()) + , counters(&countersData) + , counters_DEPRECATED(std::make_shared()) , iceHandler(iceHandler) { LUAU_ASSERT(iceHandler); } Unifier::Unifier(TypeArena* types, Mode mode, ScopePtr globalScope, const std::vector>& seen, const Location& location, - Variance variance, InternalErrorReporter* iceHandler, const std::shared_ptr& counters) + Variance variance, InternalErrorReporter* iceHandler, const std::shared_ptr& counters_DEPRECATED, UnifierCounters* counters) : types(types) , mode(mode) , globalScope(std::move(globalScope)) , log(seen) , location(location) , variance(variance) - , counters(counters ? counters : std::make_shared()) + , counters(counters ? counters : &countersData) + , counters_DEPRECATED(counters_DEPRECATED ? counters_DEPRECATED : std::make_shared()) , iceHandler(iceHandler) { LUAU_ASSERT(iceHandler); @@ -65,16 +69,26 @@ Unifier::Unifier(TypeArena* types, Mode mode, ScopePtr globalScope, const std::v void Unifier::tryUnify(TypeId superTy, TypeId subTy, bool isFunctionCall, bool isIntersection) { - counters->iterationCount = 0; + if (FFlag::LuauTypecheckOpts) + counters->iterationCount = 0; + else + counters_DEPRECATED->iterationCount = 0; + return tryUnify_(superTy, subTy, isFunctionCall, isIntersection); } void Unifier::tryUnify_(TypeId superTy, TypeId subTy, bool isFunctionCall, bool isIntersection) { - RecursionLimiter _ra(&counters->recursionCount, FInt::LuauTypeInferRecursionLimit); + RecursionLimiter _ra( + FFlag::LuauTypecheckOpts ? &counters->recursionCount : &counters_DEPRECATED->recursionCount, FInt::LuauTypeInferRecursionLimit); - ++counters->iterationCount; - if (FInt::LuauTypeInferIterationLimit > 0 && FInt::LuauTypeInferIterationLimit < counters->iterationCount) + if (FFlag::LuauTypecheckOpts) + ++counters->iterationCount; + else + ++counters_DEPRECATED->iterationCount; + + if (FInt::LuauTypeInferIterationLimit > 0 && + FInt::LuauTypeInferIterationLimit < (FFlag::LuauTypecheckOpts ? counters->iterationCount : counters_DEPRECATED->iterationCount)) { errors.push_back(TypeError{location, UnificationTooComplex{}}); return; @@ -440,7 +454,11 @@ ErrorVec Unifier::canUnify(TypePackId superTy, TypePackId subTy, bool isFunction void Unifier::tryUnify(TypePackId superTp, TypePackId subTp, bool isFunctionCall) { - counters->iterationCount = 0; + if (FFlag::LuauTypecheckOpts) + counters->iterationCount = 0; + else + counters_DEPRECATED->iterationCount = 0; + return tryUnify_(superTp, subTp, isFunctionCall); } @@ -450,10 +468,16 @@ void Unifier::tryUnify(TypePackId superTp, TypePackId subTp, bool isFunctionCall */ void Unifier::tryUnify_(TypePackId superTp, TypePackId subTp, bool isFunctionCall) { - RecursionLimiter _ra(&counters->recursionCount, FInt::LuauTypeInferRecursionLimit); + RecursionLimiter _ra( + FFlag::LuauTypecheckOpts ? &counters->recursionCount : &counters_DEPRECATED->recursionCount, FInt::LuauTypeInferRecursionLimit); - ++counters->iterationCount; - if (FInt::LuauTypeInferIterationLimit > 0 && FInt::LuauTypeInferIterationLimit < counters->iterationCount) + if (FFlag::LuauTypecheckOpts) + ++counters->iterationCount; + else + ++counters_DEPRECATED->iterationCount; + + if (FInt::LuauTypeInferIterationLimit > 0 && + FInt::LuauTypeInferIterationLimit < (FFlag::LuauTypecheckOpts ? counters->iterationCount : counters_DEPRECATED->iterationCount)) { errors.push_back(TypeError{location, UnificationTooComplex{}}); return; @@ -762,9 +786,210 @@ struct Resetter void Unifier::tryUnifyTables(TypeId left, TypeId right, bool isIntersection) { - std::unique_ptr resetter; + if (!FFlag::LuauTableSubtypingVariance) + return DEPRECATED_tryUnifyTables(left, right, isIntersection); - resetter.reset(new Resetter{&variance}); + TableTypeVar* lt = getMutable(left); + TableTypeVar* rt = getMutable(right); + if (!lt || !rt) + ice("passed non-table types to unifyTables"); + + std::vector missingProperties; + std::vector extraProperties; + + // Reminder: left is the supertype, right is the subtype. + // Width subtyping: any property in the supertype must be in the subtype, + // and the types must agree. + for (const auto& [name, prop] : lt->props) + { + const auto& r = rt->props.find(name); + if (r != rt->props.end()) + { + // TODO: read-only properties don't need invariance + Resetter resetter{&variance}; + variance = Invariant; + + Unifier innerState = makeChildUnifier(); + innerState.tryUnify_(prop.type, r->second.type); + checkChildUnifierTypeMismatch(innerState.errors, left, right); + if (innerState.errors.empty()) + log.concat(std::move(innerState.log)); + else + innerState.log.rollback(); + } + else if (rt->indexer && isString(rt->indexer->indexType)) + { + // TODO: read-only indexers don't need invariance + // TODO: really we should only allow this if prop.type is optional. + Resetter resetter{&variance}; + variance = Invariant; + + Unifier innerState = makeChildUnifier(); + innerState.tryUnify_(prop.type, rt->indexer->indexResultType); + checkChildUnifierTypeMismatch(innerState.errors, left, right); + if (innerState.errors.empty()) + log.concat(std::move(innerState.log)); + else + innerState.log.rollback(); + } + else if (isOptional(prop.type) || get(follow(prop.type))) + // TODO: this case is unsound, but without it our test suite fails. CLI-46031 + // TODO: should isOptional(anyType) be true? + {} + else if (rt->state == TableState::Free) + { + log(rt); + rt->props[name] = prop; + } + else + missingProperties.push_back(name); + } + + for (const auto& [name, prop] : rt->props) + { + if (lt->props.count(name)) + { + // If both lt and rt contain the property, then + // we're done since we already unified them above + } + else if (lt->indexer && isString(lt->indexer->indexType)) + { + // TODO: read-only indexers don't need invariance + // TODO: really we should only allow this if prop.type is optional. + Resetter resetter{&variance}; + variance = Invariant; + + Unifier innerState = makeChildUnifier(); + innerState.tryUnify_(prop.type, lt->indexer->indexResultType); + checkChildUnifierTypeMismatch(innerState.errors, left, right); + if (innerState.errors.empty()) + log.concat(std::move(innerState.log)); + else + innerState.log.rollback(); + } + else if (lt->state == TableState::Unsealed) + { + // TODO: this case is unsound when variance is Invariant, but without it lua-apps fails to typecheck. + // TODO: file a JIRA + // TODO: hopefully readonly/writeonly properties will fix this. + Property clone = prop; + clone.type = deeplyOptional(clone.type); + log(lt); + lt->props[name] = clone; + } + else if (variance == Covariant) + {} + else if (isOptional(prop.type) || get(follow(prop.type))) + // TODO: this case is unsound, but without it our test suite fails. CLI-46031 + // TODO: should isOptional(anyType) be true? + {} + else if (lt->state == TableState::Free) + { + log(lt); + lt->props[name] = prop; + } + else + extraProperties.push_back(name); + } + + // Unify indexers + if (lt->indexer && rt->indexer) + { + // TODO: read-only indexers don't need invariance + Resetter resetter{&variance}; + variance = Invariant; + + Unifier innerState = makeChildUnifier(); + innerState.tryUnify(*lt->indexer, *rt->indexer); + checkChildUnifierTypeMismatch(innerState.errors, left, right); + if (innerState.errors.empty()) + log.concat(std::move(innerState.log)); + else + innerState.log.rollback(); + } + else if (lt->indexer) + { + if (rt->state == TableState::Unsealed || rt->state == TableState::Free) + { + // passing/assigning a table without an indexer to something that has one + // e.g. table.insert(t, 1) where t is a non-sealed table and doesn't have an indexer. + // TODO: we only need to do this if the supertype's indexer is read/write + // since that can add indexed elements. + log(rt); + rt->indexer = lt->indexer; + } + } + else if (rt->indexer && variance == Invariant) + { + // Symmetric if we are invariant + if (lt->state == TableState::Unsealed || lt->state == TableState::Free) + { + log(lt); + lt->indexer = rt->indexer; + } + } + + if (!missingProperties.empty()) + { + errors.push_back(TypeError{location, MissingProperties{left, right, std::move(missingProperties)}}); + return; + } + + if (!extraProperties.empty()) + { + errors.push_back(TypeError{location, MissingProperties{left, right, std::move(extraProperties), MissingProperties::Extra}}); + return; + } + + /* + * TypeVars are commonly cyclic, so it is entirely possible + * for unifying a property of a table to change the table itself! + * We need to check for this and start over if we notice this occurring. + * + * I believe this is guaranteed to terminate eventually because this will + * only happen when a free table is bound to another table. + */ + if (lt->boundTo || rt->boundTo) + return tryUnify_(left, right); + + if (lt->state == TableState::Free) + { + log(lt); + lt->boundTo = right; + } + else if (rt->state == TableState::Free) + { + log(rt); + rt->boundTo = left; + } +} + +TypeId Unifier::deeplyOptional(TypeId ty, std::unordered_map seen) +{ + ty = follow(ty); + if (get(ty)) + return ty; + else if (isOptional(ty)) + return ty; + else if (const TableTypeVar* ttv = get(ty)) + { + TypeId& result = seen[ty]; + if (result) + return result; + result = types->addType(*ttv); + TableTypeVar* resultTtv = getMutable(result); + for (auto& [name, prop] : resultTtv->props) + prop.type = deeplyOptional(prop.type, seen); + return types->addType(UnionTypeVar{{ singletonTypes.nilType, result }});; + } + else + return types->addType(UnionTypeVar{{ singletonTypes.nilType, ty }}); +} + +void Unifier::DEPRECATED_tryUnifyTables(TypeId left, TypeId right, bool isIntersection) +{ + LUAU_ASSERT(!FFlag::LuauTableSubtypingVariance); + Resetter resetter{&variance}; variance = Invariant; TableTypeVar* lt = getMutable(left); @@ -894,10 +1119,7 @@ void Unifier::tryUnifyFreeTable(TypeId freeTypeId, TypeId otherTypeId) if (!freeTable->boundTo && otherTable->state != TableState::Free) { - if (FFlag::LuauLogTableTypeVarBoundTo) - log(freeTable); - else - log(freeTypeId); + log(freeTable); freeTable->boundTo = otherTypeId; } } @@ -1196,9 +1418,11 @@ void Unifier::tryUnify(const TableIndexer& superIndexer, const TableIndexer& sub tryUnify_(superIndexer.indexResultType, subIndexer.indexResultType); } -static void queueTypePack( +static void queueTypePack_DEPRECATED( std::vector& queue, std::unordered_set& seenTypePacks, Unifier& state, TypePackId a, TypePackId anyTypePack) { + LUAU_ASSERT(!FFlag::LuauTypecheckOpts); + while (true) { if (FFlag::LuauAddMissingFollow) @@ -1244,6 +1468,55 @@ static void queueTypePack( } } +static void queueTypePack(std::vector& queue, DenseHashSet& seenTypePacks, Unifier& state, TypePackId a, TypePackId anyTypePack) +{ + LUAU_ASSERT(FFlag::LuauTypecheckOpts); + + while (true) + { + if (FFlag::LuauAddMissingFollow) + a = follow(a); + + if (seenTypePacks.find(a)) + break; + seenTypePacks.insert(a); + + if (FFlag::LuauAddMissingFollow) + { + if (get(a)) + { + state.log(a); + *asMutable(a) = Unifiable::Bound{anyTypePack}; + } + else if (auto tp = get(a)) + { + queue.insert(queue.end(), tp->head.begin(), tp->head.end()); + if (tp->tail) + a = *tp->tail; + else + break; + } + } + else + { + if (get(a)) + { + state.log(a); + *asMutable(a) = Unifiable::Bound{anyTypePack}; + } + + if (auto tp = get(a)) + { + queue.insert(queue.end(), tp->head.begin(), tp->head.end()); + if (tp->tail) + a = *tp->tail; + else + break; + } + } + } +} + void Unifier::tryUnifyVariadics(TypePackId superTp, TypePackId subTp, bool reversed, int subOffset) { const VariadicTypePack* lv = get(superTp); @@ -1297,9 +1570,11 @@ void Unifier::tryUnifyVariadics(TypePackId superTp, TypePackId subTp, bool rever } } -static void tryUnifyWithAny( +static void tryUnifyWithAny_DEPRECATED( std::vector& queue, Unifier& state, std::unordered_set& seenTypePacks, TypeId anyType, TypePackId anyTypePack) { + LUAU_ASSERT(!FFlag::LuauTypecheckOpts); + std::unordered_set seen; while (!queue.empty()) @@ -1310,6 +1585,59 @@ static void tryUnifyWithAny( continue; seen.insert(ty); + if (get(ty)) + { + state.log(ty); + *asMutable(ty) = BoundTypeVar{anyType}; + } + else if (auto fun = get(ty)) + { + queueTypePack_DEPRECATED(queue, seenTypePacks, state, fun->argTypes, anyTypePack); + queueTypePack_DEPRECATED(queue, seenTypePacks, state, fun->retType, anyTypePack); + } + else if (auto table = get(ty)) + { + for (const auto& [_name, prop] : table->props) + queue.push_back(prop.type); + + if (table->indexer) + { + queue.push_back(table->indexer->indexType); + queue.push_back(table->indexer->indexResultType); + } + } + else if (auto mt = get(ty)) + { + queue.push_back(mt->table); + queue.push_back(mt->metatable); + } + else if (get(ty)) + { + // ClassTypeVars never contain free typevars. + } + else if (auto union_ = get(ty)) + queue.insert(queue.end(), union_->options.begin(), union_->options.end()); + else if (auto intersection = get(ty)) + queue.insert(queue.end(), intersection->parts.begin(), intersection->parts.end()); + else + { + } // Primitives, any, errors, and generics are left untouched. + } +} + +static void tryUnifyWithAny(std::vector& queue, Unifier& state, DenseHashSet& seen, DenseHashSet& seenTypePacks, + TypeId anyType, TypePackId anyTypePack) +{ + LUAU_ASSERT(FFlag::LuauTypecheckOpts); + + while (!queue.empty()) + { + TypeId ty = follow(queue.back()); + queue.pop_back(); + if (seen.find(ty)) + continue; + seen.insert(ty); + if (get(ty)) { state.log(ty); @@ -1354,14 +1682,33 @@ void Unifier::tryUnifyWithAny(TypeId any, TypeId ty) { LUAU_ASSERT(get(any) || get(any)); + if (FFlag::LuauTypecheckOpts) + { + // These types are not visited in general loop below + if (get(ty) || get(ty) || get(ty)) + return; + } + const TypePackId anyTypePack = types->addTypePack(TypePackVar{VariadicTypePack{singletonTypes.anyType}}); const TypePackId anyTP = get(any) ? anyTypePack : types->addTypePack(TypePackVar{Unifiable::Error{}}); - std::unordered_set seenTypePacks; - std::vector queue = {ty}; + if (FFlag::LuauTypecheckOpts) + { + std::vector queue = {ty}; - Luau::tryUnifyWithAny(queue, *this, seenTypePacks, singletonTypes.anyType, anyTP); + tempSeenTy.clear(); + tempSeenTp.clear(); + + Luau::tryUnifyWithAny(queue, *this, tempSeenTy, tempSeenTp, singletonTypes.anyType, anyTP); + } + else + { + std::unordered_set seenTypePacks; + std::vector queue = {ty}; + + Luau::tryUnifyWithAny_DEPRECATED(queue, *this, seenTypePacks, singletonTypes.anyType, anyTP); + } } void Unifier::tryUnifyWithAny(TypePackId any, TypePackId ty) @@ -1370,12 +1717,26 @@ void Unifier::tryUnifyWithAny(TypePackId any, TypePackId ty) const TypeId anyTy = singletonTypes.errorType; - std::unordered_set seenTypePacks; - std::vector queue; + if (FFlag::LuauTypecheckOpts) + { + std::vector queue; - queueTypePack(queue, seenTypePacks, *this, ty, any); + tempSeenTy.clear(); + tempSeenTp.clear(); - Luau::tryUnifyWithAny(queue, *this, seenTypePacks, anyTy, any); + queueTypePack(queue, tempSeenTp, *this, ty, any); + + Luau::tryUnifyWithAny(queue, *this, tempSeenTy, tempSeenTp, anyTy, any); + } + else + { + std::unordered_set seenTypePacks; + std::vector queue; + + queueTypePack_DEPRECATED(queue, seenTypePacks, *this, ty, any); + + Luau::tryUnifyWithAny_DEPRECATED(queue, *this, seenTypePacks, anyTy, any); + } } std::optional Unifier::findTablePropertyRespectingMeta(TypeId lhsType, Name name) @@ -1387,21 +1748,6 @@ std::optional Unifier::findMetatableEntry(TypeId type, std::string entry { type = follow(type); - if (!FFlag::LuauStringMetatable) - { - if (const PrimitiveTypeVar* primType = get(type)) - { - if (primType->type != PrimitiveTypeVar::String || "__index" != entry) - return std::nullopt; - - auto found = globalScope->bindings.find(AstName{"string"}); - if (found == globalScope->bindings.end()) - return std::nullopt; - else - return found->second.typeId; - } - } - std::optional metatable = getMetatable(type); if (!metatable) return std::nullopt; @@ -1427,21 +1773,36 @@ std::optional Unifier::findMetatableEntry(TypeId type, std::string entry void Unifier::occursCheck(TypeId needle, TypeId haystack) { - std::unordered_set seen; - return occursCheck(seen, needle, haystack); + std::unordered_set seen_DEPRECATED; + + if (FFlag::LuauTypecheckOpts) + tempSeenTy.clear(); + + return occursCheck(seen_DEPRECATED, tempSeenTy, needle, haystack); } -void Unifier::occursCheck(std::unordered_set& seen, TypeId needle, TypeId haystack) +void Unifier::occursCheck(std::unordered_set& seen_DEPRECATED, DenseHashSet& seen, TypeId needle, TypeId haystack) { - RecursionLimiter _ra(&counters->recursionCount, FInt::LuauTypeInferRecursionLimit); + RecursionLimiter _ra( + FFlag::LuauTypecheckOpts ? &counters->recursionCount : &counters_DEPRECATED->recursionCount, FInt::LuauTypeInferRecursionLimit); needle = follow(needle); haystack = follow(haystack); - if (seen.end() != seen.find(haystack)) - return; + if (FFlag::LuauTypecheckOpts) + { + if (seen.find(haystack)) + return; - seen.insert(haystack); + seen.insert(haystack); + } + else + { + if (seen_DEPRECATED.end() != seen_DEPRECATED.find(haystack)) + return; + + seen_DEPRECATED.insert(haystack); + } if (get(needle)) return; @@ -1458,7 +1819,7 @@ void Unifier::occursCheck(std::unordered_set& seen, TypeId needle, TypeI } auto check = [&](TypeId tv) { - occursCheck(seen, needle, tv); + occursCheck(seen_DEPRECATED, seen, needle, tv); }; if (get(haystack)) @@ -1488,19 +1849,33 @@ void Unifier::occursCheck(std::unordered_set& seen, TypeId needle, TypeI void Unifier::occursCheck(TypePackId needle, TypePackId haystack) { - std::unordered_set seen; - return occursCheck(seen, needle, haystack); + std::unordered_set seen_DEPRECATED; + + if (FFlag::LuauTypecheckOpts) + tempSeenTp.clear(); + + return occursCheck(seen_DEPRECATED, tempSeenTp, needle, haystack); } -void Unifier::occursCheck(std::unordered_set& seen, TypePackId needle, TypePackId haystack) +void Unifier::occursCheck(std::unordered_set& seen_DEPRECATED, DenseHashSet& seen, TypePackId needle, TypePackId haystack) { needle = follow(needle); haystack = follow(haystack); - if (seen.find(haystack) != seen.end()) - return; + if (FFlag::LuauTypecheckOpts) + { + if (seen.find(haystack)) + return; - seen.insert(haystack); + seen.insert(haystack); + } + else + { + if (seen_DEPRECATED.end() != seen_DEPRECATED.find(haystack)) + return; + + seen_DEPRECATED.insert(haystack); + } if (get(needle)) return; @@ -1508,7 +1883,8 @@ void Unifier::occursCheck(std::unordered_set& seen, TypePackId needl if (!get(needle)) ice("Expected needle pack to be free"); - RecursionLimiter _ra(&counters->recursionCount, FInt::LuauTypeInferRecursionLimit); + RecursionLimiter _ra( + FFlag::LuauTypecheckOpts ? &counters->recursionCount : &counters_DEPRECATED->recursionCount, FInt::LuauTypeInferRecursionLimit); while (!get(haystack)) { @@ -1528,8 +1904,8 @@ void Unifier::occursCheck(std::unordered_set& seen, TypePackId needl { if (auto f = get(FFlag::LuauAddMissingFollow ? follow(ty) : ty)) { - occursCheck(seen, needle, f->argTypes); - occursCheck(seen, needle, f->retType); + occursCheck(seen_DEPRECATED, seen, needle, f->argTypes); + occursCheck(seen_DEPRECATED, seen, needle, f->retType); } } } @@ -1546,7 +1922,7 @@ void Unifier::occursCheck(std::unordered_set& seen, TypePackId needl Unifier Unifier::makeChildUnifier() { - return Unifier{types, mode, globalScope, log.seen, location, variance, iceHandler, counters}; + return Unifier{types, mode, globalScope, log.seen, location, variance, iceHandler, counters_DEPRECATED, counters}; } bool Unifier::isNonstrictMode() const diff --git a/Ast/include/Luau/Ast.h b/Ast/include/Luau/Ast.h index df38cfec..a2189f7b 100644 --- a/Ast/include/Luau/Ast.h +++ b/Ast/include/Luau/Ast.h @@ -264,6 +264,10 @@ public: { return false; } + virtual bool visit(class AstTypePackExplicit* node) + { + return visit((class AstTypePack*)node); + } virtual bool visit(class AstTypePackVariadic* node) { return visit((class AstTypePack*)node); @@ -930,12 +934,14 @@ class AstStatTypeAlias : public AstStat public: LUAU_RTTI(AstStatTypeAlias) - AstStatTypeAlias(const Location& location, const AstName& name, const AstArray& generics, AstType* type, bool exported); + AstStatTypeAlias(const Location& location, const AstName& name, const AstArray& generics, const AstArray& genericPacks, + AstType* type, bool exported); void visit(AstVisitor* visitor) override; AstName name; AstArray generics; + AstArray genericPacks; AstType* type; bool exported; }; @@ -1007,19 +1013,28 @@ public: } }; +// Don't have Luau::Variant available, it's a bit of an overhead, but a plain struct is nice to use +struct AstTypeOrPack +{ + AstType* type = nullptr; + AstTypePack* typePack = nullptr; +}; + class AstTypeReference : public AstType { public: LUAU_RTTI(AstTypeReference) - AstTypeReference(const Location& location, std::optional prefix, AstName name, const AstArray& generics = {}); + AstTypeReference(const Location& location, std::optional prefix, AstName name, bool hasParameterList = false, + const AstArray& parameters = {}); void visit(AstVisitor* visitor) override; bool hasPrefix; + bool hasParameterList; AstName prefix; AstName name; - AstArray generics; + AstArray parameters; }; struct AstTableProp @@ -1152,6 +1167,18 @@ public: } }; +class AstTypePackExplicit : public AstTypePack +{ +public: + LUAU_RTTI(AstTypePackExplicit) + + AstTypePackExplicit(const Location& location, AstTypeList typeList); + + void visit(AstVisitor* visitor) override; + + AstTypeList typeList; +}; + class AstTypePackVariadic : public AstTypePack { public: diff --git a/Ast/include/Luau/DenseHash.h b/Ast/include/Luau/DenseHash.h index 02924e88..a7b2515a 100644 --- a/Ast/include/Luau/DenseHash.h +++ b/Ast/include/Luau/DenseHash.h @@ -136,7 +136,10 @@ public: const Key& key = ItemInterface::getKey(data[i]); if (!eq(key, empty_key)) - *newtable.insert_unsafe(key) = data[i]; + { + Item* item = newtable.insert_unsafe(key); + *item = std::move(data[i]); + } } LUAU_ASSERT(count == newtable.count); diff --git a/Ast/include/Luau/Parser.h b/Ast/include/Luau/Parser.h index e6ebd503..42c64dc9 100644 --- a/Ast/include/Luau/Parser.h +++ b/Ast/include/Luau/Parser.h @@ -218,13 +218,14 @@ private: AstTableIndexer* parseTableIndexerAnnotation(); - AstType* parseFunctionTypeAnnotation(); + AstTypeOrPack parseFunctionTypeAnnotation(bool allowPack); AstType* parseFunctionTypeAnnotationTail(const Lexeme& begin, AstArray generics, AstArray genericPacks, AstArray& params, AstArray>& paramNames, AstTypePack* varargAnnotation); AstType* parseTableTypeAnnotation(); - AstType* parseSimpleTypeAnnotation(); + AstTypeOrPack parseSimpleTypeAnnotation(bool allowPack); + AstTypeOrPack parseTypeOrPackAnnotation(); AstType* parseTypeAnnotation(TempVector& parts, const Location& begin); AstType* parseTypeAnnotation(); @@ -284,7 +285,7 @@ private: std::pair, AstArray> parseGenericTypeListIfFFlagParseGenericFunctions(); // `<' typeAnnotation[, ...] `>' - AstArray parseTypeParams(); + AstArray parseTypeParams(); AstExpr* parseString(); @@ -413,6 +414,7 @@ private: std::vector scratchLocal; std::vector scratchTableTypeProps; std::vector scratchAnnotation; + std::vector scratchTypeOrPackAnnotation; std::vector scratchDeclaredClassProps; std::vector scratchItem; std::vector scratchArgName; diff --git a/Ast/include/Luau/TimeTrace.h b/Ast/include/Luau/TimeTrace.h new file mode 100644 index 00000000..641dfd3c --- /dev/null +++ b/Ast/include/Luau/TimeTrace.h @@ -0,0 +1,223 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +#pragma once + +#include "Common.h" + +#include + +#include + +LUAU_FASTFLAG(DebugLuauTimeTracing) + +#if defined(LUAU_ENABLE_TIME_TRACE) + +namespace Luau +{ +namespace TimeTrace +{ +uint32_t getClockMicroseconds(); + +struct Token +{ + const char* name; + const char* category; +}; + +enum class EventType : uint8_t +{ + Enter, + Leave, + + ArgName, + ArgValue, +}; + +struct Event +{ + EventType type; + uint16_t token; + + union + { + uint32_t microsec; // 1 hour trace limit + uint32_t dataPos; + } data; +}; + +struct GlobalContext; +struct ThreadContext; + +GlobalContext& getGlobalContext(); + +uint16_t createToken(GlobalContext& context, const char* name, const char* category); +uint32_t createThread(GlobalContext& context, ThreadContext* threadContext); +void releaseThread(GlobalContext& context, ThreadContext* threadContext); +void flushEvents(GlobalContext& context, uint32_t threadId, const std::vector& events, const std::vector& data); + +struct ThreadContext +{ + ThreadContext() + : globalContext(getGlobalContext()) + { + threadId = createThread(globalContext, this); + } + + ~ThreadContext() + { + if (!events.empty()) + flushEvents(); + + releaseThread(globalContext, this); + } + + void flushEvents() + { + static uint16_t flushToken = createToken(globalContext, "flushEvents", "TimeTrace"); + + events.push_back({EventType::Enter, flushToken, {getClockMicroseconds()}}); + + TimeTrace::flushEvents(globalContext, threadId, events, data); + + events.clear(); + data.clear(); + + events.push_back({EventType::Leave, 0, {getClockMicroseconds()}}); + } + + void eventEnter(uint16_t token) + { + eventEnter(token, getClockMicroseconds()); + } + + void eventEnter(uint16_t token, uint32_t microsec) + { + events.push_back({EventType::Enter, token, {microsec}}); + } + + void eventLeave() + { + eventLeave(getClockMicroseconds()); + } + + void eventLeave(uint32_t microsec) + { + events.push_back({EventType::Leave, 0, {microsec}}); + + if (events.size() > kEventFlushLimit) + flushEvents(); + } + + void eventArgument(const char* name, const char* value) + { + uint32_t pos = uint32_t(data.size()); + data.insert(data.end(), name, name + strlen(name) + 1); + events.push_back({EventType::ArgName, 0, {pos}}); + + pos = uint32_t(data.size()); + data.insert(data.end(), value, value + strlen(value) + 1); + events.push_back({EventType::ArgValue, 0, {pos}}); + } + + GlobalContext& globalContext; + uint32_t threadId; + std::vector events; + std::vector data; + + static constexpr size_t kEventFlushLimit = 8192; +}; + +ThreadContext& getThreadContext(); + +struct Scope +{ + explicit Scope(ThreadContext& context, uint16_t token) + : context(context) + { + if (!FFlag::DebugLuauTimeTracing) + return; + + context.eventEnter(token); + } + + ~Scope() + { + if (!FFlag::DebugLuauTimeTracing) + return; + + context.eventLeave(); + } + + ThreadContext& context; +}; + +struct OptionalTailScope +{ + explicit OptionalTailScope(ThreadContext& context, uint16_t token, uint32_t threshold) + : context(context) + , token(token) + , threshold(threshold) + { + if (!FFlag::DebugLuauTimeTracing) + return; + + pos = uint32_t(context.events.size()); + microsec = getClockMicroseconds(); + } + + ~OptionalTailScope() + { + if (!FFlag::DebugLuauTimeTracing) + return; + + if (pos == context.events.size()) + { + uint32_t curr = getClockMicroseconds(); + + if (curr - microsec > threshold) + { + context.eventEnter(token, microsec); + context.eventLeave(curr); + } + } + } + + ThreadContext& context; + uint16_t token; + uint32_t threshold; + uint32_t microsec; + uint32_t pos; +}; + +LUAU_NOINLINE std::pair createScopeData(const char* name, const char* category); + +} // namespace TimeTrace +} // namespace Luau + +// Regular scope +#define LUAU_TIMETRACE_SCOPE(name, category) \ + static auto lttScopeStatic = Luau::TimeTrace::createScopeData(name, category); \ + Luau::TimeTrace::Scope lttScope(lttScopeStatic.second, lttScopeStatic.first) + +// A scope without nested scopes that may be skipped if the time it took is less than the threshold +#define LUAU_TIMETRACE_OPTIONAL_TAIL_SCOPE(name, category, microsec) \ + static auto lttScopeStaticOptTail = Luau::TimeTrace::createScopeData(name, category); \ + Luau::TimeTrace::OptionalTailScope lttScope(lttScopeStaticOptTail.second, lttScopeStaticOptTail.first, microsec) + +// Extra key/value data can be added to regular scopes +#define LUAU_TIMETRACE_ARGUMENT(name, value) \ + do \ + { \ + if (FFlag::DebugLuauTimeTracing) \ + lttScopeStatic.second.eventArgument(name, value); \ + } while (false) + +#else + +#define LUAU_TIMETRACE_SCOPE(name, category) +#define LUAU_TIMETRACE_OPTIONAL_TAIL_SCOPE(name, category, microsec) +#define LUAU_TIMETRACE_ARGUMENT(name, value) \ + do \ + { \ + } while (false) + +#endif diff --git a/Ast/src/Ast.cpp b/Ast/src/Ast.cpp index fff1537d..b1209faa 100644 --- a/Ast/src/Ast.cpp +++ b/Ast/src/Ast.cpp @@ -641,10 +641,12 @@ void AstStatLocalFunction::visit(AstVisitor* visitor) func->visit(visitor); } -AstStatTypeAlias::AstStatTypeAlias(const Location& location, const AstName& name, const AstArray& generics, AstType* type, bool exported) +AstStatTypeAlias::AstStatTypeAlias(const Location& location, const AstName& name, const AstArray& generics, + const AstArray& genericPacks, AstType* type, bool exported) : AstStat(ClassIndex(), location) , name(name) , generics(generics) + , genericPacks(genericPacks) , type(type) , exported(exported) { @@ -729,12 +731,14 @@ void AstStatError::visit(AstVisitor* visitor) } } -AstTypeReference::AstTypeReference(const Location& location, std::optional prefix, AstName name, const AstArray& generics) +AstTypeReference::AstTypeReference( + const Location& location, std::optional prefix, AstName name, bool hasParameterList, const AstArray& parameters) : AstType(ClassIndex(), location) , hasPrefix(bool(prefix)) + , hasParameterList(hasParameterList) , prefix(prefix ? *prefix : AstName()) , name(name) - , generics(generics) + , parameters(parameters) { } @@ -742,8 +746,13 @@ void AstTypeReference::visit(AstVisitor* visitor) { if (visitor->visit(this)) { - for (AstType* generic : generics) - generic->visit(visitor); + for (const AstTypeOrPack& param : parameters) + { + if (param.type) + param.type->visit(visitor); + else + param.typePack->visit(visitor); + } } } @@ -849,6 +858,24 @@ void AstTypeError::visit(AstVisitor* visitor) } } +AstTypePackExplicit::AstTypePackExplicit(const Location& location, AstTypeList typeList) + : AstTypePack(ClassIndex(), location) + , typeList(typeList) +{ +} + +void AstTypePackExplicit::visit(AstVisitor* visitor) +{ + if (visitor->visit(this)) + { + for (AstType* type : typeList.types) + type->visit(visitor); + + if (typeList.tailType) + typeList.tailType->visit(visitor); + } +} + AstTypePackVariadic::AstTypePackVariadic(const Location& location, AstType* variadicType) : AstTypePack(ClassIndex(), location) , variadicType(variadicType) diff --git a/Ast/src/Parser.cpp b/Ast/src/Parser.cpp index 6672efe8..40026d8b 100644 --- a/Ast/src/Parser.cpp +++ b/Ast/src/Parser.cpp @@ -1,6 +1,8 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #include "Luau/Parser.h" +#include "Luau/TimeTrace.h" + #include // Warning: If you are introducing new syntax, ensure that it is behind a separate @@ -13,6 +15,8 @@ LUAU_FASTFLAGVARIABLE(LuauParseGenericFunctions, false) LUAU_FASTFLAGVARIABLE(LuauCaptureBrokenCommentSpans, false) LUAU_FASTFLAGVARIABLE(LuauIfElseExpressionBaseSupport, false) LUAU_FASTFLAGVARIABLE(LuauIfStatementRecursionGuard, false) +LUAU_FASTFLAGVARIABLE(LuauTypeAliasPacks, false) +LUAU_FASTFLAGVARIABLE(LuauParseTypePackTypeParameters, false) namespace Luau { @@ -148,6 +152,8 @@ static bool shouldParseTypePackAnnotation(Lexer& lexer) ParseResult Parser::parse(const char* buffer, size_t bufferSize, AstNameTable& names, Allocator& allocator, ParseOptions options) { + LUAU_TIMETRACE_SCOPE("Parser::parse", "Parser"); + Parser p(buffer, bufferSize, names, allocator); try @@ -769,14 +775,14 @@ AstStat* Parser::parseTypeAlias(const Location& start, bool exported) if (!name) name = Name(nameError, lexer.current().location); - // TODO: support generic type pack parameters in type aliases CLI-39907 auto [generics, genericPacks] = parseGenericTypeList(); expectAndConsume('=', "type alias"); AstType* type = parseTypeAnnotation(); - return allocator.alloc(Location(start, type->location), name->name, generics, type, exported); + return allocator.alloc( + Location(start, type->location), name->name, generics, FFlag::LuauTypeAliasPacks ? genericPacks : AstArray{}, type, exported); } AstDeclaredClassProp Parser::parseDeclaredClassMethod() @@ -1333,7 +1339,7 @@ AstType* Parser::parseTableTypeAnnotation() // ReturnType ::= TypeAnnotation | `(' TypeList `)' // FunctionTypeAnnotation ::= [`<' varlist `>'] `(' [TypeList] `)' `->` ReturnType -AstType* Parser::parseFunctionTypeAnnotation() +AstTypeOrPack Parser::parseFunctionTypeAnnotation(bool allowPack) { incrementRecursionCounter("type annotation"); @@ -1364,14 +1370,23 @@ AstType* Parser::parseFunctionTypeAnnotation() matchRecoveryStopOnToken[Lexeme::SkinnyArrow]--; - // Not a function at all. Just a parenthesized type. - if (params.size() == 1 && !varargAnnotation && monomorphic && lexer.current().type != Lexeme::SkinnyArrow) - return params[0]; - AstArray paramTypes = copy(params); + + // Not a function at all. Just a parenthesized type. Or maybe a type pack with a single element + if (params.size() == 1 && !varargAnnotation && monomorphic && lexer.current().type != Lexeme::SkinnyArrow) + { + if (allowPack) + return {{}, allocator.alloc(begin.location, AstTypeList{paramTypes, nullptr})}; + else + return {params[0], {}}; + } + + if (lexer.current().type != Lexeme::SkinnyArrow && monomorphic && allowPack) + return {{}, allocator.alloc(begin.location, AstTypeList{paramTypes, varargAnnotation})}; + AstArray> paramNames = copy(names); - return parseFunctionTypeAnnotationTail(begin, generics, genericPacks, paramTypes, paramNames, varargAnnotation); + return {parseFunctionTypeAnnotationTail(begin, generics, genericPacks, paramTypes, paramNames, varargAnnotation), {}}; } AstType* Parser::parseFunctionTypeAnnotationTail(const Lexeme& begin, AstArray generics, AstArray genericPacks, @@ -1421,7 +1436,7 @@ AstType* Parser::parseTypeAnnotation(TempVector& parts, const Location if (c == '|') { nextLexeme(); - parts.push_back(parseSimpleTypeAnnotation()); + parts.push_back(parseSimpleTypeAnnotation(false).type); isUnion = true; } else if (c == '?') @@ -1434,7 +1449,7 @@ AstType* Parser::parseTypeAnnotation(TempVector& parts, const Location else if (c == '&') { nextLexeme(); - parts.push_back(parseSimpleTypeAnnotation()); + parts.push_back(parseSimpleTypeAnnotation(false).type); isIntersection = true; } else @@ -1462,6 +1477,30 @@ AstType* Parser::parseTypeAnnotation(TempVector& parts, const Location ParseError::raise(begin, "Composite type was not an intersection or union."); } +AstTypeOrPack Parser::parseTypeOrPackAnnotation() +{ + unsigned int oldRecursionCount = recursionCounter; + incrementRecursionCounter("type annotation"); + + Location begin = lexer.current().location; + + TempVector parts(scratchAnnotation); + + auto [type, typePack] = parseSimpleTypeAnnotation(true); + + if (typePack) + { + LUAU_ASSERT(!type); + return {{}, typePack}; + } + + parts.push_back(type); + + recursionCounter = oldRecursionCount; + + return {parseTypeAnnotation(parts, begin), {}}; +} + AstType* Parser::parseTypeAnnotation() { unsigned int oldRecursionCount = recursionCounter; @@ -1470,7 +1509,7 @@ AstType* Parser::parseTypeAnnotation() Location begin = lexer.current().location; TempVector parts(scratchAnnotation); - parts.push_back(parseSimpleTypeAnnotation()); + parts.push_back(parseSimpleTypeAnnotation(false).type); recursionCounter = oldRecursionCount; @@ -1479,7 +1518,7 @@ AstType* Parser::parseTypeAnnotation() // typeannotation ::= nil | Name[`.' Name] [ `<' typeannotation [`,' ...] `>' ] | `typeof' `(' expr `)' | `{' [PropList] `}' // | [`<' varlist `>'] `(' [TypeList] `)' `->` ReturnType -AstType* Parser::parseSimpleTypeAnnotation() +AstTypeOrPack Parser::parseSimpleTypeAnnotation(bool allowPack) { incrementRecursionCounter("type annotation"); @@ -1488,7 +1527,7 @@ AstType* Parser::parseSimpleTypeAnnotation() if (lexer.current().type == Lexeme::ReservedNil) { nextLexeme(); - return allocator.alloc(begin, std::nullopt, nameNil); + return {allocator.alloc(begin, std::nullopt, nameNil), {}}; } else if (lexer.current().type == Lexeme::Name) { @@ -1514,22 +1553,41 @@ AstType* Parser::parseSimpleTypeAnnotation() expectMatchAndConsume(')', typeofBegin); - return allocator.alloc(Location(begin, end), expr); + return {allocator.alloc(Location(begin, end), expr), {}}; } - AstArray generics = parseTypeParams(); + if (FFlag::LuauParseTypePackTypeParameters) + { + bool hasParameters = false; + AstArray parameters{}; - Location end = lexer.previousLocation(); + if (lexer.current().type == '<') + { + hasParameters = true; + parameters = parseTypeParams(); + } - return allocator.alloc(Location(begin, end), prefix, name.name, generics); + Location end = lexer.previousLocation(); + + return {allocator.alloc(Location(begin, end), prefix, name.name, hasParameters, parameters), {}}; + } + else + { + AstArray generics = parseTypeParams(); + + Location end = lexer.previousLocation(); + + // false in 'hasParameterList' as it is not used without FFlagLuauTypeAliasPacks + return {allocator.alloc(Location(begin, end), prefix, name.name, false, generics), {}}; + } } else if (lexer.current().type == '{') { - return parseTableTypeAnnotation(); + return {parseTableTypeAnnotation(), {}}; } else if (lexer.current().type == '(' || (FFlag::LuauParseGenericFunctions && lexer.current().type == '<')) { - return parseFunctionTypeAnnotation(); + return parseFunctionTypeAnnotation(allowPack); } else { @@ -1538,7 +1596,7 @@ AstType* Parser::parseSimpleTypeAnnotation() // For a missing type annoation, capture 'space' between last token and the next one location = Location(lexer.previousLocation().end, lexer.current().location.begin); - return reportTypeAnnotationError(location, {}, /*isMissing*/ true, "Expected type, got %s", lexer.current().toString().c_str()); + return {reportTypeAnnotationError(location, {}, /*isMissing*/ true, "Expected type, got %s", lexer.current().toString().c_str()), {}}; } } @@ -2312,18 +2370,59 @@ std::pair, AstArray> Parser::parseGenericTypeList() return {generics, genericPacks}; } -AstArray Parser::parseTypeParams() +AstArray Parser::parseTypeParams() { - TempVector result{scratchAnnotation}; + TempVector parameters{scratchTypeOrPackAnnotation}; if (lexer.current().type == '<') { Lexeme begin = lexer.current(); nextLexeme(); + bool seenPack = false; while (true) { - result.push_back(parseTypeAnnotation()); + if (FFlag::LuauParseTypePackTypeParameters) + { + if (shouldParseTypePackAnnotation(lexer)) + { + seenPack = true; + + auto typePack = parseTypePackAnnotation(); + + if (FFlag::LuauTypeAliasPacks) // Type packs are recorded only is we can handle them + parameters.push_back({{}, typePack}); + } + else if (lexer.current().type == '(') + { + auto [type, typePack] = parseTypeOrPackAnnotation(); + + if (typePack) + { + seenPack = true; + + if (FFlag::LuauTypeAliasPacks) // Type packs are recorded only is we can handle them + parameters.push_back({{}, typePack}); + } + else + { + parameters.push_back({type, {}}); + } + } + else if (lexer.current().type == '>' && parameters.empty()) + { + break; + } + else + { + parameters.push_back({parseTypeAnnotation(), {}}); + } + } + else + { + parameters.push_back({parseTypeAnnotation(), {}}); + } + if (lexer.current().type == ',') nextLexeme(); else @@ -2333,7 +2432,7 @@ AstArray Parser::parseTypeParams() expectMatchAndConsume('>', begin); } - return copy(result); + return copy(parameters); } AstExpr* Parser::parseString() diff --git a/Ast/src/TimeTrace.cpp b/Ast/src/TimeTrace.cpp new file mode 100644 index 00000000..e6aab20e --- /dev/null +++ b/Ast/src/TimeTrace.cpp @@ -0,0 +1,248 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +#include "Luau/TimeTrace.h" + +#include "Luau/StringUtils.h" + +#include +#include + +#include + +#ifdef _WIN32 +#include +#endif + +#ifdef __APPLE__ +#include +#include +#endif + +#include + +LUAU_FASTFLAGVARIABLE(DebugLuauTimeTracing, false) + +#if defined(LUAU_ENABLE_TIME_TRACE) + +namespace Luau +{ +namespace TimeTrace +{ +static double getClockPeriod() +{ +#if defined(_WIN32) + LARGE_INTEGER result = {}; + QueryPerformanceFrequency(&result); + return 1.0 / double(result.QuadPart); +#elif defined(__APPLE__) + mach_timebase_info_data_t result = {}; + mach_timebase_info(&result); + return double(result.numer) / double(result.denom) * 1e-9; +#elif defined(__linux__) + return 1e-9; +#else + return 1.0 / double(CLOCKS_PER_SEC); +#endif +} + +static double getClockTimestamp() +{ +#if defined(_WIN32) + LARGE_INTEGER result = {}; + QueryPerformanceCounter(&result); + return double(result.QuadPart); +#elif defined(__APPLE__) + return double(mach_absolute_time()); +#elif defined(__linux__) + timespec now; + clock_gettime(CLOCK_MONOTONIC, &now); + return now.tv_sec * 1e9 + now.tv_nsec; +#else + return double(clock()); +#endif +} + +uint32_t getClockMicroseconds() +{ + static double period = getClockPeriod() * 1e6; + static double start = getClockTimestamp(); + + return uint32_t((getClockTimestamp() - start) * period); +} + +struct GlobalContext +{ + GlobalContext() = default; + ~GlobalContext() + { + // Ideally we would want all ThreadContext destructors to run + // But in VS, not all thread_local object instances are destroyed + for (ThreadContext* context : threads) + context->flushEvents(); + + if (traceFile) + fclose(traceFile); + } + + std::mutex mutex; + std::vector threads; + uint32_t nextThreadId = 0; + std::vector tokens; + FILE* traceFile = nullptr; +}; + +GlobalContext& getGlobalContext() +{ + static GlobalContext context; + return context; +} + +uint16_t createToken(GlobalContext& context, const char* name, const char* category) +{ + std::scoped_lock lock(context.mutex); + + LUAU_ASSERT(context.tokens.size() < 64 * 1024); + + context.tokens.push_back({name, category}); + return uint16_t(context.tokens.size() - 1); +} + +uint32_t createThread(GlobalContext& context, ThreadContext* threadContext) +{ + std::scoped_lock lock(context.mutex); + + context.threads.push_back(threadContext); + + return ++context.nextThreadId; +} + +void releaseThread(GlobalContext& context, ThreadContext* threadContext) +{ + std::scoped_lock lock(context.mutex); + + if (auto it = std::find(context.threads.begin(), context.threads.end(), threadContext); it != context.threads.end()) + context.threads.erase(it); +} + +void flushEvents(GlobalContext& context, uint32_t threadId, const std::vector& events, const std::vector& data) +{ + std::scoped_lock lock(context.mutex); + + if (!context.traceFile) + { + context.traceFile = fopen("trace.json", "w"); + + if (!context.traceFile) + return; + + fprintf(context.traceFile, "[\n"); + } + + std::string temp; + const unsigned tempReserve = 64 * 1024; + temp.reserve(tempReserve); + + const char* rawData = data.data(); + + // Formatting state + bool unfinishedEnter = false; + bool unfinishedArgs = false; + + for (const Event& ev : events) + { + switch (ev.type) + { + case EventType::Enter: + { + if (unfinishedArgs) + { + formatAppend(temp, "}"); + unfinishedArgs = false; + } + + if (unfinishedEnter) + { + formatAppend(temp, "},\n"); + unfinishedEnter = false; + } + + Token& token = context.tokens[ev.token]; + + formatAppend(temp, R"({"name": "%s", "cat": "%s", "ph": "B", "ts": %u, "pid": 0, "tid": %u)", token.name, token.category, + ev.data.microsec, threadId); + unfinishedEnter = true; + } + break; + case EventType::Leave: + if (unfinishedArgs) + { + formatAppend(temp, "}"); + unfinishedArgs = false; + } + if (unfinishedEnter) + { + formatAppend(temp, "},\n"); + unfinishedEnter = false; + } + + formatAppend(temp, + R"({"ph": "E", "ts": %u, "pid": 0, "tid": %u},)" + "\n", + ev.data.microsec, threadId); + break; + case EventType::ArgName: + LUAU_ASSERT(unfinishedEnter); + + if (!unfinishedArgs) + { + formatAppend(temp, R"(, "args": { "%s": )", rawData + ev.data.dataPos); + unfinishedArgs = true; + } + else + { + formatAppend(temp, R"(, "%s": )", rawData + ev.data.dataPos); + } + break; + case EventType::ArgValue: + LUAU_ASSERT(unfinishedArgs); + formatAppend(temp, R"("%s")", rawData + ev.data.dataPos); + break; + } + + // Don't want to hit the string capacity and reallocate + if (temp.size() > tempReserve - 1024) + { + fwrite(temp.data(), 1, temp.size(), context.traceFile); + temp.clear(); + } + } + + if (unfinishedArgs) + { + formatAppend(temp, "}"); + unfinishedArgs = false; + } + if (unfinishedEnter) + { + formatAppend(temp, "},\n"); + unfinishedEnter = false; + } + + fwrite(temp.data(), 1, temp.size(), context.traceFile); + fflush(context.traceFile); +} + +ThreadContext& getThreadContext() +{ + thread_local ThreadContext context; + return context; +} + +std::pair createScopeData(const char* name, const char* category) +{ + uint16_t token = createToken(Luau::TimeTrace::getGlobalContext(), name, category); + return {token, Luau::TimeTrace::getThreadContext()}; +} +} // namespace TimeTrace +} // namespace Luau + +#endif diff --git a/CLI/Analyze.cpp b/CLI/Analyze.cpp index 920502b8..ed0552d7 100644 --- a/CLI/Analyze.cpp +++ b/CLI/Analyze.cpp @@ -111,11 +111,24 @@ struct CliFileResolver : Luau::FileResolver return Luau::SourceCode{*source, Luau::SourceCode::Module}; } + std::optional resolveModule(const Luau::ModuleInfo* context, Luau::AstExpr* node) override + { + if (Luau::AstExprConstantString* expr = node->as()) + { + Luau::ModuleName name = std::string(expr->value.data, expr->value.size) + ".lua"; + + return {{name}}; + } + + return std::nullopt; + } + bool moduleExists(const Luau::ModuleName& name) const override { return !!readFile(name); } + std::optional fromAstFragment(Luau::AstExpr* expr) const override { return std::nullopt; @@ -130,11 +143,6 @@ struct CliFileResolver : Luau::FileResolver { return std::nullopt; } - - std::optional getEnvironmentForModule(const Luau::ModuleName& name) const override - { - return std::nullopt; - } }; struct CliConfigResolver : Luau::ConfigResolver diff --git a/Compiler/src/Compiler.cpp b/Compiler/src/Compiler.cpp index 022eccb7..797ee20d 100644 --- a/Compiler/src/Compiler.cpp +++ b/Compiler/src/Compiler.cpp @@ -4,6 +4,7 @@ #include "Luau/Parser.h" #include "Luau/BytecodeBuilder.h" #include "Luau/Common.h" +#include "Luau/TimeTrace.h" #include #include @@ -137,6 +138,11 @@ struct Compiler uint32_t compileFunction(AstExprFunction* func) { + LUAU_TIMETRACE_SCOPE("Compiler::compileFunction", "Compiler"); + + if (func->debugname.value) + LUAU_TIMETRACE_ARGUMENT("name", func->debugname.value); + LUAU_ASSERT(!functions.contains(func)); LUAU_ASSERT(regTop == 0 && stackSize == 0 && localStack.empty() && upvals.empty()); @@ -3686,6 +3692,8 @@ struct Compiler void compileOrThrow(BytecodeBuilder& bytecode, AstStatBlock* root, const AstNameTable& names, const CompileOptions& options) { + LUAU_TIMETRACE_SCOPE("compileOrThrow", "Compiler"); + Compiler compiler(bytecode, options); // since access to some global objects may result in values that change over time, we block table imports @@ -3748,6 +3756,8 @@ void compileOrThrow(BytecodeBuilder& bytecode, const std::string& source, const std::string compile(const std::string& source, const CompileOptions& options, const ParseOptions& parseOptions, BytecodeEncoder* encoder) { + LUAU_TIMETRACE_SCOPE("compile", "Compiler"); + Allocator allocator; AstNameTable names(allocator); ParseResult result = Parser::parse(source.c_str(), source.size(), names, allocator, parseOptions); diff --git a/Sources.cmake b/Sources.cmake index 6f96f6ab..83ed5230 100644 --- a/Sources.cmake +++ b/Sources.cmake @@ -9,6 +9,7 @@ target_sources(Luau.Ast PRIVATE Ast/include/Luau/ParseOptions.h Ast/include/Luau/Parser.h Ast/include/Luau/StringUtils.h + Ast/include/Luau/TimeTrace.h Ast/src/Ast.cpp Ast/src/Confusables.cpp @@ -16,6 +17,7 @@ target_sources(Luau.Ast PRIVATE Ast/src/Location.cpp Ast/src/Parser.cpp Ast/src/StringUtils.cpp + Ast/src/TimeTrace.cpp ) # Luau.Compiler Sources @@ -46,6 +48,7 @@ target_sources(Luau.Analysis PRIVATE Analysis/include/Luau/Predicate.h Analysis/include/Luau/RecursionCounter.h Analysis/include/Luau/RequireTracer.h + Analysis/include/Luau/Scope.h Analysis/include/Luau/Substitution.h Analysis/include/Luau/Symbol.h Analysis/include/Luau/TopoSortStatements.h @@ -75,6 +78,7 @@ target_sources(Luau.Analysis PRIVATE Analysis/src/Module.cpp Analysis/src/Predicate.cpp Analysis/src/RequireTracer.cpp + Analysis/src/Scope.cpp Analysis/src/Substitution.cpp Analysis/src/Symbol.cpp Analysis/src/TopoSortStatements.cpp @@ -188,6 +192,7 @@ if(TARGET Luau.UnitTest) tests/TopoSort.test.cpp tests/ToString.test.cpp tests/Transpiler.test.cpp + tests/TypeInfer.aliases.test.cpp tests/TypeInfer.annotations.test.cpp tests/TypeInfer.builtins.test.cpp tests/TypeInfer.classes.test.cpp diff --git a/VM/src/ldo.cpp b/VM/src/ldo.cpp index ee4962a5..39a61597 100644 --- a/VM/src/ldo.cpp +++ b/VM/src/ldo.cpp @@ -8,9 +8,13 @@ #include "lmem.h" #include "lvm.h" -#include - +#if LUA_USE_LONGJMP #include +#include +#else +#include +#endif + #include LUAU_FASTFLAGVARIABLE(LuauExceptionMessageFix, false) @@ -51,8 +55,8 @@ l_noret luaD_throw(lua_State* L, int errcode) longjmp(jb->buf, 1); } - if (L->global->panic) - L->global->panic(L, errcode); + if (L->global->cb.panic) + L->global->cb.panic(L, errcode); abort(); } diff --git a/VM/src/lgc.cpp b/VM/src/lgc.cpp index 9b040fb5..510a9f54 100644 --- a/VM/src/lgc.cpp +++ b/VM/src/lgc.cpp @@ -16,6 +16,8 @@ LUAU_FASTFLAGVARIABLE(LuauRescanGrayAgain, false) LUAU_FASTFLAGVARIABLE(LuauRescanGrayAgainForwardBarrier, false) LUAU_FASTFLAGVARIABLE(LuauGcFullSkipInactiveThreads, false) LUAU_FASTFLAGVARIABLE(LuauShrinkWeakTables, false) +LUAU_FASTFLAGVARIABLE(LuauConsolidatedStep, false) + LUAU_FASTFLAG(LuauArrayBoundary) #define GC_SWEEPMAX 40 @@ -810,6 +812,133 @@ static size_t singlestep(lua_State* L) return cost; } +static size_t gcstep(lua_State* L, size_t limit) +{ + size_t cost = 0; + global_State* g = L->global; + switch (g->gcstate) + { + case GCSpause: + { + markroot(L); /* start a new collection */ + break; + } + case GCSpropagate: + { + if (FFlag::LuauRescanGrayAgain) + { + while (g->gray && cost < limit) + { + g->gcstats.currcycle.markitems++; + + cost += propagatemark(g); + } + + if (!g->gray) + { + // perform one iteration over 'gray again' list + g->gray = g->grayagain; + g->grayagain = NULL; + + g->gcstate = GCSpropagateagain; + } + } + else + { + while (g->gray && cost < limit) + { + g->gcstats.currcycle.markitems++; + + cost += propagatemark(g); + } + + if (!g->gray) /* no more `gray' objects */ + { + double starttimestamp = lua_clock(); + + g->gcstats.currcycle.atomicstarttimestamp = starttimestamp; + g->gcstats.currcycle.atomicstarttotalsizebytes = g->totalbytes; + + atomic(L); /* finish mark phase */ + + g->gcstats.currcycle.atomictime += lua_clock() - starttimestamp; + } + } + break; + } + case GCSpropagateagain: + { + while (g->gray && cost < limit) + { + g->gcstats.currcycle.markitems++; + + cost += propagatemark(g); + } + + if (!g->gray) /* no more `gray' objects */ + { + double starttimestamp = lua_clock(); + + g->gcstats.currcycle.atomicstarttimestamp = starttimestamp; + g->gcstats.currcycle.atomicstarttotalsizebytes = g->totalbytes; + + atomic(L); /* finish mark phase */ + + g->gcstats.currcycle.atomictime += lua_clock() - starttimestamp; + } + break; + } + case GCSsweepstring: + { + while (g->sweepstrgc < g->strt.size && cost < limit) + { + size_t traversedcount = 0; + sweepwholelist(L, &g->strt.hash[g->sweepstrgc++], &traversedcount); + + g->gcstats.currcycle.sweepitems += traversedcount; + cost += GC_SWEEPCOST; + } + + // nothing more to sweep? + if (g->sweepstrgc >= g->strt.size) + { + // sweep string buffer list and preserve used string count + uint32_t nuse = L->global->strt.nuse; + + size_t traversedcount = 0; + sweepwholelist(L, &g->strbufgc, &traversedcount); + + L->global->strt.nuse = nuse; + + g->gcstats.currcycle.sweepitems += traversedcount; + g->gcstate = GCSsweep; // end sweep-string phase + } + break; + } + case GCSsweep: + { + while (*g->sweepgc && cost < limit) + { + size_t traversedcount = 0; + g->sweepgc = sweeplist(L, g->sweepgc, GC_SWEEPMAX, &traversedcount); + + g->gcstats.currcycle.sweepitems += traversedcount; + cost += GC_SWEEPMAX * GC_SWEEPCOST; + } + + if (*g->sweepgc == NULL) + { /* nothing more to sweep? */ + shrinkbuffers(L); + g->gcstate = GCSpause; /* end collection */ + } + break; + } + default: + LUAU_ASSERT(0); + } + return cost; +} + static int64_t getheaptriggererroroffset(GCHeapTriggerStats* triggerstats, GCCycleStats* cyclestats) { // adjust for error using Proportional-Integral controller @@ -878,33 +1007,40 @@ void luaC_step(lua_State* L, bool assist) if (g->gcstate == GCSpause) startGcCycleStats(g); - if (assist) - g->gcstats.currcycle.assistwork += lim; - else - g->gcstats.currcycle.explicitwork += lim; - int lastgcstate = g->gcstate; double lastttimestamp = lua_clock(); - // always perform at least one single step - do + if (FFlag::LuauConsolidatedStep) { - lim -= singlestep(L); + size_t work = gcstep(L, lim); - // if we have switched to a different state, capture the duration of last stage - // this way we reduce the number of timer calls we make - if (lastgcstate != g->gcstate) + if (assist) + g->gcstats.currcycle.assistwork += work; + else + g->gcstats.currcycle.explicitwork += work; + } + else + { + // always perform at least one single step + do { - GC_INTERRUPT(lastgcstate); + lim -= singlestep(L); - double now = lua_clock(); + // if we have switched to a different state, capture the duration of last stage + // this way we reduce the number of timer calls we make + if (lastgcstate != g->gcstate) + { + GC_INTERRUPT(lastgcstate); - recordGcStateTime(g, lastgcstate, now - lastttimestamp, assist); + double now = lua_clock(); - lastttimestamp = now; - lastgcstate = g->gcstate; - } - } while (lim > 0 && g->gcstate != GCSpause); + recordGcStateTime(g, lastgcstate, now - lastttimestamp, assist); + + lastttimestamp = now; + lastgcstate = g->gcstate; + } + } while (lim > 0 && g->gcstate != GCSpause); + } recordGcStateTime(g, lastgcstate, lua_clock() - lastttimestamp, assist); @@ -931,7 +1067,14 @@ void luaC_step(lua_State* L, bool assist) g->GCthreshold -= debt; } - GC_INTERRUPT(g->gcstate); + if (FFlag::LuauConsolidatedStep) + { + GC_INTERRUPT(lastgcstate); + } + else + { + GC_INTERRUPT(g->gcstate); + } } void luaC_fullgc(lua_State* L) @@ -957,7 +1100,10 @@ void luaC_fullgc(lua_State* L) while (g->gcstate != GCSpause) { LUAU_ASSERT(g->gcstate == GCSsweepstring || g->gcstate == GCSsweep); - singlestep(L); + if (FFlag::LuauConsolidatedStep) + gcstep(L, SIZE_MAX); + else + singlestep(L); } finishGcCycleStats(g); @@ -968,7 +1114,10 @@ void luaC_fullgc(lua_State* L) markroot(L); while (g->gcstate != GCSpause) { - singlestep(L); + if (FFlag::LuauConsolidatedStep) + gcstep(L, SIZE_MAX); + else + singlestep(L); } /* reclaim as much buffer memory as possible (shrinkbuffers() called during sweep is incremental) */ shrinkbuffersfull(L); diff --git a/VM/src/ltablib.cpp b/VM/src/ltablib.cpp index 090e183f..de5788eb 100644 --- a/VM/src/ltablib.cpp +++ b/VM/src/ltablib.cpp @@ -9,14 +9,8 @@ #include "ldebug.h" #include "lvm.h" -LUAU_DYNAMIC_FASTFLAGVARIABLE(LuauTableMoveTelemetry, false) - LUAU_FASTFLAGVARIABLE(LuauTableFreeze, false) -bool lua_telemetry_table_move_oob_src_from = false; -bool lua_telemetry_table_move_oob_src_to = false; -bool lua_telemetry_table_move_oob_dst = false; - static int foreachi(lua_State* L) { luaL_checktype(L, 1, LUA_TTABLE); @@ -202,22 +196,6 @@ static int tmove(lua_State* L) int tt = !lua_isnoneornil(L, 5) ? 5 : 1; /* destination table */ luaL_checktype(L, tt, LUA_TTABLE); - if (DFFlag::LuauTableMoveTelemetry) - { - int nf = lua_objlen(L, 1); - int nt = lua_objlen(L, tt); - - // source index range must be in bounds in source table unless the table is empty (permits 1..#t moves) - if (!(f == 1 || (f >= 1 && f <= nf))) - lua_telemetry_table_move_oob_src_from = true; - if (!(e == nf || (e >= 1 && e <= nf))) - lua_telemetry_table_move_oob_src_to = true; - - // destination index must be in bounds in dest table or be exactly at the first empty element (permits concats) - if (!(t == nt + 1 || (t >= 1 && t <= nt + 1))) - lua_telemetry_table_move_oob_dst = true; - } - if (e >= f) { /* otherwise, nothing to move */ luaL_argcheck(L, f > 0 || e < INT_MAX + f, 3, "too many elements to move"); diff --git a/VM/src/lvmexecute.cpp b/VM/src/lvmexecute.cpp index 5f0ee922..eed2862b 100644 --- a/VM/src/lvmexecute.cpp +++ b/VM/src/lvmexecute.cpp @@ -16,8 +16,6 @@ #include -LUAU_FASTFLAGVARIABLE(LuauLoopUseSafeenv, false) - // Disable c99-designator to avoid the warning in CGOTO dispatch table #ifdef __clang__ #if __has_warning("-Wc99-designator") @@ -292,10 +290,6 @@ inline bool luau_skipstep(uint8_t op) return op == LOP_PREPVARARGS || op == LOP_BREAK; } -// declared in lbaselib.cpp, needed to support cases when pairs/ipairs have been replaced via setfenv -LUAI_FUNC int luaB_inext(lua_State* L); -LUAI_FUNC int luaB_next(lua_State* L); - template static void luau_execute(lua_State* L) { @@ -2223,8 +2217,7 @@ static void luau_execute(lua_State* L) StkId ra = VM_REG(LUAU_INSN_A(insn)); // fast-path: ipairs/inext - bool safeenv = FFlag::LuauLoopUseSafeenv ? cl->env->safeenv : ttisfunction(ra) && clvalue(ra)->isC && clvalue(ra)->c.f == luaB_inext; - if (safeenv && ttistable(ra + 1) && ttisnumber(ra + 2) && nvalue(ra + 2) == 0.0) + if (cl->env->safeenv && ttistable(ra + 1) && ttisnumber(ra + 2) && nvalue(ra + 2) == 0.0) { setpvalue(ra + 2, reinterpret_cast(uintptr_t(0))); } @@ -2304,8 +2297,7 @@ static void luau_execute(lua_State* L) StkId ra = VM_REG(LUAU_INSN_A(insn)); // fast-path: pairs/next - bool safeenv = FFlag::LuauLoopUseSafeenv ? cl->env->safeenv : ttisfunction(ra) && clvalue(ra)->isC && clvalue(ra)->c.f == luaB_next; - if (safeenv && ttistable(ra + 1) && ttisnil(ra + 2)) + if (cl->env->safeenv && ttistable(ra + 1) && ttisnil(ra + 2)) { setpvalue(ra + 2, reinterpret_cast(uintptr_t(0))); } diff --git a/VM/src/lvmload.cpp b/VM/src/lvmload.cpp index 0a232342..b932a85b 100644 --- a/VM/src/lvmload.cpp +++ b/VM/src/lvmload.cpp @@ -12,7 +12,32 @@ #include -#include +// TODO: RAII deallocation doesn't work for longjmp builds if a memory error happens +template +struct TempBuffer +{ + lua_State* L; + T* data; + size_t count; + + TempBuffer(lua_State* L, size_t count) + : L(L) + , data(luaM_newarray(L, count, T, 0)) + , count(count) + { + } + + ~TempBuffer() + { + luaM_freearray(L, data, count, T, 0); + } + + T& operator[](size_t index) + { + LUAU_ASSERT(index < count); + return data[index]; + } +}; void luaV_getimport(lua_State* L, Table* env, TValue* k, uint32_t id, bool propagatenil) { @@ -67,7 +92,7 @@ static unsigned int readVarInt(const char* data, size_t size, size_t& offset) return result; } -static TString* readString(std::vector& strings, const char* data, size_t size, size_t& offset) +static TString* readString(TempBuffer& strings, const char* data, size_t size, size_t& offset) { unsigned int id = readVarInt(data, size, offset); @@ -133,6 +158,7 @@ int luau_load(lua_State* L, const char* chunkname, const char* data, size_t size } // pause GC for the duration of deserialization - some objects we're creating aren't rooted + // TODO: if an allocation error happens mid-load, we do not unpause GC! size_t GCthreshold = L->global->GCthreshold; L->global->GCthreshold = SIZE_MAX; @@ -144,7 +170,7 @@ int luau_load(lua_State* L, const char* chunkname, const char* data, size_t size // string table unsigned int stringCount = readVarInt(data, size, offset); - std::vector strings(stringCount); + TempBuffer strings(L, stringCount); for (unsigned int i = 0; i < stringCount; ++i) { @@ -156,7 +182,7 @@ int luau_load(lua_State* L, const char* chunkname, const char* data, size_t size // proto table unsigned int protoCount = readVarInt(data, size, offset); - std::vector protos(protoCount); + TempBuffer protos(L, protoCount); for (unsigned int i = 0; i < protoCount; ++i) { diff --git a/bench/tests/deltablue.lua b/bench/tests/deltablue.lua deleted file mode 100644 index ecf246d3..00000000 --- a/bench/tests/deltablue.lua +++ /dev/null @@ -1,934 +0,0 @@ -local bench = script and require(script.Parent.bench_support) or require("bench_support") - --- Copyright 2008 the V8 project authors. All rights reserved. --- Copyright 1996 John Maloney and Mario Wolczko. - --- This program is free software; you can redistribute it and/or modify --- it under the terms of the GNU General Public License as published by --- the Free Software Foundation; either version 2 of the License, or --- (at your option) any later version. --- --- This program is distributed in the hope that it will be useful, --- but WITHOUT ANY WARRANTY; without even the implied warranty of --- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the --- GNU General Public License for more details. --- --- You should have received a copy of the GNU General Public License --- along with this program; if not, write to the Free Software --- Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA - - --- This implementation of the DeltaBlue benchmark is derived --- from the Smalltalk implementation by John Maloney and Mario --- Wolczko. Some parts have been translated directly, whereas --- others have been modified more aggresively to make it feel --- more like a JavaScript program. - - --- --- A JavaScript implementation of the DeltaBlue constraint-solving --- algorithm, as described in: --- --- "The DeltaBlue Algorithm: An Incremental Constraint Hierarchy Solver" --- Bjorn N. Freeman-Benson and John Maloney --- January 1990 Communications of the ACM, --- also available as University of Washington TR 89-08-06. --- --- Beware: this benchmark is written in a grotesque style where --- the constraint model is built by side-effects from constructors. --- I've kept it this way to avoid deviating too much from the original --- implementation. --- - -function class(base) - local T = {} - T.__index = T - - if base then - T.super = base - setmetatable(T, base) - end - - function T.new(...) - local O = {} - setmetatable(O, T) - O:constructor(...) - return O - end - - return T -end - -local planner - ---- O b j e c t M o d e l --- - -local function alert (...) print(...) end - -local OrderedCollection = class() - -function OrderedCollection:constructor() - self.elms = {} -end - -function OrderedCollection:add(elm) - self.elms[#self.elms + 1] = elm -end - -function OrderedCollection:at (index) - return self.elms[index] -end - -function OrderedCollection:size () - return #self.elms -end - -function OrderedCollection:removeFirst () - local e = self.elms[#self.elms] - self.elms[#self.elms] = nil - return e -end - -function OrderedCollection:remove (elm) - local index = 0 - local skipped = 0 - - for i = 1, #self.elms do - local value = self.elms[i] - if value ~= elm then - self.elms[index] = value - index = index + 1 - else - skipped = skipped + 1 - end - end - - local l = #self.elms - for i = 1, skipped do self.elms[l - i + 1] = nil end -end - --- --- S t r e n g t h --- - --- --- Strengths are used to measure the relative importance of constraints. --- New strengths may be inserted in the strength hierarchy without --- disrupting current constraints. Strengths cannot be created outside --- this class, so pointer comparison can be used for value comparison. --- - -local Strength = class() - -function Strength:constructor(strengthValue, name) - self.strengthValue = strengthValue - self.name = name -end - -function Strength.stronger (s1, s2) - return s1.strengthValue < s2.strengthValue -end - -function Strength.weaker (s1, s2) - return s1.strengthValue > s2.strengthValue -end - -function Strength.weakestOf (s1, s2) - return Strength.weaker(s1, s2) and s1 or s2 -end - -function Strength.strongest (s1, s2) - return Strength.stronger(s1, s2) and s1 or s2 -end - -function Strength:nextWeaker () - local v = self.strengthValue - if v == 0 then return Strength.WEAKEST - elseif v == 1 then return Strength.WEAK_DEFAULT - elseif v == 2 then return Strength.NORMAL - elseif v == 3 then return Strength.STRONG_DEFAULT - elseif v == 4 then return Strength.PREFERRED - elseif v == 5 then return Strength.REQUIRED - end -end - --- Strength constants. -Strength.REQUIRED = Strength.new(0, "required"); -Strength.STONG_PREFERRED = Strength.new(1, "strongPreferred"); -Strength.PREFERRED = Strength.new(2, "preferred"); -Strength.STRONG_DEFAULT = Strength.new(3, "strongDefault"); -Strength.NORMAL = Strength.new(4, "normal"); -Strength.WEAK_DEFAULT = Strength.new(5, "weakDefault"); -Strength.WEAKEST = Strength.new(6, "weakest"); - --- --- C o n s t r a i n t --- - --- --- An abstract class representing a system-maintainable relationship --- (or "constraint") between a set of variables. A constraint supplies --- a strength instance variable; concrete subclasses provide a means --- of storing the constrained variables and other information required --- to represent a constraint. --- - -local Constraint = class () - -function Constraint:constructor(strength) - self.strength = strength -end - --- --- Activate this constraint and attempt to satisfy it. --- -function Constraint:addConstraint () - self:addToGraph() - planner:incrementalAdd(self) -end - --- --- Attempt to find a way to enforce this constraint. If successful, --- record the solution, perhaps modifying the current dataflow --- graph. Answer the constraint that this constraint overrides, if --- there is one, or nil, if there isn't. --- Assume: I am not already satisfied. --- -function Constraint:satisfy (mark) - self:chooseMethod(mark) - if not self:isSatisfied() then - if self.strength == Strength.REQUIRED then - alert("Could not satisfy a required constraint!") - end - return nil - end - self:markInputs(mark) - local out = self:output() - local overridden = out.determinedBy - if overridden ~= nil then overridden:markUnsatisfied() end - out.determinedBy = self - if not planner:addPropagate(self, mark) then alert("Cycle encountered") end - out.mark = mark - return overridden -end - -function Constraint:destroyConstraint () - if self:isSatisfied() - then planner:incrementalRemove(self) - else self:removeFromGraph() - end -end - --- --- Normal constraints are not input constraints. An input constraint --- is one that depends on external state, such as the mouse, the --- keybord, a clock, or some arbitraty piece of imperative code. --- -function Constraint:isInput () - return false -end - - --- --- U n a r y C o n s t r a i n t --- - --- --- Abstract superclass for constraints having a single possible output --- variable. --- - -local UnaryConstraint = class(Constraint) - -function UnaryConstraint:constructor (v, strength) - UnaryConstraint.super.constructor(self, strength) - self.myOutput = v - self.satisfied = false - self:addConstraint() -end - --- --- Adds this constraint to the constraint graph --- -function UnaryConstraint:addToGraph () - self.myOutput:addConstraint(self) - self.satisfied = false -end - --- --- Decides if this constraint can be satisfied and records that --- decision. --- -function UnaryConstraint:chooseMethod (mark) - self.satisfied = (self.myOutput.mark ~= mark) - and Strength.stronger(self.strength, self.myOutput.walkStrength); -end - --- --- Returns true if this constraint is satisfied in the current solution. --- -function UnaryConstraint:isSatisfied () - return self.satisfied; -end - -function UnaryConstraint:markInputs (mark) - -- has no inputs -end - --- --- Returns the current output variable. --- -function UnaryConstraint:output () - return self.myOutput -end - --- --- Calculate the walkabout strength, the stay flag, and, if it is --- 'stay', the value for the current output of this constraint. Assume --- this constraint is satisfied. --- -function UnaryConstraint:recalculate () - self.myOutput.walkStrength = self.strength - self.myOutput.stay = not self:isInput() - if self.myOutput.stay then - self:execute() -- Stay optimization - end -end - --- --- Records that this constraint is unsatisfied --- -function UnaryConstraint:markUnsatisfied () - self.satisfied = false -end - -function UnaryConstraint:inputsKnown () - return true -end - -function UnaryConstraint:removeFromGraph () - if self.myOutput ~= nil then - self.myOutput:removeConstraint(self) - end - self.satisfied = false -end - --- --- S t a y C o n s t r a i n t --- - --- --- Variables that should, with some level of preference, stay the same. --- Planners may exploit the fact that instances, if satisfied, will not --- change their output during plan execution. This is called "stay --- optimization". --- - -local StayConstraint = class(UnaryConstraint) - -function StayConstraint:constructor(v, str) - StayConstraint.super.constructor(self, v, str) -end - -function StayConstraint:execute () - -- Stay constraints do nothing -end - --- --- E d i t C o n s t r a i n t --- - --- --- A unary input constraint used to mark a variable that the client --- wishes to change. --- - -local EditConstraint = class (UnaryConstraint) - -function EditConstraint:constructor(v, str) - EditConstraint.super.constructor(self, v, str) -end - --- --- Edits indicate that a variable is to be changed by imperative code. --- -function EditConstraint:isInput () - return true -end - -function EditConstraint:execute () - -- Edit constraints do nothing -end - --- --- B i n a r y C o n s t r a i n t --- - -local Direction = {} -Direction.NONE = 0 -Direction.FORWARD = 1 -Direction.BACKWARD = -1 - --- --- Abstract superclass for constraints having two possible output --- variables. --- - -local BinaryConstraint = class(Constraint) - -function BinaryConstraint:constructor(var1, var2, strength) - BinaryConstraint.super.constructor(self, strength); - self.v1 = var1 - self.v2 = var2 - self.direction = Direction.NONE - self:addConstraint() -end - - --- --- Decides if this constraint can be satisfied and which way it --- should flow based on the relative strength of the variables related, --- and record that decision. --- -function BinaryConstraint:chooseMethod (mark) - if self.v1.mark == mark then - self.direction = (self.v2.mark ~= mark and Strength.stronger(self.strength, self.v2.walkStrength)) and Direction.FORWARD or Direction.NONE - end - if self.v2.mark == mark then - self.direction = (self.v1.mark ~= mark and Strength.stronger(self.strength, self.v1.walkStrength)) and Direction.BACKWARD or Direction.NONE - end - if Strength.weaker(self.v1.walkStrength, self.v2.walkStrength) then - self.direction = Strength.stronger(self.strength, self.v1.walkStrength) and Direction.BACKWARD or Direction.NONE - else - self.direction = Strength.stronger(self.strength, self.v2.walkStrength) and Direction.FORWARD or Direction.BACKWARD - end -end - --- --- Add this constraint to the constraint graph --- -function BinaryConstraint:addToGraph () - self.v1:addConstraint(self) - self.v2:addConstraint(self) - self.direction = Direction.NONE -end - --- --- Answer true if this constraint is satisfied in the current solution. --- -function BinaryConstraint:isSatisfied () - return self.direction ~= Direction.NONE -end - --- --- Mark the input variable with the given mark. --- -function BinaryConstraint:markInputs (mark) - self:input().mark = mark -end - --- --- Returns the current input variable --- -function BinaryConstraint:input () - return (self.direction == Direction.FORWARD) and self.v1 or self.v2 -end - --- --- Returns the current output variable --- -function BinaryConstraint:output () - return (self.direction == Direction.FORWARD) and self.v2 or self.v1 -end - --- --- Calculate the walkabout strength, the stay flag, and, if it is --- 'stay', the value for the current output of this --- constraint. Assume this constraint is satisfied. --- -function BinaryConstraint:recalculate () - local ihn = self:input() - local out = self:output() - out.walkStrength = Strength.weakestOf(self.strength, ihn.walkStrength); - out.stay = ihn.stay - if out.stay then self:execute() end -end - --- --- Record the fact that self constraint is unsatisfied. --- -function BinaryConstraint:markUnsatisfied () - self.direction = Direction.NONE -end - -function BinaryConstraint:inputsKnown (mark) - local i = self:input() - return i.mark == mark or i.stay or i.determinedBy == nil -end - -function BinaryConstraint:removeFromGraph () - if (self.v1 ~= nil) then self.v1:removeConstraint(self) end - if (self.v2 ~= nil) then self.v2:removeConstraint(self) end - self.direction = Direction.NONE -end - --- --- S c a l e C o n s t r a i n t --- - --- --- Relates two variables by the linear scaling relationship: "v2 = --- (v1 * scale) + offset". Either v1 or v2 may be changed to maintain --- this relationship but the scale factor and offset are considered --- read-only. --- - -local ScaleConstraint = class (BinaryConstraint) - -function ScaleConstraint:constructor(src, scale, offset, dest, strength) - self.direction = Direction.NONE - self.scale = scale - self.offset = offset - ScaleConstraint.super.constructor(self, src, dest, strength) -end - - --- --- Adds this constraint to the constraint graph. --- -function ScaleConstraint:addToGraph () - ScaleConstraint.super.addToGraph(self) - self.scale:addConstraint(self) - self.offset:addConstraint(self) -end - -function ScaleConstraint:removeFromGraph () - ScaleConstraint.super.removeFromGraph(self) - if (self.scale ~= nil) then self.scale:removeConstraint(self) end - if (self.offset ~= nil) then self.offset:removeConstraint(self) end -end - -function ScaleConstraint:markInputs (mark) - ScaleConstraint.super.markInputs(self, mark); - self.offset.mark = mark - self.scale.mark = mark -end - --- --- Enforce this constraint. Assume that it is satisfied. --- -function ScaleConstraint:execute () - if self.direction == Direction.FORWARD then - self.v2.value = self.v1.value * self.scale.value + self.offset.value - else - self.v1.value = (self.v2.value - self.offset.value) / self.scale.value - end -end - --- --- Calculate the walkabout strength, the stay flag, and, if it is --- 'stay', the value for the current output of this constraint. Assume --- this constraint is satisfied. --- -function ScaleConstraint:recalculate () - local ihn = self:input() - local out = self:output() - out.walkStrength = Strength.weakestOf(self.strength, ihn.walkStrength) - out.stay = ihn.stay and self.scale.stay and self.offset.stay - if out.stay then self:execute() end -end - --- --- E q u a l i t y C o n s t r a i n t --- - --- --- Constrains two variables to have the same value. --- - -local EqualityConstraint = class (BinaryConstraint) - -function EqualityConstraint:constructor(var1, var2, strength) - EqualityConstraint.super.constructor(self, var1, var2, strength) -end - - --- --- Enforce this constraint. Assume that it is satisfied. --- -function EqualityConstraint:execute () - self:output().value = self:input().value -end - --- --- V a r i a b l e --- - --- --- A constrained variable. In addition to its value, it maintain the --- structure of the constraint graph, the current dataflow graph, and --- various parameters of interest to the DeltaBlue incremental --- constraint solver. --- -local Variable = class () - -function Variable:constructor(name, initialValue) - self.value = initialValue or 0 - self.constraints = OrderedCollection.new() - self.determinedBy = nil - self.mark = 0 - self.walkStrength = Strength.WEAKEST - self.stay = true - self.name = name -end - --- --- Add the given constraint to the set of all constraints that refer --- this variable. --- -function Variable:addConstraint (c) - self.constraints:add(c) -end - --- --- Removes all traces of c from this variable. --- -function Variable:removeConstraint (c) - self.constraints:remove(c) - if self.determinedBy == c then - self.determinedBy = nil - end -end - --- --- P l a n n e r --- - --- --- The DeltaBlue planner --- -local Planner = class() -function Planner:constructor() - self.currentMark = 0 -end - --- --- Attempt to satisfy the given constraint and, if successful, --- incrementally update the dataflow graph. Details: If satifying --- the constraint is successful, it may override a weaker constraint --- on its output. The algorithm attempts to resatisfy that --- constraint using some other method. This process is repeated --- until either a) it reaches a variable that was not previously --- determined by any constraint or b) it reaches a constraint that --- is too weak to be satisfied using any of its methods. The --- variables of constraints that have been processed are marked with --- a unique mark value so that we know where we've been. This allows --- the algorithm to avoid getting into an infinite loop even if the --- constraint graph has an inadvertent cycle. --- -function Planner:incrementalAdd (c) - local mark = self:newMark() - local overridden = c:satisfy(mark) - while overridden ~= nil do - overridden = overridden:satisfy(mark) - end -end - --- --- Entry point for retracting a constraint. Remove the given --- constraint and incrementally update the dataflow graph. --- Details: Retracting the given constraint may allow some currently --- unsatisfiable downstream constraint to be satisfied. We therefore collect --- a list of unsatisfied downstream constraints and attempt to --- satisfy each one in turn. This list is traversed by constraint --- strength, strongest first, as a heuristic for avoiding --- unnecessarily adding and then overriding weak constraints. --- Assume: c is satisfied. --- -function Planner:incrementalRemove (c) - local out = c:output() - c:markUnsatisfied() - c:removeFromGraph() - local unsatisfied = self:removePropagateFrom(out) - local strength = Strength.REQUIRED - repeat - for i = 1, unsatisfied:size() do - local u = unsatisfied:at(i) - if u.strength == strength then - self:incrementalAdd(u) - end - end - strength = strength:nextWeaker() - until strength == Strength.WEAKEST -end - --- --- Select a previously unused mark value. --- -function Planner:newMark () - self.currentMark = self.currentMark + 1 - return self.currentMark -end - --- --- Extract a plan for resatisfaction starting from the given source --- constraints, usually a set of input constraints. This method --- assumes that stay optimization is desired; the plan will contain --- only constraints whose output variables are not stay. Constraints --- that do no computation, such as stay and edit constraints, are --- not included in the plan. --- Details: The outputs of a constraint are marked when it is added --- to the plan under construction. A constraint may be appended to --- the plan when all its input variables are known. A variable is --- known if either a) the variable is marked (indicating that has --- been computed by a constraint appearing earlier in the plan), b) --- the variable is 'stay' (i.e. it is a constant at plan execution --- time), or c) the variable is not determined by any --- constraint. The last provision is for past states of history --- variables, which are not stay but which are also not computed by --- any constraint. --- Assume: sources are all satisfied. --- -local Plan -- FORWARD DECLARATION -function Planner:makePlan (sources) - local mark = self:newMark() - local plan = Plan.new() - local todo = sources - while todo:size() > 0 do - local c = todo:removeFirst() - if c:output().mark ~= mark and c:inputsKnown(mark) then - plan:addConstraint(c) - c:output().mark = mark - self:addConstraintsConsumingTo(c:output(), todo) - end - end - return plan -end - --- --- Extract a plan for resatisfying starting from the output of the --- given constraints, usually a set of input constraints. --- -function Planner:extractPlanFromConstraints (constraints) - local sources = OrderedCollection.new() - for i = 1, constraints:size() do - local c = constraints:at(i) - if c:isInput() and c:isSatisfied() then - -- not in plan already and eligible for inclusion - sources:add(c) - end - end - return self:makePlan(sources) -end - --- --- Recompute the walkabout strengths and stay flags of all variables --- downstream of the given constraint and recompute the actual --- values of all variables whose stay flag is true. If a cycle is --- detected, remove the given constraint and answer --- false. Otherwise, answer true. --- Details: Cycles are detected when a marked variable is --- encountered downstream of the given constraint. The sender is --- assumed to have marked the inputs of the given constraint with --- the given mark. Thus, encountering a marked node downstream of --- the output constraint means that there is a path from the --- constraint's output to one of its inputs. --- -function Planner:addPropagate (c, mark) - local todo = OrderedCollection.new() - todo:add(c) - while todo:size() > 0 do - local d = todo:removeFirst() - if d:output().mark == mark then - self:incrementalRemove(c) - return false - end - d:recalculate() - self:addConstraintsConsumingTo(d:output(), todo) - end - return true -end - - --- --- Update the walkabout strengths and stay flags of all variables --- downstream of the given constraint. Answer a collection of --- unsatisfied constraints sorted in order of decreasing strength. --- -function Planner:removePropagateFrom (out) - out.determinedBy = nil - out.walkStrength = Strength.WEAKEST - out.stay = true - local unsatisfied = OrderedCollection.new() - local todo = OrderedCollection.new() - todo:add(out) - while todo:size() > 0 do - local v = todo:removeFirst() - for i = 1, v.constraints:size() do - local c = v.constraints:at(i) - if not c:isSatisfied() then unsatisfied:add(c) end - end - local determining = v.determinedBy - for i = 1, v.constraints:size() do - local next = v.constraints:at(i); - if next ~= determining and next:isSatisfied() then - next:recalculate() - todo:add(next:output()) - end - end - end - return unsatisfied -end - -function Planner:addConstraintsConsumingTo (v, coll) - local determining = v.determinedBy - local cc = v.constraints - for i = 1, cc:size() do - local c = cc:at(i) - if c ~= determining and c:isSatisfied() then - coll:add(c) - end - end -end - --- --- P l a n --- - --- --- A Plan is an ordered list of constraints to be executed in sequence --- to resatisfy all currently satisfiable constraints in the face of --- one or more changing inputs. --- -Plan = class() -function Plan:constructor() - self.v = OrderedCollection.new() -end - -function Plan:addConstraint (c) - self.v:add(c) -end - -function Plan:size () - return self.v:size() -end - -function Plan:constraintAt (index) - return self.v:at(index) -end - -function Plan:execute () - for i = 1, self:size() do - local c = self:constraintAt(i) - c:execute() - end -end - --- --- M a i n --- - --- --- This is the standard DeltaBlue benchmark. A long chain of equality --- constraints is constructed with a stay constraint on one end. An --- edit constraint is then added to the opposite end and the time is --- measured for adding and removing this constraint, and extracting --- and executing a constraint satisfaction plan. There are two cases. --- In case 1, the added constraint is stronger than the stay --- constraint and values must propagate down the entire length of the --- chain. In case 2, the added constraint is weaker than the stay --- constraint so it cannot be accomodated. The cost in this case is, --- of course, very low. Typical situations lie somewhere between these --- two extremes. --- -local function chainTest(n) - planner = Planner.new() - local prev = nil - local first = nil - local last = nil - - -- Build chain of n equality constraints - for i = 0, n do - local name = "v" .. i; - local v = Variable.new(name) - if prev ~= nil then EqualityConstraint.new(prev, v, Strength.REQUIRED) end - if i == 0 then first = v end - if i == n then last = v end - prev = v - end - - StayConstraint.new(last, Strength.STRONG_DEFAULT) - local edit = EditConstraint.new(first, Strength.PREFERRED) - local edits = OrderedCollection.new() - edits:add(edit) - local plan = planner:extractPlanFromConstraints(edits) - for i = 0, 99 do - first.value = i - plan:execute() - if last.value ~= i then - alert("Chain test failed.") - end - end -end - -local function change(v, newValue) - local edit = EditConstraint.new(v, Strength.PREFERRED) - local edits = OrderedCollection.new() - edits:add(edit) - local plan = planner:extractPlanFromConstraints(edits) - for i = 1, 10 do - v.value = newValue - plan:execute() - end - edit:destroyConstraint() -end - --- --- This test constructs a two sets of variables related to each --- other by a simple linear transformation (scale and offset). The --- time is measured to change a variable on either side of the --- mapping and to change the scale and offset factors. --- -local function projectionTest(n) - planner = Planner.new(); - local scale = Variable.new("scale", 10); - local offset = Variable.new("offset", 1000); - local src = nil - local dst = nil; - - local dests = OrderedCollection.new(); - for i = 0, n - 1 do - src = Variable.new("src" .. i, i); - dst = Variable.new("dst" .. i, i); - dests:add(dst); - StayConstraint.new(src, Strength.NORMAL); - ScaleConstraint.new(src, scale, offset, dst, Strength.REQUIRED); - end - - change(src, 17) - if dst.value ~= 1170 then alert("Projection 1 failed") end - change(dst, 1050) - if src.value ~= 5 then alert("Projection 2 failed") end - change(scale, 5) - for i = 0, n - 2 do - if dests:at(i + 1).value ~= i * 5 + 1000 then - alert("Projection 3 failed") - end - end - change(offset, 2000) - for i = 0, n - 2 do - if dests:at(i + 1).value ~= i * 5 + 2000 then - alert("Projection 4 failed") - end - end -end - -function test() - local t0 = os.clock() - chainTest(1000); - projectionTest(1000); - local t1 = os.clock() - return t1-t0 -end - -bench.runCode(test, "deltablue") diff --git a/tests/Autocomplete.test.cpp b/tests/Autocomplete.test.cpp index 9cd642cf..07910a0a 100644 --- a/tests/Autocomplete.test.cpp +++ b/tests/Autocomplete.test.cpp @@ -23,19 +23,17 @@ static std::optional nullCallback(std::string tag, std::op return std::nullopt; } -struct ACFixture : Fixture +template +struct ACFixtureImpl : BaseType { AutocompleteResult autocomplete(unsigned row, unsigned column) { - return Luau::autocomplete(frontend, "MainModule", Position{row, column}, nullCallback); + return Luau::autocomplete(this->frontend, "MainModule", Position{row, column}, nullCallback); } AutocompleteResult autocomplete(char marker) { - auto i = markerPosition.find(marker); - LUAU_ASSERT(i != markerPosition.end()); - const Position& pos = i->second; - return Luau::autocomplete(frontend, "MainModule", pos, nullCallback); + return Luau::autocomplete(this->frontend, "MainModule", getPosition(marker), nullCallback); } CheckResult check(const std::string& source) @@ -45,16 +43,18 @@ struct ACFixture : Fixture filteredSource.reserve(source.size()); Position curPos(0, 0); + char prevChar{}; for (char c : source) { - if (c == '@' && !filteredSource.empty()) + if (prevChar == '@') { - char prevChar = filteredSource.back(); - filteredSource.pop_back(); - curPos.column--; // Adjust column position since we removed a character from the output - LUAU_ASSERT("Illegal marker character" && prevChar >= '0' && prevChar <= '9'); - LUAU_ASSERT("Duplicate marker found" && markerPosition.count(prevChar) == 0); - markerPosition.insert(std::pair{prevChar, curPos}); + LUAU_ASSERT("Illegal marker character" && c >= '0' && c <= '9'); + LUAU_ASSERT("Duplicate marker found" && markerPosition.count(c) == 0); + markerPosition.insert(std::pair{c, curPos}); + } + else if (c == '@') + { + // skip the '@' character } else { @@ -69,22 +69,39 @@ struct ACFixture : Fixture curPos.column++; } } + prevChar = c; } + LUAU_ASSERT("Digit expected after @ symbol" && prevChar != '@'); return Fixture::check(filteredSource); } + const Position& getPosition(char marker) const + { + auto i = markerPosition.find(marker); + LUAU_ASSERT(i != markerPosition.end()); + return i->second; + } + // Maps a marker character (0-9 inclusive) to a position in the source code. std::map markerPosition; }; +struct ACFixture : ACFixtureImpl +{ +}; + +struct UnfrozenACFixture : ACFixtureImpl +{ +}; + TEST_SUITE_BEGIN("AutocompleteTest"); TEST_CASE_FIXTURE(ACFixture, "empty_program") { - check(" "); + check(" @1"); - auto ac = autocomplete(0, 1); + auto ac = autocomplete('1'); CHECK(!ac.entryMap.empty()); CHECK(ac.entryMap.count("table")); @@ -93,26 +110,26 @@ TEST_CASE_FIXTURE(ACFixture, "empty_program") TEST_CASE_FIXTURE(ACFixture, "local_initializer") { - check("local a = "); + check("local a = @1"); - auto ac = autocomplete(0, 10); + auto ac = autocomplete('1'); CHECK(ac.entryMap.count("table")); CHECK(ac.entryMap.count("math")); } TEST_CASE_FIXTURE(ACFixture, "leave_numbers_alone") { - check("local a = 3.1"); + check("local a = 3.@11"); - auto ac = autocomplete(0, 12); + auto ac = autocomplete('1'); CHECK(ac.entryMap.empty()); } TEST_CASE_FIXTURE(ACFixture, "user_defined_globals") { - check("local myLocal = 4; "); + check("local myLocal = 4; @1"); - auto ac = autocomplete(0, 19); + auto ac = autocomplete('1'); CHECK(ac.entryMap.count("myLocal")); CHECK(ac.entryMap.count("table")); @@ -124,20 +141,20 @@ TEST_CASE_FIXTURE(ACFixture, "dont_suggest_local_before_its_definition") check(R"( local myLocal = 4 function abc() - local myInnerLocal = 1 - +@1 local myInnerLocal = 1 +@2 end - )"); +@3 )"); - auto ac = autocomplete(3, 0); + auto ac = autocomplete('1'); CHECK(ac.entryMap.count("myLocal")); CHECK(!ac.entryMap.count("myInnerLocal")); - ac = autocomplete(4, 0); + ac = autocomplete('2'); CHECK(ac.entryMap.count("myLocal")); CHECK(ac.entryMap.count("myInnerLocal")); - ac = autocomplete(6, 0); + ac = autocomplete('3'); CHECK(ac.entryMap.count("myLocal")); CHECK(!ac.entryMap.count("myInnerLocal")); } @@ -146,10 +163,10 @@ TEST_CASE_FIXTURE(ACFixture, "recursive_function") { check(R"( function foo() - end +@1 end )"); - auto ac = autocomplete(2, 0); + auto ac = autocomplete('1'); CHECK(ac.entryMap.count("foo")); } @@ -158,11 +175,11 @@ TEST_CASE_FIXTURE(ACFixture, "nested_recursive_function") check(R"( local function outer() local function inner() - end +@1 end end )"); - auto ac = autocomplete(3, 0); + auto ac = autocomplete('1'); CHECK(ac.entryMap.count("inner")); CHECK(ac.entryMap.count("outer")); } @@ -171,11 +188,11 @@ TEST_CASE_FIXTURE(ACFixture, "user_defined_local_functions_in_own_definition") { check(R"( local function abc() - +@1 end )"); - auto ac = autocomplete(2, 0); + auto ac = autocomplete('1'); CHECK(ac.entryMap.count("abc")); CHECK(ac.entryMap.count("table")); @@ -183,11 +200,11 @@ TEST_CASE_FIXTURE(ACFixture, "user_defined_local_functions_in_own_definition") check(R"( local abc = function() - +@1 end )"); - ac = autocomplete(2, 0); + ac = autocomplete('1'); CHECK(ac.entryMap.count("abc")); // FIXME: This is actually incorrect! CHECK(ac.entryMap.count("table")); @@ -202,9 +219,9 @@ TEST_CASE_FIXTURE(ACFixture, "global_functions_are_not_scoped_lexically") end end - )"); +@1 )"); - auto ac = autocomplete(6, 0); + auto ac = autocomplete('1'); CHECK(!ac.entryMap.empty()); CHECK(ac.entryMap.count("abc")); @@ -220,9 +237,9 @@ TEST_CASE_FIXTURE(ACFixture, "local_functions_fall_out_of_scope") end end - )"); +@1 )"); - auto ac = autocomplete(6, 0); + auto ac = autocomplete('1'); CHECK_NE(0, ac.entryMap.size()); CHECK(!ac.entryMap.count("abc")); @@ -233,10 +250,10 @@ TEST_CASE_FIXTURE(ACFixture, "function_parameters") check(R"( function abc(test) - end +@1 end )"); - auto ac = autocomplete(3, 0); + auto ac = autocomplete('1'); CHECK(ac.entryMap.count("test")); } @@ -244,11 +261,10 @@ TEST_CASE_FIXTURE(ACFixture, "function_parameters") TEST_CASE_FIXTURE(ACFixture, "get_member_completions") { check(R"( - local a = table. -- Line 1 - -- | Column 23 + local a = table.@1 )"); - auto ac = autocomplete(1, 24); + auto ac = autocomplete('1'); CHECK_EQ(16, ac.entryMap.size()); CHECK(ac.entryMap.count("find")); @@ -260,10 +276,10 @@ TEST_CASE_FIXTURE(ACFixture, "nested_member_completions") { check(R"( local tbl = { abc = { def = 1234, egh = false } } - tbl.abc. + tbl.abc. @1 )"); - auto ac = autocomplete(2, 17); + auto ac = autocomplete('1'); CHECK_EQ(2, ac.entryMap.size()); CHECK(ac.entryMap.count("def")); CHECK(ac.entryMap.count("egh")); @@ -274,10 +290,10 @@ TEST_CASE_FIXTURE(ACFixture, "unsealed_table") check(R"( local tbl = {} tbl.prop = 5 - tbl. + tbl.@1 )"); - auto ac = autocomplete(3, 12); + auto ac = autocomplete('1'); CHECK_EQ(1, ac.entryMap.size()); CHECK(ac.entryMap.count("prop")); } @@ -288,10 +304,10 @@ TEST_CASE_FIXTURE(ACFixture, "unsealed_table_2") local tbl = {} local inner = { prop = 5 } tbl.inner = inner - tbl.inner. + tbl.inner. @1 )"); - auto ac = autocomplete(4, 19); + auto ac = autocomplete('1'); CHECK_EQ(1, ac.entryMap.size()); CHECK(ac.entryMap.count("prop")); } @@ -302,10 +318,10 @@ TEST_CASE_FIXTURE(ACFixture, "cyclic_table") local abc = {} local def = { abc = abc } abc.def = def - abc.def. + abc.def. @1 )"); - auto ac = autocomplete(4, 17); + auto ac = autocomplete('1'); CHECK(ac.entryMap.count("abc")); } @@ -315,11 +331,11 @@ TEST_CASE_FIXTURE(ACFixture, "table_union") type t1 = { a1 : string, b2 : number } type t2 = { b2 : string, c3 : string } function func(abc : t1 | t2) - abc. + abc. @1 end )"); - auto ac = autocomplete(4, 18); + auto ac = autocomplete('1'); CHECK_EQ(1, ac.entryMap.size()); CHECK(ac.entryMap.count("b2")); } @@ -330,11 +346,11 @@ TEST_CASE_FIXTURE(ACFixture, "table_intersection") type t1 = { a1 : string, b2 : number } type t2 = { b2 : string, c3 : string } function func(abc : t1 & t2) - abc. + abc. @1 end )"); - auto ac = autocomplete(4, 18); + auto ac = autocomplete('1'); CHECK_EQ(3, ac.entryMap.size()); CHECK(ac.entryMap.count("a1")); CHECK(ac.entryMap.count("b2")); @@ -344,20 +360,19 @@ TEST_CASE_FIXTURE(ACFixture, "table_intersection") TEST_CASE_FIXTURE(ACFixture, "get_string_completions") { check(R"( - local a = ("foo"): -- Line 1 - -- | Column 26 + local a = ("foo"):@1 )"); - auto ac = autocomplete(1, 26); + auto ac = autocomplete('1'); CHECK_EQ(17, ac.entryMap.size()); } TEST_CASE_FIXTURE(ACFixture, "get_suggestions_for_new_statement") { - check(""); + check("@1"); - auto ac = autocomplete(0, 0); + auto ac = autocomplete('1'); CHECK_NE(0, ac.entryMap.size()); @@ -366,12 +381,12 @@ TEST_CASE_FIXTURE(ACFixture, "get_suggestions_for_new_statement") TEST_CASE_FIXTURE(ACFixture, "get_suggestions_for_the_very_start_of_the_script") { - check(R"( + check(R"(@1 function aaa() end )"); - auto ac = autocomplete(0, 0); + auto ac = autocomplete('1'); CHECK(ac.entryMap.count("table")); } @@ -382,11 +397,11 @@ TEST_CASE_FIXTURE(ACFixture, "method_call_inside_function_body") local game = { GetService=function(s) return 'hello' end } function a() - game: + game: @1 end )"); - auto ac = autocomplete(4, 19); + auto ac = autocomplete('1'); CHECK_NE(0, ac.entryMap.size()); @@ -396,10 +411,10 @@ TEST_CASE_FIXTURE(ACFixture, "method_call_inside_function_body") TEST_CASE_FIXTURE(ACFixture, "method_call_inside_if_conditional") { check(R"( - if table: + if table: @1 )"); - auto ac = autocomplete(1, 19); + auto ac = autocomplete('1'); CHECK_NE(0, ac.entryMap.size()); CHECK(ac.entryMap.count("concat")); @@ -411,12 +426,12 @@ TEST_CASE_FIXTURE(ACFixture, "statement_between_two_statements") check(R"( function getmyscripts() end - g + g@1 getmyscripts() )"); - auto ac = autocomplete(3, 9); + auto ac = autocomplete('1'); CHECK_NE(0, ac.entryMap.size()); @@ -431,11 +446,11 @@ TEST_CASE_FIXTURE(ACFixture, "bias_toward_inner_scope") function B() local A = {two=2} - A + A @1 end )"); - auto ac = autocomplete(6, 15); + auto ac = autocomplete('1'); CHECK(ac.entryMap.count("A")); @@ -448,12 +463,12 @@ TEST_CASE_FIXTURE(ACFixture, "bias_toward_inner_scope") TEST_CASE_FIXTURE(ACFixture, "recommend_statement_starting_keywords") { - check(""); - auto ac = autocomplete(0, 0); + check("@1"); + auto ac = autocomplete('1'); CHECK(ac.entryMap.count("local")); - check("local i = "); - auto ac2 = autocomplete(0, 10); + check("local i = @1"); + auto ac2 = autocomplete('1'); CHECK(!ac2.entryMap.count("local")); } @@ -464,9 +479,9 @@ TEST_CASE_FIXTURE(ACFixture, "do_not_overwrite_context_sensitive_kws") end - )"); +@1 )"); - auto ac = autocomplete(5, 0); + auto ac = autocomplete('1'); AutocompleteEntry entry = ac.entryMap["continue"]; CHECK(entry.kind == AutocompleteEntryKind::Binding); @@ -480,11 +495,11 @@ TEST_CASE_FIXTURE(ACFixture, "dont_offer_any_suggestions_from_within_a_comment") function foo:bar() end --[[ - foo: + foo:@1 ]] )"); - auto ac = autocomplete(6, 16); + auto ac = autocomplete('1'); CHECK_EQ(0, ac.entryMap.size()); } @@ -492,10 +507,10 @@ TEST_CASE_FIXTURE(ACFixture, "dont_offer_any_suggestions_from_within_a_comment") TEST_CASE_FIXTURE(ACFixture, "dont_offer_any_suggestions_from_the_end_of_a_comment") { check(R"( - --!strict + --!strict@1 )"); - auto ac = autocomplete(1, 17); + auto ac = autocomplete('1'); CHECK_EQ(0, ac.entryMap.size()); } @@ -505,10 +520,10 @@ TEST_CASE_FIXTURE(ACFixture, "dont_offer_any_suggestions_from_within_a_broken_co ScopedFastFlag sff{"LuauCaptureBrokenCommentSpans", true}; check(R"( - --[[ + --[[ @1 )"); - auto ac = autocomplete(1, 13); + auto ac = autocomplete('1'); CHECK_EQ(0, ac.entryMap.size()); } @@ -517,129 +532,129 @@ TEST_CASE_FIXTURE(ACFixture, "dont_offer_any_suggestions_from_within_a_broken_co { ScopedFastFlag sff{"LuauCaptureBrokenCommentSpans", true}; - check("--[["); + check("--[[@1"); - auto ac = autocomplete(0, 4); + auto ac = autocomplete('1'); CHECK_EQ(0, ac.entryMap.size()); } TEST_CASE_FIXTURE(ACFixture, "autocomplete_for_middle_keywords") { check(R"( - for x = + for x @1= )"); - auto ac1 = autocomplete(1, 14); + auto ac1 = autocomplete('1'); CHECK_EQ(ac1.entryMap.count("do"), 0); CHECK_EQ(ac1.entryMap.count("end"), 0); check(R"( - for x = 1 + for x =@1 1 )"); - auto ac2 = autocomplete(1, 15); + auto ac2 = autocomplete('1'); CHECK_EQ(ac2.entryMap.count("do"), 0); CHECK_EQ(ac2.entryMap.count("end"), 0); check(R"( - for x = 1, 2 + for x = 1,@1 2 )"); - auto ac3 = autocomplete(1, 18); + auto ac3 = autocomplete('1'); CHECK_EQ(1, ac3.entryMap.size()); CHECK_EQ(ac3.entryMap.count("do"), 1); check(R"( - for x = 1, 2, + for x = 1, @12, )"); - auto ac4 = autocomplete(1, 19); + auto ac4 = autocomplete('1'); CHECK_EQ(ac4.entryMap.count("do"), 0); CHECK_EQ(ac4.entryMap.count("end"), 0); check(R"( - for x = 1, 2, 5 + for x = 1, 2, @15 )"); - auto ac5 = autocomplete(1, 22); + auto ac5 = autocomplete('1'); CHECK_EQ(ac5.entryMap.count("do"), 1); CHECK_EQ(ac5.entryMap.count("end"), 0); check(R"( - for x = 1, 2, 5 f + for x = 1, 2, 5 f@1 )"); - auto ac6 = autocomplete(1, 25); + auto ac6 = autocomplete('1'); CHECK_EQ(ac6.entryMap.size(), 1); CHECK_EQ(ac6.entryMap.count("do"), 1); check(R"( - for x = 1, 2, 5 do + for x = 1, 2, 5 do @1 )"); - auto ac7 = autocomplete(1, 32); + auto ac7 = autocomplete('1'); CHECK_EQ(ac7.entryMap.count("end"), 1); } TEST_CASE_FIXTURE(ACFixture, "autocomplete_for_in_middle_keywords") { check(R"( - for + for @1 )"); - auto ac1 = autocomplete(1, 12); + auto ac1 = autocomplete('1'); CHECK_EQ(0, ac1.entryMap.size()); check(R"( - for x + for x@1 @2 )"); - auto ac2 = autocomplete(1, 13); + auto ac2 = autocomplete('1'); CHECK_EQ(0, ac2.entryMap.size()); - auto ac2a = autocomplete(1, 14); + auto ac2a = autocomplete('2'); CHECK_EQ(1, ac2a.entryMap.size()); CHECK_EQ(1, ac2a.entryMap.count("in")); check(R"( - for x in y + for x in y@1 )"); - auto ac3 = autocomplete(1, 18); + auto ac3 = autocomplete('1'); CHECK_EQ(ac3.entryMap.count("table"), 1); CHECK_EQ(ac3.entryMap.count("do"), 0); check(R"( - for x in y + for x in y @1 )"); - auto ac4 = autocomplete(1, 19); + auto ac4 = autocomplete('1'); CHECK_EQ(ac4.entryMap.size(), 1); CHECK_EQ(ac4.entryMap.count("do"), 1); check(R"( - for x in f f + for x in f f@1 )"); - auto ac5 = autocomplete(1, 20); + auto ac5 = autocomplete('1'); CHECK_EQ(ac5.entryMap.size(), 1); CHECK_EQ(ac5.entryMap.count("do"), 1); check(R"( - for x in y do + for x in y do @1 )"); - auto ac6 = autocomplete(1, 23); + auto ac6 = autocomplete('1'); CHECK_EQ(ac6.entryMap.count("in"), 0); CHECK_EQ(ac6.entryMap.count("table"), 1); CHECK_EQ(ac6.entryMap.count("end"), 1); CHECK_EQ(ac6.entryMap.count("function"), 1); check(R"( - for x in y do e + for x in y do e@1 )"); - auto ac7 = autocomplete(1, 23); + auto ac7 = autocomplete('1'); CHECK_EQ(ac7.entryMap.count("in"), 0); CHECK_EQ(ac7.entryMap.count("table"), 1); CHECK_EQ(ac7.entryMap.count("end"), 1); @@ -649,33 +664,33 @@ TEST_CASE_FIXTURE(ACFixture, "autocomplete_for_in_middle_keywords") TEST_CASE_FIXTURE(ACFixture, "autocomplete_while_middle_keywords") { check(R"( - while + while@1 )"); - auto ac1 = autocomplete(1, 13); + auto ac1 = autocomplete('1'); CHECK_EQ(ac1.entryMap.count("do"), 0); CHECK_EQ(ac1.entryMap.count("end"), 0); check(R"( - while true + while true @1 )"); - auto ac2 = autocomplete(1, 19); + auto ac2 = autocomplete('1'); CHECK_EQ(1, ac2.entryMap.size()); CHECK_EQ(ac2.entryMap.count("do"), 1); check(R"( - while true do + while true do @1 )"); - auto ac3 = autocomplete(1, 23); + auto ac3 = autocomplete('1'); CHECK_EQ(ac3.entryMap.count("end"), 1); check(R"( - while true d + while true d@1 )"); - auto ac4 = autocomplete(1, 20); + auto ac4 = autocomplete('1'); CHECK_EQ(1, ac4.entryMap.size()); CHECK_EQ(ac4.entryMap.count("do"), 1); } @@ -683,10 +698,10 @@ TEST_CASE_FIXTURE(ACFixture, "autocomplete_while_middle_keywords") TEST_CASE_FIXTURE(ACFixture, "autocomplete_if_middle_keywords") { check(R"( - if + if @1 )"); - auto ac1 = autocomplete(1, 13); + auto ac1 = autocomplete('1'); CHECK_EQ(ac1.entryMap.count("then"), 0); CHECK_EQ(ac1.entryMap.count("function"), 1); // FIXME: This is kind of dumb. It is technically syntactically valid but you can never do anything interesting with this. @@ -696,10 +711,10 @@ TEST_CASE_FIXTURE(ACFixture, "autocomplete_if_middle_keywords") CHECK_EQ(ac1.entryMap.count("end"), 0); check(R"( - if x + if x @1 )"); - auto ac2 = autocomplete(1, 14); + auto ac2 = autocomplete('1'); CHECK_EQ(ac2.entryMap.count("then"), 1); CHECK_EQ(ac2.entryMap.count("function"), 0); CHECK_EQ(ac2.entryMap.count("else"), 0); @@ -707,20 +722,20 @@ TEST_CASE_FIXTURE(ACFixture, "autocomplete_if_middle_keywords") CHECK_EQ(ac2.entryMap.count("end"), 0); check(R"( - if x t + if x t@1 )"); - auto ac3 = autocomplete(1, 14); + auto ac3 = autocomplete('1'); CHECK_EQ(1, ac3.entryMap.size()); CHECK_EQ(ac3.entryMap.count("then"), 1); check(R"( if x then - +@1 end )"); - auto ac4 = autocomplete(2, 0); + auto ac4 = autocomplete('1'); CHECK_EQ(ac4.entryMap.count("then"), 0); CHECK_EQ(ac4.entryMap.count("else"), 1); CHECK_EQ(ac4.entryMap.count("function"), 1); @@ -729,11 +744,11 @@ TEST_CASE_FIXTURE(ACFixture, "autocomplete_if_middle_keywords") check(R"( if x then - t + t@1 end )"); - auto ac4a = autocomplete(2, 13); + auto ac4a = autocomplete('1'); CHECK_EQ(ac4a.entryMap.count("then"), 0); CHECK_EQ(ac4a.entryMap.count("table"), 1); CHECK_EQ(ac4a.entryMap.count("else"), 1); @@ -741,12 +756,12 @@ TEST_CASE_FIXTURE(ACFixture, "autocomplete_if_middle_keywords") check(R"( if x then - +@1 elseif x then end )"); - auto ac5 = autocomplete(2, 0); + auto ac5 = autocomplete('1'); CHECK_EQ(ac5.entryMap.count("then"), 0); CHECK_EQ(ac5.entryMap.count("function"), 1); CHECK_EQ(ac5.entryMap.count("else"), 0); @@ -757,10 +772,10 @@ TEST_CASE_FIXTURE(ACFixture, "autocomplete_if_middle_keywords") TEST_CASE_FIXTURE(ACFixture, "autocomplete_until_in_repeat") { check(R"( - repeat + repeat @1 )"); - auto ac = autocomplete(1, 16); + auto ac = autocomplete('1'); CHECK_EQ(ac.entryMap.count("table"), 1); CHECK_EQ(ac.entryMap.count("until"), 1); } @@ -769,48 +784,48 @@ TEST_CASE_FIXTURE(ACFixture, "autocomplete_until_expression") { check(R"( repeat - until + until @1 )"); - auto ac = autocomplete(2, 16); + auto ac = autocomplete('1'); CHECK_EQ(ac.entryMap.count("table"), 1); } TEST_CASE_FIXTURE(ACFixture, "local_names") { check(R"( - local ab + local ab@1 )"); - auto ac1 = autocomplete(1, 16); + auto ac1 = autocomplete('1'); CHECK_EQ(ac1.entryMap.size(), 1); CHECK_EQ(ac1.entryMap.count("function"), 1); check(R"( - local ab, cd + local ab, cd@1 )"); - auto ac2 = autocomplete(1, 20); + auto ac2 = autocomplete('1'); CHECK(ac2.entryMap.empty()); } TEST_CASE_FIXTURE(ACFixture, "autocomplete_end_with_fn_exprs") { check(R"( - local function f() + local function f() @1 )"); - auto ac = autocomplete(1, 28); + auto ac = autocomplete('1'); CHECK_EQ(ac.entryMap.count("end"), 1); } TEST_CASE_FIXTURE(ACFixture, "autocomplete_end_with_lambda") { check(R"( - local a = function() local bar = foo en + local a = function() local bar = foo en@1 )"); - auto ac = autocomplete(1, 47); + auto ac = autocomplete('1'); CHECK_EQ(ac.entryMap.count("end"), 1); } @@ -818,10 +833,10 @@ TEST_CASE_FIXTURE(ACFixture, "stop_at_first_stat_when_recommending_keywords") { check(R"( repeat - for x + for x @1 )"); - auto ac1 = autocomplete(2, 18); + auto ac1 = autocomplete('1'); CHECK_EQ(ac1.entryMap.count("in"), 1); CHECK_EQ(ac1.entryMap.count("until"), 0); } @@ -829,112 +844,112 @@ TEST_CASE_FIXTURE(ACFixture, "stop_at_first_stat_when_recommending_keywords") TEST_CASE_FIXTURE(ACFixture, "autocomplete_repeat_middle_keyword") { check(R"( - repeat + repeat @1 )"); - auto ac1 = autocomplete(1, 15); + auto ac1 = autocomplete('1'); CHECK_EQ(ac1.entryMap.count("do"), 1); CHECK_EQ(ac1.entryMap.count("function"), 1); CHECK_EQ(ac1.entryMap.count("until"), 1); check(R"( - repeat f f + repeat f f@1 )"); - auto ac2 = autocomplete(1, 18); + auto ac2 = autocomplete('1'); CHECK_EQ(ac2.entryMap.count("function"), 1); CHECK_EQ(ac2.entryMap.count("until"), 1); check(R"( repeat - u + u@1 until )"); - auto ac3 = autocomplete(2, 13); + auto ac3 = autocomplete('1'); CHECK_EQ(ac3.entryMap.count("until"), 0); } TEST_CASE_FIXTURE(ACFixture, "local_function") { check(R"( - local f + local f@1 )"); - auto ac1 = autocomplete(1, 15); + auto ac1 = autocomplete('1'); CHECK_EQ(ac1.entryMap.size(), 1); CHECK_EQ(ac1.entryMap.count("function"), 1); check(R"( - local f, cd + local f@1, cd )"); - auto ac2 = autocomplete(1, 15); + auto ac2 = autocomplete('1'); CHECK(ac2.entryMap.empty()); } TEST_CASE_FIXTURE(ACFixture, "local_function") { check(R"( - local function + local function @1 )"); - auto ac = autocomplete(1, 23); + auto ac = autocomplete('1'); CHECK(ac.entryMap.empty()); check(R"( - local function s + local function @1s@2 )"); - ac = autocomplete(1, 23); + ac = autocomplete('1'); CHECK(ac.entryMap.empty()); - ac = autocomplete(1, 24); + ac = autocomplete('2'); CHECK(ac.entryMap.empty()); check(R"( - local function () + local function @1()@2 )"); - ac = autocomplete(1, 23); + ac = autocomplete('1'); CHECK(ac.entryMap.empty()); - ac = autocomplete(1, 25); + ac = autocomplete('2'); CHECK(ac.entryMap.count("end")); check(R"( - local function something + local function something@1 )"); - ac = autocomplete(1, 32); + ac = autocomplete('1'); CHECK(ac.entryMap.empty()); check(R"( local tbl = {} - function tbl.something() end + function tbl.something@1() end )"); - ac = autocomplete(2, 30); + ac = autocomplete('1'); CHECK(ac.entryMap.empty()); } TEST_CASE_FIXTURE(ACFixture, "local_function_params") { check(R"( - local function abc(def) + local function @1a@2bc(@3d@4ef)@5 @6 )"); - CHECK(autocomplete(1, 23).entryMap.empty()); - CHECK(autocomplete(1, 24).entryMap.empty()); - CHECK(autocomplete(1, 27).entryMap.empty()); - CHECK(autocomplete(1, 28).entryMap.empty()); - CHECK(!autocomplete(1, 31).entryMap.empty()); + CHECK(autocomplete('1').entryMap.empty()); + CHECK(autocomplete('2').entryMap.empty()); + CHECK(autocomplete('3').entryMap.empty()); + CHECK(autocomplete('4').entryMap.empty()); + CHECK(!autocomplete('5').entryMap.empty()); - CHECK(!autocomplete(1, 32).entryMap.empty()); + CHECK(!autocomplete('6').entryMap.empty()); check(R"( local function abc(def) - end +@1 end )"); for (unsigned int i = 23; i < 31; ++i) @@ -943,16 +958,16 @@ TEST_CASE_FIXTURE(ACFixture, "local_function_params") } CHECK(!autocomplete(1, 32).entryMap.empty()); - auto ac2 = autocomplete(2, 0); + auto ac2 = autocomplete('1'); CHECK_EQ(ac2.entryMap.count("abc"), 1); CHECK_EQ(ac2.entryMap.count("def"), 1); check(R"( - local function abc(def, ghi) + local function abc(def, ghi@1) end )"); - auto ac3 = autocomplete(1, 35); + auto ac3 = autocomplete('1'); CHECK(ac3.entryMap.empty()); } @@ -981,48 +996,48 @@ TEST_CASE_FIXTURE(ACFixture, "global_function_params") check(R"( function abc(def) - +@1 end )"); - auto ac2 = autocomplete(2, 0); + auto ac2 = autocomplete('1'); CHECK_EQ(ac2.entryMap.count("abc"), 1); CHECK_EQ(ac2.entryMap.count("def"), 1); check(R"( - function abc(def, ghi) + function abc(def, ghi@1) end )"); - auto ac3 = autocomplete(1, 29); + auto ac3 = autocomplete('1'); CHECK(ac3.entryMap.empty()); } TEST_CASE_FIXTURE(ACFixture, "arguments_to_global_lambda") { check(R"( - abc = function(def, ghi) + abc = function(def, ghi@1) end )"); - auto ac = autocomplete(1, 31); + auto ac = autocomplete('1'); CHECK(ac.entryMap.empty()); } TEST_CASE_FIXTURE(ACFixture, "function_expr_params") { check(R"( - abc = function(def) + abc = function(def) @1 )"); for (unsigned int i = 20; i < 27; ++i) { CHECK(autocomplete(1, i).entryMap.empty()); } - CHECK(!autocomplete(1, 28).entryMap.empty()); + CHECK(!autocomplete('1').entryMap.empty()); check(R"( - abc = function(def) + abc = function(def) @1 end )"); @@ -1030,25 +1045,25 @@ TEST_CASE_FIXTURE(ACFixture, "function_expr_params") { CHECK(autocomplete(1, i).entryMap.empty()); } - CHECK(!autocomplete(1, 28).entryMap.empty()); + CHECK(!autocomplete('1').entryMap.empty()); check(R"( abc = function(def) - +@1 end )"); - auto ac2 = autocomplete(2, 0); + auto ac2 = autocomplete('1'); CHECK_EQ(ac2.entryMap.count("def"), 1); } TEST_CASE_FIXTURE(ACFixture, "local_initializer") { check(R"( - local a = t + local a = t@1 )"); - auto ac = autocomplete(1, 19); + auto ac = autocomplete('1'); CHECK_EQ(ac.entryMap.count("table"), 1); CHECK_EQ(ac.entryMap.count("true"), 1); } @@ -1056,20 +1071,20 @@ TEST_CASE_FIXTURE(ACFixture, "local_initializer") TEST_CASE_FIXTURE(ACFixture, "local_initializer_2") { check(R"( - local a= + local a=@1 )"); - auto ac = autocomplete(1, 16); + auto ac = autocomplete('1'); CHECK(ac.entryMap.count("table")); } TEST_CASE_FIXTURE(ACFixture, "get_member_completions") { check(R"( - local a = 12.3 + local a = 12.@13 )"); - auto ac = autocomplete(1, 21); + auto ac = autocomplete('1'); CHECK(ac.entryMap.empty()); } @@ -1083,21 +1098,21 @@ TEST_CASE_FIXTURE(ACFixture, "sometimes_the_metatable_is_an_error") return setmetatable({x=6}, X) -- oops! end local t = T.new() - t. + t. @1 )"); - autocomplete(8, 12); + autocomplete('1'); // Don't crash! } TEST_CASE_FIXTURE(ACFixture, "local_types_builtin") { check(R"( -local a: n +local a: n@1 local b: string = "don't trip" )"); - auto ac = autocomplete(1, 10); + auto ac = autocomplete('1'); CHECK(ac.entryMap.count("nil")); CHECK(ac.entryMap.count("number")); @@ -1108,23 +1123,23 @@ TEST_CASE_FIXTURE(ACFixture, "private_types") check(R"( do type num = number - local a: nu - local b: num + local a: n@1u + local b: nu@2m end -local a: nu +local a: nu@3 )"); - auto ac = autocomplete(3, 14); + auto ac = autocomplete('1'); CHECK(ac.entryMap.count("num")); CHECK(ac.entryMap.count("number")); - ac = autocomplete(4, 15); + ac = autocomplete('2'); CHECK(ac.entryMap.count("num")); CHECK(ac.entryMap.count("number")); - ac = autocomplete(6, 11); + ac = autocomplete('3'); CHECK(!ac.entryMap.count("num")); CHECK(ac.entryMap.count("number")); @@ -1136,11 +1151,11 @@ TEST_CASE_FIXTURE(ACFixture, "type_scoping_easy") type Table = { a: number, b: number } do type Table = { x: string, y: string } - local a: T + local a: T@1 end )"); - auto ac = autocomplete(4, 14); + auto ac = autocomplete('1'); REQUIRE(ac.entryMap.count("Table")); REQUIRE(ac.entryMap["Table"].type); @@ -1198,11 +1213,11 @@ local a: aaa. TEST_CASE_FIXTURE(ACFixture, "argument_types") { check(R"( -local function f(a: n +local function f(a: n@1 local b: string = "don't trip" )"); - auto ac = autocomplete(1, 21); + auto ac = autocomplete('1'); CHECK(ac.entryMap.count("nil")); CHECK(ac.entryMap.count("number")); @@ -1211,11 +1226,11 @@ local b: string = "don't trip" TEST_CASE_FIXTURE(ACFixture, "return_types") { check(R"( -local function f(a: number): n +local function f(a: number): n@1 local b: string = "don't trip" )"); - auto ac = autocomplete(1, 30); + auto ac = autocomplete('1'); CHECK(ac.entryMap.count("nil")); CHECK(ac.entryMap.count("number")); @@ -1225,10 +1240,10 @@ TEST_CASE_FIXTURE(ACFixture, "as_types") { check(R"( local a: any = 5 -local b: number = (a :: n +local b: number = (a :: n@1 )"); - auto ac = autocomplete(2, 25); + auto ac = autocomplete('1'); CHECK(ac.entryMap.count("nil")); CHECK(ac.entryMap.count("number")); @@ -1237,34 +1252,34 @@ local b: number = (a :: n TEST_CASE_FIXTURE(ACFixture, "function_type_types") { check(R"( -local a: (n -local b: (number, (n -local c: (number, (number) -> n -local d: (number, (number) -> (number, n -local e: (n: n +local a: (n@1 +local b: (number, (n@2 +local c: (number, (number) -> n@3 +local d: (number, (number) -> (number, n@4 +local e: (n: n@5 )"); - auto ac = autocomplete(1, 11); + auto ac = autocomplete('1'); CHECK(ac.entryMap.count("nil")); CHECK(ac.entryMap.count("number")); - ac = autocomplete(2, 20); + ac = autocomplete('2'); CHECK(ac.entryMap.count("nil")); CHECK(ac.entryMap.count("number")); - ac = autocomplete(3, 31); + ac = autocomplete('3'); CHECK(ac.entryMap.count("nil")); CHECK(ac.entryMap.count("number")); - ac = autocomplete(4, 40); + ac = autocomplete('4'); CHECK(ac.entryMap.count("nil")); CHECK(ac.entryMap.count("number")); - ac = autocomplete(5, 14); + ac = autocomplete('5'); CHECK(ac.entryMap.count("nil")); CHECK(ac.entryMap.count("number")); @@ -1276,11 +1291,11 @@ TEST_CASE_FIXTURE(ACFixture, "generic_types") ScopedFastFlag luauGenericFunctions("LuauGenericFunctions", true); check(R"( -function f(a: T +function f(a: T@1 local b: string = "don't trip" )"); - auto ac = autocomplete(1, 25); + auto ac = autocomplete('1'); CHECK(ac.entryMap.count("Tee")); } @@ -1293,10 +1308,10 @@ local function target(a: number, b: string) return a + #b end local one = 4 local two = "hello" -return target(o +return target(o@1 )"); - auto ac = autocomplete(5, 15); + auto ac = autocomplete('1'); CHECK(ac.entryMap.count("one")); CHECK(ac.entryMap["one"].typeCorrect == TypeCorrectKind::Correct); @@ -1307,10 +1322,10 @@ local function target(a: number, b: string) return a + #b end local one = 4 local two = "hello" -return target(one, t +return target(one, t@1 )"); - ac = autocomplete(5, 20); + ac = autocomplete('1'); CHECK(ac.entryMap.count("two")); CHECK(ac.entryMap["two"].typeCorrect == TypeCorrectKind::Correct); @@ -1321,10 +1336,10 @@ return target(one, t local function target(a: number, b: string) return a + #b end local a = { one = 4, two = "hello" } -return target(a. +return target(a.@1 )"); - ac = autocomplete(4, 16); + ac = autocomplete('1'); CHECK(ac.entryMap.count("one")); CHECK(ac.entryMap["one"].typeCorrect == TypeCorrectKind::Correct); @@ -1334,10 +1349,10 @@ return target(a. local function target(a: number, b: string) return a + #b end local a = { one = 4, two = "hello" } -return target(a.one, a. +return target(a.one, a.@1 )"); - ac = autocomplete(4, 23); + ac = autocomplete('1'); CHECK(ac.entryMap.count("two")); CHECK(ac.entryMap["two"].typeCorrect == TypeCorrectKind::Correct); @@ -1348,10 +1363,10 @@ return target(a.one, a. local function target(a: string?) return #b end local a = { one = 4, two = "hello" } -return target(a. +return target(a.@1 )"); - ac = autocomplete(4, 16); + ac = autocomplete('1'); CHECK(ac.entryMap.count("two")); CHECK(ac.entryMap["two"].typeCorrect == TypeCorrectKind::Correct); @@ -1363,10 +1378,10 @@ TEST_CASE_FIXTURE(ACFixture, "type_correct_suggestion_in_table") check(R"( type Foo = { a: number, b: string } local a = { one = 4, two = "hello" } -local b: Foo = { a = a. +local b: Foo = { a = a.@1 )"); - auto ac = autocomplete(3, 23); + auto ac = autocomplete('1'); CHECK(ac.entryMap.count("one")); CHECK(ac.entryMap["one"].typeCorrect == TypeCorrectKind::Correct); @@ -1375,10 +1390,10 @@ local b: Foo = { a = a. check(R"( type Foo = { a: number, b: string } local a = { one = 4, two = "hello" } -local b: Foo = { b = a. +local b: Foo = { b = a.@1 )"); - ac = autocomplete(3, 23); + ac = autocomplete('1'); CHECK(ac.entryMap.count("two")); CHECK(ac.entryMap["two"].typeCorrect == TypeCorrectKind::Correct); @@ -1392,10 +1407,10 @@ local function target(a: number, b: string) return a + #b end local function bar1(a: number) return -a end local function bar2(a: string) reutrn a .. 'x' end -return target(b +return target(b@1 )"); - auto ac = autocomplete(5, 15); + auto ac = autocomplete('1'); CHECK(ac.entryMap.count("bar1")); CHECK(ac.entryMap["bar1"].typeCorrect == TypeCorrectKind::CorrectFunctionResult); @@ -1406,10 +1421,10 @@ local function target(a: number, b: string) return a + #b end local function bar1(a: number) return -a end local function bar2(a: string) return a .. 'x' end -return target(bar1, b +return target(bar1, b@1 )"); - ac = autocomplete(5, 21); + ac = autocomplete('1'); CHECK(ac.entryMap.count("bar2")); CHECK(ac.entryMap["bar2"].typeCorrect == TypeCorrectKind::CorrectFunctionResult); @@ -1420,10 +1435,10 @@ local function target(a: number, b: string) return a + #b end local function bar1(a: number): (...number) return -a, a end local function bar2(a: string) reutrn a .. 'x' end -return target(b +return target(b@1 )"); - ac = autocomplete(5, 15); + ac = autocomplete('1'); CHECK(ac.entryMap.count("bar1")); CHECK(ac.entryMap["bar1"].typeCorrect == TypeCorrectKind::CorrectFunctionResult); @@ -1433,69 +1448,69 @@ return target(b TEST_CASE_FIXTURE(ACFixture, "type_correct_local_type_suggestion") { check(R"( -local b: s = "str" +local b: s@1 = "str" )"); - auto ac = autocomplete(1, 10); + auto ac = autocomplete('1'); CHECK(ac.entryMap.count("string")); CHECK(ac.entryMap["string"].typeCorrect == TypeCorrectKind::Correct); check(R"( local function f() return "str" end -local b: s = f() +local b: s@1 = f() )"); - ac = autocomplete(2, 10); + ac = autocomplete('1'); CHECK(ac.entryMap.count("string")); CHECK(ac.entryMap["string"].typeCorrect == TypeCorrectKind::Correct); check(R"( -local b: s, c: n = "str", 2 +local b: s@1, c: n@2 = "str", 2 )"); - ac = autocomplete(1, 10); + ac = autocomplete('1'); CHECK(ac.entryMap.count("string")); CHECK(ac.entryMap["string"].typeCorrect == TypeCorrectKind::Correct); - ac = autocomplete(1, 16); + ac = autocomplete('2'); CHECK(ac.entryMap.count("number")); CHECK(ac.entryMap["number"].typeCorrect == TypeCorrectKind::Correct); check(R"( local function f() return 1, "str", 3 end -local a: b, b: n, c: s, d: n = false, f() +local a: b@1, b: n@2, c: s@3, d: n@4 = false, f() )"); - ac = autocomplete(2, 10); + ac = autocomplete('1'); CHECK(ac.entryMap.count("boolean")); CHECK(ac.entryMap["boolean"].typeCorrect == TypeCorrectKind::Correct); - ac = autocomplete(2, 16); + ac = autocomplete('2'); CHECK(ac.entryMap.count("number")); CHECK(ac.entryMap["number"].typeCorrect == TypeCorrectKind::Correct); - ac = autocomplete(2, 22); + ac = autocomplete('3'); CHECK(ac.entryMap.count("string")); CHECK(ac.entryMap["string"].typeCorrect == TypeCorrectKind::Correct); - ac = autocomplete(2, 28); + ac = autocomplete('4'); CHECK(ac.entryMap.count("number")); CHECK(ac.entryMap["number"].typeCorrect == TypeCorrectKind::Correct); check(R"( local function f(): ...number return 1, 2, 3 end -local a: boolean, b: n = false, f() +local a: boolean, b: n@1 = false, f() )"); - ac = autocomplete(2, 22); + ac = autocomplete('1'); CHECK(ac.entryMap.count("number")); CHECK(ac.entryMap["number"].typeCorrect == TypeCorrectKind::Correct); @@ -1504,46 +1519,46 @@ local a: boolean, b: n = false, f() TEST_CASE_FIXTURE(ACFixture, "type_correct_function_type_suggestion") { check(R"( -local b: (n) -> number = function(a: number, b: string) return a + #b end +local b: (n@1) -> number = function(a: number, b: string) return a + #b end )"); - auto ac = autocomplete(1, 11); + auto ac = autocomplete('1'); CHECK(ac.entryMap.count("number")); CHECK(ac.entryMap["number"].typeCorrect == TypeCorrectKind::Correct); check(R"( -local b: (number, s = function(a: number, b: string) return a + #b end +local b: (number, s@1 = function(a: number, b: string) return a + #b end )"); - ac = autocomplete(1, 19); + ac = autocomplete('1'); CHECK(ac.entryMap.count("string")); CHECK(ac.entryMap["string"].typeCorrect == TypeCorrectKind::Correct); check(R"( -local b: (number, string) -> b = function(a: number, b: string): boolean return a + #b == 0 end +local b: (number, string) -> b@1 = function(a: number, b: string): boolean return a + #b == 0 end )"); - ac = autocomplete(1, 30); + ac = autocomplete('1'); CHECK(ac.entryMap.count("boolean")); CHECK(ac.entryMap["boolean"].typeCorrect == TypeCorrectKind::Correct); check(R"( -local b: (number, ...s) = function(a: number, ...: string) return a end +local b: (number, ...s@1) = function(a: number, ...: string) return a end )"); - ac = autocomplete(1, 22); + ac = autocomplete('1'); CHECK(ac.entryMap.count("string")); CHECK(ac.entryMap["string"].typeCorrect == TypeCorrectKind::Correct); check(R"( -local b: (number) -> ...s = function(a: number): ...string return "a", "b", "c" end +local b: (number) -> ...s@1 = function(a: number): ...string return "a", "b", "c" end )"); - ac = autocomplete(1, 25); + ac = autocomplete('1'); CHECK(ac.entryMap.count("string")); CHECK(ac.entryMap["string"].typeCorrect == TypeCorrectKind::Correct); @@ -1552,24 +1567,24 @@ local b: (number) -> ...s = function(a: number): ...string return "a", "b", "c" TEST_CASE_FIXTURE(ACFixture, "type_correct_full_type_suggestion") { check(R"( -local b: = "str" +local b:@1 @2= "str" )"); - auto ac = autocomplete(1, 8); + auto ac = autocomplete('1'); CHECK(ac.entryMap.count("string")); CHECK(ac.entryMap["string"].typeCorrect == TypeCorrectKind::Correct); - ac = autocomplete(1, 9); + ac = autocomplete('2'); CHECK(ac.entryMap.count("string")); CHECK(ac.entryMap["string"].typeCorrect == TypeCorrectKind::Correct); check(R"( -local b: = function(a: number) return -a end +local b: @1= function(a: number) return -a end )"); - ac = autocomplete(1, 9); + ac = autocomplete('1'); CHECK(ac.entryMap.count("(number) -> number")); CHECK(ac.entryMap["(number) -> number"].typeCorrect == TypeCorrectKind::Correct); @@ -1580,12 +1595,12 @@ TEST_CASE_FIXTURE(ACFixture, "type_correct_argument_type_suggestion") check(R"( local function target(a: number, b: string) return a + #b end -local function d(a: n, b) +local function d(a: n@1, b) return target(a, b) end )"); - auto ac = autocomplete(3, 21); + auto ac = autocomplete('1'); CHECK(ac.entryMap.count("number")); CHECK(ac.entryMap["number"].typeCorrect == TypeCorrectKind::Correct); @@ -1593,12 +1608,12 @@ end check(R"( local function target(a: number, b: string) return a + #b end -local function d(a, b: s) +local function d(a, b: s@1) return target(a, b) end )"); - ac = autocomplete(3, 24); + ac = autocomplete('1'); CHECK(ac.entryMap.count("string")); CHECK(ac.entryMap["string"].typeCorrect == TypeCorrectKind::Correct); @@ -1606,17 +1621,17 @@ end check(R"( local function target(a: number, b: string) return a + #b end -local function d(a: , b) +local function d(a:@1 @2, b) return target(a, b) end )"); - ac = autocomplete(3, 19); + ac = autocomplete('1'); CHECK(ac.entryMap.count("number")); CHECK(ac.entryMap["number"].typeCorrect == TypeCorrectKind::Correct); - ac = autocomplete(3, 20); + ac = autocomplete('2'); CHECK(ac.entryMap.count("number")); CHECK(ac.entryMap["number"].typeCorrect == TypeCorrectKind::Correct); @@ -1624,17 +1639,17 @@ end check(R"( local function target(a: number, b: string) return a + #b end -local function d(a, b: ): number +local function d(a, b: @1)@2: number return target(a, b) end )"); - ac = autocomplete(3, 23); + ac = autocomplete('1'); CHECK(ac.entryMap.count("string")); CHECK(ac.entryMap["string"].typeCorrect == TypeCorrectKind::Correct); - ac = autocomplete(3, 24); + ac = autocomplete('2'); CHECK(ac.entryMap["string"].typeCorrect == TypeCorrectKind::None); } @@ -1644,10 +1659,10 @@ TEST_CASE_FIXTURE(ACFixture, "type_correct_expected_argument_type_suggestion") check(R"( local function target(callback: (a: number, b: string) -> number) return callback(4, "hello") end -local x = target(function(a: +local x = target(function(a: @1 )"); - auto ac = autocomplete(3, 29); + auto ac = autocomplete('1'); CHECK(ac.entryMap.count("number")); CHECK(ac.entryMap["number"].typeCorrect == TypeCorrectKind::Correct); @@ -1655,10 +1670,10 @@ local x = target(function(a: check(R"( local function target(callback: (a: number, b: string) -> number) return callback(4, "hello") end -local x = target(function(a: n +local x = target(function(a: n@1 )"); - ac = autocomplete(3, 30); + ac = autocomplete('1'); CHECK(ac.entryMap.count("number")); CHECK(ac.entryMap["number"].typeCorrect == TypeCorrectKind::Correct); @@ -1666,17 +1681,17 @@ local x = target(function(a: n check(R"( local function target(callback: (a: number, b: string) -> number) return callback(4, "hello") end -local x = target(function(a: n, b: ) +local x = target(function(a: n@1, b: @2) return a + #b end) )"); - ac = autocomplete(3, 30); + ac = autocomplete('1'); CHECK(ac.entryMap.count("number")); CHECK(ac.entryMap["number"].typeCorrect == TypeCorrectKind::Correct); - ac = autocomplete(3, 35); + ac = autocomplete('2'); CHECK(ac.entryMap.count("string")); CHECK(ac.entryMap["string"].typeCorrect == TypeCorrectKind::Correct); @@ -1684,12 +1699,12 @@ end) check(R"( local function target(callback: (...number) -> number) return callback(1, 2, 3) end -local x = target(function(a: n) +local x = target(function(a: n@1) return a end )"); - ac = autocomplete(3, 30); + ac = autocomplete('1'); CHECK(ac.entryMap.count("number")); CHECK(ac.entryMap["number"].typeCorrect == TypeCorrectKind::Correct); @@ -1700,12 +1715,12 @@ TEST_CASE_FIXTURE(ACFixture, "type_correct_expected_argument_type_pack_suggestio check(R"( local function target(callback: (...number) -> number) return callback(1, 2, 3) end -local x = target(function(...:n) +local x = target(function(...:n@1) return a end )"); - auto ac = autocomplete(3, 31); + auto ac = autocomplete('1'); CHECK(ac.entryMap.count("number")); CHECK(ac.entryMap["number"].typeCorrect == TypeCorrectKind::Correct); @@ -1713,12 +1728,12 @@ end check(R"( local function target(callback: (...number) -> number) return callback(1, 2, 3) end -local x = target(function(a:number, b:number, ...:) +local x = target(function(a:number, b:number, ...:@1) return a + b end )"); - ac = autocomplete(3, 50); + ac = autocomplete('1'); CHECK(ac.entryMap.count("number")); CHECK(ac.entryMap["number"].typeCorrect == TypeCorrectKind::Correct); @@ -1729,12 +1744,12 @@ TEST_CASE_FIXTURE(ACFixture, "type_correct_expected_return_type_suggestion") check(R"( local function target(callback: () -> number) return callback() end -local x = target(function(): n +local x = target(function(): n@1 return 1 end )"); - auto ac = autocomplete(3, 30); + auto ac = autocomplete('1'); CHECK(ac.entryMap.count("number")); CHECK(ac.entryMap["number"].typeCorrect == TypeCorrectKind::Correct); @@ -1742,12 +1757,12 @@ end check(R"( local function target(callback: () -> (number, number)) return callback() end -local x = target(function(): (number, n +local x = target(function(): (number, n@1 return 1, 2 end )"); - ac = autocomplete(3, 39); + ac = autocomplete('1'); CHECK(ac.entryMap.count("number")); CHECK(ac.entryMap["number"].typeCorrect == TypeCorrectKind::Correct); @@ -1758,12 +1773,12 @@ TEST_CASE_FIXTURE(ACFixture, "type_correct_expected_return_type_pack_suggestion" check(R"( local function target(callback: () -> ...number) return callback() end -local x = target(function(): ...n +local x = target(function(): ...n@1 return 1, 2, 3 end )"); - auto ac = autocomplete(3, 33); + auto ac = autocomplete('1'); CHECK(ac.entryMap.count("number")); CHECK(ac.entryMap["number"].typeCorrect == TypeCorrectKind::Correct); @@ -1771,12 +1786,12 @@ end check(R"( local function target(callback: () -> ...number) return callback() end -local x = target(function(): (number, number, ...n +local x = target(function(): (number, number, ...n@1 return 1, 2, 3 end )"); - ac = autocomplete(3, 50); + ac = autocomplete('1'); CHECK(ac.entryMap.count("number")); CHECK(ac.entryMap["number"].typeCorrect == TypeCorrectKind::Correct); @@ -1787,10 +1802,10 @@ TEST_CASE_FIXTURE(ACFixture, "type_correct_expected_argument_type_suggestion_opt check(R"( local function target(callback: nil | (a: number, b: string) -> number) return callback(4, "hello") end -local x = target(function(a: +local x = target(function(a: @1 )"); - auto ac = autocomplete(3, 29); + auto ac = autocomplete('1'); CHECK(ac.entryMap.count("number")); CHECK(ac.entryMap["number"].typeCorrect == TypeCorrectKind::Correct); @@ -1803,21 +1818,21 @@ local t = {} t.x = 5 function t:target(callback: (a: number, b: string) -> number) return callback(self.x, "hello") end -local x = t:target(function(a: , b: ) end) -local y = t.target(t, function(a: number, b: ) end) +local x = t:target(function(a: @1, b:@2 ) end) +local y = t.target(t, function(a: number, b: @3) end) )"); - auto ac = autocomplete(5, 31); + auto ac = autocomplete('1'); CHECK(ac.entryMap.count("number")); CHECK(ac.entryMap["number"].typeCorrect == TypeCorrectKind::Correct); - ac = autocomplete(5, 35); + ac = autocomplete('2'); CHECK(ac.entryMap.count("string")); CHECK(ac.entryMap["string"].typeCorrect == TypeCorrectKind::Correct); - ac = autocomplete(6, 45); + ac = autocomplete('3'); CHECK(ac.entryMap.count("string")); CHECK(ac.entryMap["string"].typeCorrect == TypeCorrectKind::Correct); @@ -1899,26 +1914,26 @@ TEST_CASE_FIXTURE(ACFixture, "do_not_suggest_synthetic_table_name") { check(R"( local foo = { a = 1, b = 2 } -local bar: = foo +local bar: @1= foo )"); - auto ac = autocomplete(2, 11); + auto ac = autocomplete('1'); CHECK(!ac.entryMap.count("foo")); } -// CLI-45692: Remove UnfrozenFixture here -TEST_CASE_FIXTURE(UnfrozenFixture, "type_correct_function_no_parenthesis") +// CLI-45692: Remove UnfrozenACFixture here +TEST_CASE_FIXTURE(UnfrozenACFixture, "type_correct_function_no_parenthesis") { check(R"( local function target(a: (number) -> number) return a(4) end local function bar1(a: number) return -a end local function bar2(a: string) reutrn a .. 'x' end -return target(b +return target(b@1 )"); - auto ac = autocomplete(frontend, "MainModule", Position{5, 15}, nullCallback); + auto ac = autocomplete('1'); CHECK(ac.entryMap.count("bar1")); CHECK(ac.entryMap["bar1"].typeCorrect == TypeCorrectKind::Correct); @@ -1930,16 +1945,16 @@ TEST_CASE_FIXTURE(ACFixture, "type_correct_sealed_table") { check(R"( local function f(a: { x: number, y: number }) return a.x + a.y end -local fp: = f +local fp: @1= f )"); - auto ac = autocomplete(2, 10); + auto ac = autocomplete('1'); CHECK(ac.entryMap.count("({ x: number, y: number }) -> number")); } -// CLI-45692: Remove UnfrozenFixture here -TEST_CASE_FIXTURE(UnfrozenFixture, "type_correct_keywords") +// CLI-45692: Remove UnfrozenACFixture here +TEST_CASE_FIXTURE(UnfrozenACFixture, "type_correct_keywords") { check(R"( local function a(x: boolean) end @@ -1951,33 +1966,33 @@ local function e(x: ((number) -> string) & ((boolean) -> number)) end local tru = {} local ni = false -local ac = a(t) -local bc = b(n) -local cc = c(f) -local dc = d(f) -local ec = e(f) +local ac = a(t@1) +local bc = b(n@2) +local cc = c(f@3) +local dc = d(f@4) +local ec = e(f@5) )"); - auto ac = autocomplete(frontend, "MainModule", Position{10, 14}, nullCallback); + auto ac = autocomplete('1'); CHECK(ac.entryMap.count("tru")); CHECK(ac.entryMap["tru"].typeCorrect == TypeCorrectKind::None); CHECK(ac.entryMap["true"].typeCorrect == TypeCorrectKind::Correct); CHECK(ac.entryMap["false"].typeCorrect == TypeCorrectKind::Correct); - ac = autocomplete(frontend, "MainModule", Position{11, 14}, nullCallback); + ac = autocomplete('2'); CHECK(ac.entryMap.count("ni")); CHECK(ac.entryMap["ni"].typeCorrect == TypeCorrectKind::None); CHECK(ac.entryMap["nil"].typeCorrect == TypeCorrectKind::Correct); - ac = autocomplete(frontend, "MainModule", Position{12, 14}, nullCallback); + ac = autocomplete('3'); CHECK(ac.entryMap.count("false")); CHECK(ac.entryMap["false"].typeCorrect == TypeCorrectKind::None); CHECK(ac.entryMap["function"].typeCorrect == TypeCorrectKind::Correct); - ac = autocomplete(frontend, "MainModule", Position{13, 14}, nullCallback); + ac = autocomplete('4'); CHECK(ac.entryMap["function"].typeCorrect == TypeCorrectKind::Correct); - ac = autocomplete(frontend, "MainModule", Position{14, 14}, nullCallback); + ac = autocomplete('5'); CHECK(ac.entryMap["function"].typeCorrect == TypeCorrectKind::Correct); } @@ -1988,10 +2003,10 @@ local target: ((number) -> string) & ((string) -> number)) local one = 4 local two = "hello" -return target(o) +return target(o@1) )"); - auto ac = autocomplete(5, 15); + auto ac = autocomplete('1'); CHECK(ac.entryMap.count("one")); CHECK(ac.entryMap["one"].typeCorrect == TypeCorrectKind::Correct); @@ -2002,10 +2017,10 @@ local target: ((number) -> string) & ((number) -> number)) local one = 4 local two = "hello" -return target(o) +return target(o@1) )"); - ac = autocomplete(5, 15); + ac = autocomplete('1'); CHECK(ac.entryMap.count("one")); CHECK(ac.entryMap["one"].typeCorrect == TypeCorrectKind::Correct); @@ -2016,10 +2031,10 @@ local target: ((number, number) -> string) & ((string) -> number)) local one = 4 local two = "hello" -return target(1, o) +return target(1, o@1) )"); - ac = autocomplete(5, 18); + ac = autocomplete('1'); CHECK(ac.entryMap.count("one")); CHECK(ac.entryMap["one"].typeCorrect == TypeCorrectKind::Correct); @@ -2032,10 +2047,10 @@ TEST_CASE_FIXTURE(ACFixture, "optional_members") local a = { x = 2, y = 3 } type A = typeof(a) local b: A? = a -return b. +return b.@1 )"); - auto ac = autocomplete(4, 9); + auto ac = autocomplete('1'); CHECK_EQ(2, ac.entryMap.size()); CHECK(ac.entryMap.count("x")); @@ -2045,10 +2060,10 @@ return b. local a = { x = 2, y = 3 } type A = typeof(a) local b: nil | A = a -return b. +return b.@1 )"); - ac = autocomplete(4, 9); + ac = autocomplete('1'); CHECK_EQ(2, ac.entryMap.size()); CHECK(ac.entryMap.count("x")); @@ -2056,10 +2071,10 @@ return b. check(R"( local b: nil | nil -return b. +return b.@1 )"); - ac = autocomplete(2, 9); + ac = autocomplete('1'); CHECK_EQ(0, ac.entryMap.size()); } @@ -2067,26 +2082,26 @@ return b. TEST_CASE_FIXTURE(ACFixture, "no_function_name_suggestions") { check(R"( -function na +function na@1 )"); - auto ac = autocomplete(1, 11); + auto ac = autocomplete('1'); CHECK(ac.entryMap.empty()); check(R"( -local function +local function @1 )"); - ac = autocomplete(1, 15); + ac = autocomplete('1'); CHECK(ac.entryMap.empty()); check(R"( -local function na +local function na@1 )"); - ac = autocomplete(1, 17); + ac = autocomplete('1'); CHECK(ac.entryMap.empty()); } @@ -2095,20 +2110,20 @@ TEST_CASE_FIXTURE(ACFixture, "skip_current_local") { check(R"( local other = 1 -local name = na +local name = na@1 )"); - auto ac = autocomplete(2, 15); + auto ac = autocomplete('1'); CHECK(!ac.entryMap.count("name")); CHECK(ac.entryMap.count("other")); check(R"( local other = 1 -local name, test = na +local name, test = na@1 )"); - ac = autocomplete(2, 21); + ac = autocomplete('1'); CHECK(!ac.entryMap.count("name")); CHECK(!ac.entryMap.count("test")); @@ -2119,26 +2134,26 @@ TEST_CASE_FIXTURE(ACFixture, "keyword_members") { check(R"( local a = { done = 1, forever = 2 } -local b = a.do -local c = a.for -local d = a. +local b = a.do@1 +local c = a.for@2 +local d = a.@3 do end )"); - auto ac = autocomplete(2, 14); + auto ac = autocomplete('1'); CHECK_EQ(2, ac.entryMap.size()); CHECK(ac.entryMap.count("done")); CHECK(ac.entryMap.count("forever")); - ac = autocomplete(3, 15); + ac = autocomplete('2'); CHECK_EQ(2, ac.entryMap.size()); CHECK(ac.entryMap.count("done")); CHECK(ac.entryMap.count("forever")); - ac = autocomplete(4, 12); + ac = autocomplete('3'); CHECK_EQ(2, ac.entryMap.size()); CHECK(ac.entryMap.count("done")); @@ -2150,10 +2165,10 @@ TEST_CASE_FIXTURE(ACFixture, "keyword_methods") check(R"( local a = {} function a:done() end -local b = a:do +local b = a:do@1 )"); - auto ac = autocomplete(3, 14); + auto ac = autocomplete('1'); CHECK_EQ(1, ac.entryMap.size()); CHECK(ac.entryMap.count("done")); @@ -2247,29 +2262,29 @@ local elsewhere = false local doover = false local endurance = true -if 1 then -else +if 1 then@1 +else@2 end -while false do +while false do@3 end -repeat +repeat@4 until )"); - auto ac = autocomplete(6, 9); + auto ac = autocomplete('1'); CHECK(ac.entryMap.size() == 1); CHECK(ac.entryMap.count("then")); - ac = autocomplete(7, 4); + ac = autocomplete('2'); CHECK(ac.entryMap.count("else")); CHECK(ac.entryMap.count("elseif")); - ac = autocomplete(10, 14); + ac = autocomplete('3'); CHECK(ac.entryMap.count("do")); - ac = autocomplete(13, 6); + ac = autocomplete('4'); CHECK(ac.entryMap.count("do")); // FIXME: ideally we want to handle start and end of all statements as well @@ -2284,11 +2299,11 @@ local elsewhere = false if true then return 1 -el +el@1 end )"); - auto ac = autocomplete(5, 2); + auto ac = autocomplete('1'); CHECK(ac.entryMap.count("else")); CHECK(ac.entryMap.count("elseif")); CHECK(ac.entryMap.count("elsewhere") == 0); @@ -2300,11 +2315,11 @@ if true then return 1 else return 2 -el +el@1 end )"); - ac = autocomplete(7, 2); + ac = autocomplete('1'); CHECK(ac.entryMap.count("else") == 0); CHECK(ac.entryMap.count("elseif") == 0); CHECK(ac.entryMap.count("elsewhere")); @@ -2316,10 +2331,10 @@ if true then print("1") elif true then print("2") -el +el@1 end )"); - ac = autocomplete(7, 2); + ac = autocomplete('1'); CHECK(ac.entryMap.count("else")); CHECK(ac.entryMap.count("elseif")); CHECK(ac.entryMap.count("elsewhere")); @@ -2360,30 +2375,30 @@ TEST_CASE_FIXTURE(ACFixture, "suggest_table_keys") { check(R"( type Test = { first: number, second: number } -local t: Test = { f } +local t: Test = { f@1 } )"); - auto ac = autocomplete(2, 19); + auto ac = autocomplete('1'); CHECK(ac.entryMap.count("first")); CHECK(ac.entryMap.count("second")); // Intersection check(R"( type Test = { first: number } & { second: number } -local t: Test = { f } +local t: Test = { f@1 } )"); - ac = autocomplete(2, 19); + ac = autocomplete('1'); CHECK(ac.entryMap.count("first")); CHECK(ac.entryMap.count("second")); // Union check(R"( type Test = { first: number, second: number } | { second: number, third: number } -local t: Test = { s } +local t: Test = { s@1 } )"); - ac = autocomplete(2, 19); + ac = autocomplete('1'); CHECK(ac.entryMap.count("second")); CHECK(!ac.entryMap.count("first")); CHECK(!ac.entryMap.count("third")); @@ -2391,60 +2406,60 @@ local t: Test = { s } // No parenthesis suggestion check(R"( type Test = { first: (number) -> number, second: number } -local t: Test = { f } +local t: Test = { f@1 } )"); - ac = autocomplete(2, 19); + ac = autocomplete('1'); CHECK(ac.entryMap.count("first")); CHECK(ac.entryMap["first"].parens == ParenthesesRecommendation::None); // When key is changed check(R"( type Test = { first: number, second: number } -local t: Test = { f = 2 } +local t: Test = { f@1 = 2 } )"); - ac = autocomplete(2, 19); + ac = autocomplete('1'); CHECK(ac.entryMap.count("first")); CHECK(ac.entryMap.count("second")); // Alternative key syntax check(R"( type Test = { first: number, second: number } -local t: Test = { ["f"] } +local t: Test = { ["f@1"] } )"); - ac = autocomplete(2, 21); + ac = autocomplete('1'); CHECK(ac.entryMap.count("first")); CHECK(ac.entryMap.count("second")); // Not an alternative key syntax check(R"( type Test = { first: number, second: number } -local t: Test = { "f" } +local t: Test = { "f@1" } )"); - ac = autocomplete(2, 20); + ac = autocomplete('1'); CHECK(!ac.entryMap.count("first")); CHECK(!ac.entryMap.count("second")); // Skip keys that are already defined check(R"( type Test = { first: number, second: number } -local t: Test = { first = 2, s } +local t: Test = { first = 2, s@1 } )"); - ac = autocomplete(2, 30); + ac = autocomplete('1'); CHECK(!ac.entryMap.count("first")); CHECK(ac.entryMap.count("second")); // Don't skip active key check(R"( type Test = { first: number, second: number } -local t: Test = { first } +local t: Test = { first@1 } )"); - ac = autocomplete(2, 23); + ac = autocomplete('1'); CHECK(ac.entryMap.count("first")); CHECK(ac.entryMap.count("second")); @@ -2452,22 +2467,22 @@ local t: Test = { first } check(R"( local t = { { first = 5, second = 10 }, - { f } + { f@1 } } )"); - ac = autocomplete(3, 7); + ac = autocomplete('1'); CHECK(ac.entryMap.count("first")); CHECK(ac.entryMap.count("second")); check(R"( local t = { [2] = { first = 5, second = 10 }, - [5] = { f } + [5] = { f@1 } } )"); - ac = autocomplete(3, 13); + ac = autocomplete('1'); CHECK(ac.entryMap.count("first")); CHECK(ac.entryMap.count("second")); } @@ -2502,15 +2517,15 @@ TEST_CASE_FIXTURE(ACFixture, "autocomplete_ifelse_expressions") local temp = false local even = true; local a = true -a = if t1@emp then t -a = if temp t2@ -a = if temp then e3@ -a = if temp then even e4@ -a = if temp then even elseif t5@ -a = if temp then even elseif true t6@ -a = if temp then even elseif true then t7@ -a = if temp then even elseif true then temp e8@ -a = if temp then even elseif true then temp else e9@ +a = if t@1emp then t +a = if temp t@2 +a = if temp then e@3 +a = if temp then even e@4 +a = if temp then even elseif t@5 +a = if temp then even elseif true t@6 +a = if temp then even elseif true then t@7 +a = if temp then even elseif true then temp e@8 +a = if temp then even elseif true then temp else e@9 )"); auto ac = autocomplete('1'); @@ -2573,4 +2588,20 @@ a = if temp then even elseif true then temp else e9@ } } +TEST_CASE_FIXTURE(ACFixture, "autocomplete_explicit_type_pack") +{ + ScopedFastFlag luauTypeAliasPacks("LuauTypeAliasPacks", true); + ScopedFastFlag luauParseTypePackTypeParameters("LuauParseTypePackTypeParameters", true); + + check(R"( +type A = () -> T... +local a: A<(number, s@1> + )"); + + auto ac = autocomplete('1'); + + CHECK(ac.entryMap.count("number")); + CHECK(ac.entryMap.count("string")); +} + TEST_SUITE_END(); diff --git a/tests/Fixture.cpp b/tests/Fixture.cpp index 26bc77f7..29c33f7c 100644 --- a/tests/Fixture.cpp +++ b/tests/Fixture.cpp @@ -32,6 +32,55 @@ std::optional TestFileResolver::fromAstFragment(AstExpr* expr) const return std::nullopt; } +std::optional TestFileResolver::resolveModule(const ModuleInfo* context, AstExpr* expr) +{ + if (AstExprGlobal* g = expr->as()) + { + if (g->name == "game") + return ModuleInfo{"game"}; + if (g->name == "workspace") + return ModuleInfo{"workspace"}; + if (g->name == "script") + return context ? std::optional(*context) : std::nullopt; + } + else if (AstExprIndexName* i = expr->as(); i && context) + { + if (i->index == "Parent") + { + std::string_view view = context->name; + size_t lastSeparatorIndex = view.find_last_of('/'); + + if (lastSeparatorIndex == std::string_view::npos) + return std::nullopt; + + return ModuleInfo{ModuleName(view.substr(0, lastSeparatorIndex)), context->optional}; + } + else + { + return ModuleInfo{context->name + '/' + i->index.value, context->optional}; + } + } + else if (AstExprIndexExpr* i = expr->as(); i && context) + { + if (AstExprConstantString* index = i->index->as()) + { + return ModuleInfo{context->name + '/' + std::string(index->value.data, index->value.size), context->optional}; + } + } + else if (AstExprCall* call = expr->as(); call && call->self && call->args.size >= 1 && context) + { + if (AstExprConstantString* index = call->args.data[0]->as()) + { + AstName func = call->func->as()->index; + + if (func == "GetService" && context->name == "game") + return ModuleInfo{"game/" + std::string(index->value.data, index->value.size)}; + } + } + + return std::nullopt; +} + ModuleName TestFileResolver::concat(const ModuleName& lhs, std::string_view rhs) const { return lhs + "/" + ModuleName(rhs); diff --git a/tests/Fixture.h b/tests/Fixture.h index c6294b01..1480a7f6 100644 --- a/tests/Fixture.h +++ b/tests/Fixture.h @@ -65,6 +65,8 @@ struct TestFileResolver } std::optional fromAstFragment(AstExpr* expr) const override; + std::optional resolveModule(const ModuleInfo* context, AstExpr* expr) override; + ModuleName concat(const ModuleName& lhs, std::string_view rhs) const override; std::optional getParentModuleName(const ModuleName& name) const override; diff --git a/tests/Frontend.test.cpp b/tests/Frontend.test.cpp index 3f33a5d1..fbfec636 100644 --- a/tests/Frontend.test.cpp +++ b/tests/Frontend.test.cpp @@ -58,6 +58,35 @@ struct NaiveFileResolver : NullFileResolver return std::nullopt; } + std::optional resolveModule(const ModuleInfo* context, AstExpr* expr) override + { + if (AstExprGlobal* g = expr->as()) + { + if (g->name == "Modules") + return ModuleInfo{"Modules"}; + + if (g->name == "game") + return ModuleInfo{"game"}; + } + else if (AstExprIndexName* i = expr->as()) + { + if (context) + return ModuleInfo{context->name + '/' + i->index.value, context->optional}; + } + else if (AstExprCall* call = expr->as(); call && call->self && call->args.size >= 1 && context) + { + if (AstExprConstantString* index = call->args.data[0]->as()) + { + AstName func = call->func->as()->index; + + if (func == "GetService" && context->name == "game") + return ModuleInfo{"game/" + std::string(index->value.data, index->value.size)}; + } + } + + return std::nullopt; + } + ModuleName concat(const ModuleName& lhs, std::string_view rhs) const override { return lhs + "/" + ModuleName(rhs); @@ -528,7 +557,7 @@ TEST_CASE_FIXTURE(FrontendFixture, "ignore_require_to_nonexistent_file") { fileResolver.source["Modules/A"] = R"( local Modules = script - local B = require(Modules.B :: any) + local B = require(Modules.B) :: any )"; CheckResult result = frontend.check("Modules/A"); diff --git a/tests/Linter.test.cpp b/tests/Linter.test.cpp index c8eff399..a9ed139f 100644 --- a/tests/Linter.test.cpp +++ b/tests/Linter.test.cpp @@ -1400,6 +1400,8 @@ end TEST_CASE_FIXTURE(Fixture, "TableOperations") { + ScopedFastFlag sff("LuauLinterTableMoveZero", true); + LintResult result = lintTyped(R"( local t = {} local tt = {} @@ -1417,9 +1419,12 @@ table.remove(t, 0) table.remove(t, #t-1) table.insert(t, string.find("hello", "h")) + +table.move(t, 0, #t, 1, tt) +table.move(t, 1, #t, 0, tt) )"); - REQUIRE_EQ(result.warnings.size(), 6); + REQUIRE_EQ(result.warnings.size(), 8); CHECK_EQ(result.warnings[0].text, "table.insert will insert the value before the last element, which is likely a bug; consider removing the " "second argument or wrap it in parentheses to silence"); CHECK_EQ(result.warnings[1].text, "table.insert will append the value to the table; consider removing the second argument for efficiency"); @@ -1429,6 +1434,8 @@ table.insert(t, string.find("hello", "h")) "second argument or wrap it in parentheses to silence"); CHECK_EQ(result.warnings[5].text, "table.insert may change behavior if the call returns more than one result; consider adding parentheses around second argument"); + CHECK_EQ(result.warnings[6].text, "table.move uses index 0 but arrays are 1-based; did you mean 1 instead?"); + CHECK_EQ(result.warnings[7].text, "table.move uses index 0 but arrays are 1-based; did you mean 1 instead?"); } TEST_CASE_FIXTURE(Fixture, "DuplicateConditions") diff --git a/tests/Module.test.cpp b/tests/Module.test.cpp index 1b146ed2..18f55d2c 100644 --- a/tests/Module.test.cpp +++ b/tests/Module.test.cpp @@ -1,5 +1,6 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #include "Luau/Module.h" +#include "Luau/Scope.h" #include "Fixture.h" diff --git a/tests/NonstrictMode.test.cpp b/tests/NonstrictMode.test.cpp index f3c76d55..931a8403 100644 --- a/tests/NonstrictMode.test.cpp +++ b/tests/NonstrictMode.test.cpp @@ -1,5 +1,6 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #include "Luau/Parser.h" +#include "Luau/Scope.h" #include "Luau/TypeInfer.h" #include "Luau/TypeVar.h" diff --git a/tests/Parser.test.cpp b/tests/Parser.test.cpp index cb03a7bd..a80718e4 100644 --- a/tests/Parser.test.cpp +++ b/tests/Parser.test.cpp @@ -2519,4 +2519,19 @@ TEST_CASE_FIXTURE(Fixture, "parse_if_else_expression") } } +TEST_CASE_FIXTURE(Fixture, "parse_type_pack_type_parameters") +{ + ScopedFastFlag luauParseGenericFunctions("LuauParseGenericFunctions", true); + ScopedFastFlag luauParseTypePackTypeParameters("LuauParseTypePackTypeParameters", true); + + AstStat* stat = parse(R"( +type Packed = () -> T... + +type A = Packed +type B = Packed<...number> +type C = Packed<(number, X...)> + )"); + REQUIRE(stat != nullptr); +} + TEST_SUITE_END(); diff --git a/tests/RequireTracer.test.cpp b/tests/RequireTracer.test.cpp index cbd4af29..b9fd04d6 100644 --- a/tests/RequireTracer.test.cpp +++ b/tests/RequireTracer.test.cpp @@ -57,6 +57,7 @@ TEST_CASE_FIXTURE(RequireTracerFixture, "trace_local") { AstStatBlock* block = parse(R"( local m = workspace.Foo.Bar.Baz + require(m) )"); RequireTraceResult result = traceRequires(&fileResolver, block, "ModuleName"); @@ -70,22 +71,22 @@ TEST_CASE_FIXTURE(RequireTracerFixture, "trace_local") AstExprIndexName* value = loc->values.data[0]->as(); REQUIRE(value); REQUIRE(result.exprs.contains(value)); - CHECK_EQ("workspace/Foo/Bar/Baz", result.exprs[value]); + CHECK_EQ("workspace/Foo/Bar/Baz", result.exprs[value].name); value = value->expr->as(); REQUIRE(value); REQUIRE(result.exprs.contains(value)); - CHECK_EQ("workspace/Foo/Bar", result.exprs[value]); + CHECK_EQ("workspace/Foo/Bar", result.exprs[value].name); value = value->expr->as(); REQUIRE(value); REQUIRE(result.exprs.contains(value)); - CHECK_EQ("workspace/Foo", result.exprs[value]); + CHECK_EQ("workspace/Foo", result.exprs[value].name); AstExprGlobal* workspace = value->expr->as(); REQUIRE(workspace); REQUIRE(result.exprs.contains(workspace)); - CHECK_EQ("workspace", result.exprs[workspace]); + CHECK_EQ("workspace", result.exprs[workspace].name); } TEST_CASE_FIXTURE(RequireTracerFixture, "trace_transitive_local") @@ -93,9 +94,10 @@ TEST_CASE_FIXTURE(RequireTracerFixture, "trace_transitive_local") AstStatBlock* block = parse(R"( local m = workspace.Foo.Bar.Baz local n = m.Quux + require(n) )"); - REQUIRE_EQ(2, block->body.size); + REQUIRE_EQ(3, block->body.size); RequireTraceResult result = traceRequires(&fileResolver, block, "ModuleName"); @@ -104,13 +106,13 @@ TEST_CASE_FIXTURE(RequireTracerFixture, "trace_transitive_local") REQUIRE_EQ(1, local->vars.size); REQUIRE(result.exprs.contains(local->values.data[0])); - CHECK_EQ("workspace/Foo/Bar/Baz/Quux", result.exprs[local->values.data[0]]); + CHECK_EQ("workspace/Foo/Bar/Baz/Quux", result.exprs[local->values.data[0]].name); } TEST_CASE_FIXTURE(RequireTracerFixture, "trace_function_arguments") { AstStatBlock* block = parse(R"( - local M = require(workspace.Game.Thing, workspace.Something.Else) + local M = require(workspace.Game.Thing) )"); REQUIRE_EQ(1, block->body.size); @@ -124,52 +126,9 @@ TEST_CASE_FIXTURE(RequireTracerFixture, "trace_function_arguments") AstExprCall* call = local->values.data[0]->as(); REQUIRE(call != nullptr); - REQUIRE_EQ(2, call->args.size); - - CHECK_EQ("workspace/Game/Thing", result.exprs[call->args.data[0]]); - CHECK_EQ("workspace/Something/Else", result.exprs[call->args.data[1]]); -} - -TEST_CASE_FIXTURE(RequireTracerFixture, "follow_GetService_calls") -{ - AstStatBlock* block = parse(R"( - local R = game:GetService('ReplicatedStorage').Roact - local Roact = require(R) - )"); - REQUIRE_EQ(2, block->body.size); - - RequireTraceResult result = traceRequires(&fileResolver, block, "ModuleName"); - - AstStatLocal* local = block->body.data[0]->as(); - REQUIRE(local != nullptr); - - CHECK_EQ("game/ReplicatedStorage/Roact", result.exprs[local->values.data[0]]); - - AstStatLocal* local2 = block->body.data[1]->as(); - REQUIRE(local2 != nullptr); - REQUIRE_EQ(1, local2->values.size); - - AstExprCall* call = local2->values.data[0]->as(); - REQUIRE(call != nullptr); REQUIRE_EQ(1, call->args.size); - CHECK_EQ("game/ReplicatedStorage/Roact", result.exprs[call->args.data[0]]); -} - -TEST_CASE_FIXTURE(RequireTracerFixture, "follow_WaitForChild_calls") -{ - ScopedFastFlag luauTraceRequireLookupChild("LuauTraceRequireLookupChild", true); - - AstStatBlock* block = parse(R"( -local A = require(workspace:WaitForChild('ReplicatedStorage').Content) -local B = require(workspace:FindFirstChild('ReplicatedFirst').Data) - )"); - - RequireTraceResult result = traceRequires(&fileResolver, block, "ModuleName"); - - REQUIRE_EQ(2, result.requires.size()); - CHECK_EQ("workspace/ReplicatedStorage/Content", result.requires[0].first); - CHECK_EQ("workspace/ReplicatedFirst/Data", result.requires[1].first); + CHECK_EQ("workspace/Game/Thing", result.exprs[call->args.data[0]].name); } TEST_CASE_FIXTURE(RequireTracerFixture, "follow_typeof") @@ -200,22 +159,23 @@ TEST_CASE_FIXTURE(RequireTracerFixture, "follow_typeof") REQUIRE(call != nullptr); REQUIRE_EQ(1, call->args.size); - CHECK_EQ("workspace/CoolThing", result.exprs[call->args.data[0]]); + CHECK_EQ("workspace/CoolThing", result.exprs[call->args.data[0]].name); } TEST_CASE_FIXTURE(RequireTracerFixture, "follow_string_indexexpr") { AstStatBlock* block = parse(R"( local R = game["Test"] + require(R) )"); - REQUIRE_EQ(1, block->body.size); + REQUIRE_EQ(2, block->body.size); RequireTraceResult result = traceRequires(&fileResolver, block, "ModuleName"); AstStatLocal* local = block->body.data[0]->as(); REQUIRE(local != nullptr); - CHECK_EQ("game/Test", result.exprs[local->values.data[0]]); + CHECK_EQ("game/Test", result.exprs[local->values.data[0]].name); } TEST_SUITE_END(); diff --git a/tests/ToString.test.cpp b/tests/ToString.test.cpp index d7d68c46..e18bf7cd 100644 --- a/tests/ToString.test.cpp +++ b/tests/ToString.test.cpp @@ -1,5 +1,6 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +#include "Luau/Scope.h" #include "Luau/ToString.h" #include "Fixture.h" @@ -416,8 +417,6 @@ function foo(a, b) return a(b) end TEST_CASE_FIXTURE(Fixture, "toString_the_boundTo_table_type_contained_within_a_TypePack") { - ScopedFastFlag sff{"LuauToStringFollowsBoundTo", true}; - TypeVar tv1{TableTypeVar{}}; TableTypeVar* ttv = getMutable(&tv1); ttv->state = TableState::Sealed; diff --git a/tests/TypeInfer.aliases.test.cpp b/tests/TypeInfer.aliases.test.cpp new file mode 100644 index 00000000..045f0230 --- /dev/null +++ b/tests/TypeInfer.aliases.test.cpp @@ -0,0 +1,557 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details + +#include "Fixture.h" + +#include "doctest.h" +#include "Luau/BuiltinDefinitions.h" + +using namespace Luau; + +TEST_SUITE_BEGIN("TypeAliases"); + +TEST_CASE_FIXTURE(Fixture, "cyclic_function_type_in_type_alias") +{ + ScopedFastFlag sff{"LuauOccursCheckOkWithRecursiveFunctions", true}; + + CheckResult result = check(R"( + type F = () -> F? + local function f() + return f + end + + local g: F = f + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + CHECK_EQ("t1 where t1 = () -> t1?", toString(requireType("g"))); +} + +TEST_CASE_FIXTURE(Fixture, "cyclic_types_of_named_table_fields_do_not_expand_when_stringified") +{ + CheckResult result = check(R"( + --!strict + type Node = { Parent: Node?; } + local node: Node; + node.Parent = 1 + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + TypeMismatch* tm = get(result.errors[0]); + REQUIRE(tm); + CHECK_EQ("Node?", toString(tm->wantedType)); + CHECK_EQ(typeChecker.numberType, tm->givenType); +} + +TEST_CASE_FIXTURE(Fixture, "mutually_recursive_types") +{ + CheckResult result = check(R"( + --!strict + type T = { f: a, g: U } + type U = { h: a, i: T? } + local x: T = { f = 37, g = { h = 5, i = nil } } + x.g.i = x + local y: T = { f = "hi", g = { h = "lo", i = nil } } + y.g.i = y + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "mutually_recursive_types_errors") +{ + CheckResult result = check(R"( + --!strict + type T = { f: a, g: U } + type U = { h: b, i: T? } + local x: T = { f = 37, g = { h = 5, i = nil } } + x.g.i = x + local y: T = { f = "hi", g = { h = 5, i = nil } } + y.g.i = y + )"); + + LUAU_REQUIRE_ERRORS(result); + + // We had a UAF in this example caused by not cloning type function arguments + ModulePtr module = frontend.moduleResolver.getModule("MainModule"); + unfreeze(module->interfaceTypes); + copyErrors(module->errors, module->interfaceTypes); + freeze(module->interfaceTypes); + module->internalTypes.clear(); + module->astTypes.clear(); + + // Make sure the error strings don't include "VALUELESS" + for (auto error : module->errors) + CHECK_MESSAGE(toString(error).find("VALUELESS") == std::string::npos, toString(error)); +} + +TEST_CASE_FIXTURE(Fixture, "use_table_name_and_generic_params_in_errors") +{ + CheckResult result = check(R"( + type Pair = {first: T, second: U} + local a: Pair + local b: Pair + + a = b + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + TypeMismatch* tm = get(result.errors[0]); + REQUIRE(tm); + + CHECK_EQ("Pair", toString(tm->wantedType)); + CHECK_EQ("Pair", toString(tm->givenType)); +} + +TEST_CASE_FIXTURE(Fixture, "dont_stop_typechecking_after_reporting_duplicate_type_definition") +{ + CheckResult result = check(R"( + type A = number + type A = string -- Redefinition of type 'A', previously defined at line 1 + local foo: string = 1 -- No "Type 'number' could not be converted into 'string'" + )"); + + LUAU_REQUIRE_ERROR_COUNT(2, result); +} + +TEST_CASE_FIXTURE(Fixture, "stringify_type_alias_of_recursive_template_table_type") +{ + CheckResult result = check(R"( + type Table = { a: T } + type Wrapped = Table + local l: Wrapped = 2 + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + TypeMismatch* tm = get(result.errors[0]); + REQUIRE(tm); + CHECK_EQ("Wrapped", toString(tm->wantedType)); + CHECK_EQ(typeChecker.numberType, tm->givenType); +} + +TEST_CASE_FIXTURE(Fixture, "stringify_type_alias_of_recursive_template_table_type2") +{ + CheckResult result = check(R"( + type Table = { a: T } + type Wrapped = (Table) -> string + local l: Wrapped = 2 + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + TypeMismatch* tm = get(result.errors[0]); + REQUIRE(tm); + CHECK_EQ("t1 where t1 = ({| a: t1 |}) -> string", toString(tm->wantedType)); + CHECK_EQ(typeChecker.numberType, tm->givenType); +} + +// Check that recursive intersection type doesn't generate an OOM +TEST_CASE_FIXTURE(Fixture, "cli_38393_recursive_intersection_oom") +{ + CheckResult result = check(R"( + function _(l0:(t0)&((t0)&(((t0)&((t0)->()))->(typeof(_),typeof(# _)))),l39,...):any + end + type t0 = ((typeof(_))&((t0)&(((typeof(_))&(t0))->typeof(_))),{n163:any,})->(any,typeof(_)) + _(_) + )"); + + LUAU_REQUIRE_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "type_alias_fwd_declaration_is_precise") +{ + CheckResult result = check(R"( + local foo: Id = 1 + type Id = T + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "corecursive_types_generic") +{ + const std::string code = R"( + type A = {v:T, b:B} + type B = {v:T, a:A} + local aa:A + local bb = aa + )"; + + const std::string expected = R"( + type A = {v:T, b:B} + type B = {v:T, a:A} + local aa:A + local bb:A=aa + )"; + + CHECK_EQ(expected, decorateWithTypes(code)); + CheckResult result = check(code); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "corecursive_function_types") +{ + ScopedFastFlag sff{"LuauOccursCheckOkWithRecursiveFunctions", true}; + + CheckResult result = check(R"( + type A = () -> (number, B) + type B = () -> (string, A) + local a: A + local b: B + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ("t1 where t1 = () -> (number, () -> (string, t1))", toString(requireType("a"))); + CHECK_EQ("t1 where t1 = () -> (string, () -> (number, t1))", toString(requireType("b"))); +} + +TEST_CASE_FIXTURE(Fixture, "generic_param_remap") +{ + const std::string code = R"( + -- An example of a forwarded use of a type that has different type arguments than parameters + type A = {t:T, u:U, next:A?} + local aa:A = { t = 5, u = 'hi', next = { t = 'lo', u = 8 } } + local bb = aa + )"; + + const std::string expected = R"( + + type A = {t:T, u:U, next:A?} + local aa:A = { t = 5, u = 'hi', next = { t = 'lo', u = 8 } } + local bb:A=aa + )"; + + CHECK_EQ(expected, decorateWithTypes(code)); + CheckResult result = check(code); + + LUAU_REQUIRE_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "export_type_and_type_alias_are_duplicates") +{ + CheckResult result = check(R"( + export type Foo = number + type Foo = number + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + auto dtd = get(result.errors[0]); + REQUIRE(dtd); + CHECK_EQ(dtd->name, "Foo"); +} + +TEST_CASE_FIXTURE(Fixture, "stringify_optional_parameterized_alias") +{ + ScopedFastFlag sffs3{"LuauGenericFunctions", true}; + ScopedFastFlag sffs4{"LuauParseGenericFunctions", true}; + + CheckResult result = check(R"( + type Node = { value: T, child: Node? } + + local function visitor(node: Node?) + local a: Node + + if node then + a = node.child -- Observe the output of the error message. + end + end + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + auto e = get(result.errors[0]); + CHECK_EQ("Node?", toString(e->givenType)); + CHECK_EQ("Node", toString(e->wantedType)); +} + +TEST_CASE_FIXTURE(Fixture, "general_require_multi_assign") +{ + fileResolver.source["workspace/A"] = R"( + export type myvec2 = {x: number, y: number} + return {} + )"; + + fileResolver.source["workspace/B"] = R"( + export type myvec3 = {x: number, y: number, z: number} + return {} + )"; + + fileResolver.source["workspace/C"] = R"( + local Foo, Bar = require(workspace.A), require(workspace.B) + + local a: Foo.myvec2 + local b: Bar.myvec3 + )"; + + CheckResult result = frontend.check("workspace/C"); + LUAU_REQUIRE_NO_ERRORS(result); + ModulePtr m = frontend.moduleResolver.modules["workspace/C"]; + + REQUIRE(m != nullptr); + + std::optional aTypeId = lookupName(m->getModuleScope(), "a"); + REQUIRE(aTypeId); + const Luau::TableTypeVar* aType = get(follow(*aTypeId)); + REQUIRE(aType); + REQUIRE(aType->props.size() == 2); + + std::optional bTypeId = lookupName(m->getModuleScope(), "b"); + REQUIRE(bTypeId); + const Luau::TableTypeVar* bType = get(follow(*bTypeId)); + REQUIRE(bType); + REQUIRE(bType->props.size() == 3); +} + +TEST_CASE_FIXTURE(Fixture, "type_alias_import_mutation") +{ + CheckResult result = check("type t10 = typeof(table)"); + LUAU_REQUIRE_NO_ERRORS(result); + + TypeId ty = getGlobalBinding(frontend.typeChecker, "table"); + CHECK_EQ(toString(ty), "table"); + + const TableTypeVar* ttv = get(ty); + REQUIRE(ttv); + + CHECK(ttv->instantiatedTypeParams.empty()); +} + +TEST_CASE_FIXTURE(Fixture, "type_alias_local_mutation") +{ + CheckResult result = check(R"( +type Cool = { a: number, b: string } +local c: Cool = { a = 1, b = "s" } +type NotCool = Cool +)"); + LUAU_REQUIRE_NO_ERRORS(result); + + std::optional ty = requireType("c"); + REQUIRE(ty); + CHECK_EQ(toString(*ty), "Cool"); + + const TableTypeVar* ttv = get(*ty); + REQUIRE(ttv); + + CHECK(ttv->instantiatedTypeParams.empty()); +} + +TEST_CASE_FIXTURE(Fixture, "type_alias_local_rename") +{ + CheckResult result = check(R"( +type Cool = { a: number, b: string } +type NotCool = Cool +local c: Cool = { a = 1, b = "s" } +local d: NotCool = { a = 1, b = "s" } +)"); + LUAU_REQUIRE_NO_ERRORS(result); + + std::optional ty = requireType("c"); + REQUIRE(ty); + CHECK_EQ(toString(*ty), "Cool"); + + ty = requireType("d"); + REQUIRE(ty); + CHECK_EQ(toString(*ty), "NotCool"); +} + +TEST_CASE_FIXTURE(Fixture, "type_alias_local_synthetic_mutation") +{ + CheckResult result = check(R"( +local c = { a = 1, b = "s" } +type Cool = typeof(c) +)"); + LUAU_REQUIRE_NO_ERRORS(result); + + std::optional ty = requireType("c"); + REQUIRE(ty); + + const TableTypeVar* ttv = get(*ty); + REQUIRE(ttv); + CHECK_EQ(ttv->name, "Cool"); +} + +TEST_CASE_FIXTURE(Fixture, "type_alias_of_an_imported_recursive_type") +{ + fileResolver.source["game/A"] = R"( +export type X = { a: number, b: X? } +return {} + )"; + + CheckResult aResult = frontend.check("game/A"); + LUAU_REQUIRE_NO_ERRORS(aResult); + + CheckResult bResult = check(R"( +local Import = require(game.A) +type X = Import.X + )"); + LUAU_REQUIRE_NO_ERRORS(bResult); + + std::optional ty1 = lookupImportedType("Import", "X"); + REQUIRE(ty1); + + std::optional ty2 = lookupType("X"); + REQUIRE(ty2); + + CHECK_EQ(follow(*ty1), follow(*ty2)); +} + +TEST_CASE_FIXTURE(Fixture, "type_alias_of_an_imported_recursive_generic_type") +{ + fileResolver.source["game/A"] = R"( +export type X = { a: T, b: U, C: X? } +return {} + )"; + + CheckResult aResult = frontend.check("game/A"); + LUAU_REQUIRE_NO_ERRORS(aResult); + + CheckResult bResult = check(R"( +local Import = require(game.A) +type X = Import.X + )"); + LUAU_REQUIRE_NO_ERRORS(bResult); + + std::optional ty1 = lookupImportedType("Import", "X"); + REQUIRE(ty1); + + std::optional ty2 = lookupType("X"); + REQUIRE(ty2); + + CHECK_EQ(toString(*ty1, {true}), toString(*ty2, {true})); + + bResult = check(R"( +local Import = require(game.A) +type X = Import.X + )"); + LUAU_REQUIRE_NO_ERRORS(bResult); + + ty1 = lookupImportedType("Import", "X"); + REQUIRE(ty1); + + ty2 = lookupType("X"); + REQUIRE(ty2); + + CHECK_EQ(toString(*ty1, {true}), "t1 where t1 = {| C: t1?, a: T, b: U |}"); + CHECK_EQ(toString(*ty2, {true}), "{| C: t1, a: U, b: T |} where t1 = {| C: t1, a: U, b: T |}?"); +} + +TEST_CASE_FIXTURE(Fixture, "module_export_free_type_leak") +{ + CheckResult result = check(R"( +function get() + return function(obj) return true end +end + +export type f = typeof(get()) + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "module_export_wrapped_free_type_leak") +{ + CheckResult result = check(R"( +function get() + return {a = 1, b = function(obj) return true end} +end + +export type f = typeof(get()) + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + + +TEST_CASE_FIXTURE(Fixture, "mutually_recursive_types_restriction_ok") +{ + CheckResult result = check(R"( + type Tree = { data: T, children: Forest } + type Forest = {Tree} + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "mutually_recursive_types_restriction_not_ok_1") +{ + ScopedFastFlag sff{"LuauRecursiveTypeParameterRestriction", true}; + + CheckResult result = check(R"( + -- OK because forwarded types are used with their parameters. + type Tree = { data: T, children: Forest } + type Forest = {Tree<{T}>} + )"); + + LUAU_REQUIRE_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "mutually_recursive_types_restriction_not_ok_2") +{ + ScopedFastFlag sff{"LuauRecursiveTypeParameterRestriction", true}; + + CheckResult result = check(R"( + -- Not OK because forwarded types are used with different types than their parameters. + type Forest = {Tree<{T}>} + type Tree = { data: T, children: Forest } + )"); + + LUAU_REQUIRE_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "mutually_recursive_types_swapsies_ok") +{ + CheckResult result = check(R"( + type Tree1 = { data: T, children: {Tree2} } + type Tree2 = { data: U, children: {Tree1} } + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "mutually_recursive_types_swapsies_not_ok") +{ + ScopedFastFlag sff{"LuauRecursiveTypeParameterRestriction", true}; + + CheckResult result = check(R"( + type Tree1 = { data: T, children: {Tree2} } + type Tree2 = { data: U, children: {Tree1} } + )"); + + LUAU_REQUIRE_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "free_variables_from_typeof_in_aliases") +{ + CheckResult result = check(R"( + function f(x) return x[1] end + -- x has type X? for a free type variable X + local x = f ({}) + type ContainsFree = { this: a, that: typeof(x) } + type ContainsContainsFree = { that: ContainsFree } + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "non_recursive_aliases_that_reuse_a_generic_name") +{ + ScopedFastFlag sff1{"LuauSubstitutionDontReplaceIgnoredTypes", true}; + + CheckResult result = check(R"( + type Array = { [number]: T } + type Tuple = Array + + local p: Tuple + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ("{number | string}", toString(requireType("p"), {true})); +} + +TEST_SUITE_END(); diff --git a/tests/TypeInfer.provisional.test.cpp b/tests/TypeInfer.provisional.test.cpp index 46496fdb..8bcb0242 100644 --- a/tests/TypeInfer.provisional.test.cpp +++ b/tests/TypeInfer.provisional.test.cpp @@ -30,6 +30,8 @@ TEST_SUITE_BEGIN("ProvisionalTests"); */ TEST_CASE_FIXTURE(Fixture, "typeguard_inference_incomplete") { + ScopedFastFlag luauTypeAliasPacks("LuauTypeAliasPacks", true); + const std::string code = R"( function f(a) if type(a) == "boolean" then @@ -41,11 +43,11 @@ TEST_CASE_FIXTURE(Fixture, "typeguard_inference_incomplete") )"; const std::string expected = R"( - function f(a:{fn:()->(free)}): () + function f(a:{fn:()->(free,free...)}): () if type(a) == 'boolean'then local a1:boolean=a elseif a.fn()then - local a2:{fn:()->(free)}=a + local a2:{fn:()->(free,free...)}=a end end )"; @@ -231,16 +233,7 @@ TEST_CASE_FIXTURE(Fixture, "operator_eq_completely_incompatible") local r2 = b == a )"); - if (FFlag::LuauEqConstraint) - { - LUAU_REQUIRE_NO_ERRORS(result); - } - else - { - LUAU_REQUIRE_ERROR_COUNT(2, result); - CHECK_EQ(toString(result.errors[0]), "Type '{| x: string |}?' could not be converted into 'number | string'"); - CHECK_EQ(toString(result.errors[1]), "Type 'number | string' could not be converted into '{| x: string |}?'"); - } + LUAU_REQUIRE_NO_ERRORS(result); } // Belongs in TypeInfer.refinements.test.cpp. @@ -542,6 +535,25 @@ TEST_CASE_FIXTURE(Fixture, "bail_early_on_typescript_port_of_Result_type" * doct } } +TEST_CASE_FIXTURE(Fixture, "table_subtyping_shouldn't_add_optional_properties_to_sealed_tables") +{ + CheckResult result = check(R"( + --!strict + local function setNumber(t: { p: number? }, x:number) t.p = x end + local function getString(t: { p: string? }):string return t.p or "" end + -- This shouldn't type-check! + local function oh(x:number): string + local t: {} = {} + setNumber(t, x) + return getString(t) + end + local s: string = oh(37) + )"); + + // Really this should return an error, but it doesn't + LUAU_REQUIRE_NO_ERRORS(result); +} + // Should be in TypeInfer.tables.test.cpp // It's unsound to instantiate tables containing generic methods, // since mutating properties means table properties should be invariant. diff --git a/tests/TypeInfer.refinements.test.cpp b/tests/TypeInfer.refinements.test.cpp index f2ba0ddc..31739cdc 100644 --- a/tests/TypeInfer.refinements.test.cpp +++ b/tests/TypeInfer.refinements.test.cpp @@ -1,4 +1,5 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +#include "Luau/Scope.h" #include "Luau/TypeInfer.h" #include "Fixture.h" @@ -6,7 +7,6 @@ #include "doctest.h" LUAU_FASTFLAG(LuauWeakEqConstraint) -LUAU_FASTFLAG(LuauImprovedTypeGuardPredicate2) LUAU_FASTFLAG(LuauOrPredicate) using namespace Luau; @@ -199,16 +199,8 @@ TEST_CASE_FIXTURE(Fixture, "typeguard_only_look_up_types_from_global_scope") end )"); - if (FFlag::LuauImprovedTypeGuardPredicate2) - { - LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK_EQ("Type 'number' has no overlap with 'string'", toString(result.errors[0])); - } - else - { - LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK_EQ("Type 'string' could not be converted into 'boolean'", toString(result.errors[0])); - } + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ("Type 'number' has no overlap with 'string'", toString(result.errors[0])); } TEST_CASE_FIXTURE(Fixture, "call_a_more_specific_function_using_typeguard") @@ -526,8 +518,6 @@ TEST_CASE_FIXTURE(Fixture, "narrow_property_of_a_bounded_variable") TEST_CASE_FIXTURE(Fixture, "type_narrow_to_vector") { - ScopedFastFlag sff2{"LuauImprovedTypeGuardPredicate2", true}; - CheckResult result = check(R"( local function f(x) if type(x) == "vector" then @@ -544,8 +534,6 @@ TEST_CASE_FIXTURE(Fixture, "type_narrow_to_vector") TEST_CASE_FIXTURE(Fixture, "nonoptional_type_can_narrow_to_nil_if_sense_is_true") { - ScopedFastFlag sff2{"LuauImprovedTypeGuardPredicate2", true}; - CheckResult result = check(R"( local t = {"hello"} local v = t[2] @@ -573,8 +561,6 @@ TEST_CASE_FIXTURE(Fixture, "nonoptional_type_can_narrow_to_nil_if_sense_is_true" TEST_CASE_FIXTURE(Fixture, "typeguard_not_to_be_string") { - ScopedFastFlag sff2{"LuauImprovedTypeGuardPredicate2", true}; - CheckResult result = check(R"( local function f(x: string | number | boolean) if type(x) ~= "string" then @@ -593,8 +579,6 @@ TEST_CASE_FIXTURE(Fixture, "typeguard_not_to_be_string") TEST_CASE_FIXTURE(Fixture, "typeguard_narrows_for_table") { - ScopedFastFlag sff2{"LuauImprovedTypeGuardPredicate2", true}; - CheckResult result = check(R"( local function f(x: string | {x: number} | {y: boolean}) if type(x) == "table" then @@ -613,8 +597,6 @@ TEST_CASE_FIXTURE(Fixture, "typeguard_narrows_for_table") TEST_CASE_FIXTURE(Fixture, "typeguard_narrows_for_functions") { - ScopedFastFlag sff2{"LuauImprovedTypeGuardPredicate2", true}; - CheckResult result = check(R"( local function weird(x: string | ((number) -> string)) if type(x) == "function" then @@ -698,8 +680,6 @@ struct RefinementClassFixture : Fixture TEST_CASE_FIXTURE(RefinementClassFixture, "typeguard_cast_free_table_to_vector") { - ScopedFastFlag sff2{"LuauImprovedTypeGuardPredicate2", true}; - CheckResult result = check(R"( local function f(vec) local X, Y, Z = vec.X, vec.Y, vec.Z @@ -726,8 +706,6 @@ TEST_CASE_FIXTURE(RefinementClassFixture, "typeguard_cast_free_table_to_vector") TEST_CASE_FIXTURE(RefinementClassFixture, "typeguard_cast_instance_or_vector3_to_vector") { - ScopedFastFlag sff2{"LuauImprovedTypeGuardPredicate2", true}; - CheckResult result = check(R"( local function f(x: Instance | Vector3) if typeof(x) == "Vector3" then @@ -746,8 +724,6 @@ TEST_CASE_FIXTURE(RefinementClassFixture, "typeguard_cast_instance_or_vector3_to TEST_CASE_FIXTURE(RefinementClassFixture, "type_narrow_for_all_the_userdata") { - ScopedFastFlag sff2{"LuauImprovedTypeGuardPredicate2", true}; - CheckResult result = check(R"( local function f(x: string | number | Instance | Vector3) if type(x) == "userdata" then @@ -766,10 +742,7 @@ TEST_CASE_FIXTURE(RefinementClassFixture, "type_narrow_for_all_the_userdata") TEST_CASE_FIXTURE(RefinementClassFixture, "eliminate_subclasses_of_instance") { - ScopedFastFlag sffs[] = { - {"LuauImprovedTypeGuardPredicate2", true}, - {"LuauTypeGuardPeelsAwaySubclasses", true}, - }; + ScopedFastFlag sff{"LuauTypeGuardPeelsAwaySubclasses", true}; CheckResult result = check(R"( local function f(x: Part | Folder | string) @@ -789,10 +762,7 @@ TEST_CASE_FIXTURE(RefinementClassFixture, "eliminate_subclasses_of_instance") TEST_CASE_FIXTURE(RefinementClassFixture, "narrow_this_large_union") { - ScopedFastFlag sffs[] = { - {"LuauImprovedTypeGuardPredicate2", true}, - {"LuauTypeGuardPeelsAwaySubclasses", true}, - }; + ScopedFastFlag sff{"LuauTypeGuardPeelsAwaySubclasses", true}; CheckResult result = check(R"( local function f(x: Part | Folder | Instance | string | Vector3 | any) @@ -812,10 +782,7 @@ TEST_CASE_FIXTURE(RefinementClassFixture, "narrow_this_large_union") TEST_CASE_FIXTURE(RefinementClassFixture, "x_as_any_if_x_is_instance_elseif_x_is_table") { - ScopedFastFlag sffs[] = { - {"LuauOrPredicate", true}, - {"LuauImprovedTypeGuardPredicate2", true}, - }; + ScopedFastFlag sff{"LuauOrPredicate", true}; CheckResult result = check(R"( --!nonstrict @@ -839,7 +806,6 @@ TEST_CASE_FIXTURE(RefinementClassFixture, "x_is_not_instance_or_else_not_part") { ScopedFastFlag sffs[] = { {"LuauOrPredicate", true}, - {"LuauImprovedTypeGuardPredicate2", true}, {"LuauTypeGuardPeelsAwaySubclasses", true}, }; @@ -861,8 +827,6 @@ TEST_CASE_FIXTURE(RefinementClassFixture, "x_is_not_instance_or_else_not_part") TEST_CASE_FIXTURE(Fixture, "type_guard_can_filter_for_intersection_of_tables") { - ScopedFastFlag sff2{"LuauImprovedTypeGuardPredicate2", true}; - CheckResult result = check(R"( type XYCoord = {x: number} & {y: number} local function f(t: XYCoord?) @@ -882,8 +846,6 @@ TEST_CASE_FIXTURE(Fixture, "type_guard_can_filter_for_intersection_of_tables") TEST_CASE_FIXTURE(Fixture, "type_guard_can_filter_for_overloaded_function") { - ScopedFastFlag sff2{"LuauImprovedTypeGuardPredicate2", true}; - CheckResult result = check(R"( type SomeOverloadedFunction = ((number) -> string) & ((string) -> number) local function f(g: SomeOverloadedFunction?) @@ -903,8 +865,6 @@ TEST_CASE_FIXTURE(Fixture, "type_guard_can_filter_for_overloaded_function") TEST_CASE_FIXTURE(Fixture, "type_guard_warns_on_no_overlapping_types_only_when_sense_is_true") { - ScopedFastFlag sff2{"LuauImprovedTypeGuardPredicate2", true}; - CheckResult result = check(R"( local function f(t: {x: number}) if type(t) ~= "table" then @@ -999,10 +959,7 @@ TEST_CASE_FIXTURE(Fixture, "not_a_and_not_b2") TEST_CASE_FIXTURE(Fixture, "either_number_or_string") { - ScopedFastFlag sffs[] = { - {"LuauOrPredicate", true}, - {"LuauImprovedTypeGuardPredicate2", true}, - }; + ScopedFastFlag sff{"LuauOrPredicate", true}; CheckResult result = check(R"( local function f(x: any) @@ -1036,10 +993,7 @@ TEST_CASE_FIXTURE(Fixture, "not_t_or_some_prop_of_t") TEST_CASE_FIXTURE(Fixture, "assert_a_to_be_truthy_then_assert_a_to_be_number") { - ScopedFastFlag sffs[] = { - {"LuauOrPredicate", true}, - {"LuauImprovedTypeGuardPredicate2", true}, - }; + ScopedFastFlag sff{"LuauOrPredicate", true}; CheckResult result = check(R"( local a: (number | string)? @@ -1057,10 +1011,7 @@ TEST_CASE_FIXTURE(Fixture, "assert_a_to_be_truthy_then_assert_a_to_be_number") TEST_CASE_FIXTURE(Fixture, "merge_should_be_fully_agnostic_of_hashmap_ordering") { - ScopedFastFlag sffs[] = { - {"LuauOrPredicate", true}, - {"LuauImprovedTypeGuardPredicate2", true}, - }; + ScopedFastFlag sff{"LuauOrPredicate", true}; // This bug came up because there was a mistake in Luau::merge where zipping on two maps would produce the wrong merged result. CheckResult result = check(R"( @@ -1081,10 +1032,7 @@ TEST_CASE_FIXTURE(Fixture, "merge_should_be_fully_agnostic_of_hashmap_ordering") TEST_CASE_FIXTURE(Fixture, "refine_the_correct_types_opposite_of_when_a_is_not_number_or_string") { - ScopedFastFlag sffs[] = { - {"LuauOrPredicate", true}, - {"LuauImprovedTypeGuardPredicate2", true}, - }; + ScopedFastFlag sff{"LuauOrPredicate", true}; CheckResult result = check(R"( local function f(a: string | number | boolean) diff --git a/tests/TypeInfer.tables.test.cpp b/tests/TypeInfer.tables.test.cpp index 1d1b2fae..b7f0dc7b 100644 --- a/tests/TypeInfer.tables.test.cpp +++ b/tests/TypeInfer.tables.test.cpp @@ -46,6 +46,21 @@ TEST_CASE_FIXTURE(Fixture, "augment_table") CHECK(tType->props.find("foo") != tType->props.end()); } +TEST_CASE_FIXTURE(Fixture, "augment_nested_table") +{ + CheckResult result = check("local t = { p = {} } t.p.foo = 'bar'"); + LUAU_REQUIRE_NO_ERRORS(result); + + TableTypeVar* tType = getMutable(requireType("t")); + REQUIRE(tType != nullptr); + + REQUIRE(tType->props.find("p") != tType->props.end()); + const TableTypeVar* pType = get(tType->props["p"].type); + REQUIRE(pType != nullptr); + + CHECK(pType->props.find("foo") != pType->props.end()); +} + TEST_CASE_FIXTURE(Fixture, "cannot_augment_sealed_table") { CheckResult result = check("local t = {prop=999} t.foo = 'bar'"); @@ -260,6 +275,8 @@ TEST_CASE_FIXTURE(Fixture, "open_table_unification") TEST_CASE_FIXTURE(Fixture, "open_table_unification_2") { + ScopedFastFlag sff{"LuauTableSubtypingVariance", true}; + CheckResult result = check(R"( local a = {} a.x = 99 @@ -272,10 +289,11 @@ TEST_CASE_FIXTURE(Fixture, "open_table_unification_2") LUAU_REQUIRE_ERROR_COUNT(1, result); TypeError& err = result.errors[0]; - UnknownProperty* error = get(err); + MissingProperties* error = get(err); REQUIRE(error != nullptr); + REQUIRE(error->properties.size() == 1); - CHECK_EQ(error->key, "y"); + CHECK_EQ("y", error->properties[0]); // TODO(rblanckaert): Revist when we can bind self at function creation time // CHECK_EQ(err.location, Location(Position{5, 19}, Position{5, 25})); @@ -328,6 +346,8 @@ TEST_CASE_FIXTURE(Fixture, "table_param_row_polymorphism_1") TEST_CASE_FIXTURE(Fixture, "table_param_row_polymorphism_2") { + ScopedFastFlag sff{"LuauTableSubtypingVariance", true}; + CheckResult result = check(R"( --!strict function foo(o) @@ -340,14 +360,17 @@ TEST_CASE_FIXTURE(Fixture, "table_param_row_polymorphism_2") LUAU_REQUIRE_ERROR_COUNT(1, result); - UnknownProperty* error = get(result.errors[0]); + MissingProperties* error = get(result.errors[0]); REQUIRE(error != nullptr); + REQUIRE(error->properties.size() == 1); - CHECK_EQ("baz", error->key); + CHECK_EQ("baz", error->properties[0]); } TEST_CASE_FIXTURE(Fixture, "table_param_row_polymorphism_3") { + ScopedFastFlag sff{"LuauTableSubtypingVariance", true}; + CheckResult result = check(R"( local T = {} T.bar = 'hello' @@ -359,8 +382,11 @@ TEST_CASE_FIXTURE(Fixture, "table_param_row_polymorphism_3") LUAU_REQUIRE_ERROR_COUNT(1, result); TypeError& err = result.errors[0]; - UnknownProperty* error = get(err); + MissingProperties* error = get(err); REQUIRE(error != nullptr); + REQUIRE(error->properties.size() == 1); + + CHECK_EQ("baz", error->properties[0]); // TODO(rblanckaert): Revist when we can bind self at function creation time /* @@ -448,6 +474,73 @@ TEST_CASE_FIXTURE(Fixture, "ok_to_add_property_to_free_table") dumpErrors(result); } +TEST_CASE_FIXTURE(Fixture, "okay_to_add_property_to_unsealed_tables_by_assignment") +{ + ScopedFastFlag sff{"LuauTableSubtypingVariance", true}; + + CheckResult result = check(R"( + --!strict + local t = { u = {} } + t = { u = { p = 37 } } + t = { u = { q = "hi" } } + local x = t.u.p + local y = t.u.q + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + CHECK_EQ("number?", toString(requireType("x"))); + CHECK_EQ("string?", toString(requireType("y"))); +} + +TEST_CASE_FIXTURE(Fixture, "okay_to_add_property_to_unsealed_tables_by_function_call") +{ + CheckResult result = check(R"( + --!strict + function get(x) return x.opts["MYOPT"] end + function set(x,y) x.opts["MYOPT"] = y end + local t = { opts = {} } + set(t,37) + local x = get(t) + )"); + + // Currently this errors but it shouldn't, since set only needs write access + // TODO: file a JIRA for this + LUAU_REQUIRE_ERRORS(result); + // CHECK_EQ("number?", toString(requireType("x"))); +} + +TEST_CASE_FIXTURE(Fixture, "width_subtyping") +{ + ScopedFastFlag sff{"LuauTableSubtypingVariance", true}; + + CheckResult result = check(R"( + --!strict + function f(x : { q : number }) + x.q = 8 + end + local t : { q : number, r : string } = { q = 8, r = "hi" } + f(t) + local x : string = t.r + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "width_subtyping_needs_covariance") +{ + CheckResult result = check(R"( + --!strict + function f(x : { p : { q : number }}) + x.p = { q = 8, r = 5 } + end + local t : { p : { q : number, r : string } } = { p = { q = 8, r = "hi" } } + f(t) -- Shouldn't typecheck + local x : string = t.p.r -- x is 5 + )"); + + LUAU_REQUIRE_ERRORS(result); +} + TEST_CASE_FIXTURE(Fixture, "infer_array") { CheckResult result = check(R"( @@ -676,16 +769,27 @@ TEST_CASE_FIXTURE(Fixture, "infer_indexer_for_left_unsealed_table_from_right_han LUAU_REQUIRE_NO_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "sealed_table_value_must_not_infer_an_indexer") +TEST_CASE_FIXTURE(Fixture, "sealed_table_value_can_infer_an_indexer") { + ScopedFastFlag sff{"LuauTableSubtypingVariance", true}; + CheckResult result = check(R"( local t: { a: string, [number]: string } = { a = "foo" } )"); - LUAU_REQUIRE_ERROR_COUNT(1, result); + LUAU_REQUIRE_NO_ERRORS(result); +} - TypeMismatch* tm = get(result.errors[0]); - REQUIRE(tm != nullptr); +TEST_CASE_FIXTURE(Fixture, "array_factory_function") +{ + ScopedFastFlag sff{"LuauTableSubtypingVariance", true}; + + CheckResult result = check(R"( + function empty() return {} end + local array: {string} = empty() + )"); + + LUAU_REQUIRE_NO_ERRORS(result); } TEST_CASE_FIXTURE(Fixture, "sealed_table_indexers_must_unify") @@ -756,37 +860,6 @@ TEST_CASE_FIXTURE(Fixture, "indexing_from_a_table_should_prefer_properties_when_ CHECK_MESSAGE(nullptr != get(result.errors[0]), "Expected a TypeMismatch but got " << result.errors[0]); } -TEST_CASE_FIXTURE(Fixture, "indexing_from_a_table_with_a_string") -{ - ScopedFastFlag fflag("LuauIndexTablesWithIndexers", true); - - CheckResult result = check(R"( - local t: { a: string } - function f(x: string) return t[x] end - local a = f("a") - local b = f("b") - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - - CHECK_EQ(*typeChecker.anyType, *requireType("a")); - CHECK_EQ(*typeChecker.anyType, *requireType("b")); -} - -TEST_CASE_FIXTURE(Fixture, "indexing_from_a_table_with_a_number") -{ - ScopedFastFlag fflag("LuauIndexTablesWithIndexers", true); - - CheckResult result = check(R"( - local t = { a = true } - function f(x: number) return t[x] end - )"); - - LUAU_REQUIRE_ERROR_COUNT(1, result); - - CHECK_MESSAGE(nullptr != get(result.errors[0]), "Expected a TypeMismatch but got " << result.errors[0]); -} - TEST_CASE_FIXTURE(Fixture, "assigning_to_an_unsealed_table_with_string_literal_should_infer_new_properties_over_indexer") { CheckResult result = check(R"( @@ -1392,6 +1465,8 @@ TEST_CASE_FIXTURE(Fixture, "casting_tables_with_props_into_table_with_indexer2") TEST_CASE_FIXTURE(Fixture, "casting_tables_with_props_into_table_with_indexer3") { + ScopedFastFlag sff{"LuauTableSubtypingVariance", true}; + CheckResult result = check(R"( local function foo(a: {[string]: number, a: string}) end foo({ a = 1 }) @@ -1402,8 +1477,21 @@ TEST_CASE_FIXTURE(Fixture, "casting_tables_with_props_into_table_with_indexer3") ToStringOptions o{/* exhaustive= */ true}; TypeMismatch* tm = get(result.errors[0]); REQUIRE(tm); - CHECK_EQ("string", toString(tm->wantedType, o)); - CHECK_EQ("number", toString(tm->givenType, o)); + CHECK_EQ("{| [string]: number, a: string |}", toString(tm->wantedType, o)); + CHECK_EQ("{| a: number |}", toString(tm->givenType, o)); +} + +TEST_CASE_FIXTURE(Fixture, "casting_tables_with_props_into_table_with_indexer4") +{ + CheckResult result = check(R"( + local function foo(a: {[string]: number, a: string}, i: string) + return a[i] + end + local hi: number = foo({ a = "hi" }, "a") -- shouldn't typecheck since at runtime hi is "hi" + )"); + + // This typechecks but shouldn't + LUAU_REQUIRE_NO_ERRORS(result); } TEST_CASE_FIXTURE(Fixture, "table_subtyping_with_missing_props_dont_report_multiple_errors") @@ -1446,22 +1534,32 @@ TEST_CASE_FIXTURE(Fixture, "table_subtyping_with_missing_props_dont_report_multi TEST_CASE_FIXTURE(Fixture, "table_subtyping_with_extra_props_dont_report_multiple_errors") { CheckResult result = check(R"( - local vec3 = {x = 1, y = 2, z = 3} - local vec1 = {x = 1} + local vec3 = {{x = 1, y = 2, z = 3}} + local vec1 = {{x = 1}} vec1 = vec3 )"); LUAU_REQUIRE_ERROR_COUNT(1, result); - MissingProperties* mp = get(result.errors[0]); - REQUIRE(mp); - CHECK_EQ(mp->context, MissingProperties::Extra); - REQUIRE_EQ(2, mp->properties.size()); - CHECK_EQ(mp->properties[0], "y"); - CHECK_EQ(mp->properties[1], "z"); - CHECK_EQ("vec1", toString(mp->superType)); - CHECK_EQ("vec3", toString(mp->subType)); + TypeMismatch* tm = get(result.errors[0]); + REQUIRE(tm); + CHECK_EQ("vec1", toString(tm->wantedType)); + CHECK_EQ("vec3", toString(tm->givenType)); +} + +TEST_CASE_FIXTURE(Fixture, "table_subtyping_with_extra_props_is_ok") +{ + ScopedFastFlag sff{"LuauTableSubtypingVariance", true}; + + CheckResult result = check(R"( + local vec3 = {x = 1, y = 2, z = 3} + local vec1 = {x = 1} + + vec1 = vec3 + )"); + + LUAU_REQUIRE_NO_ERRORS(result); } TEST_CASE_FIXTURE(Fixture, "type_mismatch_on_massive_table_is_cut_short") @@ -1824,4 +1922,32 @@ TEST_CASE_FIXTURE(Fixture, "invariant_table_properties_means_instantiating_table LUAU_REQUIRE_ERRORS(result); } +TEST_CASE_FIXTURE(Fixture, "table_insert_should_cope_with_optional_properties_in_nonstrict") +{ + CheckResult result = check(R"( + --!nonstrict + local buttons = {} + table.insert(buttons, { a = 1 }) + table.insert(buttons, { a = 2, b = true }) + table.insert(buttons, { a = 3 }) + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "table_insert_should_cope_with_optional_properties_in_strict") +{ + ScopedFastFlag sff{"LuauTableSubtypingVariance", true}; + + CheckResult result = check(R"( + --!strict + local buttons = {} + table.insert(buttons, { a = 1 }) + table.insert(buttons, { a = 2, b = true }) + table.insert(buttons, { a = 3 }) + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.test.cpp b/tests/TypeInfer.test.cpp index 37333b19..b75878b7 100644 --- a/tests/TypeInfer.test.cpp +++ b/tests/TypeInfer.test.cpp @@ -3,6 +3,7 @@ #include "Luau/AstQuery.h" #include "Luau/BuiltinDefinitions.h" #include "Luau/Parser.h" +#include "Luau/Scope.h" #include "Luau/TypeInfer.h" #include "Luau/TypeVar.h" #include "Luau/VisitTypeVar.h" @@ -978,23 +979,6 @@ TEST_CASE_FIXTURE(Fixture, "cyclic_function_type_in_args") CHECK_EQ("t1 where t1 = (t1) -> ()", toString(requireType("f"))); } -TEST_CASE_FIXTURE(Fixture, "cyclic_function_type_in_type_alias") -{ - ScopedFastFlag sff{"LuauOccursCheckOkWithRecursiveFunctions", true}; - - CheckResult result = check(R"( - type F = () -> F? - local function f() - return f - end - - local g: F = f - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - CHECK_EQ("t1 where t1 = () -> t1?", toString(requireType("g"))); -} - // TODO: File a Jira about this /* TEST_CASE_FIXTURE(Fixture, "unifying_vararg_pack_with_fixed_length_pack_produces_fixed_length_pack") @@ -1257,23 +1241,6 @@ TEST_CASE_FIXTURE(Fixture, "instantiate_cyclic_generic_function") REQUIRE_EQ(follow(*methodArg), follow(arg)); } -TEST_CASE_FIXTURE(Fixture, "cyclic_types_of_named_table_fields_do_not_expand_when_stringified") -{ - CheckResult result = check(R"( - --!strict - type Node = { Parent: Node?; } - local node: Node; - node.Parent = 1 - )"); - - LUAU_REQUIRE_ERROR_COUNT(1, result); - - TypeMismatch* tm = get(result.errors[0]); - REQUIRE(tm); - CHECK_EQ("Node?", toString(tm->wantedType)); - CHECK_EQ(typeChecker.numberType, tm->givenType); -} - TEST_CASE_FIXTURE(Fixture, "varlist_declared_by_for_in_loop_should_be_free") { CheckResult result = check(R"( @@ -2591,48 +2558,6 @@ TEST_CASE_FIXTURE(Fixture, "toposort_doesnt_break_mutual_recursion") dumpErrors(result); } -TEST_CASE_FIXTURE(Fixture, "mutually_recursive_types") -{ - CheckResult result = check(R"( - --!strict - type T = { f: a, g: U } - type U = { h: a, i: T? } - local x: T = { f = 37, g = { h = 5, i = nil } } - x.g.i = x - local y: T = { f = "hi", g = { h = "lo", i = nil } } - y.g.i = y - )"); - - LUAU_REQUIRE_NO_ERRORS(result); -} - -TEST_CASE_FIXTURE(Fixture, "mutually_recursive_types_errors") -{ - CheckResult result = check(R"( - --!strict - type T = { f: a, g: U } - type U = { h: b, i: T? } - local x: T = { f = 37, g = { h = 5, i = nil } } - x.g.i = x - local y: T = { f = "hi", g = { h = 5, i = nil } } - y.g.i = y - )"); - - LUAU_REQUIRE_ERRORS(result); - - // We had a UAF in this example caused by not cloning type function arguments - ModulePtr module = frontend.moduleResolver.getModule("MainModule"); - unfreeze(module->interfaceTypes); - copyErrors(module->errors, module->interfaceTypes); - freeze(module->interfaceTypes); - module->internalTypes.clear(); - module->astTypes.clear(); - - // Make sure the error strings don't include "VALUELESS" - for (auto error : module->errors) - CHECK_MESSAGE(toString(error).find("VALUELESS") == std::string::npos, toString(error)); -} - TEST_CASE_FIXTURE(Fixture, "object_constructor_can_refer_to_method_of_self") { // CLI-30902 @@ -3369,16 +3294,7 @@ TEST_CASE_FIXTURE(Fixture, "unknown_type_in_comparison") end )"); - if (FFlag::LuauEqConstraint) - { - LUAU_REQUIRE_NO_ERRORS(result); - } - else - { - LUAU_REQUIRE_ERROR_COUNT(1, result); - - REQUIRE(get(result.errors[0])); - } + LUAU_REQUIRE_NO_ERRORS(result); } TEST_CASE_FIXTURE(Fixture, "relation_op_on_any_lhs_where_rhs_maybe_has_metatable") @@ -3388,18 +3304,8 @@ TEST_CASE_FIXTURE(Fixture, "relation_op_on_any_lhs_where_rhs_maybe_has_metatable print((x == true and (x .. "y")) .. 1) )"); - if (FFlag::LuauEqConstraint) - { - LUAU_REQUIRE_ERROR_COUNT(1, result); - REQUIRE(get(result.errors[0])); - } - else - { - LUAU_REQUIRE_ERROR_COUNT(2, result); - - CHECK_EQ("Type 'boolean' could not be converted into 'number | string'", toString(result.errors[0])); - CHECK_EQ("Type 'boolean | string' could not be converted into 'number | string'", toString(result.errors[1])); - } + LUAU_REQUIRE_ERROR_COUNT(1, result); + REQUIRE(get(result.errors[0])); } TEST_CASE_FIXTURE(Fixture, "concat_op_on_string_lhs_and_free_rhs") @@ -3511,25 +3417,6 @@ _(...)(...,setfenv,_):_G() )"); } -TEST_CASE_FIXTURE(Fixture, "use_table_name_and_generic_params_in_errors") -{ - CheckResult result = check(R"( - type Pair = {first: T, second: U} - local a: Pair - local b: Pair - - a = b - )"); - - LUAU_REQUIRE_ERROR_COUNT(1, result); - - TypeMismatch* tm = get(result.errors[0]); - REQUIRE(tm); - - CHECK_EQ("Pair", toString(tm->wantedType)); - CHECK_EQ("Pair", toString(tm->givenType)); -} - TEST_CASE_FIXTURE(Fixture, "cyclic_type_packs") { // this has a risk of creating cyclic type packs, causing infinite loops / OOMs @@ -3639,17 +3526,6 @@ TEST_CASE_FIXTURE(Fixture, "for_in_loop_where_iteratee_is_free") )"); } -TEST_CASE_FIXTURE(Fixture, "dont_stop_typechecking_after_reporting_duplicate_type_definition") -{ - CheckResult result = check(R"( - type A = number - type A = string -- Redefinition of type 'A', previously defined at line 1 - local foo: string = 1 -- No "Type 'number' could not be converted into 'string'" - )"); - - LUAU_REQUIRE_ERROR_COUNT(2, result); -} - TEST_CASE_FIXTURE(Fixture, "tc_after_error_recovery") { CheckResult result = check(R"( @@ -3752,38 +3628,6 @@ TEST_CASE_FIXTURE(Fixture, "error_on_invalid_operand_types_to_relational_operato CHECK_EQ("Type 'number | string' cannot be compared with relational operator <", ge->message); } -TEST_CASE_FIXTURE(Fixture, "stringify_type_alias_of_recursive_template_table_type") -{ - CheckResult result = check(R"( - type Table = { a: T } - type Wrapped = Table - local l: Wrapped = 2 - )"); - - LUAU_REQUIRE_ERROR_COUNT(1, result); - - TypeMismatch* tm = get(result.errors[0]); - REQUIRE(tm); - CHECK_EQ("Wrapped", toString(tm->wantedType)); - CHECK_EQ(typeChecker.numberType, tm->givenType); -} - -TEST_CASE_FIXTURE(Fixture, "stringify_type_alias_of_recursive_template_table_type2") -{ - CheckResult result = check(R"( - type Table = { a: T } - type Wrapped = (Table) -> string - local l: Wrapped = 2 - )"); - - LUAU_REQUIRE_ERROR_COUNT(1, result); - - TypeMismatch* tm = get(result.errors[0]); - REQUIRE(tm); - CHECK_EQ("t1 where t1 = ({| a: t1 |}) -> string", toString(tm->wantedType)); - CHECK_EQ(typeChecker.numberType, tm->givenType); -} - TEST_CASE_FIXTURE(Fixture, "index_expr_should_be_checked") { CheckResult result = check(R"( @@ -3909,19 +3753,6 @@ TEST_CASE_FIXTURE(Fixture, "stringify_nested_unions_with_optionals") CHECK_EQ("(boolean | number | string)?", toString(tm->givenType)); } -// Check that recursive intersection type doesn't generate an OOM -TEST_CASE_FIXTURE(Fixture, "cli_38393_recursive_intersection_oom") -{ - CheckResult result = check(R"( - function _(l0:(t0)&((t0)&(((t0)&((t0)->()))->(typeof(_),typeof(# _)))),l39,...):any - end - type t0 = ((typeof(_))&((t0)&(((typeof(_))&(t0))->typeof(_))),{n163:any,})->(any,typeof(_)) - _(_) - )"); - - LUAU_REQUIRE_ERRORS(result); -} - TEST_CASE_FIXTURE(Fixture, "UnknownGlobalCompoundAssign") { // In non-strict mode, global definition is still allowed @@ -3974,16 +3805,6 @@ TEST_CASE_FIXTURE(Fixture, "loop_typecheck_crash_on_empty_optional") LUAU_REQUIRE_ERROR_COUNT(2, result); } -TEST_CASE_FIXTURE(Fixture, "type_alias_fwd_declaration_is_precise") -{ - CheckResult result = check(R"( - local foo: Id = 1 - type Id = T - )"); - - LUAU_REQUIRE_NO_ERRORS(result); -} - TEST_CASE_FIXTURE(Fixture, "cli_39932_use_unifier_in_ensure_methods") { CheckResult result = check(R"( @@ -4014,81 +3835,6 @@ end LUAU_REQUIRE_NO_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "corecursive_types_generic") -{ - const std::string code = R"( - type A = {v:T, b:B} - type B = {v:T, a:A} - local aa:A - local bb = aa - )"; - - const std::string expected = R"( - type A = {v:T, b:B} - type B = {v:T, a:A} - local aa:A - local bb:A=aa - )"; - - CHECK_EQ(expected, decorateWithTypes(code)); - CheckResult result = check(code); - - LUAU_REQUIRE_NO_ERRORS(result); -} - -TEST_CASE_FIXTURE(Fixture, "corecursive_function_types") -{ - ScopedFastFlag sff{"LuauOccursCheckOkWithRecursiveFunctions", true}; - - CheckResult result = check(R"( - type A = () -> (number, B) - type B = () -> (string, A) - local a: A - local b: B - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - - CHECK_EQ("t1 where t1 = () -> (number, () -> (string, t1))", toString(requireType("a"))); - CHECK_EQ("t1 where t1 = () -> (string, () -> (number, t1))", toString(requireType("b"))); -} - -TEST_CASE_FIXTURE(Fixture, "generic_param_remap") -{ - const std::string code = R"( - -- An example of a forwarded use of a type that has different type arguments than parameters - type A = {t:T, u:U, next:A?} - local aa:A = { t = 5, u = 'hi', next = { t = 'lo', u = 8 } } - local bb = aa - )"; - - const std::string expected = R"( - - type A = {t:T, u:U, next:A?} - local aa:A = { t = 5, u = 'hi', next = { t = 'lo', u = 8 } } - local bb:A=aa - )"; - - CHECK_EQ(expected, decorateWithTypes(code)); - CheckResult result = check(code); - - LUAU_REQUIRE_ERRORS(result); -} - -TEST_CASE_FIXTURE(Fixture, "export_type_and_type_alias_are_duplicates") -{ - CheckResult result = check(R"( - export type Foo = number - type Foo = number - )"); - - LUAU_REQUIRE_ERROR_COUNT(1, result); - - auto dtd = get(result.errors[0]); - REQUIRE(dtd); - CHECK_EQ(dtd->name, "Foo"); -} - TEST_CASE_FIXTURE(Fixture, "dont_report_type_errors_within_an_AstStatError") { CheckResult result = check(R"( @@ -4193,30 +3939,6 @@ TEST_CASE_FIXTURE(Fixture, "luau_resolves_symbols_the_same_way_lua_does") REQUIRE_MESSAGE(get(e) != nullptr, "Expected UnknownSymbol, but got " << e); } -TEST_CASE_FIXTURE(Fixture, "stringify_optional_parameterized_alias") -{ - ScopedFastFlag sffs3{"LuauGenericFunctions", true}; - ScopedFastFlag sffs4{"LuauParseGenericFunctions", true}; - - CheckResult result = check(R"( - type Node = { value: T, child: Node? } - - local function visitor(node: Node?) - local a: Node - - if node then - a = node.child -- Observe the output of the error message. - end - end - )"); - - LUAU_REQUIRE_ERROR_COUNT(1, result); - - auto e = get(result.errors[0]); - CHECK_EQ("Node?", toString(e->givenType)); - CHECK_EQ("Node", toString(e->wantedType)); -} - TEST_CASE_FIXTURE(Fixture, "operator_eq_verifies_types_do_intersect") { CheckResult result = check(R"( @@ -4272,181 +3994,6 @@ local tbl: string = require(game.A) CHECK_EQ("Type '{| def: number |}' could not be converted into 'string'", toString(result.errors[0])); } -TEST_CASE_FIXTURE(Fixture, "general_require_multi_assign") -{ - fileResolver.source["workspace/A"] = R"( - export type myvec2 = {x: number, y: number} - return {} - )"; - - fileResolver.source["workspace/B"] = R"( - export type myvec3 = {x: number, y: number, z: number} - return {} - )"; - - fileResolver.source["workspace/C"] = R"( - local Foo, Bar = require(workspace.A), require(workspace.B) - - local a: Foo.myvec2 - local b: Bar.myvec3 - )"; - - CheckResult result = frontend.check("workspace/C"); - LUAU_REQUIRE_NO_ERRORS(result); - ModulePtr m = frontend.moduleResolver.modules["workspace/C"]; - - REQUIRE(m != nullptr); - - std::optional aTypeId = lookupName(m->getModuleScope(), "a"); - REQUIRE(aTypeId); - const Luau::TableTypeVar* aType = get(follow(*aTypeId)); - REQUIRE(aType); - REQUIRE(aType->props.size() == 2); - - std::optional bTypeId = lookupName(m->getModuleScope(), "b"); - REQUIRE(bTypeId); - const Luau::TableTypeVar* bType = get(follow(*bTypeId)); - REQUIRE(bType); - REQUIRE(bType->props.size() == 3); -} - -TEST_CASE_FIXTURE(Fixture, "type_alias_import_mutation") -{ - CheckResult result = check("type t10 = typeof(table)"); - LUAU_REQUIRE_NO_ERRORS(result); - - TypeId ty = getGlobalBinding(frontend.typeChecker, "table"); - CHECK_EQ(toString(ty), "table"); - - const TableTypeVar* ttv = get(ty); - REQUIRE(ttv); - - CHECK(ttv->instantiatedTypeParams.empty()); -} - -TEST_CASE_FIXTURE(Fixture, "type_alias_local_mutation") -{ - CheckResult result = check(R"( -type Cool = { a: number, b: string } -local c: Cool = { a = 1, b = "s" } -type NotCool = Cool -)"); - LUAU_REQUIRE_NO_ERRORS(result); - - std::optional ty = requireType("c"); - REQUIRE(ty); - CHECK_EQ(toString(*ty), "Cool"); - - const TableTypeVar* ttv = get(*ty); - REQUIRE(ttv); - - CHECK(ttv->instantiatedTypeParams.empty()); -} - -TEST_CASE_FIXTURE(Fixture, "type_alias_local_rename") -{ - CheckResult result = check(R"( -type Cool = { a: number, b: string } -type NotCool = Cool -local c: Cool = { a = 1, b = "s" } -local d: NotCool = { a = 1, b = "s" } -)"); - LUAU_REQUIRE_NO_ERRORS(result); - - std::optional ty = requireType("c"); - REQUIRE(ty); - CHECK_EQ(toString(*ty), "Cool"); - - ty = requireType("d"); - REQUIRE(ty); - CHECK_EQ(toString(*ty), "NotCool"); -} - -TEST_CASE_FIXTURE(Fixture, "type_alias_local_synthetic_mutation") -{ - CheckResult result = check(R"( -local c = { a = 1, b = "s" } -type Cool = typeof(c) -)"); - LUAU_REQUIRE_NO_ERRORS(result); - - std::optional ty = requireType("c"); - REQUIRE(ty); - - const TableTypeVar* ttv = get(*ty); - REQUIRE(ttv); - CHECK_EQ(ttv->name, "Cool"); -} - -TEST_CASE_FIXTURE(Fixture, "type_alias_of_an_imported_recursive_type") -{ - ScopedFastFlag luauFixTableTypeAliasClone{"LuauFixTableTypeAliasClone", true}; - - fileResolver.source["game/A"] = R"( -export type X = { a: number, b: X? } -return {} - )"; - - CheckResult aResult = frontend.check("game/A"); - LUAU_REQUIRE_NO_ERRORS(aResult); - - CheckResult bResult = check(R"( -local Import = require(game.A) -type X = Import.X - )"); - LUAU_REQUIRE_NO_ERRORS(bResult); - - std::optional ty1 = lookupImportedType("Import", "X"); - REQUIRE(ty1); - - std::optional ty2 = lookupType("X"); - REQUIRE(ty2); - - CHECK_EQ(follow(*ty1), follow(*ty2)); -} - -TEST_CASE_FIXTURE(Fixture, "type_alias_of_an_imported_recursive_generic_type") -{ - ScopedFastFlag luauFixTableTypeAliasClone{"LuauFixTableTypeAliasClone", true}; - - fileResolver.source["game/A"] = R"( -export type X = { a: T, b: U, C: X? } -return {} - )"; - - CheckResult aResult = frontend.check("game/A"); - LUAU_REQUIRE_NO_ERRORS(aResult); - - CheckResult bResult = check(R"( -local Import = require(game.A) -type X = Import.X - )"); - LUAU_REQUIRE_NO_ERRORS(bResult); - - std::optional ty1 = lookupImportedType("Import", "X"); - REQUIRE(ty1); - - std::optional ty2 = lookupType("X"); - REQUIRE(ty2); - - CHECK_EQ(toString(*ty1, {true}), toString(*ty2, {true})); - - bResult = check(R"( -local Import = require(game.A) -type X = Import.X - )"); - LUAU_REQUIRE_NO_ERRORS(bResult); - - ty1 = lookupImportedType("Import", "X"); - REQUIRE(ty1); - - ty2 = lookupType("X"); - REQUIRE(ty2); - - CHECK_EQ(toString(*ty1, {true}), "t1 where t1 = {| C: t1?, a: T, b: U |}"); - CHECK_EQ(toString(*ty2, {true}), "{| C: t1, a: U, b: T |} where t1 = {| C: t1, a: U, b: T |}?"); -} - TEST_CASE_FIXTURE(Fixture, "nonstrict_self_mismatch_tail") { CheckResult result = check(R"( @@ -4560,32 +4107,6 @@ local c = a(2) -- too many arguments CHECK_EQ("Argument count mismatch. Function expects 1 argument, but 2 are specified", toString(result.errors[0])); } -TEST_CASE_FIXTURE(Fixture, "module_export_free_type_leak") -{ - CheckResult result = check(R"( -function get() - return function(obj) return true end -end - -export type f = typeof(get()) - )"); - - LUAU_REQUIRE_NO_ERRORS(result); -} - -TEST_CASE_FIXTURE(Fixture, "module_export_wrapped_free_type_leak") -{ - CheckResult result = check(R"( -function get() - return {a = 1, b = function(obj) return true end} -end - -export type f = typeof(get()) - )"); - - LUAU_REQUIRE_NO_ERRORS(result); -} - TEST_CASE_FIXTURE(Fixture, "custom_require_global") { CheckResult result = check(R"( @@ -4768,8 +4289,6 @@ TEST_CASE_FIXTURE(Fixture, "no_heap_use_after_free_error") TEST_CASE_FIXTURE(Fixture, "dont_invalidate_the_properties_iterator_of_free_table_when_rolled_back") { - ScopedFastFlag sff{"LuauLogTableTypeVarBoundTo", true}; - fileResolver.source["Module/Backend/Types"] = R"( export type Fiber = { return_: Fiber? @@ -4849,8 +4368,8 @@ TEST_CASE_FIXTURE(Fixture, "record_matching_overload") ModulePtr module = getMainModule(); auto it = module->astOverloadResolvedTypes.find(parentExpr); - REQUIRE(it != module->astOverloadResolvedTypes.end()); - CHECK_EQ(toString(it->second), "(number) -> number"); + REQUIRE(it); + CHECK_EQ(toString(*it), "(number) -> number"); } TEST_CASE_FIXTURE(Fixture, "infer_anonymous_function_arguments") @@ -5013,76 +4532,6 @@ g12({x=1}, {x=2}, function(x, y) return {x=x.x + y.x} end) LUAU_REQUIRE_NO_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "mutually_recursive_types_restriction_ok") -{ - CheckResult result = check(R"( - type Tree = { data: T, children: Forest } - type Forest = {Tree} - )"); - - LUAU_REQUIRE_NO_ERRORS(result); -} - -TEST_CASE_FIXTURE(Fixture, "mutually_recursive_types_restriction_not_ok_1") -{ - ScopedFastFlag sff{"LuauRecursiveTypeParameterRestriction", true}; - - CheckResult result = check(R"( - -- OK because forwarded types are used with their parameters. - type Tree = { data: T, children: Forest } - type Forest = {Tree<{T}>} - )"); - - LUAU_REQUIRE_ERRORS(result); -} - -TEST_CASE_FIXTURE(Fixture, "mutually_recursive_types_restriction_not_ok_2") -{ - ScopedFastFlag sff{"LuauRecursiveTypeParameterRestriction", true}; - - CheckResult result = check(R"( - -- Not OK because forwarded types are used with different types than their parameters. - type Forest = {Tree<{T}>} - type Tree = { data: T, children: Forest } - )"); - - LUAU_REQUIRE_ERRORS(result); -} - -TEST_CASE_FIXTURE(Fixture, "mutually_recursive_types_swapsies_ok") -{ - CheckResult result = check(R"( - type Tree1 = { data: T, children: {Tree2} } - type Tree2 = { data: U, children: {Tree1} } - - LUAU_REQUIRE_ERRORS(result); -} - -TEST_CASE_FIXTURE(Fixture, "mutually_recursive_types_swapsies_not_ok") -{ - ScopedFastFlag sff{"LuauRecursiveTypeParameterRestriction", true}; - - CheckResult result = check(R"( - type Tree1 = { data: T, children: {Tree2} } - type Tree2 = { data: U, children: {Tree1} } - )"); - - LUAU_REQUIRE_ERRORS(result); -} - -TEST_CASE_FIXTURE(Fixture, "free_variables_from_typeof_in_aliases") -{ - CheckResult result = check(R"( - function f(x) return x[1] end - -- x has type X? for a free type variable X - local x = f ({}) - type ContainsFree = { this: a, that: typeof(x) } - type ContainsContainsFree = { that: ContainsFree } - )"); - - LUAU_REQUIRE_NO_ERRORS(result); -} - TEST_CASE_FIXTURE(Fixture, "infer_generic_lib_function_function_argument") { CheckResult result = check(R"( diff --git a/tests/TypeInfer.tryUnify.test.cpp b/tests/TypeInfer.tryUnify.test.cpp index 91ac9f06..1f4b63ef 100644 --- a/tests/TypeInfer.tryUnify.test.cpp +++ b/tests/TypeInfer.tryUnify.test.cpp @@ -1,5 +1,6 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #include "Luau/Parser.h" +#include "Luau/Scope.h" #include "Luau/TypeInfer.h" #include "Luau/TypeVar.h" diff --git a/tests/TypeInfer.typePacks.cpp b/tests/TypeInfer.typePacks.cpp index 5f7f2847..3e1dedd4 100644 --- a/tests/TypeInfer.typePacks.cpp +++ b/tests/TypeInfer.typePacks.cpp @@ -294,4 +294,370 @@ end LUAU_REQUIRE_NO_ERRORS(result); } +TEST_CASE_FIXTURE(Fixture, "type_alias_type_packs") +{ + ScopedFastFlag luauParseGenericFunctions("LuauParseGenericFunctions", true); + ScopedFastFlag luauTypeAliasPacks("LuauTypeAliasPacks", true); + ScopedFastFlag luauParseTypePackTypeParameters("LuauParseTypePackTypeParameters", true); + + CheckResult result = check(R"( +type Packed = (T...) -> T... +local a: Packed<> +local b: Packed +local c: Packed + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + auto tf = lookupType("Packed"); + REQUIRE(tf); + CHECK_EQ(toString(*tf), "(T...) -> (T...)"); + CHECK_EQ(toString(requireType("a")), "() -> ()"); + CHECK_EQ(toString(requireType("b")), "(number) -> number"); + CHECK_EQ(toString(requireType("c")), "(string, number) -> (string, number)"); + + result = check(R"( +-- (U..., T) cannot be parsed right now +type Packed = { f: (a: T, U...) -> (T, U...) } +local a: Packed +local b: Packed +local c: Packed + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + tf = lookupType("Packed"); + REQUIRE(tf); + CHECK_EQ(toString(*tf), "Packed"); + CHECK_EQ(toString(*tf, {true}), "{| f: (T, U...) -> (T, U...) |}"); + + auto ttvA = get(requireType("a")); + REQUIRE(ttvA); + CHECK_EQ(toString(requireType("a")), "Packed"); + CHECK_EQ(toString(requireType("a"), {true}), "{| f: (number) -> (number) |}"); + REQUIRE(ttvA->instantiatedTypeParams.size() == 1); + REQUIRE(ttvA->instantiatedTypePackParams.size() == 1); + CHECK_EQ(toString(ttvA->instantiatedTypeParams[0], {true}), "number"); + CHECK_EQ(toString(ttvA->instantiatedTypePackParams[0], {true}), ""); + + auto ttvB = get(requireType("b")); + REQUIRE(ttvB); + CHECK_EQ(toString(requireType("b")), "Packed"); + CHECK_EQ(toString(requireType("b"), {true}), "{| f: (string, number) -> (string, number) |}"); + REQUIRE(ttvB->instantiatedTypeParams.size() == 1); + REQUIRE(ttvB->instantiatedTypePackParams.size() == 1); + CHECK_EQ(toString(ttvB->instantiatedTypeParams[0], {true}), "string"); + CHECK_EQ(toString(ttvB->instantiatedTypePackParams[0], {true}), "number"); + + auto ttvC = get(requireType("c")); + REQUIRE(ttvC); + CHECK_EQ(toString(requireType("c")), "Packed"); + CHECK_EQ(toString(requireType("c"), {true}), "{| f: (string, number, boolean) -> (string, number, boolean) |}"); + REQUIRE(ttvC->instantiatedTypeParams.size() == 1); + REQUIRE(ttvC->instantiatedTypePackParams.size() == 1); + CHECK_EQ(toString(ttvC->instantiatedTypeParams[0], {true}), "string"); + CHECK_EQ(toString(ttvC->instantiatedTypePackParams[0], {true}), "number, boolean"); +} + +TEST_CASE_FIXTURE(Fixture, "type_alias_type_packs_import") +{ + ScopedFastFlag luauParseGenericFunctions("LuauParseGenericFunctions", true); + ScopedFastFlag luauTypeAliasPacks("LuauTypeAliasPacks", true); + ScopedFastFlag luauParseTypePackTypeParameters("LuauParseTypePackTypeParameters", true); + + fileResolver.source["game/A"] = R"( +export type Packed = { a: T, b: (U...) -> () } +return {} + )"; + + CheckResult aResult = frontend.check("game/A"); + LUAU_REQUIRE_NO_ERRORS(aResult); + + CheckResult bResult = check(R"( +local Import = require(game.A) +local a: Import.Packed +local b: Import.Packed +local c: Import.Packed +local d: { a: typeof(c) } + )"); + LUAU_REQUIRE_NO_ERRORS(bResult); + + auto tf = lookupImportedType("Import", "Packed"); + REQUIRE(tf); + CHECK_EQ(toString(*tf), "Packed"); + CHECK_EQ(toString(*tf, {true}), "{| a: T, b: (U...) -> () |}"); + + CHECK_EQ(toString(requireType("a"), {true}), "{| a: number, b: () -> () |}"); + CHECK_EQ(toString(requireType("b"), {true}), "{| a: string, b: (number) -> () |}"); + CHECK_EQ(toString(requireType("c"), {true}), "{| a: string, b: (number, boolean) -> () |}"); + CHECK_EQ(toString(requireType("d")), "{| a: Packed |}"); +} + +TEST_CASE_FIXTURE(Fixture, "type_pack_type_parameters") +{ + ScopedFastFlag luauParseGenericFunctions("LuauParseGenericFunctions", true); + ScopedFastFlag luauTypeAliasPacks("LuauTypeAliasPacks", true); + ScopedFastFlag luauParseTypePackTypeParameters("LuauParseTypePackTypeParameters", true); + + fileResolver.source["game/A"] = R"( +export type Packed = { a: T, b: (U...) -> () } +return {} + )"; + + CheckResult cResult = check(R"( +local Import = require(game.A) +type Alias = Import.Packed +local a: Alias + +type B = Import.Packed +type C = Import.Packed + )"); + LUAU_REQUIRE_NO_ERRORS(cResult); + + auto tf = lookupType("Alias"); + REQUIRE(tf); + CHECK_EQ(toString(*tf), "Alias"); + CHECK_EQ(toString(*tf, {true}), "{| a: S, b: (T, R...) -> () |}"); + + CHECK_EQ(toString(requireType("a"), {true}), "{| a: string, b: (number, boolean) -> () |}"); + + tf = lookupType("B"); + REQUIRE(tf); + CHECK_EQ(toString(*tf), "B"); + CHECK_EQ(toString(*tf, {true}), "{| a: string, b: (X...) -> () |}"); + + tf = lookupType("C"); + REQUIRE(tf); + CHECK_EQ(toString(*tf), "C"); + CHECK_EQ(toString(*tf, {true}), "{| a: string, b: (number, X...) -> () |}"); +} + +TEST_CASE_FIXTURE(Fixture, "type_alias_type_packs_nested") +{ + ScopedFastFlag luauParseGenericFunctions("LuauParseGenericFunctions", true); + ScopedFastFlag luauTypeAliasPacks("LuauTypeAliasPacks", true); + ScopedFastFlag luauParseTypePackTypeParameters("LuauParseTypePackTypeParameters", true); + + CheckResult result = check(R"( +type Packed1 = (T...) -> (T...) +type Packed2 = (Packed1, T...) -> (Packed1, T...) +type Packed3 = (Packed2, T...) -> (Packed2, T...) +type Packed4 = (Packed3, T...) -> (Packed3, T...) + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + auto tf = lookupType("Packed4"); + REQUIRE(tf); + CHECK_EQ(toString(*tf), + "((((T...) -> (T...), T...) -> ((T...) -> (T...), T...), T...) -> (((T...) -> (T...), T...) -> ((T...) -> (T...), T...), T...), T...) -> " + "((((T...) -> (T...), T...) -> ((T...) -> (T...), T...), T...) -> (((T...) -> (T...), T...) -> ((T...) -> (T...), T...), T...), T...)"); +} + +TEST_CASE_FIXTURE(Fixture, "type_alias_type_pack_variadic") +{ + ScopedFastFlag luauParseGenericFunctions("LuauParseGenericFunctions", true); + ScopedFastFlag luauTypeAliasPacks("LuauTypeAliasPacks", true); + ScopedFastFlag luauParseTypePackTypeParameters("LuauParseTypePackTypeParameters", true); + + CheckResult result = check(R"( +type X = (T...) -> (string, T...) + +type D = X<...number> +type E = X<(number, ...string)> + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ(toString(*lookupType("D")), "(...number) -> (string, ...number)"); + CHECK_EQ(toString(*lookupType("E")), "(number, ...string) -> (string, number, ...string)"); +} + +TEST_CASE_FIXTURE(Fixture, "type_alias_type_pack_multi") +{ + ScopedFastFlag luauParseGenericFunctions("LuauParseGenericFunctions", true); + ScopedFastFlag luauTypeAliasPacks("LuauTypeAliasPacks", true); + ScopedFastFlag luauParseTypePackTypeParameters("LuauParseTypePackTypeParameters", true); + + CheckResult result = check(R"( +type Y = (T...) -> (U...) +type A = Y +type B = Y<(number, ...string), S...> + +type Z = (T) -> (U...) +type E = Z +type F = Z + +type W = (T, U...) -> (T, V...) +type H = W +type I = W + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ(toString(*lookupType("A")), "(S...) -> (S...)"); + CHECK_EQ(toString(*lookupType("B")), "(number, ...string) -> (S...)"); + + CHECK_EQ(toString(*lookupType("E")), "(number) -> (S...)"); + CHECK_EQ(toString(*lookupType("F")), "(number) -> (string, S...)"); + + CHECK_EQ(toString(*lookupType("H")), "(number, S...) -> (number, R...)"); + CHECK_EQ(toString(*lookupType("I")), "(number, string, S...) -> (number, R...)"); +} + +TEST_CASE_FIXTURE(Fixture, "type_alias_type_pack_explicit") +{ + ScopedFastFlag luauParseGenericFunctions("LuauParseGenericFunctions", true); + ScopedFastFlag luauTypeAliasPacks("LuauTypeAliasPacks", true); + ScopedFastFlag luauParseTypePackTypeParameters("LuauParseTypePackTypeParameters", true); + + CheckResult result = check(R"( +type X = (T...) -> (T...) + +type A = X<(S...)> +type B = X<()> +type C = X<(number)> +type D = X<(number, string)> +type E = X<(...number)> +type F = X<(string, ...number)> + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ(toString(*lookupType("A")), "(S...) -> (S...)"); + CHECK_EQ(toString(*lookupType("B")), "() -> ()"); + CHECK_EQ(toString(*lookupType("C")), "(number) -> number"); + CHECK_EQ(toString(*lookupType("D")), "(number, string) -> (number, string)"); + CHECK_EQ(toString(*lookupType("E")), "(...number) -> (...number)"); + CHECK_EQ(toString(*lookupType("F")), "(string, ...number) -> (string, ...number)"); +} + +TEST_CASE_FIXTURE(Fixture, "type_alias_type_pack_explicit_multi") +{ + ScopedFastFlag luauParseGenericFunctions("LuauParseGenericFunctions", true); + ScopedFastFlag luauTypeAliasPacks("LuauTypeAliasPacks", true); + ScopedFastFlag luauParseTypePackTypeParameters("LuauParseTypePackTypeParameters", true); + + CheckResult result = check(R"( +type Y = (T...) -> (U...) + +type A = Y<(number, string), (boolean)> +type B = Y<(), ()> +type C = Y<...string, (number, S...)> +type D = Y + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ(toString(*lookupType("A")), "(number, string) -> boolean"); + CHECK_EQ(toString(*lookupType("B")), "() -> ()"); + CHECK_EQ(toString(*lookupType("C")), "(...string) -> (number, S...)"); + CHECK_EQ(toString(*lookupType("D")), "(X...) -> (number, string, X...)"); +} + +TEST_CASE_FIXTURE(Fixture, "type_alias_type_pack_explicit_multi_tostring") +{ + ScopedFastFlag luauParseGenericFunctions("LuauParseGenericFunctions", true); + ScopedFastFlag luauTypeAliasPacks("LuauTypeAliasPacks", true); + ScopedFastFlag luauParseTypePackTypeParameters("LuauParseTypePackTypeParameters", true); + ScopedFastFlag luauInstantiatedTypeParamRecursion("LuauInstantiatedTypeParamRecursion", true); // For correct toString block + + CheckResult result = check(R"( +type Y = { f: (T...) -> (U...) } + +local a: Y<(number, string), (boolean)> +local b: Y<(), ()> + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ(toString(requireType("a")), "Y<(number, string), (boolean)>"); + CHECK_EQ(toString(requireType("b")), "Y<(), ()>"); +} + +TEST_CASE_FIXTURE(Fixture, "type_alias_backwards_compatible") +{ + ScopedFastFlag luauParseGenericFunctions("LuauParseGenericFunctions", true); + ScopedFastFlag luauTypeAliasPacks("LuauTypeAliasPacks", true); + ScopedFastFlag luauParseTypePackTypeParameters("LuauParseTypePackTypeParameters", true); + + CheckResult result = check(R"( +type X = () -> T +type Y = (T) -> U + +type A = X<(number)> +type B = Y<(number), (boolean)> +type C = Y<(number), boolean> + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ(toString(*lookupType("A")), "() -> number"); + CHECK_EQ(toString(*lookupType("B")), "(number) -> boolean"); + CHECK_EQ(toString(*lookupType("C")), "(number) -> boolean"); +} + +TEST_CASE_FIXTURE(Fixture, "type_alias_type_packs_errors") +{ + ScopedFastFlag luauParseGenericFunctions("LuauParseGenericFunctions", true); + ScopedFastFlag luauTypeAliasPacks("LuauTypeAliasPacks", true); + ScopedFastFlag luauParseTypePackTypeParameters("LuauParseTypePackTypeParameters", true); + + CheckResult result = check(R"( +type Packed = (T, U) -> (V...) +local b: Packed + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ(toString(result.errors[0]), "Generic type 'Packed' expects at least 2 type arguments, but only 1 is specified"); + + result = check(R"( +type Packed = (T, U) -> () +type B = Packed + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ(toString(result.errors[0]), "Generic type 'Packed' expects 0 type pack arguments, but 1 is specified"); + + result = check(R"( +type Packed = (T...) -> (U...) +type Other = Packed + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ(toString(result.errors[0]), "Type parameters must come before type pack parameters"); + + result = check(R"( +type Packed = (T) -> U +type Other = Packed + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ(toString(result.errors[0]), "Generic type 'Packed' expects 2 type arguments, but only 1 is specified"); + + result = check(R"( +type Packed = (T...) -> T... +local a: Packed + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ(toString(result.errors[0]), "Type parameter list is required"); + + result = check(R"( +type Packed = (T...) -> (U...) +type Other = Packed<> + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ(toString(result.errors[0]), "Generic type 'Packed' expects 2 type pack arguments, but none are specified"); + + result = check(R"( +type Packed = (T...) -> (U...) +type Other = Packed + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ(toString(result.errors[0]), "Generic type 'Packed' expects 2 type pack arguments, but only 1 is specified"); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.unionTypes.test.cpp b/tests/TypeInfer.unionTypes.test.cpp index ae4d836b..037144e2 100644 --- a/tests/TypeInfer.unionTypes.test.cpp +++ b/tests/TypeInfer.unionTypes.test.cpp @@ -237,21 +237,7 @@ TEST_CASE_FIXTURE(Fixture, "union_equality_comparisons") local z = a == c )"); - if (FFlag::LuauEqConstraint) - { - LUAU_REQUIRE_NO_ERRORS(result); - } - else - { - LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK_EQ(*typeChecker.booleanType, *requireType("x")); - CHECK_EQ(*typeChecker.booleanType, *requireType("y")); - - TypeMismatch* tm = get(result.errors[0]); - REQUIRE(tm); - CHECK_EQ("(number | string)?", toString(*tm->wantedType)); - CHECK_EQ("boolean | number", toString(*tm->givenType)); - } + LUAU_REQUIRE_NO_ERRORS(result); } TEST_CASE_FIXTURE(Fixture, "optional_union_members") diff --git a/tests/TypeVar.test.cpp b/tests/TypeVar.test.cpp index 98ce9f93..a679e3fd 100644 --- a/tests/TypeVar.test.cpp +++ b/tests/TypeVar.test.cpp @@ -1,5 +1,6 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #include "Luau/Parser.h" +#include "Luau/Scope.h" #include "Luau/TypeInfer.h" #include "Luau/TypeVar.h" diff --git a/tools/tracegraph.py b/tools/tracegraph.py new file mode 100644 index 00000000..a46423e7 --- /dev/null +++ b/tools/tracegraph.py @@ -0,0 +1,95 @@ +#!/usr/bin/python +# This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details + +# Given a trace event file, this tool generates a flame graph based on the event scopes present in the file +# The result of analysis is a .svg file which can be viewed in a browser + +import sys +import svg +import json + +class Node(svg.Node): + def __init__(self): + svg.Node.__init__(self) + self.caption = "" + self.description = "" + self.ticks = 0 + + def text(self): + return self.caption + + def title(self): + return self.caption + + def details(self, root): + return "{} ({:,} usec, {:.1%}); self: {:,} usec".format(self.description, self.width, self.width / root.width, self.ticks) + +with open(sys.argv[1]) as f: + dump = f.read() + +root = Node() + +# Finish the file +if not dump.endswith("]"): + dump += "{}]" + +data = json.loads(dump) + +stacks = {} + +for l in data: + if len(l) == 0: + continue + + # Track stack of each thread, but aggregate values together + tid = l["tid"] + + if not tid in stacks: + stacks[tid] = [] + stack = stacks[tid] + + if l["ph"] == 'B': + stack.append(l) + elif l["ph"] == 'E': + node = root + + for e in stack: + caption = e["name"] + description = '' + + if "args" in e: + for arg in e["args"]: + if len(description) != 0: + description += ", " + + description += "{}: {}".format(arg, e["args"][arg]) + + child = node.child(caption + description) + child.caption = caption + child.description = description + + node = child + + begin = stack[-1] + + ticks = l["ts"] - begin["ts"] + rawticks = ticks + + # Flame graph requires ticks without children duration + if "childts" in begin: + ticks -= begin["childts"] + + node.ticks += int(ticks) + + stack.pop() + + if len(stack): + parent = stack[-1] + + if "childts" in parent: + parent["childts"] += rawticks + else: + parent["childts"] = rawticks + +svg.layout(root, lambda n: n.ticks) +svg.display(root, "Flame Graph", "hot", flip = True) From 49b0c59eecbca8feb0996526636032742f08153d Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Thu, 4 Nov 2021 19:34:35 -0700 Subject: [PATCH 021/399] Sync to upstream/release/502 (#134) Changes: - Support for time tracing for analysis/compiler (not currently exposed through CLI) - Support for type pack arguments in type aliases (#83) - Basic support for require(path) in luau-analyze - Add a lint warning for table.move with 0 index as part of TableOperation lint - Remove last STL dependency from Luau.VM - Minor VS2022 performance tuning Co-authored-by: Rodactor --- Analysis/include/Luau/BuiltinDefinitions.h | 3 +- Analysis/include/Luau/Error.h | 1 + Analysis/include/Luau/FileResolver.h | 58 +- Analysis/include/Luau/Module.h | 10 +- Analysis/include/Luau/ModuleResolver.h | 6 - Analysis/include/Luau/RequireTracer.h | 5 +- Analysis/include/Luau/Scope.h | 67 ++ Analysis/include/Luau/Substitution.h | 14 +- Analysis/include/Luau/TypeInfer.h | 56 +- Analysis/include/Luau/TypePack.h | 3 +- Analysis/include/Luau/TypeVar.h | 22 +- Analysis/include/Luau/Unifier.h | 18 +- Analysis/src/AstQuery.cpp | 17 +- Analysis/src/Autocomplete.cpp | 49 +- Analysis/src/BuiltinDefinitions.cpp | 101 +-- Analysis/src/EmbeddedBuiltinDefinitions.cpp | 21 - Analysis/src/Error.cpp | 108 ++- Analysis/src/Frontend.cpp | 51 +- Analysis/src/IostreamHelpers.cpp | 18 +- Analysis/src/JsonEncoder.cpp | 32 +- Analysis/src/Linter.cpp | 19 +- Analysis/src/Module.cpp | 44 +- Analysis/src/RequireTracer.cpp | 215 ++++- Analysis/src/Scope.cpp | 123 +++ Analysis/src/Substitution.cpp | 34 +- Analysis/src/ToString.cpp | 131 ++- Analysis/src/Transpiler.cpp | 38 +- Analysis/src/TypeAttach.cpp | 124 ++- Analysis/src/TypeInfer.cpp | 577 ++++++------ Analysis/src/TypePack.cpp | 13 + Analysis/src/TypeUtils.cpp | 18 +- Analysis/src/TypeVar.cpp | 36 +- Analysis/src/Unifier.cpp | 500 +++++++++-- Ast/include/Luau/Ast.h | 33 +- Ast/include/Luau/DenseHash.h | 5 +- Ast/include/Luau/Parser.h | 8 +- Ast/include/Luau/TimeTrace.h | 223 +++++ Ast/src/Ast.cpp | 37 +- Ast/src/Parser.cpp | 147 ++- Ast/src/TimeTrace.cpp | 248 ++++++ CLI/Analyze.cpp | 18 +- Compiler/src/Compiler.cpp | 10 + Sources.cmake | 5 + VM/src/ldo.cpp | 12 +- VM/src/lgc.cpp | 191 +++- VM/src/ltablib.cpp | 22 - VM/src/lvmexecute.cpp | 12 +- VM/src/lvmload.cpp | 34 +- bench/tests/deltablue.lua | 934 -------------------- tests/Autocomplete.test.cpp | 853 +++++++++--------- tests/Fixture.cpp | 49 + tests/Fixture.h | 2 + tests/Frontend.test.cpp | 31 +- tests/Linter.test.cpp | 9 +- tests/Module.test.cpp | 1 + tests/NonstrictMode.test.cpp | 1 + tests/Parser.test.cpp | 15 + tests/RequireTracer.test.cpp | 68 +- tests/ToString.test.cpp | 3 +- tests/TypeInfer.aliases.test.cpp | 557 ++++++++++++ tests/TypeInfer.provisional.test.cpp | 36 +- tests/TypeInfer.refinements.test.cpp | 72 +- tests/TypeInfer.tables.test.cpp | 230 +++-- tests/TypeInfer.test.cpp | 563 +----------- tests/TypeInfer.tryUnify.test.cpp | 1 + tests/TypeInfer.typePacks.cpp | 366 ++++++++ tests/TypeInfer.unionTypes.test.cpp | 16 +- tests/TypeVar.test.cpp | 1 + tools/tracegraph.py | 95 ++ 69 files changed, 4478 insertions(+), 2962 deletions(-) create mode 100644 Analysis/include/Luau/Scope.h create mode 100644 Analysis/src/Scope.cpp create mode 100644 Ast/include/Luau/TimeTrace.h create mode 100644 Ast/src/TimeTrace.cpp delete mode 100644 bench/tests/deltablue.lua create mode 100644 tests/TypeInfer.aliases.test.cpp create mode 100644 tools/tracegraph.py diff --git a/Analysis/include/Luau/BuiltinDefinitions.h b/Analysis/include/Luau/BuiltinDefinitions.h index 8f17fff6..57a1907a 100644 --- a/Analysis/include/Luau/BuiltinDefinitions.h +++ b/Analysis/include/Luau/BuiltinDefinitions.h @@ -1,7 +1,8 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #pragma once -#include "TypeInfer.h" +#include "Luau/Scope.h" +#include "Luau/TypeInfer.h" namespace Luau { diff --git a/Analysis/include/Luau/Error.h b/Analysis/include/Luau/Error.h index 946bc928..ac6f13e9 100644 --- a/Analysis/include/Luau/Error.h +++ b/Analysis/include/Luau/Error.h @@ -120,6 +120,7 @@ struct IncorrectGenericParameterCount Name name; TypeFun typeFun; size_t actualParameters; + size_t actualPackParameters; bool operator==(const IncorrectGenericParameterCount& rhs) const; }; diff --git a/Analysis/include/Luau/FileResolver.h b/Analysis/include/Luau/FileResolver.h index 71f9464b..a05ec5e9 100644 --- a/Analysis/include/Luau/FileResolver.h +++ b/Analysis/include/Luau/FileResolver.h @@ -25,51 +25,39 @@ struct SourceCode Type type; }; +struct ModuleInfo +{ + ModuleName name; + bool optional = false; +}; + struct FileResolver { virtual ~FileResolver() {} - /** Fetch the source code associated with the provided ModuleName. - * - * FIXME: This requires a string copy! - * - * @returns The actual Lua code on success. - * @returns std::nullopt if no such file exists. When this occurs, type inference will report an UnknownRequire error. - */ virtual std::optional readSource(const ModuleName& name) = 0; - /** Does the module exist? - * - * Saves a string copy over reading the source and throwing it away. - */ - virtual bool moduleExists(const ModuleName& name) const = 0; + virtual std::optional resolveModule(const ModuleInfo* context, AstExpr* expr) + { + return std::nullopt; + } - virtual std::optional fromAstFragment(AstExpr* expr) const = 0; - - /** Given a valid module name and a string of arbitrary data, figure out the concatenation. - */ - virtual ModuleName concat(const ModuleName& lhs, std::string_view rhs) const = 0; - - /** Goes "up" a level in the hierarchy that the ModuleName represents. - * - * For instances, this is analogous to someInstance.Parent; for paths, this is equivalent to removing the last - * element of the path. Other ModuleName representations may have other ways of doing this. - * - * @returns The parent ModuleName, if one exists. - * @returns std::nullopt if there is no parent for this module name. - */ - virtual std::optional getParentModuleName(const ModuleName& name) const = 0; - - virtual std::optional getHumanReadableModuleName_(const ModuleName& name) const + virtual std::string getHumanReadableModuleName(const ModuleName& name) const { return name; } - virtual std::optional getEnvironmentForModule(const ModuleName& name) const = 0; + virtual std::optional getEnvironmentForModule(const ModuleName& name) const + { + return std::nullopt; + } - /** LanguageService only: - * std::optional fromInstance(Instance* inst) - */ + // DEPRECATED APIS + // These are going to be removed with LuauNewRequireTracer + virtual bool moduleExists(const ModuleName& name) const = 0; + virtual std::optional fromAstFragment(AstExpr* expr) const = 0; + virtual ModuleName concat(const ModuleName& lhs, std::string_view rhs) const = 0; + virtual std::optional getParentModuleName(const ModuleName& name) const = 0; }; struct NullFileResolver : FileResolver @@ -94,10 +82,6 @@ struct NullFileResolver : FileResolver { return std::nullopt; } - std::optional getEnvironmentForModule(const ModuleName& name) const override - { - return std::nullopt; - } }; } // namespace Luau diff --git a/Analysis/include/Luau/Module.h b/Analysis/include/Luau/Module.h index 413b68f4..d0844835 100644 --- a/Analysis/include/Luau/Module.h +++ b/Analysis/include/Luau/Module.h @@ -90,10 +90,12 @@ struct Module TypeArena internalTypes; std::vector> scopes; // never empty - std::unordered_map astTypes; - std::unordered_map astExpectedTypes; - std::unordered_map astOriginalCallTypes; - std::unordered_map astOverloadResolvedTypes; + + DenseHashMap astTypes{nullptr}; + DenseHashMap astExpectedTypes{nullptr}; + DenseHashMap astOriginalCallTypes{nullptr}; + DenseHashMap astOverloadResolvedTypes{nullptr}; + std::unordered_map declaredGlobals; ErrorVec errors; Mode mode; diff --git a/Analysis/include/Luau/ModuleResolver.h b/Analysis/include/Luau/ModuleResolver.h index a394a21b..d892ccd7 100644 --- a/Analysis/include/Luau/ModuleResolver.h +++ b/Analysis/include/Luau/ModuleResolver.h @@ -15,12 +15,6 @@ struct Module; using ModulePtr = std::shared_ptr; -struct ModuleInfo -{ - ModuleName name; - bool optional = false; -}; - struct ModuleResolver { virtual ~ModuleResolver() {} diff --git a/Analysis/include/Luau/RequireTracer.h b/Analysis/include/Luau/RequireTracer.h index e9778876..c25545f5 100644 --- a/Analysis/include/Luau/RequireTracer.h +++ b/Analysis/include/Luau/RequireTracer.h @@ -17,12 +17,11 @@ struct AstLocal; struct RequireTraceResult { - DenseHashMap exprs{0}; - DenseHashMap optional{0}; + DenseHashMap exprs{nullptr}; std::vector> requires; }; -RequireTraceResult traceRequires(FileResolver* fileResolver, AstStatBlock* root, ModuleName currentModuleName); +RequireTraceResult traceRequires(FileResolver* fileResolver, AstStatBlock* root, const ModuleName& currentModuleName); } // namespace Luau diff --git a/Analysis/include/Luau/Scope.h b/Analysis/include/Luau/Scope.h new file mode 100644 index 00000000..45338409 --- /dev/null +++ b/Analysis/include/Luau/Scope.h @@ -0,0 +1,67 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +#pragma once + +#include "Luau/Location.h" +#include "Luau/TypeVar.h" + +#include +#include +#include + +namespace Luau +{ + +struct Scope; + +using ScopePtr = std::shared_ptr; + +struct Binding +{ + TypeId typeId; + Location location; + bool deprecated = false; + std::string deprecatedSuggestion; + std::optional documentationSymbol; +}; + +struct Scope +{ + explicit Scope(TypePackId returnType); // root scope + explicit Scope(const ScopePtr& parent, int subLevel = 0); // child scope. Parent must not be nullptr. + + const ScopePtr parent; // null for the root + std::unordered_map bindings; + TypePackId returnType; + bool breakOk = false; + std::optional varargPack; + + TypeLevel level; + + std::unordered_map exportedTypeBindings; + std::unordered_map privateTypeBindings; + std::unordered_map typeAliasLocations; + + std::unordered_map> importedTypeBindings; + + std::optional lookup(const Symbol& name); + + std::optional lookupType(const Name& name); + std::optional lookupImportedType(const Name& moduleAlias, const Name& name); + + std::unordered_map privateTypePackBindings; + std::optional lookupPack(const Name& name); + + // WARNING: This function linearly scans for a string key of equal value! It is thus O(n**2) + std::optional linearSearchForBinding(const std::string& name, bool traverseScopeChain = true); + + RefinementMap refinements; + + // For mutually recursive type aliases, it's important that + // they use the same types for the same names. + // For instance, in `type Tree { data: T, children: Forest } type Forest = {Tree}` + // we need that the generic type `T` in both cases is the same, so we use a cache. + std::unordered_map typeAliasTypeParameters; + std::unordered_map typeAliasTypePackParameters; +}; + +} // namespace Luau diff --git a/Analysis/include/Luau/Substitution.h b/Analysis/include/Luau/Substitution.h index 6ac868f7..80a14e8f 100644 --- a/Analysis/include/Luau/Substitution.h +++ b/Analysis/include/Luau/Substitution.h @@ -52,8 +52,6 @@ // `T`, and the type of `f` are in the same SCC, which is why `f` gets // replaced. -LUAU_FASTFLAG(DebugLuauTrackOwningArena) - namespace Luau { @@ -188,20 +186,12 @@ struct Substitution : FindDirty template TypeId addType(const T& tv) { - TypeId allocated = currentModule->internalTypes.typeVars.allocate(tv); - if (FFlag::DebugLuauTrackOwningArena) - asMutable(allocated)->owningArena = ¤tModule->internalTypes; - - return allocated; + return currentModule->internalTypes.addType(tv); } template TypePackId addTypePack(const T& tp) { - TypePackId allocated = currentModule->internalTypes.typePacks.allocate(tp); - if (FFlag::DebugLuauTrackOwningArena) - asMutable(allocated)->owningArena = ¤tModule->internalTypes; - - return allocated; + return currentModule->internalTypes.addTypePack(TypePackVar{tp}); } }; diff --git a/Analysis/include/Luau/TypeInfer.h b/Analysis/include/Luau/TypeInfer.h index ec2a1a26..d701eb24 100644 --- a/Analysis/include/Luau/TypeInfer.h +++ b/Analysis/include/Luau/TypeInfer.h @@ -86,7 +86,10 @@ struct ApplyTypeFunction : Substitution { TypeLevel level; bool encounteredForwardedType; - std::unordered_map arguments; + std::unordered_map typeArguments; + std::unordered_map typePackArguments; + bool ignoreChildren(TypeId ty) override; + bool ignoreChildren(TypePackId tp) override; bool isDirty(TypeId ty) override; bool isDirty(TypePackId tp) override; TypeId clean(TypeId ty) override; @@ -328,7 +331,8 @@ private: TypeId resolveType(const ScopePtr& scope, const AstType& annotation, bool canBeGeneric = false); TypePackId resolveTypePack(const ScopePtr& scope, const AstTypeList& types); TypePackId resolveTypePack(const ScopePtr& scope, const AstTypePack& annotation); - TypeId instantiateTypeFun(const ScopePtr& scope, const TypeFun& tf, const std::vector& typeParams, const Location& location); + TypeId instantiateTypeFun(const ScopePtr& scope, const TypeFun& tf, const std::vector& typeParams, + const std::vector& typePackParams, const Location& location); // Note: `scope` must be a fresh scope. std::pair, std::vector> createGenericTypes( @@ -398,54 +402,6 @@ private: int recursionCount = 0; }; -struct Binding -{ - TypeId typeId; - Location location; - bool deprecated = false; - std::string deprecatedSuggestion; - std::optional documentationSymbol; -}; - -struct Scope -{ - explicit Scope(TypePackId returnType); // root scope - explicit Scope(const ScopePtr& parent, int subLevel = 0); // child scope. Parent must not be nullptr. - - const ScopePtr parent; // null for the root - std::unordered_map bindings; - TypePackId returnType; - bool breakOk = false; - std::optional varargPack; - - TypeLevel level; - - std::unordered_map exportedTypeBindings; - std::unordered_map privateTypeBindings; - std::unordered_map typeAliasLocations; - - std::unordered_map> importedTypeBindings; - - std::optional lookup(const Symbol& name); - - std::optional lookupType(const Name& name); - std::optional lookupImportedType(const Name& moduleAlias, const Name& name); - - std::unordered_map privateTypePackBindings; - std::optional lookupPack(const Name& name); - - // WARNING: This function linearly scans for a string key of equal value! It is thus O(n**2) - std::optional linearSearchForBinding(const std::string& name, bool traverseScopeChain = true); - - RefinementMap refinements; - - // For mutually recursive type aliases, it's important that - // they use the same types for the same names. - // For instance, in `type Tree { data: T, children: Forest } type Forest = {Tree}` - // we need that the generic type `T` in both cases is the same, so we use a cache. - std::unordered_map typeAliasParameters; -}; - // Unit test hook void setPrintLine(void (*pl)(const std::string& s)); void resetPrintLine(); diff --git a/Analysis/include/Luau/TypePack.h b/Analysis/include/Luau/TypePack.h index 0d0adce7..d987d46c 100644 --- a/Analysis/include/Luau/TypePack.h +++ b/Analysis/include/Luau/TypePack.h @@ -117,7 +117,8 @@ bool areEqual(SeenSet& seen, const TypePackVar& lhs, const TypePackVar& rhs); TypePackId follow(TypePackId tp); -size_t size(const TypePackId tp); +size_t size(TypePackId tp); +bool finite(TypePackId tp); size_t size(const TypePack& tp); std::optional first(TypePackId tp); diff --git a/Analysis/include/Luau/TypeVar.h b/Analysis/include/Luau/TypeVar.h index 2e028df3..e04aa2ca 100644 --- a/Analysis/include/Luau/TypeVar.h +++ b/Analysis/include/Luau/TypeVar.h @@ -228,6 +228,7 @@ struct TableTypeVar std::map methodDefinitionLocations; std::vector instantiatedTypeParams; + std::vector instantiatedTypePackParams; ModuleName definitionModuleName; std::optional boundTo; @@ -284,8 +285,9 @@ struct ClassTypeVar struct TypeFun { - /// These should all be generic + // These should all be generic std::vector typeParams; + std::vector typePackParams; /** The underlying type. * @@ -293,6 +295,20 @@ struct TypeFun * You must first use TypeChecker::instantiateTypeFun to turn it into a real type. */ TypeId type; + + TypeFun() = default; + TypeFun(std::vector typeParams, TypeId type) + : typeParams(std::move(typeParams)) + , type(type) + { + } + + TypeFun(std::vector typeParams, std::vector typePackParams, TypeId type) + : typeParams(std::move(typeParams)) + , typePackParams(std::move(typePackParams)) + , type(type) + { + } }; // Anything! All static checking is off. @@ -524,8 +540,4 @@ UnionTypeVarIterator end(const UnionTypeVar* utv); using TypeIdPredicate = std::function(TypeId)>; std::vector filterMap(TypeId type, TypeIdPredicate predicate); -// TEMP: Clip this prototype with FFlag::LuauStringMetatable -std::optional> magicFunctionFormat( - struct TypeChecker& typechecker, const std::shared_ptr& scope, const AstExprCall& expr, ExprResult exprResult); - } // namespace Luau diff --git a/Analysis/include/Luau/Unifier.h b/Analysis/include/Luau/Unifier.h index 0ddc3cc0..522914b2 100644 --- a/Analysis/include/Luau/Unifier.h +++ b/Analysis/include/Luau/Unifier.h @@ -36,12 +36,17 @@ struct Unifier Variance variance = Covariant; CountMismatch::Context ctx = CountMismatch::Arg; - std::shared_ptr counters; + UnifierCounters* counters; + UnifierCounters countersData; + + std::shared_ptr counters_DEPRECATED; + InternalErrorReporter* iceHandler; Unifier(TypeArena* types, Mode mode, ScopePtr globalScope, const Location& location, Variance variance, InternalErrorReporter* iceHandler); Unifier(TypeArena* types, Mode mode, ScopePtr globalScope, const std::vector>& seen, const Location& location, - Variance variance, InternalErrorReporter* iceHandler, const std::shared_ptr& counters = nullptr); + Variance variance, InternalErrorReporter* iceHandler, const std::shared_ptr& counters_DEPRECATED = nullptr, + UnifierCounters* counters = nullptr); // Test whether the two type vars unify. Never commits the result. ErrorVec canUnify(TypeId superTy, TypeId subTy); @@ -58,11 +63,13 @@ private: void tryUnifyPrimitives(TypeId superTy, TypeId subTy); void tryUnifyFunctions(TypeId superTy, TypeId subTy, bool isFunctionCall = false); void tryUnifyTables(TypeId left, TypeId right, bool isIntersection = false); + void DEPRECATED_tryUnifyTables(TypeId left, TypeId right, bool isIntersection = false); void tryUnifyFreeTable(TypeId free, TypeId other); void tryUnifySealedTables(TypeId left, TypeId right, bool isIntersection); void tryUnifyWithMetatable(TypeId metatable, TypeId other, bool reversed); void tryUnifyWithClass(TypeId superTy, TypeId subTy, bool reversed); void tryUnify(const TableIndexer& superIndexer, const TableIndexer& subIndexer); + TypeId deeplyOptional(TypeId ty, std::unordered_map seen = {}); public: void tryUnify(TypePackId superTy, TypePackId subTy, bool isFunctionCall = false); @@ -80,9 +87,9 @@ private: public: // Report an "infinite type error" if the type "needle" already occurs within "haystack" void occursCheck(TypeId needle, TypeId haystack); - void occursCheck(std::unordered_set& seen, TypeId needle, TypeId haystack); + void occursCheck(std::unordered_set& seen_DEPRECATED, DenseHashSet& seen, TypeId needle, TypeId haystack); void occursCheck(TypePackId needle, TypePackId haystack); - void occursCheck(std::unordered_set& seen, TypePackId needle, TypePackId haystack); + void occursCheck(std::unordered_set& seen_DEPRECATED, DenseHashSet& seen, TypePackId needle, TypePackId haystack); Unifier makeChildUnifier(); @@ -93,6 +100,9 @@ private: [[noreturn]] void ice(const std::string& message, const Location& location); [[noreturn]] void ice(const std::string& message); + + DenseHashSet tempSeenTy{nullptr}; + DenseHashSet tempSeenTp{nullptr}; }; } // namespace Luau diff --git a/Analysis/src/AstQuery.cpp b/Analysis/src/AstQuery.cpp index d3de1754..0aed34c0 100644 --- a/Analysis/src/AstQuery.cpp +++ b/Analysis/src/AstQuery.cpp @@ -2,6 +2,7 @@ #include "Luau/AstQuery.h" #include "Luau/Module.h" +#include "Luau/Scope.h" #include "Luau/TypeInfer.h" #include "Luau/TypeVar.h" #include "Luau/ToString.h" @@ -143,8 +144,8 @@ std::optional findTypeAtPosition(const Module& module, const SourceModul { if (auto expr = findExprAtPosition(sourceModule, pos)) { - if (auto it = module.astTypes.find(expr); it != module.astTypes.end()) - return it->second; + if (auto it = module.astTypes.find(expr)) + return *it; } return std::nullopt; @@ -154,8 +155,8 @@ std::optional findExpectedTypeAtPosition(const Module& module, const Sou { if (auto expr = findExprAtPosition(sourceModule, pos)) { - if (auto it = module.astExpectedTypes.find(expr); it != module.astExpectedTypes.end()) - return it->second; + if (auto it = module.astExpectedTypes.find(expr)) + return *it; } return std::nullopt; @@ -322,9 +323,9 @@ std::optional getDocumentationSymbolAtPosition(const Source TypeId matchingOverload = nullptr; if (parentExpr && parentExpr->is()) { - if (auto it = module.astOverloadResolvedTypes.find(parentExpr); it != module.astOverloadResolvedTypes.end()) + if (auto it = module.astOverloadResolvedTypes.find(parentExpr)) { - matchingOverload = it->second; + matchingOverload = *it; } } @@ -345,9 +346,9 @@ std::optional getDocumentationSymbolAtPosition(const Source { if (AstExprIndexName* indexName = targetExpr->as()) { - if (auto it = module.astTypes.find(indexName->expr); it != module.astTypes.end()) + if (auto it = module.astTypes.find(indexName->expr)) { - TypeId parentTy = follow(it->second); + TypeId parentTy = follow(*it); if (const TableTypeVar* ttv = get(parentTy)) { if (auto propIt = ttv->props.find(indexName->index.value); propIt != ttv->props.end()) diff --git a/Analysis/src/Autocomplete.cpp b/Analysis/src/Autocomplete.cpp index 1cfa90dc..b31d85aa 100644 --- a/Analysis/src/Autocomplete.cpp +++ b/Analysis/src/Autocomplete.cpp @@ -210,10 +210,10 @@ static TypeCorrectKind checkTypeCorrectKind(const Module& module, TypeArena* typ return TypeCorrectKind::None; auto it = module.astExpectedTypes.find(expr); - if (it == module.astExpectedTypes.end()) + if (!it) return TypeCorrectKind::None; - TypeId expectedType = follow(it->second); + TypeId expectedType = follow(*it); if (canUnify(expectedType, ty)) return TypeCorrectKind::Correct; @@ -682,10 +682,10 @@ static std::optional functionIsExpectedAt(const Module& module, AstNode* n return std::nullopt; auto it = module.astExpectedTypes.find(expr); - if (it == module.astExpectedTypes.end()) + if (!it) return std::nullopt; - TypeId expectedType = follow(it->second); + TypeId expectedType = follow(*it); if (const FunctionTypeVar* ftv = get(expectedType)) return true; @@ -784,9 +784,9 @@ AutocompleteEntryMap autocompleteTypeNames(const Module& module, Position positi if (AstExprCall* exprCall = expr->as()) { - if (auto it = module.astTypes.find(exprCall->func); it != module.astTypes.end()) + if (auto it = module.astTypes.find(exprCall->func)) { - if (const FunctionTypeVar* ftv = get(follow(it->second))) + if (const FunctionTypeVar* ftv = get(follow(*it))) { if (auto ty = tryGetTypePackTypeAt(ftv->retType, tailPos)) inferredType = *ty; @@ -798,8 +798,8 @@ AutocompleteEntryMap autocompleteTypeNames(const Module& module, Position positi if (tailPos != 0) break; - if (auto it = module.astTypes.find(expr); it != module.astTypes.end()) - inferredType = it->second; + if (auto it = module.astTypes.find(expr)) + inferredType = *it; } if (inferredType) @@ -815,10 +815,10 @@ AutocompleteEntryMap autocompleteTypeNames(const Module& module, Position positi auto tryGetExpectedFunctionType = [](const Module& module, AstExpr* expr) -> const FunctionTypeVar* { auto it = module.astExpectedTypes.find(expr); - if (it == module.astExpectedTypes.end()) + if (!it) return nullptr; - TypeId ty = follow(it->second); + TypeId ty = follow(*it); if (const FunctionTypeVar* ftv = get(ty)) return ftv; @@ -1129,9 +1129,8 @@ static void autocompleteExpression(const SourceModule& sourceModule, const Modul if (node->is()) { - auto it = module.astTypes.find(node->asExpr()); - if (it != module.astTypes.end()) - autocompleteProps(module, typeArena, it->second, PropIndexType::Point, ancestry, result); + if (auto it = module.astTypes.find(node->asExpr())) + autocompleteProps(module, typeArena, *it, PropIndexType::Point, ancestry, result); } else if (FFlag::LuauIfElseExpressionAnalysisSupport && autocompleteIfElseExpression(node, ancestry, position, result)) return; @@ -1203,13 +1202,13 @@ static std::optional getMethodContainingClass(const ModuleP return std::nullopt; } - auto parentIter = module->astTypes.find(parentExpr); - if (parentIter == module->astTypes.end()) + auto parentIt = module->astTypes.find(parentExpr); + if (!parentIt) { return std::nullopt; } - Luau::TypeId parentType = Luau::follow(parentIter->second); + Luau::TypeId parentType = Luau::follow(*parentIt); if (auto parentClass = Luau::get(parentType)) { @@ -1250,8 +1249,8 @@ static std::optional autocompleteStringParams(const Source return std::nullopt; } - auto iter = module->astTypes.find(candidate->func); - if (iter == module->astTypes.end()) + auto it = module->astTypes.find(candidate->func); + if (!it) { return std::nullopt; } @@ -1267,7 +1266,7 @@ static std::optional autocompleteStringParams(const Source return std::nullopt; }; - auto followedId = Luau::follow(iter->second); + auto followedId = Luau::follow(*it); if (auto functionType = Luau::get(followedId)) { return performCallback(functionType); @@ -1316,10 +1315,10 @@ static AutocompleteResult autocomplete(const SourceModule& sourceModule, const M if (auto indexName = node->as()) { auto it = module->astTypes.find(indexName->expr); - if (it == module->astTypes.end()) + if (!it) return {}; - TypeId ty = follow(it->second); + TypeId ty = follow(*it); PropIndexType indexType = indexName->op == ':' ? PropIndexType::Colon : PropIndexType::Point; if (isString(ty)) @@ -1447,9 +1446,9 @@ static AutocompleteResult autocomplete(const SourceModule& sourceModule, const M // If item doesn't have a key, maybe the value is actually the key if (key ? key == node : node->is() && value == node) { - if (auto it = module->astExpectedTypes.find(exprTable); it != module->astExpectedTypes.end()) + if (auto it = module->astExpectedTypes.find(exprTable)) { - auto result = autocompleteProps(*module, typeArena, it->second, PropIndexType::Key, finder.ancestry); + auto result = autocompleteProps(*module, typeArena, *it, PropIndexType::Key, finder.ancestry); // Remove keys that are already completed for (const auto& item : exprTable->items) @@ -1485,9 +1484,9 @@ static AutocompleteResult autocomplete(const SourceModule& sourceModule, const M { if (auto idxExpr = finder.ancestry.at(finder.ancestry.size() - 2)->as()) { - if (auto it = module->astTypes.find(idxExpr->expr); it != module->astTypes.end()) + if (auto it = module->astTypes.find(idxExpr->expr)) { - return {autocompleteProps(*module, typeArena, follow(it->second), PropIndexType::Point, finder.ancestry), finder.ancestry}; + return {autocompleteProps(*module, typeArena, follow(*it), PropIndexType::Point, finder.ancestry), finder.ancestry}; } } } diff --git a/Analysis/src/BuiltinDefinitions.cpp b/Analysis/src/BuiltinDefinitions.cpp index 68ad5ac9..3b0c2163 100644 --- a/Analysis/src/BuiltinDefinitions.cpp +++ b/Analysis/src/BuiltinDefinitions.cpp @@ -11,7 +11,7 @@ LUAU_FASTFLAG(LuauParseGenericFunctions) LUAU_FASTFLAG(LuauGenericFunctions) LUAU_FASTFLAG(LuauRankNTypes) -LUAU_FASTFLAG(LuauStringMetatable) +LUAU_FASTFLAG(LuauNewRequireTrace) /** FIXME: Many of these type definitions are not quite completely accurate. * @@ -218,7 +218,6 @@ void registerBuiltinTypes(TypeChecker& typeChecker) TypePackId anyTypePack = typeChecker.anyTypePack; TypePackId numberVariadicList = arena.addTypePack(TypePackVar{VariadicTypePack{numberType}}); - TypePackId stringVariadicList = arena.addTypePack(TypePackVar{VariadicTypePack{stringType}}); TypePackId listOfAtLeastOneNumber = arena.addTypePack(TypePack{{numberType}, numberVariadicList}); TypeId listOfAtLeastOneNumberToNumberType = arena.addType(FunctionTypeVar{ @@ -255,85 +254,18 @@ void registerBuiltinTypes(TypeChecker& typeChecker) TypeId genericV = arena.addType(GenericTypeVar{"V"}); TypeId mapOfKtoV = arena.addType(TableTypeVar{{}, TableIndexer(genericK, genericV), typeChecker.globalScope->level}); - if (FFlag::LuauStringMetatable) + std::optional stringMetatableTy = getMetatable(singletonTypes.stringType); + LUAU_ASSERT(stringMetatableTy); + const TableTypeVar* stringMetatableTable = get(follow(*stringMetatableTy)); + LUAU_ASSERT(stringMetatableTable); + + auto it = stringMetatableTable->props.find("__index"); + LUAU_ASSERT(it != stringMetatableTable->props.end()); + + addGlobalBinding(typeChecker, "string", it->second.type, "@luau"); + + if (!FFlag::LuauParseGenericFunctions || !FFlag::LuauGenericFunctions) { - std::optional stringMetatableTy = getMetatable(singletonTypes.stringType); - LUAU_ASSERT(stringMetatableTy); - const TableTypeVar* stringMetatableTable = get(follow(*stringMetatableTy)); - LUAU_ASSERT(stringMetatableTable); - - auto it = stringMetatableTable->props.find("__index"); - LUAU_ASSERT(it != stringMetatableTable->props.end()); - - TypeId stringLib = it->second.type; - addGlobalBinding(typeChecker, "string", stringLib, "@luau"); - } - - if (FFlag::LuauParseGenericFunctions && FFlag::LuauGenericFunctions) - { - if (!FFlag::LuauStringMetatable) - { - TypeId stringLibTy = getGlobalBinding(typeChecker, "string"); - TableTypeVar* stringLib = getMutable(stringLibTy); - TypeId replArgType = makeUnion( - arena, {stringType, - arena.addType(TableTypeVar({}, TableIndexer(stringType, stringType), typeChecker.globalScope->level, TableState::Generic)), - makeFunction(arena, std::nullopt, {stringType}, {stringType})}); - TypeId gsubFunc = makeFunction(arena, stringType, {stringType, replArgType, optionalNumber}, {stringType, numberType}); - - stringLib->props["gsub"] = makeProperty(gsubFunc, "@luau/global/string.gsub"); - } - } - else - { - if (!FFlag::LuauStringMetatable) - { - TypeId stringToStringType = makeFunction(arena, std::nullopt, {stringType}, {stringType}); - - TypeId gmatchFunc = makeFunction(arena, stringType, {stringType}, {arena.addType(FunctionTypeVar{emptyPack, stringVariadicList})}); - - TypeId replArgType = makeUnion( - arena, {stringType, - arena.addType(TableTypeVar({}, TableIndexer(stringType, stringType), typeChecker.globalScope->level, TableState::Generic)), - makeFunction(arena, std::nullopt, {stringType}, {stringType})}); - TypeId gsubFunc = makeFunction(arena, stringType, {stringType, replArgType, optionalNumber}, {stringType, numberType}); - - TypeId formatFn = arena.addType(FunctionTypeVar{arena.addTypePack(TypePack{{stringType}, anyTypePack}), oneStringPack}); - - TableTypeVar::Props stringLib = { - // FIXME string.byte "can" return a pack of numbers, but only if 2nd or 3rd arguments were supplied - {"byte", {makeFunction(arena, stringType, {optionalNumber, optionalNumber}, {optionalNumber})}}, - // FIXME char takes a variadic pack of numbers - {"char", {makeFunction(arena, std::nullopt, {numberType, optionalNumber, optionalNumber, optionalNumber}, {stringType})}}, - {"find", {makeFunction(arena, stringType, {stringType, optionalNumber, optionalBoolean}, {optionalNumber, optionalNumber})}}, - {"format", {formatFn}}, // FIXME - {"gmatch", {gmatchFunc}}, - {"gsub", {gsubFunc}}, - {"len", {makeFunction(arena, stringType, {}, {numberType})}}, - {"lower", {stringToStringType}}, - {"match", {makeFunction(arena, stringType, {stringType, optionalNumber}, {optionalString})}}, - {"rep", {makeFunction(arena, stringType, {numberType}, {stringType})}}, - {"reverse", {stringToStringType}}, - {"sub", {makeFunction(arena, stringType, {numberType, optionalNumber}, {stringType})}}, - {"upper", {stringToStringType}}, - {"split", {makeFunction(arena, stringType, {stringType, optionalString}, - {arena.addType(TableTypeVar{{}, TableIndexer{numberType, stringType}, typeChecker.globalScope->level})})}}, - {"pack", {arena.addType(FunctionTypeVar{ - arena.addTypePack(TypePack{{stringType}, anyTypePack}), - oneStringPack, - })}}, - {"packsize", {makeFunction(arena, stringType, {}, {numberType})}}, - {"unpack", {arena.addType(FunctionTypeVar{ - arena.addTypePack(TypePack{{stringType, stringType, optionalNumber}}), - anyTypePack, - })}}, - }; - - assignPropDocumentationSymbols(stringLib, "@luau/global/string"); - addGlobalBinding(typeChecker, "string", - arena.addType(TableTypeVar{stringLib, std::nullopt, typeChecker.globalScope->level, TableState::Sealed}), "@luau"); - } - TableTypeVar::Props debugLib{ {"info", {makeIntersection(arena, { @@ -601,9 +533,6 @@ void registerBuiltinTypes(TypeChecker& typeChecker) auto tableLib = getMutable(getGlobalBinding(typeChecker, "table")); attachMagicFunction(tableLib->props["pack"].type, magicFunctionPack); - auto stringLib = getMutable(getGlobalBinding(typeChecker, "string")); - attachMagicFunction(stringLib->props["format"].type, magicFunctionFormat); - attachMagicFunction(getGlobalBinding(typeChecker, "require"), magicFunctionRequire); } @@ -791,11 +720,11 @@ static std::optional> magicFunctionRequire( return std::nullopt; } - AstExpr* require = expr.args.data[0]; - - if (!checkRequirePath(typechecker, require)) + if (!checkRequirePath(typechecker, expr.args.data[0])) return std::nullopt; + const AstExpr* require = FFlag::LuauNewRequireTrace ? &expr : expr.args.data[0]; + if (auto moduleInfo = typechecker.resolver->resolveModuleInfo(typechecker.currentModuleName, *require)) return ExprResult{arena.addTypePack({typechecker.checkRequire(scope, *moduleInfo, expr.location)})}; diff --git a/Analysis/src/EmbeddedBuiltinDefinitions.cpp b/Analysis/src/EmbeddedBuiltinDefinitions.cpp index 61a63f06..1e91561a 100644 --- a/Analysis/src/EmbeddedBuiltinDefinitions.cpp +++ b/Analysis/src/EmbeddedBuiltinDefinitions.cpp @@ -206,27 +206,6 @@ std::string getBuiltinDefinitionSource() graphemes: (string, number?, number?) -> (() -> (number, number)), } - declare string: { - byte: (string, number?, number?) -> ...number, - char: (number, ...number) -> string, - find: (string, string, number?, boolean?) -> (number?, number?), - -- `string.format` has a magic function attached that will provide more type information for literal format strings. - format: (string, A...) -> string, - gmatch: (string, string) -> () -> (...string), - -- gsub is defined in C++ because we don't have syntax for describing a generic table. - len: (string) -> number, - lower: (string) -> string, - match: (string, string, number?) -> string?, - rep: (string, number) -> string, - reverse: (string) -> string, - sub: (string, number, number?) -> string, - upper: (string) -> string, - split: (string, string, string?) -> {string}, - pack: (string, A...) -> string, - packsize: (string) -> number, - unpack: (string, string, number?) -> R..., - } - -- Cannot use `typeof` here because it will produce a polytype when we expect a monotype. declare function unpack(tab: {V}, i: number?, j: number?): ...V )"; diff --git a/Analysis/src/Error.cpp b/Analysis/src/Error.cpp index a622f8a5..04d91444 100644 --- a/Analysis/src/Error.cpp +++ b/Analysis/src/Error.cpp @@ -7,9 +7,9 @@ #include -LUAU_FASTFLAG(LuauFasterStringifier) +LUAU_FASTFLAG(LuauTypeAliasPacks) -static std::string wrongNumberOfArgsString(size_t expectedCount, size_t actualCount, bool isTypeArgs = false) +static std::string wrongNumberOfArgsString_DEPRECATED(size_t expectedCount, size_t actualCount, bool isTypeArgs = false) { std::string s = "expects " + std::to_string(expectedCount) + " "; @@ -41,6 +41,52 @@ static std::string wrongNumberOfArgsString(size_t expectedCount, size_t actualCo return s; } +static std::string wrongNumberOfArgsString(size_t expectedCount, size_t actualCount, const char* argPrefix = nullptr, bool isVariadic = false) +{ + std::string s; + + if (FFlag::LuauTypeAliasPacks) + { + s = "expects "; + + if (isVariadic) + s += "at least "; + + s += std::to_string(expectedCount) + " "; + } + else + { + s = "expects " + std::to_string(expectedCount) + " "; + } + + if (argPrefix) + s += std::string(argPrefix) + " "; + + s += "argument"; + if (expectedCount != 1) + s += "s"; + + s += ", but "; + + if (actualCount == 0) + { + s += "none"; + } + else + { + if (actualCount < expectedCount) + s += "only "; + + s += std::to_string(actualCount); + } + + s += (actualCount == 1) ? " is" : " are"; + + s += " specified"; + + return s; +} + namespace Luau { @@ -127,7 +173,10 @@ struct ErrorConverter return "Function only returns " + std::to_string(e.expected) + " value" + expectedS + ". " + std::to_string(e.actual) + " are required here"; case CountMismatch::Arg: - return "Argument count mismatch. Function " + wrongNumberOfArgsString(e.expected, e.actual); + if (FFlag::LuauTypeAliasPacks) + return "Argument count mismatch. Function " + wrongNumberOfArgsString(e.expected, e.actual); + else + return "Argument count mismatch. Function " + wrongNumberOfArgsString_DEPRECATED(e.expected, e.actual); } LUAU_ASSERT(!"Unknown context"); @@ -159,13 +208,16 @@ struct ErrorConverter std::string operator()(const Luau::UnknownRequire& e) const { - return "Unknown require: " + e.modulePath; + if (e.modulePath.empty()) + return "Unknown require: unsupported path"; + else + return "Unknown require: " + e.modulePath; } std::string operator()(const Luau::IncorrectGenericParameterCount& e) const { std::string name = e.name; - if (!e.typeFun.typeParams.empty()) + if (!e.typeFun.typeParams.empty() || (FFlag::LuauTypeAliasPacks && !e.typeFun.typePackParams.empty())) { name += "<"; bool first = true; @@ -178,10 +230,37 @@ struct ErrorConverter name += toString(t); } + + if (FFlag::LuauTypeAliasPacks) + { + for (TypePackId t : e.typeFun.typePackParams) + { + if (first) + first = false; + else + name += ", "; + + name += toString(t); + } + } + name += ">"; } - return "Generic type '" + name + "' " + wrongNumberOfArgsString(e.typeFun.typeParams.size(), e.actualParameters, /*isTypeArgs*/ true); + if (FFlag::LuauTypeAliasPacks) + { + if (e.typeFun.typeParams.size() != e.actualParameters) + return "Generic type '" + name + "' " + + wrongNumberOfArgsString(e.typeFun.typeParams.size(), e.actualParameters, "type", !e.typeFun.typePackParams.empty()); + + return "Generic type '" + name + "' " + + wrongNumberOfArgsString(e.typeFun.typePackParams.size(), e.actualPackParameters, "type pack", /*isVariadic*/ false); + } + else + { + return "Generic type '" + name + "' " + + wrongNumberOfArgsString_DEPRECATED(e.typeFun.typeParams.size(), e.actualParameters, /*isTypeArgs*/ true); + } } std::string operator()(const Luau::SyntaxError& e) const @@ -470,9 +549,26 @@ bool IncorrectGenericParameterCount::operator==(const IncorrectGenericParameterC if (typeFun.typeParams.size() != rhs.typeFun.typeParams.size()) return false; + if (FFlag::LuauTypeAliasPacks) + { + if (typeFun.typePackParams.size() != rhs.typeFun.typePackParams.size()) + return false; + } + for (size_t i = 0; i < typeFun.typeParams.size(); ++i) + { if (typeFun.typeParams[i] != rhs.typeFun.typeParams[i]) return false; + } + + if (FFlag::LuauTypeAliasPacks) + { + for (size_t i = 0; i < typeFun.typePackParams.size(); ++i) + { + if (typeFun.typePackParams[i] != rhs.typeFun.typePackParams[i]) + return false; + } + } return true; } diff --git a/Analysis/src/Frontend.cpp b/Analysis/src/Frontend.cpp index 4d385ec1..b2529840 100644 --- a/Analysis/src/Frontend.cpp +++ b/Analysis/src/Frontend.cpp @@ -1,9 +1,12 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #include "Luau/Frontend.h" +#include "Luau/Common.h" #include "Luau/Config.h" #include "Luau/FileResolver.h" +#include "Luau/Scope.h" #include "Luau/StringUtils.h" +#include "Luau/TimeTrace.h" #include "Luau/TypeInfer.h" #include "Luau/Variant.h" #include "Luau/Common.h" @@ -19,6 +22,7 @@ LUAU_FASTFLAGVARIABLE(LuauSecondTypecheckKnowsTheDataModel, false) LUAU_FASTFLAGVARIABLE(LuauResolveModuleNameWithoutACurrentModule, false) LUAU_FASTFLAG(LuauTraceRequireLookupChild) LUAU_FASTFLAGVARIABLE(LuauPersistDefinitionFileTypes, false) +LUAU_FASTFLAG(LuauNewRequireTrace) namespace Luau { @@ -69,6 +73,8 @@ static void generateDocumentationSymbols(TypeId ty, const std::string& rootName) LoadDefinitionFileResult loadDefinitionFile(TypeChecker& typeChecker, ScopePtr targetScope, std::string_view source, const std::string& packageName) { + LUAU_TIMETRACE_SCOPE("loadDefinitionFile", "Frontend"); + Luau::Allocator allocator; Luau::AstNameTable names(allocator); @@ -350,6 +356,9 @@ FrontendModuleResolver::FrontendModuleResolver(Frontend* frontend) CheckResult Frontend::check(const ModuleName& name) { + LUAU_TIMETRACE_SCOPE("Frontend::check", "Frontend"); + LUAU_TIMETRACE_ARGUMENT("name", name.c_str()); + CheckResult checkResult; auto it = sourceNodes.find(name); @@ -479,6 +488,9 @@ CheckResult Frontend::check(const ModuleName& name) bool Frontend::parseGraph(std::vector& buildQueue, CheckResult& checkResult, const ModuleName& root) { + LUAU_TIMETRACE_SCOPE("Frontend::parseGraph", "Frontend"); + LUAU_TIMETRACE_ARGUMENT("root", root.c_str()); + // https://en.wikipedia.org/wiki/Topological_sorting#Depth-first_search enum Mark { @@ -597,6 +609,9 @@ ScopePtr Frontend::getModuleEnvironment(const SourceModule& module, const Config LintResult Frontend::lint(const ModuleName& name, std::optional enabledLintWarnings) { + LUAU_TIMETRACE_SCOPE("Frontend::lint", "Frontend"); + LUAU_TIMETRACE_ARGUMENT("name", name.c_str()); + CheckResult checkResult; auto [_sourceNode, sourceModule] = getSourceNode(checkResult, name); @@ -608,6 +623,8 @@ LintResult Frontend::lint(const ModuleName& name, std::optional Frontend::lintFragment(std::string_view source, std::optional enabledLintWarnings) { + LUAU_TIMETRACE_SCOPE("Frontend::lintFragment", "Frontend"); + const Config& config = configResolver->getConfig(""); SourceModule sourceModule = parse(ModuleName{}, source, config.parseOptions); @@ -627,6 +644,9 @@ std::pair Frontend::lintFragment(std::string_view sour CheckResult Frontend::check(const SourceModule& module) { + LUAU_TIMETRACE_SCOPE("Frontend::check", "Frontend"); + LUAU_TIMETRACE_ARGUMENT("module", module.name.c_str()); + const Config& config = configResolver->getConfig(module.name); Mode mode = module.mode.value_or(config.mode); @@ -648,6 +668,9 @@ CheckResult Frontend::check(const SourceModule& module) LintResult Frontend::lint(const SourceModule& module, std::optional enabledLintWarnings) { + LUAU_TIMETRACE_SCOPE("Frontend::lint", "Frontend"); + LUAU_TIMETRACE_ARGUMENT("module", module.name.c_str()); + const Config& config = configResolver->getConfig(module.name); LintOptions options = enabledLintWarnings.value_or(config.enabledLint); @@ -746,6 +769,9 @@ const SourceModule* Frontend::getSourceModule(const ModuleName& moduleName) cons // Read AST into sourceModules if necessary. Trace require()s. Report parse errors. std::pair Frontend::getSourceNode(CheckResult& checkResult, const ModuleName& name) { + LUAU_TIMETRACE_SCOPE("Frontend::getSourceNode", "Frontend"); + LUAU_TIMETRACE_ARGUMENT("name", name.c_str()); + auto it = sourceNodes.find(name); if (it != sourceNodes.end() && !it->second.dirty) { @@ -815,6 +841,9 @@ std::pair Frontend::getSourceNode(CheckResult& check */ SourceModule Frontend::parse(const ModuleName& name, std::string_view src, const ParseOptions& parseOptions) { + LUAU_TIMETRACE_SCOPE("Frontend::parse", "Frontend"); + LUAU_TIMETRACE_ARGUMENT("name", name.c_str()); + SourceModule sourceModule; double timestamp = getTimestamp(); @@ -864,20 +893,11 @@ std::optional FrontendModuleResolver::resolveModuleInfo(const Module const auto& exprs = it->second.exprs; - const ModuleName* relativeName = exprs.find(&pathExpr); - if (!relativeName || relativeName->empty()) + const ModuleInfo* info = exprs.find(&pathExpr); + if (!info || (!FFlag::LuauNewRequireTrace && info->name.empty())) return std::nullopt; - if (FFlag::LuauTraceRequireLookupChild) - { - const bool* optional = it->second.optional.find(&pathExpr); - - return {{*relativeName, optional ? *optional : false}}; - } - else - { - return {{*relativeName, false}}; - } + return *info; } const ModulePtr FrontendModuleResolver::getModule(const ModuleName& moduleName) const @@ -891,12 +911,15 @@ const ModulePtr FrontendModuleResolver::getModule(const ModuleName& moduleName) bool FrontendModuleResolver::moduleExists(const ModuleName& moduleName) const { - return frontend->fileResolver->moduleExists(moduleName); + if (FFlag::LuauNewRequireTrace) + return frontend->sourceNodes.count(moduleName) != 0; + else + return frontend->fileResolver->moduleExists(moduleName); } std::string FrontendModuleResolver::getHumanReadableModuleName(const ModuleName& moduleName) const { - return frontend->fileResolver->getHumanReadableModuleName_(moduleName).value_or(moduleName); + return frontend->fileResolver->getHumanReadableModuleName(moduleName); } ScopePtr Frontend::addEnvironment(const std::string& environmentName) diff --git a/Analysis/src/IostreamHelpers.cpp b/Analysis/src/IostreamHelpers.cpp index 84e9b77f..3b267121 100644 --- a/Analysis/src/IostreamHelpers.cpp +++ b/Analysis/src/IostreamHelpers.cpp @@ -2,6 +2,8 @@ #include "Luau/IostreamHelpers.h" #include "Luau/ToString.h" +LUAU_FASTFLAG(LuauTypeAliasPacks) + namespace Luau { @@ -92,7 +94,7 @@ std::ostream& operator<<(std::ostream& stream, const IncorrectGenericParameterCo { stream << "IncorrectGenericParameterCount { name = " << error.name; - if (!error.typeFun.typeParams.empty()) + if (!error.typeFun.typeParams.empty() || (FFlag::LuauTypeAliasPacks && !error.typeFun.typePackParams.empty())) { stream << "<"; bool first = true; @@ -105,6 +107,20 @@ std::ostream& operator<<(std::ostream& stream, const IncorrectGenericParameterCo stream << toString(t); } + + if (FFlag::LuauTypeAliasPacks) + { + for (TypePackId t : error.typeFun.typePackParams) + { + if (first) + first = false; + else + stream << ", "; + + stream << toString(t); + } + } + stream << ">"; } diff --git a/Analysis/src/JsonEncoder.cpp b/Analysis/src/JsonEncoder.cpp index a1018297..064accba 100644 --- a/Analysis/src/JsonEncoder.cpp +++ b/Analysis/src/JsonEncoder.cpp @@ -3,6 +3,9 @@ #include "Luau/Ast.h" #include "Luau/StringUtils.h" +#include "Luau/Common.h" + +LUAU_FASTFLAG(LuauTypeAliasPacks) namespace Luau { @@ -612,6 +615,12 @@ struct AstJsonEncoder : public AstVisitor writeNode(node, "AstStatTypeAlias", [&]() { PROP(name); PROP(generics); + + if (FFlag::LuauTypeAliasPacks) + { + PROP(genericPacks); + } + PROP(type); PROP(exported); }); @@ -664,13 +673,21 @@ struct AstJsonEncoder : public AstVisitor }); } + void write(struct AstTypeOrPack node) + { + if (node.type) + write(node.type); + else + write(node.typePack); + } + void write(class AstTypeReference* node) { writeNode(node, "AstTypeReference", [&]() { if (node->hasPrefix) PROP(prefix); PROP(name); - PROP(generics); + PROP(parameters); }); } @@ -734,6 +751,13 @@ struct AstJsonEncoder : public AstVisitor }); } + void write(class AstTypePackExplicit* node) + { + writeNode(node, "AstTypePackExplicit", [&]() { + PROP(typeList); + }); + } + void write(class AstTypePackVariadic* node) { writeNode(node, "AstTypePackVariadic", [&]() { @@ -1018,6 +1042,12 @@ struct AstJsonEncoder : public AstVisitor return false; } + bool visit(class AstTypePackExplicit* node) override + { + write(node); + return false; + } + bool visit(class AstTypePackVariadic* node) override { write(node); diff --git a/Analysis/src/Linter.cpp b/Analysis/src/Linter.cpp index f97f6a4a..bff947a5 100644 --- a/Analysis/src/Linter.cpp +++ b/Analysis/src/Linter.cpp @@ -3,6 +3,7 @@ #include "Luau/AstQuery.h" #include "Luau/Module.h" +#include "Luau/Scope.h" #include "Luau/TypeInfer.h" #include "Luau/StringUtils.h" #include "Luau/Common.h" @@ -12,6 +13,7 @@ #include LUAU_FASTFLAGVARIABLE(LuauLinterUnknownTypeVectorAware, false) +LUAU_FASTFLAGVARIABLE(LuauLinterTableMoveZero, false) namespace Luau { @@ -85,10 +87,10 @@ struct LintContext return std::nullopt; auto it = module->astTypes.find(expr); - if (it == module->astTypes.end()) + if (!it) return std::nullopt; - return it->second; + return *it; } }; @@ -2144,6 +2146,19 @@ private: "wrap it in parentheses to silence"); } + if (FFlag::LuauLinterTableMoveZero && func->index == "move" && node->args.size >= 4) + { + // table.move(t, 0, _, _) + if (isConstant(args[1], 0.0)) + emitWarning(*context, LintWarning::Code_TableOperations, args[1]->location, + "table.move uses index 0 but arrays are 1-based; did you mean 1 instead?"); + + // table.move(t, _, _, 0) + else if (isConstant(args[3], 0.0)) + emitWarning(*context, LintWarning::Code_TableOperations, args[3]->location, + "table.move uses index 0 but arrays are 1-based; did you mean 1 instead?"); + } + return true; } diff --git a/Analysis/src/Module.cpp b/Analysis/src/Module.cpp index f1d975fe..df6be767 100644 --- a/Analysis/src/Module.cpp +++ b/Analysis/src/Module.cpp @@ -1,6 +1,7 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #include "Luau/Module.h" +#include "Luau/Scope.h" #include "Luau/TypeInfer.h" #include "Luau/TypePack.h" #include "Luau/TypeVar.h" @@ -13,6 +14,7 @@ LUAU_FASTFLAGVARIABLE(DebugLuauFreezeArena, false) LUAU_FASTFLAGVARIABLE(DebugLuauTrackOwningArena, false) LUAU_FASTFLAG(LuauSecondTypecheckKnowsTheDataModel) LUAU_FASTFLAG(LuauCaptureBrokenCommentSpans) +LUAU_FASTFLAG(LuauTypeAliasPacks) namespace Luau { @@ -188,7 +190,7 @@ struct TypePackCloner template void defaultClone(const T& t) { - TypePackId cloned = dest.typePacks.allocate(t); + TypePackId cloned = dest.addTypePack(TypePackVar{t}); seenTypePacks[typePackId] = cloned; } @@ -197,7 +199,7 @@ struct TypePackCloner if (encounteredFreeType) *encounteredFreeType = true; - seenTypePacks[typePackId] = dest.typePacks.allocate(TypePackVar{Unifiable::Error{}}); + seenTypePacks[typePackId] = dest.addTypePack(TypePackVar{Unifiable::Error{}}); } void operator()(const Unifiable::Generic& t) @@ -219,13 +221,13 @@ struct TypePackCloner void operator()(const VariadicTypePack& t) { - TypePackId cloned = dest.typePacks.allocate(VariadicTypePack{clone(t.ty, dest, seenTypes, seenTypePacks, encounteredFreeType)}); + TypePackId cloned = dest.addTypePack(TypePackVar{VariadicTypePack{clone(t.ty, dest, seenTypes, seenTypePacks, encounteredFreeType)}}); seenTypePacks[typePackId] = cloned; } void operator()(const TypePack& t) { - TypePackId cloned = dest.typePacks.allocate(TypePack{}); + TypePackId cloned = dest.addTypePack(TypePack{}); TypePack* destTp = getMutable(cloned); LUAU_ASSERT(destTp != nullptr); seenTypePacks[typePackId] = cloned; @@ -241,7 +243,7 @@ struct TypePackCloner template void TypeCloner::defaultClone(const T& t) { - TypeId cloned = dest.typeVars.allocate(t); + TypeId cloned = dest.addType(t); seenTypes[typeId] = cloned; } @@ -250,7 +252,7 @@ void TypeCloner::operator()(const Unifiable::Free& t) if (encounteredFreeType) *encounteredFreeType = true; - seenTypes[typeId] = dest.typeVars.allocate(ErrorTypeVar{}); + seenTypes[typeId] = dest.addType(ErrorTypeVar{}); } void TypeCloner::operator()(const Unifiable::Generic& t) @@ -275,7 +277,7 @@ void TypeCloner::operator()(const PrimitiveTypeVar& t) void TypeCloner::operator()(const FunctionTypeVar& t) { - TypeId result = dest.typeVars.allocate(FunctionTypeVar{TypeLevel{0, 0}, {}, {}, nullptr, nullptr, t.definition, t.hasSelf}); + TypeId result = dest.addType(FunctionTypeVar{TypeLevel{0, 0}, {}, {}, nullptr, nullptr, t.definition, t.hasSelf}); FunctionTypeVar* ftv = getMutable(result); LUAU_ASSERT(ftv != nullptr); @@ -297,7 +299,7 @@ void TypeCloner::operator()(const FunctionTypeVar& t) void TypeCloner::operator()(const TableTypeVar& t) { - TypeId result = dest.typeVars.allocate(TableTypeVar{}); + TypeId result = dest.addType(TableTypeVar{}); TableTypeVar* ttv = getMutable(result); LUAU_ASSERT(ttv != nullptr); @@ -323,7 +325,13 @@ void TypeCloner::operator()(const TableTypeVar& t) ttv->boundTo = clone(*t.boundTo, dest, seenTypes, seenTypePacks, encounteredFreeType); for (TypeId& arg : ttv->instantiatedTypeParams) - arg = (clone(arg, dest, seenTypes, seenTypePacks, encounteredFreeType)); + arg = clone(arg, dest, seenTypes, seenTypePacks, encounteredFreeType); + + if (FFlag::LuauTypeAliasPacks) + { + for (TypePackId& arg : ttv->instantiatedTypePackParams) + arg = clone(arg, dest, seenTypes, seenTypePacks, encounteredFreeType); + } if (ttv->state == TableState::Free) { @@ -343,7 +351,7 @@ void TypeCloner::operator()(const TableTypeVar& t) void TypeCloner::operator()(const MetatableTypeVar& t) { - TypeId result = dest.typeVars.allocate(MetatableTypeVar{}); + TypeId result = dest.addType(MetatableTypeVar{}); MetatableTypeVar* mtv = getMutable(result); seenTypes[typeId] = result; @@ -353,7 +361,7 @@ void TypeCloner::operator()(const MetatableTypeVar& t) void TypeCloner::operator()(const ClassTypeVar& t) { - TypeId result = dest.typeVars.allocate(ClassTypeVar{t.name, {}, std::nullopt, std::nullopt, t.tags, t.userData}); + TypeId result = dest.addType(ClassTypeVar{t.name, {}, std::nullopt, std::nullopt, t.tags, t.userData}); ClassTypeVar* ctv = getMutable(result); seenTypes[typeId] = result; @@ -378,7 +386,7 @@ void TypeCloner::operator()(const AnyTypeVar& t) void TypeCloner::operator()(const UnionTypeVar& t) { - TypeId result = dest.typeVars.allocate(UnionTypeVar{}); + TypeId result = dest.addType(UnionTypeVar{}); seenTypes[typeId] = result; UnionTypeVar* option = getMutable(result); @@ -390,7 +398,7 @@ void TypeCloner::operator()(const UnionTypeVar& t) void TypeCloner::operator()(const IntersectionTypeVar& t) { - TypeId result = dest.typeVars.allocate(IntersectionTypeVar{}); + TypeId result = dest.addType(IntersectionTypeVar{}); seenTypes[typeId] = result; IntersectionTypeVar* option = getMutable(result); @@ -451,8 +459,14 @@ TypeId clone(TypeId typeId, TypeArena& dest, SeenTypes& seenTypes, SeenTypePacks TypeFun clone(const TypeFun& typeFun, TypeArena& dest, SeenTypes& seenTypes, SeenTypePacks& seenTypePacks, bool* encounteredFreeType) { TypeFun result; - for (TypeId param : typeFun.typeParams) - result.typeParams.push_back(clone(param, dest, seenTypes, seenTypePacks, encounteredFreeType)); + for (TypeId ty : typeFun.typeParams) + result.typeParams.push_back(clone(ty, dest, seenTypes, seenTypePacks, encounteredFreeType)); + + if (FFlag::LuauTypeAliasPacks) + { + for (TypePackId tp : typeFun.typePackParams) + result.typePackParams.push_back(clone(tp, dest, seenTypes, seenTypePacks, encounteredFreeType)); + } result.type = clone(typeFun.type, dest, seenTypes, seenTypePacks, encounteredFreeType); diff --git a/Analysis/src/RequireTracer.cpp b/Analysis/src/RequireTracer.cpp index c9e15cba..4634effd 100644 --- a/Analysis/src/RequireTracer.cpp +++ b/Analysis/src/RequireTracer.cpp @@ -5,6 +5,7 @@ #include "Luau/Module.h" LUAU_FASTFLAGVARIABLE(LuauTraceRequireLookupChild, false) +LUAU_FASTFLAGVARIABLE(LuauNewRequireTrace, false) namespace Luau { @@ -12,17 +13,18 @@ namespace Luau namespace { -struct RequireTracer : AstVisitor +struct RequireTracerOld : AstVisitor { - explicit RequireTracer(FileResolver* fileResolver, ModuleName currentModuleName) + explicit RequireTracerOld(FileResolver* fileResolver, const ModuleName& currentModuleName) : fileResolver(fileResolver) - , currentModuleName(std::move(currentModuleName)) + , currentModuleName(currentModuleName) { + LUAU_ASSERT(!FFlag::LuauNewRequireTrace); } FileResolver* const fileResolver; ModuleName currentModuleName; - DenseHashMap locals{0}; + DenseHashMap locals{nullptr}; RequireTraceResult result; std::optional fromAstFragment(AstExpr* expr) @@ -50,9 +52,9 @@ struct RequireTracer : AstVisitor AstExpr* expr = stat->values.data[i]; expr->visit(this); - const ModuleName* name = result.exprs.find(expr); - if (name) - locals[local] = *name; + const ModuleInfo* info = result.exprs.find(expr); + if (info) + locals[local] = info->name; } } @@ -63,7 +65,7 @@ struct RequireTracer : AstVisitor { std::optional name = fromAstFragment(global); if (name) - result.exprs[global] = *name; + result.exprs[global] = {*name}; return false; } @@ -72,7 +74,7 @@ struct RequireTracer : AstVisitor { const ModuleName* name = locals.find(local->local); if (name) - result.exprs[local] = *name; + result.exprs[local] = {*name}; return false; } @@ -81,16 +83,16 @@ struct RequireTracer : AstVisitor { indexName->expr->visit(this); - const ModuleName* name = result.exprs.find(indexName->expr); - if (name) + const ModuleInfo* info = result.exprs.find(indexName->expr); + if (info) { if (indexName->index == "parent" || indexName->index == "Parent") { - if (auto parent = fileResolver->getParentModuleName(*name)) - result.exprs[indexName] = *parent; + if (auto parent = fileResolver->getParentModuleName(info->name)) + result.exprs[indexName] = {*parent}; } else - result.exprs[indexName] = fileResolver->concat(*name, indexName->index.value); + result.exprs[indexName] = {fileResolver->concat(info->name, indexName->index.value)}; } return false; @@ -100,11 +102,11 @@ struct RequireTracer : AstVisitor { indexExpr->expr->visit(this); - const ModuleName* name = result.exprs.find(indexExpr->expr); + const ModuleInfo* info = result.exprs.find(indexExpr->expr); const AstExprConstantString* str = indexExpr->index->as(); - if (name && str) + if (info && str) { - result.exprs[indexExpr] = fileResolver->concat(*name, std::string_view(str->value.data, str->value.size)); + result.exprs[indexExpr] = {fileResolver->concat(info->name, std::string_view(str->value.data, str->value.size))}; } indexExpr->index->visit(this); @@ -129,8 +131,8 @@ struct RequireTracer : AstVisitor AstExprGlobal* globalName = call->func->as(); if (globalName && globalName->name == "require" && call->args.size >= 1) { - if (const ModuleName* moduleName = result.exprs.find(call->args.data[0])) - result.requires.push_back({*moduleName, call->location}); + if (const ModuleInfo* moduleInfo = result.exprs.find(call->args.data[0])) + result.requires.push_back({moduleInfo->name, call->location}); return false; } @@ -143,8 +145,8 @@ struct RequireTracer : AstVisitor if (FFlag::LuauTraceRequireLookupChild && !rootName) { - if (const ModuleName* moduleName = result.exprs.find(indexName->expr)) - rootName = *moduleName; + if (const ModuleInfo* moduleInfo = result.exprs.find(indexName->expr)) + rootName = moduleInfo->name; } if (!rootName) @@ -167,24 +169,183 @@ struct RequireTracer : AstVisitor if (v.end() != std::find(v.begin(), v.end(), '/')) return false; - result.exprs[call] = fileResolver->concat(*rootName, v); + result.exprs[call] = {fileResolver->concat(*rootName, v)}; // 'WaitForChild' can be used on modules that are not available at the typecheck time, but will be available at runtime // If we fail to find such module, we will not report an UnknownRequire error if (FFlag::LuauTraceRequireLookupChild && indexName->index == "WaitForChild") - result.optional[call] = true; + result.exprs[call].optional = true; return false; } }; +struct RequireTracer : AstVisitor +{ + RequireTracer(RequireTraceResult& result, FileResolver * fileResolver, const ModuleName& currentModuleName) + : result(result) + , fileResolver(fileResolver) + , currentModuleName(currentModuleName) + , locals(nullptr) + { + LUAU_ASSERT(FFlag::LuauNewRequireTrace); + } + + bool visit(AstExprTypeAssertion* expr) override + { + // suppress `require() :: any` + return false; + } + + bool visit(AstExprCall* expr) override + { + AstExprGlobal* global = expr->func->as(); + + if (global && global->name == "require" && expr->args.size >= 1) + requires.push_back(expr); + + return true; + } + + bool visit(AstStatLocal* stat) override + { + for (size_t i = 0; i < stat->vars.size && i < stat->values.size; ++i) + { + AstLocal* local = stat->vars.data[i]; + AstExpr* expr = stat->values.data[i]; + + // track initializing expression to be able to trace modules through locals + locals[local] = expr; + } + + return true; + } + + bool visit(AstStatAssign* stat) override + { + for (size_t i = 0; i < stat->vars.size; ++i) + { + // locals that are assigned don't have a known expression + if (AstExprLocal* expr = stat->vars.data[i]->as()) + locals[expr->local] = nullptr; + } + + return true; + } + + bool visit(AstType* node) override + { + // allow resolving require inside `typeof` annotations + return true; + } + + AstExpr* getDependent(AstExpr* node) + { + if (AstExprLocal* expr = node->as()) + return locals[expr->local]; + else if (AstExprIndexName* expr = node->as()) + return expr->expr; + else if (AstExprIndexExpr* expr = node->as()) + return expr->expr; + else if (AstExprCall* expr = node->as(); expr && expr->self) + return expr->func->as()->expr; + else + return nullptr; + } + + void process() + { + ModuleInfo moduleContext{currentModuleName}; + + // seed worklist with require arguments + work.reserve(requires.size()); + + for (AstExprCall* require: requires) + work.push_back(require->args.data[0]); + + // push all dependent expressions to the work stack; note that the vector is modified during traversal + for (size_t i = 0; i < work.size(); ++i) + if (AstExpr* dep = getDependent(work[i])) + work.push_back(dep); + + // resolve all expressions to a module info + for (size_t i = work.size(); i > 0; --i) + { + AstExpr* expr = work[i - 1]; + + // when multiple expressions depend on the same one we push it to work queue multiple times + if (result.exprs.contains(expr)) + continue; + + std::optional info; + + if (AstExpr* dep = getDependent(expr)) + { + const ModuleInfo* context = result.exprs.find(dep); + + // locals just inherit their dependent context, no resolution required + if (expr->is()) + info = context ? std::optional(*context) : std::nullopt; + else + info = fileResolver->resolveModule(context, expr); + } + else + { + info = fileResolver->resolveModule(&moduleContext, expr); + } + + if (info) + result.exprs[expr] = std::move(*info); + } + + // resolve all requires according to their argument + result.requires.reserve(requires.size()); + + for (AstExprCall* require : requires) + { + AstExpr* arg = require->args.data[0]; + + if (const ModuleInfo* info = result.exprs.find(arg)) + { + result.requires.push_back({info->name, require->location}); + + ModuleInfo infoCopy = *info; // copy *info out since next line invalidates info! + result.exprs[require] = std::move(infoCopy); + } + else + { + result.exprs[require] = {}; // mark require as unresolved + } + } + } + + RequireTraceResult& result; + FileResolver* fileResolver; + ModuleName currentModuleName; + + DenseHashMap locals; + std::vector work; + std::vector requires; +}; + } // anonymous namespace -RequireTraceResult traceRequires(FileResolver* fileResolver, AstStatBlock* root, ModuleName currentModuleName) +RequireTraceResult traceRequires(FileResolver* fileResolver, AstStatBlock* root, const ModuleName& currentModuleName) { - RequireTracer tracer{fileResolver, std::move(currentModuleName)}; - root->visit(&tracer); - return tracer.result; + if (FFlag::LuauNewRequireTrace) + { + RequireTraceResult result; + RequireTracer tracer{result, fileResolver, currentModuleName}; + root->visit(&tracer); + tracer.process(); + return result; + } + else + { + RequireTracerOld tracer{fileResolver, currentModuleName}; + root->visit(&tracer); + return tracer.result; + } } } // namespace Luau diff --git a/Analysis/src/Scope.cpp b/Analysis/src/Scope.cpp new file mode 100644 index 00000000..c30db9c2 --- /dev/null +++ b/Analysis/src/Scope.cpp @@ -0,0 +1,123 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details + +#include "Luau/Scope.h" + +namespace Luau +{ + +Scope::Scope(TypePackId returnType) + : parent(nullptr) + , returnType(returnType) + , level(TypeLevel()) +{ +} + +Scope::Scope(const ScopePtr& parent, int subLevel) + : parent(parent) + , returnType(parent->returnType) + , level(parent->level.incr()) +{ + level.subLevel = subLevel; +} + +std::optional Scope::lookup(const Symbol& name) +{ + Scope* scope = this; + + while (scope) + { + auto it = scope->bindings.find(name); + if (it != scope->bindings.end()) + return it->second.typeId; + + scope = scope->parent.get(); + } + + return std::nullopt; +} + +std::optional Scope::lookupType(const Name& name) +{ + const Scope* scope = this; + while (true) + { + auto it = scope->exportedTypeBindings.find(name); + if (it != scope->exportedTypeBindings.end()) + return it->second; + + it = scope->privateTypeBindings.find(name); + if (it != scope->privateTypeBindings.end()) + return it->second; + + if (scope->parent) + scope = scope->parent.get(); + else + return std::nullopt; + } +} + +std::optional Scope::lookupImportedType(const Name& moduleAlias, const Name& name) +{ + const Scope* scope = this; + while (scope) + { + auto it = scope->importedTypeBindings.find(moduleAlias); + if (it == scope->importedTypeBindings.end()) + { + scope = scope->parent.get(); + continue; + } + + auto it2 = it->second.find(name); + if (it2 == it->second.end()) + { + scope = scope->parent.get(); + continue; + } + + return it2->second; + } + + return std::nullopt; +} + +std::optional Scope::lookupPack(const Name& name) +{ + const Scope* scope = this; + while (true) + { + auto it = scope->privateTypePackBindings.find(name); + if (it != scope->privateTypePackBindings.end()) + return it->second; + + if (scope->parent) + scope = scope->parent.get(); + else + return std::nullopt; + } +} + +std::optional Scope::linearSearchForBinding(const std::string& name, bool traverseScopeChain) +{ + Scope* scope = this; + + while (scope) + { + for (const auto& [n, binding] : scope->bindings) + { + if (n.local && n.local->name == name.c_str()) + return binding; + else if (n.global.value && n.global == name.c_str()) + return binding; + } + + scope = scope->parent.get(); + + if (!traverseScopeChain) + break; + } + + return std::nullopt; +} + +} // namespace Luau diff --git a/Analysis/src/Substitution.cpp b/Analysis/src/Substitution.cpp index 7223998a..d861eb3d 100644 --- a/Analysis/src/Substitution.cpp +++ b/Analysis/src/Substitution.cpp @@ -6,9 +6,11 @@ #include #include -LUAU_FASTINTVARIABLE(LuauTarjanChildLimit, 0) +LUAU_FASTINTVARIABLE(LuauTarjanChildLimit, 1000) +LUAU_FASTFLAGVARIABLE(LuauSubstitutionDontReplaceIgnoredTypes, false) LUAU_FASTFLAG(LuauSecondTypecheckKnowsTheDataModel) LUAU_FASTFLAG(LuauRankNTypes) +LUAU_FASTFLAG(LuauTypeAliasPacks) namespace Luau { @@ -35,8 +37,15 @@ void Tarjan::visitChildren(TypeId ty, int index) visitChild(ttv->indexer->indexType); visitChild(ttv->indexer->indexResultType); } + for (TypeId itp : ttv->instantiatedTypeParams) visitChild(itp); + + if (FFlag::LuauTypeAliasPacks) + { + for (TypePackId itp : ttv->instantiatedTypePackParams) + visitChild(itp); + } } else if (const MetatableTypeVar* mtv = get(ty)) { @@ -332,9 +341,11 @@ std::optional Substitution::substitute(TypeId ty) return std::nullopt; for (auto [oldTy, newTy] : newTypes) - replaceChildren(newTy); + if (!FFlag::LuauSubstitutionDontReplaceIgnoredTypes || !ignoreChildren(oldTy)) + replaceChildren(newTy); for (auto [oldTp, newTp] : newPacks) - replaceChildren(newTp); + if (!FFlag::LuauSubstitutionDontReplaceIgnoredTypes || !ignoreChildren(oldTp)) + replaceChildren(newTp); TypeId newTy = replace(ty); return newTy; } @@ -350,9 +361,11 @@ std::optional Substitution::substitute(TypePackId tp) return std::nullopt; for (auto [oldTy, newTy] : newTypes) - replaceChildren(newTy); + if (!FFlag::LuauSubstitutionDontReplaceIgnoredTypes || !ignoreChildren(oldTy)) + replaceChildren(newTy); for (auto [oldTp, newTp] : newPacks) - replaceChildren(newTp); + if (!FFlag::LuauSubstitutionDontReplaceIgnoredTypes || !ignoreChildren(oldTp)) + replaceChildren(newTp); TypePackId newTp = replace(tp); return newTp; } @@ -382,6 +395,10 @@ TypeId Substitution::clone(TypeId ty) clone.name = ttv->name; clone.syntheticName = ttv->syntheticName; clone.instantiatedTypeParams = ttv->instantiatedTypeParams; + + if (FFlag::LuauTypeAliasPacks) + clone.instantiatedTypePackParams = ttv->instantiatedTypePackParams; + if (FFlag::LuauSecondTypecheckKnowsTheDataModel) clone.tags = ttv->tags; result = addType(std::move(clone)); @@ -487,8 +504,15 @@ void Substitution::replaceChildren(TypeId ty) ttv->indexer->indexType = replace(ttv->indexer->indexType); ttv->indexer->indexResultType = replace(ttv->indexer->indexResultType); } + for (TypeId& itp : ttv->instantiatedTypeParams) itp = replace(itp); + + if (FFlag::LuauTypeAliasPacks) + { + for (TypePackId& itp : ttv->instantiatedTypePackParams) + itp = replace(itp); + } } else if (MetatableTypeVar* mtv = getMutable(ty)) { diff --git a/Analysis/src/ToString.cpp b/Analysis/src/ToString.cpp index 9d2f47ba..5651af7e 100644 --- a/Analysis/src/ToString.cpp +++ b/Analysis/src/ToString.cpp @@ -1,6 +1,7 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #include "Luau/ToString.h" +#include "Luau/Scope.h" #include "Luau/TypeInfer.h" #include "Luau/TypePack.h" #include "Luau/TypeVar.h" @@ -9,10 +10,10 @@ #include #include -LUAU_FASTFLAG(LuauToStringFollowsBoundTo) LUAU_FASTFLAG(LuauExtraNilRecovery) LUAU_FASTFLAG(LuauOccursCheckOkWithRecursiveFunctions) LUAU_FASTFLAGVARIABLE(LuauInstantiatedTypeParamRecursion, false) +LUAU_FASTFLAG(LuauTypeAliasPacks) namespace Luau { @@ -59,6 +60,13 @@ struct FindCyclicTypes { for (TypeId itp : ttv.instantiatedTypeParams) visitTypeVar(itp, *this, seen); + + if (FFlag::LuauTypeAliasPacks) + { + for (TypePackId itp : ttv.instantiatedTypePackParams) + visitTypeVar(itp, *this, seen); + } + return exhaustive; } @@ -258,23 +266,60 @@ struct TypeVarStringifier void stringify(TypePackId tp); void stringify(TypePackId tpid, const std::vector>& names); - void stringify(const std::vector& types) + void stringify(const std::vector& types, const std::vector& typePacks) { - if (types.size() == 0) + if (types.size() == 0 && (!FFlag::LuauTypeAliasPacks || typePacks.size() == 0)) return; - if (types.size()) + if (types.size() || (FFlag::LuauTypeAliasPacks && typePacks.size())) state.emit("<"); - for (size_t i = 0; i < types.size(); ++i) + if (FFlag::LuauTypeAliasPacks) { - if (i > 0) - state.emit(", "); + bool first = true; - stringify(types[i]); + for (TypeId ty : types) + { + if (!first) + state.emit(", "); + first = false; + + stringify(ty); + } + + bool singleTp = typePacks.size() == 1; + + for (TypePackId tp : typePacks) + { + if (isEmpty(tp) && singleTp) + continue; + + if (!first) + state.emit(", "); + else + first = false; + + if (!singleTp) + state.emit("("); + + stringify(tp); + + if (!singleTp) + state.emit(")"); + } + } + else + { + for (size_t i = 0; i < types.size(); ++i) + { + if (i > 0) + state.emit(", "); + + stringify(types[i]); + } } - if (types.size()) + if (types.size() || (FFlag::LuauTypeAliasPacks && typePacks.size())) state.emit(">"); } @@ -388,7 +433,7 @@ struct TypeVarStringifier void operator()(TypeId, const TableTypeVar& ttv) { - if (FFlag::LuauToStringFollowsBoundTo && ttv.boundTo) + if (ttv.boundTo) return stringify(*ttv.boundTo); if (!state.exhaustive) @@ -411,14 +456,14 @@ struct TypeVarStringifier } state.emit(*ttv.name); - stringify(ttv.instantiatedTypeParams); + stringify(ttv.instantiatedTypeParams, ttv.instantiatedTypePackParams); return; } if (ttv.syntheticName) { state.result.invalid = true; state.emit(*ttv.syntheticName); - stringify(ttv.instantiatedTypeParams); + stringify(ttv.instantiatedTypeParams, ttv.instantiatedTypePackParams); return; } } @@ -900,13 +945,26 @@ ToStringResult toStringDetailed(TypeId ty, const ToStringOptions& opts) result.name += ttv->name ? *ttv->name : *ttv->syntheticName; - if (ttv->instantiatedTypeParams.empty()) + if (ttv->instantiatedTypeParams.empty() && (!FFlag::LuauTypeAliasPacks || ttv->instantiatedTypePackParams.empty())) return result; std::vector params; for (TypeId tp : ttv->instantiatedTypeParams) params.push_back(toString(tp)); + if (FFlag::LuauTypeAliasPacks) + { + // Doesn't preserve grouping of multiple type packs + // But this is under a parent block of code that is being removed later + for (TypePackId tp : ttv->instantiatedTypePackParams) + { + std::string content = toString(tp); + + if (!content.empty()) + params.push_back(std::move(content)); + } + } + result.name += "<" + join(params, ", ") + ">"; return result; } @@ -950,30 +1008,37 @@ ToStringResult toStringDetailed(TypeId ty, const ToStringOptions& opts) result.name += ttv->name ? *ttv->name : *ttv->syntheticName; - if (ttv->instantiatedTypeParams.empty()) - return result; - - result.name += "<"; - - bool first = true; - for (TypeId ty : ttv->instantiatedTypeParams) + if (FFlag::LuauTypeAliasPacks) { - if (!first) - result.name += ", "; - else - first = false; - - tvs.stringify(ty); - } - - if (opts.maxTypeLength > 0 && result.name.length() > opts.maxTypeLength) - { - result.truncated = true; - result.name += "... "; + tvs.stringify(ttv->instantiatedTypeParams, ttv->instantiatedTypePackParams); } else { - result.name += ">"; + if (ttv->instantiatedTypeParams.empty() && (!FFlag::LuauTypeAliasPacks || ttv->instantiatedTypePackParams.empty())) + return result; + + result.name += "<"; + + bool first = true; + for (TypeId ty : ttv->instantiatedTypeParams) + { + if (!first) + result.name += ", "; + else + first = false; + + tvs.stringify(ty); + } + + if (opts.maxTypeLength > 0 && result.name.length() > opts.maxTypeLength) + { + result.truncated = true; + result.name += "... "; + } + else + { + result.name += ">"; + } } return result; diff --git a/Analysis/src/Transpiler.cpp b/Analysis/src/Transpiler.cpp index 462c70ff..1b83ccdc 100644 --- a/Analysis/src/Transpiler.cpp +++ b/Analysis/src/Transpiler.cpp @@ -11,6 +11,7 @@ #include LUAU_FASTFLAG(LuauGenericFunctions) +LUAU_FASTFLAG(LuauTypeAliasPacks) namespace { @@ -280,10 +281,19 @@ struct Printer void visualizeTypePackAnnotation(const AstTypePack& annotation) { - if (const AstTypePackVariadic* variadic = annotation.as()) + if (const AstTypePackVariadic* variadicTp = annotation.as()) { writer.symbol("..."); - visualizeTypeAnnotation(*variadic->variadicType); + visualizeTypeAnnotation(*variadicTp->variadicType); + } + else if (const AstTypePackGeneric* genericTp = annotation.as()) + { + writer.symbol(genericTp->genericName.value); + writer.symbol("..."); + } + else if (const AstTypePackExplicit* explicitTp = annotation.as()) + { + visualizeTypeList(explicitTp->typeList, true); } else { @@ -807,7 +817,7 @@ struct Printer writer.keyword("type"); writer.identifier(a->name.value); - if (a->generics.size > 0) + if (a->generics.size > 0 || (FFlag::LuauTypeAliasPacks && a->genericPacks.size > 0)) { writer.symbol("<"); CommaSeparatorInserter comma(writer); @@ -817,6 +827,17 @@ struct Printer comma(); writer.identifier(o.value); } + + if (FFlag::LuauTypeAliasPacks) + { + for (auto o : a->genericPacks) + { + comma(); + writer.identifier(o.value); + writer.symbol("..."); + } + } + writer.symbol(">"); } writer.maybeSpace(a->type->location.begin, 2); @@ -960,15 +981,20 @@ struct Printer if (const auto& a = typeAnnotation.as()) { writer.write(a->name.value); - if (a->generics.size > 0) + if (a->parameters.size > 0) { CommaSeparatorInserter comma(writer); writer.symbol("<"); - for (auto o : a->generics) + for (auto o : a->parameters) { comma(); - visualizeTypeAnnotation(*o); + + if (o.type) + visualizeTypeAnnotation(*o.type); + else + visualizeTypePackAnnotation(*o.typePack); } + writer.symbol(">"); } } diff --git a/Analysis/src/TypeAttach.cpp b/Analysis/src/TypeAttach.cpp index 17c57c84..266c1986 100644 --- a/Analysis/src/TypeAttach.cpp +++ b/Analysis/src/TypeAttach.cpp @@ -5,6 +5,7 @@ #include "Luau/Module.h" #include "Luau/Parser.h" #include "Luau/RecursionCounter.h" +#include "Luau/Scope.h" #include "Luau/TypeInfer.h" #include "Luau/TypePack.h" #include "Luau/TypeVar.h" @@ -12,6 +13,7 @@ #include LUAU_FASTFLAG(LuauGenericFunctions) +LUAU_FASTFLAG(LuauTypeAliasPacks) static char* allocateString(Luau::Allocator& allocator, std::string_view contents) { @@ -33,7 +35,6 @@ static char* allocateString(Luau::Allocator& allocator, const char* format, Data namespace Luau { - class TypeRehydrationVisitor { mutable std::map seen; @@ -57,6 +58,8 @@ public: { } + AstTypePack* rehydrate(TypePackId tp) const; + AstType* operator()(const PrimitiveTypeVar& ptv) const { switch (ptv.type) @@ -85,16 +88,24 @@ public: if (ttv.name && options.bannedNames.find(*ttv.name) == options.bannedNames.end()) { - AstArray generics; - generics.size = ttv.instantiatedTypeParams.size(); - generics.data = static_cast(allocator->allocate(sizeof(AstType*) * generics.size)); + AstArray parameters; + parameters.size = ttv.instantiatedTypeParams.size(); + parameters.data = static_cast(allocator->allocate(sizeof(AstTypeOrPack) * parameters.size)); for (size_t i = 0; i < ttv.instantiatedTypeParams.size(); ++i) { - generics.data[i] = Luau::visit(*this, ttv.instantiatedTypeParams[i]->ty); + parameters.data[i] = {Luau::visit(*this, ttv.instantiatedTypeParams[i]->ty), {}}; } - return allocator->alloc(Location(), std::nullopt, AstName(ttv.name->c_str()), generics); + if (FFlag::LuauTypeAliasPacks) + { + for (size_t i = 0; i < ttv.instantiatedTypePackParams.size(); ++i) + { + parameters.data[i] = {{}, rehydrate(ttv.instantiatedTypePackParams[i])}; + } + } + + return allocator->alloc(Location(), std::nullopt, AstName(ttv.name->c_str()), parameters.size != 0, parameters); } if (hasSeen(&ttv)) @@ -222,10 +233,17 @@ public: AstTypePack* argTailAnnotation = nullptr; if (argTail) { - TypePackId tail = *argTail; - if (const VariadicTypePack* vtp = get(tail)) + if (FFlag::LuauTypeAliasPacks) { - argTailAnnotation = allocator->alloc(Location(), Luau::visit(*this, vtp->ty->ty)); + argTailAnnotation = rehydrate(*argTail); + } + else + { + TypePackId tail = *argTail; + if (const VariadicTypePack* vtp = get(tail)) + { + argTailAnnotation = allocator->alloc(Location(), Luau::visit(*this, vtp->ty->ty)); + } } } @@ -255,10 +273,17 @@ public: AstTypePack* retTailAnnotation = nullptr; if (retTail) { - TypePackId tail = *retTail; - if (const VariadicTypePack* vtp = get(tail)) + if (FFlag::LuauTypeAliasPacks) { - retTailAnnotation = allocator->alloc(Location(), Luau::visit(*this, vtp->ty->ty)); + retTailAnnotation = rehydrate(*retTail); + } + else + { + TypePackId tail = *retTail; + if (const VariadicTypePack* vtp = get(tail)) + { + retTailAnnotation = allocator->alloc(Location(), Luau::visit(*this, vtp->ty->ty)); + } } } @@ -313,6 +338,68 @@ private: const TypeRehydrationOptions& options; }; +class TypePackRehydrationVisitor +{ +public: + TypePackRehydrationVisitor(Allocator* allocator, const TypeRehydrationVisitor& typeVisitor) + : allocator(allocator) + , typeVisitor(typeVisitor) + { + } + + AstTypePack* operator()(const BoundTypePack& btp) const + { + return Luau::visit(*this, btp.boundTo->ty); + } + + AstTypePack* operator()(const TypePack& tp) const + { + AstArray head; + head.size = tp.head.size(); + head.data = static_cast(allocator->allocate(sizeof(AstType*) * tp.head.size())); + + for (size_t i = 0; i < tp.head.size(); i++) + head.data[i] = Luau::visit(typeVisitor, tp.head[i]->ty); + + AstTypePack* tail = nullptr; + + if (tp.tail) + tail = Luau::visit(*this, (*tp.tail)->ty); + + return allocator->alloc(Location(), AstTypeList{head, tail}); + } + + AstTypePack* operator()(const VariadicTypePack& vtp) const + { + return allocator->alloc(Location(), Luau::visit(typeVisitor, vtp.ty->ty)); + } + + AstTypePack* operator()(const GenericTypePack& gtp) const + { + return allocator->alloc(Location(), AstName(gtp.name.c_str())); + } + + AstTypePack* operator()(const FreeTypePack& gtp) const + { + return allocator->alloc(Location(), AstName("free")); + } + + AstTypePack* operator()(const Unifiable::Error&) const + { + return allocator->alloc(Location(), AstName("Unifiable")); + } + +private: + Allocator* allocator; + const TypeRehydrationVisitor& typeVisitor; +}; + +AstTypePack* TypeRehydrationVisitor::rehydrate(TypePackId tp) const +{ + TypePackRehydrationVisitor tprv(allocator, *this); + return Luau::visit(tprv, tp->ty); +} + class TypeAttacher : public AstVisitor { public: @@ -406,9 +493,16 @@ public: if (tail) { - TypePackId tailPack = *tail; - if (const VariadicTypePack* vtp = get(tailPack)) - variadicAnnotation = allocator->alloc(Location(), typeAst(vtp->ty)); + if (FFlag::LuauTypeAliasPacks) + { + variadicAnnotation = TypeRehydrationVisitor(allocator).rehydrate(*tail); + } + else + { + TypePackId tailPack = *tail; + if (const VariadicTypePack* vtp = get(tailPack)) + variadicAnnotation = allocator->alloc(Location(), typeAst(vtp->ty)); + } } fn->returnAnnotation = AstTypeList{typeAstPack(ret), variadicAnnotation}; diff --git a/Analysis/src/TypeInfer.cpp b/Analysis/src/TypeInfer.cpp index 5c5217e7..da93c8ec 100644 --- a/Analysis/src/TypeInfer.cpp +++ b/Analysis/src/TypeInfer.cpp @@ -5,21 +5,22 @@ #include "Luau/ModuleResolver.h" #include "Luau/Parser.h" #include "Luau/RecursionCounter.h" +#include "Luau/Scope.h" #include "Luau/Substitution.h" #include "Luau/TopoSortStatements.h" #include "Luau/ToString.h" #include "Luau/TypePack.h" #include "Luau/TypeUtils.h" #include "Luau/TypeVar.h" +#include "Luau/TimeTrace.h" #include #include LUAU_FASTFLAGVARIABLE(DebugLuauMagicTypes, false) -LUAU_FASTINTVARIABLE(LuauTypeInferRecursionLimit, 0) -LUAU_FASTINTVARIABLE(LuauTypeInferTypePackLoopLimit, 0) +LUAU_FASTINTVARIABLE(LuauTypeInferRecursionLimit, 500) +LUAU_FASTINTVARIABLE(LuauTypeInferTypePackLoopLimit, 5000) LUAU_FASTINTVARIABLE(LuauCheckRecursionLimit, 500) -LUAU_FASTFLAGVARIABLE(LuauIndexTablesWithIndexers, false) LUAU_FASTFLAGVARIABLE(LuauGenericFunctions, false) LUAU_FASTFLAGVARIABLE(LuauGenericVariadicsUnification, false) LUAU_FASTFLAG(LuauKnowsTheDataModel3) @@ -27,14 +28,11 @@ LUAU_FASTFLAG(LuauSecondTypecheckKnowsTheDataModel) LUAU_FASTFLAGVARIABLE(LuauClassPropertyAccessAsString, false) LUAU_FASTFLAGVARIABLE(LuauEqConstraint, false) LUAU_FASTFLAGVARIABLE(LuauWeakEqConstraint, false) // Eventually removed as false. -LUAU_FASTFLAGVARIABLE(LuauImprovedTypeGuardPredicate2, false) LUAU_FASTFLAG(LuauTraceRequireLookupChild) -LUAU_FASTFLAG(DebugLuauTrackOwningArena) LUAU_FASTFLAGVARIABLE(LuauCloneCorrectlyBeforeMutatingTableType, false) LUAU_FASTFLAGVARIABLE(LuauStoreMatchingOverloadFnType, false) LUAU_FASTFLAGVARIABLE(LuauRankNTypes, false) LUAU_FASTFLAGVARIABLE(LuauOrPredicate, false) -LUAU_FASTFLAGVARIABLE(LuauFixTableTypeAliasClone, false) LUAU_FASTFLAGVARIABLE(LuauExtraNilRecovery, false) LUAU_FASTFLAGVARIABLE(LuauMissingUnionPropertyError, false) LUAU_FASTFLAGVARIABLE(LuauInferReturnAssertAssign, false) @@ -45,6 +43,10 @@ LUAU_FASTFLAGVARIABLE(LuauSlightlyMoreFlexibleBinaryPredicates, false) LUAU_FASTFLAGVARIABLE(LuauInferFunctionArgsFix, false) LUAU_FASTFLAGVARIABLE(LuauFollowInTypeFunApply, false) LUAU_FASTFLAGVARIABLE(LuauIfElseExpressionAnalysisSupport, false) +LUAU_FASTFLAGVARIABLE(LuauStrictRequire, false) +LUAU_FASTFLAG(LuauSubstitutionDontReplaceIgnoredTypes) +LUAU_FASTFLAG(LuauNewRequireTrace) +LUAU_FASTFLAG(LuauTypeAliasPacks) namespace Luau { @@ -216,9 +218,8 @@ TypeChecker::TypeChecker(ModuleResolver* resolver, InternalErrorReporter* iceHan , nilType(singletonTypes.nilType) , numberType(singletonTypes.numberType) , stringType(singletonTypes.stringType) - , booleanType( - FFlag::LuauImprovedTypeGuardPredicate2 ? singletonTypes.booleanType : globalTypes.addType(PrimitiveTypeVar(PrimitiveTypeVar::Boolean))) - , threadType(FFlag::LuauImprovedTypeGuardPredicate2 ? singletonTypes.threadType : globalTypes.addType(PrimitiveTypeVar(PrimitiveTypeVar::Thread))) + , booleanType(singletonTypes.booleanType) + , threadType(singletonTypes.threadType) , anyType(singletonTypes.anyType) , errorType(singletonTypes.errorType) , optionalNumberType(globalTypes.addType(UnionTypeVar{{numberType, nilType}})) @@ -237,6 +238,9 @@ TypeChecker::TypeChecker(ModuleResolver* resolver, InternalErrorReporter* iceHan ModulePtr TypeChecker::check(const SourceModule& module, Mode mode, std::optional environmentScope) { + LUAU_TIMETRACE_SCOPE("TypeChecker::check", "TypeChecker"); + LUAU_TIMETRACE_ARGUMENT("module", module.name.c_str()); + currentModule.reset(new Module()); currentModule->type = module.type; @@ -1177,44 +1181,61 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatTypeAlias& typealias { Location location = scope->typeAliasLocations[name]; reportError(TypeError{typealias.location, DuplicateTypeDefinition{name, location}}); - bindingsMap[name] = TypeFun{binding->typeParams, errorType}; + + if (FFlag::LuauTypeAliasPacks) + bindingsMap[name] = TypeFun{binding->typeParams, binding->typePackParams, errorType}; + else + bindingsMap[name] = TypeFun{binding->typeParams, errorType}; } else { ScopePtr aliasScope = childScope(scope, typealias.location); - std::vector generics; - for (AstName generic : typealias.generics) + if (FFlag::LuauTypeAliasPacks) { - Name n = generic.value; + auto [generics, genericPacks] = createGenericTypes(aliasScope, typealias, typealias.generics, typealias.genericPacks); - // These generics are the only thing that will ever be added to aliasScope, so we can be certain that - // a collision can only occur when two generic typevars have the same name. - if (aliasScope->privateTypeBindings.end() != aliasScope->privateTypeBindings.find(n)) - { - // TODO(jhuelsman): report the exact span of the generic type parameter whose name is a duplicate. - reportError(TypeError{typealias.location, DuplicateGenericParameter{n}}); - } - - TypeId g; - if (FFlag::LuauRecursiveTypeParameterRestriction) - { - TypeId& cached = scope->typeAliasParameters[n]; - if (!cached) - cached = addType(GenericTypeVar{aliasScope->level, n}); - g = cached; - } - else - g = addType(GenericTypeVar{aliasScope->level, n}); - generics.push_back(g); - aliasScope->privateTypeBindings[n] = TypeFun{{}, g}; + TypeId ty = (FFlag::LuauRankNTypes ? freshType(aliasScope) : DEPRECATED_freshType(scope, true)); + FreeTypeVar* ftv = getMutable(ty); + LUAU_ASSERT(ftv); + ftv->forwardedTypeAlias = true; + bindingsMap[name] = {std::move(generics), std::move(genericPacks), ty}; } + else + { + std::vector generics; + for (AstName generic : typealias.generics) + { + Name n = generic.value; - TypeId ty = (FFlag::LuauRankNTypes ? freshType(aliasScope) : DEPRECATED_freshType(scope, true)); - FreeTypeVar* ftv = getMutable(ty); - LUAU_ASSERT(ftv); - ftv->forwardedTypeAlias = true; - bindingsMap[name] = {std::move(generics), ty}; + // These generics are the only thing that will ever be added to aliasScope, so we can be certain that + // a collision can only occur when two generic typevars have the same name. + if (aliasScope->privateTypeBindings.end() != aliasScope->privateTypeBindings.find(n)) + { + // TODO(jhuelsman): report the exact span of the generic type parameter whose name is a duplicate. + reportError(TypeError{typealias.location, DuplicateGenericParameter{n}}); + } + + TypeId g; + if (FFlag::LuauRecursiveTypeParameterRestriction) + { + TypeId& cached = scope->typeAliasTypeParameters[n]; + if (!cached) + cached = addType(GenericTypeVar{aliasScope->level, n}); + g = cached; + } + else + g = addType(GenericTypeVar{aliasScope->level, n}); + generics.push_back(g); + aliasScope->privateTypeBindings[n] = TypeFun{{}, g}; + } + + TypeId ty = (FFlag::LuauRankNTypes ? freshType(aliasScope) : DEPRECATED_freshType(scope, true)); + FreeTypeVar* ftv = getMutable(ty); + LUAU_ASSERT(ftv); + ftv->forwardedTypeAlias = true; + bindingsMap[name] = {std::move(generics), ty}; + } } } else @@ -1231,6 +1252,16 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatTypeAlias& typealias aliasScope->privateTypeBindings[generic->name] = TypeFun{{}, ty}; } + if (FFlag::LuauTypeAliasPacks) + { + for (TypePackId tp : binding->typePackParams) + { + auto generic = get(tp); + LUAU_ASSERT(generic); + aliasScope->privateTypePackBindings[generic->name] = tp; + } + } + TypeId ty = (FFlag::LuauRankNTypes ? resolveType(aliasScope, *typealias.type) : resolveType(aliasScope, *typealias.type, true)); if (auto ttv = getMutable(follow(ty))) { @@ -1238,7 +1269,8 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatTypeAlias& typealias if (ttv->name) { // Copy can be skipped if this is an identical alias - if (!FFlag::LuauFixTableTypeAliasClone || ttv->name != name || ttv->instantiatedTypeParams != binding->typeParams) + if (ttv->name != name || ttv->instantiatedTypeParams != binding->typeParams || + (FFlag::LuauTypeAliasPacks && ttv->instantiatedTypePackParams != binding->typePackParams)) { // This is a shallow clone, original recursive links to self are not updated TableTypeVar clone = TableTypeVar{ttv->props, ttv->indexer, ttv->level, ttv->state}; @@ -1249,6 +1281,9 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatTypeAlias& typealias clone.name = name; clone.instantiatedTypeParams = binding->typeParams; + if (FFlag::LuauTypeAliasPacks) + clone.instantiatedTypePackParams = binding->typePackParams; + ty = addType(std::move(clone)); } } @@ -1256,6 +1291,9 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatTypeAlias& typealias { ttv->name = name; ttv->instantiatedTypeParams = binding->typeParams; + + if (FFlag::LuauTypeAliasPacks) + ttv->instantiatedTypePackParams = binding->typePackParams; } } else if (auto mtv = getMutable(follow(ty))) @@ -1280,7 +1318,7 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatDeclareClass& declar } // We don't have generic classes, so this assertion _should_ never be hit. - LUAU_ASSERT(lookupType->typeParams.size() == 0); + LUAU_ASSERT(lookupType->typeParams.size() == 0 && (!FFlag::LuauTypeAliasPacks || lookupType->typePackParams.size() == 0)); superTy = lookupType->type; if (FFlag::LuauAddMissingFollow) @@ -1465,7 +1503,8 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExpr& if (FFlag::LuauStoreMatchingOverloadFnType) { - currentModule->astTypes.try_emplace(&expr, result.type); + if (!currentModule->astTypes.find(&expr)) + currentModule->astTypes[&expr] = result.type; } else { @@ -2193,7 +2232,7 @@ TypeId TypeChecker::checkRelationalOperation( * have a better, more descriptive error teed up. */ Unifier state = mkUnifier(expr.location); - if (!FFlag::LuauEqConstraint || !isEquality) + if (!isEquality) state.tryUnify(lhsType, rhsType); bool needsMetamethod = !isEquality; @@ -2262,7 +2301,7 @@ TypeId TypeChecker::checkRelationalOperation( } } - if (get(FFlag::LuauAddMissingFollow ? follow(lhsType) : lhsType) && (!FFlag::LuauEqConstraint || !isEquality)) + if (get(FFlag::LuauAddMissingFollow ? follow(lhsType) : lhsType) && !isEquality) { auto name = getIdentifierOfBaseVar(expr.left); reportError(expr.location, CannotInferBinaryOperation{expr.op, name, CannotInferBinaryOperation::Comparison}); @@ -2276,18 +2315,6 @@ TypeId TypeChecker::checkRelationalOperation( return errorType; } - if (!FFlag::LuauEqConstraint) - { - if (isEquality) - { - ErrorVec errVec = tryUnify(rhsType, lhsType, expr.location); - if (!state.errors.empty() && !errVec.empty()) - reportError(expr.location, TypeMismatch{lhsType, rhsType}); - } - else - reportErrors(state.errors); - } - return booleanType; } @@ -2443,7 +2470,7 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprBi TypeId result = checkBinaryOperation(innerScope, expr, lhs.type, rhs.type, lhs.predicates); return {result, {OrPredicate{std::move(lhs.predicates), std::move(rhs.predicates)}}}; } - else if (FFlag::LuauEqConstraint && (expr.op == AstExprBinary::CompareEq || expr.op == AstExprBinary::CompareNe)) + else if (expr.op == AstExprBinary::CompareEq || expr.op == AstExprBinary::CompareNe) { if (auto predicate = tryGetTypeGuardPredicate(expr)) return {booleanType, {std::move(*predicate)}}; @@ -2466,14 +2493,6 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprBi } else { - // Once we have EqPredicate, we should break this else branch into its' own branch. - // For now, fall through is intentional. - if (expr.op == AstExprBinary::CompareEq || expr.op == AstExprBinary::CompareNe) - { - if (auto predicate = tryGetTypeGuardPredicate(expr)) - return {booleanType, {std::move(*predicate)}}; - } - ExprResult lhs = checkExpr(scope, *expr.left); ExprResult rhs = checkExpr(scope, *expr.right); @@ -2755,12 +2774,6 @@ std::pair TypeChecker::checkLValueBinding(const ScopePtr& scope exprTable->indexer = TableIndexer{anyIfNonstrict(indexType), anyIfNonstrict(resultType)}; return std::pair(resultType, nullptr); } - else if (FFlag::LuauIndexTablesWithIndexers) - { - // We allow t[x] where x:string for tables without an indexer - unify(indexType, stringType, expr.location); - return std::pair(anyType, nullptr); - } else { TypeId resultType = freshType(scope); @@ -3076,6 +3089,13 @@ static Location getEndLocation(const AstExprFunction& function) void TypeChecker::checkFunctionBody(const ScopePtr& scope, TypeId ty, const AstExprFunction& function) { + LUAU_TIMETRACE_SCOPE("TypeChecker::checkFunctionBody", "TypeChecker"); + + if (function.debugname.value) + LUAU_TIMETRACE_ARGUMENT("name", function.debugname.value); + else + LUAU_TIMETRACE_ARGUMENT("line", std::to_string(function.location.begin.line).c_str()); + if (FunctionTypeVar* funTy = getMutable(ty)) { check(scope, *function.body); @@ -3885,6 +3905,20 @@ std::optional TypeChecker::matchRequire(const AstExprCall& call) TypeId TypeChecker::checkRequire(const ScopePtr& scope, const ModuleInfo& moduleInfo, const Location& location) { + LUAU_TIMETRACE_SCOPE("TypeChecker::checkRequire", "TypeChecker"); + LUAU_TIMETRACE_ARGUMENT("moduleInfo", moduleInfo.name.c_str()); + + if (FFlag::LuauNewRequireTrace && moduleInfo.name.empty()) + { + if (FFlag::LuauStrictRequire && currentModule->mode == Mode::Strict) + { + reportError(TypeError{location, UnknownRequire{}}); + return errorType; + } + + return anyType; + } + ModulePtr module = resolver->getModule(moduleInfo.name); if (!module) { @@ -4472,7 +4506,7 @@ TypeId TypeChecker::freshType(const ScopePtr& scope) TypeId TypeChecker::freshType(TypeLevel level) { - return currentModule->internalTypes.typeVars.allocate(TypeVar(FreeTypeVar(level))); + return currentModule->internalTypes.addType(TypeVar(FreeTypeVar(level))); } TypeId TypeChecker::DEPRECATED_freshType(const ScopePtr& scope, bool canBeGeneric) @@ -4482,11 +4516,7 @@ TypeId TypeChecker::DEPRECATED_freshType(const ScopePtr& scope, bool canBeGeneri TypeId TypeChecker::DEPRECATED_freshType(TypeLevel level, bool canBeGeneric) { - TypeId allocated = currentModule->internalTypes.typeVars.allocate(TypeVar(FreeTypeVar(level, canBeGeneric))); - if (FFlag::DebugLuauTrackOwningArena) - asMutable(allocated)->owningArena = ¤tModule->internalTypes; - - return allocated; + return currentModule->internalTypes.addType(TypeVar(FreeTypeVar(level, canBeGeneric))); } std::optional TypeChecker::filterMap(TypeId type, TypeIdPredicate predicate) @@ -4506,20 +4536,12 @@ TypeId TypeChecker::addType(const UnionTypeVar& utv) TypeId TypeChecker::addTV(TypeVar&& tv) { - TypeId allocated = currentModule->internalTypes.typeVars.allocate(std::move(tv)); - if (FFlag::DebugLuauTrackOwningArena) - asMutable(allocated)->owningArena = ¤tModule->internalTypes; - - return allocated; + return currentModule->internalTypes.addType(std::move(tv)); } TypePackId TypeChecker::addTypePack(TypePackVar&& tv) { - TypePackId allocated = currentModule->internalTypes.typePacks.allocate(std::move(tv)); - if (FFlag::DebugLuauTrackOwningArena) - asMutable(allocated)->owningArena = ¤tModule->internalTypes; - - return allocated; + return currentModule->internalTypes.addTypePack(std::move(tv)); } TypePackId TypeChecker::addTypePack(TypePack&& tp) @@ -4578,7 +4600,7 @@ TypeId TypeChecker::resolveType(const ScopePtr& scope, const AstType& annotation else if (FFlag::DebugLuauMagicTypes && lit->name == "_luau_print") { - if (lit->generics.size != 1) + if (lit->parameters.size != 1 || !lit->parameters.data[0].type) { reportError(TypeError{annotation.location, GenericError{"_luau_print requires one generic parameter"}}); return addType(ErrorTypeVar{}); @@ -4588,7 +4610,7 @@ TypeId TypeChecker::resolveType(const ScopePtr& scope, const AstType& annotation opts.exhaustive = true; opts.maxTableLength = 0; - TypeId param = resolveType(scope, *lit->generics.data[0]); + TypeId param = resolveType(scope, *lit->parameters.data[0].type); luauPrintLine(format("_luau_print\t%s\t|\t%s", toString(param, opts).c_str(), toString(lit->location).c_str())); return param; } @@ -4614,18 +4636,86 @@ TypeId TypeChecker::resolveType(const ScopePtr& scope, const AstType& annotation return addType(ErrorTypeVar{}); } - if (lit->generics.size == 0 && tf->typeParams.empty()) - return tf->type; - else if (lit->generics.size != tf->typeParams.size()) + if (lit->parameters.size == 0 && tf->typeParams.empty() && (!FFlag::LuauTypeAliasPacks || tf->typePackParams.empty())) { - reportError(TypeError{annotation.location, IncorrectGenericParameterCount{lit->name.value, *tf, lit->generics.size}}); + return tf->type; + } + else if (!FFlag::LuauTypeAliasPacks && lit->parameters.size != tf->typeParams.size()) + { + reportError(TypeError{annotation.location, IncorrectGenericParameterCount{lit->name.value, *tf, lit->parameters.size, 0}}); return addType(ErrorTypeVar{}); } + else if (FFlag::LuauTypeAliasPacks) + { + if (!lit->hasParameterList && !tf->typePackParams.empty()) + { + reportError(TypeError{annotation.location, GenericError{"Type parameter list is required"}}); + return addType(ErrorTypeVar{}); + } + + std::vector typeParams; + std::vector extraTypes; + std::vector typePackParams; + + for (size_t i = 0; i < lit->parameters.size; ++i) + { + if (AstType* type = lit->parameters.data[i].type) + { + TypeId ty = resolveType(scope, *type); + + if (typeParams.size() < tf->typeParams.size() || tf->typePackParams.empty()) + typeParams.push_back(ty); + else if (typePackParams.empty()) + extraTypes.push_back(ty); + else + reportError(TypeError{annotation.location, GenericError{"Type parameters must come before type pack parameters"}}); + } + else if (AstTypePack* typePack = lit->parameters.data[i].typePack) + { + TypePackId tp = resolveTypePack(scope, *typePack); + + // If we have collected an implicit type pack, materialize it + if (typePackParams.empty() && !extraTypes.empty()) + typePackParams.push_back(addTypePack(extraTypes)); + + // If we need more regular types, we can use single element type packs to fill those in + if (typeParams.size() < tf->typeParams.size() && size(tp) == 1 && finite(tp) && first(tp)) + typeParams.push_back(*first(tp)); + else + typePackParams.push_back(tp); + } + } + + // If we still haven't meterialized an implicit type pack, do it now + if (typePackParams.empty() && !extraTypes.empty()) + typePackParams.push_back(addTypePack(extraTypes)); + + // If we didn't combine regular types into a type pack and we're still one type pack short, provide an empty type pack + if (extraTypes.empty() && typePackParams.size() + 1 == tf->typePackParams.size()) + typePackParams.push_back(addTypePack({})); + + if (typeParams.size() != tf->typeParams.size() || typePackParams.size() != tf->typePackParams.size()) + { + reportError( + TypeError{annotation.location, IncorrectGenericParameterCount{lit->name.value, *tf, typeParams.size(), typePackParams.size()}}); + return addType(ErrorTypeVar{}); + } + + if (FFlag::LuauRecursiveTypeParameterRestriction && typeParams == tf->typeParams && typePackParams == tf->typePackParams) + { + // If the generic parameters and the type arguments are the same, we are about to + // perform an identity substitution, which we can just short-circuit. + return tf->type; + } + + return instantiateTypeFun(scope, *tf, typeParams, typePackParams, annotation.location); + } else { std::vector typeParams; - for (AstType* paramAnnot : lit->generics) - typeParams.push_back(resolveType(scope, *paramAnnot)); + + for (const auto& param : lit->parameters) + typeParams.push_back(resolveType(scope, *param.type)); if (FFlag::LuauRecursiveTypeParameterRestriction && typeParams == tf->typeParams) { @@ -4634,7 +4724,7 @@ TypeId TypeChecker::resolveType(const ScopePtr& scope, const AstType& annotation return tf->type; } - return instantiateTypeFun(scope, *tf, typeParams, annotation.location); + return instantiateTypeFun(scope, *tf, typeParams, {}, annotation.location); } } else if (const auto& table = annotation.as()) @@ -4765,6 +4855,18 @@ TypePackId TypeChecker::resolveTypePack(const ScopePtr& scope, const AstTypePack return *genericTy; } + else if (const AstTypePackExplicit* explicitTp = annotation.as()) + { + std::vector types; + + for (auto type : explicitTp->typeList.types) + types.push_back(resolveType(scope, *type)); + + if (auto tailType = explicitTp->typeList.tailType) + return addTypePack(types, resolveTypePack(scope, *tailType)); + + return addTypePack(types); + } else { ice("Unknown AstTypePack kind"); @@ -4799,12 +4901,28 @@ bool ApplyTypeFunction::isDirty(TypePackId tp) return false; } +bool ApplyTypeFunction::ignoreChildren(TypeId ty) +{ + if (FFlag::LuauSubstitutionDontReplaceIgnoredTypes && get(ty)) + return true; + else + return false; +} + +bool ApplyTypeFunction::ignoreChildren(TypePackId tp) +{ + if (FFlag::LuauSubstitutionDontReplaceIgnoredTypes && get(tp)) + return true; + else + return false; +} + TypeId ApplyTypeFunction::clean(TypeId ty) { // Really this should just replace the arguments, // but for bug-compatibility with existing code, we replace // all generics by free type variables. - TypeId& arg = arguments[ty]; + TypeId& arg = typeArguments[ty]; if (arg) return arg; else @@ -4816,17 +4934,37 @@ TypePackId ApplyTypeFunction::clean(TypePackId tp) // Really this should just replace the arguments, // but for bug-compatibility with existing code, we replace // all generics by free type variables. - return addTypePack(FreeTypePack{level}); + if (FFlag::LuauTypeAliasPacks) + { + TypePackId& arg = typePackArguments[tp]; + if (arg) + return arg; + else + return addTypePack(FreeTypePack{level}); + } + else + { + return addTypePack(FreeTypePack{level}); + } } -TypeId TypeChecker::instantiateTypeFun(const ScopePtr& scope, const TypeFun& tf, const std::vector& typeParams, const Location& location) +TypeId TypeChecker::instantiateTypeFun(const ScopePtr& scope, const TypeFun& tf, const std::vector& typeParams, + const std::vector& typePackParams, const Location& location) { - if (tf.typeParams.empty()) + if (tf.typeParams.empty() && (!FFlag::LuauTypeAliasPacks || tf.typePackParams.empty())) return tf.type; - applyTypeFunction.arguments.clear(); + applyTypeFunction.typeArguments.clear(); for (size_t i = 0; i < tf.typeParams.size(); ++i) - applyTypeFunction.arguments[tf.typeParams[i]] = typeParams[i]; + applyTypeFunction.typeArguments[tf.typeParams[i]] = typeParams[i]; + + if (FFlag::LuauTypeAliasPacks) + { + applyTypeFunction.typePackArguments.clear(); + for (size_t i = 0; i < tf.typePackParams.size(); ++i) + applyTypeFunction.typePackArguments[tf.typePackParams[i]] = typePackParams[i]; + } + applyTypeFunction.currentModule = currentModule; applyTypeFunction.level = scope->level; applyTypeFunction.encounteredForwardedType = false; @@ -4875,6 +5013,9 @@ TypeId TypeChecker::instantiateTypeFun(const ScopePtr& scope, const TypeFun& tf, if (ttv) { ttv->instantiatedTypeParams = typeParams; + + if (FFlag::LuauTypeAliasPacks) + ttv->instantiatedTypePackParams = typePackParams; } } else @@ -4890,6 +5031,9 @@ TypeId TypeChecker::instantiateTypeFun(const ScopePtr& scope, const TypeFun& tf, } ttv->instantiatedTypeParams = typeParams; + + if (FFlag::LuauTypeAliasPacks) + ttv->instantiatedTypePackParams = typePackParams; } } @@ -4899,6 +5043,8 @@ TypeId TypeChecker::instantiateTypeFun(const ScopePtr& scope, const TypeFun& tf, std::pair, std::vector> TypeChecker::createGenericTypes( const ScopePtr& scope, const AstNode& node, const AstArray& genericNames, const AstArray& genericPackNames) { + LUAU_ASSERT(scope->parent); + std::vector generics; for (const AstName& generic : genericNames) { @@ -4912,7 +5058,19 @@ std::pair, std::vector> TypeChecker::createGener reportError(TypeError{node.location, DuplicateGenericParameter{n}}); } - TypeId g = addType(Unifiable::Generic{scope->level, n}); + TypeId g; + if (FFlag::LuauRecursiveTypeParameterRestriction && FFlag::LuauTypeAliasPacks) + { + TypeId& cached = scope->parent->typeAliasTypeParameters[n]; + if (!cached) + cached = addType(GenericTypeVar{scope->level, n}); + g = cached; + } + else + { + g = addType(Unifiable::Generic{scope->level, n}); + } + generics.push_back(g); scope->privateTypeBindings[n] = TypeFun{{}, g}; } @@ -4930,7 +5088,19 @@ std::pair, std::vector> TypeChecker::createGener reportError(TypeError{node.location, DuplicateGenericParameter{n}}); } - TypePackId g = addTypePack(TypePackVar{Unifiable::Generic{scope->level, n}}); + TypePackId g; + if (FFlag::LuauRecursiveTypeParameterRestriction && FFlag::LuauTypeAliasPacks) + { + TypePackId& cached = scope->parent->typeAliasTypePackParameters[n]; + if (!cached) + cached = addTypePack(TypePackVar{Unifiable::Generic{scope->level, n}}); + g = cached; + } + else + { + g = addTypePack(TypePackVar{Unifiable::Generic{scope->level, n}}); + } + genericPacks.push_back(g); scope->privateTypePackBindings[n] = g; } @@ -5013,13 +5183,8 @@ void TypeChecker::resolve(const Predicate& predicate, ErrorVec& errVec, Refineme else if (auto isaP = get(predicate)) resolve(*isaP, errVec, refis, scope, sense); else if (auto typeguardP = get(predicate)) - { - if (FFlag::LuauImprovedTypeGuardPredicate2) - resolve(*typeguardP, errVec, refis, scope, sense); - else - DEPRECATED_resolve(*typeguardP, errVec, refis, scope, sense); - } - else if (auto eqP = get(predicate); eqP && FFlag::LuauEqConstraint) + resolve(*typeguardP, errVec, refis, scope, sense); + else if (auto eqP = get(predicate)) resolve(*eqP, errVec, refis, scope, sense); else ice("Unhandled predicate kind"); @@ -5145,7 +5310,7 @@ void TypeChecker::resolve(const IsAPredicate& isaP, ErrorVec& errVec, Refinement return isaP.ty; } } - else if (FFlag::LuauImprovedTypeGuardPredicate2) + else { auto lctv = get(option); auto rctv = get(isaP.ty); @@ -5159,19 +5324,6 @@ void TypeChecker::resolve(const IsAPredicate& isaP, ErrorVec& errVec, Refinement if (canUnify(option, isaP.ty, isaP.location).empty() == sense) return isaP.ty; } - else - { - auto lctv = get(option); - auto rctv = get(isaP.ty); - - if (lctv && rctv) - { - if (isSubclass(lctv, rctv) == sense) - return option; - else if (isSubclass(rctv, lctv) == sense) - return isaP.ty; - } - } return std::nullopt; }; @@ -5266,7 +5418,7 @@ void TypeChecker::resolve(const TypeGuardPredicate& typeguardP, ErrorVec& errVec return fail(UnknownSymbol{typeguardP.kind, UnknownSymbol::Type}); auto typeFun = globalScope->lookupType(typeguardP.kind); - if (!typeFun || !typeFun->typeParams.empty()) + if (!typeFun || !typeFun->typeParams.empty() || (FFlag::LuauTypeAliasPacks && !typeFun->typePackParams.empty())) return fail(UnknownSymbol{typeguardP.kind, UnknownSymbol::Type}); TypeId type = follow(typeFun->type); @@ -5292,7 +5444,8 @@ void TypeChecker::DEPRECATED_resolve(const TypeGuardPredicate& typeguardP, Error "userdata", // no op. Requires special handling. }; - if (auto typeFun = globalScope->lookupType(typeguardP.kind); typeFun && typeFun->typeParams.empty()) + if (auto typeFun = globalScope->lookupType(typeguardP.kind); + typeFun && typeFun->typeParams.empty() && (!FFlag::LuauTypeAliasPacks || typeFun->typePackParams.empty())) { if (auto it = std::find(primitives.begin(), primitives.end(), typeguardP.kind); it != primitives.end()) addRefinement(refis, typeguardP.lvalue, typeFun->type); @@ -5319,38 +5472,41 @@ void TypeChecker::resolve(const EqPredicate& eqP, ErrorVec& errVec, RefinementMa return; } - std::optional ty = resolveLValue(refis, scope, eqP.lvalue); - if (!ty) - return; - - std::vector lhs = options(*ty); - std::vector rhs = options(eqP.type); - - if (sense && std::any_of(lhs.begin(), lhs.end(), isUndecidable)) + if (FFlag::LuauEqConstraint) { - addRefinement(refis, eqP.lvalue, eqP.type); - return; - } - else if (sense && std::any_of(rhs.begin(), rhs.end(), isUndecidable)) - return; // Optimization: the other side has unknown types, so there's probably an overlap. Refining is no-op here. + std::optional ty = resolveLValue(refis, scope, eqP.lvalue); + if (!ty) + return; - std::unordered_set set; - for (TypeId left : lhs) - { - for (TypeId right : rhs) + std::vector lhs = options(*ty); + std::vector rhs = options(eqP.type); + + if (sense && std::any_of(lhs.begin(), lhs.end(), isUndecidable)) { - // When singleton types arrive, `isNil` here probably should be replaced with `isLiteral`. - if (canUnify(left, right, eqP.location).empty() == sense || (!sense && !isNil(left))) - set.insert(left); + addRefinement(refis, eqP.lvalue, eqP.type); + return; } + else if (sense && std::any_of(rhs.begin(), rhs.end(), isUndecidable)) + return; // Optimization: the other side has unknown types, so there's probably an overlap. Refining is no-op here. + + std::unordered_set set; + for (TypeId left : lhs) + { + for (TypeId right : rhs) + { + // When singleton types arrive, `isNil` here probably should be replaced with `isLiteral`. + if (canUnify(left, right, eqP.location).empty() == sense || (!sense && !isNil(left))) + set.insert(left); + } + } + + if (set.empty()) + return; + + std::vector viable(set.begin(), set.end()); + TypeId result = viable.size() == 1 ? viable[0] : addType(UnionTypeVar{std::move(viable)}); + addRefinement(refis, eqP.lvalue, result); } - - if (set.empty()) - return; - - std::vector viable(set.begin(), set.end()); - TypeId result = viable.size() == 1 ? viable[0] : addType(UnionTypeVar{std::move(viable)}); - addRefinement(refis, eqP.lvalue, result); } bool TypeChecker::isNonstrictMode() const @@ -5379,119 +5535,4 @@ std::vector> TypeChecker::getScopes() const return currentModule->scopes; } -Scope::Scope(TypePackId returnType) - : parent(nullptr) - , returnType(returnType) - , level(TypeLevel()) -{ -} - -Scope::Scope(const ScopePtr& parent, int subLevel) - : parent(parent) - , returnType(parent->returnType) - , level(parent->level.incr()) -{ - level.subLevel = subLevel; -} - -std::optional Scope::lookup(const Symbol& name) -{ - Scope* scope = this; - - while (scope) - { - auto it = scope->bindings.find(name); - if (it != scope->bindings.end()) - return it->second.typeId; - - scope = scope->parent.get(); - } - - return std::nullopt; -} - -std::optional Scope::lookupType(const Name& name) -{ - const Scope* scope = this; - while (true) - { - auto it = scope->exportedTypeBindings.find(name); - if (it != scope->exportedTypeBindings.end()) - return it->second; - - it = scope->privateTypeBindings.find(name); - if (it != scope->privateTypeBindings.end()) - return it->second; - - if (scope->parent) - scope = scope->parent.get(); - else - return std::nullopt; - } -} - -std::optional Scope::lookupImportedType(const Name& moduleAlias, const Name& name) -{ - const Scope* scope = this; - while (scope) - { - auto it = scope->importedTypeBindings.find(moduleAlias); - if (it == scope->importedTypeBindings.end()) - { - scope = scope->parent.get(); - continue; - } - - auto it2 = it->second.find(name); - if (it2 == it->second.end()) - { - scope = scope->parent.get(); - continue; - } - - return it2->second; - } - - return std::nullopt; -} - -std::optional Scope::lookupPack(const Name& name) -{ - const Scope* scope = this; - while (true) - { - auto it = scope->privateTypePackBindings.find(name); - if (it != scope->privateTypePackBindings.end()) - return it->second; - - if (scope->parent) - scope = scope->parent.get(); - else - return std::nullopt; - } -} - -std::optional Scope::linearSearchForBinding(const std::string& name, bool traverseScopeChain) -{ - Scope* scope = this; - - while (scope) - { - for (const auto& [n, binding] : scope->bindings) - { - if (n.local && n.local->name == name.c_str()) - return binding; - else if (n.global.value && n.global == name.c_str()) - return binding; - } - - scope = scope->parent.get(); - - if (!traverseScopeChain) - break; - } - - return std::nullopt; -} - } // namespace Luau diff --git a/Analysis/src/TypePack.cpp b/Analysis/src/TypePack.cpp index 5970f304..68a16ef0 100644 --- a/Analysis/src/TypePack.cpp +++ b/Analysis/src/TypePack.cpp @@ -209,6 +209,19 @@ size_t size(TypePackId tp) return 0; } +bool finite(TypePackId tp) +{ + tp = follow(tp); + + if (auto pack = get(tp)) + return pack->tail ? finite(*pack->tail) : true; + + if (auto pack = get(tp)) + return false; + + return true; +} + size_t size(const TypePack& tp) { size_t result = tp.head.size(); diff --git a/Analysis/src/TypeUtils.cpp b/Analysis/src/TypeUtils.cpp index b9f50978..0d9d91e0 100644 --- a/Analysis/src/TypeUtils.cpp +++ b/Analysis/src/TypeUtils.cpp @@ -1,11 +1,10 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #include "Luau/TypeUtils.h" +#include "Luau/Scope.h" #include "Luau/ToString.h" #include "Luau/TypeInfer.h" -LUAU_FASTFLAG(LuauStringMetatable) - namespace Luau { @@ -13,21 +12,6 @@ std::optional findMetatableEntry(ErrorVec& errors, const ScopePtr& globa { type = follow(type); - if (!FFlag::LuauStringMetatable) - { - if (const PrimitiveTypeVar* primType = get(type)) - { - if (primType->type != PrimitiveTypeVar::String || "__index" != entry) - return std::nullopt; - - auto it = globalScope->bindings.find(AstName{"string"}); - if (it != globalScope->bindings.end()) - return it->second.typeId; - else - return std::nullopt; - } - } - std::optional metatable = getMetatable(type); if (!metatable) return std::nullopt; diff --git a/Analysis/src/TypeVar.cpp b/Analysis/src/TypeVar.cpp index 111f4f53..e963fc74 100644 --- a/Analysis/src/TypeVar.cpp +++ b/Analysis/src/TypeVar.cpp @@ -19,11 +19,9 @@ LUAU_FASTINTVARIABLE(LuauTypeMaximumStringifierLength, 500) LUAU_FASTINTVARIABLE(LuauTableTypeMaximumStringifierLength, 0) -LUAU_FASTFLAG(LuauImprovedTypeGuardPredicate2) -LUAU_FASTFLAGVARIABLE(LuauToStringFollowsBoundTo, false) LUAU_FASTFLAG(LuauRankNTypes) -LUAU_FASTFLAGVARIABLE(LuauStringMetatable, false) LUAU_FASTFLAG(LuauTypeGuardPeelsAwaySubclasses) +LUAU_FASTFLAG(LuauTypeAliasPacks) namespace Luau { @@ -193,27 +191,11 @@ bool isOptional(TypeId ty) bool isTableIntersection(TypeId ty) { - if (FFlag::LuauImprovedTypeGuardPredicate2) - { - if (!get(follow(ty))) - return false; - - std::vector parts = flattenIntersection(ty); - return std::all_of(parts.begin(), parts.end(), getTableType); - } - else - { - if (const IntersectionTypeVar* itv = get(ty)) - { - for (TypeId part : itv->parts) - { - if (getTableType(follow(part))) - return true; - } - } - + if (!get(follow(ty))) return false; - } + + std::vector parts = flattenIntersection(ty); + return std::all_of(parts.begin(), parts.end(), getTableType); } bool isOverloadedFunction(TypeId ty) @@ -236,7 +218,7 @@ std::optional getMetatable(TypeId type) else if (const ClassTypeVar* classType = get(type)) return classType->metatable; else if (const PrimitiveTypeVar* primitiveType = get(type); - FFlag::LuauStringMetatable && primitiveType && primitiveType->metatable) + primitiveType && primitiveType->metatable) { LUAU_ASSERT(primitiveType->type == PrimitiveTypeVar::String); return primitiveType->metatable; @@ -871,6 +853,12 @@ void StateDot::visitChildren(TypeId ty, int index) } for (TypeId itp : ttv->instantiatedTypeParams) visitChild(itp, index, "typeParam"); + + if (FFlag::LuauTypeAliasPacks) + { + for (TypePackId itp : ttv->instantiatedTypePackParams) + visitChild(itp, index, "typePackParam"); + } } else if (const MetatableTypeVar* mtv = get(ty)) { diff --git a/Analysis/src/Unifier.cpp b/Analysis/src/Unifier.cpp index 96795999..56f2515f 100644 --- a/Analysis/src/Unifier.cpp +++ b/Analysis/src/Unifier.cpp @@ -3,23 +3,25 @@ #include "Luau/Common.h" #include "Luau/RecursionCounter.h" +#include "Luau/Scope.h" #include "Luau/TypePack.h" #include "Luau/TypeUtils.h" +#include "Luau/TimeTrace.h" #include LUAU_FASTINT(LuauTypeInferRecursionLimit); LUAU_FASTINT(LuauTypeInferTypePackLoopLimit); -LUAU_FASTINTVARIABLE(LuauTypeInferIterationLimit, 0); -LUAU_FASTFLAGVARIABLE(LuauLogTableTypeVarBoundTo, false) +LUAU_FASTINTVARIABLE(LuauTypeInferIterationLimit, 2000); LUAU_FASTFLAG(LuauGenericFunctions) +LUAU_FASTFLAGVARIABLE(LuauTableSubtypingVariance, false); LUAU_FASTFLAGVARIABLE(LuauDontMutatePersistentFunctions, false) LUAU_FASTFLAG(LuauRankNTypes) -LUAU_FASTFLAG(LuauStringMetatable) LUAU_FASTFLAGVARIABLE(LuauUnionHeuristic, false) LUAU_FASTFLAGVARIABLE(LuauTableUnificationEarlyTest, false) LUAU_FASTFLAGVARIABLE(LuauSealedTableUnifyOptionalFix, false) LUAU_FASTFLAGVARIABLE(LuauOccursCheckOkWithRecursiveFunctions, false) +LUAU_FASTFLAGVARIABLE(LuauTypecheckOpts, false) namespace Luau { @@ -43,21 +45,23 @@ Unifier::Unifier(TypeArena* types, Mode mode, ScopePtr globalScope, const Locati , globalScope(std::move(globalScope)) , location(location) , variance(variance) - , counters(std::make_shared()) + , counters(&countersData) + , counters_DEPRECATED(std::make_shared()) , iceHandler(iceHandler) { LUAU_ASSERT(iceHandler); } Unifier::Unifier(TypeArena* types, Mode mode, ScopePtr globalScope, const std::vector>& seen, const Location& location, - Variance variance, InternalErrorReporter* iceHandler, const std::shared_ptr& counters) + Variance variance, InternalErrorReporter* iceHandler, const std::shared_ptr& counters_DEPRECATED, UnifierCounters* counters) : types(types) , mode(mode) , globalScope(std::move(globalScope)) , log(seen) , location(location) , variance(variance) - , counters(counters ? counters : std::make_shared()) + , counters(counters ? counters : &countersData) + , counters_DEPRECATED(counters_DEPRECATED ? counters_DEPRECATED : std::make_shared()) , iceHandler(iceHandler) { LUAU_ASSERT(iceHandler); @@ -65,16 +69,26 @@ Unifier::Unifier(TypeArena* types, Mode mode, ScopePtr globalScope, const std::v void Unifier::tryUnify(TypeId superTy, TypeId subTy, bool isFunctionCall, bool isIntersection) { - counters->iterationCount = 0; + if (FFlag::LuauTypecheckOpts) + counters->iterationCount = 0; + else + counters_DEPRECATED->iterationCount = 0; + return tryUnify_(superTy, subTy, isFunctionCall, isIntersection); } void Unifier::tryUnify_(TypeId superTy, TypeId subTy, bool isFunctionCall, bool isIntersection) { - RecursionLimiter _ra(&counters->recursionCount, FInt::LuauTypeInferRecursionLimit); + RecursionLimiter _ra( + FFlag::LuauTypecheckOpts ? &counters->recursionCount : &counters_DEPRECATED->recursionCount, FInt::LuauTypeInferRecursionLimit); - ++counters->iterationCount; - if (FInt::LuauTypeInferIterationLimit > 0 && FInt::LuauTypeInferIterationLimit < counters->iterationCount) + if (FFlag::LuauTypecheckOpts) + ++counters->iterationCount; + else + ++counters_DEPRECATED->iterationCount; + + if (FInt::LuauTypeInferIterationLimit > 0 && + FInt::LuauTypeInferIterationLimit < (FFlag::LuauTypecheckOpts ? counters->iterationCount : counters_DEPRECATED->iterationCount)) { errors.push_back(TypeError{location, UnificationTooComplex{}}); return; @@ -440,7 +454,11 @@ ErrorVec Unifier::canUnify(TypePackId superTy, TypePackId subTy, bool isFunction void Unifier::tryUnify(TypePackId superTp, TypePackId subTp, bool isFunctionCall) { - counters->iterationCount = 0; + if (FFlag::LuauTypecheckOpts) + counters->iterationCount = 0; + else + counters_DEPRECATED->iterationCount = 0; + return tryUnify_(superTp, subTp, isFunctionCall); } @@ -450,10 +468,16 @@ void Unifier::tryUnify(TypePackId superTp, TypePackId subTp, bool isFunctionCall */ void Unifier::tryUnify_(TypePackId superTp, TypePackId subTp, bool isFunctionCall) { - RecursionLimiter _ra(&counters->recursionCount, FInt::LuauTypeInferRecursionLimit); + RecursionLimiter _ra( + FFlag::LuauTypecheckOpts ? &counters->recursionCount : &counters_DEPRECATED->recursionCount, FInt::LuauTypeInferRecursionLimit); - ++counters->iterationCount; - if (FInt::LuauTypeInferIterationLimit > 0 && FInt::LuauTypeInferIterationLimit < counters->iterationCount) + if (FFlag::LuauTypecheckOpts) + ++counters->iterationCount; + else + ++counters_DEPRECATED->iterationCount; + + if (FInt::LuauTypeInferIterationLimit > 0 && + FInt::LuauTypeInferIterationLimit < (FFlag::LuauTypecheckOpts ? counters->iterationCount : counters_DEPRECATED->iterationCount)) { errors.push_back(TypeError{location, UnificationTooComplex{}}); return; @@ -762,9 +786,210 @@ struct Resetter void Unifier::tryUnifyTables(TypeId left, TypeId right, bool isIntersection) { - std::unique_ptr resetter; + if (!FFlag::LuauTableSubtypingVariance) + return DEPRECATED_tryUnifyTables(left, right, isIntersection); - resetter.reset(new Resetter{&variance}); + TableTypeVar* lt = getMutable(left); + TableTypeVar* rt = getMutable(right); + if (!lt || !rt) + ice("passed non-table types to unifyTables"); + + std::vector missingProperties; + std::vector extraProperties; + + // Reminder: left is the supertype, right is the subtype. + // Width subtyping: any property in the supertype must be in the subtype, + // and the types must agree. + for (const auto& [name, prop] : lt->props) + { + const auto& r = rt->props.find(name); + if (r != rt->props.end()) + { + // TODO: read-only properties don't need invariance + Resetter resetter{&variance}; + variance = Invariant; + + Unifier innerState = makeChildUnifier(); + innerState.tryUnify_(prop.type, r->second.type); + checkChildUnifierTypeMismatch(innerState.errors, left, right); + if (innerState.errors.empty()) + log.concat(std::move(innerState.log)); + else + innerState.log.rollback(); + } + else if (rt->indexer && isString(rt->indexer->indexType)) + { + // TODO: read-only indexers don't need invariance + // TODO: really we should only allow this if prop.type is optional. + Resetter resetter{&variance}; + variance = Invariant; + + Unifier innerState = makeChildUnifier(); + innerState.tryUnify_(prop.type, rt->indexer->indexResultType); + checkChildUnifierTypeMismatch(innerState.errors, left, right); + if (innerState.errors.empty()) + log.concat(std::move(innerState.log)); + else + innerState.log.rollback(); + } + else if (isOptional(prop.type) || get(follow(prop.type))) + // TODO: this case is unsound, but without it our test suite fails. CLI-46031 + // TODO: should isOptional(anyType) be true? + {} + else if (rt->state == TableState::Free) + { + log(rt); + rt->props[name] = prop; + } + else + missingProperties.push_back(name); + } + + for (const auto& [name, prop] : rt->props) + { + if (lt->props.count(name)) + { + // If both lt and rt contain the property, then + // we're done since we already unified them above + } + else if (lt->indexer && isString(lt->indexer->indexType)) + { + // TODO: read-only indexers don't need invariance + // TODO: really we should only allow this if prop.type is optional. + Resetter resetter{&variance}; + variance = Invariant; + + Unifier innerState = makeChildUnifier(); + innerState.tryUnify_(prop.type, lt->indexer->indexResultType); + checkChildUnifierTypeMismatch(innerState.errors, left, right); + if (innerState.errors.empty()) + log.concat(std::move(innerState.log)); + else + innerState.log.rollback(); + } + else if (lt->state == TableState::Unsealed) + { + // TODO: this case is unsound when variance is Invariant, but without it lua-apps fails to typecheck. + // TODO: file a JIRA + // TODO: hopefully readonly/writeonly properties will fix this. + Property clone = prop; + clone.type = deeplyOptional(clone.type); + log(lt); + lt->props[name] = clone; + } + else if (variance == Covariant) + {} + else if (isOptional(prop.type) || get(follow(prop.type))) + // TODO: this case is unsound, but without it our test suite fails. CLI-46031 + // TODO: should isOptional(anyType) be true? + {} + else if (lt->state == TableState::Free) + { + log(lt); + lt->props[name] = prop; + } + else + extraProperties.push_back(name); + } + + // Unify indexers + if (lt->indexer && rt->indexer) + { + // TODO: read-only indexers don't need invariance + Resetter resetter{&variance}; + variance = Invariant; + + Unifier innerState = makeChildUnifier(); + innerState.tryUnify(*lt->indexer, *rt->indexer); + checkChildUnifierTypeMismatch(innerState.errors, left, right); + if (innerState.errors.empty()) + log.concat(std::move(innerState.log)); + else + innerState.log.rollback(); + } + else if (lt->indexer) + { + if (rt->state == TableState::Unsealed || rt->state == TableState::Free) + { + // passing/assigning a table without an indexer to something that has one + // e.g. table.insert(t, 1) where t is a non-sealed table and doesn't have an indexer. + // TODO: we only need to do this if the supertype's indexer is read/write + // since that can add indexed elements. + log(rt); + rt->indexer = lt->indexer; + } + } + else if (rt->indexer && variance == Invariant) + { + // Symmetric if we are invariant + if (lt->state == TableState::Unsealed || lt->state == TableState::Free) + { + log(lt); + lt->indexer = rt->indexer; + } + } + + if (!missingProperties.empty()) + { + errors.push_back(TypeError{location, MissingProperties{left, right, std::move(missingProperties)}}); + return; + } + + if (!extraProperties.empty()) + { + errors.push_back(TypeError{location, MissingProperties{left, right, std::move(extraProperties), MissingProperties::Extra}}); + return; + } + + /* + * TypeVars are commonly cyclic, so it is entirely possible + * for unifying a property of a table to change the table itself! + * We need to check for this and start over if we notice this occurring. + * + * I believe this is guaranteed to terminate eventually because this will + * only happen when a free table is bound to another table. + */ + if (lt->boundTo || rt->boundTo) + return tryUnify_(left, right); + + if (lt->state == TableState::Free) + { + log(lt); + lt->boundTo = right; + } + else if (rt->state == TableState::Free) + { + log(rt); + rt->boundTo = left; + } +} + +TypeId Unifier::deeplyOptional(TypeId ty, std::unordered_map seen) +{ + ty = follow(ty); + if (get(ty)) + return ty; + else if (isOptional(ty)) + return ty; + else if (const TableTypeVar* ttv = get(ty)) + { + TypeId& result = seen[ty]; + if (result) + return result; + result = types->addType(*ttv); + TableTypeVar* resultTtv = getMutable(result); + for (auto& [name, prop] : resultTtv->props) + prop.type = deeplyOptional(prop.type, seen); + return types->addType(UnionTypeVar{{ singletonTypes.nilType, result }});; + } + else + return types->addType(UnionTypeVar{{ singletonTypes.nilType, ty }}); +} + +void Unifier::DEPRECATED_tryUnifyTables(TypeId left, TypeId right, bool isIntersection) +{ + LUAU_ASSERT(!FFlag::LuauTableSubtypingVariance); + Resetter resetter{&variance}; variance = Invariant; TableTypeVar* lt = getMutable(left); @@ -894,10 +1119,7 @@ void Unifier::tryUnifyFreeTable(TypeId freeTypeId, TypeId otherTypeId) if (!freeTable->boundTo && otherTable->state != TableState::Free) { - if (FFlag::LuauLogTableTypeVarBoundTo) - log(freeTable); - else - log(freeTypeId); + log(freeTable); freeTable->boundTo = otherTypeId; } } @@ -1196,9 +1418,11 @@ void Unifier::tryUnify(const TableIndexer& superIndexer, const TableIndexer& sub tryUnify_(superIndexer.indexResultType, subIndexer.indexResultType); } -static void queueTypePack( +static void queueTypePack_DEPRECATED( std::vector& queue, std::unordered_set& seenTypePacks, Unifier& state, TypePackId a, TypePackId anyTypePack) { + LUAU_ASSERT(!FFlag::LuauTypecheckOpts); + while (true) { if (FFlag::LuauAddMissingFollow) @@ -1244,6 +1468,55 @@ static void queueTypePack( } } +static void queueTypePack(std::vector& queue, DenseHashSet& seenTypePacks, Unifier& state, TypePackId a, TypePackId anyTypePack) +{ + LUAU_ASSERT(FFlag::LuauTypecheckOpts); + + while (true) + { + if (FFlag::LuauAddMissingFollow) + a = follow(a); + + if (seenTypePacks.find(a)) + break; + seenTypePacks.insert(a); + + if (FFlag::LuauAddMissingFollow) + { + if (get(a)) + { + state.log(a); + *asMutable(a) = Unifiable::Bound{anyTypePack}; + } + else if (auto tp = get(a)) + { + queue.insert(queue.end(), tp->head.begin(), tp->head.end()); + if (tp->tail) + a = *tp->tail; + else + break; + } + } + else + { + if (get(a)) + { + state.log(a); + *asMutable(a) = Unifiable::Bound{anyTypePack}; + } + + if (auto tp = get(a)) + { + queue.insert(queue.end(), tp->head.begin(), tp->head.end()); + if (tp->tail) + a = *tp->tail; + else + break; + } + } + } +} + void Unifier::tryUnifyVariadics(TypePackId superTp, TypePackId subTp, bool reversed, int subOffset) { const VariadicTypePack* lv = get(superTp); @@ -1297,9 +1570,11 @@ void Unifier::tryUnifyVariadics(TypePackId superTp, TypePackId subTp, bool rever } } -static void tryUnifyWithAny( +static void tryUnifyWithAny_DEPRECATED( std::vector& queue, Unifier& state, std::unordered_set& seenTypePacks, TypeId anyType, TypePackId anyTypePack) { + LUAU_ASSERT(!FFlag::LuauTypecheckOpts); + std::unordered_set seen; while (!queue.empty()) @@ -1310,6 +1585,59 @@ static void tryUnifyWithAny( continue; seen.insert(ty); + if (get(ty)) + { + state.log(ty); + *asMutable(ty) = BoundTypeVar{anyType}; + } + else if (auto fun = get(ty)) + { + queueTypePack_DEPRECATED(queue, seenTypePacks, state, fun->argTypes, anyTypePack); + queueTypePack_DEPRECATED(queue, seenTypePacks, state, fun->retType, anyTypePack); + } + else if (auto table = get(ty)) + { + for (const auto& [_name, prop] : table->props) + queue.push_back(prop.type); + + if (table->indexer) + { + queue.push_back(table->indexer->indexType); + queue.push_back(table->indexer->indexResultType); + } + } + else if (auto mt = get(ty)) + { + queue.push_back(mt->table); + queue.push_back(mt->metatable); + } + else if (get(ty)) + { + // ClassTypeVars never contain free typevars. + } + else if (auto union_ = get(ty)) + queue.insert(queue.end(), union_->options.begin(), union_->options.end()); + else if (auto intersection = get(ty)) + queue.insert(queue.end(), intersection->parts.begin(), intersection->parts.end()); + else + { + } // Primitives, any, errors, and generics are left untouched. + } +} + +static void tryUnifyWithAny(std::vector& queue, Unifier& state, DenseHashSet& seen, DenseHashSet& seenTypePacks, + TypeId anyType, TypePackId anyTypePack) +{ + LUAU_ASSERT(FFlag::LuauTypecheckOpts); + + while (!queue.empty()) + { + TypeId ty = follow(queue.back()); + queue.pop_back(); + if (seen.find(ty)) + continue; + seen.insert(ty); + if (get(ty)) { state.log(ty); @@ -1354,14 +1682,33 @@ void Unifier::tryUnifyWithAny(TypeId any, TypeId ty) { LUAU_ASSERT(get(any) || get(any)); + if (FFlag::LuauTypecheckOpts) + { + // These types are not visited in general loop below + if (get(ty) || get(ty) || get(ty)) + return; + } + const TypePackId anyTypePack = types->addTypePack(TypePackVar{VariadicTypePack{singletonTypes.anyType}}); const TypePackId anyTP = get(any) ? anyTypePack : types->addTypePack(TypePackVar{Unifiable::Error{}}); - std::unordered_set seenTypePacks; - std::vector queue = {ty}; + if (FFlag::LuauTypecheckOpts) + { + std::vector queue = {ty}; - Luau::tryUnifyWithAny(queue, *this, seenTypePacks, singletonTypes.anyType, anyTP); + tempSeenTy.clear(); + tempSeenTp.clear(); + + Luau::tryUnifyWithAny(queue, *this, tempSeenTy, tempSeenTp, singletonTypes.anyType, anyTP); + } + else + { + std::unordered_set seenTypePacks; + std::vector queue = {ty}; + + Luau::tryUnifyWithAny_DEPRECATED(queue, *this, seenTypePacks, singletonTypes.anyType, anyTP); + } } void Unifier::tryUnifyWithAny(TypePackId any, TypePackId ty) @@ -1370,12 +1717,26 @@ void Unifier::tryUnifyWithAny(TypePackId any, TypePackId ty) const TypeId anyTy = singletonTypes.errorType; - std::unordered_set seenTypePacks; - std::vector queue; + if (FFlag::LuauTypecheckOpts) + { + std::vector queue; - queueTypePack(queue, seenTypePacks, *this, ty, any); + tempSeenTy.clear(); + tempSeenTp.clear(); - Luau::tryUnifyWithAny(queue, *this, seenTypePacks, anyTy, any); + queueTypePack(queue, tempSeenTp, *this, ty, any); + + Luau::tryUnifyWithAny(queue, *this, tempSeenTy, tempSeenTp, anyTy, any); + } + else + { + std::unordered_set seenTypePacks; + std::vector queue; + + queueTypePack_DEPRECATED(queue, seenTypePacks, *this, ty, any); + + Luau::tryUnifyWithAny_DEPRECATED(queue, *this, seenTypePacks, anyTy, any); + } } std::optional Unifier::findTablePropertyRespectingMeta(TypeId lhsType, Name name) @@ -1387,21 +1748,6 @@ std::optional Unifier::findMetatableEntry(TypeId type, std::string entry { type = follow(type); - if (!FFlag::LuauStringMetatable) - { - if (const PrimitiveTypeVar* primType = get(type)) - { - if (primType->type != PrimitiveTypeVar::String || "__index" != entry) - return std::nullopt; - - auto found = globalScope->bindings.find(AstName{"string"}); - if (found == globalScope->bindings.end()) - return std::nullopt; - else - return found->second.typeId; - } - } - std::optional metatable = getMetatable(type); if (!metatable) return std::nullopt; @@ -1427,21 +1773,36 @@ std::optional Unifier::findMetatableEntry(TypeId type, std::string entry void Unifier::occursCheck(TypeId needle, TypeId haystack) { - std::unordered_set seen; - return occursCheck(seen, needle, haystack); + std::unordered_set seen_DEPRECATED; + + if (FFlag::LuauTypecheckOpts) + tempSeenTy.clear(); + + return occursCheck(seen_DEPRECATED, tempSeenTy, needle, haystack); } -void Unifier::occursCheck(std::unordered_set& seen, TypeId needle, TypeId haystack) +void Unifier::occursCheck(std::unordered_set& seen_DEPRECATED, DenseHashSet& seen, TypeId needle, TypeId haystack) { - RecursionLimiter _ra(&counters->recursionCount, FInt::LuauTypeInferRecursionLimit); + RecursionLimiter _ra( + FFlag::LuauTypecheckOpts ? &counters->recursionCount : &counters_DEPRECATED->recursionCount, FInt::LuauTypeInferRecursionLimit); needle = follow(needle); haystack = follow(haystack); - if (seen.end() != seen.find(haystack)) - return; + if (FFlag::LuauTypecheckOpts) + { + if (seen.find(haystack)) + return; - seen.insert(haystack); + seen.insert(haystack); + } + else + { + if (seen_DEPRECATED.end() != seen_DEPRECATED.find(haystack)) + return; + + seen_DEPRECATED.insert(haystack); + } if (get(needle)) return; @@ -1458,7 +1819,7 @@ void Unifier::occursCheck(std::unordered_set& seen, TypeId needle, TypeI } auto check = [&](TypeId tv) { - occursCheck(seen, needle, tv); + occursCheck(seen_DEPRECATED, seen, needle, tv); }; if (get(haystack)) @@ -1488,19 +1849,33 @@ void Unifier::occursCheck(std::unordered_set& seen, TypeId needle, TypeI void Unifier::occursCheck(TypePackId needle, TypePackId haystack) { - std::unordered_set seen; - return occursCheck(seen, needle, haystack); + std::unordered_set seen_DEPRECATED; + + if (FFlag::LuauTypecheckOpts) + tempSeenTp.clear(); + + return occursCheck(seen_DEPRECATED, tempSeenTp, needle, haystack); } -void Unifier::occursCheck(std::unordered_set& seen, TypePackId needle, TypePackId haystack) +void Unifier::occursCheck(std::unordered_set& seen_DEPRECATED, DenseHashSet& seen, TypePackId needle, TypePackId haystack) { needle = follow(needle); haystack = follow(haystack); - if (seen.find(haystack) != seen.end()) - return; + if (FFlag::LuauTypecheckOpts) + { + if (seen.find(haystack)) + return; - seen.insert(haystack); + seen.insert(haystack); + } + else + { + if (seen_DEPRECATED.end() != seen_DEPRECATED.find(haystack)) + return; + + seen_DEPRECATED.insert(haystack); + } if (get(needle)) return; @@ -1508,7 +1883,8 @@ void Unifier::occursCheck(std::unordered_set& seen, TypePackId needl if (!get(needle)) ice("Expected needle pack to be free"); - RecursionLimiter _ra(&counters->recursionCount, FInt::LuauTypeInferRecursionLimit); + RecursionLimiter _ra( + FFlag::LuauTypecheckOpts ? &counters->recursionCount : &counters_DEPRECATED->recursionCount, FInt::LuauTypeInferRecursionLimit); while (!get(haystack)) { @@ -1528,8 +1904,8 @@ void Unifier::occursCheck(std::unordered_set& seen, TypePackId needl { if (auto f = get(FFlag::LuauAddMissingFollow ? follow(ty) : ty)) { - occursCheck(seen, needle, f->argTypes); - occursCheck(seen, needle, f->retType); + occursCheck(seen_DEPRECATED, seen, needle, f->argTypes); + occursCheck(seen_DEPRECATED, seen, needle, f->retType); } } } @@ -1546,7 +1922,7 @@ void Unifier::occursCheck(std::unordered_set& seen, TypePackId needl Unifier Unifier::makeChildUnifier() { - return Unifier{types, mode, globalScope, log.seen, location, variance, iceHandler, counters}; + return Unifier{types, mode, globalScope, log.seen, location, variance, iceHandler, counters_DEPRECATED, counters}; } bool Unifier::isNonstrictMode() const diff --git a/Ast/include/Luau/Ast.h b/Ast/include/Luau/Ast.h index df38cfec..a2189f7b 100644 --- a/Ast/include/Luau/Ast.h +++ b/Ast/include/Luau/Ast.h @@ -264,6 +264,10 @@ public: { return false; } + virtual bool visit(class AstTypePackExplicit* node) + { + return visit((class AstTypePack*)node); + } virtual bool visit(class AstTypePackVariadic* node) { return visit((class AstTypePack*)node); @@ -930,12 +934,14 @@ class AstStatTypeAlias : public AstStat public: LUAU_RTTI(AstStatTypeAlias) - AstStatTypeAlias(const Location& location, const AstName& name, const AstArray& generics, AstType* type, bool exported); + AstStatTypeAlias(const Location& location, const AstName& name, const AstArray& generics, const AstArray& genericPacks, + AstType* type, bool exported); void visit(AstVisitor* visitor) override; AstName name; AstArray generics; + AstArray genericPacks; AstType* type; bool exported; }; @@ -1007,19 +1013,28 @@ public: } }; +// Don't have Luau::Variant available, it's a bit of an overhead, but a plain struct is nice to use +struct AstTypeOrPack +{ + AstType* type = nullptr; + AstTypePack* typePack = nullptr; +}; + class AstTypeReference : public AstType { public: LUAU_RTTI(AstTypeReference) - AstTypeReference(const Location& location, std::optional prefix, AstName name, const AstArray& generics = {}); + AstTypeReference(const Location& location, std::optional prefix, AstName name, bool hasParameterList = false, + const AstArray& parameters = {}); void visit(AstVisitor* visitor) override; bool hasPrefix; + bool hasParameterList; AstName prefix; AstName name; - AstArray generics; + AstArray parameters; }; struct AstTableProp @@ -1152,6 +1167,18 @@ public: } }; +class AstTypePackExplicit : public AstTypePack +{ +public: + LUAU_RTTI(AstTypePackExplicit) + + AstTypePackExplicit(const Location& location, AstTypeList typeList); + + void visit(AstVisitor* visitor) override; + + AstTypeList typeList; +}; + class AstTypePackVariadic : public AstTypePack { public: diff --git a/Ast/include/Luau/DenseHash.h b/Ast/include/Luau/DenseHash.h index 02924e88..a7b2515a 100644 --- a/Ast/include/Luau/DenseHash.h +++ b/Ast/include/Luau/DenseHash.h @@ -136,7 +136,10 @@ public: const Key& key = ItemInterface::getKey(data[i]); if (!eq(key, empty_key)) - *newtable.insert_unsafe(key) = data[i]; + { + Item* item = newtable.insert_unsafe(key); + *item = std::move(data[i]); + } } LUAU_ASSERT(count == newtable.count); diff --git a/Ast/include/Luau/Parser.h b/Ast/include/Luau/Parser.h index e6ebd503..42c64dc9 100644 --- a/Ast/include/Luau/Parser.h +++ b/Ast/include/Luau/Parser.h @@ -218,13 +218,14 @@ private: AstTableIndexer* parseTableIndexerAnnotation(); - AstType* parseFunctionTypeAnnotation(); + AstTypeOrPack parseFunctionTypeAnnotation(bool allowPack); AstType* parseFunctionTypeAnnotationTail(const Lexeme& begin, AstArray generics, AstArray genericPacks, AstArray& params, AstArray>& paramNames, AstTypePack* varargAnnotation); AstType* parseTableTypeAnnotation(); - AstType* parseSimpleTypeAnnotation(); + AstTypeOrPack parseSimpleTypeAnnotation(bool allowPack); + AstTypeOrPack parseTypeOrPackAnnotation(); AstType* parseTypeAnnotation(TempVector& parts, const Location& begin); AstType* parseTypeAnnotation(); @@ -284,7 +285,7 @@ private: std::pair, AstArray> parseGenericTypeListIfFFlagParseGenericFunctions(); // `<' typeAnnotation[, ...] `>' - AstArray parseTypeParams(); + AstArray parseTypeParams(); AstExpr* parseString(); @@ -413,6 +414,7 @@ private: std::vector scratchLocal; std::vector scratchTableTypeProps; std::vector scratchAnnotation; + std::vector scratchTypeOrPackAnnotation; std::vector scratchDeclaredClassProps; std::vector scratchItem; std::vector scratchArgName; diff --git a/Ast/include/Luau/TimeTrace.h b/Ast/include/Luau/TimeTrace.h new file mode 100644 index 00000000..641dfd3c --- /dev/null +++ b/Ast/include/Luau/TimeTrace.h @@ -0,0 +1,223 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +#pragma once + +#include "Common.h" + +#include + +#include + +LUAU_FASTFLAG(DebugLuauTimeTracing) + +#if defined(LUAU_ENABLE_TIME_TRACE) + +namespace Luau +{ +namespace TimeTrace +{ +uint32_t getClockMicroseconds(); + +struct Token +{ + const char* name; + const char* category; +}; + +enum class EventType : uint8_t +{ + Enter, + Leave, + + ArgName, + ArgValue, +}; + +struct Event +{ + EventType type; + uint16_t token; + + union + { + uint32_t microsec; // 1 hour trace limit + uint32_t dataPos; + } data; +}; + +struct GlobalContext; +struct ThreadContext; + +GlobalContext& getGlobalContext(); + +uint16_t createToken(GlobalContext& context, const char* name, const char* category); +uint32_t createThread(GlobalContext& context, ThreadContext* threadContext); +void releaseThread(GlobalContext& context, ThreadContext* threadContext); +void flushEvents(GlobalContext& context, uint32_t threadId, const std::vector& events, const std::vector& data); + +struct ThreadContext +{ + ThreadContext() + : globalContext(getGlobalContext()) + { + threadId = createThread(globalContext, this); + } + + ~ThreadContext() + { + if (!events.empty()) + flushEvents(); + + releaseThread(globalContext, this); + } + + void flushEvents() + { + static uint16_t flushToken = createToken(globalContext, "flushEvents", "TimeTrace"); + + events.push_back({EventType::Enter, flushToken, {getClockMicroseconds()}}); + + TimeTrace::flushEvents(globalContext, threadId, events, data); + + events.clear(); + data.clear(); + + events.push_back({EventType::Leave, 0, {getClockMicroseconds()}}); + } + + void eventEnter(uint16_t token) + { + eventEnter(token, getClockMicroseconds()); + } + + void eventEnter(uint16_t token, uint32_t microsec) + { + events.push_back({EventType::Enter, token, {microsec}}); + } + + void eventLeave() + { + eventLeave(getClockMicroseconds()); + } + + void eventLeave(uint32_t microsec) + { + events.push_back({EventType::Leave, 0, {microsec}}); + + if (events.size() > kEventFlushLimit) + flushEvents(); + } + + void eventArgument(const char* name, const char* value) + { + uint32_t pos = uint32_t(data.size()); + data.insert(data.end(), name, name + strlen(name) + 1); + events.push_back({EventType::ArgName, 0, {pos}}); + + pos = uint32_t(data.size()); + data.insert(data.end(), value, value + strlen(value) + 1); + events.push_back({EventType::ArgValue, 0, {pos}}); + } + + GlobalContext& globalContext; + uint32_t threadId; + std::vector events; + std::vector data; + + static constexpr size_t kEventFlushLimit = 8192; +}; + +ThreadContext& getThreadContext(); + +struct Scope +{ + explicit Scope(ThreadContext& context, uint16_t token) + : context(context) + { + if (!FFlag::DebugLuauTimeTracing) + return; + + context.eventEnter(token); + } + + ~Scope() + { + if (!FFlag::DebugLuauTimeTracing) + return; + + context.eventLeave(); + } + + ThreadContext& context; +}; + +struct OptionalTailScope +{ + explicit OptionalTailScope(ThreadContext& context, uint16_t token, uint32_t threshold) + : context(context) + , token(token) + , threshold(threshold) + { + if (!FFlag::DebugLuauTimeTracing) + return; + + pos = uint32_t(context.events.size()); + microsec = getClockMicroseconds(); + } + + ~OptionalTailScope() + { + if (!FFlag::DebugLuauTimeTracing) + return; + + if (pos == context.events.size()) + { + uint32_t curr = getClockMicroseconds(); + + if (curr - microsec > threshold) + { + context.eventEnter(token, microsec); + context.eventLeave(curr); + } + } + } + + ThreadContext& context; + uint16_t token; + uint32_t threshold; + uint32_t microsec; + uint32_t pos; +}; + +LUAU_NOINLINE std::pair createScopeData(const char* name, const char* category); + +} // namespace TimeTrace +} // namespace Luau + +// Regular scope +#define LUAU_TIMETRACE_SCOPE(name, category) \ + static auto lttScopeStatic = Luau::TimeTrace::createScopeData(name, category); \ + Luau::TimeTrace::Scope lttScope(lttScopeStatic.second, lttScopeStatic.first) + +// A scope without nested scopes that may be skipped if the time it took is less than the threshold +#define LUAU_TIMETRACE_OPTIONAL_TAIL_SCOPE(name, category, microsec) \ + static auto lttScopeStaticOptTail = Luau::TimeTrace::createScopeData(name, category); \ + Luau::TimeTrace::OptionalTailScope lttScope(lttScopeStaticOptTail.second, lttScopeStaticOptTail.first, microsec) + +// Extra key/value data can be added to regular scopes +#define LUAU_TIMETRACE_ARGUMENT(name, value) \ + do \ + { \ + if (FFlag::DebugLuauTimeTracing) \ + lttScopeStatic.second.eventArgument(name, value); \ + } while (false) + +#else + +#define LUAU_TIMETRACE_SCOPE(name, category) +#define LUAU_TIMETRACE_OPTIONAL_TAIL_SCOPE(name, category, microsec) +#define LUAU_TIMETRACE_ARGUMENT(name, value) \ + do \ + { \ + } while (false) + +#endif diff --git a/Ast/src/Ast.cpp b/Ast/src/Ast.cpp index fff1537d..b1209faa 100644 --- a/Ast/src/Ast.cpp +++ b/Ast/src/Ast.cpp @@ -641,10 +641,12 @@ void AstStatLocalFunction::visit(AstVisitor* visitor) func->visit(visitor); } -AstStatTypeAlias::AstStatTypeAlias(const Location& location, const AstName& name, const AstArray& generics, AstType* type, bool exported) +AstStatTypeAlias::AstStatTypeAlias(const Location& location, const AstName& name, const AstArray& generics, + const AstArray& genericPacks, AstType* type, bool exported) : AstStat(ClassIndex(), location) , name(name) , generics(generics) + , genericPacks(genericPacks) , type(type) , exported(exported) { @@ -729,12 +731,14 @@ void AstStatError::visit(AstVisitor* visitor) } } -AstTypeReference::AstTypeReference(const Location& location, std::optional prefix, AstName name, const AstArray& generics) +AstTypeReference::AstTypeReference( + const Location& location, std::optional prefix, AstName name, bool hasParameterList, const AstArray& parameters) : AstType(ClassIndex(), location) , hasPrefix(bool(prefix)) + , hasParameterList(hasParameterList) , prefix(prefix ? *prefix : AstName()) , name(name) - , generics(generics) + , parameters(parameters) { } @@ -742,8 +746,13 @@ void AstTypeReference::visit(AstVisitor* visitor) { if (visitor->visit(this)) { - for (AstType* generic : generics) - generic->visit(visitor); + for (const AstTypeOrPack& param : parameters) + { + if (param.type) + param.type->visit(visitor); + else + param.typePack->visit(visitor); + } } } @@ -849,6 +858,24 @@ void AstTypeError::visit(AstVisitor* visitor) } } +AstTypePackExplicit::AstTypePackExplicit(const Location& location, AstTypeList typeList) + : AstTypePack(ClassIndex(), location) + , typeList(typeList) +{ +} + +void AstTypePackExplicit::visit(AstVisitor* visitor) +{ + if (visitor->visit(this)) + { + for (AstType* type : typeList.types) + type->visit(visitor); + + if (typeList.tailType) + typeList.tailType->visit(visitor); + } +} + AstTypePackVariadic::AstTypePackVariadic(const Location& location, AstType* variadicType) : AstTypePack(ClassIndex(), location) , variadicType(variadicType) diff --git a/Ast/src/Parser.cpp b/Ast/src/Parser.cpp index 9794a037..846bc0ba 100644 --- a/Ast/src/Parser.cpp +++ b/Ast/src/Parser.cpp @@ -1,6 +1,8 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #include "Luau/Parser.h" +#include "Luau/TimeTrace.h" + #include // Warning: If you are introducing new syntax, ensure that it is behind a separate @@ -13,6 +15,8 @@ LUAU_FASTFLAGVARIABLE(LuauParseGenericFunctions, false) LUAU_FASTFLAGVARIABLE(LuauCaptureBrokenCommentSpans, false) LUAU_FASTFLAGVARIABLE(LuauIfElseExpressionBaseSupport, false) LUAU_FASTFLAGVARIABLE(LuauIfStatementRecursionGuard, false) +LUAU_FASTFLAGVARIABLE(LuauTypeAliasPacks, false) +LUAU_FASTFLAGVARIABLE(LuauParseTypePackTypeParameters, false) namespace Luau { @@ -148,6 +152,8 @@ static bool shouldParseTypePackAnnotation(Lexer& lexer) ParseResult Parser::parse(const char* buffer, size_t bufferSize, AstNameTable& names, Allocator& allocator, ParseOptions options) { + LUAU_TIMETRACE_SCOPE("Parser::parse", "Parser"); + Parser p(buffer, bufferSize, names, allocator); try @@ -769,14 +775,14 @@ AstStat* Parser::parseTypeAlias(const Location& start, bool exported) if (!name) name = Name(nameError, lexer.current().location); - // TODO: support generic type pack parameters in type aliases CLI-39907 auto [generics, genericPacks] = parseGenericTypeList(); expectAndConsume('=', "type alias"); AstType* type = parseTypeAnnotation(); - return allocator.alloc(Location(start, type->location), name->name, generics, type, exported); + return allocator.alloc( + Location(start, type->location), name->name, generics, FFlag::LuauTypeAliasPacks ? genericPacks : AstArray{}, type, exported); } AstDeclaredClassProp Parser::parseDeclaredClassMethod() @@ -1333,7 +1339,7 @@ AstType* Parser::parseTableTypeAnnotation() // ReturnType ::= TypeAnnotation | `(' TypeList `)' // FunctionTypeAnnotation ::= [`<' varlist `>'] `(' [TypeList] `)' `->` ReturnType -AstType* Parser::parseFunctionTypeAnnotation() +AstTypeOrPack Parser::parseFunctionTypeAnnotation(bool allowPack) { incrementRecursionCounter("type annotation"); @@ -1364,14 +1370,23 @@ AstType* Parser::parseFunctionTypeAnnotation() matchRecoveryStopOnToken[Lexeme::SkinnyArrow]--; - // Not a function at all. Just a parenthesized type. - if (params.size() == 1 && !varargAnnotation && monomorphic && lexer.current().type != Lexeme::SkinnyArrow) - return params[0]; - AstArray paramTypes = copy(params); + + // Not a function at all. Just a parenthesized type. Or maybe a type pack with a single element + if (params.size() == 1 && !varargAnnotation && monomorphic && lexer.current().type != Lexeme::SkinnyArrow) + { + if (allowPack) + return {{}, allocator.alloc(begin.location, AstTypeList{paramTypes, nullptr})}; + else + return {params[0], {}}; + } + + if (lexer.current().type != Lexeme::SkinnyArrow && monomorphic && allowPack) + return {{}, allocator.alloc(begin.location, AstTypeList{paramTypes, varargAnnotation})}; + AstArray> paramNames = copy(names); - return parseFunctionTypeAnnotationTail(begin, generics, genericPacks, paramTypes, paramNames, varargAnnotation); + return {parseFunctionTypeAnnotationTail(begin, generics, genericPacks, paramTypes, paramNames, varargAnnotation), {}}; } AstType* Parser::parseFunctionTypeAnnotationTail(const Lexeme& begin, AstArray generics, AstArray genericPacks, @@ -1421,7 +1436,7 @@ AstType* Parser::parseTypeAnnotation(TempVector& parts, const Location if (c == '|') { nextLexeme(); - parts.push_back(parseSimpleTypeAnnotation()); + parts.push_back(parseSimpleTypeAnnotation(false).type); isUnion = true; } else if (c == '?') @@ -1434,7 +1449,7 @@ AstType* Parser::parseTypeAnnotation(TempVector& parts, const Location else if (c == '&') { nextLexeme(); - parts.push_back(parseSimpleTypeAnnotation()); + parts.push_back(parseSimpleTypeAnnotation(false).type); isIntersection = true; } else @@ -1462,6 +1477,30 @@ AstType* Parser::parseTypeAnnotation(TempVector& parts, const Location ParseError::raise(begin, "Composite type was not an intersection or union."); } +AstTypeOrPack Parser::parseTypeOrPackAnnotation() +{ + unsigned int oldRecursionCount = recursionCounter; + incrementRecursionCounter("type annotation"); + + Location begin = lexer.current().location; + + TempVector parts(scratchAnnotation); + + auto [type, typePack] = parseSimpleTypeAnnotation(true); + + if (typePack) + { + LUAU_ASSERT(!type); + return {{}, typePack}; + } + + parts.push_back(type); + + recursionCounter = oldRecursionCount; + + return {parseTypeAnnotation(parts, begin), {}}; +} + AstType* Parser::parseTypeAnnotation() { unsigned int oldRecursionCount = recursionCounter; @@ -1470,7 +1509,7 @@ AstType* Parser::parseTypeAnnotation() Location begin = lexer.current().location; TempVector parts(scratchAnnotation); - parts.push_back(parseSimpleTypeAnnotation()); + parts.push_back(parseSimpleTypeAnnotation(false).type); recursionCounter = oldRecursionCount; @@ -1479,7 +1518,7 @@ AstType* Parser::parseTypeAnnotation() // typeannotation ::= nil | Name[`.' Name] [ `<' typeannotation [`,' ...] `>' ] | `typeof' `(' expr `)' | `{' [PropList] `}' // | [`<' varlist `>'] `(' [TypeList] `)' `->` ReturnType -AstType* Parser::parseSimpleTypeAnnotation() +AstTypeOrPack Parser::parseSimpleTypeAnnotation(bool allowPack) { incrementRecursionCounter("type annotation"); @@ -1488,7 +1527,7 @@ AstType* Parser::parseSimpleTypeAnnotation() if (lexer.current().type == Lexeme::ReservedNil) { nextLexeme(); - return allocator.alloc(begin, std::nullopt, nameNil); + return {allocator.alloc(begin, std::nullopt, nameNil), {}}; } else if (lexer.current().type == Lexeme::Name) { @@ -1514,22 +1553,41 @@ AstType* Parser::parseSimpleTypeAnnotation() expectMatchAndConsume(')', typeofBegin); - return allocator.alloc(Location(begin, end), expr); + return {allocator.alloc(Location(begin, end), expr), {}}; } - AstArray generics = parseTypeParams(); + if (FFlag::LuauParseTypePackTypeParameters) + { + bool hasParameters = false; + AstArray parameters{}; - Location end = lexer.previousLocation(); + if (lexer.current().type == '<') + { + hasParameters = true; + parameters = parseTypeParams(); + } - return allocator.alloc(Location(begin, end), prefix, name.name, generics); + Location end = lexer.previousLocation(); + + return {allocator.alloc(Location(begin, end), prefix, name.name, hasParameters, parameters), {}}; + } + else + { + AstArray generics = parseTypeParams(); + + Location end = lexer.previousLocation(); + + // false in 'hasParameterList' as it is not used without FFlagLuauTypeAliasPacks + return {allocator.alloc(Location(begin, end), prefix, name.name, false, generics), {}}; + } } else if (lexer.current().type == '{') { - return parseTableTypeAnnotation(); + return {parseTableTypeAnnotation(), {}}; } else if (lexer.current().type == '(' || (FFlag::LuauParseGenericFunctions && lexer.current().type == '<')) { - return parseFunctionTypeAnnotation(); + return parseFunctionTypeAnnotation(allowPack); } else { @@ -1538,7 +1596,7 @@ AstType* Parser::parseSimpleTypeAnnotation() // For a missing type annotation, capture 'space' between last token and the next one location = Location(lexer.previousLocation().end, lexer.current().location.begin); - return reportTypeAnnotationError(location, {}, /*isMissing*/ true, "Expected type, got %s", lexer.current().toString().c_str()); + return {reportTypeAnnotationError(location, {}, /*isMissing*/ true, "Expected type, got %s", lexer.current().toString().c_str()), {}}; } } @@ -2312,18 +2370,59 @@ std::pair, AstArray> Parser::parseGenericTypeList() return {generics, genericPacks}; } -AstArray Parser::parseTypeParams() +AstArray Parser::parseTypeParams() { - TempVector result{scratchAnnotation}; + TempVector parameters{scratchTypeOrPackAnnotation}; if (lexer.current().type == '<') { Lexeme begin = lexer.current(); nextLexeme(); + bool seenPack = false; while (true) { - result.push_back(parseTypeAnnotation()); + if (FFlag::LuauParseTypePackTypeParameters) + { + if (shouldParseTypePackAnnotation(lexer)) + { + seenPack = true; + + auto typePack = parseTypePackAnnotation(); + + if (FFlag::LuauTypeAliasPacks) // Type packs are recorded only is we can handle them + parameters.push_back({{}, typePack}); + } + else if (lexer.current().type == '(') + { + auto [type, typePack] = parseTypeOrPackAnnotation(); + + if (typePack) + { + seenPack = true; + + if (FFlag::LuauTypeAliasPacks) // Type packs are recorded only is we can handle them + parameters.push_back({{}, typePack}); + } + else + { + parameters.push_back({type, {}}); + } + } + else if (lexer.current().type == '>' && parameters.empty()) + { + break; + } + else + { + parameters.push_back({parseTypeAnnotation(), {}}); + } + } + else + { + parameters.push_back({parseTypeAnnotation(), {}}); + } + if (lexer.current().type == ',') nextLexeme(); else @@ -2333,7 +2432,7 @@ AstArray Parser::parseTypeParams() expectMatchAndConsume('>', begin); } - return copy(result); + return copy(parameters); } AstExpr* Parser::parseString() diff --git a/Ast/src/TimeTrace.cpp b/Ast/src/TimeTrace.cpp new file mode 100644 index 00000000..e6aab20e --- /dev/null +++ b/Ast/src/TimeTrace.cpp @@ -0,0 +1,248 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +#include "Luau/TimeTrace.h" + +#include "Luau/StringUtils.h" + +#include +#include + +#include + +#ifdef _WIN32 +#include +#endif + +#ifdef __APPLE__ +#include +#include +#endif + +#include + +LUAU_FASTFLAGVARIABLE(DebugLuauTimeTracing, false) + +#if defined(LUAU_ENABLE_TIME_TRACE) + +namespace Luau +{ +namespace TimeTrace +{ +static double getClockPeriod() +{ +#if defined(_WIN32) + LARGE_INTEGER result = {}; + QueryPerformanceFrequency(&result); + return 1.0 / double(result.QuadPart); +#elif defined(__APPLE__) + mach_timebase_info_data_t result = {}; + mach_timebase_info(&result); + return double(result.numer) / double(result.denom) * 1e-9; +#elif defined(__linux__) + return 1e-9; +#else + return 1.0 / double(CLOCKS_PER_SEC); +#endif +} + +static double getClockTimestamp() +{ +#if defined(_WIN32) + LARGE_INTEGER result = {}; + QueryPerformanceCounter(&result); + return double(result.QuadPart); +#elif defined(__APPLE__) + return double(mach_absolute_time()); +#elif defined(__linux__) + timespec now; + clock_gettime(CLOCK_MONOTONIC, &now); + return now.tv_sec * 1e9 + now.tv_nsec; +#else + return double(clock()); +#endif +} + +uint32_t getClockMicroseconds() +{ + static double period = getClockPeriod() * 1e6; + static double start = getClockTimestamp(); + + return uint32_t((getClockTimestamp() - start) * period); +} + +struct GlobalContext +{ + GlobalContext() = default; + ~GlobalContext() + { + // Ideally we would want all ThreadContext destructors to run + // But in VS, not all thread_local object instances are destroyed + for (ThreadContext* context : threads) + context->flushEvents(); + + if (traceFile) + fclose(traceFile); + } + + std::mutex mutex; + std::vector threads; + uint32_t nextThreadId = 0; + std::vector tokens; + FILE* traceFile = nullptr; +}; + +GlobalContext& getGlobalContext() +{ + static GlobalContext context; + return context; +} + +uint16_t createToken(GlobalContext& context, const char* name, const char* category) +{ + std::scoped_lock lock(context.mutex); + + LUAU_ASSERT(context.tokens.size() < 64 * 1024); + + context.tokens.push_back({name, category}); + return uint16_t(context.tokens.size() - 1); +} + +uint32_t createThread(GlobalContext& context, ThreadContext* threadContext) +{ + std::scoped_lock lock(context.mutex); + + context.threads.push_back(threadContext); + + return ++context.nextThreadId; +} + +void releaseThread(GlobalContext& context, ThreadContext* threadContext) +{ + std::scoped_lock lock(context.mutex); + + if (auto it = std::find(context.threads.begin(), context.threads.end(), threadContext); it != context.threads.end()) + context.threads.erase(it); +} + +void flushEvents(GlobalContext& context, uint32_t threadId, const std::vector& events, const std::vector& data) +{ + std::scoped_lock lock(context.mutex); + + if (!context.traceFile) + { + context.traceFile = fopen("trace.json", "w"); + + if (!context.traceFile) + return; + + fprintf(context.traceFile, "[\n"); + } + + std::string temp; + const unsigned tempReserve = 64 * 1024; + temp.reserve(tempReserve); + + const char* rawData = data.data(); + + // Formatting state + bool unfinishedEnter = false; + bool unfinishedArgs = false; + + for (const Event& ev : events) + { + switch (ev.type) + { + case EventType::Enter: + { + if (unfinishedArgs) + { + formatAppend(temp, "}"); + unfinishedArgs = false; + } + + if (unfinishedEnter) + { + formatAppend(temp, "},\n"); + unfinishedEnter = false; + } + + Token& token = context.tokens[ev.token]; + + formatAppend(temp, R"({"name": "%s", "cat": "%s", "ph": "B", "ts": %u, "pid": 0, "tid": %u)", token.name, token.category, + ev.data.microsec, threadId); + unfinishedEnter = true; + } + break; + case EventType::Leave: + if (unfinishedArgs) + { + formatAppend(temp, "}"); + unfinishedArgs = false; + } + if (unfinishedEnter) + { + formatAppend(temp, "},\n"); + unfinishedEnter = false; + } + + formatAppend(temp, + R"({"ph": "E", "ts": %u, "pid": 0, "tid": %u},)" + "\n", + ev.data.microsec, threadId); + break; + case EventType::ArgName: + LUAU_ASSERT(unfinishedEnter); + + if (!unfinishedArgs) + { + formatAppend(temp, R"(, "args": { "%s": )", rawData + ev.data.dataPos); + unfinishedArgs = true; + } + else + { + formatAppend(temp, R"(, "%s": )", rawData + ev.data.dataPos); + } + break; + case EventType::ArgValue: + LUAU_ASSERT(unfinishedArgs); + formatAppend(temp, R"("%s")", rawData + ev.data.dataPos); + break; + } + + // Don't want to hit the string capacity and reallocate + if (temp.size() > tempReserve - 1024) + { + fwrite(temp.data(), 1, temp.size(), context.traceFile); + temp.clear(); + } + } + + if (unfinishedArgs) + { + formatAppend(temp, "}"); + unfinishedArgs = false; + } + if (unfinishedEnter) + { + formatAppend(temp, "},\n"); + unfinishedEnter = false; + } + + fwrite(temp.data(), 1, temp.size(), context.traceFile); + fflush(context.traceFile); +} + +ThreadContext& getThreadContext() +{ + thread_local ThreadContext context; + return context; +} + +std::pair createScopeData(const char* name, const char* category) +{ + uint16_t token = createToken(Luau::TimeTrace::getGlobalContext(), name, category); + return {token, Luau::TimeTrace::getThreadContext()}; +} +} // namespace TimeTrace +} // namespace Luau + +#endif diff --git a/CLI/Analyze.cpp b/CLI/Analyze.cpp index 920502b8..ed0552d7 100644 --- a/CLI/Analyze.cpp +++ b/CLI/Analyze.cpp @@ -111,11 +111,24 @@ struct CliFileResolver : Luau::FileResolver return Luau::SourceCode{*source, Luau::SourceCode::Module}; } + std::optional resolveModule(const Luau::ModuleInfo* context, Luau::AstExpr* node) override + { + if (Luau::AstExprConstantString* expr = node->as()) + { + Luau::ModuleName name = std::string(expr->value.data, expr->value.size) + ".lua"; + + return {{name}}; + } + + return std::nullopt; + } + bool moduleExists(const Luau::ModuleName& name) const override { return !!readFile(name); } + std::optional fromAstFragment(Luau::AstExpr* expr) const override { return std::nullopt; @@ -130,11 +143,6 @@ struct CliFileResolver : Luau::FileResolver { return std::nullopt; } - - std::optional getEnvironmentForModule(const Luau::ModuleName& name) const override - { - return std::nullopt; - } }; struct CliConfigResolver : Luau::ConfigResolver diff --git a/Compiler/src/Compiler.cpp b/Compiler/src/Compiler.cpp index 21564248..7750a1d9 100644 --- a/Compiler/src/Compiler.cpp +++ b/Compiler/src/Compiler.cpp @@ -4,6 +4,7 @@ #include "Luau/Parser.h" #include "Luau/BytecodeBuilder.h" #include "Luau/Common.h" +#include "Luau/TimeTrace.h" #include #include @@ -137,6 +138,11 @@ struct Compiler uint32_t compileFunction(AstExprFunction* func) { + LUAU_TIMETRACE_SCOPE("Compiler::compileFunction", "Compiler"); + + if (func->debugname.value) + LUAU_TIMETRACE_ARGUMENT("name", func->debugname.value); + LUAU_ASSERT(!functions.contains(func)); LUAU_ASSERT(regTop == 0 && stackSize == 0 && localStack.empty() && upvals.empty()); @@ -3686,6 +3692,8 @@ struct Compiler void compileOrThrow(BytecodeBuilder& bytecode, AstStatBlock* root, const AstNameTable& names, const CompileOptions& options) { + LUAU_TIMETRACE_SCOPE("compileOrThrow", "Compiler"); + Compiler compiler(bytecode, options); // since access to some global objects may result in values that change over time, we block table imports @@ -3748,6 +3756,8 @@ void compileOrThrow(BytecodeBuilder& bytecode, const std::string& source, const std::string compile(const std::string& source, const CompileOptions& options, const ParseOptions& parseOptions, BytecodeEncoder* encoder) { + LUAU_TIMETRACE_SCOPE("compile", "Compiler"); + Allocator allocator; AstNameTable names(allocator); ParseResult result = Parser::parse(source.c_str(), source.size(), names, allocator, parseOptions); diff --git a/Sources.cmake b/Sources.cmake index 6f96f6ab..83ed5230 100644 --- a/Sources.cmake +++ b/Sources.cmake @@ -9,6 +9,7 @@ target_sources(Luau.Ast PRIVATE Ast/include/Luau/ParseOptions.h Ast/include/Luau/Parser.h Ast/include/Luau/StringUtils.h + Ast/include/Luau/TimeTrace.h Ast/src/Ast.cpp Ast/src/Confusables.cpp @@ -16,6 +17,7 @@ target_sources(Luau.Ast PRIVATE Ast/src/Location.cpp Ast/src/Parser.cpp Ast/src/StringUtils.cpp + Ast/src/TimeTrace.cpp ) # Luau.Compiler Sources @@ -46,6 +48,7 @@ target_sources(Luau.Analysis PRIVATE Analysis/include/Luau/Predicate.h Analysis/include/Luau/RecursionCounter.h Analysis/include/Luau/RequireTracer.h + Analysis/include/Luau/Scope.h Analysis/include/Luau/Substitution.h Analysis/include/Luau/Symbol.h Analysis/include/Luau/TopoSortStatements.h @@ -75,6 +78,7 @@ target_sources(Luau.Analysis PRIVATE Analysis/src/Module.cpp Analysis/src/Predicate.cpp Analysis/src/RequireTracer.cpp + Analysis/src/Scope.cpp Analysis/src/Substitution.cpp Analysis/src/Symbol.cpp Analysis/src/TopoSortStatements.cpp @@ -188,6 +192,7 @@ if(TARGET Luau.UnitTest) tests/TopoSort.test.cpp tests/ToString.test.cpp tests/Transpiler.test.cpp + tests/TypeInfer.aliases.test.cpp tests/TypeInfer.annotations.test.cpp tests/TypeInfer.builtins.test.cpp tests/TypeInfer.classes.test.cpp diff --git a/VM/src/ldo.cpp b/VM/src/ldo.cpp index 77366714..fb4978d5 100644 --- a/VM/src/ldo.cpp +++ b/VM/src/ldo.cpp @@ -8,9 +8,13 @@ #include "lmem.h" #include "lvm.h" -#include - +#if LUA_USE_LONGJMP #include +#include +#else +#include +#endif + #include LUAU_FASTFLAGVARIABLE(LuauExceptionMessageFix, false) @@ -51,8 +55,8 @@ l_noret luaD_throw(lua_State* L, int errcode) longjmp(jb->buf, 1); } - if (L->global->panic) - L->global->panic(L, errcode); + if (L->global->cb.panic) + L->global->cb.panic(L, errcode); abort(); } diff --git a/VM/src/lgc.cpp b/VM/src/lgc.cpp index 85af403c..5de5d490 100644 --- a/VM/src/lgc.cpp +++ b/VM/src/lgc.cpp @@ -16,6 +16,8 @@ LUAU_FASTFLAGVARIABLE(LuauRescanGrayAgain, false) LUAU_FASTFLAGVARIABLE(LuauRescanGrayAgainForwardBarrier, false) LUAU_FASTFLAGVARIABLE(LuauGcFullSkipInactiveThreads, false) LUAU_FASTFLAGVARIABLE(LuauShrinkWeakTables, false) +LUAU_FASTFLAGVARIABLE(LuauConsolidatedStep, false) + LUAU_FASTFLAG(LuauArrayBoundary) #define GC_SWEEPMAX 40 @@ -810,6 +812,133 @@ static size_t singlestep(lua_State* L) return cost; } +static size_t gcstep(lua_State* L, size_t limit) +{ + size_t cost = 0; + global_State* g = L->global; + switch (g->gcstate) + { + case GCSpause: + { + markroot(L); /* start a new collection */ + break; + } + case GCSpropagate: + { + if (FFlag::LuauRescanGrayAgain) + { + while (g->gray && cost < limit) + { + g->gcstats.currcycle.markitems++; + + cost += propagatemark(g); + } + + if (!g->gray) + { + // perform one iteration over 'gray again' list + g->gray = g->grayagain; + g->grayagain = NULL; + + g->gcstate = GCSpropagateagain; + } + } + else + { + while (g->gray && cost < limit) + { + g->gcstats.currcycle.markitems++; + + cost += propagatemark(g); + } + + if (!g->gray) /* no more `gray' objects */ + { + double starttimestamp = lua_clock(); + + g->gcstats.currcycle.atomicstarttimestamp = starttimestamp; + g->gcstats.currcycle.atomicstarttotalsizebytes = g->totalbytes; + + atomic(L); /* finish mark phase */ + + g->gcstats.currcycle.atomictime += lua_clock() - starttimestamp; + } + } + break; + } + case GCSpropagateagain: + { + while (g->gray && cost < limit) + { + g->gcstats.currcycle.markitems++; + + cost += propagatemark(g); + } + + if (!g->gray) /* no more `gray' objects */ + { + double starttimestamp = lua_clock(); + + g->gcstats.currcycle.atomicstarttimestamp = starttimestamp; + g->gcstats.currcycle.atomicstarttotalsizebytes = g->totalbytes; + + atomic(L); /* finish mark phase */ + + g->gcstats.currcycle.atomictime += lua_clock() - starttimestamp; + } + break; + } + case GCSsweepstring: + { + while (g->sweepstrgc < g->strt.size && cost < limit) + { + size_t traversedcount = 0; + sweepwholelist(L, &g->strt.hash[g->sweepstrgc++], &traversedcount); + + g->gcstats.currcycle.sweepitems += traversedcount; + cost += GC_SWEEPCOST; + } + + // nothing more to sweep? + if (g->sweepstrgc >= g->strt.size) + { + // sweep string buffer list and preserve used string count + uint32_t nuse = L->global->strt.nuse; + + size_t traversedcount = 0; + sweepwholelist(L, &g->strbufgc, &traversedcount); + + L->global->strt.nuse = nuse; + + g->gcstats.currcycle.sweepitems += traversedcount; + g->gcstate = GCSsweep; // end sweep-string phase + } + break; + } + case GCSsweep: + { + while (*g->sweepgc && cost < limit) + { + size_t traversedcount = 0; + g->sweepgc = sweeplist(L, g->sweepgc, GC_SWEEPMAX, &traversedcount); + + g->gcstats.currcycle.sweepitems += traversedcount; + cost += GC_SWEEPMAX * GC_SWEEPCOST; + } + + if (*g->sweepgc == NULL) + { /* nothing more to sweep? */ + shrinkbuffers(L); + g->gcstate = GCSpause; /* end collection */ + } + break; + } + default: + LUAU_ASSERT(0); + } + return cost; +} + static int64_t getheaptriggererroroffset(GCHeapTriggerStats* triggerstats, GCCycleStats* cyclestats) { // adjust for error using Proportional-Integral controller @@ -878,33 +1007,40 @@ void luaC_step(lua_State* L, bool assist) if (g->gcstate == GCSpause) startGcCycleStats(g); - if (assist) - g->gcstats.currcycle.assistwork += lim; - else - g->gcstats.currcycle.explicitwork += lim; - int lastgcstate = g->gcstate; double lasttimestamp = lua_clock(); - // always perform at least one single step - do + if (FFlag::LuauConsolidatedStep) { - lim -= singlestep(L); + size_t work = gcstep(L, lim); - // if we have switched to a different state, capture the duration of last stage - // this way we reduce the number of timer calls we make - if (lastgcstate != g->gcstate) + if (assist) + g->gcstats.currcycle.assistwork += work; + else + g->gcstats.currcycle.explicitwork += work; + } + else + { + // always perform at least one single step + do { - GC_INTERRUPT(lastgcstate); + lim -= singlestep(L); - double now = lua_clock(); + // if we have switched to a different state, capture the duration of last stage + // this way we reduce the number of timer calls we make + if (lastgcstate != g->gcstate) + { + GC_INTERRUPT(lastgcstate); - recordGcStateTime(g, lastgcstate, now - lasttimestamp, assist); + double now = lua_clock(); - lasttimestamp = now; - lastgcstate = g->gcstate; - } - } while (lim > 0 && g->gcstate != GCSpause); + recordGcStateTime(g, lastgcstate, now - lasttimestamp, assist); + + lasttimestamp = now; + lastgcstate = g->gcstate; + } + } while (lim > 0 && g->gcstate != GCSpause); + } recordGcStateTime(g, lastgcstate, lua_clock() - lasttimestamp, assist); @@ -931,7 +1067,14 @@ void luaC_step(lua_State* L, bool assist) g->GCthreshold -= debt; } - GC_INTERRUPT(g->gcstate); + if (FFlag::LuauConsolidatedStep) + { + GC_INTERRUPT(lastgcstate); + } + else + { + GC_INTERRUPT(g->gcstate); + } } void luaC_fullgc(lua_State* L) @@ -957,7 +1100,10 @@ void luaC_fullgc(lua_State* L) while (g->gcstate != GCSpause) { LUAU_ASSERT(g->gcstate == GCSsweepstring || g->gcstate == GCSsweep); - singlestep(L); + if (FFlag::LuauConsolidatedStep) + gcstep(L, SIZE_MAX); + else + singlestep(L); } finishGcCycleStats(g); @@ -968,7 +1114,10 @@ void luaC_fullgc(lua_State* L) markroot(L); while (g->gcstate != GCSpause) { - singlestep(L); + if (FFlag::LuauConsolidatedStep) + gcstep(L, SIZE_MAX); + else + singlestep(L); } /* reclaim as much buffer memory as possible (shrinkbuffers() called during sweep is incremental) */ shrinkbuffersfull(L); diff --git a/VM/src/ltablib.cpp b/VM/src/ltablib.cpp index 090e183f..de5788eb 100644 --- a/VM/src/ltablib.cpp +++ b/VM/src/ltablib.cpp @@ -9,14 +9,8 @@ #include "ldebug.h" #include "lvm.h" -LUAU_DYNAMIC_FASTFLAGVARIABLE(LuauTableMoveTelemetry, false) - LUAU_FASTFLAGVARIABLE(LuauTableFreeze, false) -bool lua_telemetry_table_move_oob_src_from = false; -bool lua_telemetry_table_move_oob_src_to = false; -bool lua_telemetry_table_move_oob_dst = false; - static int foreachi(lua_State* L) { luaL_checktype(L, 1, LUA_TTABLE); @@ -202,22 +196,6 @@ static int tmove(lua_State* L) int tt = !lua_isnoneornil(L, 5) ? 5 : 1; /* destination table */ luaL_checktype(L, tt, LUA_TTABLE); - if (DFFlag::LuauTableMoveTelemetry) - { - int nf = lua_objlen(L, 1); - int nt = lua_objlen(L, tt); - - // source index range must be in bounds in source table unless the table is empty (permits 1..#t moves) - if (!(f == 1 || (f >= 1 && f <= nf))) - lua_telemetry_table_move_oob_src_from = true; - if (!(e == nf || (e >= 1 && e <= nf))) - lua_telemetry_table_move_oob_src_to = true; - - // destination index must be in bounds in dest table or be exactly at the first empty element (permits concats) - if (!(t == nt + 1 || (t >= 1 && t <= nt + 1))) - lua_telemetry_table_move_oob_dst = true; - } - if (e >= f) { /* otherwise, nothing to move */ luaL_argcheck(L, f > 0 || e < INT_MAX + f, 3, "too many elements to move"); diff --git a/VM/src/lvmexecute.cpp b/VM/src/lvmexecute.cpp index 5f0ee922..eed2862b 100644 --- a/VM/src/lvmexecute.cpp +++ b/VM/src/lvmexecute.cpp @@ -16,8 +16,6 @@ #include -LUAU_FASTFLAGVARIABLE(LuauLoopUseSafeenv, false) - // Disable c99-designator to avoid the warning in CGOTO dispatch table #ifdef __clang__ #if __has_warning("-Wc99-designator") @@ -292,10 +290,6 @@ inline bool luau_skipstep(uint8_t op) return op == LOP_PREPVARARGS || op == LOP_BREAK; } -// declared in lbaselib.cpp, needed to support cases when pairs/ipairs have been replaced via setfenv -LUAI_FUNC int luaB_inext(lua_State* L); -LUAI_FUNC int luaB_next(lua_State* L); - template static void luau_execute(lua_State* L) { @@ -2223,8 +2217,7 @@ static void luau_execute(lua_State* L) StkId ra = VM_REG(LUAU_INSN_A(insn)); // fast-path: ipairs/inext - bool safeenv = FFlag::LuauLoopUseSafeenv ? cl->env->safeenv : ttisfunction(ra) && clvalue(ra)->isC && clvalue(ra)->c.f == luaB_inext; - if (safeenv && ttistable(ra + 1) && ttisnumber(ra + 2) && nvalue(ra + 2) == 0.0) + if (cl->env->safeenv && ttistable(ra + 1) && ttisnumber(ra + 2) && nvalue(ra + 2) == 0.0) { setpvalue(ra + 2, reinterpret_cast(uintptr_t(0))); } @@ -2304,8 +2297,7 @@ static void luau_execute(lua_State* L) StkId ra = VM_REG(LUAU_INSN_A(insn)); // fast-path: pairs/next - bool safeenv = FFlag::LuauLoopUseSafeenv ? cl->env->safeenv : ttisfunction(ra) && clvalue(ra)->isC && clvalue(ra)->c.f == luaB_next; - if (safeenv && ttistable(ra + 1) && ttisnil(ra + 2)) + if (cl->env->safeenv && ttistable(ra + 1) && ttisnil(ra + 2)) { setpvalue(ra + 2, reinterpret_cast(uintptr_t(0))); } diff --git a/VM/src/lvmload.cpp b/VM/src/lvmload.cpp index 0a232342..b932a85b 100644 --- a/VM/src/lvmload.cpp +++ b/VM/src/lvmload.cpp @@ -12,7 +12,32 @@ #include -#include +// TODO: RAII deallocation doesn't work for longjmp builds if a memory error happens +template +struct TempBuffer +{ + lua_State* L; + T* data; + size_t count; + + TempBuffer(lua_State* L, size_t count) + : L(L) + , data(luaM_newarray(L, count, T, 0)) + , count(count) + { + } + + ~TempBuffer() + { + luaM_freearray(L, data, count, T, 0); + } + + T& operator[](size_t index) + { + LUAU_ASSERT(index < count); + return data[index]; + } +}; void luaV_getimport(lua_State* L, Table* env, TValue* k, uint32_t id, bool propagatenil) { @@ -67,7 +92,7 @@ static unsigned int readVarInt(const char* data, size_t size, size_t& offset) return result; } -static TString* readString(std::vector& strings, const char* data, size_t size, size_t& offset) +static TString* readString(TempBuffer& strings, const char* data, size_t size, size_t& offset) { unsigned int id = readVarInt(data, size, offset); @@ -133,6 +158,7 @@ int luau_load(lua_State* L, const char* chunkname, const char* data, size_t size } // pause GC for the duration of deserialization - some objects we're creating aren't rooted + // TODO: if an allocation error happens mid-load, we do not unpause GC! size_t GCthreshold = L->global->GCthreshold; L->global->GCthreshold = SIZE_MAX; @@ -144,7 +170,7 @@ int luau_load(lua_State* L, const char* chunkname, const char* data, size_t size // string table unsigned int stringCount = readVarInt(data, size, offset); - std::vector strings(stringCount); + TempBuffer strings(L, stringCount); for (unsigned int i = 0; i < stringCount; ++i) { @@ -156,7 +182,7 @@ int luau_load(lua_State* L, const char* chunkname, const char* data, size_t size // proto table unsigned int protoCount = readVarInt(data, size, offset); - std::vector protos(protoCount); + TempBuffer protos(L, protoCount); for (unsigned int i = 0; i < protoCount; ++i) { diff --git a/bench/tests/deltablue.lua b/bench/tests/deltablue.lua deleted file mode 100644 index ad18c233..00000000 --- a/bench/tests/deltablue.lua +++ /dev/null @@ -1,934 +0,0 @@ -local bench = script and require(script.Parent.bench_support) or require("bench_support") - --- Copyright 2008 the V8 project authors. All rights reserved. --- Copyright 1996 John Maloney and Mario Wolczko. - --- This program is free software; you can redistribute it and/or modify --- it under the terms of the GNU General Public License as published by --- the Free Software Foundation; either version 2 of the License, or --- (at your option) any later version. --- --- This program is distributed in the hope that it will be useful, --- but WITHOUT ANY WARRANTY; without even the implied warranty of --- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the --- GNU General Public License for more details. --- --- You should have received a copy of the GNU General Public License --- along with this program; if not, write to the Free Software --- Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA - - --- This implementation of the DeltaBlue benchmark is derived --- from the Smalltalk implementation by John Maloney and Mario --- Wolczko. Some parts have been translated directly, whereas --- others have been modified more aggressively to make it feel --- more like a JavaScript program. - - --- --- A JavaScript implementation of the DeltaBlue constraint-solving --- algorithm, as described in: --- --- "The DeltaBlue Algorithm: An Incremental Constraint Hierarchy Solver" --- Bjorn N. Freeman-Benson and John Maloney --- January 1990 Communications of the ACM, --- also available as University of Washington TR 89-08-06. --- --- Beware: this benchmark is written in a grotesque style where --- the constraint model is built by side-effects from constructors. --- I've kept it this way to avoid deviating too much from the original --- implementation. --- - -function class(base) - local T = {} - T.__index = T - - if base then - T.super = base - setmetatable(T, base) - end - - function T.new(...) - local O = {} - setmetatable(O, T) - O:constructor(...) - return O - end - - return T -end - -local planner - ---- O b j e c t M o d e l --- - -local function alert (...) print(...) end - -local OrderedCollection = class() - -function OrderedCollection:constructor() - self.elms = {} -end - -function OrderedCollection:add(elm) - self.elms[#self.elms + 1] = elm -end - -function OrderedCollection:at (index) - return self.elms[index] -end - -function OrderedCollection:size () - return #self.elms -end - -function OrderedCollection:removeFirst () - local e = self.elms[#self.elms] - self.elms[#self.elms] = nil - return e -end - -function OrderedCollection:remove (elm) - local index = 0 - local skipped = 0 - - for i = 1, #self.elms do - local value = self.elms[i] - if value ~= elm then - self.elms[index] = value - index = index + 1 - else - skipped = skipped + 1 - end - end - - local l = #self.elms - for i = 1, skipped do self.elms[l - i + 1] = nil end -end - --- --- S t r e n g t h --- - --- --- Strengths are used to measure the relative importance of constraints. --- New strengths may be inserted in the strength hierarchy without --- disrupting current constraints. Strengths cannot be created outside --- this class, so pointer comparison can be used for value comparison. --- - -local Strength = class() - -function Strength:constructor(strengthValue, name) - self.strengthValue = strengthValue - self.name = name -end - -function Strength.stronger (s1, s2) - return s1.strengthValue < s2.strengthValue -end - -function Strength.weaker (s1, s2) - return s1.strengthValue > s2.strengthValue -end - -function Strength.weakestOf (s1, s2) - return Strength.weaker(s1, s2) and s1 or s2 -end - -function Strength.strongest (s1, s2) - return Strength.stronger(s1, s2) and s1 or s2 -end - -function Strength:nextWeaker () - local v = self.strengthValue - if v == 0 then return Strength.WEAKEST - elseif v == 1 then return Strength.WEAK_DEFAULT - elseif v == 2 then return Strength.NORMAL - elseif v == 3 then return Strength.STRONG_DEFAULT - elseif v == 4 then return Strength.PREFERRED - elseif v == 5 then return Strength.REQUIRED - end -end - --- Strength constants. -Strength.REQUIRED = Strength.new(0, "required"); -Strength.STONG_PREFERRED = Strength.new(1, "strongPreferred"); -Strength.PREFERRED = Strength.new(2, "preferred"); -Strength.STRONG_DEFAULT = Strength.new(3, "strongDefault"); -Strength.NORMAL = Strength.new(4, "normal"); -Strength.WEAK_DEFAULT = Strength.new(5, "weakDefault"); -Strength.WEAKEST = Strength.new(6, "weakest"); - --- --- C o n s t r a i n t --- - --- --- An abstract class representing a system-maintainable relationship --- (or "constraint") between a set of variables. A constraint supplies --- a strength instance variable; concrete subclasses provide a means --- of storing the constrained variables and other information required --- to represent a constraint. --- - -local Constraint = class () - -function Constraint:constructor(strength) - self.strength = strength -end - --- --- Activate this constraint and attempt to satisfy it. --- -function Constraint:addConstraint () - self:addToGraph() - planner:incrementalAdd(self) -end - --- --- Attempt to find a way to enforce this constraint. If successful, --- record the solution, perhaps modifying the current dataflow --- graph. Answer the constraint that this constraint overrides, if --- there is one, or nil, if there isn't. --- Assume: I am not already satisfied. --- -function Constraint:satisfy (mark) - self:chooseMethod(mark) - if not self:isSatisfied() then - if self.strength == Strength.REQUIRED then - alert("Could not satisfy a required constraint!") - end - return nil - end - self:markInputs(mark) - local out = self:output() - local overridden = out.determinedBy - if overridden ~= nil then overridden:markUnsatisfied() end - out.determinedBy = self - if not planner:addPropagate(self, mark) then alert("Cycle encountered") end - out.mark = mark - return overridden -end - -function Constraint:destroyConstraint () - if self:isSatisfied() - then planner:incrementalRemove(self) - else self:removeFromGraph() - end -end - --- --- Normal constraints are not input constraints. An input constraint --- is one that depends on external state, such as the mouse, the --- keyboard, a clock, or some arbitrary piece of imperative code. --- -function Constraint:isInput () - return false -end - - --- --- U n a r y C o n s t r a i n t --- - --- --- Abstract superclass for constraints having a single possible output --- variable. --- - -local UnaryConstraint = class(Constraint) - -function UnaryConstraint:constructor (v, strength) - UnaryConstraint.super.constructor(self, strength) - self.myOutput = v - self.satisfied = false - self:addConstraint() -end - --- --- Adds this constraint to the constraint graph --- -function UnaryConstraint:addToGraph () - self.myOutput:addConstraint(self) - self.satisfied = false -end - --- --- Decides if this constraint can be satisfied and records that --- decision. --- -function UnaryConstraint:chooseMethod (mark) - self.satisfied = (self.myOutput.mark ~= mark) - and Strength.stronger(self.strength, self.myOutput.walkStrength); -end - --- --- Returns true if this constraint is satisfied in the current solution. --- -function UnaryConstraint:isSatisfied () - return self.satisfied; -end - -function UnaryConstraint:markInputs (mark) - -- has no inputs -end - --- --- Returns the current output variable. --- -function UnaryConstraint:output () - return self.myOutput -end - --- --- Calculate the walkabout strength, the stay flag, and, if it is --- 'stay', the value for the current output of this constraint. Assume --- this constraint is satisfied. --- -function UnaryConstraint:recalculate () - self.myOutput.walkStrength = self.strength - self.myOutput.stay = not self:isInput() - if self.myOutput.stay then - self:execute() -- Stay optimization - end -end - --- --- Records that this constraint is unsatisfied --- -function UnaryConstraint:markUnsatisfied () - self.satisfied = false -end - -function UnaryConstraint:inputsKnown () - return true -end - -function UnaryConstraint:removeFromGraph () - if self.myOutput ~= nil then - self.myOutput:removeConstraint(self) - end - self.satisfied = false -end - --- --- S t a y C o n s t r a i n t --- - --- --- Variables that should, with some level of preference, stay the same. --- Planners may exploit the fact that instances, if satisfied, will not --- change their output during plan execution. This is called "stay --- optimization". --- - -local StayConstraint = class(UnaryConstraint) - -function StayConstraint:constructor(v, str) - StayConstraint.super.constructor(self, v, str) -end - -function StayConstraint:execute () - -- Stay constraints do nothing -end - --- --- E d i t C o n s t r a i n t --- - --- --- A unary input constraint used to mark a variable that the client --- wishes to change. --- - -local EditConstraint = class (UnaryConstraint) - -function EditConstraint:constructor(v, str) - EditConstraint.super.constructor(self, v, str) -end - --- --- Edits indicate that a variable is to be changed by imperative code. --- -function EditConstraint:isInput () - return true -end - -function EditConstraint:execute () - -- Edit constraints do nothing -end - --- --- B i n a r y C o n s t r a i n t --- - -local Direction = {} -Direction.NONE = 0 -Direction.FORWARD = 1 -Direction.BACKWARD = -1 - --- --- Abstract superclass for constraints having two possible output --- variables. --- - -local BinaryConstraint = class(Constraint) - -function BinaryConstraint:constructor(var1, var2, strength) - BinaryConstraint.super.constructor(self, strength); - self.v1 = var1 - self.v2 = var2 - self.direction = Direction.NONE - self:addConstraint() -end - - --- --- Decides if this constraint can be satisfied and which way it --- should flow based on the relative strength of the variables related, --- and record that decision. --- -function BinaryConstraint:chooseMethod (mark) - if self.v1.mark == mark then - self.direction = (self.v2.mark ~= mark and Strength.stronger(self.strength, self.v2.walkStrength)) and Direction.FORWARD or Direction.NONE - end - if self.v2.mark == mark then - self.direction = (self.v1.mark ~= mark and Strength.stronger(self.strength, self.v1.walkStrength)) and Direction.BACKWARD or Direction.NONE - end - if Strength.weaker(self.v1.walkStrength, self.v2.walkStrength) then - self.direction = Strength.stronger(self.strength, self.v1.walkStrength) and Direction.BACKWARD or Direction.NONE - else - self.direction = Strength.stronger(self.strength, self.v2.walkStrength) and Direction.FORWARD or Direction.BACKWARD - end -end - --- --- Add this constraint to the constraint graph --- -function BinaryConstraint:addToGraph () - self.v1:addConstraint(self) - self.v2:addConstraint(self) - self.direction = Direction.NONE -end - --- --- Answer true if this constraint is satisfied in the current solution. --- -function BinaryConstraint:isSatisfied () - return self.direction ~= Direction.NONE -end - --- --- Mark the input variable with the given mark. --- -function BinaryConstraint:markInputs (mark) - self:input().mark = mark -end - --- --- Returns the current input variable --- -function BinaryConstraint:input () - return (self.direction == Direction.FORWARD) and self.v1 or self.v2 -end - --- --- Returns the current output variable --- -function BinaryConstraint:output () - return (self.direction == Direction.FORWARD) and self.v2 or self.v1 -end - --- --- Calculate the walkabout strength, the stay flag, and, if it is --- 'stay', the value for the current output of this --- constraint. Assume this constraint is satisfied. --- -function BinaryConstraint:recalculate () - local ihn = self:input() - local out = self:output() - out.walkStrength = Strength.weakestOf(self.strength, ihn.walkStrength); - out.stay = ihn.stay - if out.stay then self:execute() end -end - --- --- Record the fact that self constraint is unsatisfied. --- -function BinaryConstraint:markUnsatisfied () - self.direction = Direction.NONE -end - -function BinaryConstraint:inputsKnown (mark) - local i = self:input() - return i.mark == mark or i.stay or i.determinedBy == nil -end - -function BinaryConstraint:removeFromGraph () - if (self.v1 ~= nil) then self.v1:removeConstraint(self) end - if (self.v2 ~= nil) then self.v2:removeConstraint(self) end - self.direction = Direction.NONE -end - --- --- S c a l e C o n s t r a i n t --- - --- --- Relates two variables by the linear scaling relationship: "v2 = --- (v1 * scale) + offset". Either v1 or v2 may be changed to maintain --- this relationship but the scale factor and offset are considered --- read-only. --- - -local ScaleConstraint = class (BinaryConstraint) - -function ScaleConstraint:constructor(src, scale, offset, dest, strength) - self.direction = Direction.NONE - self.scale = scale - self.offset = offset - ScaleConstraint.super.constructor(self, src, dest, strength) -end - - --- --- Adds this constraint to the constraint graph. --- -function ScaleConstraint:addToGraph () - ScaleConstraint.super.addToGraph(self) - self.scale:addConstraint(self) - self.offset:addConstraint(self) -end - -function ScaleConstraint:removeFromGraph () - ScaleConstraint.super.removeFromGraph(self) - if (self.scale ~= nil) then self.scale:removeConstraint(self) end - if (self.offset ~= nil) then self.offset:removeConstraint(self) end -end - -function ScaleConstraint:markInputs (mark) - ScaleConstraint.super.markInputs(self, mark); - self.offset.mark = mark - self.scale.mark = mark -end - --- --- Enforce this constraint. Assume that it is satisfied. --- -function ScaleConstraint:execute () - if self.direction == Direction.FORWARD then - self.v2.value = self.v1.value * self.scale.value + self.offset.value - else - self.v1.value = (self.v2.value - self.offset.value) / self.scale.value - end -end - --- --- Calculate the walkabout strength, the stay flag, and, if it is --- 'stay', the value for the current output of this constraint. Assume --- this constraint is satisfied. --- -function ScaleConstraint:recalculate () - local ihn = self:input() - local out = self:output() - out.walkStrength = Strength.weakestOf(self.strength, ihn.walkStrength) - out.stay = ihn.stay and self.scale.stay and self.offset.stay - if out.stay then self:execute() end -end - --- --- E q u a l i t y C o n s t r a i n t --- - --- --- Constrains two variables to have the same value. --- - -local EqualityConstraint = class (BinaryConstraint) - -function EqualityConstraint:constructor(var1, var2, strength) - EqualityConstraint.super.constructor(self, var1, var2, strength) -end - - --- --- Enforce this constraint. Assume that it is satisfied. --- -function EqualityConstraint:execute () - self:output().value = self:input().value -end - --- --- V a r i a b l e --- - --- --- A constrained variable. In addition to its value, it maintain the --- structure of the constraint graph, the current dataflow graph, and --- various parameters of interest to the DeltaBlue incremental --- constraint solver. --- -local Variable = class () - -function Variable:constructor(name, initialValue) - self.value = initialValue or 0 - self.constraints = OrderedCollection.new() - self.determinedBy = nil - self.mark = 0 - self.walkStrength = Strength.WEAKEST - self.stay = true - self.name = name -end - --- --- Add the given constraint to the set of all constraints that refer --- this variable. --- -function Variable:addConstraint (c) - self.constraints:add(c) -end - --- --- Removes all traces of c from this variable. --- -function Variable:removeConstraint (c) - self.constraints:remove(c) - if self.determinedBy == c then - self.determinedBy = nil - end -end - --- --- P l a n n e r --- - --- --- The DeltaBlue planner --- -local Planner = class() -function Planner:constructor() - self.currentMark = 0 -end - --- --- Attempt to satisfy the given constraint and, if successful, --- incrementally update the dataflow graph. Details: If satisfying --- the constraint is successful, it may override a weaker constraint --- on its output. The algorithm attempts to resatisfy that --- constraint using some other method. This process is repeated --- until either a) it reaches a variable that was not previously --- determined by any constraint or b) it reaches a constraint that --- is too weak to be satisfied using any of its methods. The --- variables of constraints that have been processed are marked with --- a unique mark value so that we know where we've been. This allows --- the algorithm to avoid getting into an infinite loop even if the --- constraint graph has an inadvertent cycle. --- -function Planner:incrementalAdd (c) - local mark = self:newMark() - local overridden = c:satisfy(mark) - while overridden ~= nil do - overridden = overridden:satisfy(mark) - end -end - --- --- Entry point for retracting a constraint. Remove the given --- constraint and incrementally update the dataflow graph. --- Details: Retracting the given constraint may allow some currently --- unsatisfiable downstream constraint to be satisfied. We therefore collect --- a list of unsatisfied downstream constraints and attempt to --- satisfy each one in turn. This list is traversed by constraint --- strength, strongest first, as a heuristic for avoiding --- unnecessarily adding and then overriding weak constraints. --- Assume: c is satisfied. --- -function Planner:incrementalRemove (c) - local out = c:output() - c:markUnsatisfied() - c:removeFromGraph() - local unsatisfied = self:removePropagateFrom(out) - local strength = Strength.REQUIRED - repeat - for i = 1, unsatisfied:size() do - local u = unsatisfied:at(i) - if u.strength == strength then - self:incrementalAdd(u) - end - end - strength = strength:nextWeaker() - until strength == Strength.WEAKEST -end - --- --- Select a previously unused mark value. --- -function Planner:newMark () - self.currentMark = self.currentMark + 1 - return self.currentMark -end - --- --- Extract a plan for resatisfaction starting from the given source --- constraints, usually a set of input constraints. This method --- assumes that stay optimization is desired; the plan will contain --- only constraints whose output variables are not stay. Constraints --- that do no computation, such as stay and edit constraints, are --- not included in the plan. --- Details: The outputs of a constraint are marked when it is added --- to the plan under construction. A constraint may be appended to --- the plan when all its input variables are known. A variable is --- known if either a) the variable is marked (indicating that has --- been computed by a constraint appearing earlier in the plan), b) --- the variable is 'stay' (i.e. it is a constant at plan execution --- time), or c) the variable is not determined by any --- constraint. The last provision is for past states of history --- variables, which are not stay but which are also not computed by --- any constraint. --- Assume: sources are all satisfied. --- -local Plan -- FORWARD DECLARATION -function Planner:makePlan (sources) - local mark = self:newMark() - local plan = Plan.new() - local todo = sources - while todo:size() > 0 do - local c = todo:removeFirst() - if c:output().mark ~= mark and c:inputsKnown(mark) then - plan:addConstraint(c) - c:output().mark = mark - self:addConstraintsConsumingTo(c:output(), todo) - end - end - return plan -end - --- --- Extract a plan for resatisfying starting from the output of the --- given constraints, usually a set of input constraints. --- -function Planner:extractPlanFromConstraints (constraints) - local sources = OrderedCollection.new() - for i = 1, constraints:size() do - local c = constraints:at(i) - if c:isInput() and c:isSatisfied() then - -- not in plan already and eligible for inclusion - sources:add(c) - end - end - return self:makePlan(sources) -end - --- --- Recompute the walkabout strengths and stay flags of all variables --- downstream of the given constraint and recompute the actual --- values of all variables whose stay flag is true. If a cycle is --- detected, remove the given constraint and answer --- false. Otherwise, answer true. --- Details: Cycles are detected when a marked variable is --- encountered downstream of the given constraint. The sender is --- assumed to have marked the inputs of the given constraint with --- the given mark. Thus, encountering a marked node downstream of --- the output constraint means that there is a path from the --- constraint's output to one of its inputs. --- -function Planner:addPropagate (c, mark) - local todo = OrderedCollection.new() - todo:add(c) - while todo:size() > 0 do - local d = todo:removeFirst() - if d:output().mark == mark then - self:incrementalRemove(c) - return false - end - d:recalculate() - self:addConstraintsConsumingTo(d:output(), todo) - end - return true -end - - --- --- Update the walkabout strengths and stay flags of all variables --- downstream of the given constraint. Answer a collection of --- unsatisfied constraints sorted in order of decreasing strength. --- -function Planner:removePropagateFrom (out) - out.determinedBy = nil - out.walkStrength = Strength.WEAKEST - out.stay = true - local unsatisfied = OrderedCollection.new() - local todo = OrderedCollection.new() - todo:add(out) - while todo:size() > 0 do - local v = todo:removeFirst() - for i = 1, v.constraints:size() do - local c = v.constraints:at(i) - if not c:isSatisfied() then unsatisfied:add(c) end - end - local determining = v.determinedBy - for i = 1, v.constraints:size() do - local next = v.constraints:at(i); - if next ~= determining and next:isSatisfied() then - next:recalculate() - todo:add(next:output()) - end - end - end - return unsatisfied -end - -function Planner:addConstraintsConsumingTo (v, coll) - local determining = v.determinedBy - local cc = v.constraints - for i = 1, cc:size() do - local c = cc:at(i) - if c ~= determining and c:isSatisfied() then - coll:add(c) - end - end -end - --- --- P l a n --- - --- --- A Plan is an ordered list of constraints to be executed in sequence --- to resatisfy all currently satisfiable constraints in the face of --- one or more changing inputs. --- -Plan = class() -function Plan:constructor() - self.v = OrderedCollection.new() -end - -function Plan:addConstraint (c) - self.v:add(c) -end - -function Plan:size () - return self.v:size() -end - -function Plan:constraintAt (index) - return self.v:at(index) -end - -function Plan:execute () - for i = 1, self:size() do - local c = self:constraintAt(i) - c:execute() - end -end - --- --- M a i n --- - --- --- This is the standard DeltaBlue benchmark. A long chain of equality --- constraints is constructed with a stay constraint on one end. An --- edit constraint is then added to the opposite end and the time is --- measured for adding and removing this constraint, and extracting --- and executing a constraint satisfaction plan. There are two cases. --- In case 1, the added constraint is stronger than the stay --- constraint and values must propagate down the entire length of the --- chain. In case 2, the added constraint is weaker than the stay --- constraint so it cannot be accommodated. The cost in this case is, --- of course, very low. Typical situations lie somewhere between these --- two extremes. --- -local function chainTest(n) - planner = Planner.new() - local prev = nil - local first = nil - local last = nil - - -- Build chain of n equality constraints - for i = 0, n do - local name = "v" .. i; - local v = Variable.new(name) - if prev ~= nil then EqualityConstraint.new(prev, v, Strength.REQUIRED) end - if i == 0 then first = v end - if i == n then last = v end - prev = v - end - - StayConstraint.new(last, Strength.STRONG_DEFAULT) - local edit = EditConstraint.new(first, Strength.PREFERRED) - local edits = OrderedCollection.new() - edits:add(edit) - local plan = planner:extractPlanFromConstraints(edits) - for i = 0, 99 do - first.value = i - plan:execute() - if last.value ~= i then - alert("Chain test failed.") - end - end -end - -local function change(v, newValue) - local edit = EditConstraint.new(v, Strength.PREFERRED) - local edits = OrderedCollection.new() - edits:add(edit) - local plan = planner:extractPlanFromConstraints(edits) - for i = 1, 10 do - v.value = newValue - plan:execute() - end - edit:destroyConstraint() -end - --- --- This test constructs a two sets of variables related to each --- other by a simple linear transformation (scale and offset). The --- time is measured to change a variable on either side of the --- mapping and to change the scale and offset factors. --- -local function projectionTest(n) - planner = Planner.new(); - local scale = Variable.new("scale", 10); - local offset = Variable.new("offset", 1000); - local src = nil - local dst = nil; - - local dests = OrderedCollection.new(); - for i = 0, n - 1 do - src = Variable.new("src" .. i, i); - dst = Variable.new("dst" .. i, i); - dests:add(dst); - StayConstraint.new(src, Strength.NORMAL); - ScaleConstraint.new(src, scale, offset, dst, Strength.REQUIRED); - end - - change(src, 17) - if dst.value ~= 1170 then alert("Projection 1 failed") end - change(dst, 1050) - if src.value ~= 5 then alert("Projection 2 failed") end - change(scale, 5) - for i = 0, n - 2 do - if dests:at(i + 1).value ~= i * 5 + 1000 then - alert("Projection 3 failed") - end - end - change(offset, 2000) - for i = 0, n - 2 do - if dests:at(i + 1).value ~= i * 5 + 2000 then - alert("Projection 4 failed") - end - end -end - -function test() - local t0 = os.clock() - chainTest(1000); - projectionTest(1000); - local t1 = os.clock() - return t1-t0 -end - -bench.runCode(test, "deltablue") diff --git a/tests/Autocomplete.test.cpp b/tests/Autocomplete.test.cpp index 9cd642cf..07910a0a 100644 --- a/tests/Autocomplete.test.cpp +++ b/tests/Autocomplete.test.cpp @@ -23,19 +23,17 @@ static std::optional nullCallback(std::string tag, std::op return std::nullopt; } -struct ACFixture : Fixture +template +struct ACFixtureImpl : BaseType { AutocompleteResult autocomplete(unsigned row, unsigned column) { - return Luau::autocomplete(frontend, "MainModule", Position{row, column}, nullCallback); + return Luau::autocomplete(this->frontend, "MainModule", Position{row, column}, nullCallback); } AutocompleteResult autocomplete(char marker) { - auto i = markerPosition.find(marker); - LUAU_ASSERT(i != markerPosition.end()); - const Position& pos = i->second; - return Luau::autocomplete(frontend, "MainModule", pos, nullCallback); + return Luau::autocomplete(this->frontend, "MainModule", getPosition(marker), nullCallback); } CheckResult check(const std::string& source) @@ -45,16 +43,18 @@ struct ACFixture : Fixture filteredSource.reserve(source.size()); Position curPos(0, 0); + char prevChar{}; for (char c : source) { - if (c == '@' && !filteredSource.empty()) + if (prevChar == '@') { - char prevChar = filteredSource.back(); - filteredSource.pop_back(); - curPos.column--; // Adjust column position since we removed a character from the output - LUAU_ASSERT("Illegal marker character" && prevChar >= '0' && prevChar <= '9'); - LUAU_ASSERT("Duplicate marker found" && markerPosition.count(prevChar) == 0); - markerPosition.insert(std::pair{prevChar, curPos}); + LUAU_ASSERT("Illegal marker character" && c >= '0' && c <= '9'); + LUAU_ASSERT("Duplicate marker found" && markerPosition.count(c) == 0); + markerPosition.insert(std::pair{c, curPos}); + } + else if (c == '@') + { + // skip the '@' character } else { @@ -69,22 +69,39 @@ struct ACFixture : Fixture curPos.column++; } } + prevChar = c; } + LUAU_ASSERT("Digit expected after @ symbol" && prevChar != '@'); return Fixture::check(filteredSource); } + const Position& getPosition(char marker) const + { + auto i = markerPosition.find(marker); + LUAU_ASSERT(i != markerPosition.end()); + return i->second; + } + // Maps a marker character (0-9 inclusive) to a position in the source code. std::map markerPosition; }; +struct ACFixture : ACFixtureImpl +{ +}; + +struct UnfrozenACFixture : ACFixtureImpl +{ +}; + TEST_SUITE_BEGIN("AutocompleteTest"); TEST_CASE_FIXTURE(ACFixture, "empty_program") { - check(" "); + check(" @1"); - auto ac = autocomplete(0, 1); + auto ac = autocomplete('1'); CHECK(!ac.entryMap.empty()); CHECK(ac.entryMap.count("table")); @@ -93,26 +110,26 @@ TEST_CASE_FIXTURE(ACFixture, "empty_program") TEST_CASE_FIXTURE(ACFixture, "local_initializer") { - check("local a = "); + check("local a = @1"); - auto ac = autocomplete(0, 10); + auto ac = autocomplete('1'); CHECK(ac.entryMap.count("table")); CHECK(ac.entryMap.count("math")); } TEST_CASE_FIXTURE(ACFixture, "leave_numbers_alone") { - check("local a = 3.1"); + check("local a = 3.@11"); - auto ac = autocomplete(0, 12); + auto ac = autocomplete('1'); CHECK(ac.entryMap.empty()); } TEST_CASE_FIXTURE(ACFixture, "user_defined_globals") { - check("local myLocal = 4; "); + check("local myLocal = 4; @1"); - auto ac = autocomplete(0, 19); + auto ac = autocomplete('1'); CHECK(ac.entryMap.count("myLocal")); CHECK(ac.entryMap.count("table")); @@ -124,20 +141,20 @@ TEST_CASE_FIXTURE(ACFixture, "dont_suggest_local_before_its_definition") check(R"( local myLocal = 4 function abc() - local myInnerLocal = 1 - +@1 local myInnerLocal = 1 +@2 end - )"); +@3 )"); - auto ac = autocomplete(3, 0); + auto ac = autocomplete('1'); CHECK(ac.entryMap.count("myLocal")); CHECK(!ac.entryMap.count("myInnerLocal")); - ac = autocomplete(4, 0); + ac = autocomplete('2'); CHECK(ac.entryMap.count("myLocal")); CHECK(ac.entryMap.count("myInnerLocal")); - ac = autocomplete(6, 0); + ac = autocomplete('3'); CHECK(ac.entryMap.count("myLocal")); CHECK(!ac.entryMap.count("myInnerLocal")); } @@ -146,10 +163,10 @@ TEST_CASE_FIXTURE(ACFixture, "recursive_function") { check(R"( function foo() - end +@1 end )"); - auto ac = autocomplete(2, 0); + auto ac = autocomplete('1'); CHECK(ac.entryMap.count("foo")); } @@ -158,11 +175,11 @@ TEST_CASE_FIXTURE(ACFixture, "nested_recursive_function") check(R"( local function outer() local function inner() - end +@1 end end )"); - auto ac = autocomplete(3, 0); + auto ac = autocomplete('1'); CHECK(ac.entryMap.count("inner")); CHECK(ac.entryMap.count("outer")); } @@ -171,11 +188,11 @@ TEST_CASE_FIXTURE(ACFixture, "user_defined_local_functions_in_own_definition") { check(R"( local function abc() - +@1 end )"); - auto ac = autocomplete(2, 0); + auto ac = autocomplete('1'); CHECK(ac.entryMap.count("abc")); CHECK(ac.entryMap.count("table")); @@ -183,11 +200,11 @@ TEST_CASE_FIXTURE(ACFixture, "user_defined_local_functions_in_own_definition") check(R"( local abc = function() - +@1 end )"); - ac = autocomplete(2, 0); + ac = autocomplete('1'); CHECK(ac.entryMap.count("abc")); // FIXME: This is actually incorrect! CHECK(ac.entryMap.count("table")); @@ -202,9 +219,9 @@ TEST_CASE_FIXTURE(ACFixture, "global_functions_are_not_scoped_lexically") end end - )"); +@1 )"); - auto ac = autocomplete(6, 0); + auto ac = autocomplete('1'); CHECK(!ac.entryMap.empty()); CHECK(ac.entryMap.count("abc")); @@ -220,9 +237,9 @@ TEST_CASE_FIXTURE(ACFixture, "local_functions_fall_out_of_scope") end end - )"); +@1 )"); - auto ac = autocomplete(6, 0); + auto ac = autocomplete('1'); CHECK_NE(0, ac.entryMap.size()); CHECK(!ac.entryMap.count("abc")); @@ -233,10 +250,10 @@ TEST_CASE_FIXTURE(ACFixture, "function_parameters") check(R"( function abc(test) - end +@1 end )"); - auto ac = autocomplete(3, 0); + auto ac = autocomplete('1'); CHECK(ac.entryMap.count("test")); } @@ -244,11 +261,10 @@ TEST_CASE_FIXTURE(ACFixture, "function_parameters") TEST_CASE_FIXTURE(ACFixture, "get_member_completions") { check(R"( - local a = table. -- Line 1 - -- | Column 23 + local a = table.@1 )"); - auto ac = autocomplete(1, 24); + auto ac = autocomplete('1'); CHECK_EQ(16, ac.entryMap.size()); CHECK(ac.entryMap.count("find")); @@ -260,10 +276,10 @@ TEST_CASE_FIXTURE(ACFixture, "nested_member_completions") { check(R"( local tbl = { abc = { def = 1234, egh = false } } - tbl.abc. + tbl.abc. @1 )"); - auto ac = autocomplete(2, 17); + auto ac = autocomplete('1'); CHECK_EQ(2, ac.entryMap.size()); CHECK(ac.entryMap.count("def")); CHECK(ac.entryMap.count("egh")); @@ -274,10 +290,10 @@ TEST_CASE_FIXTURE(ACFixture, "unsealed_table") check(R"( local tbl = {} tbl.prop = 5 - tbl. + tbl.@1 )"); - auto ac = autocomplete(3, 12); + auto ac = autocomplete('1'); CHECK_EQ(1, ac.entryMap.size()); CHECK(ac.entryMap.count("prop")); } @@ -288,10 +304,10 @@ TEST_CASE_FIXTURE(ACFixture, "unsealed_table_2") local tbl = {} local inner = { prop = 5 } tbl.inner = inner - tbl.inner. + tbl.inner. @1 )"); - auto ac = autocomplete(4, 19); + auto ac = autocomplete('1'); CHECK_EQ(1, ac.entryMap.size()); CHECK(ac.entryMap.count("prop")); } @@ -302,10 +318,10 @@ TEST_CASE_FIXTURE(ACFixture, "cyclic_table") local abc = {} local def = { abc = abc } abc.def = def - abc.def. + abc.def. @1 )"); - auto ac = autocomplete(4, 17); + auto ac = autocomplete('1'); CHECK(ac.entryMap.count("abc")); } @@ -315,11 +331,11 @@ TEST_CASE_FIXTURE(ACFixture, "table_union") type t1 = { a1 : string, b2 : number } type t2 = { b2 : string, c3 : string } function func(abc : t1 | t2) - abc. + abc. @1 end )"); - auto ac = autocomplete(4, 18); + auto ac = autocomplete('1'); CHECK_EQ(1, ac.entryMap.size()); CHECK(ac.entryMap.count("b2")); } @@ -330,11 +346,11 @@ TEST_CASE_FIXTURE(ACFixture, "table_intersection") type t1 = { a1 : string, b2 : number } type t2 = { b2 : string, c3 : string } function func(abc : t1 & t2) - abc. + abc. @1 end )"); - auto ac = autocomplete(4, 18); + auto ac = autocomplete('1'); CHECK_EQ(3, ac.entryMap.size()); CHECK(ac.entryMap.count("a1")); CHECK(ac.entryMap.count("b2")); @@ -344,20 +360,19 @@ TEST_CASE_FIXTURE(ACFixture, "table_intersection") TEST_CASE_FIXTURE(ACFixture, "get_string_completions") { check(R"( - local a = ("foo"): -- Line 1 - -- | Column 26 + local a = ("foo"):@1 )"); - auto ac = autocomplete(1, 26); + auto ac = autocomplete('1'); CHECK_EQ(17, ac.entryMap.size()); } TEST_CASE_FIXTURE(ACFixture, "get_suggestions_for_new_statement") { - check(""); + check("@1"); - auto ac = autocomplete(0, 0); + auto ac = autocomplete('1'); CHECK_NE(0, ac.entryMap.size()); @@ -366,12 +381,12 @@ TEST_CASE_FIXTURE(ACFixture, "get_suggestions_for_new_statement") TEST_CASE_FIXTURE(ACFixture, "get_suggestions_for_the_very_start_of_the_script") { - check(R"( + check(R"(@1 function aaa() end )"); - auto ac = autocomplete(0, 0); + auto ac = autocomplete('1'); CHECK(ac.entryMap.count("table")); } @@ -382,11 +397,11 @@ TEST_CASE_FIXTURE(ACFixture, "method_call_inside_function_body") local game = { GetService=function(s) return 'hello' end } function a() - game: + game: @1 end )"); - auto ac = autocomplete(4, 19); + auto ac = autocomplete('1'); CHECK_NE(0, ac.entryMap.size()); @@ -396,10 +411,10 @@ TEST_CASE_FIXTURE(ACFixture, "method_call_inside_function_body") TEST_CASE_FIXTURE(ACFixture, "method_call_inside_if_conditional") { check(R"( - if table: + if table: @1 )"); - auto ac = autocomplete(1, 19); + auto ac = autocomplete('1'); CHECK_NE(0, ac.entryMap.size()); CHECK(ac.entryMap.count("concat")); @@ -411,12 +426,12 @@ TEST_CASE_FIXTURE(ACFixture, "statement_between_two_statements") check(R"( function getmyscripts() end - g + g@1 getmyscripts() )"); - auto ac = autocomplete(3, 9); + auto ac = autocomplete('1'); CHECK_NE(0, ac.entryMap.size()); @@ -431,11 +446,11 @@ TEST_CASE_FIXTURE(ACFixture, "bias_toward_inner_scope") function B() local A = {two=2} - A + A @1 end )"); - auto ac = autocomplete(6, 15); + auto ac = autocomplete('1'); CHECK(ac.entryMap.count("A")); @@ -448,12 +463,12 @@ TEST_CASE_FIXTURE(ACFixture, "bias_toward_inner_scope") TEST_CASE_FIXTURE(ACFixture, "recommend_statement_starting_keywords") { - check(""); - auto ac = autocomplete(0, 0); + check("@1"); + auto ac = autocomplete('1'); CHECK(ac.entryMap.count("local")); - check("local i = "); - auto ac2 = autocomplete(0, 10); + check("local i = @1"); + auto ac2 = autocomplete('1'); CHECK(!ac2.entryMap.count("local")); } @@ -464,9 +479,9 @@ TEST_CASE_FIXTURE(ACFixture, "do_not_overwrite_context_sensitive_kws") end - )"); +@1 )"); - auto ac = autocomplete(5, 0); + auto ac = autocomplete('1'); AutocompleteEntry entry = ac.entryMap["continue"]; CHECK(entry.kind == AutocompleteEntryKind::Binding); @@ -480,11 +495,11 @@ TEST_CASE_FIXTURE(ACFixture, "dont_offer_any_suggestions_from_within_a_comment") function foo:bar() end --[[ - foo: + foo:@1 ]] )"); - auto ac = autocomplete(6, 16); + auto ac = autocomplete('1'); CHECK_EQ(0, ac.entryMap.size()); } @@ -492,10 +507,10 @@ TEST_CASE_FIXTURE(ACFixture, "dont_offer_any_suggestions_from_within_a_comment") TEST_CASE_FIXTURE(ACFixture, "dont_offer_any_suggestions_from_the_end_of_a_comment") { check(R"( - --!strict + --!strict@1 )"); - auto ac = autocomplete(1, 17); + auto ac = autocomplete('1'); CHECK_EQ(0, ac.entryMap.size()); } @@ -505,10 +520,10 @@ TEST_CASE_FIXTURE(ACFixture, "dont_offer_any_suggestions_from_within_a_broken_co ScopedFastFlag sff{"LuauCaptureBrokenCommentSpans", true}; check(R"( - --[[ + --[[ @1 )"); - auto ac = autocomplete(1, 13); + auto ac = autocomplete('1'); CHECK_EQ(0, ac.entryMap.size()); } @@ -517,129 +532,129 @@ TEST_CASE_FIXTURE(ACFixture, "dont_offer_any_suggestions_from_within_a_broken_co { ScopedFastFlag sff{"LuauCaptureBrokenCommentSpans", true}; - check("--[["); + check("--[[@1"); - auto ac = autocomplete(0, 4); + auto ac = autocomplete('1'); CHECK_EQ(0, ac.entryMap.size()); } TEST_CASE_FIXTURE(ACFixture, "autocomplete_for_middle_keywords") { check(R"( - for x = + for x @1= )"); - auto ac1 = autocomplete(1, 14); + auto ac1 = autocomplete('1'); CHECK_EQ(ac1.entryMap.count("do"), 0); CHECK_EQ(ac1.entryMap.count("end"), 0); check(R"( - for x = 1 + for x =@1 1 )"); - auto ac2 = autocomplete(1, 15); + auto ac2 = autocomplete('1'); CHECK_EQ(ac2.entryMap.count("do"), 0); CHECK_EQ(ac2.entryMap.count("end"), 0); check(R"( - for x = 1, 2 + for x = 1,@1 2 )"); - auto ac3 = autocomplete(1, 18); + auto ac3 = autocomplete('1'); CHECK_EQ(1, ac3.entryMap.size()); CHECK_EQ(ac3.entryMap.count("do"), 1); check(R"( - for x = 1, 2, + for x = 1, @12, )"); - auto ac4 = autocomplete(1, 19); + auto ac4 = autocomplete('1'); CHECK_EQ(ac4.entryMap.count("do"), 0); CHECK_EQ(ac4.entryMap.count("end"), 0); check(R"( - for x = 1, 2, 5 + for x = 1, 2, @15 )"); - auto ac5 = autocomplete(1, 22); + auto ac5 = autocomplete('1'); CHECK_EQ(ac5.entryMap.count("do"), 1); CHECK_EQ(ac5.entryMap.count("end"), 0); check(R"( - for x = 1, 2, 5 f + for x = 1, 2, 5 f@1 )"); - auto ac6 = autocomplete(1, 25); + auto ac6 = autocomplete('1'); CHECK_EQ(ac6.entryMap.size(), 1); CHECK_EQ(ac6.entryMap.count("do"), 1); check(R"( - for x = 1, 2, 5 do + for x = 1, 2, 5 do @1 )"); - auto ac7 = autocomplete(1, 32); + auto ac7 = autocomplete('1'); CHECK_EQ(ac7.entryMap.count("end"), 1); } TEST_CASE_FIXTURE(ACFixture, "autocomplete_for_in_middle_keywords") { check(R"( - for + for @1 )"); - auto ac1 = autocomplete(1, 12); + auto ac1 = autocomplete('1'); CHECK_EQ(0, ac1.entryMap.size()); check(R"( - for x + for x@1 @2 )"); - auto ac2 = autocomplete(1, 13); + auto ac2 = autocomplete('1'); CHECK_EQ(0, ac2.entryMap.size()); - auto ac2a = autocomplete(1, 14); + auto ac2a = autocomplete('2'); CHECK_EQ(1, ac2a.entryMap.size()); CHECK_EQ(1, ac2a.entryMap.count("in")); check(R"( - for x in y + for x in y@1 )"); - auto ac3 = autocomplete(1, 18); + auto ac3 = autocomplete('1'); CHECK_EQ(ac3.entryMap.count("table"), 1); CHECK_EQ(ac3.entryMap.count("do"), 0); check(R"( - for x in y + for x in y @1 )"); - auto ac4 = autocomplete(1, 19); + auto ac4 = autocomplete('1'); CHECK_EQ(ac4.entryMap.size(), 1); CHECK_EQ(ac4.entryMap.count("do"), 1); check(R"( - for x in f f + for x in f f@1 )"); - auto ac5 = autocomplete(1, 20); + auto ac5 = autocomplete('1'); CHECK_EQ(ac5.entryMap.size(), 1); CHECK_EQ(ac5.entryMap.count("do"), 1); check(R"( - for x in y do + for x in y do @1 )"); - auto ac6 = autocomplete(1, 23); + auto ac6 = autocomplete('1'); CHECK_EQ(ac6.entryMap.count("in"), 0); CHECK_EQ(ac6.entryMap.count("table"), 1); CHECK_EQ(ac6.entryMap.count("end"), 1); CHECK_EQ(ac6.entryMap.count("function"), 1); check(R"( - for x in y do e + for x in y do e@1 )"); - auto ac7 = autocomplete(1, 23); + auto ac7 = autocomplete('1'); CHECK_EQ(ac7.entryMap.count("in"), 0); CHECK_EQ(ac7.entryMap.count("table"), 1); CHECK_EQ(ac7.entryMap.count("end"), 1); @@ -649,33 +664,33 @@ TEST_CASE_FIXTURE(ACFixture, "autocomplete_for_in_middle_keywords") TEST_CASE_FIXTURE(ACFixture, "autocomplete_while_middle_keywords") { check(R"( - while + while@1 )"); - auto ac1 = autocomplete(1, 13); + auto ac1 = autocomplete('1'); CHECK_EQ(ac1.entryMap.count("do"), 0); CHECK_EQ(ac1.entryMap.count("end"), 0); check(R"( - while true + while true @1 )"); - auto ac2 = autocomplete(1, 19); + auto ac2 = autocomplete('1'); CHECK_EQ(1, ac2.entryMap.size()); CHECK_EQ(ac2.entryMap.count("do"), 1); check(R"( - while true do + while true do @1 )"); - auto ac3 = autocomplete(1, 23); + auto ac3 = autocomplete('1'); CHECK_EQ(ac3.entryMap.count("end"), 1); check(R"( - while true d + while true d@1 )"); - auto ac4 = autocomplete(1, 20); + auto ac4 = autocomplete('1'); CHECK_EQ(1, ac4.entryMap.size()); CHECK_EQ(ac4.entryMap.count("do"), 1); } @@ -683,10 +698,10 @@ TEST_CASE_FIXTURE(ACFixture, "autocomplete_while_middle_keywords") TEST_CASE_FIXTURE(ACFixture, "autocomplete_if_middle_keywords") { check(R"( - if + if @1 )"); - auto ac1 = autocomplete(1, 13); + auto ac1 = autocomplete('1'); CHECK_EQ(ac1.entryMap.count("then"), 0); CHECK_EQ(ac1.entryMap.count("function"), 1); // FIXME: This is kind of dumb. It is technically syntactically valid but you can never do anything interesting with this. @@ -696,10 +711,10 @@ TEST_CASE_FIXTURE(ACFixture, "autocomplete_if_middle_keywords") CHECK_EQ(ac1.entryMap.count("end"), 0); check(R"( - if x + if x @1 )"); - auto ac2 = autocomplete(1, 14); + auto ac2 = autocomplete('1'); CHECK_EQ(ac2.entryMap.count("then"), 1); CHECK_EQ(ac2.entryMap.count("function"), 0); CHECK_EQ(ac2.entryMap.count("else"), 0); @@ -707,20 +722,20 @@ TEST_CASE_FIXTURE(ACFixture, "autocomplete_if_middle_keywords") CHECK_EQ(ac2.entryMap.count("end"), 0); check(R"( - if x t + if x t@1 )"); - auto ac3 = autocomplete(1, 14); + auto ac3 = autocomplete('1'); CHECK_EQ(1, ac3.entryMap.size()); CHECK_EQ(ac3.entryMap.count("then"), 1); check(R"( if x then - +@1 end )"); - auto ac4 = autocomplete(2, 0); + auto ac4 = autocomplete('1'); CHECK_EQ(ac4.entryMap.count("then"), 0); CHECK_EQ(ac4.entryMap.count("else"), 1); CHECK_EQ(ac4.entryMap.count("function"), 1); @@ -729,11 +744,11 @@ TEST_CASE_FIXTURE(ACFixture, "autocomplete_if_middle_keywords") check(R"( if x then - t + t@1 end )"); - auto ac4a = autocomplete(2, 13); + auto ac4a = autocomplete('1'); CHECK_EQ(ac4a.entryMap.count("then"), 0); CHECK_EQ(ac4a.entryMap.count("table"), 1); CHECK_EQ(ac4a.entryMap.count("else"), 1); @@ -741,12 +756,12 @@ TEST_CASE_FIXTURE(ACFixture, "autocomplete_if_middle_keywords") check(R"( if x then - +@1 elseif x then end )"); - auto ac5 = autocomplete(2, 0); + auto ac5 = autocomplete('1'); CHECK_EQ(ac5.entryMap.count("then"), 0); CHECK_EQ(ac5.entryMap.count("function"), 1); CHECK_EQ(ac5.entryMap.count("else"), 0); @@ -757,10 +772,10 @@ TEST_CASE_FIXTURE(ACFixture, "autocomplete_if_middle_keywords") TEST_CASE_FIXTURE(ACFixture, "autocomplete_until_in_repeat") { check(R"( - repeat + repeat @1 )"); - auto ac = autocomplete(1, 16); + auto ac = autocomplete('1'); CHECK_EQ(ac.entryMap.count("table"), 1); CHECK_EQ(ac.entryMap.count("until"), 1); } @@ -769,48 +784,48 @@ TEST_CASE_FIXTURE(ACFixture, "autocomplete_until_expression") { check(R"( repeat - until + until @1 )"); - auto ac = autocomplete(2, 16); + auto ac = autocomplete('1'); CHECK_EQ(ac.entryMap.count("table"), 1); } TEST_CASE_FIXTURE(ACFixture, "local_names") { check(R"( - local ab + local ab@1 )"); - auto ac1 = autocomplete(1, 16); + auto ac1 = autocomplete('1'); CHECK_EQ(ac1.entryMap.size(), 1); CHECK_EQ(ac1.entryMap.count("function"), 1); check(R"( - local ab, cd + local ab, cd@1 )"); - auto ac2 = autocomplete(1, 20); + auto ac2 = autocomplete('1'); CHECK(ac2.entryMap.empty()); } TEST_CASE_FIXTURE(ACFixture, "autocomplete_end_with_fn_exprs") { check(R"( - local function f() + local function f() @1 )"); - auto ac = autocomplete(1, 28); + auto ac = autocomplete('1'); CHECK_EQ(ac.entryMap.count("end"), 1); } TEST_CASE_FIXTURE(ACFixture, "autocomplete_end_with_lambda") { check(R"( - local a = function() local bar = foo en + local a = function() local bar = foo en@1 )"); - auto ac = autocomplete(1, 47); + auto ac = autocomplete('1'); CHECK_EQ(ac.entryMap.count("end"), 1); } @@ -818,10 +833,10 @@ TEST_CASE_FIXTURE(ACFixture, "stop_at_first_stat_when_recommending_keywords") { check(R"( repeat - for x + for x @1 )"); - auto ac1 = autocomplete(2, 18); + auto ac1 = autocomplete('1'); CHECK_EQ(ac1.entryMap.count("in"), 1); CHECK_EQ(ac1.entryMap.count("until"), 0); } @@ -829,112 +844,112 @@ TEST_CASE_FIXTURE(ACFixture, "stop_at_first_stat_when_recommending_keywords") TEST_CASE_FIXTURE(ACFixture, "autocomplete_repeat_middle_keyword") { check(R"( - repeat + repeat @1 )"); - auto ac1 = autocomplete(1, 15); + auto ac1 = autocomplete('1'); CHECK_EQ(ac1.entryMap.count("do"), 1); CHECK_EQ(ac1.entryMap.count("function"), 1); CHECK_EQ(ac1.entryMap.count("until"), 1); check(R"( - repeat f f + repeat f f@1 )"); - auto ac2 = autocomplete(1, 18); + auto ac2 = autocomplete('1'); CHECK_EQ(ac2.entryMap.count("function"), 1); CHECK_EQ(ac2.entryMap.count("until"), 1); check(R"( repeat - u + u@1 until )"); - auto ac3 = autocomplete(2, 13); + auto ac3 = autocomplete('1'); CHECK_EQ(ac3.entryMap.count("until"), 0); } TEST_CASE_FIXTURE(ACFixture, "local_function") { check(R"( - local f + local f@1 )"); - auto ac1 = autocomplete(1, 15); + auto ac1 = autocomplete('1'); CHECK_EQ(ac1.entryMap.size(), 1); CHECK_EQ(ac1.entryMap.count("function"), 1); check(R"( - local f, cd + local f@1, cd )"); - auto ac2 = autocomplete(1, 15); + auto ac2 = autocomplete('1'); CHECK(ac2.entryMap.empty()); } TEST_CASE_FIXTURE(ACFixture, "local_function") { check(R"( - local function + local function @1 )"); - auto ac = autocomplete(1, 23); + auto ac = autocomplete('1'); CHECK(ac.entryMap.empty()); check(R"( - local function s + local function @1s@2 )"); - ac = autocomplete(1, 23); + ac = autocomplete('1'); CHECK(ac.entryMap.empty()); - ac = autocomplete(1, 24); + ac = autocomplete('2'); CHECK(ac.entryMap.empty()); check(R"( - local function () + local function @1()@2 )"); - ac = autocomplete(1, 23); + ac = autocomplete('1'); CHECK(ac.entryMap.empty()); - ac = autocomplete(1, 25); + ac = autocomplete('2'); CHECK(ac.entryMap.count("end")); check(R"( - local function something + local function something@1 )"); - ac = autocomplete(1, 32); + ac = autocomplete('1'); CHECK(ac.entryMap.empty()); check(R"( local tbl = {} - function tbl.something() end + function tbl.something@1() end )"); - ac = autocomplete(2, 30); + ac = autocomplete('1'); CHECK(ac.entryMap.empty()); } TEST_CASE_FIXTURE(ACFixture, "local_function_params") { check(R"( - local function abc(def) + local function @1a@2bc(@3d@4ef)@5 @6 )"); - CHECK(autocomplete(1, 23).entryMap.empty()); - CHECK(autocomplete(1, 24).entryMap.empty()); - CHECK(autocomplete(1, 27).entryMap.empty()); - CHECK(autocomplete(1, 28).entryMap.empty()); - CHECK(!autocomplete(1, 31).entryMap.empty()); + CHECK(autocomplete('1').entryMap.empty()); + CHECK(autocomplete('2').entryMap.empty()); + CHECK(autocomplete('3').entryMap.empty()); + CHECK(autocomplete('4').entryMap.empty()); + CHECK(!autocomplete('5').entryMap.empty()); - CHECK(!autocomplete(1, 32).entryMap.empty()); + CHECK(!autocomplete('6').entryMap.empty()); check(R"( local function abc(def) - end +@1 end )"); for (unsigned int i = 23; i < 31; ++i) @@ -943,16 +958,16 @@ TEST_CASE_FIXTURE(ACFixture, "local_function_params") } CHECK(!autocomplete(1, 32).entryMap.empty()); - auto ac2 = autocomplete(2, 0); + auto ac2 = autocomplete('1'); CHECK_EQ(ac2.entryMap.count("abc"), 1); CHECK_EQ(ac2.entryMap.count("def"), 1); check(R"( - local function abc(def, ghi) + local function abc(def, ghi@1) end )"); - auto ac3 = autocomplete(1, 35); + auto ac3 = autocomplete('1'); CHECK(ac3.entryMap.empty()); } @@ -981,48 +996,48 @@ TEST_CASE_FIXTURE(ACFixture, "global_function_params") check(R"( function abc(def) - +@1 end )"); - auto ac2 = autocomplete(2, 0); + auto ac2 = autocomplete('1'); CHECK_EQ(ac2.entryMap.count("abc"), 1); CHECK_EQ(ac2.entryMap.count("def"), 1); check(R"( - function abc(def, ghi) + function abc(def, ghi@1) end )"); - auto ac3 = autocomplete(1, 29); + auto ac3 = autocomplete('1'); CHECK(ac3.entryMap.empty()); } TEST_CASE_FIXTURE(ACFixture, "arguments_to_global_lambda") { check(R"( - abc = function(def, ghi) + abc = function(def, ghi@1) end )"); - auto ac = autocomplete(1, 31); + auto ac = autocomplete('1'); CHECK(ac.entryMap.empty()); } TEST_CASE_FIXTURE(ACFixture, "function_expr_params") { check(R"( - abc = function(def) + abc = function(def) @1 )"); for (unsigned int i = 20; i < 27; ++i) { CHECK(autocomplete(1, i).entryMap.empty()); } - CHECK(!autocomplete(1, 28).entryMap.empty()); + CHECK(!autocomplete('1').entryMap.empty()); check(R"( - abc = function(def) + abc = function(def) @1 end )"); @@ -1030,25 +1045,25 @@ TEST_CASE_FIXTURE(ACFixture, "function_expr_params") { CHECK(autocomplete(1, i).entryMap.empty()); } - CHECK(!autocomplete(1, 28).entryMap.empty()); + CHECK(!autocomplete('1').entryMap.empty()); check(R"( abc = function(def) - +@1 end )"); - auto ac2 = autocomplete(2, 0); + auto ac2 = autocomplete('1'); CHECK_EQ(ac2.entryMap.count("def"), 1); } TEST_CASE_FIXTURE(ACFixture, "local_initializer") { check(R"( - local a = t + local a = t@1 )"); - auto ac = autocomplete(1, 19); + auto ac = autocomplete('1'); CHECK_EQ(ac.entryMap.count("table"), 1); CHECK_EQ(ac.entryMap.count("true"), 1); } @@ -1056,20 +1071,20 @@ TEST_CASE_FIXTURE(ACFixture, "local_initializer") TEST_CASE_FIXTURE(ACFixture, "local_initializer_2") { check(R"( - local a= + local a=@1 )"); - auto ac = autocomplete(1, 16); + auto ac = autocomplete('1'); CHECK(ac.entryMap.count("table")); } TEST_CASE_FIXTURE(ACFixture, "get_member_completions") { check(R"( - local a = 12.3 + local a = 12.@13 )"); - auto ac = autocomplete(1, 21); + auto ac = autocomplete('1'); CHECK(ac.entryMap.empty()); } @@ -1083,21 +1098,21 @@ TEST_CASE_FIXTURE(ACFixture, "sometimes_the_metatable_is_an_error") return setmetatable({x=6}, X) -- oops! end local t = T.new() - t. + t. @1 )"); - autocomplete(8, 12); + autocomplete('1'); // Don't crash! } TEST_CASE_FIXTURE(ACFixture, "local_types_builtin") { check(R"( -local a: n +local a: n@1 local b: string = "don't trip" )"); - auto ac = autocomplete(1, 10); + auto ac = autocomplete('1'); CHECK(ac.entryMap.count("nil")); CHECK(ac.entryMap.count("number")); @@ -1108,23 +1123,23 @@ TEST_CASE_FIXTURE(ACFixture, "private_types") check(R"( do type num = number - local a: nu - local b: num + local a: n@1u + local b: nu@2m end -local a: nu +local a: nu@3 )"); - auto ac = autocomplete(3, 14); + auto ac = autocomplete('1'); CHECK(ac.entryMap.count("num")); CHECK(ac.entryMap.count("number")); - ac = autocomplete(4, 15); + ac = autocomplete('2'); CHECK(ac.entryMap.count("num")); CHECK(ac.entryMap.count("number")); - ac = autocomplete(6, 11); + ac = autocomplete('3'); CHECK(!ac.entryMap.count("num")); CHECK(ac.entryMap.count("number")); @@ -1136,11 +1151,11 @@ TEST_CASE_FIXTURE(ACFixture, "type_scoping_easy") type Table = { a: number, b: number } do type Table = { x: string, y: string } - local a: T + local a: T@1 end )"); - auto ac = autocomplete(4, 14); + auto ac = autocomplete('1'); REQUIRE(ac.entryMap.count("Table")); REQUIRE(ac.entryMap["Table"].type); @@ -1198,11 +1213,11 @@ local a: aaa. TEST_CASE_FIXTURE(ACFixture, "argument_types") { check(R"( -local function f(a: n +local function f(a: n@1 local b: string = "don't trip" )"); - auto ac = autocomplete(1, 21); + auto ac = autocomplete('1'); CHECK(ac.entryMap.count("nil")); CHECK(ac.entryMap.count("number")); @@ -1211,11 +1226,11 @@ local b: string = "don't trip" TEST_CASE_FIXTURE(ACFixture, "return_types") { check(R"( -local function f(a: number): n +local function f(a: number): n@1 local b: string = "don't trip" )"); - auto ac = autocomplete(1, 30); + auto ac = autocomplete('1'); CHECK(ac.entryMap.count("nil")); CHECK(ac.entryMap.count("number")); @@ -1225,10 +1240,10 @@ TEST_CASE_FIXTURE(ACFixture, "as_types") { check(R"( local a: any = 5 -local b: number = (a :: n +local b: number = (a :: n@1 )"); - auto ac = autocomplete(2, 25); + auto ac = autocomplete('1'); CHECK(ac.entryMap.count("nil")); CHECK(ac.entryMap.count("number")); @@ -1237,34 +1252,34 @@ local b: number = (a :: n TEST_CASE_FIXTURE(ACFixture, "function_type_types") { check(R"( -local a: (n -local b: (number, (n -local c: (number, (number) -> n -local d: (number, (number) -> (number, n -local e: (n: n +local a: (n@1 +local b: (number, (n@2 +local c: (number, (number) -> n@3 +local d: (number, (number) -> (number, n@4 +local e: (n: n@5 )"); - auto ac = autocomplete(1, 11); + auto ac = autocomplete('1'); CHECK(ac.entryMap.count("nil")); CHECK(ac.entryMap.count("number")); - ac = autocomplete(2, 20); + ac = autocomplete('2'); CHECK(ac.entryMap.count("nil")); CHECK(ac.entryMap.count("number")); - ac = autocomplete(3, 31); + ac = autocomplete('3'); CHECK(ac.entryMap.count("nil")); CHECK(ac.entryMap.count("number")); - ac = autocomplete(4, 40); + ac = autocomplete('4'); CHECK(ac.entryMap.count("nil")); CHECK(ac.entryMap.count("number")); - ac = autocomplete(5, 14); + ac = autocomplete('5'); CHECK(ac.entryMap.count("nil")); CHECK(ac.entryMap.count("number")); @@ -1276,11 +1291,11 @@ TEST_CASE_FIXTURE(ACFixture, "generic_types") ScopedFastFlag luauGenericFunctions("LuauGenericFunctions", true); check(R"( -function f(a: T +function f(a: T@1 local b: string = "don't trip" )"); - auto ac = autocomplete(1, 25); + auto ac = autocomplete('1'); CHECK(ac.entryMap.count("Tee")); } @@ -1293,10 +1308,10 @@ local function target(a: number, b: string) return a + #b end local one = 4 local two = "hello" -return target(o +return target(o@1 )"); - auto ac = autocomplete(5, 15); + auto ac = autocomplete('1'); CHECK(ac.entryMap.count("one")); CHECK(ac.entryMap["one"].typeCorrect == TypeCorrectKind::Correct); @@ -1307,10 +1322,10 @@ local function target(a: number, b: string) return a + #b end local one = 4 local two = "hello" -return target(one, t +return target(one, t@1 )"); - ac = autocomplete(5, 20); + ac = autocomplete('1'); CHECK(ac.entryMap.count("two")); CHECK(ac.entryMap["two"].typeCorrect == TypeCorrectKind::Correct); @@ -1321,10 +1336,10 @@ return target(one, t local function target(a: number, b: string) return a + #b end local a = { one = 4, two = "hello" } -return target(a. +return target(a.@1 )"); - ac = autocomplete(4, 16); + ac = autocomplete('1'); CHECK(ac.entryMap.count("one")); CHECK(ac.entryMap["one"].typeCorrect == TypeCorrectKind::Correct); @@ -1334,10 +1349,10 @@ return target(a. local function target(a: number, b: string) return a + #b end local a = { one = 4, two = "hello" } -return target(a.one, a. +return target(a.one, a.@1 )"); - ac = autocomplete(4, 23); + ac = autocomplete('1'); CHECK(ac.entryMap.count("two")); CHECK(ac.entryMap["two"].typeCorrect == TypeCorrectKind::Correct); @@ -1348,10 +1363,10 @@ return target(a.one, a. local function target(a: string?) return #b end local a = { one = 4, two = "hello" } -return target(a. +return target(a.@1 )"); - ac = autocomplete(4, 16); + ac = autocomplete('1'); CHECK(ac.entryMap.count("two")); CHECK(ac.entryMap["two"].typeCorrect == TypeCorrectKind::Correct); @@ -1363,10 +1378,10 @@ TEST_CASE_FIXTURE(ACFixture, "type_correct_suggestion_in_table") check(R"( type Foo = { a: number, b: string } local a = { one = 4, two = "hello" } -local b: Foo = { a = a. +local b: Foo = { a = a.@1 )"); - auto ac = autocomplete(3, 23); + auto ac = autocomplete('1'); CHECK(ac.entryMap.count("one")); CHECK(ac.entryMap["one"].typeCorrect == TypeCorrectKind::Correct); @@ -1375,10 +1390,10 @@ local b: Foo = { a = a. check(R"( type Foo = { a: number, b: string } local a = { one = 4, two = "hello" } -local b: Foo = { b = a. +local b: Foo = { b = a.@1 )"); - ac = autocomplete(3, 23); + ac = autocomplete('1'); CHECK(ac.entryMap.count("two")); CHECK(ac.entryMap["two"].typeCorrect == TypeCorrectKind::Correct); @@ -1392,10 +1407,10 @@ local function target(a: number, b: string) return a + #b end local function bar1(a: number) return -a end local function bar2(a: string) reutrn a .. 'x' end -return target(b +return target(b@1 )"); - auto ac = autocomplete(5, 15); + auto ac = autocomplete('1'); CHECK(ac.entryMap.count("bar1")); CHECK(ac.entryMap["bar1"].typeCorrect == TypeCorrectKind::CorrectFunctionResult); @@ -1406,10 +1421,10 @@ local function target(a: number, b: string) return a + #b end local function bar1(a: number) return -a end local function bar2(a: string) return a .. 'x' end -return target(bar1, b +return target(bar1, b@1 )"); - ac = autocomplete(5, 21); + ac = autocomplete('1'); CHECK(ac.entryMap.count("bar2")); CHECK(ac.entryMap["bar2"].typeCorrect == TypeCorrectKind::CorrectFunctionResult); @@ -1420,10 +1435,10 @@ local function target(a: number, b: string) return a + #b end local function bar1(a: number): (...number) return -a, a end local function bar2(a: string) reutrn a .. 'x' end -return target(b +return target(b@1 )"); - ac = autocomplete(5, 15); + ac = autocomplete('1'); CHECK(ac.entryMap.count("bar1")); CHECK(ac.entryMap["bar1"].typeCorrect == TypeCorrectKind::CorrectFunctionResult); @@ -1433,69 +1448,69 @@ return target(b TEST_CASE_FIXTURE(ACFixture, "type_correct_local_type_suggestion") { check(R"( -local b: s = "str" +local b: s@1 = "str" )"); - auto ac = autocomplete(1, 10); + auto ac = autocomplete('1'); CHECK(ac.entryMap.count("string")); CHECK(ac.entryMap["string"].typeCorrect == TypeCorrectKind::Correct); check(R"( local function f() return "str" end -local b: s = f() +local b: s@1 = f() )"); - ac = autocomplete(2, 10); + ac = autocomplete('1'); CHECK(ac.entryMap.count("string")); CHECK(ac.entryMap["string"].typeCorrect == TypeCorrectKind::Correct); check(R"( -local b: s, c: n = "str", 2 +local b: s@1, c: n@2 = "str", 2 )"); - ac = autocomplete(1, 10); + ac = autocomplete('1'); CHECK(ac.entryMap.count("string")); CHECK(ac.entryMap["string"].typeCorrect == TypeCorrectKind::Correct); - ac = autocomplete(1, 16); + ac = autocomplete('2'); CHECK(ac.entryMap.count("number")); CHECK(ac.entryMap["number"].typeCorrect == TypeCorrectKind::Correct); check(R"( local function f() return 1, "str", 3 end -local a: b, b: n, c: s, d: n = false, f() +local a: b@1, b: n@2, c: s@3, d: n@4 = false, f() )"); - ac = autocomplete(2, 10); + ac = autocomplete('1'); CHECK(ac.entryMap.count("boolean")); CHECK(ac.entryMap["boolean"].typeCorrect == TypeCorrectKind::Correct); - ac = autocomplete(2, 16); + ac = autocomplete('2'); CHECK(ac.entryMap.count("number")); CHECK(ac.entryMap["number"].typeCorrect == TypeCorrectKind::Correct); - ac = autocomplete(2, 22); + ac = autocomplete('3'); CHECK(ac.entryMap.count("string")); CHECK(ac.entryMap["string"].typeCorrect == TypeCorrectKind::Correct); - ac = autocomplete(2, 28); + ac = autocomplete('4'); CHECK(ac.entryMap.count("number")); CHECK(ac.entryMap["number"].typeCorrect == TypeCorrectKind::Correct); check(R"( local function f(): ...number return 1, 2, 3 end -local a: boolean, b: n = false, f() +local a: boolean, b: n@1 = false, f() )"); - ac = autocomplete(2, 22); + ac = autocomplete('1'); CHECK(ac.entryMap.count("number")); CHECK(ac.entryMap["number"].typeCorrect == TypeCorrectKind::Correct); @@ -1504,46 +1519,46 @@ local a: boolean, b: n = false, f() TEST_CASE_FIXTURE(ACFixture, "type_correct_function_type_suggestion") { check(R"( -local b: (n) -> number = function(a: number, b: string) return a + #b end +local b: (n@1) -> number = function(a: number, b: string) return a + #b end )"); - auto ac = autocomplete(1, 11); + auto ac = autocomplete('1'); CHECK(ac.entryMap.count("number")); CHECK(ac.entryMap["number"].typeCorrect == TypeCorrectKind::Correct); check(R"( -local b: (number, s = function(a: number, b: string) return a + #b end +local b: (number, s@1 = function(a: number, b: string) return a + #b end )"); - ac = autocomplete(1, 19); + ac = autocomplete('1'); CHECK(ac.entryMap.count("string")); CHECK(ac.entryMap["string"].typeCorrect == TypeCorrectKind::Correct); check(R"( -local b: (number, string) -> b = function(a: number, b: string): boolean return a + #b == 0 end +local b: (number, string) -> b@1 = function(a: number, b: string): boolean return a + #b == 0 end )"); - ac = autocomplete(1, 30); + ac = autocomplete('1'); CHECK(ac.entryMap.count("boolean")); CHECK(ac.entryMap["boolean"].typeCorrect == TypeCorrectKind::Correct); check(R"( -local b: (number, ...s) = function(a: number, ...: string) return a end +local b: (number, ...s@1) = function(a: number, ...: string) return a end )"); - ac = autocomplete(1, 22); + ac = autocomplete('1'); CHECK(ac.entryMap.count("string")); CHECK(ac.entryMap["string"].typeCorrect == TypeCorrectKind::Correct); check(R"( -local b: (number) -> ...s = function(a: number): ...string return "a", "b", "c" end +local b: (number) -> ...s@1 = function(a: number): ...string return "a", "b", "c" end )"); - ac = autocomplete(1, 25); + ac = autocomplete('1'); CHECK(ac.entryMap.count("string")); CHECK(ac.entryMap["string"].typeCorrect == TypeCorrectKind::Correct); @@ -1552,24 +1567,24 @@ local b: (number) -> ...s = function(a: number): ...string return "a", "b", "c" TEST_CASE_FIXTURE(ACFixture, "type_correct_full_type_suggestion") { check(R"( -local b: = "str" +local b:@1 @2= "str" )"); - auto ac = autocomplete(1, 8); + auto ac = autocomplete('1'); CHECK(ac.entryMap.count("string")); CHECK(ac.entryMap["string"].typeCorrect == TypeCorrectKind::Correct); - ac = autocomplete(1, 9); + ac = autocomplete('2'); CHECK(ac.entryMap.count("string")); CHECK(ac.entryMap["string"].typeCorrect == TypeCorrectKind::Correct); check(R"( -local b: = function(a: number) return -a end +local b: @1= function(a: number) return -a end )"); - ac = autocomplete(1, 9); + ac = autocomplete('1'); CHECK(ac.entryMap.count("(number) -> number")); CHECK(ac.entryMap["(number) -> number"].typeCorrect == TypeCorrectKind::Correct); @@ -1580,12 +1595,12 @@ TEST_CASE_FIXTURE(ACFixture, "type_correct_argument_type_suggestion") check(R"( local function target(a: number, b: string) return a + #b end -local function d(a: n, b) +local function d(a: n@1, b) return target(a, b) end )"); - auto ac = autocomplete(3, 21); + auto ac = autocomplete('1'); CHECK(ac.entryMap.count("number")); CHECK(ac.entryMap["number"].typeCorrect == TypeCorrectKind::Correct); @@ -1593,12 +1608,12 @@ end check(R"( local function target(a: number, b: string) return a + #b end -local function d(a, b: s) +local function d(a, b: s@1) return target(a, b) end )"); - ac = autocomplete(3, 24); + ac = autocomplete('1'); CHECK(ac.entryMap.count("string")); CHECK(ac.entryMap["string"].typeCorrect == TypeCorrectKind::Correct); @@ -1606,17 +1621,17 @@ end check(R"( local function target(a: number, b: string) return a + #b end -local function d(a: , b) +local function d(a:@1 @2, b) return target(a, b) end )"); - ac = autocomplete(3, 19); + ac = autocomplete('1'); CHECK(ac.entryMap.count("number")); CHECK(ac.entryMap["number"].typeCorrect == TypeCorrectKind::Correct); - ac = autocomplete(3, 20); + ac = autocomplete('2'); CHECK(ac.entryMap.count("number")); CHECK(ac.entryMap["number"].typeCorrect == TypeCorrectKind::Correct); @@ -1624,17 +1639,17 @@ end check(R"( local function target(a: number, b: string) return a + #b end -local function d(a, b: ): number +local function d(a, b: @1)@2: number return target(a, b) end )"); - ac = autocomplete(3, 23); + ac = autocomplete('1'); CHECK(ac.entryMap.count("string")); CHECK(ac.entryMap["string"].typeCorrect == TypeCorrectKind::Correct); - ac = autocomplete(3, 24); + ac = autocomplete('2'); CHECK(ac.entryMap["string"].typeCorrect == TypeCorrectKind::None); } @@ -1644,10 +1659,10 @@ TEST_CASE_FIXTURE(ACFixture, "type_correct_expected_argument_type_suggestion") check(R"( local function target(callback: (a: number, b: string) -> number) return callback(4, "hello") end -local x = target(function(a: +local x = target(function(a: @1 )"); - auto ac = autocomplete(3, 29); + auto ac = autocomplete('1'); CHECK(ac.entryMap.count("number")); CHECK(ac.entryMap["number"].typeCorrect == TypeCorrectKind::Correct); @@ -1655,10 +1670,10 @@ local x = target(function(a: check(R"( local function target(callback: (a: number, b: string) -> number) return callback(4, "hello") end -local x = target(function(a: n +local x = target(function(a: n@1 )"); - ac = autocomplete(3, 30); + ac = autocomplete('1'); CHECK(ac.entryMap.count("number")); CHECK(ac.entryMap["number"].typeCorrect == TypeCorrectKind::Correct); @@ -1666,17 +1681,17 @@ local x = target(function(a: n check(R"( local function target(callback: (a: number, b: string) -> number) return callback(4, "hello") end -local x = target(function(a: n, b: ) +local x = target(function(a: n@1, b: @2) return a + #b end) )"); - ac = autocomplete(3, 30); + ac = autocomplete('1'); CHECK(ac.entryMap.count("number")); CHECK(ac.entryMap["number"].typeCorrect == TypeCorrectKind::Correct); - ac = autocomplete(3, 35); + ac = autocomplete('2'); CHECK(ac.entryMap.count("string")); CHECK(ac.entryMap["string"].typeCorrect == TypeCorrectKind::Correct); @@ -1684,12 +1699,12 @@ end) check(R"( local function target(callback: (...number) -> number) return callback(1, 2, 3) end -local x = target(function(a: n) +local x = target(function(a: n@1) return a end )"); - ac = autocomplete(3, 30); + ac = autocomplete('1'); CHECK(ac.entryMap.count("number")); CHECK(ac.entryMap["number"].typeCorrect == TypeCorrectKind::Correct); @@ -1700,12 +1715,12 @@ TEST_CASE_FIXTURE(ACFixture, "type_correct_expected_argument_type_pack_suggestio check(R"( local function target(callback: (...number) -> number) return callback(1, 2, 3) end -local x = target(function(...:n) +local x = target(function(...:n@1) return a end )"); - auto ac = autocomplete(3, 31); + auto ac = autocomplete('1'); CHECK(ac.entryMap.count("number")); CHECK(ac.entryMap["number"].typeCorrect == TypeCorrectKind::Correct); @@ -1713,12 +1728,12 @@ end check(R"( local function target(callback: (...number) -> number) return callback(1, 2, 3) end -local x = target(function(a:number, b:number, ...:) +local x = target(function(a:number, b:number, ...:@1) return a + b end )"); - ac = autocomplete(3, 50); + ac = autocomplete('1'); CHECK(ac.entryMap.count("number")); CHECK(ac.entryMap["number"].typeCorrect == TypeCorrectKind::Correct); @@ -1729,12 +1744,12 @@ TEST_CASE_FIXTURE(ACFixture, "type_correct_expected_return_type_suggestion") check(R"( local function target(callback: () -> number) return callback() end -local x = target(function(): n +local x = target(function(): n@1 return 1 end )"); - auto ac = autocomplete(3, 30); + auto ac = autocomplete('1'); CHECK(ac.entryMap.count("number")); CHECK(ac.entryMap["number"].typeCorrect == TypeCorrectKind::Correct); @@ -1742,12 +1757,12 @@ end check(R"( local function target(callback: () -> (number, number)) return callback() end -local x = target(function(): (number, n +local x = target(function(): (number, n@1 return 1, 2 end )"); - ac = autocomplete(3, 39); + ac = autocomplete('1'); CHECK(ac.entryMap.count("number")); CHECK(ac.entryMap["number"].typeCorrect == TypeCorrectKind::Correct); @@ -1758,12 +1773,12 @@ TEST_CASE_FIXTURE(ACFixture, "type_correct_expected_return_type_pack_suggestion" check(R"( local function target(callback: () -> ...number) return callback() end -local x = target(function(): ...n +local x = target(function(): ...n@1 return 1, 2, 3 end )"); - auto ac = autocomplete(3, 33); + auto ac = autocomplete('1'); CHECK(ac.entryMap.count("number")); CHECK(ac.entryMap["number"].typeCorrect == TypeCorrectKind::Correct); @@ -1771,12 +1786,12 @@ end check(R"( local function target(callback: () -> ...number) return callback() end -local x = target(function(): (number, number, ...n +local x = target(function(): (number, number, ...n@1 return 1, 2, 3 end )"); - ac = autocomplete(3, 50); + ac = autocomplete('1'); CHECK(ac.entryMap.count("number")); CHECK(ac.entryMap["number"].typeCorrect == TypeCorrectKind::Correct); @@ -1787,10 +1802,10 @@ TEST_CASE_FIXTURE(ACFixture, "type_correct_expected_argument_type_suggestion_opt check(R"( local function target(callback: nil | (a: number, b: string) -> number) return callback(4, "hello") end -local x = target(function(a: +local x = target(function(a: @1 )"); - auto ac = autocomplete(3, 29); + auto ac = autocomplete('1'); CHECK(ac.entryMap.count("number")); CHECK(ac.entryMap["number"].typeCorrect == TypeCorrectKind::Correct); @@ -1803,21 +1818,21 @@ local t = {} t.x = 5 function t:target(callback: (a: number, b: string) -> number) return callback(self.x, "hello") end -local x = t:target(function(a: , b: ) end) -local y = t.target(t, function(a: number, b: ) end) +local x = t:target(function(a: @1, b:@2 ) end) +local y = t.target(t, function(a: number, b: @3) end) )"); - auto ac = autocomplete(5, 31); + auto ac = autocomplete('1'); CHECK(ac.entryMap.count("number")); CHECK(ac.entryMap["number"].typeCorrect == TypeCorrectKind::Correct); - ac = autocomplete(5, 35); + ac = autocomplete('2'); CHECK(ac.entryMap.count("string")); CHECK(ac.entryMap["string"].typeCorrect == TypeCorrectKind::Correct); - ac = autocomplete(6, 45); + ac = autocomplete('3'); CHECK(ac.entryMap.count("string")); CHECK(ac.entryMap["string"].typeCorrect == TypeCorrectKind::Correct); @@ -1899,26 +1914,26 @@ TEST_CASE_FIXTURE(ACFixture, "do_not_suggest_synthetic_table_name") { check(R"( local foo = { a = 1, b = 2 } -local bar: = foo +local bar: @1= foo )"); - auto ac = autocomplete(2, 11); + auto ac = autocomplete('1'); CHECK(!ac.entryMap.count("foo")); } -// CLI-45692: Remove UnfrozenFixture here -TEST_CASE_FIXTURE(UnfrozenFixture, "type_correct_function_no_parenthesis") +// CLI-45692: Remove UnfrozenACFixture here +TEST_CASE_FIXTURE(UnfrozenACFixture, "type_correct_function_no_parenthesis") { check(R"( local function target(a: (number) -> number) return a(4) end local function bar1(a: number) return -a end local function bar2(a: string) reutrn a .. 'x' end -return target(b +return target(b@1 )"); - auto ac = autocomplete(frontend, "MainModule", Position{5, 15}, nullCallback); + auto ac = autocomplete('1'); CHECK(ac.entryMap.count("bar1")); CHECK(ac.entryMap["bar1"].typeCorrect == TypeCorrectKind::Correct); @@ -1930,16 +1945,16 @@ TEST_CASE_FIXTURE(ACFixture, "type_correct_sealed_table") { check(R"( local function f(a: { x: number, y: number }) return a.x + a.y end -local fp: = f +local fp: @1= f )"); - auto ac = autocomplete(2, 10); + auto ac = autocomplete('1'); CHECK(ac.entryMap.count("({ x: number, y: number }) -> number")); } -// CLI-45692: Remove UnfrozenFixture here -TEST_CASE_FIXTURE(UnfrozenFixture, "type_correct_keywords") +// CLI-45692: Remove UnfrozenACFixture here +TEST_CASE_FIXTURE(UnfrozenACFixture, "type_correct_keywords") { check(R"( local function a(x: boolean) end @@ -1951,33 +1966,33 @@ local function e(x: ((number) -> string) & ((boolean) -> number)) end local tru = {} local ni = false -local ac = a(t) -local bc = b(n) -local cc = c(f) -local dc = d(f) -local ec = e(f) +local ac = a(t@1) +local bc = b(n@2) +local cc = c(f@3) +local dc = d(f@4) +local ec = e(f@5) )"); - auto ac = autocomplete(frontend, "MainModule", Position{10, 14}, nullCallback); + auto ac = autocomplete('1'); CHECK(ac.entryMap.count("tru")); CHECK(ac.entryMap["tru"].typeCorrect == TypeCorrectKind::None); CHECK(ac.entryMap["true"].typeCorrect == TypeCorrectKind::Correct); CHECK(ac.entryMap["false"].typeCorrect == TypeCorrectKind::Correct); - ac = autocomplete(frontend, "MainModule", Position{11, 14}, nullCallback); + ac = autocomplete('2'); CHECK(ac.entryMap.count("ni")); CHECK(ac.entryMap["ni"].typeCorrect == TypeCorrectKind::None); CHECK(ac.entryMap["nil"].typeCorrect == TypeCorrectKind::Correct); - ac = autocomplete(frontend, "MainModule", Position{12, 14}, nullCallback); + ac = autocomplete('3'); CHECK(ac.entryMap.count("false")); CHECK(ac.entryMap["false"].typeCorrect == TypeCorrectKind::None); CHECK(ac.entryMap["function"].typeCorrect == TypeCorrectKind::Correct); - ac = autocomplete(frontend, "MainModule", Position{13, 14}, nullCallback); + ac = autocomplete('4'); CHECK(ac.entryMap["function"].typeCorrect == TypeCorrectKind::Correct); - ac = autocomplete(frontend, "MainModule", Position{14, 14}, nullCallback); + ac = autocomplete('5'); CHECK(ac.entryMap["function"].typeCorrect == TypeCorrectKind::Correct); } @@ -1988,10 +2003,10 @@ local target: ((number) -> string) & ((string) -> number)) local one = 4 local two = "hello" -return target(o) +return target(o@1) )"); - auto ac = autocomplete(5, 15); + auto ac = autocomplete('1'); CHECK(ac.entryMap.count("one")); CHECK(ac.entryMap["one"].typeCorrect == TypeCorrectKind::Correct); @@ -2002,10 +2017,10 @@ local target: ((number) -> string) & ((number) -> number)) local one = 4 local two = "hello" -return target(o) +return target(o@1) )"); - ac = autocomplete(5, 15); + ac = autocomplete('1'); CHECK(ac.entryMap.count("one")); CHECK(ac.entryMap["one"].typeCorrect == TypeCorrectKind::Correct); @@ -2016,10 +2031,10 @@ local target: ((number, number) -> string) & ((string) -> number)) local one = 4 local two = "hello" -return target(1, o) +return target(1, o@1) )"); - ac = autocomplete(5, 18); + ac = autocomplete('1'); CHECK(ac.entryMap.count("one")); CHECK(ac.entryMap["one"].typeCorrect == TypeCorrectKind::Correct); @@ -2032,10 +2047,10 @@ TEST_CASE_FIXTURE(ACFixture, "optional_members") local a = { x = 2, y = 3 } type A = typeof(a) local b: A? = a -return b. +return b.@1 )"); - auto ac = autocomplete(4, 9); + auto ac = autocomplete('1'); CHECK_EQ(2, ac.entryMap.size()); CHECK(ac.entryMap.count("x")); @@ -2045,10 +2060,10 @@ return b. local a = { x = 2, y = 3 } type A = typeof(a) local b: nil | A = a -return b. +return b.@1 )"); - ac = autocomplete(4, 9); + ac = autocomplete('1'); CHECK_EQ(2, ac.entryMap.size()); CHECK(ac.entryMap.count("x")); @@ -2056,10 +2071,10 @@ return b. check(R"( local b: nil | nil -return b. +return b.@1 )"); - ac = autocomplete(2, 9); + ac = autocomplete('1'); CHECK_EQ(0, ac.entryMap.size()); } @@ -2067,26 +2082,26 @@ return b. TEST_CASE_FIXTURE(ACFixture, "no_function_name_suggestions") { check(R"( -function na +function na@1 )"); - auto ac = autocomplete(1, 11); + auto ac = autocomplete('1'); CHECK(ac.entryMap.empty()); check(R"( -local function +local function @1 )"); - ac = autocomplete(1, 15); + ac = autocomplete('1'); CHECK(ac.entryMap.empty()); check(R"( -local function na +local function na@1 )"); - ac = autocomplete(1, 17); + ac = autocomplete('1'); CHECK(ac.entryMap.empty()); } @@ -2095,20 +2110,20 @@ TEST_CASE_FIXTURE(ACFixture, "skip_current_local") { check(R"( local other = 1 -local name = na +local name = na@1 )"); - auto ac = autocomplete(2, 15); + auto ac = autocomplete('1'); CHECK(!ac.entryMap.count("name")); CHECK(ac.entryMap.count("other")); check(R"( local other = 1 -local name, test = na +local name, test = na@1 )"); - ac = autocomplete(2, 21); + ac = autocomplete('1'); CHECK(!ac.entryMap.count("name")); CHECK(!ac.entryMap.count("test")); @@ -2119,26 +2134,26 @@ TEST_CASE_FIXTURE(ACFixture, "keyword_members") { check(R"( local a = { done = 1, forever = 2 } -local b = a.do -local c = a.for -local d = a. +local b = a.do@1 +local c = a.for@2 +local d = a.@3 do end )"); - auto ac = autocomplete(2, 14); + auto ac = autocomplete('1'); CHECK_EQ(2, ac.entryMap.size()); CHECK(ac.entryMap.count("done")); CHECK(ac.entryMap.count("forever")); - ac = autocomplete(3, 15); + ac = autocomplete('2'); CHECK_EQ(2, ac.entryMap.size()); CHECK(ac.entryMap.count("done")); CHECK(ac.entryMap.count("forever")); - ac = autocomplete(4, 12); + ac = autocomplete('3'); CHECK_EQ(2, ac.entryMap.size()); CHECK(ac.entryMap.count("done")); @@ -2150,10 +2165,10 @@ TEST_CASE_FIXTURE(ACFixture, "keyword_methods") check(R"( local a = {} function a:done() end -local b = a:do +local b = a:do@1 )"); - auto ac = autocomplete(3, 14); + auto ac = autocomplete('1'); CHECK_EQ(1, ac.entryMap.size()); CHECK(ac.entryMap.count("done")); @@ -2247,29 +2262,29 @@ local elsewhere = false local doover = false local endurance = true -if 1 then -else +if 1 then@1 +else@2 end -while false do +while false do@3 end -repeat +repeat@4 until )"); - auto ac = autocomplete(6, 9); + auto ac = autocomplete('1'); CHECK(ac.entryMap.size() == 1); CHECK(ac.entryMap.count("then")); - ac = autocomplete(7, 4); + ac = autocomplete('2'); CHECK(ac.entryMap.count("else")); CHECK(ac.entryMap.count("elseif")); - ac = autocomplete(10, 14); + ac = autocomplete('3'); CHECK(ac.entryMap.count("do")); - ac = autocomplete(13, 6); + ac = autocomplete('4'); CHECK(ac.entryMap.count("do")); // FIXME: ideally we want to handle start and end of all statements as well @@ -2284,11 +2299,11 @@ local elsewhere = false if true then return 1 -el +el@1 end )"); - auto ac = autocomplete(5, 2); + auto ac = autocomplete('1'); CHECK(ac.entryMap.count("else")); CHECK(ac.entryMap.count("elseif")); CHECK(ac.entryMap.count("elsewhere") == 0); @@ -2300,11 +2315,11 @@ if true then return 1 else return 2 -el +el@1 end )"); - ac = autocomplete(7, 2); + ac = autocomplete('1'); CHECK(ac.entryMap.count("else") == 0); CHECK(ac.entryMap.count("elseif") == 0); CHECK(ac.entryMap.count("elsewhere")); @@ -2316,10 +2331,10 @@ if true then print("1") elif true then print("2") -el +el@1 end )"); - ac = autocomplete(7, 2); + ac = autocomplete('1'); CHECK(ac.entryMap.count("else")); CHECK(ac.entryMap.count("elseif")); CHECK(ac.entryMap.count("elsewhere")); @@ -2360,30 +2375,30 @@ TEST_CASE_FIXTURE(ACFixture, "suggest_table_keys") { check(R"( type Test = { first: number, second: number } -local t: Test = { f } +local t: Test = { f@1 } )"); - auto ac = autocomplete(2, 19); + auto ac = autocomplete('1'); CHECK(ac.entryMap.count("first")); CHECK(ac.entryMap.count("second")); // Intersection check(R"( type Test = { first: number } & { second: number } -local t: Test = { f } +local t: Test = { f@1 } )"); - ac = autocomplete(2, 19); + ac = autocomplete('1'); CHECK(ac.entryMap.count("first")); CHECK(ac.entryMap.count("second")); // Union check(R"( type Test = { first: number, second: number } | { second: number, third: number } -local t: Test = { s } +local t: Test = { s@1 } )"); - ac = autocomplete(2, 19); + ac = autocomplete('1'); CHECK(ac.entryMap.count("second")); CHECK(!ac.entryMap.count("first")); CHECK(!ac.entryMap.count("third")); @@ -2391,60 +2406,60 @@ local t: Test = { s } // No parenthesis suggestion check(R"( type Test = { first: (number) -> number, second: number } -local t: Test = { f } +local t: Test = { f@1 } )"); - ac = autocomplete(2, 19); + ac = autocomplete('1'); CHECK(ac.entryMap.count("first")); CHECK(ac.entryMap["first"].parens == ParenthesesRecommendation::None); // When key is changed check(R"( type Test = { first: number, second: number } -local t: Test = { f = 2 } +local t: Test = { f@1 = 2 } )"); - ac = autocomplete(2, 19); + ac = autocomplete('1'); CHECK(ac.entryMap.count("first")); CHECK(ac.entryMap.count("second")); // Alternative key syntax check(R"( type Test = { first: number, second: number } -local t: Test = { ["f"] } +local t: Test = { ["f@1"] } )"); - ac = autocomplete(2, 21); + ac = autocomplete('1'); CHECK(ac.entryMap.count("first")); CHECK(ac.entryMap.count("second")); // Not an alternative key syntax check(R"( type Test = { first: number, second: number } -local t: Test = { "f" } +local t: Test = { "f@1" } )"); - ac = autocomplete(2, 20); + ac = autocomplete('1'); CHECK(!ac.entryMap.count("first")); CHECK(!ac.entryMap.count("second")); // Skip keys that are already defined check(R"( type Test = { first: number, second: number } -local t: Test = { first = 2, s } +local t: Test = { first = 2, s@1 } )"); - ac = autocomplete(2, 30); + ac = autocomplete('1'); CHECK(!ac.entryMap.count("first")); CHECK(ac.entryMap.count("second")); // Don't skip active key check(R"( type Test = { first: number, second: number } -local t: Test = { first } +local t: Test = { first@1 } )"); - ac = autocomplete(2, 23); + ac = autocomplete('1'); CHECK(ac.entryMap.count("first")); CHECK(ac.entryMap.count("second")); @@ -2452,22 +2467,22 @@ local t: Test = { first } check(R"( local t = { { first = 5, second = 10 }, - { f } + { f@1 } } )"); - ac = autocomplete(3, 7); + ac = autocomplete('1'); CHECK(ac.entryMap.count("first")); CHECK(ac.entryMap.count("second")); check(R"( local t = { [2] = { first = 5, second = 10 }, - [5] = { f } + [5] = { f@1 } } )"); - ac = autocomplete(3, 13); + ac = autocomplete('1'); CHECK(ac.entryMap.count("first")); CHECK(ac.entryMap.count("second")); } @@ -2502,15 +2517,15 @@ TEST_CASE_FIXTURE(ACFixture, "autocomplete_ifelse_expressions") local temp = false local even = true; local a = true -a = if t1@emp then t -a = if temp t2@ -a = if temp then e3@ -a = if temp then even e4@ -a = if temp then even elseif t5@ -a = if temp then even elseif true t6@ -a = if temp then even elseif true then t7@ -a = if temp then even elseif true then temp e8@ -a = if temp then even elseif true then temp else e9@ +a = if t@1emp then t +a = if temp t@2 +a = if temp then e@3 +a = if temp then even e@4 +a = if temp then even elseif t@5 +a = if temp then even elseif true t@6 +a = if temp then even elseif true then t@7 +a = if temp then even elseif true then temp e@8 +a = if temp then even elseif true then temp else e@9 )"); auto ac = autocomplete('1'); @@ -2573,4 +2588,20 @@ a = if temp then even elseif true then temp else e9@ } } +TEST_CASE_FIXTURE(ACFixture, "autocomplete_explicit_type_pack") +{ + ScopedFastFlag luauTypeAliasPacks("LuauTypeAliasPacks", true); + ScopedFastFlag luauParseTypePackTypeParameters("LuauParseTypePackTypeParameters", true); + + check(R"( +type A = () -> T... +local a: A<(number, s@1> + )"); + + auto ac = autocomplete('1'); + + CHECK(ac.entryMap.count("number")); + CHECK(ac.entryMap.count("string")); +} + TEST_SUITE_END(); diff --git a/tests/Fixture.cpp b/tests/Fixture.cpp index 26bc77f7..29c33f7c 100644 --- a/tests/Fixture.cpp +++ b/tests/Fixture.cpp @@ -32,6 +32,55 @@ std::optional TestFileResolver::fromAstFragment(AstExpr* expr) const return std::nullopt; } +std::optional TestFileResolver::resolveModule(const ModuleInfo* context, AstExpr* expr) +{ + if (AstExprGlobal* g = expr->as()) + { + if (g->name == "game") + return ModuleInfo{"game"}; + if (g->name == "workspace") + return ModuleInfo{"workspace"}; + if (g->name == "script") + return context ? std::optional(*context) : std::nullopt; + } + else if (AstExprIndexName* i = expr->as(); i && context) + { + if (i->index == "Parent") + { + std::string_view view = context->name; + size_t lastSeparatorIndex = view.find_last_of('/'); + + if (lastSeparatorIndex == std::string_view::npos) + return std::nullopt; + + return ModuleInfo{ModuleName(view.substr(0, lastSeparatorIndex)), context->optional}; + } + else + { + return ModuleInfo{context->name + '/' + i->index.value, context->optional}; + } + } + else if (AstExprIndexExpr* i = expr->as(); i && context) + { + if (AstExprConstantString* index = i->index->as()) + { + return ModuleInfo{context->name + '/' + std::string(index->value.data, index->value.size), context->optional}; + } + } + else if (AstExprCall* call = expr->as(); call && call->self && call->args.size >= 1 && context) + { + if (AstExprConstantString* index = call->args.data[0]->as()) + { + AstName func = call->func->as()->index; + + if (func == "GetService" && context->name == "game") + return ModuleInfo{"game/" + std::string(index->value.data, index->value.size)}; + } + } + + return std::nullopt; +} + ModuleName TestFileResolver::concat(const ModuleName& lhs, std::string_view rhs) const { return lhs + "/" + ModuleName(rhs); diff --git a/tests/Fixture.h b/tests/Fixture.h index c6294b01..1480a7f6 100644 --- a/tests/Fixture.h +++ b/tests/Fixture.h @@ -65,6 +65,8 @@ struct TestFileResolver } std::optional fromAstFragment(AstExpr* expr) const override; + std::optional resolveModule(const ModuleInfo* context, AstExpr* expr) override; + ModuleName concat(const ModuleName& lhs, std::string_view rhs) const override; std::optional getParentModuleName(const ModuleName& name) const override; diff --git a/tests/Frontend.test.cpp b/tests/Frontend.test.cpp index 3f33a5d1..fbfec636 100644 --- a/tests/Frontend.test.cpp +++ b/tests/Frontend.test.cpp @@ -58,6 +58,35 @@ struct NaiveFileResolver : NullFileResolver return std::nullopt; } + std::optional resolveModule(const ModuleInfo* context, AstExpr* expr) override + { + if (AstExprGlobal* g = expr->as()) + { + if (g->name == "Modules") + return ModuleInfo{"Modules"}; + + if (g->name == "game") + return ModuleInfo{"game"}; + } + else if (AstExprIndexName* i = expr->as()) + { + if (context) + return ModuleInfo{context->name + '/' + i->index.value, context->optional}; + } + else if (AstExprCall* call = expr->as(); call && call->self && call->args.size >= 1 && context) + { + if (AstExprConstantString* index = call->args.data[0]->as()) + { + AstName func = call->func->as()->index; + + if (func == "GetService" && context->name == "game") + return ModuleInfo{"game/" + std::string(index->value.data, index->value.size)}; + } + } + + return std::nullopt; + } + ModuleName concat(const ModuleName& lhs, std::string_view rhs) const override { return lhs + "/" + ModuleName(rhs); @@ -528,7 +557,7 @@ TEST_CASE_FIXTURE(FrontendFixture, "ignore_require_to_nonexistent_file") { fileResolver.source["Modules/A"] = R"( local Modules = script - local B = require(Modules.B :: any) + local B = require(Modules.B) :: any )"; CheckResult result = frontend.check("Modules/A"); diff --git a/tests/Linter.test.cpp b/tests/Linter.test.cpp index c8eff399..a9ed139f 100644 --- a/tests/Linter.test.cpp +++ b/tests/Linter.test.cpp @@ -1400,6 +1400,8 @@ end TEST_CASE_FIXTURE(Fixture, "TableOperations") { + ScopedFastFlag sff("LuauLinterTableMoveZero", true); + LintResult result = lintTyped(R"( local t = {} local tt = {} @@ -1417,9 +1419,12 @@ table.remove(t, 0) table.remove(t, #t-1) table.insert(t, string.find("hello", "h")) + +table.move(t, 0, #t, 1, tt) +table.move(t, 1, #t, 0, tt) )"); - REQUIRE_EQ(result.warnings.size(), 6); + REQUIRE_EQ(result.warnings.size(), 8); CHECK_EQ(result.warnings[0].text, "table.insert will insert the value before the last element, which is likely a bug; consider removing the " "second argument or wrap it in parentheses to silence"); CHECK_EQ(result.warnings[1].text, "table.insert will append the value to the table; consider removing the second argument for efficiency"); @@ -1429,6 +1434,8 @@ table.insert(t, string.find("hello", "h")) "second argument or wrap it in parentheses to silence"); CHECK_EQ(result.warnings[5].text, "table.insert may change behavior if the call returns more than one result; consider adding parentheses around second argument"); + CHECK_EQ(result.warnings[6].text, "table.move uses index 0 but arrays are 1-based; did you mean 1 instead?"); + CHECK_EQ(result.warnings[7].text, "table.move uses index 0 but arrays are 1-based; did you mean 1 instead?"); } TEST_CASE_FIXTURE(Fixture, "DuplicateConditions") diff --git a/tests/Module.test.cpp b/tests/Module.test.cpp index 1b146ed2..18f55d2c 100644 --- a/tests/Module.test.cpp +++ b/tests/Module.test.cpp @@ -1,5 +1,6 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #include "Luau/Module.h" +#include "Luau/Scope.h" #include "Fixture.h" diff --git a/tests/NonstrictMode.test.cpp b/tests/NonstrictMode.test.cpp index f3c76d55..931a8403 100644 --- a/tests/NonstrictMode.test.cpp +++ b/tests/NonstrictMode.test.cpp @@ -1,5 +1,6 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #include "Luau/Parser.h" +#include "Luau/Scope.h" #include "Luau/TypeInfer.h" #include "Luau/TypeVar.h" diff --git a/tests/Parser.test.cpp b/tests/Parser.test.cpp index cb03a7bd..a80718e4 100644 --- a/tests/Parser.test.cpp +++ b/tests/Parser.test.cpp @@ -2519,4 +2519,19 @@ TEST_CASE_FIXTURE(Fixture, "parse_if_else_expression") } } +TEST_CASE_FIXTURE(Fixture, "parse_type_pack_type_parameters") +{ + ScopedFastFlag luauParseGenericFunctions("LuauParseGenericFunctions", true); + ScopedFastFlag luauParseTypePackTypeParameters("LuauParseTypePackTypeParameters", true); + + AstStat* stat = parse(R"( +type Packed = () -> T... + +type A = Packed +type B = Packed<...number> +type C = Packed<(number, X...)> + )"); + REQUIRE(stat != nullptr); +} + TEST_SUITE_END(); diff --git a/tests/RequireTracer.test.cpp b/tests/RequireTracer.test.cpp index cbd4af29..b9fd04d6 100644 --- a/tests/RequireTracer.test.cpp +++ b/tests/RequireTracer.test.cpp @@ -57,6 +57,7 @@ TEST_CASE_FIXTURE(RequireTracerFixture, "trace_local") { AstStatBlock* block = parse(R"( local m = workspace.Foo.Bar.Baz + require(m) )"); RequireTraceResult result = traceRequires(&fileResolver, block, "ModuleName"); @@ -70,22 +71,22 @@ TEST_CASE_FIXTURE(RequireTracerFixture, "trace_local") AstExprIndexName* value = loc->values.data[0]->as(); REQUIRE(value); REQUIRE(result.exprs.contains(value)); - CHECK_EQ("workspace/Foo/Bar/Baz", result.exprs[value]); + CHECK_EQ("workspace/Foo/Bar/Baz", result.exprs[value].name); value = value->expr->as(); REQUIRE(value); REQUIRE(result.exprs.contains(value)); - CHECK_EQ("workspace/Foo/Bar", result.exprs[value]); + CHECK_EQ("workspace/Foo/Bar", result.exprs[value].name); value = value->expr->as(); REQUIRE(value); REQUIRE(result.exprs.contains(value)); - CHECK_EQ("workspace/Foo", result.exprs[value]); + CHECK_EQ("workspace/Foo", result.exprs[value].name); AstExprGlobal* workspace = value->expr->as(); REQUIRE(workspace); REQUIRE(result.exprs.contains(workspace)); - CHECK_EQ("workspace", result.exprs[workspace]); + CHECK_EQ("workspace", result.exprs[workspace].name); } TEST_CASE_FIXTURE(RequireTracerFixture, "trace_transitive_local") @@ -93,9 +94,10 @@ TEST_CASE_FIXTURE(RequireTracerFixture, "trace_transitive_local") AstStatBlock* block = parse(R"( local m = workspace.Foo.Bar.Baz local n = m.Quux + require(n) )"); - REQUIRE_EQ(2, block->body.size); + REQUIRE_EQ(3, block->body.size); RequireTraceResult result = traceRequires(&fileResolver, block, "ModuleName"); @@ -104,13 +106,13 @@ TEST_CASE_FIXTURE(RequireTracerFixture, "trace_transitive_local") REQUIRE_EQ(1, local->vars.size); REQUIRE(result.exprs.contains(local->values.data[0])); - CHECK_EQ("workspace/Foo/Bar/Baz/Quux", result.exprs[local->values.data[0]]); + CHECK_EQ("workspace/Foo/Bar/Baz/Quux", result.exprs[local->values.data[0]].name); } TEST_CASE_FIXTURE(RequireTracerFixture, "trace_function_arguments") { AstStatBlock* block = parse(R"( - local M = require(workspace.Game.Thing, workspace.Something.Else) + local M = require(workspace.Game.Thing) )"); REQUIRE_EQ(1, block->body.size); @@ -124,52 +126,9 @@ TEST_CASE_FIXTURE(RequireTracerFixture, "trace_function_arguments") AstExprCall* call = local->values.data[0]->as(); REQUIRE(call != nullptr); - REQUIRE_EQ(2, call->args.size); - - CHECK_EQ("workspace/Game/Thing", result.exprs[call->args.data[0]]); - CHECK_EQ("workspace/Something/Else", result.exprs[call->args.data[1]]); -} - -TEST_CASE_FIXTURE(RequireTracerFixture, "follow_GetService_calls") -{ - AstStatBlock* block = parse(R"( - local R = game:GetService('ReplicatedStorage').Roact - local Roact = require(R) - )"); - REQUIRE_EQ(2, block->body.size); - - RequireTraceResult result = traceRequires(&fileResolver, block, "ModuleName"); - - AstStatLocal* local = block->body.data[0]->as(); - REQUIRE(local != nullptr); - - CHECK_EQ("game/ReplicatedStorage/Roact", result.exprs[local->values.data[0]]); - - AstStatLocal* local2 = block->body.data[1]->as(); - REQUIRE(local2 != nullptr); - REQUIRE_EQ(1, local2->values.size); - - AstExprCall* call = local2->values.data[0]->as(); - REQUIRE(call != nullptr); REQUIRE_EQ(1, call->args.size); - CHECK_EQ("game/ReplicatedStorage/Roact", result.exprs[call->args.data[0]]); -} - -TEST_CASE_FIXTURE(RequireTracerFixture, "follow_WaitForChild_calls") -{ - ScopedFastFlag luauTraceRequireLookupChild("LuauTraceRequireLookupChild", true); - - AstStatBlock* block = parse(R"( -local A = require(workspace:WaitForChild('ReplicatedStorage').Content) -local B = require(workspace:FindFirstChild('ReplicatedFirst').Data) - )"); - - RequireTraceResult result = traceRequires(&fileResolver, block, "ModuleName"); - - REQUIRE_EQ(2, result.requires.size()); - CHECK_EQ("workspace/ReplicatedStorage/Content", result.requires[0].first); - CHECK_EQ("workspace/ReplicatedFirst/Data", result.requires[1].first); + CHECK_EQ("workspace/Game/Thing", result.exprs[call->args.data[0]].name); } TEST_CASE_FIXTURE(RequireTracerFixture, "follow_typeof") @@ -200,22 +159,23 @@ TEST_CASE_FIXTURE(RequireTracerFixture, "follow_typeof") REQUIRE(call != nullptr); REQUIRE_EQ(1, call->args.size); - CHECK_EQ("workspace/CoolThing", result.exprs[call->args.data[0]]); + CHECK_EQ("workspace/CoolThing", result.exprs[call->args.data[0]].name); } TEST_CASE_FIXTURE(RequireTracerFixture, "follow_string_indexexpr") { AstStatBlock* block = parse(R"( local R = game["Test"] + require(R) )"); - REQUIRE_EQ(1, block->body.size); + REQUIRE_EQ(2, block->body.size); RequireTraceResult result = traceRequires(&fileResolver, block, "ModuleName"); AstStatLocal* local = block->body.data[0]->as(); REQUIRE(local != nullptr); - CHECK_EQ("game/Test", result.exprs[local->values.data[0]]); + CHECK_EQ("game/Test", result.exprs[local->values.data[0]].name); } TEST_SUITE_END(); diff --git a/tests/ToString.test.cpp b/tests/ToString.test.cpp index d7d68c46..e18bf7cd 100644 --- a/tests/ToString.test.cpp +++ b/tests/ToString.test.cpp @@ -1,5 +1,6 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +#include "Luau/Scope.h" #include "Luau/ToString.h" #include "Fixture.h" @@ -416,8 +417,6 @@ function foo(a, b) return a(b) end TEST_CASE_FIXTURE(Fixture, "toString_the_boundTo_table_type_contained_within_a_TypePack") { - ScopedFastFlag sff{"LuauToStringFollowsBoundTo", true}; - TypeVar tv1{TableTypeVar{}}; TableTypeVar* ttv = getMutable(&tv1); ttv->state = TableState::Sealed; diff --git a/tests/TypeInfer.aliases.test.cpp b/tests/TypeInfer.aliases.test.cpp new file mode 100644 index 00000000..045f0230 --- /dev/null +++ b/tests/TypeInfer.aliases.test.cpp @@ -0,0 +1,557 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details + +#include "Fixture.h" + +#include "doctest.h" +#include "Luau/BuiltinDefinitions.h" + +using namespace Luau; + +TEST_SUITE_BEGIN("TypeAliases"); + +TEST_CASE_FIXTURE(Fixture, "cyclic_function_type_in_type_alias") +{ + ScopedFastFlag sff{"LuauOccursCheckOkWithRecursiveFunctions", true}; + + CheckResult result = check(R"( + type F = () -> F? + local function f() + return f + end + + local g: F = f + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + CHECK_EQ("t1 where t1 = () -> t1?", toString(requireType("g"))); +} + +TEST_CASE_FIXTURE(Fixture, "cyclic_types_of_named_table_fields_do_not_expand_when_stringified") +{ + CheckResult result = check(R"( + --!strict + type Node = { Parent: Node?; } + local node: Node; + node.Parent = 1 + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + TypeMismatch* tm = get(result.errors[0]); + REQUIRE(tm); + CHECK_EQ("Node?", toString(tm->wantedType)); + CHECK_EQ(typeChecker.numberType, tm->givenType); +} + +TEST_CASE_FIXTURE(Fixture, "mutually_recursive_types") +{ + CheckResult result = check(R"( + --!strict + type T = { f: a, g: U } + type U = { h: a, i: T? } + local x: T = { f = 37, g = { h = 5, i = nil } } + x.g.i = x + local y: T = { f = "hi", g = { h = "lo", i = nil } } + y.g.i = y + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "mutually_recursive_types_errors") +{ + CheckResult result = check(R"( + --!strict + type T = { f: a, g: U } + type U = { h: b, i: T? } + local x: T = { f = 37, g = { h = 5, i = nil } } + x.g.i = x + local y: T = { f = "hi", g = { h = 5, i = nil } } + y.g.i = y + )"); + + LUAU_REQUIRE_ERRORS(result); + + // We had a UAF in this example caused by not cloning type function arguments + ModulePtr module = frontend.moduleResolver.getModule("MainModule"); + unfreeze(module->interfaceTypes); + copyErrors(module->errors, module->interfaceTypes); + freeze(module->interfaceTypes); + module->internalTypes.clear(); + module->astTypes.clear(); + + // Make sure the error strings don't include "VALUELESS" + for (auto error : module->errors) + CHECK_MESSAGE(toString(error).find("VALUELESS") == std::string::npos, toString(error)); +} + +TEST_CASE_FIXTURE(Fixture, "use_table_name_and_generic_params_in_errors") +{ + CheckResult result = check(R"( + type Pair = {first: T, second: U} + local a: Pair + local b: Pair + + a = b + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + TypeMismatch* tm = get(result.errors[0]); + REQUIRE(tm); + + CHECK_EQ("Pair", toString(tm->wantedType)); + CHECK_EQ("Pair", toString(tm->givenType)); +} + +TEST_CASE_FIXTURE(Fixture, "dont_stop_typechecking_after_reporting_duplicate_type_definition") +{ + CheckResult result = check(R"( + type A = number + type A = string -- Redefinition of type 'A', previously defined at line 1 + local foo: string = 1 -- No "Type 'number' could not be converted into 'string'" + )"); + + LUAU_REQUIRE_ERROR_COUNT(2, result); +} + +TEST_CASE_FIXTURE(Fixture, "stringify_type_alias_of_recursive_template_table_type") +{ + CheckResult result = check(R"( + type Table = { a: T } + type Wrapped = Table + local l: Wrapped = 2 + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + TypeMismatch* tm = get(result.errors[0]); + REQUIRE(tm); + CHECK_EQ("Wrapped", toString(tm->wantedType)); + CHECK_EQ(typeChecker.numberType, tm->givenType); +} + +TEST_CASE_FIXTURE(Fixture, "stringify_type_alias_of_recursive_template_table_type2") +{ + CheckResult result = check(R"( + type Table = { a: T } + type Wrapped = (Table) -> string + local l: Wrapped = 2 + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + TypeMismatch* tm = get(result.errors[0]); + REQUIRE(tm); + CHECK_EQ("t1 where t1 = ({| a: t1 |}) -> string", toString(tm->wantedType)); + CHECK_EQ(typeChecker.numberType, tm->givenType); +} + +// Check that recursive intersection type doesn't generate an OOM +TEST_CASE_FIXTURE(Fixture, "cli_38393_recursive_intersection_oom") +{ + CheckResult result = check(R"( + function _(l0:(t0)&((t0)&(((t0)&((t0)->()))->(typeof(_),typeof(# _)))),l39,...):any + end + type t0 = ((typeof(_))&((t0)&(((typeof(_))&(t0))->typeof(_))),{n163:any,})->(any,typeof(_)) + _(_) + )"); + + LUAU_REQUIRE_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "type_alias_fwd_declaration_is_precise") +{ + CheckResult result = check(R"( + local foo: Id = 1 + type Id = T + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "corecursive_types_generic") +{ + const std::string code = R"( + type A = {v:T, b:B} + type B = {v:T, a:A} + local aa:A + local bb = aa + )"; + + const std::string expected = R"( + type A = {v:T, b:B} + type B = {v:T, a:A} + local aa:A + local bb:A=aa + )"; + + CHECK_EQ(expected, decorateWithTypes(code)); + CheckResult result = check(code); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "corecursive_function_types") +{ + ScopedFastFlag sff{"LuauOccursCheckOkWithRecursiveFunctions", true}; + + CheckResult result = check(R"( + type A = () -> (number, B) + type B = () -> (string, A) + local a: A + local b: B + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ("t1 where t1 = () -> (number, () -> (string, t1))", toString(requireType("a"))); + CHECK_EQ("t1 where t1 = () -> (string, () -> (number, t1))", toString(requireType("b"))); +} + +TEST_CASE_FIXTURE(Fixture, "generic_param_remap") +{ + const std::string code = R"( + -- An example of a forwarded use of a type that has different type arguments than parameters + type A = {t:T, u:U, next:A?} + local aa:A = { t = 5, u = 'hi', next = { t = 'lo', u = 8 } } + local bb = aa + )"; + + const std::string expected = R"( + + type A = {t:T, u:U, next:A?} + local aa:A = { t = 5, u = 'hi', next = { t = 'lo', u = 8 } } + local bb:A=aa + )"; + + CHECK_EQ(expected, decorateWithTypes(code)); + CheckResult result = check(code); + + LUAU_REQUIRE_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "export_type_and_type_alias_are_duplicates") +{ + CheckResult result = check(R"( + export type Foo = number + type Foo = number + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + auto dtd = get(result.errors[0]); + REQUIRE(dtd); + CHECK_EQ(dtd->name, "Foo"); +} + +TEST_CASE_FIXTURE(Fixture, "stringify_optional_parameterized_alias") +{ + ScopedFastFlag sffs3{"LuauGenericFunctions", true}; + ScopedFastFlag sffs4{"LuauParseGenericFunctions", true}; + + CheckResult result = check(R"( + type Node = { value: T, child: Node? } + + local function visitor(node: Node?) + local a: Node + + if node then + a = node.child -- Observe the output of the error message. + end + end + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + auto e = get(result.errors[0]); + CHECK_EQ("Node?", toString(e->givenType)); + CHECK_EQ("Node", toString(e->wantedType)); +} + +TEST_CASE_FIXTURE(Fixture, "general_require_multi_assign") +{ + fileResolver.source["workspace/A"] = R"( + export type myvec2 = {x: number, y: number} + return {} + )"; + + fileResolver.source["workspace/B"] = R"( + export type myvec3 = {x: number, y: number, z: number} + return {} + )"; + + fileResolver.source["workspace/C"] = R"( + local Foo, Bar = require(workspace.A), require(workspace.B) + + local a: Foo.myvec2 + local b: Bar.myvec3 + )"; + + CheckResult result = frontend.check("workspace/C"); + LUAU_REQUIRE_NO_ERRORS(result); + ModulePtr m = frontend.moduleResolver.modules["workspace/C"]; + + REQUIRE(m != nullptr); + + std::optional aTypeId = lookupName(m->getModuleScope(), "a"); + REQUIRE(aTypeId); + const Luau::TableTypeVar* aType = get(follow(*aTypeId)); + REQUIRE(aType); + REQUIRE(aType->props.size() == 2); + + std::optional bTypeId = lookupName(m->getModuleScope(), "b"); + REQUIRE(bTypeId); + const Luau::TableTypeVar* bType = get(follow(*bTypeId)); + REQUIRE(bType); + REQUIRE(bType->props.size() == 3); +} + +TEST_CASE_FIXTURE(Fixture, "type_alias_import_mutation") +{ + CheckResult result = check("type t10 = typeof(table)"); + LUAU_REQUIRE_NO_ERRORS(result); + + TypeId ty = getGlobalBinding(frontend.typeChecker, "table"); + CHECK_EQ(toString(ty), "table"); + + const TableTypeVar* ttv = get(ty); + REQUIRE(ttv); + + CHECK(ttv->instantiatedTypeParams.empty()); +} + +TEST_CASE_FIXTURE(Fixture, "type_alias_local_mutation") +{ + CheckResult result = check(R"( +type Cool = { a: number, b: string } +local c: Cool = { a = 1, b = "s" } +type NotCool = Cool +)"); + LUAU_REQUIRE_NO_ERRORS(result); + + std::optional ty = requireType("c"); + REQUIRE(ty); + CHECK_EQ(toString(*ty), "Cool"); + + const TableTypeVar* ttv = get(*ty); + REQUIRE(ttv); + + CHECK(ttv->instantiatedTypeParams.empty()); +} + +TEST_CASE_FIXTURE(Fixture, "type_alias_local_rename") +{ + CheckResult result = check(R"( +type Cool = { a: number, b: string } +type NotCool = Cool +local c: Cool = { a = 1, b = "s" } +local d: NotCool = { a = 1, b = "s" } +)"); + LUAU_REQUIRE_NO_ERRORS(result); + + std::optional ty = requireType("c"); + REQUIRE(ty); + CHECK_EQ(toString(*ty), "Cool"); + + ty = requireType("d"); + REQUIRE(ty); + CHECK_EQ(toString(*ty), "NotCool"); +} + +TEST_CASE_FIXTURE(Fixture, "type_alias_local_synthetic_mutation") +{ + CheckResult result = check(R"( +local c = { a = 1, b = "s" } +type Cool = typeof(c) +)"); + LUAU_REQUIRE_NO_ERRORS(result); + + std::optional ty = requireType("c"); + REQUIRE(ty); + + const TableTypeVar* ttv = get(*ty); + REQUIRE(ttv); + CHECK_EQ(ttv->name, "Cool"); +} + +TEST_CASE_FIXTURE(Fixture, "type_alias_of_an_imported_recursive_type") +{ + fileResolver.source["game/A"] = R"( +export type X = { a: number, b: X? } +return {} + )"; + + CheckResult aResult = frontend.check("game/A"); + LUAU_REQUIRE_NO_ERRORS(aResult); + + CheckResult bResult = check(R"( +local Import = require(game.A) +type X = Import.X + )"); + LUAU_REQUIRE_NO_ERRORS(bResult); + + std::optional ty1 = lookupImportedType("Import", "X"); + REQUIRE(ty1); + + std::optional ty2 = lookupType("X"); + REQUIRE(ty2); + + CHECK_EQ(follow(*ty1), follow(*ty2)); +} + +TEST_CASE_FIXTURE(Fixture, "type_alias_of_an_imported_recursive_generic_type") +{ + fileResolver.source["game/A"] = R"( +export type X = { a: T, b: U, C: X? } +return {} + )"; + + CheckResult aResult = frontend.check("game/A"); + LUAU_REQUIRE_NO_ERRORS(aResult); + + CheckResult bResult = check(R"( +local Import = require(game.A) +type X = Import.X + )"); + LUAU_REQUIRE_NO_ERRORS(bResult); + + std::optional ty1 = lookupImportedType("Import", "X"); + REQUIRE(ty1); + + std::optional ty2 = lookupType("X"); + REQUIRE(ty2); + + CHECK_EQ(toString(*ty1, {true}), toString(*ty2, {true})); + + bResult = check(R"( +local Import = require(game.A) +type X = Import.X + )"); + LUAU_REQUIRE_NO_ERRORS(bResult); + + ty1 = lookupImportedType("Import", "X"); + REQUIRE(ty1); + + ty2 = lookupType("X"); + REQUIRE(ty2); + + CHECK_EQ(toString(*ty1, {true}), "t1 where t1 = {| C: t1?, a: T, b: U |}"); + CHECK_EQ(toString(*ty2, {true}), "{| C: t1, a: U, b: T |} where t1 = {| C: t1, a: U, b: T |}?"); +} + +TEST_CASE_FIXTURE(Fixture, "module_export_free_type_leak") +{ + CheckResult result = check(R"( +function get() + return function(obj) return true end +end + +export type f = typeof(get()) + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "module_export_wrapped_free_type_leak") +{ + CheckResult result = check(R"( +function get() + return {a = 1, b = function(obj) return true end} +end + +export type f = typeof(get()) + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + + +TEST_CASE_FIXTURE(Fixture, "mutually_recursive_types_restriction_ok") +{ + CheckResult result = check(R"( + type Tree = { data: T, children: Forest } + type Forest = {Tree} + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "mutually_recursive_types_restriction_not_ok_1") +{ + ScopedFastFlag sff{"LuauRecursiveTypeParameterRestriction", true}; + + CheckResult result = check(R"( + -- OK because forwarded types are used with their parameters. + type Tree = { data: T, children: Forest } + type Forest = {Tree<{T}>} + )"); + + LUAU_REQUIRE_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "mutually_recursive_types_restriction_not_ok_2") +{ + ScopedFastFlag sff{"LuauRecursiveTypeParameterRestriction", true}; + + CheckResult result = check(R"( + -- Not OK because forwarded types are used with different types than their parameters. + type Forest = {Tree<{T}>} + type Tree = { data: T, children: Forest } + )"); + + LUAU_REQUIRE_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "mutually_recursive_types_swapsies_ok") +{ + CheckResult result = check(R"( + type Tree1 = { data: T, children: {Tree2} } + type Tree2 = { data: U, children: {Tree1} } + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "mutually_recursive_types_swapsies_not_ok") +{ + ScopedFastFlag sff{"LuauRecursiveTypeParameterRestriction", true}; + + CheckResult result = check(R"( + type Tree1 = { data: T, children: {Tree2} } + type Tree2 = { data: U, children: {Tree1} } + )"); + + LUAU_REQUIRE_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "free_variables_from_typeof_in_aliases") +{ + CheckResult result = check(R"( + function f(x) return x[1] end + -- x has type X? for a free type variable X + local x = f ({}) + type ContainsFree = { this: a, that: typeof(x) } + type ContainsContainsFree = { that: ContainsFree } + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "non_recursive_aliases_that_reuse_a_generic_name") +{ + ScopedFastFlag sff1{"LuauSubstitutionDontReplaceIgnoredTypes", true}; + + CheckResult result = check(R"( + type Array = { [number]: T } + type Tuple = Array + + local p: Tuple + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ("{number | string}", toString(requireType("p"), {true})); +} + +TEST_SUITE_END(); diff --git a/tests/TypeInfer.provisional.test.cpp b/tests/TypeInfer.provisional.test.cpp index 8108e7f3..b00fddc5 100644 --- a/tests/TypeInfer.provisional.test.cpp +++ b/tests/TypeInfer.provisional.test.cpp @@ -30,6 +30,8 @@ TEST_SUITE_BEGIN("ProvisionalTests"); */ TEST_CASE_FIXTURE(Fixture, "typeguard_inference_incomplete") { + ScopedFastFlag luauTypeAliasPacks("LuauTypeAliasPacks", true); + const std::string code = R"( function f(a) if type(a) == "boolean" then @@ -41,11 +43,11 @@ TEST_CASE_FIXTURE(Fixture, "typeguard_inference_incomplete") )"; const std::string expected = R"( - function f(a:{fn:()->(free)}): () + function f(a:{fn:()->(free,free...)}): () if type(a) == 'boolean'then local a1:boolean=a elseif a.fn()then - local a2:{fn:()->(free)}=a + local a2:{fn:()->(free,free...)}=a end end )"; @@ -231,16 +233,7 @@ TEST_CASE_FIXTURE(Fixture, "operator_eq_completely_incompatible") local r2 = b == a )"); - if (FFlag::LuauEqConstraint) - { - LUAU_REQUIRE_NO_ERRORS(result); - } - else - { - LUAU_REQUIRE_ERROR_COUNT(2, result); - CHECK_EQ(toString(result.errors[0]), "Type '{| x: string |}?' could not be converted into 'number | string'"); - CHECK_EQ(toString(result.errors[1]), "Type 'number | string' could not be converted into '{| x: string |}?'"); - } + LUAU_REQUIRE_NO_ERRORS(result); } // Belongs in TypeInfer.refinements.test.cpp. @@ -542,6 +535,25 @@ TEST_CASE_FIXTURE(Fixture, "bail_early_on_typescript_port_of_Result_type" * doct } } +TEST_CASE_FIXTURE(Fixture, "table_subtyping_shouldn't_add_optional_properties_to_sealed_tables") +{ + CheckResult result = check(R"( + --!strict + local function setNumber(t: { p: number? }, x:number) t.p = x end + local function getString(t: { p: string? }):string return t.p or "" end + -- This shouldn't type-check! + local function oh(x:number): string + local t: {} = {} + setNumber(t, x) + return getString(t) + end + local s: string = oh(37) + )"); + + // Really this should return an error, but it doesn't + LUAU_REQUIRE_NO_ERRORS(result); +} + // Should be in TypeInfer.tables.test.cpp // It's unsound to instantiate tables containing generic methods, // since mutating properties means table properties should be invariant. diff --git a/tests/TypeInfer.refinements.test.cpp b/tests/TypeInfer.refinements.test.cpp index f2ba0ddc..31739cdc 100644 --- a/tests/TypeInfer.refinements.test.cpp +++ b/tests/TypeInfer.refinements.test.cpp @@ -1,4 +1,5 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +#include "Luau/Scope.h" #include "Luau/TypeInfer.h" #include "Fixture.h" @@ -6,7 +7,6 @@ #include "doctest.h" LUAU_FASTFLAG(LuauWeakEqConstraint) -LUAU_FASTFLAG(LuauImprovedTypeGuardPredicate2) LUAU_FASTFLAG(LuauOrPredicate) using namespace Luau; @@ -199,16 +199,8 @@ TEST_CASE_FIXTURE(Fixture, "typeguard_only_look_up_types_from_global_scope") end )"); - if (FFlag::LuauImprovedTypeGuardPredicate2) - { - LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK_EQ("Type 'number' has no overlap with 'string'", toString(result.errors[0])); - } - else - { - LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK_EQ("Type 'string' could not be converted into 'boolean'", toString(result.errors[0])); - } + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ("Type 'number' has no overlap with 'string'", toString(result.errors[0])); } TEST_CASE_FIXTURE(Fixture, "call_a_more_specific_function_using_typeguard") @@ -526,8 +518,6 @@ TEST_CASE_FIXTURE(Fixture, "narrow_property_of_a_bounded_variable") TEST_CASE_FIXTURE(Fixture, "type_narrow_to_vector") { - ScopedFastFlag sff2{"LuauImprovedTypeGuardPredicate2", true}; - CheckResult result = check(R"( local function f(x) if type(x) == "vector" then @@ -544,8 +534,6 @@ TEST_CASE_FIXTURE(Fixture, "type_narrow_to_vector") TEST_CASE_FIXTURE(Fixture, "nonoptional_type_can_narrow_to_nil_if_sense_is_true") { - ScopedFastFlag sff2{"LuauImprovedTypeGuardPredicate2", true}; - CheckResult result = check(R"( local t = {"hello"} local v = t[2] @@ -573,8 +561,6 @@ TEST_CASE_FIXTURE(Fixture, "nonoptional_type_can_narrow_to_nil_if_sense_is_true" TEST_CASE_FIXTURE(Fixture, "typeguard_not_to_be_string") { - ScopedFastFlag sff2{"LuauImprovedTypeGuardPredicate2", true}; - CheckResult result = check(R"( local function f(x: string | number | boolean) if type(x) ~= "string" then @@ -593,8 +579,6 @@ TEST_CASE_FIXTURE(Fixture, "typeguard_not_to_be_string") TEST_CASE_FIXTURE(Fixture, "typeguard_narrows_for_table") { - ScopedFastFlag sff2{"LuauImprovedTypeGuardPredicate2", true}; - CheckResult result = check(R"( local function f(x: string | {x: number} | {y: boolean}) if type(x) == "table" then @@ -613,8 +597,6 @@ TEST_CASE_FIXTURE(Fixture, "typeguard_narrows_for_table") TEST_CASE_FIXTURE(Fixture, "typeguard_narrows_for_functions") { - ScopedFastFlag sff2{"LuauImprovedTypeGuardPredicate2", true}; - CheckResult result = check(R"( local function weird(x: string | ((number) -> string)) if type(x) == "function" then @@ -698,8 +680,6 @@ struct RefinementClassFixture : Fixture TEST_CASE_FIXTURE(RefinementClassFixture, "typeguard_cast_free_table_to_vector") { - ScopedFastFlag sff2{"LuauImprovedTypeGuardPredicate2", true}; - CheckResult result = check(R"( local function f(vec) local X, Y, Z = vec.X, vec.Y, vec.Z @@ -726,8 +706,6 @@ TEST_CASE_FIXTURE(RefinementClassFixture, "typeguard_cast_free_table_to_vector") TEST_CASE_FIXTURE(RefinementClassFixture, "typeguard_cast_instance_or_vector3_to_vector") { - ScopedFastFlag sff2{"LuauImprovedTypeGuardPredicate2", true}; - CheckResult result = check(R"( local function f(x: Instance | Vector3) if typeof(x) == "Vector3" then @@ -746,8 +724,6 @@ TEST_CASE_FIXTURE(RefinementClassFixture, "typeguard_cast_instance_or_vector3_to TEST_CASE_FIXTURE(RefinementClassFixture, "type_narrow_for_all_the_userdata") { - ScopedFastFlag sff2{"LuauImprovedTypeGuardPredicate2", true}; - CheckResult result = check(R"( local function f(x: string | number | Instance | Vector3) if type(x) == "userdata" then @@ -766,10 +742,7 @@ TEST_CASE_FIXTURE(RefinementClassFixture, "type_narrow_for_all_the_userdata") TEST_CASE_FIXTURE(RefinementClassFixture, "eliminate_subclasses_of_instance") { - ScopedFastFlag sffs[] = { - {"LuauImprovedTypeGuardPredicate2", true}, - {"LuauTypeGuardPeelsAwaySubclasses", true}, - }; + ScopedFastFlag sff{"LuauTypeGuardPeelsAwaySubclasses", true}; CheckResult result = check(R"( local function f(x: Part | Folder | string) @@ -789,10 +762,7 @@ TEST_CASE_FIXTURE(RefinementClassFixture, "eliminate_subclasses_of_instance") TEST_CASE_FIXTURE(RefinementClassFixture, "narrow_this_large_union") { - ScopedFastFlag sffs[] = { - {"LuauImprovedTypeGuardPredicate2", true}, - {"LuauTypeGuardPeelsAwaySubclasses", true}, - }; + ScopedFastFlag sff{"LuauTypeGuardPeelsAwaySubclasses", true}; CheckResult result = check(R"( local function f(x: Part | Folder | Instance | string | Vector3 | any) @@ -812,10 +782,7 @@ TEST_CASE_FIXTURE(RefinementClassFixture, "narrow_this_large_union") TEST_CASE_FIXTURE(RefinementClassFixture, "x_as_any_if_x_is_instance_elseif_x_is_table") { - ScopedFastFlag sffs[] = { - {"LuauOrPredicate", true}, - {"LuauImprovedTypeGuardPredicate2", true}, - }; + ScopedFastFlag sff{"LuauOrPredicate", true}; CheckResult result = check(R"( --!nonstrict @@ -839,7 +806,6 @@ TEST_CASE_FIXTURE(RefinementClassFixture, "x_is_not_instance_or_else_not_part") { ScopedFastFlag sffs[] = { {"LuauOrPredicate", true}, - {"LuauImprovedTypeGuardPredicate2", true}, {"LuauTypeGuardPeelsAwaySubclasses", true}, }; @@ -861,8 +827,6 @@ TEST_CASE_FIXTURE(RefinementClassFixture, "x_is_not_instance_or_else_not_part") TEST_CASE_FIXTURE(Fixture, "type_guard_can_filter_for_intersection_of_tables") { - ScopedFastFlag sff2{"LuauImprovedTypeGuardPredicate2", true}; - CheckResult result = check(R"( type XYCoord = {x: number} & {y: number} local function f(t: XYCoord?) @@ -882,8 +846,6 @@ TEST_CASE_FIXTURE(Fixture, "type_guard_can_filter_for_intersection_of_tables") TEST_CASE_FIXTURE(Fixture, "type_guard_can_filter_for_overloaded_function") { - ScopedFastFlag sff2{"LuauImprovedTypeGuardPredicate2", true}; - CheckResult result = check(R"( type SomeOverloadedFunction = ((number) -> string) & ((string) -> number) local function f(g: SomeOverloadedFunction?) @@ -903,8 +865,6 @@ TEST_CASE_FIXTURE(Fixture, "type_guard_can_filter_for_overloaded_function") TEST_CASE_FIXTURE(Fixture, "type_guard_warns_on_no_overlapping_types_only_when_sense_is_true") { - ScopedFastFlag sff2{"LuauImprovedTypeGuardPredicate2", true}; - CheckResult result = check(R"( local function f(t: {x: number}) if type(t) ~= "table" then @@ -999,10 +959,7 @@ TEST_CASE_FIXTURE(Fixture, "not_a_and_not_b2") TEST_CASE_FIXTURE(Fixture, "either_number_or_string") { - ScopedFastFlag sffs[] = { - {"LuauOrPredicate", true}, - {"LuauImprovedTypeGuardPredicate2", true}, - }; + ScopedFastFlag sff{"LuauOrPredicate", true}; CheckResult result = check(R"( local function f(x: any) @@ -1036,10 +993,7 @@ TEST_CASE_FIXTURE(Fixture, "not_t_or_some_prop_of_t") TEST_CASE_FIXTURE(Fixture, "assert_a_to_be_truthy_then_assert_a_to_be_number") { - ScopedFastFlag sffs[] = { - {"LuauOrPredicate", true}, - {"LuauImprovedTypeGuardPredicate2", true}, - }; + ScopedFastFlag sff{"LuauOrPredicate", true}; CheckResult result = check(R"( local a: (number | string)? @@ -1057,10 +1011,7 @@ TEST_CASE_FIXTURE(Fixture, "assert_a_to_be_truthy_then_assert_a_to_be_number") TEST_CASE_FIXTURE(Fixture, "merge_should_be_fully_agnostic_of_hashmap_ordering") { - ScopedFastFlag sffs[] = { - {"LuauOrPredicate", true}, - {"LuauImprovedTypeGuardPredicate2", true}, - }; + ScopedFastFlag sff{"LuauOrPredicate", true}; // This bug came up because there was a mistake in Luau::merge where zipping on two maps would produce the wrong merged result. CheckResult result = check(R"( @@ -1081,10 +1032,7 @@ TEST_CASE_FIXTURE(Fixture, "merge_should_be_fully_agnostic_of_hashmap_ordering") TEST_CASE_FIXTURE(Fixture, "refine_the_correct_types_opposite_of_when_a_is_not_number_or_string") { - ScopedFastFlag sffs[] = { - {"LuauOrPredicate", true}, - {"LuauImprovedTypeGuardPredicate2", true}, - }; + ScopedFastFlag sff{"LuauOrPredicate", true}; CheckResult result = check(R"( local function f(a: string | number | boolean) diff --git a/tests/TypeInfer.tables.test.cpp b/tests/TypeInfer.tables.test.cpp index 1d1b2fae..b7f0dc7b 100644 --- a/tests/TypeInfer.tables.test.cpp +++ b/tests/TypeInfer.tables.test.cpp @@ -46,6 +46,21 @@ TEST_CASE_FIXTURE(Fixture, "augment_table") CHECK(tType->props.find("foo") != tType->props.end()); } +TEST_CASE_FIXTURE(Fixture, "augment_nested_table") +{ + CheckResult result = check("local t = { p = {} } t.p.foo = 'bar'"); + LUAU_REQUIRE_NO_ERRORS(result); + + TableTypeVar* tType = getMutable(requireType("t")); + REQUIRE(tType != nullptr); + + REQUIRE(tType->props.find("p") != tType->props.end()); + const TableTypeVar* pType = get(tType->props["p"].type); + REQUIRE(pType != nullptr); + + CHECK(pType->props.find("foo") != pType->props.end()); +} + TEST_CASE_FIXTURE(Fixture, "cannot_augment_sealed_table") { CheckResult result = check("local t = {prop=999} t.foo = 'bar'"); @@ -260,6 +275,8 @@ TEST_CASE_FIXTURE(Fixture, "open_table_unification") TEST_CASE_FIXTURE(Fixture, "open_table_unification_2") { + ScopedFastFlag sff{"LuauTableSubtypingVariance", true}; + CheckResult result = check(R"( local a = {} a.x = 99 @@ -272,10 +289,11 @@ TEST_CASE_FIXTURE(Fixture, "open_table_unification_2") LUAU_REQUIRE_ERROR_COUNT(1, result); TypeError& err = result.errors[0]; - UnknownProperty* error = get(err); + MissingProperties* error = get(err); REQUIRE(error != nullptr); + REQUIRE(error->properties.size() == 1); - CHECK_EQ(error->key, "y"); + CHECK_EQ("y", error->properties[0]); // TODO(rblanckaert): Revist when we can bind self at function creation time // CHECK_EQ(err.location, Location(Position{5, 19}, Position{5, 25})); @@ -328,6 +346,8 @@ TEST_CASE_FIXTURE(Fixture, "table_param_row_polymorphism_1") TEST_CASE_FIXTURE(Fixture, "table_param_row_polymorphism_2") { + ScopedFastFlag sff{"LuauTableSubtypingVariance", true}; + CheckResult result = check(R"( --!strict function foo(o) @@ -340,14 +360,17 @@ TEST_CASE_FIXTURE(Fixture, "table_param_row_polymorphism_2") LUAU_REQUIRE_ERROR_COUNT(1, result); - UnknownProperty* error = get(result.errors[0]); + MissingProperties* error = get(result.errors[0]); REQUIRE(error != nullptr); + REQUIRE(error->properties.size() == 1); - CHECK_EQ("baz", error->key); + CHECK_EQ("baz", error->properties[0]); } TEST_CASE_FIXTURE(Fixture, "table_param_row_polymorphism_3") { + ScopedFastFlag sff{"LuauTableSubtypingVariance", true}; + CheckResult result = check(R"( local T = {} T.bar = 'hello' @@ -359,8 +382,11 @@ TEST_CASE_FIXTURE(Fixture, "table_param_row_polymorphism_3") LUAU_REQUIRE_ERROR_COUNT(1, result); TypeError& err = result.errors[0]; - UnknownProperty* error = get(err); + MissingProperties* error = get(err); REQUIRE(error != nullptr); + REQUIRE(error->properties.size() == 1); + + CHECK_EQ("baz", error->properties[0]); // TODO(rblanckaert): Revist when we can bind self at function creation time /* @@ -448,6 +474,73 @@ TEST_CASE_FIXTURE(Fixture, "ok_to_add_property_to_free_table") dumpErrors(result); } +TEST_CASE_FIXTURE(Fixture, "okay_to_add_property_to_unsealed_tables_by_assignment") +{ + ScopedFastFlag sff{"LuauTableSubtypingVariance", true}; + + CheckResult result = check(R"( + --!strict + local t = { u = {} } + t = { u = { p = 37 } } + t = { u = { q = "hi" } } + local x = t.u.p + local y = t.u.q + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + CHECK_EQ("number?", toString(requireType("x"))); + CHECK_EQ("string?", toString(requireType("y"))); +} + +TEST_CASE_FIXTURE(Fixture, "okay_to_add_property_to_unsealed_tables_by_function_call") +{ + CheckResult result = check(R"( + --!strict + function get(x) return x.opts["MYOPT"] end + function set(x,y) x.opts["MYOPT"] = y end + local t = { opts = {} } + set(t,37) + local x = get(t) + )"); + + // Currently this errors but it shouldn't, since set only needs write access + // TODO: file a JIRA for this + LUAU_REQUIRE_ERRORS(result); + // CHECK_EQ("number?", toString(requireType("x"))); +} + +TEST_CASE_FIXTURE(Fixture, "width_subtyping") +{ + ScopedFastFlag sff{"LuauTableSubtypingVariance", true}; + + CheckResult result = check(R"( + --!strict + function f(x : { q : number }) + x.q = 8 + end + local t : { q : number, r : string } = { q = 8, r = "hi" } + f(t) + local x : string = t.r + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "width_subtyping_needs_covariance") +{ + CheckResult result = check(R"( + --!strict + function f(x : { p : { q : number }}) + x.p = { q = 8, r = 5 } + end + local t : { p : { q : number, r : string } } = { p = { q = 8, r = "hi" } } + f(t) -- Shouldn't typecheck + local x : string = t.p.r -- x is 5 + )"); + + LUAU_REQUIRE_ERRORS(result); +} + TEST_CASE_FIXTURE(Fixture, "infer_array") { CheckResult result = check(R"( @@ -676,16 +769,27 @@ TEST_CASE_FIXTURE(Fixture, "infer_indexer_for_left_unsealed_table_from_right_han LUAU_REQUIRE_NO_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "sealed_table_value_must_not_infer_an_indexer") +TEST_CASE_FIXTURE(Fixture, "sealed_table_value_can_infer_an_indexer") { + ScopedFastFlag sff{"LuauTableSubtypingVariance", true}; + CheckResult result = check(R"( local t: { a: string, [number]: string } = { a = "foo" } )"); - LUAU_REQUIRE_ERROR_COUNT(1, result); + LUAU_REQUIRE_NO_ERRORS(result); +} - TypeMismatch* tm = get(result.errors[0]); - REQUIRE(tm != nullptr); +TEST_CASE_FIXTURE(Fixture, "array_factory_function") +{ + ScopedFastFlag sff{"LuauTableSubtypingVariance", true}; + + CheckResult result = check(R"( + function empty() return {} end + local array: {string} = empty() + )"); + + LUAU_REQUIRE_NO_ERRORS(result); } TEST_CASE_FIXTURE(Fixture, "sealed_table_indexers_must_unify") @@ -756,37 +860,6 @@ TEST_CASE_FIXTURE(Fixture, "indexing_from_a_table_should_prefer_properties_when_ CHECK_MESSAGE(nullptr != get(result.errors[0]), "Expected a TypeMismatch but got " << result.errors[0]); } -TEST_CASE_FIXTURE(Fixture, "indexing_from_a_table_with_a_string") -{ - ScopedFastFlag fflag("LuauIndexTablesWithIndexers", true); - - CheckResult result = check(R"( - local t: { a: string } - function f(x: string) return t[x] end - local a = f("a") - local b = f("b") - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - - CHECK_EQ(*typeChecker.anyType, *requireType("a")); - CHECK_EQ(*typeChecker.anyType, *requireType("b")); -} - -TEST_CASE_FIXTURE(Fixture, "indexing_from_a_table_with_a_number") -{ - ScopedFastFlag fflag("LuauIndexTablesWithIndexers", true); - - CheckResult result = check(R"( - local t = { a = true } - function f(x: number) return t[x] end - )"); - - LUAU_REQUIRE_ERROR_COUNT(1, result); - - CHECK_MESSAGE(nullptr != get(result.errors[0]), "Expected a TypeMismatch but got " << result.errors[0]); -} - TEST_CASE_FIXTURE(Fixture, "assigning_to_an_unsealed_table_with_string_literal_should_infer_new_properties_over_indexer") { CheckResult result = check(R"( @@ -1392,6 +1465,8 @@ TEST_CASE_FIXTURE(Fixture, "casting_tables_with_props_into_table_with_indexer2") TEST_CASE_FIXTURE(Fixture, "casting_tables_with_props_into_table_with_indexer3") { + ScopedFastFlag sff{"LuauTableSubtypingVariance", true}; + CheckResult result = check(R"( local function foo(a: {[string]: number, a: string}) end foo({ a = 1 }) @@ -1402,8 +1477,21 @@ TEST_CASE_FIXTURE(Fixture, "casting_tables_with_props_into_table_with_indexer3") ToStringOptions o{/* exhaustive= */ true}; TypeMismatch* tm = get(result.errors[0]); REQUIRE(tm); - CHECK_EQ("string", toString(tm->wantedType, o)); - CHECK_EQ("number", toString(tm->givenType, o)); + CHECK_EQ("{| [string]: number, a: string |}", toString(tm->wantedType, o)); + CHECK_EQ("{| a: number |}", toString(tm->givenType, o)); +} + +TEST_CASE_FIXTURE(Fixture, "casting_tables_with_props_into_table_with_indexer4") +{ + CheckResult result = check(R"( + local function foo(a: {[string]: number, a: string}, i: string) + return a[i] + end + local hi: number = foo({ a = "hi" }, "a") -- shouldn't typecheck since at runtime hi is "hi" + )"); + + // This typechecks but shouldn't + LUAU_REQUIRE_NO_ERRORS(result); } TEST_CASE_FIXTURE(Fixture, "table_subtyping_with_missing_props_dont_report_multiple_errors") @@ -1446,22 +1534,32 @@ TEST_CASE_FIXTURE(Fixture, "table_subtyping_with_missing_props_dont_report_multi TEST_CASE_FIXTURE(Fixture, "table_subtyping_with_extra_props_dont_report_multiple_errors") { CheckResult result = check(R"( - local vec3 = {x = 1, y = 2, z = 3} - local vec1 = {x = 1} + local vec3 = {{x = 1, y = 2, z = 3}} + local vec1 = {{x = 1}} vec1 = vec3 )"); LUAU_REQUIRE_ERROR_COUNT(1, result); - MissingProperties* mp = get(result.errors[0]); - REQUIRE(mp); - CHECK_EQ(mp->context, MissingProperties::Extra); - REQUIRE_EQ(2, mp->properties.size()); - CHECK_EQ(mp->properties[0], "y"); - CHECK_EQ(mp->properties[1], "z"); - CHECK_EQ("vec1", toString(mp->superType)); - CHECK_EQ("vec3", toString(mp->subType)); + TypeMismatch* tm = get(result.errors[0]); + REQUIRE(tm); + CHECK_EQ("vec1", toString(tm->wantedType)); + CHECK_EQ("vec3", toString(tm->givenType)); +} + +TEST_CASE_FIXTURE(Fixture, "table_subtyping_with_extra_props_is_ok") +{ + ScopedFastFlag sff{"LuauTableSubtypingVariance", true}; + + CheckResult result = check(R"( + local vec3 = {x = 1, y = 2, z = 3} + local vec1 = {x = 1} + + vec1 = vec3 + )"); + + LUAU_REQUIRE_NO_ERRORS(result); } TEST_CASE_FIXTURE(Fixture, "type_mismatch_on_massive_table_is_cut_short") @@ -1824,4 +1922,32 @@ TEST_CASE_FIXTURE(Fixture, "invariant_table_properties_means_instantiating_table LUAU_REQUIRE_ERRORS(result); } +TEST_CASE_FIXTURE(Fixture, "table_insert_should_cope_with_optional_properties_in_nonstrict") +{ + CheckResult result = check(R"( + --!nonstrict + local buttons = {} + table.insert(buttons, { a = 1 }) + table.insert(buttons, { a = 2, b = true }) + table.insert(buttons, { a = 3 }) + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "table_insert_should_cope_with_optional_properties_in_strict") +{ + ScopedFastFlag sff{"LuauTableSubtypingVariance", true}; + + CheckResult result = check(R"( + --!strict + local buttons = {} + table.insert(buttons, { a = 1 }) + table.insert(buttons, { a = 2, b = true }) + table.insert(buttons, { a = 3 }) + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.test.cpp b/tests/TypeInfer.test.cpp index 338698e7..dbc45382 100644 --- a/tests/TypeInfer.test.cpp +++ b/tests/TypeInfer.test.cpp @@ -3,6 +3,7 @@ #include "Luau/AstQuery.h" #include "Luau/BuiltinDefinitions.h" #include "Luau/Parser.h" +#include "Luau/Scope.h" #include "Luau/TypeInfer.h" #include "Luau/TypeVar.h" #include "Luau/VisitTypeVar.h" @@ -978,23 +979,6 @@ TEST_CASE_FIXTURE(Fixture, "cyclic_function_type_in_args") CHECK_EQ("t1 where t1 = (t1) -> ()", toString(requireType("f"))); } -TEST_CASE_FIXTURE(Fixture, "cyclic_function_type_in_type_alias") -{ - ScopedFastFlag sff{"LuauOccursCheckOkWithRecursiveFunctions", true}; - - CheckResult result = check(R"( - type F = () -> F? - local function f() - return f - end - - local g: F = f - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - CHECK_EQ("t1 where t1 = () -> t1?", toString(requireType("g"))); -} - // TODO: File a Jira about this /* TEST_CASE_FIXTURE(Fixture, "unifying_vararg_pack_with_fixed_length_pack_produces_fixed_length_pack") @@ -1257,23 +1241,6 @@ TEST_CASE_FIXTURE(Fixture, "instantiate_cyclic_generic_function") REQUIRE_EQ(follow(*methodArg), follow(arg)); } -TEST_CASE_FIXTURE(Fixture, "cyclic_types_of_named_table_fields_do_not_expand_when_stringified") -{ - CheckResult result = check(R"( - --!strict - type Node = { Parent: Node?; } - local node: Node; - node.Parent = 1 - )"); - - LUAU_REQUIRE_ERROR_COUNT(1, result); - - TypeMismatch* tm = get(result.errors[0]); - REQUIRE(tm); - CHECK_EQ("Node?", toString(tm->wantedType)); - CHECK_EQ(typeChecker.numberType, tm->givenType); -} - TEST_CASE_FIXTURE(Fixture, "varlist_declared_by_for_in_loop_should_be_free") { CheckResult result = check(R"( @@ -2591,48 +2558,6 @@ TEST_CASE_FIXTURE(Fixture, "toposort_doesnt_break_mutual_recursion") dumpErrors(result); } -TEST_CASE_FIXTURE(Fixture, "mutually_recursive_types") -{ - CheckResult result = check(R"( - --!strict - type T = { f: a, g: U } - type U = { h: a, i: T? } - local x: T = { f = 37, g = { h = 5, i = nil } } - x.g.i = x - local y: T = { f = "hi", g = { h = "lo", i = nil } } - y.g.i = y - )"); - - LUAU_REQUIRE_NO_ERRORS(result); -} - -TEST_CASE_FIXTURE(Fixture, "mutually_recursive_types_errors") -{ - CheckResult result = check(R"( - --!strict - type T = { f: a, g: U } - type U = { h: b, i: T? } - local x: T = { f = 37, g = { h = 5, i = nil } } - x.g.i = x - local y: T = { f = "hi", g = { h = 5, i = nil } } - y.g.i = y - )"); - - LUAU_REQUIRE_ERRORS(result); - - // We had a UAF in this example caused by not cloning type function arguments - ModulePtr module = frontend.moduleResolver.getModule("MainModule"); - unfreeze(module->interfaceTypes); - copyErrors(module->errors, module->interfaceTypes); - freeze(module->interfaceTypes); - module->internalTypes.clear(); - module->astTypes.clear(); - - // Make sure the error strings don't include "VALUELESS" - for (auto error : module->errors) - CHECK_MESSAGE(toString(error).find("VALUELESS") == std::string::npos, toString(error)); -} - TEST_CASE_FIXTURE(Fixture, "object_constructor_can_refer_to_method_of_self") { // CLI-30902 @@ -3388,16 +3313,7 @@ TEST_CASE_FIXTURE(Fixture, "unknown_type_in_comparison") end )"); - if (FFlag::LuauEqConstraint) - { - LUAU_REQUIRE_NO_ERRORS(result); - } - else - { - LUAU_REQUIRE_ERROR_COUNT(1, result); - - REQUIRE(get(result.errors[0])); - } + LUAU_REQUIRE_NO_ERRORS(result); } TEST_CASE_FIXTURE(Fixture, "relation_op_on_any_lhs_where_rhs_maybe_has_metatable") @@ -3407,18 +3323,8 @@ TEST_CASE_FIXTURE(Fixture, "relation_op_on_any_lhs_where_rhs_maybe_has_metatable print((x == true and (x .. "y")) .. 1) )"); - if (FFlag::LuauEqConstraint) - { - LUAU_REQUIRE_ERROR_COUNT(1, result); - REQUIRE(get(result.errors[0])); - } - else - { - LUAU_REQUIRE_ERROR_COUNT(2, result); - - CHECK_EQ("Type 'boolean' could not be converted into 'number | string'", toString(result.errors[0])); - CHECK_EQ("Type 'boolean | string' could not be converted into 'number | string'", toString(result.errors[1])); - } + LUAU_REQUIRE_ERROR_COUNT(1, result); + REQUIRE(get(result.errors[0])); } TEST_CASE_FIXTURE(Fixture, "concat_op_on_string_lhs_and_free_rhs") @@ -3530,25 +3436,6 @@ _(...)(...,setfenv,_):_G() )"); } -TEST_CASE_FIXTURE(Fixture, "use_table_name_and_generic_params_in_errors") -{ - CheckResult result = check(R"( - type Pair = {first: T, second: U} - local a: Pair - local b: Pair - - a = b - )"); - - LUAU_REQUIRE_ERROR_COUNT(1, result); - - TypeMismatch* tm = get(result.errors[0]); - REQUIRE(tm); - - CHECK_EQ("Pair", toString(tm->wantedType)); - CHECK_EQ("Pair", toString(tm->givenType)); -} - TEST_CASE_FIXTURE(Fixture, "cyclic_type_packs") { // this has a risk of creating cyclic type packs, causing infinite loops / OOMs @@ -3658,17 +3545,6 @@ TEST_CASE_FIXTURE(Fixture, "for_in_loop_where_iteratee_is_free") )"); } -TEST_CASE_FIXTURE(Fixture, "dont_stop_typechecking_after_reporting_duplicate_type_definition") -{ - CheckResult result = check(R"( - type A = number - type A = string -- Redefinition of type 'A', previously defined at line 1 - local foo: string = 1 -- No "Type 'number' could not be converted into 'string'" - )"); - - LUAU_REQUIRE_ERROR_COUNT(2, result); -} - TEST_CASE_FIXTURE(Fixture, "tc_after_error_recovery") { CheckResult result = check(R"( @@ -3771,38 +3647,6 @@ TEST_CASE_FIXTURE(Fixture, "error_on_invalid_operand_types_to_relational_operato CHECK_EQ("Type 'number | string' cannot be compared with relational operator <", ge->message); } -TEST_CASE_FIXTURE(Fixture, "stringify_type_alias_of_recursive_template_table_type") -{ - CheckResult result = check(R"( - type Table = { a: T } - type Wrapped = Table - local l: Wrapped = 2 - )"); - - LUAU_REQUIRE_ERROR_COUNT(1, result); - - TypeMismatch* tm = get(result.errors[0]); - REQUIRE(tm); - CHECK_EQ("Wrapped", toString(tm->wantedType)); - CHECK_EQ(typeChecker.numberType, tm->givenType); -} - -TEST_CASE_FIXTURE(Fixture, "stringify_type_alias_of_recursive_template_table_type2") -{ - CheckResult result = check(R"( - type Table = { a: T } - type Wrapped = (Table) -> string - local l: Wrapped = 2 - )"); - - LUAU_REQUIRE_ERROR_COUNT(1, result); - - TypeMismatch* tm = get(result.errors[0]); - REQUIRE(tm); - CHECK_EQ("t1 where t1 = ({| a: t1 |}) -> string", toString(tm->wantedType)); - CHECK_EQ(typeChecker.numberType, tm->givenType); -} - TEST_CASE_FIXTURE(Fixture, "index_expr_should_be_checked") { CheckResult result = check(R"( @@ -3928,19 +3772,6 @@ TEST_CASE_FIXTURE(Fixture, "stringify_nested_unions_with_optionals") CHECK_EQ("(boolean | number | string)?", toString(tm->givenType)); } -// Check that recursive intersection type doesn't generate an OOM -TEST_CASE_FIXTURE(Fixture, "cli_38393_recursive_intersection_oom") -{ - CheckResult result = check(R"( - function _(l0:(t0)&((t0)&(((t0)&((t0)->()))->(typeof(_),typeof(# _)))),l39,...):any - end - type t0 = ((typeof(_))&((t0)&(((typeof(_))&(t0))->typeof(_))),{n163:any,})->(any,typeof(_)) - _(_) - )"); - - LUAU_REQUIRE_ERRORS(result); -} - TEST_CASE_FIXTURE(Fixture, "UnknownGlobalCompoundAssign") { // In non-strict mode, global definition is still allowed @@ -3993,16 +3824,6 @@ TEST_CASE_FIXTURE(Fixture, "loop_typecheck_crash_on_empty_optional") LUAU_REQUIRE_ERROR_COUNT(2, result); } -TEST_CASE_FIXTURE(Fixture, "type_alias_fwd_declaration_is_precise") -{ - CheckResult result = check(R"( - local foo: Id = 1 - type Id = T - )"); - - LUAU_REQUIRE_NO_ERRORS(result); -} - TEST_CASE_FIXTURE(Fixture, "cli_39932_use_unifier_in_ensure_methods") { CheckResult result = check(R"( @@ -4033,81 +3854,6 @@ end LUAU_REQUIRE_NO_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "corecursive_types_generic") -{ - const std::string code = R"( - type A = {v:T, b:B} - type B = {v:T, a:A} - local aa:A - local bb = aa - )"; - - const std::string expected = R"( - type A = {v:T, b:B} - type B = {v:T, a:A} - local aa:A - local bb:A=aa - )"; - - CHECK_EQ(expected, decorateWithTypes(code)); - CheckResult result = check(code); - - LUAU_REQUIRE_NO_ERRORS(result); -} - -TEST_CASE_FIXTURE(Fixture, "corecursive_function_types") -{ - ScopedFastFlag sff{"LuauOccursCheckOkWithRecursiveFunctions", true}; - - CheckResult result = check(R"( - type A = () -> (number, B) - type B = () -> (string, A) - local a: A - local b: B - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - - CHECK_EQ("t1 where t1 = () -> (number, () -> (string, t1))", toString(requireType("a"))); - CHECK_EQ("t1 where t1 = () -> (string, () -> (number, t1))", toString(requireType("b"))); -} - -TEST_CASE_FIXTURE(Fixture, "generic_param_remap") -{ - const std::string code = R"( - -- An example of a forwarded use of a type that has different type arguments than parameters - type A = {t:T, u:U, next:A?} - local aa:A = { t = 5, u = 'hi', next = { t = 'lo', u = 8 } } - local bb = aa - )"; - - const std::string expected = R"( - - type A = {t:T, u:U, next:A?} - local aa:A = { t = 5, u = 'hi', next = { t = 'lo', u = 8 } } - local bb:A=aa - )"; - - CHECK_EQ(expected, decorateWithTypes(code)); - CheckResult result = check(code); - - LUAU_REQUIRE_ERRORS(result); -} - -TEST_CASE_FIXTURE(Fixture, "export_type_and_type_alias_are_duplicates") -{ - CheckResult result = check(R"( - export type Foo = number - type Foo = number - )"); - - LUAU_REQUIRE_ERROR_COUNT(1, result); - - auto dtd = get(result.errors[0]); - REQUIRE(dtd); - CHECK_EQ(dtd->name, "Foo"); -} - TEST_CASE_FIXTURE(Fixture, "dont_report_type_errors_within_an_AstStatError") { CheckResult result = check(R"( @@ -4212,30 +3958,6 @@ TEST_CASE_FIXTURE(Fixture, "luau_resolves_symbols_the_same_way_lua_does") REQUIRE_MESSAGE(get(e) != nullptr, "Expected UnknownSymbol, but got " << e); } -TEST_CASE_FIXTURE(Fixture, "stringify_optional_parameterized_alias") -{ - ScopedFastFlag sffs3{"LuauGenericFunctions", true}; - ScopedFastFlag sffs4{"LuauParseGenericFunctions", true}; - - CheckResult result = check(R"( - type Node = { value: T, child: Node? } - - local function visitor(node: Node?) - local a: Node - - if node then - a = node.child -- Observe the output of the error message. - end - end - )"); - - LUAU_REQUIRE_ERROR_COUNT(1, result); - - auto e = get(result.errors[0]); - CHECK_EQ("Node?", toString(e->givenType)); - CHECK_EQ("Node", toString(e->wantedType)); -} - TEST_CASE_FIXTURE(Fixture, "operator_eq_verifies_types_do_intersect") { CheckResult result = check(R"( @@ -4291,181 +4013,6 @@ local tbl: string = require(game.A) CHECK_EQ("Type '{| def: number |}' could not be converted into 'string'", toString(result.errors[0])); } -TEST_CASE_FIXTURE(Fixture, "general_require_multi_assign") -{ - fileResolver.source["workspace/A"] = R"( - export type myvec2 = {x: number, y: number} - return {} - )"; - - fileResolver.source["workspace/B"] = R"( - export type myvec3 = {x: number, y: number, z: number} - return {} - )"; - - fileResolver.source["workspace/C"] = R"( - local Foo, Bar = require(workspace.A), require(workspace.B) - - local a: Foo.myvec2 - local b: Bar.myvec3 - )"; - - CheckResult result = frontend.check("workspace/C"); - LUAU_REQUIRE_NO_ERRORS(result); - ModulePtr m = frontend.moduleResolver.modules["workspace/C"]; - - REQUIRE(m != nullptr); - - std::optional aTypeId = lookupName(m->getModuleScope(), "a"); - REQUIRE(aTypeId); - const Luau::TableTypeVar* aType = get(follow(*aTypeId)); - REQUIRE(aType); - REQUIRE(aType->props.size() == 2); - - std::optional bTypeId = lookupName(m->getModuleScope(), "b"); - REQUIRE(bTypeId); - const Luau::TableTypeVar* bType = get(follow(*bTypeId)); - REQUIRE(bType); - REQUIRE(bType->props.size() == 3); -} - -TEST_CASE_FIXTURE(Fixture, "type_alias_import_mutation") -{ - CheckResult result = check("type t10 = typeof(table)"); - LUAU_REQUIRE_NO_ERRORS(result); - - TypeId ty = getGlobalBinding(frontend.typeChecker, "table"); - CHECK_EQ(toString(ty), "table"); - - const TableTypeVar* ttv = get(ty); - REQUIRE(ttv); - - CHECK(ttv->instantiatedTypeParams.empty()); -} - -TEST_CASE_FIXTURE(Fixture, "type_alias_local_mutation") -{ - CheckResult result = check(R"( -type Cool = { a: number, b: string } -local c: Cool = { a = 1, b = "s" } -type NotCool = Cool -)"); - LUAU_REQUIRE_NO_ERRORS(result); - - std::optional ty = requireType("c"); - REQUIRE(ty); - CHECK_EQ(toString(*ty), "Cool"); - - const TableTypeVar* ttv = get(*ty); - REQUIRE(ttv); - - CHECK(ttv->instantiatedTypeParams.empty()); -} - -TEST_CASE_FIXTURE(Fixture, "type_alias_local_rename") -{ - CheckResult result = check(R"( -type Cool = { a: number, b: string } -type NotCool = Cool -local c: Cool = { a = 1, b = "s" } -local d: NotCool = { a = 1, b = "s" } -)"); - LUAU_REQUIRE_NO_ERRORS(result); - - std::optional ty = requireType("c"); - REQUIRE(ty); - CHECK_EQ(toString(*ty), "Cool"); - - ty = requireType("d"); - REQUIRE(ty); - CHECK_EQ(toString(*ty), "NotCool"); -} - -TEST_CASE_FIXTURE(Fixture, "type_alias_local_synthetic_mutation") -{ - CheckResult result = check(R"( -local c = { a = 1, b = "s" } -type Cool = typeof(c) -)"); - LUAU_REQUIRE_NO_ERRORS(result); - - std::optional ty = requireType("c"); - REQUIRE(ty); - - const TableTypeVar* ttv = get(*ty); - REQUIRE(ttv); - CHECK_EQ(ttv->name, "Cool"); -} - -TEST_CASE_FIXTURE(Fixture, "type_alias_of_an_imported_recursive_type") -{ - ScopedFastFlag luauFixTableTypeAliasClone{"LuauFixTableTypeAliasClone", true}; - - fileResolver.source["game/A"] = R"( -export type X = { a: number, b: X? } -return {} - )"; - - CheckResult aResult = frontend.check("game/A"); - LUAU_REQUIRE_NO_ERRORS(aResult); - - CheckResult bResult = check(R"( -local Import = require(game.A) -type X = Import.X - )"); - LUAU_REQUIRE_NO_ERRORS(bResult); - - std::optional ty1 = lookupImportedType("Import", "X"); - REQUIRE(ty1); - - std::optional ty2 = lookupType("X"); - REQUIRE(ty2); - - CHECK_EQ(follow(*ty1), follow(*ty2)); -} - -TEST_CASE_FIXTURE(Fixture, "type_alias_of_an_imported_recursive_generic_type") -{ - ScopedFastFlag luauFixTableTypeAliasClone{"LuauFixTableTypeAliasClone", true}; - - fileResolver.source["game/A"] = R"( -export type X = { a: T, b: U, C: X? } -return {} - )"; - - CheckResult aResult = frontend.check("game/A"); - LUAU_REQUIRE_NO_ERRORS(aResult); - - CheckResult bResult = check(R"( -local Import = require(game.A) -type X = Import.X - )"); - LUAU_REQUIRE_NO_ERRORS(bResult); - - std::optional ty1 = lookupImportedType("Import", "X"); - REQUIRE(ty1); - - std::optional ty2 = lookupType("X"); - REQUIRE(ty2); - - CHECK_EQ(toString(*ty1, {true}), toString(*ty2, {true})); - - bResult = check(R"( -local Import = require(game.A) -type X = Import.X - )"); - LUAU_REQUIRE_NO_ERRORS(bResult); - - ty1 = lookupImportedType("Import", "X"); - REQUIRE(ty1); - - ty2 = lookupType("X"); - REQUIRE(ty2); - - CHECK_EQ(toString(*ty1, {true}), "t1 where t1 = {| C: t1?, a: T, b: U |}"); - CHECK_EQ(toString(*ty2, {true}), "{| C: t1, a: U, b: T |} where t1 = {| C: t1, a: U, b: T |}?"); -} - TEST_CASE_FIXTURE(Fixture, "nonstrict_self_mismatch_tail") { CheckResult result = check(R"( @@ -4579,32 +4126,6 @@ local c = a(2) -- too many arguments CHECK_EQ("Argument count mismatch. Function expects 1 argument, but 2 are specified", toString(result.errors[0])); } -TEST_CASE_FIXTURE(Fixture, "module_export_free_type_leak") -{ - CheckResult result = check(R"( -function get() - return function(obj) return true end -end - -export type f = typeof(get()) - )"); - - LUAU_REQUIRE_NO_ERRORS(result); -} - -TEST_CASE_FIXTURE(Fixture, "module_export_wrapped_free_type_leak") -{ - CheckResult result = check(R"( -function get() - return {a = 1, b = function(obj) return true end} -end - -export type f = typeof(get()) - )"); - - LUAU_REQUIRE_NO_ERRORS(result); -} - TEST_CASE_FIXTURE(Fixture, "custom_require_global") { CheckResult result = check(R"( @@ -4787,8 +4308,6 @@ TEST_CASE_FIXTURE(Fixture, "no_heap_use_after_free_error") TEST_CASE_FIXTURE(Fixture, "dont_invalidate_the_properties_iterator_of_free_table_when_rolled_back") { - ScopedFastFlag sff{"LuauLogTableTypeVarBoundTo", true}; - fileResolver.source["Module/Backend/Types"] = R"( export type Fiber = { return_: Fiber? @@ -4868,8 +4387,8 @@ TEST_CASE_FIXTURE(Fixture, "record_matching_overload") ModulePtr module = getMainModule(); auto it = module->astOverloadResolvedTypes.find(parentExpr); - REQUIRE(it != module->astOverloadResolvedTypes.end()); - CHECK_EQ(toString(it->second), "(number) -> number"); + REQUIRE(it); + CHECK_EQ(toString(*it), "(number) -> number"); } TEST_CASE_FIXTURE(Fixture, "infer_anonymous_function_arguments") @@ -5032,76 +4551,6 @@ g12({x=1}, {x=2}, function(x, y) return {x=x.x + y.x} end) LUAU_REQUIRE_NO_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "mutually_recursive_types_restriction_ok") -{ - CheckResult result = check(R"( - type Tree = { data: T, children: Forest } - type Forest = {Tree} - )"); - - LUAU_REQUIRE_NO_ERRORS(result); -} - -TEST_CASE_FIXTURE(Fixture, "mutually_recursive_types_restriction_not_ok_1") -{ - ScopedFastFlag sff{"LuauRecursiveTypeParameterRestriction", true}; - - CheckResult result = check(R"( - -- OK because forwarded types are used with their parameters. - type Tree = { data: T, children: Forest } - type Forest = {Tree<{T}>} - )"); - - LUAU_REQUIRE_ERRORS(result); -} - -TEST_CASE_FIXTURE(Fixture, "mutually_recursive_types_restriction_not_ok_2") -{ - ScopedFastFlag sff{"LuauRecursiveTypeParameterRestriction", true}; - - CheckResult result = check(R"( - -- Not OK because forwarded types are used with different types than their parameters. - type Forest = {Tree<{T}>} - type Tree = { data: T, children: Forest } - )"); - - LUAU_REQUIRE_ERRORS(result); -} - -TEST_CASE_FIXTURE(Fixture, "mutually_recursive_types_swapsies_ok") -{ - CheckResult result = check(R"( - type Tree1 = { data: T, children: {Tree2} } - type Tree2 = { data: U, children: {Tree1} } - - LUAU_REQUIRE_ERRORS(result); -} - -TEST_CASE_FIXTURE(Fixture, "mutually_recursive_types_swapsies_not_ok") -{ - ScopedFastFlag sff{"LuauRecursiveTypeParameterRestriction", true}; - - CheckResult result = check(R"( - type Tree1 = { data: T, children: {Tree2} } - type Tree2 = { data: U, children: {Tree1} } - )"); - - LUAU_REQUIRE_ERRORS(result); -} - -TEST_CASE_FIXTURE(Fixture, "free_variables_from_typeof_in_aliases") -{ - CheckResult result = check(R"( - function f(x) return x[1] end - -- x has type X? for a free type variable X - local x = f ({}) - type ContainsFree = { this: a, that: typeof(x) } - type ContainsContainsFree = { that: ContainsFree } - )"); - - LUAU_REQUIRE_NO_ERRORS(result); -} - TEST_CASE_FIXTURE(Fixture, "infer_generic_lib_function_function_argument") { CheckResult result = check(R"( diff --git a/tests/TypeInfer.tryUnify.test.cpp b/tests/TypeInfer.tryUnify.test.cpp index 91ac9f06..1f4b63ef 100644 --- a/tests/TypeInfer.tryUnify.test.cpp +++ b/tests/TypeInfer.tryUnify.test.cpp @@ -1,5 +1,6 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #include "Luau/Parser.h" +#include "Luau/Scope.h" #include "Luau/TypeInfer.h" #include "Luau/TypeVar.h" diff --git a/tests/TypeInfer.typePacks.cpp b/tests/TypeInfer.typePacks.cpp index 5f7f2847..3e1dedd4 100644 --- a/tests/TypeInfer.typePacks.cpp +++ b/tests/TypeInfer.typePacks.cpp @@ -294,4 +294,370 @@ end LUAU_REQUIRE_NO_ERRORS(result); } +TEST_CASE_FIXTURE(Fixture, "type_alias_type_packs") +{ + ScopedFastFlag luauParseGenericFunctions("LuauParseGenericFunctions", true); + ScopedFastFlag luauTypeAliasPacks("LuauTypeAliasPacks", true); + ScopedFastFlag luauParseTypePackTypeParameters("LuauParseTypePackTypeParameters", true); + + CheckResult result = check(R"( +type Packed = (T...) -> T... +local a: Packed<> +local b: Packed +local c: Packed + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + auto tf = lookupType("Packed"); + REQUIRE(tf); + CHECK_EQ(toString(*tf), "(T...) -> (T...)"); + CHECK_EQ(toString(requireType("a")), "() -> ()"); + CHECK_EQ(toString(requireType("b")), "(number) -> number"); + CHECK_EQ(toString(requireType("c")), "(string, number) -> (string, number)"); + + result = check(R"( +-- (U..., T) cannot be parsed right now +type Packed = { f: (a: T, U...) -> (T, U...) } +local a: Packed +local b: Packed +local c: Packed + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + tf = lookupType("Packed"); + REQUIRE(tf); + CHECK_EQ(toString(*tf), "Packed"); + CHECK_EQ(toString(*tf, {true}), "{| f: (T, U...) -> (T, U...) |}"); + + auto ttvA = get(requireType("a")); + REQUIRE(ttvA); + CHECK_EQ(toString(requireType("a")), "Packed"); + CHECK_EQ(toString(requireType("a"), {true}), "{| f: (number) -> (number) |}"); + REQUIRE(ttvA->instantiatedTypeParams.size() == 1); + REQUIRE(ttvA->instantiatedTypePackParams.size() == 1); + CHECK_EQ(toString(ttvA->instantiatedTypeParams[0], {true}), "number"); + CHECK_EQ(toString(ttvA->instantiatedTypePackParams[0], {true}), ""); + + auto ttvB = get(requireType("b")); + REQUIRE(ttvB); + CHECK_EQ(toString(requireType("b")), "Packed"); + CHECK_EQ(toString(requireType("b"), {true}), "{| f: (string, number) -> (string, number) |}"); + REQUIRE(ttvB->instantiatedTypeParams.size() == 1); + REQUIRE(ttvB->instantiatedTypePackParams.size() == 1); + CHECK_EQ(toString(ttvB->instantiatedTypeParams[0], {true}), "string"); + CHECK_EQ(toString(ttvB->instantiatedTypePackParams[0], {true}), "number"); + + auto ttvC = get(requireType("c")); + REQUIRE(ttvC); + CHECK_EQ(toString(requireType("c")), "Packed"); + CHECK_EQ(toString(requireType("c"), {true}), "{| f: (string, number, boolean) -> (string, number, boolean) |}"); + REQUIRE(ttvC->instantiatedTypeParams.size() == 1); + REQUIRE(ttvC->instantiatedTypePackParams.size() == 1); + CHECK_EQ(toString(ttvC->instantiatedTypeParams[0], {true}), "string"); + CHECK_EQ(toString(ttvC->instantiatedTypePackParams[0], {true}), "number, boolean"); +} + +TEST_CASE_FIXTURE(Fixture, "type_alias_type_packs_import") +{ + ScopedFastFlag luauParseGenericFunctions("LuauParseGenericFunctions", true); + ScopedFastFlag luauTypeAliasPacks("LuauTypeAliasPacks", true); + ScopedFastFlag luauParseTypePackTypeParameters("LuauParseTypePackTypeParameters", true); + + fileResolver.source["game/A"] = R"( +export type Packed = { a: T, b: (U...) -> () } +return {} + )"; + + CheckResult aResult = frontend.check("game/A"); + LUAU_REQUIRE_NO_ERRORS(aResult); + + CheckResult bResult = check(R"( +local Import = require(game.A) +local a: Import.Packed +local b: Import.Packed +local c: Import.Packed +local d: { a: typeof(c) } + )"); + LUAU_REQUIRE_NO_ERRORS(bResult); + + auto tf = lookupImportedType("Import", "Packed"); + REQUIRE(tf); + CHECK_EQ(toString(*tf), "Packed"); + CHECK_EQ(toString(*tf, {true}), "{| a: T, b: (U...) -> () |}"); + + CHECK_EQ(toString(requireType("a"), {true}), "{| a: number, b: () -> () |}"); + CHECK_EQ(toString(requireType("b"), {true}), "{| a: string, b: (number) -> () |}"); + CHECK_EQ(toString(requireType("c"), {true}), "{| a: string, b: (number, boolean) -> () |}"); + CHECK_EQ(toString(requireType("d")), "{| a: Packed |}"); +} + +TEST_CASE_FIXTURE(Fixture, "type_pack_type_parameters") +{ + ScopedFastFlag luauParseGenericFunctions("LuauParseGenericFunctions", true); + ScopedFastFlag luauTypeAliasPacks("LuauTypeAliasPacks", true); + ScopedFastFlag luauParseTypePackTypeParameters("LuauParseTypePackTypeParameters", true); + + fileResolver.source["game/A"] = R"( +export type Packed = { a: T, b: (U...) -> () } +return {} + )"; + + CheckResult cResult = check(R"( +local Import = require(game.A) +type Alias = Import.Packed +local a: Alias + +type B = Import.Packed +type C = Import.Packed + )"); + LUAU_REQUIRE_NO_ERRORS(cResult); + + auto tf = lookupType("Alias"); + REQUIRE(tf); + CHECK_EQ(toString(*tf), "Alias"); + CHECK_EQ(toString(*tf, {true}), "{| a: S, b: (T, R...) -> () |}"); + + CHECK_EQ(toString(requireType("a"), {true}), "{| a: string, b: (number, boolean) -> () |}"); + + tf = lookupType("B"); + REQUIRE(tf); + CHECK_EQ(toString(*tf), "B"); + CHECK_EQ(toString(*tf, {true}), "{| a: string, b: (X...) -> () |}"); + + tf = lookupType("C"); + REQUIRE(tf); + CHECK_EQ(toString(*tf), "C"); + CHECK_EQ(toString(*tf, {true}), "{| a: string, b: (number, X...) -> () |}"); +} + +TEST_CASE_FIXTURE(Fixture, "type_alias_type_packs_nested") +{ + ScopedFastFlag luauParseGenericFunctions("LuauParseGenericFunctions", true); + ScopedFastFlag luauTypeAliasPacks("LuauTypeAliasPacks", true); + ScopedFastFlag luauParseTypePackTypeParameters("LuauParseTypePackTypeParameters", true); + + CheckResult result = check(R"( +type Packed1 = (T...) -> (T...) +type Packed2 = (Packed1, T...) -> (Packed1, T...) +type Packed3 = (Packed2, T...) -> (Packed2, T...) +type Packed4 = (Packed3, T...) -> (Packed3, T...) + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + auto tf = lookupType("Packed4"); + REQUIRE(tf); + CHECK_EQ(toString(*tf), + "((((T...) -> (T...), T...) -> ((T...) -> (T...), T...), T...) -> (((T...) -> (T...), T...) -> ((T...) -> (T...), T...), T...), T...) -> " + "((((T...) -> (T...), T...) -> ((T...) -> (T...), T...), T...) -> (((T...) -> (T...), T...) -> ((T...) -> (T...), T...), T...), T...)"); +} + +TEST_CASE_FIXTURE(Fixture, "type_alias_type_pack_variadic") +{ + ScopedFastFlag luauParseGenericFunctions("LuauParseGenericFunctions", true); + ScopedFastFlag luauTypeAliasPacks("LuauTypeAliasPacks", true); + ScopedFastFlag luauParseTypePackTypeParameters("LuauParseTypePackTypeParameters", true); + + CheckResult result = check(R"( +type X = (T...) -> (string, T...) + +type D = X<...number> +type E = X<(number, ...string)> + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ(toString(*lookupType("D")), "(...number) -> (string, ...number)"); + CHECK_EQ(toString(*lookupType("E")), "(number, ...string) -> (string, number, ...string)"); +} + +TEST_CASE_FIXTURE(Fixture, "type_alias_type_pack_multi") +{ + ScopedFastFlag luauParseGenericFunctions("LuauParseGenericFunctions", true); + ScopedFastFlag luauTypeAliasPacks("LuauTypeAliasPacks", true); + ScopedFastFlag luauParseTypePackTypeParameters("LuauParseTypePackTypeParameters", true); + + CheckResult result = check(R"( +type Y = (T...) -> (U...) +type A = Y +type B = Y<(number, ...string), S...> + +type Z = (T) -> (U...) +type E = Z +type F = Z + +type W = (T, U...) -> (T, V...) +type H = W +type I = W + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ(toString(*lookupType("A")), "(S...) -> (S...)"); + CHECK_EQ(toString(*lookupType("B")), "(number, ...string) -> (S...)"); + + CHECK_EQ(toString(*lookupType("E")), "(number) -> (S...)"); + CHECK_EQ(toString(*lookupType("F")), "(number) -> (string, S...)"); + + CHECK_EQ(toString(*lookupType("H")), "(number, S...) -> (number, R...)"); + CHECK_EQ(toString(*lookupType("I")), "(number, string, S...) -> (number, R...)"); +} + +TEST_CASE_FIXTURE(Fixture, "type_alias_type_pack_explicit") +{ + ScopedFastFlag luauParseGenericFunctions("LuauParseGenericFunctions", true); + ScopedFastFlag luauTypeAliasPacks("LuauTypeAliasPacks", true); + ScopedFastFlag luauParseTypePackTypeParameters("LuauParseTypePackTypeParameters", true); + + CheckResult result = check(R"( +type X = (T...) -> (T...) + +type A = X<(S...)> +type B = X<()> +type C = X<(number)> +type D = X<(number, string)> +type E = X<(...number)> +type F = X<(string, ...number)> + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ(toString(*lookupType("A")), "(S...) -> (S...)"); + CHECK_EQ(toString(*lookupType("B")), "() -> ()"); + CHECK_EQ(toString(*lookupType("C")), "(number) -> number"); + CHECK_EQ(toString(*lookupType("D")), "(number, string) -> (number, string)"); + CHECK_EQ(toString(*lookupType("E")), "(...number) -> (...number)"); + CHECK_EQ(toString(*lookupType("F")), "(string, ...number) -> (string, ...number)"); +} + +TEST_CASE_FIXTURE(Fixture, "type_alias_type_pack_explicit_multi") +{ + ScopedFastFlag luauParseGenericFunctions("LuauParseGenericFunctions", true); + ScopedFastFlag luauTypeAliasPacks("LuauTypeAliasPacks", true); + ScopedFastFlag luauParseTypePackTypeParameters("LuauParseTypePackTypeParameters", true); + + CheckResult result = check(R"( +type Y = (T...) -> (U...) + +type A = Y<(number, string), (boolean)> +type B = Y<(), ()> +type C = Y<...string, (number, S...)> +type D = Y + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ(toString(*lookupType("A")), "(number, string) -> boolean"); + CHECK_EQ(toString(*lookupType("B")), "() -> ()"); + CHECK_EQ(toString(*lookupType("C")), "(...string) -> (number, S...)"); + CHECK_EQ(toString(*lookupType("D")), "(X...) -> (number, string, X...)"); +} + +TEST_CASE_FIXTURE(Fixture, "type_alias_type_pack_explicit_multi_tostring") +{ + ScopedFastFlag luauParseGenericFunctions("LuauParseGenericFunctions", true); + ScopedFastFlag luauTypeAliasPacks("LuauTypeAliasPacks", true); + ScopedFastFlag luauParseTypePackTypeParameters("LuauParseTypePackTypeParameters", true); + ScopedFastFlag luauInstantiatedTypeParamRecursion("LuauInstantiatedTypeParamRecursion", true); // For correct toString block + + CheckResult result = check(R"( +type Y = { f: (T...) -> (U...) } + +local a: Y<(number, string), (boolean)> +local b: Y<(), ()> + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ(toString(requireType("a")), "Y<(number, string), (boolean)>"); + CHECK_EQ(toString(requireType("b")), "Y<(), ()>"); +} + +TEST_CASE_FIXTURE(Fixture, "type_alias_backwards_compatible") +{ + ScopedFastFlag luauParseGenericFunctions("LuauParseGenericFunctions", true); + ScopedFastFlag luauTypeAliasPacks("LuauTypeAliasPacks", true); + ScopedFastFlag luauParseTypePackTypeParameters("LuauParseTypePackTypeParameters", true); + + CheckResult result = check(R"( +type X = () -> T +type Y = (T) -> U + +type A = X<(number)> +type B = Y<(number), (boolean)> +type C = Y<(number), boolean> + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ(toString(*lookupType("A")), "() -> number"); + CHECK_EQ(toString(*lookupType("B")), "(number) -> boolean"); + CHECK_EQ(toString(*lookupType("C")), "(number) -> boolean"); +} + +TEST_CASE_FIXTURE(Fixture, "type_alias_type_packs_errors") +{ + ScopedFastFlag luauParseGenericFunctions("LuauParseGenericFunctions", true); + ScopedFastFlag luauTypeAliasPacks("LuauTypeAliasPacks", true); + ScopedFastFlag luauParseTypePackTypeParameters("LuauParseTypePackTypeParameters", true); + + CheckResult result = check(R"( +type Packed = (T, U) -> (V...) +local b: Packed + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ(toString(result.errors[0]), "Generic type 'Packed' expects at least 2 type arguments, but only 1 is specified"); + + result = check(R"( +type Packed = (T, U) -> () +type B = Packed + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ(toString(result.errors[0]), "Generic type 'Packed' expects 0 type pack arguments, but 1 is specified"); + + result = check(R"( +type Packed = (T...) -> (U...) +type Other = Packed + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ(toString(result.errors[0]), "Type parameters must come before type pack parameters"); + + result = check(R"( +type Packed = (T) -> U +type Other = Packed + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ(toString(result.errors[0]), "Generic type 'Packed' expects 2 type arguments, but only 1 is specified"); + + result = check(R"( +type Packed = (T...) -> T... +local a: Packed + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ(toString(result.errors[0]), "Type parameter list is required"); + + result = check(R"( +type Packed = (T...) -> (U...) +type Other = Packed<> + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ(toString(result.errors[0]), "Generic type 'Packed' expects 2 type pack arguments, but none are specified"); + + result = check(R"( +type Packed = (T...) -> (U...) +type Other = Packed + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ(toString(result.errors[0]), "Generic type 'Packed' expects 2 type pack arguments, but only 1 is specified"); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.unionTypes.test.cpp b/tests/TypeInfer.unionTypes.test.cpp index 456dbadc..c0ed25d2 100644 --- a/tests/TypeInfer.unionTypes.test.cpp +++ b/tests/TypeInfer.unionTypes.test.cpp @@ -237,21 +237,7 @@ TEST_CASE_FIXTURE(Fixture, "union_equality_comparisons") local z = a == c )"); - if (FFlag::LuauEqConstraint) - { - LUAU_REQUIRE_NO_ERRORS(result); - } - else - { - LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK_EQ(*typeChecker.booleanType, *requireType("x")); - CHECK_EQ(*typeChecker.booleanType, *requireType("y")); - - TypeMismatch* tm = get(result.errors[0]); - REQUIRE(tm); - CHECK_EQ("(number | string)?", toString(*tm->wantedType)); - CHECK_EQ("boolean | number", toString(*tm->givenType)); - } + LUAU_REQUIRE_NO_ERRORS(result); } TEST_CASE_FIXTURE(Fixture, "optional_union_members") diff --git a/tests/TypeVar.test.cpp b/tests/TypeVar.test.cpp index 98ce9f93..a679e3fd 100644 --- a/tests/TypeVar.test.cpp +++ b/tests/TypeVar.test.cpp @@ -1,5 +1,6 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #include "Luau/Parser.h" +#include "Luau/Scope.h" #include "Luau/TypeInfer.h" #include "Luau/TypeVar.h" diff --git a/tools/tracegraph.py b/tools/tracegraph.py new file mode 100644 index 00000000..a46423e7 --- /dev/null +++ b/tools/tracegraph.py @@ -0,0 +1,95 @@ +#!/usr/bin/python +# This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details + +# Given a trace event file, this tool generates a flame graph based on the event scopes present in the file +# The result of analysis is a .svg file which can be viewed in a browser + +import sys +import svg +import json + +class Node(svg.Node): + def __init__(self): + svg.Node.__init__(self) + self.caption = "" + self.description = "" + self.ticks = 0 + + def text(self): + return self.caption + + def title(self): + return self.caption + + def details(self, root): + return "{} ({:,} usec, {:.1%}); self: {:,} usec".format(self.description, self.width, self.width / root.width, self.ticks) + +with open(sys.argv[1]) as f: + dump = f.read() + +root = Node() + +# Finish the file +if not dump.endswith("]"): + dump += "{}]" + +data = json.loads(dump) + +stacks = {} + +for l in data: + if len(l) == 0: + continue + + # Track stack of each thread, but aggregate values together + tid = l["tid"] + + if not tid in stacks: + stacks[tid] = [] + stack = stacks[tid] + + if l["ph"] == 'B': + stack.append(l) + elif l["ph"] == 'E': + node = root + + for e in stack: + caption = e["name"] + description = '' + + if "args" in e: + for arg in e["args"]: + if len(description) != 0: + description += ", " + + description += "{}: {}".format(arg, e["args"][arg]) + + child = node.child(caption + description) + child.caption = caption + child.description = description + + node = child + + begin = stack[-1] + + ticks = l["ts"] - begin["ts"] + rawticks = ticks + + # Flame graph requires ticks without children duration + if "childts" in begin: + ticks -= begin["childts"] + + node.ticks += int(ticks) + + stack.pop() + + if len(stack): + parent = stack[-1] + + if "childts" in parent: + parent["childts"] += rawticks + else: + parent["childts"] = rawticks + +svg.layout(root, lambda n: n.ticks) +svg.display(root, "Flame Graph", "hot", flip = True) From 34cf695fbc35eb435dcd9fb85c3b98234fdd266c Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Thu, 4 Nov 2021 19:42:00 -0700 Subject: [PATCH 022/399] Sync to upstream/release/503 - A series of major optimizations to type checking performance on complex programs/types (up to two orders of magnitude speedup for programs involving huge tagged unions) - Fix a few issues encountered by UBSAN (and maybe fix s390x builds) - Fix gcc-11 test builds - Fix a rare corner case where luau_load wouldn't wake inactive threads which could result in a use-after-free due to GC - Fix CLI crash when error object that's not a string escapes to top level --- Analysis/include/Luau/BuiltinDefinitions.h | 1 - Analysis/include/Luau/Quantify.h | 14 + Analysis/include/Luau/ToString.h | 2 + Analysis/include/Luau/TopoSortStatements.h | 1 + Analysis/include/Luau/TxnLog.h | 30 +- Analysis/include/Luau/TypeInfer.h | 7 +- Analysis/include/Luau/TypeVar.h | 9 +- Analysis/include/Luau/Unifier.h | 20 +- Analysis/include/Luau/UnifierSharedState.h | 44 ++ Analysis/include/Luau/VisitTypeVar.h | 59 +- Analysis/src/Autocomplete.cpp | 5 +- Analysis/src/BuiltinDefinitions.cpp | 12 - Analysis/src/Error.cpp | 21 +- Analysis/src/Frontend.cpp | 11 +- Analysis/src/Module.cpp | 18 +- Analysis/src/Quantify.cpp | 90 +++ Analysis/src/RequireTracer.cpp | 6 +- Analysis/src/ToString.cpp | 25 +- Analysis/src/TopoSortStatements.cpp | 25 + Analysis/src/TxnLog.cpp | 37 +- Analysis/src/TypeAttach.cpp | 93 ++- Analysis/src/TypeInfer.cpp | 203 ++--- Analysis/src/TypeVar.cpp | 86 ++- Analysis/src/Unifier.cpp | 349 ++++++++- Ast/include/Luau/TimeTrace.h | 16 +- Ast/src/Parser.cpp | 2 +- Ast/src/TimeTrace.cpp | 5 +- CLI/Repl.cpp | 24 +- Compiler/include/Luau/Bytecode.h | 4 +- Compiler/src/Compiler.cpp | 14 +- Makefile | 2 + Sources.cmake | 3 + VM/include/lualib.h | 2 +- VM/src/lapi.cpp | 4 +- VM/src/laux.cpp | 2 +- VM/src/lcorolib.cpp | 72 +- VM/src/ldo.cpp | 14 +- VM/src/ldo.h | 2 +- VM/src/lfunc.cpp | 2 +- VM/src/lgc.cpp | 333 ++++---- VM/src/lgc.h | 8 +- VM/src/lmem.cpp | 2 +- VM/src/lstring.cpp | 2 +- VM/src/lvmload.cpp | 4 +- bench/tests/chess.lua | 849 +++++++++++++++++++++ bench/tests/shootout/scimark.lua | 2 +- tests/Autocomplete.test.cpp | 24 +- tests/Compiler.test.cpp | 84 +- tests/Conformance.test.cpp | 13 + tests/IostreamOptional.h | 7 +- tests/Linter.test.cpp | 14 +- tests/TypeInfer.aliases.test.cpp | 50 ++ tests/TypeInfer.builtins.test.cpp | 2 +- tests/TypeInfer.classes.test.cpp | 2 - tests/TypeInfer.generics.test.cpp | 21 + tests/TypeInfer.provisional.test.cpp | 25 +- tests/TypeInfer.refinements.test.cpp | 11 +- tests/TypeInfer.tables.test.cpp | 2 +- tests/TypeInfer.test.cpp | 62 +- tests/TypeInfer.tryUnify.test.cpp | 10 +- tests/TypeInfer.typePacks.cpp | 6 +- tests/TypeInfer.unionTypes.test.cpp | 25 +- tests/TypeVar.test.cpp | 60 ++ tests/conformance/closure.lua | 2 +- tests/conformance/coroutine.lua | 2 +- tests/conformance/gc.lua | 2 +- tests/conformance/locals.lua | 2 +- tests/conformance/math.lua | 2 +- tests/conformance/pm.lua | 4 +- tests/conformance/tmerror.lua | 15 + tools/gdb-printers.py | 8 +- 71 files changed, 2304 insertions(+), 687 deletions(-) create mode 100644 Analysis/include/Luau/Quantify.h create mode 100644 Analysis/include/Luau/UnifierSharedState.h create mode 100644 Analysis/src/Quantify.cpp create mode 100644 bench/tests/chess.lua create mode 100644 tests/conformance/tmerror.lua diff --git a/Analysis/include/Luau/BuiltinDefinitions.h b/Analysis/include/Luau/BuiltinDefinitions.h index 57a1907a..07d897b2 100644 --- a/Analysis/include/Luau/BuiltinDefinitions.h +++ b/Analysis/include/Luau/BuiltinDefinitions.h @@ -34,7 +34,6 @@ TypeId makeFunction( // Polymorphic std::initializer_list paramTypes, std::initializer_list paramNames, std::initializer_list retTypes); void attachMagicFunction(TypeId ty, MagicFunction fn); -void attachFunctionTag(TypeId ty, std::string constraint); Property makeProperty(TypeId ty, std::optional documentationSymbol = std::nullopt); void assignPropDocumentationSymbols(TableTypeVar::Props& props, const std::string& baseName); diff --git a/Analysis/include/Luau/Quantify.h b/Analysis/include/Luau/Quantify.h new file mode 100644 index 00000000..f46df146 --- /dev/null +++ b/Analysis/include/Luau/Quantify.h @@ -0,0 +1,14 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +#pragma once + +#include "Luau/TypeVar.h" + +namespace Luau +{ + +struct Module; +using ModulePtr = std::shared_ptr; + +void quantify(ModulePtr module, TypeId ty, TypeLevel level); + +} // namespace Luau diff --git a/Analysis/include/Luau/ToString.h b/Analysis/include/Luau/ToString.h index 0897ec85..e5683fc4 100644 --- a/Analysis/include/Luau/ToString.h +++ b/Analysis/include/Luau/ToString.h @@ -69,4 +69,6 @@ std::string toString(const TypePackVar& tp, const ToStringOptions& opts = {}); void dump(TypeId ty); void dump(TypePackId ty); +std::string generateName(size_t n); + } // namespace Luau diff --git a/Analysis/include/Luau/TopoSortStatements.h b/Analysis/include/Luau/TopoSortStatements.h index 751694f0..4a4acfa3 100644 --- a/Analysis/include/Luau/TopoSortStatements.h +++ b/Analysis/include/Luau/TopoSortStatements.h @@ -12,6 +12,7 @@ struct AstArray; class AstStat; bool containsFunctionCall(const AstStat& stat); +bool containsFunctionCallOrReturn(const AstStat& stat); bool isFunction(const AstStat& stat); void toposort(std::vector& stats); diff --git a/Analysis/include/Luau/TxnLog.h b/Analysis/include/Luau/TxnLog.h index 055441ce..322abd19 100644 --- a/Analysis/include/Luau/TxnLog.h +++ b/Analysis/include/Luau/TxnLog.h @@ -3,19 +3,37 @@ #include "Luau/TypeVar.h" +LUAU_FASTFLAG(LuauShareTxnSeen); + namespace Luau { // Log of where what TypeIds we are rebinding and what they used to be struct TxnLog { - TxnLog() = default; - - explicit TxnLog(const std::vector>& seen) - : seen(seen) + TxnLog() + : originalSeenSize(0) + , ownedSeen() + , sharedSeen(&ownedSeen) { } + explicit TxnLog(std::vector>* sharedSeen) + : originalSeenSize(sharedSeen->size()) + , ownedSeen() + , sharedSeen(sharedSeen) + { + } + + explicit TxnLog(const std::vector>& ownedSeen) + : originalSeenSize(ownedSeen.size()) + , ownedSeen(ownedSeen) + , sharedSeen(nullptr) + { + // This is deprecated! + LUAU_ASSERT(!FFlag::LuauShareTxnSeen); + } + TxnLog(const TxnLog&) = delete; TxnLog& operator=(const TxnLog&) = delete; @@ -38,9 +56,11 @@ private: std::vector> typeVarChanges; std::vector> typePackChanges; std::vector>> tableChanges; + size_t originalSeenSize; public: - std::vector> seen; // used to avoid infinite recursion when types are cyclic + std::vector> ownedSeen; // used to avoid infinite recursion when types are cyclic + std::vector>* sharedSeen; // shared with all the descendent logs }; } // namespace Luau diff --git a/Analysis/include/Luau/TypeInfer.h b/Analysis/include/Luau/TypeInfer.h index d701eb24..9d62fef0 100644 --- a/Analysis/include/Luau/TypeInfer.h +++ b/Analysis/include/Luau/TypeInfer.h @@ -11,6 +11,7 @@ #include "Luau/TypePack.h" #include "Luau/TypeVar.h" #include "Luau/Unifier.h" +#include "Luau/UnifierSharedState.h" #include #include @@ -121,7 +122,7 @@ struct TypeChecker void check(const ScopePtr& scope, const AstStatForIn& forin); void check(const ScopePtr& scope, TypeId ty, const ScopePtr& funScope, const AstStatFunction& function); void check(const ScopePtr& scope, TypeId ty, const ScopePtr& funScope, const AstStatLocalFunction& function); - void check(const ScopePtr& scope, const AstStatTypeAlias& typealias, bool forwardDeclare = false); + void check(const ScopePtr& scope, const AstStatTypeAlias& typealias, int subLevel = 0, bool forwardDeclare = false); void check(const ScopePtr& scope, const AstStatDeclareClass& declaredClass); void check(const ScopePtr& scope, const AstStatDeclareFunction& declaredFunction); @@ -336,7 +337,7 @@ private: // Note: `scope` must be a fresh scope. std::pair, std::vector> createGenericTypes( - const ScopePtr& scope, const AstNode& node, const AstArray& genericNames, const AstArray& genericPackNames); + const ScopePtr& scope, std::optional levelOpt, const AstNode& node, const AstArray& genericNames, const AstArray& genericPackNames); public: ErrorVec resolve(const PredicateVec& predicates, const ScopePtr& scope, bool sense); @@ -383,6 +384,8 @@ public: std::function prepareModuleScope; InternalErrorReporter* iceHandler; + UnifierSharedState unifierState; + public: const TypeId nilType; const TypeId numberType; diff --git a/Analysis/include/Luau/TypeVar.h b/Analysis/include/Luau/TypeVar.h index d4e4e491..9611e881 100644 --- a/Analysis/include/Luau/TypeVar.h +++ b/Analysis/include/Luau/TypeVar.h @@ -405,7 +405,7 @@ const std::string* getName(TypeId type); // Checks whether a union contains all types of another union. bool isSubset(const UnionTypeVar& super, const UnionTypeVar& sub); -// Checks if a type conains generic type binders +// Checks if a type contains generic type binders bool isGeneric(const TypeId ty); // Checks if a type may be instantiated to one containing generic type binders @@ -540,4 +540,11 @@ UnionTypeVarIterator end(const UnionTypeVar* utv); using TypeIdPredicate = std::function(TypeId)>; std::vector filterMap(TypeId type, TypeIdPredicate predicate); +void attachTag(TypeId ty, const std::string& tagName); +void attachTag(Property& prop, const std::string& tagName); + +bool hasTag(TypeId ty, const std::string& tagName); +bool hasTag(const Property& prop, const std::string& tagName); +bool hasTag(const Tags& tags, const std::string& tagName); // Do not use in new work. + } // namespace Luau diff --git a/Analysis/include/Luau/Unifier.h b/Analysis/include/Luau/Unifier.h index 522914b2..56632e33 100644 --- a/Analysis/include/Luau/Unifier.h +++ b/Analysis/include/Luau/Unifier.h @@ -6,6 +6,7 @@ #include "Luau/TxnLog.h" #include "Luau/TypeInfer.h" #include "Luau/Module.h" // FIXME: For TypeArena. It merits breaking out into its own header. +#include "Luau/UnifierSharedState.h" #include @@ -41,11 +42,14 @@ struct Unifier std::shared_ptr counters_DEPRECATED; - InternalErrorReporter* iceHandler; + UnifierSharedState& sharedState; - Unifier(TypeArena* types, Mode mode, ScopePtr globalScope, const Location& location, Variance variance, InternalErrorReporter* iceHandler); - Unifier(TypeArena* types, Mode mode, ScopePtr globalScope, const std::vector>& seen, const Location& location, - Variance variance, InternalErrorReporter* iceHandler, const std::shared_ptr& counters_DEPRECATED = nullptr, + Unifier(TypeArena* types, Mode mode, ScopePtr globalScope, const Location& location, Variance variance, UnifierSharedState& sharedState); + Unifier(TypeArena* types, Mode mode, ScopePtr globalScope, const std::vector>& ownedSeen, const Location& location, + Variance variance, UnifierSharedState& sharedState, const std::shared_ptr& counters_DEPRECATED = nullptr, + UnifierCounters* counters = nullptr); + Unifier(TypeArena* types, Mode mode, ScopePtr globalScope, std::vector>* sharedSeen, const Location& location, + Variance variance, UnifierSharedState& sharedState, const std::shared_ptr& counters_DEPRECATED = nullptr, UnifierCounters* counters = nullptr); // Test whether the two type vars unify. Never commits the result. @@ -69,7 +73,8 @@ private: void tryUnifyWithMetatable(TypeId metatable, TypeId other, bool reversed); void tryUnifyWithClass(TypeId superTy, TypeId subTy, bool reversed); void tryUnify(const TableIndexer& superIndexer, const TableIndexer& subIndexer); - TypeId deeplyOptional(TypeId ty, std::unordered_map seen = {}); + TypeId deeplyOptional(TypeId ty, std::unordered_map seen = {}); + void cacheResult(TypeId superTy, TypeId subTy); public: void tryUnify(TypePackId superTy, TypePackId subTy, bool isFunctionCall = false); @@ -101,8 +106,9 @@ private: [[noreturn]] void ice(const std::string& message, const Location& location); [[noreturn]] void ice(const std::string& message); - DenseHashSet tempSeenTy{nullptr}; - DenseHashSet tempSeenTp{nullptr}; + // Remove with FFlagLuauCacheUnifyTableResults + DenseHashSet tempSeenTy_DEPRECATED{nullptr}; + DenseHashSet tempSeenTp_DEPRECATED{nullptr}; }; } // namespace Luau diff --git a/Analysis/include/Luau/UnifierSharedState.h b/Analysis/include/Luau/UnifierSharedState.h new file mode 100644 index 00000000..f252a004 --- /dev/null +++ b/Analysis/include/Luau/UnifierSharedState.h @@ -0,0 +1,44 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +#pragma once + +#include "Luau/DenseHash.h" +#include "Luau/TypeVar.h" +#include "Luau/TypePack.h" + +#include + +namespace Luau +{ +struct InternalErrorReporter; + +struct TypeIdPairHash +{ + size_t hashOne(Luau::TypeId key) const + { + return (uintptr_t(key) >> 4) ^ (uintptr_t(key) >> 9); + } + + size_t operator()(const std::pair& x) const + { + return hashOne(x.first) ^ (hashOne(x.second) << 1); + } +}; + +struct UnifierSharedState +{ + UnifierSharedState(InternalErrorReporter* iceHandler) + : iceHandler(iceHandler) + { + } + + InternalErrorReporter* iceHandler; + + DenseHashSet seenAny{nullptr}; + DenseHashMap skipCacheForType{nullptr}; + DenseHashSet, TypeIdPairHash> cachedUnify{{nullptr, nullptr}}; + + DenseHashSet tempSeenTy{nullptr}; + DenseHashSet tempSeenTp{nullptr}; +}; + +} // namespace Luau diff --git a/Analysis/include/Luau/VisitTypeVar.h b/Analysis/include/Luau/VisitTypeVar.h index df0bd420..a866655c 100644 --- a/Analysis/include/Luau/VisitTypeVar.h +++ b/Analysis/include/Luau/VisitTypeVar.h @@ -1,9 +1,12 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #pragma once +#include "Luau/DenseHash.h" #include "Luau/TypeVar.h" #include "Luau/TypePack.h" +LUAU_FASTFLAG(LuauCacheUnifyTableResults) + namespace Luau { @@ -32,17 +35,33 @@ inline bool hasSeen(std::unordered_set& seen, const void* tv) return !seen.insert(ttv).second; } +inline bool hasSeen(DenseHashSet& seen, const void* tv) +{ + void* ttv = const_cast(tv); + + if (seen.contains(ttv)) + return true; + + seen.insert(ttv); + return false; +} + inline void unsee(std::unordered_set& seen, const void* tv) { void* ttv = const_cast(tv); seen.erase(ttv); } -template -void visit(TypePackId tp, F& f, std::unordered_set& seen); +inline void unsee(DenseHashSet& seen, const void* tv) +{ + // When DenseHashSet is used for 'visitOnce', where don't forget visited elements +} -template -void visit(TypeId ty, F& f, std::unordered_set& seen) +template +void visit(TypePackId tp, F& f, Set& seen); + +template +void visit(TypeId ty, F& f, Set& seen) { if (visit_detail::hasSeen(seen, ty)) { @@ -79,15 +98,23 @@ void visit(TypeId ty, F& f, std::unordered_set& seen) else if (auto ttv = get(ty)) { + // Some visitors want to see bound tables, that's why we visit the original type if (apply(ty, *ttv, seen, f)) { - for (auto& [_name, prop] : ttv->props) - visit(prop.type, f, seen); - - if (ttv->indexer) + if (FFlag::LuauCacheUnifyTableResults && ttv->boundTo) { - visit(ttv->indexer->indexType, f, seen); - visit(ttv->indexer->indexResultType, f, seen); + visit(*ttv->boundTo, f, seen); + } + else + { + for (auto& [_name, prop] : ttv->props) + visit(prop.type, f, seen); + + if (ttv->indexer) + { + visit(ttv->indexer->indexType, f, seen); + visit(ttv->indexer->indexResultType, f, seen); + } } } } @@ -140,8 +167,8 @@ void visit(TypeId ty, F& f, std::unordered_set& seen) visit_detail::unsee(seen, ty); } -template -void visit(TypePackId tp, F& f, std::unordered_set& seen) +template +void visit(TypePackId tp, F& f, Set& seen) { if (visit_detail::hasSeen(seen, tp)) { @@ -182,6 +209,7 @@ void visit(TypePackId tp, F& f, std::unordered_set& seen) visit_detail::unsee(seen, tp); } + } // namespace visit_detail template @@ -197,4 +225,11 @@ void visitTypeVar(TID ty, F& f) visit_detail::visit(ty, f, seen); } +template +void visitTypeVarOnce(TID ty, F& f, DenseHashSet& seen) +{ + seen.clear(); + visit_detail::visit(ty, f, seen); +} + } // namespace Luau diff --git a/Analysis/src/Autocomplete.cpp b/Analysis/src/Autocomplete.cpp index 235abf36..3c43c808 100644 --- a/Analysis/src/Autocomplete.cpp +++ b/Analysis/src/Autocomplete.cpp @@ -196,7 +196,8 @@ static TypeCorrectKind checkTypeCorrectKind(const Module& module, TypeArena* typ auto canUnify = [&typeArena, &module](TypeId expectedType, TypeId actualType) { InternalErrorReporter iceReporter; - Unifier unifier(typeArena, Mode::Strict, module.getModuleScope(), Location(), Variance::Covariant, &iceReporter); + UnifierSharedState unifierState(&iceReporter); + Unifier unifier(typeArena, Mode::Strict, module.getModuleScope(), Location(), Variance::Covariant, unifierState); unifier.tryUnify(expectedType, actualType); @@ -1460,7 +1461,7 @@ static AutocompleteResult autocomplete(const SourceModule& sourceModule, const M result.erase(std::string(stringKey->value.data, stringKey->value.size)); } - // If we know for sure that a key is being written, do not offer general epxression suggestions + // If we know for sure that a key is being written, do not offer general expression suggestions if (!key) autocompleteExpression(sourceModule, *module, typeChecker, typeArena, finder.ancestry, position, result); diff --git a/Analysis/src/BuiltinDefinitions.cpp b/Analysis/src/BuiltinDefinitions.cpp index 3b0c2163..f6f2363c 100644 --- a/Analysis/src/BuiltinDefinitions.cpp +++ b/Analysis/src/BuiltinDefinitions.cpp @@ -106,18 +106,6 @@ void attachMagicFunction(TypeId ty, MagicFunction fn) LUAU_ASSERT(!"Got a non functional type"); } -void attachFunctionTag(TypeId ty, std::string tag) -{ - if (auto ftv = getMutable(ty)) - { - ftv->tags.emplace_back(std::move(tag)); - } - else - { - LUAU_ASSERT(!"Got a non functional type"); - } -} - Property makeProperty(TypeId ty, std::optional documentationSymbol) { return { diff --git a/Analysis/src/Error.cpp b/Analysis/src/Error.cpp index 92fbffc8..04d91444 100644 --- a/Analysis/src/Error.cpp +++ b/Analysis/src/Error.cpp @@ -158,21 +158,20 @@ struct ErrorConverter std::string operator()(const Luau::CountMismatch& e) const { + const std::string expectedS = e.expected == 1 ? "" : "s"; + const std::string actualS = e.actual == 1 ? "" : "s"; + const std::string actualVerb = e.actual == 1 ? "is" : "are"; + switch (e.context) { case CountMismatch::Return: - { - const std::string expectedS = e.expected == 1 ? "" : "s"; - const std::string actualS = e.actual == 1 ? "is" : "are"; - return "Expected to return " + std::to_string(e.expected) + " value" + expectedS + ", but " + std::to_string(e.actual) + " " + actualS + - " returned here"; - } + return "Expected to return " + std::to_string(e.expected) + " value" + expectedS + ", but " + + std::to_string(e.actual) + " " + actualVerb + " returned here"; case CountMismatch::Result: - if (e.expected > e.actual) - return "Function returns " + std::to_string(e.expected) + " values but there are only " + std::to_string(e.expected) + - " values to unpack them into."; - else - return "Function only returns " + std::to_string(e.expected) + " values. " + std::to_string(e.actual) + " are required here"; + // It is alright if right hand side produces more values than the + // left hand side accepts. In this context consider only the opposite case. + return "Function only returns " + std::to_string(e.expected) + " value" + expectedS + ". " + + std::to_string(e.actual) + " are required here"; case CountMismatch::Arg: if (FFlag::LuauTypeAliasPacks) return "Argument count mismatch. Function " + wrongNumberOfArgsString(e.expected, e.actual); diff --git a/Analysis/src/Frontend.cpp b/Analysis/src/Frontend.cpp index b2529840..5e7af50c 100644 --- a/Analysis/src/Frontend.cpp +++ b/Analysis/src/Frontend.cpp @@ -23,6 +23,7 @@ LUAU_FASTFLAGVARIABLE(LuauResolveModuleNameWithoutACurrentModule, false) LUAU_FASTFLAG(LuauTraceRequireLookupChild) LUAU_FASTFLAGVARIABLE(LuauPersistDefinitionFileTypes, false) LUAU_FASTFLAG(LuauNewRequireTrace) +LUAU_FASTFLAGVARIABLE(LuauClearScopes, false) namespace Luau { @@ -248,7 +249,7 @@ struct RequireCycle // Note that this is O(V^2) for a fully connected graph and produces O(V) paths of length O(V) // However, when the graph is acyclic, this is O(V), as well as when only the first cycle is needed (stopAtFirst=true) std::vector getRequireCycles( - const std::unordered_map& sourceNodes, const SourceNode* start, bool stopAtFirst = false) + const FileResolver* resolver, const std::unordered_map& sourceNodes, const SourceNode* start, bool stopAtFirst = false) { std::vector result; @@ -282,9 +283,9 @@ std::vector getRequireCycles( if (top == start) { for (const SourceNode* node : path) - cycle.push_back(node->name); + cycle.push_back(resolver->getHumanReadableModuleName(node->name)); - cycle.push_back(top->name); + cycle.push_back(resolver->getHumanReadableModuleName(top->name)); break; } } @@ -404,7 +405,7 @@ CheckResult Frontend::check(const ModuleName& name) // however, for now getRequireCycles isn't expensive in practice on the cases we care about, and long term // all correct programs must be acyclic so this code triggers rarely if (cycleDetected) - requireCycles = getRequireCycles(sourceNodes, &sourceNode, mode == Mode::NoCheck); + requireCycles = getRequireCycles(fileResolver, sourceNodes, &sourceNode, mode == Mode::NoCheck); // This is used by the type checker to replace the resulting type of cyclic modules with any sourceModule.cyclic = !requireCycles.empty(); @@ -458,6 +459,8 @@ CheckResult Frontend::check(const ModuleName& name) module->astTypes.clear(); module->astExpectedTypes.clear(); module->astOriginalCallTypes.clear(); + if (FFlag::LuauClearScopes) + module->scopes.resize(1); } if (mode != Mode::NoCheck) diff --git a/Analysis/src/Module.cpp b/Analysis/src/Module.cpp index df6be767..2fd95896 100644 --- a/Analysis/src/Module.cpp +++ b/Analysis/src/Module.cpp @@ -15,6 +15,7 @@ LUAU_FASTFLAGVARIABLE(DebugLuauTrackOwningArena, false) LUAU_FASTFLAG(LuauSecondTypecheckKnowsTheDataModel) LUAU_FASTFLAG(LuauCaptureBrokenCommentSpans) LUAU_FASTFLAG(LuauTypeAliasPacks) +LUAU_FASTFLAGVARIABLE(LuauCloneBoundTables, false) namespace Luau { @@ -299,6 +300,14 @@ void TypeCloner::operator()(const FunctionTypeVar& t) void TypeCloner::operator()(const TableTypeVar& t) { + // If table is now bound to another one, we ignore the content of the original + if (FFlag::LuauCloneBoundTables && t.boundTo) + { + TypeId boundTo = clone(*t.boundTo, dest, seenTypes, seenTypePacks, encounteredFreeType); + seenTypes[typeId] = boundTo; + return; + } + TypeId result = dest.addType(TableTypeVar{}); TableTypeVar* ttv = getMutable(result); LUAU_ASSERT(ttv != nullptr); @@ -321,8 +330,11 @@ void TypeCloner::operator()(const TableTypeVar& t) ttv->indexer = TableIndexer{clone(t.indexer->indexType, dest, seenTypes, seenTypePacks, encounteredFreeType), clone(t.indexer->indexResultType, dest, seenTypes, seenTypePacks, encounteredFreeType)}; - if (t.boundTo) - ttv->boundTo = clone(*t.boundTo, dest, seenTypes, seenTypePacks, encounteredFreeType); + if (!FFlag::LuauCloneBoundTables) + { + if (t.boundTo) + ttv->boundTo = clone(*t.boundTo, dest, seenTypes, seenTypePacks, encounteredFreeType); + } for (TypeId& arg : ttv->instantiatedTypeParams) arg = clone(arg, dest, seenTypes, seenTypePacks, encounteredFreeType); @@ -335,7 +347,7 @@ void TypeCloner::operator()(const TableTypeVar& t) if (ttv->state == TableState::Free) { - if (!t.boundTo) + if (FFlag::LuauCloneBoundTables || !t.boundTo) { if (encounteredFreeType) *encounteredFreeType = true; diff --git a/Analysis/src/Quantify.cpp b/Analysis/src/Quantify.cpp new file mode 100644 index 00000000..bf6d81aa --- /dev/null +++ b/Analysis/src/Quantify.cpp @@ -0,0 +1,90 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details + +#include "Luau/Quantify.h" + +#include "Luau/VisitTypeVar.h" + +namespace Luau +{ + +struct Quantifier +{ + ModulePtr module; + TypeLevel level; + std::vector generics; + std::vector genericPacks; + + Quantifier(ModulePtr module, TypeLevel level) + : module(module) + , level(level) + { + } + + void cycle(TypeId) {} + void cycle(TypePackId) {} + + bool operator()(TypeId ty, const FreeTypeVar& ftv) + { + if (!level.subsumes(ftv.level)) + return false; + + *asMutable(ty) = GenericTypeVar{level}; + generics.push_back(ty); + + return false; + } + + template + bool operator()(TypeId ty, const T& t) + { + return true; + } + + template + bool operator()(TypePackId, const T&) + { + return true; + } + + bool operator()(TypeId ty, const TableTypeVar&) + { + TableTypeVar& ttv = *getMutable(ty); + + if (ttv.state == TableState::Sealed || ttv.state == TableState::Generic) + return false; + if (!level.subsumes(ttv.level)) + return false; + + if (ttv.state == TableState::Free) + ttv.state = TableState::Generic; + else if (ttv.state == TableState::Unsealed) + ttv.state = TableState::Sealed; + + ttv.level = level; + + return true; + } + + bool operator()(TypePackId tp, const FreeTypePack& ftp) + { + if (!level.subsumes(ftp.level)) + return false; + + *asMutable(tp) = GenericTypePack{level}; + genericPacks.push_back(tp); + return true; + } +}; + +void quantify(ModulePtr module, TypeId ty, TypeLevel level) +{ + Quantifier q{std::move(module), level}; + visitTypeVar(ty, q); + + FunctionTypeVar* ftv = getMutable(ty); + LUAU_ASSERT(ftv); + ftv->generics = q.generics; + ftv->genericPacks = q.genericPacks; +} + +} // namespace Luau diff --git a/Analysis/src/RequireTracer.cpp b/Analysis/src/RequireTracer.cpp index ad4d5ef4..95910b56 100644 --- a/Analysis/src/RequireTracer.cpp +++ b/Analysis/src/RequireTracer.cpp @@ -171,7 +171,7 @@ struct RequireTracerOld : AstVisitor result.exprs[call] = {fileResolver->concat(*rootName, v)}; - // 'WaitForChild' can be used on modules that are not awailable at the typecheck time, but will be awailable at runtime + // 'WaitForChild' can be used on modules that are not available at the typecheck time, but will be available at runtime // If we fail to find such module, we will not report an UnknownRequire error if (FFlag::LuauTraceRequireLookupChild && indexName->index == "WaitForChild") result.exprs[call].optional = true; @@ -182,7 +182,7 @@ struct RequireTracerOld : AstVisitor struct RequireTracer : AstVisitor { - RequireTracer(RequireTraceResult& result, FileResolver * fileResolver, const ModuleName& currentModuleName) + RequireTracer(RequireTraceResult& result, FileResolver* fileResolver, const ModuleName& currentModuleName) : result(result) , fileResolver(fileResolver) , currentModuleName(currentModuleName) @@ -260,7 +260,7 @@ struct RequireTracer : AstVisitor // seed worklist with require arguments work.reserve(requires.size()); - for (AstExprCall* require: requires) + for (AstExprCall* require : requires) work.push_back(require->args.data[0]); // push all dependent expressions to the work stack; note that the vector is modified during traversal diff --git a/Analysis/src/ToString.cpp b/Analysis/src/ToString.cpp index 5651af7e..cd8180db 100644 --- a/Analysis/src/ToString.cpp +++ b/Analysis/src/ToString.cpp @@ -10,7 +10,6 @@ #include #include -LUAU_FASTFLAG(LuauExtraNilRecovery) LUAU_FASTFLAG(LuauOccursCheckOkWithRecursiveFunctions) LUAU_FASTFLAGVARIABLE(LuauInstantiatedTypeParamRecursion, false) LUAU_FASTFLAG(LuauTypeAliasPacks) @@ -159,15 +158,6 @@ struct StringifierState seen.erase(iter); } - static std::string generateName(size_t i) - { - std::string n; - n = char('a' + i % 26); - if (i >= 26) - n += std::to_string(i / 26); - return n; - } - std::string getName(TypeId ty) { const size_t s = result.nameMap.typeVars.size(); @@ -584,8 +574,7 @@ struct TypeVarStringifier std::vector results = {}; for (auto el : &uv) { - if (FFlag::LuauExtraNilRecovery || FFlag::LuauAddMissingFollow) - el = follow(el); + el = follow(el); if (isNil(el)) { @@ -649,8 +638,7 @@ struct TypeVarStringifier std::vector results = {}; for (auto el : uv.parts) { - if (FFlag::LuauExtraNilRecovery || FFlag::LuauAddMissingFollow) - el = follow(el); + el = follow(el); std::string saved = std::move(state.result.name); @@ -1204,4 +1192,13 @@ void dump(TypePackId ty) printf("%s\n", toString(ty, opts).c_str()); } +std::string generateName(size_t i) +{ + std::string n; + n = char('a' + i % 26); + if (i >= 26) + n += std::to_string(i / 26); + return n; +} + } // namespace Luau diff --git a/Analysis/src/TopoSortStatements.cpp b/Analysis/src/TopoSortStatements.cpp index 2d356384..dba694be 100644 --- a/Analysis/src/TopoSortStatements.cpp +++ b/Analysis/src/TopoSortStatements.cpp @@ -298,8 +298,15 @@ struct ArcCollector : public AstVisitor struct ContainsFunctionCall : public AstVisitor { + bool alsoReturn = false; bool result = false; + ContainsFunctionCall() = default; + explicit ContainsFunctionCall(bool alsoReturn) + : alsoReturn(alsoReturn) + { + } + bool visit(AstExpr*) override { return !result; // short circuit if result is true @@ -318,6 +325,17 @@ struct ContainsFunctionCall : public AstVisitor return false; } + bool visit(AstStatReturn* stat) override + { + if (alsoReturn) + { + result = true; + return false; + } + else + return AstVisitor::visit(stat); + } + bool visit(AstExprFunction*) override { return false; @@ -479,6 +497,13 @@ bool containsFunctionCall(const AstStat& stat) return cfc.result; } +bool containsFunctionCallOrReturn(const AstStat& stat) +{ + detail::ContainsFunctionCall cfc{true}; + const_cast(stat).visit(&cfc); + return cfc.result; +} + bool isFunction(const AstStat& stat) { return stat.is() || stat.is(); diff --git a/Analysis/src/TxnLog.cpp b/Analysis/src/TxnLog.cpp index 702d0ca2..383bb050 100644 --- a/Analysis/src/TxnLog.cpp +++ b/Analysis/src/TxnLog.cpp @@ -5,6 +5,8 @@ #include +LUAU_FASTFLAGVARIABLE(LuauShareTxnSeen, false) + namespace Luau { @@ -33,6 +35,12 @@ void TxnLog::rollback() for (auto it = tableChanges.rbegin(); it != tableChanges.rend(); ++it) std::swap(it->first->boundTo, it->second); + + if (FFlag::LuauShareTxnSeen) + { + LUAU_ASSERT(originalSeenSize <= sharedSeen->size()); + sharedSeen->resize(originalSeenSize); + } } void TxnLog::concat(TxnLog rhs) @@ -46,27 +54,44 @@ void TxnLog::concat(TxnLog rhs) tableChanges.insert(tableChanges.end(), rhs.tableChanges.begin(), rhs.tableChanges.end()); rhs.tableChanges.clear(); - seen.swap(rhs.seen); - rhs.seen.clear(); + if (!FFlag::LuauShareTxnSeen) + { + ownedSeen.swap(rhs.ownedSeen); + rhs.ownedSeen.clear(); + } } bool TxnLog::haveSeen(TypeId lhs, TypeId rhs) { const std::pair sortedPair = (lhs > rhs) ? std::make_pair(lhs, rhs) : std::make_pair(rhs, lhs); - return (seen.end() != std::find(seen.begin(), seen.end(), sortedPair)); + if (FFlag::LuauShareTxnSeen) + return (sharedSeen->end() != std::find(sharedSeen->begin(), sharedSeen->end(), sortedPair)); + else + return (ownedSeen.end() != std::find(ownedSeen.begin(), ownedSeen.end(), sortedPair)); } void TxnLog::pushSeen(TypeId lhs, TypeId rhs) { const std::pair sortedPair = (lhs > rhs) ? std::make_pair(lhs, rhs) : std::make_pair(rhs, lhs); - seen.push_back(sortedPair); + if (FFlag::LuauShareTxnSeen) + sharedSeen->push_back(sortedPair); + else + ownedSeen.push_back(sortedPair); } void TxnLog::popSeen(TypeId lhs, TypeId rhs) { const std::pair sortedPair = (lhs > rhs) ? std::make_pair(lhs, rhs) : std::make_pair(rhs, lhs); - LUAU_ASSERT(sortedPair == seen.back()); - seen.pop_back(); + if (FFlag::LuauShareTxnSeen) + { + LUAU_ASSERT(sortedPair == sharedSeen->back()); + sharedSeen->pop_back(); + } + else + { + LUAU_ASSERT(sortedPair == ownedSeen.back()); + ownedSeen.pop_back(); + } } } // namespace Luau diff --git a/Analysis/src/TypeAttach.cpp b/Analysis/src/TypeAttach.cpp index 266c1986..49f8e0ca 100644 --- a/Analysis/src/TypeAttach.cpp +++ b/Analysis/src/TypeAttach.cpp @@ -6,6 +6,7 @@ #include "Luau/Parser.h" #include "Luau/RecursionCounter.h" #include "Luau/Scope.h" +#include "Luau/ToString.h" #include "Luau/TypeInfer.h" #include "Luau/TypePack.h" #include "Luau/TypeVar.h" @@ -33,14 +34,31 @@ static char* allocateString(Luau::Allocator& allocator, const char* format, Data return result; } +using SyntheticNames = std::unordered_map; + namespace Luau { + +static const char* getName(Allocator* allocator, SyntheticNames* syntheticNames, const Unifiable::Generic& gen) +{ + size_t s = syntheticNames->size(); + char*& n = (*syntheticNames)[&gen]; + if (!n) + { + std::string str = gen.explicitName ? gen.name : generateName(s); + n = static_cast(allocator->allocate(str.size() + 1)); + strcpy(n, str.c_str()); + } + + return n; +} + class TypeRehydrationVisitor { - mutable std::map seen; - mutable int count = 0; + std::map seen; + int count = 0; - bool hasSeen(const void* tv) const + bool hasSeen(const void* tv) { void* ttv = const_cast(tv); auto it = seen.find(ttv); @@ -52,15 +70,16 @@ class TypeRehydrationVisitor } public: - TypeRehydrationVisitor(Allocator* alloc, const TypeRehydrationOptions& options = TypeRehydrationOptions()) + TypeRehydrationVisitor(Allocator* alloc, SyntheticNames* syntheticNames, const TypeRehydrationOptions& options = TypeRehydrationOptions()) : allocator(alloc) + , syntheticNames(syntheticNames) , options(options) { } - AstTypePack* rehydrate(TypePackId tp) const; + AstTypePack* rehydrate(TypePackId tp); - AstType* operator()(const PrimitiveTypeVar& ptv) const + AstType* operator()(const PrimitiveTypeVar& ptv) { switch (ptv.type) { @@ -78,11 +97,11 @@ public: return nullptr; } } - AstType* operator()(const AnyTypeVar&) const + AstType* operator()(const AnyTypeVar&) { return allocator->alloc(Location(), std::nullopt, AstName("any")); } - AstType* operator()(const TableTypeVar& ttv) const + AstType* operator()(const TableTypeVar& ttv) { RecursionCounter counter(&count); @@ -144,12 +163,12 @@ public: return allocator->alloc(Location(), props, indexer); } - AstType* operator()(const MetatableTypeVar& mtv) const + AstType* operator()(const MetatableTypeVar& mtv) { return Luau::visit(*this, mtv.table->ty); } - AstType* operator()(const ClassTypeVar& ctv) const + AstType* operator()(const ClassTypeVar& ctv) { RecursionCounter counter(&count); @@ -176,7 +195,7 @@ public: return allocator->alloc(Location(), props); } - AstType* operator()(const FunctionTypeVar& ftv) const + AstType* operator()(const FunctionTypeVar& ftv) { RecursionCounter counter(&count); @@ -253,10 +272,12 @@ public: size_t i = 0; for (const auto& el : ftv.argNames) { + std::optional* arg = &argNames.data[i++]; + if (el) - argNames.data[i++] = {AstName(el->name.c_str()), el->location}; + new (arg) std::optional(AstArgumentName(AstName(el->name.c_str()), el->location)); else - argNames.data[i++] = {}; + new (arg) std::optional(); } AstArray returnTypes; @@ -290,23 +311,23 @@ public: return allocator->alloc( Location(), generics, genericPacks, AstTypeList{argTypes, argTailAnnotation}, argNames, AstTypeList{returnTypes, retTailAnnotation}); } - AstType* operator()(const Unifiable::Error&) const + AstType* operator()(const Unifiable::Error&) { return allocator->alloc(Location(), std::nullopt, AstName("Unifiable")); } - AstType* operator()(const GenericTypeVar& gtv) const + AstType* operator()(const GenericTypeVar& gtv) { - return allocator->alloc(Location(), std::nullopt, AstName(gtv.name.c_str())); + return allocator->alloc(Location(), std::nullopt, AstName(getName(allocator, syntheticNames, gtv))); } - AstType* operator()(const Unifiable::Bound& bound) const + AstType* operator()(const Unifiable::Bound& bound) { return Luau::visit(*this, bound.boundTo->ty); } - AstType* operator()(Unifiable::Free ftv) const + AstType* operator()(const FreeTypeVar& ftv) { return allocator->alloc(Location(), std::nullopt, AstName("free")); } - AstType* operator()(const UnionTypeVar& uv) const + AstType* operator()(const UnionTypeVar& uv) { AstArray unionTypes; unionTypes.size = uv.options.size(); @@ -317,7 +338,7 @@ public: } return allocator->alloc(Location(), unionTypes); } - AstType* operator()(const IntersectionTypeVar& uv) const + AstType* operator()(const IntersectionTypeVar& uv) { AstArray intersectionTypes; intersectionTypes.size = uv.parts.size(); @@ -328,23 +349,28 @@ public: } return allocator->alloc(Location(), intersectionTypes); } - AstType* operator()(const LazyTypeVar& ltv) const + AstType* operator()(const LazyTypeVar& ltv) { return allocator->alloc(Location(), std::nullopt, AstName("")); } private: Allocator* allocator; + SyntheticNames* syntheticNames; const TypeRehydrationOptions& options; }; class TypePackRehydrationVisitor { public: - TypePackRehydrationVisitor(Allocator* allocator, const TypeRehydrationVisitor& typeVisitor) + TypePackRehydrationVisitor(Allocator* allocator, SyntheticNames* syntheticNames, TypeRehydrationVisitor* typeVisitor) : allocator(allocator) + , syntheticNames(syntheticNames) , typeVisitor(typeVisitor) { + LUAU_ASSERT(allocator); + LUAU_ASSERT(syntheticNames); + LUAU_ASSERT(typeVisitor); } AstTypePack* operator()(const BoundTypePack& btp) const @@ -359,7 +385,7 @@ public: head.data = static_cast(allocator->allocate(sizeof(AstType*) * tp.head.size())); for (size_t i = 0; i < tp.head.size(); i++) - head.data[i] = Luau::visit(typeVisitor, tp.head[i]->ty); + head.data[i] = Luau::visit(*typeVisitor, tp.head[i]->ty); AstTypePack* tail = nullptr; @@ -371,12 +397,12 @@ public: AstTypePack* operator()(const VariadicTypePack& vtp) const { - return allocator->alloc(Location(), Luau::visit(typeVisitor, vtp.ty->ty)); + return allocator->alloc(Location(), Luau::visit(*typeVisitor, vtp.ty->ty)); } AstTypePack* operator()(const GenericTypePack& gtp) const { - return allocator->alloc(Location(), AstName(gtp.name.c_str())); + return allocator->alloc(Location(), AstName(getName(allocator, syntheticNames, gtp))); } AstTypePack* operator()(const FreeTypePack& gtp) const @@ -391,12 +417,13 @@ public: private: Allocator* allocator; - const TypeRehydrationVisitor& typeVisitor; + SyntheticNames* syntheticNames; + TypeRehydrationVisitor* typeVisitor; }; -AstTypePack* TypeRehydrationVisitor::rehydrate(TypePackId tp) const +AstTypePack* TypeRehydrationVisitor::rehydrate(TypePackId tp) { - TypePackRehydrationVisitor tprv(allocator, *this); + TypePackRehydrationVisitor tprv(allocator, syntheticNames, this); return Luau::visit(tprv, tp->ty); } @@ -431,7 +458,7 @@ public: { if (!type) return nullptr; - return Luau::visit(TypeRehydrationVisitor(allocator), (*type)->ty); + return Luau::visit(TypeRehydrationVisitor(allocator, &syntheticNames), (*type)->ty); } AstArray typeAstPack(TypePackId type) @@ -443,7 +470,7 @@ public: result.data = static_cast(allocator->allocate(sizeof(AstType*) * v.size())); for (size_t i = 0; i < v.size(); ++i) { - result.data[i] = Luau::visit(TypeRehydrationVisitor(allocator), v[i]->ty); + result.data[i] = Luau::visit(TypeRehydrationVisitor(allocator, &syntheticNames), v[i]->ty); } return result; } @@ -495,7 +522,7 @@ public: { if (FFlag::LuauTypeAliasPacks) { - variadicAnnotation = TypeRehydrationVisitor(allocator).rehydrate(*tail); + variadicAnnotation = TypeRehydrationVisitor(allocator, &syntheticNames).rehydrate(*tail); } else { @@ -515,6 +542,7 @@ public: private: Module& module; Allocator* allocator; + SyntheticNames syntheticNames; }; void attachTypeData(SourceModule& source, Module& result) @@ -525,7 +553,8 @@ void attachTypeData(SourceModule& source, Module& result) AstType* rehydrateAnnotation(TypeId type, Allocator* allocator, const TypeRehydrationOptions& options) { - return Luau::visit(TypeRehydrationVisitor(allocator, options), type->ty); + SyntheticNames syntheticNames; + return Luau::visit(TypeRehydrationVisitor(allocator, &syntheticNames, options), type->ty); } } // namespace Luau diff --git a/Analysis/src/TypeInfer.cpp b/Analysis/src/TypeInfer.cpp index 3a1fdfff..38e2e527 100644 --- a/Analysis/src/TypeInfer.cpp +++ b/Analysis/src/TypeInfer.cpp @@ -4,6 +4,7 @@ #include "Luau/Common.h" #include "Luau/ModuleResolver.h" #include "Luau/Parser.h" +#include "Luau/Quantify.h" #include "Luau/RecursionCounter.h" #include "Luau/Scope.h" #include "Luau/Substitution.h" @@ -33,18 +34,16 @@ LUAU_FASTFLAGVARIABLE(LuauCloneCorrectlyBeforeMutatingTableType, false) LUAU_FASTFLAGVARIABLE(LuauStoreMatchingOverloadFnType, false) LUAU_FASTFLAGVARIABLE(LuauRankNTypes, false) LUAU_FASTFLAGVARIABLE(LuauOrPredicate, false) -LUAU_FASTFLAGVARIABLE(LuauExtraNilRecovery, false) -LUAU_FASTFLAGVARIABLE(LuauMissingUnionPropertyError, false) LUAU_FASTFLAGVARIABLE(LuauInferReturnAssertAssign, false) LUAU_FASTFLAGVARIABLE(LuauRecursiveTypeParameterRestriction, false) LUAU_FASTFLAGVARIABLE(LuauAddMissingFollow, false) LUAU_FASTFLAGVARIABLE(LuauTypeGuardPeelsAwaySubclasses, false) LUAU_FASTFLAGVARIABLE(LuauSlightlyMoreFlexibleBinaryPredicates, false) -LUAU_FASTFLAGVARIABLE(LuauInferFunctionArgsFix, false) LUAU_FASTFLAGVARIABLE(LuauFollowInTypeFunApply, false) LUAU_FASTFLAGVARIABLE(LuauIfElseExpressionAnalysisSupport, false) LUAU_FASTFLAGVARIABLE(LuauStrictRequire, false) LUAU_FASTFLAG(LuauSubstitutionDontReplaceIgnoredTypes) +LUAU_FASTFLAGVARIABLE(LuauQuantifyInPlace2, false) LUAU_FASTFLAG(LuauNewRequireTrace) LUAU_FASTFLAG(LuauTypeAliasPacks) @@ -215,6 +214,7 @@ static bool isMetamethod(const Name& name) TypeChecker::TypeChecker(ModuleResolver* resolver, InternalErrorReporter* iceHandler) : resolver(resolver) , iceHandler(iceHandler) + , unifierState(iceHandler) , nilType(singletonTypes.nilType) , numberType(singletonTypes.numberType) , stringType(singletonTypes.stringType) @@ -370,13 +370,18 @@ void TypeChecker::checkBlock(const ScopePtr& scope, const AstStatBlock& block) return; } + int subLevel = 0; + std::vector sorted(block.body.data, block.body.data + block.body.size); toposort(sorted); for (const auto& stat : sorted) { if (const auto& typealias = stat->as()) - check(scope, *typealias, true); + { + check(scope, *typealias, subLevel, true); + ++subLevel; + } } auto protoIter = sorted.begin(); @@ -399,8 +404,6 @@ void TypeChecker::checkBlock(const ScopePtr& scope, const AstStatBlock& block) } }; - int subLevel = 0; - while (protoIter != sorted.end()) { // protoIter walks forward @@ -416,7 +419,7 @@ void TypeChecker::checkBlock(const ScopePtr& scope, const AstStatBlock& block) // ``` // These both call each other, so `f` will be ordered before `g`, so the call to `g` // is typechecked before `g` has had its body checked. For this reason, there's three - // types for each functuion: before its body is checked, during checking its body, + // types for each function: before its body is checked, during checking its body, // and after its body is checked. // // We currently treat the before-type and the during-type as the same, @@ -433,7 +436,7 @@ void TypeChecker::checkBlock(const ScopePtr& scope, const AstStatBlock& block) // function f(x:a):a local x: number = g(37) return x end // function g(x:number):number return f(x) end // ``` - if (containsFunctionCall(**protoIter)) + if (FFlag::LuauQuantifyInPlace2 ? containsFunctionCallOrReturn(**protoIter) : containsFunctionCall(**protoIter)) { while (checkIter != protoIter) { @@ -1080,7 +1083,7 @@ void TypeChecker::check(const ScopePtr& scope, TypeId ty, const ScopePtr& funSco checkFunctionBody(funScope, ty, *function.func); // If in nonstrict mode and allowing redefinition of global function, restore the previous definition type - // in case this function has a differing signature. The signature discrepency will be caught in checkBlock. + // in case this function has a differing signature. The signature discrepancy will be caught in checkBlock. if (previouslyDefined) globalBindings[name] = oldBinding; else @@ -1161,7 +1164,7 @@ void TypeChecker::check(const ScopePtr& scope, TypeId ty, const ScopePtr& funSco scope->bindings[function.name] = {quantify(scope, ty, function.name->location), function.name->location}; } -void TypeChecker::check(const ScopePtr& scope, const AstStatTypeAlias& typealias, bool forwardDeclare) +void TypeChecker::check(const ScopePtr& scope, const AstStatTypeAlias& typealias, int subLevel, bool forwardDeclare) { // This function should be called at most twice for each type alias. // Once with forwardDeclare, and once without. @@ -1189,11 +1192,12 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatTypeAlias& typealias } else { - ScopePtr aliasScope = childScope(scope, typealias.location); + ScopePtr aliasScope = + FFlag::LuauQuantifyInPlace2 ? childScope(scope, typealias.location, subLevel) : childScope(scope, typealias.location); if (FFlag::LuauTypeAliasPacks) { - auto [generics, genericPacks] = createGenericTypes(aliasScope, typealias, typealias.generics, typealias.genericPacks); + auto [generics, genericPacks] = createGenericTypes(aliasScope, scope->level, typealias, typealias.generics, typealias.genericPacks); TypeId ty = (FFlag::LuauRankNTypes ? freshType(aliasScope) : DEPRECATED_freshType(scope, true)); FreeTypeVar* ftv = getMutable(ty); @@ -1418,7 +1422,7 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatDeclareFunction& glo { ScopePtr funScope = childFunctionScope(scope, global.location); - auto [generics, genericPacks] = createGenericTypes(funScope, global, global.generics, global.genericPacks); + auto [generics, genericPacks] = createGenericTypes(funScope, std::nullopt, global, global.generics, global.genericPacks); TypePackId argPack = resolveTypePack(funScope, global.params); TypePackId retPack = resolveTypePack(funScope, global.retTypes); @@ -1610,25 +1614,11 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprIn if (std::optional ty = resolveLValue(scope, *lvalue)) return {*ty, {TruthyPredicate{std::move(*lvalue), expr.location}}}; - if (FFlag::LuauExtraNilRecovery) - lhsType = stripFromNilAndReport(lhsType, expr.expr->location); + lhsType = stripFromNilAndReport(lhsType, expr.expr->location); if (std::optional ty = getIndexTypeFromType(scope, lhsType, name, expr.location, true)) return {*ty}; - if (!FFlag::LuauMissingUnionPropertyError) - reportError(expr.indexLocation, UnknownProperty{lhsType, expr.index.value}); - - if (!FFlag::LuauExtraNilRecovery) - { - // Try to recover using a union without 'nil' options - if (std::optional strippedUnion = tryStripUnionFromNil(lhsType)) - { - if (std::optional ty = getIndexTypeFromType(scope, *strippedUnion, name, expr.location, false)) - return {*ty}; - } - } - return {errorType}; } @@ -1694,61 +1684,37 @@ std::optional TypeChecker::getIndexTypeFromType( } else if (const UnionTypeVar* utv = get(type)) { - if (FFlag::LuauMissingUnionPropertyError) + std::vector goodOptions; + std::vector badOptions; + + for (TypeId t : utv) { - std::vector goodOptions; - std::vector badOptions; + RecursionLimiter _rl(&recursionCount, FInt::LuauTypeInferRecursionLimit); - for (TypeId t : utv) - { - RecursionLimiter _rl(&recursionCount, FInt::LuauTypeInferRecursionLimit); - - if (std::optional ty = getIndexTypeFromType(scope, t, name, location, false)) - goodOptions.push_back(*ty); - else - badOptions.push_back(t); - } - - if (!badOptions.empty()) - { - if (addErrors) - { - if (goodOptions.empty()) - reportError(location, UnknownProperty{type, name}); - else - reportError(location, MissingUnionProperty{type, badOptions, name}); - } - return std::nullopt; - } - - std::vector result = reduceUnion(goodOptions); - - if (result.size() == 1) - return result[0]; - - return addType(UnionTypeVar{std::move(result)}); + if (std::optional ty = getIndexTypeFromType(scope, t, name, location, false)) + goodOptions.push_back(*ty); + else + badOptions.push_back(t); } - else + + if (!badOptions.empty()) { - std::vector options; - - for (TypeId t : utv->options) + if (addErrors) { - RecursionLimiter _rl(&recursionCount, FInt::LuauTypeInferRecursionLimit); - - if (std::optional ty = getIndexTypeFromType(scope, t, name, location, false)) - options.push_back(*ty); + if (goodOptions.empty()) + reportError(location, UnknownProperty{type, name}); else - return std::nullopt; + reportError(location, MissingUnionProperty{type, badOptions, name}); } - - std::vector result = reduceUnion(options); - - if (result.size() == 1) - return result[0]; - - return addType(UnionTypeVar{std::move(result)}); + return std::nullopt; } + + std::vector result = reduceUnion(goodOptions); + + if (result.size() == 1) + return result[0]; + + return addType(UnionTypeVar{std::move(result)}); } else if (const IntersectionTypeVar* itv = get(type)) { @@ -1765,7 +1731,7 @@ std::optional TypeChecker::getIndexTypeFromType( // If no parts of the intersection had the property we looked up for, it never existed at all. if (parts.empty()) { - if (FFlag::LuauMissingUnionPropertyError && addErrors) + if (addErrors) reportError(location, UnknownProperty{type, name}); return std::nullopt; } @@ -1779,7 +1745,7 @@ std::optional TypeChecker::getIndexTypeFromType( return addType(IntersectionTypeVar{result}); } - if (FFlag::LuauMissingUnionPropertyError && addErrors) + if (addErrors) reportError(location, UnknownProperty{type, name}); return std::nullopt; @@ -2062,8 +2028,7 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprUn case AstExprUnary::Len: tablify(operandType); - if (FFlag::LuauExtraNilRecovery) - operandType = stripFromNilAndReport(operandType, expr.location); + operandType = stripFromNilAndReport(operandType, expr.location); if (get(operandType)) return {errorType}; @@ -2635,8 +2600,7 @@ std::pair TypeChecker::checkLValueBinding(const ScopePtr& scope Name name = expr.index.value; - if (FFlag::LuauExtraNilRecovery) - lhs = stripFromNilAndReport(lhs, expr.expr->location); + lhs = stripFromNilAndReport(lhs, expr.expr->location); if (TableTypeVar* lhsTable = getMutableTableType(lhs)) { @@ -2710,8 +2674,7 @@ std::pair TypeChecker::checkLValueBinding(const ScopePtr& scope TypeId exprType = checkExpr(scope, *expr.expr).type; tablify(exprType); - if (FFlag::LuauExtraNilRecovery) - exprType = stripFromNilAndReport(exprType, expr.expr->location); + exprType = stripFromNilAndReport(exprType, expr.expr->location); TypeId indexType = checkExpr(scope, *expr.index).type; @@ -2738,10 +2701,7 @@ std::pair TypeChecker::checkLValueBinding(const ScopePtr& scope if (!exprTable) { - if (FFlag::LuauExtraNilRecovery) - reportError(TypeError{expr.expr->location, NotATable{exprType}}); - else - reportError(TypeError{expr.location, NotATable{exprType}}); + reportError(TypeError{expr.expr->location, NotATable{exprType}}); return std::pair(errorType, nullptr); } @@ -2910,7 +2870,7 @@ std::pair TypeChecker::checkFunctionSignature( if (FFlag::LuauGenericFunctions) { - std::tie(generics, genericPacks) = createGenericTypes(funScope, expr, expr.generics, expr.genericPacks); + std::tie(generics, genericPacks) = createGenericTypes(funScope, std::nullopt, expr, expr.generics, expr.genericPacks); } TypePackId retPack; @@ -3016,9 +2976,6 @@ std::pair TypeChecker::checkFunctionSignature( if (expectedArgsCurr != expectedArgsEnd) { argType = *expectedArgsCurr; - - if (!FFlag::LuauInferFunctionArgsFix) - ++expectedArgsCurr; } else if (auto expectedArgsTail = expectedArgsCurr.tail()) { @@ -3034,7 +2991,7 @@ std::pair TypeChecker::checkFunctionSignature( funScope->bindings[local] = {argType, local->location}; argTypes.push_back(argType); - if (FFlag::LuauInferFunctionArgsFix && expectedArgsCurr != expectedArgsEnd) + if (expectedArgsCurr != expectedArgsEnd) ++expectedArgsCurr; } @@ -3170,7 +3127,7 @@ void TypeChecker::checkArgumentList( const ScopePtr& scope, Unifier& state, TypePackId argPack, TypePackId paramPack, const std::vector& argLocations) { /* Important terminology refresher: - * A function requires paramaters. + * A function requires parameters. * To call a function, you supply arguments. */ TypePackIterator argIter = begin(argPack); @@ -3402,8 +3359,7 @@ ExprResult TypeChecker::checkExprPack(const ScopePtr& scope, const A if (!FFlag::LuauRankNTypes) instantiate(scope, selfType, expr.func->location); - if (FFlag::LuauExtraNilRecovery) - selfType = stripFromNilAndReport(selfType, expr.func->location); + selfType = stripFromNilAndReport(selfType, expr.func->location); if (std::optional propTy = getIndexTypeFromType(scope, selfType, indexExpr->index.value, expr.location, true)) { @@ -3412,34 +3368,8 @@ ExprResult TypeChecker::checkExprPack(const ScopePtr& scope, const A } else { - if (!FFlag::LuauMissingUnionPropertyError) - reportError(indexExpr->indexLocation, UnknownProperty{selfType, indexExpr->index.value}); - - if (!FFlag::LuauExtraNilRecovery) - { - // Try to recover using a union without 'nil' options - if (std::optional strippedUnion = tryStripUnionFromNil(selfType)) - { - if (std::optional propTy = getIndexTypeFromType(scope, *strippedUnion, indexExpr->index.value, expr.location, false)) - { - selfType = *strippedUnion; - - functionType = *propTy; - actualFunctionType = instantiate(scope, functionType, expr.func->location); - } - } - - if (!actualFunctionType) - { - functionType = errorType; - actualFunctionType = errorType; - } - } - else - { - functionType = errorType; - actualFunctionType = errorType; - } + functionType = errorType; + actualFunctionType = errorType; } } else @@ -3555,8 +3485,7 @@ std::optional> TypeChecker::checkCallOverload(const Scope TypePackId argPack, TypePack* args, const std::vector& argLocations, const ExprResult& argListResult, std::vector& overloadsThatMatchArgCount, std::vector& errors) { - if (FFlag::LuauExtraNilRecovery) - fn = stripFromNilAndReport(fn, expr.func->location); + fn = stripFromNilAndReport(fn, expr.func->location); if (get(fn)) { @@ -4283,6 +4212,12 @@ TypeId TypeChecker::quantify(const ScopePtr& scope, TypeId ty, Location location if (!ftv || !ftv->generics.empty() || !ftv->genericPacks.empty()) return ty; + if (FFlag::LuauQuantifyInPlace2) + { + Luau::quantify(currentModule, ty, scope->level); + return ty; + } + quantification.level = scope->level; quantification.generics.clear(); quantification.genericPacks.clear(); @@ -4491,12 +4426,12 @@ void TypeChecker::merge(RefinementMap& l, const RefinementMap& r) Unifier TypeChecker::mkUnifier(const Location& location) { - return Unifier{¤tModule->internalTypes, currentModule->mode, globalScope, location, Variance::Covariant, iceHandler}; + return Unifier{¤tModule->internalTypes, currentModule->mode, globalScope, location, Variance::Covariant, unifierState}; } Unifier TypeChecker::mkUnifier(const std::vector>& seen, const Location& location) { - return Unifier{¤tModule->internalTypes, currentModule->mode, globalScope, seen, location, Variance::Covariant, iceHandler}; + return Unifier{¤tModule->internalTypes, currentModule->mode, globalScope, seen, location, Variance::Covariant, unifierState}; } TypeId TypeChecker::freshType(const ScopePtr& scope) @@ -4753,7 +4688,7 @@ TypeId TypeChecker::resolveType(const ScopePtr& scope, const AstType& annotation if (FFlag::LuauGenericFunctions) { - std::tie(generics, genericPacks) = createGenericTypes(funcScope, annotation, func->generics, func->genericPacks); + std::tie(generics, genericPacks) = createGenericTypes(funcScope, std::nullopt, annotation, func->generics, func->genericPacks); } // TODO: better error message CLI-39912 @@ -5041,10 +4976,12 @@ TypeId TypeChecker::instantiateTypeFun(const ScopePtr& scope, const TypeFun& tf, } std::pair, std::vector> TypeChecker::createGenericTypes( - const ScopePtr& scope, const AstNode& node, const AstArray& genericNames, const AstArray& genericPackNames) + const ScopePtr& scope, std::optional levelOpt, const AstNode& node, const AstArray& genericNames, const AstArray& genericPackNames) { LUAU_ASSERT(scope->parent); + const TypeLevel level = (FFlag::LuauQuantifyInPlace2 && levelOpt) ? *levelOpt : scope->level; + std::vector generics; for (const AstName& generic : genericNames) { @@ -5063,12 +5000,12 @@ std::pair, std::vector> TypeChecker::createGener { TypeId& cached = scope->parent->typeAliasTypeParameters[n]; if (!cached) - cached = addType(GenericTypeVar{scope->level, n}); + cached = addType(GenericTypeVar{level, n}); g = cached; } else { - g = addType(Unifiable::Generic{scope->level, n}); + g = addType(Unifiable::Generic{level, n}); } generics.push_back(g); @@ -5093,12 +5030,12 @@ std::pair, std::vector> TypeChecker::createGener { TypePackId& cached = scope->parent->typeAliasTypePackParameters[n]; if (!cached) - cached = addTypePack(TypePackVar{Unifiable::Generic{scope->level, n}}); + cached = addTypePack(TypePackVar{Unifiable::Generic{level, n}}); g = cached; } else { - g = addTypePack(TypePackVar{Unifiable::Generic{scope->level, n}}); + g = addTypePack(TypePackVar{Unifiable::Generic{level, n}}); } genericPacks.push_back(g); diff --git a/Analysis/src/TypeVar.cpp b/Analysis/src/TypeVar.cpp index e963fc74..e82f7519 100644 --- a/Analysis/src/TypeVar.cpp +++ b/Analysis/src/TypeVar.cpp @@ -22,6 +22,7 @@ LUAU_FASTINTVARIABLE(LuauTableTypeMaximumStringifierLength, 0) LUAU_FASTFLAG(LuauRankNTypes) LUAU_FASTFLAG(LuauTypeGuardPeelsAwaySubclasses) LUAU_FASTFLAG(LuauTypeAliasPacks) +LUAU_FASTFLAGVARIABLE(LuauRefactorTagging, false) namespace Luau { @@ -217,8 +218,7 @@ std::optional getMetatable(TypeId type) return mtType->metatable; else if (const ClassTypeVar* classType = get(type)) return classType->metatable; - else if (const PrimitiveTypeVar* primitiveType = get(type); - primitiveType && primitiveType->metatable) + else if (const PrimitiveTypeVar* primitiveType = get(type); primitiveType && primitiveType->metatable) { LUAU_ASSERT(primitiveType->type == PrimitiveTypeVar::String); return primitiveType->metatable; @@ -1490,4 +1490,86 @@ std::vector filterMap(TypeId type, TypeIdPredicate predicate) return {}; } +static Tags* getTags(TypeId ty) +{ + ty = follow(ty); + + if (auto ftv = getMutable(ty)) + return &ftv->tags; + else if (auto ttv = getMutable(ty)) + return &ttv->tags; + else if (auto ctv = getMutable(ty)) + return &ctv->tags; + + return nullptr; +} + +void attachTag(TypeId ty, const std::string& tagName) +{ + if (!FFlag::LuauRefactorTagging) + { + if (auto ftv = getMutable(ty)) + { + ftv->tags.emplace_back(tagName); + } + else + { + LUAU_ASSERT(!"Got a non functional type"); + } + } + else + { + if (auto tags = getTags(ty)) + tags->push_back(tagName); + else + LUAU_ASSERT(!"This TypeId does not support tags"); + } +} + +void attachTag(Property& prop, const std::string& tagName) +{ + LUAU_ASSERT(FFlag::LuauRefactorTagging); + + prop.tags.push_back(tagName); +} + +// We would ideally not expose this because it could cause a footgun. +// If the Base class has a tag and you ask if Derived has that tag, it would return false. +// Unfortunately, there's already use cases that's hard to disentangle. For now, we expose it. +bool hasTag(const Tags& tags, const std::string& tagName) +{ + LUAU_ASSERT(FFlag::LuauRefactorTagging); + return std::find(tags.begin(), tags.end(), tagName) != tags.end(); +} + +bool hasTag(TypeId ty, const std::string& tagName) +{ + ty = follow(ty); + + // We special case classes because getTags only returns a pointer to one vector of tags. + // But classes has multiple vector of tags, represented throughout the hierarchy. + if (auto ctv = get(ty)) + { + while (ctv) + { + if (hasTag(ctv->tags, tagName)) + return true; + else if (!ctv->parent) + return false; + + ctv = get(*ctv->parent); + LUAU_ASSERT(ctv); + } + } + else if (auto tags = getTags(ty)) + return hasTag(*tags, tagName); + + return false; +} + +bool hasTag(const Property& prop, const std::string& tagName) +{ + return hasTag(prop.tags, tagName); +} + } // namespace Luau diff --git a/Analysis/src/Unifier.cpp b/Analysis/src/Unifier.cpp index 117cbc28..2539650a 100644 --- a/Analysis/src/Unifier.cpp +++ b/Analysis/src/Unifier.cpp @@ -7,6 +7,7 @@ #include "Luau/TypePack.h" #include "Luau/TypeUtils.h" #include "Luau/TimeTrace.h" +#include "Luau/VisitTypeVar.h" #include @@ -22,9 +23,99 @@ LUAU_FASTFLAGVARIABLE(LuauTableUnificationEarlyTest, false) LUAU_FASTFLAGVARIABLE(LuauSealedTableUnifyOptionalFix, false) LUAU_FASTFLAGVARIABLE(LuauOccursCheckOkWithRecursiveFunctions, false) LUAU_FASTFLAGVARIABLE(LuauTypecheckOpts, false) +LUAU_FASTFLAG(LuauShareTxnSeen); +LUAU_FASTFLAGVARIABLE(LuauCacheUnifyTableResults, false) namespace Luau { +struct SkipCacheForType +{ + SkipCacheForType(const DenseHashMap& skipCacheForType) + : skipCacheForType(skipCacheForType) + { + } + + void cycle(TypeId) {} + void cycle(TypePackId) {} + + bool operator()(TypeId ty, const FreeTypeVar& ftv) + { + result = true; + return false; + } + + bool operator()(TypeId ty, const BoundTypeVar& btv) + { + result = true; + return false; + } + + bool operator()(TypeId ty, const GenericTypeVar& btv) + { + result = true; + return false; + } + + bool operator()(TypeId ty, const TableTypeVar&) + { + TableTypeVar& ttv = *getMutable(ty); + + if (ttv.boundTo) + { + result = true; + return false; + } + + if (ttv.state != TableState::Sealed) + { + result = true; + return false; + } + + return true; + } + + template + bool operator()(TypeId ty, const T& t) + { + const bool* prev = skipCacheForType.find(ty); + + if (prev && *prev) + { + result = true; + return false; + } + + return true; + } + + template + bool operator()(TypePackId, const T&) + { + return true; + } + + bool operator()(TypePackId tp, const FreeTypePack& ftp) + { + result = true; + return false; + } + + bool operator()(TypePackId tp, const BoundTypePack& ftp) + { + result = true; + return false; + } + + bool operator()(TypePackId tp, const GenericTypePack& ftp) + { + result = true; + return false; + } + + const DenseHashMap& skipCacheForType; + bool result = false; +}; static std::optional hasUnificationTooComplex(const ErrorVec& errors) { @@ -39,7 +130,7 @@ static std::optional hasUnificationTooComplex(const ErrorVec& errors) return *it; } -Unifier::Unifier(TypeArena* types, Mode mode, ScopePtr globalScope, const Location& location, Variance variance, InternalErrorReporter* iceHandler) +Unifier::Unifier(TypeArena* types, Mode mode, ScopePtr globalScope, const Location& location, Variance variance, UnifierSharedState& sharedState) : types(types) , mode(mode) , globalScope(std::move(globalScope)) @@ -47,24 +138,39 @@ Unifier::Unifier(TypeArena* types, Mode mode, ScopePtr globalScope, const Locati , variance(variance) , counters(&countersData) , counters_DEPRECATED(std::make_shared()) - , iceHandler(iceHandler) + , sharedState(sharedState) { - LUAU_ASSERT(iceHandler); + LUAU_ASSERT(sharedState.iceHandler); } -Unifier::Unifier(TypeArena* types, Mode mode, ScopePtr globalScope, const std::vector>& seen, const Location& location, - Variance variance, InternalErrorReporter* iceHandler, const std::shared_ptr& counters_DEPRECATED, UnifierCounters* counters) +Unifier::Unifier(TypeArena* types, Mode mode, ScopePtr globalScope, const std::vector>& ownedSeen, const Location& location, + Variance variance, UnifierSharedState& sharedState, const std::shared_ptr& counters_DEPRECATED, UnifierCounters* counters) : types(types) , mode(mode) , globalScope(std::move(globalScope)) - , log(seen) + , log(ownedSeen) , location(location) , variance(variance) , counters(counters ? counters : &countersData) , counters_DEPRECATED(counters_DEPRECATED ? counters_DEPRECATED : std::make_shared()) - , iceHandler(iceHandler) + , sharedState(sharedState) { - LUAU_ASSERT(iceHandler); + LUAU_ASSERT(sharedState.iceHandler); +} + +Unifier::Unifier(TypeArena* types, Mode mode, ScopePtr globalScope, std::vector>* sharedSeen, const Location& location, + Variance variance, UnifierSharedState& sharedState, const std::shared_ptr& counters_DEPRECATED, UnifierCounters* counters) + : types(types) + , mode(mode) + , globalScope(std::move(globalScope)) + , log(sharedSeen) + , location(location) + , variance(variance) + , counters(counters ? counters : &countersData) + , counters_DEPRECATED(counters_DEPRECATED ? counters_DEPRECATED : std::make_shared()) + , sharedState(sharedState) +{ + LUAU_ASSERT(sharedState.iceHandler); } void Unifier::tryUnify(TypeId superTy, TypeId subTy, bool isFunctionCall, bool isIntersection) @@ -74,7 +180,7 @@ void Unifier::tryUnify(TypeId superTy, TypeId subTy, bool isFunctionCall, bool i else counters_DEPRECATED->iterationCount = 0; - return tryUnify_(superTy, subTy, isFunctionCall, isIntersection); + tryUnify_(superTy, subTy, isFunctionCall, isIntersection); } void Unifier::tryUnify_(TypeId superTy, TypeId subTy, bool isFunctionCall, bool isIntersection) @@ -206,6 +312,13 @@ void Unifier::tryUnify_(TypeId superTy, TypeId subTy, bool isFunctionCall, bool if (get(subTy) || get(subTy)) return tryUnifyWithAny(subTy, superTy); + bool cacheEnabled = FFlag::LuauCacheUnifyTableResults && !isFunctionCall && !isIntersection; + auto& cache = sharedState.cachedUnify; + + // What if the types are immutable and we proved their relation before + if (cacheEnabled && cache.contains({superTy, subTy}) && (variance == Covariant || cache.contains({subTy, superTy}))) + return; + // If we have seen this pair of types before, we are currently recursing into cyclic types. // Here, we assume that the types unify. If they do not, we will find out as we roll back // the stack. @@ -257,6 +370,8 @@ void Unifier::tryUnify_(TypeId superTy, TypeId subTy, bool isFunctionCall, bool if (FFlag::LuauUnionHeuristic) { + bool found = false; + const std::string* subName = getName(subTy); if (subName) { @@ -264,6 +379,21 @@ void Unifier::tryUnify_(TypeId superTy, TypeId subTy, bool isFunctionCall, bool { const std::string* optionName = getName(uv->options[i]); if (optionName && *optionName == *subName) + { + found = true; + startIndex = i; + break; + } + } + } + + if (!found && cacheEnabled) + { + for (size_t i = 0; i < uv->options.size(); ++i) + { + TypeId type = uv->options[i]; + + if (cache.contains({type, subTy}) && (variance == Covariant || cache.contains({subTy, type}))) { startIndex = i; break; @@ -311,8 +441,25 @@ void Unifier::tryUnify_(TypeId superTy, TypeId subTy, bool isFunctionCall, bool bool found = false; std::optional unificationTooComplex; - for (TypeId type : uv->parts) + size_t startIndex = 0; + + if (cacheEnabled) { + for (size_t i = 0; i < uv->parts.size(); ++i) + { + TypeId type = uv->parts[i]; + + if (cache.contains({superTy, type}) && (variance == Covariant || cache.contains({type, superTy}))) + { + startIndex = i; + break; + } + } + } + + for (size_t i = 0; i < uv->parts.size(); ++i) + { + TypeId type = uv->parts[(i + startIndex) % uv->parts.size()]; Unifier innerState = makeChildUnifier(); innerState.tryUnify_(superTy, type, isFunctionCall); @@ -342,8 +489,13 @@ void Unifier::tryUnify_(TypeId superTy, TypeId subTy, bool isFunctionCall, bool tryUnifyFunctions(superTy, subTy, isFunctionCall); else if (get(superTy) && get(subTy)) + { tryUnifyTables(superTy, subTy, isIntersection); + if (cacheEnabled && errors.empty()) + cacheResult(superTy, subTy); + } + // tryUnifyWithMetatable assumes its first argument is a MetatableTypeVar. The check is otherwise symmetrical. else if (get(superTy)) tryUnifyWithMetatable(superTy, subTy, /*reversed*/ false); @@ -364,6 +516,41 @@ void Unifier::tryUnify_(TypeId superTy, TypeId subTy, bool isFunctionCall, bool log.popSeen(superTy, subTy); } +void Unifier::cacheResult(TypeId superTy, TypeId subTy) +{ + LUAU_ASSERT(FFlag::LuauCacheUnifyTableResults); + + bool* superTyInfo = sharedState.skipCacheForType.find(superTy); + + if (superTyInfo && *superTyInfo) + return; + + bool* subTyInfo = sharedState.skipCacheForType.find(subTy); + + if (subTyInfo && *subTyInfo) + return; + + auto skipCacheFor = [this](TypeId ty) { + SkipCacheForType visitor{sharedState.skipCacheForType}; + visitTypeVarOnce(ty, visitor, sharedState.seenAny); + + sharedState.skipCacheForType[ty] = visitor.result; + + return visitor.result; + }; + + if (!superTyInfo && skipCacheFor(superTy)) + return; + + if (!subTyInfo && skipCacheFor(subTy)) + return; + + sharedState.cachedUnify.insert({superTy, subTy}); + + if (variance == Invariant) + sharedState.cachedUnify.insert({subTy, superTy}); +} + struct WeirdIter { TypePackId packId; @@ -459,7 +646,7 @@ void Unifier::tryUnify(TypePackId superTp, TypePackId subTp, bool isFunctionCall else counters_DEPRECATED->iterationCount = 0; - return tryUnify_(superTp, subTp, isFunctionCall); + tryUnify_(superTp, subTp, isFunctionCall); } /* @@ -650,11 +837,11 @@ void Unifier::tryUnify_(TypePackId superTp, TypePackId subTp, bool isFunctionCal } // This is a bit weird because we don't actually know expected vs actual. We just know - // subtype vs supertype. If we are checking a return value, we swap these to produce - // the expected error message. + // subtype vs supertype. If we are checking the values returned by a function, we swap + // these to produce the expected error message. size_t expectedSize = size(superTp); size_t actualSize = size(subTp); - if (ctx == CountMismatch::Result || ctx == CountMismatch::Return) + if (ctx == CountMismatch::Result) std::swap(expectedSize, actualSize); errors.push_back(TypeError{location, CountMismatch{expectedSize, actualSize, ctx}}); @@ -797,6 +984,40 @@ void Unifier::tryUnifyTables(TypeId left, TypeId right, bool isIntersection) std::vector missingProperties; std::vector extraProperties; + // Optimization: First test that the property sets are compatible without doing any recursive unification + if (FFlag::LuauTableUnificationEarlyTest && !rt->indexer && rt->state != TableState::Free) + { + for (const auto& [propName, superProp] : lt->props) + { + auto subIter = rt->props.find(propName); + if (subIter == rt->props.end() && !isOptional(superProp.type) && !get(follow(superProp.type))) + missingProperties.push_back(propName); + } + + if (!missingProperties.empty()) + { + errors.push_back(TypeError{location, MissingProperties{left, right, std::move(missingProperties)}}); + return; + } + } + + // And vice versa if we're invariant + if (FFlag::LuauTableUnificationEarlyTest && variance == Invariant && !lt->indexer && lt->state != TableState::Unsealed && lt->state != TableState::Free) + { + for (const auto& [propName, subProp] : rt->props) + { + auto superIter = lt->props.find(propName); + if (superIter == lt->props.end() && !isOptional(subProp.type) && !get(follow(subProp.type))) + extraProperties.push_back(propName); + } + + if (!extraProperties.empty()) + { + errors.push_back(TypeError{location, MissingProperties{left, right, std::move(extraProperties), MissingProperties::Extra}}); + return; + } + } + // Reminder: left is the supertype, right is the subtype. // Width subtyping: any property in the supertype must be in the subtype, // and the types must agree. @@ -833,9 +1054,10 @@ void Unifier::tryUnifyTables(TypeId left, TypeId right, bool isIntersection) innerState.log.rollback(); } else if (isOptional(prop.type) || get(follow(prop.type))) - // TODO: this case is unsound, but without it our test suite fails. CLI-46031 - // TODO: should isOptional(anyType) be true? - {} + // TODO: this case is unsound, but without it our test suite fails. CLI-46031 + // TODO: should isOptional(anyType) be true? + { + } else if (rt->state == TableState::Free) { log(rt); @@ -878,11 +1100,13 @@ void Unifier::tryUnifyTables(TypeId left, TypeId right, bool isIntersection) lt->props[name] = clone; } else if (variance == Covariant) - {} + { + } else if (isOptional(prop.type) || get(follow(prop.type))) - // TODO: this case is unsound, but without it our test suite fails. CLI-46031 - // TODO: should isOptional(anyType) be true? - {} + // TODO: this case is unsound, but without it our test suite fails. CLI-46031 + // TODO: should isOptional(anyType) be true? + { + } else if (lt->state == TableState::Free) { log(lt); @@ -980,10 +1204,10 @@ TypeId Unifier::deeplyOptional(TypeId ty, std::unordered_map see TableTypeVar* resultTtv = getMutable(result); for (auto& [name, prop] : resultTtv->props) prop.type = deeplyOptional(prop.type, seen); - return types->addType(UnionTypeVar{{ singletonTypes.nilType, result }});; + return types->addType(UnionTypeVar{{singletonTypes.nilType, result}}); } else - return types->addType(UnionTypeVar{{ singletonTypes.nilType, ty }}); + return types->addType(UnionTypeVar{{singletonTypes.nilType, ty}}); } void Unifier::DEPRECATED_tryUnifyTables(TypeId left, TypeId right, bool isIntersection) @@ -1247,7 +1471,7 @@ void Unifier::tryUnifySealedTables(TypeId left, TypeId right, bool isIntersectio // If the superTy/left is an immediate part of an intersection type, do not do extra-property check. // Otherwise, we would falsely generate an extra-property-error for 's' in this code: // local a: {n: number} & {s: string} = {n=1, s=""} - // When checking agaist the table '{n: number}'. + // When checking against the table '{n: number}'. if (!isIntersection && lt->state != TableState::Unsealed && !lt->indexer) { // Check for extra properties in the subTy @@ -1697,10 +1921,20 @@ void Unifier::tryUnifyWithAny(TypeId any, TypeId ty) { std::vector queue = {ty}; - tempSeenTy.clear(); - tempSeenTp.clear(); + if (FFlag::LuauCacheUnifyTableResults) + { + sharedState.tempSeenTy.clear(); + sharedState.tempSeenTp.clear(); - Luau::tryUnifyWithAny(queue, *this, tempSeenTy, tempSeenTp, singletonTypes.anyType, anyTP); + Luau::tryUnifyWithAny(queue, *this, sharedState.tempSeenTy, sharedState.tempSeenTp, singletonTypes.anyType, anyTP); + } + else + { + tempSeenTy_DEPRECATED.clear(); + tempSeenTp_DEPRECATED.clear(); + + Luau::tryUnifyWithAny(queue, *this, tempSeenTy_DEPRECATED, tempSeenTp_DEPRECATED, singletonTypes.anyType, anyTP); + } } else { @@ -1721,12 +1955,24 @@ void Unifier::tryUnifyWithAny(TypePackId any, TypePackId ty) { std::vector queue; - tempSeenTy.clear(); - tempSeenTp.clear(); + if (FFlag::LuauCacheUnifyTableResults) + { + sharedState.tempSeenTy.clear(); + sharedState.tempSeenTp.clear(); - queueTypePack(queue, tempSeenTp, *this, ty, any); + queueTypePack(queue, sharedState.tempSeenTp, *this, ty, any); - Luau::tryUnifyWithAny(queue, *this, tempSeenTy, tempSeenTp, anyTy, any); + Luau::tryUnifyWithAny(queue, *this, sharedState.tempSeenTy, sharedState.tempSeenTp, anyTy, any); + } + else + { + tempSeenTy_DEPRECATED.clear(); + tempSeenTp_DEPRECATED.clear(); + + queueTypePack(queue, tempSeenTp_DEPRECATED, *this, ty, any); + + Luau::tryUnifyWithAny(queue, *this, tempSeenTy_DEPRECATED, tempSeenTp_DEPRECATED, anyTy, any); + } } else { @@ -1775,10 +2021,20 @@ void Unifier::occursCheck(TypeId needle, TypeId haystack) { std::unordered_set seen_DEPRECATED; - if (FFlag::LuauTypecheckOpts) - tempSeenTy.clear(); + if (FFlag::LuauCacheUnifyTableResults) + { + if (FFlag::LuauTypecheckOpts) + sharedState.tempSeenTy.clear(); - return occursCheck(seen_DEPRECATED, tempSeenTy, needle, haystack); + return occursCheck(seen_DEPRECATED, sharedState.tempSeenTy, needle, haystack); + } + else + { + if (FFlag::LuauTypecheckOpts) + tempSeenTy_DEPRECATED.clear(); + + return occursCheck(seen_DEPRECATED, tempSeenTy_DEPRECATED, needle, haystack); + } } void Unifier::occursCheck(std::unordered_set& seen_DEPRECATED, DenseHashSet& seen, TypeId needle, TypeId haystack) @@ -1851,10 +2107,20 @@ void Unifier::occursCheck(TypePackId needle, TypePackId haystack) { std::unordered_set seen_DEPRECATED; - if (FFlag::LuauTypecheckOpts) - tempSeenTp.clear(); + if (FFlag::LuauCacheUnifyTableResults) + { + if (FFlag::LuauTypecheckOpts) + sharedState.tempSeenTp.clear(); - return occursCheck(seen_DEPRECATED, tempSeenTp, needle, haystack); + return occursCheck(seen_DEPRECATED, sharedState.tempSeenTp, needle, haystack); + } + else + { + if (FFlag::LuauTypecheckOpts) + tempSeenTp_DEPRECATED.clear(); + + return occursCheck(seen_DEPRECATED, tempSeenTp_DEPRECATED, needle, haystack); + } } void Unifier::occursCheck(std::unordered_set& seen_DEPRECATED, DenseHashSet& seen, TypePackId needle, TypePackId haystack) @@ -1922,7 +2188,10 @@ void Unifier::occursCheck(std::unordered_set& seen_DEPRECATED, Dense Unifier Unifier::makeChildUnifier() { - return Unifier{types, mode, globalScope, log.seen, location, variance, iceHandler, counters_DEPRECATED, counters}; + if (FFlag::LuauShareTxnSeen) + return Unifier{types, mode, globalScope, log.sharedSeen, location, variance, sharedState, counters_DEPRECATED, counters}; + else + return Unifier{types, mode, globalScope, log.ownedSeen, location, variance, sharedState, counters_DEPRECATED, counters}; } bool Unifier::isNonstrictMode() const @@ -1940,12 +2209,12 @@ void Unifier::checkChildUnifierTypeMismatch(const ErrorVec& innerErrors, TypeId void Unifier::ice(const std::string& message, const Location& location) { - iceHandler->ice(message, location); + sharedState.iceHandler->ice(message, location); } void Unifier::ice(const std::string& message) { - iceHandler->ice(message); + sharedState.iceHandler->ice(message); } } // namespace Luau diff --git a/Ast/include/Luau/TimeTrace.h b/Ast/include/Luau/TimeTrace.h index 641dfd3c..503eca61 100644 --- a/Ast/include/Luau/TimeTrace.h +++ b/Ast/include/Luau/TimeTrace.h @@ -194,20 +194,20 @@ LUAU_NOINLINE std::pair createScopeDa } // namespace Luau // Regular scope -#define LUAU_TIMETRACE_SCOPE(name, category) \ +#define LUAU_TIMETRACE_SCOPE(name, category) \ static auto lttScopeStatic = Luau::TimeTrace::createScopeData(name, category); \ Luau::TimeTrace::Scope lttScope(lttScopeStatic.second, lttScopeStatic.first) // A scope without nested scopes that may be skipped if the time it took is less than the threshold -#define LUAU_TIMETRACE_OPTIONAL_TAIL_SCOPE(name, category, microsec) \ +#define LUAU_TIMETRACE_OPTIONAL_TAIL_SCOPE(name, category, microsec) \ static auto lttScopeStaticOptTail = Luau::TimeTrace::createScopeData(name, category); \ Luau::TimeTrace::OptionalTailScope lttScope(lttScopeStaticOptTail.second, lttScopeStaticOptTail.first, microsec) // Extra key/value data can be added to regular scopes -#define LUAU_TIMETRACE_ARGUMENT(name, value) \ - do \ - { \ - if (FFlag::DebugLuauTimeTracing) \ +#define LUAU_TIMETRACE_ARGUMENT(name, value) \ + do \ + { \ + if (FFlag::DebugLuauTimeTracing) \ lttScopeStatic.second.eventArgument(name, value); \ } while (false) @@ -216,8 +216,8 @@ LUAU_NOINLINE std::pair createScopeDa #define LUAU_TIMETRACE_SCOPE(name, category) #define LUAU_TIMETRACE_OPTIONAL_TAIL_SCOPE(name, category, microsec) #define LUAU_TIMETRACE_ARGUMENT(name, value) \ - do \ - { \ + do \ + { \ } while (false) #endif diff --git a/Ast/src/Parser.cpp b/Ast/src/Parser.cpp index 40026d8b..846bc0ba 100644 --- a/Ast/src/Parser.cpp +++ b/Ast/src/Parser.cpp @@ -1593,7 +1593,7 @@ AstTypeOrPack Parser::parseSimpleTypeAnnotation(bool allowPack) { Location location = lexer.current().location; - // For a missing type annoation, capture 'space' between last token and the next one + // For a missing type annotation, capture 'space' between last token and the next one location = Location(lexer.previousLocation().end, lexer.current().location.begin); return {reportTypeAnnotationError(location, {}, /*isMissing*/ true, "Expected type, got %s", lexer.current().toString().c_str()), {}}; diff --git a/Ast/src/TimeTrace.cpp b/Ast/src/TimeTrace.cpp index e6aab20e..ded50e53 100644 --- a/Ast/src/TimeTrace.cpp +++ b/Ast/src/TimeTrace.cpp @@ -77,7 +77,10 @@ struct GlobalContext // Ideally we would want all ThreadContext destructors to run // But in VS, not all thread_local object instances are destroyed for (ThreadContext* context : threads) - context->flushEvents(); + { + if (!context->events.empty()) + context->flushEvents(); + } if (traceFile) fclose(traceFile); diff --git a/CLI/Repl.cpp b/CLI/Repl.cpp index 6baa21ea..4968d080 100644 --- a/CLI/Repl.cpp +++ b/CLI/Repl.cpp @@ -169,7 +169,17 @@ static std::string runCode(lua_State* L, const std::string& source) } else { - std::string error = (status == LUA_YIELD) ? "thread yielded unexpectedly" : lua_tostring(T, -1); + std::string error; + + if (status == LUA_YIELD) + { + error = "thread yielded unexpectedly"; + } + else if (const char* str = lua_tostring(T, -1)) + { + error = str; + } + error += "\nstack backtrace:\n"; error += lua_debugtrace(T); @@ -322,7 +332,17 @@ static bool runFile(const char* name, lua_State* GL) } else { - std::string error = (status == LUA_YIELD) ? "thread yielded unexpectedly" : lua_tostring(L, -1); + std::string error; + + if (status == LUA_YIELD) + { + error = "thread yielded unexpectedly"; + } + else if (const char* str = lua_tostring(L, -1)) + { + error = str; + } + error += "\nstacktrace:\n"; error += lua_debugtrace(L); diff --git a/Compiler/include/Luau/Bytecode.h b/Compiler/include/Luau/Bytecode.h index 07be2e74..4b03ed1c 100644 --- a/Compiler/include/Luau/Bytecode.h +++ b/Compiler/include/Luau/Bytecode.h @@ -208,14 +208,14 @@ enum LuauOpcode LOP_MODK, LOP_POWK, - // AND, OR: perform `and` or `or` operation (selecting first or second register based on whether the first one is truthful) and put the result into target register + // AND, OR: perform `and` or `or` operation (selecting first or second register based on whether the first one is truthy) and put the result into target register // A: target register // B: source register 1 // C: source register 2 LOP_AND, LOP_OR, - // ANDK, ORK: perform `and` or `or` operation (selecting source register or constant based on whether the source register is truthful) and put the result into target register + // ANDK, ORK: perform `and` or `or` operation (selecting source register or constant based on whether the source register is truthy) and put the result into target register // A: target register // B: source register // C: constant table index (0..255) diff --git a/Compiler/src/Compiler.cpp b/Compiler/src/Compiler.cpp index 797ee20d..7750a1d9 100644 --- a/Compiler/src/Compiler.cpp +++ b/Compiler/src/Compiler.cpp @@ -718,9 +718,9 @@ struct Compiler } // compile expr to target temp register - // if the expr (or not expr if onlyTruth is false) is truthful, jump via skipJump - // if the expr (or not expr if onlyTruth is false) is falseful, fall through (target isn't guaranteed to be updated in this case) - // if target is omitted, then the jump behavior is the same - skipJump or fallthrough depending on the truthfulness of the expression + // if the expr (or not expr if onlyTruth is false) is truthy, jump via skipJump + // if the expr (or not expr if onlyTruth is false) is falsy, fall through (target isn't guaranteed to be updated in this case) + // if target is omitted, then the jump behavior is the same - skipJump or fallthrough depending on the truthiness of the expression void compileConditionValue(AstExpr* node, const uint8_t* target, std::vector& skipJump, bool onlyTruth) { // Optimization: we don't need to compute constant values @@ -728,7 +728,7 @@ struct Compiler if (cv && cv->type != Constant::Type_Unknown) { - // note that we only need to compute the value if it's truthful; otherwise we cal fall through + // note that we only need to compute the value if it's truthy; otherwise we cal fall through if (cv->isTruthful() == onlyTruth) { if (target) @@ -747,7 +747,7 @@ struct Compiler case AstExprBinary::And: case AstExprBinary::Or: { - // disambiguation: there's 4 cases (we only need truthful or falseful results based on onlyTruth) + // disambiguation: there's 4 cases (we only need truthy or falsy results based on onlyTruth) // onlyTruth = 1: a and b transforms to a ? b : dontcare // onlyTruth = 1: a or b transforms to a ? a : a // onlyTruth = 0: a and b transforms to !a ? a : b @@ -791,8 +791,8 @@ struct Compiler if (target) { // since target is a temp register, we'll initialize it to 1, and then jump if the comparison is true - // if the comparison is false, we'll fallthrough and target will still be 1 but target has unspecified value for falseful results - // when we only care about falseful values instead of truthful values, the process is the same but with flipped conditionals + // if the comparison is false, we'll fallthrough and target will still be 1 but target has unspecified value for falsy results + // when we only care about falsy values instead of truthy values, the process is the same but with flipped conditionals bytecode.emitABC(LOP_LOADB, *target, onlyTruth ? 1 : 0, 0); } diff --git a/Makefile b/Makefile index 0056870b..7788251d 100644 --- a/Makefile +++ b/Makefile @@ -1,4 +1,5 @@ # This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +.SUFFIXES: MAKEFLAGS+=-r -j8 COMMA=, @@ -107,6 +108,7 @@ coverage: $(TESTS_TARGET) rm default.profraw default-flags.profraw llvm-cov show -format=html -show-instantiations=false -show-line-counts=true -show-region-summary=false -ignore-filename-regex=\(tests\|extern\)/.* -output-dir=coverage --instr-profile default.profdata build/coverage/luau-tests llvm-cov report -ignore-filename-regex=\(tests\|extern\)/.* -show-region-summary=false --instr-profile default.profdata build/coverage/luau-tests + llvm-cov export -format lcov --instr-profile default.profdata build/coverage/luau-tests >coverage.info format: find . -name '*.h' -or -name '*.cpp' | xargs clang-format -i diff --git a/Sources.cmake b/Sources.cmake index 83ed5230..c30cf77d 100644 --- a/Sources.cmake +++ b/Sources.cmake @@ -46,6 +46,7 @@ target_sources(Luau.Analysis PRIVATE Analysis/include/Luau/Module.h Analysis/include/Luau/ModuleResolver.h Analysis/include/Luau/Predicate.h + Analysis/include/Luau/Quantify.h Analysis/include/Luau/RecursionCounter.h Analysis/include/Luau/RequireTracer.h Analysis/include/Luau/Scope.h @@ -63,6 +64,7 @@ target_sources(Luau.Analysis PRIVATE Analysis/include/Luau/TypeVar.h Analysis/include/Luau/Unifiable.h Analysis/include/Luau/Unifier.h + Analysis/include/Luau/UnifierSharedState.h Analysis/include/Luau/Variant.h Analysis/include/Luau/VisitTypeVar.h @@ -77,6 +79,7 @@ target_sources(Luau.Analysis PRIVATE Analysis/src/Linter.cpp Analysis/src/Module.cpp Analysis/src/Predicate.cpp + Analysis/src/Quantify.cpp Analysis/src/RequireTracer.cpp Analysis/src/Scope.cpp Analysis/src/Substitution.cpp diff --git a/VM/include/lualib.h b/VM/include/lualib.h index 7a09ae9f..30cffaff 100644 --- a/VM/include/lualib.h +++ b/VM/include/lualib.h @@ -76,7 +76,7 @@ struct luaL_Buffer char buffer[LUA_BUFFERSIZE]; }; -// when internal buffer storage is exhaused, a mutable string value 'storage' will be placed on the stack +// when internal buffer storage is exhausted, a mutable string value 'storage' will be placed on the stack // in general, functions expect the mutable string buffer to be placed on top of the stack (top-1) // with the exception of luaL_addvalue that expects the value at the top and string buffer further away (top-2) // functions that accept a 'boxloc' support string buffer placement at any location in the stack diff --git a/VM/src/lapi.cpp b/VM/src/lapi.cpp index 01315360..f2e97c66 100644 --- a/VM/src/lapi.cpp +++ b/VM/src/lapi.cpp @@ -13,8 +13,6 @@ #include -LUAU_FASTFLAG(LuauGcFullSkipInactiveThreads) - const char* lua_ident = "$Lua: Lua 5.1.4 Copyright (C) 1994-2008 Lua.org, PUC-Rio $\n" "$Authors: R. Ierusalimschy, L. H. de Figueiredo & W. Celes $\n" "$URL: www.lua.org $\n"; @@ -1153,7 +1151,7 @@ void* lua_newuserdatadtor(lua_State* L, size_t sz, void (*dtor)(void*)) luaC_checkGC(L); luaC_checkthreadsleep(L); Udata* u = luaS_newudata(L, sz + sizeof(dtor), UTAG_IDTOR); - memcpy(u->data + sz, &dtor, sizeof(dtor)); + memcpy(&u->data + sz, &dtor, sizeof(dtor)); setuvalue(L, L->top, u); api_incr_top(L); return u->data; diff --git a/VM/src/laux.cpp b/VM/src/laux.cpp index e37618f7..2a684ee4 100644 --- a/VM/src/laux.cpp +++ b/VM/src/laux.cpp @@ -313,7 +313,7 @@ static size_t getnextbuffersize(lua_State* L, size_t currentsize, size_t desired { size_t newsize = currentsize + currentsize / 2; - // check for size oveflow + // check for size overflow if (SIZE_MAX - desiredsize < currentsize) luaL_error(L, "buffer too large"); diff --git a/VM/src/lcorolib.cpp b/VM/src/lcorolib.cpp index c0b50b96..9724c0e7 100644 --- a/VM/src/lcorolib.cpp +++ b/VM/src/lcorolib.cpp @@ -5,8 +5,6 @@ #include "lstate.h" #include "lvm.h" -LUAU_FASTFLAGVARIABLE(LuauPreferXpush, false) - #define CO_RUN 0 /* running */ #define CO_SUS 1 /* suspended */ #define CO_NOR 2 /* 'normal' (it resumed another coroutine) */ @@ -17,7 +15,7 @@ LUAU_FASTFLAGVARIABLE(LuauPreferXpush, false) static const char* const statnames[] = {"running", "suspended", "normal", "dead"}; -static int costatus(lua_State* L, lua_State* co) +static int auxstatus(lua_State* L, lua_State* co) { if (co == L) return CO_RUN; @@ -25,7 +23,7 @@ static int costatus(lua_State* L, lua_State* co) return CO_SUS; if (co->status == LUA_BREAK) return CO_NOR; - if (co->status != 0) /* some error occured */ + if (co->status != 0) /* some error occurred */ return CO_DEAD; if (co->ci != co->base_ci) /* does it have frames? */ return CO_NOR; @@ -34,11 +32,11 @@ static int costatus(lua_State* L, lua_State* co) return CO_SUS; /* initial state */ } -static int luaB_costatus(lua_State* L) +static int costatus(lua_State* L) { lua_State* co = lua_tothread(L, 1); luaL_argexpected(L, co, 1, "thread"); - lua_pushstring(L, statnames[costatus(L, co)]); + lua_pushstring(L, statnames[auxstatus(L, co)]); return 1; } @@ -47,7 +45,7 @@ static int auxresume(lua_State* L, lua_State* co, int narg) // error handling for edge cases if (co->status != LUA_YIELD) { - int status = costatus(L, co); + int status = auxstatus(L, co); if (status != CO_SUS) { lua_pushfstring(L, "cannot resume %s coroutine", statnames[status]); @@ -115,7 +113,7 @@ static int auxresumecont(lua_State* L, lua_State* co) } } -static int luaB_coresumefinish(lua_State* L, int r) +static int coresumefinish(lua_State* L, int r) { if (r < 0) { @@ -131,7 +129,7 @@ static int luaB_coresumefinish(lua_State* L, int r) } } -static int luaB_coresumey(lua_State* L) +static int coresumey(lua_State* L) { lua_State* co = lua_tothread(L, 1); luaL_argexpected(L, co, 1, "thread"); @@ -141,10 +139,10 @@ static int luaB_coresumey(lua_State* L) if (r == CO_STATUS_BREAK) return interruptThread(L, co); - return luaB_coresumefinish(L, r); + return coresumefinish(L, r); } -static int luaB_coresumecont(lua_State* L, int status) +static int coresumecont(lua_State* L, int status) { lua_State* co = lua_tothread(L, 1); luaL_argexpected(L, co, 1, "thread"); @@ -155,10 +153,10 @@ static int luaB_coresumecont(lua_State* L, int status) int r = auxresumecont(L, co); - return luaB_coresumefinish(L, r); + return coresumefinish(L, r); } -static int luaB_auxwrapfinish(lua_State* L, int r) +static int auxwrapfinish(lua_State* L, int r) { if (r < 0) { @@ -173,7 +171,7 @@ static int luaB_auxwrapfinish(lua_State* L, int r) return r; } -static int luaB_auxwrapy(lua_State* L) +static int auxwrapy(lua_State* L) { lua_State* co = lua_tothread(L, lua_upvalueindex(1)); int narg = cast_int(L->top - L->base); @@ -182,10 +180,10 @@ static int luaB_auxwrapy(lua_State* L) if (r == CO_STATUS_BREAK) return interruptThread(L, co); - return luaB_auxwrapfinish(L, r); + return auxwrapfinish(L, r); } -static int luaB_auxwrapcont(lua_State* L, int status) +static int auxwrapcont(lua_State* L, int status) { lua_State* co = lua_tothread(L, lua_upvalueindex(1)); @@ -195,62 +193,52 @@ static int luaB_auxwrapcont(lua_State* L, int status) int r = auxresumecont(L, co); - return luaB_auxwrapfinish(L, r); + return auxwrapfinish(L, r); } -static int luaB_cocreate(lua_State* L) +static int cocreate(lua_State* L) { luaL_checktype(L, 1, LUA_TFUNCTION); lua_State* NL = lua_newthread(L); - - if (FFlag::LuauPreferXpush) - { - lua_xpush(L, NL, 1); // push function on top of NL - } - else - { - lua_pushvalue(L, 1); /* move function to top */ - lua_xmove(L, NL, 1); /* move function from L to NL */ - } - + lua_xpush(L, NL, 1); // push function on top of NL return 1; } -static int luaB_cowrap(lua_State* L) +static int cowrap(lua_State* L) { - luaB_cocreate(L); + cocreate(L); - lua_pushcfunction(L, luaB_auxwrapy, NULL, 1, luaB_auxwrapcont); + lua_pushcfunction(L, auxwrapy, NULL, 1, auxwrapcont); return 1; } -static int luaB_yield(lua_State* L) +static int coyield(lua_State* L) { int nres = cast_int(L->top - L->base); return lua_yield(L, nres); } -static int luaB_corunning(lua_State* L) +static int corunning(lua_State* L) { if (lua_pushthread(L)) lua_pushnil(L); /* main thread is not a coroutine */ return 1; } -static int luaB_yieldable(lua_State* L) +static int coyieldable(lua_State* L) { lua_pushboolean(L, lua_isyieldable(L)); return 1; } static const luaL_Reg co_funcs[] = { - {"create", luaB_cocreate}, - {"running", luaB_corunning}, - {"status", luaB_costatus}, - {"wrap", luaB_cowrap}, - {"yield", luaB_yield}, - {"isyieldable", luaB_yieldable}, + {"create", cocreate}, + {"running", corunning}, + {"status", costatus}, + {"wrap", cowrap}, + {"yield", coyield}, + {"isyieldable", coyieldable}, {NULL, NULL}, }; @@ -258,7 +246,7 @@ LUALIB_API int luaopen_coroutine(lua_State* L) { luaL_register(L, LUA_COLIBNAME, co_funcs); - lua_pushcfunction(L, luaB_coresumey, "resume", 0, luaB_coresumecont); + lua_pushcfunction(L, coresumey, "resume", 0, coresumecont); lua_setfield(L, -2, "resume"); return 1; diff --git a/VM/src/ldo.cpp b/VM/src/ldo.cpp index 39a61597..328b47e6 100644 --- a/VM/src/ldo.cpp +++ b/VM/src/ldo.cpp @@ -18,6 +18,7 @@ #include LUAU_FASTFLAGVARIABLE(LuauExceptionMessageFix, false) +LUAU_FASTFLAGVARIABLE(LuauCcallRestoreFix, false) /* ** {====================================================== @@ -536,7 +537,13 @@ int luaD_pcall(lua_State* L, Pfunc func, void* u, ptrdiff_t old_top, ptrdiff_t e status = LUA_ERRERR; } - // an error occured, check if we have a protected error callback + if (FFlag::LuauCcallRestoreFix) + { + // Restore nCcalls before calling the debugprotectederror callback which may rely on the proper value to have been restored. + L->nCcalls = oldnCcalls; + } + + // an error occurred, check if we have a protected error callback if (L->global->cb.debugprotectederror) { L->global->cb.debugprotectederror(L); @@ -549,7 +556,10 @@ int luaD_pcall(lua_State* L, Pfunc func, void* u, ptrdiff_t old_top, ptrdiff_t e StkId oldtop = restorestack(L, old_top); luaF_close(L, oldtop); /* close eventual pending closures */ seterrorobj(L, status, oldtop); - L->nCcalls = oldnCcalls; + if (!FFlag::LuauCcallRestoreFix) + { + L->nCcalls = oldnCcalls; + } L->ci = restoreci(L, old_ci); L->base = L->ci->base; restore_stack_limit(L); diff --git a/VM/src/ldo.h b/VM/src/ldo.h index 4fe1c341..72807f0f 100644 --- a/VM/src/ldo.h +++ b/VM/src/ldo.h @@ -37,7 +37,7 @@ /* results from luaD_precall */ #define PCRLUA 0 /* initiated a call to a Lua function */ #define PCRC 1 /* did a call to a C function */ -#define PCRYIELD 2 /* C funtion yielded */ +#define PCRYIELD 2 /* C function yielded */ /* type of protected functions, to be ran by `runprotected' */ typedef void (*Pfunc)(lua_State* L, void* ud); diff --git a/VM/src/lfunc.cpp b/VM/src/lfunc.cpp index 0b543026..64878569 100644 --- a/VM/src/lfunc.cpp +++ b/VM/src/lfunc.cpp @@ -76,7 +76,7 @@ UpVal* luaF_findupval(lua_State* L, StkId level) if (p->v == level) { /* found a corresponding upvalue? */ if (isdead(g, obj2gco(p))) /* is it dead? */ - changewhite(obj2gco(p)); /* ressurect it */ + changewhite(obj2gco(p)); /* resurrect it */ return p; } pp = &p->next; diff --git a/VM/src/lgc.cpp b/VM/src/lgc.cpp index 510a9f54..6553009f 100644 --- a/VM/src/lgc.cpp +++ b/VM/src/lgc.cpp @@ -12,11 +12,9 @@ #include #include -LUAU_FASTFLAGVARIABLE(LuauRescanGrayAgain, false) LUAU_FASTFLAGVARIABLE(LuauRescanGrayAgainForwardBarrier, false) -LUAU_FASTFLAGVARIABLE(LuauGcFullSkipInactiveThreads, false) -LUAU_FASTFLAGVARIABLE(LuauShrinkWeakTables, false) LUAU_FASTFLAGVARIABLE(LuauConsolidatedStep, false) +LUAU_FASTFLAGVARIABLE(LuauSeparateAtomic, false) LUAU_FASTFLAG(LuauArrayBoundary) @@ -66,13 +64,18 @@ static void recordGcStateTime(global_State* g, int startgcstate, double seconds, g->gcstats.currcycle.marktime += seconds; // atomic step had to be performed during the switch and it's tracked separately - if (g->gcstate == GCSsweepstring) + if (!FFlag::LuauSeparateAtomic && g->gcstate == GCSsweepstring) g->gcstats.currcycle.marktime -= g->gcstats.currcycle.atomictime; break; + case GCSatomic: + g->gcstats.currcycle.atomictime += seconds; + break; case GCSsweepstring: case GCSsweep: g->gcstats.currcycle.sweeptime += seconds; break; + default: + LUAU_ASSERT(!"Unexpected GC state"); } if (assist) @@ -183,33 +186,15 @@ static int traversetable(global_State* g, Table* h) if (h->metatable) markobject(g, cast_to(Table*, h->metatable)); - if (FFlag::LuauShrinkWeakTables) + /* is there a weak mode? */ + if (const char* modev = gettablemode(g, h)) { - /* is there a weak mode? */ - if (const char* modev = gettablemode(g, h)) - { - weakkey = (strchr(modev, 'k') != NULL); - weakvalue = (strchr(modev, 'v') != NULL); - if (weakkey || weakvalue) - { /* is really weak? */ - h->gclist = g->weak; /* must be cleared after GC, ... */ - g->weak = obj2gco(h); /* ... so put in the appropriate list */ - } - } - } - else - { - const TValue* mode = gfasttm(g, h->metatable, TM_MODE); - if (mode && ttisstring(mode)) - { /* is there a weak mode? */ - const char* modev = svalue(mode); - weakkey = (strchr(modev, 'k') != NULL); - weakvalue = (strchr(modev, 'v') != NULL); - if (weakkey || weakvalue) - { /* is really weak? */ - h->gclist = g->weak; /* must be cleared after GC, ... */ - g->weak = obj2gco(h); /* ... so put in the appropriate list */ - } + weakkey = (strchr(modev, 'k') != NULL); + weakvalue = (strchr(modev, 'v') != NULL); + if (weakkey || weakvalue) + { /* is really weak? */ + h->gclist = g->weak; /* must be cleared after GC, ... */ + g->weak = obj2gco(h); /* ... so put in the appropriate list */ } } @@ -297,7 +282,7 @@ static void traversestack(global_State* g, lua_State* l, bool clearstack) for (StkId o = l->stack; o < l->top; o++) markvalue(g, o); /* final traversal? */ - if (g->gcstate == GCSatomic || (FFlag::LuauGcFullSkipInactiveThreads && clearstack)) + if (g->gcstate == GCSatomic || clearstack) { StkId stack_end = l->stack + l->stacksize; for (StkId o = l->top; o < stack_end; o++) /* clear not-marked stack slice */ @@ -336,28 +321,16 @@ static size_t propagatemark(global_State* g) lua_State* th = gco2th(o); g->gray = th->gclist; - if (FFlag::LuauGcFullSkipInactiveThreads) + LUAU_ASSERT(!luaC_threadsleeping(th)); + + // threads that are executing and the main thread are not deactivated + bool active = luaC_threadactive(th) || th == th->global->mainthread; + + if (!active && g->gcstate == GCSpropagate) { - LUAU_ASSERT(!luaC_threadsleeping(th)); + traversestack(g, th, /* clearstack= */ true); - // threads that are executing and the main thread are not deactivated - bool active = luaC_threadactive(th) || th == th->global->mainthread; - - if (!active && g->gcstate == GCSpropagate) - { - traversestack(g, th, /* clearstack= */ true); - - l_setbit(th->stackstate, THREAD_SLEEPINGBIT); - } - else - { - th->gclist = g->grayagain; - g->grayagain = o; - - black2gray(o); - - traversestack(g, th, /* clearstack= */ false); - } + l_setbit(th->stackstate, THREAD_SLEEPINGBIT); } else { @@ -385,12 +358,14 @@ static size_t propagatemark(global_State* g) } } -static void propagateall(global_State* g) +static size_t propagateall(global_State* g) { + size_t work = 0; while (g->gray) { - propagatemark(g); + work += propagatemark(g); } + return work; } /* @@ -415,11 +390,14 @@ static int isobjcleared(GCObject* o) /* ** clear collected entries from weaktables */ -static void cleartable(lua_State* L, GCObject* l) +static size_t cleartable(lua_State* L, GCObject* l) { + size_t work = 0; while (l) { Table* h = gco2h(l); + work += sizeof(Table) + sizeof(TValue) * h->sizearray + sizeof(LuaNode) * sizenode(h); + int i = h->sizearray; while (i--) { @@ -433,50 +411,36 @@ static void cleartable(lua_State* L, GCObject* l) { LuaNode* n = gnode(h, i); - if (FFlag::LuauShrinkWeakTables) + // non-empty entry? + if (!ttisnil(gval(n))) { - // non-empty entry? - if (!ttisnil(gval(n))) - { - // can we clear key or value? - if (iscleared(gkey(n)) || iscleared(gval(n))) - { - setnilvalue(gval(n)); /* remove value ... */ - removeentry(n); /* remove entry from table */ - } - else - { - activevalues++; - } - } - } - else - { - if (!ttisnil(gval(n)) && /* non-empty entry? */ - (iscleared(gkey(n)) || iscleared(gval(n)))) + // can we clear key or value? + if (iscleared(gkey(n)) || iscleared(gval(n))) { setnilvalue(gval(n)); /* remove value ... */ removeentry(n); /* remove entry from table */ } + else + { + activevalues++; + } } } - if (FFlag::LuauShrinkWeakTables) + if (const char* modev = gettablemode(L->global, h)) { - if (const char* modev = gettablemode(L->global, h)) + // are we allowed to shrink this weak table? + if (strchr(modev, 's')) { - // are we allowed to shrink this weak table? - if (strchr(modev, 's')) - { - // shrink at 37.5% occupancy - if (activevalues < sizenode(h) * 3 / 8) - luaH_resizehash(L, h, activevalues); - } + // shrink at 37.5% occupancy + if (activevalues < sizenode(h) * 3 / 8) + luaH_resizehash(L, h, activevalues); } } l = h->gclist; } + return work; } static void shrinkstack(lua_State* L) @@ -655,37 +619,49 @@ static void markroot(lua_State* L) g->gcstate = GCSpropagate; } -static void remarkupvals(global_State* g) +static size_t remarkupvals(global_State* g) { - UpVal* uv; - for (uv = g->uvhead.u.l.next; uv != &g->uvhead; uv = uv->u.l.next) + size_t work = 0; + for (UpVal* uv = g->uvhead.u.l.next; uv != &g->uvhead; uv = uv->u.l.next) { + work += sizeof(UpVal); LUAU_ASSERT(uv->u.l.next->u.l.prev == uv && uv->u.l.prev->u.l.next == uv); if (isgray(obj2gco(uv))) markvalue(g, uv->v); } + return work; } -static void atomic(lua_State* L) +static size_t atomic(lua_State* L) { global_State* g = L->global; - g->gcstate = GCSatomic; + size_t work = 0; + + if (FFlag::LuauSeparateAtomic) + { + LUAU_ASSERT(g->gcstate == GCSatomic); + } + else + { + g->gcstate = GCSatomic; + } + /* remark occasional upvalues of (maybe) dead threads */ - remarkupvals(g); + work += remarkupvals(g); /* traverse objects caught by write barrier and by 'remarkupvals' */ - propagateall(g); + work += propagateall(g); /* remark weak tables */ g->gray = g->weak; g->weak = NULL; LUAU_ASSERT(!iswhite(obj2gco(g->mainthread))); markobject(g, L); /* mark running thread */ markmt(g); /* mark basic metatables (again) */ - propagateall(g); + work += propagateall(g); /* remark gray again */ g->gray = g->grayagain; g->grayagain = NULL; - propagateall(g); - cleartable(L, g->weak); /* remove collected objects from weak tables */ + work += propagateall(g); + work += cleartable(L, g->weak); /* remove collected objects from weak tables */ g->weak = NULL; /* flip current white */ g->currentwhite = cast_byte(otherwhite(g)); @@ -693,7 +669,12 @@ static void atomic(lua_State* L) g->sweepgc = &g->rootgc; g->gcstate = GCSsweepstring; - GC_INTERRUPT(GCSatomic); + if (!FFlag::LuauSeparateAtomic) + { + GC_INTERRUPT(GCSatomic); + } + + return work; } static size_t singlestep(lua_State* L) @@ -705,46 +686,24 @@ static size_t singlestep(lua_State* L) case GCSpause: { markroot(L); /* start a new collection */ + LUAU_ASSERT(g->gcstate == GCSpropagate); break; } case GCSpropagate: { - if (FFlag::LuauRescanGrayAgain) + if (g->gray) { - if (g->gray) - { - g->gcstats.currcycle.markitems++; + g->gcstats.currcycle.markitems++; - cost = propagatemark(g); - } - else - { - // perform one iteration over 'gray again' list - g->gray = g->grayagain; - g->grayagain = NULL; - - g->gcstate = GCSpropagateagain; - } + cost = propagatemark(g); } else { - if (g->gray) - { - g->gcstats.currcycle.markitems++; + // perform one iteration over 'gray again' list + g->gray = g->grayagain; + g->grayagain = NULL; - cost = propagatemark(g); - } - else /* no more `gray' objects */ - { - double starttimestamp = lua_clock(); - - g->gcstats.currcycle.atomicstarttimestamp = starttimestamp; - g->gcstats.currcycle.atomicstarttotalsizebytes = g->totalbytes; - - atomic(L); /* finish mark phase */ - - g->gcstats.currcycle.atomictime += lua_clock() - starttimestamp; - } + g->gcstate = GCSpropagateagain; } break; } @@ -758,17 +717,34 @@ static size_t singlestep(lua_State* L) } else /* no more `gray' objects */ { - double starttimestamp = lua_clock(); + if (FFlag::LuauSeparateAtomic) + { + g->gcstate = GCSatomic; + } + else + { + double starttimestamp = lua_clock(); - g->gcstats.currcycle.atomicstarttimestamp = starttimestamp; - g->gcstats.currcycle.atomicstarttotalsizebytes = g->totalbytes; + g->gcstats.currcycle.atomicstarttimestamp = starttimestamp; + g->gcstats.currcycle.atomicstarttotalsizebytes = g->totalbytes; - atomic(L); /* finish mark phase */ + atomic(L); /* finish mark phase */ + LUAU_ASSERT(g->gcstate == GCSsweepstring); - g->gcstats.currcycle.atomictime += lua_clock() - starttimestamp; + g->gcstats.currcycle.atomictime += lua_clock() - starttimestamp; + } } break; } + case GCSatomic: + { + g->gcstats.currcycle.atomicstarttimestamp = lua_clock(); + g->gcstats.currcycle.atomicstarttotalsizebytes = g->totalbytes; + + cost = atomic(L); /* finish mark phase */ + LUAU_ASSERT(g->gcstate == GCSsweepstring); + break; + } case GCSsweepstring: { size_t traversedcount = 0; @@ -806,7 +782,7 @@ static size_t singlestep(lua_State* L) break; } default: - LUAU_ASSERT(0); + LUAU_ASSERT(!"Unexpected GC state"); } return cost; @@ -821,48 +797,25 @@ static size_t gcstep(lua_State* L, size_t limit) case GCSpause: { markroot(L); /* start a new collection */ + LUAU_ASSERT(g->gcstate == GCSpropagate); break; } case GCSpropagate: { - if (FFlag::LuauRescanGrayAgain) + while (g->gray && cost < limit) { - while (g->gray && cost < limit) - { - g->gcstats.currcycle.markitems++; + g->gcstats.currcycle.markitems++; - cost += propagatemark(g); - } - - if (!g->gray) - { - // perform one iteration over 'gray again' list - g->gray = g->grayagain; - g->grayagain = NULL; - - g->gcstate = GCSpropagateagain; - } + cost += propagatemark(g); } - else + + if (!g->gray) { - while (g->gray && cost < limit) - { - g->gcstats.currcycle.markitems++; + // perform one iteration over 'gray again' list + g->gray = g->grayagain; + g->grayagain = NULL; - cost += propagatemark(g); - } - - if (!g->gray) /* no more `gray' objects */ - { - double starttimestamp = lua_clock(); - - g->gcstats.currcycle.atomicstarttimestamp = starttimestamp; - g->gcstats.currcycle.atomicstarttotalsizebytes = g->totalbytes; - - atomic(L); /* finish mark phase */ - - g->gcstats.currcycle.atomictime += lua_clock() - starttimestamp; - } + g->gcstate = GCSpropagateagain; } break; } @@ -877,17 +830,34 @@ static size_t gcstep(lua_State* L, size_t limit) if (!g->gray) /* no more `gray' objects */ { - double starttimestamp = lua_clock(); + if (FFlag::LuauSeparateAtomic) + { + g->gcstate = GCSatomic; + } + else + { + double starttimestamp = lua_clock(); - g->gcstats.currcycle.atomicstarttimestamp = starttimestamp; - g->gcstats.currcycle.atomicstarttotalsizebytes = g->totalbytes; + g->gcstats.currcycle.atomicstarttimestamp = starttimestamp; + g->gcstats.currcycle.atomicstarttotalsizebytes = g->totalbytes; - atomic(L); /* finish mark phase */ + atomic(L); /* finish mark phase */ + LUAU_ASSERT(g->gcstate == GCSsweepstring); - g->gcstats.currcycle.atomictime += lua_clock() - starttimestamp; + g->gcstats.currcycle.atomictime += lua_clock() - starttimestamp; + } } break; } + case GCSatomic: + { + g->gcstats.currcycle.atomicstarttimestamp = lua_clock(); + g->gcstats.currcycle.atomicstarttotalsizebytes = g->totalbytes; + + cost = atomic(L); /* finish mark phase */ + LUAU_ASSERT(g->gcstate == GCSsweepstring); + break; + } case GCSsweepstring: { while (g->sweepstrgc < g->strt.size && cost < limit) @@ -934,7 +904,7 @@ static size_t gcstep(lua_State* L, size_t limit) break; } default: - LUAU_ASSERT(0); + LUAU_ASSERT(!"Unexpected GC state"); } return cost; } @@ -1008,7 +978,7 @@ void luaC_step(lua_State* L, bool assist) startGcCycleStats(g); int lastgcstate = g->gcstate; - double lastttimestamp = lua_clock(); + double lasttimestamp = lua_clock(); if (FFlag::LuauConsolidatedStep) { @@ -1034,15 +1004,15 @@ void luaC_step(lua_State* L, bool assist) double now = lua_clock(); - recordGcStateTime(g, lastgcstate, now - lastttimestamp, assist); + recordGcStateTime(g, lastgcstate, now - lasttimestamp, assist); - lastttimestamp = now; + lasttimestamp = now; lastgcstate = g->gcstate; } } while (lim > 0 && g->gcstate != GCSpause); } - recordGcStateTime(g, lastgcstate, lua_clock() - lastttimestamp, assist); + recordGcStateTime(g, lastgcstate, lua_clock() - lasttimestamp, assist); // at the end of the last cycle if (g->gcstate == GCSpause) @@ -1084,7 +1054,7 @@ void luaC_fullgc(lua_State* L) if (g->gcstate == GCSpause) startGcCycleStats(g); - if (g->gcstate <= GCSpropagateagain) + if (g->gcstate <= (FFlag::LuauSeparateAtomic ? GCSatomic : GCSpropagateagain)) { /* reset sweep marks to sweep all elements (returning them to white) */ g->sweepstrgc = 0; @@ -1095,7 +1065,7 @@ void luaC_fullgc(lua_State* L) g->weak = NULL; g->gcstate = GCSsweepstring; } - LUAU_ASSERT(g->gcstate != GCSpause && g->gcstate != GCSpropagate && g->gcstate != GCSpropagateagain); + LUAU_ASSERT(g->gcstate == GCSsweepstring || g->gcstate == GCSsweep); /* finish any pending sweep phase */ while (g->gcstate != GCSpause) { @@ -1143,14 +1113,11 @@ void luaC_fullgc(lua_State* L) void luaC_barrierupval(lua_State* L, GCObject* v) { - if (FFlag::LuauGcFullSkipInactiveThreads) - { - global_State* g = L->global; - LUAU_ASSERT(iswhite(v) && !isdead(g, v)); + global_State* g = L->global; + LUAU_ASSERT(iswhite(v) && !isdead(g, v)); - if (keepinvariant(g)) - reallymarkobject(g, v); - } + if (keepinvariant(g)) + reallymarkobject(g, v); } void luaC_barrierf(lua_State* L, GCObject* o, GCObject* v) @@ -1778,7 +1745,7 @@ int64_t luaC_allocationrate(lua_State* L) global_State* g = L->global; const double durationthreshold = 1e-3; // avoid measuring intervals smaller than 1ms - if (g->gcstate <= GCSpropagateagain) + if (g->gcstate <= (FFlag::LuauSeparateAtomic ? GCSatomic : GCSpropagateagain)) { double duration = lua_clock() - g->gcstats.lastcycle.endtimestamp; diff --git a/VM/src/lgc.h b/VM/src/lgc.h index dc780bba..f434e506 100644 --- a/VM/src/lgc.h +++ b/VM/src/lgc.h @@ -6,8 +6,6 @@ #include "lobject.h" #include "lstate.h" -LUAU_FASTFLAG(LuauGcFullSkipInactiveThreads) - /* ** Possible states of the Garbage Collector */ @@ -25,10 +23,10 @@ LUAU_FASTFLAG(LuauGcFullSkipInactiveThreads) ** still-black objects. The invariant is restored when sweep ends and ** all objects are white again. */ -#define keepinvariant(g) ((g)->gcstate == GCSpropagate || (g)->gcstate == GCSpropagateagain) +#define keepinvariant(g) ((g)->gcstate == GCSpropagate || (g)->gcstate == GCSpropagateagain || (g)->gcstate == GCSatomic) /* -** some userful bit tricks +** some useful bit tricks */ #define resetbits(x, m) ((x) &= cast_to(uint8_t, ~(m))) #define setbits(x, m) ((x) |= (m)) @@ -147,4 +145,4 @@ LUAI_FUNC void luaC_validate(lua_State* L); LUAI_FUNC void luaC_dump(lua_State* L, void* file, const char* (*categoryName)(lua_State* L, uint8_t memcat)); LUAI_FUNC int64_t luaC_allocationrate(lua_State* L); LUAI_FUNC void luaC_wakethread(lua_State* L); -LUAI_FUNC const char* luaC_statename(int state); \ No newline at end of file +LUAI_FUNC const char* luaC_statename(int state); diff --git a/VM/src/lmem.cpp b/VM/src/lmem.cpp index 2759f3b8..d8b265cb 100644 --- a/VM/src/lmem.cpp +++ b/VM/src/lmem.cpp @@ -199,7 +199,7 @@ static void* luaM_newblock(lua_State* L, int sizeClass) if (page->freeNext >= 0) { - block = page->data + page->freeNext; + block = &page->data + page->freeNext; ASAN_UNPOISON_MEMORY_REGION(block, page->blockSize); page->freeNext -= page->blockSize; diff --git a/VM/src/lstring.cpp b/VM/src/lstring.cpp index d77e17c9..18ee1cda 100644 --- a/VM/src/lstring.cpp +++ b/VM/src/lstring.cpp @@ -226,7 +226,7 @@ void luaS_freeudata(lua_State* L, Udata* u) void (*dtor)(void*) = nullptr; if (u->tag == UTAG_IDTOR) - memcpy(&dtor, u->data + u->len - sizeof(dtor), sizeof(dtor)); + memcpy(&dtor, &u->data + u->len - sizeof(dtor), sizeof(dtor)); else if (u->tag) dtor = L->global->udatagc[u->tag]; diff --git a/VM/src/lvmload.cpp b/VM/src/lvmload.cpp index b932a85b..a168b652 100644 --- a/VM/src/lvmload.cpp +++ b/VM/src/lvmload.cpp @@ -13,7 +13,7 @@ #include // TODO: RAII deallocation doesn't work for longjmp builds if a memory error happens -template +template struct TempBuffer { lua_State* L; @@ -346,6 +346,8 @@ int luau_load(lua_State* L, const char* chunkname, const char* data, size_t size uint32_t mainid = readVarInt(data, size, offset); Proto* main = protos[mainid]; + luaC_checkthreadsleep(L); + Closure* cl = luaF_newLclosure(L, 0, envt, main); setclvalue(L, L->top, cl); incr_top(L); diff --git a/bench/tests/chess.lua b/bench/tests/chess.lua new file mode 100644 index 00000000..87b9abfd --- /dev/null +++ b/bench/tests/chess.lua @@ -0,0 +1,849 @@ + +local bench = script and require(script.Parent.bench_support) or require("bench_support") + +local RANKS = "12345678" +local FILES = "abcdefgh" +local PieceSymbols = "PpRrNnBbQqKk" +local UnicodePieces = {"♙", "♟", "♖", "♜", "♘", "♞", "♗", "♝", "♕", "♛", "♔", "♚"} +local StartingFen = "rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - 0 1" + +-- +-- Lua 5.2 Compat +-- + +if not table.create then + function table.create(n, v) + local result = {} + for i=1,n do result[i] = v end + return result + end +end + +if not table.move then + function table.move(a, from, to, start, target) + local dx = start - from + for i=from,to do + target[i+dx] = a[i] + end + end +end + + +-- +-- Utils +-- + +local function square(s) + return RANKS:find(s:sub(2,2)) * 8 + FILES:find(s:sub(1,1)) - 9 +end + +local function squareName(n) + local file = n % 8 + local rank = (n-file)/8 + return FILES:sub(file+1,file+1) .. RANKS:sub(rank+1,rank+1) +end + +local function moveName(v ) + local from = bit32.extract(v, 6, 6) + local to = bit32.extract(v, 0, 6) + local piece = bit32.extract(v, 20, 4) + local captured = bit32.extract(v, 25, 4) + + local move = PieceSymbols:sub(piece,piece) .. ' ' .. squareName(from) .. (captured ~= 0 and 'x' or '-') .. squareName(to) + + if bit32.extract(v,14) == 1 then + if to > from then + return "O-O" + else + return "O-O-O" + end + end + + local promote = bit32.extract(v,15,4) + if promote ~= 0 then + move = move .. "=" .. PieceSymbols:sub(promote,promote) + end + return move +end + +local function ucimove(m) + local mm = squareName(bit32.extract(m, 6, 6)) .. squareName(bit32.extract(m, 0, 6)) + local promote = bit32.extract(m,15,4) + if promote > 0 then + mm = mm .. PieceSymbols:sub(promote,promote):lower() + end + return mm +end + +local _utils = {squareName, moveName} + +-- +-- Bitboards +-- + +local Bitboard = {} + + +function Bitboard:toString() + local out = {} + + local src = self.h + for x=7,0,-1 do + table.insert(out, RANKS:sub(x+1,x+1)) + table.insert(out, " ") + local bit = bit32.lshift(1,(x%4) * 8) + for x=0,7 do + if bit32.band(src, bit) ~= 0 then + table.insert(out, "x ") + else + table.insert(out, "- ") + end + bit = bit32.lshift(bit, 1) + end + if x == 4 then + src = self.l + end + table.insert(out, "\n") + end + table.insert(out, ' ' .. FILES:gsub('.', '%1 ') .. '\n') + table.insert(out, '#: ' .. self:popcnt() .. "\tl:" .. self.l .. "\th:" .. self.h) + return table.concat(out) +end + + +function Bitboard.from(l ,h ) + return setmetatable({l=l, h=h}, Bitboard) +end + +Bitboard.zero = Bitboard.from(0,0) +Bitboard.full = Bitboard.from(0xFFFFFFFF, 0xFFFFFFFF) + +local Rank1 = Bitboard.from(0x000000FF, 0) +local Rank3 = Bitboard.from(0x00FF0000, 0) +local Rank6 = Bitboard.from(0, 0x0000FF00) +local Rank8 = Bitboard.from(0, 0xFF000000) +local FileA = Bitboard.from(0x01010101, 0x01010101) +local FileB = Bitboard.from(0x02020202, 0x02020202) +local FileC = Bitboard.from(0x04040404, 0x04040404) +local FileD = Bitboard.from(0x08080808, 0x08080808) +local FileE = Bitboard.from(0x10101010, 0x10101010) +local FileF = Bitboard.from(0x20202020, 0x20202020) +local FileG = Bitboard.from(0x40404040, 0x40404040) +local FileH = Bitboard.from(0x80808080, 0x80808080) + +local _Files = {FileA, FileB, FileC, FileD, FileE, FileF, FileG, FileH} + +-- These masks are filled out below for all files +local RightMasks = {FileH} +local LeftMasks = {FileA} + + + +local function popcnt32(i) + i = i - bit32.band(bit32.rshift(i,1), 0x55555555) + i = bit32.band(i, 0x33333333) + bit32.band(bit32.rshift(i,2), 0x33333333) + return bit32.rshift(bit32.band(i + bit32.rshift(i,4), 0x0F0F0F0F) * 0x01010101, 24) +end + +function Bitboard:up() + return self:lshift(8) +end + +function Bitboard:down() + return self:rshift(8) +end + +function Bitboard:right() + return self:band(FileH:inverse()):lshift(1) +end + +function Bitboard:left() + return self:band(FileA:inverse()):rshift(1) +end + +function Bitboard:move(x,y) + local out = self + + if x < 0 then out = out:bandnot(RightMasks[-x]):lshift(-x) end + if x > 0 then out = out:bandnot(LeftMasks[x]):rshift(x) end + + if y < 0 then out = out:rshift(-8 * y) end + if y > 0 then out = out:lshift(8 * y) end + return out +end + + +function Bitboard:popcnt() + return popcnt32(self.l) + popcnt32(self.h) +end + +function Bitboard:band(other ) + return Bitboard.from(bit32.band(self.l,other.l), bit32.band(self.h, other.h)) +end + +function Bitboard:bandnot(other ) + return Bitboard.from(bit32.band(self.l,bit32.bnot(other.l)), bit32.band(self.h, bit32.bnot(other.h))) +end + +function Bitboard:bandempty(other ) + return bit32.band(self.l,other.l) == 0 and bit32.band(self.h, other.h) == 0 +end + +function Bitboard:bor(other ) + return Bitboard.from(bit32.bor(self.l,other.l), bit32.bor(self.h, other.h)) +end + +function Bitboard:bxor(other ) + return Bitboard.from(bit32.bxor(self.l,other.l), bit32.bxor(self.h, other.h)) +end + +function Bitboard:inverse() + return Bitboard.from(bit32.bxor(self.l,0xFFFFFFFF), bit32.bxor(self.h, 0xFFFFFFFF)) +end + +function Bitboard:empty() + return self.h == 0 and self.l == 0 +end + +function Bitboard:ctz() + local target = self.l + local offset = 0 + local result = 0 + + if target == 0 then + target = self.h + result = 32 + end + + if target == 0 then + return 64 + end + + while bit32.extract(target, offset) == 0 do + offset = offset + 1 + end + + return result + offset +end + +function Bitboard:ctzafter(start) + start = start + 1 + if start < 32 then + for i=start,31 do + if bit32.extract(self.l, i) == 1 then return i end + end + end + for i=math.max(32,start),63 do + if bit32.extract(self.h, i-32) == 1 then return i end + end + return 64 +end + + +function Bitboard:lshift(amt) + assert(amt >= 0) + if amt == 0 then return self end + + if amt > 31 then + return Bitboard.from(0, bit32.lshift(self.l, amt-31)) + end + + local l = bit32.lshift(self.l, amt) + local h = bit32.bor( + bit32.lshift(self.h, amt), + bit32.extract(self.l, 32-amt, amt) + ) + return Bitboard.from(l, h) +end + +function Bitboard:rshift(amt) + assert(amt >= 0) + if amt == 0 then return self end + local h = bit32.rshift(self.h, amt) + local l = bit32.bor( + bit32.rshift(self.l, amt), + bit32.lshift(bit32.extract(self.h, 0, amt), 32-amt) + ) + return Bitboard.from(l, h) +end + +function Bitboard:index(i) + if i > 31 then + return bit32.extract(self.h, i - 32) + else + return bit32.extract(self.l, i) + end +end + +function Bitboard:set(i , v) + if i > 31 then + return Bitboard.from(self.l, bit32.replace(self.h, v, i - 32)) + else + return Bitboard.from(bit32.replace(self.l, v, i), self.h) + end +end + +function Bitboard:isolate(i) + return self:band(Bitboard.some(i)) +end + +function Bitboard.some(idx ) + return Bitboard.zero:set(idx, 1) +end + +Bitboard.__index = Bitboard +Bitboard.__tostring = Bitboard.toString + +for i=2,8 do + RightMasks[i] = RightMasks[i-1]:rshift(1):bor(FileH) + LeftMasks[i] = LeftMasks[i-1]:lshift(1):bor(FileA) +end +-- +-- Board +-- + +local Board = {} + + +function Board.new() + local boards = table.create(12, Bitboard.zero) + boards.ocupied = Bitboard.zero + boards.white = Bitboard.zero + boards.black = Bitboard.zero + boards.unocupied = Bitboard.full + boards.ep = Bitboard.zero + boards.castle = Bitboard.zero + boards.toMove = 1 + boards.hm = 0 + boards.moves = 0 + boards.material = 0 + + return setmetatable(boards, Board) +end + +function Board.fromFen(fen ) + local b = Board.new() + local i = 0 + local rank = 7 + local file = 0 + + while true do + i = i + 1 + local p = fen:sub(i,i) + if p == '/' then + rank = rank - 1 + file = 0 + elseif tonumber(p) ~= nil then + file = file + tonumber(p) + else + local pidx = PieceSymbols:find(p) + if pidx == nil then break end + b[pidx] = b[pidx]:set(rank*8+file, 1) + file = file + 1 + end + end + + + local move, castle, ep, hm, m = string.match(fen, "^ ([bw]) ([KQkq-]*) ([a-h-][0-9]?) (%d*) (%d*)", i) + if move == nil then print(fen:sub(i)) end + b.toMove = move == 'w' and 1 or 2 + + if ep ~= "-" then + b.ep = Bitboard.some(square(ep)) + end + + if castle ~= "-" then + local oo = Bitboard.zero + if castle:find("K") then + oo = oo:set(7, 1) + end + if castle:find("Q") then + oo = oo:set(0, 1) + end + if castle:find("k") then + oo = oo:set(63, 1) + end + if castle:find("q") then + oo = oo:set(56, 1) + end + + b.castle = oo + end + + b.hm = hm + b.moves = m + + b:updateCache() + return b + +end + +function Board:index(idx ) + if self.white:index(idx) == 1 then + for p=1,12,2 do + if self[p]:index(idx) == 1 then + return p + end + end + else + for p=2,12,2 do + if self[p]:index(idx) == 1 then + return p + end + end + end + + return 0 +end + +function Board:updateCache() + for i=1,11,2 do + self.white = self.white:bor(self[i]) + self.black = self.black:bor(self[i+1]) + end + + self.ocupied = self.black:bor(self.white) + self.unocupied = self.ocupied:inverse() + self.material = + 100*self[1]:popcnt() - 100*self[2]:popcnt() + + 500*self[3]:popcnt() - 500*self[4]:popcnt() + + 300*self[5]:popcnt() - 300*self[6]:popcnt() + + 300*self[7]:popcnt() - 300*self[8]:popcnt() + + 900*self[9]:popcnt() - 900*self[10]:popcnt() + +end + +function Board:fen() + local out = {} + local s = 0 + local idx = 56 + for i=0,63 do + if i % 8 == 0 and i > 0 then + idx = idx - 16 + if s > 0 then + table.insert(out, '' .. s) + s = 0 + end + table.insert(out, '/') + end + local p = self:index(idx) + if p == 0 then + s = s + 1 + else + if s > 0 then + table.insert(out, '' .. s) + s = 0 + end + table.insert(out, PieceSymbols:sub(p,p)) + end + + idx = idx + 1 + end + if s > 0 then + table.insert(out, '' .. s) + end + + table.insert(out, self.toMove == 1 and ' w ' or ' b ') + if self.castle:empty() then + table.insert(out, '-') + else + if self.castle:index(7) == 1 then table.insert(out, 'K') end + if self.castle:index(0) == 1 then table.insert(out, 'Q') end + if self.castle:index(63) == 1 then table.insert(out, 'k') end + if self.castle:index(56) == 1 then table.insert(out, 'q') end + end + + table.insert(out, ' ') + if self.ep:empty() then + table.insert(out, '-') + else + table.insert(out, squareName(self.ep:ctz())) + end + + table.insert(out, ' ' .. self.hm) + table.insert(out, ' ' .. self.moves) + + return table.concat(out) +end + +function Board:pmoves(idx) + return self:generate(idx) +end + +function Board:pcaptures(idx) + return self:generate(idx):band(self.ocupied) +end + +local ROOK_SLIDES = {{1,0}, {-1,0}, {0,1}, {0,-1}} +local BISHOP_SLIDES = {{1,1}, {-1,1}, {1,-1}, {-1,-1}} +local QUEEN_SLIDES = {{1,0}, {-1,0}, {0,1}, {0,-1}, {1,1}, {-1,1}, {1,-1}, {-1,-1}} +local KNIGHT_MOVES = {{2,1}, {2,-1}, {-2,1}, {-2,-1}, {1,2}, {1,-2}, {-1,2}, {-1,-2}} + +function Board:generate(idx) + local piece = self:index(idx) + local r = Bitboard.some(idx) + local out = Bitboard.zero + local type = bit32.rshift(piece - 1, 1) + local cancapture = piece % 2 == 1 and self.black or self.white + + if piece == 0 then return Bitboard.zero end + + if type == 0 then + -- Pawn + local d = -(piece*2 - 3) + local movetwo = piece == 1 and Rank3 or Rank6 + + out = out:bor(r:move(0,d):band(self.unocupied)) + out = out:bor(out:band(movetwo):move(0,d):band(self.unocupied)) + + local captures = r:move(0,d) + captures = captures:right():bor(captures:left()) + + if not captures:bandempty(self.ep) then + out = out:bor(self.ep) + end + + captures = captures:band(cancapture) + out = out:bor(captures) + + return out + elseif type == 5 then + -- King + for x=-1,1,1 do + for y = -1,1,1 do + local w = r:move(x,y) + if self.ocupied:bandempty(w) then + out = out:bor(w) + else + if not cancapture:bandempty(w) then + out = out:bor(w) + end + end + end + end + elseif type == 2 then + -- Knight + for _,j in ipairs(KNIGHT_MOVES) do + local w = r:move(j[1],j[2]) + + if self.ocupied:bandempty(w) then + out = out:bor(w) + else + if not cancapture:bandempty(w) then + out = out:bor(w) + end + end + end + else + -- Sliders (Rook, Bishop, Queen) + local slides + if type == 1 then + slides = ROOK_SLIDES + elseif type == 3 then + slides = BISHOP_SLIDES + else + slides = QUEEN_SLIDES + end + + for _, op in ipairs(slides) do + local w = r + for i=1,7 do + w = w:move(op[1], op[2]) + if w:empty() then break end + + if self.ocupied:bandempty(w) then + out = out:bor(w) + else + if not cancapture:bandempty(w) then + out = out:bor(w) + end + break + end + end + end + end + + + return out +end + +-- 0-5 - From Square +-- 6-11 - To Square +-- 12 - is Check +-- 13 - Is EnPassent +-- 14 - Is Castle +-- 15-19 - Promotion Piece +-- 20-24 - Moved Pice +-- 25-29 - Captured Piece + + +function Board:toString(mark ) + local out = {} + for x=8,1,-1 do + table.insert(out, RANKS:sub(x,x) .. " ") + + for y=1,8 do + local n = 8*x+y-9 + local i = self:index(n) + if i == 0 then + table.insert(out, '-') + else + -- out = out .. PieceSymbols:sub(i,i) + table.insert(out, UnicodePieces[i]) + end + if mark ~= nil and mark:index(n) ~= 0 then + table.insert(out, ')') + elseif mark ~= nil and n < 63 and y < 8 and mark:index(n+1) ~= 0 then + table.insert(out, '(') + else + table.insert(out, ' ') + end + end + + table.insert(out, "\n") + end + table.insert(out, ' ' .. FILES:gsub('.', '%1 ') .. '\n') + table.insert(out, (self.toMove == 1 and "White" or "Black") .. ' e:' .. (self.material/100) .. "\n") + return table.concat(out) +end + +function Board:moveList() + local tm = self.toMove == 1 and self.white or self.black + local castle_rank = self.toMove == 1 and Rank1 or Rank8 + local out = {} + local function emit(id) + if not self:applyMove(id):illegalyChecked() then + table.insert(out, id) + end + end + + local cr = tm:band(self.castle):band(castle_rank) + if not cr:empty() then + local p = self.toMove == 1 and 11 or 12 + local tcolor = self.toMove == 1 and self.black or self.white + local kidx = self[p]:ctz() + + + local castle = bit32.replace(0, p, 20, 4) + castle = bit32.replace(castle, kidx, 6, 6) + castle = bit32.replace(castle, 1, 14) + + + local mustbeemptyl = LeftMasks[4]:bxor(FileA):band(castle_rank) + local cantbethreatened = FileD:bor(FileC):band(castle_rank):bor(self[p]) + if + not cr:bandempty(FileA) and + mustbeemptyl:bandempty(self.ocupied) and + not self:isSquareThreatened(cantbethreatened, tcolor) + then + emit(bit32.replace(castle, kidx - 2, 0, 6)) + end + + + local mustbeemptyr = RightMasks[3]:bxor(FileH):band(castle_rank) + if + not cr:bandempty(FileH) and + mustbeemptyr:bandempty(self.ocupied) and + not self:isSquareThreatened(mustbeemptyr:bor(self[p]), tcolor) + then + emit(bit32.replace(castle, kidx + 2, 0, 6)) + end + end + + local sq = tm:ctz() + repeat + local p = self:index(sq) + local moves = self:pmoves(sq) + + while not moves:empty() do + local m = moves:ctz() + moves = moves:set(m, 0) + local id = bit32.replace(m, sq, 6, 6) + id = bit32.replace(id, p, 20, 4) + local mbb = Bitboard.some(m) + if not self.ocupied:bandempty(mbb) then + id = bit32.replace(id, self:index(m), 25, 4) + end + + -- Check if pawn needs to be promoted + if p == 1 and m >= 8*7 then + for i=3,9,2 do + emit(bit32.replace(id, i, 15, 4)) + end + elseif p == 2 and m < 8 then + for i=4,10,2 do + emit(bit32.replace(id, i, 15, 4)) + end + else + emit(id) + end + end + sq = tm:ctzafter(sq) + until sq == 64 + return out +end + +function Board:illegalyChecked() + local target = self.toMove == 1 and self[PieceSymbols:find("k")] or self[PieceSymbols:find("K")] + return self:isSquareThreatened(target, self.toMove == 1 and self.white or self.black) +end + +function Board:isSquareThreatened(target , color ) + local tm = color + local sq = tm:ctz() + repeat + local moves = self:pmoves(sq) + if not moves:bandempty(target) then + return true + end + sq = color:ctzafter(sq) + until sq == 64 + return false +end + +function Board:perft(depth ) + if depth == 0 then return 1 end + if depth == 1 then + return #self:moveList() + end + local result = 0 + for k,m in ipairs(self:moveList()) do + local c = self:applyMove(m):perft(depth - 1) + if c == 0 then + -- Perft only counts leaf nodes at target depth + -- result = result + 1 + else + result = result + c + end + end + return result +end + + +function Board:applyMove(move ) + local out = Board.new() + table.move(self, 1, 12, 1, out) + local from = bit32.extract(move, 6, 6) + local to = bit32.extract(move, 0, 6) + local promote = bit32.extract(move, 15, 4) + local piece = self:index(from) + local captured = self:index(to) + local tom = Bitboard.some(to) + local isCastle = bit32.extract(move, 14) + + if piece % 2 == 0 then + out.moves = self.moves + 1 + end + + if captured == 1 or piece < 3 then + out.hm = 0 + else + out.hm = self.hm + 1 + end + out.castle = self.castle + out.toMove = self.toMove == 1 and 2 or 1 + + if isCastle == 1 then + local rank = piece == 11 and Rank1 or Rank8 + local colorOffset = piece - 11 + + out[3 + colorOffset] = out[3 + colorOffset]:bandnot(from < to and FileH or FileA) + out[3 + colorOffset] = out[3 + colorOffset]:bor((from < to and FileF or FileD):band(rank)) + + out[piece] = (from < to and FileG or FileC):band(rank) + out.castle = out.castle:bandnot(rank) + out:updateCache() + return out + end + + if piece < 3 then + local dist = math.abs(to - from) + -- Pawn moved two squares, set ep square + if dist == 16 then + out.ep = Bitboard.some((from + to) / 2) + end + + -- Remove enpasent capture + if not tom:bandempty(self.ep) then + if piece == 1 then + out[2] = out[2]:bandnot(self.ep:down()) + end + if piece == 2 then + out[1] = out[1]:bandnot(self.ep:up()) + end + end + end + + if piece == 3 or piece == 4 then + out.castle = out.castle:set(from, 0) + end + + if piece > 10 then + local rank = piece == 11 and Rank1 or Rank8 + out.castle = out.castle:bandnot(rank) + end + + out[piece] = out[piece]:set(from, 0) + if promote == 0 then + out[piece] = out[piece]:set(to, 1) + else + out[promote] = out[promote]:set(to, 1) + end + if captured ~= 0 then + out[captured] = out[captured]:set(to, 0) + end + + out:updateCache() + return out +end + +Board.__index = Board +Board.__tostring = Board.toString +-- +-- Main +-- + +local failures = 0 +local function test(fen, ply, target) + local b = Board.fromFen(fen) + if b:fen() ~= fen then + print("FEN MISMATCH", fen, b:fen()) + failures = failures + 1 + return + end + + local found = b:perft(ply) + if found ~= target then + print(fen, "Found", found, "target", target) + failures = failures + 1 + for k,v in pairs(b:moveList()) do + print(ucimove(v) .. ': ' .. (ply > 1 and b:applyMove(v):perft(ply-1) or '1')) + end + --error("Test Failure") + else + print("OK", found, fen) + end +end + +-- From https://www.chessprogramming.org/Perft_Results +-- If interpreter, computers, or algorithm gets too fast +-- feel free to go deeper + +local testCases = {} +local function addTest(...) table.insert(testCases, {...}) end + +addTest(StartingFen, 3, 8902) +addTest("r3k2r/p1ppqpb1/bn2pnp1/3PN3/1p2P3/2N2Q1p/PPPBBPPP/R3K2R w KQkq - 0 0", 2, 2039) +addTest("8/2p5/3p4/KP5r/1R3p1k/8/4P1P1/8 w - - 0 0", 3, 2812) +addTest("r3k2r/Pppp1ppp/1b3nbN/nP6/BBP1P3/q4N2/Pp1P2PP/R2Q1RK1 w kq - 0 1", 3, 9467) +addTest("rnbq1k1r/pp1Pbppp/2p5/8/2B5/8/PPP1NnPP/RNBQK2R w KQ - 1 8", 2, 1486) +addTest("r4rk1/1pp1qppp/p1np1n2/2b1p1B1/2B1P1b1/P1NP1N2/1PP1QPPP/R4RK1 w - - 0 10", 2, 2079) + + +local function chess() + for k,v in ipairs(testCases) do + test(v[1],v[2],v[3]) + end +end + +bench.runCode(chess, "chess") diff --git a/bench/tests/shootout/scimark.lua b/bench/tests/shootout/scimark.lua index 41d97bb8..ad0557b1 100644 --- a/bench/tests/shootout/scimark.lua +++ b/bench/tests/shootout/scimark.lua @@ -30,7 +30,7 @@ ------------------------------------------------------------------------------ ------------------------------------------------------------------------------ --- Modificatin to be compatible with Lua 5.3 +-- Modification to be compatible with Lua 5.3 ------------------------------------------------------------------------------ local bench = script and require(script.Parent.bench_support) or require("bench_support") diff --git a/tests/Autocomplete.test.cpp b/tests/Autocomplete.test.cpp index 07910a0a..44b8362d 100644 --- a/tests/Autocomplete.test.cpp +++ b/tests/Autocomplete.test.cpp @@ -1596,7 +1596,7 @@ TEST_CASE_FIXTURE(ACFixture, "type_correct_argument_type_suggestion") local function target(a: number, b: string) return a + #b end local function d(a: n@1, b) - return target(a, b) + return target(a, b) end )"); @@ -1609,7 +1609,7 @@ end local function target(a: number, b: string) return a + #b end local function d(a, b: s@1) - return target(a, b) + return target(a, b) end )"); @@ -1622,7 +1622,7 @@ end local function target(a: number, b: string) return a + #b end local function d(a:@1 @2, b) - return target(a, b) + return target(a, b) end )"); @@ -1640,7 +1640,7 @@ end local function target(a: number, b: string) return a + #b end local function d(a, b: @1)@2: number - return target(a, b) + return target(a, b) end )"); @@ -1682,7 +1682,7 @@ local x = target(function(a: n@1 local function target(callback: (a: number, b: string) -> number) return callback(4, "hello") end local x = target(function(a: n@1, b: @2) - return a + #b + return a + #b end) )"); @@ -1700,7 +1700,7 @@ end) local function target(callback: (...number) -> number) return callback(1, 2, 3) end local x = target(function(a: n@1) - return a + return a end )"); @@ -1716,7 +1716,7 @@ TEST_CASE_FIXTURE(ACFixture, "type_correct_expected_argument_type_pack_suggestio local function target(callback: (...number) -> number) return callback(1, 2, 3) end local x = target(function(...:n@1) - return a + return a end )"); @@ -1729,7 +1729,7 @@ end local function target(callback: (...number) -> number) return callback(1, 2, 3) end local x = target(function(a:number, b:number, ...:@1) - return a + b + return a + b end )"); @@ -1745,7 +1745,7 @@ TEST_CASE_FIXTURE(ACFixture, "type_correct_expected_return_type_suggestion") local function target(callback: () -> number) return callback() end local x = target(function(): n@1 - return 1 + return 1 end )"); @@ -1758,7 +1758,7 @@ end local function target(callback: () -> (number, number)) return callback() end local x = target(function(): (number, n@1 - return 1, 2 + return 1, 2 end )"); @@ -1774,7 +1774,7 @@ TEST_CASE_FIXTURE(ACFixture, "type_correct_expected_return_type_pack_suggestion" local function target(callback: () -> ...number) return callback() end local x = target(function(): ...n@1 - return 1, 2, 3 + return 1, 2, 3 end )"); @@ -1787,7 +1787,7 @@ end local function target(callback: () -> ...number) return callback() end local x = target(function(): (number, number, ...n@1 - return 1, 2, 3 + return 1, 2, 3 end )"); diff --git a/tests/Compiler.test.cpp b/tests/Compiler.test.cpp index 54a31a68..bbac3302 100644 --- a/tests/Compiler.test.cpp +++ b/tests/Compiler.test.cpp @@ -768,11 +768,11 @@ TEST_CASE("CaptureSelf") local MaterialsListClass = {} function MaterialsListClass:_MakeToolTip(guiElement, text) - local function updateTooltipPosition() - self._tweakingTooltipFrame = 5 - end + local function updateTooltipPosition() + self._tweakingTooltipFrame = 5 + end - updateTooltipPosition() + updateTooltipPosition() end return MaterialsListClass @@ -2001,14 +2001,14 @@ TEST_CASE("UpvaluesLoopsBytecode") { CHECK_EQ("\n" + compileFunction(R"( function test() - for i=1,10 do + for i=1,10 do i = i - foo(function() return i end) - if bar then - break - end - end - return 0 + foo(function() return i end) + if bar then + break + end + end + return 0 end )", 1), @@ -2035,14 +2035,14 @@ RETURN R0 1 CHECK_EQ("\n" + compileFunction(R"( function test() - for i in ipairs(data) do + for i in ipairs(data) do i = i - foo(function() return i end) - if bar then - break - end - end - return 0 + foo(function() return i end) + if bar then + break + end + end + return 0 end )", 1), @@ -2068,17 +2068,17 @@ RETURN R0 1 CHECK_EQ("\n" + compileFunction(R"( function test() - local i = 0 - while i < 5 do - local j + local i = 0 + while i < 5 do + local j j = i - foo(function() return j end) - i = i + 1 - if bar then - break - end - end - return 0 + foo(function() return j end) + i = i + 1 + if bar then + break + end + end + return 0 end )", 1), @@ -2105,17 +2105,17 @@ RETURN R1 1 CHECK_EQ("\n" + compileFunction(R"( function test() - local i = 0 - repeat - local j + local i = 0 + repeat + local j j = i - foo(function() return j end) - i = i + 1 - if bar then - break - end - until i < 5 - return 0 + foo(function() return j end) + i = i + 1 + if bar then + break + end + until i < 5 + return 0 end )", 1), @@ -2304,10 +2304,10 @@ local Value1, Value2, Value3 = ... local Table = {} Table.SubTable["Key"] = { - Key1 = Value1, - Key2 = Value2, - Key3 = Value3, - Key4 = true, + Key1 = Value1, + Key2 = Value2, + Key3 = Value3, + Key4 = true, } )"); diff --git a/tests/Conformance.test.cpp b/tests/Conformance.test.cpp index 5a697a49..06b3c523 100644 --- a/tests/Conformance.test.cpp +++ b/tests/Conformance.test.cpp @@ -801,4 +801,17 @@ TEST_CASE("IfElseExpression") runConformance("ifelseexpr.lua"); } +TEST_CASE("TagMethodError") +{ + ScopedFastFlag sff{"LuauCcallRestoreFix", true}; + + runConformance("tmerror.lua", [](lua_State* L) { + auto* cb = lua_callbacks(L); + + cb->debugprotectederror = [](lua_State* L) { + CHECK(lua_isyieldable(L)); + }; + }); +} + TEST_SUITE_END(); diff --git a/tests/IostreamOptional.h b/tests/IostreamOptional.h index 9f874899..e55b5b0c 100644 --- a/tests/IostreamOptional.h +++ b/tests/IostreamOptional.h @@ -2,6 +2,9 @@ #pragma once #include +#include + +namespace std { inline std::ostream& operator<<(std::ostream& lhs, const std::nullopt_t&) { @@ -9,10 +12,12 @@ inline std::ostream& operator<<(std::ostream& lhs, const std::nullopt_t&) } template -std::ostream& operator<<(std::ostream& lhs, const std::optional& t) +auto operator<<(std::ostream& lhs, const std::optional& t) -> decltype(lhs << *t) // SFINAE to only instantiate << for supported types { if (t) return lhs << *t; else return lhs << "none"; } + +} // namespace std diff --git a/tests/Linter.test.cpp b/tests/Linter.test.cpp index a9ed139f..37f1b60b 100644 --- a/tests/Linter.test.cpp +++ b/tests/Linter.test.cpp @@ -791,13 +791,13 @@ TEST_CASE_FIXTURE(Fixture, "TypeAnnotationsShouldNotProduceWarnings") { LintResult result = lint(R"(--!strict type InputData = { - id: number, - inputType: EnumItem, - inputState: EnumItem, - updated: number, - position: Vector3, - keyCode: EnumItem, - name: string + id: number, + inputType: EnumItem, + inputState: EnumItem, + updated: number, + position: Vector3, + keyCode: EnumItem, + name: string } )"); diff --git a/tests/TypeInfer.aliases.test.cpp b/tests/TypeInfer.aliases.test.cpp index 045f0230..f580604c 100644 --- a/tests/TypeInfer.aliases.test.cpp +++ b/tests/TypeInfer.aliases.test.cpp @@ -554,4 +554,54 @@ TEST_CASE_FIXTURE(Fixture, "non_recursive_aliases_that_reuse_a_generic_name") CHECK_EQ("{number | string}", toString(requireType("p"), {true})); } +/* + * We had a problem where all type aliases would be prototyped into a child scope that happened + * to have the same level. This caused a problem where, if a sibling function referred to that + * type alias in its type signature, it would erroneously be quantified away, even though it doesn't + * actually belong to the function. + * + * We solved this by ascribing a unique subLevel to each prototyped alias. + */ +TEST_CASE_FIXTURE(Fixture, "do_not_quantify_unresolved_aliases") +{ + CheckResult result = check(R"( + --!strict + + local KeyPool = {} + + local function newkey(pool: KeyPool, index) + return {} + end + + function newKeyPool() + local pool = { + available = {} :: {Key}, + } + + return setmetatable(pool, KeyPool) + end + + export type KeyPool = typeof(newKeyPool()) + export type Key = typeof(newkey(newKeyPool(), 1)) + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +/* + * We keep a cache of type alias onto TypeVar to prevent infinite types from + * being constructed via recursive or corecursive aliases. We have to adjust + * the TypeLevels of those generic TypeVars so that the unifier doesn't think + * they have improperly leaked out of their scope. + */ +TEST_CASE_FIXTURE(Fixture, "generic_typevars_are_not_considered_to_escape_their_scope_if_they_are_reused_in_multiple_aliases") +{ + CheckResult result = check(R"( + type Array = {T} + type Exclude = T + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.builtins.test.cpp b/tests/TypeInfer.builtins.test.cpp index e8974776..17e32e9f 100644 --- a/tests/TypeInfer.builtins.test.cpp +++ b/tests/TypeInfer.builtins.test.cpp @@ -359,7 +359,7 @@ TEST_CASE_FIXTURE(Fixture, "table_insert_correctly_infers_type_of_array_2_args_o CHECK_EQ(typeChecker.stringType, requireType("s")); } -TEST_CASE_FIXTURE(Fixture, "table_insert_corrrectly_infers_type_of_array_3_args_overload") +TEST_CASE_FIXTURE(Fixture, "table_insert_correctly_infers_type_of_array_3_args_overload") { CheckResult result = check(R"( local t = {} diff --git a/tests/TypeInfer.classes.test.cpp b/tests/TypeInfer.classes.test.cpp index 6da33a08..eabf7e65 100644 --- a/tests/TypeInfer.classes.test.cpp +++ b/tests/TypeInfer.classes.test.cpp @@ -437,8 +437,6 @@ TEST_CASE_FIXTURE(ClassFixture, "class_unification_type_mismatch_is_correct_orde TEST_CASE_FIXTURE(ClassFixture, "optional_class_field_access_error") { - ScopedFastFlag luauExtraNilRecovery("LuauExtraNilRecovery", true); - CheckResult result = check(R"( local b: Vector2? = nil local a = b.X + b.Z diff --git a/tests/TypeInfer.generics.test.cpp b/tests/TypeInfer.generics.test.cpp index 3a04a18f..581375a1 100644 --- a/tests/TypeInfer.generics.test.cpp +++ b/tests/TypeInfer.generics.test.cpp @@ -695,4 +695,25 @@ TEST_CASE_FIXTURE(Fixture, "typefuns_sharing_types") CHECK(requireType("y1") == requireType("y2")); } +TEST_CASE_FIXTURE(Fixture, "bound_tables_do_not_clone_original_fields") +{ + ScopedFastFlag luauRankNTypes{"LuauRankNTypes", true}; + ScopedFastFlag luauCloneBoundTables{"LuauCloneBoundTables", true}; + + CheckResult result = check(R"( +local exports = {} +local nested = {} + +nested.name = function(t, k) + local a = t.x.y + return rawget(t, k) +end + +exports.nested = nested +return exports + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.provisional.test.cpp b/tests/TypeInfer.provisional.test.cpp index 8bcb0242..419da8ad 100644 --- a/tests/TypeInfer.provisional.test.cpp +++ b/tests/TypeInfer.provisional.test.cpp @@ -9,12 +9,13 @@ #include LUAU_FASTFLAG(LuauEqConstraint) +LUAU_FASTFLAG(LuauQuantifyInPlace2) using namespace Luau; TEST_SUITE_BEGIN("ProvisionalTests"); -// These tests check for behavior that differes from the final behavior we'd +// These tests check for behavior that differs from the final behavior we'd // like to have. They serve to document the current state of the typechecker. // When making future improvements, its very likely these tests will break and // will need to be replaced. @@ -42,7 +43,7 @@ TEST_CASE_FIXTURE(Fixture, "typeguard_inference_incomplete") end )"; - const std::string expected = R"( + const std::string old_expected = R"( function f(a:{fn:()->(free,free...)}): () if type(a) == 'boolean'then local a1:boolean=a @@ -51,7 +52,21 @@ TEST_CASE_FIXTURE(Fixture, "typeguard_inference_incomplete") end end )"; - CHECK_EQ(expected, decorateWithTypes(code)); + + const std::string expected = R"( + function f(a:{fn:()->(a,b...)}): () + if type(a) == 'boolean'then + local a1:boolean=a + elseif a.fn()then + local a2:{fn:()->(a,b...)}=a + end + end + )"; + + if (FFlag::LuauQuantifyInPlace2) + CHECK_EQ(expected, decorateWithTypes(code)); + else + CHECK_EQ(old_expected, decorateWithTypes(code)); } TEST_CASE_FIXTURE(Fixture, "xpcall_returns_what_f_returns") @@ -263,8 +278,8 @@ TEST_CASE_FIXTURE(Fixture, "lvalue_equals_another_lvalue_with_no_overlap") TEST_CASE_FIXTURE(Fixture, "bail_early_if_unification_is_too_complicated" * doctest::timeout(0.5)) { - ScopedFastInt sffi{"LuauTarjanChildLimit", 50}; - ScopedFastInt sffi2{"LuauTypeInferIterationLimit", 50}; + ScopedFastInt sffi{"LuauTarjanChildLimit", 1}; + ScopedFastInt sffi2{"LuauTypeInferIterationLimit", 1}; CheckResult result = check(R"LUA( local Result diff --git a/tests/TypeInfer.refinements.test.cpp b/tests/TypeInfer.refinements.test.cpp index 31739cdc..36dcaa95 100644 --- a/tests/TypeInfer.refinements.test.cpp +++ b/tests/TypeInfer.refinements.test.cpp @@ -8,6 +8,7 @@ LUAU_FASTFLAG(LuauWeakEqConstraint) LUAU_FASTFLAG(LuauOrPredicate) +LUAU_FASTFLAG(LuauQuantifyInPlace2) using namespace Luau; @@ -698,10 +699,16 @@ TEST_CASE_FIXTURE(RefinementClassFixture, "typeguard_cast_free_table_to_vector") CHECK_EQ("Vector3", toString(requireTypeAtPosition({5, 28}))); // type(vec) == "vector" - CHECK_EQ("Type '{- X: a, Y: b, Z: c -}' could not be converted into 'Instance'", toString(result.errors[0])); + if (FFlag::LuauQuantifyInPlace2) + CHECK_EQ("Type '{+ X: a, Y: b, Z: c +}' could not be converted into 'Instance'", toString(result.errors[0])); + else + CHECK_EQ("Type '{- X: a, Y: b, Z: c -}' could not be converted into 'Instance'", toString(result.errors[0])); CHECK_EQ("*unknown*", toString(requireTypeAtPosition({7, 28}))); // typeof(vec) == "Instance" - CHECK_EQ("{- X: a, Y: b, Z: c -}", toString(requireTypeAtPosition({9, 28}))); // type(vec) ~= "vector" and typeof(vec) ~= "Instance" + if (FFlag::LuauQuantifyInPlace2) + CHECK_EQ("{+ X: a, Y: b, Z: c +}", toString(requireTypeAtPosition({9, 28}))); // type(vec) ~= "vector" and typeof(vec) ~= "Instance" + else + CHECK_EQ("{- X: a, Y: b, Z: c -}", toString(requireTypeAtPosition({9, 28}))); // type(vec) ~= "vector" and typeof(vec) ~= "Instance" } TEST_CASE_FIXTURE(RefinementClassFixture, "typeguard_cast_instance_or_vector3_to_vector") diff --git a/tests/TypeInfer.tables.test.cpp b/tests/TypeInfer.tables.test.cpp index b7f0dc7b..f1451a81 100644 --- a/tests/TypeInfer.tables.test.cpp +++ b/tests/TypeInfer.tables.test.cpp @@ -617,7 +617,7 @@ TEST_CASE_FIXTURE(Fixture, "indexers_get_quantified_too") REQUIRE_EQ(indexer.indexType, typeChecker.numberType); - REQUIRE(nullptr != get(indexer.indexResultType)); + REQUIRE(nullptr != get(follow(indexer.indexResultType))); } TEST_CASE_FIXTURE(Fixture, "indexers_quantification_2") diff --git a/tests/TypeInfer.test.cpp b/tests/TypeInfer.test.cpp index b75878b7..45381757 100644 --- a/tests/TypeInfer.test.cpp +++ b/tests/TypeInfer.test.cpp @@ -180,7 +180,12 @@ TEST_CASE_FIXTURE(Fixture, "expr_statement") TEST_CASE_FIXTURE(Fixture, "generic_function") { - CheckResult result = check("function id(x) return x end local a = id(55) local b = id(nil)"); + CheckResult result = check(R"( + function id(x) return x end + local a = id(55) + local b = id(nil) + )"); + LUAU_REQUIRE_NO_ERRORS(result); CHECK_EQ(*typeChecker.numberType, *requireType("a")); @@ -406,7 +411,7 @@ TEST_CASE_FIXTURE(Fixture, "for_in_loop_error_on_factory_not_returning_the_right for p in primes2() do print(p) end -- mismatch in argument types, prime_iter takes {}, number, we are given {}, string - for p in primes3() do print(p) end -- no errror + for p in primes3() do print(p) end -- no error )"); LUAU_REQUIRE_ERROR_COUNT(2, result); @@ -1889,7 +1894,7 @@ TEST_CASE_FIXTURE(Fixture, "infer_higher_order_function") REQUIRE_EQ(2, argVec.size()); - const FunctionTypeVar* fType = get(argVec[0]); + const FunctionTypeVar* fType = get(follow(argVec[0])); REQUIRE(fType != nullptr); std::vector fArgs = flatten(fType->argTypes).first; @@ -1926,7 +1931,7 @@ TEST_CASE_FIXTURE(Fixture, "higher_order_function_2") REQUIRE_EQ(6, argVec.size()); - const FunctionTypeVar* fType = get(argVec[0]); + const FunctionTypeVar* fType = get(follow(argVec[0])); REQUIRE(fType != nullptr); } @@ -2549,7 +2554,7 @@ TEST_CASE_FIXTURE(Fixture, "toposort_doesnt_break_mutual_recursion") --!strict local x = nil function f() g() end - -- make sure print(x) doen't get toposorted here, breaking the mutual block + -- make sure print(x) doesn't get toposorted here, breaking the mutual block function g() x = f end print(x) )"); @@ -2987,7 +2992,7 @@ TEST_CASE_FIXTURE(Fixture, "correctly_scope_locals_while") CHECK_EQ(us->name, "a"); } -TEST_CASE_FIXTURE(Fixture, "ipairs_produces_integral_indeces") +TEST_CASE_FIXTURE(Fixture, "ipairs_produces_integral_indices") { CheckResult result = check(R"( local key @@ -3176,7 +3181,24 @@ TEST_CASE_FIXTURE(Fixture, "too_many_return_values") CountMismatch* acm = get(result.errors[0]); REQUIRE(acm); - CHECK(acm->context == CountMismatch::Result); + CHECK_EQ(acm->context, CountMismatch::Result); + CHECK_EQ(acm->expected, 1); + CHECK_EQ(acm->actual, 2); +} + +TEST_CASE_FIXTURE(Fixture, "ignored_return_values") +{ + CheckResult result = check(R"( + --!strict + + function f() + return 55, "" + end + + local a = f() + )"); + + LUAU_REQUIRE_ERROR_COUNT(0, result); } TEST_CASE_FIXTURE(Fixture, "function_does_not_return_enough_values") @@ -3194,6 +3216,8 @@ TEST_CASE_FIXTURE(Fixture, "function_does_not_return_enough_values") CountMismatch* acm = get(result.errors[0]); REQUIRE(acm); CHECK_EQ(acm->context, CountMismatch::Return); + CHECK_EQ(acm->expected, 2); + CHECK_EQ(acm->actual, 1); } TEST_CASE_FIXTURE(Fixture, "typecheck_unary_minus") @@ -3823,10 +3847,10 @@ local T: any T = {} T.__index = T function T.new(...) - local self = {} - setmetatable(self, T) - self:construct(...) - return self + local self = {} + setmetatable(self, T) + self:construct(...) + return self end function T:construct(index) end @@ -4049,11 +4073,11 @@ function n:Clone() end local m = {} function m.a(x) - x:Clone() + x:Clone() end function m.b() - m.a(n) + m.a(n) end return m @@ -4374,8 +4398,6 @@ TEST_CASE_FIXTURE(Fixture, "record_matching_overload") TEST_CASE_FIXTURE(Fixture, "infer_anonymous_function_arguments") { - ScopedFastFlag luauInferFunctionArgsFix("LuauInferFunctionArgsFix", true); - // Simple direct arg to arg propagation CheckResult result = check(R"( type Table = { x: number, y: number } @@ -4385,7 +4407,7 @@ f(function(a) return a.x + a.y end) LUAU_REQUIRE_NO_ERRORS(result); - // An optional funciton is accepted, but since we already provide a function, nil can be ignored + // An optional function is accepted, but since we already provide a function, nil can be ignored result = check(R"( type Table = { x: number, y: number } local function f(a: ((Table) -> number)?) if a then return a({x = 1, y = 2}) else return 0 end end @@ -4413,7 +4435,7 @@ f(function(a: number, b, c) return c and a + b or b - a end) LUAU_REQUIRE_NO_ERRORS(result); - // Anonymous function has a varyadic pack + // Anonymous function has a variadic pack result = check(R"( type Table = { x: number, y: number } local function f(a: (Table) -> number) return a({x = 1, y = 2}) end @@ -4432,7 +4454,7 @@ f(function(a, b, c, ...) return a + b end) LUAU_REQUIRE_ERRORS(result); CHECK_EQ("Type '(number, number, a) -> number' could not be converted into '(number, number) -> number'", toString(result.errors[0])); - // Infer from varyadic packs into elements + // Infer from variadic packs into elements result = check(R"( function f(a: (...number) -> number) return a(1, 2) end f(function(a, b) return a + b end) @@ -4440,7 +4462,7 @@ f(function(a, b) return a + b end) LUAU_REQUIRE_NO_ERRORS(result); - // Infer from varyadic packs into varyadic packs + // Infer from variadic packs into variadic packs result = check(R"( type Table = { x: number, y: number } function f(a: (...Table) -> number) return a({x = 1, y = 2}, {x = 3, y = 4}) end @@ -4662,7 +4684,6 @@ TEST_CASE_FIXTURE(Fixture, "checked_prop_too_early") { ScopedFastFlag sffs[] = { {"LuauSlightlyMoreFlexibleBinaryPredicates", true}, - {"LuauExtraNilRecovery", true}, }; CheckResult result = check(R"( @@ -4679,7 +4700,6 @@ TEST_CASE_FIXTURE(Fixture, "accidentally_checked_prop_in_opposite_branch") { ScopedFastFlag sffs[] = { {"LuauSlightlyMoreFlexibleBinaryPredicates", true}, - {"LuauExtraNilRecovery", true}, }; CheckResult result = check(R"( diff --git a/tests/TypeInfer.tryUnify.test.cpp b/tests/TypeInfer.tryUnify.test.cpp index 1f4b63ef..1192a8ac 100644 --- a/tests/TypeInfer.tryUnify.test.cpp +++ b/tests/TypeInfer.tryUnify.test.cpp @@ -8,6 +8,8 @@ #include "doctest.h" +LUAU_FASTFLAG(LuauQuantifyInPlace2); + using namespace Luau; struct TryUnifyFixture : Fixture @@ -15,7 +17,8 @@ struct TryUnifyFixture : Fixture TypeArena arena; ScopePtr globalScope{new Scope{arena.addTypePack({TypeId{}})}}; InternalErrorReporter iceHandler; - Unifier state{&arena, Mode::Strict, globalScope, Location{}, Variance::Covariant, &iceHandler}; + UnifierSharedState unifierState{&iceHandler}; + Unifier state{&arena, Mode::Strict, globalScope, Location{}, Variance::Covariant, unifierState}; }; TEST_SUITE_BEGIN("TryUnifyTests"); @@ -139,7 +142,10 @@ TEST_CASE_FIXTURE(TryUnifyFixture, "typepack_unification_should_trim_free_tails" )"); LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK_EQ("(number) -> (boolean)", toString(requireType("f"))); + if (FFlag::LuauQuantifyInPlace2) + CHECK_EQ("(number) -> boolean", toString(requireType("f"))); + else + CHECK_EQ("(number) -> (boolean)", toString(requireType("f"))); } TEST_CASE_FIXTURE(TryUnifyFixture, "variadic_type_pack_unification") diff --git a/tests/TypeInfer.typePacks.cpp b/tests/TypeInfer.typePacks.cpp index 3e1dedd4..8dab2605 100644 --- a/tests/TypeInfer.typePacks.cpp +++ b/tests/TypeInfer.typePacks.cpp @@ -98,10 +98,10 @@ TEST_CASE_FIXTURE(Fixture, "higher_order_function") std::vector applyArgs = flatten(applyType->argTypes).first; REQUIRE_EQ(3, applyArgs.size()); - const FunctionTypeVar* fType = get(applyArgs[0]); + const FunctionTypeVar* fType = get(follow(applyArgs[0])); REQUIRE(fType != nullptr); - const FunctionTypeVar* gType = get(applyArgs[1]); + const FunctionTypeVar* gType = get(follow(applyArgs[1])); REQUIRE(gType != nullptr); std::vector gArgs = flatten(gType->argTypes).first; @@ -285,7 +285,7 @@ TEST_CASE_FIXTURE(Fixture, "variadic_argument_tail") { CheckResult result = check(R"( local _ = function():((...any)->(...any),()->()) - return function() end, function() end + return function() end, function() end end for y in _() do end diff --git a/tests/TypeInfer.unionTypes.test.cpp b/tests/TypeInfer.unionTypes.test.cpp index 037144e2..34c25a9f 100644 --- a/tests/TypeInfer.unionTypes.test.cpp +++ b/tests/TypeInfer.unionTypes.test.cpp @@ -52,7 +52,7 @@ TEST_CASE_FIXTURE(Fixture, "allow_more_specific_assign") LUAU_REQUIRE_NO_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "disallow_less_specifc_assign") +TEST_CASE_FIXTURE(Fixture, "disallow_less_specific_assign") { CheckResult result = check(R"( local a:number = 10 @@ -63,7 +63,7 @@ TEST_CASE_FIXTURE(Fixture, "disallow_less_specifc_assign") LUAU_REQUIRE_ERROR_COUNT(1, result); } -TEST_CASE_FIXTURE(Fixture, "disallow_less_specifc_assign2") +TEST_CASE_FIXTURE(Fixture, "disallow_less_specific_assign2") { CheckResult result = check(R"( local a:number? = 10 @@ -181,8 +181,6 @@ TEST_CASE_FIXTURE(Fixture, "index_on_a_union_type_with_one_optional_property") TEST_CASE_FIXTURE(Fixture, "index_on_a_union_type_with_missing_property") { - ScopedFastFlag luauMissingUnionPropertyError("LuauMissingUnionPropertyError", true); - CheckResult result = check(R"( type A = {x: number} type B = {} @@ -242,8 +240,6 @@ TEST_CASE_FIXTURE(Fixture, "union_equality_comparisons") TEST_CASE_FIXTURE(Fixture, "optional_union_members") { - ScopedFastFlag luauExtraNilRecovery("LuauExtraNilRecovery", true); - CheckResult result = check(R"( local a = { a = { x = 1, y = 2 }, b = 3 } type A = typeof(a) @@ -259,8 +255,6 @@ local c = bf.a.y TEST_CASE_FIXTURE(Fixture, "optional_union_functions") { - ScopedFastFlag luauExtraNilRecovery("LuauExtraNilRecovery", true); - CheckResult result = check(R"( local a = {} function a.foo(x:number, y:number) return x + y end @@ -276,8 +270,6 @@ local c = b.foo(1, 2) TEST_CASE_FIXTURE(Fixture, "optional_union_methods") { - ScopedFastFlag luauExtraNilRecovery("LuauExtraNilRecovery", true); - CheckResult result = check(R"( local a = {} function a:foo(x:number, y:number) return x + y end @@ -310,8 +302,6 @@ return f() TEST_CASE_FIXTURE(Fixture, "optional_field_access_error") { - ScopedFastFlag luauExtraNilRecovery("LuauExtraNilRecovery", true); - CheckResult result = check(R"( type A = { x: number } local b: A? = { x = 2 } @@ -327,8 +317,6 @@ local d = b.y TEST_CASE_FIXTURE(Fixture, "optional_index_error") { - ScopedFastFlag luauExtraNilRecovery("LuauExtraNilRecovery", true); - CheckResult result = check(R"( type A = {number} local a: A? = {1, 2, 3} @@ -341,8 +329,6 @@ local b = a[1] TEST_CASE_FIXTURE(Fixture, "optional_call_error") { - ScopedFastFlag luauExtraNilRecovery("LuauExtraNilRecovery", true); - CheckResult result = check(R"( type A = (number) -> number local a: A? = function(a) return -a end @@ -355,8 +341,6 @@ local b = a(4) TEST_CASE_FIXTURE(Fixture, "optional_assignment_errors") { - ScopedFastFlag luauExtraNilRecovery("LuauExtraNilRecovery", true); - CheckResult result = check(R"( type A = { x: number } local a: A? = { x = 2 } @@ -378,8 +362,6 @@ a.x = 2 TEST_CASE_FIXTURE(Fixture, "optional_length_error") { - ScopedFastFlag luauExtraNilRecovery("LuauExtraNilRecovery", true); - CheckResult result = check(R"( type A = {number} local a: A? = {1, 2, 3} @@ -392,9 +374,6 @@ local b = #a TEST_CASE_FIXTURE(Fixture, "optional_missing_key_error_details") { - ScopedFastFlag luauExtraNilRecovery("LuauExtraNilRecovery", true); - ScopedFastFlag luauMissingUnionPropertyError("LuauMissingUnionPropertyError", true); - CheckResult result = check(R"( type A = { x: number, y: number } type B = { x: number, y: number } diff --git a/tests/TypeVar.test.cpp b/tests/TypeVar.test.cpp index a679e3fd..930c1a39 100644 --- a/tests/TypeVar.test.cpp +++ b/tests/TypeVar.test.cpp @@ -265,4 +265,64 @@ TEST_CASE_FIXTURE(Fixture, "substitution_skip_failure") CHECK_EQ("{ f: t1 } where t1 = () -> { f: () -> { f: ({ f: t1 }) -> (), signal: { f: (any) -> () } } }", toString(result)); } +TEST_CASE("tagging_tables") +{ + ScopedFastFlag sff{"LuauRefactorTagging", true}; + + TypeVar ttv{TableTypeVar{}}; + CHECK(!Luau::hasTag(&ttv, "foo")); + Luau::attachTag(&ttv, "foo"); + CHECK(Luau::hasTag(&ttv, "foo")); +} + +TEST_CASE("tagging_classes") +{ + ScopedFastFlag sff{"LuauRefactorTagging", true}; + + TypeVar base{ClassTypeVar{"Base", {}, std::nullopt, std::nullopt, {}, nullptr}}; + CHECK(!Luau::hasTag(&base, "foo")); + Luau::attachTag(&base, "foo"); + CHECK(Luau::hasTag(&base, "foo")); +} + +TEST_CASE("tagging_subclasses") +{ + ScopedFastFlag sff{"LuauRefactorTagging", true}; + + TypeVar base{ClassTypeVar{"Base", {}, std::nullopt, std::nullopt, {}, nullptr}}; + TypeVar derived{ClassTypeVar{"Derived", {}, &base, std::nullopt, {}, nullptr}}; + + CHECK(!Luau::hasTag(&base, "foo")); + CHECK(!Luau::hasTag(&derived, "foo")); + + Luau::attachTag(&base, "foo"); + CHECK(Luau::hasTag(&base, "foo")); + CHECK(Luau::hasTag(&derived, "foo")); + + Luau::attachTag(&derived, "bar"); + CHECK(!Luau::hasTag(&base, "bar")); + CHECK(Luau::hasTag(&derived, "bar")); +} + +TEST_CASE("tagging_functions") +{ + ScopedFastFlag sff{"LuauRefactorTagging", true}; + + TypePackVar empty{TypePack{}}; + TypeVar ftv{FunctionTypeVar{&empty, &empty}}; + CHECK(!Luau::hasTag(&ftv, "foo")); + Luau::attachTag(&ftv, "foo"); + CHECK(Luau::hasTag(&ftv, "foo")); +} + +TEST_CASE("tagging_props") +{ + ScopedFastFlag sff{"LuauRefactorTagging", true}; + + Property prop{}; + CHECK(!Luau::hasTag(prop, "foo")); + Luau::attachTag(prop, "foo"); + CHECK(Luau::hasTag(prop, "foo")); +} + TEST_SUITE_END(); diff --git a/tests/conformance/closure.lua b/tests/conformance/closure.lua index 79f8d9c2..aac42c56 100644 --- a/tests/conformance/closure.lua +++ b/tests/conformance/closure.lua @@ -319,7 +319,7 @@ end assert(a == 5^4) --- access to locals of collected corroutines +-- access to locals of collected coroutines local C = {}; setmetatable(C, {__mode = "kv"}) local x = coroutine.wrap (function () local a = 10 diff --git a/tests/conformance/coroutine.lua b/tests/conformance/coroutine.lua index 73c3833d..75329642 100644 --- a/tests/conformance/coroutine.lua +++ b/tests/conformance/coroutine.lua @@ -185,7 +185,7 @@ end assert(a == 5^4) --- access to locals of collected corroutines +-- access to locals of collected coroutines local C = {}; setmetatable(C, {__mode = "kv"}) local x = coroutine.wrap (function () local a = 10 diff --git a/tests/conformance/gc.lua b/tests/conformance/gc.lua index fd4b4de1..4263dfda 100644 --- a/tests/conformance/gc.lua +++ b/tests/conformance/gc.lua @@ -277,7 +277,7 @@ do assert(getmetatable(o) == tt) -- create new objects during GC local a = 'xuxu'..(10+3)..'joao', {} - ___Glob = o -- ressurect object! + ___Glob = o -- resurrect object! newproxy(o) -- creates a new one with same metatable print(">>> closing state " .. "<<<\n") end diff --git a/tests/conformance/locals.lua b/tests/conformance/locals.lua index cbe5f92d..2d8d004b 100644 --- a/tests/conformance/locals.lua +++ b/tests/conformance/locals.lua @@ -117,7 +117,7 @@ if rawget(_G, "querytab") then local t = querytab(a) for k,_ in pairs(a) do a[k] = nil end - collectgarbage() -- restore GC and collect dead fiels in `a' + collectgarbage() -- restore GC and collect dead fields in `a' for i=0,t-1 do local k = querytab(a, i) assert(k == nil or type(k) == 'number' or k == 'alo') diff --git a/tests/conformance/math.lua b/tests/conformance/math.lua index 5e8b9398..d5bca44f 100644 --- a/tests/conformance/math.lua +++ b/tests/conformance/math.lua @@ -172,7 +172,7 @@ end a = nil --- testing implicit convertions +-- testing implicit conversions local a,b = '10', '20' assert(a*b == 200 and a+b == 30 and a-b == -10 and a/b == 0.5 and -b == -20) diff --git a/tests/conformance/pm.lua b/tests/conformance/pm.lua index 9a113964..263759ac 100644 --- a/tests/conformance/pm.lua +++ b/tests/conformance/pm.lua @@ -21,9 +21,9 @@ a,b = string.find('alo', '') assert(a == 1 and b == 0) a,b = string.find('a\0o a\0o a\0o', 'a', 1) -- first position assert(a == 1 and b == 1) -a,b = string.find('a\0o a\0o a\0o', 'a\0o', 2) -- starts in the midle +a,b = string.find('a\0o a\0o a\0o', 'a\0o', 2) -- starts in the middle assert(a == 5 and b == 7) -a,b = string.find('a\0o a\0o a\0o', 'a\0o', 9) -- starts in the midle +a,b = string.find('a\0o a\0o a\0o', 'a\0o', 9) -- starts in the middle assert(a == 9 and b == 11) a,b = string.find('a\0a\0a\0a\0\0ab', '\0ab', 2); -- finds at the end assert(a == 9 and b == 11); diff --git a/tests/conformance/tmerror.lua b/tests/conformance/tmerror.lua new file mode 100644 index 00000000..1ad4dd16 --- /dev/null +++ b/tests/conformance/tmerror.lua @@ -0,0 +1,15 @@ +-- This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +-- This file is based on Lua 5.x tests -- https://github.com/lua/lua/tree/master/testes + +-- Generate an error (i.e. throw an exception) inside a tag method which is indirectly +-- called via pcall. +-- This test is meant to detect a regression in handling errors inside a tag method + +local testtable = {} +setmetatable(testtable, { __index = function() error("Error") end }) + +pcall(function() + testtable.missingmethod() +end) + +return('OK') diff --git a/tools/gdb-printers.py b/tools/gdb-printers.py index c711c5e2..017b9f95 100644 --- a/tools/gdb-printers.py +++ b/tools/gdb-printers.py @@ -11,9 +11,9 @@ class VariantPrinter: return type.name + " [" + str(value) + "]" def match_printer(val): - type = val.type.strip_typedefs() - if type.name and type.name.startswith('Luau::Variant<'): - return VariantPrinter(val) - return None + type = val.type.strip_typedefs() + if type.name and type.name.startswith('Luau::Variant<'): + return VariantPrinter(val) + return None gdb.pretty_printers.append(match_printer) From c0b95b89619ee00f9de0e4bf8692fe35da8af52a Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Thu, 4 Nov 2021 23:24:39 -0700 Subject: [PATCH 023/399] Update profile.md Too much future. --- docs/_pages/profile.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/_pages/profile.md b/docs/_pages/profile.md index edb30287..72d11546 100644 --- a/docs/_pages/profile.md +++ b/docs/_pages/profile.md @@ -56,4 +56,4 @@ This profiler doesn't track leaf C functions and instead attributes the time spe slow, consider not just the work it does immediately but also the library functions it calls. This profiler tracks time consumed by Luau thread stacks; when a thread calls another thread via `coroutine.resume`, the time spent is not attributed to the parent thread that's -waiting for resume results. This limitation will be removed in the future in the future. +waiting for resume results. This limitation will be removed in the future. From 279855df91d522bb65739631e3ee74e94192b817 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Fri, 5 Nov 2021 08:47:21 -0700 Subject: [PATCH 024/399] Sync to upstream/release/503 (#135) - A series of major optimizations to type checking performance on complex programs/types (up to two orders of magnitude speedup for programs involving huge tagged unions) - Fix a few issues encountered by UBSAN (and maybe fix s390x builds) - Fix gcc-11 test builds - Fix a rare corner case where luau_load wouldn't wake inactive threads which could result in a use-after-free due to GC - Fix CLI crash when error object that's not a string escapes to top level - Fix Makefile suffixes on macOS Co-authored-by: Rodactor --- Analysis/include/Luau/BuiltinDefinitions.h | 1 - Analysis/include/Luau/Quantify.h | 14 + Analysis/include/Luau/ToString.h | 2 + Analysis/include/Luau/TopoSortStatements.h | 1 + Analysis/include/Luau/TxnLog.h | 30 +- Analysis/include/Luau/TypeInfer.h | 7 +- Analysis/include/Luau/TypeVar.h | 7 + Analysis/include/Luau/Unifier.h | 20 +- Analysis/include/Luau/UnifierSharedState.h | 44 ++ Analysis/include/Luau/VisitTypeVar.h | 59 +- Analysis/src/Autocomplete.cpp | 3 +- Analysis/src/BuiltinDefinitions.cpp | 12 - Analysis/src/Frontend.cpp | 11 +- Analysis/src/Module.cpp | 18 +- Analysis/src/Quantify.cpp | 90 +++ Analysis/src/RequireTracer.cpp | 4 +- Analysis/src/ToString.cpp | 25 +- Analysis/src/TopoSortStatements.cpp | 25 + Analysis/src/TxnLog.cpp | 37 +- Analysis/src/TypeAttach.cpp | 93 ++- Analysis/src/TypeInfer.cpp | 197 ++--- Analysis/src/TypeVar.cpp | 86 ++- Analysis/src/Unifier.cpp | 341 ++++++++- Ast/include/Luau/TimeTrace.h | 16 +- Ast/src/TimeTrace.cpp | 5 +- Makefile | 1 + Sources.cmake | 3 + VM/src/lapi.cpp | 4 +- VM/src/lcorolib.cpp | 70 +- VM/src/ldo.cpp | 12 +- VM/src/lgc.cpp | 325 ++++---- VM/src/lgc.h | 6 +- VM/src/lmem.cpp | 2 +- VM/src/lstring.cpp | 2 +- VM/src/lvmload.cpp | 4 +- bench/tests/chess.lua | 849 +++++++++++++++++++++ tests/Autocomplete.test.cpp | 24 +- tests/Compiler.test.cpp | 84 +- tests/Conformance.test.cpp | 13 + tests/IostreamOptional.h | 7 +- tests/Linter.test.cpp | 14 +- tests/TypeInfer.aliases.test.cpp | 50 ++ tests/TypeInfer.classes.test.cpp | 2 - tests/TypeInfer.generics.test.cpp | 21 + tests/TypeInfer.provisional.test.cpp | 23 +- tests/TypeInfer.refinements.test.cpp | 11 +- tests/TypeInfer.tables.test.cpp | 2 +- tests/TypeInfer.test.cpp | 27 +- tests/TypeInfer.tryUnify.test.cpp | 10 +- tests/TypeInfer.typePacks.cpp | 6 +- tests/TypeInfer.unionTypes.test.cpp | 21 - tests/TypeVar.test.cpp | 60 ++ tests/conformance/tmerror.lua | 15 + tools/gdb-printers.py | 8 +- 54 files changed, 2201 insertions(+), 623 deletions(-) create mode 100644 Analysis/include/Luau/Quantify.h create mode 100644 Analysis/include/Luau/UnifierSharedState.h create mode 100644 Analysis/src/Quantify.cpp create mode 100644 bench/tests/chess.lua create mode 100644 tests/conformance/tmerror.lua diff --git a/Analysis/include/Luau/BuiltinDefinitions.h b/Analysis/include/Luau/BuiltinDefinitions.h index 57a1907a..07d897b2 100644 --- a/Analysis/include/Luau/BuiltinDefinitions.h +++ b/Analysis/include/Luau/BuiltinDefinitions.h @@ -34,7 +34,6 @@ TypeId makeFunction( // Polymorphic std::initializer_list paramTypes, std::initializer_list paramNames, std::initializer_list retTypes); void attachMagicFunction(TypeId ty, MagicFunction fn); -void attachFunctionTag(TypeId ty, std::string constraint); Property makeProperty(TypeId ty, std::optional documentationSymbol = std::nullopt); void assignPropDocumentationSymbols(TableTypeVar::Props& props, const std::string& baseName); diff --git a/Analysis/include/Luau/Quantify.h b/Analysis/include/Luau/Quantify.h new file mode 100644 index 00000000..f46df146 --- /dev/null +++ b/Analysis/include/Luau/Quantify.h @@ -0,0 +1,14 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +#pragma once + +#include "Luau/TypeVar.h" + +namespace Luau +{ + +struct Module; +using ModulePtr = std::shared_ptr; + +void quantify(ModulePtr module, TypeId ty, TypeLevel level); + +} // namespace Luau diff --git a/Analysis/include/Luau/ToString.h b/Analysis/include/Luau/ToString.h index 0897ec85..e5683fc4 100644 --- a/Analysis/include/Luau/ToString.h +++ b/Analysis/include/Luau/ToString.h @@ -69,4 +69,6 @@ std::string toString(const TypePackVar& tp, const ToStringOptions& opts = {}); void dump(TypeId ty); void dump(TypePackId ty); +std::string generateName(size_t n); + } // namespace Luau diff --git a/Analysis/include/Luau/TopoSortStatements.h b/Analysis/include/Luau/TopoSortStatements.h index 751694f0..4a4acfa3 100644 --- a/Analysis/include/Luau/TopoSortStatements.h +++ b/Analysis/include/Luau/TopoSortStatements.h @@ -12,6 +12,7 @@ struct AstArray; class AstStat; bool containsFunctionCall(const AstStat& stat); +bool containsFunctionCallOrReturn(const AstStat& stat); bool isFunction(const AstStat& stat); void toposort(std::vector& stats); diff --git a/Analysis/include/Luau/TxnLog.h b/Analysis/include/Luau/TxnLog.h index 055441ce..322abd19 100644 --- a/Analysis/include/Luau/TxnLog.h +++ b/Analysis/include/Luau/TxnLog.h @@ -3,19 +3,37 @@ #include "Luau/TypeVar.h" +LUAU_FASTFLAG(LuauShareTxnSeen); + namespace Luau { // Log of where what TypeIds we are rebinding and what they used to be struct TxnLog { - TxnLog() = default; - - explicit TxnLog(const std::vector>& seen) - : seen(seen) + TxnLog() + : originalSeenSize(0) + , ownedSeen() + , sharedSeen(&ownedSeen) { } + explicit TxnLog(std::vector>* sharedSeen) + : originalSeenSize(sharedSeen->size()) + , ownedSeen() + , sharedSeen(sharedSeen) + { + } + + explicit TxnLog(const std::vector>& ownedSeen) + : originalSeenSize(ownedSeen.size()) + , ownedSeen(ownedSeen) + , sharedSeen(nullptr) + { + // This is deprecated! + LUAU_ASSERT(!FFlag::LuauShareTxnSeen); + } + TxnLog(const TxnLog&) = delete; TxnLog& operator=(const TxnLog&) = delete; @@ -38,9 +56,11 @@ private: std::vector> typeVarChanges; std::vector> typePackChanges; std::vector>> tableChanges; + size_t originalSeenSize; public: - std::vector> seen; // used to avoid infinite recursion when types are cyclic + std::vector> ownedSeen; // used to avoid infinite recursion when types are cyclic + std::vector>* sharedSeen; // shared with all the descendent logs }; } // namespace Luau diff --git a/Analysis/include/Luau/TypeInfer.h b/Analysis/include/Luau/TypeInfer.h index d701eb24..9d62fef0 100644 --- a/Analysis/include/Luau/TypeInfer.h +++ b/Analysis/include/Luau/TypeInfer.h @@ -11,6 +11,7 @@ #include "Luau/TypePack.h" #include "Luau/TypeVar.h" #include "Luau/Unifier.h" +#include "Luau/UnifierSharedState.h" #include #include @@ -121,7 +122,7 @@ struct TypeChecker void check(const ScopePtr& scope, const AstStatForIn& forin); void check(const ScopePtr& scope, TypeId ty, const ScopePtr& funScope, const AstStatFunction& function); void check(const ScopePtr& scope, TypeId ty, const ScopePtr& funScope, const AstStatLocalFunction& function); - void check(const ScopePtr& scope, const AstStatTypeAlias& typealias, bool forwardDeclare = false); + void check(const ScopePtr& scope, const AstStatTypeAlias& typealias, int subLevel = 0, bool forwardDeclare = false); void check(const ScopePtr& scope, const AstStatDeclareClass& declaredClass); void check(const ScopePtr& scope, const AstStatDeclareFunction& declaredFunction); @@ -336,7 +337,7 @@ private: // Note: `scope` must be a fresh scope. std::pair, std::vector> createGenericTypes( - const ScopePtr& scope, const AstNode& node, const AstArray& genericNames, const AstArray& genericPackNames); + const ScopePtr& scope, std::optional levelOpt, const AstNode& node, const AstArray& genericNames, const AstArray& genericPackNames); public: ErrorVec resolve(const PredicateVec& predicates, const ScopePtr& scope, bool sense); @@ -383,6 +384,8 @@ public: std::function prepareModuleScope; InternalErrorReporter* iceHandler; + UnifierSharedState unifierState; + public: const TypeId nilType; const TypeId numberType; diff --git a/Analysis/include/Luau/TypeVar.h b/Analysis/include/Luau/TypeVar.h index e04aa2ca..9611e881 100644 --- a/Analysis/include/Luau/TypeVar.h +++ b/Analysis/include/Luau/TypeVar.h @@ -540,4 +540,11 @@ UnionTypeVarIterator end(const UnionTypeVar* utv); using TypeIdPredicate = std::function(TypeId)>; std::vector filterMap(TypeId type, TypeIdPredicate predicate); +void attachTag(TypeId ty, const std::string& tagName); +void attachTag(Property& prop, const std::string& tagName); + +bool hasTag(TypeId ty, const std::string& tagName); +bool hasTag(const Property& prop, const std::string& tagName); +bool hasTag(const Tags& tags, const std::string& tagName); // Do not use in new work. + } // namespace Luau diff --git a/Analysis/include/Luau/Unifier.h b/Analysis/include/Luau/Unifier.h index 522914b2..56632e33 100644 --- a/Analysis/include/Luau/Unifier.h +++ b/Analysis/include/Luau/Unifier.h @@ -6,6 +6,7 @@ #include "Luau/TxnLog.h" #include "Luau/TypeInfer.h" #include "Luau/Module.h" // FIXME: For TypeArena. It merits breaking out into its own header. +#include "Luau/UnifierSharedState.h" #include @@ -41,11 +42,14 @@ struct Unifier std::shared_ptr counters_DEPRECATED; - InternalErrorReporter* iceHandler; + UnifierSharedState& sharedState; - Unifier(TypeArena* types, Mode mode, ScopePtr globalScope, const Location& location, Variance variance, InternalErrorReporter* iceHandler); - Unifier(TypeArena* types, Mode mode, ScopePtr globalScope, const std::vector>& seen, const Location& location, - Variance variance, InternalErrorReporter* iceHandler, const std::shared_ptr& counters_DEPRECATED = nullptr, + Unifier(TypeArena* types, Mode mode, ScopePtr globalScope, const Location& location, Variance variance, UnifierSharedState& sharedState); + Unifier(TypeArena* types, Mode mode, ScopePtr globalScope, const std::vector>& ownedSeen, const Location& location, + Variance variance, UnifierSharedState& sharedState, const std::shared_ptr& counters_DEPRECATED = nullptr, + UnifierCounters* counters = nullptr); + Unifier(TypeArena* types, Mode mode, ScopePtr globalScope, std::vector>* sharedSeen, const Location& location, + Variance variance, UnifierSharedState& sharedState, const std::shared_ptr& counters_DEPRECATED = nullptr, UnifierCounters* counters = nullptr); // Test whether the two type vars unify. Never commits the result. @@ -69,7 +73,8 @@ private: void tryUnifyWithMetatable(TypeId metatable, TypeId other, bool reversed); void tryUnifyWithClass(TypeId superTy, TypeId subTy, bool reversed); void tryUnify(const TableIndexer& superIndexer, const TableIndexer& subIndexer); - TypeId deeplyOptional(TypeId ty, std::unordered_map seen = {}); + TypeId deeplyOptional(TypeId ty, std::unordered_map seen = {}); + void cacheResult(TypeId superTy, TypeId subTy); public: void tryUnify(TypePackId superTy, TypePackId subTy, bool isFunctionCall = false); @@ -101,8 +106,9 @@ private: [[noreturn]] void ice(const std::string& message, const Location& location); [[noreturn]] void ice(const std::string& message); - DenseHashSet tempSeenTy{nullptr}; - DenseHashSet tempSeenTp{nullptr}; + // Remove with FFlagLuauCacheUnifyTableResults + DenseHashSet tempSeenTy_DEPRECATED{nullptr}; + DenseHashSet tempSeenTp_DEPRECATED{nullptr}; }; } // namespace Luau diff --git a/Analysis/include/Luau/UnifierSharedState.h b/Analysis/include/Luau/UnifierSharedState.h new file mode 100644 index 00000000..f252a004 --- /dev/null +++ b/Analysis/include/Luau/UnifierSharedState.h @@ -0,0 +1,44 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +#pragma once + +#include "Luau/DenseHash.h" +#include "Luau/TypeVar.h" +#include "Luau/TypePack.h" + +#include + +namespace Luau +{ +struct InternalErrorReporter; + +struct TypeIdPairHash +{ + size_t hashOne(Luau::TypeId key) const + { + return (uintptr_t(key) >> 4) ^ (uintptr_t(key) >> 9); + } + + size_t operator()(const std::pair& x) const + { + return hashOne(x.first) ^ (hashOne(x.second) << 1); + } +}; + +struct UnifierSharedState +{ + UnifierSharedState(InternalErrorReporter* iceHandler) + : iceHandler(iceHandler) + { + } + + InternalErrorReporter* iceHandler; + + DenseHashSet seenAny{nullptr}; + DenseHashMap skipCacheForType{nullptr}; + DenseHashSet, TypeIdPairHash> cachedUnify{{nullptr, nullptr}}; + + DenseHashSet tempSeenTy{nullptr}; + DenseHashSet tempSeenTp{nullptr}; +}; + +} // namespace Luau diff --git a/Analysis/include/Luau/VisitTypeVar.h b/Analysis/include/Luau/VisitTypeVar.h index df0bd420..a866655c 100644 --- a/Analysis/include/Luau/VisitTypeVar.h +++ b/Analysis/include/Luau/VisitTypeVar.h @@ -1,9 +1,12 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #pragma once +#include "Luau/DenseHash.h" #include "Luau/TypeVar.h" #include "Luau/TypePack.h" +LUAU_FASTFLAG(LuauCacheUnifyTableResults) + namespace Luau { @@ -32,17 +35,33 @@ inline bool hasSeen(std::unordered_set& seen, const void* tv) return !seen.insert(ttv).second; } +inline bool hasSeen(DenseHashSet& seen, const void* tv) +{ + void* ttv = const_cast(tv); + + if (seen.contains(ttv)) + return true; + + seen.insert(ttv); + return false; +} + inline void unsee(std::unordered_set& seen, const void* tv) { void* ttv = const_cast(tv); seen.erase(ttv); } -template -void visit(TypePackId tp, F& f, std::unordered_set& seen); +inline void unsee(DenseHashSet& seen, const void* tv) +{ + // When DenseHashSet is used for 'visitOnce', where don't forget visited elements +} -template -void visit(TypeId ty, F& f, std::unordered_set& seen) +template +void visit(TypePackId tp, F& f, Set& seen); + +template +void visit(TypeId ty, F& f, Set& seen) { if (visit_detail::hasSeen(seen, ty)) { @@ -79,15 +98,23 @@ void visit(TypeId ty, F& f, std::unordered_set& seen) else if (auto ttv = get(ty)) { + // Some visitors want to see bound tables, that's why we visit the original type if (apply(ty, *ttv, seen, f)) { - for (auto& [_name, prop] : ttv->props) - visit(prop.type, f, seen); - - if (ttv->indexer) + if (FFlag::LuauCacheUnifyTableResults && ttv->boundTo) { - visit(ttv->indexer->indexType, f, seen); - visit(ttv->indexer->indexResultType, f, seen); + visit(*ttv->boundTo, f, seen); + } + else + { + for (auto& [_name, prop] : ttv->props) + visit(prop.type, f, seen); + + if (ttv->indexer) + { + visit(ttv->indexer->indexType, f, seen); + visit(ttv->indexer->indexResultType, f, seen); + } } } } @@ -140,8 +167,8 @@ void visit(TypeId ty, F& f, std::unordered_set& seen) visit_detail::unsee(seen, ty); } -template -void visit(TypePackId tp, F& f, std::unordered_set& seen) +template +void visit(TypePackId tp, F& f, Set& seen) { if (visit_detail::hasSeen(seen, tp)) { @@ -182,6 +209,7 @@ void visit(TypePackId tp, F& f, std::unordered_set& seen) visit_detail::unsee(seen, tp); } + } // namespace visit_detail template @@ -197,4 +225,11 @@ void visitTypeVar(TID ty, F& f) visit_detail::visit(ty, f, seen); } +template +void visitTypeVarOnce(TID ty, F& f, DenseHashSet& seen) +{ + seen.clear(); + visit_detail::visit(ty, f, seen); +} + } // namespace Luau diff --git a/Analysis/src/Autocomplete.cpp b/Analysis/src/Autocomplete.cpp index b31d85aa..3c43c808 100644 --- a/Analysis/src/Autocomplete.cpp +++ b/Analysis/src/Autocomplete.cpp @@ -196,7 +196,8 @@ static TypeCorrectKind checkTypeCorrectKind(const Module& module, TypeArena* typ auto canUnify = [&typeArena, &module](TypeId expectedType, TypeId actualType) { InternalErrorReporter iceReporter; - Unifier unifier(typeArena, Mode::Strict, module.getModuleScope(), Location(), Variance::Covariant, &iceReporter); + UnifierSharedState unifierState(&iceReporter); + Unifier unifier(typeArena, Mode::Strict, module.getModuleScope(), Location(), Variance::Covariant, unifierState); unifier.tryUnify(expectedType, actualType); diff --git a/Analysis/src/BuiltinDefinitions.cpp b/Analysis/src/BuiltinDefinitions.cpp index 3b0c2163..f6f2363c 100644 --- a/Analysis/src/BuiltinDefinitions.cpp +++ b/Analysis/src/BuiltinDefinitions.cpp @@ -106,18 +106,6 @@ void attachMagicFunction(TypeId ty, MagicFunction fn) LUAU_ASSERT(!"Got a non functional type"); } -void attachFunctionTag(TypeId ty, std::string tag) -{ - if (auto ftv = getMutable(ty)) - { - ftv->tags.emplace_back(std::move(tag)); - } - else - { - LUAU_ASSERT(!"Got a non functional type"); - } -} - Property makeProperty(TypeId ty, std::optional documentationSymbol) { return { diff --git a/Analysis/src/Frontend.cpp b/Analysis/src/Frontend.cpp index b2529840..5e7af50c 100644 --- a/Analysis/src/Frontend.cpp +++ b/Analysis/src/Frontend.cpp @@ -23,6 +23,7 @@ LUAU_FASTFLAGVARIABLE(LuauResolveModuleNameWithoutACurrentModule, false) LUAU_FASTFLAG(LuauTraceRequireLookupChild) LUAU_FASTFLAGVARIABLE(LuauPersistDefinitionFileTypes, false) LUAU_FASTFLAG(LuauNewRequireTrace) +LUAU_FASTFLAGVARIABLE(LuauClearScopes, false) namespace Luau { @@ -248,7 +249,7 @@ struct RequireCycle // Note that this is O(V^2) for a fully connected graph and produces O(V) paths of length O(V) // However, when the graph is acyclic, this is O(V), as well as when only the first cycle is needed (stopAtFirst=true) std::vector getRequireCycles( - const std::unordered_map& sourceNodes, const SourceNode* start, bool stopAtFirst = false) + const FileResolver* resolver, const std::unordered_map& sourceNodes, const SourceNode* start, bool stopAtFirst = false) { std::vector result; @@ -282,9 +283,9 @@ std::vector getRequireCycles( if (top == start) { for (const SourceNode* node : path) - cycle.push_back(node->name); + cycle.push_back(resolver->getHumanReadableModuleName(node->name)); - cycle.push_back(top->name); + cycle.push_back(resolver->getHumanReadableModuleName(top->name)); break; } } @@ -404,7 +405,7 @@ CheckResult Frontend::check(const ModuleName& name) // however, for now getRequireCycles isn't expensive in practice on the cases we care about, and long term // all correct programs must be acyclic so this code triggers rarely if (cycleDetected) - requireCycles = getRequireCycles(sourceNodes, &sourceNode, mode == Mode::NoCheck); + requireCycles = getRequireCycles(fileResolver, sourceNodes, &sourceNode, mode == Mode::NoCheck); // This is used by the type checker to replace the resulting type of cyclic modules with any sourceModule.cyclic = !requireCycles.empty(); @@ -458,6 +459,8 @@ CheckResult Frontend::check(const ModuleName& name) module->astTypes.clear(); module->astExpectedTypes.clear(); module->astOriginalCallTypes.clear(); + if (FFlag::LuauClearScopes) + module->scopes.resize(1); } if (mode != Mode::NoCheck) diff --git a/Analysis/src/Module.cpp b/Analysis/src/Module.cpp index df6be767..2fd95896 100644 --- a/Analysis/src/Module.cpp +++ b/Analysis/src/Module.cpp @@ -15,6 +15,7 @@ LUAU_FASTFLAGVARIABLE(DebugLuauTrackOwningArena, false) LUAU_FASTFLAG(LuauSecondTypecheckKnowsTheDataModel) LUAU_FASTFLAG(LuauCaptureBrokenCommentSpans) LUAU_FASTFLAG(LuauTypeAliasPacks) +LUAU_FASTFLAGVARIABLE(LuauCloneBoundTables, false) namespace Luau { @@ -299,6 +300,14 @@ void TypeCloner::operator()(const FunctionTypeVar& t) void TypeCloner::operator()(const TableTypeVar& t) { + // If table is now bound to another one, we ignore the content of the original + if (FFlag::LuauCloneBoundTables && t.boundTo) + { + TypeId boundTo = clone(*t.boundTo, dest, seenTypes, seenTypePacks, encounteredFreeType); + seenTypes[typeId] = boundTo; + return; + } + TypeId result = dest.addType(TableTypeVar{}); TableTypeVar* ttv = getMutable(result); LUAU_ASSERT(ttv != nullptr); @@ -321,8 +330,11 @@ void TypeCloner::operator()(const TableTypeVar& t) ttv->indexer = TableIndexer{clone(t.indexer->indexType, dest, seenTypes, seenTypePacks, encounteredFreeType), clone(t.indexer->indexResultType, dest, seenTypes, seenTypePacks, encounteredFreeType)}; - if (t.boundTo) - ttv->boundTo = clone(*t.boundTo, dest, seenTypes, seenTypePacks, encounteredFreeType); + if (!FFlag::LuauCloneBoundTables) + { + if (t.boundTo) + ttv->boundTo = clone(*t.boundTo, dest, seenTypes, seenTypePacks, encounteredFreeType); + } for (TypeId& arg : ttv->instantiatedTypeParams) arg = clone(arg, dest, seenTypes, seenTypePacks, encounteredFreeType); @@ -335,7 +347,7 @@ void TypeCloner::operator()(const TableTypeVar& t) if (ttv->state == TableState::Free) { - if (!t.boundTo) + if (FFlag::LuauCloneBoundTables || !t.boundTo) { if (encounteredFreeType) *encounteredFreeType = true; diff --git a/Analysis/src/Quantify.cpp b/Analysis/src/Quantify.cpp new file mode 100644 index 00000000..bf6d81aa --- /dev/null +++ b/Analysis/src/Quantify.cpp @@ -0,0 +1,90 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details + +#include "Luau/Quantify.h" + +#include "Luau/VisitTypeVar.h" + +namespace Luau +{ + +struct Quantifier +{ + ModulePtr module; + TypeLevel level; + std::vector generics; + std::vector genericPacks; + + Quantifier(ModulePtr module, TypeLevel level) + : module(module) + , level(level) + { + } + + void cycle(TypeId) {} + void cycle(TypePackId) {} + + bool operator()(TypeId ty, const FreeTypeVar& ftv) + { + if (!level.subsumes(ftv.level)) + return false; + + *asMutable(ty) = GenericTypeVar{level}; + generics.push_back(ty); + + return false; + } + + template + bool operator()(TypeId ty, const T& t) + { + return true; + } + + template + bool operator()(TypePackId, const T&) + { + return true; + } + + bool operator()(TypeId ty, const TableTypeVar&) + { + TableTypeVar& ttv = *getMutable(ty); + + if (ttv.state == TableState::Sealed || ttv.state == TableState::Generic) + return false; + if (!level.subsumes(ttv.level)) + return false; + + if (ttv.state == TableState::Free) + ttv.state = TableState::Generic; + else if (ttv.state == TableState::Unsealed) + ttv.state = TableState::Sealed; + + ttv.level = level; + + return true; + } + + bool operator()(TypePackId tp, const FreeTypePack& ftp) + { + if (!level.subsumes(ftp.level)) + return false; + + *asMutable(tp) = GenericTypePack{level}; + genericPacks.push_back(tp); + return true; + } +}; + +void quantify(ModulePtr module, TypeId ty, TypeLevel level) +{ + Quantifier q{std::move(module), level}; + visitTypeVar(ty, q); + + FunctionTypeVar* ftv = getMutable(ty); + LUAU_ASSERT(ftv); + ftv->generics = q.generics; + ftv->genericPacks = q.genericPacks; +} + +} // namespace Luau diff --git a/Analysis/src/RequireTracer.cpp b/Analysis/src/RequireTracer.cpp index 4634effd..95910b56 100644 --- a/Analysis/src/RequireTracer.cpp +++ b/Analysis/src/RequireTracer.cpp @@ -182,7 +182,7 @@ struct RequireTracerOld : AstVisitor struct RequireTracer : AstVisitor { - RequireTracer(RequireTraceResult& result, FileResolver * fileResolver, const ModuleName& currentModuleName) + RequireTracer(RequireTraceResult& result, FileResolver* fileResolver, const ModuleName& currentModuleName) : result(result) , fileResolver(fileResolver) , currentModuleName(currentModuleName) @@ -260,7 +260,7 @@ struct RequireTracer : AstVisitor // seed worklist with require arguments work.reserve(requires.size()); - for (AstExprCall* require: requires) + for (AstExprCall* require : requires) work.push_back(require->args.data[0]); // push all dependent expressions to the work stack; note that the vector is modified during traversal diff --git a/Analysis/src/ToString.cpp b/Analysis/src/ToString.cpp index 5651af7e..cd8180db 100644 --- a/Analysis/src/ToString.cpp +++ b/Analysis/src/ToString.cpp @@ -10,7 +10,6 @@ #include #include -LUAU_FASTFLAG(LuauExtraNilRecovery) LUAU_FASTFLAG(LuauOccursCheckOkWithRecursiveFunctions) LUAU_FASTFLAGVARIABLE(LuauInstantiatedTypeParamRecursion, false) LUAU_FASTFLAG(LuauTypeAliasPacks) @@ -159,15 +158,6 @@ struct StringifierState seen.erase(iter); } - static std::string generateName(size_t i) - { - std::string n; - n = char('a' + i % 26); - if (i >= 26) - n += std::to_string(i / 26); - return n; - } - std::string getName(TypeId ty) { const size_t s = result.nameMap.typeVars.size(); @@ -584,8 +574,7 @@ struct TypeVarStringifier std::vector results = {}; for (auto el : &uv) { - if (FFlag::LuauExtraNilRecovery || FFlag::LuauAddMissingFollow) - el = follow(el); + el = follow(el); if (isNil(el)) { @@ -649,8 +638,7 @@ struct TypeVarStringifier std::vector results = {}; for (auto el : uv.parts) { - if (FFlag::LuauExtraNilRecovery || FFlag::LuauAddMissingFollow) - el = follow(el); + el = follow(el); std::string saved = std::move(state.result.name); @@ -1204,4 +1192,13 @@ void dump(TypePackId ty) printf("%s\n", toString(ty, opts).c_str()); } +std::string generateName(size_t i) +{ + std::string n; + n = char('a' + i % 26); + if (i >= 26) + n += std::to_string(i / 26); + return n; +} + } // namespace Luau diff --git a/Analysis/src/TopoSortStatements.cpp b/Analysis/src/TopoSortStatements.cpp index 2d356384..dba694be 100644 --- a/Analysis/src/TopoSortStatements.cpp +++ b/Analysis/src/TopoSortStatements.cpp @@ -298,8 +298,15 @@ struct ArcCollector : public AstVisitor struct ContainsFunctionCall : public AstVisitor { + bool alsoReturn = false; bool result = false; + ContainsFunctionCall() = default; + explicit ContainsFunctionCall(bool alsoReturn) + : alsoReturn(alsoReturn) + { + } + bool visit(AstExpr*) override { return !result; // short circuit if result is true @@ -318,6 +325,17 @@ struct ContainsFunctionCall : public AstVisitor return false; } + bool visit(AstStatReturn* stat) override + { + if (alsoReturn) + { + result = true; + return false; + } + else + return AstVisitor::visit(stat); + } + bool visit(AstExprFunction*) override { return false; @@ -479,6 +497,13 @@ bool containsFunctionCall(const AstStat& stat) return cfc.result; } +bool containsFunctionCallOrReturn(const AstStat& stat) +{ + detail::ContainsFunctionCall cfc{true}; + const_cast(stat).visit(&cfc); + return cfc.result; +} + bool isFunction(const AstStat& stat) { return stat.is() || stat.is(); diff --git a/Analysis/src/TxnLog.cpp b/Analysis/src/TxnLog.cpp index 702d0ca2..383bb050 100644 --- a/Analysis/src/TxnLog.cpp +++ b/Analysis/src/TxnLog.cpp @@ -5,6 +5,8 @@ #include +LUAU_FASTFLAGVARIABLE(LuauShareTxnSeen, false) + namespace Luau { @@ -33,6 +35,12 @@ void TxnLog::rollback() for (auto it = tableChanges.rbegin(); it != tableChanges.rend(); ++it) std::swap(it->first->boundTo, it->second); + + if (FFlag::LuauShareTxnSeen) + { + LUAU_ASSERT(originalSeenSize <= sharedSeen->size()); + sharedSeen->resize(originalSeenSize); + } } void TxnLog::concat(TxnLog rhs) @@ -46,27 +54,44 @@ void TxnLog::concat(TxnLog rhs) tableChanges.insert(tableChanges.end(), rhs.tableChanges.begin(), rhs.tableChanges.end()); rhs.tableChanges.clear(); - seen.swap(rhs.seen); - rhs.seen.clear(); + if (!FFlag::LuauShareTxnSeen) + { + ownedSeen.swap(rhs.ownedSeen); + rhs.ownedSeen.clear(); + } } bool TxnLog::haveSeen(TypeId lhs, TypeId rhs) { const std::pair sortedPair = (lhs > rhs) ? std::make_pair(lhs, rhs) : std::make_pair(rhs, lhs); - return (seen.end() != std::find(seen.begin(), seen.end(), sortedPair)); + if (FFlag::LuauShareTxnSeen) + return (sharedSeen->end() != std::find(sharedSeen->begin(), sharedSeen->end(), sortedPair)); + else + return (ownedSeen.end() != std::find(ownedSeen.begin(), ownedSeen.end(), sortedPair)); } void TxnLog::pushSeen(TypeId lhs, TypeId rhs) { const std::pair sortedPair = (lhs > rhs) ? std::make_pair(lhs, rhs) : std::make_pair(rhs, lhs); - seen.push_back(sortedPair); + if (FFlag::LuauShareTxnSeen) + sharedSeen->push_back(sortedPair); + else + ownedSeen.push_back(sortedPair); } void TxnLog::popSeen(TypeId lhs, TypeId rhs) { const std::pair sortedPair = (lhs > rhs) ? std::make_pair(lhs, rhs) : std::make_pair(rhs, lhs); - LUAU_ASSERT(sortedPair == seen.back()); - seen.pop_back(); + if (FFlag::LuauShareTxnSeen) + { + LUAU_ASSERT(sortedPair == sharedSeen->back()); + sharedSeen->pop_back(); + } + else + { + LUAU_ASSERT(sortedPair == ownedSeen.back()); + ownedSeen.pop_back(); + } } } // namespace Luau diff --git a/Analysis/src/TypeAttach.cpp b/Analysis/src/TypeAttach.cpp index 266c1986..49f8e0ca 100644 --- a/Analysis/src/TypeAttach.cpp +++ b/Analysis/src/TypeAttach.cpp @@ -6,6 +6,7 @@ #include "Luau/Parser.h" #include "Luau/RecursionCounter.h" #include "Luau/Scope.h" +#include "Luau/ToString.h" #include "Luau/TypeInfer.h" #include "Luau/TypePack.h" #include "Luau/TypeVar.h" @@ -33,14 +34,31 @@ static char* allocateString(Luau::Allocator& allocator, const char* format, Data return result; } +using SyntheticNames = std::unordered_map; + namespace Luau { + +static const char* getName(Allocator* allocator, SyntheticNames* syntheticNames, const Unifiable::Generic& gen) +{ + size_t s = syntheticNames->size(); + char*& n = (*syntheticNames)[&gen]; + if (!n) + { + std::string str = gen.explicitName ? gen.name : generateName(s); + n = static_cast(allocator->allocate(str.size() + 1)); + strcpy(n, str.c_str()); + } + + return n; +} + class TypeRehydrationVisitor { - mutable std::map seen; - mutable int count = 0; + std::map seen; + int count = 0; - bool hasSeen(const void* tv) const + bool hasSeen(const void* tv) { void* ttv = const_cast(tv); auto it = seen.find(ttv); @@ -52,15 +70,16 @@ class TypeRehydrationVisitor } public: - TypeRehydrationVisitor(Allocator* alloc, const TypeRehydrationOptions& options = TypeRehydrationOptions()) + TypeRehydrationVisitor(Allocator* alloc, SyntheticNames* syntheticNames, const TypeRehydrationOptions& options = TypeRehydrationOptions()) : allocator(alloc) + , syntheticNames(syntheticNames) , options(options) { } - AstTypePack* rehydrate(TypePackId tp) const; + AstTypePack* rehydrate(TypePackId tp); - AstType* operator()(const PrimitiveTypeVar& ptv) const + AstType* operator()(const PrimitiveTypeVar& ptv) { switch (ptv.type) { @@ -78,11 +97,11 @@ public: return nullptr; } } - AstType* operator()(const AnyTypeVar&) const + AstType* operator()(const AnyTypeVar&) { return allocator->alloc(Location(), std::nullopt, AstName("any")); } - AstType* operator()(const TableTypeVar& ttv) const + AstType* operator()(const TableTypeVar& ttv) { RecursionCounter counter(&count); @@ -144,12 +163,12 @@ public: return allocator->alloc(Location(), props, indexer); } - AstType* operator()(const MetatableTypeVar& mtv) const + AstType* operator()(const MetatableTypeVar& mtv) { return Luau::visit(*this, mtv.table->ty); } - AstType* operator()(const ClassTypeVar& ctv) const + AstType* operator()(const ClassTypeVar& ctv) { RecursionCounter counter(&count); @@ -176,7 +195,7 @@ public: return allocator->alloc(Location(), props); } - AstType* operator()(const FunctionTypeVar& ftv) const + AstType* operator()(const FunctionTypeVar& ftv) { RecursionCounter counter(&count); @@ -253,10 +272,12 @@ public: size_t i = 0; for (const auto& el : ftv.argNames) { + std::optional* arg = &argNames.data[i++]; + if (el) - argNames.data[i++] = {AstName(el->name.c_str()), el->location}; + new (arg) std::optional(AstArgumentName(AstName(el->name.c_str()), el->location)); else - argNames.data[i++] = {}; + new (arg) std::optional(); } AstArray returnTypes; @@ -290,23 +311,23 @@ public: return allocator->alloc( Location(), generics, genericPacks, AstTypeList{argTypes, argTailAnnotation}, argNames, AstTypeList{returnTypes, retTailAnnotation}); } - AstType* operator()(const Unifiable::Error&) const + AstType* operator()(const Unifiable::Error&) { return allocator->alloc(Location(), std::nullopt, AstName("Unifiable")); } - AstType* operator()(const GenericTypeVar& gtv) const + AstType* operator()(const GenericTypeVar& gtv) { - return allocator->alloc(Location(), std::nullopt, AstName(gtv.name.c_str())); + return allocator->alloc(Location(), std::nullopt, AstName(getName(allocator, syntheticNames, gtv))); } - AstType* operator()(const Unifiable::Bound& bound) const + AstType* operator()(const Unifiable::Bound& bound) { return Luau::visit(*this, bound.boundTo->ty); } - AstType* operator()(Unifiable::Free ftv) const + AstType* operator()(const FreeTypeVar& ftv) { return allocator->alloc(Location(), std::nullopt, AstName("free")); } - AstType* operator()(const UnionTypeVar& uv) const + AstType* operator()(const UnionTypeVar& uv) { AstArray unionTypes; unionTypes.size = uv.options.size(); @@ -317,7 +338,7 @@ public: } return allocator->alloc(Location(), unionTypes); } - AstType* operator()(const IntersectionTypeVar& uv) const + AstType* operator()(const IntersectionTypeVar& uv) { AstArray intersectionTypes; intersectionTypes.size = uv.parts.size(); @@ -328,23 +349,28 @@ public: } return allocator->alloc(Location(), intersectionTypes); } - AstType* operator()(const LazyTypeVar& ltv) const + AstType* operator()(const LazyTypeVar& ltv) { return allocator->alloc(Location(), std::nullopt, AstName("")); } private: Allocator* allocator; + SyntheticNames* syntheticNames; const TypeRehydrationOptions& options; }; class TypePackRehydrationVisitor { public: - TypePackRehydrationVisitor(Allocator* allocator, const TypeRehydrationVisitor& typeVisitor) + TypePackRehydrationVisitor(Allocator* allocator, SyntheticNames* syntheticNames, TypeRehydrationVisitor* typeVisitor) : allocator(allocator) + , syntheticNames(syntheticNames) , typeVisitor(typeVisitor) { + LUAU_ASSERT(allocator); + LUAU_ASSERT(syntheticNames); + LUAU_ASSERT(typeVisitor); } AstTypePack* operator()(const BoundTypePack& btp) const @@ -359,7 +385,7 @@ public: head.data = static_cast(allocator->allocate(sizeof(AstType*) * tp.head.size())); for (size_t i = 0; i < tp.head.size(); i++) - head.data[i] = Luau::visit(typeVisitor, tp.head[i]->ty); + head.data[i] = Luau::visit(*typeVisitor, tp.head[i]->ty); AstTypePack* tail = nullptr; @@ -371,12 +397,12 @@ public: AstTypePack* operator()(const VariadicTypePack& vtp) const { - return allocator->alloc(Location(), Luau::visit(typeVisitor, vtp.ty->ty)); + return allocator->alloc(Location(), Luau::visit(*typeVisitor, vtp.ty->ty)); } AstTypePack* operator()(const GenericTypePack& gtp) const { - return allocator->alloc(Location(), AstName(gtp.name.c_str())); + return allocator->alloc(Location(), AstName(getName(allocator, syntheticNames, gtp))); } AstTypePack* operator()(const FreeTypePack& gtp) const @@ -391,12 +417,13 @@ public: private: Allocator* allocator; - const TypeRehydrationVisitor& typeVisitor; + SyntheticNames* syntheticNames; + TypeRehydrationVisitor* typeVisitor; }; -AstTypePack* TypeRehydrationVisitor::rehydrate(TypePackId tp) const +AstTypePack* TypeRehydrationVisitor::rehydrate(TypePackId tp) { - TypePackRehydrationVisitor tprv(allocator, *this); + TypePackRehydrationVisitor tprv(allocator, syntheticNames, this); return Luau::visit(tprv, tp->ty); } @@ -431,7 +458,7 @@ public: { if (!type) return nullptr; - return Luau::visit(TypeRehydrationVisitor(allocator), (*type)->ty); + return Luau::visit(TypeRehydrationVisitor(allocator, &syntheticNames), (*type)->ty); } AstArray typeAstPack(TypePackId type) @@ -443,7 +470,7 @@ public: result.data = static_cast(allocator->allocate(sizeof(AstType*) * v.size())); for (size_t i = 0; i < v.size(); ++i) { - result.data[i] = Luau::visit(TypeRehydrationVisitor(allocator), v[i]->ty); + result.data[i] = Luau::visit(TypeRehydrationVisitor(allocator, &syntheticNames), v[i]->ty); } return result; } @@ -495,7 +522,7 @@ public: { if (FFlag::LuauTypeAliasPacks) { - variadicAnnotation = TypeRehydrationVisitor(allocator).rehydrate(*tail); + variadicAnnotation = TypeRehydrationVisitor(allocator, &syntheticNames).rehydrate(*tail); } else { @@ -515,6 +542,7 @@ public: private: Module& module; Allocator* allocator; + SyntheticNames syntheticNames; }; void attachTypeData(SourceModule& source, Module& result) @@ -525,7 +553,8 @@ void attachTypeData(SourceModule& source, Module& result) AstType* rehydrateAnnotation(TypeId type, Allocator* allocator, const TypeRehydrationOptions& options) { - return Luau::visit(TypeRehydrationVisitor(allocator, options), type->ty); + SyntheticNames syntheticNames; + return Luau::visit(TypeRehydrationVisitor(allocator, &syntheticNames, options), type->ty); } } // namespace Luau diff --git a/Analysis/src/TypeInfer.cpp b/Analysis/src/TypeInfer.cpp index da93c8ec..38e2e527 100644 --- a/Analysis/src/TypeInfer.cpp +++ b/Analysis/src/TypeInfer.cpp @@ -4,6 +4,7 @@ #include "Luau/Common.h" #include "Luau/ModuleResolver.h" #include "Luau/Parser.h" +#include "Luau/Quantify.h" #include "Luau/RecursionCounter.h" #include "Luau/Scope.h" #include "Luau/Substitution.h" @@ -33,18 +34,16 @@ LUAU_FASTFLAGVARIABLE(LuauCloneCorrectlyBeforeMutatingTableType, false) LUAU_FASTFLAGVARIABLE(LuauStoreMatchingOverloadFnType, false) LUAU_FASTFLAGVARIABLE(LuauRankNTypes, false) LUAU_FASTFLAGVARIABLE(LuauOrPredicate, false) -LUAU_FASTFLAGVARIABLE(LuauExtraNilRecovery, false) -LUAU_FASTFLAGVARIABLE(LuauMissingUnionPropertyError, false) LUAU_FASTFLAGVARIABLE(LuauInferReturnAssertAssign, false) LUAU_FASTFLAGVARIABLE(LuauRecursiveTypeParameterRestriction, false) LUAU_FASTFLAGVARIABLE(LuauAddMissingFollow, false) LUAU_FASTFLAGVARIABLE(LuauTypeGuardPeelsAwaySubclasses, false) LUAU_FASTFLAGVARIABLE(LuauSlightlyMoreFlexibleBinaryPredicates, false) -LUAU_FASTFLAGVARIABLE(LuauInferFunctionArgsFix, false) LUAU_FASTFLAGVARIABLE(LuauFollowInTypeFunApply, false) LUAU_FASTFLAGVARIABLE(LuauIfElseExpressionAnalysisSupport, false) LUAU_FASTFLAGVARIABLE(LuauStrictRequire, false) LUAU_FASTFLAG(LuauSubstitutionDontReplaceIgnoredTypes) +LUAU_FASTFLAGVARIABLE(LuauQuantifyInPlace2, false) LUAU_FASTFLAG(LuauNewRequireTrace) LUAU_FASTFLAG(LuauTypeAliasPacks) @@ -215,6 +214,7 @@ static bool isMetamethod(const Name& name) TypeChecker::TypeChecker(ModuleResolver* resolver, InternalErrorReporter* iceHandler) : resolver(resolver) , iceHandler(iceHandler) + , unifierState(iceHandler) , nilType(singletonTypes.nilType) , numberType(singletonTypes.numberType) , stringType(singletonTypes.stringType) @@ -370,13 +370,18 @@ void TypeChecker::checkBlock(const ScopePtr& scope, const AstStatBlock& block) return; } + int subLevel = 0; + std::vector sorted(block.body.data, block.body.data + block.body.size); toposort(sorted); for (const auto& stat : sorted) { if (const auto& typealias = stat->as()) - check(scope, *typealias, true); + { + check(scope, *typealias, subLevel, true); + ++subLevel; + } } auto protoIter = sorted.begin(); @@ -399,8 +404,6 @@ void TypeChecker::checkBlock(const ScopePtr& scope, const AstStatBlock& block) } }; - int subLevel = 0; - while (protoIter != sorted.end()) { // protoIter walks forward @@ -433,7 +436,7 @@ void TypeChecker::checkBlock(const ScopePtr& scope, const AstStatBlock& block) // function f(x:a):a local x: number = g(37) return x end // function g(x:number):number return f(x) end // ``` - if (containsFunctionCall(**protoIter)) + if (FFlag::LuauQuantifyInPlace2 ? containsFunctionCallOrReturn(**protoIter) : containsFunctionCall(**protoIter)) { while (checkIter != protoIter) { @@ -1161,7 +1164,7 @@ void TypeChecker::check(const ScopePtr& scope, TypeId ty, const ScopePtr& funSco scope->bindings[function.name] = {quantify(scope, ty, function.name->location), function.name->location}; } -void TypeChecker::check(const ScopePtr& scope, const AstStatTypeAlias& typealias, bool forwardDeclare) +void TypeChecker::check(const ScopePtr& scope, const AstStatTypeAlias& typealias, int subLevel, bool forwardDeclare) { // This function should be called at most twice for each type alias. // Once with forwardDeclare, and once without. @@ -1189,11 +1192,12 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatTypeAlias& typealias } else { - ScopePtr aliasScope = childScope(scope, typealias.location); + ScopePtr aliasScope = + FFlag::LuauQuantifyInPlace2 ? childScope(scope, typealias.location, subLevel) : childScope(scope, typealias.location); if (FFlag::LuauTypeAliasPacks) { - auto [generics, genericPacks] = createGenericTypes(aliasScope, typealias, typealias.generics, typealias.genericPacks); + auto [generics, genericPacks] = createGenericTypes(aliasScope, scope->level, typealias, typealias.generics, typealias.genericPacks); TypeId ty = (FFlag::LuauRankNTypes ? freshType(aliasScope) : DEPRECATED_freshType(scope, true)); FreeTypeVar* ftv = getMutable(ty); @@ -1418,7 +1422,7 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatDeclareFunction& glo { ScopePtr funScope = childFunctionScope(scope, global.location); - auto [generics, genericPacks] = createGenericTypes(funScope, global, global.generics, global.genericPacks); + auto [generics, genericPacks] = createGenericTypes(funScope, std::nullopt, global, global.generics, global.genericPacks); TypePackId argPack = resolveTypePack(funScope, global.params); TypePackId retPack = resolveTypePack(funScope, global.retTypes); @@ -1610,25 +1614,11 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprIn if (std::optional ty = resolveLValue(scope, *lvalue)) return {*ty, {TruthyPredicate{std::move(*lvalue), expr.location}}}; - if (FFlag::LuauExtraNilRecovery) - lhsType = stripFromNilAndReport(lhsType, expr.expr->location); + lhsType = stripFromNilAndReport(lhsType, expr.expr->location); if (std::optional ty = getIndexTypeFromType(scope, lhsType, name, expr.location, true)) return {*ty}; - if (!FFlag::LuauMissingUnionPropertyError) - reportError(expr.indexLocation, UnknownProperty{lhsType, expr.index.value}); - - if (!FFlag::LuauExtraNilRecovery) - { - // Try to recover using a union without 'nil' options - if (std::optional strippedUnion = tryStripUnionFromNil(lhsType)) - { - if (std::optional ty = getIndexTypeFromType(scope, *strippedUnion, name, expr.location, false)) - return {*ty}; - } - } - return {errorType}; } @@ -1694,61 +1684,37 @@ std::optional TypeChecker::getIndexTypeFromType( } else if (const UnionTypeVar* utv = get(type)) { - if (FFlag::LuauMissingUnionPropertyError) + std::vector goodOptions; + std::vector badOptions; + + for (TypeId t : utv) { - std::vector goodOptions; - std::vector badOptions; + RecursionLimiter _rl(&recursionCount, FInt::LuauTypeInferRecursionLimit); - for (TypeId t : utv) - { - RecursionLimiter _rl(&recursionCount, FInt::LuauTypeInferRecursionLimit); - - if (std::optional ty = getIndexTypeFromType(scope, t, name, location, false)) - goodOptions.push_back(*ty); - else - badOptions.push_back(t); - } - - if (!badOptions.empty()) - { - if (addErrors) - { - if (goodOptions.empty()) - reportError(location, UnknownProperty{type, name}); - else - reportError(location, MissingUnionProperty{type, badOptions, name}); - } - return std::nullopt; - } - - std::vector result = reduceUnion(goodOptions); - - if (result.size() == 1) - return result[0]; - - return addType(UnionTypeVar{std::move(result)}); + if (std::optional ty = getIndexTypeFromType(scope, t, name, location, false)) + goodOptions.push_back(*ty); + else + badOptions.push_back(t); } - else + + if (!badOptions.empty()) { - std::vector options; - - for (TypeId t : utv->options) + if (addErrors) { - RecursionLimiter _rl(&recursionCount, FInt::LuauTypeInferRecursionLimit); - - if (std::optional ty = getIndexTypeFromType(scope, t, name, location, false)) - options.push_back(*ty); + if (goodOptions.empty()) + reportError(location, UnknownProperty{type, name}); else - return std::nullopt; + reportError(location, MissingUnionProperty{type, badOptions, name}); } - - std::vector result = reduceUnion(options); - - if (result.size() == 1) - return result[0]; - - return addType(UnionTypeVar{std::move(result)}); + return std::nullopt; } + + std::vector result = reduceUnion(goodOptions); + + if (result.size() == 1) + return result[0]; + + return addType(UnionTypeVar{std::move(result)}); } else if (const IntersectionTypeVar* itv = get(type)) { @@ -1765,7 +1731,7 @@ std::optional TypeChecker::getIndexTypeFromType( // If no parts of the intersection had the property we looked up for, it never existed at all. if (parts.empty()) { - if (FFlag::LuauMissingUnionPropertyError && addErrors) + if (addErrors) reportError(location, UnknownProperty{type, name}); return std::nullopt; } @@ -1779,7 +1745,7 @@ std::optional TypeChecker::getIndexTypeFromType( return addType(IntersectionTypeVar{result}); } - if (FFlag::LuauMissingUnionPropertyError && addErrors) + if (addErrors) reportError(location, UnknownProperty{type, name}); return std::nullopt; @@ -2062,8 +2028,7 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprUn case AstExprUnary::Len: tablify(operandType); - if (FFlag::LuauExtraNilRecovery) - operandType = stripFromNilAndReport(operandType, expr.location); + operandType = stripFromNilAndReport(operandType, expr.location); if (get(operandType)) return {errorType}; @@ -2635,8 +2600,7 @@ std::pair TypeChecker::checkLValueBinding(const ScopePtr& scope Name name = expr.index.value; - if (FFlag::LuauExtraNilRecovery) - lhs = stripFromNilAndReport(lhs, expr.expr->location); + lhs = stripFromNilAndReport(lhs, expr.expr->location); if (TableTypeVar* lhsTable = getMutableTableType(lhs)) { @@ -2710,8 +2674,7 @@ std::pair TypeChecker::checkLValueBinding(const ScopePtr& scope TypeId exprType = checkExpr(scope, *expr.expr).type; tablify(exprType); - if (FFlag::LuauExtraNilRecovery) - exprType = stripFromNilAndReport(exprType, expr.expr->location); + exprType = stripFromNilAndReport(exprType, expr.expr->location); TypeId indexType = checkExpr(scope, *expr.index).type; @@ -2738,10 +2701,7 @@ std::pair TypeChecker::checkLValueBinding(const ScopePtr& scope if (!exprTable) { - if (FFlag::LuauExtraNilRecovery) - reportError(TypeError{expr.expr->location, NotATable{exprType}}); - else - reportError(TypeError{expr.location, NotATable{exprType}}); + reportError(TypeError{expr.expr->location, NotATable{exprType}}); return std::pair(errorType, nullptr); } @@ -2910,7 +2870,7 @@ std::pair TypeChecker::checkFunctionSignature( if (FFlag::LuauGenericFunctions) { - std::tie(generics, genericPacks) = createGenericTypes(funScope, expr, expr.generics, expr.genericPacks); + std::tie(generics, genericPacks) = createGenericTypes(funScope, std::nullopt, expr, expr.generics, expr.genericPacks); } TypePackId retPack; @@ -3016,9 +2976,6 @@ std::pair TypeChecker::checkFunctionSignature( if (expectedArgsCurr != expectedArgsEnd) { argType = *expectedArgsCurr; - - if (!FFlag::LuauInferFunctionArgsFix) - ++expectedArgsCurr; } else if (auto expectedArgsTail = expectedArgsCurr.tail()) { @@ -3034,7 +2991,7 @@ std::pair TypeChecker::checkFunctionSignature( funScope->bindings[local] = {argType, local->location}; argTypes.push_back(argType); - if (FFlag::LuauInferFunctionArgsFix && expectedArgsCurr != expectedArgsEnd) + if (expectedArgsCurr != expectedArgsEnd) ++expectedArgsCurr; } @@ -3402,8 +3359,7 @@ ExprResult TypeChecker::checkExprPack(const ScopePtr& scope, const A if (!FFlag::LuauRankNTypes) instantiate(scope, selfType, expr.func->location); - if (FFlag::LuauExtraNilRecovery) - selfType = stripFromNilAndReport(selfType, expr.func->location); + selfType = stripFromNilAndReport(selfType, expr.func->location); if (std::optional propTy = getIndexTypeFromType(scope, selfType, indexExpr->index.value, expr.location, true)) { @@ -3412,34 +3368,8 @@ ExprResult TypeChecker::checkExprPack(const ScopePtr& scope, const A } else { - if (!FFlag::LuauMissingUnionPropertyError) - reportError(indexExpr->indexLocation, UnknownProperty{selfType, indexExpr->index.value}); - - if (!FFlag::LuauExtraNilRecovery) - { - // Try to recover using a union without 'nil' options - if (std::optional strippedUnion = tryStripUnionFromNil(selfType)) - { - if (std::optional propTy = getIndexTypeFromType(scope, *strippedUnion, indexExpr->index.value, expr.location, false)) - { - selfType = *strippedUnion; - - functionType = *propTy; - actualFunctionType = instantiate(scope, functionType, expr.func->location); - } - } - - if (!actualFunctionType) - { - functionType = errorType; - actualFunctionType = errorType; - } - } - else - { - functionType = errorType; - actualFunctionType = errorType; - } + functionType = errorType; + actualFunctionType = errorType; } } else @@ -3555,8 +3485,7 @@ std::optional> TypeChecker::checkCallOverload(const Scope TypePackId argPack, TypePack* args, const std::vector& argLocations, const ExprResult& argListResult, std::vector& overloadsThatMatchArgCount, std::vector& errors) { - if (FFlag::LuauExtraNilRecovery) - fn = stripFromNilAndReport(fn, expr.func->location); + fn = stripFromNilAndReport(fn, expr.func->location); if (get(fn)) { @@ -4283,6 +4212,12 @@ TypeId TypeChecker::quantify(const ScopePtr& scope, TypeId ty, Location location if (!ftv || !ftv->generics.empty() || !ftv->genericPacks.empty()) return ty; + if (FFlag::LuauQuantifyInPlace2) + { + Luau::quantify(currentModule, ty, scope->level); + return ty; + } + quantification.level = scope->level; quantification.generics.clear(); quantification.genericPacks.clear(); @@ -4491,12 +4426,12 @@ void TypeChecker::merge(RefinementMap& l, const RefinementMap& r) Unifier TypeChecker::mkUnifier(const Location& location) { - return Unifier{¤tModule->internalTypes, currentModule->mode, globalScope, location, Variance::Covariant, iceHandler}; + return Unifier{¤tModule->internalTypes, currentModule->mode, globalScope, location, Variance::Covariant, unifierState}; } Unifier TypeChecker::mkUnifier(const std::vector>& seen, const Location& location) { - return Unifier{¤tModule->internalTypes, currentModule->mode, globalScope, seen, location, Variance::Covariant, iceHandler}; + return Unifier{¤tModule->internalTypes, currentModule->mode, globalScope, seen, location, Variance::Covariant, unifierState}; } TypeId TypeChecker::freshType(const ScopePtr& scope) @@ -4753,7 +4688,7 @@ TypeId TypeChecker::resolveType(const ScopePtr& scope, const AstType& annotation if (FFlag::LuauGenericFunctions) { - std::tie(generics, genericPacks) = createGenericTypes(funcScope, annotation, func->generics, func->genericPacks); + std::tie(generics, genericPacks) = createGenericTypes(funcScope, std::nullopt, annotation, func->generics, func->genericPacks); } // TODO: better error message CLI-39912 @@ -5041,10 +4976,12 @@ TypeId TypeChecker::instantiateTypeFun(const ScopePtr& scope, const TypeFun& tf, } std::pair, std::vector> TypeChecker::createGenericTypes( - const ScopePtr& scope, const AstNode& node, const AstArray& genericNames, const AstArray& genericPackNames) + const ScopePtr& scope, std::optional levelOpt, const AstNode& node, const AstArray& genericNames, const AstArray& genericPackNames) { LUAU_ASSERT(scope->parent); + const TypeLevel level = (FFlag::LuauQuantifyInPlace2 && levelOpt) ? *levelOpt : scope->level; + std::vector generics; for (const AstName& generic : genericNames) { @@ -5063,12 +5000,12 @@ std::pair, std::vector> TypeChecker::createGener { TypeId& cached = scope->parent->typeAliasTypeParameters[n]; if (!cached) - cached = addType(GenericTypeVar{scope->level, n}); + cached = addType(GenericTypeVar{level, n}); g = cached; } else { - g = addType(Unifiable::Generic{scope->level, n}); + g = addType(Unifiable::Generic{level, n}); } generics.push_back(g); @@ -5093,12 +5030,12 @@ std::pair, std::vector> TypeChecker::createGener { TypePackId& cached = scope->parent->typeAliasTypePackParameters[n]; if (!cached) - cached = addTypePack(TypePackVar{Unifiable::Generic{scope->level, n}}); + cached = addTypePack(TypePackVar{Unifiable::Generic{level, n}}); g = cached; } else { - g = addTypePack(TypePackVar{Unifiable::Generic{scope->level, n}}); + g = addTypePack(TypePackVar{Unifiable::Generic{level, n}}); } genericPacks.push_back(g); diff --git a/Analysis/src/TypeVar.cpp b/Analysis/src/TypeVar.cpp index e963fc74..e82f7519 100644 --- a/Analysis/src/TypeVar.cpp +++ b/Analysis/src/TypeVar.cpp @@ -22,6 +22,7 @@ LUAU_FASTINTVARIABLE(LuauTableTypeMaximumStringifierLength, 0) LUAU_FASTFLAG(LuauRankNTypes) LUAU_FASTFLAG(LuauTypeGuardPeelsAwaySubclasses) LUAU_FASTFLAG(LuauTypeAliasPacks) +LUAU_FASTFLAGVARIABLE(LuauRefactorTagging, false) namespace Luau { @@ -217,8 +218,7 @@ std::optional getMetatable(TypeId type) return mtType->metatable; else if (const ClassTypeVar* classType = get(type)) return classType->metatable; - else if (const PrimitiveTypeVar* primitiveType = get(type); - primitiveType && primitiveType->metatable) + else if (const PrimitiveTypeVar* primitiveType = get(type); primitiveType && primitiveType->metatable) { LUAU_ASSERT(primitiveType->type == PrimitiveTypeVar::String); return primitiveType->metatable; @@ -1490,4 +1490,86 @@ std::vector filterMap(TypeId type, TypeIdPredicate predicate) return {}; } +static Tags* getTags(TypeId ty) +{ + ty = follow(ty); + + if (auto ftv = getMutable(ty)) + return &ftv->tags; + else if (auto ttv = getMutable(ty)) + return &ttv->tags; + else if (auto ctv = getMutable(ty)) + return &ctv->tags; + + return nullptr; +} + +void attachTag(TypeId ty, const std::string& tagName) +{ + if (!FFlag::LuauRefactorTagging) + { + if (auto ftv = getMutable(ty)) + { + ftv->tags.emplace_back(tagName); + } + else + { + LUAU_ASSERT(!"Got a non functional type"); + } + } + else + { + if (auto tags = getTags(ty)) + tags->push_back(tagName); + else + LUAU_ASSERT(!"This TypeId does not support tags"); + } +} + +void attachTag(Property& prop, const std::string& tagName) +{ + LUAU_ASSERT(FFlag::LuauRefactorTagging); + + prop.tags.push_back(tagName); +} + +// We would ideally not expose this because it could cause a footgun. +// If the Base class has a tag and you ask if Derived has that tag, it would return false. +// Unfortunately, there's already use cases that's hard to disentangle. For now, we expose it. +bool hasTag(const Tags& tags, const std::string& tagName) +{ + LUAU_ASSERT(FFlag::LuauRefactorTagging); + return std::find(tags.begin(), tags.end(), tagName) != tags.end(); +} + +bool hasTag(TypeId ty, const std::string& tagName) +{ + ty = follow(ty); + + // We special case classes because getTags only returns a pointer to one vector of tags. + // But classes has multiple vector of tags, represented throughout the hierarchy. + if (auto ctv = get(ty)) + { + while (ctv) + { + if (hasTag(ctv->tags, tagName)) + return true; + else if (!ctv->parent) + return false; + + ctv = get(*ctv->parent); + LUAU_ASSERT(ctv); + } + } + else if (auto tags = getTags(ty)) + return hasTag(*tags, tagName); + + return false; +} + +bool hasTag(const Property& prop, const std::string& tagName) +{ + return hasTag(prop.tags, tagName); +} + } // namespace Luau diff --git a/Analysis/src/Unifier.cpp b/Analysis/src/Unifier.cpp index 56f2515f..2539650a 100644 --- a/Analysis/src/Unifier.cpp +++ b/Analysis/src/Unifier.cpp @@ -7,6 +7,7 @@ #include "Luau/TypePack.h" #include "Luau/TypeUtils.h" #include "Luau/TimeTrace.h" +#include "Luau/VisitTypeVar.h" #include @@ -22,9 +23,99 @@ LUAU_FASTFLAGVARIABLE(LuauTableUnificationEarlyTest, false) LUAU_FASTFLAGVARIABLE(LuauSealedTableUnifyOptionalFix, false) LUAU_FASTFLAGVARIABLE(LuauOccursCheckOkWithRecursiveFunctions, false) LUAU_FASTFLAGVARIABLE(LuauTypecheckOpts, false) +LUAU_FASTFLAG(LuauShareTxnSeen); +LUAU_FASTFLAGVARIABLE(LuauCacheUnifyTableResults, false) namespace Luau { +struct SkipCacheForType +{ + SkipCacheForType(const DenseHashMap& skipCacheForType) + : skipCacheForType(skipCacheForType) + { + } + + void cycle(TypeId) {} + void cycle(TypePackId) {} + + bool operator()(TypeId ty, const FreeTypeVar& ftv) + { + result = true; + return false; + } + + bool operator()(TypeId ty, const BoundTypeVar& btv) + { + result = true; + return false; + } + + bool operator()(TypeId ty, const GenericTypeVar& btv) + { + result = true; + return false; + } + + bool operator()(TypeId ty, const TableTypeVar&) + { + TableTypeVar& ttv = *getMutable(ty); + + if (ttv.boundTo) + { + result = true; + return false; + } + + if (ttv.state != TableState::Sealed) + { + result = true; + return false; + } + + return true; + } + + template + bool operator()(TypeId ty, const T& t) + { + const bool* prev = skipCacheForType.find(ty); + + if (prev && *prev) + { + result = true; + return false; + } + + return true; + } + + template + bool operator()(TypePackId, const T&) + { + return true; + } + + bool operator()(TypePackId tp, const FreeTypePack& ftp) + { + result = true; + return false; + } + + bool operator()(TypePackId tp, const BoundTypePack& ftp) + { + result = true; + return false; + } + + bool operator()(TypePackId tp, const GenericTypePack& ftp) + { + result = true; + return false; + } + + const DenseHashMap& skipCacheForType; + bool result = false; +}; static std::optional hasUnificationTooComplex(const ErrorVec& errors) { @@ -39,7 +130,7 @@ static std::optional hasUnificationTooComplex(const ErrorVec& errors) return *it; } -Unifier::Unifier(TypeArena* types, Mode mode, ScopePtr globalScope, const Location& location, Variance variance, InternalErrorReporter* iceHandler) +Unifier::Unifier(TypeArena* types, Mode mode, ScopePtr globalScope, const Location& location, Variance variance, UnifierSharedState& sharedState) : types(types) , mode(mode) , globalScope(std::move(globalScope)) @@ -47,24 +138,39 @@ Unifier::Unifier(TypeArena* types, Mode mode, ScopePtr globalScope, const Locati , variance(variance) , counters(&countersData) , counters_DEPRECATED(std::make_shared()) - , iceHandler(iceHandler) + , sharedState(sharedState) { - LUAU_ASSERT(iceHandler); + LUAU_ASSERT(sharedState.iceHandler); } -Unifier::Unifier(TypeArena* types, Mode mode, ScopePtr globalScope, const std::vector>& seen, const Location& location, - Variance variance, InternalErrorReporter* iceHandler, const std::shared_ptr& counters_DEPRECATED, UnifierCounters* counters) +Unifier::Unifier(TypeArena* types, Mode mode, ScopePtr globalScope, const std::vector>& ownedSeen, const Location& location, + Variance variance, UnifierSharedState& sharedState, const std::shared_ptr& counters_DEPRECATED, UnifierCounters* counters) : types(types) , mode(mode) , globalScope(std::move(globalScope)) - , log(seen) + , log(ownedSeen) , location(location) , variance(variance) , counters(counters ? counters : &countersData) , counters_DEPRECATED(counters_DEPRECATED ? counters_DEPRECATED : std::make_shared()) - , iceHandler(iceHandler) + , sharedState(sharedState) { - LUAU_ASSERT(iceHandler); + LUAU_ASSERT(sharedState.iceHandler); +} + +Unifier::Unifier(TypeArena* types, Mode mode, ScopePtr globalScope, std::vector>* sharedSeen, const Location& location, + Variance variance, UnifierSharedState& sharedState, const std::shared_ptr& counters_DEPRECATED, UnifierCounters* counters) + : types(types) + , mode(mode) + , globalScope(std::move(globalScope)) + , log(sharedSeen) + , location(location) + , variance(variance) + , counters(counters ? counters : &countersData) + , counters_DEPRECATED(counters_DEPRECATED ? counters_DEPRECATED : std::make_shared()) + , sharedState(sharedState) +{ + LUAU_ASSERT(sharedState.iceHandler); } void Unifier::tryUnify(TypeId superTy, TypeId subTy, bool isFunctionCall, bool isIntersection) @@ -74,7 +180,7 @@ void Unifier::tryUnify(TypeId superTy, TypeId subTy, bool isFunctionCall, bool i else counters_DEPRECATED->iterationCount = 0; - return tryUnify_(superTy, subTy, isFunctionCall, isIntersection); + tryUnify_(superTy, subTy, isFunctionCall, isIntersection); } void Unifier::tryUnify_(TypeId superTy, TypeId subTy, bool isFunctionCall, bool isIntersection) @@ -206,6 +312,13 @@ void Unifier::tryUnify_(TypeId superTy, TypeId subTy, bool isFunctionCall, bool if (get(subTy) || get(subTy)) return tryUnifyWithAny(subTy, superTy); + bool cacheEnabled = FFlag::LuauCacheUnifyTableResults && !isFunctionCall && !isIntersection; + auto& cache = sharedState.cachedUnify; + + // What if the types are immutable and we proved their relation before + if (cacheEnabled && cache.contains({superTy, subTy}) && (variance == Covariant || cache.contains({subTy, superTy}))) + return; + // If we have seen this pair of types before, we are currently recursing into cyclic types. // Here, we assume that the types unify. If they do not, we will find out as we roll back // the stack. @@ -257,6 +370,8 @@ void Unifier::tryUnify_(TypeId superTy, TypeId subTy, bool isFunctionCall, bool if (FFlag::LuauUnionHeuristic) { + bool found = false; + const std::string* subName = getName(subTy); if (subName) { @@ -264,6 +379,21 @@ void Unifier::tryUnify_(TypeId superTy, TypeId subTy, bool isFunctionCall, bool { const std::string* optionName = getName(uv->options[i]); if (optionName && *optionName == *subName) + { + found = true; + startIndex = i; + break; + } + } + } + + if (!found && cacheEnabled) + { + for (size_t i = 0; i < uv->options.size(); ++i) + { + TypeId type = uv->options[i]; + + if (cache.contains({type, subTy}) && (variance == Covariant || cache.contains({subTy, type}))) { startIndex = i; break; @@ -311,8 +441,25 @@ void Unifier::tryUnify_(TypeId superTy, TypeId subTy, bool isFunctionCall, bool bool found = false; std::optional unificationTooComplex; - for (TypeId type : uv->parts) + size_t startIndex = 0; + + if (cacheEnabled) { + for (size_t i = 0; i < uv->parts.size(); ++i) + { + TypeId type = uv->parts[i]; + + if (cache.contains({superTy, type}) && (variance == Covariant || cache.contains({type, superTy}))) + { + startIndex = i; + break; + } + } + } + + for (size_t i = 0; i < uv->parts.size(); ++i) + { + TypeId type = uv->parts[(i + startIndex) % uv->parts.size()]; Unifier innerState = makeChildUnifier(); innerState.tryUnify_(superTy, type, isFunctionCall); @@ -342,8 +489,13 @@ void Unifier::tryUnify_(TypeId superTy, TypeId subTy, bool isFunctionCall, bool tryUnifyFunctions(superTy, subTy, isFunctionCall); else if (get(superTy) && get(subTy)) + { tryUnifyTables(superTy, subTy, isIntersection); + if (cacheEnabled && errors.empty()) + cacheResult(superTy, subTy); + } + // tryUnifyWithMetatable assumes its first argument is a MetatableTypeVar. The check is otherwise symmetrical. else if (get(superTy)) tryUnifyWithMetatable(superTy, subTy, /*reversed*/ false); @@ -364,6 +516,41 @@ void Unifier::tryUnify_(TypeId superTy, TypeId subTy, bool isFunctionCall, bool log.popSeen(superTy, subTy); } +void Unifier::cacheResult(TypeId superTy, TypeId subTy) +{ + LUAU_ASSERT(FFlag::LuauCacheUnifyTableResults); + + bool* superTyInfo = sharedState.skipCacheForType.find(superTy); + + if (superTyInfo && *superTyInfo) + return; + + bool* subTyInfo = sharedState.skipCacheForType.find(subTy); + + if (subTyInfo && *subTyInfo) + return; + + auto skipCacheFor = [this](TypeId ty) { + SkipCacheForType visitor{sharedState.skipCacheForType}; + visitTypeVarOnce(ty, visitor, sharedState.seenAny); + + sharedState.skipCacheForType[ty] = visitor.result; + + return visitor.result; + }; + + if (!superTyInfo && skipCacheFor(superTy)) + return; + + if (!subTyInfo && skipCacheFor(subTy)) + return; + + sharedState.cachedUnify.insert({superTy, subTy}); + + if (variance == Invariant) + sharedState.cachedUnify.insert({subTy, superTy}); +} + struct WeirdIter { TypePackId packId; @@ -459,7 +646,7 @@ void Unifier::tryUnify(TypePackId superTp, TypePackId subTp, bool isFunctionCall else counters_DEPRECATED->iterationCount = 0; - return tryUnify_(superTp, subTp, isFunctionCall); + tryUnify_(superTp, subTp, isFunctionCall); } /* @@ -797,6 +984,40 @@ void Unifier::tryUnifyTables(TypeId left, TypeId right, bool isIntersection) std::vector missingProperties; std::vector extraProperties; + // Optimization: First test that the property sets are compatible without doing any recursive unification + if (FFlag::LuauTableUnificationEarlyTest && !rt->indexer && rt->state != TableState::Free) + { + for (const auto& [propName, superProp] : lt->props) + { + auto subIter = rt->props.find(propName); + if (subIter == rt->props.end() && !isOptional(superProp.type) && !get(follow(superProp.type))) + missingProperties.push_back(propName); + } + + if (!missingProperties.empty()) + { + errors.push_back(TypeError{location, MissingProperties{left, right, std::move(missingProperties)}}); + return; + } + } + + // And vice versa if we're invariant + if (FFlag::LuauTableUnificationEarlyTest && variance == Invariant && !lt->indexer && lt->state != TableState::Unsealed && lt->state != TableState::Free) + { + for (const auto& [propName, subProp] : rt->props) + { + auto superIter = lt->props.find(propName); + if (superIter == lt->props.end() && !isOptional(subProp.type) && !get(follow(subProp.type))) + extraProperties.push_back(propName); + } + + if (!extraProperties.empty()) + { + errors.push_back(TypeError{location, MissingProperties{left, right, std::move(extraProperties), MissingProperties::Extra}}); + return; + } + } + // Reminder: left is the supertype, right is the subtype. // Width subtyping: any property in the supertype must be in the subtype, // and the types must agree. @@ -833,9 +1054,10 @@ void Unifier::tryUnifyTables(TypeId left, TypeId right, bool isIntersection) innerState.log.rollback(); } else if (isOptional(prop.type) || get(follow(prop.type))) - // TODO: this case is unsound, but without it our test suite fails. CLI-46031 - // TODO: should isOptional(anyType) be true? - {} + // TODO: this case is unsound, but without it our test suite fails. CLI-46031 + // TODO: should isOptional(anyType) be true? + { + } else if (rt->state == TableState::Free) { log(rt); @@ -878,11 +1100,13 @@ void Unifier::tryUnifyTables(TypeId left, TypeId right, bool isIntersection) lt->props[name] = clone; } else if (variance == Covariant) - {} + { + } else if (isOptional(prop.type) || get(follow(prop.type))) - // TODO: this case is unsound, but without it our test suite fails. CLI-46031 - // TODO: should isOptional(anyType) be true? - {} + // TODO: this case is unsound, but without it our test suite fails. CLI-46031 + // TODO: should isOptional(anyType) be true? + { + } else if (lt->state == TableState::Free) { log(lt); @@ -980,10 +1204,10 @@ TypeId Unifier::deeplyOptional(TypeId ty, std::unordered_map see TableTypeVar* resultTtv = getMutable(result); for (auto& [name, prop] : resultTtv->props) prop.type = deeplyOptional(prop.type, seen); - return types->addType(UnionTypeVar{{ singletonTypes.nilType, result }});; + return types->addType(UnionTypeVar{{singletonTypes.nilType, result}}); } else - return types->addType(UnionTypeVar{{ singletonTypes.nilType, ty }}); + return types->addType(UnionTypeVar{{singletonTypes.nilType, ty}}); } void Unifier::DEPRECATED_tryUnifyTables(TypeId left, TypeId right, bool isIntersection) @@ -1697,10 +1921,20 @@ void Unifier::tryUnifyWithAny(TypeId any, TypeId ty) { std::vector queue = {ty}; - tempSeenTy.clear(); - tempSeenTp.clear(); + if (FFlag::LuauCacheUnifyTableResults) + { + sharedState.tempSeenTy.clear(); + sharedState.tempSeenTp.clear(); - Luau::tryUnifyWithAny(queue, *this, tempSeenTy, tempSeenTp, singletonTypes.anyType, anyTP); + Luau::tryUnifyWithAny(queue, *this, sharedState.tempSeenTy, sharedState.tempSeenTp, singletonTypes.anyType, anyTP); + } + else + { + tempSeenTy_DEPRECATED.clear(); + tempSeenTp_DEPRECATED.clear(); + + Luau::tryUnifyWithAny(queue, *this, tempSeenTy_DEPRECATED, tempSeenTp_DEPRECATED, singletonTypes.anyType, anyTP); + } } else { @@ -1721,12 +1955,24 @@ void Unifier::tryUnifyWithAny(TypePackId any, TypePackId ty) { std::vector queue; - tempSeenTy.clear(); - tempSeenTp.clear(); + if (FFlag::LuauCacheUnifyTableResults) + { + sharedState.tempSeenTy.clear(); + sharedState.tempSeenTp.clear(); - queueTypePack(queue, tempSeenTp, *this, ty, any); + queueTypePack(queue, sharedState.tempSeenTp, *this, ty, any); - Luau::tryUnifyWithAny(queue, *this, tempSeenTy, tempSeenTp, anyTy, any); + Luau::tryUnifyWithAny(queue, *this, sharedState.tempSeenTy, sharedState.tempSeenTp, anyTy, any); + } + else + { + tempSeenTy_DEPRECATED.clear(); + tempSeenTp_DEPRECATED.clear(); + + queueTypePack(queue, tempSeenTp_DEPRECATED, *this, ty, any); + + Luau::tryUnifyWithAny(queue, *this, tempSeenTy_DEPRECATED, tempSeenTp_DEPRECATED, anyTy, any); + } } else { @@ -1775,10 +2021,20 @@ void Unifier::occursCheck(TypeId needle, TypeId haystack) { std::unordered_set seen_DEPRECATED; - if (FFlag::LuauTypecheckOpts) - tempSeenTy.clear(); + if (FFlag::LuauCacheUnifyTableResults) + { + if (FFlag::LuauTypecheckOpts) + sharedState.tempSeenTy.clear(); - return occursCheck(seen_DEPRECATED, tempSeenTy, needle, haystack); + return occursCheck(seen_DEPRECATED, sharedState.tempSeenTy, needle, haystack); + } + else + { + if (FFlag::LuauTypecheckOpts) + tempSeenTy_DEPRECATED.clear(); + + return occursCheck(seen_DEPRECATED, tempSeenTy_DEPRECATED, needle, haystack); + } } void Unifier::occursCheck(std::unordered_set& seen_DEPRECATED, DenseHashSet& seen, TypeId needle, TypeId haystack) @@ -1851,10 +2107,20 @@ void Unifier::occursCheck(TypePackId needle, TypePackId haystack) { std::unordered_set seen_DEPRECATED; - if (FFlag::LuauTypecheckOpts) - tempSeenTp.clear(); + if (FFlag::LuauCacheUnifyTableResults) + { + if (FFlag::LuauTypecheckOpts) + sharedState.tempSeenTp.clear(); - return occursCheck(seen_DEPRECATED, tempSeenTp, needle, haystack); + return occursCheck(seen_DEPRECATED, sharedState.tempSeenTp, needle, haystack); + } + else + { + if (FFlag::LuauTypecheckOpts) + tempSeenTp_DEPRECATED.clear(); + + return occursCheck(seen_DEPRECATED, tempSeenTp_DEPRECATED, needle, haystack); + } } void Unifier::occursCheck(std::unordered_set& seen_DEPRECATED, DenseHashSet& seen, TypePackId needle, TypePackId haystack) @@ -1922,7 +2188,10 @@ void Unifier::occursCheck(std::unordered_set& seen_DEPRECATED, Dense Unifier Unifier::makeChildUnifier() { - return Unifier{types, mode, globalScope, log.seen, location, variance, iceHandler, counters_DEPRECATED, counters}; + if (FFlag::LuauShareTxnSeen) + return Unifier{types, mode, globalScope, log.sharedSeen, location, variance, sharedState, counters_DEPRECATED, counters}; + else + return Unifier{types, mode, globalScope, log.ownedSeen, location, variance, sharedState, counters_DEPRECATED, counters}; } bool Unifier::isNonstrictMode() const @@ -1940,12 +2209,12 @@ void Unifier::checkChildUnifierTypeMismatch(const ErrorVec& innerErrors, TypeId void Unifier::ice(const std::string& message, const Location& location) { - iceHandler->ice(message, location); + sharedState.iceHandler->ice(message, location); } void Unifier::ice(const std::string& message) { - iceHandler->ice(message); + sharedState.iceHandler->ice(message); } } // namespace Luau diff --git a/Ast/include/Luau/TimeTrace.h b/Ast/include/Luau/TimeTrace.h index 641dfd3c..503eca61 100644 --- a/Ast/include/Luau/TimeTrace.h +++ b/Ast/include/Luau/TimeTrace.h @@ -194,20 +194,20 @@ LUAU_NOINLINE std::pair createScopeDa } // namespace Luau // Regular scope -#define LUAU_TIMETRACE_SCOPE(name, category) \ +#define LUAU_TIMETRACE_SCOPE(name, category) \ static auto lttScopeStatic = Luau::TimeTrace::createScopeData(name, category); \ Luau::TimeTrace::Scope lttScope(lttScopeStatic.second, lttScopeStatic.first) // A scope without nested scopes that may be skipped if the time it took is less than the threshold -#define LUAU_TIMETRACE_OPTIONAL_TAIL_SCOPE(name, category, microsec) \ +#define LUAU_TIMETRACE_OPTIONAL_TAIL_SCOPE(name, category, microsec) \ static auto lttScopeStaticOptTail = Luau::TimeTrace::createScopeData(name, category); \ Luau::TimeTrace::OptionalTailScope lttScope(lttScopeStaticOptTail.second, lttScopeStaticOptTail.first, microsec) // Extra key/value data can be added to regular scopes -#define LUAU_TIMETRACE_ARGUMENT(name, value) \ - do \ - { \ - if (FFlag::DebugLuauTimeTracing) \ +#define LUAU_TIMETRACE_ARGUMENT(name, value) \ + do \ + { \ + if (FFlag::DebugLuauTimeTracing) \ lttScopeStatic.second.eventArgument(name, value); \ } while (false) @@ -216,8 +216,8 @@ LUAU_NOINLINE std::pair createScopeDa #define LUAU_TIMETRACE_SCOPE(name, category) #define LUAU_TIMETRACE_OPTIONAL_TAIL_SCOPE(name, category, microsec) #define LUAU_TIMETRACE_ARGUMENT(name, value) \ - do \ - { \ + do \ + { \ } while (false) #endif diff --git a/Ast/src/TimeTrace.cpp b/Ast/src/TimeTrace.cpp index e6aab20e..ded50e53 100644 --- a/Ast/src/TimeTrace.cpp +++ b/Ast/src/TimeTrace.cpp @@ -77,7 +77,10 @@ struct GlobalContext // Ideally we would want all ThreadContext destructors to run // But in VS, not all thread_local object instances are destroyed for (ThreadContext* context : threads) - context->flushEvents(); + { + if (!context->events.empty()) + context->flushEvents(); + } if (traceFile) fclose(traceFile); diff --git a/Makefile b/Makefile index dd63ffbe..ea416be4 100644 --- a/Makefile +++ b/Makefile @@ -1,4 +1,5 @@ # This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +.SUFFIXES: MAKEFLAGS+=-r -j8 COMMA=, diff --git a/Sources.cmake b/Sources.cmake index 83ed5230..c30cf77d 100644 --- a/Sources.cmake +++ b/Sources.cmake @@ -46,6 +46,7 @@ target_sources(Luau.Analysis PRIVATE Analysis/include/Luau/Module.h Analysis/include/Luau/ModuleResolver.h Analysis/include/Luau/Predicate.h + Analysis/include/Luau/Quantify.h Analysis/include/Luau/RecursionCounter.h Analysis/include/Luau/RequireTracer.h Analysis/include/Luau/Scope.h @@ -63,6 +64,7 @@ target_sources(Luau.Analysis PRIVATE Analysis/include/Luau/TypeVar.h Analysis/include/Luau/Unifiable.h Analysis/include/Luau/Unifier.h + Analysis/include/Luau/UnifierSharedState.h Analysis/include/Luau/Variant.h Analysis/include/Luau/VisitTypeVar.h @@ -77,6 +79,7 @@ target_sources(Luau.Analysis PRIVATE Analysis/src/Linter.cpp Analysis/src/Module.cpp Analysis/src/Predicate.cpp + Analysis/src/Quantify.cpp Analysis/src/RequireTracer.cpp Analysis/src/Scope.cpp Analysis/src/Substitution.cpp diff --git a/VM/src/lapi.cpp b/VM/src/lapi.cpp index 01315360..f2e97c66 100644 --- a/VM/src/lapi.cpp +++ b/VM/src/lapi.cpp @@ -13,8 +13,6 @@ #include -LUAU_FASTFLAG(LuauGcFullSkipInactiveThreads) - const char* lua_ident = "$Lua: Lua 5.1.4 Copyright (C) 1994-2008 Lua.org, PUC-Rio $\n" "$Authors: R. Ierusalimschy, L. H. de Figueiredo & W. Celes $\n" "$URL: www.lua.org $\n"; @@ -1153,7 +1151,7 @@ void* lua_newuserdatadtor(lua_State* L, size_t sz, void (*dtor)(void*)) luaC_checkGC(L); luaC_checkthreadsleep(L); Udata* u = luaS_newudata(L, sz + sizeof(dtor), UTAG_IDTOR); - memcpy(u->data + sz, &dtor, sizeof(dtor)); + memcpy(&u->data + sz, &dtor, sizeof(dtor)); setuvalue(L, L->top, u); api_incr_top(L); return u->data; diff --git a/VM/src/lcorolib.cpp b/VM/src/lcorolib.cpp index a2515ba0..9724c0e7 100644 --- a/VM/src/lcorolib.cpp +++ b/VM/src/lcorolib.cpp @@ -5,8 +5,6 @@ #include "lstate.h" #include "lvm.h" -LUAU_FASTFLAGVARIABLE(LuauPreferXpush, false) - #define CO_RUN 0 /* running */ #define CO_SUS 1 /* suspended */ #define CO_NOR 2 /* 'normal' (it resumed another coroutine) */ @@ -17,7 +15,7 @@ LUAU_FASTFLAGVARIABLE(LuauPreferXpush, false) static const char* const statnames[] = {"running", "suspended", "normal", "dead"}; -static int costatus(lua_State* L, lua_State* co) +static int auxstatus(lua_State* L, lua_State* co) { if (co == L) return CO_RUN; @@ -34,11 +32,11 @@ static int costatus(lua_State* L, lua_State* co) return CO_SUS; /* initial state */ } -static int luaB_costatus(lua_State* L) +static int costatus(lua_State* L) { lua_State* co = lua_tothread(L, 1); luaL_argexpected(L, co, 1, "thread"); - lua_pushstring(L, statnames[costatus(L, co)]); + lua_pushstring(L, statnames[auxstatus(L, co)]); return 1; } @@ -47,7 +45,7 @@ static int auxresume(lua_State* L, lua_State* co, int narg) // error handling for edge cases if (co->status != LUA_YIELD) { - int status = costatus(L, co); + int status = auxstatus(L, co); if (status != CO_SUS) { lua_pushfstring(L, "cannot resume %s coroutine", statnames[status]); @@ -115,7 +113,7 @@ static int auxresumecont(lua_State* L, lua_State* co) } } -static int luaB_coresumefinish(lua_State* L, int r) +static int coresumefinish(lua_State* L, int r) { if (r < 0) { @@ -131,7 +129,7 @@ static int luaB_coresumefinish(lua_State* L, int r) } } -static int luaB_coresumey(lua_State* L) +static int coresumey(lua_State* L) { lua_State* co = lua_tothread(L, 1); luaL_argexpected(L, co, 1, "thread"); @@ -141,10 +139,10 @@ static int luaB_coresumey(lua_State* L) if (r == CO_STATUS_BREAK) return interruptThread(L, co); - return luaB_coresumefinish(L, r); + return coresumefinish(L, r); } -static int luaB_coresumecont(lua_State* L, int status) +static int coresumecont(lua_State* L, int status) { lua_State* co = lua_tothread(L, 1); luaL_argexpected(L, co, 1, "thread"); @@ -155,10 +153,10 @@ static int luaB_coresumecont(lua_State* L, int status) int r = auxresumecont(L, co); - return luaB_coresumefinish(L, r); + return coresumefinish(L, r); } -static int luaB_auxwrapfinish(lua_State* L, int r) +static int auxwrapfinish(lua_State* L, int r) { if (r < 0) { @@ -173,7 +171,7 @@ static int luaB_auxwrapfinish(lua_State* L, int r) return r; } -static int luaB_auxwrapy(lua_State* L) +static int auxwrapy(lua_State* L) { lua_State* co = lua_tothread(L, lua_upvalueindex(1)); int narg = cast_int(L->top - L->base); @@ -182,10 +180,10 @@ static int luaB_auxwrapy(lua_State* L) if (r == CO_STATUS_BREAK) return interruptThread(L, co); - return luaB_auxwrapfinish(L, r); + return auxwrapfinish(L, r); } -static int luaB_auxwrapcont(lua_State* L, int status) +static int auxwrapcont(lua_State* L, int status) { lua_State* co = lua_tothread(L, lua_upvalueindex(1)); @@ -195,62 +193,52 @@ static int luaB_auxwrapcont(lua_State* L, int status) int r = auxresumecont(L, co); - return luaB_auxwrapfinish(L, r); + return auxwrapfinish(L, r); } -static int luaB_cocreate(lua_State* L) +static int cocreate(lua_State* L) { luaL_checktype(L, 1, LUA_TFUNCTION); lua_State* NL = lua_newthread(L); - - if (FFlag::LuauPreferXpush) - { - lua_xpush(L, NL, 1); // push function on top of NL - } - else - { - lua_pushvalue(L, 1); /* move function to top */ - lua_xmove(L, NL, 1); /* move function from L to NL */ - } - + lua_xpush(L, NL, 1); // push function on top of NL return 1; } -static int luaB_cowrap(lua_State* L) +static int cowrap(lua_State* L) { - luaB_cocreate(L); + cocreate(L); - lua_pushcfunction(L, luaB_auxwrapy, NULL, 1, luaB_auxwrapcont); + lua_pushcfunction(L, auxwrapy, NULL, 1, auxwrapcont); return 1; } -static int luaB_yield(lua_State* L) +static int coyield(lua_State* L) { int nres = cast_int(L->top - L->base); return lua_yield(L, nres); } -static int luaB_corunning(lua_State* L) +static int corunning(lua_State* L) { if (lua_pushthread(L)) lua_pushnil(L); /* main thread is not a coroutine */ return 1; } -static int luaB_yieldable(lua_State* L) +static int coyieldable(lua_State* L) { lua_pushboolean(L, lua_isyieldable(L)); return 1; } static const luaL_Reg co_funcs[] = { - {"create", luaB_cocreate}, - {"running", luaB_corunning}, - {"status", luaB_costatus}, - {"wrap", luaB_cowrap}, - {"yield", luaB_yield}, - {"isyieldable", luaB_yieldable}, + {"create", cocreate}, + {"running", corunning}, + {"status", costatus}, + {"wrap", cowrap}, + {"yield", coyield}, + {"isyieldable", coyieldable}, {NULL, NULL}, }; @@ -258,7 +246,7 @@ LUALIB_API int luaopen_coroutine(lua_State* L) { luaL_register(L, LUA_COLIBNAME, co_funcs); - lua_pushcfunction(L, luaB_coresumey, "resume", 0, luaB_coresumecont); + lua_pushcfunction(L, coresumey, "resume", 0, coresumecont); lua_setfield(L, -2, "resume"); return 1; diff --git a/VM/src/ldo.cpp b/VM/src/ldo.cpp index fb4978d5..328b47e6 100644 --- a/VM/src/ldo.cpp +++ b/VM/src/ldo.cpp @@ -18,6 +18,7 @@ #include LUAU_FASTFLAGVARIABLE(LuauExceptionMessageFix, false) +LUAU_FASTFLAGVARIABLE(LuauCcallRestoreFix, false) /* ** {====================================================== @@ -536,6 +537,12 @@ int luaD_pcall(lua_State* L, Pfunc func, void* u, ptrdiff_t old_top, ptrdiff_t e status = LUA_ERRERR; } + if (FFlag::LuauCcallRestoreFix) + { + // Restore nCcalls before calling the debugprotectederror callback which may rely on the proper value to have been restored. + L->nCcalls = oldnCcalls; + } + // an error occurred, check if we have a protected error callback if (L->global->cb.debugprotectederror) { @@ -549,7 +556,10 @@ int luaD_pcall(lua_State* L, Pfunc func, void* u, ptrdiff_t old_top, ptrdiff_t e StkId oldtop = restorestack(L, old_top); luaF_close(L, oldtop); /* close eventual pending closures */ seterrorobj(L, status, oldtop); - L->nCcalls = oldnCcalls; + if (!FFlag::LuauCcallRestoreFix) + { + L->nCcalls = oldnCcalls; + } L->ci = restoreci(L, old_ci); L->base = L->ci->base; restore_stack_limit(L); diff --git a/VM/src/lgc.cpp b/VM/src/lgc.cpp index 5de5d490..6553009f 100644 --- a/VM/src/lgc.cpp +++ b/VM/src/lgc.cpp @@ -12,11 +12,9 @@ #include #include -LUAU_FASTFLAGVARIABLE(LuauRescanGrayAgain, false) LUAU_FASTFLAGVARIABLE(LuauRescanGrayAgainForwardBarrier, false) -LUAU_FASTFLAGVARIABLE(LuauGcFullSkipInactiveThreads, false) -LUAU_FASTFLAGVARIABLE(LuauShrinkWeakTables, false) LUAU_FASTFLAGVARIABLE(LuauConsolidatedStep, false) +LUAU_FASTFLAGVARIABLE(LuauSeparateAtomic, false) LUAU_FASTFLAG(LuauArrayBoundary) @@ -66,13 +64,18 @@ static void recordGcStateTime(global_State* g, int startgcstate, double seconds, g->gcstats.currcycle.marktime += seconds; // atomic step had to be performed during the switch and it's tracked separately - if (g->gcstate == GCSsweepstring) + if (!FFlag::LuauSeparateAtomic && g->gcstate == GCSsweepstring) g->gcstats.currcycle.marktime -= g->gcstats.currcycle.atomictime; break; + case GCSatomic: + g->gcstats.currcycle.atomictime += seconds; + break; case GCSsweepstring: case GCSsweep: g->gcstats.currcycle.sweeptime += seconds; break; + default: + LUAU_ASSERT(!"Unexpected GC state"); } if (assist) @@ -183,33 +186,15 @@ static int traversetable(global_State* g, Table* h) if (h->metatable) markobject(g, cast_to(Table*, h->metatable)); - if (FFlag::LuauShrinkWeakTables) + /* is there a weak mode? */ + if (const char* modev = gettablemode(g, h)) { - /* is there a weak mode? */ - if (const char* modev = gettablemode(g, h)) - { - weakkey = (strchr(modev, 'k') != NULL); - weakvalue = (strchr(modev, 'v') != NULL); - if (weakkey || weakvalue) - { /* is really weak? */ - h->gclist = g->weak; /* must be cleared after GC, ... */ - g->weak = obj2gco(h); /* ... so put in the appropriate list */ - } - } - } - else - { - const TValue* mode = gfasttm(g, h->metatable, TM_MODE); - if (mode && ttisstring(mode)) - { /* is there a weak mode? */ - const char* modev = svalue(mode); - weakkey = (strchr(modev, 'k') != NULL); - weakvalue = (strchr(modev, 'v') != NULL); - if (weakkey || weakvalue) - { /* is really weak? */ - h->gclist = g->weak; /* must be cleared after GC, ... */ - g->weak = obj2gco(h); /* ... so put in the appropriate list */ - } + weakkey = (strchr(modev, 'k') != NULL); + weakvalue = (strchr(modev, 'v') != NULL); + if (weakkey || weakvalue) + { /* is really weak? */ + h->gclist = g->weak; /* must be cleared after GC, ... */ + g->weak = obj2gco(h); /* ... so put in the appropriate list */ } } @@ -297,7 +282,7 @@ static void traversestack(global_State* g, lua_State* l, bool clearstack) for (StkId o = l->stack; o < l->top; o++) markvalue(g, o); /* final traversal? */ - if (g->gcstate == GCSatomic || (FFlag::LuauGcFullSkipInactiveThreads && clearstack)) + if (g->gcstate == GCSatomic || clearstack) { StkId stack_end = l->stack + l->stacksize; for (StkId o = l->top; o < stack_end; o++) /* clear not-marked stack slice */ @@ -336,28 +321,16 @@ static size_t propagatemark(global_State* g) lua_State* th = gco2th(o); g->gray = th->gclist; - if (FFlag::LuauGcFullSkipInactiveThreads) + LUAU_ASSERT(!luaC_threadsleeping(th)); + + // threads that are executing and the main thread are not deactivated + bool active = luaC_threadactive(th) || th == th->global->mainthread; + + if (!active && g->gcstate == GCSpropagate) { - LUAU_ASSERT(!luaC_threadsleeping(th)); + traversestack(g, th, /* clearstack= */ true); - // threads that are executing and the main thread are not deactivated - bool active = luaC_threadactive(th) || th == th->global->mainthread; - - if (!active && g->gcstate == GCSpropagate) - { - traversestack(g, th, /* clearstack= */ true); - - l_setbit(th->stackstate, THREAD_SLEEPINGBIT); - } - else - { - th->gclist = g->grayagain; - g->grayagain = o; - - black2gray(o); - - traversestack(g, th, /* clearstack= */ false); - } + l_setbit(th->stackstate, THREAD_SLEEPINGBIT); } else { @@ -385,12 +358,14 @@ static size_t propagatemark(global_State* g) } } -static void propagateall(global_State* g) +static size_t propagateall(global_State* g) { + size_t work = 0; while (g->gray) { - propagatemark(g); + work += propagatemark(g); } + return work; } /* @@ -415,11 +390,14 @@ static int isobjcleared(GCObject* o) /* ** clear collected entries from weaktables */ -static void cleartable(lua_State* L, GCObject* l) +static size_t cleartable(lua_State* L, GCObject* l) { + size_t work = 0; while (l) { Table* h = gco2h(l); + work += sizeof(Table) + sizeof(TValue) * h->sizearray + sizeof(LuaNode) * sizenode(h); + int i = h->sizearray; while (i--) { @@ -433,50 +411,36 @@ static void cleartable(lua_State* L, GCObject* l) { LuaNode* n = gnode(h, i); - if (FFlag::LuauShrinkWeakTables) + // non-empty entry? + if (!ttisnil(gval(n))) { - // non-empty entry? - if (!ttisnil(gval(n))) - { - // can we clear key or value? - if (iscleared(gkey(n)) || iscleared(gval(n))) - { - setnilvalue(gval(n)); /* remove value ... */ - removeentry(n); /* remove entry from table */ - } - else - { - activevalues++; - } - } - } - else - { - if (!ttisnil(gval(n)) && /* non-empty entry? */ - (iscleared(gkey(n)) || iscleared(gval(n)))) + // can we clear key or value? + if (iscleared(gkey(n)) || iscleared(gval(n))) { setnilvalue(gval(n)); /* remove value ... */ removeentry(n); /* remove entry from table */ } + else + { + activevalues++; + } } } - if (FFlag::LuauShrinkWeakTables) + if (const char* modev = gettablemode(L->global, h)) { - if (const char* modev = gettablemode(L->global, h)) + // are we allowed to shrink this weak table? + if (strchr(modev, 's')) { - // are we allowed to shrink this weak table? - if (strchr(modev, 's')) - { - // shrink at 37.5% occupancy - if (activevalues < sizenode(h) * 3 / 8) - luaH_resizehash(L, h, activevalues); - } + // shrink at 37.5% occupancy + if (activevalues < sizenode(h) * 3 / 8) + luaH_resizehash(L, h, activevalues); } } l = h->gclist; } + return work; } static void shrinkstack(lua_State* L) @@ -655,37 +619,49 @@ static void markroot(lua_State* L) g->gcstate = GCSpropagate; } -static void remarkupvals(global_State* g) +static size_t remarkupvals(global_State* g) { - UpVal* uv; - for (uv = g->uvhead.u.l.next; uv != &g->uvhead; uv = uv->u.l.next) + size_t work = 0; + for (UpVal* uv = g->uvhead.u.l.next; uv != &g->uvhead; uv = uv->u.l.next) { + work += sizeof(UpVal); LUAU_ASSERT(uv->u.l.next->u.l.prev == uv && uv->u.l.prev->u.l.next == uv); if (isgray(obj2gco(uv))) markvalue(g, uv->v); } + return work; } -static void atomic(lua_State* L) +static size_t atomic(lua_State* L) { global_State* g = L->global; - g->gcstate = GCSatomic; + size_t work = 0; + + if (FFlag::LuauSeparateAtomic) + { + LUAU_ASSERT(g->gcstate == GCSatomic); + } + else + { + g->gcstate = GCSatomic; + } + /* remark occasional upvalues of (maybe) dead threads */ - remarkupvals(g); + work += remarkupvals(g); /* traverse objects caught by write barrier and by 'remarkupvals' */ - propagateall(g); + work += propagateall(g); /* remark weak tables */ g->gray = g->weak; g->weak = NULL; LUAU_ASSERT(!iswhite(obj2gco(g->mainthread))); markobject(g, L); /* mark running thread */ markmt(g); /* mark basic metatables (again) */ - propagateall(g); + work += propagateall(g); /* remark gray again */ g->gray = g->grayagain; g->grayagain = NULL; - propagateall(g); - cleartable(L, g->weak); /* remove collected objects from weak tables */ + work += propagateall(g); + work += cleartable(L, g->weak); /* remove collected objects from weak tables */ g->weak = NULL; /* flip current white */ g->currentwhite = cast_byte(otherwhite(g)); @@ -693,7 +669,12 @@ static void atomic(lua_State* L) g->sweepgc = &g->rootgc; g->gcstate = GCSsweepstring; - GC_INTERRUPT(GCSatomic); + if (!FFlag::LuauSeparateAtomic) + { + GC_INTERRUPT(GCSatomic); + } + + return work; } static size_t singlestep(lua_State* L) @@ -705,46 +686,24 @@ static size_t singlestep(lua_State* L) case GCSpause: { markroot(L); /* start a new collection */ + LUAU_ASSERT(g->gcstate == GCSpropagate); break; } case GCSpropagate: { - if (FFlag::LuauRescanGrayAgain) + if (g->gray) { - if (g->gray) - { - g->gcstats.currcycle.markitems++; + g->gcstats.currcycle.markitems++; - cost = propagatemark(g); - } - else - { - // perform one iteration over 'gray again' list - g->gray = g->grayagain; - g->grayagain = NULL; - - g->gcstate = GCSpropagateagain; - } + cost = propagatemark(g); } else { - if (g->gray) - { - g->gcstats.currcycle.markitems++; + // perform one iteration over 'gray again' list + g->gray = g->grayagain; + g->grayagain = NULL; - cost = propagatemark(g); - } - else /* no more `gray' objects */ - { - double starttimestamp = lua_clock(); - - g->gcstats.currcycle.atomicstarttimestamp = starttimestamp; - g->gcstats.currcycle.atomicstarttotalsizebytes = g->totalbytes; - - atomic(L); /* finish mark phase */ - - g->gcstats.currcycle.atomictime += lua_clock() - starttimestamp; - } + g->gcstate = GCSpropagateagain; } break; } @@ -758,17 +717,34 @@ static size_t singlestep(lua_State* L) } else /* no more `gray' objects */ { - double starttimestamp = lua_clock(); + if (FFlag::LuauSeparateAtomic) + { + g->gcstate = GCSatomic; + } + else + { + double starttimestamp = lua_clock(); - g->gcstats.currcycle.atomicstarttimestamp = starttimestamp; - g->gcstats.currcycle.atomicstarttotalsizebytes = g->totalbytes; + g->gcstats.currcycle.atomicstarttimestamp = starttimestamp; + g->gcstats.currcycle.atomicstarttotalsizebytes = g->totalbytes; - atomic(L); /* finish mark phase */ + atomic(L); /* finish mark phase */ + LUAU_ASSERT(g->gcstate == GCSsweepstring); - g->gcstats.currcycle.atomictime += lua_clock() - starttimestamp; + g->gcstats.currcycle.atomictime += lua_clock() - starttimestamp; + } } break; } + case GCSatomic: + { + g->gcstats.currcycle.atomicstarttimestamp = lua_clock(); + g->gcstats.currcycle.atomicstarttotalsizebytes = g->totalbytes; + + cost = atomic(L); /* finish mark phase */ + LUAU_ASSERT(g->gcstate == GCSsweepstring); + break; + } case GCSsweepstring: { size_t traversedcount = 0; @@ -806,7 +782,7 @@ static size_t singlestep(lua_State* L) break; } default: - LUAU_ASSERT(0); + LUAU_ASSERT(!"Unexpected GC state"); } return cost; @@ -821,48 +797,25 @@ static size_t gcstep(lua_State* L, size_t limit) case GCSpause: { markroot(L); /* start a new collection */ + LUAU_ASSERT(g->gcstate == GCSpropagate); break; } case GCSpropagate: { - if (FFlag::LuauRescanGrayAgain) + while (g->gray && cost < limit) { - while (g->gray && cost < limit) - { - g->gcstats.currcycle.markitems++; + g->gcstats.currcycle.markitems++; - cost += propagatemark(g); - } - - if (!g->gray) - { - // perform one iteration over 'gray again' list - g->gray = g->grayagain; - g->grayagain = NULL; - - g->gcstate = GCSpropagateagain; - } + cost += propagatemark(g); } - else + + if (!g->gray) { - while (g->gray && cost < limit) - { - g->gcstats.currcycle.markitems++; + // perform one iteration over 'gray again' list + g->gray = g->grayagain; + g->grayagain = NULL; - cost += propagatemark(g); - } - - if (!g->gray) /* no more `gray' objects */ - { - double starttimestamp = lua_clock(); - - g->gcstats.currcycle.atomicstarttimestamp = starttimestamp; - g->gcstats.currcycle.atomicstarttotalsizebytes = g->totalbytes; - - atomic(L); /* finish mark phase */ - - g->gcstats.currcycle.atomictime += lua_clock() - starttimestamp; - } + g->gcstate = GCSpropagateagain; } break; } @@ -877,17 +830,34 @@ static size_t gcstep(lua_State* L, size_t limit) if (!g->gray) /* no more `gray' objects */ { - double starttimestamp = lua_clock(); + if (FFlag::LuauSeparateAtomic) + { + g->gcstate = GCSatomic; + } + else + { + double starttimestamp = lua_clock(); - g->gcstats.currcycle.atomicstarttimestamp = starttimestamp; - g->gcstats.currcycle.atomicstarttotalsizebytes = g->totalbytes; + g->gcstats.currcycle.atomicstarttimestamp = starttimestamp; + g->gcstats.currcycle.atomicstarttotalsizebytes = g->totalbytes; - atomic(L); /* finish mark phase */ + atomic(L); /* finish mark phase */ + LUAU_ASSERT(g->gcstate == GCSsweepstring); - g->gcstats.currcycle.atomictime += lua_clock() - starttimestamp; + g->gcstats.currcycle.atomictime += lua_clock() - starttimestamp; + } } break; } + case GCSatomic: + { + g->gcstats.currcycle.atomicstarttimestamp = lua_clock(); + g->gcstats.currcycle.atomicstarttotalsizebytes = g->totalbytes; + + cost = atomic(L); /* finish mark phase */ + LUAU_ASSERT(g->gcstate == GCSsweepstring); + break; + } case GCSsweepstring: { while (g->sweepstrgc < g->strt.size && cost < limit) @@ -934,7 +904,7 @@ static size_t gcstep(lua_State* L, size_t limit) break; } default: - LUAU_ASSERT(0); + LUAU_ASSERT(!"Unexpected GC state"); } return cost; } @@ -1084,7 +1054,7 @@ void luaC_fullgc(lua_State* L) if (g->gcstate == GCSpause) startGcCycleStats(g); - if (g->gcstate <= GCSpropagateagain) + if (g->gcstate <= (FFlag::LuauSeparateAtomic ? GCSatomic : GCSpropagateagain)) { /* reset sweep marks to sweep all elements (returning them to white) */ g->sweepstrgc = 0; @@ -1095,7 +1065,7 @@ void luaC_fullgc(lua_State* L) g->weak = NULL; g->gcstate = GCSsweepstring; } - LUAU_ASSERT(g->gcstate != GCSpause && g->gcstate != GCSpropagate && g->gcstate != GCSpropagateagain); + LUAU_ASSERT(g->gcstate == GCSsweepstring || g->gcstate == GCSsweep); /* finish any pending sweep phase */ while (g->gcstate != GCSpause) { @@ -1143,14 +1113,11 @@ void luaC_fullgc(lua_State* L) void luaC_barrierupval(lua_State* L, GCObject* v) { - if (FFlag::LuauGcFullSkipInactiveThreads) - { - global_State* g = L->global; - LUAU_ASSERT(iswhite(v) && !isdead(g, v)); + global_State* g = L->global; + LUAU_ASSERT(iswhite(v) && !isdead(g, v)); - if (keepinvariant(g)) - reallymarkobject(g, v); - } + if (keepinvariant(g)) + reallymarkobject(g, v); } void luaC_barrierf(lua_State* L, GCObject* o, GCObject* v) @@ -1778,7 +1745,7 @@ int64_t luaC_allocationrate(lua_State* L) global_State* g = L->global; const double durationthreshold = 1e-3; // avoid measuring intervals smaller than 1ms - if (g->gcstate <= GCSpropagateagain) + if (g->gcstate <= (FFlag::LuauSeparateAtomic ? GCSatomic : GCSpropagateagain)) { double duration = lua_clock() - g->gcstats.lastcycle.endtimestamp; diff --git a/VM/src/lgc.h b/VM/src/lgc.h index 6fddd663..f434e506 100644 --- a/VM/src/lgc.h +++ b/VM/src/lgc.h @@ -6,8 +6,6 @@ #include "lobject.h" #include "lstate.h" -LUAU_FASTFLAG(LuauGcFullSkipInactiveThreads) - /* ** Possible states of the Garbage Collector */ @@ -25,7 +23,7 @@ LUAU_FASTFLAG(LuauGcFullSkipInactiveThreads) ** still-black objects. The invariant is restored when sweep ends and ** all objects are white again. */ -#define keepinvariant(g) ((g)->gcstate == GCSpropagate || (g)->gcstate == GCSpropagateagain) +#define keepinvariant(g) ((g)->gcstate == GCSpropagate || (g)->gcstate == GCSpropagateagain || (g)->gcstate == GCSatomic) /* ** some useful bit tricks @@ -147,4 +145,4 @@ LUAI_FUNC void luaC_validate(lua_State* L); LUAI_FUNC void luaC_dump(lua_State* L, void* file, const char* (*categoryName)(lua_State* L, uint8_t memcat)); LUAI_FUNC int64_t luaC_allocationrate(lua_State* L); LUAI_FUNC void luaC_wakethread(lua_State* L); -LUAI_FUNC const char* luaC_statename(int state); \ No newline at end of file +LUAI_FUNC const char* luaC_statename(int state); diff --git a/VM/src/lmem.cpp b/VM/src/lmem.cpp index 2759f3b8..d8b265cb 100644 --- a/VM/src/lmem.cpp +++ b/VM/src/lmem.cpp @@ -199,7 +199,7 @@ static void* luaM_newblock(lua_State* L, int sizeClass) if (page->freeNext >= 0) { - block = page->data + page->freeNext; + block = &page->data + page->freeNext; ASAN_UNPOISON_MEMORY_REGION(block, page->blockSize); page->freeNext -= page->blockSize; diff --git a/VM/src/lstring.cpp b/VM/src/lstring.cpp index d77e17c9..18ee1cda 100644 --- a/VM/src/lstring.cpp +++ b/VM/src/lstring.cpp @@ -226,7 +226,7 @@ void luaS_freeudata(lua_State* L, Udata* u) void (*dtor)(void*) = nullptr; if (u->tag == UTAG_IDTOR) - memcpy(&dtor, u->data + u->len - sizeof(dtor), sizeof(dtor)); + memcpy(&dtor, &u->data + u->len - sizeof(dtor), sizeof(dtor)); else if (u->tag) dtor = L->global->udatagc[u->tag]; diff --git a/VM/src/lvmload.cpp b/VM/src/lvmload.cpp index b932a85b..a168b652 100644 --- a/VM/src/lvmload.cpp +++ b/VM/src/lvmload.cpp @@ -13,7 +13,7 @@ #include // TODO: RAII deallocation doesn't work for longjmp builds if a memory error happens -template +template struct TempBuffer { lua_State* L; @@ -346,6 +346,8 @@ int luau_load(lua_State* L, const char* chunkname, const char* data, size_t size uint32_t mainid = readVarInt(data, size, offset); Proto* main = protos[mainid]; + luaC_checkthreadsleep(L); + Closure* cl = luaF_newLclosure(L, 0, envt, main); setclvalue(L, L->top, cl); incr_top(L); diff --git a/bench/tests/chess.lua b/bench/tests/chess.lua new file mode 100644 index 00000000..87b9abfd --- /dev/null +++ b/bench/tests/chess.lua @@ -0,0 +1,849 @@ + +local bench = script and require(script.Parent.bench_support) or require("bench_support") + +local RANKS = "12345678" +local FILES = "abcdefgh" +local PieceSymbols = "PpRrNnBbQqKk" +local UnicodePieces = {"♙", "♟", "♖", "♜", "♘", "♞", "♗", "♝", "♕", "♛", "♔", "♚"} +local StartingFen = "rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - 0 1" + +-- +-- Lua 5.2 Compat +-- + +if not table.create then + function table.create(n, v) + local result = {} + for i=1,n do result[i] = v end + return result + end +end + +if not table.move then + function table.move(a, from, to, start, target) + local dx = start - from + for i=from,to do + target[i+dx] = a[i] + end + end +end + + +-- +-- Utils +-- + +local function square(s) + return RANKS:find(s:sub(2,2)) * 8 + FILES:find(s:sub(1,1)) - 9 +end + +local function squareName(n) + local file = n % 8 + local rank = (n-file)/8 + return FILES:sub(file+1,file+1) .. RANKS:sub(rank+1,rank+1) +end + +local function moveName(v ) + local from = bit32.extract(v, 6, 6) + local to = bit32.extract(v, 0, 6) + local piece = bit32.extract(v, 20, 4) + local captured = bit32.extract(v, 25, 4) + + local move = PieceSymbols:sub(piece,piece) .. ' ' .. squareName(from) .. (captured ~= 0 and 'x' or '-') .. squareName(to) + + if bit32.extract(v,14) == 1 then + if to > from then + return "O-O" + else + return "O-O-O" + end + end + + local promote = bit32.extract(v,15,4) + if promote ~= 0 then + move = move .. "=" .. PieceSymbols:sub(promote,promote) + end + return move +end + +local function ucimove(m) + local mm = squareName(bit32.extract(m, 6, 6)) .. squareName(bit32.extract(m, 0, 6)) + local promote = bit32.extract(m,15,4) + if promote > 0 then + mm = mm .. PieceSymbols:sub(promote,promote):lower() + end + return mm +end + +local _utils = {squareName, moveName} + +-- +-- Bitboards +-- + +local Bitboard = {} + + +function Bitboard:toString() + local out = {} + + local src = self.h + for x=7,0,-1 do + table.insert(out, RANKS:sub(x+1,x+1)) + table.insert(out, " ") + local bit = bit32.lshift(1,(x%4) * 8) + for x=0,7 do + if bit32.band(src, bit) ~= 0 then + table.insert(out, "x ") + else + table.insert(out, "- ") + end + bit = bit32.lshift(bit, 1) + end + if x == 4 then + src = self.l + end + table.insert(out, "\n") + end + table.insert(out, ' ' .. FILES:gsub('.', '%1 ') .. '\n') + table.insert(out, '#: ' .. self:popcnt() .. "\tl:" .. self.l .. "\th:" .. self.h) + return table.concat(out) +end + + +function Bitboard.from(l ,h ) + return setmetatable({l=l, h=h}, Bitboard) +end + +Bitboard.zero = Bitboard.from(0,0) +Bitboard.full = Bitboard.from(0xFFFFFFFF, 0xFFFFFFFF) + +local Rank1 = Bitboard.from(0x000000FF, 0) +local Rank3 = Bitboard.from(0x00FF0000, 0) +local Rank6 = Bitboard.from(0, 0x0000FF00) +local Rank8 = Bitboard.from(0, 0xFF000000) +local FileA = Bitboard.from(0x01010101, 0x01010101) +local FileB = Bitboard.from(0x02020202, 0x02020202) +local FileC = Bitboard.from(0x04040404, 0x04040404) +local FileD = Bitboard.from(0x08080808, 0x08080808) +local FileE = Bitboard.from(0x10101010, 0x10101010) +local FileF = Bitboard.from(0x20202020, 0x20202020) +local FileG = Bitboard.from(0x40404040, 0x40404040) +local FileH = Bitboard.from(0x80808080, 0x80808080) + +local _Files = {FileA, FileB, FileC, FileD, FileE, FileF, FileG, FileH} + +-- These masks are filled out below for all files +local RightMasks = {FileH} +local LeftMasks = {FileA} + + + +local function popcnt32(i) + i = i - bit32.band(bit32.rshift(i,1), 0x55555555) + i = bit32.band(i, 0x33333333) + bit32.band(bit32.rshift(i,2), 0x33333333) + return bit32.rshift(bit32.band(i + bit32.rshift(i,4), 0x0F0F0F0F) * 0x01010101, 24) +end + +function Bitboard:up() + return self:lshift(8) +end + +function Bitboard:down() + return self:rshift(8) +end + +function Bitboard:right() + return self:band(FileH:inverse()):lshift(1) +end + +function Bitboard:left() + return self:band(FileA:inverse()):rshift(1) +end + +function Bitboard:move(x,y) + local out = self + + if x < 0 then out = out:bandnot(RightMasks[-x]):lshift(-x) end + if x > 0 then out = out:bandnot(LeftMasks[x]):rshift(x) end + + if y < 0 then out = out:rshift(-8 * y) end + if y > 0 then out = out:lshift(8 * y) end + return out +end + + +function Bitboard:popcnt() + return popcnt32(self.l) + popcnt32(self.h) +end + +function Bitboard:band(other ) + return Bitboard.from(bit32.band(self.l,other.l), bit32.band(self.h, other.h)) +end + +function Bitboard:bandnot(other ) + return Bitboard.from(bit32.band(self.l,bit32.bnot(other.l)), bit32.band(self.h, bit32.bnot(other.h))) +end + +function Bitboard:bandempty(other ) + return bit32.band(self.l,other.l) == 0 and bit32.band(self.h, other.h) == 0 +end + +function Bitboard:bor(other ) + return Bitboard.from(bit32.bor(self.l,other.l), bit32.bor(self.h, other.h)) +end + +function Bitboard:bxor(other ) + return Bitboard.from(bit32.bxor(self.l,other.l), bit32.bxor(self.h, other.h)) +end + +function Bitboard:inverse() + return Bitboard.from(bit32.bxor(self.l,0xFFFFFFFF), bit32.bxor(self.h, 0xFFFFFFFF)) +end + +function Bitboard:empty() + return self.h == 0 and self.l == 0 +end + +function Bitboard:ctz() + local target = self.l + local offset = 0 + local result = 0 + + if target == 0 then + target = self.h + result = 32 + end + + if target == 0 then + return 64 + end + + while bit32.extract(target, offset) == 0 do + offset = offset + 1 + end + + return result + offset +end + +function Bitboard:ctzafter(start) + start = start + 1 + if start < 32 then + for i=start,31 do + if bit32.extract(self.l, i) == 1 then return i end + end + end + for i=math.max(32,start),63 do + if bit32.extract(self.h, i-32) == 1 then return i end + end + return 64 +end + + +function Bitboard:lshift(amt) + assert(amt >= 0) + if amt == 0 then return self end + + if amt > 31 then + return Bitboard.from(0, bit32.lshift(self.l, amt-31)) + end + + local l = bit32.lshift(self.l, amt) + local h = bit32.bor( + bit32.lshift(self.h, amt), + bit32.extract(self.l, 32-amt, amt) + ) + return Bitboard.from(l, h) +end + +function Bitboard:rshift(amt) + assert(amt >= 0) + if amt == 0 then return self end + local h = bit32.rshift(self.h, amt) + local l = bit32.bor( + bit32.rshift(self.l, amt), + bit32.lshift(bit32.extract(self.h, 0, amt), 32-amt) + ) + return Bitboard.from(l, h) +end + +function Bitboard:index(i) + if i > 31 then + return bit32.extract(self.h, i - 32) + else + return bit32.extract(self.l, i) + end +end + +function Bitboard:set(i , v) + if i > 31 then + return Bitboard.from(self.l, bit32.replace(self.h, v, i - 32)) + else + return Bitboard.from(bit32.replace(self.l, v, i), self.h) + end +end + +function Bitboard:isolate(i) + return self:band(Bitboard.some(i)) +end + +function Bitboard.some(idx ) + return Bitboard.zero:set(idx, 1) +end + +Bitboard.__index = Bitboard +Bitboard.__tostring = Bitboard.toString + +for i=2,8 do + RightMasks[i] = RightMasks[i-1]:rshift(1):bor(FileH) + LeftMasks[i] = LeftMasks[i-1]:lshift(1):bor(FileA) +end +-- +-- Board +-- + +local Board = {} + + +function Board.new() + local boards = table.create(12, Bitboard.zero) + boards.ocupied = Bitboard.zero + boards.white = Bitboard.zero + boards.black = Bitboard.zero + boards.unocupied = Bitboard.full + boards.ep = Bitboard.zero + boards.castle = Bitboard.zero + boards.toMove = 1 + boards.hm = 0 + boards.moves = 0 + boards.material = 0 + + return setmetatable(boards, Board) +end + +function Board.fromFen(fen ) + local b = Board.new() + local i = 0 + local rank = 7 + local file = 0 + + while true do + i = i + 1 + local p = fen:sub(i,i) + if p == '/' then + rank = rank - 1 + file = 0 + elseif tonumber(p) ~= nil then + file = file + tonumber(p) + else + local pidx = PieceSymbols:find(p) + if pidx == nil then break end + b[pidx] = b[pidx]:set(rank*8+file, 1) + file = file + 1 + end + end + + + local move, castle, ep, hm, m = string.match(fen, "^ ([bw]) ([KQkq-]*) ([a-h-][0-9]?) (%d*) (%d*)", i) + if move == nil then print(fen:sub(i)) end + b.toMove = move == 'w' and 1 or 2 + + if ep ~= "-" then + b.ep = Bitboard.some(square(ep)) + end + + if castle ~= "-" then + local oo = Bitboard.zero + if castle:find("K") then + oo = oo:set(7, 1) + end + if castle:find("Q") then + oo = oo:set(0, 1) + end + if castle:find("k") then + oo = oo:set(63, 1) + end + if castle:find("q") then + oo = oo:set(56, 1) + end + + b.castle = oo + end + + b.hm = hm + b.moves = m + + b:updateCache() + return b + +end + +function Board:index(idx ) + if self.white:index(idx) == 1 then + for p=1,12,2 do + if self[p]:index(idx) == 1 then + return p + end + end + else + for p=2,12,2 do + if self[p]:index(idx) == 1 then + return p + end + end + end + + return 0 +end + +function Board:updateCache() + for i=1,11,2 do + self.white = self.white:bor(self[i]) + self.black = self.black:bor(self[i+1]) + end + + self.ocupied = self.black:bor(self.white) + self.unocupied = self.ocupied:inverse() + self.material = + 100*self[1]:popcnt() - 100*self[2]:popcnt() + + 500*self[3]:popcnt() - 500*self[4]:popcnt() + + 300*self[5]:popcnt() - 300*self[6]:popcnt() + + 300*self[7]:popcnt() - 300*self[8]:popcnt() + + 900*self[9]:popcnt() - 900*self[10]:popcnt() + +end + +function Board:fen() + local out = {} + local s = 0 + local idx = 56 + for i=0,63 do + if i % 8 == 0 and i > 0 then + idx = idx - 16 + if s > 0 then + table.insert(out, '' .. s) + s = 0 + end + table.insert(out, '/') + end + local p = self:index(idx) + if p == 0 then + s = s + 1 + else + if s > 0 then + table.insert(out, '' .. s) + s = 0 + end + table.insert(out, PieceSymbols:sub(p,p)) + end + + idx = idx + 1 + end + if s > 0 then + table.insert(out, '' .. s) + end + + table.insert(out, self.toMove == 1 and ' w ' or ' b ') + if self.castle:empty() then + table.insert(out, '-') + else + if self.castle:index(7) == 1 then table.insert(out, 'K') end + if self.castle:index(0) == 1 then table.insert(out, 'Q') end + if self.castle:index(63) == 1 then table.insert(out, 'k') end + if self.castle:index(56) == 1 then table.insert(out, 'q') end + end + + table.insert(out, ' ') + if self.ep:empty() then + table.insert(out, '-') + else + table.insert(out, squareName(self.ep:ctz())) + end + + table.insert(out, ' ' .. self.hm) + table.insert(out, ' ' .. self.moves) + + return table.concat(out) +end + +function Board:pmoves(idx) + return self:generate(idx) +end + +function Board:pcaptures(idx) + return self:generate(idx):band(self.ocupied) +end + +local ROOK_SLIDES = {{1,0}, {-1,0}, {0,1}, {0,-1}} +local BISHOP_SLIDES = {{1,1}, {-1,1}, {1,-1}, {-1,-1}} +local QUEEN_SLIDES = {{1,0}, {-1,0}, {0,1}, {0,-1}, {1,1}, {-1,1}, {1,-1}, {-1,-1}} +local KNIGHT_MOVES = {{2,1}, {2,-1}, {-2,1}, {-2,-1}, {1,2}, {1,-2}, {-1,2}, {-1,-2}} + +function Board:generate(idx) + local piece = self:index(idx) + local r = Bitboard.some(idx) + local out = Bitboard.zero + local type = bit32.rshift(piece - 1, 1) + local cancapture = piece % 2 == 1 and self.black or self.white + + if piece == 0 then return Bitboard.zero end + + if type == 0 then + -- Pawn + local d = -(piece*2 - 3) + local movetwo = piece == 1 and Rank3 or Rank6 + + out = out:bor(r:move(0,d):band(self.unocupied)) + out = out:bor(out:band(movetwo):move(0,d):band(self.unocupied)) + + local captures = r:move(0,d) + captures = captures:right():bor(captures:left()) + + if not captures:bandempty(self.ep) then + out = out:bor(self.ep) + end + + captures = captures:band(cancapture) + out = out:bor(captures) + + return out + elseif type == 5 then + -- King + for x=-1,1,1 do + for y = -1,1,1 do + local w = r:move(x,y) + if self.ocupied:bandempty(w) then + out = out:bor(w) + else + if not cancapture:bandempty(w) then + out = out:bor(w) + end + end + end + end + elseif type == 2 then + -- Knight + for _,j in ipairs(KNIGHT_MOVES) do + local w = r:move(j[1],j[2]) + + if self.ocupied:bandempty(w) then + out = out:bor(w) + else + if not cancapture:bandempty(w) then + out = out:bor(w) + end + end + end + else + -- Sliders (Rook, Bishop, Queen) + local slides + if type == 1 then + slides = ROOK_SLIDES + elseif type == 3 then + slides = BISHOP_SLIDES + else + slides = QUEEN_SLIDES + end + + for _, op in ipairs(slides) do + local w = r + for i=1,7 do + w = w:move(op[1], op[2]) + if w:empty() then break end + + if self.ocupied:bandempty(w) then + out = out:bor(w) + else + if not cancapture:bandempty(w) then + out = out:bor(w) + end + break + end + end + end + end + + + return out +end + +-- 0-5 - From Square +-- 6-11 - To Square +-- 12 - is Check +-- 13 - Is EnPassent +-- 14 - Is Castle +-- 15-19 - Promotion Piece +-- 20-24 - Moved Pice +-- 25-29 - Captured Piece + + +function Board:toString(mark ) + local out = {} + for x=8,1,-1 do + table.insert(out, RANKS:sub(x,x) .. " ") + + for y=1,8 do + local n = 8*x+y-9 + local i = self:index(n) + if i == 0 then + table.insert(out, '-') + else + -- out = out .. PieceSymbols:sub(i,i) + table.insert(out, UnicodePieces[i]) + end + if mark ~= nil and mark:index(n) ~= 0 then + table.insert(out, ')') + elseif mark ~= nil and n < 63 and y < 8 and mark:index(n+1) ~= 0 then + table.insert(out, '(') + else + table.insert(out, ' ') + end + end + + table.insert(out, "\n") + end + table.insert(out, ' ' .. FILES:gsub('.', '%1 ') .. '\n') + table.insert(out, (self.toMove == 1 and "White" or "Black") .. ' e:' .. (self.material/100) .. "\n") + return table.concat(out) +end + +function Board:moveList() + local tm = self.toMove == 1 and self.white or self.black + local castle_rank = self.toMove == 1 and Rank1 or Rank8 + local out = {} + local function emit(id) + if not self:applyMove(id):illegalyChecked() then + table.insert(out, id) + end + end + + local cr = tm:band(self.castle):band(castle_rank) + if not cr:empty() then + local p = self.toMove == 1 and 11 or 12 + local tcolor = self.toMove == 1 and self.black or self.white + local kidx = self[p]:ctz() + + + local castle = bit32.replace(0, p, 20, 4) + castle = bit32.replace(castle, kidx, 6, 6) + castle = bit32.replace(castle, 1, 14) + + + local mustbeemptyl = LeftMasks[4]:bxor(FileA):band(castle_rank) + local cantbethreatened = FileD:bor(FileC):band(castle_rank):bor(self[p]) + if + not cr:bandempty(FileA) and + mustbeemptyl:bandempty(self.ocupied) and + not self:isSquareThreatened(cantbethreatened, tcolor) + then + emit(bit32.replace(castle, kidx - 2, 0, 6)) + end + + + local mustbeemptyr = RightMasks[3]:bxor(FileH):band(castle_rank) + if + not cr:bandempty(FileH) and + mustbeemptyr:bandempty(self.ocupied) and + not self:isSquareThreatened(mustbeemptyr:bor(self[p]), tcolor) + then + emit(bit32.replace(castle, kidx + 2, 0, 6)) + end + end + + local sq = tm:ctz() + repeat + local p = self:index(sq) + local moves = self:pmoves(sq) + + while not moves:empty() do + local m = moves:ctz() + moves = moves:set(m, 0) + local id = bit32.replace(m, sq, 6, 6) + id = bit32.replace(id, p, 20, 4) + local mbb = Bitboard.some(m) + if not self.ocupied:bandempty(mbb) then + id = bit32.replace(id, self:index(m), 25, 4) + end + + -- Check if pawn needs to be promoted + if p == 1 and m >= 8*7 then + for i=3,9,2 do + emit(bit32.replace(id, i, 15, 4)) + end + elseif p == 2 and m < 8 then + for i=4,10,2 do + emit(bit32.replace(id, i, 15, 4)) + end + else + emit(id) + end + end + sq = tm:ctzafter(sq) + until sq == 64 + return out +end + +function Board:illegalyChecked() + local target = self.toMove == 1 and self[PieceSymbols:find("k")] or self[PieceSymbols:find("K")] + return self:isSquareThreatened(target, self.toMove == 1 and self.white or self.black) +end + +function Board:isSquareThreatened(target , color ) + local tm = color + local sq = tm:ctz() + repeat + local moves = self:pmoves(sq) + if not moves:bandempty(target) then + return true + end + sq = color:ctzafter(sq) + until sq == 64 + return false +end + +function Board:perft(depth ) + if depth == 0 then return 1 end + if depth == 1 then + return #self:moveList() + end + local result = 0 + for k,m in ipairs(self:moveList()) do + local c = self:applyMove(m):perft(depth - 1) + if c == 0 then + -- Perft only counts leaf nodes at target depth + -- result = result + 1 + else + result = result + c + end + end + return result +end + + +function Board:applyMove(move ) + local out = Board.new() + table.move(self, 1, 12, 1, out) + local from = bit32.extract(move, 6, 6) + local to = bit32.extract(move, 0, 6) + local promote = bit32.extract(move, 15, 4) + local piece = self:index(from) + local captured = self:index(to) + local tom = Bitboard.some(to) + local isCastle = bit32.extract(move, 14) + + if piece % 2 == 0 then + out.moves = self.moves + 1 + end + + if captured == 1 or piece < 3 then + out.hm = 0 + else + out.hm = self.hm + 1 + end + out.castle = self.castle + out.toMove = self.toMove == 1 and 2 or 1 + + if isCastle == 1 then + local rank = piece == 11 and Rank1 or Rank8 + local colorOffset = piece - 11 + + out[3 + colorOffset] = out[3 + colorOffset]:bandnot(from < to and FileH or FileA) + out[3 + colorOffset] = out[3 + colorOffset]:bor((from < to and FileF or FileD):band(rank)) + + out[piece] = (from < to and FileG or FileC):band(rank) + out.castle = out.castle:bandnot(rank) + out:updateCache() + return out + end + + if piece < 3 then + local dist = math.abs(to - from) + -- Pawn moved two squares, set ep square + if dist == 16 then + out.ep = Bitboard.some((from + to) / 2) + end + + -- Remove enpasent capture + if not tom:bandempty(self.ep) then + if piece == 1 then + out[2] = out[2]:bandnot(self.ep:down()) + end + if piece == 2 then + out[1] = out[1]:bandnot(self.ep:up()) + end + end + end + + if piece == 3 or piece == 4 then + out.castle = out.castle:set(from, 0) + end + + if piece > 10 then + local rank = piece == 11 and Rank1 or Rank8 + out.castle = out.castle:bandnot(rank) + end + + out[piece] = out[piece]:set(from, 0) + if promote == 0 then + out[piece] = out[piece]:set(to, 1) + else + out[promote] = out[promote]:set(to, 1) + end + if captured ~= 0 then + out[captured] = out[captured]:set(to, 0) + end + + out:updateCache() + return out +end + +Board.__index = Board +Board.__tostring = Board.toString +-- +-- Main +-- + +local failures = 0 +local function test(fen, ply, target) + local b = Board.fromFen(fen) + if b:fen() ~= fen then + print("FEN MISMATCH", fen, b:fen()) + failures = failures + 1 + return + end + + local found = b:perft(ply) + if found ~= target then + print(fen, "Found", found, "target", target) + failures = failures + 1 + for k,v in pairs(b:moveList()) do + print(ucimove(v) .. ': ' .. (ply > 1 and b:applyMove(v):perft(ply-1) or '1')) + end + --error("Test Failure") + else + print("OK", found, fen) + end +end + +-- From https://www.chessprogramming.org/Perft_Results +-- If interpreter, computers, or algorithm gets too fast +-- feel free to go deeper + +local testCases = {} +local function addTest(...) table.insert(testCases, {...}) end + +addTest(StartingFen, 3, 8902) +addTest("r3k2r/p1ppqpb1/bn2pnp1/3PN3/1p2P3/2N2Q1p/PPPBBPPP/R3K2R w KQkq - 0 0", 2, 2039) +addTest("8/2p5/3p4/KP5r/1R3p1k/8/4P1P1/8 w - - 0 0", 3, 2812) +addTest("r3k2r/Pppp1ppp/1b3nbN/nP6/BBP1P3/q4N2/Pp1P2PP/R2Q1RK1 w kq - 0 1", 3, 9467) +addTest("rnbq1k1r/pp1Pbppp/2p5/8/2B5/8/PPP1NnPP/RNBQK2R w KQ - 1 8", 2, 1486) +addTest("r4rk1/1pp1qppp/p1np1n2/2b1p1B1/2B1P1b1/P1NP1N2/1PP1QPPP/R4RK1 w - - 0 10", 2, 2079) + + +local function chess() + for k,v in ipairs(testCases) do + test(v[1],v[2],v[3]) + end +end + +bench.runCode(chess, "chess") diff --git a/tests/Autocomplete.test.cpp b/tests/Autocomplete.test.cpp index 07910a0a..44b8362d 100644 --- a/tests/Autocomplete.test.cpp +++ b/tests/Autocomplete.test.cpp @@ -1596,7 +1596,7 @@ TEST_CASE_FIXTURE(ACFixture, "type_correct_argument_type_suggestion") local function target(a: number, b: string) return a + #b end local function d(a: n@1, b) - return target(a, b) + return target(a, b) end )"); @@ -1609,7 +1609,7 @@ end local function target(a: number, b: string) return a + #b end local function d(a, b: s@1) - return target(a, b) + return target(a, b) end )"); @@ -1622,7 +1622,7 @@ end local function target(a: number, b: string) return a + #b end local function d(a:@1 @2, b) - return target(a, b) + return target(a, b) end )"); @@ -1640,7 +1640,7 @@ end local function target(a: number, b: string) return a + #b end local function d(a, b: @1)@2: number - return target(a, b) + return target(a, b) end )"); @@ -1682,7 +1682,7 @@ local x = target(function(a: n@1 local function target(callback: (a: number, b: string) -> number) return callback(4, "hello") end local x = target(function(a: n@1, b: @2) - return a + #b + return a + #b end) )"); @@ -1700,7 +1700,7 @@ end) local function target(callback: (...number) -> number) return callback(1, 2, 3) end local x = target(function(a: n@1) - return a + return a end )"); @@ -1716,7 +1716,7 @@ TEST_CASE_FIXTURE(ACFixture, "type_correct_expected_argument_type_pack_suggestio local function target(callback: (...number) -> number) return callback(1, 2, 3) end local x = target(function(...:n@1) - return a + return a end )"); @@ -1729,7 +1729,7 @@ end local function target(callback: (...number) -> number) return callback(1, 2, 3) end local x = target(function(a:number, b:number, ...:@1) - return a + b + return a + b end )"); @@ -1745,7 +1745,7 @@ TEST_CASE_FIXTURE(ACFixture, "type_correct_expected_return_type_suggestion") local function target(callback: () -> number) return callback() end local x = target(function(): n@1 - return 1 + return 1 end )"); @@ -1758,7 +1758,7 @@ end local function target(callback: () -> (number, number)) return callback() end local x = target(function(): (number, n@1 - return 1, 2 + return 1, 2 end )"); @@ -1774,7 +1774,7 @@ TEST_CASE_FIXTURE(ACFixture, "type_correct_expected_return_type_pack_suggestion" local function target(callback: () -> ...number) return callback() end local x = target(function(): ...n@1 - return 1, 2, 3 + return 1, 2, 3 end )"); @@ -1787,7 +1787,7 @@ end local function target(callback: () -> ...number) return callback() end local x = target(function(): (number, number, ...n@1 - return 1, 2, 3 + return 1, 2, 3 end )"); diff --git a/tests/Compiler.test.cpp b/tests/Compiler.test.cpp index 54a31a68..bbac3302 100644 --- a/tests/Compiler.test.cpp +++ b/tests/Compiler.test.cpp @@ -768,11 +768,11 @@ TEST_CASE("CaptureSelf") local MaterialsListClass = {} function MaterialsListClass:_MakeToolTip(guiElement, text) - local function updateTooltipPosition() - self._tweakingTooltipFrame = 5 - end + local function updateTooltipPosition() + self._tweakingTooltipFrame = 5 + end - updateTooltipPosition() + updateTooltipPosition() end return MaterialsListClass @@ -2001,14 +2001,14 @@ TEST_CASE("UpvaluesLoopsBytecode") { CHECK_EQ("\n" + compileFunction(R"( function test() - for i=1,10 do + for i=1,10 do i = i - foo(function() return i end) - if bar then - break - end - end - return 0 + foo(function() return i end) + if bar then + break + end + end + return 0 end )", 1), @@ -2035,14 +2035,14 @@ RETURN R0 1 CHECK_EQ("\n" + compileFunction(R"( function test() - for i in ipairs(data) do + for i in ipairs(data) do i = i - foo(function() return i end) - if bar then - break - end - end - return 0 + foo(function() return i end) + if bar then + break + end + end + return 0 end )", 1), @@ -2068,17 +2068,17 @@ RETURN R0 1 CHECK_EQ("\n" + compileFunction(R"( function test() - local i = 0 - while i < 5 do - local j + local i = 0 + while i < 5 do + local j j = i - foo(function() return j end) - i = i + 1 - if bar then - break - end - end - return 0 + foo(function() return j end) + i = i + 1 + if bar then + break + end + end + return 0 end )", 1), @@ -2105,17 +2105,17 @@ RETURN R1 1 CHECK_EQ("\n" + compileFunction(R"( function test() - local i = 0 - repeat - local j + local i = 0 + repeat + local j j = i - foo(function() return j end) - i = i + 1 - if bar then - break - end - until i < 5 - return 0 + foo(function() return j end) + i = i + 1 + if bar then + break + end + until i < 5 + return 0 end )", 1), @@ -2304,10 +2304,10 @@ local Value1, Value2, Value3 = ... local Table = {} Table.SubTable["Key"] = { - Key1 = Value1, - Key2 = Value2, - Key3 = Value3, - Key4 = true, + Key1 = Value1, + Key2 = Value2, + Key3 = Value3, + Key4 = true, } )"); diff --git a/tests/Conformance.test.cpp b/tests/Conformance.test.cpp index 5a697a49..06b3c523 100644 --- a/tests/Conformance.test.cpp +++ b/tests/Conformance.test.cpp @@ -801,4 +801,17 @@ TEST_CASE("IfElseExpression") runConformance("ifelseexpr.lua"); } +TEST_CASE("TagMethodError") +{ + ScopedFastFlag sff{"LuauCcallRestoreFix", true}; + + runConformance("tmerror.lua", [](lua_State* L) { + auto* cb = lua_callbacks(L); + + cb->debugprotectederror = [](lua_State* L) { + CHECK(lua_isyieldable(L)); + }; + }); +} + TEST_SUITE_END(); diff --git a/tests/IostreamOptional.h b/tests/IostreamOptional.h index 9f874899..e55b5b0c 100644 --- a/tests/IostreamOptional.h +++ b/tests/IostreamOptional.h @@ -2,6 +2,9 @@ #pragma once #include +#include + +namespace std { inline std::ostream& operator<<(std::ostream& lhs, const std::nullopt_t&) { @@ -9,10 +12,12 @@ inline std::ostream& operator<<(std::ostream& lhs, const std::nullopt_t&) } template -std::ostream& operator<<(std::ostream& lhs, const std::optional& t) +auto operator<<(std::ostream& lhs, const std::optional& t) -> decltype(lhs << *t) // SFINAE to only instantiate << for supported types { if (t) return lhs << *t; else return lhs << "none"; } + +} // namespace std diff --git a/tests/Linter.test.cpp b/tests/Linter.test.cpp index a9ed139f..37f1b60b 100644 --- a/tests/Linter.test.cpp +++ b/tests/Linter.test.cpp @@ -791,13 +791,13 @@ TEST_CASE_FIXTURE(Fixture, "TypeAnnotationsShouldNotProduceWarnings") { LintResult result = lint(R"(--!strict type InputData = { - id: number, - inputType: EnumItem, - inputState: EnumItem, - updated: number, - position: Vector3, - keyCode: EnumItem, - name: string + id: number, + inputType: EnumItem, + inputState: EnumItem, + updated: number, + position: Vector3, + keyCode: EnumItem, + name: string } )"); diff --git a/tests/TypeInfer.aliases.test.cpp b/tests/TypeInfer.aliases.test.cpp index 045f0230..f580604c 100644 --- a/tests/TypeInfer.aliases.test.cpp +++ b/tests/TypeInfer.aliases.test.cpp @@ -554,4 +554,54 @@ TEST_CASE_FIXTURE(Fixture, "non_recursive_aliases_that_reuse_a_generic_name") CHECK_EQ("{number | string}", toString(requireType("p"), {true})); } +/* + * We had a problem where all type aliases would be prototyped into a child scope that happened + * to have the same level. This caused a problem where, if a sibling function referred to that + * type alias in its type signature, it would erroneously be quantified away, even though it doesn't + * actually belong to the function. + * + * We solved this by ascribing a unique subLevel to each prototyped alias. + */ +TEST_CASE_FIXTURE(Fixture, "do_not_quantify_unresolved_aliases") +{ + CheckResult result = check(R"( + --!strict + + local KeyPool = {} + + local function newkey(pool: KeyPool, index) + return {} + end + + function newKeyPool() + local pool = { + available = {} :: {Key}, + } + + return setmetatable(pool, KeyPool) + end + + export type KeyPool = typeof(newKeyPool()) + export type Key = typeof(newkey(newKeyPool(), 1)) + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +/* + * We keep a cache of type alias onto TypeVar to prevent infinite types from + * being constructed via recursive or corecursive aliases. We have to adjust + * the TypeLevels of those generic TypeVars so that the unifier doesn't think + * they have improperly leaked out of their scope. + */ +TEST_CASE_FIXTURE(Fixture, "generic_typevars_are_not_considered_to_escape_their_scope_if_they_are_reused_in_multiple_aliases") +{ + CheckResult result = check(R"( + type Array = {T} + type Exclude = T + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.classes.test.cpp b/tests/TypeInfer.classes.test.cpp index 6da33a08..eabf7e65 100644 --- a/tests/TypeInfer.classes.test.cpp +++ b/tests/TypeInfer.classes.test.cpp @@ -437,8 +437,6 @@ TEST_CASE_FIXTURE(ClassFixture, "class_unification_type_mismatch_is_correct_orde TEST_CASE_FIXTURE(ClassFixture, "optional_class_field_access_error") { - ScopedFastFlag luauExtraNilRecovery("LuauExtraNilRecovery", true); - CheckResult result = check(R"( local b: Vector2? = nil local a = b.X + b.Z diff --git a/tests/TypeInfer.generics.test.cpp b/tests/TypeInfer.generics.test.cpp index 3a04a18f..581375a1 100644 --- a/tests/TypeInfer.generics.test.cpp +++ b/tests/TypeInfer.generics.test.cpp @@ -695,4 +695,25 @@ TEST_CASE_FIXTURE(Fixture, "typefuns_sharing_types") CHECK(requireType("y1") == requireType("y2")); } +TEST_CASE_FIXTURE(Fixture, "bound_tables_do_not_clone_original_fields") +{ + ScopedFastFlag luauRankNTypes{"LuauRankNTypes", true}; + ScopedFastFlag luauCloneBoundTables{"LuauCloneBoundTables", true}; + + CheckResult result = check(R"( +local exports = {} +local nested = {} + +nested.name = function(t, k) + local a = t.x.y + return rawget(t, k) +end + +exports.nested = nested +return exports + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.provisional.test.cpp b/tests/TypeInfer.provisional.test.cpp index b00fddc5..419da8ad 100644 --- a/tests/TypeInfer.provisional.test.cpp +++ b/tests/TypeInfer.provisional.test.cpp @@ -9,6 +9,7 @@ #include LUAU_FASTFLAG(LuauEqConstraint) +LUAU_FASTFLAG(LuauQuantifyInPlace2) using namespace Luau; @@ -42,7 +43,7 @@ TEST_CASE_FIXTURE(Fixture, "typeguard_inference_incomplete") end )"; - const std::string expected = R"( + const std::string old_expected = R"( function f(a:{fn:()->(free,free...)}): () if type(a) == 'boolean'then local a1:boolean=a @@ -51,7 +52,21 @@ TEST_CASE_FIXTURE(Fixture, "typeguard_inference_incomplete") end end )"; - CHECK_EQ(expected, decorateWithTypes(code)); + + const std::string expected = R"( + function f(a:{fn:()->(a,b...)}): () + if type(a) == 'boolean'then + local a1:boolean=a + elseif a.fn()then + local a2:{fn:()->(a,b...)}=a + end + end + )"; + + if (FFlag::LuauQuantifyInPlace2) + CHECK_EQ(expected, decorateWithTypes(code)); + else + CHECK_EQ(old_expected, decorateWithTypes(code)); } TEST_CASE_FIXTURE(Fixture, "xpcall_returns_what_f_returns") @@ -263,8 +278,8 @@ TEST_CASE_FIXTURE(Fixture, "lvalue_equals_another_lvalue_with_no_overlap") TEST_CASE_FIXTURE(Fixture, "bail_early_if_unification_is_too_complicated" * doctest::timeout(0.5)) { - ScopedFastInt sffi{"LuauTarjanChildLimit", 50}; - ScopedFastInt sffi2{"LuauTypeInferIterationLimit", 50}; + ScopedFastInt sffi{"LuauTarjanChildLimit", 1}; + ScopedFastInt sffi2{"LuauTypeInferIterationLimit", 1}; CheckResult result = check(R"LUA( local Result diff --git a/tests/TypeInfer.refinements.test.cpp b/tests/TypeInfer.refinements.test.cpp index 31739cdc..36dcaa95 100644 --- a/tests/TypeInfer.refinements.test.cpp +++ b/tests/TypeInfer.refinements.test.cpp @@ -8,6 +8,7 @@ LUAU_FASTFLAG(LuauWeakEqConstraint) LUAU_FASTFLAG(LuauOrPredicate) +LUAU_FASTFLAG(LuauQuantifyInPlace2) using namespace Luau; @@ -698,10 +699,16 @@ TEST_CASE_FIXTURE(RefinementClassFixture, "typeguard_cast_free_table_to_vector") CHECK_EQ("Vector3", toString(requireTypeAtPosition({5, 28}))); // type(vec) == "vector" - CHECK_EQ("Type '{- X: a, Y: b, Z: c -}' could not be converted into 'Instance'", toString(result.errors[0])); + if (FFlag::LuauQuantifyInPlace2) + CHECK_EQ("Type '{+ X: a, Y: b, Z: c +}' could not be converted into 'Instance'", toString(result.errors[0])); + else + CHECK_EQ("Type '{- X: a, Y: b, Z: c -}' could not be converted into 'Instance'", toString(result.errors[0])); CHECK_EQ("*unknown*", toString(requireTypeAtPosition({7, 28}))); // typeof(vec) == "Instance" - CHECK_EQ("{- X: a, Y: b, Z: c -}", toString(requireTypeAtPosition({9, 28}))); // type(vec) ~= "vector" and typeof(vec) ~= "Instance" + if (FFlag::LuauQuantifyInPlace2) + CHECK_EQ("{+ X: a, Y: b, Z: c +}", toString(requireTypeAtPosition({9, 28}))); // type(vec) ~= "vector" and typeof(vec) ~= "Instance" + else + CHECK_EQ("{- X: a, Y: b, Z: c -}", toString(requireTypeAtPosition({9, 28}))); // type(vec) ~= "vector" and typeof(vec) ~= "Instance" } TEST_CASE_FIXTURE(RefinementClassFixture, "typeguard_cast_instance_or_vector3_to_vector") diff --git a/tests/TypeInfer.tables.test.cpp b/tests/TypeInfer.tables.test.cpp index b7f0dc7b..f1451a81 100644 --- a/tests/TypeInfer.tables.test.cpp +++ b/tests/TypeInfer.tables.test.cpp @@ -617,7 +617,7 @@ TEST_CASE_FIXTURE(Fixture, "indexers_get_quantified_too") REQUIRE_EQ(indexer.indexType, typeChecker.numberType); - REQUIRE(nullptr != get(indexer.indexResultType)); + REQUIRE(nullptr != get(follow(indexer.indexResultType))); } TEST_CASE_FIXTURE(Fixture, "indexers_quantification_2") diff --git a/tests/TypeInfer.test.cpp b/tests/TypeInfer.test.cpp index dbc45382..45381757 100644 --- a/tests/TypeInfer.test.cpp +++ b/tests/TypeInfer.test.cpp @@ -180,7 +180,12 @@ TEST_CASE_FIXTURE(Fixture, "expr_statement") TEST_CASE_FIXTURE(Fixture, "generic_function") { - CheckResult result = check("function id(x) return x end local a = id(55) local b = id(nil)"); + CheckResult result = check(R"( + function id(x) return x end + local a = id(55) + local b = id(nil) + )"); + LUAU_REQUIRE_NO_ERRORS(result); CHECK_EQ(*typeChecker.numberType, *requireType("a")); @@ -1889,7 +1894,7 @@ TEST_CASE_FIXTURE(Fixture, "infer_higher_order_function") REQUIRE_EQ(2, argVec.size()); - const FunctionTypeVar* fType = get(argVec[0]); + const FunctionTypeVar* fType = get(follow(argVec[0])); REQUIRE(fType != nullptr); std::vector fArgs = flatten(fType->argTypes).first; @@ -1926,7 +1931,7 @@ TEST_CASE_FIXTURE(Fixture, "higher_order_function_2") REQUIRE_EQ(6, argVec.size()); - const FunctionTypeVar* fType = get(argVec[0]); + const FunctionTypeVar* fType = get(follow(argVec[0])); REQUIRE(fType != nullptr); } @@ -3842,10 +3847,10 @@ local T: any T = {} T.__index = T function T.new(...) - local self = {} - setmetatable(self, T) - self:construct(...) - return self + local self = {} + setmetatable(self, T) + self:construct(...) + return self end function T:construct(index) end @@ -4068,11 +4073,11 @@ function n:Clone() end local m = {} function m.a(x) - x:Clone() + x:Clone() end function m.b() - m.a(n) + m.a(n) end return m @@ -4393,8 +4398,6 @@ TEST_CASE_FIXTURE(Fixture, "record_matching_overload") TEST_CASE_FIXTURE(Fixture, "infer_anonymous_function_arguments") { - ScopedFastFlag luauInferFunctionArgsFix("LuauInferFunctionArgsFix", true); - // Simple direct arg to arg propagation CheckResult result = check(R"( type Table = { x: number, y: number } @@ -4681,7 +4684,6 @@ TEST_CASE_FIXTURE(Fixture, "checked_prop_too_early") { ScopedFastFlag sffs[] = { {"LuauSlightlyMoreFlexibleBinaryPredicates", true}, - {"LuauExtraNilRecovery", true}, }; CheckResult result = check(R"( @@ -4698,7 +4700,6 @@ TEST_CASE_FIXTURE(Fixture, "accidentally_checked_prop_in_opposite_branch") { ScopedFastFlag sffs[] = { {"LuauSlightlyMoreFlexibleBinaryPredicates", true}, - {"LuauExtraNilRecovery", true}, }; CheckResult result = check(R"( diff --git a/tests/TypeInfer.tryUnify.test.cpp b/tests/TypeInfer.tryUnify.test.cpp index 1f4b63ef..1192a8ac 100644 --- a/tests/TypeInfer.tryUnify.test.cpp +++ b/tests/TypeInfer.tryUnify.test.cpp @@ -8,6 +8,8 @@ #include "doctest.h" +LUAU_FASTFLAG(LuauQuantifyInPlace2); + using namespace Luau; struct TryUnifyFixture : Fixture @@ -15,7 +17,8 @@ struct TryUnifyFixture : Fixture TypeArena arena; ScopePtr globalScope{new Scope{arena.addTypePack({TypeId{}})}}; InternalErrorReporter iceHandler; - Unifier state{&arena, Mode::Strict, globalScope, Location{}, Variance::Covariant, &iceHandler}; + UnifierSharedState unifierState{&iceHandler}; + Unifier state{&arena, Mode::Strict, globalScope, Location{}, Variance::Covariant, unifierState}; }; TEST_SUITE_BEGIN("TryUnifyTests"); @@ -139,7 +142,10 @@ TEST_CASE_FIXTURE(TryUnifyFixture, "typepack_unification_should_trim_free_tails" )"); LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK_EQ("(number) -> (boolean)", toString(requireType("f"))); + if (FFlag::LuauQuantifyInPlace2) + CHECK_EQ("(number) -> boolean", toString(requireType("f"))); + else + CHECK_EQ("(number) -> (boolean)", toString(requireType("f"))); } TEST_CASE_FIXTURE(TryUnifyFixture, "variadic_type_pack_unification") diff --git a/tests/TypeInfer.typePacks.cpp b/tests/TypeInfer.typePacks.cpp index 3e1dedd4..8dab2605 100644 --- a/tests/TypeInfer.typePacks.cpp +++ b/tests/TypeInfer.typePacks.cpp @@ -98,10 +98,10 @@ TEST_CASE_FIXTURE(Fixture, "higher_order_function") std::vector applyArgs = flatten(applyType->argTypes).first; REQUIRE_EQ(3, applyArgs.size()); - const FunctionTypeVar* fType = get(applyArgs[0]); + const FunctionTypeVar* fType = get(follow(applyArgs[0])); REQUIRE(fType != nullptr); - const FunctionTypeVar* gType = get(applyArgs[1]); + const FunctionTypeVar* gType = get(follow(applyArgs[1])); REQUIRE(gType != nullptr); std::vector gArgs = flatten(gType->argTypes).first; @@ -285,7 +285,7 @@ TEST_CASE_FIXTURE(Fixture, "variadic_argument_tail") { CheckResult result = check(R"( local _ = function():((...any)->(...any),()->()) - return function() end, function() end + return function() end, function() end end for y in _() do end diff --git a/tests/TypeInfer.unionTypes.test.cpp b/tests/TypeInfer.unionTypes.test.cpp index c0ed25d2..34c25a9f 100644 --- a/tests/TypeInfer.unionTypes.test.cpp +++ b/tests/TypeInfer.unionTypes.test.cpp @@ -181,8 +181,6 @@ TEST_CASE_FIXTURE(Fixture, "index_on_a_union_type_with_one_optional_property") TEST_CASE_FIXTURE(Fixture, "index_on_a_union_type_with_missing_property") { - ScopedFastFlag luauMissingUnionPropertyError("LuauMissingUnionPropertyError", true); - CheckResult result = check(R"( type A = {x: number} type B = {} @@ -242,8 +240,6 @@ TEST_CASE_FIXTURE(Fixture, "union_equality_comparisons") TEST_CASE_FIXTURE(Fixture, "optional_union_members") { - ScopedFastFlag luauExtraNilRecovery("LuauExtraNilRecovery", true); - CheckResult result = check(R"( local a = { a = { x = 1, y = 2 }, b = 3 } type A = typeof(a) @@ -259,8 +255,6 @@ local c = bf.a.y TEST_CASE_FIXTURE(Fixture, "optional_union_functions") { - ScopedFastFlag luauExtraNilRecovery("LuauExtraNilRecovery", true); - CheckResult result = check(R"( local a = {} function a.foo(x:number, y:number) return x + y end @@ -276,8 +270,6 @@ local c = b.foo(1, 2) TEST_CASE_FIXTURE(Fixture, "optional_union_methods") { - ScopedFastFlag luauExtraNilRecovery("LuauExtraNilRecovery", true); - CheckResult result = check(R"( local a = {} function a:foo(x:number, y:number) return x + y end @@ -310,8 +302,6 @@ return f() TEST_CASE_FIXTURE(Fixture, "optional_field_access_error") { - ScopedFastFlag luauExtraNilRecovery("LuauExtraNilRecovery", true); - CheckResult result = check(R"( type A = { x: number } local b: A? = { x = 2 } @@ -327,8 +317,6 @@ local d = b.y TEST_CASE_FIXTURE(Fixture, "optional_index_error") { - ScopedFastFlag luauExtraNilRecovery("LuauExtraNilRecovery", true); - CheckResult result = check(R"( type A = {number} local a: A? = {1, 2, 3} @@ -341,8 +329,6 @@ local b = a[1] TEST_CASE_FIXTURE(Fixture, "optional_call_error") { - ScopedFastFlag luauExtraNilRecovery("LuauExtraNilRecovery", true); - CheckResult result = check(R"( type A = (number) -> number local a: A? = function(a) return -a end @@ -355,8 +341,6 @@ local b = a(4) TEST_CASE_FIXTURE(Fixture, "optional_assignment_errors") { - ScopedFastFlag luauExtraNilRecovery("LuauExtraNilRecovery", true); - CheckResult result = check(R"( type A = { x: number } local a: A? = { x = 2 } @@ -378,8 +362,6 @@ a.x = 2 TEST_CASE_FIXTURE(Fixture, "optional_length_error") { - ScopedFastFlag luauExtraNilRecovery("LuauExtraNilRecovery", true); - CheckResult result = check(R"( type A = {number} local a: A? = {1, 2, 3} @@ -392,9 +374,6 @@ local b = #a TEST_CASE_FIXTURE(Fixture, "optional_missing_key_error_details") { - ScopedFastFlag luauExtraNilRecovery("LuauExtraNilRecovery", true); - ScopedFastFlag luauMissingUnionPropertyError("LuauMissingUnionPropertyError", true); - CheckResult result = check(R"( type A = { x: number, y: number } type B = { x: number, y: number } diff --git a/tests/TypeVar.test.cpp b/tests/TypeVar.test.cpp index a679e3fd..930c1a39 100644 --- a/tests/TypeVar.test.cpp +++ b/tests/TypeVar.test.cpp @@ -265,4 +265,64 @@ TEST_CASE_FIXTURE(Fixture, "substitution_skip_failure") CHECK_EQ("{ f: t1 } where t1 = () -> { f: () -> { f: ({ f: t1 }) -> (), signal: { f: (any) -> () } } }", toString(result)); } +TEST_CASE("tagging_tables") +{ + ScopedFastFlag sff{"LuauRefactorTagging", true}; + + TypeVar ttv{TableTypeVar{}}; + CHECK(!Luau::hasTag(&ttv, "foo")); + Luau::attachTag(&ttv, "foo"); + CHECK(Luau::hasTag(&ttv, "foo")); +} + +TEST_CASE("tagging_classes") +{ + ScopedFastFlag sff{"LuauRefactorTagging", true}; + + TypeVar base{ClassTypeVar{"Base", {}, std::nullopt, std::nullopt, {}, nullptr}}; + CHECK(!Luau::hasTag(&base, "foo")); + Luau::attachTag(&base, "foo"); + CHECK(Luau::hasTag(&base, "foo")); +} + +TEST_CASE("tagging_subclasses") +{ + ScopedFastFlag sff{"LuauRefactorTagging", true}; + + TypeVar base{ClassTypeVar{"Base", {}, std::nullopt, std::nullopt, {}, nullptr}}; + TypeVar derived{ClassTypeVar{"Derived", {}, &base, std::nullopt, {}, nullptr}}; + + CHECK(!Luau::hasTag(&base, "foo")); + CHECK(!Luau::hasTag(&derived, "foo")); + + Luau::attachTag(&base, "foo"); + CHECK(Luau::hasTag(&base, "foo")); + CHECK(Luau::hasTag(&derived, "foo")); + + Luau::attachTag(&derived, "bar"); + CHECK(!Luau::hasTag(&base, "bar")); + CHECK(Luau::hasTag(&derived, "bar")); +} + +TEST_CASE("tagging_functions") +{ + ScopedFastFlag sff{"LuauRefactorTagging", true}; + + TypePackVar empty{TypePack{}}; + TypeVar ftv{FunctionTypeVar{&empty, &empty}}; + CHECK(!Luau::hasTag(&ftv, "foo")); + Luau::attachTag(&ftv, "foo"); + CHECK(Luau::hasTag(&ftv, "foo")); +} + +TEST_CASE("tagging_props") +{ + ScopedFastFlag sff{"LuauRefactorTagging", true}; + + Property prop{}; + CHECK(!Luau::hasTag(prop, "foo")); + Luau::attachTag(prop, "foo"); + CHECK(Luau::hasTag(prop, "foo")); +} + TEST_SUITE_END(); diff --git a/tests/conformance/tmerror.lua b/tests/conformance/tmerror.lua new file mode 100644 index 00000000..1ad4dd16 --- /dev/null +++ b/tests/conformance/tmerror.lua @@ -0,0 +1,15 @@ +-- This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +-- This file is based on Lua 5.x tests -- https://github.com/lua/lua/tree/master/testes + +-- Generate an error (i.e. throw an exception) inside a tag method which is indirectly +-- called via pcall. +-- This test is meant to detect a regression in handling errors inside a tag method + +local testtable = {} +setmetatable(testtable, { __index = function() error("Error") end }) + +pcall(function() + testtable.missingmethod() +end) + +return('OK') diff --git a/tools/gdb-printers.py b/tools/gdb-printers.py index c711c5e2..017b9f95 100644 --- a/tools/gdb-printers.py +++ b/tools/gdb-printers.py @@ -11,9 +11,9 @@ class VariantPrinter: return type.name + " [" + str(value) + "]" def match_printer(val): - type = val.type.strip_typedefs() - if type.name and type.name.startswith('Luau::Variant<'): - return VariantPrinter(val) - return None + type = val.type.strip_typedefs() + if type.name and type.name.startswith('Luau::Variant<'): + return VariantPrinter(val) + return None gdb.pretty_printers.append(match_printer) From aea1f6a718029e2fab8c336719596ef899e6aff2 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Fri, 5 Nov 2021 10:16:28 -0700 Subject: [PATCH 025/399] Update README.md Add `make` commandline as well. --- README.md | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index 81b30337..020ac34b 100644 --- a/README.md +++ b/README.md @@ -22,7 +22,7 @@ You can download the binaries from [a recent release](https://github.com/Roblox/ # Building -To build Luau tools or tests yourself, you can use CMake on all platforms, or alternatively make (on Linux/macOS). For example: +To build Luau tools or tests yourself, you can use CMake on all platforms: ```sh mkdir cmake && cd cmake @@ -31,6 +31,12 @@ cmake --build . --target Luau.Repl.CLI --config RelWithDebInfo cmake --build . --target Luau.Analyze.CLI --config RelWithDebInfo ``` +Alternatively, on Linus/macOS you can use make: + +```sh +make config=release luau luau-analyze +``` + To integrate Luau into your CMake application projects, at the minimum you'll need to depend on `Luau.Compiler` and `Luau.VM` projects. From there you need to create a new Luau state (using Lua 5.x API such as `lua_newstate`), compile source to bytecode and load it into the VM like this: ```cpp From a80fc936469efc557847796efefa72107dc10d6b Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Fri, 5 Nov 2021 12:39:27 -0700 Subject: [PATCH 026/399] Fix unused variable treatment (#139) Right now our CMake infra specifies -Wno-unused only for GCC builds, but Makefile specifies it for all builds. The intent has been to use it just for GCC, so we now do that by detecting the compiler version - this should equalize the behavior across different types of builds. Separately, latest version of clang appears to expose an unused variable that clang-10 was okay with, so fix that. (change from upstream) --- Ast/src/Parser.cpp | 5 ----- Makefile | 5 ++++- 2 files changed, 4 insertions(+), 6 deletions(-) diff --git a/Ast/src/Parser.cpp b/Ast/src/Parser.cpp index 846bc0ba..ee84542d 100644 --- a/Ast/src/Parser.cpp +++ b/Ast/src/Parser.cpp @@ -2379,15 +2379,12 @@ AstArray Parser::parseTypeParams() Lexeme begin = lexer.current(); nextLexeme(); - bool seenPack = false; while (true) { if (FFlag::LuauParseTypePackTypeParameters) { if (shouldParseTypePackAnnotation(lexer)) { - seenPack = true; - auto typePack = parseTypePackAnnotation(); if (FFlag::LuauTypeAliasPacks) // Type packs are recorded only is we can handle them @@ -2399,8 +2396,6 @@ AstArray Parser::parseTypeParams() if (typePack) { - seenPack = true; - if (FFlag::LuauTypeAliasPacks) // Type packs are recorded only is we can handle them parameters.push_back({{}, typePack}); } diff --git a/Makefile b/Makefile index ea416be4..e96a9cbd 100644 --- a/Makefile +++ b/Makefile @@ -49,7 +49,10 @@ OBJECTS=$(AST_OBJECTS) $(COMPILER_OBJECTS) $(ANALYSIS_OBJECTS) $(VM_OBJECTS) $(T CXXFLAGS=-g -Wall -Werror LDFLAGS= -CXXFLAGS+=-Wno-unused # temporary, for older gcc versions +# temporary, for older gcc versions as they treat var in `if (type var = val)` as unused +ifeq ($(findstring g++,$(shell $(CXX) --version)),g++) + CXXFLAGS+=-Wno-unused +endif # configuration-specific flags ifeq ($(config),release) From ad9b47b72fb664c518a8e55471c58a3e21986cd0 Mon Sep 17 00:00:00 2001 From: bmcq-0 <66541602+bmcq-0@users.noreply.github.com> Date: Fri, 5 Nov 2021 16:24:30 -0400 Subject: [PATCH 027/399] Fold length operations when argument is a constant string (#141) Co-authored-by: Arseny Kapoulkine --- Compiler/src/Compiler.cpp | 5 +++++ tests/Compiler.test.cpp | 11 +++++++++++ 2 files changed, 16 insertions(+) diff --git a/Compiler/src/Compiler.cpp b/Compiler/src/Compiler.cpp index 7750a1d9..4c86b6f9 100644 --- a/Compiler/src/Compiler.cpp +++ b/Compiler/src/Compiler.cpp @@ -2900,6 +2900,11 @@ struct Compiler break; case AstExprUnary::Len: + if (arg.type == Constant::Type_String) + { + result.type = Constant::Type_Number; + result.valueNumber = double(arg.valueString.size); + } break; default: diff --git a/tests/Compiler.test.cpp b/tests/Compiler.test.cpp index bbac3302..1389bfec 100644 --- a/tests/Compiler.test.cpp +++ b/tests/Compiler.test.cpp @@ -1168,6 +1168,17 @@ RETURN R0 1 )"); } +TEST_CASE("ConstantFoldStringLen") +{ + CHECK_EQ("\n" + compileFunction0("return #'string', #'', #'a', #('b')"), R"( +LOADN R0 6 +LOADN R1 0 +LOADN R2 1 +LOADN R3 1 +RETURN R0 4 +)"); +} + TEST_CASE("ConstantFoldCompare") { // ordered comparisons From 6342913533b26045a6a254519e88aff6359026a5 Mon Sep 17 00:00:00 2001 From: Rerumu <25379555+Rerumu@users.noreply.github.com> Date: Fri, 5 Nov 2021 17:38:08 -0400 Subject: [PATCH 028/399] Fix small assert ordering (#143) --- Compiler/src/Compiler.cpp | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/Compiler/src/Compiler.cpp b/Compiler/src/Compiler.cpp index 4c86b6f9..45d10cc3 100644 --- a/Compiler/src/Compiler.cpp +++ b/Compiler/src/Compiler.cpp @@ -2465,9 +2465,10 @@ struct Compiler } else if (node->is()) { + LUAU_ASSERT(!loops.empty()); + // before exiting out of the loop, we need to close all local variables that were captured in closures since loop start // normally they are closed by the enclosing blocks, including the loop block, but we're skipping that here - LUAU_ASSERT(!loops.empty()); closeLocals(loops.back().localOffset); size_t label = bytecode.emitLabel(); @@ -2478,12 +2479,13 @@ struct Compiler } else if (AstStatContinue* stat = node->as()) { + LUAU_ASSERT(!loops.empty()); + if (loops.back().untilCondition) validateContinueUntil(stat, loops.back().untilCondition); // before continuing, we need to close all local variables that were captured in closures since loop start // normally they are closed by the enclosing blocks, including the loop block, but we're skipping that here - LUAU_ASSERT(!loops.empty()); closeLocals(loops.back().localOffset); size_t label = bytecode.emitLabel(); From 1e1d1f58e9bb9b196ec983a0d4bde605d6fbdd0f Mon Sep 17 00:00:00 2001 From: "Roni N. (Kittenz)" Date: Sat, 6 Nov 2021 04:11:26 +0200 Subject: [PATCH 029/399] Look for `.luau` before `.lua` in REPL & Analyze (#97) (#124) As discussed in the issue, Luau has evolved from Lua to the point where a new default extension `.luau` would be needed. This change makes the REPL and Analyze look for `.luau` extension first and if not found, fall back to `.lua`. --- CLI/Analyze.cpp | 18 ++++++++++++++---- CLI/Repl.cpp | 10 ++++++---- 2 files changed, 20 insertions(+), 8 deletions(-) diff --git a/CLI/Analyze.cpp b/CLI/Analyze.cpp index ed0552d7..d72a865d 100644 --- a/CLI/Analyze.cpp +++ b/CLI/Analyze.cpp @@ -115,7 +115,12 @@ struct CliFileResolver : Luau::FileResolver { if (Luau::AstExprConstantString* expr = node->as()) { - Luau::ModuleName name = std::string(expr->value.data, expr->value.size) + ".lua"; + Luau::ModuleName name = std::string(expr->value.data, expr->value.size) + ".luau"; + if (!moduleExists(name)) + { + // fall back to .lua if a module with .luau doesn't exist + name = std::string(expr->value.data, expr->value.size) + ".lua"; + } return {{name}}; } @@ -236,8 +241,15 @@ int main(int argc, char** argv) if (isDirectory(argv[i])) { traverseDirectory(argv[i], [&](const std::string& name) { - if (name.length() > 4 && name.rfind(".lua") == name.length() - 4) + // Look for .luau first and if absent, fall back to .lua + if (name.length() > 5 && name.rfind(".luau") == name.length() - 5) + { failed += !analyzeFile(frontend, name.c_str(), format, annotate); + } + else if (name.length() > 4 && name.rfind(".lua") == name.length() - 4) + { + failed += !analyzeFile(frontend, name.c_str(), format, annotate); + } }); } else @@ -256,5 +268,3 @@ int main(int argc, char** argv) return (format == ReportFormat::Luacheck) ? 0 : failed; } - - diff --git a/CLI/Repl.cpp b/CLI/Repl.cpp index 4968d080..323ab144 100644 --- a/CLI/Repl.cpp +++ b/CLI/Repl.cpp @@ -51,9 +51,13 @@ static int lua_require(lua_State* L) return finishrequire(L); lua_pop(L, 1); - std::optional source = readFile(name + ".lua"); + std::optional source = readFile(name + ".luau"); if (!source) - luaL_argerrorL(L, 1, ("error loading " + name).c_str()); + { + source = readFile(name + ".lua"); // try .lua if .luau doesn't exist + if (!source) + luaL_argerrorL(L, 1, ("error loading " + name).c_str()); // if neither .luau nor .lua exist, we have an error + } // module needs to run in a new thread, isolated from the rest lua_State* GL = lua_mainthread(L); @@ -511,5 +515,3 @@ int main(int argc, char** argv) return failed; } } - - From 96b1707f8778921465d62e5dcaf5d79432a07707 Mon Sep 17 00:00:00 2001 From: Rerumu <25379555+Rerumu@users.noreply.github.com> Date: Fri, 5 Nov 2021 22:11:56 -0400 Subject: [PATCH 030/399] Fix CLI analysis reporting wrong file names (#146) --- CLI/Analyze.cpp | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-) diff --git a/CLI/Analyze.cpp b/CLI/Analyze.cpp index d72a865d..798ead06 100644 --- a/CLI/Analyze.cpp +++ b/CLI/Analyze.cpp @@ -34,8 +34,10 @@ static void report(ReportFormat format, const char* name, const Luau::Location& } } -static void reportError(ReportFormat format, const char* name, const Luau::TypeError& error) +static void reportError(ReportFormat format, const Luau::TypeError& error) { + const char* name = error.moduleName.c_str(); + if (const Luau::SyntaxError* syntaxError = Luau::get_if(&error.data)) report(format, name, error.location, "SyntaxError", syntaxError->message.c_str()); else @@ -49,7 +51,10 @@ static void reportWarning(ReportFormat format, const char* name, const Luau::Lin static bool analyzeFile(Luau::Frontend& frontend, const char* name, ReportFormat format, bool annotate) { - Luau::CheckResult cr = frontend.check(name); + Luau::CheckResult cr; + + if (frontend.isDirty(name)) + cr = frontend.check(name); if (!frontend.getSourceModule(name)) { @@ -58,7 +63,7 @@ static bool analyzeFile(Luau::Frontend& frontend, const char* name, ReportFormat } for (auto& error : cr.errors) - reportError(format, name, error); + reportError(format, error); Luau::LintResult lr = frontend.lint(name); From c6de3bd2e4b6a9e892d5690b393631cc33ef60b3 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Fri, 5 Nov 2021 19:50:29 -0700 Subject: [PATCH 031/399] Update sandbox.md Remove section on thread identity: this is not part of open-source Luau and as such is now confusing. --- docs/_pages/sandbox.md | 10 ---------- 1 file changed, 10 deletions(-) diff --git a/docs/_pages/sandbox.md b/docs/_pages/sandbox.md index 8457cd11..409a0929 100644 --- a/docs/_pages/sandbox.md +++ b/docs/_pages/sandbox.md @@ -46,16 +46,6 @@ This is using the VM feature that is not accessible from scripts, that prevents By itself this would mean that code that runs in Luau can't use globals at all, since assigning globals would fail. While this is feasible, in Roblox we solve this by creating a new global table for each script, that uses `__index` to point to the builtin global table. This safely sandboxes the builtin globals while still allowing writing globals from each script. This also means that short of exposing special shared globals from the host, all scripts are isolated from each other. -## Thread identity - -Environment-level sandboxing is sufficient to implement separation between trusted code and untrusted code, assuming that `getfenv`/`setfenv` are either unavailable (removed from the globals), or that trusted code never interfaces with untrusted code (which prevents untrusted code from ever getting access to trusted functions). When running trusted code, it's possible to inject extra globals from the host into that global table, providing access to special APIs. - -However, in some cases it's desirable to restrict access to functions that are exposed both to trusted and untrusted code. For example, both may have access to `game` global, but `game` may expose methods that should only work from trusted code. - -To achieve this, each thread in Luau has a security identity, which can only be set by the host. Newly created threads inherit identities from the parent thread, and functions exposed from the host can validate the identity of the calling thread. This makes it possible to provide APIs to trusted code while limiting the access from untrusted code. - -> Note: to achieve an even stronger guarantee of isolation between trusted and untrusted code, it's possible to run it in different Luau VMs, which is what Roblox does for extra safety. - ## `__gc` Lua 5.1 exposes a `__gc` metamethod for userdata, which can be used on proxies (`newproxy`) to hook into garbage collector. Later versions of Lua extend this mechanism to work on tables. From 16753a78f4e0f77d1542921a62b1758014d8e718 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Sun, 7 Nov 2021 08:07:57 -0800 Subject: [PATCH 032/399] Update navigation.yml Add GitHub link to top bar --- docs/_data/navigation.yml | 2 ++ 1 file changed, 2 insertions(+) diff --git a/docs/_data/navigation.yml b/docs/_data/navigation.yml index ad2e55fd..63058963 100644 --- a/docs/_data/navigation.yml +++ b/docs/_data/navigation.yml @@ -3,6 +3,8 @@ main: url: /news - title: Getting Started url: /getting-started + - title: GitHub + url: https://github.com/Roblox/luau pages: - title: Getting Started From f27580c74b09c9f0bdbc00aa77f5a3d79cc0faee Mon Sep 17 00:00:00 2001 From: ThePotato <10260415+ThePotato97@users.noreply.github.com> Date: Mon, 8 Nov 2021 07:19:28 +0000 Subject: [PATCH 033/399] Fix linux typo in README.md "Linus" to "Linux" (#174) --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 020ac34b..ffb464d0 100644 --- a/README.md +++ b/README.md @@ -31,7 +31,7 @@ cmake --build . --target Luau.Repl.CLI --config RelWithDebInfo cmake --build . --target Luau.Analyze.CLI --config RelWithDebInfo ``` -Alternatively, on Linus/macOS you can use make: +Alternatively, on Linux/macOS you can use make: ```sh make config=release luau luau-analyze From 46244d2ea7223bbe0756b21e8b75b7fd5b8d6eb0 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Sun, 7 Nov 2021 23:21:34 -0800 Subject: [PATCH 034/399] Update getting-started.md Update file extension from .lua to .luau Contributes to #97 --- docs/_pages/getting-started.md | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/docs/_pages/getting-started.md b/docs/_pages/getting-started.md index 2f6017da..f64e0ebc 100644 --- a/docs/_pages/getting-started.md +++ b/docs/_pages/getting-started.md @@ -8,7 +8,7 @@ To get started with Luau you need to use `luau` command line binary to run your ## Creating a script -To create your own testing script, create a new file with `.lua` as the extension: +To create your own testing script, create a new file with `.luau` as the extension: ```lua function ispositive(x) @@ -26,7 +26,7 @@ print(isfoo("bar")) print(isfoo(1)) ``` -You can now run the file using `luau test.lua` and analyze it using `luau-analyze test.lua`. +You can now run the file using `luau test.luau` and analyze it using `luau-analyze test.luau`. Note that there are no warnings about calling ``ispositive()`` with a string, or calling ``isfoo()`` a number. This is because the type checking uses non-strict mode by default, which is lenient in how it infers types used by the program. @@ -52,8 +52,8 @@ In this case, Luau will use the ``return x > 0`` statement to infer that ``ispos Based on Luau's type inference, the analysis tool will now flag the incorrect call to ``ispositive()``: ``` -$ luau-analyze test.lua -test.lua(7,18): TypeError: Type 'string' could not be converted into 'number' +$ luau-analyze test.luau +test.luau(7,18): TypeError: Type 'string' could not be converted into 'number' ``` ## Annotations @@ -92,9 +92,9 @@ result = ispositive(1) Oops -- we're returning string values, but we forgot to update the function return type. Since we've told Luau that ``ispositive()`` returns a boolean (and that's how we're using it), the call site isn't flagged as an error. But because the annotation doesn't match our code, we get a warning in the function body itself: ``` -$ luau-analyze test.lua -test.lua(5,9): TypeError: Type 'string' could not be converted into 'boolean' -test.lua(7,9): TypeError: Type 'string' could not be converted into 'boolean' +$ luau-analyze test.luau +test.luau(5,9): TypeError: Type 'string' could not be converted into 'boolean' +test.luau(7,9): TypeError: Type 'string' could not be converted into 'boolean' ``` The fix is simple; just change the annotation to declare the return type as a string: @@ -117,8 +117,8 @@ result = ispositive(1) Well, almost - since we declared ``result`` as a boolean, the call site is now flagged: ``` -$ luau-analyze test.lua -test.lua(12,10): TypeError: Type 'string' could not be converted into 'boolean' +$ luau-analyze test.luau +test.luau(12,10): TypeError: Type 'string' could not be converted into 'boolean' ``` If we update the type of the local variable, everything is good. Note that we could also just let Luau infer the type of ``result`` by changing it to the single line version ``local result = ispositive(1)``. From f1649a43cdf7dd9e41496913672e82b30c067234 Mon Sep 17 00:00:00 2001 From: LoganDark Date: Mon, 8 Nov 2021 08:26:03 -0800 Subject: [PATCH 035/399] Add userdata to lua_Callbacks (#168) --- VM/include/lua.h | 2 ++ 1 file changed, 2 insertions(+) diff --git a/VM/include/lua.h b/VM/include/lua.h index 2f93ad90..8e7d6468 100644 --- a/VM/include/lua.h +++ b/VM/include/lua.h @@ -346,6 +346,8 @@ struct lua_Debug * can only be changed when the VM is not running any code */ struct lua_Callbacks { + void* userdata; /* arbitrary userdata pointer that is never overwritten by Luau */ + void (*interrupt)(lua_State* L, int gc); /* gets called at safepoints (loop back edges, call/ret, gc) if set */ void (*panic)(lua_State* L, int errcode); /* gets called when an unprotected error is raised (if longjmp is used) */ From 7ca09ec100b7583eaf26adb6ceb5fbff42f50db0 Mon Sep 17 00:00:00 2001 From: LoganDark Date: Mon, 8 Nov 2021 11:09:30 -0800 Subject: [PATCH 036/399] Fix lbuiltins.cpp comment (#180) --- VM/src/lbuiltins.cpp | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/VM/src/lbuiltins.cpp b/VM/src/lbuiltins.cpp index e1c99b21..9ff0d6a8 100644 --- a/VM/src/lbuiltins.cpp +++ b/VM/src/lbuiltins.cpp @@ -20,8 +20,9 @@ // If types of the arguments mismatch, luauF_* needs to return -1 and the execution will fall back to the usual call path // If luauF_* succeeds, it needs to return *all* requested arguments, filling results with nil as appropriate. // On input, nparams refers to the actual number of arguments (0+), whereas nresults contains LUA_MULTRET for arbitrary returns or 0+ for a -// fixed-length return Because of this, and the fact that "extra" returned values will be ignored, implementations below typically check that nresults -// is <= expected number, which covers the LUA_MULTRET case. +// fixed-length return +// Because of this, and the fact that "extra" returned values will be ignored, implementations below typically check that nresults is <= expected +// number, which covers the LUA_MULTRET case. static int luauF_assert(lua_State* L, StkId res, TValue* arg0, int nresults, StkId args, int nparams) { From 7816f2596f0545629c3c303ca1d7aee46025c693 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Mon, 8 Nov 2021 15:01:57 -0800 Subject: [PATCH 037/399] Update config.yml Rename "help and support" to "questions" --- .github/ISSUE_TEMPLATE/config.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/ISSUE_TEMPLATE/config.yml b/.github/ISSUE_TEMPLATE/config.yml index 37382f13..55f9db39 100644 --- a/.github/ISSUE_TEMPLATE/config.yml +++ b/.github/ISSUE_TEMPLATE/config.yml @@ -1,5 +1,5 @@ blank_issues_enabled: false contact_links: - - name: Help and support + - name: Questions url: https://github.com/Roblox/luau/discussions about: Please use GitHub Discussions if you have questions or need help. From 773fb5bd6f9ef512d61b87fb13cd88ffc0af3581 Mon Sep 17 00:00:00 2001 From: LoganDark Date: Mon, 8 Nov 2021 15:30:51 -0800 Subject: [PATCH 038/399] Don't allow registry to be set to readonly (#184) --- VM/src/lapi.cpp | 1 + 1 file changed, 1 insertion(+) diff --git a/VM/src/lapi.cpp b/VM/src/lapi.cpp index f2e97c66..c0c78b93 100644 --- a/VM/src/lapi.cpp +++ b/VM/src/lapi.cpp @@ -703,6 +703,7 @@ void lua_setreadonly(lua_State* L, int objindex, bool value) const TValue* o = index2adr(L, objindex); api_check(L, ttistable(o)); Table* t = hvalue(o); + api_check(L, t != hvalue(registry(L))); t->readonly = value; return; } From 9523670e631751b70ed3dc07641a0d7fc9989ce1 Mon Sep 17 00:00:00 2001 From: LoganDark Date: Tue, 9 Nov 2021 07:06:25 -0800 Subject: [PATCH 039/399] Remove Roblox-specific mutable globals (#185) Instead the code that calls the compiler needs to use Compiler::mutableGlobals to disable GETIMPORT optimization. --- Compiler/include/Luau/Compiler.h | 4 ++ Compiler/src/Compiler.cpp | 30 +++++--- tests/Compiler.test.cpp | 115 +++++++++++++++++++++++++++++++ 3 files changed, 141 insertions(+), 8 deletions(-) diff --git a/Compiler/include/Luau/Compiler.h b/Compiler/include/Luau/Compiler.h index f8d67158..2935f595 100644 --- a/Compiler/include/Luau/Compiler.h +++ b/Compiler/include/Luau/Compiler.h @@ -36,6 +36,10 @@ struct CompileOptions // global builtin to construct vectors; disabled by default const char* vectorLib = nullptr; const char* vectorCtor = nullptr; + + // array of globals that are mutable; disables the import optimization for fields accessed through them + // use NULL to end the array + const char** mutableGlobals = nullptr; }; class CompileError : public std::exception diff --git a/Compiler/src/Compiler.cpp b/Compiler/src/Compiler.cpp index 45d10cc3..08cb9934 100644 --- a/Compiler/src/Compiler.cpp +++ b/Compiler/src/Compiler.cpp @@ -13,6 +13,7 @@ LUAU_FASTFLAGVARIABLE(LuauPreloadClosures, false) LUAU_FASTFLAGVARIABLE(LuauPreloadClosuresFenv, false) LUAU_FASTFLAGVARIABLE(LuauPreloadClosuresUpval, false) +LUAU_FASTFLAGVARIABLE(LuauGenericSpecialGlobals, false) LUAU_FASTFLAG(LuauIfElseExpressionBaseSupport) namespace Luau @@ -1277,7 +1278,7 @@ struct Compiler { const Global* global = globals.find(expr->name); - return options.optimizationLevel >= 1 && (!global || (!global->written && !global->special)); + return options.optimizationLevel >= 1 && (!global || (!global->written && !global->writable)); } void compileExprIndexName(AstExprIndexName* expr, uint8_t target) @@ -3447,7 +3448,7 @@ struct Compiler struct Global { - bool special = false; + bool writable = false; bool written = false; }; @@ -3505,7 +3506,7 @@ struct Compiler { Global* g = globals.find(object->name); - return !g || (!g->special && !g->written) ? Builtin{object->name, expr->index} : Builtin(); + return !g || (!g->writable && !g->written) ? Builtin{object->name, expr->index} : Builtin(); } else { @@ -3703,13 +3704,26 @@ void compileOrThrow(BytecodeBuilder& bytecode, AstStatBlock* root, const AstName Compiler compiler(bytecode, options); - // since access to some global objects may result in values that change over time, we block table imports - for (const char* global : kSpecialGlobals) + // since access to some global objects may result in values that change over time, we block imports from non-readonly tables + if (FFlag::LuauGenericSpecialGlobals) { - AstName name = names.get(global); + if (AstName name = names.get("_G"); name.value) + compiler.globals[name].writable = true; - if (name.value) - compiler.globals[name].special = true; + if (options.mutableGlobals) + for (const char** ptr = options.mutableGlobals; *ptr != NULL; ++ptr) + { + if (AstName name = names.get(*ptr); name.value) + compiler.globals[name].writable = true; + } + } + else + { + for (const char* global : kSpecialGlobals) + { + if (AstName name = names.get(global); name.value) + compiler.globals[name].writable = true; + } } // this visitor traverses the AST to analyze mutability of locals/globals, filling Local::written and Global::written diff --git a/tests/Compiler.test.cpp b/tests/Compiler.test.cpp index 1389bfec..7f03019c 100644 --- a/tests/Compiler.test.cpp +++ b/tests/Compiler.test.cpp @@ -13,6 +13,7 @@ LUAU_FASTFLAG(LuauPreloadClosures) LUAU_FASTFLAG(LuauPreloadClosuresFenv) LUAU_FASTFLAG(LuauPreloadClosuresUpval) +LUAU_FASTFLAG(LuauGenericSpecialGlobals) using namespace Luau; @@ -3670,4 +3671,118 @@ RETURN R0 0 )"); } +TEST_CASE("LuauGenericSpecialGlobals") +{ + const char* source = R"( +print() +Game.print() +Workspace.print() +_G.print() +game.print() +plugin.print() +script.print() +shared.print() +workspace.print() +)"; + + { + ScopedFastFlag genericSpecialGlobals{"LuauGenericSpecialGlobals", false}; + + // Check Roblox globals are here + CHECK_EQ("\n" + compileFunction0(source), R"( +GETIMPORT R0 1 +CALL R0 0 0 +GETIMPORT R1 3 +GETTABLEKS R0 R1 K0 +CALL R0 0 0 +GETIMPORT R1 5 +GETTABLEKS R0 R1 K0 +CALL R0 0 0 +GETIMPORT R1 7 +GETTABLEKS R0 R1 K0 +CALL R0 0 0 +GETIMPORT R1 9 +GETTABLEKS R0 R1 K0 +CALL R0 0 0 +GETIMPORT R1 11 +GETTABLEKS R0 R1 K0 +CALL R0 0 0 +GETIMPORT R1 13 +GETTABLEKS R0 R1 K0 +CALL R0 0 0 +GETIMPORT R1 15 +GETTABLEKS R0 R1 K0 +CALL R0 0 0 +GETIMPORT R1 17 +GETTABLEKS R0 R1 K0 +CALL R0 0 0 +RETURN R0 0 +)"); + } + + ScopedFastFlag genericSpecialGlobals{"LuauGenericSpecialGlobals", true}; + + // Check Roblox globals are no longer here + CHECK_EQ("\n" + compileFunction0(source), R"( +GETIMPORT R0 1 +CALL R0 0 0 +GETIMPORT R0 3 +CALL R0 0 0 +GETIMPORT R0 5 +CALL R0 0 0 +GETIMPORT R1 7 +GETTABLEKS R0 R1 K0 +CALL R0 0 0 +GETIMPORT R0 9 +CALL R0 0 0 +GETIMPORT R0 11 +CALL R0 0 0 +GETIMPORT R0 13 +CALL R0 0 0 +GETIMPORT R0 15 +CALL R0 0 0 +GETIMPORT R0 17 +CALL R0 0 0 +RETURN R0 0 +)"); + + // Check we can add them back + Luau::BytecodeBuilder bcb; + bcb.setDumpFlags(Luau::BytecodeBuilder::Dump_Code); + Luau::CompileOptions options; + const char* mutableGlobals[] = {"Game", "Workspace", "game", "plugin", "script", "shared", "workspace", NULL}; + options.mutableGlobals = &mutableGlobals[0]; + Luau::compileOrThrow(bcb, source, options); + + CHECK_EQ("\n" + bcb.dumpFunction(0), R"( +GETIMPORT R0 1 +CALL R0 0 0 +GETIMPORT R1 3 +GETTABLEKS R0 R1 K0 +CALL R0 0 0 +GETIMPORT R1 5 +GETTABLEKS R0 R1 K0 +CALL R0 0 0 +GETIMPORT R1 7 +GETTABLEKS R0 R1 K0 +CALL R0 0 0 +GETIMPORT R1 9 +GETTABLEKS R0 R1 K0 +CALL R0 0 0 +GETIMPORT R1 11 +GETTABLEKS R0 R1 K0 +CALL R0 0 0 +GETIMPORT R1 13 +GETTABLEKS R0 R1 K0 +CALL R0 0 0 +GETIMPORT R1 15 +GETTABLEKS R0 R1 K0 +CALL R0 0 0 +GETIMPORT R1 17 +GETTABLEKS R0 R1 K0 +CALL R0 0 0 +RETURN R0 0 +)"); +} + TEST_SUITE_END(); From fe1bf43b546784b257f50380c836d12b06004793 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Tue, 9 Nov 2021 10:40:14 -0800 Subject: [PATCH 040/399] RFC: bit32.countlz/countrz (#89) Co-authored-by: vegorov-rbx <75688451+vegorov-rbx@users.noreply.github.com> --- rfcs/function-bit32-countlz-countrz.md | 50 ++++++++++++++++++++++++++ 1 file changed, 50 insertions(+) create mode 100644 rfcs/function-bit32-countlz-countrz.md diff --git a/rfcs/function-bit32-countlz-countrz.md b/rfcs/function-bit32-countlz-countrz.md new file mode 100644 index 00000000..986da70c --- /dev/null +++ b/rfcs/function-bit32-countlz-countrz.md @@ -0,0 +1,50 @@ +# bit32.countlz/countrz + +## Summary + +Add bit32.countlz (count left zeroes) and bit32.countrz (count right zeroes) to accelerate bit scanning + +## Motivation + +All CPUs have instructions to determine the position of first/last set bit in an integer. These instructions have a variety of uses, the popular ones being: + +- Fast implementation of integer logarithm (essentially allowing to compute `floor(log2(value))` quickly) +- Scanning set bits in an integer, which allows efficient traversal of compact representation of bitmaps +- Allocating bits out of a bitmap quickly + +Today it's possible to approximate `countlz` using `floor` and `log` but this approximation is relatively slow; approximating `countrz` is difficult without iterating through each bit. + +## Design + +`bit32` library will gain two new functions, `countlz` and `countrz`: + +``` +function bit32.countlz(n: number): number +function bit32.countrz(n: number): number +``` + +`countlz` takes an integer number (converting the input number to a 32-bit unsigned integer as all other `bit32` functions do), and returns the number of consecutive left-most zero bits - that is, the number of most significant zero bits in a 32-bit number until the first 1. The result is in `[0, 32]` range. + +For example, when the input number is `0`, it's `32`. When the input number is `2^k`, the result is `31-k`. + +`countrz` takes an integer number (converting the input number to a 32-bit unsigned integer as all other `bit32` functions do), and returns the number of consecutive right-most zero bits - that is, +the number of least significant zero bits in a 32-bit number until the first 1. The result is in `[0, 32]` range. + +For example, when the input number is `0`, it's `32`. When the input number is `2^k`, the result is `k`. + +> Non-normative: a proof of concept implementation shows that a polyfill for `countlz` takes ~34 ns per loop iteration when computing `countlz` for an increasing number sequence, whereas +> a builtin implementation takes ~4 ns. + +## Drawbacks + +None known. + +## Alternatives + +These functions can be alternatively specified as "find the position of the most/least significant bit set" (e.g. "ffs"/"fls" for "find first set"/"find last set"). This formulation +can be more immediately useful since the bit position is usually more important than the number of bits. However, the bit position is undefined when the input number is zero, +returning a sentinel such as -1 seems non-idiomatic, and returning `nil` seems awkward for calling code. Counting functions don't have this problem. + +An early version of this proposal suggested `clz`/`ctz` (leading/trailing) as names; however, using a full verb is more consistent with other operations like shift/rotate, and left/right may be easier to understand intuitively compared to leading/trailing. left/right are used by C++20. + +Of the two functions, `countlz` is vastly more useful than `countrz`; we could implement just `countlz`, but having both is nice for symmetry. From 3ba0bdf7084f3d8eb37c64f0872d4ed912edede8 Mon Sep 17 00:00:00 2001 From: NotDSF <67711164+NotDSF@users.noreply.github.com> Date: Tue, 9 Nov 2021 22:54:53 +0000 Subject: [PATCH 041/399] Save bytecode to file through CLI (#170) Using --compile=binary it's now possible to produce binary bytecode so that it can be compiled offline and loaded into the VM. --- CLI/Repl.cpp | 46 +++++++++++++++++++++++++++++++++++++++------- 1 file changed, 39 insertions(+), 7 deletions(-) diff --git a/CLI/Repl.cpp b/CLI/Repl.cpp index 323ab144..a5bd5737 100644 --- a/CLI/Repl.cpp +++ b/CLI/Repl.cpp @@ -13,6 +13,17 @@ #include +#ifdef _WIN32 + #include + #include +#endif + +enum class CompileFormat +{ + Default, + Binary +}; + static int lua_loadstring(lua_State* L) { size_t l = 0; @@ -370,7 +381,7 @@ static void reportError(const char* name, const Luau::CompileError& error) report(name, error.getLocation(), "CompileError", error.what()); } -static bool compileFile(const char* name) +static bool compileFile(const char* name, CompileFormat format) { std::optional source = readFile(name); if (!source) @@ -387,7 +398,20 @@ static bool compileFile(const char* name) Luau::compileOrThrow(bcb, *source); - printf("%s", bcb.dumpEverything().c_str()); + switch (format) + { + case CompileFormat::Default: + printf("%s", bcb.dumpEverything().c_str()); + break; + case CompileFormat::Binary: + #ifdef _WIN32 + _setmode(_fileno(stdout), _O_BINARY); + #endif + + std::string Bytecode = bcb.getBytecode(); + fwrite(Bytecode.c_str(), 1, Bytecode.size(), stdout); + break; + } return true; } @@ -412,7 +436,7 @@ static void displayHelp(const char* argv0) printf("\n"); printf("Available modes:\n"); printf(" omitted: compile and run input files one by one\n"); - printf(" --compile: compile input files and output resulting bytecode\n"); + printf(" --compile[=format]: compile input files and output resulting formatted bytecode (binary or text)\n"); printf("\n"); printf("Available options:\n"); printf(" --profile[=N]: profile the code using N Hz sampling (default 10000) and output results to profile.out\n"); @@ -444,8 +468,16 @@ int main(int argc, char** argv) return 0; } - if (argc >= 2 && strcmp(argv[1], "--compile") == 0) - { + + if (argc >= 2 && strncmp(argv[1], "--compile", strlen("--compile")) == 0) + { + CompileFormat format = CompileFormat::Default; + + if (strcmp(argv[1], "--compile=binary") == 0) + { + format = CompileFormat::Binary; + } + int failed = 0; for (int i = 2; i < argc; ++i) @@ -457,12 +489,12 @@ int main(int argc, char** argv) { traverseDirectory(argv[i], [&](const std::string& name) { if (name.length() > 4 && name.rfind(".lua") == name.length() - 4) - failed += !compileFile(name.c_str()); + failed += !compileFile(name.c_str(), format); }); } else { - failed += !compileFile(argv[i]); + failed += !compileFile(argv[i], format); } } From f3468be92b90f8d4b3069a0bc6e8b30d04a686cd Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Tue, 9 Nov 2021 15:11:52 -0800 Subject: [PATCH 042/399] Small follow code cleanup for Repl.cpp --- CLI/Repl.cpp | 26 +++++++++++++------------- 1 file changed, 13 insertions(+), 13 deletions(-) diff --git a/CLI/Repl.cpp b/CLI/Repl.cpp index a5bd5737..1c5305a4 100644 --- a/CLI/Repl.cpp +++ b/CLI/Repl.cpp @@ -14,8 +14,8 @@ #include #ifdef _WIN32 - #include - #include +#include +#include #endif enum class CompileFormat @@ -404,12 +404,7 @@ static bool compileFile(const char* name, CompileFormat format) printf("%s", bcb.dumpEverything().c_str()); break; case CompileFormat::Binary: - #ifdef _WIN32 - _setmode(_fileno(stdout), _O_BINARY); - #endif - - std::string Bytecode = bcb.getBytecode(); - fwrite(Bytecode.c_str(), 1, Bytecode.size(), stdout); + fwrite(bcb.getBytecode().data(), 1, bcb.getBytecode().size(), stdout); break; } @@ -470,13 +465,16 @@ int main(int argc, char** argv) if (argc >= 2 && strncmp(argv[1], "--compile", strlen("--compile")) == 0) - { + { CompileFormat format = CompileFormat::Default; - if (strcmp(argv[1], "--compile=binary") == 0) - { + if (strcmp(argv[1], "--compile=binary") == 0) format = CompileFormat::Binary; - } + +#ifdef _WIN32 + if (format == CompileFormat::Binary) + _setmode(_fileno(stdout), _O_BINARY); +#endif int failed = 0; @@ -488,7 +486,9 @@ int main(int argc, char** argv) if (isDirectory(argv[i])) { traverseDirectory(argv[i], [&](const std::string& name) { - if (name.length() > 4 && name.rfind(".lua") == name.length() - 4) + if (name.length() > 5 && name.rfind(".luau") == name.length() - 5) + failed += !compileFile(name.c_str(), format); + else if (name.length() > 4 && name.rfind(".lua") == name.length() - 4) failed += !compileFile(name.c_str(), format); }); } From d6b3346f580b4b50fbdcdfe0ff748f65204c8214 Mon Sep 17 00:00:00 2001 From: LoganDark Date: Tue, 9 Nov 2021 17:02:46 -0800 Subject: [PATCH 043/399] move static_assert from ltable.h to ltable.cpp (#189) --- VM/src/ltable.cpp | 1 + VM/src/ltable.h | 1 - 2 files changed, 1 insertion(+), 1 deletion(-) diff --git a/VM/src/ltable.cpp b/VM/src/ltable.cpp index 883442ae..07d22d59 100644 --- a/VM/src/ltable.cpp +++ b/VM/src/ltable.cpp @@ -30,6 +30,7 @@ LUAU_FASTFLAGVARIABLE(LuauArrayBoundary, false) #define MAXBITS 26 #define MAXSIZE (1 << MAXBITS) +static_assert(offsetof(LuaNode, val) == 0, "Unexpected Node memory layout, pointer cast in gval2slot is incorrect"); // TKey is bitpacked for memory efficiency so we need to validate bit counts for worst case static_assert(TKey{{NULL}, 0, LUA_TDEADKEY, 0}.tt == LUA_TDEADKEY, "not enough bits for tt"); static_assert(TKey{{NULL}, 0, LUA_TNIL, MAXSIZE - 1}.next == MAXSIZE - 1, "not enough bits for next"); diff --git a/VM/src/ltable.h b/VM/src/ltable.h index f98d87b1..45061443 100644 --- a/VM/src/ltable.h +++ b/VM/src/ltable.h @@ -9,7 +9,6 @@ #define gval(n) (&(n)->val) #define gnext(n) ((n)->key.next) -static_assert(offsetof(LuaNode, val) == 0, "Unexpected Node memory layout, pointer cast below is incorrect"); #define gval2slot(t, v) int(cast_to(LuaNode*, static_cast(v)) - t->node) LUAI_FUNC const TValue* luaH_getnum(Table* t, int key); From aec8fbfd0facdba1e1f6dcb070742000d92fa5ed Mon Sep 17 00:00:00 2001 From: Pelanyo Kamara Date: Wed, 10 Nov 2021 16:40:46 +0000 Subject: [PATCH 044/399] Feature: Web REPL using Emscripten (#138) Currently doesn't include the new page into navigation since we aren't building the .js files anywhere. --- .gitignore | 1 + CLI/Repl.cpp | 44 ++++++++++++++++++++++++++++++++++ CMakeLists.txt | 38 +++++++++++++++++++++-------- docs/_data/navigation.yml | 4 ++++ docs/_includes/repl.html | 50 +++++++++++++++++++++++++++++++++++++++ docs/_pages/demo.md | 6 +++++ 6 files changed, 133 insertions(+), 10 deletions(-) create mode 100644 docs/_includes/repl.html create mode 100644 docs/_pages/demo.md diff --git a/.gitignore b/.gitignore index 0b2422ce..fa11b45b 100644 --- a/.gitignore +++ b/.gitignore @@ -5,3 +5,4 @@ ^default.prof* ^fuzz-* ^luau$ +/.vs diff --git a/CLI/Repl.cpp b/CLI/Repl.cpp index 1c5305a4..63b86a80 100644 --- a/CLI/Repl.cpp +++ b/CLI/Repl.cpp @@ -198,6 +198,11 @@ static std::string runCode(lua_State* L, const std::string& source) error += "\nstack backtrace:\n"; error += lua_debugtrace(T); +#ifdef __EMSCRIPTEN__ + // nicer formatting for errors in web repl + error = "Error:" + error; +#endif + fprintf(stdout, "%s", error.c_str()); } @@ -205,6 +210,44 @@ static std::string runCode(lua_State* L, const std::string& source) return std::string(); } +#ifdef __EMSCRIPTEN__ +extern "C" +{ + const char* executeScript(const char* source) + { + // static string for caching result (prevents dangling ptr on function exit) + static std::string result; + + // setup flags + for (Luau::FValue* flag = Luau::FValue::list; flag; flag = flag->next) + if (strncmp(flag->name, "Luau", 4) == 0) + flag->value = true; + + // create new state + std::unique_ptr globalState(luaL_newstate(), lua_close); + lua_State* L = globalState.get(); + + // setup state + setupState(L); + + // sandbox thread + luaL_sandboxthread(L); + + // run code + collect error + std::string error = runCode(L, source); + result = error; + + if (error.length()) + { + return result.c_str(); + } + return NULL; + } +} +#endif + +// Excluded from emscripten compilation to avoid -Wunused-function errors. +#ifndef __EMSCRIPTEN__ static void completeIndexer(lua_State* L, const char* editBuffer, size_t start, std::vector& completions) { std::string_view lookup = editBuffer + start; @@ -547,3 +590,4 @@ int main(int argc, char** argv) return failed; } } +#endif diff --git a/CMakeLists.txt b/CMakeLists.txt index d6598f2a..36014a98 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -17,17 +17,26 @@ add_library(Luau.VM STATIC) if(LUAU_BUILD_CLI) add_executable(Luau.Repl.CLI) - add_executable(Luau.Analyze.CLI) + if(NOT EMSCRIPTEN) + add_executable(Luau.Analyze.CLI) + else() + # add -fexceptions for emscripten to allow exceptions to be caught in C++ + set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -fexceptions") + endif() # This also adds target `name` on Linux/macOS and `name.exe` on Windows set_target_properties(Luau.Repl.CLI PROPERTIES OUTPUT_NAME luau) - set_target_properties(Luau.Analyze.CLI PROPERTIES OUTPUT_NAME luau-analyze) + + if(NOT EMSCRIPTEN) + set_target_properties(Luau.Analyze.CLI PROPERTIES OUTPUT_NAME luau-analyze) + endif() endif() -if(LUAU_BUILD_TESTS) +if(LUAU_BUILD_TESTS AND NOT EMSCRIPTEN) add_executable(Luau.UnitTest) add_executable(Luau.Conformance) endif() + include(Sources.cmake) target_compile_features(Luau.Ast PUBLIC cxx_std_17) @@ -53,10 +62,6 @@ if(MSVC) else() list(APPEND LUAU_OPTIONS -Wall) # All warnings list(APPEND LUAU_OPTIONS -Werror) # Warnings are errors - - if(CMAKE_CXX_COMPILER_ID STREQUAL "GNU") - list(APPEND LUAU_OPTIONS -Wno-unused) # GCC considers variables declared/checked in if() as unused - endif() endif() target_compile_options(Luau.Ast PRIVATE ${LUAU_OPTIONS}) @@ -65,7 +70,10 @@ target_compile_options(Luau.VM PRIVATE ${LUAU_OPTIONS}) if(LUAU_BUILD_CLI) target_compile_options(Luau.Repl.CLI PRIVATE ${LUAU_OPTIONS}) - target_compile_options(Luau.Analyze.CLI PRIVATE ${LUAU_OPTIONS}) + + if(NOT EMSCRIPTEN) + target_compile_options(Luau.Analyze.CLI PRIVATE ${LUAU_OPTIONS}) + endif() target_include_directories(Luau.Repl.CLI PRIVATE extern) target_link_libraries(Luau.Repl.CLI PRIVATE Luau.Compiler Luau.VM) @@ -74,10 +82,20 @@ if(LUAU_BUILD_CLI) target_link_libraries(Luau.Repl.CLI PRIVATE pthread) endif() - target_link_libraries(Luau.Analyze.CLI PRIVATE Luau.Analysis) + if(NOT EMSCRIPTEN) + target_link_libraries(Luau.Analyze.CLI PRIVATE Luau.Analysis) + endif() + + if(EMSCRIPTEN) + # declare exported functions to emscripten + target_link_options(Luau.Repl.CLI PRIVATE -sEXPORTED_FUNCTIONS=['_executeScript'] -sEXPORTED_RUNTIME_METHODS=['ccall','cwrap'] -fexceptions) + + # custom output directory for wasm + js file + set_target_properties(Luau.Repl.CLI PROPERTIES RUNTIME_OUTPUT_DIRECTORY ${CMAKE_SOURCE_DIR}/docs/assets/luau) + endif() endif() -if(LUAU_BUILD_TESTS) +if(LUAU_BUILD_TESTS AND NOT EMSCRIPTEN) target_compile_options(Luau.UnitTest PRIVATE ${LUAU_OPTIONS}) target_include_directories(Luau.UnitTest PRIVATE extern) target_link_libraries(Luau.UnitTest PRIVATE Luau.Analysis Luau.Compiler) diff --git a/docs/_data/navigation.yml b/docs/_data/navigation.yml index 63058963..c828b2d4 100644 --- a/docs/_data/navigation.yml +++ b/docs/_data/navigation.yml @@ -27,3 +27,7 @@ pages: url: /profile - title: Library url: /library + +# Remove demo pages until solution is found +# - title: Demo +# url: /demo diff --git a/docs/_includes/repl.html b/docs/_includes/repl.html new file mode 100644 index 00000000..73c5ba40 --- /dev/null +++ b/docs/_includes/repl.html @@ -0,0 +1,50 @@ +
+
+ +
+ +

+ + +
+

+
+ +
+ +

+ +
+
+ + + diff --git a/docs/_pages/demo.md b/docs/_pages/demo.md new file mode 100644 index 00000000..fefa4b20 --- /dev/null +++ b/docs/_pages/demo.md @@ -0,0 +1,6 @@ +--- +permalink: /demo +title: Demo +--- + +{% include repl.html %} From 4957812b62e22a589542eaa2bcc2938222cca89b Mon Sep 17 00:00:00 2001 From: dcope-rbx <91100513+dcope-rbx@users.noreply.github.com> Date: Wed, 10 Nov 2021 09:26:20 -0800 Subject: [PATCH 045/399] Added documentation related to typecasting (#191) --- docs/_pages/typecheck.md | 31 +++++++++++++++++++++++++++++++ 1 file changed, 31 insertions(+) diff --git a/docs/_pages/typecheck.md b/docs/_pages/typecheck.md index 6c00b548..fcc14e02 100644 --- a/docs/_pages/typecheck.md +++ b/docs/_pages/typecheck.md @@ -351,6 +351,30 @@ local onlyString: string = stringOrNumber -- ok local onlyNumber: number = stringOrNumber -- not ok ``` +## Typecasts + +Expressions may be typecast using `::`. Typecasting is useful for specifying the type of an expression when the automatically inferred type is too generic. + +For example, consider the following table constructor where the intent is to store a table of names: +```lua +local myTable = {names = {}} +table.insert(myTable.names, 42) -- Inserting a number ought to cause a type error, but doesn't +``` + +In order to specify the type of the `names` table a typecast may be used: + +```lua +local myTable = {names = {} :: {string}} +table.insert(myTable.names, 42) -- not ok, invalid 'number' to 'string' conversion +``` + +A typecast itself is also type checked to ensure the conversion specified is a valid one: +```lua +local number = 1 +local value = number :: any -- ok, 'number' is a subtype of 'any' +local flag = number :: boolean -- not ok, invalid 'number' to 'boolean' conversion +``` + ## Roblox types Roblox supports a rich set of classes and data types, [documented here](https://developer.roblox.com/en-us/api-reference). All of them are readily available for the type checker to use by their name (e.g. `Part` or `RaycastResult`). @@ -397,3 +421,10 @@ return module ``` There are some caveats here though. For instance, the require path must be resolvable statically, otherwise Luau cannot accurately type check it. + +### Cyclic module dependencies + +Cyclic module dependencies can cause problems for the type checker. In order to break a module dependency cycle a typecast of the module to `any` may be used: +```lua +local myModule = require(MyModule) :: any +``` \ No newline at end of file From 68fb2bdfdee6db7739bc36a0bcbf3a2251a17ab6 Mon Sep 17 00:00:00 2001 From: dcope-rbx <91100513+dcope-rbx@users.noreply.github.com> Date: Wed, 10 Nov 2021 11:53:43 -0800 Subject: [PATCH 046/399] Addressed typecasting documentation feedback (#192) --- docs/_pages/typecheck.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/_pages/typecheck.md b/docs/_pages/typecheck.md index fcc14e02..58e66bb8 100644 --- a/docs/_pages/typecheck.md +++ b/docs/_pages/typecheck.md @@ -368,11 +368,11 @@ local myTable = {names = {} :: {string}} table.insert(myTable.names, 42) -- not ok, invalid 'number' to 'string' conversion ``` -A typecast itself is also type checked to ensure the conversion specified is a valid one: +A typecast itself is also type checked to ensure the conversion is made to a subtype of the expression's type or `any`: ```lua -local number = 1 -local value = number :: any -- ok, 'number' is a subtype of 'any' -local flag = number :: boolean -- not ok, invalid 'number' to 'boolean' conversion +local numericValue = 1 +local value = numericValue :: any -- ok, all expressions may be cast to 'any' +local flag = numericValue :: boolean -- not ok, invalid 'number' to 'boolean' conversion ``` ## Roblox types From a6a2b86c9b387f72c49364ce52d755f6e31e8a3b Mon Sep 17 00:00:00 2001 From: LoganDark Date: Thu, 11 Nov 2021 06:29:59 -0800 Subject: [PATCH 047/399] Fix build on Clang 12 by removing unused variables (#195) --- Analysis/src/Autocomplete.cpp | 4 ++-- Analysis/src/TypeInfer.cpp | 2 +- Analysis/src/TypePack.cpp | 2 +- Analysis/src/TypeVar.cpp | 12 ++++++------ 4 files changed, 10 insertions(+), 10 deletions(-) diff --git a/Analysis/src/Autocomplete.cpp b/Analysis/src/Autocomplete.cpp index 3c43c808..09476f78 100644 --- a/Analysis/src/Autocomplete.cpp +++ b/Analysis/src/Autocomplete.cpp @@ -496,7 +496,7 @@ static bool canSuggestInferredType(ScopePtr scope, TypeId ty) return false; // No syntax for unnamed tables with a metatable - if (const MetatableTypeVar* mtv = get(ty)) + if (get(ty)) return false; if (const TableTypeVar* ttv = get(ty)) @@ -688,7 +688,7 @@ static std::optional functionIsExpectedAt(const Module& module, AstNode* n TypeId expectedType = follow(*it); - if (const FunctionTypeVar* ftv = get(expectedType)) + if (get(expectedType)) return true; if (const IntersectionTypeVar* itv = get(expectedType)) diff --git a/Analysis/src/TypeInfer.cpp b/Analysis/src/TypeInfer.cpp index 38e2e527..c8fc55ba 100644 --- a/Analysis/src/TypeInfer.cpp +++ b/Analysis/src/TypeInfer.cpp @@ -2129,7 +2129,7 @@ TypeId TypeChecker::checkRelationalOperation( if (!isNonstrictMode() && !isOrOp) return ty; - if (auto i = get(ty)) + if (get(ty)) { std::optional cleaned = tryStripUnionFromNil(ty); diff --git a/Analysis/src/TypePack.cpp b/Analysis/src/TypePack.cpp index 68a16ef0..22257016 100644 --- a/Analysis/src/TypePack.cpp +++ b/Analysis/src/TypePack.cpp @@ -216,7 +216,7 @@ bool finite(TypePackId tp) if (auto pack = get(tp)) return pack->tail ? finite(*pack->tail) : true; - if (auto pack = get(tp)) + if (get(tp)) return false; return true; diff --git a/Analysis/src/TypeVar.cpp b/Analysis/src/TypeVar.cpp index e82f7519..631f58f9 100644 --- a/Analysis/src/TypeVar.cpp +++ b/Analysis/src/TypeVar.cpp @@ -749,9 +749,9 @@ void StateDot::visitChild(TypeId ty, int parentIndex, const char* linkName) if (opts.duplicatePrimitives && canDuplicatePrimitive(ty)) { - if (const PrimitiveTypeVar* ptv = get(ty)) + if (get(ty)) formatAppend(result, "n%d [label=\"%s\"];\n", index, toStringDetailed(ty, {}).name.c_str()); - else if (const AnyTypeVar* atv = get(ty)) + else if (get(ty)) formatAppend(result, "n%d [label=\"any\"];\n", index); } else @@ -902,19 +902,19 @@ void StateDot::visitChildren(TypeId ty, int index) finishNodeLabel(ty); finishNode(); } - else if (const AnyTypeVar* atv = get(ty)) + else if (get(ty)) { formatAppend(result, "AnyTypeVar %d", index); finishNodeLabel(ty); finishNode(); } - else if (const PrimitiveTypeVar* ptv = get(ty)) + else if (get(ty)) { formatAppend(result, "PrimitiveTypeVar %s", toStringDetailed(ty, {}).name.c_str()); finishNodeLabel(ty); finishNode(); } - else if (const ErrorTypeVar* etv = get(ty)) + else if (get(ty)) { formatAppend(result, "ErrorTypeVar %d", index); finishNodeLabel(ty); @@ -994,7 +994,7 @@ void StateDot::visitChildren(TypePackId tp, int index) finishNodeLabel(tp); finishNode(); } - else if (const Unifiable::Error* etp = get(tp)) + else if (get(tp)) { formatAppend(result, "ErrorTypePack %d", index); finishNodeLabel(tp); From 82d74e6f73fa8d9a81c4c932c668543c08c27597 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Thu, 11 Nov 2021 18:12:39 -0800 Subject: [PATCH 048/399] Sync to upstream/release/504 --- .gitignore | 1 + Analysis/include/Luau/Error.h | 13 +- Analysis/include/Luau/FileResolver.h | 2 +- Analysis/include/Luau/Transpiler.h | 3 +- Analysis/include/Luau/TypeInfer.h | 8 +- Analysis/include/Luau/TypePack.h | 20 +- Analysis/include/Luau/TypeVar.h | 45 +- Analysis/include/Luau/Unifiable.h | 3 - Analysis/include/Luau/Unifier.h | 2 +- Analysis/src/Autocomplete.cpp | 46 +- Analysis/src/BuiltinDefinitions.cpp | 244 +-------- Analysis/src/EmbeddedBuiltinDefinitions.cpp | 22 +- Analysis/src/Error.cpp | 291 ++++++---- Analysis/src/Frontend.cpp | 9 +- Analysis/src/Linter.cpp | 10 +- Analysis/src/Module.cpp | 17 +- Analysis/src/Predicate.cpp | 4 - Analysis/src/RequireTracer.cpp | 8 +- Analysis/src/Substitution.cpp | 13 +- Analysis/src/ToString.cpp | 114 +--- Analysis/src/Transpiler.cpp | 117 +++- Analysis/src/TypeAttach.cpp | 41 +- Analysis/src/TypeInfer.cpp | 576 ++++++-------------- Analysis/src/TypePack.cpp | 8 +- Analysis/src/TypeVar.cpp | 70 ++- Analysis/src/Unifiable.cpp | 10 - Analysis/src/Unifier.cpp | 345 ++++++------ Ast/include/Luau/Parser.h | 1 - Ast/src/Parser.cpp | 51 +- CLI/Analyze.cpp | 27 +- CLI/Repl.cpp | 93 +++- CMakeLists.txt | 38 +- Compiler/include/Luau/Bytecode.h | 4 + Compiler/include/Luau/Compiler.h | 3 + Compiler/src/Compiler.cpp | 47 +- Makefile | 10 +- VM/include/lua.h | 4 + VM/src/lapi.cpp | 11 + VM/src/lbitlib.cpp | 42 ++ VM/src/lbuiltins.cpp | 54 +- VM/src/lgc.cpp | 164 +----- VM/src/lstrlib.cpp | 20 +- VM/src/ltable.cpp | 1 + VM/src/ltable.h | 1 - VM/src/ltablib.cpp | 8 - bench/tests/chess.lua | 80 +-- fuzz/luau.proto | 7 + fuzz/proto.cpp | 7 + fuzz/protoprint.cpp | 10 + tests/AstQuery.test.cpp | 1 - tests/Autocomplete.test.cpp | 3 - tests/Compiler.test.cpp | 126 +++++ tests/Conformance.test.cpp | 6 +- tests/Linter.test.cpp | 6 - tests/Parser.test.cpp | 43 +- tests/Predicate.test.cpp | 6 - tests/ToString.test.cpp | 9 - tests/Transpiler.test.cpp | 252 ++++++++- tests/TypeInfer.aliases.test.cpp | 3 - tests/TypeInfer.builtins.test.cpp | 8 - tests/TypeInfer.classes.test.cpp | 25 +- tests/TypeInfer.definitions.test.cpp | 3 - tests/TypeInfer.generics.test.cpp | 89 --- tests/TypeInfer.intersectionTypes.test.cpp | 39 ++ tests/TypeInfer.provisional.test.cpp | 7 +- tests/TypeInfer.refinements.test.cpp | 44 +- tests/TypeInfer.tables.test.cpp | 72 +++ tests/TypeInfer.test.cpp | 34 +- tests/TypeInfer.tryUnify.test.cpp | 5 - tests/TypeInfer.typePacks.cpp | 12 - tests/TypeInfer.unionTypes.test.cpp | 41 +- tests/TypeVar.test.cpp | 2 - tests/conformance/bitwise.lua | 16 + 73 files changed, 1734 insertions(+), 1843 deletions(-) diff --git a/.gitignore b/.gitignore index 0b2422ce..fa11b45b 100644 --- a/.gitignore +++ b/.gitignore @@ -5,3 +5,4 @@ ^default.prof* ^fuzz-* ^luau$ +/.vs diff --git a/Analysis/include/Luau/Error.h b/Analysis/include/Luau/Error.h index ac6f13e9..9ee75004 100644 --- a/Analysis/include/Luau/Error.h +++ b/Analysis/include/Luau/Error.h @@ -8,11 +8,20 @@ namespace Luau { +struct TypeError; struct TypeMismatch { - TypeId wantedType; - TypeId givenType; + TypeMismatch() = default; + TypeMismatch(TypeId wantedType, TypeId givenType); + TypeMismatch(TypeId wantedType, TypeId givenType, std::string reason); + TypeMismatch(TypeId wantedType, TypeId givenType, std::string reason, TypeError error); + + TypeId wantedType = nullptr; + TypeId givenType = nullptr; + + std::string reason; + std::shared_ptr error; bool operator==(const TypeMismatch& rhs) const; }; diff --git a/Analysis/include/Luau/FileResolver.h b/Analysis/include/Luau/FileResolver.h index a05ec5e9..9b74fc12 100644 --- a/Analysis/include/Luau/FileResolver.h +++ b/Analysis/include/Luau/FileResolver.h @@ -53,7 +53,7 @@ struct FileResolver } // DEPRECATED APIS - // These are going to be removed with LuauNewRequireTracer + // These are going to be removed with LuauNewRequireTrace2 virtual bool moduleExists(const ModuleName& name) const = 0; virtual std::optional fromAstFragment(AstExpr* expr) const = 0; virtual ModuleName concat(const ModuleName& lhs, std::string_view rhs) const = 0; diff --git a/Analysis/include/Luau/Transpiler.h b/Analysis/include/Luau/Transpiler.h index 817459fe..df01008c 100644 --- a/Analysis/include/Luau/Transpiler.h +++ b/Analysis/include/Luau/Transpiler.h @@ -18,6 +18,7 @@ struct TranspileResult std::string parseError; // Nonempty if the transpile failed }; +std::string toString(AstNode* node); void dump(AstNode* node); // Never fails on a well-formed AST @@ -25,6 +26,6 @@ std::string transpile(AstStatBlock& ast); std::string transpileWithTypes(AstStatBlock& block); // Only fails when parsing fails -TranspileResult transpile(std::string_view source, ParseOptions options = ParseOptions{}); +TranspileResult transpile(std::string_view source, ParseOptions options = ParseOptions{}, bool withTypes = false); } // namespace Luau diff --git a/Analysis/include/Luau/TypeInfer.h b/Analysis/include/Luau/TypeInfer.h index 9d62fef0..306ac77d 100644 --- a/Analysis/include/Luau/TypeInfer.h +++ b/Analysis/include/Luau/TypeInfer.h @@ -263,8 +263,6 @@ public: * */ TypeId instantiate(const ScopePtr& scope, TypeId ty, Location location); - // Removed by FFlag::LuauRankNTypes - TypePackId DEPRECATED_instantiate(const ScopePtr& scope, TypePackId ty, Location location); // Replace any free types or type packs by `any`. // This is used when exporting types from modules, to make sure free types don't leak. @@ -298,8 +296,6 @@ private: // Produce a new free type var. TypeId freshType(const ScopePtr& scope); TypeId freshType(TypeLevel level); - TypeId DEPRECATED_freshType(const ScopePtr& scope, bool canBeGeneric = false); - TypeId DEPRECATED_freshType(TypeLevel level, bool canBeGeneric = false); // Returns nullopt if the predicate filters down the TypeId to 0 options. std::optional filterMap(TypeId type, TypeIdPredicate predicate); @@ -326,10 +322,8 @@ private: TypePackId addTypePack(std::initializer_list&& ty); TypePackId freshTypePack(const ScopePtr& scope); TypePackId freshTypePack(TypeLevel level); - TypePackId DEPRECATED_freshTypePack(const ScopePtr& scope, bool canBeGeneric = false); - TypePackId DEPRECATED_freshTypePack(TypeLevel level, bool canBeGeneric = false); - TypeId resolveType(const ScopePtr& scope, const AstType& annotation, bool canBeGeneric = false); + TypeId resolveType(const ScopePtr& scope, const AstType& annotation); TypePackId resolveTypePack(const ScopePtr& scope, const AstTypeList& types); TypePackId resolveTypePack(const ScopePtr& scope, const AstTypePack& annotation); TypeId instantiateTypeFun(const ScopePtr& scope, const TypeFun& tf, const std::vector& typeParams, diff --git a/Analysis/include/Luau/TypePack.h b/Analysis/include/Luau/TypePack.h index d987d46c..e72808da 100644 --- a/Analysis/include/Luau/TypePack.h +++ b/Analysis/include/Luau/TypePack.h @@ -8,8 +8,6 @@ #include #include -LUAU_FASTFLAG(LuauAddMissingFollow) - namespace Luau { @@ -128,13 +126,10 @@ TypePack* asMutable(const TypePack* tp); template const T* get(TypePackId tp) { - if (FFlag::LuauAddMissingFollow) - { - LUAU_ASSERT(tp); + LUAU_ASSERT(tp); - if constexpr (!std::is_same_v) - LUAU_ASSERT(get_if(&tp->ty) == nullptr); - } + if constexpr (!std::is_same_v) + LUAU_ASSERT(get_if(&tp->ty) == nullptr); return get_if(&(tp->ty)); } @@ -142,13 +137,10 @@ const T* get(TypePackId tp) template T* getMutable(TypePackId tp) { - if (FFlag::LuauAddMissingFollow) - { - LUAU_ASSERT(tp); + LUAU_ASSERT(tp); - if constexpr (!std::is_same_v) - LUAU_ASSERT(get_if(&tp->ty) == nullptr); - } + if constexpr (!std::is_same_v) + LUAU_ASSERT(get_if(&tp->ty) == nullptr); return get_if(&(asMutable(tp)->ty)); } diff --git a/Analysis/include/Luau/TypeVar.h b/Analysis/include/Luau/TypeVar.h index 9611e881..6bd7932d 100644 --- a/Analysis/include/Luau/TypeVar.h +++ b/Analysis/include/Luau/TypeVar.h @@ -18,7 +18,6 @@ LUAU_FASTINT(LuauTableTypeMaximumStringifierLength) LUAU_FASTINT(LuauTypeMaximumStringifierLength) -LUAU_FASTFLAG(LuauAddMissingFollow) namespace Luau { @@ -413,13 +412,17 @@ bool maybeGeneric(const TypeId ty); struct SingletonTypes { - const TypeId nilType = &nilType_; - const TypeId numberType = &numberType_; - const TypeId stringType = &stringType_; - const TypeId booleanType = &booleanType_; - const TypeId threadType = &threadType_; - const TypeId anyType = &anyType_; - const TypeId errorType = &errorType_; + const TypeId nilType; + const TypeId numberType; + const TypeId stringType; + const TypeId booleanType; + const TypeId threadType; + const TypeId anyType; + const TypeId errorType; + const TypeId optionalNumberType; + + const TypePackId anyTypePack; + const TypePackId errorTypePack; SingletonTypes(); SingletonTypes(const SingletonTypes&) = delete; @@ -427,14 +430,6 @@ struct SingletonTypes private: std::unique_ptr arena; - TypeVar nilType_; - TypeVar numberType_; - TypeVar stringType_; - TypeVar booleanType_; - TypeVar threadType_; - TypeVar anyType_; - TypeVar errorType_; - TypeId makeStringMetatable(); }; @@ -472,13 +467,10 @@ TypeVar* asMutable(TypeId ty); template const T* get(TypeId tv) { - if (FFlag::LuauAddMissingFollow) - { - LUAU_ASSERT(tv); + LUAU_ASSERT(tv); - if constexpr (!std::is_same_v) - LUAU_ASSERT(get_if(&tv->ty) == nullptr); - } + if constexpr (!std::is_same_v) + LUAU_ASSERT(get_if(&tv->ty) == nullptr); return get_if(&tv->ty); } @@ -486,13 +478,10 @@ const T* get(TypeId tv) template T* getMutable(TypeId tv) { - if (FFlag::LuauAddMissingFollow) - { - LUAU_ASSERT(tv); + LUAU_ASSERT(tv); - if constexpr (!std::is_same_v) - LUAU_ASSERT(get_if(&tv->ty) == nullptr); - } + if constexpr (!std::is_same_v) + LUAU_ASSERT(get_if(&tv->ty) == nullptr); return get_if(&asMutable(tv)->ty); } diff --git a/Analysis/include/Luau/Unifiable.h b/Analysis/include/Luau/Unifiable.h index 10dbf333..c2e07e46 100644 --- a/Analysis/include/Luau/Unifiable.h +++ b/Analysis/include/Luau/Unifiable.h @@ -63,12 +63,9 @@ using Name = std::string; struct Free { explicit Free(TypeLevel level); - Free(TypeLevel level, bool DEPRECATED_canBeGeneric); int index; TypeLevel level; - // Removed by FFlag::LuauRankNTypes - bool DEPRECATED_canBeGeneric = false; // True if this free type variable is part of a mutually // recursive type alias whose definitions haven't been // resolved yet. diff --git a/Analysis/include/Luau/Unifier.h b/Analysis/include/Luau/Unifier.h index 56632e33..be0aadd0 100644 --- a/Analysis/include/Luau/Unifier.h +++ b/Analysis/include/Luau/Unifier.h @@ -87,7 +87,6 @@ private: void tryUnifyWithAny(TypePackId any, TypePackId ty); std::optional findTablePropertyRespectingMeta(TypeId lhsType, Name name); - std::optional findMetatableEntry(TypeId type, std::string entry); public: // Report an "infinite type error" if the type "needle" already occurs within "haystack" @@ -102,6 +101,7 @@ private: bool isNonstrictMode() const; void checkChildUnifierTypeMismatch(const ErrorVec& innerErrors, TypeId wantedType, TypeId givenType); + void checkChildUnifierTypeMismatch(const ErrorVec& innerErrors, const std::string& prop, TypeId wantedType, TypeId givenType); [[noreturn]] void ice(const std::string& message, const Location& location); [[noreturn]] void ice(const std::string& message); diff --git a/Analysis/src/Autocomplete.cpp b/Analysis/src/Autocomplete.cpp index 3c43c808..1c94bb68 100644 --- a/Analysis/src/Autocomplete.cpp +++ b/Analysis/src/Autocomplete.cpp @@ -12,7 +12,6 @@ #include #include -LUAU_FASTFLAG(LuauSecondTypecheckKnowsTheDataModel) LUAU_FASTFLAGVARIABLE(ElseElseIfCompletionImprovements, false); LUAU_FASTFLAG(LuauIfElseExpressionAnalysisSupport) @@ -369,20 +368,10 @@ static void autocompleteProps(const Module& module, TypeArena* typeArena, TypeId while (iter != endIter) { - if (FFlag::LuauAddMissingFollow) - { - if (isNil(*iter)) - ++iter; - else - break; - } + if (isNil(*iter)) + ++iter; else - { - if (auto primTy = Luau::get(*iter); primTy && primTy->type == PrimitiveTypeVar::NilType) - ++iter; - else - break; - } + break; } if (iter == endIter) @@ -397,21 +386,10 @@ static void autocompleteProps(const Module& module, TypeArena* typeArena, TypeId AutocompleteEntryMap inner; std::unordered_set innerSeen = seen; - if (FFlag::LuauAddMissingFollow) + if (isNil(*iter)) { - if (isNil(*iter)) - { - ++iter; - continue; - } - } - else - { - if (auto innerPrimTy = Luau::get(*iter); innerPrimTy && innerPrimTy->type == PrimitiveTypeVar::NilType) - { - ++iter; - continue; - } + ++iter; + continue; } autocompleteProps(module, typeArena, *iter, indexType, nodes, inner, innerSeen); @@ -496,7 +474,7 @@ static bool canSuggestInferredType(ScopePtr scope, TypeId ty) return false; // No syntax for unnamed tables with a metatable - if (const MetatableTypeVar* mtv = get(ty)) + if (get(ty)) return false; if (const TableTypeVar* ttv = get(ty)) @@ -688,7 +666,7 @@ static std::optional functionIsExpectedAt(const Module& module, AstNode* n TypeId expectedType = follow(*it); - if (const FunctionTypeVar* ftv = get(expectedType)) + if (get(expectedType)) return true; if (const IntersectionTypeVar* itv = get(expectedType)) @@ -1519,10 +1497,10 @@ AutocompleteResult autocomplete(Frontend& frontend, const ModuleName& moduleName return {}; TypeChecker& typeChecker = - (frontend.options.typecheckTwice && FFlag::LuauSecondTypecheckKnowsTheDataModel ? frontend.typeCheckerForAutocomplete : frontend.typeChecker); + (frontend.options.typecheckTwice ? frontend.typeCheckerForAutocomplete : frontend.typeChecker); ModulePtr module = - (frontend.options.typecheckTwice && FFlag::LuauSecondTypecheckKnowsTheDataModel ? frontend.moduleResolverForAutocomplete.getModule(moduleName) - : frontend.moduleResolver.getModule(moduleName)); + (frontend.options.typecheckTwice ? frontend.moduleResolverForAutocomplete.getModule(moduleName) + : frontend.moduleResolver.getModule(moduleName)); if (!module) return {}; @@ -1550,7 +1528,7 @@ OwningAutocompleteResult autocompleteSource(Frontend& frontend, std::string_view sourceModule->commentLocations = std::move(result.commentLocations); TypeChecker& typeChecker = - (frontend.options.typecheckTwice && FFlag::LuauSecondTypecheckKnowsTheDataModel ? frontend.typeCheckerForAutocomplete : frontend.typeChecker); + (frontend.options.typecheckTwice ? frontend.typeCheckerForAutocomplete : frontend.typeChecker); ModulePtr module = typeChecker.check(*sourceModule, Mode::Strict); diff --git a/Analysis/src/BuiltinDefinitions.cpp b/Analysis/src/BuiltinDefinitions.cpp index f6f2363c..62a06a3c 100644 --- a/Analysis/src/BuiltinDefinitions.cpp +++ b/Analysis/src/BuiltinDefinitions.cpp @@ -8,10 +8,7 @@ #include -LUAU_FASTFLAG(LuauParseGenericFunctions) -LUAU_FASTFLAG(LuauGenericFunctions) -LUAU_FASTFLAG(LuauRankNTypes) -LUAU_FASTFLAG(LuauNewRequireTrace) +LUAU_FASTFLAG(LuauNewRequireTrace2) /** FIXME: Many of these type definitions are not quite completely accurate. * @@ -185,25 +182,11 @@ void registerBuiltinTypes(TypeChecker& typeChecker) TypeId numberType = typeChecker.numberType; TypeId booleanType = typeChecker.booleanType; TypeId nilType = typeChecker.nilType; - TypeId stringType = typeChecker.stringType; - TypeId threadType = typeChecker.threadType; - TypeId anyType = typeChecker.anyType; TypeArena& arena = typeChecker.globalTypes; - TypeId optionalNumber = makeOption(typeChecker, arena, numberType); - TypeId optionalString = makeOption(typeChecker, arena, stringType); - TypeId optionalBoolean = makeOption(typeChecker, arena, booleanType); - - TypeId stringOrNumber = makeUnion(arena, {stringType, numberType}); - - TypePackId emptyPack = arena.addTypePack({}); TypePackId oneNumberPack = arena.addTypePack({numberType}); - TypePackId oneStringPack = arena.addTypePack({stringType}); TypePackId oneBooleanPack = arena.addTypePack({booleanType}); - TypePackId oneAnyPack = arena.addTypePack({anyType}); - - TypePackId anyTypePack = typeChecker.anyTypePack; TypePackId numberVariadicList = arena.addTypePack(TypePackVar{VariadicTypePack{numberType}}); TypePackId listOfAtLeastOneNumber = arena.addTypePack(TypePack{{numberType}, numberVariadicList}); @@ -215,8 +198,6 @@ void registerBuiltinTypes(TypeChecker& typeChecker) TypeId listOfAtLeastZeroNumbersToNumberType = arena.addType(FunctionTypeVar{numberVariadicList, oneNumberPack}); - TypeId stringToAnyMap = arena.addType(TableTypeVar{{}, TableIndexer(stringType, anyType), typeChecker.globalScope->level}); - LoadDefinitionFileResult loadResult = Luau::loadDefinitionFile(typeChecker, typeChecker.globalScope, getBuiltinDefinitionSource(), "@luau"); LUAU_ASSERT(loadResult.success); @@ -236,8 +217,6 @@ void registerBuiltinTypes(TypeChecker& typeChecker) ttv->props["btest"] = makeProperty(arena.addType(FunctionTypeVar{listOfAtLeastOneNumber, oneBooleanPack}), "@luau/global/bit32.btest"); } - TypeId anyFunction = arena.addType(FunctionTypeVar{anyTypePack, anyTypePack}); - TypeId genericK = arena.addType(GenericTypeVar{"K"}); TypeId genericV = arena.addType(GenericTypeVar{"V"}); TypeId mapOfKtoV = arena.addType(TableTypeVar{{}, TableIndexer(genericK, genericV), typeChecker.globalScope->level}); @@ -252,222 +231,6 @@ void registerBuiltinTypes(TypeChecker& typeChecker) addGlobalBinding(typeChecker, "string", it->second.type, "@luau"); - if (!FFlag::LuauParseGenericFunctions || !FFlag::LuauGenericFunctions) - { - TableTypeVar::Props debugLib{ - {"info", {makeIntersection(arena, - { - arena.addType(FunctionTypeVar{arena.addTypePack({typeChecker.threadType, numberType, stringType}), anyTypePack}), - arena.addType(FunctionTypeVar{arena.addTypePack({numberType, stringType}), anyTypePack}), - arena.addType(FunctionTypeVar{arena.addTypePack({anyFunction, stringType}), anyTypePack}), - })}}, - {"traceback", {makeIntersection(arena, - { - makeFunction(arena, std::nullopt, {optionalString, optionalNumber}, {stringType}), - makeFunction(arena, std::nullopt, {typeChecker.threadType, optionalString, optionalNumber}, {stringType}), - })}}, - }; - - assignPropDocumentationSymbols(debugLib, "@luau/global/debug"); - addGlobalBinding(typeChecker, "debug", - arena.addType(TableTypeVar{debugLib, std::nullopt, typeChecker.globalScope->level, Luau::TableState::Sealed}), "@luau"); - - TableTypeVar::Props utf8Lib = { - {"char", {arena.addType(FunctionTypeVar{listOfAtLeastOneNumber, oneStringPack})}}, // FIXME - {"charpattern", {stringType}}, - {"codes", {makeFunction(arena, std::nullopt, {stringType}, - {makeFunction(arena, std::nullopt, {stringType, numberType}, {numberType, numberType}), stringType, numberType})}}, - {"codepoint", - {arena.addType(FunctionTypeVar{arena.addTypePack({stringType, optionalNumber, optionalNumber}), listOfAtLeastOneNumber})}}, // FIXME - {"len", {makeFunction(arena, std::nullopt, {stringType, optionalNumber, optionalNumber}, {optionalNumber, numberType})}}, - {"offset", {makeFunction(arena, std::nullopt, {stringType, optionalNumber, optionalNumber}, {numberType})}}, - {"nfdnormalize", {makeFunction(arena, std::nullopt, {stringType}, {stringType})}}, - {"graphemes", {makeFunction(arena, std::nullopt, {stringType, optionalNumber, optionalNumber}, - {makeFunction(arena, std::nullopt, {}, {numberType, numberType})})}}, - {"nfcnormalize", {makeFunction(arena, std::nullopt, {stringType}, {stringType})}}, - }; - - assignPropDocumentationSymbols(utf8Lib, "@luau/global/utf8"); - addGlobalBinding( - typeChecker, "utf8", arena.addType(TableTypeVar{utf8Lib, std::nullopt, typeChecker.globalScope->level, TableState::Sealed}), "@luau"); - - TypeId optionalV = makeOption(typeChecker, arena, genericV); - - TypeId arrayOfV = arena.addType(TableTypeVar{{}, TableIndexer(numberType, genericV), typeChecker.globalScope->level}); - - TypePackId unpackArgsPack = arena.addTypePack(TypePack{{arrayOfV, optionalNumber, optionalNumber}}); - TypePackId unpackReturnPack = arena.addTypePack(TypePack{{}, anyTypePack}); - TypeId unpackFunc = arena.addType(FunctionTypeVar{{genericV}, {}, unpackArgsPack, unpackReturnPack}); - - TypeId packResult = arena.addType(TableTypeVar{ - TableTypeVar::Props{{"n", {numberType}}}, TableIndexer{numberType, numberType}, typeChecker.globalScope->level, TableState::Sealed}); - TypePackId packArgsPack = arena.addTypePack(TypePack{{}, anyTypePack}); - TypePackId packReturnPack = arena.addTypePack(TypePack{{packResult}}); - - TypeId comparator = makeFunction(arena, std::nullopt, {genericV, genericV}, {booleanType}); - TypeId optionalComparator = makeOption(typeChecker, arena, comparator); - - TypeId packFn = arena.addType(FunctionTypeVar(packArgsPack, packReturnPack)); - - TableTypeVar::Props tableLib = { - {"concat", {makeFunction(arena, std::nullopt, {genericV}, {}, {arrayOfV, optionalString, optionalNumber, optionalNumber}, {stringType})}}, - {"insert", {makeIntersection(arena, {makeFunction(arena, std::nullopt, {genericV}, {}, {arrayOfV, genericV}, {}), - makeFunction(arena, std::nullopt, {genericV}, {}, {arrayOfV, numberType, genericV}, {})})}}, - {"maxn", {makeFunction(arena, std::nullopt, {genericV}, {}, {arrayOfV}, {numberType})}}, - {"remove", {makeFunction(arena, std::nullopt, {genericV}, {}, {arrayOfV, optionalNumber}, {optionalV})}}, - {"sort", {makeFunction(arena, std::nullopt, {genericV}, {}, {arrayOfV, optionalComparator}, {})}}, - {"create", {makeFunction(arena, std::nullopt, {genericV}, {}, {numberType, optionalV}, {arrayOfV})}}, - {"find", {makeFunction(arena, std::nullopt, {genericV}, {}, {arrayOfV, genericV, optionalNumber}, {optionalNumber})}}, - - {"unpack", {unpackFunc}}, // FIXME - {"pack", {packFn}}, - - // Lua 5.0 compat - {"getn", {makeFunction(arena, std::nullopt, {genericV}, {}, {arrayOfV}, {numberType})}}, - {"foreach", {makeFunction(arena, std::nullopt, {genericK, genericV}, {}, - {mapOfKtoV, makeFunction(arena, std::nullopt, {genericK, genericV}, {})}, {})}}, - {"foreachi", {makeFunction(arena, std::nullopt, {genericV}, {}, {arrayOfV, makeFunction(arena, std::nullopt, {genericV}, {})}, {})}}, - - // backported from Lua 5.3 - {"move", {makeFunction(arena, std::nullopt, {genericV}, {}, {arrayOfV, numberType, numberType, numberType, arrayOfV}, {})}}, - - // added in Luau (borrowed from LuaJIT) - {"clear", {makeFunction(arena, std::nullopt, {genericK, genericV}, {}, {mapOfKtoV}, {})}}, - - {"freeze", {makeFunction(arena, std::nullopt, {genericK, genericV}, {}, {mapOfKtoV}, {mapOfKtoV})}}, - {"isfrozen", {makeFunction(arena, std::nullopt, {genericK, genericV}, {}, {mapOfKtoV}, {booleanType})}}, - }; - - assignPropDocumentationSymbols(tableLib, "@luau/global/table"); - addGlobalBinding( - typeChecker, "table", arena.addType(TableTypeVar{tableLib, std::nullopt, typeChecker.globalScope->level, TableState::Sealed}), "@luau"); - - TableTypeVar::Props coroutineLib = { - {"create", {makeFunction(arena, std::nullopt, {anyFunction}, {threadType})}}, - {"resume", {arena.addType(FunctionTypeVar{arena.addTypePack(TypePack{{threadType}, anyTypePack}), anyTypePack})}}, - {"running", {makeFunction(arena, std::nullopt, {}, {threadType})}}, - {"status", {makeFunction(arena, std::nullopt, {threadType}, {stringType})}}, - {"wrap", {makeFunction( - arena, std::nullopt, {anyFunction}, {anyType})}}, // FIXME this technically returns a function, but we can't represent this - // atm since it can be called with different arg types at different times - {"yield", {arena.addType(FunctionTypeVar{anyTypePack, anyTypePack})}}, - {"isyieldable", {makeFunction(arena, std::nullopt, {}, {booleanType})}}, - }; - - assignPropDocumentationSymbols(coroutineLib, "@luau/global/coroutine"); - addGlobalBinding(typeChecker, "coroutine", - arena.addType(TableTypeVar{coroutineLib, std::nullopt, typeChecker.globalScope->level, TableState::Sealed}), "@luau"); - - TypeId genericT = arena.addType(GenericTypeVar{"T"}); - TypeId genericR = arena.addType(GenericTypeVar{"R"}); - - // assert returns all arguments - TypePackId assertArgs = arena.addTypePack({genericT, optionalString}); - TypePackId assertRets = arena.addTypePack({genericT}); - addGlobalBinding(typeChecker, "assert", arena.addType(FunctionTypeVar{assertArgs, assertRets}), "@luau"); - - addGlobalBinding(typeChecker, "print", arena.addType(FunctionTypeVar{anyTypePack, emptyPack}), "@luau"); - - addGlobalBinding(typeChecker, "type", makeFunction(arena, std::nullopt, {genericT}, {}, {genericT}, {stringType}), "@luau"); - addGlobalBinding(typeChecker, "typeof", makeFunction(arena, std::nullopt, {genericT}, {}, {genericT}, {stringType}), "@luau"); - - addGlobalBinding(typeChecker, "error", makeFunction(arena, std::nullopt, {genericT}, {}, {genericT, optionalNumber}, {}), "@luau"); - - addGlobalBinding(typeChecker, "tostring", makeFunction(arena, std::nullopt, {genericT}, {}, {genericT}, {stringType}), "@luau"); - addGlobalBinding( - typeChecker, "tonumber", makeFunction(arena, std::nullopt, {genericT}, {}, {genericT, optionalNumber}, {numberType}), "@luau"); - - addGlobalBinding( - typeChecker, "rawequal", makeFunction(arena, std::nullopt, {genericT, genericR}, {}, {genericT, genericR}, {booleanType}), "@luau"); - addGlobalBinding( - typeChecker, "rawget", makeFunction(arena, std::nullopt, {genericK, genericV}, {}, {mapOfKtoV, genericK}, {genericV}), "@luau"); - addGlobalBinding(typeChecker, "rawset", - makeFunction(arena, std::nullopt, {genericK, genericV}, {}, {mapOfKtoV, genericK, genericV}, {mapOfKtoV}), "@luau"); - - TypePackId genericTPack = arena.addTypePack({genericT}); - TypePackId genericRPack = arena.addTypePack({genericR}); - TypeId genericArgsToReturnFunction = arena.addType( - FunctionTypeVar{{genericT, genericR}, {}, arena.addTypePack(TypePack{{}, genericTPack}), arena.addTypePack(TypePack{{}, genericRPack})}); - - TypeId setfenvArgType = makeUnion(arena, {numberType, genericArgsToReturnFunction}); - TypeId setfenvReturnType = makeOption(typeChecker, arena, genericArgsToReturnFunction); - addGlobalBinding(typeChecker, "setfenv", makeFunction(arena, std::nullopt, {setfenvArgType, stringToAnyMap}, {setfenvReturnType}), "@luau"); - - TypePackId ipairsArgsTypePack = arena.addTypePack({arrayOfV}); - - TypeId ipairsNextFunctionType = arena.addType( - FunctionTypeVar{{genericK, genericV}, {}, arena.addTypePack({arrayOfV, numberType}), arena.addTypePack({numberType, genericV})}); - - // ipairs returns 'next, Array, 0' so we would need type-level primitives and change to - // again, we have a direct reference to 'next' because ipairs returns it - // ipairs(t: Array) -> ((Array) -> (number, V), Array, 0) - TypePackId ipairsReturnTypePack = arena.addTypePack(TypePack{{ipairsNextFunctionType, arrayOfV, numberType}}); - - // ipairs(t: Array) -> ((Array) -> (number, V), Array, number) - addGlobalBinding(typeChecker, "ipairs", arena.addType(FunctionTypeVar{{genericV}, {}, ipairsArgsTypePack, ipairsReturnTypePack}), "@luau"); - - TypePackId pcallArg0FnArgs = arena.addTypePack(TypePackVar{GenericTypeVar{"A"}}); - TypePackId pcallArg0FnRet = arena.addTypePack(TypePackVar{GenericTypeVar{"R"}}); - TypeId pcallArg0 = arena.addType(FunctionTypeVar{pcallArg0FnArgs, pcallArg0FnRet}); - TypePackId pcallArgsTypePack = arena.addTypePack(TypePack{{pcallArg0}, pcallArg0FnArgs}); - - TypePackId pcallReturnTypePack = arena.addTypePack(TypePack{{booleanType}, pcallArg0FnRet}); - - // pcall(f: (A...) -> R..., args: A...) -> boolean, R... - addGlobalBinding(typeChecker, "pcall", - arena.addType(FunctionTypeVar{{}, {pcallArg0FnArgs, pcallArg0FnRet}, pcallArgsTypePack, pcallReturnTypePack}), "@luau"); - - // errors thrown by the function 'f' are propagated onto the function 'err' that accepts it. - // and either 'f' or 'err' are valid results of this xpcall - // if 'err' did throw an error, then it returns: false, "error in error handling" - // TODO: the above is not represented (nor representable) in the type annotation below. - // - // The real type of xpcall is as such: (f: (A...) -> R1..., err: (E) -> R2..., A...) -> (true, R1...) | (false, - // R2...) - TypePackId genericAPack = arena.addTypePack(TypePackVar{GenericTypeVar{"A"}}); - TypePackId genericR1Pack = arena.addTypePack(TypePackVar{GenericTypeVar{"R1"}}); - TypePackId genericR2Pack = arena.addTypePack(TypePackVar{GenericTypeVar{"R2"}}); - - TypeId genericE = arena.addType(GenericTypeVar{"E"}); - - TypeId xpcallFArg = arena.addType(FunctionTypeVar{genericAPack, genericR1Pack}); - TypeId xpcallErrArg = arena.addType(FunctionTypeVar{arena.addTypePack({genericE}), genericR2Pack}); - - TypePackId xpcallArgsPack = arena.addTypePack({{xpcallFArg, xpcallErrArg}, genericAPack}); - TypePackId xpcallRetPack = arena.addTypePack({{booleanType}, genericR1Pack}); // FIXME - - addGlobalBinding(typeChecker, "xpcall", - arena.addType(FunctionTypeVar{{genericE}, {genericAPack, genericR1Pack, genericR2Pack}, xpcallArgsPack, xpcallRetPack}), "@luau"); - - addGlobalBinding(typeChecker, "unpack", unpackFunc, "@luau"); - - TypePackId selectArgsTypePack = arena.addTypePack(TypePack{ - {stringOrNumber}, - anyTypePack // FIXME? select() is tricky. - }); - - addGlobalBinding(typeChecker, "select", arena.addType(FunctionTypeVar{selectArgsTypePack, anyTypePack}), "@luau"); - - // TODO: not completely correct. loadstring's return type should be a function or (nil, string) - TypeId loadstringFunc = arena.addType(FunctionTypeVar{anyTypePack, oneAnyPack}); - - addGlobalBinding(typeChecker, "loadstring", - makeFunction(arena, std::nullopt, {stringType, optionalString}, - { - makeOption(typeChecker, arena, loadstringFunc), - makeOption(typeChecker, arena, stringType), - }), - "@luau"); - - // a userdata object is "roughly" the same as a sealed empty table - // except `type(newproxy(false))` evaluates to "userdata" so we may need another special type here too. - // another important thing to note: the value passed in conditionally creates an empty metatable, and you have to use getmetatable, NOT - // setmetatable. - // TODO: change this to something Luau can understand how to reject `setmetatable(newproxy(false or true), {})`. - TypeId sealedTable = arena.addType(TableTypeVar(TableState::Sealed, typeChecker.globalScope->level)); - addGlobalBinding(typeChecker, "newproxy", makeFunction(arena, std::nullopt, {optionalBoolean}, {sealedTable}), "@luau"); - } - // next(t: Table, i: K | nil) -> (K, V) TypePackId nextArgsTypePack = arena.addTypePack(TypePack{{mapOfKtoV, makeOption(typeChecker, arena, genericK)}}); addGlobalBinding(typeChecker, "next", @@ -475,8 +238,7 @@ void registerBuiltinTypes(TypeChecker& typeChecker) TypePackId pairsArgsTypePack = arena.addTypePack({mapOfKtoV}); - TypeId pairsNext = (FFlag::LuauRankNTypes ? arena.addType(FunctionTypeVar{nextArgsTypePack, arena.addTypePack(TypePack{{genericK, genericV}})}) - : getGlobalBinding(typeChecker, "next")); + TypeId pairsNext = arena.addType(FunctionTypeVar{nextArgsTypePack, arena.addTypePack(TypePack{{genericK, genericV}})}); TypePackId pairsReturnTypePack = arena.addTypePack(TypePack{{pairsNext, mapOfKtoV, nilType}}); // NOTE we are missing 'i: K | nil' argument in the first return types' argument. @@ -711,7 +473,7 @@ static std::optional> magicFunctionRequire( if (!checkRequirePath(typechecker, expr.args.data[0])) return std::nullopt; - const AstExpr* require = FFlag::LuauNewRequireTrace ? &expr : expr.args.data[0]; + const AstExpr* require = FFlag::LuauNewRequireTrace2 ? &expr : expr.args.data[0]; if (auto moduleInfo = typechecker.resolver->resolveModuleInfo(typechecker.currentModuleName, *require)) return ExprResult{arena.addTypePack({typechecker.checkRequire(scope, *moduleInfo, expr.location)})}; diff --git a/Analysis/src/EmbeddedBuiltinDefinitions.cpp b/Analysis/src/EmbeddedBuiltinDefinitions.cpp index 1e91561a..96703ef1 100644 --- a/Analysis/src/EmbeddedBuiltinDefinitions.cpp +++ b/Analysis/src/EmbeddedBuiltinDefinitions.cpp @@ -1,9 +1,6 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #include "Luau/BuiltinDefinitions.h" -LUAU_FASTFLAG(LuauParseGenericFunctions) -LUAU_FASTFLAG(LuauGenericFunctions) - namespace Luau { @@ -19,6 +16,8 @@ declare bit32: { bnot: (number) -> number, extract: (number, number, number?) -> number, replace: (number, number, number, number?) -> number, + countlz: (number) -> number, + countrz: (number) -> number, } declare math: { @@ -103,15 +102,6 @@ declare _VERSION: string declare function gcinfo(): number -)BUILTIN_SRC"; - -std::string getBuiltinDefinitionSource() -{ - std::string src = kBuiltinDefinitionLuaSrc; - - if (FFlag::LuauParseGenericFunctions && FFlag::LuauGenericFunctions) - { - src += R"( declare function print(...: T...) declare function type(value: T): string @@ -208,10 +198,12 @@ std::string getBuiltinDefinitionSource() -- Cannot use `typeof` here because it will produce a polytype when we expect a monotype. declare function unpack(tab: {V}, i: number?, j: number?): ...V - )"; - } - return src; +)BUILTIN_SRC"; + +std::string getBuiltinDefinitionSource() +{ + return kBuiltinDefinitionLuaSrc; } } // namespace Luau diff --git a/Analysis/src/Error.cpp b/Analysis/src/Error.cpp index 04d91444..46ff2c72 100644 --- a/Analysis/src/Error.cpp +++ b/Analysis/src/Error.cpp @@ -94,8 +94,23 @@ struct ErrorConverter { std::string operator()(const Luau::TypeMismatch& tm) const { - ToStringOptions opts; - return "Type '" + Luau::toString(tm.givenType, opts) + "' could not be converted into '" + Luau::toString(tm.wantedType, opts) + "'"; + std::string result = "Type '" + Luau::toString(tm.givenType) + "' could not be converted into '" + Luau::toString(tm.wantedType) + "'"; + + if (tm.error) + { + result += "\ncaused by:\n "; + + if (!tm.reason.empty()) + result += tm.reason + ". "; + + result += Luau::toString(*tm.error); + } + else if (!tm.reason.empty()) + { + result += "; " + tm.reason; + } + + return result; } std::string operator()(const Luau::UnknownSymbol& e) const @@ -478,9 +493,36 @@ struct InvalidNameChecker } }; +TypeMismatch::TypeMismatch(TypeId wantedType, TypeId givenType) + : wantedType(wantedType) + , givenType(givenType) +{ +} + +TypeMismatch::TypeMismatch(TypeId wantedType, TypeId givenType, std::string reason) + : wantedType(wantedType) + , givenType(givenType) + , reason(reason) +{ +} + +TypeMismatch::TypeMismatch(TypeId wantedType, TypeId givenType, std::string reason, TypeError error) + : wantedType(wantedType) + , givenType(givenType) + , reason(reason) + , error(std::make_shared(std::move(error))) +{ +} + bool TypeMismatch::operator==(const TypeMismatch& rhs) const { - return *wantedType == *rhs.wantedType && *givenType == *rhs.givenType; + if (!!error != !!rhs.error) + return false; + + if (error && !(*error == *rhs.error)) + return false; + + return *wantedType == *rhs.wantedType && *givenType == *rhs.givenType && reason == rhs.reason; } bool UnknownSymbol::operator==(const UnknownSymbol& rhs) const @@ -690,130 +732,141 @@ bool containsParseErrorName(const TypeError& error) return Luau::visit(InvalidNameChecker{}, error.data); } -void copyErrors(ErrorVec& errors, struct TypeArena& destArena) +template +void copyError(T& e, TypeArena& destArena, SeenTypes& seenTypes, SeenTypePacks& seenTypePacks) { - SeenTypes seenTypes; - SeenTypePacks seenTypePacks; - auto clone = [&](auto&& ty) { return ::Luau::clone(ty, destArena, seenTypes, seenTypePacks); }; auto visitErrorData = [&](auto&& e) { - using T = std::decay_t; + copyError(e, destArena, seenTypes, seenTypePacks); + }; - if constexpr (false) - { - } - else if constexpr (std::is_same_v) - { - e.wantedType = clone(e.wantedType); - e.givenType = clone(e.givenType); - } - else if constexpr (std::is_same_v) - { - } - else if constexpr (std::is_same_v) - { - e.table = clone(e.table); - } - else if constexpr (std::is_same_v) - { - e.ty = clone(e.ty); - } - else if constexpr (std::is_same_v) - { - e.tableType = clone(e.tableType); - } - else if constexpr (std::is_same_v) - { - e.tableType = clone(e.tableType); - } - else if constexpr (std::is_same_v) - { - } - else if constexpr (std::is_same_v) - { - } - else if constexpr (std::is_same_v) - { - } - else if constexpr (std::is_same_v) - { - } - else if constexpr (std::is_same_v) - { - } - else if constexpr (std::is_same_v) - { - } - else if constexpr (std::is_same_v) - { - e.typeFun = clone(e.typeFun); - } - else if constexpr (std::is_same_v) - { - } - else if constexpr (std::is_same_v) - { - } - else if constexpr (std::is_same_v) - { - } - else if constexpr (std::is_same_v) - { - e.table = clone(e.table); - } - else if constexpr (std::is_same_v) - { - } - else if constexpr (std::is_same_v) - { - e.ty = clone(e.ty); - } - else if constexpr (std::is_same_v) - { - } - else if constexpr (std::is_same_v) - { - } - else if constexpr (std::is_same_v) - { - } - else if constexpr (std::is_same_v) - { - } - else if constexpr (std::is_same_v) - { - e.expectedReturnType = clone(e.expectedReturnType); - } - else if constexpr (std::is_same_v) - { - } - else if constexpr (std::is_same_v) - { - } - else if constexpr (std::is_same_v) - { - e.superType = clone(e.superType); - e.subType = clone(e.subType); - } - else if constexpr (std::is_same_v) - { - } - else if constexpr (std::is_same_v) - { - e.optional = clone(e.optional); - } - else if constexpr (std::is_same_v) - { - e.type = clone(e.type); + if constexpr (false) + { + } + else if constexpr (std::is_same_v) + { + e.wantedType = clone(e.wantedType); + e.givenType = clone(e.givenType); - for (auto& ty : e.missing) - ty = clone(ty); - } - else - static_assert(always_false_v, "Non-exhaustive type switch"); + if (e.error) + visit(visitErrorData, e.error->data); + } + else if constexpr (std::is_same_v) + { + } + else if constexpr (std::is_same_v) + { + e.table = clone(e.table); + } + else if constexpr (std::is_same_v) + { + e.ty = clone(e.ty); + } + else if constexpr (std::is_same_v) + { + e.tableType = clone(e.tableType); + } + else if constexpr (std::is_same_v) + { + e.tableType = clone(e.tableType); + } + else if constexpr (std::is_same_v) + { + } + else if constexpr (std::is_same_v) + { + } + else if constexpr (std::is_same_v) + { + } + else if constexpr (std::is_same_v) + { + } + else if constexpr (std::is_same_v) + { + } + else if constexpr (std::is_same_v) + { + } + else if constexpr (std::is_same_v) + { + e.typeFun = clone(e.typeFun); + } + else if constexpr (std::is_same_v) + { + } + else if constexpr (std::is_same_v) + { + } + else if constexpr (std::is_same_v) + { + } + else if constexpr (std::is_same_v) + { + e.table = clone(e.table); + } + else if constexpr (std::is_same_v) + { + } + else if constexpr (std::is_same_v) + { + e.ty = clone(e.ty); + } + else if constexpr (std::is_same_v) + { + } + else if constexpr (std::is_same_v) + { + } + else if constexpr (std::is_same_v) + { + } + else if constexpr (std::is_same_v) + { + } + else if constexpr (std::is_same_v) + { + e.expectedReturnType = clone(e.expectedReturnType); + } + else if constexpr (std::is_same_v) + { + } + else if constexpr (std::is_same_v) + { + } + else if constexpr (std::is_same_v) + { + e.superType = clone(e.superType); + e.subType = clone(e.subType); + } + else if constexpr (std::is_same_v) + { + } + else if constexpr (std::is_same_v) + { + e.optional = clone(e.optional); + } + else if constexpr (std::is_same_v) + { + e.type = clone(e.type); + + for (auto& ty : e.missing) + ty = clone(ty); + } + else + static_assert(always_false_v, "Non-exhaustive type switch"); +} + +void copyErrors(ErrorVec& errors, TypeArena& destArena) +{ + SeenTypes seenTypes; + SeenTypePacks seenTypePacks; + + auto visitErrorData = [&](auto&& e) { + copyError(e, destArena, seenTypes, seenTypePacks); }; LUAU_ASSERT(!destArena.typeVars.isFrozen()); diff --git a/Analysis/src/Frontend.cpp b/Analysis/src/Frontend.cpp index 5e7af50c..2f411274 100644 --- a/Analysis/src/Frontend.cpp +++ b/Analysis/src/Frontend.cpp @@ -18,11 +18,10 @@ LUAU_FASTFLAG(LuauInferInNoCheckMode) LUAU_FASTFLAGVARIABLE(LuauTypeCheckTwice, false) LUAU_FASTFLAGVARIABLE(LuauKnowsTheDataModel3, false) -LUAU_FASTFLAGVARIABLE(LuauSecondTypecheckKnowsTheDataModel, false) LUAU_FASTFLAGVARIABLE(LuauResolveModuleNameWithoutACurrentModule, false) LUAU_FASTFLAG(LuauTraceRequireLookupChild) LUAU_FASTFLAGVARIABLE(LuauPersistDefinitionFileTypes, false) -LUAU_FASTFLAG(LuauNewRequireTrace) +LUAU_FASTFLAG(LuauNewRequireTrace2) LUAU_FASTFLAGVARIABLE(LuauClearScopes, false) namespace Luau @@ -415,7 +414,7 @@ CheckResult Frontend::check(const ModuleName& name) // If we're typechecking twice, we do so. // The second typecheck is always in strict mode with DM awareness // to provide better typen information for IDE features. - if (options.typecheckTwice && FFlag::LuauSecondTypecheckKnowsTheDataModel) + if (options.typecheckTwice) { ModulePtr moduleForAutocomplete = typeCheckerForAutocomplete.check(sourceModule, Mode::Strict); moduleResolverForAutocomplete.modules[moduleName] = moduleForAutocomplete; @@ -897,7 +896,7 @@ std::optional FrontendModuleResolver::resolveModuleInfo(const Module const auto& exprs = it->second.exprs; const ModuleInfo* info = exprs.find(&pathExpr); - if (!info || (!FFlag::LuauNewRequireTrace && info->name.empty())) + if (!info || (!FFlag::LuauNewRequireTrace2 && info->name.empty())) return std::nullopt; return *info; @@ -914,7 +913,7 @@ const ModulePtr FrontendModuleResolver::getModule(const ModuleName& moduleName) bool FrontendModuleResolver::moduleExists(const ModuleName& moduleName) const { - if (FFlag::LuauNewRequireTrace) + if (FFlag::LuauNewRequireTrace2) return frontend->sourceNodes.count(moduleName) != 0; else return frontend->fileResolver->moduleExists(moduleName); diff --git a/Analysis/src/Linter.cpp b/Analysis/src/Linter.cpp index bff947a5..1a5b24fe 100644 --- a/Analysis/src/Linter.cpp +++ b/Analysis/src/Linter.cpp @@ -12,9 +12,6 @@ #include #include -LUAU_FASTFLAGVARIABLE(LuauLinterUnknownTypeVectorAware, false) -LUAU_FASTFLAGVARIABLE(LuauLinterTableMoveZero, false) - namespace Luau { @@ -1110,10 +1107,7 @@ private: if (g && g->name == "type") { - if (FFlag::LuauLinterUnknownTypeVectorAware) - validateType(arg, {Kind_Primitive, Kind_Vector}, "primitive type"); - else - validateType(arg, {Kind_Primitive}, "primitive type"); + validateType(arg, {Kind_Primitive, Kind_Vector}, "primitive type"); } else if (g && g->name == "typeof") { @@ -2146,7 +2140,7 @@ private: "wrap it in parentheses to silence"); } - if (FFlag::LuauLinterTableMoveZero && func->index == "move" && node->args.size >= 4) + if (func->index == "move" && node->args.size >= 4) { // table.move(t, 0, _, _) if (isConstant(args[1], 0.0)) diff --git a/Analysis/src/Module.cpp b/Analysis/src/Module.cpp index 2fd95896..880ffd2e 100644 --- a/Analysis/src/Module.cpp +++ b/Analysis/src/Module.cpp @@ -12,7 +12,6 @@ LUAU_FASTFLAGVARIABLE(DebugLuauFreezeArena, false) LUAU_FASTFLAGVARIABLE(DebugLuauTrackOwningArena, false) -LUAU_FASTFLAG(LuauSecondTypecheckKnowsTheDataModel) LUAU_FASTFLAG(LuauCaptureBrokenCommentSpans) LUAU_FASTFLAG(LuauTypeAliasPacks) LUAU_FASTFLAGVARIABLE(LuauCloneBoundTables, false) @@ -290,9 +289,7 @@ void TypeCloner::operator()(const FunctionTypeVar& t) for (TypePackId genericPack : t.genericPacks) ftv->genericPacks.push_back(clone(genericPack, dest, seenTypes, seenTypePacks, encounteredFreeType)); - if (FFlag::LuauSecondTypecheckKnowsTheDataModel) - ftv->tags = t.tags; - + ftv->tags = t.tags; ftv->argTypes = clone(t.argTypes, dest, seenTypes, seenTypePacks, encounteredFreeType); ftv->argNames = t.argNames; ftv->retType = clone(t.retType, dest, seenTypes, seenTypePacks, encounteredFreeType); @@ -319,12 +316,7 @@ void TypeCloner::operator()(const TableTypeVar& t) ttv->level = TypeLevel{0, 0}; for (const auto& [name, prop] : t.props) - { - if (FFlag::LuauSecondTypecheckKnowsTheDataModel) - ttv->props[name] = {clone(prop.type, dest, seenTypes, seenTypePacks, encounteredFreeType), prop.deprecated, {}, prop.location, prop.tags}; - else - ttv->props[name] = {clone(prop.type, dest, seenTypes, seenTypePacks, encounteredFreeType), prop.deprecated, {}, prop.location}; - } + ttv->props[name] = {clone(prop.type, dest, seenTypes, seenTypePacks, encounteredFreeType), prop.deprecated, {}, prop.location, prop.tags}; if (t.indexer) ttv->indexer = TableIndexer{clone(t.indexer->indexType, dest, seenTypes, seenTypePacks, encounteredFreeType), @@ -379,10 +371,7 @@ void TypeCloner::operator()(const ClassTypeVar& t) seenTypes[typeId] = result; for (const auto& [name, prop] : t.props) - if (FFlag::LuauSecondTypecheckKnowsTheDataModel) - ctv->props[name] = {clone(prop.type, dest, seenTypes, seenTypePacks, encounteredFreeType), prop.deprecated, {}, prop.location, prop.tags}; - else - ctv->props[name] = {clone(prop.type, dest, seenTypes, seenTypePacks, encounteredFreeType), prop.deprecated, {}, prop.location}; + ctv->props[name] = {clone(prop.type, dest, seenTypes, seenTypePacks, encounteredFreeType), prop.deprecated, {}, prop.location, prop.tags}; if (t.parent) ctv->parent = clone(*t.parent, dest, seenTypes, seenTypePacks, encounteredFreeType); diff --git a/Analysis/src/Predicate.cpp b/Analysis/src/Predicate.cpp index 25e63bff..848627cf 100644 --- a/Analysis/src/Predicate.cpp +++ b/Analysis/src/Predicate.cpp @@ -3,8 +3,6 @@ #include "Luau/Ast.h" -LUAU_FASTFLAG(LuauOrPredicate) - namespace Luau { @@ -60,8 +58,6 @@ std::string toString(const LValue& lvalue) void merge(RefinementMap& l, const RefinementMap& r, std::function f) { - LUAU_ASSERT(FFlag::LuauOrPredicate); - auto itL = l.begin(); auto itR = r.begin(); while (itL != l.end() && itR != r.end()) diff --git a/Analysis/src/RequireTracer.cpp b/Analysis/src/RequireTracer.cpp index 95910b56..b72f53f9 100644 --- a/Analysis/src/RequireTracer.cpp +++ b/Analysis/src/RequireTracer.cpp @@ -5,7 +5,7 @@ #include "Luau/Module.h" LUAU_FASTFLAGVARIABLE(LuauTraceRequireLookupChild, false) -LUAU_FASTFLAGVARIABLE(LuauNewRequireTrace, false) +LUAU_FASTFLAGVARIABLE(LuauNewRequireTrace2, false) namespace Luau { @@ -19,7 +19,7 @@ struct RequireTracerOld : AstVisitor : fileResolver(fileResolver) , currentModuleName(currentModuleName) { - LUAU_ASSERT(!FFlag::LuauNewRequireTrace); + LUAU_ASSERT(!FFlag::LuauNewRequireTrace2); } FileResolver* const fileResolver; @@ -188,7 +188,7 @@ struct RequireTracer : AstVisitor , currentModuleName(currentModuleName) , locals(nullptr) { - LUAU_ASSERT(FFlag::LuauNewRequireTrace); + LUAU_ASSERT(FFlag::LuauNewRequireTrace2); } bool visit(AstExprTypeAssertion* expr) override @@ -332,7 +332,7 @@ struct RequireTracer : AstVisitor RequireTraceResult traceRequires(FileResolver* fileResolver, AstStatBlock* root, const ModuleName& currentModuleName) { - if (FFlag::LuauNewRequireTrace) + if (FFlag::LuauNewRequireTrace2) { RequireTraceResult result; RequireTracer tracer{result, fileResolver, currentModuleName}; diff --git a/Analysis/src/Substitution.cpp b/Analysis/src/Substitution.cpp index d861eb3d..ca2b30f5 100644 --- a/Analysis/src/Substitution.cpp +++ b/Analysis/src/Substitution.cpp @@ -8,8 +8,6 @@ LUAU_FASTINTVARIABLE(LuauTarjanChildLimit, 1000) LUAU_FASTFLAGVARIABLE(LuauSubstitutionDontReplaceIgnoredTypes, false) -LUAU_FASTFLAG(LuauSecondTypecheckKnowsTheDataModel) -LUAU_FASTFLAG(LuauRankNTypes) LUAU_FASTFLAG(LuauTypeAliasPacks) namespace Luau @@ -19,7 +17,7 @@ void Tarjan::visitChildren(TypeId ty, int index) { ty = follow(ty); - if (FFlag::LuauRankNTypes && ignoreChildren(ty)) + if (ignoreChildren(ty)) return; if (const FunctionTypeVar* ftv = get(ty)) @@ -68,7 +66,7 @@ void Tarjan::visitChildren(TypePackId tp, int index) { tp = follow(tp); - if (FFlag::LuauRankNTypes && ignoreChildren(tp)) + if (ignoreChildren(tp)) return; if (const TypePack* tpp = get(tp)) @@ -399,8 +397,7 @@ TypeId Substitution::clone(TypeId ty) if (FFlag::LuauTypeAliasPacks) clone.instantiatedTypePackParams = ttv->instantiatedTypePackParams; - if (FFlag::LuauSecondTypecheckKnowsTheDataModel) - clone.tags = ttv->tags; + clone.tags = ttv->tags; result = addType(std::move(clone)); } else if (const MetatableTypeVar* mtv = get(ty)) @@ -486,7 +483,7 @@ void Substitution::replaceChildren(TypeId ty) { ty = follow(ty); - if (FFlag::LuauRankNTypes && ignoreChildren(ty)) + if (ignoreChildren(ty)) return; if (FunctionTypeVar* ftv = getMutable(ty)) @@ -535,7 +532,7 @@ void Substitution::replaceChildren(TypePackId tp) { tp = follow(tp); - if (FFlag::LuauRankNTypes && ignoreChildren(tp)) + if (ignoreChildren(tp)) return; if (TypePack* tpp = getMutable(tp)) diff --git a/Analysis/src/ToString.cpp b/Analysis/src/ToString.cpp index cd8180db..885fd489 100644 --- a/Analysis/src/ToString.cpp +++ b/Analysis/src/ToString.cpp @@ -11,7 +11,6 @@ #include LUAU_FASTFLAG(LuauOccursCheckOkWithRecursiveFunctions) -LUAU_FASTFLAGVARIABLE(LuauInstantiatedTypeParamRecursion, false) LUAU_FASTFLAG(LuauTypeAliasPacks) namespace Luau @@ -237,15 +236,6 @@ struct TypeVarStringifier return; } - if (!FFlag::LuauAddMissingFollow) - { - if (get(tv)) - { - state.emit(state.getName(tv)); - return; - } - } - Luau::visit( [this, tv](auto&& t) { return (*this)(tv, t); @@ -316,11 +306,7 @@ struct TypeVarStringifier void operator()(TypeId ty, const Unifiable::Free& ftv) { state.result.invalid = true; - - if (FFlag::LuauAddMissingFollow) - state.emit(state.getName(ty)); - else - state.emit(""); + state.emit(state.getName(ty)); } void operator()(TypeId, const BoundTypeVar& btv) @@ -724,16 +710,6 @@ struct TypePackStringifier return; } - if (!FFlag::LuauAddMissingFollow) - { - if (get(tp)) - { - state.emit(state.getName(tp)); - state.emit("..."); - return; - } - } - auto it = state.cycleTpNames.find(tp); if (it != state.cycleTpNames.end()) { @@ -821,16 +797,8 @@ struct TypePackStringifier void operator()(TypePackId tp, const FreeTypePack& pack) { state.result.invalid = true; - - if (FFlag::LuauAddMissingFollow) - { - state.emit(state.getName(tp)); - state.emit("..."); - } - else - { - state.emit(""); - } + state.emit(state.getName(tp)); + state.emit("..."); } void operator()(TypePackId, const BoundTypePack& btv) @@ -864,23 +832,15 @@ static void assignCycleNames(const std::unordered_set& cycles, const std std::string name; // TODO: use the stringified type list if there are no cycles - if (FFlag::LuauInstantiatedTypeParamRecursion) + if (auto ttv = get(follow(cycleTy)); !exhaustive && ttv && (ttv->syntheticName || ttv->name)) { - if (auto ttv = get(follow(cycleTy)); !exhaustive && ttv && (ttv->syntheticName || ttv->name)) - { - // If we have a cycle type in type parameters, assign a cycle name for this named table - if (std::find_if(ttv->instantiatedTypeParams.begin(), ttv->instantiatedTypeParams.end(), [&](auto&& el) { - return cycles.count(follow(el)); - }) != ttv->instantiatedTypeParams.end()) - cycleNames[cycleTy] = ttv->name ? *ttv->name : *ttv->syntheticName; + // If we have a cycle type in type parameters, assign a cycle name for this named table + if (std::find_if(ttv->instantiatedTypeParams.begin(), ttv->instantiatedTypeParams.end(), [&](auto&& el) { + return cycles.count(follow(el)); + }) != ttv->instantiatedTypeParams.end()) + cycleNames[cycleTy] = ttv->name ? *ttv->name : *ttv->syntheticName; - continue; - } - } - else - { - if (auto ttv = get(follow(cycleTy)); !exhaustive && ttv && (ttv->syntheticName || ttv->name)) - continue; + continue; } name = "t" + std::to_string(nextIndex); @@ -912,58 +872,6 @@ ToStringResult toStringDetailed(TypeId ty, const ToStringOptions& opts) ToStringResult result; - if (!FFlag::LuauInstantiatedTypeParamRecursion && !opts.exhaustive) - { - if (auto ttv = get(ty); ttv && (ttv->name || ttv->syntheticName)) - { - if (ttv->syntheticName) - result.invalid = true; - - // If scope if provided, add module name and check visibility - if (ttv->name && opts.scope) - { - auto [success, moduleName] = canUseTypeNameInScope(opts.scope, *ttv->name); - - if (!success) - result.invalid = true; - - if (moduleName) - result.name = format("%s.", moduleName->c_str()); - } - - result.name += ttv->name ? *ttv->name : *ttv->syntheticName; - - if (ttv->instantiatedTypeParams.empty() && (!FFlag::LuauTypeAliasPacks || ttv->instantiatedTypePackParams.empty())) - return result; - - std::vector params; - for (TypeId tp : ttv->instantiatedTypeParams) - params.push_back(toString(tp)); - - if (FFlag::LuauTypeAliasPacks) - { - // Doesn't preserve grouping of multiple type packs - // But this is under a parent block of code that is being removed later - for (TypePackId tp : ttv->instantiatedTypePackParams) - { - std::string content = toString(tp); - - if (!content.empty()) - params.push_back(std::move(content)); - } - } - - result.name += "<" + join(params, ", ") + ">"; - return result; - } - else if (auto mtv = get(ty); mtv && mtv->syntheticName) - { - result.invalid = true; - result.name = *mtv->syntheticName; - return result; - } - } - StringifierState state{opts, result, opts.nameMap}; std::unordered_set cycles; @@ -975,7 +883,7 @@ ToStringResult toStringDetailed(TypeId ty, const ToStringOptions& opts) TypeVarStringifier tvs{state}; - if (FFlag::LuauInstantiatedTypeParamRecursion && !opts.exhaustive) + if (!opts.exhaustive) { if (auto ttv = get(ty); ttv && (ttv->name || ttv->syntheticName)) { diff --git a/Analysis/src/Transpiler.cpp b/Analysis/src/Transpiler.cpp index 1b83ccdc..7d880af4 100644 --- a/Analysis/src/Transpiler.cpp +++ b/Analysis/src/Transpiler.cpp @@ -10,7 +10,6 @@ #include #include -LUAU_FASTFLAG(LuauGenericFunctions) LUAU_FASTFLAG(LuauTypeAliasPacks) namespace @@ -97,9 +96,6 @@ struct Writer { virtual ~Writer() {} - virtual void begin() {} - virtual void end() {} - virtual void advance(const Position&) = 0; virtual void newline() = 0; virtual void space() = 0; @@ -131,6 +127,7 @@ struct StringWriter : Writer if (pos.column < newPos.column) write(std::string(newPos.column - pos.column, ' ')); } + void maybeSpace(const Position& newPos, int reserve) override { if (pos.column + reserve < newPos.column) @@ -279,11 +276,14 @@ struct Printer writer.identifier(func->index.value); } - void visualizeTypePackAnnotation(const AstTypePack& annotation) + void visualizeTypePackAnnotation(const AstTypePack& annotation, bool forVarArg) { + advance(annotation.location.begin); if (const AstTypePackVariadic* variadicTp = annotation.as()) { - writer.symbol("..."); + if (!forVarArg) + writer.symbol("..."); + visualizeTypeAnnotation(*variadicTp->variadicType); } else if (const AstTypePackGeneric* genericTp = annotation.as()) @@ -293,6 +293,7 @@ struct Printer } else if (const AstTypePackExplicit* explicitTp = annotation.as()) { + LUAU_ASSERT(!forVarArg); visualizeTypeList(explicitTp->typeList, true); } else @@ -317,7 +318,7 @@ struct Printer // Only variadic tail if (list.types.size == 0) { - visualizeTypePackAnnotation(*list.tailType); + visualizeTypePackAnnotation(*list.tailType, false); } else { @@ -345,7 +346,7 @@ struct Printer if (list.tailType) { writer.symbol(","); - visualizeTypePackAnnotation(*list.tailType); + visualizeTypePackAnnotation(*list.tailType, false); } writer.symbol(")"); @@ -542,6 +543,7 @@ struct Printer case AstExprBinary::CompareLt: case AstExprBinary::CompareGt: writer.maybeSpace(a->right->location.begin, 2); + writer.symbol(toString(a->op)); break; case AstExprBinary::Concat: case AstExprBinary::CompareNe: @@ -550,19 +552,35 @@ struct Printer case AstExprBinary::CompareGe: case AstExprBinary::Or: writer.maybeSpace(a->right->location.begin, 3); + writer.keyword(toString(a->op)); break; case AstExprBinary::And: writer.maybeSpace(a->right->location.begin, 4); + writer.keyword(toString(a->op)); break; } - writer.symbol(toString(a->op)); - visualize(*a->right); } else if (const auto& a = expr.as()) { visualize(*a->expr); + + if (writeTypes) + { + writer.maybeSpace(a->annotation->location.begin, 2); + writer.symbol("::"); + visualizeTypeAnnotation(*a->annotation); + } + } + else if (const auto& a = expr.as()) + { + writer.keyword("if"); + visualize(*a->condition); + writer.keyword("then"); + visualize(*a->trueExpr); + writer.keyword("else"); + visualize(*a->falseExpr); } else if (const auto& a = expr.as()) { @@ -769,24 +787,31 @@ struct Printer switch (a->op) { case AstExprBinary::Add: + writer.maybeSpace(a->value->location.begin, 2); writer.symbol("+="); break; case AstExprBinary::Sub: + writer.maybeSpace(a->value->location.begin, 2); writer.symbol("-="); break; case AstExprBinary::Mul: + writer.maybeSpace(a->value->location.begin, 2); writer.symbol("*="); break; case AstExprBinary::Div: + writer.maybeSpace(a->value->location.begin, 2); writer.symbol("/="); break; case AstExprBinary::Mod: + writer.maybeSpace(a->value->location.begin, 2); writer.symbol("%="); break; case AstExprBinary::Pow: + writer.maybeSpace(a->value->location.begin, 2); writer.symbol("^="); break; case AstExprBinary::Concat: + writer.maybeSpace(a->value->location.begin, 3); writer.symbol("..="); break; default: @@ -874,7 +899,7 @@ struct Printer void visualizeFunctionBody(AstExprFunction& func) { - if (FFlag::LuauGenericFunctions && (func.generics.size > 0 || func.genericPacks.size > 0)) + if (func.generics.size > 0 || func.genericPacks.size > 0) { CommaSeparatorInserter comma(writer); writer.symbol("<"); @@ -913,12 +938,13 @@ struct Printer if (func.vararg) { comma(); + advance(func.varargLocation.begin); writer.symbol("..."); if (func.varargAnnotation) { writer.symbol(":"); - visualizeTypePackAnnotation(*func.varargAnnotation); + visualizeTypePackAnnotation(*func.varargAnnotation, true); } } @@ -980,8 +1006,14 @@ struct Printer advance(typeAnnotation.location.begin); if (const auto& a = typeAnnotation.as()) { + if (a->hasPrefix) + { + writer.write(a->prefix.value); + writer.symbol("."); + } + writer.write(a->name.value); - if (a->parameters.size > 0) + if (a->parameters.size > 0 || a->hasParameterList) { CommaSeparatorInserter comma(writer); writer.symbol("<"); @@ -992,7 +1024,7 @@ struct Printer if (o.type) visualizeTypeAnnotation(*o.type); else - visualizeTypePackAnnotation(*o.typePack); + visualizeTypePackAnnotation(*o.typePack, false); } writer.symbol(">"); @@ -1000,7 +1032,7 @@ struct Printer } else if (const auto& a = typeAnnotation.as()) { - if (FFlag::LuauGenericFunctions && (a->generics.size > 0 || a->genericPacks.size > 0)) + if (a->generics.size > 0 || a->genericPacks.size > 0) { CommaSeparatorInserter comma(writer); writer.symbol("<"); @@ -1075,7 +1107,16 @@ struct Printer auto rta = r->as(); if (rta && rta->name == "nil") { + bool wrap = l->as() || l->as(); + + if (wrap) + writer.symbol("("); + visualizeTypeAnnotation(*l); + + if (wrap) + writer.symbol(")"); + writer.symbol("?"); return; } @@ -1089,7 +1130,15 @@ struct Printer writer.symbol("|"); } + bool wrap = a->types.data[i]->as() || a->types.data[i]->as(); + + if (wrap) + writer.symbol("("); + visualizeTypeAnnotation(*a->types.data[i]); + + if (wrap) + writer.symbol(")"); } } else if (const auto& a = typeAnnotation.as()) @@ -1102,7 +1151,15 @@ struct Printer writer.symbol("&"); } + bool wrap = a->types.data[i]->as() || a->types.data[i]->as(); + + if (wrap) + writer.symbol("("); + visualizeTypeAnnotation(*a->types.data[i]); + + if (wrap) + writer.symbol(")"); } } else if (typeAnnotation.is()) @@ -1116,31 +1173,27 @@ struct Printer } }; -void dump(AstNode* node) +std::string toString(AstNode* node) { StringWriter writer; + writer.pos = node->location.begin; + Printer printer(writer); printer.writeTypes = true; if (auto statNode = dynamic_cast(node)) - { printer.visualize(*statNode); - printf("%s\n", writer.str().c_str()); - } else if (auto exprNode = dynamic_cast(node)) - { printer.visualize(*exprNode); - printf("%s\n", writer.str().c_str()); - } else if (auto typeNode = dynamic_cast(node)) - { printer.visualizeTypeAnnotation(*typeNode); - printf("%s\n", writer.str().c_str()); - } - else - { - printf("Can't dump this node\n"); - } + + return writer.str(); +} + +void dump(AstNode* node) +{ + printf("%s\n", toString(node).c_str()); } std::string transpile(AstStatBlock& block) @@ -1149,6 +1202,7 @@ std::string transpile(AstStatBlock& block) Printer(writer).visualizeBlock(block); return writer.str(); } + std::string transpileWithTypes(AstStatBlock& block) { StringWriter writer; @@ -1158,7 +1212,7 @@ std::string transpileWithTypes(AstStatBlock& block) return writer.str(); } -TranspileResult transpile(std::string_view source, ParseOptions options) +TranspileResult transpile(std::string_view source, ParseOptions options, bool withTypes) { auto allocator = Allocator{}; auto names = AstNameTable{allocator}; @@ -1176,6 +1230,9 @@ TranspileResult transpile(std::string_view source, ParseOptions options) if (!parseResult.root) return TranspileResult{"", {}, "Internal error: Parser yielded empty parse tree"}; + if (withTypes) + return TranspileResult{transpileWithTypes(*parseResult.root)}; + return TranspileResult{transpile(*parseResult.root)}; } diff --git a/Analysis/src/TypeAttach.cpp b/Analysis/src/TypeAttach.cpp index 49f8e0ca..11aa7b39 100644 --- a/Analysis/src/TypeAttach.cpp +++ b/Analysis/src/TypeAttach.cpp @@ -13,7 +13,6 @@ #include -LUAU_FASTFLAG(LuauGenericFunctions) LUAU_FASTFLAG(LuauTypeAliasPacks) static char* allocateString(Luau::Allocator& allocator, std::string_view contents) @@ -203,39 +202,23 @@ public: return allocator->alloc(Location(), std::nullopt, AstName("")); AstArray generics; - if (FFlag::LuauGenericFunctions) + generics.size = ftv.generics.size(); + generics.data = static_cast(allocator->allocate(sizeof(AstName) * generics.size)); + size_t numGenerics = 0; + for (auto it = ftv.generics.begin(); it != ftv.generics.end(); ++it) { - generics.size = ftv.generics.size(); - generics.data = static_cast(allocator->allocate(sizeof(AstName) * generics.size)); - size_t i = 0; - for (auto it = ftv.generics.begin(); it != ftv.generics.end(); ++it) - { - if (auto gtv = get(*it)) - generics.data[i++] = AstName(gtv->name.c_str()); - } - } - else - { - generics.size = 0; - generics.data = nullptr; + if (auto gtv = get(*it)) + generics.data[numGenerics++] = AstName(gtv->name.c_str()); } AstArray genericPacks; - if (FFlag::LuauGenericFunctions) + genericPacks.size = ftv.genericPacks.size(); + genericPacks.data = static_cast(allocator->allocate(sizeof(AstName) * genericPacks.size)); + size_t numGenericPacks = 0; + for (auto it = ftv.genericPacks.begin(); it != ftv.genericPacks.end(); ++it) { - genericPacks.size = ftv.genericPacks.size(); - genericPacks.data = static_cast(allocator->allocate(sizeof(AstName) * genericPacks.size)); - size_t i = 0; - for (auto it = ftv.genericPacks.begin(); it != ftv.genericPacks.end(); ++it) - { - if (auto gtv = get(*it)) - genericPacks.data[i++] = AstName(gtv->name.c_str()); - } - } - else - { - generics.size = 0; - generics.data = nullptr; + if (auto gtv = get(*it)) + genericPacks.data[numGenericPacks++] = AstName(gtv->name.c_str()); } AstArray argTypes; diff --git a/Analysis/src/TypeInfer.cpp b/Analysis/src/TypeInfer.cpp index 38e2e527..8fad1af9 100644 --- a/Analysis/src/TypeInfer.cpp +++ b/Analysis/src/TypeInfer.cpp @@ -22,29 +22,19 @@ LUAU_FASTFLAGVARIABLE(DebugLuauMagicTypes, false) LUAU_FASTINTVARIABLE(LuauTypeInferRecursionLimit, 500) LUAU_FASTINTVARIABLE(LuauTypeInferTypePackLoopLimit, 5000) LUAU_FASTINTVARIABLE(LuauCheckRecursionLimit, 500) -LUAU_FASTFLAGVARIABLE(LuauGenericFunctions, false) -LUAU_FASTFLAGVARIABLE(LuauGenericVariadicsUnification, false) LUAU_FASTFLAG(LuauKnowsTheDataModel3) -LUAU_FASTFLAG(LuauSecondTypecheckKnowsTheDataModel) LUAU_FASTFLAGVARIABLE(LuauClassPropertyAccessAsString, false) LUAU_FASTFLAGVARIABLE(LuauEqConstraint, false) LUAU_FASTFLAGVARIABLE(LuauWeakEqConstraint, false) // Eventually removed as false. LUAU_FASTFLAG(LuauTraceRequireLookupChild) LUAU_FASTFLAGVARIABLE(LuauCloneCorrectlyBeforeMutatingTableType, false) LUAU_FASTFLAGVARIABLE(LuauStoreMatchingOverloadFnType, false) -LUAU_FASTFLAGVARIABLE(LuauRankNTypes, false) -LUAU_FASTFLAGVARIABLE(LuauOrPredicate, false) -LUAU_FASTFLAGVARIABLE(LuauInferReturnAssertAssign, false) LUAU_FASTFLAGVARIABLE(LuauRecursiveTypeParameterRestriction, false) -LUAU_FASTFLAGVARIABLE(LuauAddMissingFollow, false) -LUAU_FASTFLAGVARIABLE(LuauTypeGuardPeelsAwaySubclasses, false) -LUAU_FASTFLAGVARIABLE(LuauSlightlyMoreFlexibleBinaryPredicates, false) -LUAU_FASTFLAGVARIABLE(LuauFollowInTypeFunApply, false) LUAU_FASTFLAGVARIABLE(LuauIfElseExpressionAnalysisSupport, false) LUAU_FASTFLAGVARIABLE(LuauStrictRequire, false) LUAU_FASTFLAG(LuauSubstitutionDontReplaceIgnoredTypes) LUAU_FASTFLAGVARIABLE(LuauQuantifyInPlace2, false) -LUAU_FASTFLAG(LuauNewRequireTrace) +LUAU_FASTFLAG(LuauNewRequireTrace2) LUAU_FASTFLAG(LuauTypeAliasPacks) namespace Luau @@ -222,9 +212,9 @@ TypeChecker::TypeChecker(ModuleResolver* resolver, InternalErrorReporter* iceHan , threadType(singletonTypes.threadType) , anyType(singletonTypes.anyType) , errorType(singletonTypes.errorType) - , optionalNumberType(globalTypes.addType(UnionTypeVar{{numberType, nilType}})) - , anyTypePack(globalTypes.addTypePack(TypePackVar{VariadicTypePack{singletonTypes.anyType}, true})) - , errorTypePack(globalTypes.addTypePack(TypePackVar{Unifiable::Error{}})) + , optionalNumberType(singletonTypes.optionalNumberType) + , anyTypePack(singletonTypes.anyTypePack) + , errorTypePack(singletonTypes.errorTypePack) { globalScope = std::make_shared(globalTypes.addTypePack(TypePackVar{FreeTypePack{TypeLevel{}}})); @@ -251,10 +241,8 @@ ModulePtr TypeChecker::check(const SourceModule& module, Mode mode, std::optiona if (module.cyclic) moduleScope->returnType = addTypePack(TypePack{{anyType}, std::nullopt}); - else if (FFlag::LuauRankNTypes) - moduleScope->returnType = freshTypePack(moduleScope); else - moduleScope->returnType = DEPRECATED_freshTypePack(moduleScope, true); + moduleScope->returnType = freshTypePack(moduleScope); moduleScope->varargPack = anyTypePack; @@ -268,7 +256,7 @@ ModulePtr TypeChecker::check(const SourceModule& module, Mode mode, std::optiona checkBlock(moduleScope, *module.root); - if (get(FFlag::LuauAddMissingFollow ? follow(moduleScope->returnType) : moduleScope->returnType)) + if (get(follow(moduleScope->returnType))) moduleScope->returnType = addTypePack(TypePack{{}, std::nullopt}); else moduleScope->returnType = anyify(moduleScope, moduleScope->returnType, Location{}); @@ -326,7 +314,7 @@ void TypeChecker::check(const ScopePtr& scope, const AstStat& program) check(scope, *typealias); else if (auto global = program.as()) { - TypeId globalType = (FFlag::LuauRankNTypes ? resolveType(scope, *global->type) : resolveType(scope, *global->type, true)); + TypeId globalType = resolveType(scope, *global->type); Name globalName(global->name.value); currentModule->declaredGlobals[globalName] = globalType; @@ -494,7 +482,7 @@ LUAU_NOINLINE void TypeChecker::checkBlockTypeAliases(const ScopePtr& scope, std Name name = typealias->name.value; TypeId type = bindings[name].type; - if (get(FFlag::LuauAddMissingFollow ? follow(type) : type)) + if (get(follow(type))) { *asMutable(type) = ErrorTypeVar{}; reportError(TypeError{typealias->location, OccursCheckFailed{}}); @@ -607,26 +595,22 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatRepeat& statement) void TypeChecker::check(const ScopePtr& scope, const AstStatReturn& return_) { std::vector> expectedTypes; + expectedTypes.reserve(return_.list.size); - if (FFlag::LuauInferReturnAssertAssign) + TypePackIterator expectedRetCurr = begin(scope->returnType); + TypePackIterator expectedRetEnd = end(scope->returnType); + + for (size_t i = 0; i < return_.list.size; ++i) { - expectedTypes.reserve(return_.list.size); - - TypePackIterator expectedRetCurr = begin(scope->returnType); - TypePackIterator expectedRetEnd = end(scope->returnType); - - for (size_t i = 0; i < return_.list.size; ++i) + if (expectedRetCurr != expectedRetEnd) { - if (expectedRetCurr != expectedRetEnd) - { - expectedTypes.push_back(*expectedRetCurr); - ++expectedRetCurr; - } - else if (auto expectedArgsTail = expectedRetCurr.tail()) - { - if (const VariadicTypePack* vtp = get(follow(*expectedArgsTail))) - expectedTypes.push_back(vtp->ty); - } + expectedTypes.push_back(*expectedRetCurr); + ++expectedRetCurr; + } + else if (auto expectedArgsTail = expectedRetCurr.tail()) + { + if (const VariadicTypePack* vtp = get(follow(*expectedArgsTail))) + expectedTypes.push_back(vtp->ty); } } @@ -672,34 +656,30 @@ ErrorVec TypeChecker::tryUnify_(Id left, Id right, const Location& location) void TypeChecker::check(const ScopePtr& scope, const AstStatAssign& assign) { std::vector> expectedTypes; + expectedTypes.reserve(assign.vars.size); - if (FFlag::LuauInferReturnAssertAssign) + ScopePtr moduleScope = currentModule->getModuleScope(); + + for (size_t i = 0; i < assign.vars.size; ++i) { - expectedTypes.reserve(assign.vars.size); + AstExpr* dest = assign.vars.data[i]; - ScopePtr moduleScope = currentModule->getModuleScope(); - - for (size_t i = 0; i < assign.vars.size; ++i) + if (auto a = dest->as()) { - AstExpr* dest = assign.vars.data[i]; - - if (auto a = dest->as()) - { - // AstExprLocal l-values will have to be checked again because their type might have been mutated during checkExprList later - expectedTypes.push_back(scope->lookup(a->local)); - } - else if (auto a = dest->as()) - { - // AstExprGlobal l-values lookup is inlined here to avoid creating a global binding before checkExprList - if (auto it = moduleScope->bindings.find(a->name); it != moduleScope->bindings.end()) - expectedTypes.push_back(it->second.typeId); - else - expectedTypes.push_back(std::nullopt); - } + // AstExprLocal l-values will have to be checked again because their type might have been mutated during checkExprList later + expectedTypes.push_back(scope->lookup(a->local)); + } + else if (auto a = dest->as()) + { + // AstExprGlobal l-values lookup is inlined here to avoid creating a global binding before checkExprList + if (auto it = moduleScope->bindings.find(a->name); it != moduleScope->bindings.end()) + expectedTypes.push_back(it->second.typeId); else - { - expectedTypes.push_back(checkLValue(scope, *dest)); - } + expectedTypes.push_back(std::nullopt); + } + else + { + expectedTypes.push_back(checkLValue(scope, *dest)); } } @@ -715,7 +695,7 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatAssign& assign) AstExpr* dest = assign.vars.data[i]; TypeId left = nullptr; - if (!FFlag::LuauInferReturnAssertAssign || dest->is() || dest->is()) + if (dest->is() || dest->is()) left = checkLValue(scope, *dest); else left = *expectedTypes[i]; @@ -751,11 +731,7 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatAssign& assign) if (right) { - if (FFlag::LuauGenericFunctions && !maybeGeneric(left) && isGeneric(right)) - right = instantiate(scope, right, loc); - - if (!FFlag::LuauGenericFunctions && get(FFlag::LuauAddMissingFollow ? follow(left) : left) && - get(FFlag::LuauAddMissingFollow ? follow(right) : right)) + if (!maybeGeneric(left) && isGeneric(right)) right = instantiate(scope, right, loc); // Setting a table entry to nil doesn't mean nil is the type of the indexer, it is just deleting the entry @@ -766,7 +742,7 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatAssign& assign) if (!destTableTypeReceivingNil || !destTableTypeReceivingNil->indexer) { // In nonstrict mode, any assignments where the lhs is free and rhs isn't a function, we give it any typevar. - if (isNonstrictMode() && get(FFlag::LuauAddMissingFollow ? follow(left) : left) && !get(follow(right))) + if (isNonstrictMode() && get(follow(left)) && !get(follow(right))) unify(left, anyType, loc); else unify(left, right, loc); @@ -815,7 +791,7 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatLocal& local) if (annotation) { - ty = (FFlag::LuauRankNTypes ? resolveType(scope, *annotation) : resolveType(scope, *annotation, true)); + ty = resolveType(scope, *annotation); // If the annotation type has an error, treat it as if there was no annotation if (get(follow(ty))) @@ -823,23 +799,19 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatLocal& local) } if (!ty) - ty = rhsIsTable ? (FFlag::LuauRankNTypes ? freshType(scope) : DEPRECATED_freshType(scope, true)) - : isNonstrictMode() ? anyType : (FFlag::LuauRankNTypes ? freshType(scope) : DEPRECATED_freshType(scope, true)); + ty = rhsIsTable ? freshType(scope) : isNonstrictMode() ? anyType : freshType(scope); varBindings.emplace_back(vars[i], Binding{ty, vars[i]->location}); variableTypes.push_back(ty); expectedTypes.push_back(ty); - if (FFlag::LuauGenericFunctions) - instantiateGenerics.push_back(annotation != nullptr && !maybeGeneric(ty)); - else - instantiateGenerics.push_back(annotation != nullptr && get(FFlag::LuauAddMissingFollow ? follow(ty) : ty)); + instantiateGenerics.push_back(annotation != nullptr && !maybeGeneric(ty)); } if (local.values.size > 0) { - TypePackId variablePack = addTypePack(variableTypes, FFlag::LuauRankNTypes ? freshTypePack(scope) : DEPRECATED_freshTypePack(scope, true)); + TypePackId variablePack = addTypePack(variableTypes, freshTypePack(scope)); TypePackId valuePack = checkExprList(scope, local.location, local.values, /* substituteFreeForNil= */ true, instantiateGenerics, expectedTypes).type; @@ -979,8 +951,6 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatForIn& forin) { AstExprCall* exprCall = firstValue->as(); callRetPack = checkExprPack(scope, *exprCall).type; - if (!FFlag::LuauRankNTypes) - callRetPack = DEPRECATED_instantiate(scope, callRetPack, exprCall->location); callRetPack = follow(callRetPack); if (get(callRetPack)) @@ -998,8 +968,7 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatForIn& forin) else { iterTy = *first(callRetPack); - if (FFlag::LuauRankNTypes) - iterTy = instantiate(scope, iterTy, exprCall->location); + iterTy = instantiate(scope, iterTy, exprCall->location); } } else @@ -1158,10 +1127,7 @@ void TypeChecker::check(const ScopePtr& scope, TypeId ty, const ScopePtr& funSco checkFunctionBody(funScope, ty, *function.func); - if (FFlag::LuauGenericFunctions) - scope->bindings[function.name] = {quantify(funScope, ty, function.name->location), function.name->location}; - else - scope->bindings[function.name] = {quantify(scope, ty, function.name->location), function.name->location}; + scope->bindings[function.name] = {quantify(funScope, ty, function.name->location), function.name->location}; } void TypeChecker::check(const ScopePtr& scope, const AstStatTypeAlias& typealias, int subLevel, bool forwardDeclare) @@ -1199,7 +1165,7 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatTypeAlias& typealias { auto [generics, genericPacks] = createGenericTypes(aliasScope, scope->level, typealias, typealias.generics, typealias.genericPacks); - TypeId ty = (FFlag::LuauRankNTypes ? freshType(aliasScope) : DEPRECATED_freshType(scope, true)); + TypeId ty = freshType(aliasScope); FreeTypeVar* ftv = getMutable(ty); LUAU_ASSERT(ftv); ftv->forwardedTypeAlias = true; @@ -1234,7 +1200,7 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatTypeAlias& typealias aliasScope->privateTypeBindings[n] = TypeFun{{}, g}; } - TypeId ty = (FFlag::LuauRankNTypes ? freshType(aliasScope) : DEPRECATED_freshType(scope, true)); + TypeId ty = freshType(aliasScope); FreeTypeVar* ftv = getMutable(ty); LUAU_ASSERT(ftv); ftv->forwardedTypeAlias = true; @@ -1266,7 +1232,7 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatTypeAlias& typealias } } - TypeId ty = (FFlag::LuauRankNTypes ? resolveType(aliasScope, *typealias.type) : resolveType(aliasScope, *typealias.type, true)); + TypeId ty = resolveType(aliasScope, *typealias.type); if (auto ttv = getMutable(follow(ty))) { // If the table is already named and we want to rename the type function, we have to bind new alias to a copy @@ -1325,25 +1291,12 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatDeclareClass& declar LUAU_ASSERT(lookupType->typeParams.size() == 0 && (!FFlag::LuauTypeAliasPacks || lookupType->typePackParams.size() == 0)); superTy = lookupType->type; - if (FFlag::LuauAddMissingFollow) + if (!get(follow(*superTy))) { - if (!get(follow(*superTy))) - { - reportError(declaredClass.location, GenericError{format("Cannot use non-class type '%s' as a superclass of class '%s'", - superName.c_str(), declaredClass.name.value)}); + reportError(declaredClass.location, + GenericError{format("Cannot use non-class type '%s' as a superclass of class '%s'", superName.c_str(), declaredClass.name.value)}); - return; - } - } - else - { - if (const ClassTypeVar* superCtv = get(*superTy); !superCtv) - { - reportError(declaredClass.location, GenericError{format("Cannot use non-class type '%s' as a superclass of class '%s'", - superName.c_str(), declaredClass.name.value)}); - - return; - } + return; } } @@ -1558,8 +1511,8 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprVa } else if (auto ftp = get(varargPack)) { - TypeId head = (FFlag::LuauRankNTypes ? freshType(scope) : DEPRECATED_freshType(scope, ftp->DEPRECATED_canBeGeneric)); - TypePackId tail = (FFlag::LuauRankNTypes ? freshTypePack(scope) : DEPRECATED_freshTypePack(scope, ftp->DEPRECATED_canBeGeneric)); + TypeId head = freshType(scope); + TypePackId tail = freshTypePack(scope); *asMutable(varargPack) = TypePack{{head}, tail}; return {head}; } @@ -1567,7 +1520,7 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprVa return {errorType}; else if (auto vtp = get(varargPack)) return {vtp->ty}; - else if (FFlag::LuauGenericVariadicsUnification && get(varargPack)) + else if (get(varargPack)) { // TODO: Better error? reportError(expr.location, GenericError{"Trying to get a type from a variadic type parameter"}); @@ -1588,7 +1541,7 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprCa } else if (auto ftp = get(retPack)) { - TypeId head = (FFlag::LuauRankNTypes ? freshType(scope) : DEPRECATED_freshType(scope, ftp->DEPRECATED_canBeGeneric)); + TypeId head = freshType(scope); TypePackId pack = addTypePack(TypePackVar{TypePack{{head}, freshTypePack(scope)}}); unify(retPack, pack, expr.location); return {head, std::move(result.predicates)}; @@ -1667,7 +1620,7 @@ std::optional TypeChecker::getIndexTypeFromType( } else if (tableType->state == TableState::Free) { - TypeId result = (FFlag::LuauRankNTypes ? freshType(scope) : DEPRECATED_freshType(scope, true)); + TypeId result = freshType(scope); tableType->props[name] = {result}; return result; } @@ -2129,7 +2082,7 @@ TypeId TypeChecker::checkRelationalOperation( if (!isNonstrictMode() && !isOrOp) return ty; - if (auto i = get(ty)) + if (get(ty)) { std::optional cleaned = tryStripUnionFromNil(ty); @@ -2158,16 +2111,9 @@ TypeId TypeChecker::checkRelationalOperation( { if (expr.op == AstExprBinary::Or && subexp->op == AstExprBinary::And) { - if (FFlag::LuauSlightlyMoreFlexibleBinaryPredicates) - { - ScopePtr subScope = childScope(scope, subexp->location); - reportErrors(resolve(predicates, subScope, true)); - return unionOfTypes(rhsType, stripNil(checkExpr(subScope, *subexp->right).type, true), expr.location); - } - else - { - return unionOfTypes(rhsType, checkExpr(scope, *subexp->right).type, expr.location); - } + ScopePtr subScope = childScope(scope, subexp->location); + reportErrors(resolve(predicates, subScope, true)); + return unionOfTypes(rhsType, stripNil(checkExpr(subScope, *subexp->right).type, true), expr.location); } } @@ -2217,10 +2163,8 @@ TypeId TypeChecker::checkRelationalOperation( std::string metamethodName = opToMetaTableEntry(expr.op); - std::optional leftMetatable = - isString(lhsType) ? std::nullopt : getMetatable(FFlag::LuauAddMissingFollow ? follow(lhsType) : lhsType); - std::optional rightMetatable = - isString(rhsType) ? std::nullopt : getMetatable(FFlag::LuauAddMissingFollow ? follow(rhsType) : rhsType); + std::optional leftMetatable = isString(lhsType) ? std::nullopt : getMetatable(follow(lhsType)); + std::optional rightMetatable = isString(rhsType) ? std::nullopt : getMetatable(follow(rhsType)); if (bool(leftMetatable) != bool(rightMetatable) && leftMetatable != rightMetatable) { @@ -2266,7 +2210,7 @@ TypeId TypeChecker::checkRelationalOperation( } } - if (get(FFlag::LuauAddMissingFollow ? follow(lhsType) : lhsType) && !isEquality) + if (get(follow(lhsType)) && !isEquality) { auto name = getIdentifierOfBaseVar(expr.left); reportError(expr.location, CannotInferBinaryOperation{expr.op, name, CannotInferBinaryOperation::Comparison}); @@ -2417,12 +2361,10 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprBi resolve(lhs.predicates, innerScope, true); ExprResult rhs = checkExpr(innerScope, *expr.right); - if (!FFlag::LuauSlightlyMoreFlexibleBinaryPredicates) - resolve(rhs.predicates, innerScope, true); return {checkBinaryOperation(innerScope, expr, lhs.type, rhs.type), {AndPredicate{std::move(lhs.predicates), std::move(rhs.predicates)}}}; } - else if (FFlag::LuauOrPredicate && expr.op == AstExprBinary::Or) + else if (expr.op == AstExprBinary::Or) { ExprResult lhs = checkExpr(scope, *expr.left); @@ -2468,19 +2410,8 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprBi ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprTypeAssertion& expr) { - ExprResult result; - TypeId annotationType; - - if (FFlag::LuauInferReturnAssertAssign) - { - annotationType = (FFlag::LuauRankNTypes ? resolveType(scope, *expr.annotation) : resolveType(scope, *expr.annotation, true)); - result = checkExpr(scope, *expr.expr, annotationType); - } - else - { - result = checkExpr(scope, *expr.expr); - annotationType = (FFlag::LuauRankNTypes ? resolveType(scope, *expr.annotation) : resolveType(scope, *expr.annotation, true)); - } + TypeId annotationType = resolveType(scope, *expr.annotation); + ExprResult result = checkExpr(scope, *expr.expr, annotationType); ErrorVec errorVec = canUnify(result.type, annotationType, expr.location); if (!errorVec.empty()) @@ -2570,23 +2501,16 @@ std::pair TypeChecker::checkLValueBinding(const ScopePtr& scope if (it != moduleScope->bindings.end()) return std::pair(it->second.typeId, &it->second.typeId); - if (isNonstrictMode() || FFlag::LuauSecondTypecheckKnowsTheDataModel) - { - TypeId result = (FFlag::LuauGenericFunctions && FFlag::LuauRankNTypes ? freshType(scope) : DEPRECATED_freshType(moduleScope, true)); + TypeId result = freshType(scope); + Binding& binding = moduleScope->bindings[expr.name]; + binding = {result, expr.location}; - Binding& binding = moduleScope->bindings[expr.name]; - binding = {result, expr.location}; + // If we're in strict mode, we want to report defining a global as an error, + // but still add it to the bindings, so that autocomplete includes it in completions. + if (!isNonstrictMode()) + reportError(TypeError{expr.location, UnknownSymbol{name, UnknownSymbol::Binding}}); - // If we're in strict mode, we want to report defining a global as an error, - // but still add it to the bindings, so that autocomplete includes it in completions. - if (!isNonstrictMode()) - reportError(TypeError{expr.location, UnknownSymbol{name, UnknownSymbol::Binding}}); - - return std::pair(result, &binding.typeId); - } - - reportError(TypeError{expr.location, UnknownSymbol{name, UnknownSymbol::Binding}}); - return std::pair(errorType, nullptr); + return std::pair(result, &binding.typeId); } std::pair TypeChecker::checkLValueBinding(const ScopePtr& scope, const AstExprIndexName& expr) @@ -2611,7 +2535,7 @@ std::pair TypeChecker::checkLValueBinding(const ScopePtr& scope } else if (lhsTable->state == TableState::Unsealed || lhsTable->state == TableState::Free) { - TypeId theType = (FFlag::LuauRankNTypes ? freshType(scope) : DEPRECATED_freshType(scope, true)); + TypeId theType = freshType(scope); Property& property = lhsTable->props[name]; property.type = theType; property.location = expr.indexLocation; @@ -2683,7 +2607,7 @@ std::pair TypeChecker::checkLValueBinding(const ScopePtr& scope AstExprConstantString* value = expr.index->as(); - if (value && FFlag::LuauClassPropertyAccessAsString) + if (value) { if (const ClassTypeVar* exprClass = get(exprType)) { @@ -2714,7 +2638,7 @@ std::pair TypeChecker::checkLValueBinding(const ScopePtr& scope } else if (exprTable->state == TableState::Unsealed || exprTable->state == TableState::Free) { - TypeId resultType = (FFlag::LuauRankNTypes ? freshType(scope) : DEPRECATED_freshType(scope, true)); + TypeId resultType = freshType(scope); Property& property = exprTable->props[value->value.data]; property.type = resultType; property.location = expr.index->location; @@ -2730,7 +2654,7 @@ std::pair TypeChecker::checkLValueBinding(const ScopePtr& scope } else if (exprTable->state == TableState::Unsealed || exprTable->state == TableState::Free) { - TypeId resultType = (FFlag::LuauRankNTypes ? freshType(scope) : DEPRECATED_freshType(scope, true)); + TypeId resultType = freshType(scope); exprTable->indexer = TableIndexer{anyIfNonstrict(indexType), anyIfNonstrict(resultType)}; return std::pair(resultType, nullptr); } @@ -2758,7 +2682,7 @@ TypeId TypeChecker::checkFunctionName(const ScopePtr& scope, AstExpr& funName) } else { - TypeId ty = (FFlag::LuauRankNTypes ? freshType(scope) : DEPRECATED_freshType(scope, true)); + TypeId ty = freshType(scope); globalScope->bindings[name] = {ty, funName.location}; return ty; } @@ -2768,7 +2692,7 @@ TypeId TypeChecker::checkFunctionName(const ScopePtr& scope, AstExpr& funName) Symbol name = localName->local; Binding& binding = scope->bindings[name]; if (binding.typeId == nullptr) - binding = {(FFlag::LuauRankNTypes ? freshType(scope) : DEPRECATED_freshType(scope, true)), funName.location}; + binding = {freshType(scope), funName.location}; return binding.typeId; } @@ -2798,7 +2722,7 @@ TypeId TypeChecker::checkFunctionName(const ScopePtr& scope, AstExpr& funName) Property& property = ttv->props[name]; - property.type = (FFlag::LuauRankNTypes ? freshType(scope) : DEPRECATED_freshType(scope, true)); + property.type = freshType(scope); property.location = indexName->indexLocation; ttv->methodDefinitionLocations[name] = funName.location; return property.type; @@ -2865,22 +2789,11 @@ std::pair TypeChecker::checkFunctionSignature( expectedFunctionType = nullptr; } - std::vector generics; - std::vector genericPacks; - - if (FFlag::LuauGenericFunctions) - { - std::tie(generics, genericPacks) = createGenericTypes(funScope, std::nullopt, expr, expr.generics, expr.genericPacks); - } + auto [generics, genericPacks] = createGenericTypes(funScope, std::nullopt, expr, expr.generics, expr.genericPacks); TypePackId retPack; if (expr.hasReturnAnnotation) - { - if (FFlag::LuauGenericFunctions) - retPack = resolveTypePack(funScope, expr.returnAnnotation); - else - retPack = resolveTypePack(scope, expr.returnAnnotation); - } + retPack = resolveTypePack(funScope, expr.returnAnnotation); else if (isNonstrictMode()) retPack = anyTypePack; else if (expectedFunctionType) @@ -2889,24 +2802,17 @@ std::pair TypeChecker::checkFunctionSignature( // Do not infer 'nil' as function return type if (!tail && head.size() == 1 && isNil(head[0])) - retPack = FFlag::LuauGenericFunctions ? freshTypePack(funScope) : freshTypePack(scope); + retPack = freshTypePack(funScope); else retPack = addTypePack(head, tail); } - else if (FFlag::LuauGenericFunctions) - retPack = freshTypePack(funScope); else - retPack = freshTypePack(scope); + retPack = freshTypePack(funScope); if (expr.vararg) { if (expr.varargAnnotation) - { - if (FFlag::LuauGenericFunctions) - funScope->varargPack = resolveTypePack(funScope, *expr.varargAnnotation); - else - funScope->varargPack = resolveTypePack(scope, *expr.varargAnnotation); - } + funScope->varargPack = resolveTypePack(funScope, *expr.varargAnnotation); else { if (expectedFunctionType && !isNonstrictMode()) @@ -2963,7 +2869,7 @@ std::pair TypeChecker::checkFunctionSignature( if (local->annotation) { - argType = resolveType((FFlag::LuauGenericFunctions ? funScope : scope), *local->annotation); + argType = resolveType(funScope, *local->annotation); // If the annotation type has an error, treat it as if there was no annotation if (get(follow(argType))) @@ -3022,7 +2928,7 @@ static bool allowsNoReturnValues(const TypePackId tp) { for (TypeId ty : tp) { - if (!get(FFlag::LuauAddMissingFollow ? follow(ty) : ty)) + if (!get(follow(ty))) { return false; } @@ -3058,7 +2964,7 @@ void TypeChecker::checkFunctionBody(const ScopePtr& scope, TypeId ty, const AstE check(scope, *function.body); // We explicitly don't follow here to check if we have a 'true' free type instead of bound one - if (FFlag::LuauAddMissingFollow ? get_if(&funTy->retType->ty) : get(funTy->retType)) + if (get_if(&funTy->retType->ty)) *asMutable(funTy->retType) = TypePack{{}, std::nullopt}; bool reachesImplicitReturn = getFallthrough(function.body) != nullptr; @@ -3287,7 +3193,7 @@ void TypeChecker::checkArgumentList( return; } - else if (FFlag::LuauGenericVariadicsUnification && get(tail)) + else if (get(tail)) { // Create a type pack out of the remaining argument types // and unify it with the tail. @@ -3310,7 +3216,7 @@ void TypeChecker::checkArgumentList( return; } - else if (FFlag::LuauRankNTypes && get(tail)) + else if (get(tail)) { // For this case, we want the error span to cover every errant extra parameter Location location = state.location; @@ -3323,10 +3229,7 @@ void TypeChecker::checkArgumentList( } else { - if (FFlag::LuauRankNTypes) - unifyWithInstantiationIfNeeded(scope, *paramIter, *argIter, state); - else - state.tryUnify(*paramIter, *argIter, /*isFunctionCall*/ false); + unifyWithInstantiationIfNeeded(scope, *paramIter, *argIter, state); ++argIter; ++paramIter; } @@ -3356,9 +3259,6 @@ ExprResult TypeChecker::checkExprPack(const ScopePtr& scope, const A ice("method call expression has no 'self'"); selfType = checkExpr(scope, *indexExpr->expr).type; - if (!FFlag::LuauRankNTypes) - instantiate(scope, selfType, expr.func->location); - selfType = stripFromNilAndReport(selfType, expr.func->location); if (std::optional propTy = getIndexTypeFromType(scope, selfType, indexExpr->index.value, expr.location, true)) @@ -3393,8 +3293,7 @@ ExprResult TypeChecker::checkExprPack(const ScopePtr& scope, const A std::vector> expectedTypes = getExpectedTypesForCall(overloads, expr.args.size, expr.self); ExprResult argListResult = checkExprList(scope, expr.location, expr.args, false, {}, expectedTypes); - TypePackId argList = argListResult.type; - TypePackId argPack = (FFlag::LuauRankNTypes ? argList : DEPRECATED_instantiate(scope, argList, expr.location)); + TypePackId argPack = argListResult.type; if (get(argPack)) return ExprResult{errorTypePack}; @@ -3526,8 +3425,7 @@ std::optional> TypeChecker::checkCallOverload(const Scope metaArgLocations.insert(metaArgLocations.begin(), expr.func->location); TypeId fn = *ty; - if (FFlag::LuauRankNTypes) - fn = instantiate(scope, fn, expr.func->location); + fn = instantiate(scope, fn, expr.func->location); return checkCallOverload( scope, expr, fn, retPack, metaCallArgPack, metaCallArgs, metaArgLocations, argListResult, overloadsThatMatchArgCount, errors); @@ -3800,7 +3698,7 @@ ExprResult TypeChecker::checkExprList(const ScopePtr& scope, const L TypeId actualType = substituteFreeForNil && expr->is() ? freshType(scope) : type; - if (instantiateGenerics.size() > i && instantiateGenerics[i] && (FFlag::LuauGenericFunctions || get(actualType))) + if (instantiateGenerics.size() > i && instantiateGenerics[i]) actualType = instantiate(scope, actualType, expr->location); if (expectedType) @@ -3837,7 +3735,7 @@ TypeId TypeChecker::checkRequire(const ScopePtr& scope, const ModuleInfo& module LUAU_TIMETRACE_SCOPE("TypeChecker::checkRequire", "TypeChecker"); LUAU_TIMETRACE_ARGUMENT("moduleInfo", moduleInfo.name.c_str()); - if (FFlag::LuauNewRequireTrace && moduleInfo.name.empty()) + if (FFlag::LuauNewRequireTrace2 && moduleInfo.name.empty()) { if (FFlag::LuauStrictRequire && currentModule->mode == Mode::Strict) { @@ -3922,7 +3820,6 @@ bool TypeChecker::unify(TypePackId left, TypePackId right, const Location& locat bool TypeChecker::unifyWithInstantiationIfNeeded(const ScopePtr& scope, TypeId left, TypeId right, const Location& location) { - LUAU_ASSERT(FFlag::LuauRankNTypes); Unifier state = mkUnifier(location); unifyWithInstantiationIfNeeded(scope, left, right, state); @@ -3933,7 +3830,6 @@ bool TypeChecker::unifyWithInstantiationIfNeeded(const ScopePtr& scope, TypeId l void TypeChecker::unifyWithInstantiationIfNeeded(const ScopePtr& scope, TypeId left, TypeId right, Unifier& state) { - LUAU_ASSERT(FFlag::LuauRankNTypes); if (!maybeGeneric(right)) // Quick check to see if we definitely can't instantiate state.tryUnify(left, right, /*isFunctionCall*/ false); @@ -3973,19 +3869,7 @@ void TypeChecker::unifyWithInstantiationIfNeeded(const ScopePtr& scope, TypeId l bool Instantiation::isDirty(TypeId ty) { - if (FFlag::LuauRankNTypes) - { - if (get(ty)) - return true; - else - return false; - } - - if (const FunctionTypeVar* ftv = get(ty)) - return !ftv->generics.empty() || !ftv->genericPacks.empty(); - else if (const TableTypeVar* ttv = get(ty)) - return ttv->state == TableState::Generic; - else if (get(ty)) + if (get(ty)) return true; else return false; @@ -3993,18 +3877,11 @@ bool Instantiation::isDirty(TypeId ty) bool Instantiation::isDirty(TypePackId tp) { - if (FFlag::LuauRankNTypes) - return false; - - if (get(tp)) - return true; - else - return false; + return false; } bool Instantiation::ignoreChildren(TypeId ty) { - LUAU_ASSERT(FFlag::LuauRankNTypes); if (get(ty)) return true; else @@ -4013,63 +3890,38 @@ bool Instantiation::ignoreChildren(TypeId ty) TypeId Instantiation::clean(TypeId ty) { - LUAU_ASSERT(isDirty(ty)); + const FunctionTypeVar* ftv = get(ty); + LUAU_ASSERT(ftv); - if (const FunctionTypeVar* ftv = get(ty)) - { - FunctionTypeVar clone = FunctionTypeVar{level, ftv->argTypes, ftv->retType, ftv->definition, ftv->hasSelf}; - clone.magicFunction = ftv->magicFunction; - clone.tags = ftv->tags; - clone.argNames = ftv->argNames; - TypeId result = addType(std::move(clone)); + FunctionTypeVar clone = FunctionTypeVar{level, ftv->argTypes, ftv->retType, ftv->definition, ftv->hasSelf}; + clone.magicFunction = ftv->magicFunction; + clone.tags = ftv->tags; + clone.argNames = ftv->argNames; + TypeId result = addType(std::move(clone)); - if (FFlag::LuauRankNTypes) - { - // Annoyingly, we have to do this even if there are no generics, - // to replace any generic tables. - replaceGenerics.level = level; - replaceGenerics.currentModule = currentModule; - replaceGenerics.generics.assign(ftv->generics.begin(), ftv->generics.end()); - replaceGenerics.genericPacks.assign(ftv->genericPacks.begin(), ftv->genericPacks.end()); + // Annoyingly, we have to do this even if there are no generics, + // to replace any generic tables. + replaceGenerics.level = level; + replaceGenerics.currentModule = currentModule; + replaceGenerics.generics.assign(ftv->generics.begin(), ftv->generics.end()); + replaceGenerics.genericPacks.assign(ftv->genericPacks.begin(), ftv->genericPacks.end()); - // TODO: What to do if this returns nullopt? - // We don't have access to the error-reporting machinery - result = replaceGenerics.substitute(result).value_or(result); - } + // TODO: What to do if this returns nullopt? + // We don't have access to the error-reporting machinery + result = replaceGenerics.substitute(result).value_or(result); - asMutable(result)->documentationSymbol = ty->documentationSymbol; - return result; - } - else if (const TableTypeVar* ttv = get(ty)) - { - LUAU_ASSERT(!FFlag::LuauRankNTypes); - TableTypeVar clone = TableTypeVar{ttv->props, ttv->indexer, level, TableState::Free}; - clone.methodDefinitionLocations = ttv->methodDefinitionLocations; - clone.definitionModuleName = ttv->definitionModuleName; - TypeId result = addType(std::move(clone)); - - asMutable(result)->documentationSymbol = ty->documentationSymbol; - return result; - } - else - { - LUAU_ASSERT(!FFlag::LuauRankNTypes); - TypeId result = addType(FreeTypeVar{level}); - - asMutable(result)->documentationSymbol = ty->documentationSymbol; - return result; - } + asMutable(result)->documentationSymbol = ty->documentationSymbol; + return result; } TypePackId Instantiation::clean(TypePackId tp) { - LUAU_ASSERT(!FFlag::LuauRankNTypes); - return addTypePack(TypePackVar(FreeTypePack{level})); + LUAU_ASSERT(false); + return tp; } bool ReplaceGenerics::ignoreChildren(TypeId ty) { - LUAU_ASSERT(FFlag::LuauRankNTypes); if (const FunctionTypeVar* ftv = get(ty)) // We aren't recursing in the case of a generic function which // binds the same generics. This can happen if, for example, there's recursive types. @@ -4083,7 +3935,6 @@ bool ReplaceGenerics::ignoreChildren(TypeId ty) bool ReplaceGenerics::isDirty(TypeId ty) { - LUAU_ASSERT(FFlag::LuauRankNTypes); if (const TableTypeVar* ttv = get(ty)) return ttv->state == TableState::Generic; else if (get(ty)) @@ -4094,7 +3945,6 @@ bool ReplaceGenerics::isDirty(TypeId ty) bool ReplaceGenerics::isDirty(TypePackId tp) { - LUAU_ASSERT(FFlag::LuauRankNTypes); if (get(tp)) return std::find(genericPacks.begin(), genericPacks.end(), tp) != genericPacks.end(); else @@ -4255,21 +4105,6 @@ TypeId TypeChecker::instantiate(const ScopePtr& scope, TypeId ty, Location locat } } -TypePackId TypeChecker::DEPRECATED_instantiate(const ScopePtr& scope, TypePackId ty, Location location) -{ - LUAU_ASSERT(!FFlag::LuauRankNTypes); - instantiation.level = scope->level; - instantiation.currentModule = currentModule; - std::optional instantiated = instantiation.substitute(ty); - if (instantiated.has_value()) - return *instantiated; - else - { - reportError(location, UnificationTooComplex{}); - return errorTypePack; - } -} - TypeId TypeChecker::anyify(const ScopePtr& scope, TypeId ty, Location location) { anyification.anyType = anyType; @@ -4444,16 +4279,6 @@ TypeId TypeChecker::freshType(TypeLevel level) return currentModule->internalTypes.addType(TypeVar(FreeTypeVar(level))); } -TypeId TypeChecker::DEPRECATED_freshType(const ScopePtr& scope, bool canBeGeneric) -{ - return DEPRECATED_freshType(scope->level, canBeGeneric); -} - -TypeId TypeChecker::DEPRECATED_freshType(TypeLevel level, bool canBeGeneric) -{ - return currentModule->internalTypes.addType(TypeVar(FreeTypeVar(level, canBeGeneric))); -} - std::optional TypeChecker::filterMap(TypeId type, TypeIdPredicate predicate) { std::vector types = Luau::filterMap(type, predicate); @@ -4509,21 +4334,8 @@ TypePackId TypeChecker::freshTypePack(TypeLevel level) return addTypePack(TypePackVar(FreeTypePack(level))); } -TypePackId TypeChecker::DEPRECATED_freshTypePack(const ScopePtr& scope, bool canBeGeneric) +TypeId TypeChecker::resolveType(const ScopePtr& scope, const AstType& annotation) { - return DEPRECATED_freshTypePack(scope->level, canBeGeneric); -} - -TypePackId TypeChecker::DEPRECATED_freshTypePack(TypeLevel level, bool canBeGeneric) -{ - return addTypePack(TypePackVar(FreeTypePack(level, canBeGeneric))); -} - -TypeId TypeChecker::resolveType(const ScopePtr& scope, const AstType& annotation, bool DEPRECATED_canBeGeneric) -{ - if (DEPRECATED_canBeGeneric) - LUAU_ASSERT(!FFlag::LuauRankNTypes); - if (const auto& lit = annotation.as()) { std::optional tf; @@ -4668,11 +4480,11 @@ TypeId TypeChecker::resolveType(const ScopePtr& scope, const AstType& annotation std::optional tableIndexer; for (const auto& prop : table->props) - props[prop.name.value] = {resolveType(scope, *prop.type, DEPRECATED_canBeGeneric)}; + props[prop.name.value] = {resolveType(scope, *prop.type)}; if (const auto& indexer = table->indexer) tableIndexer = TableIndexer( - resolveType(scope, *indexer->indexType, DEPRECATED_canBeGeneric), resolveType(scope, *indexer->resultType, DEPRECATED_canBeGeneric)); + resolveType(scope, *indexer->indexType), resolveType(scope, *indexer->resultType)); return addType(TableTypeVar{ props, tableIndexer, scope->level, @@ -4683,17 +4495,7 @@ TypeId TypeChecker::resolveType(const ScopePtr& scope, const AstType& annotation { ScopePtr funcScope = childScope(scope, func->location); - std::vector generics; - std::vector genericPacks; - - if (FFlag::LuauGenericFunctions) - { - std::tie(generics, genericPacks) = createGenericTypes(funcScope, std::nullopt, annotation, func->generics, func->genericPacks); - } - - // TODO: better error message CLI-39912 - if (FFlag::LuauGenericFunctions && !FFlag::LuauRankNTypes && !DEPRECATED_canBeGeneric && (generics.size() > 0 || genericPacks.size() > 0)) - reportError(TypeError{annotation.location, GenericError{"generic function where only monotypes are allowed"}}); + auto [generics, genericPacks] = createGenericTypes(funcScope, std::nullopt, annotation, func->generics, func->genericPacks); TypePackId argTypes = resolveTypePack(funcScope, func->argTypes); TypePackId retTypes = resolveTypePack(funcScope, func->returnTypes); @@ -4716,16 +4518,13 @@ TypeId TypeChecker::resolveType(const ScopePtr& scope, const AstType& annotation else if (auto typeOf = annotation.as()) { TypeId ty = checkExpr(scope, *typeOf->expr).type; - // TODO: better error message CLI-39912 - if (FFlag::LuauGenericFunctions && !FFlag::LuauRankNTypes && !DEPRECATED_canBeGeneric && isGeneric(ty)) - reportError(TypeError{annotation.location, GenericError{"typeof produced a polytype where only monotypes are allowed"}}); return ty; } else if (const auto& un = annotation.as()) { std::vector types; for (AstType* ann : un->types) - types.push_back(resolveType(scope, *ann, DEPRECATED_canBeGeneric)); + types.push_back(resolveType(scope, *ann)); return addType(UnionTypeVar{types}); } @@ -4733,7 +4532,7 @@ TypeId TypeChecker::resolveType(const ScopePtr& scope, const AstType& annotation { std::vector types; for (AstType* ann : un->types) - types.push_back(resolveType(scope, *ann, DEPRECATED_canBeGeneric)); + types.push_back(resolveType(scope, *ann)); return addType(IntersectionTypeVar{types}); } @@ -4919,9 +4718,8 @@ TypeId TypeChecker::instantiateTypeFun(const ScopePtr& scope, const TypeFun& tf, if (FFlag::LuauCloneCorrectlyBeforeMutatingTableType) { - // TODO: CLI-46926 it's a bad idea to rename the type whether we follow through the BoundTypeVar or not - TypeId target = FFlag::LuauFollowInTypeFunApply ? follow(instantiated) : instantiated; - + // TODO: CLI-46926 it's not a good idea to rename the type here + TypeId target = follow(instantiated); bool needsClone = follow(tf.type) == target; TableTypeVar* ttv = getMutableTableType(target); @@ -5152,31 +4950,18 @@ void TypeChecker::resolve(const TruthyPredicate& truthyP, ErrorVec& errVec, Refi void TypeChecker::resolve(const AndPredicate& andP, ErrorVec& errVec, RefinementMap& refis, const ScopePtr& scope, bool sense) { - if (FFlag::LuauOrPredicate) + if (!sense) { - if (!sense) - { - OrPredicate orP{ - {NotPredicate{std::move(andP.lhs)}}, - {NotPredicate{std::move(andP.rhs)}}, - }; + OrPredicate orP{ + {NotPredicate{std::move(andP.lhs)}}, + {NotPredicate{std::move(andP.rhs)}}, + }; - return resolve(orP, errVec, refis, scope, !sense); - } - - resolve(andP.lhs, errVec, refis, scope, sense); - resolve(andP.rhs, errVec, refis, scope, sense); + return resolve(orP, errVec, refis, scope, !sense); } - else - { - // And predicate is currently not resolvable when sense is false. 'not (a and b)' is synonymous with '(not a) or (not b)'. - // TODO: implement environment merging to permit this case. - if (!sense) - return; - resolve(andP.lhs, errVec, refis, scope, sense); - resolve(andP.rhs, errVec, refis, scope, sense); - } + resolve(andP.lhs, errVec, refis, scope, sense); + resolve(andP.rhs, errVec, refis, scope, sense); } void TypeChecker::resolve(const OrPredicate& orP, ErrorVec& errVec, RefinementMap& refis, const ScopePtr& scope, bool sense) @@ -5207,58 +4992,41 @@ void TypeChecker::resolve(const OrPredicate& orP, ErrorVec& errVec, RefinementMa void TypeChecker::resolve(const IsAPredicate& isaP, ErrorVec& errVec, RefinementMap& refis, const ScopePtr& scope, bool sense) { auto predicate = [&](TypeId option) -> std::optional { - if (FFlag::LuauTypeGuardPeelsAwaySubclasses) - { - // This by itself is not truly enough to determine that A is stronger than B or vice versa. - // The best unambiguous way about this would be to have a function that returns the relationship ordering of a pair. - // i.e. TypeRelationship relationshipOf(TypeId superTy, TypeId subTy) - bool optionIsSubtype = canUnify(isaP.ty, option, isaP.location).empty(); - bool targetIsSubtype = canUnify(option, isaP.ty, isaP.location).empty(); + // This by itself is not truly enough to determine that A is stronger than B or vice versa. + // The best unambiguous way about this would be to have a function that returns the relationship ordering of a pair. + // i.e. TypeRelationship relationshipOf(TypeId superTy, TypeId subTy) + bool optionIsSubtype = canUnify(isaP.ty, option, isaP.location).empty(); + bool targetIsSubtype = canUnify(option, isaP.ty, isaP.location).empty(); - // If A is a superset of B, then if sense is true, we promote A to B, otherwise we keep A. - if (!optionIsSubtype && targetIsSubtype) + // If A is a superset of B, then if sense is true, we promote A to B, otherwise we keep A. + if (!optionIsSubtype && targetIsSubtype) + return sense ? isaP.ty : option; + + // If A is a subset of B, then if sense is true we pick A, otherwise we eliminate A. + if (optionIsSubtype && !targetIsSubtype) + return sense ? std::optional(option) : std::nullopt; + + // If neither has any relationship, we only return A if sense is false. + if (!optionIsSubtype && !targetIsSubtype) + return sense ? std::nullopt : std::optional(option); + + // If both are subtypes, then we're in one of the two situations: + // 1. Instance₁ <: Instance₂ ∧ Instance₂ <: Instance₁ + // 2. any <: Instance ∧ Instance <: any + // Right now, we have to look at the types to see if they were undecidables. + // By this point, we also know free tables are also subtypes and supertypes. + if (optionIsSubtype && targetIsSubtype) + { + // We can only have (any, Instance) because the rhs is never undecidable right now. + // So we can just return the right hand side immediately. + + // typeof(x) == "Instance" where x : any + auto ttv = get(option); + if (isUndecidable(option) || (ttv && ttv->state == TableState::Free)) return sense ? isaP.ty : option; - // If A is a subset of B, then if sense is true we pick A, otherwise we eliminate A. - if (optionIsSubtype && !targetIsSubtype) - return sense ? std::optional(option) : std::nullopt; - - // If neither has any relationship, we only return A if sense is false. - if (!optionIsSubtype && !targetIsSubtype) - return sense ? std::nullopt : std::optional(option); - - // If both are subtypes, then we're in one of the two situations: - // 1. Instance₁ <: Instance₂ ∧ Instance₂ <: Instance₁ - // 2. any <: Instance ∧ Instance <: any - // Right now, we have to look at the types to see if they were undecidables. - // By this point, we also know free tables are also subtypes and supertypes. - if (optionIsSubtype && targetIsSubtype) - { - // We can only have (any, Instance) because the rhs is never undecidable right now. - // So we can just return the right hand side immediately. - - // typeof(x) == "Instance" where x : any - auto ttv = get(option); - if (isUndecidable(option) || (ttv && ttv->state == TableState::Free)) - return sense ? isaP.ty : option; - - // typeof(x) == "Instance" where x : Instance - if (sense) - return isaP.ty; - } - } - else - { - auto lctv = get(option); - auto rctv = get(isaP.ty); - - if (isSubclass(lctv, rctv) == sense) - return option; - - if (isSubclass(rctv, lctv) == sense) - return isaP.ty; - - if (canUnify(option, isaP.ty, isaP.location).empty() == sense) + // typeof(x) == "Instance" where x : Instance + if (sense) return isaP.ty; } @@ -5457,7 +5225,7 @@ std::vector TypeChecker::unTypePack(const ScopePtr& scope, TypePackId tp TypePack* expectedPack = getMutable(expectedTypePack); LUAU_ASSERT(expectedPack); for (size_t i = 0; i < expectedLength; ++i) - expectedPack->head.push_back(FFlag::LuauRankNTypes ? freshType(scope) : DEPRECATED_freshType(scope, true)); + expectedPack->head.push_back(freshType(scope)); unify(expectedTypePack, tp, location); diff --git a/Analysis/src/TypePack.cpp b/Analysis/src/TypePack.cpp index 68a16ef0..228b1926 100644 --- a/Analysis/src/TypePack.cpp +++ b/Analysis/src/TypePack.cpp @@ -97,7 +97,7 @@ TypePackIterator begin(TypePackId tp) TypePackIterator end(TypePackId tp) { - return FFlag::LuauAddMissingFollow ? TypePackIterator{} : TypePackIterator{nullptr}; + return TypePackIterator{}; } bool areEqual(SeenSet& seen, const TypePackVar& lhs, const TypePackVar& rhs) @@ -203,7 +203,7 @@ TypePackId follow(TypePackId tp) size_t size(TypePackId tp) { - if (auto pack = get(FFlag::LuauAddMissingFollow ? follow(tp) : tp)) + if (auto pack = get(follow(tp))) return size(*pack); else return 0; @@ -216,7 +216,7 @@ bool finite(TypePackId tp) if (auto pack = get(tp)) return pack->tail ? finite(*pack->tail) : true; - if (auto pack = get(tp)) + if (get(tp)) return false; return true; @@ -227,7 +227,7 @@ size_t size(const TypePack& tp) size_t result = tp.head.size(); if (tp.tail) { - const TypePack* tail = get(FFlag::LuauAddMissingFollow ? follow(*tp.tail) : *tp.tail); + const TypePack* tail = get(follow(*tp.tail)); if (tail) result += size(*tail); } diff --git a/Analysis/src/TypeVar.cpp b/Analysis/src/TypeVar.cpp index e82f7519..cd447ca2 100644 --- a/Analysis/src/TypeVar.cpp +++ b/Analysis/src/TypeVar.cpp @@ -19,8 +19,6 @@ LUAU_FASTINTVARIABLE(LuauTypeMaximumStringifierLength, 500) LUAU_FASTINTVARIABLE(LuauTableTypeMaximumStringifierLength, 0) -LUAU_FASTFLAG(LuauRankNTypes) -LUAU_FASTFLAG(LuauTypeGuardPeelsAwaySubclasses) LUAU_FASTFLAG(LuauTypeAliasPacks) LUAU_FASTFLAGVARIABLE(LuauRefactorTagging, false) @@ -42,7 +40,7 @@ TypeId follow(TypeId t) }; auto force = [](TypeId ty) { - if (auto ltv = FFlag::LuauAddMissingFollow ? get_if(&ty->ty) : get(ty)) + if (auto ltv = get_if(&ty->ty)) { TypeId res = ltv->thunk(); if (get(res)) @@ -296,7 +294,7 @@ bool maybeGeneric(TypeId ty) { ty = follow(ty); if (auto ftv = get(ty)) - return FFlag::LuauRankNTypes || ftv->DEPRECATED_canBeGeneric; + return true; else if (auto ttv = get(ty)) { // TODO: recurse on table types CLI-39914 @@ -545,15 +543,30 @@ TypeId makeFunction(TypeArena& arena, std::optional selfType, std::initi std::initializer_list genericPacks, std::initializer_list paramTypes, std::initializer_list paramNames, std::initializer_list retTypes); +static TypeVar nilType_{PrimitiveTypeVar{PrimitiveTypeVar::NilType}, /*persistent*/ true}; +static TypeVar numberType_{PrimitiveTypeVar{PrimitiveTypeVar::Number}, /*persistent*/ true}; +static TypeVar stringType_{PrimitiveTypeVar{PrimitiveTypeVar::String}, /*persistent*/ true}; +static TypeVar booleanType_{PrimitiveTypeVar{PrimitiveTypeVar::Boolean}, /*persistent*/ true}; +static TypeVar threadType_{PrimitiveTypeVar{PrimitiveTypeVar::Thread}, /*persistent*/ true}; +static TypeVar anyType_{AnyTypeVar{}}; +static TypeVar errorType_{ErrorTypeVar{}}; +static TypeVar optionalNumberType_{UnionTypeVar{{&numberType_, &nilType_}}}; + +static TypePackVar anyTypePack_{VariadicTypePack{&anyType_}, true}; +static TypePackVar errorTypePack_{Unifiable::Error{}}; + SingletonTypes::SingletonTypes() - : arena(new TypeArena) - , nilType_{PrimitiveTypeVar{PrimitiveTypeVar::NilType}, /*persistent*/ true} - , numberType_{PrimitiveTypeVar{PrimitiveTypeVar::Number}, /*persistent*/ true} - , stringType_{PrimitiveTypeVar{PrimitiveTypeVar::String}, /*persistent*/ true} - , booleanType_{PrimitiveTypeVar{PrimitiveTypeVar::Boolean}, /*persistent*/ true} - , threadType_{PrimitiveTypeVar{PrimitiveTypeVar::Thread}, /*persistent*/ true} - , anyType_{AnyTypeVar{}} - , errorType_{ErrorTypeVar{}} + : nilType(&nilType_) + , numberType(&numberType_) + , stringType(&stringType_) + , booleanType(&booleanType_) + , threadType(&threadType_) + , anyType(&anyType_) + , errorType(&errorType_) + , optionalNumberType(&optionalNumberType_) + , anyTypePack(&anyTypePack_) + , errorTypePack(&errorTypePack_) + , arena(new TypeArena) { TypeId stringMetatable = makeStringMetatable(); stringType_.ty = PrimitiveTypeVar{PrimitiveTypeVar::String, makeStringMetatable()}; @@ -749,9 +762,9 @@ void StateDot::visitChild(TypeId ty, int parentIndex, const char* linkName) if (opts.duplicatePrimitives && canDuplicatePrimitive(ty)) { - if (const PrimitiveTypeVar* ptv = get(ty)) + if (get(ty)) formatAppend(result, "n%d [label=\"%s\"];\n", index, toStringDetailed(ty, {}).name.c_str()); - else if (const AnyTypeVar* atv = get(ty)) + else if (get(ty)) formatAppend(result, "n%d [label=\"any\"];\n", index); } else @@ -902,19 +915,19 @@ void StateDot::visitChildren(TypeId ty, int index) finishNodeLabel(ty); finishNode(); } - else if (const AnyTypeVar* atv = get(ty)) + else if (get(ty)) { formatAppend(result, "AnyTypeVar %d", index); finishNodeLabel(ty); finishNode(); } - else if (const PrimitiveTypeVar* ptv = get(ty)) + else if (get(ty)) { formatAppend(result, "PrimitiveTypeVar %s", toStringDetailed(ty, {}).name.c_str()); finishNodeLabel(ty); finishNode(); } - else if (const ErrorTypeVar* etv = get(ty)) + else if (get(ty)) { formatAppend(result, "ErrorTypeVar %d", index); finishNodeLabel(ty); @@ -994,7 +1007,7 @@ void StateDot::visitChildren(TypePackId tp, int index) finishNodeLabel(tp); finishNode(); } - else if (const Unifiable::Error* etp = get(tp)) + else if (get(tp)) { formatAppend(result, "ErrorTypePack %d", index); finishNodeLabel(tp); @@ -1372,24 +1385,6 @@ UnionTypeVarIterator end(const UnionTypeVar* utv) return UnionTypeVarIterator{}; } -static std::vector DEPRECATED_filterMap(TypeId type, TypeIdPredicate predicate) -{ - std::vector result; - - if (auto utv = get(follow(type))) - { - for (TypeId option : utv) - { - if (auto out = predicate(follow(option))) - result.push_back(*out); - } - } - else if (auto out = predicate(follow(type))) - return {*out}; - - return result; -} - static std::vector parseFormatString(TypeChecker& typechecker, const char* data, size_t size) { const char* options = "cdiouxXeEfgGqs"; @@ -1470,9 +1465,6 @@ std::optional> magicFunctionFormat( std::vector filterMap(TypeId type, TypeIdPredicate predicate) { - if (!FFlag::LuauTypeGuardPeelsAwaySubclasses) - return DEPRECATED_filterMap(type, predicate); - type = follow(type); if (auto utv = get(type)) diff --git a/Analysis/src/Unifiable.cpp b/Analysis/src/Unifiable.cpp index cef07833..dc554664 100644 --- a/Analysis/src/Unifiable.cpp +++ b/Analysis/src/Unifiable.cpp @@ -1,8 +1,6 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #include "Luau/Unifiable.h" -LUAU_FASTFLAG(LuauRankNTypes) - namespace Luau { namespace Unifiable @@ -14,14 +12,6 @@ Free::Free(TypeLevel level) { } -Free::Free(TypeLevel level, bool DEPRECATED_canBeGeneric) - : index(++nextIndex) - , level(level) - , DEPRECATED_canBeGeneric(DEPRECATED_canBeGeneric) -{ - LUAU_ASSERT(!FFlag::LuauRankNTypes); -} - int Free::nextIndex = 0; Generic::Generic() diff --git a/Analysis/src/Unifier.cpp b/Analysis/src/Unifier.cpp index 2539650a..82f621b6 100644 --- a/Analysis/src/Unifier.cpp +++ b/Analysis/src/Unifier.cpp @@ -14,17 +14,15 @@ LUAU_FASTINT(LuauTypeInferRecursionLimit); LUAU_FASTINT(LuauTypeInferTypePackLoopLimit); LUAU_FASTINTVARIABLE(LuauTypeInferIterationLimit, 2000); -LUAU_FASTFLAG(LuauGenericFunctions) LUAU_FASTFLAGVARIABLE(LuauTableSubtypingVariance, false); -LUAU_FASTFLAGVARIABLE(LuauDontMutatePersistentFunctions, false) -LUAU_FASTFLAG(LuauRankNTypes) LUAU_FASTFLAGVARIABLE(LuauUnionHeuristic, false) LUAU_FASTFLAGVARIABLE(LuauTableUnificationEarlyTest, false) -LUAU_FASTFLAGVARIABLE(LuauSealedTableUnifyOptionalFix, false) LUAU_FASTFLAGVARIABLE(LuauOccursCheckOkWithRecursiveFunctions, false) LUAU_FASTFLAGVARIABLE(LuauTypecheckOpts, false) LUAU_FASTFLAG(LuauShareTxnSeen); LUAU_FASTFLAGVARIABLE(LuauCacheUnifyTableResults, false) +LUAU_FASTFLAGVARIABLE(LuauExtendedTypeMismatchError, false) +LUAU_FASTFLAGVARIABLE(LuauExtendedClassMismatchError, false) namespace Luau { @@ -219,17 +217,12 @@ void Unifier::tryUnify_(TypeId superTy, TypeId subTy, bool isFunctionCall, bool *asMutable(subTy) = BoundTypeVar(superTy); } - if (!FFlag::LuauRankNTypes) - l->DEPRECATED_canBeGeneric &= r->DEPRECATED_canBeGeneric; - return; } - else if (l && r && FFlag::LuauGenericFunctions) + else if (l && r) { log(superTy); occursCheck(superTy, subTy); - if (!FFlag::LuauRankNTypes) - r->DEPRECATED_canBeGeneric &= l->DEPRECATED_canBeGeneric; r->level = min(r->level, l->level); *asMutable(superTy) = BoundTypeVar(subTy); return; @@ -240,7 +233,7 @@ void Unifier::tryUnify_(TypeId superTy, TypeId subTy, bool isFunctionCall, bool // Unification can't change the level of a generic. auto rightGeneric = get(subTy); - if (FFlag::LuauRankNTypes && rightGeneric && !rightGeneric->level.subsumes(l->level)) + if (rightGeneric && !rightGeneric->level.subsumes(l->level)) { // TODO: a more informative error message? CLI-39912 errors.push_back(TypeError{location, GenericError{"Generic subtype escaping scope"}}); @@ -266,31 +259,13 @@ void Unifier::tryUnify_(TypeId superTy, TypeId subTy, bool isFunctionCall, bool // Unification can't change the level of a generic. auto leftGeneric = get(superTy); - if (FFlag::LuauRankNTypes && leftGeneric && !leftGeneric->level.subsumes(r->level)) + if (leftGeneric && !leftGeneric->level.subsumes(r->level)) { // TODO: a more informative error message? CLI-39912 errors.push_back(TypeError{location, GenericError{"Generic supertype escaping scope"}}); return; } - // This is the old code which is just wrong - auto wrongGeneric = get(subTy); // Guaranteed to be null - if (!FFlag::LuauRankNTypes && FFlag::LuauGenericFunctions && wrongGeneric && r->level.subsumes(wrongGeneric->level)) - { - // This code is unreachable! Should we just remove it? - // TODO: a more informative error message? CLI-39912 - errors.push_back(TypeError{location, GenericError{"Generic supertype escaping scope"}}); - return; - } - - // Check if we're unifying a monotype with a polytype - if (FFlag::LuauGenericFunctions && !FFlag::LuauRankNTypes && !r->DEPRECATED_canBeGeneric && isGeneric(superTy)) - { - // TODO: a more informative error message? CLI-39912 - errors.push_back(TypeError{location, GenericError{"Failed to unify a polytype with a monotype"}}); - return; - } - if (!get(subTy)) { if (auto leftLevel = getMutableLevel(superTy)) @@ -333,6 +308,7 @@ void Unifier::tryUnify_(TypeId superTy, TypeId subTy, bool isFunctionCall, bool // A | B <: T if A <: T and B <: T bool failed = false; std::optional unificationTooComplex; + std::optional firstFailedOption; size_t count = uv->options.size(); size_t i = 0; @@ -345,7 +321,13 @@ void Unifier::tryUnify_(TypeId superTy, TypeId subTy, bool isFunctionCall, bool if (auto e = hasUnificationTooComplex(innerState.errors)) unificationTooComplex = e; else if (!innerState.errors.empty()) + { + // 'nil' option is skipped from extended report because we present the type in a special way - 'T?' + if (FFlag::LuauExtendedTypeMismatchError && !firstFailedOption && !isNil(type)) + firstFailedOption = {innerState.errors.front()}; + failed = true; + } if (i != count - 1) innerState.log.rollback(); @@ -358,7 +340,12 @@ void Unifier::tryUnify_(TypeId superTy, TypeId subTy, bool isFunctionCall, bool if (unificationTooComplex) errors.push_back(*unificationTooComplex); else if (failed) - errors.push_back(TypeError{location, TypeMismatch{superTy, subTy}}); + { + if (FFlag::LuauExtendedTypeMismatchError && firstFailedOption) + errors.push_back(TypeError{location, TypeMismatch{superTy, subTy, "Not all union options are compatible", *firstFailedOption}}); + else + errors.push_back(TypeError{location, TypeMismatch{superTy, subTy}}); + } } else if (const UnionTypeVar* uv = get(superTy)) { @@ -425,14 +412,49 @@ void Unifier::tryUnify_(TypeId superTy, TypeId subTy, bool isFunctionCall, bool if (unificationTooComplex) errors.push_back(*unificationTooComplex); else if (!found) - errors.push_back(TypeError{location, TypeMismatch{superTy, subTy}}); + { + if (FFlag::LuauExtendedTypeMismatchError) + errors.push_back(TypeError{location, TypeMismatch{superTy, subTy, "none of the union options are compatible"}}); + else + errors.push_back(TypeError{location, TypeMismatch{superTy, subTy}}); + } } else if (const IntersectionTypeVar* uv = get(superTy)) { - // T <: A & B if A <: T and B <: T - for (TypeId type : uv->parts) + if (FFlag::LuauExtendedTypeMismatchError) { - tryUnify_(type, subTy, /*isFunctionCall*/ false, /*isIntersection*/ true); + std::optional unificationTooComplex; + std::optional firstFailedOption; + + // T <: A & B if A <: T and B <: T + for (TypeId type : uv->parts) + { + Unifier innerState = makeChildUnifier(); + innerState.tryUnify_(type, subTy, /*isFunctionCall*/ false, /*isIntersection*/ true); + + if (auto e = hasUnificationTooComplex(innerState.errors)) + unificationTooComplex = e; + else if (!innerState.errors.empty()) + { + if (!firstFailedOption) + firstFailedOption = {innerState.errors.front()}; + } + + log.concat(std::move(innerState.log)); + } + + if (unificationTooComplex) + errors.push_back(*unificationTooComplex); + else if (firstFailedOption) + errors.push_back(TypeError{location, TypeMismatch{superTy, subTy, "Not all intersection parts are compatible", *firstFailedOption}}); + } + else + { + // T <: A & B if A <: T and B <: T + for (TypeId type : uv->parts) + { + tryUnify_(type, subTy, /*isFunctionCall*/ false, /*isIntersection*/ true); + } } } else if (const IntersectionTypeVar* uv = get(subTy)) @@ -480,7 +502,12 @@ void Unifier::tryUnify_(TypeId superTy, TypeId subTy, bool isFunctionCall, bool if (unificationTooComplex) errors.push_back(*unificationTooComplex); else if (!found) - errors.push_back(TypeError{location, TypeMismatch{superTy, subTy}}); + { + if (FFlag::LuauExtendedTypeMismatchError) + errors.push_back(TypeError{location, TypeMismatch{superTy, subTy, "none of the intersection parts are compatible"}}); + else + errors.push_back(TypeError{location, TypeMismatch{superTy, subTy}}); + } } else if (get(superTy) && get(subTy)) tryUnifyPrimitives(superTy, subTy); @@ -773,8 +800,8 @@ void Unifier::tryUnify_(TypePackId superTp, TypePackId subTp, bool isFunctionCal // If both are at the end, we're done if (!superIter.good() && !subIter.good()) { - const bool lFreeTail = l->tail && get(FFlag::LuauAddMissingFollow ? follow(*l->tail) : *l->tail) != nullptr; - const bool rFreeTail = r->tail && get(FFlag::LuauAddMissingFollow ? follow(*r->tail) : *r->tail) != nullptr; + const bool lFreeTail = l->tail && get(follow(*l->tail)) != nullptr; + const bool rFreeTail = r->tail && get(follow(*r->tail)) != nullptr; if (lFreeTail && rFreeTail) tryUnify_(*l->tail, *r->tail); else if (lFreeTail) @@ -812,7 +839,7 @@ void Unifier::tryUnify_(TypePackId superTp, TypePackId subTp, bool isFunctionCal } // In nonstrict mode, any also marks an optional argument. - else if (superIter.good() && isNonstrictMode() && get(FFlag::LuauAddMissingFollow ? follow(*superIter) : *superIter)) + else if (superIter.good() && isNonstrictMode() && get(follow(*superIter))) { superIter.advance(); continue; @@ -887,24 +914,21 @@ void Unifier::tryUnifyFunctions(TypeId superTy, TypeId subTy, bool isFunctionCal ice("passed non-function types to unifyFunction"); size_t numGenerics = lf->generics.size(); - if (FFlag::LuauGenericFunctions && numGenerics != rf->generics.size()) + if (numGenerics != rf->generics.size()) { numGenerics = std::min(lf->generics.size(), rf->generics.size()); errors.push_back(TypeError{location, TypeMismatch{superTy, subTy}}); } size_t numGenericPacks = lf->genericPacks.size(); - if (FFlag::LuauGenericFunctions && numGenericPacks != rf->genericPacks.size()) + if (numGenericPacks != rf->genericPacks.size()) { numGenericPacks = std::min(lf->genericPacks.size(), rf->genericPacks.size()); errors.push_back(TypeError{location, TypeMismatch{superTy, subTy}}); } - if (FFlag::LuauGenericFunctions) - { - for (size_t i = 0; i < numGenerics; i++) - log.pushSeen(lf->generics[i], rf->generics[i]); - } + for (size_t i = 0; i < numGenerics; i++) + log.pushSeen(lf->generics[i], rf->generics[i]); CountMismatch::Context context = ctx; @@ -931,22 +955,19 @@ void Unifier::tryUnifyFunctions(TypeId superTy, TypeId subTy, bool isFunctionCal tryUnify_(lf->retType, rf->retType); } - if (lf->definition && !rf->definition && (!FFlag::LuauDontMutatePersistentFunctions || !subTy->persistent)) + if (lf->definition && !rf->definition && !subTy->persistent) { rf->definition = lf->definition; } - else if (!lf->definition && rf->definition && (!FFlag::LuauDontMutatePersistentFunctions || !superTy->persistent)) + else if (!lf->definition && rf->definition && !superTy->persistent) { lf->definition = rf->definition; } ctx = context; - if (FFlag::LuauGenericFunctions) - { - for (int i = int(numGenerics) - 1; 0 <= i; i--) - log.popSeen(lf->generics[i], rf->generics[i]); - } + for (int i = int(numGenerics) - 1; 0 <= i; i--) + log.popSeen(lf->generics[i], rf->generics[i]); } namespace @@ -1032,7 +1053,12 @@ void Unifier::tryUnifyTables(TypeId left, TypeId right, bool isIntersection) Unifier innerState = makeChildUnifier(); innerState.tryUnify_(prop.type, r->second.type); - checkChildUnifierTypeMismatch(innerState.errors, left, right); + + if (FFlag::LuauExtendedTypeMismatchError) + checkChildUnifierTypeMismatch(innerState.errors, name, left, right); + else + checkChildUnifierTypeMismatch(innerState.errors, left, right); + if (innerState.errors.empty()) log.concat(std::move(innerState.log)); else @@ -1047,7 +1073,12 @@ void Unifier::tryUnifyTables(TypeId left, TypeId right, bool isIntersection) Unifier innerState = makeChildUnifier(); innerState.tryUnify_(prop.type, rt->indexer->indexResultType); - checkChildUnifierTypeMismatch(innerState.errors, left, right); + + if (FFlag::LuauExtendedTypeMismatchError) + checkChildUnifierTypeMismatch(innerState.errors, name, left, right); + else + checkChildUnifierTypeMismatch(innerState.errors, left, right); + if (innerState.errors.empty()) log.concat(std::move(innerState.log)); else @@ -1083,7 +1114,12 @@ void Unifier::tryUnifyTables(TypeId left, TypeId right, bool isIntersection) Unifier innerState = makeChildUnifier(); innerState.tryUnify_(prop.type, lt->indexer->indexResultType); - checkChildUnifierTypeMismatch(innerState.errors, left, right); + + if (FFlag::LuauExtendedTypeMismatchError) + checkChildUnifierTypeMismatch(innerState.errors, name, left, right); + else + checkChildUnifierTypeMismatch(innerState.errors, left, right); + if (innerState.errors.empty()) log.concat(std::move(innerState.log)); else @@ -1384,21 +1420,8 @@ void Unifier::tryUnifySealedTables(TypeId left, TypeId right, bool isIntersectio const auto& r = rt->props.find(it.first); if (r == rt->props.end()) { - if (FFlag::LuauSealedTableUnifyOptionalFix) - { - if (isOptional(it.second.type)) - continue; - } - else - { - if (get(it.second.type)) - { - const UnionTypeVar* possiblyOptional = get(it.second.type); - const std::vector& options = possiblyOptional->options; - if (options.end() != std::find_if(options.begin(), options.end(), isNil)) - continue; - } - } + if (isOptional(it.second.type)) + continue; missingPropertiesInSuper.push_back(it.first); @@ -1482,21 +1505,8 @@ void Unifier::tryUnifySealedTables(TypeId left, TypeId right, bool isIntersectio const auto& r = lt->props.find(it.first); if (r == lt->props.end()) { - if (FFlag::LuauSealedTableUnifyOptionalFix) - { - if (isOptional(it.second.type)) - continue; - } - else - { - if (get(it.second.type)) - { - const UnionTypeVar* possiblyOptional = get(it.second.type); - const std::vector& options = possiblyOptional->options; - if (options.end() != std::find_if(options.begin(), options.end(), isNil)) - continue; - } - } + if (isOptional(it.second.type)) + continue; extraPropertiesInSub.push_back(it.first); } @@ -1526,7 +1536,18 @@ void Unifier::tryUnifyWithMetatable(TypeId metatable, TypeId other, bool reverse innerState.tryUnify_(lhs->table, rhs->table); innerState.tryUnify_(lhs->metatable, rhs->metatable); - checkChildUnifierTypeMismatch(innerState.errors, reversed ? other : metatable, reversed ? metatable : other); + if (FFlag::LuauExtendedTypeMismatchError) + { + if (auto e = hasUnificationTooComplex(innerState.errors)) + errors.push_back(*e); + else if (!innerState.errors.empty()) + errors.push_back( + TypeError{location, TypeMismatch{reversed ? other : metatable, reversed ? metatable : other, "", innerState.errors.front()}}); + } + else + { + checkChildUnifierTypeMismatch(innerState.errors, reversed ? other : metatable, reversed ? metatable : other); + } log.concat(std::move(innerState.log)); } @@ -1613,10 +1634,34 @@ void Unifier::tryUnifyWithClass(TypeId superTy, TypeId subTy, bool reversed) { ok = false; errors.push_back(TypeError{location, UnknownProperty{superTy, propName}}); - tryUnify_(prop.type, singletonTypes.errorType); + + if (!FFlag::LuauExtendedClassMismatchError) + tryUnify_(prop.type, singletonTypes.errorType); } else - tryUnify_(prop.type, classProp->type); + { + if (FFlag::LuauExtendedClassMismatchError) + { + Unifier innerState = makeChildUnifier(); + innerState.tryUnify_(prop.type, classProp->type); + + checkChildUnifierTypeMismatch(innerState.errors, propName, reversed ? subTy : superTy, reversed ? superTy : subTy); + + if (innerState.errors.empty()) + { + log.concat(std::move(innerState.log)); + } + else + { + ok = false; + innerState.log.rollback(); + } + } + else + { + tryUnify_(prop.type, classProp->type); + } + } } if (table->indexer) @@ -1649,45 +1694,24 @@ static void queueTypePack_DEPRECATED( while (true) { - if (FFlag::LuauAddMissingFollow) - a = follow(a); + a = follow(a); if (seenTypePacks.count(a)) break; seenTypePacks.insert(a); - if (FFlag::LuauAddMissingFollow) + if (get(a)) { - if (get(a)) - { - state.log(a); - *asMutable(a) = Unifiable::Bound{anyTypePack}; - } - else if (auto tp = get(a)) - { - queue.insert(queue.end(), tp->head.begin(), tp->head.end()); - if (tp->tail) - a = *tp->tail; - else - break; - } + state.log(a); + *asMutable(a) = Unifiable::Bound{anyTypePack}; } - else + else if (auto tp = get(a)) { - if (get(a)) - { - state.log(a); - *asMutable(a) = Unifiable::Bound{anyTypePack}; - } - - if (auto tp = get(a)) - { - queue.insert(queue.end(), tp->head.begin(), tp->head.end()); - if (tp->tail) - a = *tp->tail; - else - break; - } + queue.insert(queue.end(), tp->head.begin(), tp->head.end()); + if (tp->tail) + a = *tp->tail; + else + break; } } } @@ -1698,45 +1722,24 @@ static void queueTypePack(std::vector& queue, DenseHashSet& while (true) { - if (FFlag::LuauAddMissingFollow) - a = follow(a); + a = follow(a); if (seenTypePacks.find(a)) break; seenTypePacks.insert(a); - if (FFlag::LuauAddMissingFollow) + if (get(a)) { - if (get(a)) - { - state.log(a); - *asMutable(a) = Unifiable::Bound{anyTypePack}; - } - else if (auto tp = get(a)) - { - queue.insert(queue.end(), tp->head.begin(), tp->head.end()); - if (tp->tail) - a = *tp->tail; - else - break; - } + state.log(a); + *asMutable(a) = Unifiable::Bound{anyTypePack}; } - else + else if (auto tp = get(a)) { - if (get(a)) - { - state.log(a); - *asMutable(a) = Unifiable::Bound{anyTypePack}; - } - - if (auto tp = get(a)) - { - queue.insert(queue.end(), tp->head.begin(), tp->head.end()); - if (tp->tail) - a = *tp->tail; - else - break; - } + queue.insert(queue.end(), tp->head.begin(), tp->head.end()); + if (tp->tail) + a = *tp->tail; + else + break; } } } @@ -1990,33 +1993,6 @@ std::optional Unifier::findTablePropertyRespectingMeta(TypeId lhsType, N return Luau::findTablePropertyRespectingMeta(errors, globalScope, lhsType, name, location); } -std::optional Unifier::findMetatableEntry(TypeId type, std::string entry) -{ - type = follow(type); - - std::optional metatable = getMetatable(type); - if (!metatable) - return std::nullopt; - - TypeId unwrapped = follow(*metatable); - - if (get(unwrapped)) - return singletonTypes.anyType; - - const TableTypeVar* mtt = getTableType(unwrapped); - if (!mtt) - { - errors.push_back(TypeError{location, GenericError{"Metatable was not a table."}}); - return std::nullopt; - } - - auto it = mtt->props.find(entry); - if (it != mtt->props.end()) - return it->second.type; - else - return std::nullopt; -} - void Unifier::occursCheck(TypeId needle, TypeId haystack) { std::unordered_set seen_DEPRECATED; @@ -2168,7 +2144,7 @@ void Unifier::occursCheck(std::unordered_set& seen_DEPRECATED, Dense { for (const auto& ty : a->head) { - if (auto f = get(FFlag::LuauAddMissingFollow ? follow(ty) : ty)) + if (auto f = get(follow(ty))) { occursCheck(seen_DEPRECATED, seen, needle, f->argTypes); occursCheck(seen_DEPRECATED, seen, needle, f->retType); @@ -2207,6 +2183,17 @@ void Unifier::checkChildUnifierTypeMismatch(const ErrorVec& innerErrors, TypeId errors.push_back(TypeError{location, TypeMismatch{wantedType, givenType}}); } +void Unifier::checkChildUnifierTypeMismatch(const ErrorVec& innerErrors, const std::string& prop, TypeId wantedType, TypeId givenType) +{ + LUAU_ASSERT(FFlag::LuauExtendedTypeMismatchError || FFlag::LuauExtendedClassMismatchError); + + if (auto e = hasUnificationTooComplex(innerErrors)) + errors.push_back(*e); + else if (!innerErrors.empty()) + errors.push_back( + TypeError{location, TypeMismatch{wantedType, givenType, format("Property '%s' is not compatible", prop.c_str()), innerErrors.front()}}); +} + void Unifier::ice(const std::string& message, const Location& location) { sharedState.iceHandler->ice(message, location); diff --git a/Ast/include/Luau/Parser.h b/Ast/include/Luau/Parser.h index 42c64dc9..39c7d925 100644 --- a/Ast/include/Luau/Parser.h +++ b/Ast/include/Luau/Parser.h @@ -282,7 +282,6 @@ private: // `<' namelist `>' std::pair, AstArray> parseGenericTypeList(); - std::pair, AstArray> parseGenericTypeListIfFFlagParseGenericFunctions(); // `<' typeAnnotation[, ...] `>' AstArray parseTypeParams(); diff --git a/Ast/src/Parser.cpp b/Ast/src/Parser.cpp index 846bc0ba..a1bad65e 100644 --- a/Ast/src/Parser.cpp +++ b/Ast/src/Parser.cpp @@ -10,13 +10,13 @@ // See docs/SyntaxChanges.md for an explanation. LUAU_FASTINTVARIABLE(LuauRecursionLimit, 1000) LUAU_FASTINTVARIABLE(LuauParseErrorLimit, 100) -LUAU_FASTFLAGVARIABLE(LuauGenericFunctionsParserFix, false) -LUAU_FASTFLAGVARIABLE(LuauParseGenericFunctions, false) LUAU_FASTFLAGVARIABLE(LuauCaptureBrokenCommentSpans, false) LUAU_FASTFLAGVARIABLE(LuauIfElseExpressionBaseSupport, false) LUAU_FASTFLAGVARIABLE(LuauIfStatementRecursionGuard, false) LUAU_FASTFLAGVARIABLE(LuauTypeAliasPacks, false) LUAU_FASTFLAGVARIABLE(LuauParseTypePackTypeParameters, false) +LUAU_FASTFLAGVARIABLE(LuauFixAmbiguousErrorRecoveryInAssign, false) +LUAU_FASTFLAGVARIABLE(LuauParseGenericFunctionTypeBegin, false) namespace Luau { @@ -957,7 +957,7 @@ AstStat* Parser::parseAssignment(AstExpr* initial) { nextLexeme(); - AstExpr* expr = parsePrimaryExpr(/* asStatement= */ false); + AstExpr* expr = parsePrimaryExpr(/* asStatement= */ FFlag::LuauFixAmbiguousErrorRecoveryInAssign); if (!isExprLValue(expr)) expr = reportExprError(expr->location, copy({expr}), "Assigned expression must be a variable or a field"); @@ -995,7 +995,7 @@ std::pair Parser::parseFunctionBody( { Location start = matchFunction.location; - auto [generics, genericPacks] = parseGenericTypeListIfFFlagParseGenericFunctions(); + auto [generics, genericPacks] = parseGenericTypeList(); Lexeme matchParen = lexer.current(); expectAndConsume('(', "function"); @@ -1343,19 +1343,18 @@ AstTypeOrPack Parser::parseFunctionTypeAnnotation(bool allowPack) { incrementRecursionCounter("type annotation"); - bool monomorphic = !(FFlag::LuauParseGenericFunctions && lexer.current().type == '<'); - - auto [generics, genericPacks] = parseGenericTypeListIfFFlagParseGenericFunctions(); + bool monomorphic = lexer.current().type != '<'; Lexeme begin = lexer.current(); - if (FFlag::LuauGenericFunctionsParserFix) - expectAndConsume('(', "function parameters"); - else - { - LUAU_ASSERT(begin.type == '('); - nextLexeme(); // ( - } + auto [generics, genericPacks] = parseGenericTypeList(); + + Lexeme parameterStart = lexer.current(); + + if (!FFlag::LuauParseGenericFunctionTypeBegin) + begin = parameterStart; + + expectAndConsume('(', "function parameters"); matchRecoveryStopOnToken[Lexeme::SkinnyArrow]++; @@ -1366,7 +1365,7 @@ AstTypeOrPack Parser::parseFunctionTypeAnnotation(bool allowPack) if (lexer.current().type != ')') varargAnnotation = parseTypeList(params, names); - expectMatchAndConsume(')', begin, true); + expectMatchAndConsume(')', parameterStart, true); matchRecoveryStopOnToken[Lexeme::SkinnyArrow]--; @@ -1585,7 +1584,7 @@ AstTypeOrPack Parser::parseSimpleTypeAnnotation(bool allowPack) { return {parseTableTypeAnnotation(), {}}; } - else if (lexer.current().type == '(' || (FFlag::LuauParseGenericFunctions && lexer.current().type == '<')) + else if (lexer.current().type == '(' || lexer.current().type == '<') { return parseFunctionTypeAnnotation(allowPack); } @@ -2315,19 +2314,6 @@ Parser::Name Parser::parseIndexName(const char* context, const Position& previou return Name(nameError, location); } -std::pair, AstArray> Parser::parseGenericTypeListIfFFlagParseGenericFunctions() -{ - if (FFlag::LuauParseGenericFunctions) - return Parser::parseGenericTypeList(); - AstArray generics; - AstArray genericPacks; - generics.size = 0; - generics.data = nullptr; - genericPacks.size = 0; - genericPacks.data = nullptr; - return std::pair(generics, genericPacks); -} - std::pair, AstArray> Parser::parseGenericTypeList() { TempVector names{scratchName}; @@ -2342,7 +2328,7 @@ std::pair, AstArray> Parser::parseGenericTypeList() while (true) { AstName name = parseName().name; - if (FFlag::LuauParseGenericFunctions && lexer.current().type == Lexeme::Dot3) + if (lexer.current().type == Lexeme::Dot3) { seenPack = true; nextLexeme(); @@ -2379,15 +2365,12 @@ AstArray Parser::parseTypeParams() Lexeme begin = lexer.current(); nextLexeme(); - bool seenPack = false; while (true) { if (FFlag::LuauParseTypePackTypeParameters) { if (shouldParseTypePackAnnotation(lexer)) { - seenPack = true; - auto typePack = parseTypePackAnnotation(); if (FFlag::LuauTypeAliasPacks) // Type packs are recorded only is we can handle them @@ -2399,8 +2382,6 @@ AstArray Parser::parseTypeParams() if (typePack) { - seenPack = true; - if (FFlag::LuauTypeAliasPacks) // Type packs are recorded only is we can handle them parameters.push_back({{}, typePack}); } diff --git a/CLI/Analyze.cpp b/CLI/Analyze.cpp index ed0552d7..9ab10aaf 100644 --- a/CLI/Analyze.cpp +++ b/CLI/Analyze.cpp @@ -34,8 +34,10 @@ static void report(ReportFormat format, const char* name, const Luau::Location& } } -static void reportError(ReportFormat format, const char* name, const Luau::TypeError& error) +static void reportError(ReportFormat format, const Luau::TypeError& error) { + const char* name = error.moduleName.c_str(); + if (const Luau::SyntaxError* syntaxError = Luau::get_if(&error.data)) report(format, name, error.location, "SyntaxError", syntaxError->message.c_str()); else @@ -49,7 +51,10 @@ static void reportWarning(ReportFormat format, const char* name, const Luau::Lin static bool analyzeFile(Luau::Frontend& frontend, const char* name, ReportFormat format, bool annotate) { - Luau::CheckResult cr = frontend.check(name); + Luau::CheckResult cr; + + if (frontend.isDirty(name)) + cr = frontend.check(name); if (!frontend.getSourceModule(name)) { @@ -58,7 +63,7 @@ static bool analyzeFile(Luau::Frontend& frontend, const char* name, ReportFormat } for (auto& error : cr.errors) - reportError(format, name, error); + reportError(format, error); Luau::LintResult lr = frontend.lint(name); @@ -115,7 +120,12 @@ struct CliFileResolver : Luau::FileResolver { if (Luau::AstExprConstantString* expr = node->as()) { - Luau::ModuleName name = std::string(expr->value.data, expr->value.size) + ".lua"; + Luau::ModuleName name = std::string(expr->value.data, expr->value.size) + ".luau"; + if (!moduleExists(name)) + { + // fall back to .lua if a module with .luau doesn't exist + name = std::string(expr->value.data, expr->value.size) + ".lua"; + } return {{name}}; } @@ -236,8 +246,15 @@ int main(int argc, char** argv) if (isDirectory(argv[i])) { traverseDirectory(argv[i], [&](const std::string& name) { - if (name.length() > 4 && name.rfind(".lua") == name.length() - 4) + // Look for .luau first and if absent, fall back to .lua + if (name.length() > 5 && name.rfind(".luau") == name.length() - 5) + { failed += !analyzeFile(frontend, name.c_str(), format, annotate); + } + else if (name.length() > 4 && name.rfind(".lua") == name.length() - 4) + { + failed += !analyzeFile(frontend, name.c_str(), format, annotate); + } }); } else diff --git a/CLI/Repl.cpp b/CLI/Repl.cpp index 4968d080..5c904cca 100644 --- a/CLI/Repl.cpp +++ b/CLI/Repl.cpp @@ -13,6 +13,17 @@ #include +#ifdef _WIN32 +#include +#include +#endif + +enum class CompileFormat +{ + Default, + Binary +}; + static int lua_loadstring(lua_State* L) { size_t l = 0; @@ -51,9 +62,13 @@ static int lua_require(lua_State* L) return finishrequire(L); lua_pop(L, 1); - std::optional source = readFile(name + ".lua"); + std::optional source = readFile(name + ".luau"); if (!source) - luaL_argerrorL(L, 1, ("error loading " + name).c_str()); + { + source = readFile(name + ".lua"); // try .lua if .luau doesn't exist + if (!source) + luaL_argerrorL(L, 1, ("error loading " + name).c_str()); // if neither .luau nor .lua exist, we have an error + } // module needs to run in a new thread, isolated from the rest lua_State* GL = lua_mainthread(L); @@ -183,6 +198,11 @@ static std::string runCode(lua_State* L, const std::string& source) error += "\nstack backtrace:\n"; error += lua_debugtrace(T); +#ifdef __EMSCRIPTEN__ + // nicer formatting for errors in web repl + error = "Error:" + error; +#endif + fprintf(stdout, "%s", error.c_str()); } @@ -190,6 +210,39 @@ static std::string runCode(lua_State* L, const std::string& source) return std::string(); } +#ifdef __EMSCRIPTEN__ +extern "C" +{ + const char* executeScript(const char* source) + { + // setup flags + for (Luau::FValue* flag = Luau::FValue::list; flag; flag = flag->next) + if (strncmp(flag->name, "Luau", 4) == 0) + flag->value = true; + + // create new state + std::unique_ptr globalState(luaL_newstate(), lua_close); + lua_State* L = globalState.get(); + + // setup state + setupState(L); + + // sandbox thread + luaL_sandboxthread(L); + + // static string for caching result (prevents dangling ptr on function exit) + static std::string result; + + // run code + collect error + result = runCode(L, source); + + return result.empty() ? NULL : result.c_str(); + } +} +#endif + +// Excluded from emscripten compilation to avoid -Wunused-function errors. +#ifndef __EMSCRIPTEN__ static void completeIndexer(lua_State* L, const char* editBuffer, size_t start, std::vector& completions) { std::string_view lookup = editBuffer + start; @@ -366,7 +419,7 @@ static void reportError(const char* name, const Luau::CompileError& error) report(name, error.getLocation(), "CompileError", error.what()); } -static bool compileFile(const char* name) +static bool compileFile(const char* name, CompileFormat format) { std::optional source = readFile(name); if (!source) @@ -383,7 +436,15 @@ static bool compileFile(const char* name) Luau::compileOrThrow(bcb, *source); - printf("%s", bcb.dumpEverything().c_str()); + switch (format) + { + case CompileFormat::Default: + printf("%s", bcb.dumpEverything().c_str()); + break; + case CompileFormat::Binary: + fwrite(bcb.getBytecode().data(), 1, bcb.getBytecode().size(), stdout); + break; + } return true; } @@ -408,7 +469,7 @@ static void displayHelp(const char* argv0) printf("\n"); printf("Available modes:\n"); printf(" omitted: compile and run input files one by one\n"); - printf(" --compile: compile input files and output resulting bytecode\n"); + printf(" --compile[=format]: compile input files and output resulting formatted bytecode (binary or text)\n"); printf("\n"); printf("Available options:\n"); printf(" --profile[=N]: profile the code using N Hz sampling (default 10000) and output results to profile.out\n"); @@ -440,8 +501,19 @@ int main(int argc, char** argv) return 0; } - if (argc >= 2 && strcmp(argv[1], "--compile") == 0) + + if (argc >= 2 && strncmp(argv[1], "--compile", strlen("--compile")) == 0) { + CompileFormat format = CompileFormat::Default; + + if (strcmp(argv[1], "--compile=binary") == 0) + format = CompileFormat::Binary; + +#ifdef _WIN32 + if (format == CompileFormat::Binary) + _setmode(_fileno(stdout), _O_BINARY); +#endif + int failed = 0; for (int i = 2; i < argc; ++i) @@ -452,13 +524,15 @@ int main(int argc, char** argv) if (isDirectory(argv[i])) { traverseDirectory(argv[i], [&](const std::string& name) { - if (name.length() > 4 && name.rfind(".lua") == name.length() - 4) - failed += !compileFile(name.c_str()); + if (name.length() > 5 && name.rfind(".luau") == name.length() - 5) + failed += !compileFile(name.c_str(), format); + else if (name.length() > 4 && name.rfind(".lua") == name.length() - 4) + failed += !compileFile(name.c_str(), format); }); } else { - failed += !compileFile(argv[i]); + failed += !compileFile(argv[i], format); } } @@ -511,5 +585,6 @@ int main(int argc, char** argv) return failed; } } +#endif diff --git a/CMakeLists.txt b/CMakeLists.txt index d6598f2a..36014a98 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -17,17 +17,26 @@ add_library(Luau.VM STATIC) if(LUAU_BUILD_CLI) add_executable(Luau.Repl.CLI) - add_executable(Luau.Analyze.CLI) + if(NOT EMSCRIPTEN) + add_executable(Luau.Analyze.CLI) + else() + # add -fexceptions for emscripten to allow exceptions to be caught in C++ + set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -fexceptions") + endif() # This also adds target `name` on Linux/macOS and `name.exe` on Windows set_target_properties(Luau.Repl.CLI PROPERTIES OUTPUT_NAME luau) - set_target_properties(Luau.Analyze.CLI PROPERTIES OUTPUT_NAME luau-analyze) + + if(NOT EMSCRIPTEN) + set_target_properties(Luau.Analyze.CLI PROPERTIES OUTPUT_NAME luau-analyze) + endif() endif() -if(LUAU_BUILD_TESTS) +if(LUAU_BUILD_TESTS AND NOT EMSCRIPTEN) add_executable(Luau.UnitTest) add_executable(Luau.Conformance) endif() + include(Sources.cmake) target_compile_features(Luau.Ast PUBLIC cxx_std_17) @@ -53,10 +62,6 @@ if(MSVC) else() list(APPEND LUAU_OPTIONS -Wall) # All warnings list(APPEND LUAU_OPTIONS -Werror) # Warnings are errors - - if(CMAKE_CXX_COMPILER_ID STREQUAL "GNU") - list(APPEND LUAU_OPTIONS -Wno-unused) # GCC considers variables declared/checked in if() as unused - endif() endif() target_compile_options(Luau.Ast PRIVATE ${LUAU_OPTIONS}) @@ -65,7 +70,10 @@ target_compile_options(Luau.VM PRIVATE ${LUAU_OPTIONS}) if(LUAU_BUILD_CLI) target_compile_options(Luau.Repl.CLI PRIVATE ${LUAU_OPTIONS}) - target_compile_options(Luau.Analyze.CLI PRIVATE ${LUAU_OPTIONS}) + + if(NOT EMSCRIPTEN) + target_compile_options(Luau.Analyze.CLI PRIVATE ${LUAU_OPTIONS}) + endif() target_include_directories(Luau.Repl.CLI PRIVATE extern) target_link_libraries(Luau.Repl.CLI PRIVATE Luau.Compiler Luau.VM) @@ -74,10 +82,20 @@ if(LUAU_BUILD_CLI) target_link_libraries(Luau.Repl.CLI PRIVATE pthread) endif() - target_link_libraries(Luau.Analyze.CLI PRIVATE Luau.Analysis) + if(NOT EMSCRIPTEN) + target_link_libraries(Luau.Analyze.CLI PRIVATE Luau.Analysis) + endif() + + if(EMSCRIPTEN) + # declare exported functions to emscripten + target_link_options(Luau.Repl.CLI PRIVATE -sEXPORTED_FUNCTIONS=['_executeScript'] -sEXPORTED_RUNTIME_METHODS=['ccall','cwrap'] -fexceptions) + + # custom output directory for wasm + js file + set_target_properties(Luau.Repl.CLI PROPERTIES RUNTIME_OUTPUT_DIRECTORY ${CMAKE_SOURCE_DIR}/docs/assets/luau) + endif() endif() -if(LUAU_BUILD_TESTS) +if(LUAU_BUILD_TESTS AND NOT EMSCRIPTEN) target_compile_options(Luau.UnitTest PRIVATE ${LUAU_OPTIONS}) target_include_directories(Luau.UnitTest PRIVATE extern) target_link_libraries(Luau.UnitTest PRIVATE Luau.Analysis Luau.Compiler) diff --git a/Compiler/include/Luau/Bytecode.h b/Compiler/include/Luau/Bytecode.h index 4b03ed1c..71631d10 100644 --- a/Compiler/include/Luau/Bytecode.h +++ b/Compiler/include/Luau/Bytecode.h @@ -467,6 +467,10 @@ enum LuauBuiltinFunction // vector ctor LBF_VECTOR, + + // bit32.count + LBF_BIT32_COUNTLZ, + LBF_BIT32_COUNTRZ, }; // Capture type, used in LOP_CAPTURE diff --git a/Compiler/include/Luau/Compiler.h b/Compiler/include/Luau/Compiler.h index f8d67158..4f88e602 100644 --- a/Compiler/include/Luau/Compiler.h +++ b/Compiler/include/Luau/Compiler.h @@ -36,6 +36,9 @@ struct CompileOptions // global builtin to construct vectors; disabled by default const char* vectorLib = nullptr; const char* vectorCtor = nullptr; + + // null-terminated array of globals that are mutable; disables the import optimization for fields accessed through these + const char** mutableGlobals = nullptr; }; class CompileError : public std::exception diff --git a/Compiler/src/Compiler.cpp b/Compiler/src/Compiler.cpp index 7750a1d9..9712f02f 100644 --- a/Compiler/src/Compiler.cpp +++ b/Compiler/src/Compiler.cpp @@ -13,7 +13,9 @@ LUAU_FASTFLAGVARIABLE(LuauPreloadClosures, false) LUAU_FASTFLAGVARIABLE(LuauPreloadClosuresFenv, false) LUAU_FASTFLAGVARIABLE(LuauPreloadClosuresUpval, false) +LUAU_FASTFLAGVARIABLE(LuauGenericSpecialGlobals, false) LUAU_FASTFLAG(LuauIfElseExpressionBaseSupport) +LUAU_FASTFLAGVARIABLE(LuauBit32CountBuiltin, false) namespace Luau { @@ -22,6 +24,7 @@ static const uint32_t kMaxRegisterCount = 255; static const uint32_t kMaxUpvalueCount = 200; static const uint32_t kMaxLocalCount = 200; +// TODO: Remove with LuauGenericSpecialGlobals static const char* kSpecialGlobals[] = {"Game", "Workspace", "_G", "game", "plugin", "script", "shared", "workspace"}; CompileError::CompileError(const Location& location, const std::string& message) @@ -1277,7 +1280,7 @@ struct Compiler { const Global* global = globals.find(expr->name); - return options.optimizationLevel >= 1 && (!global || (!global->written && !global->special)); + return options.optimizationLevel >= 1 && (!global || (!global->written && !global->writable)); } void compileExprIndexName(AstExprIndexName* expr, uint8_t target) @@ -2465,9 +2468,10 @@ struct Compiler } else if (node->is()) { + LUAU_ASSERT(!loops.empty()); + // before exiting out of the loop, we need to close all local variables that were captured in closures since loop start // normally they are closed by the enclosing blocks, including the loop block, but we're skipping that here - LUAU_ASSERT(!loops.empty()); closeLocals(loops.back().localOffset); size_t label = bytecode.emitLabel(); @@ -2478,12 +2482,13 @@ struct Compiler } else if (AstStatContinue* stat = node->as()) { + LUAU_ASSERT(!loops.empty()); + if (loops.back().untilCondition) validateContinueUntil(stat, loops.back().untilCondition); // before continuing, we need to close all local variables that were captured in closures since loop start // normally they are closed by the enclosing blocks, including the loop block, but we're skipping that here - LUAU_ASSERT(!loops.empty()); closeLocals(loops.back().localOffset); size_t label = bytecode.emitLabel(); @@ -2900,6 +2905,11 @@ struct Compiler break; case AstExprUnary::Len: + if (arg.type == Constant::Type_String) + { + result.type = Constant::Type_Number; + result.valueNumber = double(arg.valueString.size); + } break; default: @@ -3440,7 +3450,7 @@ struct Compiler struct Global { - bool special = false; + bool writable = false; bool written = false; }; @@ -3498,7 +3508,7 @@ struct Compiler { Global* g = globals.find(object->name); - return !g || (!g->special && !g->written) ? Builtin{object->name, expr->index} : Builtin(); + return !g || (!g->writable && !g->written) ? Builtin{object->name, expr->index} : Builtin(); } else { @@ -3629,6 +3639,10 @@ struct Compiler return LBF_BIT32_RROTATE; if (builtin.method == "rshift") return LBF_BIT32_RSHIFT; + if (builtin.method == "countlz" && FFlag::LuauBit32CountBuiltin) + return LBF_BIT32_COUNTLZ; + if (builtin.method == "countrz" && FFlag::LuauBit32CountBuiltin) + return LBF_BIT32_COUNTRZ; } if (builtin.object == "string") @@ -3696,13 +3710,24 @@ void compileOrThrow(BytecodeBuilder& bytecode, AstStatBlock* root, const AstName Compiler compiler(bytecode, options); - // since access to some global objects may result in values that change over time, we block table imports - for (const char* global : kSpecialGlobals) + // since access to some global objects may result in values that change over time, we block imports from non-readonly tables + if (FFlag::LuauGenericSpecialGlobals) { - AstName name = names.get(global); + if (AstName name = names.get("_G"); name.value) + compiler.globals[name].writable = true; - if (name.value) - compiler.globals[name].special = true; + if (options.mutableGlobals) + for (const char** ptr = options.mutableGlobals; *ptr; ++ptr) + if (AstName name = names.get(*ptr); name.value) + compiler.globals[name].writable = true; + } + else + { + for (const char* global : kSpecialGlobals) + { + if (AstName name = names.get(global); name.value) + compiler.globals[name].writable = true; + } } // this visitor traverses the AST to analyze mutability of locals/globals, filling Local::written and Global::written @@ -3717,7 +3742,7 @@ void compileOrThrow(BytecodeBuilder& bytecode, AstStatBlock* root, const AstName } // this visitor tracks calls to getfenv/setfenv and disables some optimizations when they are found - if (FFlag::LuauPreloadClosuresFenv && options.optimizationLevel >= 1) + if (FFlag::LuauPreloadClosuresFenv && options.optimizationLevel >= 1 && (names.get("getfenv").value || names.get("setfenv").value)) { Compiler::FenvVisitor fenvVisitor(compiler.getfenvUsed, compiler.setfenvUsed); root->visit(&fenvVisitor); diff --git a/Makefile b/Makefile index 7788251d..5d51b3d4 100644 --- a/Makefile +++ b/Makefile @@ -49,7 +49,10 @@ OBJECTS=$(AST_OBJECTS) $(COMPILER_OBJECTS) $(ANALYSIS_OBJECTS) $(VM_OBJECTS) $(T CXXFLAGS=-g -Wall -Werror LDFLAGS= -CXXFLAGS+=-Wno-unused # temporary, for older gcc versions +# temporary, for older gcc versions as they treat var in `if (type var = val)` as unused +ifeq ($(findstring g++,$(shell $(CXX) --version)),g++) + CXXFLAGS+=-Wno-unused +endif # configuration-specific flags ifeq ($(config),release) @@ -134,12 +137,11 @@ $(TESTS_TARGET) $(REPL_CLI_TARGET) $(ANALYZE_CLI_TARGET): # executable targets for fuzzing fuzz-%: $(BUILD)/fuzz/%.cpp.o $(ANALYSIS_TARGET) $(COMPILER_TARGET) $(AST_TARGET) $(VM_TARGET) + $(CXX) $^ $(LDFLAGS) -o $@ + fuzz-proto: $(BUILD)/fuzz/proto.cpp.o $(BUILD)/fuzz/protoprint.cpp.o $(BUILD)/fuzz/luau.pb.cpp.o $(ANALYSIS_TARGET) $(COMPILER_TARGET) $(AST_TARGET) $(VM_TARGET) | build/libprotobuf-mutator fuzz-prototest: $(BUILD)/fuzz/prototest.cpp.o $(BUILD)/fuzz/protoprint.cpp.o $(BUILD)/fuzz/luau.pb.cpp.o $(ANALYSIS_TARGET) $(COMPILER_TARGET) $(AST_TARGET) $(VM_TARGET) | build/libprotobuf-mutator -fuzz-%: - $(CXX) $^ $(LDFLAGS) -o $@ - # static library targets $(AST_TARGET): $(AST_OBJECTS) $(COMPILER_TARGET): $(COMPILER_OBJECTS) diff --git a/VM/include/lua.h b/VM/include/lua.h index 2f93ad90..a9d3e875 100644 --- a/VM/include/lua.h +++ b/VM/include/lua.h @@ -213,6 +213,8 @@ LUA_API int lua_resume(lua_State* L, lua_State* from, int narg); LUA_API int lua_resumeerror(lua_State* L, lua_State* from); LUA_API int lua_status(lua_State* L); LUA_API int lua_isyieldable(lua_State* L); +LUA_API void* lua_getthreaddata(lua_State* L); +LUA_API void lua_setthreaddata(lua_State* L, void* data); /* ** garbage-collection function and options @@ -346,6 +348,8 @@ struct lua_Debug * can only be changed when the VM is not running any code */ struct lua_Callbacks { + void* userdata; /* arbitrary userdata pointer that is never overwritten by Luau */ + void (*interrupt)(lua_State* L, int gc); /* gets called at safepoints (loop back edges, call/ret, gc) if set */ void (*panic)(lua_State* L, int errcode); /* gets called when an unprotected error is raised (if longjmp is used) */ diff --git a/VM/src/lapi.cpp b/VM/src/lapi.cpp index f2e97c66..7e742644 100644 --- a/VM/src/lapi.cpp +++ b/VM/src/lapi.cpp @@ -703,6 +703,7 @@ void lua_setreadonly(lua_State* L, int objindex, bool value) const TValue* o = index2adr(L, objindex); api_check(L, ttistable(o)); Table* t = hvalue(o); + api_check(L, t != hvalue(registry(L))); t->readonly = value; return; } @@ -987,6 +988,16 @@ int lua_status(lua_State* L) return L->status; } +void* lua_getthreaddata(lua_State* L) +{ + return L->userdata; +} + +void lua_setthreaddata(lua_State* L, void* data) +{ + L->userdata = data; +} + /* ** Garbage-collection function */ diff --git a/VM/src/lbitlib.cpp b/VM/src/lbitlib.cpp index 0754a351..c72fe674 100644 --- a/VM/src/lbitlib.cpp +++ b/VM/src/lbitlib.cpp @@ -4,6 +4,8 @@ #include "lnumutils.h" +LUAU_FASTFLAGVARIABLE(LuauBit32Count, false) + #define ALLONES ~0u #define NBITS int(8 * sizeof(unsigned)) @@ -177,6 +179,44 @@ static int b_replace(lua_State* L) return 1; } +static int b_countlz(lua_State* L) +{ + if (!FFlag::LuauBit32Count) + luaL_error(L, "bit32.countlz isn't enabled"); + + b_uint v = luaL_checkunsigned(L, 1); + + b_uint r = NBITS; + for (int i = 0; i < NBITS; ++i) + if (v & (1u << (NBITS - 1 - i))) + { + r = i; + break; + } + + lua_pushunsigned(L, r); + return 1; +} + +static int b_countrz(lua_State* L) +{ + if (!FFlag::LuauBit32Count) + luaL_error(L, "bit32.countrz isn't enabled"); + + b_uint v = luaL_checkunsigned(L, 1); + + b_uint r = NBITS; + for (int i = 0; i < NBITS; ++i) + if (v & (1u << i)) + { + r = i; + break; + } + + lua_pushunsigned(L, r); + return 1; +} + static const luaL_Reg bitlib[] = { {"arshift", b_arshift}, {"band", b_and}, @@ -190,6 +230,8 @@ static const luaL_Reg bitlib[] = { {"replace", b_replace}, {"rrotate", b_rrot}, {"rshift", b_rshift}, + {"countlz", b_countlz}, + {"countrz", b_countrz}, {NULL, NULL}, }; diff --git a/VM/src/lbuiltins.cpp b/VM/src/lbuiltins.cpp index e1c99b21..9ab57ac9 100644 --- a/VM/src/lbuiltins.cpp +++ b/VM/src/lbuiltins.cpp @@ -20,8 +20,9 @@ // If types of the arguments mismatch, luauF_* needs to return -1 and the execution will fall back to the usual call path // If luauF_* succeeds, it needs to return *all* requested arguments, filling results with nil as appropriate. // On input, nparams refers to the actual number of arguments (0+), whereas nresults contains LUA_MULTRET for arbitrary returns or 0+ for a -// fixed-length return Because of this, and the fact that "extra" returned values will be ignored, implementations below typically check that nresults -// is <= expected number, which covers the LUA_MULTRET case. +// fixed-length return +// Because of this, and the fact that "extra" returned values will be ignored, implementations below typically check that nresults is <= expected +// number, which covers the LUA_MULTRET case. static int luauF_assert(lua_State* L, StkId res, TValue* arg0, int nresults, StkId args, int nparams) { @@ -1030,6 +1031,52 @@ static int luauF_vector(lua_State* L, StkId res, TValue* arg0, int nresults, Stk return -1; } +static int luauF_countlz(lua_State* L, StkId res, TValue* arg0, int nresults, StkId args, int nparams) +{ + if (nparams >= 1 && nresults <= 1 && ttisnumber(arg0)) + { + double a1 = nvalue(arg0); + + unsigned n; + luai_num2unsigned(n, a1); + +#ifdef _MSC_VER + unsigned long rl; + int r = _BitScanReverse(&rl, n) ? 31 - int(rl) : 32; +#else + int r = n == 0 ? 32 : __builtin_clz(n); +#endif + + setnvalue(res, double(r)); + return 1; + } + + return -1; +} + +static int luauF_countrz(lua_State* L, StkId res, TValue* arg0, int nresults, StkId args, int nparams) +{ + if (nparams >= 1 && nresults <= 1 && ttisnumber(arg0)) + { + double a1 = nvalue(arg0); + + unsigned n; + luai_num2unsigned(n, a1); + +#ifdef _MSC_VER + unsigned long rl; + int r = _BitScanForward(&rl, n) ? int(rl) : 32; +#else + int r = n == 0 ? 32 : __builtin_ctz(n); +#endif + + setnvalue(res, double(r)); + return 1; + } + + return -1; +} + luau_FastFunction luauF_table[256] = { NULL, luauF_assert, @@ -1096,4 +1143,7 @@ luau_FastFunction luauF_table[256] = { luauF_tunpack, luauF_vector, + + luauF_countlz, + luauF_countrz, }; diff --git a/VM/src/lgc.cpp b/VM/src/lgc.cpp index 6553009f..11f79d1a 100644 --- a/VM/src/lgc.cpp +++ b/VM/src/lgc.cpp @@ -13,7 +13,6 @@ #include LUAU_FASTFLAGVARIABLE(LuauRescanGrayAgainForwardBarrier, false) -LUAU_FASTFLAGVARIABLE(LuauConsolidatedStep, false) LUAU_FASTFLAGVARIABLE(LuauSeparateAtomic, false) LUAU_FASTFLAG(LuauArrayBoundary) @@ -677,117 +676,6 @@ static size_t atomic(lua_State* L) return work; } -static size_t singlestep(lua_State* L) -{ - size_t cost = 0; - global_State* g = L->global; - switch (g->gcstate) - { - case GCSpause: - { - markroot(L); /* start a new collection */ - LUAU_ASSERT(g->gcstate == GCSpropagate); - break; - } - case GCSpropagate: - { - if (g->gray) - { - g->gcstats.currcycle.markitems++; - - cost = propagatemark(g); - } - else - { - // perform one iteration over 'gray again' list - g->gray = g->grayagain; - g->grayagain = NULL; - - g->gcstate = GCSpropagateagain; - } - break; - } - case GCSpropagateagain: - { - if (g->gray) - { - g->gcstats.currcycle.markitems++; - - cost = propagatemark(g); - } - else /* no more `gray' objects */ - { - if (FFlag::LuauSeparateAtomic) - { - g->gcstate = GCSatomic; - } - else - { - double starttimestamp = lua_clock(); - - g->gcstats.currcycle.atomicstarttimestamp = starttimestamp; - g->gcstats.currcycle.atomicstarttotalsizebytes = g->totalbytes; - - atomic(L); /* finish mark phase */ - LUAU_ASSERT(g->gcstate == GCSsweepstring); - - g->gcstats.currcycle.atomictime += lua_clock() - starttimestamp; - } - } - break; - } - case GCSatomic: - { - g->gcstats.currcycle.atomicstarttimestamp = lua_clock(); - g->gcstats.currcycle.atomicstarttotalsizebytes = g->totalbytes; - - cost = atomic(L); /* finish mark phase */ - LUAU_ASSERT(g->gcstate == GCSsweepstring); - break; - } - case GCSsweepstring: - { - size_t traversedcount = 0; - sweepwholelist(L, &g->strt.hash[g->sweepstrgc++], &traversedcount); - - // nothing more to sweep? - if (g->sweepstrgc >= g->strt.size) - { - // sweep string buffer list and preserve used string count - uint32_t nuse = L->global->strt.nuse; - sweepwholelist(L, &g->strbufgc, &traversedcount); - L->global->strt.nuse = nuse; - - g->gcstate = GCSsweep; // end sweep-string phase - } - - g->gcstats.currcycle.sweepitems += traversedcount; - - cost = GC_SWEEPCOST; - break; - } - case GCSsweep: - { - size_t traversedcount = 0; - g->sweepgc = sweeplist(L, g->sweepgc, GC_SWEEPMAX, &traversedcount); - - g->gcstats.currcycle.sweepitems += traversedcount; - - if (*g->sweepgc == NULL) - { /* nothing more to sweep? */ - shrinkbuffers(L); - g->gcstate = GCSpause; /* end collection */ - } - cost = GC_SWEEPMAX * GC_SWEEPCOST; - break; - } - default: - LUAU_ASSERT(!"Unexpected GC state"); - } - - return cost; -} - static size_t gcstep(lua_State* L, size_t limit) { size_t cost = 0; @@ -980,37 +868,12 @@ void luaC_step(lua_State* L, bool assist) int lastgcstate = g->gcstate; double lasttimestamp = lua_clock(); - if (FFlag::LuauConsolidatedStep) - { - size_t work = gcstep(L, lim); + size_t work = gcstep(L, lim); - if (assist) - g->gcstats.currcycle.assistwork += work; - else - g->gcstats.currcycle.explicitwork += work; - } + if (assist) + g->gcstats.currcycle.assistwork += work; else - { - // always perform at least one single step - do - { - lim -= singlestep(L); - - // if we have switched to a different state, capture the duration of last stage - // this way we reduce the number of timer calls we make - if (lastgcstate != g->gcstate) - { - GC_INTERRUPT(lastgcstate); - - double now = lua_clock(); - - recordGcStateTime(g, lastgcstate, now - lasttimestamp, assist); - - lasttimestamp = now; - lastgcstate = g->gcstate; - } - } while (lim > 0 && g->gcstate != GCSpause); - } + g->gcstats.currcycle.explicitwork += work; recordGcStateTime(g, lastgcstate, lua_clock() - lasttimestamp, assist); @@ -1037,14 +900,7 @@ void luaC_step(lua_State* L, bool assist) g->GCthreshold -= debt; } - if (FFlag::LuauConsolidatedStep) - { - GC_INTERRUPT(lastgcstate); - } - else - { - GC_INTERRUPT(g->gcstate); - } + GC_INTERRUPT(lastgcstate); } void luaC_fullgc(lua_State* L) @@ -1070,10 +926,7 @@ void luaC_fullgc(lua_State* L) while (g->gcstate != GCSpause) { LUAU_ASSERT(g->gcstate == GCSsweepstring || g->gcstate == GCSsweep); - if (FFlag::LuauConsolidatedStep) - gcstep(L, SIZE_MAX); - else - singlestep(L); + gcstep(L, SIZE_MAX); } finishGcCycleStats(g); @@ -1084,10 +937,7 @@ void luaC_fullgc(lua_State* L) markroot(L); while (g->gcstate != GCSpause) { - if (FFlag::LuauConsolidatedStep) - gcstep(L, SIZE_MAX); - else - singlestep(L); + gcstep(L, SIZE_MAX); } /* reclaim as much buffer memory as possible (shrinkbuffers() called during sweep is incremental) */ shrinkbuffersfull(L); diff --git a/VM/src/lstrlib.cpp b/VM/src/lstrlib.cpp index a9db3727..80a34483 100644 --- a/VM/src/lstrlib.cpp +++ b/VM/src/lstrlib.cpp @@ -8,6 +8,8 @@ #include #include +LUAU_FASTFLAGVARIABLE(LuauStrPackUBCastFix, false) + /* macro to `unsign' a character */ #define uchar(c) ((unsigned char)(c)) @@ -1404,10 +1406,20 @@ static int str_pack(lua_State* L) } case Kuint: { /* unsigned integers */ - unsigned long long n = (unsigned long long)luaL_checknumber(L, arg); - if (size < SZINT) /* need overflow check? */ - luaL_argcheck(L, n < ((unsigned long long)1 << (size * NB)), arg, "unsigned overflow"); - packint(&b, n, h.islittle, size, 0); + if (FFlag::LuauStrPackUBCastFix) + { + long long n = (long long)luaL_checknumber(L, arg); + if (size < SZINT) /* need overflow check? */ + luaL_argcheck(L, (unsigned long long)n < ((unsigned long long)1 << (size * NB)), arg, "unsigned overflow"); + packint(&b, (unsigned long long)n, h.islittle, size, 0); + } + else + { + unsigned long long n = (unsigned long long)luaL_checknumber(L, arg); + if (size < SZINT) /* need overflow check? */ + luaL_argcheck(L, n < ((unsigned long long)1 << (size * NB)), arg, "unsigned overflow"); + packint(&b, n, h.islittle, size, 0); + } break; } case Kfloat: diff --git a/VM/src/ltable.cpp b/VM/src/ltable.cpp index 883442ae..07d22d59 100644 --- a/VM/src/ltable.cpp +++ b/VM/src/ltable.cpp @@ -30,6 +30,7 @@ LUAU_FASTFLAGVARIABLE(LuauArrayBoundary, false) #define MAXBITS 26 #define MAXSIZE (1 << MAXBITS) +static_assert(offsetof(LuaNode, val) == 0, "Unexpected Node memory layout, pointer cast in gval2slot is incorrect"); // TKey is bitpacked for memory efficiency so we need to validate bit counts for worst case static_assert(TKey{{NULL}, 0, LUA_TDEADKEY, 0}.tt == LUA_TDEADKEY, "not enough bits for tt"); static_assert(TKey{{NULL}, 0, LUA_TNIL, MAXSIZE - 1}.next == MAXSIZE - 1, "not enough bits for next"); diff --git a/VM/src/ltable.h b/VM/src/ltable.h index f98d87b1..45061443 100644 --- a/VM/src/ltable.h +++ b/VM/src/ltable.h @@ -9,7 +9,6 @@ #define gval(n) (&(n)->val) #define gnext(n) ((n)->key.next) -static_assert(offsetof(LuaNode, val) == 0, "Unexpected Node memory layout, pointer cast below is incorrect"); #define gval2slot(t, v) int(cast_to(LuaNode*, static_cast(v)) - t->node) LUAI_FUNC const TValue* luaH_getnum(Table* t, int key); diff --git a/VM/src/ltablib.cpp b/VM/src/ltablib.cpp index de5788eb..37025818 100644 --- a/VM/src/ltablib.cpp +++ b/VM/src/ltablib.cpp @@ -9,8 +9,6 @@ #include "ldebug.h" #include "lvm.h" -LUAU_FASTFLAGVARIABLE(LuauTableFreeze, false) - static int foreachi(lua_State* L) { luaL_checktype(L, 1, LUA_TTABLE); @@ -491,9 +489,6 @@ static int tclear(lua_State* L) static int tfreeze(lua_State* L) { - if (!FFlag::LuauTableFreeze) - luaG_runerror(L, "table.freeze is disabled"); - luaL_checktype(L, 1, LUA_TTABLE); luaL_argcheck(L, !lua_getreadonly(L, 1), 1, "table is already frozen"); luaL_argcheck(L, !luaL_getmetafield(L, 1, "__metatable"), 1, "table has a protected metatable"); @@ -506,9 +501,6 @@ static int tfreeze(lua_State* L) static int tisfrozen(lua_State* L) { - if (!FFlag::LuauTableFreeze) - luaG_runerror(L, "table.isfrozen is disabled"); - luaL_checktype(L, 1, LUA_TTABLE); lua_pushboolean(L, lua_getreadonly(L, 1)); diff --git a/bench/tests/chess.lua b/bench/tests/chess.lua index 87b9abfd..f6ae2cc6 100644 --- a/bench/tests/chess.lua +++ b/bench/tests/chess.lua @@ -205,38 +205,48 @@ function Bitboard:empty() return self.h == 0 and self.l == 0 end -function Bitboard:ctz() - local target = self.l - local offset = 0 - local result = 0 - - if target == 0 then - target = self.h - result = 32 +if not bit32.countrz then + local function ctz(v) + if v == 0 then return 32 end + local offset = 0 + while bit32.extract(v, offset) == 0 do + offset = offset + 1 + end + return offset end - - if target == 0 then - return 64 - end - - while bit32.extract(target, offset) == 0 do - offset = offset + 1 - end - - return result + offset -end - -function Bitboard:ctzafter(start) - start = start + 1 - if start < 32 then - for i=start,31 do - if bit32.extract(self.l, i) == 1 then return i end + function Bitboard:ctz() + local result = ctz(self.l) + if result == 32 then + return ctz(self.h) + 32 + else + return result end end - for i=math.max(32,start),63 do - if bit32.extract(self.h, i-32) == 1 then return i end + function Bitboard:ctzafter(start) + start = start + 1 + if start < 32 then + for i=start,31 do + if bit32.extract(self.l, i) == 1 then return i end + end + end + for i=math.max(32,start),63 do + if bit32.extract(self.h, i-32) == 1 then return i end + end + return 64 + end +else + function Bitboard:ctz() + local result = bit32.countrz(self.l) + if result == 32 then + return bit32.countrz(self.h) + 32 + else + return result + end + end + function Bitboard:ctzafter(start) + local masked = self:band(Bitboard.full:lshift(start+1)) + return masked:ctz() end - return 64 end @@ -245,7 +255,7 @@ function Bitboard:lshift(amt) if amt == 0 then return self end if amt > 31 then - return Bitboard.from(0, bit32.lshift(self.l, amt-31)) + return Bitboard.from(0, bit32.lshift(self.l, amt-32)) end local l = bit32.lshift(self.l, amt) @@ -832,12 +842,12 @@ end local testCases = {} local function addTest(...) table.insert(testCases, {...}) end -addTest(StartingFen, 3, 8902) -addTest("r3k2r/p1ppqpb1/bn2pnp1/3PN3/1p2P3/2N2Q1p/PPPBBPPP/R3K2R w KQkq - 0 0", 2, 2039) -addTest("8/2p5/3p4/KP5r/1R3p1k/8/4P1P1/8 w - - 0 0", 3, 2812) -addTest("r3k2r/Pppp1ppp/1b3nbN/nP6/BBP1P3/q4N2/Pp1P2PP/R2Q1RK1 w kq - 0 1", 3, 9467) -addTest("rnbq1k1r/pp1Pbppp/2p5/8/2B5/8/PPP1NnPP/RNBQK2R w KQ - 1 8", 2, 1486) -addTest("r4rk1/1pp1qppp/p1np1n2/2b1p1B1/2B1P1b1/P1NP1N2/1PP1QPPP/R4RK1 w - - 0 10", 2, 2079) +addTest(StartingFen, 2, 400) +addTest("r3k2r/p1ppqpb1/bn2pnp1/3PN3/1p2P3/2N2Q1p/PPPBBPPP/R3K2R w KQkq - 0 0", 1, 48) +addTest("8/2p5/3p4/KP5r/1R3p1k/8/4P1P1/8 w - - 0 0", 2, 191) +addTest("r3k2r/Pppp1ppp/1b3nbN/nP6/BBP1P3/q4N2/Pp1P2PP/R2Q1RK1 w kq - 0 1", 2, 264) +addTest("rnbq1k1r/pp1Pbppp/2p5/8/2B5/8/PPP1NnPP/RNBQK2R w KQ - 1 8", 1, 44) +addTest("r4rk1/1pp1qppp/p1np1n2/2b1p1B1/2B1P1b1/P1NP1N2/1PP1QPPP/R4RK1 w - - 0 10", 1, 46) local function chess() diff --git a/fuzz/luau.proto b/fuzz/luau.proto index 41a1d077..c78fcf31 100644 --- a/fuzz/luau.proto +++ b/fuzz/luau.proto @@ -19,6 +19,7 @@ message Expr { ExprTable table = 13; ExprUnary unary = 14; ExprBinary binary = 15; + ExprIfElse ifelse = 16; } } @@ -149,6 +150,12 @@ message ExprBinary { required Expr right = 3; } +message ExprIfElse { + required Expr cond = 1; + required Expr then = 2; + required Expr else = 3; +} + message LValue { oneof lvalue_oneof { ExprLocal local = 1; diff --git a/fuzz/proto.cpp b/fuzz/proto.cpp index 6c230b67..c85fac7d 100644 --- a/fuzz/proto.cpp +++ b/fuzz/proto.cpp @@ -11,6 +11,7 @@ #include "Luau/BytecodeBuilder.h" #include "Luau/Common.h" #include "Luau/ToString.h" +#include "Luau/Transpiler.h" #include "lua.h" #include "lualib.h" @@ -23,6 +24,7 @@ const bool kFuzzLinter = true; const bool kFuzzTypeck = true; const bool kFuzzVM = true; const bool kFuzzTypes = true; +const bool kFuzzTranspile = true; static_assert(!(kFuzzVM && !kFuzzCompiler), "VM requires the compiler!"); @@ -242,6 +244,11 @@ DEFINE_PROTO_FUZZER(const luau::StatBlock& message) } } + if (kFuzzTranspile && parseResult.root) + { + transpileWithTypes(*parseResult.root); + } + // run resulting bytecode if (kFuzzVM && bytecode.size()) { diff --git a/fuzz/protoprint.cpp b/fuzz/protoprint.cpp index 2c861a55..e61b6936 100644 --- a/fuzz/protoprint.cpp +++ b/fuzz/protoprint.cpp @@ -476,6 +476,16 @@ struct ProtoToLuau print(expr.right()); } + void print(const luau::ExprIfElse& expr) + { + source += " if "; + print(expr.cond()); + source += " then "; + print(expr.then()); + source += " else "; + print(expr.else_()); + } + void print(const luau::LValue& expr) { if (expr.has_local()) diff --git a/tests/AstQuery.test.cpp b/tests/AstQuery.test.cpp index dd49e675..aa53a92b 100644 --- a/tests/AstQuery.test.cpp +++ b/tests/AstQuery.test.cpp @@ -45,7 +45,6 @@ TEST_CASE_FIXTURE(DocumentationSymbolFixture, "prop") TEST_CASE_FIXTURE(DocumentationSymbolFixture, "event_callback_arg") { ScopedFastFlag sffs[] = { - {"LuauDontMutatePersistentFunctions", true}, {"LuauPersistDefinitionFileTypes", true}, }; diff --git a/tests/Autocomplete.test.cpp b/tests/Autocomplete.test.cpp index 44b8362d..8a7798f3 100644 --- a/tests/Autocomplete.test.cpp +++ b/tests/Autocomplete.test.cpp @@ -1287,9 +1287,6 @@ local e: (n: n@5 TEST_CASE_FIXTURE(ACFixture, "generic_types") { - ScopedFastFlag luauParseGenericFunctions("LuauParseGenericFunctions", true); - ScopedFastFlag luauGenericFunctions("LuauGenericFunctions", true); - check(R"( function f(a: T@1 local b: string = "don't trip" diff --git a/tests/Compiler.test.cpp b/tests/Compiler.test.cpp index bbac3302..7f03019c 100644 --- a/tests/Compiler.test.cpp +++ b/tests/Compiler.test.cpp @@ -13,6 +13,7 @@ LUAU_FASTFLAG(LuauPreloadClosures) LUAU_FASTFLAG(LuauPreloadClosuresFenv) LUAU_FASTFLAG(LuauPreloadClosuresUpval) +LUAU_FASTFLAG(LuauGenericSpecialGlobals) using namespace Luau; @@ -1168,6 +1169,17 @@ RETURN R0 1 )"); } +TEST_CASE("ConstantFoldStringLen") +{ + CHECK_EQ("\n" + compileFunction0("return #'string', #'', #'a', #('b')"), R"( +LOADN R0 6 +LOADN R1 0 +LOADN R2 1 +LOADN R3 1 +RETURN R0 4 +)"); +} + TEST_CASE("ConstantFoldCompare") { // ordered comparisons @@ -3659,4 +3671,118 @@ RETURN R0 0 )"); } +TEST_CASE("LuauGenericSpecialGlobals") +{ + const char* source = R"( +print() +Game.print() +Workspace.print() +_G.print() +game.print() +plugin.print() +script.print() +shared.print() +workspace.print() +)"; + + { + ScopedFastFlag genericSpecialGlobals{"LuauGenericSpecialGlobals", false}; + + // Check Roblox globals are here + CHECK_EQ("\n" + compileFunction0(source), R"( +GETIMPORT R0 1 +CALL R0 0 0 +GETIMPORT R1 3 +GETTABLEKS R0 R1 K0 +CALL R0 0 0 +GETIMPORT R1 5 +GETTABLEKS R0 R1 K0 +CALL R0 0 0 +GETIMPORT R1 7 +GETTABLEKS R0 R1 K0 +CALL R0 0 0 +GETIMPORT R1 9 +GETTABLEKS R0 R1 K0 +CALL R0 0 0 +GETIMPORT R1 11 +GETTABLEKS R0 R1 K0 +CALL R0 0 0 +GETIMPORT R1 13 +GETTABLEKS R0 R1 K0 +CALL R0 0 0 +GETIMPORT R1 15 +GETTABLEKS R0 R1 K0 +CALL R0 0 0 +GETIMPORT R1 17 +GETTABLEKS R0 R1 K0 +CALL R0 0 0 +RETURN R0 0 +)"); + } + + ScopedFastFlag genericSpecialGlobals{"LuauGenericSpecialGlobals", true}; + + // Check Roblox globals are no longer here + CHECK_EQ("\n" + compileFunction0(source), R"( +GETIMPORT R0 1 +CALL R0 0 0 +GETIMPORT R0 3 +CALL R0 0 0 +GETIMPORT R0 5 +CALL R0 0 0 +GETIMPORT R1 7 +GETTABLEKS R0 R1 K0 +CALL R0 0 0 +GETIMPORT R0 9 +CALL R0 0 0 +GETIMPORT R0 11 +CALL R0 0 0 +GETIMPORT R0 13 +CALL R0 0 0 +GETIMPORT R0 15 +CALL R0 0 0 +GETIMPORT R0 17 +CALL R0 0 0 +RETURN R0 0 +)"); + + // Check we can add them back + Luau::BytecodeBuilder bcb; + bcb.setDumpFlags(Luau::BytecodeBuilder::Dump_Code); + Luau::CompileOptions options; + const char* mutableGlobals[] = {"Game", "Workspace", "game", "plugin", "script", "shared", "workspace", NULL}; + options.mutableGlobals = &mutableGlobals[0]; + Luau::compileOrThrow(bcb, source, options); + + CHECK_EQ("\n" + bcb.dumpFunction(0), R"( +GETIMPORT R0 1 +CALL R0 0 0 +GETIMPORT R1 3 +GETTABLEKS R0 R1 K0 +CALL R0 0 0 +GETIMPORT R1 5 +GETTABLEKS R0 R1 K0 +CALL R0 0 0 +GETIMPORT R1 7 +GETTABLEKS R0 R1 K0 +CALL R0 0 0 +GETIMPORT R1 9 +GETTABLEKS R0 R1 K0 +CALL R0 0 0 +GETIMPORT R1 11 +GETTABLEKS R0 R1 K0 +CALL R0 0 0 +GETIMPORT R1 13 +GETTABLEKS R0 R1 K0 +CALL R0 0 0 +GETIMPORT R1 15 +GETTABLEKS R0 R1 K0 +CALL R0 0 0 +GETIMPORT R1 17 +GETTABLEKS R0 R1 K0 +CALL R0 0 0 +RETURN R0 0 +)"); +} + TEST_SUITE_END(); diff --git a/tests/Conformance.test.cpp b/tests/Conformance.test.cpp index 06b3c523..c1b790b9 100644 --- a/tests/Conformance.test.cpp +++ b/tests/Conformance.test.cpp @@ -240,8 +240,6 @@ TEST_CASE("Math") TEST_CASE("Table") { - ScopedFastFlag sff("LuauTableFreeze", true); - runConformance("nextvar.lua"); } @@ -322,6 +320,8 @@ TEST_CASE("GC") TEST_CASE("Bitwise") { + ScopedFastFlag sff("LuauBit32Count", true); + runConformance("bitwise.lua"); } @@ -359,6 +359,8 @@ TEST_CASE("PCall") TEST_CASE("Pack") { + ScopedFastFlag sff{ "LuauStrPackUBCastFix", true }; + runConformance("tpack.lua"); } diff --git a/tests/Linter.test.cpp b/tests/Linter.test.cpp index 37f1b60b..7ba40c50 100644 --- a/tests/Linter.test.cpp +++ b/tests/Linter.test.cpp @@ -479,10 +479,6 @@ return foo1 TEST_CASE_FIXTURE(Fixture, "UnknownType") { - ScopedFastFlag sff{"LuauLinterUnknownTypeVectorAware", true}; - - SourceModule sm; - unfreeze(typeChecker.globalTypes); TableTypeVar::Props instanceProps{ {"ClassName", {typeChecker.anyType}}, @@ -1400,8 +1396,6 @@ end TEST_CASE_FIXTURE(Fixture, "TableOperations") { - ScopedFastFlag sff("LuauLinterTableMoveZero", true); - LintResult result = lintTyped(R"( local t = {} local tt = {} diff --git a/tests/Parser.test.cpp b/tests/Parser.test.cpp index a80718e4..e3e6ce6d 100644 --- a/tests/Parser.test.cpp +++ b/tests/Parser.test.cpp @@ -7,6 +7,8 @@ #include "doctest.h" +LUAU_FASTFLAG(LuauFixAmbiguousErrorRecoveryInAssign) + using namespace Luau; namespace @@ -625,10 +627,6 @@ TEST_CASE_FIXTURE(Fixture, "parse_error_messages") )"), "Cannot have more than one table indexer"); - ScopedFastFlag sffs1{"LuauGenericFunctions", true}; - ScopedFastFlag sffs2{"LuauGenericFunctionsParserFix", true}; - ScopedFastFlag sffs3{"LuauParseGenericFunctions", true}; - CHECK_EQ(getParseError(R"( type T = foo )"), @@ -1624,6 +1622,20 @@ TEST_CASE_FIXTURE(Fixture, "parse_error_confusing_function_call") "statements"); CHECK(result3.errors.size() == 1); + + auto result4 = matchParseError(R"( + local t = {} + function f() return t end + t.x, (f) + ().y = 5, 6 + )", + "Ambiguous syntax: this looks like an argument list for a function call, but could also be a start of new statement; use ';' to separate " + "statements"); + + if (FFlag::LuauFixAmbiguousErrorRecoveryInAssign) + CHECK(result4.errors.size() == 1); + else + CHECK(result4.errors.size() == 5); } TEST_CASE_FIXTURE(Fixture, "parse_error_varargs") @@ -1824,9 +1836,6 @@ TEST_CASE_FIXTURE(Fixture, "variadic_definition_parsing") TEST_CASE_FIXTURE(Fixture, "generic_pack_parsing") { - // Doesn't need LuauGenericFunctions - ScopedFastFlag sffs{"LuauParseGenericFunctions", true}; - ParseResult result = parseEx(R"( function f(...: a...) end @@ -1861,9 +1870,6 @@ TEST_CASE_FIXTURE(Fixture, "generic_pack_parsing") TEST_CASE_FIXTURE(Fixture, "generic_function_declaration_parsing") { - // Doesn't need LuauGenericFunctions - ScopedFastFlag sffs{"LuauParseGenericFunctions", true}; - ParseResult result = parseEx(R"( declare function f() )"); @@ -1953,12 +1959,7 @@ TEST_CASE_FIXTURE(Fixture, "function_type_named_arguments") matchParseError("type MyFunc = (a: number, b: string, c: number) -> (d: number, e: string, f: number)", "Expected '->' when parsing function type, got "); - { - ScopedFastFlag luauParseGenericFunctions{"LuauParseGenericFunctions", true}; - ScopedFastFlag luauGenericFunctionsParserFix{"LuauGenericFunctionsParserFix", true}; - - matchParseError("type MyFunc = (number) -> (d: number) -> number", "Expected '->' when parsing function type, got '<'"); - } + matchParseError("type MyFunc = (number) -> (d: number) -> number", "Expected '->' when parsing function type, got '<'"); } TEST_SUITE_END(); @@ -2362,8 +2363,6 @@ type Fn = ( CHECK_EQ("Expected '->' when parsing function type, got ')'", e.getErrors().front().getMessage()); } - ScopedFastFlag sffs3{"LuauParseGenericFunctions", true}; - try { parse(R"(type Fn = (any, string | number | ()) -> any)"); @@ -2397,8 +2396,6 @@ TEST_CASE_FIXTURE(Fixture, "AstName_comparison") TEST_CASE_FIXTURE(Fixture, "generic_type_list_recovery") { - ScopedFastFlag luauParseGenericFunctions{"LuauParseGenericFunctions", true}; - try { parse(R"( @@ -2521,7 +2518,6 @@ TEST_CASE_FIXTURE(Fixture, "parse_if_else_expression") TEST_CASE_FIXTURE(Fixture, "parse_type_pack_type_parameters") { - ScopedFastFlag luauParseGenericFunctions("LuauParseGenericFunctions", true); ScopedFastFlag luauParseTypePackTypeParameters("LuauParseTypePackTypeParameters", true); AstStat* stat = parse(R"( @@ -2534,4 +2530,9 @@ type C = Packed<(number, X...)> REQUIRE(stat != nullptr); } +TEST_CASE_FIXTURE(Fixture, "function_type_matching_parenthesis") +{ + matchParseError("local a: (number -> string", "Expected ')' (to close '(' at column 13), got '->'"); +} + TEST_SUITE_END(); diff --git a/tests/Predicate.test.cpp b/tests/Predicate.test.cpp index bb5a93c5..7081693e 100644 --- a/tests/Predicate.test.cpp +++ b/tests/Predicate.test.cpp @@ -33,8 +33,6 @@ TEST_SUITE_BEGIN("Predicate"); TEST_CASE_FIXTURE(Fixture, "Luau_merge_hashmap_order") { - ScopedFastFlag sff{"LuauOrPredicate", true}; - RefinementMap m{ {"b", typeChecker.stringType}, {"c", typeChecker.numberType}, @@ -61,8 +59,6 @@ TEST_CASE_FIXTURE(Fixture, "Luau_merge_hashmap_order") TEST_CASE_FIXTURE(Fixture, "Luau_merge_hashmap_order2") { - ScopedFastFlag sff{"LuauOrPredicate", true}; - RefinementMap m{ {"a", typeChecker.stringType}, {"b", typeChecker.stringType}, @@ -89,8 +85,6 @@ TEST_CASE_FIXTURE(Fixture, "Luau_merge_hashmap_order2") TEST_CASE_FIXTURE(Fixture, "one_map_has_overlap_at_end_whereas_other_has_it_in_start") { - ScopedFastFlag sff{"LuauOrPredicate", true}; - RefinementMap m{ {"a", typeChecker.stringType}, {"b", typeChecker.numberType}, diff --git a/tests/ToString.test.cpp b/tests/ToString.test.cpp index e18bf7cd..b076e9ad 100644 --- a/tests/ToString.test.cpp +++ b/tests/ToString.test.cpp @@ -259,9 +259,6 @@ TEST_CASE_FIXTURE(Fixture, "function_type_with_argument_names") TEST_CASE_FIXTURE(Fixture, "function_type_with_argument_names_generic") { - ScopedFastFlag luauGenericFunctions{"LuauGenericFunctions", true}; - ScopedFastFlag luauParseGenericFunctions{"LuauParseGenericFunctions", true}; - CheckResult result = check("local function f(n: number, ...: a...): (a...) return ... end"); LUAU_REQUIRE_NO_ERRORS(result); @@ -340,10 +337,6 @@ TEST_CASE_FIXTURE(Fixture, "toStringDetailed") TEST_CASE_FIXTURE(Fixture, "toStringDetailed2") { - ScopedFastFlag sff[] = { - {"LuauGenericFunctions", true}, - }; - CheckResult result = check(R"( local base = {} function base:one() return 1 end @@ -468,8 +461,6 @@ TEST_CASE_FIXTURE(Fixture, "no_parentheses_around_cyclic_function_type_in_inters TEST_CASE_FIXTURE(Fixture, "self_recursive_instantiated_param") { - ScopedFastFlag luauInstantiatedTypeParamRecursion{"LuauInstantiatedTypeParamRecursion", true}; - TypeVar tableTy{TableTypeVar{}}; TableTypeVar* ttv = getMutable(&tableTy); ttv->name = "Table"; diff --git a/tests/Transpiler.test.cpp b/tests/Transpiler.test.cpp index bfff60f9..928c03a3 100644 --- a/tests/Transpiler.test.cpp +++ b/tests/Transpiler.test.cpp @@ -21,7 +21,7 @@ local function isPortal(element) return false end - return element.component==Core.Portal + return element.component == Core.Portal end )"; @@ -223,12 +223,24 @@ TEST_CASE("escaped_strings") CHECK_EQ(code, transpile(code).code); } +TEST_CASE("escaped_strings_2") +{ + const std::string code = R"( local s="\a\b\f\n\r\t\v\'\"\\" )"; + CHECK_EQ(code, transpile(code).code); +} + TEST_CASE("need_a_space_between_number_literals_and_dots") { const std::string code = R"( return point and math.ceil(point* 100000* 100)/ 100000 .. '%'or '' )"; CHECK_EQ(code, transpile(code).code); } +TEST_CASE("binary_keywords") +{ + const std::string code = "local c = a0 ._ or b0 ._"; + CHECK_EQ(code, transpile(code).code); +} + TEST_CASE("do_blocks") { const std::string code = R"( @@ -364,10 +376,10 @@ TEST_CASE_FIXTURE(Fixture, "type_lists_should_be_emitted_correctly") )"; std::string expected = R"( - local a:(string,number,...string)->(string,...number)=function(a:string,b:number,...:...string): (string,...number) + local a:(string,number,...string)->(string,...number)=function(a:string,b:number,...:string): (string,...number) end - local b:(...string)->(...number)=function(...:...string): ...number + local b:(...string)->(...number)=function(...:string): ...number end local c:()->()=function(): () @@ -400,4 +412,238 @@ TEST_CASE_FIXTURE(Fixture, "function_type_location") CHECK_EQ(expected, actual); } +TEST_CASE_FIXTURE(Fixture, "transpile_type_assertion") +{ + std::string code = "local a = 5 :: number"; + + CHECK_EQ(code, transpile(code, {}, true).code); +} + +TEST_CASE_FIXTURE(Fixture, "transpile_if_then_else") +{ + ScopedFastFlag luauIfElseExpressionBaseSupport("LuauIfElseExpressionBaseSupport", true); + + std::string code = "local a = if 1 then 2 else 3"; + + CHECK_EQ(code, transpile(code).code); +} + +TEST_CASE_FIXTURE(Fixture, "transpile_type_reference_import") +{ + fileResolver.source["game/A"] = R"( +export type Type = { a: number } +return {} + )"; + + std::string code = R"( +local Import = require(game.A) +local a: Import.Type + )"; + + CHECK_EQ(code, transpile(code, {}, true).code); +} + +TEST_CASE_FIXTURE(Fixture, "transpile_type_packs") +{ + ScopedFastFlag luauTypeAliasPacks("LuauTypeAliasPacks", true); + ScopedFastFlag luauParseTypePackTypeParameters("LuauParseTypePackTypeParameters", true); + + std::string code = R"( +type Packed = (T...)->(T...) +local a: Packed<> +local b: Packed<(number, string)> + )"; + + CHECK_EQ(code, transpile(code, {}, true).code); +} + +TEST_CASE_FIXTURE(Fixture, "transpile_union_type_nested") +{ + std::string code = "local a: ((number)->(string))|((string)->(string))"; + + CHECK_EQ(code, transpile(code, {}, true).code); +} + +TEST_CASE_FIXTURE(Fixture, "transpile_union_type_nested_2") +{ + std::string code = "local a: (number&string)|(string&boolean)"; + + CHECK_EQ(code, transpile(code, {}, true).code); +} + +TEST_CASE_FIXTURE(Fixture, "transpile_union_type_nested_3") +{ + std::string code = "local a: nil | (string & number)"; + + CHECK_EQ("local a: ( string & number)?", transpile(code, {}, true).code); +} + +TEST_CASE_FIXTURE(Fixture, "transpile_intersection_type_nested") +{ + std::string code = "local a: ((number)->(string))&((string)->(string))"; + + CHECK_EQ(code, transpile(code, {}, true).code); +} + +TEST_CASE_FIXTURE(Fixture, "transpile_intersection_type_nested_2") +{ + std::string code = "local a: (number|string)&(string|boolean)"; + + CHECK_EQ(code, transpile(code, {}, true).code); +} + +TEST_CASE_FIXTURE(Fixture, "transpile_varargs") +{ + std::string code = "local function f(...) return ... end"; + + CHECK_EQ(code, transpile(code, {}, true).code); +} + +TEST_CASE_FIXTURE(Fixture, "transpile_index_expr") +{ + std::string code = "local a = {1, 2, 3} local b = a[2]"; + + CHECK_EQ(code, transpile(code, {}, true).code); +} + +TEST_CASE_FIXTURE(Fixture, "transpile_unary") +{ + std::string code = R"( +local a = 1 +local b = -1 +local c = true +local d = not c +local e = 'hello' +local d = #e + )"; + + CHECK_EQ(code, transpile(code, {}, true).code); +} + +TEST_CASE_FIXTURE(Fixture, "transpile_break_continue") +{ + std::string code = R"( +local a, b, c +repeat + if a then break end + if b then continue end +until c + )"; + + CHECK_EQ(code, transpile(code, {}, true).code); +} + +TEST_CASE_FIXTURE(Fixture, "transpile_compound_assignmenr") +{ + std::string code = R"( +local a = 1 +a += 2 +a -= 3 +a *= 4 +a /= 5 +a %= 6 +a ^= 7 +a ..= ' - result' + )"; + + CHECK_EQ(code, transpile(code, {}, true).code); +} + +TEST_CASE_FIXTURE(Fixture, "transpile_assign_multiple") +{ + std::string code = "a, b, c = 1, 2, 3"; + + CHECK_EQ(code, transpile(code, {}, true).code); +} + +TEST_CASE_FIXTURE(Fixture, "transpile_generic_function") +{ + ScopedFastFlag luauParseGenericFunctionTypeBegin("LuauParseGenericFunctionTypeBegin", true); + + std::string code = R"( +local function foo(a: T, ...: S...) return 1 end +local f: (T, S...)->(number) = foo + )"; + + CHECK_EQ(code, transpile(code, {}, true).code); +} + +TEST_CASE_FIXTURE(Fixture, "transpile_union_reverse") +{ + std::string code = "local a: nil | number"; + + CHECK_EQ("local a: number?", transpile(code, {}, true).code); +} + +TEST_CASE_FIXTURE(Fixture, "transpile_for_in_multiple") +{ + std::string code = "for k,v in next,{}do print(k,v) end"; + + CHECK_EQ(code, transpile(code, {}, true).code); +} + +TEST_CASE_FIXTURE(Fixture, "transpile_error_expr") +{ + std::string code = "local a = f:-"; + + auto allocator = Allocator{}; + auto names = AstNameTable{allocator}; + ParseResult parseResult = Parser::parse(code.data(), code.size(), names, allocator, {}); + + CHECK_EQ("local a = (error-expr: f.%error-id%)-(error-expr)", transpileWithTypes(*parseResult.root)); +} + +TEST_CASE_FIXTURE(Fixture, "transpile_error_stat") +{ + std::string code = "-"; + + auto allocator = Allocator{}; + auto names = AstNameTable{allocator}; + ParseResult parseResult = Parser::parse(code.data(), code.size(), names, allocator, {}); + + CHECK_EQ("(error-stat: (error-expr))", transpileWithTypes(*parseResult.root)); +} + +TEST_CASE_FIXTURE(Fixture, "transpile_error_type") +{ + std::string code = "local a: "; + + auto allocator = Allocator{}; + auto names = AstNameTable{allocator}; + ParseResult parseResult = Parser::parse(code.data(), code.size(), names, allocator, {}); + + CHECK_EQ("local a:%error-type%", transpileWithTypes(*parseResult.root)); +} + +TEST_CASE_FIXTURE(Fixture, "transpile_parse_error") +{ + std::string code = "local a = -"; + + auto result = transpile(code); + CHECK_EQ("", result.code); + CHECK_EQ("Expected identifier when parsing expression, got ", result.parseError); +} + +TEST_CASE_FIXTURE(Fixture, "transpile_to_string") +{ + std::string code = "local a: string = 'hello'"; + + auto allocator = Allocator{}; + auto names = AstNameTable{allocator}; + ParseResult parseResult = Parser::parse(code.data(), code.size(), names, allocator, {}); + + REQUIRE(parseResult.root); + REQUIRE(parseResult.root->body.size == 1); + AstStatLocal* statLocal = parseResult.root->body.data[0]->as(); + REQUIRE(statLocal); + CHECK_EQ("local a: string = 'hello'", toString(statLocal)); + REQUIRE(statLocal->vars.size == 1); + AstLocal* local = statLocal->vars.data[0]; + REQUIRE(local->annotation); + CHECK_EQ("string", toString(local->annotation)); + REQUIRE(statLocal->values.size == 1); + AstExpr* expr = statLocal->values.data[0]; + CHECK_EQ("'hello'", toString(expr)); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.aliases.test.cpp b/tests/TypeInfer.aliases.test.cpp index f580604c..c27f8083 100644 --- a/tests/TypeInfer.aliases.test.cpp +++ b/tests/TypeInfer.aliases.test.cpp @@ -247,9 +247,6 @@ TEST_CASE_FIXTURE(Fixture, "export_type_and_type_alias_are_duplicates") TEST_CASE_FIXTURE(Fixture, "stringify_optional_parameterized_alias") { - ScopedFastFlag sffs3{"LuauGenericFunctions", true}; - ScopedFastFlag sffs4{"LuauParseGenericFunctions", true}; - CheckResult result = check(R"( type Node = { value: T, child: Node? } diff --git a/tests/TypeInfer.builtins.test.cpp b/tests/TypeInfer.builtins.test.cpp index 17e32e9f..1e2eae14 100644 --- a/tests/TypeInfer.builtins.test.cpp +++ b/tests/TypeInfer.builtins.test.cpp @@ -444,8 +444,6 @@ TEST_CASE_FIXTURE(Fixture, "os_time_takes_optional_date_table") TEST_CASE_FIXTURE(Fixture, "thread_is_a_type") { - ScopedFastFlag sff{"LuauDontMutatePersistentFunctions", true}; - CheckResult result = check(R"( local co = coroutine.create(function() end) )"); @@ -456,8 +454,6 @@ TEST_CASE_FIXTURE(Fixture, "thread_is_a_type") TEST_CASE_FIXTURE(Fixture, "coroutine_resume_anything_goes") { - ScopedFastFlag sff{"LuauDontMutatePersistentFunctions", true}; - CheckResult result = check(R"( local function nifty(x, y) print(x, y) @@ -476,8 +472,6 @@ TEST_CASE_FIXTURE(Fixture, "coroutine_resume_anything_goes") TEST_CASE_FIXTURE(Fixture, "coroutine_wrap_anything_goes") { - ScopedFastFlag sff{"LuauDontMutatePersistentFunctions", true}; - CheckResult result = check(R"( --!nonstrict local function nifty(x, y) @@ -822,8 +816,6 @@ TEST_CASE_FIXTURE(Fixture, "string_format_report_all_type_errors_at_correct_posi TEST_CASE_FIXTURE(Fixture, "dont_add_definitions_to_persistent_types") { - ScopedFastFlag sff{"LuauDontMutatePersistentFunctions", true}; - CheckResult result = check(R"( local f = math.sin local function g(x) return math.sin(x) end diff --git a/tests/TypeInfer.classes.test.cpp b/tests/TypeInfer.classes.test.cpp index eabf7e65..1ff23fe6 100644 --- a/tests/TypeInfer.classes.test.cpp +++ b/tests/TypeInfer.classes.test.cpp @@ -232,8 +232,6 @@ TEST_CASE_FIXTURE(ClassFixture, "can_assign_to_prop_of_base_class") TEST_CASE_FIXTURE(ClassFixture, "can_read_prop_of_base_class_using_string") { - ScopedFastFlag luauClassPropertyAccessAsString("LuauClassPropertyAccessAsString", true); - CheckResult result = check(R"( local c = ChildClass.New() local x = 1 + c["BaseField"] @@ -244,8 +242,6 @@ TEST_CASE_FIXTURE(ClassFixture, "can_read_prop_of_base_class_using_string") TEST_CASE_FIXTURE(ClassFixture, "can_assign_to_prop_of_base_class_using_string") { - ScopedFastFlag luauClassPropertyAccessAsString("LuauClassPropertyAccessAsString", true); - CheckResult result = check(R"( local c = ChildClass.New() c["BaseField"] = 444 @@ -451,4 +447,25 @@ b.X = 2 -- real Vector2.X is also read-only CHECK_EQ("Value of type 'Vector2?' could be nil", toString(result.errors[3])); } +TEST_CASE_FIXTURE(ClassFixture, "detailed_class_unification_error") +{ + ScopedFastFlag luauExtendedClassMismatchError{"LuauExtendedClassMismatchError", true}; + + CheckResult result = check(R"( +local function foo(v) + return v.X :: number + string.len(v.Y) +end + +local a: Vector2 +local b = foo +b(a) + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ(R"(Type 'Vector2' could not be converted into '{- X: a, Y: string -}' +caused by: + Property 'Y' is not compatible. Type 'number' could not be converted into 'string')", + toString(result.errors[0])); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.definitions.test.cpp b/tests/TypeInfer.definitions.test.cpp index 41e3e45a..2652486b 100644 --- a/tests/TypeInfer.definitions.test.cpp +++ b/tests/TypeInfer.definitions.test.cpp @@ -171,9 +171,6 @@ TEST_CASE_FIXTURE(Fixture, "no_cyclic_defined_classes") TEST_CASE_FIXTURE(Fixture, "declaring_generic_functions") { - ScopedFastFlag sffs{"LuauGenericFunctions", true}; - ScopedFastFlag sffs4{"LuauParseGenericFunctions", true}; - loadDefinition(R"( declare function f(a: a, b: b): string declare function g(...: a...): b... diff --git a/tests/TypeInfer.generics.test.cpp b/tests/TypeInfer.generics.test.cpp index 581375a1..de2f0154 100644 --- a/tests/TypeInfer.generics.test.cpp +++ b/tests/TypeInfer.generics.test.cpp @@ -13,8 +13,6 @@ TEST_SUITE_BEGIN("GenericsTests"); TEST_CASE_FIXTURE(Fixture, "check_generic_function") { - ScopedFastFlag sffs{"LuauGenericFunctions", true}; - ScopedFastFlag sffs2{"LuauParseGenericFunctions", true}; CheckResult result = check(R"( function id(x:a): a return x @@ -27,8 +25,6 @@ TEST_CASE_FIXTURE(Fixture, "check_generic_function") TEST_CASE_FIXTURE(Fixture, "check_generic_local_function") { - ScopedFastFlag sffs{"LuauGenericFunctions", true}; - ScopedFastFlag sffs2{"LuauParseGenericFunctions", true}; CheckResult result = check(R"( local function id(x:a): a return x @@ -41,10 +37,6 @@ TEST_CASE_FIXTURE(Fixture, "check_generic_local_function") TEST_CASE_FIXTURE(Fixture, "check_generic_typepack_function") { - ScopedFastFlag sffs{"LuauGenericFunctions", true}; - ScopedFastFlag sffs4{"LuauGenericVariadicsUnification", true}; - ScopedFastFlag sffs5{"LuauParseGenericFunctions", true}; - CheckResult result = check(R"( function id(...: a...): (a...) return ... end local x: string, y: boolean = id("hi", true) @@ -56,8 +48,6 @@ TEST_CASE_FIXTURE(Fixture, "check_generic_typepack_function") TEST_CASE_FIXTURE(Fixture, "types_before_typepacks") { - ScopedFastFlag sffs{"LuauGenericFunctions", true}; - ScopedFastFlag sffs2{"LuauParseGenericFunctions", true}; CheckResult result = check(R"( function f() end )"); @@ -66,8 +56,6 @@ TEST_CASE_FIXTURE(Fixture, "types_before_typepacks") TEST_CASE_FIXTURE(Fixture, "local_vars_can_be_polytypes") { - ScopedFastFlag sffs{"LuauGenericFunctions", true}; - ScopedFastFlag sffs2{"LuauParseGenericFunctions", true}; CheckResult result = check(R"( local function id(x:a):a return x end local f: (a)->a = id @@ -79,7 +67,6 @@ TEST_CASE_FIXTURE(Fixture, "local_vars_can_be_polytypes") TEST_CASE_FIXTURE(Fixture, "inferred_local_vars_can_be_polytypes") { - ScopedFastFlag sffs{"LuauGenericFunctions", true}; CheckResult result = check(R"( local function id(x) return x end print("This is bogus") -- TODO: CLI-39916 @@ -92,7 +79,6 @@ TEST_CASE_FIXTURE(Fixture, "inferred_local_vars_can_be_polytypes") TEST_CASE_FIXTURE(Fixture, "local_vars_can_be_instantiated_polytypes") { - ScopedFastFlag sffs{"LuauGenericFunctions", true}; CheckResult result = check(R"( local function id(x) return x end print("This is bogus") -- TODO: CLI-39916 @@ -104,8 +90,6 @@ TEST_CASE_FIXTURE(Fixture, "local_vars_can_be_instantiated_polytypes") TEST_CASE_FIXTURE(Fixture, "properties_can_be_polytypes") { - ScopedFastFlag sffs{"LuauGenericFunctions", true}; - ScopedFastFlag sffs2{"LuauParseGenericFunctions", true}; CheckResult result = check(R"( local t = {} t.m = function(x: a):a return x end @@ -117,8 +101,6 @@ TEST_CASE_FIXTURE(Fixture, "properties_can_be_polytypes") TEST_CASE_FIXTURE(Fixture, "properties_can_be_instantiated_polytypes") { - ScopedFastFlag sffs{"LuauGenericFunctions", true}; - ScopedFastFlag sffs2{"LuauParseGenericFunctions", true}; CheckResult result = check(R"( local t: { m: (number)->number } = { m = function(x:number) return x+1 end } local function id(x:a):a return x end @@ -129,8 +111,6 @@ TEST_CASE_FIXTURE(Fixture, "properties_can_be_instantiated_polytypes") TEST_CASE_FIXTURE(Fixture, "check_nested_generic_function") { - ScopedFastFlag sffs{"LuauGenericFunctions", true}; - ScopedFastFlag sffs2{"LuauParseGenericFunctions", true}; CheckResult result = check(R"( local function f() local function id(x:a): a @@ -145,8 +125,6 @@ TEST_CASE_FIXTURE(Fixture, "check_nested_generic_function") TEST_CASE_FIXTURE(Fixture, "check_recursive_generic_function") { - ScopedFastFlag sffs{"LuauGenericFunctions", true}; - ScopedFastFlag sffs2{"LuauParseGenericFunctions", true}; CheckResult result = check(R"( local function id(x:a):a local y: string = id("hi") @@ -159,8 +137,6 @@ TEST_CASE_FIXTURE(Fixture, "check_recursive_generic_function") TEST_CASE_FIXTURE(Fixture, "check_mutual_generic_functions") { - ScopedFastFlag sffs{"LuauGenericFunctions", true}; - ScopedFastFlag sffs2{"LuauParseGenericFunctions", true}; CheckResult result = check(R"( local id2 local function id1(x:a):a @@ -179,8 +155,6 @@ TEST_CASE_FIXTURE(Fixture, "check_mutual_generic_functions") TEST_CASE_FIXTURE(Fixture, "generic_functions_in_types") { - ScopedFastFlag sffs{"LuauGenericFunctions", true}; - ScopedFastFlag sffs2{"LuauParseGenericFunctions", true}; CheckResult result = check(R"( type T = { id: (a) -> a } local x: T = { id = function(x:a):a return x end } @@ -192,8 +166,6 @@ TEST_CASE_FIXTURE(Fixture, "generic_functions_in_types") TEST_CASE_FIXTURE(Fixture, "generic_factories") { - ScopedFastFlag sffs{"LuauGenericFunctions", true}; - ScopedFastFlag sffs2{"LuauParseGenericFunctions", true}; CheckResult result = check(R"( type T = { id: (a) -> a } type Factory = { build: () -> T } @@ -215,10 +187,6 @@ TEST_CASE_FIXTURE(Fixture, "generic_factories") TEST_CASE_FIXTURE(Fixture, "factories_of_generics") { - ScopedFastFlag sffs{"LuauGenericFunctions", true}; - ScopedFastFlag sffs2{"LuauParseGenericFunctions", true}; - ScopedFastFlag sffs3{"LuauRankNTypes", true}; - CheckResult result = check(R"( type T = { id: (a) -> a } type Factory = { build: () -> T } @@ -241,7 +209,6 @@ TEST_CASE_FIXTURE(Fixture, "factories_of_generics") TEST_CASE_FIXTURE(Fixture, "infer_generic_function") { - ScopedFastFlag sffs{"LuauGenericFunctions", true}; CheckResult result = check(R"( function id(x) return x @@ -265,7 +232,6 @@ TEST_CASE_FIXTURE(Fixture, "infer_generic_function") TEST_CASE_FIXTURE(Fixture, "infer_generic_local_function") { - ScopedFastFlag sffs{"LuauGenericFunctions", true}; CheckResult result = check(R"( local function id(x) return x @@ -289,7 +255,6 @@ TEST_CASE_FIXTURE(Fixture, "infer_generic_local_function") TEST_CASE_FIXTURE(Fixture, "infer_nested_generic_function") { - ScopedFastFlag sffs{"LuauGenericFunctions", true}; CheckResult result = check(R"( local function f() local function id(x) @@ -304,7 +269,6 @@ TEST_CASE_FIXTURE(Fixture, "infer_nested_generic_function") TEST_CASE_FIXTURE(Fixture, "infer_generic_methods") { - ScopedFastFlag sffs{"LuauGenericFunctions", true}; CheckResult result = check(R"( local x = {} function x:id(x) return x end @@ -316,7 +280,6 @@ TEST_CASE_FIXTURE(Fixture, "infer_generic_methods") TEST_CASE_FIXTURE(Fixture, "calling_self_generic_methods") { - ScopedFastFlag sffs{"LuauGenericFunctions", true}; CheckResult result = check(R"( local x = {} function x:id(x) return x end @@ -331,8 +294,6 @@ TEST_CASE_FIXTURE(Fixture, "calling_self_generic_methods") TEST_CASE_FIXTURE(Fixture, "infer_generic_property") { - ScopedFastFlag sffs{"LuauGenericFunctions", true}; - ScopedFastFlag sffs2{"LuauRankNTypes", true}; CheckResult result = check(R"( local t = {} t.m = function(x) return x end @@ -344,9 +305,6 @@ TEST_CASE_FIXTURE(Fixture, "infer_generic_property") TEST_CASE_FIXTURE(Fixture, "function_arguments_can_be_polytypes") { - ScopedFastFlag sffs{"LuauGenericFunctions", true}; - ScopedFastFlag sffs2{"LuauParseGenericFunctions", true}; - ScopedFastFlag sffs3{"LuauRankNTypes", true}; CheckResult result = check(R"( local function f(g: (a)->a) local x: number = g(37) @@ -358,9 +316,6 @@ TEST_CASE_FIXTURE(Fixture, "function_arguments_can_be_polytypes") TEST_CASE_FIXTURE(Fixture, "function_results_can_be_polytypes") { - ScopedFastFlag sffs{"LuauGenericFunctions", true}; - ScopedFastFlag sffs2{"LuauParseGenericFunctions", true}; - ScopedFastFlag sffs3{"LuauRankNTypes", true}; CheckResult result = check(R"( local function f() : (a)->a local function id(x:a):a return x end @@ -372,9 +327,6 @@ TEST_CASE_FIXTURE(Fixture, "function_results_can_be_polytypes") TEST_CASE_FIXTURE(Fixture, "type_parameters_can_be_polytypes") { - ScopedFastFlag sffs1{"LuauGenericFunctions", true}; - ScopedFastFlag sffs2{"LuauParseGenericFunctions", true}; - ScopedFastFlag sffs3{"LuauRankNTypes", true}; CheckResult result = check(R"( local function id(x:a):a return x end local f: (a)->a = id(id) @@ -384,7 +336,6 @@ TEST_CASE_FIXTURE(Fixture, "type_parameters_can_be_polytypes") TEST_CASE_FIXTURE(Fixture, "dont_leak_generic_types") { - ScopedFastFlag sffs{"LuauGenericFunctions", true}; CheckResult result = check(R"( local function f(y) -- this will only typecheck if we infer z: any @@ -406,7 +357,6 @@ TEST_CASE_FIXTURE(Fixture, "dont_leak_generic_types") TEST_CASE_FIXTURE(Fixture, "dont_leak_inferred_generic_types") { - ScopedFastFlag sffs{"LuauGenericFunctions", true}; CheckResult result = check(R"( local function f(y) local z = y @@ -423,12 +373,6 @@ TEST_CASE_FIXTURE(Fixture, "dont_leak_inferred_generic_types") TEST_CASE_FIXTURE(Fixture, "dont_substitute_bound_types") { - ScopedFastFlag sffs[] = { - {"LuauGenericFunctions", true}, - {"LuauParseGenericFunctions", true}, - {"LuauRankNTypes", true}, - }; - CheckResult result = check(R"( type T = { m: (a) -> T } function f(t : T) @@ -440,10 +384,6 @@ TEST_CASE_FIXTURE(Fixture, "dont_substitute_bound_types") TEST_CASE_FIXTURE(Fixture, "dont_unify_bound_types") { - ScopedFastFlag sffs{"LuauGenericFunctions", true}; - ScopedFastFlag sffs2{"LuauParseGenericFunctions", true}; - ScopedFastFlag sffs3{"LuauRankNTypes", true}; - CheckResult result = check(R"( type F = () -> (a, b) -> a type G = (b, b) -> b @@ -470,7 +410,6 @@ TEST_CASE_FIXTURE(Fixture, "mutable_state_polymorphism") // Replaying the classic problem with polymorphism and mutable state in Luau // See, e.g. Tofte (1990) // https://www.sciencedirect.com/science/article/pii/089054019090018D. - ScopedFastFlag sffs{"LuauGenericFunctions", true}; CheckResult result = check(R"( --!strict -- Our old friend the polymorphic identity function @@ -508,7 +447,6 @@ TEST_CASE_FIXTURE(Fixture, "mutable_state_polymorphism") TEST_CASE_FIXTURE(Fixture, "rank_N_types_via_typeof") { - ScopedFastFlag sffs{"LuauGenericFunctions", false}; CheckResult result = check(R"( --!strict local function id(x) return x end @@ -531,8 +469,6 @@ TEST_CASE_FIXTURE(Fixture, "rank_N_types_via_typeof") TEST_CASE_FIXTURE(Fixture, "duplicate_generic_types") { - ScopedFastFlag sffs{"LuauGenericFunctions", true}; - ScopedFastFlag sffs2{"LuauParseGenericFunctions", true}; CheckResult result = check(R"( function f(x:a):a return x end )"); @@ -541,7 +477,6 @@ TEST_CASE_FIXTURE(Fixture, "duplicate_generic_types") TEST_CASE_FIXTURE(Fixture, "duplicate_generic_type_packs") { - ScopedFastFlag sffs{"LuauGenericFunctions", true}; CheckResult result = check(R"( function f() end )"); @@ -550,7 +485,6 @@ TEST_CASE_FIXTURE(Fixture, "duplicate_generic_type_packs") TEST_CASE_FIXTURE(Fixture, "typepacks_before_types") { - ScopedFastFlag sffs{"LuauGenericFunctions", true}; CheckResult result = check(R"( function f() end )"); @@ -559,9 +493,6 @@ TEST_CASE_FIXTURE(Fixture, "typepacks_before_types") TEST_CASE_FIXTURE(Fixture, "variadic_generics") { - ScopedFastFlag sffs{"LuauGenericFunctions", true}; - ScopedFastFlag sffs3{"LuauParseGenericFunctions", true}; - CheckResult result = check(R"( function f(...: a) end @@ -573,9 +504,6 @@ TEST_CASE_FIXTURE(Fixture, "variadic_generics") TEST_CASE_FIXTURE(Fixture, "generic_type_pack_syntax") { - ScopedFastFlag sffs{"LuauGenericFunctions", true}; - ScopedFastFlag sffs4{"LuauParseGenericFunctions", true}; - CheckResult result = check(R"( function f(...: a...): (a...) return ... end )"); @@ -586,10 +514,6 @@ TEST_CASE_FIXTURE(Fixture, "generic_type_pack_syntax") TEST_CASE_FIXTURE(Fixture, "generic_type_pack_parentheses") { - ScopedFastFlag sffs{"LuauGenericFunctions", true}; - ScopedFastFlag sffs4{"LuauGenericVariadicsUnification", true}; - ScopedFastFlag sffs5{"LuauParseGenericFunctions", true}; - CheckResult result = check(R"( function f(...: a...): any return (...) end )"); @@ -599,9 +523,6 @@ TEST_CASE_FIXTURE(Fixture, "generic_type_pack_parentheses") TEST_CASE_FIXTURE(Fixture, "better_mismatch_error_messages") { - ScopedFastFlag sffs{"LuauGenericFunctions", true}; - ScopedFastFlag sffs5{"LuauParseGenericFunctions", true}; - CheckResult result = check(R"( function f(...: T...) return ... @@ -626,9 +547,6 @@ TEST_CASE_FIXTURE(Fixture, "better_mismatch_error_messages") TEST_CASE_FIXTURE(Fixture, "reject_clashing_generic_and_pack_names") { - ScopedFastFlag sffs{"LuauGenericFunctions", true}; - ScopedFastFlag sffs3{"LuauParseGenericFunctions", true}; - CheckResult result = check(R"( function f() end )"); @@ -641,8 +559,6 @@ TEST_CASE_FIXTURE(Fixture, "reject_clashing_generic_and_pack_names") TEST_CASE_FIXTURE(Fixture, "instantiation_sharing_types") { - ScopedFastFlag sffs1{"LuauGenericFunctions", true}; - CheckResult result = check(R"( function f(z) local o = {} @@ -665,8 +581,6 @@ TEST_CASE_FIXTURE(Fixture, "instantiation_sharing_types") TEST_CASE_FIXTURE(Fixture, "quantification_sharing_types") { - ScopedFastFlag sffs1{"LuauGenericFunctions", true}; - CheckResult result = check(R"( function f(x) return {5} end function g(x, y) return f(x) end @@ -680,8 +594,6 @@ TEST_CASE_FIXTURE(Fixture, "quantification_sharing_types") TEST_CASE_FIXTURE(Fixture, "typefuns_sharing_types") { - ScopedFastFlag sffs1{"LuauGenericFunctions", true}; - CheckResult result = check(R"( type T = { x: {a}, y: {number} } local o1: T = { x = {true}, y = {5} } @@ -697,7 +609,6 @@ TEST_CASE_FIXTURE(Fixture, "typefuns_sharing_types") TEST_CASE_FIXTURE(Fixture, "bound_tables_do_not_clone_original_fields") { - ScopedFastFlag luauRankNTypes{"LuauRankNTypes", true}; ScopedFastFlag luauCloneBoundTables{"LuauCloneBoundTables", true}; CheckResult result = check(R"( diff --git a/tests/TypeInfer.intersectionTypes.test.cpp b/tests/TypeInfer.intersectionTypes.test.cpp index 9685f4f3..893bc2b3 100644 --- a/tests/TypeInfer.intersectionTypes.test.cpp +++ b/tests/TypeInfer.intersectionTypes.test.cpp @@ -341,4 +341,43 @@ TEST_CASE_FIXTURE(Fixture, "table_intersection_setmetatable") LUAU_REQUIRE_NO_ERRORS(result); } +TEST_CASE_FIXTURE(Fixture, "error_detailed_intersection_part") +{ + ScopedFastFlag luauExtendedTypeMismatchError{"LuauExtendedTypeMismatchError", true}; + + CheckResult result = check(R"( +type X = { x: number } +type Y = { y: number } +type Z = { z: number } + +type XYZ = X & Y & Z + +local a: XYZ = 3 + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ(toString(result.errors[0]), R"(Type 'number' could not be converted into 'X & Y & Z' +caused by: + Not all intersection parts are compatible. Type 'number' could not be converted into 'X')"); +} + +TEST_CASE_FIXTURE(Fixture, "error_detailed_intersection_all") +{ + ScopedFastFlag luauExtendedTypeMismatchError{"LuauExtendedTypeMismatchError", true}; + + CheckResult result = check(R"( +type X = { x: number } +type Y = { y: number } +type Z = { z: number } + +type XYZ = X & Y & Z + +local a: XYZ +local b: number = a + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ(toString(result.errors[0]), R"(Type 'X & Y & Z' could not be converted into 'number'; none of the intersection parts are compatible)"); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.provisional.test.cpp b/tests/TypeInfer.provisional.test.cpp index 419da8ad..e5c14dde 100644 --- a/tests/TypeInfer.provisional.test.cpp +++ b/tests/TypeInfer.provisional.test.cpp @@ -194,9 +194,6 @@ TEST_CASE_FIXTURE(Fixture, "normal_conditional_expression_has_refinements") // Luau currently doesn't yet know how to allow assignments when the binding was refined. TEST_CASE_FIXTURE(Fixture, "while_body_are_also_refined") { - ScopedFastFlag sffs2{"LuauGenericFunctions", true}; - ScopedFastFlag sffs5{"LuauParseGenericFunctions", true}; - CheckResult result = check(R"( type Node = { value: T, child: Node? } @@ -596,11 +593,9 @@ TEST_CASE_FIXTURE(Fixture, "invariant_table_properties_means_instantiating_table TEST_CASE_FIXTURE(Fixture, "self_recursive_instantiated_param") { ScopedFastFlag luauCloneCorrectlyBeforeMutatingTableType{"LuauCloneCorrectlyBeforeMutatingTableType", true}; - ScopedFastFlag luauFollowInTypeFunApply{"LuauFollowInTypeFunApply", true}; - ScopedFastFlag luauInstantiatedTypeParamRecursion{"LuauInstantiatedTypeParamRecursion", true}; // Mutability in type function application right now can create strange recursive types - // TODO: instantiation right now is problematic, it this example should either leave the Table type alone + // TODO: instantiation right now is problematic, in this example should either leave the Table type alone // or it should rename the type to 'Self' so that the result will be 'Self' CheckResult result = check(R"( type Table = { a: number } diff --git a/tests/TypeInfer.refinements.test.cpp b/tests/TypeInfer.refinements.test.cpp index 36dcaa95..733fc39b 100644 --- a/tests/TypeInfer.refinements.test.cpp +++ b/tests/TypeInfer.refinements.test.cpp @@ -7,7 +7,6 @@ #include "doctest.h" LUAU_FASTFLAG(LuauWeakEqConstraint) -LUAU_FASTFLAG(LuauOrPredicate) LUAU_FASTFLAG(LuauQuantifyInPlace2) using namespace Luau; @@ -133,11 +132,8 @@ TEST_CASE_FIXTURE(Fixture, "or_predicate_with_truthy_predicates") CHECK_EQ("string?", toString(requireTypeAtPosition({3, 26}))); CHECK_EQ("number?", toString(requireTypeAtPosition({4, 26}))); - if (FFlag::LuauOrPredicate) - { - CHECK_EQ("nil", toString(requireTypeAtPosition({6, 26}))); - CHECK_EQ("nil", toString(requireTypeAtPosition({7, 26}))); - } + CHECK_EQ("nil", toString(requireTypeAtPosition({6, 26}))); + CHECK_EQ("nil", toString(requireTypeAtPosition({7, 26}))); } TEST_CASE_FIXTURE(Fixture, "type_assertion_expr_carry_its_constraints") @@ -283,6 +279,8 @@ TEST_CASE_FIXTURE(Fixture, "assert_non_binary_expressions_actually_resolve_const TEST_CASE_FIXTURE(Fixture, "assign_table_with_refined_property_with_a_similar_type_is_illegal") { + ScopedFastFlag luauTableSubtypingVariance{"LuauTableSubtypingVariance", true}; + ScopedFastFlag luauExtendedTypeMismatchError{"LuauExtendedTypeMismatchError", true}; CheckResult result = check(R"( local t: {x: number?} = {x = nil} @@ -293,7 +291,10 @@ TEST_CASE_FIXTURE(Fixture, "assign_table_with_refined_property_with_a_similar_ty )"); LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK_EQ("Type '{| x: number? |}' could not be converted into '{| x: number |}'", toString(result.errors[0])); + CHECK_EQ(R"(Type '{| x: number? |}' could not be converted into '{| x: number |}' +caused by: + Property 'x' is not compatible. Type 'number?' could not be converted into 'number')", + toString(result.errors[0])); } TEST_CASE_FIXTURE(Fixture, "lvalue_is_equal_to_another_lvalue") @@ -749,8 +750,6 @@ TEST_CASE_FIXTURE(RefinementClassFixture, "type_narrow_for_all_the_userdata") TEST_CASE_FIXTURE(RefinementClassFixture, "eliminate_subclasses_of_instance") { - ScopedFastFlag sff{"LuauTypeGuardPeelsAwaySubclasses", true}; - CheckResult result = check(R"( local function f(x: Part | Folder | string) if typeof(x) == "Instance" then @@ -769,8 +768,6 @@ TEST_CASE_FIXTURE(RefinementClassFixture, "eliminate_subclasses_of_instance") TEST_CASE_FIXTURE(RefinementClassFixture, "narrow_this_large_union") { - ScopedFastFlag sff{"LuauTypeGuardPeelsAwaySubclasses", true}; - CheckResult result = check(R"( local function f(x: Part | Folder | Instance | string | Vector3 | any) if typeof(x) == "Instance" then @@ -789,8 +786,6 @@ TEST_CASE_FIXTURE(RefinementClassFixture, "narrow_this_large_union") TEST_CASE_FIXTURE(RefinementClassFixture, "x_as_any_if_x_is_instance_elseif_x_is_table") { - ScopedFastFlag sff{"LuauOrPredicate", true}; - CheckResult result = check(R"( --!nonstrict @@ -811,11 +806,6 @@ TEST_CASE_FIXTURE(RefinementClassFixture, "x_as_any_if_x_is_instance_elseif_x_is TEST_CASE_FIXTURE(RefinementClassFixture, "x_is_not_instance_or_else_not_part") { - ScopedFastFlag sffs[] = { - {"LuauOrPredicate", true}, - {"LuauTypeGuardPeelsAwaySubclasses", true}, - }; - CheckResult result = check(R"( local function f(x: Part | Folder | string) if typeof(x) ~= "Instance" or not x:IsA("Part") then @@ -890,8 +880,6 @@ TEST_CASE_FIXTURE(Fixture, "type_guard_warns_on_no_overlapping_types_only_when_s TEST_CASE_FIXTURE(Fixture, "not_a_or_not_b") { - ScopedFastFlag sff{"LuauOrPredicate", true}; - CheckResult result = check(R"( local function f(a: number?, b: number?) if (not a) or (not b) then @@ -909,8 +897,6 @@ TEST_CASE_FIXTURE(Fixture, "not_a_or_not_b") TEST_CASE_FIXTURE(Fixture, "not_a_or_not_b2") { - ScopedFastFlag sff{"LuauOrPredicate", true}; - CheckResult result = check(R"( local function f(a: number?, b: number?) if not (a and b) then @@ -928,8 +914,6 @@ TEST_CASE_FIXTURE(Fixture, "not_a_or_not_b2") TEST_CASE_FIXTURE(Fixture, "not_a_and_not_b") { - ScopedFastFlag sff{"LuauOrPredicate", true}; - CheckResult result = check(R"( local function f(a: number?, b: number?) if (not a) and (not b) then @@ -947,8 +931,6 @@ TEST_CASE_FIXTURE(Fixture, "not_a_and_not_b") TEST_CASE_FIXTURE(Fixture, "not_a_and_not_b2") { - ScopedFastFlag sff{"LuauOrPredicate", true}; - CheckResult result = check(R"( local function f(a: number?, b: number?) if not (a or b) then @@ -966,8 +948,6 @@ TEST_CASE_FIXTURE(Fixture, "not_a_and_not_b2") TEST_CASE_FIXTURE(Fixture, "either_number_or_string") { - ScopedFastFlag sff{"LuauOrPredicate", true}; - CheckResult result = check(R"( local function f(x: any) if type(x) == "number" or type(x) == "string" then @@ -983,8 +963,6 @@ TEST_CASE_FIXTURE(Fixture, "either_number_or_string") TEST_CASE_FIXTURE(Fixture, "not_t_or_some_prop_of_t") { - ScopedFastFlag sff{"LuauOrPredicate", true}; - CheckResult result = check(R"( local function f(t: {x: boolean}?) if not t or t.x then @@ -1000,8 +978,6 @@ TEST_CASE_FIXTURE(Fixture, "not_t_or_some_prop_of_t") TEST_CASE_FIXTURE(Fixture, "assert_a_to_be_truthy_then_assert_a_to_be_number") { - ScopedFastFlag sff{"LuauOrPredicate", true}; - CheckResult result = check(R"( local a: (number | string)? assert(a) @@ -1018,8 +994,6 @@ TEST_CASE_FIXTURE(Fixture, "assert_a_to_be_truthy_then_assert_a_to_be_number") TEST_CASE_FIXTURE(Fixture, "merge_should_be_fully_agnostic_of_hashmap_ordering") { - ScopedFastFlag sff{"LuauOrPredicate", true}; - // This bug came up because there was a mistake in Luau::merge where zipping on two maps would produce the wrong merged result. CheckResult result = check(R"( local function f(b: string | { x: string }, a) @@ -1039,8 +1013,6 @@ TEST_CASE_FIXTURE(Fixture, "merge_should_be_fully_agnostic_of_hashmap_ordering") TEST_CASE_FIXTURE(Fixture, "refine_the_correct_types_opposite_of_when_a_is_not_number_or_string") { - ScopedFastFlag sff{"LuauOrPredicate", true}; - CheckResult result = check(R"( local function f(a: string | number | boolean) if type(a) ~= "number" and type(a) ~= "string" then diff --git a/tests/TypeInfer.tables.test.cpp b/tests/TypeInfer.tables.test.cpp index f1451a81..c3694be7 100644 --- a/tests/TypeInfer.tables.test.cpp +++ b/tests/TypeInfer.tables.test.cpp @@ -1950,4 +1950,76 @@ TEST_CASE_FIXTURE(Fixture, "table_insert_should_cope_with_optional_properties_in LUAU_REQUIRE_NO_ERRORS(result); } +TEST_CASE_FIXTURE(Fixture, "error_detailed_prop") +{ + ScopedFastFlag luauTableSubtypingVariance{"LuauTableSubtypingVariance", true}; // Only for new path + ScopedFastFlag luauExtendedTypeMismatchError{"LuauExtendedTypeMismatchError", true}; + + CheckResult result = check(R"( +type A = { x: number, y: number } +type B = { x: number, y: string } + +local a: A +local b: B = a + )"); + + LUAU_REQUIRE_ERRORS(result); + CHECK_EQ(toString(result.errors[0]), R"(Type 'A' could not be converted into 'B' +caused by: + Property 'y' is not compatible. Type 'number' could not be converted into 'string')"); +} + +TEST_CASE_FIXTURE(Fixture, "error_detailed_prop_nested") +{ + ScopedFastFlag luauTableSubtypingVariance{"LuauTableSubtypingVariance", true}; // Only for new path + ScopedFastFlag luauExtendedTypeMismatchError{"LuauExtendedTypeMismatchError", true}; + + CheckResult result = check(R"( +type AS = { x: number, y: number } +type BS = { x: number, y: string } + +type A = { a: boolean, b: AS } +type B = { a: boolean, b: BS } + +local a: A +local b: B = a + )"); + + LUAU_REQUIRE_ERRORS(result); + CHECK_EQ(toString(result.errors[0]), R"(Type 'A' could not be converted into 'B' +caused by: + Property 'b' is not compatible. Type 'AS' could not be converted into 'BS' +caused by: + Property 'y' is not compatible. Type 'number' could not be converted into 'string')"); +} + +TEST_CASE_FIXTURE(Fixture, "error_detailed_metatable_prop") +{ + ScopedFastFlag luauTableSubtypingVariance{"LuauTableSubtypingVariance", true}; // Only for new path + ScopedFastFlag luauExtendedTypeMismatchError{"LuauExtendedTypeMismatchError", true}; + + CheckResult result = check(R"( +local a1 = setmetatable({ x = 2, y = 3 }, { __call = function(s) end }); +local b1 = setmetatable({ x = 2, y = "hello" }, { __call = function(s) end }); +local c1: typeof(a1) = b1 + +local a2 = setmetatable({ x = 2, y = 3 }, { __call = function(s) end }); +local b2 = setmetatable({ x = 2, y = 4 }, { __call = function(s, t) end }); +local c2: typeof(a2) = b2 + )"); + + LUAU_REQUIRE_ERROR_COUNT(2, result); + CHECK_EQ(toString(result.errors[0]), R"(Type 'b1' could not be converted into 'a1' +caused by: + Type '{| x: number, y: string |}' could not be converted into '{| x: number, y: number |}' +caused by: + Property 'y' is not compatible. Type 'string' could not be converted into 'number')"); + + CHECK_EQ(toString(result.errors[1]), R"(Type 'b2' could not be converted into 'a2' +caused by: + Type '{| __call: (a, b) -> () |}' could not be converted into '{| __call: (a) -> () |}' +caused by: + Property '__call' is not compatible. Type '(a, b) -> ()' could not be converted into '(a) -> ()')"); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.test.cpp b/tests/TypeInfer.test.cpp index 45381757..30d9130a 100644 --- a/tests/TypeInfer.test.cpp +++ b/tests/TypeInfer.test.cpp @@ -3926,8 +3926,6 @@ local b: number = 1 or a TEST_CASE_FIXTURE(Fixture, "no_lossy_function_type") { - ScopedFastFlag sffs2{"LuauGenericFunctions", true}; - CheckResult result = check(R"( --!strict local tbl = {} @@ -4493,10 +4491,6 @@ f(function(x) print(x) end) TEST_CASE_FIXTURE(Fixture, "infer_generic_function_function_argument") { - ScopedFastFlag luauGenericFunctions("LuauGenericFunctions", true); - ScopedFastFlag luauParseGenericFunctions("LuauParseGenericFunctions", true); - ScopedFastFlag luauRankNTypes("LuauRankNTypes", true); - CheckResult result = check(R"( local function sum(x: a, y: a, f: (a, a) -> a) return f(x, y) end return sum(2, 3, function(a, b) return a + b end) @@ -4525,10 +4519,6 @@ local r = foldl(a, {s=0,c=0}, function(a, b) return {s = a.s + b, c = a.c + 1} e TEST_CASE_FIXTURE(Fixture, "infer_generic_function_function_argument_overloaded") { - ScopedFastFlag luauGenericFunctions("LuauGenericFunctions", true); - ScopedFastFlag luauParseGenericFunctions("LuauParseGenericFunctions", true); - ScopedFastFlag luauRankNTypes("LuauRankNTypes", true); - CheckResult result = check(R"( local function g1(a: T, f: (T) -> T) return f(a) end local function g2(a: T, b: T, f: (T, T) -> T) return f(a, b) end @@ -4579,10 +4569,6 @@ local a: TableWithFunc = { x = 3, y = 4, f = function(a, b) return a + b end } TEST_CASE_FIXTURE(Fixture, "do_not_infer_generic_functions") { - ScopedFastFlag luauGenericFunctions("LuauGenericFunctions", true); - ScopedFastFlag luauParseGenericFunctions("LuauParseGenericFunctions", true); - ScopedFastFlag luauRankNTypes("LuauRankNTypes", true); - CheckResult result = check(R"( local function sum(x: a, y: a, f: (a, a) -> a) return f(x, y) end @@ -4600,8 +4586,6 @@ local c = sumrec(function(x, y, f) return f(x, y) end) -- type binders are not i TEST_CASE_FIXTURE(Fixture, "infer_return_value_type") { - ScopedFastFlag luauInferReturnAssertAssign("LuauInferReturnAssertAssign", true); - CheckResult result = check(R"( local function f(): {string|number} return {1, "b", 3} @@ -4625,8 +4609,6 @@ end TEST_CASE_FIXTURE(Fixture, "infer_type_assertion_value_type") { - ScopedFastFlag luauInferReturnAssertAssign("LuauInferReturnAssertAssign", true); - CheckResult result = check(R"( local function f() return {4, "b", 3} :: {string|number} @@ -4638,8 +4620,6 @@ end TEST_CASE_FIXTURE(Fixture, "infer_assignment_value_types") { - ScopedFastFlag luauInferReturnAssertAssign("LuauInferReturnAssertAssign", true); - CheckResult result = check(R"( local a: (number, number) -> number = function(a, b) return a - b end @@ -4655,8 +4635,6 @@ b, c = {2, "s"}, {"b", 4} TEST_CASE_FIXTURE(Fixture, "infer_assignment_value_types_mutable_lval") { - ScopedFastFlag luauInferReturnAssertAssign("LuauInferReturnAssertAssign", true); - CheckResult result = check(R"( local a = {} a.x = 2 @@ -4668,8 +4646,6 @@ a = setmetatable(a, { __call = function(x) end }) TEST_CASE_FIXTURE(Fixture, "refine_and_or") { - ScopedFastFlag sff{"LuauSlightlyMoreFlexibleBinaryPredicates", true}; - CheckResult result = check(R"( local t: {x: number?}? = {x = nil} local u = t and t.x or 5 @@ -4682,10 +4658,6 @@ TEST_CASE_FIXTURE(Fixture, "refine_and_or") TEST_CASE_FIXTURE(Fixture, "checked_prop_too_early") { - ScopedFastFlag sffs[] = { - {"LuauSlightlyMoreFlexibleBinaryPredicates", true}, - }; - CheckResult result = check(R"( local t: {x: number?}? = {x = nil} local u = t.x and t or 5 @@ -4698,10 +4670,6 @@ TEST_CASE_FIXTURE(Fixture, "checked_prop_too_early") TEST_CASE_FIXTURE(Fixture, "accidentally_checked_prop_in_opposite_branch") { - ScopedFastFlag sffs[] = { - {"LuauSlightlyMoreFlexibleBinaryPredicates", true}, - }; - CheckResult result = check(R"( local t: {x: number?}? = {x = nil} local u = t and t.x == 5 or t.x == 31337 @@ -4714,7 +4682,7 @@ TEST_CASE_FIXTURE(Fixture, "accidentally_checked_prop_in_opposite_branch") TEST_CASE_FIXTURE(Fixture, "substitution_with_bound_table") { - ScopedFastFlag luauFollowInTypeFunApply("LuauFollowInTypeFunApply", true); + ScopedFastFlag luauCloneCorrectlyBeforeMutatingTableType{"LuauCloneCorrectlyBeforeMutatingTableType", true}; CheckResult result = check(R"( type A = { x: number } diff --git a/tests/TypeInfer.tryUnify.test.cpp b/tests/TypeInfer.tryUnify.test.cpp index 1192a8ac..2d697fc9 100644 --- a/tests/TypeInfer.tryUnify.test.cpp +++ b/tests/TypeInfer.tryUnify.test.cpp @@ -178,9 +178,6 @@ TEST_CASE_FIXTURE(TryUnifyFixture, "unifying_variadic_pack_with_error_should_wor TEST_CASE_FIXTURE(TryUnifyFixture, "variadics_should_use_reversed_properly") { - ScopedFastFlag sffs2{"LuauGenericFunctions", true}; - ScopedFastFlag sffs4{"LuauParseGenericFunctions", true}; - CheckResult result = check(R"( --!strict local function f(...: T): ...T @@ -199,8 +196,6 @@ TEST_CASE_FIXTURE(TryUnifyFixture, "variadics_should_use_reversed_properly") TEST_CASE_FIXTURE(TryUnifyFixture, "cli_41095_concat_log_in_sealed_table_unification") { - ScopedFastFlag sffs2("LuauGenericFunctions", true); - CheckResult result = check(R"( --!strict table.insert() diff --git a/tests/TypeInfer.typePacks.cpp b/tests/TypeInfer.typePacks.cpp index 8dab2605..c6de0abf 100644 --- a/tests/TypeInfer.typePacks.cpp +++ b/tests/TypeInfer.typePacks.cpp @@ -296,7 +296,6 @@ end TEST_CASE_FIXTURE(Fixture, "type_alias_type_packs") { - ScopedFastFlag luauParseGenericFunctions("LuauParseGenericFunctions", true); ScopedFastFlag luauTypeAliasPacks("LuauTypeAliasPacks", true); ScopedFastFlag luauParseTypePackTypeParameters("LuauParseTypePackTypeParameters", true); @@ -361,7 +360,6 @@ local c: Packed TEST_CASE_FIXTURE(Fixture, "type_alias_type_packs_import") { - ScopedFastFlag luauParseGenericFunctions("LuauParseGenericFunctions", true); ScopedFastFlag luauTypeAliasPacks("LuauTypeAliasPacks", true); ScopedFastFlag luauParseTypePackTypeParameters("LuauParseTypePackTypeParameters", true); @@ -395,7 +393,6 @@ local d: { a: typeof(c) } TEST_CASE_FIXTURE(Fixture, "type_pack_type_parameters") { - ScopedFastFlag luauParseGenericFunctions("LuauParseGenericFunctions", true); ScopedFastFlag luauTypeAliasPacks("LuauTypeAliasPacks", true); ScopedFastFlag luauParseTypePackTypeParameters("LuauParseTypePackTypeParameters", true); @@ -434,7 +431,6 @@ type C = Import.Packed TEST_CASE_FIXTURE(Fixture, "type_alias_type_packs_nested") { - ScopedFastFlag luauParseGenericFunctions("LuauParseGenericFunctions", true); ScopedFastFlag luauTypeAliasPacks("LuauTypeAliasPacks", true); ScopedFastFlag luauParseTypePackTypeParameters("LuauParseTypePackTypeParameters", true); @@ -456,7 +452,6 @@ type Packed4 = (Packed3, T...) -> (Packed3, T...) TEST_CASE_FIXTURE(Fixture, "type_alias_type_pack_variadic") { - ScopedFastFlag luauParseGenericFunctions("LuauParseGenericFunctions", true); ScopedFastFlag luauTypeAliasPacks("LuauTypeAliasPacks", true); ScopedFastFlag luauParseTypePackTypeParameters("LuauParseTypePackTypeParameters", true); @@ -475,7 +470,6 @@ type E = X<(number, ...string)> TEST_CASE_FIXTURE(Fixture, "type_alias_type_pack_multi") { - ScopedFastFlag luauParseGenericFunctions("LuauParseGenericFunctions", true); ScopedFastFlag luauTypeAliasPacks("LuauTypeAliasPacks", true); ScopedFastFlag luauParseTypePackTypeParameters("LuauParseTypePackTypeParameters", true); @@ -507,7 +501,6 @@ type I = W TEST_CASE_FIXTURE(Fixture, "type_alias_type_pack_explicit") { - ScopedFastFlag luauParseGenericFunctions("LuauParseGenericFunctions", true); ScopedFastFlag luauTypeAliasPacks("LuauTypeAliasPacks", true); ScopedFastFlag luauParseTypePackTypeParameters("LuauParseTypePackTypeParameters", true); @@ -534,7 +527,6 @@ type F = X<(string, ...number)> TEST_CASE_FIXTURE(Fixture, "type_alias_type_pack_explicit_multi") { - ScopedFastFlag luauParseGenericFunctions("LuauParseGenericFunctions", true); ScopedFastFlag luauTypeAliasPacks("LuauTypeAliasPacks", true); ScopedFastFlag luauParseTypePackTypeParameters("LuauParseTypePackTypeParameters", true); @@ -557,10 +549,8 @@ type D = Y TEST_CASE_FIXTURE(Fixture, "type_alias_type_pack_explicit_multi_tostring") { - ScopedFastFlag luauParseGenericFunctions("LuauParseGenericFunctions", true); ScopedFastFlag luauTypeAliasPacks("LuauTypeAliasPacks", true); ScopedFastFlag luauParseTypePackTypeParameters("LuauParseTypePackTypeParameters", true); - ScopedFastFlag luauInstantiatedTypeParamRecursion("LuauInstantiatedTypeParamRecursion", true); // For correct toString block CheckResult result = check(R"( type Y = { f: (T...) -> (U...) } @@ -577,7 +567,6 @@ local b: Y<(), ()> TEST_CASE_FIXTURE(Fixture, "type_alias_backwards_compatible") { - ScopedFastFlag luauParseGenericFunctions("LuauParseGenericFunctions", true); ScopedFastFlag luauTypeAliasPacks("LuauTypeAliasPacks", true); ScopedFastFlag luauParseTypePackTypeParameters("LuauParseTypePackTypeParameters", true); @@ -599,7 +588,6 @@ type C = Y<(number), boolean> TEST_CASE_FIXTURE(Fixture, "type_alias_type_packs_errors") { - ScopedFastFlag luauParseGenericFunctions("LuauParseGenericFunctions", true); ScopedFastFlag luauTypeAliasPacks("LuauTypeAliasPacks", true); ScopedFastFlag luauParseTypePackTypeParameters("LuauParseTypePackTypeParameters", true); diff --git a/tests/TypeInfer.unionTypes.test.cpp b/tests/TypeInfer.unionTypes.test.cpp index 34c25a9f..9f29b642 100644 --- a/tests/TypeInfer.unionTypes.test.cpp +++ b/tests/TypeInfer.unionTypes.test.cpp @@ -400,8 +400,6 @@ local e = a.z TEST_CASE_FIXTURE(Fixture, "unify_sealed_table_union_check") { - ScopedFastFlag luauSealedTableUnifyOptionalFix("LuauSealedTableUnifyOptionalFix", true); - CheckResult result = check(R"( local x: { x: number } = { x = 3 } type A = number? @@ -426,4 +424,43 @@ y = x LUAU_REQUIRE_NO_ERRORS(result); } +TEST_CASE_FIXTURE(Fixture, "error_detailed_union_part") +{ + ScopedFastFlag luauExtendedTypeMismatchError{"LuauExtendedTypeMismatchError", true}; + + CheckResult result = check(R"( +type X = { x: number } +type Y = { y: number } +type Z = { z: number } + +type XYZ = X | Y | Z + +local a: XYZ +local b: { w: number } = a + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ(toString(result.errors[0]), R"(Type 'X | Y | Z' could not be converted into '{| w: number |}' +caused by: + Not all union options are compatible. Table type 'X' not compatible with type '{| w: number |}' because the former is missing field 'w')"); +} + +TEST_CASE_FIXTURE(Fixture, "error_detailed_union_all") +{ + ScopedFastFlag luauExtendedTypeMismatchError{"LuauExtendedTypeMismatchError", true}; + + CheckResult result = check(R"( +type X = { x: number } +type Y = { y: number } +type Z = { z: number } + +type XYZ = X | Y | Z + +local a: XYZ = { w = 4 } + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ(toString(result.errors[0]), R"(Type 'a' could not be converted into 'X | Y | Z'; none of the union options are compatible)"); +} + TEST_SUITE_END(); diff --git a/tests/TypeVar.test.cpp b/tests/TypeVar.test.cpp index 930c1a39..91efa818 100644 --- a/tests/TypeVar.test.cpp +++ b/tests/TypeVar.test.cpp @@ -11,8 +11,6 @@ using namespace Luau; -LUAU_FASTFLAG(LuauGenericFunctions); - TEST_SUITE_BEGIN("TypeVarTests"); TEST_CASE_FIXTURE(Fixture, "primitives_are_equal") diff --git a/tests/conformance/bitwise.lua b/tests/conformance/bitwise.lua index 6efa5960..13be3f94 100644 --- a/tests/conformance/bitwise.lua +++ b/tests/conformance/bitwise.lua @@ -113,6 +113,20 @@ assert(bit32.replace(0, -1, 4) == 2^4) assert(bit32.replace(-1, 0, 31) == 2^31 - 1) assert(bit32.replace(-1, 0, 1, 2) == 2^32 - 7) +-- testing countlz/countrc +assert(bit32.countlz(0) == 32) +assert(bit32.countlz(42) == 26) +assert(bit32.countlz(0xffffffff) == 0) +assert(bit32.countlz(0x80000000) == 0) +assert(bit32.countlz(0x7fffffff) == 1) + +assert(bit32.countrz(0) == 32) +assert(bit32.countrz(1) == 0) +assert(bit32.countrz(42) == 1) +assert(bit32.countrz(0x80000000) == 31) +assert(bit32.countrz(0x40000000) == 30) +assert(bit32.countrz(0x7fffffff) == 0) + --[[ This test verifies a fix in luauF_replace() where if the 4th parameter was not a number, but the first three are numbers, it will @@ -136,5 +150,7 @@ assert(bit32.bxor("1", 3) == 2) assert(bit32.bxor(1, "3") == 2) assert(bit32.btest(1, "3") == true) assert(bit32.btest("1", 3) == true) +assert(bit32.countlz("42") == 26) +assert(bit32.countrz("42") == 1) return('OK') From 8fe0dc0b6d553dc053a43f5ed6607c9c4d1a26c0 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Thu, 11 Nov 2021 18:23:34 -0800 Subject: [PATCH 049/399] Fix build --- VM/src/lbitlib.cpp | 1 + 1 file changed, 1 insertion(+) diff --git a/VM/src/lbitlib.cpp b/VM/src/lbitlib.cpp index c72fe674..907c43c4 100644 --- a/VM/src/lbitlib.cpp +++ b/VM/src/lbitlib.cpp @@ -2,6 +2,7 @@ // This code is based on Lua 5.x implementation licensed under MIT License; see lua_LICENSE.txt for details #include "lualib.h" +#include "lcommon.h" #include "lnumutils.h" LUAU_FASTFLAGVARIABLE(LuauBit32Count, false) From 863d3ff6ffa64398a6d13dc8c189bae30fefd557 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Thu, 11 Nov 2021 19:42:50 -0800 Subject: [PATCH 050/399] Attempt to work around non-sensical error --- Analysis/src/TypeInfer.cpp | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/Analysis/src/TypeInfer.cpp b/Analysis/src/TypeInfer.cpp index 8fad1af9..b27f3e17 100644 --- a/Analysis/src/TypeInfer.cpp +++ b/Analysis/src/TypeInfer.cpp @@ -5030,7 +5030,8 @@ void TypeChecker::resolve(const IsAPredicate& isaP, ErrorVec& errVec, Refinement return isaP.ty; } - return std::nullopt; + std::optional res = std::nullopt; + return res; }; std::optional ty = resolveLValue(refis, scope, isaP.lvalue); From 3c3541aba84d9209b6098a5c6ae01727ab11ec32 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Thu, 11 Nov 2021 20:36:53 -0800 Subject: [PATCH 051/399] Add a comment --- Analysis/src/TypeInfer.cpp | 1 + 1 file changed, 1 insertion(+) diff --git a/Analysis/src/TypeInfer.cpp b/Analysis/src/TypeInfer.cpp index b27f3e17..a6696efd 100644 --- a/Analysis/src/TypeInfer.cpp +++ b/Analysis/src/TypeInfer.cpp @@ -5030,6 +5030,7 @@ void TypeChecker::resolve(const IsAPredicate& isaP, ErrorVec& errVec, Refinement return isaP.ty; } + // local variable works around an odd gcc 9.3 warning: may be used uninitialized std::optional res = std::nullopt; return res; }; From d47b2f1dfe9531dbcf2cea1f37e7cc7280896b04 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Fri, 12 Nov 2021 06:27:34 -0800 Subject: [PATCH 052/399] Sync to upstream/release/504 (#200) - Type mismatch errors now show detailed information for compound types, highlighting the mismatching component - Fix string.pack bug on ARM when packing negative numbers using unsigned formats - Implement bit32.countlz/countrz (RFC RFC: bit32.countlz/countrz #89) - Minor compiler throughput optimization (~2% faster compilation) - Improve transpiler behavior for edge cases and better test coverage (not exposed through CLI at the moment) - Improve error recovery when parsing invalid assignments - Build fixes for fuzzing targets --- Analysis/include/Luau/Error.h | 13 +- Analysis/include/Luau/FileResolver.h | 2 +- Analysis/include/Luau/Transpiler.h | 3 +- Analysis/include/Luau/TypeInfer.h | 8 +- Analysis/include/Luau/TypePack.h | 20 +- Analysis/include/Luau/TypeVar.h | 45 +- Analysis/include/Luau/Unifiable.h | 3 - Analysis/include/Luau/Unifier.h | 2 +- Analysis/src/Autocomplete.cpp | 42 +- Analysis/src/BuiltinDefinitions.cpp | 244 +-------- Analysis/src/EmbeddedBuiltinDefinitions.cpp | 22 +- Analysis/src/Error.cpp | 291 ++++++---- Analysis/src/Frontend.cpp | 9 +- Analysis/src/Linter.cpp | 10 +- Analysis/src/Module.cpp | 17 +- Analysis/src/Predicate.cpp | 4 - Analysis/src/RequireTracer.cpp | 8 +- Analysis/src/Substitution.cpp | 13 +- Analysis/src/ToString.cpp | 114 +--- Analysis/src/Transpiler.cpp | 117 +++- Analysis/src/TypeAttach.cpp | 41 +- Analysis/src/TypeInfer.cpp | 578 ++++++-------------- Analysis/src/TypePack.cpp | 6 +- Analysis/src/TypeVar.cpp | 58 +- Analysis/src/Unifiable.cpp | 10 - Analysis/src/Unifier.cpp | 345 ++++++------ Ast/include/Luau/Parser.h | 1 - Ast/src/Parser.cpp | 46 +- CLI/Repl.cpp | 18 +- Compiler/include/Luau/Bytecode.h | 4 + Compiler/include/Luau/Compiler.h | 3 +- Compiler/src/Compiler.cpp | 12 +- Makefile | 5 +- VM/include/lua.h | 2 + VM/src/lapi.cpp | 10 + VM/src/lbitlib.cpp | 43 ++ VM/src/lbuiltins.cpp | 49 ++ VM/src/lgc.cpp | 164 +----- VM/src/lstrlib.cpp | 20 +- VM/src/ltablib.cpp | 8 - bench/tests/chess.lua | 80 +-- fuzz/luau.proto | 7 + fuzz/proto.cpp | 7 + fuzz/protoprint.cpp | 10 + tests/AstQuery.test.cpp | 1 - tests/Autocomplete.test.cpp | 3 - tests/Conformance.test.cpp | 6 +- tests/Linter.test.cpp | 6 - tests/Parser.test.cpp | 43 +- tests/Predicate.test.cpp | 6 - tests/ToString.test.cpp | 9 - tests/Transpiler.test.cpp | 252 ++++++++- tests/TypeInfer.aliases.test.cpp | 3 - tests/TypeInfer.builtins.test.cpp | 8 - tests/TypeInfer.classes.test.cpp | 25 +- tests/TypeInfer.definitions.test.cpp | 3 - tests/TypeInfer.generics.test.cpp | 89 --- tests/TypeInfer.intersectionTypes.test.cpp | 39 ++ tests/TypeInfer.provisional.test.cpp | 7 +- tests/TypeInfer.refinements.test.cpp | 44 +- tests/TypeInfer.tables.test.cpp | 72 +++ tests/TypeInfer.test.cpp | 34 +- tests/TypeInfer.tryUnify.test.cpp | 5 - tests/TypeInfer.typePacks.cpp | 12 - tests/TypeInfer.unionTypes.test.cpp | 41 +- tests/TypeVar.test.cpp | 2 - tests/conformance/bitwise.lua | 16 + 67 files changed, 1433 insertions(+), 1807 deletions(-) diff --git a/Analysis/include/Luau/Error.h b/Analysis/include/Luau/Error.h index ac6f13e9..9ee75004 100644 --- a/Analysis/include/Luau/Error.h +++ b/Analysis/include/Luau/Error.h @@ -8,11 +8,20 @@ namespace Luau { +struct TypeError; struct TypeMismatch { - TypeId wantedType; - TypeId givenType; + TypeMismatch() = default; + TypeMismatch(TypeId wantedType, TypeId givenType); + TypeMismatch(TypeId wantedType, TypeId givenType, std::string reason); + TypeMismatch(TypeId wantedType, TypeId givenType, std::string reason, TypeError error); + + TypeId wantedType = nullptr; + TypeId givenType = nullptr; + + std::string reason; + std::shared_ptr error; bool operator==(const TypeMismatch& rhs) const; }; diff --git a/Analysis/include/Luau/FileResolver.h b/Analysis/include/Luau/FileResolver.h index a05ec5e9..9b74fc12 100644 --- a/Analysis/include/Luau/FileResolver.h +++ b/Analysis/include/Luau/FileResolver.h @@ -53,7 +53,7 @@ struct FileResolver } // DEPRECATED APIS - // These are going to be removed with LuauNewRequireTracer + // These are going to be removed with LuauNewRequireTrace2 virtual bool moduleExists(const ModuleName& name) const = 0; virtual std::optional fromAstFragment(AstExpr* expr) const = 0; virtual ModuleName concat(const ModuleName& lhs, std::string_view rhs) const = 0; diff --git a/Analysis/include/Luau/Transpiler.h b/Analysis/include/Luau/Transpiler.h index 817459fe..df01008c 100644 --- a/Analysis/include/Luau/Transpiler.h +++ b/Analysis/include/Luau/Transpiler.h @@ -18,6 +18,7 @@ struct TranspileResult std::string parseError; // Nonempty if the transpile failed }; +std::string toString(AstNode* node); void dump(AstNode* node); // Never fails on a well-formed AST @@ -25,6 +26,6 @@ std::string transpile(AstStatBlock& ast); std::string transpileWithTypes(AstStatBlock& block); // Only fails when parsing fails -TranspileResult transpile(std::string_view source, ParseOptions options = ParseOptions{}); +TranspileResult transpile(std::string_view source, ParseOptions options = ParseOptions{}, bool withTypes = false); } // namespace Luau diff --git a/Analysis/include/Luau/TypeInfer.h b/Analysis/include/Luau/TypeInfer.h index 9d62fef0..306ac77d 100644 --- a/Analysis/include/Luau/TypeInfer.h +++ b/Analysis/include/Luau/TypeInfer.h @@ -263,8 +263,6 @@ public: * */ TypeId instantiate(const ScopePtr& scope, TypeId ty, Location location); - // Removed by FFlag::LuauRankNTypes - TypePackId DEPRECATED_instantiate(const ScopePtr& scope, TypePackId ty, Location location); // Replace any free types or type packs by `any`. // This is used when exporting types from modules, to make sure free types don't leak. @@ -298,8 +296,6 @@ private: // Produce a new free type var. TypeId freshType(const ScopePtr& scope); TypeId freshType(TypeLevel level); - TypeId DEPRECATED_freshType(const ScopePtr& scope, bool canBeGeneric = false); - TypeId DEPRECATED_freshType(TypeLevel level, bool canBeGeneric = false); // Returns nullopt if the predicate filters down the TypeId to 0 options. std::optional filterMap(TypeId type, TypeIdPredicate predicate); @@ -326,10 +322,8 @@ private: TypePackId addTypePack(std::initializer_list&& ty); TypePackId freshTypePack(const ScopePtr& scope); TypePackId freshTypePack(TypeLevel level); - TypePackId DEPRECATED_freshTypePack(const ScopePtr& scope, bool canBeGeneric = false); - TypePackId DEPRECATED_freshTypePack(TypeLevel level, bool canBeGeneric = false); - TypeId resolveType(const ScopePtr& scope, const AstType& annotation, bool canBeGeneric = false); + TypeId resolveType(const ScopePtr& scope, const AstType& annotation); TypePackId resolveTypePack(const ScopePtr& scope, const AstTypeList& types); TypePackId resolveTypePack(const ScopePtr& scope, const AstTypePack& annotation); TypeId instantiateTypeFun(const ScopePtr& scope, const TypeFun& tf, const std::vector& typeParams, diff --git a/Analysis/include/Luau/TypePack.h b/Analysis/include/Luau/TypePack.h index d987d46c..e72808da 100644 --- a/Analysis/include/Luau/TypePack.h +++ b/Analysis/include/Luau/TypePack.h @@ -8,8 +8,6 @@ #include #include -LUAU_FASTFLAG(LuauAddMissingFollow) - namespace Luau { @@ -128,13 +126,10 @@ TypePack* asMutable(const TypePack* tp); template const T* get(TypePackId tp) { - if (FFlag::LuauAddMissingFollow) - { - LUAU_ASSERT(tp); + LUAU_ASSERT(tp); - if constexpr (!std::is_same_v) - LUAU_ASSERT(get_if(&tp->ty) == nullptr); - } + if constexpr (!std::is_same_v) + LUAU_ASSERT(get_if(&tp->ty) == nullptr); return get_if(&(tp->ty)); } @@ -142,13 +137,10 @@ const T* get(TypePackId tp) template T* getMutable(TypePackId tp) { - if (FFlag::LuauAddMissingFollow) - { - LUAU_ASSERT(tp); + LUAU_ASSERT(tp); - if constexpr (!std::is_same_v) - LUAU_ASSERT(get_if(&tp->ty) == nullptr); - } + if constexpr (!std::is_same_v) + LUAU_ASSERT(get_if(&tp->ty) == nullptr); return get_if(&(asMutable(tp)->ty)); } diff --git a/Analysis/include/Luau/TypeVar.h b/Analysis/include/Luau/TypeVar.h index 9611e881..6bd7932d 100644 --- a/Analysis/include/Luau/TypeVar.h +++ b/Analysis/include/Luau/TypeVar.h @@ -18,7 +18,6 @@ LUAU_FASTINT(LuauTableTypeMaximumStringifierLength) LUAU_FASTINT(LuauTypeMaximumStringifierLength) -LUAU_FASTFLAG(LuauAddMissingFollow) namespace Luau { @@ -413,13 +412,17 @@ bool maybeGeneric(const TypeId ty); struct SingletonTypes { - const TypeId nilType = &nilType_; - const TypeId numberType = &numberType_; - const TypeId stringType = &stringType_; - const TypeId booleanType = &booleanType_; - const TypeId threadType = &threadType_; - const TypeId anyType = &anyType_; - const TypeId errorType = &errorType_; + const TypeId nilType; + const TypeId numberType; + const TypeId stringType; + const TypeId booleanType; + const TypeId threadType; + const TypeId anyType; + const TypeId errorType; + const TypeId optionalNumberType; + + const TypePackId anyTypePack; + const TypePackId errorTypePack; SingletonTypes(); SingletonTypes(const SingletonTypes&) = delete; @@ -427,14 +430,6 @@ struct SingletonTypes private: std::unique_ptr arena; - TypeVar nilType_; - TypeVar numberType_; - TypeVar stringType_; - TypeVar booleanType_; - TypeVar threadType_; - TypeVar anyType_; - TypeVar errorType_; - TypeId makeStringMetatable(); }; @@ -472,13 +467,10 @@ TypeVar* asMutable(TypeId ty); template const T* get(TypeId tv) { - if (FFlag::LuauAddMissingFollow) - { - LUAU_ASSERT(tv); + LUAU_ASSERT(tv); - if constexpr (!std::is_same_v) - LUAU_ASSERT(get_if(&tv->ty) == nullptr); - } + if constexpr (!std::is_same_v) + LUAU_ASSERT(get_if(&tv->ty) == nullptr); return get_if(&tv->ty); } @@ -486,13 +478,10 @@ const T* get(TypeId tv) template T* getMutable(TypeId tv) { - if (FFlag::LuauAddMissingFollow) - { - LUAU_ASSERT(tv); + LUAU_ASSERT(tv); - if constexpr (!std::is_same_v) - LUAU_ASSERT(get_if(&tv->ty) == nullptr); - } + if constexpr (!std::is_same_v) + LUAU_ASSERT(get_if(&tv->ty) == nullptr); return get_if(&asMutable(tv)->ty); } diff --git a/Analysis/include/Luau/Unifiable.h b/Analysis/include/Luau/Unifiable.h index 10dbf333..c2e07e46 100644 --- a/Analysis/include/Luau/Unifiable.h +++ b/Analysis/include/Luau/Unifiable.h @@ -63,12 +63,9 @@ using Name = std::string; struct Free { explicit Free(TypeLevel level); - Free(TypeLevel level, bool DEPRECATED_canBeGeneric); int index; TypeLevel level; - // Removed by FFlag::LuauRankNTypes - bool DEPRECATED_canBeGeneric = false; // True if this free type variable is part of a mutually // recursive type alias whose definitions haven't been // resolved yet. diff --git a/Analysis/include/Luau/Unifier.h b/Analysis/include/Luau/Unifier.h index 56632e33..be0aadd0 100644 --- a/Analysis/include/Luau/Unifier.h +++ b/Analysis/include/Luau/Unifier.h @@ -87,7 +87,6 @@ private: void tryUnifyWithAny(TypePackId any, TypePackId ty); std::optional findTablePropertyRespectingMeta(TypeId lhsType, Name name); - std::optional findMetatableEntry(TypeId type, std::string entry); public: // Report an "infinite type error" if the type "needle" already occurs within "haystack" @@ -102,6 +101,7 @@ private: bool isNonstrictMode() const; void checkChildUnifierTypeMismatch(const ErrorVec& innerErrors, TypeId wantedType, TypeId givenType); + void checkChildUnifierTypeMismatch(const ErrorVec& innerErrors, const std::string& prop, TypeId wantedType, TypeId givenType); [[noreturn]] void ice(const std::string& message, const Location& location); [[noreturn]] void ice(const std::string& message); diff --git a/Analysis/src/Autocomplete.cpp b/Analysis/src/Autocomplete.cpp index 09476f78..1c94bb68 100644 --- a/Analysis/src/Autocomplete.cpp +++ b/Analysis/src/Autocomplete.cpp @@ -12,7 +12,6 @@ #include #include -LUAU_FASTFLAG(LuauSecondTypecheckKnowsTheDataModel) LUAU_FASTFLAGVARIABLE(ElseElseIfCompletionImprovements, false); LUAU_FASTFLAG(LuauIfElseExpressionAnalysisSupport) @@ -369,20 +368,10 @@ static void autocompleteProps(const Module& module, TypeArena* typeArena, TypeId while (iter != endIter) { - if (FFlag::LuauAddMissingFollow) - { - if (isNil(*iter)) - ++iter; - else - break; - } + if (isNil(*iter)) + ++iter; else - { - if (auto primTy = Luau::get(*iter); primTy && primTy->type == PrimitiveTypeVar::NilType) - ++iter; - else - break; - } + break; } if (iter == endIter) @@ -397,21 +386,10 @@ static void autocompleteProps(const Module& module, TypeArena* typeArena, TypeId AutocompleteEntryMap inner; std::unordered_set innerSeen = seen; - if (FFlag::LuauAddMissingFollow) + if (isNil(*iter)) { - if (isNil(*iter)) - { - ++iter; - continue; - } - } - else - { - if (auto innerPrimTy = Luau::get(*iter); innerPrimTy && innerPrimTy->type == PrimitiveTypeVar::NilType) - { - ++iter; - continue; - } + ++iter; + continue; } autocompleteProps(module, typeArena, *iter, indexType, nodes, inner, innerSeen); @@ -1519,10 +1497,10 @@ AutocompleteResult autocomplete(Frontend& frontend, const ModuleName& moduleName return {}; TypeChecker& typeChecker = - (frontend.options.typecheckTwice && FFlag::LuauSecondTypecheckKnowsTheDataModel ? frontend.typeCheckerForAutocomplete : frontend.typeChecker); + (frontend.options.typecheckTwice ? frontend.typeCheckerForAutocomplete : frontend.typeChecker); ModulePtr module = - (frontend.options.typecheckTwice && FFlag::LuauSecondTypecheckKnowsTheDataModel ? frontend.moduleResolverForAutocomplete.getModule(moduleName) - : frontend.moduleResolver.getModule(moduleName)); + (frontend.options.typecheckTwice ? frontend.moduleResolverForAutocomplete.getModule(moduleName) + : frontend.moduleResolver.getModule(moduleName)); if (!module) return {}; @@ -1550,7 +1528,7 @@ OwningAutocompleteResult autocompleteSource(Frontend& frontend, std::string_view sourceModule->commentLocations = std::move(result.commentLocations); TypeChecker& typeChecker = - (frontend.options.typecheckTwice && FFlag::LuauSecondTypecheckKnowsTheDataModel ? frontend.typeCheckerForAutocomplete : frontend.typeChecker); + (frontend.options.typecheckTwice ? frontend.typeCheckerForAutocomplete : frontend.typeChecker); ModulePtr module = typeChecker.check(*sourceModule, Mode::Strict); diff --git a/Analysis/src/BuiltinDefinitions.cpp b/Analysis/src/BuiltinDefinitions.cpp index f6f2363c..62a06a3c 100644 --- a/Analysis/src/BuiltinDefinitions.cpp +++ b/Analysis/src/BuiltinDefinitions.cpp @@ -8,10 +8,7 @@ #include -LUAU_FASTFLAG(LuauParseGenericFunctions) -LUAU_FASTFLAG(LuauGenericFunctions) -LUAU_FASTFLAG(LuauRankNTypes) -LUAU_FASTFLAG(LuauNewRequireTrace) +LUAU_FASTFLAG(LuauNewRequireTrace2) /** FIXME: Many of these type definitions are not quite completely accurate. * @@ -185,25 +182,11 @@ void registerBuiltinTypes(TypeChecker& typeChecker) TypeId numberType = typeChecker.numberType; TypeId booleanType = typeChecker.booleanType; TypeId nilType = typeChecker.nilType; - TypeId stringType = typeChecker.stringType; - TypeId threadType = typeChecker.threadType; - TypeId anyType = typeChecker.anyType; TypeArena& arena = typeChecker.globalTypes; - TypeId optionalNumber = makeOption(typeChecker, arena, numberType); - TypeId optionalString = makeOption(typeChecker, arena, stringType); - TypeId optionalBoolean = makeOption(typeChecker, arena, booleanType); - - TypeId stringOrNumber = makeUnion(arena, {stringType, numberType}); - - TypePackId emptyPack = arena.addTypePack({}); TypePackId oneNumberPack = arena.addTypePack({numberType}); - TypePackId oneStringPack = arena.addTypePack({stringType}); TypePackId oneBooleanPack = arena.addTypePack({booleanType}); - TypePackId oneAnyPack = arena.addTypePack({anyType}); - - TypePackId anyTypePack = typeChecker.anyTypePack; TypePackId numberVariadicList = arena.addTypePack(TypePackVar{VariadicTypePack{numberType}}); TypePackId listOfAtLeastOneNumber = arena.addTypePack(TypePack{{numberType}, numberVariadicList}); @@ -215,8 +198,6 @@ void registerBuiltinTypes(TypeChecker& typeChecker) TypeId listOfAtLeastZeroNumbersToNumberType = arena.addType(FunctionTypeVar{numberVariadicList, oneNumberPack}); - TypeId stringToAnyMap = arena.addType(TableTypeVar{{}, TableIndexer(stringType, anyType), typeChecker.globalScope->level}); - LoadDefinitionFileResult loadResult = Luau::loadDefinitionFile(typeChecker, typeChecker.globalScope, getBuiltinDefinitionSource(), "@luau"); LUAU_ASSERT(loadResult.success); @@ -236,8 +217,6 @@ void registerBuiltinTypes(TypeChecker& typeChecker) ttv->props["btest"] = makeProperty(arena.addType(FunctionTypeVar{listOfAtLeastOneNumber, oneBooleanPack}), "@luau/global/bit32.btest"); } - TypeId anyFunction = arena.addType(FunctionTypeVar{anyTypePack, anyTypePack}); - TypeId genericK = arena.addType(GenericTypeVar{"K"}); TypeId genericV = arena.addType(GenericTypeVar{"V"}); TypeId mapOfKtoV = arena.addType(TableTypeVar{{}, TableIndexer(genericK, genericV), typeChecker.globalScope->level}); @@ -252,222 +231,6 @@ void registerBuiltinTypes(TypeChecker& typeChecker) addGlobalBinding(typeChecker, "string", it->second.type, "@luau"); - if (!FFlag::LuauParseGenericFunctions || !FFlag::LuauGenericFunctions) - { - TableTypeVar::Props debugLib{ - {"info", {makeIntersection(arena, - { - arena.addType(FunctionTypeVar{arena.addTypePack({typeChecker.threadType, numberType, stringType}), anyTypePack}), - arena.addType(FunctionTypeVar{arena.addTypePack({numberType, stringType}), anyTypePack}), - arena.addType(FunctionTypeVar{arena.addTypePack({anyFunction, stringType}), anyTypePack}), - })}}, - {"traceback", {makeIntersection(arena, - { - makeFunction(arena, std::nullopt, {optionalString, optionalNumber}, {stringType}), - makeFunction(arena, std::nullopt, {typeChecker.threadType, optionalString, optionalNumber}, {stringType}), - })}}, - }; - - assignPropDocumentationSymbols(debugLib, "@luau/global/debug"); - addGlobalBinding(typeChecker, "debug", - arena.addType(TableTypeVar{debugLib, std::nullopt, typeChecker.globalScope->level, Luau::TableState::Sealed}), "@luau"); - - TableTypeVar::Props utf8Lib = { - {"char", {arena.addType(FunctionTypeVar{listOfAtLeastOneNumber, oneStringPack})}}, // FIXME - {"charpattern", {stringType}}, - {"codes", {makeFunction(arena, std::nullopt, {stringType}, - {makeFunction(arena, std::nullopt, {stringType, numberType}, {numberType, numberType}), stringType, numberType})}}, - {"codepoint", - {arena.addType(FunctionTypeVar{arena.addTypePack({stringType, optionalNumber, optionalNumber}), listOfAtLeastOneNumber})}}, // FIXME - {"len", {makeFunction(arena, std::nullopt, {stringType, optionalNumber, optionalNumber}, {optionalNumber, numberType})}}, - {"offset", {makeFunction(arena, std::nullopt, {stringType, optionalNumber, optionalNumber}, {numberType})}}, - {"nfdnormalize", {makeFunction(arena, std::nullopt, {stringType}, {stringType})}}, - {"graphemes", {makeFunction(arena, std::nullopt, {stringType, optionalNumber, optionalNumber}, - {makeFunction(arena, std::nullopt, {}, {numberType, numberType})})}}, - {"nfcnormalize", {makeFunction(arena, std::nullopt, {stringType}, {stringType})}}, - }; - - assignPropDocumentationSymbols(utf8Lib, "@luau/global/utf8"); - addGlobalBinding( - typeChecker, "utf8", arena.addType(TableTypeVar{utf8Lib, std::nullopt, typeChecker.globalScope->level, TableState::Sealed}), "@luau"); - - TypeId optionalV = makeOption(typeChecker, arena, genericV); - - TypeId arrayOfV = arena.addType(TableTypeVar{{}, TableIndexer(numberType, genericV), typeChecker.globalScope->level}); - - TypePackId unpackArgsPack = arena.addTypePack(TypePack{{arrayOfV, optionalNumber, optionalNumber}}); - TypePackId unpackReturnPack = arena.addTypePack(TypePack{{}, anyTypePack}); - TypeId unpackFunc = arena.addType(FunctionTypeVar{{genericV}, {}, unpackArgsPack, unpackReturnPack}); - - TypeId packResult = arena.addType(TableTypeVar{ - TableTypeVar::Props{{"n", {numberType}}}, TableIndexer{numberType, numberType}, typeChecker.globalScope->level, TableState::Sealed}); - TypePackId packArgsPack = arena.addTypePack(TypePack{{}, anyTypePack}); - TypePackId packReturnPack = arena.addTypePack(TypePack{{packResult}}); - - TypeId comparator = makeFunction(arena, std::nullopt, {genericV, genericV}, {booleanType}); - TypeId optionalComparator = makeOption(typeChecker, arena, comparator); - - TypeId packFn = arena.addType(FunctionTypeVar(packArgsPack, packReturnPack)); - - TableTypeVar::Props tableLib = { - {"concat", {makeFunction(arena, std::nullopt, {genericV}, {}, {arrayOfV, optionalString, optionalNumber, optionalNumber}, {stringType})}}, - {"insert", {makeIntersection(arena, {makeFunction(arena, std::nullopt, {genericV}, {}, {arrayOfV, genericV}, {}), - makeFunction(arena, std::nullopt, {genericV}, {}, {arrayOfV, numberType, genericV}, {})})}}, - {"maxn", {makeFunction(arena, std::nullopt, {genericV}, {}, {arrayOfV}, {numberType})}}, - {"remove", {makeFunction(arena, std::nullopt, {genericV}, {}, {arrayOfV, optionalNumber}, {optionalV})}}, - {"sort", {makeFunction(arena, std::nullopt, {genericV}, {}, {arrayOfV, optionalComparator}, {})}}, - {"create", {makeFunction(arena, std::nullopt, {genericV}, {}, {numberType, optionalV}, {arrayOfV})}}, - {"find", {makeFunction(arena, std::nullopt, {genericV}, {}, {arrayOfV, genericV, optionalNumber}, {optionalNumber})}}, - - {"unpack", {unpackFunc}}, // FIXME - {"pack", {packFn}}, - - // Lua 5.0 compat - {"getn", {makeFunction(arena, std::nullopt, {genericV}, {}, {arrayOfV}, {numberType})}}, - {"foreach", {makeFunction(arena, std::nullopt, {genericK, genericV}, {}, - {mapOfKtoV, makeFunction(arena, std::nullopt, {genericK, genericV}, {})}, {})}}, - {"foreachi", {makeFunction(arena, std::nullopt, {genericV}, {}, {arrayOfV, makeFunction(arena, std::nullopt, {genericV}, {})}, {})}}, - - // backported from Lua 5.3 - {"move", {makeFunction(arena, std::nullopt, {genericV}, {}, {arrayOfV, numberType, numberType, numberType, arrayOfV}, {})}}, - - // added in Luau (borrowed from LuaJIT) - {"clear", {makeFunction(arena, std::nullopt, {genericK, genericV}, {}, {mapOfKtoV}, {})}}, - - {"freeze", {makeFunction(arena, std::nullopt, {genericK, genericV}, {}, {mapOfKtoV}, {mapOfKtoV})}}, - {"isfrozen", {makeFunction(arena, std::nullopt, {genericK, genericV}, {}, {mapOfKtoV}, {booleanType})}}, - }; - - assignPropDocumentationSymbols(tableLib, "@luau/global/table"); - addGlobalBinding( - typeChecker, "table", arena.addType(TableTypeVar{tableLib, std::nullopt, typeChecker.globalScope->level, TableState::Sealed}), "@luau"); - - TableTypeVar::Props coroutineLib = { - {"create", {makeFunction(arena, std::nullopt, {anyFunction}, {threadType})}}, - {"resume", {arena.addType(FunctionTypeVar{arena.addTypePack(TypePack{{threadType}, anyTypePack}), anyTypePack})}}, - {"running", {makeFunction(arena, std::nullopt, {}, {threadType})}}, - {"status", {makeFunction(arena, std::nullopt, {threadType}, {stringType})}}, - {"wrap", {makeFunction( - arena, std::nullopt, {anyFunction}, {anyType})}}, // FIXME this technically returns a function, but we can't represent this - // atm since it can be called with different arg types at different times - {"yield", {arena.addType(FunctionTypeVar{anyTypePack, anyTypePack})}}, - {"isyieldable", {makeFunction(arena, std::nullopt, {}, {booleanType})}}, - }; - - assignPropDocumentationSymbols(coroutineLib, "@luau/global/coroutine"); - addGlobalBinding(typeChecker, "coroutine", - arena.addType(TableTypeVar{coroutineLib, std::nullopt, typeChecker.globalScope->level, TableState::Sealed}), "@luau"); - - TypeId genericT = arena.addType(GenericTypeVar{"T"}); - TypeId genericR = arena.addType(GenericTypeVar{"R"}); - - // assert returns all arguments - TypePackId assertArgs = arena.addTypePack({genericT, optionalString}); - TypePackId assertRets = arena.addTypePack({genericT}); - addGlobalBinding(typeChecker, "assert", arena.addType(FunctionTypeVar{assertArgs, assertRets}), "@luau"); - - addGlobalBinding(typeChecker, "print", arena.addType(FunctionTypeVar{anyTypePack, emptyPack}), "@luau"); - - addGlobalBinding(typeChecker, "type", makeFunction(arena, std::nullopt, {genericT}, {}, {genericT}, {stringType}), "@luau"); - addGlobalBinding(typeChecker, "typeof", makeFunction(arena, std::nullopt, {genericT}, {}, {genericT}, {stringType}), "@luau"); - - addGlobalBinding(typeChecker, "error", makeFunction(arena, std::nullopt, {genericT}, {}, {genericT, optionalNumber}, {}), "@luau"); - - addGlobalBinding(typeChecker, "tostring", makeFunction(arena, std::nullopt, {genericT}, {}, {genericT}, {stringType}), "@luau"); - addGlobalBinding( - typeChecker, "tonumber", makeFunction(arena, std::nullopt, {genericT}, {}, {genericT, optionalNumber}, {numberType}), "@luau"); - - addGlobalBinding( - typeChecker, "rawequal", makeFunction(arena, std::nullopt, {genericT, genericR}, {}, {genericT, genericR}, {booleanType}), "@luau"); - addGlobalBinding( - typeChecker, "rawget", makeFunction(arena, std::nullopt, {genericK, genericV}, {}, {mapOfKtoV, genericK}, {genericV}), "@luau"); - addGlobalBinding(typeChecker, "rawset", - makeFunction(arena, std::nullopt, {genericK, genericV}, {}, {mapOfKtoV, genericK, genericV}, {mapOfKtoV}), "@luau"); - - TypePackId genericTPack = arena.addTypePack({genericT}); - TypePackId genericRPack = arena.addTypePack({genericR}); - TypeId genericArgsToReturnFunction = arena.addType( - FunctionTypeVar{{genericT, genericR}, {}, arena.addTypePack(TypePack{{}, genericTPack}), arena.addTypePack(TypePack{{}, genericRPack})}); - - TypeId setfenvArgType = makeUnion(arena, {numberType, genericArgsToReturnFunction}); - TypeId setfenvReturnType = makeOption(typeChecker, arena, genericArgsToReturnFunction); - addGlobalBinding(typeChecker, "setfenv", makeFunction(arena, std::nullopt, {setfenvArgType, stringToAnyMap}, {setfenvReturnType}), "@luau"); - - TypePackId ipairsArgsTypePack = arena.addTypePack({arrayOfV}); - - TypeId ipairsNextFunctionType = arena.addType( - FunctionTypeVar{{genericK, genericV}, {}, arena.addTypePack({arrayOfV, numberType}), arena.addTypePack({numberType, genericV})}); - - // ipairs returns 'next, Array, 0' so we would need type-level primitives and change to - // again, we have a direct reference to 'next' because ipairs returns it - // ipairs(t: Array) -> ((Array) -> (number, V), Array, 0) - TypePackId ipairsReturnTypePack = arena.addTypePack(TypePack{{ipairsNextFunctionType, arrayOfV, numberType}}); - - // ipairs(t: Array) -> ((Array) -> (number, V), Array, number) - addGlobalBinding(typeChecker, "ipairs", arena.addType(FunctionTypeVar{{genericV}, {}, ipairsArgsTypePack, ipairsReturnTypePack}), "@luau"); - - TypePackId pcallArg0FnArgs = arena.addTypePack(TypePackVar{GenericTypeVar{"A"}}); - TypePackId pcallArg0FnRet = arena.addTypePack(TypePackVar{GenericTypeVar{"R"}}); - TypeId pcallArg0 = arena.addType(FunctionTypeVar{pcallArg0FnArgs, pcallArg0FnRet}); - TypePackId pcallArgsTypePack = arena.addTypePack(TypePack{{pcallArg0}, pcallArg0FnArgs}); - - TypePackId pcallReturnTypePack = arena.addTypePack(TypePack{{booleanType}, pcallArg0FnRet}); - - // pcall(f: (A...) -> R..., args: A...) -> boolean, R... - addGlobalBinding(typeChecker, "pcall", - arena.addType(FunctionTypeVar{{}, {pcallArg0FnArgs, pcallArg0FnRet}, pcallArgsTypePack, pcallReturnTypePack}), "@luau"); - - // errors thrown by the function 'f' are propagated onto the function 'err' that accepts it. - // and either 'f' or 'err' are valid results of this xpcall - // if 'err' did throw an error, then it returns: false, "error in error handling" - // TODO: the above is not represented (nor representable) in the type annotation below. - // - // The real type of xpcall is as such: (f: (A...) -> R1..., err: (E) -> R2..., A...) -> (true, R1...) | (false, - // R2...) - TypePackId genericAPack = arena.addTypePack(TypePackVar{GenericTypeVar{"A"}}); - TypePackId genericR1Pack = arena.addTypePack(TypePackVar{GenericTypeVar{"R1"}}); - TypePackId genericR2Pack = arena.addTypePack(TypePackVar{GenericTypeVar{"R2"}}); - - TypeId genericE = arena.addType(GenericTypeVar{"E"}); - - TypeId xpcallFArg = arena.addType(FunctionTypeVar{genericAPack, genericR1Pack}); - TypeId xpcallErrArg = arena.addType(FunctionTypeVar{arena.addTypePack({genericE}), genericR2Pack}); - - TypePackId xpcallArgsPack = arena.addTypePack({{xpcallFArg, xpcallErrArg}, genericAPack}); - TypePackId xpcallRetPack = arena.addTypePack({{booleanType}, genericR1Pack}); // FIXME - - addGlobalBinding(typeChecker, "xpcall", - arena.addType(FunctionTypeVar{{genericE}, {genericAPack, genericR1Pack, genericR2Pack}, xpcallArgsPack, xpcallRetPack}), "@luau"); - - addGlobalBinding(typeChecker, "unpack", unpackFunc, "@luau"); - - TypePackId selectArgsTypePack = arena.addTypePack(TypePack{ - {stringOrNumber}, - anyTypePack // FIXME? select() is tricky. - }); - - addGlobalBinding(typeChecker, "select", arena.addType(FunctionTypeVar{selectArgsTypePack, anyTypePack}), "@luau"); - - // TODO: not completely correct. loadstring's return type should be a function or (nil, string) - TypeId loadstringFunc = arena.addType(FunctionTypeVar{anyTypePack, oneAnyPack}); - - addGlobalBinding(typeChecker, "loadstring", - makeFunction(arena, std::nullopt, {stringType, optionalString}, - { - makeOption(typeChecker, arena, loadstringFunc), - makeOption(typeChecker, arena, stringType), - }), - "@luau"); - - // a userdata object is "roughly" the same as a sealed empty table - // except `type(newproxy(false))` evaluates to "userdata" so we may need another special type here too. - // another important thing to note: the value passed in conditionally creates an empty metatable, and you have to use getmetatable, NOT - // setmetatable. - // TODO: change this to something Luau can understand how to reject `setmetatable(newproxy(false or true), {})`. - TypeId sealedTable = arena.addType(TableTypeVar(TableState::Sealed, typeChecker.globalScope->level)); - addGlobalBinding(typeChecker, "newproxy", makeFunction(arena, std::nullopt, {optionalBoolean}, {sealedTable}), "@luau"); - } - // next(t: Table, i: K | nil) -> (K, V) TypePackId nextArgsTypePack = arena.addTypePack(TypePack{{mapOfKtoV, makeOption(typeChecker, arena, genericK)}}); addGlobalBinding(typeChecker, "next", @@ -475,8 +238,7 @@ void registerBuiltinTypes(TypeChecker& typeChecker) TypePackId pairsArgsTypePack = arena.addTypePack({mapOfKtoV}); - TypeId pairsNext = (FFlag::LuauRankNTypes ? arena.addType(FunctionTypeVar{nextArgsTypePack, arena.addTypePack(TypePack{{genericK, genericV}})}) - : getGlobalBinding(typeChecker, "next")); + TypeId pairsNext = arena.addType(FunctionTypeVar{nextArgsTypePack, arena.addTypePack(TypePack{{genericK, genericV}})}); TypePackId pairsReturnTypePack = arena.addTypePack(TypePack{{pairsNext, mapOfKtoV, nilType}}); // NOTE we are missing 'i: K | nil' argument in the first return types' argument. @@ -711,7 +473,7 @@ static std::optional> magicFunctionRequire( if (!checkRequirePath(typechecker, expr.args.data[0])) return std::nullopt; - const AstExpr* require = FFlag::LuauNewRequireTrace ? &expr : expr.args.data[0]; + const AstExpr* require = FFlag::LuauNewRequireTrace2 ? &expr : expr.args.data[0]; if (auto moduleInfo = typechecker.resolver->resolveModuleInfo(typechecker.currentModuleName, *require)) return ExprResult{arena.addTypePack({typechecker.checkRequire(scope, *moduleInfo, expr.location)})}; diff --git a/Analysis/src/EmbeddedBuiltinDefinitions.cpp b/Analysis/src/EmbeddedBuiltinDefinitions.cpp index 1e91561a..96703ef1 100644 --- a/Analysis/src/EmbeddedBuiltinDefinitions.cpp +++ b/Analysis/src/EmbeddedBuiltinDefinitions.cpp @@ -1,9 +1,6 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #include "Luau/BuiltinDefinitions.h" -LUAU_FASTFLAG(LuauParseGenericFunctions) -LUAU_FASTFLAG(LuauGenericFunctions) - namespace Luau { @@ -19,6 +16,8 @@ declare bit32: { bnot: (number) -> number, extract: (number, number, number?) -> number, replace: (number, number, number, number?) -> number, + countlz: (number) -> number, + countrz: (number) -> number, } declare math: { @@ -103,15 +102,6 @@ declare _VERSION: string declare function gcinfo(): number -)BUILTIN_SRC"; - -std::string getBuiltinDefinitionSource() -{ - std::string src = kBuiltinDefinitionLuaSrc; - - if (FFlag::LuauParseGenericFunctions && FFlag::LuauGenericFunctions) - { - src += R"( declare function print(...: T...) declare function type(value: T): string @@ -208,10 +198,12 @@ std::string getBuiltinDefinitionSource() -- Cannot use `typeof` here because it will produce a polytype when we expect a monotype. declare function unpack(tab: {V}, i: number?, j: number?): ...V - )"; - } - return src; +)BUILTIN_SRC"; + +std::string getBuiltinDefinitionSource() +{ + return kBuiltinDefinitionLuaSrc; } } // namespace Luau diff --git a/Analysis/src/Error.cpp b/Analysis/src/Error.cpp index 04d91444..46ff2c72 100644 --- a/Analysis/src/Error.cpp +++ b/Analysis/src/Error.cpp @@ -94,8 +94,23 @@ struct ErrorConverter { std::string operator()(const Luau::TypeMismatch& tm) const { - ToStringOptions opts; - return "Type '" + Luau::toString(tm.givenType, opts) + "' could not be converted into '" + Luau::toString(tm.wantedType, opts) + "'"; + std::string result = "Type '" + Luau::toString(tm.givenType) + "' could not be converted into '" + Luau::toString(tm.wantedType) + "'"; + + if (tm.error) + { + result += "\ncaused by:\n "; + + if (!tm.reason.empty()) + result += tm.reason + ". "; + + result += Luau::toString(*tm.error); + } + else if (!tm.reason.empty()) + { + result += "; " + tm.reason; + } + + return result; } std::string operator()(const Luau::UnknownSymbol& e) const @@ -478,9 +493,36 @@ struct InvalidNameChecker } }; +TypeMismatch::TypeMismatch(TypeId wantedType, TypeId givenType) + : wantedType(wantedType) + , givenType(givenType) +{ +} + +TypeMismatch::TypeMismatch(TypeId wantedType, TypeId givenType, std::string reason) + : wantedType(wantedType) + , givenType(givenType) + , reason(reason) +{ +} + +TypeMismatch::TypeMismatch(TypeId wantedType, TypeId givenType, std::string reason, TypeError error) + : wantedType(wantedType) + , givenType(givenType) + , reason(reason) + , error(std::make_shared(std::move(error))) +{ +} + bool TypeMismatch::operator==(const TypeMismatch& rhs) const { - return *wantedType == *rhs.wantedType && *givenType == *rhs.givenType; + if (!!error != !!rhs.error) + return false; + + if (error && !(*error == *rhs.error)) + return false; + + return *wantedType == *rhs.wantedType && *givenType == *rhs.givenType && reason == rhs.reason; } bool UnknownSymbol::operator==(const UnknownSymbol& rhs) const @@ -690,130 +732,141 @@ bool containsParseErrorName(const TypeError& error) return Luau::visit(InvalidNameChecker{}, error.data); } -void copyErrors(ErrorVec& errors, struct TypeArena& destArena) +template +void copyError(T& e, TypeArena& destArena, SeenTypes& seenTypes, SeenTypePacks& seenTypePacks) { - SeenTypes seenTypes; - SeenTypePacks seenTypePacks; - auto clone = [&](auto&& ty) { return ::Luau::clone(ty, destArena, seenTypes, seenTypePacks); }; auto visitErrorData = [&](auto&& e) { - using T = std::decay_t; + copyError(e, destArena, seenTypes, seenTypePacks); + }; - if constexpr (false) - { - } - else if constexpr (std::is_same_v) - { - e.wantedType = clone(e.wantedType); - e.givenType = clone(e.givenType); - } - else if constexpr (std::is_same_v) - { - } - else if constexpr (std::is_same_v) - { - e.table = clone(e.table); - } - else if constexpr (std::is_same_v) - { - e.ty = clone(e.ty); - } - else if constexpr (std::is_same_v) - { - e.tableType = clone(e.tableType); - } - else if constexpr (std::is_same_v) - { - e.tableType = clone(e.tableType); - } - else if constexpr (std::is_same_v) - { - } - else if constexpr (std::is_same_v) - { - } - else if constexpr (std::is_same_v) - { - } - else if constexpr (std::is_same_v) - { - } - else if constexpr (std::is_same_v) - { - } - else if constexpr (std::is_same_v) - { - } - else if constexpr (std::is_same_v) - { - e.typeFun = clone(e.typeFun); - } - else if constexpr (std::is_same_v) - { - } - else if constexpr (std::is_same_v) - { - } - else if constexpr (std::is_same_v) - { - } - else if constexpr (std::is_same_v) - { - e.table = clone(e.table); - } - else if constexpr (std::is_same_v) - { - } - else if constexpr (std::is_same_v) - { - e.ty = clone(e.ty); - } - else if constexpr (std::is_same_v) - { - } - else if constexpr (std::is_same_v) - { - } - else if constexpr (std::is_same_v) - { - } - else if constexpr (std::is_same_v) - { - } - else if constexpr (std::is_same_v) - { - e.expectedReturnType = clone(e.expectedReturnType); - } - else if constexpr (std::is_same_v) - { - } - else if constexpr (std::is_same_v) - { - } - else if constexpr (std::is_same_v) - { - e.superType = clone(e.superType); - e.subType = clone(e.subType); - } - else if constexpr (std::is_same_v) - { - } - else if constexpr (std::is_same_v) - { - e.optional = clone(e.optional); - } - else if constexpr (std::is_same_v) - { - e.type = clone(e.type); + if constexpr (false) + { + } + else if constexpr (std::is_same_v) + { + e.wantedType = clone(e.wantedType); + e.givenType = clone(e.givenType); - for (auto& ty : e.missing) - ty = clone(ty); - } - else - static_assert(always_false_v, "Non-exhaustive type switch"); + if (e.error) + visit(visitErrorData, e.error->data); + } + else if constexpr (std::is_same_v) + { + } + else if constexpr (std::is_same_v) + { + e.table = clone(e.table); + } + else if constexpr (std::is_same_v) + { + e.ty = clone(e.ty); + } + else if constexpr (std::is_same_v) + { + e.tableType = clone(e.tableType); + } + else if constexpr (std::is_same_v) + { + e.tableType = clone(e.tableType); + } + else if constexpr (std::is_same_v) + { + } + else if constexpr (std::is_same_v) + { + } + else if constexpr (std::is_same_v) + { + } + else if constexpr (std::is_same_v) + { + } + else if constexpr (std::is_same_v) + { + } + else if constexpr (std::is_same_v) + { + } + else if constexpr (std::is_same_v) + { + e.typeFun = clone(e.typeFun); + } + else if constexpr (std::is_same_v) + { + } + else if constexpr (std::is_same_v) + { + } + else if constexpr (std::is_same_v) + { + } + else if constexpr (std::is_same_v) + { + e.table = clone(e.table); + } + else if constexpr (std::is_same_v) + { + } + else if constexpr (std::is_same_v) + { + e.ty = clone(e.ty); + } + else if constexpr (std::is_same_v) + { + } + else if constexpr (std::is_same_v) + { + } + else if constexpr (std::is_same_v) + { + } + else if constexpr (std::is_same_v) + { + } + else if constexpr (std::is_same_v) + { + e.expectedReturnType = clone(e.expectedReturnType); + } + else if constexpr (std::is_same_v) + { + } + else if constexpr (std::is_same_v) + { + } + else if constexpr (std::is_same_v) + { + e.superType = clone(e.superType); + e.subType = clone(e.subType); + } + else if constexpr (std::is_same_v) + { + } + else if constexpr (std::is_same_v) + { + e.optional = clone(e.optional); + } + else if constexpr (std::is_same_v) + { + e.type = clone(e.type); + + for (auto& ty : e.missing) + ty = clone(ty); + } + else + static_assert(always_false_v, "Non-exhaustive type switch"); +} + +void copyErrors(ErrorVec& errors, TypeArena& destArena) +{ + SeenTypes seenTypes; + SeenTypePacks seenTypePacks; + + auto visitErrorData = [&](auto&& e) { + copyError(e, destArena, seenTypes, seenTypePacks); }; LUAU_ASSERT(!destArena.typeVars.isFrozen()); diff --git a/Analysis/src/Frontend.cpp b/Analysis/src/Frontend.cpp index 5e7af50c..2f411274 100644 --- a/Analysis/src/Frontend.cpp +++ b/Analysis/src/Frontend.cpp @@ -18,11 +18,10 @@ LUAU_FASTFLAG(LuauInferInNoCheckMode) LUAU_FASTFLAGVARIABLE(LuauTypeCheckTwice, false) LUAU_FASTFLAGVARIABLE(LuauKnowsTheDataModel3, false) -LUAU_FASTFLAGVARIABLE(LuauSecondTypecheckKnowsTheDataModel, false) LUAU_FASTFLAGVARIABLE(LuauResolveModuleNameWithoutACurrentModule, false) LUAU_FASTFLAG(LuauTraceRequireLookupChild) LUAU_FASTFLAGVARIABLE(LuauPersistDefinitionFileTypes, false) -LUAU_FASTFLAG(LuauNewRequireTrace) +LUAU_FASTFLAG(LuauNewRequireTrace2) LUAU_FASTFLAGVARIABLE(LuauClearScopes, false) namespace Luau @@ -415,7 +414,7 @@ CheckResult Frontend::check(const ModuleName& name) // If we're typechecking twice, we do so. // The second typecheck is always in strict mode with DM awareness // to provide better typen information for IDE features. - if (options.typecheckTwice && FFlag::LuauSecondTypecheckKnowsTheDataModel) + if (options.typecheckTwice) { ModulePtr moduleForAutocomplete = typeCheckerForAutocomplete.check(sourceModule, Mode::Strict); moduleResolverForAutocomplete.modules[moduleName] = moduleForAutocomplete; @@ -897,7 +896,7 @@ std::optional FrontendModuleResolver::resolveModuleInfo(const Module const auto& exprs = it->second.exprs; const ModuleInfo* info = exprs.find(&pathExpr); - if (!info || (!FFlag::LuauNewRequireTrace && info->name.empty())) + if (!info || (!FFlag::LuauNewRequireTrace2 && info->name.empty())) return std::nullopt; return *info; @@ -914,7 +913,7 @@ const ModulePtr FrontendModuleResolver::getModule(const ModuleName& moduleName) bool FrontendModuleResolver::moduleExists(const ModuleName& moduleName) const { - if (FFlag::LuauNewRequireTrace) + if (FFlag::LuauNewRequireTrace2) return frontend->sourceNodes.count(moduleName) != 0; else return frontend->fileResolver->moduleExists(moduleName); diff --git a/Analysis/src/Linter.cpp b/Analysis/src/Linter.cpp index bff947a5..1a5b24fe 100644 --- a/Analysis/src/Linter.cpp +++ b/Analysis/src/Linter.cpp @@ -12,9 +12,6 @@ #include #include -LUAU_FASTFLAGVARIABLE(LuauLinterUnknownTypeVectorAware, false) -LUAU_FASTFLAGVARIABLE(LuauLinterTableMoveZero, false) - namespace Luau { @@ -1110,10 +1107,7 @@ private: if (g && g->name == "type") { - if (FFlag::LuauLinterUnknownTypeVectorAware) - validateType(arg, {Kind_Primitive, Kind_Vector}, "primitive type"); - else - validateType(arg, {Kind_Primitive}, "primitive type"); + validateType(arg, {Kind_Primitive, Kind_Vector}, "primitive type"); } else if (g && g->name == "typeof") { @@ -2146,7 +2140,7 @@ private: "wrap it in parentheses to silence"); } - if (FFlag::LuauLinterTableMoveZero && func->index == "move" && node->args.size >= 4) + if (func->index == "move" && node->args.size >= 4) { // table.move(t, 0, _, _) if (isConstant(args[1], 0.0)) diff --git a/Analysis/src/Module.cpp b/Analysis/src/Module.cpp index 2fd95896..880ffd2e 100644 --- a/Analysis/src/Module.cpp +++ b/Analysis/src/Module.cpp @@ -12,7 +12,6 @@ LUAU_FASTFLAGVARIABLE(DebugLuauFreezeArena, false) LUAU_FASTFLAGVARIABLE(DebugLuauTrackOwningArena, false) -LUAU_FASTFLAG(LuauSecondTypecheckKnowsTheDataModel) LUAU_FASTFLAG(LuauCaptureBrokenCommentSpans) LUAU_FASTFLAG(LuauTypeAliasPacks) LUAU_FASTFLAGVARIABLE(LuauCloneBoundTables, false) @@ -290,9 +289,7 @@ void TypeCloner::operator()(const FunctionTypeVar& t) for (TypePackId genericPack : t.genericPacks) ftv->genericPacks.push_back(clone(genericPack, dest, seenTypes, seenTypePacks, encounteredFreeType)); - if (FFlag::LuauSecondTypecheckKnowsTheDataModel) - ftv->tags = t.tags; - + ftv->tags = t.tags; ftv->argTypes = clone(t.argTypes, dest, seenTypes, seenTypePacks, encounteredFreeType); ftv->argNames = t.argNames; ftv->retType = clone(t.retType, dest, seenTypes, seenTypePacks, encounteredFreeType); @@ -319,12 +316,7 @@ void TypeCloner::operator()(const TableTypeVar& t) ttv->level = TypeLevel{0, 0}; for (const auto& [name, prop] : t.props) - { - if (FFlag::LuauSecondTypecheckKnowsTheDataModel) - ttv->props[name] = {clone(prop.type, dest, seenTypes, seenTypePacks, encounteredFreeType), prop.deprecated, {}, prop.location, prop.tags}; - else - ttv->props[name] = {clone(prop.type, dest, seenTypes, seenTypePacks, encounteredFreeType), prop.deprecated, {}, prop.location}; - } + ttv->props[name] = {clone(prop.type, dest, seenTypes, seenTypePacks, encounteredFreeType), prop.deprecated, {}, prop.location, prop.tags}; if (t.indexer) ttv->indexer = TableIndexer{clone(t.indexer->indexType, dest, seenTypes, seenTypePacks, encounteredFreeType), @@ -379,10 +371,7 @@ void TypeCloner::operator()(const ClassTypeVar& t) seenTypes[typeId] = result; for (const auto& [name, prop] : t.props) - if (FFlag::LuauSecondTypecheckKnowsTheDataModel) - ctv->props[name] = {clone(prop.type, dest, seenTypes, seenTypePacks, encounteredFreeType), prop.deprecated, {}, prop.location, prop.tags}; - else - ctv->props[name] = {clone(prop.type, dest, seenTypes, seenTypePacks, encounteredFreeType), prop.deprecated, {}, prop.location}; + ctv->props[name] = {clone(prop.type, dest, seenTypes, seenTypePacks, encounteredFreeType), prop.deprecated, {}, prop.location, prop.tags}; if (t.parent) ctv->parent = clone(*t.parent, dest, seenTypes, seenTypePacks, encounteredFreeType); diff --git a/Analysis/src/Predicate.cpp b/Analysis/src/Predicate.cpp index 25e63bff..848627cf 100644 --- a/Analysis/src/Predicate.cpp +++ b/Analysis/src/Predicate.cpp @@ -3,8 +3,6 @@ #include "Luau/Ast.h" -LUAU_FASTFLAG(LuauOrPredicate) - namespace Luau { @@ -60,8 +58,6 @@ std::string toString(const LValue& lvalue) void merge(RefinementMap& l, const RefinementMap& r, std::function f) { - LUAU_ASSERT(FFlag::LuauOrPredicate); - auto itL = l.begin(); auto itR = r.begin(); while (itL != l.end() && itR != r.end()) diff --git a/Analysis/src/RequireTracer.cpp b/Analysis/src/RequireTracer.cpp index 95910b56..b72f53f9 100644 --- a/Analysis/src/RequireTracer.cpp +++ b/Analysis/src/RequireTracer.cpp @@ -5,7 +5,7 @@ #include "Luau/Module.h" LUAU_FASTFLAGVARIABLE(LuauTraceRequireLookupChild, false) -LUAU_FASTFLAGVARIABLE(LuauNewRequireTrace, false) +LUAU_FASTFLAGVARIABLE(LuauNewRequireTrace2, false) namespace Luau { @@ -19,7 +19,7 @@ struct RequireTracerOld : AstVisitor : fileResolver(fileResolver) , currentModuleName(currentModuleName) { - LUAU_ASSERT(!FFlag::LuauNewRequireTrace); + LUAU_ASSERT(!FFlag::LuauNewRequireTrace2); } FileResolver* const fileResolver; @@ -188,7 +188,7 @@ struct RequireTracer : AstVisitor , currentModuleName(currentModuleName) , locals(nullptr) { - LUAU_ASSERT(FFlag::LuauNewRequireTrace); + LUAU_ASSERT(FFlag::LuauNewRequireTrace2); } bool visit(AstExprTypeAssertion* expr) override @@ -332,7 +332,7 @@ struct RequireTracer : AstVisitor RequireTraceResult traceRequires(FileResolver* fileResolver, AstStatBlock* root, const ModuleName& currentModuleName) { - if (FFlag::LuauNewRequireTrace) + if (FFlag::LuauNewRequireTrace2) { RequireTraceResult result; RequireTracer tracer{result, fileResolver, currentModuleName}; diff --git a/Analysis/src/Substitution.cpp b/Analysis/src/Substitution.cpp index d861eb3d..ca2b30f5 100644 --- a/Analysis/src/Substitution.cpp +++ b/Analysis/src/Substitution.cpp @@ -8,8 +8,6 @@ LUAU_FASTINTVARIABLE(LuauTarjanChildLimit, 1000) LUAU_FASTFLAGVARIABLE(LuauSubstitutionDontReplaceIgnoredTypes, false) -LUAU_FASTFLAG(LuauSecondTypecheckKnowsTheDataModel) -LUAU_FASTFLAG(LuauRankNTypes) LUAU_FASTFLAG(LuauTypeAliasPacks) namespace Luau @@ -19,7 +17,7 @@ void Tarjan::visitChildren(TypeId ty, int index) { ty = follow(ty); - if (FFlag::LuauRankNTypes && ignoreChildren(ty)) + if (ignoreChildren(ty)) return; if (const FunctionTypeVar* ftv = get(ty)) @@ -68,7 +66,7 @@ void Tarjan::visitChildren(TypePackId tp, int index) { tp = follow(tp); - if (FFlag::LuauRankNTypes && ignoreChildren(tp)) + if (ignoreChildren(tp)) return; if (const TypePack* tpp = get(tp)) @@ -399,8 +397,7 @@ TypeId Substitution::clone(TypeId ty) if (FFlag::LuauTypeAliasPacks) clone.instantiatedTypePackParams = ttv->instantiatedTypePackParams; - if (FFlag::LuauSecondTypecheckKnowsTheDataModel) - clone.tags = ttv->tags; + clone.tags = ttv->tags; result = addType(std::move(clone)); } else if (const MetatableTypeVar* mtv = get(ty)) @@ -486,7 +483,7 @@ void Substitution::replaceChildren(TypeId ty) { ty = follow(ty); - if (FFlag::LuauRankNTypes && ignoreChildren(ty)) + if (ignoreChildren(ty)) return; if (FunctionTypeVar* ftv = getMutable(ty)) @@ -535,7 +532,7 @@ void Substitution::replaceChildren(TypePackId tp) { tp = follow(tp); - if (FFlag::LuauRankNTypes && ignoreChildren(tp)) + if (ignoreChildren(tp)) return; if (TypePack* tpp = getMutable(tp)) diff --git a/Analysis/src/ToString.cpp b/Analysis/src/ToString.cpp index cd8180db..885fd489 100644 --- a/Analysis/src/ToString.cpp +++ b/Analysis/src/ToString.cpp @@ -11,7 +11,6 @@ #include LUAU_FASTFLAG(LuauOccursCheckOkWithRecursiveFunctions) -LUAU_FASTFLAGVARIABLE(LuauInstantiatedTypeParamRecursion, false) LUAU_FASTFLAG(LuauTypeAliasPacks) namespace Luau @@ -237,15 +236,6 @@ struct TypeVarStringifier return; } - if (!FFlag::LuauAddMissingFollow) - { - if (get(tv)) - { - state.emit(state.getName(tv)); - return; - } - } - Luau::visit( [this, tv](auto&& t) { return (*this)(tv, t); @@ -316,11 +306,7 @@ struct TypeVarStringifier void operator()(TypeId ty, const Unifiable::Free& ftv) { state.result.invalid = true; - - if (FFlag::LuauAddMissingFollow) - state.emit(state.getName(ty)); - else - state.emit(""); + state.emit(state.getName(ty)); } void operator()(TypeId, const BoundTypeVar& btv) @@ -724,16 +710,6 @@ struct TypePackStringifier return; } - if (!FFlag::LuauAddMissingFollow) - { - if (get(tp)) - { - state.emit(state.getName(tp)); - state.emit("..."); - return; - } - } - auto it = state.cycleTpNames.find(tp); if (it != state.cycleTpNames.end()) { @@ -821,16 +797,8 @@ struct TypePackStringifier void operator()(TypePackId tp, const FreeTypePack& pack) { state.result.invalid = true; - - if (FFlag::LuauAddMissingFollow) - { - state.emit(state.getName(tp)); - state.emit("..."); - } - else - { - state.emit(""); - } + state.emit(state.getName(tp)); + state.emit("..."); } void operator()(TypePackId, const BoundTypePack& btv) @@ -864,23 +832,15 @@ static void assignCycleNames(const std::unordered_set& cycles, const std std::string name; // TODO: use the stringified type list if there are no cycles - if (FFlag::LuauInstantiatedTypeParamRecursion) + if (auto ttv = get(follow(cycleTy)); !exhaustive && ttv && (ttv->syntheticName || ttv->name)) { - if (auto ttv = get(follow(cycleTy)); !exhaustive && ttv && (ttv->syntheticName || ttv->name)) - { - // If we have a cycle type in type parameters, assign a cycle name for this named table - if (std::find_if(ttv->instantiatedTypeParams.begin(), ttv->instantiatedTypeParams.end(), [&](auto&& el) { - return cycles.count(follow(el)); - }) != ttv->instantiatedTypeParams.end()) - cycleNames[cycleTy] = ttv->name ? *ttv->name : *ttv->syntheticName; + // If we have a cycle type in type parameters, assign a cycle name for this named table + if (std::find_if(ttv->instantiatedTypeParams.begin(), ttv->instantiatedTypeParams.end(), [&](auto&& el) { + return cycles.count(follow(el)); + }) != ttv->instantiatedTypeParams.end()) + cycleNames[cycleTy] = ttv->name ? *ttv->name : *ttv->syntheticName; - continue; - } - } - else - { - if (auto ttv = get(follow(cycleTy)); !exhaustive && ttv && (ttv->syntheticName || ttv->name)) - continue; + continue; } name = "t" + std::to_string(nextIndex); @@ -912,58 +872,6 @@ ToStringResult toStringDetailed(TypeId ty, const ToStringOptions& opts) ToStringResult result; - if (!FFlag::LuauInstantiatedTypeParamRecursion && !opts.exhaustive) - { - if (auto ttv = get(ty); ttv && (ttv->name || ttv->syntheticName)) - { - if (ttv->syntheticName) - result.invalid = true; - - // If scope if provided, add module name and check visibility - if (ttv->name && opts.scope) - { - auto [success, moduleName] = canUseTypeNameInScope(opts.scope, *ttv->name); - - if (!success) - result.invalid = true; - - if (moduleName) - result.name = format("%s.", moduleName->c_str()); - } - - result.name += ttv->name ? *ttv->name : *ttv->syntheticName; - - if (ttv->instantiatedTypeParams.empty() && (!FFlag::LuauTypeAliasPacks || ttv->instantiatedTypePackParams.empty())) - return result; - - std::vector params; - for (TypeId tp : ttv->instantiatedTypeParams) - params.push_back(toString(tp)); - - if (FFlag::LuauTypeAliasPacks) - { - // Doesn't preserve grouping of multiple type packs - // But this is under a parent block of code that is being removed later - for (TypePackId tp : ttv->instantiatedTypePackParams) - { - std::string content = toString(tp); - - if (!content.empty()) - params.push_back(std::move(content)); - } - } - - result.name += "<" + join(params, ", ") + ">"; - return result; - } - else if (auto mtv = get(ty); mtv && mtv->syntheticName) - { - result.invalid = true; - result.name = *mtv->syntheticName; - return result; - } - } - StringifierState state{opts, result, opts.nameMap}; std::unordered_set cycles; @@ -975,7 +883,7 @@ ToStringResult toStringDetailed(TypeId ty, const ToStringOptions& opts) TypeVarStringifier tvs{state}; - if (FFlag::LuauInstantiatedTypeParamRecursion && !opts.exhaustive) + if (!opts.exhaustive) { if (auto ttv = get(ty); ttv && (ttv->name || ttv->syntheticName)) { diff --git a/Analysis/src/Transpiler.cpp b/Analysis/src/Transpiler.cpp index 1b83ccdc..7d880af4 100644 --- a/Analysis/src/Transpiler.cpp +++ b/Analysis/src/Transpiler.cpp @@ -10,7 +10,6 @@ #include #include -LUAU_FASTFLAG(LuauGenericFunctions) LUAU_FASTFLAG(LuauTypeAliasPacks) namespace @@ -97,9 +96,6 @@ struct Writer { virtual ~Writer() {} - virtual void begin() {} - virtual void end() {} - virtual void advance(const Position&) = 0; virtual void newline() = 0; virtual void space() = 0; @@ -131,6 +127,7 @@ struct StringWriter : Writer if (pos.column < newPos.column) write(std::string(newPos.column - pos.column, ' ')); } + void maybeSpace(const Position& newPos, int reserve) override { if (pos.column + reserve < newPos.column) @@ -279,11 +276,14 @@ struct Printer writer.identifier(func->index.value); } - void visualizeTypePackAnnotation(const AstTypePack& annotation) + void visualizeTypePackAnnotation(const AstTypePack& annotation, bool forVarArg) { + advance(annotation.location.begin); if (const AstTypePackVariadic* variadicTp = annotation.as()) { - writer.symbol("..."); + if (!forVarArg) + writer.symbol("..."); + visualizeTypeAnnotation(*variadicTp->variadicType); } else if (const AstTypePackGeneric* genericTp = annotation.as()) @@ -293,6 +293,7 @@ struct Printer } else if (const AstTypePackExplicit* explicitTp = annotation.as()) { + LUAU_ASSERT(!forVarArg); visualizeTypeList(explicitTp->typeList, true); } else @@ -317,7 +318,7 @@ struct Printer // Only variadic tail if (list.types.size == 0) { - visualizeTypePackAnnotation(*list.tailType); + visualizeTypePackAnnotation(*list.tailType, false); } else { @@ -345,7 +346,7 @@ struct Printer if (list.tailType) { writer.symbol(","); - visualizeTypePackAnnotation(*list.tailType); + visualizeTypePackAnnotation(*list.tailType, false); } writer.symbol(")"); @@ -542,6 +543,7 @@ struct Printer case AstExprBinary::CompareLt: case AstExprBinary::CompareGt: writer.maybeSpace(a->right->location.begin, 2); + writer.symbol(toString(a->op)); break; case AstExprBinary::Concat: case AstExprBinary::CompareNe: @@ -550,19 +552,35 @@ struct Printer case AstExprBinary::CompareGe: case AstExprBinary::Or: writer.maybeSpace(a->right->location.begin, 3); + writer.keyword(toString(a->op)); break; case AstExprBinary::And: writer.maybeSpace(a->right->location.begin, 4); + writer.keyword(toString(a->op)); break; } - writer.symbol(toString(a->op)); - visualize(*a->right); } else if (const auto& a = expr.as()) { visualize(*a->expr); + + if (writeTypes) + { + writer.maybeSpace(a->annotation->location.begin, 2); + writer.symbol("::"); + visualizeTypeAnnotation(*a->annotation); + } + } + else if (const auto& a = expr.as()) + { + writer.keyword("if"); + visualize(*a->condition); + writer.keyword("then"); + visualize(*a->trueExpr); + writer.keyword("else"); + visualize(*a->falseExpr); } else if (const auto& a = expr.as()) { @@ -769,24 +787,31 @@ struct Printer switch (a->op) { case AstExprBinary::Add: + writer.maybeSpace(a->value->location.begin, 2); writer.symbol("+="); break; case AstExprBinary::Sub: + writer.maybeSpace(a->value->location.begin, 2); writer.symbol("-="); break; case AstExprBinary::Mul: + writer.maybeSpace(a->value->location.begin, 2); writer.symbol("*="); break; case AstExprBinary::Div: + writer.maybeSpace(a->value->location.begin, 2); writer.symbol("/="); break; case AstExprBinary::Mod: + writer.maybeSpace(a->value->location.begin, 2); writer.symbol("%="); break; case AstExprBinary::Pow: + writer.maybeSpace(a->value->location.begin, 2); writer.symbol("^="); break; case AstExprBinary::Concat: + writer.maybeSpace(a->value->location.begin, 3); writer.symbol("..="); break; default: @@ -874,7 +899,7 @@ struct Printer void visualizeFunctionBody(AstExprFunction& func) { - if (FFlag::LuauGenericFunctions && (func.generics.size > 0 || func.genericPacks.size > 0)) + if (func.generics.size > 0 || func.genericPacks.size > 0) { CommaSeparatorInserter comma(writer); writer.symbol("<"); @@ -913,12 +938,13 @@ struct Printer if (func.vararg) { comma(); + advance(func.varargLocation.begin); writer.symbol("..."); if (func.varargAnnotation) { writer.symbol(":"); - visualizeTypePackAnnotation(*func.varargAnnotation); + visualizeTypePackAnnotation(*func.varargAnnotation, true); } } @@ -980,8 +1006,14 @@ struct Printer advance(typeAnnotation.location.begin); if (const auto& a = typeAnnotation.as()) { + if (a->hasPrefix) + { + writer.write(a->prefix.value); + writer.symbol("."); + } + writer.write(a->name.value); - if (a->parameters.size > 0) + if (a->parameters.size > 0 || a->hasParameterList) { CommaSeparatorInserter comma(writer); writer.symbol("<"); @@ -992,7 +1024,7 @@ struct Printer if (o.type) visualizeTypeAnnotation(*o.type); else - visualizeTypePackAnnotation(*o.typePack); + visualizeTypePackAnnotation(*o.typePack, false); } writer.symbol(">"); @@ -1000,7 +1032,7 @@ struct Printer } else if (const auto& a = typeAnnotation.as()) { - if (FFlag::LuauGenericFunctions && (a->generics.size > 0 || a->genericPacks.size > 0)) + if (a->generics.size > 0 || a->genericPacks.size > 0) { CommaSeparatorInserter comma(writer); writer.symbol("<"); @@ -1075,7 +1107,16 @@ struct Printer auto rta = r->as(); if (rta && rta->name == "nil") { + bool wrap = l->as() || l->as(); + + if (wrap) + writer.symbol("("); + visualizeTypeAnnotation(*l); + + if (wrap) + writer.symbol(")"); + writer.symbol("?"); return; } @@ -1089,7 +1130,15 @@ struct Printer writer.symbol("|"); } + bool wrap = a->types.data[i]->as() || a->types.data[i]->as(); + + if (wrap) + writer.symbol("("); + visualizeTypeAnnotation(*a->types.data[i]); + + if (wrap) + writer.symbol(")"); } } else if (const auto& a = typeAnnotation.as()) @@ -1102,7 +1151,15 @@ struct Printer writer.symbol("&"); } + bool wrap = a->types.data[i]->as() || a->types.data[i]->as(); + + if (wrap) + writer.symbol("("); + visualizeTypeAnnotation(*a->types.data[i]); + + if (wrap) + writer.symbol(")"); } } else if (typeAnnotation.is()) @@ -1116,31 +1173,27 @@ struct Printer } }; -void dump(AstNode* node) +std::string toString(AstNode* node) { StringWriter writer; + writer.pos = node->location.begin; + Printer printer(writer); printer.writeTypes = true; if (auto statNode = dynamic_cast(node)) - { printer.visualize(*statNode); - printf("%s\n", writer.str().c_str()); - } else if (auto exprNode = dynamic_cast(node)) - { printer.visualize(*exprNode); - printf("%s\n", writer.str().c_str()); - } else if (auto typeNode = dynamic_cast(node)) - { printer.visualizeTypeAnnotation(*typeNode); - printf("%s\n", writer.str().c_str()); - } - else - { - printf("Can't dump this node\n"); - } + + return writer.str(); +} + +void dump(AstNode* node) +{ + printf("%s\n", toString(node).c_str()); } std::string transpile(AstStatBlock& block) @@ -1149,6 +1202,7 @@ std::string transpile(AstStatBlock& block) Printer(writer).visualizeBlock(block); return writer.str(); } + std::string transpileWithTypes(AstStatBlock& block) { StringWriter writer; @@ -1158,7 +1212,7 @@ std::string transpileWithTypes(AstStatBlock& block) return writer.str(); } -TranspileResult transpile(std::string_view source, ParseOptions options) +TranspileResult transpile(std::string_view source, ParseOptions options, bool withTypes) { auto allocator = Allocator{}; auto names = AstNameTable{allocator}; @@ -1176,6 +1230,9 @@ TranspileResult transpile(std::string_view source, ParseOptions options) if (!parseResult.root) return TranspileResult{"", {}, "Internal error: Parser yielded empty parse tree"}; + if (withTypes) + return TranspileResult{transpileWithTypes(*parseResult.root)}; + return TranspileResult{transpile(*parseResult.root)}; } diff --git a/Analysis/src/TypeAttach.cpp b/Analysis/src/TypeAttach.cpp index 49f8e0ca..11aa7b39 100644 --- a/Analysis/src/TypeAttach.cpp +++ b/Analysis/src/TypeAttach.cpp @@ -13,7 +13,6 @@ #include -LUAU_FASTFLAG(LuauGenericFunctions) LUAU_FASTFLAG(LuauTypeAliasPacks) static char* allocateString(Luau::Allocator& allocator, std::string_view contents) @@ -203,39 +202,23 @@ public: return allocator->alloc(Location(), std::nullopt, AstName("")); AstArray generics; - if (FFlag::LuauGenericFunctions) + generics.size = ftv.generics.size(); + generics.data = static_cast(allocator->allocate(sizeof(AstName) * generics.size)); + size_t numGenerics = 0; + for (auto it = ftv.generics.begin(); it != ftv.generics.end(); ++it) { - generics.size = ftv.generics.size(); - generics.data = static_cast(allocator->allocate(sizeof(AstName) * generics.size)); - size_t i = 0; - for (auto it = ftv.generics.begin(); it != ftv.generics.end(); ++it) - { - if (auto gtv = get(*it)) - generics.data[i++] = AstName(gtv->name.c_str()); - } - } - else - { - generics.size = 0; - generics.data = nullptr; + if (auto gtv = get(*it)) + generics.data[numGenerics++] = AstName(gtv->name.c_str()); } AstArray genericPacks; - if (FFlag::LuauGenericFunctions) + genericPacks.size = ftv.genericPacks.size(); + genericPacks.data = static_cast(allocator->allocate(sizeof(AstName) * genericPacks.size)); + size_t numGenericPacks = 0; + for (auto it = ftv.genericPacks.begin(); it != ftv.genericPacks.end(); ++it) { - genericPacks.size = ftv.genericPacks.size(); - genericPacks.data = static_cast(allocator->allocate(sizeof(AstName) * genericPacks.size)); - size_t i = 0; - for (auto it = ftv.genericPacks.begin(); it != ftv.genericPacks.end(); ++it) - { - if (auto gtv = get(*it)) - genericPacks.data[i++] = AstName(gtv->name.c_str()); - } - } - else - { - generics.size = 0; - generics.data = nullptr; + if (auto gtv = get(*it)) + genericPacks.data[numGenericPacks++] = AstName(gtv->name.c_str()); } AstArray argTypes; diff --git a/Analysis/src/TypeInfer.cpp b/Analysis/src/TypeInfer.cpp index c8fc55ba..a6696efd 100644 --- a/Analysis/src/TypeInfer.cpp +++ b/Analysis/src/TypeInfer.cpp @@ -22,29 +22,19 @@ LUAU_FASTFLAGVARIABLE(DebugLuauMagicTypes, false) LUAU_FASTINTVARIABLE(LuauTypeInferRecursionLimit, 500) LUAU_FASTINTVARIABLE(LuauTypeInferTypePackLoopLimit, 5000) LUAU_FASTINTVARIABLE(LuauCheckRecursionLimit, 500) -LUAU_FASTFLAGVARIABLE(LuauGenericFunctions, false) -LUAU_FASTFLAGVARIABLE(LuauGenericVariadicsUnification, false) LUAU_FASTFLAG(LuauKnowsTheDataModel3) -LUAU_FASTFLAG(LuauSecondTypecheckKnowsTheDataModel) LUAU_FASTFLAGVARIABLE(LuauClassPropertyAccessAsString, false) LUAU_FASTFLAGVARIABLE(LuauEqConstraint, false) LUAU_FASTFLAGVARIABLE(LuauWeakEqConstraint, false) // Eventually removed as false. LUAU_FASTFLAG(LuauTraceRequireLookupChild) LUAU_FASTFLAGVARIABLE(LuauCloneCorrectlyBeforeMutatingTableType, false) LUAU_FASTFLAGVARIABLE(LuauStoreMatchingOverloadFnType, false) -LUAU_FASTFLAGVARIABLE(LuauRankNTypes, false) -LUAU_FASTFLAGVARIABLE(LuauOrPredicate, false) -LUAU_FASTFLAGVARIABLE(LuauInferReturnAssertAssign, false) LUAU_FASTFLAGVARIABLE(LuauRecursiveTypeParameterRestriction, false) -LUAU_FASTFLAGVARIABLE(LuauAddMissingFollow, false) -LUAU_FASTFLAGVARIABLE(LuauTypeGuardPeelsAwaySubclasses, false) -LUAU_FASTFLAGVARIABLE(LuauSlightlyMoreFlexibleBinaryPredicates, false) -LUAU_FASTFLAGVARIABLE(LuauFollowInTypeFunApply, false) LUAU_FASTFLAGVARIABLE(LuauIfElseExpressionAnalysisSupport, false) LUAU_FASTFLAGVARIABLE(LuauStrictRequire, false) LUAU_FASTFLAG(LuauSubstitutionDontReplaceIgnoredTypes) LUAU_FASTFLAGVARIABLE(LuauQuantifyInPlace2, false) -LUAU_FASTFLAG(LuauNewRequireTrace) +LUAU_FASTFLAG(LuauNewRequireTrace2) LUAU_FASTFLAG(LuauTypeAliasPacks) namespace Luau @@ -222,9 +212,9 @@ TypeChecker::TypeChecker(ModuleResolver* resolver, InternalErrorReporter* iceHan , threadType(singletonTypes.threadType) , anyType(singletonTypes.anyType) , errorType(singletonTypes.errorType) - , optionalNumberType(globalTypes.addType(UnionTypeVar{{numberType, nilType}})) - , anyTypePack(globalTypes.addTypePack(TypePackVar{VariadicTypePack{singletonTypes.anyType}, true})) - , errorTypePack(globalTypes.addTypePack(TypePackVar{Unifiable::Error{}})) + , optionalNumberType(singletonTypes.optionalNumberType) + , anyTypePack(singletonTypes.anyTypePack) + , errorTypePack(singletonTypes.errorTypePack) { globalScope = std::make_shared(globalTypes.addTypePack(TypePackVar{FreeTypePack{TypeLevel{}}})); @@ -251,10 +241,8 @@ ModulePtr TypeChecker::check(const SourceModule& module, Mode mode, std::optiona if (module.cyclic) moduleScope->returnType = addTypePack(TypePack{{anyType}, std::nullopt}); - else if (FFlag::LuauRankNTypes) - moduleScope->returnType = freshTypePack(moduleScope); else - moduleScope->returnType = DEPRECATED_freshTypePack(moduleScope, true); + moduleScope->returnType = freshTypePack(moduleScope); moduleScope->varargPack = anyTypePack; @@ -268,7 +256,7 @@ ModulePtr TypeChecker::check(const SourceModule& module, Mode mode, std::optiona checkBlock(moduleScope, *module.root); - if (get(FFlag::LuauAddMissingFollow ? follow(moduleScope->returnType) : moduleScope->returnType)) + if (get(follow(moduleScope->returnType))) moduleScope->returnType = addTypePack(TypePack{{}, std::nullopt}); else moduleScope->returnType = anyify(moduleScope, moduleScope->returnType, Location{}); @@ -326,7 +314,7 @@ void TypeChecker::check(const ScopePtr& scope, const AstStat& program) check(scope, *typealias); else if (auto global = program.as()) { - TypeId globalType = (FFlag::LuauRankNTypes ? resolveType(scope, *global->type) : resolveType(scope, *global->type, true)); + TypeId globalType = resolveType(scope, *global->type); Name globalName(global->name.value); currentModule->declaredGlobals[globalName] = globalType; @@ -494,7 +482,7 @@ LUAU_NOINLINE void TypeChecker::checkBlockTypeAliases(const ScopePtr& scope, std Name name = typealias->name.value; TypeId type = bindings[name].type; - if (get(FFlag::LuauAddMissingFollow ? follow(type) : type)) + if (get(follow(type))) { *asMutable(type) = ErrorTypeVar{}; reportError(TypeError{typealias->location, OccursCheckFailed{}}); @@ -607,26 +595,22 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatRepeat& statement) void TypeChecker::check(const ScopePtr& scope, const AstStatReturn& return_) { std::vector> expectedTypes; + expectedTypes.reserve(return_.list.size); - if (FFlag::LuauInferReturnAssertAssign) + TypePackIterator expectedRetCurr = begin(scope->returnType); + TypePackIterator expectedRetEnd = end(scope->returnType); + + for (size_t i = 0; i < return_.list.size; ++i) { - expectedTypes.reserve(return_.list.size); - - TypePackIterator expectedRetCurr = begin(scope->returnType); - TypePackIterator expectedRetEnd = end(scope->returnType); - - for (size_t i = 0; i < return_.list.size; ++i) + if (expectedRetCurr != expectedRetEnd) { - if (expectedRetCurr != expectedRetEnd) - { - expectedTypes.push_back(*expectedRetCurr); - ++expectedRetCurr; - } - else if (auto expectedArgsTail = expectedRetCurr.tail()) - { - if (const VariadicTypePack* vtp = get(follow(*expectedArgsTail))) - expectedTypes.push_back(vtp->ty); - } + expectedTypes.push_back(*expectedRetCurr); + ++expectedRetCurr; + } + else if (auto expectedArgsTail = expectedRetCurr.tail()) + { + if (const VariadicTypePack* vtp = get(follow(*expectedArgsTail))) + expectedTypes.push_back(vtp->ty); } } @@ -672,34 +656,30 @@ ErrorVec TypeChecker::tryUnify_(Id left, Id right, const Location& location) void TypeChecker::check(const ScopePtr& scope, const AstStatAssign& assign) { std::vector> expectedTypes; + expectedTypes.reserve(assign.vars.size); - if (FFlag::LuauInferReturnAssertAssign) + ScopePtr moduleScope = currentModule->getModuleScope(); + + for (size_t i = 0; i < assign.vars.size; ++i) { - expectedTypes.reserve(assign.vars.size); + AstExpr* dest = assign.vars.data[i]; - ScopePtr moduleScope = currentModule->getModuleScope(); - - for (size_t i = 0; i < assign.vars.size; ++i) + if (auto a = dest->as()) { - AstExpr* dest = assign.vars.data[i]; - - if (auto a = dest->as()) - { - // AstExprLocal l-values will have to be checked again because their type might have been mutated during checkExprList later - expectedTypes.push_back(scope->lookup(a->local)); - } - else if (auto a = dest->as()) - { - // AstExprGlobal l-values lookup is inlined here to avoid creating a global binding before checkExprList - if (auto it = moduleScope->bindings.find(a->name); it != moduleScope->bindings.end()) - expectedTypes.push_back(it->second.typeId); - else - expectedTypes.push_back(std::nullopt); - } + // AstExprLocal l-values will have to be checked again because their type might have been mutated during checkExprList later + expectedTypes.push_back(scope->lookup(a->local)); + } + else if (auto a = dest->as()) + { + // AstExprGlobal l-values lookup is inlined here to avoid creating a global binding before checkExprList + if (auto it = moduleScope->bindings.find(a->name); it != moduleScope->bindings.end()) + expectedTypes.push_back(it->second.typeId); else - { - expectedTypes.push_back(checkLValue(scope, *dest)); - } + expectedTypes.push_back(std::nullopt); + } + else + { + expectedTypes.push_back(checkLValue(scope, *dest)); } } @@ -715,7 +695,7 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatAssign& assign) AstExpr* dest = assign.vars.data[i]; TypeId left = nullptr; - if (!FFlag::LuauInferReturnAssertAssign || dest->is() || dest->is()) + if (dest->is() || dest->is()) left = checkLValue(scope, *dest); else left = *expectedTypes[i]; @@ -751,11 +731,7 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatAssign& assign) if (right) { - if (FFlag::LuauGenericFunctions && !maybeGeneric(left) && isGeneric(right)) - right = instantiate(scope, right, loc); - - if (!FFlag::LuauGenericFunctions && get(FFlag::LuauAddMissingFollow ? follow(left) : left) && - get(FFlag::LuauAddMissingFollow ? follow(right) : right)) + if (!maybeGeneric(left) && isGeneric(right)) right = instantiate(scope, right, loc); // Setting a table entry to nil doesn't mean nil is the type of the indexer, it is just deleting the entry @@ -766,7 +742,7 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatAssign& assign) if (!destTableTypeReceivingNil || !destTableTypeReceivingNil->indexer) { // In nonstrict mode, any assignments where the lhs is free and rhs isn't a function, we give it any typevar. - if (isNonstrictMode() && get(FFlag::LuauAddMissingFollow ? follow(left) : left) && !get(follow(right))) + if (isNonstrictMode() && get(follow(left)) && !get(follow(right))) unify(left, anyType, loc); else unify(left, right, loc); @@ -815,7 +791,7 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatLocal& local) if (annotation) { - ty = (FFlag::LuauRankNTypes ? resolveType(scope, *annotation) : resolveType(scope, *annotation, true)); + ty = resolveType(scope, *annotation); // If the annotation type has an error, treat it as if there was no annotation if (get(follow(ty))) @@ -823,23 +799,19 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatLocal& local) } if (!ty) - ty = rhsIsTable ? (FFlag::LuauRankNTypes ? freshType(scope) : DEPRECATED_freshType(scope, true)) - : isNonstrictMode() ? anyType : (FFlag::LuauRankNTypes ? freshType(scope) : DEPRECATED_freshType(scope, true)); + ty = rhsIsTable ? freshType(scope) : isNonstrictMode() ? anyType : freshType(scope); varBindings.emplace_back(vars[i], Binding{ty, vars[i]->location}); variableTypes.push_back(ty); expectedTypes.push_back(ty); - if (FFlag::LuauGenericFunctions) - instantiateGenerics.push_back(annotation != nullptr && !maybeGeneric(ty)); - else - instantiateGenerics.push_back(annotation != nullptr && get(FFlag::LuauAddMissingFollow ? follow(ty) : ty)); + instantiateGenerics.push_back(annotation != nullptr && !maybeGeneric(ty)); } if (local.values.size > 0) { - TypePackId variablePack = addTypePack(variableTypes, FFlag::LuauRankNTypes ? freshTypePack(scope) : DEPRECATED_freshTypePack(scope, true)); + TypePackId variablePack = addTypePack(variableTypes, freshTypePack(scope)); TypePackId valuePack = checkExprList(scope, local.location, local.values, /* substituteFreeForNil= */ true, instantiateGenerics, expectedTypes).type; @@ -979,8 +951,6 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatForIn& forin) { AstExprCall* exprCall = firstValue->as(); callRetPack = checkExprPack(scope, *exprCall).type; - if (!FFlag::LuauRankNTypes) - callRetPack = DEPRECATED_instantiate(scope, callRetPack, exprCall->location); callRetPack = follow(callRetPack); if (get(callRetPack)) @@ -998,8 +968,7 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatForIn& forin) else { iterTy = *first(callRetPack); - if (FFlag::LuauRankNTypes) - iterTy = instantiate(scope, iterTy, exprCall->location); + iterTy = instantiate(scope, iterTy, exprCall->location); } } else @@ -1158,10 +1127,7 @@ void TypeChecker::check(const ScopePtr& scope, TypeId ty, const ScopePtr& funSco checkFunctionBody(funScope, ty, *function.func); - if (FFlag::LuauGenericFunctions) - scope->bindings[function.name] = {quantify(funScope, ty, function.name->location), function.name->location}; - else - scope->bindings[function.name] = {quantify(scope, ty, function.name->location), function.name->location}; + scope->bindings[function.name] = {quantify(funScope, ty, function.name->location), function.name->location}; } void TypeChecker::check(const ScopePtr& scope, const AstStatTypeAlias& typealias, int subLevel, bool forwardDeclare) @@ -1199,7 +1165,7 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatTypeAlias& typealias { auto [generics, genericPacks] = createGenericTypes(aliasScope, scope->level, typealias, typealias.generics, typealias.genericPacks); - TypeId ty = (FFlag::LuauRankNTypes ? freshType(aliasScope) : DEPRECATED_freshType(scope, true)); + TypeId ty = freshType(aliasScope); FreeTypeVar* ftv = getMutable(ty); LUAU_ASSERT(ftv); ftv->forwardedTypeAlias = true; @@ -1234,7 +1200,7 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatTypeAlias& typealias aliasScope->privateTypeBindings[n] = TypeFun{{}, g}; } - TypeId ty = (FFlag::LuauRankNTypes ? freshType(aliasScope) : DEPRECATED_freshType(scope, true)); + TypeId ty = freshType(aliasScope); FreeTypeVar* ftv = getMutable(ty); LUAU_ASSERT(ftv); ftv->forwardedTypeAlias = true; @@ -1266,7 +1232,7 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatTypeAlias& typealias } } - TypeId ty = (FFlag::LuauRankNTypes ? resolveType(aliasScope, *typealias.type) : resolveType(aliasScope, *typealias.type, true)); + TypeId ty = resolveType(aliasScope, *typealias.type); if (auto ttv = getMutable(follow(ty))) { // If the table is already named and we want to rename the type function, we have to bind new alias to a copy @@ -1325,25 +1291,12 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatDeclareClass& declar LUAU_ASSERT(lookupType->typeParams.size() == 0 && (!FFlag::LuauTypeAliasPacks || lookupType->typePackParams.size() == 0)); superTy = lookupType->type; - if (FFlag::LuauAddMissingFollow) + if (!get(follow(*superTy))) { - if (!get(follow(*superTy))) - { - reportError(declaredClass.location, GenericError{format("Cannot use non-class type '%s' as a superclass of class '%s'", - superName.c_str(), declaredClass.name.value)}); + reportError(declaredClass.location, + GenericError{format("Cannot use non-class type '%s' as a superclass of class '%s'", superName.c_str(), declaredClass.name.value)}); - return; - } - } - else - { - if (const ClassTypeVar* superCtv = get(*superTy); !superCtv) - { - reportError(declaredClass.location, GenericError{format("Cannot use non-class type '%s' as a superclass of class '%s'", - superName.c_str(), declaredClass.name.value)}); - - return; - } + return; } } @@ -1558,8 +1511,8 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprVa } else if (auto ftp = get(varargPack)) { - TypeId head = (FFlag::LuauRankNTypes ? freshType(scope) : DEPRECATED_freshType(scope, ftp->DEPRECATED_canBeGeneric)); - TypePackId tail = (FFlag::LuauRankNTypes ? freshTypePack(scope) : DEPRECATED_freshTypePack(scope, ftp->DEPRECATED_canBeGeneric)); + TypeId head = freshType(scope); + TypePackId tail = freshTypePack(scope); *asMutable(varargPack) = TypePack{{head}, tail}; return {head}; } @@ -1567,7 +1520,7 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprVa return {errorType}; else if (auto vtp = get(varargPack)) return {vtp->ty}; - else if (FFlag::LuauGenericVariadicsUnification && get(varargPack)) + else if (get(varargPack)) { // TODO: Better error? reportError(expr.location, GenericError{"Trying to get a type from a variadic type parameter"}); @@ -1588,7 +1541,7 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprCa } else if (auto ftp = get(retPack)) { - TypeId head = (FFlag::LuauRankNTypes ? freshType(scope) : DEPRECATED_freshType(scope, ftp->DEPRECATED_canBeGeneric)); + TypeId head = freshType(scope); TypePackId pack = addTypePack(TypePackVar{TypePack{{head}, freshTypePack(scope)}}); unify(retPack, pack, expr.location); return {head, std::move(result.predicates)}; @@ -1667,7 +1620,7 @@ std::optional TypeChecker::getIndexTypeFromType( } else if (tableType->state == TableState::Free) { - TypeId result = (FFlag::LuauRankNTypes ? freshType(scope) : DEPRECATED_freshType(scope, true)); + TypeId result = freshType(scope); tableType->props[name] = {result}; return result; } @@ -2158,16 +2111,9 @@ TypeId TypeChecker::checkRelationalOperation( { if (expr.op == AstExprBinary::Or && subexp->op == AstExprBinary::And) { - if (FFlag::LuauSlightlyMoreFlexibleBinaryPredicates) - { - ScopePtr subScope = childScope(scope, subexp->location); - reportErrors(resolve(predicates, subScope, true)); - return unionOfTypes(rhsType, stripNil(checkExpr(subScope, *subexp->right).type, true), expr.location); - } - else - { - return unionOfTypes(rhsType, checkExpr(scope, *subexp->right).type, expr.location); - } + ScopePtr subScope = childScope(scope, subexp->location); + reportErrors(resolve(predicates, subScope, true)); + return unionOfTypes(rhsType, stripNil(checkExpr(subScope, *subexp->right).type, true), expr.location); } } @@ -2217,10 +2163,8 @@ TypeId TypeChecker::checkRelationalOperation( std::string metamethodName = opToMetaTableEntry(expr.op); - std::optional leftMetatable = - isString(lhsType) ? std::nullopt : getMetatable(FFlag::LuauAddMissingFollow ? follow(lhsType) : lhsType); - std::optional rightMetatable = - isString(rhsType) ? std::nullopt : getMetatable(FFlag::LuauAddMissingFollow ? follow(rhsType) : rhsType); + std::optional leftMetatable = isString(lhsType) ? std::nullopt : getMetatable(follow(lhsType)); + std::optional rightMetatable = isString(rhsType) ? std::nullopt : getMetatable(follow(rhsType)); if (bool(leftMetatable) != bool(rightMetatable) && leftMetatable != rightMetatable) { @@ -2266,7 +2210,7 @@ TypeId TypeChecker::checkRelationalOperation( } } - if (get(FFlag::LuauAddMissingFollow ? follow(lhsType) : lhsType) && !isEquality) + if (get(follow(lhsType)) && !isEquality) { auto name = getIdentifierOfBaseVar(expr.left); reportError(expr.location, CannotInferBinaryOperation{expr.op, name, CannotInferBinaryOperation::Comparison}); @@ -2417,12 +2361,10 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprBi resolve(lhs.predicates, innerScope, true); ExprResult rhs = checkExpr(innerScope, *expr.right); - if (!FFlag::LuauSlightlyMoreFlexibleBinaryPredicates) - resolve(rhs.predicates, innerScope, true); return {checkBinaryOperation(innerScope, expr, lhs.type, rhs.type), {AndPredicate{std::move(lhs.predicates), std::move(rhs.predicates)}}}; } - else if (FFlag::LuauOrPredicate && expr.op == AstExprBinary::Or) + else if (expr.op == AstExprBinary::Or) { ExprResult lhs = checkExpr(scope, *expr.left); @@ -2468,19 +2410,8 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprBi ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprTypeAssertion& expr) { - ExprResult result; - TypeId annotationType; - - if (FFlag::LuauInferReturnAssertAssign) - { - annotationType = (FFlag::LuauRankNTypes ? resolveType(scope, *expr.annotation) : resolveType(scope, *expr.annotation, true)); - result = checkExpr(scope, *expr.expr, annotationType); - } - else - { - result = checkExpr(scope, *expr.expr); - annotationType = (FFlag::LuauRankNTypes ? resolveType(scope, *expr.annotation) : resolveType(scope, *expr.annotation, true)); - } + TypeId annotationType = resolveType(scope, *expr.annotation); + ExprResult result = checkExpr(scope, *expr.expr, annotationType); ErrorVec errorVec = canUnify(result.type, annotationType, expr.location); if (!errorVec.empty()) @@ -2570,23 +2501,16 @@ std::pair TypeChecker::checkLValueBinding(const ScopePtr& scope if (it != moduleScope->bindings.end()) return std::pair(it->second.typeId, &it->second.typeId); - if (isNonstrictMode() || FFlag::LuauSecondTypecheckKnowsTheDataModel) - { - TypeId result = (FFlag::LuauGenericFunctions && FFlag::LuauRankNTypes ? freshType(scope) : DEPRECATED_freshType(moduleScope, true)); + TypeId result = freshType(scope); + Binding& binding = moduleScope->bindings[expr.name]; + binding = {result, expr.location}; - Binding& binding = moduleScope->bindings[expr.name]; - binding = {result, expr.location}; + // If we're in strict mode, we want to report defining a global as an error, + // but still add it to the bindings, so that autocomplete includes it in completions. + if (!isNonstrictMode()) + reportError(TypeError{expr.location, UnknownSymbol{name, UnknownSymbol::Binding}}); - // If we're in strict mode, we want to report defining a global as an error, - // but still add it to the bindings, so that autocomplete includes it in completions. - if (!isNonstrictMode()) - reportError(TypeError{expr.location, UnknownSymbol{name, UnknownSymbol::Binding}}); - - return std::pair(result, &binding.typeId); - } - - reportError(TypeError{expr.location, UnknownSymbol{name, UnknownSymbol::Binding}}); - return std::pair(errorType, nullptr); + return std::pair(result, &binding.typeId); } std::pair TypeChecker::checkLValueBinding(const ScopePtr& scope, const AstExprIndexName& expr) @@ -2611,7 +2535,7 @@ std::pair TypeChecker::checkLValueBinding(const ScopePtr& scope } else if (lhsTable->state == TableState::Unsealed || lhsTable->state == TableState::Free) { - TypeId theType = (FFlag::LuauRankNTypes ? freshType(scope) : DEPRECATED_freshType(scope, true)); + TypeId theType = freshType(scope); Property& property = lhsTable->props[name]; property.type = theType; property.location = expr.indexLocation; @@ -2683,7 +2607,7 @@ std::pair TypeChecker::checkLValueBinding(const ScopePtr& scope AstExprConstantString* value = expr.index->as(); - if (value && FFlag::LuauClassPropertyAccessAsString) + if (value) { if (const ClassTypeVar* exprClass = get(exprType)) { @@ -2714,7 +2638,7 @@ std::pair TypeChecker::checkLValueBinding(const ScopePtr& scope } else if (exprTable->state == TableState::Unsealed || exprTable->state == TableState::Free) { - TypeId resultType = (FFlag::LuauRankNTypes ? freshType(scope) : DEPRECATED_freshType(scope, true)); + TypeId resultType = freshType(scope); Property& property = exprTable->props[value->value.data]; property.type = resultType; property.location = expr.index->location; @@ -2730,7 +2654,7 @@ std::pair TypeChecker::checkLValueBinding(const ScopePtr& scope } else if (exprTable->state == TableState::Unsealed || exprTable->state == TableState::Free) { - TypeId resultType = (FFlag::LuauRankNTypes ? freshType(scope) : DEPRECATED_freshType(scope, true)); + TypeId resultType = freshType(scope); exprTable->indexer = TableIndexer{anyIfNonstrict(indexType), anyIfNonstrict(resultType)}; return std::pair(resultType, nullptr); } @@ -2758,7 +2682,7 @@ TypeId TypeChecker::checkFunctionName(const ScopePtr& scope, AstExpr& funName) } else { - TypeId ty = (FFlag::LuauRankNTypes ? freshType(scope) : DEPRECATED_freshType(scope, true)); + TypeId ty = freshType(scope); globalScope->bindings[name] = {ty, funName.location}; return ty; } @@ -2768,7 +2692,7 @@ TypeId TypeChecker::checkFunctionName(const ScopePtr& scope, AstExpr& funName) Symbol name = localName->local; Binding& binding = scope->bindings[name]; if (binding.typeId == nullptr) - binding = {(FFlag::LuauRankNTypes ? freshType(scope) : DEPRECATED_freshType(scope, true)), funName.location}; + binding = {freshType(scope), funName.location}; return binding.typeId; } @@ -2798,7 +2722,7 @@ TypeId TypeChecker::checkFunctionName(const ScopePtr& scope, AstExpr& funName) Property& property = ttv->props[name]; - property.type = (FFlag::LuauRankNTypes ? freshType(scope) : DEPRECATED_freshType(scope, true)); + property.type = freshType(scope); property.location = indexName->indexLocation; ttv->methodDefinitionLocations[name] = funName.location; return property.type; @@ -2865,22 +2789,11 @@ std::pair TypeChecker::checkFunctionSignature( expectedFunctionType = nullptr; } - std::vector generics; - std::vector genericPacks; - - if (FFlag::LuauGenericFunctions) - { - std::tie(generics, genericPacks) = createGenericTypes(funScope, std::nullopt, expr, expr.generics, expr.genericPacks); - } + auto [generics, genericPacks] = createGenericTypes(funScope, std::nullopt, expr, expr.generics, expr.genericPacks); TypePackId retPack; if (expr.hasReturnAnnotation) - { - if (FFlag::LuauGenericFunctions) - retPack = resolveTypePack(funScope, expr.returnAnnotation); - else - retPack = resolveTypePack(scope, expr.returnAnnotation); - } + retPack = resolveTypePack(funScope, expr.returnAnnotation); else if (isNonstrictMode()) retPack = anyTypePack; else if (expectedFunctionType) @@ -2889,24 +2802,17 @@ std::pair TypeChecker::checkFunctionSignature( // Do not infer 'nil' as function return type if (!tail && head.size() == 1 && isNil(head[0])) - retPack = FFlag::LuauGenericFunctions ? freshTypePack(funScope) : freshTypePack(scope); + retPack = freshTypePack(funScope); else retPack = addTypePack(head, tail); } - else if (FFlag::LuauGenericFunctions) - retPack = freshTypePack(funScope); else - retPack = freshTypePack(scope); + retPack = freshTypePack(funScope); if (expr.vararg) { if (expr.varargAnnotation) - { - if (FFlag::LuauGenericFunctions) - funScope->varargPack = resolveTypePack(funScope, *expr.varargAnnotation); - else - funScope->varargPack = resolveTypePack(scope, *expr.varargAnnotation); - } + funScope->varargPack = resolveTypePack(funScope, *expr.varargAnnotation); else { if (expectedFunctionType && !isNonstrictMode()) @@ -2963,7 +2869,7 @@ std::pair TypeChecker::checkFunctionSignature( if (local->annotation) { - argType = resolveType((FFlag::LuauGenericFunctions ? funScope : scope), *local->annotation); + argType = resolveType(funScope, *local->annotation); // If the annotation type has an error, treat it as if there was no annotation if (get(follow(argType))) @@ -3022,7 +2928,7 @@ static bool allowsNoReturnValues(const TypePackId tp) { for (TypeId ty : tp) { - if (!get(FFlag::LuauAddMissingFollow ? follow(ty) : ty)) + if (!get(follow(ty))) { return false; } @@ -3058,7 +2964,7 @@ void TypeChecker::checkFunctionBody(const ScopePtr& scope, TypeId ty, const AstE check(scope, *function.body); // We explicitly don't follow here to check if we have a 'true' free type instead of bound one - if (FFlag::LuauAddMissingFollow ? get_if(&funTy->retType->ty) : get(funTy->retType)) + if (get_if(&funTy->retType->ty)) *asMutable(funTy->retType) = TypePack{{}, std::nullopt}; bool reachesImplicitReturn = getFallthrough(function.body) != nullptr; @@ -3287,7 +3193,7 @@ void TypeChecker::checkArgumentList( return; } - else if (FFlag::LuauGenericVariadicsUnification && get(tail)) + else if (get(tail)) { // Create a type pack out of the remaining argument types // and unify it with the tail. @@ -3310,7 +3216,7 @@ void TypeChecker::checkArgumentList( return; } - else if (FFlag::LuauRankNTypes && get(tail)) + else if (get(tail)) { // For this case, we want the error span to cover every errant extra parameter Location location = state.location; @@ -3323,10 +3229,7 @@ void TypeChecker::checkArgumentList( } else { - if (FFlag::LuauRankNTypes) - unifyWithInstantiationIfNeeded(scope, *paramIter, *argIter, state); - else - state.tryUnify(*paramIter, *argIter, /*isFunctionCall*/ false); + unifyWithInstantiationIfNeeded(scope, *paramIter, *argIter, state); ++argIter; ++paramIter; } @@ -3356,9 +3259,6 @@ ExprResult TypeChecker::checkExprPack(const ScopePtr& scope, const A ice("method call expression has no 'self'"); selfType = checkExpr(scope, *indexExpr->expr).type; - if (!FFlag::LuauRankNTypes) - instantiate(scope, selfType, expr.func->location); - selfType = stripFromNilAndReport(selfType, expr.func->location); if (std::optional propTy = getIndexTypeFromType(scope, selfType, indexExpr->index.value, expr.location, true)) @@ -3393,8 +3293,7 @@ ExprResult TypeChecker::checkExprPack(const ScopePtr& scope, const A std::vector> expectedTypes = getExpectedTypesForCall(overloads, expr.args.size, expr.self); ExprResult argListResult = checkExprList(scope, expr.location, expr.args, false, {}, expectedTypes); - TypePackId argList = argListResult.type; - TypePackId argPack = (FFlag::LuauRankNTypes ? argList : DEPRECATED_instantiate(scope, argList, expr.location)); + TypePackId argPack = argListResult.type; if (get(argPack)) return ExprResult{errorTypePack}; @@ -3526,8 +3425,7 @@ std::optional> TypeChecker::checkCallOverload(const Scope metaArgLocations.insert(metaArgLocations.begin(), expr.func->location); TypeId fn = *ty; - if (FFlag::LuauRankNTypes) - fn = instantiate(scope, fn, expr.func->location); + fn = instantiate(scope, fn, expr.func->location); return checkCallOverload( scope, expr, fn, retPack, metaCallArgPack, metaCallArgs, metaArgLocations, argListResult, overloadsThatMatchArgCount, errors); @@ -3800,7 +3698,7 @@ ExprResult TypeChecker::checkExprList(const ScopePtr& scope, const L TypeId actualType = substituteFreeForNil && expr->is() ? freshType(scope) : type; - if (instantiateGenerics.size() > i && instantiateGenerics[i] && (FFlag::LuauGenericFunctions || get(actualType))) + if (instantiateGenerics.size() > i && instantiateGenerics[i]) actualType = instantiate(scope, actualType, expr->location); if (expectedType) @@ -3837,7 +3735,7 @@ TypeId TypeChecker::checkRequire(const ScopePtr& scope, const ModuleInfo& module LUAU_TIMETRACE_SCOPE("TypeChecker::checkRequire", "TypeChecker"); LUAU_TIMETRACE_ARGUMENT("moduleInfo", moduleInfo.name.c_str()); - if (FFlag::LuauNewRequireTrace && moduleInfo.name.empty()) + if (FFlag::LuauNewRequireTrace2 && moduleInfo.name.empty()) { if (FFlag::LuauStrictRequire && currentModule->mode == Mode::Strict) { @@ -3922,7 +3820,6 @@ bool TypeChecker::unify(TypePackId left, TypePackId right, const Location& locat bool TypeChecker::unifyWithInstantiationIfNeeded(const ScopePtr& scope, TypeId left, TypeId right, const Location& location) { - LUAU_ASSERT(FFlag::LuauRankNTypes); Unifier state = mkUnifier(location); unifyWithInstantiationIfNeeded(scope, left, right, state); @@ -3933,7 +3830,6 @@ bool TypeChecker::unifyWithInstantiationIfNeeded(const ScopePtr& scope, TypeId l void TypeChecker::unifyWithInstantiationIfNeeded(const ScopePtr& scope, TypeId left, TypeId right, Unifier& state) { - LUAU_ASSERT(FFlag::LuauRankNTypes); if (!maybeGeneric(right)) // Quick check to see if we definitely can't instantiate state.tryUnify(left, right, /*isFunctionCall*/ false); @@ -3973,19 +3869,7 @@ void TypeChecker::unifyWithInstantiationIfNeeded(const ScopePtr& scope, TypeId l bool Instantiation::isDirty(TypeId ty) { - if (FFlag::LuauRankNTypes) - { - if (get(ty)) - return true; - else - return false; - } - - if (const FunctionTypeVar* ftv = get(ty)) - return !ftv->generics.empty() || !ftv->genericPacks.empty(); - else if (const TableTypeVar* ttv = get(ty)) - return ttv->state == TableState::Generic; - else if (get(ty)) + if (get(ty)) return true; else return false; @@ -3993,18 +3877,11 @@ bool Instantiation::isDirty(TypeId ty) bool Instantiation::isDirty(TypePackId tp) { - if (FFlag::LuauRankNTypes) - return false; - - if (get(tp)) - return true; - else - return false; + return false; } bool Instantiation::ignoreChildren(TypeId ty) { - LUAU_ASSERT(FFlag::LuauRankNTypes); if (get(ty)) return true; else @@ -4013,63 +3890,38 @@ bool Instantiation::ignoreChildren(TypeId ty) TypeId Instantiation::clean(TypeId ty) { - LUAU_ASSERT(isDirty(ty)); + const FunctionTypeVar* ftv = get(ty); + LUAU_ASSERT(ftv); - if (const FunctionTypeVar* ftv = get(ty)) - { - FunctionTypeVar clone = FunctionTypeVar{level, ftv->argTypes, ftv->retType, ftv->definition, ftv->hasSelf}; - clone.magicFunction = ftv->magicFunction; - clone.tags = ftv->tags; - clone.argNames = ftv->argNames; - TypeId result = addType(std::move(clone)); + FunctionTypeVar clone = FunctionTypeVar{level, ftv->argTypes, ftv->retType, ftv->definition, ftv->hasSelf}; + clone.magicFunction = ftv->magicFunction; + clone.tags = ftv->tags; + clone.argNames = ftv->argNames; + TypeId result = addType(std::move(clone)); - if (FFlag::LuauRankNTypes) - { - // Annoyingly, we have to do this even if there are no generics, - // to replace any generic tables. - replaceGenerics.level = level; - replaceGenerics.currentModule = currentModule; - replaceGenerics.generics.assign(ftv->generics.begin(), ftv->generics.end()); - replaceGenerics.genericPacks.assign(ftv->genericPacks.begin(), ftv->genericPacks.end()); + // Annoyingly, we have to do this even if there are no generics, + // to replace any generic tables. + replaceGenerics.level = level; + replaceGenerics.currentModule = currentModule; + replaceGenerics.generics.assign(ftv->generics.begin(), ftv->generics.end()); + replaceGenerics.genericPacks.assign(ftv->genericPacks.begin(), ftv->genericPacks.end()); - // TODO: What to do if this returns nullopt? - // We don't have access to the error-reporting machinery - result = replaceGenerics.substitute(result).value_or(result); - } + // TODO: What to do if this returns nullopt? + // We don't have access to the error-reporting machinery + result = replaceGenerics.substitute(result).value_or(result); - asMutable(result)->documentationSymbol = ty->documentationSymbol; - return result; - } - else if (const TableTypeVar* ttv = get(ty)) - { - LUAU_ASSERT(!FFlag::LuauRankNTypes); - TableTypeVar clone = TableTypeVar{ttv->props, ttv->indexer, level, TableState::Free}; - clone.methodDefinitionLocations = ttv->methodDefinitionLocations; - clone.definitionModuleName = ttv->definitionModuleName; - TypeId result = addType(std::move(clone)); - - asMutable(result)->documentationSymbol = ty->documentationSymbol; - return result; - } - else - { - LUAU_ASSERT(!FFlag::LuauRankNTypes); - TypeId result = addType(FreeTypeVar{level}); - - asMutable(result)->documentationSymbol = ty->documentationSymbol; - return result; - } + asMutable(result)->documentationSymbol = ty->documentationSymbol; + return result; } TypePackId Instantiation::clean(TypePackId tp) { - LUAU_ASSERT(!FFlag::LuauRankNTypes); - return addTypePack(TypePackVar(FreeTypePack{level})); + LUAU_ASSERT(false); + return tp; } bool ReplaceGenerics::ignoreChildren(TypeId ty) { - LUAU_ASSERT(FFlag::LuauRankNTypes); if (const FunctionTypeVar* ftv = get(ty)) // We aren't recursing in the case of a generic function which // binds the same generics. This can happen if, for example, there's recursive types. @@ -4083,7 +3935,6 @@ bool ReplaceGenerics::ignoreChildren(TypeId ty) bool ReplaceGenerics::isDirty(TypeId ty) { - LUAU_ASSERT(FFlag::LuauRankNTypes); if (const TableTypeVar* ttv = get(ty)) return ttv->state == TableState::Generic; else if (get(ty)) @@ -4094,7 +3945,6 @@ bool ReplaceGenerics::isDirty(TypeId ty) bool ReplaceGenerics::isDirty(TypePackId tp) { - LUAU_ASSERT(FFlag::LuauRankNTypes); if (get(tp)) return std::find(genericPacks.begin(), genericPacks.end(), tp) != genericPacks.end(); else @@ -4255,21 +4105,6 @@ TypeId TypeChecker::instantiate(const ScopePtr& scope, TypeId ty, Location locat } } -TypePackId TypeChecker::DEPRECATED_instantiate(const ScopePtr& scope, TypePackId ty, Location location) -{ - LUAU_ASSERT(!FFlag::LuauRankNTypes); - instantiation.level = scope->level; - instantiation.currentModule = currentModule; - std::optional instantiated = instantiation.substitute(ty); - if (instantiated.has_value()) - return *instantiated; - else - { - reportError(location, UnificationTooComplex{}); - return errorTypePack; - } -} - TypeId TypeChecker::anyify(const ScopePtr& scope, TypeId ty, Location location) { anyification.anyType = anyType; @@ -4444,16 +4279,6 @@ TypeId TypeChecker::freshType(TypeLevel level) return currentModule->internalTypes.addType(TypeVar(FreeTypeVar(level))); } -TypeId TypeChecker::DEPRECATED_freshType(const ScopePtr& scope, bool canBeGeneric) -{ - return DEPRECATED_freshType(scope->level, canBeGeneric); -} - -TypeId TypeChecker::DEPRECATED_freshType(TypeLevel level, bool canBeGeneric) -{ - return currentModule->internalTypes.addType(TypeVar(FreeTypeVar(level, canBeGeneric))); -} - std::optional TypeChecker::filterMap(TypeId type, TypeIdPredicate predicate) { std::vector types = Luau::filterMap(type, predicate); @@ -4509,21 +4334,8 @@ TypePackId TypeChecker::freshTypePack(TypeLevel level) return addTypePack(TypePackVar(FreeTypePack(level))); } -TypePackId TypeChecker::DEPRECATED_freshTypePack(const ScopePtr& scope, bool canBeGeneric) +TypeId TypeChecker::resolveType(const ScopePtr& scope, const AstType& annotation) { - return DEPRECATED_freshTypePack(scope->level, canBeGeneric); -} - -TypePackId TypeChecker::DEPRECATED_freshTypePack(TypeLevel level, bool canBeGeneric) -{ - return addTypePack(TypePackVar(FreeTypePack(level, canBeGeneric))); -} - -TypeId TypeChecker::resolveType(const ScopePtr& scope, const AstType& annotation, bool DEPRECATED_canBeGeneric) -{ - if (DEPRECATED_canBeGeneric) - LUAU_ASSERT(!FFlag::LuauRankNTypes); - if (const auto& lit = annotation.as()) { std::optional tf; @@ -4668,11 +4480,11 @@ TypeId TypeChecker::resolveType(const ScopePtr& scope, const AstType& annotation std::optional tableIndexer; for (const auto& prop : table->props) - props[prop.name.value] = {resolveType(scope, *prop.type, DEPRECATED_canBeGeneric)}; + props[prop.name.value] = {resolveType(scope, *prop.type)}; if (const auto& indexer = table->indexer) tableIndexer = TableIndexer( - resolveType(scope, *indexer->indexType, DEPRECATED_canBeGeneric), resolveType(scope, *indexer->resultType, DEPRECATED_canBeGeneric)); + resolveType(scope, *indexer->indexType), resolveType(scope, *indexer->resultType)); return addType(TableTypeVar{ props, tableIndexer, scope->level, @@ -4683,17 +4495,7 @@ TypeId TypeChecker::resolveType(const ScopePtr& scope, const AstType& annotation { ScopePtr funcScope = childScope(scope, func->location); - std::vector generics; - std::vector genericPacks; - - if (FFlag::LuauGenericFunctions) - { - std::tie(generics, genericPacks) = createGenericTypes(funcScope, std::nullopt, annotation, func->generics, func->genericPacks); - } - - // TODO: better error message CLI-39912 - if (FFlag::LuauGenericFunctions && !FFlag::LuauRankNTypes && !DEPRECATED_canBeGeneric && (generics.size() > 0 || genericPacks.size() > 0)) - reportError(TypeError{annotation.location, GenericError{"generic function where only monotypes are allowed"}}); + auto [generics, genericPacks] = createGenericTypes(funcScope, std::nullopt, annotation, func->generics, func->genericPacks); TypePackId argTypes = resolveTypePack(funcScope, func->argTypes); TypePackId retTypes = resolveTypePack(funcScope, func->returnTypes); @@ -4716,16 +4518,13 @@ TypeId TypeChecker::resolveType(const ScopePtr& scope, const AstType& annotation else if (auto typeOf = annotation.as()) { TypeId ty = checkExpr(scope, *typeOf->expr).type; - // TODO: better error message CLI-39912 - if (FFlag::LuauGenericFunctions && !FFlag::LuauRankNTypes && !DEPRECATED_canBeGeneric && isGeneric(ty)) - reportError(TypeError{annotation.location, GenericError{"typeof produced a polytype where only monotypes are allowed"}}); return ty; } else if (const auto& un = annotation.as()) { std::vector types; for (AstType* ann : un->types) - types.push_back(resolveType(scope, *ann, DEPRECATED_canBeGeneric)); + types.push_back(resolveType(scope, *ann)); return addType(UnionTypeVar{types}); } @@ -4733,7 +4532,7 @@ TypeId TypeChecker::resolveType(const ScopePtr& scope, const AstType& annotation { std::vector types; for (AstType* ann : un->types) - types.push_back(resolveType(scope, *ann, DEPRECATED_canBeGeneric)); + types.push_back(resolveType(scope, *ann)); return addType(IntersectionTypeVar{types}); } @@ -4919,9 +4718,8 @@ TypeId TypeChecker::instantiateTypeFun(const ScopePtr& scope, const TypeFun& tf, if (FFlag::LuauCloneCorrectlyBeforeMutatingTableType) { - // TODO: CLI-46926 it's a bad idea to rename the type whether we follow through the BoundTypeVar or not - TypeId target = FFlag::LuauFollowInTypeFunApply ? follow(instantiated) : instantiated; - + // TODO: CLI-46926 it's not a good idea to rename the type here + TypeId target = follow(instantiated); bool needsClone = follow(tf.type) == target; TableTypeVar* ttv = getMutableTableType(target); @@ -5152,31 +4950,18 @@ void TypeChecker::resolve(const TruthyPredicate& truthyP, ErrorVec& errVec, Refi void TypeChecker::resolve(const AndPredicate& andP, ErrorVec& errVec, RefinementMap& refis, const ScopePtr& scope, bool sense) { - if (FFlag::LuauOrPredicate) + if (!sense) { - if (!sense) - { - OrPredicate orP{ - {NotPredicate{std::move(andP.lhs)}}, - {NotPredicate{std::move(andP.rhs)}}, - }; + OrPredicate orP{ + {NotPredicate{std::move(andP.lhs)}}, + {NotPredicate{std::move(andP.rhs)}}, + }; - return resolve(orP, errVec, refis, scope, !sense); - } - - resolve(andP.lhs, errVec, refis, scope, sense); - resolve(andP.rhs, errVec, refis, scope, sense); + return resolve(orP, errVec, refis, scope, !sense); } - else - { - // And predicate is currently not resolvable when sense is false. 'not (a and b)' is synonymous with '(not a) or (not b)'. - // TODO: implement environment merging to permit this case. - if (!sense) - return; - resolve(andP.lhs, errVec, refis, scope, sense); - resolve(andP.rhs, errVec, refis, scope, sense); - } + resolve(andP.lhs, errVec, refis, scope, sense); + resolve(andP.rhs, errVec, refis, scope, sense); } void TypeChecker::resolve(const OrPredicate& orP, ErrorVec& errVec, RefinementMap& refis, const ScopePtr& scope, bool sense) @@ -5207,62 +4992,47 @@ void TypeChecker::resolve(const OrPredicate& orP, ErrorVec& errVec, RefinementMa void TypeChecker::resolve(const IsAPredicate& isaP, ErrorVec& errVec, RefinementMap& refis, const ScopePtr& scope, bool sense) { auto predicate = [&](TypeId option) -> std::optional { - if (FFlag::LuauTypeGuardPeelsAwaySubclasses) - { - // This by itself is not truly enough to determine that A is stronger than B or vice versa. - // The best unambiguous way about this would be to have a function that returns the relationship ordering of a pair. - // i.e. TypeRelationship relationshipOf(TypeId superTy, TypeId subTy) - bool optionIsSubtype = canUnify(isaP.ty, option, isaP.location).empty(); - bool targetIsSubtype = canUnify(option, isaP.ty, isaP.location).empty(); + // This by itself is not truly enough to determine that A is stronger than B or vice versa. + // The best unambiguous way about this would be to have a function that returns the relationship ordering of a pair. + // i.e. TypeRelationship relationshipOf(TypeId superTy, TypeId subTy) + bool optionIsSubtype = canUnify(isaP.ty, option, isaP.location).empty(); + bool targetIsSubtype = canUnify(option, isaP.ty, isaP.location).empty(); - // If A is a superset of B, then if sense is true, we promote A to B, otherwise we keep A. - if (!optionIsSubtype && targetIsSubtype) + // If A is a superset of B, then if sense is true, we promote A to B, otherwise we keep A. + if (!optionIsSubtype && targetIsSubtype) + return sense ? isaP.ty : option; + + // If A is a subset of B, then if sense is true we pick A, otherwise we eliminate A. + if (optionIsSubtype && !targetIsSubtype) + return sense ? std::optional(option) : std::nullopt; + + // If neither has any relationship, we only return A if sense is false. + if (!optionIsSubtype && !targetIsSubtype) + return sense ? std::nullopt : std::optional(option); + + // If both are subtypes, then we're in one of the two situations: + // 1. Instance₁ <: Instance₂ ∧ Instance₂ <: Instance₁ + // 2. any <: Instance ∧ Instance <: any + // Right now, we have to look at the types to see if they were undecidables. + // By this point, we also know free tables are also subtypes and supertypes. + if (optionIsSubtype && targetIsSubtype) + { + // We can only have (any, Instance) because the rhs is never undecidable right now. + // So we can just return the right hand side immediately. + + // typeof(x) == "Instance" where x : any + auto ttv = get(option); + if (isUndecidable(option) || (ttv && ttv->state == TableState::Free)) return sense ? isaP.ty : option; - // If A is a subset of B, then if sense is true we pick A, otherwise we eliminate A. - if (optionIsSubtype && !targetIsSubtype) - return sense ? std::optional(option) : std::nullopt; - - // If neither has any relationship, we only return A if sense is false. - if (!optionIsSubtype && !targetIsSubtype) - return sense ? std::nullopt : std::optional(option); - - // If both are subtypes, then we're in one of the two situations: - // 1. Instance₁ <: Instance₂ ∧ Instance₂ <: Instance₁ - // 2. any <: Instance ∧ Instance <: any - // Right now, we have to look at the types to see if they were undecidables. - // By this point, we also know free tables are also subtypes and supertypes. - if (optionIsSubtype && targetIsSubtype) - { - // We can only have (any, Instance) because the rhs is never undecidable right now. - // So we can just return the right hand side immediately. - - // typeof(x) == "Instance" where x : any - auto ttv = get(option); - if (isUndecidable(option) || (ttv && ttv->state == TableState::Free)) - return sense ? isaP.ty : option; - - // typeof(x) == "Instance" where x : Instance - if (sense) - return isaP.ty; - } - } - else - { - auto lctv = get(option); - auto rctv = get(isaP.ty); - - if (isSubclass(lctv, rctv) == sense) - return option; - - if (isSubclass(rctv, lctv) == sense) - return isaP.ty; - - if (canUnify(option, isaP.ty, isaP.location).empty() == sense) + // typeof(x) == "Instance" where x : Instance + if (sense) return isaP.ty; } - return std::nullopt; + // local variable works around an odd gcc 9.3 warning: may be used uninitialized + std::optional res = std::nullopt; + return res; }; std::optional ty = resolveLValue(refis, scope, isaP.lvalue); @@ -5457,7 +5227,7 @@ std::vector TypeChecker::unTypePack(const ScopePtr& scope, TypePackId tp TypePack* expectedPack = getMutable(expectedTypePack); LUAU_ASSERT(expectedPack); for (size_t i = 0; i < expectedLength; ++i) - expectedPack->head.push_back(FFlag::LuauRankNTypes ? freshType(scope) : DEPRECATED_freshType(scope, true)); + expectedPack->head.push_back(freshType(scope)); unify(expectedTypePack, tp, location); diff --git a/Analysis/src/TypePack.cpp b/Analysis/src/TypePack.cpp index 22257016..228b1926 100644 --- a/Analysis/src/TypePack.cpp +++ b/Analysis/src/TypePack.cpp @@ -97,7 +97,7 @@ TypePackIterator begin(TypePackId tp) TypePackIterator end(TypePackId tp) { - return FFlag::LuauAddMissingFollow ? TypePackIterator{} : TypePackIterator{nullptr}; + return TypePackIterator{}; } bool areEqual(SeenSet& seen, const TypePackVar& lhs, const TypePackVar& rhs) @@ -203,7 +203,7 @@ TypePackId follow(TypePackId tp) size_t size(TypePackId tp) { - if (auto pack = get(FFlag::LuauAddMissingFollow ? follow(tp) : tp)) + if (auto pack = get(follow(tp))) return size(*pack); else return 0; @@ -227,7 +227,7 @@ size_t size(const TypePack& tp) size_t result = tp.head.size(); if (tp.tail) { - const TypePack* tail = get(FFlag::LuauAddMissingFollow ? follow(*tp.tail) : *tp.tail); + const TypePack* tail = get(follow(*tp.tail)); if (tail) result += size(*tail); } diff --git a/Analysis/src/TypeVar.cpp b/Analysis/src/TypeVar.cpp index 631f58f9..cd447ca2 100644 --- a/Analysis/src/TypeVar.cpp +++ b/Analysis/src/TypeVar.cpp @@ -19,8 +19,6 @@ LUAU_FASTINTVARIABLE(LuauTypeMaximumStringifierLength, 500) LUAU_FASTINTVARIABLE(LuauTableTypeMaximumStringifierLength, 0) -LUAU_FASTFLAG(LuauRankNTypes) -LUAU_FASTFLAG(LuauTypeGuardPeelsAwaySubclasses) LUAU_FASTFLAG(LuauTypeAliasPacks) LUAU_FASTFLAGVARIABLE(LuauRefactorTagging, false) @@ -42,7 +40,7 @@ TypeId follow(TypeId t) }; auto force = [](TypeId ty) { - if (auto ltv = FFlag::LuauAddMissingFollow ? get_if(&ty->ty) : get(ty)) + if (auto ltv = get_if(&ty->ty)) { TypeId res = ltv->thunk(); if (get(res)) @@ -296,7 +294,7 @@ bool maybeGeneric(TypeId ty) { ty = follow(ty); if (auto ftv = get(ty)) - return FFlag::LuauRankNTypes || ftv->DEPRECATED_canBeGeneric; + return true; else if (auto ttv = get(ty)) { // TODO: recurse on table types CLI-39914 @@ -545,15 +543,30 @@ TypeId makeFunction(TypeArena& arena, std::optional selfType, std::initi std::initializer_list genericPacks, std::initializer_list paramTypes, std::initializer_list paramNames, std::initializer_list retTypes); +static TypeVar nilType_{PrimitiveTypeVar{PrimitiveTypeVar::NilType}, /*persistent*/ true}; +static TypeVar numberType_{PrimitiveTypeVar{PrimitiveTypeVar::Number}, /*persistent*/ true}; +static TypeVar stringType_{PrimitiveTypeVar{PrimitiveTypeVar::String}, /*persistent*/ true}; +static TypeVar booleanType_{PrimitiveTypeVar{PrimitiveTypeVar::Boolean}, /*persistent*/ true}; +static TypeVar threadType_{PrimitiveTypeVar{PrimitiveTypeVar::Thread}, /*persistent*/ true}; +static TypeVar anyType_{AnyTypeVar{}}; +static TypeVar errorType_{ErrorTypeVar{}}; +static TypeVar optionalNumberType_{UnionTypeVar{{&numberType_, &nilType_}}}; + +static TypePackVar anyTypePack_{VariadicTypePack{&anyType_}, true}; +static TypePackVar errorTypePack_{Unifiable::Error{}}; + SingletonTypes::SingletonTypes() - : arena(new TypeArena) - , nilType_{PrimitiveTypeVar{PrimitiveTypeVar::NilType}, /*persistent*/ true} - , numberType_{PrimitiveTypeVar{PrimitiveTypeVar::Number}, /*persistent*/ true} - , stringType_{PrimitiveTypeVar{PrimitiveTypeVar::String}, /*persistent*/ true} - , booleanType_{PrimitiveTypeVar{PrimitiveTypeVar::Boolean}, /*persistent*/ true} - , threadType_{PrimitiveTypeVar{PrimitiveTypeVar::Thread}, /*persistent*/ true} - , anyType_{AnyTypeVar{}} - , errorType_{ErrorTypeVar{}} + : nilType(&nilType_) + , numberType(&numberType_) + , stringType(&stringType_) + , booleanType(&booleanType_) + , threadType(&threadType_) + , anyType(&anyType_) + , errorType(&errorType_) + , optionalNumberType(&optionalNumberType_) + , anyTypePack(&anyTypePack_) + , errorTypePack(&errorTypePack_) + , arena(new TypeArena) { TypeId stringMetatable = makeStringMetatable(); stringType_.ty = PrimitiveTypeVar{PrimitiveTypeVar::String, makeStringMetatable()}; @@ -1372,24 +1385,6 @@ UnionTypeVarIterator end(const UnionTypeVar* utv) return UnionTypeVarIterator{}; } -static std::vector DEPRECATED_filterMap(TypeId type, TypeIdPredicate predicate) -{ - std::vector result; - - if (auto utv = get(follow(type))) - { - for (TypeId option : utv) - { - if (auto out = predicate(follow(option))) - result.push_back(*out); - } - } - else if (auto out = predicate(follow(type))) - return {*out}; - - return result; -} - static std::vector parseFormatString(TypeChecker& typechecker, const char* data, size_t size) { const char* options = "cdiouxXeEfgGqs"; @@ -1470,9 +1465,6 @@ std::optional> magicFunctionFormat( std::vector filterMap(TypeId type, TypeIdPredicate predicate) { - if (!FFlag::LuauTypeGuardPeelsAwaySubclasses) - return DEPRECATED_filterMap(type, predicate); - type = follow(type); if (auto utv = get(type)) diff --git a/Analysis/src/Unifiable.cpp b/Analysis/src/Unifiable.cpp index cef07833..dc554664 100644 --- a/Analysis/src/Unifiable.cpp +++ b/Analysis/src/Unifiable.cpp @@ -1,8 +1,6 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #include "Luau/Unifiable.h" -LUAU_FASTFLAG(LuauRankNTypes) - namespace Luau { namespace Unifiable @@ -14,14 +12,6 @@ Free::Free(TypeLevel level) { } -Free::Free(TypeLevel level, bool DEPRECATED_canBeGeneric) - : index(++nextIndex) - , level(level) - , DEPRECATED_canBeGeneric(DEPRECATED_canBeGeneric) -{ - LUAU_ASSERT(!FFlag::LuauRankNTypes); -} - int Free::nextIndex = 0; Generic::Generic() diff --git a/Analysis/src/Unifier.cpp b/Analysis/src/Unifier.cpp index 2539650a..82f621b6 100644 --- a/Analysis/src/Unifier.cpp +++ b/Analysis/src/Unifier.cpp @@ -14,17 +14,15 @@ LUAU_FASTINT(LuauTypeInferRecursionLimit); LUAU_FASTINT(LuauTypeInferTypePackLoopLimit); LUAU_FASTINTVARIABLE(LuauTypeInferIterationLimit, 2000); -LUAU_FASTFLAG(LuauGenericFunctions) LUAU_FASTFLAGVARIABLE(LuauTableSubtypingVariance, false); -LUAU_FASTFLAGVARIABLE(LuauDontMutatePersistentFunctions, false) -LUAU_FASTFLAG(LuauRankNTypes) LUAU_FASTFLAGVARIABLE(LuauUnionHeuristic, false) LUAU_FASTFLAGVARIABLE(LuauTableUnificationEarlyTest, false) -LUAU_FASTFLAGVARIABLE(LuauSealedTableUnifyOptionalFix, false) LUAU_FASTFLAGVARIABLE(LuauOccursCheckOkWithRecursiveFunctions, false) LUAU_FASTFLAGVARIABLE(LuauTypecheckOpts, false) LUAU_FASTFLAG(LuauShareTxnSeen); LUAU_FASTFLAGVARIABLE(LuauCacheUnifyTableResults, false) +LUAU_FASTFLAGVARIABLE(LuauExtendedTypeMismatchError, false) +LUAU_FASTFLAGVARIABLE(LuauExtendedClassMismatchError, false) namespace Luau { @@ -219,17 +217,12 @@ void Unifier::tryUnify_(TypeId superTy, TypeId subTy, bool isFunctionCall, bool *asMutable(subTy) = BoundTypeVar(superTy); } - if (!FFlag::LuauRankNTypes) - l->DEPRECATED_canBeGeneric &= r->DEPRECATED_canBeGeneric; - return; } - else if (l && r && FFlag::LuauGenericFunctions) + else if (l && r) { log(superTy); occursCheck(superTy, subTy); - if (!FFlag::LuauRankNTypes) - r->DEPRECATED_canBeGeneric &= l->DEPRECATED_canBeGeneric; r->level = min(r->level, l->level); *asMutable(superTy) = BoundTypeVar(subTy); return; @@ -240,7 +233,7 @@ void Unifier::tryUnify_(TypeId superTy, TypeId subTy, bool isFunctionCall, bool // Unification can't change the level of a generic. auto rightGeneric = get(subTy); - if (FFlag::LuauRankNTypes && rightGeneric && !rightGeneric->level.subsumes(l->level)) + if (rightGeneric && !rightGeneric->level.subsumes(l->level)) { // TODO: a more informative error message? CLI-39912 errors.push_back(TypeError{location, GenericError{"Generic subtype escaping scope"}}); @@ -266,31 +259,13 @@ void Unifier::tryUnify_(TypeId superTy, TypeId subTy, bool isFunctionCall, bool // Unification can't change the level of a generic. auto leftGeneric = get(superTy); - if (FFlag::LuauRankNTypes && leftGeneric && !leftGeneric->level.subsumes(r->level)) + if (leftGeneric && !leftGeneric->level.subsumes(r->level)) { // TODO: a more informative error message? CLI-39912 errors.push_back(TypeError{location, GenericError{"Generic supertype escaping scope"}}); return; } - // This is the old code which is just wrong - auto wrongGeneric = get(subTy); // Guaranteed to be null - if (!FFlag::LuauRankNTypes && FFlag::LuauGenericFunctions && wrongGeneric && r->level.subsumes(wrongGeneric->level)) - { - // This code is unreachable! Should we just remove it? - // TODO: a more informative error message? CLI-39912 - errors.push_back(TypeError{location, GenericError{"Generic supertype escaping scope"}}); - return; - } - - // Check if we're unifying a monotype with a polytype - if (FFlag::LuauGenericFunctions && !FFlag::LuauRankNTypes && !r->DEPRECATED_canBeGeneric && isGeneric(superTy)) - { - // TODO: a more informative error message? CLI-39912 - errors.push_back(TypeError{location, GenericError{"Failed to unify a polytype with a monotype"}}); - return; - } - if (!get(subTy)) { if (auto leftLevel = getMutableLevel(superTy)) @@ -333,6 +308,7 @@ void Unifier::tryUnify_(TypeId superTy, TypeId subTy, bool isFunctionCall, bool // A | B <: T if A <: T and B <: T bool failed = false; std::optional unificationTooComplex; + std::optional firstFailedOption; size_t count = uv->options.size(); size_t i = 0; @@ -345,7 +321,13 @@ void Unifier::tryUnify_(TypeId superTy, TypeId subTy, bool isFunctionCall, bool if (auto e = hasUnificationTooComplex(innerState.errors)) unificationTooComplex = e; else if (!innerState.errors.empty()) + { + // 'nil' option is skipped from extended report because we present the type in a special way - 'T?' + if (FFlag::LuauExtendedTypeMismatchError && !firstFailedOption && !isNil(type)) + firstFailedOption = {innerState.errors.front()}; + failed = true; + } if (i != count - 1) innerState.log.rollback(); @@ -358,7 +340,12 @@ void Unifier::tryUnify_(TypeId superTy, TypeId subTy, bool isFunctionCall, bool if (unificationTooComplex) errors.push_back(*unificationTooComplex); else if (failed) - errors.push_back(TypeError{location, TypeMismatch{superTy, subTy}}); + { + if (FFlag::LuauExtendedTypeMismatchError && firstFailedOption) + errors.push_back(TypeError{location, TypeMismatch{superTy, subTy, "Not all union options are compatible", *firstFailedOption}}); + else + errors.push_back(TypeError{location, TypeMismatch{superTy, subTy}}); + } } else if (const UnionTypeVar* uv = get(superTy)) { @@ -425,14 +412,49 @@ void Unifier::tryUnify_(TypeId superTy, TypeId subTy, bool isFunctionCall, bool if (unificationTooComplex) errors.push_back(*unificationTooComplex); else if (!found) - errors.push_back(TypeError{location, TypeMismatch{superTy, subTy}}); + { + if (FFlag::LuauExtendedTypeMismatchError) + errors.push_back(TypeError{location, TypeMismatch{superTy, subTy, "none of the union options are compatible"}}); + else + errors.push_back(TypeError{location, TypeMismatch{superTy, subTy}}); + } } else if (const IntersectionTypeVar* uv = get(superTy)) { - // T <: A & B if A <: T and B <: T - for (TypeId type : uv->parts) + if (FFlag::LuauExtendedTypeMismatchError) { - tryUnify_(type, subTy, /*isFunctionCall*/ false, /*isIntersection*/ true); + std::optional unificationTooComplex; + std::optional firstFailedOption; + + // T <: A & B if A <: T and B <: T + for (TypeId type : uv->parts) + { + Unifier innerState = makeChildUnifier(); + innerState.tryUnify_(type, subTy, /*isFunctionCall*/ false, /*isIntersection*/ true); + + if (auto e = hasUnificationTooComplex(innerState.errors)) + unificationTooComplex = e; + else if (!innerState.errors.empty()) + { + if (!firstFailedOption) + firstFailedOption = {innerState.errors.front()}; + } + + log.concat(std::move(innerState.log)); + } + + if (unificationTooComplex) + errors.push_back(*unificationTooComplex); + else if (firstFailedOption) + errors.push_back(TypeError{location, TypeMismatch{superTy, subTy, "Not all intersection parts are compatible", *firstFailedOption}}); + } + else + { + // T <: A & B if A <: T and B <: T + for (TypeId type : uv->parts) + { + tryUnify_(type, subTy, /*isFunctionCall*/ false, /*isIntersection*/ true); + } } } else if (const IntersectionTypeVar* uv = get(subTy)) @@ -480,7 +502,12 @@ void Unifier::tryUnify_(TypeId superTy, TypeId subTy, bool isFunctionCall, bool if (unificationTooComplex) errors.push_back(*unificationTooComplex); else if (!found) - errors.push_back(TypeError{location, TypeMismatch{superTy, subTy}}); + { + if (FFlag::LuauExtendedTypeMismatchError) + errors.push_back(TypeError{location, TypeMismatch{superTy, subTy, "none of the intersection parts are compatible"}}); + else + errors.push_back(TypeError{location, TypeMismatch{superTy, subTy}}); + } } else if (get(superTy) && get(subTy)) tryUnifyPrimitives(superTy, subTy); @@ -773,8 +800,8 @@ void Unifier::tryUnify_(TypePackId superTp, TypePackId subTp, bool isFunctionCal // If both are at the end, we're done if (!superIter.good() && !subIter.good()) { - const bool lFreeTail = l->tail && get(FFlag::LuauAddMissingFollow ? follow(*l->tail) : *l->tail) != nullptr; - const bool rFreeTail = r->tail && get(FFlag::LuauAddMissingFollow ? follow(*r->tail) : *r->tail) != nullptr; + const bool lFreeTail = l->tail && get(follow(*l->tail)) != nullptr; + const bool rFreeTail = r->tail && get(follow(*r->tail)) != nullptr; if (lFreeTail && rFreeTail) tryUnify_(*l->tail, *r->tail); else if (lFreeTail) @@ -812,7 +839,7 @@ void Unifier::tryUnify_(TypePackId superTp, TypePackId subTp, bool isFunctionCal } // In nonstrict mode, any also marks an optional argument. - else if (superIter.good() && isNonstrictMode() && get(FFlag::LuauAddMissingFollow ? follow(*superIter) : *superIter)) + else if (superIter.good() && isNonstrictMode() && get(follow(*superIter))) { superIter.advance(); continue; @@ -887,24 +914,21 @@ void Unifier::tryUnifyFunctions(TypeId superTy, TypeId subTy, bool isFunctionCal ice("passed non-function types to unifyFunction"); size_t numGenerics = lf->generics.size(); - if (FFlag::LuauGenericFunctions && numGenerics != rf->generics.size()) + if (numGenerics != rf->generics.size()) { numGenerics = std::min(lf->generics.size(), rf->generics.size()); errors.push_back(TypeError{location, TypeMismatch{superTy, subTy}}); } size_t numGenericPacks = lf->genericPacks.size(); - if (FFlag::LuauGenericFunctions && numGenericPacks != rf->genericPacks.size()) + if (numGenericPacks != rf->genericPacks.size()) { numGenericPacks = std::min(lf->genericPacks.size(), rf->genericPacks.size()); errors.push_back(TypeError{location, TypeMismatch{superTy, subTy}}); } - if (FFlag::LuauGenericFunctions) - { - for (size_t i = 0; i < numGenerics; i++) - log.pushSeen(lf->generics[i], rf->generics[i]); - } + for (size_t i = 0; i < numGenerics; i++) + log.pushSeen(lf->generics[i], rf->generics[i]); CountMismatch::Context context = ctx; @@ -931,22 +955,19 @@ void Unifier::tryUnifyFunctions(TypeId superTy, TypeId subTy, bool isFunctionCal tryUnify_(lf->retType, rf->retType); } - if (lf->definition && !rf->definition && (!FFlag::LuauDontMutatePersistentFunctions || !subTy->persistent)) + if (lf->definition && !rf->definition && !subTy->persistent) { rf->definition = lf->definition; } - else if (!lf->definition && rf->definition && (!FFlag::LuauDontMutatePersistentFunctions || !superTy->persistent)) + else if (!lf->definition && rf->definition && !superTy->persistent) { lf->definition = rf->definition; } ctx = context; - if (FFlag::LuauGenericFunctions) - { - for (int i = int(numGenerics) - 1; 0 <= i; i--) - log.popSeen(lf->generics[i], rf->generics[i]); - } + for (int i = int(numGenerics) - 1; 0 <= i; i--) + log.popSeen(lf->generics[i], rf->generics[i]); } namespace @@ -1032,7 +1053,12 @@ void Unifier::tryUnifyTables(TypeId left, TypeId right, bool isIntersection) Unifier innerState = makeChildUnifier(); innerState.tryUnify_(prop.type, r->second.type); - checkChildUnifierTypeMismatch(innerState.errors, left, right); + + if (FFlag::LuauExtendedTypeMismatchError) + checkChildUnifierTypeMismatch(innerState.errors, name, left, right); + else + checkChildUnifierTypeMismatch(innerState.errors, left, right); + if (innerState.errors.empty()) log.concat(std::move(innerState.log)); else @@ -1047,7 +1073,12 @@ void Unifier::tryUnifyTables(TypeId left, TypeId right, bool isIntersection) Unifier innerState = makeChildUnifier(); innerState.tryUnify_(prop.type, rt->indexer->indexResultType); - checkChildUnifierTypeMismatch(innerState.errors, left, right); + + if (FFlag::LuauExtendedTypeMismatchError) + checkChildUnifierTypeMismatch(innerState.errors, name, left, right); + else + checkChildUnifierTypeMismatch(innerState.errors, left, right); + if (innerState.errors.empty()) log.concat(std::move(innerState.log)); else @@ -1083,7 +1114,12 @@ void Unifier::tryUnifyTables(TypeId left, TypeId right, bool isIntersection) Unifier innerState = makeChildUnifier(); innerState.tryUnify_(prop.type, lt->indexer->indexResultType); - checkChildUnifierTypeMismatch(innerState.errors, left, right); + + if (FFlag::LuauExtendedTypeMismatchError) + checkChildUnifierTypeMismatch(innerState.errors, name, left, right); + else + checkChildUnifierTypeMismatch(innerState.errors, left, right); + if (innerState.errors.empty()) log.concat(std::move(innerState.log)); else @@ -1384,21 +1420,8 @@ void Unifier::tryUnifySealedTables(TypeId left, TypeId right, bool isIntersectio const auto& r = rt->props.find(it.first); if (r == rt->props.end()) { - if (FFlag::LuauSealedTableUnifyOptionalFix) - { - if (isOptional(it.second.type)) - continue; - } - else - { - if (get(it.second.type)) - { - const UnionTypeVar* possiblyOptional = get(it.second.type); - const std::vector& options = possiblyOptional->options; - if (options.end() != std::find_if(options.begin(), options.end(), isNil)) - continue; - } - } + if (isOptional(it.second.type)) + continue; missingPropertiesInSuper.push_back(it.first); @@ -1482,21 +1505,8 @@ void Unifier::tryUnifySealedTables(TypeId left, TypeId right, bool isIntersectio const auto& r = lt->props.find(it.first); if (r == lt->props.end()) { - if (FFlag::LuauSealedTableUnifyOptionalFix) - { - if (isOptional(it.second.type)) - continue; - } - else - { - if (get(it.second.type)) - { - const UnionTypeVar* possiblyOptional = get(it.second.type); - const std::vector& options = possiblyOptional->options; - if (options.end() != std::find_if(options.begin(), options.end(), isNil)) - continue; - } - } + if (isOptional(it.second.type)) + continue; extraPropertiesInSub.push_back(it.first); } @@ -1526,7 +1536,18 @@ void Unifier::tryUnifyWithMetatable(TypeId metatable, TypeId other, bool reverse innerState.tryUnify_(lhs->table, rhs->table); innerState.tryUnify_(lhs->metatable, rhs->metatable); - checkChildUnifierTypeMismatch(innerState.errors, reversed ? other : metatable, reversed ? metatable : other); + if (FFlag::LuauExtendedTypeMismatchError) + { + if (auto e = hasUnificationTooComplex(innerState.errors)) + errors.push_back(*e); + else if (!innerState.errors.empty()) + errors.push_back( + TypeError{location, TypeMismatch{reversed ? other : metatable, reversed ? metatable : other, "", innerState.errors.front()}}); + } + else + { + checkChildUnifierTypeMismatch(innerState.errors, reversed ? other : metatable, reversed ? metatable : other); + } log.concat(std::move(innerState.log)); } @@ -1613,10 +1634,34 @@ void Unifier::tryUnifyWithClass(TypeId superTy, TypeId subTy, bool reversed) { ok = false; errors.push_back(TypeError{location, UnknownProperty{superTy, propName}}); - tryUnify_(prop.type, singletonTypes.errorType); + + if (!FFlag::LuauExtendedClassMismatchError) + tryUnify_(prop.type, singletonTypes.errorType); } else - tryUnify_(prop.type, classProp->type); + { + if (FFlag::LuauExtendedClassMismatchError) + { + Unifier innerState = makeChildUnifier(); + innerState.tryUnify_(prop.type, classProp->type); + + checkChildUnifierTypeMismatch(innerState.errors, propName, reversed ? subTy : superTy, reversed ? superTy : subTy); + + if (innerState.errors.empty()) + { + log.concat(std::move(innerState.log)); + } + else + { + ok = false; + innerState.log.rollback(); + } + } + else + { + tryUnify_(prop.type, classProp->type); + } + } } if (table->indexer) @@ -1649,45 +1694,24 @@ static void queueTypePack_DEPRECATED( while (true) { - if (FFlag::LuauAddMissingFollow) - a = follow(a); + a = follow(a); if (seenTypePacks.count(a)) break; seenTypePacks.insert(a); - if (FFlag::LuauAddMissingFollow) + if (get(a)) { - if (get(a)) - { - state.log(a); - *asMutable(a) = Unifiable::Bound{anyTypePack}; - } - else if (auto tp = get(a)) - { - queue.insert(queue.end(), tp->head.begin(), tp->head.end()); - if (tp->tail) - a = *tp->tail; - else - break; - } + state.log(a); + *asMutable(a) = Unifiable::Bound{anyTypePack}; } - else + else if (auto tp = get(a)) { - if (get(a)) - { - state.log(a); - *asMutable(a) = Unifiable::Bound{anyTypePack}; - } - - if (auto tp = get(a)) - { - queue.insert(queue.end(), tp->head.begin(), tp->head.end()); - if (tp->tail) - a = *tp->tail; - else - break; - } + queue.insert(queue.end(), tp->head.begin(), tp->head.end()); + if (tp->tail) + a = *tp->tail; + else + break; } } } @@ -1698,45 +1722,24 @@ static void queueTypePack(std::vector& queue, DenseHashSet& while (true) { - if (FFlag::LuauAddMissingFollow) - a = follow(a); + a = follow(a); if (seenTypePacks.find(a)) break; seenTypePacks.insert(a); - if (FFlag::LuauAddMissingFollow) + if (get(a)) { - if (get(a)) - { - state.log(a); - *asMutable(a) = Unifiable::Bound{anyTypePack}; - } - else if (auto tp = get(a)) - { - queue.insert(queue.end(), tp->head.begin(), tp->head.end()); - if (tp->tail) - a = *tp->tail; - else - break; - } + state.log(a); + *asMutable(a) = Unifiable::Bound{anyTypePack}; } - else + else if (auto tp = get(a)) { - if (get(a)) - { - state.log(a); - *asMutable(a) = Unifiable::Bound{anyTypePack}; - } - - if (auto tp = get(a)) - { - queue.insert(queue.end(), tp->head.begin(), tp->head.end()); - if (tp->tail) - a = *tp->tail; - else - break; - } + queue.insert(queue.end(), tp->head.begin(), tp->head.end()); + if (tp->tail) + a = *tp->tail; + else + break; } } } @@ -1990,33 +1993,6 @@ std::optional Unifier::findTablePropertyRespectingMeta(TypeId lhsType, N return Luau::findTablePropertyRespectingMeta(errors, globalScope, lhsType, name, location); } -std::optional Unifier::findMetatableEntry(TypeId type, std::string entry) -{ - type = follow(type); - - std::optional metatable = getMetatable(type); - if (!metatable) - return std::nullopt; - - TypeId unwrapped = follow(*metatable); - - if (get(unwrapped)) - return singletonTypes.anyType; - - const TableTypeVar* mtt = getTableType(unwrapped); - if (!mtt) - { - errors.push_back(TypeError{location, GenericError{"Metatable was not a table."}}); - return std::nullopt; - } - - auto it = mtt->props.find(entry); - if (it != mtt->props.end()) - return it->second.type; - else - return std::nullopt; -} - void Unifier::occursCheck(TypeId needle, TypeId haystack) { std::unordered_set seen_DEPRECATED; @@ -2168,7 +2144,7 @@ void Unifier::occursCheck(std::unordered_set& seen_DEPRECATED, Dense { for (const auto& ty : a->head) { - if (auto f = get(FFlag::LuauAddMissingFollow ? follow(ty) : ty)) + if (auto f = get(follow(ty))) { occursCheck(seen_DEPRECATED, seen, needle, f->argTypes); occursCheck(seen_DEPRECATED, seen, needle, f->retType); @@ -2207,6 +2183,17 @@ void Unifier::checkChildUnifierTypeMismatch(const ErrorVec& innerErrors, TypeId errors.push_back(TypeError{location, TypeMismatch{wantedType, givenType}}); } +void Unifier::checkChildUnifierTypeMismatch(const ErrorVec& innerErrors, const std::string& prop, TypeId wantedType, TypeId givenType) +{ + LUAU_ASSERT(FFlag::LuauExtendedTypeMismatchError || FFlag::LuauExtendedClassMismatchError); + + if (auto e = hasUnificationTooComplex(innerErrors)) + errors.push_back(*e); + else if (!innerErrors.empty()) + errors.push_back( + TypeError{location, TypeMismatch{wantedType, givenType, format("Property '%s' is not compatible", prop.c_str()), innerErrors.front()}}); +} + void Unifier::ice(const std::string& message, const Location& location) { sharedState.iceHandler->ice(message, location); diff --git a/Ast/include/Luau/Parser.h b/Ast/include/Luau/Parser.h index 42c64dc9..39c7d925 100644 --- a/Ast/include/Luau/Parser.h +++ b/Ast/include/Luau/Parser.h @@ -282,7 +282,6 @@ private: // `<' namelist `>' std::pair, AstArray> parseGenericTypeList(); - std::pair, AstArray> parseGenericTypeListIfFFlagParseGenericFunctions(); // `<' typeAnnotation[, ...] `>' AstArray parseTypeParams(); diff --git a/Ast/src/Parser.cpp b/Ast/src/Parser.cpp index ee84542d..a1bad65e 100644 --- a/Ast/src/Parser.cpp +++ b/Ast/src/Parser.cpp @@ -10,13 +10,13 @@ // See docs/SyntaxChanges.md for an explanation. LUAU_FASTINTVARIABLE(LuauRecursionLimit, 1000) LUAU_FASTINTVARIABLE(LuauParseErrorLimit, 100) -LUAU_FASTFLAGVARIABLE(LuauGenericFunctionsParserFix, false) -LUAU_FASTFLAGVARIABLE(LuauParseGenericFunctions, false) LUAU_FASTFLAGVARIABLE(LuauCaptureBrokenCommentSpans, false) LUAU_FASTFLAGVARIABLE(LuauIfElseExpressionBaseSupport, false) LUAU_FASTFLAGVARIABLE(LuauIfStatementRecursionGuard, false) LUAU_FASTFLAGVARIABLE(LuauTypeAliasPacks, false) LUAU_FASTFLAGVARIABLE(LuauParseTypePackTypeParameters, false) +LUAU_FASTFLAGVARIABLE(LuauFixAmbiguousErrorRecoveryInAssign, false) +LUAU_FASTFLAGVARIABLE(LuauParseGenericFunctionTypeBegin, false) namespace Luau { @@ -957,7 +957,7 @@ AstStat* Parser::parseAssignment(AstExpr* initial) { nextLexeme(); - AstExpr* expr = parsePrimaryExpr(/* asStatement= */ false); + AstExpr* expr = parsePrimaryExpr(/* asStatement= */ FFlag::LuauFixAmbiguousErrorRecoveryInAssign); if (!isExprLValue(expr)) expr = reportExprError(expr->location, copy({expr}), "Assigned expression must be a variable or a field"); @@ -995,7 +995,7 @@ std::pair Parser::parseFunctionBody( { Location start = matchFunction.location; - auto [generics, genericPacks] = parseGenericTypeListIfFFlagParseGenericFunctions(); + auto [generics, genericPacks] = parseGenericTypeList(); Lexeme matchParen = lexer.current(); expectAndConsume('(', "function"); @@ -1343,19 +1343,18 @@ AstTypeOrPack Parser::parseFunctionTypeAnnotation(bool allowPack) { incrementRecursionCounter("type annotation"); - bool monomorphic = !(FFlag::LuauParseGenericFunctions && lexer.current().type == '<'); - - auto [generics, genericPacks] = parseGenericTypeListIfFFlagParseGenericFunctions(); + bool monomorphic = lexer.current().type != '<'; Lexeme begin = lexer.current(); - if (FFlag::LuauGenericFunctionsParserFix) - expectAndConsume('(', "function parameters"); - else - { - LUAU_ASSERT(begin.type == '('); - nextLexeme(); // ( - } + auto [generics, genericPacks] = parseGenericTypeList(); + + Lexeme parameterStart = lexer.current(); + + if (!FFlag::LuauParseGenericFunctionTypeBegin) + begin = parameterStart; + + expectAndConsume('(', "function parameters"); matchRecoveryStopOnToken[Lexeme::SkinnyArrow]++; @@ -1366,7 +1365,7 @@ AstTypeOrPack Parser::parseFunctionTypeAnnotation(bool allowPack) if (lexer.current().type != ')') varargAnnotation = parseTypeList(params, names); - expectMatchAndConsume(')', begin, true); + expectMatchAndConsume(')', parameterStart, true); matchRecoveryStopOnToken[Lexeme::SkinnyArrow]--; @@ -1585,7 +1584,7 @@ AstTypeOrPack Parser::parseSimpleTypeAnnotation(bool allowPack) { return {parseTableTypeAnnotation(), {}}; } - else if (lexer.current().type == '(' || (FFlag::LuauParseGenericFunctions && lexer.current().type == '<')) + else if (lexer.current().type == '(' || lexer.current().type == '<') { return parseFunctionTypeAnnotation(allowPack); } @@ -2315,19 +2314,6 @@ Parser::Name Parser::parseIndexName(const char* context, const Position& previou return Name(nameError, location); } -std::pair, AstArray> Parser::parseGenericTypeListIfFFlagParseGenericFunctions() -{ - if (FFlag::LuauParseGenericFunctions) - return Parser::parseGenericTypeList(); - AstArray generics; - AstArray genericPacks; - generics.size = 0; - generics.data = nullptr; - genericPacks.size = 0; - genericPacks.data = nullptr; - return std::pair(generics, genericPacks); -} - std::pair, AstArray> Parser::parseGenericTypeList() { TempVector names{scratchName}; @@ -2342,7 +2328,7 @@ std::pair, AstArray> Parser::parseGenericTypeList() while (true) { AstName name = parseName().name; - if (FFlag::LuauParseGenericFunctions && lexer.current().type == Lexeme::Dot3) + if (lexer.current().type == Lexeme::Dot3) { seenPack = true; nextLexeme(); diff --git a/CLI/Repl.cpp b/CLI/Repl.cpp index 63b86a80..0d804be5 100644 --- a/CLI/Repl.cpp +++ b/CLI/Repl.cpp @@ -215,9 +215,6 @@ extern "C" { const char* executeScript(const char* source) { - // static string for caching result (prevents dangling ptr on function exit) - static std::string result; - // setup flags for (Luau::FValue* flag = Luau::FValue::list; flag; flag = flag->next) if (strncmp(flag->name, "Luau", 4) == 0) @@ -233,15 +230,13 @@ extern "C" // sandbox thread luaL_sandboxthread(L); + // static string for caching result (prevents dangling ptr on function exit) + static std::string result; + // run code + collect error - std::string error = runCode(L, source); - result = error; - - if (error.length()) - { - return result.c_str(); - } - return NULL; + result = runCode(L, source); + + return result.empty() ? NULL : result.c_str(); } } #endif @@ -591,3 +586,4 @@ int main(int argc, char** argv) } } #endif + diff --git a/Compiler/include/Luau/Bytecode.h b/Compiler/include/Luau/Bytecode.h index 4b03ed1c..71631d10 100644 --- a/Compiler/include/Luau/Bytecode.h +++ b/Compiler/include/Luau/Bytecode.h @@ -467,6 +467,10 @@ enum LuauBuiltinFunction // vector ctor LBF_VECTOR, + + // bit32.count + LBF_BIT32_COUNTLZ, + LBF_BIT32_COUNTRZ, }; // Capture type, used in LOP_CAPTURE diff --git a/Compiler/include/Luau/Compiler.h b/Compiler/include/Luau/Compiler.h index 2935f595..4f88e602 100644 --- a/Compiler/include/Luau/Compiler.h +++ b/Compiler/include/Luau/Compiler.h @@ -37,8 +37,7 @@ struct CompileOptions const char* vectorLib = nullptr; const char* vectorCtor = nullptr; - // array of globals that are mutable; disables the import optimization for fields accessed through them - // use NULL to end the array + // null-terminated array of globals that are mutable; disables the import optimization for fields accessed through these const char** mutableGlobals = nullptr; }; diff --git a/Compiler/src/Compiler.cpp b/Compiler/src/Compiler.cpp index 08cb9934..9712f02f 100644 --- a/Compiler/src/Compiler.cpp +++ b/Compiler/src/Compiler.cpp @@ -15,6 +15,7 @@ LUAU_FASTFLAGVARIABLE(LuauPreloadClosuresFenv, false) LUAU_FASTFLAGVARIABLE(LuauPreloadClosuresUpval, false) LUAU_FASTFLAGVARIABLE(LuauGenericSpecialGlobals, false) LUAU_FASTFLAG(LuauIfElseExpressionBaseSupport) +LUAU_FASTFLAGVARIABLE(LuauBit32CountBuiltin, false) namespace Luau { @@ -23,6 +24,7 @@ static const uint32_t kMaxRegisterCount = 255; static const uint32_t kMaxUpvalueCount = 200; static const uint32_t kMaxLocalCount = 200; +// TODO: Remove with LuauGenericSpecialGlobals static const char* kSpecialGlobals[] = {"Game", "Workspace", "_G", "game", "plugin", "script", "shared", "workspace"}; CompileError::CompileError(const Location& location, const std::string& message) @@ -3637,6 +3639,10 @@ struct Compiler return LBF_BIT32_RROTATE; if (builtin.method == "rshift") return LBF_BIT32_RSHIFT; + if (builtin.method == "countlz" && FFlag::LuauBit32CountBuiltin) + return LBF_BIT32_COUNTLZ; + if (builtin.method == "countrz" && FFlag::LuauBit32CountBuiltin) + return LBF_BIT32_COUNTRZ; } if (builtin.object == "string") @@ -3711,11 +3717,9 @@ void compileOrThrow(BytecodeBuilder& bytecode, AstStatBlock* root, const AstName compiler.globals[name].writable = true; if (options.mutableGlobals) - for (const char** ptr = options.mutableGlobals; *ptr != NULL; ++ptr) - { + for (const char** ptr = options.mutableGlobals; *ptr; ++ptr) if (AstName name = names.get(*ptr); name.value) compiler.globals[name].writable = true; - } } else { @@ -3738,7 +3742,7 @@ void compileOrThrow(BytecodeBuilder& bytecode, AstStatBlock* root, const AstName } // this visitor tracks calls to getfenv/setfenv and disables some optimizations when they are found - if (FFlag::LuauPreloadClosuresFenv && options.optimizationLevel >= 1) + if (FFlag::LuauPreloadClosuresFenv && options.optimizationLevel >= 1 && (names.get("getfenv").value || names.get("setfenv").value)) { Compiler::FenvVisitor fenvVisitor(compiler.getfenvUsed, compiler.setfenvUsed); root->visit(&fenvVisitor); diff --git a/Makefile b/Makefile index e96a9cbd..ffe43092 100644 --- a/Makefile +++ b/Makefile @@ -137,12 +137,11 @@ $(TESTS_TARGET) $(REPL_CLI_TARGET) $(ANALYZE_CLI_TARGET): # executable targets for fuzzing fuzz-%: $(BUILD)/fuzz/%.cpp.o $(ANALYSIS_TARGET) $(COMPILER_TARGET) $(AST_TARGET) $(VM_TARGET) + $(CXX) $^ $(LDFLAGS) -o $@ + fuzz-proto: $(BUILD)/fuzz/proto.cpp.o $(BUILD)/fuzz/protoprint.cpp.o $(BUILD)/fuzz/luau.pb.cpp.o $(ANALYSIS_TARGET) $(COMPILER_TARGET) $(AST_TARGET) $(VM_TARGET) | build/libprotobuf-mutator fuzz-prototest: $(BUILD)/fuzz/prototest.cpp.o $(BUILD)/fuzz/protoprint.cpp.o $(BUILD)/fuzz/luau.pb.cpp.o $(ANALYSIS_TARGET) $(COMPILER_TARGET) $(AST_TARGET) $(VM_TARGET) | build/libprotobuf-mutator -fuzz-%: - $(CXX) $^ $(LDFLAGS) -o $@ - # static library targets $(AST_TARGET): $(AST_OBJECTS) $(COMPILER_TARGET): $(COMPILER_OBJECTS) diff --git a/VM/include/lua.h b/VM/include/lua.h index 8e7d6468..a9d3e875 100644 --- a/VM/include/lua.h +++ b/VM/include/lua.h @@ -213,6 +213,8 @@ LUA_API int lua_resume(lua_State* L, lua_State* from, int narg); LUA_API int lua_resumeerror(lua_State* L, lua_State* from); LUA_API int lua_status(lua_State* L); LUA_API int lua_isyieldable(lua_State* L); +LUA_API void* lua_getthreaddata(lua_State* L); +LUA_API void lua_setthreaddata(lua_State* L, void* data); /* ** garbage-collection function and options diff --git a/VM/src/lapi.cpp b/VM/src/lapi.cpp index c0c78b93..7e742644 100644 --- a/VM/src/lapi.cpp +++ b/VM/src/lapi.cpp @@ -988,6 +988,16 @@ int lua_status(lua_State* L) return L->status; } +void* lua_getthreaddata(lua_State* L) +{ + return L->userdata; +} + +void lua_setthreaddata(lua_State* L, void* data) +{ + L->userdata = data; +} + /* ** Garbage-collection function */ diff --git a/VM/src/lbitlib.cpp b/VM/src/lbitlib.cpp index 0754a351..907c43c4 100644 --- a/VM/src/lbitlib.cpp +++ b/VM/src/lbitlib.cpp @@ -2,8 +2,11 @@ // This code is based on Lua 5.x implementation licensed under MIT License; see lua_LICENSE.txt for details #include "lualib.h" +#include "lcommon.h" #include "lnumutils.h" +LUAU_FASTFLAGVARIABLE(LuauBit32Count, false) + #define ALLONES ~0u #define NBITS int(8 * sizeof(unsigned)) @@ -177,6 +180,44 @@ static int b_replace(lua_State* L) return 1; } +static int b_countlz(lua_State* L) +{ + if (!FFlag::LuauBit32Count) + luaL_error(L, "bit32.countlz isn't enabled"); + + b_uint v = luaL_checkunsigned(L, 1); + + b_uint r = NBITS; + for (int i = 0; i < NBITS; ++i) + if (v & (1u << (NBITS - 1 - i))) + { + r = i; + break; + } + + lua_pushunsigned(L, r); + return 1; +} + +static int b_countrz(lua_State* L) +{ + if (!FFlag::LuauBit32Count) + luaL_error(L, "bit32.countrz isn't enabled"); + + b_uint v = luaL_checkunsigned(L, 1); + + b_uint r = NBITS; + for (int i = 0; i < NBITS; ++i) + if (v & (1u << i)) + { + r = i; + break; + } + + lua_pushunsigned(L, r); + return 1; +} + static const luaL_Reg bitlib[] = { {"arshift", b_arshift}, {"band", b_and}, @@ -190,6 +231,8 @@ static const luaL_Reg bitlib[] = { {"replace", b_replace}, {"rrotate", b_rrot}, {"rshift", b_rshift}, + {"countlz", b_countlz}, + {"countrz", b_countrz}, {NULL, NULL}, }; diff --git a/VM/src/lbuiltins.cpp b/VM/src/lbuiltins.cpp index 9ff0d6a8..9ab57ac9 100644 --- a/VM/src/lbuiltins.cpp +++ b/VM/src/lbuiltins.cpp @@ -1031,6 +1031,52 @@ static int luauF_vector(lua_State* L, StkId res, TValue* arg0, int nresults, Stk return -1; } +static int luauF_countlz(lua_State* L, StkId res, TValue* arg0, int nresults, StkId args, int nparams) +{ + if (nparams >= 1 && nresults <= 1 && ttisnumber(arg0)) + { + double a1 = nvalue(arg0); + + unsigned n; + luai_num2unsigned(n, a1); + +#ifdef _MSC_VER + unsigned long rl; + int r = _BitScanReverse(&rl, n) ? 31 - int(rl) : 32; +#else + int r = n == 0 ? 32 : __builtin_clz(n); +#endif + + setnvalue(res, double(r)); + return 1; + } + + return -1; +} + +static int luauF_countrz(lua_State* L, StkId res, TValue* arg0, int nresults, StkId args, int nparams) +{ + if (nparams >= 1 && nresults <= 1 && ttisnumber(arg0)) + { + double a1 = nvalue(arg0); + + unsigned n; + luai_num2unsigned(n, a1); + +#ifdef _MSC_VER + unsigned long rl; + int r = _BitScanForward(&rl, n) ? int(rl) : 32; +#else + int r = n == 0 ? 32 : __builtin_ctz(n); +#endif + + setnvalue(res, double(r)); + return 1; + } + + return -1; +} + luau_FastFunction luauF_table[256] = { NULL, luauF_assert, @@ -1097,4 +1143,7 @@ luau_FastFunction luauF_table[256] = { luauF_tunpack, luauF_vector, + + luauF_countlz, + luauF_countrz, }; diff --git a/VM/src/lgc.cpp b/VM/src/lgc.cpp index 6553009f..11f79d1a 100644 --- a/VM/src/lgc.cpp +++ b/VM/src/lgc.cpp @@ -13,7 +13,6 @@ #include LUAU_FASTFLAGVARIABLE(LuauRescanGrayAgainForwardBarrier, false) -LUAU_FASTFLAGVARIABLE(LuauConsolidatedStep, false) LUAU_FASTFLAGVARIABLE(LuauSeparateAtomic, false) LUAU_FASTFLAG(LuauArrayBoundary) @@ -677,117 +676,6 @@ static size_t atomic(lua_State* L) return work; } -static size_t singlestep(lua_State* L) -{ - size_t cost = 0; - global_State* g = L->global; - switch (g->gcstate) - { - case GCSpause: - { - markroot(L); /* start a new collection */ - LUAU_ASSERT(g->gcstate == GCSpropagate); - break; - } - case GCSpropagate: - { - if (g->gray) - { - g->gcstats.currcycle.markitems++; - - cost = propagatemark(g); - } - else - { - // perform one iteration over 'gray again' list - g->gray = g->grayagain; - g->grayagain = NULL; - - g->gcstate = GCSpropagateagain; - } - break; - } - case GCSpropagateagain: - { - if (g->gray) - { - g->gcstats.currcycle.markitems++; - - cost = propagatemark(g); - } - else /* no more `gray' objects */ - { - if (FFlag::LuauSeparateAtomic) - { - g->gcstate = GCSatomic; - } - else - { - double starttimestamp = lua_clock(); - - g->gcstats.currcycle.atomicstarttimestamp = starttimestamp; - g->gcstats.currcycle.atomicstarttotalsizebytes = g->totalbytes; - - atomic(L); /* finish mark phase */ - LUAU_ASSERT(g->gcstate == GCSsweepstring); - - g->gcstats.currcycle.atomictime += lua_clock() - starttimestamp; - } - } - break; - } - case GCSatomic: - { - g->gcstats.currcycle.atomicstarttimestamp = lua_clock(); - g->gcstats.currcycle.atomicstarttotalsizebytes = g->totalbytes; - - cost = atomic(L); /* finish mark phase */ - LUAU_ASSERT(g->gcstate == GCSsweepstring); - break; - } - case GCSsweepstring: - { - size_t traversedcount = 0; - sweepwholelist(L, &g->strt.hash[g->sweepstrgc++], &traversedcount); - - // nothing more to sweep? - if (g->sweepstrgc >= g->strt.size) - { - // sweep string buffer list and preserve used string count - uint32_t nuse = L->global->strt.nuse; - sweepwholelist(L, &g->strbufgc, &traversedcount); - L->global->strt.nuse = nuse; - - g->gcstate = GCSsweep; // end sweep-string phase - } - - g->gcstats.currcycle.sweepitems += traversedcount; - - cost = GC_SWEEPCOST; - break; - } - case GCSsweep: - { - size_t traversedcount = 0; - g->sweepgc = sweeplist(L, g->sweepgc, GC_SWEEPMAX, &traversedcount); - - g->gcstats.currcycle.sweepitems += traversedcount; - - if (*g->sweepgc == NULL) - { /* nothing more to sweep? */ - shrinkbuffers(L); - g->gcstate = GCSpause; /* end collection */ - } - cost = GC_SWEEPMAX * GC_SWEEPCOST; - break; - } - default: - LUAU_ASSERT(!"Unexpected GC state"); - } - - return cost; -} - static size_t gcstep(lua_State* L, size_t limit) { size_t cost = 0; @@ -980,37 +868,12 @@ void luaC_step(lua_State* L, bool assist) int lastgcstate = g->gcstate; double lasttimestamp = lua_clock(); - if (FFlag::LuauConsolidatedStep) - { - size_t work = gcstep(L, lim); + size_t work = gcstep(L, lim); - if (assist) - g->gcstats.currcycle.assistwork += work; - else - g->gcstats.currcycle.explicitwork += work; - } + if (assist) + g->gcstats.currcycle.assistwork += work; else - { - // always perform at least one single step - do - { - lim -= singlestep(L); - - // if we have switched to a different state, capture the duration of last stage - // this way we reduce the number of timer calls we make - if (lastgcstate != g->gcstate) - { - GC_INTERRUPT(lastgcstate); - - double now = lua_clock(); - - recordGcStateTime(g, lastgcstate, now - lasttimestamp, assist); - - lasttimestamp = now; - lastgcstate = g->gcstate; - } - } while (lim > 0 && g->gcstate != GCSpause); - } + g->gcstats.currcycle.explicitwork += work; recordGcStateTime(g, lastgcstate, lua_clock() - lasttimestamp, assist); @@ -1037,14 +900,7 @@ void luaC_step(lua_State* L, bool assist) g->GCthreshold -= debt; } - if (FFlag::LuauConsolidatedStep) - { - GC_INTERRUPT(lastgcstate); - } - else - { - GC_INTERRUPT(g->gcstate); - } + GC_INTERRUPT(lastgcstate); } void luaC_fullgc(lua_State* L) @@ -1070,10 +926,7 @@ void luaC_fullgc(lua_State* L) while (g->gcstate != GCSpause) { LUAU_ASSERT(g->gcstate == GCSsweepstring || g->gcstate == GCSsweep); - if (FFlag::LuauConsolidatedStep) - gcstep(L, SIZE_MAX); - else - singlestep(L); + gcstep(L, SIZE_MAX); } finishGcCycleStats(g); @@ -1084,10 +937,7 @@ void luaC_fullgc(lua_State* L) markroot(L); while (g->gcstate != GCSpause) { - if (FFlag::LuauConsolidatedStep) - gcstep(L, SIZE_MAX); - else - singlestep(L); + gcstep(L, SIZE_MAX); } /* reclaim as much buffer memory as possible (shrinkbuffers() called during sweep is incremental) */ shrinkbuffersfull(L); diff --git a/VM/src/lstrlib.cpp b/VM/src/lstrlib.cpp index a9db3727..80a34483 100644 --- a/VM/src/lstrlib.cpp +++ b/VM/src/lstrlib.cpp @@ -8,6 +8,8 @@ #include #include +LUAU_FASTFLAGVARIABLE(LuauStrPackUBCastFix, false) + /* macro to `unsign' a character */ #define uchar(c) ((unsigned char)(c)) @@ -1404,10 +1406,20 @@ static int str_pack(lua_State* L) } case Kuint: { /* unsigned integers */ - unsigned long long n = (unsigned long long)luaL_checknumber(L, arg); - if (size < SZINT) /* need overflow check? */ - luaL_argcheck(L, n < ((unsigned long long)1 << (size * NB)), arg, "unsigned overflow"); - packint(&b, n, h.islittle, size, 0); + if (FFlag::LuauStrPackUBCastFix) + { + long long n = (long long)luaL_checknumber(L, arg); + if (size < SZINT) /* need overflow check? */ + luaL_argcheck(L, (unsigned long long)n < ((unsigned long long)1 << (size * NB)), arg, "unsigned overflow"); + packint(&b, (unsigned long long)n, h.islittle, size, 0); + } + else + { + unsigned long long n = (unsigned long long)luaL_checknumber(L, arg); + if (size < SZINT) /* need overflow check? */ + luaL_argcheck(L, n < ((unsigned long long)1 << (size * NB)), arg, "unsigned overflow"); + packint(&b, n, h.islittle, size, 0); + } break; } case Kfloat: diff --git a/VM/src/ltablib.cpp b/VM/src/ltablib.cpp index de5788eb..37025818 100644 --- a/VM/src/ltablib.cpp +++ b/VM/src/ltablib.cpp @@ -9,8 +9,6 @@ #include "ldebug.h" #include "lvm.h" -LUAU_FASTFLAGVARIABLE(LuauTableFreeze, false) - static int foreachi(lua_State* L) { luaL_checktype(L, 1, LUA_TTABLE); @@ -491,9 +489,6 @@ static int tclear(lua_State* L) static int tfreeze(lua_State* L) { - if (!FFlag::LuauTableFreeze) - luaG_runerror(L, "table.freeze is disabled"); - luaL_checktype(L, 1, LUA_TTABLE); luaL_argcheck(L, !lua_getreadonly(L, 1), 1, "table is already frozen"); luaL_argcheck(L, !luaL_getmetafield(L, 1, "__metatable"), 1, "table has a protected metatable"); @@ -506,9 +501,6 @@ static int tfreeze(lua_State* L) static int tisfrozen(lua_State* L) { - if (!FFlag::LuauTableFreeze) - luaG_runerror(L, "table.isfrozen is disabled"); - luaL_checktype(L, 1, LUA_TTABLE); lua_pushboolean(L, lua_getreadonly(L, 1)); diff --git a/bench/tests/chess.lua b/bench/tests/chess.lua index 87b9abfd..f6ae2cc6 100644 --- a/bench/tests/chess.lua +++ b/bench/tests/chess.lua @@ -205,38 +205,48 @@ function Bitboard:empty() return self.h == 0 and self.l == 0 end -function Bitboard:ctz() - local target = self.l - local offset = 0 - local result = 0 - - if target == 0 then - target = self.h - result = 32 +if not bit32.countrz then + local function ctz(v) + if v == 0 then return 32 end + local offset = 0 + while bit32.extract(v, offset) == 0 do + offset = offset + 1 + end + return offset end - - if target == 0 then - return 64 - end - - while bit32.extract(target, offset) == 0 do - offset = offset + 1 - end - - return result + offset -end - -function Bitboard:ctzafter(start) - start = start + 1 - if start < 32 then - for i=start,31 do - if bit32.extract(self.l, i) == 1 then return i end + function Bitboard:ctz() + local result = ctz(self.l) + if result == 32 then + return ctz(self.h) + 32 + else + return result end end - for i=math.max(32,start),63 do - if bit32.extract(self.h, i-32) == 1 then return i end + function Bitboard:ctzafter(start) + start = start + 1 + if start < 32 then + for i=start,31 do + if bit32.extract(self.l, i) == 1 then return i end + end + end + for i=math.max(32,start),63 do + if bit32.extract(self.h, i-32) == 1 then return i end + end + return 64 + end +else + function Bitboard:ctz() + local result = bit32.countrz(self.l) + if result == 32 then + return bit32.countrz(self.h) + 32 + else + return result + end + end + function Bitboard:ctzafter(start) + local masked = self:band(Bitboard.full:lshift(start+1)) + return masked:ctz() end - return 64 end @@ -245,7 +255,7 @@ function Bitboard:lshift(amt) if amt == 0 then return self end if amt > 31 then - return Bitboard.from(0, bit32.lshift(self.l, amt-31)) + return Bitboard.from(0, bit32.lshift(self.l, amt-32)) end local l = bit32.lshift(self.l, amt) @@ -832,12 +842,12 @@ end local testCases = {} local function addTest(...) table.insert(testCases, {...}) end -addTest(StartingFen, 3, 8902) -addTest("r3k2r/p1ppqpb1/bn2pnp1/3PN3/1p2P3/2N2Q1p/PPPBBPPP/R3K2R w KQkq - 0 0", 2, 2039) -addTest("8/2p5/3p4/KP5r/1R3p1k/8/4P1P1/8 w - - 0 0", 3, 2812) -addTest("r3k2r/Pppp1ppp/1b3nbN/nP6/BBP1P3/q4N2/Pp1P2PP/R2Q1RK1 w kq - 0 1", 3, 9467) -addTest("rnbq1k1r/pp1Pbppp/2p5/8/2B5/8/PPP1NnPP/RNBQK2R w KQ - 1 8", 2, 1486) -addTest("r4rk1/1pp1qppp/p1np1n2/2b1p1B1/2B1P1b1/P1NP1N2/1PP1QPPP/R4RK1 w - - 0 10", 2, 2079) +addTest(StartingFen, 2, 400) +addTest("r3k2r/p1ppqpb1/bn2pnp1/3PN3/1p2P3/2N2Q1p/PPPBBPPP/R3K2R w KQkq - 0 0", 1, 48) +addTest("8/2p5/3p4/KP5r/1R3p1k/8/4P1P1/8 w - - 0 0", 2, 191) +addTest("r3k2r/Pppp1ppp/1b3nbN/nP6/BBP1P3/q4N2/Pp1P2PP/R2Q1RK1 w kq - 0 1", 2, 264) +addTest("rnbq1k1r/pp1Pbppp/2p5/8/2B5/8/PPP1NnPP/RNBQK2R w KQ - 1 8", 1, 44) +addTest("r4rk1/1pp1qppp/p1np1n2/2b1p1B1/2B1P1b1/P1NP1N2/1PP1QPPP/R4RK1 w - - 0 10", 1, 46) local function chess() diff --git a/fuzz/luau.proto b/fuzz/luau.proto index 41a1d077..c78fcf31 100644 --- a/fuzz/luau.proto +++ b/fuzz/luau.proto @@ -19,6 +19,7 @@ message Expr { ExprTable table = 13; ExprUnary unary = 14; ExprBinary binary = 15; + ExprIfElse ifelse = 16; } } @@ -149,6 +150,12 @@ message ExprBinary { required Expr right = 3; } +message ExprIfElse { + required Expr cond = 1; + required Expr then = 2; + required Expr else = 3; +} + message LValue { oneof lvalue_oneof { ExprLocal local = 1; diff --git a/fuzz/proto.cpp b/fuzz/proto.cpp index 6c230b67..c85fac7d 100644 --- a/fuzz/proto.cpp +++ b/fuzz/proto.cpp @@ -11,6 +11,7 @@ #include "Luau/BytecodeBuilder.h" #include "Luau/Common.h" #include "Luau/ToString.h" +#include "Luau/Transpiler.h" #include "lua.h" #include "lualib.h" @@ -23,6 +24,7 @@ const bool kFuzzLinter = true; const bool kFuzzTypeck = true; const bool kFuzzVM = true; const bool kFuzzTypes = true; +const bool kFuzzTranspile = true; static_assert(!(kFuzzVM && !kFuzzCompiler), "VM requires the compiler!"); @@ -242,6 +244,11 @@ DEFINE_PROTO_FUZZER(const luau::StatBlock& message) } } + if (kFuzzTranspile && parseResult.root) + { + transpileWithTypes(*parseResult.root); + } + // run resulting bytecode if (kFuzzVM && bytecode.size()) { diff --git a/fuzz/protoprint.cpp b/fuzz/protoprint.cpp index 2c861a55..e61b6936 100644 --- a/fuzz/protoprint.cpp +++ b/fuzz/protoprint.cpp @@ -476,6 +476,16 @@ struct ProtoToLuau print(expr.right()); } + void print(const luau::ExprIfElse& expr) + { + source += " if "; + print(expr.cond()); + source += " then "; + print(expr.then()); + source += " else "; + print(expr.else_()); + } + void print(const luau::LValue& expr) { if (expr.has_local()) diff --git a/tests/AstQuery.test.cpp b/tests/AstQuery.test.cpp index dd49e675..aa53a92b 100644 --- a/tests/AstQuery.test.cpp +++ b/tests/AstQuery.test.cpp @@ -45,7 +45,6 @@ TEST_CASE_FIXTURE(DocumentationSymbolFixture, "prop") TEST_CASE_FIXTURE(DocumentationSymbolFixture, "event_callback_arg") { ScopedFastFlag sffs[] = { - {"LuauDontMutatePersistentFunctions", true}, {"LuauPersistDefinitionFileTypes", true}, }; diff --git a/tests/Autocomplete.test.cpp b/tests/Autocomplete.test.cpp index 44b8362d..8a7798f3 100644 --- a/tests/Autocomplete.test.cpp +++ b/tests/Autocomplete.test.cpp @@ -1287,9 +1287,6 @@ local e: (n: n@5 TEST_CASE_FIXTURE(ACFixture, "generic_types") { - ScopedFastFlag luauParseGenericFunctions("LuauParseGenericFunctions", true); - ScopedFastFlag luauGenericFunctions("LuauGenericFunctions", true); - check(R"( function f(a: T@1 local b: string = "don't trip" diff --git a/tests/Conformance.test.cpp b/tests/Conformance.test.cpp index 06b3c523..c1b790b9 100644 --- a/tests/Conformance.test.cpp +++ b/tests/Conformance.test.cpp @@ -240,8 +240,6 @@ TEST_CASE("Math") TEST_CASE("Table") { - ScopedFastFlag sff("LuauTableFreeze", true); - runConformance("nextvar.lua"); } @@ -322,6 +320,8 @@ TEST_CASE("GC") TEST_CASE("Bitwise") { + ScopedFastFlag sff("LuauBit32Count", true); + runConformance("bitwise.lua"); } @@ -359,6 +359,8 @@ TEST_CASE("PCall") TEST_CASE("Pack") { + ScopedFastFlag sff{ "LuauStrPackUBCastFix", true }; + runConformance("tpack.lua"); } diff --git a/tests/Linter.test.cpp b/tests/Linter.test.cpp index 37f1b60b..7ba40c50 100644 --- a/tests/Linter.test.cpp +++ b/tests/Linter.test.cpp @@ -479,10 +479,6 @@ return foo1 TEST_CASE_FIXTURE(Fixture, "UnknownType") { - ScopedFastFlag sff{"LuauLinterUnknownTypeVectorAware", true}; - - SourceModule sm; - unfreeze(typeChecker.globalTypes); TableTypeVar::Props instanceProps{ {"ClassName", {typeChecker.anyType}}, @@ -1400,8 +1396,6 @@ end TEST_CASE_FIXTURE(Fixture, "TableOperations") { - ScopedFastFlag sff("LuauLinterTableMoveZero", true); - LintResult result = lintTyped(R"( local t = {} local tt = {} diff --git a/tests/Parser.test.cpp b/tests/Parser.test.cpp index a80718e4..e3e6ce6d 100644 --- a/tests/Parser.test.cpp +++ b/tests/Parser.test.cpp @@ -7,6 +7,8 @@ #include "doctest.h" +LUAU_FASTFLAG(LuauFixAmbiguousErrorRecoveryInAssign) + using namespace Luau; namespace @@ -625,10 +627,6 @@ TEST_CASE_FIXTURE(Fixture, "parse_error_messages") )"), "Cannot have more than one table indexer"); - ScopedFastFlag sffs1{"LuauGenericFunctions", true}; - ScopedFastFlag sffs2{"LuauGenericFunctionsParserFix", true}; - ScopedFastFlag sffs3{"LuauParseGenericFunctions", true}; - CHECK_EQ(getParseError(R"( type T = foo )"), @@ -1624,6 +1622,20 @@ TEST_CASE_FIXTURE(Fixture, "parse_error_confusing_function_call") "statements"); CHECK(result3.errors.size() == 1); + + auto result4 = matchParseError(R"( + local t = {} + function f() return t end + t.x, (f) + ().y = 5, 6 + )", + "Ambiguous syntax: this looks like an argument list for a function call, but could also be a start of new statement; use ';' to separate " + "statements"); + + if (FFlag::LuauFixAmbiguousErrorRecoveryInAssign) + CHECK(result4.errors.size() == 1); + else + CHECK(result4.errors.size() == 5); } TEST_CASE_FIXTURE(Fixture, "parse_error_varargs") @@ -1824,9 +1836,6 @@ TEST_CASE_FIXTURE(Fixture, "variadic_definition_parsing") TEST_CASE_FIXTURE(Fixture, "generic_pack_parsing") { - // Doesn't need LuauGenericFunctions - ScopedFastFlag sffs{"LuauParseGenericFunctions", true}; - ParseResult result = parseEx(R"( function f(...: a...) end @@ -1861,9 +1870,6 @@ TEST_CASE_FIXTURE(Fixture, "generic_pack_parsing") TEST_CASE_FIXTURE(Fixture, "generic_function_declaration_parsing") { - // Doesn't need LuauGenericFunctions - ScopedFastFlag sffs{"LuauParseGenericFunctions", true}; - ParseResult result = parseEx(R"( declare function f() )"); @@ -1953,12 +1959,7 @@ TEST_CASE_FIXTURE(Fixture, "function_type_named_arguments") matchParseError("type MyFunc = (a: number, b: string, c: number) -> (d: number, e: string, f: number)", "Expected '->' when parsing function type, got "); - { - ScopedFastFlag luauParseGenericFunctions{"LuauParseGenericFunctions", true}; - ScopedFastFlag luauGenericFunctionsParserFix{"LuauGenericFunctionsParserFix", true}; - - matchParseError("type MyFunc = (number) -> (d: number) -> number", "Expected '->' when parsing function type, got '<'"); - } + matchParseError("type MyFunc = (number) -> (d: number) -> number", "Expected '->' when parsing function type, got '<'"); } TEST_SUITE_END(); @@ -2362,8 +2363,6 @@ type Fn = ( CHECK_EQ("Expected '->' when parsing function type, got ')'", e.getErrors().front().getMessage()); } - ScopedFastFlag sffs3{"LuauParseGenericFunctions", true}; - try { parse(R"(type Fn = (any, string | number | ()) -> any)"); @@ -2397,8 +2396,6 @@ TEST_CASE_FIXTURE(Fixture, "AstName_comparison") TEST_CASE_FIXTURE(Fixture, "generic_type_list_recovery") { - ScopedFastFlag luauParseGenericFunctions{"LuauParseGenericFunctions", true}; - try { parse(R"( @@ -2521,7 +2518,6 @@ TEST_CASE_FIXTURE(Fixture, "parse_if_else_expression") TEST_CASE_FIXTURE(Fixture, "parse_type_pack_type_parameters") { - ScopedFastFlag luauParseGenericFunctions("LuauParseGenericFunctions", true); ScopedFastFlag luauParseTypePackTypeParameters("LuauParseTypePackTypeParameters", true); AstStat* stat = parse(R"( @@ -2534,4 +2530,9 @@ type C = Packed<(number, X...)> REQUIRE(stat != nullptr); } +TEST_CASE_FIXTURE(Fixture, "function_type_matching_parenthesis") +{ + matchParseError("local a: (number -> string", "Expected ')' (to close '(' at column 13), got '->'"); +} + TEST_SUITE_END(); diff --git a/tests/Predicate.test.cpp b/tests/Predicate.test.cpp index bb5a93c5..7081693e 100644 --- a/tests/Predicate.test.cpp +++ b/tests/Predicate.test.cpp @@ -33,8 +33,6 @@ TEST_SUITE_BEGIN("Predicate"); TEST_CASE_FIXTURE(Fixture, "Luau_merge_hashmap_order") { - ScopedFastFlag sff{"LuauOrPredicate", true}; - RefinementMap m{ {"b", typeChecker.stringType}, {"c", typeChecker.numberType}, @@ -61,8 +59,6 @@ TEST_CASE_FIXTURE(Fixture, "Luau_merge_hashmap_order") TEST_CASE_FIXTURE(Fixture, "Luau_merge_hashmap_order2") { - ScopedFastFlag sff{"LuauOrPredicate", true}; - RefinementMap m{ {"a", typeChecker.stringType}, {"b", typeChecker.stringType}, @@ -89,8 +85,6 @@ TEST_CASE_FIXTURE(Fixture, "Luau_merge_hashmap_order2") TEST_CASE_FIXTURE(Fixture, "one_map_has_overlap_at_end_whereas_other_has_it_in_start") { - ScopedFastFlag sff{"LuauOrPredicate", true}; - RefinementMap m{ {"a", typeChecker.stringType}, {"b", typeChecker.numberType}, diff --git a/tests/ToString.test.cpp b/tests/ToString.test.cpp index e18bf7cd..b076e9ad 100644 --- a/tests/ToString.test.cpp +++ b/tests/ToString.test.cpp @@ -259,9 +259,6 @@ TEST_CASE_FIXTURE(Fixture, "function_type_with_argument_names") TEST_CASE_FIXTURE(Fixture, "function_type_with_argument_names_generic") { - ScopedFastFlag luauGenericFunctions{"LuauGenericFunctions", true}; - ScopedFastFlag luauParseGenericFunctions{"LuauParseGenericFunctions", true}; - CheckResult result = check("local function f(n: number, ...: a...): (a...) return ... end"); LUAU_REQUIRE_NO_ERRORS(result); @@ -340,10 +337,6 @@ TEST_CASE_FIXTURE(Fixture, "toStringDetailed") TEST_CASE_FIXTURE(Fixture, "toStringDetailed2") { - ScopedFastFlag sff[] = { - {"LuauGenericFunctions", true}, - }; - CheckResult result = check(R"( local base = {} function base:one() return 1 end @@ -468,8 +461,6 @@ TEST_CASE_FIXTURE(Fixture, "no_parentheses_around_cyclic_function_type_in_inters TEST_CASE_FIXTURE(Fixture, "self_recursive_instantiated_param") { - ScopedFastFlag luauInstantiatedTypeParamRecursion{"LuauInstantiatedTypeParamRecursion", true}; - TypeVar tableTy{TableTypeVar{}}; TableTypeVar* ttv = getMutable(&tableTy); ttv->name = "Table"; diff --git a/tests/Transpiler.test.cpp b/tests/Transpiler.test.cpp index bfff60f9..928c03a3 100644 --- a/tests/Transpiler.test.cpp +++ b/tests/Transpiler.test.cpp @@ -21,7 +21,7 @@ local function isPortal(element) return false end - return element.component==Core.Portal + return element.component == Core.Portal end )"; @@ -223,12 +223,24 @@ TEST_CASE("escaped_strings") CHECK_EQ(code, transpile(code).code); } +TEST_CASE("escaped_strings_2") +{ + const std::string code = R"( local s="\a\b\f\n\r\t\v\'\"\\" )"; + CHECK_EQ(code, transpile(code).code); +} + TEST_CASE("need_a_space_between_number_literals_and_dots") { const std::string code = R"( return point and math.ceil(point* 100000* 100)/ 100000 .. '%'or '' )"; CHECK_EQ(code, transpile(code).code); } +TEST_CASE("binary_keywords") +{ + const std::string code = "local c = a0 ._ or b0 ._"; + CHECK_EQ(code, transpile(code).code); +} + TEST_CASE("do_blocks") { const std::string code = R"( @@ -364,10 +376,10 @@ TEST_CASE_FIXTURE(Fixture, "type_lists_should_be_emitted_correctly") )"; std::string expected = R"( - local a:(string,number,...string)->(string,...number)=function(a:string,b:number,...:...string): (string,...number) + local a:(string,number,...string)->(string,...number)=function(a:string,b:number,...:string): (string,...number) end - local b:(...string)->(...number)=function(...:...string): ...number + local b:(...string)->(...number)=function(...:string): ...number end local c:()->()=function(): () @@ -400,4 +412,238 @@ TEST_CASE_FIXTURE(Fixture, "function_type_location") CHECK_EQ(expected, actual); } +TEST_CASE_FIXTURE(Fixture, "transpile_type_assertion") +{ + std::string code = "local a = 5 :: number"; + + CHECK_EQ(code, transpile(code, {}, true).code); +} + +TEST_CASE_FIXTURE(Fixture, "transpile_if_then_else") +{ + ScopedFastFlag luauIfElseExpressionBaseSupport("LuauIfElseExpressionBaseSupport", true); + + std::string code = "local a = if 1 then 2 else 3"; + + CHECK_EQ(code, transpile(code).code); +} + +TEST_CASE_FIXTURE(Fixture, "transpile_type_reference_import") +{ + fileResolver.source["game/A"] = R"( +export type Type = { a: number } +return {} + )"; + + std::string code = R"( +local Import = require(game.A) +local a: Import.Type + )"; + + CHECK_EQ(code, transpile(code, {}, true).code); +} + +TEST_CASE_FIXTURE(Fixture, "transpile_type_packs") +{ + ScopedFastFlag luauTypeAliasPacks("LuauTypeAliasPacks", true); + ScopedFastFlag luauParseTypePackTypeParameters("LuauParseTypePackTypeParameters", true); + + std::string code = R"( +type Packed = (T...)->(T...) +local a: Packed<> +local b: Packed<(number, string)> + )"; + + CHECK_EQ(code, transpile(code, {}, true).code); +} + +TEST_CASE_FIXTURE(Fixture, "transpile_union_type_nested") +{ + std::string code = "local a: ((number)->(string))|((string)->(string))"; + + CHECK_EQ(code, transpile(code, {}, true).code); +} + +TEST_CASE_FIXTURE(Fixture, "transpile_union_type_nested_2") +{ + std::string code = "local a: (number&string)|(string&boolean)"; + + CHECK_EQ(code, transpile(code, {}, true).code); +} + +TEST_CASE_FIXTURE(Fixture, "transpile_union_type_nested_3") +{ + std::string code = "local a: nil | (string & number)"; + + CHECK_EQ("local a: ( string & number)?", transpile(code, {}, true).code); +} + +TEST_CASE_FIXTURE(Fixture, "transpile_intersection_type_nested") +{ + std::string code = "local a: ((number)->(string))&((string)->(string))"; + + CHECK_EQ(code, transpile(code, {}, true).code); +} + +TEST_CASE_FIXTURE(Fixture, "transpile_intersection_type_nested_2") +{ + std::string code = "local a: (number|string)&(string|boolean)"; + + CHECK_EQ(code, transpile(code, {}, true).code); +} + +TEST_CASE_FIXTURE(Fixture, "transpile_varargs") +{ + std::string code = "local function f(...) return ... end"; + + CHECK_EQ(code, transpile(code, {}, true).code); +} + +TEST_CASE_FIXTURE(Fixture, "transpile_index_expr") +{ + std::string code = "local a = {1, 2, 3} local b = a[2]"; + + CHECK_EQ(code, transpile(code, {}, true).code); +} + +TEST_CASE_FIXTURE(Fixture, "transpile_unary") +{ + std::string code = R"( +local a = 1 +local b = -1 +local c = true +local d = not c +local e = 'hello' +local d = #e + )"; + + CHECK_EQ(code, transpile(code, {}, true).code); +} + +TEST_CASE_FIXTURE(Fixture, "transpile_break_continue") +{ + std::string code = R"( +local a, b, c +repeat + if a then break end + if b then continue end +until c + )"; + + CHECK_EQ(code, transpile(code, {}, true).code); +} + +TEST_CASE_FIXTURE(Fixture, "transpile_compound_assignmenr") +{ + std::string code = R"( +local a = 1 +a += 2 +a -= 3 +a *= 4 +a /= 5 +a %= 6 +a ^= 7 +a ..= ' - result' + )"; + + CHECK_EQ(code, transpile(code, {}, true).code); +} + +TEST_CASE_FIXTURE(Fixture, "transpile_assign_multiple") +{ + std::string code = "a, b, c = 1, 2, 3"; + + CHECK_EQ(code, transpile(code, {}, true).code); +} + +TEST_CASE_FIXTURE(Fixture, "transpile_generic_function") +{ + ScopedFastFlag luauParseGenericFunctionTypeBegin("LuauParseGenericFunctionTypeBegin", true); + + std::string code = R"( +local function foo(a: T, ...: S...) return 1 end +local f: (T, S...)->(number) = foo + )"; + + CHECK_EQ(code, transpile(code, {}, true).code); +} + +TEST_CASE_FIXTURE(Fixture, "transpile_union_reverse") +{ + std::string code = "local a: nil | number"; + + CHECK_EQ("local a: number?", transpile(code, {}, true).code); +} + +TEST_CASE_FIXTURE(Fixture, "transpile_for_in_multiple") +{ + std::string code = "for k,v in next,{}do print(k,v) end"; + + CHECK_EQ(code, transpile(code, {}, true).code); +} + +TEST_CASE_FIXTURE(Fixture, "transpile_error_expr") +{ + std::string code = "local a = f:-"; + + auto allocator = Allocator{}; + auto names = AstNameTable{allocator}; + ParseResult parseResult = Parser::parse(code.data(), code.size(), names, allocator, {}); + + CHECK_EQ("local a = (error-expr: f.%error-id%)-(error-expr)", transpileWithTypes(*parseResult.root)); +} + +TEST_CASE_FIXTURE(Fixture, "transpile_error_stat") +{ + std::string code = "-"; + + auto allocator = Allocator{}; + auto names = AstNameTable{allocator}; + ParseResult parseResult = Parser::parse(code.data(), code.size(), names, allocator, {}); + + CHECK_EQ("(error-stat: (error-expr))", transpileWithTypes(*parseResult.root)); +} + +TEST_CASE_FIXTURE(Fixture, "transpile_error_type") +{ + std::string code = "local a: "; + + auto allocator = Allocator{}; + auto names = AstNameTable{allocator}; + ParseResult parseResult = Parser::parse(code.data(), code.size(), names, allocator, {}); + + CHECK_EQ("local a:%error-type%", transpileWithTypes(*parseResult.root)); +} + +TEST_CASE_FIXTURE(Fixture, "transpile_parse_error") +{ + std::string code = "local a = -"; + + auto result = transpile(code); + CHECK_EQ("", result.code); + CHECK_EQ("Expected identifier when parsing expression, got ", result.parseError); +} + +TEST_CASE_FIXTURE(Fixture, "transpile_to_string") +{ + std::string code = "local a: string = 'hello'"; + + auto allocator = Allocator{}; + auto names = AstNameTable{allocator}; + ParseResult parseResult = Parser::parse(code.data(), code.size(), names, allocator, {}); + + REQUIRE(parseResult.root); + REQUIRE(parseResult.root->body.size == 1); + AstStatLocal* statLocal = parseResult.root->body.data[0]->as(); + REQUIRE(statLocal); + CHECK_EQ("local a: string = 'hello'", toString(statLocal)); + REQUIRE(statLocal->vars.size == 1); + AstLocal* local = statLocal->vars.data[0]; + REQUIRE(local->annotation); + CHECK_EQ("string", toString(local->annotation)); + REQUIRE(statLocal->values.size == 1); + AstExpr* expr = statLocal->values.data[0]; + CHECK_EQ("'hello'", toString(expr)); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.aliases.test.cpp b/tests/TypeInfer.aliases.test.cpp index f580604c..c27f8083 100644 --- a/tests/TypeInfer.aliases.test.cpp +++ b/tests/TypeInfer.aliases.test.cpp @@ -247,9 +247,6 @@ TEST_CASE_FIXTURE(Fixture, "export_type_and_type_alias_are_duplicates") TEST_CASE_FIXTURE(Fixture, "stringify_optional_parameterized_alias") { - ScopedFastFlag sffs3{"LuauGenericFunctions", true}; - ScopedFastFlag sffs4{"LuauParseGenericFunctions", true}; - CheckResult result = check(R"( type Node = { value: T, child: Node? } diff --git a/tests/TypeInfer.builtins.test.cpp b/tests/TypeInfer.builtins.test.cpp index 17e32e9f..1e2eae14 100644 --- a/tests/TypeInfer.builtins.test.cpp +++ b/tests/TypeInfer.builtins.test.cpp @@ -444,8 +444,6 @@ TEST_CASE_FIXTURE(Fixture, "os_time_takes_optional_date_table") TEST_CASE_FIXTURE(Fixture, "thread_is_a_type") { - ScopedFastFlag sff{"LuauDontMutatePersistentFunctions", true}; - CheckResult result = check(R"( local co = coroutine.create(function() end) )"); @@ -456,8 +454,6 @@ TEST_CASE_FIXTURE(Fixture, "thread_is_a_type") TEST_CASE_FIXTURE(Fixture, "coroutine_resume_anything_goes") { - ScopedFastFlag sff{"LuauDontMutatePersistentFunctions", true}; - CheckResult result = check(R"( local function nifty(x, y) print(x, y) @@ -476,8 +472,6 @@ TEST_CASE_FIXTURE(Fixture, "coroutine_resume_anything_goes") TEST_CASE_FIXTURE(Fixture, "coroutine_wrap_anything_goes") { - ScopedFastFlag sff{"LuauDontMutatePersistentFunctions", true}; - CheckResult result = check(R"( --!nonstrict local function nifty(x, y) @@ -822,8 +816,6 @@ TEST_CASE_FIXTURE(Fixture, "string_format_report_all_type_errors_at_correct_posi TEST_CASE_FIXTURE(Fixture, "dont_add_definitions_to_persistent_types") { - ScopedFastFlag sff{"LuauDontMutatePersistentFunctions", true}; - CheckResult result = check(R"( local f = math.sin local function g(x) return math.sin(x) end diff --git a/tests/TypeInfer.classes.test.cpp b/tests/TypeInfer.classes.test.cpp index eabf7e65..1ff23fe6 100644 --- a/tests/TypeInfer.classes.test.cpp +++ b/tests/TypeInfer.classes.test.cpp @@ -232,8 +232,6 @@ TEST_CASE_FIXTURE(ClassFixture, "can_assign_to_prop_of_base_class") TEST_CASE_FIXTURE(ClassFixture, "can_read_prop_of_base_class_using_string") { - ScopedFastFlag luauClassPropertyAccessAsString("LuauClassPropertyAccessAsString", true); - CheckResult result = check(R"( local c = ChildClass.New() local x = 1 + c["BaseField"] @@ -244,8 +242,6 @@ TEST_CASE_FIXTURE(ClassFixture, "can_read_prop_of_base_class_using_string") TEST_CASE_FIXTURE(ClassFixture, "can_assign_to_prop_of_base_class_using_string") { - ScopedFastFlag luauClassPropertyAccessAsString("LuauClassPropertyAccessAsString", true); - CheckResult result = check(R"( local c = ChildClass.New() c["BaseField"] = 444 @@ -451,4 +447,25 @@ b.X = 2 -- real Vector2.X is also read-only CHECK_EQ("Value of type 'Vector2?' could be nil", toString(result.errors[3])); } +TEST_CASE_FIXTURE(ClassFixture, "detailed_class_unification_error") +{ + ScopedFastFlag luauExtendedClassMismatchError{"LuauExtendedClassMismatchError", true}; + + CheckResult result = check(R"( +local function foo(v) + return v.X :: number + string.len(v.Y) +end + +local a: Vector2 +local b = foo +b(a) + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ(R"(Type 'Vector2' could not be converted into '{- X: a, Y: string -}' +caused by: + Property 'Y' is not compatible. Type 'number' could not be converted into 'string')", + toString(result.errors[0])); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.definitions.test.cpp b/tests/TypeInfer.definitions.test.cpp index 41e3e45a..2652486b 100644 --- a/tests/TypeInfer.definitions.test.cpp +++ b/tests/TypeInfer.definitions.test.cpp @@ -171,9 +171,6 @@ TEST_CASE_FIXTURE(Fixture, "no_cyclic_defined_classes") TEST_CASE_FIXTURE(Fixture, "declaring_generic_functions") { - ScopedFastFlag sffs{"LuauGenericFunctions", true}; - ScopedFastFlag sffs4{"LuauParseGenericFunctions", true}; - loadDefinition(R"( declare function f(a: a, b: b): string declare function g(...: a...): b... diff --git a/tests/TypeInfer.generics.test.cpp b/tests/TypeInfer.generics.test.cpp index 581375a1..de2f0154 100644 --- a/tests/TypeInfer.generics.test.cpp +++ b/tests/TypeInfer.generics.test.cpp @@ -13,8 +13,6 @@ TEST_SUITE_BEGIN("GenericsTests"); TEST_CASE_FIXTURE(Fixture, "check_generic_function") { - ScopedFastFlag sffs{"LuauGenericFunctions", true}; - ScopedFastFlag sffs2{"LuauParseGenericFunctions", true}; CheckResult result = check(R"( function id(x:a): a return x @@ -27,8 +25,6 @@ TEST_CASE_FIXTURE(Fixture, "check_generic_function") TEST_CASE_FIXTURE(Fixture, "check_generic_local_function") { - ScopedFastFlag sffs{"LuauGenericFunctions", true}; - ScopedFastFlag sffs2{"LuauParseGenericFunctions", true}; CheckResult result = check(R"( local function id(x:a): a return x @@ -41,10 +37,6 @@ TEST_CASE_FIXTURE(Fixture, "check_generic_local_function") TEST_CASE_FIXTURE(Fixture, "check_generic_typepack_function") { - ScopedFastFlag sffs{"LuauGenericFunctions", true}; - ScopedFastFlag sffs4{"LuauGenericVariadicsUnification", true}; - ScopedFastFlag sffs5{"LuauParseGenericFunctions", true}; - CheckResult result = check(R"( function id(...: a...): (a...) return ... end local x: string, y: boolean = id("hi", true) @@ -56,8 +48,6 @@ TEST_CASE_FIXTURE(Fixture, "check_generic_typepack_function") TEST_CASE_FIXTURE(Fixture, "types_before_typepacks") { - ScopedFastFlag sffs{"LuauGenericFunctions", true}; - ScopedFastFlag sffs2{"LuauParseGenericFunctions", true}; CheckResult result = check(R"( function f() end )"); @@ -66,8 +56,6 @@ TEST_CASE_FIXTURE(Fixture, "types_before_typepacks") TEST_CASE_FIXTURE(Fixture, "local_vars_can_be_polytypes") { - ScopedFastFlag sffs{"LuauGenericFunctions", true}; - ScopedFastFlag sffs2{"LuauParseGenericFunctions", true}; CheckResult result = check(R"( local function id(x:a):a return x end local f: (a)->a = id @@ -79,7 +67,6 @@ TEST_CASE_FIXTURE(Fixture, "local_vars_can_be_polytypes") TEST_CASE_FIXTURE(Fixture, "inferred_local_vars_can_be_polytypes") { - ScopedFastFlag sffs{"LuauGenericFunctions", true}; CheckResult result = check(R"( local function id(x) return x end print("This is bogus") -- TODO: CLI-39916 @@ -92,7 +79,6 @@ TEST_CASE_FIXTURE(Fixture, "inferred_local_vars_can_be_polytypes") TEST_CASE_FIXTURE(Fixture, "local_vars_can_be_instantiated_polytypes") { - ScopedFastFlag sffs{"LuauGenericFunctions", true}; CheckResult result = check(R"( local function id(x) return x end print("This is bogus") -- TODO: CLI-39916 @@ -104,8 +90,6 @@ TEST_CASE_FIXTURE(Fixture, "local_vars_can_be_instantiated_polytypes") TEST_CASE_FIXTURE(Fixture, "properties_can_be_polytypes") { - ScopedFastFlag sffs{"LuauGenericFunctions", true}; - ScopedFastFlag sffs2{"LuauParseGenericFunctions", true}; CheckResult result = check(R"( local t = {} t.m = function(x: a):a return x end @@ -117,8 +101,6 @@ TEST_CASE_FIXTURE(Fixture, "properties_can_be_polytypes") TEST_CASE_FIXTURE(Fixture, "properties_can_be_instantiated_polytypes") { - ScopedFastFlag sffs{"LuauGenericFunctions", true}; - ScopedFastFlag sffs2{"LuauParseGenericFunctions", true}; CheckResult result = check(R"( local t: { m: (number)->number } = { m = function(x:number) return x+1 end } local function id(x:a):a return x end @@ -129,8 +111,6 @@ TEST_CASE_FIXTURE(Fixture, "properties_can_be_instantiated_polytypes") TEST_CASE_FIXTURE(Fixture, "check_nested_generic_function") { - ScopedFastFlag sffs{"LuauGenericFunctions", true}; - ScopedFastFlag sffs2{"LuauParseGenericFunctions", true}; CheckResult result = check(R"( local function f() local function id(x:a): a @@ -145,8 +125,6 @@ TEST_CASE_FIXTURE(Fixture, "check_nested_generic_function") TEST_CASE_FIXTURE(Fixture, "check_recursive_generic_function") { - ScopedFastFlag sffs{"LuauGenericFunctions", true}; - ScopedFastFlag sffs2{"LuauParseGenericFunctions", true}; CheckResult result = check(R"( local function id(x:a):a local y: string = id("hi") @@ -159,8 +137,6 @@ TEST_CASE_FIXTURE(Fixture, "check_recursive_generic_function") TEST_CASE_FIXTURE(Fixture, "check_mutual_generic_functions") { - ScopedFastFlag sffs{"LuauGenericFunctions", true}; - ScopedFastFlag sffs2{"LuauParseGenericFunctions", true}; CheckResult result = check(R"( local id2 local function id1(x:a):a @@ -179,8 +155,6 @@ TEST_CASE_FIXTURE(Fixture, "check_mutual_generic_functions") TEST_CASE_FIXTURE(Fixture, "generic_functions_in_types") { - ScopedFastFlag sffs{"LuauGenericFunctions", true}; - ScopedFastFlag sffs2{"LuauParseGenericFunctions", true}; CheckResult result = check(R"( type T = { id: (a) -> a } local x: T = { id = function(x:a):a return x end } @@ -192,8 +166,6 @@ TEST_CASE_FIXTURE(Fixture, "generic_functions_in_types") TEST_CASE_FIXTURE(Fixture, "generic_factories") { - ScopedFastFlag sffs{"LuauGenericFunctions", true}; - ScopedFastFlag sffs2{"LuauParseGenericFunctions", true}; CheckResult result = check(R"( type T = { id: (a) -> a } type Factory = { build: () -> T } @@ -215,10 +187,6 @@ TEST_CASE_FIXTURE(Fixture, "generic_factories") TEST_CASE_FIXTURE(Fixture, "factories_of_generics") { - ScopedFastFlag sffs{"LuauGenericFunctions", true}; - ScopedFastFlag sffs2{"LuauParseGenericFunctions", true}; - ScopedFastFlag sffs3{"LuauRankNTypes", true}; - CheckResult result = check(R"( type T = { id: (a) -> a } type Factory = { build: () -> T } @@ -241,7 +209,6 @@ TEST_CASE_FIXTURE(Fixture, "factories_of_generics") TEST_CASE_FIXTURE(Fixture, "infer_generic_function") { - ScopedFastFlag sffs{"LuauGenericFunctions", true}; CheckResult result = check(R"( function id(x) return x @@ -265,7 +232,6 @@ TEST_CASE_FIXTURE(Fixture, "infer_generic_function") TEST_CASE_FIXTURE(Fixture, "infer_generic_local_function") { - ScopedFastFlag sffs{"LuauGenericFunctions", true}; CheckResult result = check(R"( local function id(x) return x @@ -289,7 +255,6 @@ TEST_CASE_FIXTURE(Fixture, "infer_generic_local_function") TEST_CASE_FIXTURE(Fixture, "infer_nested_generic_function") { - ScopedFastFlag sffs{"LuauGenericFunctions", true}; CheckResult result = check(R"( local function f() local function id(x) @@ -304,7 +269,6 @@ TEST_CASE_FIXTURE(Fixture, "infer_nested_generic_function") TEST_CASE_FIXTURE(Fixture, "infer_generic_methods") { - ScopedFastFlag sffs{"LuauGenericFunctions", true}; CheckResult result = check(R"( local x = {} function x:id(x) return x end @@ -316,7 +280,6 @@ TEST_CASE_FIXTURE(Fixture, "infer_generic_methods") TEST_CASE_FIXTURE(Fixture, "calling_self_generic_methods") { - ScopedFastFlag sffs{"LuauGenericFunctions", true}; CheckResult result = check(R"( local x = {} function x:id(x) return x end @@ -331,8 +294,6 @@ TEST_CASE_FIXTURE(Fixture, "calling_self_generic_methods") TEST_CASE_FIXTURE(Fixture, "infer_generic_property") { - ScopedFastFlag sffs{"LuauGenericFunctions", true}; - ScopedFastFlag sffs2{"LuauRankNTypes", true}; CheckResult result = check(R"( local t = {} t.m = function(x) return x end @@ -344,9 +305,6 @@ TEST_CASE_FIXTURE(Fixture, "infer_generic_property") TEST_CASE_FIXTURE(Fixture, "function_arguments_can_be_polytypes") { - ScopedFastFlag sffs{"LuauGenericFunctions", true}; - ScopedFastFlag sffs2{"LuauParseGenericFunctions", true}; - ScopedFastFlag sffs3{"LuauRankNTypes", true}; CheckResult result = check(R"( local function f(g: (a)->a) local x: number = g(37) @@ -358,9 +316,6 @@ TEST_CASE_FIXTURE(Fixture, "function_arguments_can_be_polytypes") TEST_CASE_FIXTURE(Fixture, "function_results_can_be_polytypes") { - ScopedFastFlag sffs{"LuauGenericFunctions", true}; - ScopedFastFlag sffs2{"LuauParseGenericFunctions", true}; - ScopedFastFlag sffs3{"LuauRankNTypes", true}; CheckResult result = check(R"( local function f() : (a)->a local function id(x:a):a return x end @@ -372,9 +327,6 @@ TEST_CASE_FIXTURE(Fixture, "function_results_can_be_polytypes") TEST_CASE_FIXTURE(Fixture, "type_parameters_can_be_polytypes") { - ScopedFastFlag sffs1{"LuauGenericFunctions", true}; - ScopedFastFlag sffs2{"LuauParseGenericFunctions", true}; - ScopedFastFlag sffs3{"LuauRankNTypes", true}; CheckResult result = check(R"( local function id(x:a):a return x end local f: (a)->a = id(id) @@ -384,7 +336,6 @@ TEST_CASE_FIXTURE(Fixture, "type_parameters_can_be_polytypes") TEST_CASE_FIXTURE(Fixture, "dont_leak_generic_types") { - ScopedFastFlag sffs{"LuauGenericFunctions", true}; CheckResult result = check(R"( local function f(y) -- this will only typecheck if we infer z: any @@ -406,7 +357,6 @@ TEST_CASE_FIXTURE(Fixture, "dont_leak_generic_types") TEST_CASE_FIXTURE(Fixture, "dont_leak_inferred_generic_types") { - ScopedFastFlag sffs{"LuauGenericFunctions", true}; CheckResult result = check(R"( local function f(y) local z = y @@ -423,12 +373,6 @@ TEST_CASE_FIXTURE(Fixture, "dont_leak_inferred_generic_types") TEST_CASE_FIXTURE(Fixture, "dont_substitute_bound_types") { - ScopedFastFlag sffs[] = { - {"LuauGenericFunctions", true}, - {"LuauParseGenericFunctions", true}, - {"LuauRankNTypes", true}, - }; - CheckResult result = check(R"( type T = { m: (a) -> T } function f(t : T) @@ -440,10 +384,6 @@ TEST_CASE_FIXTURE(Fixture, "dont_substitute_bound_types") TEST_CASE_FIXTURE(Fixture, "dont_unify_bound_types") { - ScopedFastFlag sffs{"LuauGenericFunctions", true}; - ScopedFastFlag sffs2{"LuauParseGenericFunctions", true}; - ScopedFastFlag sffs3{"LuauRankNTypes", true}; - CheckResult result = check(R"( type F = () -> (a, b) -> a type G = (b, b) -> b @@ -470,7 +410,6 @@ TEST_CASE_FIXTURE(Fixture, "mutable_state_polymorphism") // Replaying the classic problem with polymorphism and mutable state in Luau // See, e.g. Tofte (1990) // https://www.sciencedirect.com/science/article/pii/089054019090018D. - ScopedFastFlag sffs{"LuauGenericFunctions", true}; CheckResult result = check(R"( --!strict -- Our old friend the polymorphic identity function @@ -508,7 +447,6 @@ TEST_CASE_FIXTURE(Fixture, "mutable_state_polymorphism") TEST_CASE_FIXTURE(Fixture, "rank_N_types_via_typeof") { - ScopedFastFlag sffs{"LuauGenericFunctions", false}; CheckResult result = check(R"( --!strict local function id(x) return x end @@ -531,8 +469,6 @@ TEST_CASE_FIXTURE(Fixture, "rank_N_types_via_typeof") TEST_CASE_FIXTURE(Fixture, "duplicate_generic_types") { - ScopedFastFlag sffs{"LuauGenericFunctions", true}; - ScopedFastFlag sffs2{"LuauParseGenericFunctions", true}; CheckResult result = check(R"( function f(x:a):a return x end )"); @@ -541,7 +477,6 @@ TEST_CASE_FIXTURE(Fixture, "duplicate_generic_types") TEST_CASE_FIXTURE(Fixture, "duplicate_generic_type_packs") { - ScopedFastFlag sffs{"LuauGenericFunctions", true}; CheckResult result = check(R"( function f() end )"); @@ -550,7 +485,6 @@ TEST_CASE_FIXTURE(Fixture, "duplicate_generic_type_packs") TEST_CASE_FIXTURE(Fixture, "typepacks_before_types") { - ScopedFastFlag sffs{"LuauGenericFunctions", true}; CheckResult result = check(R"( function f() end )"); @@ -559,9 +493,6 @@ TEST_CASE_FIXTURE(Fixture, "typepacks_before_types") TEST_CASE_FIXTURE(Fixture, "variadic_generics") { - ScopedFastFlag sffs{"LuauGenericFunctions", true}; - ScopedFastFlag sffs3{"LuauParseGenericFunctions", true}; - CheckResult result = check(R"( function f(...: a) end @@ -573,9 +504,6 @@ TEST_CASE_FIXTURE(Fixture, "variadic_generics") TEST_CASE_FIXTURE(Fixture, "generic_type_pack_syntax") { - ScopedFastFlag sffs{"LuauGenericFunctions", true}; - ScopedFastFlag sffs4{"LuauParseGenericFunctions", true}; - CheckResult result = check(R"( function f(...: a...): (a...) return ... end )"); @@ -586,10 +514,6 @@ TEST_CASE_FIXTURE(Fixture, "generic_type_pack_syntax") TEST_CASE_FIXTURE(Fixture, "generic_type_pack_parentheses") { - ScopedFastFlag sffs{"LuauGenericFunctions", true}; - ScopedFastFlag sffs4{"LuauGenericVariadicsUnification", true}; - ScopedFastFlag sffs5{"LuauParseGenericFunctions", true}; - CheckResult result = check(R"( function f(...: a...): any return (...) end )"); @@ -599,9 +523,6 @@ TEST_CASE_FIXTURE(Fixture, "generic_type_pack_parentheses") TEST_CASE_FIXTURE(Fixture, "better_mismatch_error_messages") { - ScopedFastFlag sffs{"LuauGenericFunctions", true}; - ScopedFastFlag sffs5{"LuauParseGenericFunctions", true}; - CheckResult result = check(R"( function f(...: T...) return ... @@ -626,9 +547,6 @@ TEST_CASE_FIXTURE(Fixture, "better_mismatch_error_messages") TEST_CASE_FIXTURE(Fixture, "reject_clashing_generic_and_pack_names") { - ScopedFastFlag sffs{"LuauGenericFunctions", true}; - ScopedFastFlag sffs3{"LuauParseGenericFunctions", true}; - CheckResult result = check(R"( function f() end )"); @@ -641,8 +559,6 @@ TEST_CASE_FIXTURE(Fixture, "reject_clashing_generic_and_pack_names") TEST_CASE_FIXTURE(Fixture, "instantiation_sharing_types") { - ScopedFastFlag sffs1{"LuauGenericFunctions", true}; - CheckResult result = check(R"( function f(z) local o = {} @@ -665,8 +581,6 @@ TEST_CASE_FIXTURE(Fixture, "instantiation_sharing_types") TEST_CASE_FIXTURE(Fixture, "quantification_sharing_types") { - ScopedFastFlag sffs1{"LuauGenericFunctions", true}; - CheckResult result = check(R"( function f(x) return {5} end function g(x, y) return f(x) end @@ -680,8 +594,6 @@ TEST_CASE_FIXTURE(Fixture, "quantification_sharing_types") TEST_CASE_FIXTURE(Fixture, "typefuns_sharing_types") { - ScopedFastFlag sffs1{"LuauGenericFunctions", true}; - CheckResult result = check(R"( type T = { x: {a}, y: {number} } local o1: T = { x = {true}, y = {5} } @@ -697,7 +609,6 @@ TEST_CASE_FIXTURE(Fixture, "typefuns_sharing_types") TEST_CASE_FIXTURE(Fixture, "bound_tables_do_not_clone_original_fields") { - ScopedFastFlag luauRankNTypes{"LuauRankNTypes", true}; ScopedFastFlag luauCloneBoundTables{"LuauCloneBoundTables", true}; CheckResult result = check(R"( diff --git a/tests/TypeInfer.intersectionTypes.test.cpp b/tests/TypeInfer.intersectionTypes.test.cpp index 9685f4f3..893bc2b3 100644 --- a/tests/TypeInfer.intersectionTypes.test.cpp +++ b/tests/TypeInfer.intersectionTypes.test.cpp @@ -341,4 +341,43 @@ TEST_CASE_FIXTURE(Fixture, "table_intersection_setmetatable") LUAU_REQUIRE_NO_ERRORS(result); } +TEST_CASE_FIXTURE(Fixture, "error_detailed_intersection_part") +{ + ScopedFastFlag luauExtendedTypeMismatchError{"LuauExtendedTypeMismatchError", true}; + + CheckResult result = check(R"( +type X = { x: number } +type Y = { y: number } +type Z = { z: number } + +type XYZ = X & Y & Z + +local a: XYZ = 3 + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ(toString(result.errors[0]), R"(Type 'number' could not be converted into 'X & Y & Z' +caused by: + Not all intersection parts are compatible. Type 'number' could not be converted into 'X')"); +} + +TEST_CASE_FIXTURE(Fixture, "error_detailed_intersection_all") +{ + ScopedFastFlag luauExtendedTypeMismatchError{"LuauExtendedTypeMismatchError", true}; + + CheckResult result = check(R"( +type X = { x: number } +type Y = { y: number } +type Z = { z: number } + +type XYZ = X & Y & Z + +local a: XYZ +local b: number = a + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ(toString(result.errors[0]), R"(Type 'X & Y & Z' could not be converted into 'number'; none of the intersection parts are compatible)"); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.provisional.test.cpp b/tests/TypeInfer.provisional.test.cpp index 419da8ad..e5c14dde 100644 --- a/tests/TypeInfer.provisional.test.cpp +++ b/tests/TypeInfer.provisional.test.cpp @@ -194,9 +194,6 @@ TEST_CASE_FIXTURE(Fixture, "normal_conditional_expression_has_refinements") // Luau currently doesn't yet know how to allow assignments when the binding was refined. TEST_CASE_FIXTURE(Fixture, "while_body_are_also_refined") { - ScopedFastFlag sffs2{"LuauGenericFunctions", true}; - ScopedFastFlag sffs5{"LuauParseGenericFunctions", true}; - CheckResult result = check(R"( type Node = { value: T, child: Node? } @@ -596,11 +593,9 @@ TEST_CASE_FIXTURE(Fixture, "invariant_table_properties_means_instantiating_table TEST_CASE_FIXTURE(Fixture, "self_recursive_instantiated_param") { ScopedFastFlag luauCloneCorrectlyBeforeMutatingTableType{"LuauCloneCorrectlyBeforeMutatingTableType", true}; - ScopedFastFlag luauFollowInTypeFunApply{"LuauFollowInTypeFunApply", true}; - ScopedFastFlag luauInstantiatedTypeParamRecursion{"LuauInstantiatedTypeParamRecursion", true}; // Mutability in type function application right now can create strange recursive types - // TODO: instantiation right now is problematic, it this example should either leave the Table type alone + // TODO: instantiation right now is problematic, in this example should either leave the Table type alone // or it should rename the type to 'Self' so that the result will be 'Self
' CheckResult result = check(R"( type Table = { a: number } diff --git a/tests/TypeInfer.refinements.test.cpp b/tests/TypeInfer.refinements.test.cpp index 36dcaa95..733fc39b 100644 --- a/tests/TypeInfer.refinements.test.cpp +++ b/tests/TypeInfer.refinements.test.cpp @@ -7,7 +7,6 @@ #include "doctest.h" LUAU_FASTFLAG(LuauWeakEqConstraint) -LUAU_FASTFLAG(LuauOrPredicate) LUAU_FASTFLAG(LuauQuantifyInPlace2) using namespace Luau; @@ -133,11 +132,8 @@ TEST_CASE_FIXTURE(Fixture, "or_predicate_with_truthy_predicates") CHECK_EQ("string?", toString(requireTypeAtPosition({3, 26}))); CHECK_EQ("number?", toString(requireTypeAtPosition({4, 26}))); - if (FFlag::LuauOrPredicate) - { - CHECK_EQ("nil", toString(requireTypeAtPosition({6, 26}))); - CHECK_EQ("nil", toString(requireTypeAtPosition({7, 26}))); - } + CHECK_EQ("nil", toString(requireTypeAtPosition({6, 26}))); + CHECK_EQ("nil", toString(requireTypeAtPosition({7, 26}))); } TEST_CASE_FIXTURE(Fixture, "type_assertion_expr_carry_its_constraints") @@ -283,6 +279,8 @@ TEST_CASE_FIXTURE(Fixture, "assert_non_binary_expressions_actually_resolve_const TEST_CASE_FIXTURE(Fixture, "assign_table_with_refined_property_with_a_similar_type_is_illegal") { + ScopedFastFlag luauTableSubtypingVariance{"LuauTableSubtypingVariance", true}; + ScopedFastFlag luauExtendedTypeMismatchError{"LuauExtendedTypeMismatchError", true}; CheckResult result = check(R"( local t: {x: number?} = {x = nil} @@ -293,7 +291,10 @@ TEST_CASE_FIXTURE(Fixture, "assign_table_with_refined_property_with_a_similar_ty )"); LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK_EQ("Type '{| x: number? |}' could not be converted into '{| x: number |}'", toString(result.errors[0])); + CHECK_EQ(R"(Type '{| x: number? |}' could not be converted into '{| x: number |}' +caused by: + Property 'x' is not compatible. Type 'number?' could not be converted into 'number')", + toString(result.errors[0])); } TEST_CASE_FIXTURE(Fixture, "lvalue_is_equal_to_another_lvalue") @@ -749,8 +750,6 @@ TEST_CASE_FIXTURE(RefinementClassFixture, "type_narrow_for_all_the_userdata") TEST_CASE_FIXTURE(RefinementClassFixture, "eliminate_subclasses_of_instance") { - ScopedFastFlag sff{"LuauTypeGuardPeelsAwaySubclasses", true}; - CheckResult result = check(R"( local function f(x: Part | Folder | string) if typeof(x) == "Instance" then @@ -769,8 +768,6 @@ TEST_CASE_FIXTURE(RefinementClassFixture, "eliminate_subclasses_of_instance") TEST_CASE_FIXTURE(RefinementClassFixture, "narrow_this_large_union") { - ScopedFastFlag sff{"LuauTypeGuardPeelsAwaySubclasses", true}; - CheckResult result = check(R"( local function f(x: Part | Folder | Instance | string | Vector3 | any) if typeof(x) == "Instance" then @@ -789,8 +786,6 @@ TEST_CASE_FIXTURE(RefinementClassFixture, "narrow_this_large_union") TEST_CASE_FIXTURE(RefinementClassFixture, "x_as_any_if_x_is_instance_elseif_x_is_table") { - ScopedFastFlag sff{"LuauOrPredicate", true}; - CheckResult result = check(R"( --!nonstrict @@ -811,11 +806,6 @@ TEST_CASE_FIXTURE(RefinementClassFixture, "x_as_any_if_x_is_instance_elseif_x_is TEST_CASE_FIXTURE(RefinementClassFixture, "x_is_not_instance_or_else_not_part") { - ScopedFastFlag sffs[] = { - {"LuauOrPredicate", true}, - {"LuauTypeGuardPeelsAwaySubclasses", true}, - }; - CheckResult result = check(R"( local function f(x: Part | Folder | string) if typeof(x) ~= "Instance" or not x:IsA("Part") then @@ -890,8 +880,6 @@ TEST_CASE_FIXTURE(Fixture, "type_guard_warns_on_no_overlapping_types_only_when_s TEST_CASE_FIXTURE(Fixture, "not_a_or_not_b") { - ScopedFastFlag sff{"LuauOrPredicate", true}; - CheckResult result = check(R"( local function f(a: number?, b: number?) if (not a) or (not b) then @@ -909,8 +897,6 @@ TEST_CASE_FIXTURE(Fixture, "not_a_or_not_b") TEST_CASE_FIXTURE(Fixture, "not_a_or_not_b2") { - ScopedFastFlag sff{"LuauOrPredicate", true}; - CheckResult result = check(R"( local function f(a: number?, b: number?) if not (a and b) then @@ -928,8 +914,6 @@ TEST_CASE_FIXTURE(Fixture, "not_a_or_not_b2") TEST_CASE_FIXTURE(Fixture, "not_a_and_not_b") { - ScopedFastFlag sff{"LuauOrPredicate", true}; - CheckResult result = check(R"( local function f(a: number?, b: number?) if (not a) and (not b) then @@ -947,8 +931,6 @@ TEST_CASE_FIXTURE(Fixture, "not_a_and_not_b") TEST_CASE_FIXTURE(Fixture, "not_a_and_not_b2") { - ScopedFastFlag sff{"LuauOrPredicate", true}; - CheckResult result = check(R"( local function f(a: number?, b: number?) if not (a or b) then @@ -966,8 +948,6 @@ TEST_CASE_FIXTURE(Fixture, "not_a_and_not_b2") TEST_CASE_FIXTURE(Fixture, "either_number_or_string") { - ScopedFastFlag sff{"LuauOrPredicate", true}; - CheckResult result = check(R"( local function f(x: any) if type(x) == "number" or type(x) == "string" then @@ -983,8 +963,6 @@ TEST_CASE_FIXTURE(Fixture, "either_number_or_string") TEST_CASE_FIXTURE(Fixture, "not_t_or_some_prop_of_t") { - ScopedFastFlag sff{"LuauOrPredicate", true}; - CheckResult result = check(R"( local function f(t: {x: boolean}?) if not t or t.x then @@ -1000,8 +978,6 @@ TEST_CASE_FIXTURE(Fixture, "not_t_or_some_prop_of_t") TEST_CASE_FIXTURE(Fixture, "assert_a_to_be_truthy_then_assert_a_to_be_number") { - ScopedFastFlag sff{"LuauOrPredicate", true}; - CheckResult result = check(R"( local a: (number | string)? assert(a) @@ -1018,8 +994,6 @@ TEST_CASE_FIXTURE(Fixture, "assert_a_to_be_truthy_then_assert_a_to_be_number") TEST_CASE_FIXTURE(Fixture, "merge_should_be_fully_agnostic_of_hashmap_ordering") { - ScopedFastFlag sff{"LuauOrPredicate", true}; - // This bug came up because there was a mistake in Luau::merge where zipping on two maps would produce the wrong merged result. CheckResult result = check(R"( local function f(b: string | { x: string }, a) @@ -1039,8 +1013,6 @@ TEST_CASE_FIXTURE(Fixture, "merge_should_be_fully_agnostic_of_hashmap_ordering") TEST_CASE_FIXTURE(Fixture, "refine_the_correct_types_opposite_of_when_a_is_not_number_or_string") { - ScopedFastFlag sff{"LuauOrPredicate", true}; - CheckResult result = check(R"( local function f(a: string | number | boolean) if type(a) ~= "number" and type(a) ~= "string" then diff --git a/tests/TypeInfer.tables.test.cpp b/tests/TypeInfer.tables.test.cpp index f1451a81..c3694be7 100644 --- a/tests/TypeInfer.tables.test.cpp +++ b/tests/TypeInfer.tables.test.cpp @@ -1950,4 +1950,76 @@ TEST_CASE_FIXTURE(Fixture, "table_insert_should_cope_with_optional_properties_in LUAU_REQUIRE_NO_ERRORS(result); } +TEST_CASE_FIXTURE(Fixture, "error_detailed_prop") +{ + ScopedFastFlag luauTableSubtypingVariance{"LuauTableSubtypingVariance", true}; // Only for new path + ScopedFastFlag luauExtendedTypeMismatchError{"LuauExtendedTypeMismatchError", true}; + + CheckResult result = check(R"( +type A = { x: number, y: number } +type B = { x: number, y: string } + +local a: A +local b: B = a + )"); + + LUAU_REQUIRE_ERRORS(result); + CHECK_EQ(toString(result.errors[0]), R"(Type 'A' could not be converted into 'B' +caused by: + Property 'y' is not compatible. Type 'number' could not be converted into 'string')"); +} + +TEST_CASE_FIXTURE(Fixture, "error_detailed_prop_nested") +{ + ScopedFastFlag luauTableSubtypingVariance{"LuauTableSubtypingVariance", true}; // Only for new path + ScopedFastFlag luauExtendedTypeMismatchError{"LuauExtendedTypeMismatchError", true}; + + CheckResult result = check(R"( +type AS = { x: number, y: number } +type BS = { x: number, y: string } + +type A = { a: boolean, b: AS } +type B = { a: boolean, b: BS } + +local a: A +local b: B = a + )"); + + LUAU_REQUIRE_ERRORS(result); + CHECK_EQ(toString(result.errors[0]), R"(Type 'A' could not be converted into 'B' +caused by: + Property 'b' is not compatible. Type 'AS' could not be converted into 'BS' +caused by: + Property 'y' is not compatible. Type 'number' could not be converted into 'string')"); +} + +TEST_CASE_FIXTURE(Fixture, "error_detailed_metatable_prop") +{ + ScopedFastFlag luauTableSubtypingVariance{"LuauTableSubtypingVariance", true}; // Only for new path + ScopedFastFlag luauExtendedTypeMismatchError{"LuauExtendedTypeMismatchError", true}; + + CheckResult result = check(R"( +local a1 = setmetatable({ x = 2, y = 3 }, { __call = function(s) end }); +local b1 = setmetatable({ x = 2, y = "hello" }, { __call = function(s) end }); +local c1: typeof(a1) = b1 + +local a2 = setmetatable({ x = 2, y = 3 }, { __call = function(s) end }); +local b2 = setmetatable({ x = 2, y = 4 }, { __call = function(s, t) end }); +local c2: typeof(a2) = b2 + )"); + + LUAU_REQUIRE_ERROR_COUNT(2, result); + CHECK_EQ(toString(result.errors[0]), R"(Type 'b1' could not be converted into 'a1' +caused by: + Type '{| x: number, y: string |}' could not be converted into '{| x: number, y: number |}' +caused by: + Property 'y' is not compatible. Type 'string' could not be converted into 'number')"); + + CHECK_EQ(toString(result.errors[1]), R"(Type 'b2' could not be converted into 'a2' +caused by: + Type '{| __call: (a, b) -> () |}' could not be converted into '{| __call: (a) -> () |}' +caused by: + Property '__call' is not compatible. Type '(a, b) -> ()' could not be converted into '(a) -> ()')"); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.test.cpp b/tests/TypeInfer.test.cpp index 45381757..30d9130a 100644 --- a/tests/TypeInfer.test.cpp +++ b/tests/TypeInfer.test.cpp @@ -3926,8 +3926,6 @@ local b: number = 1 or a TEST_CASE_FIXTURE(Fixture, "no_lossy_function_type") { - ScopedFastFlag sffs2{"LuauGenericFunctions", true}; - CheckResult result = check(R"( --!strict local tbl = {} @@ -4493,10 +4491,6 @@ f(function(x) print(x) end) TEST_CASE_FIXTURE(Fixture, "infer_generic_function_function_argument") { - ScopedFastFlag luauGenericFunctions("LuauGenericFunctions", true); - ScopedFastFlag luauParseGenericFunctions("LuauParseGenericFunctions", true); - ScopedFastFlag luauRankNTypes("LuauRankNTypes", true); - CheckResult result = check(R"( local function sum(x: a, y: a, f: (a, a) -> a) return f(x, y) end return sum(2, 3, function(a, b) return a + b end) @@ -4525,10 +4519,6 @@ local r = foldl(a, {s=0,c=0}, function(a, b) return {s = a.s + b, c = a.c + 1} e TEST_CASE_FIXTURE(Fixture, "infer_generic_function_function_argument_overloaded") { - ScopedFastFlag luauGenericFunctions("LuauGenericFunctions", true); - ScopedFastFlag luauParseGenericFunctions("LuauParseGenericFunctions", true); - ScopedFastFlag luauRankNTypes("LuauRankNTypes", true); - CheckResult result = check(R"( local function g1(a: T, f: (T) -> T) return f(a) end local function g2(a: T, b: T, f: (T, T) -> T) return f(a, b) end @@ -4579,10 +4569,6 @@ local a: TableWithFunc = { x = 3, y = 4, f = function(a, b) return a + b end } TEST_CASE_FIXTURE(Fixture, "do_not_infer_generic_functions") { - ScopedFastFlag luauGenericFunctions("LuauGenericFunctions", true); - ScopedFastFlag luauParseGenericFunctions("LuauParseGenericFunctions", true); - ScopedFastFlag luauRankNTypes("LuauRankNTypes", true); - CheckResult result = check(R"( local function sum(x: a, y: a, f: (a, a) -> a) return f(x, y) end @@ -4600,8 +4586,6 @@ local c = sumrec(function(x, y, f) return f(x, y) end) -- type binders are not i TEST_CASE_FIXTURE(Fixture, "infer_return_value_type") { - ScopedFastFlag luauInferReturnAssertAssign("LuauInferReturnAssertAssign", true); - CheckResult result = check(R"( local function f(): {string|number} return {1, "b", 3} @@ -4625,8 +4609,6 @@ end TEST_CASE_FIXTURE(Fixture, "infer_type_assertion_value_type") { - ScopedFastFlag luauInferReturnAssertAssign("LuauInferReturnAssertAssign", true); - CheckResult result = check(R"( local function f() return {4, "b", 3} :: {string|number} @@ -4638,8 +4620,6 @@ end TEST_CASE_FIXTURE(Fixture, "infer_assignment_value_types") { - ScopedFastFlag luauInferReturnAssertAssign("LuauInferReturnAssertAssign", true); - CheckResult result = check(R"( local a: (number, number) -> number = function(a, b) return a - b end @@ -4655,8 +4635,6 @@ b, c = {2, "s"}, {"b", 4} TEST_CASE_FIXTURE(Fixture, "infer_assignment_value_types_mutable_lval") { - ScopedFastFlag luauInferReturnAssertAssign("LuauInferReturnAssertAssign", true); - CheckResult result = check(R"( local a = {} a.x = 2 @@ -4668,8 +4646,6 @@ a = setmetatable(a, { __call = function(x) end }) TEST_CASE_FIXTURE(Fixture, "refine_and_or") { - ScopedFastFlag sff{"LuauSlightlyMoreFlexibleBinaryPredicates", true}; - CheckResult result = check(R"( local t: {x: number?}? = {x = nil} local u = t and t.x or 5 @@ -4682,10 +4658,6 @@ TEST_CASE_FIXTURE(Fixture, "refine_and_or") TEST_CASE_FIXTURE(Fixture, "checked_prop_too_early") { - ScopedFastFlag sffs[] = { - {"LuauSlightlyMoreFlexibleBinaryPredicates", true}, - }; - CheckResult result = check(R"( local t: {x: number?}? = {x = nil} local u = t.x and t or 5 @@ -4698,10 +4670,6 @@ TEST_CASE_FIXTURE(Fixture, "checked_prop_too_early") TEST_CASE_FIXTURE(Fixture, "accidentally_checked_prop_in_opposite_branch") { - ScopedFastFlag sffs[] = { - {"LuauSlightlyMoreFlexibleBinaryPredicates", true}, - }; - CheckResult result = check(R"( local t: {x: number?}? = {x = nil} local u = t and t.x == 5 or t.x == 31337 @@ -4714,7 +4682,7 @@ TEST_CASE_FIXTURE(Fixture, "accidentally_checked_prop_in_opposite_branch") TEST_CASE_FIXTURE(Fixture, "substitution_with_bound_table") { - ScopedFastFlag luauFollowInTypeFunApply("LuauFollowInTypeFunApply", true); + ScopedFastFlag luauCloneCorrectlyBeforeMutatingTableType{"LuauCloneCorrectlyBeforeMutatingTableType", true}; CheckResult result = check(R"( type A = { x: number } diff --git a/tests/TypeInfer.tryUnify.test.cpp b/tests/TypeInfer.tryUnify.test.cpp index 1192a8ac..2d697fc9 100644 --- a/tests/TypeInfer.tryUnify.test.cpp +++ b/tests/TypeInfer.tryUnify.test.cpp @@ -178,9 +178,6 @@ TEST_CASE_FIXTURE(TryUnifyFixture, "unifying_variadic_pack_with_error_should_wor TEST_CASE_FIXTURE(TryUnifyFixture, "variadics_should_use_reversed_properly") { - ScopedFastFlag sffs2{"LuauGenericFunctions", true}; - ScopedFastFlag sffs4{"LuauParseGenericFunctions", true}; - CheckResult result = check(R"( --!strict local function f(...: T): ...T @@ -199,8 +196,6 @@ TEST_CASE_FIXTURE(TryUnifyFixture, "variadics_should_use_reversed_properly") TEST_CASE_FIXTURE(TryUnifyFixture, "cli_41095_concat_log_in_sealed_table_unification") { - ScopedFastFlag sffs2("LuauGenericFunctions", true); - CheckResult result = check(R"( --!strict table.insert() diff --git a/tests/TypeInfer.typePacks.cpp b/tests/TypeInfer.typePacks.cpp index 8dab2605..c6de0abf 100644 --- a/tests/TypeInfer.typePacks.cpp +++ b/tests/TypeInfer.typePacks.cpp @@ -296,7 +296,6 @@ end TEST_CASE_FIXTURE(Fixture, "type_alias_type_packs") { - ScopedFastFlag luauParseGenericFunctions("LuauParseGenericFunctions", true); ScopedFastFlag luauTypeAliasPacks("LuauTypeAliasPacks", true); ScopedFastFlag luauParseTypePackTypeParameters("LuauParseTypePackTypeParameters", true); @@ -361,7 +360,6 @@ local c: Packed TEST_CASE_FIXTURE(Fixture, "type_alias_type_packs_import") { - ScopedFastFlag luauParseGenericFunctions("LuauParseGenericFunctions", true); ScopedFastFlag luauTypeAliasPacks("LuauTypeAliasPacks", true); ScopedFastFlag luauParseTypePackTypeParameters("LuauParseTypePackTypeParameters", true); @@ -395,7 +393,6 @@ local d: { a: typeof(c) } TEST_CASE_FIXTURE(Fixture, "type_pack_type_parameters") { - ScopedFastFlag luauParseGenericFunctions("LuauParseGenericFunctions", true); ScopedFastFlag luauTypeAliasPacks("LuauTypeAliasPacks", true); ScopedFastFlag luauParseTypePackTypeParameters("LuauParseTypePackTypeParameters", true); @@ -434,7 +431,6 @@ type C = Import.Packed TEST_CASE_FIXTURE(Fixture, "type_alias_type_packs_nested") { - ScopedFastFlag luauParseGenericFunctions("LuauParseGenericFunctions", true); ScopedFastFlag luauTypeAliasPacks("LuauTypeAliasPacks", true); ScopedFastFlag luauParseTypePackTypeParameters("LuauParseTypePackTypeParameters", true); @@ -456,7 +452,6 @@ type Packed4 = (Packed3, T...) -> (Packed3, T...) TEST_CASE_FIXTURE(Fixture, "type_alias_type_pack_variadic") { - ScopedFastFlag luauParseGenericFunctions("LuauParseGenericFunctions", true); ScopedFastFlag luauTypeAliasPacks("LuauTypeAliasPacks", true); ScopedFastFlag luauParseTypePackTypeParameters("LuauParseTypePackTypeParameters", true); @@ -475,7 +470,6 @@ type E = X<(number, ...string)> TEST_CASE_FIXTURE(Fixture, "type_alias_type_pack_multi") { - ScopedFastFlag luauParseGenericFunctions("LuauParseGenericFunctions", true); ScopedFastFlag luauTypeAliasPacks("LuauTypeAliasPacks", true); ScopedFastFlag luauParseTypePackTypeParameters("LuauParseTypePackTypeParameters", true); @@ -507,7 +501,6 @@ type I = W TEST_CASE_FIXTURE(Fixture, "type_alias_type_pack_explicit") { - ScopedFastFlag luauParseGenericFunctions("LuauParseGenericFunctions", true); ScopedFastFlag luauTypeAliasPacks("LuauTypeAliasPacks", true); ScopedFastFlag luauParseTypePackTypeParameters("LuauParseTypePackTypeParameters", true); @@ -534,7 +527,6 @@ type F = X<(string, ...number)> TEST_CASE_FIXTURE(Fixture, "type_alias_type_pack_explicit_multi") { - ScopedFastFlag luauParseGenericFunctions("LuauParseGenericFunctions", true); ScopedFastFlag luauTypeAliasPacks("LuauTypeAliasPacks", true); ScopedFastFlag luauParseTypePackTypeParameters("LuauParseTypePackTypeParameters", true); @@ -557,10 +549,8 @@ type D = Y TEST_CASE_FIXTURE(Fixture, "type_alias_type_pack_explicit_multi_tostring") { - ScopedFastFlag luauParseGenericFunctions("LuauParseGenericFunctions", true); ScopedFastFlag luauTypeAliasPacks("LuauTypeAliasPacks", true); ScopedFastFlag luauParseTypePackTypeParameters("LuauParseTypePackTypeParameters", true); - ScopedFastFlag luauInstantiatedTypeParamRecursion("LuauInstantiatedTypeParamRecursion", true); // For correct toString block CheckResult result = check(R"( type Y = { f: (T...) -> (U...) } @@ -577,7 +567,6 @@ local b: Y<(), ()> TEST_CASE_FIXTURE(Fixture, "type_alias_backwards_compatible") { - ScopedFastFlag luauParseGenericFunctions("LuauParseGenericFunctions", true); ScopedFastFlag luauTypeAliasPacks("LuauTypeAliasPacks", true); ScopedFastFlag luauParseTypePackTypeParameters("LuauParseTypePackTypeParameters", true); @@ -599,7 +588,6 @@ type C = Y<(number), boolean> TEST_CASE_FIXTURE(Fixture, "type_alias_type_packs_errors") { - ScopedFastFlag luauParseGenericFunctions("LuauParseGenericFunctions", true); ScopedFastFlag luauTypeAliasPacks("LuauTypeAliasPacks", true); ScopedFastFlag luauParseTypePackTypeParameters("LuauParseTypePackTypeParameters", true); diff --git a/tests/TypeInfer.unionTypes.test.cpp b/tests/TypeInfer.unionTypes.test.cpp index 34c25a9f..9f29b642 100644 --- a/tests/TypeInfer.unionTypes.test.cpp +++ b/tests/TypeInfer.unionTypes.test.cpp @@ -400,8 +400,6 @@ local e = a.z TEST_CASE_FIXTURE(Fixture, "unify_sealed_table_union_check") { - ScopedFastFlag luauSealedTableUnifyOptionalFix("LuauSealedTableUnifyOptionalFix", true); - CheckResult result = check(R"( local x: { x: number } = { x = 3 } type A = number? @@ -426,4 +424,43 @@ y = x LUAU_REQUIRE_NO_ERRORS(result); } +TEST_CASE_FIXTURE(Fixture, "error_detailed_union_part") +{ + ScopedFastFlag luauExtendedTypeMismatchError{"LuauExtendedTypeMismatchError", true}; + + CheckResult result = check(R"( +type X = { x: number } +type Y = { y: number } +type Z = { z: number } + +type XYZ = X | Y | Z + +local a: XYZ +local b: { w: number } = a + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ(toString(result.errors[0]), R"(Type 'X | Y | Z' could not be converted into '{| w: number |}' +caused by: + Not all union options are compatible. Table type 'X' not compatible with type '{| w: number |}' because the former is missing field 'w')"); +} + +TEST_CASE_FIXTURE(Fixture, "error_detailed_union_all") +{ + ScopedFastFlag luauExtendedTypeMismatchError{"LuauExtendedTypeMismatchError", true}; + + CheckResult result = check(R"( +type X = { x: number } +type Y = { y: number } +type Z = { z: number } + +type XYZ = X | Y | Z + +local a: XYZ = { w = 4 } + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ(toString(result.errors[0]), R"(Type 'a' could not be converted into 'X | Y | Z'; none of the union options are compatible)"); +} + TEST_SUITE_END(); diff --git a/tests/TypeVar.test.cpp b/tests/TypeVar.test.cpp index 930c1a39..91efa818 100644 --- a/tests/TypeVar.test.cpp +++ b/tests/TypeVar.test.cpp @@ -11,8 +11,6 @@ using namespace Luau; -LUAU_FASTFLAG(LuauGenericFunctions); - TEST_SUITE_BEGIN("TypeVarTests"); TEST_CASE_FIXTURE(Fixture, "primitives_are_equal") diff --git a/tests/conformance/bitwise.lua b/tests/conformance/bitwise.lua index 6efa5960..13be3f94 100644 --- a/tests/conformance/bitwise.lua +++ b/tests/conformance/bitwise.lua @@ -113,6 +113,20 @@ assert(bit32.replace(0, -1, 4) == 2^4) assert(bit32.replace(-1, 0, 31) == 2^31 - 1) assert(bit32.replace(-1, 0, 1, 2) == 2^32 - 7) +-- testing countlz/countrc +assert(bit32.countlz(0) == 32) +assert(bit32.countlz(42) == 26) +assert(bit32.countlz(0xffffffff) == 0) +assert(bit32.countlz(0x80000000) == 0) +assert(bit32.countlz(0x7fffffff) == 1) + +assert(bit32.countrz(0) == 32) +assert(bit32.countrz(1) == 0) +assert(bit32.countrz(42) == 1) +assert(bit32.countrz(0x80000000) == 31) +assert(bit32.countrz(0x40000000) == 30) +assert(bit32.countrz(0x7fffffff) == 0) + --[[ This test verifies a fix in luauF_replace() where if the 4th parameter was not a number, but the first three are numbers, it will @@ -136,5 +150,7 @@ assert(bit32.bxor("1", 3) == 2) assert(bit32.bxor(1, "3") == 2) assert(bit32.btest(1, "3") == true) assert(bit32.btest("1", 3) == true) +assert(bit32.countlz("42") == 26) +assert(bit32.countrz("42") == 1) return('OK') From 70ffc8a01d3f9fc72e232ed9c53af3e789b11eb9 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Fri, 12 Nov 2021 06:54:00 -0800 Subject: [PATCH 053/399] RFC: Do not allow method call on string literals (#145) --- rfcs/syntax-method-call-on-string-literals.md | 70 ------------------- 1 file changed, 70 deletions(-) delete mode 100644 rfcs/syntax-method-call-on-string-literals.md diff --git a/rfcs/syntax-method-call-on-string-literals.md b/rfcs/syntax-method-call-on-string-literals.md deleted file mode 100644 index 877774d0..00000000 --- a/rfcs/syntax-method-call-on-string-literals.md +++ /dev/null @@ -1,70 +0,0 @@ -# Allow method call on string literals - -> Note: this RFC was adapted from an internal proposal that predates RFC process - -## Summary - -Allow string literals to be indexed on without parentheses or from an identifier. That is, the following snippet will become legal under this proposal: - -```lua -print("Hello, %s!":format("world")) -print("0123456789ABCDEF":sub(i, i)) -``` - -## Motivation - -Experienced Lua developers occasionally run into this paper-cut even after years of working with the language. Programmers in Lua frequently wants to format a user-facing message using a constant string, but the parser will not accept it as legible syntax. - -## Design - -Formally, the proposal is to move the `String` parser from `exp` to `prefixexp`: - -```diff - var ::= Name | prefixexp `[´ exp `]´ | prefixexp `.´ Name -- exp ::= nil | false | true | Number | String | `...´ | function | -+ exp ::= nil | false | true | Number | `...´ | function - | prefixexp | tableconstructor | exp binop exp | unop exp -- prefixexp ::= var | functioncall | `(´ exp `)´ -+ prefixexp ::= String | var | functioncall | `(´ exp `)´ - functioncall ::= prefixexp args | prefixexp `:´ Name args -``` - -By itself, this change introduces an additional ambiguity because of the combination of non-significant whitespace and function calls with string literal as the first argument without the use of parentheses. - -Consider code like this: - -```lua -local foo = bar -("fmt"):format(...) -``` - -The grammar for this sequence suggests that the above is a function call to bar with a single string literal argument, "fmt", and format method is called on the result. This is a consequence of line endings not being significant, but humans don't read the code like this, and are likely to think that here, format is called on the string literal "fmt". - -Because of this, Lua 5.1 produces a syntax error whenever function call arguments start on the next line. Luau has the same error production rule; future versions of Lua remove this restriction but it's not clear that we want to remove this as this does help prevent errors. - -The grammar today also allows calling functions with string literals as their first (and only) argument without the use of parentheses; bar "fmt" is a function call. This is helpful when defining embedded domain-specific languages. - -By itself, this proposal thus would create a similar ambiguity in code like this: - -```lua -local foo = bar -"fmt":format(...) -``` - -While we could extend the line-based error check to include function literal arguments, this is not syntactically backwards compatible and as such may break existing code. A simpler and more conservative solution is to disallow string literal as the leading token of a new statement - there are no cases when this is valid today, so it's safe to limit this. - -Doing so would prohibit code like this: - -```lua -"fmt":format(...) -``` - -However, there are no methods on the string object where code like this would be meaningful. As such, in addition to changing the grammar wrt string literals, we will add an extra ambiguity error whenever a statement starts with a string literal. - -## Drawbacks - -None known. - -## Alternatives - -The infallible parser could be mended in this exact scenario to report a more friendly error message. We decided not to do this because there is more value to gain by simply supporting the main proposal. From b7d26b371a31795d6c4e2e164fce7375c764bcc3 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Fri, 12 Nov 2021 06:56:25 -0800 Subject: [PATCH 054/399] Use -Werror in CI only (#201) We keep getting compat reports for warnings in various compiler versions. While we can keep merging PRs to resolve these warnings, it would be nice if the users of other compilers or compiler versions weren't blocked on us fixing this. As such, this change disables Werror by default and only enables it when requested, which happens in CI in test builds. --- .github/workflows/build.yml | 8 ++++---- Analysis/src/TypeInfer.cpp | 4 ++-- Analysis/src/TypeVar.cpp | 2 +- CMakeLists.txt | 12 ++++++++++-- Makefile | 10 ++++++++-- 5 files changed, 25 insertions(+), 11 deletions(-) diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml index cdee2f6c..506a4c89 100644 --- a/.github/workflows/build.yml +++ b/.github/workflows/build.yml @@ -27,13 +27,13 @@ jobs: - uses: actions/checkout@v1 - name: make test run: | - make -j2 config=sanitize test + make -j2 config=sanitize werror=1 test - name: make test w/flags run: | - make -j2 config=sanitize flags=true test + make -j2 config=sanitize werror=1 flags=true test - name: make cli run: | - make -j2 config=sanitize luau luau-analyze # match config with tests to improve build time + make -j2 config=sanitize werror=1 luau luau-analyze # match config with tests to improve build time ./luau tests/conformance/assert.lua ./luau-analyze tests/conformance/assert.lua @@ -45,7 +45,7 @@ jobs: steps: - uses: actions/checkout@v1 - name: cmake configure - run: cmake . -A ${{matrix.arch}} + run: cmake . -A ${{matrix.arch}} -DLUAU_WERROR=ON - name: cmake test shell: bash # necessary for fail-fast run: | diff --git a/Analysis/src/TypeInfer.cpp b/Analysis/src/TypeInfer.cpp index a6696efd..96b79923 100644 --- a/Analysis/src/TypeInfer.cpp +++ b/Analysis/src/TypeInfer.cpp @@ -1509,7 +1509,7 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprVa std::vector types = flatten(varargPack).first; return {!types.empty() ? types[0] : nilType}; } - else if (auto ftp = get(varargPack)) + else if (get(varargPack)) { TypeId head = freshType(scope); TypePackId tail = freshTypePack(scope); @@ -1539,7 +1539,7 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprCa { return {pack->head.empty() ? nilType : pack->head[0], std::move(result.predicates)}; } - else if (auto ftp = get(retPack)) + else if (get(retPack)) { TypeId head = freshType(scope); TypePackId pack = addTypePack(TypePackVar{TypePack{{head}, freshTypePack(scope)}}); diff --git a/Analysis/src/TypeVar.cpp b/Analysis/src/TypeVar.cpp index cd447ca2..4ee5cb82 100644 --- a/Analysis/src/TypeVar.cpp +++ b/Analysis/src/TypeVar.cpp @@ -293,7 +293,7 @@ bool isGeneric(TypeId ty) bool maybeGeneric(TypeId ty) { ty = follow(ty); - if (auto ftv = get(ty)) + if (get(ty)) return true; else if (auto ttv = get(ty)) { diff --git a/CMakeLists.txt b/CMakeLists.txt index 36014a98..d9ef9c05 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -9,6 +9,7 @@ project(Luau LANGUAGES CXX) option(LUAU_BUILD_CLI "Build CLI" ON) option(LUAU_BUILD_TESTS "Build tests" ON) +option(LUAU_WERROR "Warnings as errors" OFF) add_library(Luau.Ast STATIC) add_library(Luau.Compiler STATIC) @@ -57,11 +58,18 @@ set(LUAU_OPTIONS) if(MSVC) list(APPEND LUAU_OPTIONS /D_CRT_SECURE_NO_WARNINGS) # We need to use the portable CRT functions. - list(APPEND LUAU_OPTIONS /WX) # Warnings are errors list(APPEND LUAU_OPTIONS /MP) # Distribute single project compilation across multiple cores else() list(APPEND LUAU_OPTIONS -Wall) # All warnings - list(APPEND LUAU_OPTIONS -Werror) # Warnings are errors +endif() + +# Enabled in CI; we should be warning free on our main compiler versions but don't guarantee being warning free everywhere +if(LUAU_WERROR) + if(MSVC) + list(APPEND LUAU_OPTIONS /WX) # Warnings are errors + else() + list(APPEND LUAU_OPTIONS -Werror) # Warnings are errors + endif() endif() target_compile_options(Luau.Ast PRIVATE ${LUAU_OPTIONS}) diff --git a/Makefile b/Makefile index ffe43092..ce461af7 100644 --- a/Makefile +++ b/Makefile @@ -46,14 +46,20 @@ endif OBJECTS=$(AST_OBJECTS) $(COMPILER_OBJECTS) $(ANALYSIS_OBJECTS) $(VM_OBJECTS) $(TESTS_OBJECTS) $(CLI_OBJECTS) $(FUZZ_OBJECTS) # common flags -CXXFLAGS=-g -Wall -Werror +CXXFLAGS=-g -Wall LDFLAGS= -# temporary, for older gcc versions as they treat var in `if (type var = val)` as unused +# some gcc versions treat var in `if (type var = val)` as unused +# some gcc versions treat variables used in constexpr if blocks as unused ifeq ($(findstring g++,$(shell $(CXX) --version)),g++) CXXFLAGS+=-Wno-unused endif +# enabled in CI; we should be warning free on our main compiler versions but don't guarantee being warning free everywhere +ifneq ($(werror),) + CXXFLAGS+=-Werror +endif + # configuration-specific flags ifeq ($(config),release) CXXFLAGS+=-O2 -DNDEBUG From d11e8277c2c1094444a6da7f6749f9013636a3be Mon Sep 17 00:00:00 2001 From: ThePotato <10260415+ThePotato97@users.noreply.github.com> Date: Fri, 12 Nov 2021 14:58:34 +0000 Subject: [PATCH 055/399] Fixes IFTODT error while compiling from an android device (#199) Co-authored-by: Arseny Kapoulkine --- CLI/FileUtils.cpp | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/CLI/FileUtils.cpp b/CLI/FileUtils.cpp index 0702b74f..fff5e429 100644 --- a/CLI/FileUtils.cpp +++ b/CLI/FileUtils.cpp @@ -142,6 +142,7 @@ static bool traverseDirectoryRec(const std::string& path, const std::function Date: Mon, 15 Nov 2021 12:04:26 -0800 Subject: [PATCH 056/399] Create SECURITY.md Fixes #197. --- SECURITY.md | 14 ++++++++++++++ 1 file changed, 14 insertions(+) create mode 100644 SECURITY.md diff --git a/SECURITY.md b/SECURITY.md new file mode 100644 index 00000000..3fc9f66d --- /dev/null +++ b/SECURITY.md @@ -0,0 +1,14 @@ +# Security Guarantees + +Luau provides a safe sandbox that scripts can not escape from, short of vulnerabilities in custom C functions exposed by the host. This includes the virtual machine and builtin libraries. + +Any source code can not result in memory safety errors or crashes during its compilation or execution. Violations of memory safety are considered vulnerabilities. + +Note that Luau does not provide termination guarantees - some code may exhaust CPU or RAM resources on the system during compilation or execution. + +The runtime expects valid bytecode as an input. Feeding bytecode that was not produced by Luau compiler into the VM is not supported and +doesn't come with any security guarantees; make sure to sign the bytecode when it crosses a network or file system boundary to avoid tampering. + +# Reporting a Vulnerability + +You can report security bugs via [Hackerone](https://hackerone.com/roblox). Please refer to the linked page for rules of the bounty program. From e7a443daa825ec82e6d24249a2fcd06b58681160 Mon Sep 17 00:00:00 2001 From: fpliu Date: Tue, 16 Nov 2021 07:01:36 +0800 Subject: [PATCH 057/399] Fix Android linking issue with libpthread (#203) --- CMakeLists.txt | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/CMakeLists.txt b/CMakeLists.txt index d9ef9c05..9c69521e 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -87,7 +87,10 @@ if(LUAU_BUILD_CLI) target_link_libraries(Luau.Repl.CLI PRIVATE Luau.Compiler Luau.VM) if(UNIX) - target_link_libraries(Luau.Repl.CLI PRIVATE pthread) + find_library(LIBPTHREAD pthread) + if (LIBPTHREAD) + target_link_libraries(Luau.Repl.CLI PRIVATE pthread) + endif() endif() if(NOT EMSCRIPTEN) From 59366ad7f8861bf0b0ae76bcaa4d07a357064e12 Mon Sep 17 00:00:00 2001 From: Alan Jeffrey <403333+asajeffrey@users.noreply.github.com> Date: Tue, 16 Nov 2021 14:48:01 -0600 Subject: [PATCH 058/399] Clarified parsing properties of tables in the presence of singleton types (#207) --- rfcs/syntax-singleton-types.md | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/rfcs/syntax-singleton-types.md b/rfcs/syntax-singleton-types.md index 749006c7..26ea3028 100644 --- a/rfcs/syntax-singleton-types.md +++ b/rfcs/syntax-singleton-types.md @@ -48,6 +48,18 @@ type Animals = "Dog" | "Cat" | "Bird" type TrueOrNil = true? ``` +Adding constant strings as type means that it is now legal to write +`{["foo"]:T}` as a table type. This should be parsed as a property, +not an indexer. For example: +```lua + type T = { + ["foo"]: number, + ["$$bar"]: string, + baz: boolean, + } +``` +The table type `T` is a table with three properties and no indexer. + ### Semantics You are allowed to provide a constant value to the generic primitive type. From a02086260b45c4eb60b8550cf445d96f5eefd54c Mon Sep 17 00:00:00 2001 From: ccuser44 <68124053+ccuser44@users.noreply.github.com> Date: Wed, 17 Nov 2021 16:42:04 +0200 Subject: [PATCH 059/399] Added note to docs about not using os.difftime for new work (#210) Co-authored-by: Arseny Kapoulkine Fixes #194. --- docs/_pages/library.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/_pages/library.md b/docs/_pages/library.md index 0a6e3ec9..28a9f389 100644 --- a/docs/_pages/library.md +++ b/docs/_pages/library.md @@ -759,7 +759,7 @@ Otherwise, `s` is interpreted as a [date format string](https://www.cplusplus.co function os.difftime(a: number, b: number): number ``` -Calculates the difference in seconds between `a` and `b`; provided for compatibility. +Calculates the difference in seconds between `a` and `b`; provided for compatibility only. Please use `a - b` instead. ``` function os.time(t: table?): number From 09ad884ca8a41b0fb0218f5260036b65187ca2d1 Mon Sep 17 00:00:00 2001 From: ccuser44 <68124053+ccuser44@users.noreply.github.com> Date: Wed, 17 Nov 2021 16:42:14 +0200 Subject: [PATCH 060/399] Update SECURITY.md (#209) In some use cases it is better to encrypt the bytecode, while on others you may want to do both. --- SECURITY.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/SECURITY.md b/SECURITY.md index 3fc9f66d..ad929775 100644 --- a/SECURITY.md +++ b/SECURITY.md @@ -7,7 +7,7 @@ Any source code can not result in memory safety errors or crashes during its com Note that Luau does not provide termination guarantees - some code may exhaust CPU or RAM resources on the system during compilation or execution. The runtime expects valid bytecode as an input. Feeding bytecode that was not produced by Luau compiler into the VM is not supported and -doesn't come with any security guarantees; make sure to sign the bytecode when it crosses a network or file system boundary to avoid tampering. +doesn't come with any security guarantees; make sure to sign and/or encrypt the bytecode when it crosses a network or file system boundary to avoid tampering. # Reporting a Vulnerability From 4265e58ad10035eb6025d927a8eee8c98259389e Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Wed, 17 Nov 2021 06:49:49 -0800 Subject: [PATCH 061/399] RFC: coroutine.close (#88) --- rfcs/function-coroutine-close.md | 34 ++++++++++++++++++++++++++++++++ 1 file changed, 34 insertions(+) create mode 100644 rfcs/function-coroutine-close.md diff --git a/rfcs/function-coroutine-close.md b/rfcs/function-coroutine-close.md new file mode 100644 index 00000000..635050c4 --- /dev/null +++ b/rfcs/function-coroutine-close.md @@ -0,0 +1,34 @@ +# coroutine.close + +## Summary + +Add `coroutine.close` function from Lua 5.4 that takes a suspended coroutine and makes it "dead" (non-runnable). + +## Motivation + +When implementing various higher level objects on top of coroutines, such as promises, it can be useful to cancel the coroutine execution externally - when the caller is not +interested in getting the results anymore, execution can be aborted. Since coroutines don't provide a way to do that externally, this requires the framework to implement +cancellation on top of coroutines by keeping extra status/token and checking that token in all places where the coroutine is resumed. + +Since coroutine execution can be aborted with an error at any point, coroutines already implement support for "dead" status. If it were possible to externally transition a coroutine +to that status, it would be easier to implement cancellable promises on top of coroutines. + +## Design + +We implement Lua 5.4 behavior exactly with the exception of to-be-closed variables that we don't support. Quoting Lua 5.4 manual: + +> coroutine.close (co) +> Closes coroutine co, that is, puts the coroutine in a dead state. The given coroutine must be dead or suspended. In case of error (either the original error that stopped the coroutine or errors in closing methods), returns false plus the error object; otherwise returns true. + +The `co` argument must be a coroutine object (of type `thread`). + +After closing the coroutine, it gets transitioned to dead state which means that `coroutine.status` will return `"dead"` and attempts to resume the coroutine will fail. In addition, the coroutine stack (which can be accessed via `debug.traceback` or `debug.info`) will become empty. Calling `coroutine.close` on a closed coroutine will return `true` - after closing, the coroutine transitions into a "dead" state with no error information. + +## Drawbacks + +None known, as this function doesn't introduce any existing states to coroutines, and is similar to running the coroutine to completion/error. + +## Alternatives + +Lua's name for this function is likely in part motivated by to-be-closed variables that we don't support. As such, a more appropriate name could be `coroutine.cancel` which also +aligns with use cases better. However, since the semantics is otherwise the same, using the same name as Lua 5.4 reduces library fragmentation. From 60e6e86adb5c7a687153989bc471fc05fc4637e8 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Thu, 18 Nov 2021 14:21:07 -0800 Subject: [PATCH 062/399] Sync to upstream/release/505 --- Analysis/include/Luau/Documentation.h | 11 +- Analysis/include/Luau/ToString.h | 11 +- Analysis/include/Luau/TypeInfer.h | 24 +- Analysis/include/Luau/TypeVar.h | 87 ++++- Analysis/include/Luau/Unifiable.h | 2 + Analysis/include/Luau/Unifier.h | 1 + Analysis/src/Autocomplete.cpp | 33 +- Analysis/src/EmbeddedBuiltinDefinitions.cpp | 1 + Analysis/src/Error.cpp | 8 +- Analysis/src/Frontend.cpp | 4 +- Analysis/src/Module.cpp | 16 +- Analysis/src/ToString.cpp | 115 +++++- Analysis/src/Transpiler.cpp | 55 --- Analysis/src/TypeAttach.cpp | 16 + Analysis/src/TypeInfer.cpp | 374 ++++++++++++------- Analysis/src/TypePack.cpp | 1 - Analysis/src/TypeVar.cpp | 50 ++- Analysis/src/Unifier.cpp | 64 +++- Ast/include/Luau/Ast.h | 32 ++ Ast/include/Luau/Parser.h | 1 + Ast/include/Luau/StringUtils.h | 2 + Ast/src/Ast.cpp | 22 ++ Ast/src/Parser.cpp | 68 +++- Ast/src/StringUtils.cpp | 58 +++ CLI/Analyze.cpp | 28 +- CLI/FileUtils.cpp | 46 ++- CLI/FileUtils.h | 3 + CLI/Repl.cpp | 81 ++--- CMakeLists.txt | 17 +- Compiler/include/Luau/Compiler.h | 4 +- Compiler/include/luacode.h | 39 ++ Compiler/src/Compiler.cpp | 46 +-- Compiler/src/lcode.cpp | 29 ++ Makefile | 10 +- Sources.cmake | 3 + VM/include/lua.h | 18 +- VM/include/lualib.h | 6 +- VM/src/lapi.cpp | 10 +- VM/src/lbaselib.cpp | 8 +- VM/src/lbitlib.cpp | 1 + VM/src/lcorolib.cpp | 36 +- VM/src/ldebug.cpp | 10 +- VM/src/ldo.cpp | 6 +- VM/src/linit.cpp | 2 +- VM/src/lstate.cpp | 28 ++ VM/src/lstrlib.cpp | 2 +- VM/src/lutf8lib.cpp | 2 +- bench/bench.py | 32 +- fuzz/proto.cpp | 2 +- tests/Autocomplete.test.cpp | 16 +- tests/Compiler.test.cpp | 63 +--- tests/Conformance.test.cpp | 97 ++--- tests/IostreamOptional.h | 3 +- tests/Module.test.cpp | 6 +- tests/ToString.test.cpp | 111 +++++- tests/TypeInfer.aliases.test.cpp | 2 +- tests/TypeInfer.annotations.test.cpp | 4 +- tests/TypeInfer.generics.test.cpp | 6 +- tests/TypeInfer.refinements.test.cpp | 3 +- tests/TypeInfer.singletons.test.cpp | 377 ++++++++++++++++++++ tests/TypeInfer.test.cpp | 87 +++-- tests/TypeInfer.tryUnify.test.cpp | 30 +- tests/TypeInfer.unionTypes.test.cpp | 5 +- tests/conformance/coroutine.lua | 54 +++ tests/conformance/debugger.lua | 9 + 65 files changed, 1819 insertions(+), 579 deletions(-) create mode 100644 Compiler/include/luacode.h create mode 100644 Compiler/src/lcode.cpp create mode 100644 tests/TypeInfer.singletons.test.cpp diff --git a/Analysis/include/Luau/Documentation.h b/Analysis/include/Luau/Documentation.h index 7b609b4f..68ff3a7c 100644 --- a/Analysis/include/Luau/Documentation.h +++ b/Analysis/include/Luau/Documentation.h @@ -12,10 +12,17 @@ namespace Luau struct FunctionDocumentation; struct TableDocumentation; struct OverloadedFunctionDocumentation; +struct BasicDocumentation; -using Documentation = Luau::Variant; +using Documentation = Luau::Variant; using DocumentationSymbol = std::string; +struct BasicDocumentation +{ + std::string documentation; + std::string learnMoreLink; +}; + struct FunctionParameterDocumentation { std::string name; @@ -29,6 +36,7 @@ struct FunctionDocumentation std::string documentation; std::vector parameters; std::vector returns; + std::string learnMoreLink; }; struct OverloadedFunctionDocumentation @@ -43,6 +51,7 @@ struct TableDocumentation { std::string documentation; Luau::DenseHashMap keys; + std::string learnMoreLink; }; using DocumentationDatabase = Luau::DenseHashMap; diff --git a/Analysis/include/Luau/ToString.h b/Analysis/include/Luau/ToString.h index e5683fc4..50379c1c 100644 --- a/Analysis/include/Luau/ToString.h +++ b/Analysis/include/Luau/ToString.h @@ -23,10 +23,11 @@ struct ToStringNameMap struct ToStringOptions { - bool exhaustive = false; // If true, we produce complete output rather than comprehensible output - bool useLineBreaks = false; // If true, we insert new lines to separate long results such as table entries/metatable. - bool functionTypeArguments = false; // If true, output function type argument names when they are available - bool hideTableKind = false; // If true, all tables will be surrounded with plain '{}' + bool exhaustive = false; // If true, we produce complete output rather than comprehensible output + bool useLineBreaks = false; // If true, we insert new lines to separate long results such as table entries/metatable. + bool functionTypeArguments = false; // If true, output function type argument names when they are available + bool hideTableKind = false; // If true, all tables will be surrounded with plain '{}' + bool hideNamedFunctionTypeParameters = false; // If true, type parameters of functions will be hidden at top-level. size_t maxTableLength = size_t(FInt::LuauTableTypeMaximumStringifierLength); // Only applied to TableTypeVars size_t maxTypeLength = size_t(FInt::LuauTypeMaximumStringifierLength); std::optional nameMap; @@ -64,6 +65,8 @@ inline std::string toString(TypePackId ty) std::string toString(const TypeVar& tv, const ToStringOptions& opts = {}); std::string toString(const TypePackVar& tp, const ToStringOptions& opts = {}); +std::string toStringNamedFunction(const std::string& prefix, const FunctionTypeVar& ftv, ToStringOptions opts = {}); + // It could be useful to see the text representation of a type during a debugging session instead of exploring the content of the class // These functions will dump the type to stdout and can be evaluated in Watch/Immediate windows or as gdb/lldb expression void dump(TypeId ty); diff --git a/Analysis/include/Luau/TypeInfer.h b/Analysis/include/Luau/TypeInfer.h index 306ac77d..78d642c5 100644 --- a/Analysis/include/Luau/TypeInfer.h +++ b/Analysis/include/Luau/TypeInfer.h @@ -175,10 +175,10 @@ struct TypeChecker std::vector> getExpectedTypesForCall(const std::vector& overloads, size_t argumentCount, bool selfCall); std::optional> checkCallOverload(const ScopePtr& scope, const AstExprCall& expr, TypeId fn, TypePackId retPack, TypePackId argPack, TypePack* args, const std::vector& argLocations, const ExprResult& argListResult, - std::vector& overloadsThatMatchArgCount, std::vector& errors); + std::vector& overloadsThatMatchArgCount, std::vector& overloadsThatDont, std::vector& errors); bool handleSelfCallMismatch(const ScopePtr& scope, const AstExprCall& expr, TypePack* args, const std::vector& argLocations, const std::vector& errors); - ExprResult reportOverloadResolutionError(const ScopePtr& scope, const AstExprCall& expr, TypePackId retPack, TypePackId argPack, + void reportOverloadResolutionError(const ScopePtr& scope, const AstExprCall& expr, TypePackId retPack, TypePackId argPack, const std::vector& argLocations, const std::vector& overloads, const std::vector& overloadsThatMatchArgCount, const std::vector& errors); @@ -282,6 +282,14 @@ public: // Wrapper for merge(l, r, toUnion) but without the lambda junk. void merge(RefinementMap& l, const RefinementMap& r); + // Produce an "emergency backup type" for recovery from type errors. + // This comes in two flavours, depening on whether or not we can make a good guess + // for an error recovery type. + TypeId errorRecoveryType(TypeId guess); + TypePackId errorRecoveryTypePack(TypePackId guess); + TypeId errorRecoveryType(const ScopePtr& scope); + TypePackId errorRecoveryTypePack(const ScopePtr& scope); + private: void prepareErrorsForDisplay(ErrorVec& errVec); void diagnoseMissingTableKey(UnknownProperty* utk, TypeErrorData& data); @@ -297,6 +305,10 @@ private: TypeId freshType(const ScopePtr& scope); TypeId freshType(TypeLevel level); + // Produce a new singleton type var. + TypeId singletonType(bool value); + TypeId singletonType(std::string value); + // Returns nullopt if the predicate filters down the TypeId to 0 options. std::optional filterMap(TypeId type, TypeIdPredicate predicate); @@ -330,8 +342,8 @@ private: const std::vector& typePackParams, const Location& location); // Note: `scope` must be a fresh scope. - std::pair, std::vector> createGenericTypes( - const ScopePtr& scope, std::optional levelOpt, const AstNode& node, const AstArray& genericNames, const AstArray& genericPackNames); + std::pair, std::vector> createGenericTypes(const ScopePtr& scope, std::optional levelOpt, + const AstNode& node, const AstArray& genericNames, const AstArray& genericPackNames); public: ErrorVec resolve(const PredicateVec& predicates, const ScopePtr& scope, bool sense); @@ -347,7 +359,6 @@ private: void resolve(const OrPredicate& orP, ErrorVec& errVec, RefinementMap& refis, const ScopePtr& scope, bool sense); void resolve(const IsAPredicate& isaP, ErrorVec& errVec, RefinementMap& refis, const ScopePtr& scope, bool sense); void resolve(const TypeGuardPredicate& typeguardP, ErrorVec& errVec, RefinementMap& refis, const ScopePtr& scope, bool sense); - void DEPRECATED_resolve(const TypeGuardPredicate& typeguardP, ErrorVec& errVec, RefinementMap& refis, const ScopePtr& scope, bool sense); void resolve(const EqPredicate& eqP, ErrorVec& errVec, RefinementMap& refis, const ScopePtr& scope, bool sense); bool isNonstrictMode() const; @@ -387,12 +398,9 @@ public: const TypeId booleanType; const TypeId threadType; const TypeId anyType; - - const TypeId errorType; const TypeId optionalNumberType; const TypePackId anyTypePack; - const TypePackId errorTypePack; private: int checkRecursionCount = 0; diff --git a/Analysis/include/Luau/TypeVar.h b/Analysis/include/Luau/TypeVar.h index 6bd7932d..093ea431 100644 --- a/Analysis/include/Luau/TypeVar.h +++ b/Analysis/include/Luau/TypeVar.h @@ -108,6 +108,79 @@ struct PrimitiveTypeVar } }; +// Singleton types https://github.com/Roblox/luau/blob/master/rfcs/syntax-singleton-types.md +// Types for true and false +struct BoolSingleton +{ + bool value; + + bool operator==(const BoolSingleton& rhs) const + { + return value == rhs.value; + } + + bool operator!=(const BoolSingleton& rhs) const + { + return !(*this == rhs); + } +}; + +// Types for "foo", "bar" etc. +struct StringSingleton +{ + std::string value; + + bool operator==(const StringSingleton& rhs) const + { + return value == rhs.value; + } + + bool operator!=(const StringSingleton& rhs) const + { + return !(*this == rhs); + } +}; + +// No type for float singletons, partly because === isn't any equalivalence on floats +// (NaN != NaN). + +using SingletonVariant = Luau::Variant; + +struct SingletonTypeVar +{ + explicit SingletonTypeVar(const SingletonVariant& variant) + : variant(variant) + { + } + + explicit SingletonTypeVar(SingletonVariant&& variant) + : variant(std::move(variant)) + { + } + + // Default operator== is C++20. + bool operator==(const SingletonTypeVar& rhs) const + { + return variant == rhs.variant; + } + + bool operator!=(const SingletonTypeVar& rhs) const + { + return !(*this == rhs); + } + + SingletonVariant variant; +}; + +template +const T* get(const SingletonTypeVar* stv) +{ + if (stv) + return get_if(&stv->variant); + else + return nullptr; +} + struct FunctionArgument { Name name; @@ -332,8 +405,8 @@ struct LazyTypeVar using ErrorTypeVar = Unifiable::Error; -using TypeVariant = Unifiable::Variant; +using TypeVariant = Unifiable::Variant; struct TypeVar final { @@ -410,6 +483,9 @@ bool isGeneric(const TypeId ty); // Checks if a type may be instantiated to one containing generic type binders bool maybeGeneric(const TypeId ty); +// Checks if a type is of the form T1|...|Tn where one of the Ti is a singleton +bool maybeSingleton(TypeId ty); + struct SingletonTypes { const TypeId nilType; @@ -418,16 +494,19 @@ struct SingletonTypes const TypeId booleanType; const TypeId threadType; const TypeId anyType; - const TypeId errorType; const TypeId optionalNumberType; const TypePackId anyTypePack; - const TypePackId errorTypePack; SingletonTypes(); SingletonTypes(const SingletonTypes&) = delete; void operator=(const SingletonTypes&) = delete; + TypeId errorRecoveryType(TypeId guess); + TypePackId errorRecoveryTypePack(TypePackId guess); + TypeId errorRecoveryType(); + TypePackId errorRecoveryTypePack(); + private: std::unique_ptr arena; TypeId makeStringMetatable(); diff --git a/Analysis/include/Luau/Unifiable.h b/Analysis/include/Luau/Unifiable.h index c2e07e46..b47610fc 100644 --- a/Analysis/include/Luau/Unifiable.h +++ b/Analysis/include/Luau/Unifiable.h @@ -105,6 +105,8 @@ private: struct Error { + // This constructor has to be public, since it's used in TypeVar and TypePack, + // but shouldn't be called directly. Please use errorRecoveryType() instead. Error(); int index; diff --git a/Analysis/include/Luau/Unifier.h b/Analysis/include/Luau/Unifier.h index be0aadd0..503034a1 100644 --- a/Analysis/include/Luau/Unifier.h +++ b/Analysis/include/Luau/Unifier.h @@ -65,6 +65,7 @@ struct Unifier private: void tryUnify_(TypeId superTy, TypeId subTy, bool isFunctionCall = false, bool isIntersection = false); void tryUnifyPrimitives(TypeId superTy, TypeId subTy); + void tryUnifySingletons(TypeId superTy, TypeId subTy); void tryUnifyFunctions(TypeId superTy, TypeId subTy, bool isFunctionCall = false); void tryUnifyTables(TypeId left, TypeId right, bool isIntersection = false); void DEPRECATED_tryUnifyTables(TypeId left, TypeId right, bool isIntersection = false); diff --git a/Analysis/src/Autocomplete.cpp b/Analysis/src/Autocomplete.cpp index 1c94bb68..6fc0b3f8 100644 --- a/Analysis/src/Autocomplete.cpp +++ b/Analysis/src/Autocomplete.cpp @@ -14,6 +14,7 @@ LUAU_FASTFLAGVARIABLE(ElseElseIfCompletionImprovements, false); LUAU_FASTFLAG(LuauIfElseExpressionAnalysisSupport) +LUAU_FASTFLAGVARIABLE(LuauAutocompleteAvoidMutation, false); static const std::unordered_set kStatementStartingKeywords = { "while", "if", "local", "repeat", "function", "do", "for", "return", "break", "continue", "type", "export"}; @@ -198,11 +199,24 @@ static TypeCorrectKind checkTypeCorrectKind(const Module& module, TypeArena* typ UnifierSharedState unifierState(&iceReporter); Unifier unifier(typeArena, Mode::Strict, module.getModuleScope(), Location(), Variance::Covariant, unifierState); - unifier.tryUnify(expectedType, actualType); + if (FFlag::LuauAutocompleteAvoidMutation) + { + SeenTypes seenTypes; + SeenTypePacks seenTypePacks; + expectedType = clone(expectedType, *typeArena, seenTypes, seenTypePacks, nullptr); + actualType = clone(actualType, *typeArena, seenTypes, seenTypePacks, nullptr); - bool ok = unifier.errors.empty(); - unifier.log.rollback(); - return ok; + auto errors = unifier.canUnify(expectedType, actualType); + return errors.empty(); + } + else + { + unifier.tryUnify(expectedType, actualType); + + bool ok = unifier.errors.empty(); + unifier.log.rollback(); + return ok; + } }; auto expr = node->asExpr(); @@ -1496,11 +1510,9 @@ AutocompleteResult autocomplete(Frontend& frontend, const ModuleName& moduleName if (!sourceModule) return {}; - TypeChecker& typeChecker = - (frontend.options.typecheckTwice ? frontend.typeCheckerForAutocomplete : frontend.typeChecker); - ModulePtr module = - (frontend.options.typecheckTwice ? frontend.moduleResolverForAutocomplete.getModule(moduleName) - : frontend.moduleResolver.getModule(moduleName)); + TypeChecker& typeChecker = (frontend.options.typecheckTwice ? frontend.typeCheckerForAutocomplete : frontend.typeChecker); + ModulePtr module = (frontend.options.typecheckTwice ? frontend.moduleResolverForAutocomplete.getModule(moduleName) + : frontend.moduleResolver.getModule(moduleName)); if (!module) return {}; @@ -1527,8 +1539,7 @@ OwningAutocompleteResult autocompleteSource(Frontend& frontend, std::string_view sourceModule->mode = Mode::Strict; sourceModule->commentLocations = std::move(result.commentLocations); - TypeChecker& typeChecker = - (frontend.options.typecheckTwice ? frontend.typeCheckerForAutocomplete : frontend.typeChecker); + TypeChecker& typeChecker = (frontend.options.typecheckTwice ? frontend.typeCheckerForAutocomplete : frontend.typeChecker); ModulePtr module = typeChecker.check(*sourceModule, Mode::Strict); diff --git a/Analysis/src/EmbeddedBuiltinDefinitions.cpp b/Analysis/src/EmbeddedBuiltinDefinitions.cpp index 96703ef1..9f5c8250 100644 --- a/Analysis/src/EmbeddedBuiltinDefinitions.cpp +++ b/Analysis/src/EmbeddedBuiltinDefinitions.cpp @@ -153,6 +153,7 @@ declare function gcinfo(): number wrap: ((A...) -> R...) -> any, yield: (A...) -> R..., isyieldable: () -> boolean, + close: (thread) -> (boolean, any?) } declare table: { diff --git a/Analysis/src/Error.cpp b/Analysis/src/Error.cpp index 46ff2c72..f80d50a7 100644 --- a/Analysis/src/Error.cpp +++ b/Analysis/src/Error.cpp @@ -180,13 +180,13 @@ struct ErrorConverter switch (e.context) { case CountMismatch::Return: - return "Expected to return " + std::to_string(e.expected) + " value" + expectedS + ", but " + - std::to_string(e.actual) + " " + actualVerb + " returned here"; + return "Expected to return " + std::to_string(e.expected) + " value" + expectedS + ", but " + std::to_string(e.actual) + " " + + actualVerb + " returned here"; case CountMismatch::Result: // It is alright if right hand side produces more values than the // left hand side accepts. In this context consider only the opposite case. - return "Function only returns " + std::to_string(e.expected) + " value" + expectedS + ". " + - std::to_string(e.actual) + " are required here"; + return "Function only returns " + std::to_string(e.expected) + " value" + expectedS + ". " + std::to_string(e.actual) + + " are required here"; case CountMismatch::Arg: if (FFlag::LuauTypeAliasPacks) return "Argument count mismatch. Function " + wrongNumberOfArgsString(e.expected, e.actual); diff --git a/Analysis/src/Frontend.cpp b/Analysis/src/Frontend.cpp index 2f411274..1e97705d 100644 --- a/Analysis/src/Frontend.cpp +++ b/Analysis/src/Frontend.cpp @@ -22,7 +22,6 @@ LUAU_FASTFLAGVARIABLE(LuauResolveModuleNameWithoutACurrentModule, false) LUAU_FASTFLAG(LuauTraceRequireLookupChild) LUAU_FASTFLAGVARIABLE(LuauPersistDefinitionFileTypes, false) LUAU_FASTFLAG(LuauNewRequireTrace2) -LUAU_FASTFLAGVARIABLE(LuauClearScopes, false) namespace Luau { @@ -458,8 +457,7 @@ CheckResult Frontend::check(const ModuleName& name) module->astTypes.clear(); module->astExpectedTypes.clear(); module->astOriginalCallTypes.clear(); - if (FFlag::LuauClearScopes) - module->scopes.resize(1); + module->scopes.resize(1); } if (mode != Mode::NoCheck) diff --git a/Analysis/src/Module.cpp b/Analysis/src/Module.cpp index 880ffd2e..32a0646a 100644 --- a/Analysis/src/Module.cpp +++ b/Analysis/src/Module.cpp @@ -161,6 +161,7 @@ struct TypeCloner void operator()(const Unifiable::Bound& t); void operator()(const Unifiable::Error& t); void operator()(const PrimitiveTypeVar& t); + void operator()(const SingletonTypeVar& t); void operator()(const FunctionTypeVar& t); void operator()(const TableTypeVar& t); void operator()(const MetatableTypeVar& t); @@ -199,7 +200,9 @@ struct TypePackCloner if (encounteredFreeType) *encounteredFreeType = true; - seenTypePacks[typePackId] = dest.addTypePack(TypePackVar{Unifiable::Error{}}); + TypePackId err = singletonTypes.errorRecoveryTypePack(singletonTypes.anyTypePack); + TypePackId cloned = dest.addTypePack(*err); + seenTypePacks[typePackId] = cloned; } void operator()(const Unifiable::Generic& t) @@ -251,8 +254,9 @@ void TypeCloner::operator()(const Unifiable::Free& t) { if (encounteredFreeType) *encounteredFreeType = true; - - seenTypes[typeId] = dest.addType(ErrorTypeVar{}); + TypeId err = singletonTypes.errorRecoveryType(singletonTypes.anyType); + TypeId cloned = dest.addType(*err); + seenTypes[typeId] = cloned; } void TypeCloner::operator()(const Unifiable::Generic& t) @@ -270,11 +274,17 @@ void TypeCloner::operator()(const Unifiable::Error& t) { defaultClone(t); } + void TypeCloner::operator()(const PrimitiveTypeVar& t) { defaultClone(t); } +void TypeCloner::operator()(const SingletonTypeVar& t) +{ + defaultClone(t); +} + void TypeCloner::operator()(const FunctionTypeVar& t) { TypeId result = dest.addType(FunctionTypeVar{TypeLevel{0, 0}, {}, {}, nullptr, nullptr, t.definition, t.hasSelf}); diff --git a/Analysis/src/ToString.cpp b/Analysis/src/ToString.cpp index 885fd489..735bfa50 100644 --- a/Analysis/src/ToString.cpp +++ b/Analysis/src/ToString.cpp @@ -350,6 +350,23 @@ struct TypeVarStringifier } } + void operator()(TypeId, const SingletonTypeVar& stv) + { + if (const BoolSingleton* bs = Luau::get(&stv)) + state.emit(bs->value ? "true" : "false"); + else if (const StringSingleton* ss = Luau::get(&stv)) + { + state.emit("\""); + state.emit(escape(ss->value)); + state.emit("\""); + } + else + { + LUAU_ASSERT(!"Unknown singleton type"); + throw std::runtime_error("Unknown singleton type"); + } + } + void operator()(TypeId, const FunctionTypeVar& ftv) { if (state.hasSeen(&ftv)) @@ -359,6 +376,7 @@ struct TypeVarStringifier return; } + // We should not be respecting opts.hideNamedFunctionTypeParameters here. if (ftv.generics.size() > 0 || ftv.genericPacks.size() > 0) { state.emit("<"); @@ -514,7 +532,14 @@ struct TypeVarStringifier break; } - state.emit(name); + if (isIdentifier(name)) + state.emit(name); + else + { + state.emit("[\""); + state.emit(escape(name)); + state.emit("\"]"); + } state.emit(": "); stringify(prop.type); comma = true; @@ -1084,6 +1109,94 @@ std::string toString(const TypePackVar& tp, const ToStringOptions& opts) return toString(const_cast(&tp), std::move(opts)); } +std::string toStringNamedFunction(const std::string& prefix, const FunctionTypeVar& ftv, ToStringOptions opts) +{ + std::string s = prefix; + + auto toString_ = [&opts](TypeId ty) -> std::string { + ToStringResult res = toStringDetailed(ty, opts); + opts.nameMap = std::move(res.nameMap); + return res.name; + }; + + auto toStringPack_ = [&opts](TypePackId ty) -> std::string { + ToStringResult res = toStringDetailed(ty, opts); + opts.nameMap = std::move(res.nameMap); + return res.name; + }; + + if (!opts.hideNamedFunctionTypeParameters && (!ftv.generics.empty() || !ftv.genericPacks.empty())) + { + s += "<"; + + bool first = true; + for (TypeId g : ftv.generics) + { + if (!first) + s += ", "; + first = false; + s += toString_(g); + } + + for (TypePackId gp : ftv.genericPacks) + { + if (!first) + s += ", "; + first = false; + s += toStringPack_(gp); + } + + s += ">"; + } + + s += "("; + + auto argPackIter = begin(ftv.argTypes); + auto argNameIter = ftv.argNames.begin(); + + bool first = true; + while (argPackIter != end(ftv.argTypes)) + { + if (!first) + s += ", "; + first = false; + + // argNames is guaranteed to be equal to argTypes iff argNames is not empty. + // We don't currently respect opts.functionTypeArguments. I don't think this function should. + if (!ftv.argNames.empty()) + s += (*argNameIter ? (*argNameIter)->name : "_") + ": "; + s += toString_(*argPackIter); + + ++argPackIter; + if (!ftv.argNames.empty()) + { + LUAU_ASSERT(argNameIter != ftv.argNames.end()); + ++argNameIter; + } + } + + if (argPackIter.tail()) + { + if (auto vtp = get(*argPackIter.tail())) + s += ", ...: " + toString_(vtp->ty); + else + s += ", ...: " + toStringPack_(*argPackIter.tail()); + } + + s += "): "; + + size_t retSize = size(ftv.retType); + bool hasTail = !finite(ftv.retType); + if (retSize == 0 && !hasTail) + s += "()"; + else if ((retSize == 0 && hasTail) || (retSize == 1 && !hasTail)) + s += toStringPack_(ftv.retType); + else + s += "(" + toStringPack_(ftv.retType) + ")"; + + return s; +} + void dump(TypeId ty) { ToStringOptions opts; diff --git a/Analysis/src/Transpiler.cpp b/Analysis/src/Transpiler.cpp index 7d880af4..6627fbe3 100644 --- a/Analysis/src/Transpiler.cpp +++ b/Analysis/src/Transpiler.cpp @@ -14,61 +14,6 @@ LUAU_FASTFLAG(LuauTypeAliasPacks) namespace { - -std::string escape(std::string_view s) -{ - std::string r; - r.reserve(s.size() + 50); // arbitrary number to guess how many characters we'll be inserting - - for (uint8_t c : s) - { - if (c >= ' ' && c != '\\' && c != '\'' && c != '\"') - r += c; - else - { - r += '\\'; - - switch (c) - { - case '\a': - r += 'a'; - break; - case '\b': - r += 'b'; - break; - case '\f': - r += 'f'; - break; - case '\n': - r += 'n'; - break; - case '\r': - r += 'r'; - break; - case '\t': - r += 't'; - break; - case '\v': - r += 'v'; - break; - case '\'': - r += '\''; - break; - case '\"': - r += '\"'; - break; - case '\\': - r += '\\'; - break; - default: - Luau::formatAppend(r, "%03u", c); - } - } - } - - return r; -} - bool isIdentifierStartChar(char c) { return (c >= 'A' && c <= 'Z') || (c >= 'a' && c <= 'z') || c == '_'; diff --git a/Analysis/src/TypeAttach.cpp b/Analysis/src/TypeAttach.cpp index 11aa7b39..af6d2543 100644 --- a/Analysis/src/TypeAttach.cpp +++ b/Analysis/src/TypeAttach.cpp @@ -96,6 +96,22 @@ public: return nullptr; } } + + AstType* operator()(const SingletonTypeVar& stv) + { + if (const BoolSingleton* bs = get(&stv)) + return allocator->alloc(Location(), bs->value); + else if (const StringSingleton* ss = get(&stv)) + { + AstArray value; + value.data = const_cast(ss->value.c_str()); + value.size = strlen(value.data); + return allocator->alloc(Location(), value); + } + else + return nullptr; + } + AstType* operator()(const AnyTypeVar&) { return allocator->alloc(Location(), std::nullopt, AstName("any")); diff --git a/Analysis/src/TypeInfer.cpp b/Analysis/src/TypeInfer.cpp index 8fad1af9..b2ae94c7 100644 --- a/Analysis/src/TypeInfer.cpp +++ b/Analysis/src/TypeInfer.cpp @@ -36,6 +36,9 @@ LUAU_FASTFLAG(LuauSubstitutionDontReplaceIgnoredTypes) LUAU_FASTFLAGVARIABLE(LuauQuantifyInPlace2, false) LUAU_FASTFLAG(LuauNewRequireTrace2) LUAU_FASTFLAG(LuauTypeAliasPacks) +LUAU_FASTFLAGVARIABLE(LuauSingletonTypes, false) +LUAU_FASTFLAGVARIABLE(LuauExpectedTypesOfProperties, false) +LUAU_FASTFLAGVARIABLE(LuauErrorRecoveryType, false) namespace Luau { @@ -211,10 +214,8 @@ TypeChecker::TypeChecker(ModuleResolver* resolver, InternalErrorReporter* iceHan , booleanType(singletonTypes.booleanType) , threadType(singletonTypes.threadType) , anyType(singletonTypes.anyType) - , errorType(singletonTypes.errorType) , optionalNumberType(singletonTypes.optionalNumberType) , anyTypePack(singletonTypes.anyTypePack) - , errorTypePack(singletonTypes.errorTypePack) { globalScope = std::make_shared(globalTypes.addTypePack(TypePackVar{FreeTypePack{TypeLevel{}}})); @@ -484,7 +485,7 @@ LUAU_NOINLINE void TypeChecker::checkBlockTypeAliases(const ScopePtr& scope, std TypeId type = bindings[name].type; if (get(follow(type))) { - *asMutable(type) = ErrorTypeVar{}; + *asMutable(type) = *errorRecoveryType(anyType); reportError(TypeError{typealias->location, OccursCheckFailed{}}); } } @@ -719,7 +720,7 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatAssign& assign) else if (auto tail = valueIter.tail()) { if (get(*tail)) - right = errorType; + right = errorRecoveryType(scope); else if (auto vtp = get(*tail)) right = vtp->ty; else if (get(*tail)) @@ -961,7 +962,7 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatForIn& forin) else if (get(callRetPack) || !first(callRetPack)) { for (TypeId var : varTypes) - unify(var, errorType, forin.location); + unify(var, errorRecoveryType(scope), forin.location); return check(loopScope, *forin.body); } @@ -979,7 +980,7 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatForIn& forin) const FunctionTypeVar* iterFunc = get(iterTy); if (!iterFunc) { - TypeId varTy = get(iterTy) ? anyType : errorType; + TypeId varTy = get(iterTy) ? anyType : errorRecoveryType(loopScope); for (TypeId var : varTypes) unify(var, varTy, forin.location); @@ -1152,9 +1153,9 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatTypeAlias& typealias reportError(TypeError{typealias.location, DuplicateTypeDefinition{name, location}}); if (FFlag::LuauTypeAliasPacks) - bindingsMap[name] = TypeFun{binding->typeParams, binding->typePackParams, errorType}; + bindingsMap[name] = TypeFun{binding->typeParams, binding->typePackParams, errorRecoveryType(anyType)}; else - bindingsMap[name] = TypeFun{binding->typeParams, errorType}; + bindingsMap[name] = TypeFun{binding->typeParams, errorRecoveryType(anyType)}; } else { @@ -1398,7 +1399,7 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExpr& if (FInt::LuauCheckRecursionLimit > 0 && checkRecursionCount >= FInt::LuauCheckRecursionLimit) { reportErrorCodeTooComplex(expr.location); - return {errorType}; + return {errorRecoveryType(scope)}; } ExprResult result; @@ -1407,12 +1408,22 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExpr& result = checkExpr(scope, *a->expr); else if (expr.is()) result = {nilType}; - else if (expr.is()) - result = {booleanType}; + else if (const AstExprConstantBool* bexpr = expr.as()) + { + if (FFlag::LuauSingletonTypes && expectedType && maybeSingleton(*expectedType)) + result = {singletonType(bexpr->value)}; + else + result = {booleanType}; + } + else if (const AstExprConstantString* sexpr = expr.as()) + { + if (FFlag::LuauSingletonTypes && expectedType && maybeSingleton(*expectedType)) + result = {singletonType(std::string(sexpr->value.data, sexpr->value.size))}; + else + result = {stringType}; + } else if (expr.is()) result = {numberType}; - else if (expr.is()) - result = {stringType}; else if (auto a = expr.as()) result = checkExpr(scope, *a); else if (auto a = expr.as()) @@ -1485,7 +1496,7 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprLo // TODO: tempting to ice here, but this breaks very often because our toposort doesn't enforce this constraint // ice("AstExprLocal exists but no binding definition for it?", expr.location); reportError(TypeError{expr.location, UnknownSymbol{expr.local->name.value, UnknownSymbol::Binding}}); - return {errorType}; + return {errorRecoveryType(scope)}; } ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprGlobal& expr) @@ -1497,7 +1508,7 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprGl return {*ty, {TruthyPredicate{std::move(*lvalue), expr.location}}}; reportError(TypeError{expr.location, UnknownSymbol{expr.name.value, UnknownSymbol::Binding}}); - return {errorType}; + return {errorRecoveryType(scope)}; } ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprVarargs& expr) @@ -1509,7 +1520,7 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprVa std::vector types = flatten(varargPack).first; return {!types.empty() ? types[0] : nilType}; } - else if (auto ftp = get(varargPack)) + else if (get(varargPack)) { TypeId head = freshType(scope); TypePackId tail = freshTypePack(scope); @@ -1517,14 +1528,14 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprVa return {head}; } if (get(varargPack)) - return {errorType}; + return {errorRecoveryType(scope)}; else if (auto vtp = get(varargPack)) return {vtp->ty}; else if (get(varargPack)) { // TODO: Better error? reportError(expr.location, GenericError{"Trying to get a type from a variadic type parameter"}); - return {errorType}; + return {errorRecoveryType(scope)}; } else ice("Unknown TypePack type in checkExpr(AstExprVarargs)!"); @@ -1539,7 +1550,7 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprCa { return {pack->head.empty() ? nilType : pack->head[0], std::move(result.predicates)}; } - else if (auto ftp = get(retPack)) + else if (get(retPack)) { TypeId head = freshType(scope); TypePackId pack = addTypePack(TypePackVar{TypePack{{head}, freshTypePack(scope)}}); @@ -1547,7 +1558,7 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprCa return {head, std::move(result.predicates)}; } if (get(retPack)) - return {errorType, std::move(result.predicates)}; + return {errorRecoveryType(scope), std::move(result.predicates)}; else if (auto vtp = get(retPack)) return {vtp->ty, std::move(result.predicates)}; else if (get(retPack)) @@ -1572,7 +1583,7 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprIn if (std::optional ty = getIndexTypeFromType(scope, lhsType, name, expr.location, true)) return {*ty}; - return {errorType}; + return {errorRecoveryType(scope)}; } std::optional TypeChecker::findTablePropertyRespectingMeta(TypeId lhsType, Name name, const Location& location) @@ -1876,6 +1887,7 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprTa std::vector> fieldTypes(expr.items.size); const TableTypeVar* expectedTable = nullptr; + const UnionTypeVar* expectedUnion = nullptr; std::optional expectedIndexType; std::optional expectedIndexResultType; @@ -1894,6 +1906,9 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprTa } } } + else if (FFlag::LuauExpectedTypesOfProperties) + if (const UnionTypeVar* utv = get(follow(*expectedType))) + expectedUnion = utv; } for (size_t i = 0; i < expr.items.size; ++i) @@ -1916,6 +1931,18 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprTa if (auto prop = expectedTable->props.find(key->value.data); prop != expectedTable->props.end()) expectedResultType = prop->second.type; } + else if (FFlag::LuauExpectedTypesOfProperties && expectedUnion) + { + std::vector expectedResultTypes; + for (TypeId expectedOption : expectedUnion) + if (const TableTypeVar* ttv = get(follow(expectedOption))) + if (auto prop = ttv->props.find(key->value.data); prop != ttv->props.end()) + expectedResultTypes.push_back(prop->second.type); + if (expectedResultTypes.size() == 1) + expectedResultType = expectedResultTypes[0]; + else if (expectedResultTypes.size() > 1) + expectedResultType = addType(UnionTypeVar{expectedResultTypes}); + } } else { @@ -1958,21 +1985,22 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprUn { TypeId actualFunctionType = instantiate(scope, *fnt, expr.location); TypePackId arguments = addTypePack({operandType}); - TypePackId retType = freshTypePack(scope); - TypeId expectedFunctionType = addType(FunctionTypeVar(scope->level, arguments, retType)); + TypePackId retTypePack = freshTypePack(scope); + TypeId expectedFunctionType = addType(FunctionTypeVar(scope->level, arguments, retTypePack)); Unifier state = mkUnifier(expr.location); state.tryUnify(expectedFunctionType, actualFunctionType, /*isFunctionCall*/ true); + TypeId retType = first(retTypePack).value_or(nilType); if (!state.errors.empty()) - return {errorType}; + retType = errorRecoveryType(retType); - return {first(retType).value_or(nilType)}; + return {retType}; } reportError(expr.location, GenericError{format("Unary operator '%s' not supported by type '%s'", toString(expr.op).c_str(), toString(operandType).c_str())}); - return {errorType}; + return {errorRecoveryType(scope)}; } reportErrors(tryUnify(numberType, operandType, expr.location)); @@ -1984,7 +2012,7 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprUn operandType = stripFromNilAndReport(operandType, expr.location); if (get(operandType)) - return {errorType}; + return {errorRecoveryType(scope)}; if (get(operandType)) return {numberType}; // Not strictly correct: metatables permit overriding this @@ -2044,7 +2072,7 @@ TypeId TypeChecker::unionOfTypes(TypeId a, TypeId b, const Location& location, b if (unify(a, b, location)) return a; - return errorType; + return errorRecoveryType(anyType); } if (*a == *b) @@ -2166,11 +2194,13 @@ TypeId TypeChecker::checkRelationalOperation( std::optional leftMetatable = isString(lhsType) ? std::nullopt : getMetatable(follow(lhsType)); std::optional rightMetatable = isString(rhsType) ? std::nullopt : getMetatable(follow(rhsType)); + // TODO: this check seems odd, the second part is redundant + // is it meant to be if (leftMetatable && rightMetatable && leftMetatable != rightMetatable) if (bool(leftMetatable) != bool(rightMetatable) && leftMetatable != rightMetatable) { reportError(expr.location, GenericError{format("Types %s and %s cannot be compared with %s because they do not have the same metatable", toString(lhsType).c_str(), toString(rhsType).c_str(), toString(expr.op).c_str())}); - return errorType; + return errorRecoveryType(booleanType); } if (leftMetatable) @@ -2188,7 +2218,7 @@ TypeId TypeChecker::checkRelationalOperation( if (!state.errors.empty()) { reportError(expr.location, GenericError{format("Metamethod '%s' must return type 'boolean'", metamethodName.c_str())}); - return errorType; + return errorRecoveryType(booleanType); } } } @@ -2206,7 +2236,7 @@ TypeId TypeChecker::checkRelationalOperation( { reportError( expr.location, GenericError{format("Table %s does not offer metamethod %s", toString(lhsType).c_str(), metamethodName.c_str())}); - return errorType; + return errorRecoveryType(booleanType); } } @@ -2214,14 +2244,14 @@ TypeId TypeChecker::checkRelationalOperation( { auto name = getIdentifierOfBaseVar(expr.left); reportError(expr.location, CannotInferBinaryOperation{expr.op, name, CannotInferBinaryOperation::Comparison}); - return errorType; + return errorRecoveryType(booleanType); } if (needsMetamethod) { reportError(expr.location, GenericError{format("Type %s cannot be compared with %s because it has no metatable", toString(lhsType).c_str(), toString(expr.op).c_str())}); - return errorType; + return errorRecoveryType(booleanType); } return booleanType; @@ -2266,7 +2296,8 @@ TypeId TypeChecker::checkBinaryOperation( { auto name = getIdentifierOfBaseVar(expr.left); reportError(expr.location, CannotInferBinaryOperation{expr.op, name, CannotInferBinaryOperation::Operation}); - return errorType; + if (!FFlag::LuauErrorRecoveryType) + return errorRecoveryType(scope); } // If we know nothing at all about the lhs type, we can usually say nothing about the result. @@ -2296,18 +2327,33 @@ TypeId TypeChecker::checkBinaryOperation( auto checkMetatableCall = [this, &scope, &expr](TypeId fnt, TypeId lhst, TypeId rhst) -> TypeId { TypeId actualFunctionType = instantiate(scope, fnt, expr.location); TypePackId arguments = addTypePack({lhst, rhst}); - TypePackId retType = freshTypePack(scope); - TypeId expectedFunctionType = addType(FunctionTypeVar(scope->level, arguments, retType)); + TypePackId retTypePack = freshTypePack(scope); + TypeId expectedFunctionType = addType(FunctionTypeVar(scope->level, arguments, retTypePack)); Unifier state = mkUnifier(expr.location); state.tryUnify(expectedFunctionType, actualFunctionType, /*isFunctionCall*/ true); reportErrors(state.errors); + bool hasErrors = !state.errors.empty(); - if (!state.errors.empty()) - return errorType; + if (FFlag::LuauErrorRecoveryType && hasErrors) + { + // If there are unification errors, the return type may still be unknown + // so we loosen the argument types to see if that helps. + TypePackId fallbackArguments = freshTypePack(scope); + TypeId fallbackFunctionType = addType(FunctionTypeVar(scope->level, fallbackArguments, retTypePack)); + state.log.rollback(); + state.errors.clear(); + state.tryUnify(fallbackFunctionType, actualFunctionType, /*isFunctionCall*/ true); + if (!state.errors.empty()) + state.log.rollback(); + } - return first(retType).value_or(nilType); + TypeId retType = first(retTypePack).value_or(nilType); + if (hasErrors) + retType = errorRecoveryType(retType); + + return retType; }; std::string op = opToMetaTableEntry(expr.op); @@ -2321,7 +2367,8 @@ TypeId TypeChecker::checkBinaryOperation( reportError(expr.location, GenericError{format("Binary operator '%s' not supported by types '%s' and '%s'", toString(expr.op).c_str(), toString(lhsType).c_str(), toString(rhsType).c_str())}); - return errorType; + + return errorRecoveryType(scope); } switch (expr.op) @@ -2414,11 +2461,9 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprTy ExprResult result = checkExpr(scope, *expr.expr, annotationType); ErrorVec errorVec = canUnify(result.type, annotationType, expr.location); + reportErrors(errorVec); if (!errorVec.empty()) - { - reportErrors(errorVec); - return {errorType, std::move(result.predicates)}; - } + annotationType = errorRecoveryType(annotationType); return {annotationType, std::move(result.predicates)}; } @@ -2434,7 +2479,7 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprEr // any type errors that may arise from it are going to be useless. currentModule->errors.resize(oldSize); - return {errorType}; + return {errorRecoveryType(scope)}; } ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprIfElse& expr) @@ -2476,7 +2521,7 @@ std::pair TypeChecker::checkLValueBinding(const ScopePtr& scope { for (AstExpr* expr : a->expressions) checkExpr(scope, *expr); - return std::pair(errorType, nullptr); + return {errorRecoveryType(scope), nullptr}; } else ice("Unexpected AST node in checkLValue", expr.location); @@ -2488,7 +2533,7 @@ std::pair TypeChecker::checkLValueBinding(const ScopePtr& scope return {*ty, nullptr}; reportError(expr.location, UnknownSymbol{expr.local->name.value, UnknownSymbol::Binding}); - return {errorType, nullptr}; + return {errorRecoveryType(scope), nullptr}; } std::pair TypeChecker::checkLValueBinding(const ScopePtr& scope, const AstExprGlobal& expr) @@ -2545,24 +2590,25 @@ std::pair TypeChecker::checkLValueBinding(const ScopePtr& scope { Unifier state = mkUnifier(expr.location); state.tryUnify(indexer->indexType, stringType); + TypeId retType = indexer->indexResultType; if (!state.errors.empty()) { state.log.rollback(); reportError(expr.location, UnknownProperty{lhs, name}); - return std::pair(errorType, nullptr); + retType = errorRecoveryType(retType); } - return std::pair(indexer->indexResultType, nullptr); + return std::pair(retType, nullptr); } else if (lhsTable->state == TableState::Sealed) { reportError(TypeError{expr.location, CannotExtendTable{lhs, CannotExtendTable::Property, name}}); - return std::pair(errorType, nullptr); + return std::pair(errorRecoveryType(scope), nullptr); } else { reportError(TypeError{expr.location, GenericError{"Internal error: generic tables are not lvalues"}}); - return std::pair(errorType, nullptr); + return std::pair(errorRecoveryType(scope), nullptr); } } else if (const ClassTypeVar* lhsClass = get(lhs)) @@ -2571,7 +2617,7 @@ std::pair TypeChecker::checkLValueBinding(const ScopePtr& scope if (!prop) { reportError(TypeError{expr.location, UnknownProperty{lhs, name}}); - return std::pair(errorType, nullptr); + return std::pair(errorRecoveryType(scope), nullptr); } return std::pair(prop->type, nullptr); @@ -2585,12 +2631,12 @@ std::pair TypeChecker::checkLValueBinding(const ScopePtr& scope if (isTableIntersection(lhs)) { reportError(TypeError{expr.location, CannotExtendTable{lhs, CannotExtendTable::Property, name}}); - return std::pair(errorType, nullptr); + return std::pair(errorRecoveryType(scope), nullptr); } } reportError(TypeError{expr.location, NotATable{lhs}}); - return std::pair(errorType, nullptr); + return std::pair(errorRecoveryType(scope), nullptr); } std::pair TypeChecker::checkLValueBinding(const ScopePtr& scope, const AstExprIndexExpr& expr) @@ -2615,7 +2661,7 @@ std::pair TypeChecker::checkLValueBinding(const ScopePtr& scope if (!prop) { reportError(TypeError{expr.location, UnknownProperty{exprType, value->value.data}}); - return std::pair(errorType, nullptr); + return std::pair(errorRecoveryType(scope), nullptr); } return std::pair(prop->type, nullptr); } @@ -2626,7 +2672,7 @@ std::pair TypeChecker::checkLValueBinding(const ScopePtr& scope if (!exprTable) { reportError(TypeError{expr.expr->location, NotATable{exprType}}); - return std::pair(errorType, nullptr); + return std::pair(errorRecoveryType(scope), nullptr); } if (value) @@ -2678,7 +2724,7 @@ TypeId TypeChecker::checkFunctionName(const ScopePtr& scope, AstExpr& funName) if (isNonstrictMode()) return globalScope->bindings[name].typeId; - return errorType; + return errorRecoveryType(scope); } else { @@ -2705,20 +2751,21 @@ TypeId TypeChecker::checkFunctionName(const ScopePtr& scope, AstExpr& funName) TableTypeVar* ttv = getMutableTableType(lhsType); if (!ttv) { - if (!isTableIntersection(lhsType)) + if (!FFlag::LuauErrorRecoveryType && !isTableIntersection(lhsType)) + // This error now gets reported when we check the function body. reportError(TypeError{funName.location, OnlyTablesCanHaveMethods{lhsType}}); - return errorType; + return errorRecoveryType(scope); } // Cannot extend sealed table, but we dont report an error here because it will be reported during AstStatFunction check if (lhsType->persistent || ttv->state == TableState::Sealed) - return errorType; + return errorRecoveryType(scope); Name name = indexName->index.value; if (ttv->props.count(name)) - return errorType; + return errorRecoveryType(scope); Property& property = ttv->props[name]; @@ -2728,9 +2775,7 @@ TypeId TypeChecker::checkFunctionName(const ScopePtr& scope, AstExpr& funName) return property.type; } else if (funName.is()) - { - return errorType; - } + return errorRecoveryType(scope); else { ice("Unexpected AST node type", funName.location); @@ -2991,7 +3036,7 @@ ExprResult TypeChecker::checkExprPack(const ScopePtr& scope, const A else if (expr.is()) { if (!scope->varargPack) - return {addTypePack({addType(ErrorTypeVar())})}; + return {errorRecoveryTypePack(scope)}; return {*scope->varargPack}; } @@ -3095,10 +3140,9 @@ void TypeChecker::checkArgumentList( if (get(tail)) { // Unify remaining parameters so we don't leave any free-types hanging around. - TypeId argTy = errorType; while (paramIter != endIter) { - state.tryUnify(*paramIter, argTy); + state.tryUnify(*paramIter, errorRecoveryType(anyType)); ++paramIter; } return; @@ -3157,7 +3201,7 @@ void TypeChecker::checkArgumentList( { while (argIter != endIter) { - unify(*argIter, errorType, state.location); + unify(*argIter, errorRecoveryType(scope), state.location); ++argIter; } // For this case, we want the error span to cover every errant extra parameter @@ -3246,7 +3290,8 @@ ExprResult TypeChecker::checkExprPack(const ScopePtr& scope, const A // For each overload // Compare parameter and argument types // Report any errors (also speculate dot vs colon warnings!) - // If there are no errors, return the resulting return type + // Return the resulting return type (even if there are errors) + // If there are no matching overloads, unify with (a...) -> (b...) and return b... TypeId selfType = nullptr; TypeId functionType = nullptr; @@ -3268,8 +3313,8 @@ ExprResult TypeChecker::checkExprPack(const ScopePtr& scope, const A } else { - functionType = errorType; - actualFunctionType = errorType; + functionType = errorRecoveryType(scope); + actualFunctionType = functionType; } } else @@ -3296,7 +3341,7 @@ ExprResult TypeChecker::checkExprPack(const ScopePtr& scope, const A TypePackId argPack = argListResult.type; if (get(argPack)) - return ExprResult{errorTypePack}; + return {errorRecoveryTypePack(scope)}; TypePack* args = getMutable(argPack); LUAU_ASSERT(args != nullptr); @@ -3314,19 +3359,34 @@ ExprResult TypeChecker::checkExprPack(const ScopePtr& scope, const A std::vector errors; // errors encountered for each overload std::vector overloadsThatMatchArgCount; + std::vector overloadsThatDont; for (TypeId fn : overloads) { fn = follow(fn); - if (auto ret = checkCallOverload(scope, expr, fn, retPack, argPack, args, argLocations, argListResult, overloadsThatMatchArgCount, errors)) + if (auto ret = checkCallOverload( + scope, expr, fn, retPack, argPack, args, argLocations, argListResult, overloadsThatMatchArgCount, overloadsThatDont, errors)) return *ret; } if (handleSelfCallMismatch(scope, expr, args, argLocations, errors)) return {retPack}; - return reportOverloadResolutionError(scope, expr, retPack, argPack, argLocations, overloads, overloadsThatMatchArgCount, errors); + reportOverloadResolutionError(scope, expr, retPack, argPack, argLocations, overloads, overloadsThatMatchArgCount, errors); + + if (FFlag::LuauErrorRecoveryType) + { + const FunctionTypeVar* overload = nullptr; + if (!overloadsThatMatchArgCount.empty()) + overload = get(overloadsThatMatchArgCount[0]); + if (!overload && !overloadsThatDont.empty()) + overload = get(overloadsThatDont[0]); + if (overload) + return {errorRecoveryTypePack(overload->retType)}; + } + + return {errorRecoveryTypePack(retPack)}; } std::vector> TypeChecker::getExpectedTypesForCall(const std::vector& overloads, size_t argumentCount, bool selfCall) @@ -3382,7 +3442,7 @@ std::vector> TypeChecker::getExpectedTypesForCall(const st std::optional> TypeChecker::checkCallOverload(const ScopePtr& scope, const AstExprCall& expr, TypeId fn, TypePackId retPack, TypePackId argPack, TypePack* args, const std::vector& argLocations, const ExprResult& argListResult, - std::vector& overloadsThatMatchArgCount, std::vector& errors) + std::vector& overloadsThatMatchArgCount, std::vector& overloadsThatDont, std::vector& errors) { fn = stripFromNilAndReport(fn, expr.func->location); @@ -3394,7 +3454,7 @@ std::optional> TypeChecker::checkCallOverload(const Scope if (get(fn)) { - return {{addTypePack(TypePackVar{Unifiable::Error{}})}}; + return {{errorRecoveryTypePack(scope)}}; } if (get(fn)) @@ -3427,14 +3487,14 @@ std::optional> TypeChecker::checkCallOverload(const Scope TypeId fn = *ty; fn = instantiate(scope, fn, expr.func->location); - return checkCallOverload( - scope, expr, fn, retPack, metaCallArgPack, metaCallArgs, metaArgLocations, argListResult, overloadsThatMatchArgCount, errors); + return checkCallOverload(scope, expr, fn, retPack, metaCallArgPack, metaCallArgs, metaArgLocations, argListResult, + overloadsThatMatchArgCount, overloadsThatDont, errors); } } reportError(TypeError{expr.func->location, CannotCallNonFunction{fn}}); - unify(retPack, errorTypePack, expr.func->location); - return {{errorTypePack}}; + unify(retPack, errorRecoveryTypePack(scope), expr.func->location); + return {{errorRecoveryTypePack(retPack)}}; } // When this function type has magic functions and did return something, we select that overload instead. @@ -3476,6 +3536,8 @@ std::optional> TypeChecker::checkCallOverload(const Scope if (!argMismatch) overloadsThatMatchArgCount.push_back(fn); + else if (FFlag::LuauErrorRecoveryType) + overloadsThatDont.push_back(fn); errors.emplace_back(std::move(state.errors), args->head, ftv); state.log.rollback(); @@ -3586,14 +3648,14 @@ bool TypeChecker::handleSelfCallMismatch(const ScopePtr& scope, const AstExprCal return false; } -ExprResult TypeChecker::reportOverloadResolutionError(const ScopePtr& scope, const AstExprCall& expr, TypePackId retPack, - TypePackId argPack, const std::vector& argLocations, const std::vector& overloads, - const std::vector& overloadsThatMatchArgCount, const std::vector& errors) +void TypeChecker::reportOverloadResolutionError(const ScopePtr& scope, const AstExprCall& expr, TypePackId retPack, TypePackId argPack, + const std::vector& argLocations, const std::vector& overloads, const std::vector& overloadsThatMatchArgCount, + const std::vector& errors) { if (overloads.size() == 1) { reportErrors(std::get<0>(errors.front())); - return {errorTypePack}; + return; } std::vector overloadTypes = overloadsThatMatchArgCount; @@ -3622,7 +3684,7 @@ ExprResult TypeChecker::reportOverloadResolutionError(const ScopePtr // If only one overload matched, we don't need this error because we provided the previous errors. if (overloadsThatMatchArgCount.size() == 1) - return {errorTypePack}; + return; } std::string s; @@ -3655,7 +3717,7 @@ ExprResult TypeChecker::reportOverloadResolutionError(const ScopePtr reportError(expr.func->location, ExtraInformation{"Other overloads are also not viable: " + s}); // No viable overload - return {errorTypePack}; + return; } ExprResult TypeChecker::checkExprList(const ScopePtr& scope, const Location& location, const AstArray& exprs, @@ -3740,7 +3802,7 @@ TypeId TypeChecker::checkRequire(const ScopePtr& scope, const ModuleInfo& module if (FFlag::LuauStrictRequire && currentModule->mode == Mode::Strict) { reportError(TypeError{location, UnknownRequire{}}); - return errorType; + return errorRecoveryType(anyType); } return anyType; @@ -3758,14 +3820,14 @@ TypeId TypeChecker::checkRequire(const ScopePtr& scope, const ModuleInfo& module reportError(TypeError{location, UnknownRequire{reportedModulePath}}); } - return errorType; + return errorRecoveryType(scope); } if (module->type != SourceCode::Module) { std::string humanReadableName = resolver->getHumanReadableModuleName(moduleInfo.name); reportError(location, IllegalRequire{humanReadableName, "Module is not a ModuleScript. It cannot be required."}); - return errorType; + return errorRecoveryType(scope); } std::optional moduleType = first(module->getModuleScope()->returnType); @@ -3773,7 +3835,7 @@ TypeId TypeChecker::checkRequire(const ScopePtr& scope, const ModuleInfo& module { std::string humanReadableName = resolver->getHumanReadableModuleName(moduleInfo.name); reportError(location, IllegalRequire{humanReadableName, "Module does not return exactly 1 value. It cannot be required."}); - return errorType; + return errorRecoveryType(scope); } SeenTypes seenTypes; @@ -4078,7 +4140,7 @@ TypeId TypeChecker::quantify(const ScopePtr& scope, TypeId ty, Location location if (!qty.has_value()) { reportError(location, UnificationTooComplex{}); - return errorType; + return errorRecoveryType(scope); } if (ty == *qty) @@ -4101,7 +4163,7 @@ TypeId TypeChecker::instantiate(const ScopePtr& scope, TypeId ty, Location locat else { reportError(location, UnificationTooComplex{}); - return errorType; + return errorRecoveryType(scope); } } @@ -4116,7 +4178,7 @@ TypeId TypeChecker::anyify(const ScopePtr& scope, TypeId ty, Location location) else { reportError(location, UnificationTooComplex{}); - return errorType; + return errorRecoveryType(anyType); } } @@ -4131,7 +4193,7 @@ TypePackId TypeChecker::anyify(const ScopePtr& scope, TypePackId ty, Location lo else { reportError(location, UnificationTooComplex{}); - return errorTypePack; + return errorRecoveryTypePack(anyTypePack); } } @@ -4279,6 +4341,38 @@ TypeId TypeChecker::freshType(TypeLevel level) return currentModule->internalTypes.addType(TypeVar(FreeTypeVar(level))); } +TypeId TypeChecker::singletonType(bool value) +{ + // TODO: cache singleton types + return currentModule->internalTypes.addType(TypeVar(SingletonTypeVar(BoolSingleton{value}))); +} + +TypeId TypeChecker::singletonType(std::string value) +{ + // TODO: cache singleton types + return currentModule->internalTypes.addType(TypeVar(SingletonTypeVar(StringSingleton{std::move(value)}))); +} + +TypeId TypeChecker::errorRecoveryType(const ScopePtr& scope) +{ + return singletonTypes.errorRecoveryType(); +} + +TypeId TypeChecker::errorRecoveryType(TypeId guess) +{ + return singletonTypes.errorRecoveryType(guess); +} + +TypePackId TypeChecker::errorRecoveryTypePack(const ScopePtr& scope) +{ + return singletonTypes.errorRecoveryTypePack(); +} + +TypePackId TypeChecker::errorRecoveryTypePack(TypePackId guess) +{ + return singletonTypes.errorRecoveryTypePack(guess); +} + std::optional TypeChecker::filterMap(TypeId type, TypeIdPredicate predicate) { std::vector types = Luau::filterMap(type, predicate); @@ -4350,7 +4444,7 @@ TypeId TypeChecker::resolveType(const ScopePtr& scope, const AstType& annotation if (lit->parameters.size != 1 || !lit->parameters.data[0].type) { reportError(TypeError{annotation.location, GenericError{"_luau_print requires one generic parameter"}}); - return addType(ErrorTypeVar{}); + return errorRecoveryType(anyType); } ToStringOptions opts; @@ -4368,7 +4462,7 @@ TypeId TypeChecker::resolveType(const ScopePtr& scope, const AstType& annotation if (!tf) { if (lit->name == Parser::errorName) - return addType(ErrorTypeVar{}); + return errorRecoveryType(scope); std::string typeName; if (lit->hasPrefix) @@ -4380,7 +4474,7 @@ TypeId TypeChecker::resolveType(const ScopePtr& scope, const AstType& annotation else reportError(TypeError{annotation.location, UnknownSymbol{typeName, UnknownSymbol::Type}}); - return addType(ErrorTypeVar{}); + return errorRecoveryType(scope); } if (lit->parameters.size == 0 && tf->typeParams.empty() && (!FFlag::LuauTypeAliasPacks || tf->typePackParams.empty())) @@ -4390,14 +4484,17 @@ TypeId TypeChecker::resolveType(const ScopePtr& scope, const AstType& annotation else if (!FFlag::LuauTypeAliasPacks && lit->parameters.size != tf->typeParams.size()) { reportError(TypeError{annotation.location, IncorrectGenericParameterCount{lit->name.value, *tf, lit->parameters.size, 0}}); - return addType(ErrorTypeVar{}); + if (!FFlag::LuauErrorRecoveryType) + return errorRecoveryType(scope); } - else if (FFlag::LuauTypeAliasPacks) + + if (FFlag::LuauTypeAliasPacks) { if (!lit->hasParameterList && !tf->typePackParams.empty()) { reportError(TypeError{annotation.location, GenericError{"Type parameter list is required"}}); - return addType(ErrorTypeVar{}); + if (!FFlag::LuauErrorRecoveryType) + return errorRecoveryType(scope); } std::vector typeParams; @@ -4445,7 +4542,17 @@ TypeId TypeChecker::resolveType(const ScopePtr& scope, const AstType& annotation { reportError( TypeError{annotation.location, IncorrectGenericParameterCount{lit->name.value, *tf, typeParams.size(), typePackParams.size()}}); - return addType(ErrorTypeVar{}); + + if (FFlag::LuauErrorRecoveryType) + { + // Pad the types out with error recovery types + while (typeParams.size() < tf->typeParams.size()) + typeParams.push_back(errorRecoveryType(scope)); + while (typePackParams.size() < tf->typePackParams.size()) + typePackParams.push_back(errorRecoveryTypePack(scope)); + } + else + return errorRecoveryType(scope); } if (FFlag::LuauRecursiveTypeParameterRestriction && typeParams == tf->typeParams && typePackParams == tf->typePackParams) @@ -4464,6 +4571,14 @@ TypeId TypeChecker::resolveType(const ScopePtr& scope, const AstType& annotation for (const auto& param : lit->parameters) typeParams.push_back(resolveType(scope, *param.type)); + if (FFlag::LuauErrorRecoveryType) + { + // If there aren't enough type parameters, pad them out with error recovery types + // (we've already reported the error) + while (typeParams.size() < lit->parameters.size) + typeParams.push_back(errorRecoveryType(scope)); + } + if (FFlag::LuauRecursiveTypeParameterRestriction && typeParams == tf->typeParams) { // If the generic parameters and the type arguments are the same, we are about to @@ -4483,8 +4598,7 @@ TypeId TypeChecker::resolveType(const ScopePtr& scope, const AstType& annotation props[prop.name.value] = {resolveType(scope, *prop.type)}; if (const auto& indexer = table->indexer) - tableIndexer = TableIndexer( - resolveType(scope, *indexer->indexType), resolveType(scope, *indexer->resultType)); + tableIndexer = TableIndexer(resolveType(scope, *indexer->indexType), resolveType(scope, *indexer->resultType)); return addType(TableTypeVar{ props, tableIndexer, scope->level, @@ -4536,14 +4650,20 @@ TypeId TypeChecker::resolveType(const ScopePtr& scope, const AstType& annotation return addType(IntersectionTypeVar{types}); } - else if (annotation.is()) + else if (const auto& tsb = annotation.as()) { - return addType(ErrorTypeVar{}); + return singletonType(tsb->value); } + else if (const auto& tss = annotation.as()) + { + return singletonType(std::string(tss->value.data, tss->value.size)); + } + else if (annotation.is()) + return errorRecoveryType(scope); else { reportError(TypeError{annotation.location, GenericError{"Unknown type annotation?"}}); - return addType(ErrorTypeVar{}); + return errorRecoveryType(scope); } } @@ -4584,7 +4704,7 @@ TypePackId TypeChecker::resolveTypePack(const ScopePtr& scope, const AstTypePack else reportError(TypeError{generic->location, UnknownSymbol{genericName, UnknownSymbol::Type}}); - return addTypePack(TypePackVar{Unifiable::Error{}}); + return errorRecoveryTypePack(scope); } return *genericTy; @@ -4706,12 +4826,12 @@ TypeId TypeChecker::instantiateTypeFun(const ScopePtr& scope, const TypeFun& tf, if (!maybeInstantiated.has_value()) { reportError(location, UnificationTooComplex{}); - return errorType; + return errorRecoveryType(scope); } if (FFlag::LuauRecursiveTypeParameterRestriction && applyTypeFunction.encounteredForwardedType) { reportError(TypeError{location, GenericError{"Recursive type being used with different parameters"}}); - return errorType; + return errorRecoveryType(scope); } TypeId instantiated = *maybeInstantiated; @@ -4773,8 +4893,8 @@ TypeId TypeChecker::instantiateTypeFun(const ScopePtr& scope, const TypeFun& tf, return instantiated; } -std::pair, std::vector> TypeChecker::createGenericTypes( - const ScopePtr& scope, std::optional levelOpt, const AstNode& node, const AstArray& genericNames, const AstArray& genericPackNames) +std::pair, std::vector> TypeChecker::createGenericTypes(const ScopePtr& scope, std::optional levelOpt, + const AstNode& node, const AstArray& genericNames, const AstArray& genericPackNames) { LUAU_ASSERT(scope->parent); @@ -5030,7 +5150,9 @@ void TypeChecker::resolve(const IsAPredicate& isaP, ErrorVec& errVec, Refinement return isaP.ty; } - return std::nullopt; + // local variable works around an odd gcc 9.3 warning: may be used uninitialized + std::optional res = std::nullopt; + return res; }; std::optional ty = resolveLValue(refis, scope, isaP.lvalue); @@ -5041,7 +5163,7 @@ void TypeChecker::resolve(const IsAPredicate& isaP, ErrorVec& errVec, Refinement addRefinement(refis, isaP.lvalue, *result); else { - addRefinement(refis, isaP.lvalue, errorType); + addRefinement(refis, isaP.lvalue, errorRecoveryType(scope)); errVec.push_back(TypeError{isaP.location, TypeMismatch{isaP.ty, *ty}}); } } @@ -5105,7 +5227,7 @@ void TypeChecker::resolve(const TypeGuardPredicate& typeguardP, ErrorVec& errVec addRefinement(refis, typeguardP.lvalue, *result); else { - addRefinement(refis, typeguardP.lvalue, errorType); + addRefinement(refis, typeguardP.lvalue, errorRecoveryType(scope)); if (sense) errVec.push_back( TypeError{typeguardP.location, GenericError{"Type '" + toString(*ty) + "' has no overlap with '" + typeguardP.kind + "'"}}); @@ -5116,7 +5238,7 @@ void TypeChecker::resolve(const TypeGuardPredicate& typeguardP, ErrorVec& errVec auto fail = [&](const TypeErrorData& err) { errVec.push_back(TypeError{typeguardP.location, err}); - addRefinement(refis, typeguardP.lvalue, errorType); + addRefinement(refis, typeguardP.lvalue, errorRecoveryType(scope)); }; if (!typeguardP.isTypeof) @@ -5137,28 +5259,6 @@ void TypeChecker::resolve(const TypeGuardPredicate& typeguardP, ErrorVec& errVec return resolve(IsAPredicate{std::move(typeguardP.lvalue), typeguardP.location, type}, errVec, refis, scope, sense); } -void TypeChecker::DEPRECATED_resolve(const TypeGuardPredicate& typeguardP, ErrorVec& errVec, RefinementMap& refis, const ScopePtr& scope, bool sense) -{ - if (!sense) - return; - - static std::vector primitives{ - "string", "number", "boolean", "nil", "thread", - "table", // no op. Requires special handling. - "function", // no op. Requires special handling. - "userdata", // no op. Requires special handling. - }; - - if (auto typeFun = globalScope->lookupType(typeguardP.kind); - typeFun && typeFun->typeParams.empty() && (!FFlag::LuauTypeAliasPacks || typeFun->typePackParams.empty())) - { - if (auto it = std::find(primitives.begin(), primitives.end(), typeguardP.kind); it != primitives.end()) - addRefinement(refis, typeguardP.lvalue, typeFun->type); - else if (typeguardP.isTypeof) - addRefinement(refis, typeguardP.lvalue, typeFun->type); - } -} - void TypeChecker::resolve(const EqPredicate& eqP, ErrorVec& errVec, RefinementMap& refis, const ScopePtr& scope, bool sense) { // This refinement will require success typing to do everything correctly. For now, we can get most of the way there. diff --git a/Analysis/src/TypePack.cpp b/Analysis/src/TypePack.cpp index 228b1926..d3221c73 100644 --- a/Analysis/src/TypePack.cpp +++ b/Analysis/src/TypePack.cpp @@ -286,5 +286,4 @@ TypePack* asMutable(const TypePack* tp) { return const_cast(tp); } - } // namespace Luau diff --git a/Analysis/src/TypeVar.cpp b/Analysis/src/TypeVar.cpp index cd447ca2..924bf082 100644 --- a/Analysis/src/TypeVar.cpp +++ b/Analysis/src/TypeVar.cpp @@ -21,6 +21,7 @@ LUAU_FASTINTVARIABLE(LuauTypeMaximumStringifierLength, 500) LUAU_FASTINTVARIABLE(LuauTableTypeMaximumStringifierLength, 0) LUAU_FASTFLAG(LuauTypeAliasPacks) LUAU_FASTFLAGVARIABLE(LuauRefactorTagging, false) +LUAU_FASTFLAG(LuauErrorRecoveryType) namespace Luau { @@ -293,7 +294,7 @@ bool isGeneric(TypeId ty) bool maybeGeneric(TypeId ty) { ty = follow(ty); - if (auto ftv = get(ty)) + if (get(ty)) return true; else if (auto ttv = get(ty)) { @@ -305,6 +306,18 @@ bool maybeGeneric(TypeId ty) return isGeneric(ty); } +bool maybeSingleton(TypeId ty) +{ + ty = follow(ty); + if (get(ty)) + return true; + if (const UnionTypeVar* utv = get(ty)) + for (TypeId option : utv) + if (get(follow(option))) + return true; + return false; +} + FunctionTypeVar::FunctionTypeVar(TypePackId argTypes, TypePackId retType, std::optional defn, bool hasSelf) : argTypes(argTypes) , retType(retType) @@ -562,10 +575,8 @@ SingletonTypes::SingletonTypes() , booleanType(&booleanType_) , threadType(&threadType_) , anyType(&anyType_) - , errorType(&errorType_) , optionalNumberType(&optionalNumberType_) , anyTypePack(&anyTypePack_) - , errorTypePack(&errorTypePack_) , arena(new TypeArena) { TypeId stringMetatable = makeStringMetatable(); @@ -634,6 +645,32 @@ TypeId SingletonTypes::makeStringMetatable() return arena->addType(TableTypeVar{{{{"__index", {tableType}}}}, std::nullopt, TypeLevel{}, TableState::Sealed}); } +TypeId SingletonTypes::errorRecoveryType() +{ + return &errorType_; +} + +TypePackId SingletonTypes::errorRecoveryTypePack() +{ + return &errorTypePack_; +} + +TypeId SingletonTypes::errorRecoveryType(TypeId guess) +{ + if (FFlag::LuauErrorRecoveryType) + return guess; + else + return &errorType_; +} + +TypePackId SingletonTypes::errorRecoveryTypePack(TypePackId guess) +{ + if (FFlag::LuauErrorRecoveryType) + return guess; + else + return &errorTypePack_; +} + SingletonTypes singletonTypes; void persist(TypeId ty) @@ -1141,6 +1178,11 @@ struct QVarFinder return false; } + bool operator()(const SingletonTypeVar&) const + { + return false; + } + bool operator()(const FunctionTypeVar& ftv) const { if (hasGeneric(ftv.argTypes)) @@ -1412,7 +1454,7 @@ static std::vector parseFormatString(TypeChecker& typechecker, const cha else if (strchr(options, data[i])) result.push_back(typechecker.numberType); else - result.push_back(typechecker.errorType); + result.push_back(typechecker.errorRecoveryType(typechecker.anyType)); } } diff --git a/Analysis/src/Unifier.cpp b/Analysis/src/Unifier.cpp index 82f621b6..e1a52be4 100644 --- a/Analysis/src/Unifier.cpp +++ b/Analysis/src/Unifier.cpp @@ -22,7 +22,9 @@ LUAU_FASTFLAGVARIABLE(LuauTypecheckOpts, false) LUAU_FASTFLAG(LuauShareTxnSeen); LUAU_FASTFLAGVARIABLE(LuauCacheUnifyTableResults, false) LUAU_FASTFLAGVARIABLE(LuauExtendedTypeMismatchError, false) +LUAU_FASTFLAG(LuauSingletonTypes) LUAU_FASTFLAGVARIABLE(LuauExtendedClassMismatchError, false) +LUAU_FASTFLAG(LuauErrorRecoveryType); namespace Luau { @@ -211,6 +213,7 @@ void Unifier::tryUnify_(TypeId superTy, TypeId subTy, bool isFunctionCall, bool { occursCheck(subTy, superTy); + // The occurrence check might have caused superTy no longer to be a free type if (!get(subTy)) { log(subTy); @@ -221,10 +224,20 @@ void Unifier::tryUnify_(TypeId superTy, TypeId subTy, bool isFunctionCall, bool } else if (l && r) { - log(superTy); + if (!FFlag::LuauErrorRecoveryType) + log(superTy); occursCheck(superTy, subTy); r->level = min(r->level, l->level); - *asMutable(superTy) = BoundTypeVar(subTy); + + // The occurrence check might have caused superTy no longer to be a free type + if (!FFlag::LuauErrorRecoveryType) + *asMutable(superTy) = BoundTypeVar(subTy); + else if (!get(superTy)) + { + log(superTy); + *asMutable(superTy) = BoundTypeVar(subTy); + } + return; } else if (l) @@ -240,6 +253,7 @@ void Unifier::tryUnify_(TypeId superTy, TypeId subTy, bool isFunctionCall, bool return; } + // The occurrence check might have caused superTy no longer to be a free type if (!get(superTy)) { if (auto rightLevel = getMutableLevel(subTy)) @@ -251,6 +265,7 @@ void Unifier::tryUnify_(TypeId superTy, TypeId subTy, bool isFunctionCall, bool log(superTy); *asMutable(superTy) = BoundTypeVar(subTy); } + return; } else if (r) @@ -512,6 +527,9 @@ void Unifier::tryUnify_(TypeId superTy, TypeId subTy, bool isFunctionCall, bool else if (get(superTy) && get(subTy)) tryUnifyPrimitives(superTy, subTy); + else if (FFlag::LuauSingletonTypes && (get(superTy) || get(superTy)) && get(subTy)) + tryUnifySingletons(superTy, subTy); + else if (get(superTy) && get(subTy)) tryUnifyFunctions(superTy, subTy, isFunctionCall); @@ -723,17 +741,18 @@ void Unifier::tryUnify_(TypePackId superTp, TypePackId subTp, bool isFunctionCal { occursCheck(superTp, subTp); + // The occurrence check might have caused superTp no longer to be a free type if (!get(superTp)) { log(superTp); *asMutable(superTp) = Unifiable::Bound(subTp); } } - else if (get(subTp)) { occursCheck(subTp, superTp); + // The occurrence check might have caused superTp no longer to be a free type if (!get(subTp)) { log(subTp); @@ -874,13 +893,13 @@ void Unifier::tryUnify_(TypePackId superTp, TypePackId subTp, bool isFunctionCal while (superIter.good()) { - tryUnify_(singletonTypes.errorType, *superIter); + tryUnify_(singletonTypes.errorRecoveryType(), *superIter); superIter.advance(); } while (subIter.good()) { - tryUnify_(singletonTypes.errorType, *subIter); + tryUnify_(singletonTypes.errorRecoveryType(), *subIter); subIter.advance(); } @@ -906,6 +925,27 @@ void Unifier::tryUnifyPrimitives(TypeId superTy, TypeId subTy) errors.push_back(TypeError{location, TypeMismatch{superTy, subTy}}); } +void Unifier::tryUnifySingletons(TypeId superTy, TypeId subTy) +{ + const PrimitiveTypeVar* lp = get(superTy); + const SingletonTypeVar* ls = get(superTy); + const SingletonTypeVar* rs = get(subTy); + + if ((!lp && !ls) || !rs) + ice("passed non singleton/primitive types to unifySingletons"); + + if (ls && *ls == *rs) + return; + + if (lp && lp->type == PrimitiveTypeVar::Boolean && get(rs) && variance == Covariant) + return; + + if (lp && lp->type == PrimitiveTypeVar::String && get(rs) && variance == Covariant) + return; + + errors.push_back(TypeError{location, TypeMismatch{superTy, subTy}}); +} + void Unifier::tryUnifyFunctions(TypeId superTy, TypeId subTy, bool isFunctionCall) { FunctionTypeVar* lf = getMutable(superTy); @@ -1023,7 +1063,8 @@ void Unifier::tryUnifyTables(TypeId left, TypeId right, bool isIntersection) } // And vice versa if we're invariant - if (FFlag::LuauTableUnificationEarlyTest && variance == Invariant && !lt->indexer && lt->state != TableState::Unsealed && lt->state != TableState::Free) + if (FFlag::LuauTableUnificationEarlyTest && variance == Invariant && !lt->indexer && lt->state != TableState::Unsealed && + lt->state != TableState::Free) { for (const auto& [propName, subProp] : rt->props) { @@ -1038,7 +1079,7 @@ void Unifier::tryUnifyTables(TypeId left, TypeId right, bool isIntersection) return; } } - + // Reminder: left is the supertype, right is the subtype. // Width subtyping: any property in the supertype must be in the subtype, // and the types must agree. @@ -1634,9 +1675,8 @@ void Unifier::tryUnifyWithClass(TypeId superTy, TypeId subTy, bool reversed) { ok = false; errors.push_back(TypeError{location, UnknownProperty{superTy, propName}}); - if (!FFlag::LuauExtendedClassMismatchError) - tryUnify_(prop.type, singletonTypes.errorType); + tryUnify_(prop.type, singletonTypes.errorRecoveryType()); } else { @@ -1952,7 +1992,7 @@ void Unifier::tryUnifyWithAny(TypePackId any, TypePackId ty) { LUAU_ASSERT(get(any)); - const TypeId anyTy = singletonTypes.errorType; + const TypeId anyTy = singletonTypes.errorRecoveryType(); if (FFlag::LuauTypecheckOpts) { @@ -2046,7 +2086,7 @@ void Unifier::occursCheck(std::unordered_set& seen_DEPRECATED, DenseHash { errors.push_back(TypeError{location, OccursCheckFailed{}}); log(needle); - *asMutable(needle) = ErrorTypeVar{}; + *asMutable(needle) = *singletonTypes.errorRecoveryType(); return; } @@ -2134,7 +2174,7 @@ void Unifier::occursCheck(std::unordered_set& seen_DEPRECATED, Dense { errors.push_back(TypeError{location, OccursCheckFailed{}}); log(needle); - *asMutable(needle) = ErrorTypeVar{}; + *asMutable(needle) = *singletonTypes.errorRecoveryTypePack(); return; } diff --git a/Ast/include/Luau/Ast.h b/Ast/include/Luau/Ast.h index a2189f7b..5b4bfa03 100644 --- a/Ast/include/Luau/Ast.h +++ b/Ast/include/Luau/Ast.h @@ -255,6 +255,14 @@ public: { return visit((class AstType*)node); } + virtual bool visit(class AstTypeSingletonBool* node) + { + return visit((class AstType*)node); + } + virtual bool visit(class AstTypeSingletonString* node) + { + return visit((class AstType*)node); + } virtual bool visit(class AstTypeError* node) { return visit((class AstType*)node); @@ -1158,6 +1166,30 @@ public: unsigned messageIndex; }; +class AstTypeSingletonBool : public AstType +{ +public: + LUAU_RTTI(AstTypeSingletonBool) + + AstTypeSingletonBool(const Location& location, bool value); + + void visit(AstVisitor* visitor) override; + + bool value; +}; + +class AstTypeSingletonString : public AstType +{ +public: + LUAU_RTTI(AstTypeSingletonString) + + AstTypeSingletonString(const Location& location, const AstArray& value); + + void visit(AstVisitor* visitor) override; + + const AstArray value; +}; + class AstTypePack : public AstNode { public: diff --git a/Ast/include/Luau/Parser.h b/Ast/include/Luau/Parser.h index 39c7d925..87ebc48b 100644 --- a/Ast/include/Luau/Parser.h +++ b/Ast/include/Luau/Parser.h @@ -286,6 +286,7 @@ private: // `<' typeAnnotation[, ...] `>' AstArray parseTypeParams(); + std::optional> parseCharArray(); AstExpr* parseString(); AstLocal* pushLocal(const Binding& binding); diff --git a/Ast/include/Luau/StringUtils.h b/Ast/include/Luau/StringUtils.h index 4f7673fa..6ecf0606 100644 --- a/Ast/include/Luau/StringUtils.h +++ b/Ast/include/Luau/StringUtils.h @@ -34,4 +34,6 @@ bool equalsLower(std::string_view lhs, std::string_view rhs); size_t hashRange(const char* data, size_t size); +std::string escape(std::string_view s); +bool isIdentifier(std::string_view s); } // namespace Luau diff --git a/Ast/src/Ast.cpp b/Ast/src/Ast.cpp index b1209faa..e709894d 100644 --- a/Ast/src/Ast.cpp +++ b/Ast/src/Ast.cpp @@ -841,6 +841,28 @@ void AstTypeIntersection::visit(AstVisitor* visitor) } } +AstTypeSingletonBool::AstTypeSingletonBool(const Location& location, bool value) + : AstType(ClassIndex(), location) + , value(value) +{ +} + +void AstTypeSingletonBool::visit(AstVisitor* visitor) +{ + visitor->visit(this); +} + +AstTypeSingletonString::AstTypeSingletonString(const Location& location, const AstArray& value) + : AstType(ClassIndex(), location) + , value(value) +{ +} + +void AstTypeSingletonString::visit(AstVisitor* visitor) +{ + visitor->visit(this); +} + AstTypeError::AstTypeError(const Location& location, const AstArray& types, bool isMissing, unsigned messageIndex) : AstType(ClassIndex(), location) , types(types) diff --git a/Ast/src/Parser.cpp b/Ast/src/Parser.cpp index a1bad65e..bc63e37d 100644 --- a/Ast/src/Parser.cpp +++ b/Ast/src/Parser.cpp @@ -16,6 +16,7 @@ LUAU_FASTFLAGVARIABLE(LuauIfStatementRecursionGuard, false) LUAU_FASTFLAGVARIABLE(LuauTypeAliasPacks, false) LUAU_FASTFLAGVARIABLE(LuauParseTypePackTypeParameters, false) LUAU_FASTFLAGVARIABLE(LuauFixAmbiguousErrorRecoveryInAssign, false) +LUAU_FASTFLAGVARIABLE(LuauParseSingletonTypes, false) LUAU_FASTFLAGVARIABLE(LuauParseGenericFunctionTypeBegin, false) namespace Luau @@ -1278,7 +1279,27 @@ AstType* Parser::parseTableTypeAnnotation() while (lexer.current().type != '}') { - if (lexer.current().type == '[') + if (FFlag::LuauParseSingletonTypes && lexer.current().type == '[' && + (lexer.lookahead().type == Lexeme::RawString || lexer.lookahead().type == Lexeme::QuotedString)) + { + const Lexeme begin = lexer.current(); + nextLexeme(); // [ + std::optional> chars = parseCharArray(); + + expectMatchAndConsume(']', begin); + expectAndConsume(':', "table field"); + + AstType* type = parseTypeAnnotation(); + + // TODO: since AstName conains a char*, it can't contain null + bool containsNull = chars && (strnlen(chars->data, chars->size) < chars->size); + + if (chars && !containsNull) + props.push_back({AstName(chars->data), begin.location, type}); + else + report(begin.location, "String literal contains malformed escape sequence"); + } + else if (lexer.current().type == '[') { if (indexer) { @@ -1528,6 +1549,32 @@ AstTypeOrPack Parser::parseSimpleTypeAnnotation(bool allowPack) nextLexeme(); return {allocator.alloc(begin, std::nullopt, nameNil), {}}; } + else if (FFlag::LuauParseSingletonTypes && lexer.current().type == Lexeme::ReservedTrue) + { + nextLexeme(); + return {allocator.alloc(begin, true)}; + } + else if (FFlag::LuauParseSingletonTypes && lexer.current().type == Lexeme::ReservedFalse) + { + nextLexeme(); + return {allocator.alloc(begin, false)}; + } + else if (FFlag::LuauParseSingletonTypes && (lexer.current().type == Lexeme::RawString || lexer.current().type == Lexeme::QuotedString)) + { + if (std::optional> value = parseCharArray()) + { + AstArray svalue = *value; + return {allocator.alloc(begin, svalue)}; + } + else + return {reportTypeAnnotationError(begin, {}, /*isMissing*/ false, "String literal contains malformed escape sequence")}; + } + else if (FFlag::LuauParseSingletonTypes && lexer.current().type == Lexeme::BrokenString) + { + Location location = lexer.current().location; + nextLexeme(); + return {reportTypeAnnotationError(location, {}, /*isMissing*/ false, "Malformed string")}; + } else if (lexer.current().type == Lexeme::Name) { std::optional prefix; @@ -2416,7 +2463,7 @@ AstArray Parser::parseTypeParams() return copy(parameters); } -AstExpr* Parser::parseString() +std::optional> Parser::parseCharArray() { LUAU_ASSERT(lexer.current().type == Lexeme::QuotedString || lexer.current().type == Lexeme::RawString); @@ -2426,11 +2473,8 @@ AstExpr* Parser::parseString() { if (!Lexer::fixupQuotedString(scratchData)) { - Location location = lexer.current().location; - nextLexeme(); - - return reportExprError(location, {}, "String literal contains malformed escape sequence"); + return std::nullopt; } } else @@ -2438,12 +2482,18 @@ AstExpr* Parser::parseString() Lexer::fixupMultilineString(scratchData); } - Location start = lexer.current().location; AstArray value = copy(scratchData); - nextLexeme(); + return value; +} - return allocator.alloc(start, value); +AstExpr* Parser::parseString() +{ + Location location = lexer.current().location; + if (std::optional> value = parseCharArray()) + return allocator.alloc(location, *value); + else + return reportExprError(location, {}, "String literal contains malformed escape sequence"); } AstLocal* Parser::pushLocal(const Binding& binding) diff --git a/Ast/src/StringUtils.cpp b/Ast/src/StringUtils.cpp index 24b2283a..9c7fed31 100644 --- a/Ast/src/StringUtils.cpp +++ b/Ast/src/StringUtils.cpp @@ -225,4 +225,62 @@ size_t hashRange(const char* data, size_t size) return hash; } +bool isIdentifier(std::string_view s) +{ + return (s.find_first_not_of("abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890_") == std::string::npos); +} + +std::string escape(std::string_view s) +{ + std::string r; + r.reserve(s.size() + 50); // arbitrary number to guess how many characters we'll be inserting + + for (uint8_t c : s) + { + if (c >= ' ' && c != '\\' && c != '\'' && c != '\"') + r += c; + else + { + r += '\\'; + + switch (c) + { + case '\a': + r += 'a'; + break; + case '\b': + r += 'b'; + break; + case '\f': + r += 'f'; + break; + case '\n': + r += 'n'; + break; + case '\r': + r += 'r'; + break; + case '\t': + r += 't'; + break; + case '\v': + r += 'v'; + break; + case '\'': + r += '\''; + break; + case '\"': + r += '\"'; + break; + case '\\': + r += '\\'; + break; + default: + Luau::formatAppend(r, "%03u", c); + } + } + } + + return r; +} } // namespace Luau diff --git a/CLI/Analyze.cpp b/CLI/Analyze.cpp index 9ab10aaf..ebdd7896 100644 --- a/CLI/Analyze.cpp +++ b/CLI/Analyze.cpp @@ -236,32 +236,12 @@ int main(int argc, char** argv) Luau::registerBuiltinTypes(frontend.typeChecker); Luau::freeze(frontend.typeChecker.globalTypes); + std::vector files = getSourceFiles(argc, argv); + int failed = 0; - for (int i = 1; i < argc; ++i) - { - if (argv[i][0] == '-') - continue; - - if (isDirectory(argv[i])) - { - traverseDirectory(argv[i], [&](const std::string& name) { - // Look for .luau first and if absent, fall back to .lua - if (name.length() > 5 && name.rfind(".luau") == name.length() - 5) - { - failed += !analyzeFile(frontend, name.c_str(), format, annotate); - } - else if (name.length() > 4 && name.rfind(".lua") == name.length() - 4) - { - failed += !analyzeFile(frontend, name.c_str(), format, annotate); - } - }); - } - else - { - failed += !analyzeFile(frontend, argv[i], format, annotate); - } - } + for (const std::string& path : files) + failed += !analyzeFile(frontend, path.c_str(), format, annotate); if (!configResolver.configErrors.empty()) { diff --git a/CLI/FileUtils.cpp b/CLI/FileUtils.cpp index 0702b74f..b3c9557b 100644 --- a/CLI/FileUtils.cpp +++ b/CLI/FileUtils.cpp @@ -142,6 +142,7 @@ static bool traverseDirectoryRec(const std::string& path, const std::function getParentPath(const std::string& path) return ""; } + +static std::string getExtension(const std::string& path) +{ + std::string::size_type dot = path.find_last_of(".\\/"); + + if (dot == std::string::npos || path[dot] != '.') + return ""; + + return path.substr(dot); +} + +std::vector getSourceFiles(int argc, char** argv) +{ + std::vector files; + + for (int i = 1; i < argc; ++i) + { + if (argv[i][0] == '-') + continue; + + if (isDirectory(argv[i])) + { + traverseDirectory(argv[i], [&](const std::string& name) { + std::string ext = getExtension(name); + + if (ext == ".lua" || ext == ".luau") + files.push_back(name); + }); + } + else + { + files.push_back(argv[i]); + } + } + + return files; +} diff --git a/CLI/FileUtils.h b/CLI/FileUtils.h index f7fbe8af..da11f512 100644 --- a/CLI/FileUtils.h +++ b/CLI/FileUtils.h @@ -4,6 +4,7 @@ #include #include #include +#include std::optional readFile(const std::string& name); @@ -12,3 +13,5 @@ bool traverseDirectory(const std::string& path, const std::function getParentPath(const std::string& path); + +std::vector getSourceFiles(int argc, char** argv); diff --git a/CLI/Repl.cpp b/CLI/Repl.cpp index 5c904cca..b29cd6f9 100644 --- a/CLI/Repl.cpp +++ b/CLI/Repl.cpp @@ -20,7 +20,7 @@ enum class CompileFormat { - Default, + Text, Binary }; @@ -33,7 +33,7 @@ static int lua_loadstring(lua_State* L) lua_setsafeenv(L, LUA_ENVIRONINDEX, false); std::string bytecode = Luau::compile(std::string(s, l)); - if (luau_load(L, chunkname, bytecode.data(), bytecode.size()) == 0) + if (luau_load(L, chunkname, bytecode.data(), bytecode.size(), 0) == 0) return 1; lua_pushnil(L); @@ -80,7 +80,7 @@ static int lua_require(lua_State* L) // now we can compile & run module on the new thread std::string bytecode = Luau::compile(*source); - if (luau_load(ML, chunkname.c_str(), bytecode.data(), bytecode.size()) == 0) + if (luau_load(ML, chunkname.c_str(), bytecode.data(), bytecode.size(), 0) == 0) { int status = lua_resume(ML, L, 0); @@ -151,7 +151,7 @@ static std::string runCode(lua_State* L, const std::string& source) { std::string bytecode = Luau::compile(source); - if (luau_load(L, "=stdin", bytecode.data(), bytecode.size()) != 0) + if (luau_load(L, "=stdin", bytecode.data(), bytecode.size(), 0) != 0) { size_t len; const char* msg = lua_tolstring(L, -1, &len); @@ -370,7 +370,7 @@ static bool runFile(const char* name, lua_State* GL) std::string bytecode = Luau::compile(*source); int status = 0; - if (luau_load(L, chunkname.c_str(), bytecode.data(), bytecode.size()) == 0) + if (luau_load(L, chunkname.c_str(), bytecode.data(), bytecode.size(), 0) == 0) { status = lua_resume(L, NULL, 0); } @@ -379,11 +379,7 @@ static bool runFile(const char* name, lua_State* GL) status = LUA_ERRSYNTAX; } - if (status == 0) - { - return true; - } - else + if (status != 0) { std::string error; @@ -400,8 +396,10 @@ static bool runFile(const char* name, lua_State* GL) error += lua_debugtrace(L); fprintf(stderr, "%s", error.c_str()); - return false; } + + lua_pop(GL, 1); + return status == 0; } static void report(const char* name, const Luau::Location& location, const char* type, const char* message) @@ -431,14 +429,18 @@ static bool compileFile(const char* name, CompileFormat format) try { Luau::BytecodeBuilder bcb; - bcb.setDumpFlags(Luau::BytecodeBuilder::Dump_Code | Luau::BytecodeBuilder::Dump_Source); - bcb.setDumpSource(*source); + + if (format == CompileFormat::Text) + { + bcb.setDumpFlags(Luau::BytecodeBuilder::Dump_Code | Luau::BytecodeBuilder::Dump_Source); + bcb.setDumpSource(*source); + } Luau::compileOrThrow(bcb, *source); switch (format) { - case CompileFormat::Default: + case CompileFormat::Text: printf("%s", bcb.dumpEverything().c_str()); break; case CompileFormat::Binary: @@ -504,7 +506,7 @@ int main(int argc, char** argv) if (argc >= 2 && strncmp(argv[1], "--compile", strlen("--compile")) == 0) { - CompileFormat format = CompileFormat::Default; + CompileFormat format = CompileFormat::Text; if (strcmp(argv[1], "--compile=binary") == 0) format = CompileFormat::Binary; @@ -514,27 +516,12 @@ int main(int argc, char** argv) _setmode(_fileno(stdout), _O_BINARY); #endif + std::vector files = getSourceFiles(argc, argv); + int failed = 0; - for (int i = 2; i < argc; ++i) - { - if (argv[i][0] == '-') - continue; - - if (isDirectory(argv[i])) - { - traverseDirectory(argv[i], [&](const std::string& name) { - if (name.length() > 5 && name.rfind(".luau") == name.length() - 5) - failed += !compileFile(name.c_str(), format); - else if (name.length() > 4 && name.rfind(".lua") == name.length() - 4) - failed += !compileFile(name.c_str(), format); - }); - } - else - { - failed += !compileFile(argv[i], format); - } - } + for (const std::string& path : files) + failed += !compileFile(path.c_str(), format); return failed; } @@ -548,33 +535,25 @@ int main(int argc, char** argv) int profile = 0; for (int i = 1; i < argc; ++i) + { + if (argv[i][0] != '-') + continue; + if (strcmp(argv[i], "--profile") == 0) profile = 10000; // default to 10 KHz else if (strncmp(argv[i], "--profile=", 10) == 0) profile = atoi(argv[i] + 10); + } if (profile) profilerStart(L, profile); + std::vector files = getSourceFiles(argc, argv); + int failed = 0; - for (int i = 1; i < argc; ++i) - { - if (argv[i][0] == '-') - continue; - - if (isDirectory(argv[i])) - { - traverseDirectory(argv[i], [&](const std::string& name) { - if (name.length() > 4 && name.rfind(".lua") == name.length() - 4) - failed += !runFile(name.c_str(), L); - }); - } - else - { - failed += !runFile(argv[i], L); - } - } + for (const std::string& path : files) + failed += !runFile(path.c_str(), L); if (profile) { diff --git a/CMakeLists.txt b/CMakeLists.txt index 36014a98..9c69521e 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -9,6 +9,7 @@ project(Luau LANGUAGES CXX) option(LUAU_BUILD_CLI "Build CLI" ON) option(LUAU_BUILD_TESTS "Build tests" ON) +option(LUAU_WERROR "Warnings as errors" OFF) add_library(Luau.Ast STATIC) add_library(Luau.Compiler STATIC) @@ -57,11 +58,18 @@ set(LUAU_OPTIONS) if(MSVC) list(APPEND LUAU_OPTIONS /D_CRT_SECURE_NO_WARNINGS) # We need to use the portable CRT functions. - list(APPEND LUAU_OPTIONS /WX) # Warnings are errors list(APPEND LUAU_OPTIONS /MP) # Distribute single project compilation across multiple cores else() list(APPEND LUAU_OPTIONS -Wall) # All warnings - list(APPEND LUAU_OPTIONS -Werror) # Warnings are errors +endif() + +# Enabled in CI; we should be warning free on our main compiler versions but don't guarantee being warning free everywhere +if(LUAU_WERROR) + if(MSVC) + list(APPEND LUAU_OPTIONS /WX) # Warnings are errors + else() + list(APPEND LUAU_OPTIONS -Werror) # Warnings are errors + endif() endif() target_compile_options(Luau.Ast PRIVATE ${LUAU_OPTIONS}) @@ -79,7 +87,10 @@ if(LUAU_BUILD_CLI) target_link_libraries(Luau.Repl.CLI PRIVATE Luau.Compiler Luau.VM) if(UNIX) - target_link_libraries(Luau.Repl.CLI PRIVATE pthread) + find_library(LIBPTHREAD pthread) + if (LIBPTHREAD) + target_link_libraries(Luau.Repl.CLI PRIVATE pthread) + endif() endif() if(NOT EMSCRIPTEN) diff --git a/Compiler/include/Luau/Compiler.h b/Compiler/include/Luau/Compiler.h index 4f88e602..65e962da 100644 --- a/Compiler/include/Luau/Compiler.h +++ b/Compiler/include/Luau/Compiler.h @@ -13,11 +13,9 @@ class AstNameTable; class BytecodeBuilder; class BytecodeEncoder; +// Note: this structure is duplicated in luacode.h, don't forget to change these in sync! struct CompileOptions { - // default bytecode version target; can be used to compile code for older clients - int bytecodeVersion = 1; - // 0 - no optimization // 1 - baseline optimization level that doesn't prevent debuggability // 2 - includes optimizations that harm debuggability such as inlining diff --git a/Compiler/include/luacode.h b/Compiler/include/luacode.h new file mode 100644 index 00000000..e235a2e7 --- /dev/null +++ b/Compiler/include/luacode.h @@ -0,0 +1,39 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +#pragma once + +#include + +/* Can be used to reconfigure visibility/exports for public APIs */ +#ifndef LUACODE_API +#define LUACODE_API extern +#endif + +typedef struct lua_CompileOptions lua_CompileOptions; + +struct lua_CompileOptions +{ + // 0 - no optimization + // 1 - baseline optimization level that doesn't prevent debuggability + // 2 - includes optimizations that harm debuggability such as inlining + int optimizationLevel; // default=1 + + // 0 - no debugging support + // 1 - line info & function names only; sufficient for backtraces + // 2 - full debug info with local & upvalue names; necessary for debugger + int debugLevel; // default=1 + + // 0 - no code coverage support + // 1 - statement coverage + // 2 - statement and expression coverage (verbose) + int coverageLevel; // default=0 + + // global builtin to construct vectors; disabled by default + const char* vectorLib; + const char* vectorCtor; + + // null-terminated array of globals that are mutable; disables the import optimization for fields accessed through these + const char** mutableGlobals; +}; + +/* compile source to bytecode; when source compilation fails, the resulting bytecode contains the encoded error. use free() to destroy */ +LUACODE_API char* luau_compile(const char* source, size_t size, lua_CompileOptions* options, size_t* outsize); diff --git a/Compiler/src/Compiler.cpp b/Compiler/src/Compiler.cpp index 9712f02f..5b93c1dc 100644 --- a/Compiler/src/Compiler.cpp +++ b/Compiler/src/Compiler.cpp @@ -11,9 +11,6 @@ #include LUAU_FASTFLAGVARIABLE(LuauPreloadClosures, false) -LUAU_FASTFLAGVARIABLE(LuauPreloadClosuresFenv, false) -LUAU_FASTFLAGVARIABLE(LuauPreloadClosuresUpval, false) -LUAU_FASTFLAGVARIABLE(LuauGenericSpecialGlobals, false) LUAU_FASTFLAG(LuauIfElseExpressionBaseSupport) LUAU_FASTFLAGVARIABLE(LuauBit32CountBuiltin, false) @@ -24,9 +21,6 @@ static const uint32_t kMaxRegisterCount = 255; static const uint32_t kMaxUpvalueCount = 200; static const uint32_t kMaxLocalCount = 200; -// TODO: Remove with LuauGenericSpecialGlobals -static const char* kSpecialGlobals[] = {"Game", "Workspace", "_G", "game", "plugin", "script", "shared", "workspace"}; - CompileError::CompileError(const Location& location, const std::string& message) : location(location) , message(message) @@ -466,7 +460,7 @@ struct Compiler bool shared = false; - if (FFlag::LuauPreloadClosuresUpval) + if (FFlag::LuauPreloadClosures) { // Optimization: when closure has no upvalues, or upvalues are safe to share, instead of allocating it every time we can share closure // objects (this breaks assumptions about function identity which can lead to setfenv not working as expected, so we disable this when it @@ -482,18 +476,6 @@ struct Compiler } } } - // Optimization: when closure has no upvalues, instead of allocating it every time we can share closure objects - // (this breaks assumptions about function identity which can lead to setfenv not working as expected, so we disable this when it is used) - else if (FFlag::LuauPreloadClosures && options.optimizationLevel >= 1 && f->upvals.empty() && !setfenvUsed) - { - int32_t cid = bytecode.addConstantClosure(f->id); - - if (cid >= 0 && cid < 32768) - { - bytecode.emitAD(LOP_DUPCLOSURE, target, cid); - return; - } - } if (!shared) bytecode.emitAD(LOP_NEWCLOSURE, target, pid); @@ -3298,8 +3280,7 @@ struct Compiler bool visit(AstStatLocalFunction* node) override { // record local->function association for some optimizations - if (FFlag::LuauPreloadClosuresUpval) - self->locals[node->name].func = node->func; + self->locals[node->name].func = node->func; return true; } @@ -3711,24 +3692,13 @@ void compileOrThrow(BytecodeBuilder& bytecode, AstStatBlock* root, const AstName Compiler compiler(bytecode, options); // since access to some global objects may result in values that change over time, we block imports from non-readonly tables - if (FFlag::LuauGenericSpecialGlobals) - { - if (AstName name = names.get("_G"); name.value) - compiler.globals[name].writable = true; + if (AstName name = names.get("_G"); name.value) + compiler.globals[name].writable = true; - if (options.mutableGlobals) - for (const char** ptr = options.mutableGlobals; *ptr; ++ptr) - if (AstName name = names.get(*ptr); name.value) - compiler.globals[name].writable = true; - } - else - { - for (const char* global : kSpecialGlobals) - { - if (AstName name = names.get(global); name.value) + if (options.mutableGlobals) + for (const char** ptr = options.mutableGlobals; *ptr; ++ptr) + if (AstName name = names.get(*ptr); name.value) compiler.globals[name].writable = true; - } - } // this visitor traverses the AST to analyze mutability of locals/globals, filling Local::written and Global::written Compiler::AssignmentVisitor assignmentVisitor(&compiler); @@ -3742,7 +3712,7 @@ void compileOrThrow(BytecodeBuilder& bytecode, AstStatBlock* root, const AstName } // this visitor tracks calls to getfenv/setfenv and disables some optimizations when they are found - if (FFlag::LuauPreloadClosuresFenv && options.optimizationLevel >= 1 && (names.get("getfenv").value || names.get("setfenv").value)) + if (options.optimizationLevel >= 1 && (names.get("getfenv").value || names.get("setfenv").value)) { Compiler::FenvVisitor fenvVisitor(compiler.getfenvUsed, compiler.setfenvUsed); root->visit(&fenvVisitor); diff --git a/Compiler/src/lcode.cpp b/Compiler/src/lcode.cpp new file mode 100644 index 00000000..ee150b17 --- /dev/null +++ b/Compiler/src/lcode.cpp @@ -0,0 +1,29 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +#include "luacode.h" + +#include "Luau/Compiler.h" + +#include + +char* luau_compile(const char* source, size_t size, lua_CompileOptions* options, size_t* outsize) +{ + LUAU_ASSERT(outsize); + + Luau::CompileOptions opts; + + if (options) + { + static_assert(sizeof(lua_CompileOptions) == sizeof(Luau::CompileOptions), "C and C++ interface must match"); + memcpy(static_cast(&opts), options, sizeof(opts)); + } + + std::string result = compile(std::string(source, size), opts); + + char* copy = static_cast(malloc(result.size())); + if (!copy) + return nullptr; + + memcpy(copy, result.data(), result.size()); + *outsize = result.size(); + return copy; +} diff --git a/Makefile b/Makefile index 5d51b3d4..cab3d43f 100644 --- a/Makefile +++ b/Makefile @@ -46,14 +46,20 @@ endif OBJECTS=$(AST_OBJECTS) $(COMPILER_OBJECTS) $(ANALYSIS_OBJECTS) $(VM_OBJECTS) $(TESTS_OBJECTS) $(CLI_OBJECTS) $(FUZZ_OBJECTS) # common flags -CXXFLAGS=-g -Wall -Werror +CXXFLAGS=-g -Wall LDFLAGS= -# temporary, for older gcc versions as they treat var in `if (type var = val)` as unused +# some gcc versions treat var in `if (type var = val)` as unused +# some gcc versions treat variables used in constexpr if blocks as unused ifeq ($(findstring g++,$(shell $(CXX) --version)),g++) CXXFLAGS+=-Wno-unused endif +# enabled in CI; we should be warning free on our main compiler versions but don't guarantee being warning free everywhere +ifneq ($(werror),) + CXXFLAGS+=-Werror +endif + # configuration-specific flags ifeq ($(config),release) CXXFLAGS+=-O2 -DNDEBUG diff --git a/Sources.cmake b/Sources.cmake index c30cf77d..23b931c6 100644 --- a/Sources.cmake +++ b/Sources.cmake @@ -25,9 +25,11 @@ target_sources(Luau.Compiler PRIVATE Compiler/include/Luau/Bytecode.h Compiler/include/Luau/BytecodeBuilder.h Compiler/include/Luau/Compiler.h + Compiler/include/luacode.h Compiler/src/BytecodeBuilder.cpp Compiler/src/Compiler.cpp + Compiler/src/lcode.cpp ) # Luau.Analysis Sources @@ -204,6 +206,7 @@ if(TARGET Luau.UnitTest) tests/TypeInfer.intersectionTypes.test.cpp tests/TypeInfer.provisional.test.cpp tests/TypeInfer.refinements.test.cpp + tests/TypeInfer.singletons.test.cpp tests/TypeInfer.tables.test.cpp tests/TypeInfer.test.cpp tests/TypeInfer.tryUnify.test.cpp diff --git a/VM/include/lua.h b/VM/include/lua.h index a9d3e875..1568d191 100644 --- a/VM/include/lua.h +++ b/VM/include/lua.h @@ -102,6 +102,8 @@ LUA_API lua_State* lua_newstate(lua_Alloc f, void* ud); LUA_API void lua_close(lua_State* L); LUA_API lua_State* lua_newthread(lua_State* L); LUA_API lua_State* lua_mainthread(lua_State* L); +LUA_API void lua_resetthread(lua_State* L); +LUA_API int lua_isthreadreset(lua_State* L); /* ** basic stack manipulation @@ -162,8 +164,7 @@ LUA_API void lua_pushlstring(lua_State* L, const char* s, size_t l); LUA_API void lua_pushstring(lua_State* L, const char* s); LUA_API const char* lua_pushvfstring(lua_State* L, const char* fmt, va_list argp); LUA_API LUA_PRINTF_ATTR(2, 3) const char* lua_pushfstringL(lua_State* L, const char* fmt, ...); -LUA_API void lua_pushcfunction( - lua_State* L, lua_CFunction fn, const char* debugname = NULL, int nup = 0, lua_Continuation cont = NULL); +LUA_API void lua_pushcclosurek(lua_State* L, lua_CFunction fn, const char* debugname, int nup, lua_Continuation cont); LUA_API void lua_pushboolean(lua_State* L, int b); LUA_API void lua_pushlightuserdata(lua_State* L, void* p); LUA_API int lua_pushthread(lua_State* L); @@ -178,9 +179,9 @@ LUA_API void lua_rawget(lua_State* L, int idx); LUA_API void lua_rawgeti(lua_State* L, int idx, int n); LUA_API void lua_createtable(lua_State* L, int narr, int nrec); -LUA_API void lua_setreadonly(lua_State* L, int idx, bool value); +LUA_API void lua_setreadonly(lua_State* L, int idx, int enabled); LUA_API int lua_getreadonly(lua_State* L, int idx); -LUA_API void lua_setsafeenv(lua_State* L, int idx, bool value); +LUA_API void lua_setsafeenv(lua_State* L, int idx, int enabled); LUA_API void* lua_newuserdata(lua_State* L, size_t sz, int tag); LUA_API void* lua_newuserdatadtor(lua_State* L, size_t sz, void (*dtor)(void*)); @@ -200,7 +201,7 @@ LUA_API int lua_setfenv(lua_State* L, int idx); /* ** `load' and `call' functions (load and run Luau bytecode) */ -LUA_API int luau_load(lua_State* L, const char* chunkname, const char* data, size_t size, int env = 0); +LUA_API int luau_load(lua_State* L, const char* chunkname, const char* data, size_t size, int env); LUA_API void lua_call(lua_State* L, int nargs, int nresults); LUA_API int lua_pcall(lua_State* L, int nargs, int nresults, int errfunc); @@ -293,6 +294,8 @@ LUA_API void lua_unref(lua_State* L, int ref); #define lua_isnoneornil(L, n) (lua_type(L, (n)) <= LUA_TNIL) #define lua_pushliteral(L, s) lua_pushlstring(L, "" s, (sizeof(s) / sizeof(char)) - 1) +#define lua_pushcfunction(L, fn, debugname) lua_pushcclosurek(L, fn, debugname, 0, NULL) +#define lua_pushcclosure(L, fn, debugname, nup) lua_pushcclosurek(L, fn, debugname, nup, NULL) #define lua_setglobal(L, s) lua_setfield(L, LUA_GLOBALSINDEX, (s)) #define lua_getglobal(L, s) lua_getfield(L, LUA_GLOBALSINDEX, (s)) @@ -319,8 +322,8 @@ LUA_API const char* lua_setlocal(lua_State* L, int level, int n); LUA_API const char* lua_getupvalue(lua_State* L, int funcindex, int n); LUA_API const char* lua_setupvalue(lua_State* L, int funcindex, int n); -LUA_API void lua_singlestep(lua_State* L, bool singlestep); -LUA_API void lua_breakpoint(lua_State* L, int funcindex, int line, bool enable); +LUA_API void lua_singlestep(lua_State* L, int enabled); +LUA_API void lua_breakpoint(lua_State* L, int funcindex, int line, int enabled); /* Warning: this function is not thread-safe since it stores the result in a shared global array! Only use for debugging. */ LUA_API const char* lua_debugtrace(lua_State* L); @@ -361,6 +364,7 @@ struct lua_Callbacks void (*debuginterrupt)(lua_State* L, lua_Debug* ar); /* gets called when thread execution is interrupted by break in another thread */ void (*debugprotectederror)(lua_State* L); /* gets called when protected call results in an error */ }; +typedef struct lua_Callbacks lua_Callbacks; LUA_API lua_Callbacks* lua_callbacks(lua_State* L); diff --git a/VM/include/lualib.h b/VM/include/lualib.h index 30cffaff..fa836955 100644 --- a/VM/include/lualib.h +++ b/VM/include/lualib.h @@ -8,11 +8,12 @@ #define luaL_typeerror(L, narg, tname) luaL_typeerrorL(L, narg, tname) #define luaL_argerror(L, narg, extramsg) luaL_argerrorL(L, narg, extramsg) -typedef struct luaL_Reg +struct luaL_Reg { const char* name; lua_CFunction func; -} luaL_Reg; +}; +typedef struct luaL_Reg luaL_Reg; LUALIB_API void luaL_register(lua_State* L, const char* libname, const luaL_Reg* l); LUALIB_API int luaL_getmetafield(lua_State* L, int obj, const char* e); @@ -75,6 +76,7 @@ struct luaL_Buffer struct TString* storage; char buffer[LUA_BUFFERSIZE]; }; +typedef struct luaL_Buffer luaL_Buffer; // when internal buffer storage is exhausted, a mutable string value 'storage' will be placed on the stack // in general, functions expect the mutable string buffer to be placed on top of the stack (top-1) diff --git a/VM/src/lapi.cpp b/VM/src/lapi.cpp index 7e742644..a79ba0d4 100644 --- a/VM/src/lapi.cpp +++ b/VM/src/lapi.cpp @@ -593,7 +593,7 @@ const char* lua_pushfstringL(lua_State* L, const char* fmt, ...) return ret; } -void lua_pushcfunction(lua_State* L, lua_CFunction fn, const char* debugname, int nup, lua_Continuation cont) +void lua_pushcclosurek(lua_State* L, lua_CFunction fn, const char* debugname, int nup, lua_Continuation cont) { luaC_checkGC(L); luaC_checkthreadsleep(L); @@ -698,13 +698,13 @@ void lua_createtable(lua_State* L, int narray, int nrec) return; } -void lua_setreadonly(lua_State* L, int objindex, bool value) +void lua_setreadonly(lua_State* L, int objindex, int enabled) { const TValue* o = index2adr(L, objindex); api_check(L, ttistable(o)); Table* t = hvalue(o); api_check(L, t != hvalue(registry(L))); - t->readonly = value; + t->readonly = bool(enabled); return; } @@ -717,12 +717,12 @@ int lua_getreadonly(lua_State* L, int objindex) return res; } -void lua_setsafeenv(lua_State* L, int objindex, bool value) +void lua_setsafeenv(lua_State* L, int objindex, int enabled) { const TValue* o = index2adr(L, objindex); api_check(L, ttistable(o)); Table* t = hvalue(o); - t->safeenv = value; + t->safeenv = bool(enabled); return; } diff --git a/VM/src/lbaselib.cpp b/VM/src/lbaselib.cpp index 87fc1631..61798e2b 100644 --- a/VM/src/lbaselib.cpp +++ b/VM/src/lbaselib.cpp @@ -436,8 +436,8 @@ static const luaL_Reg base_funcs[] = { static void auxopen(lua_State* L, const char* name, lua_CFunction f, lua_CFunction u) { - lua_pushcfunction(L, u); - lua_pushcfunction(L, f, name, 1); + lua_pushcfunction(L, u, NULL); + lua_pushcclosure(L, f, name, 1); lua_setfield(L, -2, name); } @@ -456,10 +456,10 @@ LUALIB_API int luaopen_base(lua_State* L) auxopen(L, "ipairs", luaB_ipairs, luaB_inext); auxopen(L, "pairs", luaB_pairs, luaB_next); - lua_pushcfunction(L, luaB_pcally, "pcall", 0, luaB_pcallcont); + lua_pushcclosurek(L, luaB_pcally, "pcall", 0, luaB_pcallcont); lua_setfield(L, -2, "pcall"); - lua_pushcfunction(L, luaB_xpcally, "xpcall", 0, luaB_xpcallcont); + lua_pushcclosurek(L, luaB_xpcally, "xpcall", 0, luaB_xpcallcont); lua_setfield(L, -2, "xpcall"); return 1; diff --git a/VM/src/lbitlib.cpp b/VM/src/lbitlib.cpp index c72fe674..907c43c4 100644 --- a/VM/src/lbitlib.cpp +++ b/VM/src/lbitlib.cpp @@ -2,6 +2,7 @@ // This code is based on Lua 5.x implementation licensed under MIT License; see lua_LICENSE.txt for details #include "lualib.h" +#include "lcommon.h" #include "lnumutils.h" LUAU_FASTFLAGVARIABLE(LuauBit32Count, false) diff --git a/VM/src/lcorolib.cpp b/VM/src/lcorolib.cpp index 9724c0e7..0178fae8 100644 --- a/VM/src/lcorolib.cpp +++ b/VM/src/lcorolib.cpp @@ -5,6 +5,8 @@ #include "lstate.h" #include "lvm.h" +LUAU_FASTFLAGVARIABLE(LuauCoroutineClose, false) + #define CO_RUN 0 /* running */ #define CO_SUS 1 /* suspended */ #define CO_NOR 2 /* 'normal' (it resumed another coroutine) */ @@ -208,8 +210,7 @@ static int cowrap(lua_State* L) { cocreate(L); - lua_pushcfunction(L, auxwrapy, NULL, 1, auxwrapcont); - + lua_pushcclosurek(L, auxwrapy, NULL, 1, auxwrapcont); return 1; } @@ -232,6 +233,34 @@ static int coyieldable(lua_State* L) return 1; } +static int coclose(lua_State* L) +{ + if (!FFlag::LuauCoroutineClose) + luaL_error(L, "coroutine.close is not enabled"); + + lua_State* co = lua_tothread(L, 1); + luaL_argexpected(L, co, 1, "thread"); + + int status = auxstatus(L, co); + if (status != CO_DEAD && status != CO_SUS) + luaL_error(L, "cannot close %s coroutine", statnames[status]); + + if (co->status == LUA_OK || co->status == LUA_YIELD) + { + lua_pushboolean(L, true); + lua_resetthread(co); + return 1; + } + else + { + lua_pushboolean(L, false); + if (lua_gettop(co)) + lua_xmove(co, L, 1); /* move error message */ + lua_resetthread(co); + return 2; + } +} + static const luaL_Reg co_funcs[] = { {"create", cocreate}, {"running", corunning}, @@ -239,6 +268,7 @@ static const luaL_Reg co_funcs[] = { {"wrap", cowrap}, {"yield", coyield}, {"isyieldable", coyieldable}, + {"close", coclose}, {NULL, NULL}, }; @@ -246,7 +276,7 @@ LUALIB_API int luaopen_coroutine(lua_State* L) { luaL_register(L, LUA_COLIBNAME, co_funcs); - lua_pushcfunction(L, coresumey, "resume", 0, coresumecont); + lua_pushcclosurek(L, coresumey, "resume", 0, coresumecont); lua_setfield(L, -2, "resume"); return 1; diff --git a/VM/src/ldebug.cpp b/VM/src/ldebug.cpp index 1890e682..d77f84ef 100644 --- a/VM/src/ldebug.cpp +++ b/VM/src/ldebug.cpp @@ -316,7 +316,7 @@ void luaG_breakpoint(lua_State* L, Proto* p, int line, bool enable) p->debuginsn[j] = LUAU_INSN_OP(p->code[j]); } - uint8_t op = enable ? LOP_BREAK : LUAU_INSN_OP(p->code[i]); + uint8_t op = enable ? LOP_BREAK : LUAU_INSN_OP(p->debuginsn[i]); // patch just the opcode byte, leave arguments alone p->code[i] &= ~0xff; @@ -357,17 +357,17 @@ int luaG_getline(Proto* p, int pc) return p->abslineinfo[pc >> p->linegaplog2] + p->lineinfo[pc]; } -void lua_singlestep(lua_State* L, bool singlestep) +void lua_singlestep(lua_State* L, int enabled) { - L->singlestep = singlestep; + L->singlestep = bool(enabled); } -void lua_breakpoint(lua_State* L, int funcindex, int line, bool enable) +void lua_breakpoint(lua_State* L, int funcindex, int line, int enabled) { const TValue* func = luaA_toobject(L, funcindex); api_check(L, ttisfunction(func) && !clvalue(func)->isC); - luaG_breakpoint(L, clvalue(func)->l.p, line, enable); + luaG_breakpoint(L, clvalue(func)->l.p, line, bool(enabled)); } static size_t append(char* buf, size_t bufsize, size_t offset, const char* data) diff --git a/VM/src/ldo.cpp b/VM/src/ldo.cpp index 328b47e6..1259d461 100644 --- a/VM/src/ldo.cpp +++ b/VM/src/ldo.cpp @@ -19,6 +19,7 @@ LUAU_FASTFLAGVARIABLE(LuauExceptionMessageFix, false) LUAU_FASTFLAGVARIABLE(LuauCcallRestoreFix, false) +LUAU_FASTFLAG(LuauCoroutineClose) /* ** {====================================================== @@ -300,7 +301,10 @@ static void resume(lua_State* L, void* ud) if (L->status == 0) { // start coroutine - LUAU_ASSERT(L->ci == L->base_ci && firstArg > L->base); + LUAU_ASSERT(L->ci == L->base_ci && firstArg >= L->base); + if (FFlag::LuauCoroutineClose && firstArg == L->base) + luaG_runerror(L, "cannot resume dead coroutine"); + if (luau_precall(L, firstArg - 1, LUA_MULTRET) != PCRLUA) return; diff --git a/VM/src/linit.cpp b/VM/src/linit.cpp index bf5e738f..4e40165a 100644 --- a/VM/src/linit.cpp +++ b/VM/src/linit.cpp @@ -22,7 +22,7 @@ LUALIB_API void luaL_openlibs(lua_State* L) const luaL_Reg* lib = lualibs; for (; lib->func; lib++) { - lua_pushcfunction(L, lib->func); + lua_pushcfunction(L, lib->func, NULL); lua_pushstring(L, lib->name); lua_call(L, 1, 0); } diff --git a/VM/src/lstate.cpp b/VM/src/lstate.cpp index 0b2dfb69..24e97063 100644 --- a/VM/src/lstate.cpp +++ b/VM/src/lstate.cpp @@ -124,6 +124,34 @@ void luaE_freethread(lua_State* L, lua_State* L1) luaM_free(L, L1, sizeof(lua_State), L1->memcat); } +void lua_resetthread(lua_State* L) +{ + /* close upvalues before clearing anything */ + luaF_close(L, L->stack); + /* clear call frames */ + CallInfo* ci = L->base_ci; + ci->func = L->stack; + ci->base = ci->func + 1; + ci->top = ci->base + LUA_MINSTACK; + setnilvalue(ci->func); + L->ci = ci; + luaD_reallocCI(L, BASIC_CI_SIZE); + /* clear thread state */ + L->status = LUA_OK; + L->base = L->ci->base; + L->top = L->ci->base; + L->nCcalls = L->baseCcalls = 0; + /* clear thread stack */ + luaD_reallocstack(L, BASIC_STACK_SIZE); + for (int i = 0; i < L->stacksize; i++) + setnilvalue(L->stack + i); +} + +int lua_isthreadreset(lua_State* L) +{ + return L->ci == L->base_ci && L->base == L->top && L->status == LUA_OK; +} + lua_State* lua_newstate(lua_Alloc f, void* ud) { int i; diff --git a/VM/src/lstrlib.cpp b/VM/src/lstrlib.cpp index 80a34483..b576f809 100644 --- a/VM/src/lstrlib.cpp +++ b/VM/src/lstrlib.cpp @@ -748,7 +748,7 @@ static int gmatch(lua_State* L) luaL_checkstring(L, 2); lua_settop(L, 2); lua_pushinteger(L, 0); - lua_pushcfunction(L, gmatch_aux, NULL, 3); + lua_pushcclosure(L, gmatch_aux, NULL, 3); return 1; } diff --git a/VM/src/lutf8lib.cpp b/VM/src/lutf8lib.cpp index 6a026296..378de3d0 100644 --- a/VM/src/lutf8lib.cpp +++ b/VM/src/lutf8lib.cpp @@ -265,7 +265,7 @@ static int iter_aux(lua_State* L) static int iter_codes(lua_State* L) { luaL_checkstring(L, 1); - lua_pushcfunction(L, iter_aux); + lua_pushcfunction(L, iter_aux, NULL); lua_pushvalue(L, 1); lua_pushinteger(L, 0); return 3; diff --git a/bench/bench.py b/bench/bench.py index b23ca891..39f219f3 100644 --- a/bench/bench.py +++ b/bench/bench.py @@ -25,8 +25,8 @@ try: import scipy from scipy import stats except ModuleNotFoundError: - print("scipy package is required") - exit(1) + print("Warning: scipy package is not installed, confidence values will not be available") + stats = None scriptdir = os.path.dirname(os.path.realpath(__file__)) defaultVm = 'luau.exe' if os.name == "nt" else './luau' @@ -200,11 +200,14 @@ def finalizeResult(result): result.sampleStdDev = math.sqrt(sumOfSquares / (result.count - 1)) result.unbiasedEst = result.sampleStdDev * result.sampleStdDev - # Two-tailed distribution with 95% conf. - tValue = stats.t.ppf(1 - 0.05 / 2, result.count - 1) + if stats: + # Two-tailed distribution with 95% conf. + tValue = stats.t.ppf(1 - 0.05 / 2, result.count - 1) - # Compute confidence interval - result.sampleConfidenceInterval = tValue * result.sampleStdDev / math.sqrt(result.count) + # Compute confidence interval + result.sampleConfidenceInterval = tValue * result.sampleStdDev / math.sqrt(result.count) + else: + result.sampleConfidenceInterval = result.sampleStdDev else: result.sampleStdDev = 0 result.unbiasedEst = 0 @@ -377,14 +380,19 @@ def analyzeResult(subdir, main, comparisons): tStat = abs(main.avg - compare.avg) / (pooledStdDev * math.sqrt(2 / main.count)) degreesOfFreedom = 2 * main.count - 2 - # Two-tailed distribution with 95% conf. - tCritical = stats.t.ppf(1 - 0.05 / 2, degreesOfFreedom) + if stats: + # Two-tailed distribution with 95% conf. + tCritical = stats.t.ppf(1 - 0.05 / 2, degreesOfFreedom) - noSignificantDifference = tStat < tCritical + noSignificantDifference = tStat < tCritical + pValue = 2 * (1 - stats.t.cdf(tStat, df = degreesOfFreedom)) + else: + noSignificantDifference = None + pValue = -1 - pValue = 2 * (1 - stats.t.cdf(tStat, df = degreesOfFreedom)) - - if noSignificantDifference: + if noSignificantDifference is None: + verdict = "" + elif noSignificantDifference: verdict = "likely same" elif main.avg < compare.avg: verdict = "likely worse" diff --git a/fuzz/proto.cpp b/fuzz/proto.cpp index c85fac7d..ae2399e4 100644 --- a/fuzz/proto.cpp +++ b/fuzz/proto.cpp @@ -257,7 +257,7 @@ DEFINE_PROTO_FUZZER(const luau::StatBlock& message) lua_State* L = lua_newthread(globalState); luaL_sandboxthread(L); - if (luau_load(L, "=fuzz", bytecode.data(), bytecode.size()) == 0) + if (luau_load(L, "=fuzz", bytecode.data(), bytecode.size(), 0) == 0) { interruptDeadline = std::chrono::system_clock::now() + kInterruptTimeout; diff --git a/tests/Autocomplete.test.cpp b/tests/Autocomplete.test.cpp index 8a7798f3..5a7c8602 100644 --- a/tests/Autocomplete.test.cpp +++ b/tests/Autocomplete.test.cpp @@ -91,10 +91,6 @@ struct ACFixture : ACFixtureImpl { }; -struct UnfrozenACFixture : ACFixtureImpl -{ -}; - TEST_SUITE_BEGIN("AutocompleteTest"); TEST_CASE_FIXTURE(ACFixture, "empty_program") @@ -1919,9 +1915,10 @@ local bar: @1= foo CHECK(!ac.entryMap.count("foo")); } -// CLI-45692: Remove UnfrozenACFixture here -TEST_CASE_FIXTURE(UnfrozenACFixture, "type_correct_function_no_parenthesis") +TEST_CASE_FIXTURE(ACFixture, "type_correct_function_no_parenthesis") { + ScopedFastFlag luauAutocompleteAvoidMutation("LuauAutocompleteAvoidMutation", true); + check(R"( local function target(a: (number) -> number) return a(4) end local function bar1(a: number) return -a end @@ -1950,9 +1947,10 @@ local fp: @1= f CHECK(ac.entryMap.count("({ x: number, y: number }) -> number")); } -// CLI-45692: Remove UnfrozenACFixture here -TEST_CASE_FIXTURE(UnfrozenACFixture, "type_correct_keywords") +TEST_CASE_FIXTURE(ACFixture, "type_correct_keywords") { + ScopedFastFlag luauAutocompleteAvoidMutation("LuauAutocompleteAvoidMutation", true); + check(R"( local function a(x: boolean) end local function b(x: number?) end @@ -2484,7 +2482,7 @@ local t = { CHECK(ac.entryMap.count("second")); } -TEST_CASE_FIXTURE(Fixture, "autocomplete_documentation_symbols") +TEST_CASE_FIXTURE(UnfrozenFixture, "autocomplete_documentation_symbols") { loadDefinition(R"( declare y: { diff --git a/tests/Compiler.test.cpp b/tests/Compiler.test.cpp index 7f03019c..4ce8d08a 100644 --- a/tests/Compiler.test.cpp +++ b/tests/Compiler.test.cpp @@ -11,8 +11,6 @@ #include LUAU_FASTFLAG(LuauPreloadClosures) -LUAU_FASTFLAG(LuauPreloadClosuresFenv) -LUAU_FASTFLAG(LuauPreloadClosuresUpval) LUAU_FASTFLAG(LuauGenericSpecialGlobals) using namespace Luau; @@ -2797,7 +2795,7 @@ CAPTURE UPVAL U1 RETURN R0 1 )"); - if (FFlag::LuauPreloadClosuresUpval) + if (FFlag::LuauPreloadClosures) { // recursive capture CHECK_EQ("\n" + compileFunction("local function foo() return foo() end", 1), R"( @@ -3479,15 +3477,13 @@ CAPTURE VAL R0 RETURN R1 1 )"); - if (FFlag::LuauPreloadClosuresFenv) - { - // if they don't need upvalues but we sense that environment may be modified, we disable this to avoid fenv-related identity confusion - CHECK_EQ("\n" + compileFunction(R"( + // if they don't need upvalues but we sense that environment may be modified, we disable this to avoid fenv-related identity confusion + CHECK_EQ("\n" + compileFunction(R"( setfenv(1, {}) return function() print("hi") end )", - 1), - R"( + 1), + R"( GETIMPORT R0 1 LOADN R1 1 NEWTABLE R2 0 0 @@ -3496,23 +3492,21 @@ NEWCLOSURE R0 P0 RETURN R0 1 )"); - // note that fenv analysis isn't flow-sensitive right now, which is sort of a feature - CHECK_EQ("\n" + compileFunction(R"( + // note that fenv analysis isn't flow-sensitive right now, which is sort of a feature + CHECK_EQ("\n" + compileFunction(R"( if false then setfenv(1, {}) end return function() print("hi") end )", - 1), - R"( + 1), + R"( NEWCLOSURE R0 P0 RETURN R0 1 )"); - } } TEST_CASE("SharedClosure") { ScopedFastFlag sff1("LuauPreloadClosures", true); - ScopedFastFlag sff2("LuauPreloadClosuresUpval", true); // closures can be shared even if functions refer to upvalues, as long as upvalues are top-level CHECK_EQ("\n" + compileFunction(R"( @@ -3671,7 +3665,7 @@ RETURN R0 0 )"); } -TEST_CASE("LuauGenericSpecialGlobals") +TEST_CASE("MutableGlobals") { const char* source = R"( print() @@ -3685,43 +3679,6 @@ shared.print() workspace.print() )"; - { - ScopedFastFlag genericSpecialGlobals{"LuauGenericSpecialGlobals", false}; - - // Check Roblox globals are here - CHECK_EQ("\n" + compileFunction0(source), R"( -GETIMPORT R0 1 -CALL R0 0 0 -GETIMPORT R1 3 -GETTABLEKS R0 R1 K0 -CALL R0 0 0 -GETIMPORT R1 5 -GETTABLEKS R0 R1 K0 -CALL R0 0 0 -GETIMPORT R1 7 -GETTABLEKS R0 R1 K0 -CALL R0 0 0 -GETIMPORT R1 9 -GETTABLEKS R0 R1 K0 -CALL R0 0 0 -GETIMPORT R1 11 -GETTABLEKS R0 R1 K0 -CALL R0 0 0 -GETIMPORT R1 13 -GETTABLEKS R0 R1 K0 -CALL R0 0 0 -GETIMPORT R1 15 -GETTABLEKS R0 R1 K0 -CALL R0 0 0 -GETIMPORT R1 17 -GETTABLEKS R0 R1 K0 -CALL R0 0 0 -RETURN R0 0 -)"); - } - - ScopedFastFlag genericSpecialGlobals{"LuauGenericSpecialGlobals", true}; - // Check Roblox globals are no longer here CHECK_EQ("\n" + compileFunction0(source), R"( GETIMPORT R0 1 diff --git a/tests/Conformance.test.cpp b/tests/Conformance.test.cpp index c1b790b9..e495a213 100644 --- a/tests/Conformance.test.cpp +++ b/tests/Conformance.test.cpp @@ -1,5 +1,7 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details -#include "Luau/Compiler.h" +#include "lua.h" +#include "lualib.h" +#include "luacode.h" #include "Luau/BuiltinDefinitions.h" #include "Luau/ModuleResolver.h" @@ -10,9 +12,6 @@ #include "doctest.h" #include "ScopedFlags.h" -#include "lua.h" -#include "lualib.h" - #include #include @@ -49,8 +48,12 @@ static int lua_loadstring(lua_State* L) lua_setsafeenv(L, LUA_ENVIRONINDEX, false); - std::string bytecode = Luau::compile(std::string(s, l)); - if (luau_load(L, chunkname, bytecode.data(), bytecode.size()) == 0) + size_t bytecodeSize = 0; + char* bytecode = luau_compile(s, l, nullptr, &bytecodeSize); + int result = luau_load(L, chunkname, bytecode, bytecodeSize, 0); + free(bytecode); + + if (result == 0) return 1; lua_pushnil(L); @@ -179,21 +182,17 @@ static StateRef runConformance( std::string chunkname = "=" + std::string(name); - Luau::CompileOptions copts; + lua_CompileOptions copts = {}; + copts.optimizationLevel = 1; // default copts.debugLevel = 2; // for debugger tests copts.vectorCtor = "vector"; // for vector tests - std::string bytecode = Luau::compile(source, copts); - int status = 0; + size_t bytecodeSize = 0; + char* bytecode = luau_compile(source.data(), source.size(), &copts, &bytecodeSize); + int result = luau_load(L, chunkname.c_str(), bytecode, bytecodeSize, 0); + free(bytecode); - if (luau_load(L, chunkname.c_str(), bytecode.data(), bytecode.size()) == 0) - { - status = lua_resume(L, nullptr, 0); - } - else - { - status = LUA_ERRSYNTAX; - } + int status = (result == 0) ? lua_resume(L, nullptr, 0) : LUA_ERRSYNTAX; while (yield && (status == LUA_YIELD || status == LUA_BREAK)) { @@ -332,53 +331,61 @@ TEST_CASE("UTF8") TEST_CASE("Coroutine") { + ScopedFastFlag sff("LuauCoroutineClose", true); + runConformance("coroutine.lua"); } +static int cxxthrow(lua_State* L) +{ +#if LUA_USE_LONGJMP + luaL_error(L, "oops"); +#else + throw std::runtime_error("oops"); +#endif +} + TEST_CASE("PCall") { runConformance("pcall.lua", [](lua_State* L) { - lua_pushcfunction(L, [](lua_State* L) -> int { -#if LUA_USE_LONGJMP - luaL_error(L, "oops"); -#else - throw std::runtime_error("oops"); -#endif - }); + lua_pushcfunction(L, cxxthrow, "cxxthrow"); lua_setglobal(L, "cxxthrow"); - lua_pushcfunction(L, [](lua_State* L) -> int { - lua_State* co = lua_tothread(L, 1); - lua_xmove(L, co, 1); - lua_resumeerror(co, L); - return 0; - }); + lua_pushcfunction( + L, + [](lua_State* L) -> int { + lua_State* co = lua_tothread(L, 1); + lua_xmove(L, co, 1); + lua_resumeerror(co, L); + return 0; + }, + "resumeerror"); lua_setglobal(L, "resumeerror"); }); } TEST_CASE("Pack") { - ScopedFastFlag sff{ "LuauStrPackUBCastFix", true }; - + ScopedFastFlag sff{"LuauStrPackUBCastFix", true}; + runConformance("tpack.lua"); } TEST_CASE("Vector") { runConformance("vector.lua", [](lua_State* L) { - lua_pushcfunction(L, lua_vector); + lua_pushcfunction(L, lua_vector, "vector"); lua_setglobal(L, "vector"); lua_pushvector(L, 0.0f, 0.0f, 0.0f); luaL_newmetatable(L, "vector"); lua_pushstring(L, "__index"); - lua_pushcfunction(L, lua_vector_index); + lua_pushcfunction(L, lua_vector_index, nullptr); lua_settable(L, -3); lua_pushstring(L, "__namecall"); - lua_pushcfunction(L, lua_vector_namecall); + lua_pushcfunction(L, lua_vector_namecall, nullptr); lua_settable(L, -3); lua_setreadonly(L, -1, true); @@ -513,15 +520,19 @@ TEST_CASE("Debugger") }; // add breakpoint() function - lua_pushcfunction(L, [](lua_State* L) -> int { - int line = luaL_checkinteger(L, 1); + lua_pushcfunction( + L, + [](lua_State* L) -> int { + int line = luaL_checkinteger(L, 1); + bool enabled = lua_isboolean(L, 2) ? lua_toboolean(L, 2) : true; - lua_Debug ar = {}; - lua_getinfo(L, 1, "f", &ar); + lua_Debug ar = {}; + lua_getinfo(L, 1, "f", &ar); - lua_breakpoint(L, -1, line, true); - return 0; - }); + lua_breakpoint(L, -1, line, enabled); + return 0; + }, + "breakpoint"); lua_setglobal(L, "breakpoint"); }, [](lua_State* L) { @@ -744,7 +755,7 @@ TEST_CASE("ExceptionObject") if (nsize == 0) { free(ptr); - return NULL; + return nullptr; } else if (nsize > 512 * 1024) { diff --git a/tests/IostreamOptional.h b/tests/IostreamOptional.h index e55b5b0c..e0756bad 100644 --- a/tests/IostreamOptional.h +++ b/tests/IostreamOptional.h @@ -4,7 +4,8 @@ #include #include -namespace std { +namespace std +{ inline std::ostream& operator<<(std::ostream& lhs, const std::nullopt_t&) { diff --git a/tests/Module.test.cpp b/tests/Module.test.cpp index 18f55d2c..7a3543c7 100644 --- a/tests/Module.test.cpp +++ b/tests/Module.test.cpp @@ -203,6 +203,8 @@ TEST_CASE_FIXTURE(Fixture, "clone_class") TEST_CASE_FIXTURE(Fixture, "clone_sanitize_free_types") { + ScopedFastFlag sff{"LuauErrorRecoveryType", true}; + TypeVar freeTy(FreeTypeVar{TypeLevel{}}); TypePackVar freeTp(FreeTypePack{TypeLevel{}}); @@ -212,12 +214,12 @@ TEST_CASE_FIXTURE(Fixture, "clone_sanitize_free_types") bool encounteredFreeType = false; TypeId clonedTy = clone(&freeTy, dest, seenTypes, seenTypePacks, &encounteredFreeType); - CHECK(Luau::get(clonedTy)); + CHECK_EQ("any", toString(clonedTy)); CHECK(encounteredFreeType); encounteredFreeType = false; TypePackId clonedTp = clone(&freeTp, dest, seenTypes, seenTypePacks, &encounteredFreeType); - CHECK(Luau::get(clonedTp)); + CHECK_EQ("...any", toString(clonedTp)); CHECK(encounteredFreeType); } diff --git a/tests/ToString.test.cpp b/tests/ToString.test.cpp index b076e9ad..80a258f5 100644 --- a/tests/ToString.test.cpp +++ b/tests/ToString.test.cpp @@ -198,7 +198,8 @@ TEST_CASE_FIXTURE(Fixture, "stringifying_table_type_correctly_use_matching_table TypeVar tv{ttv}; - ToStringOptions o{/* exhaustive= */ false, /* useLineBreaks= */ false, /* functionTypeArguments= */ false, /* hideTableKind= */ false, 40}; + ToStringOptions o; + o.maxTableLength = 40; CHECK_EQ(toString(&tv, o), "{| a: number, b: number, c: number, d: number, e: number, ... 5 more ... |}"); } @@ -395,7 +396,7 @@ local function target(callback: nil) return callback(4, "hello") end )"); LUAU_REQUIRE_ERRORS(result); - CHECK_EQ(toString(requireType("target")), "(nil) -> (*unknown*)"); + CHECK_EQ("(nil) -> (*unknown*)", toString(requireType("target"))); } TEST_CASE_FIXTURE(Fixture, "toStringGenericPack") @@ -469,4 +470,110 @@ TEST_CASE_FIXTURE(Fixture, "self_recursive_instantiated_param") CHECK_EQ(toString(tableTy), "Table
"); } +TEST_CASE_FIXTURE(Fixture, "toStringNamedFunction_id") +{ + CheckResult result = check(R"( + local function id(x) return x end + )"); + + TypeId ty = requireType("id"); + const FunctionTypeVar* ftv = get(follow(ty)); + + CHECK_EQ("id(x: a): a", toStringNamedFunction("id", *ftv)); +} + +TEST_CASE_FIXTURE(Fixture, "toStringNamedFunction_map") +{ + CheckResult result = check(R"( + local function map(arr, fn) + local t = {} + for i = 0, #arr do + t[i] = fn(arr[i]) + end + return t + end + )"); + + TypeId ty = requireType("map"); + const FunctionTypeVar* ftv = get(follow(ty)); + + CHECK_EQ("map(arr: {a}, fn: (a) -> b): {b}", toStringNamedFunction("map", *ftv)); +} + +TEST_CASE("toStringNamedFunction_unit_f") +{ + TypePackVar empty{TypePack{}}; + FunctionTypeVar ftv{&empty, &empty, {}, false}; + CHECK_EQ("f(): ()", toStringNamedFunction("f", ftv)); +} + +TEST_CASE_FIXTURE(Fixture, "toStringNamedFunction_variadics") +{ + CheckResult result = check(R"( + local function f(x: a, ...): (a, a, b...) + return x, x, ... + end + )"); + + TypeId ty = requireType("f"); + auto ftv = get(follow(ty)); + + CHECK_EQ("f(x: a, ...: any): (a, a, b...)", toStringNamedFunction("f", *ftv)); +} + +TEST_CASE_FIXTURE(Fixture, "toStringNamedFunction_variadics2") +{ + CheckResult result = check(R"( + local function f(): ...number + return 1, 2, 3 + end + )"); + + TypeId ty = requireType("f"); + auto ftv = get(follow(ty)); + + CHECK_EQ("f(): ...number", toStringNamedFunction("f", *ftv)); +} + +TEST_CASE_FIXTURE(Fixture, "toStringNamedFunction_variadics3") +{ + CheckResult result = check(R"( + local function f(): (string, ...number) + return 'a', 1, 2, 3 + end + )"); + + TypeId ty = requireType("f"); + auto ftv = get(follow(ty)); + + CHECK_EQ("f(): (string, ...number)", toStringNamedFunction("f", *ftv)); +} + +TEST_CASE_FIXTURE(Fixture, "toStringNamedFunction_type_annotation_has_partial_argnames") +{ + CheckResult result = check(R"( + local f: (number, y: number) -> number + )"); + + TypeId ty = requireType("f"); + auto ftv = get(follow(ty)); + + CHECK_EQ("f(_: number, y: number): number", toStringNamedFunction("f", *ftv)); +} + +TEST_CASE_FIXTURE(Fixture, "toStringNamedFunction_hide_type_params") +{ + CheckResult result = check(R"( + local function f(x: T, g: (T) -> U)): () + end + )"); + + TypeId ty = requireType("f"); + auto ftv = get(follow(ty)); + + ToStringOptions opts; + opts.hideNamedFunctionTypeParameters = true; + CHECK_EQ("f(x: T, g: (T) -> U): ()", toStringNamedFunction("f", *ftv, opts)); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.aliases.test.cpp b/tests/TypeInfer.aliases.test.cpp index c27f8083..74ce155c 100644 --- a/tests/TypeInfer.aliases.test.cpp +++ b/tests/TypeInfer.aliases.test.cpp @@ -109,7 +109,7 @@ TEST_CASE_FIXTURE(Fixture, "dont_stop_typechecking_after_reporting_duplicate_typ CheckResult result = check(R"( type A = number type A = string -- Redefinition of type 'A', previously defined at line 1 - local foo: string = 1 -- No "Type 'number' could not be converted into 'string'" + local foo: string = 1 -- "Type 'number' could not be converted into 'string'" )"); LUAU_REQUIRE_ERROR_COUNT(2, result); diff --git a/tests/TypeInfer.annotations.test.cpp b/tests/TypeInfer.annotations.test.cpp index 2e400164..091c2f01 100644 --- a/tests/TypeInfer.annotations.test.cpp +++ b/tests/TypeInfer.annotations.test.cpp @@ -381,6 +381,8 @@ TEST_CASE_FIXTURE(Fixture, "typeof_expr") TEST_CASE_FIXTURE(Fixture, "corecursive_types_error_on_tight_loop") { + ScopedFastFlag sff{"LuauErrorRecoveryType", true}; + CheckResult result = check(R"( type A = B type B = A @@ -390,7 +392,7 @@ TEST_CASE_FIXTURE(Fixture, "corecursive_types_error_on_tight_loop") )"); TypeId fType = requireType("aa"); - const ErrorTypeVar* ftv = get(follow(fType)); + const AnyTypeVar* ftv = get(follow(fType)); REQUIRE(ftv != nullptr); REQUIRE(!result.errors.empty()); } diff --git a/tests/TypeInfer.generics.test.cpp b/tests/TypeInfer.generics.test.cpp index de2f0154..88c2dc85 100644 --- a/tests/TypeInfer.generics.test.cpp +++ b/tests/TypeInfer.generics.test.cpp @@ -289,7 +289,7 @@ TEST_CASE_FIXTURE(Fixture, "calling_self_generic_methods") end )"); // TODO: Should typecheck but currently errors CLI-39916 - LUAU_REQUIRE_ERROR_COUNT(1, result); + LUAU_REQUIRE_ERRORS(result); } TEST_CASE_FIXTURE(Fixture, "infer_generic_property") @@ -352,7 +352,7 @@ TEST_CASE_FIXTURE(Fixture, "dont_leak_generic_types") -- so this assignment should fail local b: boolean = f(true) )"); - LUAU_REQUIRE_ERROR_COUNT(2, result); + LUAU_REQUIRE_ERRORS(result); } TEST_CASE_FIXTURE(Fixture, "dont_leak_inferred_generic_types") @@ -368,7 +368,7 @@ TEST_CASE_FIXTURE(Fixture, "dont_leak_inferred_generic_types") local y: number = id(37) end )"); - LUAU_REQUIRE_ERROR_COUNT(1, result); + LUAU_REQUIRE_ERRORS(result); } TEST_CASE_FIXTURE(Fixture, "dont_substitute_bound_types") diff --git a/tests/TypeInfer.refinements.test.cpp b/tests/TypeInfer.refinements.test.cpp index 733fc39b..fe8e7ff9 100644 --- a/tests/TypeInfer.refinements.test.cpp +++ b/tests/TypeInfer.refinements.test.cpp @@ -704,9 +704,10 @@ TEST_CASE_FIXTURE(RefinementClassFixture, "typeguard_cast_free_table_to_vector") CHECK_EQ("Type '{+ X: a, Y: b, Z: c +}' could not be converted into 'Instance'", toString(result.errors[0])); else CHECK_EQ("Type '{- X: a, Y: b, Z: c -}' could not be converted into 'Instance'", toString(result.errors[0])); + CHECK_EQ("*unknown*", toString(requireTypeAtPosition({7, 28}))); // typeof(vec) == "Instance" - if (FFlag::LuauQuantifyInPlace2) + if (FFlag::LuauQuantifyInPlace2) CHECK_EQ("{+ X: a, Y: b, Z: c +}", toString(requireTypeAtPosition({9, 28}))); // type(vec) ~= "vector" and typeof(vec) ~= "Instance" else CHECK_EQ("{- X: a, Y: b, Z: c -}", toString(requireTypeAtPosition({9, 28}))); // type(vec) ~= "vector" and typeof(vec) ~= "Instance" diff --git a/tests/TypeInfer.singletons.test.cpp b/tests/TypeInfer.singletons.test.cpp new file mode 100644 index 00000000..5f95efd5 --- /dev/null +++ b/tests/TypeInfer.singletons.test.cpp @@ -0,0 +1,377 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details + +#include "Fixture.h" + +#include "doctest.h" +#include "Luau/BuiltinDefinitions.h" + +using namespace Luau; + +TEST_SUITE_BEGIN("TypeSingletons"); + +TEST_CASE_FIXTURE(Fixture, "bool_singletons") +{ + ScopedFastFlag sffs[] = { + {"LuauSingletonTypes", true}, + {"LuauParseSingletonTypes", true}, + }; + + CheckResult result = check(R"( + local a: true = true + local b: false = false + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "string_singletons") +{ + ScopedFastFlag sffs[] = { + {"LuauSingletonTypes", true}, + {"LuauParseSingletonTypes", true}, + }; + + CheckResult result = check(R"( + local a: "foo" = "foo" + local b: "bar" = "bar" + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "bool_singletons_mismatch") +{ + ScopedFastFlag sffs[] = { + {"LuauSingletonTypes", true}, + {"LuauParseSingletonTypes", true}, + }; + + CheckResult result = check(R"( + local a: true = false + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ("Type 'false' could not be converted into 'true'", toString(result.errors[0])); +} + +TEST_CASE_FIXTURE(Fixture, "string_singletons_mismatch") +{ + ScopedFastFlag sffs[] = { + {"LuauSingletonTypes", true}, + {"LuauParseSingletonTypes", true}, + }; + + CheckResult result = check(R"( + local a: "foo" = "bar" + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ("Type '\"bar\"' could not be converted into '\"foo\"'", toString(result.errors[0])); +} + +TEST_CASE_FIXTURE(Fixture, "string_singletons_escape_chars") +{ + ScopedFastFlag sffs[] = { + {"LuauSingletonTypes", true}, + {"LuauParseSingletonTypes", true}, + }; + + CheckResult result = check(R"( + local a: "\n" = "\000\r" + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ(R"(Type '"\000\r"' could not be converted into '"\n"')", toString(result.errors[0])); +} + +TEST_CASE_FIXTURE(Fixture, "bool_singleton_subtype") +{ + ScopedFastFlag sffs[] = { + {"LuauSingletonTypes", true}, + {"LuauParseSingletonTypes", true}, + }; + + CheckResult result = check(R"( + local a: true = true + local b: boolean = a + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "string_singleton_subtype") +{ + ScopedFastFlag sffs[] = { + {"LuauSingletonTypes", true}, + {"LuauParseSingletonTypes", true}, + }; + + CheckResult result = check(R"( + local a: "foo" = "foo" + local b: string = a + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "function_call_with_singletons") +{ + ScopedFastFlag sffs[] = { + {"LuauSingletonTypes", true}, + {"LuauParseSingletonTypes", true}, + }; + + CheckResult result = check(R"( + function f(a: true, b: "foo") end + f(true, "foo") + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "function_call_with_singletons_mismatch") +{ + ScopedFastFlag sffs[] = { + {"LuauSingletonTypes", true}, + {"LuauParseSingletonTypes", true}, + }; + + CheckResult result = check(R"( + function f(a: true, b: "foo") end + f(true, "bar") + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ("Type '\"bar\"' could not be converted into '\"foo\"'", toString(result.errors[0])); +} + +TEST_CASE_FIXTURE(Fixture, "overloaded_function_call_with_singletons") +{ + ScopedFastFlag sffs[] = { + {"LuauSingletonTypes", true}, + {"LuauParseSingletonTypes", true}, + }; + + CheckResult result = check(R"( + function f(a, b) end + local g : ((true, string) -> ()) & ((false, number) -> ()) = (f::any) + g(true, "foo") + g(false, 37) + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "overloaded_function_call_with_singletons_mismatch") +{ + ScopedFastFlag sffs[] = { + {"LuauSingletonTypes", true}, + {"LuauParseSingletonTypes", true}, + }; + + CheckResult result = check(R"( + function f(a, b) end + local g : ((true, string) -> ()) & ((false, number) -> ()) = (f::any) + g(true, 37) + )"); + + LUAU_REQUIRE_ERROR_COUNT(2, result); + CHECK_EQ("Type 'number' could not be converted into 'string'", toString(result.errors[0])); + CHECK_EQ("Other overloads are also not viable: (false, number) -> ()", toString(result.errors[1])); +} + +TEST_CASE_FIXTURE(Fixture, "enums_using_singletons") +{ + ScopedFastFlag sffs[] = { + {"LuauSingletonTypes", true}, + {"LuauParseSingletonTypes", true}, + }; + + CheckResult result = check(R"( + type MyEnum = "foo" | "bar" | "baz" + local a : MyEnum = "foo" + local b : MyEnum = "bar" + local c : MyEnum = "baz" + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "enums_using_singletons_mismatch") +{ + ScopedFastFlag sffs[] = { + {"LuauSingletonTypes", true}, + {"LuauParseSingletonTypes", true}, + {"LuauExtendedTypeMismatchError", true}, + }; + + CheckResult result = check(R"( + type MyEnum = "foo" | "bar" | "baz" + local a : MyEnum = "bang" + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ("Type '\"bang\"' could not be converted into '\"bar\" | \"baz\" | \"foo\"'; none of the union options are compatible", + toString(result.errors[0])); +} + +TEST_CASE_FIXTURE(Fixture, "enums_using_singletons_subtyping") +{ + ScopedFastFlag sffs[] = { + {"LuauSingletonTypes", true}, + {"LuauParseSingletonTypes", true}, + }; + + CheckResult result = check(R"( + type MyEnum1 = "foo" | "bar" + type MyEnum2 = MyEnum1 | "baz" + local a : MyEnum1 = "foo" + local b : MyEnum2 = a + local c : string = b + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "tagged_unions_using_singletons") +{ + ScopedFastFlag sffs[] = { + {"LuauSingletonTypes", true}, + {"LuauParseSingletonTypes", true}, + {"LuauExpectedTypesOfProperties", true}, + }; + + CheckResult result = check(R"( + type Dog = { tag: "Dog", howls: boolean } + type Cat = { tag: "Cat", meows: boolean } + type Animal = Dog | Cat + local a : Dog = { tag = "Dog", howls = true } + local b : Animal = { tag = "Cat", meows = true } + local c : Animal = a + c = b + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "tagged_unions_using_singletons_mismatch") +{ + ScopedFastFlag sffs[] = { + {"LuauSingletonTypes", true}, + {"LuauParseSingletonTypes", true}, + }; + + CheckResult result = check(R"( + type Dog = { tag: "Dog", howls: boolean } + type Cat = { tag: "Cat", meows: boolean } + type Animal = Dog | Cat + local a : Animal = { tag = "Cat", howls = true } + )"); + + LUAU_REQUIRE_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "tagged_unions_immutable_tag") +{ + ScopedFastFlag sffs[] = { + {"LuauSingletonTypes", true}, + {"LuauParseSingletonTypes", true}, + }; + + CheckResult result = check(R"( + type Dog = { tag: "Dog", howls: boolean } + type Cat = { tag: "Cat", meows: boolean } + type Animal = Dog | Cat + local a : Animal = { tag = "Cat", meows = true } + a.tag = "Dog" + )"); + + LUAU_REQUIRE_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "table_properties_singleton_strings") +{ + ScopedFastFlag sffs[] = { + {"LuauParseSingletonTypes", true}, + }; + + CheckResult result = check(R"( + --!strict + type T = { + ["foo"] : number, + ["$$bar"] : string, + baz : boolean + } + local t: T = { + ["foo"] = 37, + ["$$bar"] = "hi", + baz = true + } + local a: number = t.foo + local b: string = t["$$bar"] + local c: boolean = t.baz + t.foo = 5 + t["$$bar"] = "lo" + t.baz = false + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} +TEST_CASE_FIXTURE(Fixture, "table_properties_singleton_strings_mismatch") +{ + ScopedFastFlag sffs[] = { + {"LuauParseSingletonTypes", true}, + }; + + CheckResult result = check(R"( + --!strict + type T = { + ["$$bar"] : string, + } + local t: T = { + ["$$bar"] = "hi", + } + t["$$bar"] = 5 + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ("Type 'number' could not be converted into 'string'", toString(result.errors[0])); +} + +TEST_CASE_FIXTURE(Fixture, "table_properties_alias_or_parens_is_indexer") +{ + ScopedFastFlag sffs[] = { + {"LuauParseSingletonTypes", true}, + }; + + CheckResult result = check(R"( + --!strict + type S = "bar" + type T = { + [("foo")] : number, + [S] : string, + } + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ("Syntax error: Cannot have more than one table indexer", toString(result.errors[0])); +} + +TEST_CASE_FIXTURE(Fixture, "table_properties_type_error_escapes") +{ + ScopedFastFlag sffs[] = { + {"LuauParseSingletonTypes", true}, + }; + + CheckResult result = check(R"( + --!strict + local x: { ["<>"] : number } + x = { ["\n"] = 5 } + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ(R"(Table type '{| ["\n"]: number |}' not compatible with type '{| ["<>"]: number |}' because the former is missing field '<>')", + toString(result.errors[0])); +} + +TEST_SUITE_END(); diff --git a/tests/TypeInfer.test.cpp b/tests/TypeInfer.test.cpp index 30d9130a..99fd8339 100644 --- a/tests/TypeInfer.test.cpp +++ b/tests/TypeInfer.test.cpp @@ -362,7 +362,7 @@ TEST_CASE_FIXTURE(Fixture, "for_in_loop_on_error") CHECK_EQ(2, result.errors.size()); TypeId p = requireType("p"); - CHECK_EQ(*p, *typeChecker.errorType); + CHECK_EQ("*unknown*", toString(p)); } TEST_CASE_FIXTURE(Fixture, "for_in_loop_on_non_function") @@ -480,7 +480,7 @@ TEST_CASE_FIXTURE(Fixture, "for_in_loop_iterator_returns_any2") LUAU_REQUIRE_NO_ERRORS(result); - CHECK_EQ(typeChecker.anyType, requireType("a")); + CHECK_EQ("any", toString(requireType("a"))); } TEST_CASE_FIXTURE(Fixture, "for_in_loop_iterator_is_any") @@ -496,7 +496,7 @@ TEST_CASE_FIXTURE(Fixture, "for_in_loop_iterator_is_any") LUAU_REQUIRE_NO_ERRORS(result); - CHECK_EQ(typeChecker.anyType, requireType("a")); + CHECK_EQ("any", toString(requireType("a"))); } TEST_CASE_FIXTURE(Fixture, "for_in_loop_iterator_is_any2") @@ -512,7 +512,7 @@ TEST_CASE_FIXTURE(Fixture, "for_in_loop_iterator_is_any2") LUAU_REQUIRE_NO_ERRORS(result); - CHECK_EQ(typeChecker.anyType, requireType("a")); + CHECK_EQ("any", toString(requireType("a"))); } TEST_CASE_FIXTURE(Fixture, "for_in_loop_iterator_is_error") @@ -526,7 +526,7 @@ TEST_CASE_FIXTURE(Fixture, "for_in_loop_iterator_is_error") LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK_EQ(typeChecker.errorType, requireType("a")); + CHECK_EQ("*unknown*", toString(requireType("a"))); } TEST_CASE_FIXTURE(Fixture, "for_in_loop_iterator_is_error2") @@ -542,7 +542,7 @@ TEST_CASE_FIXTURE(Fixture, "for_in_loop_iterator_is_error2") LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK_EQ(typeChecker.errorType, requireType("a")); + CHECK_EQ("*unknown*", toString(requireType("a"))); } TEST_CASE_FIXTURE(Fixture, "for_in_loop_with_custom_iterator") @@ -673,7 +673,7 @@ TEST_CASE_FIXTURE(Fixture, "string_index") REQUIRE(nat); CHECK_EQ("string", toString(nat->ty)); - CHECK(get(requireType("t"))); + CHECK_EQ("*unknown*", toString(requireType("t"))); } TEST_CASE_FIXTURE(Fixture, "length_of_error_type_does_not_produce_an_error") @@ -1456,7 +1456,7 @@ TEST_CASE_FIXTURE(Fixture, "require_module_that_does_not_export") auto hootyType = requireType(bModule, "Hooty"); - CHECK_MESSAGE(get(follow(hootyType)) != nullptr, "Should be an error: " << toString(hootyType)); + CHECK_EQ("*unknown*", toString(hootyType)); } TEST_CASE_FIXTURE(Fixture, "warn_on_lowercase_parent_property") @@ -2032,7 +2032,7 @@ TEST_CASE_FIXTURE(Fixture, "higher_order_function_4") CHECK_EQ(*arg0->indexer->indexResultType, *arg1Args[1]); } -TEST_CASE_FIXTURE(Fixture, "error_types_propagate") +TEST_CASE_FIXTURE(Fixture, "type_errors_infer_types") { CheckResult result = check(R"( local err = (true).x @@ -2049,10 +2049,10 @@ TEST_CASE_FIXTURE(Fixture, "error_types_propagate") CHECK_EQ("boolean", toString(err->table)); CHECK_EQ("x", err->key); - CHECK(nullptr != get(requireType("c"))); - CHECK(nullptr != get(requireType("d"))); - CHECK(nullptr != get(requireType("e"))); - CHECK(nullptr != get(requireType("f"))); + CHECK_EQ("*unknown*", toString(requireType("c"))); + CHECK_EQ("*unknown*", toString(requireType("d"))); + CHECK_EQ("*unknown*", toString(requireType("e"))); + CHECK_EQ("*unknown*", toString(requireType("f"))); } TEST_CASE_FIXTURE(Fixture, "calling_error_type_yields_error") @@ -2068,7 +2068,7 @@ TEST_CASE_FIXTURE(Fixture, "calling_error_type_yields_error") CHECK_EQ("unknown", err->name); - CHECK(nullptr != get(requireType("a"))); + CHECK_EQ("*unknown*", toString(requireType("a"))); } TEST_CASE_FIXTURE(Fixture, "chain_calling_error_type_yields_error") @@ -2077,9 +2077,7 @@ TEST_CASE_FIXTURE(Fixture, "chain_calling_error_type_yields_error") local a = Utility.Create "Foo" {} )"); - TypeId aType = requireType("a"); - - REQUIRE_MESSAGE(nullptr != get(aType), "Not an error: " << toString(aType)); + CHECK_EQ("*unknown*", toString(requireType("a"))); } TEST_CASE_FIXTURE(Fixture, "primitive_arith_no_metatable") @@ -2146,6 +2144,8 @@ TEST_CASE_FIXTURE(Fixture, "some_primitive_binary_ops") TEST_CASE_FIXTURE(Fixture, "typecheck_overloaded_multiply_that_is_an_intersection") { + ScopedFastFlag sff{"LuauErrorRecoveryType", true}; + CheckResult result = check(R"( --!strict local Vec3 = {} @@ -2175,11 +2175,13 @@ TEST_CASE_FIXTURE(Fixture, "typecheck_overloaded_multiply_that_is_an_intersectio CHECK_EQ("Vec3", toString(requireType("b"))); CHECK_EQ("Vec3", toString(requireType("c"))); CHECK_EQ("Vec3", toString(requireType("d"))); - CHECK(get(requireType("e"))); + CHECK_EQ("Vec3", toString(requireType("e"))); } TEST_CASE_FIXTURE(Fixture, "typecheck_overloaded_multiply_that_is_an_intersection_on_rhs") { + ScopedFastFlag sff{"LuauErrorRecoveryType", true}; + CheckResult result = check(R"( --!strict local Vec3 = {} @@ -2209,7 +2211,7 @@ TEST_CASE_FIXTURE(Fixture, "typecheck_overloaded_multiply_that_is_an_intersectio CHECK_EQ("Vec3", toString(requireType("b"))); CHECK_EQ("Vec3", toString(requireType("c"))); CHECK_EQ("Vec3", toString(requireType("d"))); - CHECK(get(requireType("e"))); + CHECK_EQ("Vec3", toString(requireType("e"))); } TEST_CASE_FIXTURE(Fixture, "compare_numbers") @@ -2901,6 +2903,8 @@ end TEST_CASE_FIXTURE(Fixture, "CheckMethodsOfNumber") { + ScopedFastFlag sff{"LuauErrorRecoveryType", true}; + CheckResult result = check(R"( local x: number = 9999 function x:y(z: number) @@ -2908,7 +2912,7 @@ function x:y(z: number) end )"); - LUAU_REQUIRE_ERROR_COUNT(3, result); + LUAU_REQUIRE_ERROR_COUNT(2, result); } TEST_CASE_FIXTURE(Fixture, "CheckMethodsOfError") @@ -2920,7 +2924,7 @@ function x:y(z: number) end )"); - LUAU_REQUIRE_ERROR_COUNT(2, result); + LUAU_REQUIRE_ERRORS(result); } TEST_CASE_FIXTURE(Fixture, "CallOrOfFunctions") @@ -3799,7 +3803,7 @@ TEST_CASE_FIXTURE(Fixture, "UnknownGlobalCompoundAssign") print(a) )"); - LUAU_REQUIRE_ERROR_COUNT(2, result); + LUAU_REQUIRE_ERRORS(result); CHECK_EQ(toString(result.errors[0]), "Unknown global 'a'"); } @@ -4215,7 +4219,7 @@ TEST_CASE_FIXTURE(Fixture, "no_stack_overflow_from_quantifying") std::optional t0 = getMainModule()->getModuleScope()->lookupType("t0"); REQUIRE(t0); - CHECK(get(t0->type)); + CHECK_EQ("*unknown*", toString(t0->type)); auto it = std::find_if(result.errors.begin(), result.errors.end(), [](TypeError& err) { return get(err); @@ -4238,7 +4242,7 @@ TEST_CASE_FIXTURE(Fixture, "no_stack_overflow_from_isoptional") std::optional t0 = getMainModule()->getModuleScope()->lookupType("t0"); REQUIRE(t0); - CHECK(get(t0->type)); + CHECK_EQ("*unknown*", toString(t0->type)); auto it = std::find_if(result.errors.begin(), result.errors.end(), [](TypeError& err) { return get(err); @@ -4394,6 +4398,25 @@ TEST_CASE_FIXTURE(Fixture, "record_matching_overload") CHECK_EQ(toString(*it), "(number) -> number"); } +TEST_CASE_FIXTURE(Fixture, "return_type_by_overload") +{ + ScopedFastFlag sff{"LuauErrorRecoveryType", true}; + + CheckResult result = check(R"( + type Overload = ((string) -> string) & ((number, number) -> number) + local abc: Overload + local x = abc(true) + local y = abc(true,true) + local z = abc(true,true,true) + )"); + + LUAU_REQUIRE_ERRORS(result); + CHECK_EQ("string", toString(requireType("x"))); + CHECK_EQ("number", toString(requireType("y"))); + // Should this be string|number? + CHECK_EQ("string", toString(requireType("z"))); +} + TEST_CASE_FIXTURE(Fixture, "infer_anonymous_function_arguments") { // Simple direct arg to arg propagation @@ -4740,4 +4763,20 @@ TEST_CASE_FIXTURE(Fixture, "tc_if_else_expressions3") } } +TEST_CASE_FIXTURE(Fixture, "type_error_addition") +{ + CheckResult result = check(R"( +--!strict +local foo = makesandwich() +local bar = foo.nutrition + 100 + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + // We should definitely get this error + CHECK_EQ("Unknown global 'makesandwich'", toString(result.errors[0])); + // We get this error if makesandwich() returns a free type + // CHECK_EQ("Unknown type used in + operation; consider adding a type annotation to 'foo'", toString(result.errors[1])); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.tryUnify.test.cpp b/tests/TypeInfer.tryUnify.test.cpp index 2d697fc9..9f9a007f 100644 --- a/tests/TypeInfer.tryUnify.test.cpp +++ b/tests/TypeInfer.tryUnify.test.cpp @@ -121,9 +121,26 @@ TEST_CASE_FIXTURE(TryUnifyFixture, "members_of_failed_typepack_unification_are_u LUAU_REQUIRE_ERROR_COUNT(1, result); - TypeId bType = requireType("b"); + CHECK_EQ("a", toString(requireType("a"))); + CHECK_EQ("*unknown*", toString(requireType("b"))); +} - CHECK_MESSAGE(get(bType), "Should be an error: " << toString(bType)); +TEST_CASE_FIXTURE(TryUnifyFixture, "result_of_failed_typepack_unification_is_constrained") +{ + ScopedFastFlag sff{"LuauErrorRecoveryType", true}; + + CheckResult result = check(R"( + function f(arg: number) return arg end + local a + local b + local c = f(a, b) + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + CHECK_EQ("a", toString(requireType("a"))); + CHECK_EQ("*unknown*", toString(requireType("b"))); + CHECK_EQ("number", toString(requireType("c"))); } TEST_CASE_FIXTURE(TryUnifyFixture, "typepack_unification_should_trim_free_tails") @@ -167,15 +184,6 @@ TEST_CASE_FIXTURE(TryUnifyFixture, "variadic_tails_respect_progress") CHECK(state.errors.empty()); } -TEST_CASE_FIXTURE(TryUnifyFixture, "unifying_variadic_pack_with_error_should_work") -{ - TypePackId variadicPack = arena.addTypePack(TypePackVar{VariadicTypePack{typeChecker.numberType}}); - TypePackId errorPack = arena.addTypePack(TypePack{{typeChecker.numberType}, arena.addTypePack(TypePackVar{Unifiable::Error{}})}); - - state.tryUnify(variadicPack, errorPack); - REQUIRE_EQ(0, state.errors.size()); -} - TEST_CASE_FIXTURE(TryUnifyFixture, "variadics_should_use_reversed_properly") { CheckResult result = check(R"( diff --git a/tests/TypeInfer.unionTypes.test.cpp b/tests/TypeInfer.unionTypes.test.cpp index 9f29b642..48496b89 100644 --- a/tests/TypeInfer.unionTypes.test.cpp +++ b/tests/TypeInfer.unionTypes.test.cpp @@ -200,8 +200,7 @@ TEST_CASE_FIXTURE(Fixture, "index_on_a_union_type_with_missing_property") CHECK_EQ(mup->missing[0], *bTy); CHECK_EQ(mup->key, "x"); - TypeId r = requireType("r"); - CHECK_MESSAGE(get(r), "Expected error, got " << toString(r)); + CHECK_EQ("*unknown*", toString(requireType("r"))); } TEST_CASE_FIXTURE(Fixture, "index_on_a_union_type_with_one_property_of_type_any") @@ -283,7 +282,7 @@ local c = b:foo(1, 2) CHECK_EQ("Value of type 'A?' could be nil", toString(result.errors[0])); } -TEST_CASE_FIXTURE(Fixture, "optional_union_follow") +TEST_CASE_FIXTURE(UnfrozenFixture, "optional_union_follow") { CheckResult result = check(R"( local y: number? = 2 diff --git a/tests/conformance/coroutine.lua b/tests/conformance/coroutine.lua index 75329642..4d9b1295 100644 --- a/tests/conformance/coroutine.lua +++ b/tests/conformance/coroutine.lua @@ -319,4 +319,58 @@ for i=0,30 do assert(#T2 == 1 or T2[#T2] == 42) end +-- test coroutine.close +do + -- ok to close a dead coroutine + local co = coroutine.create(type) + assert(coroutine.resume(co, "testing 'coroutine.close'")) + assert(coroutine.status(co) == "dead") + local st, msg = coroutine.close(co) + assert(st and msg == nil) + -- also ok to close it again + st, msg = coroutine.close(co) + assert(st and msg == nil) + + + -- cannot close the running coroutine + coroutine.wrap(function() + local st, msg = pcall(coroutine.close, coroutine.running()) + assert(not st and string.find(msg, "running")) + end)() + + -- cannot close a "normal" coroutine + coroutine.wrap(function() + local co = coroutine.running() + coroutine.wrap(function () + local st, msg = pcall(coroutine.close, co) + assert(not st and string.find(msg, "normal")) + end)() + end)() + + -- closing a coroutine after an error + local co = coroutine.create(error) + local obj = {42} + local st, msg = coroutine.resume(co, obj) + assert(not st and msg == obj) + st, msg = coroutine.close(co) + assert(not st and msg == obj) + -- after closing, no more errors + st, msg = coroutine.close(co) + assert(st and msg == nil) + + -- closing a coroutine that has outstanding upvalues + local f + local co = coroutine.create(function() + local a = 42 + f = function() return a end + coroutine.yield() + a = 20 + end) + coroutine.resume(co) + assert(f() == 42) + st, msg = coroutine.close(co) + assert(st and msg == nil) + assert(f() == 42) +end + return'OK' diff --git a/tests/conformance/debugger.lua b/tests/conformance/debugger.lua index 5e69fc6b..6ba99fb9 100644 --- a/tests/conformance/debugger.lua +++ b/tests/conformance/debugger.lua @@ -45,4 +45,13 @@ breakpoint(38) -- break inside corobad() local co = coroutine.create(corobad) assert(coroutine.resume(co) == false) -- this breaks, resumes and dies! +function bar() + print("in bar") +end + +breakpoint(49) +breakpoint(49, false) -- validate that disabling breakpoints works + +bar() + return 'OK' From 3f1508c83aafe0e20864e2873a6ec3f19eccea2a Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Fri, 19 Nov 2021 08:10:07 -0800 Subject: [PATCH 063/399] Sync to upstream/release/505 (#216) - Improve error recovery during type checking - Initial (not fully complete) implementation for singleton types (RFC RFC: Singleton types #37) - Implement a C-friendly interface for compiler (luacode.h) - Remove C++ features from lua.h (removed default arguments from luau_load and lua_pushcfunction) - Fix lua_breakpoint behavior when enabled=false - Implement coroutine.close (RFC RFC: coroutine.close #88) Note, this introduces small breaking changes in lua.h: - luau_load env argument is now required, pass an extra 0 - lua_pushcfunction now must be called with 3 arguments; if you were calling it with 2 arguments, pass an extra NULL; if you were calling it with 4, use lua_pushcclosure. These changes are necessary to make sure lua.h can be used from pure C - the future release will make it possible by adding an option to luaconf.h to change function name mangling to be C-compatible. We don't anticipate breaking the FFI interface in the future, but this change was necessary to restore C compatibility. Closes #121 Fixes #213 --- Analysis/include/Luau/Documentation.h | 11 +- Analysis/include/Luau/ToString.h | 11 +- Analysis/include/Luau/TypeInfer.h | 24 +- Analysis/include/Luau/TypeVar.h | 87 ++++- Analysis/include/Luau/Unifiable.h | 2 + Analysis/include/Luau/Unifier.h | 1 + Analysis/src/Autocomplete.cpp | 33 +- Analysis/src/EmbeddedBuiltinDefinitions.cpp | 1 + Analysis/src/Error.cpp | 8 +- Analysis/src/Frontend.cpp | 4 +- Analysis/src/Module.cpp | 16 +- Analysis/src/ToString.cpp | 115 +++++- Analysis/src/Transpiler.cpp | 55 --- Analysis/src/TypeAttach.cpp | 16 + Analysis/src/TypeInfer.cpp | 366 ++++++++++++------- Analysis/src/TypePack.cpp | 1 - Analysis/src/TypeVar.cpp | 48 ++- Analysis/src/Unifier.cpp | 64 +++- Ast/include/Luau/Ast.h | 32 ++ Ast/include/Luau/Parser.h | 1 + Ast/include/Luau/StringUtils.h | 2 + Ast/src/Ast.cpp | 22 ++ Ast/src/Parser.cpp | 68 +++- Ast/src/StringUtils.cpp | 58 +++ CLI/Analyze.cpp | 28 +- CLI/FileUtils.cpp | 37 ++ CLI/FileUtils.h | 3 + CLI/Repl.cpp | 81 ++--- Compiler/include/Luau/Compiler.h | 4 +- Compiler/include/luacode.h | 39 ++ Compiler/src/Compiler.cpp | 46 +-- Compiler/src/lcode.cpp | 29 ++ Sources.cmake | 3 + VM/include/lua.h | 18 +- VM/include/lualib.h | 6 +- VM/src/lapi.cpp | 10 +- VM/src/lbaselib.cpp | 8 +- VM/src/lcorolib.cpp | 36 +- VM/src/ldebug.cpp | 10 +- VM/src/ldo.cpp | 6 +- VM/src/linit.cpp | 2 +- VM/src/lstate.cpp | 28 ++ VM/src/lstrlib.cpp | 2 +- VM/src/lutf8lib.cpp | 2 +- bench/bench.py | 32 +- fuzz/proto.cpp | 2 +- tests/Autocomplete.test.cpp | 16 +- tests/Compiler.test.cpp | 63 +--- tests/Conformance.test.cpp | 97 ++--- tests/IostreamOptional.h | 3 +- tests/Module.test.cpp | 6 +- tests/ToString.test.cpp | 111 +++++- tests/TypeInfer.aliases.test.cpp | 2 +- tests/TypeInfer.annotations.test.cpp | 4 +- tests/TypeInfer.generics.test.cpp | 6 +- tests/TypeInfer.refinements.test.cpp | 3 +- tests/TypeInfer.singletons.test.cpp | 377 ++++++++++++++++++++ tests/TypeInfer.test.cpp | 87 +++-- tests/TypeInfer.tryUnify.test.cpp | 30 +- tests/TypeInfer.unionTypes.test.cpp | 5 +- tests/conformance/coroutine.lua | 54 +++ tests/conformance/debugger.lua | 9 + 62 files changed, 1785 insertions(+), 566 deletions(-) create mode 100644 Compiler/include/luacode.h create mode 100644 Compiler/src/lcode.cpp create mode 100644 tests/TypeInfer.singletons.test.cpp diff --git a/Analysis/include/Luau/Documentation.h b/Analysis/include/Luau/Documentation.h index 7b609b4f..68ff3a7c 100644 --- a/Analysis/include/Luau/Documentation.h +++ b/Analysis/include/Luau/Documentation.h @@ -12,10 +12,17 @@ namespace Luau struct FunctionDocumentation; struct TableDocumentation; struct OverloadedFunctionDocumentation; +struct BasicDocumentation; -using Documentation = Luau::Variant; +using Documentation = Luau::Variant; using DocumentationSymbol = std::string; +struct BasicDocumentation +{ + std::string documentation; + std::string learnMoreLink; +}; + struct FunctionParameterDocumentation { std::string name; @@ -29,6 +36,7 @@ struct FunctionDocumentation std::string documentation; std::vector parameters; std::vector returns; + std::string learnMoreLink; }; struct OverloadedFunctionDocumentation @@ -43,6 +51,7 @@ struct TableDocumentation { std::string documentation; Luau::DenseHashMap keys; + std::string learnMoreLink; }; using DocumentationDatabase = Luau::DenseHashMap; diff --git a/Analysis/include/Luau/ToString.h b/Analysis/include/Luau/ToString.h index e5683fc4..50379c1c 100644 --- a/Analysis/include/Luau/ToString.h +++ b/Analysis/include/Luau/ToString.h @@ -23,10 +23,11 @@ struct ToStringNameMap struct ToStringOptions { - bool exhaustive = false; // If true, we produce complete output rather than comprehensible output - bool useLineBreaks = false; // If true, we insert new lines to separate long results such as table entries/metatable. - bool functionTypeArguments = false; // If true, output function type argument names when they are available - bool hideTableKind = false; // If true, all tables will be surrounded with plain '{}' + bool exhaustive = false; // If true, we produce complete output rather than comprehensible output + bool useLineBreaks = false; // If true, we insert new lines to separate long results such as table entries/metatable. + bool functionTypeArguments = false; // If true, output function type argument names when they are available + bool hideTableKind = false; // If true, all tables will be surrounded with plain '{}' + bool hideNamedFunctionTypeParameters = false; // If true, type parameters of functions will be hidden at top-level. size_t maxTableLength = size_t(FInt::LuauTableTypeMaximumStringifierLength); // Only applied to TableTypeVars size_t maxTypeLength = size_t(FInt::LuauTypeMaximumStringifierLength); std::optional nameMap; @@ -64,6 +65,8 @@ inline std::string toString(TypePackId ty) std::string toString(const TypeVar& tv, const ToStringOptions& opts = {}); std::string toString(const TypePackVar& tp, const ToStringOptions& opts = {}); +std::string toStringNamedFunction(const std::string& prefix, const FunctionTypeVar& ftv, ToStringOptions opts = {}); + // It could be useful to see the text representation of a type during a debugging session instead of exploring the content of the class // These functions will dump the type to stdout and can be evaluated in Watch/Immediate windows or as gdb/lldb expression void dump(TypeId ty); diff --git a/Analysis/include/Luau/TypeInfer.h b/Analysis/include/Luau/TypeInfer.h index 306ac77d..78d642c5 100644 --- a/Analysis/include/Luau/TypeInfer.h +++ b/Analysis/include/Luau/TypeInfer.h @@ -175,10 +175,10 @@ struct TypeChecker std::vector> getExpectedTypesForCall(const std::vector& overloads, size_t argumentCount, bool selfCall); std::optional> checkCallOverload(const ScopePtr& scope, const AstExprCall& expr, TypeId fn, TypePackId retPack, TypePackId argPack, TypePack* args, const std::vector& argLocations, const ExprResult& argListResult, - std::vector& overloadsThatMatchArgCount, std::vector& errors); + std::vector& overloadsThatMatchArgCount, std::vector& overloadsThatDont, std::vector& errors); bool handleSelfCallMismatch(const ScopePtr& scope, const AstExprCall& expr, TypePack* args, const std::vector& argLocations, const std::vector& errors); - ExprResult reportOverloadResolutionError(const ScopePtr& scope, const AstExprCall& expr, TypePackId retPack, TypePackId argPack, + void reportOverloadResolutionError(const ScopePtr& scope, const AstExprCall& expr, TypePackId retPack, TypePackId argPack, const std::vector& argLocations, const std::vector& overloads, const std::vector& overloadsThatMatchArgCount, const std::vector& errors); @@ -282,6 +282,14 @@ public: // Wrapper for merge(l, r, toUnion) but without the lambda junk. void merge(RefinementMap& l, const RefinementMap& r); + // Produce an "emergency backup type" for recovery from type errors. + // This comes in two flavours, depening on whether or not we can make a good guess + // for an error recovery type. + TypeId errorRecoveryType(TypeId guess); + TypePackId errorRecoveryTypePack(TypePackId guess); + TypeId errorRecoveryType(const ScopePtr& scope); + TypePackId errorRecoveryTypePack(const ScopePtr& scope); + private: void prepareErrorsForDisplay(ErrorVec& errVec); void diagnoseMissingTableKey(UnknownProperty* utk, TypeErrorData& data); @@ -297,6 +305,10 @@ private: TypeId freshType(const ScopePtr& scope); TypeId freshType(TypeLevel level); + // Produce a new singleton type var. + TypeId singletonType(bool value); + TypeId singletonType(std::string value); + // Returns nullopt if the predicate filters down the TypeId to 0 options. std::optional filterMap(TypeId type, TypeIdPredicate predicate); @@ -330,8 +342,8 @@ private: const std::vector& typePackParams, const Location& location); // Note: `scope` must be a fresh scope. - std::pair, std::vector> createGenericTypes( - const ScopePtr& scope, std::optional levelOpt, const AstNode& node, const AstArray& genericNames, const AstArray& genericPackNames); + std::pair, std::vector> createGenericTypes(const ScopePtr& scope, std::optional levelOpt, + const AstNode& node, const AstArray& genericNames, const AstArray& genericPackNames); public: ErrorVec resolve(const PredicateVec& predicates, const ScopePtr& scope, bool sense); @@ -347,7 +359,6 @@ private: void resolve(const OrPredicate& orP, ErrorVec& errVec, RefinementMap& refis, const ScopePtr& scope, bool sense); void resolve(const IsAPredicate& isaP, ErrorVec& errVec, RefinementMap& refis, const ScopePtr& scope, bool sense); void resolve(const TypeGuardPredicate& typeguardP, ErrorVec& errVec, RefinementMap& refis, const ScopePtr& scope, bool sense); - void DEPRECATED_resolve(const TypeGuardPredicate& typeguardP, ErrorVec& errVec, RefinementMap& refis, const ScopePtr& scope, bool sense); void resolve(const EqPredicate& eqP, ErrorVec& errVec, RefinementMap& refis, const ScopePtr& scope, bool sense); bool isNonstrictMode() const; @@ -387,12 +398,9 @@ public: const TypeId booleanType; const TypeId threadType; const TypeId anyType; - - const TypeId errorType; const TypeId optionalNumberType; const TypePackId anyTypePack; - const TypePackId errorTypePack; private: int checkRecursionCount = 0; diff --git a/Analysis/include/Luau/TypeVar.h b/Analysis/include/Luau/TypeVar.h index 6bd7932d..093ea431 100644 --- a/Analysis/include/Luau/TypeVar.h +++ b/Analysis/include/Luau/TypeVar.h @@ -108,6 +108,79 @@ struct PrimitiveTypeVar } }; +// Singleton types https://github.com/Roblox/luau/blob/master/rfcs/syntax-singleton-types.md +// Types for true and false +struct BoolSingleton +{ + bool value; + + bool operator==(const BoolSingleton& rhs) const + { + return value == rhs.value; + } + + bool operator!=(const BoolSingleton& rhs) const + { + return !(*this == rhs); + } +}; + +// Types for "foo", "bar" etc. +struct StringSingleton +{ + std::string value; + + bool operator==(const StringSingleton& rhs) const + { + return value == rhs.value; + } + + bool operator!=(const StringSingleton& rhs) const + { + return !(*this == rhs); + } +}; + +// No type for float singletons, partly because === isn't any equalivalence on floats +// (NaN != NaN). + +using SingletonVariant = Luau::Variant; + +struct SingletonTypeVar +{ + explicit SingletonTypeVar(const SingletonVariant& variant) + : variant(variant) + { + } + + explicit SingletonTypeVar(SingletonVariant&& variant) + : variant(std::move(variant)) + { + } + + // Default operator== is C++20. + bool operator==(const SingletonTypeVar& rhs) const + { + return variant == rhs.variant; + } + + bool operator!=(const SingletonTypeVar& rhs) const + { + return !(*this == rhs); + } + + SingletonVariant variant; +}; + +template +const T* get(const SingletonTypeVar* stv) +{ + if (stv) + return get_if(&stv->variant); + else + return nullptr; +} + struct FunctionArgument { Name name; @@ -332,8 +405,8 @@ struct LazyTypeVar using ErrorTypeVar = Unifiable::Error; -using TypeVariant = Unifiable::Variant; +using TypeVariant = Unifiable::Variant; struct TypeVar final { @@ -410,6 +483,9 @@ bool isGeneric(const TypeId ty); // Checks if a type may be instantiated to one containing generic type binders bool maybeGeneric(const TypeId ty); +// Checks if a type is of the form T1|...|Tn where one of the Ti is a singleton +bool maybeSingleton(TypeId ty); + struct SingletonTypes { const TypeId nilType; @@ -418,16 +494,19 @@ struct SingletonTypes const TypeId booleanType; const TypeId threadType; const TypeId anyType; - const TypeId errorType; const TypeId optionalNumberType; const TypePackId anyTypePack; - const TypePackId errorTypePack; SingletonTypes(); SingletonTypes(const SingletonTypes&) = delete; void operator=(const SingletonTypes&) = delete; + TypeId errorRecoveryType(TypeId guess); + TypePackId errorRecoveryTypePack(TypePackId guess); + TypeId errorRecoveryType(); + TypePackId errorRecoveryTypePack(); + private: std::unique_ptr arena; TypeId makeStringMetatable(); diff --git a/Analysis/include/Luau/Unifiable.h b/Analysis/include/Luau/Unifiable.h index c2e07e46..b47610fc 100644 --- a/Analysis/include/Luau/Unifiable.h +++ b/Analysis/include/Luau/Unifiable.h @@ -105,6 +105,8 @@ private: struct Error { + // This constructor has to be public, since it's used in TypeVar and TypePack, + // but shouldn't be called directly. Please use errorRecoveryType() instead. Error(); int index; diff --git a/Analysis/include/Luau/Unifier.h b/Analysis/include/Luau/Unifier.h index be0aadd0..503034a1 100644 --- a/Analysis/include/Luau/Unifier.h +++ b/Analysis/include/Luau/Unifier.h @@ -65,6 +65,7 @@ struct Unifier private: void tryUnify_(TypeId superTy, TypeId subTy, bool isFunctionCall = false, bool isIntersection = false); void tryUnifyPrimitives(TypeId superTy, TypeId subTy); + void tryUnifySingletons(TypeId superTy, TypeId subTy); void tryUnifyFunctions(TypeId superTy, TypeId subTy, bool isFunctionCall = false); void tryUnifyTables(TypeId left, TypeId right, bool isIntersection = false); void DEPRECATED_tryUnifyTables(TypeId left, TypeId right, bool isIntersection = false); diff --git a/Analysis/src/Autocomplete.cpp b/Analysis/src/Autocomplete.cpp index 1c94bb68..6fc0b3f8 100644 --- a/Analysis/src/Autocomplete.cpp +++ b/Analysis/src/Autocomplete.cpp @@ -14,6 +14,7 @@ LUAU_FASTFLAGVARIABLE(ElseElseIfCompletionImprovements, false); LUAU_FASTFLAG(LuauIfElseExpressionAnalysisSupport) +LUAU_FASTFLAGVARIABLE(LuauAutocompleteAvoidMutation, false); static const std::unordered_set kStatementStartingKeywords = { "while", "if", "local", "repeat", "function", "do", "for", "return", "break", "continue", "type", "export"}; @@ -198,11 +199,24 @@ static TypeCorrectKind checkTypeCorrectKind(const Module& module, TypeArena* typ UnifierSharedState unifierState(&iceReporter); Unifier unifier(typeArena, Mode::Strict, module.getModuleScope(), Location(), Variance::Covariant, unifierState); - unifier.tryUnify(expectedType, actualType); + if (FFlag::LuauAutocompleteAvoidMutation) + { + SeenTypes seenTypes; + SeenTypePacks seenTypePacks; + expectedType = clone(expectedType, *typeArena, seenTypes, seenTypePacks, nullptr); + actualType = clone(actualType, *typeArena, seenTypes, seenTypePacks, nullptr); - bool ok = unifier.errors.empty(); - unifier.log.rollback(); - return ok; + auto errors = unifier.canUnify(expectedType, actualType); + return errors.empty(); + } + else + { + unifier.tryUnify(expectedType, actualType); + + bool ok = unifier.errors.empty(); + unifier.log.rollback(); + return ok; + } }; auto expr = node->asExpr(); @@ -1496,11 +1510,9 @@ AutocompleteResult autocomplete(Frontend& frontend, const ModuleName& moduleName if (!sourceModule) return {}; - TypeChecker& typeChecker = - (frontend.options.typecheckTwice ? frontend.typeCheckerForAutocomplete : frontend.typeChecker); - ModulePtr module = - (frontend.options.typecheckTwice ? frontend.moduleResolverForAutocomplete.getModule(moduleName) - : frontend.moduleResolver.getModule(moduleName)); + TypeChecker& typeChecker = (frontend.options.typecheckTwice ? frontend.typeCheckerForAutocomplete : frontend.typeChecker); + ModulePtr module = (frontend.options.typecheckTwice ? frontend.moduleResolverForAutocomplete.getModule(moduleName) + : frontend.moduleResolver.getModule(moduleName)); if (!module) return {}; @@ -1527,8 +1539,7 @@ OwningAutocompleteResult autocompleteSource(Frontend& frontend, std::string_view sourceModule->mode = Mode::Strict; sourceModule->commentLocations = std::move(result.commentLocations); - TypeChecker& typeChecker = - (frontend.options.typecheckTwice ? frontend.typeCheckerForAutocomplete : frontend.typeChecker); + TypeChecker& typeChecker = (frontend.options.typecheckTwice ? frontend.typeCheckerForAutocomplete : frontend.typeChecker); ModulePtr module = typeChecker.check(*sourceModule, Mode::Strict); diff --git a/Analysis/src/EmbeddedBuiltinDefinitions.cpp b/Analysis/src/EmbeddedBuiltinDefinitions.cpp index 96703ef1..9f5c8250 100644 --- a/Analysis/src/EmbeddedBuiltinDefinitions.cpp +++ b/Analysis/src/EmbeddedBuiltinDefinitions.cpp @@ -153,6 +153,7 @@ declare function gcinfo(): number wrap: ((A...) -> R...) -> any, yield: (A...) -> R..., isyieldable: () -> boolean, + close: (thread) -> (boolean, any?) } declare table: { diff --git a/Analysis/src/Error.cpp b/Analysis/src/Error.cpp index 46ff2c72..f80d50a7 100644 --- a/Analysis/src/Error.cpp +++ b/Analysis/src/Error.cpp @@ -180,13 +180,13 @@ struct ErrorConverter switch (e.context) { case CountMismatch::Return: - return "Expected to return " + std::to_string(e.expected) + " value" + expectedS + ", but " + - std::to_string(e.actual) + " " + actualVerb + " returned here"; + return "Expected to return " + std::to_string(e.expected) + " value" + expectedS + ", but " + std::to_string(e.actual) + " " + + actualVerb + " returned here"; case CountMismatch::Result: // It is alright if right hand side produces more values than the // left hand side accepts. In this context consider only the opposite case. - return "Function only returns " + std::to_string(e.expected) + " value" + expectedS + ". " + - std::to_string(e.actual) + " are required here"; + return "Function only returns " + std::to_string(e.expected) + " value" + expectedS + ". " + std::to_string(e.actual) + + " are required here"; case CountMismatch::Arg: if (FFlag::LuauTypeAliasPacks) return "Argument count mismatch. Function " + wrongNumberOfArgsString(e.expected, e.actual); diff --git a/Analysis/src/Frontend.cpp b/Analysis/src/Frontend.cpp index 2f411274..1e97705d 100644 --- a/Analysis/src/Frontend.cpp +++ b/Analysis/src/Frontend.cpp @@ -22,7 +22,6 @@ LUAU_FASTFLAGVARIABLE(LuauResolveModuleNameWithoutACurrentModule, false) LUAU_FASTFLAG(LuauTraceRequireLookupChild) LUAU_FASTFLAGVARIABLE(LuauPersistDefinitionFileTypes, false) LUAU_FASTFLAG(LuauNewRequireTrace2) -LUAU_FASTFLAGVARIABLE(LuauClearScopes, false) namespace Luau { @@ -458,8 +457,7 @@ CheckResult Frontend::check(const ModuleName& name) module->astTypes.clear(); module->astExpectedTypes.clear(); module->astOriginalCallTypes.clear(); - if (FFlag::LuauClearScopes) - module->scopes.resize(1); + module->scopes.resize(1); } if (mode != Mode::NoCheck) diff --git a/Analysis/src/Module.cpp b/Analysis/src/Module.cpp index 880ffd2e..32a0646a 100644 --- a/Analysis/src/Module.cpp +++ b/Analysis/src/Module.cpp @@ -161,6 +161,7 @@ struct TypeCloner void operator()(const Unifiable::Bound& t); void operator()(const Unifiable::Error& t); void operator()(const PrimitiveTypeVar& t); + void operator()(const SingletonTypeVar& t); void operator()(const FunctionTypeVar& t); void operator()(const TableTypeVar& t); void operator()(const MetatableTypeVar& t); @@ -199,7 +200,9 @@ struct TypePackCloner if (encounteredFreeType) *encounteredFreeType = true; - seenTypePacks[typePackId] = dest.addTypePack(TypePackVar{Unifiable::Error{}}); + TypePackId err = singletonTypes.errorRecoveryTypePack(singletonTypes.anyTypePack); + TypePackId cloned = dest.addTypePack(*err); + seenTypePacks[typePackId] = cloned; } void operator()(const Unifiable::Generic& t) @@ -251,8 +254,9 @@ void TypeCloner::operator()(const Unifiable::Free& t) { if (encounteredFreeType) *encounteredFreeType = true; - - seenTypes[typeId] = dest.addType(ErrorTypeVar{}); + TypeId err = singletonTypes.errorRecoveryType(singletonTypes.anyType); + TypeId cloned = dest.addType(*err); + seenTypes[typeId] = cloned; } void TypeCloner::operator()(const Unifiable::Generic& t) @@ -270,11 +274,17 @@ void TypeCloner::operator()(const Unifiable::Error& t) { defaultClone(t); } + void TypeCloner::operator()(const PrimitiveTypeVar& t) { defaultClone(t); } +void TypeCloner::operator()(const SingletonTypeVar& t) +{ + defaultClone(t); +} + void TypeCloner::operator()(const FunctionTypeVar& t) { TypeId result = dest.addType(FunctionTypeVar{TypeLevel{0, 0}, {}, {}, nullptr, nullptr, t.definition, t.hasSelf}); diff --git a/Analysis/src/ToString.cpp b/Analysis/src/ToString.cpp index 885fd489..735bfa50 100644 --- a/Analysis/src/ToString.cpp +++ b/Analysis/src/ToString.cpp @@ -350,6 +350,23 @@ struct TypeVarStringifier } } + void operator()(TypeId, const SingletonTypeVar& stv) + { + if (const BoolSingleton* bs = Luau::get(&stv)) + state.emit(bs->value ? "true" : "false"); + else if (const StringSingleton* ss = Luau::get(&stv)) + { + state.emit("\""); + state.emit(escape(ss->value)); + state.emit("\""); + } + else + { + LUAU_ASSERT(!"Unknown singleton type"); + throw std::runtime_error("Unknown singleton type"); + } + } + void operator()(TypeId, const FunctionTypeVar& ftv) { if (state.hasSeen(&ftv)) @@ -359,6 +376,7 @@ struct TypeVarStringifier return; } + // We should not be respecting opts.hideNamedFunctionTypeParameters here. if (ftv.generics.size() > 0 || ftv.genericPacks.size() > 0) { state.emit("<"); @@ -514,7 +532,14 @@ struct TypeVarStringifier break; } - state.emit(name); + if (isIdentifier(name)) + state.emit(name); + else + { + state.emit("[\""); + state.emit(escape(name)); + state.emit("\"]"); + } state.emit(": "); stringify(prop.type); comma = true; @@ -1084,6 +1109,94 @@ std::string toString(const TypePackVar& tp, const ToStringOptions& opts) return toString(const_cast(&tp), std::move(opts)); } +std::string toStringNamedFunction(const std::string& prefix, const FunctionTypeVar& ftv, ToStringOptions opts) +{ + std::string s = prefix; + + auto toString_ = [&opts](TypeId ty) -> std::string { + ToStringResult res = toStringDetailed(ty, opts); + opts.nameMap = std::move(res.nameMap); + return res.name; + }; + + auto toStringPack_ = [&opts](TypePackId ty) -> std::string { + ToStringResult res = toStringDetailed(ty, opts); + opts.nameMap = std::move(res.nameMap); + return res.name; + }; + + if (!opts.hideNamedFunctionTypeParameters && (!ftv.generics.empty() || !ftv.genericPacks.empty())) + { + s += "<"; + + bool first = true; + for (TypeId g : ftv.generics) + { + if (!first) + s += ", "; + first = false; + s += toString_(g); + } + + for (TypePackId gp : ftv.genericPacks) + { + if (!first) + s += ", "; + first = false; + s += toStringPack_(gp); + } + + s += ">"; + } + + s += "("; + + auto argPackIter = begin(ftv.argTypes); + auto argNameIter = ftv.argNames.begin(); + + bool first = true; + while (argPackIter != end(ftv.argTypes)) + { + if (!first) + s += ", "; + first = false; + + // argNames is guaranteed to be equal to argTypes iff argNames is not empty. + // We don't currently respect opts.functionTypeArguments. I don't think this function should. + if (!ftv.argNames.empty()) + s += (*argNameIter ? (*argNameIter)->name : "_") + ": "; + s += toString_(*argPackIter); + + ++argPackIter; + if (!ftv.argNames.empty()) + { + LUAU_ASSERT(argNameIter != ftv.argNames.end()); + ++argNameIter; + } + } + + if (argPackIter.tail()) + { + if (auto vtp = get(*argPackIter.tail())) + s += ", ...: " + toString_(vtp->ty); + else + s += ", ...: " + toStringPack_(*argPackIter.tail()); + } + + s += "): "; + + size_t retSize = size(ftv.retType); + bool hasTail = !finite(ftv.retType); + if (retSize == 0 && !hasTail) + s += "()"; + else if ((retSize == 0 && hasTail) || (retSize == 1 && !hasTail)) + s += toStringPack_(ftv.retType); + else + s += "(" + toStringPack_(ftv.retType) + ")"; + + return s; +} + void dump(TypeId ty) { ToStringOptions opts; diff --git a/Analysis/src/Transpiler.cpp b/Analysis/src/Transpiler.cpp index 7d880af4..6627fbe3 100644 --- a/Analysis/src/Transpiler.cpp +++ b/Analysis/src/Transpiler.cpp @@ -14,61 +14,6 @@ LUAU_FASTFLAG(LuauTypeAliasPacks) namespace { - -std::string escape(std::string_view s) -{ - std::string r; - r.reserve(s.size() + 50); // arbitrary number to guess how many characters we'll be inserting - - for (uint8_t c : s) - { - if (c >= ' ' && c != '\\' && c != '\'' && c != '\"') - r += c; - else - { - r += '\\'; - - switch (c) - { - case '\a': - r += 'a'; - break; - case '\b': - r += 'b'; - break; - case '\f': - r += 'f'; - break; - case '\n': - r += 'n'; - break; - case '\r': - r += 'r'; - break; - case '\t': - r += 't'; - break; - case '\v': - r += 'v'; - break; - case '\'': - r += '\''; - break; - case '\"': - r += '\"'; - break; - case '\\': - r += '\\'; - break; - default: - Luau::formatAppend(r, "%03u", c); - } - } - } - - return r; -} - bool isIdentifierStartChar(char c) { return (c >= 'A' && c <= 'Z') || (c >= 'a' && c <= 'z') || c == '_'; diff --git a/Analysis/src/TypeAttach.cpp b/Analysis/src/TypeAttach.cpp index 11aa7b39..af6d2543 100644 --- a/Analysis/src/TypeAttach.cpp +++ b/Analysis/src/TypeAttach.cpp @@ -96,6 +96,22 @@ public: return nullptr; } } + + AstType* operator()(const SingletonTypeVar& stv) + { + if (const BoolSingleton* bs = get(&stv)) + return allocator->alloc(Location(), bs->value); + else if (const StringSingleton* ss = get(&stv)) + { + AstArray value; + value.data = const_cast(ss->value.c_str()); + value.size = strlen(value.data); + return allocator->alloc(Location(), value); + } + else + return nullptr; + } + AstType* operator()(const AnyTypeVar&) { return allocator->alloc(Location(), std::nullopt, AstName("any")); diff --git a/Analysis/src/TypeInfer.cpp b/Analysis/src/TypeInfer.cpp index 96b79923..b2ae94c7 100644 --- a/Analysis/src/TypeInfer.cpp +++ b/Analysis/src/TypeInfer.cpp @@ -36,6 +36,9 @@ LUAU_FASTFLAG(LuauSubstitutionDontReplaceIgnoredTypes) LUAU_FASTFLAGVARIABLE(LuauQuantifyInPlace2, false) LUAU_FASTFLAG(LuauNewRequireTrace2) LUAU_FASTFLAG(LuauTypeAliasPacks) +LUAU_FASTFLAGVARIABLE(LuauSingletonTypes, false) +LUAU_FASTFLAGVARIABLE(LuauExpectedTypesOfProperties, false) +LUAU_FASTFLAGVARIABLE(LuauErrorRecoveryType, false) namespace Luau { @@ -211,10 +214,8 @@ TypeChecker::TypeChecker(ModuleResolver* resolver, InternalErrorReporter* iceHan , booleanType(singletonTypes.booleanType) , threadType(singletonTypes.threadType) , anyType(singletonTypes.anyType) - , errorType(singletonTypes.errorType) , optionalNumberType(singletonTypes.optionalNumberType) , anyTypePack(singletonTypes.anyTypePack) - , errorTypePack(singletonTypes.errorTypePack) { globalScope = std::make_shared(globalTypes.addTypePack(TypePackVar{FreeTypePack{TypeLevel{}}})); @@ -484,7 +485,7 @@ LUAU_NOINLINE void TypeChecker::checkBlockTypeAliases(const ScopePtr& scope, std TypeId type = bindings[name].type; if (get(follow(type))) { - *asMutable(type) = ErrorTypeVar{}; + *asMutable(type) = *errorRecoveryType(anyType); reportError(TypeError{typealias->location, OccursCheckFailed{}}); } } @@ -719,7 +720,7 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatAssign& assign) else if (auto tail = valueIter.tail()) { if (get(*tail)) - right = errorType; + right = errorRecoveryType(scope); else if (auto vtp = get(*tail)) right = vtp->ty; else if (get(*tail)) @@ -961,7 +962,7 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatForIn& forin) else if (get(callRetPack) || !first(callRetPack)) { for (TypeId var : varTypes) - unify(var, errorType, forin.location); + unify(var, errorRecoveryType(scope), forin.location); return check(loopScope, *forin.body); } @@ -979,7 +980,7 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatForIn& forin) const FunctionTypeVar* iterFunc = get(iterTy); if (!iterFunc) { - TypeId varTy = get(iterTy) ? anyType : errorType; + TypeId varTy = get(iterTy) ? anyType : errorRecoveryType(loopScope); for (TypeId var : varTypes) unify(var, varTy, forin.location); @@ -1152,9 +1153,9 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatTypeAlias& typealias reportError(TypeError{typealias.location, DuplicateTypeDefinition{name, location}}); if (FFlag::LuauTypeAliasPacks) - bindingsMap[name] = TypeFun{binding->typeParams, binding->typePackParams, errorType}; + bindingsMap[name] = TypeFun{binding->typeParams, binding->typePackParams, errorRecoveryType(anyType)}; else - bindingsMap[name] = TypeFun{binding->typeParams, errorType}; + bindingsMap[name] = TypeFun{binding->typeParams, errorRecoveryType(anyType)}; } else { @@ -1398,7 +1399,7 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExpr& if (FInt::LuauCheckRecursionLimit > 0 && checkRecursionCount >= FInt::LuauCheckRecursionLimit) { reportErrorCodeTooComplex(expr.location); - return {errorType}; + return {errorRecoveryType(scope)}; } ExprResult result; @@ -1407,12 +1408,22 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExpr& result = checkExpr(scope, *a->expr); else if (expr.is()) result = {nilType}; - else if (expr.is()) - result = {booleanType}; + else if (const AstExprConstantBool* bexpr = expr.as()) + { + if (FFlag::LuauSingletonTypes && expectedType && maybeSingleton(*expectedType)) + result = {singletonType(bexpr->value)}; + else + result = {booleanType}; + } + else if (const AstExprConstantString* sexpr = expr.as()) + { + if (FFlag::LuauSingletonTypes && expectedType && maybeSingleton(*expectedType)) + result = {singletonType(std::string(sexpr->value.data, sexpr->value.size))}; + else + result = {stringType}; + } else if (expr.is()) result = {numberType}; - else if (expr.is()) - result = {stringType}; else if (auto a = expr.as()) result = checkExpr(scope, *a); else if (auto a = expr.as()) @@ -1485,7 +1496,7 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprLo // TODO: tempting to ice here, but this breaks very often because our toposort doesn't enforce this constraint // ice("AstExprLocal exists but no binding definition for it?", expr.location); reportError(TypeError{expr.location, UnknownSymbol{expr.local->name.value, UnknownSymbol::Binding}}); - return {errorType}; + return {errorRecoveryType(scope)}; } ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprGlobal& expr) @@ -1497,7 +1508,7 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprGl return {*ty, {TruthyPredicate{std::move(*lvalue), expr.location}}}; reportError(TypeError{expr.location, UnknownSymbol{expr.name.value, UnknownSymbol::Binding}}); - return {errorType}; + return {errorRecoveryType(scope)}; } ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprVarargs& expr) @@ -1517,14 +1528,14 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprVa return {head}; } if (get(varargPack)) - return {errorType}; + return {errorRecoveryType(scope)}; else if (auto vtp = get(varargPack)) return {vtp->ty}; else if (get(varargPack)) { // TODO: Better error? reportError(expr.location, GenericError{"Trying to get a type from a variadic type parameter"}); - return {errorType}; + return {errorRecoveryType(scope)}; } else ice("Unknown TypePack type in checkExpr(AstExprVarargs)!"); @@ -1547,7 +1558,7 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprCa return {head, std::move(result.predicates)}; } if (get(retPack)) - return {errorType, std::move(result.predicates)}; + return {errorRecoveryType(scope), std::move(result.predicates)}; else if (auto vtp = get(retPack)) return {vtp->ty, std::move(result.predicates)}; else if (get(retPack)) @@ -1572,7 +1583,7 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprIn if (std::optional ty = getIndexTypeFromType(scope, lhsType, name, expr.location, true)) return {*ty}; - return {errorType}; + return {errorRecoveryType(scope)}; } std::optional TypeChecker::findTablePropertyRespectingMeta(TypeId lhsType, Name name, const Location& location) @@ -1876,6 +1887,7 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprTa std::vector> fieldTypes(expr.items.size); const TableTypeVar* expectedTable = nullptr; + const UnionTypeVar* expectedUnion = nullptr; std::optional expectedIndexType; std::optional expectedIndexResultType; @@ -1894,6 +1906,9 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprTa } } } + else if (FFlag::LuauExpectedTypesOfProperties) + if (const UnionTypeVar* utv = get(follow(*expectedType))) + expectedUnion = utv; } for (size_t i = 0; i < expr.items.size; ++i) @@ -1916,6 +1931,18 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprTa if (auto prop = expectedTable->props.find(key->value.data); prop != expectedTable->props.end()) expectedResultType = prop->second.type; } + else if (FFlag::LuauExpectedTypesOfProperties && expectedUnion) + { + std::vector expectedResultTypes; + for (TypeId expectedOption : expectedUnion) + if (const TableTypeVar* ttv = get(follow(expectedOption))) + if (auto prop = ttv->props.find(key->value.data); prop != ttv->props.end()) + expectedResultTypes.push_back(prop->second.type); + if (expectedResultTypes.size() == 1) + expectedResultType = expectedResultTypes[0]; + else if (expectedResultTypes.size() > 1) + expectedResultType = addType(UnionTypeVar{expectedResultTypes}); + } } else { @@ -1958,21 +1985,22 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprUn { TypeId actualFunctionType = instantiate(scope, *fnt, expr.location); TypePackId arguments = addTypePack({operandType}); - TypePackId retType = freshTypePack(scope); - TypeId expectedFunctionType = addType(FunctionTypeVar(scope->level, arguments, retType)); + TypePackId retTypePack = freshTypePack(scope); + TypeId expectedFunctionType = addType(FunctionTypeVar(scope->level, arguments, retTypePack)); Unifier state = mkUnifier(expr.location); state.tryUnify(expectedFunctionType, actualFunctionType, /*isFunctionCall*/ true); + TypeId retType = first(retTypePack).value_or(nilType); if (!state.errors.empty()) - return {errorType}; + retType = errorRecoveryType(retType); - return {first(retType).value_or(nilType)}; + return {retType}; } reportError(expr.location, GenericError{format("Unary operator '%s' not supported by type '%s'", toString(expr.op).c_str(), toString(operandType).c_str())}); - return {errorType}; + return {errorRecoveryType(scope)}; } reportErrors(tryUnify(numberType, operandType, expr.location)); @@ -1984,7 +2012,7 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprUn operandType = stripFromNilAndReport(operandType, expr.location); if (get(operandType)) - return {errorType}; + return {errorRecoveryType(scope)}; if (get(operandType)) return {numberType}; // Not strictly correct: metatables permit overriding this @@ -2044,7 +2072,7 @@ TypeId TypeChecker::unionOfTypes(TypeId a, TypeId b, const Location& location, b if (unify(a, b, location)) return a; - return errorType; + return errorRecoveryType(anyType); } if (*a == *b) @@ -2166,11 +2194,13 @@ TypeId TypeChecker::checkRelationalOperation( std::optional leftMetatable = isString(lhsType) ? std::nullopt : getMetatable(follow(lhsType)); std::optional rightMetatable = isString(rhsType) ? std::nullopt : getMetatable(follow(rhsType)); + // TODO: this check seems odd, the second part is redundant + // is it meant to be if (leftMetatable && rightMetatable && leftMetatable != rightMetatable) if (bool(leftMetatable) != bool(rightMetatable) && leftMetatable != rightMetatable) { reportError(expr.location, GenericError{format("Types %s and %s cannot be compared with %s because they do not have the same metatable", toString(lhsType).c_str(), toString(rhsType).c_str(), toString(expr.op).c_str())}); - return errorType; + return errorRecoveryType(booleanType); } if (leftMetatable) @@ -2188,7 +2218,7 @@ TypeId TypeChecker::checkRelationalOperation( if (!state.errors.empty()) { reportError(expr.location, GenericError{format("Metamethod '%s' must return type 'boolean'", metamethodName.c_str())}); - return errorType; + return errorRecoveryType(booleanType); } } } @@ -2206,7 +2236,7 @@ TypeId TypeChecker::checkRelationalOperation( { reportError( expr.location, GenericError{format("Table %s does not offer metamethod %s", toString(lhsType).c_str(), metamethodName.c_str())}); - return errorType; + return errorRecoveryType(booleanType); } } @@ -2214,14 +2244,14 @@ TypeId TypeChecker::checkRelationalOperation( { auto name = getIdentifierOfBaseVar(expr.left); reportError(expr.location, CannotInferBinaryOperation{expr.op, name, CannotInferBinaryOperation::Comparison}); - return errorType; + return errorRecoveryType(booleanType); } if (needsMetamethod) { reportError(expr.location, GenericError{format("Type %s cannot be compared with %s because it has no metatable", toString(lhsType).c_str(), toString(expr.op).c_str())}); - return errorType; + return errorRecoveryType(booleanType); } return booleanType; @@ -2266,7 +2296,8 @@ TypeId TypeChecker::checkBinaryOperation( { auto name = getIdentifierOfBaseVar(expr.left); reportError(expr.location, CannotInferBinaryOperation{expr.op, name, CannotInferBinaryOperation::Operation}); - return errorType; + if (!FFlag::LuauErrorRecoveryType) + return errorRecoveryType(scope); } // If we know nothing at all about the lhs type, we can usually say nothing about the result. @@ -2296,18 +2327,33 @@ TypeId TypeChecker::checkBinaryOperation( auto checkMetatableCall = [this, &scope, &expr](TypeId fnt, TypeId lhst, TypeId rhst) -> TypeId { TypeId actualFunctionType = instantiate(scope, fnt, expr.location); TypePackId arguments = addTypePack({lhst, rhst}); - TypePackId retType = freshTypePack(scope); - TypeId expectedFunctionType = addType(FunctionTypeVar(scope->level, arguments, retType)); + TypePackId retTypePack = freshTypePack(scope); + TypeId expectedFunctionType = addType(FunctionTypeVar(scope->level, arguments, retTypePack)); Unifier state = mkUnifier(expr.location); state.tryUnify(expectedFunctionType, actualFunctionType, /*isFunctionCall*/ true); reportErrors(state.errors); + bool hasErrors = !state.errors.empty(); - if (!state.errors.empty()) - return errorType; + if (FFlag::LuauErrorRecoveryType && hasErrors) + { + // If there are unification errors, the return type may still be unknown + // so we loosen the argument types to see if that helps. + TypePackId fallbackArguments = freshTypePack(scope); + TypeId fallbackFunctionType = addType(FunctionTypeVar(scope->level, fallbackArguments, retTypePack)); + state.log.rollback(); + state.errors.clear(); + state.tryUnify(fallbackFunctionType, actualFunctionType, /*isFunctionCall*/ true); + if (!state.errors.empty()) + state.log.rollback(); + } - return first(retType).value_or(nilType); + TypeId retType = first(retTypePack).value_or(nilType); + if (hasErrors) + retType = errorRecoveryType(retType); + + return retType; }; std::string op = opToMetaTableEntry(expr.op); @@ -2321,7 +2367,8 @@ TypeId TypeChecker::checkBinaryOperation( reportError(expr.location, GenericError{format("Binary operator '%s' not supported by types '%s' and '%s'", toString(expr.op).c_str(), toString(lhsType).c_str(), toString(rhsType).c_str())}); - return errorType; + + return errorRecoveryType(scope); } switch (expr.op) @@ -2414,11 +2461,9 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprTy ExprResult result = checkExpr(scope, *expr.expr, annotationType); ErrorVec errorVec = canUnify(result.type, annotationType, expr.location); + reportErrors(errorVec); if (!errorVec.empty()) - { - reportErrors(errorVec); - return {errorType, std::move(result.predicates)}; - } + annotationType = errorRecoveryType(annotationType); return {annotationType, std::move(result.predicates)}; } @@ -2434,7 +2479,7 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprEr // any type errors that may arise from it are going to be useless. currentModule->errors.resize(oldSize); - return {errorType}; + return {errorRecoveryType(scope)}; } ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprIfElse& expr) @@ -2476,7 +2521,7 @@ std::pair TypeChecker::checkLValueBinding(const ScopePtr& scope { for (AstExpr* expr : a->expressions) checkExpr(scope, *expr); - return std::pair(errorType, nullptr); + return {errorRecoveryType(scope), nullptr}; } else ice("Unexpected AST node in checkLValue", expr.location); @@ -2488,7 +2533,7 @@ std::pair TypeChecker::checkLValueBinding(const ScopePtr& scope return {*ty, nullptr}; reportError(expr.location, UnknownSymbol{expr.local->name.value, UnknownSymbol::Binding}); - return {errorType, nullptr}; + return {errorRecoveryType(scope), nullptr}; } std::pair TypeChecker::checkLValueBinding(const ScopePtr& scope, const AstExprGlobal& expr) @@ -2545,24 +2590,25 @@ std::pair TypeChecker::checkLValueBinding(const ScopePtr& scope { Unifier state = mkUnifier(expr.location); state.tryUnify(indexer->indexType, stringType); + TypeId retType = indexer->indexResultType; if (!state.errors.empty()) { state.log.rollback(); reportError(expr.location, UnknownProperty{lhs, name}); - return std::pair(errorType, nullptr); + retType = errorRecoveryType(retType); } - return std::pair(indexer->indexResultType, nullptr); + return std::pair(retType, nullptr); } else if (lhsTable->state == TableState::Sealed) { reportError(TypeError{expr.location, CannotExtendTable{lhs, CannotExtendTable::Property, name}}); - return std::pair(errorType, nullptr); + return std::pair(errorRecoveryType(scope), nullptr); } else { reportError(TypeError{expr.location, GenericError{"Internal error: generic tables are not lvalues"}}); - return std::pair(errorType, nullptr); + return std::pair(errorRecoveryType(scope), nullptr); } } else if (const ClassTypeVar* lhsClass = get(lhs)) @@ -2571,7 +2617,7 @@ std::pair TypeChecker::checkLValueBinding(const ScopePtr& scope if (!prop) { reportError(TypeError{expr.location, UnknownProperty{lhs, name}}); - return std::pair(errorType, nullptr); + return std::pair(errorRecoveryType(scope), nullptr); } return std::pair(prop->type, nullptr); @@ -2585,12 +2631,12 @@ std::pair TypeChecker::checkLValueBinding(const ScopePtr& scope if (isTableIntersection(lhs)) { reportError(TypeError{expr.location, CannotExtendTable{lhs, CannotExtendTable::Property, name}}); - return std::pair(errorType, nullptr); + return std::pair(errorRecoveryType(scope), nullptr); } } reportError(TypeError{expr.location, NotATable{lhs}}); - return std::pair(errorType, nullptr); + return std::pair(errorRecoveryType(scope), nullptr); } std::pair TypeChecker::checkLValueBinding(const ScopePtr& scope, const AstExprIndexExpr& expr) @@ -2615,7 +2661,7 @@ std::pair TypeChecker::checkLValueBinding(const ScopePtr& scope if (!prop) { reportError(TypeError{expr.location, UnknownProperty{exprType, value->value.data}}); - return std::pair(errorType, nullptr); + return std::pair(errorRecoveryType(scope), nullptr); } return std::pair(prop->type, nullptr); } @@ -2626,7 +2672,7 @@ std::pair TypeChecker::checkLValueBinding(const ScopePtr& scope if (!exprTable) { reportError(TypeError{expr.expr->location, NotATable{exprType}}); - return std::pair(errorType, nullptr); + return std::pair(errorRecoveryType(scope), nullptr); } if (value) @@ -2678,7 +2724,7 @@ TypeId TypeChecker::checkFunctionName(const ScopePtr& scope, AstExpr& funName) if (isNonstrictMode()) return globalScope->bindings[name].typeId; - return errorType; + return errorRecoveryType(scope); } else { @@ -2705,20 +2751,21 @@ TypeId TypeChecker::checkFunctionName(const ScopePtr& scope, AstExpr& funName) TableTypeVar* ttv = getMutableTableType(lhsType); if (!ttv) { - if (!isTableIntersection(lhsType)) + if (!FFlag::LuauErrorRecoveryType && !isTableIntersection(lhsType)) + // This error now gets reported when we check the function body. reportError(TypeError{funName.location, OnlyTablesCanHaveMethods{lhsType}}); - return errorType; + return errorRecoveryType(scope); } // Cannot extend sealed table, but we dont report an error here because it will be reported during AstStatFunction check if (lhsType->persistent || ttv->state == TableState::Sealed) - return errorType; + return errorRecoveryType(scope); Name name = indexName->index.value; if (ttv->props.count(name)) - return errorType; + return errorRecoveryType(scope); Property& property = ttv->props[name]; @@ -2728,9 +2775,7 @@ TypeId TypeChecker::checkFunctionName(const ScopePtr& scope, AstExpr& funName) return property.type; } else if (funName.is()) - { - return errorType; - } + return errorRecoveryType(scope); else { ice("Unexpected AST node type", funName.location); @@ -2991,7 +3036,7 @@ ExprResult TypeChecker::checkExprPack(const ScopePtr& scope, const A else if (expr.is()) { if (!scope->varargPack) - return {addTypePack({addType(ErrorTypeVar())})}; + return {errorRecoveryTypePack(scope)}; return {*scope->varargPack}; } @@ -3095,10 +3140,9 @@ void TypeChecker::checkArgumentList( if (get(tail)) { // Unify remaining parameters so we don't leave any free-types hanging around. - TypeId argTy = errorType; while (paramIter != endIter) { - state.tryUnify(*paramIter, argTy); + state.tryUnify(*paramIter, errorRecoveryType(anyType)); ++paramIter; } return; @@ -3157,7 +3201,7 @@ void TypeChecker::checkArgumentList( { while (argIter != endIter) { - unify(*argIter, errorType, state.location); + unify(*argIter, errorRecoveryType(scope), state.location); ++argIter; } // For this case, we want the error span to cover every errant extra parameter @@ -3246,7 +3290,8 @@ ExprResult TypeChecker::checkExprPack(const ScopePtr& scope, const A // For each overload // Compare parameter and argument types // Report any errors (also speculate dot vs colon warnings!) - // If there are no errors, return the resulting return type + // Return the resulting return type (even if there are errors) + // If there are no matching overloads, unify with (a...) -> (b...) and return b... TypeId selfType = nullptr; TypeId functionType = nullptr; @@ -3268,8 +3313,8 @@ ExprResult TypeChecker::checkExprPack(const ScopePtr& scope, const A } else { - functionType = errorType; - actualFunctionType = errorType; + functionType = errorRecoveryType(scope); + actualFunctionType = functionType; } } else @@ -3296,7 +3341,7 @@ ExprResult TypeChecker::checkExprPack(const ScopePtr& scope, const A TypePackId argPack = argListResult.type; if (get(argPack)) - return ExprResult{errorTypePack}; + return {errorRecoveryTypePack(scope)}; TypePack* args = getMutable(argPack); LUAU_ASSERT(args != nullptr); @@ -3314,19 +3359,34 @@ ExprResult TypeChecker::checkExprPack(const ScopePtr& scope, const A std::vector errors; // errors encountered for each overload std::vector overloadsThatMatchArgCount; + std::vector overloadsThatDont; for (TypeId fn : overloads) { fn = follow(fn); - if (auto ret = checkCallOverload(scope, expr, fn, retPack, argPack, args, argLocations, argListResult, overloadsThatMatchArgCount, errors)) + if (auto ret = checkCallOverload( + scope, expr, fn, retPack, argPack, args, argLocations, argListResult, overloadsThatMatchArgCount, overloadsThatDont, errors)) return *ret; } if (handleSelfCallMismatch(scope, expr, args, argLocations, errors)) return {retPack}; - return reportOverloadResolutionError(scope, expr, retPack, argPack, argLocations, overloads, overloadsThatMatchArgCount, errors); + reportOverloadResolutionError(scope, expr, retPack, argPack, argLocations, overloads, overloadsThatMatchArgCount, errors); + + if (FFlag::LuauErrorRecoveryType) + { + const FunctionTypeVar* overload = nullptr; + if (!overloadsThatMatchArgCount.empty()) + overload = get(overloadsThatMatchArgCount[0]); + if (!overload && !overloadsThatDont.empty()) + overload = get(overloadsThatDont[0]); + if (overload) + return {errorRecoveryTypePack(overload->retType)}; + } + + return {errorRecoveryTypePack(retPack)}; } std::vector> TypeChecker::getExpectedTypesForCall(const std::vector& overloads, size_t argumentCount, bool selfCall) @@ -3382,7 +3442,7 @@ std::vector> TypeChecker::getExpectedTypesForCall(const st std::optional> TypeChecker::checkCallOverload(const ScopePtr& scope, const AstExprCall& expr, TypeId fn, TypePackId retPack, TypePackId argPack, TypePack* args, const std::vector& argLocations, const ExprResult& argListResult, - std::vector& overloadsThatMatchArgCount, std::vector& errors) + std::vector& overloadsThatMatchArgCount, std::vector& overloadsThatDont, std::vector& errors) { fn = stripFromNilAndReport(fn, expr.func->location); @@ -3394,7 +3454,7 @@ std::optional> TypeChecker::checkCallOverload(const Scope if (get(fn)) { - return {{addTypePack(TypePackVar{Unifiable::Error{}})}}; + return {{errorRecoveryTypePack(scope)}}; } if (get(fn)) @@ -3427,14 +3487,14 @@ std::optional> TypeChecker::checkCallOverload(const Scope TypeId fn = *ty; fn = instantiate(scope, fn, expr.func->location); - return checkCallOverload( - scope, expr, fn, retPack, metaCallArgPack, metaCallArgs, metaArgLocations, argListResult, overloadsThatMatchArgCount, errors); + return checkCallOverload(scope, expr, fn, retPack, metaCallArgPack, metaCallArgs, metaArgLocations, argListResult, + overloadsThatMatchArgCount, overloadsThatDont, errors); } } reportError(TypeError{expr.func->location, CannotCallNonFunction{fn}}); - unify(retPack, errorTypePack, expr.func->location); - return {{errorTypePack}}; + unify(retPack, errorRecoveryTypePack(scope), expr.func->location); + return {{errorRecoveryTypePack(retPack)}}; } // When this function type has magic functions and did return something, we select that overload instead. @@ -3476,6 +3536,8 @@ std::optional> TypeChecker::checkCallOverload(const Scope if (!argMismatch) overloadsThatMatchArgCount.push_back(fn); + else if (FFlag::LuauErrorRecoveryType) + overloadsThatDont.push_back(fn); errors.emplace_back(std::move(state.errors), args->head, ftv); state.log.rollback(); @@ -3586,14 +3648,14 @@ bool TypeChecker::handleSelfCallMismatch(const ScopePtr& scope, const AstExprCal return false; } -ExprResult TypeChecker::reportOverloadResolutionError(const ScopePtr& scope, const AstExprCall& expr, TypePackId retPack, - TypePackId argPack, const std::vector& argLocations, const std::vector& overloads, - const std::vector& overloadsThatMatchArgCount, const std::vector& errors) +void TypeChecker::reportOverloadResolutionError(const ScopePtr& scope, const AstExprCall& expr, TypePackId retPack, TypePackId argPack, + const std::vector& argLocations, const std::vector& overloads, const std::vector& overloadsThatMatchArgCount, + const std::vector& errors) { if (overloads.size() == 1) { reportErrors(std::get<0>(errors.front())); - return {errorTypePack}; + return; } std::vector overloadTypes = overloadsThatMatchArgCount; @@ -3622,7 +3684,7 @@ ExprResult TypeChecker::reportOverloadResolutionError(const ScopePtr // If only one overload matched, we don't need this error because we provided the previous errors. if (overloadsThatMatchArgCount.size() == 1) - return {errorTypePack}; + return; } std::string s; @@ -3655,7 +3717,7 @@ ExprResult TypeChecker::reportOverloadResolutionError(const ScopePtr reportError(expr.func->location, ExtraInformation{"Other overloads are also not viable: " + s}); // No viable overload - return {errorTypePack}; + return; } ExprResult TypeChecker::checkExprList(const ScopePtr& scope, const Location& location, const AstArray& exprs, @@ -3740,7 +3802,7 @@ TypeId TypeChecker::checkRequire(const ScopePtr& scope, const ModuleInfo& module if (FFlag::LuauStrictRequire && currentModule->mode == Mode::Strict) { reportError(TypeError{location, UnknownRequire{}}); - return errorType; + return errorRecoveryType(anyType); } return anyType; @@ -3758,14 +3820,14 @@ TypeId TypeChecker::checkRequire(const ScopePtr& scope, const ModuleInfo& module reportError(TypeError{location, UnknownRequire{reportedModulePath}}); } - return errorType; + return errorRecoveryType(scope); } if (module->type != SourceCode::Module) { std::string humanReadableName = resolver->getHumanReadableModuleName(moduleInfo.name); reportError(location, IllegalRequire{humanReadableName, "Module is not a ModuleScript. It cannot be required."}); - return errorType; + return errorRecoveryType(scope); } std::optional moduleType = first(module->getModuleScope()->returnType); @@ -3773,7 +3835,7 @@ TypeId TypeChecker::checkRequire(const ScopePtr& scope, const ModuleInfo& module { std::string humanReadableName = resolver->getHumanReadableModuleName(moduleInfo.name); reportError(location, IllegalRequire{humanReadableName, "Module does not return exactly 1 value. It cannot be required."}); - return errorType; + return errorRecoveryType(scope); } SeenTypes seenTypes; @@ -4078,7 +4140,7 @@ TypeId TypeChecker::quantify(const ScopePtr& scope, TypeId ty, Location location if (!qty.has_value()) { reportError(location, UnificationTooComplex{}); - return errorType; + return errorRecoveryType(scope); } if (ty == *qty) @@ -4101,7 +4163,7 @@ TypeId TypeChecker::instantiate(const ScopePtr& scope, TypeId ty, Location locat else { reportError(location, UnificationTooComplex{}); - return errorType; + return errorRecoveryType(scope); } } @@ -4116,7 +4178,7 @@ TypeId TypeChecker::anyify(const ScopePtr& scope, TypeId ty, Location location) else { reportError(location, UnificationTooComplex{}); - return errorType; + return errorRecoveryType(anyType); } } @@ -4131,7 +4193,7 @@ TypePackId TypeChecker::anyify(const ScopePtr& scope, TypePackId ty, Location lo else { reportError(location, UnificationTooComplex{}); - return errorTypePack; + return errorRecoveryTypePack(anyTypePack); } } @@ -4279,6 +4341,38 @@ TypeId TypeChecker::freshType(TypeLevel level) return currentModule->internalTypes.addType(TypeVar(FreeTypeVar(level))); } +TypeId TypeChecker::singletonType(bool value) +{ + // TODO: cache singleton types + return currentModule->internalTypes.addType(TypeVar(SingletonTypeVar(BoolSingleton{value}))); +} + +TypeId TypeChecker::singletonType(std::string value) +{ + // TODO: cache singleton types + return currentModule->internalTypes.addType(TypeVar(SingletonTypeVar(StringSingleton{std::move(value)}))); +} + +TypeId TypeChecker::errorRecoveryType(const ScopePtr& scope) +{ + return singletonTypes.errorRecoveryType(); +} + +TypeId TypeChecker::errorRecoveryType(TypeId guess) +{ + return singletonTypes.errorRecoveryType(guess); +} + +TypePackId TypeChecker::errorRecoveryTypePack(const ScopePtr& scope) +{ + return singletonTypes.errorRecoveryTypePack(); +} + +TypePackId TypeChecker::errorRecoveryTypePack(TypePackId guess) +{ + return singletonTypes.errorRecoveryTypePack(guess); +} + std::optional TypeChecker::filterMap(TypeId type, TypeIdPredicate predicate) { std::vector types = Luau::filterMap(type, predicate); @@ -4350,7 +4444,7 @@ TypeId TypeChecker::resolveType(const ScopePtr& scope, const AstType& annotation if (lit->parameters.size != 1 || !lit->parameters.data[0].type) { reportError(TypeError{annotation.location, GenericError{"_luau_print requires one generic parameter"}}); - return addType(ErrorTypeVar{}); + return errorRecoveryType(anyType); } ToStringOptions opts; @@ -4368,7 +4462,7 @@ TypeId TypeChecker::resolveType(const ScopePtr& scope, const AstType& annotation if (!tf) { if (lit->name == Parser::errorName) - return addType(ErrorTypeVar{}); + return errorRecoveryType(scope); std::string typeName; if (lit->hasPrefix) @@ -4380,7 +4474,7 @@ TypeId TypeChecker::resolveType(const ScopePtr& scope, const AstType& annotation else reportError(TypeError{annotation.location, UnknownSymbol{typeName, UnknownSymbol::Type}}); - return addType(ErrorTypeVar{}); + return errorRecoveryType(scope); } if (lit->parameters.size == 0 && tf->typeParams.empty() && (!FFlag::LuauTypeAliasPacks || tf->typePackParams.empty())) @@ -4390,14 +4484,17 @@ TypeId TypeChecker::resolveType(const ScopePtr& scope, const AstType& annotation else if (!FFlag::LuauTypeAliasPacks && lit->parameters.size != tf->typeParams.size()) { reportError(TypeError{annotation.location, IncorrectGenericParameterCount{lit->name.value, *tf, lit->parameters.size, 0}}); - return addType(ErrorTypeVar{}); + if (!FFlag::LuauErrorRecoveryType) + return errorRecoveryType(scope); } - else if (FFlag::LuauTypeAliasPacks) + + if (FFlag::LuauTypeAliasPacks) { if (!lit->hasParameterList && !tf->typePackParams.empty()) { reportError(TypeError{annotation.location, GenericError{"Type parameter list is required"}}); - return addType(ErrorTypeVar{}); + if (!FFlag::LuauErrorRecoveryType) + return errorRecoveryType(scope); } std::vector typeParams; @@ -4445,7 +4542,17 @@ TypeId TypeChecker::resolveType(const ScopePtr& scope, const AstType& annotation { reportError( TypeError{annotation.location, IncorrectGenericParameterCount{lit->name.value, *tf, typeParams.size(), typePackParams.size()}}); - return addType(ErrorTypeVar{}); + + if (FFlag::LuauErrorRecoveryType) + { + // Pad the types out with error recovery types + while (typeParams.size() < tf->typeParams.size()) + typeParams.push_back(errorRecoveryType(scope)); + while (typePackParams.size() < tf->typePackParams.size()) + typePackParams.push_back(errorRecoveryTypePack(scope)); + } + else + return errorRecoveryType(scope); } if (FFlag::LuauRecursiveTypeParameterRestriction && typeParams == tf->typeParams && typePackParams == tf->typePackParams) @@ -4464,6 +4571,14 @@ TypeId TypeChecker::resolveType(const ScopePtr& scope, const AstType& annotation for (const auto& param : lit->parameters) typeParams.push_back(resolveType(scope, *param.type)); + if (FFlag::LuauErrorRecoveryType) + { + // If there aren't enough type parameters, pad them out with error recovery types + // (we've already reported the error) + while (typeParams.size() < lit->parameters.size) + typeParams.push_back(errorRecoveryType(scope)); + } + if (FFlag::LuauRecursiveTypeParameterRestriction && typeParams == tf->typeParams) { // If the generic parameters and the type arguments are the same, we are about to @@ -4483,8 +4598,7 @@ TypeId TypeChecker::resolveType(const ScopePtr& scope, const AstType& annotation props[prop.name.value] = {resolveType(scope, *prop.type)}; if (const auto& indexer = table->indexer) - tableIndexer = TableIndexer( - resolveType(scope, *indexer->indexType), resolveType(scope, *indexer->resultType)); + tableIndexer = TableIndexer(resolveType(scope, *indexer->indexType), resolveType(scope, *indexer->resultType)); return addType(TableTypeVar{ props, tableIndexer, scope->level, @@ -4536,14 +4650,20 @@ TypeId TypeChecker::resolveType(const ScopePtr& scope, const AstType& annotation return addType(IntersectionTypeVar{types}); } - else if (annotation.is()) + else if (const auto& tsb = annotation.as()) { - return addType(ErrorTypeVar{}); + return singletonType(tsb->value); } + else if (const auto& tss = annotation.as()) + { + return singletonType(std::string(tss->value.data, tss->value.size)); + } + else if (annotation.is()) + return errorRecoveryType(scope); else { reportError(TypeError{annotation.location, GenericError{"Unknown type annotation?"}}); - return addType(ErrorTypeVar{}); + return errorRecoveryType(scope); } } @@ -4584,7 +4704,7 @@ TypePackId TypeChecker::resolveTypePack(const ScopePtr& scope, const AstTypePack else reportError(TypeError{generic->location, UnknownSymbol{genericName, UnknownSymbol::Type}}); - return addTypePack(TypePackVar{Unifiable::Error{}}); + return errorRecoveryTypePack(scope); } return *genericTy; @@ -4706,12 +4826,12 @@ TypeId TypeChecker::instantiateTypeFun(const ScopePtr& scope, const TypeFun& tf, if (!maybeInstantiated.has_value()) { reportError(location, UnificationTooComplex{}); - return errorType; + return errorRecoveryType(scope); } if (FFlag::LuauRecursiveTypeParameterRestriction && applyTypeFunction.encounteredForwardedType) { reportError(TypeError{location, GenericError{"Recursive type being used with different parameters"}}); - return errorType; + return errorRecoveryType(scope); } TypeId instantiated = *maybeInstantiated; @@ -4773,8 +4893,8 @@ TypeId TypeChecker::instantiateTypeFun(const ScopePtr& scope, const TypeFun& tf, return instantiated; } -std::pair, std::vector> TypeChecker::createGenericTypes( - const ScopePtr& scope, std::optional levelOpt, const AstNode& node, const AstArray& genericNames, const AstArray& genericPackNames) +std::pair, std::vector> TypeChecker::createGenericTypes(const ScopePtr& scope, std::optional levelOpt, + const AstNode& node, const AstArray& genericNames, const AstArray& genericPackNames) { LUAU_ASSERT(scope->parent); @@ -5043,7 +5163,7 @@ void TypeChecker::resolve(const IsAPredicate& isaP, ErrorVec& errVec, Refinement addRefinement(refis, isaP.lvalue, *result); else { - addRefinement(refis, isaP.lvalue, errorType); + addRefinement(refis, isaP.lvalue, errorRecoveryType(scope)); errVec.push_back(TypeError{isaP.location, TypeMismatch{isaP.ty, *ty}}); } } @@ -5107,7 +5227,7 @@ void TypeChecker::resolve(const TypeGuardPredicate& typeguardP, ErrorVec& errVec addRefinement(refis, typeguardP.lvalue, *result); else { - addRefinement(refis, typeguardP.lvalue, errorType); + addRefinement(refis, typeguardP.lvalue, errorRecoveryType(scope)); if (sense) errVec.push_back( TypeError{typeguardP.location, GenericError{"Type '" + toString(*ty) + "' has no overlap with '" + typeguardP.kind + "'"}}); @@ -5118,7 +5238,7 @@ void TypeChecker::resolve(const TypeGuardPredicate& typeguardP, ErrorVec& errVec auto fail = [&](const TypeErrorData& err) { errVec.push_back(TypeError{typeguardP.location, err}); - addRefinement(refis, typeguardP.lvalue, errorType); + addRefinement(refis, typeguardP.lvalue, errorRecoveryType(scope)); }; if (!typeguardP.isTypeof) @@ -5139,28 +5259,6 @@ void TypeChecker::resolve(const TypeGuardPredicate& typeguardP, ErrorVec& errVec return resolve(IsAPredicate{std::move(typeguardP.lvalue), typeguardP.location, type}, errVec, refis, scope, sense); } -void TypeChecker::DEPRECATED_resolve(const TypeGuardPredicate& typeguardP, ErrorVec& errVec, RefinementMap& refis, const ScopePtr& scope, bool sense) -{ - if (!sense) - return; - - static std::vector primitives{ - "string", "number", "boolean", "nil", "thread", - "table", // no op. Requires special handling. - "function", // no op. Requires special handling. - "userdata", // no op. Requires special handling. - }; - - if (auto typeFun = globalScope->lookupType(typeguardP.kind); - typeFun && typeFun->typeParams.empty() && (!FFlag::LuauTypeAliasPacks || typeFun->typePackParams.empty())) - { - if (auto it = std::find(primitives.begin(), primitives.end(), typeguardP.kind); it != primitives.end()) - addRefinement(refis, typeguardP.lvalue, typeFun->type); - else if (typeguardP.isTypeof) - addRefinement(refis, typeguardP.lvalue, typeFun->type); - } -} - void TypeChecker::resolve(const EqPredicate& eqP, ErrorVec& errVec, RefinementMap& refis, const ScopePtr& scope, bool sense) { // This refinement will require success typing to do everything correctly. For now, we can get most of the way there. diff --git a/Analysis/src/TypePack.cpp b/Analysis/src/TypePack.cpp index 228b1926..d3221c73 100644 --- a/Analysis/src/TypePack.cpp +++ b/Analysis/src/TypePack.cpp @@ -286,5 +286,4 @@ TypePack* asMutable(const TypePack* tp) { return const_cast(tp); } - } // namespace Luau diff --git a/Analysis/src/TypeVar.cpp b/Analysis/src/TypeVar.cpp index 4ee5cb82..924bf082 100644 --- a/Analysis/src/TypeVar.cpp +++ b/Analysis/src/TypeVar.cpp @@ -21,6 +21,7 @@ LUAU_FASTINTVARIABLE(LuauTypeMaximumStringifierLength, 500) LUAU_FASTINTVARIABLE(LuauTableTypeMaximumStringifierLength, 0) LUAU_FASTFLAG(LuauTypeAliasPacks) LUAU_FASTFLAGVARIABLE(LuauRefactorTagging, false) +LUAU_FASTFLAG(LuauErrorRecoveryType) namespace Luau { @@ -305,6 +306,18 @@ bool maybeGeneric(TypeId ty) return isGeneric(ty); } +bool maybeSingleton(TypeId ty) +{ + ty = follow(ty); + if (get(ty)) + return true; + if (const UnionTypeVar* utv = get(ty)) + for (TypeId option : utv) + if (get(follow(option))) + return true; + return false; +} + FunctionTypeVar::FunctionTypeVar(TypePackId argTypes, TypePackId retType, std::optional defn, bool hasSelf) : argTypes(argTypes) , retType(retType) @@ -562,10 +575,8 @@ SingletonTypes::SingletonTypes() , booleanType(&booleanType_) , threadType(&threadType_) , anyType(&anyType_) - , errorType(&errorType_) , optionalNumberType(&optionalNumberType_) , anyTypePack(&anyTypePack_) - , errorTypePack(&errorTypePack_) , arena(new TypeArena) { TypeId stringMetatable = makeStringMetatable(); @@ -634,6 +645,32 @@ TypeId SingletonTypes::makeStringMetatable() return arena->addType(TableTypeVar{{{{"__index", {tableType}}}}, std::nullopt, TypeLevel{}, TableState::Sealed}); } +TypeId SingletonTypes::errorRecoveryType() +{ + return &errorType_; +} + +TypePackId SingletonTypes::errorRecoveryTypePack() +{ + return &errorTypePack_; +} + +TypeId SingletonTypes::errorRecoveryType(TypeId guess) +{ + if (FFlag::LuauErrorRecoveryType) + return guess; + else + return &errorType_; +} + +TypePackId SingletonTypes::errorRecoveryTypePack(TypePackId guess) +{ + if (FFlag::LuauErrorRecoveryType) + return guess; + else + return &errorTypePack_; +} + SingletonTypes singletonTypes; void persist(TypeId ty) @@ -1141,6 +1178,11 @@ struct QVarFinder return false; } + bool operator()(const SingletonTypeVar&) const + { + return false; + } + bool operator()(const FunctionTypeVar& ftv) const { if (hasGeneric(ftv.argTypes)) @@ -1412,7 +1454,7 @@ static std::vector parseFormatString(TypeChecker& typechecker, const cha else if (strchr(options, data[i])) result.push_back(typechecker.numberType); else - result.push_back(typechecker.errorType); + result.push_back(typechecker.errorRecoveryType(typechecker.anyType)); } } diff --git a/Analysis/src/Unifier.cpp b/Analysis/src/Unifier.cpp index 82f621b6..e1a52be4 100644 --- a/Analysis/src/Unifier.cpp +++ b/Analysis/src/Unifier.cpp @@ -22,7 +22,9 @@ LUAU_FASTFLAGVARIABLE(LuauTypecheckOpts, false) LUAU_FASTFLAG(LuauShareTxnSeen); LUAU_FASTFLAGVARIABLE(LuauCacheUnifyTableResults, false) LUAU_FASTFLAGVARIABLE(LuauExtendedTypeMismatchError, false) +LUAU_FASTFLAG(LuauSingletonTypes) LUAU_FASTFLAGVARIABLE(LuauExtendedClassMismatchError, false) +LUAU_FASTFLAG(LuauErrorRecoveryType); namespace Luau { @@ -211,6 +213,7 @@ void Unifier::tryUnify_(TypeId superTy, TypeId subTy, bool isFunctionCall, bool { occursCheck(subTy, superTy); + // The occurrence check might have caused superTy no longer to be a free type if (!get(subTy)) { log(subTy); @@ -221,10 +224,20 @@ void Unifier::tryUnify_(TypeId superTy, TypeId subTy, bool isFunctionCall, bool } else if (l && r) { - log(superTy); + if (!FFlag::LuauErrorRecoveryType) + log(superTy); occursCheck(superTy, subTy); r->level = min(r->level, l->level); - *asMutable(superTy) = BoundTypeVar(subTy); + + // The occurrence check might have caused superTy no longer to be a free type + if (!FFlag::LuauErrorRecoveryType) + *asMutable(superTy) = BoundTypeVar(subTy); + else if (!get(superTy)) + { + log(superTy); + *asMutable(superTy) = BoundTypeVar(subTy); + } + return; } else if (l) @@ -240,6 +253,7 @@ void Unifier::tryUnify_(TypeId superTy, TypeId subTy, bool isFunctionCall, bool return; } + // The occurrence check might have caused superTy no longer to be a free type if (!get(superTy)) { if (auto rightLevel = getMutableLevel(subTy)) @@ -251,6 +265,7 @@ void Unifier::tryUnify_(TypeId superTy, TypeId subTy, bool isFunctionCall, bool log(superTy); *asMutable(superTy) = BoundTypeVar(subTy); } + return; } else if (r) @@ -512,6 +527,9 @@ void Unifier::tryUnify_(TypeId superTy, TypeId subTy, bool isFunctionCall, bool else if (get(superTy) && get(subTy)) tryUnifyPrimitives(superTy, subTy); + else if (FFlag::LuauSingletonTypes && (get(superTy) || get(superTy)) && get(subTy)) + tryUnifySingletons(superTy, subTy); + else if (get(superTy) && get(subTy)) tryUnifyFunctions(superTy, subTy, isFunctionCall); @@ -723,17 +741,18 @@ void Unifier::tryUnify_(TypePackId superTp, TypePackId subTp, bool isFunctionCal { occursCheck(superTp, subTp); + // The occurrence check might have caused superTp no longer to be a free type if (!get(superTp)) { log(superTp); *asMutable(superTp) = Unifiable::Bound(subTp); } } - else if (get(subTp)) { occursCheck(subTp, superTp); + // The occurrence check might have caused superTp no longer to be a free type if (!get(subTp)) { log(subTp); @@ -874,13 +893,13 @@ void Unifier::tryUnify_(TypePackId superTp, TypePackId subTp, bool isFunctionCal while (superIter.good()) { - tryUnify_(singletonTypes.errorType, *superIter); + tryUnify_(singletonTypes.errorRecoveryType(), *superIter); superIter.advance(); } while (subIter.good()) { - tryUnify_(singletonTypes.errorType, *subIter); + tryUnify_(singletonTypes.errorRecoveryType(), *subIter); subIter.advance(); } @@ -906,6 +925,27 @@ void Unifier::tryUnifyPrimitives(TypeId superTy, TypeId subTy) errors.push_back(TypeError{location, TypeMismatch{superTy, subTy}}); } +void Unifier::tryUnifySingletons(TypeId superTy, TypeId subTy) +{ + const PrimitiveTypeVar* lp = get(superTy); + const SingletonTypeVar* ls = get(superTy); + const SingletonTypeVar* rs = get(subTy); + + if ((!lp && !ls) || !rs) + ice("passed non singleton/primitive types to unifySingletons"); + + if (ls && *ls == *rs) + return; + + if (lp && lp->type == PrimitiveTypeVar::Boolean && get(rs) && variance == Covariant) + return; + + if (lp && lp->type == PrimitiveTypeVar::String && get(rs) && variance == Covariant) + return; + + errors.push_back(TypeError{location, TypeMismatch{superTy, subTy}}); +} + void Unifier::tryUnifyFunctions(TypeId superTy, TypeId subTy, bool isFunctionCall) { FunctionTypeVar* lf = getMutable(superTy); @@ -1023,7 +1063,8 @@ void Unifier::tryUnifyTables(TypeId left, TypeId right, bool isIntersection) } // And vice versa if we're invariant - if (FFlag::LuauTableUnificationEarlyTest && variance == Invariant && !lt->indexer && lt->state != TableState::Unsealed && lt->state != TableState::Free) + if (FFlag::LuauTableUnificationEarlyTest && variance == Invariant && !lt->indexer && lt->state != TableState::Unsealed && + lt->state != TableState::Free) { for (const auto& [propName, subProp] : rt->props) { @@ -1038,7 +1079,7 @@ void Unifier::tryUnifyTables(TypeId left, TypeId right, bool isIntersection) return; } } - + // Reminder: left is the supertype, right is the subtype. // Width subtyping: any property in the supertype must be in the subtype, // and the types must agree. @@ -1634,9 +1675,8 @@ void Unifier::tryUnifyWithClass(TypeId superTy, TypeId subTy, bool reversed) { ok = false; errors.push_back(TypeError{location, UnknownProperty{superTy, propName}}); - if (!FFlag::LuauExtendedClassMismatchError) - tryUnify_(prop.type, singletonTypes.errorType); + tryUnify_(prop.type, singletonTypes.errorRecoveryType()); } else { @@ -1952,7 +1992,7 @@ void Unifier::tryUnifyWithAny(TypePackId any, TypePackId ty) { LUAU_ASSERT(get(any)); - const TypeId anyTy = singletonTypes.errorType; + const TypeId anyTy = singletonTypes.errorRecoveryType(); if (FFlag::LuauTypecheckOpts) { @@ -2046,7 +2086,7 @@ void Unifier::occursCheck(std::unordered_set& seen_DEPRECATED, DenseHash { errors.push_back(TypeError{location, OccursCheckFailed{}}); log(needle); - *asMutable(needle) = ErrorTypeVar{}; + *asMutable(needle) = *singletonTypes.errorRecoveryType(); return; } @@ -2134,7 +2174,7 @@ void Unifier::occursCheck(std::unordered_set& seen_DEPRECATED, Dense { errors.push_back(TypeError{location, OccursCheckFailed{}}); log(needle); - *asMutable(needle) = ErrorTypeVar{}; + *asMutable(needle) = *singletonTypes.errorRecoveryTypePack(); return; } diff --git a/Ast/include/Luau/Ast.h b/Ast/include/Luau/Ast.h index a2189f7b..5b4bfa03 100644 --- a/Ast/include/Luau/Ast.h +++ b/Ast/include/Luau/Ast.h @@ -255,6 +255,14 @@ public: { return visit((class AstType*)node); } + virtual bool visit(class AstTypeSingletonBool* node) + { + return visit((class AstType*)node); + } + virtual bool visit(class AstTypeSingletonString* node) + { + return visit((class AstType*)node); + } virtual bool visit(class AstTypeError* node) { return visit((class AstType*)node); @@ -1158,6 +1166,30 @@ public: unsigned messageIndex; }; +class AstTypeSingletonBool : public AstType +{ +public: + LUAU_RTTI(AstTypeSingletonBool) + + AstTypeSingletonBool(const Location& location, bool value); + + void visit(AstVisitor* visitor) override; + + bool value; +}; + +class AstTypeSingletonString : public AstType +{ +public: + LUAU_RTTI(AstTypeSingletonString) + + AstTypeSingletonString(const Location& location, const AstArray& value); + + void visit(AstVisitor* visitor) override; + + const AstArray value; +}; + class AstTypePack : public AstNode { public: diff --git a/Ast/include/Luau/Parser.h b/Ast/include/Luau/Parser.h index 39c7d925..87ebc48b 100644 --- a/Ast/include/Luau/Parser.h +++ b/Ast/include/Luau/Parser.h @@ -286,6 +286,7 @@ private: // `<' typeAnnotation[, ...] `>' AstArray parseTypeParams(); + std::optional> parseCharArray(); AstExpr* parseString(); AstLocal* pushLocal(const Binding& binding); diff --git a/Ast/include/Luau/StringUtils.h b/Ast/include/Luau/StringUtils.h index 4f7673fa..6ecf0606 100644 --- a/Ast/include/Luau/StringUtils.h +++ b/Ast/include/Luau/StringUtils.h @@ -34,4 +34,6 @@ bool equalsLower(std::string_view lhs, std::string_view rhs); size_t hashRange(const char* data, size_t size); +std::string escape(std::string_view s); +bool isIdentifier(std::string_view s); } // namespace Luau diff --git a/Ast/src/Ast.cpp b/Ast/src/Ast.cpp index b1209faa..e709894d 100644 --- a/Ast/src/Ast.cpp +++ b/Ast/src/Ast.cpp @@ -841,6 +841,28 @@ void AstTypeIntersection::visit(AstVisitor* visitor) } } +AstTypeSingletonBool::AstTypeSingletonBool(const Location& location, bool value) + : AstType(ClassIndex(), location) + , value(value) +{ +} + +void AstTypeSingletonBool::visit(AstVisitor* visitor) +{ + visitor->visit(this); +} + +AstTypeSingletonString::AstTypeSingletonString(const Location& location, const AstArray& value) + : AstType(ClassIndex(), location) + , value(value) +{ +} + +void AstTypeSingletonString::visit(AstVisitor* visitor) +{ + visitor->visit(this); +} + AstTypeError::AstTypeError(const Location& location, const AstArray& types, bool isMissing, unsigned messageIndex) : AstType(ClassIndex(), location) , types(types) diff --git a/Ast/src/Parser.cpp b/Ast/src/Parser.cpp index a1bad65e..bc63e37d 100644 --- a/Ast/src/Parser.cpp +++ b/Ast/src/Parser.cpp @@ -16,6 +16,7 @@ LUAU_FASTFLAGVARIABLE(LuauIfStatementRecursionGuard, false) LUAU_FASTFLAGVARIABLE(LuauTypeAliasPacks, false) LUAU_FASTFLAGVARIABLE(LuauParseTypePackTypeParameters, false) LUAU_FASTFLAGVARIABLE(LuauFixAmbiguousErrorRecoveryInAssign, false) +LUAU_FASTFLAGVARIABLE(LuauParseSingletonTypes, false) LUAU_FASTFLAGVARIABLE(LuauParseGenericFunctionTypeBegin, false) namespace Luau @@ -1278,7 +1279,27 @@ AstType* Parser::parseTableTypeAnnotation() while (lexer.current().type != '}') { - if (lexer.current().type == '[') + if (FFlag::LuauParseSingletonTypes && lexer.current().type == '[' && + (lexer.lookahead().type == Lexeme::RawString || lexer.lookahead().type == Lexeme::QuotedString)) + { + const Lexeme begin = lexer.current(); + nextLexeme(); // [ + std::optional> chars = parseCharArray(); + + expectMatchAndConsume(']', begin); + expectAndConsume(':', "table field"); + + AstType* type = parseTypeAnnotation(); + + // TODO: since AstName conains a char*, it can't contain null + bool containsNull = chars && (strnlen(chars->data, chars->size) < chars->size); + + if (chars && !containsNull) + props.push_back({AstName(chars->data), begin.location, type}); + else + report(begin.location, "String literal contains malformed escape sequence"); + } + else if (lexer.current().type == '[') { if (indexer) { @@ -1528,6 +1549,32 @@ AstTypeOrPack Parser::parseSimpleTypeAnnotation(bool allowPack) nextLexeme(); return {allocator.alloc(begin, std::nullopt, nameNil), {}}; } + else if (FFlag::LuauParseSingletonTypes && lexer.current().type == Lexeme::ReservedTrue) + { + nextLexeme(); + return {allocator.alloc(begin, true)}; + } + else if (FFlag::LuauParseSingletonTypes && lexer.current().type == Lexeme::ReservedFalse) + { + nextLexeme(); + return {allocator.alloc(begin, false)}; + } + else if (FFlag::LuauParseSingletonTypes && (lexer.current().type == Lexeme::RawString || lexer.current().type == Lexeme::QuotedString)) + { + if (std::optional> value = parseCharArray()) + { + AstArray svalue = *value; + return {allocator.alloc(begin, svalue)}; + } + else + return {reportTypeAnnotationError(begin, {}, /*isMissing*/ false, "String literal contains malformed escape sequence")}; + } + else if (FFlag::LuauParseSingletonTypes && lexer.current().type == Lexeme::BrokenString) + { + Location location = lexer.current().location; + nextLexeme(); + return {reportTypeAnnotationError(location, {}, /*isMissing*/ false, "Malformed string")}; + } else if (lexer.current().type == Lexeme::Name) { std::optional prefix; @@ -2416,7 +2463,7 @@ AstArray Parser::parseTypeParams() return copy(parameters); } -AstExpr* Parser::parseString() +std::optional> Parser::parseCharArray() { LUAU_ASSERT(lexer.current().type == Lexeme::QuotedString || lexer.current().type == Lexeme::RawString); @@ -2426,11 +2473,8 @@ AstExpr* Parser::parseString() { if (!Lexer::fixupQuotedString(scratchData)) { - Location location = lexer.current().location; - nextLexeme(); - - return reportExprError(location, {}, "String literal contains malformed escape sequence"); + return std::nullopt; } } else @@ -2438,12 +2482,18 @@ AstExpr* Parser::parseString() Lexer::fixupMultilineString(scratchData); } - Location start = lexer.current().location; AstArray value = copy(scratchData); - nextLexeme(); + return value; +} - return allocator.alloc(start, value); +AstExpr* Parser::parseString() +{ + Location location = lexer.current().location; + if (std::optional> value = parseCharArray()) + return allocator.alloc(location, *value); + else + return reportExprError(location, {}, "String literal contains malformed escape sequence"); } AstLocal* Parser::pushLocal(const Binding& binding) diff --git a/Ast/src/StringUtils.cpp b/Ast/src/StringUtils.cpp index 24b2283a..9c7fed31 100644 --- a/Ast/src/StringUtils.cpp +++ b/Ast/src/StringUtils.cpp @@ -225,4 +225,62 @@ size_t hashRange(const char* data, size_t size) return hash; } +bool isIdentifier(std::string_view s) +{ + return (s.find_first_not_of("abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890_") == std::string::npos); +} + +std::string escape(std::string_view s) +{ + std::string r; + r.reserve(s.size() + 50); // arbitrary number to guess how many characters we'll be inserting + + for (uint8_t c : s) + { + if (c >= ' ' && c != '\\' && c != '\'' && c != '\"') + r += c; + else + { + r += '\\'; + + switch (c) + { + case '\a': + r += 'a'; + break; + case '\b': + r += 'b'; + break; + case '\f': + r += 'f'; + break; + case '\n': + r += 'n'; + break; + case '\r': + r += 'r'; + break; + case '\t': + r += 't'; + break; + case '\v': + r += 'v'; + break; + case '\'': + r += '\''; + break; + case '\"': + r += '\"'; + break; + case '\\': + r += '\\'; + break; + default: + Luau::formatAppend(r, "%03u", c); + } + } + } + + return r; +} } // namespace Luau diff --git a/CLI/Analyze.cpp b/CLI/Analyze.cpp index 798ead06..4194d5ae 100644 --- a/CLI/Analyze.cpp +++ b/CLI/Analyze.cpp @@ -236,32 +236,12 @@ int main(int argc, char** argv) Luau::registerBuiltinTypes(frontend.typeChecker); Luau::freeze(frontend.typeChecker.globalTypes); + std::vector files = getSourceFiles(argc, argv); + int failed = 0; - for (int i = 1; i < argc; ++i) - { - if (argv[i][0] == '-') - continue; - - if (isDirectory(argv[i])) - { - traverseDirectory(argv[i], [&](const std::string& name) { - // Look for .luau first and if absent, fall back to .lua - if (name.length() > 5 && name.rfind(".luau") == name.length() - 5) - { - failed += !analyzeFile(frontend, name.c_str(), format, annotate); - } - else if (name.length() > 4 && name.rfind(".lua") == name.length() - 4) - { - failed += !analyzeFile(frontend, name.c_str(), format, annotate); - } - }); - } - else - { - failed += !analyzeFile(frontend, argv[i], format, annotate); - } - } + for (const std::string& path : files) + failed += !analyzeFile(frontend, path.c_str(), format, annotate); if (!configResolver.configErrors.empty()) { diff --git a/CLI/FileUtils.cpp b/CLI/FileUtils.cpp index fff5e429..b3c9557b 100644 --- a/CLI/FileUtils.cpp +++ b/CLI/FileUtils.cpp @@ -223,3 +223,40 @@ std::optional getParentPath(const std::string& path) return ""; } + +static std::string getExtension(const std::string& path) +{ + std::string::size_type dot = path.find_last_of(".\\/"); + + if (dot == std::string::npos || path[dot] != '.') + return ""; + + return path.substr(dot); +} + +std::vector getSourceFiles(int argc, char** argv) +{ + std::vector files; + + for (int i = 1; i < argc; ++i) + { + if (argv[i][0] == '-') + continue; + + if (isDirectory(argv[i])) + { + traverseDirectory(argv[i], [&](const std::string& name) { + std::string ext = getExtension(name); + + if (ext == ".lua" || ext == ".luau") + files.push_back(name); + }); + } + else + { + files.push_back(argv[i]); + } + } + + return files; +} diff --git a/CLI/FileUtils.h b/CLI/FileUtils.h index f7fbe8af..da11f512 100644 --- a/CLI/FileUtils.h +++ b/CLI/FileUtils.h @@ -4,6 +4,7 @@ #include #include #include +#include std::optional readFile(const std::string& name); @@ -12,3 +13,5 @@ bool traverseDirectory(const std::string& path, const std::function getParentPath(const std::string& path); + +std::vector getSourceFiles(int argc, char** argv); diff --git a/CLI/Repl.cpp b/CLI/Repl.cpp index 0d804be5..410674fa 100644 --- a/CLI/Repl.cpp +++ b/CLI/Repl.cpp @@ -20,7 +20,7 @@ enum class CompileFormat { - Default, + Text, Binary }; @@ -33,7 +33,7 @@ static int lua_loadstring(lua_State* L) lua_setsafeenv(L, LUA_ENVIRONINDEX, false); std::string bytecode = Luau::compile(std::string(s, l)); - if (luau_load(L, chunkname, bytecode.data(), bytecode.size()) == 0) + if (luau_load(L, chunkname, bytecode.data(), bytecode.size(), 0) == 0) return 1; lua_pushnil(L); @@ -80,7 +80,7 @@ static int lua_require(lua_State* L) // now we can compile & run module on the new thread std::string bytecode = Luau::compile(*source); - if (luau_load(ML, chunkname.c_str(), bytecode.data(), bytecode.size()) == 0) + if (luau_load(ML, chunkname.c_str(), bytecode.data(), bytecode.size(), 0) == 0) { int status = lua_resume(ML, L, 0); @@ -151,7 +151,7 @@ static std::string runCode(lua_State* L, const std::string& source) { std::string bytecode = Luau::compile(source); - if (luau_load(L, "=stdin", bytecode.data(), bytecode.size()) != 0) + if (luau_load(L, "=stdin", bytecode.data(), bytecode.size(), 0) != 0) { size_t len; const char* msg = lua_tolstring(L, -1, &len); @@ -370,7 +370,7 @@ static bool runFile(const char* name, lua_State* GL) std::string bytecode = Luau::compile(*source); int status = 0; - if (luau_load(L, chunkname.c_str(), bytecode.data(), bytecode.size()) == 0) + if (luau_load(L, chunkname.c_str(), bytecode.data(), bytecode.size(), 0) == 0) { status = lua_resume(L, NULL, 0); } @@ -379,11 +379,7 @@ static bool runFile(const char* name, lua_State* GL) status = LUA_ERRSYNTAX; } - if (status == 0) - { - return true; - } - else + if (status != 0) { std::string error; @@ -400,8 +396,10 @@ static bool runFile(const char* name, lua_State* GL) error += lua_debugtrace(L); fprintf(stderr, "%s", error.c_str()); - return false; } + + lua_pop(GL, 1); + return status == 0; } static void report(const char* name, const Luau::Location& location, const char* type, const char* message) @@ -431,14 +429,18 @@ static bool compileFile(const char* name, CompileFormat format) try { Luau::BytecodeBuilder bcb; - bcb.setDumpFlags(Luau::BytecodeBuilder::Dump_Code | Luau::BytecodeBuilder::Dump_Source); - bcb.setDumpSource(*source); + + if (format == CompileFormat::Text) + { + bcb.setDumpFlags(Luau::BytecodeBuilder::Dump_Code | Luau::BytecodeBuilder::Dump_Source); + bcb.setDumpSource(*source); + } Luau::compileOrThrow(bcb, *source); switch (format) { - case CompileFormat::Default: + case CompileFormat::Text: printf("%s", bcb.dumpEverything().c_str()); break; case CompileFormat::Binary: @@ -504,7 +506,7 @@ int main(int argc, char** argv) if (argc >= 2 && strncmp(argv[1], "--compile", strlen("--compile")) == 0) { - CompileFormat format = CompileFormat::Default; + CompileFormat format = CompileFormat::Text; if (strcmp(argv[1], "--compile=binary") == 0) format = CompileFormat::Binary; @@ -514,27 +516,12 @@ int main(int argc, char** argv) _setmode(_fileno(stdout), _O_BINARY); #endif + std::vector files = getSourceFiles(argc, argv); + int failed = 0; - for (int i = 2; i < argc; ++i) - { - if (argv[i][0] == '-') - continue; - - if (isDirectory(argv[i])) - { - traverseDirectory(argv[i], [&](const std::string& name) { - if (name.length() > 5 && name.rfind(".luau") == name.length() - 5) - failed += !compileFile(name.c_str(), format); - else if (name.length() > 4 && name.rfind(".lua") == name.length() - 4) - failed += !compileFile(name.c_str(), format); - }); - } - else - { - failed += !compileFile(argv[i], format); - } - } + for (const std::string& path : files) + failed += !compileFile(path.c_str(), format); return failed; } @@ -548,33 +535,25 @@ int main(int argc, char** argv) int profile = 0; for (int i = 1; i < argc; ++i) + { + if (argv[i][0] != '-') + continue; + if (strcmp(argv[i], "--profile") == 0) profile = 10000; // default to 10 KHz else if (strncmp(argv[i], "--profile=", 10) == 0) profile = atoi(argv[i] + 10); + } if (profile) profilerStart(L, profile); + std::vector files = getSourceFiles(argc, argv); + int failed = 0; - for (int i = 1; i < argc; ++i) - { - if (argv[i][0] == '-') - continue; - - if (isDirectory(argv[i])) - { - traverseDirectory(argv[i], [&](const std::string& name) { - if (name.length() > 4 && name.rfind(".lua") == name.length() - 4) - failed += !runFile(name.c_str(), L); - }); - } - else - { - failed += !runFile(argv[i], L); - } - } + for (const std::string& path : files) + failed += !runFile(path.c_str(), L); if (profile) { diff --git a/Compiler/include/Luau/Compiler.h b/Compiler/include/Luau/Compiler.h index 4f88e602..65e962da 100644 --- a/Compiler/include/Luau/Compiler.h +++ b/Compiler/include/Luau/Compiler.h @@ -13,11 +13,9 @@ class AstNameTable; class BytecodeBuilder; class BytecodeEncoder; +// Note: this structure is duplicated in luacode.h, don't forget to change these in sync! struct CompileOptions { - // default bytecode version target; can be used to compile code for older clients - int bytecodeVersion = 1; - // 0 - no optimization // 1 - baseline optimization level that doesn't prevent debuggability // 2 - includes optimizations that harm debuggability such as inlining diff --git a/Compiler/include/luacode.h b/Compiler/include/luacode.h new file mode 100644 index 00000000..e235a2e7 --- /dev/null +++ b/Compiler/include/luacode.h @@ -0,0 +1,39 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +#pragma once + +#include + +/* Can be used to reconfigure visibility/exports for public APIs */ +#ifndef LUACODE_API +#define LUACODE_API extern +#endif + +typedef struct lua_CompileOptions lua_CompileOptions; + +struct lua_CompileOptions +{ + // 0 - no optimization + // 1 - baseline optimization level that doesn't prevent debuggability + // 2 - includes optimizations that harm debuggability such as inlining + int optimizationLevel; // default=1 + + // 0 - no debugging support + // 1 - line info & function names only; sufficient for backtraces + // 2 - full debug info with local & upvalue names; necessary for debugger + int debugLevel; // default=1 + + // 0 - no code coverage support + // 1 - statement coverage + // 2 - statement and expression coverage (verbose) + int coverageLevel; // default=0 + + // global builtin to construct vectors; disabled by default + const char* vectorLib; + const char* vectorCtor; + + // null-terminated array of globals that are mutable; disables the import optimization for fields accessed through these + const char** mutableGlobals; +}; + +/* compile source to bytecode; when source compilation fails, the resulting bytecode contains the encoded error. use free() to destroy */ +LUACODE_API char* luau_compile(const char* source, size_t size, lua_CompileOptions* options, size_t* outsize); diff --git a/Compiler/src/Compiler.cpp b/Compiler/src/Compiler.cpp index 9712f02f..5b93c1dc 100644 --- a/Compiler/src/Compiler.cpp +++ b/Compiler/src/Compiler.cpp @@ -11,9 +11,6 @@ #include LUAU_FASTFLAGVARIABLE(LuauPreloadClosures, false) -LUAU_FASTFLAGVARIABLE(LuauPreloadClosuresFenv, false) -LUAU_FASTFLAGVARIABLE(LuauPreloadClosuresUpval, false) -LUAU_FASTFLAGVARIABLE(LuauGenericSpecialGlobals, false) LUAU_FASTFLAG(LuauIfElseExpressionBaseSupport) LUAU_FASTFLAGVARIABLE(LuauBit32CountBuiltin, false) @@ -24,9 +21,6 @@ static const uint32_t kMaxRegisterCount = 255; static const uint32_t kMaxUpvalueCount = 200; static const uint32_t kMaxLocalCount = 200; -// TODO: Remove with LuauGenericSpecialGlobals -static const char* kSpecialGlobals[] = {"Game", "Workspace", "_G", "game", "plugin", "script", "shared", "workspace"}; - CompileError::CompileError(const Location& location, const std::string& message) : location(location) , message(message) @@ -466,7 +460,7 @@ struct Compiler bool shared = false; - if (FFlag::LuauPreloadClosuresUpval) + if (FFlag::LuauPreloadClosures) { // Optimization: when closure has no upvalues, or upvalues are safe to share, instead of allocating it every time we can share closure // objects (this breaks assumptions about function identity which can lead to setfenv not working as expected, so we disable this when it @@ -482,18 +476,6 @@ struct Compiler } } } - // Optimization: when closure has no upvalues, instead of allocating it every time we can share closure objects - // (this breaks assumptions about function identity which can lead to setfenv not working as expected, so we disable this when it is used) - else if (FFlag::LuauPreloadClosures && options.optimizationLevel >= 1 && f->upvals.empty() && !setfenvUsed) - { - int32_t cid = bytecode.addConstantClosure(f->id); - - if (cid >= 0 && cid < 32768) - { - bytecode.emitAD(LOP_DUPCLOSURE, target, cid); - return; - } - } if (!shared) bytecode.emitAD(LOP_NEWCLOSURE, target, pid); @@ -3298,8 +3280,7 @@ struct Compiler bool visit(AstStatLocalFunction* node) override { // record local->function association for some optimizations - if (FFlag::LuauPreloadClosuresUpval) - self->locals[node->name].func = node->func; + self->locals[node->name].func = node->func; return true; } @@ -3711,24 +3692,13 @@ void compileOrThrow(BytecodeBuilder& bytecode, AstStatBlock* root, const AstName Compiler compiler(bytecode, options); // since access to some global objects may result in values that change over time, we block imports from non-readonly tables - if (FFlag::LuauGenericSpecialGlobals) - { - if (AstName name = names.get("_G"); name.value) - compiler.globals[name].writable = true; + if (AstName name = names.get("_G"); name.value) + compiler.globals[name].writable = true; - if (options.mutableGlobals) - for (const char** ptr = options.mutableGlobals; *ptr; ++ptr) - if (AstName name = names.get(*ptr); name.value) - compiler.globals[name].writable = true; - } - else - { - for (const char* global : kSpecialGlobals) - { - if (AstName name = names.get(global); name.value) + if (options.mutableGlobals) + for (const char** ptr = options.mutableGlobals; *ptr; ++ptr) + if (AstName name = names.get(*ptr); name.value) compiler.globals[name].writable = true; - } - } // this visitor traverses the AST to analyze mutability of locals/globals, filling Local::written and Global::written Compiler::AssignmentVisitor assignmentVisitor(&compiler); @@ -3742,7 +3712,7 @@ void compileOrThrow(BytecodeBuilder& bytecode, AstStatBlock* root, const AstName } // this visitor tracks calls to getfenv/setfenv and disables some optimizations when they are found - if (FFlag::LuauPreloadClosuresFenv && options.optimizationLevel >= 1 && (names.get("getfenv").value || names.get("setfenv").value)) + if (options.optimizationLevel >= 1 && (names.get("getfenv").value || names.get("setfenv").value)) { Compiler::FenvVisitor fenvVisitor(compiler.getfenvUsed, compiler.setfenvUsed); root->visit(&fenvVisitor); diff --git a/Compiler/src/lcode.cpp b/Compiler/src/lcode.cpp new file mode 100644 index 00000000..ee150b17 --- /dev/null +++ b/Compiler/src/lcode.cpp @@ -0,0 +1,29 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +#include "luacode.h" + +#include "Luau/Compiler.h" + +#include + +char* luau_compile(const char* source, size_t size, lua_CompileOptions* options, size_t* outsize) +{ + LUAU_ASSERT(outsize); + + Luau::CompileOptions opts; + + if (options) + { + static_assert(sizeof(lua_CompileOptions) == sizeof(Luau::CompileOptions), "C and C++ interface must match"); + memcpy(static_cast(&opts), options, sizeof(opts)); + } + + std::string result = compile(std::string(source, size), opts); + + char* copy = static_cast(malloc(result.size())); + if (!copy) + return nullptr; + + memcpy(copy, result.data(), result.size()); + *outsize = result.size(); + return copy; +} diff --git a/Sources.cmake b/Sources.cmake index c30cf77d..23b931c6 100644 --- a/Sources.cmake +++ b/Sources.cmake @@ -25,9 +25,11 @@ target_sources(Luau.Compiler PRIVATE Compiler/include/Luau/Bytecode.h Compiler/include/Luau/BytecodeBuilder.h Compiler/include/Luau/Compiler.h + Compiler/include/luacode.h Compiler/src/BytecodeBuilder.cpp Compiler/src/Compiler.cpp + Compiler/src/lcode.cpp ) # Luau.Analysis Sources @@ -204,6 +206,7 @@ if(TARGET Luau.UnitTest) tests/TypeInfer.intersectionTypes.test.cpp tests/TypeInfer.provisional.test.cpp tests/TypeInfer.refinements.test.cpp + tests/TypeInfer.singletons.test.cpp tests/TypeInfer.tables.test.cpp tests/TypeInfer.test.cpp tests/TypeInfer.tryUnify.test.cpp diff --git a/VM/include/lua.h b/VM/include/lua.h index a9d3e875..1568d191 100644 --- a/VM/include/lua.h +++ b/VM/include/lua.h @@ -102,6 +102,8 @@ LUA_API lua_State* lua_newstate(lua_Alloc f, void* ud); LUA_API void lua_close(lua_State* L); LUA_API lua_State* lua_newthread(lua_State* L); LUA_API lua_State* lua_mainthread(lua_State* L); +LUA_API void lua_resetthread(lua_State* L); +LUA_API int lua_isthreadreset(lua_State* L); /* ** basic stack manipulation @@ -162,8 +164,7 @@ LUA_API void lua_pushlstring(lua_State* L, const char* s, size_t l); LUA_API void lua_pushstring(lua_State* L, const char* s); LUA_API const char* lua_pushvfstring(lua_State* L, const char* fmt, va_list argp); LUA_API LUA_PRINTF_ATTR(2, 3) const char* lua_pushfstringL(lua_State* L, const char* fmt, ...); -LUA_API void lua_pushcfunction( - lua_State* L, lua_CFunction fn, const char* debugname = NULL, int nup = 0, lua_Continuation cont = NULL); +LUA_API void lua_pushcclosurek(lua_State* L, lua_CFunction fn, const char* debugname, int nup, lua_Continuation cont); LUA_API void lua_pushboolean(lua_State* L, int b); LUA_API void lua_pushlightuserdata(lua_State* L, void* p); LUA_API int lua_pushthread(lua_State* L); @@ -178,9 +179,9 @@ LUA_API void lua_rawget(lua_State* L, int idx); LUA_API void lua_rawgeti(lua_State* L, int idx, int n); LUA_API void lua_createtable(lua_State* L, int narr, int nrec); -LUA_API void lua_setreadonly(lua_State* L, int idx, bool value); +LUA_API void lua_setreadonly(lua_State* L, int idx, int enabled); LUA_API int lua_getreadonly(lua_State* L, int idx); -LUA_API void lua_setsafeenv(lua_State* L, int idx, bool value); +LUA_API void lua_setsafeenv(lua_State* L, int idx, int enabled); LUA_API void* lua_newuserdata(lua_State* L, size_t sz, int tag); LUA_API void* lua_newuserdatadtor(lua_State* L, size_t sz, void (*dtor)(void*)); @@ -200,7 +201,7 @@ LUA_API int lua_setfenv(lua_State* L, int idx); /* ** `load' and `call' functions (load and run Luau bytecode) */ -LUA_API int luau_load(lua_State* L, const char* chunkname, const char* data, size_t size, int env = 0); +LUA_API int luau_load(lua_State* L, const char* chunkname, const char* data, size_t size, int env); LUA_API void lua_call(lua_State* L, int nargs, int nresults); LUA_API int lua_pcall(lua_State* L, int nargs, int nresults, int errfunc); @@ -293,6 +294,8 @@ LUA_API void lua_unref(lua_State* L, int ref); #define lua_isnoneornil(L, n) (lua_type(L, (n)) <= LUA_TNIL) #define lua_pushliteral(L, s) lua_pushlstring(L, "" s, (sizeof(s) / sizeof(char)) - 1) +#define lua_pushcfunction(L, fn, debugname) lua_pushcclosurek(L, fn, debugname, 0, NULL) +#define lua_pushcclosure(L, fn, debugname, nup) lua_pushcclosurek(L, fn, debugname, nup, NULL) #define lua_setglobal(L, s) lua_setfield(L, LUA_GLOBALSINDEX, (s)) #define lua_getglobal(L, s) lua_getfield(L, LUA_GLOBALSINDEX, (s)) @@ -319,8 +322,8 @@ LUA_API const char* lua_setlocal(lua_State* L, int level, int n); LUA_API const char* lua_getupvalue(lua_State* L, int funcindex, int n); LUA_API const char* lua_setupvalue(lua_State* L, int funcindex, int n); -LUA_API void lua_singlestep(lua_State* L, bool singlestep); -LUA_API void lua_breakpoint(lua_State* L, int funcindex, int line, bool enable); +LUA_API void lua_singlestep(lua_State* L, int enabled); +LUA_API void lua_breakpoint(lua_State* L, int funcindex, int line, int enabled); /* Warning: this function is not thread-safe since it stores the result in a shared global array! Only use for debugging. */ LUA_API const char* lua_debugtrace(lua_State* L); @@ -361,6 +364,7 @@ struct lua_Callbacks void (*debuginterrupt)(lua_State* L, lua_Debug* ar); /* gets called when thread execution is interrupted by break in another thread */ void (*debugprotectederror)(lua_State* L); /* gets called when protected call results in an error */ }; +typedef struct lua_Callbacks lua_Callbacks; LUA_API lua_Callbacks* lua_callbacks(lua_State* L); diff --git a/VM/include/lualib.h b/VM/include/lualib.h index 30cffaff..fa836955 100644 --- a/VM/include/lualib.h +++ b/VM/include/lualib.h @@ -8,11 +8,12 @@ #define luaL_typeerror(L, narg, tname) luaL_typeerrorL(L, narg, tname) #define luaL_argerror(L, narg, extramsg) luaL_argerrorL(L, narg, extramsg) -typedef struct luaL_Reg +struct luaL_Reg { const char* name; lua_CFunction func; -} luaL_Reg; +}; +typedef struct luaL_Reg luaL_Reg; LUALIB_API void luaL_register(lua_State* L, const char* libname, const luaL_Reg* l); LUALIB_API int luaL_getmetafield(lua_State* L, int obj, const char* e); @@ -75,6 +76,7 @@ struct luaL_Buffer struct TString* storage; char buffer[LUA_BUFFERSIZE]; }; +typedef struct luaL_Buffer luaL_Buffer; // when internal buffer storage is exhausted, a mutable string value 'storage' will be placed on the stack // in general, functions expect the mutable string buffer to be placed on top of the stack (top-1) diff --git a/VM/src/lapi.cpp b/VM/src/lapi.cpp index 7e742644..a79ba0d4 100644 --- a/VM/src/lapi.cpp +++ b/VM/src/lapi.cpp @@ -593,7 +593,7 @@ const char* lua_pushfstringL(lua_State* L, const char* fmt, ...) return ret; } -void lua_pushcfunction(lua_State* L, lua_CFunction fn, const char* debugname, int nup, lua_Continuation cont) +void lua_pushcclosurek(lua_State* L, lua_CFunction fn, const char* debugname, int nup, lua_Continuation cont) { luaC_checkGC(L); luaC_checkthreadsleep(L); @@ -698,13 +698,13 @@ void lua_createtable(lua_State* L, int narray, int nrec) return; } -void lua_setreadonly(lua_State* L, int objindex, bool value) +void lua_setreadonly(lua_State* L, int objindex, int enabled) { const TValue* o = index2adr(L, objindex); api_check(L, ttistable(o)); Table* t = hvalue(o); api_check(L, t != hvalue(registry(L))); - t->readonly = value; + t->readonly = bool(enabled); return; } @@ -717,12 +717,12 @@ int lua_getreadonly(lua_State* L, int objindex) return res; } -void lua_setsafeenv(lua_State* L, int objindex, bool value) +void lua_setsafeenv(lua_State* L, int objindex, int enabled) { const TValue* o = index2adr(L, objindex); api_check(L, ttistable(o)); Table* t = hvalue(o); - t->safeenv = value; + t->safeenv = bool(enabled); return; } diff --git a/VM/src/lbaselib.cpp b/VM/src/lbaselib.cpp index 87fc1631..61798e2b 100644 --- a/VM/src/lbaselib.cpp +++ b/VM/src/lbaselib.cpp @@ -436,8 +436,8 @@ static const luaL_Reg base_funcs[] = { static void auxopen(lua_State* L, const char* name, lua_CFunction f, lua_CFunction u) { - lua_pushcfunction(L, u); - lua_pushcfunction(L, f, name, 1); + lua_pushcfunction(L, u, NULL); + lua_pushcclosure(L, f, name, 1); lua_setfield(L, -2, name); } @@ -456,10 +456,10 @@ LUALIB_API int luaopen_base(lua_State* L) auxopen(L, "ipairs", luaB_ipairs, luaB_inext); auxopen(L, "pairs", luaB_pairs, luaB_next); - lua_pushcfunction(L, luaB_pcally, "pcall", 0, luaB_pcallcont); + lua_pushcclosurek(L, luaB_pcally, "pcall", 0, luaB_pcallcont); lua_setfield(L, -2, "pcall"); - lua_pushcfunction(L, luaB_xpcally, "xpcall", 0, luaB_xpcallcont); + lua_pushcclosurek(L, luaB_xpcally, "xpcall", 0, luaB_xpcallcont); lua_setfield(L, -2, "xpcall"); return 1; diff --git a/VM/src/lcorolib.cpp b/VM/src/lcorolib.cpp index 9724c0e7..0178fae8 100644 --- a/VM/src/lcorolib.cpp +++ b/VM/src/lcorolib.cpp @@ -5,6 +5,8 @@ #include "lstate.h" #include "lvm.h" +LUAU_FASTFLAGVARIABLE(LuauCoroutineClose, false) + #define CO_RUN 0 /* running */ #define CO_SUS 1 /* suspended */ #define CO_NOR 2 /* 'normal' (it resumed another coroutine) */ @@ -208,8 +210,7 @@ static int cowrap(lua_State* L) { cocreate(L); - lua_pushcfunction(L, auxwrapy, NULL, 1, auxwrapcont); - + lua_pushcclosurek(L, auxwrapy, NULL, 1, auxwrapcont); return 1; } @@ -232,6 +233,34 @@ static int coyieldable(lua_State* L) return 1; } +static int coclose(lua_State* L) +{ + if (!FFlag::LuauCoroutineClose) + luaL_error(L, "coroutine.close is not enabled"); + + lua_State* co = lua_tothread(L, 1); + luaL_argexpected(L, co, 1, "thread"); + + int status = auxstatus(L, co); + if (status != CO_DEAD && status != CO_SUS) + luaL_error(L, "cannot close %s coroutine", statnames[status]); + + if (co->status == LUA_OK || co->status == LUA_YIELD) + { + lua_pushboolean(L, true); + lua_resetthread(co); + return 1; + } + else + { + lua_pushboolean(L, false); + if (lua_gettop(co)) + lua_xmove(co, L, 1); /* move error message */ + lua_resetthread(co); + return 2; + } +} + static const luaL_Reg co_funcs[] = { {"create", cocreate}, {"running", corunning}, @@ -239,6 +268,7 @@ static const luaL_Reg co_funcs[] = { {"wrap", cowrap}, {"yield", coyield}, {"isyieldable", coyieldable}, + {"close", coclose}, {NULL, NULL}, }; @@ -246,7 +276,7 @@ LUALIB_API int luaopen_coroutine(lua_State* L) { luaL_register(L, LUA_COLIBNAME, co_funcs); - lua_pushcfunction(L, coresumey, "resume", 0, coresumecont); + lua_pushcclosurek(L, coresumey, "resume", 0, coresumecont); lua_setfield(L, -2, "resume"); return 1; diff --git a/VM/src/ldebug.cpp b/VM/src/ldebug.cpp index 1890e682..d77f84ef 100644 --- a/VM/src/ldebug.cpp +++ b/VM/src/ldebug.cpp @@ -316,7 +316,7 @@ void luaG_breakpoint(lua_State* L, Proto* p, int line, bool enable) p->debuginsn[j] = LUAU_INSN_OP(p->code[j]); } - uint8_t op = enable ? LOP_BREAK : LUAU_INSN_OP(p->code[i]); + uint8_t op = enable ? LOP_BREAK : LUAU_INSN_OP(p->debuginsn[i]); // patch just the opcode byte, leave arguments alone p->code[i] &= ~0xff; @@ -357,17 +357,17 @@ int luaG_getline(Proto* p, int pc) return p->abslineinfo[pc >> p->linegaplog2] + p->lineinfo[pc]; } -void lua_singlestep(lua_State* L, bool singlestep) +void lua_singlestep(lua_State* L, int enabled) { - L->singlestep = singlestep; + L->singlestep = bool(enabled); } -void lua_breakpoint(lua_State* L, int funcindex, int line, bool enable) +void lua_breakpoint(lua_State* L, int funcindex, int line, int enabled) { const TValue* func = luaA_toobject(L, funcindex); api_check(L, ttisfunction(func) && !clvalue(func)->isC); - luaG_breakpoint(L, clvalue(func)->l.p, line, enable); + luaG_breakpoint(L, clvalue(func)->l.p, line, bool(enabled)); } static size_t append(char* buf, size_t bufsize, size_t offset, const char* data) diff --git a/VM/src/ldo.cpp b/VM/src/ldo.cpp index 328b47e6..1259d461 100644 --- a/VM/src/ldo.cpp +++ b/VM/src/ldo.cpp @@ -19,6 +19,7 @@ LUAU_FASTFLAGVARIABLE(LuauExceptionMessageFix, false) LUAU_FASTFLAGVARIABLE(LuauCcallRestoreFix, false) +LUAU_FASTFLAG(LuauCoroutineClose) /* ** {====================================================== @@ -300,7 +301,10 @@ static void resume(lua_State* L, void* ud) if (L->status == 0) { // start coroutine - LUAU_ASSERT(L->ci == L->base_ci && firstArg > L->base); + LUAU_ASSERT(L->ci == L->base_ci && firstArg >= L->base); + if (FFlag::LuauCoroutineClose && firstArg == L->base) + luaG_runerror(L, "cannot resume dead coroutine"); + if (luau_precall(L, firstArg - 1, LUA_MULTRET) != PCRLUA) return; diff --git a/VM/src/linit.cpp b/VM/src/linit.cpp index bf5e738f..4e40165a 100644 --- a/VM/src/linit.cpp +++ b/VM/src/linit.cpp @@ -22,7 +22,7 @@ LUALIB_API void luaL_openlibs(lua_State* L) const luaL_Reg* lib = lualibs; for (; lib->func; lib++) { - lua_pushcfunction(L, lib->func); + lua_pushcfunction(L, lib->func, NULL); lua_pushstring(L, lib->name); lua_call(L, 1, 0); } diff --git a/VM/src/lstate.cpp b/VM/src/lstate.cpp index 0b2dfb69..24e97063 100644 --- a/VM/src/lstate.cpp +++ b/VM/src/lstate.cpp @@ -124,6 +124,34 @@ void luaE_freethread(lua_State* L, lua_State* L1) luaM_free(L, L1, sizeof(lua_State), L1->memcat); } +void lua_resetthread(lua_State* L) +{ + /* close upvalues before clearing anything */ + luaF_close(L, L->stack); + /* clear call frames */ + CallInfo* ci = L->base_ci; + ci->func = L->stack; + ci->base = ci->func + 1; + ci->top = ci->base + LUA_MINSTACK; + setnilvalue(ci->func); + L->ci = ci; + luaD_reallocCI(L, BASIC_CI_SIZE); + /* clear thread state */ + L->status = LUA_OK; + L->base = L->ci->base; + L->top = L->ci->base; + L->nCcalls = L->baseCcalls = 0; + /* clear thread stack */ + luaD_reallocstack(L, BASIC_STACK_SIZE); + for (int i = 0; i < L->stacksize; i++) + setnilvalue(L->stack + i); +} + +int lua_isthreadreset(lua_State* L) +{ + return L->ci == L->base_ci && L->base == L->top && L->status == LUA_OK; +} + lua_State* lua_newstate(lua_Alloc f, void* ud) { int i; diff --git a/VM/src/lstrlib.cpp b/VM/src/lstrlib.cpp index 80a34483..b576f809 100644 --- a/VM/src/lstrlib.cpp +++ b/VM/src/lstrlib.cpp @@ -748,7 +748,7 @@ static int gmatch(lua_State* L) luaL_checkstring(L, 2); lua_settop(L, 2); lua_pushinteger(L, 0); - lua_pushcfunction(L, gmatch_aux, NULL, 3); + lua_pushcclosure(L, gmatch_aux, NULL, 3); return 1; } diff --git a/VM/src/lutf8lib.cpp b/VM/src/lutf8lib.cpp index 6a026296..378de3d0 100644 --- a/VM/src/lutf8lib.cpp +++ b/VM/src/lutf8lib.cpp @@ -265,7 +265,7 @@ static int iter_aux(lua_State* L) static int iter_codes(lua_State* L) { luaL_checkstring(L, 1); - lua_pushcfunction(L, iter_aux); + lua_pushcfunction(L, iter_aux, NULL); lua_pushvalue(L, 1); lua_pushinteger(L, 0); return 3; diff --git a/bench/bench.py b/bench/bench.py index b23ca891..39f219f3 100644 --- a/bench/bench.py +++ b/bench/bench.py @@ -25,8 +25,8 @@ try: import scipy from scipy import stats except ModuleNotFoundError: - print("scipy package is required") - exit(1) + print("Warning: scipy package is not installed, confidence values will not be available") + stats = None scriptdir = os.path.dirname(os.path.realpath(__file__)) defaultVm = 'luau.exe' if os.name == "nt" else './luau' @@ -200,11 +200,14 @@ def finalizeResult(result): result.sampleStdDev = math.sqrt(sumOfSquares / (result.count - 1)) result.unbiasedEst = result.sampleStdDev * result.sampleStdDev - # Two-tailed distribution with 95% conf. - tValue = stats.t.ppf(1 - 0.05 / 2, result.count - 1) + if stats: + # Two-tailed distribution with 95% conf. + tValue = stats.t.ppf(1 - 0.05 / 2, result.count - 1) - # Compute confidence interval - result.sampleConfidenceInterval = tValue * result.sampleStdDev / math.sqrt(result.count) + # Compute confidence interval + result.sampleConfidenceInterval = tValue * result.sampleStdDev / math.sqrt(result.count) + else: + result.sampleConfidenceInterval = result.sampleStdDev else: result.sampleStdDev = 0 result.unbiasedEst = 0 @@ -377,14 +380,19 @@ def analyzeResult(subdir, main, comparisons): tStat = abs(main.avg - compare.avg) / (pooledStdDev * math.sqrt(2 / main.count)) degreesOfFreedom = 2 * main.count - 2 - # Two-tailed distribution with 95% conf. - tCritical = stats.t.ppf(1 - 0.05 / 2, degreesOfFreedom) + if stats: + # Two-tailed distribution with 95% conf. + tCritical = stats.t.ppf(1 - 0.05 / 2, degreesOfFreedom) - noSignificantDifference = tStat < tCritical + noSignificantDifference = tStat < tCritical + pValue = 2 * (1 - stats.t.cdf(tStat, df = degreesOfFreedom)) + else: + noSignificantDifference = None + pValue = -1 - pValue = 2 * (1 - stats.t.cdf(tStat, df = degreesOfFreedom)) - - if noSignificantDifference: + if noSignificantDifference is None: + verdict = "" + elif noSignificantDifference: verdict = "likely same" elif main.avg < compare.avg: verdict = "likely worse" diff --git a/fuzz/proto.cpp b/fuzz/proto.cpp index c85fac7d..ae2399e4 100644 --- a/fuzz/proto.cpp +++ b/fuzz/proto.cpp @@ -257,7 +257,7 @@ DEFINE_PROTO_FUZZER(const luau::StatBlock& message) lua_State* L = lua_newthread(globalState); luaL_sandboxthread(L); - if (luau_load(L, "=fuzz", bytecode.data(), bytecode.size()) == 0) + if (luau_load(L, "=fuzz", bytecode.data(), bytecode.size(), 0) == 0) { interruptDeadline = std::chrono::system_clock::now() + kInterruptTimeout; diff --git a/tests/Autocomplete.test.cpp b/tests/Autocomplete.test.cpp index 8a7798f3..5a7c8602 100644 --- a/tests/Autocomplete.test.cpp +++ b/tests/Autocomplete.test.cpp @@ -91,10 +91,6 @@ struct ACFixture : ACFixtureImpl { }; -struct UnfrozenACFixture : ACFixtureImpl -{ -}; - TEST_SUITE_BEGIN("AutocompleteTest"); TEST_CASE_FIXTURE(ACFixture, "empty_program") @@ -1919,9 +1915,10 @@ local bar: @1= foo CHECK(!ac.entryMap.count("foo")); } -// CLI-45692: Remove UnfrozenACFixture here -TEST_CASE_FIXTURE(UnfrozenACFixture, "type_correct_function_no_parenthesis") +TEST_CASE_FIXTURE(ACFixture, "type_correct_function_no_parenthesis") { + ScopedFastFlag luauAutocompleteAvoidMutation("LuauAutocompleteAvoidMutation", true); + check(R"( local function target(a: (number) -> number) return a(4) end local function bar1(a: number) return -a end @@ -1950,9 +1947,10 @@ local fp: @1= f CHECK(ac.entryMap.count("({ x: number, y: number }) -> number")); } -// CLI-45692: Remove UnfrozenACFixture here -TEST_CASE_FIXTURE(UnfrozenACFixture, "type_correct_keywords") +TEST_CASE_FIXTURE(ACFixture, "type_correct_keywords") { + ScopedFastFlag luauAutocompleteAvoidMutation("LuauAutocompleteAvoidMutation", true); + check(R"( local function a(x: boolean) end local function b(x: number?) end @@ -2484,7 +2482,7 @@ local t = { CHECK(ac.entryMap.count("second")); } -TEST_CASE_FIXTURE(Fixture, "autocomplete_documentation_symbols") +TEST_CASE_FIXTURE(UnfrozenFixture, "autocomplete_documentation_symbols") { loadDefinition(R"( declare y: { diff --git a/tests/Compiler.test.cpp b/tests/Compiler.test.cpp index 7f03019c..4ce8d08a 100644 --- a/tests/Compiler.test.cpp +++ b/tests/Compiler.test.cpp @@ -11,8 +11,6 @@ #include LUAU_FASTFLAG(LuauPreloadClosures) -LUAU_FASTFLAG(LuauPreloadClosuresFenv) -LUAU_FASTFLAG(LuauPreloadClosuresUpval) LUAU_FASTFLAG(LuauGenericSpecialGlobals) using namespace Luau; @@ -2797,7 +2795,7 @@ CAPTURE UPVAL U1 RETURN R0 1 )"); - if (FFlag::LuauPreloadClosuresUpval) + if (FFlag::LuauPreloadClosures) { // recursive capture CHECK_EQ("\n" + compileFunction("local function foo() return foo() end", 1), R"( @@ -3479,15 +3477,13 @@ CAPTURE VAL R0 RETURN R1 1 )"); - if (FFlag::LuauPreloadClosuresFenv) - { - // if they don't need upvalues but we sense that environment may be modified, we disable this to avoid fenv-related identity confusion - CHECK_EQ("\n" + compileFunction(R"( + // if they don't need upvalues but we sense that environment may be modified, we disable this to avoid fenv-related identity confusion + CHECK_EQ("\n" + compileFunction(R"( setfenv(1, {}) return function() print("hi") end )", - 1), - R"( + 1), + R"( GETIMPORT R0 1 LOADN R1 1 NEWTABLE R2 0 0 @@ -3496,23 +3492,21 @@ NEWCLOSURE R0 P0 RETURN R0 1 )"); - // note that fenv analysis isn't flow-sensitive right now, which is sort of a feature - CHECK_EQ("\n" + compileFunction(R"( + // note that fenv analysis isn't flow-sensitive right now, which is sort of a feature + CHECK_EQ("\n" + compileFunction(R"( if false then setfenv(1, {}) end return function() print("hi") end )", - 1), - R"( + 1), + R"( NEWCLOSURE R0 P0 RETURN R0 1 )"); - } } TEST_CASE("SharedClosure") { ScopedFastFlag sff1("LuauPreloadClosures", true); - ScopedFastFlag sff2("LuauPreloadClosuresUpval", true); // closures can be shared even if functions refer to upvalues, as long as upvalues are top-level CHECK_EQ("\n" + compileFunction(R"( @@ -3671,7 +3665,7 @@ RETURN R0 0 )"); } -TEST_CASE("LuauGenericSpecialGlobals") +TEST_CASE("MutableGlobals") { const char* source = R"( print() @@ -3685,43 +3679,6 @@ shared.print() workspace.print() )"; - { - ScopedFastFlag genericSpecialGlobals{"LuauGenericSpecialGlobals", false}; - - // Check Roblox globals are here - CHECK_EQ("\n" + compileFunction0(source), R"( -GETIMPORT R0 1 -CALL R0 0 0 -GETIMPORT R1 3 -GETTABLEKS R0 R1 K0 -CALL R0 0 0 -GETIMPORT R1 5 -GETTABLEKS R0 R1 K0 -CALL R0 0 0 -GETIMPORT R1 7 -GETTABLEKS R0 R1 K0 -CALL R0 0 0 -GETIMPORT R1 9 -GETTABLEKS R0 R1 K0 -CALL R0 0 0 -GETIMPORT R1 11 -GETTABLEKS R0 R1 K0 -CALL R0 0 0 -GETIMPORT R1 13 -GETTABLEKS R0 R1 K0 -CALL R0 0 0 -GETIMPORT R1 15 -GETTABLEKS R0 R1 K0 -CALL R0 0 0 -GETIMPORT R1 17 -GETTABLEKS R0 R1 K0 -CALL R0 0 0 -RETURN R0 0 -)"); - } - - ScopedFastFlag genericSpecialGlobals{"LuauGenericSpecialGlobals", true}; - // Check Roblox globals are no longer here CHECK_EQ("\n" + compileFunction0(source), R"( GETIMPORT R0 1 diff --git a/tests/Conformance.test.cpp b/tests/Conformance.test.cpp index c1b790b9..e495a213 100644 --- a/tests/Conformance.test.cpp +++ b/tests/Conformance.test.cpp @@ -1,5 +1,7 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details -#include "Luau/Compiler.h" +#include "lua.h" +#include "lualib.h" +#include "luacode.h" #include "Luau/BuiltinDefinitions.h" #include "Luau/ModuleResolver.h" @@ -10,9 +12,6 @@ #include "doctest.h" #include "ScopedFlags.h" -#include "lua.h" -#include "lualib.h" - #include #include @@ -49,8 +48,12 @@ static int lua_loadstring(lua_State* L) lua_setsafeenv(L, LUA_ENVIRONINDEX, false); - std::string bytecode = Luau::compile(std::string(s, l)); - if (luau_load(L, chunkname, bytecode.data(), bytecode.size()) == 0) + size_t bytecodeSize = 0; + char* bytecode = luau_compile(s, l, nullptr, &bytecodeSize); + int result = luau_load(L, chunkname, bytecode, bytecodeSize, 0); + free(bytecode); + + if (result == 0) return 1; lua_pushnil(L); @@ -179,21 +182,17 @@ static StateRef runConformance( std::string chunkname = "=" + std::string(name); - Luau::CompileOptions copts; + lua_CompileOptions copts = {}; + copts.optimizationLevel = 1; // default copts.debugLevel = 2; // for debugger tests copts.vectorCtor = "vector"; // for vector tests - std::string bytecode = Luau::compile(source, copts); - int status = 0; + size_t bytecodeSize = 0; + char* bytecode = luau_compile(source.data(), source.size(), &copts, &bytecodeSize); + int result = luau_load(L, chunkname.c_str(), bytecode, bytecodeSize, 0); + free(bytecode); - if (luau_load(L, chunkname.c_str(), bytecode.data(), bytecode.size()) == 0) - { - status = lua_resume(L, nullptr, 0); - } - else - { - status = LUA_ERRSYNTAX; - } + int status = (result == 0) ? lua_resume(L, nullptr, 0) : LUA_ERRSYNTAX; while (yield && (status == LUA_YIELD || status == LUA_BREAK)) { @@ -332,53 +331,61 @@ TEST_CASE("UTF8") TEST_CASE("Coroutine") { + ScopedFastFlag sff("LuauCoroutineClose", true); + runConformance("coroutine.lua"); } +static int cxxthrow(lua_State* L) +{ +#if LUA_USE_LONGJMP + luaL_error(L, "oops"); +#else + throw std::runtime_error("oops"); +#endif +} + TEST_CASE("PCall") { runConformance("pcall.lua", [](lua_State* L) { - lua_pushcfunction(L, [](lua_State* L) -> int { -#if LUA_USE_LONGJMP - luaL_error(L, "oops"); -#else - throw std::runtime_error("oops"); -#endif - }); + lua_pushcfunction(L, cxxthrow, "cxxthrow"); lua_setglobal(L, "cxxthrow"); - lua_pushcfunction(L, [](lua_State* L) -> int { - lua_State* co = lua_tothread(L, 1); - lua_xmove(L, co, 1); - lua_resumeerror(co, L); - return 0; - }); + lua_pushcfunction( + L, + [](lua_State* L) -> int { + lua_State* co = lua_tothread(L, 1); + lua_xmove(L, co, 1); + lua_resumeerror(co, L); + return 0; + }, + "resumeerror"); lua_setglobal(L, "resumeerror"); }); } TEST_CASE("Pack") { - ScopedFastFlag sff{ "LuauStrPackUBCastFix", true }; - + ScopedFastFlag sff{"LuauStrPackUBCastFix", true}; + runConformance("tpack.lua"); } TEST_CASE("Vector") { runConformance("vector.lua", [](lua_State* L) { - lua_pushcfunction(L, lua_vector); + lua_pushcfunction(L, lua_vector, "vector"); lua_setglobal(L, "vector"); lua_pushvector(L, 0.0f, 0.0f, 0.0f); luaL_newmetatable(L, "vector"); lua_pushstring(L, "__index"); - lua_pushcfunction(L, lua_vector_index); + lua_pushcfunction(L, lua_vector_index, nullptr); lua_settable(L, -3); lua_pushstring(L, "__namecall"); - lua_pushcfunction(L, lua_vector_namecall); + lua_pushcfunction(L, lua_vector_namecall, nullptr); lua_settable(L, -3); lua_setreadonly(L, -1, true); @@ -513,15 +520,19 @@ TEST_CASE("Debugger") }; // add breakpoint() function - lua_pushcfunction(L, [](lua_State* L) -> int { - int line = luaL_checkinteger(L, 1); + lua_pushcfunction( + L, + [](lua_State* L) -> int { + int line = luaL_checkinteger(L, 1); + bool enabled = lua_isboolean(L, 2) ? lua_toboolean(L, 2) : true; - lua_Debug ar = {}; - lua_getinfo(L, 1, "f", &ar); + lua_Debug ar = {}; + lua_getinfo(L, 1, "f", &ar); - lua_breakpoint(L, -1, line, true); - return 0; - }); + lua_breakpoint(L, -1, line, enabled); + return 0; + }, + "breakpoint"); lua_setglobal(L, "breakpoint"); }, [](lua_State* L) { @@ -744,7 +755,7 @@ TEST_CASE("ExceptionObject") if (nsize == 0) { free(ptr); - return NULL; + return nullptr; } else if (nsize > 512 * 1024) { diff --git a/tests/IostreamOptional.h b/tests/IostreamOptional.h index e55b5b0c..e0756bad 100644 --- a/tests/IostreamOptional.h +++ b/tests/IostreamOptional.h @@ -4,7 +4,8 @@ #include #include -namespace std { +namespace std +{ inline std::ostream& operator<<(std::ostream& lhs, const std::nullopt_t&) { diff --git a/tests/Module.test.cpp b/tests/Module.test.cpp index 18f55d2c..7a3543c7 100644 --- a/tests/Module.test.cpp +++ b/tests/Module.test.cpp @@ -203,6 +203,8 @@ TEST_CASE_FIXTURE(Fixture, "clone_class") TEST_CASE_FIXTURE(Fixture, "clone_sanitize_free_types") { + ScopedFastFlag sff{"LuauErrorRecoveryType", true}; + TypeVar freeTy(FreeTypeVar{TypeLevel{}}); TypePackVar freeTp(FreeTypePack{TypeLevel{}}); @@ -212,12 +214,12 @@ TEST_CASE_FIXTURE(Fixture, "clone_sanitize_free_types") bool encounteredFreeType = false; TypeId clonedTy = clone(&freeTy, dest, seenTypes, seenTypePacks, &encounteredFreeType); - CHECK(Luau::get(clonedTy)); + CHECK_EQ("any", toString(clonedTy)); CHECK(encounteredFreeType); encounteredFreeType = false; TypePackId clonedTp = clone(&freeTp, dest, seenTypes, seenTypePacks, &encounteredFreeType); - CHECK(Luau::get(clonedTp)); + CHECK_EQ("...any", toString(clonedTp)); CHECK(encounteredFreeType); } diff --git a/tests/ToString.test.cpp b/tests/ToString.test.cpp index b076e9ad..80a258f5 100644 --- a/tests/ToString.test.cpp +++ b/tests/ToString.test.cpp @@ -198,7 +198,8 @@ TEST_CASE_FIXTURE(Fixture, "stringifying_table_type_correctly_use_matching_table TypeVar tv{ttv}; - ToStringOptions o{/* exhaustive= */ false, /* useLineBreaks= */ false, /* functionTypeArguments= */ false, /* hideTableKind= */ false, 40}; + ToStringOptions o; + o.maxTableLength = 40; CHECK_EQ(toString(&tv, o), "{| a: number, b: number, c: number, d: number, e: number, ... 5 more ... |}"); } @@ -395,7 +396,7 @@ local function target(callback: nil) return callback(4, "hello") end )"); LUAU_REQUIRE_ERRORS(result); - CHECK_EQ(toString(requireType("target")), "(nil) -> (*unknown*)"); + CHECK_EQ("(nil) -> (*unknown*)", toString(requireType("target"))); } TEST_CASE_FIXTURE(Fixture, "toStringGenericPack") @@ -469,4 +470,110 @@ TEST_CASE_FIXTURE(Fixture, "self_recursive_instantiated_param") CHECK_EQ(toString(tableTy), "Table
"); } +TEST_CASE_FIXTURE(Fixture, "toStringNamedFunction_id") +{ + CheckResult result = check(R"( + local function id(x) return x end + )"); + + TypeId ty = requireType("id"); + const FunctionTypeVar* ftv = get(follow(ty)); + + CHECK_EQ("id(x: a): a", toStringNamedFunction("id", *ftv)); +} + +TEST_CASE_FIXTURE(Fixture, "toStringNamedFunction_map") +{ + CheckResult result = check(R"( + local function map(arr, fn) + local t = {} + for i = 0, #arr do + t[i] = fn(arr[i]) + end + return t + end + )"); + + TypeId ty = requireType("map"); + const FunctionTypeVar* ftv = get(follow(ty)); + + CHECK_EQ("map(arr: {a}, fn: (a) -> b): {b}", toStringNamedFunction("map", *ftv)); +} + +TEST_CASE("toStringNamedFunction_unit_f") +{ + TypePackVar empty{TypePack{}}; + FunctionTypeVar ftv{&empty, &empty, {}, false}; + CHECK_EQ("f(): ()", toStringNamedFunction("f", ftv)); +} + +TEST_CASE_FIXTURE(Fixture, "toStringNamedFunction_variadics") +{ + CheckResult result = check(R"( + local function f(x: a, ...): (a, a, b...) + return x, x, ... + end + )"); + + TypeId ty = requireType("f"); + auto ftv = get(follow(ty)); + + CHECK_EQ("f(x: a, ...: any): (a, a, b...)", toStringNamedFunction("f", *ftv)); +} + +TEST_CASE_FIXTURE(Fixture, "toStringNamedFunction_variadics2") +{ + CheckResult result = check(R"( + local function f(): ...number + return 1, 2, 3 + end + )"); + + TypeId ty = requireType("f"); + auto ftv = get(follow(ty)); + + CHECK_EQ("f(): ...number", toStringNamedFunction("f", *ftv)); +} + +TEST_CASE_FIXTURE(Fixture, "toStringNamedFunction_variadics3") +{ + CheckResult result = check(R"( + local function f(): (string, ...number) + return 'a', 1, 2, 3 + end + )"); + + TypeId ty = requireType("f"); + auto ftv = get(follow(ty)); + + CHECK_EQ("f(): (string, ...number)", toStringNamedFunction("f", *ftv)); +} + +TEST_CASE_FIXTURE(Fixture, "toStringNamedFunction_type_annotation_has_partial_argnames") +{ + CheckResult result = check(R"( + local f: (number, y: number) -> number + )"); + + TypeId ty = requireType("f"); + auto ftv = get(follow(ty)); + + CHECK_EQ("f(_: number, y: number): number", toStringNamedFunction("f", *ftv)); +} + +TEST_CASE_FIXTURE(Fixture, "toStringNamedFunction_hide_type_params") +{ + CheckResult result = check(R"( + local function f(x: T, g: (T) -> U)): () + end + )"); + + TypeId ty = requireType("f"); + auto ftv = get(follow(ty)); + + ToStringOptions opts; + opts.hideNamedFunctionTypeParameters = true; + CHECK_EQ("f(x: T, g: (T) -> U): ()", toStringNamedFunction("f", *ftv, opts)); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.aliases.test.cpp b/tests/TypeInfer.aliases.test.cpp index c27f8083..74ce155c 100644 --- a/tests/TypeInfer.aliases.test.cpp +++ b/tests/TypeInfer.aliases.test.cpp @@ -109,7 +109,7 @@ TEST_CASE_FIXTURE(Fixture, "dont_stop_typechecking_after_reporting_duplicate_typ CheckResult result = check(R"( type A = number type A = string -- Redefinition of type 'A', previously defined at line 1 - local foo: string = 1 -- No "Type 'number' could not be converted into 'string'" + local foo: string = 1 -- "Type 'number' could not be converted into 'string'" )"); LUAU_REQUIRE_ERROR_COUNT(2, result); diff --git a/tests/TypeInfer.annotations.test.cpp b/tests/TypeInfer.annotations.test.cpp index 2e400164..091c2f01 100644 --- a/tests/TypeInfer.annotations.test.cpp +++ b/tests/TypeInfer.annotations.test.cpp @@ -381,6 +381,8 @@ TEST_CASE_FIXTURE(Fixture, "typeof_expr") TEST_CASE_FIXTURE(Fixture, "corecursive_types_error_on_tight_loop") { + ScopedFastFlag sff{"LuauErrorRecoveryType", true}; + CheckResult result = check(R"( type A = B type B = A @@ -390,7 +392,7 @@ TEST_CASE_FIXTURE(Fixture, "corecursive_types_error_on_tight_loop") )"); TypeId fType = requireType("aa"); - const ErrorTypeVar* ftv = get(follow(fType)); + const AnyTypeVar* ftv = get(follow(fType)); REQUIRE(ftv != nullptr); REQUIRE(!result.errors.empty()); } diff --git a/tests/TypeInfer.generics.test.cpp b/tests/TypeInfer.generics.test.cpp index de2f0154..88c2dc85 100644 --- a/tests/TypeInfer.generics.test.cpp +++ b/tests/TypeInfer.generics.test.cpp @@ -289,7 +289,7 @@ TEST_CASE_FIXTURE(Fixture, "calling_self_generic_methods") end )"); // TODO: Should typecheck but currently errors CLI-39916 - LUAU_REQUIRE_ERROR_COUNT(1, result); + LUAU_REQUIRE_ERRORS(result); } TEST_CASE_FIXTURE(Fixture, "infer_generic_property") @@ -352,7 +352,7 @@ TEST_CASE_FIXTURE(Fixture, "dont_leak_generic_types") -- so this assignment should fail local b: boolean = f(true) )"); - LUAU_REQUIRE_ERROR_COUNT(2, result); + LUAU_REQUIRE_ERRORS(result); } TEST_CASE_FIXTURE(Fixture, "dont_leak_inferred_generic_types") @@ -368,7 +368,7 @@ TEST_CASE_FIXTURE(Fixture, "dont_leak_inferred_generic_types") local y: number = id(37) end )"); - LUAU_REQUIRE_ERROR_COUNT(1, result); + LUAU_REQUIRE_ERRORS(result); } TEST_CASE_FIXTURE(Fixture, "dont_substitute_bound_types") diff --git a/tests/TypeInfer.refinements.test.cpp b/tests/TypeInfer.refinements.test.cpp index 733fc39b..fe8e7ff9 100644 --- a/tests/TypeInfer.refinements.test.cpp +++ b/tests/TypeInfer.refinements.test.cpp @@ -704,9 +704,10 @@ TEST_CASE_FIXTURE(RefinementClassFixture, "typeguard_cast_free_table_to_vector") CHECK_EQ("Type '{+ X: a, Y: b, Z: c +}' could not be converted into 'Instance'", toString(result.errors[0])); else CHECK_EQ("Type '{- X: a, Y: b, Z: c -}' could not be converted into 'Instance'", toString(result.errors[0])); + CHECK_EQ("*unknown*", toString(requireTypeAtPosition({7, 28}))); // typeof(vec) == "Instance" - if (FFlag::LuauQuantifyInPlace2) + if (FFlag::LuauQuantifyInPlace2) CHECK_EQ("{+ X: a, Y: b, Z: c +}", toString(requireTypeAtPosition({9, 28}))); // type(vec) ~= "vector" and typeof(vec) ~= "Instance" else CHECK_EQ("{- X: a, Y: b, Z: c -}", toString(requireTypeAtPosition({9, 28}))); // type(vec) ~= "vector" and typeof(vec) ~= "Instance" diff --git a/tests/TypeInfer.singletons.test.cpp b/tests/TypeInfer.singletons.test.cpp new file mode 100644 index 00000000..5f95efd5 --- /dev/null +++ b/tests/TypeInfer.singletons.test.cpp @@ -0,0 +1,377 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details + +#include "Fixture.h" + +#include "doctest.h" +#include "Luau/BuiltinDefinitions.h" + +using namespace Luau; + +TEST_SUITE_BEGIN("TypeSingletons"); + +TEST_CASE_FIXTURE(Fixture, "bool_singletons") +{ + ScopedFastFlag sffs[] = { + {"LuauSingletonTypes", true}, + {"LuauParseSingletonTypes", true}, + }; + + CheckResult result = check(R"( + local a: true = true + local b: false = false + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "string_singletons") +{ + ScopedFastFlag sffs[] = { + {"LuauSingletonTypes", true}, + {"LuauParseSingletonTypes", true}, + }; + + CheckResult result = check(R"( + local a: "foo" = "foo" + local b: "bar" = "bar" + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "bool_singletons_mismatch") +{ + ScopedFastFlag sffs[] = { + {"LuauSingletonTypes", true}, + {"LuauParseSingletonTypes", true}, + }; + + CheckResult result = check(R"( + local a: true = false + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ("Type 'false' could not be converted into 'true'", toString(result.errors[0])); +} + +TEST_CASE_FIXTURE(Fixture, "string_singletons_mismatch") +{ + ScopedFastFlag sffs[] = { + {"LuauSingletonTypes", true}, + {"LuauParseSingletonTypes", true}, + }; + + CheckResult result = check(R"( + local a: "foo" = "bar" + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ("Type '\"bar\"' could not be converted into '\"foo\"'", toString(result.errors[0])); +} + +TEST_CASE_FIXTURE(Fixture, "string_singletons_escape_chars") +{ + ScopedFastFlag sffs[] = { + {"LuauSingletonTypes", true}, + {"LuauParseSingletonTypes", true}, + }; + + CheckResult result = check(R"( + local a: "\n" = "\000\r" + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ(R"(Type '"\000\r"' could not be converted into '"\n"')", toString(result.errors[0])); +} + +TEST_CASE_FIXTURE(Fixture, "bool_singleton_subtype") +{ + ScopedFastFlag sffs[] = { + {"LuauSingletonTypes", true}, + {"LuauParseSingletonTypes", true}, + }; + + CheckResult result = check(R"( + local a: true = true + local b: boolean = a + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "string_singleton_subtype") +{ + ScopedFastFlag sffs[] = { + {"LuauSingletonTypes", true}, + {"LuauParseSingletonTypes", true}, + }; + + CheckResult result = check(R"( + local a: "foo" = "foo" + local b: string = a + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "function_call_with_singletons") +{ + ScopedFastFlag sffs[] = { + {"LuauSingletonTypes", true}, + {"LuauParseSingletonTypes", true}, + }; + + CheckResult result = check(R"( + function f(a: true, b: "foo") end + f(true, "foo") + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "function_call_with_singletons_mismatch") +{ + ScopedFastFlag sffs[] = { + {"LuauSingletonTypes", true}, + {"LuauParseSingletonTypes", true}, + }; + + CheckResult result = check(R"( + function f(a: true, b: "foo") end + f(true, "bar") + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ("Type '\"bar\"' could not be converted into '\"foo\"'", toString(result.errors[0])); +} + +TEST_CASE_FIXTURE(Fixture, "overloaded_function_call_with_singletons") +{ + ScopedFastFlag sffs[] = { + {"LuauSingletonTypes", true}, + {"LuauParseSingletonTypes", true}, + }; + + CheckResult result = check(R"( + function f(a, b) end + local g : ((true, string) -> ()) & ((false, number) -> ()) = (f::any) + g(true, "foo") + g(false, 37) + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "overloaded_function_call_with_singletons_mismatch") +{ + ScopedFastFlag sffs[] = { + {"LuauSingletonTypes", true}, + {"LuauParseSingletonTypes", true}, + }; + + CheckResult result = check(R"( + function f(a, b) end + local g : ((true, string) -> ()) & ((false, number) -> ()) = (f::any) + g(true, 37) + )"); + + LUAU_REQUIRE_ERROR_COUNT(2, result); + CHECK_EQ("Type 'number' could not be converted into 'string'", toString(result.errors[0])); + CHECK_EQ("Other overloads are also not viable: (false, number) -> ()", toString(result.errors[1])); +} + +TEST_CASE_FIXTURE(Fixture, "enums_using_singletons") +{ + ScopedFastFlag sffs[] = { + {"LuauSingletonTypes", true}, + {"LuauParseSingletonTypes", true}, + }; + + CheckResult result = check(R"( + type MyEnum = "foo" | "bar" | "baz" + local a : MyEnum = "foo" + local b : MyEnum = "bar" + local c : MyEnum = "baz" + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "enums_using_singletons_mismatch") +{ + ScopedFastFlag sffs[] = { + {"LuauSingletonTypes", true}, + {"LuauParseSingletonTypes", true}, + {"LuauExtendedTypeMismatchError", true}, + }; + + CheckResult result = check(R"( + type MyEnum = "foo" | "bar" | "baz" + local a : MyEnum = "bang" + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ("Type '\"bang\"' could not be converted into '\"bar\" | \"baz\" | \"foo\"'; none of the union options are compatible", + toString(result.errors[0])); +} + +TEST_CASE_FIXTURE(Fixture, "enums_using_singletons_subtyping") +{ + ScopedFastFlag sffs[] = { + {"LuauSingletonTypes", true}, + {"LuauParseSingletonTypes", true}, + }; + + CheckResult result = check(R"( + type MyEnum1 = "foo" | "bar" + type MyEnum2 = MyEnum1 | "baz" + local a : MyEnum1 = "foo" + local b : MyEnum2 = a + local c : string = b + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "tagged_unions_using_singletons") +{ + ScopedFastFlag sffs[] = { + {"LuauSingletonTypes", true}, + {"LuauParseSingletonTypes", true}, + {"LuauExpectedTypesOfProperties", true}, + }; + + CheckResult result = check(R"( + type Dog = { tag: "Dog", howls: boolean } + type Cat = { tag: "Cat", meows: boolean } + type Animal = Dog | Cat + local a : Dog = { tag = "Dog", howls = true } + local b : Animal = { tag = "Cat", meows = true } + local c : Animal = a + c = b + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "tagged_unions_using_singletons_mismatch") +{ + ScopedFastFlag sffs[] = { + {"LuauSingletonTypes", true}, + {"LuauParseSingletonTypes", true}, + }; + + CheckResult result = check(R"( + type Dog = { tag: "Dog", howls: boolean } + type Cat = { tag: "Cat", meows: boolean } + type Animal = Dog | Cat + local a : Animal = { tag = "Cat", howls = true } + )"); + + LUAU_REQUIRE_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "tagged_unions_immutable_tag") +{ + ScopedFastFlag sffs[] = { + {"LuauSingletonTypes", true}, + {"LuauParseSingletonTypes", true}, + }; + + CheckResult result = check(R"( + type Dog = { tag: "Dog", howls: boolean } + type Cat = { tag: "Cat", meows: boolean } + type Animal = Dog | Cat + local a : Animal = { tag = "Cat", meows = true } + a.tag = "Dog" + )"); + + LUAU_REQUIRE_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "table_properties_singleton_strings") +{ + ScopedFastFlag sffs[] = { + {"LuauParseSingletonTypes", true}, + }; + + CheckResult result = check(R"( + --!strict + type T = { + ["foo"] : number, + ["$$bar"] : string, + baz : boolean + } + local t: T = { + ["foo"] = 37, + ["$$bar"] = "hi", + baz = true + } + local a: number = t.foo + local b: string = t["$$bar"] + local c: boolean = t.baz + t.foo = 5 + t["$$bar"] = "lo" + t.baz = false + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} +TEST_CASE_FIXTURE(Fixture, "table_properties_singleton_strings_mismatch") +{ + ScopedFastFlag sffs[] = { + {"LuauParseSingletonTypes", true}, + }; + + CheckResult result = check(R"( + --!strict + type T = { + ["$$bar"] : string, + } + local t: T = { + ["$$bar"] = "hi", + } + t["$$bar"] = 5 + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ("Type 'number' could not be converted into 'string'", toString(result.errors[0])); +} + +TEST_CASE_FIXTURE(Fixture, "table_properties_alias_or_parens_is_indexer") +{ + ScopedFastFlag sffs[] = { + {"LuauParseSingletonTypes", true}, + }; + + CheckResult result = check(R"( + --!strict + type S = "bar" + type T = { + [("foo")] : number, + [S] : string, + } + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ("Syntax error: Cannot have more than one table indexer", toString(result.errors[0])); +} + +TEST_CASE_FIXTURE(Fixture, "table_properties_type_error_escapes") +{ + ScopedFastFlag sffs[] = { + {"LuauParseSingletonTypes", true}, + }; + + CheckResult result = check(R"( + --!strict + local x: { ["<>"] : number } + x = { ["\n"] = 5 } + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ(R"(Table type '{| ["\n"]: number |}' not compatible with type '{| ["<>"]: number |}' because the former is missing field '<>')", + toString(result.errors[0])); +} + +TEST_SUITE_END(); diff --git a/tests/TypeInfer.test.cpp b/tests/TypeInfer.test.cpp index 30d9130a..99fd8339 100644 --- a/tests/TypeInfer.test.cpp +++ b/tests/TypeInfer.test.cpp @@ -362,7 +362,7 @@ TEST_CASE_FIXTURE(Fixture, "for_in_loop_on_error") CHECK_EQ(2, result.errors.size()); TypeId p = requireType("p"); - CHECK_EQ(*p, *typeChecker.errorType); + CHECK_EQ("*unknown*", toString(p)); } TEST_CASE_FIXTURE(Fixture, "for_in_loop_on_non_function") @@ -480,7 +480,7 @@ TEST_CASE_FIXTURE(Fixture, "for_in_loop_iterator_returns_any2") LUAU_REQUIRE_NO_ERRORS(result); - CHECK_EQ(typeChecker.anyType, requireType("a")); + CHECK_EQ("any", toString(requireType("a"))); } TEST_CASE_FIXTURE(Fixture, "for_in_loop_iterator_is_any") @@ -496,7 +496,7 @@ TEST_CASE_FIXTURE(Fixture, "for_in_loop_iterator_is_any") LUAU_REQUIRE_NO_ERRORS(result); - CHECK_EQ(typeChecker.anyType, requireType("a")); + CHECK_EQ("any", toString(requireType("a"))); } TEST_CASE_FIXTURE(Fixture, "for_in_loop_iterator_is_any2") @@ -512,7 +512,7 @@ TEST_CASE_FIXTURE(Fixture, "for_in_loop_iterator_is_any2") LUAU_REQUIRE_NO_ERRORS(result); - CHECK_EQ(typeChecker.anyType, requireType("a")); + CHECK_EQ("any", toString(requireType("a"))); } TEST_CASE_FIXTURE(Fixture, "for_in_loop_iterator_is_error") @@ -526,7 +526,7 @@ TEST_CASE_FIXTURE(Fixture, "for_in_loop_iterator_is_error") LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK_EQ(typeChecker.errorType, requireType("a")); + CHECK_EQ("*unknown*", toString(requireType("a"))); } TEST_CASE_FIXTURE(Fixture, "for_in_loop_iterator_is_error2") @@ -542,7 +542,7 @@ TEST_CASE_FIXTURE(Fixture, "for_in_loop_iterator_is_error2") LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK_EQ(typeChecker.errorType, requireType("a")); + CHECK_EQ("*unknown*", toString(requireType("a"))); } TEST_CASE_FIXTURE(Fixture, "for_in_loop_with_custom_iterator") @@ -673,7 +673,7 @@ TEST_CASE_FIXTURE(Fixture, "string_index") REQUIRE(nat); CHECK_EQ("string", toString(nat->ty)); - CHECK(get(requireType("t"))); + CHECK_EQ("*unknown*", toString(requireType("t"))); } TEST_CASE_FIXTURE(Fixture, "length_of_error_type_does_not_produce_an_error") @@ -1456,7 +1456,7 @@ TEST_CASE_FIXTURE(Fixture, "require_module_that_does_not_export") auto hootyType = requireType(bModule, "Hooty"); - CHECK_MESSAGE(get(follow(hootyType)) != nullptr, "Should be an error: " << toString(hootyType)); + CHECK_EQ("*unknown*", toString(hootyType)); } TEST_CASE_FIXTURE(Fixture, "warn_on_lowercase_parent_property") @@ -2032,7 +2032,7 @@ TEST_CASE_FIXTURE(Fixture, "higher_order_function_4") CHECK_EQ(*arg0->indexer->indexResultType, *arg1Args[1]); } -TEST_CASE_FIXTURE(Fixture, "error_types_propagate") +TEST_CASE_FIXTURE(Fixture, "type_errors_infer_types") { CheckResult result = check(R"( local err = (true).x @@ -2049,10 +2049,10 @@ TEST_CASE_FIXTURE(Fixture, "error_types_propagate") CHECK_EQ("boolean", toString(err->table)); CHECK_EQ("x", err->key); - CHECK(nullptr != get(requireType("c"))); - CHECK(nullptr != get(requireType("d"))); - CHECK(nullptr != get(requireType("e"))); - CHECK(nullptr != get(requireType("f"))); + CHECK_EQ("*unknown*", toString(requireType("c"))); + CHECK_EQ("*unknown*", toString(requireType("d"))); + CHECK_EQ("*unknown*", toString(requireType("e"))); + CHECK_EQ("*unknown*", toString(requireType("f"))); } TEST_CASE_FIXTURE(Fixture, "calling_error_type_yields_error") @@ -2068,7 +2068,7 @@ TEST_CASE_FIXTURE(Fixture, "calling_error_type_yields_error") CHECK_EQ("unknown", err->name); - CHECK(nullptr != get(requireType("a"))); + CHECK_EQ("*unknown*", toString(requireType("a"))); } TEST_CASE_FIXTURE(Fixture, "chain_calling_error_type_yields_error") @@ -2077,9 +2077,7 @@ TEST_CASE_FIXTURE(Fixture, "chain_calling_error_type_yields_error") local a = Utility.Create "Foo" {} )"); - TypeId aType = requireType("a"); - - REQUIRE_MESSAGE(nullptr != get(aType), "Not an error: " << toString(aType)); + CHECK_EQ("*unknown*", toString(requireType("a"))); } TEST_CASE_FIXTURE(Fixture, "primitive_arith_no_metatable") @@ -2146,6 +2144,8 @@ TEST_CASE_FIXTURE(Fixture, "some_primitive_binary_ops") TEST_CASE_FIXTURE(Fixture, "typecheck_overloaded_multiply_that_is_an_intersection") { + ScopedFastFlag sff{"LuauErrorRecoveryType", true}; + CheckResult result = check(R"( --!strict local Vec3 = {} @@ -2175,11 +2175,13 @@ TEST_CASE_FIXTURE(Fixture, "typecheck_overloaded_multiply_that_is_an_intersectio CHECK_EQ("Vec3", toString(requireType("b"))); CHECK_EQ("Vec3", toString(requireType("c"))); CHECK_EQ("Vec3", toString(requireType("d"))); - CHECK(get(requireType("e"))); + CHECK_EQ("Vec3", toString(requireType("e"))); } TEST_CASE_FIXTURE(Fixture, "typecheck_overloaded_multiply_that_is_an_intersection_on_rhs") { + ScopedFastFlag sff{"LuauErrorRecoveryType", true}; + CheckResult result = check(R"( --!strict local Vec3 = {} @@ -2209,7 +2211,7 @@ TEST_CASE_FIXTURE(Fixture, "typecheck_overloaded_multiply_that_is_an_intersectio CHECK_EQ("Vec3", toString(requireType("b"))); CHECK_EQ("Vec3", toString(requireType("c"))); CHECK_EQ("Vec3", toString(requireType("d"))); - CHECK(get(requireType("e"))); + CHECK_EQ("Vec3", toString(requireType("e"))); } TEST_CASE_FIXTURE(Fixture, "compare_numbers") @@ -2901,6 +2903,8 @@ end TEST_CASE_FIXTURE(Fixture, "CheckMethodsOfNumber") { + ScopedFastFlag sff{"LuauErrorRecoveryType", true}; + CheckResult result = check(R"( local x: number = 9999 function x:y(z: number) @@ -2908,7 +2912,7 @@ function x:y(z: number) end )"); - LUAU_REQUIRE_ERROR_COUNT(3, result); + LUAU_REQUIRE_ERROR_COUNT(2, result); } TEST_CASE_FIXTURE(Fixture, "CheckMethodsOfError") @@ -2920,7 +2924,7 @@ function x:y(z: number) end )"); - LUAU_REQUIRE_ERROR_COUNT(2, result); + LUAU_REQUIRE_ERRORS(result); } TEST_CASE_FIXTURE(Fixture, "CallOrOfFunctions") @@ -3799,7 +3803,7 @@ TEST_CASE_FIXTURE(Fixture, "UnknownGlobalCompoundAssign") print(a) )"); - LUAU_REQUIRE_ERROR_COUNT(2, result); + LUAU_REQUIRE_ERRORS(result); CHECK_EQ(toString(result.errors[0]), "Unknown global 'a'"); } @@ -4215,7 +4219,7 @@ TEST_CASE_FIXTURE(Fixture, "no_stack_overflow_from_quantifying") std::optional t0 = getMainModule()->getModuleScope()->lookupType("t0"); REQUIRE(t0); - CHECK(get(t0->type)); + CHECK_EQ("*unknown*", toString(t0->type)); auto it = std::find_if(result.errors.begin(), result.errors.end(), [](TypeError& err) { return get(err); @@ -4238,7 +4242,7 @@ TEST_CASE_FIXTURE(Fixture, "no_stack_overflow_from_isoptional") std::optional t0 = getMainModule()->getModuleScope()->lookupType("t0"); REQUIRE(t0); - CHECK(get(t0->type)); + CHECK_EQ("*unknown*", toString(t0->type)); auto it = std::find_if(result.errors.begin(), result.errors.end(), [](TypeError& err) { return get(err); @@ -4394,6 +4398,25 @@ TEST_CASE_FIXTURE(Fixture, "record_matching_overload") CHECK_EQ(toString(*it), "(number) -> number"); } +TEST_CASE_FIXTURE(Fixture, "return_type_by_overload") +{ + ScopedFastFlag sff{"LuauErrorRecoveryType", true}; + + CheckResult result = check(R"( + type Overload = ((string) -> string) & ((number, number) -> number) + local abc: Overload + local x = abc(true) + local y = abc(true,true) + local z = abc(true,true,true) + )"); + + LUAU_REQUIRE_ERRORS(result); + CHECK_EQ("string", toString(requireType("x"))); + CHECK_EQ("number", toString(requireType("y"))); + // Should this be string|number? + CHECK_EQ("string", toString(requireType("z"))); +} + TEST_CASE_FIXTURE(Fixture, "infer_anonymous_function_arguments") { // Simple direct arg to arg propagation @@ -4740,4 +4763,20 @@ TEST_CASE_FIXTURE(Fixture, "tc_if_else_expressions3") } } +TEST_CASE_FIXTURE(Fixture, "type_error_addition") +{ + CheckResult result = check(R"( +--!strict +local foo = makesandwich() +local bar = foo.nutrition + 100 + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + // We should definitely get this error + CHECK_EQ("Unknown global 'makesandwich'", toString(result.errors[0])); + // We get this error if makesandwich() returns a free type + // CHECK_EQ("Unknown type used in + operation; consider adding a type annotation to 'foo'", toString(result.errors[1])); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.tryUnify.test.cpp b/tests/TypeInfer.tryUnify.test.cpp index 2d697fc9..9f9a007f 100644 --- a/tests/TypeInfer.tryUnify.test.cpp +++ b/tests/TypeInfer.tryUnify.test.cpp @@ -121,9 +121,26 @@ TEST_CASE_FIXTURE(TryUnifyFixture, "members_of_failed_typepack_unification_are_u LUAU_REQUIRE_ERROR_COUNT(1, result); - TypeId bType = requireType("b"); + CHECK_EQ("a", toString(requireType("a"))); + CHECK_EQ("*unknown*", toString(requireType("b"))); +} - CHECK_MESSAGE(get(bType), "Should be an error: " << toString(bType)); +TEST_CASE_FIXTURE(TryUnifyFixture, "result_of_failed_typepack_unification_is_constrained") +{ + ScopedFastFlag sff{"LuauErrorRecoveryType", true}; + + CheckResult result = check(R"( + function f(arg: number) return arg end + local a + local b + local c = f(a, b) + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + CHECK_EQ("a", toString(requireType("a"))); + CHECK_EQ("*unknown*", toString(requireType("b"))); + CHECK_EQ("number", toString(requireType("c"))); } TEST_CASE_FIXTURE(TryUnifyFixture, "typepack_unification_should_trim_free_tails") @@ -167,15 +184,6 @@ TEST_CASE_FIXTURE(TryUnifyFixture, "variadic_tails_respect_progress") CHECK(state.errors.empty()); } -TEST_CASE_FIXTURE(TryUnifyFixture, "unifying_variadic_pack_with_error_should_work") -{ - TypePackId variadicPack = arena.addTypePack(TypePackVar{VariadicTypePack{typeChecker.numberType}}); - TypePackId errorPack = arena.addTypePack(TypePack{{typeChecker.numberType}, arena.addTypePack(TypePackVar{Unifiable::Error{}})}); - - state.tryUnify(variadicPack, errorPack); - REQUIRE_EQ(0, state.errors.size()); -} - TEST_CASE_FIXTURE(TryUnifyFixture, "variadics_should_use_reversed_properly") { CheckResult result = check(R"( diff --git a/tests/TypeInfer.unionTypes.test.cpp b/tests/TypeInfer.unionTypes.test.cpp index 9f29b642..48496b89 100644 --- a/tests/TypeInfer.unionTypes.test.cpp +++ b/tests/TypeInfer.unionTypes.test.cpp @@ -200,8 +200,7 @@ TEST_CASE_FIXTURE(Fixture, "index_on_a_union_type_with_missing_property") CHECK_EQ(mup->missing[0], *bTy); CHECK_EQ(mup->key, "x"); - TypeId r = requireType("r"); - CHECK_MESSAGE(get(r), "Expected error, got " << toString(r)); + CHECK_EQ("*unknown*", toString(requireType("r"))); } TEST_CASE_FIXTURE(Fixture, "index_on_a_union_type_with_one_property_of_type_any") @@ -283,7 +282,7 @@ local c = b:foo(1, 2) CHECK_EQ("Value of type 'A?' could be nil", toString(result.errors[0])); } -TEST_CASE_FIXTURE(Fixture, "optional_union_follow") +TEST_CASE_FIXTURE(UnfrozenFixture, "optional_union_follow") { CheckResult result = check(R"( local y: number? = 2 diff --git a/tests/conformance/coroutine.lua b/tests/conformance/coroutine.lua index 75329642..4d9b1295 100644 --- a/tests/conformance/coroutine.lua +++ b/tests/conformance/coroutine.lua @@ -319,4 +319,58 @@ for i=0,30 do assert(#T2 == 1 or T2[#T2] == 42) end +-- test coroutine.close +do + -- ok to close a dead coroutine + local co = coroutine.create(type) + assert(coroutine.resume(co, "testing 'coroutine.close'")) + assert(coroutine.status(co) == "dead") + local st, msg = coroutine.close(co) + assert(st and msg == nil) + -- also ok to close it again + st, msg = coroutine.close(co) + assert(st and msg == nil) + + + -- cannot close the running coroutine + coroutine.wrap(function() + local st, msg = pcall(coroutine.close, coroutine.running()) + assert(not st and string.find(msg, "running")) + end)() + + -- cannot close a "normal" coroutine + coroutine.wrap(function() + local co = coroutine.running() + coroutine.wrap(function () + local st, msg = pcall(coroutine.close, co) + assert(not st and string.find(msg, "normal")) + end)() + end)() + + -- closing a coroutine after an error + local co = coroutine.create(error) + local obj = {42} + local st, msg = coroutine.resume(co, obj) + assert(not st and msg == obj) + st, msg = coroutine.close(co) + assert(not st and msg == obj) + -- after closing, no more errors + st, msg = coroutine.close(co) + assert(st and msg == nil) + + -- closing a coroutine that has outstanding upvalues + local f + local co = coroutine.create(function() + local a = 42 + f = function() return a end + coroutine.yield() + a = 20 + end) + coroutine.resume(co) + assert(f() == 42) + st, msg = coroutine.close(co) + assert(st and msg == nil) + assert(f() == 42) +end + return'OK' diff --git a/tests/conformance/debugger.lua b/tests/conformance/debugger.lua index 5e69fc6b..6ba99fb9 100644 --- a/tests/conformance/debugger.lua +++ b/tests/conformance/debugger.lua @@ -45,4 +45,13 @@ breakpoint(38) -- break inside corobad() local co = coroutine.create(corobad) assert(coroutine.resume(co) == false) -- this breaks, resumes and dies! +function bar() + print("in bar") +end + +breakpoint(49) +breakpoint(49, false) -- validate that disabling breakpoints works + +bar() + return 'OK' From 100710c9f66334c2e98294c67a05de997d04cbe8 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Fri, 19 Nov 2021 08:15:56 -0800 Subject: [PATCH 064/399] Update README.md Switch to luau_compile and specify env for luau_load --- README.md | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index ffb464d0..d79a0bd6 100644 --- a/README.md +++ b/README.md @@ -40,8 +40,13 @@ make config=release luau luau-analyze To integrate Luau into your CMake application projects, at the minimum you'll need to depend on `Luau.Compiler` and `Luau.VM` projects. From there you need to create a new Luau state (using Lua 5.x API such as `lua_newstate`), compile source to bytecode and load it into the VM like this: ```cpp -std::string bytecode = Luau::compile(source); // needs Luau/Compiler.h include -if (luau_load(L, chunkname, bytecode.data(), bytecode.size()) == 0) +// needs lua.h and luacode.h +size_t bytecodeSize = 0; +char* bytecode = luau_compile(source, strlen(source), NULL, &bytecodeSize); +int result = luau_load(L, chunkname, bytecode, bytecodeSize, 0); +free(bytecode); + +if (result == 0) return 1; /* return chunk main function */ ``` From ec8a5643ccdb11c7236fc9f4b873548f1828f12b Mon Sep 17 00:00:00 2001 From: Josh Soref <2119212+jsoref@users.noreply.github.com> Date: Fri, 19 Nov 2021 13:45:53 -0500 Subject: [PATCH 065/399] Improve readability (#206) Co-authored-by: Josh Soref Co-authored-by: Arseny Kapoulkine --- bench/gc/test_LB_mandel.lua | 2 +- bench/tests/shootout/mandel.lua | 2 +- bench/tests/shootout/qt.lua | 10 +++++----- tests/conformance/basic.lua | 2 +- tests/conformance/closure.lua | 8 +------- tests/conformance/constructs.lua | 2 +- tests/conformance/coroutine.lua | 2 +- tests/conformance/datetime.lua | 2 +- tests/conformance/debug.lua | 2 +- tests/conformance/errors.lua | 32 ++++++++++++++++---------------- tests/conformance/gc.lua | 8 ++++---- tests/conformance/nextvar.lua | 6 +++--- tests/conformance/pcall.lua | 2 +- tests/conformance/utf8.lua | 2 +- 14 files changed, 38 insertions(+), 44 deletions(-) diff --git a/bench/gc/test_LB_mandel.lua b/bench/gc/test_LB_mandel.lua index 4be78502..fe5b4eb2 100644 --- a/bench/gc/test_LB_mandel.lua +++ b/bench/gc/test_LB_mandel.lua @@ -88,7 +88,7 @@ for i=1,N do local y=ymin+(j-1)*dy S = S + level(x,y) end - -- if i % 10 == 0 then print(collectgarbage"count") end + -- if i % 10 == 0 then print(collectgarbage("count")) end end print(S) diff --git a/bench/tests/shootout/mandel.lua b/bench/tests/shootout/mandel.lua index 4be78502..fe5b4eb2 100644 --- a/bench/tests/shootout/mandel.lua +++ b/bench/tests/shootout/mandel.lua @@ -88,7 +88,7 @@ for i=1,N do local y=ymin+(j-1)*dy S = S + level(x,y) end - -- if i % 10 == 0 then print(collectgarbage"count") end + -- if i % 10 == 0 then print(collectgarbage("count")) end end print(S) diff --git a/bench/tests/shootout/qt.lua b/bench/tests/shootout/qt.lua index de962a74..79cbe38b 100644 --- a/bench/tests/shootout/qt.lua +++ b/bench/tests/shootout/qt.lua @@ -275,7 +275,7 @@ local function memory(s) local t=os.clock() --local dt=string.format("%f",t-t0) local dt=t-t0 - --io.stdout:write(s,"\t",dt," sec\t",t," sec\t",math.floor(collectgarbage"count"/1024),"M\n") + --io.stdout:write(s,"\t",dt," sec\t",t," sec\t",math.floor(collectgarbage("count")/1024),"M\n") t0=t end @@ -286,7 +286,7 @@ local function do_(f,s) end local function julia(l,a,b) -memory"begin" +memory("begin") cx=a cy=b root=newcell() exterior=newcell() exterior.color=white @@ -297,14 +297,14 @@ memory"begin" do_(update,"update") repeat N=0 color(root,Rxmin,Rxmax,Rymin,Rymax) --print("color",N) - until N==0 memory"color" + until N==0 memory("color") repeat N=0 prewhite(root,Rxmin,Rxmax,Rymin,Rymax) --print("prewhite",N) - until N==0 memory"prewhite" + until N==0 memory("prewhite") do_(recolor,"recolor") do_(colorup,"colorup") --print("colorup",N) local g,b=do_(area,"area") --print("area",g,b,g+b) - show(i) memory"output" + show(i) memory("output") --print("edges",nE) end end diff --git a/tests/conformance/basic.lua b/tests/conformance/basic.lua index 687fff1e..693f68df 100644 --- a/tests/conformance/basic.lua +++ b/tests/conformance/basic.lua @@ -876,4 +876,4 @@ assert(concat(typeof(5), typeof(nil), typeof({}), typeof(newproxy())) == "number testgetfenv() -- DONT MOVE THIS LINE -return'OK' +return 'OK' diff --git a/tests/conformance/closure.lua b/tests/conformance/closure.lua index aac42c56..f32d5bdc 100644 --- a/tests/conformance/closure.lua +++ b/tests/conformance/closure.lua @@ -419,11 +419,5 @@ co = coroutine.create(function () return loadstring("return a")() end) -a = {a = 15} --- debug.setfenv(co, a) --- assert(debug.getfenv(co) == a) --- assert(select(2, coroutine.resume(co)) == a) --- assert(select(2, coroutine.resume(co)) == a.a) - -return'OK' +return 'OK' diff --git a/tests/conformance/constructs.lua b/tests/conformance/constructs.lua index 16c63b00..f133501f 100644 --- a/tests/conformance/constructs.lua +++ b/tests/conformance/constructs.lua @@ -237,4 +237,4 @@ repeat i = i+1 until i==c -return'OK' +return 'OK' diff --git a/tests/conformance/coroutine.lua b/tests/conformance/coroutine.lua index 4d9b1295..f2ecc96b 100644 --- a/tests/conformance/coroutine.lua +++ b/tests/conformance/coroutine.lua @@ -373,4 +373,4 @@ do assert(f() == 42) end -return'OK' +return 'OK' diff --git a/tests/conformance/datetime.lua b/tests/conformance/datetime.lua index 21ef60d7..ca35cf2f 100644 --- a/tests/conformance/datetime.lua +++ b/tests/conformance/datetime.lua @@ -74,4 +74,4 @@ assert(os.difftime(t1,t2) == 60*2-19) assert(os.time({ year = 1970, day = 1, month = 1, hour = 0}) == 0) -return'OK' +return 'OK' diff --git a/tests/conformance/debug.lua b/tests/conformance/debug.lua index ee79a14f..9cf3c742 100644 --- a/tests/conformance/debug.lua +++ b/tests/conformance/debug.lua @@ -98,4 +98,4 @@ assert(quuz(function(...) end) == "0 true") assert(quuz(function(a, b) end) == "2 false") assert(quuz(function(a, b, ...) end) == "2 true") -return'OK' +return 'OK' diff --git a/tests/conformance/errors.lua b/tests/conformance/errors.lua index eded14e9..d5ff215b 100644 --- a/tests/conformance/errors.lua +++ b/tests/conformance/errors.lua @@ -34,15 +34,15 @@ assert(doit("error('hi', 0)") == 'hi') assert(doit("unpack({}, 1, n=2^30)")) assert(doit("a=math.sin()")) assert(not doit("tostring(1)") and doit("tostring()")) -assert(doit"tonumber()") -assert(doit"repeat until 1; a") +assert(doit("tonumber()")) +assert(doit("repeat until 1; a")) checksyntax("break label", "", "label", 1) -assert(doit";") -assert(doit"a=1;;") -assert(doit"return;;") -assert(doit"assert(false)") -assert(doit"assert(nil)") -assert(doit"a=math.sin\n(3)") +assert(doit(";")) +assert(doit("a=1;;")) +assert(doit("return;;")) +assert(doit("assert(false)")) +assert(doit("assert(nil)")) +assert(doit("a=math.sin\n(3)")) assert(doit("function a (... , ...) end")) assert(doit("function a (, ...) end")) @@ -59,7 +59,7 @@ checkmessage("a=1; local a,bbbb=2,3; a = math.sin(1) and bbbb(3)", "local 'bbbb'") checkmessage("a={}; do local a=1 end a:bbbb(3)", "method 'bbbb'") checkmessage("local a={}; a.bbbb(3)", "field 'bbbb'") -assert(not string.find(doit"a={13}; local bbbb=1; a[bbbb](3)", "'bbbb'")) +assert(not string.find(doit("a={13}; local bbbb=1; a[bbbb](3)"), "'bbbb'")) checkmessage("a={13}; local bbbb=1; a[bbbb](3)", "number") aaa = nil @@ -67,14 +67,14 @@ checkmessage("aaa.bbb:ddd(9)", "global 'aaa'") checkmessage("local aaa={bbb=1}; aaa.bbb:ddd(9)", "field 'bbb'") checkmessage("local aaa={bbb={}}; aaa.bbb:ddd(9)", "method 'ddd'") checkmessage("local a,b,c; (function () a = b+1 end)()", "upvalue 'b'") -assert(not doit"local aaa={bbb={ddd=next}}; aaa.bbb:ddd(nil)") +assert(not doit("local aaa={bbb={ddd=next}}; aaa.bbb:ddd(nil)")) checkmessage("b=1; local aaa='a'; x=aaa+b", "local 'aaa'") checkmessage("aaa={}; x=3/aaa", "global 'aaa'") checkmessage("aaa='2'; b=nil;x=aaa*b", "global 'b'") checkmessage("aaa={}; x=-aaa", "global 'aaa'") -assert(not string.find(doit"aaa={}; x=(aaa or aaa)+(aaa and aaa)", "'aaa'")) -assert(not string.find(doit"aaa={}; (aaa or aaa)()", "'aaa'")) +assert(not string.find(doit("aaa={}; x=(aaa or aaa)+(aaa and aaa)"), "'aaa'")) +assert(not string.find(doit("aaa={}; (aaa or aaa)()"), "'aaa'")) checkmessage([[aaa=9 repeat until 3==3 @@ -122,10 +122,10 @@ function lineerror (s) return line and line+0 end -assert(lineerror"local a\n for i=1,'a' do \n print(i) \n end" == 2) --- assert(lineerror"\n local a \n for k,v in 3 \n do \n print(k) \n end" == 3) --- assert(lineerror"\n\n for k,v in \n 3 \n do \n print(k) \n end" == 4) -assert(lineerror"function a.x.y ()\na=a+1\nend" == 1) +assert(lineerror("local a\n for i=1,'a' do \n print(i) \n end") == 2) +-- assert(lineerror("\n local a \n for k,v in 3 \n do \n print(k) \n end") == 3) +-- assert(lineerror("\n\n for k,v in \n 3 \n do \n print(k) \n end") == 4) +assert(lineerror("function a.x.y ()\na=a+1\nend") == 1) local p = [[ function g() f() end diff --git a/tests/conformance/gc.lua b/tests/conformance/gc.lua index 4263dfda..6d9eb854 100644 --- a/tests/conformance/gc.lua +++ b/tests/conformance/gc.lua @@ -77,7 +77,7 @@ end local function dosteps (siz) collectgarbage() - collectgarbage"stop" + collectgarbage("stop") local a = {} for i=1,100 do a[i] = {{}}; local b = {} end local x = gcinfo() @@ -99,11 +99,11 @@ assert(dosteps(10000) == 1) do local x = gcinfo() collectgarbage() - collectgarbage"stop" + collectgarbage("stop") repeat local a = {} until gcinfo() > 1000 - collectgarbage"restart" + collectgarbage("restart") repeat local a = {} until gcinfo() < 1000 @@ -123,7 +123,7 @@ for n in pairs(b) do end b = nil collectgarbage() -for n in pairs(a) do error'cannot be here' end +for n in pairs(a) do error("cannot be here") end for i=1,lim do a[i] = i end for i=1,lim do assert(a[i] == i) end diff --git a/tests/conformance/nextvar.lua b/tests/conformance/nextvar.lua index 7f9b7596..94ba5ccf 100644 --- a/tests/conformance/nextvar.lua +++ b/tests/conformance/nextvar.lua @@ -368,9 +368,9 @@ assert(next(a,nil) == 1000 and next(a,1000) == nil) assert(next({}) == nil) assert(next({}, nil) == nil) -for a,b in pairs{} do error"not here" end -for i=1,0 do error'not here' end -for i=0,1,-1 do error'not here' end +for a,b in pairs{} do error("not here") end +for i=1,0 do error("not here") end +for i=0,1,-1 do error("not here") end a = nil; for i=1,1 do assert(not a); a=1 end; assert(a) a = nil; for i=1,1,-1 do assert(not a); a=1 end; assert(a) diff --git a/tests/conformance/pcall.lua b/tests/conformance/pcall.lua index a2072d2c..84ac2ba1 100644 --- a/tests/conformance/pcall.lua +++ b/tests/conformance/pcall.lua @@ -144,4 +144,4 @@ coroutine.resume(co) resumeerror(co, "fail") checkresults({ true, false, "fail" }, coroutine.resume(co)) -return'OK' +return 'OK' diff --git a/tests/conformance/utf8.lua b/tests/conformance/utf8.lua index 024cb16d..bfd7a1ac 100644 --- a/tests/conformance/utf8.lua +++ b/tests/conformance/utf8.lua @@ -205,4 +205,4 @@ for p, c in string.gmatch(x, "()(" .. utf8.charpattern .. ")") do end end -return'OK' +return 'OK' From 2fa5b9c329f6758bfbbfcceaff14209d0e0bdb9b Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Sun, 21 Nov 2021 20:07:44 -0800 Subject: [PATCH 066/399] Update repl.html Try using a release artifact --- docs/_includes/repl.html | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/_includes/repl.html b/docs/_includes/repl.html index 73c5ba40..1ff81fb8 100644 --- a/docs/_includes/repl.html +++ b/docs/_includes/repl.html @@ -47,4 +47,4 @@ } } - + From ffed1845620d98d5f48e5baa1d1da90b059b520d Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Sun, 21 Nov 2021 20:12:08 -0800 Subject: [PATCH 067/399] Update navigation.yml Add demo to top-level nav --- docs/_data/navigation.yml | 2 ++ 1 file changed, 2 insertions(+) diff --git a/docs/_data/navigation.yml b/docs/_data/navigation.yml index c828b2d4..7982fbc1 100644 --- a/docs/_data/navigation.yml +++ b/docs/_data/navigation.yml @@ -3,6 +3,8 @@ main: url: /news - title: Getting Started url: /getting-started + - title: Demo + url: /demo - title: GitHub url: https://github.com/Roblox/luau From cecd50fb062fbac8f4c63808c7de832578dcd035 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Sun, 21 Nov 2021 20:12:21 -0800 Subject: [PATCH 068/399] Update navigation.yml Remove leftover comments --- docs/_data/navigation.yml | 4 ---- 1 file changed, 4 deletions(-) diff --git a/docs/_data/navigation.yml b/docs/_data/navigation.yml index 7982fbc1..83172d80 100644 --- a/docs/_data/navigation.yml +++ b/docs/_data/navigation.yml @@ -29,7 +29,3 @@ pages: url: /profile - title: Library url: /library - -# Remove demo pages until solution is found -# - title: Demo -# url: /demo From a5bb3ee2af6ab06942e511f1ac6aa810d2e1fa5d Mon Sep 17 00:00:00 2001 From: Pelanyo Kamara Date: Mon, 22 Nov 2021 15:42:11 +0000 Subject: [PATCH 069/399] Add luaL_checkboolean and luaL_optboolean (#221) Co-authored-by: Arseny Kapoulkine --- VM/include/lualib.h | 3 +++ VM/src/laux.cpp | 16 ++++++++++++++++ tests/Conformance.test.cpp | 2 +- 3 files changed, 20 insertions(+), 1 deletion(-) diff --git a/VM/include/lualib.h b/VM/include/lualib.h index fa836955..54b008ff 100644 --- a/VM/include/lualib.h +++ b/VM/include/lualib.h @@ -25,6 +25,9 @@ LUALIB_API const char* luaL_optlstring(lua_State* L, int numArg, const char* def LUALIB_API double luaL_checknumber(lua_State* L, int numArg); LUALIB_API double luaL_optnumber(lua_State* L, int nArg, double def); +LUALIB_API int luaL_checkboolean(lua_State* L, int narg); +LUALIB_API int luaL_optboolean(lua_State* L, int narg, int def); + LUALIB_API int luaL_checkinteger(lua_State* L, int numArg); LUALIB_API int luaL_optinteger(lua_State* L, int nArg, int def); LUALIB_API unsigned luaL_checkunsigned(lua_State* L, int numArg); diff --git a/VM/src/laux.cpp b/VM/src/laux.cpp index 2a684ee4..5257271a 100644 --- a/VM/src/laux.cpp +++ b/VM/src/laux.cpp @@ -183,6 +183,22 @@ LUALIB_API double luaL_optnumber(lua_State* L, int narg, double def) return luaL_opt(L, luaL_checknumber, narg, def); } +LUALIB_API int luaL_checkboolean(lua_State* L, int narg) +{ + // This checks specifically for boolean values, ignoring + // all other truthy/falsy values. If the desired result + // is true if value is present then lua_toboolean should + // directly be used instead. + if (!lua_isboolean(L, narg)) + tag_error(L, narg, LUA_TBOOLEAN); + return lua_toboolean(L, narg); +} + +LUALIB_API int luaL_optboolean(lua_State* L, int narg, int def) +{ + return luaL_opt(L, luaL_checkboolean, narg, def); +} + LUALIB_API int luaL_checkinteger(lua_State* L, int narg) { int isnum; diff --git a/tests/Conformance.test.cpp b/tests/Conformance.test.cpp index e495a213..22977219 100644 --- a/tests/Conformance.test.cpp +++ b/tests/Conformance.test.cpp @@ -524,7 +524,7 @@ TEST_CASE("Debugger") L, [](lua_State* L) -> int { int line = luaL_checkinteger(L, 1); - bool enabled = lua_isboolean(L, 2) ? lua_toboolean(L, 2) : true; + bool enabled = luaL_optboolean(L, 2, true); lua_Debug ar = {}; lua_getinfo(L, 1, "f", &ar); From 2740f69f326a9b18e622571084bf0a03beff13b2 Mon Sep 17 00:00:00 2001 From: petrihakkinen Date: Mon, 22 Nov 2021 17:42:33 +0200 Subject: [PATCH 070/399] Expand vectors to 4 components using compile time switch (#214) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: Petri Häkkinen Co-authored-by: Arseny Kapoulkine --- VM/include/lua.h | 4 ++++ VM/include/luaconf.h | 4 ++++ VM/src/lapi.cpp | 13 +++++++++++-- VM/src/laux.cpp | 4 ++++ VM/src/lbuiltins.cpp | 12 +++++++++++- VM/src/lmem.cpp | 8 +++++++- VM/src/lnumutils.h | 8 ++++++++ VM/src/lobject.cpp | 2 +- VM/src/lobject.h | 23 ++++++++++++++++++----- VM/src/ltable.cpp | 19 +++++++++++++------ VM/src/lvmexecute.cpp | 30 ++++++++++++++++++------------ VM/src/lvmutils.cpp | 18 +++++++++--------- tests/Conformance.test.cpp | 11 +++++++++++ tests/conformance/vector.lua | 31 ++++++++++++++++++++++++++----- 14 files changed, 145 insertions(+), 42 deletions(-) diff --git a/VM/include/lua.h b/VM/include/lua.h index 1568d191..ebe7bbc5 100644 --- a/VM/include/lua.h +++ b/VM/include/lua.h @@ -159,7 +159,11 @@ LUA_API void lua_pushnil(lua_State* L); LUA_API void lua_pushnumber(lua_State* L, double n); LUA_API void lua_pushinteger(lua_State* L, int n); LUA_API void lua_pushunsigned(lua_State* L, unsigned n); +#if LUA_VECTOR_SIZE == 4 +LUA_API void lua_pushvector(lua_State* L, float x, float y, float z, float w); +#else LUA_API void lua_pushvector(lua_State* L, float x, float y, float z); +#endif LUA_API void lua_pushlstring(lua_State* L, const char* s, size_t l); LUA_API void lua_pushstring(lua_State* L, const char* s); LUA_API const char* lua_pushvfstring(lua_State* L, const char* fmt, va_list argp); diff --git a/VM/include/luaconf.h b/VM/include/luaconf.h index aa008a24..cce70a20 100644 --- a/VM/include/luaconf.h +++ b/VM/include/luaconf.h @@ -122,3 +122,7 @@ void* s; \ long l; \ } + +#define LUA_VECTOR_SIZE 3 /* must be 3 or 4 */ + +#define LUA_EXTRA_SIZE LUA_VECTOR_SIZE - 2 diff --git a/VM/src/lapi.cpp b/VM/src/lapi.cpp index a79ba0d4..832849ac 100644 --- a/VM/src/lapi.cpp +++ b/VM/src/lapi.cpp @@ -550,12 +550,21 @@ void lua_pushunsigned(lua_State* L, unsigned u) return; } -void lua_pushvector(lua_State* L, float x, float y, float z) +#if LUA_VECTOR_SIZE == 4 +void lua_pushvector(lua_State* L, float x, float y, float z, float w) { - setvvalue(L->top, x, y, z); + setvvalue(L->top, x, y, z, w); api_incr_top(L); return; } +#else +void lua_pushvector(lua_State* L, float x, float y, float z) +{ + setvvalue(L->top, x, y, z, 0.0f); + api_incr_top(L); + return; +} +#endif void lua_pushlstring(lua_State* L, const char* s, size_t len) { diff --git a/VM/src/laux.cpp b/VM/src/laux.cpp index 5257271a..c5133a58 100644 --- a/VM/src/laux.cpp +++ b/VM/src/laux.cpp @@ -478,7 +478,11 @@ LUALIB_API const char* luaL_tolstring(lua_State* L, int idx, size_t* len) case LUA_TVECTOR: { const float* v = lua_tovector(L, idx); +#if LUA_VECTOR_SIZE == 4 + lua_pushfstring(L, LUA_NUMBER_FMT ", " LUA_NUMBER_FMT ", " LUA_NUMBER_FMT ", " LUA_NUMBER_FMT, v[0], v[1], v[2], v[3]); +#else lua_pushfstring(L, LUA_NUMBER_FMT ", " LUA_NUMBER_FMT ", " LUA_NUMBER_FMT, v[0], v[1], v[2]); +#endif break; } default: diff --git a/VM/src/lbuiltins.cpp b/VM/src/lbuiltins.cpp index 9ab57ac9..34e9ebc1 100644 --- a/VM/src/lbuiltins.cpp +++ b/VM/src/lbuiltins.cpp @@ -1018,13 +1018,23 @@ static int luauF_tunpack(lua_State* L, StkId res, TValue* arg0, int nresults, St static int luauF_vector(lua_State* L, StkId res, TValue* arg0, int nresults, StkId args, int nparams) { +#if LUA_VECTOR_SIZE == 4 + if (nparams >= 4 && nresults <= 1 && ttisnumber(arg0) && ttisnumber(args) && ttisnumber(args + 1) && ttisnumber(args + 2)) +#else if (nparams >= 3 && nresults <= 1 && ttisnumber(arg0) && ttisnumber(args) && ttisnumber(args + 1)) +#endif { double x = nvalue(arg0); double y = nvalue(args); double z = nvalue(args + 1); - setvvalue(res, float(x), float(y), float(z)); +#if LUA_VECTOR_SIZE == 4 + double w = nvalue(args + 2); + setvvalue(res, float(x), float(y), float(z), float(w)); +#else + setvvalue(res, float(x), float(y), float(z), 0.0f); +#endif + return 1; } diff --git a/VM/src/lmem.cpp b/VM/src/lmem.cpp index d8b265cb..9f9d4a98 100644 --- a/VM/src/lmem.cpp +++ b/VM/src/lmem.cpp @@ -33,11 +33,17 @@ #define ABISWITCH(x64, ms32, gcc32) (sizeof(void*) == 8 ? x64 : ms32) #endif +#if LUA_VECTOR_SIZE == 4 +static_assert(sizeof(TValue) == ABISWITCH(24, 24, 24), "size mismatch for value"); +static_assert(sizeof(LuaNode) == ABISWITCH(48, 48, 48), "size mismatch for table entry"); +#else static_assert(sizeof(TValue) == ABISWITCH(16, 16, 16), "size mismatch for value"); +static_assert(sizeof(LuaNode) == ABISWITCH(32, 32, 32), "size mismatch for table entry"); +#endif + static_assert(offsetof(TString, data) == ABISWITCH(24, 20, 20), "size mismatch for string header"); static_assert(offsetof(Udata, data) == ABISWITCH(24, 16, 16), "size mismatch for userdata header"); static_assert(sizeof(Table) == ABISWITCH(56, 36, 36), "size mismatch for table header"); -static_assert(sizeof(LuaNode) == ABISWITCH(32, 32, 32), "size mismatch for table entry"); const size_t kSizeClasses = LUA_SIZECLASSES; const size_t kMaxSmallSize = 512; diff --git a/VM/src/lnumutils.h b/VM/src/lnumutils.h index 43f8014b..67f832dc 100644 --- a/VM/src/lnumutils.h +++ b/VM/src/lnumutils.h @@ -18,12 +18,20 @@ inline bool luai_veceq(const float* a, const float* b) { +#if LUA_VECTOR_SIZE == 4 + return a[0] == b[0] && a[1] == b[1] && a[2] == b[2] && a[3] == b[3]; +#else return a[0] == b[0] && a[1] == b[1] && a[2] == b[2]; +#endif } inline bool luai_vecisnan(const float* a) { +#if LUA_VECTOR_SIZE == 4 + return a[0] != a[0] || a[1] != a[1] || a[2] != a[2] || a[3] != a[3]; +#else return a[0] != a[0] || a[1] != a[1] || a[2] != a[2]; +#endif } LUAU_FASTMATH_BEGIN diff --git a/VM/src/lobject.cpp b/VM/src/lobject.cpp index bf13e6e9..370c7b28 100644 --- a/VM/src/lobject.cpp +++ b/VM/src/lobject.cpp @@ -15,7 +15,7 @@ -const TValue luaO_nilobject_ = {{NULL}, LUA_TNIL}; +const TValue luaO_nilobject_ = {{NULL}, {0}, LUA_TNIL}; int luaO_log2(unsigned int x) { diff --git a/VM/src/lobject.h b/VM/src/lobject.h index c5f2e2f4..ba040af6 100644 --- a/VM/src/lobject.h +++ b/VM/src/lobject.h @@ -47,7 +47,7 @@ typedef union typedef struct lua_TValue { Value value; - int extra; + int extra[LUA_EXTRA_SIZE]; int tt; } TValue; @@ -105,7 +105,19 @@ typedef struct lua_TValue i_o->tt = LUA_TNUMBER; \ } -#define setvvalue(obj, x, y, z) \ +#if LUA_VECTOR_SIZE == 4 +#define setvvalue(obj, x, y, z, w) \ + { \ + TValue* i_o = (obj); \ + float* i_v = i_o->value.v; \ + i_v[0] = (x); \ + i_v[1] = (y); \ + i_v[2] = (z); \ + i_v[3] = (w); \ + i_o->tt = LUA_TVECTOR; \ + } +#else +#define setvvalue(obj, x, y, z, w) \ { \ TValue* i_o = (obj); \ float* i_v = i_o->value.v; \ @@ -114,6 +126,7 @@ typedef struct lua_TValue i_v[2] = (z); \ i_o->tt = LUA_TVECTOR; \ } +#endif #define setpvalue(obj, x) \ { \ @@ -364,7 +377,7 @@ typedef struct Closure typedef struct TKey { ::Value value; - int extra; + int extra[LUA_EXTRA_SIZE]; unsigned tt : 4; int next : 28; /* for chaining */ } TKey; @@ -381,7 +394,7 @@ typedef struct LuaNode LuaNode* n_ = (node); \ const TValue* i_o = (obj); \ n_->key.value = i_o->value; \ - n_->key.extra = i_o->extra; \ + memcpy(n_->key.extra, i_o->extra, sizeof(n_->key.extra)); \ n_->key.tt = i_o->tt; \ checkliveness(L->global, i_o); \ } @@ -392,7 +405,7 @@ typedef struct LuaNode TValue* i_o = (obj); \ const LuaNode* n_ = (node); \ i_o->value = n_->key.value; \ - i_o->extra = n_->key.extra; \ + memcpy(i_o->extra, n_->key.extra, sizeof(i_o->extra)); \ i_o->tt = n_->key.tt; \ checkliveness(L->global, i_o); \ } diff --git a/VM/src/ltable.cpp b/VM/src/ltable.cpp index 07d22d59..0b55fcea 100644 --- a/VM/src/ltable.cpp +++ b/VM/src/ltable.cpp @@ -31,18 +31,19 @@ LUAU_FASTFLAGVARIABLE(LuauArrayBoundary, false) #define MAXSIZE (1 << MAXBITS) static_assert(offsetof(LuaNode, val) == 0, "Unexpected Node memory layout, pointer cast in gval2slot is incorrect"); + // TKey is bitpacked for memory efficiency so we need to validate bit counts for worst case -static_assert(TKey{{NULL}, 0, LUA_TDEADKEY, 0}.tt == LUA_TDEADKEY, "not enough bits for tt"); -static_assert(TKey{{NULL}, 0, LUA_TNIL, MAXSIZE - 1}.next == MAXSIZE - 1, "not enough bits for next"); -static_assert(TKey{{NULL}, 0, LUA_TNIL, -(MAXSIZE - 1)}.next == -(MAXSIZE - 1), "not enough bits for next"); +static_assert(TKey{{NULL}, {0}, LUA_TDEADKEY, 0}.tt == LUA_TDEADKEY, "not enough bits for tt"); +static_assert(TKey{{NULL}, {0}, LUA_TNIL, MAXSIZE - 1}.next == MAXSIZE - 1, "not enough bits for next"); +static_assert(TKey{{NULL}, {0}, LUA_TNIL, -(MAXSIZE - 1)}.next == -(MAXSIZE - 1), "not enough bits for next"); // reset cache of absent metamethods, cache is updated in luaT_gettm #define invalidateTMcache(t) t->flags = 0 // empty hash data points to dummynode so that we can always dereference it const LuaNode luaH_dummynode = { - {{NULL}, 0, LUA_TNIL}, /* value */ - {{NULL}, 0, LUA_TNIL, 0} /* key */ + {{NULL}, {0}, LUA_TNIL}, /* value */ + {{NULL}, {0}, LUA_TNIL, 0} /* key */ }; #define dummynode (&luaH_dummynode) @@ -96,7 +97,7 @@ static LuaNode* hashnum(const Table* t, double n) static LuaNode* hashvec(const Table* t, const float* v) { - unsigned int i[3]; + unsigned int i[LUA_VECTOR_SIZE]; memcpy(i, v, sizeof(i)); // convert -0 to 0 to make sure they hash to the same value @@ -112,6 +113,12 @@ static LuaNode* hashvec(const Table* t, const float* v) // Optimized Spatial Hashing for Collision Detection of Deformable Objects unsigned int h = (i[0] * 73856093) ^ (i[1] * 19349663) ^ (i[2] * 83492791); +#if LUA_VECTOR_SIZE == 4 + i[3] = (i[3] == 0x8000000) ? 0 : i[3]; + i[3] ^= i[3] >> 17; + h ^= i[3] * 39916801; +#endif + return hashpow2(t, h); } diff --git a/VM/src/lvmexecute.cpp b/VM/src/lvmexecute.cpp index eed2862b..bf8d493e 100644 --- a/VM/src/lvmexecute.cpp +++ b/VM/src/lvmexecute.cpp @@ -601,7 +601,13 @@ static void luau_execute(lua_State* L) const char* name = getstr(tsvalue(kv)); int ic = (name[0] | ' ') - 'x'; - if (unsigned(ic) < 3 && name[1] == '\0') +#if LUA_VECTOR_SIZE == 4 + // 'w' is before 'x' in ascii, so ic is -1 when indexing with 'w' + if (ic == -1) + ic = 3; +#endif + + if (unsigned(ic) < LUA_VECTOR_SIZE && name[1] == '\0') { setnvalue(ra, rb->value.v[ic]); VM_NEXT(); @@ -1526,7 +1532,7 @@ static void luau_execute(lua_State* L) { const float* vb = rb->value.v; const float* vc = rc->value.v; - setvvalue(ra, vb[0] + vc[0], vb[1] + vc[1], vb[2] + vc[2]); + setvvalue(ra, vb[0] + vc[0], vb[1] + vc[1], vb[2] + vc[2], vb[3] + vc[3]); VM_NEXT(); } else @@ -1572,7 +1578,7 @@ static void luau_execute(lua_State* L) { const float* vb = rb->value.v; const float* vc = rc->value.v; - setvvalue(ra, vb[0] - vc[0], vb[1] - vc[1], vb[2] - vc[2]); + setvvalue(ra, vb[0] - vc[0], vb[1] - vc[1], vb[2] - vc[2], vb[3] - vc[3]); VM_NEXT(); } else @@ -1618,21 +1624,21 @@ static void luau_execute(lua_State* L) { const float* vb = rb->value.v; float vc = cast_to(float, nvalue(rc)); - setvvalue(ra, vb[0] * vc, vb[1] * vc, vb[2] * vc); + setvvalue(ra, vb[0] * vc, vb[1] * vc, vb[2] * vc, vb[3] * vc); VM_NEXT(); } else if (ttisvector(rb) && ttisvector(rc)) { const float* vb = rb->value.v; const float* vc = rc->value.v; - setvvalue(ra, vb[0] * vc[0], vb[1] * vc[1], vb[2] * vc[2]); + setvvalue(ra, vb[0] * vc[0], vb[1] * vc[1], vb[2] * vc[2], vb[3] * vc[3]); VM_NEXT(); } else if (ttisnumber(rb) && ttisvector(rc)) { float vb = cast_to(float, nvalue(rb)); const float* vc = rc->value.v; - setvvalue(ra, vb * vc[0], vb * vc[1], vb * vc[2]); + setvvalue(ra, vb * vc[0], vb * vc[1], vb * vc[2], vb * vc[3]); VM_NEXT(); } else @@ -1679,21 +1685,21 @@ static void luau_execute(lua_State* L) { const float* vb = rb->value.v; float vc = cast_to(float, nvalue(rc)); - setvvalue(ra, vb[0] / vc, vb[1] / vc, vb[2] / vc); + setvvalue(ra, vb[0] / vc, vb[1] / vc, vb[2] / vc, vb[3] / vc); VM_NEXT(); } else if (ttisvector(rb) && ttisvector(rc)) { const float* vb = rb->value.v; const float* vc = rc->value.v; - setvvalue(ra, vb[0] / vc[0], vb[1] / vc[1], vb[2] / vc[2]); + setvvalue(ra, vb[0] / vc[0], vb[1] / vc[1], vb[2] / vc[2], vb[3] / vc[3]); VM_NEXT(); } else if (ttisnumber(rb) && ttisvector(rc)) { float vb = cast_to(float, nvalue(rb)); const float* vc = rc->value.v; - setvvalue(ra, vb / vc[0], vb / vc[1], vb / vc[2]); + setvvalue(ra, vb / vc[0], vb / vc[1], vb / vc[2], vb / vc[3]); VM_NEXT(); } else @@ -1826,7 +1832,7 @@ static void luau_execute(lua_State* L) { const float* vb = rb->value.v; float vc = cast_to(float, nvalue(kv)); - setvvalue(ra, vb[0] * vc, vb[1] * vc, vb[2] * vc); + setvvalue(ra, vb[0] * vc, vb[1] * vc, vb[2] * vc, vb[3] * vc); VM_NEXT(); } else @@ -1872,7 +1878,7 @@ static void luau_execute(lua_State* L) { const float* vb = rb->value.v; float vc = cast_to(float, nvalue(kv)); - setvvalue(ra, vb[0] / vc, vb[1] / vc, vb[2] / vc); + setvvalue(ra, vb[0] / vc, vb[1] / vc, vb[2] / vc, vb[3] / vc); VM_NEXT(); } else @@ -2037,7 +2043,7 @@ static void luau_execute(lua_State* L) else if (ttisvector(rb)) { const float* vb = rb->value.v; - setvvalue(ra, -vb[0], -vb[1], -vb[2]); + setvvalue(ra, -vb[0], -vb[1], -vb[2], -vb[3]); VM_NEXT(); } else diff --git a/VM/src/lvmutils.cpp b/VM/src/lvmutils.cpp index f52e8e74..740a4cfd 100644 --- a/VM/src/lvmutils.cpp +++ b/VM/src/lvmutils.cpp @@ -401,19 +401,19 @@ void luaV_doarith(lua_State* L, StkId ra, const TValue* rb, const TValue* rc, TM switch (op) { case TM_ADD: - setvvalue(ra, vb[0] + vc[0], vb[1] + vc[1], vb[2] + vc[2]); + setvvalue(ra, vb[0] + vc[0], vb[1] + vc[1], vb[2] + vc[2], vb[3] + vc[3]); return; case TM_SUB: - setvvalue(ra, vb[0] - vc[0], vb[1] - vc[1], vb[2] - vc[2]); + setvvalue(ra, vb[0] - vc[0], vb[1] - vc[1], vb[2] - vc[2], vb[3] - vc[3]); return; case TM_MUL: - setvvalue(ra, vb[0] * vc[0], vb[1] * vc[1], vb[2] * vc[2]); + setvvalue(ra, vb[0] * vc[0], vb[1] * vc[1], vb[2] * vc[2], vb[3] * vc[3]); return; case TM_DIV: - setvvalue(ra, vb[0] / vc[0], vb[1] / vc[1], vb[2] / vc[2]); + setvvalue(ra, vb[0] / vc[0], vb[1] / vc[1], vb[2] / vc[2], vb[3] / vc[3]); return; case TM_UNM: - setvvalue(ra, -vb[0], -vb[1], -vb[2]); + setvvalue(ra, -vb[0], -vb[1], -vb[2], -vb[3]); return; default: break; @@ -430,10 +430,10 @@ void luaV_doarith(lua_State* L, StkId ra, const TValue* rb, const TValue* rc, TM switch (op) { case TM_MUL: - setvvalue(ra, vb[0] * nc, vb[1] * nc, vb[2] * nc); + setvvalue(ra, vb[0] * nc, vb[1] * nc, vb[2] * nc, vb[3] * nc); return; case TM_DIV: - setvvalue(ra, vb[0] / nc, vb[1] / nc, vb[2] / nc); + setvvalue(ra, vb[0] / nc, vb[1] / nc, vb[2] / nc, vb[3] / nc); return; default: break; @@ -451,10 +451,10 @@ void luaV_doarith(lua_State* L, StkId ra, const TValue* rb, const TValue* rc, TM switch (op) { case TM_MUL: - setvvalue(ra, nb * vc[0], nb * vc[1], nb * vc[2]); + setvvalue(ra, nb * vc[0], nb * vc[1], nb * vc[2], nb * vc[3]); return; case TM_DIV: - setvvalue(ra, nb / vc[0], nb / vc[1], nb / vc[2]); + setvvalue(ra, nb / vc[0], nb / vc[1], nb / vc[2], nb / vc[3]); return; default: break; diff --git a/tests/Conformance.test.cpp b/tests/Conformance.test.cpp index 22977219..a573ae42 100644 --- a/tests/Conformance.test.cpp +++ b/tests/Conformance.test.cpp @@ -67,7 +67,12 @@ static int lua_vector(lua_State* L) double y = luaL_checknumber(L, 2); double z = luaL_checknumber(L, 3); +#if LUA_VECTOR_SIZE == 4 + double w = luaL_optnumber(L, 4, 0.0); + lua_pushvector(L, float(x), float(y), float(z), float(w)); +#else lua_pushvector(L, float(x), float(y), float(z)); +#endif return 1; } @@ -373,11 +378,17 @@ TEST_CASE("Pack") TEST_CASE("Vector") { + ScopedFastFlag sff{"LuauIfElseExpressionBaseSupport", true}; + runConformance("vector.lua", [](lua_State* L) { lua_pushcfunction(L, lua_vector, "vector"); lua_setglobal(L, "vector"); +#if LUA_VECTOR_SIZE == 4 + lua_pushvector(L, 0.0f, 0.0f, 0.0f, 0.0f); +#else lua_pushvector(L, 0.0f, 0.0f, 0.0f); +#endif luaL_newmetatable(L, "vector"); lua_pushstring(L, "__index"); diff --git a/tests/conformance/vector.lua b/tests/conformance/vector.lua index 620f646a..7d18bda3 100644 --- a/tests/conformance/vector.lua +++ b/tests/conformance/vector.lua @@ -1,6 +1,9 @@ -- This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details print('testing vectors') +-- detect vector size +local vector_size = if pcall(function() return vector(0, 0, 0).w end) then 4 else 3 + -- equality assert(vector(1, 2, 3) == vector(1, 2, 3)) assert(vector(0, 1, 2) == vector(-0, 1, 2)) @@ -13,8 +16,14 @@ assert(not rawequal(vector(1, 2, 3), vector(1, 2, 4))) -- type & tostring assert(type(vector(1, 2, 3)) == "vector") -assert(tostring(vector(1, 2, 3)) == "1, 2, 3") -assert(tostring(vector(-1, 2, 0.5)) == "-1, 2, 0.5") + +if vector_size == 4 then + assert(tostring(vector(1, 2, 3, 4)) == "1, 2, 3, 4") + assert(tostring(vector(-1, 2, 0.5, 0)) == "-1, 2, 0.5, 0") +else + assert(tostring(vector(1, 2, 3)) == "1, 2, 3") + assert(tostring(vector(-1, 2, 0.5)) == "-1, 2, 0.5") +end local t = {} @@ -42,12 +51,19 @@ assert(8 * vector(8, 16, 24) == vector(64, 128, 192)); assert(vector(1, 2, 4) * '8' == vector(8, 16, 32)); assert('8' * vector(8, 16, 24) == vector(64, 128, 192)); -assert(vector(1, 2, 4) / vector(8, 16, 24) == vector(1/8, 2/16, 4/24)); +if vector_size == 4 then + assert(vector(1, 2, 4, 8) / vector(8, 16, 24, 32) == vector(1/8, 2/16, 4/24, 8/32)); + assert(8 / vector(8, 16, 24, 32) == vector(1, 1/2, 1/3, 1/4)); + assert('8' / vector(8, 16, 24, 32) == vector(1, 1/2, 1/3, 1/4)); +else + assert(vector(1, 2, 4) / vector(8, 16, 24, 1) == vector(1/8, 2/16, 4/24)); + assert(8 / vector(8, 16, 24) == vector(1, 1/2, 1/3)); + assert('8' / vector(8, 16, 24) == vector(1, 1/2, 1/3)); +end + assert(vector(1, 2, 4) / 8 == vector(1/8, 1/4, 1/2)); assert(vector(1, 2, 4) / (1 / val) == vector(1/8, 2/8, 4/8)); -assert(8 / vector(8, 16, 24) == vector(1, 1/2, 1/3)); assert(vector(1, 2, 4) / '8' == vector(1/8, 1/4, 1/2)); -assert('8' / vector(8, 16, 24) == vector(1, 1/2, 1/3)); assert(-vector(1, 2, 4) == vector(-1, -2, -4)); @@ -71,4 +87,9 @@ assert(pcall(function() local t = {} rawset(t, vector(0/0, 2, 3), 1) end) == fal -- make sure we cover both builtin and C impl assert(vector(1, 2, 4) == vector("1", "2", "4")) +-- additional checks for 4-component vectors +if vector_size == 4 then + assert(vector(1, 2, 3, 4).w == 4) +end + return 'OK' From 5961261a1ce0a5eeb180b1ac3ea8718b2bec0be7 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Mon, 22 Nov 2021 09:59:15 -0800 Subject: [PATCH 071/399] Add web workflow to build Repl with Emscripten (#222) This also separates Emscripten build into a new target / source to make it more decoupled. --- .github/workflows/build.yml | 19 ++++++ .github/workflows/release.yml | 23 ++++++++ CLI/Repl.cpp | 39 ------------- CLI/Web.cpp | 106 ++++++++++++++++++++++++++++++++++ CMakeLists.txt | 57 +++++++++--------- Sources.cmake | 6 ++ 6 files changed, 184 insertions(+), 66 deletions(-) create mode 100644 CLI/Web.cpp diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml index 506a4c89..430ac0cb 100644 --- a/.github/workflows/build.yml +++ b/.github/workflows/build.yml @@ -83,3 +83,22 @@ jobs: with: name: coverage path: coverage + + web: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v1 + - uses: actions/checkout@v2 + with: + repository: emscripten-core/emsdk + path: emsdk + - name: emsdk install + run: | + cd emsdk + ./emsdk install latest + ./emsdk activate latest + - name: make + run: | + source emsdk/emsdk_env.sh + emcmake cmake . -DLUAU_BUILD_WEB=ON -DCMAKE_BUILD_TYPE=Release + make -j2 Luau.Web diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index 2fd8d3d5..e5cd952b 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -33,3 +33,26 @@ jobs: with: name: luau-${{matrix.os}} path: Release\luau*.exe + + web: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v1 + - uses: actions/checkout@v2 + with: + repository: emscripten-core/emsdk + path: emsdk + - name: emsdk install + run: | + cd emsdk + ./emsdk install latest + ./emsdk activate latest + - name: make + run: | + source emsdk/emsdk_env.sh + emcmake cmake . -DLUAU_BUILD_WEB=ON -DCMAKE_BUILD_TYPE=Release + make -j2 Luau.Web + - uses: actions/upload-artifact@v2 + with: + name: Luau.Web.js + path: Luau.Web.js diff --git a/CLI/Repl.cpp b/CLI/Repl.cpp index 410674fa..89a6037b 100644 --- a/CLI/Repl.cpp +++ b/CLI/Repl.cpp @@ -198,11 +198,6 @@ static std::string runCode(lua_State* L, const std::string& source) error += "\nstack backtrace:\n"; error += lua_debugtrace(T); -#ifdef __EMSCRIPTEN__ - // nicer formatting for errors in web repl - error = "Error:" + error; -#endif - fprintf(stdout, "%s", error.c_str()); } @@ -210,39 +205,6 @@ static std::string runCode(lua_State* L, const std::string& source) return std::string(); } -#ifdef __EMSCRIPTEN__ -extern "C" -{ - const char* executeScript(const char* source) - { - // setup flags - for (Luau::FValue* flag = Luau::FValue::list; flag; flag = flag->next) - if (strncmp(flag->name, "Luau", 4) == 0) - flag->value = true; - - // create new state - std::unique_ptr globalState(luaL_newstate(), lua_close); - lua_State* L = globalState.get(); - - // setup state - setupState(L); - - // sandbox thread - luaL_sandboxthread(L); - - // static string for caching result (prevents dangling ptr on function exit) - static std::string result; - - // run code + collect error - result = runCode(L, source); - - return result.empty() ? NULL : result.c_str(); - } -} -#endif - -// Excluded from emscripten compilation to avoid -Wunused-function errors. -#ifndef __EMSCRIPTEN__ static void completeIndexer(lua_State* L, const char* editBuffer, size_t start, std::vector& completions) { std::string_view lookup = editBuffer + start; @@ -564,5 +526,4 @@ int main(int argc, char** argv) return failed; } } -#endif diff --git a/CLI/Web.cpp b/CLI/Web.cpp new file mode 100644 index 00000000..cf5c831e --- /dev/null +++ b/CLI/Web.cpp @@ -0,0 +1,106 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +#include "lua.h" +#include "lualib.h" +#include "luacode.h" + +#include "Luau/Common.h" + +#include + +#include + +static void setupState(lua_State* L) +{ + luaL_openlibs(L); + + luaL_sandbox(L); +} + +static std::string runCode(lua_State* L, const std::string& source) +{ + size_t bytecodeSize = 0; + char* bytecode = luau_compile(source.data(), source.length(), nullptr, &bytecodeSize); + int result = luau_load(L, "=stdin", bytecode, bytecodeSize, 0); + free(bytecode); + + if (result != 0) + { + size_t len; + const char* msg = lua_tolstring(L, -1, &len); + + std::string error(msg, len); + lua_pop(L, 1); + + return error; + } + + lua_State* T = lua_newthread(L); + + lua_pushvalue(L, -2); + lua_remove(L, -3); + lua_xmove(L, T, 1); + + int status = lua_resume(T, NULL, 0); + + if (status == 0) + { + int n = lua_gettop(T); + + if (n) + { + luaL_checkstack(T, LUA_MINSTACK, "too many results to print"); + lua_getglobal(T, "print"); + lua_insert(T, 1); + lua_pcall(T, n, 0, 0); + } + } + else + { + std::string error; + + if (status == LUA_YIELD) + { + error = "thread yielded unexpectedly"; + } + else if (const char* str = lua_tostring(T, -1)) + { + error = str; + } + + error += "\nstack backtrace:\n"; + error += lua_debugtrace(T); + + error = "Error:" + error; + + fprintf(stdout, "%s", error.c_str()); + } + + lua_pop(L, 1); + return std::string(); +} + +extern "C" const char* executeScript(const char* source) +{ + // setup flags + for (Luau::FValue* flag = Luau::FValue::list; flag; flag = flag->next) + if (strncmp(flag->name, "Luau", 4) == 0) + flag->value = true; + + // create new state + std::unique_ptr globalState(luaL_newstate(), lua_close); + lua_State* L = globalState.get(); + + // setup state + setupState(L); + + // sandbox thread + luaL_sandboxthread(L); + + // static string for caching result (prevents dangling ptr on function exit) + static std::string result; + + // run code + collect error + result = runCode(L, source); + + return result.empty() ? NULL : result.c_str(); +} diff --git a/CMakeLists.txt b/CMakeLists.txt index 9c69521e..bafc59e5 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -9,6 +9,7 @@ project(Luau LANGUAGES CXX) option(LUAU_BUILD_CLI "Build CLI" ON) option(LUAU_BUILD_TESTS "Build tests" ON) +option(LUAU_BUILD_WEB "Build Web module" OFF) option(LUAU_WERROR "Warnings as errors" OFF) add_library(Luau.Ast STATIC) @@ -18,26 +19,22 @@ add_library(Luau.VM STATIC) if(LUAU_BUILD_CLI) add_executable(Luau.Repl.CLI) - if(NOT EMSCRIPTEN) - add_executable(Luau.Analyze.CLI) - else() - # add -fexceptions for emscripten to allow exceptions to be caught in C++ - set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -fexceptions") - endif() + add_executable(Luau.Analyze.CLI) # This also adds target `name` on Linux/macOS and `name.exe` on Windows set_target_properties(Luau.Repl.CLI PROPERTIES OUTPUT_NAME luau) - - if(NOT EMSCRIPTEN) - set_target_properties(Luau.Analyze.CLI PROPERTIES OUTPUT_NAME luau-analyze) - endif() + set_target_properties(Luau.Analyze.CLI PROPERTIES OUTPUT_NAME luau-analyze) endif() -if(LUAU_BUILD_TESTS AND NOT EMSCRIPTEN) +if(LUAU_BUILD_TESTS) add_executable(Luau.UnitTest) add_executable(Luau.Conformance) endif() +if(LUAU_BUILD_WEB) + add_executable(Luau.Web) +endif() + include(Sources.cmake) target_compile_features(Luau.Ast PUBLIC cxx_std_17) @@ -72,16 +69,18 @@ if(LUAU_WERROR) endif() endif() +if(LUAU_BUILD_WEB) + # add -fexceptions for emscripten to allow exceptions to be caught in C++ + list(APPEND LUAU_OPTIONS -fexceptions) +endif() + target_compile_options(Luau.Ast PRIVATE ${LUAU_OPTIONS}) target_compile_options(Luau.Analysis PRIVATE ${LUAU_OPTIONS}) target_compile_options(Luau.VM PRIVATE ${LUAU_OPTIONS}) if(LUAU_BUILD_CLI) target_compile_options(Luau.Repl.CLI PRIVATE ${LUAU_OPTIONS}) - - if(NOT EMSCRIPTEN) - target_compile_options(Luau.Analyze.CLI PRIVATE ${LUAU_OPTIONS}) - endif() + target_compile_options(Luau.Analyze.CLI PRIVATE ${LUAU_OPTIONS}) target_include_directories(Luau.Repl.CLI PRIVATE extern) target_link_libraries(Luau.Repl.CLI PRIVATE Luau.Compiler Luau.VM) @@ -93,20 +92,10 @@ if(LUAU_BUILD_CLI) endif() endif() - if(NOT EMSCRIPTEN) - target_link_libraries(Luau.Analyze.CLI PRIVATE Luau.Analysis) - endif() - - if(EMSCRIPTEN) - # declare exported functions to emscripten - target_link_options(Luau.Repl.CLI PRIVATE -sEXPORTED_FUNCTIONS=['_executeScript'] -sEXPORTED_RUNTIME_METHODS=['ccall','cwrap'] -fexceptions) - - # custom output directory for wasm + js file - set_target_properties(Luau.Repl.CLI PROPERTIES RUNTIME_OUTPUT_DIRECTORY ${CMAKE_SOURCE_DIR}/docs/assets/luau) - endif() + target_link_libraries(Luau.Analyze.CLI PRIVATE Luau.Analysis) endif() -if(LUAU_BUILD_TESTS AND NOT EMSCRIPTEN) +if(LUAU_BUILD_TESTS) target_compile_options(Luau.UnitTest PRIVATE ${LUAU_OPTIONS}) target_include_directories(Luau.UnitTest PRIVATE extern) target_link_libraries(Luau.UnitTest PRIVATE Luau.Analysis Luau.Compiler) @@ -115,3 +104,17 @@ if(LUAU_BUILD_TESTS AND NOT EMSCRIPTEN) target_include_directories(Luau.Conformance PRIVATE extern) target_link_libraries(Luau.Conformance PRIVATE Luau.Analysis Luau.Compiler Luau.VM) endif() + +if(LUAU_BUILD_WEB) + target_compile_options(Luau.Web PRIVATE ${LUAU_OPTIONS}) + target_link_libraries(Luau.Web PRIVATE Luau.Compiler Luau.VM) + + # declare exported functions to emscripten + target_link_options(Luau.Web PRIVATE -sEXPORTED_FUNCTIONS=['_executeScript'] -sEXPORTED_RUNTIME_METHODS=['ccall','cwrap']) + + # add -fexceptions for emscripten to allow exceptions to be caught in C++ + target_link_options(Luau.Web PRIVATE -fexceptions) + + # the output is a single .js file with an embedded wasm blob + target_link_options(Luau.Web PRIVATE -sSINGLE_FILE=1) +endif() diff --git a/Sources.cmake b/Sources.cmake index 23b931c6..440cc3ec 100644 --- a/Sources.cmake +++ b/Sources.cmake @@ -224,3 +224,9 @@ if(TARGET Luau.Conformance) tests/Conformance.test.cpp tests/main.cpp) endif() + +if(TARGET Luau.Web) + # Luau.Web Sources + target_sources(Luau.Web PRIVATE + CLI/Web.cpp) +endif() From a26024fb4b6dd9b02f22ab369dbaa839e9d0a525 Mon Sep 17 00:00:00 2001 From: Tiffany Bennett Date: Mon, 22 Nov 2021 12:54:27 -0800 Subject: [PATCH 072/399] Use latest release url (#227) --- docs/_includes/repl.html | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/_includes/repl.html b/docs/_includes/repl.html index 1ff81fb8..9d9b5895 100644 --- a/docs/_includes/repl.html +++ b/docs/_includes/repl.html @@ -47,4 +47,4 @@ } } - + From 6958716ccd3b1e377c9682b393e1d84309a5b896 Mon Sep 17 00:00:00 2001 From: Alexander McCord <11488393+alexmccord@users.noreply.github.com> Date: Mon, 22 Nov 2021 14:59:38 -0800 Subject: [PATCH 073/399] RFC: String interpolation (#165) --- rfcs/syntax-string-interpolation.md | 142 ++++++++++++++++++++++++++++ 1 file changed, 142 insertions(+) create mode 100644 rfcs/syntax-string-interpolation.md diff --git a/rfcs/syntax-string-interpolation.md b/rfcs/syntax-string-interpolation.md new file mode 100644 index 00000000..208143a0 --- /dev/null +++ b/rfcs/syntax-string-interpolation.md @@ -0,0 +1,142 @@ +# String interpolation + +## Summary + +New string interpolation syntax. + +## Motivation + +The problems with `string.format` are many. + +1. Must be exact about the types and its corresponding value. +2. Using `%d` is the idiomatic default for most people, but this loses precision. + * `%d` casts the number into `long long`, which has a lower max value than `double` and does not support decimals. + * `%f` by default will format to the millionths, e.g. `5.5` is `5.500000`. + * `%g` by default will format up to the hundred thousandths, e.g. `5.5` is `5.5` and `5.5312389` is `5.53123`. It will also convert the number to scientific notation when it encounters a number equal to or greater than 10^6. + * To not lose too much precision, you need to use `%s`, but even so the type checker assumes you actually wanted strings. +3. No support for `boolean`. You must use `%s` **and** call `tostring`. +4. No support for values implementing the `__tostring` metamethod. +5. Using `%` is in itself a dangerous operation within `string.format`. + * `"Your health is %d% so you need to heal up."` causes a runtime error because `% so` is actually parsed as `(%s)o` and now requires a corresponding string. +6. Having to use parentheses around string literals just to call a method of it. + +## Design + +To fix all of those issues, we need to do a few things. + +1. A new string interpolation expression (fixes #5, #6) +2. Extend `string.format` to accept values of arbitrary types (fixes #1, #2, #3, #4) + +Because we care about backward compatibility, we need some new syntax in order to not change the meaning of existing strings. There are a few components of this new expression: + +1. A string chunk (`` `...{ ``, `}...{`, and `` }...` ``) where `...` is a range of 0 to many characters. + * `\` escapes `` ` ``, `{`, and itself `\`. + * Restriction: the string interpolation literal must have at least one value to interpolate. We do not need 3 ways to express a single line string literal. + * The pairs must be on the same line (unless a `\` escapes the newline) but expressions needn't be on the same line. +2. An expression between the braces. This is the value that will be interpolated into the string. +3. Formatting specification may follow after the expression, delimited by an unambiguous character. + * Restriction: the formatting specification must be constant at parse time. + * In the absence of an explicit formatting specification, the `%*` token will be used. + * For now, we explicitly reject any formatting specification syntax. A future extension may be introduced to extend the syntax with an optional specification. + +To put the above into formal EBNF grammar: + +``` +stringinterp ::= exp { exp} +``` + +Which, in actual Luau code, will look like the following: + +``` +local world = "world" +print(`Hello {world}!`) +--> Hello world! + +local combo = {5, 2, 8, 9} +print(`The lock combinations are: {table.concat(combo, ", ")}`) +--> The lock combinations are: 5, 2, 8, 9 + +local set1 = Set.new({0, 1, 3}) +local set2 = Set.new({0, 5, 4}) +print(`{set1} ∪ {set2} = {Set.union(set1, set2)}`) +--> {0, 1, 3} ∪ {0, 5, 4} = {0, 1, 3, 4, 5} + +-- For illustrative purposes. These are illegal specifically because they don't interpolate anything. +print(`Some example escaping the braces \{like so}`) +print(`backslash \ that escapes the space is not a part of the string...`) +print(`backslash \\ will escape the second backslash...`) +print(`Some text that also includes \`...`) +--> Some example escaping the braces {like so} +--> backslash that escapes the space is not a part of the string... +--> backslash \ will escape the second backslash... +--> Some text that also includes `... +``` + +As for how newlines are handled, they are handled the same as other string literals. Any text between the `{}` delimiters are not considered part of the string, hence newlines are OK. The main thing is that one opening pair will scan until either a closing pair is encountered, or an unescaped newline. + +``` +local name = "Luau" + +print(`Welcome to { + name +}!`) +--> Welcome to Luau! + +print(`Welcome to \ +{name}!`) +--> Welcome to +-- Luau! +``` + +This expression will not be allowed to come after a `prefixexp`. I believe this is fully additive, so a future RFC may allow this. So for now, we explicitly reject the following: + +``` +local name = "world" +print`Hello {name}` +``` + +Since the string interpolation expression is going to be lowered into a `string.format` call, we'll also need to extend `string.format`. The bare minimum to support the lowering is to add a new token whose definition is to perform a `tostring` call. `%*` is currently an invalid token, so this is a backward compatible extension. This RFC shall define `%*` to have the same behavior as if `tostring` was called. + +```lua +print(string.format("%* %*", 1, 2)) +--> 1 2 +``` + +The offset must always be within bound of the numbers of values passed to `string.format`. + +```lua +local function return_one_thing() return "hi" end +local function return_two_nils() return nil, nil end + +print(string.format("%*", return_one_thing())) +--> "hi" + +print(string.format("%*", Set.new({1, 2, 3}))) +--> {1, 2, 3} + +print(string.format("%* %*", return_two_nils())) +--> nil nil + +print(string.format("%* %* %*", return_two_nils())) +--> error: value #3 is missing, got 2 +``` + +## Drawbacks + +If we want to use backticks for other purposes, it may introduce some potential ambiguity. One option to solve that is to only ever produce string interpolation tokens from the context of an expression. This is messy but doable because the parser and the lexer are already implemented to work in tandem. The other option is to pick a different delimiter syntax to keep backticks available for use in the future. + +If we were to naively compile the expression into a `string.format` call, then implementation details would be observable if you write `` `Your health is {hp}% so you need to heal up.` ``. When lowering the expression, we would need to implicitly insert a `%` character anytime one shows up in a string interpolation token. Otherwise attempting to run this will produce a runtime error where the `%s` token is missing its corresponding string value. + +## Alternatives + +Rather than coming up with a new syntax (which doesn't help issue #5 and #6) and extending `string.format` to accept an extra token, we could just make `%s` call `tostring` and be done. However, doing so would cause programs to be more lenient and the type checker would have no way to infer strings from a `string.format` call. To preserve that, we would need a different token anyway. + +Language | Syntax | Conclusion +----------:|:----------------------|:----------- +Python | `f'Hello {name}'` | Rejected because it's ambiguous with function call syntax. +Swift | `"Hello \(name)"` | Rejected because it changes the meaning of existing strings. +Ruby | `"Hello #{name}"` | Rejected because it changes the meaning of existing strings. +JavaScript | `` `Hello ${name}` `` | Viable option as long as we don't intend to use backticks for other purposes. +C# | `$"Hello {name}"` | Viable option and guarantees no ambiguities with future syntax. + +This leaves us with only two syntax that already exists in other programming languages. The current proposal are for backticks, so the only backward compatible alternative are `$""` literals. We don't necessarily need to use `$` symbol here, but if we were to choose a different symbol, `#` cannot be used. I picked backticks because it doesn't require us to add a stack of closing delimiters in the lexer to make sure each nested string interpolation literals are correctly closed with its opening pair. You only have to count them. From 5740686124137536aa93fa740983bb590677a660 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Tue, 23 Nov 2021 08:26:28 -0800 Subject: [PATCH 074/399] Ignore errors during upload coverage (#236) --- .github/workflows/build.yml | 1 + 1 file changed, 1 insertion(+) diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml index 430ac0cb..78da7b63 100644 --- a/.github/workflows/build.yml +++ b/.github/workflows/build.yml @@ -79,6 +79,7 @@ jobs: with: path-to-lcov: ./coverage.info github-token: ${{ secrets.GITHUB_TOKEN }} + continue-on-error: true - uses: actions/upload-artifact@v2 with: name: coverage From 6b2b179aa6d6c68adee535f36b1160ec630ecf61 Mon Sep 17 00:00:00 2001 From: vegorov-rbx <75688451+vegorov-rbx@users.noreply.github.com> Date: Tue, 23 Nov 2021 10:03:20 -0800 Subject: [PATCH 075/399] Mark 'Type alias type packs' RFC as implemented (#237) --- rfcs/syntax-type-alias-type-packs.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/rfcs/syntax-type-alias-type-packs.md b/rfcs/syntax-type-alias-type-packs.md index aa088299..d5bb6065 100644 --- a/rfcs/syntax-type-alias-type-packs.md +++ b/rfcs/syntax-type-alias-type-packs.md @@ -1,5 +1,7 @@ # Type alias type packs +**Status**: Implemented + ## Summary Provide semantics for referencing type packs inside the body of a type alias declaration From dd02420f707d24f41607d441abbbb3c977fa7e48 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Tue, 23 Nov 2021 11:44:18 -0800 Subject: [PATCH 076/399] Update build.yml Enable debug mode in coveralls action to diagnose https://github.com/lemurheavy/coveralls-public/issues/1595 --- .github/workflows/build.yml | 2 ++ 1 file changed, 2 insertions(+) diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml index 78da7b63..c2b4448f 100644 --- a/.github/workflows/build.yml +++ b/.github/workflows/build.yml @@ -66,6 +66,8 @@ jobs: coverage: runs-on: ubuntu-latest + env: + NODE_COVERALLS_DEBUG: 1 steps: - uses: actions/checkout@v1 - name: install From 222f03bbdafe16eff0b5072ddc8e2866b3408b95 Mon Sep 17 00:00:00 2001 From: Baileyeatspizza <68239467+Baileyeatspizza@users.noreply.github.com> Date: Mon, 29 Nov 2021 16:13:55 +0000 Subject: [PATCH 077/399] Update lmathlib.cpp (#241) --- VM/src/lmathlib.cpp | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/VM/src/lmathlib.cpp b/VM/src/lmathlib.cpp index 8e476a52..50bcf70c 100644 --- a/VM/src/lmathlib.cpp +++ b/VM/src/lmathlib.cpp @@ -385,8 +385,7 @@ static int math_sign(lua_State* L) static int math_round(lua_State* L) { - double v = luaL_checknumber(L, 1); - lua_pushnumber(L, round(v)); + lua_pushnumber(L, round(luaL_checknumber(L, 1))); return 1; } From f86d4c6995418e489a55be0100159009492778ff Mon Sep 17 00:00:00 2001 From: Lana Octavia Date: Mon, 29 Nov 2021 11:14:06 -0500 Subject: [PATCH 078/399] Removed LUALIB_API from source file method bodies (#235) --- VM/src/laux.cpp | 66 ++++++++++++++++++++++----------------------- VM/src/lbaselib.cpp | 2 +- VM/src/lbitlib.cpp | 2 +- VM/src/lcorolib.cpp | 2 +- VM/src/ldblib.cpp | 2 +- VM/src/linit.cpp | 8 +++--- VM/src/lmathlib.cpp | 2 +- VM/src/loslib.cpp | 2 +- VM/src/lstrlib.cpp | 2 +- VM/src/ltablib.cpp | 2 +- VM/src/lutf8lib.cpp | 2 +- 11 files changed, 46 insertions(+), 46 deletions(-) diff --git a/VM/src/laux.cpp b/VM/src/laux.cpp index c5133a58..a5e54358 100644 --- a/VM/src/laux.cpp +++ b/VM/src/laux.cpp @@ -30,7 +30,7 @@ static const char* currfuncname(lua_State* L) return debugname; } -LUALIB_API l_noret luaL_argerrorL(lua_State* L, int narg, const char* extramsg) +l_noret luaL_argerrorL(lua_State* L, int narg, const char* extramsg) { const char* fname = currfuncname(L); @@ -40,7 +40,7 @@ LUALIB_API l_noret luaL_argerrorL(lua_State* L, int narg, const char* extramsg) luaL_error(L, "invalid argument #%d (%s)", narg, extramsg); } -LUALIB_API l_noret luaL_typeerrorL(lua_State* L, int narg, const char* tname) +l_noret luaL_typeerrorL(lua_State* L, int narg, const char* tname) { const char* fname = currfuncname(L); const TValue* obj = luaA_toobject(L, narg); @@ -66,7 +66,7 @@ static l_noret tag_error(lua_State* L, int narg, int tag) luaL_typeerrorL(L, narg, lua_typename(L, tag)); } -LUALIB_API void luaL_where(lua_State* L, int level) +void luaL_where(lua_State* L, int level) { lua_Debug ar; if (lua_getinfo(L, level, "sl", &ar) && ar.currentline > 0) @@ -77,7 +77,7 @@ LUALIB_API void luaL_where(lua_State* L, int level) lua_pushliteral(L, ""); /* else, no information available... */ } -LUALIB_API l_noret luaL_errorL(lua_State* L, const char* fmt, ...) +l_noret luaL_errorL(lua_State* L, const char* fmt, ...) { va_list argp; va_start(argp, fmt); @@ -90,7 +90,7 @@ LUALIB_API l_noret luaL_errorL(lua_State* L, const char* fmt, ...) /* }====================================================== */ -LUALIB_API int luaL_checkoption(lua_State* L, int narg, const char* def, const char* const lst[]) +int luaL_checkoption(lua_State* L, int narg, const char* def, const char* const lst[]) { const char* name = (def) ? luaL_optstring(L, narg, def) : luaL_checkstring(L, narg); int i; @@ -101,7 +101,7 @@ LUALIB_API int luaL_checkoption(lua_State* L, int narg, const char* def, const c luaL_argerrorL(L, narg, msg); } -LUALIB_API int luaL_newmetatable(lua_State* L, const char* tname) +int luaL_newmetatable(lua_State* L, const char* tname) { lua_getfield(L, LUA_REGISTRYINDEX, tname); /* get registry.name */ if (!lua_isnil(L, -1)) /* name already in use? */ @@ -113,7 +113,7 @@ LUALIB_API int luaL_newmetatable(lua_State* L, const char* tname) return 1; } -LUALIB_API void* luaL_checkudata(lua_State* L, int ud, const char* tname) +void* luaL_checkudata(lua_State* L, int ud, const char* tname) { void* p = lua_touserdata(L, ud); if (p != NULL) @@ -131,25 +131,25 @@ LUALIB_API void* luaL_checkudata(lua_State* L, int ud, const char* tname) luaL_typeerrorL(L, ud, tname); /* else error */ } -LUALIB_API void luaL_checkstack(lua_State* L, int space, const char* mes) +void luaL_checkstack(lua_State* L, int space, const char* mes) { if (!lua_checkstack(L, space)) luaL_error(L, "stack overflow (%s)", mes); } -LUALIB_API void luaL_checktype(lua_State* L, int narg, int t) +void luaL_checktype(lua_State* L, int narg, int t) { if (lua_type(L, narg) != t) tag_error(L, narg, t); } -LUALIB_API void luaL_checkany(lua_State* L, int narg) +void luaL_checkany(lua_State* L, int narg) { if (lua_type(L, narg) == LUA_TNONE) luaL_error(L, "missing argument #%d", narg); } -LUALIB_API const char* luaL_checklstring(lua_State* L, int narg, size_t* len) +const char* luaL_checklstring(lua_State* L, int narg, size_t* len) { const char* s = lua_tolstring(L, narg, len); if (!s) @@ -157,7 +157,7 @@ LUALIB_API const char* luaL_checklstring(lua_State* L, int narg, size_t* len) return s; } -LUALIB_API const char* luaL_optlstring(lua_State* L, int narg, const char* def, size_t* len) +const char* luaL_optlstring(lua_State* L, int narg, const char* def, size_t* len) { if (lua_isnoneornil(L, narg)) { @@ -169,7 +169,7 @@ LUALIB_API const char* luaL_optlstring(lua_State* L, int narg, const char* def, return luaL_checklstring(L, narg, len); } -LUALIB_API double luaL_checknumber(lua_State* L, int narg) +double luaL_checknumber(lua_State* L, int narg) { int isnum; double d = lua_tonumberx(L, narg, &isnum); @@ -178,12 +178,12 @@ LUALIB_API double luaL_checknumber(lua_State* L, int narg) return d; } -LUALIB_API double luaL_optnumber(lua_State* L, int narg, double def) +double luaL_optnumber(lua_State* L, int narg, double def) { return luaL_opt(L, luaL_checknumber, narg, def); } -LUALIB_API int luaL_checkboolean(lua_State* L, int narg) +int luaL_checkboolean(lua_State* L, int narg) { // This checks specifically for boolean values, ignoring // all other truthy/falsy values. If the desired result @@ -194,12 +194,12 @@ LUALIB_API int luaL_checkboolean(lua_State* L, int narg) return lua_toboolean(L, narg); } -LUALIB_API int luaL_optboolean(lua_State* L, int narg, int def) +int luaL_optboolean(lua_State* L, int narg, int def) { return luaL_opt(L, luaL_checkboolean, narg, def); } -LUALIB_API int luaL_checkinteger(lua_State* L, int narg) +int luaL_checkinteger(lua_State* L, int narg) { int isnum; int d = lua_tointegerx(L, narg, &isnum); @@ -208,12 +208,12 @@ LUALIB_API int luaL_checkinteger(lua_State* L, int narg) return d; } -LUALIB_API int luaL_optinteger(lua_State* L, int narg, int def) +int luaL_optinteger(lua_State* L, int narg, int def) { return luaL_opt(L, luaL_checkinteger, narg, def); } -LUALIB_API unsigned luaL_checkunsigned(lua_State* L, int narg) +unsigned luaL_checkunsigned(lua_State* L, int narg) { int isnum; unsigned d = lua_tounsignedx(L, narg, &isnum); @@ -222,12 +222,12 @@ LUALIB_API unsigned luaL_checkunsigned(lua_State* L, int narg) return d; } -LUALIB_API unsigned luaL_optunsigned(lua_State* L, int narg, unsigned def) +unsigned luaL_optunsigned(lua_State* L, int narg, unsigned def) { return luaL_opt(L, luaL_checkunsigned, narg, def); } -LUALIB_API int luaL_getmetafield(lua_State* L, int obj, const char* event) +int luaL_getmetafield(lua_State* L, int obj, const char* event) { if (!lua_getmetatable(L, obj)) /* no metatable? */ return 0; @@ -245,7 +245,7 @@ LUALIB_API int luaL_getmetafield(lua_State* L, int obj, const char* event) } } -LUALIB_API int luaL_callmeta(lua_State* L, int obj, const char* event) +int luaL_callmeta(lua_State* L, int obj, const char* event) { obj = abs_index(L, obj); if (!luaL_getmetafield(L, obj, event)) /* no metafield? */ @@ -263,7 +263,7 @@ static int libsize(const luaL_Reg* l) return size; } -LUALIB_API void luaL_register(lua_State* L, const char* libname, const luaL_Reg* l) +void luaL_register(lua_State* L, const char* libname, const luaL_Reg* l) { if (libname) { @@ -289,7 +289,7 @@ LUALIB_API void luaL_register(lua_State* L, const char* libname, const luaL_Reg* } } -LUALIB_API const char* luaL_findtable(lua_State* L, int idx, const char* fname, int szhint) +const char* luaL_findtable(lua_State* L, int idx, const char* fname, int szhint) { const char* e; lua_pushvalue(L, idx); @@ -340,7 +340,7 @@ static size_t getnextbuffersize(lua_State* L, size_t currentsize, size_t desired return newsize; } -LUALIB_API void luaL_buffinit(lua_State* L, luaL_Buffer* B) +void luaL_buffinit(lua_State* L, luaL_Buffer* B) { // start with an internal buffer B->p = B->buffer; @@ -350,14 +350,14 @@ LUALIB_API void luaL_buffinit(lua_State* L, luaL_Buffer* B) B->storage = nullptr; } -LUALIB_API char* luaL_buffinitsize(lua_State* L, luaL_Buffer* B, size_t size) +char* luaL_buffinitsize(lua_State* L, luaL_Buffer* B, size_t size) { luaL_buffinit(L, B); luaL_reservebuffer(B, size, -1); return B->p; } -LUALIB_API char* luaL_extendbuffer(luaL_Buffer* B, size_t additionalsize, int boxloc) +char* luaL_extendbuffer(luaL_Buffer* B, size_t additionalsize, int boxloc) { lua_State* L = B->L; @@ -388,13 +388,13 @@ LUALIB_API char* luaL_extendbuffer(luaL_Buffer* B, size_t additionalsize, int bo return B->p; } -LUALIB_API void luaL_reservebuffer(luaL_Buffer* B, size_t size, int boxloc) +void luaL_reservebuffer(luaL_Buffer* B, size_t size, int boxloc) { if (size_t(B->end - B->p) < size) luaL_extendbuffer(B, size - (B->end - B->p), boxloc); } -LUALIB_API void luaL_addlstring(luaL_Buffer* B, const char* s, size_t len) +void luaL_addlstring(luaL_Buffer* B, const char* s, size_t len) { if (size_t(B->end - B->p) < len) luaL_extendbuffer(B, len - (B->end - B->p), -1); @@ -403,7 +403,7 @@ LUALIB_API void luaL_addlstring(luaL_Buffer* B, const char* s, size_t len) B->p += len; } -LUALIB_API void luaL_addvalue(luaL_Buffer* B) +void luaL_addvalue(luaL_Buffer* B) { lua_State* L = B->L; @@ -420,7 +420,7 @@ LUALIB_API void luaL_addvalue(luaL_Buffer* B) } } -LUALIB_API void luaL_pushresult(luaL_Buffer* B) +void luaL_pushresult(luaL_Buffer* B) { lua_State* L = B->L; @@ -444,7 +444,7 @@ LUALIB_API void luaL_pushresult(luaL_Buffer* B) } } -LUALIB_API void luaL_pushresultsize(luaL_Buffer* B, size_t size) +void luaL_pushresultsize(luaL_Buffer* B, size_t size) { B->p += size; luaL_pushresult(B); @@ -452,7 +452,7 @@ LUALIB_API void luaL_pushresultsize(luaL_Buffer* B, size_t size) /* }====================================================== */ -LUALIB_API const char* luaL_tolstring(lua_State* L, int idx, size_t* len) +const char* luaL_tolstring(lua_State* L, int idx, size_t* len) { if (luaL_callmeta(L, idx, "__tostring")) /* is there a metafield? */ { diff --git a/VM/src/lbaselib.cpp b/VM/src/lbaselib.cpp index 61798e2b..c3217f86 100644 --- a/VM/src/lbaselib.cpp +++ b/VM/src/lbaselib.cpp @@ -441,7 +441,7 @@ static void auxopen(lua_State* L, const char* name, lua_CFunction f, lua_CFuncti lua_setfield(L, -2, name); } -LUALIB_API int luaopen_base(lua_State* L) +int luaopen_base(lua_State* L) { /* set global _G */ lua_pushvalue(L, LUA_GLOBALSINDEX); diff --git a/VM/src/lbitlib.cpp b/VM/src/lbitlib.cpp index 907c43c4..8b511edf 100644 --- a/VM/src/lbitlib.cpp +++ b/VM/src/lbitlib.cpp @@ -236,7 +236,7 @@ static const luaL_Reg bitlib[] = { {NULL, NULL}, }; -LUALIB_API int luaopen_bit32(lua_State* L) +int luaopen_bit32(lua_State* L) { luaL_register(L, LUA_BITLIBNAME, bitlib); diff --git a/VM/src/lcorolib.cpp b/VM/src/lcorolib.cpp index 0178fae8..abcde779 100644 --- a/VM/src/lcorolib.cpp +++ b/VM/src/lcorolib.cpp @@ -272,7 +272,7 @@ static const luaL_Reg co_funcs[] = { {NULL, NULL}, }; -LUALIB_API int luaopen_coroutine(lua_State* L) +int luaopen_coroutine(lua_State* L) { luaL_register(L, LUA_COLIBNAME, co_funcs); diff --git a/VM/src/ldblib.cpp b/VM/src/ldblib.cpp index 965d2b3d..93d8703a 100644 --- a/VM/src/ldblib.cpp +++ b/VM/src/ldblib.cpp @@ -160,7 +160,7 @@ static const luaL_Reg dblib[] = { {NULL, NULL}, }; -LUALIB_API int luaopen_debug(lua_State* L) +int luaopen_debug(lua_State* L) { luaL_register(L, LUA_DBLIBNAME, dblib); return 1; diff --git a/VM/src/linit.cpp b/VM/src/linit.cpp index 4e40165a..0792c971 100644 --- a/VM/src/linit.cpp +++ b/VM/src/linit.cpp @@ -17,7 +17,7 @@ static const luaL_Reg lualibs[] = { {NULL, NULL}, }; -LUALIB_API void luaL_openlibs(lua_State* L) +void luaL_openlibs(lua_State* L) { const luaL_Reg* lib = lualibs; for (; lib->func; lib++) @@ -28,7 +28,7 @@ LUALIB_API void luaL_openlibs(lua_State* L) } } -LUALIB_API void luaL_sandbox(lua_State* L) +void luaL_sandbox(lua_State* L) { // set all libraries to read-only lua_pushnil(L); @@ -51,7 +51,7 @@ LUALIB_API void luaL_sandbox(lua_State* L) lua_setsafeenv(L, LUA_GLOBALSINDEX, true); } -LUALIB_API void luaL_sandboxthread(lua_State* L) +void luaL_sandboxthread(lua_State* L) { // create new global table that proxies reads to original table lua_newtable(L); @@ -81,7 +81,7 @@ static void* l_alloc(lua_State* L, void* ud, void* ptr, size_t osize, size_t nsi return realloc(ptr, nsize); } -LUALIB_API lua_State* luaL_newstate(void) +lua_State* luaL_newstate(void) { return lua_newstate(l_alloc, NULL); } diff --git a/VM/src/lmathlib.cpp b/VM/src/lmathlib.cpp index 50bcf70c..a6e7b494 100644 --- a/VM/src/lmathlib.cpp +++ b/VM/src/lmathlib.cpp @@ -428,7 +428,7 @@ static const luaL_Reg mathlib[] = { /* ** Open math library */ -LUALIB_API int luaopen_math(lua_State* L) +int luaopen_math(lua_State* L) { uint64_t seed = uintptr_t(L); seed ^= time(NULL); diff --git a/VM/src/loslib.cpp b/VM/src/loslib.cpp index 8eaef60c..b5901865 100644 --- a/VM/src/loslib.cpp +++ b/VM/src/loslib.cpp @@ -186,7 +186,7 @@ static const luaL_Reg syslib[] = { {NULL, NULL}, }; -LUALIB_API int luaopen_os(lua_State* L) +int luaopen_os(lua_State* L) { luaL_register(L, LUA_OSLIBNAME, syslib); return 1; diff --git a/VM/src/lstrlib.cpp b/VM/src/lstrlib.cpp index b576f809..0b3054ae 100644 --- a/VM/src/lstrlib.cpp +++ b/VM/src/lstrlib.cpp @@ -1657,7 +1657,7 @@ static void createmetatable(lua_State* L) /* ** Open string library */ -LUALIB_API int luaopen_string(lua_State* L) +int luaopen_string(lua_State* L) { luaL_register(L, LUA_STRLIBNAME, strlib); createmetatable(L); diff --git a/VM/src/ltablib.cpp b/VM/src/ltablib.cpp index 37025818..0d3374ef 100644 --- a/VM/src/ltablib.cpp +++ b/VM/src/ltablib.cpp @@ -527,7 +527,7 @@ static const luaL_Reg tab_funcs[] = { {NULL, NULL}, }; -LUALIB_API int luaopen_table(lua_State* L) +int luaopen_table(lua_State* L) { luaL_register(L, LUA_TABLIBNAME, tab_funcs); diff --git a/VM/src/lutf8lib.cpp b/VM/src/lutf8lib.cpp index 378de3d0..8bc8200a 100644 --- a/VM/src/lutf8lib.cpp +++ b/VM/src/lutf8lib.cpp @@ -283,7 +283,7 @@ static const luaL_Reg funcs[] = { {NULL, NULL}, }; -LUALIB_API int luaopen_utf8(lua_State* L) +int luaopen_utf8(lua_State* L) { luaL_register(L, LUA_UTF8LIBNAME, funcs); From 9aa9ff12dd75173ecba5b76cbb111f9f953b7088 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Petri=20H=C3=A4kkinen?= Date: Tue, 30 Nov 2021 18:14:12 +0200 Subject: [PATCH 079/399] Add LUA_GCCOUNTB option for lua_gc (#254) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: Petri Häkkinen --- VM/include/lua.h | 1 + VM/src/lapi.cpp | 5 +++++ 2 files changed, 6 insertions(+) diff --git a/VM/include/lua.h b/VM/include/lua.h index ebe7bbc5..a2c72213 100644 --- a/VM/include/lua.h +++ b/VM/include/lua.h @@ -231,6 +231,7 @@ enum lua_GCOp LUA_GCRESTART, LUA_GCCOLLECT, LUA_GCCOUNT, + LUA_GCCOUNTB, LUA_GCISRUNNING, // garbage collection is handled by 'assists' that perform some amount of GC work matching pace of allocation diff --git a/VM/src/lapi.cpp b/VM/src/lapi.cpp index 832849ac..3f4c0fd6 100644 --- a/VM/src/lapi.cpp +++ b/VM/src/lapi.cpp @@ -1039,6 +1039,11 @@ int lua_gc(lua_State* L, int what, int data) res = cast_int(g->totalbytes >> 10); break; } + case LUA_GCCOUNTB: + { + res = cast_int(g->totalbytes & 1023); + break; + } case LUA_GCISRUNNING: { res = (g->GCthreshold != SIZE_MAX); From 677994b243176c538f1f936b78604420095daed9 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Petri=20H=C3=A4kkinen?= Date: Tue, 30 Nov 2021 18:14:28 +0200 Subject: [PATCH 080/399] Fix: luaL_sandbox leaves value on the stack (#253) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: Petri Häkkinen --- VM/src/linit.cpp | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/VM/src/linit.cpp b/VM/src/linit.cpp index 0792c971..c93f431f 100644 --- a/VM/src/linit.cpp +++ b/VM/src/linit.cpp @@ -44,7 +44,7 @@ void luaL_sandbox(lua_State* L) lua_pushliteral(L, ""); lua_getmetatable(L, -1); lua_setreadonly(L, -1, true); - lua_pop(L, 1); + lua_pop(L, 2); // set globals to readonly and activate safeenv since the env is immutable lua_setreadonly(L, LUA_GLOBALSINDEX, true); From f185e9f5db29e939c74efd6c196c745b90a53b39 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Tue, 30 Nov 2021 08:21:11 -0800 Subject: [PATCH 081/399] Update performance.md (#252) Add documentation for closure allocation elision. --- docs/_pages/performance.md | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/docs/_pages/performance.md b/docs/_pages/performance.md index f54c5ab8..4e87ec5b 100644 --- a/docs/_pages/performance.md +++ b/docs/_pages/performance.md @@ -134,6 +134,12 @@ Lua implements upvalues as garbage collected objects that can point directly at Note that "immutable" in this case only refers to the variable itself - if the variable isn't assigned to it can be captured by value, even if it's a table that has its contents change. +## Closure caching + +With optimized upvalue storage, creating new closures (function objects) is more efficient but still requires allocating a new object every time. This can be problematic for cases when functions are passed to algorithms like `table.sort` or functions like `pcall`, as it results in excessive allocation traffic which then leads to more work for garbage collector. + +To make closure creation cheaper, Luau compiler implements closure caching - when multiple executions of the same function expression are guaranteed to result in the function object that is semantically identical, the compiler may cache the closure and always return the same object. This changes the function identity which may affect code that uses function objects as table keys, but preserves the calling semantics - compiler will only do this if calling the original (cached) function behaves the same way as a newly created function would. The heuristics used for this optimization are subject to change; currently, the compiler will cache closures that have no upvalues, or all upvalues are immutable (see previous section) and are declared at the module scope, as the module scope is (almost always) evaluated only once. + ## Fast memory allocator Similarly to LuaJIT, but unlike vanilla Lua, Luau implements a custom allocator that is highly specialized and tuned to the common allocation workloads we see. The allocator design is inspired by classic pool allocators as well as the excellent `mimalloc`, but through careful domain-specific tuning it beats all general purpose allocators we've tested, including `rpmalloc`, `mimalloc`, `jemalloc`, `ptmalloc` and `tcmalloc`. From 6801c65090daeb7f82e5e6d84e25213926b97a9c Mon Sep 17 00:00:00 2001 From: vegorov-rbx <75688451+vegorov-rbx@users.noreply.github.com> Date: Tue, 30 Nov 2021 14:00:23 -0800 Subject: [PATCH 082/399] Documentation for type packs (#257) * Documentation for type packs * Update docs/_pages/typecheck.md Co-authored-by: Arseny Kapoulkine * Add a note about the difference between ...T and T... Fix a typo at the start as well. Co-authored-by: Arseny Kapoulkine --- docs/_pages/typecheck.md | 60 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 60 insertions(+) diff --git a/docs/_pages/typecheck.md b/docs/_pages/typecheck.md index 58e66bb8..3580d66e 100644 --- a/docs/_pages/typecheck.md +++ b/docs/_pages/typecheck.md @@ -287,6 +287,66 @@ In type annotations, this is written as `...T`: type F = (...number) -> ...string ``` +## Type packs + +Multiple function return values as well as the function variadic parameter use a type pack to represent a list of types. + +When a type alias is defined, generic type pack parameters can be used after the type parameters: + +```lua +type Signal = { f: (T, U...) -> (), data: T } +``` + +> Keep in mind that `...T` is a variadic type pack (many elements of the same type `T`), while `U...` is a generic type pack that can contain zero or more types and they don't have to be the same. + +It is also possible for a generic function to reference a generic type pack from the generics list: + +```lua +local function call(s: Signal, ...: U...) + s.f(s.data, ...) +end +``` + +Generic types with type packs can be instantiated by providing a type pack: + +```lua +local signal: Signal = -- + +call(signal, 1, 2, false) +``` + +There are also other ways to instantiate types with generic type pack parameters: + +```lua +type A = (T) -> U... + +type B = A -- with a variadic type pack +type C = A -- with a generic type pack +type D = A -- with an empty type pack +``` + +Trailing type pack argument can also be provided without parentheses by specifying variadic type arguments: + +```lua +type List = (Head, Rest...) -> () + +type B = List -- Rest... is () +type C = List -- Rest is (string, boolean) + +type Returns = () -> T... + +-- When there are no type parameters, the list can be left empty +type D = Returns<> -- T... is () +``` + +Type pack parameters are not limited to a single one, as many as required can be specified: + +```lua +type Callback = { f: (Args...) -> Rets... } + +type A = Callback<(number, string), ...number> +``` + ## Typing idiomatic OOP One common pattern we see throughout Roblox is this OOP idiom. A downside with this pattern is that it does not automatically create a type binding for an instance of that class, so one has to write `type Account = typeof(Account.new("", 0))`. From 955f9fa754d2f7bf60dfcb81e112e50192f0a878 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Tue, 30 Nov 2021 15:26:28 -0800 Subject: [PATCH 083/399] Update index.md Studio => luau-analyze --- docs/index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/index.md b/docs/index.md index d7a344f8..3bea3b15 100644 --- a/docs/index.md +++ b/docs/index.md @@ -38,7 +38,7 @@ feature_row3: - title: Analysis excerpt: > - To make it easier to write correct code, Luau comes with a set of analysis tools that can surface common mistakes. These consist of a linter and a type checker, colloquially known as script analysis, and can be used from [Roblox Studio](https://developer.roblox.com/en-us/articles/The-Script-Analysis-Tool). The linting passes are [described here](lint), and the type checking user guide can [be found here](typecheck). + To make it easier to write correct code, Luau comes with a set of analysis tools that can surface common mistakes. These consist of a linter and a type checker, colloquially known as script analysis, and are integrated into `luau-analyze` command line executable. The linting passes are [described here](lint), and the type checking user guide can [be found here](typecheck). - title: Performance From 35e497b533571f7d439ef0ce622d10227d12baf4 Mon Sep 17 00:00:00 2001 From: kunitoki Date: Wed, 1 Dec 2021 02:03:18 +0100 Subject: [PATCH 084/399] Allow reconfiguring VM defaults (#260) Co-authored-by: Lucio Asnaghi --- VM/include/luaconf.h | 34 ++++++++++++++++++++++++++++++++++ 1 file changed, 34 insertions(+) diff --git a/VM/include/luaconf.h b/VM/include/luaconf.h index cce70a20..a01a1481 100644 --- a/VM/include/luaconf.h +++ b/VM/include/luaconf.h @@ -34,7 +34,10 @@ #endif /* Can be used to reconfigure visibility/exports for public APIs */ +#ifndef LUA_API #define LUA_API extern +#endif + #define LUALIB_API LUA_API /* Can be used to reconfigure visibility for internal APIs */ @@ -47,10 +50,14 @@ #endif /* Can be used to reconfigure internal error handling to use longjmp instead of C++ EH */ +#ifndef LUA_USE_LONGJMP #define LUA_USE_LONGJMP 0 +#endif /* LUA_IDSIZE gives the maximum size for the description of the source */ +#ifndef LUA_IDSIZE #define LUA_IDSIZE 256 +#endif /* @@ LUAI_GCGOAL defines the desired top heap size in relation to the live heap @@ -59,7 +66,9 @@ ** mean larger GC pauses which mean slower collection.) You can also change ** this value dynamically. */ +#ifndef LUAI_GCGOAL #define LUAI_GCGOAL 200 /* 200% (allow heap to double compared to live heap size) */ +#endif /* @@ LUAI_GCSTEPMUL / LUAI_GCSTEPSIZE define the default speed of garbage collection @@ -69,38 +78,63 @@ ** CHANGE it if you want to change the granularity of the garbage ** collection. */ +#ifndef LUAI_GCSTEPMUL #define LUAI_GCSTEPMUL 200 /* GC runs 'twice the speed' of memory allocation */ +#endif + +#ifndef LUAI_GCSTEPSIZE #define LUAI_GCSTEPSIZE 1 /* GC runs every KB of memory allocation */ +#endif /* LUA_MINSTACK is the guaranteed number of Lua stack slots available to a C function */ +#ifndef LUA_MINSTACK #define LUA_MINSTACK 20 +#endif /* LUAI_MAXCSTACK limits the number of Lua stack slots that a C function can use */ +#ifndef LUAI_MAXCSTACK #define LUAI_MAXCSTACK 8000 +#endif /* LUAI_MAXCALLS limits the number of nested calls */ +#ifndef LUAI_MAXCALLS #define LUAI_MAXCALLS 20000 +#endif /* LUAI_MAXCCALLS is the maximum depth for nested C calls; this limit depends on native stack size */ +#ifndef LUAI_MAXCCALLS #define LUAI_MAXCCALLS 200 +#endif /* buffer size used for on-stack string operations; this limit depends on native stack size */ +#ifndef LUA_BUFFERSIZE #define LUA_BUFFERSIZE 512 +#endif /* number of valid Lua userdata tags */ +#ifndef LUA_UTAG_LIMIT #define LUA_UTAG_LIMIT 128 +#endif /* upper bound for number of size classes used by page allocator */ +#ifndef LUA_SIZECLASSES #define LUA_SIZECLASSES 32 +#endif /* available number of separate memory categories */ +#ifndef LUA_MEMORY_CATEGORIES #define LUA_MEMORY_CATEGORIES 256 +#endif /* minimum size for the string table (must be power of 2) */ +#ifndef LUA_MINSTRTABSIZE #define LUA_MINSTRTABSIZE 32 +#endif /* maximum number of captures supported by pattern matching */ +#ifndef LUA_MAXCAPTURES #define LUA_MAXCAPTURES 32 +#endif /* }================================================================== */ From bf6cf4a69e05e1b7488a88f82c00deacb5c5d646 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Petri=20H=C3=A4kkinen?= Date: Wed, 1 Dec 2021 20:44:38 +0200 Subject: [PATCH 085/399] Fix luau_load 'env' to work with absolute stack index & add lua_absindex (#263) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: Petri Häkkinen --- VM/include/lua.h | 2 ++ VM/src/lapi.cpp | 6 ++++++ VM/src/lvmload.cpp | 6 +++--- 3 files changed, 11 insertions(+), 3 deletions(-) diff --git a/VM/include/lua.h b/VM/include/lua.h index a2c72213..77d57f90 100644 --- a/VM/include/lua.h +++ b/VM/include/lua.h @@ -21,6 +21,7 @@ #define LUA_ENVIRONINDEX (-10001) #define LUA_GLOBALSINDEX (-10002) #define lua_upvalueindex(i) (LUA_GLOBALSINDEX - (i)) +#define lua_ispseudo(i) ((i) <= LUA_REGISTRYINDEX) /* thread status; 0 is OK */ enum lua_Status @@ -108,6 +109,7 @@ LUA_API int lua_isthreadreset(lua_State* L); /* ** basic stack manipulation */ +LUA_API int lua_absindex(lua_State* L, int idx); LUA_API int lua_gettop(lua_State* L); LUA_API void lua_settop(lua_State* L, int idx); LUA_API void lua_pushvalue(lua_State* L, int idx); diff --git a/VM/src/lapi.cpp b/VM/src/lapi.cpp index 3f4c0fd6..25c29e3b 100644 --- a/VM/src/lapi.cpp +++ b/VM/src/lapi.cpp @@ -170,6 +170,12 @@ lua_State* lua_mainthread(lua_State* L) ** basic stack manipulation */ +int lua_absindex(lua_State* L, int idx) +{ + api_check(L, (idx > 0 && idx <= L->top - L->base) || (idx < 0 && -idx <= L->top - L->base) || lua_ispseudo(idx)); + return idx > 0 || lua_ispseudo(idx) ? idx : cast_int(L->top - L->base) + idx + 1; +} + int lua_gettop(lua_State* L) { return cast_int(L->top - L->base); diff --git a/VM/src/lvmload.cpp b/VM/src/lvmload.cpp index a168b652..add3588d 100644 --- a/VM/src/lvmload.cpp +++ b/VM/src/lvmload.cpp @@ -9,6 +9,7 @@ #include "lgc.h" #include "lmem.h" #include "lbytecode.h" +#include "lapi.h" #include @@ -162,9 +163,8 @@ int luau_load(lua_State* L, const char* chunkname, const char* data, size_t size size_t GCthreshold = L->global->GCthreshold; L->global->GCthreshold = SIZE_MAX; - // env is 0 for current environment and a stack relative index otherwise - LUAU_ASSERT(env <= 0 && L->top - L->base >= -env); - Table* envt = (env == 0) ? hvalue(gt(L)) : hvalue(L->top + env); + // env is 0 for current environment and a stack index otherwise + Table* envt = (env == 0) ? hvalue(gt(L)) : hvalue(luaA_toobject(L, env)); TString* source = luaS_new(L, chunkname); From d2bf2870e89f6b71d8656b2ff487dc96e6e92400 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Petri=20H=C3=A4kkinen?= Date: Wed, 1 Dec 2021 21:03:08 +0200 Subject: [PATCH 086/399] Add lua_isvector, luaL_checkvector and luaL_optvector (#261) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: Petri Häkkinen Co-authored-by: Arseny Kapoulkine --- VM/include/lua.h | 1 + VM/include/lualib.h | 3 +++ VM/src/laux.cpp | 13 +++++++++++++ tests/Conformance.test.cpp | 39 ++++++++++++++++---------------------- 4 files changed, 33 insertions(+), 23 deletions(-) diff --git a/VM/include/lua.h b/VM/include/lua.h index 77d57f90..f9da943b 100644 --- a/VM/include/lua.h +++ b/VM/include/lua.h @@ -296,6 +296,7 @@ LUA_API void lua_unref(lua_State* L, int ref); #define lua_islightuserdata(L, n) (lua_type(L, (n)) == LUA_TLIGHTUSERDATA) #define lua_isnil(L, n) (lua_type(L, (n)) == LUA_TNIL) #define lua_isboolean(L, n) (lua_type(L, (n)) == LUA_TBOOLEAN) +#define lua_isvector(L, n) (lua_type(L, (n)) == LUA_TVECTOR) #define lua_isthread(L, n) (lua_type(L, (n)) == LUA_TTHREAD) #define lua_isnone(L, n) (lua_type(L, (n)) == LUA_TNONE) #define lua_isnoneornil(L, n) (lua_type(L, (n)) <= LUA_TNIL) diff --git a/VM/include/lualib.h b/VM/include/lualib.h index 54b008ff..baf27b47 100644 --- a/VM/include/lualib.h +++ b/VM/include/lualib.h @@ -33,6 +33,9 @@ LUALIB_API int luaL_optinteger(lua_State* L, int nArg, int def); LUALIB_API unsigned luaL_checkunsigned(lua_State* L, int numArg); LUALIB_API unsigned luaL_optunsigned(lua_State* L, int numArg, unsigned def); +LUALIB_API const float* luaL_checkvector(lua_State* L, int narg); +LUALIB_API const float* luaL_optvector(lua_State* L, int narg, const float* def); + LUALIB_API void luaL_checkstack(lua_State* L, int sz, const char* msg); LUALIB_API void luaL_checktype(lua_State* L, int narg, int t); LUALIB_API void luaL_checkany(lua_State* L, int narg); diff --git a/VM/src/laux.cpp b/VM/src/laux.cpp index a5e54358..7ed2a62e 100644 --- a/VM/src/laux.cpp +++ b/VM/src/laux.cpp @@ -227,6 +227,19 @@ unsigned luaL_optunsigned(lua_State* L, int narg, unsigned def) return luaL_opt(L, luaL_checkunsigned, narg, def); } +const float* luaL_checkvector(lua_State* L, int narg) +{ + const float* v = lua_tovector(L, narg); + if (!v) + tag_error(L, narg, LUA_TVECTOR); + return v; +} + +const float* luaL_optvector(lua_State* L, int narg, const float* def) +{ + return luaL_opt(L, luaL_checkvector, narg, def); +} + int luaL_getmetafield(lua_State* L, int obj, const char* event) { if (!lua_getmetatable(L, obj)) /* no metatable? */ diff --git a/tests/Conformance.test.cpp b/tests/Conformance.test.cpp index a573ae42..5ee84244 100644 --- a/tests/Conformance.test.cpp +++ b/tests/Conformance.test.cpp @@ -78,38 +78,31 @@ static int lua_vector(lua_State* L) static int lua_vector_dot(lua_State* L) { - const float* a = lua_tovector(L, 1); - const float* b = lua_tovector(L, 2); + const float* a = luaL_checkvector(L, 1); + const float* b = luaL_checkvector(L, 2); - if (a && b) - { - lua_pushnumber(L, a[0] * b[0] + a[1] * b[1] + a[2] * b[2]); - return 1; - } - - throw std::runtime_error("invalid arguments to vector:Dot"); + lua_pushnumber(L, a[0] * b[0] + a[1] * b[1] + a[2] * b[2]); + return 1; } static int lua_vector_index(lua_State* L) { + const float* v = luaL_checkvector(L, 1); const char* name = luaL_checkstring(L, 2); - if (const float* v = lua_tovector(L, 1)) + if (strcmp(name, "Magnitude") == 0) { - if (strcmp(name, "Magnitude") == 0) - { - lua_pushnumber(L, sqrtf(v[0] * v[0] + v[1] * v[1] + v[2] * v[2])); - return 1; - } - - if (strcmp(name, "Dot") == 0) - { - lua_pushcfunction(L, lua_vector_dot, "Dot"); - return 1; - } + lua_pushnumber(L, sqrtf(v[0] * v[0] + v[1] * v[1] + v[2] * v[2])); + return 1; } - throw std::runtime_error(Luau::format("%s is not a valid member of vector", name)); + if (strcmp(name, "Dot") == 0) + { + lua_pushcfunction(L, lua_vector_dot, "Dot"); + return 1; + } + + luaL_error(L, "%s is not a valid member of vector", name); } static int lua_vector_namecall(lua_State* L) @@ -120,7 +113,7 @@ static int lua_vector_namecall(lua_State* L) return lua_vector_dot(L); } - throw std::runtime_error(Luau::format("%s is not a valid method of vector", luaL_checkstring(L, 1))); + luaL_error(L, "%s is not a valid method of vector", luaL_checkstring(L, 1)); } int lua_silence(lua_State* L) From 2aff9bb859b651e4c5351dd09d98fe4823c9bbe8 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Thu, 2 Dec 2021 11:36:40 -0800 Subject: [PATCH 087/399] Add documentation for bit32.count* and coroutine.close (#268) --- docs/_pages/library.md | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) diff --git a/docs/_pages/library.md b/docs/_pages/library.md index 28a9f389..324670f6 100644 --- a/docs/_pages/library.md +++ b/docs/_pages/library.md @@ -624,6 +624,12 @@ function coroutine.resume(co: thread, args: ...any): (boolean, ...any) Resumes the coroutine and passes the arguments along to the suspension point. When the coroutine yields or finishes, returns `true` and all values returned at the suspension point. If an error is raised during coroutine resumption, this function returns `false` and the error object, similarly to `pcall`. +``` +function coroutine.close(co: thread): (boolean, any?) +``` + +Closes the coroutine which puts coroutine in the dead state. The coroutine must be dead or suspended - in particular it can't be currently running. If the coroutine that's being closed was in an error state, returns `false` along with an error object; otherwise returns `true`. After closing, the coroutine can't be resumed and the coroutine stack becomes empty. + ## bit32 library All functions in the `bit32` library treat input numbers as 32-bit unsigned integers in `[0..4294967295]` range. The bit positions start at 0 where 0 corresponds to the least significant bit. @@ -700,6 +706,18 @@ function bit32.rshift(n: number, i: number): number Shifts `n` to the right by `i` bits (if `i` is negative, a left shift is performed instead). +``` +function bit32.countlz(n: number): number +``` + +Returns the number of consecutive zero bits in the 32-bit representation of `n` starting from the left-most (most significant) bit. Returns 32 if `n` is zero. + +``` +function bit32.countrz(n: number): number +``` + +Returns the number of consecutive zero bits in the 32-bit representation of `n` starting from the right-most (least significant) bit. Returns 32 if `n` is zero. + ## utf8 library Strings in Luau can contain arbitrary bytes; however, in many applications strings representing text contain UTF8 encoded data by convention, that can be inspected and manipulated using `utf8` library. From 864221e6678dba6e30c0ad7c0a850dcb20cd52c9 Mon Sep 17 00:00:00 2001 From: Lily Brown Date: Thu, 2 Dec 2021 13:22:02 -0800 Subject: [PATCH 088/399] November 2021 Luau recap (#264) Co-authored-by: Arseny Kapoulkine --- .../2021-11-29-luau-recap-november-2021.md | 77 +++++++++++++++++++ 1 file changed, 77 insertions(+) create mode 100644 docs/_posts/2021-11-29-luau-recap-november-2021.md diff --git a/docs/_posts/2021-11-29-luau-recap-november-2021.md b/docs/_posts/2021-11-29-luau-recap-november-2021.md new file mode 100644 index 00000000..381b2455 --- /dev/null +++ b/docs/_posts/2021-11-29-luau-recap-november-2021.md @@ -0,0 +1,77 @@ +--- +layout: single +title: "Luau Recap: November 2021" +--- + +Luau is our new language that you can read more about at [https://luau-lang.org](https://luau-lang.org). + +[Cross-posted to the [Roblox Developer Forum](https://devforum.roblox.com/t/luau-recap-november-2021/).] + +## Type packs in type aliases + +Type packs are the construct Luau uses to represent a sequence of types. We've had syntax for generic type packs for a while now, and it sees use in generic functions, but it hasn't been available in type aliases. That has changed, and it is now syntactically legal to write the following type alias: +```lua +type X = () -> A... +type Y = X +``` + +We've also added support for explicit type packs. Previously, it was impossible to instantiate a generic with two or more type pack parameters, because it wasn't clear where the first pack ended and the second one began. We have introduced a new syntax for this use case: +``` +type Fn = (P...) -> R... +type X = Fn<(number, string), (string, number)> +``` + +For more information, check out [the documentation](https://luau-lang.org/typecheck#type-packs) or [the RFC](https://github.com/Roblox/luau/blob/f86d4c6995418e489a55be0100159009492778ff/rfcs/syntax-type-alias-type-packs.md) for this feature. + +## Luau is open-source! + +We announced this in early November but it deserves repeating: Luau is now an open-source project! You can use Luau outside of Roblox, subject to MIT License, and - importantly - we accept contributions. + +Many changes contributed by community, both Roblox and external, have been merged since we've made Luau open source. Of note are two visible changes that shipped on Roblox platform: + +- The type error "Expected to return X values, but Y values are returned here" actually had X and Y swapped! This is now fixed. +- Luau compiler dutifully computed the length of the string when using `#` operator on a string literal; this is now fixed and `#"foo"` compiles to 3. + +You might think that C++ is a scary language and you can't contribute to Luau. If so, you'd be happy to know that the contents of https://luau-lang.org, where we host our documentation, is also hosted on GitHub in the same repository (https://github.com/Roblox/luau/tree/master/docs) and that we'd love the community to contribute improvements to documentation among other changes! For example see [issues in this list](https://github.com/Roblox/luau/issues?q=is%3Aissue+is%3Aopen+label%3A%22pr+welcome%22) that start with "Documentation", but all other changes and additions to documentation are also welcome. + +## Library improvements + +```lua +function bit32.countlz(n: number): number +function bit32.countrz(n: number): number +``` +Given a number, returns the number of preceding left or trailing right-hand bits that are `0`. + +See [the RFC for these functions](https://github.com/Roblox/luau/blob/f86d4c6995418e489a55be0100159009492778ff/rfcs/function-bit32-countlz-countrz.md) for more information. + +## Type checking improvements + +We have enabled a rewrite of how Luau handles `require` tracing. This has two main effects: firstly, in strict mode, `require` statements that Luau can't resolve will trigger type errors; secondly, Luau now understands the `FindFirstAncestor` method in `require` expressions. + +Luau now warns when the index to `table.move` is 0, as this is non-idiomatic and performs poorly. If this behavior is intentional, wrap the index in parentheses to suppress the warning. + +Luau now provides additional context in table and class type mismatch errors. + +## Performance improvements + +We have enabled several changes that aim to avoid allocating a new closure object in cases where it's not necessary to. This is helpful in cases where many closures are being allocated; in our benchmark suite, the two benchmarks that allocate a large number of closures improved by 15% and 5%, respectively. + +When checking union types, we now try possibilities whose synthetic names match. This will speed up type checking unions in cases where synthetic names are populated. + +We have also enabled an optimization that shares state in a hot path on the type checker. This will improve type checking performance. + +The Luau VM now attempts to cache the length of tables' array portion. This change showed a small performance improvement in benchmarks, and should speed up `#` expressions. + +The Luau type checker now caches a specific category of table unification results. This can improve type checking performance significantly when the same set of types is used frequently. + +When Luau is not retaining type graphs, the type checker now discards more of a module's type surface after type checking it. This improves memory usage significantly. + +## Bug fixes + +We've fixed a bug where on ARM systems (mobile), packing negative numbers using unsigned formats in `string.pack` would produce the wrong result. + +We've fixed an issue with type aliases that reuse generic type names that caused them to be instantiated incorrectly. + +We've corrected a subtle bug that could cause free types to leak into a table type when a free table is bound to that table. + +We've fixed an issue that could cause Luau to report an infinitely recursive type error when the type was not infinitely recursive. From eed18acec8677b380fdbb7f424d79e1c7dae4273 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Thu, 2 Dec 2021 15:20:08 -0800 Subject: [PATCH 089/399] Sync to upstream/release/506 --- Analysis/include/Luau/FileResolver.h | 23 - Analysis/include/Luau/Module.h | 12 +- Analysis/include/Luau/ToDot.h | 31 ++ Analysis/include/Luau/TxnLog.h | 9 - Analysis/include/Luau/TypeInfer.h | 1 - Analysis/include/Luau/TypeVar.h | 15 - Analysis/include/Luau/Unifier.h | 25 +- Analysis/include/Luau/UnifierSharedState.h | 8 + Analysis/include/Luau/VisitTypeVar.h | 4 +- Analysis/src/Autocomplete.cpp | 72 ++- Analysis/src/BuiltinDefinitions.cpp | 6 +- Analysis/src/Error.cpp | 116 +---- Analysis/src/Frontend.cpp | 27 +- Analysis/src/IostreamHelpers.cpp | 19 +- Analysis/src/JsonEncoder.cpp | 9 +- Analysis/src/Module.cpp | 132 ++--- Analysis/src/Quantify.cpp | 13 +- Analysis/src/RequireTracer.cpp | 195 +------ Analysis/src/Substitution.cpp | 29 +- Analysis/src/ToDot.cpp | 378 ++++++++++++++ Analysis/src/ToString.cpp | 172 +++---- Analysis/src/Transpiler.cpp | 15 +- Analysis/src/TxnLog.cpp | 37 +- Analysis/src/TypeAttach.cpp | 52 +- Analysis/src/TypeInfer.cpp | 331 +++++------- Analysis/src/TypeVar.cpp | 364 -------------- Analysis/src/Unifier.cpp | 306 ++--------- Ast/src/Parser.cpp | 74 +-- CLI/Analyze.cpp | 23 +- CLI/Repl.cpp | 39 -- CLI/Web.cpp | 106 ++++ CMakeLists.txt | 57 ++- Compiler/src/Compiler.cpp | 14 +- Makefile | 6 +- Sources.cmake | 10 + VM/include/lua.h | 11 +- VM/include/luaconf.h | 38 ++ VM/include/lualib.h | 6 + VM/src/lapi.cpp | 67 ++- VM/src/laux.cpp | 95 ++-- VM/src/lbaselib.cpp | 4 +- VM/src/lbitlib.cpp | 2 +- VM/src/lbuiltins.cpp | 12 +- VM/src/lcorolib.cpp | 2 +- VM/src/ldblib.cpp | 2 +- VM/src/ldo.cpp | 71 +-- VM/src/lgc.cpp | 548 +------------------- VM/src/lgcdebug.cpp | 558 +++++++++++++++++++++ VM/src/linit.cpp | 10 +- VM/src/lmathlib.cpp | 5 +- VM/src/lmem.cpp | 8 +- VM/src/lnumutils.h | 8 + VM/src/lobject.cpp | 2 +- VM/src/lobject.h | 23 +- VM/src/loslib.cpp | 2 +- VM/src/lstrlib.cpp | 2 +- VM/src/ltable.cpp | 19 +- VM/src/ltablib.cpp | 2 +- VM/src/lutf8lib.cpp | 2 +- VM/src/lvmexecute.cpp | 30 +- VM/src/lvmload.cpp | 6 +- VM/src/lvmutils.cpp | 18 +- bench/gc/test_LB_mandel.lua | 2 +- bench/tests/shootout/mandel.lua | 2 +- bench/tests/shootout/qt.lua | 10 +- fuzz/proto.cpp | 4 +- tests/AstQuery.test.cpp | 23 + tests/Autocomplete.test.cpp | 40 +- tests/Compiler.test.cpp | 168 +++++++ tests/Conformance.test.cpp | 116 +++-- tests/Fixture.cpp | 38 +- tests/Fixture.h | 5 +- tests/Frontend.test.cpp | 17 - tests/Linter.test.cpp | 16 + tests/Module.test.cpp | 66 ++- tests/Parser.test.cpp | 2 - tests/ToDot.test.cpp | 366 ++++++++++++++ tests/Transpiler.test.cpp | 3 - tests/TypeInfer.aliases.test.cpp | 2 - tests/TypeInfer.generics.test.cpp | 21 +- tests/TypeInfer.provisional.test.cpp | 2 - tests/TypeInfer.tables.test.cpp | 70 +++ tests/TypeInfer.test.cpp | 20 + tests/TypeInfer.typePacks.cpp | 33 -- tests/TypeVar.test.cpp | 45 ++ tests/conformance/apicalls.lua | 8 +- tests/conformance/basic.lua | 5 +- tests/conformance/closure.lua | 8 +- tests/conformance/constructs.lua | 2 +- tests/conformance/coroutine.lua | 2 +- tests/conformance/datetime.lua | 2 +- tests/conformance/debug.lua | 2 +- tests/conformance/errors.lua | 32 +- tests/conformance/gc.lua | 8 +- tests/conformance/nextvar.lua | 6 +- tests/conformance/pcall.lua | 2 +- tests/conformance/utf8.lua | 2 +- tests/conformance/vector.lua | 31 +- tools/svg.py | 9 +- 99 files changed, 2905 insertions(+), 2568 deletions(-) create mode 100644 Analysis/include/Luau/ToDot.h create mode 100644 Analysis/src/ToDot.cpp create mode 100644 CLI/Web.cpp create mode 100644 VM/src/lgcdebug.cpp create mode 100644 tests/ToDot.test.cpp diff --git a/Analysis/include/Luau/FileResolver.h b/Analysis/include/Luau/FileResolver.h index 9b74fc12..0fdcce16 100644 --- a/Analysis/include/Luau/FileResolver.h +++ b/Analysis/include/Luau/FileResolver.h @@ -51,13 +51,6 @@ struct FileResolver { return std::nullopt; } - - // DEPRECATED APIS - // These are going to be removed with LuauNewRequireTrace2 - virtual bool moduleExists(const ModuleName& name) const = 0; - virtual std::optional fromAstFragment(AstExpr* expr) const = 0; - virtual ModuleName concat(const ModuleName& lhs, std::string_view rhs) const = 0; - virtual std::optional getParentModuleName(const ModuleName& name) const = 0; }; struct NullFileResolver : FileResolver @@ -66,22 +59,6 @@ struct NullFileResolver : FileResolver { return std::nullopt; } - bool moduleExists(const ModuleName& name) const override - { - return false; - } - std::optional fromAstFragment(AstExpr* expr) const override - { - return std::nullopt; - } - ModuleName concat(const ModuleName& lhs, std::string_view rhs) const override - { - return lhs; - } - std::optional getParentModuleName(const ModuleName& name) const override - { - return std::nullopt; - } }; } // namespace Luau diff --git a/Analysis/include/Luau/Module.h b/Analysis/include/Luau/Module.h index d0844835..2e41674b 100644 --- a/Analysis/include/Luau/Module.h +++ b/Analysis/include/Luau/Module.h @@ -78,9 +78,15 @@ void unfreeze(TypeArena& arena); using SeenTypes = std::unordered_map; using SeenTypePacks = std::unordered_map; -TypePackId clone(TypePackId tp, TypeArena& dest, SeenTypes& seenTypes, SeenTypePacks& seenTypePacks, bool* encounteredFreeType = nullptr); -TypeId clone(TypeId tp, TypeArena& dest, SeenTypes& seenTypes, SeenTypePacks& seenTypePacks, bool* encounteredFreeType = nullptr); -TypeFun clone(const TypeFun& typeFun, TypeArena& dest, SeenTypes& seenTypes, SeenTypePacks& seenTypePacks, bool* encounteredFreeType = nullptr); +struct CloneState +{ + int recursionCount = 0; + bool encounteredFreeType = false; +}; + +TypePackId clone(TypePackId tp, TypeArena& dest, SeenTypes& seenTypes, SeenTypePacks& seenTypePacks, CloneState& cloneState); +TypeId clone(TypeId tp, TypeArena& dest, SeenTypes& seenTypes, SeenTypePacks& seenTypePacks, CloneState& cloneState); +TypeFun clone(const TypeFun& typeFun, TypeArena& dest, SeenTypes& seenTypes, SeenTypePacks& seenTypePacks, CloneState& cloneState); struct Module { diff --git a/Analysis/include/Luau/ToDot.h b/Analysis/include/Luau/ToDot.h new file mode 100644 index 00000000..ce518d3a --- /dev/null +++ b/Analysis/include/Luau/ToDot.h @@ -0,0 +1,31 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +#pragma once + +#include "Luau/Common.h" + +#include + +namespace Luau +{ +struct TypeVar; +using TypeId = const TypeVar*; + +struct TypePackVar; +using TypePackId = const TypePackVar*; + +struct ToDotOptions +{ + bool showPointers = true; // Show pointer value in the node label + bool duplicatePrimitives = true; // Display primitive types and 'any' as separate nodes +}; + +std::string toDot(TypeId ty, const ToDotOptions& opts); +std::string toDot(TypePackId tp, const ToDotOptions& opts); + +std::string toDot(TypeId ty); +std::string toDot(TypePackId tp); + +void dumpDot(TypeId ty); +void dumpDot(TypePackId tp); + +} // namespace Luau diff --git a/Analysis/include/Luau/TxnLog.h b/Analysis/include/Luau/TxnLog.h index 322abd19..29988a3b 100644 --- a/Analysis/include/Luau/TxnLog.h +++ b/Analysis/include/Luau/TxnLog.h @@ -25,15 +25,6 @@ struct TxnLog { } - explicit TxnLog(const std::vector>& ownedSeen) - : originalSeenSize(ownedSeen.size()) - , ownedSeen(ownedSeen) - , sharedSeen(nullptr) - { - // This is deprecated! - LUAU_ASSERT(!FFlag::LuauShareTxnSeen); - } - TxnLog(const TxnLog&) = delete; TxnLog& operator=(const TxnLog&) = delete; diff --git a/Analysis/include/Luau/TypeInfer.h b/Analysis/include/Luau/TypeInfer.h index 78d642c5..9f553bc1 100644 --- a/Analysis/include/Luau/TypeInfer.h +++ b/Analysis/include/Luau/TypeInfer.h @@ -297,7 +297,6 @@ private: private: Unifier mkUnifier(const Location& location); - Unifier mkUnifier(const std::vector>& seen, const Location& location); // These functions are only safe to call when we are in the process of typechecking a module. diff --git a/Analysis/include/Luau/TypeVar.h b/Analysis/include/Luau/TypeVar.h index 093ea431..8c4c2f34 100644 --- a/Analysis/include/Luau/TypeVar.h +++ b/Analysis/include/Luau/TypeVar.h @@ -517,21 +517,6 @@ extern SingletonTypes singletonTypes; void persist(TypeId ty); void persist(TypePackId tp); -struct ToDotOptions -{ - bool showPointers = true; // Show pointer value in the node label - bool duplicatePrimitives = true; // Display primitive types and 'any' as separate nodes -}; - -std::string toDot(TypeId ty, const ToDotOptions& opts); -std::string toDot(TypePackId tp, const ToDotOptions& opts); - -std::string toDot(TypeId ty); -std::string toDot(TypePackId tp); - -void dumpDot(TypeId ty); -void dumpDot(TypePackId tp); - const TypeLevel* getLevel(TypeId ty); TypeLevel* getMutableLevel(TypeId ty); diff --git a/Analysis/include/Luau/Unifier.h b/Analysis/include/Luau/Unifier.h index 503034a1..4588cdd8 100644 --- a/Analysis/include/Luau/Unifier.h +++ b/Analysis/include/Luau/Unifier.h @@ -19,12 +19,6 @@ enum Variance Invariant }; -struct UnifierCounters -{ - int recursionCount = 0; - int iterationCount = 0; -}; - struct Unifier { TypeArena* const types; @@ -37,20 +31,11 @@ struct Unifier Variance variance = Covariant; CountMismatch::Context ctx = CountMismatch::Arg; - UnifierCounters* counters; - UnifierCounters countersData; - - std::shared_ptr counters_DEPRECATED; - UnifierSharedState& sharedState; Unifier(TypeArena* types, Mode mode, ScopePtr globalScope, const Location& location, Variance variance, UnifierSharedState& sharedState); - Unifier(TypeArena* types, Mode mode, ScopePtr globalScope, const std::vector>& ownedSeen, const Location& location, - Variance variance, UnifierSharedState& sharedState, const std::shared_ptr& counters_DEPRECATED = nullptr, - UnifierCounters* counters = nullptr); Unifier(TypeArena* types, Mode mode, ScopePtr globalScope, std::vector>* sharedSeen, const Location& location, - Variance variance, UnifierSharedState& sharedState, const std::shared_ptr& counters_DEPRECATED = nullptr, - UnifierCounters* counters = nullptr); + Variance variance, UnifierSharedState& sharedState); // Test whether the two type vars unify. Never commits the result. ErrorVec canUnify(TypeId superTy, TypeId subTy); @@ -92,9 +77,9 @@ private: public: // Report an "infinite type error" if the type "needle" already occurs within "haystack" void occursCheck(TypeId needle, TypeId haystack); - void occursCheck(std::unordered_set& seen_DEPRECATED, DenseHashSet& seen, TypeId needle, TypeId haystack); + void occursCheck(DenseHashSet& seen, TypeId needle, TypeId haystack); void occursCheck(TypePackId needle, TypePackId haystack); - void occursCheck(std::unordered_set& seen_DEPRECATED, DenseHashSet& seen, TypePackId needle, TypePackId haystack); + void occursCheck(DenseHashSet& seen, TypePackId needle, TypePackId haystack); Unifier makeChildUnifier(); @@ -106,10 +91,6 @@ private: [[noreturn]] void ice(const std::string& message, const Location& location); [[noreturn]] void ice(const std::string& message); - - // Remove with FFlagLuauCacheUnifyTableResults - DenseHashSet tempSeenTy_DEPRECATED{nullptr}; - DenseHashSet tempSeenTp_DEPRECATED{nullptr}; }; } // namespace Luau diff --git a/Analysis/include/Luau/UnifierSharedState.h b/Analysis/include/Luau/UnifierSharedState.h index f252a004..88997c41 100644 --- a/Analysis/include/Luau/UnifierSharedState.h +++ b/Analysis/include/Luau/UnifierSharedState.h @@ -24,6 +24,12 @@ struct TypeIdPairHash } }; +struct UnifierCounters +{ + int recursionCount = 0; + int iterationCount = 0; +}; + struct UnifierSharedState { UnifierSharedState(InternalErrorReporter* iceHandler) @@ -39,6 +45,8 @@ struct UnifierSharedState DenseHashSet tempSeenTy{nullptr}; DenseHashSet tempSeenTp{nullptr}; + + UnifierCounters counters; }; } // namespace Luau diff --git a/Analysis/include/Luau/VisitTypeVar.h b/Analysis/include/Luau/VisitTypeVar.h index a866655c..740854b3 100644 --- a/Analysis/include/Luau/VisitTypeVar.h +++ b/Analysis/include/Luau/VisitTypeVar.h @@ -5,8 +5,6 @@ #include "Luau/TypeVar.h" #include "Luau/TypePack.h" -LUAU_FASTFLAG(LuauCacheUnifyTableResults) - namespace Luau { @@ -101,7 +99,7 @@ void visit(TypeId ty, F& f, Set& seen) // Some visitors want to see bound tables, that's why we visit the original type if (apply(ty, *ttv, seen, f)) { - if (FFlag::LuauCacheUnifyTableResults && ttv->boundTo) + if (ttv->boundTo) { visit(*ttv->boundTo, f, seen); } diff --git a/Analysis/src/Autocomplete.cpp b/Analysis/src/Autocomplete.cpp index 6fc0b3f8..db2d1d0e 100644 --- a/Analysis/src/Autocomplete.cpp +++ b/Analysis/src/Autocomplete.cpp @@ -12,9 +12,9 @@ #include #include -LUAU_FASTFLAGVARIABLE(ElseElseIfCompletionImprovements, false); LUAU_FASTFLAG(LuauIfElseExpressionAnalysisSupport) LUAU_FASTFLAGVARIABLE(LuauAutocompleteAvoidMutation, false); +LUAU_FASTFLAGVARIABLE(LuauAutocompletePreferToCallFunctions, false); static const std::unordered_set kStatementStartingKeywords = { "while", "if", "local", "repeat", "function", "do", "for", "return", "break", "continue", "type", "export"}; @@ -203,8 +203,9 @@ static TypeCorrectKind checkTypeCorrectKind(const Module& module, TypeArena* typ { SeenTypes seenTypes; SeenTypePacks seenTypePacks; - expectedType = clone(expectedType, *typeArena, seenTypes, seenTypePacks, nullptr); - actualType = clone(actualType, *typeArena, seenTypes, seenTypePacks, nullptr); + CloneState cloneState; + expectedType = clone(expectedType, *typeArena, seenTypes, seenTypePacks, cloneState); + actualType = clone(actualType, *typeArena, seenTypes, seenTypePacks, cloneState); auto errors = unifier.canUnify(expectedType, actualType); return errors.empty(); @@ -229,28 +230,51 @@ static TypeCorrectKind checkTypeCorrectKind(const Module& module, TypeArena* typ TypeId expectedType = follow(*it); - if (canUnify(expectedType, ty)) - return TypeCorrectKind::Correct; - - // We also want to suggest functions that return compatible result - const FunctionTypeVar* ftv = get(ty); - - if (!ftv) - return TypeCorrectKind::None; - - auto [retHead, retTail] = flatten(ftv->retType); - - if (!retHead.empty()) - return canUnify(expectedType, retHead.front()) ? TypeCorrectKind::CorrectFunctionResult : TypeCorrectKind::None; - - // We might only have a variadic tail pack, check if the element is compatible - if (retTail) + if (FFlag::LuauAutocompletePreferToCallFunctions) { - if (const VariadicTypePack* vtp = get(follow(*retTail))) - return canUnify(expectedType, vtp->ty) ? TypeCorrectKind::CorrectFunctionResult : TypeCorrectKind::None; - } + // We also want to suggest functions that return compatible result + if (const FunctionTypeVar* ftv = get(ty)) + { + auto [retHead, retTail] = flatten(ftv->retType); - return TypeCorrectKind::None; + if (!retHead.empty() && canUnify(expectedType, retHead.front())) + return TypeCorrectKind::CorrectFunctionResult; + + // We might only have a variadic tail pack, check if the element is compatible + if (retTail) + { + if (const VariadicTypePack* vtp = get(follow(*retTail)); vtp && canUnify(expectedType, vtp->ty)) + return TypeCorrectKind::CorrectFunctionResult; + } + } + + return canUnify(expectedType, ty) ? TypeCorrectKind::Correct : TypeCorrectKind::None; + } + else + { + if (canUnify(expectedType, ty)) + return TypeCorrectKind::Correct; + + // We also want to suggest functions that return compatible result + const FunctionTypeVar* ftv = get(ty); + + if (!ftv) + return TypeCorrectKind::None; + + auto [retHead, retTail] = flatten(ftv->retType); + + if (!retHead.empty()) + return canUnify(expectedType, retHead.front()) ? TypeCorrectKind::CorrectFunctionResult : TypeCorrectKind::None; + + // We might only have a variadic tail pack, check if the element is compatible + if (retTail) + { + if (const VariadicTypePack* vtp = get(follow(*retTail))) + return canUnify(expectedType, vtp->ty) ? TypeCorrectKind::CorrectFunctionResult : TypeCorrectKind::None; + } + + return TypeCorrectKind::None; + } } enum class PropIndexType @@ -1413,7 +1437,7 @@ static AutocompleteResult autocomplete(const SourceModule& sourceModule, const M else if (AstStatWhile* statWhile = extractStat(finder.ancestry); statWhile && !statWhile->hasDo) return {{{"do", AutocompleteEntry{AutocompleteEntryKind::Keyword}}}, finder.ancestry}; - else if (AstStatIf* statIf = node->as(); FFlag::ElseElseIfCompletionImprovements && statIf && !statIf->hasElse) + else if (AstStatIf* statIf = node->as(); statIf && !statIf->hasElse) { return {{{"else", AutocompleteEntry{AutocompleteEntryKind::Keyword}}, {"elseif", AutocompleteEntry{AutocompleteEntryKind::Keyword}}}, finder.ancestry}; diff --git a/Analysis/src/BuiltinDefinitions.cpp b/Analysis/src/BuiltinDefinitions.cpp index 62a06a3c..bac94a2b 100644 --- a/Analysis/src/BuiltinDefinitions.cpp +++ b/Analysis/src/BuiltinDefinitions.cpp @@ -8,8 +8,6 @@ #include -LUAU_FASTFLAG(LuauNewRequireTrace2) - /** FIXME: Many of these type definitions are not quite completely accurate. * * Some of them require richer generics than we have. For instance, we do not yet have a way to talk @@ -473,9 +471,7 @@ static std::optional> magicFunctionRequire( if (!checkRequirePath(typechecker, expr.args.data[0])) return std::nullopt; - const AstExpr* require = FFlag::LuauNewRequireTrace2 ? &expr : expr.args.data[0]; - - if (auto moduleInfo = typechecker.resolver->resolveModuleInfo(typechecker.currentModuleName, *require)) + if (auto moduleInfo = typechecker.resolver->resolveModuleInfo(typechecker.currentModuleName, expr)) return ExprResult{arena.addTypePack({typechecker.checkRequire(scope, *moduleInfo, expr.location)})}; return std::nullopt; diff --git a/Analysis/src/Error.cpp b/Analysis/src/Error.cpp index f80d50a7..8334bd62 100644 --- a/Analysis/src/Error.cpp +++ b/Analysis/src/Error.cpp @@ -7,57 +7,14 @@ #include -LUAU_FASTFLAG(LuauTypeAliasPacks) - -static std::string wrongNumberOfArgsString_DEPRECATED(size_t expectedCount, size_t actualCount, bool isTypeArgs = false) -{ - std::string s = "expects " + std::to_string(expectedCount) + " "; - - if (isTypeArgs) - s += "type "; - - s += "argument"; - if (expectedCount != 1) - s += "s"; - - s += ", but "; - - if (actualCount == 0) - { - s += "none"; - } - else - { - if (actualCount < expectedCount) - s += "only "; - - s += std::to_string(actualCount); - } - - s += (actualCount == 1) ? " is" : " are"; - - s += " specified"; - - return s; -} - static std::string wrongNumberOfArgsString(size_t expectedCount, size_t actualCount, const char* argPrefix = nullptr, bool isVariadic = false) { - std::string s; + std::string s = "expects "; - if (FFlag::LuauTypeAliasPacks) - { - s = "expects "; + if (isVariadic) + s += "at least "; - if (isVariadic) - s += "at least "; - - s += std::to_string(expectedCount) + " "; - } - else - { - s = "expects " + std::to_string(expectedCount) + " "; - } + s += std::to_string(expectedCount) + " "; if (argPrefix) s += std::string(argPrefix) + " "; @@ -188,10 +145,7 @@ struct ErrorConverter return "Function only returns " + std::to_string(e.expected) + " value" + expectedS + ". " + std::to_string(e.actual) + " are required here"; case CountMismatch::Arg: - if (FFlag::LuauTypeAliasPacks) - return "Argument count mismatch. Function " + wrongNumberOfArgsString(e.expected, e.actual); - else - return "Argument count mismatch. Function " + wrongNumberOfArgsString_DEPRECATED(e.expected, e.actual); + return "Argument count mismatch. Function " + wrongNumberOfArgsString(e.expected, e.actual); } LUAU_ASSERT(!"Unknown context"); @@ -232,7 +186,7 @@ struct ErrorConverter std::string operator()(const Luau::IncorrectGenericParameterCount& e) const { std::string name = e.name; - if (!e.typeFun.typeParams.empty() || (FFlag::LuauTypeAliasPacks && !e.typeFun.typePackParams.empty())) + if (!e.typeFun.typeParams.empty() || !e.typeFun.typePackParams.empty()) { name += "<"; bool first = true; @@ -246,36 +200,25 @@ struct ErrorConverter name += toString(t); } - if (FFlag::LuauTypeAliasPacks) + for (TypePackId t : e.typeFun.typePackParams) { - for (TypePackId t : e.typeFun.typePackParams) - { - if (first) - first = false; - else - name += ", "; + if (first) + first = false; + else + name += ", "; - name += toString(t); - } + name += toString(t); } name += ">"; } - if (FFlag::LuauTypeAliasPacks) - { - if (e.typeFun.typeParams.size() != e.actualParameters) - return "Generic type '" + name + "' " + - wrongNumberOfArgsString(e.typeFun.typeParams.size(), e.actualParameters, "type", !e.typeFun.typePackParams.empty()); + if (e.typeFun.typeParams.size() != e.actualParameters) + return "Generic type '" + name + "' " + + wrongNumberOfArgsString(e.typeFun.typeParams.size(), e.actualParameters, "type", !e.typeFun.typePackParams.empty()); - return "Generic type '" + name + "' " + - wrongNumberOfArgsString(e.typeFun.typePackParams.size(), e.actualPackParameters, "type pack", /*isVariadic*/ false); - } - else - { - return "Generic type '" + name + "' " + - wrongNumberOfArgsString_DEPRECATED(e.typeFun.typeParams.size(), e.actualParameters, /*isTypeArgs*/ true); - } + return "Generic type '" + name + "' " + + wrongNumberOfArgsString(e.typeFun.typePackParams.size(), e.actualPackParameters, "type pack", /*isVariadic*/ false); } std::string operator()(const Luau::SyntaxError& e) const @@ -591,11 +534,8 @@ bool IncorrectGenericParameterCount::operator==(const IncorrectGenericParameterC if (typeFun.typeParams.size() != rhs.typeFun.typeParams.size()) return false; - if (FFlag::LuauTypeAliasPacks) - { - if (typeFun.typePackParams.size() != rhs.typeFun.typePackParams.size()) - return false; - } + if (typeFun.typePackParams.size() != rhs.typeFun.typePackParams.size()) + return false; for (size_t i = 0; i < typeFun.typeParams.size(); ++i) { @@ -603,13 +543,10 @@ bool IncorrectGenericParameterCount::operator==(const IncorrectGenericParameterC return false; } - if (FFlag::LuauTypeAliasPacks) + for (size_t i = 0; i < typeFun.typePackParams.size(); ++i) { - for (size_t i = 0; i < typeFun.typePackParams.size(); ++i) - { - if (typeFun.typePackParams[i] != rhs.typeFun.typePackParams[i]) - return false; - } + if (typeFun.typePackParams[i] != rhs.typeFun.typePackParams[i]) + return false; } return true; @@ -733,14 +670,14 @@ bool containsParseErrorName(const TypeError& error) } template -void copyError(T& e, TypeArena& destArena, SeenTypes& seenTypes, SeenTypePacks& seenTypePacks) +void copyError(T& e, TypeArena& destArena, SeenTypes& seenTypes, SeenTypePacks& seenTypePacks, CloneState cloneState) { auto clone = [&](auto&& ty) { - return ::Luau::clone(ty, destArena, seenTypes, seenTypePacks); + return ::Luau::clone(ty, destArena, seenTypes, seenTypePacks, cloneState); }; auto visitErrorData = [&](auto&& e) { - copyError(e, destArena, seenTypes, seenTypePacks); + copyError(e, destArena, seenTypes, seenTypePacks, cloneState); }; if constexpr (false) @@ -864,9 +801,10 @@ void copyErrors(ErrorVec& errors, TypeArena& destArena) { SeenTypes seenTypes; SeenTypePacks seenTypePacks; + CloneState cloneState; auto visitErrorData = [&](auto&& e) { - copyError(e, destArena, seenTypes, seenTypePacks); + copyError(e, destArena, seenTypes, seenTypePacks, cloneState); }; LUAU_ASSERT(!destArena.typeVars.isFrozen()); diff --git a/Analysis/src/Frontend.cpp b/Analysis/src/Frontend.cpp index 1e97705d..e332f07d 100644 --- a/Analysis/src/Frontend.cpp +++ b/Analysis/src/Frontend.cpp @@ -18,10 +18,7 @@ LUAU_FASTFLAG(LuauInferInNoCheckMode) LUAU_FASTFLAGVARIABLE(LuauTypeCheckTwice, false) LUAU_FASTFLAGVARIABLE(LuauKnowsTheDataModel3, false) -LUAU_FASTFLAGVARIABLE(LuauResolveModuleNameWithoutACurrentModule, false) -LUAU_FASTFLAG(LuauTraceRequireLookupChild) LUAU_FASTFLAGVARIABLE(LuauPersistDefinitionFileTypes, false) -LUAU_FASTFLAG(LuauNewRequireTrace2) namespace Luau { @@ -96,10 +93,11 @@ LoadDefinitionFileResult loadDefinitionFile(TypeChecker& typeChecker, ScopePtr t SeenTypes seenTypes; SeenTypePacks seenTypePacks; + CloneState cloneState; for (const auto& [name, ty] : checkedModule->declaredGlobals) { - TypeId globalTy = clone(ty, typeChecker.globalTypes, seenTypes, seenTypePacks); + TypeId globalTy = clone(ty, typeChecker.globalTypes, seenTypes, seenTypePacks, cloneState); std::string documentationSymbol = packageName + "/global/" + name; generateDocumentationSymbols(globalTy, documentationSymbol); targetScope->bindings[typeChecker.globalNames.names->getOrAdd(name.c_str())] = {globalTy, Location(), false, {}, documentationSymbol}; @@ -110,7 +108,7 @@ LoadDefinitionFileResult loadDefinitionFile(TypeChecker& typeChecker, ScopePtr t for (const auto& [name, ty] : checkedModule->getModuleScope()->exportedTypeBindings) { - TypeFun globalTy = clone(ty, typeChecker.globalTypes, seenTypes, seenTypePacks); + TypeFun globalTy = clone(ty, typeChecker.globalTypes, seenTypes, seenTypePacks, cloneState); std::string documentationSymbol = packageName + "/globaltype/" + name; generateDocumentationSymbols(globalTy.type, documentationSymbol); targetScope->exportedTypeBindings[name] = globalTy; @@ -427,15 +425,16 @@ CheckResult Frontend::check(const ModuleName& name) SeenTypes seenTypes; SeenTypePacks seenTypePacks; + CloneState cloneState; for (const auto& [expr, strictTy] : strictModule->astTypes) - module->astTypes[expr] = clone(strictTy, module->interfaceTypes, seenTypes, seenTypePacks); + module->astTypes[expr] = clone(strictTy, module->interfaceTypes, seenTypes, seenTypePacks, cloneState); for (const auto& [expr, strictTy] : strictModule->astOriginalCallTypes) - module->astOriginalCallTypes[expr] = clone(strictTy, module->interfaceTypes, seenTypes, seenTypePacks); + module->astOriginalCallTypes[expr] = clone(strictTy, module->interfaceTypes, seenTypes, seenTypePacks, cloneState); for (const auto& [expr, strictTy] : strictModule->astExpectedTypes) - module->astExpectedTypes[expr] = clone(strictTy, module->interfaceTypes, seenTypes, seenTypePacks); + module->astExpectedTypes[expr] = clone(strictTy, module->interfaceTypes, seenTypes, seenTypePacks, cloneState); } stats.timeCheck += getTimestamp() - timestamp; @@ -885,16 +884,13 @@ std::optional FrontendModuleResolver::resolveModuleInfo(const Module // If we can't find the current module name, that's because we bypassed the frontend's initializer // and called typeChecker.check directly. (This is done by autocompleteSource, for example). // In that case, requires will always fail. - if (FFlag::LuauResolveModuleNameWithoutACurrentModule) - return std::nullopt; - else - throw std::runtime_error("Frontend::resolveModuleName: Unknown currentModuleName '" + currentModuleName + "'"); + return std::nullopt; } const auto& exprs = it->second.exprs; const ModuleInfo* info = exprs.find(&pathExpr); - if (!info || (!FFlag::LuauNewRequireTrace2 && info->name.empty())) + if (!info) return std::nullopt; return *info; @@ -911,10 +907,7 @@ const ModulePtr FrontendModuleResolver::getModule(const ModuleName& moduleName) bool FrontendModuleResolver::moduleExists(const ModuleName& moduleName) const { - if (FFlag::LuauNewRequireTrace2) - return frontend->sourceNodes.count(moduleName) != 0; - else - return frontend->fileResolver->moduleExists(moduleName); + return frontend->sourceNodes.count(moduleName) != 0; } std::string FrontendModuleResolver::getHumanReadableModuleName(const ModuleName& moduleName) const diff --git a/Analysis/src/IostreamHelpers.cpp b/Analysis/src/IostreamHelpers.cpp index 3b267121..ac46b5a4 100644 --- a/Analysis/src/IostreamHelpers.cpp +++ b/Analysis/src/IostreamHelpers.cpp @@ -2,8 +2,6 @@ #include "Luau/IostreamHelpers.h" #include "Luau/ToString.h" -LUAU_FASTFLAG(LuauTypeAliasPacks) - namespace Luau { @@ -94,7 +92,7 @@ std::ostream& operator<<(std::ostream& stream, const IncorrectGenericParameterCo { stream << "IncorrectGenericParameterCount { name = " << error.name; - if (!error.typeFun.typeParams.empty() || (FFlag::LuauTypeAliasPacks && !error.typeFun.typePackParams.empty())) + if (!error.typeFun.typeParams.empty() || !error.typeFun.typePackParams.empty()) { stream << "<"; bool first = true; @@ -108,17 +106,14 @@ std::ostream& operator<<(std::ostream& stream, const IncorrectGenericParameterCo stream << toString(t); } - if (FFlag::LuauTypeAliasPacks) + for (TypePackId t : error.typeFun.typePackParams) { - for (TypePackId t : error.typeFun.typePackParams) - { - if (first) - first = false; - else - stream << ", "; + if (first) + first = false; + else + stream << ", "; - stream << toString(t); - } + stream << toString(t); } stream << ">"; diff --git a/Analysis/src/JsonEncoder.cpp b/Analysis/src/JsonEncoder.cpp index 064accba..c7f623ee 100644 --- a/Analysis/src/JsonEncoder.cpp +++ b/Analysis/src/JsonEncoder.cpp @@ -5,8 +5,6 @@ #include "Luau/StringUtils.h" #include "Luau/Common.h" -LUAU_FASTFLAG(LuauTypeAliasPacks) - namespace Luau { @@ -615,12 +613,7 @@ struct AstJsonEncoder : public AstVisitor writeNode(node, "AstStatTypeAlias", [&]() { PROP(name); PROP(generics); - - if (FFlag::LuauTypeAliasPacks) - { - PROP(genericPacks); - } - + PROP(genericPacks); PROP(type); PROP(exported); }); diff --git a/Analysis/src/Module.cpp b/Analysis/src/Module.cpp index 32a0646a..b4b6eb42 100644 --- a/Analysis/src/Module.cpp +++ b/Analysis/src/Module.cpp @@ -1,20 +1,20 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #include "Luau/Module.h" +#include "Luau/Common.h" +#include "Luau/RecursionCounter.h" #include "Luau/Scope.h" #include "Luau/TypeInfer.h" #include "Luau/TypePack.h" #include "Luau/TypeVar.h" #include "Luau/VisitTypeVar.h" -#include "Luau/Common.h" #include LUAU_FASTFLAGVARIABLE(DebugLuauFreezeArena, false) LUAU_FASTFLAGVARIABLE(DebugLuauTrackOwningArena, false) LUAU_FASTFLAG(LuauCaptureBrokenCommentSpans) -LUAU_FASTFLAG(LuauTypeAliasPacks) -LUAU_FASTFLAGVARIABLE(LuauCloneBoundTables, false) +LUAU_FASTINTVARIABLE(LuauTypeCloneRecursionLimit, 0) namespace Luau { @@ -120,12 +120,6 @@ TypePackId TypeArena::addTypePack(TypePackVar tp) return allocated; } -using SeenTypes = std::unordered_map; -using SeenTypePacks = std::unordered_map; - -TypePackId clone(TypePackId tp, TypeArena& dest, SeenTypes& seenTypes, SeenTypePacks& seenTypePacks, bool* encounteredFreeType); -TypeId clone(TypeId tp, TypeArena& dest, SeenTypes& seenTypes, SeenTypePacks& seenTypePacks, bool* encounteredFreeType); - namespace { @@ -138,11 +132,12 @@ struct TypePackCloner; struct TypeCloner { - TypeCloner(TypeArena& dest, TypeId typeId, SeenTypes& seenTypes, SeenTypePacks& seenTypePacks) + TypeCloner(TypeArena& dest, TypeId typeId, SeenTypes& seenTypes, SeenTypePacks& seenTypePacks, CloneState& cloneState) : dest(dest) , typeId(typeId) , seenTypes(seenTypes) , seenTypePacks(seenTypePacks) + , cloneState(cloneState) { } @@ -150,8 +145,7 @@ struct TypeCloner TypeId typeId; SeenTypes& seenTypes; SeenTypePacks& seenTypePacks; - - bool* encounteredFreeType = nullptr; + CloneState& cloneState; template void defaultClone(const T& t); @@ -178,13 +172,14 @@ struct TypePackCloner TypePackId typePackId; SeenTypes& seenTypes; SeenTypePacks& seenTypePacks; - bool* encounteredFreeType = nullptr; + CloneState& cloneState; - TypePackCloner(TypeArena& dest, TypePackId typePackId, SeenTypes& seenTypes, SeenTypePacks& seenTypePacks) + TypePackCloner(TypeArena& dest, TypePackId typePackId, SeenTypes& seenTypes, SeenTypePacks& seenTypePacks, CloneState& cloneState) : dest(dest) , typePackId(typePackId) , seenTypes(seenTypes) , seenTypePacks(seenTypePacks) + , cloneState(cloneState) { } @@ -197,8 +192,7 @@ struct TypePackCloner void operator()(const Unifiable::Free& t) { - if (encounteredFreeType) - *encounteredFreeType = true; + cloneState.encounteredFreeType = true; TypePackId err = singletonTypes.errorRecoveryTypePack(singletonTypes.anyTypePack); TypePackId cloned = dest.addTypePack(*err); @@ -218,13 +212,13 @@ struct TypePackCloner // We just need to be sure that we rewrite pointers both to the binder and the bindee to the same pointer. void operator()(const Unifiable::Bound& t) { - TypePackId cloned = clone(t.boundTo, dest, seenTypes, seenTypePacks, encounteredFreeType); + TypePackId cloned = clone(t.boundTo, dest, seenTypes, seenTypePacks, cloneState); seenTypePacks[typePackId] = cloned; } void operator()(const VariadicTypePack& t) { - TypePackId cloned = dest.addTypePack(TypePackVar{VariadicTypePack{clone(t.ty, dest, seenTypes, seenTypePacks, encounteredFreeType)}}); + TypePackId cloned = dest.addTypePack(TypePackVar{VariadicTypePack{clone(t.ty, dest, seenTypes, seenTypePacks, cloneState)}}); seenTypePacks[typePackId] = cloned; } @@ -236,10 +230,10 @@ struct TypePackCloner seenTypePacks[typePackId] = cloned; for (TypeId ty : t.head) - destTp->head.push_back(clone(ty, dest, seenTypes, seenTypePacks, encounteredFreeType)); + destTp->head.push_back(clone(ty, dest, seenTypes, seenTypePacks, cloneState)); if (t.tail) - destTp->tail = clone(*t.tail, dest, seenTypes, seenTypePacks, encounteredFreeType); + destTp->tail = clone(*t.tail, dest, seenTypes, seenTypePacks, cloneState); } }; @@ -252,8 +246,7 @@ void TypeCloner::defaultClone(const T& t) void TypeCloner::operator()(const Unifiable::Free& t) { - if (encounteredFreeType) - *encounteredFreeType = true; + cloneState.encounteredFreeType = true; TypeId err = singletonTypes.errorRecoveryType(singletonTypes.anyType); TypeId cloned = dest.addType(*err); seenTypes[typeId] = cloned; @@ -266,7 +259,7 @@ void TypeCloner::operator()(const Unifiable::Generic& t) void TypeCloner::operator()(const Unifiable::Bound& t) { - TypeId boundTo = clone(t.boundTo, dest, seenTypes, seenTypePacks, encounteredFreeType); + TypeId boundTo = clone(t.boundTo, dest, seenTypes, seenTypePacks, cloneState); seenTypes[typeId] = boundTo; } @@ -294,23 +287,23 @@ void TypeCloner::operator()(const FunctionTypeVar& t) seenTypes[typeId] = result; for (TypeId generic : t.generics) - ftv->generics.push_back(clone(generic, dest, seenTypes, seenTypePacks, encounteredFreeType)); + ftv->generics.push_back(clone(generic, dest, seenTypes, seenTypePacks, cloneState)); for (TypePackId genericPack : t.genericPacks) - ftv->genericPacks.push_back(clone(genericPack, dest, seenTypes, seenTypePacks, encounteredFreeType)); + ftv->genericPacks.push_back(clone(genericPack, dest, seenTypes, seenTypePacks, cloneState)); ftv->tags = t.tags; - ftv->argTypes = clone(t.argTypes, dest, seenTypes, seenTypePacks, encounteredFreeType); + ftv->argTypes = clone(t.argTypes, dest, seenTypes, seenTypePacks, cloneState); ftv->argNames = t.argNames; - ftv->retType = clone(t.retType, dest, seenTypes, seenTypePacks, encounteredFreeType); + ftv->retType = clone(t.retType, dest, seenTypes, seenTypePacks, cloneState); } void TypeCloner::operator()(const TableTypeVar& t) { // If table is now bound to another one, we ignore the content of the original - if (FFlag::LuauCloneBoundTables && t.boundTo) + if (t.boundTo) { - TypeId boundTo = clone(*t.boundTo, dest, seenTypes, seenTypePacks, encounteredFreeType); + TypeId boundTo = clone(*t.boundTo, dest, seenTypes, seenTypePacks, cloneState); seenTypes[typeId] = boundTo; return; } @@ -326,34 +319,21 @@ void TypeCloner::operator()(const TableTypeVar& t) ttv->level = TypeLevel{0, 0}; for (const auto& [name, prop] : t.props) - ttv->props[name] = {clone(prop.type, dest, seenTypes, seenTypePacks, encounteredFreeType), prop.deprecated, {}, prop.location, prop.tags}; + ttv->props[name] = {clone(prop.type, dest, seenTypes, seenTypePacks, cloneState), prop.deprecated, {}, prop.location, prop.tags}; if (t.indexer) - ttv->indexer = TableIndexer{clone(t.indexer->indexType, dest, seenTypes, seenTypePacks, encounteredFreeType), - clone(t.indexer->indexResultType, dest, seenTypes, seenTypePacks, encounteredFreeType)}; - - if (!FFlag::LuauCloneBoundTables) - { - if (t.boundTo) - ttv->boundTo = clone(*t.boundTo, dest, seenTypes, seenTypePacks, encounteredFreeType); - } + ttv->indexer = TableIndexer{clone(t.indexer->indexType, dest, seenTypes, seenTypePacks, cloneState), + clone(t.indexer->indexResultType, dest, seenTypes, seenTypePacks, cloneState)}; for (TypeId& arg : ttv->instantiatedTypeParams) - arg = clone(arg, dest, seenTypes, seenTypePacks, encounteredFreeType); + arg = clone(arg, dest, seenTypes, seenTypePacks, cloneState); - if (FFlag::LuauTypeAliasPacks) - { - for (TypePackId& arg : ttv->instantiatedTypePackParams) - arg = clone(arg, dest, seenTypes, seenTypePacks, encounteredFreeType); - } + for (TypePackId& arg : ttv->instantiatedTypePackParams) + arg = clone(arg, dest, seenTypes, seenTypePacks, cloneState); if (ttv->state == TableState::Free) { - if (FFlag::LuauCloneBoundTables || !t.boundTo) - { - if (encounteredFreeType) - *encounteredFreeType = true; - } + cloneState.encounteredFreeType = true; ttv->state = TableState::Sealed; } @@ -369,8 +349,8 @@ void TypeCloner::operator()(const MetatableTypeVar& t) MetatableTypeVar* mtv = getMutable(result); seenTypes[typeId] = result; - mtv->table = clone(t.table, dest, seenTypes, seenTypePacks, encounteredFreeType); - mtv->metatable = clone(t.metatable, dest, seenTypes, seenTypePacks, encounteredFreeType); + mtv->table = clone(t.table, dest, seenTypes, seenTypePacks, cloneState); + mtv->metatable = clone(t.metatable, dest, seenTypes, seenTypePacks, cloneState); } void TypeCloner::operator()(const ClassTypeVar& t) @@ -381,13 +361,13 @@ void TypeCloner::operator()(const ClassTypeVar& t) seenTypes[typeId] = result; for (const auto& [name, prop] : t.props) - ctv->props[name] = {clone(prop.type, dest, seenTypes, seenTypePacks, encounteredFreeType), prop.deprecated, {}, prop.location, prop.tags}; + ctv->props[name] = {clone(prop.type, dest, seenTypes, seenTypePacks, cloneState), prop.deprecated, {}, prop.location, prop.tags}; if (t.parent) - ctv->parent = clone(*t.parent, dest, seenTypes, seenTypePacks, encounteredFreeType); + ctv->parent = clone(*t.parent, dest, seenTypes, seenTypePacks, cloneState); if (t.metatable) - ctv->metatable = clone(*t.metatable, dest, seenTypes, seenTypePacks, encounteredFreeType); + ctv->metatable = clone(*t.metatable, dest, seenTypes, seenTypePacks, cloneState); } void TypeCloner::operator()(const AnyTypeVar& t) @@ -404,7 +384,7 @@ void TypeCloner::operator()(const UnionTypeVar& t) LUAU_ASSERT(option != nullptr); for (TypeId ty : t.options) - option->options.push_back(clone(ty, dest, seenTypes, seenTypePacks, encounteredFreeType)); + option->options.push_back(clone(ty, dest, seenTypes, seenTypePacks, cloneState)); } void TypeCloner::operator()(const IntersectionTypeVar& t) @@ -416,7 +396,7 @@ void TypeCloner::operator()(const IntersectionTypeVar& t) LUAU_ASSERT(option != nullptr); for (TypeId ty : t.parts) - option->parts.push_back(clone(ty, dest, seenTypes, seenTypePacks, encounteredFreeType)); + option->parts.push_back(clone(ty, dest, seenTypes, seenTypePacks, cloneState)); } void TypeCloner::operator()(const LazyTypeVar& t) @@ -426,17 +406,18 @@ void TypeCloner::operator()(const LazyTypeVar& t) } // anonymous namespace -TypePackId clone(TypePackId tp, TypeArena& dest, SeenTypes& seenTypes, SeenTypePacks& seenTypePacks, bool* encounteredFreeType) +TypePackId clone(TypePackId tp, TypeArena& dest, SeenTypes& seenTypes, SeenTypePacks& seenTypePacks, CloneState& cloneState) { if (tp->persistent) return tp; + RecursionLimiter _ra(&cloneState.recursionCount, FInt::LuauTypeCloneRecursionLimit); + TypePackId& res = seenTypePacks[tp]; if (res == nullptr) { - TypePackCloner cloner{dest, tp, seenTypes, seenTypePacks}; - cloner.encounteredFreeType = encounteredFreeType; + TypePackCloner cloner{dest, tp, seenTypes, seenTypePacks, cloneState}; Luau::visit(cloner, tp->ty); // Mutates the storage that 'res' points into. } @@ -446,17 +427,18 @@ TypePackId clone(TypePackId tp, TypeArena& dest, SeenTypes& seenTypes, SeenTypeP return res; } -TypeId clone(TypeId typeId, TypeArena& dest, SeenTypes& seenTypes, SeenTypePacks& seenTypePacks, bool* encounteredFreeType) +TypeId clone(TypeId typeId, TypeArena& dest, SeenTypes& seenTypes, SeenTypePacks& seenTypePacks, CloneState& cloneState) { if (typeId->persistent) return typeId; + RecursionLimiter _ra(&cloneState.recursionCount, FInt::LuauTypeCloneRecursionLimit); + TypeId& res = seenTypes[typeId]; if (res == nullptr) { - TypeCloner cloner{dest, typeId, seenTypes, seenTypePacks}; - cloner.encounteredFreeType = encounteredFreeType; + TypeCloner cloner{dest, typeId, seenTypes, seenTypePacks, cloneState}; Luau::visit(cloner, typeId->ty); // Mutates the storage that 'res' points into. asMutable(res)->documentationSymbol = typeId->documentationSymbol; } @@ -467,19 +449,16 @@ TypeId clone(TypeId typeId, TypeArena& dest, SeenTypes& seenTypes, SeenTypePacks return res; } -TypeFun clone(const TypeFun& typeFun, TypeArena& dest, SeenTypes& seenTypes, SeenTypePacks& seenTypePacks, bool* encounteredFreeType) +TypeFun clone(const TypeFun& typeFun, TypeArena& dest, SeenTypes& seenTypes, SeenTypePacks& seenTypePacks, CloneState& cloneState) { TypeFun result; for (TypeId ty : typeFun.typeParams) - result.typeParams.push_back(clone(ty, dest, seenTypes, seenTypePacks, encounteredFreeType)); + result.typeParams.push_back(clone(ty, dest, seenTypes, seenTypePacks, cloneState)); - if (FFlag::LuauTypeAliasPacks) - { - for (TypePackId tp : typeFun.typePackParams) - result.typePackParams.push_back(clone(tp, dest, seenTypes, seenTypePacks, encounteredFreeType)); - } + for (TypePackId tp : typeFun.typePackParams) + result.typePackParams.push_back(clone(tp, dest, seenTypes, seenTypePacks, cloneState)); - result.type = clone(typeFun.type, dest, seenTypes, seenTypePacks, encounteredFreeType); + result.type = clone(typeFun.type, dest, seenTypes, seenTypePacks, cloneState); return result; } @@ -519,19 +498,18 @@ bool Module::clonePublicInterface() LUAU_ASSERT(interfaceTypes.typeVars.empty()); LUAU_ASSERT(interfaceTypes.typePacks.empty()); - bool encounteredFreeType = false; - - SeenTypePacks seenTypePacks; SeenTypes seenTypes; + SeenTypePacks seenTypePacks; + CloneState cloneState; ScopePtr moduleScope = getModuleScope(); - moduleScope->returnType = clone(moduleScope->returnType, interfaceTypes, seenTypes, seenTypePacks, &encounteredFreeType); + moduleScope->returnType = clone(moduleScope->returnType, interfaceTypes, seenTypes, seenTypePacks, cloneState); if (moduleScope->varargPack) - moduleScope->varargPack = clone(*moduleScope->varargPack, interfaceTypes, seenTypes, seenTypePacks, &encounteredFreeType); + moduleScope->varargPack = clone(*moduleScope->varargPack, interfaceTypes, seenTypes, seenTypePacks, cloneState); for (auto& pair : moduleScope->exportedTypeBindings) - pair.second = clone(pair.second, interfaceTypes, seenTypes, seenTypePacks, &encounteredFreeType); + pair.second = clone(pair.second, interfaceTypes, seenTypes, seenTypePacks, cloneState); for (TypeId ty : moduleScope->returnType) if (get(follow(ty))) @@ -540,7 +518,7 @@ bool Module::clonePublicInterface() freeze(internalTypes); freeze(interfaceTypes); - return encounteredFreeType; + return cloneState.encounteredFreeType; } } // namespace Luau diff --git a/Analysis/src/Quantify.cpp b/Analysis/src/Quantify.cpp index bf6d81aa..c773e208 100644 --- a/Analysis/src/Quantify.cpp +++ b/Analysis/src/Quantify.cpp @@ -4,6 +4,8 @@ #include "Luau/VisitTypeVar.h" +LUAU_FASTFLAGVARIABLE(LuauQuantifyVisitOnce, false) + namespace Luau { @@ -79,7 +81,16 @@ struct Quantifier void quantify(ModulePtr module, TypeId ty, TypeLevel level) { Quantifier q{std::move(module), level}; - visitTypeVar(ty, q); + + if (FFlag::LuauQuantifyVisitOnce) + { + DenseHashSet seen{nullptr}; + visitTypeVarOnce(ty, q, seen); + } + else + { + visitTypeVar(ty, q); + } FunctionTypeVar* ftv = getMutable(ty); LUAU_ASSERT(ftv); diff --git a/Analysis/src/RequireTracer.cpp b/Analysis/src/RequireTracer.cpp index b72f53f9..8ed245fb 100644 --- a/Analysis/src/RequireTracer.cpp +++ b/Analysis/src/RequireTracer.cpp @@ -4,182 +4,9 @@ #include "Luau/Ast.h" #include "Luau/Module.h" -LUAU_FASTFLAGVARIABLE(LuauTraceRequireLookupChild, false) -LUAU_FASTFLAGVARIABLE(LuauNewRequireTrace2, false) - namespace Luau { -namespace -{ - -struct RequireTracerOld : AstVisitor -{ - explicit RequireTracerOld(FileResolver* fileResolver, const ModuleName& currentModuleName) - : fileResolver(fileResolver) - , currentModuleName(currentModuleName) - { - LUAU_ASSERT(!FFlag::LuauNewRequireTrace2); - } - - FileResolver* const fileResolver; - ModuleName currentModuleName; - DenseHashMap locals{nullptr}; - RequireTraceResult result; - - std::optional fromAstFragment(AstExpr* expr) - { - if (auto g = expr->as(); g && g->name == "script") - return currentModuleName; - - return fileResolver->fromAstFragment(expr); - } - - bool visit(AstStatLocal* stat) override - { - for (size_t i = 0; i < stat->vars.size; ++i) - { - AstLocal* local = stat->vars.data[i]; - - if (local->annotation) - { - if (AstTypeTypeof* ann = local->annotation->as()) - ann->expr->visit(this); - } - - if (i < stat->values.size) - { - AstExpr* expr = stat->values.data[i]; - expr->visit(this); - - const ModuleInfo* info = result.exprs.find(expr); - if (info) - locals[local] = info->name; - } - } - - return false; - } - - bool visit(AstExprGlobal* global) override - { - std::optional name = fromAstFragment(global); - if (name) - result.exprs[global] = {*name}; - - return false; - } - - bool visit(AstExprLocal* local) override - { - const ModuleName* name = locals.find(local->local); - if (name) - result.exprs[local] = {*name}; - - return false; - } - - bool visit(AstExprIndexName* indexName) override - { - indexName->expr->visit(this); - - const ModuleInfo* info = result.exprs.find(indexName->expr); - if (info) - { - if (indexName->index == "parent" || indexName->index == "Parent") - { - if (auto parent = fileResolver->getParentModuleName(info->name)) - result.exprs[indexName] = {*parent}; - } - else - result.exprs[indexName] = {fileResolver->concat(info->name, indexName->index.value)}; - } - - return false; - } - - bool visit(AstExprIndexExpr* indexExpr) override - { - indexExpr->expr->visit(this); - - const ModuleInfo* info = result.exprs.find(indexExpr->expr); - const AstExprConstantString* str = indexExpr->index->as(); - if (info && str) - { - result.exprs[indexExpr] = {fileResolver->concat(info->name, std::string_view(str->value.data, str->value.size))}; - } - - indexExpr->index->visit(this); - - return false; - } - - bool visit(AstExprTypeAssertion* expr) override - { - return false; - } - - // If we see game:GetService("StringLiteral") or Game:GetService("StringLiteral"), then rewrite to game.StringLiteral. - // Else traverse arguments and trace requires to them. - bool visit(AstExprCall* call) override - { - for (AstExpr* arg : call->args) - arg->visit(this); - - call->func->visit(this); - - AstExprGlobal* globalName = call->func->as(); - if (globalName && globalName->name == "require" && call->args.size >= 1) - { - if (const ModuleInfo* moduleInfo = result.exprs.find(call->args.data[0])) - result.requires.push_back({moduleInfo->name, call->location}); - - return false; - } - - AstExprIndexName* indexName = call->func->as(); - if (!indexName) - return false; - - std::optional rootName = fromAstFragment(indexName->expr); - - if (FFlag::LuauTraceRequireLookupChild && !rootName) - { - if (const ModuleInfo* moduleInfo = result.exprs.find(indexName->expr)) - rootName = moduleInfo->name; - } - - if (!rootName) - return false; - - bool supportedLookup = indexName->index == "GetService" || - (FFlag::LuauTraceRequireLookupChild && (indexName->index == "FindFirstChild" || indexName->index == "WaitForChild")); - - if (!supportedLookup) - return false; - - if (call->args.size != 1) - return false; - - AstExprConstantString* name = call->args.data[0]->as(); - if (!name) - return false; - - std::string_view v{name->value.data, name->value.size}; - if (v.end() != std::find(v.begin(), v.end(), '/')) - return false; - - result.exprs[call] = {fileResolver->concat(*rootName, v)}; - - // 'WaitForChild' can be used on modules that are not available at the typecheck time, but will be available at runtime - // If we fail to find such module, we will not report an UnknownRequire error - if (FFlag::LuauTraceRequireLookupChild && indexName->index == "WaitForChild") - result.exprs[call].optional = true; - - return false; - } -}; - struct RequireTracer : AstVisitor { RequireTracer(RequireTraceResult& result, FileResolver* fileResolver, const ModuleName& currentModuleName) @@ -188,7 +15,6 @@ struct RequireTracer : AstVisitor , currentModuleName(currentModuleName) , locals(nullptr) { - LUAU_ASSERT(FFlag::LuauNewRequireTrace2); } bool visit(AstExprTypeAssertion* expr) override @@ -328,24 +154,13 @@ struct RequireTracer : AstVisitor std::vector requires; }; -} // anonymous namespace - RequireTraceResult traceRequires(FileResolver* fileResolver, AstStatBlock* root, const ModuleName& currentModuleName) { - if (FFlag::LuauNewRequireTrace2) - { - RequireTraceResult result; - RequireTracer tracer{result, fileResolver, currentModuleName}; - root->visit(&tracer); - tracer.process(); - return result; - } - else - { - RequireTracerOld tracer{fileResolver, currentModuleName}; - root->visit(&tracer); - return tracer.result; - } + RequireTraceResult result; + RequireTracer tracer{result, fileResolver, currentModuleName}; + root->visit(&tracer); + tracer.process(); + return result; } } // namespace Luau diff --git a/Analysis/src/Substitution.cpp b/Analysis/src/Substitution.cpp index ca2b30f5..3d004bee 100644 --- a/Analysis/src/Substitution.cpp +++ b/Analysis/src/Substitution.cpp @@ -7,8 +7,6 @@ #include LUAU_FASTINTVARIABLE(LuauTarjanChildLimit, 1000) -LUAU_FASTFLAGVARIABLE(LuauSubstitutionDontReplaceIgnoredTypes, false) -LUAU_FASTFLAG(LuauTypeAliasPacks) namespace Luau { @@ -39,11 +37,8 @@ void Tarjan::visitChildren(TypeId ty, int index) for (TypeId itp : ttv->instantiatedTypeParams) visitChild(itp); - if (FFlag::LuauTypeAliasPacks) - { - for (TypePackId itp : ttv->instantiatedTypePackParams) - visitChild(itp); - } + for (TypePackId itp : ttv->instantiatedTypePackParams) + visitChild(itp); } else if (const MetatableTypeVar* mtv = get(ty)) { @@ -339,10 +334,10 @@ std::optional Substitution::substitute(TypeId ty) return std::nullopt; for (auto [oldTy, newTy] : newTypes) - if (!FFlag::LuauSubstitutionDontReplaceIgnoredTypes || !ignoreChildren(oldTy)) + if (!ignoreChildren(oldTy)) replaceChildren(newTy); for (auto [oldTp, newTp] : newPacks) - if (!FFlag::LuauSubstitutionDontReplaceIgnoredTypes || !ignoreChildren(oldTp)) + if (!ignoreChildren(oldTp)) replaceChildren(newTp); TypeId newTy = replace(ty); return newTy; @@ -359,10 +354,10 @@ std::optional Substitution::substitute(TypePackId tp) return std::nullopt; for (auto [oldTy, newTy] : newTypes) - if (!FFlag::LuauSubstitutionDontReplaceIgnoredTypes || !ignoreChildren(oldTy)) + if (!ignoreChildren(oldTy)) replaceChildren(newTy); for (auto [oldTp, newTp] : newPacks) - if (!FFlag::LuauSubstitutionDontReplaceIgnoredTypes || !ignoreChildren(oldTp)) + if (!ignoreChildren(oldTp)) replaceChildren(newTp); TypePackId newTp = replace(tp); return newTp; @@ -393,10 +388,7 @@ TypeId Substitution::clone(TypeId ty) clone.name = ttv->name; clone.syntheticName = ttv->syntheticName; clone.instantiatedTypeParams = ttv->instantiatedTypeParams; - - if (FFlag::LuauTypeAliasPacks) - clone.instantiatedTypePackParams = ttv->instantiatedTypePackParams; - + clone.instantiatedTypePackParams = ttv->instantiatedTypePackParams; clone.tags = ttv->tags; result = addType(std::move(clone)); } @@ -505,11 +497,8 @@ void Substitution::replaceChildren(TypeId ty) for (TypeId& itp : ttv->instantiatedTypeParams) itp = replace(itp); - if (FFlag::LuauTypeAliasPacks) - { - for (TypePackId& itp : ttv->instantiatedTypePackParams) - itp = replace(itp); - } + for (TypePackId& itp : ttv->instantiatedTypePackParams) + itp = replace(itp); } else if (MetatableTypeVar* mtv = getMutable(ty)) { diff --git a/Analysis/src/ToDot.cpp b/Analysis/src/ToDot.cpp new file mode 100644 index 00000000..df9d4188 --- /dev/null +++ b/Analysis/src/ToDot.cpp @@ -0,0 +1,378 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +#include "Luau/ToDot.h" + +#include "Luau/ToString.h" +#include "Luau/TypePack.h" +#include "Luau/TypeVar.h" +#include "Luau/StringUtils.h" + +#include +#include + +namespace Luau +{ + +namespace +{ + +struct StateDot +{ + StateDot(ToDotOptions opts) + : opts(opts) + { + } + + ToDotOptions opts; + + std::unordered_set seenTy; + std::unordered_set seenTp; + std::unordered_map tyToIndex; + std::unordered_map tpToIndex; + int nextIndex = 1; + std::string result; + + bool canDuplicatePrimitive(TypeId ty); + + void visitChildren(TypeId ty, int index); + void visitChildren(TypePackId ty, int index); + + void visitChild(TypeId ty, int parentIndex, const char* linkName = nullptr); + void visitChild(TypePackId tp, int parentIndex, const char* linkName = nullptr); + + void startNode(int index); + void finishNode(); + + void startNodeLabel(); + void finishNodeLabel(TypeId ty); + void finishNodeLabel(TypePackId tp); +}; + +bool StateDot::canDuplicatePrimitive(TypeId ty) +{ + if (get(ty)) + return false; + + return get(ty) || get(ty); +} + +void StateDot::visitChild(TypeId ty, int parentIndex, const char* linkName) +{ + if (!tyToIndex.count(ty) || (opts.duplicatePrimitives && canDuplicatePrimitive(ty))) + tyToIndex[ty] = nextIndex++; + + int index = tyToIndex[ty]; + + if (parentIndex != 0) + { + if (linkName) + formatAppend(result, "n%d -> n%d [label=\"%s\"];\n", parentIndex, index, linkName); + else + formatAppend(result, "n%d -> n%d;\n", parentIndex, index); + } + + if (opts.duplicatePrimitives && canDuplicatePrimitive(ty)) + { + if (get(ty)) + formatAppend(result, "n%d [label=\"%s\"];\n", index, toStringDetailed(ty, {}).name.c_str()); + else if (get(ty)) + formatAppend(result, "n%d [label=\"any\"];\n", index); + } + else + { + visitChildren(ty, index); + } +} + +void StateDot::visitChild(TypePackId tp, int parentIndex, const char* linkName) +{ + if (!tpToIndex.count(tp)) + tpToIndex[tp] = nextIndex++; + + if (parentIndex != 0) + { + if (linkName) + formatAppend(result, "n%d -> n%d [label=\"%s\"];\n", parentIndex, tpToIndex[tp], linkName); + else + formatAppend(result, "n%d -> n%d;\n", parentIndex, tpToIndex[tp]); + } + + visitChildren(tp, tpToIndex[tp]); +} + +void StateDot::startNode(int index) +{ + formatAppend(result, "n%d [", index); +} + +void StateDot::finishNode() +{ + formatAppend(result, "];\n"); +} + +void StateDot::startNodeLabel() +{ + formatAppend(result, "label=\""); +} + +void StateDot::finishNodeLabel(TypeId ty) +{ + if (opts.showPointers) + formatAppend(result, "\n0x%p", ty); + // additional common attributes can be added here as well + result += "\""; +} + +void StateDot::finishNodeLabel(TypePackId tp) +{ + if (opts.showPointers) + formatAppend(result, "\n0x%p", tp); + // additional common attributes can be added here as well + result += "\""; +} + +void StateDot::visitChildren(TypeId ty, int index) +{ + if (seenTy.count(ty)) + return; + seenTy.insert(ty); + + startNode(index); + startNodeLabel(); + + if (const BoundTypeVar* btv = get(ty)) + { + formatAppend(result, "BoundTypeVar %d", index); + finishNodeLabel(ty); + finishNode(); + + visitChild(btv->boundTo, index); + } + else if (const FunctionTypeVar* ftv = get(ty)) + { + formatAppend(result, "FunctionTypeVar %d", index); + finishNodeLabel(ty); + finishNode(); + + visitChild(ftv->argTypes, index, "arg"); + visitChild(ftv->retType, index, "ret"); + } + else if (const TableTypeVar* ttv = get(ty)) + { + if (ttv->name) + formatAppend(result, "TableTypeVar %s", ttv->name->c_str()); + else if (ttv->syntheticName) + formatAppend(result, "TableTypeVar %s", ttv->syntheticName->c_str()); + else + formatAppend(result, "TableTypeVar %d", index); + finishNodeLabel(ty); + finishNode(); + + if (ttv->boundTo) + return visitChild(*ttv->boundTo, index, "boundTo"); + + for (const auto& [name, prop] : ttv->props) + visitChild(prop.type, index, name.c_str()); + if (ttv->indexer) + { + visitChild(ttv->indexer->indexType, index, "[index]"); + visitChild(ttv->indexer->indexResultType, index, "[value]"); + } + for (TypeId itp : ttv->instantiatedTypeParams) + visitChild(itp, index, "typeParam"); + + for (TypePackId itp : ttv->instantiatedTypePackParams) + visitChild(itp, index, "typePackParam"); + } + else if (const MetatableTypeVar* mtv = get(ty)) + { + formatAppend(result, "MetatableTypeVar %d", index); + finishNodeLabel(ty); + finishNode(); + + visitChild(mtv->table, index, "table"); + visitChild(mtv->metatable, index, "metatable"); + } + else if (const UnionTypeVar* utv = get(ty)) + { + formatAppend(result, "UnionTypeVar %d", index); + finishNodeLabel(ty); + finishNode(); + + for (TypeId opt : utv->options) + visitChild(opt, index); + } + else if (const IntersectionTypeVar* itv = get(ty)) + { + formatAppend(result, "IntersectionTypeVar %d", index); + finishNodeLabel(ty); + finishNode(); + + for (TypeId part : itv->parts) + visitChild(part, index); + } + else if (const GenericTypeVar* gtv = get(ty)) + { + if (gtv->explicitName) + formatAppend(result, "GenericTypeVar %s", gtv->name.c_str()); + else + formatAppend(result, "GenericTypeVar %d", index); + finishNodeLabel(ty); + finishNode(); + } + else if (const FreeTypeVar* ftv = get(ty)) + { + formatAppend(result, "FreeTypeVar %d", index); + finishNodeLabel(ty); + finishNode(); + } + else if (get(ty)) + { + formatAppend(result, "AnyTypeVar %d", index); + finishNodeLabel(ty); + finishNode(); + } + else if (get(ty)) + { + formatAppend(result, "PrimitiveTypeVar %s", toStringDetailed(ty, {}).name.c_str()); + finishNodeLabel(ty); + finishNode(); + } + else if (get(ty)) + { + formatAppend(result, "ErrorTypeVar %d", index); + finishNodeLabel(ty); + finishNode(); + } + else if (const ClassTypeVar* ctv = get(ty)) + { + formatAppend(result, "ClassTypeVar %s", ctv->name.c_str()); + finishNodeLabel(ty); + finishNode(); + + for (const auto& [name, prop] : ctv->props) + visitChild(prop.type, index, name.c_str()); + + if (ctv->parent) + visitChild(*ctv->parent, index, "[parent]"); + + if (ctv->metatable) + visitChild(*ctv->metatable, index, "[metatable]"); + } + else + { + LUAU_ASSERT(!"unknown type kind"); + finishNodeLabel(ty); + finishNode(); + } +} + +void StateDot::visitChildren(TypePackId tp, int index) +{ + if (seenTp.count(tp)) + return; + seenTp.insert(tp); + + startNode(index); + startNodeLabel(); + + if (const BoundTypePack* btp = get(tp)) + { + formatAppend(result, "BoundTypePack %d", index); + finishNodeLabel(tp); + finishNode(); + + visitChild(btp->boundTo, index); + } + else if (const TypePack* tpp = get(tp)) + { + formatAppend(result, "TypePack %d", index); + finishNodeLabel(tp); + finishNode(); + + for (TypeId tv : tpp->head) + visitChild(tv, index); + if (tpp->tail) + visitChild(*tpp->tail, index, "tail"); + } + else if (const VariadicTypePack* vtp = get(tp)) + { + formatAppend(result, "VariadicTypePack %d", index); + finishNodeLabel(tp); + finishNode(); + + visitChild(vtp->ty, index); + } + else if (const FreeTypePack* ftp = get(tp)) + { + formatAppend(result, "FreeTypePack %d", index); + finishNodeLabel(tp); + finishNode(); + } + else if (const GenericTypePack* gtp = get(tp)) + { + if (gtp->explicitName) + formatAppend(result, "GenericTypePack %s", gtp->name.c_str()); + else + formatAppend(result, "GenericTypePack %d", index); + finishNodeLabel(tp); + finishNode(); + } + else if (get(tp)) + { + formatAppend(result, "ErrorTypePack %d", index); + finishNodeLabel(tp); + finishNode(); + } + else + { + LUAU_ASSERT(!"unknown type pack kind"); + finishNodeLabel(tp); + finishNode(); + } +} + +} // namespace + +std::string toDot(TypeId ty, const ToDotOptions& opts) +{ + StateDot state{opts}; + + state.result = "digraph graphname {\n"; + state.visitChild(ty, 0); + state.result += "}"; + + return state.result; +} + +std::string toDot(TypePackId tp, const ToDotOptions& opts) +{ + StateDot state{opts}; + + state.result = "digraph graphname {\n"; + state.visitChild(tp, 0); + state.result += "}"; + + return state.result; +} + +std::string toDot(TypeId ty) +{ + return toDot(ty, {}); +} + +std::string toDot(TypePackId tp) +{ + return toDot(tp, {}); +} + +void dumpDot(TypeId ty) +{ + printf("%s\n", toDot(ty).c_str()); +} + +void dumpDot(TypePackId tp) +{ + printf("%s\n", toDot(tp).c_str()); +} + +} // namespace Luau diff --git a/Analysis/src/ToString.cpp b/Analysis/src/ToString.cpp index 735bfa50..6322096c 100644 --- a/Analysis/src/ToString.cpp +++ b/Analysis/src/ToString.cpp @@ -11,7 +11,7 @@ #include LUAU_FASTFLAG(LuauOccursCheckOkWithRecursiveFunctions) -LUAU_FASTFLAG(LuauTypeAliasPacks) +LUAU_FASTFLAGVARIABLE(LuauFunctionArgumentNameSize, false) namespace Luau { @@ -59,11 +59,8 @@ struct FindCyclicTypes for (TypeId itp : ttv.instantiatedTypeParams) visitTypeVar(itp, *this, seen); - if (FFlag::LuauTypeAliasPacks) - { - for (TypePackId itp : ttv.instantiatedTypePackParams) - visitTypeVar(itp, *this, seen); - } + for (TypePackId itp : ttv.instantiatedTypePackParams) + visitTypeVar(itp, *this, seen); return exhaustive; } @@ -248,58 +245,45 @@ struct TypeVarStringifier void stringify(const std::vector& types, const std::vector& typePacks) { - if (types.size() == 0 && (!FFlag::LuauTypeAliasPacks || typePacks.size() == 0)) + if (types.size() == 0 && typePacks.size() == 0) return; - if (types.size() || (FFlag::LuauTypeAliasPacks && typePacks.size())) + if (types.size() || typePacks.size()) state.emit("<"); - if (FFlag::LuauTypeAliasPacks) - { - bool first = true; + bool first = true; - for (TypeId ty : types) - { - if (!first) - state.emit(", "); + for (TypeId ty : types) + { + if (!first) + state.emit(", "); + first = false; + + stringify(ty); + } + + bool singleTp = typePacks.size() == 1; + + for (TypePackId tp : typePacks) + { + if (isEmpty(tp) && singleTp) + continue; + + if (!first) + state.emit(", "); + else first = false; - stringify(ty); - } + if (!singleTp) + state.emit("("); - bool singleTp = typePacks.size() == 1; + stringify(tp); - for (TypePackId tp : typePacks) - { - if (isEmpty(tp) && singleTp) - continue; - - if (!first) - state.emit(", "); - else - first = false; - - if (!singleTp) - state.emit("("); - - stringify(tp); - - if (!singleTp) - state.emit(")"); - } - } - else - { - for (size_t i = 0; i < types.size(); ++i) - { - if (i > 0) - state.emit(", "); - - stringify(types[i]); - } + if (!singleTp) + state.emit(")"); } - if (types.size() || (FFlag::LuauTypeAliasPacks && typePacks.size())) + if (types.size() || typePacks.size()) state.emit(">"); } @@ -767,12 +751,23 @@ struct TypePackStringifier else state.emit(", "); - LUAU_ASSERT(elemNames.empty() || elemIndex < elemNames.size()); - - if (!elemNames.empty() && elemNames[elemIndex]) + if (FFlag::LuauFunctionArgumentNameSize) { - state.emit(elemNames[elemIndex]->name); - state.emit(": "); + if (elemIndex < elemNames.size() && elemNames[elemIndex]) + { + state.emit(elemNames[elemIndex]->name); + state.emit(": "); + } + } + else + { + LUAU_ASSERT(elemNames.empty() || elemIndex < elemNames.size()); + + if (!elemNames.empty() && elemNames[elemIndex]) + { + state.emit(elemNames[elemIndex]->name); + state.emit(": "); + } } elemIndex++; @@ -929,38 +924,7 @@ ToStringResult toStringDetailed(TypeId ty, const ToStringOptions& opts) result.name += ttv->name ? *ttv->name : *ttv->syntheticName; - if (FFlag::LuauTypeAliasPacks) - { - tvs.stringify(ttv->instantiatedTypeParams, ttv->instantiatedTypePackParams); - } - else - { - if (ttv->instantiatedTypeParams.empty() && (!FFlag::LuauTypeAliasPacks || ttv->instantiatedTypePackParams.empty())) - return result; - - result.name += "<"; - - bool first = true; - for (TypeId ty : ttv->instantiatedTypeParams) - { - if (!first) - result.name += ", "; - else - first = false; - - tvs.stringify(ty); - } - - if (opts.maxTypeLength > 0 && result.name.length() > opts.maxTypeLength) - { - result.truncated = true; - result.name += "... "; - } - else - { - result.name += ">"; - } - } + tvs.stringify(ttv->instantiatedTypeParams, ttv->instantiatedTypePackParams); return result; } @@ -1161,17 +1125,37 @@ std::string toStringNamedFunction(const std::string& prefix, const FunctionTypeV s += ", "; first = false; - // argNames is guaranteed to be equal to argTypes iff argNames is not empty. - // We don't currently respect opts.functionTypeArguments. I don't think this function should. - if (!ftv.argNames.empty()) - s += (*argNameIter ? (*argNameIter)->name : "_") + ": "; - s += toString_(*argPackIter); - - ++argPackIter; - if (!ftv.argNames.empty()) + if (FFlag::LuauFunctionArgumentNameSize) { - LUAU_ASSERT(argNameIter != ftv.argNames.end()); - ++argNameIter; + // We don't currently respect opts.functionTypeArguments. I don't think this function should. + if (argNameIter != ftv.argNames.end()) + { + s += (*argNameIter ? (*argNameIter)->name : "_") + ": "; + ++argNameIter; + } + else + { + s += "_: "; + } + } + else + { + // argNames is guaranteed to be equal to argTypes iff argNames is not empty. + // We don't currently respect opts.functionTypeArguments. I don't think this function should. + if (!ftv.argNames.empty()) + s += (*argNameIter ? (*argNameIter)->name : "_") + ": "; + } + + s += toString_(*argPackIter); + ++argPackIter; + + if (!FFlag::LuauFunctionArgumentNameSize) + { + if (!ftv.argNames.empty()) + { + LUAU_ASSERT(argNameIter != ftv.argNames.end()); + ++argNameIter; + } } } diff --git a/Analysis/src/Transpiler.cpp b/Analysis/src/Transpiler.cpp index 6627fbe3..8e13ea5b 100644 --- a/Analysis/src/Transpiler.cpp +++ b/Analysis/src/Transpiler.cpp @@ -10,8 +10,6 @@ #include #include -LUAU_FASTFLAG(LuauTypeAliasPacks) - namespace { bool isIdentifierStartChar(char c) @@ -787,7 +785,7 @@ struct Printer writer.keyword("type"); writer.identifier(a->name.value); - if (a->generics.size > 0 || (FFlag::LuauTypeAliasPacks && a->genericPacks.size > 0)) + if (a->generics.size > 0 || a->genericPacks.size > 0) { writer.symbol("<"); CommaSeparatorInserter comma(writer); @@ -798,14 +796,11 @@ struct Printer writer.identifier(o.value); } - if (FFlag::LuauTypeAliasPacks) + for (auto o : a->genericPacks) { - for (auto o : a->genericPacks) - { - comma(); - writer.identifier(o.value); - writer.symbol("..."); - } + comma(); + writer.identifier(o.value); + writer.symbol("..."); } writer.symbol(">"); diff --git a/Analysis/src/TxnLog.cpp b/Analysis/src/TxnLog.cpp index 383bb050..f6a61581 100644 --- a/Analysis/src/TxnLog.cpp +++ b/Analysis/src/TxnLog.cpp @@ -5,8 +5,6 @@ #include -LUAU_FASTFLAGVARIABLE(LuauShareTxnSeen, false) - namespace Luau { @@ -36,11 +34,8 @@ void TxnLog::rollback() for (auto it = tableChanges.rbegin(); it != tableChanges.rend(); ++it) std::swap(it->first->boundTo, it->second); - if (FFlag::LuauShareTxnSeen) - { - LUAU_ASSERT(originalSeenSize <= sharedSeen->size()); - sharedSeen->resize(originalSeenSize); - } + LUAU_ASSERT(originalSeenSize <= sharedSeen->size()); + sharedSeen->resize(originalSeenSize); } void TxnLog::concat(TxnLog rhs) @@ -53,45 +48,25 @@ void TxnLog::concat(TxnLog rhs) tableChanges.insert(tableChanges.end(), rhs.tableChanges.begin(), rhs.tableChanges.end()); rhs.tableChanges.clear(); - - if (!FFlag::LuauShareTxnSeen) - { - ownedSeen.swap(rhs.ownedSeen); - rhs.ownedSeen.clear(); - } } bool TxnLog::haveSeen(TypeId lhs, TypeId rhs) { const std::pair sortedPair = (lhs > rhs) ? std::make_pair(lhs, rhs) : std::make_pair(rhs, lhs); - if (FFlag::LuauShareTxnSeen) - return (sharedSeen->end() != std::find(sharedSeen->begin(), sharedSeen->end(), sortedPair)); - else - return (ownedSeen.end() != std::find(ownedSeen.begin(), ownedSeen.end(), sortedPair)); + return (sharedSeen->end() != std::find(sharedSeen->begin(), sharedSeen->end(), sortedPair)); } void TxnLog::pushSeen(TypeId lhs, TypeId rhs) { const std::pair sortedPair = (lhs > rhs) ? std::make_pair(lhs, rhs) : std::make_pair(rhs, lhs); - if (FFlag::LuauShareTxnSeen) - sharedSeen->push_back(sortedPair); - else - ownedSeen.push_back(sortedPair); + sharedSeen->push_back(sortedPair); } void TxnLog::popSeen(TypeId lhs, TypeId rhs) { const std::pair sortedPair = (lhs > rhs) ? std::make_pair(lhs, rhs) : std::make_pair(rhs, lhs); - if (FFlag::LuauShareTxnSeen) - { - LUAU_ASSERT(sortedPair == sharedSeen->back()); - sharedSeen->pop_back(); - } - else - { - LUAU_ASSERT(sortedPair == ownedSeen.back()); - ownedSeen.pop_back(); - } + LUAU_ASSERT(sortedPair == sharedSeen->back()); + sharedSeen->pop_back(); } } // namespace Luau diff --git a/Analysis/src/TypeAttach.cpp b/Analysis/src/TypeAttach.cpp index af6d2543..9e61c792 100644 --- a/Analysis/src/TypeAttach.cpp +++ b/Analysis/src/TypeAttach.cpp @@ -13,8 +13,6 @@ #include -LUAU_FASTFLAG(LuauTypeAliasPacks) - static char* allocateString(Luau::Allocator& allocator, std::string_view contents) { char* result = (char*)allocator.allocate(contents.size() + 1); @@ -131,12 +129,9 @@ public: parameters.data[i] = {Luau::visit(*this, ttv.instantiatedTypeParams[i]->ty), {}}; } - if (FFlag::LuauTypeAliasPacks) + for (size_t i = 0; i < ttv.instantiatedTypePackParams.size(); ++i) { - for (size_t i = 0; i < ttv.instantiatedTypePackParams.size(); ++i) - { - parameters.data[i] = {{}, rehydrate(ttv.instantiatedTypePackParams[i])}; - } + parameters.data[i] = {{}, rehydrate(ttv.instantiatedTypePackParams[i])}; } return allocator->alloc(Location(), std::nullopt, AstName(ttv.name->c_str()), parameters.size != 0, parameters); @@ -250,20 +245,7 @@ public: AstTypePack* argTailAnnotation = nullptr; if (argTail) - { - if (FFlag::LuauTypeAliasPacks) - { - argTailAnnotation = rehydrate(*argTail); - } - else - { - TypePackId tail = *argTail; - if (const VariadicTypePack* vtp = get(tail)) - { - argTailAnnotation = allocator->alloc(Location(), Luau::visit(*this, vtp->ty->ty)); - } - } - } + argTailAnnotation = rehydrate(*argTail); AstArray> argNames; argNames.size = ftv.argNames.size(); @@ -292,20 +274,7 @@ public: AstTypePack* retTailAnnotation = nullptr; if (retTail) - { - if (FFlag::LuauTypeAliasPacks) - { - retTailAnnotation = rehydrate(*retTail); - } - else - { - TypePackId tail = *retTail; - if (const VariadicTypePack* vtp = get(tail)) - { - retTailAnnotation = allocator->alloc(Location(), Luau::visit(*this, vtp->ty->ty)); - } - } - } + retTailAnnotation = rehydrate(*retTail); return allocator->alloc( Location(), generics, genericPacks, AstTypeList{argTypes, argTailAnnotation}, argNames, AstTypeList{returnTypes, retTailAnnotation}); @@ -518,18 +487,7 @@ public: const auto& [v, tail] = flatten(ret); if (tail) - { - if (FFlag::LuauTypeAliasPacks) - { - variadicAnnotation = TypeRehydrationVisitor(allocator, &syntheticNames).rehydrate(*tail); - } - else - { - TypePackId tailPack = *tail; - if (const VariadicTypePack* vtp = get(tailPack)) - variadicAnnotation = allocator->alloc(Location(), typeAst(vtp->ty)); - } - } + variadicAnnotation = TypeRehydrationVisitor(allocator, &syntheticNames).rehydrate(*tail); fn->returnAnnotation = AstTypeList{typeAstPack(ret), variadicAnnotation}; } diff --git a/Analysis/src/TypeInfer.cpp b/Analysis/src/TypeInfer.cpp index b2ae94c7..617bf482 100644 --- a/Analysis/src/TypeInfer.cpp +++ b/Analysis/src/TypeInfer.cpp @@ -23,22 +23,20 @@ LUAU_FASTINTVARIABLE(LuauTypeInferRecursionLimit, 500) LUAU_FASTINTVARIABLE(LuauTypeInferTypePackLoopLimit, 5000) LUAU_FASTINTVARIABLE(LuauCheckRecursionLimit, 500) LUAU_FASTFLAG(LuauKnowsTheDataModel3) -LUAU_FASTFLAGVARIABLE(LuauClassPropertyAccessAsString, false) LUAU_FASTFLAGVARIABLE(LuauEqConstraint, false) LUAU_FASTFLAGVARIABLE(LuauWeakEqConstraint, false) // Eventually removed as false. -LUAU_FASTFLAG(LuauTraceRequireLookupChild) LUAU_FASTFLAGVARIABLE(LuauCloneCorrectlyBeforeMutatingTableType, false) LUAU_FASTFLAGVARIABLE(LuauStoreMatchingOverloadFnType, false) LUAU_FASTFLAGVARIABLE(LuauRecursiveTypeParameterRestriction, false) LUAU_FASTFLAGVARIABLE(LuauIfElseExpressionAnalysisSupport, false) LUAU_FASTFLAGVARIABLE(LuauStrictRequire, false) -LUAU_FASTFLAG(LuauSubstitutionDontReplaceIgnoredTypes) LUAU_FASTFLAGVARIABLE(LuauQuantifyInPlace2, false) -LUAU_FASTFLAG(LuauNewRequireTrace2) -LUAU_FASTFLAG(LuauTypeAliasPacks) LUAU_FASTFLAGVARIABLE(LuauSingletonTypes, false) LUAU_FASTFLAGVARIABLE(LuauExpectedTypesOfProperties, false) LUAU_FASTFLAGVARIABLE(LuauErrorRecoveryType, false) +LUAU_FASTFLAGVARIABLE(LuauPropertiesGetExpectedType, false) +LUAU_FASTFLAGVARIABLE(LuauTailArgumentTypeInfo, false) +LUAU_FASTFLAGVARIABLE(LuauModuleRequireErrorPack, false) namespace Luau { @@ -562,12 +560,6 @@ ErrorVec TypeChecker::canUnify(TypePackId left, TypePackId right, const Location return canUnify_(left, right, location); } -ErrorVec TypeChecker::canUnify(const std::vector>& seen, TypeId superTy, TypeId subTy, const Location& location) -{ - Unifier state = mkUnifier(seen, location); - return state.canUnify(superTy, subTy); -} - template ErrorVec TypeChecker::canUnify_(Id superTy, Id subTy, const Location& location) { @@ -1152,61 +1144,20 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatTypeAlias& typealias Location location = scope->typeAliasLocations[name]; reportError(TypeError{typealias.location, DuplicateTypeDefinition{name, location}}); - if (FFlag::LuauTypeAliasPacks) - bindingsMap[name] = TypeFun{binding->typeParams, binding->typePackParams, errorRecoveryType(anyType)}; - else - bindingsMap[name] = TypeFun{binding->typeParams, errorRecoveryType(anyType)}; + bindingsMap[name] = TypeFun{binding->typeParams, binding->typePackParams, errorRecoveryType(anyType)}; } else { ScopePtr aliasScope = FFlag::LuauQuantifyInPlace2 ? childScope(scope, typealias.location, subLevel) : childScope(scope, typealias.location); - if (FFlag::LuauTypeAliasPacks) - { - auto [generics, genericPacks] = createGenericTypes(aliasScope, scope->level, typealias, typealias.generics, typealias.genericPacks); + auto [generics, genericPacks] = createGenericTypes(aliasScope, scope->level, typealias, typealias.generics, typealias.genericPacks); - TypeId ty = freshType(aliasScope); - FreeTypeVar* ftv = getMutable(ty); - LUAU_ASSERT(ftv); - ftv->forwardedTypeAlias = true; - bindingsMap[name] = {std::move(generics), std::move(genericPacks), ty}; - } - else - { - std::vector generics; - for (AstName generic : typealias.generics) - { - Name n = generic.value; - - // These generics are the only thing that will ever be added to aliasScope, so we can be certain that - // a collision can only occur when two generic typevars have the same name. - if (aliasScope->privateTypeBindings.end() != aliasScope->privateTypeBindings.find(n)) - { - // TODO(jhuelsman): report the exact span of the generic type parameter whose name is a duplicate. - reportError(TypeError{typealias.location, DuplicateGenericParameter{n}}); - } - - TypeId g; - if (FFlag::LuauRecursiveTypeParameterRestriction) - { - TypeId& cached = scope->typeAliasTypeParameters[n]; - if (!cached) - cached = addType(GenericTypeVar{aliasScope->level, n}); - g = cached; - } - else - g = addType(GenericTypeVar{aliasScope->level, n}); - generics.push_back(g); - aliasScope->privateTypeBindings[n] = TypeFun{{}, g}; - } - - TypeId ty = freshType(aliasScope); - FreeTypeVar* ftv = getMutable(ty); - LUAU_ASSERT(ftv); - ftv->forwardedTypeAlias = true; - bindingsMap[name] = {std::move(generics), ty}; - } + TypeId ty = freshType(aliasScope); + FreeTypeVar* ftv = getMutable(ty); + LUAU_ASSERT(ftv); + ftv->forwardedTypeAlias = true; + bindingsMap[name] = {std::move(generics), std::move(genericPacks), ty}; } } else @@ -1223,14 +1174,11 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatTypeAlias& typealias aliasScope->privateTypeBindings[generic->name] = TypeFun{{}, ty}; } - if (FFlag::LuauTypeAliasPacks) + for (TypePackId tp : binding->typePackParams) { - for (TypePackId tp : binding->typePackParams) - { - auto generic = get(tp); - LUAU_ASSERT(generic); - aliasScope->privateTypePackBindings[generic->name] = tp; - } + auto generic = get(tp); + LUAU_ASSERT(generic); + aliasScope->privateTypePackBindings[generic->name] = tp; } TypeId ty = resolveType(aliasScope, *typealias.type); @@ -1241,19 +1189,16 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatTypeAlias& typealias { // Copy can be skipped if this is an identical alias if (ttv->name != name || ttv->instantiatedTypeParams != binding->typeParams || - (FFlag::LuauTypeAliasPacks && ttv->instantiatedTypePackParams != binding->typePackParams)) + ttv->instantiatedTypePackParams != binding->typePackParams) { // This is a shallow clone, original recursive links to self are not updated TableTypeVar clone = TableTypeVar{ttv->props, ttv->indexer, ttv->level, ttv->state}; clone.methodDefinitionLocations = ttv->methodDefinitionLocations; clone.definitionModuleName = ttv->definitionModuleName; - clone.name = name; clone.instantiatedTypeParams = binding->typeParams; - - if (FFlag::LuauTypeAliasPacks) - clone.instantiatedTypePackParams = binding->typePackParams; + clone.instantiatedTypePackParams = binding->typePackParams; ty = addType(std::move(clone)); } @@ -1262,9 +1207,7 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatTypeAlias& typealias { ttv->name = name; ttv->instantiatedTypeParams = binding->typeParams; - - if (FFlag::LuauTypeAliasPacks) - ttv->instantiatedTypePackParams = binding->typePackParams; + ttv->instantiatedTypePackParams = binding->typePackParams; } } else if (auto mtv = getMutable(follow(ty))) @@ -1289,7 +1232,7 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatDeclareClass& declar } // We don't have generic classes, so this assertion _should_ never be hit. - LUAU_ASSERT(lookupType->typeParams.size() == 0 && (!FFlag::LuauTypeAliasPacks || lookupType->typePackParams.size() == 0)); + LUAU_ASSERT(lookupType->typeParams.size() == 0 && lookupType->typePackParams.size() == 0); superTy = lookupType->type; if (!get(follow(*superTy))) @@ -1851,6 +1794,24 @@ TypeId TypeChecker::checkExprTable( if (isNonstrictMode() && !getTableType(exprType) && !get(exprType)) exprType = anyType; + if (FFlag::LuauPropertiesGetExpectedType && expectedTable) + { + auto it = expectedTable->props.find(key->value.data); + if (it != expectedTable->props.end()) + { + Property expectedProp = it->second; + ErrorVec errors = tryUnify(expectedProp.type, exprType, k->location); + if (errors.empty()) + exprType = expectedProp.type; + } + else if (expectedTable->indexer && isString(expectedTable->indexer->indexType)) + { + ErrorVec errors = tryUnify(expectedTable->indexer->indexResultType, exprType, k->location); + if (errors.empty()) + exprType = expectedTable->indexer->indexResultType; + } + } + props[key->value.data] = {exprType, /* deprecated */ false, {}, k->location}; } else @@ -3744,17 +3705,29 @@ ExprResult TypeChecker::checkExprList(const ScopePtr& scope, const L for (size_t i = 0; i < exprs.size; ++i) { AstExpr* expr = exprs.data[i]; + std::optional expectedType = i < expectedTypes.size() ? expectedTypes[i] : std::nullopt; if (i == lastIndex && (expr->is() || expr->is())) { auto [typePack, exprPredicates] = checkExprPack(scope, *expr); insert(exprPredicates); + if (FFlag::LuauTailArgumentTypeInfo) + { + if (std::optional firstTy = first(typePack)) + { + if (!currentModule->astTypes.find(expr)) + currentModule->astTypes[expr] = follow(*firstTy); + } + + if (expectedType) + currentModule->astExpectedTypes[expr] = *expectedType; + } + tp->tail = typePack; } else { - std::optional expectedType = i < expectedTypes.size() ? expectedTypes[i] : std::nullopt; auto [type, exprPredicates] = checkExpr(scope, *expr, expectedType); insert(exprPredicates); @@ -3797,7 +3770,7 @@ TypeId TypeChecker::checkRequire(const ScopePtr& scope, const ModuleInfo& module LUAU_TIMETRACE_SCOPE("TypeChecker::checkRequire", "TypeChecker"); LUAU_TIMETRACE_ARGUMENT("moduleInfo", moduleInfo.name.c_str()); - if (FFlag::LuauNewRequireTrace2 && moduleInfo.name.empty()) + if (moduleInfo.name.empty()) { if (FFlag::LuauStrictRequire && currentModule->mode == Mode::Strict) { @@ -3814,7 +3787,7 @@ TypeId TypeChecker::checkRequire(const ScopePtr& scope, const ModuleInfo& module // There are two reasons why we might fail to find the module: // either the file does not exist or there's a cycle. If there's a cycle // we will already have reported the error. - if (!resolver->moduleExists(moduleInfo.name) && (FFlag::LuauTraceRequireLookupChild ? !moduleInfo.optional : true)) + if (!resolver->moduleExists(moduleInfo.name) && !moduleInfo.optional) { std::string reportedModulePath = resolver->getHumanReadableModuleName(moduleInfo.name); reportError(TypeError{location, UnknownRequire{reportedModulePath}}); @@ -3830,7 +3803,12 @@ TypeId TypeChecker::checkRequire(const ScopePtr& scope, const ModuleInfo& module return errorRecoveryType(scope); } - std::optional moduleType = first(module->getModuleScope()->returnType); + TypePackId modulePack = module->getModuleScope()->returnType; + + if (FFlag::LuauModuleRequireErrorPack && get(modulePack)) + return errorRecoveryType(scope); + + std::optional moduleType = first(modulePack); if (!moduleType) { std::string humanReadableName = resolver->getHumanReadableModuleName(moduleInfo.name); @@ -3840,7 +3818,8 @@ TypeId TypeChecker::checkRequire(const ScopePtr& scope, const ModuleInfo& module SeenTypes seenTypes; SeenTypePacks seenTypePacks; - return clone(*moduleType, currentModule->internalTypes, seenTypes, seenTypePacks); + CloneState cloneState; + return clone(*moduleType, currentModule->internalTypes, seenTypes, seenTypePacks, cloneState); } void TypeChecker::tablify(TypeId type) @@ -4326,11 +4305,6 @@ Unifier TypeChecker::mkUnifier(const Location& location) return Unifier{¤tModule->internalTypes, currentModule->mode, globalScope, location, Variance::Covariant, unifierState}; } -Unifier TypeChecker::mkUnifier(const std::vector>& seen, const Location& location) -{ - return Unifier{¤tModule->internalTypes, currentModule->mode, globalScope, seen, location, Variance::Covariant, unifierState}; -} - TypeId TypeChecker::freshType(const ScopePtr& scope) { return freshType(scope->level); @@ -4477,117 +4451,82 @@ TypeId TypeChecker::resolveType(const ScopePtr& scope, const AstType& annotation return errorRecoveryType(scope); } - if (lit->parameters.size == 0 && tf->typeParams.empty() && (!FFlag::LuauTypeAliasPacks || tf->typePackParams.empty())) - { + if (lit->parameters.size == 0 && tf->typeParams.empty() && tf->typePackParams.empty()) return tf->type; - } - else if (!FFlag::LuauTypeAliasPacks && lit->parameters.size != tf->typeParams.size()) + + if (!lit->hasParameterList && !tf->typePackParams.empty()) { - reportError(TypeError{annotation.location, IncorrectGenericParameterCount{lit->name.value, *tf, lit->parameters.size, 0}}); + reportError(TypeError{annotation.location, GenericError{"Type parameter list is required"}}); if (!FFlag::LuauErrorRecoveryType) return errorRecoveryType(scope); } - if (FFlag::LuauTypeAliasPacks) + std::vector typeParams; + std::vector extraTypes; + std::vector typePackParams; + + for (size_t i = 0; i < lit->parameters.size; ++i) { - if (!lit->hasParameterList && !tf->typePackParams.empty()) + if (AstType* type = lit->parameters.data[i].type) { - reportError(TypeError{annotation.location, GenericError{"Type parameter list is required"}}); - if (!FFlag::LuauErrorRecoveryType) - return errorRecoveryType(scope); - } + TypeId ty = resolveType(scope, *type); - std::vector typeParams; - std::vector extraTypes; - std::vector typePackParams; - - for (size_t i = 0; i < lit->parameters.size; ++i) - { - if (AstType* type = lit->parameters.data[i].type) - { - TypeId ty = resolveType(scope, *type); - - if (typeParams.size() < tf->typeParams.size() || tf->typePackParams.empty()) - typeParams.push_back(ty); - else if (typePackParams.empty()) - extraTypes.push_back(ty); - else - reportError(TypeError{annotation.location, GenericError{"Type parameters must come before type pack parameters"}}); - } - else if (AstTypePack* typePack = lit->parameters.data[i].typePack) - { - TypePackId tp = resolveTypePack(scope, *typePack); - - // If we have collected an implicit type pack, materialize it - if (typePackParams.empty() && !extraTypes.empty()) - typePackParams.push_back(addTypePack(extraTypes)); - - // If we need more regular types, we can use single element type packs to fill those in - if (typeParams.size() < tf->typeParams.size() && size(tp) == 1 && finite(tp) && first(tp)) - typeParams.push_back(*first(tp)); - else - typePackParams.push_back(tp); - } - } - - // If we still haven't meterialized an implicit type pack, do it now - if (typePackParams.empty() && !extraTypes.empty()) - typePackParams.push_back(addTypePack(extraTypes)); - - // If we didn't combine regular types into a type pack and we're still one type pack short, provide an empty type pack - if (extraTypes.empty() && typePackParams.size() + 1 == tf->typePackParams.size()) - typePackParams.push_back(addTypePack({})); - - if (typeParams.size() != tf->typeParams.size() || typePackParams.size() != tf->typePackParams.size()) - { - reportError( - TypeError{annotation.location, IncorrectGenericParameterCount{lit->name.value, *tf, typeParams.size(), typePackParams.size()}}); - - if (FFlag::LuauErrorRecoveryType) - { - // Pad the types out with error recovery types - while (typeParams.size() < tf->typeParams.size()) - typeParams.push_back(errorRecoveryType(scope)); - while (typePackParams.size() < tf->typePackParams.size()) - typePackParams.push_back(errorRecoveryTypePack(scope)); - } + if (typeParams.size() < tf->typeParams.size() || tf->typePackParams.empty()) + typeParams.push_back(ty); + else if (typePackParams.empty()) + extraTypes.push_back(ty); else - return errorRecoveryType(scope); + reportError(TypeError{annotation.location, GenericError{"Type parameters must come before type pack parameters"}}); } - - if (FFlag::LuauRecursiveTypeParameterRestriction && typeParams == tf->typeParams && typePackParams == tf->typePackParams) + else if (AstTypePack* typePack = lit->parameters.data[i].typePack) { - // If the generic parameters and the type arguments are the same, we are about to - // perform an identity substitution, which we can just short-circuit. - return tf->type; + TypePackId tp = resolveTypePack(scope, *typePack); + + // If we have collected an implicit type pack, materialize it + if (typePackParams.empty() && !extraTypes.empty()) + typePackParams.push_back(addTypePack(extraTypes)); + + // If we need more regular types, we can use single element type packs to fill those in + if (typeParams.size() < tf->typeParams.size() && size(tp) == 1 && finite(tp) && first(tp)) + typeParams.push_back(*first(tp)); + else + typePackParams.push_back(tp); } - - return instantiateTypeFun(scope, *tf, typeParams, typePackParams, annotation.location); } - else - { - std::vector typeParams; - for (const auto& param : lit->parameters) - typeParams.push_back(resolveType(scope, *param.type)); + // If we still haven't meterialized an implicit type pack, do it now + if (typePackParams.empty() && !extraTypes.empty()) + typePackParams.push_back(addTypePack(extraTypes)); + + // If we didn't combine regular types into a type pack and we're still one type pack short, provide an empty type pack + if (extraTypes.empty() && typePackParams.size() + 1 == tf->typePackParams.size()) + typePackParams.push_back(addTypePack({})); + + if (typeParams.size() != tf->typeParams.size() || typePackParams.size() != tf->typePackParams.size()) + { + reportError( + TypeError{annotation.location, IncorrectGenericParameterCount{lit->name.value, *tf, typeParams.size(), typePackParams.size()}}); if (FFlag::LuauErrorRecoveryType) { - // If there aren't enough type parameters, pad them out with error recovery types - // (we've already reported the error) - while (typeParams.size() < lit->parameters.size) + // Pad the types out with error recovery types + while (typeParams.size() < tf->typeParams.size()) typeParams.push_back(errorRecoveryType(scope)); + while (typePackParams.size() < tf->typePackParams.size()) + typePackParams.push_back(errorRecoveryTypePack(scope)); } - - if (FFlag::LuauRecursiveTypeParameterRestriction && typeParams == tf->typeParams) - { - // If the generic parameters and the type arguments are the same, we are about to - // perform an identity substitution, which we can just short-circuit. - return tf->type; - } - - return instantiateTypeFun(scope, *tf, typeParams, {}, annotation.location); + else + return errorRecoveryType(scope); } + + if (FFlag::LuauRecursiveTypeParameterRestriction && typeParams == tf->typeParams && typePackParams == tf->typePackParams) + { + // If the generic parameters and the type arguments are the same, we are about to + // perform an identity substitution, which we can just short-circuit. + return tf->type; + } + + return instantiateTypeFun(scope, *tf, typeParams, typePackParams, annotation.location); } else if (const auto& table = annotation.as()) { @@ -4757,7 +4696,7 @@ bool ApplyTypeFunction::isDirty(TypePackId tp) bool ApplyTypeFunction::ignoreChildren(TypeId ty) { - if (FFlag::LuauSubstitutionDontReplaceIgnoredTypes && get(ty)) + if (get(ty)) return true; else return false; @@ -4765,7 +4704,7 @@ bool ApplyTypeFunction::ignoreChildren(TypeId ty) bool ApplyTypeFunction::ignoreChildren(TypePackId tp) { - if (FFlag::LuauSubstitutionDontReplaceIgnoredTypes && get(tp)) + if (get(tp)) return true; else return false; @@ -4788,36 +4727,26 @@ TypePackId ApplyTypeFunction::clean(TypePackId tp) // Really this should just replace the arguments, // but for bug-compatibility with existing code, we replace // all generics by free type variables. - if (FFlag::LuauTypeAliasPacks) - { - TypePackId& arg = typePackArguments[tp]; - if (arg) - return arg; - else - return addTypePack(FreeTypePack{level}); - } + TypePackId& arg = typePackArguments[tp]; + if (arg) + return arg; else - { return addTypePack(FreeTypePack{level}); - } } TypeId TypeChecker::instantiateTypeFun(const ScopePtr& scope, const TypeFun& tf, const std::vector& typeParams, const std::vector& typePackParams, const Location& location) { - if (tf.typeParams.empty() && (!FFlag::LuauTypeAliasPacks || tf.typePackParams.empty())) + if (tf.typeParams.empty() && tf.typePackParams.empty()) return tf.type; applyTypeFunction.typeArguments.clear(); for (size_t i = 0; i < tf.typeParams.size(); ++i) applyTypeFunction.typeArguments[tf.typeParams[i]] = typeParams[i]; - if (FFlag::LuauTypeAliasPacks) - { - applyTypeFunction.typePackArguments.clear(); - for (size_t i = 0; i < tf.typePackParams.size(); ++i) - applyTypeFunction.typePackArguments[tf.typePackParams[i]] = typePackParams[i]; - } + applyTypeFunction.typePackArguments.clear(); + for (size_t i = 0; i < tf.typePackParams.size(); ++i) + applyTypeFunction.typePackArguments[tf.typePackParams[i]] = typePackParams[i]; applyTypeFunction.currentModule = currentModule; applyTypeFunction.level = scope->level; @@ -4866,9 +4795,7 @@ TypeId TypeChecker::instantiateTypeFun(const ScopePtr& scope, const TypeFun& tf, if (ttv) { ttv->instantiatedTypeParams = typeParams; - - if (FFlag::LuauTypeAliasPacks) - ttv->instantiatedTypePackParams = typePackParams; + ttv->instantiatedTypePackParams = typePackParams; } } else @@ -4884,9 +4811,7 @@ TypeId TypeChecker::instantiateTypeFun(const ScopePtr& scope, const TypeFun& tf, } ttv->instantiatedTypeParams = typeParams; - - if (FFlag::LuauTypeAliasPacks) - ttv->instantiatedTypePackParams = typePackParams; + ttv->instantiatedTypePackParams = typePackParams; } } @@ -4914,7 +4839,7 @@ std::pair, std::vector> TypeChecker::createGener } TypeId g; - if (FFlag::LuauRecursiveTypeParameterRestriction && FFlag::LuauTypeAliasPacks) + if (FFlag::LuauRecursiveTypeParameterRestriction) { TypeId& cached = scope->parent->typeAliasTypeParameters[n]; if (!cached) @@ -4944,7 +4869,7 @@ std::pair, std::vector> TypeChecker::createGener } TypePackId g; - if (FFlag::LuauRecursiveTypeParameterRestriction && FFlag::LuauTypeAliasPacks) + if (FFlag::LuauRecursiveTypeParameterRestriction) { TypePackId& cached = scope->parent->typeAliasTypePackParameters[n]; if (!cached) @@ -5245,7 +5170,7 @@ void TypeChecker::resolve(const TypeGuardPredicate& typeguardP, ErrorVec& errVec return fail(UnknownSymbol{typeguardP.kind, UnknownSymbol::Type}); auto typeFun = globalScope->lookupType(typeguardP.kind); - if (!typeFun || !typeFun->typeParams.empty() || (FFlag::LuauTypeAliasPacks && !typeFun->typePackParams.empty())) + if (!typeFun || !typeFun->typeParams.empty() || !typeFun->typePackParams.empty()) return fail(UnknownSymbol{typeguardP.kind, UnknownSymbol::Type}); TypeId type = follow(typeFun->type); diff --git a/Analysis/src/TypeVar.cpp b/Analysis/src/TypeVar.cpp index 924bf082..62715af5 100644 --- a/Analysis/src/TypeVar.cpp +++ b/Analysis/src/TypeVar.cpp @@ -19,7 +19,6 @@ LUAU_FASTINTVARIABLE(LuauTypeMaximumStringifierLength, 500) LUAU_FASTINTVARIABLE(LuauTableTypeMaximumStringifierLength, 0) -LUAU_FASTFLAG(LuauTypeAliasPacks) LUAU_FASTFLAGVARIABLE(LuauRefactorTagging, false) LUAU_FASTFLAG(LuauErrorRecoveryType) @@ -739,369 +738,6 @@ void persist(TypePackId tp) } } -namespace -{ - -struct StateDot -{ - StateDot(ToDotOptions opts) - : opts(opts) - { - } - - ToDotOptions opts; - - std::unordered_set seenTy; - std::unordered_set seenTp; - std::unordered_map tyToIndex; - std::unordered_map tpToIndex; - int nextIndex = 1; - std::string result; - - bool canDuplicatePrimitive(TypeId ty); - - void visitChildren(TypeId ty, int index); - void visitChildren(TypePackId ty, int index); - - void visitChild(TypeId ty, int parentIndex, const char* linkName = nullptr); - void visitChild(TypePackId tp, int parentIndex, const char* linkName = nullptr); - - void startNode(int index); - void finishNode(); - - void startNodeLabel(); - void finishNodeLabel(TypeId ty); - void finishNodeLabel(TypePackId tp); -}; - -bool StateDot::canDuplicatePrimitive(TypeId ty) -{ - if (get(ty)) - return false; - - return get(ty) || get(ty); -} - -void StateDot::visitChild(TypeId ty, int parentIndex, const char* linkName) -{ - if (!tyToIndex.count(ty) || (opts.duplicatePrimitives && canDuplicatePrimitive(ty))) - tyToIndex[ty] = nextIndex++; - - int index = tyToIndex[ty]; - - if (parentIndex != 0) - { - if (linkName) - formatAppend(result, "n%d -> n%d [label=\"%s\"];\n", parentIndex, index, linkName); - else - formatAppend(result, "n%d -> n%d;\n", parentIndex, index); - } - - if (opts.duplicatePrimitives && canDuplicatePrimitive(ty)) - { - if (get(ty)) - formatAppend(result, "n%d [label=\"%s\"];\n", index, toStringDetailed(ty, {}).name.c_str()); - else if (get(ty)) - formatAppend(result, "n%d [label=\"any\"];\n", index); - } - else - { - visitChildren(ty, index); - } -} - -void StateDot::visitChild(TypePackId tp, int parentIndex, const char* linkName) -{ - if (!tpToIndex.count(tp)) - tpToIndex[tp] = nextIndex++; - - if (linkName) - formatAppend(result, "n%d -> n%d [label=\"%s\"];\n", parentIndex, tpToIndex[tp], linkName); - else - formatAppend(result, "n%d -> n%d;\n", parentIndex, tpToIndex[tp]); - - visitChildren(tp, tpToIndex[tp]); -} - -void StateDot::startNode(int index) -{ - formatAppend(result, "n%d [", index); -} - -void StateDot::finishNode() -{ - formatAppend(result, "];\n"); -} - -void StateDot::startNodeLabel() -{ - formatAppend(result, "label=\""); -} - -void StateDot::finishNodeLabel(TypeId ty) -{ - if (opts.showPointers) - formatAppend(result, "\n0x%p", ty); - // additional common attributes can be added here as well - result += "\""; -} - -void StateDot::finishNodeLabel(TypePackId tp) -{ - if (opts.showPointers) - formatAppend(result, "\n0x%p", tp); - // additional common attributes can be added here as well - result += "\""; -} - -void StateDot::visitChildren(TypeId ty, int index) -{ - if (seenTy.count(ty)) - return; - seenTy.insert(ty); - - startNode(index); - startNodeLabel(); - - if (const BoundTypeVar* btv = get(ty)) - { - formatAppend(result, "BoundTypeVar %d", index); - finishNodeLabel(ty); - finishNode(); - - visitChild(btv->boundTo, index); - } - else if (const FunctionTypeVar* ftv = get(ty)) - { - formatAppend(result, "FunctionTypeVar %d", index); - finishNodeLabel(ty); - finishNode(); - - visitChild(ftv->argTypes, index, "arg"); - visitChild(ftv->retType, index, "ret"); - } - else if (const TableTypeVar* ttv = get(ty)) - { - if (ttv->name) - formatAppend(result, "TableTypeVar %s", ttv->name->c_str()); - else if (ttv->syntheticName) - formatAppend(result, "TableTypeVar %s", ttv->syntheticName->c_str()); - else - formatAppend(result, "TableTypeVar %d", index); - finishNodeLabel(ty); - finishNode(); - - if (ttv->boundTo) - return visitChild(*ttv->boundTo, index, "boundTo"); - - for (const auto& [name, prop] : ttv->props) - visitChild(prop.type, index, name.c_str()); - if (ttv->indexer) - { - visitChild(ttv->indexer->indexType, index, "[index]"); - visitChild(ttv->indexer->indexResultType, index, "[value]"); - } - for (TypeId itp : ttv->instantiatedTypeParams) - visitChild(itp, index, "typeParam"); - - if (FFlag::LuauTypeAliasPacks) - { - for (TypePackId itp : ttv->instantiatedTypePackParams) - visitChild(itp, index, "typePackParam"); - } - } - else if (const MetatableTypeVar* mtv = get(ty)) - { - formatAppend(result, "MetatableTypeVar %d", index); - finishNodeLabel(ty); - finishNode(); - - visitChild(mtv->table, index, "table"); - visitChild(mtv->metatable, index, "metatable"); - } - else if (const UnionTypeVar* utv = get(ty)) - { - formatAppend(result, "UnionTypeVar %d", index); - finishNodeLabel(ty); - finishNode(); - - for (TypeId opt : utv->options) - visitChild(opt, index); - } - else if (const IntersectionTypeVar* itv = get(ty)) - { - formatAppend(result, "IntersectionTypeVar %d", index); - finishNodeLabel(ty); - finishNode(); - - for (TypeId part : itv->parts) - visitChild(part, index); - } - else if (const GenericTypeVar* gtv = get(ty)) - { - if (gtv->explicitName) - formatAppend(result, "GenericTypeVar %s", gtv->name.c_str()); - else - formatAppend(result, "GenericTypeVar %d", index); - finishNodeLabel(ty); - finishNode(); - } - else if (const FreeTypeVar* ftv = get(ty)) - { - formatAppend(result, "FreeTypeVar %d", ftv->index); - finishNodeLabel(ty); - finishNode(); - } - else if (get(ty)) - { - formatAppend(result, "AnyTypeVar %d", index); - finishNodeLabel(ty); - finishNode(); - } - else if (get(ty)) - { - formatAppend(result, "PrimitiveTypeVar %s", toStringDetailed(ty, {}).name.c_str()); - finishNodeLabel(ty); - finishNode(); - } - else if (get(ty)) - { - formatAppend(result, "ErrorTypeVar %d", index); - finishNodeLabel(ty); - finishNode(); - } - else if (const ClassTypeVar* ctv = get(ty)) - { - formatAppend(result, "ClassTypeVar %s", ctv->name.c_str()); - finishNodeLabel(ty); - finishNode(); - - for (const auto& [name, prop] : ctv->props) - visitChild(prop.type, index, name.c_str()); - - if (ctv->parent) - visitChild(*ctv->parent, index, "[parent]"); - - if (ctv->metatable) - visitChild(*ctv->metatable, index, "[metatable]"); - } - else - { - LUAU_ASSERT(!"unknown type kind"); - finishNodeLabel(ty); - finishNode(); - } -} - -void StateDot::visitChildren(TypePackId tp, int index) -{ - if (seenTp.count(tp)) - return; - seenTp.insert(tp); - - startNode(index); - startNodeLabel(); - - if (const BoundTypePack* btp = get(tp)) - { - formatAppend(result, "BoundTypePack %d", index); - finishNodeLabel(tp); - finishNode(); - - visitChild(btp->boundTo, index); - } - else if (const TypePack* tpp = get(tp)) - { - formatAppend(result, "TypePack %d", index); - finishNodeLabel(tp); - finishNode(); - - for (TypeId tv : tpp->head) - visitChild(tv, index); - if (tpp->tail) - visitChild(*tpp->tail, index, "tail"); - } - else if (const VariadicTypePack* vtp = get(tp)) - { - formatAppend(result, "VariadicTypePack %d", index); - finishNodeLabel(tp); - finishNode(); - - visitChild(vtp->ty, index); - } - else if (const FreeTypePack* ftp = get(tp)) - { - formatAppend(result, "FreeTypePack %d", ftp->index); - finishNodeLabel(tp); - finishNode(); - } - else if (const GenericTypePack* gtp = get(tp)) - { - if (gtp->explicitName) - formatAppend(result, "GenericTypePack %s", gtp->name.c_str()); - else - formatAppend(result, "GenericTypePack %d", gtp->index); - finishNodeLabel(tp); - finishNode(); - } - else if (get(tp)) - { - formatAppend(result, "ErrorTypePack %d", index); - finishNodeLabel(tp); - finishNode(); - } - else - { - LUAU_ASSERT(!"unknown type pack kind"); - finishNodeLabel(tp); - finishNode(); - } -} - -} // namespace - -std::string toDot(TypeId ty, const ToDotOptions& opts) -{ - StateDot state{opts}; - - state.result = "digraph graphname {\n"; - state.visitChild(ty, 0); - state.result += "}"; - - return state.result; -} - -std::string toDot(TypePackId tp, const ToDotOptions& opts) -{ - StateDot state{opts}; - - state.result = "digraph graphname {\n"; - state.visitChild(tp, 0); - state.result += "}"; - - return state.result; -} - -std::string toDot(TypeId ty) -{ - return toDot(ty, {}); -} - -std::string toDot(TypePackId tp) -{ - return toDot(tp, {}); -} - -void dumpDot(TypeId ty) -{ - printf("%s\n", toDot(ty).c_str()); -} - -void dumpDot(TypePackId tp) -{ - printf("%s\n", toDot(tp).c_str()); -} - const TypeLevel* getLevel(TypeId ty) { ty = follow(ty); diff --git a/Analysis/src/Unifier.cpp b/Analysis/src/Unifier.cpp index e1a52be4..d0b18837 100644 --- a/Analysis/src/Unifier.cpp +++ b/Analysis/src/Unifier.cpp @@ -18,9 +18,6 @@ LUAU_FASTFLAGVARIABLE(LuauTableSubtypingVariance, false); LUAU_FASTFLAGVARIABLE(LuauUnionHeuristic, false) LUAU_FASTFLAGVARIABLE(LuauTableUnificationEarlyTest, false) LUAU_FASTFLAGVARIABLE(LuauOccursCheckOkWithRecursiveFunctions, false) -LUAU_FASTFLAGVARIABLE(LuauTypecheckOpts, false) -LUAU_FASTFLAG(LuauShareTxnSeen); -LUAU_FASTFLAGVARIABLE(LuauCacheUnifyTableResults, false) LUAU_FASTFLAGVARIABLE(LuauExtendedTypeMismatchError, false) LUAU_FASTFLAG(LuauSingletonTypes) LUAU_FASTFLAGVARIABLE(LuauExtendedClassMismatchError, false) @@ -136,38 +133,19 @@ Unifier::Unifier(TypeArena* types, Mode mode, ScopePtr globalScope, const Locati , globalScope(std::move(globalScope)) , location(location) , variance(variance) - , counters(&countersData) - , counters_DEPRECATED(std::make_shared()) - , sharedState(sharedState) -{ - LUAU_ASSERT(sharedState.iceHandler); -} - -Unifier::Unifier(TypeArena* types, Mode mode, ScopePtr globalScope, const std::vector>& ownedSeen, const Location& location, - Variance variance, UnifierSharedState& sharedState, const std::shared_ptr& counters_DEPRECATED, UnifierCounters* counters) - : types(types) - , mode(mode) - , globalScope(std::move(globalScope)) - , log(ownedSeen) - , location(location) - , variance(variance) - , counters(counters ? counters : &countersData) - , counters_DEPRECATED(counters_DEPRECATED ? counters_DEPRECATED : std::make_shared()) , sharedState(sharedState) { LUAU_ASSERT(sharedState.iceHandler); } Unifier::Unifier(TypeArena* types, Mode mode, ScopePtr globalScope, std::vector>* sharedSeen, const Location& location, - Variance variance, UnifierSharedState& sharedState, const std::shared_ptr& counters_DEPRECATED, UnifierCounters* counters) + Variance variance, UnifierSharedState& sharedState) : types(types) , mode(mode) , globalScope(std::move(globalScope)) , log(sharedSeen) , location(location) , variance(variance) - , counters(counters ? counters : &countersData) - , counters_DEPRECATED(counters_DEPRECATED ? counters_DEPRECATED : std::make_shared()) , sharedState(sharedState) { LUAU_ASSERT(sharedState.iceHandler); @@ -175,26 +153,18 @@ Unifier::Unifier(TypeArena* types, Mode mode, ScopePtr globalScope, std::vector< void Unifier::tryUnify(TypeId superTy, TypeId subTy, bool isFunctionCall, bool isIntersection) { - if (FFlag::LuauTypecheckOpts) - counters->iterationCount = 0; - else - counters_DEPRECATED->iterationCount = 0; + sharedState.counters.iterationCount = 0; tryUnify_(superTy, subTy, isFunctionCall, isIntersection); } void Unifier::tryUnify_(TypeId superTy, TypeId subTy, bool isFunctionCall, bool isIntersection) { - RecursionLimiter _ra( - FFlag::LuauTypecheckOpts ? &counters->recursionCount : &counters_DEPRECATED->recursionCount, FInt::LuauTypeInferRecursionLimit); + RecursionLimiter _ra(&sharedState.counters.recursionCount, FInt::LuauTypeInferRecursionLimit); - if (FFlag::LuauTypecheckOpts) - ++counters->iterationCount; - else - ++counters_DEPRECATED->iterationCount; + ++sharedState.counters.iterationCount; - if (FInt::LuauTypeInferIterationLimit > 0 && - FInt::LuauTypeInferIterationLimit < (FFlag::LuauTypecheckOpts ? counters->iterationCount : counters_DEPRECATED->iterationCount)) + if (FInt::LuauTypeInferIterationLimit > 0 && FInt::LuauTypeInferIterationLimit < sharedState.counters.iterationCount) { errors.push_back(TypeError{location, UnificationTooComplex{}}); return; @@ -302,7 +272,7 @@ void Unifier::tryUnify_(TypeId superTy, TypeId subTy, bool isFunctionCall, bool if (get(subTy) || get(subTy)) return tryUnifyWithAny(subTy, superTy); - bool cacheEnabled = FFlag::LuauCacheUnifyTableResults && !isFunctionCall && !isIntersection; + bool cacheEnabled = !isFunctionCall && !isIntersection; auto& cache = sharedState.cachedUnify; // What if the types are immutable and we proved their relation before @@ -563,8 +533,6 @@ void Unifier::tryUnify_(TypeId superTy, TypeId subTy, bool isFunctionCall, bool void Unifier::cacheResult(TypeId superTy, TypeId subTy) { - LUAU_ASSERT(FFlag::LuauCacheUnifyTableResults); - bool* superTyInfo = sharedState.skipCacheForType.find(superTy); if (superTyInfo && *superTyInfo) @@ -686,10 +654,7 @@ ErrorVec Unifier::canUnify(TypePackId superTy, TypePackId subTy, bool isFunction void Unifier::tryUnify(TypePackId superTp, TypePackId subTp, bool isFunctionCall) { - if (FFlag::LuauTypecheckOpts) - counters->iterationCount = 0; - else - counters_DEPRECATED->iterationCount = 0; + sharedState.counters.iterationCount = 0; tryUnify_(superTp, subTp, isFunctionCall); } @@ -700,16 +665,11 @@ void Unifier::tryUnify(TypePackId superTp, TypePackId subTp, bool isFunctionCall */ void Unifier::tryUnify_(TypePackId superTp, TypePackId subTp, bool isFunctionCall) { - RecursionLimiter _ra( - FFlag::LuauTypecheckOpts ? &counters->recursionCount : &counters_DEPRECATED->recursionCount, FInt::LuauTypeInferRecursionLimit); + RecursionLimiter _ra(&sharedState.counters.recursionCount, FInt::LuauTypeInferRecursionLimit); - if (FFlag::LuauTypecheckOpts) - ++counters->iterationCount; - else - ++counters_DEPRECATED->iterationCount; + ++sharedState.counters.iterationCount; - if (FInt::LuauTypeInferIterationLimit > 0 && - FInt::LuauTypeInferIterationLimit < (FFlag::LuauTypecheckOpts ? counters->iterationCount : counters_DEPRECATED->iterationCount)) + if (FInt::LuauTypeInferIterationLimit > 0 && FInt::LuauTypeInferIterationLimit < sharedState.counters.iterationCount) { errors.push_back(TypeError{location, UnificationTooComplex{}}); return; @@ -1727,39 +1687,8 @@ void Unifier::tryUnify(const TableIndexer& superIndexer, const TableIndexer& sub tryUnify_(superIndexer.indexResultType, subIndexer.indexResultType); } -static void queueTypePack_DEPRECATED( - std::vector& queue, std::unordered_set& seenTypePacks, Unifier& state, TypePackId a, TypePackId anyTypePack) -{ - LUAU_ASSERT(!FFlag::LuauTypecheckOpts); - - while (true) - { - a = follow(a); - - if (seenTypePacks.count(a)) - break; - seenTypePacks.insert(a); - - if (get(a)) - { - state.log(a); - *asMutable(a) = Unifiable::Bound{anyTypePack}; - } - else if (auto tp = get(a)) - { - queue.insert(queue.end(), tp->head.begin(), tp->head.end()); - if (tp->tail) - a = *tp->tail; - else - break; - } - } -} - static void queueTypePack(std::vector& queue, DenseHashSet& seenTypePacks, Unifier& state, TypePackId a, TypePackId anyTypePack) { - LUAU_ASSERT(FFlag::LuauTypecheckOpts); - while (true) { a = follow(a); @@ -1837,66 +1766,9 @@ void Unifier::tryUnifyVariadics(TypePackId superTp, TypePackId subTp, bool rever } } -static void tryUnifyWithAny_DEPRECATED( - std::vector& queue, Unifier& state, std::unordered_set& seenTypePacks, TypeId anyType, TypePackId anyTypePack) -{ - LUAU_ASSERT(!FFlag::LuauTypecheckOpts); - - std::unordered_set seen; - - while (!queue.empty()) - { - TypeId ty = follow(queue.back()); - queue.pop_back(); - if (seen.count(ty)) - continue; - seen.insert(ty); - - if (get(ty)) - { - state.log(ty); - *asMutable(ty) = BoundTypeVar{anyType}; - } - else if (auto fun = get(ty)) - { - queueTypePack_DEPRECATED(queue, seenTypePacks, state, fun->argTypes, anyTypePack); - queueTypePack_DEPRECATED(queue, seenTypePacks, state, fun->retType, anyTypePack); - } - else if (auto table = get(ty)) - { - for (const auto& [_name, prop] : table->props) - queue.push_back(prop.type); - - if (table->indexer) - { - queue.push_back(table->indexer->indexType); - queue.push_back(table->indexer->indexResultType); - } - } - else if (auto mt = get(ty)) - { - queue.push_back(mt->table); - queue.push_back(mt->metatable); - } - else if (get(ty)) - { - // ClassTypeVars never contain free typevars. - } - else if (auto union_ = get(ty)) - queue.insert(queue.end(), union_->options.begin(), union_->options.end()); - else if (auto intersection = get(ty)) - queue.insert(queue.end(), intersection->parts.begin(), intersection->parts.end()); - else - { - } // Primitives, any, errors, and generics are left untouched. - } -} - static void tryUnifyWithAny(std::vector& queue, Unifier& state, DenseHashSet& seen, DenseHashSet& seenTypePacks, TypeId anyType, TypePackId anyTypePack) { - LUAU_ASSERT(FFlag::LuauTypecheckOpts); - while (!queue.empty()) { TypeId ty = follow(queue.back()); @@ -1949,43 +1821,20 @@ void Unifier::tryUnifyWithAny(TypeId any, TypeId ty) { LUAU_ASSERT(get(any) || get(any)); - if (FFlag::LuauTypecheckOpts) - { - // These types are not visited in general loop below - if (get(ty) || get(ty) || get(ty)) - return; - } + // These types are not visited in general loop below + if (get(ty) || get(ty) || get(ty)) + return; const TypePackId anyTypePack = types->addTypePack(TypePackVar{VariadicTypePack{singletonTypes.anyType}}); const TypePackId anyTP = get(any) ? anyTypePack : types->addTypePack(TypePackVar{Unifiable::Error{}}); - if (FFlag::LuauTypecheckOpts) - { - std::vector queue = {ty}; + std::vector queue = {ty}; - if (FFlag::LuauCacheUnifyTableResults) - { - sharedState.tempSeenTy.clear(); - sharedState.tempSeenTp.clear(); + sharedState.tempSeenTy.clear(); + sharedState.tempSeenTp.clear(); - Luau::tryUnifyWithAny(queue, *this, sharedState.tempSeenTy, sharedState.tempSeenTp, singletonTypes.anyType, anyTP); - } - else - { - tempSeenTy_DEPRECATED.clear(); - tempSeenTp_DEPRECATED.clear(); - - Luau::tryUnifyWithAny(queue, *this, tempSeenTy_DEPRECATED, tempSeenTp_DEPRECATED, singletonTypes.anyType, anyTP); - } - } - else - { - std::unordered_set seenTypePacks; - std::vector queue = {ty}; - - Luau::tryUnifyWithAny_DEPRECATED(queue, *this, seenTypePacks, singletonTypes.anyType, anyTP); - } + Luau::tryUnifyWithAny(queue, *this, sharedState.tempSeenTy, sharedState.tempSeenTp, singletonTypes.anyType, anyTP); } void Unifier::tryUnifyWithAny(TypePackId any, TypePackId ty) @@ -1994,38 +1843,14 @@ void Unifier::tryUnifyWithAny(TypePackId any, TypePackId ty) const TypeId anyTy = singletonTypes.errorRecoveryType(); - if (FFlag::LuauTypecheckOpts) - { - std::vector queue; + std::vector queue; - if (FFlag::LuauCacheUnifyTableResults) - { - sharedState.tempSeenTy.clear(); - sharedState.tempSeenTp.clear(); + sharedState.tempSeenTy.clear(); + sharedState.tempSeenTp.clear(); - queueTypePack(queue, sharedState.tempSeenTp, *this, ty, any); + queueTypePack(queue, sharedState.tempSeenTp, *this, ty, any); - Luau::tryUnifyWithAny(queue, *this, sharedState.tempSeenTy, sharedState.tempSeenTp, anyTy, any); - } - else - { - tempSeenTy_DEPRECATED.clear(); - tempSeenTp_DEPRECATED.clear(); - - queueTypePack(queue, tempSeenTp_DEPRECATED, *this, ty, any); - - Luau::tryUnifyWithAny(queue, *this, tempSeenTy_DEPRECATED, tempSeenTp_DEPRECATED, anyTy, any); - } - } - else - { - std::unordered_set seenTypePacks; - std::vector queue; - - queueTypePack_DEPRECATED(queue, seenTypePacks, *this, ty, any); - - Luau::tryUnifyWithAny_DEPRECATED(queue, *this, seenTypePacks, anyTy, any); - } + Luau::tryUnifyWithAny(queue, *this, sharedState.tempSeenTy, sharedState.tempSeenTp, anyTy, any); } std::optional Unifier::findTablePropertyRespectingMeta(TypeId lhsType, Name name) @@ -2035,46 +1860,22 @@ std::optional Unifier::findTablePropertyRespectingMeta(TypeId lhsType, N void Unifier::occursCheck(TypeId needle, TypeId haystack) { - std::unordered_set seen_DEPRECATED; + sharedState.tempSeenTy.clear(); - if (FFlag::LuauCacheUnifyTableResults) - { - if (FFlag::LuauTypecheckOpts) - sharedState.tempSeenTy.clear(); - - return occursCheck(seen_DEPRECATED, sharedState.tempSeenTy, needle, haystack); - } - else - { - if (FFlag::LuauTypecheckOpts) - tempSeenTy_DEPRECATED.clear(); - - return occursCheck(seen_DEPRECATED, tempSeenTy_DEPRECATED, needle, haystack); - } + return occursCheck(sharedState.tempSeenTy, needle, haystack); } -void Unifier::occursCheck(std::unordered_set& seen_DEPRECATED, DenseHashSet& seen, TypeId needle, TypeId haystack) +void Unifier::occursCheck(DenseHashSet& seen, TypeId needle, TypeId haystack) { - RecursionLimiter _ra( - FFlag::LuauTypecheckOpts ? &counters->recursionCount : &counters_DEPRECATED->recursionCount, FInt::LuauTypeInferRecursionLimit); + RecursionLimiter _ra(&sharedState.counters.recursionCount, FInt::LuauTypeInferRecursionLimit); needle = follow(needle); haystack = follow(haystack); - if (FFlag::LuauTypecheckOpts) - { - if (seen.find(haystack)) - return; + if (seen.find(haystack)) + return; - seen.insert(haystack); - } - else - { - if (seen_DEPRECATED.end() != seen_DEPRECATED.find(haystack)) - return; - - seen_DEPRECATED.insert(haystack); - } + seen.insert(haystack); if (get(needle)) return; @@ -2091,7 +1892,7 @@ void Unifier::occursCheck(std::unordered_set& seen_DEPRECATED, DenseHash } auto check = [&](TypeId tv) { - occursCheck(seen_DEPRECATED, seen, needle, tv); + occursCheck(seen, needle, tv); }; if (get(haystack)) @@ -2121,43 +1922,20 @@ void Unifier::occursCheck(std::unordered_set& seen_DEPRECATED, DenseHash void Unifier::occursCheck(TypePackId needle, TypePackId haystack) { - std::unordered_set seen_DEPRECATED; + sharedState.tempSeenTp.clear(); - if (FFlag::LuauCacheUnifyTableResults) - { - if (FFlag::LuauTypecheckOpts) - sharedState.tempSeenTp.clear(); - - return occursCheck(seen_DEPRECATED, sharedState.tempSeenTp, needle, haystack); - } - else - { - if (FFlag::LuauTypecheckOpts) - tempSeenTp_DEPRECATED.clear(); - - return occursCheck(seen_DEPRECATED, tempSeenTp_DEPRECATED, needle, haystack); - } + return occursCheck(sharedState.tempSeenTp, needle, haystack); } -void Unifier::occursCheck(std::unordered_set& seen_DEPRECATED, DenseHashSet& seen, TypePackId needle, TypePackId haystack) +void Unifier::occursCheck(DenseHashSet& seen, TypePackId needle, TypePackId haystack) { needle = follow(needle); haystack = follow(haystack); - if (FFlag::LuauTypecheckOpts) - { - if (seen.find(haystack)) - return; + if (seen.find(haystack)) + return; - seen.insert(haystack); - } - else - { - if (seen_DEPRECATED.end() != seen_DEPRECATED.find(haystack)) - return; - - seen_DEPRECATED.insert(haystack); - } + seen.insert(haystack); if (get(needle)) return; @@ -2165,8 +1943,7 @@ void Unifier::occursCheck(std::unordered_set& seen_DEPRECATED, Dense if (!get(needle)) ice("Expected needle pack to be free"); - RecursionLimiter _ra( - FFlag::LuauTypecheckOpts ? &counters->recursionCount : &counters_DEPRECATED->recursionCount, FInt::LuauTypeInferRecursionLimit); + RecursionLimiter _ra(&sharedState.counters.recursionCount, FInt::LuauTypeInferRecursionLimit); while (!get(haystack)) { @@ -2186,8 +1963,8 @@ void Unifier::occursCheck(std::unordered_set& seen_DEPRECATED, Dense { if (auto f = get(follow(ty))) { - occursCheck(seen_DEPRECATED, seen, needle, f->argTypes); - occursCheck(seen_DEPRECATED, seen, needle, f->retType); + occursCheck(seen, needle, f->argTypes); + occursCheck(seen, needle, f->retType); } } } @@ -2204,10 +1981,7 @@ void Unifier::occursCheck(std::unordered_set& seen_DEPRECATED, Dense Unifier Unifier::makeChildUnifier() { - if (FFlag::LuauShareTxnSeen) - return Unifier{types, mode, globalScope, log.sharedSeen, location, variance, sharedState, counters_DEPRECATED, counters}; - else - return Unifier{types, mode, globalScope, log.ownedSeen, location, variance, sharedState, counters_DEPRECATED, counters}; + return Unifier{types, mode, globalScope, log.sharedSeen, location, variance, sharedState}; } bool Unifier::isNonstrictMode() const diff --git a/Ast/src/Parser.cpp b/Ast/src/Parser.cpp index bc63e37d..3d0d5b7e 100644 --- a/Ast/src/Parser.cpp +++ b/Ast/src/Parser.cpp @@ -13,8 +13,6 @@ LUAU_FASTINTVARIABLE(LuauParseErrorLimit, 100) LUAU_FASTFLAGVARIABLE(LuauCaptureBrokenCommentSpans, false) LUAU_FASTFLAGVARIABLE(LuauIfElseExpressionBaseSupport, false) LUAU_FASTFLAGVARIABLE(LuauIfStatementRecursionGuard, false) -LUAU_FASTFLAGVARIABLE(LuauTypeAliasPacks, false) -LUAU_FASTFLAGVARIABLE(LuauParseTypePackTypeParameters, false) LUAU_FASTFLAGVARIABLE(LuauFixAmbiguousErrorRecoveryInAssign, false) LUAU_FASTFLAGVARIABLE(LuauParseSingletonTypes, false) LUAU_FASTFLAGVARIABLE(LuauParseGenericFunctionTypeBegin, false) @@ -782,8 +780,7 @@ AstStat* Parser::parseTypeAlias(const Location& start, bool exported) AstType* type = parseTypeAnnotation(); - return allocator.alloc( - Location(start, type->location), name->name, generics, FFlag::LuauTypeAliasPacks ? genericPacks : AstArray{}, type, exported); + return allocator.alloc(Location(start, type->location), name->name, generics, genericPacks, type, exported); } AstDeclaredClassProp Parser::parseDeclaredClassMethod() @@ -1602,30 +1599,18 @@ AstTypeOrPack Parser::parseSimpleTypeAnnotation(bool allowPack) return {allocator.alloc(Location(begin, end), expr), {}}; } - if (FFlag::LuauParseTypePackTypeParameters) + bool hasParameters = false; + AstArray parameters{}; + + if (lexer.current().type == '<') { - bool hasParameters = false; - AstArray parameters{}; - - if (lexer.current().type == '<') - { - hasParameters = true; - parameters = parseTypeParams(); - } - - Location end = lexer.previousLocation(); - - return {allocator.alloc(Location(begin, end), prefix, name.name, hasParameters, parameters), {}}; + hasParameters = true; + parameters = parseTypeParams(); } - else - { - AstArray generics = parseTypeParams(); - Location end = lexer.previousLocation(); + Location end = lexer.previousLocation(); - // false in 'hasParameterList' as it is not used without FFlagLuauTypeAliasPacks - return {allocator.alloc(Location(begin, end), prefix, name.name, false, generics), {}}; - } + return {allocator.alloc(Location(begin, end), prefix, name.name, hasParameters, parameters), {}}; } else if (lexer.current().type == '{') { @@ -2414,37 +2399,24 @@ AstArray Parser::parseTypeParams() while (true) { - if (FFlag::LuauParseTypePackTypeParameters) + if (shouldParseTypePackAnnotation(lexer)) { - if (shouldParseTypePackAnnotation(lexer)) - { - auto typePack = parseTypePackAnnotation(); + auto typePack = parseTypePackAnnotation(); - if (FFlag::LuauTypeAliasPacks) // Type packs are recorded only is we can handle them - parameters.push_back({{}, typePack}); - } - else if (lexer.current().type == '(') - { - auto [type, typePack] = parseTypeOrPackAnnotation(); + parameters.push_back({{}, typePack}); + } + else if (lexer.current().type == '(') + { + auto [type, typePack] = parseTypeOrPackAnnotation(); - if (typePack) - { - if (FFlag::LuauTypeAliasPacks) // Type packs are recorded only is we can handle them - parameters.push_back({{}, typePack}); - } - else - { - parameters.push_back({type, {}}); - } - } - else if (lexer.current().type == '>' && parameters.empty()) - { - break; - } + if (typePack) + parameters.push_back({{}, typePack}); else - { - parameters.push_back({parseTypeAnnotation(), {}}); - } + parameters.push_back({type, {}}); + } + else if (lexer.current().type == '>' && parameters.empty()) + { + break; } else { diff --git a/CLI/Analyze.cpp b/CLI/Analyze.cpp index ebdd7896..9230d80d 100644 --- a/CLI/Analyze.cpp +++ b/CLI/Analyze.cpp @@ -121,7 +121,7 @@ struct CliFileResolver : Luau::FileResolver if (Luau::AstExprConstantString* expr = node->as()) { Luau::ModuleName name = std::string(expr->value.data, expr->value.size) + ".luau"; - if (!moduleExists(name)) + if (!readFile(name)) { // fall back to .lua if a module with .luau doesn't exist name = std::string(expr->value.data, expr->value.size) + ".lua"; @@ -132,27 +132,6 @@ struct CliFileResolver : Luau::FileResolver return std::nullopt; } - - bool moduleExists(const Luau::ModuleName& name) const override - { - return !!readFile(name); - } - - - std::optional fromAstFragment(Luau::AstExpr* expr) const override - { - return std::nullopt; - } - - Luau::ModuleName concat(const Luau::ModuleName& lhs, std::string_view rhs) const override - { - return lhs + "/" + std::string(rhs); - } - - std::optional getParentModuleName(const Luau::ModuleName& name) const override - { - return std::nullopt; - } }; struct CliConfigResolver : Luau::ConfigResolver diff --git a/CLI/Repl.cpp b/CLI/Repl.cpp index b29cd6f9..2cdd0062 100644 --- a/CLI/Repl.cpp +++ b/CLI/Repl.cpp @@ -198,11 +198,6 @@ static std::string runCode(lua_State* L, const std::string& source) error += "\nstack backtrace:\n"; error += lua_debugtrace(T); -#ifdef __EMSCRIPTEN__ - // nicer formatting for errors in web repl - error = "Error:" + error; -#endif - fprintf(stdout, "%s", error.c_str()); } @@ -210,39 +205,6 @@ static std::string runCode(lua_State* L, const std::string& source) return std::string(); } -#ifdef __EMSCRIPTEN__ -extern "C" -{ - const char* executeScript(const char* source) - { - // setup flags - for (Luau::FValue* flag = Luau::FValue::list; flag; flag = flag->next) - if (strncmp(flag->name, "Luau", 4) == 0) - flag->value = true; - - // create new state - std::unique_ptr globalState(luaL_newstate(), lua_close); - lua_State* L = globalState.get(); - - // setup state - setupState(L); - - // sandbox thread - luaL_sandboxthread(L); - - // static string for caching result (prevents dangling ptr on function exit) - static std::string result; - - // run code + collect error - result = runCode(L, source); - - return result.empty() ? NULL : result.c_str(); - } -} -#endif - -// Excluded from emscripten compilation to avoid -Wunused-function errors. -#ifndef __EMSCRIPTEN__ static void completeIndexer(lua_State* L, const char* editBuffer, size_t start, std::vector& completions) { std::string_view lookup = editBuffer + start; @@ -564,6 +526,5 @@ int main(int argc, char** argv) return failed; } } -#endif diff --git a/CLI/Web.cpp b/CLI/Web.cpp new file mode 100644 index 00000000..cf5c831e --- /dev/null +++ b/CLI/Web.cpp @@ -0,0 +1,106 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +#include "lua.h" +#include "lualib.h" +#include "luacode.h" + +#include "Luau/Common.h" + +#include + +#include + +static void setupState(lua_State* L) +{ + luaL_openlibs(L); + + luaL_sandbox(L); +} + +static std::string runCode(lua_State* L, const std::string& source) +{ + size_t bytecodeSize = 0; + char* bytecode = luau_compile(source.data(), source.length(), nullptr, &bytecodeSize); + int result = luau_load(L, "=stdin", bytecode, bytecodeSize, 0); + free(bytecode); + + if (result != 0) + { + size_t len; + const char* msg = lua_tolstring(L, -1, &len); + + std::string error(msg, len); + lua_pop(L, 1); + + return error; + } + + lua_State* T = lua_newthread(L); + + lua_pushvalue(L, -2); + lua_remove(L, -3); + lua_xmove(L, T, 1); + + int status = lua_resume(T, NULL, 0); + + if (status == 0) + { + int n = lua_gettop(T); + + if (n) + { + luaL_checkstack(T, LUA_MINSTACK, "too many results to print"); + lua_getglobal(T, "print"); + lua_insert(T, 1); + lua_pcall(T, n, 0, 0); + } + } + else + { + std::string error; + + if (status == LUA_YIELD) + { + error = "thread yielded unexpectedly"; + } + else if (const char* str = lua_tostring(T, -1)) + { + error = str; + } + + error += "\nstack backtrace:\n"; + error += lua_debugtrace(T); + + error = "Error:" + error; + + fprintf(stdout, "%s", error.c_str()); + } + + lua_pop(L, 1); + return std::string(); +} + +extern "C" const char* executeScript(const char* source) +{ + // setup flags + for (Luau::FValue* flag = Luau::FValue::list; flag; flag = flag->next) + if (strncmp(flag->name, "Luau", 4) == 0) + flag->value = true; + + // create new state + std::unique_ptr globalState(luaL_newstate(), lua_close); + lua_State* L = globalState.get(); + + // setup state + setupState(L); + + // sandbox thread + luaL_sandboxthread(L); + + // static string for caching result (prevents dangling ptr on function exit) + static std::string result; + + // run code + collect error + result = runCode(L, source); + + return result.empty() ? NULL : result.c_str(); +} diff --git a/CMakeLists.txt b/CMakeLists.txt index 9c69521e..bafc59e5 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -9,6 +9,7 @@ project(Luau LANGUAGES CXX) option(LUAU_BUILD_CLI "Build CLI" ON) option(LUAU_BUILD_TESTS "Build tests" ON) +option(LUAU_BUILD_WEB "Build Web module" OFF) option(LUAU_WERROR "Warnings as errors" OFF) add_library(Luau.Ast STATIC) @@ -18,26 +19,22 @@ add_library(Luau.VM STATIC) if(LUAU_BUILD_CLI) add_executable(Luau.Repl.CLI) - if(NOT EMSCRIPTEN) - add_executable(Luau.Analyze.CLI) - else() - # add -fexceptions for emscripten to allow exceptions to be caught in C++ - set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -fexceptions") - endif() + add_executable(Luau.Analyze.CLI) # This also adds target `name` on Linux/macOS and `name.exe` on Windows set_target_properties(Luau.Repl.CLI PROPERTIES OUTPUT_NAME luau) - - if(NOT EMSCRIPTEN) - set_target_properties(Luau.Analyze.CLI PROPERTIES OUTPUT_NAME luau-analyze) - endif() + set_target_properties(Luau.Analyze.CLI PROPERTIES OUTPUT_NAME luau-analyze) endif() -if(LUAU_BUILD_TESTS AND NOT EMSCRIPTEN) +if(LUAU_BUILD_TESTS) add_executable(Luau.UnitTest) add_executable(Luau.Conformance) endif() +if(LUAU_BUILD_WEB) + add_executable(Luau.Web) +endif() + include(Sources.cmake) target_compile_features(Luau.Ast PUBLIC cxx_std_17) @@ -72,16 +69,18 @@ if(LUAU_WERROR) endif() endif() +if(LUAU_BUILD_WEB) + # add -fexceptions for emscripten to allow exceptions to be caught in C++ + list(APPEND LUAU_OPTIONS -fexceptions) +endif() + target_compile_options(Luau.Ast PRIVATE ${LUAU_OPTIONS}) target_compile_options(Luau.Analysis PRIVATE ${LUAU_OPTIONS}) target_compile_options(Luau.VM PRIVATE ${LUAU_OPTIONS}) if(LUAU_BUILD_CLI) target_compile_options(Luau.Repl.CLI PRIVATE ${LUAU_OPTIONS}) - - if(NOT EMSCRIPTEN) - target_compile_options(Luau.Analyze.CLI PRIVATE ${LUAU_OPTIONS}) - endif() + target_compile_options(Luau.Analyze.CLI PRIVATE ${LUAU_OPTIONS}) target_include_directories(Luau.Repl.CLI PRIVATE extern) target_link_libraries(Luau.Repl.CLI PRIVATE Luau.Compiler Luau.VM) @@ -93,20 +92,10 @@ if(LUAU_BUILD_CLI) endif() endif() - if(NOT EMSCRIPTEN) - target_link_libraries(Luau.Analyze.CLI PRIVATE Luau.Analysis) - endif() - - if(EMSCRIPTEN) - # declare exported functions to emscripten - target_link_options(Luau.Repl.CLI PRIVATE -sEXPORTED_FUNCTIONS=['_executeScript'] -sEXPORTED_RUNTIME_METHODS=['ccall','cwrap'] -fexceptions) - - # custom output directory for wasm + js file - set_target_properties(Luau.Repl.CLI PROPERTIES RUNTIME_OUTPUT_DIRECTORY ${CMAKE_SOURCE_DIR}/docs/assets/luau) - endif() + target_link_libraries(Luau.Analyze.CLI PRIVATE Luau.Analysis) endif() -if(LUAU_BUILD_TESTS AND NOT EMSCRIPTEN) +if(LUAU_BUILD_TESTS) target_compile_options(Luau.UnitTest PRIVATE ${LUAU_OPTIONS}) target_include_directories(Luau.UnitTest PRIVATE extern) target_link_libraries(Luau.UnitTest PRIVATE Luau.Analysis Luau.Compiler) @@ -115,3 +104,17 @@ if(LUAU_BUILD_TESTS AND NOT EMSCRIPTEN) target_include_directories(Luau.Conformance PRIVATE extern) target_link_libraries(Luau.Conformance PRIVATE Luau.Analysis Luau.Compiler Luau.VM) endif() + +if(LUAU_BUILD_WEB) + target_compile_options(Luau.Web PRIVATE ${LUAU_OPTIONS}) + target_link_libraries(Luau.Web PRIVATE Luau.Compiler Luau.VM) + + # declare exported functions to emscripten + target_link_options(Luau.Web PRIVATE -sEXPORTED_FUNCTIONS=['_executeScript'] -sEXPORTED_RUNTIME_METHODS=['ccall','cwrap']) + + # add -fexceptions for emscripten to allow exceptions to be caught in C++ + target_link_options(Luau.Web PRIVATE -fexceptions) + + # the output is a single .js file with an embedded wasm blob + target_link_options(Luau.Web PRIVATE -sSINGLE_FILE=1) +endif() diff --git a/Compiler/src/Compiler.cpp b/Compiler/src/Compiler.cpp index 5b93c1dc..2c1e85ff 100644 --- a/Compiler/src/Compiler.cpp +++ b/Compiler/src/Compiler.cpp @@ -321,13 +321,15 @@ struct Compiler compileExprTempTop(expr->args.data[i], uint8_t(regs + 1 + expr->self + i)); } - setDebugLine(expr->func); + setDebugLineEnd(expr->func); if (expr->self) { AstExprIndexName* fi = expr->func->as(); LUAU_ASSERT(fi); + setDebugLine(fi->indexLocation); + BytecodeBuilder::StringRef iname = sref(fi->index); int32_t cid = bytecode.addConstantString(iname); if (cid < 0) @@ -1313,6 +1315,8 @@ struct Compiler RegScope rs(this); uint8_t reg = compileExprAuto(expr->expr, rs); + setDebugLine(expr->indexLocation); + BytecodeBuilder::StringRef iname = sref(expr->index); int32_t cid = bytecode.addConstantString(iname); if (cid < 0) @@ -2710,6 +2714,12 @@ struct Compiler bytecode.setDebugLine(node->location.begin.line + 1); } + void setDebugLine(const Location& location) + { + if (options.debugLevel >= 1) + bytecode.setDebugLine(location.begin.line + 1); + } + void setDebugLineEnd(AstNode* node) { if (options.debugLevel >= 1) @@ -3650,7 +3660,7 @@ struct Compiler { if (options.vectorLib) { - if (builtin.object == options.vectorLib && builtin.method == options.vectorCtor) + if (builtin.isMethod(options.vectorLib, options.vectorCtor)) return LBF_VECTOR; } else diff --git a/Makefile b/Makefile index cab3d43f..15c7ff7a 100644 --- a/Makefile +++ b/Makefile @@ -35,7 +35,7 @@ ANALYZE_CLI_SOURCES=CLI/FileUtils.cpp CLI/Analyze.cpp ANALYZE_CLI_OBJECTS=$(ANALYZE_CLI_SOURCES:%=$(BUILD)/%.o) ANALYZE_CLI_TARGET=$(BUILD)/luau-analyze -FUZZ_SOURCES=$(wildcard fuzz/*.cpp) +FUZZ_SOURCES=$(wildcard fuzz/*.cpp) fuzz/luau.pb.cpp FUZZ_OBJECTS=$(FUZZ_SOURCES:%=$(BUILD)/%.o) TESTS_ARGS= @@ -167,8 +167,8 @@ fuzz/luau.pb.cpp: fuzz/luau.proto build/libprotobuf-mutator cd fuzz && ../build/libprotobuf-mutator/external.protobuf/bin/protoc luau.proto --cpp_out=. mv fuzz/luau.pb.cc fuzz/luau.pb.cpp -$(BUILD)/fuzz/proto.cpp.o: build/libprotobuf-mutator -$(BUILD)/fuzz/protoprint.cpp.o: build/libprotobuf-mutator +$(BUILD)/fuzz/proto.cpp.o: fuzz/luau.pb.cpp +$(BUILD)/fuzz/protoprint.cpp.o: fuzz/luau.pb.cpp build/libprotobuf-mutator: git clone https://github.com/google/libprotobuf-mutator build/libprotobuf-mutator diff --git a/Sources.cmake b/Sources.cmake index 23b931c6..57df9b91 100644 --- a/Sources.cmake +++ b/Sources.cmake @@ -54,6 +54,7 @@ target_sources(Luau.Analysis PRIVATE Analysis/include/Luau/Scope.h Analysis/include/Luau/Substitution.h Analysis/include/Luau/Symbol.h + Analysis/include/Luau/ToDot.h Analysis/include/Luau/TopoSortStatements.h Analysis/include/Luau/ToString.h Analysis/include/Luau/Transpiler.h @@ -86,6 +87,7 @@ target_sources(Luau.Analysis PRIVATE Analysis/src/Scope.cpp Analysis/src/Substitution.cpp Analysis/src/Symbol.cpp + Analysis/src/ToDot.cpp Analysis/src/TopoSortStatements.cpp Analysis/src/ToString.cpp Analysis/src/Transpiler.cpp @@ -118,6 +120,7 @@ target_sources(Luau.VM PRIVATE VM/src/ldo.cpp VM/src/lfunc.cpp VM/src/lgc.cpp + VM/src/lgcdebug.cpp VM/src/linit.cpp VM/src/lmathlib.cpp VM/src/lmem.cpp @@ -194,6 +197,7 @@ if(TARGET Luau.UnitTest) tests/RequireTracer.test.cpp tests/StringUtils.test.cpp tests/Symbol.test.cpp + tests/ToDot.test.cpp tests/TopoSort.test.cpp tests/ToString.test.cpp tests/Transpiler.test.cpp @@ -224,3 +228,9 @@ if(TARGET Luau.Conformance) tests/Conformance.test.cpp tests/main.cpp) endif() + +if(TARGET Luau.Web) + # Luau.Web Sources + target_sources(Luau.Web PRIVATE + CLI/Web.cpp) +endif() diff --git a/VM/include/lua.h b/VM/include/lua.h index 1568d191..7078acd0 100644 --- a/VM/include/lua.h +++ b/VM/include/lua.h @@ -21,6 +21,7 @@ #define LUA_ENVIRONINDEX (-10001) #define LUA_GLOBALSINDEX (-10002) #define lua_upvalueindex(i) (LUA_GLOBALSINDEX - (i)) +#define lua_ispseudo(i) ((i) <= LUA_REGISTRYINDEX) /* thread status; 0 is OK */ enum lua_Status @@ -108,6 +109,7 @@ LUA_API int lua_isthreadreset(lua_State* L); /* ** basic stack manipulation */ +LUA_API int lua_absindex(lua_State* L, int idx); LUA_API int lua_gettop(lua_State* L); LUA_API void lua_settop(lua_State* L, int idx); LUA_API void lua_pushvalue(lua_State* L, int idx); @@ -159,7 +161,11 @@ LUA_API void lua_pushnil(lua_State* L); LUA_API void lua_pushnumber(lua_State* L, double n); LUA_API void lua_pushinteger(lua_State* L, int n); LUA_API void lua_pushunsigned(lua_State* L, unsigned n); +#if LUA_VECTOR_SIZE == 4 +LUA_API void lua_pushvector(lua_State* L, float x, float y, float z, float w); +#else LUA_API void lua_pushvector(lua_State* L, float x, float y, float z); +#endif LUA_API void lua_pushlstring(lua_State* L, const char* s, size_t l); LUA_API void lua_pushstring(lua_State* L, const char* s); LUA_API const char* lua_pushvfstring(lua_State* L, const char* fmt, va_list argp); @@ -183,7 +189,7 @@ LUA_API void lua_setreadonly(lua_State* L, int idx, int enabled); LUA_API int lua_getreadonly(lua_State* L, int idx); LUA_API void lua_setsafeenv(lua_State* L, int idx, int enabled); -LUA_API void* lua_newuserdata(lua_State* L, size_t sz, int tag); +LUA_API void* lua_newuserdatatagged(lua_State* L, size_t sz, int tag); LUA_API void* lua_newuserdatadtor(lua_State* L, size_t sz, void (*dtor)(void*)); LUA_API int lua_getmetatable(lua_State* L, int objindex); LUA_API void lua_getfenv(lua_State* L, int idx); @@ -227,6 +233,7 @@ enum lua_GCOp LUA_GCRESTART, LUA_GCCOLLECT, LUA_GCCOUNT, + LUA_GCCOUNTB, LUA_GCISRUNNING, // garbage collection is handled by 'assists' that perform some amount of GC work matching pace of allocation @@ -281,6 +288,7 @@ LUA_API void lua_unref(lua_State* L, int ref); #define lua_pop(L, n) lua_settop(L, -(n)-1) #define lua_newtable(L) lua_createtable(L, 0, 0) +#define lua_newuserdata(L, s) lua_newuserdatatagged(L, s, 0) #define lua_strlen(L, i) lua_objlen(L, (i)) @@ -289,6 +297,7 @@ LUA_API void lua_unref(lua_State* L, int ref); #define lua_islightuserdata(L, n) (lua_type(L, (n)) == LUA_TLIGHTUSERDATA) #define lua_isnil(L, n) (lua_type(L, (n)) == LUA_TNIL) #define lua_isboolean(L, n) (lua_type(L, (n)) == LUA_TBOOLEAN) +#define lua_isvector(L, n) (lua_type(L, (n)) == LUA_TVECTOR) #define lua_isthread(L, n) (lua_type(L, (n)) == LUA_TTHREAD) #define lua_isnone(L, n) (lua_type(L, (n)) == LUA_TNONE) #define lua_isnoneornil(L, n) (lua_type(L, (n)) <= LUA_TNIL) diff --git a/VM/include/luaconf.h b/VM/include/luaconf.h index aa008a24..a01a1481 100644 --- a/VM/include/luaconf.h +++ b/VM/include/luaconf.h @@ -34,7 +34,10 @@ #endif /* Can be used to reconfigure visibility/exports for public APIs */ +#ifndef LUA_API #define LUA_API extern +#endif + #define LUALIB_API LUA_API /* Can be used to reconfigure visibility for internal APIs */ @@ -47,10 +50,14 @@ #endif /* Can be used to reconfigure internal error handling to use longjmp instead of C++ EH */ +#ifndef LUA_USE_LONGJMP #define LUA_USE_LONGJMP 0 +#endif /* LUA_IDSIZE gives the maximum size for the description of the source */ +#ifndef LUA_IDSIZE #define LUA_IDSIZE 256 +#endif /* @@ LUAI_GCGOAL defines the desired top heap size in relation to the live heap @@ -59,7 +66,9 @@ ** mean larger GC pauses which mean slower collection.) You can also change ** this value dynamically. */ +#ifndef LUAI_GCGOAL #define LUAI_GCGOAL 200 /* 200% (allow heap to double compared to live heap size) */ +#endif /* @@ LUAI_GCSTEPMUL / LUAI_GCSTEPSIZE define the default speed of garbage collection @@ -69,38 +78,63 @@ ** CHANGE it if you want to change the granularity of the garbage ** collection. */ +#ifndef LUAI_GCSTEPMUL #define LUAI_GCSTEPMUL 200 /* GC runs 'twice the speed' of memory allocation */ +#endif + +#ifndef LUAI_GCSTEPSIZE #define LUAI_GCSTEPSIZE 1 /* GC runs every KB of memory allocation */ +#endif /* LUA_MINSTACK is the guaranteed number of Lua stack slots available to a C function */ +#ifndef LUA_MINSTACK #define LUA_MINSTACK 20 +#endif /* LUAI_MAXCSTACK limits the number of Lua stack slots that a C function can use */ +#ifndef LUAI_MAXCSTACK #define LUAI_MAXCSTACK 8000 +#endif /* LUAI_MAXCALLS limits the number of nested calls */ +#ifndef LUAI_MAXCALLS #define LUAI_MAXCALLS 20000 +#endif /* LUAI_MAXCCALLS is the maximum depth for nested C calls; this limit depends on native stack size */ +#ifndef LUAI_MAXCCALLS #define LUAI_MAXCCALLS 200 +#endif /* buffer size used for on-stack string operations; this limit depends on native stack size */ +#ifndef LUA_BUFFERSIZE #define LUA_BUFFERSIZE 512 +#endif /* number of valid Lua userdata tags */ +#ifndef LUA_UTAG_LIMIT #define LUA_UTAG_LIMIT 128 +#endif /* upper bound for number of size classes used by page allocator */ +#ifndef LUA_SIZECLASSES #define LUA_SIZECLASSES 32 +#endif /* available number of separate memory categories */ +#ifndef LUA_MEMORY_CATEGORIES #define LUA_MEMORY_CATEGORIES 256 +#endif /* minimum size for the string table (must be power of 2) */ +#ifndef LUA_MINSTRTABSIZE #define LUA_MINSTRTABSIZE 32 +#endif /* maximum number of captures supported by pattern matching */ +#ifndef LUA_MAXCAPTURES #define LUA_MAXCAPTURES 32 +#endif /* }================================================================== */ @@ -122,3 +156,7 @@ void* s; \ long l; \ } + +#define LUA_VECTOR_SIZE 3 /* must be 3 or 4 */ + +#define LUA_EXTRA_SIZE LUA_VECTOR_SIZE - 2 diff --git a/VM/include/lualib.h b/VM/include/lualib.h index fa836955..baf27b47 100644 --- a/VM/include/lualib.h +++ b/VM/include/lualib.h @@ -25,11 +25,17 @@ LUALIB_API const char* luaL_optlstring(lua_State* L, int numArg, const char* def LUALIB_API double luaL_checknumber(lua_State* L, int numArg); LUALIB_API double luaL_optnumber(lua_State* L, int nArg, double def); +LUALIB_API int luaL_checkboolean(lua_State* L, int narg); +LUALIB_API int luaL_optboolean(lua_State* L, int narg, int def); + LUALIB_API int luaL_checkinteger(lua_State* L, int numArg); LUALIB_API int luaL_optinteger(lua_State* L, int nArg, int def); LUALIB_API unsigned luaL_checkunsigned(lua_State* L, int numArg); LUALIB_API unsigned luaL_optunsigned(lua_State* L, int numArg, unsigned def); +LUALIB_API const float* luaL_checkvector(lua_State* L, int narg); +LUALIB_API const float* luaL_optvector(lua_State* L, int narg, const float* def); + LUALIB_API void luaL_checkstack(lua_State* L, int sz, const char* msg); LUALIB_API void luaL_checktype(lua_State* L, int narg, int t); LUALIB_API void luaL_checkany(lua_State* L, int narg); diff --git a/VM/src/lapi.cpp b/VM/src/lapi.cpp index a79ba0d4..76043b9c 100644 --- a/VM/src/lapi.cpp +++ b/VM/src/lapi.cpp @@ -13,6 +13,8 @@ #include +LUAU_FASTFLAG(LuauActivateBeforeExec) + const char* lua_ident = "$Lua: Lua 5.1.4 Copyright (C) 1994-2008 Lua.org, PUC-Rio $\n" "$Authors: R. Ierusalimschy, L. H. de Figueiredo & W. Celes $\n" "$URL: www.lua.org $\n"; @@ -170,6 +172,12 @@ lua_State* lua_mainthread(lua_State* L) ** basic stack manipulation */ +int lua_absindex(lua_State* L, int idx) +{ + api_check(L, (idx > 0 && idx <= L->top - L->base) || (idx < 0 && -idx <= L->top - L->base) || lua_ispseudo(idx)); + return idx > 0 || lua_ispseudo(idx) ? idx : cast_int(L->top - L->base) + idx + 1; +} + int lua_gettop(lua_State* L) { return cast_int(L->top - L->base); @@ -550,12 +558,21 @@ void lua_pushunsigned(lua_State* L, unsigned u) return; } -void lua_pushvector(lua_State* L, float x, float y, float z) +#if LUA_VECTOR_SIZE == 4 +void lua_pushvector(lua_State* L, float x, float y, float z, float w) { - setvvalue(L->top, x, y, z); + setvvalue(L->top, x, y, z, w); api_incr_top(L); return; } +#else +void lua_pushvector(lua_State* L, float x, float y, float z) +{ + setvvalue(L->top, x, y, z, 0.0f); + api_incr_top(L); + return; +} +#endif void lua_pushlstring(lua_State* L, const char* s, size_t len) { @@ -922,14 +939,21 @@ void lua_call(lua_State* L, int nargs, int nresults) checkresults(L, nargs, nresults); func = L->top - (nargs + 1); - int wasActive = luaC_threadactive(L); - l_setbit(L->stackstate, THREAD_ACTIVEBIT); - luaC_checkthreadsleep(L); + if (FFlag::LuauActivateBeforeExec) + { + luaD_call(L, func, nresults); + } + else + { + int oldactive = luaC_threadactive(L); + l_setbit(L->stackstate, THREAD_ACTIVEBIT); + luaC_checkthreadsleep(L); - luaD_call(L, func, nresults); + luaD_call(L, func, nresults); - if (!wasActive) - resetbit(L->stackstate, THREAD_ACTIVEBIT); + if (!oldactive) + resetbit(L->stackstate, THREAD_ACTIVEBIT); + } adjustresults(L, nresults); return; @@ -970,14 +994,21 @@ int lua_pcall(lua_State* L, int nargs, int nresults, int errfunc) c.func = L->top - (nargs + 1); /* function to be called */ c.nresults = nresults; - int wasActive = luaC_threadactive(L); - l_setbit(L->stackstate, THREAD_ACTIVEBIT); - luaC_checkthreadsleep(L); + if (FFlag::LuauActivateBeforeExec) + { + status = luaD_pcall(L, f_call, &c, savestack(L, c.func), func); + } + else + { + int oldactive = luaC_threadactive(L); + l_setbit(L->stackstate, THREAD_ACTIVEBIT); + luaC_checkthreadsleep(L); - status = luaD_pcall(L, f_call, &c, savestack(L, c.func), func); + status = luaD_pcall(L, f_call, &c, savestack(L, c.func), func); - if (!wasActive) - resetbit(L->stackstate, THREAD_ACTIVEBIT); + if (!oldactive) + resetbit(L->stackstate, THREAD_ACTIVEBIT); + } adjustresults(L, nresults); return status; @@ -1030,6 +1061,11 @@ int lua_gc(lua_State* L, int what, int data) res = cast_int(g->totalbytes >> 10); break; } + case LUA_GCCOUNTB: + { + res = cast_int(g->totalbytes & 1023); + break; + } case LUA_GCISRUNNING: { res = (g->GCthreshold != SIZE_MAX); @@ -1146,7 +1182,7 @@ void lua_concat(lua_State* L, int n) return; } -void* lua_newuserdata(lua_State* L, size_t sz, int tag) +void* lua_newuserdatatagged(lua_State* L, size_t sz, int tag) { api_check(L, unsigned(tag) < LUA_UTAG_LIMIT); luaC_checkGC(L); @@ -1231,6 +1267,7 @@ uintptr_t lua_encodepointer(lua_State* L, uintptr_t p) int lua_ref(lua_State* L, int idx) { + api_check(L, idx != LUA_REGISTRYINDEX); /* idx is a stack index for value */ int ref = LUA_REFNIL; global_State* g = L->global; StkId p = index2adr(L, idx); diff --git a/VM/src/laux.cpp b/VM/src/laux.cpp index 2a684ee4..7ed2a62e 100644 --- a/VM/src/laux.cpp +++ b/VM/src/laux.cpp @@ -30,7 +30,7 @@ static const char* currfuncname(lua_State* L) return debugname; } -LUALIB_API l_noret luaL_argerrorL(lua_State* L, int narg, const char* extramsg) +l_noret luaL_argerrorL(lua_State* L, int narg, const char* extramsg) { const char* fname = currfuncname(L); @@ -40,7 +40,7 @@ LUALIB_API l_noret luaL_argerrorL(lua_State* L, int narg, const char* extramsg) luaL_error(L, "invalid argument #%d (%s)", narg, extramsg); } -LUALIB_API l_noret luaL_typeerrorL(lua_State* L, int narg, const char* tname) +l_noret luaL_typeerrorL(lua_State* L, int narg, const char* tname) { const char* fname = currfuncname(L); const TValue* obj = luaA_toobject(L, narg); @@ -66,7 +66,7 @@ static l_noret tag_error(lua_State* L, int narg, int tag) luaL_typeerrorL(L, narg, lua_typename(L, tag)); } -LUALIB_API void luaL_where(lua_State* L, int level) +void luaL_where(lua_State* L, int level) { lua_Debug ar; if (lua_getinfo(L, level, "sl", &ar) && ar.currentline > 0) @@ -77,7 +77,7 @@ LUALIB_API void luaL_where(lua_State* L, int level) lua_pushliteral(L, ""); /* else, no information available... */ } -LUALIB_API l_noret luaL_errorL(lua_State* L, const char* fmt, ...) +l_noret luaL_errorL(lua_State* L, const char* fmt, ...) { va_list argp; va_start(argp, fmt); @@ -90,7 +90,7 @@ LUALIB_API l_noret luaL_errorL(lua_State* L, const char* fmt, ...) /* }====================================================== */ -LUALIB_API int luaL_checkoption(lua_State* L, int narg, const char* def, const char* const lst[]) +int luaL_checkoption(lua_State* L, int narg, const char* def, const char* const lst[]) { const char* name = (def) ? luaL_optstring(L, narg, def) : luaL_checkstring(L, narg); int i; @@ -101,7 +101,7 @@ LUALIB_API int luaL_checkoption(lua_State* L, int narg, const char* def, const c luaL_argerrorL(L, narg, msg); } -LUALIB_API int luaL_newmetatable(lua_State* L, const char* tname) +int luaL_newmetatable(lua_State* L, const char* tname) { lua_getfield(L, LUA_REGISTRYINDEX, tname); /* get registry.name */ if (!lua_isnil(L, -1)) /* name already in use? */ @@ -113,7 +113,7 @@ LUALIB_API int luaL_newmetatable(lua_State* L, const char* tname) return 1; } -LUALIB_API void* luaL_checkudata(lua_State* L, int ud, const char* tname) +void* luaL_checkudata(lua_State* L, int ud, const char* tname) { void* p = lua_touserdata(L, ud); if (p != NULL) @@ -131,25 +131,25 @@ LUALIB_API void* luaL_checkudata(lua_State* L, int ud, const char* tname) luaL_typeerrorL(L, ud, tname); /* else error */ } -LUALIB_API void luaL_checkstack(lua_State* L, int space, const char* mes) +void luaL_checkstack(lua_State* L, int space, const char* mes) { if (!lua_checkstack(L, space)) luaL_error(L, "stack overflow (%s)", mes); } -LUALIB_API void luaL_checktype(lua_State* L, int narg, int t) +void luaL_checktype(lua_State* L, int narg, int t) { if (lua_type(L, narg) != t) tag_error(L, narg, t); } -LUALIB_API void luaL_checkany(lua_State* L, int narg) +void luaL_checkany(lua_State* L, int narg) { if (lua_type(L, narg) == LUA_TNONE) luaL_error(L, "missing argument #%d", narg); } -LUALIB_API const char* luaL_checklstring(lua_State* L, int narg, size_t* len) +const char* luaL_checklstring(lua_State* L, int narg, size_t* len) { const char* s = lua_tolstring(L, narg, len); if (!s) @@ -157,7 +157,7 @@ LUALIB_API const char* luaL_checklstring(lua_State* L, int narg, size_t* len) return s; } -LUALIB_API const char* luaL_optlstring(lua_State* L, int narg, const char* def, size_t* len) +const char* luaL_optlstring(lua_State* L, int narg, const char* def, size_t* len) { if (lua_isnoneornil(L, narg)) { @@ -169,7 +169,7 @@ LUALIB_API const char* luaL_optlstring(lua_State* L, int narg, const char* def, return luaL_checklstring(L, narg, len); } -LUALIB_API double luaL_checknumber(lua_State* L, int narg) +double luaL_checknumber(lua_State* L, int narg) { int isnum; double d = lua_tonumberx(L, narg, &isnum); @@ -178,12 +178,28 @@ LUALIB_API double luaL_checknumber(lua_State* L, int narg) return d; } -LUALIB_API double luaL_optnumber(lua_State* L, int narg, double def) +double luaL_optnumber(lua_State* L, int narg, double def) { return luaL_opt(L, luaL_checknumber, narg, def); } -LUALIB_API int luaL_checkinteger(lua_State* L, int narg) +int luaL_checkboolean(lua_State* L, int narg) +{ + // This checks specifically for boolean values, ignoring + // all other truthy/falsy values. If the desired result + // is true if value is present then lua_toboolean should + // directly be used instead. + if (!lua_isboolean(L, narg)) + tag_error(L, narg, LUA_TBOOLEAN); + return lua_toboolean(L, narg); +} + +int luaL_optboolean(lua_State* L, int narg, int def) +{ + return luaL_opt(L, luaL_checkboolean, narg, def); +} + +int luaL_checkinteger(lua_State* L, int narg) { int isnum; int d = lua_tointegerx(L, narg, &isnum); @@ -192,12 +208,12 @@ LUALIB_API int luaL_checkinteger(lua_State* L, int narg) return d; } -LUALIB_API int luaL_optinteger(lua_State* L, int narg, int def) +int luaL_optinteger(lua_State* L, int narg, int def) { return luaL_opt(L, luaL_checkinteger, narg, def); } -LUALIB_API unsigned luaL_checkunsigned(lua_State* L, int narg) +unsigned luaL_checkunsigned(lua_State* L, int narg) { int isnum; unsigned d = lua_tounsignedx(L, narg, &isnum); @@ -206,12 +222,25 @@ LUALIB_API unsigned luaL_checkunsigned(lua_State* L, int narg) return d; } -LUALIB_API unsigned luaL_optunsigned(lua_State* L, int narg, unsigned def) +unsigned luaL_optunsigned(lua_State* L, int narg, unsigned def) { return luaL_opt(L, luaL_checkunsigned, narg, def); } -LUALIB_API int luaL_getmetafield(lua_State* L, int obj, const char* event) +const float* luaL_checkvector(lua_State* L, int narg) +{ + const float* v = lua_tovector(L, narg); + if (!v) + tag_error(L, narg, LUA_TVECTOR); + return v; +} + +const float* luaL_optvector(lua_State* L, int narg, const float* def) +{ + return luaL_opt(L, luaL_checkvector, narg, def); +} + +int luaL_getmetafield(lua_State* L, int obj, const char* event) { if (!lua_getmetatable(L, obj)) /* no metatable? */ return 0; @@ -229,7 +258,7 @@ LUALIB_API int luaL_getmetafield(lua_State* L, int obj, const char* event) } } -LUALIB_API int luaL_callmeta(lua_State* L, int obj, const char* event) +int luaL_callmeta(lua_State* L, int obj, const char* event) { obj = abs_index(L, obj); if (!luaL_getmetafield(L, obj, event)) /* no metafield? */ @@ -247,7 +276,7 @@ static int libsize(const luaL_Reg* l) return size; } -LUALIB_API void luaL_register(lua_State* L, const char* libname, const luaL_Reg* l) +void luaL_register(lua_State* L, const char* libname, const luaL_Reg* l) { if (libname) { @@ -273,7 +302,7 @@ LUALIB_API void luaL_register(lua_State* L, const char* libname, const luaL_Reg* } } -LUALIB_API const char* luaL_findtable(lua_State* L, int idx, const char* fname, int szhint) +const char* luaL_findtable(lua_State* L, int idx, const char* fname, int szhint) { const char* e; lua_pushvalue(L, idx); @@ -324,7 +353,7 @@ static size_t getnextbuffersize(lua_State* L, size_t currentsize, size_t desired return newsize; } -LUALIB_API void luaL_buffinit(lua_State* L, luaL_Buffer* B) +void luaL_buffinit(lua_State* L, luaL_Buffer* B) { // start with an internal buffer B->p = B->buffer; @@ -334,14 +363,14 @@ LUALIB_API void luaL_buffinit(lua_State* L, luaL_Buffer* B) B->storage = nullptr; } -LUALIB_API char* luaL_buffinitsize(lua_State* L, luaL_Buffer* B, size_t size) +char* luaL_buffinitsize(lua_State* L, luaL_Buffer* B, size_t size) { luaL_buffinit(L, B); luaL_reservebuffer(B, size, -1); return B->p; } -LUALIB_API char* luaL_extendbuffer(luaL_Buffer* B, size_t additionalsize, int boxloc) +char* luaL_extendbuffer(luaL_Buffer* B, size_t additionalsize, int boxloc) { lua_State* L = B->L; @@ -372,13 +401,13 @@ LUALIB_API char* luaL_extendbuffer(luaL_Buffer* B, size_t additionalsize, int bo return B->p; } -LUALIB_API void luaL_reservebuffer(luaL_Buffer* B, size_t size, int boxloc) +void luaL_reservebuffer(luaL_Buffer* B, size_t size, int boxloc) { if (size_t(B->end - B->p) < size) luaL_extendbuffer(B, size - (B->end - B->p), boxloc); } -LUALIB_API void luaL_addlstring(luaL_Buffer* B, const char* s, size_t len) +void luaL_addlstring(luaL_Buffer* B, const char* s, size_t len) { if (size_t(B->end - B->p) < len) luaL_extendbuffer(B, len - (B->end - B->p), -1); @@ -387,7 +416,7 @@ LUALIB_API void luaL_addlstring(luaL_Buffer* B, const char* s, size_t len) B->p += len; } -LUALIB_API void luaL_addvalue(luaL_Buffer* B) +void luaL_addvalue(luaL_Buffer* B) { lua_State* L = B->L; @@ -404,7 +433,7 @@ LUALIB_API void luaL_addvalue(luaL_Buffer* B) } } -LUALIB_API void luaL_pushresult(luaL_Buffer* B) +void luaL_pushresult(luaL_Buffer* B) { lua_State* L = B->L; @@ -428,7 +457,7 @@ LUALIB_API void luaL_pushresult(luaL_Buffer* B) } } -LUALIB_API void luaL_pushresultsize(luaL_Buffer* B, size_t size) +void luaL_pushresultsize(luaL_Buffer* B, size_t size) { B->p += size; luaL_pushresult(B); @@ -436,7 +465,7 @@ LUALIB_API void luaL_pushresultsize(luaL_Buffer* B, size_t size) /* }====================================================== */ -LUALIB_API const char* luaL_tolstring(lua_State* L, int idx, size_t* len) +const char* luaL_tolstring(lua_State* L, int idx, size_t* len) { if (luaL_callmeta(L, idx, "__tostring")) /* is there a metafield? */ { @@ -462,7 +491,11 @@ LUALIB_API const char* luaL_tolstring(lua_State* L, int idx, size_t* len) case LUA_TVECTOR: { const float* v = lua_tovector(L, idx); +#if LUA_VECTOR_SIZE == 4 + lua_pushfstring(L, LUA_NUMBER_FMT ", " LUA_NUMBER_FMT ", " LUA_NUMBER_FMT ", " LUA_NUMBER_FMT, v[0], v[1], v[2], v[3]); +#else lua_pushfstring(L, LUA_NUMBER_FMT ", " LUA_NUMBER_FMT ", " LUA_NUMBER_FMT, v[0], v[1], v[2]); +#endif break; } default: diff --git a/VM/src/lbaselib.cpp b/VM/src/lbaselib.cpp index 61798e2b..881c804d 100644 --- a/VM/src/lbaselib.cpp +++ b/VM/src/lbaselib.cpp @@ -401,7 +401,7 @@ static int luaB_newproxy(lua_State* L) bool needsmt = lua_toboolean(L, 1); - lua_newuserdata(L, 0, 0); + lua_newuserdata(L, 0); if (needsmt) { @@ -441,7 +441,7 @@ static void auxopen(lua_State* L, const char* name, lua_CFunction f, lua_CFuncti lua_setfield(L, -2, name); } -LUALIB_API int luaopen_base(lua_State* L) +int luaopen_base(lua_State* L) { /* set global _G */ lua_pushvalue(L, LUA_GLOBALSINDEX); diff --git a/VM/src/lbitlib.cpp b/VM/src/lbitlib.cpp index 907c43c4..8b511edf 100644 --- a/VM/src/lbitlib.cpp +++ b/VM/src/lbitlib.cpp @@ -236,7 +236,7 @@ static const luaL_Reg bitlib[] = { {NULL, NULL}, }; -LUALIB_API int luaopen_bit32(lua_State* L) +int luaopen_bit32(lua_State* L) { luaL_register(L, LUA_BITLIBNAME, bitlib); diff --git a/VM/src/lbuiltins.cpp b/VM/src/lbuiltins.cpp index 9ab57ac9..34e9ebc1 100644 --- a/VM/src/lbuiltins.cpp +++ b/VM/src/lbuiltins.cpp @@ -1018,13 +1018,23 @@ static int luauF_tunpack(lua_State* L, StkId res, TValue* arg0, int nresults, St static int luauF_vector(lua_State* L, StkId res, TValue* arg0, int nresults, StkId args, int nparams) { +#if LUA_VECTOR_SIZE == 4 + if (nparams >= 4 && nresults <= 1 && ttisnumber(arg0) && ttisnumber(args) && ttisnumber(args + 1) && ttisnumber(args + 2)) +#else if (nparams >= 3 && nresults <= 1 && ttisnumber(arg0) && ttisnumber(args) && ttisnumber(args + 1)) +#endif { double x = nvalue(arg0); double y = nvalue(args); double z = nvalue(args + 1); - setvvalue(res, float(x), float(y), float(z)); +#if LUA_VECTOR_SIZE == 4 + double w = nvalue(args + 2); + setvvalue(res, float(x), float(y), float(z), float(w)); +#else + setvvalue(res, float(x), float(y), float(z), 0.0f); +#endif + return 1; } diff --git a/VM/src/lcorolib.cpp b/VM/src/lcorolib.cpp index 0178fae8..abcde779 100644 --- a/VM/src/lcorolib.cpp +++ b/VM/src/lcorolib.cpp @@ -272,7 +272,7 @@ static const luaL_Reg co_funcs[] = { {NULL, NULL}, }; -LUALIB_API int luaopen_coroutine(lua_State* L) +int luaopen_coroutine(lua_State* L) { luaL_register(L, LUA_COLIBNAME, co_funcs); diff --git a/VM/src/ldblib.cpp b/VM/src/ldblib.cpp index 965d2b3d..93d8703a 100644 --- a/VM/src/ldblib.cpp +++ b/VM/src/ldblib.cpp @@ -160,7 +160,7 @@ static const luaL_Reg dblib[] = { {NULL, NULL}, }; -LUALIB_API int luaopen_debug(lua_State* L) +int luaopen_debug(lua_State* L) { luaL_register(L, LUA_DBLIBNAME, dblib); return 1; diff --git a/VM/src/ldo.cpp b/VM/src/ldo.cpp index 1259d461..62bbdb7c 100644 --- a/VM/src/ldo.cpp +++ b/VM/src/ldo.cpp @@ -17,9 +17,9 @@ #include -LUAU_FASTFLAGVARIABLE(LuauExceptionMessageFix, false) LUAU_FASTFLAGVARIABLE(LuauCcallRestoreFix, false) LUAU_FASTFLAG(LuauCoroutineClose) +LUAU_FASTFLAGVARIABLE(LuauActivateBeforeExec, false) /* ** {====================================================== @@ -74,35 +74,28 @@ public: const char* what() const throw() override { - if (FFlag::LuauExceptionMessageFix) + // LUA_ERRRUN/LUA_ERRSYNTAX pass an object on the stack which is intended to describe the error. + if (status == LUA_ERRRUN || status == LUA_ERRSYNTAX) { - // LUA_ERRRUN/LUA_ERRSYNTAX pass an object on the stack which is intended to describe the error. - if (status == LUA_ERRRUN || status == LUA_ERRSYNTAX) + // Conversion to a string could still fail. For example if a user passes a non-string/non-number argument to `error()`. + if (const char* str = lua_tostring(L, -1)) { - // Conversion to a string could still fail. For example if a user passes a non-string/non-number argument to `error()`. - if (const char* str = lua_tostring(L, -1)) - { - return str; - } - } - - switch (status) - { - case LUA_ERRRUN: - return "lua_exception: LUA_ERRRUN (no string/number provided as description)"; - case LUA_ERRSYNTAX: - return "lua_exception: LUA_ERRSYNTAX (no string/number provided as description)"; - case LUA_ERRMEM: - return "lua_exception: " LUA_MEMERRMSG; - case LUA_ERRERR: - return "lua_exception: " LUA_ERRERRMSG; - default: - return "lua_exception: unexpected exception status"; + return str; } } - else + + switch (status) { - return lua_tostring(L, -1); + case LUA_ERRRUN: + return "lua_exception: LUA_ERRRUN (no string/number provided as description)"; + case LUA_ERRSYNTAX: + return "lua_exception: LUA_ERRSYNTAX (no string/number provided as description)"; + case LUA_ERRMEM: + return "lua_exception: " LUA_MEMERRMSG; + case LUA_ERRERR: + return "lua_exception: " LUA_ERRERRMSG; + default: + return "lua_exception: unexpected exception status"; } } @@ -234,7 +227,22 @@ void luaD_call(lua_State* L, StkId func, int nResults) if (luau_precall(L, func, nResults) == PCRLUA) { /* is a Lua function? */ L->ci->flags |= LUA_CALLINFO_RETURN; /* luau_execute will stop after returning from the stack frame */ - luau_execute(L); /* call it */ + + if (FFlag::LuauActivateBeforeExec) + { + int oldactive = luaC_threadactive(L); + l_setbit(L->stackstate, THREAD_ACTIVEBIT); + luaC_checkthreadsleep(L); + + luau_execute(L); /* call it */ + + if (!oldactive) + resetbit(L->stackstate, THREAD_ACTIVEBIT); + } + else + { + luau_execute(L); /* call it */ + } } L->nCcalls--; luaC_checkGC(L); @@ -527,10 +535,10 @@ static void restore_stack_limit(lua_State* L) int luaD_pcall(lua_State* L, Pfunc func, void* u, ptrdiff_t old_top, ptrdiff_t ef) { - int status; unsigned short oldnCcalls = L->nCcalls; ptrdiff_t old_ci = saveci(L, L->ci); - status = luaD_rawrunprotected(L, func, u); + int oldactive = luaC_threadactive(L); + int status = luaD_rawrunprotected(L, func, u); if (status != 0) { // call user-defined error function (used in xpcall) @@ -541,6 +549,13 @@ int luaD_pcall(lua_State* L, Pfunc func, void* u, ptrdiff_t old_top, ptrdiff_t e status = LUA_ERRERR; } + if (FFlag::LuauActivateBeforeExec) + { + // since the call failed with an error, we might have to reset the 'active' thread state + if (!oldactive) + resetbit(L->stackstate, THREAD_ACTIVEBIT); + } + if (FFlag::LuauCcallRestoreFix) { // Restore nCcalls before calling the debugprotectederror callback which may rely on the proper value to have been restored. diff --git a/VM/src/lgc.cpp b/VM/src/lgc.cpp index 11f79d1a..ab416041 100644 --- a/VM/src/lgc.cpp +++ b/VM/src/lgc.cpp @@ -10,9 +10,7 @@ #include "ldo.h" #include -#include -LUAU_FASTFLAGVARIABLE(LuauRescanGrayAgainForwardBarrier, false) LUAU_FASTFLAGVARIABLE(LuauSeparateAtomic, false) LUAU_FASTFLAG(LuauArrayBoundary) @@ -988,7 +986,7 @@ void luaC_barriertable(lua_State* L, Table* t, GCObject* v) GCObject* o = obj2gco(t); // in the second propagation stage, table assignment barrier works as a forward barrier - if (FFlag::LuauRescanGrayAgainForwardBarrier && g->gcstate == GCSpropagateagain) + if (g->gcstate == GCSpropagateagain) { LUAU_ASSERT(isblack(o) && iswhite(v) && !isdead(g, v) && !isdead(g, o)); reallymarkobject(g, v); @@ -1044,550 +1042,6 @@ void luaC_linkupval(lua_State* L, UpVal* uv) } } -static void validateobjref(global_State* g, GCObject* f, GCObject* t) -{ - LUAU_ASSERT(!isdead(g, t)); - - if (keepinvariant(g)) - { - /* basic incremental invariant: black can't point to white */ - LUAU_ASSERT(!(isblack(f) && iswhite(t))); - } -} - -static void validateref(global_State* g, GCObject* f, TValue* v) -{ - if (iscollectable(v)) - { - LUAU_ASSERT(ttype(v) == gcvalue(v)->gch.tt); - validateobjref(g, f, gcvalue(v)); - } -} - -static void validatetable(global_State* g, Table* h) -{ - int sizenode = 1 << h->lsizenode; - - if (FFlag::LuauArrayBoundary) - LUAU_ASSERT(h->lastfree <= sizenode); - else - LUAU_ASSERT(h->lastfree >= 0 && h->lastfree <= sizenode); - - if (h->metatable) - validateobjref(g, obj2gco(h), obj2gco(h->metatable)); - - for (int i = 0; i < h->sizearray; ++i) - validateref(g, obj2gco(h), &h->array[i]); - - for (int i = 0; i < sizenode; ++i) - { - LuaNode* n = &h->node[i]; - - LUAU_ASSERT(ttype(gkey(n)) != LUA_TDEADKEY || ttisnil(gval(n))); - LUAU_ASSERT(i + gnext(n) >= 0 && i + gnext(n) < sizenode); - - if (!ttisnil(gval(n))) - { - TValue k = {}; - k.tt = gkey(n)->tt; - k.value = gkey(n)->value; - - validateref(g, obj2gco(h), &k); - validateref(g, obj2gco(h), gval(n)); - } - } -} - -static void validateclosure(global_State* g, Closure* cl) -{ - validateobjref(g, obj2gco(cl), obj2gco(cl->env)); - - if (cl->isC) - { - for (int i = 0; i < cl->nupvalues; ++i) - validateref(g, obj2gco(cl), &cl->c.upvals[i]); - } - else - { - LUAU_ASSERT(cl->nupvalues == cl->l.p->nups); - - validateobjref(g, obj2gco(cl), obj2gco(cl->l.p)); - - for (int i = 0; i < cl->nupvalues; ++i) - validateref(g, obj2gco(cl), &cl->l.uprefs[i]); - } -} - -static void validatestack(global_State* g, lua_State* l) -{ - validateref(g, obj2gco(l), gt(l)); - - for (CallInfo* ci = l->base_ci; ci <= l->ci; ++ci) - { - LUAU_ASSERT(l->stack <= ci->base); - LUAU_ASSERT(ci->func <= ci->base && ci->base <= ci->top); - LUAU_ASSERT(ci->top <= l->stack_last); - } - - // note: stack refs can violate gc invariant so we only check for liveness - for (StkId o = l->stack; o < l->top; ++o) - checkliveness(g, o); - - if (l->namecall) - validateobjref(g, obj2gco(l), obj2gco(l->namecall)); - - for (GCObject* uv = l->openupval; uv; uv = uv->gch.next) - { - LUAU_ASSERT(uv->gch.tt == LUA_TUPVAL); - LUAU_ASSERT(gco2uv(uv)->v != &gco2uv(uv)->u.value); - } -} - -static void validateproto(global_State* g, Proto* f) -{ - if (f->source) - validateobjref(g, obj2gco(f), obj2gco(f->source)); - - if (f->debugname) - validateobjref(g, obj2gco(f), obj2gco(f->debugname)); - - for (int i = 0; i < f->sizek; ++i) - validateref(g, obj2gco(f), &f->k[i]); - - for (int i = 0; i < f->sizeupvalues; ++i) - if (f->upvalues[i]) - validateobjref(g, obj2gco(f), obj2gco(f->upvalues[i])); - - for (int i = 0; i < f->sizep; ++i) - if (f->p[i]) - validateobjref(g, obj2gco(f), obj2gco(f->p[i])); - - for (int i = 0; i < f->sizelocvars; i++) - if (f->locvars[i].varname) - validateobjref(g, obj2gco(f), obj2gco(f->locvars[i].varname)); -} - -static void validateobj(global_State* g, GCObject* o) -{ - /* dead objects can only occur during sweep */ - if (isdead(g, o)) - { - LUAU_ASSERT(g->gcstate == GCSsweepstring || g->gcstate == GCSsweep); - return; - } - - switch (o->gch.tt) - { - case LUA_TSTRING: - break; - - case LUA_TTABLE: - validatetable(g, gco2h(o)); - break; - - case LUA_TFUNCTION: - validateclosure(g, gco2cl(o)); - break; - - case LUA_TUSERDATA: - if (gco2u(o)->metatable) - validateobjref(g, o, obj2gco(gco2u(o)->metatable)); - break; - - case LUA_TTHREAD: - validatestack(g, gco2th(o)); - break; - - case LUA_TPROTO: - validateproto(g, gco2p(o)); - break; - - case LUA_TUPVAL: - validateref(g, o, gco2uv(o)->v); - break; - - default: - LUAU_ASSERT(!"unexpected object type"); - } -} - -static void validatelist(global_State* g, GCObject* o) -{ - while (o) - { - validateobj(g, o); - - o = o->gch.next; - } -} - -static void validategraylist(global_State* g, GCObject* o) -{ - if (!keepinvariant(g)) - return; - - while (o) - { - LUAU_ASSERT(isgray(o)); - - switch (o->gch.tt) - { - case LUA_TTABLE: - o = gco2h(o)->gclist; - break; - case LUA_TFUNCTION: - o = gco2cl(o)->gclist; - break; - case LUA_TTHREAD: - o = gco2th(o)->gclist; - break; - case LUA_TPROTO: - o = gco2p(o)->gclist; - break; - default: - LUAU_ASSERT(!"unknown object in gray list"); - return; - } - } -} - -void luaC_validate(lua_State* L) -{ - global_State* g = L->global; - - LUAU_ASSERT(!isdead(g, obj2gco(g->mainthread))); - checkliveness(g, &g->registry); - - for (int i = 0; i < LUA_T_COUNT; ++i) - if (g->mt[i]) - LUAU_ASSERT(!isdead(g, obj2gco(g->mt[i]))); - - validategraylist(g, g->weak); - validategraylist(g, g->gray); - validategraylist(g, g->grayagain); - - for (int i = 0; i < g->strt.size; ++i) - validatelist(g, g->strt.hash[i]); - - validatelist(g, g->rootgc); - validatelist(g, g->strbufgc); - - for (UpVal* uv = g->uvhead.u.l.next; uv != &g->uvhead; uv = uv->u.l.next) - { - LUAU_ASSERT(uv->tt == LUA_TUPVAL); - LUAU_ASSERT(uv->v != &uv->u.value); - LUAU_ASSERT(uv->u.l.next->u.l.prev == uv && uv->u.l.prev->u.l.next == uv); - } -} - -inline bool safejson(char ch) -{ - return unsigned(ch) < 128 && ch >= 32 && ch != '\\' && ch != '\"'; -} - -static void dumpref(FILE* f, GCObject* o) -{ - fprintf(f, "\"%p\"", o); -} - -static void dumprefs(FILE* f, TValue* data, size_t size) -{ - bool first = true; - - for (size_t i = 0; i < size; ++i) - { - if (iscollectable(&data[i])) - { - if (!first) - fputc(',', f); - first = false; - - dumpref(f, gcvalue(&data[i])); - } - } -} - -static void dumpstringdata(FILE* f, const char* data, size_t len) -{ - for (size_t i = 0; i < len; ++i) - fputc(safejson(data[i]) ? data[i] : '?', f); -} - -static void dumpstring(FILE* f, TString* ts) -{ - fprintf(f, "{\"type\":\"string\",\"cat\":%d,\"size\":%d,\"data\":\"", ts->memcat, int(sizestring(ts->len))); - dumpstringdata(f, ts->data, ts->len); - fprintf(f, "\"}"); -} - -static void dumptable(FILE* f, Table* h) -{ - size_t size = sizeof(Table) + (h->node == &luaH_dummynode ? 0 : sizenode(h) * sizeof(LuaNode)) + h->sizearray * sizeof(TValue); - - fprintf(f, "{\"type\":\"table\",\"cat\":%d,\"size\":%d", h->memcat, int(size)); - - if (h->node != &luaH_dummynode) - { - fprintf(f, ",\"pairs\":["); - - bool first = true; - - for (int i = 0; i < sizenode(h); ++i) - { - const LuaNode& n = h->node[i]; - - if (!ttisnil(&n.val) && (iscollectable(&n.key) || iscollectable(&n.val))) - { - if (!first) - fputc(',', f); - first = false; - - if (iscollectable(&n.key)) - dumpref(f, gcvalue(&n.key)); - else - fprintf(f, "null"); - - fputc(',', f); - - if (iscollectable(&n.val)) - dumpref(f, gcvalue(&n.val)); - else - fprintf(f, "null"); - } - } - - fprintf(f, "]"); - } - if (h->sizearray) - { - fprintf(f, ",\"array\":["); - dumprefs(f, h->array, h->sizearray); - fprintf(f, "]"); - } - if (h->metatable) - { - fprintf(f, ",\"metatable\":"); - dumpref(f, obj2gco(h->metatable)); - } - fprintf(f, "}"); -} - -static void dumpclosure(FILE* f, Closure* cl) -{ - fprintf(f, "{\"type\":\"function\",\"cat\":%d,\"size\":%d", cl->memcat, - cl->isC ? int(sizeCclosure(cl->nupvalues)) : int(sizeLclosure(cl->nupvalues))); - - fprintf(f, ",\"env\":"); - dumpref(f, obj2gco(cl->env)); - if (cl->isC) - { - if (cl->nupvalues) - { - fprintf(f, ",\"upvalues\":["); - dumprefs(f, cl->c.upvals, cl->nupvalues); - fprintf(f, "]"); - } - } - else - { - fprintf(f, ",\"proto\":"); - dumpref(f, obj2gco(cl->l.p)); - if (cl->nupvalues) - { - fprintf(f, ",\"upvalues\":["); - dumprefs(f, cl->l.uprefs, cl->nupvalues); - fprintf(f, "]"); - } - } - fprintf(f, "}"); -} - -static void dumpudata(FILE* f, Udata* u) -{ - fprintf(f, "{\"type\":\"userdata\",\"cat\":%d,\"size\":%d,\"tag\":%d", u->memcat, int(sizeudata(u->len)), u->tag); - - if (u->metatable) - { - fprintf(f, ",\"metatable\":"); - dumpref(f, obj2gco(u->metatable)); - } - fprintf(f, "}"); -} - -static void dumpthread(FILE* f, lua_State* th) -{ - size_t size = sizeof(lua_State) + sizeof(TValue) * th->stacksize + sizeof(CallInfo) * th->size_ci; - - fprintf(f, "{\"type\":\"thread\",\"cat\":%d,\"size\":%d", th->memcat, int(size)); - - if (iscollectable(&th->l_gt)) - { - fprintf(f, ",\"env\":"); - dumpref(f, gcvalue(&th->l_gt)); - } - - Closure* tcl = 0; - for (CallInfo* ci = th->base_ci; ci <= th->ci; ++ci) - { - if (ttisfunction(ci->func)) - { - tcl = clvalue(ci->func); - break; - } - } - - if (tcl && !tcl->isC && tcl->l.p->source) - { - Proto* p = tcl->l.p; - - fprintf(f, ",\"source\":\""); - dumpstringdata(f, p->source->data, p->source->len); - fprintf(f, "\",\"line\":%d", p->abslineinfo ? p->abslineinfo[0] : 0); - } - - if (th->top > th->stack) - { - fprintf(f, ",\"stack\":["); - dumprefs(f, th->stack, th->top - th->stack); - fprintf(f, "]"); - } - fprintf(f, "}"); -} - -static void dumpproto(FILE* f, Proto* p) -{ - size_t size = sizeof(Proto) + sizeof(Instruction) * p->sizecode + sizeof(Proto*) * p->sizep + sizeof(TValue) * p->sizek + p->sizelineinfo + - sizeof(LocVar) * p->sizelocvars + sizeof(TString*) * p->sizeupvalues; - - fprintf(f, "{\"type\":\"proto\",\"cat\":%d,\"size\":%d", p->memcat, int(size)); - - if (p->source) - { - fprintf(f, ",\"source\":\""); - dumpstringdata(f, p->source->data, p->source->len); - fprintf(f, "\",\"line\":%d", p->abslineinfo ? p->abslineinfo[0] : 0); - } - - if (p->sizek) - { - fprintf(f, ",\"constants\":["); - dumprefs(f, p->k, p->sizek); - fprintf(f, "]"); - } - - if (p->sizep) - { - fprintf(f, ",\"protos\":["); - for (int i = 0; i < p->sizep; ++i) - { - if (i != 0) - fputc(',', f); - dumpref(f, obj2gco(p->p[i])); - } - fprintf(f, "]"); - } - - fprintf(f, "}"); -} - -static void dumpupval(FILE* f, UpVal* uv) -{ - fprintf(f, "{\"type\":\"upvalue\",\"cat\":%d,\"size\":%d", uv->memcat, int(sizeof(UpVal))); - - if (iscollectable(uv->v)) - { - fprintf(f, ",\"object\":"); - dumpref(f, gcvalue(uv->v)); - } - fprintf(f, "}"); -} - -static void dumpobj(FILE* f, GCObject* o) -{ - switch (o->gch.tt) - { - case LUA_TSTRING: - return dumpstring(f, gco2ts(o)); - - case LUA_TTABLE: - return dumptable(f, gco2h(o)); - - case LUA_TFUNCTION: - return dumpclosure(f, gco2cl(o)); - - case LUA_TUSERDATA: - return dumpudata(f, gco2u(o)); - - case LUA_TTHREAD: - return dumpthread(f, gco2th(o)); - - case LUA_TPROTO: - return dumpproto(f, gco2p(o)); - - case LUA_TUPVAL: - return dumpupval(f, gco2uv(o)); - - default: - LUAU_ASSERT(0); - } -} - -static void dumplist(FILE* f, GCObject* o) -{ - while (o) - { - dumpref(f, o); - fputc(':', f); - dumpobj(f, o); - fputc(',', f); - fputc('\n', f); - - // thread has additional list containing collectable objects that are not present in rootgc - if (o->gch.tt == LUA_TTHREAD) - dumplist(f, gco2th(o)->openupval); - - o = o->gch.next; - } -} - -void luaC_dump(lua_State* L, void* file, const char* (*categoryName)(lua_State* L, uint8_t memcat)) -{ - global_State* g = L->global; - FILE* f = static_cast(file); - - fprintf(f, "{\"objects\":{\n"); - dumplist(f, g->rootgc); - dumplist(f, g->strbufgc); - for (int i = 0; i < g->strt.size; ++i) - dumplist(f, g->strt.hash[i]); - - fprintf(f, "\"0\":{\"type\":\"userdata\",\"cat\":0,\"size\":0}\n"); // to avoid issues with trailing , - fprintf(f, "},\"roots\":{\n"); - fprintf(f, "\"mainthread\":"); - dumpref(f, obj2gco(g->mainthread)); - fprintf(f, ",\"registry\":"); - dumpref(f, gcvalue(&g->registry)); - - fprintf(f, "},\"stats\":{\n"); - - fprintf(f, "\"size\":%d,\n", int(g->totalbytes)); - - fprintf(f, "\"categories\":{\n"); - for (int i = 0; i < LUA_MEMORY_CATEGORIES; i++) - { - if (size_t bytes = g->memcatbytes[i]) - { - if (categoryName) - fprintf(f, "\"%d\":{\"name\":\"%s\", \"size\":%d},\n", i, categoryName(L, i), int(bytes)); - else - fprintf(f, "\"%d\":{\"size\":%d},\n", i, int(bytes)); - } - } - fprintf(f, "\"none\":{}\n"); // to avoid issues with trailing , - fprintf(f, "}\n"); - fprintf(f, "}}\n"); -} - // measure the allocation rate in bytes/sec // returns -1 if allocation rate cannot be measured int64_t luaC_allocationrate(lua_State* L) diff --git a/VM/src/lgcdebug.cpp b/VM/src/lgcdebug.cpp new file mode 100644 index 00000000..a79e7b95 --- /dev/null +++ b/VM/src/lgcdebug.cpp @@ -0,0 +1,558 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +// This code is based on Lua 5.x implementation licensed under MIT License; see lua_LICENSE.txt for details +#include "lgc.h" + +#include "lobject.h" +#include "lstate.h" +#include "ltable.h" +#include "lfunc.h" +#include "lstring.h" + +#include +#include + +LUAU_FASTFLAG(LuauArrayBoundary) + +static void validateobjref(global_State* g, GCObject* f, GCObject* t) +{ + LUAU_ASSERT(!isdead(g, t)); + + if (keepinvariant(g)) + { + /* basic incremental invariant: black can't point to white */ + LUAU_ASSERT(!(isblack(f) && iswhite(t))); + } +} + +static void validateref(global_State* g, GCObject* f, TValue* v) +{ + if (iscollectable(v)) + { + LUAU_ASSERT(ttype(v) == gcvalue(v)->gch.tt); + validateobjref(g, f, gcvalue(v)); + } +} + +static void validatetable(global_State* g, Table* h) +{ + int sizenode = 1 << h->lsizenode; + + if (FFlag::LuauArrayBoundary) + LUAU_ASSERT(h->lastfree <= sizenode); + else + LUAU_ASSERT(h->lastfree >= 0 && h->lastfree <= sizenode); + + if (h->metatable) + validateobjref(g, obj2gco(h), obj2gco(h->metatable)); + + for (int i = 0; i < h->sizearray; ++i) + validateref(g, obj2gco(h), &h->array[i]); + + for (int i = 0; i < sizenode; ++i) + { + LuaNode* n = &h->node[i]; + + LUAU_ASSERT(ttype(gkey(n)) != LUA_TDEADKEY || ttisnil(gval(n))); + LUAU_ASSERT(i + gnext(n) >= 0 && i + gnext(n) < sizenode); + + if (!ttisnil(gval(n))) + { + TValue k = {}; + k.tt = gkey(n)->tt; + k.value = gkey(n)->value; + + validateref(g, obj2gco(h), &k); + validateref(g, obj2gco(h), gval(n)); + } + } +} + +static void validateclosure(global_State* g, Closure* cl) +{ + validateobjref(g, obj2gco(cl), obj2gco(cl->env)); + + if (cl->isC) + { + for (int i = 0; i < cl->nupvalues; ++i) + validateref(g, obj2gco(cl), &cl->c.upvals[i]); + } + else + { + LUAU_ASSERT(cl->nupvalues == cl->l.p->nups); + + validateobjref(g, obj2gco(cl), obj2gco(cl->l.p)); + + for (int i = 0; i < cl->nupvalues; ++i) + validateref(g, obj2gco(cl), &cl->l.uprefs[i]); + } +} + +static void validatestack(global_State* g, lua_State* l) +{ + validateref(g, obj2gco(l), gt(l)); + + for (CallInfo* ci = l->base_ci; ci <= l->ci; ++ci) + { + LUAU_ASSERT(l->stack <= ci->base); + LUAU_ASSERT(ci->func <= ci->base && ci->base <= ci->top); + LUAU_ASSERT(ci->top <= l->stack_last); + } + + // note: stack refs can violate gc invariant so we only check for liveness + for (StkId o = l->stack; o < l->top; ++o) + checkliveness(g, o); + + if (l->namecall) + validateobjref(g, obj2gco(l), obj2gco(l->namecall)); + + for (GCObject* uv = l->openupval; uv; uv = uv->gch.next) + { + LUAU_ASSERT(uv->gch.tt == LUA_TUPVAL); + LUAU_ASSERT(gco2uv(uv)->v != &gco2uv(uv)->u.value); + } +} + +static void validateproto(global_State* g, Proto* f) +{ + if (f->source) + validateobjref(g, obj2gco(f), obj2gco(f->source)); + + if (f->debugname) + validateobjref(g, obj2gco(f), obj2gco(f->debugname)); + + for (int i = 0; i < f->sizek; ++i) + validateref(g, obj2gco(f), &f->k[i]); + + for (int i = 0; i < f->sizeupvalues; ++i) + if (f->upvalues[i]) + validateobjref(g, obj2gco(f), obj2gco(f->upvalues[i])); + + for (int i = 0; i < f->sizep; ++i) + if (f->p[i]) + validateobjref(g, obj2gco(f), obj2gco(f->p[i])); + + for (int i = 0; i < f->sizelocvars; i++) + if (f->locvars[i].varname) + validateobjref(g, obj2gco(f), obj2gco(f->locvars[i].varname)); +} + +static void validateobj(global_State* g, GCObject* o) +{ + /* dead objects can only occur during sweep */ + if (isdead(g, o)) + { + LUAU_ASSERT(g->gcstate == GCSsweepstring || g->gcstate == GCSsweep); + return; + } + + switch (o->gch.tt) + { + case LUA_TSTRING: + break; + + case LUA_TTABLE: + validatetable(g, gco2h(o)); + break; + + case LUA_TFUNCTION: + validateclosure(g, gco2cl(o)); + break; + + case LUA_TUSERDATA: + if (gco2u(o)->metatable) + validateobjref(g, o, obj2gco(gco2u(o)->metatable)); + break; + + case LUA_TTHREAD: + validatestack(g, gco2th(o)); + break; + + case LUA_TPROTO: + validateproto(g, gco2p(o)); + break; + + case LUA_TUPVAL: + validateref(g, o, gco2uv(o)->v); + break; + + default: + LUAU_ASSERT(!"unexpected object type"); + } +} + +static void validatelist(global_State* g, GCObject* o) +{ + while (o) + { + validateobj(g, o); + + o = o->gch.next; + } +} + +static void validategraylist(global_State* g, GCObject* o) +{ + if (!keepinvariant(g)) + return; + + while (o) + { + LUAU_ASSERT(isgray(o)); + + switch (o->gch.tt) + { + case LUA_TTABLE: + o = gco2h(o)->gclist; + break; + case LUA_TFUNCTION: + o = gco2cl(o)->gclist; + break; + case LUA_TTHREAD: + o = gco2th(o)->gclist; + break; + case LUA_TPROTO: + o = gco2p(o)->gclist; + break; + default: + LUAU_ASSERT(!"unknown object in gray list"); + return; + } + } +} + +void luaC_validate(lua_State* L) +{ + global_State* g = L->global; + + LUAU_ASSERT(!isdead(g, obj2gco(g->mainthread))); + checkliveness(g, &g->registry); + + for (int i = 0; i < LUA_T_COUNT; ++i) + if (g->mt[i]) + LUAU_ASSERT(!isdead(g, obj2gco(g->mt[i]))); + + validategraylist(g, g->weak); + validategraylist(g, g->gray); + validategraylist(g, g->grayagain); + + for (int i = 0; i < g->strt.size; ++i) + validatelist(g, g->strt.hash[i]); + + validatelist(g, g->rootgc); + validatelist(g, g->strbufgc); + + for (UpVal* uv = g->uvhead.u.l.next; uv != &g->uvhead; uv = uv->u.l.next) + { + LUAU_ASSERT(uv->tt == LUA_TUPVAL); + LUAU_ASSERT(uv->v != &uv->u.value); + LUAU_ASSERT(uv->u.l.next->u.l.prev == uv && uv->u.l.prev->u.l.next == uv); + } +} + +inline bool safejson(char ch) +{ + return unsigned(ch) < 128 && ch >= 32 && ch != '\\' && ch != '\"'; +} + +static void dumpref(FILE* f, GCObject* o) +{ + fprintf(f, "\"%p\"", o); +} + +static void dumprefs(FILE* f, TValue* data, size_t size) +{ + bool first = true; + + for (size_t i = 0; i < size; ++i) + { + if (iscollectable(&data[i])) + { + if (!first) + fputc(',', f); + first = false; + + dumpref(f, gcvalue(&data[i])); + } + } +} + +static void dumpstringdata(FILE* f, const char* data, size_t len) +{ + for (size_t i = 0; i < len; ++i) + fputc(safejson(data[i]) ? data[i] : '?', f); +} + +static void dumpstring(FILE* f, TString* ts) +{ + fprintf(f, "{\"type\":\"string\",\"cat\":%d,\"size\":%d,\"data\":\"", ts->memcat, int(sizestring(ts->len))); + dumpstringdata(f, ts->data, ts->len); + fprintf(f, "\"}"); +} + +static void dumptable(FILE* f, Table* h) +{ + size_t size = sizeof(Table) + (h->node == &luaH_dummynode ? 0 : sizenode(h) * sizeof(LuaNode)) + h->sizearray * sizeof(TValue); + + fprintf(f, "{\"type\":\"table\",\"cat\":%d,\"size\":%d", h->memcat, int(size)); + + if (h->node != &luaH_dummynode) + { + fprintf(f, ",\"pairs\":["); + + bool first = true; + + for (int i = 0; i < sizenode(h); ++i) + { + const LuaNode& n = h->node[i]; + + if (!ttisnil(&n.val) && (iscollectable(&n.key) || iscollectable(&n.val))) + { + if (!first) + fputc(',', f); + first = false; + + if (iscollectable(&n.key)) + dumpref(f, gcvalue(&n.key)); + else + fprintf(f, "null"); + + fputc(',', f); + + if (iscollectable(&n.val)) + dumpref(f, gcvalue(&n.val)); + else + fprintf(f, "null"); + } + } + + fprintf(f, "]"); + } + if (h->sizearray) + { + fprintf(f, ",\"array\":["); + dumprefs(f, h->array, h->sizearray); + fprintf(f, "]"); + } + if (h->metatable) + { + fprintf(f, ",\"metatable\":"); + dumpref(f, obj2gco(h->metatable)); + } + fprintf(f, "}"); +} + +static void dumpclosure(FILE* f, Closure* cl) +{ + fprintf(f, "{\"type\":\"function\",\"cat\":%d,\"size\":%d", cl->memcat, + cl->isC ? int(sizeCclosure(cl->nupvalues)) : int(sizeLclosure(cl->nupvalues))); + + fprintf(f, ",\"env\":"); + dumpref(f, obj2gco(cl->env)); + if (cl->isC) + { + if (cl->nupvalues) + { + fprintf(f, ",\"upvalues\":["); + dumprefs(f, cl->c.upvals, cl->nupvalues); + fprintf(f, "]"); + } + } + else + { + fprintf(f, ",\"proto\":"); + dumpref(f, obj2gco(cl->l.p)); + if (cl->nupvalues) + { + fprintf(f, ",\"upvalues\":["); + dumprefs(f, cl->l.uprefs, cl->nupvalues); + fprintf(f, "]"); + } + } + fprintf(f, "}"); +} + +static void dumpudata(FILE* f, Udata* u) +{ + fprintf(f, "{\"type\":\"userdata\",\"cat\":%d,\"size\":%d,\"tag\":%d", u->memcat, int(sizeudata(u->len)), u->tag); + + if (u->metatable) + { + fprintf(f, ",\"metatable\":"); + dumpref(f, obj2gco(u->metatable)); + } + fprintf(f, "}"); +} + +static void dumpthread(FILE* f, lua_State* th) +{ + size_t size = sizeof(lua_State) + sizeof(TValue) * th->stacksize + sizeof(CallInfo) * th->size_ci; + + fprintf(f, "{\"type\":\"thread\",\"cat\":%d,\"size\":%d", th->memcat, int(size)); + + if (iscollectable(&th->l_gt)) + { + fprintf(f, ",\"env\":"); + dumpref(f, gcvalue(&th->l_gt)); + } + + Closure* tcl = 0; + for (CallInfo* ci = th->base_ci; ci <= th->ci; ++ci) + { + if (ttisfunction(ci->func)) + { + tcl = clvalue(ci->func); + break; + } + } + + if (tcl && !tcl->isC && tcl->l.p->source) + { + Proto* p = tcl->l.p; + + fprintf(f, ",\"source\":\""); + dumpstringdata(f, p->source->data, p->source->len); + fprintf(f, "\",\"line\":%d", p->abslineinfo ? p->abslineinfo[0] : 0); + } + + if (th->top > th->stack) + { + fprintf(f, ",\"stack\":["); + dumprefs(f, th->stack, th->top - th->stack); + fprintf(f, "]"); + } + fprintf(f, "}"); +} + +static void dumpproto(FILE* f, Proto* p) +{ + size_t size = sizeof(Proto) + sizeof(Instruction) * p->sizecode + sizeof(Proto*) * p->sizep + sizeof(TValue) * p->sizek + p->sizelineinfo + + sizeof(LocVar) * p->sizelocvars + sizeof(TString*) * p->sizeupvalues; + + fprintf(f, "{\"type\":\"proto\",\"cat\":%d,\"size\":%d", p->memcat, int(size)); + + if (p->source) + { + fprintf(f, ",\"source\":\""); + dumpstringdata(f, p->source->data, p->source->len); + fprintf(f, "\",\"line\":%d", p->abslineinfo ? p->abslineinfo[0] : 0); + } + + if (p->sizek) + { + fprintf(f, ",\"constants\":["); + dumprefs(f, p->k, p->sizek); + fprintf(f, "]"); + } + + if (p->sizep) + { + fprintf(f, ",\"protos\":["); + for (int i = 0; i < p->sizep; ++i) + { + if (i != 0) + fputc(',', f); + dumpref(f, obj2gco(p->p[i])); + } + fprintf(f, "]"); + } + + fprintf(f, "}"); +} + +static void dumpupval(FILE* f, UpVal* uv) +{ + fprintf(f, "{\"type\":\"upvalue\",\"cat\":%d,\"size\":%d", uv->memcat, int(sizeof(UpVal))); + + if (iscollectable(uv->v)) + { + fprintf(f, ",\"object\":"); + dumpref(f, gcvalue(uv->v)); + } + fprintf(f, "}"); +} + +static void dumpobj(FILE* f, GCObject* o) +{ + switch (o->gch.tt) + { + case LUA_TSTRING: + return dumpstring(f, gco2ts(o)); + + case LUA_TTABLE: + return dumptable(f, gco2h(o)); + + case LUA_TFUNCTION: + return dumpclosure(f, gco2cl(o)); + + case LUA_TUSERDATA: + return dumpudata(f, gco2u(o)); + + case LUA_TTHREAD: + return dumpthread(f, gco2th(o)); + + case LUA_TPROTO: + return dumpproto(f, gco2p(o)); + + case LUA_TUPVAL: + return dumpupval(f, gco2uv(o)); + + default: + LUAU_ASSERT(0); + } +} + +static void dumplist(FILE* f, GCObject* o) +{ + while (o) + { + dumpref(f, o); + fputc(':', f); + dumpobj(f, o); + fputc(',', f); + fputc('\n', f); + + // thread has additional list containing collectable objects that are not present in rootgc + if (o->gch.tt == LUA_TTHREAD) + dumplist(f, gco2th(o)->openupval); + + o = o->gch.next; + } +} + +void luaC_dump(lua_State* L, void* file, const char* (*categoryName)(lua_State* L, uint8_t memcat)) +{ + global_State* g = L->global; + FILE* f = static_cast(file); + + fprintf(f, "{\"objects\":{\n"); + dumplist(f, g->rootgc); + dumplist(f, g->strbufgc); + for (int i = 0; i < g->strt.size; ++i) + dumplist(f, g->strt.hash[i]); + + fprintf(f, "\"0\":{\"type\":\"userdata\",\"cat\":0,\"size\":0}\n"); // to avoid issues with trailing , + fprintf(f, "},\"roots\":{\n"); + fprintf(f, "\"mainthread\":"); + dumpref(f, obj2gco(g->mainthread)); + fprintf(f, ",\"registry\":"); + dumpref(f, gcvalue(&g->registry)); + + fprintf(f, "},\"stats\":{\n"); + + fprintf(f, "\"size\":%d,\n", int(g->totalbytes)); + + fprintf(f, "\"categories\":{\n"); + for (int i = 0; i < LUA_MEMORY_CATEGORIES; i++) + { + if (size_t bytes = g->memcatbytes[i]) + { + if (categoryName) + fprintf(f, "\"%d\":{\"name\":\"%s\", \"size\":%d},\n", i, categoryName(L, i), int(bytes)); + else + fprintf(f, "\"%d\":{\"size\":%d},\n", i, int(bytes)); + } + } + fprintf(f, "\"none\":{}\n"); // to avoid issues with trailing , + fprintf(f, "}\n"); + fprintf(f, "}}\n"); +} diff --git a/VM/src/linit.cpp b/VM/src/linit.cpp index 4e40165a..c93f431f 100644 --- a/VM/src/linit.cpp +++ b/VM/src/linit.cpp @@ -17,7 +17,7 @@ static const luaL_Reg lualibs[] = { {NULL, NULL}, }; -LUALIB_API void luaL_openlibs(lua_State* L) +void luaL_openlibs(lua_State* L) { const luaL_Reg* lib = lualibs; for (; lib->func; lib++) @@ -28,7 +28,7 @@ LUALIB_API void luaL_openlibs(lua_State* L) } } -LUALIB_API void luaL_sandbox(lua_State* L) +void luaL_sandbox(lua_State* L) { // set all libraries to read-only lua_pushnil(L); @@ -44,14 +44,14 @@ LUALIB_API void luaL_sandbox(lua_State* L) lua_pushliteral(L, ""); lua_getmetatable(L, -1); lua_setreadonly(L, -1, true); - lua_pop(L, 1); + lua_pop(L, 2); // set globals to readonly and activate safeenv since the env is immutable lua_setreadonly(L, LUA_GLOBALSINDEX, true); lua_setsafeenv(L, LUA_GLOBALSINDEX, true); } -LUALIB_API void luaL_sandboxthread(lua_State* L) +void luaL_sandboxthread(lua_State* L) { // create new global table that proxies reads to original table lua_newtable(L); @@ -81,7 +81,7 @@ static void* l_alloc(lua_State* L, void* ud, void* ptr, size_t osize, size_t nsi return realloc(ptr, nsize); } -LUALIB_API lua_State* luaL_newstate(void) +lua_State* luaL_newstate(void) { return lua_newstate(l_alloc, NULL); } diff --git a/VM/src/lmathlib.cpp b/VM/src/lmathlib.cpp index 8e476a52..a6e7b494 100644 --- a/VM/src/lmathlib.cpp +++ b/VM/src/lmathlib.cpp @@ -385,8 +385,7 @@ static int math_sign(lua_State* L) static int math_round(lua_State* L) { - double v = luaL_checknumber(L, 1); - lua_pushnumber(L, round(v)); + lua_pushnumber(L, round(luaL_checknumber(L, 1))); return 1; } @@ -429,7 +428,7 @@ static const luaL_Reg mathlib[] = { /* ** Open math library */ -LUALIB_API int luaopen_math(lua_State* L) +int luaopen_math(lua_State* L) { uint64_t seed = uintptr_t(L); seed ^= time(NULL); diff --git a/VM/src/lmem.cpp b/VM/src/lmem.cpp index d8b265cb..9f9d4a98 100644 --- a/VM/src/lmem.cpp +++ b/VM/src/lmem.cpp @@ -33,11 +33,17 @@ #define ABISWITCH(x64, ms32, gcc32) (sizeof(void*) == 8 ? x64 : ms32) #endif +#if LUA_VECTOR_SIZE == 4 +static_assert(sizeof(TValue) == ABISWITCH(24, 24, 24), "size mismatch for value"); +static_assert(sizeof(LuaNode) == ABISWITCH(48, 48, 48), "size mismatch for table entry"); +#else static_assert(sizeof(TValue) == ABISWITCH(16, 16, 16), "size mismatch for value"); +static_assert(sizeof(LuaNode) == ABISWITCH(32, 32, 32), "size mismatch for table entry"); +#endif + static_assert(offsetof(TString, data) == ABISWITCH(24, 20, 20), "size mismatch for string header"); static_assert(offsetof(Udata, data) == ABISWITCH(24, 16, 16), "size mismatch for userdata header"); static_assert(sizeof(Table) == ABISWITCH(56, 36, 36), "size mismatch for table header"); -static_assert(sizeof(LuaNode) == ABISWITCH(32, 32, 32), "size mismatch for table entry"); const size_t kSizeClasses = LUA_SIZECLASSES; const size_t kMaxSmallSize = 512; diff --git a/VM/src/lnumutils.h b/VM/src/lnumutils.h index 43f8014b..67f832dc 100644 --- a/VM/src/lnumutils.h +++ b/VM/src/lnumutils.h @@ -18,12 +18,20 @@ inline bool luai_veceq(const float* a, const float* b) { +#if LUA_VECTOR_SIZE == 4 + return a[0] == b[0] && a[1] == b[1] && a[2] == b[2] && a[3] == b[3]; +#else return a[0] == b[0] && a[1] == b[1] && a[2] == b[2]; +#endif } inline bool luai_vecisnan(const float* a) { +#if LUA_VECTOR_SIZE == 4 + return a[0] != a[0] || a[1] != a[1] || a[2] != a[2] || a[3] != a[3]; +#else return a[0] != a[0] || a[1] != a[1] || a[2] != a[2]; +#endif } LUAU_FASTMATH_BEGIN diff --git a/VM/src/lobject.cpp b/VM/src/lobject.cpp index bf13e6e9..370c7b28 100644 --- a/VM/src/lobject.cpp +++ b/VM/src/lobject.cpp @@ -15,7 +15,7 @@ -const TValue luaO_nilobject_ = {{NULL}, LUA_TNIL}; +const TValue luaO_nilobject_ = {{NULL}, {0}, LUA_TNIL}; int luaO_log2(unsigned int x) { diff --git a/VM/src/lobject.h b/VM/src/lobject.h index c5f2e2f4..ba040af6 100644 --- a/VM/src/lobject.h +++ b/VM/src/lobject.h @@ -47,7 +47,7 @@ typedef union typedef struct lua_TValue { Value value; - int extra; + int extra[LUA_EXTRA_SIZE]; int tt; } TValue; @@ -105,7 +105,19 @@ typedef struct lua_TValue i_o->tt = LUA_TNUMBER; \ } -#define setvvalue(obj, x, y, z) \ +#if LUA_VECTOR_SIZE == 4 +#define setvvalue(obj, x, y, z, w) \ + { \ + TValue* i_o = (obj); \ + float* i_v = i_o->value.v; \ + i_v[0] = (x); \ + i_v[1] = (y); \ + i_v[2] = (z); \ + i_v[3] = (w); \ + i_o->tt = LUA_TVECTOR; \ + } +#else +#define setvvalue(obj, x, y, z, w) \ { \ TValue* i_o = (obj); \ float* i_v = i_o->value.v; \ @@ -114,6 +126,7 @@ typedef struct lua_TValue i_v[2] = (z); \ i_o->tt = LUA_TVECTOR; \ } +#endif #define setpvalue(obj, x) \ { \ @@ -364,7 +377,7 @@ typedef struct Closure typedef struct TKey { ::Value value; - int extra; + int extra[LUA_EXTRA_SIZE]; unsigned tt : 4; int next : 28; /* for chaining */ } TKey; @@ -381,7 +394,7 @@ typedef struct LuaNode LuaNode* n_ = (node); \ const TValue* i_o = (obj); \ n_->key.value = i_o->value; \ - n_->key.extra = i_o->extra; \ + memcpy(n_->key.extra, i_o->extra, sizeof(n_->key.extra)); \ n_->key.tt = i_o->tt; \ checkliveness(L->global, i_o); \ } @@ -392,7 +405,7 @@ typedef struct LuaNode TValue* i_o = (obj); \ const LuaNode* n_ = (node); \ i_o->value = n_->key.value; \ - i_o->extra = n_->key.extra; \ + memcpy(i_o->extra, n_->key.extra, sizeof(i_o->extra)); \ i_o->tt = n_->key.tt; \ checkliveness(L->global, i_o); \ } diff --git a/VM/src/loslib.cpp b/VM/src/loslib.cpp index 8eaef60c..b5901865 100644 --- a/VM/src/loslib.cpp +++ b/VM/src/loslib.cpp @@ -186,7 +186,7 @@ static const luaL_Reg syslib[] = { {NULL, NULL}, }; -LUALIB_API int luaopen_os(lua_State* L) +int luaopen_os(lua_State* L) { luaL_register(L, LUA_OSLIBNAME, syslib); return 1; diff --git a/VM/src/lstrlib.cpp b/VM/src/lstrlib.cpp index b576f809..0b3054ae 100644 --- a/VM/src/lstrlib.cpp +++ b/VM/src/lstrlib.cpp @@ -1657,7 +1657,7 @@ static void createmetatable(lua_State* L) /* ** Open string library */ -LUALIB_API int luaopen_string(lua_State* L) +int luaopen_string(lua_State* L) { luaL_register(L, LUA_STRLIBNAME, strlib); createmetatable(L); diff --git a/VM/src/ltable.cpp b/VM/src/ltable.cpp index 07d22d59..0b55fcea 100644 --- a/VM/src/ltable.cpp +++ b/VM/src/ltable.cpp @@ -31,18 +31,19 @@ LUAU_FASTFLAGVARIABLE(LuauArrayBoundary, false) #define MAXSIZE (1 << MAXBITS) static_assert(offsetof(LuaNode, val) == 0, "Unexpected Node memory layout, pointer cast in gval2slot is incorrect"); + // TKey is bitpacked for memory efficiency so we need to validate bit counts for worst case -static_assert(TKey{{NULL}, 0, LUA_TDEADKEY, 0}.tt == LUA_TDEADKEY, "not enough bits for tt"); -static_assert(TKey{{NULL}, 0, LUA_TNIL, MAXSIZE - 1}.next == MAXSIZE - 1, "not enough bits for next"); -static_assert(TKey{{NULL}, 0, LUA_TNIL, -(MAXSIZE - 1)}.next == -(MAXSIZE - 1), "not enough bits for next"); +static_assert(TKey{{NULL}, {0}, LUA_TDEADKEY, 0}.tt == LUA_TDEADKEY, "not enough bits for tt"); +static_assert(TKey{{NULL}, {0}, LUA_TNIL, MAXSIZE - 1}.next == MAXSIZE - 1, "not enough bits for next"); +static_assert(TKey{{NULL}, {0}, LUA_TNIL, -(MAXSIZE - 1)}.next == -(MAXSIZE - 1), "not enough bits for next"); // reset cache of absent metamethods, cache is updated in luaT_gettm #define invalidateTMcache(t) t->flags = 0 // empty hash data points to dummynode so that we can always dereference it const LuaNode luaH_dummynode = { - {{NULL}, 0, LUA_TNIL}, /* value */ - {{NULL}, 0, LUA_TNIL, 0} /* key */ + {{NULL}, {0}, LUA_TNIL}, /* value */ + {{NULL}, {0}, LUA_TNIL, 0} /* key */ }; #define dummynode (&luaH_dummynode) @@ -96,7 +97,7 @@ static LuaNode* hashnum(const Table* t, double n) static LuaNode* hashvec(const Table* t, const float* v) { - unsigned int i[3]; + unsigned int i[LUA_VECTOR_SIZE]; memcpy(i, v, sizeof(i)); // convert -0 to 0 to make sure they hash to the same value @@ -112,6 +113,12 @@ static LuaNode* hashvec(const Table* t, const float* v) // Optimized Spatial Hashing for Collision Detection of Deformable Objects unsigned int h = (i[0] * 73856093) ^ (i[1] * 19349663) ^ (i[2] * 83492791); +#if LUA_VECTOR_SIZE == 4 + i[3] = (i[3] == 0x8000000) ? 0 : i[3]; + i[3] ^= i[3] >> 17; + h ^= i[3] * 39916801; +#endif + return hashpow2(t, h); } diff --git a/VM/src/ltablib.cpp b/VM/src/ltablib.cpp index 37025818..0d3374ef 100644 --- a/VM/src/ltablib.cpp +++ b/VM/src/ltablib.cpp @@ -527,7 +527,7 @@ static const luaL_Reg tab_funcs[] = { {NULL, NULL}, }; -LUALIB_API int luaopen_table(lua_State* L) +int luaopen_table(lua_State* L) { luaL_register(L, LUA_TABLIBNAME, tab_funcs); diff --git a/VM/src/lutf8lib.cpp b/VM/src/lutf8lib.cpp index 378de3d0..8bc8200a 100644 --- a/VM/src/lutf8lib.cpp +++ b/VM/src/lutf8lib.cpp @@ -283,7 +283,7 @@ static const luaL_Reg funcs[] = { {NULL, NULL}, }; -LUALIB_API int luaopen_utf8(lua_State* L) +int luaopen_utf8(lua_State* L) { luaL_register(L, LUA_UTF8LIBNAME, funcs); diff --git a/VM/src/lvmexecute.cpp b/VM/src/lvmexecute.cpp index eed2862b..bf8d493e 100644 --- a/VM/src/lvmexecute.cpp +++ b/VM/src/lvmexecute.cpp @@ -601,7 +601,13 @@ static void luau_execute(lua_State* L) const char* name = getstr(tsvalue(kv)); int ic = (name[0] | ' ') - 'x'; - if (unsigned(ic) < 3 && name[1] == '\0') +#if LUA_VECTOR_SIZE == 4 + // 'w' is before 'x' in ascii, so ic is -1 when indexing with 'w' + if (ic == -1) + ic = 3; +#endif + + if (unsigned(ic) < LUA_VECTOR_SIZE && name[1] == '\0') { setnvalue(ra, rb->value.v[ic]); VM_NEXT(); @@ -1526,7 +1532,7 @@ static void luau_execute(lua_State* L) { const float* vb = rb->value.v; const float* vc = rc->value.v; - setvvalue(ra, vb[0] + vc[0], vb[1] + vc[1], vb[2] + vc[2]); + setvvalue(ra, vb[0] + vc[0], vb[1] + vc[1], vb[2] + vc[2], vb[3] + vc[3]); VM_NEXT(); } else @@ -1572,7 +1578,7 @@ static void luau_execute(lua_State* L) { const float* vb = rb->value.v; const float* vc = rc->value.v; - setvvalue(ra, vb[0] - vc[0], vb[1] - vc[1], vb[2] - vc[2]); + setvvalue(ra, vb[0] - vc[0], vb[1] - vc[1], vb[2] - vc[2], vb[3] - vc[3]); VM_NEXT(); } else @@ -1618,21 +1624,21 @@ static void luau_execute(lua_State* L) { const float* vb = rb->value.v; float vc = cast_to(float, nvalue(rc)); - setvvalue(ra, vb[0] * vc, vb[1] * vc, vb[2] * vc); + setvvalue(ra, vb[0] * vc, vb[1] * vc, vb[2] * vc, vb[3] * vc); VM_NEXT(); } else if (ttisvector(rb) && ttisvector(rc)) { const float* vb = rb->value.v; const float* vc = rc->value.v; - setvvalue(ra, vb[0] * vc[0], vb[1] * vc[1], vb[2] * vc[2]); + setvvalue(ra, vb[0] * vc[0], vb[1] * vc[1], vb[2] * vc[2], vb[3] * vc[3]); VM_NEXT(); } else if (ttisnumber(rb) && ttisvector(rc)) { float vb = cast_to(float, nvalue(rb)); const float* vc = rc->value.v; - setvvalue(ra, vb * vc[0], vb * vc[1], vb * vc[2]); + setvvalue(ra, vb * vc[0], vb * vc[1], vb * vc[2], vb * vc[3]); VM_NEXT(); } else @@ -1679,21 +1685,21 @@ static void luau_execute(lua_State* L) { const float* vb = rb->value.v; float vc = cast_to(float, nvalue(rc)); - setvvalue(ra, vb[0] / vc, vb[1] / vc, vb[2] / vc); + setvvalue(ra, vb[0] / vc, vb[1] / vc, vb[2] / vc, vb[3] / vc); VM_NEXT(); } else if (ttisvector(rb) && ttisvector(rc)) { const float* vb = rb->value.v; const float* vc = rc->value.v; - setvvalue(ra, vb[0] / vc[0], vb[1] / vc[1], vb[2] / vc[2]); + setvvalue(ra, vb[0] / vc[0], vb[1] / vc[1], vb[2] / vc[2], vb[3] / vc[3]); VM_NEXT(); } else if (ttisnumber(rb) && ttisvector(rc)) { float vb = cast_to(float, nvalue(rb)); const float* vc = rc->value.v; - setvvalue(ra, vb / vc[0], vb / vc[1], vb / vc[2]); + setvvalue(ra, vb / vc[0], vb / vc[1], vb / vc[2], vb / vc[3]); VM_NEXT(); } else @@ -1826,7 +1832,7 @@ static void luau_execute(lua_State* L) { const float* vb = rb->value.v; float vc = cast_to(float, nvalue(kv)); - setvvalue(ra, vb[0] * vc, vb[1] * vc, vb[2] * vc); + setvvalue(ra, vb[0] * vc, vb[1] * vc, vb[2] * vc, vb[3] * vc); VM_NEXT(); } else @@ -1872,7 +1878,7 @@ static void luau_execute(lua_State* L) { const float* vb = rb->value.v; float vc = cast_to(float, nvalue(kv)); - setvvalue(ra, vb[0] / vc, vb[1] / vc, vb[2] / vc); + setvvalue(ra, vb[0] / vc, vb[1] / vc, vb[2] / vc, vb[3] / vc); VM_NEXT(); } else @@ -2037,7 +2043,7 @@ static void luau_execute(lua_State* L) else if (ttisvector(rb)) { const float* vb = rb->value.v; - setvvalue(ra, -vb[0], -vb[1], -vb[2]); + setvvalue(ra, -vb[0], -vb[1], -vb[2], -vb[3]); VM_NEXT(); } else diff --git a/VM/src/lvmload.cpp b/VM/src/lvmload.cpp index a168b652..add3588d 100644 --- a/VM/src/lvmload.cpp +++ b/VM/src/lvmload.cpp @@ -9,6 +9,7 @@ #include "lgc.h" #include "lmem.h" #include "lbytecode.h" +#include "lapi.h" #include @@ -162,9 +163,8 @@ int luau_load(lua_State* L, const char* chunkname, const char* data, size_t size size_t GCthreshold = L->global->GCthreshold; L->global->GCthreshold = SIZE_MAX; - // env is 0 for current environment and a stack relative index otherwise - LUAU_ASSERT(env <= 0 && L->top - L->base >= -env); - Table* envt = (env == 0) ? hvalue(gt(L)) : hvalue(L->top + env); + // env is 0 for current environment and a stack index otherwise + Table* envt = (env == 0) ? hvalue(gt(L)) : hvalue(luaA_toobject(L, env)); TString* source = luaS_new(L, chunkname); diff --git a/VM/src/lvmutils.cpp b/VM/src/lvmutils.cpp index f52e8e74..740a4cfd 100644 --- a/VM/src/lvmutils.cpp +++ b/VM/src/lvmutils.cpp @@ -401,19 +401,19 @@ void luaV_doarith(lua_State* L, StkId ra, const TValue* rb, const TValue* rc, TM switch (op) { case TM_ADD: - setvvalue(ra, vb[0] + vc[0], vb[1] + vc[1], vb[2] + vc[2]); + setvvalue(ra, vb[0] + vc[0], vb[1] + vc[1], vb[2] + vc[2], vb[3] + vc[3]); return; case TM_SUB: - setvvalue(ra, vb[0] - vc[0], vb[1] - vc[1], vb[2] - vc[2]); + setvvalue(ra, vb[0] - vc[0], vb[1] - vc[1], vb[2] - vc[2], vb[3] - vc[3]); return; case TM_MUL: - setvvalue(ra, vb[0] * vc[0], vb[1] * vc[1], vb[2] * vc[2]); + setvvalue(ra, vb[0] * vc[0], vb[1] * vc[1], vb[2] * vc[2], vb[3] * vc[3]); return; case TM_DIV: - setvvalue(ra, vb[0] / vc[0], vb[1] / vc[1], vb[2] / vc[2]); + setvvalue(ra, vb[0] / vc[0], vb[1] / vc[1], vb[2] / vc[2], vb[3] / vc[3]); return; case TM_UNM: - setvvalue(ra, -vb[0], -vb[1], -vb[2]); + setvvalue(ra, -vb[0], -vb[1], -vb[2], -vb[3]); return; default: break; @@ -430,10 +430,10 @@ void luaV_doarith(lua_State* L, StkId ra, const TValue* rb, const TValue* rc, TM switch (op) { case TM_MUL: - setvvalue(ra, vb[0] * nc, vb[1] * nc, vb[2] * nc); + setvvalue(ra, vb[0] * nc, vb[1] * nc, vb[2] * nc, vb[3] * nc); return; case TM_DIV: - setvvalue(ra, vb[0] / nc, vb[1] / nc, vb[2] / nc); + setvvalue(ra, vb[0] / nc, vb[1] / nc, vb[2] / nc, vb[3] / nc); return; default: break; @@ -451,10 +451,10 @@ void luaV_doarith(lua_State* L, StkId ra, const TValue* rb, const TValue* rc, TM switch (op) { case TM_MUL: - setvvalue(ra, nb * vc[0], nb * vc[1], nb * vc[2]); + setvvalue(ra, nb * vc[0], nb * vc[1], nb * vc[2], nb * vc[3]); return; case TM_DIV: - setvvalue(ra, nb / vc[0], nb / vc[1], nb / vc[2]); + setvvalue(ra, nb / vc[0], nb / vc[1], nb / vc[2], nb / vc[3]); return; default: break; diff --git a/bench/gc/test_LB_mandel.lua b/bench/gc/test_LB_mandel.lua index 4be78502..fe5b4eb2 100644 --- a/bench/gc/test_LB_mandel.lua +++ b/bench/gc/test_LB_mandel.lua @@ -88,7 +88,7 @@ for i=1,N do local y=ymin+(j-1)*dy S = S + level(x,y) end - -- if i % 10 == 0 then print(collectgarbage"count") end + -- if i % 10 == 0 then print(collectgarbage("count")) end end print(S) diff --git a/bench/tests/shootout/mandel.lua b/bench/tests/shootout/mandel.lua index 4be78502..fe5b4eb2 100644 --- a/bench/tests/shootout/mandel.lua +++ b/bench/tests/shootout/mandel.lua @@ -88,7 +88,7 @@ for i=1,N do local y=ymin+(j-1)*dy S = S + level(x,y) end - -- if i % 10 == 0 then print(collectgarbage"count") end + -- if i % 10 == 0 then print(collectgarbage("count")) end end print(S) diff --git a/bench/tests/shootout/qt.lua b/bench/tests/shootout/qt.lua index de962a74..79cbe38b 100644 --- a/bench/tests/shootout/qt.lua +++ b/bench/tests/shootout/qt.lua @@ -275,7 +275,7 @@ local function memory(s) local t=os.clock() --local dt=string.format("%f",t-t0) local dt=t-t0 - --io.stdout:write(s,"\t",dt," sec\t",t," sec\t",math.floor(collectgarbage"count"/1024),"M\n") + --io.stdout:write(s,"\t",dt," sec\t",t," sec\t",math.floor(collectgarbage("count")/1024),"M\n") t0=t end @@ -286,7 +286,7 @@ local function do_(f,s) end local function julia(l,a,b) -memory"begin" +memory("begin") cx=a cy=b root=newcell() exterior=newcell() exterior.color=white @@ -297,14 +297,14 @@ memory"begin" do_(update,"update") repeat N=0 color(root,Rxmin,Rxmax,Rymin,Rymax) --print("color",N) - until N==0 memory"color" + until N==0 memory("color") repeat N=0 prewhite(root,Rxmin,Rxmax,Rymin,Rymax) --print("prewhite",N) - until N==0 memory"prewhite" + until N==0 memory("prewhite") do_(recolor,"recolor") do_(colorup,"colorup") --print("colorup",N) local g,b=do_(area,"area") --print("area",g,b,g+b) - show(i) memory"output" + show(i) memory("output") --print("edges",nE) end end diff --git a/fuzz/proto.cpp b/fuzz/proto.cpp index ae2399e4..27e53492 100644 --- a/fuzz/proto.cpp +++ b/fuzz/proto.cpp @@ -23,9 +23,11 @@ const bool kFuzzCompiler = true; const bool kFuzzLinter = true; const bool kFuzzTypeck = true; const bool kFuzzVM = true; -const bool kFuzzTypes = true; const bool kFuzzTranspile = true; +// Should we generate type annotations? +const bool kFuzzTypes = true; + static_assert(!(kFuzzVM && !kFuzzCompiler), "VM requires the compiler!"); std::string protoprint(const luau::StatBlock& stat, bool types); diff --git a/tests/AstQuery.test.cpp b/tests/AstQuery.test.cpp index aa53a92b..2090b014 100644 --- a/tests/AstQuery.test.cpp +++ b/tests/AstQuery.test.cpp @@ -78,3 +78,26 @@ TEST_CASE_FIXTURE(DocumentationSymbolFixture, "overloaded_fn") } TEST_SUITE_END(); + +TEST_SUITE_BEGIN("AstQuery"); + +TEST_CASE_FIXTURE(Fixture, "last_argument_function_call_type") +{ + ScopedFastFlag luauTailArgumentTypeInfo{"LuauTailArgumentTypeInfo", true}; + + check(R"( +local function foo() return 2 end +local function bar(a: number) return -a end +bar(foo()) + )"); + + auto oty = findTypeAtPosition(Position(3, 7)); + REQUIRE(oty); + CHECK_EQ("number", toString(*oty)); + + auto expectedOty = findExpectedTypeAtPosition(Position(3, 7)); + REQUIRE(expectedOty); + CHECK_EQ("number", toString(*expectedOty)); +} + +TEST_SUITE_END(); diff --git a/tests/Autocomplete.test.cpp b/tests/Autocomplete.test.cpp index 5a7c8602..3b74a99e 100644 --- a/tests/Autocomplete.test.cpp +++ b/tests/Autocomplete.test.cpp @@ -1935,6 +1935,39 @@ return target(b@1 CHECK(ac.entryMap["bar2"].typeCorrect == TypeCorrectKind::None); } +TEST_CASE_FIXTURE(ACFixture, "function_in_assignment_has_parentheses") +{ + ScopedFastFlag luauAutocompleteAvoidMutation("LuauAutocompleteAvoidMutation", true); + ScopedFastFlag luauAutocompletePreferToCallFunctions("LuauAutocompletePreferToCallFunctions", true); + + check(R"( +local function bar(a: number) return -a end +local abc = b@1 + )"); + + auto ac = autocomplete('1'); + + CHECK(ac.entryMap.count("bar")); + CHECK(ac.entryMap["bar"].parens == ParenthesesRecommendation::CursorInside); +} + +TEST_CASE_FIXTURE(ACFixture, "function_result_passed_to_function_has_parentheses") +{ + ScopedFastFlag luauAutocompleteAvoidMutation("LuauAutocompleteAvoidMutation", true); + ScopedFastFlag luauAutocompletePreferToCallFunctions("LuauAutocompletePreferToCallFunctions", true); + + check(R"( +local function foo() return 1 end +local function bar(a: number) return -a end +local abc = bar(@1) + )"); + + auto ac = autocomplete('1'); + + CHECK(ac.entryMap.count("foo")); + CHECK(ac.entryMap["foo"].parens == ParenthesesRecommendation::CursorAfter); +} + TEST_CASE_FIXTURE(ACFixture, "type_correct_sealed_table") { check(R"( @@ -2210,8 +2243,6 @@ TEST_CASE_FIXTURE(ACFixture, "autocompleteSource") TEST_CASE_FIXTURE(ACFixture, "autocompleteSource_require") { - ScopedFastFlag luauResolveModuleNameWithoutACurrentModule("LuauResolveModuleNameWithoutACurrentModule", true); - std::string_view source = R"( local a = require(w -- Line 1 -- | Column 27 @@ -2287,8 +2318,6 @@ until TEST_CASE_FIXTURE(ACFixture, "if_then_else_elseif_completions") { - ScopedFastFlag sff{"ElseElseIfCompletionImprovements", true}; - check(R"( local elsewhere = false @@ -2585,9 +2614,6 @@ a = if temp then even elseif true then temp else e@9 TEST_CASE_FIXTURE(ACFixture, "autocomplete_explicit_type_pack") { - ScopedFastFlag luauTypeAliasPacks("LuauTypeAliasPacks", true); - ScopedFastFlag luauParseTypePackTypeParameters("LuauParseTypePackTypeParameters", true); - check(R"( type A = () -> T... local a: A<(number, s@1> diff --git a/tests/Compiler.test.cpp b/tests/Compiler.test.cpp index 4ce8d08a..6ba39ada 100644 --- a/tests/Compiler.test.cpp +++ b/tests/Compiler.test.cpp @@ -1057,6 +1057,18 @@ RETURN R0 1 CHECK_EQ("\n" + compileFunction0("return if false then 10 else 20"), R"( LOADN R0 20 RETURN R0 1 +)"); + + // codegen for a true constant condition with non-constant expressions + CHECK_EQ("\n" + compileFunction0("return if true then {} else error()"), R"( +NEWTABLE R0 0 0 +RETURN R0 1 +)"); + + // codegen for a false constant condition with non-constant expressions + CHECK_EQ("\n" + compileFunction0("return if false then error() else {}"), R"( +NEWTABLE R0 0 0 +RETURN R0 1 )"); // codegen for a false (in this case 'nil') constant condition @@ -2360,6 +2372,58 @@ Foo:Bar( )"); } +TEST_CASE("DebugLineInfoCallChain") +{ + Luau::BytecodeBuilder bcb; + bcb.setDumpFlags(Luau::BytecodeBuilder::Dump_Code | Luau::BytecodeBuilder::Dump_Lines); + Luau::compileOrThrow(bcb, R"( +local Foo = ... + +Foo +:Bar(1) +:Baz(2) +.Qux(3) +)"); + + CHECK_EQ("\n" + bcb.dumpFunction(0), R"( +2: GETVARARGS R0 1 +5: LOADN R4 1 +5: NAMECALL R2 R0 K0 +5: CALL R2 2 1 +6: LOADN R4 2 +6: NAMECALL R2 R2 K1 +6: CALL R2 2 1 +7: GETTABLEKS R1 R2 K2 +7: LOADN R2 3 +7: CALL R1 1 0 +8: RETURN R0 0 +)"); +} + +TEST_CASE("DebugLineInfoFastCall") +{ + Luau::BytecodeBuilder bcb; + bcb.setDumpFlags(Luau::BytecodeBuilder::Dump_Code | Luau::BytecodeBuilder::Dump_Lines); + Luau::compileOrThrow(bcb, R"( +local Foo, Bar = ... + +return + math.max( + Foo, + Bar) +)"); + + CHECK_EQ("\n" + bcb.dumpFunction(0), R"( +2: GETVARARGS R0 2 +5: FASTCALL2 18 R0 R1 +5 +5: MOVE R3 R0 +5: MOVE R4 R1 +5: GETIMPORT R2 2 +5: CALL R2 2 -1 +5: RETURN R2 -1 +)"); +} + TEST_CASE("DebugSource") { const char* source = R"( @@ -3742,4 +3806,108 @@ RETURN R0 0 )"); } +TEST_CASE("ConstantsNoFolding") +{ + const char* source = "return nil, true, 42, 'hello'"; + + Luau::BytecodeBuilder bcb; + bcb.setDumpFlags(Luau::BytecodeBuilder::Dump_Code); + Luau::CompileOptions options; + options.optimizationLevel = 0; + Luau::compileOrThrow(bcb, source, options); + + CHECK_EQ("\n" + bcb.dumpFunction(0), R"( +LOADNIL R0 +LOADB R1 1 +LOADK R2 K0 +LOADK R3 K1 +RETURN R0 4 +)"); +} + +TEST_CASE("VectorFastCall") +{ + const char* source = "return Vector3.new(1, 2, 3)"; + + Luau::BytecodeBuilder bcb; + bcb.setDumpFlags(Luau::BytecodeBuilder::Dump_Code); + Luau::CompileOptions options; + options.vectorLib = "Vector3"; + options.vectorCtor = "new"; + Luau::compileOrThrow(bcb, source, options); + + CHECK_EQ("\n" + bcb.dumpFunction(0), R"( +LOADN R1 1 +LOADN R2 2 +LOADN R3 3 +FASTCALL 54 +2 +GETIMPORT R0 2 +CALL R0 3 -1 +RETURN R0 -1 +)"); +} + +TEST_CASE("TypeAssertion") +{ + // validate that type assertions work with the compiler and that the code inside type assertion isn't evaluated + CHECK_EQ("\n" + compileFunction0(R"( +print(foo() :: typeof(error("compile time"))) +)"), + R"( +GETIMPORT R0 1 +GETIMPORT R1 3 +CALL R1 0 1 +CALL R0 1 0 +RETURN R0 0 +)"); + + // note that above, foo() is treated as single-arg function; removing type assertion changes the bytecode + CHECK_EQ("\n" + compileFunction0(R"( +print(foo()) +)"), + R"( +GETIMPORT R0 1 +GETIMPORT R1 3 +CALL R1 0 -1 +CALL R0 -1 0 +RETURN R0 0 +)"); +} + +TEST_CASE("Arithmetics") +{ + // basic arithmetics codegen with non-constants + CHECK_EQ("\n" + compileFunction0(R"( +local a, b = ... +return a + b, a - b, a / b, a * b, a % b, a ^ b +)"), + R"( +GETVARARGS R0 2 +ADD R2 R0 R1 +SUB R3 R0 R1 +DIV R4 R0 R1 +MUL R5 R0 R1 +MOD R6 R0 R1 +POW R7 R0 R1 +RETURN R2 6 +)"); + + // basic arithmetics codegen with constants on the right side + // note that we don't simplify these expressions as we don't know the type of a + CHECK_EQ("\n" + compileFunction0(R"( +local a = ... +return a + 1, a - 1, a / 1, a * 1, a % 1, a ^ 1 +)"), + R"( +GETVARARGS R0 1 +ADDK R1 R0 K0 +SUBK R2 R0 K0 +DIVK R3 R0 K0 +MULK R4 R0 K0 +MODK R5 R0 K0 +POWK R6 R0 K0 +RETURN R1 6 +)"); +} + TEST_SUITE_END(); diff --git a/tests/Conformance.test.cpp b/tests/Conformance.test.cpp index e495a213..b2aad316 100644 --- a/tests/Conformance.test.cpp +++ b/tests/Conformance.test.cpp @@ -67,44 +67,42 @@ static int lua_vector(lua_State* L) double y = luaL_checknumber(L, 2); double z = luaL_checknumber(L, 3); +#if LUA_VECTOR_SIZE == 4 + double w = luaL_optnumber(L, 4, 0.0); + lua_pushvector(L, float(x), float(y), float(z), float(w)); +#else lua_pushvector(L, float(x), float(y), float(z)); +#endif return 1; } static int lua_vector_dot(lua_State* L) { - const float* a = lua_tovector(L, 1); - const float* b = lua_tovector(L, 2); + const float* a = luaL_checkvector(L, 1); + const float* b = luaL_checkvector(L, 2); - if (a && b) - { - lua_pushnumber(L, a[0] * b[0] + a[1] * b[1] + a[2] * b[2]); - return 1; - } - - throw std::runtime_error("invalid arguments to vector:Dot"); + lua_pushnumber(L, a[0] * b[0] + a[1] * b[1] + a[2] * b[2]); + return 1; } static int lua_vector_index(lua_State* L) { + const float* v = luaL_checkvector(L, 1); const char* name = luaL_checkstring(L, 2); - if (const float* v = lua_tovector(L, 1)) + if (strcmp(name, "Magnitude") == 0) { - if (strcmp(name, "Magnitude") == 0) - { - lua_pushnumber(L, sqrtf(v[0] * v[0] + v[1] * v[1] + v[2] * v[2])); - return 1; - } - - if (strcmp(name, "Dot") == 0) - { - lua_pushcfunction(L, lua_vector_dot, "Dot"); - return 1; - } + lua_pushnumber(L, sqrtf(v[0] * v[0] + v[1] * v[1] + v[2] * v[2])); + return 1; } - throw std::runtime_error(Luau::format("%s is not a valid member of vector", name)); + if (strcmp(name, "Dot") == 0) + { + lua_pushcfunction(L, lua_vector_dot, "Dot"); + return 1; + } + + luaL_error(L, "%s is not a valid member of vector", name); } static int lua_vector_namecall(lua_State* L) @@ -115,7 +113,7 @@ static int lua_vector_namecall(lua_State* L) return lua_vector_dot(L); } - throw std::runtime_error(Luau::format("%s is not a valid method of vector", luaL_checkstring(L, 1))); + luaL_error(L, "%s is not a valid method of vector", luaL_checkstring(L, 1)); } int lua_silence(lua_State* L) @@ -373,11 +371,17 @@ TEST_CASE("Pack") TEST_CASE("Vector") { + ScopedFastFlag sff{"LuauIfElseExpressionBaseSupport", true}; + runConformance("vector.lua", [](lua_State* L) { lua_pushcfunction(L, lua_vector, "vector"); lua_setglobal(L, "vector"); +#if LUA_VECTOR_SIZE == 4 + lua_pushvector(L, 0.0f, 0.0f, 0.0f, 0.0f); +#else lua_pushvector(L, 0.0f, 0.0f, 0.0f); +#endif luaL_newmetatable(L, "vector"); lua_pushstring(L, "__index"); @@ -504,6 +508,9 @@ TEST_CASE("Debugger") cb->debugbreak = [](lua_State* L, lua_Debug* ar) { breakhits++; + // make sure we can trace the stack for every breakpoint we hit + lua_debugtrace(L); + // for every breakpoint, we break on the first invocation and continue on second // this allows us to easily step off breakpoints // (real implementaiton may require singlestepping) @@ -524,7 +531,7 @@ TEST_CASE("Debugger") L, [](lua_State* L) -> int { int line = luaL_checkinteger(L, 1); - bool enabled = lua_isboolean(L, 2) ? lua_toboolean(L, 2) : true; + bool enabled = luaL_optboolean(L, 2, true); lua_Debug ar = {}; lua_getinfo(L, 1, "f", &ar); @@ -699,21 +706,52 @@ TEST_CASE("ApiFunctionCalls") StateRef globalState = runConformance("apicalls.lua"); lua_State* L = globalState.get(); - lua_getfield(L, LUA_GLOBALSINDEX, "add"); - lua_pushnumber(L, 40); - lua_pushnumber(L, 2); - lua_call(L, 2, 1); - CHECK(lua_isnumber(L, -1)); - CHECK(lua_tonumber(L, -1) == 42); - lua_pop(L, 1); + // lua_call + { + lua_getfield(L, LUA_GLOBALSINDEX, "add"); + lua_pushnumber(L, 40); + lua_pushnumber(L, 2); + lua_call(L, 2, 1); + CHECK(lua_isnumber(L, -1)); + CHECK(lua_tonumber(L, -1) == 42); + lua_pop(L, 1); + } - lua_getfield(L, LUA_GLOBALSINDEX, "add"); - lua_pushnumber(L, 40); - lua_pushnumber(L, 2); - lua_pcall(L, 2, 1, 0); - CHECK(lua_isnumber(L, -1)); - CHECK(lua_tonumber(L, -1) == 42); - lua_pop(L, 1); + // lua_pcall + { + lua_getfield(L, LUA_GLOBALSINDEX, "add"); + lua_pushnumber(L, 40); + lua_pushnumber(L, 2); + lua_pcall(L, 2, 1, 0); + CHECK(lua_isnumber(L, -1)); + CHECK(lua_tonumber(L, -1) == 42); + lua_pop(L, 1); + } + + // lua_equal with a sleeping thread wake up + { + ScopedFastFlag luauActivateBeforeExec("LuauActivateBeforeExec", true); + + lua_State* L2 = lua_newthread(L); + + lua_getfield(L2, LUA_GLOBALSINDEX, "create_with_tm"); + lua_pushnumber(L2, 42); + lua_pcall(L2, 1, 1, 0); + + lua_getfield(L2, LUA_GLOBALSINDEX, "create_with_tm"); + lua_pushnumber(L2, 42); + lua_pcall(L2, 1, 1, 0); + + // Reset GC + lua_gc(L2, LUA_GCCOLLECT, 0); + + // Try to mark 'L2' as sleeping + // Can't control GC precisely, even in tests + lua_gc(L2, LUA_GCSTEP, 8); + + CHECK(lua_equal(L2, -1, -2) == 1); + lua_pop(L2, 2); + } } static bool endsWith(const std::string& str, const std::string& suffix) @@ -727,8 +765,6 @@ static bool endsWith(const std::string& str, const std::string& suffix) #if !LUA_USE_LONGJMP TEST_CASE("ExceptionObject") { - ScopedFastFlag sff("LuauExceptionMessageFix", true); - struct ExceptionResult { bool exceptionGenerated; diff --git a/tests/Fixture.cpp b/tests/Fixture.cpp index 29c33f7c..36d6f561 100644 --- a/tests/Fixture.cpp +++ b/tests/Fixture.cpp @@ -19,19 +19,6 @@ static const char* mainModuleName = "MainModule"; namespace Luau { -std::optional TestFileResolver::fromAstFragment(AstExpr* expr) const -{ - auto g = expr->as(); - if (!g) - return std::nullopt; - - std::string_view value = g->name.value; - if (value == "game" || value == "Game" || value == "workspace" || value == "Workspace" || value == "script" || value == "Script") - return ModuleName(value); - - return std::nullopt; -} - std::optional TestFileResolver::resolveModule(const ModuleInfo* context, AstExpr* expr) { if (AstExprGlobal* g = expr->as()) @@ -81,24 +68,6 @@ std::optional TestFileResolver::resolveModule(const ModuleInfo* cont return std::nullopt; } -ModuleName TestFileResolver::concat(const ModuleName& lhs, std::string_view rhs) const -{ - return lhs + "/" + ModuleName(rhs); -} - -std::optional TestFileResolver::getParentModuleName(const ModuleName& name) const -{ - std::string_view view = name; - const size_t lastSeparatorIndex = view.find_last_of('/'); - - if (lastSeparatorIndex != std::string_view::npos) - { - return ModuleName(view.substr(0, lastSeparatorIndex)); - } - - return std::nullopt; -} - std::string TestFileResolver::getHumanReadableModuleName(const ModuleName& name) const { return name; @@ -324,6 +293,13 @@ std::optional Fixture::findTypeAtPosition(Position position) return Luau::findTypeAtPosition(*module, *sourceModule, position); } +std::optional Fixture::findExpectedTypeAtPosition(Position position) +{ + ModulePtr module = getMainModule(); + SourceModule* sourceModule = getMainSourceModule(); + return Luau::findExpectedTypeAtPosition(*module, *sourceModule, position); +} + TypeId Fixture::requireTypeAtPosition(Position position) { auto ty = findTypeAtPosition(position); diff --git a/tests/Fixture.h b/tests/Fixture.h index 1480a7f6..de2b7381 100644 --- a/tests/Fixture.h +++ b/tests/Fixture.h @@ -64,12 +64,8 @@ struct TestFileResolver return SourceCode{it->second, sourceType}; } - std::optional fromAstFragment(AstExpr* expr) const override; std::optional resolveModule(const ModuleInfo* context, AstExpr* expr) override; - ModuleName concat(const ModuleName& lhs, std::string_view rhs) const override; - std::optional getParentModuleName(const ModuleName& name) const override; - std::string getHumanReadableModuleName(const ModuleName& name) const override; std::optional getEnvironmentForModule(const ModuleName& name) const override; @@ -126,6 +122,7 @@ struct Fixture std::optional findTypeAtPosition(Position position); TypeId requireTypeAtPosition(Position position); + std::optional findExpectedTypeAtPosition(Position position); std::optional lookupType(const std::string& name); std::optional lookupImportedType(const std::string& moduleAlias, const std::string& name); diff --git a/tests/Frontend.test.cpp b/tests/Frontend.test.cpp index fbfec636..51fcd3d6 100644 --- a/tests/Frontend.test.cpp +++ b/tests/Frontend.test.cpp @@ -46,18 +46,6 @@ NaiveModuleResolver naiveModuleResolver; struct NaiveFileResolver : NullFileResolver { - std::optional fromAstFragment(AstExpr* expr) const override - { - AstExprGlobal* g = expr->as(); - if (g && g->name == "Modules") - return "Modules"; - - if (g && g->name == "game") - return "game"; - - return std::nullopt; - } - std::optional resolveModule(const ModuleInfo* context, AstExpr* expr) override { if (AstExprGlobal* g = expr->as()) @@ -86,11 +74,6 @@ struct NaiveFileResolver : NullFileResolver return std::nullopt; } - - ModuleName concat(const ModuleName& lhs, std::string_view rhs) const override - { - return lhs + "/" + ModuleName(rhs); - } }; } // namespace diff --git a/tests/Linter.test.cpp b/tests/Linter.test.cpp index 7ba40c50..1d13df28 100644 --- a/tests/Linter.test.cpp +++ b/tests/Linter.test.cpp @@ -1469,6 +1469,22 @@ _ = true and true or false -- no warning since this is is a common pattern used CHECK_EQ(result.warnings[6].location.begin.line + 1, 19); } +TEST_CASE_FIXTURE(Fixture, "DuplicateConditionsExpr") +{ + LintResult result = lint(R"( +local correct, opaque = ... + +if correct({a = 1, b = 2 * (-2), c = opaque.path['with']("calls")}) then +elseif correct({a = 1, b = 2 * (-2), c = opaque.path['with']("calls")}) then +elseif correct({a = 1, b = 2 * (-2), c = opaque.path['with']("calls", false)}) then +end +)"); + + REQUIRE_EQ(result.warnings.size(), 1); + CHECK_EQ(result.warnings[0].text, "Condition has already been checked on line 4"); + CHECK_EQ(result.warnings[0].location.begin.line + 1, 5); +} + TEST_CASE_FIXTURE(Fixture, "DuplicateLocal") { LintResult result = lint(R"( diff --git a/tests/Module.test.cpp b/tests/Module.test.cpp index 7a3543c7..2800d2fe 100644 --- a/tests/Module.test.cpp +++ b/tests/Module.test.cpp @@ -44,9 +44,10 @@ TEST_CASE_FIXTURE(Fixture, "dont_clone_persistent_primitive") SeenTypes seenTypes; SeenTypePacks seenTypePacks; + CloneState cloneState; // numberType is persistent. We leave it as-is. - TypeId newNumber = clone(typeChecker.numberType, dest, seenTypes, seenTypePacks); + TypeId newNumber = clone(typeChecker.numberType, dest, seenTypes, seenTypePacks, cloneState); CHECK_EQ(newNumber, typeChecker.numberType); } @@ -56,12 +57,13 @@ TEST_CASE_FIXTURE(Fixture, "deepClone_non_persistent_primitive") SeenTypes seenTypes; SeenTypePacks seenTypePacks; + CloneState cloneState; // Create a new number type that isn't persistent unfreeze(typeChecker.globalTypes); TypeId oldNumber = typeChecker.globalTypes.addType(PrimitiveTypeVar{PrimitiveTypeVar::Number}); freeze(typeChecker.globalTypes); - TypeId newNumber = clone(oldNumber, dest, seenTypes, seenTypePacks); + TypeId newNumber = clone(oldNumber, dest, seenTypes, seenTypePacks, cloneState); CHECK_NE(newNumber, oldNumber); CHECK_EQ(*oldNumber, *newNumber); @@ -89,9 +91,10 @@ TEST_CASE_FIXTURE(Fixture, "deepClone_cyclic_table") SeenTypes seenTypes; SeenTypePacks seenTypePacks; + CloneState cloneState; TypeArena dest; - TypeId counterCopy = clone(counterType, dest, seenTypes, seenTypePacks); + TypeId counterCopy = clone(counterType, dest, seenTypes, seenTypePacks, cloneState); TableTypeVar* ttv = getMutable(counterCopy); REQUIRE(ttv != nullptr); @@ -142,11 +145,12 @@ TEST_CASE_FIXTURE(Fixture, "deepClone_union") SeenTypes seenTypes; SeenTypePacks seenTypePacks; + CloneState cloneState; unfreeze(typeChecker.globalTypes); TypeId oldUnion = typeChecker.globalTypes.addType(UnionTypeVar{{typeChecker.numberType, typeChecker.stringType}}); freeze(typeChecker.globalTypes); - TypeId newUnion = clone(oldUnion, dest, seenTypes, seenTypePacks); + TypeId newUnion = clone(oldUnion, dest, seenTypes, seenTypePacks, cloneState); CHECK_NE(newUnion, oldUnion); CHECK_EQ("number | string", toString(newUnion)); @@ -159,11 +163,12 @@ TEST_CASE_FIXTURE(Fixture, "deepClone_intersection") SeenTypes seenTypes; SeenTypePacks seenTypePacks; + CloneState cloneState; unfreeze(typeChecker.globalTypes); TypeId oldIntersection = typeChecker.globalTypes.addType(IntersectionTypeVar{{typeChecker.numberType, typeChecker.stringType}}); freeze(typeChecker.globalTypes); - TypeId newIntersection = clone(oldIntersection, dest, seenTypes, seenTypePacks); + TypeId newIntersection = clone(oldIntersection, dest, seenTypes, seenTypePacks, cloneState); CHECK_NE(newIntersection, oldIntersection); CHECK_EQ("number & string", toString(newIntersection)); @@ -188,8 +193,9 @@ TEST_CASE_FIXTURE(Fixture, "clone_class") SeenTypes seenTypes; SeenTypePacks seenTypePacks; + CloneState cloneState; - TypeId cloned = clone(&exampleClass, dest, seenTypes, seenTypePacks); + TypeId cloned = clone(&exampleClass, dest, seenTypes, seenTypePacks, cloneState); const ClassTypeVar* ctv = get(cloned); REQUIRE(ctv != nullptr); @@ -211,16 +217,16 @@ TEST_CASE_FIXTURE(Fixture, "clone_sanitize_free_types") TypeArena dest; SeenTypes seenTypes; SeenTypePacks seenTypePacks; + CloneState cloneState; - bool encounteredFreeType = false; - TypeId clonedTy = clone(&freeTy, dest, seenTypes, seenTypePacks, &encounteredFreeType); + TypeId clonedTy = clone(&freeTy, dest, seenTypes, seenTypePacks, cloneState); CHECK_EQ("any", toString(clonedTy)); - CHECK(encounteredFreeType); + CHECK(cloneState.encounteredFreeType); - encounteredFreeType = false; - TypePackId clonedTp = clone(&freeTp, dest, seenTypes, seenTypePacks, &encounteredFreeType); + cloneState = {}; + TypePackId clonedTp = clone(&freeTp, dest, seenTypes, seenTypePacks, cloneState); CHECK_EQ("...any", toString(clonedTp)); - CHECK(encounteredFreeType); + CHECK(cloneState.encounteredFreeType); } TEST_CASE_FIXTURE(Fixture, "clone_seal_free_tables") @@ -232,12 +238,12 @@ TEST_CASE_FIXTURE(Fixture, "clone_seal_free_tables") TypeArena dest; SeenTypes seenTypes; SeenTypePacks seenTypePacks; + CloneState cloneState; - bool encounteredFreeType = false; - TypeId cloned = clone(&tableTy, dest, seenTypes, seenTypePacks, &encounteredFreeType); + TypeId cloned = clone(&tableTy, dest, seenTypes, seenTypePacks, cloneState); const TableTypeVar* clonedTtv = get(cloned); CHECK_EQ(clonedTtv->state, TableState::Sealed); - CHECK(encounteredFreeType); + CHECK(cloneState.encounteredFreeType); } TEST_CASE_FIXTURE(Fixture, "clone_self_property") @@ -267,4 +273,34 @@ TEST_CASE_FIXTURE(Fixture, "clone_self_property") "dot or pass 1 extra nil to suppress this warning"); } +TEST_CASE_FIXTURE(Fixture, "clone_recursion_limit") +{ +#if defined(_DEBUG) || defined(_NOOPT) + int limit = 250; +#else + int limit = 500; +#endif + ScopedFastInt luauTypeCloneRecursionLimit{"LuauTypeCloneRecursionLimit", limit}; + + TypeArena src; + + TypeId table = src.addType(TableTypeVar{}); + TypeId nested = table; + + for (unsigned i = 0; i < limit + 100; i++) + { + TableTypeVar* ttv = getMutable(nested); + + ttv->props["a"].type = src.addType(TableTypeVar{}); + nested = ttv->props["a"].type; + } + + TypeArena dest; + SeenTypes seenTypes; + SeenTypePacks seenTypePacks; + CloneState cloneState; + + CHECK_THROWS_AS(clone(table, dest, seenTypes, seenTypePacks, cloneState), std::runtime_error); +} + TEST_SUITE_END(); diff --git a/tests/Parser.test.cpp b/tests/Parser.test.cpp index e3e6ce6d..72d3a9a6 100644 --- a/tests/Parser.test.cpp +++ b/tests/Parser.test.cpp @@ -2518,8 +2518,6 @@ TEST_CASE_FIXTURE(Fixture, "parse_if_else_expression") TEST_CASE_FIXTURE(Fixture, "parse_type_pack_type_parameters") { - ScopedFastFlag luauParseTypePackTypeParameters("LuauParseTypePackTypeParameters", true); - AstStat* stat = parse(R"( type Packed = () -> T... diff --git a/tests/ToDot.test.cpp b/tests/ToDot.test.cpp new file mode 100644 index 00000000..0ca9c994 --- /dev/null +++ b/tests/ToDot.test.cpp @@ -0,0 +1,366 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details + +#include "Luau/Scope.h" +#include "Luau/ToDot.h" + +#include "Fixture.h" + +#include "doctest.h" + +using namespace Luau; + +struct ToDotClassFixture : Fixture +{ + ToDotClassFixture() + { + TypeArena& arena = typeChecker.globalTypes; + + unfreeze(arena); + + TypeId baseClassMetaType = arena.addType(TableTypeVar{}); + + TypeId baseClassInstanceType = arena.addType(ClassTypeVar{"BaseClass", {}, std::nullopt, baseClassMetaType, {}, {}}); + getMutable(baseClassInstanceType)->props = { + {"BaseField", {typeChecker.numberType}}, + }; + typeChecker.globalScope->exportedTypeBindings["BaseClass"] = TypeFun{{}, baseClassInstanceType}; + + TypeId childClassInstanceType = arena.addType(ClassTypeVar{"ChildClass", {}, baseClassInstanceType, std::nullopt, {}, {}}); + getMutable(childClassInstanceType)->props = { + {"ChildField", {typeChecker.stringType}}, + }; + typeChecker.globalScope->exportedTypeBindings["ChildClass"] = TypeFun{{}, childClassInstanceType}; + + freeze(arena); + } +}; + +TEST_SUITE_BEGIN("ToDot"); + +TEST_CASE_FIXTURE(Fixture, "primitive") +{ + CheckResult result = check(R"( +local a: nil +local b: number +local c: any +)"); + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_NE("nil", toDot(requireType("a"))); + + CHECK_EQ(R"(digraph graphname { +n1 [label="number"]; +})", + toDot(requireType("b"))); + + CHECK_EQ(R"(digraph graphname { +n1 [label="any"]; +})", + toDot(requireType("c"))); + + ToDotOptions opts; + opts.showPointers = false; + opts.duplicatePrimitives = false; + + CHECK_EQ(R"(digraph graphname { +n1 [label="PrimitiveTypeVar number"]; +})", + toDot(requireType("b"), opts)); + + CHECK_EQ(R"(digraph graphname { +n1 [label="AnyTypeVar 1"]; +})", + toDot(requireType("c"), opts)); +} + +TEST_CASE_FIXTURE(Fixture, "bound") +{ + CheckResult result = check(R"( +local a = 444 +local b = a +)"); + LUAU_REQUIRE_NO_ERRORS(result); + + std::optional ty = getType("b"); + REQUIRE(bool(ty)); + + ToDotOptions opts; + opts.showPointers = false; + CHECK_EQ(R"(digraph graphname { +n1 [label="BoundTypeVar 1"]; +n1 -> n2; +n2 [label="number"]; +})", + toDot(*ty, opts)); +} + +TEST_CASE_FIXTURE(Fixture, "function") +{ + ScopedFastFlag luauQuantifyInPlace2{"LuauQuantifyInPlace2", true}; + + CheckResult result = check(R"( +local function f(a, ...: string) return a end +)"); + LUAU_REQUIRE_NO_ERRORS(result); + + ToDotOptions opts; + opts.showPointers = false; + CHECK_EQ(R"(digraph graphname { +n1 [label="FunctionTypeVar 1"]; +n1 -> n2 [label="arg"]; +n2 [label="TypePack 2"]; +n2 -> n3; +n3 [label="GenericTypeVar 3"]; +n2 -> n4 [label="tail"]; +n4 [label="VariadicTypePack 4"]; +n4 -> n5; +n5 [label="string"]; +n1 -> n6 [label="ret"]; +n6 [label="BoundTypePack 6"]; +n6 -> n7; +n7 [label="TypePack 7"]; +n7 -> n3; +})", + toDot(requireType("f"), opts)); +} + +TEST_CASE_FIXTURE(Fixture, "union") +{ + CheckResult result = check(R"( +local a: string | number +)"); + LUAU_REQUIRE_NO_ERRORS(result); + + ToDotOptions opts; + opts.showPointers = false; + CHECK_EQ(R"(digraph graphname { +n1 [label="UnionTypeVar 1"]; +n1 -> n2; +n2 [label="string"]; +n1 -> n3; +n3 [label="number"]; +})", + toDot(requireType("a"), opts)); +} + +TEST_CASE_FIXTURE(Fixture, "intersection") +{ + CheckResult result = check(R"( +local a: string & number -- uninhabited +)"); + LUAU_REQUIRE_NO_ERRORS(result); + + ToDotOptions opts; + opts.showPointers = false; + CHECK_EQ(R"(digraph graphname { +n1 [label="IntersectionTypeVar 1"]; +n1 -> n2; +n2 [label="string"]; +n1 -> n3; +n3 [label="number"]; +})", + toDot(requireType("a"), opts)); +} + +TEST_CASE_FIXTURE(Fixture, "table") +{ + CheckResult result = check(R"( +type A = { x: T, y: (U...) -> (), [string]: any } +local a: A +)"); + LUAU_REQUIRE_NO_ERRORS(result); + + ToDotOptions opts; + opts.showPointers = false; + CHECK_EQ(R"(digraph graphname { +n1 [label="TableTypeVar A"]; +n1 -> n2 [label="x"]; +n2 [label="number"]; +n1 -> n3 [label="y"]; +n3 [label="FunctionTypeVar 3"]; +n3 -> n4 [label="arg"]; +n4 [label="VariadicTypePack 4"]; +n4 -> n5; +n5 [label="string"]; +n3 -> n6 [label="ret"]; +n6 [label="TypePack 6"]; +n1 -> n7 [label="[index]"]; +n7 [label="string"]; +n1 -> n8 [label="[value]"]; +n8 [label="any"]; +n1 -> n9 [label="typeParam"]; +n9 [label="number"]; +n1 -> n4 [label="typePackParam"]; +})", + toDot(requireType("a"), opts)); + + // Extra coverage with pointers (unstable values) + (void)toDot(requireType("a")); +} + +TEST_CASE_FIXTURE(Fixture, "metatable") +{ + CheckResult result = check(R"( +local a: typeof(setmetatable({}, {})) +)"); + LUAU_REQUIRE_NO_ERRORS(result); + + ToDotOptions opts; + opts.showPointers = false; + CHECK_EQ(R"(digraph graphname { +n1 [label="MetatableTypeVar 1"]; +n1 -> n2 [label="table"]; +n2 [label="TableTypeVar 2"]; +n1 -> n3 [label="metatable"]; +n3 [label="TableTypeVar 3"]; +})", + toDot(requireType("a"), opts)); +} + +TEST_CASE_FIXTURE(Fixture, "free") +{ + TypeVar type{TypeVariant{FreeTypeVar{TypeLevel{0, 0}}}}; + + ToDotOptions opts; + opts.showPointers = false; + CHECK_EQ(R"(digraph graphname { +n1 [label="FreeTypeVar 1"]; +})", + toDot(&type, opts)); +} + +TEST_CASE_FIXTURE(Fixture, "error") +{ + TypeVar type{TypeVariant{ErrorTypeVar{}}}; + + ToDotOptions opts; + opts.showPointers = false; + CHECK_EQ(R"(digraph graphname { +n1 [label="ErrorTypeVar 1"]; +})", + toDot(&type, opts)); +} + +TEST_CASE_FIXTURE(Fixture, "generic") +{ + TypeVar type{TypeVariant{GenericTypeVar{"T"}}}; + + ToDotOptions opts; + opts.showPointers = false; + CHECK_EQ(R"(digraph graphname { +n1 [label="GenericTypeVar T"]; +})", + toDot(&type, opts)); +} + +TEST_CASE_FIXTURE(ToDotClassFixture, "class") +{ + CheckResult result = check(R"( +local a: ChildClass +)"); + LUAU_REQUIRE_NO_ERRORS(result); + + ToDotOptions opts; + opts.showPointers = false; + CHECK_EQ(R"(digraph graphname { +n1 [label="ClassTypeVar ChildClass"]; +n1 -> n2 [label="ChildField"]; +n2 [label="string"]; +n1 -> n3 [label="[parent]"]; +n3 [label="ClassTypeVar BaseClass"]; +n3 -> n4 [label="BaseField"]; +n4 [label="number"]; +n3 -> n5 [label="[metatable]"]; +n5 [label="TableTypeVar 5"]; +})", + toDot(requireType("a"), opts)); +} + +TEST_CASE_FIXTURE(Fixture, "free_pack") +{ + TypePackVar pack{TypePackVariant{FreeTypePack{TypeLevel{0, 0}}}}; + + ToDotOptions opts; + opts.showPointers = false; + CHECK_EQ(R"(digraph graphname { +n1 [label="FreeTypePack 1"]; +})", + toDot(&pack, opts)); +} + +TEST_CASE_FIXTURE(Fixture, "error_pack") +{ + TypePackVar pack{TypePackVariant{Unifiable::Error{}}}; + + ToDotOptions opts; + opts.showPointers = false; + CHECK_EQ(R"(digraph graphname { +n1 [label="ErrorTypePack 1"]; +})", + toDot(&pack, opts)); + + // Extra coverage with pointers (unstable values) + (void)toDot(&pack); +} + +TEST_CASE_FIXTURE(Fixture, "generic_pack") +{ + TypePackVar pack1{TypePackVariant{GenericTypePack{}}}; + TypePackVar pack2{TypePackVariant{GenericTypePack{"T"}}}; + + ToDotOptions opts; + opts.showPointers = false; + CHECK_EQ(R"(digraph graphname { +n1 [label="GenericTypePack 1"]; +})", + toDot(&pack1, opts)); + + CHECK_EQ(R"(digraph graphname { +n1 [label="GenericTypePack T"]; +})", + toDot(&pack2, opts)); +} + +TEST_CASE_FIXTURE(Fixture, "bound_pack") +{ + TypePackVar pack{TypePackVariant{TypePack{{typeChecker.numberType}, {}}}}; + TypePackVar bound{TypePackVariant{BoundTypePack{&pack}}}; + + ToDotOptions opts; + opts.showPointers = false; + CHECK_EQ(R"(digraph graphname { +n1 [label="BoundTypePack 1"]; +n1 -> n2; +n2 [label="TypePack 2"]; +n2 -> n3; +n3 [label="number"]; +})", + toDot(&bound, opts)); +} + +TEST_CASE_FIXTURE(Fixture, "bound_table") +{ + CheckResult result = check(R"( +local a = {x=2} +local b +b.x = 2 +b = a +)"); + LUAU_REQUIRE_NO_ERRORS(result); + + std::optional ty = getType("b"); + REQUIRE(bool(ty)); + + ToDotOptions opts; + opts.showPointers = false; + CHECK_EQ(R"(digraph graphname { +n1 [label="TableTypeVar 1"]; +n1 -> n2 [label="boundTo"]; +n2 [label="TableTypeVar a"]; +n2 -> n3 [label="x"]; +n3 [label="number"]; +})", + toDot(*ty, opts)); +} + +TEST_SUITE_END(); diff --git a/tests/Transpiler.test.cpp b/tests/Transpiler.test.cpp index 928c03a3..327fa0bb 100644 --- a/tests/Transpiler.test.cpp +++ b/tests/Transpiler.test.cpp @@ -445,9 +445,6 @@ local a: Import.Type TEST_CASE_FIXTURE(Fixture, "transpile_type_packs") { - ScopedFastFlag luauTypeAliasPacks("LuauTypeAliasPacks", true); - ScopedFastFlag luauParseTypePackTypeParameters("LuauParseTypePackTypeParameters", true); - std::string code = R"( type Packed = (T...)->(T...) local a: Packed<> diff --git a/tests/TypeInfer.aliases.test.cpp b/tests/TypeInfer.aliases.test.cpp index 74ce155c..822bd727 100644 --- a/tests/TypeInfer.aliases.test.cpp +++ b/tests/TypeInfer.aliases.test.cpp @@ -537,8 +537,6 @@ TEST_CASE_FIXTURE(Fixture, "free_variables_from_typeof_in_aliases") TEST_CASE_FIXTURE(Fixture, "non_recursive_aliases_that_reuse_a_generic_name") { - ScopedFastFlag sff1{"LuauSubstitutionDontReplaceIgnoredTypes", true}; - CheckResult result = check(R"( type Array = { [number]: T } type Tuple = Array diff --git a/tests/TypeInfer.generics.test.cpp b/tests/TypeInfer.generics.test.cpp index 88c2dc85..aba50891 100644 --- a/tests/TypeInfer.generics.test.cpp +++ b/tests/TypeInfer.generics.test.cpp @@ -609,8 +609,6 @@ TEST_CASE_FIXTURE(Fixture, "typefuns_sharing_types") TEST_CASE_FIXTURE(Fixture, "bound_tables_do_not_clone_original_fields") { - ScopedFastFlag luauCloneBoundTables{"LuauCloneBoundTables", true}; - CheckResult result = check(R"( local exports = {} local nested = {} @@ -627,4 +625,23 @@ return exports LUAU_REQUIRE_NO_ERRORS(result); } +TEST_CASE_FIXTURE(Fixture, "instantiated_function_argument_names") +{ + ScopedFastFlag luauFunctionArgumentNameSize{"LuauFunctionArgumentNameSize", true}; + + CheckResult result = check(R"( +local function f(a: T, ...: U...) end + +f(1, 2, 3) + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + auto ty = findTypeAtPosition(Position(3, 0)); + REQUIRE(ty); + ToStringOptions opts; + opts.functionTypeArguments = true; + CHECK_EQ(toString(*ty, opts), "(a: number, number, number) -> ()"); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.provisional.test.cpp b/tests/TypeInfer.provisional.test.cpp index e5c14dde..e6d3d4d4 100644 --- a/tests/TypeInfer.provisional.test.cpp +++ b/tests/TypeInfer.provisional.test.cpp @@ -31,8 +31,6 @@ TEST_SUITE_BEGIN("ProvisionalTests"); */ TEST_CASE_FIXTURE(Fixture, "typeguard_inference_incomplete") { - ScopedFastFlag luauTypeAliasPacks("LuauTypeAliasPacks", true); - const std::string code = R"( function f(a) if type(a) == "boolean" then diff --git a/tests/TypeInfer.tables.test.cpp b/tests/TypeInfer.tables.test.cpp index c3694be7..cb72faaf 100644 --- a/tests/TypeInfer.tables.test.cpp +++ b/tests/TypeInfer.tables.test.cpp @@ -2022,4 +2022,74 @@ caused by: Property '__call' is not compatible. Type '(a, b) -> ()' could not be converted into '(a) -> ()')"); } +TEST_CASE_FIXTURE(Fixture, "explicitly_typed_table") +{ + ScopedFastFlag sffs[] { + {"LuauPropertiesGetExpectedType", true}, + {"LuauExpectedTypesOfProperties", true}, + {"LuauTableSubtypingVariance", true}, + }; + + CheckResult result = check(R"( +--!strict +type Super = { x : number } +type Sub = { x : number, y: number } +type HasSuper = { p : Super } +type HasSub = { p : Sub } +local a: HasSuper = { p = { x = 5, y = 7 }} +a.p = { x = 9 } + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "explicitly_typed_table_error") +{ + ScopedFastFlag sffs[] { + {"LuauPropertiesGetExpectedType", true}, + {"LuauExpectedTypesOfProperties", true}, + {"LuauTableSubtypingVariance", true}, + {"LuauExtendedTypeMismatchError", true}, + }; + + CheckResult result = check(R"( +--!strict +type Super = { x : number } +type Sub = { x : number, y: number } +type HasSuper = { p : Super } +type HasSub = { p : Sub } +local tmp = { p = { x = 5, y = 7 }} +local a: HasSuper = tmp +a.p = { x = 9 } +-- needs to be an error because +local y: number = tmp.p.y + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ(toString(result.errors[0]), R"(Type 'tmp' could not be converted into 'HasSuper' +caused by: + Property 'p' is not compatible. Table type '{| x: number, y: number |}' not compatible with type 'Super' because the former has extra field 'y')"); +} + +TEST_CASE_FIXTURE(Fixture, "explicitly_typed_table_with_indexer") +{ + ScopedFastFlag sffs[] { + {"LuauPropertiesGetExpectedType", true}, + {"LuauExpectedTypesOfProperties", true}, + {"LuauTableSubtypingVariance", true}, + }; + + CheckResult result = check(R"( +--!strict +type Super = { x : number } +type Sub = { x : number, y: number } +type HasSuper = { [string] : Super } +type HasSub = { [string] : Sub } +local a: HasSuper = { p = { x = 5, y = 7 }} +a.p = { x = 9 } + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.test.cpp b/tests/TypeInfer.test.cpp index 99fd8339..e3222a41 100644 --- a/tests/TypeInfer.test.cpp +++ b/tests/TypeInfer.test.cpp @@ -4779,4 +4779,24 @@ local bar = foo.nutrition + 100 // CHECK_EQ("Unknown type used in + operation; consider adding a type annotation to 'foo'", toString(result.errors[1])); } +TEST_CASE_FIXTURE(Fixture, "require_failed_module") +{ + ScopedFastFlag luauModuleRequireErrorPack{"LuauModuleRequireErrorPack", true}; + + fileResolver.source["game/A"] = R"( +return unfortunately() + )"; + + CheckResult aResult = frontend.check("game/A"); + LUAU_REQUIRE_ERRORS(aResult); + + CheckResult result = check(R"( +local ModuleA = require(game.A) + )"); + LUAU_REQUIRE_NO_ERRORS(result); + + std::optional oty = requireType("ModuleA"); + CHECK_EQ("*unknown*", toString(*oty)); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.typePacks.cpp b/tests/TypeInfer.typePacks.cpp index c6de0abf..3f4420cd 100644 --- a/tests/TypeInfer.typePacks.cpp +++ b/tests/TypeInfer.typePacks.cpp @@ -296,9 +296,6 @@ end TEST_CASE_FIXTURE(Fixture, "type_alias_type_packs") { - ScopedFastFlag luauTypeAliasPacks("LuauTypeAliasPacks", true); - ScopedFastFlag luauParseTypePackTypeParameters("LuauParseTypePackTypeParameters", true); - CheckResult result = check(R"( type Packed = (T...) -> T... local a: Packed<> @@ -360,9 +357,6 @@ local c: Packed TEST_CASE_FIXTURE(Fixture, "type_alias_type_packs_import") { - ScopedFastFlag luauTypeAliasPacks("LuauTypeAliasPacks", true); - ScopedFastFlag luauParseTypePackTypeParameters("LuauParseTypePackTypeParameters", true); - fileResolver.source["game/A"] = R"( export type Packed = { a: T, b: (U...) -> () } return {} @@ -393,9 +387,6 @@ local d: { a: typeof(c) } TEST_CASE_FIXTURE(Fixture, "type_pack_type_parameters") { - ScopedFastFlag luauTypeAliasPacks("LuauTypeAliasPacks", true); - ScopedFastFlag luauParseTypePackTypeParameters("LuauParseTypePackTypeParameters", true); - fileResolver.source["game/A"] = R"( export type Packed = { a: T, b: (U...) -> () } return {} @@ -431,9 +422,6 @@ type C = Import.Packed TEST_CASE_FIXTURE(Fixture, "type_alias_type_packs_nested") { - ScopedFastFlag luauTypeAliasPacks("LuauTypeAliasPacks", true); - ScopedFastFlag luauParseTypePackTypeParameters("LuauParseTypePackTypeParameters", true); - CheckResult result = check(R"( type Packed1 = (T...) -> (T...) type Packed2 = (Packed1, T...) -> (Packed1, T...) @@ -452,9 +440,6 @@ type Packed4 = (Packed3, T...) -> (Packed3, T...) TEST_CASE_FIXTURE(Fixture, "type_alias_type_pack_variadic") { - ScopedFastFlag luauTypeAliasPacks("LuauTypeAliasPacks", true); - ScopedFastFlag luauParseTypePackTypeParameters("LuauParseTypePackTypeParameters", true); - CheckResult result = check(R"( type X = (T...) -> (string, T...) @@ -470,9 +455,6 @@ type E = X<(number, ...string)> TEST_CASE_FIXTURE(Fixture, "type_alias_type_pack_multi") { - ScopedFastFlag luauTypeAliasPacks("LuauTypeAliasPacks", true); - ScopedFastFlag luauParseTypePackTypeParameters("LuauParseTypePackTypeParameters", true); - CheckResult result = check(R"( type Y = (T...) -> (U...) type A = Y @@ -501,9 +483,6 @@ type I = W TEST_CASE_FIXTURE(Fixture, "type_alias_type_pack_explicit") { - ScopedFastFlag luauTypeAliasPacks("LuauTypeAliasPacks", true); - ScopedFastFlag luauParseTypePackTypeParameters("LuauParseTypePackTypeParameters", true); - CheckResult result = check(R"( type X = (T...) -> (T...) @@ -527,9 +506,6 @@ type F = X<(string, ...number)> TEST_CASE_FIXTURE(Fixture, "type_alias_type_pack_explicit_multi") { - ScopedFastFlag luauTypeAliasPacks("LuauTypeAliasPacks", true); - ScopedFastFlag luauParseTypePackTypeParameters("LuauParseTypePackTypeParameters", true); - CheckResult result = check(R"( type Y = (T...) -> (U...) @@ -549,9 +525,6 @@ type D = Y TEST_CASE_FIXTURE(Fixture, "type_alias_type_pack_explicit_multi_tostring") { - ScopedFastFlag luauTypeAliasPacks("LuauTypeAliasPacks", true); - ScopedFastFlag luauParseTypePackTypeParameters("LuauParseTypePackTypeParameters", true); - CheckResult result = check(R"( type Y = { f: (T...) -> (U...) } @@ -567,9 +540,6 @@ local b: Y<(), ()> TEST_CASE_FIXTURE(Fixture, "type_alias_backwards_compatible") { - ScopedFastFlag luauTypeAliasPacks("LuauTypeAliasPacks", true); - ScopedFastFlag luauParseTypePackTypeParameters("LuauParseTypePackTypeParameters", true); - CheckResult result = check(R"( type X = () -> T type Y = (T) -> U @@ -588,9 +558,6 @@ type C = Y<(number), boolean> TEST_CASE_FIXTURE(Fixture, "type_alias_type_packs_errors") { - ScopedFastFlag luauTypeAliasPacks("LuauTypeAliasPacks", true); - ScopedFastFlag luauParseTypePackTypeParameters("LuauParseTypePackTypeParameters", true); - CheckResult result = check(R"( type Packed = (T, U) -> (V...) local b: Packed diff --git a/tests/TypeVar.test.cpp b/tests/TypeVar.test.cpp index 91efa818..13db923e 100644 --- a/tests/TypeVar.test.cpp +++ b/tests/TypeVar.test.cpp @@ -3,6 +3,7 @@ #include "Luau/Scope.h" #include "Luau/TypeInfer.h" #include "Luau/TypeVar.h" +#include "Luau/VisitTypeVar.h" #include "Fixture.h" #include "ScopedFlags.h" @@ -323,4 +324,48 @@ TEST_CASE("tagging_props") CHECK(Luau::hasTag(prop, "foo")); } +struct VisitCountTracker +{ + std::unordered_map tyVisits; + std::unordered_map tpVisits; + + void cycle(TypeId) {} + void cycle(TypePackId) {} + + template + bool operator()(TypeId ty, const T& t) + { + tyVisits[ty]++; + return true; + } + + template + bool operator()(TypePackId tp, const T&) + { + tpVisits[tp]++; + return true; + } +}; + +TEST_CASE_FIXTURE(Fixture, "visit_once") +{ + CheckResult result = check(R"( +type T = { a: number, b: () -> () } +local b: (T, T, T) -> T +)"); + LUAU_REQUIRE_NO_ERRORS(result); + + TypeId bType = requireType("b"); + + VisitCountTracker tester; + DenseHashSet seen{nullptr}; + visitTypeVarOnce(bType, tester, seen); + + for (auto [_, count] : tester.tyVisits) + CHECK_EQ(count, 1); + + for (auto [_, count] : tester.tpVisits) + CHECK_EQ(count, 1); +} + TEST_SUITE_END(); diff --git a/tests/conformance/apicalls.lua b/tests/conformance/apicalls.lua index 5e03b055..7a4058b5 100644 --- a/tests/conformance/apicalls.lua +++ b/tests/conformance/apicalls.lua @@ -2,7 +2,13 @@ print('testing function calls through API') function add(a, b) - return a + b + return a + b +end + +local m = { __eq = function(a, b) return a.a == b.a end } + +function create_with_tm(x) + return setmetatable({ a = x }, m) end return('OK') diff --git a/tests/conformance/basic.lua b/tests/conformance/basic.lua index 687fff1e..188b8ebc 100644 --- a/tests/conformance/basic.lua +++ b/tests/conformance/basic.lua @@ -441,7 +441,8 @@ assert((function() a = {} b = {} mt = { __eq = function(l, r) return #l == #r en assert((function() a = {} b = {} function eq(l, r) return #l == #r end setmetatable(a, {__eq = eq}) setmetatable(b, {__eq = eq}) return concat(a == b, a ~= b) end)() == "true,false") assert((function() a = {} b = {} setmetatable(a, {__eq = function(l, r) return #l == #r end}) setmetatable(b, {__eq = function(l, r) return #l == #r end}) return concat(a == b, a ~= b) end)() == "false,true") --- userdata, reference equality (no mt) +-- userdata, reference equality (no mt or mt.__eq) +assert((function() a = newproxy() return concat(a == newproxy(),a ~= newproxy()) end)() == "false,true") assert((function() a = newproxy(true) return concat(a == newproxy(true),a ~= newproxy(true)) end)() == "false,true") -- rawequal @@ -876,4 +877,4 @@ assert(concat(typeof(5), typeof(nil), typeof({}), typeof(newproxy())) == "number testgetfenv() -- DONT MOVE THIS LINE -return'OK' +return 'OK' diff --git a/tests/conformance/closure.lua b/tests/conformance/closure.lua index aac42c56..f32d5bdc 100644 --- a/tests/conformance/closure.lua +++ b/tests/conformance/closure.lua @@ -419,11 +419,5 @@ co = coroutine.create(function () return loadstring("return a")() end) -a = {a = 15} --- debug.setfenv(co, a) --- assert(debug.getfenv(co) == a) --- assert(select(2, coroutine.resume(co)) == a) --- assert(select(2, coroutine.resume(co)) == a.a) - -return'OK' +return 'OK' diff --git a/tests/conformance/constructs.lua b/tests/conformance/constructs.lua index 16c63b00..f133501f 100644 --- a/tests/conformance/constructs.lua +++ b/tests/conformance/constructs.lua @@ -237,4 +237,4 @@ repeat i = i+1 until i==c -return'OK' +return 'OK' diff --git a/tests/conformance/coroutine.lua b/tests/conformance/coroutine.lua index 4d9b1295..f2ecc96b 100644 --- a/tests/conformance/coroutine.lua +++ b/tests/conformance/coroutine.lua @@ -373,4 +373,4 @@ do assert(f() == 42) end -return'OK' +return 'OK' diff --git a/tests/conformance/datetime.lua b/tests/conformance/datetime.lua index 21ef60d7..ca35cf2f 100644 --- a/tests/conformance/datetime.lua +++ b/tests/conformance/datetime.lua @@ -74,4 +74,4 @@ assert(os.difftime(t1,t2) == 60*2-19) assert(os.time({ year = 1970, day = 1, month = 1, hour = 0}) == 0) -return'OK' +return 'OK' diff --git a/tests/conformance/debug.lua b/tests/conformance/debug.lua index ee79a14f..9cf3c742 100644 --- a/tests/conformance/debug.lua +++ b/tests/conformance/debug.lua @@ -98,4 +98,4 @@ assert(quuz(function(...) end) == "0 true") assert(quuz(function(a, b) end) == "2 false") assert(quuz(function(a, b, ...) end) == "2 true") -return'OK' +return 'OK' diff --git a/tests/conformance/errors.lua b/tests/conformance/errors.lua index eded14e9..d5ff215b 100644 --- a/tests/conformance/errors.lua +++ b/tests/conformance/errors.lua @@ -34,15 +34,15 @@ assert(doit("error('hi', 0)") == 'hi') assert(doit("unpack({}, 1, n=2^30)")) assert(doit("a=math.sin()")) assert(not doit("tostring(1)") and doit("tostring()")) -assert(doit"tonumber()") -assert(doit"repeat until 1; a") +assert(doit("tonumber()")) +assert(doit("repeat until 1; a")) checksyntax("break label", "", "label", 1) -assert(doit";") -assert(doit"a=1;;") -assert(doit"return;;") -assert(doit"assert(false)") -assert(doit"assert(nil)") -assert(doit"a=math.sin\n(3)") +assert(doit(";")) +assert(doit("a=1;;")) +assert(doit("return;;")) +assert(doit("assert(false)")) +assert(doit("assert(nil)")) +assert(doit("a=math.sin\n(3)")) assert(doit("function a (... , ...) end")) assert(doit("function a (, ...) end")) @@ -59,7 +59,7 @@ checkmessage("a=1; local a,bbbb=2,3; a = math.sin(1) and bbbb(3)", "local 'bbbb'") checkmessage("a={}; do local a=1 end a:bbbb(3)", "method 'bbbb'") checkmessage("local a={}; a.bbbb(3)", "field 'bbbb'") -assert(not string.find(doit"a={13}; local bbbb=1; a[bbbb](3)", "'bbbb'")) +assert(not string.find(doit("a={13}; local bbbb=1; a[bbbb](3)"), "'bbbb'")) checkmessage("a={13}; local bbbb=1; a[bbbb](3)", "number") aaa = nil @@ -67,14 +67,14 @@ checkmessage("aaa.bbb:ddd(9)", "global 'aaa'") checkmessage("local aaa={bbb=1}; aaa.bbb:ddd(9)", "field 'bbb'") checkmessage("local aaa={bbb={}}; aaa.bbb:ddd(9)", "method 'ddd'") checkmessage("local a,b,c; (function () a = b+1 end)()", "upvalue 'b'") -assert(not doit"local aaa={bbb={ddd=next}}; aaa.bbb:ddd(nil)") +assert(not doit("local aaa={bbb={ddd=next}}; aaa.bbb:ddd(nil)")) checkmessage("b=1; local aaa='a'; x=aaa+b", "local 'aaa'") checkmessage("aaa={}; x=3/aaa", "global 'aaa'") checkmessage("aaa='2'; b=nil;x=aaa*b", "global 'b'") checkmessage("aaa={}; x=-aaa", "global 'aaa'") -assert(not string.find(doit"aaa={}; x=(aaa or aaa)+(aaa and aaa)", "'aaa'")) -assert(not string.find(doit"aaa={}; (aaa or aaa)()", "'aaa'")) +assert(not string.find(doit("aaa={}; x=(aaa or aaa)+(aaa and aaa)"), "'aaa'")) +assert(not string.find(doit("aaa={}; (aaa or aaa)()"), "'aaa'")) checkmessage([[aaa=9 repeat until 3==3 @@ -122,10 +122,10 @@ function lineerror (s) return line and line+0 end -assert(lineerror"local a\n for i=1,'a' do \n print(i) \n end" == 2) --- assert(lineerror"\n local a \n for k,v in 3 \n do \n print(k) \n end" == 3) --- assert(lineerror"\n\n for k,v in \n 3 \n do \n print(k) \n end" == 4) -assert(lineerror"function a.x.y ()\na=a+1\nend" == 1) +assert(lineerror("local a\n for i=1,'a' do \n print(i) \n end") == 2) +-- assert(lineerror("\n local a \n for k,v in 3 \n do \n print(k) \n end") == 3) +-- assert(lineerror("\n\n for k,v in \n 3 \n do \n print(k) \n end") == 4) +assert(lineerror("function a.x.y ()\na=a+1\nend") == 1) local p = [[ function g() f() end diff --git a/tests/conformance/gc.lua b/tests/conformance/gc.lua index 4263dfda..6d9eb854 100644 --- a/tests/conformance/gc.lua +++ b/tests/conformance/gc.lua @@ -77,7 +77,7 @@ end local function dosteps (siz) collectgarbage() - collectgarbage"stop" + collectgarbage("stop") local a = {} for i=1,100 do a[i] = {{}}; local b = {} end local x = gcinfo() @@ -99,11 +99,11 @@ assert(dosteps(10000) == 1) do local x = gcinfo() collectgarbage() - collectgarbage"stop" + collectgarbage("stop") repeat local a = {} until gcinfo() > 1000 - collectgarbage"restart" + collectgarbage("restart") repeat local a = {} until gcinfo() < 1000 @@ -123,7 +123,7 @@ for n in pairs(b) do end b = nil collectgarbage() -for n in pairs(a) do error'cannot be here' end +for n in pairs(a) do error("cannot be here") end for i=1,lim do a[i] = i end for i=1,lim do assert(a[i] == i) end diff --git a/tests/conformance/nextvar.lua b/tests/conformance/nextvar.lua index 7f9b7596..94ba5ccf 100644 --- a/tests/conformance/nextvar.lua +++ b/tests/conformance/nextvar.lua @@ -368,9 +368,9 @@ assert(next(a,nil) == 1000 and next(a,1000) == nil) assert(next({}) == nil) assert(next({}, nil) == nil) -for a,b in pairs{} do error"not here" end -for i=1,0 do error'not here' end -for i=0,1,-1 do error'not here' end +for a,b in pairs{} do error("not here") end +for i=1,0 do error("not here") end +for i=0,1,-1 do error("not here") end a = nil; for i=1,1 do assert(not a); a=1 end; assert(a) a = nil; for i=1,1,-1 do assert(not a); a=1 end; assert(a) diff --git a/tests/conformance/pcall.lua b/tests/conformance/pcall.lua index a2072d2c..84ac2ba1 100644 --- a/tests/conformance/pcall.lua +++ b/tests/conformance/pcall.lua @@ -144,4 +144,4 @@ coroutine.resume(co) resumeerror(co, "fail") checkresults({ true, false, "fail" }, coroutine.resume(co)) -return'OK' +return 'OK' diff --git a/tests/conformance/utf8.lua b/tests/conformance/utf8.lua index 024cb16d..bfd7a1ac 100644 --- a/tests/conformance/utf8.lua +++ b/tests/conformance/utf8.lua @@ -205,4 +205,4 @@ for p, c in string.gmatch(x, "()(" .. utf8.charpattern .. ")") do end end -return'OK' +return 'OK' diff --git a/tests/conformance/vector.lua b/tests/conformance/vector.lua index 620f646a..7d18bda3 100644 --- a/tests/conformance/vector.lua +++ b/tests/conformance/vector.lua @@ -1,6 +1,9 @@ -- This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details print('testing vectors') +-- detect vector size +local vector_size = if pcall(function() return vector(0, 0, 0).w end) then 4 else 3 + -- equality assert(vector(1, 2, 3) == vector(1, 2, 3)) assert(vector(0, 1, 2) == vector(-0, 1, 2)) @@ -13,8 +16,14 @@ assert(not rawequal(vector(1, 2, 3), vector(1, 2, 4))) -- type & tostring assert(type(vector(1, 2, 3)) == "vector") -assert(tostring(vector(1, 2, 3)) == "1, 2, 3") -assert(tostring(vector(-1, 2, 0.5)) == "-1, 2, 0.5") + +if vector_size == 4 then + assert(tostring(vector(1, 2, 3, 4)) == "1, 2, 3, 4") + assert(tostring(vector(-1, 2, 0.5, 0)) == "-1, 2, 0.5, 0") +else + assert(tostring(vector(1, 2, 3)) == "1, 2, 3") + assert(tostring(vector(-1, 2, 0.5)) == "-1, 2, 0.5") +end local t = {} @@ -42,12 +51,19 @@ assert(8 * vector(8, 16, 24) == vector(64, 128, 192)); assert(vector(1, 2, 4) * '8' == vector(8, 16, 32)); assert('8' * vector(8, 16, 24) == vector(64, 128, 192)); -assert(vector(1, 2, 4) / vector(8, 16, 24) == vector(1/8, 2/16, 4/24)); +if vector_size == 4 then + assert(vector(1, 2, 4, 8) / vector(8, 16, 24, 32) == vector(1/8, 2/16, 4/24, 8/32)); + assert(8 / vector(8, 16, 24, 32) == vector(1, 1/2, 1/3, 1/4)); + assert('8' / vector(8, 16, 24, 32) == vector(1, 1/2, 1/3, 1/4)); +else + assert(vector(1, 2, 4) / vector(8, 16, 24, 1) == vector(1/8, 2/16, 4/24)); + assert(8 / vector(8, 16, 24) == vector(1, 1/2, 1/3)); + assert('8' / vector(8, 16, 24) == vector(1, 1/2, 1/3)); +end + assert(vector(1, 2, 4) / 8 == vector(1/8, 1/4, 1/2)); assert(vector(1, 2, 4) / (1 / val) == vector(1/8, 2/8, 4/8)); -assert(8 / vector(8, 16, 24) == vector(1, 1/2, 1/3)); assert(vector(1, 2, 4) / '8' == vector(1/8, 1/4, 1/2)); -assert('8' / vector(8, 16, 24) == vector(1, 1/2, 1/3)); assert(-vector(1, 2, 4) == vector(-1, -2, -4)); @@ -71,4 +87,9 @@ assert(pcall(function() local t = {} rawset(t, vector(0/0, 2, 3), 1) end) == fal -- make sure we cover both builtin and C impl assert(vector(1, 2, 4) == vector("1", "2", "4")) +-- additional checks for 4-component vectors +if vector_size == 4 then + assert(vector(1, 2, 3, 4).w == 4) +end + return 'OK' diff --git a/tools/svg.py b/tools/svg.py index 3b3bb28c..99853fb6 100644 --- a/tools/svg.py +++ b/tools/svg.py @@ -458,13 +458,16 @@ def display(root, title, colors, flip = False): framewidth = 1200 - 20 + def pixels(x): + return float(x) / root.width * framewidth if root.width > 0 else 0 + for n in root.subtree(): - if n.width / root.width * framewidth < 0.1: + if pixels(n.width) < 0.1: continue - x = 10 + n.offset / root.width * framewidth + x = 10 + pixels(n.offset) y = (maxdepth - 1 - n.depth if flip else n.depth) * 16 + 3 * 16 - width = n.width / root.width * framewidth + width = pixels(n.width) height = 15 if colors == "cold": From e440729e2bb98aba0bdb41109e257de522b857e4 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Thu, 2 Dec 2021 15:46:33 -0800 Subject: [PATCH 090/399] Fix signed/unsigned mismatch warning + lower limit to match upstream --- tests/Module.test.cpp | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/tests/Module.test.cpp b/tests/Module.test.cpp index 2800d2fe..e3993cc5 100644 --- a/tests/Module.test.cpp +++ b/tests/Module.test.cpp @@ -278,7 +278,7 @@ TEST_CASE_FIXTURE(Fixture, "clone_recursion_limit") #if defined(_DEBUG) || defined(_NOOPT) int limit = 250; #else - int limit = 500; + int limit = 400; #endif ScopedFastInt luauTypeCloneRecursionLimit{"LuauTypeCloneRecursionLimit", limit}; @@ -287,7 +287,7 @@ TEST_CASE_FIXTURE(Fixture, "clone_recursion_limit") TypeId table = src.addType(TableTypeVar{}); TypeId nested = table; - for (unsigned i = 0; i < limit + 100; i++) + for (int i = 0; i < limit + 100; i++) { TableTypeVar* ttv = getMutable(nested); From f5ec6df7ba0b112359932d1e5032804d4ea016fa Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Thu, 2 Dec 2021 15:55:06 -0800 Subject: [PATCH 091/399] Update build.yml Disable continue-on-error for coverage because hiding the error actually makes it difficult to debug now :) --- .github/workflows/build.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml index c2b4448f..3782886c 100644 --- a/.github/workflows/build.yml +++ b/.github/workflows/build.yml @@ -81,7 +81,7 @@ jobs: with: path-to-lcov: ./coverage.info github-token: ${{ secrets.GITHUB_TOKEN }} - continue-on-error: true + continue-on-error: false - uses: actions/upload-artifact@v2 with: name: coverage From 32fb6d10a79464d7fd70813ab3daf8534540d81e Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Thu, 2 Dec 2021 22:41:04 -0800 Subject: [PATCH 092/399] Sync to upstream/release/506 (#270) - Fix some cases where type checking would overflow the native stack - Improve autocomplete behavior when assigning a partially written function call (not currently exposed through command line tools) - Improve autocomplete type inference feedback for some expressions where previously the type would not be known - Improve quantification performance during type checking for large types - Improve type checking for table literals when the expected type of the table is known because of a type annotation - Fix type checking errors in cases where required module has errors in the resulting type - Fix debug line information for multi-line chained call sequences (Add function name information for "attempt to call a nil value" #255) - lua_newuserdata now takes 2 arguments to match Lua/LuaJIT APIs better; lua_newuserdatatagged should be used if the third argument was non-0. - lua_ref can no longer be used with LUA_REGISTRYINDEX to prevent mistakes when migrating Lua FFI (Inconsistency with lua_ref #247) - Fix assertions and possible crashes when executing script code indirectly via metatable dispatch from lua_equal/lua_lessthan/lua_getfield/etc. (Hitting a crash in an assert after lua_equal is called. #259) - Fix flamegraph scripts to run under Python 2 --- Analysis/include/Luau/FileResolver.h | 23 - Analysis/include/Luau/Module.h | 12 +- Analysis/include/Luau/ToDot.h | 31 ++ Analysis/include/Luau/TxnLog.h | 9 - Analysis/include/Luau/TypeInfer.h | 1 - Analysis/include/Luau/TypeVar.h | 15 - Analysis/include/Luau/Unifier.h | 25 +- Analysis/include/Luau/UnifierSharedState.h | 8 + Analysis/include/Luau/VisitTypeVar.h | 4 +- Analysis/src/Autocomplete.cpp | 72 ++- Analysis/src/BuiltinDefinitions.cpp | 6 +- Analysis/src/Error.cpp | 116 +---- Analysis/src/Frontend.cpp | 27 +- Analysis/src/IostreamHelpers.cpp | 19 +- Analysis/src/JsonEncoder.cpp | 9 +- Analysis/src/Module.cpp | 132 ++--- Analysis/src/Quantify.cpp | 13 +- Analysis/src/RequireTracer.cpp | 195 +------ Analysis/src/Substitution.cpp | 29 +- Analysis/src/ToDot.cpp | 378 ++++++++++++++ Analysis/src/ToString.cpp | 172 +++---- Analysis/src/Transpiler.cpp | 15 +- Analysis/src/TxnLog.cpp | 37 +- Analysis/src/TypeAttach.cpp | 52 +- Analysis/src/TypeInfer.cpp | 331 +++++------- Analysis/src/TypeVar.cpp | 364 -------------- Analysis/src/Unifier.cpp | 306 ++--------- Ast/src/Parser.cpp | 74 +-- CLI/Analyze.cpp | 23 +- CLI/Repl.cpp | 1 - Compiler/src/Compiler.cpp | 14 +- Makefile | 6 +- Sources.cmake | 4 + VM/include/lua.h | 3 +- VM/src/lapi.cpp | 43 +- VM/src/lbaselib.cpp | 2 +- VM/src/ldo.cpp | 71 +-- VM/src/lgc.cpp | 548 +------------------- VM/src/lgcdebug.cpp | 558 +++++++++++++++++++++ fuzz/proto.cpp | 4 +- tests/AstQuery.test.cpp | 23 + tests/Autocomplete.test.cpp | 40 +- tests/Compiler.test.cpp | 168 +++++++ tests/Conformance.test.cpp | 64 ++- tests/Fixture.cpp | 38 +- tests/Fixture.h | 5 +- tests/Frontend.test.cpp | 17 - tests/Linter.test.cpp | 16 + tests/Module.test.cpp | 66 ++- tests/Parser.test.cpp | 2 - tests/ToDot.test.cpp | 366 ++++++++++++++ tests/Transpiler.test.cpp | 3 - tests/TypeInfer.aliases.test.cpp | 2 - tests/TypeInfer.generics.test.cpp | 21 +- tests/TypeInfer.provisional.test.cpp | 2 - tests/TypeInfer.tables.test.cpp | 70 +++ tests/TypeInfer.test.cpp | 20 + tests/TypeInfer.typePacks.cpp | 33 -- tests/TypeVar.test.cpp | 45 ++ tests/conformance/apicalls.lua | 8 +- tests/conformance/basic.lua | 3 +- tools/svg.py | 9 +- 62 files changed, 2430 insertions(+), 2343 deletions(-) create mode 100644 Analysis/include/Luau/ToDot.h create mode 100644 Analysis/src/ToDot.cpp create mode 100644 VM/src/lgcdebug.cpp create mode 100644 tests/ToDot.test.cpp diff --git a/Analysis/include/Luau/FileResolver.h b/Analysis/include/Luau/FileResolver.h index 9b74fc12..0fdcce16 100644 --- a/Analysis/include/Luau/FileResolver.h +++ b/Analysis/include/Luau/FileResolver.h @@ -51,13 +51,6 @@ struct FileResolver { return std::nullopt; } - - // DEPRECATED APIS - // These are going to be removed with LuauNewRequireTrace2 - virtual bool moduleExists(const ModuleName& name) const = 0; - virtual std::optional fromAstFragment(AstExpr* expr) const = 0; - virtual ModuleName concat(const ModuleName& lhs, std::string_view rhs) const = 0; - virtual std::optional getParentModuleName(const ModuleName& name) const = 0; }; struct NullFileResolver : FileResolver @@ -66,22 +59,6 @@ struct NullFileResolver : FileResolver { return std::nullopt; } - bool moduleExists(const ModuleName& name) const override - { - return false; - } - std::optional fromAstFragment(AstExpr* expr) const override - { - return std::nullopt; - } - ModuleName concat(const ModuleName& lhs, std::string_view rhs) const override - { - return lhs; - } - std::optional getParentModuleName(const ModuleName& name) const override - { - return std::nullopt; - } }; } // namespace Luau diff --git a/Analysis/include/Luau/Module.h b/Analysis/include/Luau/Module.h index d0844835..2e41674b 100644 --- a/Analysis/include/Luau/Module.h +++ b/Analysis/include/Luau/Module.h @@ -78,9 +78,15 @@ void unfreeze(TypeArena& arena); using SeenTypes = std::unordered_map; using SeenTypePacks = std::unordered_map; -TypePackId clone(TypePackId tp, TypeArena& dest, SeenTypes& seenTypes, SeenTypePacks& seenTypePacks, bool* encounteredFreeType = nullptr); -TypeId clone(TypeId tp, TypeArena& dest, SeenTypes& seenTypes, SeenTypePacks& seenTypePacks, bool* encounteredFreeType = nullptr); -TypeFun clone(const TypeFun& typeFun, TypeArena& dest, SeenTypes& seenTypes, SeenTypePacks& seenTypePacks, bool* encounteredFreeType = nullptr); +struct CloneState +{ + int recursionCount = 0; + bool encounteredFreeType = false; +}; + +TypePackId clone(TypePackId tp, TypeArena& dest, SeenTypes& seenTypes, SeenTypePacks& seenTypePacks, CloneState& cloneState); +TypeId clone(TypeId tp, TypeArena& dest, SeenTypes& seenTypes, SeenTypePacks& seenTypePacks, CloneState& cloneState); +TypeFun clone(const TypeFun& typeFun, TypeArena& dest, SeenTypes& seenTypes, SeenTypePacks& seenTypePacks, CloneState& cloneState); struct Module { diff --git a/Analysis/include/Luau/ToDot.h b/Analysis/include/Luau/ToDot.h new file mode 100644 index 00000000..ce518d3a --- /dev/null +++ b/Analysis/include/Luau/ToDot.h @@ -0,0 +1,31 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +#pragma once + +#include "Luau/Common.h" + +#include + +namespace Luau +{ +struct TypeVar; +using TypeId = const TypeVar*; + +struct TypePackVar; +using TypePackId = const TypePackVar*; + +struct ToDotOptions +{ + bool showPointers = true; // Show pointer value in the node label + bool duplicatePrimitives = true; // Display primitive types and 'any' as separate nodes +}; + +std::string toDot(TypeId ty, const ToDotOptions& opts); +std::string toDot(TypePackId tp, const ToDotOptions& opts); + +std::string toDot(TypeId ty); +std::string toDot(TypePackId tp); + +void dumpDot(TypeId ty); +void dumpDot(TypePackId tp); + +} // namespace Luau diff --git a/Analysis/include/Luau/TxnLog.h b/Analysis/include/Luau/TxnLog.h index 322abd19..29988a3b 100644 --- a/Analysis/include/Luau/TxnLog.h +++ b/Analysis/include/Luau/TxnLog.h @@ -25,15 +25,6 @@ struct TxnLog { } - explicit TxnLog(const std::vector>& ownedSeen) - : originalSeenSize(ownedSeen.size()) - , ownedSeen(ownedSeen) - , sharedSeen(nullptr) - { - // This is deprecated! - LUAU_ASSERT(!FFlag::LuauShareTxnSeen); - } - TxnLog(const TxnLog&) = delete; TxnLog& operator=(const TxnLog&) = delete; diff --git a/Analysis/include/Luau/TypeInfer.h b/Analysis/include/Luau/TypeInfer.h index 78d642c5..9f553bc1 100644 --- a/Analysis/include/Luau/TypeInfer.h +++ b/Analysis/include/Luau/TypeInfer.h @@ -297,7 +297,6 @@ private: private: Unifier mkUnifier(const Location& location); - Unifier mkUnifier(const std::vector>& seen, const Location& location); // These functions are only safe to call when we are in the process of typechecking a module. diff --git a/Analysis/include/Luau/TypeVar.h b/Analysis/include/Luau/TypeVar.h index 093ea431..8c4c2f34 100644 --- a/Analysis/include/Luau/TypeVar.h +++ b/Analysis/include/Luau/TypeVar.h @@ -517,21 +517,6 @@ extern SingletonTypes singletonTypes; void persist(TypeId ty); void persist(TypePackId tp); -struct ToDotOptions -{ - bool showPointers = true; // Show pointer value in the node label - bool duplicatePrimitives = true; // Display primitive types and 'any' as separate nodes -}; - -std::string toDot(TypeId ty, const ToDotOptions& opts); -std::string toDot(TypePackId tp, const ToDotOptions& opts); - -std::string toDot(TypeId ty); -std::string toDot(TypePackId tp); - -void dumpDot(TypeId ty); -void dumpDot(TypePackId tp); - const TypeLevel* getLevel(TypeId ty); TypeLevel* getMutableLevel(TypeId ty); diff --git a/Analysis/include/Luau/Unifier.h b/Analysis/include/Luau/Unifier.h index 503034a1..4588cdd8 100644 --- a/Analysis/include/Luau/Unifier.h +++ b/Analysis/include/Luau/Unifier.h @@ -19,12 +19,6 @@ enum Variance Invariant }; -struct UnifierCounters -{ - int recursionCount = 0; - int iterationCount = 0; -}; - struct Unifier { TypeArena* const types; @@ -37,20 +31,11 @@ struct Unifier Variance variance = Covariant; CountMismatch::Context ctx = CountMismatch::Arg; - UnifierCounters* counters; - UnifierCounters countersData; - - std::shared_ptr counters_DEPRECATED; - UnifierSharedState& sharedState; Unifier(TypeArena* types, Mode mode, ScopePtr globalScope, const Location& location, Variance variance, UnifierSharedState& sharedState); - Unifier(TypeArena* types, Mode mode, ScopePtr globalScope, const std::vector>& ownedSeen, const Location& location, - Variance variance, UnifierSharedState& sharedState, const std::shared_ptr& counters_DEPRECATED = nullptr, - UnifierCounters* counters = nullptr); Unifier(TypeArena* types, Mode mode, ScopePtr globalScope, std::vector>* sharedSeen, const Location& location, - Variance variance, UnifierSharedState& sharedState, const std::shared_ptr& counters_DEPRECATED = nullptr, - UnifierCounters* counters = nullptr); + Variance variance, UnifierSharedState& sharedState); // Test whether the two type vars unify. Never commits the result. ErrorVec canUnify(TypeId superTy, TypeId subTy); @@ -92,9 +77,9 @@ private: public: // Report an "infinite type error" if the type "needle" already occurs within "haystack" void occursCheck(TypeId needle, TypeId haystack); - void occursCheck(std::unordered_set& seen_DEPRECATED, DenseHashSet& seen, TypeId needle, TypeId haystack); + void occursCheck(DenseHashSet& seen, TypeId needle, TypeId haystack); void occursCheck(TypePackId needle, TypePackId haystack); - void occursCheck(std::unordered_set& seen_DEPRECATED, DenseHashSet& seen, TypePackId needle, TypePackId haystack); + void occursCheck(DenseHashSet& seen, TypePackId needle, TypePackId haystack); Unifier makeChildUnifier(); @@ -106,10 +91,6 @@ private: [[noreturn]] void ice(const std::string& message, const Location& location); [[noreturn]] void ice(const std::string& message); - - // Remove with FFlagLuauCacheUnifyTableResults - DenseHashSet tempSeenTy_DEPRECATED{nullptr}; - DenseHashSet tempSeenTp_DEPRECATED{nullptr}; }; } // namespace Luau diff --git a/Analysis/include/Luau/UnifierSharedState.h b/Analysis/include/Luau/UnifierSharedState.h index f252a004..88997c41 100644 --- a/Analysis/include/Luau/UnifierSharedState.h +++ b/Analysis/include/Luau/UnifierSharedState.h @@ -24,6 +24,12 @@ struct TypeIdPairHash } }; +struct UnifierCounters +{ + int recursionCount = 0; + int iterationCount = 0; +}; + struct UnifierSharedState { UnifierSharedState(InternalErrorReporter* iceHandler) @@ -39,6 +45,8 @@ struct UnifierSharedState DenseHashSet tempSeenTy{nullptr}; DenseHashSet tempSeenTp{nullptr}; + + UnifierCounters counters; }; } // namespace Luau diff --git a/Analysis/include/Luau/VisitTypeVar.h b/Analysis/include/Luau/VisitTypeVar.h index a866655c..740854b3 100644 --- a/Analysis/include/Luau/VisitTypeVar.h +++ b/Analysis/include/Luau/VisitTypeVar.h @@ -5,8 +5,6 @@ #include "Luau/TypeVar.h" #include "Luau/TypePack.h" -LUAU_FASTFLAG(LuauCacheUnifyTableResults) - namespace Luau { @@ -101,7 +99,7 @@ void visit(TypeId ty, F& f, Set& seen) // Some visitors want to see bound tables, that's why we visit the original type if (apply(ty, *ttv, seen, f)) { - if (FFlag::LuauCacheUnifyTableResults && ttv->boundTo) + if (ttv->boundTo) { visit(*ttv->boundTo, f, seen); } diff --git a/Analysis/src/Autocomplete.cpp b/Analysis/src/Autocomplete.cpp index 6fc0b3f8..db2d1d0e 100644 --- a/Analysis/src/Autocomplete.cpp +++ b/Analysis/src/Autocomplete.cpp @@ -12,9 +12,9 @@ #include #include -LUAU_FASTFLAGVARIABLE(ElseElseIfCompletionImprovements, false); LUAU_FASTFLAG(LuauIfElseExpressionAnalysisSupport) LUAU_FASTFLAGVARIABLE(LuauAutocompleteAvoidMutation, false); +LUAU_FASTFLAGVARIABLE(LuauAutocompletePreferToCallFunctions, false); static const std::unordered_set kStatementStartingKeywords = { "while", "if", "local", "repeat", "function", "do", "for", "return", "break", "continue", "type", "export"}; @@ -203,8 +203,9 @@ static TypeCorrectKind checkTypeCorrectKind(const Module& module, TypeArena* typ { SeenTypes seenTypes; SeenTypePacks seenTypePacks; - expectedType = clone(expectedType, *typeArena, seenTypes, seenTypePacks, nullptr); - actualType = clone(actualType, *typeArena, seenTypes, seenTypePacks, nullptr); + CloneState cloneState; + expectedType = clone(expectedType, *typeArena, seenTypes, seenTypePacks, cloneState); + actualType = clone(actualType, *typeArena, seenTypes, seenTypePacks, cloneState); auto errors = unifier.canUnify(expectedType, actualType); return errors.empty(); @@ -229,28 +230,51 @@ static TypeCorrectKind checkTypeCorrectKind(const Module& module, TypeArena* typ TypeId expectedType = follow(*it); - if (canUnify(expectedType, ty)) - return TypeCorrectKind::Correct; - - // We also want to suggest functions that return compatible result - const FunctionTypeVar* ftv = get(ty); - - if (!ftv) - return TypeCorrectKind::None; - - auto [retHead, retTail] = flatten(ftv->retType); - - if (!retHead.empty()) - return canUnify(expectedType, retHead.front()) ? TypeCorrectKind::CorrectFunctionResult : TypeCorrectKind::None; - - // We might only have a variadic tail pack, check if the element is compatible - if (retTail) + if (FFlag::LuauAutocompletePreferToCallFunctions) { - if (const VariadicTypePack* vtp = get(follow(*retTail))) - return canUnify(expectedType, vtp->ty) ? TypeCorrectKind::CorrectFunctionResult : TypeCorrectKind::None; - } + // We also want to suggest functions that return compatible result + if (const FunctionTypeVar* ftv = get(ty)) + { + auto [retHead, retTail] = flatten(ftv->retType); - return TypeCorrectKind::None; + if (!retHead.empty() && canUnify(expectedType, retHead.front())) + return TypeCorrectKind::CorrectFunctionResult; + + // We might only have a variadic tail pack, check if the element is compatible + if (retTail) + { + if (const VariadicTypePack* vtp = get(follow(*retTail)); vtp && canUnify(expectedType, vtp->ty)) + return TypeCorrectKind::CorrectFunctionResult; + } + } + + return canUnify(expectedType, ty) ? TypeCorrectKind::Correct : TypeCorrectKind::None; + } + else + { + if (canUnify(expectedType, ty)) + return TypeCorrectKind::Correct; + + // We also want to suggest functions that return compatible result + const FunctionTypeVar* ftv = get(ty); + + if (!ftv) + return TypeCorrectKind::None; + + auto [retHead, retTail] = flatten(ftv->retType); + + if (!retHead.empty()) + return canUnify(expectedType, retHead.front()) ? TypeCorrectKind::CorrectFunctionResult : TypeCorrectKind::None; + + // We might only have a variadic tail pack, check if the element is compatible + if (retTail) + { + if (const VariadicTypePack* vtp = get(follow(*retTail))) + return canUnify(expectedType, vtp->ty) ? TypeCorrectKind::CorrectFunctionResult : TypeCorrectKind::None; + } + + return TypeCorrectKind::None; + } } enum class PropIndexType @@ -1413,7 +1437,7 @@ static AutocompleteResult autocomplete(const SourceModule& sourceModule, const M else if (AstStatWhile* statWhile = extractStat(finder.ancestry); statWhile && !statWhile->hasDo) return {{{"do", AutocompleteEntry{AutocompleteEntryKind::Keyword}}}, finder.ancestry}; - else if (AstStatIf* statIf = node->as(); FFlag::ElseElseIfCompletionImprovements && statIf && !statIf->hasElse) + else if (AstStatIf* statIf = node->as(); statIf && !statIf->hasElse) { return {{{"else", AutocompleteEntry{AutocompleteEntryKind::Keyword}}, {"elseif", AutocompleteEntry{AutocompleteEntryKind::Keyword}}}, finder.ancestry}; diff --git a/Analysis/src/BuiltinDefinitions.cpp b/Analysis/src/BuiltinDefinitions.cpp index 62a06a3c..bac94a2b 100644 --- a/Analysis/src/BuiltinDefinitions.cpp +++ b/Analysis/src/BuiltinDefinitions.cpp @@ -8,8 +8,6 @@ #include -LUAU_FASTFLAG(LuauNewRequireTrace2) - /** FIXME: Many of these type definitions are not quite completely accurate. * * Some of them require richer generics than we have. For instance, we do not yet have a way to talk @@ -473,9 +471,7 @@ static std::optional> magicFunctionRequire( if (!checkRequirePath(typechecker, expr.args.data[0])) return std::nullopt; - const AstExpr* require = FFlag::LuauNewRequireTrace2 ? &expr : expr.args.data[0]; - - if (auto moduleInfo = typechecker.resolver->resolveModuleInfo(typechecker.currentModuleName, *require)) + if (auto moduleInfo = typechecker.resolver->resolveModuleInfo(typechecker.currentModuleName, expr)) return ExprResult{arena.addTypePack({typechecker.checkRequire(scope, *moduleInfo, expr.location)})}; return std::nullopt; diff --git a/Analysis/src/Error.cpp b/Analysis/src/Error.cpp index f80d50a7..8334bd62 100644 --- a/Analysis/src/Error.cpp +++ b/Analysis/src/Error.cpp @@ -7,57 +7,14 @@ #include -LUAU_FASTFLAG(LuauTypeAliasPacks) - -static std::string wrongNumberOfArgsString_DEPRECATED(size_t expectedCount, size_t actualCount, bool isTypeArgs = false) -{ - std::string s = "expects " + std::to_string(expectedCount) + " "; - - if (isTypeArgs) - s += "type "; - - s += "argument"; - if (expectedCount != 1) - s += "s"; - - s += ", but "; - - if (actualCount == 0) - { - s += "none"; - } - else - { - if (actualCount < expectedCount) - s += "only "; - - s += std::to_string(actualCount); - } - - s += (actualCount == 1) ? " is" : " are"; - - s += " specified"; - - return s; -} - static std::string wrongNumberOfArgsString(size_t expectedCount, size_t actualCount, const char* argPrefix = nullptr, bool isVariadic = false) { - std::string s; + std::string s = "expects "; - if (FFlag::LuauTypeAliasPacks) - { - s = "expects "; + if (isVariadic) + s += "at least "; - if (isVariadic) - s += "at least "; - - s += std::to_string(expectedCount) + " "; - } - else - { - s = "expects " + std::to_string(expectedCount) + " "; - } + s += std::to_string(expectedCount) + " "; if (argPrefix) s += std::string(argPrefix) + " "; @@ -188,10 +145,7 @@ struct ErrorConverter return "Function only returns " + std::to_string(e.expected) + " value" + expectedS + ". " + std::to_string(e.actual) + " are required here"; case CountMismatch::Arg: - if (FFlag::LuauTypeAliasPacks) - return "Argument count mismatch. Function " + wrongNumberOfArgsString(e.expected, e.actual); - else - return "Argument count mismatch. Function " + wrongNumberOfArgsString_DEPRECATED(e.expected, e.actual); + return "Argument count mismatch. Function " + wrongNumberOfArgsString(e.expected, e.actual); } LUAU_ASSERT(!"Unknown context"); @@ -232,7 +186,7 @@ struct ErrorConverter std::string operator()(const Luau::IncorrectGenericParameterCount& e) const { std::string name = e.name; - if (!e.typeFun.typeParams.empty() || (FFlag::LuauTypeAliasPacks && !e.typeFun.typePackParams.empty())) + if (!e.typeFun.typeParams.empty() || !e.typeFun.typePackParams.empty()) { name += "<"; bool first = true; @@ -246,36 +200,25 @@ struct ErrorConverter name += toString(t); } - if (FFlag::LuauTypeAliasPacks) + for (TypePackId t : e.typeFun.typePackParams) { - for (TypePackId t : e.typeFun.typePackParams) - { - if (first) - first = false; - else - name += ", "; + if (first) + first = false; + else + name += ", "; - name += toString(t); - } + name += toString(t); } name += ">"; } - if (FFlag::LuauTypeAliasPacks) - { - if (e.typeFun.typeParams.size() != e.actualParameters) - return "Generic type '" + name + "' " + - wrongNumberOfArgsString(e.typeFun.typeParams.size(), e.actualParameters, "type", !e.typeFun.typePackParams.empty()); + if (e.typeFun.typeParams.size() != e.actualParameters) + return "Generic type '" + name + "' " + + wrongNumberOfArgsString(e.typeFun.typeParams.size(), e.actualParameters, "type", !e.typeFun.typePackParams.empty()); - return "Generic type '" + name + "' " + - wrongNumberOfArgsString(e.typeFun.typePackParams.size(), e.actualPackParameters, "type pack", /*isVariadic*/ false); - } - else - { - return "Generic type '" + name + "' " + - wrongNumberOfArgsString_DEPRECATED(e.typeFun.typeParams.size(), e.actualParameters, /*isTypeArgs*/ true); - } + return "Generic type '" + name + "' " + + wrongNumberOfArgsString(e.typeFun.typePackParams.size(), e.actualPackParameters, "type pack", /*isVariadic*/ false); } std::string operator()(const Luau::SyntaxError& e) const @@ -591,11 +534,8 @@ bool IncorrectGenericParameterCount::operator==(const IncorrectGenericParameterC if (typeFun.typeParams.size() != rhs.typeFun.typeParams.size()) return false; - if (FFlag::LuauTypeAliasPacks) - { - if (typeFun.typePackParams.size() != rhs.typeFun.typePackParams.size()) - return false; - } + if (typeFun.typePackParams.size() != rhs.typeFun.typePackParams.size()) + return false; for (size_t i = 0; i < typeFun.typeParams.size(); ++i) { @@ -603,13 +543,10 @@ bool IncorrectGenericParameterCount::operator==(const IncorrectGenericParameterC return false; } - if (FFlag::LuauTypeAliasPacks) + for (size_t i = 0; i < typeFun.typePackParams.size(); ++i) { - for (size_t i = 0; i < typeFun.typePackParams.size(); ++i) - { - if (typeFun.typePackParams[i] != rhs.typeFun.typePackParams[i]) - return false; - } + if (typeFun.typePackParams[i] != rhs.typeFun.typePackParams[i]) + return false; } return true; @@ -733,14 +670,14 @@ bool containsParseErrorName(const TypeError& error) } template -void copyError(T& e, TypeArena& destArena, SeenTypes& seenTypes, SeenTypePacks& seenTypePacks) +void copyError(T& e, TypeArena& destArena, SeenTypes& seenTypes, SeenTypePacks& seenTypePacks, CloneState cloneState) { auto clone = [&](auto&& ty) { - return ::Luau::clone(ty, destArena, seenTypes, seenTypePacks); + return ::Luau::clone(ty, destArena, seenTypes, seenTypePacks, cloneState); }; auto visitErrorData = [&](auto&& e) { - copyError(e, destArena, seenTypes, seenTypePacks); + copyError(e, destArena, seenTypes, seenTypePacks, cloneState); }; if constexpr (false) @@ -864,9 +801,10 @@ void copyErrors(ErrorVec& errors, TypeArena& destArena) { SeenTypes seenTypes; SeenTypePacks seenTypePacks; + CloneState cloneState; auto visitErrorData = [&](auto&& e) { - copyError(e, destArena, seenTypes, seenTypePacks); + copyError(e, destArena, seenTypes, seenTypePacks, cloneState); }; LUAU_ASSERT(!destArena.typeVars.isFrozen()); diff --git a/Analysis/src/Frontend.cpp b/Analysis/src/Frontend.cpp index 1e97705d..e332f07d 100644 --- a/Analysis/src/Frontend.cpp +++ b/Analysis/src/Frontend.cpp @@ -18,10 +18,7 @@ LUAU_FASTFLAG(LuauInferInNoCheckMode) LUAU_FASTFLAGVARIABLE(LuauTypeCheckTwice, false) LUAU_FASTFLAGVARIABLE(LuauKnowsTheDataModel3, false) -LUAU_FASTFLAGVARIABLE(LuauResolveModuleNameWithoutACurrentModule, false) -LUAU_FASTFLAG(LuauTraceRequireLookupChild) LUAU_FASTFLAGVARIABLE(LuauPersistDefinitionFileTypes, false) -LUAU_FASTFLAG(LuauNewRequireTrace2) namespace Luau { @@ -96,10 +93,11 @@ LoadDefinitionFileResult loadDefinitionFile(TypeChecker& typeChecker, ScopePtr t SeenTypes seenTypes; SeenTypePacks seenTypePacks; + CloneState cloneState; for (const auto& [name, ty] : checkedModule->declaredGlobals) { - TypeId globalTy = clone(ty, typeChecker.globalTypes, seenTypes, seenTypePacks); + TypeId globalTy = clone(ty, typeChecker.globalTypes, seenTypes, seenTypePacks, cloneState); std::string documentationSymbol = packageName + "/global/" + name; generateDocumentationSymbols(globalTy, documentationSymbol); targetScope->bindings[typeChecker.globalNames.names->getOrAdd(name.c_str())] = {globalTy, Location(), false, {}, documentationSymbol}; @@ -110,7 +108,7 @@ LoadDefinitionFileResult loadDefinitionFile(TypeChecker& typeChecker, ScopePtr t for (const auto& [name, ty] : checkedModule->getModuleScope()->exportedTypeBindings) { - TypeFun globalTy = clone(ty, typeChecker.globalTypes, seenTypes, seenTypePacks); + TypeFun globalTy = clone(ty, typeChecker.globalTypes, seenTypes, seenTypePacks, cloneState); std::string documentationSymbol = packageName + "/globaltype/" + name; generateDocumentationSymbols(globalTy.type, documentationSymbol); targetScope->exportedTypeBindings[name] = globalTy; @@ -427,15 +425,16 @@ CheckResult Frontend::check(const ModuleName& name) SeenTypes seenTypes; SeenTypePacks seenTypePacks; + CloneState cloneState; for (const auto& [expr, strictTy] : strictModule->astTypes) - module->astTypes[expr] = clone(strictTy, module->interfaceTypes, seenTypes, seenTypePacks); + module->astTypes[expr] = clone(strictTy, module->interfaceTypes, seenTypes, seenTypePacks, cloneState); for (const auto& [expr, strictTy] : strictModule->astOriginalCallTypes) - module->astOriginalCallTypes[expr] = clone(strictTy, module->interfaceTypes, seenTypes, seenTypePacks); + module->astOriginalCallTypes[expr] = clone(strictTy, module->interfaceTypes, seenTypes, seenTypePacks, cloneState); for (const auto& [expr, strictTy] : strictModule->astExpectedTypes) - module->astExpectedTypes[expr] = clone(strictTy, module->interfaceTypes, seenTypes, seenTypePacks); + module->astExpectedTypes[expr] = clone(strictTy, module->interfaceTypes, seenTypes, seenTypePacks, cloneState); } stats.timeCheck += getTimestamp() - timestamp; @@ -885,16 +884,13 @@ std::optional FrontendModuleResolver::resolveModuleInfo(const Module // If we can't find the current module name, that's because we bypassed the frontend's initializer // and called typeChecker.check directly. (This is done by autocompleteSource, for example). // In that case, requires will always fail. - if (FFlag::LuauResolveModuleNameWithoutACurrentModule) - return std::nullopt; - else - throw std::runtime_error("Frontend::resolveModuleName: Unknown currentModuleName '" + currentModuleName + "'"); + return std::nullopt; } const auto& exprs = it->second.exprs; const ModuleInfo* info = exprs.find(&pathExpr); - if (!info || (!FFlag::LuauNewRequireTrace2 && info->name.empty())) + if (!info) return std::nullopt; return *info; @@ -911,10 +907,7 @@ const ModulePtr FrontendModuleResolver::getModule(const ModuleName& moduleName) bool FrontendModuleResolver::moduleExists(const ModuleName& moduleName) const { - if (FFlag::LuauNewRequireTrace2) - return frontend->sourceNodes.count(moduleName) != 0; - else - return frontend->fileResolver->moduleExists(moduleName); + return frontend->sourceNodes.count(moduleName) != 0; } std::string FrontendModuleResolver::getHumanReadableModuleName(const ModuleName& moduleName) const diff --git a/Analysis/src/IostreamHelpers.cpp b/Analysis/src/IostreamHelpers.cpp index 3b267121..ac46b5a4 100644 --- a/Analysis/src/IostreamHelpers.cpp +++ b/Analysis/src/IostreamHelpers.cpp @@ -2,8 +2,6 @@ #include "Luau/IostreamHelpers.h" #include "Luau/ToString.h" -LUAU_FASTFLAG(LuauTypeAliasPacks) - namespace Luau { @@ -94,7 +92,7 @@ std::ostream& operator<<(std::ostream& stream, const IncorrectGenericParameterCo { stream << "IncorrectGenericParameterCount { name = " << error.name; - if (!error.typeFun.typeParams.empty() || (FFlag::LuauTypeAliasPacks && !error.typeFun.typePackParams.empty())) + if (!error.typeFun.typeParams.empty() || !error.typeFun.typePackParams.empty()) { stream << "<"; bool first = true; @@ -108,17 +106,14 @@ std::ostream& operator<<(std::ostream& stream, const IncorrectGenericParameterCo stream << toString(t); } - if (FFlag::LuauTypeAliasPacks) + for (TypePackId t : error.typeFun.typePackParams) { - for (TypePackId t : error.typeFun.typePackParams) - { - if (first) - first = false; - else - stream << ", "; + if (first) + first = false; + else + stream << ", "; - stream << toString(t); - } + stream << toString(t); } stream << ">"; diff --git a/Analysis/src/JsonEncoder.cpp b/Analysis/src/JsonEncoder.cpp index 064accba..c7f623ee 100644 --- a/Analysis/src/JsonEncoder.cpp +++ b/Analysis/src/JsonEncoder.cpp @@ -5,8 +5,6 @@ #include "Luau/StringUtils.h" #include "Luau/Common.h" -LUAU_FASTFLAG(LuauTypeAliasPacks) - namespace Luau { @@ -615,12 +613,7 @@ struct AstJsonEncoder : public AstVisitor writeNode(node, "AstStatTypeAlias", [&]() { PROP(name); PROP(generics); - - if (FFlag::LuauTypeAliasPacks) - { - PROP(genericPacks); - } - + PROP(genericPacks); PROP(type); PROP(exported); }); diff --git a/Analysis/src/Module.cpp b/Analysis/src/Module.cpp index 32a0646a..b4b6eb42 100644 --- a/Analysis/src/Module.cpp +++ b/Analysis/src/Module.cpp @@ -1,20 +1,20 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #include "Luau/Module.h" +#include "Luau/Common.h" +#include "Luau/RecursionCounter.h" #include "Luau/Scope.h" #include "Luau/TypeInfer.h" #include "Luau/TypePack.h" #include "Luau/TypeVar.h" #include "Luau/VisitTypeVar.h" -#include "Luau/Common.h" #include LUAU_FASTFLAGVARIABLE(DebugLuauFreezeArena, false) LUAU_FASTFLAGVARIABLE(DebugLuauTrackOwningArena, false) LUAU_FASTFLAG(LuauCaptureBrokenCommentSpans) -LUAU_FASTFLAG(LuauTypeAliasPacks) -LUAU_FASTFLAGVARIABLE(LuauCloneBoundTables, false) +LUAU_FASTINTVARIABLE(LuauTypeCloneRecursionLimit, 0) namespace Luau { @@ -120,12 +120,6 @@ TypePackId TypeArena::addTypePack(TypePackVar tp) return allocated; } -using SeenTypes = std::unordered_map; -using SeenTypePacks = std::unordered_map; - -TypePackId clone(TypePackId tp, TypeArena& dest, SeenTypes& seenTypes, SeenTypePacks& seenTypePacks, bool* encounteredFreeType); -TypeId clone(TypeId tp, TypeArena& dest, SeenTypes& seenTypes, SeenTypePacks& seenTypePacks, bool* encounteredFreeType); - namespace { @@ -138,11 +132,12 @@ struct TypePackCloner; struct TypeCloner { - TypeCloner(TypeArena& dest, TypeId typeId, SeenTypes& seenTypes, SeenTypePacks& seenTypePacks) + TypeCloner(TypeArena& dest, TypeId typeId, SeenTypes& seenTypes, SeenTypePacks& seenTypePacks, CloneState& cloneState) : dest(dest) , typeId(typeId) , seenTypes(seenTypes) , seenTypePacks(seenTypePacks) + , cloneState(cloneState) { } @@ -150,8 +145,7 @@ struct TypeCloner TypeId typeId; SeenTypes& seenTypes; SeenTypePacks& seenTypePacks; - - bool* encounteredFreeType = nullptr; + CloneState& cloneState; template void defaultClone(const T& t); @@ -178,13 +172,14 @@ struct TypePackCloner TypePackId typePackId; SeenTypes& seenTypes; SeenTypePacks& seenTypePacks; - bool* encounteredFreeType = nullptr; + CloneState& cloneState; - TypePackCloner(TypeArena& dest, TypePackId typePackId, SeenTypes& seenTypes, SeenTypePacks& seenTypePacks) + TypePackCloner(TypeArena& dest, TypePackId typePackId, SeenTypes& seenTypes, SeenTypePacks& seenTypePacks, CloneState& cloneState) : dest(dest) , typePackId(typePackId) , seenTypes(seenTypes) , seenTypePacks(seenTypePacks) + , cloneState(cloneState) { } @@ -197,8 +192,7 @@ struct TypePackCloner void operator()(const Unifiable::Free& t) { - if (encounteredFreeType) - *encounteredFreeType = true; + cloneState.encounteredFreeType = true; TypePackId err = singletonTypes.errorRecoveryTypePack(singletonTypes.anyTypePack); TypePackId cloned = dest.addTypePack(*err); @@ -218,13 +212,13 @@ struct TypePackCloner // We just need to be sure that we rewrite pointers both to the binder and the bindee to the same pointer. void operator()(const Unifiable::Bound& t) { - TypePackId cloned = clone(t.boundTo, dest, seenTypes, seenTypePacks, encounteredFreeType); + TypePackId cloned = clone(t.boundTo, dest, seenTypes, seenTypePacks, cloneState); seenTypePacks[typePackId] = cloned; } void operator()(const VariadicTypePack& t) { - TypePackId cloned = dest.addTypePack(TypePackVar{VariadicTypePack{clone(t.ty, dest, seenTypes, seenTypePacks, encounteredFreeType)}}); + TypePackId cloned = dest.addTypePack(TypePackVar{VariadicTypePack{clone(t.ty, dest, seenTypes, seenTypePacks, cloneState)}}); seenTypePacks[typePackId] = cloned; } @@ -236,10 +230,10 @@ struct TypePackCloner seenTypePacks[typePackId] = cloned; for (TypeId ty : t.head) - destTp->head.push_back(clone(ty, dest, seenTypes, seenTypePacks, encounteredFreeType)); + destTp->head.push_back(clone(ty, dest, seenTypes, seenTypePacks, cloneState)); if (t.tail) - destTp->tail = clone(*t.tail, dest, seenTypes, seenTypePacks, encounteredFreeType); + destTp->tail = clone(*t.tail, dest, seenTypes, seenTypePacks, cloneState); } }; @@ -252,8 +246,7 @@ void TypeCloner::defaultClone(const T& t) void TypeCloner::operator()(const Unifiable::Free& t) { - if (encounteredFreeType) - *encounteredFreeType = true; + cloneState.encounteredFreeType = true; TypeId err = singletonTypes.errorRecoveryType(singletonTypes.anyType); TypeId cloned = dest.addType(*err); seenTypes[typeId] = cloned; @@ -266,7 +259,7 @@ void TypeCloner::operator()(const Unifiable::Generic& t) void TypeCloner::operator()(const Unifiable::Bound& t) { - TypeId boundTo = clone(t.boundTo, dest, seenTypes, seenTypePacks, encounteredFreeType); + TypeId boundTo = clone(t.boundTo, dest, seenTypes, seenTypePacks, cloneState); seenTypes[typeId] = boundTo; } @@ -294,23 +287,23 @@ void TypeCloner::operator()(const FunctionTypeVar& t) seenTypes[typeId] = result; for (TypeId generic : t.generics) - ftv->generics.push_back(clone(generic, dest, seenTypes, seenTypePacks, encounteredFreeType)); + ftv->generics.push_back(clone(generic, dest, seenTypes, seenTypePacks, cloneState)); for (TypePackId genericPack : t.genericPacks) - ftv->genericPacks.push_back(clone(genericPack, dest, seenTypes, seenTypePacks, encounteredFreeType)); + ftv->genericPacks.push_back(clone(genericPack, dest, seenTypes, seenTypePacks, cloneState)); ftv->tags = t.tags; - ftv->argTypes = clone(t.argTypes, dest, seenTypes, seenTypePacks, encounteredFreeType); + ftv->argTypes = clone(t.argTypes, dest, seenTypes, seenTypePacks, cloneState); ftv->argNames = t.argNames; - ftv->retType = clone(t.retType, dest, seenTypes, seenTypePacks, encounteredFreeType); + ftv->retType = clone(t.retType, dest, seenTypes, seenTypePacks, cloneState); } void TypeCloner::operator()(const TableTypeVar& t) { // If table is now bound to another one, we ignore the content of the original - if (FFlag::LuauCloneBoundTables && t.boundTo) + if (t.boundTo) { - TypeId boundTo = clone(*t.boundTo, dest, seenTypes, seenTypePacks, encounteredFreeType); + TypeId boundTo = clone(*t.boundTo, dest, seenTypes, seenTypePacks, cloneState); seenTypes[typeId] = boundTo; return; } @@ -326,34 +319,21 @@ void TypeCloner::operator()(const TableTypeVar& t) ttv->level = TypeLevel{0, 0}; for (const auto& [name, prop] : t.props) - ttv->props[name] = {clone(prop.type, dest, seenTypes, seenTypePacks, encounteredFreeType), prop.deprecated, {}, prop.location, prop.tags}; + ttv->props[name] = {clone(prop.type, dest, seenTypes, seenTypePacks, cloneState), prop.deprecated, {}, prop.location, prop.tags}; if (t.indexer) - ttv->indexer = TableIndexer{clone(t.indexer->indexType, dest, seenTypes, seenTypePacks, encounteredFreeType), - clone(t.indexer->indexResultType, dest, seenTypes, seenTypePacks, encounteredFreeType)}; - - if (!FFlag::LuauCloneBoundTables) - { - if (t.boundTo) - ttv->boundTo = clone(*t.boundTo, dest, seenTypes, seenTypePacks, encounteredFreeType); - } + ttv->indexer = TableIndexer{clone(t.indexer->indexType, dest, seenTypes, seenTypePacks, cloneState), + clone(t.indexer->indexResultType, dest, seenTypes, seenTypePacks, cloneState)}; for (TypeId& arg : ttv->instantiatedTypeParams) - arg = clone(arg, dest, seenTypes, seenTypePacks, encounteredFreeType); + arg = clone(arg, dest, seenTypes, seenTypePacks, cloneState); - if (FFlag::LuauTypeAliasPacks) - { - for (TypePackId& arg : ttv->instantiatedTypePackParams) - arg = clone(arg, dest, seenTypes, seenTypePacks, encounteredFreeType); - } + for (TypePackId& arg : ttv->instantiatedTypePackParams) + arg = clone(arg, dest, seenTypes, seenTypePacks, cloneState); if (ttv->state == TableState::Free) { - if (FFlag::LuauCloneBoundTables || !t.boundTo) - { - if (encounteredFreeType) - *encounteredFreeType = true; - } + cloneState.encounteredFreeType = true; ttv->state = TableState::Sealed; } @@ -369,8 +349,8 @@ void TypeCloner::operator()(const MetatableTypeVar& t) MetatableTypeVar* mtv = getMutable(result); seenTypes[typeId] = result; - mtv->table = clone(t.table, dest, seenTypes, seenTypePacks, encounteredFreeType); - mtv->metatable = clone(t.metatable, dest, seenTypes, seenTypePacks, encounteredFreeType); + mtv->table = clone(t.table, dest, seenTypes, seenTypePacks, cloneState); + mtv->metatable = clone(t.metatable, dest, seenTypes, seenTypePacks, cloneState); } void TypeCloner::operator()(const ClassTypeVar& t) @@ -381,13 +361,13 @@ void TypeCloner::operator()(const ClassTypeVar& t) seenTypes[typeId] = result; for (const auto& [name, prop] : t.props) - ctv->props[name] = {clone(prop.type, dest, seenTypes, seenTypePacks, encounteredFreeType), prop.deprecated, {}, prop.location, prop.tags}; + ctv->props[name] = {clone(prop.type, dest, seenTypes, seenTypePacks, cloneState), prop.deprecated, {}, prop.location, prop.tags}; if (t.parent) - ctv->parent = clone(*t.parent, dest, seenTypes, seenTypePacks, encounteredFreeType); + ctv->parent = clone(*t.parent, dest, seenTypes, seenTypePacks, cloneState); if (t.metatable) - ctv->metatable = clone(*t.metatable, dest, seenTypes, seenTypePacks, encounteredFreeType); + ctv->metatable = clone(*t.metatable, dest, seenTypes, seenTypePacks, cloneState); } void TypeCloner::operator()(const AnyTypeVar& t) @@ -404,7 +384,7 @@ void TypeCloner::operator()(const UnionTypeVar& t) LUAU_ASSERT(option != nullptr); for (TypeId ty : t.options) - option->options.push_back(clone(ty, dest, seenTypes, seenTypePacks, encounteredFreeType)); + option->options.push_back(clone(ty, dest, seenTypes, seenTypePacks, cloneState)); } void TypeCloner::operator()(const IntersectionTypeVar& t) @@ -416,7 +396,7 @@ void TypeCloner::operator()(const IntersectionTypeVar& t) LUAU_ASSERT(option != nullptr); for (TypeId ty : t.parts) - option->parts.push_back(clone(ty, dest, seenTypes, seenTypePacks, encounteredFreeType)); + option->parts.push_back(clone(ty, dest, seenTypes, seenTypePacks, cloneState)); } void TypeCloner::operator()(const LazyTypeVar& t) @@ -426,17 +406,18 @@ void TypeCloner::operator()(const LazyTypeVar& t) } // anonymous namespace -TypePackId clone(TypePackId tp, TypeArena& dest, SeenTypes& seenTypes, SeenTypePacks& seenTypePacks, bool* encounteredFreeType) +TypePackId clone(TypePackId tp, TypeArena& dest, SeenTypes& seenTypes, SeenTypePacks& seenTypePacks, CloneState& cloneState) { if (tp->persistent) return tp; + RecursionLimiter _ra(&cloneState.recursionCount, FInt::LuauTypeCloneRecursionLimit); + TypePackId& res = seenTypePacks[tp]; if (res == nullptr) { - TypePackCloner cloner{dest, tp, seenTypes, seenTypePacks}; - cloner.encounteredFreeType = encounteredFreeType; + TypePackCloner cloner{dest, tp, seenTypes, seenTypePacks, cloneState}; Luau::visit(cloner, tp->ty); // Mutates the storage that 'res' points into. } @@ -446,17 +427,18 @@ TypePackId clone(TypePackId tp, TypeArena& dest, SeenTypes& seenTypes, SeenTypeP return res; } -TypeId clone(TypeId typeId, TypeArena& dest, SeenTypes& seenTypes, SeenTypePacks& seenTypePacks, bool* encounteredFreeType) +TypeId clone(TypeId typeId, TypeArena& dest, SeenTypes& seenTypes, SeenTypePacks& seenTypePacks, CloneState& cloneState) { if (typeId->persistent) return typeId; + RecursionLimiter _ra(&cloneState.recursionCount, FInt::LuauTypeCloneRecursionLimit); + TypeId& res = seenTypes[typeId]; if (res == nullptr) { - TypeCloner cloner{dest, typeId, seenTypes, seenTypePacks}; - cloner.encounteredFreeType = encounteredFreeType; + TypeCloner cloner{dest, typeId, seenTypes, seenTypePacks, cloneState}; Luau::visit(cloner, typeId->ty); // Mutates the storage that 'res' points into. asMutable(res)->documentationSymbol = typeId->documentationSymbol; } @@ -467,19 +449,16 @@ TypeId clone(TypeId typeId, TypeArena& dest, SeenTypes& seenTypes, SeenTypePacks return res; } -TypeFun clone(const TypeFun& typeFun, TypeArena& dest, SeenTypes& seenTypes, SeenTypePacks& seenTypePacks, bool* encounteredFreeType) +TypeFun clone(const TypeFun& typeFun, TypeArena& dest, SeenTypes& seenTypes, SeenTypePacks& seenTypePacks, CloneState& cloneState) { TypeFun result; for (TypeId ty : typeFun.typeParams) - result.typeParams.push_back(clone(ty, dest, seenTypes, seenTypePacks, encounteredFreeType)); + result.typeParams.push_back(clone(ty, dest, seenTypes, seenTypePacks, cloneState)); - if (FFlag::LuauTypeAliasPacks) - { - for (TypePackId tp : typeFun.typePackParams) - result.typePackParams.push_back(clone(tp, dest, seenTypes, seenTypePacks, encounteredFreeType)); - } + for (TypePackId tp : typeFun.typePackParams) + result.typePackParams.push_back(clone(tp, dest, seenTypes, seenTypePacks, cloneState)); - result.type = clone(typeFun.type, dest, seenTypes, seenTypePacks, encounteredFreeType); + result.type = clone(typeFun.type, dest, seenTypes, seenTypePacks, cloneState); return result; } @@ -519,19 +498,18 @@ bool Module::clonePublicInterface() LUAU_ASSERT(interfaceTypes.typeVars.empty()); LUAU_ASSERT(interfaceTypes.typePacks.empty()); - bool encounteredFreeType = false; - - SeenTypePacks seenTypePacks; SeenTypes seenTypes; + SeenTypePacks seenTypePacks; + CloneState cloneState; ScopePtr moduleScope = getModuleScope(); - moduleScope->returnType = clone(moduleScope->returnType, interfaceTypes, seenTypes, seenTypePacks, &encounteredFreeType); + moduleScope->returnType = clone(moduleScope->returnType, interfaceTypes, seenTypes, seenTypePacks, cloneState); if (moduleScope->varargPack) - moduleScope->varargPack = clone(*moduleScope->varargPack, interfaceTypes, seenTypes, seenTypePacks, &encounteredFreeType); + moduleScope->varargPack = clone(*moduleScope->varargPack, interfaceTypes, seenTypes, seenTypePacks, cloneState); for (auto& pair : moduleScope->exportedTypeBindings) - pair.second = clone(pair.second, interfaceTypes, seenTypes, seenTypePacks, &encounteredFreeType); + pair.second = clone(pair.second, interfaceTypes, seenTypes, seenTypePacks, cloneState); for (TypeId ty : moduleScope->returnType) if (get(follow(ty))) @@ -540,7 +518,7 @@ bool Module::clonePublicInterface() freeze(internalTypes); freeze(interfaceTypes); - return encounteredFreeType; + return cloneState.encounteredFreeType; } } // namespace Luau diff --git a/Analysis/src/Quantify.cpp b/Analysis/src/Quantify.cpp index bf6d81aa..c773e208 100644 --- a/Analysis/src/Quantify.cpp +++ b/Analysis/src/Quantify.cpp @@ -4,6 +4,8 @@ #include "Luau/VisitTypeVar.h" +LUAU_FASTFLAGVARIABLE(LuauQuantifyVisitOnce, false) + namespace Luau { @@ -79,7 +81,16 @@ struct Quantifier void quantify(ModulePtr module, TypeId ty, TypeLevel level) { Quantifier q{std::move(module), level}; - visitTypeVar(ty, q); + + if (FFlag::LuauQuantifyVisitOnce) + { + DenseHashSet seen{nullptr}; + visitTypeVarOnce(ty, q, seen); + } + else + { + visitTypeVar(ty, q); + } FunctionTypeVar* ftv = getMutable(ty); LUAU_ASSERT(ftv); diff --git a/Analysis/src/RequireTracer.cpp b/Analysis/src/RequireTracer.cpp index b72f53f9..8ed245fb 100644 --- a/Analysis/src/RequireTracer.cpp +++ b/Analysis/src/RequireTracer.cpp @@ -4,182 +4,9 @@ #include "Luau/Ast.h" #include "Luau/Module.h" -LUAU_FASTFLAGVARIABLE(LuauTraceRequireLookupChild, false) -LUAU_FASTFLAGVARIABLE(LuauNewRequireTrace2, false) - namespace Luau { -namespace -{ - -struct RequireTracerOld : AstVisitor -{ - explicit RequireTracerOld(FileResolver* fileResolver, const ModuleName& currentModuleName) - : fileResolver(fileResolver) - , currentModuleName(currentModuleName) - { - LUAU_ASSERT(!FFlag::LuauNewRequireTrace2); - } - - FileResolver* const fileResolver; - ModuleName currentModuleName; - DenseHashMap locals{nullptr}; - RequireTraceResult result; - - std::optional fromAstFragment(AstExpr* expr) - { - if (auto g = expr->as(); g && g->name == "script") - return currentModuleName; - - return fileResolver->fromAstFragment(expr); - } - - bool visit(AstStatLocal* stat) override - { - for (size_t i = 0; i < stat->vars.size; ++i) - { - AstLocal* local = stat->vars.data[i]; - - if (local->annotation) - { - if (AstTypeTypeof* ann = local->annotation->as()) - ann->expr->visit(this); - } - - if (i < stat->values.size) - { - AstExpr* expr = stat->values.data[i]; - expr->visit(this); - - const ModuleInfo* info = result.exprs.find(expr); - if (info) - locals[local] = info->name; - } - } - - return false; - } - - bool visit(AstExprGlobal* global) override - { - std::optional name = fromAstFragment(global); - if (name) - result.exprs[global] = {*name}; - - return false; - } - - bool visit(AstExprLocal* local) override - { - const ModuleName* name = locals.find(local->local); - if (name) - result.exprs[local] = {*name}; - - return false; - } - - bool visit(AstExprIndexName* indexName) override - { - indexName->expr->visit(this); - - const ModuleInfo* info = result.exprs.find(indexName->expr); - if (info) - { - if (indexName->index == "parent" || indexName->index == "Parent") - { - if (auto parent = fileResolver->getParentModuleName(info->name)) - result.exprs[indexName] = {*parent}; - } - else - result.exprs[indexName] = {fileResolver->concat(info->name, indexName->index.value)}; - } - - return false; - } - - bool visit(AstExprIndexExpr* indexExpr) override - { - indexExpr->expr->visit(this); - - const ModuleInfo* info = result.exprs.find(indexExpr->expr); - const AstExprConstantString* str = indexExpr->index->as(); - if (info && str) - { - result.exprs[indexExpr] = {fileResolver->concat(info->name, std::string_view(str->value.data, str->value.size))}; - } - - indexExpr->index->visit(this); - - return false; - } - - bool visit(AstExprTypeAssertion* expr) override - { - return false; - } - - // If we see game:GetService("StringLiteral") or Game:GetService("StringLiteral"), then rewrite to game.StringLiteral. - // Else traverse arguments and trace requires to them. - bool visit(AstExprCall* call) override - { - for (AstExpr* arg : call->args) - arg->visit(this); - - call->func->visit(this); - - AstExprGlobal* globalName = call->func->as(); - if (globalName && globalName->name == "require" && call->args.size >= 1) - { - if (const ModuleInfo* moduleInfo = result.exprs.find(call->args.data[0])) - result.requires.push_back({moduleInfo->name, call->location}); - - return false; - } - - AstExprIndexName* indexName = call->func->as(); - if (!indexName) - return false; - - std::optional rootName = fromAstFragment(indexName->expr); - - if (FFlag::LuauTraceRequireLookupChild && !rootName) - { - if (const ModuleInfo* moduleInfo = result.exprs.find(indexName->expr)) - rootName = moduleInfo->name; - } - - if (!rootName) - return false; - - bool supportedLookup = indexName->index == "GetService" || - (FFlag::LuauTraceRequireLookupChild && (indexName->index == "FindFirstChild" || indexName->index == "WaitForChild")); - - if (!supportedLookup) - return false; - - if (call->args.size != 1) - return false; - - AstExprConstantString* name = call->args.data[0]->as(); - if (!name) - return false; - - std::string_view v{name->value.data, name->value.size}; - if (v.end() != std::find(v.begin(), v.end(), '/')) - return false; - - result.exprs[call] = {fileResolver->concat(*rootName, v)}; - - // 'WaitForChild' can be used on modules that are not available at the typecheck time, but will be available at runtime - // If we fail to find such module, we will not report an UnknownRequire error - if (FFlag::LuauTraceRequireLookupChild && indexName->index == "WaitForChild") - result.exprs[call].optional = true; - - return false; - } -}; - struct RequireTracer : AstVisitor { RequireTracer(RequireTraceResult& result, FileResolver* fileResolver, const ModuleName& currentModuleName) @@ -188,7 +15,6 @@ struct RequireTracer : AstVisitor , currentModuleName(currentModuleName) , locals(nullptr) { - LUAU_ASSERT(FFlag::LuauNewRequireTrace2); } bool visit(AstExprTypeAssertion* expr) override @@ -328,24 +154,13 @@ struct RequireTracer : AstVisitor std::vector requires; }; -} // anonymous namespace - RequireTraceResult traceRequires(FileResolver* fileResolver, AstStatBlock* root, const ModuleName& currentModuleName) { - if (FFlag::LuauNewRequireTrace2) - { - RequireTraceResult result; - RequireTracer tracer{result, fileResolver, currentModuleName}; - root->visit(&tracer); - tracer.process(); - return result; - } - else - { - RequireTracerOld tracer{fileResolver, currentModuleName}; - root->visit(&tracer); - return tracer.result; - } + RequireTraceResult result; + RequireTracer tracer{result, fileResolver, currentModuleName}; + root->visit(&tracer); + tracer.process(); + return result; } } // namespace Luau diff --git a/Analysis/src/Substitution.cpp b/Analysis/src/Substitution.cpp index ca2b30f5..3d004bee 100644 --- a/Analysis/src/Substitution.cpp +++ b/Analysis/src/Substitution.cpp @@ -7,8 +7,6 @@ #include LUAU_FASTINTVARIABLE(LuauTarjanChildLimit, 1000) -LUAU_FASTFLAGVARIABLE(LuauSubstitutionDontReplaceIgnoredTypes, false) -LUAU_FASTFLAG(LuauTypeAliasPacks) namespace Luau { @@ -39,11 +37,8 @@ void Tarjan::visitChildren(TypeId ty, int index) for (TypeId itp : ttv->instantiatedTypeParams) visitChild(itp); - if (FFlag::LuauTypeAliasPacks) - { - for (TypePackId itp : ttv->instantiatedTypePackParams) - visitChild(itp); - } + for (TypePackId itp : ttv->instantiatedTypePackParams) + visitChild(itp); } else if (const MetatableTypeVar* mtv = get(ty)) { @@ -339,10 +334,10 @@ std::optional Substitution::substitute(TypeId ty) return std::nullopt; for (auto [oldTy, newTy] : newTypes) - if (!FFlag::LuauSubstitutionDontReplaceIgnoredTypes || !ignoreChildren(oldTy)) + if (!ignoreChildren(oldTy)) replaceChildren(newTy); for (auto [oldTp, newTp] : newPacks) - if (!FFlag::LuauSubstitutionDontReplaceIgnoredTypes || !ignoreChildren(oldTp)) + if (!ignoreChildren(oldTp)) replaceChildren(newTp); TypeId newTy = replace(ty); return newTy; @@ -359,10 +354,10 @@ std::optional Substitution::substitute(TypePackId tp) return std::nullopt; for (auto [oldTy, newTy] : newTypes) - if (!FFlag::LuauSubstitutionDontReplaceIgnoredTypes || !ignoreChildren(oldTy)) + if (!ignoreChildren(oldTy)) replaceChildren(newTy); for (auto [oldTp, newTp] : newPacks) - if (!FFlag::LuauSubstitutionDontReplaceIgnoredTypes || !ignoreChildren(oldTp)) + if (!ignoreChildren(oldTp)) replaceChildren(newTp); TypePackId newTp = replace(tp); return newTp; @@ -393,10 +388,7 @@ TypeId Substitution::clone(TypeId ty) clone.name = ttv->name; clone.syntheticName = ttv->syntheticName; clone.instantiatedTypeParams = ttv->instantiatedTypeParams; - - if (FFlag::LuauTypeAliasPacks) - clone.instantiatedTypePackParams = ttv->instantiatedTypePackParams; - + clone.instantiatedTypePackParams = ttv->instantiatedTypePackParams; clone.tags = ttv->tags; result = addType(std::move(clone)); } @@ -505,11 +497,8 @@ void Substitution::replaceChildren(TypeId ty) for (TypeId& itp : ttv->instantiatedTypeParams) itp = replace(itp); - if (FFlag::LuauTypeAliasPacks) - { - for (TypePackId& itp : ttv->instantiatedTypePackParams) - itp = replace(itp); - } + for (TypePackId& itp : ttv->instantiatedTypePackParams) + itp = replace(itp); } else if (MetatableTypeVar* mtv = getMutable(ty)) { diff --git a/Analysis/src/ToDot.cpp b/Analysis/src/ToDot.cpp new file mode 100644 index 00000000..df9d4188 --- /dev/null +++ b/Analysis/src/ToDot.cpp @@ -0,0 +1,378 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +#include "Luau/ToDot.h" + +#include "Luau/ToString.h" +#include "Luau/TypePack.h" +#include "Luau/TypeVar.h" +#include "Luau/StringUtils.h" + +#include +#include + +namespace Luau +{ + +namespace +{ + +struct StateDot +{ + StateDot(ToDotOptions opts) + : opts(opts) + { + } + + ToDotOptions opts; + + std::unordered_set seenTy; + std::unordered_set seenTp; + std::unordered_map tyToIndex; + std::unordered_map tpToIndex; + int nextIndex = 1; + std::string result; + + bool canDuplicatePrimitive(TypeId ty); + + void visitChildren(TypeId ty, int index); + void visitChildren(TypePackId ty, int index); + + void visitChild(TypeId ty, int parentIndex, const char* linkName = nullptr); + void visitChild(TypePackId tp, int parentIndex, const char* linkName = nullptr); + + void startNode(int index); + void finishNode(); + + void startNodeLabel(); + void finishNodeLabel(TypeId ty); + void finishNodeLabel(TypePackId tp); +}; + +bool StateDot::canDuplicatePrimitive(TypeId ty) +{ + if (get(ty)) + return false; + + return get(ty) || get(ty); +} + +void StateDot::visitChild(TypeId ty, int parentIndex, const char* linkName) +{ + if (!tyToIndex.count(ty) || (opts.duplicatePrimitives && canDuplicatePrimitive(ty))) + tyToIndex[ty] = nextIndex++; + + int index = tyToIndex[ty]; + + if (parentIndex != 0) + { + if (linkName) + formatAppend(result, "n%d -> n%d [label=\"%s\"];\n", parentIndex, index, linkName); + else + formatAppend(result, "n%d -> n%d;\n", parentIndex, index); + } + + if (opts.duplicatePrimitives && canDuplicatePrimitive(ty)) + { + if (get(ty)) + formatAppend(result, "n%d [label=\"%s\"];\n", index, toStringDetailed(ty, {}).name.c_str()); + else if (get(ty)) + formatAppend(result, "n%d [label=\"any\"];\n", index); + } + else + { + visitChildren(ty, index); + } +} + +void StateDot::visitChild(TypePackId tp, int parentIndex, const char* linkName) +{ + if (!tpToIndex.count(tp)) + tpToIndex[tp] = nextIndex++; + + if (parentIndex != 0) + { + if (linkName) + formatAppend(result, "n%d -> n%d [label=\"%s\"];\n", parentIndex, tpToIndex[tp], linkName); + else + formatAppend(result, "n%d -> n%d;\n", parentIndex, tpToIndex[tp]); + } + + visitChildren(tp, tpToIndex[tp]); +} + +void StateDot::startNode(int index) +{ + formatAppend(result, "n%d [", index); +} + +void StateDot::finishNode() +{ + formatAppend(result, "];\n"); +} + +void StateDot::startNodeLabel() +{ + formatAppend(result, "label=\""); +} + +void StateDot::finishNodeLabel(TypeId ty) +{ + if (opts.showPointers) + formatAppend(result, "\n0x%p", ty); + // additional common attributes can be added here as well + result += "\""; +} + +void StateDot::finishNodeLabel(TypePackId tp) +{ + if (opts.showPointers) + formatAppend(result, "\n0x%p", tp); + // additional common attributes can be added here as well + result += "\""; +} + +void StateDot::visitChildren(TypeId ty, int index) +{ + if (seenTy.count(ty)) + return; + seenTy.insert(ty); + + startNode(index); + startNodeLabel(); + + if (const BoundTypeVar* btv = get(ty)) + { + formatAppend(result, "BoundTypeVar %d", index); + finishNodeLabel(ty); + finishNode(); + + visitChild(btv->boundTo, index); + } + else if (const FunctionTypeVar* ftv = get(ty)) + { + formatAppend(result, "FunctionTypeVar %d", index); + finishNodeLabel(ty); + finishNode(); + + visitChild(ftv->argTypes, index, "arg"); + visitChild(ftv->retType, index, "ret"); + } + else if (const TableTypeVar* ttv = get(ty)) + { + if (ttv->name) + formatAppend(result, "TableTypeVar %s", ttv->name->c_str()); + else if (ttv->syntheticName) + formatAppend(result, "TableTypeVar %s", ttv->syntheticName->c_str()); + else + formatAppend(result, "TableTypeVar %d", index); + finishNodeLabel(ty); + finishNode(); + + if (ttv->boundTo) + return visitChild(*ttv->boundTo, index, "boundTo"); + + for (const auto& [name, prop] : ttv->props) + visitChild(prop.type, index, name.c_str()); + if (ttv->indexer) + { + visitChild(ttv->indexer->indexType, index, "[index]"); + visitChild(ttv->indexer->indexResultType, index, "[value]"); + } + for (TypeId itp : ttv->instantiatedTypeParams) + visitChild(itp, index, "typeParam"); + + for (TypePackId itp : ttv->instantiatedTypePackParams) + visitChild(itp, index, "typePackParam"); + } + else if (const MetatableTypeVar* mtv = get(ty)) + { + formatAppend(result, "MetatableTypeVar %d", index); + finishNodeLabel(ty); + finishNode(); + + visitChild(mtv->table, index, "table"); + visitChild(mtv->metatable, index, "metatable"); + } + else if (const UnionTypeVar* utv = get(ty)) + { + formatAppend(result, "UnionTypeVar %d", index); + finishNodeLabel(ty); + finishNode(); + + for (TypeId opt : utv->options) + visitChild(opt, index); + } + else if (const IntersectionTypeVar* itv = get(ty)) + { + formatAppend(result, "IntersectionTypeVar %d", index); + finishNodeLabel(ty); + finishNode(); + + for (TypeId part : itv->parts) + visitChild(part, index); + } + else if (const GenericTypeVar* gtv = get(ty)) + { + if (gtv->explicitName) + formatAppend(result, "GenericTypeVar %s", gtv->name.c_str()); + else + formatAppend(result, "GenericTypeVar %d", index); + finishNodeLabel(ty); + finishNode(); + } + else if (const FreeTypeVar* ftv = get(ty)) + { + formatAppend(result, "FreeTypeVar %d", index); + finishNodeLabel(ty); + finishNode(); + } + else if (get(ty)) + { + formatAppend(result, "AnyTypeVar %d", index); + finishNodeLabel(ty); + finishNode(); + } + else if (get(ty)) + { + formatAppend(result, "PrimitiveTypeVar %s", toStringDetailed(ty, {}).name.c_str()); + finishNodeLabel(ty); + finishNode(); + } + else if (get(ty)) + { + formatAppend(result, "ErrorTypeVar %d", index); + finishNodeLabel(ty); + finishNode(); + } + else if (const ClassTypeVar* ctv = get(ty)) + { + formatAppend(result, "ClassTypeVar %s", ctv->name.c_str()); + finishNodeLabel(ty); + finishNode(); + + for (const auto& [name, prop] : ctv->props) + visitChild(prop.type, index, name.c_str()); + + if (ctv->parent) + visitChild(*ctv->parent, index, "[parent]"); + + if (ctv->metatable) + visitChild(*ctv->metatable, index, "[metatable]"); + } + else + { + LUAU_ASSERT(!"unknown type kind"); + finishNodeLabel(ty); + finishNode(); + } +} + +void StateDot::visitChildren(TypePackId tp, int index) +{ + if (seenTp.count(tp)) + return; + seenTp.insert(tp); + + startNode(index); + startNodeLabel(); + + if (const BoundTypePack* btp = get(tp)) + { + formatAppend(result, "BoundTypePack %d", index); + finishNodeLabel(tp); + finishNode(); + + visitChild(btp->boundTo, index); + } + else if (const TypePack* tpp = get(tp)) + { + formatAppend(result, "TypePack %d", index); + finishNodeLabel(tp); + finishNode(); + + for (TypeId tv : tpp->head) + visitChild(tv, index); + if (tpp->tail) + visitChild(*tpp->tail, index, "tail"); + } + else if (const VariadicTypePack* vtp = get(tp)) + { + formatAppend(result, "VariadicTypePack %d", index); + finishNodeLabel(tp); + finishNode(); + + visitChild(vtp->ty, index); + } + else if (const FreeTypePack* ftp = get(tp)) + { + formatAppend(result, "FreeTypePack %d", index); + finishNodeLabel(tp); + finishNode(); + } + else if (const GenericTypePack* gtp = get(tp)) + { + if (gtp->explicitName) + formatAppend(result, "GenericTypePack %s", gtp->name.c_str()); + else + formatAppend(result, "GenericTypePack %d", index); + finishNodeLabel(tp); + finishNode(); + } + else if (get(tp)) + { + formatAppend(result, "ErrorTypePack %d", index); + finishNodeLabel(tp); + finishNode(); + } + else + { + LUAU_ASSERT(!"unknown type pack kind"); + finishNodeLabel(tp); + finishNode(); + } +} + +} // namespace + +std::string toDot(TypeId ty, const ToDotOptions& opts) +{ + StateDot state{opts}; + + state.result = "digraph graphname {\n"; + state.visitChild(ty, 0); + state.result += "}"; + + return state.result; +} + +std::string toDot(TypePackId tp, const ToDotOptions& opts) +{ + StateDot state{opts}; + + state.result = "digraph graphname {\n"; + state.visitChild(tp, 0); + state.result += "}"; + + return state.result; +} + +std::string toDot(TypeId ty) +{ + return toDot(ty, {}); +} + +std::string toDot(TypePackId tp) +{ + return toDot(tp, {}); +} + +void dumpDot(TypeId ty) +{ + printf("%s\n", toDot(ty).c_str()); +} + +void dumpDot(TypePackId tp) +{ + printf("%s\n", toDot(tp).c_str()); +} + +} // namespace Luau diff --git a/Analysis/src/ToString.cpp b/Analysis/src/ToString.cpp index 735bfa50..6322096c 100644 --- a/Analysis/src/ToString.cpp +++ b/Analysis/src/ToString.cpp @@ -11,7 +11,7 @@ #include LUAU_FASTFLAG(LuauOccursCheckOkWithRecursiveFunctions) -LUAU_FASTFLAG(LuauTypeAliasPacks) +LUAU_FASTFLAGVARIABLE(LuauFunctionArgumentNameSize, false) namespace Luau { @@ -59,11 +59,8 @@ struct FindCyclicTypes for (TypeId itp : ttv.instantiatedTypeParams) visitTypeVar(itp, *this, seen); - if (FFlag::LuauTypeAliasPacks) - { - for (TypePackId itp : ttv.instantiatedTypePackParams) - visitTypeVar(itp, *this, seen); - } + for (TypePackId itp : ttv.instantiatedTypePackParams) + visitTypeVar(itp, *this, seen); return exhaustive; } @@ -248,58 +245,45 @@ struct TypeVarStringifier void stringify(const std::vector& types, const std::vector& typePacks) { - if (types.size() == 0 && (!FFlag::LuauTypeAliasPacks || typePacks.size() == 0)) + if (types.size() == 0 && typePacks.size() == 0) return; - if (types.size() || (FFlag::LuauTypeAliasPacks && typePacks.size())) + if (types.size() || typePacks.size()) state.emit("<"); - if (FFlag::LuauTypeAliasPacks) - { - bool first = true; + bool first = true; - for (TypeId ty : types) - { - if (!first) - state.emit(", "); + for (TypeId ty : types) + { + if (!first) + state.emit(", "); + first = false; + + stringify(ty); + } + + bool singleTp = typePacks.size() == 1; + + for (TypePackId tp : typePacks) + { + if (isEmpty(tp) && singleTp) + continue; + + if (!first) + state.emit(", "); + else first = false; - stringify(ty); - } + if (!singleTp) + state.emit("("); - bool singleTp = typePacks.size() == 1; + stringify(tp); - for (TypePackId tp : typePacks) - { - if (isEmpty(tp) && singleTp) - continue; - - if (!first) - state.emit(", "); - else - first = false; - - if (!singleTp) - state.emit("("); - - stringify(tp); - - if (!singleTp) - state.emit(")"); - } - } - else - { - for (size_t i = 0; i < types.size(); ++i) - { - if (i > 0) - state.emit(", "); - - stringify(types[i]); - } + if (!singleTp) + state.emit(")"); } - if (types.size() || (FFlag::LuauTypeAliasPacks && typePacks.size())) + if (types.size() || typePacks.size()) state.emit(">"); } @@ -767,12 +751,23 @@ struct TypePackStringifier else state.emit(", "); - LUAU_ASSERT(elemNames.empty() || elemIndex < elemNames.size()); - - if (!elemNames.empty() && elemNames[elemIndex]) + if (FFlag::LuauFunctionArgumentNameSize) { - state.emit(elemNames[elemIndex]->name); - state.emit(": "); + if (elemIndex < elemNames.size() && elemNames[elemIndex]) + { + state.emit(elemNames[elemIndex]->name); + state.emit(": "); + } + } + else + { + LUAU_ASSERT(elemNames.empty() || elemIndex < elemNames.size()); + + if (!elemNames.empty() && elemNames[elemIndex]) + { + state.emit(elemNames[elemIndex]->name); + state.emit(": "); + } } elemIndex++; @@ -929,38 +924,7 @@ ToStringResult toStringDetailed(TypeId ty, const ToStringOptions& opts) result.name += ttv->name ? *ttv->name : *ttv->syntheticName; - if (FFlag::LuauTypeAliasPacks) - { - tvs.stringify(ttv->instantiatedTypeParams, ttv->instantiatedTypePackParams); - } - else - { - if (ttv->instantiatedTypeParams.empty() && (!FFlag::LuauTypeAliasPacks || ttv->instantiatedTypePackParams.empty())) - return result; - - result.name += "<"; - - bool first = true; - for (TypeId ty : ttv->instantiatedTypeParams) - { - if (!first) - result.name += ", "; - else - first = false; - - tvs.stringify(ty); - } - - if (opts.maxTypeLength > 0 && result.name.length() > opts.maxTypeLength) - { - result.truncated = true; - result.name += "... "; - } - else - { - result.name += ">"; - } - } + tvs.stringify(ttv->instantiatedTypeParams, ttv->instantiatedTypePackParams); return result; } @@ -1161,17 +1125,37 @@ std::string toStringNamedFunction(const std::string& prefix, const FunctionTypeV s += ", "; first = false; - // argNames is guaranteed to be equal to argTypes iff argNames is not empty. - // We don't currently respect opts.functionTypeArguments. I don't think this function should. - if (!ftv.argNames.empty()) - s += (*argNameIter ? (*argNameIter)->name : "_") + ": "; - s += toString_(*argPackIter); - - ++argPackIter; - if (!ftv.argNames.empty()) + if (FFlag::LuauFunctionArgumentNameSize) { - LUAU_ASSERT(argNameIter != ftv.argNames.end()); - ++argNameIter; + // We don't currently respect opts.functionTypeArguments. I don't think this function should. + if (argNameIter != ftv.argNames.end()) + { + s += (*argNameIter ? (*argNameIter)->name : "_") + ": "; + ++argNameIter; + } + else + { + s += "_: "; + } + } + else + { + // argNames is guaranteed to be equal to argTypes iff argNames is not empty. + // We don't currently respect opts.functionTypeArguments. I don't think this function should. + if (!ftv.argNames.empty()) + s += (*argNameIter ? (*argNameIter)->name : "_") + ": "; + } + + s += toString_(*argPackIter); + ++argPackIter; + + if (!FFlag::LuauFunctionArgumentNameSize) + { + if (!ftv.argNames.empty()) + { + LUAU_ASSERT(argNameIter != ftv.argNames.end()); + ++argNameIter; + } } } diff --git a/Analysis/src/Transpiler.cpp b/Analysis/src/Transpiler.cpp index 6627fbe3..8e13ea5b 100644 --- a/Analysis/src/Transpiler.cpp +++ b/Analysis/src/Transpiler.cpp @@ -10,8 +10,6 @@ #include #include -LUAU_FASTFLAG(LuauTypeAliasPacks) - namespace { bool isIdentifierStartChar(char c) @@ -787,7 +785,7 @@ struct Printer writer.keyword("type"); writer.identifier(a->name.value); - if (a->generics.size > 0 || (FFlag::LuauTypeAliasPacks && a->genericPacks.size > 0)) + if (a->generics.size > 0 || a->genericPacks.size > 0) { writer.symbol("<"); CommaSeparatorInserter comma(writer); @@ -798,14 +796,11 @@ struct Printer writer.identifier(o.value); } - if (FFlag::LuauTypeAliasPacks) + for (auto o : a->genericPacks) { - for (auto o : a->genericPacks) - { - comma(); - writer.identifier(o.value); - writer.symbol("..."); - } + comma(); + writer.identifier(o.value); + writer.symbol("..."); } writer.symbol(">"); diff --git a/Analysis/src/TxnLog.cpp b/Analysis/src/TxnLog.cpp index 383bb050..f6a61581 100644 --- a/Analysis/src/TxnLog.cpp +++ b/Analysis/src/TxnLog.cpp @@ -5,8 +5,6 @@ #include -LUAU_FASTFLAGVARIABLE(LuauShareTxnSeen, false) - namespace Luau { @@ -36,11 +34,8 @@ void TxnLog::rollback() for (auto it = tableChanges.rbegin(); it != tableChanges.rend(); ++it) std::swap(it->first->boundTo, it->second); - if (FFlag::LuauShareTxnSeen) - { - LUAU_ASSERT(originalSeenSize <= sharedSeen->size()); - sharedSeen->resize(originalSeenSize); - } + LUAU_ASSERT(originalSeenSize <= sharedSeen->size()); + sharedSeen->resize(originalSeenSize); } void TxnLog::concat(TxnLog rhs) @@ -53,45 +48,25 @@ void TxnLog::concat(TxnLog rhs) tableChanges.insert(tableChanges.end(), rhs.tableChanges.begin(), rhs.tableChanges.end()); rhs.tableChanges.clear(); - - if (!FFlag::LuauShareTxnSeen) - { - ownedSeen.swap(rhs.ownedSeen); - rhs.ownedSeen.clear(); - } } bool TxnLog::haveSeen(TypeId lhs, TypeId rhs) { const std::pair sortedPair = (lhs > rhs) ? std::make_pair(lhs, rhs) : std::make_pair(rhs, lhs); - if (FFlag::LuauShareTxnSeen) - return (sharedSeen->end() != std::find(sharedSeen->begin(), sharedSeen->end(), sortedPair)); - else - return (ownedSeen.end() != std::find(ownedSeen.begin(), ownedSeen.end(), sortedPair)); + return (sharedSeen->end() != std::find(sharedSeen->begin(), sharedSeen->end(), sortedPair)); } void TxnLog::pushSeen(TypeId lhs, TypeId rhs) { const std::pair sortedPair = (lhs > rhs) ? std::make_pair(lhs, rhs) : std::make_pair(rhs, lhs); - if (FFlag::LuauShareTxnSeen) - sharedSeen->push_back(sortedPair); - else - ownedSeen.push_back(sortedPair); + sharedSeen->push_back(sortedPair); } void TxnLog::popSeen(TypeId lhs, TypeId rhs) { const std::pair sortedPair = (lhs > rhs) ? std::make_pair(lhs, rhs) : std::make_pair(rhs, lhs); - if (FFlag::LuauShareTxnSeen) - { - LUAU_ASSERT(sortedPair == sharedSeen->back()); - sharedSeen->pop_back(); - } - else - { - LUAU_ASSERT(sortedPair == ownedSeen.back()); - ownedSeen.pop_back(); - } + LUAU_ASSERT(sortedPair == sharedSeen->back()); + sharedSeen->pop_back(); } } // namespace Luau diff --git a/Analysis/src/TypeAttach.cpp b/Analysis/src/TypeAttach.cpp index af6d2543..9e61c792 100644 --- a/Analysis/src/TypeAttach.cpp +++ b/Analysis/src/TypeAttach.cpp @@ -13,8 +13,6 @@ #include -LUAU_FASTFLAG(LuauTypeAliasPacks) - static char* allocateString(Luau::Allocator& allocator, std::string_view contents) { char* result = (char*)allocator.allocate(contents.size() + 1); @@ -131,12 +129,9 @@ public: parameters.data[i] = {Luau::visit(*this, ttv.instantiatedTypeParams[i]->ty), {}}; } - if (FFlag::LuauTypeAliasPacks) + for (size_t i = 0; i < ttv.instantiatedTypePackParams.size(); ++i) { - for (size_t i = 0; i < ttv.instantiatedTypePackParams.size(); ++i) - { - parameters.data[i] = {{}, rehydrate(ttv.instantiatedTypePackParams[i])}; - } + parameters.data[i] = {{}, rehydrate(ttv.instantiatedTypePackParams[i])}; } return allocator->alloc(Location(), std::nullopt, AstName(ttv.name->c_str()), parameters.size != 0, parameters); @@ -250,20 +245,7 @@ public: AstTypePack* argTailAnnotation = nullptr; if (argTail) - { - if (FFlag::LuauTypeAliasPacks) - { - argTailAnnotation = rehydrate(*argTail); - } - else - { - TypePackId tail = *argTail; - if (const VariadicTypePack* vtp = get(tail)) - { - argTailAnnotation = allocator->alloc(Location(), Luau::visit(*this, vtp->ty->ty)); - } - } - } + argTailAnnotation = rehydrate(*argTail); AstArray> argNames; argNames.size = ftv.argNames.size(); @@ -292,20 +274,7 @@ public: AstTypePack* retTailAnnotation = nullptr; if (retTail) - { - if (FFlag::LuauTypeAliasPacks) - { - retTailAnnotation = rehydrate(*retTail); - } - else - { - TypePackId tail = *retTail; - if (const VariadicTypePack* vtp = get(tail)) - { - retTailAnnotation = allocator->alloc(Location(), Luau::visit(*this, vtp->ty->ty)); - } - } - } + retTailAnnotation = rehydrate(*retTail); return allocator->alloc( Location(), generics, genericPacks, AstTypeList{argTypes, argTailAnnotation}, argNames, AstTypeList{returnTypes, retTailAnnotation}); @@ -518,18 +487,7 @@ public: const auto& [v, tail] = flatten(ret); if (tail) - { - if (FFlag::LuauTypeAliasPacks) - { - variadicAnnotation = TypeRehydrationVisitor(allocator, &syntheticNames).rehydrate(*tail); - } - else - { - TypePackId tailPack = *tail; - if (const VariadicTypePack* vtp = get(tailPack)) - variadicAnnotation = allocator->alloc(Location(), typeAst(vtp->ty)); - } - } + variadicAnnotation = TypeRehydrationVisitor(allocator, &syntheticNames).rehydrate(*tail); fn->returnAnnotation = AstTypeList{typeAstPack(ret), variadicAnnotation}; } diff --git a/Analysis/src/TypeInfer.cpp b/Analysis/src/TypeInfer.cpp index b2ae94c7..617bf482 100644 --- a/Analysis/src/TypeInfer.cpp +++ b/Analysis/src/TypeInfer.cpp @@ -23,22 +23,20 @@ LUAU_FASTINTVARIABLE(LuauTypeInferRecursionLimit, 500) LUAU_FASTINTVARIABLE(LuauTypeInferTypePackLoopLimit, 5000) LUAU_FASTINTVARIABLE(LuauCheckRecursionLimit, 500) LUAU_FASTFLAG(LuauKnowsTheDataModel3) -LUAU_FASTFLAGVARIABLE(LuauClassPropertyAccessAsString, false) LUAU_FASTFLAGVARIABLE(LuauEqConstraint, false) LUAU_FASTFLAGVARIABLE(LuauWeakEqConstraint, false) // Eventually removed as false. -LUAU_FASTFLAG(LuauTraceRequireLookupChild) LUAU_FASTFLAGVARIABLE(LuauCloneCorrectlyBeforeMutatingTableType, false) LUAU_FASTFLAGVARIABLE(LuauStoreMatchingOverloadFnType, false) LUAU_FASTFLAGVARIABLE(LuauRecursiveTypeParameterRestriction, false) LUAU_FASTFLAGVARIABLE(LuauIfElseExpressionAnalysisSupport, false) LUAU_FASTFLAGVARIABLE(LuauStrictRequire, false) -LUAU_FASTFLAG(LuauSubstitutionDontReplaceIgnoredTypes) LUAU_FASTFLAGVARIABLE(LuauQuantifyInPlace2, false) -LUAU_FASTFLAG(LuauNewRequireTrace2) -LUAU_FASTFLAG(LuauTypeAliasPacks) LUAU_FASTFLAGVARIABLE(LuauSingletonTypes, false) LUAU_FASTFLAGVARIABLE(LuauExpectedTypesOfProperties, false) LUAU_FASTFLAGVARIABLE(LuauErrorRecoveryType, false) +LUAU_FASTFLAGVARIABLE(LuauPropertiesGetExpectedType, false) +LUAU_FASTFLAGVARIABLE(LuauTailArgumentTypeInfo, false) +LUAU_FASTFLAGVARIABLE(LuauModuleRequireErrorPack, false) namespace Luau { @@ -562,12 +560,6 @@ ErrorVec TypeChecker::canUnify(TypePackId left, TypePackId right, const Location return canUnify_(left, right, location); } -ErrorVec TypeChecker::canUnify(const std::vector>& seen, TypeId superTy, TypeId subTy, const Location& location) -{ - Unifier state = mkUnifier(seen, location); - return state.canUnify(superTy, subTy); -} - template ErrorVec TypeChecker::canUnify_(Id superTy, Id subTy, const Location& location) { @@ -1152,61 +1144,20 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatTypeAlias& typealias Location location = scope->typeAliasLocations[name]; reportError(TypeError{typealias.location, DuplicateTypeDefinition{name, location}}); - if (FFlag::LuauTypeAliasPacks) - bindingsMap[name] = TypeFun{binding->typeParams, binding->typePackParams, errorRecoveryType(anyType)}; - else - bindingsMap[name] = TypeFun{binding->typeParams, errorRecoveryType(anyType)}; + bindingsMap[name] = TypeFun{binding->typeParams, binding->typePackParams, errorRecoveryType(anyType)}; } else { ScopePtr aliasScope = FFlag::LuauQuantifyInPlace2 ? childScope(scope, typealias.location, subLevel) : childScope(scope, typealias.location); - if (FFlag::LuauTypeAliasPacks) - { - auto [generics, genericPacks] = createGenericTypes(aliasScope, scope->level, typealias, typealias.generics, typealias.genericPacks); + auto [generics, genericPacks] = createGenericTypes(aliasScope, scope->level, typealias, typealias.generics, typealias.genericPacks); - TypeId ty = freshType(aliasScope); - FreeTypeVar* ftv = getMutable(ty); - LUAU_ASSERT(ftv); - ftv->forwardedTypeAlias = true; - bindingsMap[name] = {std::move(generics), std::move(genericPacks), ty}; - } - else - { - std::vector generics; - for (AstName generic : typealias.generics) - { - Name n = generic.value; - - // These generics are the only thing that will ever be added to aliasScope, so we can be certain that - // a collision can only occur when two generic typevars have the same name. - if (aliasScope->privateTypeBindings.end() != aliasScope->privateTypeBindings.find(n)) - { - // TODO(jhuelsman): report the exact span of the generic type parameter whose name is a duplicate. - reportError(TypeError{typealias.location, DuplicateGenericParameter{n}}); - } - - TypeId g; - if (FFlag::LuauRecursiveTypeParameterRestriction) - { - TypeId& cached = scope->typeAliasTypeParameters[n]; - if (!cached) - cached = addType(GenericTypeVar{aliasScope->level, n}); - g = cached; - } - else - g = addType(GenericTypeVar{aliasScope->level, n}); - generics.push_back(g); - aliasScope->privateTypeBindings[n] = TypeFun{{}, g}; - } - - TypeId ty = freshType(aliasScope); - FreeTypeVar* ftv = getMutable(ty); - LUAU_ASSERT(ftv); - ftv->forwardedTypeAlias = true; - bindingsMap[name] = {std::move(generics), ty}; - } + TypeId ty = freshType(aliasScope); + FreeTypeVar* ftv = getMutable(ty); + LUAU_ASSERT(ftv); + ftv->forwardedTypeAlias = true; + bindingsMap[name] = {std::move(generics), std::move(genericPacks), ty}; } } else @@ -1223,14 +1174,11 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatTypeAlias& typealias aliasScope->privateTypeBindings[generic->name] = TypeFun{{}, ty}; } - if (FFlag::LuauTypeAliasPacks) + for (TypePackId tp : binding->typePackParams) { - for (TypePackId tp : binding->typePackParams) - { - auto generic = get(tp); - LUAU_ASSERT(generic); - aliasScope->privateTypePackBindings[generic->name] = tp; - } + auto generic = get(tp); + LUAU_ASSERT(generic); + aliasScope->privateTypePackBindings[generic->name] = tp; } TypeId ty = resolveType(aliasScope, *typealias.type); @@ -1241,19 +1189,16 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatTypeAlias& typealias { // Copy can be skipped if this is an identical alias if (ttv->name != name || ttv->instantiatedTypeParams != binding->typeParams || - (FFlag::LuauTypeAliasPacks && ttv->instantiatedTypePackParams != binding->typePackParams)) + ttv->instantiatedTypePackParams != binding->typePackParams) { // This is a shallow clone, original recursive links to self are not updated TableTypeVar clone = TableTypeVar{ttv->props, ttv->indexer, ttv->level, ttv->state}; clone.methodDefinitionLocations = ttv->methodDefinitionLocations; clone.definitionModuleName = ttv->definitionModuleName; - clone.name = name; clone.instantiatedTypeParams = binding->typeParams; - - if (FFlag::LuauTypeAliasPacks) - clone.instantiatedTypePackParams = binding->typePackParams; + clone.instantiatedTypePackParams = binding->typePackParams; ty = addType(std::move(clone)); } @@ -1262,9 +1207,7 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatTypeAlias& typealias { ttv->name = name; ttv->instantiatedTypeParams = binding->typeParams; - - if (FFlag::LuauTypeAliasPacks) - ttv->instantiatedTypePackParams = binding->typePackParams; + ttv->instantiatedTypePackParams = binding->typePackParams; } } else if (auto mtv = getMutable(follow(ty))) @@ -1289,7 +1232,7 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatDeclareClass& declar } // We don't have generic classes, so this assertion _should_ never be hit. - LUAU_ASSERT(lookupType->typeParams.size() == 0 && (!FFlag::LuauTypeAliasPacks || lookupType->typePackParams.size() == 0)); + LUAU_ASSERT(lookupType->typeParams.size() == 0 && lookupType->typePackParams.size() == 0); superTy = lookupType->type; if (!get(follow(*superTy))) @@ -1851,6 +1794,24 @@ TypeId TypeChecker::checkExprTable( if (isNonstrictMode() && !getTableType(exprType) && !get(exprType)) exprType = anyType; + if (FFlag::LuauPropertiesGetExpectedType && expectedTable) + { + auto it = expectedTable->props.find(key->value.data); + if (it != expectedTable->props.end()) + { + Property expectedProp = it->second; + ErrorVec errors = tryUnify(expectedProp.type, exprType, k->location); + if (errors.empty()) + exprType = expectedProp.type; + } + else if (expectedTable->indexer && isString(expectedTable->indexer->indexType)) + { + ErrorVec errors = tryUnify(expectedTable->indexer->indexResultType, exprType, k->location); + if (errors.empty()) + exprType = expectedTable->indexer->indexResultType; + } + } + props[key->value.data] = {exprType, /* deprecated */ false, {}, k->location}; } else @@ -3744,17 +3705,29 @@ ExprResult TypeChecker::checkExprList(const ScopePtr& scope, const L for (size_t i = 0; i < exprs.size; ++i) { AstExpr* expr = exprs.data[i]; + std::optional expectedType = i < expectedTypes.size() ? expectedTypes[i] : std::nullopt; if (i == lastIndex && (expr->is() || expr->is())) { auto [typePack, exprPredicates] = checkExprPack(scope, *expr); insert(exprPredicates); + if (FFlag::LuauTailArgumentTypeInfo) + { + if (std::optional firstTy = first(typePack)) + { + if (!currentModule->astTypes.find(expr)) + currentModule->astTypes[expr] = follow(*firstTy); + } + + if (expectedType) + currentModule->astExpectedTypes[expr] = *expectedType; + } + tp->tail = typePack; } else { - std::optional expectedType = i < expectedTypes.size() ? expectedTypes[i] : std::nullopt; auto [type, exprPredicates] = checkExpr(scope, *expr, expectedType); insert(exprPredicates); @@ -3797,7 +3770,7 @@ TypeId TypeChecker::checkRequire(const ScopePtr& scope, const ModuleInfo& module LUAU_TIMETRACE_SCOPE("TypeChecker::checkRequire", "TypeChecker"); LUAU_TIMETRACE_ARGUMENT("moduleInfo", moduleInfo.name.c_str()); - if (FFlag::LuauNewRequireTrace2 && moduleInfo.name.empty()) + if (moduleInfo.name.empty()) { if (FFlag::LuauStrictRequire && currentModule->mode == Mode::Strict) { @@ -3814,7 +3787,7 @@ TypeId TypeChecker::checkRequire(const ScopePtr& scope, const ModuleInfo& module // There are two reasons why we might fail to find the module: // either the file does not exist or there's a cycle. If there's a cycle // we will already have reported the error. - if (!resolver->moduleExists(moduleInfo.name) && (FFlag::LuauTraceRequireLookupChild ? !moduleInfo.optional : true)) + if (!resolver->moduleExists(moduleInfo.name) && !moduleInfo.optional) { std::string reportedModulePath = resolver->getHumanReadableModuleName(moduleInfo.name); reportError(TypeError{location, UnknownRequire{reportedModulePath}}); @@ -3830,7 +3803,12 @@ TypeId TypeChecker::checkRequire(const ScopePtr& scope, const ModuleInfo& module return errorRecoveryType(scope); } - std::optional moduleType = first(module->getModuleScope()->returnType); + TypePackId modulePack = module->getModuleScope()->returnType; + + if (FFlag::LuauModuleRequireErrorPack && get(modulePack)) + return errorRecoveryType(scope); + + std::optional moduleType = first(modulePack); if (!moduleType) { std::string humanReadableName = resolver->getHumanReadableModuleName(moduleInfo.name); @@ -3840,7 +3818,8 @@ TypeId TypeChecker::checkRequire(const ScopePtr& scope, const ModuleInfo& module SeenTypes seenTypes; SeenTypePacks seenTypePacks; - return clone(*moduleType, currentModule->internalTypes, seenTypes, seenTypePacks); + CloneState cloneState; + return clone(*moduleType, currentModule->internalTypes, seenTypes, seenTypePacks, cloneState); } void TypeChecker::tablify(TypeId type) @@ -4326,11 +4305,6 @@ Unifier TypeChecker::mkUnifier(const Location& location) return Unifier{¤tModule->internalTypes, currentModule->mode, globalScope, location, Variance::Covariant, unifierState}; } -Unifier TypeChecker::mkUnifier(const std::vector>& seen, const Location& location) -{ - return Unifier{¤tModule->internalTypes, currentModule->mode, globalScope, seen, location, Variance::Covariant, unifierState}; -} - TypeId TypeChecker::freshType(const ScopePtr& scope) { return freshType(scope->level); @@ -4477,117 +4451,82 @@ TypeId TypeChecker::resolveType(const ScopePtr& scope, const AstType& annotation return errorRecoveryType(scope); } - if (lit->parameters.size == 0 && tf->typeParams.empty() && (!FFlag::LuauTypeAliasPacks || tf->typePackParams.empty())) - { + if (lit->parameters.size == 0 && tf->typeParams.empty() && tf->typePackParams.empty()) return tf->type; - } - else if (!FFlag::LuauTypeAliasPacks && lit->parameters.size != tf->typeParams.size()) + + if (!lit->hasParameterList && !tf->typePackParams.empty()) { - reportError(TypeError{annotation.location, IncorrectGenericParameterCount{lit->name.value, *tf, lit->parameters.size, 0}}); + reportError(TypeError{annotation.location, GenericError{"Type parameter list is required"}}); if (!FFlag::LuauErrorRecoveryType) return errorRecoveryType(scope); } - if (FFlag::LuauTypeAliasPacks) + std::vector typeParams; + std::vector extraTypes; + std::vector typePackParams; + + for (size_t i = 0; i < lit->parameters.size; ++i) { - if (!lit->hasParameterList && !tf->typePackParams.empty()) + if (AstType* type = lit->parameters.data[i].type) { - reportError(TypeError{annotation.location, GenericError{"Type parameter list is required"}}); - if (!FFlag::LuauErrorRecoveryType) - return errorRecoveryType(scope); - } + TypeId ty = resolveType(scope, *type); - std::vector typeParams; - std::vector extraTypes; - std::vector typePackParams; - - for (size_t i = 0; i < lit->parameters.size; ++i) - { - if (AstType* type = lit->parameters.data[i].type) - { - TypeId ty = resolveType(scope, *type); - - if (typeParams.size() < tf->typeParams.size() || tf->typePackParams.empty()) - typeParams.push_back(ty); - else if (typePackParams.empty()) - extraTypes.push_back(ty); - else - reportError(TypeError{annotation.location, GenericError{"Type parameters must come before type pack parameters"}}); - } - else if (AstTypePack* typePack = lit->parameters.data[i].typePack) - { - TypePackId tp = resolveTypePack(scope, *typePack); - - // If we have collected an implicit type pack, materialize it - if (typePackParams.empty() && !extraTypes.empty()) - typePackParams.push_back(addTypePack(extraTypes)); - - // If we need more regular types, we can use single element type packs to fill those in - if (typeParams.size() < tf->typeParams.size() && size(tp) == 1 && finite(tp) && first(tp)) - typeParams.push_back(*first(tp)); - else - typePackParams.push_back(tp); - } - } - - // If we still haven't meterialized an implicit type pack, do it now - if (typePackParams.empty() && !extraTypes.empty()) - typePackParams.push_back(addTypePack(extraTypes)); - - // If we didn't combine regular types into a type pack and we're still one type pack short, provide an empty type pack - if (extraTypes.empty() && typePackParams.size() + 1 == tf->typePackParams.size()) - typePackParams.push_back(addTypePack({})); - - if (typeParams.size() != tf->typeParams.size() || typePackParams.size() != tf->typePackParams.size()) - { - reportError( - TypeError{annotation.location, IncorrectGenericParameterCount{lit->name.value, *tf, typeParams.size(), typePackParams.size()}}); - - if (FFlag::LuauErrorRecoveryType) - { - // Pad the types out with error recovery types - while (typeParams.size() < tf->typeParams.size()) - typeParams.push_back(errorRecoveryType(scope)); - while (typePackParams.size() < tf->typePackParams.size()) - typePackParams.push_back(errorRecoveryTypePack(scope)); - } + if (typeParams.size() < tf->typeParams.size() || tf->typePackParams.empty()) + typeParams.push_back(ty); + else if (typePackParams.empty()) + extraTypes.push_back(ty); else - return errorRecoveryType(scope); + reportError(TypeError{annotation.location, GenericError{"Type parameters must come before type pack parameters"}}); } - - if (FFlag::LuauRecursiveTypeParameterRestriction && typeParams == tf->typeParams && typePackParams == tf->typePackParams) + else if (AstTypePack* typePack = lit->parameters.data[i].typePack) { - // If the generic parameters and the type arguments are the same, we are about to - // perform an identity substitution, which we can just short-circuit. - return tf->type; + TypePackId tp = resolveTypePack(scope, *typePack); + + // If we have collected an implicit type pack, materialize it + if (typePackParams.empty() && !extraTypes.empty()) + typePackParams.push_back(addTypePack(extraTypes)); + + // If we need more regular types, we can use single element type packs to fill those in + if (typeParams.size() < tf->typeParams.size() && size(tp) == 1 && finite(tp) && first(tp)) + typeParams.push_back(*first(tp)); + else + typePackParams.push_back(tp); } - - return instantiateTypeFun(scope, *tf, typeParams, typePackParams, annotation.location); } - else - { - std::vector typeParams; - for (const auto& param : lit->parameters) - typeParams.push_back(resolveType(scope, *param.type)); + // If we still haven't meterialized an implicit type pack, do it now + if (typePackParams.empty() && !extraTypes.empty()) + typePackParams.push_back(addTypePack(extraTypes)); + + // If we didn't combine regular types into a type pack and we're still one type pack short, provide an empty type pack + if (extraTypes.empty() && typePackParams.size() + 1 == tf->typePackParams.size()) + typePackParams.push_back(addTypePack({})); + + if (typeParams.size() != tf->typeParams.size() || typePackParams.size() != tf->typePackParams.size()) + { + reportError( + TypeError{annotation.location, IncorrectGenericParameterCount{lit->name.value, *tf, typeParams.size(), typePackParams.size()}}); if (FFlag::LuauErrorRecoveryType) { - // If there aren't enough type parameters, pad them out with error recovery types - // (we've already reported the error) - while (typeParams.size() < lit->parameters.size) + // Pad the types out with error recovery types + while (typeParams.size() < tf->typeParams.size()) typeParams.push_back(errorRecoveryType(scope)); + while (typePackParams.size() < tf->typePackParams.size()) + typePackParams.push_back(errorRecoveryTypePack(scope)); } - - if (FFlag::LuauRecursiveTypeParameterRestriction && typeParams == tf->typeParams) - { - // If the generic parameters and the type arguments are the same, we are about to - // perform an identity substitution, which we can just short-circuit. - return tf->type; - } - - return instantiateTypeFun(scope, *tf, typeParams, {}, annotation.location); + else + return errorRecoveryType(scope); } + + if (FFlag::LuauRecursiveTypeParameterRestriction && typeParams == tf->typeParams && typePackParams == tf->typePackParams) + { + // If the generic parameters and the type arguments are the same, we are about to + // perform an identity substitution, which we can just short-circuit. + return tf->type; + } + + return instantiateTypeFun(scope, *tf, typeParams, typePackParams, annotation.location); } else if (const auto& table = annotation.as()) { @@ -4757,7 +4696,7 @@ bool ApplyTypeFunction::isDirty(TypePackId tp) bool ApplyTypeFunction::ignoreChildren(TypeId ty) { - if (FFlag::LuauSubstitutionDontReplaceIgnoredTypes && get(ty)) + if (get(ty)) return true; else return false; @@ -4765,7 +4704,7 @@ bool ApplyTypeFunction::ignoreChildren(TypeId ty) bool ApplyTypeFunction::ignoreChildren(TypePackId tp) { - if (FFlag::LuauSubstitutionDontReplaceIgnoredTypes && get(tp)) + if (get(tp)) return true; else return false; @@ -4788,36 +4727,26 @@ TypePackId ApplyTypeFunction::clean(TypePackId tp) // Really this should just replace the arguments, // but for bug-compatibility with existing code, we replace // all generics by free type variables. - if (FFlag::LuauTypeAliasPacks) - { - TypePackId& arg = typePackArguments[tp]; - if (arg) - return arg; - else - return addTypePack(FreeTypePack{level}); - } + TypePackId& arg = typePackArguments[tp]; + if (arg) + return arg; else - { return addTypePack(FreeTypePack{level}); - } } TypeId TypeChecker::instantiateTypeFun(const ScopePtr& scope, const TypeFun& tf, const std::vector& typeParams, const std::vector& typePackParams, const Location& location) { - if (tf.typeParams.empty() && (!FFlag::LuauTypeAliasPacks || tf.typePackParams.empty())) + if (tf.typeParams.empty() && tf.typePackParams.empty()) return tf.type; applyTypeFunction.typeArguments.clear(); for (size_t i = 0; i < tf.typeParams.size(); ++i) applyTypeFunction.typeArguments[tf.typeParams[i]] = typeParams[i]; - if (FFlag::LuauTypeAliasPacks) - { - applyTypeFunction.typePackArguments.clear(); - for (size_t i = 0; i < tf.typePackParams.size(); ++i) - applyTypeFunction.typePackArguments[tf.typePackParams[i]] = typePackParams[i]; - } + applyTypeFunction.typePackArguments.clear(); + for (size_t i = 0; i < tf.typePackParams.size(); ++i) + applyTypeFunction.typePackArguments[tf.typePackParams[i]] = typePackParams[i]; applyTypeFunction.currentModule = currentModule; applyTypeFunction.level = scope->level; @@ -4866,9 +4795,7 @@ TypeId TypeChecker::instantiateTypeFun(const ScopePtr& scope, const TypeFun& tf, if (ttv) { ttv->instantiatedTypeParams = typeParams; - - if (FFlag::LuauTypeAliasPacks) - ttv->instantiatedTypePackParams = typePackParams; + ttv->instantiatedTypePackParams = typePackParams; } } else @@ -4884,9 +4811,7 @@ TypeId TypeChecker::instantiateTypeFun(const ScopePtr& scope, const TypeFun& tf, } ttv->instantiatedTypeParams = typeParams; - - if (FFlag::LuauTypeAliasPacks) - ttv->instantiatedTypePackParams = typePackParams; + ttv->instantiatedTypePackParams = typePackParams; } } @@ -4914,7 +4839,7 @@ std::pair, std::vector> TypeChecker::createGener } TypeId g; - if (FFlag::LuauRecursiveTypeParameterRestriction && FFlag::LuauTypeAliasPacks) + if (FFlag::LuauRecursiveTypeParameterRestriction) { TypeId& cached = scope->parent->typeAliasTypeParameters[n]; if (!cached) @@ -4944,7 +4869,7 @@ std::pair, std::vector> TypeChecker::createGener } TypePackId g; - if (FFlag::LuauRecursiveTypeParameterRestriction && FFlag::LuauTypeAliasPacks) + if (FFlag::LuauRecursiveTypeParameterRestriction) { TypePackId& cached = scope->parent->typeAliasTypePackParameters[n]; if (!cached) @@ -5245,7 +5170,7 @@ void TypeChecker::resolve(const TypeGuardPredicate& typeguardP, ErrorVec& errVec return fail(UnknownSymbol{typeguardP.kind, UnknownSymbol::Type}); auto typeFun = globalScope->lookupType(typeguardP.kind); - if (!typeFun || !typeFun->typeParams.empty() || (FFlag::LuauTypeAliasPacks && !typeFun->typePackParams.empty())) + if (!typeFun || !typeFun->typeParams.empty() || !typeFun->typePackParams.empty()) return fail(UnknownSymbol{typeguardP.kind, UnknownSymbol::Type}); TypeId type = follow(typeFun->type); diff --git a/Analysis/src/TypeVar.cpp b/Analysis/src/TypeVar.cpp index 924bf082..62715af5 100644 --- a/Analysis/src/TypeVar.cpp +++ b/Analysis/src/TypeVar.cpp @@ -19,7 +19,6 @@ LUAU_FASTINTVARIABLE(LuauTypeMaximumStringifierLength, 500) LUAU_FASTINTVARIABLE(LuauTableTypeMaximumStringifierLength, 0) -LUAU_FASTFLAG(LuauTypeAliasPacks) LUAU_FASTFLAGVARIABLE(LuauRefactorTagging, false) LUAU_FASTFLAG(LuauErrorRecoveryType) @@ -739,369 +738,6 @@ void persist(TypePackId tp) } } -namespace -{ - -struct StateDot -{ - StateDot(ToDotOptions opts) - : opts(opts) - { - } - - ToDotOptions opts; - - std::unordered_set seenTy; - std::unordered_set seenTp; - std::unordered_map tyToIndex; - std::unordered_map tpToIndex; - int nextIndex = 1; - std::string result; - - bool canDuplicatePrimitive(TypeId ty); - - void visitChildren(TypeId ty, int index); - void visitChildren(TypePackId ty, int index); - - void visitChild(TypeId ty, int parentIndex, const char* linkName = nullptr); - void visitChild(TypePackId tp, int parentIndex, const char* linkName = nullptr); - - void startNode(int index); - void finishNode(); - - void startNodeLabel(); - void finishNodeLabel(TypeId ty); - void finishNodeLabel(TypePackId tp); -}; - -bool StateDot::canDuplicatePrimitive(TypeId ty) -{ - if (get(ty)) - return false; - - return get(ty) || get(ty); -} - -void StateDot::visitChild(TypeId ty, int parentIndex, const char* linkName) -{ - if (!tyToIndex.count(ty) || (opts.duplicatePrimitives && canDuplicatePrimitive(ty))) - tyToIndex[ty] = nextIndex++; - - int index = tyToIndex[ty]; - - if (parentIndex != 0) - { - if (linkName) - formatAppend(result, "n%d -> n%d [label=\"%s\"];\n", parentIndex, index, linkName); - else - formatAppend(result, "n%d -> n%d;\n", parentIndex, index); - } - - if (opts.duplicatePrimitives && canDuplicatePrimitive(ty)) - { - if (get(ty)) - formatAppend(result, "n%d [label=\"%s\"];\n", index, toStringDetailed(ty, {}).name.c_str()); - else if (get(ty)) - formatAppend(result, "n%d [label=\"any\"];\n", index); - } - else - { - visitChildren(ty, index); - } -} - -void StateDot::visitChild(TypePackId tp, int parentIndex, const char* linkName) -{ - if (!tpToIndex.count(tp)) - tpToIndex[tp] = nextIndex++; - - if (linkName) - formatAppend(result, "n%d -> n%d [label=\"%s\"];\n", parentIndex, tpToIndex[tp], linkName); - else - formatAppend(result, "n%d -> n%d;\n", parentIndex, tpToIndex[tp]); - - visitChildren(tp, tpToIndex[tp]); -} - -void StateDot::startNode(int index) -{ - formatAppend(result, "n%d [", index); -} - -void StateDot::finishNode() -{ - formatAppend(result, "];\n"); -} - -void StateDot::startNodeLabel() -{ - formatAppend(result, "label=\""); -} - -void StateDot::finishNodeLabel(TypeId ty) -{ - if (opts.showPointers) - formatAppend(result, "\n0x%p", ty); - // additional common attributes can be added here as well - result += "\""; -} - -void StateDot::finishNodeLabel(TypePackId tp) -{ - if (opts.showPointers) - formatAppend(result, "\n0x%p", tp); - // additional common attributes can be added here as well - result += "\""; -} - -void StateDot::visitChildren(TypeId ty, int index) -{ - if (seenTy.count(ty)) - return; - seenTy.insert(ty); - - startNode(index); - startNodeLabel(); - - if (const BoundTypeVar* btv = get(ty)) - { - formatAppend(result, "BoundTypeVar %d", index); - finishNodeLabel(ty); - finishNode(); - - visitChild(btv->boundTo, index); - } - else if (const FunctionTypeVar* ftv = get(ty)) - { - formatAppend(result, "FunctionTypeVar %d", index); - finishNodeLabel(ty); - finishNode(); - - visitChild(ftv->argTypes, index, "arg"); - visitChild(ftv->retType, index, "ret"); - } - else if (const TableTypeVar* ttv = get(ty)) - { - if (ttv->name) - formatAppend(result, "TableTypeVar %s", ttv->name->c_str()); - else if (ttv->syntheticName) - formatAppend(result, "TableTypeVar %s", ttv->syntheticName->c_str()); - else - formatAppend(result, "TableTypeVar %d", index); - finishNodeLabel(ty); - finishNode(); - - if (ttv->boundTo) - return visitChild(*ttv->boundTo, index, "boundTo"); - - for (const auto& [name, prop] : ttv->props) - visitChild(prop.type, index, name.c_str()); - if (ttv->indexer) - { - visitChild(ttv->indexer->indexType, index, "[index]"); - visitChild(ttv->indexer->indexResultType, index, "[value]"); - } - for (TypeId itp : ttv->instantiatedTypeParams) - visitChild(itp, index, "typeParam"); - - if (FFlag::LuauTypeAliasPacks) - { - for (TypePackId itp : ttv->instantiatedTypePackParams) - visitChild(itp, index, "typePackParam"); - } - } - else if (const MetatableTypeVar* mtv = get(ty)) - { - formatAppend(result, "MetatableTypeVar %d", index); - finishNodeLabel(ty); - finishNode(); - - visitChild(mtv->table, index, "table"); - visitChild(mtv->metatable, index, "metatable"); - } - else if (const UnionTypeVar* utv = get(ty)) - { - formatAppend(result, "UnionTypeVar %d", index); - finishNodeLabel(ty); - finishNode(); - - for (TypeId opt : utv->options) - visitChild(opt, index); - } - else if (const IntersectionTypeVar* itv = get(ty)) - { - formatAppend(result, "IntersectionTypeVar %d", index); - finishNodeLabel(ty); - finishNode(); - - for (TypeId part : itv->parts) - visitChild(part, index); - } - else if (const GenericTypeVar* gtv = get(ty)) - { - if (gtv->explicitName) - formatAppend(result, "GenericTypeVar %s", gtv->name.c_str()); - else - formatAppend(result, "GenericTypeVar %d", index); - finishNodeLabel(ty); - finishNode(); - } - else if (const FreeTypeVar* ftv = get(ty)) - { - formatAppend(result, "FreeTypeVar %d", ftv->index); - finishNodeLabel(ty); - finishNode(); - } - else if (get(ty)) - { - formatAppend(result, "AnyTypeVar %d", index); - finishNodeLabel(ty); - finishNode(); - } - else if (get(ty)) - { - formatAppend(result, "PrimitiveTypeVar %s", toStringDetailed(ty, {}).name.c_str()); - finishNodeLabel(ty); - finishNode(); - } - else if (get(ty)) - { - formatAppend(result, "ErrorTypeVar %d", index); - finishNodeLabel(ty); - finishNode(); - } - else if (const ClassTypeVar* ctv = get(ty)) - { - formatAppend(result, "ClassTypeVar %s", ctv->name.c_str()); - finishNodeLabel(ty); - finishNode(); - - for (const auto& [name, prop] : ctv->props) - visitChild(prop.type, index, name.c_str()); - - if (ctv->parent) - visitChild(*ctv->parent, index, "[parent]"); - - if (ctv->metatable) - visitChild(*ctv->metatable, index, "[metatable]"); - } - else - { - LUAU_ASSERT(!"unknown type kind"); - finishNodeLabel(ty); - finishNode(); - } -} - -void StateDot::visitChildren(TypePackId tp, int index) -{ - if (seenTp.count(tp)) - return; - seenTp.insert(tp); - - startNode(index); - startNodeLabel(); - - if (const BoundTypePack* btp = get(tp)) - { - formatAppend(result, "BoundTypePack %d", index); - finishNodeLabel(tp); - finishNode(); - - visitChild(btp->boundTo, index); - } - else if (const TypePack* tpp = get(tp)) - { - formatAppend(result, "TypePack %d", index); - finishNodeLabel(tp); - finishNode(); - - for (TypeId tv : tpp->head) - visitChild(tv, index); - if (tpp->tail) - visitChild(*tpp->tail, index, "tail"); - } - else if (const VariadicTypePack* vtp = get(tp)) - { - formatAppend(result, "VariadicTypePack %d", index); - finishNodeLabel(tp); - finishNode(); - - visitChild(vtp->ty, index); - } - else if (const FreeTypePack* ftp = get(tp)) - { - formatAppend(result, "FreeTypePack %d", ftp->index); - finishNodeLabel(tp); - finishNode(); - } - else if (const GenericTypePack* gtp = get(tp)) - { - if (gtp->explicitName) - formatAppend(result, "GenericTypePack %s", gtp->name.c_str()); - else - formatAppend(result, "GenericTypePack %d", gtp->index); - finishNodeLabel(tp); - finishNode(); - } - else if (get(tp)) - { - formatAppend(result, "ErrorTypePack %d", index); - finishNodeLabel(tp); - finishNode(); - } - else - { - LUAU_ASSERT(!"unknown type pack kind"); - finishNodeLabel(tp); - finishNode(); - } -} - -} // namespace - -std::string toDot(TypeId ty, const ToDotOptions& opts) -{ - StateDot state{opts}; - - state.result = "digraph graphname {\n"; - state.visitChild(ty, 0); - state.result += "}"; - - return state.result; -} - -std::string toDot(TypePackId tp, const ToDotOptions& opts) -{ - StateDot state{opts}; - - state.result = "digraph graphname {\n"; - state.visitChild(tp, 0); - state.result += "}"; - - return state.result; -} - -std::string toDot(TypeId ty) -{ - return toDot(ty, {}); -} - -std::string toDot(TypePackId tp) -{ - return toDot(tp, {}); -} - -void dumpDot(TypeId ty) -{ - printf("%s\n", toDot(ty).c_str()); -} - -void dumpDot(TypePackId tp) -{ - printf("%s\n", toDot(tp).c_str()); -} - const TypeLevel* getLevel(TypeId ty) { ty = follow(ty); diff --git a/Analysis/src/Unifier.cpp b/Analysis/src/Unifier.cpp index e1a52be4..d0b18837 100644 --- a/Analysis/src/Unifier.cpp +++ b/Analysis/src/Unifier.cpp @@ -18,9 +18,6 @@ LUAU_FASTFLAGVARIABLE(LuauTableSubtypingVariance, false); LUAU_FASTFLAGVARIABLE(LuauUnionHeuristic, false) LUAU_FASTFLAGVARIABLE(LuauTableUnificationEarlyTest, false) LUAU_FASTFLAGVARIABLE(LuauOccursCheckOkWithRecursiveFunctions, false) -LUAU_FASTFLAGVARIABLE(LuauTypecheckOpts, false) -LUAU_FASTFLAG(LuauShareTxnSeen); -LUAU_FASTFLAGVARIABLE(LuauCacheUnifyTableResults, false) LUAU_FASTFLAGVARIABLE(LuauExtendedTypeMismatchError, false) LUAU_FASTFLAG(LuauSingletonTypes) LUAU_FASTFLAGVARIABLE(LuauExtendedClassMismatchError, false) @@ -136,38 +133,19 @@ Unifier::Unifier(TypeArena* types, Mode mode, ScopePtr globalScope, const Locati , globalScope(std::move(globalScope)) , location(location) , variance(variance) - , counters(&countersData) - , counters_DEPRECATED(std::make_shared()) - , sharedState(sharedState) -{ - LUAU_ASSERT(sharedState.iceHandler); -} - -Unifier::Unifier(TypeArena* types, Mode mode, ScopePtr globalScope, const std::vector>& ownedSeen, const Location& location, - Variance variance, UnifierSharedState& sharedState, const std::shared_ptr& counters_DEPRECATED, UnifierCounters* counters) - : types(types) - , mode(mode) - , globalScope(std::move(globalScope)) - , log(ownedSeen) - , location(location) - , variance(variance) - , counters(counters ? counters : &countersData) - , counters_DEPRECATED(counters_DEPRECATED ? counters_DEPRECATED : std::make_shared()) , sharedState(sharedState) { LUAU_ASSERT(sharedState.iceHandler); } Unifier::Unifier(TypeArena* types, Mode mode, ScopePtr globalScope, std::vector>* sharedSeen, const Location& location, - Variance variance, UnifierSharedState& sharedState, const std::shared_ptr& counters_DEPRECATED, UnifierCounters* counters) + Variance variance, UnifierSharedState& sharedState) : types(types) , mode(mode) , globalScope(std::move(globalScope)) , log(sharedSeen) , location(location) , variance(variance) - , counters(counters ? counters : &countersData) - , counters_DEPRECATED(counters_DEPRECATED ? counters_DEPRECATED : std::make_shared()) , sharedState(sharedState) { LUAU_ASSERT(sharedState.iceHandler); @@ -175,26 +153,18 @@ Unifier::Unifier(TypeArena* types, Mode mode, ScopePtr globalScope, std::vector< void Unifier::tryUnify(TypeId superTy, TypeId subTy, bool isFunctionCall, bool isIntersection) { - if (FFlag::LuauTypecheckOpts) - counters->iterationCount = 0; - else - counters_DEPRECATED->iterationCount = 0; + sharedState.counters.iterationCount = 0; tryUnify_(superTy, subTy, isFunctionCall, isIntersection); } void Unifier::tryUnify_(TypeId superTy, TypeId subTy, bool isFunctionCall, bool isIntersection) { - RecursionLimiter _ra( - FFlag::LuauTypecheckOpts ? &counters->recursionCount : &counters_DEPRECATED->recursionCount, FInt::LuauTypeInferRecursionLimit); + RecursionLimiter _ra(&sharedState.counters.recursionCount, FInt::LuauTypeInferRecursionLimit); - if (FFlag::LuauTypecheckOpts) - ++counters->iterationCount; - else - ++counters_DEPRECATED->iterationCount; + ++sharedState.counters.iterationCount; - if (FInt::LuauTypeInferIterationLimit > 0 && - FInt::LuauTypeInferIterationLimit < (FFlag::LuauTypecheckOpts ? counters->iterationCount : counters_DEPRECATED->iterationCount)) + if (FInt::LuauTypeInferIterationLimit > 0 && FInt::LuauTypeInferIterationLimit < sharedState.counters.iterationCount) { errors.push_back(TypeError{location, UnificationTooComplex{}}); return; @@ -302,7 +272,7 @@ void Unifier::tryUnify_(TypeId superTy, TypeId subTy, bool isFunctionCall, bool if (get(subTy) || get(subTy)) return tryUnifyWithAny(subTy, superTy); - bool cacheEnabled = FFlag::LuauCacheUnifyTableResults && !isFunctionCall && !isIntersection; + bool cacheEnabled = !isFunctionCall && !isIntersection; auto& cache = sharedState.cachedUnify; // What if the types are immutable and we proved their relation before @@ -563,8 +533,6 @@ void Unifier::tryUnify_(TypeId superTy, TypeId subTy, bool isFunctionCall, bool void Unifier::cacheResult(TypeId superTy, TypeId subTy) { - LUAU_ASSERT(FFlag::LuauCacheUnifyTableResults); - bool* superTyInfo = sharedState.skipCacheForType.find(superTy); if (superTyInfo && *superTyInfo) @@ -686,10 +654,7 @@ ErrorVec Unifier::canUnify(TypePackId superTy, TypePackId subTy, bool isFunction void Unifier::tryUnify(TypePackId superTp, TypePackId subTp, bool isFunctionCall) { - if (FFlag::LuauTypecheckOpts) - counters->iterationCount = 0; - else - counters_DEPRECATED->iterationCount = 0; + sharedState.counters.iterationCount = 0; tryUnify_(superTp, subTp, isFunctionCall); } @@ -700,16 +665,11 @@ void Unifier::tryUnify(TypePackId superTp, TypePackId subTp, bool isFunctionCall */ void Unifier::tryUnify_(TypePackId superTp, TypePackId subTp, bool isFunctionCall) { - RecursionLimiter _ra( - FFlag::LuauTypecheckOpts ? &counters->recursionCount : &counters_DEPRECATED->recursionCount, FInt::LuauTypeInferRecursionLimit); + RecursionLimiter _ra(&sharedState.counters.recursionCount, FInt::LuauTypeInferRecursionLimit); - if (FFlag::LuauTypecheckOpts) - ++counters->iterationCount; - else - ++counters_DEPRECATED->iterationCount; + ++sharedState.counters.iterationCount; - if (FInt::LuauTypeInferIterationLimit > 0 && - FInt::LuauTypeInferIterationLimit < (FFlag::LuauTypecheckOpts ? counters->iterationCount : counters_DEPRECATED->iterationCount)) + if (FInt::LuauTypeInferIterationLimit > 0 && FInt::LuauTypeInferIterationLimit < sharedState.counters.iterationCount) { errors.push_back(TypeError{location, UnificationTooComplex{}}); return; @@ -1727,39 +1687,8 @@ void Unifier::tryUnify(const TableIndexer& superIndexer, const TableIndexer& sub tryUnify_(superIndexer.indexResultType, subIndexer.indexResultType); } -static void queueTypePack_DEPRECATED( - std::vector& queue, std::unordered_set& seenTypePacks, Unifier& state, TypePackId a, TypePackId anyTypePack) -{ - LUAU_ASSERT(!FFlag::LuauTypecheckOpts); - - while (true) - { - a = follow(a); - - if (seenTypePacks.count(a)) - break; - seenTypePacks.insert(a); - - if (get(a)) - { - state.log(a); - *asMutable(a) = Unifiable::Bound{anyTypePack}; - } - else if (auto tp = get(a)) - { - queue.insert(queue.end(), tp->head.begin(), tp->head.end()); - if (tp->tail) - a = *tp->tail; - else - break; - } - } -} - static void queueTypePack(std::vector& queue, DenseHashSet& seenTypePacks, Unifier& state, TypePackId a, TypePackId anyTypePack) { - LUAU_ASSERT(FFlag::LuauTypecheckOpts); - while (true) { a = follow(a); @@ -1837,66 +1766,9 @@ void Unifier::tryUnifyVariadics(TypePackId superTp, TypePackId subTp, bool rever } } -static void tryUnifyWithAny_DEPRECATED( - std::vector& queue, Unifier& state, std::unordered_set& seenTypePacks, TypeId anyType, TypePackId anyTypePack) -{ - LUAU_ASSERT(!FFlag::LuauTypecheckOpts); - - std::unordered_set seen; - - while (!queue.empty()) - { - TypeId ty = follow(queue.back()); - queue.pop_back(); - if (seen.count(ty)) - continue; - seen.insert(ty); - - if (get(ty)) - { - state.log(ty); - *asMutable(ty) = BoundTypeVar{anyType}; - } - else if (auto fun = get(ty)) - { - queueTypePack_DEPRECATED(queue, seenTypePacks, state, fun->argTypes, anyTypePack); - queueTypePack_DEPRECATED(queue, seenTypePacks, state, fun->retType, anyTypePack); - } - else if (auto table = get(ty)) - { - for (const auto& [_name, prop] : table->props) - queue.push_back(prop.type); - - if (table->indexer) - { - queue.push_back(table->indexer->indexType); - queue.push_back(table->indexer->indexResultType); - } - } - else if (auto mt = get(ty)) - { - queue.push_back(mt->table); - queue.push_back(mt->metatable); - } - else if (get(ty)) - { - // ClassTypeVars never contain free typevars. - } - else if (auto union_ = get(ty)) - queue.insert(queue.end(), union_->options.begin(), union_->options.end()); - else if (auto intersection = get(ty)) - queue.insert(queue.end(), intersection->parts.begin(), intersection->parts.end()); - else - { - } // Primitives, any, errors, and generics are left untouched. - } -} - static void tryUnifyWithAny(std::vector& queue, Unifier& state, DenseHashSet& seen, DenseHashSet& seenTypePacks, TypeId anyType, TypePackId anyTypePack) { - LUAU_ASSERT(FFlag::LuauTypecheckOpts); - while (!queue.empty()) { TypeId ty = follow(queue.back()); @@ -1949,43 +1821,20 @@ void Unifier::tryUnifyWithAny(TypeId any, TypeId ty) { LUAU_ASSERT(get(any) || get(any)); - if (FFlag::LuauTypecheckOpts) - { - // These types are not visited in general loop below - if (get(ty) || get(ty) || get(ty)) - return; - } + // These types are not visited in general loop below + if (get(ty) || get(ty) || get(ty)) + return; const TypePackId anyTypePack = types->addTypePack(TypePackVar{VariadicTypePack{singletonTypes.anyType}}); const TypePackId anyTP = get(any) ? anyTypePack : types->addTypePack(TypePackVar{Unifiable::Error{}}); - if (FFlag::LuauTypecheckOpts) - { - std::vector queue = {ty}; + std::vector queue = {ty}; - if (FFlag::LuauCacheUnifyTableResults) - { - sharedState.tempSeenTy.clear(); - sharedState.tempSeenTp.clear(); + sharedState.tempSeenTy.clear(); + sharedState.tempSeenTp.clear(); - Luau::tryUnifyWithAny(queue, *this, sharedState.tempSeenTy, sharedState.tempSeenTp, singletonTypes.anyType, anyTP); - } - else - { - tempSeenTy_DEPRECATED.clear(); - tempSeenTp_DEPRECATED.clear(); - - Luau::tryUnifyWithAny(queue, *this, tempSeenTy_DEPRECATED, tempSeenTp_DEPRECATED, singletonTypes.anyType, anyTP); - } - } - else - { - std::unordered_set seenTypePacks; - std::vector queue = {ty}; - - Luau::tryUnifyWithAny_DEPRECATED(queue, *this, seenTypePacks, singletonTypes.anyType, anyTP); - } + Luau::tryUnifyWithAny(queue, *this, sharedState.tempSeenTy, sharedState.tempSeenTp, singletonTypes.anyType, anyTP); } void Unifier::tryUnifyWithAny(TypePackId any, TypePackId ty) @@ -1994,38 +1843,14 @@ void Unifier::tryUnifyWithAny(TypePackId any, TypePackId ty) const TypeId anyTy = singletonTypes.errorRecoveryType(); - if (FFlag::LuauTypecheckOpts) - { - std::vector queue; + std::vector queue; - if (FFlag::LuauCacheUnifyTableResults) - { - sharedState.tempSeenTy.clear(); - sharedState.tempSeenTp.clear(); + sharedState.tempSeenTy.clear(); + sharedState.tempSeenTp.clear(); - queueTypePack(queue, sharedState.tempSeenTp, *this, ty, any); + queueTypePack(queue, sharedState.tempSeenTp, *this, ty, any); - Luau::tryUnifyWithAny(queue, *this, sharedState.tempSeenTy, sharedState.tempSeenTp, anyTy, any); - } - else - { - tempSeenTy_DEPRECATED.clear(); - tempSeenTp_DEPRECATED.clear(); - - queueTypePack(queue, tempSeenTp_DEPRECATED, *this, ty, any); - - Luau::tryUnifyWithAny(queue, *this, tempSeenTy_DEPRECATED, tempSeenTp_DEPRECATED, anyTy, any); - } - } - else - { - std::unordered_set seenTypePacks; - std::vector queue; - - queueTypePack_DEPRECATED(queue, seenTypePacks, *this, ty, any); - - Luau::tryUnifyWithAny_DEPRECATED(queue, *this, seenTypePacks, anyTy, any); - } + Luau::tryUnifyWithAny(queue, *this, sharedState.tempSeenTy, sharedState.tempSeenTp, anyTy, any); } std::optional Unifier::findTablePropertyRespectingMeta(TypeId lhsType, Name name) @@ -2035,46 +1860,22 @@ std::optional Unifier::findTablePropertyRespectingMeta(TypeId lhsType, N void Unifier::occursCheck(TypeId needle, TypeId haystack) { - std::unordered_set seen_DEPRECATED; + sharedState.tempSeenTy.clear(); - if (FFlag::LuauCacheUnifyTableResults) - { - if (FFlag::LuauTypecheckOpts) - sharedState.tempSeenTy.clear(); - - return occursCheck(seen_DEPRECATED, sharedState.tempSeenTy, needle, haystack); - } - else - { - if (FFlag::LuauTypecheckOpts) - tempSeenTy_DEPRECATED.clear(); - - return occursCheck(seen_DEPRECATED, tempSeenTy_DEPRECATED, needle, haystack); - } + return occursCheck(sharedState.tempSeenTy, needle, haystack); } -void Unifier::occursCheck(std::unordered_set& seen_DEPRECATED, DenseHashSet& seen, TypeId needle, TypeId haystack) +void Unifier::occursCheck(DenseHashSet& seen, TypeId needle, TypeId haystack) { - RecursionLimiter _ra( - FFlag::LuauTypecheckOpts ? &counters->recursionCount : &counters_DEPRECATED->recursionCount, FInt::LuauTypeInferRecursionLimit); + RecursionLimiter _ra(&sharedState.counters.recursionCount, FInt::LuauTypeInferRecursionLimit); needle = follow(needle); haystack = follow(haystack); - if (FFlag::LuauTypecheckOpts) - { - if (seen.find(haystack)) - return; + if (seen.find(haystack)) + return; - seen.insert(haystack); - } - else - { - if (seen_DEPRECATED.end() != seen_DEPRECATED.find(haystack)) - return; - - seen_DEPRECATED.insert(haystack); - } + seen.insert(haystack); if (get(needle)) return; @@ -2091,7 +1892,7 @@ void Unifier::occursCheck(std::unordered_set& seen_DEPRECATED, DenseHash } auto check = [&](TypeId tv) { - occursCheck(seen_DEPRECATED, seen, needle, tv); + occursCheck(seen, needle, tv); }; if (get(haystack)) @@ -2121,43 +1922,20 @@ void Unifier::occursCheck(std::unordered_set& seen_DEPRECATED, DenseHash void Unifier::occursCheck(TypePackId needle, TypePackId haystack) { - std::unordered_set seen_DEPRECATED; + sharedState.tempSeenTp.clear(); - if (FFlag::LuauCacheUnifyTableResults) - { - if (FFlag::LuauTypecheckOpts) - sharedState.tempSeenTp.clear(); - - return occursCheck(seen_DEPRECATED, sharedState.tempSeenTp, needle, haystack); - } - else - { - if (FFlag::LuauTypecheckOpts) - tempSeenTp_DEPRECATED.clear(); - - return occursCheck(seen_DEPRECATED, tempSeenTp_DEPRECATED, needle, haystack); - } + return occursCheck(sharedState.tempSeenTp, needle, haystack); } -void Unifier::occursCheck(std::unordered_set& seen_DEPRECATED, DenseHashSet& seen, TypePackId needle, TypePackId haystack) +void Unifier::occursCheck(DenseHashSet& seen, TypePackId needle, TypePackId haystack) { needle = follow(needle); haystack = follow(haystack); - if (FFlag::LuauTypecheckOpts) - { - if (seen.find(haystack)) - return; + if (seen.find(haystack)) + return; - seen.insert(haystack); - } - else - { - if (seen_DEPRECATED.end() != seen_DEPRECATED.find(haystack)) - return; - - seen_DEPRECATED.insert(haystack); - } + seen.insert(haystack); if (get(needle)) return; @@ -2165,8 +1943,7 @@ void Unifier::occursCheck(std::unordered_set& seen_DEPRECATED, Dense if (!get(needle)) ice("Expected needle pack to be free"); - RecursionLimiter _ra( - FFlag::LuauTypecheckOpts ? &counters->recursionCount : &counters_DEPRECATED->recursionCount, FInt::LuauTypeInferRecursionLimit); + RecursionLimiter _ra(&sharedState.counters.recursionCount, FInt::LuauTypeInferRecursionLimit); while (!get(haystack)) { @@ -2186,8 +1963,8 @@ void Unifier::occursCheck(std::unordered_set& seen_DEPRECATED, Dense { if (auto f = get(follow(ty))) { - occursCheck(seen_DEPRECATED, seen, needle, f->argTypes); - occursCheck(seen_DEPRECATED, seen, needle, f->retType); + occursCheck(seen, needle, f->argTypes); + occursCheck(seen, needle, f->retType); } } } @@ -2204,10 +1981,7 @@ void Unifier::occursCheck(std::unordered_set& seen_DEPRECATED, Dense Unifier Unifier::makeChildUnifier() { - if (FFlag::LuauShareTxnSeen) - return Unifier{types, mode, globalScope, log.sharedSeen, location, variance, sharedState, counters_DEPRECATED, counters}; - else - return Unifier{types, mode, globalScope, log.ownedSeen, location, variance, sharedState, counters_DEPRECATED, counters}; + return Unifier{types, mode, globalScope, log.sharedSeen, location, variance, sharedState}; } bool Unifier::isNonstrictMode() const diff --git a/Ast/src/Parser.cpp b/Ast/src/Parser.cpp index bc63e37d..3d0d5b7e 100644 --- a/Ast/src/Parser.cpp +++ b/Ast/src/Parser.cpp @@ -13,8 +13,6 @@ LUAU_FASTINTVARIABLE(LuauParseErrorLimit, 100) LUAU_FASTFLAGVARIABLE(LuauCaptureBrokenCommentSpans, false) LUAU_FASTFLAGVARIABLE(LuauIfElseExpressionBaseSupport, false) LUAU_FASTFLAGVARIABLE(LuauIfStatementRecursionGuard, false) -LUAU_FASTFLAGVARIABLE(LuauTypeAliasPacks, false) -LUAU_FASTFLAGVARIABLE(LuauParseTypePackTypeParameters, false) LUAU_FASTFLAGVARIABLE(LuauFixAmbiguousErrorRecoveryInAssign, false) LUAU_FASTFLAGVARIABLE(LuauParseSingletonTypes, false) LUAU_FASTFLAGVARIABLE(LuauParseGenericFunctionTypeBegin, false) @@ -782,8 +780,7 @@ AstStat* Parser::parseTypeAlias(const Location& start, bool exported) AstType* type = parseTypeAnnotation(); - return allocator.alloc( - Location(start, type->location), name->name, generics, FFlag::LuauTypeAliasPacks ? genericPacks : AstArray{}, type, exported); + return allocator.alloc(Location(start, type->location), name->name, generics, genericPacks, type, exported); } AstDeclaredClassProp Parser::parseDeclaredClassMethod() @@ -1602,30 +1599,18 @@ AstTypeOrPack Parser::parseSimpleTypeAnnotation(bool allowPack) return {allocator.alloc(Location(begin, end), expr), {}}; } - if (FFlag::LuauParseTypePackTypeParameters) + bool hasParameters = false; + AstArray parameters{}; + + if (lexer.current().type == '<') { - bool hasParameters = false; - AstArray parameters{}; - - if (lexer.current().type == '<') - { - hasParameters = true; - parameters = parseTypeParams(); - } - - Location end = lexer.previousLocation(); - - return {allocator.alloc(Location(begin, end), prefix, name.name, hasParameters, parameters), {}}; + hasParameters = true; + parameters = parseTypeParams(); } - else - { - AstArray generics = parseTypeParams(); - Location end = lexer.previousLocation(); + Location end = lexer.previousLocation(); - // false in 'hasParameterList' as it is not used without FFlagLuauTypeAliasPacks - return {allocator.alloc(Location(begin, end), prefix, name.name, false, generics), {}}; - } + return {allocator.alloc(Location(begin, end), prefix, name.name, hasParameters, parameters), {}}; } else if (lexer.current().type == '{') { @@ -2414,37 +2399,24 @@ AstArray Parser::parseTypeParams() while (true) { - if (FFlag::LuauParseTypePackTypeParameters) + if (shouldParseTypePackAnnotation(lexer)) { - if (shouldParseTypePackAnnotation(lexer)) - { - auto typePack = parseTypePackAnnotation(); + auto typePack = parseTypePackAnnotation(); - if (FFlag::LuauTypeAliasPacks) // Type packs are recorded only is we can handle them - parameters.push_back({{}, typePack}); - } - else if (lexer.current().type == '(') - { - auto [type, typePack] = parseTypeOrPackAnnotation(); + parameters.push_back({{}, typePack}); + } + else if (lexer.current().type == '(') + { + auto [type, typePack] = parseTypeOrPackAnnotation(); - if (typePack) - { - if (FFlag::LuauTypeAliasPacks) // Type packs are recorded only is we can handle them - parameters.push_back({{}, typePack}); - } - else - { - parameters.push_back({type, {}}); - } - } - else if (lexer.current().type == '>' && parameters.empty()) - { - break; - } + if (typePack) + parameters.push_back({{}, typePack}); else - { - parameters.push_back({parseTypeAnnotation(), {}}); - } + parameters.push_back({type, {}}); + } + else if (lexer.current().type == '>' && parameters.empty()) + { + break; } else { diff --git a/CLI/Analyze.cpp b/CLI/Analyze.cpp index 4194d5ae..cec51e61 100644 --- a/CLI/Analyze.cpp +++ b/CLI/Analyze.cpp @@ -121,7 +121,7 @@ struct CliFileResolver : Luau::FileResolver if (Luau::AstExprConstantString* expr = node->as()) { Luau::ModuleName name = std::string(expr->value.data, expr->value.size) + ".luau"; - if (!moduleExists(name)) + if (!readFile(name)) { // fall back to .lua if a module with .luau doesn't exist name = std::string(expr->value.data, expr->value.size) + ".lua"; @@ -132,27 +132,6 @@ struct CliFileResolver : Luau::FileResolver return std::nullopt; } - - bool moduleExists(const Luau::ModuleName& name) const override - { - return !!readFile(name); - } - - - std::optional fromAstFragment(Luau::AstExpr* expr) const override - { - return std::nullopt; - } - - Luau::ModuleName concat(const Luau::ModuleName& lhs, std::string_view rhs) const override - { - return lhs + "/" + std::string(rhs); - } - - std::optional getParentModuleName(const Luau::ModuleName& name) const override - { - return std::nullopt; - } }; struct CliConfigResolver : Luau::ConfigResolver diff --git a/CLI/Repl.cpp b/CLI/Repl.cpp index 89a6037b..76493a5e 100644 --- a/CLI/Repl.cpp +++ b/CLI/Repl.cpp @@ -526,4 +526,3 @@ int main(int argc, char** argv) return failed; } } - diff --git a/Compiler/src/Compiler.cpp b/Compiler/src/Compiler.cpp index 5b93c1dc..2c1e85ff 100644 --- a/Compiler/src/Compiler.cpp +++ b/Compiler/src/Compiler.cpp @@ -321,13 +321,15 @@ struct Compiler compileExprTempTop(expr->args.data[i], uint8_t(regs + 1 + expr->self + i)); } - setDebugLine(expr->func); + setDebugLineEnd(expr->func); if (expr->self) { AstExprIndexName* fi = expr->func->as(); LUAU_ASSERT(fi); + setDebugLine(fi->indexLocation); + BytecodeBuilder::StringRef iname = sref(fi->index); int32_t cid = bytecode.addConstantString(iname); if (cid < 0) @@ -1313,6 +1315,8 @@ struct Compiler RegScope rs(this); uint8_t reg = compileExprAuto(expr->expr, rs); + setDebugLine(expr->indexLocation); + BytecodeBuilder::StringRef iname = sref(expr->index); int32_t cid = bytecode.addConstantString(iname); if (cid < 0) @@ -2710,6 +2714,12 @@ struct Compiler bytecode.setDebugLine(node->location.begin.line + 1); } + void setDebugLine(const Location& location) + { + if (options.debugLevel >= 1) + bytecode.setDebugLine(location.begin.line + 1); + } + void setDebugLineEnd(AstNode* node) { if (options.debugLevel >= 1) @@ -3650,7 +3660,7 @@ struct Compiler { if (options.vectorLib) { - if (builtin.object == options.vectorLib && builtin.method == options.vectorCtor) + if (builtin.isMethod(options.vectorLib, options.vectorCtor)) return LBF_VECTOR; } else diff --git a/Makefile b/Makefile index ce461af7..7355040f 100644 --- a/Makefile +++ b/Makefile @@ -35,7 +35,7 @@ ANALYZE_CLI_SOURCES=CLI/FileUtils.cpp CLI/Analyze.cpp ANALYZE_CLI_OBJECTS=$(ANALYZE_CLI_SOURCES:%=$(BUILD)/%.o) ANALYZE_CLI_TARGET=$(BUILD)/luau-analyze -FUZZ_SOURCES=$(wildcard fuzz/*.cpp) +FUZZ_SOURCES=$(wildcard fuzz/*.cpp) fuzz/luau.pb.cpp FUZZ_OBJECTS=$(FUZZ_SOURCES:%=$(BUILD)/%.o) TESTS_ARGS= @@ -167,8 +167,8 @@ fuzz/luau.pb.cpp: fuzz/luau.proto build/libprotobuf-mutator cd fuzz && ../build/libprotobuf-mutator/external.protobuf/bin/protoc luau.proto --cpp_out=. mv fuzz/luau.pb.cc fuzz/luau.pb.cpp -$(BUILD)/fuzz/proto.cpp.o: build/libprotobuf-mutator -$(BUILD)/fuzz/protoprint.cpp.o: build/libprotobuf-mutator +$(BUILD)/fuzz/proto.cpp.o: fuzz/luau.pb.cpp +$(BUILD)/fuzz/protoprint.cpp.o: fuzz/luau.pb.cpp build/libprotobuf-mutator: git clone https://github.com/google/libprotobuf-mutator build/libprotobuf-mutator diff --git a/Sources.cmake b/Sources.cmake index 440cc3ec..57df9b91 100644 --- a/Sources.cmake +++ b/Sources.cmake @@ -54,6 +54,7 @@ target_sources(Luau.Analysis PRIVATE Analysis/include/Luau/Scope.h Analysis/include/Luau/Substitution.h Analysis/include/Luau/Symbol.h + Analysis/include/Luau/ToDot.h Analysis/include/Luau/TopoSortStatements.h Analysis/include/Luau/ToString.h Analysis/include/Luau/Transpiler.h @@ -86,6 +87,7 @@ target_sources(Luau.Analysis PRIVATE Analysis/src/Scope.cpp Analysis/src/Substitution.cpp Analysis/src/Symbol.cpp + Analysis/src/ToDot.cpp Analysis/src/TopoSortStatements.cpp Analysis/src/ToString.cpp Analysis/src/Transpiler.cpp @@ -118,6 +120,7 @@ target_sources(Luau.VM PRIVATE VM/src/ldo.cpp VM/src/lfunc.cpp VM/src/lgc.cpp + VM/src/lgcdebug.cpp VM/src/linit.cpp VM/src/lmathlib.cpp VM/src/lmem.cpp @@ -194,6 +197,7 @@ if(TARGET Luau.UnitTest) tests/RequireTracer.test.cpp tests/StringUtils.test.cpp tests/Symbol.test.cpp + tests/ToDot.test.cpp tests/TopoSort.test.cpp tests/ToString.test.cpp tests/Transpiler.test.cpp diff --git a/VM/include/lua.h b/VM/include/lua.h index f9da943b..7078acd0 100644 --- a/VM/include/lua.h +++ b/VM/include/lua.h @@ -189,7 +189,7 @@ LUA_API void lua_setreadonly(lua_State* L, int idx, int enabled); LUA_API int lua_getreadonly(lua_State* L, int idx); LUA_API void lua_setsafeenv(lua_State* L, int idx, int enabled); -LUA_API void* lua_newuserdata(lua_State* L, size_t sz, int tag); +LUA_API void* lua_newuserdatatagged(lua_State* L, size_t sz, int tag); LUA_API void* lua_newuserdatadtor(lua_State* L, size_t sz, void (*dtor)(void*)); LUA_API int lua_getmetatable(lua_State* L, int objindex); LUA_API void lua_getfenv(lua_State* L, int idx); @@ -288,6 +288,7 @@ LUA_API void lua_unref(lua_State* L, int ref); #define lua_pop(L, n) lua_settop(L, -(n)-1) #define lua_newtable(L) lua_createtable(L, 0, 0) +#define lua_newuserdata(L, s) lua_newuserdatatagged(L, s, 0) #define lua_strlen(L, i) lua_objlen(L, (i)) diff --git a/VM/src/lapi.cpp b/VM/src/lapi.cpp index 25c29e3b..76043b9c 100644 --- a/VM/src/lapi.cpp +++ b/VM/src/lapi.cpp @@ -13,6 +13,8 @@ #include +LUAU_FASTFLAG(LuauActivateBeforeExec) + const char* lua_ident = "$Lua: Lua 5.1.4 Copyright (C) 1994-2008 Lua.org, PUC-Rio $\n" "$Authors: R. Ierusalimschy, L. H. de Figueiredo & W. Celes $\n" "$URL: www.lua.org $\n"; @@ -937,14 +939,21 @@ void lua_call(lua_State* L, int nargs, int nresults) checkresults(L, nargs, nresults); func = L->top - (nargs + 1); - int wasActive = luaC_threadactive(L); - l_setbit(L->stackstate, THREAD_ACTIVEBIT); - luaC_checkthreadsleep(L); + if (FFlag::LuauActivateBeforeExec) + { + luaD_call(L, func, nresults); + } + else + { + int oldactive = luaC_threadactive(L); + l_setbit(L->stackstate, THREAD_ACTIVEBIT); + luaC_checkthreadsleep(L); - luaD_call(L, func, nresults); + luaD_call(L, func, nresults); - if (!wasActive) - resetbit(L->stackstate, THREAD_ACTIVEBIT); + if (!oldactive) + resetbit(L->stackstate, THREAD_ACTIVEBIT); + } adjustresults(L, nresults); return; @@ -985,14 +994,21 @@ int lua_pcall(lua_State* L, int nargs, int nresults, int errfunc) c.func = L->top - (nargs + 1); /* function to be called */ c.nresults = nresults; - int wasActive = luaC_threadactive(L); - l_setbit(L->stackstate, THREAD_ACTIVEBIT); - luaC_checkthreadsleep(L); + if (FFlag::LuauActivateBeforeExec) + { + status = luaD_pcall(L, f_call, &c, savestack(L, c.func), func); + } + else + { + int oldactive = luaC_threadactive(L); + l_setbit(L->stackstate, THREAD_ACTIVEBIT); + luaC_checkthreadsleep(L); - status = luaD_pcall(L, f_call, &c, savestack(L, c.func), func); + status = luaD_pcall(L, f_call, &c, savestack(L, c.func), func); - if (!wasActive) - resetbit(L->stackstate, THREAD_ACTIVEBIT); + if (!oldactive) + resetbit(L->stackstate, THREAD_ACTIVEBIT); + } adjustresults(L, nresults); return status; @@ -1166,7 +1182,7 @@ void lua_concat(lua_State* L, int n) return; } -void* lua_newuserdata(lua_State* L, size_t sz, int tag) +void* lua_newuserdatatagged(lua_State* L, size_t sz, int tag) { api_check(L, unsigned(tag) < LUA_UTAG_LIMIT); luaC_checkGC(L); @@ -1251,6 +1267,7 @@ uintptr_t lua_encodepointer(lua_State* L, uintptr_t p) int lua_ref(lua_State* L, int idx) { + api_check(L, idx != LUA_REGISTRYINDEX); /* idx is a stack index for value */ int ref = LUA_REFNIL; global_State* g = L->global; StkId p = index2adr(L, idx); diff --git a/VM/src/lbaselib.cpp b/VM/src/lbaselib.cpp index c3217f86..881c804d 100644 --- a/VM/src/lbaselib.cpp +++ b/VM/src/lbaselib.cpp @@ -401,7 +401,7 @@ static int luaB_newproxy(lua_State* L) bool needsmt = lua_toboolean(L, 1); - lua_newuserdata(L, 0, 0); + lua_newuserdata(L, 0); if (needsmt) { diff --git a/VM/src/ldo.cpp b/VM/src/ldo.cpp index 1259d461..62bbdb7c 100644 --- a/VM/src/ldo.cpp +++ b/VM/src/ldo.cpp @@ -17,9 +17,9 @@ #include -LUAU_FASTFLAGVARIABLE(LuauExceptionMessageFix, false) LUAU_FASTFLAGVARIABLE(LuauCcallRestoreFix, false) LUAU_FASTFLAG(LuauCoroutineClose) +LUAU_FASTFLAGVARIABLE(LuauActivateBeforeExec, false) /* ** {====================================================== @@ -74,35 +74,28 @@ public: const char* what() const throw() override { - if (FFlag::LuauExceptionMessageFix) + // LUA_ERRRUN/LUA_ERRSYNTAX pass an object on the stack which is intended to describe the error. + if (status == LUA_ERRRUN || status == LUA_ERRSYNTAX) { - // LUA_ERRRUN/LUA_ERRSYNTAX pass an object on the stack which is intended to describe the error. - if (status == LUA_ERRRUN || status == LUA_ERRSYNTAX) + // Conversion to a string could still fail. For example if a user passes a non-string/non-number argument to `error()`. + if (const char* str = lua_tostring(L, -1)) { - // Conversion to a string could still fail. For example if a user passes a non-string/non-number argument to `error()`. - if (const char* str = lua_tostring(L, -1)) - { - return str; - } - } - - switch (status) - { - case LUA_ERRRUN: - return "lua_exception: LUA_ERRRUN (no string/number provided as description)"; - case LUA_ERRSYNTAX: - return "lua_exception: LUA_ERRSYNTAX (no string/number provided as description)"; - case LUA_ERRMEM: - return "lua_exception: " LUA_MEMERRMSG; - case LUA_ERRERR: - return "lua_exception: " LUA_ERRERRMSG; - default: - return "lua_exception: unexpected exception status"; + return str; } } - else + + switch (status) { - return lua_tostring(L, -1); + case LUA_ERRRUN: + return "lua_exception: LUA_ERRRUN (no string/number provided as description)"; + case LUA_ERRSYNTAX: + return "lua_exception: LUA_ERRSYNTAX (no string/number provided as description)"; + case LUA_ERRMEM: + return "lua_exception: " LUA_MEMERRMSG; + case LUA_ERRERR: + return "lua_exception: " LUA_ERRERRMSG; + default: + return "lua_exception: unexpected exception status"; } } @@ -234,7 +227,22 @@ void luaD_call(lua_State* L, StkId func, int nResults) if (luau_precall(L, func, nResults) == PCRLUA) { /* is a Lua function? */ L->ci->flags |= LUA_CALLINFO_RETURN; /* luau_execute will stop after returning from the stack frame */ - luau_execute(L); /* call it */ + + if (FFlag::LuauActivateBeforeExec) + { + int oldactive = luaC_threadactive(L); + l_setbit(L->stackstate, THREAD_ACTIVEBIT); + luaC_checkthreadsleep(L); + + luau_execute(L); /* call it */ + + if (!oldactive) + resetbit(L->stackstate, THREAD_ACTIVEBIT); + } + else + { + luau_execute(L); /* call it */ + } } L->nCcalls--; luaC_checkGC(L); @@ -527,10 +535,10 @@ static void restore_stack_limit(lua_State* L) int luaD_pcall(lua_State* L, Pfunc func, void* u, ptrdiff_t old_top, ptrdiff_t ef) { - int status; unsigned short oldnCcalls = L->nCcalls; ptrdiff_t old_ci = saveci(L, L->ci); - status = luaD_rawrunprotected(L, func, u); + int oldactive = luaC_threadactive(L); + int status = luaD_rawrunprotected(L, func, u); if (status != 0) { // call user-defined error function (used in xpcall) @@ -541,6 +549,13 @@ int luaD_pcall(lua_State* L, Pfunc func, void* u, ptrdiff_t old_top, ptrdiff_t e status = LUA_ERRERR; } + if (FFlag::LuauActivateBeforeExec) + { + // since the call failed with an error, we might have to reset the 'active' thread state + if (!oldactive) + resetbit(L->stackstate, THREAD_ACTIVEBIT); + } + if (FFlag::LuauCcallRestoreFix) { // Restore nCcalls before calling the debugprotectederror callback which may rely on the proper value to have been restored. diff --git a/VM/src/lgc.cpp b/VM/src/lgc.cpp index 11f79d1a..ab416041 100644 --- a/VM/src/lgc.cpp +++ b/VM/src/lgc.cpp @@ -10,9 +10,7 @@ #include "ldo.h" #include -#include -LUAU_FASTFLAGVARIABLE(LuauRescanGrayAgainForwardBarrier, false) LUAU_FASTFLAGVARIABLE(LuauSeparateAtomic, false) LUAU_FASTFLAG(LuauArrayBoundary) @@ -988,7 +986,7 @@ void luaC_barriertable(lua_State* L, Table* t, GCObject* v) GCObject* o = obj2gco(t); // in the second propagation stage, table assignment barrier works as a forward barrier - if (FFlag::LuauRescanGrayAgainForwardBarrier && g->gcstate == GCSpropagateagain) + if (g->gcstate == GCSpropagateagain) { LUAU_ASSERT(isblack(o) && iswhite(v) && !isdead(g, v) && !isdead(g, o)); reallymarkobject(g, v); @@ -1044,550 +1042,6 @@ void luaC_linkupval(lua_State* L, UpVal* uv) } } -static void validateobjref(global_State* g, GCObject* f, GCObject* t) -{ - LUAU_ASSERT(!isdead(g, t)); - - if (keepinvariant(g)) - { - /* basic incremental invariant: black can't point to white */ - LUAU_ASSERT(!(isblack(f) && iswhite(t))); - } -} - -static void validateref(global_State* g, GCObject* f, TValue* v) -{ - if (iscollectable(v)) - { - LUAU_ASSERT(ttype(v) == gcvalue(v)->gch.tt); - validateobjref(g, f, gcvalue(v)); - } -} - -static void validatetable(global_State* g, Table* h) -{ - int sizenode = 1 << h->lsizenode; - - if (FFlag::LuauArrayBoundary) - LUAU_ASSERT(h->lastfree <= sizenode); - else - LUAU_ASSERT(h->lastfree >= 0 && h->lastfree <= sizenode); - - if (h->metatable) - validateobjref(g, obj2gco(h), obj2gco(h->metatable)); - - for (int i = 0; i < h->sizearray; ++i) - validateref(g, obj2gco(h), &h->array[i]); - - for (int i = 0; i < sizenode; ++i) - { - LuaNode* n = &h->node[i]; - - LUAU_ASSERT(ttype(gkey(n)) != LUA_TDEADKEY || ttisnil(gval(n))); - LUAU_ASSERT(i + gnext(n) >= 0 && i + gnext(n) < sizenode); - - if (!ttisnil(gval(n))) - { - TValue k = {}; - k.tt = gkey(n)->tt; - k.value = gkey(n)->value; - - validateref(g, obj2gco(h), &k); - validateref(g, obj2gco(h), gval(n)); - } - } -} - -static void validateclosure(global_State* g, Closure* cl) -{ - validateobjref(g, obj2gco(cl), obj2gco(cl->env)); - - if (cl->isC) - { - for (int i = 0; i < cl->nupvalues; ++i) - validateref(g, obj2gco(cl), &cl->c.upvals[i]); - } - else - { - LUAU_ASSERT(cl->nupvalues == cl->l.p->nups); - - validateobjref(g, obj2gco(cl), obj2gco(cl->l.p)); - - for (int i = 0; i < cl->nupvalues; ++i) - validateref(g, obj2gco(cl), &cl->l.uprefs[i]); - } -} - -static void validatestack(global_State* g, lua_State* l) -{ - validateref(g, obj2gco(l), gt(l)); - - for (CallInfo* ci = l->base_ci; ci <= l->ci; ++ci) - { - LUAU_ASSERT(l->stack <= ci->base); - LUAU_ASSERT(ci->func <= ci->base && ci->base <= ci->top); - LUAU_ASSERT(ci->top <= l->stack_last); - } - - // note: stack refs can violate gc invariant so we only check for liveness - for (StkId o = l->stack; o < l->top; ++o) - checkliveness(g, o); - - if (l->namecall) - validateobjref(g, obj2gco(l), obj2gco(l->namecall)); - - for (GCObject* uv = l->openupval; uv; uv = uv->gch.next) - { - LUAU_ASSERT(uv->gch.tt == LUA_TUPVAL); - LUAU_ASSERT(gco2uv(uv)->v != &gco2uv(uv)->u.value); - } -} - -static void validateproto(global_State* g, Proto* f) -{ - if (f->source) - validateobjref(g, obj2gco(f), obj2gco(f->source)); - - if (f->debugname) - validateobjref(g, obj2gco(f), obj2gco(f->debugname)); - - for (int i = 0; i < f->sizek; ++i) - validateref(g, obj2gco(f), &f->k[i]); - - for (int i = 0; i < f->sizeupvalues; ++i) - if (f->upvalues[i]) - validateobjref(g, obj2gco(f), obj2gco(f->upvalues[i])); - - for (int i = 0; i < f->sizep; ++i) - if (f->p[i]) - validateobjref(g, obj2gco(f), obj2gco(f->p[i])); - - for (int i = 0; i < f->sizelocvars; i++) - if (f->locvars[i].varname) - validateobjref(g, obj2gco(f), obj2gco(f->locvars[i].varname)); -} - -static void validateobj(global_State* g, GCObject* o) -{ - /* dead objects can only occur during sweep */ - if (isdead(g, o)) - { - LUAU_ASSERT(g->gcstate == GCSsweepstring || g->gcstate == GCSsweep); - return; - } - - switch (o->gch.tt) - { - case LUA_TSTRING: - break; - - case LUA_TTABLE: - validatetable(g, gco2h(o)); - break; - - case LUA_TFUNCTION: - validateclosure(g, gco2cl(o)); - break; - - case LUA_TUSERDATA: - if (gco2u(o)->metatable) - validateobjref(g, o, obj2gco(gco2u(o)->metatable)); - break; - - case LUA_TTHREAD: - validatestack(g, gco2th(o)); - break; - - case LUA_TPROTO: - validateproto(g, gco2p(o)); - break; - - case LUA_TUPVAL: - validateref(g, o, gco2uv(o)->v); - break; - - default: - LUAU_ASSERT(!"unexpected object type"); - } -} - -static void validatelist(global_State* g, GCObject* o) -{ - while (o) - { - validateobj(g, o); - - o = o->gch.next; - } -} - -static void validategraylist(global_State* g, GCObject* o) -{ - if (!keepinvariant(g)) - return; - - while (o) - { - LUAU_ASSERT(isgray(o)); - - switch (o->gch.tt) - { - case LUA_TTABLE: - o = gco2h(o)->gclist; - break; - case LUA_TFUNCTION: - o = gco2cl(o)->gclist; - break; - case LUA_TTHREAD: - o = gco2th(o)->gclist; - break; - case LUA_TPROTO: - o = gco2p(o)->gclist; - break; - default: - LUAU_ASSERT(!"unknown object in gray list"); - return; - } - } -} - -void luaC_validate(lua_State* L) -{ - global_State* g = L->global; - - LUAU_ASSERT(!isdead(g, obj2gco(g->mainthread))); - checkliveness(g, &g->registry); - - for (int i = 0; i < LUA_T_COUNT; ++i) - if (g->mt[i]) - LUAU_ASSERT(!isdead(g, obj2gco(g->mt[i]))); - - validategraylist(g, g->weak); - validategraylist(g, g->gray); - validategraylist(g, g->grayagain); - - for (int i = 0; i < g->strt.size; ++i) - validatelist(g, g->strt.hash[i]); - - validatelist(g, g->rootgc); - validatelist(g, g->strbufgc); - - for (UpVal* uv = g->uvhead.u.l.next; uv != &g->uvhead; uv = uv->u.l.next) - { - LUAU_ASSERT(uv->tt == LUA_TUPVAL); - LUAU_ASSERT(uv->v != &uv->u.value); - LUAU_ASSERT(uv->u.l.next->u.l.prev == uv && uv->u.l.prev->u.l.next == uv); - } -} - -inline bool safejson(char ch) -{ - return unsigned(ch) < 128 && ch >= 32 && ch != '\\' && ch != '\"'; -} - -static void dumpref(FILE* f, GCObject* o) -{ - fprintf(f, "\"%p\"", o); -} - -static void dumprefs(FILE* f, TValue* data, size_t size) -{ - bool first = true; - - for (size_t i = 0; i < size; ++i) - { - if (iscollectable(&data[i])) - { - if (!first) - fputc(',', f); - first = false; - - dumpref(f, gcvalue(&data[i])); - } - } -} - -static void dumpstringdata(FILE* f, const char* data, size_t len) -{ - for (size_t i = 0; i < len; ++i) - fputc(safejson(data[i]) ? data[i] : '?', f); -} - -static void dumpstring(FILE* f, TString* ts) -{ - fprintf(f, "{\"type\":\"string\",\"cat\":%d,\"size\":%d,\"data\":\"", ts->memcat, int(sizestring(ts->len))); - dumpstringdata(f, ts->data, ts->len); - fprintf(f, "\"}"); -} - -static void dumptable(FILE* f, Table* h) -{ - size_t size = sizeof(Table) + (h->node == &luaH_dummynode ? 0 : sizenode(h) * sizeof(LuaNode)) + h->sizearray * sizeof(TValue); - - fprintf(f, "{\"type\":\"table\",\"cat\":%d,\"size\":%d", h->memcat, int(size)); - - if (h->node != &luaH_dummynode) - { - fprintf(f, ",\"pairs\":["); - - bool first = true; - - for (int i = 0; i < sizenode(h); ++i) - { - const LuaNode& n = h->node[i]; - - if (!ttisnil(&n.val) && (iscollectable(&n.key) || iscollectable(&n.val))) - { - if (!first) - fputc(',', f); - first = false; - - if (iscollectable(&n.key)) - dumpref(f, gcvalue(&n.key)); - else - fprintf(f, "null"); - - fputc(',', f); - - if (iscollectable(&n.val)) - dumpref(f, gcvalue(&n.val)); - else - fprintf(f, "null"); - } - } - - fprintf(f, "]"); - } - if (h->sizearray) - { - fprintf(f, ",\"array\":["); - dumprefs(f, h->array, h->sizearray); - fprintf(f, "]"); - } - if (h->metatable) - { - fprintf(f, ",\"metatable\":"); - dumpref(f, obj2gco(h->metatable)); - } - fprintf(f, "}"); -} - -static void dumpclosure(FILE* f, Closure* cl) -{ - fprintf(f, "{\"type\":\"function\",\"cat\":%d,\"size\":%d", cl->memcat, - cl->isC ? int(sizeCclosure(cl->nupvalues)) : int(sizeLclosure(cl->nupvalues))); - - fprintf(f, ",\"env\":"); - dumpref(f, obj2gco(cl->env)); - if (cl->isC) - { - if (cl->nupvalues) - { - fprintf(f, ",\"upvalues\":["); - dumprefs(f, cl->c.upvals, cl->nupvalues); - fprintf(f, "]"); - } - } - else - { - fprintf(f, ",\"proto\":"); - dumpref(f, obj2gco(cl->l.p)); - if (cl->nupvalues) - { - fprintf(f, ",\"upvalues\":["); - dumprefs(f, cl->l.uprefs, cl->nupvalues); - fprintf(f, "]"); - } - } - fprintf(f, "}"); -} - -static void dumpudata(FILE* f, Udata* u) -{ - fprintf(f, "{\"type\":\"userdata\",\"cat\":%d,\"size\":%d,\"tag\":%d", u->memcat, int(sizeudata(u->len)), u->tag); - - if (u->metatable) - { - fprintf(f, ",\"metatable\":"); - dumpref(f, obj2gco(u->metatable)); - } - fprintf(f, "}"); -} - -static void dumpthread(FILE* f, lua_State* th) -{ - size_t size = sizeof(lua_State) + sizeof(TValue) * th->stacksize + sizeof(CallInfo) * th->size_ci; - - fprintf(f, "{\"type\":\"thread\",\"cat\":%d,\"size\":%d", th->memcat, int(size)); - - if (iscollectable(&th->l_gt)) - { - fprintf(f, ",\"env\":"); - dumpref(f, gcvalue(&th->l_gt)); - } - - Closure* tcl = 0; - for (CallInfo* ci = th->base_ci; ci <= th->ci; ++ci) - { - if (ttisfunction(ci->func)) - { - tcl = clvalue(ci->func); - break; - } - } - - if (tcl && !tcl->isC && tcl->l.p->source) - { - Proto* p = tcl->l.p; - - fprintf(f, ",\"source\":\""); - dumpstringdata(f, p->source->data, p->source->len); - fprintf(f, "\",\"line\":%d", p->abslineinfo ? p->abslineinfo[0] : 0); - } - - if (th->top > th->stack) - { - fprintf(f, ",\"stack\":["); - dumprefs(f, th->stack, th->top - th->stack); - fprintf(f, "]"); - } - fprintf(f, "}"); -} - -static void dumpproto(FILE* f, Proto* p) -{ - size_t size = sizeof(Proto) + sizeof(Instruction) * p->sizecode + sizeof(Proto*) * p->sizep + sizeof(TValue) * p->sizek + p->sizelineinfo + - sizeof(LocVar) * p->sizelocvars + sizeof(TString*) * p->sizeupvalues; - - fprintf(f, "{\"type\":\"proto\",\"cat\":%d,\"size\":%d", p->memcat, int(size)); - - if (p->source) - { - fprintf(f, ",\"source\":\""); - dumpstringdata(f, p->source->data, p->source->len); - fprintf(f, "\",\"line\":%d", p->abslineinfo ? p->abslineinfo[0] : 0); - } - - if (p->sizek) - { - fprintf(f, ",\"constants\":["); - dumprefs(f, p->k, p->sizek); - fprintf(f, "]"); - } - - if (p->sizep) - { - fprintf(f, ",\"protos\":["); - for (int i = 0; i < p->sizep; ++i) - { - if (i != 0) - fputc(',', f); - dumpref(f, obj2gco(p->p[i])); - } - fprintf(f, "]"); - } - - fprintf(f, "}"); -} - -static void dumpupval(FILE* f, UpVal* uv) -{ - fprintf(f, "{\"type\":\"upvalue\",\"cat\":%d,\"size\":%d", uv->memcat, int(sizeof(UpVal))); - - if (iscollectable(uv->v)) - { - fprintf(f, ",\"object\":"); - dumpref(f, gcvalue(uv->v)); - } - fprintf(f, "}"); -} - -static void dumpobj(FILE* f, GCObject* o) -{ - switch (o->gch.tt) - { - case LUA_TSTRING: - return dumpstring(f, gco2ts(o)); - - case LUA_TTABLE: - return dumptable(f, gco2h(o)); - - case LUA_TFUNCTION: - return dumpclosure(f, gco2cl(o)); - - case LUA_TUSERDATA: - return dumpudata(f, gco2u(o)); - - case LUA_TTHREAD: - return dumpthread(f, gco2th(o)); - - case LUA_TPROTO: - return dumpproto(f, gco2p(o)); - - case LUA_TUPVAL: - return dumpupval(f, gco2uv(o)); - - default: - LUAU_ASSERT(0); - } -} - -static void dumplist(FILE* f, GCObject* o) -{ - while (o) - { - dumpref(f, o); - fputc(':', f); - dumpobj(f, o); - fputc(',', f); - fputc('\n', f); - - // thread has additional list containing collectable objects that are not present in rootgc - if (o->gch.tt == LUA_TTHREAD) - dumplist(f, gco2th(o)->openupval); - - o = o->gch.next; - } -} - -void luaC_dump(lua_State* L, void* file, const char* (*categoryName)(lua_State* L, uint8_t memcat)) -{ - global_State* g = L->global; - FILE* f = static_cast(file); - - fprintf(f, "{\"objects\":{\n"); - dumplist(f, g->rootgc); - dumplist(f, g->strbufgc); - for (int i = 0; i < g->strt.size; ++i) - dumplist(f, g->strt.hash[i]); - - fprintf(f, "\"0\":{\"type\":\"userdata\",\"cat\":0,\"size\":0}\n"); // to avoid issues with trailing , - fprintf(f, "},\"roots\":{\n"); - fprintf(f, "\"mainthread\":"); - dumpref(f, obj2gco(g->mainthread)); - fprintf(f, ",\"registry\":"); - dumpref(f, gcvalue(&g->registry)); - - fprintf(f, "},\"stats\":{\n"); - - fprintf(f, "\"size\":%d,\n", int(g->totalbytes)); - - fprintf(f, "\"categories\":{\n"); - for (int i = 0; i < LUA_MEMORY_CATEGORIES; i++) - { - if (size_t bytes = g->memcatbytes[i]) - { - if (categoryName) - fprintf(f, "\"%d\":{\"name\":\"%s\", \"size\":%d},\n", i, categoryName(L, i), int(bytes)); - else - fprintf(f, "\"%d\":{\"size\":%d},\n", i, int(bytes)); - } - } - fprintf(f, "\"none\":{}\n"); // to avoid issues with trailing , - fprintf(f, "}\n"); - fprintf(f, "}}\n"); -} - // measure the allocation rate in bytes/sec // returns -1 if allocation rate cannot be measured int64_t luaC_allocationrate(lua_State* L) diff --git a/VM/src/lgcdebug.cpp b/VM/src/lgcdebug.cpp new file mode 100644 index 00000000..a79e7b95 --- /dev/null +++ b/VM/src/lgcdebug.cpp @@ -0,0 +1,558 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +// This code is based on Lua 5.x implementation licensed under MIT License; see lua_LICENSE.txt for details +#include "lgc.h" + +#include "lobject.h" +#include "lstate.h" +#include "ltable.h" +#include "lfunc.h" +#include "lstring.h" + +#include +#include + +LUAU_FASTFLAG(LuauArrayBoundary) + +static void validateobjref(global_State* g, GCObject* f, GCObject* t) +{ + LUAU_ASSERT(!isdead(g, t)); + + if (keepinvariant(g)) + { + /* basic incremental invariant: black can't point to white */ + LUAU_ASSERT(!(isblack(f) && iswhite(t))); + } +} + +static void validateref(global_State* g, GCObject* f, TValue* v) +{ + if (iscollectable(v)) + { + LUAU_ASSERT(ttype(v) == gcvalue(v)->gch.tt); + validateobjref(g, f, gcvalue(v)); + } +} + +static void validatetable(global_State* g, Table* h) +{ + int sizenode = 1 << h->lsizenode; + + if (FFlag::LuauArrayBoundary) + LUAU_ASSERT(h->lastfree <= sizenode); + else + LUAU_ASSERT(h->lastfree >= 0 && h->lastfree <= sizenode); + + if (h->metatable) + validateobjref(g, obj2gco(h), obj2gco(h->metatable)); + + for (int i = 0; i < h->sizearray; ++i) + validateref(g, obj2gco(h), &h->array[i]); + + for (int i = 0; i < sizenode; ++i) + { + LuaNode* n = &h->node[i]; + + LUAU_ASSERT(ttype(gkey(n)) != LUA_TDEADKEY || ttisnil(gval(n))); + LUAU_ASSERT(i + gnext(n) >= 0 && i + gnext(n) < sizenode); + + if (!ttisnil(gval(n))) + { + TValue k = {}; + k.tt = gkey(n)->tt; + k.value = gkey(n)->value; + + validateref(g, obj2gco(h), &k); + validateref(g, obj2gco(h), gval(n)); + } + } +} + +static void validateclosure(global_State* g, Closure* cl) +{ + validateobjref(g, obj2gco(cl), obj2gco(cl->env)); + + if (cl->isC) + { + for (int i = 0; i < cl->nupvalues; ++i) + validateref(g, obj2gco(cl), &cl->c.upvals[i]); + } + else + { + LUAU_ASSERT(cl->nupvalues == cl->l.p->nups); + + validateobjref(g, obj2gco(cl), obj2gco(cl->l.p)); + + for (int i = 0; i < cl->nupvalues; ++i) + validateref(g, obj2gco(cl), &cl->l.uprefs[i]); + } +} + +static void validatestack(global_State* g, lua_State* l) +{ + validateref(g, obj2gco(l), gt(l)); + + for (CallInfo* ci = l->base_ci; ci <= l->ci; ++ci) + { + LUAU_ASSERT(l->stack <= ci->base); + LUAU_ASSERT(ci->func <= ci->base && ci->base <= ci->top); + LUAU_ASSERT(ci->top <= l->stack_last); + } + + // note: stack refs can violate gc invariant so we only check for liveness + for (StkId o = l->stack; o < l->top; ++o) + checkliveness(g, o); + + if (l->namecall) + validateobjref(g, obj2gco(l), obj2gco(l->namecall)); + + for (GCObject* uv = l->openupval; uv; uv = uv->gch.next) + { + LUAU_ASSERT(uv->gch.tt == LUA_TUPVAL); + LUAU_ASSERT(gco2uv(uv)->v != &gco2uv(uv)->u.value); + } +} + +static void validateproto(global_State* g, Proto* f) +{ + if (f->source) + validateobjref(g, obj2gco(f), obj2gco(f->source)); + + if (f->debugname) + validateobjref(g, obj2gco(f), obj2gco(f->debugname)); + + for (int i = 0; i < f->sizek; ++i) + validateref(g, obj2gco(f), &f->k[i]); + + for (int i = 0; i < f->sizeupvalues; ++i) + if (f->upvalues[i]) + validateobjref(g, obj2gco(f), obj2gco(f->upvalues[i])); + + for (int i = 0; i < f->sizep; ++i) + if (f->p[i]) + validateobjref(g, obj2gco(f), obj2gco(f->p[i])); + + for (int i = 0; i < f->sizelocvars; i++) + if (f->locvars[i].varname) + validateobjref(g, obj2gco(f), obj2gco(f->locvars[i].varname)); +} + +static void validateobj(global_State* g, GCObject* o) +{ + /* dead objects can only occur during sweep */ + if (isdead(g, o)) + { + LUAU_ASSERT(g->gcstate == GCSsweepstring || g->gcstate == GCSsweep); + return; + } + + switch (o->gch.tt) + { + case LUA_TSTRING: + break; + + case LUA_TTABLE: + validatetable(g, gco2h(o)); + break; + + case LUA_TFUNCTION: + validateclosure(g, gco2cl(o)); + break; + + case LUA_TUSERDATA: + if (gco2u(o)->metatable) + validateobjref(g, o, obj2gco(gco2u(o)->metatable)); + break; + + case LUA_TTHREAD: + validatestack(g, gco2th(o)); + break; + + case LUA_TPROTO: + validateproto(g, gco2p(o)); + break; + + case LUA_TUPVAL: + validateref(g, o, gco2uv(o)->v); + break; + + default: + LUAU_ASSERT(!"unexpected object type"); + } +} + +static void validatelist(global_State* g, GCObject* o) +{ + while (o) + { + validateobj(g, o); + + o = o->gch.next; + } +} + +static void validategraylist(global_State* g, GCObject* o) +{ + if (!keepinvariant(g)) + return; + + while (o) + { + LUAU_ASSERT(isgray(o)); + + switch (o->gch.tt) + { + case LUA_TTABLE: + o = gco2h(o)->gclist; + break; + case LUA_TFUNCTION: + o = gco2cl(o)->gclist; + break; + case LUA_TTHREAD: + o = gco2th(o)->gclist; + break; + case LUA_TPROTO: + o = gco2p(o)->gclist; + break; + default: + LUAU_ASSERT(!"unknown object in gray list"); + return; + } + } +} + +void luaC_validate(lua_State* L) +{ + global_State* g = L->global; + + LUAU_ASSERT(!isdead(g, obj2gco(g->mainthread))); + checkliveness(g, &g->registry); + + for (int i = 0; i < LUA_T_COUNT; ++i) + if (g->mt[i]) + LUAU_ASSERT(!isdead(g, obj2gco(g->mt[i]))); + + validategraylist(g, g->weak); + validategraylist(g, g->gray); + validategraylist(g, g->grayagain); + + for (int i = 0; i < g->strt.size; ++i) + validatelist(g, g->strt.hash[i]); + + validatelist(g, g->rootgc); + validatelist(g, g->strbufgc); + + for (UpVal* uv = g->uvhead.u.l.next; uv != &g->uvhead; uv = uv->u.l.next) + { + LUAU_ASSERT(uv->tt == LUA_TUPVAL); + LUAU_ASSERT(uv->v != &uv->u.value); + LUAU_ASSERT(uv->u.l.next->u.l.prev == uv && uv->u.l.prev->u.l.next == uv); + } +} + +inline bool safejson(char ch) +{ + return unsigned(ch) < 128 && ch >= 32 && ch != '\\' && ch != '\"'; +} + +static void dumpref(FILE* f, GCObject* o) +{ + fprintf(f, "\"%p\"", o); +} + +static void dumprefs(FILE* f, TValue* data, size_t size) +{ + bool first = true; + + for (size_t i = 0; i < size; ++i) + { + if (iscollectable(&data[i])) + { + if (!first) + fputc(',', f); + first = false; + + dumpref(f, gcvalue(&data[i])); + } + } +} + +static void dumpstringdata(FILE* f, const char* data, size_t len) +{ + for (size_t i = 0; i < len; ++i) + fputc(safejson(data[i]) ? data[i] : '?', f); +} + +static void dumpstring(FILE* f, TString* ts) +{ + fprintf(f, "{\"type\":\"string\",\"cat\":%d,\"size\":%d,\"data\":\"", ts->memcat, int(sizestring(ts->len))); + dumpstringdata(f, ts->data, ts->len); + fprintf(f, "\"}"); +} + +static void dumptable(FILE* f, Table* h) +{ + size_t size = sizeof(Table) + (h->node == &luaH_dummynode ? 0 : sizenode(h) * sizeof(LuaNode)) + h->sizearray * sizeof(TValue); + + fprintf(f, "{\"type\":\"table\",\"cat\":%d,\"size\":%d", h->memcat, int(size)); + + if (h->node != &luaH_dummynode) + { + fprintf(f, ",\"pairs\":["); + + bool first = true; + + for (int i = 0; i < sizenode(h); ++i) + { + const LuaNode& n = h->node[i]; + + if (!ttisnil(&n.val) && (iscollectable(&n.key) || iscollectable(&n.val))) + { + if (!first) + fputc(',', f); + first = false; + + if (iscollectable(&n.key)) + dumpref(f, gcvalue(&n.key)); + else + fprintf(f, "null"); + + fputc(',', f); + + if (iscollectable(&n.val)) + dumpref(f, gcvalue(&n.val)); + else + fprintf(f, "null"); + } + } + + fprintf(f, "]"); + } + if (h->sizearray) + { + fprintf(f, ",\"array\":["); + dumprefs(f, h->array, h->sizearray); + fprintf(f, "]"); + } + if (h->metatable) + { + fprintf(f, ",\"metatable\":"); + dumpref(f, obj2gco(h->metatable)); + } + fprintf(f, "}"); +} + +static void dumpclosure(FILE* f, Closure* cl) +{ + fprintf(f, "{\"type\":\"function\",\"cat\":%d,\"size\":%d", cl->memcat, + cl->isC ? int(sizeCclosure(cl->nupvalues)) : int(sizeLclosure(cl->nupvalues))); + + fprintf(f, ",\"env\":"); + dumpref(f, obj2gco(cl->env)); + if (cl->isC) + { + if (cl->nupvalues) + { + fprintf(f, ",\"upvalues\":["); + dumprefs(f, cl->c.upvals, cl->nupvalues); + fprintf(f, "]"); + } + } + else + { + fprintf(f, ",\"proto\":"); + dumpref(f, obj2gco(cl->l.p)); + if (cl->nupvalues) + { + fprintf(f, ",\"upvalues\":["); + dumprefs(f, cl->l.uprefs, cl->nupvalues); + fprintf(f, "]"); + } + } + fprintf(f, "}"); +} + +static void dumpudata(FILE* f, Udata* u) +{ + fprintf(f, "{\"type\":\"userdata\",\"cat\":%d,\"size\":%d,\"tag\":%d", u->memcat, int(sizeudata(u->len)), u->tag); + + if (u->metatable) + { + fprintf(f, ",\"metatable\":"); + dumpref(f, obj2gco(u->metatable)); + } + fprintf(f, "}"); +} + +static void dumpthread(FILE* f, lua_State* th) +{ + size_t size = sizeof(lua_State) + sizeof(TValue) * th->stacksize + sizeof(CallInfo) * th->size_ci; + + fprintf(f, "{\"type\":\"thread\",\"cat\":%d,\"size\":%d", th->memcat, int(size)); + + if (iscollectable(&th->l_gt)) + { + fprintf(f, ",\"env\":"); + dumpref(f, gcvalue(&th->l_gt)); + } + + Closure* tcl = 0; + for (CallInfo* ci = th->base_ci; ci <= th->ci; ++ci) + { + if (ttisfunction(ci->func)) + { + tcl = clvalue(ci->func); + break; + } + } + + if (tcl && !tcl->isC && tcl->l.p->source) + { + Proto* p = tcl->l.p; + + fprintf(f, ",\"source\":\""); + dumpstringdata(f, p->source->data, p->source->len); + fprintf(f, "\",\"line\":%d", p->abslineinfo ? p->abslineinfo[0] : 0); + } + + if (th->top > th->stack) + { + fprintf(f, ",\"stack\":["); + dumprefs(f, th->stack, th->top - th->stack); + fprintf(f, "]"); + } + fprintf(f, "}"); +} + +static void dumpproto(FILE* f, Proto* p) +{ + size_t size = sizeof(Proto) + sizeof(Instruction) * p->sizecode + sizeof(Proto*) * p->sizep + sizeof(TValue) * p->sizek + p->sizelineinfo + + sizeof(LocVar) * p->sizelocvars + sizeof(TString*) * p->sizeupvalues; + + fprintf(f, "{\"type\":\"proto\",\"cat\":%d,\"size\":%d", p->memcat, int(size)); + + if (p->source) + { + fprintf(f, ",\"source\":\""); + dumpstringdata(f, p->source->data, p->source->len); + fprintf(f, "\",\"line\":%d", p->abslineinfo ? p->abslineinfo[0] : 0); + } + + if (p->sizek) + { + fprintf(f, ",\"constants\":["); + dumprefs(f, p->k, p->sizek); + fprintf(f, "]"); + } + + if (p->sizep) + { + fprintf(f, ",\"protos\":["); + for (int i = 0; i < p->sizep; ++i) + { + if (i != 0) + fputc(',', f); + dumpref(f, obj2gco(p->p[i])); + } + fprintf(f, "]"); + } + + fprintf(f, "}"); +} + +static void dumpupval(FILE* f, UpVal* uv) +{ + fprintf(f, "{\"type\":\"upvalue\",\"cat\":%d,\"size\":%d", uv->memcat, int(sizeof(UpVal))); + + if (iscollectable(uv->v)) + { + fprintf(f, ",\"object\":"); + dumpref(f, gcvalue(uv->v)); + } + fprintf(f, "}"); +} + +static void dumpobj(FILE* f, GCObject* o) +{ + switch (o->gch.tt) + { + case LUA_TSTRING: + return dumpstring(f, gco2ts(o)); + + case LUA_TTABLE: + return dumptable(f, gco2h(o)); + + case LUA_TFUNCTION: + return dumpclosure(f, gco2cl(o)); + + case LUA_TUSERDATA: + return dumpudata(f, gco2u(o)); + + case LUA_TTHREAD: + return dumpthread(f, gco2th(o)); + + case LUA_TPROTO: + return dumpproto(f, gco2p(o)); + + case LUA_TUPVAL: + return dumpupval(f, gco2uv(o)); + + default: + LUAU_ASSERT(0); + } +} + +static void dumplist(FILE* f, GCObject* o) +{ + while (o) + { + dumpref(f, o); + fputc(':', f); + dumpobj(f, o); + fputc(',', f); + fputc('\n', f); + + // thread has additional list containing collectable objects that are not present in rootgc + if (o->gch.tt == LUA_TTHREAD) + dumplist(f, gco2th(o)->openupval); + + o = o->gch.next; + } +} + +void luaC_dump(lua_State* L, void* file, const char* (*categoryName)(lua_State* L, uint8_t memcat)) +{ + global_State* g = L->global; + FILE* f = static_cast(file); + + fprintf(f, "{\"objects\":{\n"); + dumplist(f, g->rootgc); + dumplist(f, g->strbufgc); + for (int i = 0; i < g->strt.size; ++i) + dumplist(f, g->strt.hash[i]); + + fprintf(f, "\"0\":{\"type\":\"userdata\",\"cat\":0,\"size\":0}\n"); // to avoid issues with trailing , + fprintf(f, "},\"roots\":{\n"); + fprintf(f, "\"mainthread\":"); + dumpref(f, obj2gco(g->mainthread)); + fprintf(f, ",\"registry\":"); + dumpref(f, gcvalue(&g->registry)); + + fprintf(f, "},\"stats\":{\n"); + + fprintf(f, "\"size\":%d,\n", int(g->totalbytes)); + + fprintf(f, "\"categories\":{\n"); + for (int i = 0; i < LUA_MEMORY_CATEGORIES; i++) + { + if (size_t bytes = g->memcatbytes[i]) + { + if (categoryName) + fprintf(f, "\"%d\":{\"name\":\"%s\", \"size\":%d},\n", i, categoryName(L, i), int(bytes)); + else + fprintf(f, "\"%d\":{\"size\":%d},\n", i, int(bytes)); + } + } + fprintf(f, "\"none\":{}\n"); // to avoid issues with trailing , + fprintf(f, "}\n"); + fprintf(f, "}}\n"); +} diff --git a/fuzz/proto.cpp b/fuzz/proto.cpp index ae2399e4..27e53492 100644 --- a/fuzz/proto.cpp +++ b/fuzz/proto.cpp @@ -23,9 +23,11 @@ const bool kFuzzCompiler = true; const bool kFuzzLinter = true; const bool kFuzzTypeck = true; const bool kFuzzVM = true; -const bool kFuzzTypes = true; const bool kFuzzTranspile = true; +// Should we generate type annotations? +const bool kFuzzTypes = true; + static_assert(!(kFuzzVM && !kFuzzCompiler), "VM requires the compiler!"); std::string protoprint(const luau::StatBlock& stat, bool types); diff --git a/tests/AstQuery.test.cpp b/tests/AstQuery.test.cpp index aa53a92b..2090b014 100644 --- a/tests/AstQuery.test.cpp +++ b/tests/AstQuery.test.cpp @@ -78,3 +78,26 @@ TEST_CASE_FIXTURE(DocumentationSymbolFixture, "overloaded_fn") } TEST_SUITE_END(); + +TEST_SUITE_BEGIN("AstQuery"); + +TEST_CASE_FIXTURE(Fixture, "last_argument_function_call_type") +{ + ScopedFastFlag luauTailArgumentTypeInfo{"LuauTailArgumentTypeInfo", true}; + + check(R"( +local function foo() return 2 end +local function bar(a: number) return -a end +bar(foo()) + )"); + + auto oty = findTypeAtPosition(Position(3, 7)); + REQUIRE(oty); + CHECK_EQ("number", toString(*oty)); + + auto expectedOty = findExpectedTypeAtPosition(Position(3, 7)); + REQUIRE(expectedOty); + CHECK_EQ("number", toString(*expectedOty)); +} + +TEST_SUITE_END(); diff --git a/tests/Autocomplete.test.cpp b/tests/Autocomplete.test.cpp index 5a7c8602..3b74a99e 100644 --- a/tests/Autocomplete.test.cpp +++ b/tests/Autocomplete.test.cpp @@ -1935,6 +1935,39 @@ return target(b@1 CHECK(ac.entryMap["bar2"].typeCorrect == TypeCorrectKind::None); } +TEST_CASE_FIXTURE(ACFixture, "function_in_assignment_has_parentheses") +{ + ScopedFastFlag luauAutocompleteAvoidMutation("LuauAutocompleteAvoidMutation", true); + ScopedFastFlag luauAutocompletePreferToCallFunctions("LuauAutocompletePreferToCallFunctions", true); + + check(R"( +local function bar(a: number) return -a end +local abc = b@1 + )"); + + auto ac = autocomplete('1'); + + CHECK(ac.entryMap.count("bar")); + CHECK(ac.entryMap["bar"].parens == ParenthesesRecommendation::CursorInside); +} + +TEST_CASE_FIXTURE(ACFixture, "function_result_passed_to_function_has_parentheses") +{ + ScopedFastFlag luauAutocompleteAvoidMutation("LuauAutocompleteAvoidMutation", true); + ScopedFastFlag luauAutocompletePreferToCallFunctions("LuauAutocompletePreferToCallFunctions", true); + + check(R"( +local function foo() return 1 end +local function bar(a: number) return -a end +local abc = bar(@1) + )"); + + auto ac = autocomplete('1'); + + CHECK(ac.entryMap.count("foo")); + CHECK(ac.entryMap["foo"].parens == ParenthesesRecommendation::CursorAfter); +} + TEST_CASE_FIXTURE(ACFixture, "type_correct_sealed_table") { check(R"( @@ -2210,8 +2243,6 @@ TEST_CASE_FIXTURE(ACFixture, "autocompleteSource") TEST_CASE_FIXTURE(ACFixture, "autocompleteSource_require") { - ScopedFastFlag luauResolveModuleNameWithoutACurrentModule("LuauResolveModuleNameWithoutACurrentModule", true); - std::string_view source = R"( local a = require(w -- Line 1 -- | Column 27 @@ -2287,8 +2318,6 @@ until TEST_CASE_FIXTURE(ACFixture, "if_then_else_elseif_completions") { - ScopedFastFlag sff{"ElseElseIfCompletionImprovements", true}; - check(R"( local elsewhere = false @@ -2585,9 +2614,6 @@ a = if temp then even elseif true then temp else e@9 TEST_CASE_FIXTURE(ACFixture, "autocomplete_explicit_type_pack") { - ScopedFastFlag luauTypeAliasPacks("LuauTypeAliasPacks", true); - ScopedFastFlag luauParseTypePackTypeParameters("LuauParseTypePackTypeParameters", true); - check(R"( type A = () -> T... local a: A<(number, s@1> diff --git a/tests/Compiler.test.cpp b/tests/Compiler.test.cpp index 4ce8d08a..6ba39ada 100644 --- a/tests/Compiler.test.cpp +++ b/tests/Compiler.test.cpp @@ -1057,6 +1057,18 @@ RETURN R0 1 CHECK_EQ("\n" + compileFunction0("return if false then 10 else 20"), R"( LOADN R0 20 RETURN R0 1 +)"); + + // codegen for a true constant condition with non-constant expressions + CHECK_EQ("\n" + compileFunction0("return if true then {} else error()"), R"( +NEWTABLE R0 0 0 +RETURN R0 1 +)"); + + // codegen for a false constant condition with non-constant expressions + CHECK_EQ("\n" + compileFunction0("return if false then error() else {}"), R"( +NEWTABLE R0 0 0 +RETURN R0 1 )"); // codegen for a false (in this case 'nil') constant condition @@ -2360,6 +2372,58 @@ Foo:Bar( )"); } +TEST_CASE("DebugLineInfoCallChain") +{ + Luau::BytecodeBuilder bcb; + bcb.setDumpFlags(Luau::BytecodeBuilder::Dump_Code | Luau::BytecodeBuilder::Dump_Lines); + Luau::compileOrThrow(bcb, R"( +local Foo = ... + +Foo +:Bar(1) +:Baz(2) +.Qux(3) +)"); + + CHECK_EQ("\n" + bcb.dumpFunction(0), R"( +2: GETVARARGS R0 1 +5: LOADN R4 1 +5: NAMECALL R2 R0 K0 +5: CALL R2 2 1 +6: LOADN R4 2 +6: NAMECALL R2 R2 K1 +6: CALL R2 2 1 +7: GETTABLEKS R1 R2 K2 +7: LOADN R2 3 +7: CALL R1 1 0 +8: RETURN R0 0 +)"); +} + +TEST_CASE("DebugLineInfoFastCall") +{ + Luau::BytecodeBuilder bcb; + bcb.setDumpFlags(Luau::BytecodeBuilder::Dump_Code | Luau::BytecodeBuilder::Dump_Lines); + Luau::compileOrThrow(bcb, R"( +local Foo, Bar = ... + +return + math.max( + Foo, + Bar) +)"); + + CHECK_EQ("\n" + bcb.dumpFunction(0), R"( +2: GETVARARGS R0 2 +5: FASTCALL2 18 R0 R1 +5 +5: MOVE R3 R0 +5: MOVE R4 R1 +5: GETIMPORT R2 2 +5: CALL R2 2 -1 +5: RETURN R2 -1 +)"); +} + TEST_CASE("DebugSource") { const char* source = R"( @@ -3742,4 +3806,108 @@ RETURN R0 0 )"); } +TEST_CASE("ConstantsNoFolding") +{ + const char* source = "return nil, true, 42, 'hello'"; + + Luau::BytecodeBuilder bcb; + bcb.setDumpFlags(Luau::BytecodeBuilder::Dump_Code); + Luau::CompileOptions options; + options.optimizationLevel = 0; + Luau::compileOrThrow(bcb, source, options); + + CHECK_EQ("\n" + bcb.dumpFunction(0), R"( +LOADNIL R0 +LOADB R1 1 +LOADK R2 K0 +LOADK R3 K1 +RETURN R0 4 +)"); +} + +TEST_CASE("VectorFastCall") +{ + const char* source = "return Vector3.new(1, 2, 3)"; + + Luau::BytecodeBuilder bcb; + bcb.setDumpFlags(Luau::BytecodeBuilder::Dump_Code); + Luau::CompileOptions options; + options.vectorLib = "Vector3"; + options.vectorCtor = "new"; + Luau::compileOrThrow(bcb, source, options); + + CHECK_EQ("\n" + bcb.dumpFunction(0), R"( +LOADN R1 1 +LOADN R2 2 +LOADN R3 3 +FASTCALL 54 +2 +GETIMPORT R0 2 +CALL R0 3 -1 +RETURN R0 -1 +)"); +} + +TEST_CASE("TypeAssertion") +{ + // validate that type assertions work with the compiler and that the code inside type assertion isn't evaluated + CHECK_EQ("\n" + compileFunction0(R"( +print(foo() :: typeof(error("compile time"))) +)"), + R"( +GETIMPORT R0 1 +GETIMPORT R1 3 +CALL R1 0 1 +CALL R0 1 0 +RETURN R0 0 +)"); + + // note that above, foo() is treated as single-arg function; removing type assertion changes the bytecode + CHECK_EQ("\n" + compileFunction0(R"( +print(foo()) +)"), + R"( +GETIMPORT R0 1 +GETIMPORT R1 3 +CALL R1 0 -1 +CALL R0 -1 0 +RETURN R0 0 +)"); +} + +TEST_CASE("Arithmetics") +{ + // basic arithmetics codegen with non-constants + CHECK_EQ("\n" + compileFunction0(R"( +local a, b = ... +return a + b, a - b, a / b, a * b, a % b, a ^ b +)"), + R"( +GETVARARGS R0 2 +ADD R2 R0 R1 +SUB R3 R0 R1 +DIV R4 R0 R1 +MUL R5 R0 R1 +MOD R6 R0 R1 +POW R7 R0 R1 +RETURN R2 6 +)"); + + // basic arithmetics codegen with constants on the right side + // note that we don't simplify these expressions as we don't know the type of a + CHECK_EQ("\n" + compileFunction0(R"( +local a = ... +return a + 1, a - 1, a / 1, a * 1, a % 1, a ^ 1 +)"), + R"( +GETVARARGS R0 1 +ADDK R1 R0 K0 +SUBK R2 R0 K0 +DIVK R3 R0 K0 +MULK R4 R0 K0 +MODK R5 R0 K0 +POWK R6 R0 K0 +RETURN R1 6 +)"); +} + TEST_SUITE_END(); diff --git a/tests/Conformance.test.cpp b/tests/Conformance.test.cpp index 5ee84244..b2aad316 100644 --- a/tests/Conformance.test.cpp +++ b/tests/Conformance.test.cpp @@ -508,6 +508,9 @@ TEST_CASE("Debugger") cb->debugbreak = [](lua_State* L, lua_Debug* ar) { breakhits++; + // make sure we can trace the stack for every breakpoint we hit + lua_debugtrace(L); + // for every breakpoint, we break on the first invocation and continue on second // this allows us to easily step off breakpoints // (real implementaiton may require singlestepping) @@ -703,21 +706,52 @@ TEST_CASE("ApiFunctionCalls") StateRef globalState = runConformance("apicalls.lua"); lua_State* L = globalState.get(); - lua_getfield(L, LUA_GLOBALSINDEX, "add"); - lua_pushnumber(L, 40); - lua_pushnumber(L, 2); - lua_call(L, 2, 1); - CHECK(lua_isnumber(L, -1)); - CHECK(lua_tonumber(L, -1) == 42); - lua_pop(L, 1); + // lua_call + { + lua_getfield(L, LUA_GLOBALSINDEX, "add"); + lua_pushnumber(L, 40); + lua_pushnumber(L, 2); + lua_call(L, 2, 1); + CHECK(lua_isnumber(L, -1)); + CHECK(lua_tonumber(L, -1) == 42); + lua_pop(L, 1); + } - lua_getfield(L, LUA_GLOBALSINDEX, "add"); - lua_pushnumber(L, 40); - lua_pushnumber(L, 2); - lua_pcall(L, 2, 1, 0); - CHECK(lua_isnumber(L, -1)); - CHECK(lua_tonumber(L, -1) == 42); - lua_pop(L, 1); + // lua_pcall + { + lua_getfield(L, LUA_GLOBALSINDEX, "add"); + lua_pushnumber(L, 40); + lua_pushnumber(L, 2); + lua_pcall(L, 2, 1, 0); + CHECK(lua_isnumber(L, -1)); + CHECK(lua_tonumber(L, -1) == 42); + lua_pop(L, 1); + } + + // lua_equal with a sleeping thread wake up + { + ScopedFastFlag luauActivateBeforeExec("LuauActivateBeforeExec", true); + + lua_State* L2 = lua_newthread(L); + + lua_getfield(L2, LUA_GLOBALSINDEX, "create_with_tm"); + lua_pushnumber(L2, 42); + lua_pcall(L2, 1, 1, 0); + + lua_getfield(L2, LUA_GLOBALSINDEX, "create_with_tm"); + lua_pushnumber(L2, 42); + lua_pcall(L2, 1, 1, 0); + + // Reset GC + lua_gc(L2, LUA_GCCOLLECT, 0); + + // Try to mark 'L2' as sleeping + // Can't control GC precisely, even in tests + lua_gc(L2, LUA_GCSTEP, 8); + + CHECK(lua_equal(L2, -1, -2) == 1); + lua_pop(L2, 2); + } } static bool endsWith(const std::string& str, const std::string& suffix) @@ -731,8 +765,6 @@ static bool endsWith(const std::string& str, const std::string& suffix) #if !LUA_USE_LONGJMP TEST_CASE("ExceptionObject") { - ScopedFastFlag sff("LuauExceptionMessageFix", true); - struct ExceptionResult { bool exceptionGenerated; diff --git a/tests/Fixture.cpp b/tests/Fixture.cpp index 29c33f7c..36d6f561 100644 --- a/tests/Fixture.cpp +++ b/tests/Fixture.cpp @@ -19,19 +19,6 @@ static const char* mainModuleName = "MainModule"; namespace Luau { -std::optional TestFileResolver::fromAstFragment(AstExpr* expr) const -{ - auto g = expr->as(); - if (!g) - return std::nullopt; - - std::string_view value = g->name.value; - if (value == "game" || value == "Game" || value == "workspace" || value == "Workspace" || value == "script" || value == "Script") - return ModuleName(value); - - return std::nullopt; -} - std::optional TestFileResolver::resolveModule(const ModuleInfo* context, AstExpr* expr) { if (AstExprGlobal* g = expr->as()) @@ -81,24 +68,6 @@ std::optional TestFileResolver::resolveModule(const ModuleInfo* cont return std::nullopt; } -ModuleName TestFileResolver::concat(const ModuleName& lhs, std::string_view rhs) const -{ - return lhs + "/" + ModuleName(rhs); -} - -std::optional TestFileResolver::getParentModuleName(const ModuleName& name) const -{ - std::string_view view = name; - const size_t lastSeparatorIndex = view.find_last_of('/'); - - if (lastSeparatorIndex != std::string_view::npos) - { - return ModuleName(view.substr(0, lastSeparatorIndex)); - } - - return std::nullopt; -} - std::string TestFileResolver::getHumanReadableModuleName(const ModuleName& name) const { return name; @@ -324,6 +293,13 @@ std::optional Fixture::findTypeAtPosition(Position position) return Luau::findTypeAtPosition(*module, *sourceModule, position); } +std::optional Fixture::findExpectedTypeAtPosition(Position position) +{ + ModulePtr module = getMainModule(); + SourceModule* sourceModule = getMainSourceModule(); + return Luau::findExpectedTypeAtPosition(*module, *sourceModule, position); +} + TypeId Fixture::requireTypeAtPosition(Position position) { auto ty = findTypeAtPosition(position); diff --git a/tests/Fixture.h b/tests/Fixture.h index 1480a7f6..de2b7381 100644 --- a/tests/Fixture.h +++ b/tests/Fixture.h @@ -64,12 +64,8 @@ struct TestFileResolver return SourceCode{it->second, sourceType}; } - std::optional fromAstFragment(AstExpr* expr) const override; std::optional resolveModule(const ModuleInfo* context, AstExpr* expr) override; - ModuleName concat(const ModuleName& lhs, std::string_view rhs) const override; - std::optional getParentModuleName(const ModuleName& name) const override; - std::string getHumanReadableModuleName(const ModuleName& name) const override; std::optional getEnvironmentForModule(const ModuleName& name) const override; @@ -126,6 +122,7 @@ struct Fixture std::optional findTypeAtPosition(Position position); TypeId requireTypeAtPosition(Position position); + std::optional findExpectedTypeAtPosition(Position position); std::optional lookupType(const std::string& name); std::optional lookupImportedType(const std::string& moduleAlias, const std::string& name); diff --git a/tests/Frontend.test.cpp b/tests/Frontend.test.cpp index fbfec636..51fcd3d6 100644 --- a/tests/Frontend.test.cpp +++ b/tests/Frontend.test.cpp @@ -46,18 +46,6 @@ NaiveModuleResolver naiveModuleResolver; struct NaiveFileResolver : NullFileResolver { - std::optional fromAstFragment(AstExpr* expr) const override - { - AstExprGlobal* g = expr->as(); - if (g && g->name == "Modules") - return "Modules"; - - if (g && g->name == "game") - return "game"; - - return std::nullopt; - } - std::optional resolveModule(const ModuleInfo* context, AstExpr* expr) override { if (AstExprGlobal* g = expr->as()) @@ -86,11 +74,6 @@ struct NaiveFileResolver : NullFileResolver return std::nullopt; } - - ModuleName concat(const ModuleName& lhs, std::string_view rhs) const override - { - return lhs + "/" + ModuleName(rhs); - } }; } // namespace diff --git a/tests/Linter.test.cpp b/tests/Linter.test.cpp index 7ba40c50..1d13df28 100644 --- a/tests/Linter.test.cpp +++ b/tests/Linter.test.cpp @@ -1469,6 +1469,22 @@ _ = true and true or false -- no warning since this is is a common pattern used CHECK_EQ(result.warnings[6].location.begin.line + 1, 19); } +TEST_CASE_FIXTURE(Fixture, "DuplicateConditionsExpr") +{ + LintResult result = lint(R"( +local correct, opaque = ... + +if correct({a = 1, b = 2 * (-2), c = opaque.path['with']("calls")}) then +elseif correct({a = 1, b = 2 * (-2), c = opaque.path['with']("calls")}) then +elseif correct({a = 1, b = 2 * (-2), c = opaque.path['with']("calls", false)}) then +end +)"); + + REQUIRE_EQ(result.warnings.size(), 1); + CHECK_EQ(result.warnings[0].text, "Condition has already been checked on line 4"); + CHECK_EQ(result.warnings[0].location.begin.line + 1, 5); +} + TEST_CASE_FIXTURE(Fixture, "DuplicateLocal") { LintResult result = lint(R"( diff --git a/tests/Module.test.cpp b/tests/Module.test.cpp index 7a3543c7..e3993cc5 100644 --- a/tests/Module.test.cpp +++ b/tests/Module.test.cpp @@ -44,9 +44,10 @@ TEST_CASE_FIXTURE(Fixture, "dont_clone_persistent_primitive") SeenTypes seenTypes; SeenTypePacks seenTypePacks; + CloneState cloneState; // numberType is persistent. We leave it as-is. - TypeId newNumber = clone(typeChecker.numberType, dest, seenTypes, seenTypePacks); + TypeId newNumber = clone(typeChecker.numberType, dest, seenTypes, seenTypePacks, cloneState); CHECK_EQ(newNumber, typeChecker.numberType); } @@ -56,12 +57,13 @@ TEST_CASE_FIXTURE(Fixture, "deepClone_non_persistent_primitive") SeenTypes seenTypes; SeenTypePacks seenTypePacks; + CloneState cloneState; // Create a new number type that isn't persistent unfreeze(typeChecker.globalTypes); TypeId oldNumber = typeChecker.globalTypes.addType(PrimitiveTypeVar{PrimitiveTypeVar::Number}); freeze(typeChecker.globalTypes); - TypeId newNumber = clone(oldNumber, dest, seenTypes, seenTypePacks); + TypeId newNumber = clone(oldNumber, dest, seenTypes, seenTypePacks, cloneState); CHECK_NE(newNumber, oldNumber); CHECK_EQ(*oldNumber, *newNumber); @@ -89,9 +91,10 @@ TEST_CASE_FIXTURE(Fixture, "deepClone_cyclic_table") SeenTypes seenTypes; SeenTypePacks seenTypePacks; + CloneState cloneState; TypeArena dest; - TypeId counterCopy = clone(counterType, dest, seenTypes, seenTypePacks); + TypeId counterCopy = clone(counterType, dest, seenTypes, seenTypePacks, cloneState); TableTypeVar* ttv = getMutable(counterCopy); REQUIRE(ttv != nullptr); @@ -142,11 +145,12 @@ TEST_CASE_FIXTURE(Fixture, "deepClone_union") SeenTypes seenTypes; SeenTypePacks seenTypePacks; + CloneState cloneState; unfreeze(typeChecker.globalTypes); TypeId oldUnion = typeChecker.globalTypes.addType(UnionTypeVar{{typeChecker.numberType, typeChecker.stringType}}); freeze(typeChecker.globalTypes); - TypeId newUnion = clone(oldUnion, dest, seenTypes, seenTypePacks); + TypeId newUnion = clone(oldUnion, dest, seenTypes, seenTypePacks, cloneState); CHECK_NE(newUnion, oldUnion); CHECK_EQ("number | string", toString(newUnion)); @@ -159,11 +163,12 @@ TEST_CASE_FIXTURE(Fixture, "deepClone_intersection") SeenTypes seenTypes; SeenTypePacks seenTypePacks; + CloneState cloneState; unfreeze(typeChecker.globalTypes); TypeId oldIntersection = typeChecker.globalTypes.addType(IntersectionTypeVar{{typeChecker.numberType, typeChecker.stringType}}); freeze(typeChecker.globalTypes); - TypeId newIntersection = clone(oldIntersection, dest, seenTypes, seenTypePacks); + TypeId newIntersection = clone(oldIntersection, dest, seenTypes, seenTypePacks, cloneState); CHECK_NE(newIntersection, oldIntersection); CHECK_EQ("number & string", toString(newIntersection)); @@ -188,8 +193,9 @@ TEST_CASE_FIXTURE(Fixture, "clone_class") SeenTypes seenTypes; SeenTypePacks seenTypePacks; + CloneState cloneState; - TypeId cloned = clone(&exampleClass, dest, seenTypes, seenTypePacks); + TypeId cloned = clone(&exampleClass, dest, seenTypes, seenTypePacks, cloneState); const ClassTypeVar* ctv = get(cloned); REQUIRE(ctv != nullptr); @@ -211,16 +217,16 @@ TEST_CASE_FIXTURE(Fixture, "clone_sanitize_free_types") TypeArena dest; SeenTypes seenTypes; SeenTypePacks seenTypePacks; + CloneState cloneState; - bool encounteredFreeType = false; - TypeId clonedTy = clone(&freeTy, dest, seenTypes, seenTypePacks, &encounteredFreeType); + TypeId clonedTy = clone(&freeTy, dest, seenTypes, seenTypePacks, cloneState); CHECK_EQ("any", toString(clonedTy)); - CHECK(encounteredFreeType); + CHECK(cloneState.encounteredFreeType); - encounteredFreeType = false; - TypePackId clonedTp = clone(&freeTp, dest, seenTypes, seenTypePacks, &encounteredFreeType); + cloneState = {}; + TypePackId clonedTp = clone(&freeTp, dest, seenTypes, seenTypePacks, cloneState); CHECK_EQ("...any", toString(clonedTp)); - CHECK(encounteredFreeType); + CHECK(cloneState.encounteredFreeType); } TEST_CASE_FIXTURE(Fixture, "clone_seal_free_tables") @@ -232,12 +238,12 @@ TEST_CASE_FIXTURE(Fixture, "clone_seal_free_tables") TypeArena dest; SeenTypes seenTypes; SeenTypePacks seenTypePacks; + CloneState cloneState; - bool encounteredFreeType = false; - TypeId cloned = clone(&tableTy, dest, seenTypes, seenTypePacks, &encounteredFreeType); + TypeId cloned = clone(&tableTy, dest, seenTypes, seenTypePacks, cloneState); const TableTypeVar* clonedTtv = get(cloned); CHECK_EQ(clonedTtv->state, TableState::Sealed); - CHECK(encounteredFreeType); + CHECK(cloneState.encounteredFreeType); } TEST_CASE_FIXTURE(Fixture, "clone_self_property") @@ -267,4 +273,34 @@ TEST_CASE_FIXTURE(Fixture, "clone_self_property") "dot or pass 1 extra nil to suppress this warning"); } +TEST_CASE_FIXTURE(Fixture, "clone_recursion_limit") +{ +#if defined(_DEBUG) || defined(_NOOPT) + int limit = 250; +#else + int limit = 400; +#endif + ScopedFastInt luauTypeCloneRecursionLimit{"LuauTypeCloneRecursionLimit", limit}; + + TypeArena src; + + TypeId table = src.addType(TableTypeVar{}); + TypeId nested = table; + + for (int i = 0; i < limit + 100; i++) + { + TableTypeVar* ttv = getMutable(nested); + + ttv->props["a"].type = src.addType(TableTypeVar{}); + nested = ttv->props["a"].type; + } + + TypeArena dest; + SeenTypes seenTypes; + SeenTypePacks seenTypePacks; + CloneState cloneState; + + CHECK_THROWS_AS(clone(table, dest, seenTypes, seenTypePacks, cloneState), std::runtime_error); +} + TEST_SUITE_END(); diff --git a/tests/Parser.test.cpp b/tests/Parser.test.cpp index e3e6ce6d..72d3a9a6 100644 --- a/tests/Parser.test.cpp +++ b/tests/Parser.test.cpp @@ -2518,8 +2518,6 @@ TEST_CASE_FIXTURE(Fixture, "parse_if_else_expression") TEST_CASE_FIXTURE(Fixture, "parse_type_pack_type_parameters") { - ScopedFastFlag luauParseTypePackTypeParameters("LuauParseTypePackTypeParameters", true); - AstStat* stat = parse(R"( type Packed = () -> T... diff --git a/tests/ToDot.test.cpp b/tests/ToDot.test.cpp new file mode 100644 index 00000000..0ca9c994 --- /dev/null +++ b/tests/ToDot.test.cpp @@ -0,0 +1,366 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details + +#include "Luau/Scope.h" +#include "Luau/ToDot.h" + +#include "Fixture.h" + +#include "doctest.h" + +using namespace Luau; + +struct ToDotClassFixture : Fixture +{ + ToDotClassFixture() + { + TypeArena& arena = typeChecker.globalTypes; + + unfreeze(arena); + + TypeId baseClassMetaType = arena.addType(TableTypeVar{}); + + TypeId baseClassInstanceType = arena.addType(ClassTypeVar{"BaseClass", {}, std::nullopt, baseClassMetaType, {}, {}}); + getMutable(baseClassInstanceType)->props = { + {"BaseField", {typeChecker.numberType}}, + }; + typeChecker.globalScope->exportedTypeBindings["BaseClass"] = TypeFun{{}, baseClassInstanceType}; + + TypeId childClassInstanceType = arena.addType(ClassTypeVar{"ChildClass", {}, baseClassInstanceType, std::nullopt, {}, {}}); + getMutable(childClassInstanceType)->props = { + {"ChildField", {typeChecker.stringType}}, + }; + typeChecker.globalScope->exportedTypeBindings["ChildClass"] = TypeFun{{}, childClassInstanceType}; + + freeze(arena); + } +}; + +TEST_SUITE_BEGIN("ToDot"); + +TEST_CASE_FIXTURE(Fixture, "primitive") +{ + CheckResult result = check(R"( +local a: nil +local b: number +local c: any +)"); + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_NE("nil", toDot(requireType("a"))); + + CHECK_EQ(R"(digraph graphname { +n1 [label="number"]; +})", + toDot(requireType("b"))); + + CHECK_EQ(R"(digraph graphname { +n1 [label="any"]; +})", + toDot(requireType("c"))); + + ToDotOptions opts; + opts.showPointers = false; + opts.duplicatePrimitives = false; + + CHECK_EQ(R"(digraph graphname { +n1 [label="PrimitiveTypeVar number"]; +})", + toDot(requireType("b"), opts)); + + CHECK_EQ(R"(digraph graphname { +n1 [label="AnyTypeVar 1"]; +})", + toDot(requireType("c"), opts)); +} + +TEST_CASE_FIXTURE(Fixture, "bound") +{ + CheckResult result = check(R"( +local a = 444 +local b = a +)"); + LUAU_REQUIRE_NO_ERRORS(result); + + std::optional ty = getType("b"); + REQUIRE(bool(ty)); + + ToDotOptions opts; + opts.showPointers = false; + CHECK_EQ(R"(digraph graphname { +n1 [label="BoundTypeVar 1"]; +n1 -> n2; +n2 [label="number"]; +})", + toDot(*ty, opts)); +} + +TEST_CASE_FIXTURE(Fixture, "function") +{ + ScopedFastFlag luauQuantifyInPlace2{"LuauQuantifyInPlace2", true}; + + CheckResult result = check(R"( +local function f(a, ...: string) return a end +)"); + LUAU_REQUIRE_NO_ERRORS(result); + + ToDotOptions opts; + opts.showPointers = false; + CHECK_EQ(R"(digraph graphname { +n1 [label="FunctionTypeVar 1"]; +n1 -> n2 [label="arg"]; +n2 [label="TypePack 2"]; +n2 -> n3; +n3 [label="GenericTypeVar 3"]; +n2 -> n4 [label="tail"]; +n4 [label="VariadicTypePack 4"]; +n4 -> n5; +n5 [label="string"]; +n1 -> n6 [label="ret"]; +n6 [label="BoundTypePack 6"]; +n6 -> n7; +n7 [label="TypePack 7"]; +n7 -> n3; +})", + toDot(requireType("f"), opts)); +} + +TEST_CASE_FIXTURE(Fixture, "union") +{ + CheckResult result = check(R"( +local a: string | number +)"); + LUAU_REQUIRE_NO_ERRORS(result); + + ToDotOptions opts; + opts.showPointers = false; + CHECK_EQ(R"(digraph graphname { +n1 [label="UnionTypeVar 1"]; +n1 -> n2; +n2 [label="string"]; +n1 -> n3; +n3 [label="number"]; +})", + toDot(requireType("a"), opts)); +} + +TEST_CASE_FIXTURE(Fixture, "intersection") +{ + CheckResult result = check(R"( +local a: string & number -- uninhabited +)"); + LUAU_REQUIRE_NO_ERRORS(result); + + ToDotOptions opts; + opts.showPointers = false; + CHECK_EQ(R"(digraph graphname { +n1 [label="IntersectionTypeVar 1"]; +n1 -> n2; +n2 [label="string"]; +n1 -> n3; +n3 [label="number"]; +})", + toDot(requireType("a"), opts)); +} + +TEST_CASE_FIXTURE(Fixture, "table") +{ + CheckResult result = check(R"( +type A = { x: T, y: (U...) -> (), [string]: any } +local a: A +)"); + LUAU_REQUIRE_NO_ERRORS(result); + + ToDotOptions opts; + opts.showPointers = false; + CHECK_EQ(R"(digraph graphname { +n1 [label="TableTypeVar A"]; +n1 -> n2 [label="x"]; +n2 [label="number"]; +n1 -> n3 [label="y"]; +n3 [label="FunctionTypeVar 3"]; +n3 -> n4 [label="arg"]; +n4 [label="VariadicTypePack 4"]; +n4 -> n5; +n5 [label="string"]; +n3 -> n6 [label="ret"]; +n6 [label="TypePack 6"]; +n1 -> n7 [label="[index]"]; +n7 [label="string"]; +n1 -> n8 [label="[value]"]; +n8 [label="any"]; +n1 -> n9 [label="typeParam"]; +n9 [label="number"]; +n1 -> n4 [label="typePackParam"]; +})", + toDot(requireType("a"), opts)); + + // Extra coverage with pointers (unstable values) + (void)toDot(requireType("a")); +} + +TEST_CASE_FIXTURE(Fixture, "metatable") +{ + CheckResult result = check(R"( +local a: typeof(setmetatable({}, {})) +)"); + LUAU_REQUIRE_NO_ERRORS(result); + + ToDotOptions opts; + opts.showPointers = false; + CHECK_EQ(R"(digraph graphname { +n1 [label="MetatableTypeVar 1"]; +n1 -> n2 [label="table"]; +n2 [label="TableTypeVar 2"]; +n1 -> n3 [label="metatable"]; +n3 [label="TableTypeVar 3"]; +})", + toDot(requireType("a"), opts)); +} + +TEST_CASE_FIXTURE(Fixture, "free") +{ + TypeVar type{TypeVariant{FreeTypeVar{TypeLevel{0, 0}}}}; + + ToDotOptions opts; + opts.showPointers = false; + CHECK_EQ(R"(digraph graphname { +n1 [label="FreeTypeVar 1"]; +})", + toDot(&type, opts)); +} + +TEST_CASE_FIXTURE(Fixture, "error") +{ + TypeVar type{TypeVariant{ErrorTypeVar{}}}; + + ToDotOptions opts; + opts.showPointers = false; + CHECK_EQ(R"(digraph graphname { +n1 [label="ErrorTypeVar 1"]; +})", + toDot(&type, opts)); +} + +TEST_CASE_FIXTURE(Fixture, "generic") +{ + TypeVar type{TypeVariant{GenericTypeVar{"T"}}}; + + ToDotOptions opts; + opts.showPointers = false; + CHECK_EQ(R"(digraph graphname { +n1 [label="GenericTypeVar T"]; +})", + toDot(&type, opts)); +} + +TEST_CASE_FIXTURE(ToDotClassFixture, "class") +{ + CheckResult result = check(R"( +local a: ChildClass +)"); + LUAU_REQUIRE_NO_ERRORS(result); + + ToDotOptions opts; + opts.showPointers = false; + CHECK_EQ(R"(digraph graphname { +n1 [label="ClassTypeVar ChildClass"]; +n1 -> n2 [label="ChildField"]; +n2 [label="string"]; +n1 -> n3 [label="[parent]"]; +n3 [label="ClassTypeVar BaseClass"]; +n3 -> n4 [label="BaseField"]; +n4 [label="number"]; +n3 -> n5 [label="[metatable]"]; +n5 [label="TableTypeVar 5"]; +})", + toDot(requireType("a"), opts)); +} + +TEST_CASE_FIXTURE(Fixture, "free_pack") +{ + TypePackVar pack{TypePackVariant{FreeTypePack{TypeLevel{0, 0}}}}; + + ToDotOptions opts; + opts.showPointers = false; + CHECK_EQ(R"(digraph graphname { +n1 [label="FreeTypePack 1"]; +})", + toDot(&pack, opts)); +} + +TEST_CASE_FIXTURE(Fixture, "error_pack") +{ + TypePackVar pack{TypePackVariant{Unifiable::Error{}}}; + + ToDotOptions opts; + opts.showPointers = false; + CHECK_EQ(R"(digraph graphname { +n1 [label="ErrorTypePack 1"]; +})", + toDot(&pack, opts)); + + // Extra coverage with pointers (unstable values) + (void)toDot(&pack); +} + +TEST_CASE_FIXTURE(Fixture, "generic_pack") +{ + TypePackVar pack1{TypePackVariant{GenericTypePack{}}}; + TypePackVar pack2{TypePackVariant{GenericTypePack{"T"}}}; + + ToDotOptions opts; + opts.showPointers = false; + CHECK_EQ(R"(digraph graphname { +n1 [label="GenericTypePack 1"]; +})", + toDot(&pack1, opts)); + + CHECK_EQ(R"(digraph graphname { +n1 [label="GenericTypePack T"]; +})", + toDot(&pack2, opts)); +} + +TEST_CASE_FIXTURE(Fixture, "bound_pack") +{ + TypePackVar pack{TypePackVariant{TypePack{{typeChecker.numberType}, {}}}}; + TypePackVar bound{TypePackVariant{BoundTypePack{&pack}}}; + + ToDotOptions opts; + opts.showPointers = false; + CHECK_EQ(R"(digraph graphname { +n1 [label="BoundTypePack 1"]; +n1 -> n2; +n2 [label="TypePack 2"]; +n2 -> n3; +n3 [label="number"]; +})", + toDot(&bound, opts)); +} + +TEST_CASE_FIXTURE(Fixture, "bound_table") +{ + CheckResult result = check(R"( +local a = {x=2} +local b +b.x = 2 +b = a +)"); + LUAU_REQUIRE_NO_ERRORS(result); + + std::optional ty = getType("b"); + REQUIRE(bool(ty)); + + ToDotOptions opts; + opts.showPointers = false; + CHECK_EQ(R"(digraph graphname { +n1 [label="TableTypeVar 1"]; +n1 -> n2 [label="boundTo"]; +n2 [label="TableTypeVar a"]; +n2 -> n3 [label="x"]; +n3 [label="number"]; +})", + toDot(*ty, opts)); +} + +TEST_SUITE_END(); diff --git a/tests/Transpiler.test.cpp b/tests/Transpiler.test.cpp index 928c03a3..327fa0bb 100644 --- a/tests/Transpiler.test.cpp +++ b/tests/Transpiler.test.cpp @@ -445,9 +445,6 @@ local a: Import.Type TEST_CASE_FIXTURE(Fixture, "transpile_type_packs") { - ScopedFastFlag luauTypeAliasPacks("LuauTypeAliasPacks", true); - ScopedFastFlag luauParseTypePackTypeParameters("LuauParseTypePackTypeParameters", true); - std::string code = R"( type Packed = (T...)->(T...) local a: Packed<> diff --git a/tests/TypeInfer.aliases.test.cpp b/tests/TypeInfer.aliases.test.cpp index 74ce155c..822bd727 100644 --- a/tests/TypeInfer.aliases.test.cpp +++ b/tests/TypeInfer.aliases.test.cpp @@ -537,8 +537,6 @@ TEST_CASE_FIXTURE(Fixture, "free_variables_from_typeof_in_aliases") TEST_CASE_FIXTURE(Fixture, "non_recursive_aliases_that_reuse_a_generic_name") { - ScopedFastFlag sff1{"LuauSubstitutionDontReplaceIgnoredTypes", true}; - CheckResult result = check(R"( type Array = { [number]: T } type Tuple = Array diff --git a/tests/TypeInfer.generics.test.cpp b/tests/TypeInfer.generics.test.cpp index 88c2dc85..aba50891 100644 --- a/tests/TypeInfer.generics.test.cpp +++ b/tests/TypeInfer.generics.test.cpp @@ -609,8 +609,6 @@ TEST_CASE_FIXTURE(Fixture, "typefuns_sharing_types") TEST_CASE_FIXTURE(Fixture, "bound_tables_do_not_clone_original_fields") { - ScopedFastFlag luauCloneBoundTables{"LuauCloneBoundTables", true}; - CheckResult result = check(R"( local exports = {} local nested = {} @@ -627,4 +625,23 @@ return exports LUAU_REQUIRE_NO_ERRORS(result); } +TEST_CASE_FIXTURE(Fixture, "instantiated_function_argument_names") +{ + ScopedFastFlag luauFunctionArgumentNameSize{"LuauFunctionArgumentNameSize", true}; + + CheckResult result = check(R"( +local function f(a: T, ...: U...) end + +f(1, 2, 3) + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + auto ty = findTypeAtPosition(Position(3, 0)); + REQUIRE(ty); + ToStringOptions opts; + opts.functionTypeArguments = true; + CHECK_EQ(toString(*ty, opts), "(a: number, number, number) -> ()"); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.provisional.test.cpp b/tests/TypeInfer.provisional.test.cpp index e5c14dde..e6d3d4d4 100644 --- a/tests/TypeInfer.provisional.test.cpp +++ b/tests/TypeInfer.provisional.test.cpp @@ -31,8 +31,6 @@ TEST_SUITE_BEGIN("ProvisionalTests"); */ TEST_CASE_FIXTURE(Fixture, "typeguard_inference_incomplete") { - ScopedFastFlag luauTypeAliasPacks("LuauTypeAliasPacks", true); - const std::string code = R"( function f(a) if type(a) == "boolean" then diff --git a/tests/TypeInfer.tables.test.cpp b/tests/TypeInfer.tables.test.cpp index c3694be7..cb72faaf 100644 --- a/tests/TypeInfer.tables.test.cpp +++ b/tests/TypeInfer.tables.test.cpp @@ -2022,4 +2022,74 @@ caused by: Property '__call' is not compatible. Type '(a, b) -> ()' could not be converted into '(a) -> ()')"); } +TEST_CASE_FIXTURE(Fixture, "explicitly_typed_table") +{ + ScopedFastFlag sffs[] { + {"LuauPropertiesGetExpectedType", true}, + {"LuauExpectedTypesOfProperties", true}, + {"LuauTableSubtypingVariance", true}, + }; + + CheckResult result = check(R"( +--!strict +type Super = { x : number } +type Sub = { x : number, y: number } +type HasSuper = { p : Super } +type HasSub = { p : Sub } +local a: HasSuper = { p = { x = 5, y = 7 }} +a.p = { x = 9 } + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "explicitly_typed_table_error") +{ + ScopedFastFlag sffs[] { + {"LuauPropertiesGetExpectedType", true}, + {"LuauExpectedTypesOfProperties", true}, + {"LuauTableSubtypingVariance", true}, + {"LuauExtendedTypeMismatchError", true}, + }; + + CheckResult result = check(R"( +--!strict +type Super = { x : number } +type Sub = { x : number, y: number } +type HasSuper = { p : Super } +type HasSub = { p : Sub } +local tmp = { p = { x = 5, y = 7 }} +local a: HasSuper = tmp +a.p = { x = 9 } +-- needs to be an error because +local y: number = tmp.p.y + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ(toString(result.errors[0]), R"(Type 'tmp' could not be converted into 'HasSuper' +caused by: + Property 'p' is not compatible. Table type '{| x: number, y: number |}' not compatible with type 'Super' because the former has extra field 'y')"); +} + +TEST_CASE_FIXTURE(Fixture, "explicitly_typed_table_with_indexer") +{ + ScopedFastFlag sffs[] { + {"LuauPropertiesGetExpectedType", true}, + {"LuauExpectedTypesOfProperties", true}, + {"LuauTableSubtypingVariance", true}, + }; + + CheckResult result = check(R"( +--!strict +type Super = { x : number } +type Sub = { x : number, y: number } +type HasSuper = { [string] : Super } +type HasSub = { [string] : Sub } +local a: HasSuper = { p = { x = 5, y = 7 }} +a.p = { x = 9 } + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.test.cpp b/tests/TypeInfer.test.cpp index 99fd8339..e3222a41 100644 --- a/tests/TypeInfer.test.cpp +++ b/tests/TypeInfer.test.cpp @@ -4779,4 +4779,24 @@ local bar = foo.nutrition + 100 // CHECK_EQ("Unknown type used in + operation; consider adding a type annotation to 'foo'", toString(result.errors[1])); } +TEST_CASE_FIXTURE(Fixture, "require_failed_module") +{ + ScopedFastFlag luauModuleRequireErrorPack{"LuauModuleRequireErrorPack", true}; + + fileResolver.source["game/A"] = R"( +return unfortunately() + )"; + + CheckResult aResult = frontend.check("game/A"); + LUAU_REQUIRE_ERRORS(aResult); + + CheckResult result = check(R"( +local ModuleA = require(game.A) + )"); + LUAU_REQUIRE_NO_ERRORS(result); + + std::optional oty = requireType("ModuleA"); + CHECK_EQ("*unknown*", toString(*oty)); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.typePacks.cpp b/tests/TypeInfer.typePacks.cpp index c6de0abf..3f4420cd 100644 --- a/tests/TypeInfer.typePacks.cpp +++ b/tests/TypeInfer.typePacks.cpp @@ -296,9 +296,6 @@ end TEST_CASE_FIXTURE(Fixture, "type_alias_type_packs") { - ScopedFastFlag luauTypeAliasPacks("LuauTypeAliasPacks", true); - ScopedFastFlag luauParseTypePackTypeParameters("LuauParseTypePackTypeParameters", true); - CheckResult result = check(R"( type Packed = (T...) -> T... local a: Packed<> @@ -360,9 +357,6 @@ local c: Packed TEST_CASE_FIXTURE(Fixture, "type_alias_type_packs_import") { - ScopedFastFlag luauTypeAliasPacks("LuauTypeAliasPacks", true); - ScopedFastFlag luauParseTypePackTypeParameters("LuauParseTypePackTypeParameters", true); - fileResolver.source["game/A"] = R"( export type Packed = { a: T, b: (U...) -> () } return {} @@ -393,9 +387,6 @@ local d: { a: typeof(c) } TEST_CASE_FIXTURE(Fixture, "type_pack_type_parameters") { - ScopedFastFlag luauTypeAliasPacks("LuauTypeAliasPacks", true); - ScopedFastFlag luauParseTypePackTypeParameters("LuauParseTypePackTypeParameters", true); - fileResolver.source["game/A"] = R"( export type Packed = { a: T, b: (U...) -> () } return {} @@ -431,9 +422,6 @@ type C = Import.Packed TEST_CASE_FIXTURE(Fixture, "type_alias_type_packs_nested") { - ScopedFastFlag luauTypeAliasPacks("LuauTypeAliasPacks", true); - ScopedFastFlag luauParseTypePackTypeParameters("LuauParseTypePackTypeParameters", true); - CheckResult result = check(R"( type Packed1 = (T...) -> (T...) type Packed2 = (Packed1, T...) -> (Packed1, T...) @@ -452,9 +440,6 @@ type Packed4 = (Packed3, T...) -> (Packed3, T...) TEST_CASE_FIXTURE(Fixture, "type_alias_type_pack_variadic") { - ScopedFastFlag luauTypeAliasPacks("LuauTypeAliasPacks", true); - ScopedFastFlag luauParseTypePackTypeParameters("LuauParseTypePackTypeParameters", true); - CheckResult result = check(R"( type X = (T...) -> (string, T...) @@ -470,9 +455,6 @@ type E = X<(number, ...string)> TEST_CASE_FIXTURE(Fixture, "type_alias_type_pack_multi") { - ScopedFastFlag luauTypeAliasPacks("LuauTypeAliasPacks", true); - ScopedFastFlag luauParseTypePackTypeParameters("LuauParseTypePackTypeParameters", true); - CheckResult result = check(R"( type Y = (T...) -> (U...) type A = Y @@ -501,9 +483,6 @@ type I = W TEST_CASE_FIXTURE(Fixture, "type_alias_type_pack_explicit") { - ScopedFastFlag luauTypeAliasPacks("LuauTypeAliasPacks", true); - ScopedFastFlag luauParseTypePackTypeParameters("LuauParseTypePackTypeParameters", true); - CheckResult result = check(R"( type X = (T...) -> (T...) @@ -527,9 +506,6 @@ type F = X<(string, ...number)> TEST_CASE_FIXTURE(Fixture, "type_alias_type_pack_explicit_multi") { - ScopedFastFlag luauTypeAliasPacks("LuauTypeAliasPacks", true); - ScopedFastFlag luauParseTypePackTypeParameters("LuauParseTypePackTypeParameters", true); - CheckResult result = check(R"( type Y = (T...) -> (U...) @@ -549,9 +525,6 @@ type D = Y TEST_CASE_FIXTURE(Fixture, "type_alias_type_pack_explicit_multi_tostring") { - ScopedFastFlag luauTypeAliasPacks("LuauTypeAliasPacks", true); - ScopedFastFlag luauParseTypePackTypeParameters("LuauParseTypePackTypeParameters", true); - CheckResult result = check(R"( type Y = { f: (T...) -> (U...) } @@ -567,9 +540,6 @@ local b: Y<(), ()> TEST_CASE_FIXTURE(Fixture, "type_alias_backwards_compatible") { - ScopedFastFlag luauTypeAliasPacks("LuauTypeAliasPacks", true); - ScopedFastFlag luauParseTypePackTypeParameters("LuauParseTypePackTypeParameters", true); - CheckResult result = check(R"( type X = () -> T type Y = (T) -> U @@ -588,9 +558,6 @@ type C = Y<(number), boolean> TEST_CASE_FIXTURE(Fixture, "type_alias_type_packs_errors") { - ScopedFastFlag luauTypeAliasPacks("LuauTypeAliasPacks", true); - ScopedFastFlag luauParseTypePackTypeParameters("LuauParseTypePackTypeParameters", true); - CheckResult result = check(R"( type Packed = (T, U) -> (V...) local b: Packed diff --git a/tests/TypeVar.test.cpp b/tests/TypeVar.test.cpp index 91efa818..13db923e 100644 --- a/tests/TypeVar.test.cpp +++ b/tests/TypeVar.test.cpp @@ -3,6 +3,7 @@ #include "Luau/Scope.h" #include "Luau/TypeInfer.h" #include "Luau/TypeVar.h" +#include "Luau/VisitTypeVar.h" #include "Fixture.h" #include "ScopedFlags.h" @@ -323,4 +324,48 @@ TEST_CASE("tagging_props") CHECK(Luau::hasTag(prop, "foo")); } +struct VisitCountTracker +{ + std::unordered_map tyVisits; + std::unordered_map tpVisits; + + void cycle(TypeId) {} + void cycle(TypePackId) {} + + template + bool operator()(TypeId ty, const T& t) + { + tyVisits[ty]++; + return true; + } + + template + bool operator()(TypePackId tp, const T&) + { + tpVisits[tp]++; + return true; + } +}; + +TEST_CASE_FIXTURE(Fixture, "visit_once") +{ + CheckResult result = check(R"( +type T = { a: number, b: () -> () } +local b: (T, T, T) -> T +)"); + LUAU_REQUIRE_NO_ERRORS(result); + + TypeId bType = requireType("b"); + + VisitCountTracker tester; + DenseHashSet seen{nullptr}; + visitTypeVarOnce(bType, tester, seen); + + for (auto [_, count] : tester.tyVisits) + CHECK_EQ(count, 1); + + for (auto [_, count] : tester.tpVisits) + CHECK_EQ(count, 1); +} + TEST_SUITE_END(); diff --git a/tests/conformance/apicalls.lua b/tests/conformance/apicalls.lua index 5e03b055..7a4058b5 100644 --- a/tests/conformance/apicalls.lua +++ b/tests/conformance/apicalls.lua @@ -2,7 +2,13 @@ print('testing function calls through API') function add(a, b) - return a + b + return a + b +end + +local m = { __eq = function(a, b) return a.a == b.a end } + +function create_with_tm(x) + return setmetatable({ a = x }, m) end return('OK') diff --git a/tests/conformance/basic.lua b/tests/conformance/basic.lua index 693f68df..188b8ebc 100644 --- a/tests/conformance/basic.lua +++ b/tests/conformance/basic.lua @@ -441,7 +441,8 @@ assert((function() a = {} b = {} mt = { __eq = function(l, r) return #l == #r en assert((function() a = {} b = {} function eq(l, r) return #l == #r end setmetatable(a, {__eq = eq}) setmetatable(b, {__eq = eq}) return concat(a == b, a ~= b) end)() == "true,false") assert((function() a = {} b = {} setmetatable(a, {__eq = function(l, r) return #l == #r end}) setmetatable(b, {__eq = function(l, r) return #l == #r end}) return concat(a == b, a ~= b) end)() == "false,true") --- userdata, reference equality (no mt) +-- userdata, reference equality (no mt or mt.__eq) +assert((function() a = newproxy() return concat(a == newproxy(),a ~= newproxy()) end)() == "false,true") assert((function() a = newproxy(true) return concat(a == newproxy(true),a ~= newproxy(true)) end)() == "false,true") -- rawequal diff --git a/tools/svg.py b/tools/svg.py index 3b3bb28c..99853fb6 100644 --- a/tools/svg.py +++ b/tools/svg.py @@ -458,13 +458,16 @@ def display(root, title, colors, flip = False): framewidth = 1200 - 20 + def pixels(x): + return float(x) / root.width * framewidth if root.width > 0 else 0 + for n in root.subtree(): - if n.width / root.width * framewidth < 0.1: + if pixels(n.width) < 0.1: continue - x = 10 + n.offset / root.width * framewidth + x = 10 + pixels(n.offset) y = (maxdepth - 1 - n.depth if flip else n.depth) * 16 + 3 * 16 - width = n.width / root.width * framewidth + width = pixels(n.width) height = 15 if colors == "cold": From 7257f34d89fddfefeedc70cc2023a29851d6d517 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Thu, 2 Dec 2021 23:08:45 -0800 Subject: [PATCH 093/399] Update CONTRIBUTING.md The language around creating RFCs seems a bit redundant now. --- CONTRIBUTING.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 6eee61d3..460e4b43 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -26,10 +26,9 @@ If you're thinking of adding a new feature to the language, library, analysis to Luau team has internal priorities and a roadmap that may or may not align with specific features, so before starting to work on a feature please submit an issue describing the missing feature that you'd like to add. For features that result in observable change of language syntax or semantics, you'd need to [create an RFC](https://github.com/Roblox/luau/blob/master/rfcs/README.md) to make sure that the feature is needed and well designed. -Similarly to the above, please create an issue first so that we can see if we should go through with an RFC process. Finally, please note that Luau tries to carry a minimal feature set. All features must be evaluated not just for the benefits that they provide, but also for the downsides/costs in terms of language simplicity, maintainability, cross-feature interaction etc. -As such, feature requests may not be accepted, or may get to an RFC stage and get rejected there - don't expect Luau to gain a feature just because another programming language has it. +As such, feature requests may not be accepted even if a comprehensive RFC is written - don't expect Luau to gain a feature just because another programming language has it. ## Code style From f63ddae898efe6a75bd3df8e5799198a69c47bed Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Fri, 3 Dec 2021 10:40:31 -0800 Subject: [PATCH 094/399] Enable LuauActivateBeforeExec to activate the fix early (#275) Fixes #259. --- VM/src/ldo.cpp | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/VM/src/ldo.cpp b/VM/src/ldo.cpp index 62bbdb7c..43289ab4 100644 --- a/VM/src/ldo.cpp +++ b/VM/src/ldo.cpp @@ -19,7 +19,7 @@ LUAU_FASTFLAGVARIABLE(LuauCcallRestoreFix, false) LUAU_FASTFLAG(LuauCoroutineClose) -LUAU_FASTFLAGVARIABLE(LuauActivateBeforeExec, false) +LUAU_FASTFLAGVARIABLE(LuauActivateBeforeExec, true) /* ** {====================================================== From 9488f2379dce651c1dfb345b69ab97845520c64c Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Fri, 3 Dec 2021 11:10:41 -0800 Subject: [PATCH 095/399] Mark bit32.count* RFC as implemented --- rfcs/function-bit32-countlz-countrz.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/rfcs/function-bit32-countlz-countrz.md b/rfcs/function-bit32-countlz-countrz.md index 986da70c..d2439f72 100644 --- a/rfcs/function-bit32-countlz-countrz.md +++ b/rfcs/function-bit32-countlz-countrz.md @@ -4,6 +4,8 @@ Add bit32.countlz (count left zeroes) and bit32.countrz (count right zeroes) to accelerate bit scanning +**Status**: Implemented + ## Motivation All CPUs have instructions to determine the position of first/last set bit in an integer. These instructions have a variety of uses, the popular ones being: From 3eff1b214e8c81ae6021109aca2e2f1130fe15f1 Mon Sep 17 00:00:00 2001 From: Slappy826 Date: Tue, 7 Dec 2021 10:14:24 -0600 Subject: [PATCH 096/399] Fix spelling error of 'ROBLOX' (#277) * Remove roblox prefix on comment --- VM/src/lvmutils.cpp | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/VM/src/lvmutils.cpp b/VM/src/lvmutils.cpp index 740a4cfd..5d802277 100644 --- a/VM/src/lvmutils.cpp +++ b/VM/src/lvmutils.cpp @@ -53,7 +53,7 @@ const float* luaV_tovector(const TValue* obj) static void callTMres(lua_State* L, StkId res, const TValue* f, const TValue* p1, const TValue* p2) { ptrdiff_t result = savestack(L, res); - // RBOLOX: using stack room beyond top is technically safe here, but for very complicated reasons: + // using stack room beyond top is technically safe here, but for very complicated reasons: // * The stack guarantees 1 + EXTRA_STACK room beyond stack_last (see luaD_reallocstack) will be allocated // * we cannot move luaD_checkstack above because the arguments are *sometimes* pointers to the lua // stack and checkstack may invalidate those pointers @@ -74,7 +74,7 @@ static void callTMres(lua_State* L, StkId res, const TValue* f, const TValue* p1 static void callTM(lua_State* L, const TValue* f, const TValue* p1, const TValue* p2, const TValue* p3) { - // RBOLOX: using stack room beyond top is technically safe here, but for very complicated reasons: + // using stack room beyond top is technically safe here, but for very complicated reasons: // * The stack guarantees 1 + EXTRA_STACK room beyond stack_last (see luaD_reallocstack) will be allocated // * we cannot move luaD_checkstack above because the arguments are *sometimes* pointers to the lua // stack and checkstack may invalidate those pointers From 2807dcb7589205f13ee42de111a730ad4f836626 Mon Sep 17 00:00:00 2001 From: Braeden Date: Tue, 7 Dec 2021 10:14:35 -0600 Subject: [PATCH 097/399] Update JsonEncoder.cpp (#279) fix JsonEncoder so that AstArray elems are given commas seperating them --- Analysis/src/JsonEncoder.cpp | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Analysis/src/JsonEncoder.cpp b/Analysis/src/JsonEncoder.cpp index c7f623ee..af6bffc4 100644 --- a/Analysis/src/JsonEncoder.cpp +++ b/Analysis/src/JsonEncoder.cpp @@ -262,7 +262,7 @@ struct AstJsonEncoder : public AstVisitor if (comma) writeRaw(","); else - comma = false; + comma = true; write(a); } From 71adace16e260bc1b08e56e58d245eb4fa574c66 Mon Sep 17 00:00:00 2001 From: vegorov-rbx <75688451+vegorov-rbx@users.noreply.github.com> Date: Wed, 8 Dec 2021 04:21:47 -0800 Subject: [PATCH 098/399] RFC: Amend 'Default type alias type parameters' for type pack parameters (#238) * Do not allow regular type assignment to a type pack as a default parameter * With type pack support in type aliases, this second form with an empty list is now supported * Update rfcs/syntax-default-type-alias-type-parameters.md Co-authored-by: Alan Jeffrey <403333+asajeffrey@users.noreply.github.com> * Update syntax-default-type-alias-type-parameters.md Even more examples Co-authored-by: Alan Jeffrey <403333+asajeffrey@users.noreply.github.com> --- rfcs/syntax-default-type-alias-type-parameters.md | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) diff --git a/rfcs/syntax-default-type-alias-type-parameters.md b/rfcs/syntax-default-type-alias-type-parameters.md index 23bd71a8..15d23c81 100644 --- a/rfcs/syntax-default-type-alias-type-parameters.md +++ b/rfcs/syntax-default-type-alias-type-parameters.md @@ -48,9 +48,12 @@ type A = ... -- not allowed Default type parameter values are also allowed for type packs: ```lua -type A -- ok, variadic type pack -type C -- ok, type pack with one element -type D -- ok +type A -- ok, variadic type pack +type B -- ok, type pack with no elements +type C -- ok, type pack with one element +type D -- ok, type pack with two elements +type E -- ok, variadic type pack with a different first element +type F -- ok, same type pack as T... ``` --- @@ -68,7 +71,7 @@ If all type parameters have a default type value, it is now possible to referenc type All = ... local a: All -- ok -local b: All<> -- not allowed +local b: All<> -- ok as well ``` If type is exported from a module, default type parameter values will still be available when module is imported. From 2e6a2090c3a32a3a4daaa9e63ec265e84ae9c480 Mon Sep 17 00:00:00 2001 From: JohnnyMorganz Date: Wed, 8 Dec 2021 17:17:28 +0000 Subject: [PATCH 099/399] Add Grammar documentation (#266) Co-authored-by: Alexander McCord <11488393+alexmccord@users.noreply.github.com> Co-authored-by: Arseny Kapoulkine --- docs/_data/navigation.yml | 2 + docs/_pages/grammar.md | 82 +++++++++++++++++++++++++++++++++++++++ 2 files changed, 84 insertions(+) create mode 100644 docs/_pages/grammar.md diff --git a/docs/_data/navigation.yml b/docs/_data/navigation.yml index 83172d80..ba337ea5 100644 --- a/docs/_data/navigation.yml +++ b/docs/_data/navigation.yml @@ -29,3 +29,5 @@ pages: url: /profile - title: Library url: /library + - title: Grammar + url: /grammar diff --git a/docs/_pages/grammar.md b/docs/_pages/grammar.md new file mode 100644 index 00000000..74a5c8aa --- /dev/null +++ b/docs/_pages/grammar.md @@ -0,0 +1,82 @@ +--- +permalink: /grammar +title: Grammar +toc: true +--- + +This is the complete syntax grammar for Luau in EBNF. More information about the terminal nodes String and Number +is available in the [syntax section](syntax). + +> Note: this grammar is currently missing type pack syntax for generic arguments + +```ebnf +chunk ::= {stat [`;']} [laststat [`;']] +block ::= chunk +stat ::= varlist `=' explist | + var compoundop exp | + functioncall | + do block end | + while exp do block end | + repeat block until exp | + if exp then block {elseif exp then block} [else block] end | + for binding `=' exp `,' exp [`,' exp] do block end | + for bindinglist in explist do block end | + function funcname funcbody | + local function NAME funcbody | + local bindinglist [`=' explist] | + [export] type NAME [`<' GenericTypeList `>'] `=' Type + +laststat ::= return [explist] | break | continue + +funcname ::= NAME {`.' NAME} [`:' NAME] +funcbody ::= `(' [parlist] `)' [`:' ReturnType] block end +parlist ::= bindinglist [`,' `...'] | `...' + +explist ::= {exp `,'} exp +namelist ::= NAME {`,' NAME} + +binding ::= NAME [`:' TypeAnnotation] +bindinglist ::= binding [`,' bindinglist] (* equivalent of Lua 5.1 `namelist`, except with optional type annotations *) + +var ::= NAME | prefixexp `[' exp `]' | prefixexp `.' Name +varlist ::= var {`,' var} +prefixexp ::= var | functioncall | `(' exp `)' +functioncall ::= prefixexp funcargs | prefixexp `:' NAME funcargs + +exp ::= (asexp | unop exp) { binop exp } +ifelseexp ::= if exp then exp {elseif exp then exp} else exp +asexp ::= simpleexp [`::' Type] +simpleexp ::= NUMBER | STRING | nil | true | false | `...' | tableconstructor | function body | prefixexp | ifelseexp +funcargs ::= `(' [explist] `)' | tableconstructor | STRING + +tableconstructor ::= `{' [fieldlist] `}' +fieldlist ::= field {fieldsep field} [fieldsep] +field ::= `[' exp `]' `=' exp | NAME `=' exp | exp +fieldsep ::= `,' | `;' + +compoundop :: `+=' | `-=' | `*=' | `/=' | `%=' | `^=' | `..=' +binop ::= `+' | `-' | `*' | `/' | `^' | `%' | `..' | `<' | `<=' | `>' | `>=' | `==' | `~=' | and | or +unop ::= `-' | not | `#' + +SimpleType ::= + nil | + NAME[`.' NAME] [ `<' TypeList `>' ] | + `typeof' `(' exp `)' | + TableType | + FunctionType + +Type ::= + SimpleType [`?`] | + SimpleType [`|` Type] | + SimpleType [`&` Type] + +GenericTypeList ::= NAME [`...'] {`,' NAME [`...']} +TypeList ::= Type [`,' TypeList] | ...Type +ReturnType ::= Type | `(' TypeList `)' +TableIndexer ::= `[' Type `]' `:' Type +TableProp ::= NAME `:' Type +TablePropOrIndexer ::= TableProp | TableIndexer +PropList ::= TablePropOrIndexer {fieldsep TablePropOrIndexer} [fieldsep] +TableType ::= `{' PropList `}' +FunctionType ::= [`<' GenericTypeList `>'] `(' [TypeList] `)' `->` ReturnType +``` From 12ef94df5ecdf03fd7f08d43eaa46d56209e6d95 Mon Sep 17 00:00:00 2001 From: Rerumu <25379555+Rerumu@users.noreply.github.com> Date: Thu, 9 Dec 2021 15:37:48 -0500 Subject: [PATCH 100/399] Add shebang support (#149) Co-authored-by: Arseny Kapoulkine --- CLI/FileUtils.cpp | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/CLI/FileUtils.cpp b/CLI/FileUtils.cpp index b3c9557b..cb993dfe 100644 --- a/CLI/FileUtils.cpp +++ b/CLI/FileUtils.cpp @@ -67,6 +67,10 @@ std::optional readFile(const std::string& name) if (read != size_t(length)) return std::nullopt; + // Skip first line if it's a shebang + if (length > 2 && result[0] == '#' && result[1] == '!') + result.erase(0, result.find('\n')); + return result; } From 88be067c0b8a2774c296563a9a463111e259896c Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Thu, 9 Dec 2021 17:50:29 -0800 Subject: [PATCH 101/399] Switch coverage build to checkout@v2 (#285) Attempt to fix coverage builds by using checkout@v2 instead of v1 which might fix the detacthed HEAD issue. On the off chance it doesn't, add extra logging around git specifically. --- .github/workflows/build.yml | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml index 3782886c..b797e91b 100644 --- a/.github/workflows/build.yml +++ b/.github/workflows/build.yml @@ -66,22 +66,24 @@ jobs: coverage: runs-on: ubuntu-latest - env: - NODE_COVERALLS_DEBUG: 1 steps: - - uses: actions/checkout@v1 + - uses: actions/checkout@v2 - name: install run: | sudo apt install llvm - name: make coverage run: | CXX=clang++-10 make -j2 config=coverage coverage + - name: debug coverage + run: | + git status + git log -5 + echo SHA: $GITHUB_SHA - name: upload coverage uses: coverallsapp/github-action@master with: path-to-lcov: ./coverage.info github-token: ${{ secrets.GITHUB_TOKEN }} - continue-on-error: false - uses: actions/upload-artifact@v2 with: name: coverage From a8673f0f99885da92597d6efb4feaf0f14d59991 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Fri, 10 Dec 2021 13:17:10 -0800 Subject: [PATCH 102/399] Sync to upstream/release/507-pre This doesn't contain all changes for 507 yet but we might want to do the Luau 0.507 release a bit earlier to end the year sooner. --- Analysis/include/Luau/Error.h | 11 +- Analysis/include/Luau/IostreamHelpers.h | 4 + Analysis/include/Luau/ToString.h | 4 +- Analysis/include/Luau/TypeInfer.h | 7 +- Analysis/include/Luau/TypeVar.h | 8 +- Analysis/include/Luau/Unifiable.h | 11 +- Analysis/include/Luau/Unifier.h | 3 + Analysis/src/Autocomplete.cpp | 115 +++++++-- Analysis/src/BuiltinDefinitions.cpp | 9 +- Analysis/src/EmbeddedBuiltinDefinitions.cpp | 12 +- Analysis/src/Error.cpp | 17 +- Analysis/src/IostreamHelpers.cpp | 6 + Analysis/src/JsonEncoder.cpp | 4 +- Analysis/src/Module.cpp | 19 +- Analysis/src/Predicate.cpp | 2 +- Analysis/src/ToString.cpp | 38 ++- Analysis/src/TypeInfer.cpp | 191 +++++++++----- Analysis/src/TypeUtils.cpp | 6 +- Analysis/src/TypeVar.cpp | 220 ++++------------ Analysis/src/Unifier.cpp | 266 +++++++++++++++++--- Ast/src/Parser.cpp | 5 +- CLI/Analyze.cpp | 20 +- CLI/Coverage.cpp | 88 +++++++ CLI/Coverage.h | 10 + CLI/FileUtils.cpp | 4 + CLI/Profiler.cpp | 8 +- CLI/Profiler.h | 2 +- CLI/Repl.cpp | 35 ++- Compiler/src/Compiler.cpp | 22 +- Makefile | 6 +- Sources.cmake | 4 + VM/include/lua.h | 4 + VM/src/lapi.cpp | 162 ++++++------ VM/src/ldebug.cpp | 63 +++++ VM/src/lgc.cpp | 49 +--- VM/src/lgcdebug.cpp | 1 + VM/src/lobject.h | 10 +- VM/src/lstring.cpp | 29 --- VM/src/lstring.h | 7 - VM/src/ludata.cpp | 37 +++ VM/src/ludata.h | 13 + VM/src/lvmexecute.cpp | 14 +- VM/src/lvmutils.cpp | 4 +- bench/tests/sunspider/3d-raytrace.lua | 15 +- tests/Autocomplete.test.cpp | 55 +++- tests/Compiler.test.cpp | 49 +--- tests/Conformance.test.cpp | 100 ++++++-- tests/Module.test.cpp | 4 +- tests/Parser.test.cpp | 4 - tests/TypeInfer.annotations.test.cpp | 30 +++ tests/TypeInfer.builtins.test.cpp | 26 ++ tests/TypeInfer.generics.test.cpp | 40 +++ tests/TypeInfer.refinements.test.cpp | 17 +- tests/TypeInfer.singletons.test.cpp | 50 ++++ tests/TypeInfer.tables.test.cpp | 61 +++-- tests/TypeInfer.test.cpp | 239 +++++++++++++++++- tests/TypeInfer.tryUnify.test.cpp | 28 +++ tests/TypeInfer.unionTypes.test.cpp | 16 ++ tests/conformance/coverage.lua | 64 +++++ 59 files changed, 1704 insertions(+), 644 deletions(-) create mode 100644 CLI/Coverage.cpp create mode 100644 CLI/Coverage.h create mode 100644 VM/src/ludata.cpp create mode 100644 VM/src/ludata.h create mode 100644 tests/conformance/coverage.lua diff --git a/Analysis/include/Luau/Error.h b/Analysis/include/Luau/Error.h index 9ee75004..aff3c4d9 100644 --- a/Analysis/include/Luau/Error.h +++ b/Analysis/include/Luau/Error.h @@ -277,11 +277,20 @@ struct MissingUnionProperty bool operator==(const MissingUnionProperty& rhs) const; }; +struct TypesAreUnrelated +{ + TypeId left; + TypeId right; + + bool operator==(const TypesAreUnrelated& rhs) const; +}; + using TypeErrorData = Variant; + DuplicateGenericParameter, CannotInferBinaryOperation, MissingProperties, SwappedGenericTypeParameter, OptionalValueAccess, MissingUnionProperty, + TypesAreUnrelated>; struct TypeError { diff --git a/Analysis/include/Luau/IostreamHelpers.h b/Analysis/include/Luau/IostreamHelpers.h index f9e9cd48..ee994296 100644 --- a/Analysis/include/Luau/IostreamHelpers.h +++ b/Analysis/include/Luau/IostreamHelpers.h @@ -36,6 +36,10 @@ std::ostream& operator<<(std::ostream& lhs, const IllegalRequire& error); std::ostream& operator<<(std::ostream& lhs, const ModuleHasCyclicDependency& error); std::ostream& operator<<(std::ostream& lhs, const DuplicateGenericParameter& error); std::ostream& operator<<(std::ostream& lhs, const CannotInferBinaryOperation& error); +std::ostream& operator<<(std::ostream& lhs, const SwappedGenericTypeParameter& error); +std::ostream& operator<<(std::ostream& lhs, const OptionalValueAccess& error); +std::ostream& operator<<(std::ostream& lhs, const MissingUnionProperty& error); +std::ostream& operator<<(std::ostream& lhs, const TypesAreUnrelated& error); std::ostream& operator<<(std::ostream& lhs, const TableState& tv); std::ostream& operator<<(std::ostream& lhs, const TypeVar& tv); diff --git a/Analysis/include/Luau/ToString.h b/Analysis/include/Luau/ToString.h index 50379c1c..a97bf6d6 100644 --- a/Analysis/include/Luau/ToString.h +++ b/Analysis/include/Luau/ToString.h @@ -69,8 +69,8 @@ std::string toStringNamedFunction(const std::string& prefix, const FunctionTypeV // It could be useful to see the text representation of a type during a debugging session instead of exploring the content of the class // These functions will dump the type to stdout and can be evaluated in Watch/Immediate windows or as gdb/lldb expression -void dump(TypeId ty); -void dump(TypePackId ty); +std::string dump(TypeId ty); +std::string dump(TypePackId ty); std::string generateName(size_t n); diff --git a/Analysis/include/Luau/TypeInfer.h b/Analysis/include/Luau/TypeInfer.h index 9f553bc1..451976e4 100644 --- a/Analysis/include/Luau/TypeInfer.h +++ b/Analysis/include/Luau/TypeInfer.h @@ -156,13 +156,14 @@ struct TypeChecker // Returns both the type of the lvalue and its binding (if the caller wants to mutate the binding). // Note: the binding may be null. + // TODO: remove second return value with FFlagLuauUpdateFunctionNameBinding std::pair checkLValueBinding(const ScopePtr& scope, const AstExpr& expr); std::pair checkLValueBinding(const ScopePtr& scope, const AstExprLocal& expr); std::pair checkLValueBinding(const ScopePtr& scope, const AstExprGlobal& expr); std::pair checkLValueBinding(const ScopePtr& scope, const AstExprIndexName& expr); std::pair checkLValueBinding(const ScopePtr& scope, const AstExprIndexExpr& expr); - TypeId checkFunctionName(const ScopePtr& scope, AstExpr& funName); + TypeId checkFunctionName(const ScopePtr& scope, AstExpr& funName, TypeLevel level); std::pair checkFunctionSignature(const ScopePtr& scope, int subLevel, const AstExprFunction& expr, std::optional originalNameLoc, std::optional expectedType); void checkFunctionBody(const ScopePtr& scope, TypeId type, const AstExprFunction& function); @@ -174,7 +175,7 @@ struct TypeChecker ExprResult checkExprPack(const ScopePtr& scope, const AstExprCall& expr); std::vector> getExpectedTypesForCall(const std::vector& overloads, size_t argumentCount, bool selfCall); std::optional> checkCallOverload(const ScopePtr& scope, const AstExprCall& expr, TypeId fn, TypePackId retPack, - TypePackId argPack, TypePack* args, const std::vector& argLocations, const ExprResult& argListResult, + TypePackId argPack, TypePack* args, const std::vector* argLocations, const ExprResult& argListResult, std::vector& overloadsThatMatchArgCount, std::vector& overloadsThatDont, std::vector& errors); bool handleSelfCallMismatch(const ScopePtr& scope, const AstExprCall& expr, TypePack* args, const std::vector& argLocations, const std::vector& errors); @@ -277,7 +278,7 @@ public: [[noreturn]] void ice(const std::string& message); ScopePtr childFunctionScope(const ScopePtr& parent, const Location& location, int subLevel = 0); - ScopePtr childScope(const ScopePtr& parent, const Location& location, int subLevel = 0); + ScopePtr childScope(const ScopePtr& parent, const Location& location); // Wrapper for merge(l, r, toUnion) but without the lambda junk. void merge(RefinementMap& l, const RefinementMap& r); diff --git a/Analysis/include/Luau/TypeVar.h b/Analysis/include/Luau/TypeVar.h index 8c4c2f34..f6829ec3 100644 --- a/Analysis/include/Luau/TypeVar.h +++ b/Analysis/include/Luau/TypeVar.h @@ -499,6 +499,7 @@ struct SingletonTypes const TypePackId anyTypePack; SingletonTypes(); + ~SingletonTypes(); SingletonTypes(const SingletonTypes&) = delete; void operator=(const SingletonTypes&) = delete; @@ -509,10 +510,12 @@ struct SingletonTypes private: std::unique_ptr arena; + bool debugFreezeArena = false; + TypeId makeStringMetatable(); }; -extern SingletonTypes singletonTypes; +SingletonTypes& getSingletonTypes(); void persist(TypeId ty); void persist(TypePackId tp); @@ -523,9 +526,6 @@ TypeLevel* getMutableLevel(TypeId ty); const Property* lookupClassProp(const ClassTypeVar* cls, const Name& name); bool isSubclass(const ClassTypeVar* cls, const ClassTypeVar* parent); -bool hasGeneric(TypeId ty); -bool hasGeneric(TypePackId tp); - TypeVar* asMutable(TypeId ty); template diff --git a/Analysis/include/Luau/Unifiable.h b/Analysis/include/Luau/Unifiable.h index b47610fc..e8eafe68 100644 --- a/Analysis/include/Luau/Unifiable.h +++ b/Analysis/include/Luau/Unifiable.h @@ -24,7 +24,7 @@ struct TypeLevel int level = 0; int subLevel = 0; - // Returns true if the typelevel "this" is "bigger" than rhs + // Returns true if the level of "this" belongs to an equal or larger scope than that of rhs bool subsumes(const TypeLevel& rhs) const { if (level < rhs.level) @@ -38,6 +38,15 @@ struct TypeLevel return false; } + // Returns true if the level of "this" belongs to a larger (not equal) scope than that of rhs + bool subsumesStrict(const TypeLevel& rhs) const + { + if (level == rhs.level && subLevel == rhs.subLevel) + return false; + else + return subsumes(rhs); + } + TypeLevel incr() const { TypeLevel result; diff --git a/Analysis/include/Luau/Unifier.h b/Analysis/include/Luau/Unifier.h index 4588cdd8..7681b966 100644 --- a/Analysis/include/Luau/Unifier.h +++ b/Analysis/include/Luau/Unifier.h @@ -91,6 +91,9 @@ private: [[noreturn]] void ice(const std::string& message, const Location& location); [[noreturn]] void ice(const std::string& message); + + // Available after regular type pack unification errors + std::optional firstPackErrorPos; }; } // namespace Luau diff --git a/Analysis/src/Autocomplete.cpp b/Analysis/src/Autocomplete.cpp index db2d1d0e..4b583792 100644 --- a/Analysis/src/Autocomplete.cpp +++ b/Analysis/src/Autocomplete.cpp @@ -15,6 +15,7 @@ LUAU_FASTFLAG(LuauIfElseExpressionAnalysisSupport) LUAU_FASTFLAGVARIABLE(LuauAutocompleteAvoidMutation, false); LUAU_FASTFLAGVARIABLE(LuauAutocompletePreferToCallFunctions, false); +LUAU_FASTFLAGVARIABLE(LuauAutocompleteFirstArg, false); static const std::unordered_set kStatementStartingKeywords = { "while", "if", "local", "repeat", "function", "do", "for", "return", "break", "continue", "type", "export"}; @@ -190,7 +191,48 @@ static ParenthesesRecommendation getParenRecommendation(TypeId id, const std::ve return ParenthesesRecommendation::None; } -static TypeCorrectKind checkTypeCorrectKind(const Module& module, TypeArena* typeArena, AstNode* node, TypeId ty) +static std::optional findExpectedTypeAt(const Module& module, AstNode* node, Position position) +{ + LUAU_ASSERT(FFlag::LuauAutocompleteFirstArg); + + auto expr = node->asExpr(); + if (!expr) + return std::nullopt; + + // Extra care for first function call argument location + // When we don't have anything inside () yet, we also don't have an AST node to base our lookup + if (AstExprCall* exprCall = expr->as()) + { + if (exprCall->args.size == 0 && exprCall->argLocation.contains(position)) + { + auto it = module.astTypes.find(exprCall->func); + + if (!it) + return std::nullopt; + + const FunctionTypeVar* ftv = get(follow(*it)); + + if (!ftv) + return std::nullopt; + + auto [head, tail] = flatten(ftv->argTypes); + unsigned index = exprCall->self ? 1 : 0; + + if (index < head.size()) + return head[index]; + + return std::nullopt; + } + } + + auto it = module.astExpectedTypes.find(expr); + if (!it) + return std::nullopt; + + return *it; +} + +static TypeCorrectKind checkTypeCorrectKind(const Module& module, TypeArena* typeArena, AstNode* node, Position position, TypeId ty) { ty = follow(ty); @@ -220,15 +262,29 @@ static TypeCorrectKind checkTypeCorrectKind(const Module& module, TypeArena* typ } }; - auto expr = node->asExpr(); - if (!expr) - return TypeCorrectKind::None; + TypeId expectedType; - auto it = module.astExpectedTypes.find(expr); - if (!it) - return TypeCorrectKind::None; + if (FFlag::LuauAutocompleteFirstArg) + { + auto typeAtPosition = findExpectedTypeAt(module, node, position); - TypeId expectedType = follow(*it); + if (!typeAtPosition) + return TypeCorrectKind::None; + + expectedType = follow(*typeAtPosition); + } + else + { + auto expr = node->asExpr(); + if (!expr) + return TypeCorrectKind::None; + + auto it = module.astExpectedTypes.find(expr); + if (!it) + return TypeCorrectKind::None; + + expectedType = follow(*it); + } if (FFlag::LuauAutocompletePreferToCallFunctions) { @@ -333,8 +389,8 @@ static void autocompleteProps(const Module& module, TypeArena* typeArena, TypeId if (result.count(name) == 0 && name != Parser::errorName) { Luau::TypeId type = Luau::follow(prop.type); - TypeCorrectKind typeCorrect = - indexType == PropIndexType::Key ? TypeCorrectKind::Correct : checkTypeCorrectKind(module, typeArena, nodes.back(), type); + TypeCorrectKind typeCorrect = indexType == PropIndexType::Key ? TypeCorrectKind::Correct + : checkTypeCorrectKind(module, typeArena, nodes.back(), {{}, {}}, type); ParenthesesRecommendation parens = indexType == PropIndexType::Key ? ParenthesesRecommendation::None : getParenRecommendation(type, nodes, typeCorrect); @@ -692,17 +748,31 @@ std::optional returnFirstNonnullOptionOfType(const UnionTypeVar* utv) return ret; } -static std::optional functionIsExpectedAt(const Module& module, AstNode* node) +static std::optional functionIsExpectedAt(const Module& module, AstNode* node, Position position) { - auto expr = node->asExpr(); - if (!expr) - return std::nullopt; + TypeId expectedType; - auto it = module.astExpectedTypes.find(expr); - if (!it) - return std::nullopt; + if (FFlag::LuauAutocompleteFirstArg) + { + auto typeAtPosition = findExpectedTypeAt(module, node, position); - TypeId expectedType = follow(*it); + if (!typeAtPosition) + return std::nullopt; + + expectedType = follow(*typeAtPosition); + } + else + { + auto expr = node->asExpr(); + if (!expr) + return std::nullopt; + + auto it = module.astExpectedTypes.find(expr); + if (!it) + return std::nullopt; + + expectedType = follow(*it); + } if (get(expectedType)) return true; @@ -1171,7 +1241,7 @@ static void autocompleteExpression(const SourceModule& sourceModule, const Modul std::string n = toString(name); if (!result.count(n)) { - TypeCorrectKind typeCorrect = checkTypeCorrectKind(module, typeArena, node, binding.typeId); + TypeCorrectKind typeCorrect = checkTypeCorrectKind(module, typeArena, node, position, binding.typeId); result[n] = {AutocompleteEntryKind::Binding, binding.typeId, binding.deprecated, false, typeCorrect, std::nullopt, std::nullopt, binding.documentationSymbol, {}, getParenRecommendation(binding.typeId, ancestry, typeCorrect)}; @@ -1181,9 +1251,10 @@ static void autocompleteExpression(const SourceModule& sourceModule, const Modul scope = scope->parent; } - TypeCorrectKind correctForNil = checkTypeCorrectKind(module, typeArena, node, typeChecker.nilType); - TypeCorrectKind correctForBoolean = checkTypeCorrectKind(module, typeArena, node, typeChecker.booleanType); - TypeCorrectKind correctForFunction = functionIsExpectedAt(module, node).value_or(false) ? TypeCorrectKind::Correct : TypeCorrectKind::None; + TypeCorrectKind correctForNil = checkTypeCorrectKind(module, typeArena, node, position, typeChecker.nilType); + TypeCorrectKind correctForBoolean = checkTypeCorrectKind(module, typeArena, node, position, typeChecker.booleanType); + TypeCorrectKind correctForFunction = + functionIsExpectedAt(module, node, position).value_or(false) ? TypeCorrectKind::Correct : TypeCorrectKind::None; if (FFlag::LuauIfElseExpressionAnalysisSupport) result["if"] = {AutocompleteEntryKind::Keyword, std::nullopt, false, false}; diff --git a/Analysis/src/BuiltinDefinitions.cpp b/Analysis/src/BuiltinDefinitions.cpp index bac94a2b..d527414a 100644 --- a/Analysis/src/BuiltinDefinitions.cpp +++ b/Analysis/src/BuiltinDefinitions.cpp @@ -217,9 +217,9 @@ void registerBuiltinTypes(TypeChecker& typeChecker) TypeId genericK = arena.addType(GenericTypeVar{"K"}); TypeId genericV = arena.addType(GenericTypeVar{"V"}); - TypeId mapOfKtoV = arena.addType(TableTypeVar{{}, TableIndexer(genericK, genericV), typeChecker.globalScope->level}); + TypeId mapOfKtoV = arena.addType(TableTypeVar{{}, TableIndexer(genericK, genericV), typeChecker.globalScope->level, TableState::Generic}); - std::optional stringMetatableTy = getMetatable(singletonTypes.stringType); + std::optional stringMetatableTy = getMetatable(getSingletonTypes().stringType); LUAU_ASSERT(stringMetatableTy); const TableTypeVar* stringMetatableTable = get(follow(*stringMetatableTy)); LUAU_ASSERT(stringMetatableTable); @@ -271,7 +271,10 @@ void registerBuiltinTypes(TypeChecker& typeChecker) persist(pair.second.typeId); if (TableTypeVar* ttv = getMutable(pair.second.typeId)) - ttv->name = toString(pair.first); + { + if (!ttv->name) + ttv->name = toString(pair.first); + } } attachMagicFunction(getGlobalBinding(typeChecker, "assert"), magicFunctionAssert); diff --git a/Analysis/src/EmbeddedBuiltinDefinitions.cpp b/Analysis/src/EmbeddedBuiltinDefinitions.cpp index 9f5c8250..d0afa742 100644 --- a/Analysis/src/EmbeddedBuiltinDefinitions.cpp +++ b/Analysis/src/EmbeddedBuiltinDefinitions.cpp @@ -1,6 +1,8 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #include "Luau/BuiltinDefinitions.h" +LUAU_FASTFLAGVARIABLE(LuauFixTonumberReturnType, false) + namespace Luau { @@ -113,7 +115,6 @@ declare function gcinfo(): number declare function error(message: T, level: number?) declare function tostring(value: T): string - declare function tonumber(value: T, radix: number?): number declare function rawequal(a: T1, b: T2): boolean declare function rawget(tab: {[K]: V}, k: K): V @@ -204,7 +205,14 @@ declare function gcinfo(): number std::string getBuiltinDefinitionSource() { - return kBuiltinDefinitionLuaSrc; + std::string result = kBuiltinDefinitionLuaSrc; + + if (FFlag::LuauFixTonumberReturnType) + result += "declare function tonumber(value: T, radix: number?): number?\n"; + else + result += "declare function tonumber(value: T, radix: number?): number\n"; + + return result; } } // namespace Luau diff --git a/Analysis/src/Error.cpp b/Analysis/src/Error.cpp index 8334bd62..ce832c6b 100644 --- a/Analysis/src/Error.cpp +++ b/Analysis/src/Error.cpp @@ -58,7 +58,7 @@ struct ErrorConverter result += "\ncaused by:\n "; if (!tm.reason.empty()) - result += tm.reason + ". "; + result += tm.reason + " "; result += Luau::toString(*tm.error); } @@ -410,6 +410,11 @@ struct ErrorConverter return ss + " in the type '" + toString(e.type) + "'"; } + + std::string operator()(const TypesAreUnrelated& e) const + { + return "Cannot cast '" + toString(e.left) + "' into '" + toString(e.right) + "' because the types are unrelated"; + } }; struct InvalidNameChecker @@ -658,6 +663,11 @@ bool MissingUnionProperty::operator==(const MissingUnionProperty& rhs) const return *type == *rhs.type && key == rhs.key; } +bool TypesAreUnrelated::operator==(const TypesAreUnrelated& rhs) const +{ + return left == rhs.left && right == rhs.right; +} + std::string toString(const TypeError& error) { ErrorConverter converter; @@ -793,6 +803,11 @@ void copyError(T& e, TypeArena& destArena, SeenTypes& seenTypes, SeenTypePacks& for (auto& ty : e.missing) ty = clone(ty); } + else if constexpr (std::is_same_v) + { + e.left = clone(e.left); + e.right = clone(e.right); + } else static_assert(always_false_v, "Non-exhaustive type switch"); } diff --git a/Analysis/src/IostreamHelpers.cpp b/Analysis/src/IostreamHelpers.cpp index ac46b5a4..5bc76ade 100644 --- a/Analysis/src/IostreamHelpers.cpp +++ b/Analysis/src/IostreamHelpers.cpp @@ -262,6 +262,12 @@ std::ostream& operator<<(std::ostream& stream, const MissingUnionProperty& error return stream << " }, key = '" + error.key + "' }"; } +std::ostream& operator<<(std::ostream& stream, const TypesAreUnrelated& error) +{ + stream << "TypesAreUnrelated { left = '" + toString(error.left) + "', right = '" + toString(error.right) + "' }"; + return stream; +} + std::ostream& operator<<(std::ostream& stream, const TableState& tv) { return stream << static_cast::type>(tv); diff --git a/Analysis/src/JsonEncoder.cpp b/Analysis/src/JsonEncoder.cpp index c7f623ee..23491a5a 100644 --- a/Analysis/src/JsonEncoder.cpp +++ b/Analysis/src/JsonEncoder.cpp @@ -262,7 +262,7 @@ struct AstJsonEncoder : public AstVisitor if (comma) writeRaw(","); else - comma = false; + comma = true; write(a); } @@ -379,7 +379,7 @@ struct AstJsonEncoder : public AstVisitor if (comma) writeRaw(","); else - comma = false; + comma = true; write(prop); } }); diff --git a/Analysis/src/Module.cpp b/Analysis/src/Module.cpp index b4b6eb42..e1e53c97 100644 --- a/Analysis/src/Module.cpp +++ b/Analysis/src/Module.cpp @@ -13,7 +13,6 @@ LUAU_FASTFLAGVARIABLE(DebugLuauFreezeArena, false) LUAU_FASTFLAGVARIABLE(DebugLuauTrackOwningArena, false) -LUAU_FASTFLAG(LuauCaptureBrokenCommentSpans) LUAU_FASTINTVARIABLE(LuauTypeCloneRecursionLimit, 0) namespace Luau @@ -23,7 +22,7 @@ static bool contains(Position pos, Comment comment) { if (comment.location.contains(pos)) return true; - else if (FFlag::LuauCaptureBrokenCommentSpans && comment.type == Lexeme::BrokenComment && + else if (comment.type == Lexeme::BrokenComment && comment.location.begin <= pos) // Broken comments are broken specifically because they don't have an end return true; else if (comment.type == Lexeme::Comment && comment.location.end == pos) @@ -194,7 +193,7 @@ struct TypePackCloner { cloneState.encounteredFreeType = true; - TypePackId err = singletonTypes.errorRecoveryTypePack(singletonTypes.anyTypePack); + TypePackId err = getSingletonTypes().errorRecoveryTypePack(getSingletonTypes().anyTypePack); TypePackId cloned = dest.addTypePack(*err); seenTypePacks[typePackId] = cloned; } @@ -247,7 +246,7 @@ void TypeCloner::defaultClone(const T& t) void TypeCloner::operator()(const Unifiable::Free& t) { cloneState.encounteredFreeType = true; - TypeId err = singletonTypes.errorRecoveryType(singletonTypes.anyType); + TypeId err = getSingletonTypes().errorRecoveryType(getSingletonTypes().anyType); TypeId cloned = dest.addType(*err); seenTypes[typeId] = cloned; } @@ -421,9 +420,6 @@ TypePackId clone(TypePackId tp, TypeArena& dest, SeenTypes& seenTypes, SeenTypeP Luau::visit(cloner, tp->ty); // Mutates the storage that 'res' points into. } - if (FFlag::DebugLuauTrackOwningArena) - asMutable(res)->owningArena = &dest; - return res; } @@ -440,12 +436,11 @@ TypeId clone(TypeId typeId, TypeArena& dest, SeenTypes& seenTypes, SeenTypePacks { TypeCloner cloner{dest, typeId, seenTypes, seenTypePacks, cloneState}; Luau::visit(cloner, typeId->ty); // Mutates the storage that 'res' points into. + + // TODO: Make this work when the arena of 'res' might be frozen asMutable(res)->documentationSymbol = typeId->documentationSymbol; } - if (FFlag::DebugLuauTrackOwningArena) - asMutable(res)->owningArena = &dest; - return res; } @@ -508,8 +503,8 @@ bool Module::clonePublicInterface() if (moduleScope->varargPack) moduleScope->varargPack = clone(*moduleScope->varargPack, interfaceTypes, seenTypes, seenTypePacks, cloneState); - for (auto& pair : moduleScope->exportedTypeBindings) - pair.second = clone(pair.second, interfaceTypes, seenTypes, seenTypePacks, cloneState); + for (auto& [name, tf] : moduleScope->exportedTypeBindings) + tf = clone(tf, interfaceTypes, seenTypes, seenTypePacks, cloneState); for (TypeId ty : moduleScope->returnType) if (get(follow(ty))) diff --git a/Analysis/src/Predicate.cpp b/Analysis/src/Predicate.cpp index 848627cf..7bd8001e 100644 --- a/Analysis/src/Predicate.cpp +++ b/Analysis/src/Predicate.cpp @@ -24,7 +24,7 @@ std::optional tryGetLValue(const AstExpr& node) else if (auto indexexpr = expr->as()) { if (auto lvalue = tryGetLValue(*indexexpr->expr)) - if (auto string = indexexpr->expr->as()) + if (auto string = indexexpr->index->as()) return Field{std::make_shared(*lvalue), std::string(string->value.data, string->value.size)}; } diff --git a/Analysis/src/ToString.cpp b/Analysis/src/ToString.cpp index 6322096c..a6be5348 100644 --- a/Analysis/src/ToString.cpp +++ b/Analysis/src/ToString.cpp @@ -13,6 +13,13 @@ LUAU_FASTFLAG(LuauOccursCheckOkWithRecursiveFunctions) LUAU_FASTFLAGVARIABLE(LuauFunctionArgumentNameSize, false) +/* + * Prefix generic typenames with gen- + * Additionally, free types will be prefixed with free- and suffixed with their level. eg free-a-4 + * Fair warning: Setting this will break a lot of Luau unit tests. + */ +LUAU_FASTFLAGVARIABLE(DebugLuauVerboseTypeNames, false) + namespace Luau { @@ -290,7 +297,15 @@ struct TypeVarStringifier void operator()(TypeId ty, const Unifiable::Free& ftv) { state.result.invalid = true; + if (FFlag::DebugLuauVerboseTypeNames) + state.emit("free-"); state.emit(state.getName(ty)); + + if (FFlag::DebugLuauVerboseTypeNames) + { + state.emit("-"); + state.emit(std::to_string(ftv.level.level)); + } } void operator()(TypeId, const BoundTypeVar& btv) @@ -802,6 +817,8 @@ struct TypePackStringifier void operator()(TypePackId tp, const GenericTypePack& pack) { + if (FFlag::DebugLuauVerboseTypeNames) + state.emit("gen-"); if (pack.explicitName) { state.result.nameMap.typePacks[tp] = pack.name; @@ -817,7 +834,16 @@ struct TypePackStringifier void operator()(TypePackId tp, const FreeTypePack& pack) { state.result.invalid = true; + if (FFlag::DebugLuauVerboseTypeNames) + state.emit("free-"); state.emit(state.getName(tp)); + + if (FFlag::DebugLuauVerboseTypeNames) + { + state.emit("-"); + state.emit(std::to_string(pack.level.level)); + } + state.emit("..."); } @@ -1181,20 +1207,24 @@ std::string toStringNamedFunction(const std::string& prefix, const FunctionTypeV return s; } -void dump(TypeId ty) +std::string dump(TypeId ty) { ToStringOptions opts; opts.exhaustive = true; opts.functionTypeArguments = true; - printf("%s\n", toString(ty, opts).c_str()); + std::string s = toString(ty, opts); + printf("%s\n", s.c_str()); + return s; } -void dump(TypePackId ty) +std::string dump(TypePackId ty) { ToStringOptions opts; opts.exhaustive = true; opts.functionTypeArguments = true; - printf("%s\n", toString(ty, opts).c_str()); + std::string s = toString(ty, opts); + printf("%s\n", s.c_str()); + return s; } std::string generateName(size_t i) diff --git a/Analysis/src/TypeInfer.cpp b/Analysis/src/TypeInfer.cpp index 617bf482..abbc2901 100644 --- a/Analysis/src/TypeInfer.cpp +++ b/Analysis/src/TypeInfer.cpp @@ -9,9 +9,9 @@ #include "Luau/Scope.h" #include "Luau/Substitution.h" #include "Luau/TopoSortStatements.h" -#include "Luau/ToString.h" #include "Luau/TypePack.h" #include "Luau/TypeUtils.h" +#include "Luau/ToString.h" #include "Luau/TypeVar.h" #include "Luau/TimeTrace.h" @@ -29,7 +29,6 @@ LUAU_FASTFLAGVARIABLE(LuauCloneCorrectlyBeforeMutatingTableType, false) LUAU_FASTFLAGVARIABLE(LuauStoreMatchingOverloadFnType, false) LUAU_FASTFLAGVARIABLE(LuauRecursiveTypeParameterRestriction, false) LUAU_FASTFLAGVARIABLE(LuauIfElseExpressionAnalysisSupport, false) -LUAU_FASTFLAGVARIABLE(LuauStrictRequire, false) LUAU_FASTFLAGVARIABLE(LuauQuantifyInPlace2, false) LUAU_FASTFLAGVARIABLE(LuauSingletonTypes, false) LUAU_FASTFLAGVARIABLE(LuauExpectedTypesOfProperties, false) @@ -37,6 +36,12 @@ LUAU_FASTFLAGVARIABLE(LuauErrorRecoveryType, false) LUAU_FASTFLAGVARIABLE(LuauPropertiesGetExpectedType, false) LUAU_FASTFLAGVARIABLE(LuauTailArgumentTypeInfo, false) LUAU_FASTFLAGVARIABLE(LuauModuleRequireErrorPack, false) +LUAU_FASTFLAGVARIABLE(LuauRefiLookupFromIndexExpr, false) +LUAU_FASTFLAGVARIABLE(LuauProperTypeLevels, false) +LUAU_FASTFLAGVARIABLE(LuauAscribeCorrectLevelToInferredProperitesOfFreeTables, false) +LUAU_FASTFLAGVARIABLE(LuauFixRecursiveMetatableCall, false) +LUAU_FASTFLAGVARIABLE(LuauBidirectionalAsExpr, false) +LUAU_FASTFLAGVARIABLE(LuauUpdateFunctionNameBinding, false) namespace Luau { @@ -206,14 +211,14 @@ TypeChecker::TypeChecker(ModuleResolver* resolver, InternalErrorReporter* iceHan : resolver(resolver) , iceHandler(iceHandler) , unifierState(iceHandler) - , nilType(singletonTypes.nilType) - , numberType(singletonTypes.numberType) - , stringType(singletonTypes.stringType) - , booleanType(singletonTypes.booleanType) - , threadType(singletonTypes.threadType) - , anyType(singletonTypes.anyType) - , optionalNumberType(singletonTypes.optionalNumberType) - , anyTypePack(singletonTypes.anyTypePack) + , nilType(getSingletonTypes().nilType) + , numberType(getSingletonTypes().numberType) + , stringType(getSingletonTypes().stringType) + , booleanType(getSingletonTypes().booleanType) + , threadType(getSingletonTypes().threadType) + , anyType(getSingletonTypes().anyType) + , optionalNumberType(getSingletonTypes().optionalNumberType) + , anyTypePack(getSingletonTypes().anyTypePack) { globalScope = std::make_shared(globalTypes.addTypePack(TypePackVar{FreeTypePack{TypeLevel{}}})); @@ -443,7 +448,7 @@ void TypeChecker::checkBlock(const ScopePtr& scope, const AstStatBlock& block) functionDecls[*protoIter] = pair; ++subLevel; - TypeId leftType = checkFunctionName(scope, *fun->name); + TypeId leftType = checkFunctionName(scope, *fun->name, funScope->level); unify(leftType, funTy, fun->location); } else if (auto fun = (*protoIter)->as()) @@ -711,14 +716,15 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatAssign& assign) } else if (auto tail = valueIter.tail()) { - if (get(*tail)) + TypePackId tailPack = follow(*tail); + if (get(tailPack)) right = errorRecoveryType(scope); - else if (auto vtp = get(*tail)) + else if (auto vtp = get(tailPack)) right = vtp->ty; - else if (get(*tail)) + else if (get(tailPack)) { - *asMutable(*tail) = TypePack{{left}}; - growingPack = getMutable(*tail); + *asMutable(tailPack) = TypePack{{left}}; + growingPack = getMutable(tailPack); } } @@ -1107,8 +1113,27 @@ void TypeChecker::check(const ScopePtr& scope, TypeId ty, const ScopePtr& funSco unify(leftType, ty, function.location); - if (leftTypeBinding) - *leftTypeBinding = follow(quantify(funScope, leftType, function.name->location)); + if (FFlag::LuauUpdateFunctionNameBinding) + { + LUAU_ASSERT(function.name->is() || function.name->is()); + + if (auto exprIndexName = function.name->as()) + { + if (auto typeIt = currentModule->astTypes.find(exprIndexName->expr)) + { + if (auto ttv = getMutableTableType(*typeIt)) + { + if (auto it = ttv->props.find(exprIndexName->index.value); it != ttv->props.end()) + it->second.type = follow(quantify(funScope, leftType, function.name->location)); + } + } + } + } + else + { + if (leftTypeBinding) + *leftTypeBinding = follow(quantify(funScope, leftType, function.name->location)); + } } } @@ -1148,8 +1173,10 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatTypeAlias& typealias } else { - ScopePtr aliasScope = - FFlag::LuauQuantifyInPlace2 ? childScope(scope, typealias.location, subLevel) : childScope(scope, typealias.location); + ScopePtr aliasScope = childScope(scope, typealias.location); + aliasScope->level = scope->level.incr(); + if (FFlag::LuauProperTypeLevels) + aliasScope->level.subLevel = subLevel; auto [generics, genericPacks] = createGenericTypes(aliasScope, scope->level, typealias, typealias.generics, typealias.genericPacks); @@ -1166,6 +1193,7 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatTypeAlias& typealias ice("Not predeclared"); ScopePtr aliasScope = childScope(scope, typealias.location); + aliasScope->level = scope->level.incr(); for (TypeId ty : binding->typeParams) { @@ -1505,9 +1533,9 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprCa else if (auto vtp = get(retPack)) return {vtp->ty, std::move(result.predicates)}; else if (get(retPack)) - ice("Unexpected abstract type pack!"); + ice("Unexpected abstract type pack!", expr.location); else - ice("Unknown TypePack type!"); + ice("Unknown TypePack type!", expr.location); } ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprIndexName& expr) @@ -1574,7 +1602,7 @@ std::optional TypeChecker::getIndexTypeFromType( } else if (tableType->state == TableState::Free) { - TypeId result = freshType(scope); + TypeId result = FFlag::LuauAscribeCorrectLevelToInferredProperitesOfFreeTables ? freshType(tableType->level) : freshType(scope); tableType->props[name] = {result}; return result; } @@ -1738,7 +1766,16 @@ TypeId TypeChecker::stripFromNilAndReport(TypeId ty, const Location& location) ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprIndexExpr& expr) { - return {checkLValue(scope, expr)}; + TypeId ty = checkLValue(scope, expr); + + if (FFlag::LuauRefiLookupFromIndexExpr) + { + if (std::optional lvalue = tryGetLValue(expr)) + if (std::optional refiTy = resolveLValue(scope, *lvalue)) + return {*refiTy, {TruthyPredicate{std::move(*lvalue), expr.location}}}; + } + + return {ty}; } ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprFunction& expr, std::optional expectedType) @@ -2421,12 +2458,27 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprTy TypeId annotationType = resolveType(scope, *expr.annotation); ExprResult result = checkExpr(scope, *expr.expr, annotationType); - ErrorVec errorVec = canUnify(result.type, annotationType, expr.location); - reportErrors(errorVec); - if (!errorVec.empty()) - annotationType = errorRecoveryType(annotationType); + if (FFlag::LuauBidirectionalAsExpr) + { + // Note: As an optimization, we try 'number <: number | string' first, as that is the more likely case. + if (canUnify(result.type, annotationType, expr.location).empty()) + return {annotationType, std::move(result.predicates)}; - return {annotationType, std::move(result.predicates)}; + if (canUnify(annotationType, result.type, expr.location).empty()) + return {annotationType, std::move(result.predicates)}; + + reportError(expr.location, TypesAreUnrelated{result.type, annotationType}); + return {errorRecoveryType(annotationType), std::move(result.predicates)}; + } + else + { + ErrorVec errorVec = canUnify(result.type, annotationType, expr.location); + reportErrors(errorVec); + if (!errorVec.empty()) + annotationType = errorRecoveryType(annotationType); + + return {annotationType, std::move(result.predicates)}; + } } ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprError& expr) @@ -2674,8 +2726,15 @@ std::pair TypeChecker::checkLValueBinding(const ScopePtr& scope // Answers the question: "Can I define another function with this name?" // Primarily about detecting duplicates. -TypeId TypeChecker::checkFunctionName(const ScopePtr& scope, AstExpr& funName) +TypeId TypeChecker::checkFunctionName(const ScopePtr& scope, AstExpr& funName, TypeLevel level) { + auto freshTy = [&]() { + if (FFlag::LuauProperTypeLevels) + return freshType(level); + else + return freshType(scope); + }; + if (auto globalName = funName.as()) { const ScopePtr& globalScope = currentModule->getModuleScope(); @@ -2689,7 +2748,7 @@ TypeId TypeChecker::checkFunctionName(const ScopePtr& scope, AstExpr& funName) } else { - TypeId ty = freshType(scope); + TypeId ty = freshTy(); globalScope->bindings[name] = {ty, funName.location}; return ty; } @@ -2699,7 +2758,7 @@ TypeId TypeChecker::checkFunctionName(const ScopePtr& scope, AstExpr& funName) Symbol name = localName->local; Binding& binding = scope->bindings[name]; if (binding.typeId == nullptr) - binding = {freshType(scope), funName.location}; + binding = {freshTy(), funName.location}; return binding.typeId; } @@ -2730,7 +2789,7 @@ TypeId TypeChecker::checkFunctionName(const ScopePtr& scope, AstExpr& funName) Property& property = ttv->props[name]; - property.type = freshType(scope); + property.type = freshTy(); property.location = indexName->indexLocation; ttv->methodDefinitionLocations[name] = funName.location; return property.type; @@ -3327,7 +3386,7 @@ ExprResult TypeChecker::checkExprPack(const ScopePtr& scope, const A fn = follow(fn); if (auto ret = checkCallOverload( - scope, expr, fn, retPack, argPack, args, argLocations, argListResult, overloadsThatMatchArgCount, overloadsThatDont, errors)) + scope, expr, fn, retPack, argPack, args, &argLocations, argListResult, overloadsThatMatchArgCount, overloadsThatDont, errors)) return *ret; } @@ -3402,9 +3461,11 @@ std::vector> TypeChecker::getExpectedTypesForCall(const st } std::optional> TypeChecker::checkCallOverload(const ScopePtr& scope, const AstExprCall& expr, TypeId fn, TypePackId retPack, - TypePackId argPack, TypePack* args, const std::vector& argLocations, const ExprResult& argListResult, + TypePackId argPack, TypePack* args, const std::vector* argLocations, const ExprResult& argListResult, std::vector& overloadsThatMatchArgCount, std::vector& overloadsThatDont, std::vector& errors) { + LUAU_ASSERT(argLocations); + fn = stripFromNilAndReport(fn, expr.func->location); if (get(fn)) @@ -3428,31 +3489,44 @@ std::optional> TypeChecker::checkCallOverload(const Scope return {{retPack}}; } - const FunctionTypeVar* ftv = get(fn); - if (!ftv) + std::vector metaArgLocations; + + // Might be a callable table + if (const MetatableTypeVar* mttv = get(fn)) { - // Might be a callable table - if (const MetatableTypeVar* mttv = get(fn)) + if (std::optional ty = getIndexTypeFromType(scope, mttv->metatable, "__call", expr.func->location, false)) { - if (std::optional ty = getIndexTypeFromType(scope, mttv->metatable, "__call", expr.func->location, false)) + // Construct arguments with 'self' added in front + TypePackId metaCallArgPack = addTypePack(TypePackVar(TypePack{args->head, args->tail})); + + TypePack* metaCallArgs = getMutable(metaCallArgPack); + metaCallArgs->head.insert(metaCallArgs->head.begin(), fn); + + metaArgLocations = *argLocations; + metaArgLocations.insert(metaArgLocations.begin(), expr.func->location); + + if (FFlag::LuauFixRecursiveMetatableCall) { - // Construct arguments with 'self' added in front - TypePackId metaCallArgPack = addTypePack(TypePackVar(TypePack{args->head, args->tail})); - - TypePack* metaCallArgs = getMutable(metaCallArgPack); - metaCallArgs->head.insert(metaCallArgs->head.begin(), fn); - - std::vector metaArgLocations = argLocations; - metaArgLocations.insert(metaArgLocations.begin(), expr.func->location); + fn = instantiate(scope, *ty, expr.func->location); + argPack = metaCallArgPack; + args = metaCallArgs; + argLocations = &metaArgLocations; + } + else + { TypeId fn = *ty; fn = instantiate(scope, fn, expr.func->location); - return checkCallOverload(scope, expr, fn, retPack, metaCallArgPack, metaCallArgs, metaArgLocations, argListResult, + return checkCallOverload(scope, expr, fn, retPack, metaCallArgPack, metaCallArgs, &metaArgLocations, argListResult, overloadsThatMatchArgCount, overloadsThatDont, errors); } } + } + const FunctionTypeVar* ftv = get(fn); + if (!ftv) + { reportError(TypeError{expr.func->location, CannotCallNonFunction{fn}}); unify(retPack, errorRecoveryTypePack(scope), expr.func->location); return {{errorRecoveryTypePack(retPack)}}; @@ -3477,7 +3551,7 @@ std::optional> TypeChecker::checkCallOverload(const Scope return {}; } - checkArgumentList(scope, state, argPack, ftv->argTypes, argLocations); + checkArgumentList(scope, state, argPack, ftv->argTypes, *argLocations); if (!state.errors.empty()) { @@ -3772,7 +3846,7 @@ TypeId TypeChecker::checkRequire(const ScopePtr& scope, const ModuleInfo& module if (moduleInfo.name.empty()) { - if (FFlag::LuauStrictRequire && currentModule->mode == Mode::Strict) + if (currentModule->mode == Mode::Strict) { reportError(TypeError{location, UnknownRequire{}}); return errorRecoveryType(anyType); @@ -4268,9 +4342,11 @@ ScopePtr TypeChecker::childFunctionScope(const ScopePtr& parent, const Location& } // Creates a new Scope and carries forward the varargs from the parent. -ScopePtr TypeChecker::childScope(const ScopePtr& parent, const Location& location, int subLevel) +ScopePtr TypeChecker::childScope(const ScopePtr& parent, const Location& location) { - ScopePtr scope = std::make_shared(parent, subLevel); + ScopePtr scope = std::make_shared(parent); + if (FFlag::LuauProperTypeLevels) + scope->level = parent->level; scope->varargPack = parent->varargPack; currentModule->scopes.push_back(std::make_pair(location, scope)); @@ -4329,22 +4405,22 @@ TypeId TypeChecker::singletonType(std::string value) TypeId TypeChecker::errorRecoveryType(const ScopePtr& scope) { - return singletonTypes.errorRecoveryType(); + return getSingletonTypes().errorRecoveryType(); } TypeId TypeChecker::errorRecoveryType(TypeId guess) { - return singletonTypes.errorRecoveryType(guess); + return getSingletonTypes().errorRecoveryType(guess); } TypePackId TypeChecker::errorRecoveryTypePack(const ScopePtr& scope) { - return singletonTypes.errorRecoveryTypePack(); + return getSingletonTypes().errorRecoveryTypePack(); } TypePackId TypeChecker::errorRecoveryTypePack(TypePackId guess) { - return singletonTypes.errorRecoveryTypePack(guess); + return getSingletonTypes().errorRecoveryTypePack(guess); } std::optional TypeChecker::filterMap(TypeId type, TypeIdPredicate predicate) @@ -4547,6 +4623,7 @@ TypeId TypeChecker::resolveType(const ScopePtr& scope, const AstType& annotation else if (const auto& func = annotation.as()) { ScopePtr funcScope = childScope(scope, func->location); + funcScope->level = scope->level.incr(); auto [generics, genericPacks] = createGenericTypes(funcScope, std::nullopt, annotation, func->generics, func->genericPacks); diff --git a/Analysis/src/TypeUtils.cpp b/Analysis/src/TypeUtils.cpp index 0d9d91e0..8c6d5e49 100644 --- a/Analysis/src/TypeUtils.cpp +++ b/Analysis/src/TypeUtils.cpp @@ -19,7 +19,7 @@ std::optional findMetatableEntry(ErrorVec& errors, const ScopePtr& globa TypeId unwrapped = follow(*metatable); if (get(unwrapped)) - return singletonTypes.anyType; + return getSingletonTypes().anyType; const TableTypeVar* mtt = getTableType(unwrapped); if (!mtt) @@ -61,12 +61,12 @@ std::optional findTablePropertyRespectingMeta(ErrorVec& errors, const Sc { std::optional r = first(follow(itf->retType)); if (!r) - return singletonTypes.nilType; + return getSingletonTypes().nilType; else return *r; } else if (get(index)) - return singletonTypes.anyType; + return getSingletonTypes().anyType; else errors.push_back(TypeError{location, GenericError{"__index should either be a function or table. Got " + toString(index)}}); diff --git a/Analysis/src/TypeVar.cpp b/Analysis/src/TypeVar.cpp index 62715af5..571b13ca 100644 --- a/Analysis/src/TypeVar.cpp +++ b/Analysis/src/TypeVar.cpp @@ -21,6 +21,7 @@ LUAU_FASTINTVARIABLE(LuauTypeMaximumStringifierLength, 500) LUAU_FASTINTVARIABLE(LuauTableTypeMaximumStringifierLength, 0) LUAU_FASTFLAGVARIABLE(LuauRefactorTagging, false) LUAU_FASTFLAG(LuauErrorRecoveryType) +LUAU_FASTFLAG(DebugLuauFreezeArena) namespace Luau { @@ -579,11 +580,25 @@ SingletonTypes::SingletonTypes() , arena(new TypeArena) { TypeId stringMetatable = makeStringMetatable(); - stringType_.ty = PrimitiveTypeVar{PrimitiveTypeVar::String, makeStringMetatable()}; + stringType_.ty = PrimitiveTypeVar{PrimitiveTypeVar::String, stringMetatable}; persist(stringMetatable); + + debugFreezeArena = FFlag::DebugLuauFreezeArena; freeze(*arena); } +SingletonTypes::~SingletonTypes() +{ + // Destroy the arena with the same memory management flags it was created with + bool prevFlag = FFlag::DebugLuauFreezeArena; + FFlag::DebugLuauFreezeArena.value = debugFreezeArena; + + unfreeze(*arena); + arena.reset(nullptr); + + FFlag::DebugLuauFreezeArena.value = prevFlag; +} + TypeId SingletonTypes::makeStringMetatable() { const TypeId optionalNumber = arena->addType(UnionTypeVar{{nilType, numberType}}); @@ -641,6 +656,9 @@ TypeId SingletonTypes::makeStringMetatable() TypeId tableType = arena->addType(TableTypeVar{std::move(stringLib), std::nullopt, TypeLevel{}, TableState::Sealed}); + if (TableTypeVar* ttv = getMutable(tableType)) + ttv->name = "string"; + return arena->addType(TableTypeVar{{{{"__index", {tableType}}}}, std::nullopt, TypeLevel{}, TableState::Sealed}); } @@ -670,7 +688,11 @@ TypePackId SingletonTypes::errorRecoveryTypePack(TypePackId guess) return &errorTypePack_; } -SingletonTypes singletonTypes; +SingletonTypes& getSingletonTypes() +{ + static SingletonTypes singletonTypes; + return singletonTypes; +} void persist(TypeId ty) { @@ -719,6 +741,18 @@ void persist(TypeId ty) for (TypeId opt : itv->parts) queue.push_back(opt); } + else if (auto mtv = get(t)) + { + queue.push_back(mtv->table); + queue.push_back(mtv->metatable); + } + else if (get(t) || get(t) || get(t) || get(t) || get(t)) + { + } + else + { + LUAU_ASSERT(!"TypeId is not supported in a persist call"); + } } } @@ -736,6 +770,17 @@ void persist(TypePackId tp) if (p->tail) persist(*p->tail); } + else if (auto vtp = get(tp)) + { + persist(vtp->ty); + } + else if (get(tp)) + { + } + else + { + LUAU_ASSERT(!"TypePackId is not supported in a persist call"); + } } const TypeLevel* getLevel(TypeId ty) @@ -757,167 +802,6 @@ TypeLevel* getMutableLevel(TypeId ty) return const_cast(getLevel(ty)); } -struct QVarFinder -{ - mutable DenseHashSet seen; - - QVarFinder() - : seen(nullptr) - { - } - - bool hasSeen(const void* tv) const - { - if (seen.contains(tv)) - return true; - - seen.insert(tv); - return false; - } - - bool hasGeneric(TypeId tid) const - { - if (hasSeen(&tid->ty)) - return false; - - return Luau::visit(*this, tid->ty); - } - - bool hasGeneric(TypePackId tp) const - { - if (hasSeen(&tp->ty)) - return false; - - return Luau::visit(*this, tp->ty); - } - - bool operator()(const Unifiable::Free&) const - { - return false; - } - - bool operator()(const Unifiable::Bound& bound) const - { - return hasGeneric(bound.boundTo); - } - - bool operator()(const Unifiable::Generic&) const - { - return true; - } - bool operator()(const Unifiable::Error&) const - { - return false; - } - bool operator()(const PrimitiveTypeVar&) const - { - return false; - } - - bool operator()(const SingletonTypeVar&) const - { - return false; - } - - bool operator()(const FunctionTypeVar& ftv) const - { - if (hasGeneric(ftv.argTypes)) - return true; - return hasGeneric(ftv.retType); - } - - bool operator()(const TableTypeVar& ttv) const - { - if (ttv.state == TableState::Generic) - return true; - - if (ttv.indexer) - { - if (hasGeneric(ttv.indexer->indexType)) - return true; - if (hasGeneric(ttv.indexer->indexResultType)) - return true; - } - - for (const auto& [_name, prop] : ttv.props) - { - if (hasGeneric(prop.type)) - return true; - } - - return false; - } - - bool operator()(const MetatableTypeVar& mtv) const - { - return hasGeneric(mtv.table) || hasGeneric(mtv.metatable); - } - - bool operator()(const ClassTypeVar& ctv) const - { - for (const auto& [name, prop] : ctv.props) - { - if (hasGeneric(prop.type)) - return true; - } - - if (ctv.parent) - return hasGeneric(*ctv.parent); - - return false; - } - - bool operator()(const AnyTypeVar&) const - { - return false; - } - - bool operator()(const UnionTypeVar& utv) const - { - for (TypeId tid : utv.options) - if (hasGeneric(tid)) - return true; - - return false; - } - - bool operator()(const IntersectionTypeVar& utv) const - { - for (TypeId tid : utv.parts) - if (hasGeneric(tid)) - return true; - - return false; - } - - bool operator()(const LazyTypeVar&) const - { - return false; - } - - bool operator()(const Unifiable::Bound& bound) const - { - return hasGeneric(bound.boundTo); - } - - bool operator()(const TypePack& pack) const - { - for (TypeId ty : pack.head) - if (hasGeneric(ty)) - return true; - - if (pack.tail) - return hasGeneric(*pack.tail); - - return false; - } - - bool operator()(const VariadicTypePack& pack) const - { - return hasGeneric(pack.ty); - } -}; - const Property* lookupClassProp(const ClassTypeVar* cls, const Name& name) { while (cls) @@ -953,16 +837,6 @@ bool isSubclass(const ClassTypeVar* cls, const ClassTypeVar* parent) return false; } -bool hasGeneric(TypeId ty) -{ - return Luau::visit(QVarFinder{}, ty->ty); -} - -bool hasGeneric(TypePackId tp) -{ - return Luau::visit(QVarFinder{}, tp->ty); -} - UnionTypeVarIterator::UnionTypeVarIterator(const UnionTypeVar* utv) { LUAU_ASSERT(utv); diff --git a/Analysis/src/Unifier.cpp b/Analysis/src/Unifier.cpp index d0b18837..c5aab856 100644 --- a/Analysis/src/Unifier.cpp +++ b/Analysis/src/Unifier.cpp @@ -14,7 +14,7 @@ LUAU_FASTINT(LuauTypeInferRecursionLimit); LUAU_FASTINT(LuauTypeInferTypePackLoopLimit); LUAU_FASTINTVARIABLE(LuauTypeInferIterationLimit, 2000); -LUAU_FASTFLAGVARIABLE(LuauTableSubtypingVariance, false); +LUAU_FASTFLAGVARIABLE(LuauTableSubtypingVariance2, false); LUAU_FASTFLAGVARIABLE(LuauUnionHeuristic, false) LUAU_FASTFLAGVARIABLE(LuauTableUnificationEarlyTest, false) LUAU_FASTFLAGVARIABLE(LuauOccursCheckOkWithRecursiveFunctions, false) @@ -22,9 +22,82 @@ LUAU_FASTFLAGVARIABLE(LuauExtendedTypeMismatchError, false) LUAU_FASTFLAG(LuauSingletonTypes) LUAU_FASTFLAGVARIABLE(LuauExtendedClassMismatchError, false) LUAU_FASTFLAG(LuauErrorRecoveryType); +LUAU_FASTFLAG(LuauProperTypeLevels); +LUAU_FASTFLAGVARIABLE(LuauExtendedUnionMismatchError, false) +LUAU_FASTFLAGVARIABLE(LuauExtendedFunctionMismatchError, false) namespace Luau { + +struct PromoteTypeLevels +{ + TxnLog& log; + TypeLevel minLevel; + + explicit PromoteTypeLevels(TxnLog& log, TypeLevel minLevel) + : log(log) + , minLevel(minLevel) + {} + + template + void promote(TID ty, T* t) + { + LUAU_ASSERT(t); + if (minLevel.subsumesStrict(t->level)) + { + log(ty); + t->level = minLevel; + } + } + + template + void cycle(TID) {} + + template + bool operator()(TID, const T&) + { + return true; + } + + bool operator()(TypeId ty, const FreeTypeVar&) + { + promote(ty, getMutable(ty)); + return true; + } + + bool operator()(TypeId ty, const FunctionTypeVar&) + { + promote(ty, getMutable(ty)); + return true; + } + + bool operator()(TypeId ty, const TableTypeVar&) + { + promote(ty, getMutable(ty)); + return true; + } + + bool operator()(TypePackId tp, const FreeTypePack&) + { + promote(tp, getMutable(tp)); + return true; + } +}; + +void promoteTypeLevels(TxnLog& log, TypeLevel minLevel, TypeId ty) +{ + PromoteTypeLevels ptl{log, minLevel}; + DenseHashSet seen{nullptr}; + visitTypeVarOnce(ty, ptl, seen); +} + +void promoteTypeLevels(TxnLog& log, TypeLevel minLevel, TypePackId tp) +{ + PromoteTypeLevels ptl{log, minLevel}; + DenseHashSet seen{nullptr}; + visitTypeVarOnce(tp, ptl, seen); +} + struct SkipCacheForType { SkipCacheForType(const DenseHashMap& skipCacheForType) @@ -127,6 +200,29 @@ static std::optional hasUnificationTooComplex(const ErrorVec& errors) return *it; } +// Used for tagged union matching heuristic, returns first singleton type field +static std::optional> getTableMatchTag(TypeId type) +{ + LUAU_ASSERT(FFlag::LuauExtendedUnionMismatchError); + + type = follow(type); + + if (auto ttv = get(type)) + { + for (auto&& [name, prop] : ttv->props) + { + if (auto sing = get(follow(prop.type))) + return {{name, sing}}; + } + } + else if (auto mttv = get(type)) + { + return getTableMatchTag(mttv->table); + } + + return std::nullopt; +} + Unifier::Unifier(TypeArena* types, Mode mode, ScopePtr globalScope, const Location& location, Variance variance, UnifierSharedState& sharedState) : types(types) , mode(mode) @@ -214,9 +310,11 @@ void Unifier::tryUnify_(TypeId superTy, TypeId subTy, bool isFunctionCall, bool { occursCheck(superTy, subTy); + TypeLevel superLevel = l->level; + // Unification can't change the level of a generic. auto rightGeneric = get(subTy); - if (rightGeneric && !rightGeneric->level.subsumes(l->level)) + if (rightGeneric && !rightGeneric->level.subsumes(superLevel)) { // TODO: a more informative error message? CLI-39912 errors.push_back(TypeError{location, GenericError{"Generic subtype escaping scope"}}); @@ -226,7 +324,9 @@ void Unifier::tryUnify_(TypeId superTy, TypeId subTy, bool isFunctionCall, bool // The occurrence check might have caused superTy no longer to be a free type if (!get(superTy)) { - if (auto rightLevel = getMutableLevel(subTy)) + if (FFlag::LuauProperTypeLevels) + promoteTypeLevels(log, superLevel, subTy); + else if (auto rightLevel = getMutableLevel(subTy)) { if (!rightLevel->subsumes(l->level)) *rightLevel = l->level; @@ -240,6 +340,8 @@ void Unifier::tryUnify_(TypeId superTy, TypeId subTy, bool isFunctionCall, bool } else if (r) { + TypeLevel subLevel = r->level; + occursCheck(subTy, superTy); // Unification can't change the level of a generic. @@ -253,10 +355,16 @@ void Unifier::tryUnify_(TypeId superTy, TypeId subTy, bool isFunctionCall, bool if (!get(subTy)) { - if (auto leftLevel = getMutableLevel(superTy)) + if (FFlag::LuauProperTypeLevels) + promoteTypeLevels(log, subLevel, superTy); + + if (auto superLevel = getMutableLevel(superTy)) { - if (!leftLevel->subsumes(r->level)) - *leftLevel = r->level; + if (!superLevel->subsumes(r->level)) + { + log(superTy); + *superLevel = r->level; + } } log(subTy); @@ -327,7 +435,7 @@ void Unifier::tryUnify_(TypeId superTy, TypeId subTy, bool isFunctionCall, bool else if (failed) { if (FFlag::LuauExtendedTypeMismatchError && firstFailedOption) - errors.push_back(TypeError{location, TypeMismatch{superTy, subTy, "Not all union options are compatible", *firstFailedOption}}); + errors.push_back(TypeError{location, TypeMismatch{superTy, subTy, "Not all union options are compatible.", *firstFailedOption}}); else errors.push_back(TypeError{location, TypeMismatch{superTy, subTy}}); } @@ -338,28 +446,46 @@ void Unifier::tryUnify_(TypeId superTy, TypeId subTy, bool isFunctionCall, bool bool found = false; std::optional unificationTooComplex; + size_t failedOptionCount = 0; + std::optional failedOption; + + bool foundHeuristic = false; size_t startIndex = 0; if (FFlag::LuauUnionHeuristic) { - bool found = false; - - const std::string* subName = getName(subTy); - if (subName) + if (const std::string* subName = getName(subTy)) { for (size_t i = 0; i < uv->options.size(); ++i) { const std::string* optionName = getName(uv->options[i]); if (optionName && *optionName == *subName) { - found = true; + foundHeuristic = true; startIndex = i; break; } } } - if (!found && cacheEnabled) + if (FFlag::LuauExtendedUnionMismatchError) + { + if (auto subMatchTag = getTableMatchTag(subTy)) + { + for (size_t i = 0; i < uv->options.size(); ++i) + { + auto optionMatchTag = getTableMatchTag(uv->options[i]); + if (optionMatchTag && optionMatchTag->first == subMatchTag->first && *optionMatchTag->second == *subMatchTag->second) + { + foundHeuristic = true; + startIndex = i; + break; + } + } + } + } + + if (!foundHeuristic && cacheEnabled) { for (size_t i = 0; i < uv->options.size(); ++i) { @@ -390,15 +516,27 @@ void Unifier::tryUnify_(TypeId superTy, TypeId subTy, bool isFunctionCall, bool { unificationTooComplex = e; } + else if (FFlag::LuauExtendedUnionMismatchError && !isNil(type)) + { + failedOptionCount++; + + if (!failedOption) + failedOption = {innerState.errors.front()}; + } innerState.log.rollback(); } if (unificationTooComplex) + { errors.push_back(*unificationTooComplex); + } else if (!found) { - if (FFlag::LuauExtendedTypeMismatchError) + if (FFlag::LuauExtendedUnionMismatchError && (failedOptionCount == 1 || foundHeuristic) && failedOption) + errors.push_back( + TypeError{location, TypeMismatch{superTy, subTy, "None of the union options are compatible. For example:", *failedOption}}); + else if (FFlag::LuauExtendedTypeMismatchError) errors.push_back(TypeError{location, TypeMismatch{superTy, subTy, "none of the union options are compatible"}}); else errors.push_back(TypeError{location, TypeMismatch{superTy, subTy}}); @@ -431,7 +569,7 @@ void Unifier::tryUnify_(TypeId superTy, TypeId subTy, bool isFunctionCall, bool if (unificationTooComplex) errors.push_back(*unificationTooComplex); else if (firstFailedOption) - errors.push_back(TypeError{location, TypeMismatch{superTy, subTy, "Not all intersection parts are compatible", *firstFailedOption}}); + errors.push_back(TypeError{location, TypeMismatch{superTy, subTy, "Not all intersection parts are compatible.", *firstFailedOption}}); } else { @@ -771,6 +909,10 @@ void Unifier::tryUnify_(TypePackId superTp, TypePackId subTp, bool isFunctionCal if (superIter.good() && subIter.good()) { tryUnify_(*superIter, *subIter); + + if (FFlag::LuauExtendedFunctionMismatchError && !errors.empty() && !firstPackErrorPos) + firstPackErrorPos = loopCount; + superIter.advance(); subIter.advance(); continue; @@ -853,13 +995,13 @@ void Unifier::tryUnify_(TypePackId superTp, TypePackId subTp, bool isFunctionCal while (superIter.good()) { - tryUnify_(singletonTypes.errorRecoveryType(), *superIter); + tryUnify_(getSingletonTypes().errorRecoveryType(), *superIter); superIter.advance(); } while (subIter.good()) { - tryUnify_(singletonTypes.errorRecoveryType(), *subIter); + tryUnify_(getSingletonTypes().errorRecoveryType(), *subIter); subIter.advance(); } @@ -917,14 +1059,22 @@ void Unifier::tryUnifyFunctions(TypeId superTy, TypeId subTy, bool isFunctionCal if (numGenerics != rf->generics.size()) { numGenerics = std::min(lf->generics.size(), rf->generics.size()); - errors.push_back(TypeError{location, TypeMismatch{superTy, subTy}}); + + if (FFlag::LuauExtendedFunctionMismatchError) + errors.push_back(TypeError{location, TypeMismatch{superTy, subTy, "different number of generic type parameters"}}); + else + errors.push_back(TypeError{location, TypeMismatch{superTy, subTy}}); } size_t numGenericPacks = lf->genericPacks.size(); if (numGenericPacks != rf->genericPacks.size()) { numGenericPacks = std::min(lf->genericPacks.size(), rf->genericPacks.size()); - errors.push_back(TypeError{location, TypeMismatch{superTy, subTy}}); + + if (FFlag::LuauExtendedFunctionMismatchError) + errors.push_back(TypeError{location, TypeMismatch{superTy, subTy, "different number of generic type pack parameters"}}); + else + errors.push_back(TypeError{location, TypeMismatch{superTy, subTy}}); } for (size_t i = 0; i < numGenerics; i++) @@ -936,13 +1086,49 @@ void Unifier::tryUnifyFunctions(TypeId superTy, TypeId subTy, bool isFunctionCal { Unifier innerState = makeChildUnifier(); - ctx = CountMismatch::Arg; - innerState.tryUnify_(rf->argTypes, lf->argTypes, isFunctionCall); + if (FFlag::LuauExtendedFunctionMismatchError) + { + innerState.ctx = CountMismatch::Arg; + innerState.tryUnify_(rf->argTypes, lf->argTypes, isFunctionCall); - ctx = CountMismatch::Result; - innerState.tryUnify_(lf->retType, rf->retType); + bool reported = !innerState.errors.empty(); - checkChildUnifierTypeMismatch(innerState.errors, superTy, subTy); + if (auto e = hasUnificationTooComplex(innerState.errors)) + errors.push_back(*e); + else if (!innerState.errors.empty() && innerState.firstPackErrorPos) + errors.push_back( + TypeError{location, TypeMismatch{superTy, subTy, format("Argument #%d type is not compatible.", *innerState.firstPackErrorPos), + innerState.errors.front()}}); + else if (!innerState.errors.empty()) + errors.push_back(TypeError{location, TypeMismatch{superTy, subTy, "", innerState.errors.front()}}); + + innerState.ctx = CountMismatch::Result; + innerState.tryUnify_(lf->retType, rf->retType); + + if (!reported) + { + if (auto e = hasUnificationTooComplex(innerState.errors)) + errors.push_back(*e); + else if (!innerState.errors.empty() && size(lf->retType) == 1 && finite(lf->retType)) + errors.push_back(TypeError{location, TypeMismatch{superTy, subTy, "Return type is not compatible.", innerState.errors.front()}}); + else if (!innerState.errors.empty() && innerState.firstPackErrorPos) + errors.push_back( + TypeError{location, TypeMismatch{superTy, subTy, format("Return #%d type is not compatible.", *innerState.firstPackErrorPos), + innerState.errors.front()}}); + else if (!innerState.errors.empty()) + errors.push_back(TypeError{location, TypeMismatch{superTy, subTy, "", innerState.errors.front()}}); + } + } + else + { + ctx = CountMismatch::Arg; + innerState.tryUnify_(rf->argTypes, lf->argTypes, isFunctionCall); + + ctx = CountMismatch::Result; + innerState.tryUnify_(lf->retType, rf->retType); + + checkChildUnifierTypeMismatch(innerState.errors, superTy, subTy); + } log.concat(std::move(innerState.log)); } @@ -994,7 +1180,7 @@ struct Resetter void Unifier::tryUnifyTables(TypeId left, TypeId right, bool isIntersection) { - if (!FFlag::LuauTableSubtypingVariance) + if (!FFlag::LuauTableSubtypingVariance2) return DEPRECATED_tryUnifyTables(left, right, isIntersection); TableTypeVar* lt = getMutable(left); @@ -1133,7 +1319,7 @@ void Unifier::tryUnifyTables(TypeId left, TypeId right, bool isIntersection) // TODO: hopefully readonly/writeonly properties will fix this. Property clone = prop; clone.type = deeplyOptional(clone.type); - log(lt); + log(left); lt->props[name] = clone; } else if (variance == Covariant) @@ -1146,7 +1332,7 @@ void Unifier::tryUnifyTables(TypeId left, TypeId right, bool isIntersection) } else if (lt->state == TableState::Free) { - log(lt); + log(left); lt->props[name] = prop; } else @@ -1176,7 +1362,7 @@ void Unifier::tryUnifyTables(TypeId left, TypeId right, bool isIntersection) // e.g. table.insert(t, 1) where t is a non-sealed table and doesn't have an indexer. // TODO: we only need to do this if the supertype's indexer is read/write // since that can add indexed elements. - log(rt); + log(right); rt->indexer = lt->indexer; } } @@ -1185,7 +1371,7 @@ void Unifier::tryUnifyTables(TypeId left, TypeId right, bool isIntersection) // Symmetric if we are invariant if (lt->state == TableState::Unsealed || lt->state == TableState::Free) { - log(lt); + log(left); lt->indexer = rt->indexer; } } @@ -1241,15 +1427,15 @@ TypeId Unifier::deeplyOptional(TypeId ty, std::unordered_map see TableTypeVar* resultTtv = getMutable(result); for (auto& [name, prop] : resultTtv->props) prop.type = deeplyOptional(prop.type, seen); - return types->addType(UnionTypeVar{{singletonTypes.nilType, result}}); + return types->addType(UnionTypeVar{{getSingletonTypes().nilType, result}}); } else - return types->addType(UnionTypeVar{{singletonTypes.nilType, ty}}); + return types->addType(UnionTypeVar{{getSingletonTypes().nilType, ty}}); } void Unifier::DEPRECATED_tryUnifyTables(TypeId left, TypeId right, bool isIntersection) { - LUAU_ASSERT(!FFlag::LuauTableSubtypingVariance); + LUAU_ASSERT(!FFlag::LuauTableSubtypingVariance2); Resetter resetter{&variance}; variance = Invariant; @@ -1467,7 +1653,7 @@ void Unifier::tryUnifySealedTables(TypeId left, TypeId right, bool isIntersectio } else if (lt->indexer) { - innerState.tryUnify_(lt->indexer->indexType, singletonTypes.stringType); + innerState.tryUnify_(lt->indexer->indexType, getSingletonTypes().stringType); // We already try to unify properties in both tables. // Skip those and just look for the ones remaining and see if they fit into the indexer. for (const auto& [name, type] : rt->props) @@ -1636,7 +1822,7 @@ void Unifier::tryUnifyWithClass(TypeId superTy, TypeId subTy, bool reversed) ok = false; errors.push_back(TypeError{location, UnknownProperty{superTy, propName}}); if (!FFlag::LuauExtendedClassMismatchError) - tryUnify_(prop.type, singletonTypes.errorRecoveryType()); + tryUnify_(prop.type, getSingletonTypes().errorRecoveryType()); } else { @@ -1825,7 +2011,7 @@ void Unifier::tryUnifyWithAny(TypeId any, TypeId ty) if (get(ty) || get(ty) || get(ty)) return; - const TypePackId anyTypePack = types->addTypePack(TypePackVar{VariadicTypePack{singletonTypes.anyType}}); + const TypePackId anyTypePack = types->addTypePack(TypePackVar{VariadicTypePack{getSingletonTypes().anyType}}); const TypePackId anyTP = get(any) ? anyTypePack : types->addTypePack(TypePackVar{Unifiable::Error{}}); @@ -1834,14 +2020,14 @@ void Unifier::tryUnifyWithAny(TypeId any, TypeId ty) sharedState.tempSeenTy.clear(); sharedState.tempSeenTp.clear(); - Luau::tryUnifyWithAny(queue, *this, sharedState.tempSeenTy, sharedState.tempSeenTp, singletonTypes.anyType, anyTP); + Luau::tryUnifyWithAny(queue, *this, sharedState.tempSeenTy, sharedState.tempSeenTp, getSingletonTypes().anyType, anyTP); } void Unifier::tryUnifyWithAny(TypePackId any, TypePackId ty) { LUAU_ASSERT(get(any)); - const TypeId anyTy = singletonTypes.errorRecoveryType(); + const TypeId anyTy = getSingletonTypes().errorRecoveryType(); std::vector queue; @@ -1887,7 +2073,7 @@ void Unifier::occursCheck(DenseHashSet& seen, TypeId needle, TypeId hays { errors.push_back(TypeError{location, OccursCheckFailed{}}); log(needle); - *asMutable(needle) = *singletonTypes.errorRecoveryType(); + *asMutable(needle) = *getSingletonTypes().errorRecoveryType(); return; } @@ -1951,7 +2137,7 @@ void Unifier::occursCheck(DenseHashSet& seen, TypePackId needle, Typ { errors.push_back(TypeError{location, OccursCheckFailed{}}); log(needle); - *asMutable(needle) = *singletonTypes.errorRecoveryTypePack(); + *asMutable(needle) = *getSingletonTypes().errorRecoveryTypePack(); return; } @@ -2005,7 +2191,7 @@ void Unifier::checkChildUnifierTypeMismatch(const ErrorVec& innerErrors, const s errors.push_back(*e); else if (!innerErrors.empty()) errors.push_back( - TypeError{location, TypeMismatch{wantedType, givenType, format("Property '%s' is not compatible", prop.c_str()), innerErrors.front()}}); + TypeError{location, TypeMismatch{wantedType, givenType, format("Property '%s' is not compatible.", prop.c_str()), innerErrors.front()}}); } void Unifier::ice(const std::string& message, const Location& location) diff --git a/Ast/src/Parser.cpp b/Ast/src/Parser.cpp index 3d0d5b7e..dd24f27c 100644 --- a/Ast/src/Parser.cpp +++ b/Ast/src/Parser.cpp @@ -10,7 +10,6 @@ // See docs/SyntaxChanges.md for an explanation. LUAU_FASTINTVARIABLE(LuauRecursionLimit, 1000) LUAU_FASTINTVARIABLE(LuauParseErrorLimit, 100) -LUAU_FASTFLAGVARIABLE(LuauCaptureBrokenCommentSpans, false) LUAU_FASTFLAGVARIABLE(LuauIfElseExpressionBaseSupport, false) LUAU_FASTFLAGVARIABLE(LuauIfStatementRecursionGuard, false) LUAU_FASTFLAGVARIABLE(LuauFixAmbiguousErrorRecoveryInAssign, false) @@ -159,7 +158,7 @@ ParseResult Parser::parse(const char* buffer, size_t bufferSize, AstNameTable& n { std::vector hotcomments; - while (isComment(p.lexer.current()) || (FFlag::LuauCaptureBrokenCommentSpans && p.lexer.current().type == Lexeme::BrokenComment)) + while (isComment(p.lexer.current()) || p.lexer.current().type == Lexeme::BrokenComment) { const char* text = p.lexer.current().data; unsigned int length = p.lexer.current().length; @@ -2780,7 +2779,7 @@ const Lexeme& Parser::nextLexeme() const Lexeme& lexeme = lexer.next(/*skipComments*/ false); // Subtlety: Broken comments are weird because we record them as comments AND pass them to the parser as a lexeme. // The parser will turn this into a proper syntax error. - if (FFlag::LuauCaptureBrokenCommentSpans && lexeme.type == Lexeme::BrokenComment) + if (lexeme.type == Lexeme::BrokenComment) commentLocations.push_back(Comment{lexeme.type, lexeme.location}); if (isComment(lexeme)) commentLocations.push_back(Comment{lexeme.type, lexeme.location}); diff --git a/CLI/Analyze.cpp b/CLI/Analyze.cpp index 9230d80d..aecb619a 100644 --- a/CLI/Analyze.cpp +++ b/CLI/Analyze.cpp @@ -11,26 +11,33 @@ enum class ReportFormat { Default, - Luacheck + Luacheck, + Gnu, }; -static void report(ReportFormat format, const char* name, const Luau::Location& location, const char* type, const char* message) +static void report(ReportFormat format, const char* name, const Luau::Location& loc, const char* type, const char* message) { switch (format) { case ReportFormat::Default: - fprintf(stderr, "%s(%d,%d): %s: %s\n", name, location.begin.line + 1, location.begin.column + 1, type, message); + fprintf(stderr, "%s(%d,%d): %s: %s\n", name, loc.begin.line + 1, loc.begin.column + 1, type, message); break; case ReportFormat::Luacheck: { // Note: luacheck's end column is inclusive but our end column is exclusive // In addition, luacheck doesn't support multi-line messages, so if the error is multiline we'll fake end column as 100 and hope for the best - int columnEnd = (location.begin.line == location.end.line) ? location.end.column : 100; + int columnEnd = (loc.begin.line == loc.end.line) ? loc.end.column : 100; - fprintf(stdout, "%s:%d:%d-%d: (W0) %s: %s\n", name, location.begin.line + 1, location.begin.column + 1, columnEnd, type, message); + // Use stdout to match luacheck behavior + fprintf(stdout, "%s:%d:%d-%d: (W0) %s: %s\n", name, loc.begin.line + 1, loc.begin.column + 1, columnEnd, type, message); break; } + + case ReportFormat::Gnu: + // Note: GNU end column is inclusive but our end column is exclusive + fprintf(stderr, "%s:%d.%d-%d.%d: %s: %s\n", name, loc.begin.line + 1, loc.begin.column + 1, loc.end.line + 1, loc.end.column, type, message); + break; } } @@ -97,6 +104,7 @@ static void displayHelp(const char* argv0) printf("\n"); printf("Available options:\n"); printf(" --formatter=plain: report analysis errors in Luacheck-compatible format\n"); + printf(" --formatter=gnu: report analysis errors in GNU-compatible format\n"); } static int assertionHandler(const char* expr, const char* file, int line) @@ -201,6 +209,8 @@ int main(int argc, char** argv) if (strcmp(argv[i], "--formatter=plain") == 0) format = ReportFormat::Luacheck; + else if (strcmp(argv[i], "--formatter=gnu") == 0) + format = ReportFormat::Gnu; else if (strcmp(argv[i], "--annotate") == 0) annotate = true; } diff --git a/CLI/Coverage.cpp b/CLI/Coverage.cpp new file mode 100644 index 00000000..254df3f0 --- /dev/null +++ b/CLI/Coverage.cpp @@ -0,0 +1,88 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +#include "Coverage.h" + +#include "lua.h" + +#include +#include + +struct Coverage +{ + lua_State* L = nullptr; + std::vector functions; +} gCoverage; + +void coverageInit(lua_State* L) +{ + gCoverage.L = lua_mainthread(L); +} + +bool coverageActive() +{ + return gCoverage.L != nullptr; +} + +void coverageTrack(lua_State* L, int funcindex) +{ + int ref = lua_ref(L, funcindex); + gCoverage.functions.push_back(ref); +} + +static void coverageCallback(void* context, const char* function, int linedefined, int depth, const int* hits, size_t size) +{ + FILE* f = static_cast(context); + + std::string name; + + if (depth == 0) + name = "
"; + else if (function) + name = std::string(function) + ":" + std::to_string(linedefined); + else + name = ":" + std::to_string(linedefined); + + fprintf(f, "FN:%d,%s\n", linedefined, name.c_str()); + + for (size_t i = 0; i < size; ++i) + if (hits[i] != -1) + { + fprintf(f, "FNDA:%d,%s\n", hits[i], name.c_str()); + break; + } + + for (size_t i = 0; i < size; ++i) + if (hits[i] != -1) + fprintf(f, "DA:%d,%d\n", int(i), hits[i]); +} + +void coverageDump(const char* path) +{ + lua_State* L = gCoverage.L; + + FILE* f = fopen(path, "w"); + if (!f) + { + fprintf(stderr, "Error opening coverage %s\n", path); + return; + } + + fprintf(f, "TN:\n"); + + for (int fref: gCoverage.functions) + { + lua_getref(L, fref); + + lua_Debug ar = {}; + lua_getinfo(L, -1, "s", &ar); + + fprintf(f, "SF:%s\n", ar.short_src); + lua_getcoverage(L, -1, f, coverageCallback); + fprintf(f, "end_of_record\n"); + + lua_pop(L, 1); + } + + fclose(f); + + printf("Coverage dump written to %s (%d functions)\n", path, int(gCoverage.functions.size())); +} diff --git a/CLI/Coverage.h b/CLI/Coverage.h new file mode 100644 index 00000000..74be4e5c --- /dev/null +++ b/CLI/Coverage.h @@ -0,0 +1,10 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +#pragma once + +struct lua_State; + +void coverageInit(lua_State* L); +bool coverageActive(); + +void coverageTrack(lua_State* L, int funcindex); +void coverageDump(const char* path); diff --git a/CLI/FileUtils.cpp b/CLI/FileUtils.cpp index b3c9557b..cb993dfe 100644 --- a/CLI/FileUtils.cpp +++ b/CLI/FileUtils.cpp @@ -67,6 +67,10 @@ std::optional readFile(const std::string& name) if (read != size_t(length)) return std::nullopt; + // Skip first line if it's a shebang + if (length > 2 && result[0] == '#' && result[1] == '!') + result.erase(0, result.find('\n')); + return result; } diff --git a/CLI/Profiler.cpp b/CLI/Profiler.cpp index c6d15a7f..30a171f0 100644 --- a/CLI/Profiler.cpp +++ b/CLI/Profiler.cpp @@ -110,12 +110,12 @@ void profilerStop() gProfiler.thread.join(); } -void profilerDump(const char* name) +void profilerDump(const char* path) { - FILE* f = fopen(name, "wb"); + FILE* f = fopen(path, "wb"); if (!f) { - fprintf(stderr, "Error opening profile %s\n", name); + fprintf(stderr, "Error opening profile %s\n", path); return; } @@ -129,7 +129,7 @@ void profilerDump(const char* name) fclose(f); - printf("Profiler dump written to %s (total runtime %.3f seconds, %lld samples, %lld stacks)\n", name, double(total) / 1e6, + printf("Profiler dump written to %s (total runtime %.3f seconds, %lld samples, %lld stacks)\n", path, double(total) / 1e6, static_cast(gProfiler.samples.load()), static_cast(gProfiler.data.size())); uint64_t totalgc = 0; diff --git a/CLI/Profiler.h b/CLI/Profiler.h index 0a407e47..67b1acfd 100644 --- a/CLI/Profiler.h +++ b/CLI/Profiler.h @@ -5,4 +5,4 @@ struct lua_State; void profilerStart(lua_State* L, int frequency); void profilerStop(); -void profilerDump(const char* name); \ No newline at end of file +void profilerDump(const char* path); diff --git a/CLI/Repl.cpp b/CLI/Repl.cpp index 2cdd0062..35c02f2c 100644 --- a/CLI/Repl.cpp +++ b/CLI/Repl.cpp @@ -8,6 +8,7 @@ #include "FileUtils.h" #include "Profiler.h" +#include "Coverage.h" #include "linenoise.hpp" @@ -24,6 +25,16 @@ enum class CompileFormat Binary }; +static Luau::CompileOptions copts() +{ + Luau::CompileOptions result = {}; + result.optimizationLevel = 1; + result.debugLevel = 1; + result.coverageLevel = coverageActive() ? 2 : 0; + + return result; +} + static int lua_loadstring(lua_State* L) { size_t l = 0; @@ -32,7 +43,7 @@ static int lua_loadstring(lua_State* L) lua_setsafeenv(L, LUA_ENVIRONINDEX, false); - std::string bytecode = Luau::compile(std::string(s, l)); + std::string bytecode = Luau::compile(std::string(s, l), copts()); if (luau_load(L, chunkname, bytecode.data(), bytecode.size(), 0) == 0) return 1; @@ -79,9 +90,12 @@ static int lua_require(lua_State* L) luaL_sandboxthread(ML); // now we can compile & run module on the new thread - std::string bytecode = Luau::compile(*source); + std::string bytecode = Luau::compile(*source, copts()); if (luau_load(ML, chunkname.c_str(), bytecode.data(), bytecode.size(), 0) == 0) { + if (coverageActive()) + coverageTrack(ML, -1); + int status = lua_resume(ML, L, 0); if (status == 0) @@ -149,7 +163,7 @@ static void setupState(lua_State* L) static std::string runCode(lua_State* L, const std::string& source) { - std::string bytecode = Luau::compile(source); + std::string bytecode = Luau::compile(source, copts()); if (luau_load(L, "=stdin", bytecode.data(), bytecode.size(), 0) != 0) { @@ -329,11 +343,14 @@ static bool runFile(const char* name, lua_State* GL) std::string chunkname = "=" + std::string(name); - std::string bytecode = Luau::compile(*source); + std::string bytecode = Luau::compile(*source, copts()); int status = 0; if (luau_load(L, chunkname.c_str(), bytecode.data(), bytecode.size(), 0) == 0) { + if (coverageActive()) + coverageTrack(L, -1); + status = lua_resume(L, NULL, 0); } else @@ -437,6 +454,7 @@ static void displayHelp(const char* argv0) printf("\n"); printf("Available options:\n"); printf(" --profile[=N]: profile the code using N Hz sampling (default 10000) and output results to profile.out\n"); + printf(" --coverage: collect code coverage while running the code and output results to coverage.out\n"); } static int assertionHandler(const char* expr, const char* file, int line) @@ -495,6 +513,7 @@ int main(int argc, char** argv) setupState(L); int profile = 0; + bool coverage = false; for (int i = 1; i < argc; ++i) { @@ -505,11 +524,16 @@ int main(int argc, char** argv) profile = 10000; // default to 10 KHz else if (strncmp(argv[i], "--profile=", 10) == 0) profile = atoi(argv[i] + 10); + else if (strcmp(argv[i], "--coverage") == 0) + coverage = true; } if (profile) profilerStart(L, profile); + if (coverage) + coverageInit(L); + std::vector files = getSourceFiles(argc, argv); int failed = 0; @@ -523,6 +547,9 @@ int main(int argc, char** argv) profilerDump("profile.out"); } + if (coverage) + coverageDump("coverage.out"); + return failed; } } diff --git a/Compiler/src/Compiler.cpp b/Compiler/src/Compiler.cpp index 2c1e85ff..8f74ffed 100644 --- a/Compiler/src/Compiler.cpp +++ b/Compiler/src/Compiler.cpp @@ -10,7 +10,6 @@ #include #include -LUAU_FASTFLAGVARIABLE(LuauPreloadClosures, false) LUAU_FASTFLAG(LuauIfElseExpressionBaseSupport) LUAU_FASTFLAGVARIABLE(LuauBit32CountBuiltin, false) @@ -462,20 +461,17 @@ struct Compiler bool shared = false; - if (FFlag::LuauPreloadClosures) + // Optimization: when closure has no upvalues, or upvalues are safe to share, instead of allocating it every time we can share closure + // objects (this breaks assumptions about function identity which can lead to setfenv not working as expected, so we disable this when it + // is used) + if (options.optimizationLevel >= 1 && shouldShareClosure(expr) && !setfenvUsed) { - // Optimization: when closure has no upvalues, or upvalues are safe to share, instead of allocating it every time we can share closure - // objects (this breaks assumptions about function identity which can lead to setfenv not working as expected, so we disable this when it - // is used) - if (options.optimizationLevel >= 1 && shouldShareClosure(expr) && !setfenvUsed) - { - int32_t cid = bytecode.addConstantClosure(f->id); + int32_t cid = bytecode.addConstantClosure(f->id); - if (cid >= 0 && cid < 32768) - { - bytecode.emitAD(LOP_DUPCLOSURE, target, cid); - shared = true; - } + if (cid >= 0 && cid < 32768) + { + bytecode.emitAD(LOP_DUPCLOSURE, target, cid); + shared = true; } } diff --git a/Makefile b/Makefile index 15c7ff7a..b144cac6 100644 --- a/Makefile +++ b/Makefile @@ -27,7 +27,7 @@ TESTS_SOURCES=$(wildcard tests/*.cpp) TESTS_OBJECTS=$(TESTS_SOURCES:%=$(BUILD)/%.o) TESTS_TARGET=$(BUILD)/luau-tests -REPL_CLI_SOURCES=CLI/FileUtils.cpp CLI/Profiler.cpp CLI/Repl.cpp +REPL_CLI_SOURCES=CLI/FileUtils.cpp CLI/Profiler.cpp CLI/Coverage.cpp CLI/Repl.cpp REPL_CLI_OBJECTS=$(REPL_CLI_SOURCES:%=$(BUILD)/%.o) REPL_CLI_TARGET=$(BUILD)/luau @@ -128,10 +128,10 @@ luau-size: luau # executable target aliases luau: $(REPL_CLI_TARGET) - cp $^ $@ + ln -fs $^ $@ luau-analyze: $(ANALYZE_CLI_TARGET) - cp $^ $@ + ln -fs $^ $@ # executable targets $(TESTS_TARGET): $(TESTS_OBJECTS) $(ANALYSIS_TARGET) $(COMPILER_TARGET) $(AST_TARGET) $(VM_TARGET) diff --git a/Sources.cmake b/Sources.cmake index 57df9b91..14834b3a 100644 --- a/Sources.cmake +++ b/Sources.cmake @@ -133,6 +133,7 @@ target_sources(Luau.VM PRIVATE VM/src/ltable.cpp VM/src/ltablib.cpp VM/src/ltm.cpp + VM/src/ludata.cpp VM/src/lutf8lib.cpp VM/src/lvmexecute.cpp VM/src/lvmload.cpp @@ -152,12 +153,15 @@ target_sources(Luau.VM PRIVATE VM/src/lstring.h VM/src/ltable.h VM/src/ltm.h + VM/src/ludata.h VM/src/lvm.h ) if(TARGET Luau.Repl.CLI) # Luau.Repl.CLI Sources target_sources(Luau.Repl.CLI PRIVATE + CLI/Coverage.h + CLI/Coverage.cpp CLI/FileUtils.h CLI/FileUtils.cpp CLI/Profiler.h diff --git a/VM/include/lua.h b/VM/include/lua.h index 7078acd0..55902160 100644 --- a/VM/include/lua.h +++ b/VM/include/lua.h @@ -334,6 +334,10 @@ LUA_API const char* lua_setupvalue(lua_State* L, int funcindex, int n); LUA_API void lua_singlestep(lua_State* L, int enabled); LUA_API void lua_breakpoint(lua_State* L, int funcindex, int line, int enabled); +typedef void (*lua_Coverage)(void* context, const char* function, int linedefined, int depth, const int* hits, size_t size); + +LUA_API void lua_getcoverage(lua_State* L, int funcindex, void* context, lua_Coverage callback); + /* Warning: this function is not thread-safe since it stores the result in a shared global array! Only use for debugging. */ LUA_API const char* lua_debugtrace(lua_State* L); diff --git a/VM/src/lapi.cpp b/VM/src/lapi.cpp index 76043b9c..a65b0325 100644 --- a/VM/src/lapi.cpp +++ b/VM/src/lapi.cpp @@ -8,6 +8,7 @@ #include "lfunc.h" #include "lgc.h" #include "ldo.h" +#include "ludata.h" #include "lvm.h" #include "lnumutils.h" @@ -43,36 +44,30 @@ static Table* getcurrenv(lua_State* L) } } -static LUAU_NOINLINE TValue* index2adrslow(lua_State* L, int idx) +static LUAU_NOINLINE TValue* pseudo2addr(lua_State* L, int idx) { - api_check(L, idx <= 0); - if (idx > LUA_REGISTRYINDEX) + api_check(L, lua_ispseudo(idx)); + switch (idx) + { /* pseudo-indices */ + case LUA_REGISTRYINDEX: + return registry(L); + case LUA_ENVIRONINDEX: { - api_check(L, idx != 0 && -idx <= L->top - L->base); - return L->top + idx; + sethvalue(L, &L->env, getcurrenv(L)); + return &L->env; + } + case LUA_GLOBALSINDEX: + return gt(L); + default: + { + Closure* func = curr_func(L); + idx = LUA_GLOBALSINDEX - idx; + return (idx <= func->nupvalues) ? &func->c.upvals[idx - 1] : cast_to(TValue*, luaO_nilobject); + } } - else - switch (idx) - { /* pseudo-indices */ - case LUA_REGISTRYINDEX: - return registry(L); - case LUA_ENVIRONINDEX: - { - sethvalue(L, &L->env, getcurrenv(L)); - return &L->env; - } - case LUA_GLOBALSINDEX: - return gt(L); - default: - { - Closure* func = curr_func(L); - idx = LUA_GLOBALSINDEX - idx; - return (idx <= func->nupvalues) ? &func->c.upvals[idx - 1] : cast_to(TValue*, luaO_nilobject); - } - } } -static LUAU_FORCEINLINE TValue* index2adr(lua_State* L, int idx) +static LUAU_FORCEINLINE TValue* index2addr(lua_State* L, int idx) { if (idx > 0) { @@ -83,15 +78,20 @@ static LUAU_FORCEINLINE TValue* index2adr(lua_State* L, int idx) else return o; } + else if (idx > LUA_REGISTRYINDEX) + { + api_check(L, idx != 0 && -idx <= L->top - L->base); + return L->top + idx; + } else { - return index2adrslow(L, idx); + return pseudo2addr(L, idx); } } const TValue* luaA_toobject(lua_State* L, int idx) { - StkId p = index2adr(L, idx); + StkId p = index2addr(L, idx); return (p == luaO_nilobject) ? NULL : p; } @@ -145,7 +145,7 @@ void lua_xpush(lua_State* from, lua_State* to, int idx) { api_check(from, from->global == to->global); luaC_checkthreadsleep(to); - setobj2s(to, to->top, index2adr(from, idx)); + setobj2s(to, to->top, index2addr(from, idx)); api_incr_top(to); return; } @@ -202,7 +202,7 @@ void lua_settop(lua_State* L, int idx) void lua_remove(lua_State* L, int idx) { - StkId p = index2adr(L, idx); + StkId p = index2addr(L, idx); api_checkvalidindex(L, p); while (++p < L->top) setobjs2s(L, p - 1, p); @@ -213,7 +213,7 @@ void lua_remove(lua_State* L, int idx) void lua_insert(lua_State* L, int idx) { luaC_checkthreadsleep(L); - StkId p = index2adr(L, idx); + StkId p = index2addr(L, idx); api_checkvalidindex(L, p); for (StkId q = L->top; q > p; q--) setobjs2s(L, q, q - 1); @@ -228,7 +228,7 @@ void lua_replace(lua_State* L, int idx) luaG_runerror(L, "no calling environment"); api_checknelems(L, 1); luaC_checkthreadsleep(L); - StkId o = index2adr(L, idx); + StkId o = index2addr(L, idx); api_checkvalidindex(L, o); if (idx == LUA_ENVIRONINDEX) { @@ -250,7 +250,7 @@ void lua_replace(lua_State* L, int idx) void lua_pushvalue(lua_State* L, int idx) { luaC_checkthreadsleep(L); - StkId o = index2adr(L, idx); + StkId o = index2addr(L, idx); setobj2s(L, L->top, o); api_incr_top(L); return; @@ -262,7 +262,7 @@ void lua_pushvalue(lua_State* L, int idx) int lua_type(lua_State* L, int idx) { - StkId o = index2adr(L, idx); + StkId o = index2addr(L, idx); return (o == luaO_nilobject) ? LUA_TNONE : ttype(o); } @@ -273,20 +273,20 @@ const char* lua_typename(lua_State* L, int t) int lua_iscfunction(lua_State* L, int idx) { - StkId o = index2adr(L, idx); + StkId o = index2addr(L, idx); return iscfunction(o); } int lua_isLfunction(lua_State* L, int idx) { - StkId o = index2adr(L, idx); + StkId o = index2addr(L, idx); return isLfunction(o); } int lua_isnumber(lua_State* L, int idx) { TValue n; - const TValue* o = index2adr(L, idx); + const TValue* o = index2addr(L, idx); return tonumber(o, &n); } @@ -298,14 +298,14 @@ int lua_isstring(lua_State* L, int idx) int lua_isuserdata(lua_State* L, int idx) { - const TValue* o = index2adr(L, idx); + const TValue* o = index2addr(L, idx); return (ttisuserdata(o) || ttislightuserdata(o)); } int lua_rawequal(lua_State* L, int index1, int index2) { - StkId o1 = index2adr(L, index1); - StkId o2 = index2adr(L, index2); + StkId o1 = index2addr(L, index1); + StkId o2 = index2addr(L, index2); return (o1 == luaO_nilobject || o2 == luaO_nilobject) ? 0 : luaO_rawequalObj(o1, o2); } @@ -313,8 +313,8 @@ int lua_equal(lua_State* L, int index1, int index2) { StkId o1, o2; int i; - o1 = index2adr(L, index1); - o2 = index2adr(L, index2); + o1 = index2addr(L, index1); + o2 = index2addr(L, index2); i = (o1 == luaO_nilobject || o2 == luaO_nilobject) ? 0 : equalobj(L, o1, o2); return i; } @@ -323,8 +323,8 @@ int lua_lessthan(lua_State* L, int index1, int index2) { StkId o1, o2; int i; - o1 = index2adr(L, index1); - o2 = index2adr(L, index2); + o1 = index2addr(L, index1); + o2 = index2addr(L, index2); i = (o1 == luaO_nilobject || o2 == luaO_nilobject) ? 0 : luaV_lessthan(L, o1, o2); return i; } @@ -332,7 +332,7 @@ int lua_lessthan(lua_State* L, int index1, int index2) double lua_tonumberx(lua_State* L, int idx, int* isnum) { TValue n; - const TValue* o = index2adr(L, idx); + const TValue* o = index2addr(L, idx); if (tonumber(o, &n)) { if (isnum) @@ -350,7 +350,7 @@ double lua_tonumberx(lua_State* L, int idx, int* isnum) int lua_tointegerx(lua_State* L, int idx, int* isnum) { TValue n; - const TValue* o = index2adr(L, idx); + const TValue* o = index2addr(L, idx); if (tonumber(o, &n)) { int res; @@ -371,7 +371,7 @@ int lua_tointegerx(lua_State* L, int idx, int* isnum) unsigned lua_tounsignedx(lua_State* L, int idx, int* isnum) { TValue n; - const TValue* o = index2adr(L, idx); + const TValue* o = index2addr(L, idx); if (tonumber(o, &n)) { unsigned res; @@ -391,13 +391,13 @@ unsigned lua_tounsignedx(lua_State* L, int idx, int* isnum) int lua_toboolean(lua_State* L, int idx) { - const TValue* o = index2adr(L, idx); + const TValue* o = index2addr(L, idx); return !l_isfalse(o); } const char* lua_tolstring(lua_State* L, int idx, size_t* len) { - StkId o = index2adr(L, idx); + StkId o = index2addr(L, idx); if (!ttisstring(o)) { luaC_checkthreadsleep(L); @@ -408,7 +408,7 @@ const char* lua_tolstring(lua_State* L, int idx, size_t* len) return NULL; } luaC_checkGC(L); - o = index2adr(L, idx); /* previous call may reallocate the stack */ + o = index2addr(L, idx); /* previous call may reallocate the stack */ } if (len != NULL) *len = tsvalue(o)->len; @@ -417,7 +417,7 @@ const char* lua_tolstring(lua_State* L, int idx, size_t* len) const char* lua_tostringatom(lua_State* L, int idx, int* atom) { - StkId o = index2adr(L, idx); + StkId o = index2addr(L, idx); if (!ttisstring(o)) return NULL; const TString* s = tsvalue(o); @@ -438,7 +438,7 @@ const char* lua_namecallatom(lua_State* L, int* atom) const float* lua_tovector(lua_State* L, int idx) { - StkId o = index2adr(L, idx); + StkId o = index2addr(L, idx); if (!ttisvector(o)) { return NULL; @@ -448,7 +448,7 @@ const float* lua_tovector(lua_State* L, int idx) int lua_objlen(lua_State* L, int idx) { - StkId o = index2adr(L, idx); + StkId o = index2addr(L, idx); switch (ttype(o)) { case LUA_TSTRING: @@ -469,13 +469,13 @@ int lua_objlen(lua_State* L, int idx) lua_CFunction lua_tocfunction(lua_State* L, int idx) { - StkId o = index2adr(L, idx); + StkId o = index2addr(L, idx); return (!iscfunction(o)) ? NULL : cast_to(lua_CFunction, clvalue(o)->c.f); } void* lua_touserdata(lua_State* L, int idx) { - StkId o = index2adr(L, idx); + StkId o = index2addr(L, idx); switch (ttype(o)) { case LUA_TUSERDATA: @@ -489,13 +489,13 @@ void* lua_touserdata(lua_State* L, int idx) void* lua_touserdatatagged(lua_State* L, int idx, int tag) { - StkId o = index2adr(L, idx); + StkId o = index2addr(L, idx); return (ttisuserdata(o) && uvalue(o)->tag == tag) ? uvalue(o)->data : NULL; } int lua_userdatatag(lua_State* L, int idx) { - StkId o = index2adr(L, idx); + StkId o = index2addr(L, idx); if (ttisuserdata(o)) return uvalue(o)->tag; return -1; @@ -503,13 +503,13 @@ int lua_userdatatag(lua_State* L, int idx) lua_State* lua_tothread(lua_State* L, int idx) { - StkId o = index2adr(L, idx); + StkId o = index2addr(L, idx); return (!ttisthread(o)) ? NULL : thvalue(o); } const void* lua_topointer(lua_State* L, int idx) { - StkId o = index2adr(L, idx); + StkId o = index2addr(L, idx); switch (ttype(o)) { case LUA_TTABLE: @@ -657,7 +657,7 @@ int lua_pushthread(lua_State* L) void lua_gettable(lua_State* L, int idx) { luaC_checkthreadsleep(L); - StkId t = index2adr(L, idx); + StkId t = index2addr(L, idx); api_checkvalidindex(L, t); luaV_gettable(L, t, L->top - 1, L->top - 1); return; @@ -666,7 +666,7 @@ void lua_gettable(lua_State* L, int idx) void lua_getfield(lua_State* L, int idx, const char* k) { luaC_checkthreadsleep(L); - StkId t = index2adr(L, idx); + StkId t = index2addr(L, idx); api_checkvalidindex(L, t); TValue key; setsvalue(L, &key, luaS_new(L, k)); @@ -678,7 +678,7 @@ void lua_getfield(lua_State* L, int idx, const char* k) void lua_rawgetfield(lua_State* L, int idx, const char* k) { luaC_checkthreadsleep(L); - StkId t = index2adr(L, idx); + StkId t = index2addr(L, idx); api_check(L, ttistable(t)); TValue key; setsvalue(L, &key, luaS_new(L, k)); @@ -690,7 +690,7 @@ void lua_rawgetfield(lua_State* L, int idx, const char* k) void lua_rawget(lua_State* L, int idx) { luaC_checkthreadsleep(L); - StkId t = index2adr(L, idx); + StkId t = index2addr(L, idx); api_check(L, ttistable(t)); setobj2s(L, L->top - 1, luaH_get(hvalue(t), L->top - 1)); return; @@ -699,7 +699,7 @@ void lua_rawget(lua_State* L, int idx) void lua_rawgeti(lua_State* L, int idx, int n) { luaC_checkthreadsleep(L); - StkId t = index2adr(L, idx); + StkId t = index2addr(L, idx); api_check(L, ttistable(t)); setobj2s(L, L->top, luaH_getnum(hvalue(t), n)); api_incr_top(L); @@ -717,7 +717,7 @@ void lua_createtable(lua_State* L, int narray, int nrec) void lua_setreadonly(lua_State* L, int objindex, int enabled) { - const TValue* o = index2adr(L, objindex); + const TValue* o = index2addr(L, objindex); api_check(L, ttistable(o)); Table* t = hvalue(o); api_check(L, t != hvalue(registry(L))); @@ -727,7 +727,7 @@ void lua_setreadonly(lua_State* L, int objindex, int enabled) int lua_getreadonly(lua_State* L, int objindex) { - const TValue* o = index2adr(L, objindex); + const TValue* o = index2addr(L, objindex); api_check(L, ttistable(o)); Table* t = hvalue(o); int res = t->readonly; @@ -736,7 +736,7 @@ int lua_getreadonly(lua_State* L, int objindex) void lua_setsafeenv(lua_State* L, int objindex, int enabled) { - const TValue* o = index2adr(L, objindex); + const TValue* o = index2addr(L, objindex); api_check(L, ttistable(o)); Table* t = hvalue(o); t->safeenv = bool(enabled); @@ -748,7 +748,7 @@ int lua_getmetatable(lua_State* L, int objindex) const TValue* obj; Table* mt = NULL; int res; - obj = index2adr(L, objindex); + obj = index2addr(L, objindex); switch (ttype(obj)) { case LUA_TTABLE: @@ -775,7 +775,7 @@ int lua_getmetatable(lua_State* L, int objindex) void lua_getfenv(lua_State* L, int idx) { StkId o; - o = index2adr(L, idx); + o = index2addr(L, idx); api_checkvalidindex(L, o); switch (ttype(o)) { @@ -801,7 +801,7 @@ void lua_settable(lua_State* L, int idx) { StkId t; api_checknelems(L, 2); - t = index2adr(L, idx); + t = index2addr(L, idx); api_checkvalidindex(L, t); luaV_settable(L, t, L->top - 2, L->top - 1); L->top -= 2; /* pop index and value */ @@ -813,7 +813,7 @@ void lua_setfield(lua_State* L, int idx, const char* k) StkId t; TValue key; api_checknelems(L, 1); - t = index2adr(L, idx); + t = index2addr(L, idx); api_checkvalidindex(L, t); setsvalue(L, &key, luaS_new(L, k)); luaV_settable(L, t, &key, L->top - 1); @@ -825,7 +825,7 @@ void lua_rawset(lua_State* L, int idx) { StkId t; api_checknelems(L, 2); - t = index2adr(L, idx); + t = index2addr(L, idx); api_check(L, ttistable(t)); if (hvalue(t)->readonly) luaG_runerror(L, "Attempt to modify a readonly table"); @@ -839,7 +839,7 @@ void lua_rawseti(lua_State* L, int idx, int n) { StkId o; api_checknelems(L, 1); - o = index2adr(L, idx); + o = index2addr(L, idx); api_check(L, ttistable(o)); if (hvalue(o)->readonly) luaG_runerror(L, "Attempt to modify a readonly table"); @@ -854,7 +854,7 @@ int lua_setmetatable(lua_State* L, int objindex) TValue* obj; Table* mt; api_checknelems(L, 1); - obj = index2adr(L, objindex); + obj = index2addr(L, objindex); api_checkvalidindex(L, obj); if (ttisnil(L->top - 1)) mt = NULL; @@ -896,7 +896,7 @@ int lua_setfenv(lua_State* L, int idx) StkId o; int res = 1; api_checknelems(L, 1); - o = index2adr(L, idx); + o = index2addr(L, idx); api_checkvalidindex(L, o); api_check(L, ttistable(L->top - 1)); switch (ttype(o)) @@ -987,7 +987,7 @@ int lua_pcall(lua_State* L, int nargs, int nresults, int errfunc) func = 0; else { - StkId o = index2adr(L, errfunc); + StkId o = index2addr(L, errfunc); api_checkvalidindex(L, o); func = savestack(L, o); } @@ -1150,7 +1150,7 @@ l_noret lua_error(lua_State* L) int lua_next(lua_State* L, int idx) { luaC_checkthreadsleep(L); - StkId t = index2adr(L, idx); + StkId t = index2addr(L, idx); api_check(L, ttistable(t)); int more = luaH_next(L, hvalue(t), L->top - 1); if (more) @@ -1187,7 +1187,7 @@ void* lua_newuserdatatagged(lua_State* L, size_t sz, int tag) api_check(L, unsigned(tag) < LUA_UTAG_LIMIT); luaC_checkGC(L); luaC_checkthreadsleep(L); - Udata* u = luaS_newudata(L, sz, tag); + Udata* u = luaU_newudata(L, sz, tag); setuvalue(L, L->top, u); api_incr_top(L); return u->data; @@ -1197,7 +1197,7 @@ void* lua_newuserdatadtor(lua_State* L, size_t sz, void (*dtor)(void*)) { luaC_checkGC(L); luaC_checkthreadsleep(L); - Udata* u = luaS_newudata(L, sz + sizeof(dtor), UTAG_IDTOR); + Udata* u = luaU_newudata(L, sz + sizeof(dtor), UTAG_IDTOR); memcpy(&u->data + sz, &dtor, sizeof(dtor)); setuvalue(L, L->top, u); api_incr_top(L); @@ -1232,7 +1232,7 @@ const char* lua_getupvalue(lua_State* L, int funcindex, int n) { luaC_checkthreadsleep(L); TValue* val; - const char* name = aux_upvalue(index2adr(L, funcindex), n, &val); + const char* name = aux_upvalue(index2addr(L, funcindex), n, &val); if (name) { setobj2s(L, L->top, val); @@ -1246,7 +1246,7 @@ const char* lua_setupvalue(lua_State* L, int funcindex, int n) const char* name; TValue* val; StkId fi; - fi = index2adr(L, funcindex); + fi = index2addr(L, funcindex); api_checknelems(L, 1); name = aux_upvalue(fi, n, &val); if (name) @@ -1270,7 +1270,7 @@ int lua_ref(lua_State* L, int idx) api_check(L, idx != LUA_REGISTRYINDEX); /* idx is a stack index for value */ int ref = LUA_REFNIL; global_State* g = L->global; - StkId p = index2adr(L, idx); + StkId p = index2addr(L, idx); if (!ttisnil(p)) { Table* reg = hvalue(registry(L)); diff --git a/VM/src/ldebug.cpp b/VM/src/ldebug.cpp index d77f84ef..9fe1885f 100644 --- a/VM/src/ldebug.cpp +++ b/VM/src/ldebug.cpp @@ -370,6 +370,69 @@ void lua_breakpoint(lua_State* L, int funcindex, int line, int enabled) luaG_breakpoint(L, clvalue(func)->l.p, line, bool(enabled)); } +static int getmaxline(Proto* p) +{ + int result = -1; + + for (int i = 0; i < p->sizecode; ++i) + { + int line = luaG_getline(p, i); + result = result < line ? line : result; + } + + for (int i = 0; i < p->sizep; ++i) + { + int psize = getmaxline(p->p[i]); + result = result < psize ? psize : result; + } + + return result; +} + +static void getcoverage(Proto* p, int depth, int* buffer, size_t size, void* context, lua_Coverage callback) +{ + memset(buffer, -1, size * sizeof(int)); + + for (int i = 0; i < p->sizecode; ++i) + { + Instruction insn = p->code[i]; + if (LUAU_INSN_OP(insn) != LOP_COVERAGE) + continue; + + int line = luaG_getline(p, i); + int hits = LUAU_INSN_E(insn); + + LUAU_ASSERT(size_t(line) < size); + buffer[line] = buffer[line] < hits ? hits : buffer[line]; + } + + const char* debugname = p->debugname ? getstr(p->debugname) : NULL; + int linedefined = luaG_getline(p, 0); + + callback(context, debugname, linedefined, depth, buffer, size); + + for (int i = 0; i < p->sizep; ++i) + getcoverage(p->p[i], depth + 1, buffer, size, context, callback); +} + +void lua_getcoverage(lua_State* L, int funcindex, void* context, lua_Coverage callback) +{ + const TValue* func = luaA_toobject(L, funcindex); + api_check(L, ttisfunction(func) && !clvalue(func)->isC); + + Proto* p = clvalue(func)->l.p; + + size_t size = getmaxline(p) + 1; + if (size == 0) + return; + + int* buffer = luaM_newarray(L, size, int, 0); + + getcoverage(p, 0, buffer, size, context, callback); + + luaM_freearray(L, buffer, size, int, 0); +} + static size_t append(char* buf, size_t bufsize, size_t offset, const char* data) { size_t size = strlen(data); diff --git a/VM/src/lgc.cpp b/VM/src/lgc.cpp index ab416041..7393fc74 100644 --- a/VM/src/lgc.cpp +++ b/VM/src/lgc.cpp @@ -8,11 +8,10 @@ #include "lfunc.h" #include "lstring.h" #include "ldo.h" +#include "ludata.h" #include -LUAU_FASTFLAGVARIABLE(LuauSeparateAtomic, false) - LUAU_FASTFLAG(LuauArrayBoundary) #define GC_SWEEPMAX 40 @@ -59,10 +58,6 @@ static void recordGcStateTime(global_State* g, int startgcstate, double seconds, case GCSpropagate: case GCSpropagateagain: g->gcstats.currcycle.marktime += seconds; - - // atomic step had to be performed during the switch and it's tracked separately - if (!FFlag::LuauSeparateAtomic && g->gcstate == GCSsweepstring) - g->gcstats.currcycle.marktime -= g->gcstats.currcycle.atomictime; break; case GCSatomic: g->gcstats.currcycle.atomictime += seconds; @@ -488,7 +483,7 @@ static void freeobj(lua_State* L, GCObject* o) luaS_free(L, gco2ts(o)); break; case LUA_TUSERDATA: - luaS_freeudata(L, gco2u(o)); + luaU_freeudata(L, gco2u(o)); break; default: LUAU_ASSERT(0); @@ -632,17 +627,9 @@ static size_t remarkupvals(global_State* g) static size_t atomic(lua_State* L) { global_State* g = L->global; + LUAU_ASSERT(g->gcstate == GCSatomic); + size_t work = 0; - - if (FFlag::LuauSeparateAtomic) - { - LUAU_ASSERT(g->gcstate == GCSatomic); - } - else - { - g->gcstate = GCSatomic; - } - /* remark occasional upvalues of (maybe) dead threads */ work += remarkupvals(g); /* traverse objects caught by write barrier and by 'remarkupvals' */ @@ -666,11 +653,6 @@ static size_t atomic(lua_State* L) g->sweepgc = &g->rootgc; g->gcstate = GCSsweepstring; - if (!FFlag::LuauSeparateAtomic) - { - GC_INTERRUPT(GCSatomic); - } - return work; } @@ -716,22 +698,7 @@ static size_t gcstep(lua_State* L, size_t limit) if (!g->gray) /* no more `gray' objects */ { - if (FFlag::LuauSeparateAtomic) - { - g->gcstate = GCSatomic; - } - else - { - double starttimestamp = lua_clock(); - - g->gcstats.currcycle.atomicstarttimestamp = starttimestamp; - g->gcstats.currcycle.atomicstarttotalsizebytes = g->totalbytes; - - atomic(L); /* finish mark phase */ - LUAU_ASSERT(g->gcstate == GCSsweepstring); - - g->gcstats.currcycle.atomictime += lua_clock() - starttimestamp; - } + g->gcstate = GCSatomic; } break; } @@ -853,7 +820,7 @@ static size_t getheaptrigger(global_State* g, size_t heapgoal) void luaC_step(lua_State* L, bool assist) { global_State* g = L->global; - ptrdiff_t lim = (g->gcstepsize / 100) * g->gcstepmul; /* how much to work */ + int lim = (g->gcstepsize / 100) * g->gcstepmul; /* how much to work */ LUAU_ASSERT(g->totalbytes >= g->GCthreshold); size_t debt = g->totalbytes - g->GCthreshold; @@ -908,7 +875,7 @@ void luaC_fullgc(lua_State* L) if (g->gcstate == GCSpause) startGcCycleStats(g); - if (g->gcstate <= (FFlag::LuauSeparateAtomic ? GCSatomic : GCSpropagateagain)) + if (g->gcstate <= GCSatomic) { /* reset sweep marks to sweep all elements (returning them to white) */ g->sweepstrgc = 0; @@ -1049,7 +1016,7 @@ int64_t luaC_allocationrate(lua_State* L) global_State* g = L->global; const double durationthreshold = 1e-3; // avoid measuring intervals smaller than 1ms - if (g->gcstate <= (FFlag::LuauSeparateAtomic ? GCSatomic : GCSpropagateagain)) + if (g->gcstate <= GCSatomic) { double duration = lua_clock() - g->gcstats.lastcycle.endtimestamp; diff --git a/VM/src/lgcdebug.cpp b/VM/src/lgcdebug.cpp index a79e7b95..f6f7a878 100644 --- a/VM/src/lgcdebug.cpp +++ b/VM/src/lgcdebug.cpp @@ -7,6 +7,7 @@ #include "ltable.h" #include "lfunc.h" #include "lstring.h" +#include "ludata.h" #include #include diff --git a/VM/src/lobject.h b/VM/src/lobject.h index ba040af6..fd0a15b7 100644 --- a/VM/src/lobject.h +++ b/VM/src/lobject.h @@ -78,15 +78,7 @@ typedef struct lua_TValue #define thvalue(o) check_exp(ttisthread(o), &(o)->value.gc->th) #define upvalue(o) check_exp(ttisupval(o), &(o)->value.gc->uv) -// beware bit magic: a value is false if it's nil or boolean false -// baseline implementation: (ttisnil(o) || (ttisboolean(o) && bvalue(o) == 0)) -// we'd like a branchless version of this which helps with performance, and a very fast version -// so our strategy is to always read the boolean value (not using bvalue(o) because that asserts when type isn't boolean) -// we then combine it with type to produce 0/1 as follows: -// - when type is nil (0), & makes the result 0 -// - when type is boolean (1), we effectively only look at the bottom bit, so result is 0 iff boolean value is 0 -// - when type is different, it must have some of the top bits set - we keep all top bits of boolean value so the result is non-0 -#define l_isfalse(o) (!(((o)->value.b | ~1) & ttype(o))) +#define l_isfalse(o) (ttisnil(o) || (ttisboolean(o) && bvalue(o) == 0)) /* ** for internal debug only diff --git a/VM/src/lstring.cpp b/VM/src/lstring.cpp index 18ee1cda..a9e90d17 100644 --- a/VM/src/lstring.cpp +++ b/VM/src/lstring.cpp @@ -206,32 +206,3 @@ void luaS_free(lua_State* L, TString* ts) L->global->strt.nuse--; luaM_free(L, ts, sizestring(ts->len), ts->memcat); } - -Udata* luaS_newudata(lua_State* L, size_t s, int tag) -{ - if (s > INT_MAX - sizeof(Udata)) - luaM_toobig(L); - Udata* u = luaM_new(L, Udata, sizeudata(s), L->activememcat); - luaC_link(L, u, LUA_TUSERDATA); - u->len = int(s); - u->metatable = NULL; - LUAU_ASSERT(tag >= 0 && tag <= 255); - u->tag = uint8_t(tag); - return u; -} - -void luaS_freeudata(lua_State* L, Udata* u) -{ - LUAU_ASSERT(u->tag < LUA_UTAG_LIMIT || u->tag == UTAG_IDTOR); - - void (*dtor)(void*) = nullptr; - if (u->tag == UTAG_IDTOR) - memcpy(&dtor, &u->data + u->len - sizeof(dtor), sizeof(dtor)); - else if (u->tag) - dtor = L->global->udatagc[u->tag]; - - if (dtor) - dtor(u->data); - - luaM_free(L, u, sizeudata(u->len), u->memcat); -} diff --git a/VM/src/lstring.h b/VM/src/lstring.h index 612da28d..3fd0bd39 100644 --- a/VM/src/lstring.h +++ b/VM/src/lstring.h @@ -8,11 +8,7 @@ /* string size limit */ #define MAXSSIZE (1 << 30) -/* special tag value is used for user data with inline dtors */ -#define UTAG_IDTOR LUA_UTAG_LIMIT - #define sizestring(len) (offsetof(TString, data) + len + 1) -#define sizeudata(len) (offsetof(Udata, data) + len) #define luaS_new(L, s) (luaS_newlstr(L, s, strlen(s))) #define luaS_newliteral(L, s) (luaS_newlstr(L, "" s, (sizeof(s) / sizeof(char)) - 1)) @@ -26,8 +22,5 @@ LUAI_FUNC void luaS_resize(lua_State* L, int newsize); LUAI_FUNC TString* luaS_newlstr(lua_State* L, const char* str, size_t l); LUAI_FUNC void luaS_free(lua_State* L, TString* ts); -LUAI_FUNC Udata* luaS_newudata(lua_State* L, size_t s, int tag); -LUAI_FUNC void luaS_freeudata(lua_State* L, Udata* u); - LUAI_FUNC TString* luaS_bufstart(lua_State* L, size_t size); LUAI_FUNC TString* luaS_buffinish(lua_State* L, TString* ts); diff --git a/VM/src/ludata.cpp b/VM/src/ludata.cpp new file mode 100644 index 00000000..d180c388 --- /dev/null +++ b/VM/src/ludata.cpp @@ -0,0 +1,37 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +// This code is based on Lua 5.x implementation licensed under MIT License; see lua_LICENSE.txt for details +#include "ludata.h" + +#include "lgc.h" +#include "lmem.h" + +#include + +Udata* luaU_newudata(lua_State* L, size_t s, int tag) +{ + if (s > INT_MAX - sizeof(Udata)) + luaM_toobig(L); + Udata* u = luaM_new(L, Udata, sizeudata(s), L->activememcat); + luaC_link(L, u, LUA_TUSERDATA); + u->len = int(s); + u->metatable = NULL; + LUAU_ASSERT(tag >= 0 && tag <= 255); + u->tag = uint8_t(tag); + return u; +} + +void luaU_freeudata(lua_State* L, Udata* u) +{ + LUAU_ASSERT(u->tag < LUA_UTAG_LIMIT || u->tag == UTAG_IDTOR); + + void (*dtor)(void*) = nullptr; + if (u->tag == UTAG_IDTOR) + memcpy(&dtor, &u->data + u->len - sizeof(dtor), sizeof(dtor)); + else if (u->tag) + dtor = L->global->udatagc[u->tag]; + + if (dtor) + dtor(u->data); + + luaM_free(L, u, sizeudata(u->len), u->memcat); +} diff --git a/VM/src/ludata.h b/VM/src/ludata.h new file mode 100644 index 00000000..59cb85bd --- /dev/null +++ b/VM/src/ludata.h @@ -0,0 +1,13 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +// This code is based on Lua 5.x implementation licensed under MIT License; see lua_LICENSE.txt for details +#pragma once + +#include "lobject.h" + +/* special tag value is used for user data with inline dtors */ +#define UTAG_IDTOR LUA_UTAG_LIMIT + +#define sizeudata(len) (offsetof(Udata, data) + len) + +LUAI_FUNC Udata* luaU_newudata(lua_State* L, size_t s, int tag); +LUAI_FUNC void luaU_freeudata(lua_State* L, Udata* u); diff --git a/VM/src/lvmexecute.cpp b/VM/src/lvmexecute.cpp index bf8d493e..cebeeb58 100644 --- a/VM/src/lvmexecute.cpp +++ b/VM/src/lvmexecute.cpp @@ -63,7 +63,8 @@ #define VM_KV(i) (LUAU_ASSERT(unsigned(i) < unsigned(cl->l.p->sizek)), &k[i]) #define VM_UV(i) (LUAU_ASSERT(unsigned(i) < unsigned(cl->nupvalues)), &cl->l.uprefs[i]) -#define VM_PATCH_C(pc, slot) ((uint8_t*)(pc))[3] = uint8_t(slot) +#define VM_PATCH_C(pc, slot) *const_cast(pc) = ((uint8_t(slot) << 24) | (0x00ffffffu & *(pc))) +#define VM_PATCH_E(pc, slot) *const_cast(pc) = ((uint32_t(slot) << 8) | (0x000000ffu & *(pc))) // NOTE: If debugging the Luau code, disable this macro to prevent timeouts from // occurring when tracing code in Visual Studio / XCode @@ -120,7 +121,7 @@ */ #if VM_USE_CGOTO #define VM_CASE(op) CASE_##op: -#define VM_NEXT() goto*(SingleStep ? &&dispatch : kDispatchTable[*(uint8_t*)pc]) +#define VM_NEXT() goto*(SingleStep ? &&dispatch : kDispatchTable[LUAU_INSN_OP(*pc)]) #define VM_CONTINUE(op) goto* kDispatchTable[uint8_t(op)] #else #define VM_CASE(op) case op: @@ -325,7 +326,7 @@ static void luau_execute(lua_State* L) // ... and singlestep logic :) if (SingleStep) { - if (L->global->cb.debugstep && !luau_skipstep(*(uint8_t*)pc)) + if (L->global->cb.debugstep && !luau_skipstep(LUAU_INSN_OP(*pc))) { VM_PROTECT(luau_callhook(L, L->global->cb.debugstep, NULL)); @@ -335,13 +336,12 @@ static void luau_execute(lua_State* L) } #if VM_USE_CGOTO - VM_CONTINUE(*(uint8_t*)pc); + VM_CONTINUE(LUAU_INSN_OP(*pc)); #endif } #if !VM_USE_CGOTO - // Note: this assumes that LUAU_INSN_OP() decodes the first byte (aka least significant byte in the little endian encoding) - size_t dispatchOp = *(uint8_t*)pc; + size_t dispatchOp = LUAU_INSN_OP(*pc); dispatchContinue: switch (dispatchOp) @@ -2577,7 +2577,7 @@ static void luau_execute(lua_State* L) // update hits with saturated add and patch the instruction in place hits = (hits < (1 << 23) - 1) ? hits + 1 : hits; - ((uint32_t*)pc)[-1] = LOP_COVERAGE | (uint32_t(hits) << 8); + VM_PATCH_E(pc - 1, hits); VM_NEXT(); } diff --git a/VM/src/lvmutils.cpp b/VM/src/lvmutils.cpp index 740a4cfd..5d802277 100644 --- a/VM/src/lvmutils.cpp +++ b/VM/src/lvmutils.cpp @@ -53,7 +53,7 @@ const float* luaV_tovector(const TValue* obj) static void callTMres(lua_State* L, StkId res, const TValue* f, const TValue* p1, const TValue* p2) { ptrdiff_t result = savestack(L, res); - // RBOLOX: using stack room beyond top is technically safe here, but for very complicated reasons: + // using stack room beyond top is technically safe here, but for very complicated reasons: // * The stack guarantees 1 + EXTRA_STACK room beyond stack_last (see luaD_reallocstack) will be allocated // * we cannot move luaD_checkstack above because the arguments are *sometimes* pointers to the lua // stack and checkstack may invalidate those pointers @@ -74,7 +74,7 @@ static void callTMres(lua_State* L, StkId res, const TValue* f, const TValue* p1 static void callTM(lua_State* L, const TValue* f, const TValue* p1, const TValue* p2, const TValue* p3) { - // RBOLOX: using stack room beyond top is technically safe here, but for very complicated reasons: + // using stack room beyond top is technically safe here, but for very complicated reasons: // * The stack guarantees 1 + EXTRA_STACK room beyond stack_last (see luaD_reallocstack) will be allocated // * we cannot move luaD_checkstack above because the arguments are *sometimes* pointers to the lua // stack and checkstack may invalidate those pointers diff --git a/bench/tests/sunspider/3d-raytrace.lua b/bench/tests/sunspider/3d-raytrace.lua index 60e4f61e..c8f6b5dc 100644 --- a/bench/tests/sunspider/3d-raytrace.lua +++ b/bench/tests/sunspider/3d-raytrace.lua @@ -451,15 +451,16 @@ function raytraceScene() end function arrayToCanvasCommands(pixels) - local s = 'Test\nvar pixels = ['; + local s = {}; + table.insert(s, 'Test\nvar pixels = ['); for y = 0,size-1 do - s = s .. "["; + table.insert(s, "["); for x = 0,size-1 do - s = s .. "[" .. math.floor(pixels[y + 1][x + 1][1] * 255) .. "," .. math.floor(pixels[y + 1][x + 1][2] * 255) .. "," .. math.floor(pixels[y + 1][x + 1][3] * 255) .. "],"; + table.insert(s, "[" .. math.floor(pixels[y + 1][x + 1][1] * 255) .. "," .. math.floor(pixels[y + 1][x + 1][2] * 255) .. "," .. math.floor(pixels[y + 1][x + 1][3] * 255) .. "],"); end - s = s .. "],"; + table.insert(s, "],"); end - s = s .. '];\n var canvas = document.getElementById("renderCanvas").getContext("2d");\n\ + table.insert(s, '];\n var canvas = document.getElementById("renderCanvas").getContext("2d");\n\ \n\ \n\ var size = ' .. size .. ';\n\ @@ -479,9 +480,9 @@ for (var y = 0; y < size; y++) {\n\ canvas.setFillColor(l[0], l[1], l[2], 1);\n\ canvas.fillRect(x, y, 1, 1);\n\ }\n\ -}'; +}'); - return s; + return table.concat(s); end testOutput = arrayToCanvasCommands(raytraceScene()); diff --git a/tests/Autocomplete.test.cpp b/tests/Autocomplete.test.cpp index 3b74a99e..62a9999b 100644 --- a/tests/Autocomplete.test.cpp +++ b/tests/Autocomplete.test.cpp @@ -513,8 +513,6 @@ TEST_CASE_FIXTURE(ACFixture, "dont_offer_any_suggestions_from_the_end_of_a_comme TEST_CASE_FIXTURE(ACFixture, "dont_offer_any_suggestions_from_within_a_broken_comment") { - ScopedFastFlag sff{"LuauCaptureBrokenCommentSpans", true}; - check(R"( --[[ @1 )"); @@ -526,8 +524,6 @@ TEST_CASE_FIXTURE(ACFixture, "dont_offer_any_suggestions_from_within_a_broken_co TEST_CASE_FIXTURE(ACFixture, "dont_offer_any_suggestions_from_within_a_broken_comment_at_the_very_end_of_the_file") { - ScopedFastFlag sff{"LuauCaptureBrokenCommentSpans", true}; - check("--[[@1"); auto ac = autocomplete('1'); @@ -2625,4 +2621,55 @@ local a: A<(number, s@1> CHECK(ac.entryMap.count("string")); } +TEST_CASE_FIXTURE(ACFixture, "autocomplete_first_function_arg_expected_type") +{ + ScopedFastFlag luauAutocompleteAvoidMutation("LuauAutocompleteAvoidMutation", true); + ScopedFastFlag luauAutocompleteFirstArg("LuauAutocompleteFirstArg", true); + + check(R"( +local function foo1() return 1 end +local function foo2() return "1" end + +local function bar0() return "got" .. a end +local function bar1(a: number) return "got " .. a end +local function bar2(a: number, b: string) return "got " .. a .. b end + +local t = {} +function t:bar1(a: number) return "got " .. a end + +local r1 = bar0(@1) +local r2 = bar1(@2) +local r3 = bar2(@3) +local r4 = t:bar1(@4) + )"); + + auto ac = autocomplete('1'); + + REQUIRE(ac.entryMap.count("foo1")); + CHECK(ac.entryMap["foo1"].typeCorrect == TypeCorrectKind::None); + REQUIRE(ac.entryMap.count("foo2")); + CHECK(ac.entryMap["foo2"].typeCorrect == TypeCorrectKind::None); + + ac = autocomplete('2'); + + REQUIRE(ac.entryMap.count("foo1")); + CHECK(ac.entryMap["foo1"].typeCorrect == TypeCorrectKind::CorrectFunctionResult); + REQUIRE(ac.entryMap.count("foo2")); + CHECK(ac.entryMap["foo2"].typeCorrect == TypeCorrectKind::None); + + ac = autocomplete('3'); + + REQUIRE(ac.entryMap.count("foo1")); + CHECK(ac.entryMap["foo1"].typeCorrect == TypeCorrectKind::CorrectFunctionResult); + REQUIRE(ac.entryMap.count("foo2")); + CHECK(ac.entryMap["foo2"].typeCorrect == TypeCorrectKind::None); + + ac = autocomplete('4'); + + REQUIRE(ac.entryMap.count("foo1")); + CHECK(ac.entryMap["foo1"].typeCorrect == TypeCorrectKind::CorrectFunctionResult); + REQUIRE(ac.entryMap.count("foo2")); + CHECK(ac.entryMap["foo2"].typeCorrect == TypeCorrectKind::None); +} + TEST_SUITE_END(); diff --git a/tests/Compiler.test.cpp b/tests/Compiler.test.cpp index 6ba39ada..95811b3f 100644 --- a/tests/Compiler.test.cpp +++ b/tests/Compiler.test.cpp @@ -10,9 +10,6 @@ #include #include -LUAU_FASTFLAG(LuauPreloadClosures) -LUAU_FASTFLAG(LuauGenericSpecialGlobals) - using namespace Luau; static std::string compileFunction(const char* source, uint32_t id) @@ -74,20 +71,10 @@ TEST_CASE("BasicFunction") bcb.setDumpFlags(Luau::BytecodeBuilder::Dump_Code); Luau::compileOrThrow(bcb, "local function foo(a, b) return b end"); - if (FFlag::LuauPreloadClosures) - { - CHECK_EQ("\n" + bcb.dumpFunction(1), R"( + CHECK_EQ("\n" + bcb.dumpFunction(1), R"( DUPCLOSURE R0 K0 RETURN R0 0 )"); - } - else - { - CHECK_EQ("\n" + bcb.dumpFunction(1), R"( -NEWCLOSURE R0 P0 -RETURN R0 0 -)"); - } CHECK_EQ("\n" + bcb.dumpFunction(0), R"( RETURN R1 1 @@ -2859,47 +2846,35 @@ CAPTURE UPVAL U1 RETURN R0 1 )"); - if (FFlag::LuauPreloadClosures) - { - // recursive capture - CHECK_EQ("\n" + compileFunction("local function foo() return foo() end", 1), R"( + // recursive capture + CHECK_EQ("\n" + compileFunction("local function foo() return foo() end", 1), R"( DUPCLOSURE R0 K0 CAPTURE VAL R0 RETURN R0 0 )"); - // multi-level recursive capture - CHECK_EQ("\n" + compileFunction("local function foo() return function() return foo() end end", 1), R"( + // multi-level recursive capture + CHECK_EQ("\n" + compileFunction("local function foo() return function() return foo() end end", 1), R"( DUPCLOSURE R0 K0 CAPTURE UPVAL U0 RETURN R0 1 )"); - // multi-level recursive capture where function isn't top-level - // note: this should probably be optimized to DUPCLOSURE but doing that requires a different upval tracking flow in the compiler - CHECK_EQ("\n" + compileFunction(R"( + // multi-level recursive capture where function isn't top-level + // note: this should probably be optimized to DUPCLOSURE but doing that requires a different upval tracking flow in the compiler + CHECK_EQ("\n" + compileFunction(R"( local function foo() local function bar() return function() return bar() end end end )", - 1), - R"( + 1), + R"( NEWCLOSURE R0 P0 CAPTURE UPVAL U0 RETURN R0 1 )"); - } - else - { - // recursive capture - CHECK_EQ("\n" + compileFunction("local function foo() return foo() end", 1), R"( -NEWCLOSURE R0 P0 -CAPTURE VAL R0 -RETURN R0 0 -)"); - } } TEST_CASE("OutOfLocals") @@ -3504,8 +3479,6 @@ local t = { TEST_CASE("ConstantClosure") { - ScopedFastFlag sff("LuauPreloadClosures", true); - // closures without upvalues are created when bytecode is loaded CHECK_EQ("\n" + compileFunction(R"( return function() end @@ -3570,8 +3543,6 @@ RETURN R0 1 TEST_CASE("SharedClosure") { - ScopedFastFlag sff1("LuauPreloadClosures", true); - // closures can be shared even if functions refer to upvalues, as long as upvalues are top-level CHECK_EQ("\n" + compileFunction(R"( local val = ... diff --git a/tests/Conformance.test.cpp b/tests/Conformance.test.cpp index b2aad316..b055a38e 100644 --- a/tests/Conformance.test.cpp +++ b/tests/Conformance.test.cpp @@ -123,8 +123,8 @@ int lua_silence(lua_State* L) using StateRef = std::unique_ptr; -static StateRef runConformance( - const char* name, void (*setup)(lua_State* L) = nullptr, void (*yield)(lua_State* L) = nullptr, lua_State* initialLuaState = nullptr) +static StateRef runConformance(const char* name, void (*setup)(lua_State* L) = nullptr, void (*yield)(lua_State* L) = nullptr, + lua_State* initialLuaState = nullptr, lua_CompileOptions* copts = nullptr) { std::string path = __FILE__; path.erase(path.find_last_of("\\/")); @@ -180,13 +180,8 @@ static StateRef runConformance( std::string chunkname = "=" + std::string(name); - lua_CompileOptions copts = {}; - copts.optimizationLevel = 1; // default - copts.debugLevel = 2; // for debugger tests - copts.vectorCtor = "vector"; // for vector tests - size_t bytecodeSize = 0; - char* bytecode = luau_compile(source.data(), source.size(), &copts, &bytecodeSize); + char* bytecode = luau_compile(source.data(), source.size(), copts, &bytecodeSize); int result = luau_load(L, chunkname.c_str(), bytecode, bytecodeSize, 0); free(bytecode); @@ -373,29 +368,37 @@ TEST_CASE("Vector") { ScopedFastFlag sff{"LuauIfElseExpressionBaseSupport", true}; - runConformance("vector.lua", [](lua_State* L) { - lua_pushcfunction(L, lua_vector, "vector"); - lua_setglobal(L, "vector"); + lua_CompileOptions copts = {}; + copts.optimizationLevel = 1; + copts.debugLevel = 1; + copts.vectorCtor = "vector"; + + runConformance( + "vector.lua", + [](lua_State* L) { + lua_pushcfunction(L, lua_vector, "vector"); + lua_setglobal(L, "vector"); #if LUA_VECTOR_SIZE == 4 - lua_pushvector(L, 0.0f, 0.0f, 0.0f, 0.0f); + lua_pushvector(L, 0.0f, 0.0f, 0.0f, 0.0f); #else - lua_pushvector(L, 0.0f, 0.0f, 0.0f); + lua_pushvector(L, 0.0f, 0.0f, 0.0f); #endif - luaL_newmetatable(L, "vector"); + luaL_newmetatable(L, "vector"); - lua_pushstring(L, "__index"); - lua_pushcfunction(L, lua_vector_index, nullptr); - lua_settable(L, -3); + lua_pushstring(L, "__index"); + lua_pushcfunction(L, lua_vector_index, nullptr); + lua_settable(L, -3); - lua_pushstring(L, "__namecall"); - lua_pushcfunction(L, lua_vector_namecall, nullptr); - lua_settable(L, -3); + lua_pushstring(L, "__namecall"); + lua_pushcfunction(L, lua_vector_namecall, nullptr); + lua_settable(L, -3); - lua_setreadonly(L, -1, true); - lua_setmetatable(L, -2); - lua_pop(L, 1); - }); + lua_setreadonly(L, -1, true); + lua_setmetatable(L, -2); + lua_pop(L, 1); + }, + nullptr, nullptr, &copts); } static void populateRTTI(lua_State* L, Luau::TypeId type) @@ -499,6 +502,10 @@ TEST_CASE("Debugger") breakhits = 0; interruptedthread = nullptr; + lua_CompileOptions copts = {}; + copts.optimizationLevel = 1; + copts.debugLevel = 2; + runConformance( "debugger.lua", [](lua_State* L) { @@ -614,7 +621,8 @@ TEST_CASE("Debugger") lua_resume(interruptedthread, nullptr, 0); interruptedthread = nullptr; } - }); + }, + nullptr, &copts); CHECK(breakhits == 10); // 2 hits per breakpoint } @@ -863,4 +871,46 @@ TEST_CASE("TagMethodError") }); } +TEST_CASE("Coverage") +{ + lua_CompileOptions copts = {}; + copts.optimizationLevel = 1; + copts.debugLevel = 1; + copts.coverageLevel = 2; + + runConformance( + "coverage.lua", + [](lua_State* L) { + lua_pushcfunction( + L, + [](lua_State* L) -> int { + luaL_argexpected(L, lua_isLfunction(L, 1), 1, "function"); + + lua_newtable(L); + lua_getcoverage(L, 1, L, [](void* context, const char* function, int linedefined, int depth, const int* hits, size_t size) { + lua_State* L = static_cast(context); + + lua_newtable(L); + + lua_pushstring(L, function); + lua_setfield(L, -2, "name"); + + for (size_t i = 0; i < size; ++i) + if (hits[i] != -1) + { + lua_pushinteger(L, hits[i]); + lua_rawseti(L, -2, int(i)); + } + + lua_rawseti(L, -2, lua_objlen(L, -2) + 1); + }); + + return 1; + }, + "getcoverage"); + lua_setglobal(L, "getcoverage"); + }, + nullptr, nullptr, &copts); +} + TEST_SUITE_END(); diff --git a/tests/Module.test.cpp b/tests/Module.test.cpp index 2800d2fe..e3993cc5 100644 --- a/tests/Module.test.cpp +++ b/tests/Module.test.cpp @@ -278,7 +278,7 @@ TEST_CASE_FIXTURE(Fixture, "clone_recursion_limit") #if defined(_DEBUG) || defined(_NOOPT) int limit = 250; #else - int limit = 500; + int limit = 400; #endif ScopedFastInt luauTypeCloneRecursionLimit{"LuauTypeCloneRecursionLimit", limit}; @@ -287,7 +287,7 @@ TEST_CASE_FIXTURE(Fixture, "clone_recursion_limit") TypeId table = src.addType(TableTypeVar{}); TypeId nested = table; - for (unsigned i = 0; i < limit + 100; i++) + for (int i = 0; i < limit + 100; i++) { TableTypeVar* ttv = getMutable(nested); diff --git a/tests/Parser.test.cpp b/tests/Parser.test.cpp index 72d3a9a6..5abcb09a 100644 --- a/tests/Parser.test.cpp +++ b/tests/Parser.test.cpp @@ -2303,8 +2303,6 @@ TEST_CASE_FIXTURE(Fixture, "capture_comments") TEST_CASE_FIXTURE(Fixture, "capture_broken_comment_at_the_start_of_the_file") { - ScopedFastFlag sff{"LuauCaptureBrokenCommentSpans", true}; - ParseOptions options; options.captureComments = true; @@ -2319,8 +2317,6 @@ TEST_CASE_FIXTURE(Fixture, "capture_broken_comment_at_the_start_of_the_file") TEST_CASE_FIXTURE(Fixture, "capture_broken_comment") { - ScopedFastFlag sff{"LuauCaptureBrokenCommentSpans", true}; - ParseOptions options; options.captureComments = true; diff --git a/tests/TypeInfer.annotations.test.cpp b/tests/TypeInfer.annotations.test.cpp index 091c2f01..71ff4e1b 100644 --- a/tests/TypeInfer.annotations.test.cpp +++ b/tests/TypeInfer.annotations.test.cpp @@ -207,6 +207,36 @@ TEST_CASE_FIXTURE(Fixture, "as_expr_does_not_propagate_type_info") CHECK_EQ("number", toString(requireType("b"))); } +TEST_CASE_FIXTURE(Fixture, "as_expr_is_bidirectional") +{ + ScopedFastFlag sff{"LuauBidirectionalAsExpr", true}; + + CheckResult result = check(R"( + local a = 55 :: number? + local b = a :: number + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ("number?", toString(requireType("a"))); + CHECK_EQ("number", toString(requireType("b"))); +} + +TEST_CASE_FIXTURE(Fixture, "as_expr_warns_on_unrelated_cast") +{ + ScopedFastFlag sff{"LuauBidirectionalAsExpr", true}; + ScopedFastFlag sff2{"LuauErrorRecoveryType", true}; + + CheckResult result = check(R"( + local a = 55 :: string + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + CHECK_EQ("Cannot cast 'number' into 'string' because the types are unrelated", toString(result.errors[0])); + CHECK_EQ("string", toString(requireType("a"))); +} + TEST_CASE_FIXTURE(Fixture, "type_annotations_inside_function_bodies") { CheckResult result = check(R"( diff --git a/tests/TypeInfer.builtins.test.cpp b/tests/TypeInfer.builtins.test.cpp index 1e2eae14..1d8135d4 100644 --- a/tests/TypeInfer.builtins.test.cpp +++ b/tests/TypeInfer.builtins.test.cpp @@ -7,6 +7,8 @@ #include "doctest.h" +LUAU_FASTFLAG(LuauFixTonumberReturnType) + using namespace Luau; TEST_SUITE_BEGIN("BuiltinTests"); @@ -814,6 +816,30 @@ TEST_CASE_FIXTURE(Fixture, "string_format_report_all_type_errors_at_correct_posi CHECK_EQ(TypeErrorData(TypeMismatch{stringType, booleanType}), result.errors[2].data); } +TEST_CASE_FIXTURE(Fixture, "tonumber_returns_optional_number_type") +{ + CheckResult result = check(R"( + --!strict + local b: number = tonumber('asdf') + )"); + + if (FFlag::LuauFixTonumberReturnType) + { + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ("Type 'number?' could not be converted into 'number'", toString(result.errors[0])); + } +} + +TEST_CASE_FIXTURE(Fixture, "tonumber_returns_optional_number_type2") +{ + CheckResult result = check(R"( + --!strict + local b: number = tonumber('asdf') or 1 + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + TEST_CASE_FIXTURE(Fixture, "dont_add_definitions_to_persistent_types") { CheckResult result = check(R"( diff --git a/tests/TypeInfer.generics.test.cpp b/tests/TypeInfer.generics.test.cpp index aba50891..b62044fa 100644 --- a/tests/TypeInfer.generics.test.cpp +++ b/tests/TypeInfer.generics.test.cpp @@ -9,6 +9,8 @@ using namespace Luau; +LUAU_FASTFLAG(LuauExtendedFunctionMismatchError) + TEST_SUITE_BEGIN("GenericsTests"); TEST_CASE_FIXTURE(Fixture, "check_generic_function") @@ -644,4 +646,42 @@ f(1, 2, 3) CHECK_EQ(toString(*ty, opts), "(a: number, number, number) -> ()"); } +TEST_CASE_FIXTURE(Fixture, "error_detailed_function_mismatch_generic_types") +{ + CheckResult result = check(R"( +type C = () -> () +type D = () -> () + +local c: C +local d: D = c + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + if (FFlag::LuauExtendedFunctionMismatchError) + CHECK_EQ( + toString(result.errors[0]), R"(Type '() -> ()' could not be converted into '() -> ()'; different number of generic type parameters)"); + else + CHECK_EQ(toString(result.errors[0]), R"(Type '() -> ()' could not be converted into '() -> ()')"); +} + +TEST_CASE_FIXTURE(Fixture, "error_detailed_function_mismatch_generic_pack") +{ + CheckResult result = check(R"( +type C = () -> () +type D = () -> () + +local c: C +local d: D = c + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + if (FFlag::LuauExtendedFunctionMismatchError) + CHECK_EQ(toString(result.errors[0]), + R"(Type '() -> ()' could not be converted into '() -> ()'; different number of generic type pack parameters)"); + else + CHECK_EQ(toString(result.errors[0]), R"(Type '() -> ()' could not be converted into '() -> ()')"); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.refinements.test.cpp b/tests/TypeInfer.refinements.test.cpp index fe8e7ff9..688680c1 100644 --- a/tests/TypeInfer.refinements.test.cpp +++ b/tests/TypeInfer.refinements.test.cpp @@ -279,7 +279,7 @@ TEST_CASE_FIXTURE(Fixture, "assert_non_binary_expressions_actually_resolve_const TEST_CASE_FIXTURE(Fixture, "assign_table_with_refined_property_with_a_similar_type_is_illegal") { - ScopedFastFlag luauTableSubtypingVariance{"LuauTableSubtypingVariance", true}; + ScopedFastFlag LuauTableSubtypingVariance2{"LuauTableSubtypingVariance2", true}; ScopedFastFlag luauExtendedTypeMismatchError{"LuauExtendedTypeMismatchError", true}; CheckResult result = check(R"( @@ -1085,4 +1085,19 @@ TEST_CASE_FIXTURE(Fixture, "type_comparison_ifelse_expression") CHECK_EQ("any", toString(requireTypeAtPosition({6, 66}))); } +TEST_CASE_FIXTURE(Fixture, "apply_refinements_on_astexprindexexpr_whose_subscript_expr_is_constant_string") +{ + ScopedFastFlag sff{"LuauRefiLookupFromIndexExpr", true}; + + CheckResult result = check(R"( + type T = { [string]: { prop: number }? } + local t: T = {} + if t["hello"] then + local foo = t["hello"].prop + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.singletons.test.cpp b/tests/TypeInfer.singletons.test.cpp index 5f95efd5..1621ef32 100644 --- a/tests/TypeInfer.singletons.test.cpp +++ b/tests/TypeInfer.singletons.test.cpp @@ -374,4 +374,54 @@ TEST_CASE_FIXTURE(Fixture, "table_properties_type_error_escapes") toString(result.errors[0])); } +TEST_CASE_FIXTURE(Fixture, "error_detailed_tagged_union_mismatch_string") +{ + ScopedFastFlag sffs[] = { + {"LuauSingletonTypes", true}, + {"LuauParseSingletonTypes", true}, + {"LuauUnionHeuristic", true}, + {"LuauExpectedTypesOfProperties", true}, + {"LuauExtendedUnionMismatchError", true}, + }; + + CheckResult result = check(R"( +type Cat = { tag: 'cat', catfood: string } +type Dog = { tag: 'dog', dogfood: string } +type Animal = Cat | Dog + +local a: Animal = { tag = 'cat', cafood = 'something' } + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ(R"(Type 'a' could not be converted into 'Cat | Dog' +caused by: + None of the union options are compatible. For example: Table type 'a' not compatible with type 'Cat' because the former is missing field 'catfood')", + toString(result.errors[0])); +} + +TEST_CASE_FIXTURE(Fixture, "error_detailed_tagged_union_mismatch_bool") +{ + ScopedFastFlag sffs[] = { + {"LuauSingletonTypes", true}, + {"LuauParseSingletonTypes", true}, + {"LuauUnionHeuristic", true}, + {"LuauExpectedTypesOfProperties", true}, + {"LuauExtendedUnionMismatchError", true}, + }; + + CheckResult result = check(R"( +type Good = { success: true, result: string } +type Bad = { success: false, error: string } +type Result = Good | Bad + +local a: Result = { success = false, result = 'something' } + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ(R"(Type 'a' could not be converted into 'Bad | Good' +caused by: + None of the union options are compatible. For example: Table type 'a' not compatible with type 'Bad' because the former is missing field 'error')", + toString(result.errors[0])); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.tables.test.cpp b/tests/TypeInfer.tables.test.cpp index cb72faaf..3ea9b80c 100644 --- a/tests/TypeInfer.tables.test.cpp +++ b/tests/TypeInfer.tables.test.cpp @@ -12,6 +12,8 @@ using namespace Luau; +LUAU_FASTFLAG(LuauExtendedFunctionMismatchError) + TEST_SUITE_BEGIN("TableTests"); TEST_CASE_FIXTURE(Fixture, "basic") @@ -275,7 +277,7 @@ TEST_CASE_FIXTURE(Fixture, "open_table_unification") TEST_CASE_FIXTURE(Fixture, "open_table_unification_2") { - ScopedFastFlag sff{"LuauTableSubtypingVariance", true}; + ScopedFastFlag sff{"LuauTableSubtypingVariance2", true}; CheckResult result = check(R"( local a = {} @@ -346,7 +348,7 @@ TEST_CASE_FIXTURE(Fixture, "table_param_row_polymorphism_1") TEST_CASE_FIXTURE(Fixture, "table_param_row_polymorphism_2") { - ScopedFastFlag sff{"LuauTableSubtypingVariance", true}; + ScopedFastFlag sff{"LuauTableSubtypingVariance2", true}; CheckResult result = check(R"( --!strict @@ -369,7 +371,7 @@ TEST_CASE_FIXTURE(Fixture, "table_param_row_polymorphism_2") TEST_CASE_FIXTURE(Fixture, "table_param_row_polymorphism_3") { - ScopedFastFlag sff{"LuauTableSubtypingVariance", true}; + ScopedFastFlag sff{"LuauTableSubtypingVariance2", true}; CheckResult result = check(R"( local T = {} @@ -476,7 +478,7 @@ TEST_CASE_FIXTURE(Fixture, "ok_to_add_property_to_free_table") TEST_CASE_FIXTURE(Fixture, "okay_to_add_property_to_unsealed_tables_by_assignment") { - ScopedFastFlag sff{"LuauTableSubtypingVariance", true}; + ScopedFastFlag sff{"LuauTableSubtypingVariance2", true}; CheckResult result = check(R"( --!strict @@ -511,7 +513,7 @@ TEST_CASE_FIXTURE(Fixture, "okay_to_add_property_to_unsealed_tables_by_function_ TEST_CASE_FIXTURE(Fixture, "width_subtyping") { - ScopedFastFlag sff{"LuauTableSubtypingVariance", true}; + ScopedFastFlag sff{"LuauTableSubtypingVariance2", true}; CheckResult result = check(R"( --!strict @@ -771,7 +773,7 @@ TEST_CASE_FIXTURE(Fixture, "infer_indexer_for_left_unsealed_table_from_right_han TEST_CASE_FIXTURE(Fixture, "sealed_table_value_can_infer_an_indexer") { - ScopedFastFlag sff{"LuauTableSubtypingVariance", true}; + ScopedFastFlag sff{"LuauTableSubtypingVariance2", true}; CheckResult result = check(R"( local t: { a: string, [number]: string } = { a = "foo" } @@ -782,7 +784,7 @@ TEST_CASE_FIXTURE(Fixture, "sealed_table_value_can_infer_an_indexer") TEST_CASE_FIXTURE(Fixture, "array_factory_function") { - ScopedFastFlag sff{"LuauTableSubtypingVariance", true}; + ScopedFastFlag sff{"LuauTableSubtypingVariance2", true}; CheckResult result = check(R"( function empty() return {} end @@ -1465,7 +1467,7 @@ TEST_CASE_FIXTURE(Fixture, "casting_tables_with_props_into_table_with_indexer2") TEST_CASE_FIXTURE(Fixture, "casting_tables_with_props_into_table_with_indexer3") { - ScopedFastFlag sff{"LuauTableSubtypingVariance", true}; + ScopedFastFlag sff{"LuauTableSubtypingVariance2", true}; CheckResult result = check(R"( local function foo(a: {[string]: number, a: string}) end @@ -1550,7 +1552,7 @@ TEST_CASE_FIXTURE(Fixture, "table_subtyping_with_extra_props_dont_report_multipl TEST_CASE_FIXTURE(Fixture, "table_subtyping_with_extra_props_is_ok") { - ScopedFastFlag sff{"LuauTableSubtypingVariance", true}; + ScopedFastFlag sff{"LuauTableSubtypingVariance2", true}; CheckResult result = check(R"( local vec3 = {x = 1, y = 2, z = 3} @@ -1937,7 +1939,7 @@ TEST_CASE_FIXTURE(Fixture, "table_insert_should_cope_with_optional_properties_in TEST_CASE_FIXTURE(Fixture, "table_insert_should_cope_with_optional_properties_in_strict") { - ScopedFastFlag sff{"LuauTableSubtypingVariance", true}; + ScopedFastFlag sff{"LuauTableSubtypingVariance2", true}; CheckResult result = check(R"( --!strict @@ -1952,7 +1954,7 @@ TEST_CASE_FIXTURE(Fixture, "table_insert_should_cope_with_optional_properties_in TEST_CASE_FIXTURE(Fixture, "error_detailed_prop") { - ScopedFastFlag luauTableSubtypingVariance{"LuauTableSubtypingVariance", true}; // Only for new path + ScopedFastFlag LuauTableSubtypingVariance2{"LuauTableSubtypingVariance2", true}; // Only for new path ScopedFastFlag luauExtendedTypeMismatchError{"LuauExtendedTypeMismatchError", true}; CheckResult result = check(R"( @@ -1971,7 +1973,7 @@ caused by: TEST_CASE_FIXTURE(Fixture, "error_detailed_prop_nested") { - ScopedFastFlag luauTableSubtypingVariance{"LuauTableSubtypingVariance", true}; // Only for new path + ScopedFastFlag LuauTableSubtypingVariance2{"LuauTableSubtypingVariance2", true}; // Only for new path ScopedFastFlag luauExtendedTypeMismatchError{"LuauExtendedTypeMismatchError", true}; CheckResult result = check(R"( @@ -1995,7 +1997,7 @@ caused by: TEST_CASE_FIXTURE(Fixture, "error_detailed_metatable_prop") { - ScopedFastFlag luauTableSubtypingVariance{"LuauTableSubtypingVariance", true}; // Only for new path + ScopedFastFlag LuauTableSubtypingVariance2{"LuauTableSubtypingVariance2", true}; // Only for new path ScopedFastFlag luauExtendedTypeMismatchError{"LuauExtendedTypeMismatchError", true}; CheckResult result = check(R"( @@ -2015,11 +2017,22 @@ caused by: caused by: Property 'y' is not compatible. Type 'string' could not be converted into 'number')"); - CHECK_EQ(toString(result.errors[1]), R"(Type 'b2' could not be converted into 'a2' + if (FFlag::LuauExtendedFunctionMismatchError) + { + CHECK_EQ(toString(result.errors[1]), R"(Type 'b2' could not be converted into 'a2' +caused by: + Type '{| __call: (a, b) -> () |}' could not be converted into '{| __call: (a) -> () |}' +caused by: + Property '__call' is not compatible. Type '(a, b) -> ()' could not be converted into '(a) -> ()'; different number of generic type parameters)"); + } + else + { + CHECK_EQ(toString(result.errors[1]), R"(Type 'b2' could not be converted into 'a2' caused by: Type '{| __call: (a, b) -> () |}' could not be converted into '{| __call: (a) -> () |}' caused by: Property '__call' is not compatible. Type '(a, b) -> ()' could not be converted into '(a) -> ()')"); + } } TEST_CASE_FIXTURE(Fixture, "explicitly_typed_table") @@ -2027,7 +2040,7 @@ TEST_CASE_FIXTURE(Fixture, "explicitly_typed_table") ScopedFastFlag sffs[] { {"LuauPropertiesGetExpectedType", true}, {"LuauExpectedTypesOfProperties", true}, - {"LuauTableSubtypingVariance", true}, + {"LuauTableSubtypingVariance2", true}, }; CheckResult result = check(R"( @@ -2048,7 +2061,7 @@ TEST_CASE_FIXTURE(Fixture, "explicitly_typed_table_error") ScopedFastFlag sffs[] { {"LuauPropertiesGetExpectedType", true}, {"LuauExpectedTypesOfProperties", true}, - {"LuauTableSubtypingVariance", true}, + {"LuauTableSubtypingVariance2", true}, {"LuauExtendedTypeMismatchError", true}, }; @@ -2076,7 +2089,7 @@ TEST_CASE_FIXTURE(Fixture, "explicitly_typed_table_with_indexer") ScopedFastFlag sffs[] { {"LuauPropertiesGetExpectedType", true}, {"LuauExpectedTypesOfProperties", true}, - {"LuauTableSubtypingVariance", true}, + {"LuauTableSubtypingVariance2", true}, }; CheckResult result = check(R"( @@ -2092,4 +2105,18 @@ a.p = { x = 9 } LUAU_REQUIRE_NO_ERRORS(result); } +TEST_CASE_FIXTURE(Fixture, "recursive_metatable_type_call") +{ + ScopedFastFlag luauFixRecursiveMetatableCall{"LuauFixRecursiveMetatableCall", true}; + + CheckResult result = check(R"( +local b +b = setmetatable({}, {__call = b}) +b() + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ(toString(result.errors[0]), R"(Cannot call non-function t1 where t1 = { @metatable {| __call: t1 |}, { } })"); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.test.cpp b/tests/TypeInfer.test.cpp index e3222a41..ad9ea827 100644 --- a/tests/TypeInfer.test.cpp +++ b/tests/TypeInfer.test.cpp @@ -16,6 +16,7 @@ LUAU_FASTFLAG(LuauFixLocationSpanTableIndexExpr) LUAU_FASTFLAG(LuauEqConstraint) +LUAU_FASTFLAG(LuauExtendedFunctionMismatchError) using namespace Luau; @@ -2084,7 +2085,7 @@ TEST_CASE_FIXTURE(Fixture, "primitive_arith_no_metatable") { CheckResult result = check(R"( function add(a: number, b: string) - return a + tonumber(b), a .. b + return a + (tonumber(b) :: number), a .. b end local n, s = add(2,"3") )"); @@ -2485,7 +2486,7 @@ TEST_CASE_FIXTURE(Fixture, "inferring_crazy_table_should_also_be_quick") CheckResult result = check(R"( --!strict function f(U) - U(w:s(an):c()():c():U(s):c():c():U(s):c():U(s):cU()):c():U(s):c():U(s):c():c():U(s):c():U(s):cU() + U(w:s(an):c()():c():U(s):c():c():U(s):c():U(s):cU()):c():U(s):c():U(s):c():c():U(s):c():U(s):cU() end )"); @@ -3329,7 +3330,7 @@ TEST_CASE_FIXTURE(Fixture, "relation_op_on_any_lhs_where_rhs_maybe_has_metatable { CheckResult result = check(R"( local x - print((x == true and (x .. "y")) .. 1) + print((x == true and (x .. "y")) .. 1) )"); LUAU_REQUIRE_ERROR_COUNT(1, result); @@ -4473,7 +4474,18 @@ f(function(a, b, c, ...) return a + b end) )"); LUAU_REQUIRE_ERRORS(result); - CHECK_EQ("Type '(number, number, a) -> number' could not be converted into '(number, number) -> number'", toString(result.errors[0])); + + if (FFlag::LuauExtendedFunctionMismatchError) + { + CHECK_EQ(R"(Type '(number, number, a) -> number' could not be converted into '(number, number) -> number' +caused by: + Argument count mismatch. Function expects 3 arguments, but only 2 are specified)", + toString(result.errors[0])); + } + else + { + CHECK_EQ(R"(Type '(number, number, a) -> number' could not be converted into '(number, number) -> number')", toString(result.errors[0])); + } // Infer from variadic packs into elements result = check(R"( @@ -4604,7 +4616,17 @@ local c = sumrec(function(x, y, f) return f(x, y) end) -- type binders are not i )"); LUAU_REQUIRE_ERRORS(result); - CHECK_EQ("Type '(a, b, (a, b) -> (c...)) -> (c...)' could not be converted into '(a, a, (a, a) -> a) -> a'", toString(result.errors[0])); + if (FFlag::LuauExtendedFunctionMismatchError) + { + CHECK_EQ( + "Type '(a, b, (a, b) -> (c...)) -> (c...)' could not be converted into '(a, a, (a, a) -> a) -> a'; different number of generic type " + "parameters", + toString(result.errors[0])); + } + else + { + CHECK_EQ("Type '(a, b, (a, b) -> (c...)) -> (c...)' could not be converted into '(a, a, (a, a) -> a) -> a'", toString(result.errors[0])); + } } TEST_CASE_FIXTURE(Fixture, "infer_return_value_type") @@ -4799,4 +4821,211 @@ local ModuleA = require(game.A) CHECK_EQ("*unknown*", toString(*oty)); } +/* + * If it wasn't instantly obvious, we have the fuzzer to thank for this gem of a test. + * + * We had an issue here where the scope for the `if` block here would + * have an elevated TypeLevel even though there is no function nesting going on. + * This would result in a free typevar for the type of _ that was much higher than + * it should be. This type would be erroneously quantified in the definition of `aaa`. + * This in turn caused an ice when evaluating `_()` in the while loop. + */ +TEST_CASE_FIXTURE(Fixture, "free_typevars_introduced_within_control_flow_constructs_do_not_get_an_elevated_TypeLevel") +{ + check(R"( + --!strict + if _ then + _[_], _ = nil + _() + end + + local aaa = function():typeof(_) return 1 end + + if aaa then + while _() do + end + end + )"); + + // No ice()? No problem. +} + +/* + * This is a bit elaborate. Bear with me. + * + * The type of _ becomes free with the first statement. With the second, we unify it with a function. + * + * At this point, it is important that the newly created fresh types of this new function type are promoted + * to the same level as the original free type. If we do not, they are incorrectly ascribed the level of the + * containing function. + * + * If this is allowed to happen, the final lambda erroneously quantifies the type of _ to something ridiculous + * just before we typecheck the invocation to _. + */ +TEST_CASE_FIXTURE(Fixture, "fuzzer_found_this") +{ + check(R"( + l0, _ = nil + + local function p() + _() + end + + a = _( + function():(typeof(p),typeof(_)) + end + )[nil] + )"); +} + +/* + * We had an issue where part of the type of pairs() was an unsealed table. + * This test depends on FFlagDebugLuauFreezeArena to trigger it. + */ +TEST_CASE_FIXTURE(Fixture, "pairs_parameters_are_not_unsealed_tables") +{ + check(R"( + function _(l0:{n0:any}) + _ = pairs + end + )"); +} + +TEST_CASE_FIXTURE(Fixture, "inferred_methods_of_free_tables_have_the_same_level_as_the_enclosing_table") +{ + check(R"( + function Base64FileReader(data) + local reader = {} + local index: number + + function reader:PeekByte() + return data:byte(index) + end + + function reader:Byte() + return data:byte(index - 1) + end + + return reader + end + + Base64FileReader() + + function ReadMidiEvents(data) + + local reader = Base64FileReader(data) + + while reader:HasMore() do + (reader:Byte() % 128) + end + end + )"); +} + +TEST_CASE_FIXTURE(Fixture, "error_detailed_function_mismatch_arg_count") +{ + ScopedFastFlag luauExtendedFunctionMismatchError{"LuauExtendedFunctionMismatchError", true}; + + CheckResult result = check(R"( +type A = (number, number) -> string +type B = (number) -> string + +local a: A +local b: B = a + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ(toString(result.errors[0]), R"(Type '(number, number) -> string' could not be converted into '(number) -> string' +caused by: + Argument count mismatch. Function expects 2 arguments, but only 1 is specified)"); +} + +TEST_CASE_FIXTURE(Fixture, "error_detailed_function_mismatch_arg") +{ + ScopedFastFlag luauExtendedFunctionMismatchError{"LuauExtendedFunctionMismatchError", true}; + + CheckResult result = check(R"( +type A = (number, number) -> string +type B = (number, string) -> string + +local a: A +local b: B = a + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ(toString(result.errors[0]), R"(Type '(number, number) -> string' could not be converted into '(number, string) -> string' +caused by: + Argument #2 type is not compatible. Type 'string' could not be converted into 'number')"); +} + +TEST_CASE_FIXTURE(Fixture, "error_detailed_function_mismatch_ret_count") +{ + ScopedFastFlag luauExtendedFunctionMismatchError{"LuauExtendedFunctionMismatchError", true}; + + CheckResult result = check(R"( +type A = (number, number) -> (number) +type B = (number, number) -> (number, boolean) + +local a: A +local b: B = a + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ(toString(result.errors[0]), R"(Type '(number, number) -> number' could not be converted into '(number, number) -> (number, boolean)' +caused by: + Function only returns 1 value. 2 are required here)"); +} + +TEST_CASE_FIXTURE(Fixture, "error_detailed_function_mismatch_ret") +{ + ScopedFastFlag luauExtendedFunctionMismatchError{"LuauExtendedFunctionMismatchError", true}; + + CheckResult result = check(R"( +type A = (number, number) -> string +type B = (number, number) -> number + +local a: A +local b: B = a + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ(toString(result.errors[0]), R"(Type '(number, number) -> string' could not be converted into '(number, number) -> number' +caused by: + Return type is not compatible. Type 'string' could not be converted into 'number')"); +} + +TEST_CASE_FIXTURE(Fixture, "error_detailed_function_mismatch_ret_mult") +{ + ScopedFastFlag luauExtendedFunctionMismatchError{"LuauExtendedFunctionMismatchError", true}; + + CheckResult result = check(R"( +type A = (number, number) -> (number, string) +type B = (number, number) -> (number, boolean) + +local a: A +local b: B = a + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ( + toString(result.errors[0]), R"(Type '(number, number) -> (number, string)' could not be converted into '(number, number) -> (number, boolean)' +caused by: + Return #2 type is not compatible. Type 'string' could not be converted into 'boolean')"); +} + +TEST_CASE_FIXTURE(Fixture, "table_function_check_use_after_free") +{ + ScopedFastFlag luauUnifyFunctionCheckResult{"LuauUpdateFunctionNameBinding", true}; + + CheckResult result = check(R"( +local t = {} + +function t.x(value) + for k,v in pairs(t) do end +end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.tryUnify.test.cpp b/tests/TypeInfer.tryUnify.test.cpp index 9f9a007f..f55b46a4 100644 --- a/tests/TypeInfer.tryUnify.test.cpp +++ b/tests/TypeInfer.tryUnify.test.cpp @@ -214,4 +214,32 @@ TEST_CASE_FIXTURE(TryUnifyFixture, "cli_41095_concat_log_in_sealed_table_unifica CHECK_EQ(toString(result.errors[1]), "Available overloads: ({a}, a) -> (); and ({a}, number, a) -> ()"); } +TEST_CASE_FIXTURE(TryUnifyFixture, "undo_new_prop_on_unsealed_table") +{ + ScopedFastFlag flags[] = { + {"LuauTableSubtypingVariance2", true}, + }; + // I am not sure how to make this happen in Luau code. + + TypeId unsealedTable = arena.addType(TableTypeVar{TableState::Unsealed, TypeLevel{}}); + TypeId sealedTable = arena.addType(TableTypeVar{ + {{"prop", Property{getSingletonTypes().numberType}}}, + std::nullopt, + TypeLevel{}, + TableState::Sealed + }); + + const TableTypeVar* ttv = get(unsealedTable); + REQUIRE(ttv); + + state.tryUnify(unsealedTable, sealedTable); + + // To be honest, it's really quite spooky here that we're amending an unsealed table in this case. + CHECK(!ttv->props.empty()); + + state.log.rollback(); + + CHECK(ttv->props.empty()); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.unionTypes.test.cpp b/tests/TypeInfer.unionTypes.test.cpp index 48496b89..b095a0db 100644 --- a/tests/TypeInfer.unionTypes.test.cpp +++ b/tests/TypeInfer.unionTypes.test.cpp @@ -462,4 +462,20 @@ local a: XYZ = { w = 4 } CHECK_EQ(toString(result.errors[0]), R"(Type 'a' could not be converted into 'X | Y | Z'; none of the union options are compatible)"); } +TEST_CASE_FIXTURE(Fixture, "error_detailed_optional") +{ + ScopedFastFlag luauExtendedUnionMismatchError{"LuauExtendedUnionMismatchError", true}; + + CheckResult result = check(R"( +type X = { x: number } + +local a: X? = { w = 4 } + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ(toString(result.errors[0]), R"(Type 'a' could not be converted into 'X?' +caused by: + None of the union options are compatible. For example: Table type 'a' not compatible with type 'X' because the former is missing field 'x')"); +} + TEST_SUITE_END(); diff --git a/tests/conformance/coverage.lua b/tests/conformance/coverage.lua new file mode 100644 index 00000000..f899603f --- /dev/null +++ b/tests/conformance/coverage.lua @@ -0,0 +1,64 @@ +-- This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +print("testing coverage") + +function foo() + local x = 1 + local y = 2 + assert(x + y) +end + +function bar() + local function one(x) + return x + end + + local two = function(x) + return x + end + + one(1) +end + +function validate(stats, hits, misses) + local checked = {} + + for _,l in ipairs(hits) do + if not (stats[l] and stats[l] > 0) then + return false, string.format("expected line %d to be hit", l) + end + checked[l] = true + end + + for _,l in ipairs(misses) do + if not (stats[l] and stats[l] == 0) then + return false, string.format("expected line %d to be missed", l) + end + checked[l] = true + end + + for k,v in pairs(stats) do + if type(k) == "number" and not checked[k] then + return false, string.format("expected line %d to be absent", k) + end + end + + return true +end + +foo() +c = getcoverage(foo) +assert(#c == 1) +assert(c[1].name == "foo") +assert(validate(c[1], {5, 6, 7}, {})) + +bar() +c = getcoverage(bar) +assert(#c == 3) +assert(c[1].name == "bar") +assert(validate(c[1], {11, 15, 19}, {})) +assert(c[2].name == "one") +assert(validate(c[2], {12}, {})) +assert(c[3].name == nil) +assert(validate(c[3], {}, {16})) + +return 'OK' From f2e6a8f4a57d4b1e92c1828dbf7632404237438c Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Fri, 10 Dec 2021 14:05:05 -0800 Subject: [PATCH 103/399] Sync to upstream/release/507-pre (#286) This doesn't contain all changes for 507 yet but we might want to do the Luau 0.507 release a bit earlier to end the year sooner. Changes: - Type ascription (::) now permits casts between related types in both directions, allowing to refine or loosen the type (RFC #56) - Fix type definition for tonumber to return number? since the input string isn't guaranteed to contain a valid number - Fix type refinements for field access via [] - Many stability fixes for type checker - Provide extra information in error messages for type mismatches in more cases - Improve performance of type checking for large unions when union members are string literals - Add coverage reporting support to Repl (--coverage command line argument) and lua_getcoverage C API - Work around code signing issues during Makefile builds on macOS - Improve performance of truthiness checks in some cases, particularly on Apple M1, resulting in 10-25% perf gains on qsort benchmark depending on the CPU/compiler - Fix support for little-endian systems; IBM s390x here we go! --- Analysis/include/Luau/Error.h | 11 +- Analysis/include/Luau/IostreamHelpers.h | 4 + Analysis/include/Luau/ToString.h | 4 +- Analysis/include/Luau/TypeInfer.h | 7 +- Analysis/include/Luau/TypeVar.h | 8 +- Analysis/include/Luau/Unifiable.h | 11 +- Analysis/include/Luau/Unifier.h | 3 + Analysis/src/Autocomplete.cpp | 115 +++++++-- Analysis/src/BuiltinDefinitions.cpp | 9 +- Analysis/src/EmbeddedBuiltinDefinitions.cpp | 12 +- Analysis/src/Error.cpp | 17 +- Analysis/src/IostreamHelpers.cpp | 6 + Analysis/src/JsonEncoder.cpp | 2 +- Analysis/src/Module.cpp | 19 +- Analysis/src/Predicate.cpp | 2 +- Analysis/src/ToString.cpp | 38 ++- Analysis/src/TypeInfer.cpp | 191 +++++++++----- Analysis/src/TypeUtils.cpp | 6 +- Analysis/src/TypeVar.cpp | 220 ++++------------ Analysis/src/Unifier.cpp | 266 +++++++++++++++++--- Ast/src/Parser.cpp | 5 +- CLI/Analyze.cpp | 20 +- CLI/Coverage.cpp | 88 +++++++ CLI/Coverage.h | 10 + CLI/Profiler.cpp | 8 +- CLI/Profiler.h | 2 +- CLI/Repl.cpp | 35 ++- Compiler/src/Compiler.cpp | 22 +- Makefile | 6 +- Sources.cmake | 4 + VM/include/lua.h | 4 + VM/src/lapi.cpp | 162 ++++++------ VM/src/ldebug.cpp | 63 +++++ VM/src/lgc.cpp | 49 +--- VM/src/lgcdebug.cpp | 1 + VM/src/lobject.h | 10 +- VM/src/lstring.cpp | 29 --- VM/src/lstring.h | 7 - VM/src/ludata.cpp | 37 +++ VM/src/ludata.h | 13 + VM/src/lvmexecute.cpp | 14 +- bench/tests/sunspider/3d-raytrace.lua | 15 +- tests/Autocomplete.test.cpp | 55 +++- tests/Compiler.test.cpp | 49 +--- tests/Conformance.test.cpp | 100 ++++++-- tests/Parser.test.cpp | 4 - tests/TypeInfer.annotations.test.cpp | 30 +++ tests/TypeInfer.builtins.test.cpp | 26 ++ tests/TypeInfer.generics.test.cpp | 40 +++ tests/TypeInfer.refinements.test.cpp | 17 +- tests/TypeInfer.singletons.test.cpp | 50 ++++ tests/TypeInfer.tables.test.cpp | 61 +++-- tests/TypeInfer.test.cpp | 239 +++++++++++++++++- tests/TypeInfer.tryUnify.test.cpp | 28 +++ tests/TypeInfer.unionTypes.test.cpp | 16 ++ tests/conformance/coverage.lua | 64 +++++ 56 files changed, 1695 insertions(+), 639 deletions(-) create mode 100644 CLI/Coverage.cpp create mode 100644 CLI/Coverage.h create mode 100644 VM/src/ludata.cpp create mode 100644 VM/src/ludata.h create mode 100644 tests/conformance/coverage.lua diff --git a/Analysis/include/Luau/Error.h b/Analysis/include/Luau/Error.h index 9ee75004..aff3c4d9 100644 --- a/Analysis/include/Luau/Error.h +++ b/Analysis/include/Luau/Error.h @@ -277,11 +277,20 @@ struct MissingUnionProperty bool operator==(const MissingUnionProperty& rhs) const; }; +struct TypesAreUnrelated +{ + TypeId left; + TypeId right; + + bool operator==(const TypesAreUnrelated& rhs) const; +}; + using TypeErrorData = Variant; + DuplicateGenericParameter, CannotInferBinaryOperation, MissingProperties, SwappedGenericTypeParameter, OptionalValueAccess, MissingUnionProperty, + TypesAreUnrelated>; struct TypeError { diff --git a/Analysis/include/Luau/IostreamHelpers.h b/Analysis/include/Luau/IostreamHelpers.h index f9e9cd48..ee994296 100644 --- a/Analysis/include/Luau/IostreamHelpers.h +++ b/Analysis/include/Luau/IostreamHelpers.h @@ -36,6 +36,10 @@ std::ostream& operator<<(std::ostream& lhs, const IllegalRequire& error); std::ostream& operator<<(std::ostream& lhs, const ModuleHasCyclicDependency& error); std::ostream& operator<<(std::ostream& lhs, const DuplicateGenericParameter& error); std::ostream& operator<<(std::ostream& lhs, const CannotInferBinaryOperation& error); +std::ostream& operator<<(std::ostream& lhs, const SwappedGenericTypeParameter& error); +std::ostream& operator<<(std::ostream& lhs, const OptionalValueAccess& error); +std::ostream& operator<<(std::ostream& lhs, const MissingUnionProperty& error); +std::ostream& operator<<(std::ostream& lhs, const TypesAreUnrelated& error); std::ostream& operator<<(std::ostream& lhs, const TableState& tv); std::ostream& operator<<(std::ostream& lhs, const TypeVar& tv); diff --git a/Analysis/include/Luau/ToString.h b/Analysis/include/Luau/ToString.h index 50379c1c..a97bf6d6 100644 --- a/Analysis/include/Luau/ToString.h +++ b/Analysis/include/Luau/ToString.h @@ -69,8 +69,8 @@ std::string toStringNamedFunction(const std::string& prefix, const FunctionTypeV // It could be useful to see the text representation of a type during a debugging session instead of exploring the content of the class // These functions will dump the type to stdout and can be evaluated in Watch/Immediate windows or as gdb/lldb expression -void dump(TypeId ty); -void dump(TypePackId ty); +std::string dump(TypeId ty); +std::string dump(TypePackId ty); std::string generateName(size_t n); diff --git a/Analysis/include/Luau/TypeInfer.h b/Analysis/include/Luau/TypeInfer.h index 9f553bc1..451976e4 100644 --- a/Analysis/include/Luau/TypeInfer.h +++ b/Analysis/include/Luau/TypeInfer.h @@ -156,13 +156,14 @@ struct TypeChecker // Returns both the type of the lvalue and its binding (if the caller wants to mutate the binding). // Note: the binding may be null. + // TODO: remove second return value with FFlagLuauUpdateFunctionNameBinding std::pair checkLValueBinding(const ScopePtr& scope, const AstExpr& expr); std::pair checkLValueBinding(const ScopePtr& scope, const AstExprLocal& expr); std::pair checkLValueBinding(const ScopePtr& scope, const AstExprGlobal& expr); std::pair checkLValueBinding(const ScopePtr& scope, const AstExprIndexName& expr); std::pair checkLValueBinding(const ScopePtr& scope, const AstExprIndexExpr& expr); - TypeId checkFunctionName(const ScopePtr& scope, AstExpr& funName); + TypeId checkFunctionName(const ScopePtr& scope, AstExpr& funName, TypeLevel level); std::pair checkFunctionSignature(const ScopePtr& scope, int subLevel, const AstExprFunction& expr, std::optional originalNameLoc, std::optional expectedType); void checkFunctionBody(const ScopePtr& scope, TypeId type, const AstExprFunction& function); @@ -174,7 +175,7 @@ struct TypeChecker ExprResult checkExprPack(const ScopePtr& scope, const AstExprCall& expr); std::vector> getExpectedTypesForCall(const std::vector& overloads, size_t argumentCount, bool selfCall); std::optional> checkCallOverload(const ScopePtr& scope, const AstExprCall& expr, TypeId fn, TypePackId retPack, - TypePackId argPack, TypePack* args, const std::vector& argLocations, const ExprResult& argListResult, + TypePackId argPack, TypePack* args, const std::vector* argLocations, const ExprResult& argListResult, std::vector& overloadsThatMatchArgCount, std::vector& overloadsThatDont, std::vector& errors); bool handleSelfCallMismatch(const ScopePtr& scope, const AstExprCall& expr, TypePack* args, const std::vector& argLocations, const std::vector& errors); @@ -277,7 +278,7 @@ public: [[noreturn]] void ice(const std::string& message); ScopePtr childFunctionScope(const ScopePtr& parent, const Location& location, int subLevel = 0); - ScopePtr childScope(const ScopePtr& parent, const Location& location, int subLevel = 0); + ScopePtr childScope(const ScopePtr& parent, const Location& location); // Wrapper for merge(l, r, toUnion) but without the lambda junk. void merge(RefinementMap& l, const RefinementMap& r); diff --git a/Analysis/include/Luau/TypeVar.h b/Analysis/include/Luau/TypeVar.h index 8c4c2f34..f6829ec3 100644 --- a/Analysis/include/Luau/TypeVar.h +++ b/Analysis/include/Luau/TypeVar.h @@ -499,6 +499,7 @@ struct SingletonTypes const TypePackId anyTypePack; SingletonTypes(); + ~SingletonTypes(); SingletonTypes(const SingletonTypes&) = delete; void operator=(const SingletonTypes&) = delete; @@ -509,10 +510,12 @@ struct SingletonTypes private: std::unique_ptr arena; + bool debugFreezeArena = false; + TypeId makeStringMetatable(); }; -extern SingletonTypes singletonTypes; +SingletonTypes& getSingletonTypes(); void persist(TypeId ty); void persist(TypePackId tp); @@ -523,9 +526,6 @@ TypeLevel* getMutableLevel(TypeId ty); const Property* lookupClassProp(const ClassTypeVar* cls, const Name& name); bool isSubclass(const ClassTypeVar* cls, const ClassTypeVar* parent); -bool hasGeneric(TypeId ty); -bool hasGeneric(TypePackId tp); - TypeVar* asMutable(TypeId ty); template diff --git a/Analysis/include/Luau/Unifiable.h b/Analysis/include/Luau/Unifiable.h index b47610fc..e8eafe68 100644 --- a/Analysis/include/Luau/Unifiable.h +++ b/Analysis/include/Luau/Unifiable.h @@ -24,7 +24,7 @@ struct TypeLevel int level = 0; int subLevel = 0; - // Returns true if the typelevel "this" is "bigger" than rhs + // Returns true if the level of "this" belongs to an equal or larger scope than that of rhs bool subsumes(const TypeLevel& rhs) const { if (level < rhs.level) @@ -38,6 +38,15 @@ struct TypeLevel return false; } + // Returns true if the level of "this" belongs to a larger (not equal) scope than that of rhs + bool subsumesStrict(const TypeLevel& rhs) const + { + if (level == rhs.level && subLevel == rhs.subLevel) + return false; + else + return subsumes(rhs); + } + TypeLevel incr() const { TypeLevel result; diff --git a/Analysis/include/Luau/Unifier.h b/Analysis/include/Luau/Unifier.h index 4588cdd8..7681b966 100644 --- a/Analysis/include/Luau/Unifier.h +++ b/Analysis/include/Luau/Unifier.h @@ -91,6 +91,9 @@ private: [[noreturn]] void ice(const std::string& message, const Location& location); [[noreturn]] void ice(const std::string& message); + + // Available after regular type pack unification errors + std::optional firstPackErrorPos; }; } // namespace Luau diff --git a/Analysis/src/Autocomplete.cpp b/Analysis/src/Autocomplete.cpp index db2d1d0e..4b583792 100644 --- a/Analysis/src/Autocomplete.cpp +++ b/Analysis/src/Autocomplete.cpp @@ -15,6 +15,7 @@ LUAU_FASTFLAG(LuauIfElseExpressionAnalysisSupport) LUAU_FASTFLAGVARIABLE(LuauAutocompleteAvoidMutation, false); LUAU_FASTFLAGVARIABLE(LuauAutocompletePreferToCallFunctions, false); +LUAU_FASTFLAGVARIABLE(LuauAutocompleteFirstArg, false); static const std::unordered_set kStatementStartingKeywords = { "while", "if", "local", "repeat", "function", "do", "for", "return", "break", "continue", "type", "export"}; @@ -190,7 +191,48 @@ static ParenthesesRecommendation getParenRecommendation(TypeId id, const std::ve return ParenthesesRecommendation::None; } -static TypeCorrectKind checkTypeCorrectKind(const Module& module, TypeArena* typeArena, AstNode* node, TypeId ty) +static std::optional findExpectedTypeAt(const Module& module, AstNode* node, Position position) +{ + LUAU_ASSERT(FFlag::LuauAutocompleteFirstArg); + + auto expr = node->asExpr(); + if (!expr) + return std::nullopt; + + // Extra care for first function call argument location + // When we don't have anything inside () yet, we also don't have an AST node to base our lookup + if (AstExprCall* exprCall = expr->as()) + { + if (exprCall->args.size == 0 && exprCall->argLocation.contains(position)) + { + auto it = module.astTypes.find(exprCall->func); + + if (!it) + return std::nullopt; + + const FunctionTypeVar* ftv = get(follow(*it)); + + if (!ftv) + return std::nullopt; + + auto [head, tail] = flatten(ftv->argTypes); + unsigned index = exprCall->self ? 1 : 0; + + if (index < head.size()) + return head[index]; + + return std::nullopt; + } + } + + auto it = module.astExpectedTypes.find(expr); + if (!it) + return std::nullopt; + + return *it; +} + +static TypeCorrectKind checkTypeCorrectKind(const Module& module, TypeArena* typeArena, AstNode* node, Position position, TypeId ty) { ty = follow(ty); @@ -220,15 +262,29 @@ static TypeCorrectKind checkTypeCorrectKind(const Module& module, TypeArena* typ } }; - auto expr = node->asExpr(); - if (!expr) - return TypeCorrectKind::None; + TypeId expectedType; - auto it = module.astExpectedTypes.find(expr); - if (!it) - return TypeCorrectKind::None; + if (FFlag::LuauAutocompleteFirstArg) + { + auto typeAtPosition = findExpectedTypeAt(module, node, position); - TypeId expectedType = follow(*it); + if (!typeAtPosition) + return TypeCorrectKind::None; + + expectedType = follow(*typeAtPosition); + } + else + { + auto expr = node->asExpr(); + if (!expr) + return TypeCorrectKind::None; + + auto it = module.astExpectedTypes.find(expr); + if (!it) + return TypeCorrectKind::None; + + expectedType = follow(*it); + } if (FFlag::LuauAutocompletePreferToCallFunctions) { @@ -333,8 +389,8 @@ static void autocompleteProps(const Module& module, TypeArena* typeArena, TypeId if (result.count(name) == 0 && name != Parser::errorName) { Luau::TypeId type = Luau::follow(prop.type); - TypeCorrectKind typeCorrect = - indexType == PropIndexType::Key ? TypeCorrectKind::Correct : checkTypeCorrectKind(module, typeArena, nodes.back(), type); + TypeCorrectKind typeCorrect = indexType == PropIndexType::Key ? TypeCorrectKind::Correct + : checkTypeCorrectKind(module, typeArena, nodes.back(), {{}, {}}, type); ParenthesesRecommendation parens = indexType == PropIndexType::Key ? ParenthesesRecommendation::None : getParenRecommendation(type, nodes, typeCorrect); @@ -692,17 +748,31 @@ std::optional returnFirstNonnullOptionOfType(const UnionTypeVar* utv) return ret; } -static std::optional functionIsExpectedAt(const Module& module, AstNode* node) +static std::optional functionIsExpectedAt(const Module& module, AstNode* node, Position position) { - auto expr = node->asExpr(); - if (!expr) - return std::nullopt; + TypeId expectedType; - auto it = module.astExpectedTypes.find(expr); - if (!it) - return std::nullopt; + if (FFlag::LuauAutocompleteFirstArg) + { + auto typeAtPosition = findExpectedTypeAt(module, node, position); - TypeId expectedType = follow(*it); + if (!typeAtPosition) + return std::nullopt; + + expectedType = follow(*typeAtPosition); + } + else + { + auto expr = node->asExpr(); + if (!expr) + return std::nullopt; + + auto it = module.astExpectedTypes.find(expr); + if (!it) + return std::nullopt; + + expectedType = follow(*it); + } if (get(expectedType)) return true; @@ -1171,7 +1241,7 @@ static void autocompleteExpression(const SourceModule& sourceModule, const Modul std::string n = toString(name); if (!result.count(n)) { - TypeCorrectKind typeCorrect = checkTypeCorrectKind(module, typeArena, node, binding.typeId); + TypeCorrectKind typeCorrect = checkTypeCorrectKind(module, typeArena, node, position, binding.typeId); result[n] = {AutocompleteEntryKind::Binding, binding.typeId, binding.deprecated, false, typeCorrect, std::nullopt, std::nullopt, binding.documentationSymbol, {}, getParenRecommendation(binding.typeId, ancestry, typeCorrect)}; @@ -1181,9 +1251,10 @@ static void autocompleteExpression(const SourceModule& sourceModule, const Modul scope = scope->parent; } - TypeCorrectKind correctForNil = checkTypeCorrectKind(module, typeArena, node, typeChecker.nilType); - TypeCorrectKind correctForBoolean = checkTypeCorrectKind(module, typeArena, node, typeChecker.booleanType); - TypeCorrectKind correctForFunction = functionIsExpectedAt(module, node).value_or(false) ? TypeCorrectKind::Correct : TypeCorrectKind::None; + TypeCorrectKind correctForNil = checkTypeCorrectKind(module, typeArena, node, position, typeChecker.nilType); + TypeCorrectKind correctForBoolean = checkTypeCorrectKind(module, typeArena, node, position, typeChecker.booleanType); + TypeCorrectKind correctForFunction = + functionIsExpectedAt(module, node, position).value_or(false) ? TypeCorrectKind::Correct : TypeCorrectKind::None; if (FFlag::LuauIfElseExpressionAnalysisSupport) result["if"] = {AutocompleteEntryKind::Keyword, std::nullopt, false, false}; diff --git a/Analysis/src/BuiltinDefinitions.cpp b/Analysis/src/BuiltinDefinitions.cpp index bac94a2b..d527414a 100644 --- a/Analysis/src/BuiltinDefinitions.cpp +++ b/Analysis/src/BuiltinDefinitions.cpp @@ -217,9 +217,9 @@ void registerBuiltinTypes(TypeChecker& typeChecker) TypeId genericK = arena.addType(GenericTypeVar{"K"}); TypeId genericV = arena.addType(GenericTypeVar{"V"}); - TypeId mapOfKtoV = arena.addType(TableTypeVar{{}, TableIndexer(genericK, genericV), typeChecker.globalScope->level}); + TypeId mapOfKtoV = arena.addType(TableTypeVar{{}, TableIndexer(genericK, genericV), typeChecker.globalScope->level, TableState::Generic}); - std::optional stringMetatableTy = getMetatable(singletonTypes.stringType); + std::optional stringMetatableTy = getMetatable(getSingletonTypes().stringType); LUAU_ASSERT(stringMetatableTy); const TableTypeVar* stringMetatableTable = get(follow(*stringMetatableTy)); LUAU_ASSERT(stringMetatableTable); @@ -271,7 +271,10 @@ void registerBuiltinTypes(TypeChecker& typeChecker) persist(pair.second.typeId); if (TableTypeVar* ttv = getMutable(pair.second.typeId)) - ttv->name = toString(pair.first); + { + if (!ttv->name) + ttv->name = toString(pair.first); + } } attachMagicFunction(getGlobalBinding(typeChecker, "assert"), magicFunctionAssert); diff --git a/Analysis/src/EmbeddedBuiltinDefinitions.cpp b/Analysis/src/EmbeddedBuiltinDefinitions.cpp index 9f5c8250..d0afa742 100644 --- a/Analysis/src/EmbeddedBuiltinDefinitions.cpp +++ b/Analysis/src/EmbeddedBuiltinDefinitions.cpp @@ -1,6 +1,8 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #include "Luau/BuiltinDefinitions.h" +LUAU_FASTFLAGVARIABLE(LuauFixTonumberReturnType, false) + namespace Luau { @@ -113,7 +115,6 @@ declare function gcinfo(): number declare function error(message: T, level: number?) declare function tostring(value: T): string - declare function tonumber(value: T, radix: number?): number declare function rawequal(a: T1, b: T2): boolean declare function rawget(tab: {[K]: V}, k: K): V @@ -204,7 +205,14 @@ declare function gcinfo(): number std::string getBuiltinDefinitionSource() { - return kBuiltinDefinitionLuaSrc; + std::string result = kBuiltinDefinitionLuaSrc; + + if (FFlag::LuauFixTonumberReturnType) + result += "declare function tonumber(value: T, radix: number?): number?\n"; + else + result += "declare function tonumber(value: T, radix: number?): number\n"; + + return result; } } // namespace Luau diff --git a/Analysis/src/Error.cpp b/Analysis/src/Error.cpp index 8334bd62..ce832c6b 100644 --- a/Analysis/src/Error.cpp +++ b/Analysis/src/Error.cpp @@ -58,7 +58,7 @@ struct ErrorConverter result += "\ncaused by:\n "; if (!tm.reason.empty()) - result += tm.reason + ". "; + result += tm.reason + " "; result += Luau::toString(*tm.error); } @@ -410,6 +410,11 @@ struct ErrorConverter return ss + " in the type '" + toString(e.type) + "'"; } + + std::string operator()(const TypesAreUnrelated& e) const + { + return "Cannot cast '" + toString(e.left) + "' into '" + toString(e.right) + "' because the types are unrelated"; + } }; struct InvalidNameChecker @@ -658,6 +663,11 @@ bool MissingUnionProperty::operator==(const MissingUnionProperty& rhs) const return *type == *rhs.type && key == rhs.key; } +bool TypesAreUnrelated::operator==(const TypesAreUnrelated& rhs) const +{ + return left == rhs.left && right == rhs.right; +} + std::string toString(const TypeError& error) { ErrorConverter converter; @@ -793,6 +803,11 @@ void copyError(T& e, TypeArena& destArena, SeenTypes& seenTypes, SeenTypePacks& for (auto& ty : e.missing) ty = clone(ty); } + else if constexpr (std::is_same_v) + { + e.left = clone(e.left); + e.right = clone(e.right); + } else static_assert(always_false_v, "Non-exhaustive type switch"); } diff --git a/Analysis/src/IostreamHelpers.cpp b/Analysis/src/IostreamHelpers.cpp index ac46b5a4..5bc76ade 100644 --- a/Analysis/src/IostreamHelpers.cpp +++ b/Analysis/src/IostreamHelpers.cpp @@ -262,6 +262,12 @@ std::ostream& operator<<(std::ostream& stream, const MissingUnionProperty& error return stream << " }, key = '" + error.key + "' }"; } +std::ostream& operator<<(std::ostream& stream, const TypesAreUnrelated& error) +{ + stream << "TypesAreUnrelated { left = '" + toString(error.left) + "', right = '" + toString(error.right) + "' }"; + return stream; +} + std::ostream& operator<<(std::ostream& stream, const TableState& tv) { return stream << static_cast::type>(tv); diff --git a/Analysis/src/JsonEncoder.cpp b/Analysis/src/JsonEncoder.cpp index af6bffc4..23491a5a 100644 --- a/Analysis/src/JsonEncoder.cpp +++ b/Analysis/src/JsonEncoder.cpp @@ -379,7 +379,7 @@ struct AstJsonEncoder : public AstVisitor if (comma) writeRaw(","); else - comma = false; + comma = true; write(prop); } }); diff --git a/Analysis/src/Module.cpp b/Analysis/src/Module.cpp index b4b6eb42..e1e53c97 100644 --- a/Analysis/src/Module.cpp +++ b/Analysis/src/Module.cpp @@ -13,7 +13,6 @@ LUAU_FASTFLAGVARIABLE(DebugLuauFreezeArena, false) LUAU_FASTFLAGVARIABLE(DebugLuauTrackOwningArena, false) -LUAU_FASTFLAG(LuauCaptureBrokenCommentSpans) LUAU_FASTINTVARIABLE(LuauTypeCloneRecursionLimit, 0) namespace Luau @@ -23,7 +22,7 @@ static bool contains(Position pos, Comment comment) { if (comment.location.contains(pos)) return true; - else if (FFlag::LuauCaptureBrokenCommentSpans && comment.type == Lexeme::BrokenComment && + else if (comment.type == Lexeme::BrokenComment && comment.location.begin <= pos) // Broken comments are broken specifically because they don't have an end return true; else if (comment.type == Lexeme::Comment && comment.location.end == pos) @@ -194,7 +193,7 @@ struct TypePackCloner { cloneState.encounteredFreeType = true; - TypePackId err = singletonTypes.errorRecoveryTypePack(singletonTypes.anyTypePack); + TypePackId err = getSingletonTypes().errorRecoveryTypePack(getSingletonTypes().anyTypePack); TypePackId cloned = dest.addTypePack(*err); seenTypePacks[typePackId] = cloned; } @@ -247,7 +246,7 @@ void TypeCloner::defaultClone(const T& t) void TypeCloner::operator()(const Unifiable::Free& t) { cloneState.encounteredFreeType = true; - TypeId err = singletonTypes.errorRecoveryType(singletonTypes.anyType); + TypeId err = getSingletonTypes().errorRecoveryType(getSingletonTypes().anyType); TypeId cloned = dest.addType(*err); seenTypes[typeId] = cloned; } @@ -421,9 +420,6 @@ TypePackId clone(TypePackId tp, TypeArena& dest, SeenTypes& seenTypes, SeenTypeP Luau::visit(cloner, tp->ty); // Mutates the storage that 'res' points into. } - if (FFlag::DebugLuauTrackOwningArena) - asMutable(res)->owningArena = &dest; - return res; } @@ -440,12 +436,11 @@ TypeId clone(TypeId typeId, TypeArena& dest, SeenTypes& seenTypes, SeenTypePacks { TypeCloner cloner{dest, typeId, seenTypes, seenTypePacks, cloneState}; Luau::visit(cloner, typeId->ty); // Mutates the storage that 'res' points into. + + // TODO: Make this work when the arena of 'res' might be frozen asMutable(res)->documentationSymbol = typeId->documentationSymbol; } - if (FFlag::DebugLuauTrackOwningArena) - asMutable(res)->owningArena = &dest; - return res; } @@ -508,8 +503,8 @@ bool Module::clonePublicInterface() if (moduleScope->varargPack) moduleScope->varargPack = clone(*moduleScope->varargPack, interfaceTypes, seenTypes, seenTypePacks, cloneState); - for (auto& pair : moduleScope->exportedTypeBindings) - pair.second = clone(pair.second, interfaceTypes, seenTypes, seenTypePacks, cloneState); + for (auto& [name, tf] : moduleScope->exportedTypeBindings) + tf = clone(tf, interfaceTypes, seenTypes, seenTypePacks, cloneState); for (TypeId ty : moduleScope->returnType) if (get(follow(ty))) diff --git a/Analysis/src/Predicate.cpp b/Analysis/src/Predicate.cpp index 848627cf..7bd8001e 100644 --- a/Analysis/src/Predicate.cpp +++ b/Analysis/src/Predicate.cpp @@ -24,7 +24,7 @@ std::optional tryGetLValue(const AstExpr& node) else if (auto indexexpr = expr->as()) { if (auto lvalue = tryGetLValue(*indexexpr->expr)) - if (auto string = indexexpr->expr->as()) + if (auto string = indexexpr->index->as()) return Field{std::make_shared(*lvalue), std::string(string->value.data, string->value.size)}; } diff --git a/Analysis/src/ToString.cpp b/Analysis/src/ToString.cpp index 6322096c..a6be5348 100644 --- a/Analysis/src/ToString.cpp +++ b/Analysis/src/ToString.cpp @@ -13,6 +13,13 @@ LUAU_FASTFLAG(LuauOccursCheckOkWithRecursiveFunctions) LUAU_FASTFLAGVARIABLE(LuauFunctionArgumentNameSize, false) +/* + * Prefix generic typenames with gen- + * Additionally, free types will be prefixed with free- and suffixed with their level. eg free-a-4 + * Fair warning: Setting this will break a lot of Luau unit tests. + */ +LUAU_FASTFLAGVARIABLE(DebugLuauVerboseTypeNames, false) + namespace Luau { @@ -290,7 +297,15 @@ struct TypeVarStringifier void operator()(TypeId ty, const Unifiable::Free& ftv) { state.result.invalid = true; + if (FFlag::DebugLuauVerboseTypeNames) + state.emit("free-"); state.emit(state.getName(ty)); + + if (FFlag::DebugLuauVerboseTypeNames) + { + state.emit("-"); + state.emit(std::to_string(ftv.level.level)); + } } void operator()(TypeId, const BoundTypeVar& btv) @@ -802,6 +817,8 @@ struct TypePackStringifier void operator()(TypePackId tp, const GenericTypePack& pack) { + if (FFlag::DebugLuauVerboseTypeNames) + state.emit("gen-"); if (pack.explicitName) { state.result.nameMap.typePacks[tp] = pack.name; @@ -817,7 +834,16 @@ struct TypePackStringifier void operator()(TypePackId tp, const FreeTypePack& pack) { state.result.invalid = true; + if (FFlag::DebugLuauVerboseTypeNames) + state.emit("free-"); state.emit(state.getName(tp)); + + if (FFlag::DebugLuauVerboseTypeNames) + { + state.emit("-"); + state.emit(std::to_string(pack.level.level)); + } + state.emit("..."); } @@ -1181,20 +1207,24 @@ std::string toStringNamedFunction(const std::string& prefix, const FunctionTypeV return s; } -void dump(TypeId ty) +std::string dump(TypeId ty) { ToStringOptions opts; opts.exhaustive = true; opts.functionTypeArguments = true; - printf("%s\n", toString(ty, opts).c_str()); + std::string s = toString(ty, opts); + printf("%s\n", s.c_str()); + return s; } -void dump(TypePackId ty) +std::string dump(TypePackId ty) { ToStringOptions opts; opts.exhaustive = true; opts.functionTypeArguments = true; - printf("%s\n", toString(ty, opts).c_str()); + std::string s = toString(ty, opts); + printf("%s\n", s.c_str()); + return s; } std::string generateName(size_t i) diff --git a/Analysis/src/TypeInfer.cpp b/Analysis/src/TypeInfer.cpp index 617bf482..abbc2901 100644 --- a/Analysis/src/TypeInfer.cpp +++ b/Analysis/src/TypeInfer.cpp @@ -9,9 +9,9 @@ #include "Luau/Scope.h" #include "Luau/Substitution.h" #include "Luau/TopoSortStatements.h" -#include "Luau/ToString.h" #include "Luau/TypePack.h" #include "Luau/TypeUtils.h" +#include "Luau/ToString.h" #include "Luau/TypeVar.h" #include "Luau/TimeTrace.h" @@ -29,7 +29,6 @@ LUAU_FASTFLAGVARIABLE(LuauCloneCorrectlyBeforeMutatingTableType, false) LUAU_FASTFLAGVARIABLE(LuauStoreMatchingOverloadFnType, false) LUAU_FASTFLAGVARIABLE(LuauRecursiveTypeParameterRestriction, false) LUAU_FASTFLAGVARIABLE(LuauIfElseExpressionAnalysisSupport, false) -LUAU_FASTFLAGVARIABLE(LuauStrictRequire, false) LUAU_FASTFLAGVARIABLE(LuauQuantifyInPlace2, false) LUAU_FASTFLAGVARIABLE(LuauSingletonTypes, false) LUAU_FASTFLAGVARIABLE(LuauExpectedTypesOfProperties, false) @@ -37,6 +36,12 @@ LUAU_FASTFLAGVARIABLE(LuauErrorRecoveryType, false) LUAU_FASTFLAGVARIABLE(LuauPropertiesGetExpectedType, false) LUAU_FASTFLAGVARIABLE(LuauTailArgumentTypeInfo, false) LUAU_FASTFLAGVARIABLE(LuauModuleRequireErrorPack, false) +LUAU_FASTFLAGVARIABLE(LuauRefiLookupFromIndexExpr, false) +LUAU_FASTFLAGVARIABLE(LuauProperTypeLevels, false) +LUAU_FASTFLAGVARIABLE(LuauAscribeCorrectLevelToInferredProperitesOfFreeTables, false) +LUAU_FASTFLAGVARIABLE(LuauFixRecursiveMetatableCall, false) +LUAU_FASTFLAGVARIABLE(LuauBidirectionalAsExpr, false) +LUAU_FASTFLAGVARIABLE(LuauUpdateFunctionNameBinding, false) namespace Luau { @@ -206,14 +211,14 @@ TypeChecker::TypeChecker(ModuleResolver* resolver, InternalErrorReporter* iceHan : resolver(resolver) , iceHandler(iceHandler) , unifierState(iceHandler) - , nilType(singletonTypes.nilType) - , numberType(singletonTypes.numberType) - , stringType(singletonTypes.stringType) - , booleanType(singletonTypes.booleanType) - , threadType(singletonTypes.threadType) - , anyType(singletonTypes.anyType) - , optionalNumberType(singletonTypes.optionalNumberType) - , anyTypePack(singletonTypes.anyTypePack) + , nilType(getSingletonTypes().nilType) + , numberType(getSingletonTypes().numberType) + , stringType(getSingletonTypes().stringType) + , booleanType(getSingletonTypes().booleanType) + , threadType(getSingletonTypes().threadType) + , anyType(getSingletonTypes().anyType) + , optionalNumberType(getSingletonTypes().optionalNumberType) + , anyTypePack(getSingletonTypes().anyTypePack) { globalScope = std::make_shared(globalTypes.addTypePack(TypePackVar{FreeTypePack{TypeLevel{}}})); @@ -443,7 +448,7 @@ void TypeChecker::checkBlock(const ScopePtr& scope, const AstStatBlock& block) functionDecls[*protoIter] = pair; ++subLevel; - TypeId leftType = checkFunctionName(scope, *fun->name); + TypeId leftType = checkFunctionName(scope, *fun->name, funScope->level); unify(leftType, funTy, fun->location); } else if (auto fun = (*protoIter)->as()) @@ -711,14 +716,15 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatAssign& assign) } else if (auto tail = valueIter.tail()) { - if (get(*tail)) + TypePackId tailPack = follow(*tail); + if (get(tailPack)) right = errorRecoveryType(scope); - else if (auto vtp = get(*tail)) + else if (auto vtp = get(tailPack)) right = vtp->ty; - else if (get(*tail)) + else if (get(tailPack)) { - *asMutable(*tail) = TypePack{{left}}; - growingPack = getMutable(*tail); + *asMutable(tailPack) = TypePack{{left}}; + growingPack = getMutable(tailPack); } } @@ -1107,8 +1113,27 @@ void TypeChecker::check(const ScopePtr& scope, TypeId ty, const ScopePtr& funSco unify(leftType, ty, function.location); - if (leftTypeBinding) - *leftTypeBinding = follow(quantify(funScope, leftType, function.name->location)); + if (FFlag::LuauUpdateFunctionNameBinding) + { + LUAU_ASSERT(function.name->is() || function.name->is()); + + if (auto exprIndexName = function.name->as()) + { + if (auto typeIt = currentModule->astTypes.find(exprIndexName->expr)) + { + if (auto ttv = getMutableTableType(*typeIt)) + { + if (auto it = ttv->props.find(exprIndexName->index.value); it != ttv->props.end()) + it->second.type = follow(quantify(funScope, leftType, function.name->location)); + } + } + } + } + else + { + if (leftTypeBinding) + *leftTypeBinding = follow(quantify(funScope, leftType, function.name->location)); + } } } @@ -1148,8 +1173,10 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatTypeAlias& typealias } else { - ScopePtr aliasScope = - FFlag::LuauQuantifyInPlace2 ? childScope(scope, typealias.location, subLevel) : childScope(scope, typealias.location); + ScopePtr aliasScope = childScope(scope, typealias.location); + aliasScope->level = scope->level.incr(); + if (FFlag::LuauProperTypeLevels) + aliasScope->level.subLevel = subLevel; auto [generics, genericPacks] = createGenericTypes(aliasScope, scope->level, typealias, typealias.generics, typealias.genericPacks); @@ -1166,6 +1193,7 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatTypeAlias& typealias ice("Not predeclared"); ScopePtr aliasScope = childScope(scope, typealias.location); + aliasScope->level = scope->level.incr(); for (TypeId ty : binding->typeParams) { @@ -1505,9 +1533,9 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprCa else if (auto vtp = get(retPack)) return {vtp->ty, std::move(result.predicates)}; else if (get(retPack)) - ice("Unexpected abstract type pack!"); + ice("Unexpected abstract type pack!", expr.location); else - ice("Unknown TypePack type!"); + ice("Unknown TypePack type!", expr.location); } ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprIndexName& expr) @@ -1574,7 +1602,7 @@ std::optional TypeChecker::getIndexTypeFromType( } else if (tableType->state == TableState::Free) { - TypeId result = freshType(scope); + TypeId result = FFlag::LuauAscribeCorrectLevelToInferredProperitesOfFreeTables ? freshType(tableType->level) : freshType(scope); tableType->props[name] = {result}; return result; } @@ -1738,7 +1766,16 @@ TypeId TypeChecker::stripFromNilAndReport(TypeId ty, const Location& location) ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprIndexExpr& expr) { - return {checkLValue(scope, expr)}; + TypeId ty = checkLValue(scope, expr); + + if (FFlag::LuauRefiLookupFromIndexExpr) + { + if (std::optional lvalue = tryGetLValue(expr)) + if (std::optional refiTy = resolveLValue(scope, *lvalue)) + return {*refiTy, {TruthyPredicate{std::move(*lvalue), expr.location}}}; + } + + return {ty}; } ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprFunction& expr, std::optional expectedType) @@ -2421,12 +2458,27 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprTy TypeId annotationType = resolveType(scope, *expr.annotation); ExprResult result = checkExpr(scope, *expr.expr, annotationType); - ErrorVec errorVec = canUnify(result.type, annotationType, expr.location); - reportErrors(errorVec); - if (!errorVec.empty()) - annotationType = errorRecoveryType(annotationType); + if (FFlag::LuauBidirectionalAsExpr) + { + // Note: As an optimization, we try 'number <: number | string' first, as that is the more likely case. + if (canUnify(result.type, annotationType, expr.location).empty()) + return {annotationType, std::move(result.predicates)}; - return {annotationType, std::move(result.predicates)}; + if (canUnify(annotationType, result.type, expr.location).empty()) + return {annotationType, std::move(result.predicates)}; + + reportError(expr.location, TypesAreUnrelated{result.type, annotationType}); + return {errorRecoveryType(annotationType), std::move(result.predicates)}; + } + else + { + ErrorVec errorVec = canUnify(result.type, annotationType, expr.location); + reportErrors(errorVec); + if (!errorVec.empty()) + annotationType = errorRecoveryType(annotationType); + + return {annotationType, std::move(result.predicates)}; + } } ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprError& expr) @@ -2674,8 +2726,15 @@ std::pair TypeChecker::checkLValueBinding(const ScopePtr& scope // Answers the question: "Can I define another function with this name?" // Primarily about detecting duplicates. -TypeId TypeChecker::checkFunctionName(const ScopePtr& scope, AstExpr& funName) +TypeId TypeChecker::checkFunctionName(const ScopePtr& scope, AstExpr& funName, TypeLevel level) { + auto freshTy = [&]() { + if (FFlag::LuauProperTypeLevels) + return freshType(level); + else + return freshType(scope); + }; + if (auto globalName = funName.as()) { const ScopePtr& globalScope = currentModule->getModuleScope(); @@ -2689,7 +2748,7 @@ TypeId TypeChecker::checkFunctionName(const ScopePtr& scope, AstExpr& funName) } else { - TypeId ty = freshType(scope); + TypeId ty = freshTy(); globalScope->bindings[name] = {ty, funName.location}; return ty; } @@ -2699,7 +2758,7 @@ TypeId TypeChecker::checkFunctionName(const ScopePtr& scope, AstExpr& funName) Symbol name = localName->local; Binding& binding = scope->bindings[name]; if (binding.typeId == nullptr) - binding = {freshType(scope), funName.location}; + binding = {freshTy(), funName.location}; return binding.typeId; } @@ -2730,7 +2789,7 @@ TypeId TypeChecker::checkFunctionName(const ScopePtr& scope, AstExpr& funName) Property& property = ttv->props[name]; - property.type = freshType(scope); + property.type = freshTy(); property.location = indexName->indexLocation; ttv->methodDefinitionLocations[name] = funName.location; return property.type; @@ -3327,7 +3386,7 @@ ExprResult TypeChecker::checkExprPack(const ScopePtr& scope, const A fn = follow(fn); if (auto ret = checkCallOverload( - scope, expr, fn, retPack, argPack, args, argLocations, argListResult, overloadsThatMatchArgCount, overloadsThatDont, errors)) + scope, expr, fn, retPack, argPack, args, &argLocations, argListResult, overloadsThatMatchArgCount, overloadsThatDont, errors)) return *ret; } @@ -3402,9 +3461,11 @@ std::vector> TypeChecker::getExpectedTypesForCall(const st } std::optional> TypeChecker::checkCallOverload(const ScopePtr& scope, const AstExprCall& expr, TypeId fn, TypePackId retPack, - TypePackId argPack, TypePack* args, const std::vector& argLocations, const ExprResult& argListResult, + TypePackId argPack, TypePack* args, const std::vector* argLocations, const ExprResult& argListResult, std::vector& overloadsThatMatchArgCount, std::vector& overloadsThatDont, std::vector& errors) { + LUAU_ASSERT(argLocations); + fn = stripFromNilAndReport(fn, expr.func->location); if (get(fn)) @@ -3428,31 +3489,44 @@ std::optional> TypeChecker::checkCallOverload(const Scope return {{retPack}}; } - const FunctionTypeVar* ftv = get(fn); - if (!ftv) + std::vector metaArgLocations; + + // Might be a callable table + if (const MetatableTypeVar* mttv = get(fn)) { - // Might be a callable table - if (const MetatableTypeVar* mttv = get(fn)) + if (std::optional ty = getIndexTypeFromType(scope, mttv->metatable, "__call", expr.func->location, false)) { - if (std::optional ty = getIndexTypeFromType(scope, mttv->metatable, "__call", expr.func->location, false)) + // Construct arguments with 'self' added in front + TypePackId metaCallArgPack = addTypePack(TypePackVar(TypePack{args->head, args->tail})); + + TypePack* metaCallArgs = getMutable(metaCallArgPack); + metaCallArgs->head.insert(metaCallArgs->head.begin(), fn); + + metaArgLocations = *argLocations; + metaArgLocations.insert(metaArgLocations.begin(), expr.func->location); + + if (FFlag::LuauFixRecursiveMetatableCall) { - // Construct arguments with 'self' added in front - TypePackId metaCallArgPack = addTypePack(TypePackVar(TypePack{args->head, args->tail})); - - TypePack* metaCallArgs = getMutable(metaCallArgPack); - metaCallArgs->head.insert(metaCallArgs->head.begin(), fn); - - std::vector metaArgLocations = argLocations; - metaArgLocations.insert(metaArgLocations.begin(), expr.func->location); + fn = instantiate(scope, *ty, expr.func->location); + argPack = metaCallArgPack; + args = metaCallArgs; + argLocations = &metaArgLocations; + } + else + { TypeId fn = *ty; fn = instantiate(scope, fn, expr.func->location); - return checkCallOverload(scope, expr, fn, retPack, metaCallArgPack, metaCallArgs, metaArgLocations, argListResult, + return checkCallOverload(scope, expr, fn, retPack, metaCallArgPack, metaCallArgs, &metaArgLocations, argListResult, overloadsThatMatchArgCount, overloadsThatDont, errors); } } + } + const FunctionTypeVar* ftv = get(fn); + if (!ftv) + { reportError(TypeError{expr.func->location, CannotCallNonFunction{fn}}); unify(retPack, errorRecoveryTypePack(scope), expr.func->location); return {{errorRecoveryTypePack(retPack)}}; @@ -3477,7 +3551,7 @@ std::optional> TypeChecker::checkCallOverload(const Scope return {}; } - checkArgumentList(scope, state, argPack, ftv->argTypes, argLocations); + checkArgumentList(scope, state, argPack, ftv->argTypes, *argLocations); if (!state.errors.empty()) { @@ -3772,7 +3846,7 @@ TypeId TypeChecker::checkRequire(const ScopePtr& scope, const ModuleInfo& module if (moduleInfo.name.empty()) { - if (FFlag::LuauStrictRequire && currentModule->mode == Mode::Strict) + if (currentModule->mode == Mode::Strict) { reportError(TypeError{location, UnknownRequire{}}); return errorRecoveryType(anyType); @@ -4268,9 +4342,11 @@ ScopePtr TypeChecker::childFunctionScope(const ScopePtr& parent, const Location& } // Creates a new Scope and carries forward the varargs from the parent. -ScopePtr TypeChecker::childScope(const ScopePtr& parent, const Location& location, int subLevel) +ScopePtr TypeChecker::childScope(const ScopePtr& parent, const Location& location) { - ScopePtr scope = std::make_shared(parent, subLevel); + ScopePtr scope = std::make_shared(parent); + if (FFlag::LuauProperTypeLevels) + scope->level = parent->level; scope->varargPack = parent->varargPack; currentModule->scopes.push_back(std::make_pair(location, scope)); @@ -4329,22 +4405,22 @@ TypeId TypeChecker::singletonType(std::string value) TypeId TypeChecker::errorRecoveryType(const ScopePtr& scope) { - return singletonTypes.errorRecoveryType(); + return getSingletonTypes().errorRecoveryType(); } TypeId TypeChecker::errorRecoveryType(TypeId guess) { - return singletonTypes.errorRecoveryType(guess); + return getSingletonTypes().errorRecoveryType(guess); } TypePackId TypeChecker::errorRecoveryTypePack(const ScopePtr& scope) { - return singletonTypes.errorRecoveryTypePack(); + return getSingletonTypes().errorRecoveryTypePack(); } TypePackId TypeChecker::errorRecoveryTypePack(TypePackId guess) { - return singletonTypes.errorRecoveryTypePack(guess); + return getSingletonTypes().errorRecoveryTypePack(guess); } std::optional TypeChecker::filterMap(TypeId type, TypeIdPredicate predicate) @@ -4547,6 +4623,7 @@ TypeId TypeChecker::resolveType(const ScopePtr& scope, const AstType& annotation else if (const auto& func = annotation.as()) { ScopePtr funcScope = childScope(scope, func->location); + funcScope->level = scope->level.incr(); auto [generics, genericPacks] = createGenericTypes(funcScope, std::nullopt, annotation, func->generics, func->genericPacks); diff --git a/Analysis/src/TypeUtils.cpp b/Analysis/src/TypeUtils.cpp index 0d9d91e0..8c6d5e49 100644 --- a/Analysis/src/TypeUtils.cpp +++ b/Analysis/src/TypeUtils.cpp @@ -19,7 +19,7 @@ std::optional findMetatableEntry(ErrorVec& errors, const ScopePtr& globa TypeId unwrapped = follow(*metatable); if (get(unwrapped)) - return singletonTypes.anyType; + return getSingletonTypes().anyType; const TableTypeVar* mtt = getTableType(unwrapped); if (!mtt) @@ -61,12 +61,12 @@ std::optional findTablePropertyRespectingMeta(ErrorVec& errors, const Sc { std::optional r = first(follow(itf->retType)); if (!r) - return singletonTypes.nilType; + return getSingletonTypes().nilType; else return *r; } else if (get(index)) - return singletonTypes.anyType; + return getSingletonTypes().anyType; else errors.push_back(TypeError{location, GenericError{"__index should either be a function or table. Got " + toString(index)}}); diff --git a/Analysis/src/TypeVar.cpp b/Analysis/src/TypeVar.cpp index 62715af5..571b13ca 100644 --- a/Analysis/src/TypeVar.cpp +++ b/Analysis/src/TypeVar.cpp @@ -21,6 +21,7 @@ LUAU_FASTINTVARIABLE(LuauTypeMaximumStringifierLength, 500) LUAU_FASTINTVARIABLE(LuauTableTypeMaximumStringifierLength, 0) LUAU_FASTFLAGVARIABLE(LuauRefactorTagging, false) LUAU_FASTFLAG(LuauErrorRecoveryType) +LUAU_FASTFLAG(DebugLuauFreezeArena) namespace Luau { @@ -579,11 +580,25 @@ SingletonTypes::SingletonTypes() , arena(new TypeArena) { TypeId stringMetatable = makeStringMetatable(); - stringType_.ty = PrimitiveTypeVar{PrimitiveTypeVar::String, makeStringMetatable()}; + stringType_.ty = PrimitiveTypeVar{PrimitiveTypeVar::String, stringMetatable}; persist(stringMetatable); + + debugFreezeArena = FFlag::DebugLuauFreezeArena; freeze(*arena); } +SingletonTypes::~SingletonTypes() +{ + // Destroy the arena with the same memory management flags it was created with + bool prevFlag = FFlag::DebugLuauFreezeArena; + FFlag::DebugLuauFreezeArena.value = debugFreezeArena; + + unfreeze(*arena); + arena.reset(nullptr); + + FFlag::DebugLuauFreezeArena.value = prevFlag; +} + TypeId SingletonTypes::makeStringMetatable() { const TypeId optionalNumber = arena->addType(UnionTypeVar{{nilType, numberType}}); @@ -641,6 +656,9 @@ TypeId SingletonTypes::makeStringMetatable() TypeId tableType = arena->addType(TableTypeVar{std::move(stringLib), std::nullopt, TypeLevel{}, TableState::Sealed}); + if (TableTypeVar* ttv = getMutable(tableType)) + ttv->name = "string"; + return arena->addType(TableTypeVar{{{{"__index", {tableType}}}}, std::nullopt, TypeLevel{}, TableState::Sealed}); } @@ -670,7 +688,11 @@ TypePackId SingletonTypes::errorRecoveryTypePack(TypePackId guess) return &errorTypePack_; } -SingletonTypes singletonTypes; +SingletonTypes& getSingletonTypes() +{ + static SingletonTypes singletonTypes; + return singletonTypes; +} void persist(TypeId ty) { @@ -719,6 +741,18 @@ void persist(TypeId ty) for (TypeId opt : itv->parts) queue.push_back(opt); } + else if (auto mtv = get(t)) + { + queue.push_back(mtv->table); + queue.push_back(mtv->metatable); + } + else if (get(t) || get(t) || get(t) || get(t) || get(t)) + { + } + else + { + LUAU_ASSERT(!"TypeId is not supported in a persist call"); + } } } @@ -736,6 +770,17 @@ void persist(TypePackId tp) if (p->tail) persist(*p->tail); } + else if (auto vtp = get(tp)) + { + persist(vtp->ty); + } + else if (get(tp)) + { + } + else + { + LUAU_ASSERT(!"TypePackId is not supported in a persist call"); + } } const TypeLevel* getLevel(TypeId ty) @@ -757,167 +802,6 @@ TypeLevel* getMutableLevel(TypeId ty) return const_cast(getLevel(ty)); } -struct QVarFinder -{ - mutable DenseHashSet seen; - - QVarFinder() - : seen(nullptr) - { - } - - bool hasSeen(const void* tv) const - { - if (seen.contains(tv)) - return true; - - seen.insert(tv); - return false; - } - - bool hasGeneric(TypeId tid) const - { - if (hasSeen(&tid->ty)) - return false; - - return Luau::visit(*this, tid->ty); - } - - bool hasGeneric(TypePackId tp) const - { - if (hasSeen(&tp->ty)) - return false; - - return Luau::visit(*this, tp->ty); - } - - bool operator()(const Unifiable::Free&) const - { - return false; - } - - bool operator()(const Unifiable::Bound& bound) const - { - return hasGeneric(bound.boundTo); - } - - bool operator()(const Unifiable::Generic&) const - { - return true; - } - bool operator()(const Unifiable::Error&) const - { - return false; - } - bool operator()(const PrimitiveTypeVar&) const - { - return false; - } - - bool operator()(const SingletonTypeVar&) const - { - return false; - } - - bool operator()(const FunctionTypeVar& ftv) const - { - if (hasGeneric(ftv.argTypes)) - return true; - return hasGeneric(ftv.retType); - } - - bool operator()(const TableTypeVar& ttv) const - { - if (ttv.state == TableState::Generic) - return true; - - if (ttv.indexer) - { - if (hasGeneric(ttv.indexer->indexType)) - return true; - if (hasGeneric(ttv.indexer->indexResultType)) - return true; - } - - for (const auto& [_name, prop] : ttv.props) - { - if (hasGeneric(prop.type)) - return true; - } - - return false; - } - - bool operator()(const MetatableTypeVar& mtv) const - { - return hasGeneric(mtv.table) || hasGeneric(mtv.metatable); - } - - bool operator()(const ClassTypeVar& ctv) const - { - for (const auto& [name, prop] : ctv.props) - { - if (hasGeneric(prop.type)) - return true; - } - - if (ctv.parent) - return hasGeneric(*ctv.parent); - - return false; - } - - bool operator()(const AnyTypeVar&) const - { - return false; - } - - bool operator()(const UnionTypeVar& utv) const - { - for (TypeId tid : utv.options) - if (hasGeneric(tid)) - return true; - - return false; - } - - bool operator()(const IntersectionTypeVar& utv) const - { - for (TypeId tid : utv.parts) - if (hasGeneric(tid)) - return true; - - return false; - } - - bool operator()(const LazyTypeVar&) const - { - return false; - } - - bool operator()(const Unifiable::Bound& bound) const - { - return hasGeneric(bound.boundTo); - } - - bool operator()(const TypePack& pack) const - { - for (TypeId ty : pack.head) - if (hasGeneric(ty)) - return true; - - if (pack.tail) - return hasGeneric(*pack.tail); - - return false; - } - - bool operator()(const VariadicTypePack& pack) const - { - return hasGeneric(pack.ty); - } -}; - const Property* lookupClassProp(const ClassTypeVar* cls, const Name& name) { while (cls) @@ -953,16 +837,6 @@ bool isSubclass(const ClassTypeVar* cls, const ClassTypeVar* parent) return false; } -bool hasGeneric(TypeId ty) -{ - return Luau::visit(QVarFinder{}, ty->ty); -} - -bool hasGeneric(TypePackId tp) -{ - return Luau::visit(QVarFinder{}, tp->ty); -} - UnionTypeVarIterator::UnionTypeVarIterator(const UnionTypeVar* utv) { LUAU_ASSERT(utv); diff --git a/Analysis/src/Unifier.cpp b/Analysis/src/Unifier.cpp index d0b18837..c5aab856 100644 --- a/Analysis/src/Unifier.cpp +++ b/Analysis/src/Unifier.cpp @@ -14,7 +14,7 @@ LUAU_FASTINT(LuauTypeInferRecursionLimit); LUAU_FASTINT(LuauTypeInferTypePackLoopLimit); LUAU_FASTINTVARIABLE(LuauTypeInferIterationLimit, 2000); -LUAU_FASTFLAGVARIABLE(LuauTableSubtypingVariance, false); +LUAU_FASTFLAGVARIABLE(LuauTableSubtypingVariance2, false); LUAU_FASTFLAGVARIABLE(LuauUnionHeuristic, false) LUAU_FASTFLAGVARIABLE(LuauTableUnificationEarlyTest, false) LUAU_FASTFLAGVARIABLE(LuauOccursCheckOkWithRecursiveFunctions, false) @@ -22,9 +22,82 @@ LUAU_FASTFLAGVARIABLE(LuauExtendedTypeMismatchError, false) LUAU_FASTFLAG(LuauSingletonTypes) LUAU_FASTFLAGVARIABLE(LuauExtendedClassMismatchError, false) LUAU_FASTFLAG(LuauErrorRecoveryType); +LUAU_FASTFLAG(LuauProperTypeLevels); +LUAU_FASTFLAGVARIABLE(LuauExtendedUnionMismatchError, false) +LUAU_FASTFLAGVARIABLE(LuauExtendedFunctionMismatchError, false) namespace Luau { + +struct PromoteTypeLevels +{ + TxnLog& log; + TypeLevel minLevel; + + explicit PromoteTypeLevels(TxnLog& log, TypeLevel minLevel) + : log(log) + , minLevel(minLevel) + {} + + template + void promote(TID ty, T* t) + { + LUAU_ASSERT(t); + if (minLevel.subsumesStrict(t->level)) + { + log(ty); + t->level = minLevel; + } + } + + template + void cycle(TID) {} + + template + bool operator()(TID, const T&) + { + return true; + } + + bool operator()(TypeId ty, const FreeTypeVar&) + { + promote(ty, getMutable(ty)); + return true; + } + + bool operator()(TypeId ty, const FunctionTypeVar&) + { + promote(ty, getMutable(ty)); + return true; + } + + bool operator()(TypeId ty, const TableTypeVar&) + { + promote(ty, getMutable(ty)); + return true; + } + + bool operator()(TypePackId tp, const FreeTypePack&) + { + promote(tp, getMutable(tp)); + return true; + } +}; + +void promoteTypeLevels(TxnLog& log, TypeLevel minLevel, TypeId ty) +{ + PromoteTypeLevels ptl{log, minLevel}; + DenseHashSet seen{nullptr}; + visitTypeVarOnce(ty, ptl, seen); +} + +void promoteTypeLevels(TxnLog& log, TypeLevel minLevel, TypePackId tp) +{ + PromoteTypeLevels ptl{log, minLevel}; + DenseHashSet seen{nullptr}; + visitTypeVarOnce(tp, ptl, seen); +} + struct SkipCacheForType { SkipCacheForType(const DenseHashMap& skipCacheForType) @@ -127,6 +200,29 @@ static std::optional hasUnificationTooComplex(const ErrorVec& errors) return *it; } +// Used for tagged union matching heuristic, returns first singleton type field +static std::optional> getTableMatchTag(TypeId type) +{ + LUAU_ASSERT(FFlag::LuauExtendedUnionMismatchError); + + type = follow(type); + + if (auto ttv = get(type)) + { + for (auto&& [name, prop] : ttv->props) + { + if (auto sing = get(follow(prop.type))) + return {{name, sing}}; + } + } + else if (auto mttv = get(type)) + { + return getTableMatchTag(mttv->table); + } + + return std::nullopt; +} + Unifier::Unifier(TypeArena* types, Mode mode, ScopePtr globalScope, const Location& location, Variance variance, UnifierSharedState& sharedState) : types(types) , mode(mode) @@ -214,9 +310,11 @@ void Unifier::tryUnify_(TypeId superTy, TypeId subTy, bool isFunctionCall, bool { occursCheck(superTy, subTy); + TypeLevel superLevel = l->level; + // Unification can't change the level of a generic. auto rightGeneric = get(subTy); - if (rightGeneric && !rightGeneric->level.subsumes(l->level)) + if (rightGeneric && !rightGeneric->level.subsumes(superLevel)) { // TODO: a more informative error message? CLI-39912 errors.push_back(TypeError{location, GenericError{"Generic subtype escaping scope"}}); @@ -226,7 +324,9 @@ void Unifier::tryUnify_(TypeId superTy, TypeId subTy, bool isFunctionCall, bool // The occurrence check might have caused superTy no longer to be a free type if (!get(superTy)) { - if (auto rightLevel = getMutableLevel(subTy)) + if (FFlag::LuauProperTypeLevels) + promoteTypeLevels(log, superLevel, subTy); + else if (auto rightLevel = getMutableLevel(subTy)) { if (!rightLevel->subsumes(l->level)) *rightLevel = l->level; @@ -240,6 +340,8 @@ void Unifier::tryUnify_(TypeId superTy, TypeId subTy, bool isFunctionCall, bool } else if (r) { + TypeLevel subLevel = r->level; + occursCheck(subTy, superTy); // Unification can't change the level of a generic. @@ -253,10 +355,16 @@ void Unifier::tryUnify_(TypeId superTy, TypeId subTy, bool isFunctionCall, bool if (!get(subTy)) { - if (auto leftLevel = getMutableLevel(superTy)) + if (FFlag::LuauProperTypeLevels) + promoteTypeLevels(log, subLevel, superTy); + + if (auto superLevel = getMutableLevel(superTy)) { - if (!leftLevel->subsumes(r->level)) - *leftLevel = r->level; + if (!superLevel->subsumes(r->level)) + { + log(superTy); + *superLevel = r->level; + } } log(subTy); @@ -327,7 +435,7 @@ void Unifier::tryUnify_(TypeId superTy, TypeId subTy, bool isFunctionCall, bool else if (failed) { if (FFlag::LuauExtendedTypeMismatchError && firstFailedOption) - errors.push_back(TypeError{location, TypeMismatch{superTy, subTy, "Not all union options are compatible", *firstFailedOption}}); + errors.push_back(TypeError{location, TypeMismatch{superTy, subTy, "Not all union options are compatible.", *firstFailedOption}}); else errors.push_back(TypeError{location, TypeMismatch{superTy, subTy}}); } @@ -338,28 +446,46 @@ void Unifier::tryUnify_(TypeId superTy, TypeId subTy, bool isFunctionCall, bool bool found = false; std::optional unificationTooComplex; + size_t failedOptionCount = 0; + std::optional failedOption; + + bool foundHeuristic = false; size_t startIndex = 0; if (FFlag::LuauUnionHeuristic) { - bool found = false; - - const std::string* subName = getName(subTy); - if (subName) + if (const std::string* subName = getName(subTy)) { for (size_t i = 0; i < uv->options.size(); ++i) { const std::string* optionName = getName(uv->options[i]); if (optionName && *optionName == *subName) { - found = true; + foundHeuristic = true; startIndex = i; break; } } } - if (!found && cacheEnabled) + if (FFlag::LuauExtendedUnionMismatchError) + { + if (auto subMatchTag = getTableMatchTag(subTy)) + { + for (size_t i = 0; i < uv->options.size(); ++i) + { + auto optionMatchTag = getTableMatchTag(uv->options[i]); + if (optionMatchTag && optionMatchTag->first == subMatchTag->first && *optionMatchTag->second == *subMatchTag->second) + { + foundHeuristic = true; + startIndex = i; + break; + } + } + } + } + + if (!foundHeuristic && cacheEnabled) { for (size_t i = 0; i < uv->options.size(); ++i) { @@ -390,15 +516,27 @@ void Unifier::tryUnify_(TypeId superTy, TypeId subTy, bool isFunctionCall, bool { unificationTooComplex = e; } + else if (FFlag::LuauExtendedUnionMismatchError && !isNil(type)) + { + failedOptionCount++; + + if (!failedOption) + failedOption = {innerState.errors.front()}; + } innerState.log.rollback(); } if (unificationTooComplex) + { errors.push_back(*unificationTooComplex); + } else if (!found) { - if (FFlag::LuauExtendedTypeMismatchError) + if (FFlag::LuauExtendedUnionMismatchError && (failedOptionCount == 1 || foundHeuristic) && failedOption) + errors.push_back( + TypeError{location, TypeMismatch{superTy, subTy, "None of the union options are compatible. For example:", *failedOption}}); + else if (FFlag::LuauExtendedTypeMismatchError) errors.push_back(TypeError{location, TypeMismatch{superTy, subTy, "none of the union options are compatible"}}); else errors.push_back(TypeError{location, TypeMismatch{superTy, subTy}}); @@ -431,7 +569,7 @@ void Unifier::tryUnify_(TypeId superTy, TypeId subTy, bool isFunctionCall, bool if (unificationTooComplex) errors.push_back(*unificationTooComplex); else if (firstFailedOption) - errors.push_back(TypeError{location, TypeMismatch{superTy, subTy, "Not all intersection parts are compatible", *firstFailedOption}}); + errors.push_back(TypeError{location, TypeMismatch{superTy, subTy, "Not all intersection parts are compatible.", *firstFailedOption}}); } else { @@ -771,6 +909,10 @@ void Unifier::tryUnify_(TypePackId superTp, TypePackId subTp, bool isFunctionCal if (superIter.good() && subIter.good()) { tryUnify_(*superIter, *subIter); + + if (FFlag::LuauExtendedFunctionMismatchError && !errors.empty() && !firstPackErrorPos) + firstPackErrorPos = loopCount; + superIter.advance(); subIter.advance(); continue; @@ -853,13 +995,13 @@ void Unifier::tryUnify_(TypePackId superTp, TypePackId subTp, bool isFunctionCal while (superIter.good()) { - tryUnify_(singletonTypes.errorRecoveryType(), *superIter); + tryUnify_(getSingletonTypes().errorRecoveryType(), *superIter); superIter.advance(); } while (subIter.good()) { - tryUnify_(singletonTypes.errorRecoveryType(), *subIter); + tryUnify_(getSingletonTypes().errorRecoveryType(), *subIter); subIter.advance(); } @@ -917,14 +1059,22 @@ void Unifier::tryUnifyFunctions(TypeId superTy, TypeId subTy, bool isFunctionCal if (numGenerics != rf->generics.size()) { numGenerics = std::min(lf->generics.size(), rf->generics.size()); - errors.push_back(TypeError{location, TypeMismatch{superTy, subTy}}); + + if (FFlag::LuauExtendedFunctionMismatchError) + errors.push_back(TypeError{location, TypeMismatch{superTy, subTy, "different number of generic type parameters"}}); + else + errors.push_back(TypeError{location, TypeMismatch{superTy, subTy}}); } size_t numGenericPacks = lf->genericPacks.size(); if (numGenericPacks != rf->genericPacks.size()) { numGenericPacks = std::min(lf->genericPacks.size(), rf->genericPacks.size()); - errors.push_back(TypeError{location, TypeMismatch{superTy, subTy}}); + + if (FFlag::LuauExtendedFunctionMismatchError) + errors.push_back(TypeError{location, TypeMismatch{superTy, subTy, "different number of generic type pack parameters"}}); + else + errors.push_back(TypeError{location, TypeMismatch{superTy, subTy}}); } for (size_t i = 0; i < numGenerics; i++) @@ -936,13 +1086,49 @@ void Unifier::tryUnifyFunctions(TypeId superTy, TypeId subTy, bool isFunctionCal { Unifier innerState = makeChildUnifier(); - ctx = CountMismatch::Arg; - innerState.tryUnify_(rf->argTypes, lf->argTypes, isFunctionCall); + if (FFlag::LuauExtendedFunctionMismatchError) + { + innerState.ctx = CountMismatch::Arg; + innerState.tryUnify_(rf->argTypes, lf->argTypes, isFunctionCall); - ctx = CountMismatch::Result; - innerState.tryUnify_(lf->retType, rf->retType); + bool reported = !innerState.errors.empty(); - checkChildUnifierTypeMismatch(innerState.errors, superTy, subTy); + if (auto e = hasUnificationTooComplex(innerState.errors)) + errors.push_back(*e); + else if (!innerState.errors.empty() && innerState.firstPackErrorPos) + errors.push_back( + TypeError{location, TypeMismatch{superTy, subTy, format("Argument #%d type is not compatible.", *innerState.firstPackErrorPos), + innerState.errors.front()}}); + else if (!innerState.errors.empty()) + errors.push_back(TypeError{location, TypeMismatch{superTy, subTy, "", innerState.errors.front()}}); + + innerState.ctx = CountMismatch::Result; + innerState.tryUnify_(lf->retType, rf->retType); + + if (!reported) + { + if (auto e = hasUnificationTooComplex(innerState.errors)) + errors.push_back(*e); + else if (!innerState.errors.empty() && size(lf->retType) == 1 && finite(lf->retType)) + errors.push_back(TypeError{location, TypeMismatch{superTy, subTy, "Return type is not compatible.", innerState.errors.front()}}); + else if (!innerState.errors.empty() && innerState.firstPackErrorPos) + errors.push_back( + TypeError{location, TypeMismatch{superTy, subTy, format("Return #%d type is not compatible.", *innerState.firstPackErrorPos), + innerState.errors.front()}}); + else if (!innerState.errors.empty()) + errors.push_back(TypeError{location, TypeMismatch{superTy, subTy, "", innerState.errors.front()}}); + } + } + else + { + ctx = CountMismatch::Arg; + innerState.tryUnify_(rf->argTypes, lf->argTypes, isFunctionCall); + + ctx = CountMismatch::Result; + innerState.tryUnify_(lf->retType, rf->retType); + + checkChildUnifierTypeMismatch(innerState.errors, superTy, subTy); + } log.concat(std::move(innerState.log)); } @@ -994,7 +1180,7 @@ struct Resetter void Unifier::tryUnifyTables(TypeId left, TypeId right, bool isIntersection) { - if (!FFlag::LuauTableSubtypingVariance) + if (!FFlag::LuauTableSubtypingVariance2) return DEPRECATED_tryUnifyTables(left, right, isIntersection); TableTypeVar* lt = getMutable(left); @@ -1133,7 +1319,7 @@ void Unifier::tryUnifyTables(TypeId left, TypeId right, bool isIntersection) // TODO: hopefully readonly/writeonly properties will fix this. Property clone = prop; clone.type = deeplyOptional(clone.type); - log(lt); + log(left); lt->props[name] = clone; } else if (variance == Covariant) @@ -1146,7 +1332,7 @@ void Unifier::tryUnifyTables(TypeId left, TypeId right, bool isIntersection) } else if (lt->state == TableState::Free) { - log(lt); + log(left); lt->props[name] = prop; } else @@ -1176,7 +1362,7 @@ void Unifier::tryUnifyTables(TypeId left, TypeId right, bool isIntersection) // e.g. table.insert(t, 1) where t is a non-sealed table and doesn't have an indexer. // TODO: we only need to do this if the supertype's indexer is read/write // since that can add indexed elements. - log(rt); + log(right); rt->indexer = lt->indexer; } } @@ -1185,7 +1371,7 @@ void Unifier::tryUnifyTables(TypeId left, TypeId right, bool isIntersection) // Symmetric if we are invariant if (lt->state == TableState::Unsealed || lt->state == TableState::Free) { - log(lt); + log(left); lt->indexer = rt->indexer; } } @@ -1241,15 +1427,15 @@ TypeId Unifier::deeplyOptional(TypeId ty, std::unordered_map see TableTypeVar* resultTtv = getMutable(result); for (auto& [name, prop] : resultTtv->props) prop.type = deeplyOptional(prop.type, seen); - return types->addType(UnionTypeVar{{singletonTypes.nilType, result}}); + return types->addType(UnionTypeVar{{getSingletonTypes().nilType, result}}); } else - return types->addType(UnionTypeVar{{singletonTypes.nilType, ty}}); + return types->addType(UnionTypeVar{{getSingletonTypes().nilType, ty}}); } void Unifier::DEPRECATED_tryUnifyTables(TypeId left, TypeId right, bool isIntersection) { - LUAU_ASSERT(!FFlag::LuauTableSubtypingVariance); + LUAU_ASSERT(!FFlag::LuauTableSubtypingVariance2); Resetter resetter{&variance}; variance = Invariant; @@ -1467,7 +1653,7 @@ void Unifier::tryUnifySealedTables(TypeId left, TypeId right, bool isIntersectio } else if (lt->indexer) { - innerState.tryUnify_(lt->indexer->indexType, singletonTypes.stringType); + innerState.tryUnify_(lt->indexer->indexType, getSingletonTypes().stringType); // We already try to unify properties in both tables. // Skip those and just look for the ones remaining and see if they fit into the indexer. for (const auto& [name, type] : rt->props) @@ -1636,7 +1822,7 @@ void Unifier::tryUnifyWithClass(TypeId superTy, TypeId subTy, bool reversed) ok = false; errors.push_back(TypeError{location, UnknownProperty{superTy, propName}}); if (!FFlag::LuauExtendedClassMismatchError) - tryUnify_(prop.type, singletonTypes.errorRecoveryType()); + tryUnify_(prop.type, getSingletonTypes().errorRecoveryType()); } else { @@ -1825,7 +2011,7 @@ void Unifier::tryUnifyWithAny(TypeId any, TypeId ty) if (get(ty) || get(ty) || get(ty)) return; - const TypePackId anyTypePack = types->addTypePack(TypePackVar{VariadicTypePack{singletonTypes.anyType}}); + const TypePackId anyTypePack = types->addTypePack(TypePackVar{VariadicTypePack{getSingletonTypes().anyType}}); const TypePackId anyTP = get(any) ? anyTypePack : types->addTypePack(TypePackVar{Unifiable::Error{}}); @@ -1834,14 +2020,14 @@ void Unifier::tryUnifyWithAny(TypeId any, TypeId ty) sharedState.tempSeenTy.clear(); sharedState.tempSeenTp.clear(); - Luau::tryUnifyWithAny(queue, *this, sharedState.tempSeenTy, sharedState.tempSeenTp, singletonTypes.anyType, anyTP); + Luau::tryUnifyWithAny(queue, *this, sharedState.tempSeenTy, sharedState.tempSeenTp, getSingletonTypes().anyType, anyTP); } void Unifier::tryUnifyWithAny(TypePackId any, TypePackId ty) { LUAU_ASSERT(get(any)); - const TypeId anyTy = singletonTypes.errorRecoveryType(); + const TypeId anyTy = getSingletonTypes().errorRecoveryType(); std::vector queue; @@ -1887,7 +2073,7 @@ void Unifier::occursCheck(DenseHashSet& seen, TypeId needle, TypeId hays { errors.push_back(TypeError{location, OccursCheckFailed{}}); log(needle); - *asMutable(needle) = *singletonTypes.errorRecoveryType(); + *asMutable(needle) = *getSingletonTypes().errorRecoveryType(); return; } @@ -1951,7 +2137,7 @@ void Unifier::occursCheck(DenseHashSet& seen, TypePackId needle, Typ { errors.push_back(TypeError{location, OccursCheckFailed{}}); log(needle); - *asMutable(needle) = *singletonTypes.errorRecoveryTypePack(); + *asMutable(needle) = *getSingletonTypes().errorRecoveryTypePack(); return; } @@ -2005,7 +2191,7 @@ void Unifier::checkChildUnifierTypeMismatch(const ErrorVec& innerErrors, const s errors.push_back(*e); else if (!innerErrors.empty()) errors.push_back( - TypeError{location, TypeMismatch{wantedType, givenType, format("Property '%s' is not compatible", prop.c_str()), innerErrors.front()}}); + TypeError{location, TypeMismatch{wantedType, givenType, format("Property '%s' is not compatible.", prop.c_str()), innerErrors.front()}}); } void Unifier::ice(const std::string& message, const Location& location) diff --git a/Ast/src/Parser.cpp b/Ast/src/Parser.cpp index 3d0d5b7e..dd24f27c 100644 --- a/Ast/src/Parser.cpp +++ b/Ast/src/Parser.cpp @@ -10,7 +10,6 @@ // See docs/SyntaxChanges.md for an explanation. LUAU_FASTINTVARIABLE(LuauRecursionLimit, 1000) LUAU_FASTINTVARIABLE(LuauParseErrorLimit, 100) -LUAU_FASTFLAGVARIABLE(LuauCaptureBrokenCommentSpans, false) LUAU_FASTFLAGVARIABLE(LuauIfElseExpressionBaseSupport, false) LUAU_FASTFLAGVARIABLE(LuauIfStatementRecursionGuard, false) LUAU_FASTFLAGVARIABLE(LuauFixAmbiguousErrorRecoveryInAssign, false) @@ -159,7 +158,7 @@ ParseResult Parser::parse(const char* buffer, size_t bufferSize, AstNameTable& n { std::vector hotcomments; - while (isComment(p.lexer.current()) || (FFlag::LuauCaptureBrokenCommentSpans && p.lexer.current().type == Lexeme::BrokenComment)) + while (isComment(p.lexer.current()) || p.lexer.current().type == Lexeme::BrokenComment) { const char* text = p.lexer.current().data; unsigned int length = p.lexer.current().length; @@ -2780,7 +2779,7 @@ const Lexeme& Parser::nextLexeme() const Lexeme& lexeme = lexer.next(/*skipComments*/ false); // Subtlety: Broken comments are weird because we record them as comments AND pass them to the parser as a lexeme. // The parser will turn this into a proper syntax error. - if (FFlag::LuauCaptureBrokenCommentSpans && lexeme.type == Lexeme::BrokenComment) + if (lexeme.type == Lexeme::BrokenComment) commentLocations.push_back(Comment{lexeme.type, lexeme.location}); if (isComment(lexeme)) commentLocations.push_back(Comment{lexeme.type, lexeme.location}); diff --git a/CLI/Analyze.cpp b/CLI/Analyze.cpp index cec51e61..5318044b 100644 --- a/CLI/Analyze.cpp +++ b/CLI/Analyze.cpp @@ -11,26 +11,33 @@ enum class ReportFormat { Default, - Luacheck + Luacheck, + Gnu, }; -static void report(ReportFormat format, const char* name, const Luau::Location& location, const char* type, const char* message) +static void report(ReportFormat format, const char* name, const Luau::Location& loc, const char* type, const char* message) { switch (format) { case ReportFormat::Default: - fprintf(stderr, "%s(%d,%d): %s: %s\n", name, location.begin.line + 1, location.begin.column + 1, type, message); + fprintf(stderr, "%s(%d,%d): %s: %s\n", name, loc.begin.line + 1, loc.begin.column + 1, type, message); break; case ReportFormat::Luacheck: { // Note: luacheck's end column is inclusive but our end column is exclusive // In addition, luacheck doesn't support multi-line messages, so if the error is multiline we'll fake end column as 100 and hope for the best - int columnEnd = (location.begin.line == location.end.line) ? location.end.column : 100; + int columnEnd = (loc.begin.line == loc.end.line) ? loc.end.column : 100; - fprintf(stdout, "%s:%d:%d-%d: (W0) %s: %s\n", name, location.begin.line + 1, location.begin.column + 1, columnEnd, type, message); + // Use stdout to match luacheck behavior + fprintf(stdout, "%s:%d:%d-%d: (W0) %s: %s\n", name, loc.begin.line + 1, loc.begin.column + 1, columnEnd, type, message); break; } + + case ReportFormat::Gnu: + // Note: GNU end column is inclusive but our end column is exclusive + fprintf(stderr, "%s:%d.%d-%d.%d: %s: %s\n", name, loc.begin.line + 1, loc.begin.column + 1, loc.end.line + 1, loc.end.column, type, message); + break; } } @@ -97,6 +104,7 @@ static void displayHelp(const char* argv0) printf("\n"); printf("Available options:\n"); printf(" --formatter=plain: report analysis errors in Luacheck-compatible format\n"); + printf(" --formatter=gnu: report analysis errors in GNU-compatible format\n"); } static int assertionHandler(const char* expr, const char* file, int line) @@ -201,6 +209,8 @@ int main(int argc, char** argv) if (strcmp(argv[i], "--formatter=plain") == 0) format = ReportFormat::Luacheck; + else if (strcmp(argv[i], "--formatter=gnu") == 0) + format = ReportFormat::Gnu; else if (strcmp(argv[i], "--annotate") == 0) annotate = true; } diff --git a/CLI/Coverage.cpp b/CLI/Coverage.cpp new file mode 100644 index 00000000..254df3f0 --- /dev/null +++ b/CLI/Coverage.cpp @@ -0,0 +1,88 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +#include "Coverage.h" + +#include "lua.h" + +#include +#include + +struct Coverage +{ + lua_State* L = nullptr; + std::vector functions; +} gCoverage; + +void coverageInit(lua_State* L) +{ + gCoverage.L = lua_mainthread(L); +} + +bool coverageActive() +{ + return gCoverage.L != nullptr; +} + +void coverageTrack(lua_State* L, int funcindex) +{ + int ref = lua_ref(L, funcindex); + gCoverage.functions.push_back(ref); +} + +static void coverageCallback(void* context, const char* function, int linedefined, int depth, const int* hits, size_t size) +{ + FILE* f = static_cast(context); + + std::string name; + + if (depth == 0) + name = "
"; + else if (function) + name = std::string(function) + ":" + std::to_string(linedefined); + else + name = ":" + std::to_string(linedefined); + + fprintf(f, "FN:%d,%s\n", linedefined, name.c_str()); + + for (size_t i = 0; i < size; ++i) + if (hits[i] != -1) + { + fprintf(f, "FNDA:%d,%s\n", hits[i], name.c_str()); + break; + } + + for (size_t i = 0; i < size; ++i) + if (hits[i] != -1) + fprintf(f, "DA:%d,%d\n", int(i), hits[i]); +} + +void coverageDump(const char* path) +{ + lua_State* L = gCoverage.L; + + FILE* f = fopen(path, "w"); + if (!f) + { + fprintf(stderr, "Error opening coverage %s\n", path); + return; + } + + fprintf(f, "TN:\n"); + + for (int fref: gCoverage.functions) + { + lua_getref(L, fref); + + lua_Debug ar = {}; + lua_getinfo(L, -1, "s", &ar); + + fprintf(f, "SF:%s\n", ar.short_src); + lua_getcoverage(L, -1, f, coverageCallback); + fprintf(f, "end_of_record\n"); + + lua_pop(L, 1); + } + + fclose(f); + + printf("Coverage dump written to %s (%d functions)\n", path, int(gCoverage.functions.size())); +} diff --git a/CLI/Coverage.h b/CLI/Coverage.h new file mode 100644 index 00000000..74be4e5c --- /dev/null +++ b/CLI/Coverage.h @@ -0,0 +1,10 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +#pragma once + +struct lua_State; + +void coverageInit(lua_State* L); +bool coverageActive(); + +void coverageTrack(lua_State* L, int funcindex); +void coverageDump(const char* path); diff --git a/CLI/Profiler.cpp b/CLI/Profiler.cpp index c6d15a7f..30a171f0 100644 --- a/CLI/Profiler.cpp +++ b/CLI/Profiler.cpp @@ -110,12 +110,12 @@ void profilerStop() gProfiler.thread.join(); } -void profilerDump(const char* name) +void profilerDump(const char* path) { - FILE* f = fopen(name, "wb"); + FILE* f = fopen(path, "wb"); if (!f) { - fprintf(stderr, "Error opening profile %s\n", name); + fprintf(stderr, "Error opening profile %s\n", path); return; } @@ -129,7 +129,7 @@ void profilerDump(const char* name) fclose(f); - printf("Profiler dump written to %s (total runtime %.3f seconds, %lld samples, %lld stacks)\n", name, double(total) / 1e6, + printf("Profiler dump written to %s (total runtime %.3f seconds, %lld samples, %lld stacks)\n", path, double(total) / 1e6, static_cast(gProfiler.samples.load()), static_cast(gProfiler.data.size())); uint64_t totalgc = 0; diff --git a/CLI/Profiler.h b/CLI/Profiler.h index 0a407e47..67b1acfd 100644 --- a/CLI/Profiler.h +++ b/CLI/Profiler.h @@ -5,4 +5,4 @@ struct lua_State; void profilerStart(lua_State* L, int frequency); void profilerStop(); -void profilerDump(const char* name); \ No newline at end of file +void profilerDump(const char* path); diff --git a/CLI/Repl.cpp b/CLI/Repl.cpp index 76493a5e..5d3324ab 100644 --- a/CLI/Repl.cpp +++ b/CLI/Repl.cpp @@ -8,6 +8,7 @@ #include "FileUtils.h" #include "Profiler.h" +#include "Coverage.h" #include "linenoise.hpp" @@ -24,6 +25,16 @@ enum class CompileFormat Binary }; +static Luau::CompileOptions copts() +{ + Luau::CompileOptions result = {}; + result.optimizationLevel = 1; + result.debugLevel = 1; + result.coverageLevel = coverageActive() ? 2 : 0; + + return result; +} + static int lua_loadstring(lua_State* L) { size_t l = 0; @@ -32,7 +43,7 @@ static int lua_loadstring(lua_State* L) lua_setsafeenv(L, LUA_ENVIRONINDEX, false); - std::string bytecode = Luau::compile(std::string(s, l)); + std::string bytecode = Luau::compile(std::string(s, l), copts()); if (luau_load(L, chunkname, bytecode.data(), bytecode.size(), 0) == 0) return 1; @@ -79,9 +90,12 @@ static int lua_require(lua_State* L) luaL_sandboxthread(ML); // now we can compile & run module on the new thread - std::string bytecode = Luau::compile(*source); + std::string bytecode = Luau::compile(*source, copts()); if (luau_load(ML, chunkname.c_str(), bytecode.data(), bytecode.size(), 0) == 0) { + if (coverageActive()) + coverageTrack(ML, -1); + int status = lua_resume(ML, L, 0); if (status == 0) @@ -149,7 +163,7 @@ static void setupState(lua_State* L) static std::string runCode(lua_State* L, const std::string& source) { - std::string bytecode = Luau::compile(source); + std::string bytecode = Luau::compile(source, copts()); if (luau_load(L, "=stdin", bytecode.data(), bytecode.size(), 0) != 0) { @@ -329,11 +343,14 @@ static bool runFile(const char* name, lua_State* GL) std::string chunkname = "=" + std::string(name); - std::string bytecode = Luau::compile(*source); + std::string bytecode = Luau::compile(*source, copts()); int status = 0; if (luau_load(L, chunkname.c_str(), bytecode.data(), bytecode.size(), 0) == 0) { + if (coverageActive()) + coverageTrack(L, -1); + status = lua_resume(L, NULL, 0); } else @@ -437,6 +454,7 @@ static void displayHelp(const char* argv0) printf("\n"); printf("Available options:\n"); printf(" --profile[=N]: profile the code using N Hz sampling (default 10000) and output results to profile.out\n"); + printf(" --coverage: collect code coverage while running the code and output results to coverage.out\n"); } static int assertionHandler(const char* expr, const char* file, int line) @@ -495,6 +513,7 @@ int main(int argc, char** argv) setupState(L); int profile = 0; + bool coverage = false; for (int i = 1; i < argc; ++i) { @@ -505,11 +524,16 @@ int main(int argc, char** argv) profile = 10000; // default to 10 KHz else if (strncmp(argv[i], "--profile=", 10) == 0) profile = atoi(argv[i] + 10); + else if (strcmp(argv[i], "--coverage") == 0) + coverage = true; } if (profile) profilerStart(L, profile); + if (coverage) + coverageInit(L); + std::vector files = getSourceFiles(argc, argv); int failed = 0; @@ -523,6 +547,9 @@ int main(int argc, char** argv) profilerDump("profile.out"); } + if (coverage) + coverageDump("coverage.out"); + return failed; } } diff --git a/Compiler/src/Compiler.cpp b/Compiler/src/Compiler.cpp index 2c1e85ff..8f74ffed 100644 --- a/Compiler/src/Compiler.cpp +++ b/Compiler/src/Compiler.cpp @@ -10,7 +10,6 @@ #include #include -LUAU_FASTFLAGVARIABLE(LuauPreloadClosures, false) LUAU_FASTFLAG(LuauIfElseExpressionBaseSupport) LUAU_FASTFLAGVARIABLE(LuauBit32CountBuiltin, false) @@ -462,20 +461,17 @@ struct Compiler bool shared = false; - if (FFlag::LuauPreloadClosures) + // Optimization: when closure has no upvalues, or upvalues are safe to share, instead of allocating it every time we can share closure + // objects (this breaks assumptions about function identity which can lead to setfenv not working as expected, so we disable this when it + // is used) + if (options.optimizationLevel >= 1 && shouldShareClosure(expr) && !setfenvUsed) { - // Optimization: when closure has no upvalues, or upvalues are safe to share, instead of allocating it every time we can share closure - // objects (this breaks assumptions about function identity which can lead to setfenv not working as expected, so we disable this when it - // is used) - if (options.optimizationLevel >= 1 && shouldShareClosure(expr) && !setfenvUsed) - { - int32_t cid = bytecode.addConstantClosure(f->id); + int32_t cid = bytecode.addConstantClosure(f->id); - if (cid >= 0 && cid < 32768) - { - bytecode.emitAD(LOP_DUPCLOSURE, target, cid); - shared = true; - } + if (cid >= 0 && cid < 32768) + { + bytecode.emitAD(LOP_DUPCLOSURE, target, cid); + shared = true; } } diff --git a/Makefile b/Makefile index 7355040f..fe5a2d9c 100644 --- a/Makefile +++ b/Makefile @@ -27,7 +27,7 @@ TESTS_SOURCES=$(wildcard tests/*.cpp) TESTS_OBJECTS=$(TESTS_SOURCES:%=$(BUILD)/%.o) TESTS_TARGET=$(BUILD)/luau-tests -REPL_CLI_SOURCES=CLI/FileUtils.cpp CLI/Profiler.cpp CLI/Repl.cpp +REPL_CLI_SOURCES=CLI/FileUtils.cpp CLI/Profiler.cpp CLI/Coverage.cpp CLI/Repl.cpp REPL_CLI_OBJECTS=$(REPL_CLI_SOURCES:%=$(BUILD)/%.o) REPL_CLI_TARGET=$(BUILD)/luau @@ -128,10 +128,10 @@ luau-size: luau # executable target aliases luau: $(REPL_CLI_TARGET) - cp $^ $@ + ln -fs $^ $@ luau-analyze: $(ANALYZE_CLI_TARGET) - cp $^ $@ + ln -fs $^ $@ # executable targets $(TESTS_TARGET): $(TESTS_OBJECTS) $(ANALYSIS_TARGET) $(COMPILER_TARGET) $(AST_TARGET) $(VM_TARGET) diff --git a/Sources.cmake b/Sources.cmake index 57df9b91..14834b3a 100644 --- a/Sources.cmake +++ b/Sources.cmake @@ -133,6 +133,7 @@ target_sources(Luau.VM PRIVATE VM/src/ltable.cpp VM/src/ltablib.cpp VM/src/ltm.cpp + VM/src/ludata.cpp VM/src/lutf8lib.cpp VM/src/lvmexecute.cpp VM/src/lvmload.cpp @@ -152,12 +153,15 @@ target_sources(Luau.VM PRIVATE VM/src/lstring.h VM/src/ltable.h VM/src/ltm.h + VM/src/ludata.h VM/src/lvm.h ) if(TARGET Luau.Repl.CLI) # Luau.Repl.CLI Sources target_sources(Luau.Repl.CLI PRIVATE + CLI/Coverage.h + CLI/Coverage.cpp CLI/FileUtils.h CLI/FileUtils.cpp CLI/Profiler.h diff --git a/VM/include/lua.h b/VM/include/lua.h index 7078acd0..55902160 100644 --- a/VM/include/lua.h +++ b/VM/include/lua.h @@ -334,6 +334,10 @@ LUA_API const char* lua_setupvalue(lua_State* L, int funcindex, int n); LUA_API void lua_singlestep(lua_State* L, int enabled); LUA_API void lua_breakpoint(lua_State* L, int funcindex, int line, int enabled); +typedef void (*lua_Coverage)(void* context, const char* function, int linedefined, int depth, const int* hits, size_t size); + +LUA_API void lua_getcoverage(lua_State* L, int funcindex, void* context, lua_Coverage callback); + /* Warning: this function is not thread-safe since it stores the result in a shared global array! Only use for debugging. */ LUA_API const char* lua_debugtrace(lua_State* L); diff --git a/VM/src/lapi.cpp b/VM/src/lapi.cpp index 76043b9c..a65b0325 100644 --- a/VM/src/lapi.cpp +++ b/VM/src/lapi.cpp @@ -8,6 +8,7 @@ #include "lfunc.h" #include "lgc.h" #include "ldo.h" +#include "ludata.h" #include "lvm.h" #include "lnumutils.h" @@ -43,36 +44,30 @@ static Table* getcurrenv(lua_State* L) } } -static LUAU_NOINLINE TValue* index2adrslow(lua_State* L, int idx) +static LUAU_NOINLINE TValue* pseudo2addr(lua_State* L, int idx) { - api_check(L, idx <= 0); - if (idx > LUA_REGISTRYINDEX) + api_check(L, lua_ispseudo(idx)); + switch (idx) + { /* pseudo-indices */ + case LUA_REGISTRYINDEX: + return registry(L); + case LUA_ENVIRONINDEX: { - api_check(L, idx != 0 && -idx <= L->top - L->base); - return L->top + idx; + sethvalue(L, &L->env, getcurrenv(L)); + return &L->env; + } + case LUA_GLOBALSINDEX: + return gt(L); + default: + { + Closure* func = curr_func(L); + idx = LUA_GLOBALSINDEX - idx; + return (idx <= func->nupvalues) ? &func->c.upvals[idx - 1] : cast_to(TValue*, luaO_nilobject); + } } - else - switch (idx) - { /* pseudo-indices */ - case LUA_REGISTRYINDEX: - return registry(L); - case LUA_ENVIRONINDEX: - { - sethvalue(L, &L->env, getcurrenv(L)); - return &L->env; - } - case LUA_GLOBALSINDEX: - return gt(L); - default: - { - Closure* func = curr_func(L); - idx = LUA_GLOBALSINDEX - idx; - return (idx <= func->nupvalues) ? &func->c.upvals[idx - 1] : cast_to(TValue*, luaO_nilobject); - } - } } -static LUAU_FORCEINLINE TValue* index2adr(lua_State* L, int idx) +static LUAU_FORCEINLINE TValue* index2addr(lua_State* L, int idx) { if (idx > 0) { @@ -83,15 +78,20 @@ static LUAU_FORCEINLINE TValue* index2adr(lua_State* L, int idx) else return o; } + else if (idx > LUA_REGISTRYINDEX) + { + api_check(L, idx != 0 && -idx <= L->top - L->base); + return L->top + idx; + } else { - return index2adrslow(L, idx); + return pseudo2addr(L, idx); } } const TValue* luaA_toobject(lua_State* L, int idx) { - StkId p = index2adr(L, idx); + StkId p = index2addr(L, idx); return (p == luaO_nilobject) ? NULL : p; } @@ -145,7 +145,7 @@ void lua_xpush(lua_State* from, lua_State* to, int idx) { api_check(from, from->global == to->global); luaC_checkthreadsleep(to); - setobj2s(to, to->top, index2adr(from, idx)); + setobj2s(to, to->top, index2addr(from, idx)); api_incr_top(to); return; } @@ -202,7 +202,7 @@ void lua_settop(lua_State* L, int idx) void lua_remove(lua_State* L, int idx) { - StkId p = index2adr(L, idx); + StkId p = index2addr(L, idx); api_checkvalidindex(L, p); while (++p < L->top) setobjs2s(L, p - 1, p); @@ -213,7 +213,7 @@ void lua_remove(lua_State* L, int idx) void lua_insert(lua_State* L, int idx) { luaC_checkthreadsleep(L); - StkId p = index2adr(L, idx); + StkId p = index2addr(L, idx); api_checkvalidindex(L, p); for (StkId q = L->top; q > p; q--) setobjs2s(L, q, q - 1); @@ -228,7 +228,7 @@ void lua_replace(lua_State* L, int idx) luaG_runerror(L, "no calling environment"); api_checknelems(L, 1); luaC_checkthreadsleep(L); - StkId o = index2adr(L, idx); + StkId o = index2addr(L, idx); api_checkvalidindex(L, o); if (idx == LUA_ENVIRONINDEX) { @@ -250,7 +250,7 @@ void lua_replace(lua_State* L, int idx) void lua_pushvalue(lua_State* L, int idx) { luaC_checkthreadsleep(L); - StkId o = index2adr(L, idx); + StkId o = index2addr(L, idx); setobj2s(L, L->top, o); api_incr_top(L); return; @@ -262,7 +262,7 @@ void lua_pushvalue(lua_State* L, int idx) int lua_type(lua_State* L, int idx) { - StkId o = index2adr(L, idx); + StkId o = index2addr(L, idx); return (o == luaO_nilobject) ? LUA_TNONE : ttype(o); } @@ -273,20 +273,20 @@ const char* lua_typename(lua_State* L, int t) int lua_iscfunction(lua_State* L, int idx) { - StkId o = index2adr(L, idx); + StkId o = index2addr(L, idx); return iscfunction(o); } int lua_isLfunction(lua_State* L, int idx) { - StkId o = index2adr(L, idx); + StkId o = index2addr(L, idx); return isLfunction(o); } int lua_isnumber(lua_State* L, int idx) { TValue n; - const TValue* o = index2adr(L, idx); + const TValue* o = index2addr(L, idx); return tonumber(o, &n); } @@ -298,14 +298,14 @@ int lua_isstring(lua_State* L, int idx) int lua_isuserdata(lua_State* L, int idx) { - const TValue* o = index2adr(L, idx); + const TValue* o = index2addr(L, idx); return (ttisuserdata(o) || ttislightuserdata(o)); } int lua_rawequal(lua_State* L, int index1, int index2) { - StkId o1 = index2adr(L, index1); - StkId o2 = index2adr(L, index2); + StkId o1 = index2addr(L, index1); + StkId o2 = index2addr(L, index2); return (o1 == luaO_nilobject || o2 == luaO_nilobject) ? 0 : luaO_rawequalObj(o1, o2); } @@ -313,8 +313,8 @@ int lua_equal(lua_State* L, int index1, int index2) { StkId o1, o2; int i; - o1 = index2adr(L, index1); - o2 = index2adr(L, index2); + o1 = index2addr(L, index1); + o2 = index2addr(L, index2); i = (o1 == luaO_nilobject || o2 == luaO_nilobject) ? 0 : equalobj(L, o1, o2); return i; } @@ -323,8 +323,8 @@ int lua_lessthan(lua_State* L, int index1, int index2) { StkId o1, o2; int i; - o1 = index2adr(L, index1); - o2 = index2adr(L, index2); + o1 = index2addr(L, index1); + o2 = index2addr(L, index2); i = (o1 == luaO_nilobject || o2 == luaO_nilobject) ? 0 : luaV_lessthan(L, o1, o2); return i; } @@ -332,7 +332,7 @@ int lua_lessthan(lua_State* L, int index1, int index2) double lua_tonumberx(lua_State* L, int idx, int* isnum) { TValue n; - const TValue* o = index2adr(L, idx); + const TValue* o = index2addr(L, idx); if (tonumber(o, &n)) { if (isnum) @@ -350,7 +350,7 @@ double lua_tonumberx(lua_State* L, int idx, int* isnum) int lua_tointegerx(lua_State* L, int idx, int* isnum) { TValue n; - const TValue* o = index2adr(L, idx); + const TValue* o = index2addr(L, idx); if (tonumber(o, &n)) { int res; @@ -371,7 +371,7 @@ int lua_tointegerx(lua_State* L, int idx, int* isnum) unsigned lua_tounsignedx(lua_State* L, int idx, int* isnum) { TValue n; - const TValue* o = index2adr(L, idx); + const TValue* o = index2addr(L, idx); if (tonumber(o, &n)) { unsigned res; @@ -391,13 +391,13 @@ unsigned lua_tounsignedx(lua_State* L, int idx, int* isnum) int lua_toboolean(lua_State* L, int idx) { - const TValue* o = index2adr(L, idx); + const TValue* o = index2addr(L, idx); return !l_isfalse(o); } const char* lua_tolstring(lua_State* L, int idx, size_t* len) { - StkId o = index2adr(L, idx); + StkId o = index2addr(L, idx); if (!ttisstring(o)) { luaC_checkthreadsleep(L); @@ -408,7 +408,7 @@ const char* lua_tolstring(lua_State* L, int idx, size_t* len) return NULL; } luaC_checkGC(L); - o = index2adr(L, idx); /* previous call may reallocate the stack */ + o = index2addr(L, idx); /* previous call may reallocate the stack */ } if (len != NULL) *len = tsvalue(o)->len; @@ -417,7 +417,7 @@ const char* lua_tolstring(lua_State* L, int idx, size_t* len) const char* lua_tostringatom(lua_State* L, int idx, int* atom) { - StkId o = index2adr(L, idx); + StkId o = index2addr(L, idx); if (!ttisstring(o)) return NULL; const TString* s = tsvalue(o); @@ -438,7 +438,7 @@ const char* lua_namecallatom(lua_State* L, int* atom) const float* lua_tovector(lua_State* L, int idx) { - StkId o = index2adr(L, idx); + StkId o = index2addr(L, idx); if (!ttisvector(o)) { return NULL; @@ -448,7 +448,7 @@ const float* lua_tovector(lua_State* L, int idx) int lua_objlen(lua_State* L, int idx) { - StkId o = index2adr(L, idx); + StkId o = index2addr(L, idx); switch (ttype(o)) { case LUA_TSTRING: @@ -469,13 +469,13 @@ int lua_objlen(lua_State* L, int idx) lua_CFunction lua_tocfunction(lua_State* L, int idx) { - StkId o = index2adr(L, idx); + StkId o = index2addr(L, idx); return (!iscfunction(o)) ? NULL : cast_to(lua_CFunction, clvalue(o)->c.f); } void* lua_touserdata(lua_State* L, int idx) { - StkId o = index2adr(L, idx); + StkId o = index2addr(L, idx); switch (ttype(o)) { case LUA_TUSERDATA: @@ -489,13 +489,13 @@ void* lua_touserdata(lua_State* L, int idx) void* lua_touserdatatagged(lua_State* L, int idx, int tag) { - StkId o = index2adr(L, idx); + StkId o = index2addr(L, idx); return (ttisuserdata(o) && uvalue(o)->tag == tag) ? uvalue(o)->data : NULL; } int lua_userdatatag(lua_State* L, int idx) { - StkId o = index2adr(L, idx); + StkId o = index2addr(L, idx); if (ttisuserdata(o)) return uvalue(o)->tag; return -1; @@ -503,13 +503,13 @@ int lua_userdatatag(lua_State* L, int idx) lua_State* lua_tothread(lua_State* L, int idx) { - StkId o = index2adr(L, idx); + StkId o = index2addr(L, idx); return (!ttisthread(o)) ? NULL : thvalue(o); } const void* lua_topointer(lua_State* L, int idx) { - StkId o = index2adr(L, idx); + StkId o = index2addr(L, idx); switch (ttype(o)) { case LUA_TTABLE: @@ -657,7 +657,7 @@ int lua_pushthread(lua_State* L) void lua_gettable(lua_State* L, int idx) { luaC_checkthreadsleep(L); - StkId t = index2adr(L, idx); + StkId t = index2addr(L, idx); api_checkvalidindex(L, t); luaV_gettable(L, t, L->top - 1, L->top - 1); return; @@ -666,7 +666,7 @@ void lua_gettable(lua_State* L, int idx) void lua_getfield(lua_State* L, int idx, const char* k) { luaC_checkthreadsleep(L); - StkId t = index2adr(L, idx); + StkId t = index2addr(L, idx); api_checkvalidindex(L, t); TValue key; setsvalue(L, &key, luaS_new(L, k)); @@ -678,7 +678,7 @@ void lua_getfield(lua_State* L, int idx, const char* k) void lua_rawgetfield(lua_State* L, int idx, const char* k) { luaC_checkthreadsleep(L); - StkId t = index2adr(L, idx); + StkId t = index2addr(L, idx); api_check(L, ttistable(t)); TValue key; setsvalue(L, &key, luaS_new(L, k)); @@ -690,7 +690,7 @@ void lua_rawgetfield(lua_State* L, int idx, const char* k) void lua_rawget(lua_State* L, int idx) { luaC_checkthreadsleep(L); - StkId t = index2adr(L, idx); + StkId t = index2addr(L, idx); api_check(L, ttistable(t)); setobj2s(L, L->top - 1, luaH_get(hvalue(t), L->top - 1)); return; @@ -699,7 +699,7 @@ void lua_rawget(lua_State* L, int idx) void lua_rawgeti(lua_State* L, int idx, int n) { luaC_checkthreadsleep(L); - StkId t = index2adr(L, idx); + StkId t = index2addr(L, idx); api_check(L, ttistable(t)); setobj2s(L, L->top, luaH_getnum(hvalue(t), n)); api_incr_top(L); @@ -717,7 +717,7 @@ void lua_createtable(lua_State* L, int narray, int nrec) void lua_setreadonly(lua_State* L, int objindex, int enabled) { - const TValue* o = index2adr(L, objindex); + const TValue* o = index2addr(L, objindex); api_check(L, ttistable(o)); Table* t = hvalue(o); api_check(L, t != hvalue(registry(L))); @@ -727,7 +727,7 @@ void lua_setreadonly(lua_State* L, int objindex, int enabled) int lua_getreadonly(lua_State* L, int objindex) { - const TValue* o = index2adr(L, objindex); + const TValue* o = index2addr(L, objindex); api_check(L, ttistable(o)); Table* t = hvalue(o); int res = t->readonly; @@ -736,7 +736,7 @@ int lua_getreadonly(lua_State* L, int objindex) void lua_setsafeenv(lua_State* L, int objindex, int enabled) { - const TValue* o = index2adr(L, objindex); + const TValue* o = index2addr(L, objindex); api_check(L, ttistable(o)); Table* t = hvalue(o); t->safeenv = bool(enabled); @@ -748,7 +748,7 @@ int lua_getmetatable(lua_State* L, int objindex) const TValue* obj; Table* mt = NULL; int res; - obj = index2adr(L, objindex); + obj = index2addr(L, objindex); switch (ttype(obj)) { case LUA_TTABLE: @@ -775,7 +775,7 @@ int lua_getmetatable(lua_State* L, int objindex) void lua_getfenv(lua_State* L, int idx) { StkId o; - o = index2adr(L, idx); + o = index2addr(L, idx); api_checkvalidindex(L, o); switch (ttype(o)) { @@ -801,7 +801,7 @@ void lua_settable(lua_State* L, int idx) { StkId t; api_checknelems(L, 2); - t = index2adr(L, idx); + t = index2addr(L, idx); api_checkvalidindex(L, t); luaV_settable(L, t, L->top - 2, L->top - 1); L->top -= 2; /* pop index and value */ @@ -813,7 +813,7 @@ void lua_setfield(lua_State* L, int idx, const char* k) StkId t; TValue key; api_checknelems(L, 1); - t = index2adr(L, idx); + t = index2addr(L, idx); api_checkvalidindex(L, t); setsvalue(L, &key, luaS_new(L, k)); luaV_settable(L, t, &key, L->top - 1); @@ -825,7 +825,7 @@ void lua_rawset(lua_State* L, int idx) { StkId t; api_checknelems(L, 2); - t = index2adr(L, idx); + t = index2addr(L, idx); api_check(L, ttistable(t)); if (hvalue(t)->readonly) luaG_runerror(L, "Attempt to modify a readonly table"); @@ -839,7 +839,7 @@ void lua_rawseti(lua_State* L, int idx, int n) { StkId o; api_checknelems(L, 1); - o = index2adr(L, idx); + o = index2addr(L, idx); api_check(L, ttistable(o)); if (hvalue(o)->readonly) luaG_runerror(L, "Attempt to modify a readonly table"); @@ -854,7 +854,7 @@ int lua_setmetatable(lua_State* L, int objindex) TValue* obj; Table* mt; api_checknelems(L, 1); - obj = index2adr(L, objindex); + obj = index2addr(L, objindex); api_checkvalidindex(L, obj); if (ttisnil(L->top - 1)) mt = NULL; @@ -896,7 +896,7 @@ int lua_setfenv(lua_State* L, int idx) StkId o; int res = 1; api_checknelems(L, 1); - o = index2adr(L, idx); + o = index2addr(L, idx); api_checkvalidindex(L, o); api_check(L, ttistable(L->top - 1)); switch (ttype(o)) @@ -987,7 +987,7 @@ int lua_pcall(lua_State* L, int nargs, int nresults, int errfunc) func = 0; else { - StkId o = index2adr(L, errfunc); + StkId o = index2addr(L, errfunc); api_checkvalidindex(L, o); func = savestack(L, o); } @@ -1150,7 +1150,7 @@ l_noret lua_error(lua_State* L) int lua_next(lua_State* L, int idx) { luaC_checkthreadsleep(L); - StkId t = index2adr(L, idx); + StkId t = index2addr(L, idx); api_check(L, ttistable(t)); int more = luaH_next(L, hvalue(t), L->top - 1); if (more) @@ -1187,7 +1187,7 @@ void* lua_newuserdatatagged(lua_State* L, size_t sz, int tag) api_check(L, unsigned(tag) < LUA_UTAG_LIMIT); luaC_checkGC(L); luaC_checkthreadsleep(L); - Udata* u = luaS_newudata(L, sz, tag); + Udata* u = luaU_newudata(L, sz, tag); setuvalue(L, L->top, u); api_incr_top(L); return u->data; @@ -1197,7 +1197,7 @@ void* lua_newuserdatadtor(lua_State* L, size_t sz, void (*dtor)(void*)) { luaC_checkGC(L); luaC_checkthreadsleep(L); - Udata* u = luaS_newudata(L, sz + sizeof(dtor), UTAG_IDTOR); + Udata* u = luaU_newudata(L, sz + sizeof(dtor), UTAG_IDTOR); memcpy(&u->data + sz, &dtor, sizeof(dtor)); setuvalue(L, L->top, u); api_incr_top(L); @@ -1232,7 +1232,7 @@ const char* lua_getupvalue(lua_State* L, int funcindex, int n) { luaC_checkthreadsleep(L); TValue* val; - const char* name = aux_upvalue(index2adr(L, funcindex), n, &val); + const char* name = aux_upvalue(index2addr(L, funcindex), n, &val); if (name) { setobj2s(L, L->top, val); @@ -1246,7 +1246,7 @@ const char* lua_setupvalue(lua_State* L, int funcindex, int n) const char* name; TValue* val; StkId fi; - fi = index2adr(L, funcindex); + fi = index2addr(L, funcindex); api_checknelems(L, 1); name = aux_upvalue(fi, n, &val); if (name) @@ -1270,7 +1270,7 @@ int lua_ref(lua_State* L, int idx) api_check(L, idx != LUA_REGISTRYINDEX); /* idx is a stack index for value */ int ref = LUA_REFNIL; global_State* g = L->global; - StkId p = index2adr(L, idx); + StkId p = index2addr(L, idx); if (!ttisnil(p)) { Table* reg = hvalue(registry(L)); diff --git a/VM/src/ldebug.cpp b/VM/src/ldebug.cpp index d77f84ef..9fe1885f 100644 --- a/VM/src/ldebug.cpp +++ b/VM/src/ldebug.cpp @@ -370,6 +370,69 @@ void lua_breakpoint(lua_State* L, int funcindex, int line, int enabled) luaG_breakpoint(L, clvalue(func)->l.p, line, bool(enabled)); } +static int getmaxline(Proto* p) +{ + int result = -1; + + for (int i = 0; i < p->sizecode; ++i) + { + int line = luaG_getline(p, i); + result = result < line ? line : result; + } + + for (int i = 0; i < p->sizep; ++i) + { + int psize = getmaxline(p->p[i]); + result = result < psize ? psize : result; + } + + return result; +} + +static void getcoverage(Proto* p, int depth, int* buffer, size_t size, void* context, lua_Coverage callback) +{ + memset(buffer, -1, size * sizeof(int)); + + for (int i = 0; i < p->sizecode; ++i) + { + Instruction insn = p->code[i]; + if (LUAU_INSN_OP(insn) != LOP_COVERAGE) + continue; + + int line = luaG_getline(p, i); + int hits = LUAU_INSN_E(insn); + + LUAU_ASSERT(size_t(line) < size); + buffer[line] = buffer[line] < hits ? hits : buffer[line]; + } + + const char* debugname = p->debugname ? getstr(p->debugname) : NULL; + int linedefined = luaG_getline(p, 0); + + callback(context, debugname, linedefined, depth, buffer, size); + + for (int i = 0; i < p->sizep; ++i) + getcoverage(p->p[i], depth + 1, buffer, size, context, callback); +} + +void lua_getcoverage(lua_State* L, int funcindex, void* context, lua_Coverage callback) +{ + const TValue* func = luaA_toobject(L, funcindex); + api_check(L, ttisfunction(func) && !clvalue(func)->isC); + + Proto* p = clvalue(func)->l.p; + + size_t size = getmaxline(p) + 1; + if (size == 0) + return; + + int* buffer = luaM_newarray(L, size, int, 0); + + getcoverage(p, 0, buffer, size, context, callback); + + luaM_freearray(L, buffer, size, int, 0); +} + static size_t append(char* buf, size_t bufsize, size_t offset, const char* data) { size_t size = strlen(data); diff --git a/VM/src/lgc.cpp b/VM/src/lgc.cpp index ab416041..7393fc74 100644 --- a/VM/src/lgc.cpp +++ b/VM/src/lgc.cpp @@ -8,11 +8,10 @@ #include "lfunc.h" #include "lstring.h" #include "ldo.h" +#include "ludata.h" #include -LUAU_FASTFLAGVARIABLE(LuauSeparateAtomic, false) - LUAU_FASTFLAG(LuauArrayBoundary) #define GC_SWEEPMAX 40 @@ -59,10 +58,6 @@ static void recordGcStateTime(global_State* g, int startgcstate, double seconds, case GCSpropagate: case GCSpropagateagain: g->gcstats.currcycle.marktime += seconds; - - // atomic step had to be performed during the switch and it's tracked separately - if (!FFlag::LuauSeparateAtomic && g->gcstate == GCSsweepstring) - g->gcstats.currcycle.marktime -= g->gcstats.currcycle.atomictime; break; case GCSatomic: g->gcstats.currcycle.atomictime += seconds; @@ -488,7 +483,7 @@ static void freeobj(lua_State* L, GCObject* o) luaS_free(L, gco2ts(o)); break; case LUA_TUSERDATA: - luaS_freeudata(L, gco2u(o)); + luaU_freeudata(L, gco2u(o)); break; default: LUAU_ASSERT(0); @@ -632,17 +627,9 @@ static size_t remarkupvals(global_State* g) static size_t atomic(lua_State* L) { global_State* g = L->global; + LUAU_ASSERT(g->gcstate == GCSatomic); + size_t work = 0; - - if (FFlag::LuauSeparateAtomic) - { - LUAU_ASSERT(g->gcstate == GCSatomic); - } - else - { - g->gcstate = GCSatomic; - } - /* remark occasional upvalues of (maybe) dead threads */ work += remarkupvals(g); /* traverse objects caught by write barrier and by 'remarkupvals' */ @@ -666,11 +653,6 @@ static size_t atomic(lua_State* L) g->sweepgc = &g->rootgc; g->gcstate = GCSsweepstring; - if (!FFlag::LuauSeparateAtomic) - { - GC_INTERRUPT(GCSatomic); - } - return work; } @@ -716,22 +698,7 @@ static size_t gcstep(lua_State* L, size_t limit) if (!g->gray) /* no more `gray' objects */ { - if (FFlag::LuauSeparateAtomic) - { - g->gcstate = GCSatomic; - } - else - { - double starttimestamp = lua_clock(); - - g->gcstats.currcycle.atomicstarttimestamp = starttimestamp; - g->gcstats.currcycle.atomicstarttotalsizebytes = g->totalbytes; - - atomic(L); /* finish mark phase */ - LUAU_ASSERT(g->gcstate == GCSsweepstring); - - g->gcstats.currcycle.atomictime += lua_clock() - starttimestamp; - } + g->gcstate = GCSatomic; } break; } @@ -853,7 +820,7 @@ static size_t getheaptrigger(global_State* g, size_t heapgoal) void luaC_step(lua_State* L, bool assist) { global_State* g = L->global; - ptrdiff_t lim = (g->gcstepsize / 100) * g->gcstepmul; /* how much to work */ + int lim = (g->gcstepsize / 100) * g->gcstepmul; /* how much to work */ LUAU_ASSERT(g->totalbytes >= g->GCthreshold); size_t debt = g->totalbytes - g->GCthreshold; @@ -908,7 +875,7 @@ void luaC_fullgc(lua_State* L) if (g->gcstate == GCSpause) startGcCycleStats(g); - if (g->gcstate <= (FFlag::LuauSeparateAtomic ? GCSatomic : GCSpropagateagain)) + if (g->gcstate <= GCSatomic) { /* reset sweep marks to sweep all elements (returning them to white) */ g->sweepstrgc = 0; @@ -1049,7 +1016,7 @@ int64_t luaC_allocationrate(lua_State* L) global_State* g = L->global; const double durationthreshold = 1e-3; // avoid measuring intervals smaller than 1ms - if (g->gcstate <= (FFlag::LuauSeparateAtomic ? GCSatomic : GCSpropagateagain)) + if (g->gcstate <= GCSatomic) { double duration = lua_clock() - g->gcstats.lastcycle.endtimestamp; diff --git a/VM/src/lgcdebug.cpp b/VM/src/lgcdebug.cpp index a79e7b95..f6f7a878 100644 --- a/VM/src/lgcdebug.cpp +++ b/VM/src/lgcdebug.cpp @@ -7,6 +7,7 @@ #include "ltable.h" #include "lfunc.h" #include "lstring.h" +#include "ludata.h" #include #include diff --git a/VM/src/lobject.h b/VM/src/lobject.h index ba040af6..fd0a15b7 100644 --- a/VM/src/lobject.h +++ b/VM/src/lobject.h @@ -78,15 +78,7 @@ typedef struct lua_TValue #define thvalue(o) check_exp(ttisthread(o), &(o)->value.gc->th) #define upvalue(o) check_exp(ttisupval(o), &(o)->value.gc->uv) -// beware bit magic: a value is false if it's nil or boolean false -// baseline implementation: (ttisnil(o) || (ttisboolean(o) && bvalue(o) == 0)) -// we'd like a branchless version of this which helps with performance, and a very fast version -// so our strategy is to always read the boolean value (not using bvalue(o) because that asserts when type isn't boolean) -// we then combine it with type to produce 0/1 as follows: -// - when type is nil (0), & makes the result 0 -// - when type is boolean (1), we effectively only look at the bottom bit, so result is 0 iff boolean value is 0 -// - when type is different, it must have some of the top bits set - we keep all top bits of boolean value so the result is non-0 -#define l_isfalse(o) (!(((o)->value.b | ~1) & ttype(o))) +#define l_isfalse(o) (ttisnil(o) || (ttisboolean(o) && bvalue(o) == 0)) /* ** for internal debug only diff --git a/VM/src/lstring.cpp b/VM/src/lstring.cpp index 18ee1cda..a9e90d17 100644 --- a/VM/src/lstring.cpp +++ b/VM/src/lstring.cpp @@ -206,32 +206,3 @@ void luaS_free(lua_State* L, TString* ts) L->global->strt.nuse--; luaM_free(L, ts, sizestring(ts->len), ts->memcat); } - -Udata* luaS_newudata(lua_State* L, size_t s, int tag) -{ - if (s > INT_MAX - sizeof(Udata)) - luaM_toobig(L); - Udata* u = luaM_new(L, Udata, sizeudata(s), L->activememcat); - luaC_link(L, u, LUA_TUSERDATA); - u->len = int(s); - u->metatable = NULL; - LUAU_ASSERT(tag >= 0 && tag <= 255); - u->tag = uint8_t(tag); - return u; -} - -void luaS_freeudata(lua_State* L, Udata* u) -{ - LUAU_ASSERT(u->tag < LUA_UTAG_LIMIT || u->tag == UTAG_IDTOR); - - void (*dtor)(void*) = nullptr; - if (u->tag == UTAG_IDTOR) - memcpy(&dtor, &u->data + u->len - sizeof(dtor), sizeof(dtor)); - else if (u->tag) - dtor = L->global->udatagc[u->tag]; - - if (dtor) - dtor(u->data); - - luaM_free(L, u, sizeudata(u->len), u->memcat); -} diff --git a/VM/src/lstring.h b/VM/src/lstring.h index 612da28d..3fd0bd39 100644 --- a/VM/src/lstring.h +++ b/VM/src/lstring.h @@ -8,11 +8,7 @@ /* string size limit */ #define MAXSSIZE (1 << 30) -/* special tag value is used for user data with inline dtors */ -#define UTAG_IDTOR LUA_UTAG_LIMIT - #define sizestring(len) (offsetof(TString, data) + len + 1) -#define sizeudata(len) (offsetof(Udata, data) + len) #define luaS_new(L, s) (luaS_newlstr(L, s, strlen(s))) #define luaS_newliteral(L, s) (luaS_newlstr(L, "" s, (sizeof(s) / sizeof(char)) - 1)) @@ -26,8 +22,5 @@ LUAI_FUNC void luaS_resize(lua_State* L, int newsize); LUAI_FUNC TString* luaS_newlstr(lua_State* L, const char* str, size_t l); LUAI_FUNC void luaS_free(lua_State* L, TString* ts); -LUAI_FUNC Udata* luaS_newudata(lua_State* L, size_t s, int tag); -LUAI_FUNC void luaS_freeudata(lua_State* L, Udata* u); - LUAI_FUNC TString* luaS_bufstart(lua_State* L, size_t size); LUAI_FUNC TString* luaS_buffinish(lua_State* L, TString* ts); diff --git a/VM/src/ludata.cpp b/VM/src/ludata.cpp new file mode 100644 index 00000000..d180c388 --- /dev/null +++ b/VM/src/ludata.cpp @@ -0,0 +1,37 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +// This code is based on Lua 5.x implementation licensed under MIT License; see lua_LICENSE.txt for details +#include "ludata.h" + +#include "lgc.h" +#include "lmem.h" + +#include + +Udata* luaU_newudata(lua_State* L, size_t s, int tag) +{ + if (s > INT_MAX - sizeof(Udata)) + luaM_toobig(L); + Udata* u = luaM_new(L, Udata, sizeudata(s), L->activememcat); + luaC_link(L, u, LUA_TUSERDATA); + u->len = int(s); + u->metatable = NULL; + LUAU_ASSERT(tag >= 0 && tag <= 255); + u->tag = uint8_t(tag); + return u; +} + +void luaU_freeudata(lua_State* L, Udata* u) +{ + LUAU_ASSERT(u->tag < LUA_UTAG_LIMIT || u->tag == UTAG_IDTOR); + + void (*dtor)(void*) = nullptr; + if (u->tag == UTAG_IDTOR) + memcpy(&dtor, &u->data + u->len - sizeof(dtor), sizeof(dtor)); + else if (u->tag) + dtor = L->global->udatagc[u->tag]; + + if (dtor) + dtor(u->data); + + luaM_free(L, u, sizeudata(u->len), u->memcat); +} diff --git a/VM/src/ludata.h b/VM/src/ludata.h new file mode 100644 index 00000000..59cb85bd --- /dev/null +++ b/VM/src/ludata.h @@ -0,0 +1,13 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +// This code is based on Lua 5.x implementation licensed under MIT License; see lua_LICENSE.txt for details +#pragma once + +#include "lobject.h" + +/* special tag value is used for user data with inline dtors */ +#define UTAG_IDTOR LUA_UTAG_LIMIT + +#define sizeudata(len) (offsetof(Udata, data) + len) + +LUAI_FUNC Udata* luaU_newudata(lua_State* L, size_t s, int tag); +LUAI_FUNC void luaU_freeudata(lua_State* L, Udata* u); diff --git a/VM/src/lvmexecute.cpp b/VM/src/lvmexecute.cpp index bf8d493e..cebeeb58 100644 --- a/VM/src/lvmexecute.cpp +++ b/VM/src/lvmexecute.cpp @@ -63,7 +63,8 @@ #define VM_KV(i) (LUAU_ASSERT(unsigned(i) < unsigned(cl->l.p->sizek)), &k[i]) #define VM_UV(i) (LUAU_ASSERT(unsigned(i) < unsigned(cl->nupvalues)), &cl->l.uprefs[i]) -#define VM_PATCH_C(pc, slot) ((uint8_t*)(pc))[3] = uint8_t(slot) +#define VM_PATCH_C(pc, slot) *const_cast(pc) = ((uint8_t(slot) << 24) | (0x00ffffffu & *(pc))) +#define VM_PATCH_E(pc, slot) *const_cast(pc) = ((uint32_t(slot) << 8) | (0x000000ffu & *(pc))) // NOTE: If debugging the Luau code, disable this macro to prevent timeouts from // occurring when tracing code in Visual Studio / XCode @@ -120,7 +121,7 @@ */ #if VM_USE_CGOTO #define VM_CASE(op) CASE_##op: -#define VM_NEXT() goto*(SingleStep ? &&dispatch : kDispatchTable[*(uint8_t*)pc]) +#define VM_NEXT() goto*(SingleStep ? &&dispatch : kDispatchTable[LUAU_INSN_OP(*pc)]) #define VM_CONTINUE(op) goto* kDispatchTable[uint8_t(op)] #else #define VM_CASE(op) case op: @@ -325,7 +326,7 @@ static void luau_execute(lua_State* L) // ... and singlestep logic :) if (SingleStep) { - if (L->global->cb.debugstep && !luau_skipstep(*(uint8_t*)pc)) + if (L->global->cb.debugstep && !luau_skipstep(LUAU_INSN_OP(*pc))) { VM_PROTECT(luau_callhook(L, L->global->cb.debugstep, NULL)); @@ -335,13 +336,12 @@ static void luau_execute(lua_State* L) } #if VM_USE_CGOTO - VM_CONTINUE(*(uint8_t*)pc); + VM_CONTINUE(LUAU_INSN_OP(*pc)); #endif } #if !VM_USE_CGOTO - // Note: this assumes that LUAU_INSN_OP() decodes the first byte (aka least significant byte in the little endian encoding) - size_t dispatchOp = *(uint8_t*)pc; + size_t dispatchOp = LUAU_INSN_OP(*pc); dispatchContinue: switch (dispatchOp) @@ -2577,7 +2577,7 @@ static void luau_execute(lua_State* L) // update hits with saturated add and patch the instruction in place hits = (hits < (1 << 23) - 1) ? hits + 1 : hits; - ((uint32_t*)pc)[-1] = LOP_COVERAGE | (uint32_t(hits) << 8); + VM_PATCH_E(pc - 1, hits); VM_NEXT(); } diff --git a/bench/tests/sunspider/3d-raytrace.lua b/bench/tests/sunspider/3d-raytrace.lua index 60e4f61e..c8f6b5dc 100644 --- a/bench/tests/sunspider/3d-raytrace.lua +++ b/bench/tests/sunspider/3d-raytrace.lua @@ -451,15 +451,16 @@ function raytraceScene() end function arrayToCanvasCommands(pixels) - local s = 'Test\nvar pixels = ['; + local s = {}; + table.insert(s, 'Test\nvar pixels = ['); for y = 0,size-1 do - s = s .. "["; + table.insert(s, "["); for x = 0,size-1 do - s = s .. "[" .. math.floor(pixels[y + 1][x + 1][1] * 255) .. "," .. math.floor(pixels[y + 1][x + 1][2] * 255) .. "," .. math.floor(pixels[y + 1][x + 1][3] * 255) .. "],"; + table.insert(s, "[" .. math.floor(pixels[y + 1][x + 1][1] * 255) .. "," .. math.floor(pixels[y + 1][x + 1][2] * 255) .. "," .. math.floor(pixels[y + 1][x + 1][3] * 255) .. "],"); end - s = s .. "],"; + table.insert(s, "],"); end - s = s .. '];\n var canvas = document.getElementById("renderCanvas").getContext("2d");\n\ + table.insert(s, '];\n var canvas = document.getElementById("renderCanvas").getContext("2d");\n\ \n\ \n\ var size = ' .. size .. ';\n\ @@ -479,9 +480,9 @@ for (var y = 0; y < size; y++) {\n\ canvas.setFillColor(l[0], l[1], l[2], 1);\n\ canvas.fillRect(x, y, 1, 1);\n\ }\n\ -}'; +}'); - return s; + return table.concat(s); end testOutput = arrayToCanvasCommands(raytraceScene()); diff --git a/tests/Autocomplete.test.cpp b/tests/Autocomplete.test.cpp index 3b74a99e..62a9999b 100644 --- a/tests/Autocomplete.test.cpp +++ b/tests/Autocomplete.test.cpp @@ -513,8 +513,6 @@ TEST_CASE_FIXTURE(ACFixture, "dont_offer_any_suggestions_from_the_end_of_a_comme TEST_CASE_FIXTURE(ACFixture, "dont_offer_any_suggestions_from_within_a_broken_comment") { - ScopedFastFlag sff{"LuauCaptureBrokenCommentSpans", true}; - check(R"( --[[ @1 )"); @@ -526,8 +524,6 @@ TEST_CASE_FIXTURE(ACFixture, "dont_offer_any_suggestions_from_within_a_broken_co TEST_CASE_FIXTURE(ACFixture, "dont_offer_any_suggestions_from_within_a_broken_comment_at_the_very_end_of_the_file") { - ScopedFastFlag sff{"LuauCaptureBrokenCommentSpans", true}; - check("--[[@1"); auto ac = autocomplete('1'); @@ -2625,4 +2621,55 @@ local a: A<(number, s@1> CHECK(ac.entryMap.count("string")); } +TEST_CASE_FIXTURE(ACFixture, "autocomplete_first_function_arg_expected_type") +{ + ScopedFastFlag luauAutocompleteAvoidMutation("LuauAutocompleteAvoidMutation", true); + ScopedFastFlag luauAutocompleteFirstArg("LuauAutocompleteFirstArg", true); + + check(R"( +local function foo1() return 1 end +local function foo2() return "1" end + +local function bar0() return "got" .. a end +local function bar1(a: number) return "got " .. a end +local function bar2(a: number, b: string) return "got " .. a .. b end + +local t = {} +function t:bar1(a: number) return "got " .. a end + +local r1 = bar0(@1) +local r2 = bar1(@2) +local r3 = bar2(@3) +local r4 = t:bar1(@4) + )"); + + auto ac = autocomplete('1'); + + REQUIRE(ac.entryMap.count("foo1")); + CHECK(ac.entryMap["foo1"].typeCorrect == TypeCorrectKind::None); + REQUIRE(ac.entryMap.count("foo2")); + CHECK(ac.entryMap["foo2"].typeCorrect == TypeCorrectKind::None); + + ac = autocomplete('2'); + + REQUIRE(ac.entryMap.count("foo1")); + CHECK(ac.entryMap["foo1"].typeCorrect == TypeCorrectKind::CorrectFunctionResult); + REQUIRE(ac.entryMap.count("foo2")); + CHECK(ac.entryMap["foo2"].typeCorrect == TypeCorrectKind::None); + + ac = autocomplete('3'); + + REQUIRE(ac.entryMap.count("foo1")); + CHECK(ac.entryMap["foo1"].typeCorrect == TypeCorrectKind::CorrectFunctionResult); + REQUIRE(ac.entryMap.count("foo2")); + CHECK(ac.entryMap["foo2"].typeCorrect == TypeCorrectKind::None); + + ac = autocomplete('4'); + + REQUIRE(ac.entryMap.count("foo1")); + CHECK(ac.entryMap["foo1"].typeCorrect == TypeCorrectKind::CorrectFunctionResult); + REQUIRE(ac.entryMap.count("foo2")); + CHECK(ac.entryMap["foo2"].typeCorrect == TypeCorrectKind::None); +} + TEST_SUITE_END(); diff --git a/tests/Compiler.test.cpp b/tests/Compiler.test.cpp index 6ba39ada..95811b3f 100644 --- a/tests/Compiler.test.cpp +++ b/tests/Compiler.test.cpp @@ -10,9 +10,6 @@ #include #include -LUAU_FASTFLAG(LuauPreloadClosures) -LUAU_FASTFLAG(LuauGenericSpecialGlobals) - using namespace Luau; static std::string compileFunction(const char* source, uint32_t id) @@ -74,20 +71,10 @@ TEST_CASE("BasicFunction") bcb.setDumpFlags(Luau::BytecodeBuilder::Dump_Code); Luau::compileOrThrow(bcb, "local function foo(a, b) return b end"); - if (FFlag::LuauPreloadClosures) - { - CHECK_EQ("\n" + bcb.dumpFunction(1), R"( + CHECK_EQ("\n" + bcb.dumpFunction(1), R"( DUPCLOSURE R0 K0 RETURN R0 0 )"); - } - else - { - CHECK_EQ("\n" + bcb.dumpFunction(1), R"( -NEWCLOSURE R0 P0 -RETURN R0 0 -)"); - } CHECK_EQ("\n" + bcb.dumpFunction(0), R"( RETURN R1 1 @@ -2859,47 +2846,35 @@ CAPTURE UPVAL U1 RETURN R0 1 )"); - if (FFlag::LuauPreloadClosures) - { - // recursive capture - CHECK_EQ("\n" + compileFunction("local function foo() return foo() end", 1), R"( + // recursive capture + CHECK_EQ("\n" + compileFunction("local function foo() return foo() end", 1), R"( DUPCLOSURE R0 K0 CAPTURE VAL R0 RETURN R0 0 )"); - // multi-level recursive capture - CHECK_EQ("\n" + compileFunction("local function foo() return function() return foo() end end", 1), R"( + // multi-level recursive capture + CHECK_EQ("\n" + compileFunction("local function foo() return function() return foo() end end", 1), R"( DUPCLOSURE R0 K0 CAPTURE UPVAL U0 RETURN R0 1 )"); - // multi-level recursive capture where function isn't top-level - // note: this should probably be optimized to DUPCLOSURE but doing that requires a different upval tracking flow in the compiler - CHECK_EQ("\n" + compileFunction(R"( + // multi-level recursive capture where function isn't top-level + // note: this should probably be optimized to DUPCLOSURE but doing that requires a different upval tracking flow in the compiler + CHECK_EQ("\n" + compileFunction(R"( local function foo() local function bar() return function() return bar() end end end )", - 1), - R"( + 1), + R"( NEWCLOSURE R0 P0 CAPTURE UPVAL U0 RETURN R0 1 )"); - } - else - { - // recursive capture - CHECK_EQ("\n" + compileFunction("local function foo() return foo() end", 1), R"( -NEWCLOSURE R0 P0 -CAPTURE VAL R0 -RETURN R0 0 -)"); - } } TEST_CASE("OutOfLocals") @@ -3504,8 +3479,6 @@ local t = { TEST_CASE("ConstantClosure") { - ScopedFastFlag sff("LuauPreloadClosures", true); - // closures without upvalues are created when bytecode is loaded CHECK_EQ("\n" + compileFunction(R"( return function() end @@ -3570,8 +3543,6 @@ RETURN R0 1 TEST_CASE("SharedClosure") { - ScopedFastFlag sff1("LuauPreloadClosures", true); - // closures can be shared even if functions refer to upvalues, as long as upvalues are top-level CHECK_EQ("\n" + compileFunction(R"( local val = ... diff --git a/tests/Conformance.test.cpp b/tests/Conformance.test.cpp index b2aad316..b055a38e 100644 --- a/tests/Conformance.test.cpp +++ b/tests/Conformance.test.cpp @@ -123,8 +123,8 @@ int lua_silence(lua_State* L) using StateRef = std::unique_ptr; -static StateRef runConformance( - const char* name, void (*setup)(lua_State* L) = nullptr, void (*yield)(lua_State* L) = nullptr, lua_State* initialLuaState = nullptr) +static StateRef runConformance(const char* name, void (*setup)(lua_State* L) = nullptr, void (*yield)(lua_State* L) = nullptr, + lua_State* initialLuaState = nullptr, lua_CompileOptions* copts = nullptr) { std::string path = __FILE__; path.erase(path.find_last_of("\\/")); @@ -180,13 +180,8 @@ static StateRef runConformance( std::string chunkname = "=" + std::string(name); - lua_CompileOptions copts = {}; - copts.optimizationLevel = 1; // default - copts.debugLevel = 2; // for debugger tests - copts.vectorCtor = "vector"; // for vector tests - size_t bytecodeSize = 0; - char* bytecode = luau_compile(source.data(), source.size(), &copts, &bytecodeSize); + char* bytecode = luau_compile(source.data(), source.size(), copts, &bytecodeSize); int result = luau_load(L, chunkname.c_str(), bytecode, bytecodeSize, 0); free(bytecode); @@ -373,29 +368,37 @@ TEST_CASE("Vector") { ScopedFastFlag sff{"LuauIfElseExpressionBaseSupport", true}; - runConformance("vector.lua", [](lua_State* L) { - lua_pushcfunction(L, lua_vector, "vector"); - lua_setglobal(L, "vector"); + lua_CompileOptions copts = {}; + copts.optimizationLevel = 1; + copts.debugLevel = 1; + copts.vectorCtor = "vector"; + + runConformance( + "vector.lua", + [](lua_State* L) { + lua_pushcfunction(L, lua_vector, "vector"); + lua_setglobal(L, "vector"); #if LUA_VECTOR_SIZE == 4 - lua_pushvector(L, 0.0f, 0.0f, 0.0f, 0.0f); + lua_pushvector(L, 0.0f, 0.0f, 0.0f, 0.0f); #else - lua_pushvector(L, 0.0f, 0.0f, 0.0f); + lua_pushvector(L, 0.0f, 0.0f, 0.0f); #endif - luaL_newmetatable(L, "vector"); + luaL_newmetatable(L, "vector"); - lua_pushstring(L, "__index"); - lua_pushcfunction(L, lua_vector_index, nullptr); - lua_settable(L, -3); + lua_pushstring(L, "__index"); + lua_pushcfunction(L, lua_vector_index, nullptr); + lua_settable(L, -3); - lua_pushstring(L, "__namecall"); - lua_pushcfunction(L, lua_vector_namecall, nullptr); - lua_settable(L, -3); + lua_pushstring(L, "__namecall"); + lua_pushcfunction(L, lua_vector_namecall, nullptr); + lua_settable(L, -3); - lua_setreadonly(L, -1, true); - lua_setmetatable(L, -2); - lua_pop(L, 1); - }); + lua_setreadonly(L, -1, true); + lua_setmetatable(L, -2); + lua_pop(L, 1); + }, + nullptr, nullptr, &copts); } static void populateRTTI(lua_State* L, Luau::TypeId type) @@ -499,6 +502,10 @@ TEST_CASE("Debugger") breakhits = 0; interruptedthread = nullptr; + lua_CompileOptions copts = {}; + copts.optimizationLevel = 1; + copts.debugLevel = 2; + runConformance( "debugger.lua", [](lua_State* L) { @@ -614,7 +621,8 @@ TEST_CASE("Debugger") lua_resume(interruptedthread, nullptr, 0); interruptedthread = nullptr; } - }); + }, + nullptr, &copts); CHECK(breakhits == 10); // 2 hits per breakpoint } @@ -863,4 +871,46 @@ TEST_CASE("TagMethodError") }); } +TEST_CASE("Coverage") +{ + lua_CompileOptions copts = {}; + copts.optimizationLevel = 1; + copts.debugLevel = 1; + copts.coverageLevel = 2; + + runConformance( + "coverage.lua", + [](lua_State* L) { + lua_pushcfunction( + L, + [](lua_State* L) -> int { + luaL_argexpected(L, lua_isLfunction(L, 1), 1, "function"); + + lua_newtable(L); + lua_getcoverage(L, 1, L, [](void* context, const char* function, int linedefined, int depth, const int* hits, size_t size) { + lua_State* L = static_cast(context); + + lua_newtable(L); + + lua_pushstring(L, function); + lua_setfield(L, -2, "name"); + + for (size_t i = 0; i < size; ++i) + if (hits[i] != -1) + { + lua_pushinteger(L, hits[i]); + lua_rawseti(L, -2, int(i)); + } + + lua_rawseti(L, -2, lua_objlen(L, -2) + 1); + }); + + return 1; + }, + "getcoverage"); + lua_setglobal(L, "getcoverage"); + }, + nullptr, nullptr, &copts); +} + TEST_SUITE_END(); diff --git a/tests/Parser.test.cpp b/tests/Parser.test.cpp index 72d3a9a6..5abcb09a 100644 --- a/tests/Parser.test.cpp +++ b/tests/Parser.test.cpp @@ -2303,8 +2303,6 @@ TEST_CASE_FIXTURE(Fixture, "capture_comments") TEST_CASE_FIXTURE(Fixture, "capture_broken_comment_at_the_start_of_the_file") { - ScopedFastFlag sff{"LuauCaptureBrokenCommentSpans", true}; - ParseOptions options; options.captureComments = true; @@ -2319,8 +2317,6 @@ TEST_CASE_FIXTURE(Fixture, "capture_broken_comment_at_the_start_of_the_file") TEST_CASE_FIXTURE(Fixture, "capture_broken_comment") { - ScopedFastFlag sff{"LuauCaptureBrokenCommentSpans", true}; - ParseOptions options; options.captureComments = true; diff --git a/tests/TypeInfer.annotations.test.cpp b/tests/TypeInfer.annotations.test.cpp index 091c2f01..71ff4e1b 100644 --- a/tests/TypeInfer.annotations.test.cpp +++ b/tests/TypeInfer.annotations.test.cpp @@ -207,6 +207,36 @@ TEST_CASE_FIXTURE(Fixture, "as_expr_does_not_propagate_type_info") CHECK_EQ("number", toString(requireType("b"))); } +TEST_CASE_FIXTURE(Fixture, "as_expr_is_bidirectional") +{ + ScopedFastFlag sff{"LuauBidirectionalAsExpr", true}; + + CheckResult result = check(R"( + local a = 55 :: number? + local b = a :: number + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ("number?", toString(requireType("a"))); + CHECK_EQ("number", toString(requireType("b"))); +} + +TEST_CASE_FIXTURE(Fixture, "as_expr_warns_on_unrelated_cast") +{ + ScopedFastFlag sff{"LuauBidirectionalAsExpr", true}; + ScopedFastFlag sff2{"LuauErrorRecoveryType", true}; + + CheckResult result = check(R"( + local a = 55 :: string + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + CHECK_EQ("Cannot cast 'number' into 'string' because the types are unrelated", toString(result.errors[0])); + CHECK_EQ("string", toString(requireType("a"))); +} + TEST_CASE_FIXTURE(Fixture, "type_annotations_inside_function_bodies") { CheckResult result = check(R"( diff --git a/tests/TypeInfer.builtins.test.cpp b/tests/TypeInfer.builtins.test.cpp index 1e2eae14..1d8135d4 100644 --- a/tests/TypeInfer.builtins.test.cpp +++ b/tests/TypeInfer.builtins.test.cpp @@ -7,6 +7,8 @@ #include "doctest.h" +LUAU_FASTFLAG(LuauFixTonumberReturnType) + using namespace Luau; TEST_SUITE_BEGIN("BuiltinTests"); @@ -814,6 +816,30 @@ TEST_CASE_FIXTURE(Fixture, "string_format_report_all_type_errors_at_correct_posi CHECK_EQ(TypeErrorData(TypeMismatch{stringType, booleanType}), result.errors[2].data); } +TEST_CASE_FIXTURE(Fixture, "tonumber_returns_optional_number_type") +{ + CheckResult result = check(R"( + --!strict + local b: number = tonumber('asdf') + )"); + + if (FFlag::LuauFixTonumberReturnType) + { + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ("Type 'number?' could not be converted into 'number'", toString(result.errors[0])); + } +} + +TEST_CASE_FIXTURE(Fixture, "tonumber_returns_optional_number_type2") +{ + CheckResult result = check(R"( + --!strict + local b: number = tonumber('asdf') or 1 + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + TEST_CASE_FIXTURE(Fixture, "dont_add_definitions_to_persistent_types") { CheckResult result = check(R"( diff --git a/tests/TypeInfer.generics.test.cpp b/tests/TypeInfer.generics.test.cpp index aba50891..b62044fa 100644 --- a/tests/TypeInfer.generics.test.cpp +++ b/tests/TypeInfer.generics.test.cpp @@ -9,6 +9,8 @@ using namespace Luau; +LUAU_FASTFLAG(LuauExtendedFunctionMismatchError) + TEST_SUITE_BEGIN("GenericsTests"); TEST_CASE_FIXTURE(Fixture, "check_generic_function") @@ -644,4 +646,42 @@ f(1, 2, 3) CHECK_EQ(toString(*ty, opts), "(a: number, number, number) -> ()"); } +TEST_CASE_FIXTURE(Fixture, "error_detailed_function_mismatch_generic_types") +{ + CheckResult result = check(R"( +type C = () -> () +type D = () -> () + +local c: C +local d: D = c + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + if (FFlag::LuauExtendedFunctionMismatchError) + CHECK_EQ( + toString(result.errors[0]), R"(Type '() -> ()' could not be converted into '() -> ()'; different number of generic type parameters)"); + else + CHECK_EQ(toString(result.errors[0]), R"(Type '() -> ()' could not be converted into '() -> ()')"); +} + +TEST_CASE_FIXTURE(Fixture, "error_detailed_function_mismatch_generic_pack") +{ + CheckResult result = check(R"( +type C = () -> () +type D = () -> () + +local c: C +local d: D = c + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + if (FFlag::LuauExtendedFunctionMismatchError) + CHECK_EQ(toString(result.errors[0]), + R"(Type '() -> ()' could not be converted into '() -> ()'; different number of generic type pack parameters)"); + else + CHECK_EQ(toString(result.errors[0]), R"(Type '() -> ()' could not be converted into '() -> ()')"); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.refinements.test.cpp b/tests/TypeInfer.refinements.test.cpp index fe8e7ff9..688680c1 100644 --- a/tests/TypeInfer.refinements.test.cpp +++ b/tests/TypeInfer.refinements.test.cpp @@ -279,7 +279,7 @@ TEST_CASE_FIXTURE(Fixture, "assert_non_binary_expressions_actually_resolve_const TEST_CASE_FIXTURE(Fixture, "assign_table_with_refined_property_with_a_similar_type_is_illegal") { - ScopedFastFlag luauTableSubtypingVariance{"LuauTableSubtypingVariance", true}; + ScopedFastFlag LuauTableSubtypingVariance2{"LuauTableSubtypingVariance2", true}; ScopedFastFlag luauExtendedTypeMismatchError{"LuauExtendedTypeMismatchError", true}; CheckResult result = check(R"( @@ -1085,4 +1085,19 @@ TEST_CASE_FIXTURE(Fixture, "type_comparison_ifelse_expression") CHECK_EQ("any", toString(requireTypeAtPosition({6, 66}))); } +TEST_CASE_FIXTURE(Fixture, "apply_refinements_on_astexprindexexpr_whose_subscript_expr_is_constant_string") +{ + ScopedFastFlag sff{"LuauRefiLookupFromIndexExpr", true}; + + CheckResult result = check(R"( + type T = { [string]: { prop: number }? } + local t: T = {} + if t["hello"] then + local foo = t["hello"].prop + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.singletons.test.cpp b/tests/TypeInfer.singletons.test.cpp index 5f95efd5..1621ef32 100644 --- a/tests/TypeInfer.singletons.test.cpp +++ b/tests/TypeInfer.singletons.test.cpp @@ -374,4 +374,54 @@ TEST_CASE_FIXTURE(Fixture, "table_properties_type_error_escapes") toString(result.errors[0])); } +TEST_CASE_FIXTURE(Fixture, "error_detailed_tagged_union_mismatch_string") +{ + ScopedFastFlag sffs[] = { + {"LuauSingletonTypes", true}, + {"LuauParseSingletonTypes", true}, + {"LuauUnionHeuristic", true}, + {"LuauExpectedTypesOfProperties", true}, + {"LuauExtendedUnionMismatchError", true}, + }; + + CheckResult result = check(R"( +type Cat = { tag: 'cat', catfood: string } +type Dog = { tag: 'dog', dogfood: string } +type Animal = Cat | Dog + +local a: Animal = { tag = 'cat', cafood = 'something' } + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ(R"(Type 'a' could not be converted into 'Cat | Dog' +caused by: + None of the union options are compatible. For example: Table type 'a' not compatible with type 'Cat' because the former is missing field 'catfood')", + toString(result.errors[0])); +} + +TEST_CASE_FIXTURE(Fixture, "error_detailed_tagged_union_mismatch_bool") +{ + ScopedFastFlag sffs[] = { + {"LuauSingletonTypes", true}, + {"LuauParseSingletonTypes", true}, + {"LuauUnionHeuristic", true}, + {"LuauExpectedTypesOfProperties", true}, + {"LuauExtendedUnionMismatchError", true}, + }; + + CheckResult result = check(R"( +type Good = { success: true, result: string } +type Bad = { success: false, error: string } +type Result = Good | Bad + +local a: Result = { success = false, result = 'something' } + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ(R"(Type 'a' could not be converted into 'Bad | Good' +caused by: + None of the union options are compatible. For example: Table type 'a' not compatible with type 'Bad' because the former is missing field 'error')", + toString(result.errors[0])); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.tables.test.cpp b/tests/TypeInfer.tables.test.cpp index cb72faaf..3ea9b80c 100644 --- a/tests/TypeInfer.tables.test.cpp +++ b/tests/TypeInfer.tables.test.cpp @@ -12,6 +12,8 @@ using namespace Luau; +LUAU_FASTFLAG(LuauExtendedFunctionMismatchError) + TEST_SUITE_BEGIN("TableTests"); TEST_CASE_FIXTURE(Fixture, "basic") @@ -275,7 +277,7 @@ TEST_CASE_FIXTURE(Fixture, "open_table_unification") TEST_CASE_FIXTURE(Fixture, "open_table_unification_2") { - ScopedFastFlag sff{"LuauTableSubtypingVariance", true}; + ScopedFastFlag sff{"LuauTableSubtypingVariance2", true}; CheckResult result = check(R"( local a = {} @@ -346,7 +348,7 @@ TEST_CASE_FIXTURE(Fixture, "table_param_row_polymorphism_1") TEST_CASE_FIXTURE(Fixture, "table_param_row_polymorphism_2") { - ScopedFastFlag sff{"LuauTableSubtypingVariance", true}; + ScopedFastFlag sff{"LuauTableSubtypingVariance2", true}; CheckResult result = check(R"( --!strict @@ -369,7 +371,7 @@ TEST_CASE_FIXTURE(Fixture, "table_param_row_polymorphism_2") TEST_CASE_FIXTURE(Fixture, "table_param_row_polymorphism_3") { - ScopedFastFlag sff{"LuauTableSubtypingVariance", true}; + ScopedFastFlag sff{"LuauTableSubtypingVariance2", true}; CheckResult result = check(R"( local T = {} @@ -476,7 +478,7 @@ TEST_CASE_FIXTURE(Fixture, "ok_to_add_property_to_free_table") TEST_CASE_FIXTURE(Fixture, "okay_to_add_property_to_unsealed_tables_by_assignment") { - ScopedFastFlag sff{"LuauTableSubtypingVariance", true}; + ScopedFastFlag sff{"LuauTableSubtypingVariance2", true}; CheckResult result = check(R"( --!strict @@ -511,7 +513,7 @@ TEST_CASE_FIXTURE(Fixture, "okay_to_add_property_to_unsealed_tables_by_function_ TEST_CASE_FIXTURE(Fixture, "width_subtyping") { - ScopedFastFlag sff{"LuauTableSubtypingVariance", true}; + ScopedFastFlag sff{"LuauTableSubtypingVariance2", true}; CheckResult result = check(R"( --!strict @@ -771,7 +773,7 @@ TEST_CASE_FIXTURE(Fixture, "infer_indexer_for_left_unsealed_table_from_right_han TEST_CASE_FIXTURE(Fixture, "sealed_table_value_can_infer_an_indexer") { - ScopedFastFlag sff{"LuauTableSubtypingVariance", true}; + ScopedFastFlag sff{"LuauTableSubtypingVariance2", true}; CheckResult result = check(R"( local t: { a: string, [number]: string } = { a = "foo" } @@ -782,7 +784,7 @@ TEST_CASE_FIXTURE(Fixture, "sealed_table_value_can_infer_an_indexer") TEST_CASE_FIXTURE(Fixture, "array_factory_function") { - ScopedFastFlag sff{"LuauTableSubtypingVariance", true}; + ScopedFastFlag sff{"LuauTableSubtypingVariance2", true}; CheckResult result = check(R"( function empty() return {} end @@ -1465,7 +1467,7 @@ TEST_CASE_FIXTURE(Fixture, "casting_tables_with_props_into_table_with_indexer2") TEST_CASE_FIXTURE(Fixture, "casting_tables_with_props_into_table_with_indexer3") { - ScopedFastFlag sff{"LuauTableSubtypingVariance", true}; + ScopedFastFlag sff{"LuauTableSubtypingVariance2", true}; CheckResult result = check(R"( local function foo(a: {[string]: number, a: string}) end @@ -1550,7 +1552,7 @@ TEST_CASE_FIXTURE(Fixture, "table_subtyping_with_extra_props_dont_report_multipl TEST_CASE_FIXTURE(Fixture, "table_subtyping_with_extra_props_is_ok") { - ScopedFastFlag sff{"LuauTableSubtypingVariance", true}; + ScopedFastFlag sff{"LuauTableSubtypingVariance2", true}; CheckResult result = check(R"( local vec3 = {x = 1, y = 2, z = 3} @@ -1937,7 +1939,7 @@ TEST_CASE_FIXTURE(Fixture, "table_insert_should_cope_with_optional_properties_in TEST_CASE_FIXTURE(Fixture, "table_insert_should_cope_with_optional_properties_in_strict") { - ScopedFastFlag sff{"LuauTableSubtypingVariance", true}; + ScopedFastFlag sff{"LuauTableSubtypingVariance2", true}; CheckResult result = check(R"( --!strict @@ -1952,7 +1954,7 @@ TEST_CASE_FIXTURE(Fixture, "table_insert_should_cope_with_optional_properties_in TEST_CASE_FIXTURE(Fixture, "error_detailed_prop") { - ScopedFastFlag luauTableSubtypingVariance{"LuauTableSubtypingVariance", true}; // Only for new path + ScopedFastFlag LuauTableSubtypingVariance2{"LuauTableSubtypingVariance2", true}; // Only for new path ScopedFastFlag luauExtendedTypeMismatchError{"LuauExtendedTypeMismatchError", true}; CheckResult result = check(R"( @@ -1971,7 +1973,7 @@ caused by: TEST_CASE_FIXTURE(Fixture, "error_detailed_prop_nested") { - ScopedFastFlag luauTableSubtypingVariance{"LuauTableSubtypingVariance", true}; // Only for new path + ScopedFastFlag LuauTableSubtypingVariance2{"LuauTableSubtypingVariance2", true}; // Only for new path ScopedFastFlag luauExtendedTypeMismatchError{"LuauExtendedTypeMismatchError", true}; CheckResult result = check(R"( @@ -1995,7 +1997,7 @@ caused by: TEST_CASE_FIXTURE(Fixture, "error_detailed_metatable_prop") { - ScopedFastFlag luauTableSubtypingVariance{"LuauTableSubtypingVariance", true}; // Only for new path + ScopedFastFlag LuauTableSubtypingVariance2{"LuauTableSubtypingVariance2", true}; // Only for new path ScopedFastFlag luauExtendedTypeMismatchError{"LuauExtendedTypeMismatchError", true}; CheckResult result = check(R"( @@ -2015,11 +2017,22 @@ caused by: caused by: Property 'y' is not compatible. Type 'string' could not be converted into 'number')"); - CHECK_EQ(toString(result.errors[1]), R"(Type 'b2' could not be converted into 'a2' + if (FFlag::LuauExtendedFunctionMismatchError) + { + CHECK_EQ(toString(result.errors[1]), R"(Type 'b2' could not be converted into 'a2' +caused by: + Type '{| __call: (a, b) -> () |}' could not be converted into '{| __call: (a) -> () |}' +caused by: + Property '__call' is not compatible. Type '(a, b) -> ()' could not be converted into '(a) -> ()'; different number of generic type parameters)"); + } + else + { + CHECK_EQ(toString(result.errors[1]), R"(Type 'b2' could not be converted into 'a2' caused by: Type '{| __call: (a, b) -> () |}' could not be converted into '{| __call: (a) -> () |}' caused by: Property '__call' is not compatible. Type '(a, b) -> ()' could not be converted into '(a) -> ()')"); + } } TEST_CASE_FIXTURE(Fixture, "explicitly_typed_table") @@ -2027,7 +2040,7 @@ TEST_CASE_FIXTURE(Fixture, "explicitly_typed_table") ScopedFastFlag sffs[] { {"LuauPropertiesGetExpectedType", true}, {"LuauExpectedTypesOfProperties", true}, - {"LuauTableSubtypingVariance", true}, + {"LuauTableSubtypingVariance2", true}, }; CheckResult result = check(R"( @@ -2048,7 +2061,7 @@ TEST_CASE_FIXTURE(Fixture, "explicitly_typed_table_error") ScopedFastFlag sffs[] { {"LuauPropertiesGetExpectedType", true}, {"LuauExpectedTypesOfProperties", true}, - {"LuauTableSubtypingVariance", true}, + {"LuauTableSubtypingVariance2", true}, {"LuauExtendedTypeMismatchError", true}, }; @@ -2076,7 +2089,7 @@ TEST_CASE_FIXTURE(Fixture, "explicitly_typed_table_with_indexer") ScopedFastFlag sffs[] { {"LuauPropertiesGetExpectedType", true}, {"LuauExpectedTypesOfProperties", true}, - {"LuauTableSubtypingVariance", true}, + {"LuauTableSubtypingVariance2", true}, }; CheckResult result = check(R"( @@ -2092,4 +2105,18 @@ a.p = { x = 9 } LUAU_REQUIRE_NO_ERRORS(result); } +TEST_CASE_FIXTURE(Fixture, "recursive_metatable_type_call") +{ + ScopedFastFlag luauFixRecursiveMetatableCall{"LuauFixRecursiveMetatableCall", true}; + + CheckResult result = check(R"( +local b +b = setmetatable({}, {__call = b}) +b() + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ(toString(result.errors[0]), R"(Cannot call non-function t1 where t1 = { @metatable {| __call: t1 |}, { } })"); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.test.cpp b/tests/TypeInfer.test.cpp index e3222a41..ad9ea827 100644 --- a/tests/TypeInfer.test.cpp +++ b/tests/TypeInfer.test.cpp @@ -16,6 +16,7 @@ LUAU_FASTFLAG(LuauFixLocationSpanTableIndexExpr) LUAU_FASTFLAG(LuauEqConstraint) +LUAU_FASTFLAG(LuauExtendedFunctionMismatchError) using namespace Luau; @@ -2084,7 +2085,7 @@ TEST_CASE_FIXTURE(Fixture, "primitive_arith_no_metatable") { CheckResult result = check(R"( function add(a: number, b: string) - return a + tonumber(b), a .. b + return a + (tonumber(b) :: number), a .. b end local n, s = add(2,"3") )"); @@ -2485,7 +2486,7 @@ TEST_CASE_FIXTURE(Fixture, "inferring_crazy_table_should_also_be_quick") CheckResult result = check(R"( --!strict function f(U) - U(w:s(an):c()():c():U(s):c():c():U(s):c():U(s):cU()):c():U(s):c():U(s):c():c():U(s):c():U(s):cU() + U(w:s(an):c()():c():U(s):c():c():U(s):c():U(s):cU()):c():U(s):c():U(s):c():c():U(s):c():U(s):cU() end )"); @@ -3329,7 +3330,7 @@ TEST_CASE_FIXTURE(Fixture, "relation_op_on_any_lhs_where_rhs_maybe_has_metatable { CheckResult result = check(R"( local x - print((x == true and (x .. "y")) .. 1) + print((x == true and (x .. "y")) .. 1) )"); LUAU_REQUIRE_ERROR_COUNT(1, result); @@ -4473,7 +4474,18 @@ f(function(a, b, c, ...) return a + b end) )"); LUAU_REQUIRE_ERRORS(result); - CHECK_EQ("Type '(number, number, a) -> number' could not be converted into '(number, number) -> number'", toString(result.errors[0])); + + if (FFlag::LuauExtendedFunctionMismatchError) + { + CHECK_EQ(R"(Type '(number, number, a) -> number' could not be converted into '(number, number) -> number' +caused by: + Argument count mismatch. Function expects 3 arguments, but only 2 are specified)", + toString(result.errors[0])); + } + else + { + CHECK_EQ(R"(Type '(number, number, a) -> number' could not be converted into '(number, number) -> number')", toString(result.errors[0])); + } // Infer from variadic packs into elements result = check(R"( @@ -4604,7 +4616,17 @@ local c = sumrec(function(x, y, f) return f(x, y) end) -- type binders are not i )"); LUAU_REQUIRE_ERRORS(result); - CHECK_EQ("Type '(a, b, (a, b) -> (c...)) -> (c...)' could not be converted into '(a, a, (a, a) -> a) -> a'", toString(result.errors[0])); + if (FFlag::LuauExtendedFunctionMismatchError) + { + CHECK_EQ( + "Type '(a, b, (a, b) -> (c...)) -> (c...)' could not be converted into '(a, a, (a, a) -> a) -> a'; different number of generic type " + "parameters", + toString(result.errors[0])); + } + else + { + CHECK_EQ("Type '(a, b, (a, b) -> (c...)) -> (c...)' could not be converted into '(a, a, (a, a) -> a) -> a'", toString(result.errors[0])); + } } TEST_CASE_FIXTURE(Fixture, "infer_return_value_type") @@ -4799,4 +4821,211 @@ local ModuleA = require(game.A) CHECK_EQ("*unknown*", toString(*oty)); } +/* + * If it wasn't instantly obvious, we have the fuzzer to thank for this gem of a test. + * + * We had an issue here where the scope for the `if` block here would + * have an elevated TypeLevel even though there is no function nesting going on. + * This would result in a free typevar for the type of _ that was much higher than + * it should be. This type would be erroneously quantified in the definition of `aaa`. + * This in turn caused an ice when evaluating `_()` in the while loop. + */ +TEST_CASE_FIXTURE(Fixture, "free_typevars_introduced_within_control_flow_constructs_do_not_get_an_elevated_TypeLevel") +{ + check(R"( + --!strict + if _ then + _[_], _ = nil + _() + end + + local aaa = function():typeof(_) return 1 end + + if aaa then + while _() do + end + end + )"); + + // No ice()? No problem. +} + +/* + * This is a bit elaborate. Bear with me. + * + * The type of _ becomes free with the first statement. With the second, we unify it with a function. + * + * At this point, it is important that the newly created fresh types of this new function type are promoted + * to the same level as the original free type. If we do not, they are incorrectly ascribed the level of the + * containing function. + * + * If this is allowed to happen, the final lambda erroneously quantifies the type of _ to something ridiculous + * just before we typecheck the invocation to _. + */ +TEST_CASE_FIXTURE(Fixture, "fuzzer_found_this") +{ + check(R"( + l0, _ = nil + + local function p() + _() + end + + a = _( + function():(typeof(p),typeof(_)) + end + )[nil] + )"); +} + +/* + * We had an issue where part of the type of pairs() was an unsealed table. + * This test depends on FFlagDebugLuauFreezeArena to trigger it. + */ +TEST_CASE_FIXTURE(Fixture, "pairs_parameters_are_not_unsealed_tables") +{ + check(R"( + function _(l0:{n0:any}) + _ = pairs + end + )"); +} + +TEST_CASE_FIXTURE(Fixture, "inferred_methods_of_free_tables_have_the_same_level_as_the_enclosing_table") +{ + check(R"( + function Base64FileReader(data) + local reader = {} + local index: number + + function reader:PeekByte() + return data:byte(index) + end + + function reader:Byte() + return data:byte(index - 1) + end + + return reader + end + + Base64FileReader() + + function ReadMidiEvents(data) + + local reader = Base64FileReader(data) + + while reader:HasMore() do + (reader:Byte() % 128) + end + end + )"); +} + +TEST_CASE_FIXTURE(Fixture, "error_detailed_function_mismatch_arg_count") +{ + ScopedFastFlag luauExtendedFunctionMismatchError{"LuauExtendedFunctionMismatchError", true}; + + CheckResult result = check(R"( +type A = (number, number) -> string +type B = (number) -> string + +local a: A +local b: B = a + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ(toString(result.errors[0]), R"(Type '(number, number) -> string' could not be converted into '(number) -> string' +caused by: + Argument count mismatch. Function expects 2 arguments, but only 1 is specified)"); +} + +TEST_CASE_FIXTURE(Fixture, "error_detailed_function_mismatch_arg") +{ + ScopedFastFlag luauExtendedFunctionMismatchError{"LuauExtendedFunctionMismatchError", true}; + + CheckResult result = check(R"( +type A = (number, number) -> string +type B = (number, string) -> string + +local a: A +local b: B = a + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ(toString(result.errors[0]), R"(Type '(number, number) -> string' could not be converted into '(number, string) -> string' +caused by: + Argument #2 type is not compatible. Type 'string' could not be converted into 'number')"); +} + +TEST_CASE_FIXTURE(Fixture, "error_detailed_function_mismatch_ret_count") +{ + ScopedFastFlag luauExtendedFunctionMismatchError{"LuauExtendedFunctionMismatchError", true}; + + CheckResult result = check(R"( +type A = (number, number) -> (number) +type B = (number, number) -> (number, boolean) + +local a: A +local b: B = a + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ(toString(result.errors[0]), R"(Type '(number, number) -> number' could not be converted into '(number, number) -> (number, boolean)' +caused by: + Function only returns 1 value. 2 are required here)"); +} + +TEST_CASE_FIXTURE(Fixture, "error_detailed_function_mismatch_ret") +{ + ScopedFastFlag luauExtendedFunctionMismatchError{"LuauExtendedFunctionMismatchError", true}; + + CheckResult result = check(R"( +type A = (number, number) -> string +type B = (number, number) -> number + +local a: A +local b: B = a + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ(toString(result.errors[0]), R"(Type '(number, number) -> string' could not be converted into '(number, number) -> number' +caused by: + Return type is not compatible. Type 'string' could not be converted into 'number')"); +} + +TEST_CASE_FIXTURE(Fixture, "error_detailed_function_mismatch_ret_mult") +{ + ScopedFastFlag luauExtendedFunctionMismatchError{"LuauExtendedFunctionMismatchError", true}; + + CheckResult result = check(R"( +type A = (number, number) -> (number, string) +type B = (number, number) -> (number, boolean) + +local a: A +local b: B = a + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ( + toString(result.errors[0]), R"(Type '(number, number) -> (number, string)' could not be converted into '(number, number) -> (number, boolean)' +caused by: + Return #2 type is not compatible. Type 'string' could not be converted into 'boolean')"); +} + +TEST_CASE_FIXTURE(Fixture, "table_function_check_use_after_free") +{ + ScopedFastFlag luauUnifyFunctionCheckResult{"LuauUpdateFunctionNameBinding", true}; + + CheckResult result = check(R"( +local t = {} + +function t.x(value) + for k,v in pairs(t) do end +end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.tryUnify.test.cpp b/tests/TypeInfer.tryUnify.test.cpp index 9f9a007f..f55b46a4 100644 --- a/tests/TypeInfer.tryUnify.test.cpp +++ b/tests/TypeInfer.tryUnify.test.cpp @@ -214,4 +214,32 @@ TEST_CASE_FIXTURE(TryUnifyFixture, "cli_41095_concat_log_in_sealed_table_unifica CHECK_EQ(toString(result.errors[1]), "Available overloads: ({a}, a) -> (); and ({a}, number, a) -> ()"); } +TEST_CASE_FIXTURE(TryUnifyFixture, "undo_new_prop_on_unsealed_table") +{ + ScopedFastFlag flags[] = { + {"LuauTableSubtypingVariance2", true}, + }; + // I am not sure how to make this happen in Luau code. + + TypeId unsealedTable = arena.addType(TableTypeVar{TableState::Unsealed, TypeLevel{}}); + TypeId sealedTable = arena.addType(TableTypeVar{ + {{"prop", Property{getSingletonTypes().numberType}}}, + std::nullopt, + TypeLevel{}, + TableState::Sealed + }); + + const TableTypeVar* ttv = get(unsealedTable); + REQUIRE(ttv); + + state.tryUnify(unsealedTable, sealedTable); + + // To be honest, it's really quite spooky here that we're amending an unsealed table in this case. + CHECK(!ttv->props.empty()); + + state.log.rollback(); + + CHECK(ttv->props.empty()); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.unionTypes.test.cpp b/tests/TypeInfer.unionTypes.test.cpp index 48496b89..b095a0db 100644 --- a/tests/TypeInfer.unionTypes.test.cpp +++ b/tests/TypeInfer.unionTypes.test.cpp @@ -462,4 +462,20 @@ local a: XYZ = { w = 4 } CHECK_EQ(toString(result.errors[0]), R"(Type 'a' could not be converted into 'X | Y | Z'; none of the union options are compatible)"); } +TEST_CASE_FIXTURE(Fixture, "error_detailed_optional") +{ + ScopedFastFlag luauExtendedUnionMismatchError{"LuauExtendedUnionMismatchError", true}; + + CheckResult result = check(R"( +type X = { x: number } + +local a: X? = { w = 4 } + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ(toString(result.errors[0]), R"(Type 'a' could not be converted into 'X?' +caused by: + None of the union options are compatible. For example: Table type 'a' not compatible with type 'X' because the former is missing field 'x')"); +} + TEST_SUITE_END(); diff --git a/tests/conformance/coverage.lua b/tests/conformance/coverage.lua new file mode 100644 index 00000000..f899603f --- /dev/null +++ b/tests/conformance/coverage.lua @@ -0,0 +1,64 @@ +-- This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +print("testing coverage") + +function foo() + local x = 1 + local y = 2 + assert(x + y) +end + +function bar() + local function one(x) + return x + end + + local two = function(x) + return x + end + + one(1) +end + +function validate(stats, hits, misses) + local checked = {} + + for _,l in ipairs(hits) do + if not (stats[l] and stats[l] > 0) then + return false, string.format("expected line %d to be hit", l) + end + checked[l] = true + end + + for _,l in ipairs(misses) do + if not (stats[l] and stats[l] == 0) then + return false, string.format("expected line %d to be missed", l) + end + checked[l] = true + end + + for k,v in pairs(stats) do + if type(k) == "number" and not checked[k] then + return false, string.format("expected line %d to be absent", k) + end + end + + return true +end + +foo() +c = getcoverage(foo) +assert(#c == 1) +assert(c[1].name == "foo") +assert(validate(c[1], {5, 6, 7}, {})) + +bar() +c = getcoverage(bar) +assert(#c == 3) +assert(c[1].name == "bar") +assert(validate(c[1], {11, 15, 19}, {})) +assert(c[2].name == "one") +assert(validate(c[2], {12}, {})) +assert(c[3].name == nil) +assert(validate(c[3], {}, {16})) + +return 'OK' From 019eeeb8532e43963ba24c03fd47e7708cd3c2cd Mon Sep 17 00:00:00 2001 From: Pelanyo Kamara Date: Wed, 15 Dec 2021 17:23:26 +0000 Subject: [PATCH 104/399] Add CodeMirror Editor for Web Demo (#228) Also remove timestamps from output --- docs/_includes/repl.html | 105 +++++++++++++++++++---- docs/assets/js/luau_mode.js | 167 ++++++++++++++++++++++++++++++++++++ 2 files changed, 253 insertions(+), 19 deletions(-) create mode 100644 docs/assets/js/luau_mode.js diff --git a/docs/_includes/repl.html b/docs/_includes/repl.html index 9d9b5895..2c6b8f7d 100644 --- a/docs/_includes/repl.html +++ b/docs/_includes/repl.html @@ -1,31 +1,99 @@
- +
- -

- - + +
+ + +
-

- + +
-

- +
+ +
+ + + + + + + + + + + - diff --git a/docs/assets/js/luau_mode.js b/docs/assets/js/luau_mode.js new file mode 100644 index 00000000..0a0b933a --- /dev/null +++ b/docs/assets/js/luau_mode.js @@ -0,0 +1,167 @@ +// CodeMirror, copyright (c) by Marijn Haverbeke and others +// Distributed under an MIT license: https://codemirror.net/LICENSE + +// Luau mode. Based on Lua mode from CodeMirror and Franciszek Wawrzak (https://codemirror.net/mode/lua/lua.js) + +(function(mod) { + if (typeof exports == "object" && typeof module == "object") // CommonJS + mod(require("../../lib/codemirror")); + else if (typeof define == "function" && define.amd) // AMD + define(["../../lib/codemirror"], mod); + else // Plain browser env + mod(CodeMirror); +})(function(CodeMirror) { + "use strict"; + + CodeMirror.defineMode("luau", function(_, parserConfig) { + var indentUnit = 4; + + function prefixRE(words) { + return new RegExp("^(?:" + words.join("|") + ")", "i"); + } + function wordRE(words) { + return new RegExp("^(?:" + words.join("|") + ")$", "i"); + } + var specials = wordRE(parserConfig.specials || ["type"]); + + // long list of standard functions from lua manual + var builtins = wordRE([ + "_G","_VERSION","assert","error","getfenv","getmetatable","ipairs","load", "loadstring","next","pairs","pcall", + "print","rawequal","rawget","rawset","require","select","setfenv","setmetatable","tonumber","tostring","type", + "unpack","xpcall", + + "coroutine.create","coroutine.resume","coroutine.running","coroutine.status","coroutine.wrap","coroutine.yield", + + "debug.debug","debug.getfenv","debug.gethook","debug.getinfo","debug.getlocal","debug.getmetatable", + "debug.getregistry","debug.getupvalue","debug.setfenv","debug.sethook","debug.setlocal","debug.setmetatable", + "debug.setupvalue","debug.traceback", + + "math.abs","math.acos","math.asin","math.atan","math.atan2","math.ceil","math.cos","math.cosh","math.deg", + "math.exp","math.floor","math.fmod","math.frexp","math.huge","math.ldexp","math.log","math.log10","math.max", + "math.min","math.modf","math.pi","math.pow","math.rad","math.random","math.randomseed","math.sin","math.sinh", + "math.sqrt","math.tan","math.tanh", + + "os.clock","os.date","os.difftime","os.execute","os.exit","os.getenv","os.remove","os.rename","os.setlocale", + "os.time","os.tmpname", + + "string.byte","string.char","string.dump","string.find","string.format","string.gmatch","string.gsub", + "string.len","string.lower","string.match","string.rep","string.reverse","string.sub","string.upper", + + "table.concat","table.insert","table.maxn","table.remove","table.sort" + ]); + var keywords = wordRE(["and","break","elseif","false","nil","not","or","return", + "true","function", "end", "if", "then", "else", "do", + "while", "repeat", "until", "for", "in", "local", "continue" ]); + + var indentTokens = wordRE(["function", "if","repeat","do", "\\(", "{"]); + var dedentTokens = wordRE(["end", "until", "\\)", "}"]); + var dedentPartial = prefixRE(["end", "until", "\\)", "}", "else", "elseif"]); + + function readBracket(stream) { + var level = 0; + while (stream.eat("=")) ++level; + stream.eat("["); + return level; + } + + function normal(stream, state) { + var ch = stream.next(); + if (ch == "-" && stream.eat("-")) { + if (stream.eat("[") && stream.eat("[")) + return (state.cur = bracketed(readBracket(stream), "comment"))(stream, state); + stream.skipToEnd(); + return "comment"; + } + if (ch == "\"" || ch == "'") + return (state.cur = string(ch))(stream, state); + if (ch == "[" && /[\[=]/.test(stream.peek())) + return (state.cur = bracketed(readBracket(stream), "string"))(stream, state); + if (/\d/.test(ch)) { + stream.eatWhile(/[\w.%]/); + return "number"; + } + if (/[\w_]/.test(ch)) { + stream.eatWhile(/[\w\\\-_.]/); + return "variable"; + } + return null; + } + + function bracketed(level, style) { + return function(stream, state) { + var curlev = null, ch; + while ((ch = stream.next()) != null) { + if (curlev == null) { + if (ch == "]") curlev = 0; + } else if (ch == "=") { + ++curlev; + } else if (ch == "]" && curlev == level) { + state.cur = normal; + break; + } else { + curlev = null; + } + } + return style; + }; + } + + function string(quote) { + return function(stream, state) { + var escaped = false, ch; + while ((ch = stream.next()) != null) { + if (ch == quote && !escaped) { + break; + } + escaped = !escaped && ch == "\\"; + } + + if (!escaped) { + state.cur = normal; + } + return "string"; + }; + } + + return { + startState: function(basecol) { + return {basecol: basecol || 0, indentDepth: 0, cur: normal}; + }, + + token: function(stream, state) { + if (stream.eatSpace()) { + return null; + } + var style = state.cur(stream, state); + var word = stream.current(); + if (style == "variable") { + if (keywords.test(word)) { + style = "keyword"; + } else if (builtins.test(word)) { + style = "builtin"; + } else if (specials.test(word)) { + style = "variable-2"; + } + } + if ((style != "comment") && (style != "string")) { + if (indentTokens.test(word)) { + ++state.indentDepth; + } else if (dedentTokens.test(word)) { + --state.indentDepth; + } + } + return style; + }, + + indent: function(state, textAfter) { + var closing = dedentPartial.test(textAfter); + return state.basecol + indentUnit * (state.indentDepth - (closing ? 1 : 0)); + }, + + electricInput: /^\s*(?:end|until|else|\)|\})$/, + lineComment: "--", + blockCommentStart: "--[[", + blockCommentEnd: "]]" + }}); + CodeMirror.defineMIME("text/x-luau", "luau"); +}); \ No newline at end of file From 9e7e779c028e9233226158d07dff098b25acc954 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Mon, 20 Dec 2021 15:36:41 -0800 Subject: [PATCH 105/399] Quality of life improvements to web demo (#297) - Upgrade CodeMirror to 5.65 - Enable matching paren highlighting via an addon - Remove extra buttons and replace clear output with a checkbox - Highlight error line on parsing/execution error - Change demo layout to wide to increase available width --- docs/_includes/repl.html | 76 +++++++++++++++------------------------- docs/_pages/demo.md | 1 + 2 files changed, 29 insertions(+), 48 deletions(-) diff --git a/docs/_includes/repl.html b/docs/_includes/repl.html index 2c6b8f7d..caeda822 100644 --- a/docs/_includes/repl.html +++ b/docs/_includes/repl.html @@ -4,66 +4,38 @@
- - + + Clear Output
-
-
- -
- - + + + @@ -88,9 +60,14 @@ } }); - // Misc Functions + var lastError = undefined; + function output(text) { - output_box = document.getElementById("output"); + var output_clear = document.getElementById("output-clear"); + var output_box = document.getElementById("output"); + if (output_clear.checked) { + output_box.value = ''; + } output_box.value += text.replace('stdin:', '') + "\n"; // scroll to bottom output_box.scrollTop = output_box.scrollHeight; @@ -100,18 +77,21 @@ 'print': function (msg) { output(msg) } }; - function clearInput() { - editor.setValue(""); - } - - function clearOutput() { - document.getElementById("output").value = ""; - } - function executeScript() { + if (lastError) { + editor.removeLineClass(lastError, "background", "line-error"); + lastError = undefined; + } + var err = Module.ccall('executeScript', 'string', ['string'], [editor.getValue()]); if (err) { - output('Error:' + err.replace('stdin:', '')); + var err_text = err.replace('stdin:', ''); + output('Error:' + err_text); + + var err_line = parseInt(err_text); + if (err_line) { + lastError = editor.addLineClass(err_line-1, "background", "line-error"); + } } } diff --git a/docs/_pages/demo.md b/docs/_pages/demo.md index fefa4b20..dbba8e6e 100644 --- a/docs/_pages/demo.md +++ b/docs/_pages/demo.md @@ -1,6 +1,7 @@ --- permalink: /demo title: Demo +classes: wide --- {% include repl.html %} From 5e7648947b702ba8971ba32ecec5f1682890eeb6 Mon Sep 17 00:00:00 2001 From: boyned//Kampfkarren Date: Mon, 20 Dec 2021 15:38:15 -0800 Subject: [PATCH 106/399] RFC: Safe navigation operator (#142) Co-authored-by: Alexander McCord <11488393+alexmccord@users.noreply.github.com> --- rfcs/syntax-safe-navigation-operator.md | 102 ++++++++++++++++++++++++ 1 file changed, 102 insertions(+) create mode 100644 rfcs/syntax-safe-navigation-operator.md diff --git a/rfcs/syntax-safe-navigation-operator.md b/rfcs/syntax-safe-navigation-operator.md new file mode 100644 index 00000000..c98f3957 --- /dev/null +++ b/rfcs/syntax-safe-navigation-operator.md @@ -0,0 +1,102 @@ +# Safe navigation postfix operator (?) + +## Summary + +Introduce syntax to navigate through `nil` values, or short-circuit with `nil` if it was encountered. + + +## Motivation + +nil values are very common in Lua, and take care to prevent runtime errors. + +Currently, attempting to index `dog.name` while caring for `dog` being nil requires some form of the following: + +```lua +local dogName = nil +if dog ~= nil then + dogName = dog.name +end +``` + +...or the unusual to read... + +```lua +local dogName = dog and dog.name +``` + +...which will return `false` if `dog` is `false`, instead of throwing an error because of the index of `false.name`. + +Luau provides the if...else expression making this turn into: + +```lua +local dogName = if dog == nil then nil else dog.name +``` + +...but this is fairly clunky for such a common expression. + +## Design + +The safe navigation operator will make all of these smooth, by supporting `x?.y` to safely index nil values. `dog?.name` would resolve to `nil` if `dog` was nil, or the name otherwise. + +The previous example turns into `local dogName = dog?.name` (or just using `dog?.name` elsewhere). + +Failing the nil-safety check early would make the entire expression nil, for instance `dog?.body.legs` would resolve to `nil` if `dog` is nil, rather than resolve `dog?.body` into nil, then turning into `nil.legs`. + +```lua +dog?.name --[[ is the same as ]] if dog == nil then nil else dog.name +``` + +The short-circuiting is limited within the expression. + +```lua +dog?.owner.name -- This will return nil if `dog` is nil +(dog?.owner).name -- `(dog?.owner)` resolves to nil, of which `name` is then indexed. This will error at runtime if `dog` is nil. + +dog?.legs + 3 -- `dog?.legs` is resolved on its own, meaning this will error at runtime if it is nil (`nil + 3`) +``` + +The operator must be used in the context of either a call or an index, and so: + +```lua +local value = x? +``` + +...would be invalid syntax. + +This syntax would be based on expressions, and not identifiers, meaning that `(x or y)?.call()` would be valid syntax. + +### Type +If the expression is typed as an optional, then the resulting type would be the final expression, also optional. Otherwise, it'll just be the resulting type if `?` wasn't used. + +```lua +local optionalObject: { name: string }? +local optionalObjectName = optionalObject?.name -- resolves to `string?` + +local nonOptionalObject: { name: string } +local nonOptionalObjectName = nonOptionalObject?.name -- resolves to `string` +``` + +### Calling + +This RFC only specifies `x?.y` as an index method. `x?:y()` is currently unspecified, and `x?.y(args)` as a syntax will be reserved (will error if you try to use it). + +While being able to support `dog?.getName()` is useful, it provides [some logistical issues for the language](https://github.com/Roblox/luau/pull/142#issuecomment-990563536). + +`x?.y(args)` will be reserved both so that this can potentially be resolved later down the line if something comes up, but also because it would be a guaranteed runtime error under this RFC: `dog?.getName()` will first index `dog?.getName`, which will return nil, then will attempt to call it. + +### Assignment +`x?.y = z` is not supported, and will be reported as a syntax error. + +## Drawbacks + +As with all syntax additions, this adds complexity to the parsing of expressions, and the execution of cancelling the rest of the expression could prove challenging. + +Furthermore, with the proposed syntax, it might lock off other uses of `?` within code (and not types) for the future as being ambiguous. + +## Alternatives + +Doing nothing is an option, as current standard if-checks already work, as well as the `and` trick in other use cases, but as shown before this can create some hard to read code, and nil values are common enough that the safe navigation operator is welcome. + +Supporting optional calls/indexes, such as `x?[1]` and `x?()`, while not out of scope, are likely too fringe to support, while adding on a significant amount of parsing difficulty, especially in the case of shorthand function calls, such as `x?{}` and `x?""`. + +It is possible to make `x?.y = z` resolve to only setting `x.y` if `x` is nil, but assignments silently failing can be seen as surprising. From fedd9a5f78c66232c5940644ec1e1865cd64b7cf Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Mon, 20 Dec 2021 15:40:38 -0800 Subject: [PATCH 107/399] docs: Fix label syntax --- docs/_includes/repl.html | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/docs/_includes/repl.html b/docs/_includes/repl.html index caeda822..5f609262 100644 --- a/docs/_includes/repl.html +++ b/docs/_includes/repl.html @@ -5,7 +5,8 @@
- Clear Output + +
From d079201a6e27c91e983eaa7d8ac630d4aed79554 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Mon, 20 Dec 2021 15:44:51 -0800 Subject: [PATCH 108/399] Fix repeated calls to print() clearing output --- docs/_includes/repl.html | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/docs/_includes/repl.html b/docs/_includes/repl.html index 5f609262..fa9993a5 100644 --- a/docs/_includes/repl.html +++ b/docs/_includes/repl.html @@ -64,11 +64,7 @@ var lastError = undefined; function output(text) { - var output_clear = document.getElementById("output-clear"); var output_box = document.getElementById("output"); - if (output_clear.checked) { - output_box.value = ''; - } output_box.value += text.replace('stdin:', '') + "\n"; // scroll to bottom output_box.scrollTop = output_box.scrollHeight; @@ -84,6 +80,12 @@ lastError = undefined; } + var output_clear = document.getElementById("output-clear"); + if (output_clear.checked) { + var output_box = document.getElementById("output"); + output_box.value = ''; + } + var err = Module.ccall('executeScript', 'string', ['string'], [editor.getValue()]); if (err) { var err_text = err.replace('stdin:', ''); From 65177c425c294dd18fb74136afb2622d02de8c7c Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Mon, 27 Dec 2021 12:48:58 -0800 Subject: [PATCH 109/399] Update grammar.md This changes the grammar to follow the EBNF rules more rigorously, most significantly quoting all keywords. --- docs/_pages/grammar.md | 108 ++++++++++++++++++++--------------------- 1 file changed, 54 insertions(+), 54 deletions(-) diff --git a/docs/_pages/grammar.md b/docs/_pages/grammar.md index 74a5c8aa..0ba49b33 100644 --- a/docs/_pages/grammar.md +++ b/docs/_pages/grammar.md @@ -10,73 +10,73 @@ is available in the [syntax section](syntax). > Note: this grammar is currently missing type pack syntax for generic arguments ```ebnf -chunk ::= {stat [`;']} [laststat [`;']] -block ::= chunk -stat ::= varlist `=' explist | +chunk = block +block = {stat [';']} [laststat [';']] +stat = varlist '=' explist | var compoundop exp | functioncall | - do block end | - while exp do block end | - repeat block until exp | - if exp then block {elseif exp then block} [else block] end | - for binding `=' exp `,' exp [`,' exp] do block end | - for bindinglist in explist do block end | - function funcname funcbody | - local function NAME funcbody | - local bindinglist [`=' explist] | - [export] type NAME [`<' GenericTypeList `>'] `=' Type + 'do' block 'end' | + 'while' exp 'do' block 'end' | + 'repeat' block 'until' exp | + 'if' exp 'then' block {'elseif' exp 'then' block} ['else' block] 'end' | + 'for' binding '=' exp ',' exp [',' exp] 'do' block 'end' | + 'for' bindinglist 'in' explist 'do' block 'end' | + 'function' funcname funcbody | + 'local' 'function' NAME funcbody | + 'local' bindinglist ['=' explist] | + ['export'] type NAME ['<' GenericTypeList '>'] '=' Type -laststat ::= return [explist] | break | continue +laststat = 'return' [explist] | 'break' | 'continue' -funcname ::= NAME {`.' NAME} [`:' NAME] -funcbody ::= `(' [parlist] `)' [`:' ReturnType] block end -parlist ::= bindinglist [`,' `...'] | `...' +funcname = NAME {'.' NAME} [':' NAME] +funcbody = '(' [parlist] ')' [':' ReturnType] block 'end' +parlist = bindinglist [',' '...'] | '...' -explist ::= {exp `,'} exp -namelist ::= NAME {`,' NAME} +explist = {exp ','} exp +namelist = NAME {',' NAME} -binding ::= NAME [`:' TypeAnnotation] -bindinglist ::= binding [`,' bindinglist] (* equivalent of Lua 5.1 `namelist`, except with optional type annotations *) +binding = NAME [':' TypeAnnotation] +bindinglist = binding [',' bindinglist] (* equivalent of Lua 5.1 'namelist', except with optional type annotations *) -var ::= NAME | prefixexp `[' exp `]' | prefixexp `.' Name -varlist ::= var {`,' var} -prefixexp ::= var | functioncall | `(' exp `)' -functioncall ::= prefixexp funcargs | prefixexp `:' NAME funcargs +var = NAME | prefixexp '[' exp ']' | prefixexp '.' Name +varlist = var {',' var} +prefixexp = var | functioncall | '(' exp ')' +functioncall = prefixexp funcargs | prefixexp ':' NAME funcargs -exp ::= (asexp | unop exp) { binop exp } -ifelseexp ::= if exp then exp {elseif exp then exp} else exp -asexp ::= simpleexp [`::' Type] -simpleexp ::= NUMBER | STRING | nil | true | false | `...' | tableconstructor | function body | prefixexp | ifelseexp -funcargs ::= `(' [explist] `)' | tableconstructor | STRING +exp = (asexp | unop exp) { binop exp } +ifelseexp = 'if' exp 'then' exp {'elseif' exp 'then' exp} 'else' exp +asexp = simpleexp ['::' Type] +simpleexp = NUMBER | STRING | 'nil' | 'true' | 'false' | '...' | tableconstructor | 'function' body | prefixexp | ifelseexp +funcargs = '(' [explist] ')' | tableconstructor | STRING -tableconstructor ::= `{' [fieldlist] `}' -fieldlist ::= field {fieldsep field} [fieldsep] -field ::= `[' exp `]' `=' exp | NAME `=' exp | exp -fieldsep ::= `,' | `;' +tableconstructor = '{' [fieldlist] '}' +fieldlist = field {fieldsep field} [fieldsep] +field = '[' exp ']' '=' exp | NAME '=' exp | exp +fieldsep = ',' | ';' -compoundop :: `+=' | `-=' | `*=' | `/=' | `%=' | `^=' | `..=' -binop ::= `+' | `-' | `*' | `/' | `^' | `%' | `..' | `<' | `<=' | `>' | `>=' | `==' | `~=' | and | or -unop ::= `-' | not | `#' +compoundop :: '+=' | '-=' | '*=' | '/=' | '%=' | '^=' | '..=' +binop = '+' | '-' | '*' | '/' | '^' | '%' | '..' | '<' | '<=' | '>' | '>=' | '==' | '~=' | 'and' | 'or' +unop = '-' | 'not' | '#' -SimpleType ::= - nil | - NAME[`.' NAME] [ `<' TypeList `>' ] | - `typeof' `(' exp `)' | +SimpleType = + 'nil' | + NAME ['.' NAME] [ '<' TypeList '>' ] | + 'typeof' '(' exp ')' | TableType | FunctionType -Type ::= - SimpleType [`?`] | - SimpleType [`|` Type] | - SimpleType [`&` Type] +Type = + SimpleType ['?'] | + SimpleType ['|' Type] | + SimpleType ['&' Type] -GenericTypeList ::= NAME [`...'] {`,' NAME [`...']} -TypeList ::= Type [`,' TypeList] | ...Type -ReturnType ::= Type | `(' TypeList `)' -TableIndexer ::= `[' Type `]' `:' Type -TableProp ::= NAME `:' Type -TablePropOrIndexer ::= TableProp | TableIndexer -PropList ::= TablePropOrIndexer {fieldsep TablePropOrIndexer} [fieldsep] -TableType ::= `{' PropList `}' -FunctionType ::= [`<' GenericTypeList `>'] `(' [TypeList] `)' `->` ReturnType +GenericTypeList = NAME ['...'] {',' NAME ['...']} +TypeList = Type [',' TypeList] | '...' Type +ReturnType = Type | '(' TypeList ')' +TableIndexer = '[' Type ']' ':' Type +TableProp = NAME ':' Type +TablePropOrIndexer = TableProp | TableIndexer +PropList = TablePropOrIndexer {fieldsep TablePropOrIndexer} [fieldsep] +TableType = '{' PropList '}' +FunctionType = ['<' GenericTypeList '>'] '(' [TypeList] ')' '->' ReturnType ``` From 6203bf6ac55d1ced482304c36648929ef387a100 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Mon, 27 Dec 2021 12:51:23 -0800 Subject: [PATCH 110/399] Update grammar.md Remove TOC & mark page as wide. --- docs/_pages/grammar.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/_pages/grammar.md b/docs/_pages/grammar.md index 0ba49b33..d1767934 100644 --- a/docs/_pages/grammar.md +++ b/docs/_pages/grammar.md @@ -1,7 +1,7 @@ --- permalink: /grammar title: Grammar -toc: true +classes: wide --- This is the complete syntax grammar for Luau in EBNF. More information about the terminal nodes String and Number From fa35884e5b95ddcd8ec8401fb9151ba291fcbfbf Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Mon, 27 Dec 2021 13:08:56 -0800 Subject: [PATCH 111/399] Update library.md A few small tweaks and fixes. --- docs/_pages/library.md | 17 +++++++++-------- 1 file changed, 9 insertions(+), 8 deletions(-) diff --git a/docs/_pages/library.md b/docs/_pages/library.md index 324670f6..834bd73b 100644 --- a/docs/_pages/library.md +++ b/docs/_pages/library.md @@ -19,10 +19,10 @@ function assert(value: T, message: string?): T ``` `assert` checks if the value is truthy; if it's not (which means it's `false` or `nil`), it raises an error. The error message can be customized with an optional parameter. -Upon success the function returns the `condition` argument. +Upon success the function returns the `value` argument. ``` -function error(object: any, level: number?) +function error(obj: any, level: number?) ``` `error` raises an error with the specified object. Note that errors don't have to be strings, although they often are by convention; various error handling mechanisms like `pcall` @@ -110,7 +110,7 @@ Changes metatable for the given table. Note that unlike `getmetatable`, this fun function tonumber(s: string, base: number?): number? ``` -Converts the input string to the number in base `base` (default 10) and returns the resulting number. If the conversion fails, returns `nil` instead. +Converts the input string to the number in base `base` (default 10) and returns the resulting number. If the conversion fails (that is, if the input string doesn't represent a valid number in the specified base), returns `nil` instead. ``` function tostring(obj: any): string @@ -161,7 +161,7 @@ Note that `f` can yield, which results in the entire coroutine yielding as well. function unpack(a: {V}, f: number?, t: number?): ...V ``` -Returns all values of `a` with indices in `[f..t]` range. `f` defaults to 1 and `t` defaults to `#a`. +Returns all values of `a` with indices in `[f..t]` range. `f` defaults to 1 and `t` defaults to `#a`. Note that this is equivalent to `table.unpack`. ## math library @@ -346,13 +346,14 @@ Returns 3D Perlin noise value for the point `(x, y, z)` (`y` and `z` default to function math.clamp(n: number, min: number, max: number): number ``` -Returns `n` if the number is in `[min, max]` range; otherwise, returns `min` when `n < min`, and `max` otherwise. The function errors if `min > max`. +Returns `n` if the number is in `[min, max]` range; otherwise, returns `min` when `n < min`, and `max` otherwise. If `n` is NaN, may or may not return NaN. +The function errors if `min > max`. ``` function math.sign(n: number): number ``` -Returns `-1` if `n` is negative, `1` if `n` is positive, and `0` if `n` is zero. +Returns `-1` if `n` is negative, `1` if `n` is positive, and `0` if `n` is zero or NaN. ``` function math.round(n: number): number @@ -384,7 +385,7 @@ Iterates over numeric keys of the table in `[1..#t]` range in order; for each ke function table.getn(t: {V}): number ``` -Returns the length of table `t` (aka `#t`). +Returns the length of table `t` (equivalent to `#t`). ``` function table.maxn(t: {V}): number @@ -518,7 +519,7 @@ When `f` is a string, the substitution uses the string as a replacement. When `f function string.len(s: string): number ``` -Returns the number of bytes in the string. +Returns the number of bytes in the string (equivalent to `#s`). ``` function string.lower(s: string): string From 73b7bcb2dab8102e0bea72cd46ee953cb6697f78 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Tue, 4 Jan 2022 15:04:25 -0800 Subject: [PATCH 112/399] docs: Move Luau.Web.js fetch to the end of the page This was moved in the CodeMirror change, but that may break Module setup in the embedded @@ -98,3 +96,5 @@ } } + + From 82587bef294a9b146da54aa8bff76c0cc1d823a8 Mon Sep 17 00:00:00 2001 From: Alan Jeffrey <403333+asajeffrey@users.noreply.github.com> Date: Thu, 6 Jan 2022 12:48:09 -0600 Subject: [PATCH 113/399] RFC: Fix an unsoundness issue around stripping optional properties (#276) * Fix an unsoundness issue around stripping optional properties Co-authored-by: vegorov-rbx <75688451+vegorov-rbx@users.noreply.github.com> --- rfcs/unsealed-table-literals.md | 76 +++++++++++++++++++ ...le-subtyping-strips-optional-properties.md | 66 ++++++++++++++++ 2 files changed, 142 insertions(+) create mode 100644 rfcs/unsealed-table-literals.md create mode 100644 rfcs/unsealed-table-subtyping-strips-optional-properties.md diff --git a/rfcs/unsealed-table-literals.md b/rfcs/unsealed-table-literals.md new file mode 100644 index 00000000..320bf7ca --- /dev/null +++ b/rfcs/unsealed-table-literals.md @@ -0,0 +1,76 @@ +# Unsealed table literals + +## Summary + +Currently the only way to create an unsealed table is as an empty table literal `{}`. +This RFC proposes making all table literals unsealed. + +## Motivation + +Table types can be *sealed* or *unsealed*. These are different in that: + +* Unsealed table types are *precise*: if a table has unsealed type `{ p: number, q: string }` + then it is guaranteed to have only properties `p` and `q`. + +* Sealed tables support *width subtyping*: if a table has sealed type `{ p: number }` + then it is guaranteed to have at least property `p`, so we allow `{ p: number, q: string }` + to be treated as a subtype of `{ p: number }` + +* Unsealed tables can have properties added to them: if `t` has unsealed type + `{ p: number }` then after the assignment `t.q = "hi"`, `t`'s type is updated to be + `{ p: number, q: string }`. + +* Unsealed tables are subtypes of sealed tables. + +Currently the only way to create an unsealed table is using an empty table literal, so +```lua + local t = {} + t.p = 5 + t.q = "hi" +``` +typechecks, but +```lua + local t = { p = 5 } + t.q = "hi" +``` +does not. + +This causes problems in examples, in particular developers +may initialize properties but not methods: +```lua + local t = { p = 5 } + function t.f() return t.p end +``` + +## Design + +The proposed change is straightforward: make all table literals unsealed. + +## Drawbacks + +Making all table literals unsealed is a conservative change, it only removes type errors. + +It does encourage developers to add new properties to tables during initialization, which +may be considered poor style. + +It does mean that some spelling mistakes will not be caught, for example +```lua +local t = {x = 1, y = 2} +if foo then + t.z = 3 -- is z a typo or intentional 2-vs-3 choice? +end +``` + +In particular, we no longer warn about adding properties to array-like tables. +```lua +local a = {1,2,3} +a.p = 5 +``` + +## Alternatives + +We could introduce a new table state for unsealed-but-precise +tables. The trade-off is that that would be more precise, at the cost +of adding user-visible complexity to the type system. + +We could continue to treat array-like tables as sealed. \ No newline at end of file diff --git a/rfcs/unsealed-table-subtyping-strips-optional-properties.md b/rfcs/unsealed-table-subtyping-strips-optional-properties.md new file mode 100644 index 00000000..deecfdb3 --- /dev/null +++ b/rfcs/unsealed-table-subtyping-strips-optional-properties.md @@ -0,0 +1,66 @@ +# Only strip optional properties from unsealed tables during subtyping + +## Summary + +Currently subtyping allows optional properties to be stripped from table types during subtyping. +This RFC proposes only allowing that when the subtype is unsealed and the supertype is sealed. + +## Motivation + +Table types can be *sealed* or *unsealed*. These are different in that: + +* Unsealed table types are *precise*: if a table has unsealed type `{ p: number, q: string }` + then it is guaranteed to have only properties `p` and `q`. + +* Sealed tables support *width subtyping*: if a table has sealed type `{ p: number }` + then it is guaranteed to have at least property `p`, so we allow `{ p: number, q: string }` + to be treated as a subtype of `{ p: number }` + +* Unsealed tables can have properties added to them: if `t` has unsealed type + `{ p: number }` then after the assignment `t.q = "hi"`, `t`'s type is updated to be + `{ p: number, q: string }`. + +* Unsealed tables are subtypes of sealed tables. + +Currently we allow subtyping to strip away optional fields +as long as the supertype is sealed. +This is necessary for examples, for instance: +```lua + local t : { p: number, q: string? } = { p = 5, q = "hi" } + t = { p = 7 } +``` +typechecks because `{ p : number }` is a subtype of +`{ p : number, q : string? }`. Unfortunately this is not sound, +since sealed tables support width subtyping: +```lua + local t : { p: number, q: string? } = { p = 5, q = "hi" } + local u : { p: number } = { p = 5, q = false } + t = u +``` + +## Design + +The fix for this source of unsoundness is twofold: + +1. make all table literals unsealed, and +2. only allow stripping optional properties from when the + supertype is sealed and the subtype is unsealed. + +This RFC is for (2). There is a [separate RFC](unsealed-table-literals.md) for (1). + +## Drawbacks + +This introduces new type errors (it has to, since it is fixing a source of +unsoundness). This means that there are now false positives such as: +```lua + local t : { p: number, q: string? } = { p = 5, q = "hi" } + local u : { p: number } = { p = 5, q = "lo" } + t = u +``` +These false positives are so similar to sources of unsoundness +that it is difficult to see how to allow them soundly. + +## Alternatives + +We could just live with unsoundness. + From a9aa4faf24e6cea1ac0e33d0054a7328a35f9d4a Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Thu, 6 Jan 2022 14:08:56 -0800 Subject: [PATCH 114/399] Sync to upstream/release/508 This version isn't for release because we've skipped some internal numbers due to year-end schedule changes, but it's better to merge separately. --- Analysis/include/Luau/LValue.h | 63 +++++++ Analysis/include/Luau/Predicate.h | 34 +--- Analysis/include/Luau/TypeInfer.h | 1 + Analysis/src/{Predicate.cpp => LValue.cpp} | 88 ++++++++- Analysis/src/TypeInfer.cpp | 83 ++++++++- Analysis/src/TypeVar.cpp | 11 +- Analysis/src/Unifier.cpp | 123 ++++--------- Ast/src/Parser.cpp | 4 - Sources.cmake | 5 +- VM/src/lbaselib.cpp | 8 +- tests/Autocomplete.test.cpp | 6 +- tests/LValue.test.cpp | 198 +++++++++++++++++++++ tests/Predicate.test.cpp | 117 ------------ tests/Symbol.test.cpp | 33 +++- tests/Transpiler.test.cpp | 2 - tests/TypeInfer.builtins.test.cpp | 12 +- tests/TypeInfer.classes.test.cpp | 2 - tests/TypeInfer.intersectionTypes.test.cpp | 4 - tests/TypeInfer.refinements.test.cpp | 37 +++- tests/TypeInfer.singletons.test.cpp | 1 - tests/TypeInfer.tables.test.cpp | 4 - tests/TypeInfer.test.cpp | 13 ++ tests/TypeInfer.unionTypes.test.cpp | 4 - tests/conformance/math.lua | 1 + 24 files changed, 570 insertions(+), 284 deletions(-) create mode 100644 Analysis/include/Luau/LValue.h rename Analysis/src/{Predicate.cpp => LValue.cpp} (50%) create mode 100644 tests/LValue.test.cpp delete mode 100644 tests/Predicate.test.cpp diff --git a/Analysis/include/Luau/LValue.h b/Analysis/include/Luau/LValue.h new file mode 100644 index 00000000..8fd96f05 --- /dev/null +++ b/Analysis/include/Luau/LValue.h @@ -0,0 +1,63 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +#pragma once + +#include "Luau/Variant.h" +#include "Luau/Symbol.h" + +#include // TODO: Kill with LuauLValueAsKey. +#include +#include + +namespace Luau +{ + +struct TypeVar; +using TypeId = const TypeVar*; + +struct Field; +using LValue = Variant; + +struct Field +{ + std::shared_ptr parent; + std::string key; + + bool operator==(const Field& rhs) const; + bool operator!=(const Field& rhs) const; +}; + +struct LValueHasher +{ + size_t operator()(const LValue& lvalue) const; +}; + +const LValue* baseof(const LValue& lvalue); + +std::optional tryGetLValue(const class AstExpr& expr); + +// Utility function: breaks down an LValue to get at the Symbol, and reverses the vector of keys. +std::pair> getFullName(const LValue& lvalue); + +// Kill with LuauLValueAsKey. +std::string toString(const LValue& lvalue); + +template +const T* get(const LValue& lvalue) +{ + return get_if(&lvalue); +} + +using NEW_RefinementMap = std::unordered_map; +using DEPRECATED_RefinementMap = std::map; + +// Transient. Kill with LuauLValueAsKey. +struct RefinementMap +{ + NEW_RefinementMap NEW_refinements; + DEPRECATED_RefinementMap DEPRECATED_refinements; +}; + +void merge(RefinementMap& l, const RefinementMap& r, std::function f); +void addRefinement(RefinementMap& refis, const LValue& lvalue, TypeId ty); + +} // namespace Luau diff --git a/Analysis/include/Luau/Predicate.h b/Analysis/include/Luau/Predicate.h index a5e8b6ae..df93b4f4 100644 --- a/Analysis/include/Luau/Predicate.h +++ b/Analysis/include/Luau/Predicate.h @@ -1,12 +1,10 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #pragma once -#include "Luau/Variant.h" #include "Luau/Location.h" -#include "Luau/Symbol.h" +#include "Luau/LValue.h" +#include "Luau/Variant.h" -#include -#include #include namespace Luau @@ -15,34 +13,6 @@ namespace Luau struct TypeVar; using TypeId = const TypeVar*; -struct Field; -using LValue = Variant; - -struct Field -{ - std::shared_ptr parent; // TODO: Eventually use unique_ptr to enforce non-copyable trait. - std::string key; -}; - -std::optional tryGetLValue(const class AstExpr& expr); - -// Utility function: breaks down an LValue to get at the Symbol, and reverses the vector of keys. -std::pair> getFullName(const LValue& lvalue); - -std::string toString(const LValue& lvalue); - -template -const T* get(const LValue& lvalue) -{ - return get_if(&lvalue); -} - -// Key is a stringified encoding of an LValue. -using RefinementMap = std::map; - -void merge(RefinementMap& l, const RefinementMap& r, std::function f); -void addRefinement(RefinementMap& refis, const LValue& lvalue, TypeId ty); - struct TruthyPredicate; struct IsAPredicate; struct TypeGuardPredicate; diff --git a/Analysis/include/Luau/TypeInfer.h b/Analysis/include/Luau/TypeInfer.h index 451976e4..862f50d7 100644 --- a/Analysis/include/Luau/TypeInfer.h +++ b/Analysis/include/Luau/TypeInfer.h @@ -350,6 +350,7 @@ public: private: std::optional resolveLValue(const ScopePtr& scope, const LValue& lvalue); + std::optional DEPRECATED_resolveLValue(const ScopePtr& scope, const LValue& lvalue); std::optional resolveLValue(const RefinementMap& refis, const ScopePtr& scope, const LValue& lvalue); void resolve(const PredicateVec& predicates, ErrorVec& errVec, RefinementMap& refis, const ScopePtr& scope, bool sense, bool fromOr = false); diff --git a/Analysis/src/Predicate.cpp b/Analysis/src/LValue.cpp similarity index 50% rename from Analysis/src/Predicate.cpp rename to Analysis/src/LValue.cpp index 7bd8001e..da6804c6 100644 --- a/Analysis/src/Predicate.cpp +++ b/Analysis/src/LValue.cpp @@ -1,11 +1,59 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details -#include "Luau/Predicate.h" +#include "Luau/LValue.h" #include "Luau/Ast.h" +#include + +LUAU_FASTFLAG(LuauLValueAsKey) + namespace Luau { +bool Field::operator==(const Field& rhs) const +{ + LUAU_ASSERT(parent && rhs.parent); + return key == rhs.key && (parent == rhs.parent || *parent == *rhs.parent); +} + +bool Field::operator!=(const Field& rhs) const +{ + return !(*this == rhs); +} + +size_t LValueHasher::operator()(const LValue& lvalue) const +{ + // Most likely doesn't produce high quality hashes, but we're probably ok enough with it. + // When an evidence is shown that operator==(LValue) is used more often than it should, we can have a look at improving the hash quality. + size_t acc = 0; + size_t offset = 0; + + const LValue* current = &lvalue; + while (current) + { + if (auto field = get(*current)) + acc ^= (std::hash{}(field->key) << 1) >> ++offset; + else if (auto symbol = get(*current)) + acc ^= std::hash{}(*symbol) << 1; + else + LUAU_ASSERT(!"Hash not accumulated for this new LValue alternative."); + + current = baseof(*current); + } + + return acc; +} + +const LValue* baseof(const LValue& lvalue) +{ + if (auto field = get(lvalue)) + return field->parent.get(); + + auto symbol = get(lvalue); + LUAU_ASSERT(symbol); + return nullptr; // Base of root is null. +} + std::optional tryGetLValue(const AstExpr& node) { const AstExpr* expr = &node; @@ -38,15 +86,15 @@ std::pair> getFullName(const LValue& lvalue) while (auto field = get(*current)) { keys.push_back(field->key); - current = field->parent.get(); - if (!current) - LUAU_ASSERT(!"LValue root is a Field?"); + current = baseof(*current); } const Symbol* symbol = get(*current); + LUAU_ASSERT(symbol); return {*symbol, std::vector(keys.rbegin(), keys.rend())}; } +// Kill with LuauLValueAsKey. std::string toString(const LValue& lvalue) { auto [symbol, keys] = getFullName(lvalue); @@ -56,7 +104,18 @@ std::string toString(const LValue& lvalue) return s; } -void merge(RefinementMap& l, const RefinementMap& r, std::function f) +static void merge(NEW_RefinementMap& l, const NEW_RefinementMap& r, std::function f) +{ + for (const auto& [k, a] : r) + { + if (auto it = l.find(k); it != l.end()) + l[k] = f(it->second, a); + else + l[k] = a; + } +} + +static void merge(DEPRECATED_RefinementMap& l, const DEPRECATED_RefinementMap& r, std::function f) { auto itL = l.begin(); auto itR = r.begin(); @@ -69,21 +128,32 @@ void merge(RefinementMap& l, const RefinementMap& r, std::functionfirst > k) + else if (itL->first < k) + ++itL; + else { l[k] = a; ++itR; } - else - ++itL; } l.insert(itR, r.end()); } +void merge(RefinementMap& l, const RefinementMap& r, std::function f) +{ + if (FFlag::LuauLValueAsKey) + return merge(l.NEW_refinements, r.NEW_refinements, f); + else + return merge(l.DEPRECATED_refinements, r.DEPRECATED_refinements, f); +} + void addRefinement(RefinementMap& refis, const LValue& lvalue, TypeId ty) { - refis[toString(lvalue)] = ty; + if (FFlag::LuauLValueAsKey) + refis.NEW_refinements[lvalue] = ty; + else + refis.DEPRECATED_refinements[toString(lvalue)] = ty; } } // namespace Luau diff --git a/Analysis/src/TypeInfer.cpp b/Analysis/src/TypeInfer.cpp index abbc2901..e29b6ec6 100644 --- a/Analysis/src/TypeInfer.cpp +++ b/Analysis/src/TypeInfer.cpp @@ -36,6 +36,7 @@ LUAU_FASTFLAGVARIABLE(LuauErrorRecoveryType, false) LUAU_FASTFLAGVARIABLE(LuauPropertiesGetExpectedType, false) LUAU_FASTFLAGVARIABLE(LuauTailArgumentTypeInfo, false) LUAU_FASTFLAGVARIABLE(LuauModuleRequireErrorPack, false) +LUAU_FASTFLAGVARIABLE(LuauLValueAsKey, false) LUAU_FASTFLAGVARIABLE(LuauRefiLookupFromIndexExpr, false) LUAU_FASTFLAGVARIABLE(LuauProperTypeLevels, false) LUAU_FASTFLAGVARIABLE(LuauAscribeCorrectLevelToInferredProperitesOfFreeTables, false) @@ -1626,6 +1627,10 @@ std::optional TypeChecker::getIndexTypeFromType( { RecursionLimiter _rl(&recursionCount, FInt::LuauTypeInferRecursionLimit); + // Not needed when we normalize types. + if (FFlag::LuauLValueAsKey && get(follow(t))) + return t; + if (std::optional ty = getIndexTypeFromType(scope, t, name, location, false)) goodOptions.push_back(*ty); else @@ -4967,13 +4972,83 @@ std::pair, std::vector> TypeChecker::createGener std::optional TypeChecker::resolveLValue(const ScopePtr& scope, const LValue& lvalue) { - std::string path = toString(lvalue); + if (!FFlag::LuauLValueAsKey) + return DEPRECATED_resolveLValue(scope, lvalue); + + // We want to be walking the Scope parents. + // We'll also want to walk up the LValue path. As we do this, we need to save each LValue because we must walk back. + // For example: + // There exists an entry t.x. + // We are asked to look for t.x.y. + // We need to search in the provided Scope. Find t.x.y first. + // We fail to find t.x.y. Try t.x. We found it. Now we must return the type of the property y from the mapped-to type of t.x. + // If we completely fail to find the Symbol t but the Scope has that entry, then we should walk that all the way through and terminate. + const auto& [symbol, keys] = getFullName(lvalue); + + ScopePtr currentScope = scope; + while (currentScope) + { + std::optional found; + + std::vector childKeys; + const LValue* currentLValue = &lvalue; + while (currentLValue) + { + if (auto it = currentScope->refinements.NEW_refinements.find(*currentLValue); it != currentScope->refinements.NEW_refinements.end()) + { + found = it->second; + break; + } + + childKeys.push_back(*currentLValue); + currentLValue = baseof(*currentLValue); + } + + if (!found) + { + // Should not be using scope->lookup. This is already recursive. + if (auto it = currentScope->bindings.find(symbol); it != currentScope->bindings.end()) + found = it->second.typeId; + else + { + // Nothing exists in this Scope. Just skip and try the parent one. + currentScope = currentScope->parent; + continue; + } + } + + for (auto it = childKeys.rbegin(); it != childKeys.rend(); ++it) + { + const LValue& key = *it; + + // Symbol can happen. Skip. + if (get(key)) + continue; + else if (auto field = get(key)) + { + found = getIndexTypeFromType(scope, *found, field->key, Location(), false); + if (!found) + return std::nullopt; // Turns out this type doesn't have the property at all. We're done. + } + else + LUAU_ASSERT(!"New LValue alternative not handled here."); + } + + return found; + } + + // No entry for it at all. Can happen when LValue root is a global. + return std::nullopt; +} + +std::optional TypeChecker::DEPRECATED_resolveLValue(const ScopePtr& scope, const LValue& lvalue) +{ auto [symbol, keys] = getFullName(lvalue); ScopePtr currentScope = scope; while (currentScope) { - if (auto it = currentScope->refinements.find(path); it != currentScope->refinements.end()) + if (auto it = currentScope->refinements.DEPRECATED_refinements.find(toString(lvalue)); it != currentScope->refinements.DEPRECATED_refinements.end()) return it->second; // Should not be using scope->lookup. This is already recursive. @@ -5000,7 +5075,9 @@ std::optional TypeChecker::resolveLValue(const ScopePtr& scope, const LV std::optional TypeChecker::resolveLValue(const RefinementMap& refis, const ScopePtr& scope, const LValue& lvalue) { - if (auto it = refis.find(toString(lvalue)); it != refis.end()) + if (auto it = refis.DEPRECATED_refinements.find(toString(lvalue)); it != refis.DEPRECATED_refinements.end()) + return it->second; + else if (auto it = refis.NEW_refinements.find(lvalue); it != refis.NEW_refinements.end()) return it->second; else return resolveLValue(scope, lvalue); diff --git a/Analysis/src/TypeVar.cpp b/Analysis/src/TypeVar.cpp index 571b13ca..fb75aa02 100644 --- a/Analysis/src/TypeVar.cpp +++ b/Analysis/src/TypeVar.cpp @@ -996,18 +996,19 @@ std::optional> magicFunctionFormat( std::vector expected = parseFormatString(typechecker, fmt->value.data, fmt->value.size); const auto& [params, tail] = flatten(paramPack); - const size_t dataOffset = 1; + size_t paramOffset = 1; + size_t dataOffset = expr.self ? 0 : 1; // unify the prefix one argument at a time - for (size_t i = 0; i < expected.size() && i + dataOffset < params.size(); ++i) + for (size_t i = 0; i < expected.size() && i + paramOffset < params.size(); ++i) { - Location location = expr.args.data[std::min(i, expr.args.size - 1)]->location; + Location location = expr.args.data[std::min(i + dataOffset, expr.args.size - 1)]->location; - typechecker.unify(expected[i], params[i + dataOffset], location); + typechecker.unify(expected[i], params[i + paramOffset], location); } // if we know the argument count or if we have too many arguments for sure, we can issue an error - const size_t actualParamSize = params.size() - dataOffset; + size_t actualParamSize = params.size() - paramOffset; if (expected.size() != actualParamSize && (!tail || expected.size() < actualParamSize)) typechecker.reportError(TypeError{expr.location, CountMismatch{expected.size(), actualParamSize}}); diff --git a/Analysis/src/Unifier.cpp b/Analysis/src/Unifier.cpp index c5aab856..43ea37e7 100644 --- a/Analysis/src/Unifier.cpp +++ b/Analysis/src/Unifier.cpp @@ -18,9 +18,7 @@ LUAU_FASTFLAGVARIABLE(LuauTableSubtypingVariance2, false); LUAU_FASTFLAGVARIABLE(LuauUnionHeuristic, false) LUAU_FASTFLAGVARIABLE(LuauTableUnificationEarlyTest, false) LUAU_FASTFLAGVARIABLE(LuauOccursCheckOkWithRecursiveFunctions, false) -LUAU_FASTFLAGVARIABLE(LuauExtendedTypeMismatchError, false) LUAU_FASTFLAG(LuauSingletonTypes) -LUAU_FASTFLAGVARIABLE(LuauExtendedClassMismatchError, false) LUAU_FASTFLAG(LuauErrorRecoveryType); LUAU_FASTFLAG(LuauProperTypeLevels); LUAU_FASTFLAGVARIABLE(LuauExtendedUnionMismatchError, false) @@ -416,7 +414,7 @@ void Unifier::tryUnify_(TypeId superTy, TypeId subTy, bool isFunctionCall, bool else if (!innerState.errors.empty()) { // 'nil' option is skipped from extended report because we present the type in a special way - 'T?' - if (FFlag::LuauExtendedTypeMismatchError && !firstFailedOption && !isNil(type)) + if (!firstFailedOption && !isNil(type)) firstFailedOption = {innerState.errors.front()}; failed = true; @@ -434,7 +432,7 @@ void Unifier::tryUnify_(TypeId superTy, TypeId subTy, bool isFunctionCall, bool errors.push_back(*unificationTooComplex); else if (failed) { - if (FFlag::LuauExtendedTypeMismatchError && firstFailedOption) + if (firstFailedOption) errors.push_back(TypeError{location, TypeMismatch{superTy, subTy, "Not all union options are compatible.", *firstFailedOption}}); else errors.push_back(TypeError{location, TypeMismatch{superTy, subTy}}); @@ -536,49 +534,36 @@ void Unifier::tryUnify_(TypeId superTy, TypeId subTy, bool isFunctionCall, bool if (FFlag::LuauExtendedUnionMismatchError && (failedOptionCount == 1 || foundHeuristic) && failedOption) errors.push_back( TypeError{location, TypeMismatch{superTy, subTy, "None of the union options are compatible. For example:", *failedOption}}); - else if (FFlag::LuauExtendedTypeMismatchError) - errors.push_back(TypeError{location, TypeMismatch{superTy, subTy, "none of the union options are compatible"}}); else - errors.push_back(TypeError{location, TypeMismatch{superTy, subTy}}); + errors.push_back(TypeError{location, TypeMismatch{superTy, subTy, "none of the union options are compatible"}}); } } else if (const IntersectionTypeVar* uv = get(superTy)) { - if (FFlag::LuauExtendedTypeMismatchError) + std::optional unificationTooComplex; + std::optional firstFailedOption; + + // T <: A & B if A <: T and B <: T + for (TypeId type : uv->parts) { - std::optional unificationTooComplex; - std::optional firstFailedOption; + Unifier innerState = makeChildUnifier(); + innerState.tryUnify_(type, subTy, /*isFunctionCall*/ false, /*isIntersection*/ true); - // T <: A & B if A <: T and B <: T - for (TypeId type : uv->parts) + if (auto e = hasUnificationTooComplex(innerState.errors)) + unificationTooComplex = e; + else if (!innerState.errors.empty()) { - Unifier innerState = makeChildUnifier(); - innerState.tryUnify_(type, subTy, /*isFunctionCall*/ false, /*isIntersection*/ true); - - if (auto e = hasUnificationTooComplex(innerState.errors)) - unificationTooComplex = e; - else if (!innerState.errors.empty()) - { - if (!firstFailedOption) - firstFailedOption = {innerState.errors.front()}; - } - - log.concat(std::move(innerState.log)); + if (!firstFailedOption) + firstFailedOption = {innerState.errors.front()}; } - if (unificationTooComplex) - errors.push_back(*unificationTooComplex); - else if (firstFailedOption) - errors.push_back(TypeError{location, TypeMismatch{superTy, subTy, "Not all intersection parts are compatible.", *firstFailedOption}}); - } - else - { - // T <: A & B if A <: T and B <: T - for (TypeId type : uv->parts) - { - tryUnify_(type, subTy, /*isFunctionCall*/ false, /*isIntersection*/ true); - } + log.concat(std::move(innerState.log)); } + + if (unificationTooComplex) + errors.push_back(*unificationTooComplex); + else if (firstFailedOption) + errors.push_back(TypeError{location, TypeMismatch{superTy, subTy, "Not all intersection parts are compatible.", *firstFailedOption}}); } else if (const IntersectionTypeVar* uv = get(subTy)) { @@ -626,10 +611,7 @@ void Unifier::tryUnify_(TypeId superTy, TypeId subTy, bool isFunctionCall, bool errors.push_back(*unificationTooComplex); else if (!found) { - if (FFlag::LuauExtendedTypeMismatchError) - errors.push_back(TypeError{location, TypeMismatch{superTy, subTy, "none of the intersection parts are compatible"}}); - else - errors.push_back(TypeError{location, TypeMismatch{superTy, subTy}}); + errors.push_back(TypeError{location, TypeMismatch{superTy, subTy, "none of the intersection parts are compatible"}}); } } else if (get(superTy) && get(subTy)) @@ -1241,10 +1223,7 @@ void Unifier::tryUnifyTables(TypeId left, TypeId right, bool isIntersection) Unifier innerState = makeChildUnifier(); innerState.tryUnify_(prop.type, r->second.type); - if (FFlag::LuauExtendedTypeMismatchError) - checkChildUnifierTypeMismatch(innerState.errors, name, left, right); - else - checkChildUnifierTypeMismatch(innerState.errors, left, right); + checkChildUnifierTypeMismatch(innerState.errors, name, left, right); if (innerState.errors.empty()) log.concat(std::move(innerState.log)); @@ -1261,10 +1240,7 @@ void Unifier::tryUnifyTables(TypeId left, TypeId right, bool isIntersection) Unifier innerState = makeChildUnifier(); innerState.tryUnify_(prop.type, rt->indexer->indexResultType); - if (FFlag::LuauExtendedTypeMismatchError) - checkChildUnifierTypeMismatch(innerState.errors, name, left, right); - else - checkChildUnifierTypeMismatch(innerState.errors, left, right); + checkChildUnifierTypeMismatch(innerState.errors, name, left, right); if (innerState.errors.empty()) log.concat(std::move(innerState.log)); @@ -1302,10 +1278,7 @@ void Unifier::tryUnifyTables(TypeId left, TypeId right, bool isIntersection) Unifier innerState = makeChildUnifier(); innerState.tryUnify_(prop.type, lt->indexer->indexResultType); - if (FFlag::LuauExtendedTypeMismatchError) - checkChildUnifierTypeMismatch(innerState.errors, name, left, right); - else - checkChildUnifierTypeMismatch(innerState.errors, left, right); + checkChildUnifierTypeMismatch(innerState.errors, name, left, right); if (innerState.errors.empty()) log.concat(std::move(innerState.log)); @@ -1723,18 +1696,11 @@ void Unifier::tryUnifyWithMetatable(TypeId metatable, TypeId other, bool reverse innerState.tryUnify_(lhs->table, rhs->table); innerState.tryUnify_(lhs->metatable, rhs->metatable); - if (FFlag::LuauExtendedTypeMismatchError) - { - if (auto e = hasUnificationTooComplex(innerState.errors)) - errors.push_back(*e); - else if (!innerState.errors.empty()) - errors.push_back( - TypeError{location, TypeMismatch{reversed ? other : metatable, reversed ? metatable : other, "", innerState.errors.front()}}); - } - else - { - checkChildUnifierTypeMismatch(innerState.errors, reversed ? other : metatable, reversed ? metatable : other); - } + if (auto e = hasUnificationTooComplex(innerState.errors)) + errors.push_back(*e); + else if (!innerState.errors.empty()) + errors.push_back( + TypeError{location, TypeMismatch{reversed ? other : metatable, reversed ? metatable : other, "", innerState.errors.front()}}); log.concat(std::move(innerState.log)); } @@ -1821,31 +1787,22 @@ void Unifier::tryUnifyWithClass(TypeId superTy, TypeId subTy, bool reversed) { ok = false; errors.push_back(TypeError{location, UnknownProperty{superTy, propName}}); - if (!FFlag::LuauExtendedClassMismatchError) - tryUnify_(prop.type, getSingletonTypes().errorRecoveryType()); } else { - if (FFlag::LuauExtendedClassMismatchError) + Unifier innerState = makeChildUnifier(); + innerState.tryUnify_(prop.type, classProp->type); + + checkChildUnifierTypeMismatch(innerState.errors, propName, reversed ? subTy : superTy, reversed ? superTy : subTy); + + if (innerState.errors.empty()) { - Unifier innerState = makeChildUnifier(); - innerState.tryUnify_(prop.type, classProp->type); - - checkChildUnifierTypeMismatch(innerState.errors, propName, reversed ? subTy : superTy, reversed ? superTy : subTy); - - if (innerState.errors.empty()) - { - log.concat(std::move(innerState.log)); - } - else - { - ok = false; - innerState.log.rollback(); - } + log.concat(std::move(innerState.log)); } else { - tryUnify_(prop.type, classProp->type); + ok = false; + innerState.log.rollback(); } } } @@ -2185,8 +2142,6 @@ void Unifier::checkChildUnifierTypeMismatch(const ErrorVec& innerErrors, TypeId void Unifier::checkChildUnifierTypeMismatch(const ErrorVec& innerErrors, const std::string& prop, TypeId wantedType, TypeId givenType) { - LUAU_ASSERT(FFlag::LuauExtendedTypeMismatchError || FFlag::LuauExtendedClassMismatchError); - if (auto e = hasUnificationTooComplex(innerErrors)) errors.push_back(*e); else if (!innerErrors.empty()) diff --git a/Ast/src/Parser.cpp b/Ast/src/Parser.cpp index dd24f27c..72f61649 100644 --- a/Ast/src/Parser.cpp +++ b/Ast/src/Parser.cpp @@ -14,7 +14,6 @@ LUAU_FASTFLAGVARIABLE(LuauIfElseExpressionBaseSupport, false) LUAU_FASTFLAGVARIABLE(LuauIfStatementRecursionGuard, false) LUAU_FASTFLAGVARIABLE(LuauFixAmbiguousErrorRecoveryInAssign, false) LUAU_FASTFLAGVARIABLE(LuauParseSingletonTypes, false) -LUAU_FASTFLAGVARIABLE(LuauParseGenericFunctionTypeBegin, false) namespace Luau { @@ -1368,9 +1367,6 @@ AstTypeOrPack Parser::parseFunctionTypeAnnotation(bool allowPack) Lexeme parameterStart = lexer.current(); - if (!FFlag::LuauParseGenericFunctionTypeBegin) - begin = parameterStart; - expectAndConsume('(', "function parameters"); matchRecoveryStopOnToken[Lexeme::SkinnyArrow]++; diff --git a/Sources.cmake b/Sources.cmake index 14834b3a..a7153eb3 100644 --- a/Sources.cmake +++ b/Sources.cmake @@ -45,6 +45,7 @@ target_sources(Luau.Analysis PRIVATE Analysis/include/Luau/IostreamHelpers.h Analysis/include/Luau/JsonEncoder.h Analysis/include/Luau/Linter.h + Analysis/include/Luau/LValue.h Analysis/include/Luau/Module.h Analysis/include/Luau/ModuleResolver.h Analysis/include/Luau/Predicate.h @@ -80,8 +81,8 @@ target_sources(Luau.Analysis PRIVATE Analysis/src/IostreamHelpers.cpp Analysis/src/JsonEncoder.cpp Analysis/src/Linter.cpp + Analysis/src/LValue.cpp Analysis/src/Module.cpp - Analysis/src/Predicate.cpp Analysis/src/Quantify.cpp Analysis/src/RequireTracer.cpp Analysis/src/Scope.cpp @@ -194,10 +195,10 @@ if(TARGET Luau.UnitTest) tests/Frontend.test.cpp tests/JsonEncoder.test.cpp tests/Linter.test.cpp + tests/LValue.test.cpp tests/Module.test.cpp tests/NonstrictMode.test.cpp tests/Parser.test.cpp - tests/Predicate.test.cpp tests/RequireTracer.test.cpp tests/StringUtils.test.cpp tests/Symbol.test.cpp diff --git a/VM/src/lbaselib.cpp b/VM/src/lbaselib.cpp index 881c804d..988fd315 100644 --- a/VM/src/lbaselib.cpp +++ b/VM/src/lbaselib.cpp @@ -36,12 +36,14 @@ static int luaB_tonumber(lua_State* L) int base = luaL_optinteger(L, 2, 10); if (base == 10) { /* standard conversion */ - luaL_checkany(L, 1); - if (lua_isnumber(L, 1)) + int isnum = 0; + double n = lua_tonumberx(L, 1, &isnum); + if (isnum) { - lua_pushnumber(L, lua_tonumber(L, 1)); + lua_pushnumber(L, n); return 1; } + luaL_checkany(L, 1); /* error if we don't have any argument */ } else { diff --git a/tests/Autocomplete.test.cpp b/tests/Autocomplete.test.cpp index 62a9999b..8ca09c0e 100644 --- a/tests/Autocomplete.test.cpp +++ b/tests/Autocomplete.test.cpp @@ -1394,7 +1394,7 @@ TEST_CASE_FIXTURE(ACFixture, "type_correct_function_return_types") check(R"( local function target(a: number, b: string) return a + #b end local function bar1(a: number) return -a end -local function bar2(a: string) reutrn a .. 'x' end +local function bar2(a: string) return a .. 'x' end return target(b@1 )"); @@ -1422,7 +1422,7 @@ return target(bar1, b@1 check(R"( local function target(a: number, b: string) return a + #b end local function bar1(a: number): (...number) return -a, a end -local function bar2(a: string) reutrn a .. 'x' end +local function bar2(a: string) return a .. 'x' end return target(b@1 )"); @@ -1918,7 +1918,7 @@ TEST_CASE_FIXTURE(ACFixture, "type_correct_function_no_parenthesis") check(R"( local function target(a: (number) -> number) return a(4) end local function bar1(a: number) return -a end -local function bar2(a: string) reutrn a .. 'x' end +local function bar2(a: string) return a .. 'x' end return target(b@1 )"); diff --git a/tests/LValue.test.cpp b/tests/LValue.test.cpp new file mode 100644 index 00000000..8a092779 --- /dev/null +++ b/tests/LValue.test.cpp @@ -0,0 +1,198 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +#include "Luau/TypeInfer.h" + +#include "Fixture.h" +#include "ScopedFlags.h" + +#include "doctest.h" + +using namespace Luau; + +static void merge(TypeArena& arena, RefinementMap& l, const RefinementMap& r) +{ + Luau::merge(l, r, [&arena](TypeId a, TypeId b) -> TypeId { + // TODO: normalize here also. + std::unordered_set s; + + if (auto utv = get(follow(a))) + s.insert(begin(utv), end(utv)); + else + s.insert(a); + + if (auto utv = get(follow(b))) + s.insert(begin(utv), end(utv)); + else + s.insert(b); + + std::vector options(s.begin(), s.end()); + return options.size() == 1 ? options[0] : arena.addType(UnionTypeVar{std::move(options)}); + }); +} + +static LValue mkSymbol(const std::string& s) +{ + return Symbol{AstName{s.data()}}; +} + +TEST_SUITE_BEGIN("LValue"); + +TEST_CASE("Luau_merge_hashmap_order") +{ + ScopedFastFlag sff{"LuauLValueAsKey", true}; + + std::string a = "a"; + std::string b = "b"; + std::string c = "c"; + + RefinementMap m{{ + {mkSymbol(b), getSingletonTypes().stringType}, + {mkSymbol(c), getSingletonTypes().numberType}, + }}; + + RefinementMap other{{ + {mkSymbol(a), getSingletonTypes().stringType}, + {mkSymbol(b), getSingletonTypes().stringType}, + {mkSymbol(c), getSingletonTypes().booleanType}, + }}; + + TypeArena arena; + merge(arena, m, other); + + REQUIRE_EQ(3, m.NEW_refinements.size()); + REQUIRE(m.NEW_refinements.count(mkSymbol(a))); + REQUIRE(m.NEW_refinements.count(mkSymbol(b))); + REQUIRE(m.NEW_refinements.count(mkSymbol(c))); + + CHECK_EQ("string", toString(m.NEW_refinements[mkSymbol(a)])); + CHECK_EQ("string", toString(m.NEW_refinements[mkSymbol(b)])); + CHECK_EQ("boolean | number", toString(m.NEW_refinements[mkSymbol(c)])); +} + +TEST_CASE("Luau_merge_hashmap_order2") +{ + ScopedFastFlag sff{"LuauLValueAsKey", true}; + + std::string a = "a"; + std::string b = "b"; + std::string c = "c"; + + RefinementMap m{{ + {mkSymbol(a), getSingletonTypes().stringType}, + {mkSymbol(b), getSingletonTypes().stringType}, + {mkSymbol(c), getSingletonTypes().numberType}, + }}; + + RefinementMap other{{ + {mkSymbol(b), getSingletonTypes().stringType}, + {mkSymbol(c), getSingletonTypes().booleanType}, + }}; + + TypeArena arena; + merge(arena, m, other); + + REQUIRE_EQ(3, m.NEW_refinements.size()); + REQUIRE(m.NEW_refinements.count(mkSymbol(a))); + REQUIRE(m.NEW_refinements.count(mkSymbol(b))); + REQUIRE(m.NEW_refinements.count(mkSymbol(c))); + + CHECK_EQ("string", toString(m.NEW_refinements[mkSymbol(a)])); + CHECK_EQ("string", toString(m.NEW_refinements[mkSymbol(b)])); + CHECK_EQ("boolean | number", toString(m.NEW_refinements[mkSymbol(c)])); +} + +TEST_CASE("one_map_has_overlap_at_end_whereas_other_has_it_in_start") +{ + ScopedFastFlag sff{"LuauLValueAsKey", true}; + + std::string a = "a"; + std::string b = "b"; + std::string c = "c"; + std::string d = "d"; + std::string e = "e"; + + RefinementMap m{{ + {mkSymbol(a), getSingletonTypes().stringType}, + {mkSymbol(b), getSingletonTypes().numberType}, + {mkSymbol(c), getSingletonTypes().booleanType}, + }}; + + RefinementMap other{{ + {mkSymbol(c), getSingletonTypes().stringType}, + {mkSymbol(d), getSingletonTypes().numberType}, + {mkSymbol(e), getSingletonTypes().booleanType}, + }}; + + TypeArena arena; + merge(arena, m, other); + + REQUIRE_EQ(5, m.NEW_refinements.size()); + REQUIRE(m.NEW_refinements.count(mkSymbol(a))); + REQUIRE(m.NEW_refinements.count(mkSymbol(b))); + REQUIRE(m.NEW_refinements.count(mkSymbol(c))); + REQUIRE(m.NEW_refinements.count(mkSymbol(d))); + REQUIRE(m.NEW_refinements.count(mkSymbol(e))); + + CHECK_EQ("string", toString(m.NEW_refinements[mkSymbol(a)])); + CHECK_EQ("number", toString(m.NEW_refinements[mkSymbol(b)])); + CHECK_EQ("boolean | string", toString(m.NEW_refinements[mkSymbol(c)])); + CHECK_EQ("number", toString(m.NEW_refinements[mkSymbol(d)])); + CHECK_EQ("boolean", toString(m.NEW_refinements[mkSymbol(e)])); +} + +TEST_CASE("hashing_lvalue_global_prop_access") +{ + std::string t1 = "t"; + std::string x1 = "x"; + + LValue t_x1{Field{std::make_shared(Symbol{AstName{t1.data()}}), x1}}; + + std::string t2 = "t"; + std::string x2 = "x"; + + LValue t_x2{Field{std::make_shared(Symbol{AstName{t2.data()}}), x2}}; + + CHECK_EQ(t_x1, t_x1); + CHECK_EQ(t_x1, t_x2); + CHECK_EQ(t_x2, t_x2); + + CHECK_EQ(LValueHasher{}(t_x1), LValueHasher{}(t_x1)); + CHECK_EQ(LValueHasher{}(t_x1), LValueHasher{}(t_x2)); + CHECK_EQ(LValueHasher{}(t_x2), LValueHasher{}(t_x2)); + + NEW_RefinementMap m; + m[t_x1] = getSingletonTypes().stringType; + m[t_x2] = getSingletonTypes().numberType; + + CHECK_EQ(1, m.size()); +} + +TEST_CASE("hashing_lvalue_local_prop_access") +{ + std::string t1 = "t"; + std::string x1 = "x"; + + AstLocal localt1{AstName{t1.data()}, Location(), nullptr, 0, 0, nullptr}; + LValue t_x1{Field{std::make_shared(Symbol{&localt1}), x1}}; + + std::string t2 = "t"; + std::string x2 = "x"; + + AstLocal localt2{AstName{t2.data()}, Location(), &localt1, 0, 0, nullptr}; + LValue t_x2{Field{std::make_shared(Symbol{&localt2}), x2}}; + + CHECK_EQ(t_x1, t_x1); + CHECK_NE(t_x1, t_x2); + CHECK_EQ(t_x2, t_x2); + + CHECK_EQ(LValueHasher{}(t_x1), LValueHasher{}(t_x1)); + CHECK_NE(LValueHasher{}(t_x1), LValueHasher{}(t_x2)); + CHECK_EQ(LValueHasher{}(t_x2), LValueHasher{}(t_x2)); + + NEW_RefinementMap m; + m[t_x1] = getSingletonTypes().stringType; + m[t_x2] = getSingletonTypes().numberType; + + CHECK_EQ(2, m.size()); +} + +TEST_SUITE_END(); diff --git a/tests/Predicate.test.cpp b/tests/Predicate.test.cpp deleted file mode 100644 index 7081693e..00000000 --- a/tests/Predicate.test.cpp +++ /dev/null @@ -1,117 +0,0 @@ -// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details -#include "Luau/TypeInfer.h" - -#include "Fixture.h" -#include "ScopedFlags.h" - -#include "doctest.h" - -using namespace Luau; - -static void merge(TypeArena& arena, RefinementMap& l, const RefinementMap& r) -{ - Luau::merge(l, r, [&arena](TypeId a, TypeId b) -> TypeId { - // TODO: normalize here also. - std::unordered_set s; - - if (auto utv = get(follow(a))) - s.insert(begin(utv), end(utv)); - else - s.insert(a); - - if (auto utv = get(follow(b))) - s.insert(begin(utv), end(utv)); - else - s.insert(b); - - std::vector options(s.begin(), s.end()); - return options.size() == 1 ? options[0] : arena.addType(UnionTypeVar{std::move(options)}); - }); -} - -TEST_SUITE_BEGIN("Predicate"); - -TEST_CASE_FIXTURE(Fixture, "Luau_merge_hashmap_order") -{ - RefinementMap m{ - {"b", typeChecker.stringType}, - {"c", typeChecker.numberType}, - }; - - RefinementMap other{ - {"a", typeChecker.stringType}, - {"b", typeChecker.stringType}, - {"c", typeChecker.booleanType}, - }; - - TypeArena arena; - merge(arena, m, other); - - REQUIRE_EQ(3, m.size()); - REQUIRE(m.count("a")); - REQUIRE(m.count("b")); - REQUIRE(m.count("c")); - - CHECK_EQ("string", toString(m["a"])); - CHECK_EQ("string", toString(m["b"])); - CHECK_EQ("boolean | number", toString(m["c"])); -} - -TEST_CASE_FIXTURE(Fixture, "Luau_merge_hashmap_order2") -{ - RefinementMap m{ - {"a", typeChecker.stringType}, - {"b", typeChecker.stringType}, - {"c", typeChecker.numberType}, - }; - - RefinementMap other{ - {"b", typeChecker.stringType}, - {"c", typeChecker.booleanType}, - }; - - TypeArena arena; - merge(arena, m, other); - - REQUIRE_EQ(3, m.size()); - REQUIRE(m.count("a")); - REQUIRE(m.count("b")); - REQUIRE(m.count("c")); - - CHECK_EQ("string", toString(m["a"])); - CHECK_EQ("string", toString(m["b"])); - CHECK_EQ("boolean | number", toString(m["c"])); -} - -TEST_CASE_FIXTURE(Fixture, "one_map_has_overlap_at_end_whereas_other_has_it_in_start") -{ - RefinementMap m{ - {"a", typeChecker.stringType}, - {"b", typeChecker.numberType}, - {"c", typeChecker.booleanType}, - }; - - RefinementMap other{ - {"c", typeChecker.stringType}, - {"d", typeChecker.numberType}, - {"e", typeChecker.booleanType}, - }; - - TypeArena arena; - merge(arena, m, other); - - REQUIRE_EQ(5, m.size()); - REQUIRE(m.count("a")); - REQUIRE(m.count("b")); - REQUIRE(m.count("c")); - REQUIRE(m.count("d")); - REQUIRE(m.count("e")); - - CHECK_EQ("string", toString(m["a"])); - CHECK_EQ("number", toString(m["b"])); - CHECK_EQ("boolean | string", toString(m["c"])); - CHECK_EQ("number", toString(m["d"])); - CHECK_EQ("boolean", toString(m["e"])); -} - -TEST_SUITE_END(); diff --git a/tests/Symbol.test.cpp b/tests/Symbol.test.cpp index 44fe3a3c..e7d2973b 100644 --- a/tests/Symbol.test.cpp +++ b/tests/Symbol.test.cpp @@ -10,7 +10,7 @@ using namespace Luau; TEST_SUITE_BEGIN("SymbolTests"); -TEST_CASE("hashing") +TEST_CASE("hashing_globals") { std::string s1 = "name"; std::string s2 = "name"; @@ -31,10 +31,37 @@ TEST_CASE("hashing") CHECK_EQ(std::hash()(two), std::hash()(two)); std::unordered_map theMap; - theMap[AstName{s1.data()}] = 5; - theMap[AstName{s2.data()}] = 1; + theMap[n1] = 5; + theMap[n2] = 1; REQUIRE_EQ(1, theMap.size()); } +TEST_CASE("hashing_locals") +{ + std::string s1 = "name"; + std::string s2 = "name"; + + // These two names point to distinct memory areas. + AstLocal one{AstName{s1.data()}, Location(), nullptr, 0, 0, nullptr}; + AstLocal two{AstName{s2.data()}, Location(), &one, 0, 0, nullptr}; + + Symbol n1{&one}; + Symbol n2{&two}; + + CHECK(n1 == n1); + CHECK(n1 != n2); + CHECK(n2 == n2); + + CHECK_EQ(std::hash()(&one), std::hash()(&one)); + CHECK_NE(std::hash()(&one), std::hash()(&two)); + CHECK_EQ(std::hash()(&two), std::hash()(&two)); + + std::unordered_map theMap; + theMap[n1] = 5; + theMap[n2] = 1; + + REQUIRE_EQ(2, theMap.size()); +} + TEST_SUITE_END(); diff --git a/tests/Transpiler.test.cpp b/tests/Transpiler.test.cpp index 327fa0bb..47c3883c 100644 --- a/tests/Transpiler.test.cpp +++ b/tests/Transpiler.test.cpp @@ -555,8 +555,6 @@ TEST_CASE_FIXTURE(Fixture, "transpile_assign_multiple") TEST_CASE_FIXTURE(Fixture, "transpile_generic_function") { - ScopedFastFlag luauParseGenericFunctionTypeBegin("LuauParseGenericFunctionTypeBegin", true); - std::string code = R"( local function foo(a: T, ...: S...) return 1 end local f: (T, S...)->(number) = foo diff --git a/tests/TypeInfer.builtins.test.cpp b/tests/TypeInfer.builtins.test.cpp index 1d8135d4..506279b9 100644 --- a/tests/TypeInfer.builtins.test.cpp +++ b/tests/TypeInfer.builtins.test.cpp @@ -798,13 +798,14 @@ TEST_CASE_FIXTURE(Fixture, "string_format_report_all_type_errors_at_correct_posi { CheckResult result = check(R"( ("%s%d%s"):format(1, "hello", true) + string.format("%s%d%s", 1, "hello", true) )"); TypeId stringType = typeChecker.stringType; TypeId numberType = typeChecker.numberType; TypeId booleanType = typeChecker.booleanType; - LUAU_REQUIRE_ERROR_COUNT(3, result); + LUAU_REQUIRE_ERROR_COUNT(6, result); CHECK_EQ(Location(Position{1, 26}, Position{1, 27}), result.errors[0].location); CHECK_EQ(TypeErrorData(TypeMismatch{stringType, numberType}), result.errors[0].data); @@ -814,6 +815,15 @@ TEST_CASE_FIXTURE(Fixture, "string_format_report_all_type_errors_at_correct_posi CHECK_EQ(Location(Position{1, 38}, Position{1, 42}), result.errors[2].location); CHECK_EQ(TypeErrorData(TypeMismatch{stringType, booleanType}), result.errors[2].data); + + CHECK_EQ(Location(Position{2, 32}, Position{2, 33}), result.errors[3].location); + CHECK_EQ(TypeErrorData(TypeMismatch{stringType, numberType}), result.errors[3].data); + + CHECK_EQ(Location(Position{2, 35}, Position{2, 42}), result.errors[4].location); + CHECK_EQ(TypeErrorData(TypeMismatch{numberType, stringType}), result.errors[4].data); + + CHECK_EQ(Location(Position{2, 44}, Position{2, 48}), result.errors[5].location); + CHECK_EQ(TypeErrorData(TypeMismatch{stringType, booleanType}), result.errors[5].data); } TEST_CASE_FIXTURE(Fixture, "tonumber_returns_optional_number_type") diff --git a/tests/TypeInfer.classes.test.cpp b/tests/TypeInfer.classes.test.cpp index 1ff23fe6..0283ae19 100644 --- a/tests/TypeInfer.classes.test.cpp +++ b/tests/TypeInfer.classes.test.cpp @@ -449,8 +449,6 @@ b.X = 2 -- real Vector2.X is also read-only TEST_CASE_FIXTURE(ClassFixture, "detailed_class_unification_error") { - ScopedFastFlag luauExtendedClassMismatchError{"LuauExtendedClassMismatchError", true}; - CheckResult result = check(R"( local function foo(v) return v.X :: number + string.len(v.Y) diff --git a/tests/TypeInfer.intersectionTypes.test.cpp b/tests/TypeInfer.intersectionTypes.test.cpp index 893bc2b3..93c0baf6 100644 --- a/tests/TypeInfer.intersectionTypes.test.cpp +++ b/tests/TypeInfer.intersectionTypes.test.cpp @@ -343,8 +343,6 @@ TEST_CASE_FIXTURE(Fixture, "table_intersection_setmetatable") TEST_CASE_FIXTURE(Fixture, "error_detailed_intersection_part") { - ScopedFastFlag luauExtendedTypeMismatchError{"LuauExtendedTypeMismatchError", true}; - CheckResult result = check(R"( type X = { x: number } type Y = { y: number } @@ -363,8 +361,6 @@ caused by: TEST_CASE_FIXTURE(Fixture, "error_detailed_intersection_all") { - ScopedFastFlag luauExtendedTypeMismatchError{"LuauExtendedTypeMismatchError", true}; - CheckResult result = check(R"( type X = { x: number } type Y = { y: number } diff --git a/tests/TypeInfer.refinements.test.cpp b/tests/TypeInfer.refinements.test.cpp index 688680c1..503b613f 100644 --- a/tests/TypeInfer.refinements.test.cpp +++ b/tests/TypeInfer.refinements.test.cpp @@ -280,7 +280,6 @@ TEST_CASE_FIXTURE(Fixture, "assert_non_binary_expressions_actually_resolve_const TEST_CASE_FIXTURE(Fixture, "assign_table_with_refined_property_with_a_similar_type_is_illegal") { ScopedFastFlag LuauTableSubtypingVariance2{"LuauTableSubtypingVariance2", true}; - ScopedFastFlag luauExtendedTypeMismatchError{"LuauExtendedTypeMismatchError", true}; CheckResult result = check(R"( local t: {x: number?} = {x = nil} @@ -1085,6 +1084,41 @@ TEST_CASE_FIXTURE(Fixture, "type_comparison_ifelse_expression") CHECK_EQ("any", toString(requireTypeAtPosition({6, 66}))); } +TEST_CASE_FIXTURE(Fixture, "correctly_lookup_a_shadowed_local_that_which_was_previously_refined") +{ + ScopedFastFlag sff{"LuauLValueAsKey", true}; + + CheckResult result = check(R"( + local foo: string? = "hi" + assert(foo) + local foo: number = 5 + print(foo:sub(1, 1)) + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + CHECK_EQ("Type 'number' does not have key 'sub'", toString(result.errors[0])); +} + +TEST_CASE_FIXTURE(Fixture, "correctly_lookup_property_whose_base_was_previously_refined") +{ + ScopedFastFlag sff{"LuauLValueAsKey", true}; + + CheckResult result = check(R"( + type T = {x: string | number} + local t: T? = {x = "hi"} + if t then + if type(t.x) == "string" then + local foo = t.x + end + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ("string", toString(requireTypeAtPosition({5, 30}))); +} + TEST_CASE_FIXTURE(Fixture, "apply_refinements_on_astexprindexexpr_whose_subscript_expr_is_constant_string") { ScopedFastFlag sff{"LuauRefiLookupFromIndexExpr", true}; @@ -1092,6 +1126,7 @@ TEST_CASE_FIXTURE(Fixture, "apply_refinements_on_astexprindexexpr_whose_subscrip CheckResult result = check(R"( type T = { [string]: { prop: number }? } local t: T = {} + if t["hello"] then local foo = t["hello"].prop end diff --git a/tests/TypeInfer.singletons.test.cpp b/tests/TypeInfer.singletons.test.cpp index 1621ef32..68dc1b4f 100644 --- a/tests/TypeInfer.singletons.test.cpp +++ b/tests/TypeInfer.singletons.test.cpp @@ -202,7 +202,6 @@ TEST_CASE_FIXTURE(Fixture, "enums_using_singletons_mismatch") ScopedFastFlag sffs[] = { {"LuauSingletonTypes", true}, {"LuauParseSingletonTypes", true}, - {"LuauExtendedTypeMismatchError", true}, }; CheckResult result = check(R"( diff --git a/tests/TypeInfer.tables.test.cpp b/tests/TypeInfer.tables.test.cpp index 3ea9b80c..80f40407 100644 --- a/tests/TypeInfer.tables.test.cpp +++ b/tests/TypeInfer.tables.test.cpp @@ -1955,7 +1955,6 @@ TEST_CASE_FIXTURE(Fixture, "table_insert_should_cope_with_optional_properties_in TEST_CASE_FIXTURE(Fixture, "error_detailed_prop") { ScopedFastFlag LuauTableSubtypingVariance2{"LuauTableSubtypingVariance2", true}; // Only for new path - ScopedFastFlag luauExtendedTypeMismatchError{"LuauExtendedTypeMismatchError", true}; CheckResult result = check(R"( type A = { x: number, y: number } @@ -1974,7 +1973,6 @@ caused by: TEST_CASE_FIXTURE(Fixture, "error_detailed_prop_nested") { ScopedFastFlag LuauTableSubtypingVariance2{"LuauTableSubtypingVariance2", true}; // Only for new path - ScopedFastFlag luauExtendedTypeMismatchError{"LuauExtendedTypeMismatchError", true}; CheckResult result = check(R"( type AS = { x: number, y: number } @@ -1998,7 +1996,6 @@ caused by: TEST_CASE_FIXTURE(Fixture, "error_detailed_metatable_prop") { ScopedFastFlag LuauTableSubtypingVariance2{"LuauTableSubtypingVariance2", true}; // Only for new path - ScopedFastFlag luauExtendedTypeMismatchError{"LuauExtendedTypeMismatchError", true}; CheckResult result = check(R"( local a1 = setmetatable({ x = 2, y = 3 }, { __call = function(s) end }); @@ -2062,7 +2059,6 @@ TEST_CASE_FIXTURE(Fixture, "explicitly_typed_table_error") {"LuauPropertiesGetExpectedType", true}, {"LuauExpectedTypesOfProperties", true}, {"LuauTableSubtypingVariance2", true}, - {"LuauExtendedTypeMismatchError", true}, }; CheckResult result = check(R"( diff --git a/tests/TypeInfer.test.cpp b/tests/TypeInfer.test.cpp index ad9ea827..76324556 100644 --- a/tests/TypeInfer.test.cpp +++ b/tests/TypeInfer.test.cpp @@ -5013,6 +5013,19 @@ caused by: Return #2 type is not compatible. Type 'string' could not be converted into 'boolean')"); } +TEST_CASE_FIXTURE(Fixture, "prop_access_on_any_with_other_options") +{ + ScopedFastFlag sff{"LuauLValueAsKey", true}; + + CheckResult result = check(R"( + local function f(thing: any | string) + local foo = thing.SomeRandomKey + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + TEST_CASE_FIXTURE(Fixture, "table_function_check_use_after_free") { ScopedFastFlag luauUnifyFunctionCheckResult{"LuauUpdateFunctionNameBinding", true}; diff --git a/tests/TypeInfer.unionTypes.test.cpp b/tests/TypeInfer.unionTypes.test.cpp index b095a0db..2357869e 100644 --- a/tests/TypeInfer.unionTypes.test.cpp +++ b/tests/TypeInfer.unionTypes.test.cpp @@ -425,8 +425,6 @@ y = x TEST_CASE_FIXTURE(Fixture, "error_detailed_union_part") { - ScopedFastFlag luauExtendedTypeMismatchError{"LuauExtendedTypeMismatchError", true}; - CheckResult result = check(R"( type X = { x: number } type Y = { y: number } @@ -446,8 +444,6 @@ caused by: TEST_CASE_FIXTURE(Fixture, "error_detailed_union_all") { - ScopedFastFlag luauExtendedTypeMismatchError{"LuauExtendedTypeMismatchError", true}; - CheckResult result = check(R"( type X = { x: number } type Y = { y: number } diff --git a/tests/conformance/math.lua b/tests/conformance/math.lua index d5bca44f..bfea0e1f 100644 --- a/tests/conformance/math.lua +++ b/tests/conformance/math.lua @@ -26,6 +26,7 @@ function f(...) end end +assert(pcall(tonumber) == false) assert(tonumber{} == nil) assert(tonumber'+0.01' == 1/100 and tonumber'+.01' == 0.01 and tonumber'.01' == 0.01 and tonumber'-1.' == -1 and From 44ccd8282244228a3a3108cb8e5237c45b9d92c2 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Thu, 6 Jan 2022 14:10:07 -0800 Subject: [PATCH 115/399] Sync to upstream/release/509 --- .gitignore | 18 +- Analysis/include/Luau/Frontend.h | 10 +- Analysis/include/Luau/TxnLog.h | 280 ++- Analysis/include/Luau/TypeInfer.h | 40 +- Analysis/include/Luau/TypePack.h | 8 + Analysis/include/Luau/TypeVar.h | 1 + Analysis/include/Luau/TypedAllocator.h | 23 +- Analysis/include/Luau/Unifier.h | 48 +- Analysis/src/Autocomplete.cpp | 33 +- Analysis/src/EmbeddedBuiltinDefinitions.cpp | 7 +- Analysis/src/Frontend.cpp | 28 +- Analysis/src/Module.cpp | 2 +- Analysis/src/Quantify.cpp | 14 +- Analysis/src/ToString.cpp | 48 +- Analysis/src/TxnLog.cpp | 295 ++- Analysis/src/TypeInfer.cpp | 863 ++++--- Analysis/src/TypePack.cpp | 51 +- Analysis/src/TypeVar.cpp | 19 +- Analysis/src/TypedAllocator.cpp | 1 + Analysis/src/Unifier.cpp | 2342 +++++++++++++------ Ast/include/Luau/Common.h | 8 +- CLI/Analyze.cpp | 2 +- CLI/Repl.cpp | 46 +- CLI/Web.cpp | 24 +- Compiler/include/Luau/Bytecode.h | 1 + Compiler/include/Luau/BytecodeBuilder.h | 2 + Compiler/src/BytecodeBuilder.cpp | 69 +- Compiler/src/Compiler.cpp | 7 +- Sources.cmake | 1 + VM/include/luaconf.h | 4 - VM/src/lapi.cpp | 34 +- VM/src/laux.cpp | 38 +- VM/src/lbitlib.cpp | 8 - VM/src/ldebug.cpp | 17 +- VM/src/ldo.cpp | 29 +- VM/src/lgc.cpp | 2 - VM/src/lgcdebug.cpp | 7 +- VM/src/lnumprint.cpp | 375 +++ VM/src/lnumutils.h | 7 +- VM/src/lobject.h | 1 + VM/src/ltable.cpp | 6 +- VM/src/lvmload.cpp | 28 +- VM/src/lvmutils.cpp | 7 +- fuzz/number.cpp | 35 + tests/AstQuery.test.cpp | 2 - tests/Autocomplete.test.cpp | 43 +- tests/Conformance.test.cpp | 31 +- tests/Fixture.cpp | 5 - tests/Fixture.h | 9 - tests/Frontend.test.cpp | 7 +- tests/TypeInfer.annotations.test.cpp | 2 +- tests/TypeInfer.builtins.test.cpp | 37 +- tests/TypeInfer.generics.test.cpp | 2 - tests/TypeInfer.test.cpp | 2 - tests/TypeInfer.tryUnify.test.cpp | 54 +- tests/TypeInfer.typePacks.cpp | 14 +- tests/TypeInfer.unionTypes.test.cpp | 10 +- tests/TypeVar.test.cpp | 4 +- tests/conformance/debug.lua | 9 + tests/conformance/strconv.lua | 51 + tests/main.cpp | 23 +- tools/numprint.py | 82 + 62 files changed, 3901 insertions(+), 1375 deletions(-) create mode 100644 VM/src/lnumprint.cpp create mode 100644 fuzz/number.cpp create mode 100644 tests/conformance/strconv.lua create mode 100644 tools/numprint.py diff --git a/.gitignore b/.gitignore index fa11b45b..5688dff5 100644 --- a/.gitignore +++ b/.gitignore @@ -1,8 +1,10 @@ -^build/ -^coverage/ -^fuzz/luau.pb.* -^crash-* -^default.prof* -^fuzz-* -^luau$ -/.vs +/build/ +/build[.-]*/ +/coverage/ +/.vs/ +/.vscode/ +/fuzz/luau.pb.* +/crash-* +/default.prof* +/fuzz-* +/luau diff --git a/Analysis/include/Luau/Frontend.h b/Analysis/include/Luau/Frontend.h index 07a0296a..1f64db30 100644 --- a/Analysis/include/Luau/Frontend.h +++ b/Analysis/include/Luau/Frontend.h @@ -68,7 +68,7 @@ struct FrontendOptions // is complete. bool retainFullTypeGraphs = false; - // When true, we run typechecking twice, one in the regular mode, ond once in strict mode + // When true, we run typechecking twice, once in the regular mode, and once in strict mode // in order to get more precise type information (e.g. for autocomplete). bool typecheckTwice = false; }; @@ -109,18 +109,18 @@ struct Frontend Frontend(FileResolver* fileResolver, ConfigResolver* configResolver, const FrontendOptions& options = {}); - CheckResult check(const ModuleName& name); // new shininess - LintResult lint(const ModuleName& name, std::optional enabledLintWarnings = {}); + CheckResult check(const ModuleName& name, std::optional optionOverride = {}); // new shininess + LintResult lint(const ModuleName& name, std::optional enabledLintWarnings = {}); /** Lint some code that has no associated DataModel object * * Since this source fragment has no name, we cannot cache its AST. Instead, * we return it to the caller to use as they wish. */ - std::pair lintFragment(std::string_view source, std::optional enabledLintWarnings = {}); + std::pair lintFragment(std::string_view source, std::optional enabledLintWarnings = {}); CheckResult check(const SourceModule& module); // OLD. TODO KILL - LintResult lint(const SourceModule& module, std::optional enabledLintWarnings = {}); + LintResult lint(const SourceModule& module, std::optional enabledLintWarnings = {}); bool isDirty(const ModuleName& name) const; void markDirty(const ModuleName& name, std::vector* markedDirty = nullptr); diff --git a/Analysis/include/Luau/TxnLog.h b/Analysis/include/Luau/TxnLog.h index 29988a3b..dc45bebf 100644 --- a/Analysis/include/Luau/TxnLog.h +++ b/Analysis/include/Luau/TxnLog.h @@ -1,7 +1,11 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #pragma once +#include +#include + #include "Luau/TypeVar.h" +#include "Luau/TypePack.h" LUAU_FASTFLAG(LuauShareTxnSeen); @@ -9,27 +13,28 @@ namespace Luau { // Log of where what TypeIds we are rebinding and what they used to be -struct TxnLog +// Remove with LuauUseCommitTxnLog +struct DEPRECATED_TxnLog { - TxnLog() + DEPRECATED_TxnLog() : originalSeenSize(0) , ownedSeen() , sharedSeen(&ownedSeen) { } - explicit TxnLog(std::vector>* sharedSeen) + explicit DEPRECATED_TxnLog(std::vector>* sharedSeen) : originalSeenSize(sharedSeen->size()) , ownedSeen() , sharedSeen(sharedSeen) { } - TxnLog(const TxnLog&) = delete; - TxnLog& operator=(const TxnLog&) = delete; + DEPRECATED_TxnLog(const DEPRECATED_TxnLog&) = delete; + DEPRECATED_TxnLog& operator=(const DEPRECATED_TxnLog&) = delete; - TxnLog(TxnLog&&) = default; - TxnLog& operator=(TxnLog&&) = default; + DEPRECATED_TxnLog(DEPRECATED_TxnLog&&) = default; + DEPRECATED_TxnLog& operator=(DEPRECATED_TxnLog&&) = default; void operator()(TypeId a); void operator()(TypePackId a); @@ -37,7 +42,7 @@ struct TxnLog void rollback(); - void concat(TxnLog rhs); + void concat(DEPRECATED_TxnLog rhs); bool haveSeen(TypeId lhs, TypeId rhs); void pushSeen(TypeId lhs, TypeId rhs); @@ -54,4 +59,263 @@ public: std::vector>* sharedSeen; // shared with all the descendent logs }; +// Pending state for a TypeVar. Generated by a TxnLog and committed via +// TxnLog::commit. +struct PendingType +{ + // The pending TypeVar state. + TypeVar pending; + + explicit PendingType(TypeVar state) + : pending(std::move(state)) + { + } +}; + +// Pending state for a TypePackVar. Generated by a TxnLog and committed via +// TxnLog::commit. +struct PendingTypePack +{ + // The pending TypePackVar state. + TypePackVar pending; + + explicit PendingTypePack(TypePackVar state) + : pending(std::move(state)) + { + } +}; + +template +T* getMutable(PendingType* pending) +{ + // We use getMutable here because this state is intended to be mutated freely. + return getMutable(&pending->pending); +} + +template +T* getMutable(PendingTypePack* pending) +{ + // We use getMutable here because this state is intended to be mutated freely. + return getMutable(&pending->pending); +} + +// Log of what TypeIds we are rebinding, to be committed later. +struct TxnLog +{ + TxnLog() + : ownedSeen() + , sharedSeen(&ownedSeen) + { + } + + explicit TxnLog(TxnLog* parent) + : parent(parent) + { + if (parent) + { + sharedSeen = parent->sharedSeen; + } + else + { + sharedSeen = &ownedSeen; + } + } + + explicit TxnLog(std::vector>* sharedSeen) + : sharedSeen(sharedSeen) + { + } + + TxnLog(TxnLog* parent, std::vector>* sharedSeen) + : parent(parent) + , sharedSeen(sharedSeen) + { + } + + TxnLog(const TxnLog&) = delete; + TxnLog& operator=(const TxnLog&) = delete; + + TxnLog(TxnLog&&) = default; + TxnLog& operator=(TxnLog&&) = default; + + // Gets an empty TxnLog pointer. This is useful for constructs that + // take a TxnLog, like TypePackIterator - use the empty log if you + // don't have a TxnLog to give it. + static const TxnLog* empty(); + + // Joins another TxnLog onto this one. You should use std::move to avoid + // copying the rhs TxnLog. + // + // If both logs talk about the same type, pack, or table, the rhs takes + // priority. + void concat(TxnLog rhs); + + // Commits the TxnLog, rebinding all type pointers to their pending states. + // Clears the TxnLog afterwards. + void commit(); + + // Clears the TxnLog without committing any pending changes. + void clear(); + + // Computes an inverse of this TxnLog at the current time. + // This method should be called before commit is called in order to give an + // accurate result. Committing the inverse of a TxnLog will undo the changes + // made by commit, assuming the inverse log is accurate. + TxnLog inverse(); + + bool haveSeen(TypeId lhs, TypeId rhs) const; + void pushSeen(TypeId lhs, TypeId rhs); + void popSeen(TypeId lhs, TypeId rhs); + + // Queues a type for modification. The original type will not change until commit + // is called. Use pending to get the pending state. + // + // The pointer returned lives until `commit` or `clear` is called. + PendingType* queue(TypeId ty); + + // Queues a type pack for modification. The original type pack will not change + // until commit is called. Use pending to get the pending state. + // + // The pointer returned lives until `commit` or `clear` is called. + PendingTypePack* queue(TypePackId tp); + + // Returns the pending state of a type, or nullptr if there isn't any. It is important + // to note that this pending state is not transitive: the pending state may reference + // non-pending types freely, so you may need to call pending multiple times to view the + // entire pending state of a type graph. + // + // The pointer returned lives until `commit` or `clear` is called. + PendingType* pending(TypeId ty) const; + + // Returns the pending state of a type pack, or nullptr if there isn't any. It is + // important to note that this pending state is not transitive: the pending state may + // reference non-pending types freely, so you may need to call pending multiple times + // to view the entire pending state of a type graph. + // + // The pointer returned lives until `commit` or `clear` is called. + PendingTypePack* pending(TypePackId tp) const; + + // Queues a replacement of a type with another type. + // + // The pointer returned lives until `commit` or `clear` is called. + PendingType* replace(TypeId ty, TypeVar replacement); + + // Queues a replacement of a type pack with another type pack. + // + // The pointer returned lives until `commit` or `clear` is called. + PendingTypePack* replace(TypePackId tp, TypePackVar replacement); + + // Queues a replacement of a table type with another table type that is bound + // to a specific value. + // + // The pointer returned lives until `commit` or `clear` is called. + PendingType* bindTable(TypeId ty, std::optional newBoundTo); + + // Queues a replacement of a type with a level with a duplicate of that type + // with a new type level. + // + // The pointer returned lives until `commit` or `clear` is called. + PendingType* changeLevel(TypeId ty, TypeLevel newLevel); + + // Queues a replacement of a type pack with a level with a duplicate of that + // type pack with a new type level. + // + // The pointer returned lives until `commit` or `clear` is called. + PendingTypePack* changeLevel(TypePackId tp, TypeLevel newLevel); + + // Queues a replacement of a table type with another table type with a new + // indexer. + // + // The pointer returned lives until `commit` or `clear` is called. + PendingType* changeIndexer(TypeId ty, std::optional indexer); + + // Returns the type level of the pending state of the type, or the level of that + // type, if no pending state exists. If the type doesn't have a notion of a level, + // returns nullopt. If the pending state doesn't have a notion of a level, but the + // original state does, returns nullopt. + std::optional getLevel(TypeId ty) const; + + // Follows a type, accounting for pending type states. The returned type may have + // pending state; you should use `pending` or `get` to find out. + TypeId follow(TypeId ty); + + // Follows a type pack, accounting for pending type states. The returned type pack + // may have pending state; you should use `pending` or `get` to find out. + TypePackId follow(TypePackId tp) const; + + // Replaces a given type's state with a new variant. Returns the new pending state + // of that type. + // + // The pointer returned lives until `commit` or `clear` is called. + template + PendingType* replace(TypeId ty, T replacement) + { + return replace(ty, TypeVar(replacement)); + } + + // Replaces a given type pack's state with a new variant. Returns the new + // pending state of that type pack. + // + // The pointer returned lives until `commit` or `clear` is called. + template + PendingTypePack* replace(TypePackId tp, T replacement) + { + return replace(tp, TypePackVar(replacement)); + } + + // Returns T if a given type or type pack is this variant, respecting the + // log's pending state. + // + // Do not retain this pointer; it has the potential to be invalidated when + // commit or clear is called. + template + T* getMutable(TID ty) const + { + auto* pendingTy = pending(ty); + if (pendingTy) + return Luau::getMutable(pendingTy); + + return Luau::getMutable(ty); + } + + // Returns whether a given type or type pack is a given state, respecting the + // log's pending state. + // + // This method will not assert if called on a BoundTypeVar or BoundTypePack. + template + bool is(TID ty) const + { + // We do not use getMutable here because this method can be called on + // BoundTypeVars, which triggers an assertion. + auto* pendingTy = pending(ty); + if (pendingTy) + return Luau::get_if(&pendingTy->pending.ty) != nullptr; + + return Luau::get_if(&ty->ty) != nullptr; + } + +private: + // unique_ptr is used to give us stable pointers across insertions into the + // map. Otherwise, it would be really easy to accidentally invalidate the + // pointers returned from queue/pending. + // + // We can't use a DenseHashMap here because we need a non-const iterator + // over the map when we concatenate. + std::unordered_map> typeVarChanges; + std::unordered_map> typePackChanges; + + TxnLog* parent = nullptr; + + // Owned version of sharedSeen. This should not be accessed directly in + // TxnLogs; use sharedSeen instead. This field exists because in the tree + // of TxnLogs, the root must own its seen set. In all descendant TxnLogs, + // this is an empty vector. + std::vector> ownedSeen; + +public: + // Used to avoid infinite recursion when types are cyclic. + // Shared with all the descendent TxnLogs. + std::vector>* sharedSeen; +}; + } // namespace Luau diff --git a/Analysis/include/Luau/TypeInfer.h b/Analysis/include/Luau/TypeInfer.h index 862f50d7..312283b0 100644 --- a/Analysis/include/Luau/TypeInfer.h +++ b/Analysis/include/Luau/TypeInfer.h @@ -198,32 +198,32 @@ struct TypeChecker */ TypeId anyIfNonstrict(TypeId ty) const; - /** Attempt to unify the types left and right. Treat any failures as type errors - * in the final typecheck report. + /** Attempt to unify the types. + * Treat any failures as type errors in the final typecheck report. */ - bool unify(TypeId left, TypeId right, const Location& location); - bool unify(TypePackId left, TypePackId right, const Location& location, CountMismatch::Context ctx = CountMismatch::Context::Arg); + bool unify(TypeId subTy, TypeId superTy, const Location& location); + bool unify(TypePackId subTy, TypePackId superTy, const Location& location, CountMismatch::Context ctx = CountMismatch::Context::Arg); - /** Attempt to unify the types left and right. - * If this fails, and the right type can be instantiated, do so and try unification again. + /** Attempt to unify the types. + * If this fails, and the subTy type can be instantiated, do so and try unification again. */ - bool unifyWithInstantiationIfNeeded(const ScopePtr& scope, TypeId left, TypeId right, const Location& location); - void unifyWithInstantiationIfNeeded(const ScopePtr& scope, TypeId left, TypeId right, Unifier& state); + bool unifyWithInstantiationIfNeeded(const ScopePtr& scope, TypeId subTy, TypeId superTy, const Location& location); + void unifyWithInstantiationIfNeeded(const ScopePtr& scope, TypeId subTy, TypeId superTy, Unifier& state); - /** Attempt to unify left with right. + /** Attempt to unify. * If there are errors, undo everything and return the errors. * If there are no errors, commit and return an empty error vector. */ - ErrorVec tryUnify(TypeId left, TypeId right, const Location& location); - ErrorVec tryUnify(TypePackId left, TypePackId right, const Location& location); + template + ErrorVec tryUnify_(Id subTy, Id superTy, const Location& location); + ErrorVec tryUnify(TypeId subTy, TypeId superTy, const Location& location); + ErrorVec tryUnify(TypePackId subTy, TypePackId superTy, const Location& location); // Test whether the two type vars unify. Never commits the result. - ErrorVec canUnify(TypeId superTy, TypeId subTy, const Location& location); - ErrorVec canUnify(TypePackId superTy, TypePackId subTy, const Location& location); - - // Variant that takes a preexisting 'seen' set. We need this in certain cases to avoid infinitely recursing - // into cyclic types. - ErrorVec canUnify(const std::vector>& seen, TypeId left, TypeId right, const Location& location); + template + ErrorVec canUnify_(Id subTy, Id superTy, const Location& location); + ErrorVec canUnify(TypeId subTy, TypeId superTy, const Location& location); + ErrorVec canUnify(TypePackId subTy, TypePackId superTy, const Location& location); std::optional findMetatableEntry(TypeId type, std::string entry, const Location& location); std::optional findTablePropertyRespectingMeta(TypeId lhsType, Name name, const Location& location); @@ -237,12 +237,6 @@ struct TypeChecker std::optional tryStripUnionFromNil(TypeId ty); TypeId stripFromNilAndReport(TypeId ty, const Location& location); - template - ErrorVec tryUnify_(Id left, Id right, const Location& location); - - template - ErrorVec canUnify_(Id left, Id right, const Location& location); - public: /* * Convert monotype into a a polytype, by replacing any metavariables in descendant scopes diff --git a/Analysis/include/Luau/TypePack.h b/Analysis/include/Luau/TypePack.h index e72808da..ca588ccb 100644 --- a/Analysis/include/Luau/TypePack.h +++ b/Analysis/include/Luau/TypePack.h @@ -18,6 +18,8 @@ struct VariadicTypePack; struct TypePackVar; +struct TxnLog; + using TypePackId = const TypePackVar*; using FreeTypePack = Unifiable::Free; using BoundTypePack = Unifiable::Bound; @@ -84,6 +86,7 @@ struct TypePackIterator TypePackIterator() = default; explicit TypePackIterator(TypePackId tp); + TypePackIterator(TypePackId tp, const TxnLog* log); TypePackIterator& operator++(); TypePackIterator operator++(int); @@ -104,9 +107,13 @@ private: TypePackId currentTypePack = nullptr; const TypePack* tp = nullptr; size_t currentIndex = 0; + + // Only used if LuauUseCommittingTxnLog is true. + const TxnLog* log; }; TypePackIterator begin(TypePackId tp); +TypePackIterator begin(TypePackId tp, TxnLog* log); TypePackIterator end(TypePackId tp); using SeenSet = std::set>; @@ -114,6 +121,7 @@ using SeenSet = std::set>; bool areEqual(SeenSet& seen, const TypePackVar& lhs, const TypePackVar& rhs); TypePackId follow(TypePackId tp); +TypePackId follow(TypePackId tp, std::function mapper); size_t size(TypePackId tp); bool finite(TypePackId tp); diff --git a/Analysis/include/Luau/TypeVar.h b/Analysis/include/Luau/TypeVar.h index f6829ec3..d6e17714 100644 --- a/Analysis/include/Luau/TypeVar.h +++ b/Analysis/include/Luau/TypeVar.h @@ -453,6 +453,7 @@ bool areEqual(SeenSet& seen, const TypeVar& lhs, const TypeVar& rhs); // Follow BoundTypeVars until we get to something real TypeId follow(TypeId t); +TypeId follow(TypeId t, std::function mapper); std::vector flattenIntersection(TypeId ty); diff --git a/Analysis/include/Luau/TypedAllocator.h b/Analysis/include/Luau/TypedAllocator.h index 0ded1489..64227e7c 100644 --- a/Analysis/include/Luau/TypedAllocator.h +++ b/Analysis/include/Luau/TypedAllocator.h @@ -6,6 +6,8 @@ #include #include +LUAU_FASTFLAG(LuauTypedAllocatorZeroStart) + namespace Luau { @@ -20,7 +22,10 @@ class TypedAllocator public: TypedAllocator() { - appendBlock(); + if (FFlag::LuauTypedAllocatorZeroStart) + currentBlockSize = kBlockSize; + else + appendBlock(); } ~TypedAllocator() @@ -59,12 +64,18 @@ public: bool empty() const { - return stuff.size() == 1 && currentBlockSize == 0; + if (FFlag::LuauTypedAllocatorZeroStart) + return stuff.empty(); + else + return stuff.size() == 1 && currentBlockSize == 0; } size_t size() const { - return kBlockSize * (stuff.size() - 1) + currentBlockSize; + if (FFlag::LuauTypedAllocatorZeroStart) + return stuff.empty() ? 0 : kBlockSize * (stuff.size() - 1) + currentBlockSize; + else + return kBlockSize * (stuff.size() - 1) + currentBlockSize; } void clear() @@ -72,7 +83,11 @@ public: if (frozen) unfreeze(); free(); - appendBlock(); + + if (FFlag::LuauTypedAllocatorZeroStart) + currentBlockSize = kBlockSize; + else + appendBlock(); } void freeze() diff --git a/Analysis/include/Luau/Unifier.h b/Analysis/include/Luau/Unifier.h index 7681b966..a3be739a 100644 --- a/Analysis/include/Luau/Unifier.h +++ b/Analysis/include/Luau/Unifier.h @@ -25,6 +25,7 @@ struct Unifier Mode mode; ScopePtr globalScope; // sigh. Needed solely to get at string's metatable. + DEPRECATED_TxnLog DEPRECATED_log; TxnLog log; ErrorVec errors; Location location; @@ -33,44 +34,45 @@ struct Unifier UnifierSharedState& sharedState; - Unifier(TypeArena* types, Mode mode, ScopePtr globalScope, const Location& location, Variance variance, UnifierSharedState& sharedState); + Unifier(TypeArena* types, Mode mode, ScopePtr globalScope, const Location& location, Variance variance, UnifierSharedState& sharedState, + TxnLog* parentLog = nullptr); Unifier(TypeArena* types, Mode mode, ScopePtr globalScope, std::vector>* sharedSeen, const Location& location, - Variance variance, UnifierSharedState& sharedState); + Variance variance, UnifierSharedState& sharedState, TxnLog* parentLog = nullptr); // Test whether the two type vars unify. Never commits the result. - ErrorVec canUnify(TypeId superTy, TypeId subTy); - ErrorVec canUnify(TypePackId superTy, TypePackId subTy, bool isFunctionCall = false); + ErrorVec canUnify(TypeId subTy, TypeId superTy); + ErrorVec canUnify(TypePackId subTy, TypePackId superTy, bool isFunctionCall = false); - /** Attempt to unify left with right. + /** Attempt to unify. * Populate the vector errors with any type errors that may arise. * Populate the transaction log with the set of TypeIds that need to be reset to undo the unification attempt. */ - void tryUnify(TypeId superTy, TypeId subTy, bool isFunctionCall = false, bool isIntersection = false); + void tryUnify(TypeId subTy, TypeId superTy, bool isFunctionCall = false, bool isIntersection = false); private: - void tryUnify_(TypeId superTy, TypeId subTy, bool isFunctionCall = false, bool isIntersection = false); - void tryUnifyPrimitives(TypeId superTy, TypeId subTy); - void tryUnifySingletons(TypeId superTy, TypeId subTy); - void tryUnifyFunctions(TypeId superTy, TypeId subTy, bool isFunctionCall = false); - void tryUnifyTables(TypeId left, TypeId right, bool isIntersection = false); - void DEPRECATED_tryUnifyTables(TypeId left, TypeId right, bool isIntersection = false); - void tryUnifyFreeTable(TypeId free, TypeId other); - void tryUnifySealedTables(TypeId left, TypeId right, bool isIntersection); - void tryUnifyWithMetatable(TypeId metatable, TypeId other, bool reversed); - void tryUnifyWithClass(TypeId superTy, TypeId subTy, bool reversed); - void tryUnify(const TableIndexer& superIndexer, const TableIndexer& subIndexer); + void tryUnify_(TypeId subTy, TypeId superTy, bool isFunctionCall = false, bool isIntersection = false); + void tryUnifyPrimitives(TypeId subTy, TypeId superTy); + void tryUnifySingletons(TypeId subTy, TypeId superTy); + void tryUnifyFunctions(TypeId subTy, TypeId superTy, bool isFunctionCall = false); + void tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection = false); + void DEPRECATED_tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection = false); + void tryUnifyFreeTable(TypeId subTy, TypeId superTy); + void tryUnifySealedTables(TypeId subTy, TypeId superTy, bool isIntersection); + void tryUnifyWithMetatable(TypeId subTy, TypeId superTy, bool reversed); + void tryUnifyWithClass(TypeId subTy, TypeId superTy, bool reversed); + void tryUnifyIndexer(const TableIndexer& subIndexer, const TableIndexer& superIndexer); TypeId deeplyOptional(TypeId ty, std::unordered_map seen = {}); - void cacheResult(TypeId superTy, TypeId subTy); + void cacheResult(TypeId subTy, TypeId superTy); public: - void tryUnify(TypePackId superTy, TypePackId subTy, bool isFunctionCall = false); + void tryUnify(TypePackId subTy, TypePackId superTy, bool isFunctionCall = false); private: - void tryUnify_(TypePackId superTy, TypePackId subTy, bool isFunctionCall = false); - void tryUnifyVariadics(TypePackId superTy, TypePackId subTy, bool reversed, int subOffset = 0); + void tryUnify_(TypePackId subTy, TypePackId superTy, bool isFunctionCall = false); + void tryUnifyVariadics(TypePackId subTy, TypePackId superTy, bool reversed, int subOffset = 0); - void tryUnifyWithAny(TypeId any, TypeId ty); - void tryUnifyWithAny(TypePackId any, TypePackId ty); + void tryUnifyWithAny(TypeId subTy, TypeId anyTy); + void tryUnifyWithAny(TypePackId subTy, TypePackId anyTp); std::optional findTablePropertyRespectingMeta(TypeId lhsType, Name name); diff --git a/Analysis/src/Autocomplete.cpp b/Analysis/src/Autocomplete.cpp index 4b583792..67ebd075 100644 --- a/Analysis/src/Autocomplete.cpp +++ b/Analysis/src/Autocomplete.cpp @@ -12,10 +12,12 @@ #include #include +LUAU_FASTFLAG(LuauUseCommittingTxnLog) LUAU_FASTFLAG(LuauIfElseExpressionAnalysisSupport) LUAU_FASTFLAGVARIABLE(LuauAutocompleteAvoidMutation, false); LUAU_FASTFLAGVARIABLE(LuauAutocompletePreferToCallFunctions, false); LUAU_FASTFLAGVARIABLE(LuauAutocompleteFirstArg, false); +LUAU_FASTFLAGVARIABLE(LuauCompleteBrokenStringParams, false); static const std::unordered_set kStatementStartingKeywords = { "while", "if", "local", "repeat", "function", "do", "for", "return", "break", "continue", "type", "export"}; @@ -236,28 +238,31 @@ static TypeCorrectKind checkTypeCorrectKind(const Module& module, TypeArena* typ { ty = follow(ty); - auto canUnify = [&typeArena, &module](TypeId expectedType, TypeId actualType) { + auto canUnify = [&typeArena, &module](TypeId subTy, TypeId superTy) { InternalErrorReporter iceReporter; UnifierSharedState unifierState(&iceReporter); Unifier unifier(typeArena, Mode::Strict, module.getModuleScope(), Location(), Variance::Covariant, unifierState); - if (FFlag::LuauAutocompleteAvoidMutation) + if (FFlag::LuauAutocompleteAvoidMutation && !FFlag::LuauUseCommittingTxnLog) { SeenTypes seenTypes; SeenTypePacks seenTypePacks; CloneState cloneState; - expectedType = clone(expectedType, *typeArena, seenTypes, seenTypePacks, cloneState); - actualType = clone(actualType, *typeArena, seenTypes, seenTypePacks, cloneState); + superTy = clone(superTy, *typeArena, seenTypes, seenTypePacks, cloneState); + subTy = clone(subTy, *typeArena, seenTypes, seenTypePacks, cloneState); - auto errors = unifier.canUnify(expectedType, actualType); + auto errors = unifier.canUnify(subTy, superTy); return errors.empty(); } else { - unifier.tryUnify(expectedType, actualType); + unifier.tryUnify(subTy, superTy); bool ok = unifier.errors.empty(); - unifier.log.rollback(); + + if (!FFlag::LuauUseCommittingTxnLog) + unifier.DEPRECATED_log.rollback(); + return ok; } }; @@ -293,22 +298,22 @@ static TypeCorrectKind checkTypeCorrectKind(const Module& module, TypeArena* typ { auto [retHead, retTail] = flatten(ftv->retType); - if (!retHead.empty() && canUnify(expectedType, retHead.front())) + if (!retHead.empty() && canUnify(retHead.front(), expectedType)) return TypeCorrectKind::CorrectFunctionResult; // We might only have a variadic tail pack, check if the element is compatible if (retTail) { - if (const VariadicTypePack* vtp = get(follow(*retTail)); vtp && canUnify(expectedType, vtp->ty)) + if (const VariadicTypePack* vtp = get(follow(*retTail)); vtp && canUnify(vtp->ty, expectedType)) return TypeCorrectKind::CorrectFunctionResult; } } - return canUnify(expectedType, ty) ? TypeCorrectKind::Correct : TypeCorrectKind::None; + return canUnify(ty, expectedType) ? TypeCorrectKind::Correct : TypeCorrectKind::None; } else { - if (canUnify(expectedType, ty)) + if (canUnify(ty, expectedType)) return TypeCorrectKind::Correct; // We also want to suggest functions that return compatible result @@ -320,13 +325,13 @@ static TypeCorrectKind checkTypeCorrectKind(const Module& module, TypeArena* typ auto [retHead, retTail] = flatten(ftv->retType); if (!retHead.empty()) - return canUnify(expectedType, retHead.front()) ? TypeCorrectKind::CorrectFunctionResult : TypeCorrectKind::None; + return canUnify(retHead.front(), expectedType) ? TypeCorrectKind::CorrectFunctionResult : TypeCorrectKind::None; // We might only have a variadic tail pack, check if the element is compatible if (retTail) { if (const VariadicTypePack* vtp = get(follow(*retTail))) - return canUnify(expectedType, vtp->ty) ? TypeCorrectKind::CorrectFunctionResult : TypeCorrectKind::None; + return canUnify(vtp->ty, expectedType) ? TypeCorrectKind::CorrectFunctionResult : TypeCorrectKind::None; } return TypeCorrectKind::None; @@ -1319,7 +1324,7 @@ static std::optional autocompleteStringParams(const Source return std::nullopt; } - if (!nodes.back()->is()) + if (!nodes.back()->is() && (!FFlag::LuauCompleteBrokenStringParams || !nodes.back()->is())) { return std::nullopt; } diff --git a/Analysis/src/EmbeddedBuiltinDefinitions.cpp b/Analysis/src/EmbeddedBuiltinDefinitions.cpp index d0afa742..24982506 100644 --- a/Analysis/src/EmbeddedBuiltinDefinitions.cpp +++ b/Analysis/src/EmbeddedBuiltinDefinitions.cpp @@ -138,12 +138,7 @@ declare function gcinfo(): number -- (nil, string). declare function loadstring(src: string, chunkname: string?): (((A...) -> any)?, string?) - -- a userdata object is "roughly" the same as a sealed empty table - -- except `type(newproxy(false))` evaluates to "userdata" so we may need another special type here too. - -- another important thing to note: the value passed in conditionally creates an empty metatable, and you have to use getmetatable, NOT - -- setmetatable. - -- FIXME: change this to something Luau can understand how to reject `setmetatable(newproxy(false or true), {})`. - declare function newproxy(mt: boolean?): {} + declare function newproxy(mt: boolean?): any declare coroutine: { create: ((A...) -> R...) -> thread, diff --git a/Analysis/src/Frontend.cpp b/Analysis/src/Frontend.cpp index e332f07d..fe4b6529 100644 --- a/Analysis/src/Frontend.cpp +++ b/Analysis/src/Frontend.cpp @@ -351,7 +351,7 @@ FrontendModuleResolver::FrontendModuleResolver(Frontend* frontend) { } -CheckResult Frontend::check(const ModuleName& name) +CheckResult Frontend::check(const ModuleName& name, std::optional optionOverride) { LUAU_TIMETRACE_SCOPE("Frontend::check", "Frontend"); LUAU_TIMETRACE_ARGUMENT("name", name.c_str()); @@ -372,6 +372,8 @@ CheckResult Frontend::check(const ModuleName& name) std::vector buildQueue; bool cycleDetected = parseGraph(buildQueue, checkResult, name); + FrontendOptions frontendOptions = optionOverride.value_or(options); + // Keep track of which AST nodes we've reported cycles in std::unordered_set reportedCycles; @@ -411,31 +413,11 @@ CheckResult Frontend::check(const ModuleName& name) // If we're typechecking twice, we do so. // The second typecheck is always in strict mode with DM awareness // to provide better typen information for IDE features. - if (options.typecheckTwice) + if (frontendOptions.typecheckTwice) { ModulePtr moduleForAutocomplete = typeCheckerForAutocomplete.check(sourceModule, Mode::Strict); moduleResolverForAutocomplete.modules[moduleName] = moduleForAutocomplete; } - else if (options.retainFullTypeGraphs && options.typecheckTwice && mode != Mode::Strict) - { - ModulePtr strictModule = typeChecker.check(sourceModule, Mode::Strict, environmentScope); - module->astTypes.clear(); - module->astOriginalCallTypes.clear(); - module->astExpectedTypes.clear(); - - SeenTypes seenTypes; - SeenTypePacks seenTypePacks; - CloneState cloneState; - - for (const auto& [expr, strictTy] : strictModule->astTypes) - module->astTypes[expr] = clone(strictTy, module->interfaceTypes, seenTypes, seenTypePacks, cloneState); - - for (const auto& [expr, strictTy] : strictModule->astOriginalCallTypes) - module->astOriginalCallTypes[expr] = clone(strictTy, module->interfaceTypes, seenTypes, seenTypePacks, cloneState); - - for (const auto& [expr, strictTy] : strictModule->astExpectedTypes) - module->astExpectedTypes[expr] = clone(strictTy, module->interfaceTypes, seenTypes, seenTypePacks, cloneState); - } stats.timeCheck += getTimestamp() - timestamp; stats.filesStrict += mode == Mode::Strict; @@ -444,7 +426,7 @@ CheckResult Frontend::check(const ModuleName& name) if (module == nullptr) throw std::runtime_error("Frontend::check produced a nullptr module for " + moduleName); - if (!options.retainFullTypeGraphs) + if (!frontendOptions.retainFullTypeGraphs) { // copyErrors needs to allocate into interfaceTypes as it copies // types out of internalTypes, so we unfreeze it here. diff --git a/Analysis/src/Module.cpp b/Analysis/src/Module.cpp index e1e53c97..cff85897 100644 --- a/Analysis/src/Module.cpp +++ b/Analysis/src/Module.cpp @@ -13,7 +13,7 @@ LUAU_FASTFLAGVARIABLE(DebugLuauFreezeArena, false) LUAU_FASTFLAGVARIABLE(DebugLuauTrackOwningArena, false) -LUAU_FASTINTVARIABLE(LuauTypeCloneRecursionLimit, 0) +LUAU_FASTINTVARIABLE(LuauTypeCloneRecursionLimit, 300) namespace Luau { diff --git a/Analysis/src/Quantify.cpp b/Analysis/src/Quantify.cpp index c773e208..04ebffc1 100644 --- a/Analysis/src/Quantify.cpp +++ b/Analysis/src/Quantify.cpp @@ -4,8 +4,6 @@ #include "Luau/VisitTypeVar.h" -LUAU_FASTFLAGVARIABLE(LuauQuantifyVisitOnce, false) - namespace Luau { @@ -81,16 +79,8 @@ struct Quantifier void quantify(ModulePtr module, TypeId ty, TypeLevel level) { Quantifier q{std::move(module), level}; - - if (FFlag::LuauQuantifyVisitOnce) - { - DenseHashSet seen{nullptr}; - visitTypeVarOnce(ty, q, seen); - } - else - { - visitTypeVar(ty, q); - } + DenseHashSet seen{nullptr}; + visitTypeVarOnce(ty, q, seen); FunctionTypeVar* ftv = getMutable(ty); LUAU_ASSERT(ftv); diff --git a/Analysis/src/ToString.cpp b/Analysis/src/ToString.cpp index a6be5348..889dd6dc 100644 --- a/Analysis/src/ToString.cpp +++ b/Analysis/src/ToString.cpp @@ -11,7 +11,6 @@ #include LUAU_FASTFLAG(LuauOccursCheckOkWithRecursiveFunctions) -LUAU_FASTFLAGVARIABLE(LuauFunctionArgumentNameSize, false) /* * Prefix generic typenames with gen- @@ -766,24 +765,12 @@ struct TypePackStringifier else state.emit(", "); - if (FFlag::LuauFunctionArgumentNameSize) + if (elemIndex < elemNames.size() && elemNames[elemIndex]) { - if (elemIndex < elemNames.size() && elemNames[elemIndex]) - { - state.emit(elemNames[elemIndex]->name); - state.emit(": "); - } + state.emit(elemNames[elemIndex]->name); + state.emit(": "); } - else - { - LUAU_ASSERT(elemNames.empty() || elemIndex < elemNames.size()); - if (!elemNames.empty() && elemNames[elemIndex]) - { - state.emit(elemNames[elemIndex]->name); - state.emit(": "); - } - } elemIndex++; stringify(typeId); @@ -1151,38 +1138,19 @@ std::string toStringNamedFunction(const std::string& prefix, const FunctionTypeV s += ", "; first = false; - if (FFlag::LuauFunctionArgumentNameSize) + // We don't currently respect opts.functionTypeArguments. I don't think this function should. + if (argNameIter != ftv.argNames.end()) { - // We don't currently respect opts.functionTypeArguments. I don't think this function should. - if (argNameIter != ftv.argNames.end()) - { - s += (*argNameIter ? (*argNameIter)->name : "_") + ": "; - ++argNameIter; - } - else - { - s += "_: "; - } + s += (*argNameIter ? (*argNameIter)->name : "_") + ": "; + ++argNameIter; } else { - // argNames is guaranteed to be equal to argTypes iff argNames is not empty. - // We don't currently respect opts.functionTypeArguments. I don't think this function should. - if (!ftv.argNames.empty()) - s += (*argNameIter ? (*argNameIter)->name : "_") + ": "; + s += "_: "; } s += toString_(*argPackIter); ++argPackIter; - - if (!FFlag::LuauFunctionArgumentNameSize) - { - if (!ftv.argNames.empty()) - { - LUAU_ASSERT(argNameIter != ftv.argNames.end()); - ++argNameIter; - } - } } if (argPackIter.tail()) diff --git a/Analysis/src/TxnLog.cpp b/Analysis/src/TxnLog.cpp index f6a61581..a46ac0c3 100644 --- a/Analysis/src/TxnLog.cpp +++ b/Analysis/src/TxnLog.cpp @@ -4,27 +4,34 @@ #include "Luau/TypePack.h" #include +#include + +LUAU_FASTFLAGVARIABLE(LuauUseCommittingTxnLog, false) namespace Luau { -void TxnLog::operator()(TypeId a) +void DEPRECATED_TxnLog::operator()(TypeId a) { + LUAU_ASSERT(!FFlag::LuauUseCommittingTxnLog); typeVarChanges.emplace_back(a, *a); } -void TxnLog::operator()(TypePackId a) +void DEPRECATED_TxnLog::operator()(TypePackId a) { + LUAU_ASSERT(!FFlag::LuauUseCommittingTxnLog); typePackChanges.emplace_back(a, *a); } -void TxnLog::operator()(TableTypeVar* a) +void DEPRECATED_TxnLog::operator()(TableTypeVar* a) { + LUAU_ASSERT(!FFlag::LuauUseCommittingTxnLog); tableChanges.emplace_back(a, a->boundTo); } -void TxnLog::rollback() +void DEPRECATED_TxnLog::rollback() { + LUAU_ASSERT(!FFlag::LuauUseCommittingTxnLog); for (auto it = typeVarChanges.rbegin(); it != typeVarChanges.rend(); ++it) std::swap(*asMutable(it->first), it->second); @@ -38,8 +45,9 @@ void TxnLog::rollback() sharedSeen->resize(originalSeenSize); } -void TxnLog::concat(TxnLog rhs) +void DEPRECATED_TxnLog::concat(DEPRECATED_TxnLog rhs) { + LUAU_ASSERT(!FFlag::LuauUseCommittingTxnLog); typeVarChanges.insert(typeVarChanges.end(), rhs.typeVarChanges.begin(), rhs.typeVarChanges.end()); rhs.typeVarChanges.clear(); @@ -50,23 +58,298 @@ void TxnLog::concat(TxnLog rhs) rhs.tableChanges.clear(); } -bool TxnLog::haveSeen(TypeId lhs, TypeId rhs) +bool DEPRECATED_TxnLog::haveSeen(TypeId lhs, TypeId rhs) { + LUAU_ASSERT(!FFlag::LuauUseCommittingTxnLog); const std::pair sortedPair = (lhs > rhs) ? std::make_pair(lhs, rhs) : std::make_pair(rhs, lhs); return (sharedSeen->end() != std::find(sharedSeen->begin(), sharedSeen->end(), sortedPair)); } +void DEPRECATED_TxnLog::pushSeen(TypeId lhs, TypeId rhs) +{ + LUAU_ASSERT(!FFlag::LuauUseCommittingTxnLog); + const std::pair sortedPair = (lhs > rhs) ? std::make_pair(lhs, rhs) : std::make_pair(rhs, lhs); + sharedSeen->push_back(sortedPair); +} + +void DEPRECATED_TxnLog::popSeen(TypeId lhs, TypeId rhs) +{ + LUAU_ASSERT(!FFlag::LuauUseCommittingTxnLog); + const std::pair sortedPair = (lhs > rhs) ? std::make_pair(lhs, rhs) : std::make_pair(rhs, lhs); + LUAU_ASSERT(sortedPair == sharedSeen->back()); + sharedSeen->pop_back(); +} + +static const TxnLog emptyLog; + +const TxnLog* TxnLog::empty() +{ + return &emptyLog; +} + +void TxnLog::concat(TxnLog rhs) +{ + LUAU_ASSERT(FFlag::LuauUseCommittingTxnLog); + + for (auto& [ty, rep] : rhs.typeVarChanges) + typeVarChanges[ty] = std::move(rep); + + for (auto& [tp, rep] : rhs.typePackChanges) + typePackChanges[tp] = std::move(rep); +} + +void TxnLog::commit() +{ + LUAU_ASSERT(FFlag::LuauUseCommittingTxnLog); + + for (auto& [ty, rep] : typeVarChanges) + *asMutable(ty) = rep.get()->pending; + + for (auto& [tp, rep] : typePackChanges) + *asMutable(tp) = rep.get()->pending; + + clear(); +} + +void TxnLog::clear() +{ + LUAU_ASSERT(FFlag::LuauUseCommittingTxnLog); + + typeVarChanges.clear(); + typePackChanges.clear(); +} + +TxnLog TxnLog::inverse() +{ + LUAU_ASSERT(FFlag::LuauUseCommittingTxnLog); + + TxnLog inversed(sharedSeen); + + for (auto& [ty, _rep] : typeVarChanges) + inversed.typeVarChanges[ty] = std::make_unique(*ty); + + for (auto& [tp, _rep] : typePackChanges) + inversed.typePackChanges[tp] = std::make_unique(*tp); + + return inversed; +} + +bool TxnLog::haveSeen(TypeId lhs, TypeId rhs) const +{ + LUAU_ASSERT(FFlag::LuauUseCommittingTxnLog); + + const std::pair sortedPair = (lhs > rhs) ? std::make_pair(lhs, rhs) : std::make_pair(rhs, lhs); + if (sharedSeen->end() != std::find(sharedSeen->begin(), sharedSeen->end(), sortedPair)) + { + return true; + } + + if (parent) + { + return parent->haveSeen(lhs, rhs); + } + + return false; +} + void TxnLog::pushSeen(TypeId lhs, TypeId rhs) { + LUAU_ASSERT(FFlag::LuauUseCommittingTxnLog); + const std::pair sortedPair = (lhs > rhs) ? std::make_pair(lhs, rhs) : std::make_pair(rhs, lhs); sharedSeen->push_back(sortedPair); } void TxnLog::popSeen(TypeId lhs, TypeId rhs) { + LUAU_ASSERT(FFlag::LuauUseCommittingTxnLog); + const std::pair sortedPair = (lhs > rhs) ? std::make_pair(lhs, rhs) : std::make_pair(rhs, lhs); LUAU_ASSERT(sortedPair == sharedSeen->back()); sharedSeen->pop_back(); } +PendingType* TxnLog::queue(TypeId ty) +{ + LUAU_ASSERT(FFlag::LuauUseCommittingTxnLog); + LUAU_ASSERT(!ty->persistent); + + // Explicitly don't look in ancestors. If we have discovered something new + // about this type, we don't want to mutate the parent's state. + auto& pending = typeVarChanges[ty]; + if (!pending) + pending = std::make_unique(*ty); + + return pending.get(); +} + +PendingTypePack* TxnLog::queue(TypePackId tp) +{ + LUAU_ASSERT(FFlag::LuauUseCommittingTxnLog); + LUAU_ASSERT(!tp->persistent); + + // Explicitly don't look in ancestors. If we have discovered something new + // about this type, we don't want to mutate the parent's state. + auto& pending = typePackChanges[tp]; + if (!pending) + pending = std::make_unique(*tp); + + return pending.get(); +} + +PendingType* TxnLog::pending(TypeId ty) const +{ + LUAU_ASSERT(FFlag::LuauUseCommittingTxnLog); + + for (const TxnLog* current = this; current; current = current->parent) + { + if (auto it = current->typeVarChanges.find(ty); it != current->typeVarChanges.end()) + return it->second.get(); + } + + return nullptr; +} + +PendingTypePack* TxnLog::pending(TypePackId tp) const +{ + LUAU_ASSERT(FFlag::LuauUseCommittingTxnLog); + + for (const TxnLog* current = this; current; current = current->parent) + { + if (auto it = current->typePackChanges.find(tp); it != current->typePackChanges.end()) + return it->second.get(); + } + + return nullptr; +} + +PendingType* TxnLog::replace(TypeId ty, TypeVar replacement) +{ + LUAU_ASSERT(FFlag::LuauUseCommittingTxnLog); + + PendingType* newTy = queue(ty); + newTy->pending = replacement; + return newTy; +} + +PendingTypePack* TxnLog::replace(TypePackId tp, TypePackVar replacement) +{ + LUAU_ASSERT(FFlag::LuauUseCommittingTxnLog); + + PendingTypePack* newTp = queue(tp); + newTp->pending = replacement; + return newTp; +} + +PendingType* TxnLog::bindTable(TypeId ty, std::optional newBoundTo) +{ + LUAU_ASSERT(FFlag::LuauUseCommittingTxnLog); + LUAU_ASSERT(get(ty)); + + PendingType* newTy = queue(ty); + if (TableTypeVar* ttv = Luau::getMutable(newTy)) + ttv->boundTo = newBoundTo; + + return newTy; +} + +PendingType* TxnLog::changeLevel(TypeId ty, TypeLevel newLevel) +{ + LUAU_ASSERT(FFlag::LuauUseCommittingTxnLog); + LUAU_ASSERT(get(ty) || get(ty) || get(ty)); + + PendingType* newTy = queue(ty); + if (FreeTypeVar* ftv = Luau::getMutable(newTy)) + { + ftv->level = newLevel; + } + else if (TableTypeVar* ttv = Luau::getMutable(newTy)) + { + LUAU_ASSERT(ttv->state == TableState::Free || ttv->state == TableState::Generic); + ttv->level = newLevel; + } + else if (FunctionTypeVar* ftv = Luau::getMutable(newTy)) + { + ftv->level = newLevel; + } + + return newTy; +} + +PendingTypePack* TxnLog::changeLevel(TypePackId tp, TypeLevel newLevel) +{ + LUAU_ASSERT(FFlag::LuauUseCommittingTxnLog); + LUAU_ASSERT(get(tp)); + + PendingTypePack* newTp = queue(tp); + if (FreeTypePack* ftp = Luau::getMutable(newTp)) + { + ftp->level = newLevel; + } + + return newTp; +} + +PendingType* TxnLog::changeIndexer(TypeId ty, std::optional indexer) +{ + LUAU_ASSERT(FFlag::LuauUseCommittingTxnLog); + LUAU_ASSERT(get(ty)); + + PendingType* newTy = queue(ty); + if (TableTypeVar* ttv = Luau::getMutable(newTy)) + { + ttv->indexer = indexer; + } + + return newTy; +} + +std::optional TxnLog::getLevel(TypeId ty) const +{ + LUAU_ASSERT(FFlag::LuauUseCommittingTxnLog); + + if (FreeTypeVar* ftv = getMutable(ty)) + return ftv->level; + else if (TableTypeVar* ttv = getMutable(ty); ttv && (ttv->state == TableState::Free || ttv->state == TableState::Generic)) + return ttv->level; + else if (FunctionTypeVar* ftv = getMutable(ty)) + return ftv->level; + + return std::nullopt; +} + +TypeId TxnLog::follow(TypeId ty) +{ + LUAU_ASSERT(FFlag::LuauUseCommittingTxnLog); + + return Luau::follow(ty, [this](TypeId ty) { + PendingType* state = this->pending(ty); + + if (state == nullptr) + return ty; + + // Ugly: Fabricate a TypeId that doesn't adhere to most of the invariants + // that normally apply. This is safe because follow will only call get<> + // on the returned pointer. + return const_cast(&state->pending); + }); +} + +TypePackId TxnLog::follow(TypePackId tp) const +{ + LUAU_ASSERT(FFlag::LuauUseCommittingTxnLog); + + return Luau::follow(tp, [this](TypePackId tp) { + PendingTypePack* state = this->pending(tp); + + if (state == nullptr) + return tp; + + // Ugly: Fabricate a TypePackId that doesn't adhere to most of the + // invariants that normally apply. This is safe because follow will + // only call get<> on the returned pointer. + return const_cast(&state->pending); + }); +} + } // namespace Luau diff --git a/Analysis/src/TypeInfer.cpp b/Analysis/src/TypeInfer.cpp index e29b6ec6..1689a5c3 100644 --- a/Analysis/src/TypeInfer.cpp +++ b/Analysis/src/TypeInfer.cpp @@ -27,15 +27,16 @@ LUAU_FASTFLAGVARIABLE(LuauEqConstraint, false) LUAU_FASTFLAGVARIABLE(LuauWeakEqConstraint, false) // Eventually removed as false. LUAU_FASTFLAGVARIABLE(LuauCloneCorrectlyBeforeMutatingTableType, false) LUAU_FASTFLAGVARIABLE(LuauStoreMatchingOverloadFnType, false) +LUAU_FASTFLAG(LuauUseCommittingTxnLog) +LUAU_FASTFLAGVARIABLE(DebugLuauFreezeDuringUnification, false) LUAU_FASTFLAGVARIABLE(LuauRecursiveTypeParameterRestriction, false) LUAU_FASTFLAGVARIABLE(LuauIfElseExpressionAnalysisSupport, false) LUAU_FASTFLAGVARIABLE(LuauQuantifyInPlace2, false) +LUAU_FASTFLAGVARIABLE(LuauSealExports, false) LUAU_FASTFLAGVARIABLE(LuauSingletonTypes, false) LUAU_FASTFLAGVARIABLE(LuauExpectedTypesOfProperties, false) LUAU_FASTFLAGVARIABLE(LuauErrorRecoveryType, false) LUAU_FASTFLAGVARIABLE(LuauPropertiesGetExpectedType, false) -LUAU_FASTFLAGVARIABLE(LuauTailArgumentTypeInfo, false) -LUAU_FASTFLAGVARIABLE(LuauModuleRequireErrorPack, false) LUAU_FASTFLAGVARIABLE(LuauLValueAsKey, false) LUAU_FASTFLAGVARIABLE(LuauRefiLookupFromIndexExpr, false) LUAU_FASTFLAGVARIABLE(LuauProperTypeLevels, false) @@ -450,7 +451,7 @@ void TypeChecker::checkBlock(const ScopePtr& scope, const AstStatBlock& block) ++subLevel; TypeId leftType = checkFunctionName(scope, *fun->name, funScope->level); - unify(leftType, funTy, fun->location); + unify(funTy, leftType, fun->location); } else if (auto fun = (*protoIter)->as()) { @@ -556,21 +557,21 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatIf& statement) } } -ErrorVec TypeChecker::canUnify(TypeId left, TypeId right, const Location& location) -{ - return canUnify_(left, right, location); -} - -ErrorVec TypeChecker::canUnify(TypePackId left, TypePackId right, const Location& location) -{ - return canUnify_(left, right, location); -} - template -ErrorVec TypeChecker::canUnify_(Id superTy, Id subTy, const Location& location) +ErrorVec TypeChecker::canUnify_(Id subTy, Id superTy, const Location& location) { Unifier state = mkUnifier(location); - return state.canUnify(superTy, subTy); + return state.canUnify(subTy, superTy); +} + +ErrorVec TypeChecker::canUnify(TypeId subTy, TypeId superTy, const Location& location) +{ + return canUnify_(subTy, superTy, location); +} + +ErrorVec TypeChecker::canUnify(TypePackId subTy, TypePackId superTy, const Location& location) +{ + return canUnify_(subTy, superTy, location); } void TypeChecker::check(const ScopePtr& scope, const AstStatWhile& statement) @@ -619,7 +620,7 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatReturn& return_) // start typechecking everything across module boundaries. if (isNonstrictMode() && follow(scope->returnType) == follow(currentModule->getModuleScope()->returnType)) { - ErrorVec errors = tryUnify(scope->returnType, retPack, return_.location); + ErrorVec errors = tryUnify(retPack, scope->returnType, return_.location); if (!errors.empty()) currentModule->getModuleScope()->returnType = addTypePack({anyType}); @@ -627,31 +628,41 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatReturn& return_) return; } - unify(scope->returnType, retPack, return_.location, CountMismatch::Context::Return); -} - -ErrorVec TypeChecker::tryUnify(TypeId left, TypeId right, const Location& location) -{ - return tryUnify_(left, right, location); -} - -ErrorVec TypeChecker::tryUnify(TypePackId left, TypePackId right, const Location& location) -{ - return tryUnify_(left, right, location); + unify(retPack, scope->returnType, return_.location, CountMismatch::Context::Return); } template -ErrorVec TypeChecker::tryUnify_(Id left, Id right, const Location& location) +ErrorVec TypeChecker::tryUnify_(Id subTy, Id superTy, const Location& location) { Unifier state = mkUnifier(location); - state.tryUnify(left, right); - if (!state.errors.empty()) - state.log.rollback(); + if (FFlag::LuauUseCommittingTxnLog && FFlag::DebugLuauFreezeDuringUnification) + freeze(currentModule->internalTypes); + + state.tryUnify(subTy, superTy); + + if (FFlag::LuauUseCommittingTxnLog && FFlag::DebugLuauFreezeDuringUnification) + unfreeze(currentModule->internalTypes); + + if (!state.errors.empty() && !FFlag::LuauUseCommittingTxnLog) + state.DEPRECATED_log.rollback(); + + if (state.errors.empty() && FFlag::LuauUseCommittingTxnLog) + state.log.commit(); return state.errors; } +ErrorVec TypeChecker::tryUnify(TypeId subTy, TypeId superTy, const Location& location) +{ + return tryUnify_(subTy, superTy, location); +} + +ErrorVec TypeChecker::tryUnify(TypePackId subTy, TypePackId superTy, const Location& location) +{ + return tryUnify_(subTy, superTy, location); +} + void TypeChecker::check(const ScopePtr& scope, const AstStatAssign& assign) { std::vector> expectedTypes; @@ -743,9 +754,9 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatAssign& assign) { // In nonstrict mode, any assignments where the lhs is free and rhs isn't a function, we give it any typevar. if (isNonstrictMode() && get(follow(left)) && !get(follow(right))) - unify(left, anyType, loc); + unify(anyType, left, loc); else - unify(left, right, loc); + unify(right, left, loc); } } } @@ -760,7 +771,7 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatCompoundAssign& assi TypeId result = checkBinaryOperation(scope, expr, left, right); - unify(left, result, assign.location); + unify(result, left, assign.location); } void TypeChecker::check(const ScopePtr& scope, const AstStatLocal& local) @@ -817,9 +828,12 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatLocal& local) Unifier state = mkUnifier(local.location); state.ctx = CountMismatch::Result; - state.tryUnify(variablePack, valuePack); + state.tryUnify(valuePack, variablePack); reportErrors(state.errors); + if (FFlag::LuauUseCommittingTxnLog) + state.log.commit(); + // In the code 'local T = {}', we wish to ascribe the name 'T' to the type of the table for error-reporting purposes. // We also want to do this for 'local T = setmetatable(...)'. if (local.vars.size == 1 && local.values.size == 1) @@ -889,7 +903,7 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatFor& expr) TypeId loopVarType = numberType; if (expr.var->annotation) - unify(resolveType(scope, *expr.var->annotation), loopVarType, expr.location); + unify(loopVarType, resolveType(scope, *expr.var->annotation), expr.location); loopScope->bindings[expr.var] = {loopVarType, expr.var->location}; @@ -899,11 +913,11 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatFor& expr) if (!expr.to) ice("Bad AstStatFor has no to expr"); - unify(loopVarType, checkExpr(loopScope, *expr.from).type, expr.from->location); - unify(loopVarType, checkExpr(loopScope, *expr.to).type, expr.to->location); + unify(checkExpr(loopScope, *expr.from).type, loopVarType, expr.from->location); + unify(checkExpr(loopScope, *expr.to).type, loopVarType, expr.to->location); if (expr.step) - unify(loopVarType, checkExpr(loopScope, *expr.step).type, expr.step->location); + unify(checkExpr(loopScope, *expr.step).type, loopVarType, expr.step->location); check(loopScope, *expr.body); } @@ -956,12 +970,12 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatForIn& forin) if (get(callRetPack)) { iterTy = freshType(scope); - unify(addTypePack({{iterTy}, freshTypePack(scope)}), callRetPack, forin.location); + unify(callRetPack, addTypePack({{iterTy}, freshTypePack(scope)}), forin.location); } else if (get(callRetPack) || !first(callRetPack)) { for (TypeId var : varTypes) - unify(var, errorRecoveryType(scope), forin.location); + unify(errorRecoveryType(scope), var, forin.location); return check(loopScope, *forin.body); } @@ -982,7 +996,7 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatForIn& forin) TypeId varTy = get(iterTy) ? anyType : errorRecoveryType(loopScope); for (TypeId var : varTypes) - unify(var, varTy, forin.location); + unify(varTy, var, forin.location); if (!get(iterTy) && !get(iterTy) && !get(iterTy)) reportError(TypeError{firstValue->location, TypeMismatch{globalScope->bindings[AstName{"next"}].typeId, iterTy}}); @@ -1010,6 +1024,9 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatForIn& forin) Unifier state = mkUnifier(firstValue->location); checkArgumentList(loopScope, state, argPack, iterFunc->argTypes, /*argLocations*/ {}); + if (FFlag::LuauUseCommittingTxnLog) + state.log.commit(); + reportErrors(state.errors); } @@ -1024,10 +1041,10 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatForIn& forin) AstExprCall exprCall{Location(start, end), firstValue, arguments, /* self= */ false, Location()}; TypePackId retPack = checkExprPack(scope, exprCall).type; - unify(varPack, retPack, forin.location); + unify(retPack, varPack, forin.location); } else - unify(varPack, iterFunc->retType, forin.location); + unify(iterFunc->retType, varPack, forin.location); check(loopScope, *forin.body); } @@ -1112,7 +1129,7 @@ void TypeChecker::check(const ScopePtr& scope, TypeId ty, const ScopePtr& funSco checkFunctionBody(funScope, ty, *function.func); - unify(leftType, ty, function.location); + unify(ty, leftType, function.location); if (FFlag::LuauUpdateFunctionNameBinding) { @@ -1242,7 +1259,7 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatTypeAlias& typealias else if (auto mtv = getMutable(follow(ty))) mtv->syntheticName = name; - unify(bindingsMap[name].type, ty, typealias.location); + unify(ty, bindingsMap[name].type, typealias.location); } } @@ -1526,7 +1543,7 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprCa { TypeId head = freshType(scope); TypePackId pack = addTypePack(TypePackVar{TypePack{{head}, freshTypePack(scope)}}); - unify(retPack, pack, expr.location); + unify(pack, retPack, expr.location); return {head, std::move(result.predicates)}; } if (get(retPack)) @@ -1598,7 +1615,7 @@ std::optional TypeChecker::getIndexTypeFromType( return it->second.type; else if (auto indexer = tableType->indexer) { - tryUnify(indexer->indexType, stringType, location); + tryUnify(stringType, indexer->indexType, location); return indexer->indexResultType; } else if (tableType->state == TableState::Free) @@ -1824,7 +1841,7 @@ TypeId TypeChecker::checkExprTable( indexer = expectedTable->indexer; if (indexer) - unify(indexer->indexResultType, valueType, value->location); + unify(valueType, indexer->indexResultType, value->location); else indexer = TableIndexer{numberType, anyIfNonstrict(valueType)}; } @@ -1842,13 +1859,13 @@ TypeId TypeChecker::checkExprTable( if (it != expectedTable->props.end()) { Property expectedProp = it->second; - ErrorVec errors = tryUnify(expectedProp.type, exprType, k->location); + ErrorVec errors = tryUnify(exprType, expectedProp.type, k->location); if (errors.empty()) exprType = expectedProp.type; } else if (expectedTable->indexer && isString(expectedTable->indexer->indexType)) { - ErrorVec errors = tryUnify(expectedTable->indexer->indexResultType, exprType, k->location); + ErrorVec errors = tryUnify(exprType, expectedTable->indexer->indexResultType, k->location); if (errors.empty()) exprType = expectedTable->indexer->indexResultType; } @@ -1863,8 +1880,8 @@ TypeId TypeChecker::checkExprTable( if (indexer) { - unify(indexer->indexType, keyType, k->location); - unify(indexer->indexResultType, valueType, value->location); + unify(keyType, indexer->indexType, k->location); + unify(valueType, indexer->indexResultType, value->location); } else if (isNonstrictMode()) { @@ -1992,7 +2009,10 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprUn TypeId expectedFunctionType = addType(FunctionTypeVar(scope->level, arguments, retTypePack)); Unifier state = mkUnifier(expr.location); - state.tryUnify(expectedFunctionType, actualFunctionType, /*isFunctionCall*/ true); + state.tryUnify(actualFunctionType, expectedFunctionType, /*isFunctionCall*/ true); + + if (FFlag::LuauUseCommittingTxnLog) + state.log.commit(); TypeId retType = first(retTypePack).value_or(nilType); if (!state.errors.empty()) @@ -2006,7 +2026,7 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprUn return {errorRecoveryType(scope)}; } - reportErrors(tryUnify(numberType, operandType, expr.location)); + reportErrors(tryUnify(operandType, numberType, expr.location)); return {numberType}; } case AstExprUnary::Len: @@ -2072,7 +2092,7 @@ TypeId TypeChecker::unionOfTypes(TypeId a, TypeId b, const Location& location, b { if (unifyFreeTypes && (get(a) || get(b))) { - if (unify(a, b, location)) + if (unify(b, a, location)) return a; return errorRecoveryType(anyType); @@ -2175,7 +2195,13 @@ TypeId TypeChecker::checkRelationalOperation( */ Unifier state = mkUnifier(expr.location); if (!isEquality) - state.tryUnify(lhsType, rhsType); + { + state.tryUnify(rhsType, lhsType); + + if (FFlag::LuauUseCommittingTxnLog) + state.log.commit(); + } + bool needsMetamethod = !isEquality; @@ -2216,13 +2242,16 @@ TypeId TypeChecker::checkRelationalOperation( if (isEquality) { Unifier state = mkUnifier(expr.location); - state.tryUnify(ftv->retType, addTypePack({booleanType})); + state.tryUnify(addTypePack({booleanType}), ftv->retType); if (!state.errors.empty()) { reportError(expr.location, GenericError{format("Metamethod '%s' must return type 'boolean'", metamethodName.c_str())}); return errorRecoveryType(booleanType); } + + if (FFlag::LuauUseCommittingTxnLog) + state.log.commit(); } } @@ -2230,7 +2259,10 @@ TypeId TypeChecker::checkRelationalOperation( TypeId actualFunctionType = addType(FunctionTypeVar(scope->level, addTypePack({lhsType, rhsType}), addTypePack({booleanType}))); state.tryUnify( - instantiate(scope, *metamethod, expr.location), instantiate(scope, actualFunctionType, expr.location), /*isFunctionCall*/ true); + instantiate(scope, actualFunctionType, expr.location), instantiate(scope, *metamethod, expr.location), /*isFunctionCall*/ true); + + if (FFlag::LuauUseCommittingTxnLog) + state.log.commit(); reportErrors(state.errors); return booleanType; @@ -2323,7 +2355,7 @@ TypeId TypeChecker::checkBinaryOperation( } if (get(rhsType)) - unify(lhsType, rhsType, expr.location); + unify(rhsType, lhsType, expr.location); if (typeCouldHaveMetatable(lhsType) || typeCouldHaveMetatable(rhsType)) { @@ -2334,7 +2366,7 @@ TypeId TypeChecker::checkBinaryOperation( TypeId expectedFunctionType = addType(FunctionTypeVar(scope->level, arguments, retTypePack)); Unifier state = mkUnifier(expr.location); - state.tryUnify(expectedFunctionType, actualFunctionType, /*isFunctionCall*/ true); + state.tryUnify(actualFunctionType, expectedFunctionType, /*isFunctionCall*/ true); reportErrors(state.errors); bool hasErrors = !state.errors.empty(); @@ -2345,11 +2377,28 @@ TypeId TypeChecker::checkBinaryOperation( // so we loosen the argument types to see if that helps. TypePackId fallbackArguments = freshTypePack(scope); TypeId fallbackFunctionType = addType(FunctionTypeVar(scope->level, fallbackArguments, retTypePack)); - state.log.rollback(); state.errors.clear(); - state.tryUnify(fallbackFunctionType, actualFunctionType, /*isFunctionCall*/ true); - if (!state.errors.empty()) - state.log.rollback(); + + if (FFlag::LuauUseCommittingTxnLog) + { + state.log.clear(); + } + else + { + state.DEPRECATED_log.rollback(); + } + + state.tryUnify(actualFunctionType, fallbackFunctionType, /*isFunctionCall*/ true); + + if (FFlag::LuauUseCommittingTxnLog && state.errors.empty()) + state.log.commit(); + else if (!state.errors.empty() && !FFlag::LuauUseCommittingTxnLog) + state.DEPRECATED_log.rollback(); + } + + if (FFlag::LuauUseCommittingTxnLog && !hasErrors) + { + state.log.commit(); } TypeId retType = first(retTypePack).value_or(nilType); @@ -2377,8 +2426,8 @@ TypeId TypeChecker::checkBinaryOperation( switch (expr.op) { case AstExprBinary::Concat: - reportErrors(tryUnify(addType(UnionTypeVar{{stringType, numberType}}), lhsType, expr.left->location)); - reportErrors(tryUnify(addType(UnionTypeVar{{stringType, numberType}}), rhsType, expr.right->location)); + reportErrors(tryUnify(lhsType, addType(UnionTypeVar{{stringType, numberType}}), expr.left->location)); + reportErrors(tryUnify(rhsType, addType(UnionTypeVar{{stringType, numberType}}), expr.right->location)); return stringType; case AstExprBinary::Add: case AstExprBinary::Sub: @@ -2386,8 +2435,8 @@ TypeId TypeChecker::checkBinaryOperation( case AstExprBinary::Div: case AstExprBinary::Mod: case AstExprBinary::Pow: - reportErrors(tryUnify(numberType, lhsType, expr.left->location)); - reportErrors(tryUnify(numberType, rhsType, expr.right->location)); + reportErrors(tryUnify(lhsType, numberType, expr.left->location)); + reportErrors(tryUnify(rhsType, numberType, expr.right->location)); return numberType; default: // These should have been handled with checkRelationalOperation @@ -2466,10 +2515,10 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprTy if (FFlag::LuauBidirectionalAsExpr) { // Note: As an optimization, we try 'number <: number | string' first, as that is the more likely case. - if (canUnify(result.type, annotationType, expr.location).empty()) + if (canUnify(annotationType, result.type, expr.location).empty()) return {annotationType, std::move(result.predicates)}; - if (canUnify(annotationType, result.type, expr.location).empty()) + if (canUnify(result.type, annotationType, expr.location).empty()) return {annotationType, std::move(result.predicates)}; reportError(expr.location, TypesAreUnrelated{result.type, annotationType}); @@ -2477,7 +2526,7 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprTy } else { - ErrorVec errorVec = canUnify(result.type, annotationType, expr.location); + ErrorVec errorVec = canUnify(annotationType, result.type, expr.location); reportErrors(errorVec); if (!errorVec.empty()) annotationType = errorRecoveryType(annotationType); @@ -2512,7 +2561,7 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprIf resolve(result.predicates, falseScope, false); ExprResult falseType = checkExpr(falseScope, *expr.falseExpr); - unify(trueType.type, falseType.type, expr.location); + unify(falseType.type, trueType.type, expr.location); // TODO: normalize(UnionTypeVar{{trueType, falseType}}) // For now both trueType and falseType must be the same type. @@ -2607,14 +2656,18 @@ std::pair TypeChecker::checkLValueBinding(const ScopePtr& scope else if (auto indexer = lhsTable->indexer) { Unifier state = mkUnifier(expr.location); - state.tryUnify(indexer->indexType, stringType); + state.tryUnify(stringType, indexer->indexType); TypeId retType = indexer->indexResultType; if (!state.errors.empty()) { - state.log.rollback(); + if (!FFlag::LuauUseCommittingTxnLog) + state.DEPRECATED_log.rollback(); + reportError(expr.location, UnknownProperty{lhs, name}); retType = errorRecoveryType(retType); } + else if (FFlag::LuauUseCommittingTxnLog) + state.log.commit(); return std::pair(retType, nullptr); } @@ -2713,7 +2766,7 @@ std::pair TypeChecker::checkLValueBinding(const ScopePtr& scope if (exprTable->indexer) { const TableIndexer& indexer = *exprTable->indexer; - unify(indexer.indexType, indexType, expr.index->location); + unify(indexType, indexer.indexType, expr.index->location); return std::pair(indexer.indexResultType, nullptr); } else if (exprTable->state == TableState::Unsealed || exprTable->state == TableState::Free) @@ -3106,204 +3159,402 @@ void TypeChecker::checkArgumentList( * A function requires parameters. * To call a function, you supply arguments. */ - TypePackIterator argIter = begin(argPack); - TypePackIterator paramIter = begin(paramPack); + TypePackIterator argIter = begin(argPack, &state.log); + TypePackIterator paramIter = begin(paramPack, &state.log); TypePackIterator endIter = end(argPack); // Important subtlety: All end TypePackIterators are equivalent size_t paramIndex = 0; size_t minParams = getMinParameterCount(paramPack); - while (true) + if (FFlag::LuauUseCommittingTxnLog) { - state.location = paramIndex < argLocations.size() ? argLocations[paramIndex] : state.location; - - if (argIter == endIter && paramIter == endIter) + while (true) { - std::optional argTail = argIter.tail(); - std::optional paramTail = paramIter.tail(); + state.location = paramIndex < argLocations.size() ? argLocations[paramIndex] : state.location; - // If we hit the end of both type packs simultaneously, then there are definitely no further type - // errors to report. All we need to do is tie up any free tails. - // - // If one side has a free tail and the other has none at all, we create an empty pack and bind the - // free tail to that. - - if (argTail) + if (argIter == endIter && paramIter == endIter) { - if (get(*argTail)) + std::optional argTail = argIter.tail(); + std::optional paramTail = paramIter.tail(); + + // If we hit the end of both type packs simultaneously, then there are definitely no further type + // errors to report. All we need to do is tie up any free tails. + // + // If one side has a free tail and the other has none at all, we create an empty pack and bind the + // free tail to that. + + if (argTail) { - if (paramTail) - state.tryUnify(*argTail, *paramTail); + if (state.log.getMutable(state.log.follow(*argTail))) + { + if (paramTail) + state.tryUnify(*paramTail, *argTail); + else + state.log.replace(*argTail, TypePackVar(TypePack{{}})); + } + } + else if (paramTail) + { + // argTail is definitely empty + if (state.log.getMutable(state.log.follow(*paramTail))) + state.log.replace(*paramTail, TypePackVar(TypePack{{}})); + } + + return; + } + else if (argIter == endIter) + { + // Not enough arguments. + + // Might be ok if we are forwarding a vararg along. This is a common thing to occur in nonstrict mode. + if (argIter.tail()) + { + TypePackId tail = *argIter.tail(); + if (state.log.getMutable(tail)) + { + // Unify remaining parameters so we don't leave any free-types hanging around. + while (paramIter != endIter) + { + state.tryUnify(errorRecoveryType(anyType), *paramIter); + ++paramIter; + } + return; + } + else if (auto vtp = state.log.getMutable(tail)) + { + while (paramIter != endIter) + { + state.tryUnify(vtp->ty, *paramIter); + ++paramIter; + } + + return; + } + else if (state.log.getMutable(tail)) + { + std::vector rest; + rest.reserve(std::distance(paramIter, endIter)); + while (paramIter != endIter) + { + rest.push_back(*paramIter); + ++paramIter; + } + + TypePackId varPack = addTypePack(TypePackVar{TypePack{rest, paramIter.tail()}}); + state.tryUnify(varPack, tail); + return; + } + } + + // If any remaining unfulfilled parameters are nonoptional, this is a problem. + while (paramIter != endIter) + { + TypeId t = state.log.follow(*paramIter); + if (isOptional(t)) + { + } // ok + else if (state.log.getMutable(t)) + { + } // ok + else if (isNonstrictMode() && state.log.getMutable(t)) + { + } // ok else { - state.log(*argTail); - *asMutable(*argTail) = TypePack{{}}; + state.errors.push_back(TypeError{state.location, CountMismatch{minParams, paramIndex}}); + return; + } + ++paramIter; + } + } + else if (paramIter == endIter) + { + // too many parameters passed + if (!paramIter.tail()) + { + while (argIter != endIter) + { + // The use of unify here is deliberate. We don't want this unification + // to be undoable. + unify(errorRecoveryType(scope), *argIter, state.location); + ++argIter; + } + // For this case, we want the error span to cover every errant extra parameter + Location location = state.location; + if (!argLocations.empty()) + location = {state.location.begin, argLocations.back().end}; + state.errors.push_back(TypeError{location, CountMismatch{minParams, std::distance(begin(argPack), end(argPack))}}); + return; + } + TypePackId tail = state.log.follow(*paramIter.tail()); + + if (state.log.getMutable(tail)) + { + // Function is variadic. Ok. + return; + } + else if (auto vtp = state.log.getMutable(tail)) + { + // Function is variadic and requires that all subsequent parameters + // be compatible with a type. + size_t argIndex = paramIndex; + while (argIter != endIter) + { + Location location = state.location; + + if (argIndex < argLocations.size()) + location = argLocations[argIndex]; + + unify(*argIter, vtp->ty, location); + ++argIter; + ++argIndex; + } + + return; + } + else if (state.log.getMutable(tail)) + { + // Create a type pack out of the remaining argument types + // and unify it with the tail. + std::vector rest; + rest.reserve(std::distance(argIter, endIter)); + while (argIter != endIter) + { + rest.push_back(*argIter); + ++argIter; + } + + TypePackId varPack = addTypePack(TypePackVar{TypePack{rest, argIter.tail()}}); + state.tryUnify(varPack, tail); + return; + } + else if (state.log.getMutable(tail)) + { + state.log.replace(tail, TypePackVar(TypePack{{}})); + return; + } + else if (state.log.getMutable(tail)) + { + // For this case, we want the error span to cover every errant extra parameter + Location location = state.location; + if (!argLocations.empty()) + location = {state.location.begin, argLocations.back().end}; + // TODO: Better error message? + state.errors.push_back(TypeError{location, CountMismatch{minParams, std::distance(begin(argPack), end(argPack))}}); + return; + } + } + else + { + unifyWithInstantiationIfNeeded(scope, *argIter, *paramIter, state); + ++argIter; + ++paramIter; + } + + ++paramIndex; + } + } + else + { + while (true) + { + state.location = paramIndex < argLocations.size() ? argLocations[paramIndex] : state.location; + + if (argIter == endIter && paramIter == endIter) + { + std::optional argTail = argIter.tail(); + std::optional paramTail = paramIter.tail(); + + // If we hit the end of both type packs simultaneously, then there are definitely no further type + // errors to report. All we need to do is tie up any free tails. + // + // If one side has a free tail and the other has none at all, we create an empty pack and bind the + // free tail to that. + + if (argTail) + { + if (get(*argTail)) + { + if (paramTail) + state.tryUnify(*paramTail, *argTail); + else + { + state.DEPRECATED_log(*argTail); + *asMutable(*argTail) = TypePack{{}}; + } } } - } - else if (paramTail) - { - // argTail is definitely empty - if (get(*paramTail)) + else if (paramTail) { - state.log(*paramTail); - *asMutable(*paramTail) = TypePack{{}}; + // argTail is definitely empty + if (get(*paramTail)) + { + state.DEPRECATED_log(*paramTail); + *asMutable(*paramTail) = TypePack{{}}; + } + } + + return; + } + else if (argIter == endIter) + { + // Not enough arguments. + + // Might be ok if we are forwarding a vararg along. This is a common thing to occur in nonstrict mode. + if (argIter.tail()) + { + TypePackId tail = *argIter.tail(); + if (get(tail)) + { + // Unify remaining parameters so we don't leave any free-types hanging around. + while (paramIter != endIter) + { + state.tryUnify(*paramIter, errorRecoveryType(anyType)); + ++paramIter; + } + return; + } + else if (auto vtp = get(tail)) + { + while (paramIter != endIter) + { + state.tryUnify(*paramIter, vtp->ty); + ++paramIter; + } + + return; + } + else if (get(tail)) + { + std::vector rest; + rest.reserve(std::distance(paramIter, endIter)); + while (paramIter != endIter) + { + rest.push_back(*paramIter); + ++paramIter; + } + + TypePackId varPack = addTypePack(TypePackVar{TypePack{rest, paramIter.tail()}}); + state.tryUnify(varPack, tail); + return; + } + } + + // If any remaining unfulfilled parameters are nonoptional, this is a problem. + while (paramIter != endIter) + { + TypeId t = follow(*paramIter); + if (isOptional(t)) + { + } // ok + else if (get(t)) + { + } // ok + else if (isNonstrictMode() && get(t)) + { + } // ok + else + { + state.errors.push_back(TypeError{state.location, CountMismatch{minParams, paramIndex}}); + return; + } + ++paramIter; } } - - return; - } - else if (argIter == endIter) - { - // Not enough arguments. - - // Might be ok if we are forwarding a vararg along. This is a common thing to occur in nonstrict mode. - if (argIter.tail()) + else if (paramIter == endIter) { - TypePackId tail = *argIter.tail(); + // too many parameters passed + if (!paramIter.tail()) + { + while (argIter != endIter) + { + unify(*argIter, errorRecoveryType(scope), state.location); + ++argIter; + } + // For this case, we want the error span to cover every errant extra parameter + Location location = state.location; + if (!argLocations.empty()) + location = {state.location.begin, argLocations.back().end}; + state.errors.push_back(TypeError{location, CountMismatch{minParams, std::distance(begin(argPack), end(argPack))}}); + return; + } + TypePackId tail = *paramIter.tail(); + if (get(tail)) { - // Unify remaining parameters so we don't leave any free-types hanging around. - while (paramIter != endIter) - { - state.tryUnify(*paramIter, errorRecoveryType(anyType)); - ++paramIter; - } + // Function is variadic. Ok. return; } else if (auto vtp = get(tail)) { - while (paramIter != endIter) + // Function is variadic and requires that all subsequent parameters + // be compatible with a type. + size_t argIndex = paramIndex; + while (argIter != endIter) { - state.tryUnify(*paramIter, vtp->ty); - ++paramIter; + Location location = state.location; + + if (argIndex < argLocations.size()) + location = argLocations[argIndex]; + + unify(*argIter, vtp->ty, location); + ++argIter; + ++argIndex; } return; } else if (get(tail)) { + // Create a type pack out of the remaining argument types + // and unify it with the tail. std::vector rest; - rest.reserve(std::distance(paramIter, endIter)); - while (paramIter != endIter) + rest.reserve(std::distance(argIter, endIter)); + while (argIter != endIter) { - rest.push_back(*paramIter); - ++paramIter; + rest.push_back(*argIter); + ++argIter; } - TypePackId varPack = addTypePack(TypePackVar{TypePack{rest, paramIter.tail()}}); + TypePackId varPack = addTypePack(TypePackVar{TypePack{rest, argIter.tail()}}); state.tryUnify(tail, varPack); return; } - } + else if (get(tail)) + { + if (FFlag::LuauUseCommittingTxnLog) + { + state.log.replace(tail, TypePackVar(TypePack{{}})); + } + else + { + state.DEPRECATED_log(tail); + *asMutable(tail) = TypePack{}; + } - // If any remaining unfulfilled parameters are nonoptional, this is a problem. - while (paramIter != endIter) - { - TypeId t = follow(*paramIter); - if (isOptional(t)) - { - } // ok - else if (get(t)) - { - } // ok - else if (isNonstrictMode() && get(t)) - { - } // ok - else - { - state.errors.push_back(TypeError{state.location, CountMismatch{minParams, paramIndex}}); return; } + else if (get(tail)) + { + // For this case, we want the error span to cover every errant extra parameter + Location location = state.location; + if (!argLocations.empty()) + location = {state.location.begin, argLocations.back().end}; + // TODO: Better error message? + state.errors.push_back(TypeError{location, CountMismatch{minParams, std::distance(begin(argPack), end(argPack))}}); + return; + } + } + else + { + unifyWithInstantiationIfNeeded(scope, *argIter, *paramIter, state); + ++argIter; ++paramIter; } + + ++paramIndex; } - else if (paramIter == endIter) - { - // too many parameters passed - if (!paramIter.tail()) - { - while (argIter != endIter) - { - unify(*argIter, errorRecoveryType(scope), state.location); - ++argIter; - } - // For this case, we want the error span to cover every errant extra parameter - Location location = state.location; - if (!argLocations.empty()) - location = {state.location.begin, argLocations.back().end}; - state.errors.push_back(TypeError{location, CountMismatch{minParams, std::distance(begin(argPack), end(argPack))}}); - return; - } - TypePackId tail = *paramIter.tail(); - - if (get(tail)) - { - // Function is variadic. Ok. - return; - } - else if (auto vtp = get(tail)) - { - // Function is variadic and requires that all subsequent parameters - // be compatible with a type. - size_t argIndex = paramIndex; - while (argIter != endIter) - { - Location location = state.location; - - if (argIndex < argLocations.size()) - location = argLocations[argIndex]; - - unify(vtp->ty, *argIter, location); - ++argIter; - ++argIndex; - } - - return; - } - else if (get(tail)) - { - // Create a type pack out of the remaining argument types - // and unify it with the tail. - std::vector rest; - rest.reserve(std::distance(argIter, endIter)); - while (argIter != endIter) - { - rest.push_back(*argIter); - ++argIter; - } - - TypePackId varPack = addTypePack(TypePackVar{TypePack{rest, argIter.tail()}}); - state.tryUnify(tail, varPack); - return; - } - else if (get(tail)) - { - state.log(tail); - *asMutable(tail) = TypePack{}; - - return; - } - else if (get(tail)) - { - // For this case, we want the error span to cover every errant extra parameter - Location location = state.location; - if (!argLocations.empty()) - location = {state.location.begin, argLocations.back().end}; - // TODO: Better error message? - state.errors.push_back(TypeError{location, CountMismatch{minParams, std::distance(begin(argPack), end(argPack))}}); - return; - } - } - else - { - unifyWithInstantiationIfNeeded(scope, *paramIter, *argIter, state); - ++argIter; - ++paramIter; - } - - ++paramIndex; } } @@ -3475,7 +3726,7 @@ std::optional> TypeChecker::checkCallOverload(const Scope if (get(fn)) { - unify(argPack, anyTypePack, expr.location); + unify(anyTypePack, argPack, expr.location); return {{anyTypePack}}; } @@ -3490,7 +3741,7 @@ std::optional> TypeChecker::checkCallOverload(const Scope // has been instantiated, so is a monotype. We can therefore // unify it with a monomorphic function. TypeId r = addType(FunctionTypeVar(scope->level, argPack, retPack)); - unify(r, fn, expr.location); + unify(fn, r, expr.location); return {{retPack}}; } @@ -3533,7 +3784,7 @@ std::optional> TypeChecker::checkCallOverload(const Scope if (!ftv) { reportError(TypeError{expr.func->location, CannotCallNonFunction{fn}}); - unify(retPack, errorRecoveryTypePack(scope), expr.func->location); + unify(errorRecoveryTypePack(scope), retPack, expr.func->location); return {{errorRecoveryTypePack(retPack)}}; } @@ -3552,7 +3803,9 @@ std::optional> TypeChecker::checkCallOverload(const Scope checkArgumentList(scope, state, retPack, ftv->retType, /*argLocations*/ {}); if (!state.errors.empty()) { - state.log.rollback(); + if (!FFlag::LuauUseCommittingTxnLog) + state.DEPRECATED_log.rollback(); + return {}; } @@ -3580,10 +3833,15 @@ std::optional> TypeChecker::checkCallOverload(const Scope overloadsThatDont.push_back(fn); errors.emplace_back(std::move(state.errors), args->head, ftv); - state.log.rollback(); + + if (!FFlag::LuauUseCommittingTxnLog) + state.DEPRECATED_log.rollback(); } else { + if (FFlag::LuauUseCommittingTxnLog) + state.log.commit(); + if (isNonstrictMode() && !expr.self && expr.func->is() && ftv->hasSelf) { // If we are running in nonstrict mode, passing fewer arguments than the function is declared to take AND @@ -3640,6 +3898,9 @@ bool TypeChecker::handleSelfCallMismatch(const ScopePtr& scope, const AstExprCal if (editedState.errors.empty()) { + if (FFlag::LuauUseCommittingTxnLog) + editedState.log.commit(); + reportError(TypeError{expr.location, FunctionDoesNotTakeSelf{}}); // This is a little bit suspect: If this overload would work with a . replaced by a : // we eagerly assume that that's what you actually meant and we commit to it. @@ -3648,8 +3909,8 @@ bool TypeChecker::handleSelfCallMismatch(const ScopePtr& scope, const AstExprCal // checkArgumentList(scope, editedState, retPack, ftv->retType, retLocations, CountMismatch::Return); return true; } - else - editedState.log.rollback(); + else if (!FFlag::LuauUseCommittingTxnLog) + editedState.DEPRECATED_log.rollback(); } else if (ftv->hasSelf) { @@ -3671,6 +3932,9 @@ bool TypeChecker::handleSelfCallMismatch(const ScopePtr& scope, const AstExprCal if (editedState.errors.empty()) { + if (FFlag::LuauUseCommittingTxnLog) + editedState.log.commit(); + reportError(TypeError{expr.location, FunctionRequiresSelf{}}); // This is a little bit suspect: If this overload would work with a : replaced by a . // we eagerly assume that that's what you actually meant and we commit to it. @@ -3679,8 +3943,8 @@ bool TypeChecker::handleSelfCallMismatch(const ScopePtr& scope, const AstExprCal // checkArgumentList(scope, editedState, retPack, ftv->retType, retLocations, CountMismatch::Return); return true; } - else - editedState.log.rollback(); + else if (!FFlag::LuauUseCommittingTxnLog) + editedState.DEPRECATED_log.rollback(); } } } @@ -3740,6 +4004,9 @@ void TypeChecker::reportOverloadResolutionError(const ScopePtr& scope, const Ast checkArgumentList(scope, state, argPack, ftv->argTypes, argLocations); } + if (FFlag::LuauUseCommittingTxnLog && state.errors.empty()) + state.log.commit(); + if (i > 0) s += "; "; @@ -3748,7 +4015,8 @@ void TypeChecker::reportOverloadResolutionError(const ScopePtr& scope, const Ast s += toString(overload); - state.log.rollback(); + if (!FFlag::LuauUseCommittingTxnLog) + state.DEPRECATED_log.rollback(); } if (overloadsThatMatchArgCount.size() == 0) @@ -3781,6 +4049,8 @@ ExprResult TypeChecker::checkExprList(const ScopePtr& scope, const L Unifier state = mkUnifier(location); + std::vector inverseLogs; + for (size_t i = 0; i < exprs.size; ++i) { AstExpr* expr = exprs.data[i]; @@ -3791,18 +4061,15 @@ ExprResult TypeChecker::checkExprList(const ScopePtr& scope, const L auto [typePack, exprPredicates] = checkExprPack(scope, *expr); insert(exprPredicates); - if (FFlag::LuauTailArgumentTypeInfo) + if (std::optional firstTy = first(typePack)) { - if (std::optional firstTy = first(typePack)) - { - if (!currentModule->astTypes.find(expr)) - currentModule->astTypes[expr] = follow(*firstTy); - } - - if (expectedType) - currentModule->astExpectedTypes[expr] = *expectedType; + if (!currentModule->astTypes.find(expr)) + currentModule->astTypes[expr] = follow(*firstTy); } + if (expectedType) + currentModule->astExpectedTypes[expr] = *expectedType; + tp->tail = typePack; } else @@ -3816,13 +4083,31 @@ ExprResult TypeChecker::checkExprList(const ScopePtr& scope, const L actualType = instantiate(scope, actualType, expr->location); if (expectedType) - state.tryUnify(*expectedType, actualType); + { + state.tryUnify(actualType, *expectedType); + + // Ugly: In future iterations of the loop, we might need the state of the unification we + // just performed. There's not a great way to pass that into checkExpr. Instead, we store + // the inverse of the current log, and commit it. When we're done, we'll commit all the + // inverses. This isn't optimal, and a better solution is welcome here. + if (FFlag::LuauUseCommittingTxnLog) + { + inverseLogs.push_back(state.log.inverse()); + state.log.commit(); + } + } tp->head.push_back(actualType); } } - state.log.rollback(); + if (FFlag::LuauUseCommittingTxnLog) + { + for (TxnLog& log : inverseLogs) + log.commit(); + } + else + state.DEPRECATED_log.rollback(); return {pack, predicates}; } @@ -3884,7 +4169,7 @@ TypeId TypeChecker::checkRequire(const ScopePtr& scope, const ModuleInfo& module TypePackId modulePack = module->getModuleScope()->returnType; - if (FFlag::LuauModuleRequireErrorPack && get(modulePack)) + if (get(modulePack)) return errorRecoveryType(scope); std::optional moduleType = first(modulePack); @@ -3917,72 +4202,94 @@ TypeId TypeChecker::anyIfNonstrict(TypeId ty) const return ty; } -bool TypeChecker::unify(TypeId left, TypeId right, const Location& location) +bool TypeChecker::unify(TypeId subTy, TypeId superTy, const Location& location) { Unifier state = mkUnifier(location); - state.tryUnify(left, right); + state.tryUnify(subTy, superTy); + + if (FFlag::LuauUseCommittingTxnLog) + state.log.commit(); reportErrors(state.errors); return state.errors.empty(); } -bool TypeChecker::unify(TypePackId left, TypePackId right, const Location& location, CountMismatch::Context ctx) +bool TypeChecker::unify(TypePackId subTy, TypePackId superTy, const Location& location, CountMismatch::Context ctx) { Unifier state = mkUnifier(location); state.ctx = ctx; - state.tryUnify(left, right); + state.tryUnify(subTy, superTy); + + if (FFlag::LuauUseCommittingTxnLog) + state.log.commit(); reportErrors(state.errors); return state.errors.empty(); } -bool TypeChecker::unifyWithInstantiationIfNeeded(const ScopePtr& scope, TypeId left, TypeId right, const Location& location) +bool TypeChecker::unifyWithInstantiationIfNeeded(const ScopePtr& scope, TypeId subTy, TypeId superTy, const Location& location) { Unifier state = mkUnifier(location); - unifyWithInstantiationIfNeeded(scope, left, right, state); + unifyWithInstantiationIfNeeded(scope, subTy, superTy, state); + + if (FFlag::LuauUseCommittingTxnLog) + state.log.commit(); reportErrors(state.errors); return state.errors.empty(); } -void TypeChecker::unifyWithInstantiationIfNeeded(const ScopePtr& scope, TypeId left, TypeId right, Unifier& state) +void TypeChecker::unifyWithInstantiationIfNeeded(const ScopePtr& scope, TypeId subTy, TypeId superTy, Unifier& state) { - if (!maybeGeneric(right)) + if (!maybeGeneric(subTy)) // Quick check to see if we definitely can't instantiate - state.tryUnify(left, right, /*isFunctionCall*/ false); - else if (!maybeGeneric(left) && isGeneric(right)) + state.tryUnify(subTy, superTy, /*isFunctionCall*/ false); + else if (!maybeGeneric(superTy) && isGeneric(subTy)) { // Quick check to see if we definitely have to instantiate - TypeId instantiated = instantiate(scope, right, state.location); - state.tryUnify(left, instantiated, /*isFunctionCall*/ false); + TypeId instantiated = instantiate(scope, subTy, state.location); + state.tryUnify(instantiated, superTy, /*isFunctionCall*/ false); } else { // First try unifying with the original uninstantiated type // but if that fails, try the instantiated one. Unifier child = state.makeChildUnifier(); - child.tryUnify(left, right, /*isFunctionCall*/ false); + child.tryUnify(subTy, superTy, /*isFunctionCall*/ false); if (!child.errors.empty()) { - TypeId instantiated = instantiate(scope, right, state.location); - if (right == instantiated) + TypeId instantiated = instantiate(scope, subTy, state.location); + if (subTy == instantiated) { // Instantiating the argument made no difference, so just report any child errors - state.log.concat(std::move(child.log)); + if (FFlag::LuauUseCommittingTxnLog) + state.log.concat(std::move(child.log)); + else + state.DEPRECATED_log.concat(std::move(child.DEPRECATED_log)); + state.errors.insert(state.errors.end(), child.errors.begin(), child.errors.end()); } else { - child.log.rollback(); - state.tryUnify(left, instantiated, /*isFunctionCall*/ false); + if (!FFlag::LuauUseCommittingTxnLog) + child.DEPRECATED_log.rollback(); + + state.tryUnify(instantiated, superTy, /*isFunctionCall*/ false); } } else { - state.log.concat(std::move(child.log)); + if (FFlag::LuauUseCommittingTxnLog) + { + state.log.concat(std::move(child.log)); + } + else + { + state.DEPRECATED_log.concat(std::move(child.DEPRECATED_log)); + } } } } @@ -4139,7 +4446,7 @@ TypePackId Quantification::clean(TypePackId tp) bool Anyification::isDirty(TypeId ty) { if (const TableTypeVar* ttv = get(ty)) - return (ttv->state == TableState::Free); + return (ttv->state == TableState::Free || (FFlag::LuauSealExports && ttv->state == TableState::Unsealed)); else if (get(ty)) return true; else @@ -4162,6 +4469,12 @@ TypeId Anyification::clean(TypeId ty) TableTypeVar clone = TableTypeVar{ttv->props, ttv->indexer, ttv->level, TableState::Sealed}; clone.methodDefinitionLocations = ttv->methodDefinitionLocations; clone.definitionModuleName = ttv->definitionModuleName; + if (FFlag::LuauSealExports) + { + clone.name = ttv->name; + clone.syntheticName = ttv->syntheticName; + clone.tags = ttv->tags; + } return addType(std::move(clone)); } else @@ -5194,8 +5507,8 @@ void TypeChecker::resolve(const IsAPredicate& isaP, ErrorVec& errVec, Refinement // This by itself is not truly enough to determine that A is stronger than B or vice versa. // The best unambiguous way about this would be to have a function that returns the relationship ordering of a pair. // i.e. TypeRelationship relationshipOf(TypeId superTy, TypeId subTy) - bool optionIsSubtype = canUnify(isaP.ty, option, isaP.location).empty(); - bool targetIsSubtype = canUnify(option, isaP.ty, isaP.location).empty(); + bool optionIsSubtype = canUnify(option, isaP.ty, isaP.location).empty(); + bool targetIsSubtype = canUnify(isaP.ty, option, isaP.location).empty(); // If A is a superset of B, then if sense is true, we promote A to B, otherwise we keep A. if (!optionIsSubtype && targetIsSubtype) @@ -5379,7 +5692,7 @@ void TypeChecker::resolve(const EqPredicate& eqP, ErrorVec& errVec, RefinementMa for (TypeId right : rhs) { // When singleton types arrive, `isNil` here probably should be replaced with `isLiteral`. - if (canUnify(left, right, eqP.location).empty() == sense || (!sense && !isNil(left))) + if (canUnify(right, left, eqP.location).empty() == sense || (!sense && !isNil(left))) set.insert(left); } } @@ -5406,7 +5719,7 @@ std::vector TypeChecker::unTypePack(const ScopePtr& scope, TypePackId tp for (size_t i = 0; i < expectedLength; ++i) expectedPack->head.push_back(freshType(scope)); - unify(expectedTypePack, tp, location); + unify(tp, expectedTypePack, location); for (TypeId& tp : expectedPack->head) tp = follow(tp); diff --git a/Analysis/src/TypePack.cpp b/Analysis/src/TypePack.cpp index d3221c73..b15548a8 100644 --- a/Analysis/src/TypePack.cpp +++ b/Analysis/src/TypePack.cpp @@ -1,8 +1,12 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #include "Luau/TypePack.h" +#include "Luau/TxnLog.h" + #include +LUAU_FASTFLAG(LuauUseCommittingTxnLog) + namespace Luau { @@ -35,14 +39,28 @@ TypePackVar& TypePackVar::operator=(TypePackVariant&& tp) } TypePackIterator::TypePackIterator(TypePackId typePack) + : TypePackIterator(typePack, TxnLog::empty()) +{ +} + +TypePackIterator::TypePackIterator(TypePackId typePack, const TxnLog* log) : currentTypePack(follow(typePack)) , tp(get(currentTypePack)) , currentIndex(0) + , log(log) { while (tp && tp->head.empty()) { - currentTypePack = tp->tail ? follow(*tp->tail) : nullptr; - tp = currentTypePack ? get(currentTypePack) : nullptr; + if (FFlag::LuauUseCommittingTxnLog) + { + currentTypePack = tp->tail ? log->follow(*tp->tail) : nullptr; + tp = currentTypePack ? log->getMutable(currentTypePack) : nullptr; + } + else + { + currentTypePack = tp->tail ? follow(*tp->tail) : nullptr; + tp = currentTypePack ? get(currentTypePack) : nullptr; + } } } @@ -53,8 +71,17 @@ TypePackIterator& TypePackIterator::operator++() ++currentIndex; while (tp && currentIndex >= tp->head.size()) { - currentTypePack = tp->tail ? follow(*tp->tail) : nullptr; - tp = currentTypePack ? get(currentTypePack) : nullptr; + if (FFlag::LuauUseCommittingTxnLog) + { + currentTypePack = tp->tail ? log->follow(*tp->tail) : nullptr; + tp = currentTypePack ? log->getMutable(currentTypePack) : nullptr; + } + else + { + currentTypePack = tp->tail ? follow(*tp->tail) : nullptr; + tp = currentTypePack ? get(currentTypePack) : nullptr; + } + currentIndex = 0; } @@ -95,6 +122,11 @@ TypePackIterator begin(TypePackId tp) return TypePackIterator{tp}; } +TypePackIterator begin(TypePackId tp, TxnLog* log) +{ + return TypePackIterator{tp, log}; +} + TypePackIterator end(TypePackId tp) { return TypePackIterator{}; @@ -160,8 +192,15 @@ bool areEqual(SeenSet& seen, const TypePackVar& lhs, const TypePackVar& rhs) TypePackId follow(TypePackId tp) { - auto advance = [](TypePackId ty) -> std::optional { - if (const Unifiable::Bound* btv = get>(ty)) + return follow(tp, [](TypePackId t) { + return t; + }); +} + +TypePackId follow(TypePackId tp, std::function mapper) +{ + auto advance = [&mapper](TypePackId ty) -> std::optional { + if (const Unifiable::Bound* btv = get>(mapper(ty))) return btv->boundTo; else return std::nullopt; diff --git a/Analysis/src/TypeVar.cpp b/Analysis/src/TypeVar.cpp index fb75aa02..4cab79c8 100644 --- a/Analysis/src/TypeVar.cpp +++ b/Analysis/src/TypeVar.cpp @@ -31,17 +31,24 @@ std::optional> magicFunctionFormat( TypeId follow(TypeId t) { - auto advance = [](TypeId ty) -> std::optional { - if (auto btv = get>(ty)) + return follow(t, [](TypeId t) { + return t; + }); +} + +TypeId follow(TypeId t, std::function mapper) +{ + auto advance = [&mapper](TypeId ty) -> std::optional { + if (auto btv = get>(mapper(ty))) return btv->boundTo; - else if (auto ttv = get(ty)) + else if (auto ttv = get(mapper(ty))) return ttv->boundTo; else return std::nullopt; }; - auto force = [](TypeId ty) { - if (auto ltv = get_if(&ty->ty)) + auto force = [&mapper](TypeId ty) { + if (auto ltv = get_if(&mapper(ty)->ty)) { TypeId res = ltv->thunk(); if (get(res)) @@ -1004,7 +1011,7 @@ std::optional> magicFunctionFormat( { Location location = expr.args.data[std::min(i + dataOffset, expr.args.size - 1)]->location; - typechecker.unify(expected[i], params[i + paramOffset], location); + typechecker.unify(params[i + paramOffset], expected[i], location); } // if we know the argument count or if we have too many arguments for sure, we can issue an error diff --git a/Analysis/src/TypedAllocator.cpp b/Analysis/src/TypedAllocator.cpp index f037351e..1f7ef8c2 100644 --- a/Analysis/src/TypedAllocator.cpp +++ b/Analysis/src/TypedAllocator.cpp @@ -20,6 +20,7 @@ const size_t kPageSize = sysconf(_SC_PAGESIZE); #include LUAU_FASTFLAG(DebugLuauFreezeArena) +LUAU_FASTFLAGVARIABLE(LuauTypedAllocatorZeroStart, false) namespace Luau { diff --git a/Analysis/src/Unifier.cpp b/Analysis/src/Unifier.cpp index 43ea37e7..393a84a7 100644 --- a/Analysis/src/Unifier.cpp +++ b/Analysis/src/Unifier.cpp @@ -13,6 +13,7 @@ LUAU_FASTINT(LuauTypeInferRecursionLimit); LUAU_FASTINT(LuauTypeInferTypePackLoopLimit); +LUAU_FASTFLAG(LuauUseCommittingTxnLog) LUAU_FASTINTVARIABLE(LuauTypeInferIterationLimit, 2000); LUAU_FASTFLAGVARIABLE(LuauTableSubtypingVariance2, false); LUAU_FASTFLAGVARIABLE(LuauUnionHeuristic, false) @@ -29,27 +30,39 @@ namespace Luau struct PromoteTypeLevels { + DEPRECATED_TxnLog& DEPRECATED_log; TxnLog& log; TypeLevel minLevel; - explicit PromoteTypeLevels(TxnLog& log, TypeLevel minLevel) - : log(log) + explicit PromoteTypeLevels(DEPRECATED_TxnLog& DEPRECATED_log, TxnLog& log, TypeLevel minLevel) + : DEPRECATED_log(DEPRECATED_log) + , log(log) , minLevel(minLevel) - {} + { + } - template + template void promote(TID ty, T* t) { LUAU_ASSERT(t); if (minLevel.subsumesStrict(t->level)) { - log(ty); - t->level = minLevel; + if (FFlag::LuauUseCommittingTxnLog) + { + log.changeLevel(ty, minLevel); + } + else + { + DEPRECATED_log(ty); + t->level = minLevel; + } } } template - void cycle(TID) {} + void cycle(TID) + { + } template bool operator()(TID, const T&) @@ -59,39 +72,47 @@ struct PromoteTypeLevels bool operator()(TypeId ty, const FreeTypeVar&) { - promote(ty, getMutable(ty)); + // Surprise, it's actually a BoundTypeVar that hasn't been committed yet. + // Calling getMutable on this will trigger an assertion. + if (FFlag::LuauUseCommittingTxnLog && !log.is(ty)) + return true; + + promote(ty, FFlag::LuauUseCommittingTxnLog ? log.getMutable(ty) : getMutable(ty)); return true; } bool operator()(TypeId ty, const FunctionTypeVar&) { - promote(ty, getMutable(ty)); + promote(ty, FFlag::LuauUseCommittingTxnLog ? log.getMutable(ty) : getMutable(ty)); return true; } - bool operator()(TypeId ty, const TableTypeVar&) + bool operator()(TypeId ty, const TableTypeVar& ttv) { - promote(ty, getMutable(ty)); + if (ttv.state != TableState::Free && ttv.state != TableState::Generic) + return true; + + promote(ty, FFlag::LuauUseCommittingTxnLog ? log.getMutable(ty) : getMutable(ty)); return true; } bool operator()(TypePackId tp, const FreeTypePack&) { - promote(tp, getMutable(tp)); + promote(tp, FFlag::LuauUseCommittingTxnLog ? log.getMutable(tp) : getMutable(tp)); return true; } }; -void promoteTypeLevels(TxnLog& log, TypeLevel minLevel, TypeId ty) +void promoteTypeLevels(DEPRECATED_TxnLog& DEPRECATED_log, TxnLog& log, TypeLevel minLevel, TypeId ty) { - PromoteTypeLevels ptl{log, minLevel}; + PromoteTypeLevels ptl{DEPRECATED_log, log, minLevel}; DenseHashSet seen{nullptr}; visitTypeVarOnce(ty, ptl, seen); } -void promoteTypeLevels(TxnLog& log, TypeLevel minLevel, TypePackId tp) +void promoteTypeLevels(DEPRECATED_TxnLog& DEPRECATED_log, TxnLog& log, TypeLevel minLevel, TypePackId tp) { - PromoteTypeLevels ptl{log, minLevel}; + PromoteTypeLevels ptl{DEPRECATED_log, log, minLevel}; DenseHashSet seen{nullptr}; visitTypeVarOnce(tp, ptl, seen); } @@ -221,10 +242,12 @@ static std::optional> getTableMat return std::nullopt; } -Unifier::Unifier(TypeArena* types, Mode mode, ScopePtr globalScope, const Location& location, Variance variance, UnifierSharedState& sharedState) +Unifier::Unifier(TypeArena* types, Mode mode, ScopePtr globalScope, const Location& location, Variance variance, UnifierSharedState& sharedState, + TxnLog* parentLog) : types(types) , mode(mode) , globalScope(std::move(globalScope)) + , log(parentLog) , location(location) , variance(variance) , sharedState(sharedState) @@ -233,11 +256,12 @@ Unifier::Unifier(TypeArena* types, Mode mode, ScopePtr globalScope, const Locati } Unifier::Unifier(TypeArena* types, Mode mode, ScopePtr globalScope, std::vector>* sharedSeen, const Location& location, - Variance variance, UnifierSharedState& sharedState) + Variance variance, UnifierSharedState& sharedState, TxnLog* parentLog) : types(types) , mode(mode) , globalScope(std::move(globalScope)) - , log(sharedSeen) + , DEPRECATED_log(sharedSeen) + , log(parentLog, sharedSeen) , location(location) , variance(variance) , sharedState(sharedState) @@ -245,14 +269,14 @@ Unifier::Unifier(TypeArena* types, Mode mode, ScopePtr globalScope, std::vector< LUAU_ASSERT(sharedState.iceHandler); } -void Unifier::tryUnify(TypeId superTy, TypeId subTy, bool isFunctionCall, bool isIntersection) +void Unifier::tryUnify(TypeId subTy, TypeId superTy, bool isFunctionCall, bool isIntersection) { sharedState.counters.iterationCount = 0; - tryUnify_(superTy, subTy, isFunctionCall, isIntersection); + tryUnify_(subTy, superTy, isFunctionCall, isIntersection); } -void Unifier::tryUnify_(TypeId superTy, TypeId subTy, bool isFunctionCall, bool isIntersection) +void Unifier::tryUnify_(TypeId subTy, TypeId superTy, bool isFunctionCall, bool isIntersection) { RecursionLimiter _ra(&sharedState.counters.recursionCount, FInt::LuauTypeInferRecursionLimit); @@ -264,55 +288,112 @@ void Unifier::tryUnify_(TypeId superTy, TypeId subTy, bool isFunctionCall, bool return; } - superTy = follow(superTy); - subTy = follow(subTy); + if (FFlag::LuauUseCommittingTxnLog) + { + superTy = log.follow(superTy); + subTy = log.follow(subTy); + } + else + { + superTy = follow(superTy); + subTy = follow(subTy); + } if (superTy == subTy) return; - auto l = getMutable(superTy); - auto r = getMutable(subTy); + auto superFree = getMutable(superTy); + auto subFree = getMutable(subTy); - if (l && r && l->level.subsumes(r->level)) + if (FFlag::LuauUseCommittingTxnLog) + { + superFree = log.getMutable(superTy); + subFree = log.getMutable(subTy); + } + + if (superFree && subFree && superFree->level.subsumes(subFree->level)) { occursCheck(subTy, superTy); // The occurrence check might have caused superTy no longer to be a free type - if (!get(subTy)) + bool occursFailed = false; + if (FFlag::LuauUseCommittingTxnLog) + occursFailed = bool(log.getMutable(subTy)); + else + occursFailed = bool(get(subTy)); + + if (!occursFailed) { - log(subTy); - *asMutable(subTy) = BoundTypeVar(superTy); + if (FFlag::LuauUseCommittingTxnLog) + { + log.replace(subTy, BoundTypeVar(superTy)); + } + else + { + DEPRECATED_log(subTy); + *asMutable(subTy) = BoundTypeVar(superTy); + } } return; } - else if (l && r) + else if (superFree && subFree) { - if (!FFlag::LuauErrorRecoveryType) - log(superTy); - occursCheck(superTy, subTy); - r->level = min(r->level, l->level); - - // The occurrence check might have caused superTy no longer to be a free type - if (!FFlag::LuauErrorRecoveryType) - *asMutable(superTy) = BoundTypeVar(subTy); - else if (!get(superTy)) + if (!FFlag::LuauErrorRecoveryType && !FFlag::LuauUseCommittingTxnLog) + { + DEPRECATED_log(superTy); + subFree->level = min(subFree->level, superFree->level); + } + + occursCheck(superTy, subTy); + + bool occursFailed = false; + if (FFlag::LuauUseCommittingTxnLog) + occursFailed = bool(log.getMutable(superTy)); + else + occursFailed = bool(get(superTy)); + + if (!FFlag::LuauErrorRecoveryType && !FFlag::LuauUseCommittingTxnLog) { - log(superTy); *asMutable(superTy) = BoundTypeVar(subTy); + return; + } + + if (!occursFailed) + { + if (FFlag::LuauUseCommittingTxnLog) + { + if (superFree->level.subsumes(subFree->level)) + { + log.changeLevel(subTy, superFree->level); + } + + log.replace(superTy, BoundTypeVar(subTy)); + } + else + { + DEPRECATED_log(superTy); + *asMutable(superTy) = BoundTypeVar(subTy); + subFree->level = min(subFree->level, superFree->level); + } } return; } - else if (l) + else if (superFree) { occursCheck(superTy, subTy); + bool occursFailed = false; + if (FFlag::LuauUseCommittingTxnLog) + occursFailed = bool(log.getMutable(superTy)); + else + occursFailed = bool(get(superTy)); - TypeLevel superLevel = l->level; + TypeLevel superLevel = superFree->level; // Unification can't change the level of a generic. - auto rightGeneric = get(subTy); - if (rightGeneric && !rightGeneric->level.subsumes(superLevel)) + auto subGeneric = FFlag::LuauUseCommittingTxnLog ? log.getMutable(subTy) : get(subTy); + if (subGeneric && !subGeneric->level.subsumes(superLevel)) { // TODO: a more informative error message? CLI-39912 errors.push_back(TypeError{location, GenericError{"Generic subtype escaping scope"}}); @@ -320,63 +401,83 @@ void Unifier::tryUnify_(TypeId superTy, TypeId subTy, bool isFunctionCall, bool } // The occurrence check might have caused superTy no longer to be a free type - if (!get(superTy)) + if (!occursFailed) { - if (FFlag::LuauProperTypeLevels) - promoteTypeLevels(log, superLevel, subTy); - else if (auto rightLevel = getMutableLevel(subTy)) + if (FFlag::LuauUseCommittingTxnLog) { - if (!rightLevel->subsumes(l->level)) - *rightLevel = l->level; + promoteTypeLevels(DEPRECATED_log, log, superLevel, subTy); + log.replace(superTy, BoundTypeVar(subTy)); } + else + { + if (FFlag::LuauProperTypeLevels) + promoteTypeLevels(DEPRECATED_log, log, superLevel, subTy); + else if (auto subLevel = getMutableLevel(subTy)) + { + if (!subLevel->subsumes(superFree->level)) + *subLevel = superFree->level; + } - log(superTy); - *asMutable(superTy) = BoundTypeVar(subTy); + DEPRECATED_log(superTy); + *asMutable(superTy) = BoundTypeVar(subTy); + } } return; } - else if (r) + else if (subFree) { - TypeLevel subLevel = r->level; + TypeLevel subLevel = subFree->level; occursCheck(subTy, superTy); + bool occursFailed = false; + if (FFlag::LuauUseCommittingTxnLog) + occursFailed = bool(log.getMutable(subTy)); + else + occursFailed = bool(get(subTy)); // Unification can't change the level of a generic. - auto leftGeneric = get(superTy); - if (leftGeneric && !leftGeneric->level.subsumes(r->level)) + auto superGeneric = FFlag::LuauUseCommittingTxnLog ? log.getMutable(superTy) : get(superTy); + if (superGeneric && !superGeneric->level.subsumes(subFree->level)) { // TODO: a more informative error message? CLI-39912 errors.push_back(TypeError{location, GenericError{"Generic supertype escaping scope"}}); return; } - if (!get(subTy)) + if (!occursFailed) { - if (FFlag::LuauProperTypeLevels) - promoteTypeLevels(log, subLevel, superTy); - - if (auto superLevel = getMutableLevel(superTy)) + if (FFlag::LuauUseCommittingTxnLog) { - if (!superLevel->subsumes(r->level)) - { - log(superTy); - *superLevel = r->level; - } + promoteTypeLevels(DEPRECATED_log, log, subLevel, superTy); + log.replace(subTy, BoundTypeVar(superTy)); } + else + { + if (FFlag::LuauProperTypeLevels) + promoteTypeLevels(DEPRECATED_log, log, subLevel, superTy); + else if (auto superLevel = getMutableLevel(superTy)) + { + if (!superLevel->subsumes(subFree->level)) + { + DEPRECATED_log(superTy); + *superLevel = subFree->level; + } + } - log(subTy); - *asMutable(subTy) = BoundTypeVar(superTy); + DEPRECATED_log(subTy); + *asMutable(subTy) = BoundTypeVar(superTy); + } } return; } if (get(superTy) || get(superTy)) - return tryUnifyWithAny(superTy, subTy); + return tryUnifyWithAny(subTy, superTy); if (get(subTy) || get(subTy)) - return tryUnifyWithAny(subTy, superTy); + return tryUnifyWithAny(superTy, subTy); bool cacheEnabled = !isFunctionCall && !isIntersection; auto& cache = sharedState.cachedUnify; @@ -389,12 +490,22 @@ void Unifier::tryUnify_(TypeId superTy, TypeId subTy, bool isFunctionCall, bool // Here, we assume that the types unify. If they do not, we will find out as we roll back // the stack. - if (log.haveSeen(superTy, subTy)) - return; + if (FFlag::LuauUseCommittingTxnLog) + { + if (log.haveSeen(superTy, subTy)) + return; - log.pushSeen(superTy, subTy); + log.pushSeen(superTy, subTy); + } + else + { + if (DEPRECATED_log.haveSeen(superTy, subTy)) + return; - if (const UnionTypeVar* uv = get(subTy)) + DEPRECATED_log.pushSeen(superTy, subTy); + } + + if (const UnionTypeVar* uv = FFlag::LuauUseCommittingTxnLog ? log.getMutable(subTy) : get(subTy)) { // A | B <: T if A <: T and B <: T bool failed = false; @@ -407,7 +518,7 @@ void Unifier::tryUnify_(TypeId superTy, TypeId subTy, bool isFunctionCall, bool for (TypeId type : uv->options) { Unifier innerState = makeChildUnifier(); - innerState.tryUnify_(superTy, type); + innerState.tryUnify_(type, superTy); if (auto e = hasUnificationTooComplex(innerState.errors)) unificationTooComplex = e; @@ -420,10 +531,24 @@ void Unifier::tryUnify_(TypeId superTy, TypeId subTy, bool isFunctionCall, bool failed = true; } - if (i != count - 1) - innerState.log.rollback(); + if (FFlag::LuauUseCommittingTxnLog) + { + if (i == count - 1) + { + log.concat(std::move(innerState.log)); + } + } else - log.concat(std::move(innerState.log)); + { + if (i != count - 1) + { + innerState.DEPRECATED_log.rollback(); + } + else + { + DEPRECATED_log.concat(std::move(innerState.DEPRECATED_log)); + } + } ++i; } @@ -438,7 +563,7 @@ void Unifier::tryUnify_(TypeId superTy, TypeId subTy, bool isFunctionCall, bool errors.push_back(TypeError{location, TypeMismatch{superTy, subTy}}); } } - else if (const UnionTypeVar* uv = get(superTy)) + else if (const UnionTypeVar* uv = FFlag::LuauUseCommittingTxnLog ? log.getMutable(superTy) : get(superTy)) { // T <: A | B if T <: A or T <: B bool found = false; @@ -502,12 +627,16 @@ void Unifier::tryUnify_(TypeId superTy, TypeId subTy, bool isFunctionCall, bool { TypeId type = uv->options[(i + startIndex) % uv->options.size()]; Unifier innerState = makeChildUnifier(); - innerState.tryUnify_(type, subTy, isFunctionCall); + innerState.tryUnify_(subTy, type, isFunctionCall); if (innerState.errors.empty()) { found = true; - log.concat(std::move(innerState.log)); + if (FFlag::LuauUseCommittingTxnLog) + log.concat(std::move(innerState.log)); + else + DEPRECATED_log.concat(std::move(innerState.DEPRECATED_log)); + break; } else if (auto e = hasUnificationTooComplex(innerState.errors)) @@ -522,7 +651,8 @@ void Unifier::tryUnify_(TypeId superTy, TypeId subTy, bool isFunctionCall, bool failedOption = {innerState.errors.front()}; } - innerState.log.rollback(); + if (!FFlag::LuauUseCommittingTxnLog) + innerState.DEPRECATED_log.rollback(); } if (unificationTooComplex) @@ -538,7 +668,8 @@ void Unifier::tryUnify_(TypeId superTy, TypeId subTy, bool isFunctionCall, bool errors.push_back(TypeError{location, TypeMismatch{superTy, subTy, "none of the union options are compatible"}}); } } - else if (const IntersectionTypeVar* uv = get(superTy)) + else if (const IntersectionTypeVar* uv = + FFlag::LuauUseCommittingTxnLog ? log.getMutable(superTy) : get(superTy)) { std::optional unificationTooComplex; std::optional firstFailedOption; @@ -547,7 +678,7 @@ void Unifier::tryUnify_(TypeId superTy, TypeId subTy, bool isFunctionCall, bool for (TypeId type : uv->parts) { Unifier innerState = makeChildUnifier(); - innerState.tryUnify_(type, subTy, /*isFunctionCall*/ false, /*isIntersection*/ true); + innerState.tryUnify_(subTy, type, /*isFunctionCall*/ false, /*isIntersection*/ true); if (auto e = hasUnificationTooComplex(innerState.errors)) unificationTooComplex = e; @@ -557,7 +688,10 @@ void Unifier::tryUnify_(TypeId superTy, TypeId subTy, bool isFunctionCall, bool firstFailedOption = {innerState.errors.front()}; } - log.concat(std::move(innerState.log)); + if (FFlag::LuauUseCommittingTxnLog) + log.concat(std::move(innerState.log)); + else + DEPRECATED_log.concat(std::move(innerState.DEPRECATED_log)); } if (unificationTooComplex) @@ -565,7 +699,8 @@ void Unifier::tryUnify_(TypeId superTy, TypeId subTy, bool isFunctionCall, bool else if (firstFailedOption) errors.push_back(TypeError{location, TypeMismatch{superTy, subTy, "Not all intersection parts are compatible.", *firstFailedOption}}); } - else if (const IntersectionTypeVar* uv = get(subTy)) + else if (const IntersectionTypeVar* uv = + FFlag::LuauUseCommittingTxnLog ? log.getMutable(subTy) : get(subTy)) { // A & B <: T if T <: A or T <: B bool found = false; @@ -591,12 +726,15 @@ void Unifier::tryUnify_(TypeId superTy, TypeId subTy, bool isFunctionCall, bool { TypeId type = uv->parts[(i + startIndex) % uv->parts.size()]; Unifier innerState = makeChildUnifier(); - innerState.tryUnify_(superTy, type, isFunctionCall); + innerState.tryUnify_(type, superTy, isFunctionCall); if (innerState.errors.empty()) { found = true; - log.concat(std::move(innerState.log)); + if (FFlag::LuauUseCommittingTxnLog) + log.concat(std::move(innerState.log)); + else + DEPRECATED_log.concat(std::move(innerState.DEPRECATED_log)); break; } else if (auto e = hasUnificationTooComplex(innerState.errors)) @@ -604,7 +742,8 @@ void Unifier::tryUnify_(TypeId superTy, TypeId subTy, bool isFunctionCall, bool unificationTooComplex = e; } - innerState.log.rollback(); + if (!FFlag::LuauUseCommittingTxnLog) + innerState.DEPRECATED_log.rollback(); } if (unificationTooComplex) @@ -614,44 +753,56 @@ void Unifier::tryUnify_(TypeId superTy, TypeId subTy, bool isFunctionCall, bool errors.push_back(TypeError{location, TypeMismatch{superTy, subTy, "none of the intersection parts are compatible"}}); } } - else if (get(superTy) && get(subTy)) - tryUnifyPrimitives(superTy, subTy); + else if ((FFlag::LuauUseCommittingTxnLog && log.getMutable(superTy) && log.getMutable(subTy)) || + (!FFlag::LuauUseCommittingTxnLog && get(superTy) && get(subTy))) + tryUnifyPrimitives(subTy, superTy); - else if (FFlag::LuauSingletonTypes && (get(superTy) || get(superTy)) && get(subTy)) - tryUnifySingletons(superTy, subTy); + else if (FFlag::LuauSingletonTypes && + ((FFlag::LuauUseCommittingTxnLog ? log.getMutable(superTy) : get(superTy)) || + (FFlag::LuauUseCommittingTxnLog ? log.getMutable(superTy) : get(superTy))) && + (FFlag::LuauUseCommittingTxnLog ? log.getMutable(subTy) : get(subTy))) + tryUnifySingletons(subTy, superTy); - else if (get(superTy) && get(subTy)) - tryUnifyFunctions(superTy, subTy, isFunctionCall); + else if ((FFlag::LuauUseCommittingTxnLog && log.getMutable(superTy) && log.getMutable(subTy)) || + (!FFlag::LuauUseCommittingTxnLog && get(superTy) && get(subTy))) + tryUnifyFunctions(subTy, superTy, isFunctionCall); - else if (get(superTy) && get(subTy)) + else if ((FFlag::LuauUseCommittingTxnLog && log.getMutable(superTy) && log.getMutable(subTy)) || + (!FFlag::LuauUseCommittingTxnLog && get(superTy) && get(subTy))) { - tryUnifyTables(superTy, subTy, isIntersection); + tryUnifyTables(subTy, superTy, isIntersection); if (cacheEnabled && errors.empty()) - cacheResult(superTy, subTy); + cacheResult(subTy, superTy); } // tryUnifyWithMetatable assumes its first argument is a MetatableTypeVar. The check is otherwise symmetrical. - else if (get(superTy)) - tryUnifyWithMetatable(superTy, subTy, /*reversed*/ false); - else if (get(subTy)) - tryUnifyWithMetatable(subTy, superTy, /*reversed*/ true); + else if ((FFlag::LuauUseCommittingTxnLog && log.getMutable(superTy)) || + (!FFlag::LuauUseCommittingTxnLog && get(superTy))) + tryUnifyWithMetatable(subTy, superTy, /*reversed*/ false); + else if ((FFlag::LuauUseCommittingTxnLog && log.getMutable(subTy)) || + (!FFlag::LuauUseCommittingTxnLog && get(subTy))) + tryUnifyWithMetatable(superTy, subTy, /*reversed*/ true); - else if (get(superTy)) - tryUnifyWithClass(superTy, subTy, /*reversed*/ false); + else if ((FFlag::LuauUseCommittingTxnLog && log.getMutable(superTy)) || + (!FFlag::LuauUseCommittingTxnLog && get(superTy))) + tryUnifyWithClass(subTy, superTy, /*reversed*/ false); // Unification of nonclasses with classes is almost, but not quite symmetrical. // The order in which we perform this test is significant in the case that both types are classes. - else if (get(subTy)) - tryUnifyWithClass(superTy, subTy, /*reversed*/ true); + else if ((FFlag::LuauUseCommittingTxnLog && log.getMutable(subTy)) || (!FFlag::LuauUseCommittingTxnLog && get(subTy))) + tryUnifyWithClass(subTy, superTy, /*reversed*/ true); else errors.push_back(TypeError{location, TypeMismatch{superTy, subTy}}); - log.popSeen(superTy, subTy); + if (FFlag::LuauUseCommittingTxnLog) + log.popSeen(superTy, subTy); + else + DEPRECATED_log.popSeen(superTy, subTy); } -void Unifier::cacheResult(TypeId superTy, TypeId subTy) +void Unifier::cacheResult(TypeId subTy, TypeId superTy) { bool* superTyInfo = sharedState.skipCacheForType.find(superTy); @@ -684,7 +835,7 @@ void Unifier::cacheResult(TypeId superTy, TypeId subTy) sharedState.cachedUnify.insert({subTy, superTy}); } -struct WeirdIter +struct DEPRECATED_WeirdIter { TypePackId packId; const TypePack* pack; @@ -692,7 +843,7 @@ struct WeirdIter bool growing; TypeLevel level; - WeirdIter(TypePackId packId) + DEPRECATED_WeirdIter(TypePackId packId) : packId(packId) , pack(get(packId)) , index(0) @@ -705,7 +856,7 @@ struct WeirdIter } } - WeirdIter(const WeirdIter&) = default; + DEPRECATED_WeirdIter(const DEPRECATED_WeirdIter&) = default; const TypeId& operator*() { @@ -756,34 +907,152 @@ struct WeirdIter } }; -ErrorVec Unifier::canUnify(TypeId superTy, TypeId subTy) +struct WeirdIter +{ + TypePackId packId; + TxnLog& log; + TypePack* pack; + size_t index; + bool growing; + TypeLevel level; + + WeirdIter(TypePackId packId, TxnLog& log) + : packId(packId) + , log(log) + , pack(log.getMutable(packId)) + , index(0) + , growing(false) + { + while (pack && pack->head.empty() && pack->tail) + { + packId = *pack->tail; + pack = log.getMutable(packId); + } + } + + WeirdIter(const WeirdIter&) = default; + + TypeId& operator*() + { + LUAU_ASSERT(good()); + return pack->head[index]; + } + + bool good() const + { + return pack != nullptr && index < pack->head.size(); + } + + bool advance() + { + if (!pack) + return good(); + + if (index < pack->head.size()) + ++index; + + if (growing || index < pack->head.size()) + return good(); + + if (pack->tail) + { + packId = log.follow(*pack->tail); + pack = log.getMutable(packId); + index = 0; + } + + return good(); + } + + bool canGrow() const + { + return nullptr != log.getMutable(packId); + } + + void grow(TypePackId newTail) + { + LUAU_ASSERT(canGrow()); + LUAU_ASSERT(log.getMutable(newTail)); + + level = log.getMutable(packId)->level; + log.replace(packId, Unifiable::Bound(newTail)); + packId = newTail; + pack = log.getMutable(newTail); + index = 0; + growing = true; + } + + void pushType(TypeId ty) + { + LUAU_ASSERT(pack); + PendingTypePack* pendingPack = log.queue(packId); + if (TypePack* pending = getMutable(pendingPack)) + { + pending->head.push_back(ty); + // We've potentially just replaced the TypePack* that we need to look + // in. We need to replace pack. + pack = pending; + } + else + { + LUAU_ASSERT(!"Pending state for this pack was not a TypePack"); + } + } +}; + +ErrorVec Unifier::canUnify(TypeId subTy, TypeId superTy) { Unifier s = makeChildUnifier(); - s.tryUnify_(superTy, subTy); - s.log.rollback(); + s.tryUnify_(subTy, superTy); + + if (!FFlag::LuauUseCommittingTxnLog) + s.DEPRECATED_log.rollback(); + return s.errors; } -ErrorVec Unifier::canUnify(TypePackId superTy, TypePackId subTy, bool isFunctionCall) +ErrorVec Unifier::canUnify(TypePackId subTy, TypePackId superTy, bool isFunctionCall) { Unifier s = makeChildUnifier(); - s.tryUnify_(superTy, subTy, isFunctionCall); - s.log.rollback(); + s.tryUnify_(subTy, superTy, isFunctionCall); + + if (!FFlag::LuauUseCommittingTxnLog) + s.DEPRECATED_log.rollback(); + return s.errors; } -void Unifier::tryUnify(TypePackId superTp, TypePackId subTp, bool isFunctionCall) +void Unifier::tryUnify(TypePackId subTp, TypePackId superTp, bool isFunctionCall) { sharedState.counters.iterationCount = 0; - tryUnify_(superTp, subTp, isFunctionCall); + tryUnify_(subTp, superTp, isFunctionCall); +} + +static std::pair, std::optional> logAwareFlatten(TypePackId tp, const TxnLog& log) +{ + tp = log.follow(tp); + + std::vector flattened; + std::optional tail = std::nullopt; + + TypePackIterator it(tp, &log); + + for (; it != end(tp); ++it) + { + flattened.push_back(*it); + } + + tail = it.tail(); + + return {flattened, tail}; } /* * This is quite tricky: we are walking two rope-like structures and unifying corresponding elements. * If one is longer than the other, but the short end is free, we grow it to the required length. */ -void Unifier::tryUnify_(TypePackId superTp, TypePackId subTp, bool isFunctionCall) +void Unifier::tryUnify_(TypePackId subTp, TypePackId superTp, bool isFunctionCall) { RecursionLimiter _ra(&sharedState.counters.recursionCount, FInt::LuauTypeInferRecursionLimit); @@ -795,252 +1064,458 @@ void Unifier::tryUnify_(TypePackId superTp, TypePackId subTp, bool isFunctionCal return; } - superTp = follow(superTp); - subTp = follow(subTp); - - while (auto r = get(subTp)) + if (FFlag::LuauUseCommittingTxnLog) { - if (r->head.empty() && r->tail) - subTp = follow(*r->tail); - else - break; - } + superTp = log.follow(superTp); + subTp = log.follow(subTp); - while (auto l = get(superTp)) - { - if (l->head.empty() && l->tail) - superTp = follow(*l->tail); - else - break; - } - - if (superTp == subTp) - return; - - if (get(superTp)) - { - occursCheck(superTp, subTp); - - // The occurrence check might have caused superTp no longer to be a free type - if (!get(superTp)) + while (auto tp = log.getMutable(subTp)) { - log(superTp); - *asMutable(superTp) = Unifiable::Bound(subTp); - } - } - else if (get(subTp)) - { - occursCheck(subTp, superTp); - - // The occurrence check might have caused superTp no longer to be a free type - if (!get(subTp)) - { - log(subTp); - *asMutable(subTp) = Unifiable::Bound(superTp); - } - } - - else if (get(superTp)) - tryUnifyWithAny(superTp, subTp); - - else if (get(subTp)) - tryUnifyWithAny(subTp, superTp); - - else if (get(superTp)) - tryUnifyVariadics(superTp, subTp, false); - else if (get(subTp)) - tryUnifyVariadics(subTp, superTp, true); - - else if (get(superTp) && get(subTp)) - { - auto l = get(superTp); - auto r = get(subTp); - - // If the size of two heads does not match, but both packs have free tail - // We set the sentinel variable to say so to avoid growing it forever. - auto [superTypes, superTail] = flatten(superTp); - auto [subTypes, subTail] = flatten(subTp); - - bool noInfiniteGrowth = - (superTypes.size() != subTypes.size()) && (superTail && get(*superTail)) && (subTail && get(*subTail)); - - auto superIter = WeirdIter{superTp}; - auto subIter = WeirdIter{subTp}; - - auto mkFreshType = [this](TypeLevel level) { - return types->freshType(level); - }; - - const TypePackId emptyTp = types->addTypePack(TypePack{{}, std::nullopt}); - - int loopCount = 0; - - do - { - if (FInt::LuauTypeInferTypePackLoopLimit > 0 && loopCount >= FInt::LuauTypeInferTypePackLoopLimit) - ice("Detected possibly infinite TypePack growth"); - - ++loopCount; - - if (superIter.good() && subIter.growing) - asMutable(subIter.pack)->head.push_back(mkFreshType(subIter.level)); - - if (subIter.good() && superIter.growing) - asMutable(superIter.pack)->head.push_back(mkFreshType(superIter.level)); - - if (superIter.good() && subIter.good()) - { - tryUnify_(*superIter, *subIter); - - if (FFlag::LuauExtendedFunctionMismatchError && !errors.empty() && !firstPackErrorPos) - firstPackErrorPos = loopCount; - - superIter.advance(); - subIter.advance(); - continue; - } - - // If both are at the end, we're done - if (!superIter.good() && !subIter.good()) - { - const bool lFreeTail = l->tail && get(follow(*l->tail)) != nullptr; - const bool rFreeTail = r->tail && get(follow(*r->tail)) != nullptr; - if (lFreeTail && rFreeTail) - tryUnify_(*l->tail, *r->tail); - else if (lFreeTail) - tryUnify_(*l->tail, emptyTp); - else if (rFreeTail) - tryUnify_(*r->tail, emptyTp); - - break; - } - - // If both tails are free, bind one to the other and call it a day - if (superIter.canGrow() && subIter.canGrow()) - return tryUnify_(*superIter.pack->tail, *subIter.pack->tail); - - // If just one side is free on its tail, grow it to fit the other side. - // FIXME: The tail-most tail of the growing pack should be the same as the tail-most tail of the non-growing pack. - if (superIter.canGrow()) - superIter.grow(types->addTypePack(TypePackVar(TypePack{}))); - - else if (subIter.canGrow()) - subIter.grow(types->addTypePack(TypePackVar(TypePack{}))); - + if (tp->head.empty() && tp->tail) + subTp = log.follow(*tp->tail); else + break; + } + + while (auto tp = log.getMutable(superTp)) + { + if (tp->head.empty() && tp->tail) + superTp = log.follow(*tp->tail); + else + break; + } + + if (superTp == subTp) + return; + + if (log.getMutable(superTp)) + { + occursCheck(superTp, subTp); + + if (!log.getMutable(superTp)) { - // A union type including nil marks an optional argument - if (superIter.good() && isOptional(*superIter)) - { - superIter.advance(); - continue; - } - else if (subIter.good() && isOptional(*subIter)) - { - subIter.advance(); - continue; - } - - // In nonstrict mode, any also marks an optional argument. - else if (superIter.good() && isNonstrictMode() && get(follow(*superIter))) - { - superIter.advance(); - continue; - } - - if (get(superIter.packId)) - { - tryUnifyVariadics(superIter.packId, subIter.packId, false, int(subIter.index)); - return; - } - - if (get(subIter.packId)) - { - tryUnifyVariadics(subIter.packId, superIter.packId, true, int(superIter.index)); - return; - } - - if (!isFunctionCall && subIter.good()) - { - // Sometimes it is ok to pass too many arguments - return; - } - - // This is a bit weird because we don't actually know expected vs actual. We just know - // subtype vs supertype. If we are checking the values returned by a function, we swap - // these to produce the expected error message. - size_t expectedSize = size(superTp); - size_t actualSize = size(subTp); - if (ctx == CountMismatch::Result) - std::swap(expectedSize, actualSize); - errors.push_back(TypeError{location, CountMismatch{expectedSize, actualSize, ctx}}); - - while (superIter.good()) - { - tryUnify_(getSingletonTypes().errorRecoveryType(), *superIter); - superIter.advance(); - } - - while (subIter.good()) - { - tryUnify_(getSingletonTypes().errorRecoveryType(), *subIter); - subIter.advance(); - } - - return; + log.replace(superTp, Unifiable::Bound(subTp)); } + } + else if (log.getMutable(subTp)) + { + occursCheck(subTp, superTp); - } while (!noInfiniteGrowth); + if (!log.getMutable(subTp)) + { + log.replace(subTp, Unifiable::Bound(superTp)); + } + } + else if (log.getMutable(superTp)) + tryUnifyWithAny(subTp, superTp); + else if (log.getMutable(subTp)) + tryUnifyWithAny(superTp, subTp); + else if (log.getMutable(superTp)) + tryUnifyVariadics(subTp, superTp, false); + else if (log.getMutable(subTp)) + tryUnifyVariadics(superTp, subTp, true); + else if (log.getMutable(superTp) && log.getMutable(subTp)) + { + auto superTpv = log.getMutable(superTp); + auto subTpv = log.getMutable(subTp); + + // If the size of two heads does not match, but both packs have free tail + // We set the sentinel variable to say so to avoid growing it forever. + auto [superTypes, superTail] = logAwareFlatten(superTp, log); + auto [subTypes, subTail] = logAwareFlatten(subTp, log); + + bool noInfiniteGrowth = + (superTypes.size() != subTypes.size()) && (superTail && get(*superTail)) && (subTail && get(*subTail)); + + auto superIter = WeirdIter(superTp, log); + auto subIter = WeirdIter(subTp, log); + + auto mkFreshType = [this](TypeLevel level) { + return types->freshType(level); + }; + + const TypePackId emptyTp = types->addTypePack(TypePack{{}, std::nullopt}); + + int loopCount = 0; + + do + { + if (FInt::LuauTypeInferTypePackLoopLimit > 0 && loopCount >= FInt::LuauTypeInferTypePackLoopLimit) + ice("Detected possibly infinite TypePack growth"); + + ++loopCount; + + if (superIter.good() && subIter.growing) + { + subIter.pushType(mkFreshType(subIter.level)); + } + + if (subIter.good() && superIter.growing) + { + superIter.pushType(mkFreshType(superIter.level)); + } + + if (superIter.good() && subIter.good()) + { + tryUnify_(*subIter, *superIter); + + if (FFlag::LuauExtendedFunctionMismatchError && !errors.empty() && !firstPackErrorPos) + firstPackErrorPos = loopCount; + + superIter.advance(); + subIter.advance(); + continue; + } + + // If both are at the end, we're done + if (!superIter.good() && !subIter.good()) + { + const bool lFreeTail = superTpv->tail && log.getMutable(log.follow(*superTpv->tail)) != nullptr; + const bool rFreeTail = subTpv->tail && log.getMutable(log.follow(*subTpv->tail)) != nullptr; + if (lFreeTail && rFreeTail) + tryUnify_(*subTpv->tail, *superTpv->tail); + else if (lFreeTail) + tryUnify_(emptyTp, *superTpv->tail); + else if (rFreeTail) + tryUnify_(emptyTp, *subTpv->tail); + + break; + } + + // If both tails are free, bind one to the other and call it a day + if (superIter.canGrow() && subIter.canGrow()) + return tryUnify_(*subIter.pack->tail, *superIter.pack->tail); + + // If just one side is free on its tail, grow it to fit the other side. + // FIXME: The tail-most tail of the growing pack should be the same as the tail-most tail of the non-growing pack. + if (superIter.canGrow()) + superIter.grow(types->addTypePack(TypePackVar(TypePack{}))); + else if (subIter.canGrow()) + subIter.grow(types->addTypePack(TypePackVar(TypePack{}))); + else + { + // A union type including nil marks an optional argument + if (superIter.good() && isOptional(*superIter)) + { + superIter.advance(); + continue; + } + else if (subIter.good() && isOptional(*subIter)) + { + subIter.advance(); + continue; + } + + // In nonstrict mode, any also marks an optional argument. + else if (superIter.good() && isNonstrictMode() && log.getMutable(log.follow(*superIter))) + { + superIter.advance(); + continue; + } + + if (log.getMutable(superIter.packId)) + { + tryUnifyVariadics(subIter.packId, superIter.packId, false, int(subIter.index)); + return; + } + + if (log.getMutable(subIter.packId)) + { + tryUnifyVariadics(superIter.packId, subIter.packId, true, int(superIter.index)); + return; + } + + if (!isFunctionCall && subIter.good()) + { + // Sometimes it is ok to pass too many arguments + return; + } + + // This is a bit weird because we don't actually know expected vs actual. We just know + // subtype vs supertype. If we are checking the values returned by a function, we swap + // these to produce the expected error message. + size_t expectedSize = size(superTp); + size_t actualSize = size(subTp); + if (ctx == CountMismatch::Result) + std::swap(expectedSize, actualSize); + errors.push_back(TypeError{location, CountMismatch{expectedSize, actualSize, ctx}}); + + while (superIter.good()) + { + tryUnify_(*superIter, getSingletonTypes().errorRecoveryType()); + superIter.advance(); + } + + while (subIter.good()) + { + tryUnify_(*subIter, getSingletonTypes().errorRecoveryType()); + subIter.advance(); + } + + return; + } + + } while (!noInfiniteGrowth); + } + else + { + errors.push_back(TypeError{location, GenericError{"Failed to unify type packs"}}); + } } else { - errors.push_back(TypeError{location, GenericError{"Failed to unify type packs"}}); + superTp = follow(superTp); + subTp = follow(subTp); + + while (auto tp = get(subTp)) + { + if (tp->head.empty() && tp->tail) + subTp = follow(*tp->tail); + else + break; + } + + while (auto tp = get(superTp)) + { + if (tp->head.empty() && tp->tail) + superTp = follow(*tp->tail); + else + break; + } + + if (superTp == subTp) + return; + + if (get(superTp)) + { + occursCheck(superTp, subTp); + + if (!get(superTp)) + { + DEPRECATED_log(superTp); + *asMutable(superTp) = Unifiable::Bound(subTp); + } + } + else if (get(subTp)) + { + occursCheck(subTp, superTp); + + if (!get(subTp)) + { + DEPRECATED_log(subTp); + *asMutable(subTp) = Unifiable::Bound(superTp); + } + } + + else if (get(superTp)) + tryUnifyWithAny(subTp, superTp); + + else if (get(subTp)) + tryUnifyWithAny(superTp, subTp); + + else if (get(superTp)) + tryUnifyVariadics(subTp, superTp, false); + else if (get(subTp)) + tryUnifyVariadics(superTp, subTp, true); + + else if (get(superTp) && get(subTp)) + { + auto superTpv = get(superTp); + auto subTpv = get(subTp); + + // If the size of two heads does not match, but both packs have free tail + // We set the sentinel variable to say so to avoid growing it forever. + auto [superTypes, superTail] = flatten(superTp); + auto [subTypes, subTail] = flatten(subTp); + + bool noInfiniteGrowth = + (superTypes.size() != subTypes.size()) && (superTail && get(*superTail)) && (subTail && get(*subTail)); + + auto superIter = DEPRECATED_WeirdIter{superTp}; + auto subIter = DEPRECATED_WeirdIter{subTp}; + + auto mkFreshType = [this](TypeLevel level) { + return types->freshType(level); + }; + + const TypePackId emptyTp = types->addTypePack(TypePack{{}, std::nullopt}); + + int loopCount = 0; + + do + { + if (FInt::LuauTypeInferTypePackLoopLimit > 0 && loopCount >= FInt::LuauTypeInferTypePackLoopLimit) + ice("Detected possibly infinite TypePack growth"); + + ++loopCount; + + if (superIter.good() && subIter.growing) + asMutable(subIter.pack)->head.push_back(mkFreshType(subIter.level)); + + if (subIter.good() && superIter.growing) + asMutable(superIter.pack)->head.push_back(mkFreshType(superIter.level)); + + if (superIter.good() && subIter.good()) + { + tryUnify_(*subIter, *superIter); + + if (FFlag::LuauExtendedFunctionMismatchError && !errors.empty() && !firstPackErrorPos) + firstPackErrorPos = loopCount; + + superIter.advance(); + subIter.advance(); + continue; + } + + // If both are at the end, we're done + if (!superIter.good() && !subIter.good()) + { + const bool lFreeTail = superTpv->tail && get(follow(*superTpv->tail)) != nullptr; + const bool rFreeTail = subTpv->tail && get(follow(*subTpv->tail)) != nullptr; + if (lFreeTail && rFreeTail) + tryUnify_(*subTpv->tail, *superTpv->tail); + else if (lFreeTail) + tryUnify_(emptyTp, *superTpv->tail); + else if (rFreeTail) + tryUnify_(emptyTp, *subTpv->tail); + + break; + } + + // If both tails are free, bind one to the other and call it a day + if (superIter.canGrow() && subIter.canGrow()) + return tryUnify_(*subIter.pack->tail, *superIter.pack->tail); + + // If just one side is free on its tail, grow it to fit the other side. + // FIXME: The tail-most tail of the growing pack should be the same as the tail-most tail of the non-growing pack. + if (superIter.canGrow()) + superIter.grow(types->addTypePack(TypePackVar(TypePack{}))); + + else if (subIter.canGrow()) + subIter.grow(types->addTypePack(TypePackVar(TypePack{}))); + + else + { + // A union type including nil marks an optional argument + if (superIter.good() && isOptional(*superIter)) + { + superIter.advance(); + continue; + } + else if (subIter.good() && isOptional(*subIter)) + { + subIter.advance(); + continue; + } + + // In nonstrict mode, any also marks an optional argument. + else if (superIter.good() && isNonstrictMode() && get(follow(*superIter))) + { + superIter.advance(); + continue; + } + + if (get(superIter.packId)) + { + tryUnifyVariadics(subIter.packId, superIter.packId, false, int(subIter.index)); + return; + } + + if (get(subIter.packId)) + { + tryUnifyVariadics(superIter.packId, subIter.packId, true, int(superIter.index)); + return; + } + + if (!isFunctionCall && subIter.good()) + { + // Sometimes it is ok to pass too many arguments + return; + } + + // This is a bit weird because we don't actually know expected vs actual. We just know + // subtype vs supertype. If we are checking the values returned by a function, we swap + // these to produce the expected error message. + size_t expectedSize = size(superTp); + size_t actualSize = size(subTp); + if (ctx == CountMismatch::Result) + std::swap(expectedSize, actualSize); + errors.push_back(TypeError{location, CountMismatch{expectedSize, actualSize, ctx}}); + + while (superIter.good()) + { + tryUnify_(*superIter, getSingletonTypes().errorRecoveryType()); + superIter.advance(); + } + + while (subIter.good()) + { + tryUnify_(*subIter, getSingletonTypes().errorRecoveryType()); + subIter.advance(); + } + + return; + } + + } while (!noInfiniteGrowth); + } + else + { + errors.push_back(TypeError{location, GenericError{"Failed to unify type packs"}}); + } } } -void Unifier::tryUnifyPrimitives(TypeId superTy, TypeId subTy) +void Unifier::tryUnifyPrimitives(TypeId subTy, TypeId superTy) { - const PrimitiveTypeVar* lp = get(superTy); - const PrimitiveTypeVar* rp = get(subTy); - if (!lp || !rp) + const PrimitiveTypeVar* superPrim = get(superTy); + const PrimitiveTypeVar* subPrim = get(subTy); + if (!superPrim || !subPrim) ice("passed non primitive types to unifyPrimitives"); - if (lp->type != rp->type) + if (superPrim->type != subPrim->type) errors.push_back(TypeError{location, TypeMismatch{superTy, subTy}}); } -void Unifier::tryUnifySingletons(TypeId superTy, TypeId subTy) +void Unifier::tryUnifySingletons(TypeId subTy, TypeId superTy) { - const PrimitiveTypeVar* lp = get(superTy); - const SingletonTypeVar* ls = get(superTy); - const SingletonTypeVar* rs = get(subTy); + const PrimitiveTypeVar* superPrim = get(superTy); + const SingletonTypeVar* superSingleton = get(superTy); + const SingletonTypeVar* subSingleton = get(subTy); - if ((!lp && !ls) || !rs) + if ((!superPrim && !superSingleton) || !subSingleton) ice("passed non singleton/primitive types to unifySingletons"); - if (ls && *ls == *rs) + if (superSingleton && *superSingleton == *subSingleton) return; - if (lp && lp->type == PrimitiveTypeVar::Boolean && get(rs) && variance == Covariant) + if (superPrim && superPrim->type == PrimitiveTypeVar::Boolean && get(subSingleton) && variance == Covariant) return; - if (lp && lp->type == PrimitiveTypeVar::String && get(rs) && variance == Covariant) + if (superPrim && superPrim->type == PrimitiveTypeVar::String && get(subSingleton) && variance == Covariant) return; errors.push_back(TypeError{location, TypeMismatch{superTy, subTy}}); } -void Unifier::tryUnifyFunctions(TypeId superTy, TypeId subTy, bool isFunctionCall) +void Unifier::tryUnifyFunctions(TypeId subTy, TypeId superTy, bool isFunctionCall) { - FunctionTypeVar* lf = getMutable(superTy); - FunctionTypeVar* rf = getMutable(subTy); - if (!lf || !rf) + FunctionTypeVar* superFunction = getMutable(superTy); + FunctionTypeVar* subFunction = getMutable(subTy); + + if (FFlag::LuauUseCommittingTxnLog) + { + superFunction = log.getMutable(superTy); + subFunction = log.getMutable(subTy); + } + + if (!superFunction || !subFunction) ice("passed non-function types to unifyFunction"); - size_t numGenerics = lf->generics.size(); - if (numGenerics != rf->generics.size()) + size_t numGenerics = superFunction->generics.size(); + if (numGenerics != subFunction->generics.size()) { - numGenerics = std::min(lf->generics.size(), rf->generics.size()); + numGenerics = std::min(superFunction->generics.size(), subFunction->generics.size()); if (FFlag::LuauExtendedFunctionMismatchError) errors.push_back(TypeError{location, TypeMismatch{superTy, subTy, "different number of generic type parameters"}}); @@ -1048,10 +1523,10 @@ void Unifier::tryUnifyFunctions(TypeId superTy, TypeId subTy, bool isFunctionCal errors.push_back(TypeError{location, TypeMismatch{superTy, subTy}}); } - size_t numGenericPacks = lf->genericPacks.size(); - if (numGenericPacks != rf->genericPacks.size()) + size_t numGenericPacks = superFunction->genericPacks.size(); + if (numGenericPacks != subFunction->genericPacks.size()) { - numGenericPacks = std::min(lf->genericPacks.size(), rf->genericPacks.size()); + numGenericPacks = std::min(superFunction->genericPacks.size(), subFunction->genericPacks.size()); if (FFlag::LuauExtendedFunctionMismatchError) errors.push_back(TypeError{location, TypeMismatch{superTy, subTy, "different number of generic type pack parameters"}}); @@ -1060,7 +1535,12 @@ void Unifier::tryUnifyFunctions(TypeId superTy, TypeId subTy, bool isFunctionCal } for (size_t i = 0; i < numGenerics; i++) - log.pushSeen(lf->generics[i], rf->generics[i]); + { + if (FFlag::LuauUseCommittingTxnLog) + log.pushSeen(superFunction->generics[i], subFunction->generics[i]); + else + DEPRECATED_log.pushSeen(superFunction->generics[i], subFunction->generics[i]); + } CountMismatch::Context context = ctx; @@ -1071,7 +1551,7 @@ void Unifier::tryUnifyFunctions(TypeId superTy, TypeId subTy, bool isFunctionCal if (FFlag::LuauExtendedFunctionMismatchError) { innerState.ctx = CountMismatch::Arg; - innerState.tryUnify_(rf->argTypes, lf->argTypes, isFunctionCall); + innerState.tryUnify_(superFunction->argTypes, subFunction->argTypes, isFunctionCall); bool reported = !innerState.errors.empty(); @@ -1085,13 +1565,13 @@ void Unifier::tryUnifyFunctions(TypeId superTy, TypeId subTy, bool isFunctionCal errors.push_back(TypeError{location, TypeMismatch{superTy, subTy, "", innerState.errors.front()}}); innerState.ctx = CountMismatch::Result; - innerState.tryUnify_(lf->retType, rf->retType); + innerState.tryUnify_(subFunction->retType, superFunction->retType); if (!reported) { if (auto e = hasUnificationTooComplex(innerState.errors)) errors.push_back(*e); - else if (!innerState.errors.empty() && size(lf->retType) == 1 && finite(lf->retType)) + else if (!innerState.errors.empty() && size(superFunction->retType) == 1 && finite(superFunction->retType)) errors.push_back(TypeError{location, TypeMismatch{superTy, subTy, "Return type is not compatible.", innerState.errors.front()}}); else if (!innerState.errors.empty() && innerState.firstPackErrorPos) errors.push_back( @@ -1104,38 +1584,70 @@ void Unifier::tryUnifyFunctions(TypeId superTy, TypeId subTy, bool isFunctionCal else { ctx = CountMismatch::Arg; - innerState.tryUnify_(rf->argTypes, lf->argTypes, isFunctionCall); + innerState.tryUnify_(superFunction->argTypes, subFunction->argTypes, isFunctionCall); ctx = CountMismatch::Result; - innerState.tryUnify_(lf->retType, rf->retType); + innerState.tryUnify_(subFunction->retType, superFunction->retType); checkChildUnifierTypeMismatch(innerState.errors, superTy, subTy); } - log.concat(std::move(innerState.log)); + if (FFlag::LuauUseCommittingTxnLog) + { + log.concat(std::move(innerState.log)); + } + else + { + DEPRECATED_log.concat(std::move(innerState.DEPRECATED_log)); + } } else { ctx = CountMismatch::Arg; - tryUnify_(rf->argTypes, lf->argTypes, isFunctionCall); + tryUnify_(superFunction->argTypes, subFunction->argTypes, isFunctionCall); ctx = CountMismatch::Result; - tryUnify_(lf->retType, rf->retType); + tryUnify_(subFunction->retType, superFunction->retType); } - if (lf->definition && !rf->definition && !subTy->persistent) + if (FFlag::LuauUseCommittingTxnLog) { - rf->definition = lf->definition; + if (superFunction->definition && !subFunction->definition && !subTy->persistent) + { + PendingType* newSubTy = log.queue(subTy); + FunctionTypeVar* newSubFtv = getMutable(newSubTy); + LUAU_ASSERT(newSubFtv); + newSubFtv->definition = superFunction->definition; + } + else if (!superFunction->definition && subFunction->definition && !superTy->persistent) + { + PendingType* newSuperTy = log.queue(superTy); + FunctionTypeVar* newSuperFtv = getMutable(newSuperTy); + LUAU_ASSERT(newSuperFtv); + newSuperFtv->definition = subFunction->definition; + } } - else if (!lf->definition && rf->definition && !superTy->persistent) + else { - lf->definition = rf->definition; + if (superFunction->definition && !subFunction->definition && !subTy->persistent) + { + subFunction->definition = superFunction->definition; + } + else if (!superFunction->definition && subFunction->definition && !superTy->persistent) + { + superFunction->definition = subFunction->definition; + } } ctx = context; for (int i = int(numGenerics) - 1; 0 <= i; i--) - log.popSeen(lf->generics[i], rf->generics[i]); + { + if (FFlag::LuauUseCommittingTxnLog) + log.popSeen(superFunction->generics[i], subFunction->generics[i]); + else + DEPRECATED_log.popSeen(superFunction->generics[i], subFunction->generics[i]); + } } namespace @@ -1160,77 +1672,84 @@ struct Resetter } // namespace -void Unifier::tryUnifyTables(TypeId left, TypeId right, bool isIntersection) +void Unifier::tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) { if (!FFlag::LuauTableSubtypingVariance2) - return DEPRECATED_tryUnifyTables(left, right, isIntersection); + return DEPRECATED_tryUnifyTables(subTy, superTy, isIntersection); - TableTypeVar* lt = getMutable(left); - TableTypeVar* rt = getMutable(right); - if (!lt || !rt) + TableTypeVar* superTable = getMutable(superTy); + TableTypeVar* subTable = getMutable(subTy); + if (!superTable || !subTable) ice("passed non-table types to unifyTables"); std::vector missingProperties; std::vector extraProperties; // Optimization: First test that the property sets are compatible without doing any recursive unification - if (FFlag::LuauTableUnificationEarlyTest && !rt->indexer && rt->state != TableState::Free) + if (FFlag::LuauTableUnificationEarlyTest && !subTable->indexer && subTable->state != TableState::Free) { - for (const auto& [propName, superProp] : lt->props) + for (const auto& [propName, superProp] : superTable->props) { - auto subIter = rt->props.find(propName); - if (subIter == rt->props.end() && !isOptional(superProp.type) && !get(follow(superProp.type))) + auto subIter = subTable->props.find(propName); + if (subIter == subTable->props.end() && !isOptional(superProp.type) && !get(follow(superProp.type))) missingProperties.push_back(propName); } if (!missingProperties.empty()) { - errors.push_back(TypeError{location, MissingProperties{left, right, std::move(missingProperties)}}); + errors.push_back(TypeError{location, MissingProperties{superTy, subTy, std::move(missingProperties)}}); return; } } // And vice versa if we're invariant - if (FFlag::LuauTableUnificationEarlyTest && variance == Invariant && !lt->indexer && lt->state != TableState::Unsealed && - lt->state != TableState::Free) + if (FFlag::LuauTableUnificationEarlyTest && variance == Invariant && !superTable->indexer && superTable->state != TableState::Unsealed && + superTable->state != TableState::Free) { - for (const auto& [propName, subProp] : rt->props) + for (const auto& [propName, subProp] : subTable->props) { - auto superIter = lt->props.find(propName); - if (superIter == lt->props.end() && !isOptional(subProp.type) && !get(follow(subProp.type))) + auto superIter = superTable->props.find(propName); + if (superIter == superTable->props.end() && !isOptional(subProp.type) && !get(follow(subProp.type))) extraProperties.push_back(propName); } if (!extraProperties.empty()) { - errors.push_back(TypeError{location, MissingProperties{left, right, std::move(extraProperties), MissingProperties::Extra}}); + errors.push_back(TypeError{location, MissingProperties{superTy, subTy, std::move(extraProperties), MissingProperties::Extra}}); return; } } - // Reminder: left is the supertype, right is the subtype. // Width subtyping: any property in the supertype must be in the subtype, // and the types must agree. - for (const auto& [name, prop] : lt->props) + for (const auto& [name, prop] : superTable->props) { - const auto& r = rt->props.find(name); - if (r != rt->props.end()) + const auto& r = subTable->props.find(name); + if (r != subTable->props.end()) { // TODO: read-only properties don't need invariance Resetter resetter{&variance}; variance = Invariant; Unifier innerState = makeChildUnifier(); - innerState.tryUnify_(prop.type, r->second.type); + innerState.tryUnify_(r->second.type, prop.type); - checkChildUnifierTypeMismatch(innerState.errors, name, left, right); + checkChildUnifierTypeMismatch(innerState.errors, name, superTy, subTy); - if (innerState.errors.empty()) - log.concat(std::move(innerState.log)); + if (FFlag::LuauUseCommittingTxnLog) + { + if (innerState.errors.empty()) + log.concat(std::move(innerState.log)); + } else - innerState.log.rollback(); + { + if (innerState.errors.empty()) + DEPRECATED_log.concat(std::move(innerState.DEPRECATED_log)); + else + innerState.DEPRECATED_log.rollback(); + } } - else if (rt->indexer && isString(rt->indexer->indexType)) + else if (subTable->indexer && isString(subTable->indexer->indexType)) { // TODO: read-only indexers don't need invariance // TODO: really we should only allow this if prop.type is optional. @@ -1238,37 +1757,55 @@ void Unifier::tryUnifyTables(TypeId left, TypeId right, bool isIntersection) variance = Invariant; Unifier innerState = makeChildUnifier(); - innerState.tryUnify_(prop.type, rt->indexer->indexResultType); + innerState.tryUnify_(subTable->indexer->indexResultType, prop.type); - checkChildUnifierTypeMismatch(innerState.errors, name, left, right); + checkChildUnifierTypeMismatch(innerState.errors, name, superTy, subTy); - if (innerState.errors.empty()) - log.concat(std::move(innerState.log)); + if (FFlag::LuauUseCommittingTxnLog) + { + if (innerState.errors.empty()) + log.concat(std::move(innerState.log)); + } else - innerState.log.rollback(); + { + if (innerState.errors.empty()) + DEPRECATED_log.concat(std::move(innerState.DEPRECATED_log)); + else + innerState.DEPRECATED_log.rollback(); + } } else if (isOptional(prop.type) || get(follow(prop.type))) // TODO: this case is unsound, but without it our test suite fails. CLI-46031 // TODO: should isOptional(anyType) be true? { } - else if (rt->state == TableState::Free) + else if (subTable->state == TableState::Free) { - log(rt); - rt->props[name] = prop; + if (FFlag::LuauUseCommittingTxnLog) + { + PendingType* pendingSub = log.queue(subTy); + TableTypeVar* ttv = getMutable(pendingSub); + LUAU_ASSERT(ttv); + ttv->props[name] = prop; + } + else + { + DEPRECATED_log(subTy); + subTable->props[name] = prop; + } } else missingProperties.push_back(name); } - for (const auto& [name, prop] : rt->props) + for (const auto& [name, prop] : subTable->props) { - if (lt->props.count(name)) + if (superTable->props.count(name)) { // If both lt and rt contain the property, then // we're done since we already unified them above } - else if (lt->indexer && isString(lt->indexer->indexType)) + else if (superTable->indexer && isString(superTable->indexer->indexType)) { // TODO: read-only indexers don't need invariance // TODO: really we should only allow this if prop.type is optional. @@ -1276,24 +1813,42 @@ void Unifier::tryUnifyTables(TypeId left, TypeId right, bool isIntersection) variance = Invariant; Unifier innerState = makeChildUnifier(); - innerState.tryUnify_(prop.type, lt->indexer->indexResultType); + innerState.tryUnify_(superTable->indexer->indexResultType, prop.type); - checkChildUnifierTypeMismatch(innerState.errors, name, left, right); + checkChildUnifierTypeMismatch(innerState.errors, name, superTy, subTy); - if (innerState.errors.empty()) - log.concat(std::move(innerState.log)); + if (FFlag::LuauUseCommittingTxnLog) + { + if (innerState.errors.empty()) + log.concat(std::move(innerState.log)); + } else - innerState.log.rollback(); + { + if (innerState.errors.empty()) + DEPRECATED_log.concat(std::move(innerState.DEPRECATED_log)); + else + innerState.DEPRECATED_log.rollback(); + } } - else if (lt->state == TableState::Unsealed) + else if (superTable->state == TableState::Unsealed) { // TODO: this case is unsound when variance is Invariant, but without it lua-apps fails to typecheck. // TODO: file a JIRA // TODO: hopefully readonly/writeonly properties will fix this. Property clone = prop; clone.type = deeplyOptional(clone.type); - log(left); - lt->props[name] = clone; + + if (FFlag::LuauUseCommittingTxnLog) + { + PendingType* pendingSuper = log.queue(superTy); + TableTypeVar* pendingSuperTtv = getMutable(pendingSuper); + pendingSuperTtv->props[name] = clone; + } + else + { + DEPRECATED_log(superTy); + superTable->props[name] = clone; + } } else if (variance == Covariant) { @@ -1303,61 +1858,93 @@ void Unifier::tryUnifyTables(TypeId left, TypeId right, bool isIntersection) // TODO: should isOptional(anyType) be true? { } - else if (lt->state == TableState::Free) + else if (superTable->state == TableState::Free) { - log(left); - lt->props[name] = prop; + if (FFlag::LuauUseCommittingTxnLog) + { + PendingType* pendingSuper = log.queue(superTy); + TableTypeVar* pendingSuperTtv = getMutable(pendingSuper); + pendingSuperTtv->props[name] = prop; + } + else + { + DEPRECATED_log(superTy); + superTable->props[name] = prop; + } } else extraProperties.push_back(name); } // Unify indexers - if (lt->indexer && rt->indexer) + if (superTable->indexer && subTable->indexer) { // TODO: read-only indexers don't need invariance Resetter resetter{&variance}; variance = Invariant; Unifier innerState = makeChildUnifier(); - innerState.tryUnify(*lt->indexer, *rt->indexer); - checkChildUnifierTypeMismatch(innerState.errors, left, right); - if (innerState.errors.empty()) - log.concat(std::move(innerState.log)); + innerState.tryUnifyIndexer(*subTable->indexer, *superTable->indexer); + checkChildUnifierTypeMismatch(innerState.errors, superTy, subTy); + + if (FFlag::LuauUseCommittingTxnLog) + { + if (innerState.errors.empty()) + log.concat(std::move(innerState.log)); + } else - innerState.log.rollback(); + { + if (innerState.errors.empty()) + DEPRECATED_log.concat(std::move(innerState.DEPRECATED_log)); + else + innerState.DEPRECATED_log.rollback(); + } } - else if (lt->indexer) + else if (superTable->indexer) { - if (rt->state == TableState::Unsealed || rt->state == TableState::Free) + if (subTable->state == TableState::Unsealed || subTable->state == TableState::Free) { // passing/assigning a table without an indexer to something that has one // e.g. table.insert(t, 1) where t is a non-sealed table and doesn't have an indexer. // TODO: we only need to do this if the supertype's indexer is read/write // since that can add indexed elements. - log(right); - rt->indexer = lt->indexer; + if (FFlag::LuauUseCommittingTxnLog) + { + log.changeIndexer(subTy, superTable->indexer); + } + else + { + DEPRECATED_log(subTy); + subTable->indexer = superTable->indexer; + } } } - else if (rt->indexer && variance == Invariant) + else if (subTable->indexer && variance == Invariant) { // Symmetric if we are invariant - if (lt->state == TableState::Unsealed || lt->state == TableState::Free) + if (superTable->state == TableState::Unsealed || superTable->state == TableState::Free) { - log(left); - lt->indexer = rt->indexer; + if (FFlag::LuauUseCommittingTxnLog) + { + log.changeIndexer(superTy, subTable->indexer); + } + else + { + DEPRECATED_log(superTy); + superTable->indexer = subTable->indexer; + } } } if (!missingProperties.empty()) { - errors.push_back(TypeError{location, MissingProperties{left, right, std::move(missingProperties)}}); + errors.push_back(TypeError{location, MissingProperties{superTy, subTy, std::move(missingProperties)}}); return; } if (!extraProperties.empty()) { - errors.push_back(TypeError{location, MissingProperties{left, right, std::move(extraProperties), MissingProperties::Extra}}); + errors.push_back(TypeError{location, MissingProperties{superTy, subTy, std::move(extraProperties), MissingProperties::Extra}}); return; } @@ -1369,18 +1956,32 @@ void Unifier::tryUnifyTables(TypeId left, TypeId right, bool isIntersection) * I believe this is guaranteed to terminate eventually because this will * only happen when a free table is bound to another table. */ - if (lt->boundTo || rt->boundTo) - return tryUnify_(left, right); + if (superTable->boundTo || subTable->boundTo) + return tryUnify_(subTy, superTy); - if (lt->state == TableState::Free) + if (superTable->state == TableState::Free) { - log(lt); - lt->boundTo = right; + if (FFlag::LuauUseCommittingTxnLog) + { + log.bindTable(superTy, subTy); + } + else + { + DEPRECATED_log(superTable); + superTable->boundTo = subTy; + } } - else if (rt->state == TableState::Free) + else if (subTable->state == TableState::Free) { - log(rt); - rt->boundTo = left; + if (FFlag::LuauUseCommittingTxnLog) + { + log.bindTable(subTy, superTy); + } + else + { + DEPRECATED_log(subTy); + subTable->boundTo = superTy; + } } } @@ -1406,99 +2007,129 @@ TypeId Unifier::deeplyOptional(TypeId ty, std::unordered_map see return types->addType(UnionTypeVar{{getSingletonTypes().nilType, ty}}); } -void Unifier::DEPRECATED_tryUnifyTables(TypeId left, TypeId right, bool isIntersection) +void Unifier::DEPRECATED_tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) { LUAU_ASSERT(!FFlag::LuauTableSubtypingVariance2); Resetter resetter{&variance}; variance = Invariant; - TableTypeVar* lt = getMutable(left); - TableTypeVar* rt = getMutable(right); - if (!lt || !rt) + TableTypeVar* superTable = getMutable(superTy); + TableTypeVar* subTable = getMutable(subTy); + + if (FFlag::LuauUseCommittingTxnLog) + { + superTable = log.getMutable(superTy); + subTable = log.getMutable(subTy); + } + + if (!superTable || !subTable) ice("passed non-table types to unifyTables"); - if (lt->state == TableState::Sealed && rt->state == TableState::Sealed) - return tryUnifySealedTables(left, right, isIntersection); - else if ((lt->state == TableState::Sealed && rt->state == TableState::Unsealed) || - (lt->state == TableState::Unsealed && rt->state == TableState::Sealed)) - return tryUnifySealedTables(left, right, isIntersection); - else if ((lt->state == TableState::Sealed && rt->state == TableState::Generic) || - (lt->state == TableState::Generic && rt->state == TableState::Sealed)) - errors.push_back(TypeError{location, TypeMismatch{left, right}}); - else if ((lt->state == TableState::Free) != (rt->state == TableState::Free)) // one table is free and the other is not + if (superTable->state == TableState::Sealed && subTable->state == TableState::Sealed) + return tryUnifySealedTables(subTy, superTy, isIntersection); + else if ((superTable->state == TableState::Sealed && subTable->state == TableState::Unsealed) || + (superTable->state == TableState::Unsealed && subTable->state == TableState::Sealed)) + return tryUnifySealedTables(subTy, superTy, isIntersection); + else if ((superTable->state == TableState::Sealed && subTable->state == TableState::Generic) || + (superTable->state == TableState::Generic && subTable->state == TableState::Sealed)) + errors.push_back(TypeError{location, TypeMismatch{superTy, subTy}}); + else if ((superTable->state == TableState::Free) != (subTable->state == TableState::Free)) // one table is free and the other is not { - TypeId freeTypeId = rt->state == TableState::Free ? right : left; - TypeId otherTypeId = rt->state == TableState::Free ? left : right; + TypeId freeTypeId = subTable->state == TableState::Free ? subTy : superTy; + TypeId otherTypeId = subTable->state == TableState::Free ? superTy : subTy; - return tryUnifyFreeTable(freeTypeId, otherTypeId); + return tryUnifyFreeTable(otherTypeId, freeTypeId); } - else if (lt->state == TableState::Free && rt->state == TableState::Free) + else if (superTable->state == TableState::Free && subTable->state == TableState::Free) { - tryUnifyFreeTable(left, right); + tryUnifyFreeTable(subTy, superTy); // avoid creating a cycle when the types are already pointing at each other - if (follow(left) != follow(right)) + if (follow(superTy) != follow(subTy)) { - log(lt); - lt->boundTo = right; + if (FFlag::LuauUseCommittingTxnLog) + { + log.bindTable(superTy, subTy); + } + else + { + DEPRECATED_log(superTable); + superTable->boundTo = subTy; + } } return; } - else if (lt->state != TableState::Sealed && rt->state != TableState::Sealed) + else if (superTable->state != TableState::Sealed && subTable->state != TableState::Sealed) { // All free tables are checked in one of the branches above - LUAU_ASSERT(lt->state != TableState::Free); - LUAU_ASSERT(rt->state != TableState::Free); + LUAU_ASSERT(superTable->state != TableState::Free); + LUAU_ASSERT(subTable->state != TableState::Free); // Tables must have exactly the same props and their types must all unify // I honestly have no idea if this is remotely close to reasonable. - for (const auto& [name, prop] : lt->props) + for (const auto& [name, prop] : superTable->props) { - const auto& r = rt->props.find(name); - if (r == rt->props.end()) - errors.push_back(TypeError{location, UnknownProperty{right, name}}); + const auto& r = subTable->props.find(name); + if (r == subTable->props.end()) + errors.push_back(TypeError{location, UnknownProperty{subTy, name}}); else - tryUnify_(prop.type, r->second.type); + tryUnify_(r->second.type, prop.type); } - if (lt->indexer && rt->indexer) - tryUnify(*lt->indexer, *rt->indexer); - else if (lt->indexer) + if (superTable->indexer && subTable->indexer) + tryUnifyIndexer(*subTable->indexer, *superTable->indexer); + else if (superTable->indexer) { // passing/assigning a table without an indexer to something that has one // e.g. table.insert(t, 1) where t is a non-sealed table and doesn't have an indexer. - if (rt->state == TableState::Unsealed) - rt->indexer = lt->indexer; + if (subTable->state == TableState::Unsealed) + { + if (FFlag::LuauUseCommittingTxnLog) + { + log.changeIndexer(subTy, superTable->indexer); + } + else + { + subTable->indexer = superTable->indexer; + } + } else - errors.push_back(TypeError{location, CannotExtendTable{right, CannotExtendTable::Indexer}}); + errors.push_back(TypeError{location, CannotExtendTable{subTy, CannotExtendTable::Indexer}}); } } - else if (lt->state == TableState::Sealed) + else if (superTable->state == TableState::Sealed) { // lt is sealed and so it must be possible for rt to have precisely the same shape // Verify that this is the case, then bind rt to lt. ice("unsealed tables are not working yet", location); } - else if (rt->state == TableState::Sealed) - return tryUnifyTables(right, left, isIntersection); + else if (subTable->state == TableState::Sealed) + return tryUnifyTables(superTy, subTy, isIntersection); else ice("tryUnifyTables"); } -void Unifier::tryUnifyFreeTable(TypeId freeTypeId, TypeId otherTypeId) +void Unifier::tryUnifyFreeTable(TypeId subTy, TypeId superTy) { - TableTypeVar* freeTable = getMutable(freeTypeId); - TableTypeVar* otherTable = getMutable(otherTypeId); - if (!freeTable || !otherTable) + TableTypeVar* freeTable = getMutable(superTy); + TableTypeVar* subTable = getMutable(subTy); + + if (FFlag::LuauUseCommittingTxnLog) + { + freeTable = log.getMutable(superTy); + subTable = log.getMutable(subTy); + } + + if (!freeTable || !subTable) ice("passed non-table types to tryUnifyFreeTable"); // Any properties in freeTable must unify with those in otherTable. // Then bind freeTable to otherTable. for (const auto& [freeName, freeProp] : freeTable->props) { - if (auto otherProp = findTablePropertyRespectingMeta(otherTypeId, freeName)) + if (auto subProp = findTablePropertyRespectingMeta(subTy, freeName)) { - tryUnify_(*otherProp, freeProp.type); + tryUnify_(freeProp.type, *subProp); /* * TypeVars are commonly cyclic, so it is entirely possible @@ -1508,84 +2139,133 @@ void Unifier::tryUnifyFreeTable(TypeId freeTypeId, TypeId otherTypeId) * I believe this is guaranteed to terminate eventually because this will * only happen when a free table is bound to another table. */ - if (!get(freeTypeId) || !get(otherTypeId)) - return tryUnify_(freeTypeId, otherTypeId); + if (FFlag::LuauUseCommittingTxnLog) + { + if (!log.getMutable(superTy) || !log.getMutable(subTy)) + return tryUnify_(subTy, superTy); - if (freeTable->boundTo) - return tryUnify_(freeTypeId, otherTypeId); + if (TableTypeVar* pendingFreeTtv = log.getMutable(superTy); pendingFreeTtv && pendingFreeTtv->boundTo) + return tryUnify_(subTy, superTy); + } + else + { + if (!get(superTy) || !get(subTy)) + return tryUnify_(subTy, superTy); + + if (freeTable->boundTo) + return tryUnify_(subTy, superTy); + } } else { // If the other table is also free, then we are learning that it has more // properties than we previously thought. Else, it is an error. - if (otherTable->state == TableState::Free) - otherTable->props.insert({freeName, freeProp}); + if (subTable->state == TableState::Free) + { + if (FFlag::LuauUseCommittingTxnLog) + { + PendingType* pendingSub = log.queue(subTy); + TableTypeVar* pendingSubTtv = getMutable(pendingSub); + LUAU_ASSERT(pendingSubTtv); + pendingSubTtv->props.insert({freeName, freeProp}); + } + else + { + subTable->props.insert({freeName, freeProp}); + } + } else - errors.push_back(TypeError{location, UnknownProperty{otherTypeId, freeName}}); + errors.push_back(TypeError{location, UnknownProperty{subTy, freeName}}); } } - if (freeTable->indexer && otherTable->indexer) + if (freeTable->indexer && subTable->indexer) { Unifier innerState = makeChildUnifier(); - innerState.tryUnify(*freeTable->indexer, *otherTable->indexer); + innerState.tryUnifyIndexer(*subTable->indexer, *freeTable->indexer); - checkChildUnifierTypeMismatch(innerState.errors, freeTypeId, otherTypeId); + checkChildUnifierTypeMismatch(innerState.errors, superTy, subTy); - log.concat(std::move(innerState.log)); + if (FFlag::LuauUseCommittingTxnLog) + log.concat(std::move(innerState.log)); + else + DEPRECATED_log.concat(std::move(innerState.DEPRECATED_log)); } - else if (otherTable->state == TableState::Free && freeTable->indexer) - freeTable->indexer = otherTable->indexer; - - if (!freeTable->boundTo && otherTable->state != TableState::Free) + else if (subTable->state == TableState::Free && freeTable->indexer) { - log(freeTable); - freeTable->boundTo = otherTypeId; + if (FFlag::LuauUseCommittingTxnLog) + { + log.changeIndexer(superTy, subTable->indexer); + } + else + { + freeTable->indexer = subTable->indexer; + } + } + + if (!freeTable->boundTo && subTable->state != TableState::Free) + { + if (FFlag::LuauUseCommittingTxnLog) + { + log.bindTable(superTy, subTy); + } + else + { + DEPRECATED_log(freeTable); + freeTable->boundTo = subTy; + } } } -void Unifier::tryUnifySealedTables(TypeId left, TypeId right, bool isIntersection) +void Unifier::tryUnifySealedTables(TypeId subTy, TypeId superTy, bool isIntersection) { - TableTypeVar* lt = getMutable(left); - TableTypeVar* rt = getMutable(right); - if (!lt || !rt) + TableTypeVar* superTable = getMutable(superTy); + TableTypeVar* subTable = getMutable(subTy); + + if (FFlag::LuauUseCommittingTxnLog) + { + superTable = log.getMutable(superTy); + subTable = log.getMutable(subTy); + } + + if (!superTable || !subTable) ice("passed non-table types to unifySealedTables"); Unifier innerState = makeChildUnifier(); std::vector missingPropertiesInSuper; - bool isUnnamedTable = rt->name == std::nullopt && rt->syntheticName == std::nullopt; + bool isUnnamedTable = subTable->name == std::nullopt && subTable->syntheticName == std::nullopt; bool errorReported = false; // Optimization: First test that the property sets are compatible without doing any recursive unification - if (FFlag::LuauTableUnificationEarlyTest && !rt->indexer) + if (FFlag::LuauTableUnificationEarlyTest && !subTable->indexer) { - for (const auto& [propName, superProp] : lt->props) + for (const auto& [propName, superProp] : superTable->props) { - auto subIter = rt->props.find(propName); - if (subIter == rt->props.end() && !isOptional(superProp.type)) + auto subIter = subTable->props.find(propName); + if (subIter == subTable->props.end() && !isOptional(superProp.type)) missingPropertiesInSuper.push_back(propName); } if (!missingPropertiesInSuper.empty()) { - errors.push_back(TypeError{location, MissingProperties{left, right, std::move(missingPropertiesInSuper)}}); + errors.push_back(TypeError{location, MissingProperties{superTy, subTy, std::move(missingPropertiesInSuper)}}); return; } } // Tables must have exactly the same props and their types must all unify - for (const auto& it : lt->props) + for (const auto& it : superTable->props) { - const auto& r = rt->props.find(it.first); - if (r == rt->props.end()) + const auto& r = subTable->props.find(it.first); + if (r == subTable->props.end()) { if (isOptional(it.second.type)) continue; missingPropertiesInSuper.push_back(it.first); - innerState.errors.push_back(TypeError{location, TypeMismatch{left, right}}); + innerState.errors.push_back(TypeError{location, TypeMismatch{superTy, subTy}}); } else { @@ -1594,7 +2274,7 @@ void Unifier::tryUnifySealedTables(TypeId left, TypeId right, bool isIntersectio size_t oldErrorSize = innerState.errors.size(); Location old = innerState.location; innerState.location = *r->second.location; - innerState.tryUnify_(it.second.type, r->second.type); + innerState.tryUnify_(r->second.type, it.second.type); innerState.location = old; if (oldErrorSize != innerState.errors.size() && !errorReported) @@ -1605,113 +2285,165 @@ void Unifier::tryUnifySealedTables(TypeId left, TypeId right, bool isIntersectio } else { - innerState.tryUnify_(it.second.type, r->second.type); + innerState.tryUnify_(r->second.type, it.second.type); } } } - if (lt->indexer || rt->indexer) + if (superTable->indexer || subTable->indexer) { - if (lt->indexer && rt->indexer) - innerState.tryUnify(*lt->indexer, *rt->indexer); - else if (rt->state == TableState::Unsealed) + if (FFlag::LuauUseCommittingTxnLog) { - if (lt->indexer && !rt->indexer) - rt->indexer = lt->indexer; - } - else if (lt->state == TableState::Unsealed) - { - if (rt->indexer && !lt->indexer) - lt->indexer = rt->indexer; - } - else if (lt->indexer) - { - innerState.tryUnify_(lt->indexer->indexType, getSingletonTypes().stringType); - // We already try to unify properties in both tables. - // Skip those and just look for the ones remaining and see if they fit into the indexer. - for (const auto& [name, type] : rt->props) + if (superTable->indexer && subTable->indexer) + innerState.tryUnifyIndexer(*subTable->indexer, *superTable->indexer); + else if (subTable->state == TableState::Unsealed) { - const auto& it = lt->props.find(name); - if (it == lt->props.end()) - innerState.tryUnify_(lt->indexer->indexResultType, type.type); + if (superTable->indexer && !subTable->indexer) + { + log.changeIndexer(subTy, superTable->indexer); + } } + else if (superTable->state == TableState::Unsealed) + { + if (subTable->indexer && !superTable->indexer) + { + log.changeIndexer(superTy, subTable->indexer); + } + } + else if (superTable->indexer) + { + innerState.tryUnify_(getSingletonTypes().stringType, superTable->indexer->indexType); + for (const auto& [name, type] : subTable->props) + { + const auto& it = superTable->props.find(name); + if (it == superTable->props.end()) + innerState.tryUnify_(type.type, superTable->indexer->indexResultType); + } + } + else + innerState.errors.push_back(TypeError{location, TypeMismatch{superTy, subTy}}); } else - innerState.errors.push_back(TypeError{location, TypeMismatch{left, right}}); + { + if (superTable->indexer && subTable->indexer) + innerState.tryUnifyIndexer(*subTable->indexer, *superTable->indexer); + else if (subTable->state == TableState::Unsealed) + { + if (superTable->indexer && !subTable->indexer) + subTable->indexer = superTable->indexer; + } + else if (superTable->state == TableState::Unsealed) + { + if (subTable->indexer && !superTable->indexer) + superTable->indexer = subTable->indexer; + } + else if (superTable->indexer) + { + innerState.tryUnify_(getSingletonTypes().stringType, superTable->indexer->indexType); + // We already try to unify properties in both tables. + // Skip those and just look for the ones remaining and see if they fit into the indexer. + for (const auto& [name, type] : subTable->props) + { + const auto& it = superTable->props.find(name); + if (it == superTable->props.end()) + innerState.tryUnify_(type.type, superTable->indexer->indexResultType); + } + } + else + innerState.errors.push_back(TypeError{location, TypeMismatch{superTy, subTy}}); + } } - log.concat(std::move(innerState.log)); + if (FFlag::LuauUseCommittingTxnLog) + { + if (!errorReported) + log.concat(std::move(innerState.log)); + } + else + DEPRECATED_log.concat(std::move(innerState.DEPRECATED_log)); if (errorReported) return; if (!missingPropertiesInSuper.empty()) { - errors.push_back(TypeError{location, MissingProperties{left, right, std::move(missingPropertiesInSuper)}}); + errors.push_back(TypeError{location, MissingProperties{superTy, subTy, std::move(missingPropertiesInSuper)}}); return; } - // If the superTy/left is an immediate part of an intersection type, do not do extra-property check. + // If the superTy is an immediate part of an intersection type, do not do extra-property check. // Otherwise, we would falsely generate an extra-property-error for 's' in this code: // local a: {n: number} & {s: string} = {n=1, s=""} // When checking against the table '{n: number}'. - if (!isIntersection && lt->state != TableState::Unsealed && !lt->indexer) + if (!isIntersection && superTable->state != TableState::Unsealed && !superTable->indexer) { // Check for extra properties in the subTy std::vector extraPropertiesInSub; - for (const auto& it : rt->props) + for (const auto& [subKey, subProp] : subTable->props) { - const auto& r = lt->props.find(it.first); - if (r == lt->props.end()) + const auto& superIt = superTable->props.find(subKey); + if (superIt == superTable->props.end()) { - if (isOptional(it.second.type)) + if (isOptional(subProp.type)) continue; - extraPropertiesInSub.push_back(it.first); + extraPropertiesInSub.push_back(subKey); } } if (!extraPropertiesInSub.empty()) { - errors.push_back(TypeError{location, MissingProperties{left, right, std::move(extraPropertiesInSub), MissingProperties::Extra}}); + errors.push_back(TypeError{location, MissingProperties{superTy, subTy, std::move(extraPropertiesInSub), MissingProperties::Extra}}); return; } } - checkChildUnifierTypeMismatch(innerState.errors, left, right); + checkChildUnifierTypeMismatch(innerState.errors, superTy, subTy); } -void Unifier::tryUnifyWithMetatable(TypeId metatable, TypeId other, bool reversed) +void Unifier::tryUnifyWithMetatable(TypeId subTy, TypeId superTy, bool reversed) { - const MetatableTypeVar* lhs = get(metatable); - if (!lhs) + const MetatableTypeVar* superMetatable = get(superTy); + if (!superMetatable) ice("tryUnifyMetatable invoked with non-metatable TypeVar"); - TypeError mismatchError = TypeError{location, TypeMismatch{reversed ? other : metatable, reversed ? metatable : other}}; + TypeError mismatchError = TypeError{location, TypeMismatch{reversed ? subTy : superTy, reversed ? superTy : subTy}}; - if (const MetatableTypeVar* rhs = get(other)) + if (const MetatableTypeVar* subMetatable = + FFlag::LuauUseCommittingTxnLog ? log.getMutable(subTy) : get(subTy)) { Unifier innerState = makeChildUnifier(); - innerState.tryUnify_(lhs->table, rhs->table); - innerState.tryUnify_(lhs->metatable, rhs->metatable); + innerState.tryUnify_(subMetatable->table, superMetatable->table); + innerState.tryUnify_(subMetatable->metatable, superMetatable->metatable); if (auto e = hasUnificationTooComplex(innerState.errors)) errors.push_back(*e); else if (!innerState.errors.empty()) errors.push_back( - TypeError{location, TypeMismatch{reversed ? other : metatable, reversed ? metatable : other, "", innerState.errors.front()}}); + TypeError{location, TypeMismatch{reversed ? subTy : superTy, reversed ? superTy : subTy, "", innerState.errors.front()}}); - log.concat(std::move(innerState.log)); + if (FFlag::LuauUseCommittingTxnLog) + log.concat(std::move(innerState.log)); + else + DEPRECATED_log.concat(std::move(innerState.DEPRECATED_log)); } - else if (TableTypeVar* rhs = getMutable(other)) + else if (TableTypeVar* subTable = FFlag::LuauUseCommittingTxnLog ? log.getMutable(subTy) : getMutable(subTy)) { - switch (rhs->state) + switch (subTable->state) { case TableState::Free: { - tryUnify_(lhs->table, other); - rhs->boundTo = metatable; + tryUnify_(subTy, superMetatable->table); + + if (FFlag::LuauUseCommittingTxnLog) + { + log.bindTable(subTy, superTy); + } + else + { + subTable->boundTo = superTy; + } break; } @@ -1722,7 +2454,8 @@ void Unifier::tryUnifyWithMetatable(TypeId metatable, TypeId other, bool reverse errors.push_back(mismatchError); } } - else if (get(other) || get(other)) + else if (FFlag::LuauUseCommittingTxnLog ? (log.getMutable(subTy) || log.getMutable(subTy)) + : (get(subTy) || get(subTy))) { } else @@ -1732,7 +2465,7 @@ void Unifier::tryUnifyWithMetatable(TypeId metatable, TypeId other, bool reverse } // Class unification is almost, but not quite symmetrical. We use the 'reversed' boolean to indicate which scenario we are evaluating. -void Unifier::tryUnifyWithClass(TypeId superTy, TypeId subTy, bool reversed) +void Unifier::tryUnifyWithClass(TypeId subTy, TypeId superTy, bool reversed) { if (reversed) std::swap(superTy, subTy); @@ -1763,7 +2496,7 @@ void Unifier::tryUnifyWithClass(TypeId superTy, TypeId subTy, bool reversed) } ice("Illegal variance setting!"); } - else if (TableTypeVar* table = getMutable(subTy)) + else if (TableTypeVar* subTable = getMutable(subTy)) { /** * A free table is something whose shape we do not exactly know yet. @@ -1775,12 +2508,12 @@ void Unifier::tryUnifyWithClass(TypeId superTy, TypeId subTy, bool reversed) * * Tables that are not free are known to be actual tables. */ - if (table->state != TableState::Free) + if (subTable->state != TableState::Free) return fail(); bool ok = true; - for (const auto& [propName, prop] : table->props) + for (const auto& [propName, prop] : subTable->props) { const Property* classProp = lookupClassProp(superClass, propName); if (!classProp) @@ -1791,23 +2524,37 @@ void Unifier::tryUnifyWithClass(TypeId superTy, TypeId subTy, bool reversed) else { Unifier innerState = makeChildUnifier(); - innerState.tryUnify_(prop.type, classProp->type); + innerState.tryUnify_(classProp->type, prop.type); checkChildUnifierTypeMismatch(innerState.errors, propName, reversed ? subTy : superTy, reversed ? superTy : subTy); - if (innerState.errors.empty()) + if (FFlag::LuauUseCommittingTxnLog) { - log.concat(std::move(innerState.log)); + if (innerState.errors.empty()) + { + log.concat(std::move(innerState.log)); + } + else + { + ok = false; + } } else { - ok = false; - innerState.log.rollback(); + if (innerState.errors.empty()) + { + DEPRECATED_log.concat(std::move(innerState.DEPRECATED_log)); + } + else + { + ok = false; + innerState.DEPRECATED_log.rollback(); + } } } } - if (table->indexer) + if (subTable->indexer) { ok = false; std::string msg = "Class " + superClass->name + " does not have an indexer"; @@ -1817,17 +2564,24 @@ void Unifier::tryUnifyWithClass(TypeId superTy, TypeId subTy, bool reversed) if (!ok) return; - log(table); - table->boundTo = superTy; + if (FFlag::LuauUseCommittingTxnLog) + { + log.bindTable(subTy, superTy); + } + else + { + DEPRECATED_log(subTable); + subTable->boundTo = superTy; + } } else return fail(); } -void Unifier::tryUnify(const TableIndexer& superIndexer, const TableIndexer& subIndexer) +void Unifier::tryUnifyIndexer(const TableIndexer& subIndexer, const TableIndexer& superIndexer) { - tryUnify_(superIndexer.indexType, subIndexer.indexType); - tryUnify_(superIndexer.indexResultType, subIndexer.indexResultType); + tryUnify_(subIndexer.indexType, superIndexer.indexType); + tryUnify_(subIndexer.indexResultType, superIndexer.indexResultType); } static void queueTypePack(std::vector& queue, DenseHashSet& seenTypePacks, Unifier& state, TypePackId a, TypePackId anyTypePack) @@ -1840,54 +2594,85 @@ static void queueTypePack(std::vector& queue, DenseHashSet& break; seenTypePacks.insert(a); - if (get(a)) + if (FFlag::LuauUseCommittingTxnLog) { - state.log(a); - *asMutable(a) = Unifiable::Bound{anyTypePack}; + if (state.log.getMutable(a)) + { + state.log.replace(a, Unifiable::Bound{anyTypePack}); + } + else if (auto tp = state.log.getMutable(a)) + { + queue.insert(queue.end(), tp->head.begin(), tp->head.end()); + if (tp->tail) + a = *tp->tail; + else + break; + } } - else if (auto tp = get(a)) + else { - queue.insert(queue.end(), tp->head.begin(), tp->head.end()); - if (tp->tail) - a = *tp->tail; - else - break; + if (get(a)) + { + state.DEPRECATED_log(a); + *asMutable(a) = Unifiable::Bound{anyTypePack}; + } + else if (auto tp = get(a)) + { + queue.insert(queue.end(), tp->head.begin(), tp->head.end()); + if (tp->tail) + a = *tp->tail; + else + break; + } } } } -void Unifier::tryUnifyVariadics(TypePackId superTp, TypePackId subTp, bool reversed, int subOffset) +void Unifier::tryUnifyVariadics(TypePackId subTp, TypePackId superTp, bool reversed, int subOffset) { - const VariadicTypePack* lv = get(superTp); - if (!lv) + const VariadicTypePack* superVariadic = get(superTp); + + if (FFlag::LuauUseCommittingTxnLog) + { + superVariadic = log.getMutable(superTp); + } + + if (!superVariadic) ice("passed non-variadic pack to tryUnifyVariadics"); - if (const VariadicTypePack* rv = get(subTp)) - tryUnify_(reversed ? rv->ty : lv->ty, reversed ? lv->ty : rv->ty); + if (const VariadicTypePack* subVariadic = get(subTp)) + tryUnify_(reversed ? superVariadic->ty : subVariadic->ty, reversed ? subVariadic->ty : superVariadic->ty); else if (get(subTp)) { - TypePackIterator rIter = begin(subTp); - TypePackIterator rEnd = end(subTp); + TypePackIterator subIter = begin(subTp, &log); + TypePackIterator subEnd = end(subTp); - std::advance(rIter, subOffset); + std::advance(subIter, subOffset); - while (rIter != rEnd) + while (subIter != subEnd) { - tryUnify_(reversed ? *rIter : lv->ty, reversed ? lv->ty : *rIter); - ++rIter; + tryUnify_(reversed ? superVariadic->ty : *subIter, reversed ? *subIter : superVariadic->ty); + ++subIter; } - if (std::optional maybeTail = rIter.tail()) + if (std::optional maybeTail = subIter.tail()) { TypePackId tail = follow(*maybeTail); if (get(tail)) { - log(tail); - *asMutable(tail) = BoundTypePack{superTp}; + if (FFlag::LuauUseCommittingTxnLog) + { + log.replace(tail, BoundTypePack(superTp)); + } + else + { + DEPRECATED_log(tail); + *asMutable(tail) = BoundTypePack{superTp}; + } } else if (const VariadicTypePack* vtp = get(tail)) { - tryUnify_(lv->ty, vtp->ty); + tryUnify_(vtp->ty, superVariadic->ty); } else if (get(tail)) { @@ -1914,65 +2699,113 @@ static void tryUnifyWithAny(std::vector& queue, Unifier& state, DenseHas { while (!queue.empty()) { - TypeId ty = follow(queue.back()); - queue.pop_back(); - if (seen.find(ty)) - continue; - seen.insert(ty); + if (FFlag::LuauUseCommittingTxnLog) + { + TypeId ty = state.log.follow(queue.back()); + queue.pop_back(); + if (seen.find(ty)) + continue; + seen.insert(ty); - if (get(ty)) - { - state.log(ty); - *asMutable(ty) = BoundTypeVar{anyType}; - } - else if (auto fun = get(ty)) - { - queueTypePack(queue, seenTypePacks, state, fun->argTypes, anyTypePack); - queueTypePack(queue, seenTypePacks, state, fun->retType, anyTypePack); - } - else if (auto table = get(ty)) - { - for (const auto& [_name, prop] : table->props) - queue.push_back(prop.type); - - if (table->indexer) + if (state.log.getMutable(ty)) { - queue.push_back(table->indexer->indexType); - queue.push_back(table->indexer->indexResultType); + state.log.replace(ty, BoundTypeVar{anyType}); } + else if (auto fun = state.log.getMutable(ty)) + { + queueTypePack(queue, seenTypePacks, state, fun->argTypes, anyTypePack); + queueTypePack(queue, seenTypePacks, state, fun->retType, anyTypePack); + } + else if (auto table = state.log.getMutable(ty)) + { + for (const auto& [_name, prop] : table->props) + queue.push_back(prop.type); + + if (table->indexer) + { + queue.push_back(table->indexer->indexType); + queue.push_back(table->indexer->indexResultType); + } + } + else if (auto mt = state.log.getMutable(ty)) + { + queue.push_back(mt->table); + queue.push_back(mt->metatable); + } + else if (state.log.getMutable(ty)) + { + // ClassTypeVars never contain free typevars. + } + else if (auto union_ = state.log.getMutable(ty)) + queue.insert(queue.end(), union_->options.begin(), union_->options.end()); + else if (auto intersection = state.log.getMutable(ty)) + queue.insert(queue.end(), intersection->parts.begin(), intersection->parts.end()); + else + { + } // Primitives, any, errors, and generics are left untouched. } - else if (auto mt = get(ty)) - { - queue.push_back(mt->table); - queue.push_back(mt->metatable); - } - else if (get(ty)) - { - // ClassTypeVars never contain free typevars. - } - else if (auto union_ = get(ty)) - queue.insert(queue.end(), union_->options.begin(), union_->options.end()); - else if (auto intersection = get(ty)) - queue.insert(queue.end(), intersection->parts.begin(), intersection->parts.end()); else { - } // Primitives, any, errors, and generics are left untouched. + TypeId ty = follow(queue.back()); + queue.pop_back(); + if (seen.find(ty)) + continue; + seen.insert(ty); + + if (get(ty)) + { + state.DEPRECATED_log(ty); + *asMutable(ty) = BoundTypeVar{anyType}; + } + else if (auto fun = get(ty)) + { + queueTypePack(queue, seenTypePacks, state, fun->argTypes, anyTypePack); + queueTypePack(queue, seenTypePacks, state, fun->retType, anyTypePack); + } + else if (auto table = get(ty)) + { + for (const auto& [_name, prop] : table->props) + queue.push_back(prop.type); + + if (table->indexer) + { + queue.push_back(table->indexer->indexType); + queue.push_back(table->indexer->indexResultType); + } + } + else if (auto mt = get(ty)) + { + queue.push_back(mt->table); + queue.push_back(mt->metatable); + } + else if (get(ty)) + { + // ClassTypeVars never contain free typevars. + } + else if (auto union_ = get(ty)) + queue.insert(queue.end(), union_->options.begin(), union_->options.end()); + else if (auto intersection = get(ty)) + queue.insert(queue.end(), intersection->parts.begin(), intersection->parts.end()); + else + { + } // Primitives, any, errors, and generics are left untouched. + } } } -void Unifier::tryUnifyWithAny(TypeId any, TypeId ty) +void Unifier::tryUnifyWithAny(TypeId subTy, TypeId anyTy) { - LUAU_ASSERT(get(any) || get(any)); + LUAU_ASSERT(get(anyTy) || get(anyTy)); // These types are not visited in general loop below - if (get(ty) || get(ty) || get(ty)) + if (get(subTy) || get(subTy) || get(subTy)) return; const TypePackId anyTypePack = types->addTypePack(TypePackVar{VariadicTypePack{getSingletonTypes().anyType}}); - const TypePackId anyTP = get(any) ? anyTypePack : types->addTypePack(TypePackVar{Unifiable::Error{}}); + const TypePackId anyTP = get(anyTy) ? anyTypePack : types->addTypePack(TypePackVar{Unifiable::Error{}}); - std::vector queue = {ty}; + std::vector queue = {subTy}; sharedState.tempSeenTy.clear(); sharedState.tempSeenTp.clear(); @@ -1980,9 +2813,9 @@ void Unifier::tryUnifyWithAny(TypeId any, TypeId ty) Luau::tryUnifyWithAny(queue, *this, sharedState.tempSeenTy, sharedState.tempSeenTp, getSingletonTypes().anyType, anyTP); } -void Unifier::tryUnifyWithAny(TypePackId any, TypePackId ty) +void Unifier::tryUnifyWithAny(TypePackId subTy, TypePackId anyTp) { - LUAU_ASSERT(get(any)); + LUAU_ASSERT(get(anyTp)); const TypeId anyTy = getSingletonTypes().errorRecoveryType(); @@ -1991,9 +2824,9 @@ void Unifier::tryUnifyWithAny(TypePackId any, TypePackId ty) sharedState.tempSeenTy.clear(); sharedState.tempSeenTp.clear(); - queueTypePack(queue, sharedState.tempSeenTp, *this, ty, any); + queueTypePack(queue, sharedState.tempSeenTp, *this, subTy, anyTp); - Luau::tryUnifyWithAny(queue, *this, sharedState.tempSeenTy, sharedState.tempSeenTp, anyTy, any); + Luau::tryUnifyWithAny(queue, *this, sharedState.tempSeenTy, sharedState.tempSeenTp, anyTy, anyTp); } std::optional Unifier::findTablePropertyRespectingMeta(TypeId lhsType, Name name) @@ -2012,54 +2845,105 @@ void Unifier::occursCheck(DenseHashSet& seen, TypeId needle, TypeId hays { RecursionLimiter _ra(&sharedState.counters.recursionCount, FInt::LuauTypeInferRecursionLimit); - needle = follow(needle); - haystack = follow(haystack); - - if (seen.find(haystack)) - return; - - seen.insert(haystack); - - if (get(needle)) - return; - - if (!get(needle)) - ice("Expected needle to be free"); - - if (needle == haystack) - { - errors.push_back(TypeError{location, OccursCheckFailed{}}); - log(needle); - *asMutable(needle) = *getSingletonTypes().errorRecoveryType(); - return; - } - auto check = [&](TypeId tv) { occursCheck(seen, needle, tv); }; - if (get(haystack)) - return; - else if (auto a = get(haystack)) + if (FFlag::LuauUseCommittingTxnLog) { - if (!FFlag::LuauOccursCheckOkWithRecursiveFunctions) - { - for (TypeId ty : a->argTypes) - check(ty); + needle = log.follow(needle); + haystack = log.follow(haystack); - for (TypeId ty : a->retType) + if (seen.find(haystack)) + return; + + seen.insert(haystack); + + if (log.getMutable(needle)) + return; + + if (!log.getMutable(needle)) + ice("Expected needle to be free"); + + if (needle == haystack) + { + errors.push_back(TypeError{location, OccursCheckFailed{}}); + log.replace(needle, *getSingletonTypes().errorRecoveryType()); + + return; + } + + if (log.getMutable(haystack)) + return; + else if (auto a = log.getMutable(haystack)) + { + if (!FFlag::LuauOccursCheckOkWithRecursiveFunctions) + { + for (TypePackIterator it(a->argTypes, &log); it != end(a->argTypes); ++it) + check(*it); + + for (TypePackIterator it(a->retType, &log); it != end(a->retType); ++it) + check(*it); + } + } + else if (auto a = log.getMutable(haystack)) + { + for (TypeId ty : a->options) + check(ty); + } + else if (auto a = log.getMutable(haystack)) + { + for (TypeId ty : a->parts) check(ty); } } - else if (auto a = get(haystack)) + else { - for (TypeId ty : a->options) - check(ty); - } - else if (auto a = get(haystack)) - { - for (TypeId ty : a->parts) - check(ty); + needle = follow(needle); + haystack = follow(haystack); + + if (seen.find(haystack)) + return; + + seen.insert(haystack); + + if (get(needle)) + return; + + if (!get(needle)) + ice("Expected needle to be free"); + + if (needle == haystack) + { + errors.push_back(TypeError{location, OccursCheckFailed{}}); + DEPRECATED_log(needle); + *asMutable(needle) = *getSingletonTypes().errorRecoveryType(); + return; + } + + if (get(haystack)) + return; + else if (auto a = get(haystack)) + { + if (!FFlag::LuauOccursCheckOkWithRecursiveFunctions) + { + for (TypeId ty : a->argTypes) + check(ty); + + for (TypeId ty : a->retType) + check(ty); + } + } + else if (auto a = get(haystack)) + { + for (TypeId ty : a->options) + check(ty); + } + else if (auto a = get(haystack)) + { + for (TypeId ty : a->parts) + check(ty); + } } } @@ -2072,59 +2956,115 @@ void Unifier::occursCheck(TypePackId needle, TypePackId haystack) void Unifier::occursCheck(DenseHashSet& seen, TypePackId needle, TypePackId haystack) { - needle = follow(needle); - haystack = follow(haystack); - - if (seen.find(haystack)) - return; - - seen.insert(haystack); - - if (get(needle)) - return; - - if (!get(needle)) - ice("Expected needle pack to be free"); - - RecursionLimiter _ra(&sharedState.counters.recursionCount, FInt::LuauTypeInferRecursionLimit); - - while (!get(haystack)) + if (FFlag::LuauUseCommittingTxnLog) { - if (needle == haystack) - { - errors.push_back(TypeError{location, OccursCheckFailed{}}); - log(needle); - *asMutable(needle) = *getSingletonTypes().errorRecoveryTypePack(); - return; - } + needle = log.follow(needle); + haystack = log.follow(haystack); - if (auto a = get(haystack)) + if (seen.find(haystack)) + return; + + seen.insert(haystack); + + if (log.getMutable(needle)) + return; + + if (!get(needle)) + ice("Expected needle pack to be free"); + + RecursionLimiter _ra(&sharedState.counters.recursionCount, FInt::LuauTypeInferRecursionLimit); + + while (!log.getMutable(haystack)) { - if (!FFlag::LuauOccursCheckOkWithRecursiveFunctions) + if (needle == haystack) + { + errors.push_back(TypeError{location, OccursCheckFailed{}}); + log.replace(needle, *getSingletonTypes().errorRecoveryTypePack()); + + return; + } + + if (auto a = get(haystack)) { for (const auto& ty : a->head) { - if (auto f = get(follow(ty))) + if (!FFlag::LuauOccursCheckOkWithRecursiveFunctions) { - occursCheck(seen, needle, f->argTypes); - occursCheck(seen, needle, f->retType); + if (auto f = log.getMutable(log.follow(ty))) + { + occursCheck(seen, needle, f->argTypes); + occursCheck(seen, needle, f->retType); + } } } + + if (a->tail) + { + haystack = follow(*a->tail); + continue; + } + } + break; + } + } + else + { + needle = follow(needle); + haystack = follow(haystack); + + if (seen.find(haystack)) + return; + + seen.insert(haystack); + + if (get(needle)) + return; + + if (!get(needle)) + ice("Expected needle pack to be free"); + + RecursionLimiter _ra(&sharedState.counters.recursionCount, FInt::LuauTypeInferRecursionLimit); + + while (!get(haystack)) + { + if (needle == haystack) + { + errors.push_back(TypeError{location, OccursCheckFailed{}}); + DEPRECATED_log(needle); + *asMutable(needle) = *getSingletonTypes().errorRecoveryTypePack(); } - if (a->tail) + if (auto a = get(haystack)) { - haystack = follow(*a->tail); - continue; + if (!FFlag::LuauOccursCheckOkWithRecursiveFunctions) + { + for (const auto& ty : a->head) + { + if (auto f = get(follow(ty))) + { + occursCheck(seen, needle, f->argTypes); + occursCheck(seen, needle, f->retType); + } + } + } + + if (a->tail) + { + haystack = follow(*a->tail); + continue; + } } + break; } - break; } } Unifier Unifier::makeChildUnifier() { - return Unifier{types, mode, globalScope, log.sharedSeen, location, variance, sharedState}; + if (FFlag::LuauUseCommittingTxnLog) + return Unifier{types, mode, globalScope, log.sharedSeen, location, variance, sharedState, &log}; + else + return Unifier{types, mode, globalScope, DEPRECATED_log.sharedSeen, location, variance, sharedState, &log}; } bool Unifier::isNonstrictMode() const diff --git a/Ast/include/Luau/Common.h b/Ast/include/Luau/Common.h index 63cd3df4..fbb03a9e 100644 --- a/Ast/include/Luau/Common.h +++ b/Ast/include/Luau/Common.h @@ -29,7 +29,7 @@ namespace Luau { -using AssertHandler = int (*)(const char* expression, const char* file, int line); +using AssertHandler = int (*)(const char* expression, const char* file, int line, const char* function); inline AssertHandler& assertHandler() { @@ -37,10 +37,10 @@ inline AssertHandler& assertHandler() return handler; } -inline int assertCallHandler(const char* expression, const char* file, int line) +inline int assertCallHandler(const char* expression, const char* file, int line, const char* function) { if (AssertHandler handler = assertHandler()) - return handler(expression, file, line); + return handler(expression, file, line, function); return 1; } @@ -48,7 +48,7 @@ inline int assertCallHandler(const char* expression, const char* file, int line) } // namespace Luau #if !defined(NDEBUG) || defined(LUAU_ENABLE_ASSERT) -#define LUAU_ASSERT(expr) ((void)(!!(expr) || (Luau::assertCallHandler(#expr, __FILE__, __LINE__) && (LUAU_DEBUGBREAK(), 0)))) +#define LUAU_ASSERT(expr) ((void)(!!(expr) || (Luau::assertCallHandler(#expr, __FILE__, __LINE__, __FUNCTION__) && (LUAU_DEBUGBREAK(), 0)))) #define LUAU_ASSERTENABLED #else #define LUAU_ASSERT(expr) (void)sizeof(!!(expr)) diff --git a/CLI/Analyze.cpp b/CLI/Analyze.cpp index aecb619a..54a9a26f 100644 --- a/CLI/Analyze.cpp +++ b/CLI/Analyze.cpp @@ -107,7 +107,7 @@ static void displayHelp(const char* argv0) printf(" --formatter=gnu: report analysis errors in GNU-compatible format\n"); } -static int assertionHandler(const char* expr, const char* file, int line) +static int assertionHandler(const char* expr, const char* file, int line, const char* function) { printf("%s(%d): ASSERTION FAILED: %s\n", file, line, expr); return 1; diff --git a/CLI/Repl.cpp b/CLI/Repl.cpp index 35c02f2c..26d4333a 100644 --- a/CLI/Repl.cpp +++ b/CLI/Repl.cpp @@ -235,11 +235,14 @@ static void completeIndexer(lua_State* L, const char* editBuffer, size_t start, while (lua_next(L, -2) != 0) { - // table, key, value - std::string_view key = lua_tostring(L, -2); + if (lua_type(L, -2) == LUA_TSTRING) + { + // table, key, value + std::string_view key = lua_tostring(L, -2); - if (!key.empty() && Luau::startsWith(key, prefix)) - completions.push_back(editBuffer + std::string(key.substr(prefix.size()))); + if (!key.empty() && Luau::startsWith(key, prefix)) + completions.push_back(editBuffer + std::string(key.substr(prefix.size()))); + } lua_pop(L, 1); } @@ -253,7 +256,7 @@ static void completeIndexer(lua_State* L, const char* editBuffer, size_t start, lua_rawget(L, -2); lua_remove(L, -2); - if (lua_isnil(L, -1)) + if (!lua_istable(L, -1)) break; lookup.remove_prefix(dot + 1); @@ -266,7 +269,7 @@ static void completeIndexer(lua_State* L, const char* editBuffer, size_t start, static void completeRepl(lua_State* L, const char* editBuffer, std::vector& completions) { size_t start = strlen(editBuffer); - while (start > 0 && (isalnum(editBuffer[start - 1]) || editBuffer[start - 1] == '.')) + while (start > 0 && (isalnum(editBuffer[start - 1]) || editBuffer[start - 1] == '.' || editBuffer[start - 1] == '_')) start--; // look the value up in current global table first @@ -278,6 +281,34 @@ static void completeRepl(lua_State* L, const char* editBuffer, std::vector globalState(luaL_newstate(), lua_close); @@ -292,6 +323,7 @@ static void runRepl() }); std::string buffer; + LinenoiseScopedHistory scopedHistory; for (;;) { @@ -457,7 +489,7 @@ static void displayHelp(const char* argv0) printf(" --coverage: collect code coverage while running the code and output results to coverage.out\n"); } -static int assertionHandler(const char* expr, const char* file, int line) +static int assertionHandler(const char* expr, const char* file, int line, const char* function) { printf("%s(%d): ASSERTION FAILED: %s\n", file, line, expr); return 1; diff --git a/CLI/Web.cpp b/CLI/Web.cpp index cf5c831e..416a79f2 100644 --- a/CLI/Web.cpp +++ b/CLI/Web.cpp @@ -53,30 +53,38 @@ static std::string runCode(lua_State* L, const std::string& source) lua_insert(T, 1); lua_pcall(T, n, 0, 0); } + + lua_pop(L, 1); // pop T + return std::string(); } else { std::string error; + lua_Debug ar; + if (lua_getinfo(L, 0, "sln", &ar)) + { + error += ar.short_src; + error += ':'; + error += std::to_string(ar.currentline); + error += ": "; + } + if (status == LUA_YIELD) { - error = "thread yielded unexpectedly"; + error += "thread yielded unexpectedly"; } else if (const char* str = lua_tostring(T, -1)) { - error = str; + error += str; } error += "\nstack backtrace:\n"; error += lua_debugtrace(T); - error = "Error:" + error; - - fprintf(stdout, "%s", error.c_str()); + lua_pop(L, 1); // pop T + return error; } - - lua_pop(L, 1); - return std::string(); } extern "C" const char* executeScript(const char* source) diff --git a/Compiler/include/Luau/Bytecode.h b/Compiler/include/Luau/Bytecode.h index 71631d10..d9694d7d 100644 --- a/Compiler/include/Luau/Bytecode.h +++ b/Compiler/include/Luau/Bytecode.h @@ -377,6 +377,7 @@ enum LuauBytecodeTag { // Bytecode version LBC_VERSION = 1, + LBC_VERSION_FUTURE = 2, // TODO: This will be removed in favor of LBC_VERSION with LuauBytecodeV2Force // Types of constant table entries LBC_CONSTANT_NIL = 0, LBC_CONSTANT_BOOLEAN, diff --git a/Compiler/include/Luau/BytecodeBuilder.h b/Compiler/include/Luau/BytecodeBuilder.h index d4ebad6b..287bf4ee 100644 --- a/Compiler/include/Luau/BytecodeBuilder.h +++ b/Compiler/include/Luau/BytecodeBuilder.h @@ -74,6 +74,7 @@ public: void expandJumps(); void setDebugFunctionName(StringRef name); + void setDebugFunctionLineDefined(int line); void setDebugLine(int line); void pushDebugLocal(StringRef name, uint8_t reg, uint32_t startpc, uint32_t endpc); void pushDebugUpval(StringRef name); @@ -162,6 +163,7 @@ private: bool isvararg = false; unsigned int debugname = 0; + int debuglinedefined = 0; std::string dump; std::string dumpname; diff --git a/Compiler/src/BytecodeBuilder.cpp b/Compiler/src/BytecodeBuilder.cpp index 3280c8a4..2d31c409 100644 --- a/Compiler/src/BytecodeBuilder.cpp +++ b/Compiler/src/BytecodeBuilder.cpp @@ -6,6 +6,8 @@ #include #include +LUAU_FASTFLAGVARIABLE(LuauBytecodeV2Write, false) + namespace Luau { @@ -81,6 +83,52 @@ static int getOpLength(LuauOpcode op) } } +inline bool isJumpD(LuauOpcode op) +{ + switch (op) + { + case LOP_JUMP: + case LOP_JUMPIF: + case LOP_JUMPIFNOT: + case LOP_JUMPIFEQ: + case LOP_JUMPIFLE: + case LOP_JUMPIFLT: + case LOP_JUMPIFNOTEQ: + case LOP_JUMPIFNOTLE: + case LOP_JUMPIFNOTLT: + case LOP_FORNPREP: + case LOP_FORNLOOP: + case LOP_FORGLOOP: + case LOP_FORGPREP_INEXT: + case LOP_FORGLOOP_INEXT: + case LOP_FORGPREP_NEXT: + case LOP_FORGLOOP_NEXT: + case LOP_JUMPBACK: + case LOP_JUMPIFEQK: + case LOP_JUMPIFNOTEQK: + return true; + + default: + return false; + } +} + +inline bool isSkipC(LuauOpcode op) +{ + switch (op) + { + case LOP_LOADB: + case LOP_FASTCALL: + case LOP_FASTCALL1: + case LOP_FASTCALL2: + case LOP_FASTCALL2K: + return true; + + default: + return false; + } +} + bool BytecodeBuilder::StringRef::operator==(const StringRef& other) const { return (data && other.data) ? (length == other.length && memcmp(data, other.data, length) == 0) : (data == other.data); @@ -365,13 +413,7 @@ bool BytecodeBuilder::patchJumpD(size_t jumpLabel, size_t targetLabel) unsigned int jumpInsn = insns[jumpLabel]; (void)jumpInsn; - LUAU_ASSERT(LUAU_INSN_OP(jumpInsn) == LOP_JUMP || LUAU_INSN_OP(jumpInsn) == LOP_JUMPIF || LUAU_INSN_OP(jumpInsn) == LOP_JUMPIFNOT || - LUAU_INSN_OP(jumpInsn) == LOP_JUMPIFEQ || LUAU_INSN_OP(jumpInsn) == LOP_JUMPIFLE || LUAU_INSN_OP(jumpInsn) == LOP_JUMPIFLT || - LUAU_INSN_OP(jumpInsn) == LOP_JUMPIFNOTEQ || LUAU_INSN_OP(jumpInsn) == LOP_JUMPIFNOTLE || LUAU_INSN_OP(jumpInsn) == LOP_JUMPIFNOTLT || - LUAU_INSN_OP(jumpInsn) == LOP_FORNPREP || LUAU_INSN_OP(jumpInsn) == LOP_FORNLOOP || LUAU_INSN_OP(jumpInsn) == LOP_FORGLOOP || - LUAU_INSN_OP(jumpInsn) == LOP_FORGPREP_INEXT || LUAU_INSN_OP(jumpInsn) == LOP_FORGLOOP_INEXT || - LUAU_INSN_OP(jumpInsn) == LOP_FORGPREP_NEXT || LUAU_INSN_OP(jumpInsn) == LOP_FORGLOOP_NEXT || - LUAU_INSN_OP(jumpInsn) == LOP_JUMPBACK || LUAU_INSN_OP(jumpInsn) == LOP_JUMPIFEQK || LUAU_INSN_OP(jumpInsn) == LOP_JUMPIFNOTEQK); + LUAU_ASSERT(isJumpD(LuauOpcode(LUAU_INSN_OP(jumpInsn)))); LUAU_ASSERT(LUAU_INSN_D(jumpInsn) == 0); LUAU_ASSERT(targetLabel <= insns.size()); @@ -403,8 +445,7 @@ bool BytecodeBuilder::patchSkipC(size_t jumpLabel, size_t targetLabel) unsigned int jumpInsn = insns[jumpLabel]; (void)jumpInsn; - LUAU_ASSERT(LUAU_INSN_OP(jumpInsn) == LOP_FASTCALL || LUAU_INSN_OP(jumpInsn) == LOP_FASTCALL1 || LUAU_INSN_OP(jumpInsn) == LOP_FASTCALL2 || - LUAU_INSN_OP(jumpInsn) == LOP_FASTCALL2K); + LUAU_ASSERT(isSkipC(LuauOpcode(LUAU_INSN_OP(jumpInsn)))); LUAU_ASSERT(LUAU_INSN_C(jumpInsn) == 0); int offset = int(targetLabel) - int(jumpLabel) - 1; @@ -428,6 +469,11 @@ void BytecodeBuilder::setDebugFunctionName(StringRef name) functions[currentFunction].dumpname = std::string(name.data, name.length); } +void BytecodeBuilder::setDebugFunctionLineDefined(int line) +{ + functions[currentFunction].debuglinedefined = line; +} + void BytecodeBuilder::setDebugLine(int line) { debugLine = line; @@ -464,7 +510,7 @@ uint32_t BytecodeBuilder::getDebugPC() const void BytecodeBuilder::finalize() { LUAU_ASSERT(bytecode.empty()); - bytecode = char(LBC_VERSION); + bytecode = char(FFlag::LuauBytecodeV2Write ? LBC_VERSION_FUTURE : LBC_VERSION); writeStringTable(bytecode); @@ -565,6 +611,9 @@ void BytecodeBuilder::writeFunction(std::string& ss, uint32_t id) const writeVarInt(ss, child); // debug info + if (FFlag::LuauBytecodeV2Write) + writeVarInt(ss, func.debuglinedefined); + writeVarInt(ss, func.debugname); bool hasLines = true; diff --git a/Compiler/src/Compiler.cpp b/Compiler/src/Compiler.cpp index 8f74ffed..6ae49027 100644 --- a/Compiler/src/Compiler.cpp +++ b/Compiler/src/Compiler.cpp @@ -11,7 +11,6 @@ #include LUAU_FASTFLAG(LuauIfElseExpressionBaseSupport) -LUAU_FASTFLAGVARIABLE(LuauBit32CountBuiltin, false) namespace Luau { @@ -179,6 +178,8 @@ struct Compiler if (options.optimizationLevel >= 1 && options.debugLevel >= 2) gatherConstUpvals(func); + bytecode.setDebugFunctionLineDefined(func->location.begin.line + 1); + if (options.debugLevel >= 1 && func->debugname.value) bytecode.setDebugFunctionName(sref(func->debugname)); @@ -3626,9 +3627,9 @@ struct Compiler return LBF_BIT32_RROTATE; if (builtin.method == "rshift") return LBF_BIT32_RSHIFT; - if (builtin.method == "countlz" && FFlag::LuauBit32CountBuiltin) + if (builtin.method == "countlz") return LBF_BIT32_COUNTLZ; - if (builtin.method == "countrz" && FFlag::LuauBit32CountBuiltin) + if (builtin.method == "countrz") return LBF_BIT32_COUNTRZ; } diff --git a/Sources.cmake b/Sources.cmake index a7153eb3..5dd486aa 100644 --- a/Sources.cmake +++ b/Sources.cmake @@ -125,6 +125,7 @@ target_sources(Luau.VM PRIVATE VM/src/linit.cpp VM/src/lmathlib.cpp VM/src/lmem.cpp + VM/src/lnumprint.cpp VM/src/lobject.cpp VM/src/loslib.cpp VM/src/lperf.cpp diff --git a/VM/include/luaconf.h b/VM/include/luaconf.h index a01a1481..7e0832e7 100644 --- a/VM/include/luaconf.h +++ b/VM/include/luaconf.h @@ -138,10 +138,6 @@ /* }================================================================== */ -/* Default number printing format and the string length limit */ -#define LUA_NUMBER_FMT "%.14g" -#define LUAI_MAXNUMBER2STR 32 /* 16 digits, sign, point, and \0 */ - /* @@ LUAI_USER_ALIGNMENT_T is a type that requires maximum alignment. ** CHANGE it if your system requires alignments larger than double. (For diff --git a/VM/src/lapi.cpp b/VM/src/lapi.cpp index a65b0325..c98b9590 100644 --- a/VM/src/lapi.cpp +++ b/VM/src/lapi.cpp @@ -14,8 +14,6 @@ #include -LUAU_FASTFLAG(LuauActivateBeforeExec) - const char* lua_ident = "$Lua: Lua 5.1.4 Copyright (C) 1994-2008 Lua.org, PUC-Rio $\n" "$Authors: R. Ierusalimschy, L. H. de Figueiredo & W. Celes $\n" "$URL: www.lua.org $\n"; @@ -939,21 +937,7 @@ void lua_call(lua_State* L, int nargs, int nresults) checkresults(L, nargs, nresults); func = L->top - (nargs + 1); - if (FFlag::LuauActivateBeforeExec) - { - luaD_call(L, func, nresults); - } - else - { - int oldactive = luaC_threadactive(L); - l_setbit(L->stackstate, THREAD_ACTIVEBIT); - luaC_checkthreadsleep(L); - - luaD_call(L, func, nresults); - - if (!oldactive) - resetbit(L->stackstate, THREAD_ACTIVEBIT); - } + luaD_call(L, func, nresults); adjustresults(L, nresults); return; @@ -994,21 +978,7 @@ int lua_pcall(lua_State* L, int nargs, int nresults, int errfunc) c.func = L->top - (nargs + 1); /* function to be called */ c.nresults = nresults; - if (FFlag::LuauActivateBeforeExec) - { - status = luaD_pcall(L, f_call, &c, savestack(L, c.func), func); - } - else - { - int oldactive = luaC_threadactive(L); - l_setbit(L->stackstate, THREAD_ACTIVEBIT); - luaC_checkthreadsleep(L); - - status = luaD_pcall(L, f_call, &c, savestack(L, c.func), func); - - if (!oldactive) - resetbit(L->stackstate, THREAD_ACTIVEBIT); - } + status = luaD_pcall(L, f_call, &c, savestack(L, c.func), func); adjustresults(L, nresults); return status; diff --git a/VM/src/laux.cpp b/VM/src/laux.cpp index 7ed2a62e..71975a52 100644 --- a/VM/src/laux.cpp +++ b/VM/src/laux.cpp @@ -7,9 +7,12 @@ #include "lstring.h" #include "lapi.h" #include "lgc.h" +#include "lnumutils.h" #include +LUAU_FASTFLAG(LuauSchubfach) + /* convert a stack index to positive */ #define abs_index(L, i) ((i) > 0 || (i) <= LUA_REGISTRYINDEX ? (i) : lua_gettop(L) + (i) + 1) @@ -477,7 +480,17 @@ const char* luaL_tolstring(lua_State* L, int idx, size_t* len) switch (lua_type(L, idx)) { case LUA_TNUMBER: - lua_pushstring(L, lua_tostring(L, idx)); + if (FFlag::LuauSchubfach) + { + double n = lua_tonumber(L, idx); + char s[LUAI_MAXNUM2STR]; + char* e = luai_num2str(s, n); + lua_pushlstring(L, s, e - s); + } + else + { + lua_pushstring(L, lua_tostring(L, idx)); + } break; case LUA_TSTRING: lua_pushvalue(L, idx); @@ -491,11 +504,30 @@ const char* luaL_tolstring(lua_State* L, int idx, size_t* len) case LUA_TVECTOR: { const float* v = lua_tovector(L, idx); + + if (FFlag::LuauSchubfach) + { + char s[LUAI_MAXNUM2STR * LUA_VECTOR_SIZE]; + char* e = s; + for (int i = 0; i < LUA_VECTOR_SIZE; ++i) + { + if (i != 0) + { + *e++ = ','; + *e++ = ' '; + } + e = luai_num2str(e, v[i]); + } + lua_pushlstring(L, s, e - s); + } + else + { #if LUA_VECTOR_SIZE == 4 - lua_pushfstring(L, LUA_NUMBER_FMT ", " LUA_NUMBER_FMT ", " LUA_NUMBER_FMT ", " LUA_NUMBER_FMT, v[0], v[1], v[2], v[3]); + lua_pushfstring(L, LUA_NUMBER_FMT ", " LUA_NUMBER_FMT ", " LUA_NUMBER_FMT ", " LUA_NUMBER_FMT, v[0], v[1], v[2], v[3]); #else - lua_pushfstring(L, LUA_NUMBER_FMT ", " LUA_NUMBER_FMT ", " LUA_NUMBER_FMT, v[0], v[1], v[2]); + lua_pushfstring(L, LUA_NUMBER_FMT ", " LUA_NUMBER_FMT ", " LUA_NUMBER_FMT, v[0], v[1], v[2]); #endif + } break; } default: diff --git a/VM/src/lbitlib.cpp b/VM/src/lbitlib.cpp index 8b511edf..093400f2 100644 --- a/VM/src/lbitlib.cpp +++ b/VM/src/lbitlib.cpp @@ -5,8 +5,6 @@ #include "lcommon.h" #include "lnumutils.h" -LUAU_FASTFLAGVARIABLE(LuauBit32Count, false) - #define ALLONES ~0u #define NBITS int(8 * sizeof(unsigned)) @@ -182,9 +180,6 @@ static int b_replace(lua_State* L) static int b_countlz(lua_State* L) { - if (!FFlag::LuauBit32Count) - luaL_error(L, "bit32.countlz isn't enabled"); - b_uint v = luaL_checkunsigned(L, 1); b_uint r = NBITS; @@ -201,9 +196,6 @@ static int b_countlz(lua_State* L) static int b_countrz(lua_State* L) { - if (!FFlag::LuauBit32Count) - luaL_error(L, "bit32.countrz isn't enabled"); - b_uint v = luaL_checkunsigned(L, 1); b_uint r = NBITS; diff --git a/VM/src/ldebug.cpp b/VM/src/ldebug.cpp index 9fe1885f..2b5382bb 100644 --- a/VM/src/ldebug.cpp +++ b/VM/src/ldebug.cpp @@ -12,6 +12,9 @@ #include #include +LUAU_FASTFLAG(LuauBytecodeV2Read) +LUAU_FASTFLAG(LuauBytecodeV2Force) + static const char* getfuncname(Closure* f); static int currentpc(lua_State* L, CallInfo* ci) @@ -89,6 +92,16 @@ const char* lua_setlocal(lua_State* L, int level, int n) return name; } +static int getlinedefined(Proto* p) +{ + if (FFlag::LuauBytecodeV2Force) + return p->linedefined; + else if (FFlag::LuauBytecodeV2Read && p->linedefined >= 0) + return p->linedefined; + else + return luaG_getline(p, 0); +} + static int auxgetinfo(lua_State* L, const char* what, lua_Debug* ar, Closure* f, CallInfo* ci) { int status = 1; @@ -108,7 +121,7 @@ static int auxgetinfo(lua_State* L, const char* what, lua_Debug* ar, Closure* f, { ar->source = getstr(f->l.p->source); ar->what = "Lua"; - ar->linedefined = luaG_getline(f->l.p, 0); + ar->linedefined = getlinedefined(f->l.p); } luaO_chunkid(ar->short_src, ar->source, LUA_IDSIZE); break; @@ -121,7 +134,7 @@ static int auxgetinfo(lua_State* L, const char* what, lua_Debug* ar, Closure* f, } else { - ar->currentline = f->isC ? -1 : luaG_getline(f->l.p, 0); + ar->currentline = f->isC ? -1 : getlinedefined(f->l.p); } break; diff --git a/VM/src/ldo.cpp b/VM/src/ldo.cpp index 62bbdb7c..eb47971a 100644 --- a/VM/src/ldo.cpp +++ b/VM/src/ldo.cpp @@ -19,7 +19,6 @@ LUAU_FASTFLAGVARIABLE(LuauCcallRestoreFix, false) LUAU_FASTFLAG(LuauCoroutineClose) -LUAU_FASTFLAGVARIABLE(LuauActivateBeforeExec, false) /* ** {====================================================== @@ -228,21 +227,14 @@ void luaD_call(lua_State* L, StkId func, int nResults) { /* is a Lua function? */ L->ci->flags |= LUA_CALLINFO_RETURN; /* luau_execute will stop after returning from the stack frame */ - if (FFlag::LuauActivateBeforeExec) - { - int oldactive = luaC_threadactive(L); - l_setbit(L->stackstate, THREAD_ACTIVEBIT); - luaC_checkthreadsleep(L); + int oldactive = luaC_threadactive(L); + l_setbit(L->stackstate, THREAD_ACTIVEBIT); + luaC_checkthreadsleep(L); - luau_execute(L); /* call it */ + luau_execute(L); /* call it */ - if (!oldactive) - resetbit(L->stackstate, THREAD_ACTIVEBIT); - } - else - { - luau_execute(L); /* call it */ - } + if (!oldactive) + resetbit(L->stackstate, THREAD_ACTIVEBIT); } L->nCcalls--; luaC_checkGC(L); @@ -549,12 +541,9 @@ int luaD_pcall(lua_State* L, Pfunc func, void* u, ptrdiff_t old_top, ptrdiff_t e status = LUA_ERRERR; } - if (FFlag::LuauActivateBeforeExec) - { - // since the call failed with an error, we might have to reset the 'active' thread state - if (!oldactive) - resetbit(L->stackstate, THREAD_ACTIVEBIT); - } + // since the call failed with an error, we might have to reset the 'active' thread state + if (!oldactive) + resetbit(L->stackstate, THREAD_ACTIVEBIT); if (FFlag::LuauCcallRestoreFix) { diff --git a/VM/src/lgc.cpp b/VM/src/lgc.cpp index 7393fc74..76ef7a06 100644 --- a/VM/src/lgc.cpp +++ b/VM/src/lgc.cpp @@ -12,8 +12,6 @@ #include -LUAU_FASTFLAG(LuauArrayBoundary) - #define GC_SWEEPMAX 40 #define GC_SWEEPCOST 10 diff --git a/VM/src/lgcdebug.cpp b/VM/src/lgcdebug.cpp index f6f7a878..c66de9c1 100644 --- a/VM/src/lgcdebug.cpp +++ b/VM/src/lgcdebug.cpp @@ -12,8 +12,6 @@ #include #include -LUAU_FASTFLAG(LuauArrayBoundary) - static void validateobjref(global_State* g, GCObject* f, GCObject* t) { LUAU_ASSERT(!isdead(g, t)); @@ -38,10 +36,7 @@ static void validatetable(global_State* g, Table* h) { int sizenode = 1 << h->lsizenode; - if (FFlag::LuauArrayBoundary) - LUAU_ASSERT(h->lastfree <= sizenode); - else - LUAU_ASSERT(h->lastfree >= 0 && h->lastfree <= sizenode); + LUAU_ASSERT(h->lastfree <= sizenode); if (h->metatable) validateobjref(g, obj2gco(h), obj2gco(h->metatable)); diff --git a/VM/src/lnumprint.cpp b/VM/src/lnumprint.cpp new file mode 100644 index 00000000..2fd0f1bb --- /dev/null +++ b/VM/src/lnumprint.cpp @@ -0,0 +1,375 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +// This code is based on Lua 5.x implementation licensed under MIT License; see lua_LICENSE.txt for details +#include "luaconf.h" +#include "lnumutils.h" + +#include "lcommon.h" + +#include +#include // TODO: Remove with LuauSchubfach + +#ifdef _MSC_VER +#include +#endif + +// This work is based on: +// Raffaello Giulietti. The Schubfach way to render doubles. 2021 +// https://drive.google.com/file/d/1IEeATSVnEE6TkrHlCYNY2GjaraBjOT4f/edit + +// The code uses the notation from the paper for local variables where appropriate, and refers to paper sections/figures/results. + +LUAU_FASTFLAGVARIABLE(LuauSchubfach, false) + +// 9.8.2. Precomputed table for 128-bit overestimates of powers of 10 (see figure 3 for table bounds) +// To avoid storing 616 128-bit numbers directly we use a technique inspired by Dragonbox implementation and store 16 consecutive +// powers using a 128-bit baseline and a bitvector with 1-bit scale and 3-bit offset for the delta between each entry and base*5^k +static const int kPow10TableMin = -292; +static const int kPow10TableMax = 324; + +// clang-format off +static const uint64_t kPow5Table[16] = { + 0x8000000000000000, 0xa000000000000000, 0xc800000000000000, 0xfa00000000000000, 0x9c40000000000000, 0xc350000000000000, + 0xf424000000000000, 0x9896800000000000, 0xbebc200000000000, 0xee6b280000000000, 0x9502f90000000000, 0xba43b74000000000, + 0xe8d4a51000000000, 0x9184e72a00000000, 0xb5e620f480000000, 0xe35fa931a0000000, +}; +static const uint64_t kPow10Table[(kPow10TableMax - kPow10TableMin + 1 + 15) / 16][3] = { + {0xff77b1fcbebcdc4f, 0x25e8e89c13bb0f7b, 0x333443443333443b}, {0x8dd01fad907ffc3b, 0xae3da7d97f6792e4, 0xbbb3ab3cb3ba3cbc}, + {0x9d71ac8fada6c9b5, 0x6f773fc3603db4aa, 0x4ba4bc4bb4bb4bcc}, {0xaecc49914078536d, 0x58fae9f773886e19, 0x3ba3bc33b43b43bb}, + {0xc21094364dfb5636, 0x985915fc12f542e5, 0x33b43b43a33b33cb}, {0xd77485cb25823ac7, 0x7d633293366b828c, 0x34b44c444343443c}, + {0xef340a98172aace4, 0x86fb897116c87c35, 0x333343333343334b}, {0x84c8d4dfd2c63f3b, 0x29ecd9f40041e074, 0xccaccbbcbcbb4bbc}, + {0x936b9fcebb25c995, 0xcab10dd900beec35, 0x3ab3ab3ab3bb3bbb}, {0xa3ab66580d5fdaf5, 0xc13e60d0d2e0ebbb, 0x4cc3dc4db4db4dbb}, + {0xb5b5ada8aaff80b8, 0x0d819992132456bb, 0x33b33a34c33b34ab}, {0xc9bcff6034c13052, 0xfc89b393dd02f0b6, 0x33c33b44b43c34bc}, + {0xdff9772470297ebd, 0x59787e2b93bc56f8, 0x43b444444443434c}, {0xf8a95fcf88747d94, 0x75a44c6397ce912b, 0x443334343443343b}, + {0x8a08f0f8bf0f156b, 0x1b8e9ecb641b5900, 0xbbabab3aa3ab4ccc}, {0x993fe2c6d07b7fab, 0xe546a8038efe402a, 0x4cb4bc4db4db4bcc}, + {0xaa242499697392d2, 0xdde50bd1d5d0b9ea, 0x3ba3ba3bb33b33bc}, {0xbce5086492111aea, 0x88f4bb1ca6bcf585, 0x44b44c44c44c43cb}, + {0xd1b71758e219652b, 0xd3c36113404ea4a9, 0x44c44c44c444443b}, {0xe8d4a51000000000, 0x0000000000000000, 0x444444444444444c}, + {0x813f3978f8940984, 0x4000000000000000, 0xcccccccccccccccc}, {0x8f7e32ce7bea5c6f, 0xe4820023a2000000, 0xbba3bc4cc4cc4ccc}, + {0x9f4f2726179a2245, 0x01d762422c946591, 0x4aa3bb3aa3ba3bab}, {0xb0de65388cc8ada8, 0x3b25a55f43294bcc, 0x3ca33b33b44b43bc}, + {0xc45d1df942711d9a, 0x3ba5d0bd324f8395, 0x44c44c34c44b44cb}, {0xda01ee641a708de9, 0xe80e6f4820cc9496, 0x33b33b343333333c}, + {0xf209787bb47d6b84, 0xc0678c5dbd23a49b, 0x443444444443443b}, {0x865b86925b9bc5c2, 0x0b8a2392ba45a9b3, 0xdbccbcccb4cb3bbb}, + {0x952ab45cfa97a0b2, 0xdd945a747bf26184, 0x3bc4bb4ab3ca3cbc}, {0xa59bc234db398c25, 0x43fab9837e699096, 0x3bb3ac3ab3bb33ac}, + {0xb7dcbf5354e9bece, 0x0c11ed6d538aeb30, 0x33b43b43b34c34dc}, {0xcc20ce9bd35c78a5, 0x31ec038df7b441f5, 0x34c44c43c44b44cb}, + {0xe2a0b5dc971f303a, 0x2e44ae64840fd61e, 0x333333333333333c}, {0xfb9b7cd9a4a7443c, 0x169840ef017da3b2, 0x433344443333344c}, + {0x8bab8eefb6409c1a, 0x1ad089b6c2f7548f, 0xdcbdcc3cc4cc4bcb}, {0x9b10a4e5e9913128, 0xca7cf2b4191c8327, 0x3ab3cb3bc3bb4bbb}, + {0xac2820d9623bf429, 0x546345fa9fbdcd45, 0x3bb3cc43c43c43cb}, {0xbf21e44003acdd2c, 0xe0470a63e6bd56c4, 0x44b34a43b44c44bc}, + {0xd433179d9c8cb841, 0x5fa60692a46151ec, 0x43a33a33a333333c}, +}; +// clang-format on + +static const char kDigitTable[] = "0001020304050607080910111213141516171819202122232425262728293031323334353637383940414243444546474849" + "5051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899"; + +// x*y => 128-bit product (lo+hi) +inline uint64_t mul128(uint64_t x, uint64_t y, uint64_t* hi) +{ +#if defined(_MSC_VER) && defined(_M_X64) + return _umul128(x, y, hi); +#elif defined(__SIZEOF_INT128__) + unsigned __int128 r = x; + r *= y; + *hi = uint64_t(r >> 64); + return uint64_t(r); +#else + uint32_t x0 = uint32_t(x), x1 = uint32_t(x >> 32); + uint32_t y0 = uint32_t(y), y1 = uint32_t(y >> 32); + uint64_t p11 = uint64_t(x1) * y1, p01 = uint64_t(x0) * y1; + uint64_t p10 = uint64_t(x1) * y0, p00 = uint64_t(x0) * y0; + uint64_t mid = p10 + (p00 >> 32) + uint32_t(p01); + uint64_t r0 = (mid << 32) | uint32_t(p00); + uint64_t r1 = p11 + (mid >> 32) + (p01 >> 32); + *hi = r1; + return r0; +#endif +} + +// (x*y)>>64 => 128-bit product (lo+hi) +inline uint64_t mul192hi(uint64_t xhi, uint64_t xlo, uint64_t y, uint64_t* hi) +{ + uint64_t z2; + uint64_t z1 = mul128(xhi, y, &z2); + + uint64_t z1c; + uint64_t z0 = mul128(xlo, y, &z1c); + (void)z0; + + z1 += z1c; + z2 += (z1 < z1c); + + *hi = z2; + return z1; +} + +// 9.3. Rounding to odd (+ figure 8 + result 23) +inline uint64_t roundodd(uint64_t ghi, uint64_t glo, uint64_t cp) +{ + uint64_t xhi; + uint64_t xlo = mul128(glo, cp, &xhi); + (void)xlo; + + uint64_t yhi; + uint64_t ylo = mul128(ghi, cp, &yhi); + + uint64_t z = ylo + xhi; + return (yhi + (z < xhi)) | (z > 1); +} + +struct Decimal +{ + uint64_t s; + int k; +}; + +static Decimal schubfach(int exponent, uint64_t fraction) +{ + // Extract c & q such that c*2^q == |v| + uint64_t c = fraction; + int q = exponent - 1023 - 51; + + if (exponent != 0) // normal numbers have implicit leading 1 + { + c |= (1ull << 52); + q--; + } + + // 8.3. Fast path for integers + if (unsigned(-q) < 53 && (c & ((1ull << (-q)) - 1)) == 0) + return {c >> (-q), 0}; + + // 5. Rounding interval + int irr = (c == (1ull << 52) && q != -1074); // Qmin + int out = int(c & 1); + + // 9.8.1. Boundaries for c + uint64_t cbl = 4 * c - 2 + irr; + uint64_t cb = 4 * c; + uint64_t cbr = 4 * c + 2; + + // 9.1. Computing k and h + const int Q = 20; + const int C = 315652; // floor(2^Q * log10(2)) + const int A = -131008; // floor(2^Q * log10(3/4)) + const int C2 = 3483294; // floor(2^Q * log2(10)) + int k = (q * C + (irr ? A : 0)) >> Q; + int h = q + ((-k * C2) >> Q) + 1; // see (9) in 9.9 + + // 9.8.2. Overestimates of powers of 10 + // Recover 10^-k fraction using compact tables generated by tools/numutils.py + // The 128-bit fraction is encoded as 128-bit baseline * power-of-5 * scale + offset + LUAU_ASSERT(-k >= kPow10TableMin && -k <= kPow10TableMax); + int gtoff = -k - kPow10TableMin; + const uint64_t* gt = kPow10Table[gtoff >> 4]; + + uint64_t ghi; + uint64_t glo = mul192hi(gt[0], gt[1], kPow5Table[gtoff & 15], &ghi); + + // Apply 1-bit scale + 3-bit offset; note, offset is intentionally applied without carry, numutils.py validates that this is sufficient + int gterr = (gt[2] >> ((gtoff & 15) * 4)) & 15; + int gtscale = gterr >> 3; + + ghi <<= gtscale; + ghi += (glo >> 63) & gtscale; + glo <<= gtscale; + glo -= (gterr & 7) - 4; + + // 9.9. Boundaries for v + uint64_t vbl = roundodd(ghi, glo, cbl << h); + uint64_t vb = roundodd(ghi, glo, cb << h); + uint64_t vbr = roundodd(ghi, glo, cbr << h); + + // Main algorithm; see figure 7 + figure 9 + uint64_t s = vb / 4; + + if (s >= 10) + { + uint64_t sp = s / 10; + + bool upin = vbl + out <= 40 * sp; + bool wpin = vbr >= 40 * sp + 40 + out; + + if (upin != wpin) + return {sp + wpin, k + 1}; + } + + // Figure 7 contains the algorithm to select between u (s) and w (s+1) + // rup computes the last 4 conditions in that algorithm + // rup is only used when uin == win, but since these branches predict poorly we use branchless selects + bool uin = vbl + out <= 4 * s; + bool win = 4 * s + 4 + out <= vbr; + bool rup = vb >= 4 * s + 2 + 1 - (s & 1); + + return {s + (uin != win ? win : rup), k}; +} + +static char* printspecial(char* buf, int sign, uint64_t fraction) +{ + if (fraction == 0) + { + memcpy(buf, ("-inf") + (1 - sign), 4); + return buf + 3 + sign; + } + else + { + memcpy(buf, "nan", 4); + return buf + 3; + } +} + +static char* printunsignedrev(char* end, uint64_t num) +{ + while (num >= 10000) + { + unsigned int tail = unsigned(num % 10000); + + memcpy(end - 4, &kDigitTable[int(tail / 100) * 2], 2); + memcpy(end - 2, &kDigitTable[int(tail % 100) * 2], 2); + num /= 10000; + end -= 4; + } + + unsigned int rest = unsigned(num); + + while (rest >= 10) + { + memcpy(end - 2, &kDigitTable[int(rest % 100) * 2], 2); + rest /= 100; + end -= 2; + } + + if (rest) + { + end[-1] = '0' + int(rest); + end -= 1; + } + + return end; +} + +static char* printexp(char* buf, int num) +{ + *buf++ = 'e'; + *buf++ = num < 0 ? '-' : '+'; + + int v = num < 0 ? -num : num; + + if (v >= 100) + { + *buf++ = '0' + (v / 100); + v %= 100; + } + + memcpy(buf, &kDigitTable[v * 2], 2); + return buf + 2; +} + +inline char* trimzero(char* end) +{ + while (end[-1] == '0') + end--; + + return end; +} + +// We use fixed-length memcpy/memset since they lower to fast SIMD+scalar writes; the target buffers should have padding space +#define fastmemcpy(dst, src, size, sizefast) check_exp((size) <= sizefast, memcpy(dst, src, sizefast)) +#define fastmemset(dst, val, size, sizefast) check_exp((size) <= sizefast, memset(dst, val, sizefast)) + +char* luai_num2str(char* buf, double n) +{ + if (!FFlag::LuauSchubfach) + { + snprintf(buf, LUAI_MAXNUM2STR, LUA_NUMBER_FMT, n); + return buf + strlen(buf); + } + + // IEEE-754 + union + { + double v; + uint64_t bits; + } v = {n}; + int sign = int(v.bits >> 63); + int exponent = int(v.bits >> 52) & 2047; + uint64_t fraction = v.bits & ((1ull << 52) - 1); + + // specials + if (LUAU_UNLIKELY(exponent == 0x7ff)) + return printspecial(buf, sign, fraction); + + // sign bit + *buf = '-'; + buf += sign; + + // zero + if (exponent == 0 && fraction == 0) + { + buf[0] = '0'; + return buf + 1; + } + + // convert binary to decimal using Schubfach + Decimal d = schubfach(exponent, fraction); + LUAU_ASSERT(d.s < uint64_t(1e17)); + + // print the decimal to a temporary buffer; we'll need to insert the decimal point and figure out the format + char decbuf[40]; + char* decend = decbuf + 20; // significand needs at most 17 digits; the rest of the buffer may be copied using fixed length memcpy + char* dec = printunsignedrev(decend, d.s); + + int declen = int(decend - dec); + LUAU_ASSERT(declen <= 17); + + int dot = declen + d.k; + + // the limits are somewhat arbitrary but changing them may require changing fastmemset/fastmemcpy sizes below + if (dot >= -5 && dot <= 21) + { + // fixed point format + if (dot <= 0) + { + buf[0] = '0'; + buf[1] = '.'; + + fastmemset(buf + 2, '0', -dot, 5); + fastmemcpy(buf + 2 + (-dot), dec, declen, 17); + + return trimzero(buf + 2 + (-dot) + declen); + } + else if (dot == declen) + { + // no dot + fastmemcpy(buf, dec, dot, 17); + + return buf + dot; + } + else if (dot < declen) + { + // dot in the middle + fastmemcpy(buf, dec, dot, 16); + + buf[dot] = '.'; + + fastmemcpy(buf + dot + 1, dec + dot, declen - dot, 16); + + return trimzero(buf + declen + 1); + } + else + { + // no dot, zero padding + fastmemcpy(buf, dec, declen, 17); + fastmemset(buf + declen, '0', dot - declen, 8); + + return buf + dot; + } + } + else + { + // scientific format + buf[0] = dec[0]; + buf[1] = '.'; + fastmemcpy(buf + 2, dec + 1, declen - 1, 16); + + char* exp = trimzero(buf + declen + 1); + + return printexp(exp, dot - 1); + } +} diff --git a/VM/src/lnumutils.h b/VM/src/lnumutils.h index 67f832dc..fba07bc3 100644 --- a/VM/src/lnumutils.h +++ b/VM/src/lnumutils.h @@ -3,7 +3,6 @@ #pragma once #include -#include #define luai_numadd(a, b) ((a) + (b)) #define luai_numsub(a, b) ((a) - (b)) @@ -56,5 +55,9 @@ LUAU_FASTMATH_END #define luai_num2unsigned(i, n) ((i) = (unsigned)(long long)(n)) #endif -#define luai_num2str(s, n) snprintf((s), sizeof(s), LUA_NUMBER_FMT, (n)) +#define LUA_NUMBER_FMT "%.14g" /* TODO: Remove with LuauSchubfach */ +#define LUAI_MAXNUM2STR 48 + +LUAI_FUNC char* luai_num2str(char* buf, double n); + #define luai_str2num(s, p) strtod((s), (p)) diff --git a/VM/src/lobject.h b/VM/src/lobject.h index fd0a15b7..b642cf78 100644 --- a/VM/src/lobject.h +++ b/VM/src/lobject.h @@ -289,6 +289,7 @@ typedef struct Proto int sizek; int sizelineinfo; int linegaplog2; + int linedefined; uint8_t nups; /* number of upvalues */ diff --git a/VM/src/ltable.cpp b/VM/src/ltable.cpp index 0b55fcea..83b59f3f 100644 --- a/VM/src/ltable.cpp +++ b/VM/src/ltable.cpp @@ -24,8 +24,6 @@ #include -LUAU_FASTFLAGVARIABLE(LuauArrayBoundary, false) - // max size of both array and hash part is 2^MAXBITS #define MAXBITS 26 #define MAXSIZE (1 << MAXBITS) @@ -222,7 +220,7 @@ int luaH_next(lua_State* L, Table* t, StkId key) #define maybesetaboundary(t, boundary) \ { \ - if (FFlag::LuauArrayBoundary && t->aboundary <= 0) \ + if (t->aboundary <= 0) \ t->aboundary = -int(boundary); \ } @@ -705,7 +703,7 @@ int luaH_getn(Table* t) { int boundary = getaboundary(t); - if (FFlag::LuauArrayBoundary && boundary > 0) + if (boundary > 0) { if (!ttisnil(&t->array[t->sizearray - 1]) && t->node == dummynode) return t->sizearray; /* fast-path: the end of the array in `t' already refers to a boundary */ diff --git a/VM/src/lvmload.cpp b/VM/src/lvmload.cpp index add3588d..cdb276c0 100644 --- a/VM/src/lvmload.cpp +++ b/VM/src/lvmload.cpp @@ -13,6 +13,9 @@ #include +LUAU_FASTFLAGVARIABLE(LuauBytecodeV2Read, false) +LUAU_FASTFLAGVARIABLE(LuauBytecodeV2Force, false) + // TODO: RAII deallocation doesn't work for longjmp builds if a memory error happens template struct TempBuffer @@ -146,15 +149,19 @@ int luau_load(lua_State* L, const char* chunkname, const char* data, size_t size uint8_t version = read(data, size, offset); // 0 means the rest of the bytecode is the error message - if (version == 0 || version != LBC_VERSION) + if (version == 0) { char chunkid[LUA_IDSIZE]; luaO_chunkid(chunkid, chunkname, LUA_IDSIZE); + lua_pushfstring(L, "%s%.*s", chunkid, int(size - offset), data + offset); + return 1; + } - if (version == 0) - lua_pushfstring(L, "%s%.*s", chunkid, int(size - offset), data + offset); - else - lua_pushfstring(L, "%s: bytecode version mismatch", chunkid); + if (FFlag::LuauBytecodeV2Force ? (version != LBC_VERSION_FUTURE) : FFlag::LuauBytecodeV2Read ? (version != LBC_VERSION && version != LBC_VERSION_FUTURE) : (version != LBC_VERSION)) + { + char chunkid[LUA_IDSIZE]; + luaO_chunkid(chunkid, chunkname, LUA_IDSIZE); + lua_pushfstring(L, "%s: bytecode version mismatch (expected %d, got %d)", chunkid, FFlag::LuauBytecodeV2Force ? LBC_VERSION_FUTURE : LBC_VERSION, version); return 1; } @@ -285,6 +292,11 @@ int luau_load(lua_State* L, const char* chunkname, const char* data, size_t size p->p[j] = protos[fid]; } + if (FFlag::LuauBytecodeV2Force || (FFlag::LuauBytecodeV2Read && version == LBC_VERSION_FUTURE)) + p->linedefined = readVarInt(data, size, offset); + else + p->linedefined = -1; + p->debugname = readString(strings, data, size, offset); uint8_t lineinfo = read(data, size, offset); @@ -307,11 +319,11 @@ int luau_load(lua_State* L, const char* chunkname, const char* data, size_t size p->lineinfo[j] = lastoffset; } - int lastLine = 0; + int lastline = 0; for (int j = 0; j < intervals; ++j) { - lastLine += read(data, size, offset); - p->abslineinfo[j] = lastLine; + lastline += read(data, size, offset); + p->abslineinfo[j] = lastline; } } diff --git a/VM/src/lvmutils.cpp b/VM/src/lvmutils.cpp index 5d802277..31dd59c8 100644 --- a/VM/src/lvmutils.cpp +++ b/VM/src/lvmutils.cpp @@ -34,10 +34,11 @@ int luaV_tostring(lua_State* L, StkId obj) return 0; else { - char s[LUAI_MAXNUMBER2STR]; + char s[LUAI_MAXNUM2STR]; double n = nvalue(obj); - luai_num2str(s, n); - setsvalue2s(L, obj, luaS_new(L, s)); + char* e = luai_num2str(s, n); + LUAU_ASSERT(e < s + sizeof(s)); + setsvalue2s(L, obj, luaS_newlstr(L, s, e - s)); return 1; } } diff --git a/fuzz/number.cpp b/fuzz/number.cpp new file mode 100644 index 00000000..70447409 --- /dev/null +++ b/fuzz/number.cpp @@ -0,0 +1,35 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +#include "Luau/Common.h" + +#include +#include +#include +#include + +LUAU_FASTFLAG(LuauSchubfach); + +#define LUAI_MAXNUM2STR 48 + +char* luai_num2str(char* buf, double n); + +extern "C" int LLVMFuzzerTestOneInput(const uint8_t* Data, size_t Size) +{ + if (Size < 8) + return 0; + + FFlag::LuauSchubfach.value = true; + + double num; + memcpy(&num, Data, 8); + + char buf[LUAI_MAXNUM2STR]; + char* end = luai_num2str(buf, num); + LUAU_ASSERT(end < buf + sizeof(buf)); + + *end = 0; + + double rec = strtod(buf, nullptr); + + LUAU_ASSERT(rec == num || (rec != rec && num != num)); + return 0; +} diff --git a/tests/AstQuery.test.cpp b/tests/AstQuery.test.cpp index 2090b014..41b553b5 100644 --- a/tests/AstQuery.test.cpp +++ b/tests/AstQuery.test.cpp @@ -83,8 +83,6 @@ TEST_SUITE_BEGIN("AstQuery"); TEST_CASE_FIXTURE(Fixture, "last_argument_function_call_type") { - ScopedFastFlag luauTailArgumentTypeInfo{"LuauTailArgumentTypeInfo", true}; - check(R"( local function foo() return 2 end local function bar(a: number) return -a end diff --git a/tests/Autocomplete.test.cpp b/tests/Autocomplete.test.cpp index 8ca09c0e..210db7ee 100644 --- a/tests/Autocomplete.test.cpp +++ b/tests/Autocomplete.test.cpp @@ -15,6 +15,7 @@ LUAU_FASTFLAG(LuauTraceTypesInNonstrictMode2) LUAU_FASTFLAG(LuauSetMetatableDoesNotTimeTravel) +LUAU_FASTFLAG(LuauUseCommittingTxnLog) using namespace Luau; @@ -1911,11 +1912,14 @@ local bar: @1= foo CHECK(!ac.entryMap.count("foo")); } -TEST_CASE_FIXTURE(ACFixture, "type_correct_function_no_parenthesis") +// Switch back to TEST_CASE_FIXTURE with regular ACFixture when removing the +// LuauUseCommittingTxnLog flag. +TEST_CASE("type_correct_function_no_parenthesis") { - ScopedFastFlag luauAutocompleteAvoidMutation("LuauAutocompleteAvoidMutation", true); + ScopedFastFlag sff_LuauUseCommittingTxnLog = ScopedFastFlag("LuauUseCommittingTxnLog", true); + ACFixture fix; - check(R"( + fix.check(R"( local function target(a: (number) -> number) return a(4) end local function bar1(a: number) return -a end local function bar2(a: string) return a .. 'x' end @@ -1923,7 +1927,7 @@ local function bar2(a: string) return a .. 'x' end return target(b@1 )"); - auto ac = autocomplete('1'); + auto ac = fix.autocomplete('1'); CHECK(ac.entryMap.count("bar1")); CHECK(ac.entryMap["bar1"].typeCorrect == TypeCorrectKind::Correct); @@ -1976,11 +1980,14 @@ local fp: @1= f CHECK(ac.entryMap.count("({ x: number, y: number }) -> number")); } -TEST_CASE_FIXTURE(ACFixture, "type_correct_keywords") +// Switch back to TEST_CASE_FIXTURE with regular ACFixture when removing the +// LuauUseCommittingTxnLog flag. +TEST_CASE("type_correct_keywords") { - ScopedFastFlag luauAutocompleteAvoidMutation("LuauAutocompleteAvoidMutation", true); + ScopedFastFlag sff_LuauUseCommittingTxnLog = ScopedFastFlag("LuauUseCommittingTxnLog", true); + ACFixture fix; - check(R"( + fix.check(R"( local function a(x: boolean) end local function b(x: number?) end local function c(x: (number) -> string) end @@ -1997,26 +2004,26 @@ local dc = d(f@4) local ec = e(f@5) )"); - auto ac = autocomplete('1'); + auto ac = fix.autocomplete('1'); CHECK(ac.entryMap.count("tru")); CHECK(ac.entryMap["tru"].typeCorrect == TypeCorrectKind::None); CHECK(ac.entryMap["true"].typeCorrect == TypeCorrectKind::Correct); CHECK(ac.entryMap["false"].typeCorrect == TypeCorrectKind::Correct); - ac = autocomplete('2'); + ac = fix.autocomplete('2'); CHECK(ac.entryMap.count("ni")); CHECK(ac.entryMap["ni"].typeCorrect == TypeCorrectKind::None); CHECK(ac.entryMap["nil"].typeCorrect == TypeCorrectKind::Correct); - ac = autocomplete('3'); + ac = fix.autocomplete('3'); CHECK(ac.entryMap.count("false")); CHECK(ac.entryMap["false"].typeCorrect == TypeCorrectKind::None); CHECK(ac.entryMap["function"].typeCorrect == TypeCorrectKind::Correct); - ac = autocomplete('4'); + ac = fix.autocomplete('4'); CHECK(ac.entryMap["function"].typeCorrect == TypeCorrectKind::Correct); - ac = autocomplete('5'); + ac = fix.autocomplete('5'); CHECK(ac.entryMap["function"].typeCorrect == TypeCorrectKind::Correct); } @@ -2507,21 +2514,23 @@ local t = { CHECK(ac.entryMap.count("second")); } -TEST_CASE_FIXTURE(UnfrozenFixture, "autocomplete_documentation_symbols") +TEST_CASE("autocomplete_documentation_symbols") { - loadDefinition(R"( + Fixture fix(FFlag::LuauUseCommittingTxnLog); + + fix.loadDefinition(R"( declare y: { x: number, } )"); - fileResolver.source["Module/A"] = R"( + fix.fileResolver.source["Module/A"] = R"( local a = y. )"; - frontend.check("Module/A"); + fix.frontend.check("Module/A"); - auto ac = autocomplete(frontend, "Module/A", Position{1, 21}, nullCallback); + auto ac = autocomplete(fix.frontend, "Module/A", Position{1, 21}, nullCallback); REQUIRE(ac.entryMap.count("x")); CHECK_EQ(ac.entryMap["x"].documentationSymbol, "@test/global/y.x"); diff --git a/tests/Conformance.test.cpp b/tests/Conformance.test.cpp index b055a38e..663b329e 100644 --- a/tests/Conformance.test.cpp +++ b/tests/Conformance.test.cpp @@ -13,8 +13,11 @@ #include "ScopedFlags.h" #include +#include #include +extern bool verbose; + static int lua_collectgarbage(lua_State* L) { static const char* const opts[] = {"stop", "restart", "collect", "count", "isrunning", "step", "setgoal", "setstepmul", "setstepsize", nullptr}; @@ -146,15 +149,21 @@ static StateRef runConformance(const char* name, void (*setup)(lua_State* L) = n luaL_openlibs(L); // Register a few global functions for conformance tests - static const luaL_Reg funcs[] = { + std::vector funcs = { {"collectgarbage", lua_collectgarbage}, {"loadstring", lua_loadstring}, - {"print", lua_silence}, // Disable print() by default; comment this out to enable debug prints in tests - {nullptr, nullptr}, }; + if (!verbose) + { + funcs.push_back({"print", lua_silence}); + } + + // "null" terminate the list of functions to register + funcs.push_back({nullptr, nullptr}); + lua_pushvalue(L, LUA_GLOBALSINDEX); - luaL_register(L, nullptr, funcs); + luaL_register(L, nullptr, funcs.data()); lua_pop(L, 1); // In some configurations we have a larger C stack consumption which trips some conformance tests @@ -312,8 +321,6 @@ TEST_CASE("GC") TEST_CASE("Bitwise") { - ScopedFastFlag sff("LuauBit32Count", true); - runConformance("bitwise.lua"); } @@ -491,6 +498,9 @@ TEST_CASE("DateTime") TEST_CASE("Debug") { + ScopedFastFlag sffr("LuauBytecodeV2Read", true); + ScopedFastFlag sffw("LuauBytecodeV2Write", true); + runConformance("debug.lua"); } @@ -738,8 +748,6 @@ TEST_CASE("ApiFunctionCalls") // lua_equal with a sleeping thread wake up { - ScopedFastFlag luauActivateBeforeExec("LuauActivateBeforeExec", true); - lua_State* L2 = lua_newthread(L); lua_getfield(L2, LUA_GLOBALSINDEX, "create_with_tm"); @@ -913,4 +921,11 @@ TEST_CASE("Coverage") nullptr, nullptr, &copts); } +TEST_CASE("StringConversion") +{ + ScopedFastFlag sff{"LuauSchubfach", true}; + + runConformance("strconv.lua"); +} + TEST_SUITE_END(); diff --git a/tests/Fixture.cpp b/tests/Fixture.cpp index 36d6f561..ca4281a0 100644 --- a/tests/Fixture.cpp +++ b/tests/Fixture.cpp @@ -103,11 +103,6 @@ Fixture::~Fixture() Luau::resetPrintLine(); } -UnfrozenFixture::UnfrozenFixture() - : Fixture(false) -{ -} - AstStatBlock* Fixture::parse(const std::string& source, const ParseOptions& parseOptions) { sourceModule.reset(new SourceModule); diff --git a/tests/Fixture.h b/tests/Fixture.h index de2b7381..e01632ea 100644 --- a/tests/Fixture.h +++ b/tests/Fixture.h @@ -152,15 +152,6 @@ struct Fixture LoadDefinitionFileResult loadDefinition(const std::string& source); }; -// Disables arena freezing for a given test case. -// Do not use this in new tests. If you are running into access violations, you -// are violating Luau's memory model - the fix is not to use UnfrozenFixture. -// Related: CLI-45692 -struct UnfrozenFixture : Fixture -{ - UnfrozenFixture(); -}; - ModuleName fromString(std::string_view name); template diff --git a/tests/Frontend.test.cpp b/tests/Frontend.test.cpp index 51fcd3d6..405f26e0 100644 --- a/tests/Frontend.test.cpp +++ b/tests/Frontend.test.cpp @@ -914,6 +914,8 @@ TEST_CASE_FIXTURE(FrontendFixture, "typecheck_twice_for_ast_types") TEST_CASE_FIXTURE(FrontendFixture, "imported_table_modification_2") { + ScopedFastFlag sffs("LuauSealExports", true); + frontend.options.retainFullTypeGraphs = false; fileResolver.source["Module/A"] = R"( @@ -927,7 +929,7 @@ return a; --!nonstrict local a = require(script.Parent.A) local b = {} -function a:b() end -- this should error, but doesn't +function a:b() end -- this should error, since A doesn't define a:b() return b )"; @@ -942,8 +944,7 @@ a:b() -- this should error, since A doesn't define a:b() LUAU_REQUIRE_NO_ERRORS(resultA); CheckResult resultB = frontend.check("Module/B"); - // TODO (CLI-45592): this should error, since we shouldn't be adding properties to objects from other modules - LUAU_REQUIRE_NO_ERRORS(resultB); + LUAU_REQUIRE_ERRORS(resultB); CheckResult resultC = frontend.check("Module/C"); LUAU_REQUIRE_ERRORS(resultC); diff --git a/tests/TypeInfer.annotations.test.cpp b/tests/TypeInfer.annotations.test.cpp index 71ff4e1b..275782b3 100644 --- a/tests/TypeInfer.annotations.test.cpp +++ b/tests/TypeInfer.annotations.test.cpp @@ -620,7 +620,7 @@ struct AssertionCatcher { tripped = 0; oldhook = Luau::assertHandler(); - Luau::assertHandler() = [](const char* expr, const char* file, int line) -> int { + Luau::assertHandler() = [](const char* expr, const char* file, int line, const char* function) -> int { ++tripped; return 0; }; diff --git a/tests/TypeInfer.builtins.test.cpp b/tests/TypeInfer.builtins.test.cpp index 506279b9..5e08654a 100644 --- a/tests/TypeInfer.builtins.test.cpp +++ b/tests/TypeInfer.builtins.test.cpp @@ -11,6 +11,8 @@ LUAU_FASTFLAG(LuauFixTonumberReturnType) using namespace Luau; +LUAU_FASTFLAG(LuauUseCommittingTxnLog) + TEST_SUITE_BEGIN("BuiltinTests"); TEST_CASE_FIXTURE(Fixture, "math_things_are_defined") @@ -444,19 +446,28 @@ TEST_CASE_FIXTURE(Fixture, "os_time_takes_optional_date_table") CHECK_EQ(*typeChecker.numberType, *requireType("n3")); } -TEST_CASE_FIXTURE(Fixture, "thread_is_a_type") +// Switch back to TEST_CASE_FIXTURE with regular Fixture when removing the +// LuauUseCommittingTxnLog flag. +TEST_CASE("thread_is_a_type") { - CheckResult result = check(R"( + Fixture fix(FFlag::LuauUseCommittingTxnLog); + + CheckResult result = fix.check(R"( local co = coroutine.create(function() end) )"); - LUAU_REQUIRE_NO_ERRORS(result); - CHECK_EQ(*typeChecker.threadType, *requireType("co")); + // Replace with LUAU_REQUIRE_NO_ERRORS(result) when using TEST_CASE_FIXTURE. + CHECK(result.errors.size() == 0); + CHECK_EQ(*fix.typeChecker.threadType, *fix.requireType("co")); } -TEST_CASE_FIXTURE(Fixture, "coroutine_resume_anything_goes") +// Switch back to TEST_CASE_FIXTURE with regular Fixture when removing the +// LuauUseCommittingTxnLog flag. +TEST_CASE("coroutine_resume_anything_goes") { - CheckResult result = check(R"( + Fixture fix(FFlag::LuauUseCommittingTxnLog); + + CheckResult result = fix.check(R"( local function nifty(x, y) print(x, y) local z = coroutine.yield(1, 2) @@ -469,12 +480,17 @@ TEST_CASE_FIXTURE(Fixture, "coroutine_resume_anything_goes") local answer = coroutine.resume(co, 3) )"); - LUAU_REQUIRE_NO_ERRORS(result); + // Replace with LUAU_REQUIRE_NO_ERRORS(result) when using TEST_CASE_FIXTURE. + CHECK(result.errors.size() == 0); } -TEST_CASE_FIXTURE(Fixture, "coroutine_wrap_anything_goes") +// Switch back to TEST_CASE_FIXTURE with regular Fixture when removing the +// LuauUseCommittingTxnLog flag. +TEST_CASE("coroutine_wrap_anything_goes") { - CheckResult result = check(R"( + Fixture fix(FFlag::LuauUseCommittingTxnLog); + + CheckResult result = fix.check(R"( --!nonstrict local function nifty(x, y) print(x, y) @@ -488,7 +504,8 @@ TEST_CASE_FIXTURE(Fixture, "coroutine_wrap_anything_goes") local answer = f(3) )"); - LUAU_REQUIRE_NO_ERRORS(result); + // Replace with LUAU_REQUIRE_NO_ERRORS(result) when using TEST_CASE_FIXTURE. + CHECK(result.errors.size() == 0); } TEST_CASE_FIXTURE(Fixture, "setmetatable_should_not_mutate_persisted_types") diff --git a/tests/TypeInfer.generics.test.cpp b/tests/TypeInfer.generics.test.cpp index b62044fa..114679e3 100644 --- a/tests/TypeInfer.generics.test.cpp +++ b/tests/TypeInfer.generics.test.cpp @@ -629,8 +629,6 @@ return exports TEST_CASE_FIXTURE(Fixture, "instantiated_function_argument_names") { - ScopedFastFlag luauFunctionArgumentNameSize{"LuauFunctionArgumentNameSize", true}; - CheckResult result = check(R"( local function f(a: T, ...: U...) end diff --git a/tests/TypeInfer.test.cpp b/tests/TypeInfer.test.cpp index 76324556..f70f3b1c 100644 --- a/tests/TypeInfer.test.cpp +++ b/tests/TypeInfer.test.cpp @@ -4803,8 +4803,6 @@ local bar = foo.nutrition + 100 TEST_CASE_FIXTURE(Fixture, "require_failed_module") { - ScopedFastFlag luauModuleRequireErrorPack{"LuauModuleRequireErrorPack", true}; - fileResolver.source["game/A"] = R"( return unfortunately() )"; diff --git a/tests/TypeInfer.tryUnify.test.cpp b/tests/TypeInfer.tryUnify.test.cpp index f55b46a4..1e790eba 100644 --- a/tests/TypeInfer.tryUnify.test.cpp +++ b/tests/TypeInfer.tryUnify.test.cpp @@ -12,6 +12,8 @@ LUAU_FASTFLAG(LuauQuantifyInPlace2); using namespace Luau; +LUAU_FASTFLAG(LuauUseCommittingTxnLog) + struct TryUnifyFixture : Fixture { TypeArena arena; @@ -28,7 +30,7 @@ TEST_CASE_FIXTURE(TryUnifyFixture, "primitives_unify") TypeVar numberOne{TypeVariant{PrimitiveTypeVar{PrimitiveTypeVar::Number}}}; TypeVar numberTwo = numberOne; - state.tryUnify(&numberOne, &numberTwo); + state.tryUnify(&numberTwo, &numberOne); CHECK(state.errors.empty()); } @@ -41,9 +43,12 @@ TEST_CASE_FIXTURE(TryUnifyFixture, "compatible_functions_are_unified") TypeVar functionTwo{TypeVariant{ FunctionTypeVar(arena.addTypePack({arena.freshType(globalScope->level)}), arena.addTypePack({arena.freshType(globalScope->level)}))}}; - state.tryUnify(&functionOne, &functionTwo); + state.tryUnify(&functionTwo, &functionOne); CHECK(state.errors.empty()); + if (FFlag::LuauUseCommittingTxnLog) + state.log.commit(); + CHECK_EQ(functionOne, functionTwo); } @@ -61,7 +66,7 @@ TEST_CASE_FIXTURE(TryUnifyFixture, "incompatible_functions_are_preserved") TypeVar functionTwoSaved = functionTwo; - state.tryUnify(&functionOne, &functionTwo); + state.tryUnify(&functionTwo, &functionOne); CHECK(!state.errors.empty()); CHECK_EQ(functionOne, functionOneSaved); @@ -80,10 +85,13 @@ TEST_CASE_FIXTURE(TryUnifyFixture, "tables_can_be_unified") CHECK_NE(*getMutable(&tableOne)->props["foo"].type, *getMutable(&tableTwo)->props["foo"].type); - state.tryUnify(&tableOne, &tableTwo); + state.tryUnify(&tableTwo, &tableOne); CHECK(state.errors.empty()); + if (FFlag::LuauUseCommittingTxnLog) + state.log.commit(); + CHECK_EQ(*getMutable(&tableOne)->props["foo"].type, *getMutable(&tableTwo)->props["foo"].type); } @@ -101,11 +109,12 @@ TEST_CASE_FIXTURE(TryUnifyFixture, "incompatible_tables_are_preserved") CHECK_NE(*getMutable(&tableOne)->props["foo"].type, *getMutable(&tableTwo)->props["foo"].type); - state.tryUnify(&tableOne, &tableTwo); + state.tryUnify(&tableTwo, &tableOne); CHECK_EQ(1, state.errors.size()); - state.log.rollback(); + if (!FFlag::LuauUseCommittingTxnLog) + state.DEPRECATED_log.rollback(); CHECK_NE(*getMutable(&tableOne)->props["foo"].type, *getMutable(&tableTwo)->props["foo"].type); } @@ -170,7 +179,7 @@ TEST_CASE_FIXTURE(TryUnifyFixture, "variadic_type_pack_unification") TypePackVar testPack{TypePack{{typeChecker.numberType, typeChecker.stringType}, std::nullopt}}; TypePackVar variadicPack{VariadicTypePack{typeChecker.numberType}}; - state.tryUnify(&variadicPack, &testPack); + state.tryUnify(&testPack, &variadicPack); CHECK(!state.errors.empty()); } @@ -180,7 +189,7 @@ TEST_CASE_FIXTURE(TryUnifyFixture, "variadic_tails_respect_progress") TypePackVar a{TypePack{{typeChecker.numberType, typeChecker.stringType, typeChecker.booleanType, typeChecker.booleanType}}}; TypePackVar b{TypePack{{typeChecker.numberType, typeChecker.stringType}, &variadicPack}}; - state.tryUnify(&a, &b); + state.tryUnify(&b, &a); CHECK(state.errors.empty()); } @@ -214,32 +223,41 @@ TEST_CASE_FIXTURE(TryUnifyFixture, "cli_41095_concat_log_in_sealed_table_unifica CHECK_EQ(toString(result.errors[1]), "Available overloads: ({a}, a) -> (); and ({a}, number, a) -> ()"); } -TEST_CASE_FIXTURE(TryUnifyFixture, "undo_new_prop_on_unsealed_table") +TEST_CASE("undo_new_prop_on_unsealed_table") { ScopedFastFlag flags[] = { {"LuauTableSubtypingVariance2", true}, + // This test makes no sense with a committing TxnLog. + {"LuauUseCommittingTxnLog", false}, }; // I am not sure how to make this happen in Luau code. - TypeId unsealedTable = arena.addType(TableTypeVar{TableState::Unsealed, TypeLevel{}}); - TypeId sealedTable = arena.addType(TableTypeVar{ - {{"prop", Property{getSingletonTypes().numberType}}}, - std::nullopt, - TypeLevel{}, - TableState::Sealed - }); + TryUnifyFixture fix; + + TypeId unsealedTable = fix.arena.addType(TableTypeVar{TableState::Unsealed, TypeLevel{}}); + TypeId sealedTable = + fix.arena.addType(TableTypeVar{{{"prop", Property{getSingletonTypes().numberType}}}, std::nullopt, TypeLevel{}, TableState::Sealed}); const TableTypeVar* ttv = get(unsealedTable); REQUIRE(ttv); - state.tryUnify(unsealedTable, sealedTable); + fix.state.tryUnify(sealedTable, unsealedTable); // To be honest, it's really quite spooky here that we're amending an unsealed table in this case. CHECK(!ttv->props.empty()); - state.log.rollback(); + fix.state.DEPRECATED_log.rollback(); CHECK(ttv->props.empty()); } +TEST_CASE_FIXTURE(TryUnifyFixture, "free_tail_is_grown_properly") +{ + TypePackId threeNumbers = arena.addTypePack(TypePack{{typeChecker.numberType, typeChecker.numberType, typeChecker.numberType}, std::nullopt}); + TypePackId numberAndFreeTail = arena.addTypePack(TypePack{{typeChecker.numberType}, arena.addTypePack(TypePackVar{FreeTypePack{TypeLevel{}}})}); + + ErrorVec unifyErrors = state.canUnify(numberAndFreeTail, threeNumbers); + CHECK(unifyErrors.size() == 0); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.typePacks.cpp b/tests/TypeInfer.typePacks.cpp index 3f4420cd..5d37b032 100644 --- a/tests/TypeInfer.typePacks.cpp +++ b/tests/TypeInfer.typePacks.cpp @@ -10,6 +10,8 @@ using namespace Luau; +LUAU_FASTFLAG(LuauUseCommittingTxnLog) + TEST_SUITE_BEGIN("TypePackTests"); TEST_CASE_FIXTURE(Fixture, "infer_multi_return") @@ -263,10 +265,13 @@ TEST_CASE_FIXTURE(Fixture, "variadic_pack_syntax") CHECK_EQ(toString(requireType("foo")), "(...number) -> ()"); } -// CLI-45791 -TEST_CASE_FIXTURE(UnfrozenFixture, "type_pack_hidden_free_tail_infinite_growth") +// Switch back to TEST_CASE_FIXTURE with regular Fixture when removing the +// LuauUseCommittingTxnLog flag. +TEST_CASE("type_pack_hidden_free_tail_infinite_growth") { - CheckResult result = check(R"( + Fixture fix(FFlag::LuauUseCommittingTxnLog); + + CheckResult result = fix.check(R"( --!nonstrict if _ then _[function(l0)end],l0 = _ @@ -278,7 +283,8 @@ elseif _ then end )"); - LUAU_REQUIRE_ERRORS(result); + // Switch back to LUAU_REQUIRE_ERRORS(result) when using TEST_CASE_FIXTURE. + CHECK(result.errors.size() > 0); } TEST_CASE_FIXTURE(Fixture, "variadic_argument_tail") diff --git a/tests/TypeInfer.unionTypes.test.cpp b/tests/TypeInfer.unionTypes.test.cpp index 2357869e..b54ba996 100644 --- a/tests/TypeInfer.unionTypes.test.cpp +++ b/tests/TypeInfer.unionTypes.test.cpp @@ -8,6 +8,7 @@ #include "doctest.h" LUAU_FASTFLAG(LuauEqConstraint) +LUAU_FASTFLAG(LuauUseCommittingTxnLog) using namespace Luau; @@ -282,16 +283,19 @@ local c = b:foo(1, 2) CHECK_EQ("Value of type 'A?' could be nil", toString(result.errors[0])); } -TEST_CASE_FIXTURE(UnfrozenFixture, "optional_union_follow") +TEST_CASE("optional_union_follow") { - CheckResult result = check(R"( + Fixture fix(FFlag::LuauUseCommittingTxnLog); + + CheckResult result = fix.check(R"( local y: number? = 2 local x = y local function f(a: number, b: typeof(x), c: typeof(x)) return -a end return f() )"); - LUAU_REQUIRE_ERROR_COUNT(1, result); + REQUIRE_EQ(result.errors.size(), 1); + // LUAU_REQUIRE_ERROR_COUNT(1, result); auto acm = get(result.errors[0]); REQUIRE(acm); diff --git a/tests/TypeVar.test.cpp b/tests/TypeVar.test.cpp index 13db923e..2e0d149e 100644 --- a/tests/TypeVar.test.cpp +++ b/tests/TypeVar.test.cpp @@ -185,6 +185,8 @@ TEST_CASE_FIXTURE(Fixture, "UnionTypeVarIterator_with_empty_union") TEST_CASE_FIXTURE(Fixture, "substitution_skip_failure") { + ScopedFastFlag sff{"LuauSealExports", true}; + TypeVar ftv11{FreeTypeVar{TypeLevel{}}}; TypePackVar tp24{TypePack{{&ftv11}}}; @@ -261,7 +263,7 @@ TEST_CASE_FIXTURE(Fixture, "substitution_skip_failure") TypeId result = typeChecker.anyify(typeChecker.globalScope, root, Location{}); - CHECK_EQ("{ f: t1 } where t1 = () -> { f: () -> { f: ({ f: t1 }) -> (), signal: { f: (any) -> () } } }", toString(result)); + CHECK_EQ("{| f: t1 |} where t1 = () -> {| f: () -> {| f: ({| f: t1 |}) -> (), signal: {| f: (any) -> () |} |} |}", toString(result)); } TEST_CASE("tagging_tables") diff --git a/tests/conformance/debug.lua b/tests/conformance/debug.lua index 9cf3c742..8c96ab33 100644 --- a/tests/conformance/debug.lua +++ b/tests/conformance/debug.lua @@ -98,4 +98,13 @@ assert(quuz(function(...) end) == "0 true") assert(quuz(function(a, b) end) == "2 false") assert(quuz(function(a, b, ...) end) == "2 true") +-- info linedefined & line +function testlinedefined() + local line = debug.info(1, "l") + local linedefined = debug.info(testlinedefined, "l") + assert(linedefined + 1 == line) +end + +testlinedefined() + return 'OK' diff --git a/tests/conformance/strconv.lua b/tests/conformance/strconv.lua new file mode 100644 index 00000000..85ad0295 --- /dev/null +++ b/tests/conformance/strconv.lua @@ -0,0 +1,51 @@ +-- This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +-- This file is based on Lua 5.x tests -- https://github.com/lua/lua/tree/master/testes +print("testing string-number conversion") + +-- zero +assert(tostring(0) == "0") +assert(tostring(0/-1) == "-0") + +-- specials +assert(tostring(1/0) == "inf") +assert(tostring(-1/0) == "-inf") +assert(tostring(0/0) == "nan") + +-- integers +assert(tostring(1) == "1") +assert(tostring(42) == "42") +assert(tostring(-4294967296) == "-4294967296") +assert(tostring(9007199254740991) == "9007199254740991") + +-- decimals +assert(tostring(0.5) == "0.5") +assert(tostring(0.1) == "0.1") +assert(tostring(-0.17) == "-0.17") +assert(tostring(math.pi) == "3.141592653589793") + +-- fuzzing corpus +assert(tostring(5.4536123983019448e-311) == "5.453612398302e-311") +assert(tostring(5.4834368411298348e-311) == "5.48343684113e-311") +assert(tostring(4.4154895841930002e-305) == "4.415489584193e-305") +assert(tostring(1125968630513728) == "1125968630513728") +assert(tostring(3.3951932655938423e-313) == "3.3951932656e-313") +assert(tostring(1.625) == "1.625") +assert(tostring(4.9406564584124654e-324) == "5.e-324") +assert(tostring(2.0049288280105384) == "2.0049288280105384") +assert(tostring(3.0517578125e-05) == "0.000030517578125") +assert(tostring(1.383544921875) == "1.383544921875") +assert(tostring(3.0053350932691001) == "3.0053350932691") +assert(tostring(0.0001373291015625) == "0.0001373291015625") +assert(tostring(-1.9490628022799998e+289) == "-1.94906280228e+289") +assert(tostring(-0.00610404721867928) == "-0.00610404721867928") +assert(tostring(0.00014495849609375) == "0.00014495849609375") +assert(tostring(0.453125) == "0.453125") +assert(tostring(-4.2375343999999997e+73) == "-4.2375344e+73") +assert(tostring(1.3202313930270133e-192) == "1.3202313930270133e-192") +assert(tostring(3.6984408976312836e+19) == "36984408976312840000") +assert(tostring(2.0563000527063302) == "2.05630005270633") +assert(tostring(4.8970527433648997e-260) == "4.8970527433649e-260") +assert(tostring(1.62890625) == "1.62890625") +assert(tostring(1.1295093211933533e+65) == "1.1295093211933533e+65") + +return "OK" diff --git a/tests/main.cpp b/tests/main.cpp index ed17070c..cd24e100 100644 --- a/tests/main.cpp +++ b/tests/main.cpp @@ -2,6 +2,8 @@ #include "Luau/Common.h" #define DOCTEST_CONFIG_IMPLEMENT +// Our calls to parseOption/parseFlag don't provide a prefix so set the prefix to the empty string. +#define DOCTEST_CONFIG_OPTIONS_PREFIX "" #include "doctest.h" #ifdef _WIN32 @@ -18,6 +20,10 @@ #include +// Indicates if verbose output is enabled. +// Currently, this enables output from lua's 'print', but other verbose output could be enabled eventually. +bool verbose = false; + static bool skipFastFlag(const char* flagName) { if (strncmp(flagName, "Test", 4) == 0) @@ -46,7 +52,7 @@ static bool debuggerPresent() #endif } -static int assertionHandler(const char* expr, const char* file, int line) +static int assertionHandler(const char* expr, const char* file, int line, const char* function) { if (debuggerPresent()) LUAU_DEBUGBREAK(); @@ -235,6 +241,11 @@ int main(int argc, char** argv) return 0; } + if (doctest::parseFlag(argc, argv, "--verbose")) + { + verbose = true; + } + if (std::vector flags; doctest::parseCommaSepArgs(argc, argv, "--fflags=", flags)) setFastFlags(flags); @@ -261,7 +272,15 @@ int main(int argc, char** argv) } } - return context.run(); + int result = context.run(); + if (doctest::parseFlag(argc, argv, "--help") || doctest::parseFlag(argc, argv, "-h")) + { + printf("Additional command line options:\n"); + printf(" --verbose Enables verbose output (e.g. lua 'print' statements)\n"); + printf(" --fflags= Sets specified fast flags\n"); + printf(" --list-fflags List all fast flags\n"); + } + return result; } diff --git a/tools/numprint.py b/tools/numprint.py new file mode 100644 index 00000000..47ad36d9 --- /dev/null +++ b/tools/numprint.py @@ -0,0 +1,82 @@ +#!/usr/bin/python +# This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details + +# This code can be used to generate power tables for Schubfach algorithm (see lnumprint.cpp) + +import math +import sys + +(_, pow10min, pow10max, compact) = sys.argv +pow10min = int(pow10min) +pow10max = int(pow10max) +compact = compact == "True" + +# extract high 128 bits of the value +def high128(n, roundup): + L = math.ceil(math.log2(n)) + + r = 0 + for i in range(L - 128, L): + if i >= 0 and (n & (1 << i)) != 0: + r |= (1 << (i - L + 128)) + + return r + (1 if roundup else 0) + +def pow10approx(n): + if n == 0: + return 1 << 127 + elif n > 0: + return high128(10**n, 5**n >= 2**128) + else: + # 10^-n is a binary fraction that can't be represented in floating point + # we need to extract top 128 bits of the fraction starting from the first 1 + # to get there, we need to divide 2^k by 10^n for a sufficiently large k and repeat the extraction process + p = 10**-n + k = 2**128 * 16**-n # this guarantees that the fraction has more than 128 extra bits + return high128(k // p, True) + +def pow5_64(n): + assert(n >= 0) + if n == 0: + return 1 << 63 + else: + return high128(5**n, False) >> 64 + +if not compact: + print("// kPow10Table", pow10min, "..", pow10max) + print("{") + for p in range(pow10min, pow10max + 1): + h = hex(pow10approx(p))[2:] + assert(len(h) == 32) + print(" {0x%s, 0x%s}," % (h[0:16].upper(), h[16:32].upper())) + print("}") +else: + print("// kPow5Table") + print("{") + for i in range(16): + print(" " + hex(pow5_64(i)) + ",") + print("}") + print("// kPow10Table", pow10min, "..", pow10max) + print("{") + for p in range(pow10min, pow10max + 1, 16): + base = pow10approx(p) + errw = 0 + for i in range(16): + real = pow10approx(p + i) + appr = (base * pow5_64(i)) >> 64 + scale = 1 if appr < (1 << 127) else 0 # 1-bit scale + + offset = (appr << scale) - real + assert(offset >= -4 and offset <= 3) # 3-bit offset + assert((appr << scale) >> 64 == real >> 64) # offset only affects low half + assert((appr << scale) - offset == real) # validate full reconstruction + + err = (scale << 3) | (offset + 4) + errw |= err << (i * 4) + + hbase = hex(base)[2:] + assert(len(hbase) == 32) + assert(errw < 1 << 64) + + print(" {0x%s, 0x%s, 0x%16x}," % (hbase[0:16], hbase[16:32], errw)) + print("}") From d323237b6cf8f5e6147a9e5d78eab96e896f6eeb Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Thu, 6 Jan 2022 15:26:14 -0800 Subject: [PATCH 116/399] Sync to upstream/release/508 (#301) This version isn't for release because we've skipped some internal numbers due to year-end schedule changes, but it's better to merge separately. --- Analysis/include/Luau/LValue.h | 63 +++++++ Analysis/include/Luau/Predicate.h | 34 +--- Analysis/include/Luau/TypeInfer.h | 1 + Analysis/src/{Predicate.cpp => LValue.cpp} | 88 ++++++++- Analysis/src/TypeInfer.cpp | 83 ++++++++- Analysis/src/TypeVar.cpp | 11 +- Analysis/src/Unifier.cpp | 123 ++++--------- Ast/src/Parser.cpp | 4 - Sources.cmake | 5 +- VM/src/lbaselib.cpp | 8 +- tests/Autocomplete.test.cpp | 6 +- tests/LValue.test.cpp | 198 +++++++++++++++++++++ tests/Predicate.test.cpp | 117 ------------ tests/Symbol.test.cpp | 33 +++- tests/Transpiler.test.cpp | 2 - tests/TypeInfer.builtins.test.cpp | 12 +- tests/TypeInfer.classes.test.cpp | 2 - tests/TypeInfer.intersectionTypes.test.cpp | 4 - tests/TypeInfer.refinements.test.cpp | 37 +++- tests/TypeInfer.singletons.test.cpp | 1 - tests/TypeInfer.tables.test.cpp | 4 - tests/TypeInfer.test.cpp | 13 ++ tests/TypeInfer.unionTypes.test.cpp | 4 - tests/conformance/math.lua | 1 + 24 files changed, 570 insertions(+), 284 deletions(-) create mode 100644 Analysis/include/Luau/LValue.h rename Analysis/src/{Predicate.cpp => LValue.cpp} (50%) create mode 100644 tests/LValue.test.cpp delete mode 100644 tests/Predicate.test.cpp diff --git a/Analysis/include/Luau/LValue.h b/Analysis/include/Luau/LValue.h new file mode 100644 index 00000000..8fd96f05 --- /dev/null +++ b/Analysis/include/Luau/LValue.h @@ -0,0 +1,63 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +#pragma once + +#include "Luau/Variant.h" +#include "Luau/Symbol.h" + +#include // TODO: Kill with LuauLValueAsKey. +#include +#include + +namespace Luau +{ + +struct TypeVar; +using TypeId = const TypeVar*; + +struct Field; +using LValue = Variant; + +struct Field +{ + std::shared_ptr parent; + std::string key; + + bool operator==(const Field& rhs) const; + bool operator!=(const Field& rhs) const; +}; + +struct LValueHasher +{ + size_t operator()(const LValue& lvalue) const; +}; + +const LValue* baseof(const LValue& lvalue); + +std::optional tryGetLValue(const class AstExpr& expr); + +// Utility function: breaks down an LValue to get at the Symbol, and reverses the vector of keys. +std::pair> getFullName(const LValue& lvalue); + +// Kill with LuauLValueAsKey. +std::string toString(const LValue& lvalue); + +template +const T* get(const LValue& lvalue) +{ + return get_if(&lvalue); +} + +using NEW_RefinementMap = std::unordered_map; +using DEPRECATED_RefinementMap = std::map; + +// Transient. Kill with LuauLValueAsKey. +struct RefinementMap +{ + NEW_RefinementMap NEW_refinements; + DEPRECATED_RefinementMap DEPRECATED_refinements; +}; + +void merge(RefinementMap& l, const RefinementMap& r, std::function f); +void addRefinement(RefinementMap& refis, const LValue& lvalue, TypeId ty); + +} // namespace Luau diff --git a/Analysis/include/Luau/Predicate.h b/Analysis/include/Luau/Predicate.h index a5e8b6ae..df93b4f4 100644 --- a/Analysis/include/Luau/Predicate.h +++ b/Analysis/include/Luau/Predicate.h @@ -1,12 +1,10 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #pragma once -#include "Luau/Variant.h" #include "Luau/Location.h" -#include "Luau/Symbol.h" +#include "Luau/LValue.h" +#include "Luau/Variant.h" -#include -#include #include namespace Luau @@ -15,34 +13,6 @@ namespace Luau struct TypeVar; using TypeId = const TypeVar*; -struct Field; -using LValue = Variant; - -struct Field -{ - std::shared_ptr parent; // TODO: Eventually use unique_ptr to enforce non-copyable trait. - std::string key; -}; - -std::optional tryGetLValue(const class AstExpr& expr); - -// Utility function: breaks down an LValue to get at the Symbol, and reverses the vector of keys. -std::pair> getFullName(const LValue& lvalue); - -std::string toString(const LValue& lvalue); - -template -const T* get(const LValue& lvalue) -{ - return get_if(&lvalue); -} - -// Key is a stringified encoding of an LValue. -using RefinementMap = std::map; - -void merge(RefinementMap& l, const RefinementMap& r, std::function f); -void addRefinement(RefinementMap& refis, const LValue& lvalue, TypeId ty); - struct TruthyPredicate; struct IsAPredicate; struct TypeGuardPredicate; diff --git a/Analysis/include/Luau/TypeInfer.h b/Analysis/include/Luau/TypeInfer.h index 451976e4..862f50d7 100644 --- a/Analysis/include/Luau/TypeInfer.h +++ b/Analysis/include/Luau/TypeInfer.h @@ -350,6 +350,7 @@ public: private: std::optional resolveLValue(const ScopePtr& scope, const LValue& lvalue); + std::optional DEPRECATED_resolveLValue(const ScopePtr& scope, const LValue& lvalue); std::optional resolveLValue(const RefinementMap& refis, const ScopePtr& scope, const LValue& lvalue); void resolve(const PredicateVec& predicates, ErrorVec& errVec, RefinementMap& refis, const ScopePtr& scope, bool sense, bool fromOr = false); diff --git a/Analysis/src/Predicate.cpp b/Analysis/src/LValue.cpp similarity index 50% rename from Analysis/src/Predicate.cpp rename to Analysis/src/LValue.cpp index 7bd8001e..da6804c6 100644 --- a/Analysis/src/Predicate.cpp +++ b/Analysis/src/LValue.cpp @@ -1,11 +1,59 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details -#include "Luau/Predicate.h" +#include "Luau/LValue.h" #include "Luau/Ast.h" +#include + +LUAU_FASTFLAG(LuauLValueAsKey) + namespace Luau { +bool Field::operator==(const Field& rhs) const +{ + LUAU_ASSERT(parent && rhs.parent); + return key == rhs.key && (parent == rhs.parent || *parent == *rhs.parent); +} + +bool Field::operator!=(const Field& rhs) const +{ + return !(*this == rhs); +} + +size_t LValueHasher::operator()(const LValue& lvalue) const +{ + // Most likely doesn't produce high quality hashes, but we're probably ok enough with it. + // When an evidence is shown that operator==(LValue) is used more often than it should, we can have a look at improving the hash quality. + size_t acc = 0; + size_t offset = 0; + + const LValue* current = &lvalue; + while (current) + { + if (auto field = get(*current)) + acc ^= (std::hash{}(field->key) << 1) >> ++offset; + else if (auto symbol = get(*current)) + acc ^= std::hash{}(*symbol) << 1; + else + LUAU_ASSERT(!"Hash not accumulated for this new LValue alternative."); + + current = baseof(*current); + } + + return acc; +} + +const LValue* baseof(const LValue& lvalue) +{ + if (auto field = get(lvalue)) + return field->parent.get(); + + auto symbol = get(lvalue); + LUAU_ASSERT(symbol); + return nullptr; // Base of root is null. +} + std::optional tryGetLValue(const AstExpr& node) { const AstExpr* expr = &node; @@ -38,15 +86,15 @@ std::pair> getFullName(const LValue& lvalue) while (auto field = get(*current)) { keys.push_back(field->key); - current = field->parent.get(); - if (!current) - LUAU_ASSERT(!"LValue root is a Field?"); + current = baseof(*current); } const Symbol* symbol = get(*current); + LUAU_ASSERT(symbol); return {*symbol, std::vector(keys.rbegin(), keys.rend())}; } +// Kill with LuauLValueAsKey. std::string toString(const LValue& lvalue) { auto [symbol, keys] = getFullName(lvalue); @@ -56,7 +104,18 @@ std::string toString(const LValue& lvalue) return s; } -void merge(RefinementMap& l, const RefinementMap& r, std::function f) +static void merge(NEW_RefinementMap& l, const NEW_RefinementMap& r, std::function f) +{ + for (const auto& [k, a] : r) + { + if (auto it = l.find(k); it != l.end()) + l[k] = f(it->second, a); + else + l[k] = a; + } +} + +static void merge(DEPRECATED_RefinementMap& l, const DEPRECATED_RefinementMap& r, std::function f) { auto itL = l.begin(); auto itR = r.begin(); @@ -69,21 +128,32 @@ void merge(RefinementMap& l, const RefinementMap& r, std::functionfirst > k) + else if (itL->first < k) + ++itL; + else { l[k] = a; ++itR; } - else - ++itL; } l.insert(itR, r.end()); } +void merge(RefinementMap& l, const RefinementMap& r, std::function f) +{ + if (FFlag::LuauLValueAsKey) + return merge(l.NEW_refinements, r.NEW_refinements, f); + else + return merge(l.DEPRECATED_refinements, r.DEPRECATED_refinements, f); +} + void addRefinement(RefinementMap& refis, const LValue& lvalue, TypeId ty) { - refis[toString(lvalue)] = ty; + if (FFlag::LuauLValueAsKey) + refis.NEW_refinements[lvalue] = ty; + else + refis.DEPRECATED_refinements[toString(lvalue)] = ty; } } // namespace Luau diff --git a/Analysis/src/TypeInfer.cpp b/Analysis/src/TypeInfer.cpp index abbc2901..e29b6ec6 100644 --- a/Analysis/src/TypeInfer.cpp +++ b/Analysis/src/TypeInfer.cpp @@ -36,6 +36,7 @@ LUAU_FASTFLAGVARIABLE(LuauErrorRecoveryType, false) LUAU_FASTFLAGVARIABLE(LuauPropertiesGetExpectedType, false) LUAU_FASTFLAGVARIABLE(LuauTailArgumentTypeInfo, false) LUAU_FASTFLAGVARIABLE(LuauModuleRequireErrorPack, false) +LUAU_FASTFLAGVARIABLE(LuauLValueAsKey, false) LUAU_FASTFLAGVARIABLE(LuauRefiLookupFromIndexExpr, false) LUAU_FASTFLAGVARIABLE(LuauProperTypeLevels, false) LUAU_FASTFLAGVARIABLE(LuauAscribeCorrectLevelToInferredProperitesOfFreeTables, false) @@ -1626,6 +1627,10 @@ std::optional TypeChecker::getIndexTypeFromType( { RecursionLimiter _rl(&recursionCount, FInt::LuauTypeInferRecursionLimit); + // Not needed when we normalize types. + if (FFlag::LuauLValueAsKey && get(follow(t))) + return t; + if (std::optional ty = getIndexTypeFromType(scope, t, name, location, false)) goodOptions.push_back(*ty); else @@ -4967,13 +4972,83 @@ std::pair, std::vector> TypeChecker::createGener std::optional TypeChecker::resolveLValue(const ScopePtr& scope, const LValue& lvalue) { - std::string path = toString(lvalue); + if (!FFlag::LuauLValueAsKey) + return DEPRECATED_resolveLValue(scope, lvalue); + + // We want to be walking the Scope parents. + // We'll also want to walk up the LValue path. As we do this, we need to save each LValue because we must walk back. + // For example: + // There exists an entry t.x. + // We are asked to look for t.x.y. + // We need to search in the provided Scope. Find t.x.y first. + // We fail to find t.x.y. Try t.x. We found it. Now we must return the type of the property y from the mapped-to type of t.x. + // If we completely fail to find the Symbol t but the Scope has that entry, then we should walk that all the way through and terminate. + const auto& [symbol, keys] = getFullName(lvalue); + + ScopePtr currentScope = scope; + while (currentScope) + { + std::optional found; + + std::vector childKeys; + const LValue* currentLValue = &lvalue; + while (currentLValue) + { + if (auto it = currentScope->refinements.NEW_refinements.find(*currentLValue); it != currentScope->refinements.NEW_refinements.end()) + { + found = it->second; + break; + } + + childKeys.push_back(*currentLValue); + currentLValue = baseof(*currentLValue); + } + + if (!found) + { + // Should not be using scope->lookup. This is already recursive. + if (auto it = currentScope->bindings.find(symbol); it != currentScope->bindings.end()) + found = it->second.typeId; + else + { + // Nothing exists in this Scope. Just skip and try the parent one. + currentScope = currentScope->parent; + continue; + } + } + + for (auto it = childKeys.rbegin(); it != childKeys.rend(); ++it) + { + const LValue& key = *it; + + // Symbol can happen. Skip. + if (get(key)) + continue; + else if (auto field = get(key)) + { + found = getIndexTypeFromType(scope, *found, field->key, Location(), false); + if (!found) + return std::nullopt; // Turns out this type doesn't have the property at all. We're done. + } + else + LUAU_ASSERT(!"New LValue alternative not handled here."); + } + + return found; + } + + // No entry for it at all. Can happen when LValue root is a global. + return std::nullopt; +} + +std::optional TypeChecker::DEPRECATED_resolveLValue(const ScopePtr& scope, const LValue& lvalue) +{ auto [symbol, keys] = getFullName(lvalue); ScopePtr currentScope = scope; while (currentScope) { - if (auto it = currentScope->refinements.find(path); it != currentScope->refinements.end()) + if (auto it = currentScope->refinements.DEPRECATED_refinements.find(toString(lvalue)); it != currentScope->refinements.DEPRECATED_refinements.end()) return it->second; // Should not be using scope->lookup. This is already recursive. @@ -5000,7 +5075,9 @@ std::optional TypeChecker::resolveLValue(const ScopePtr& scope, const LV std::optional TypeChecker::resolveLValue(const RefinementMap& refis, const ScopePtr& scope, const LValue& lvalue) { - if (auto it = refis.find(toString(lvalue)); it != refis.end()) + if (auto it = refis.DEPRECATED_refinements.find(toString(lvalue)); it != refis.DEPRECATED_refinements.end()) + return it->second; + else if (auto it = refis.NEW_refinements.find(lvalue); it != refis.NEW_refinements.end()) return it->second; else return resolveLValue(scope, lvalue); diff --git a/Analysis/src/TypeVar.cpp b/Analysis/src/TypeVar.cpp index 571b13ca..fb75aa02 100644 --- a/Analysis/src/TypeVar.cpp +++ b/Analysis/src/TypeVar.cpp @@ -996,18 +996,19 @@ std::optional> magicFunctionFormat( std::vector expected = parseFormatString(typechecker, fmt->value.data, fmt->value.size); const auto& [params, tail] = flatten(paramPack); - const size_t dataOffset = 1; + size_t paramOffset = 1; + size_t dataOffset = expr.self ? 0 : 1; // unify the prefix one argument at a time - for (size_t i = 0; i < expected.size() && i + dataOffset < params.size(); ++i) + for (size_t i = 0; i < expected.size() && i + paramOffset < params.size(); ++i) { - Location location = expr.args.data[std::min(i, expr.args.size - 1)]->location; + Location location = expr.args.data[std::min(i + dataOffset, expr.args.size - 1)]->location; - typechecker.unify(expected[i], params[i + dataOffset], location); + typechecker.unify(expected[i], params[i + paramOffset], location); } // if we know the argument count or if we have too many arguments for sure, we can issue an error - const size_t actualParamSize = params.size() - dataOffset; + size_t actualParamSize = params.size() - paramOffset; if (expected.size() != actualParamSize && (!tail || expected.size() < actualParamSize)) typechecker.reportError(TypeError{expr.location, CountMismatch{expected.size(), actualParamSize}}); diff --git a/Analysis/src/Unifier.cpp b/Analysis/src/Unifier.cpp index c5aab856..43ea37e7 100644 --- a/Analysis/src/Unifier.cpp +++ b/Analysis/src/Unifier.cpp @@ -18,9 +18,7 @@ LUAU_FASTFLAGVARIABLE(LuauTableSubtypingVariance2, false); LUAU_FASTFLAGVARIABLE(LuauUnionHeuristic, false) LUAU_FASTFLAGVARIABLE(LuauTableUnificationEarlyTest, false) LUAU_FASTFLAGVARIABLE(LuauOccursCheckOkWithRecursiveFunctions, false) -LUAU_FASTFLAGVARIABLE(LuauExtendedTypeMismatchError, false) LUAU_FASTFLAG(LuauSingletonTypes) -LUAU_FASTFLAGVARIABLE(LuauExtendedClassMismatchError, false) LUAU_FASTFLAG(LuauErrorRecoveryType); LUAU_FASTFLAG(LuauProperTypeLevels); LUAU_FASTFLAGVARIABLE(LuauExtendedUnionMismatchError, false) @@ -416,7 +414,7 @@ void Unifier::tryUnify_(TypeId superTy, TypeId subTy, bool isFunctionCall, bool else if (!innerState.errors.empty()) { // 'nil' option is skipped from extended report because we present the type in a special way - 'T?' - if (FFlag::LuauExtendedTypeMismatchError && !firstFailedOption && !isNil(type)) + if (!firstFailedOption && !isNil(type)) firstFailedOption = {innerState.errors.front()}; failed = true; @@ -434,7 +432,7 @@ void Unifier::tryUnify_(TypeId superTy, TypeId subTy, bool isFunctionCall, bool errors.push_back(*unificationTooComplex); else if (failed) { - if (FFlag::LuauExtendedTypeMismatchError && firstFailedOption) + if (firstFailedOption) errors.push_back(TypeError{location, TypeMismatch{superTy, subTy, "Not all union options are compatible.", *firstFailedOption}}); else errors.push_back(TypeError{location, TypeMismatch{superTy, subTy}}); @@ -536,49 +534,36 @@ void Unifier::tryUnify_(TypeId superTy, TypeId subTy, bool isFunctionCall, bool if (FFlag::LuauExtendedUnionMismatchError && (failedOptionCount == 1 || foundHeuristic) && failedOption) errors.push_back( TypeError{location, TypeMismatch{superTy, subTy, "None of the union options are compatible. For example:", *failedOption}}); - else if (FFlag::LuauExtendedTypeMismatchError) - errors.push_back(TypeError{location, TypeMismatch{superTy, subTy, "none of the union options are compatible"}}); else - errors.push_back(TypeError{location, TypeMismatch{superTy, subTy}}); + errors.push_back(TypeError{location, TypeMismatch{superTy, subTy, "none of the union options are compatible"}}); } } else if (const IntersectionTypeVar* uv = get(superTy)) { - if (FFlag::LuauExtendedTypeMismatchError) + std::optional unificationTooComplex; + std::optional firstFailedOption; + + // T <: A & B if A <: T and B <: T + for (TypeId type : uv->parts) { - std::optional unificationTooComplex; - std::optional firstFailedOption; + Unifier innerState = makeChildUnifier(); + innerState.tryUnify_(type, subTy, /*isFunctionCall*/ false, /*isIntersection*/ true); - // T <: A & B if A <: T and B <: T - for (TypeId type : uv->parts) + if (auto e = hasUnificationTooComplex(innerState.errors)) + unificationTooComplex = e; + else if (!innerState.errors.empty()) { - Unifier innerState = makeChildUnifier(); - innerState.tryUnify_(type, subTy, /*isFunctionCall*/ false, /*isIntersection*/ true); - - if (auto e = hasUnificationTooComplex(innerState.errors)) - unificationTooComplex = e; - else if (!innerState.errors.empty()) - { - if (!firstFailedOption) - firstFailedOption = {innerState.errors.front()}; - } - - log.concat(std::move(innerState.log)); + if (!firstFailedOption) + firstFailedOption = {innerState.errors.front()}; } - if (unificationTooComplex) - errors.push_back(*unificationTooComplex); - else if (firstFailedOption) - errors.push_back(TypeError{location, TypeMismatch{superTy, subTy, "Not all intersection parts are compatible.", *firstFailedOption}}); - } - else - { - // T <: A & B if A <: T and B <: T - for (TypeId type : uv->parts) - { - tryUnify_(type, subTy, /*isFunctionCall*/ false, /*isIntersection*/ true); - } + log.concat(std::move(innerState.log)); } + + if (unificationTooComplex) + errors.push_back(*unificationTooComplex); + else if (firstFailedOption) + errors.push_back(TypeError{location, TypeMismatch{superTy, subTy, "Not all intersection parts are compatible.", *firstFailedOption}}); } else if (const IntersectionTypeVar* uv = get(subTy)) { @@ -626,10 +611,7 @@ void Unifier::tryUnify_(TypeId superTy, TypeId subTy, bool isFunctionCall, bool errors.push_back(*unificationTooComplex); else if (!found) { - if (FFlag::LuauExtendedTypeMismatchError) - errors.push_back(TypeError{location, TypeMismatch{superTy, subTy, "none of the intersection parts are compatible"}}); - else - errors.push_back(TypeError{location, TypeMismatch{superTy, subTy}}); + errors.push_back(TypeError{location, TypeMismatch{superTy, subTy, "none of the intersection parts are compatible"}}); } } else if (get(superTy) && get(subTy)) @@ -1241,10 +1223,7 @@ void Unifier::tryUnifyTables(TypeId left, TypeId right, bool isIntersection) Unifier innerState = makeChildUnifier(); innerState.tryUnify_(prop.type, r->second.type); - if (FFlag::LuauExtendedTypeMismatchError) - checkChildUnifierTypeMismatch(innerState.errors, name, left, right); - else - checkChildUnifierTypeMismatch(innerState.errors, left, right); + checkChildUnifierTypeMismatch(innerState.errors, name, left, right); if (innerState.errors.empty()) log.concat(std::move(innerState.log)); @@ -1261,10 +1240,7 @@ void Unifier::tryUnifyTables(TypeId left, TypeId right, bool isIntersection) Unifier innerState = makeChildUnifier(); innerState.tryUnify_(prop.type, rt->indexer->indexResultType); - if (FFlag::LuauExtendedTypeMismatchError) - checkChildUnifierTypeMismatch(innerState.errors, name, left, right); - else - checkChildUnifierTypeMismatch(innerState.errors, left, right); + checkChildUnifierTypeMismatch(innerState.errors, name, left, right); if (innerState.errors.empty()) log.concat(std::move(innerState.log)); @@ -1302,10 +1278,7 @@ void Unifier::tryUnifyTables(TypeId left, TypeId right, bool isIntersection) Unifier innerState = makeChildUnifier(); innerState.tryUnify_(prop.type, lt->indexer->indexResultType); - if (FFlag::LuauExtendedTypeMismatchError) - checkChildUnifierTypeMismatch(innerState.errors, name, left, right); - else - checkChildUnifierTypeMismatch(innerState.errors, left, right); + checkChildUnifierTypeMismatch(innerState.errors, name, left, right); if (innerState.errors.empty()) log.concat(std::move(innerState.log)); @@ -1723,18 +1696,11 @@ void Unifier::tryUnifyWithMetatable(TypeId metatable, TypeId other, bool reverse innerState.tryUnify_(lhs->table, rhs->table); innerState.tryUnify_(lhs->metatable, rhs->metatable); - if (FFlag::LuauExtendedTypeMismatchError) - { - if (auto e = hasUnificationTooComplex(innerState.errors)) - errors.push_back(*e); - else if (!innerState.errors.empty()) - errors.push_back( - TypeError{location, TypeMismatch{reversed ? other : metatable, reversed ? metatable : other, "", innerState.errors.front()}}); - } - else - { - checkChildUnifierTypeMismatch(innerState.errors, reversed ? other : metatable, reversed ? metatable : other); - } + if (auto e = hasUnificationTooComplex(innerState.errors)) + errors.push_back(*e); + else if (!innerState.errors.empty()) + errors.push_back( + TypeError{location, TypeMismatch{reversed ? other : metatable, reversed ? metatable : other, "", innerState.errors.front()}}); log.concat(std::move(innerState.log)); } @@ -1821,31 +1787,22 @@ void Unifier::tryUnifyWithClass(TypeId superTy, TypeId subTy, bool reversed) { ok = false; errors.push_back(TypeError{location, UnknownProperty{superTy, propName}}); - if (!FFlag::LuauExtendedClassMismatchError) - tryUnify_(prop.type, getSingletonTypes().errorRecoveryType()); } else { - if (FFlag::LuauExtendedClassMismatchError) + Unifier innerState = makeChildUnifier(); + innerState.tryUnify_(prop.type, classProp->type); + + checkChildUnifierTypeMismatch(innerState.errors, propName, reversed ? subTy : superTy, reversed ? superTy : subTy); + + if (innerState.errors.empty()) { - Unifier innerState = makeChildUnifier(); - innerState.tryUnify_(prop.type, classProp->type); - - checkChildUnifierTypeMismatch(innerState.errors, propName, reversed ? subTy : superTy, reversed ? superTy : subTy); - - if (innerState.errors.empty()) - { - log.concat(std::move(innerState.log)); - } - else - { - ok = false; - innerState.log.rollback(); - } + log.concat(std::move(innerState.log)); } else { - tryUnify_(prop.type, classProp->type); + ok = false; + innerState.log.rollback(); } } } @@ -2185,8 +2142,6 @@ void Unifier::checkChildUnifierTypeMismatch(const ErrorVec& innerErrors, TypeId void Unifier::checkChildUnifierTypeMismatch(const ErrorVec& innerErrors, const std::string& prop, TypeId wantedType, TypeId givenType) { - LUAU_ASSERT(FFlag::LuauExtendedTypeMismatchError || FFlag::LuauExtendedClassMismatchError); - if (auto e = hasUnificationTooComplex(innerErrors)) errors.push_back(*e); else if (!innerErrors.empty()) diff --git a/Ast/src/Parser.cpp b/Ast/src/Parser.cpp index dd24f27c..72f61649 100644 --- a/Ast/src/Parser.cpp +++ b/Ast/src/Parser.cpp @@ -14,7 +14,6 @@ LUAU_FASTFLAGVARIABLE(LuauIfElseExpressionBaseSupport, false) LUAU_FASTFLAGVARIABLE(LuauIfStatementRecursionGuard, false) LUAU_FASTFLAGVARIABLE(LuauFixAmbiguousErrorRecoveryInAssign, false) LUAU_FASTFLAGVARIABLE(LuauParseSingletonTypes, false) -LUAU_FASTFLAGVARIABLE(LuauParseGenericFunctionTypeBegin, false) namespace Luau { @@ -1368,9 +1367,6 @@ AstTypeOrPack Parser::parseFunctionTypeAnnotation(bool allowPack) Lexeme parameterStart = lexer.current(); - if (!FFlag::LuauParseGenericFunctionTypeBegin) - begin = parameterStart; - expectAndConsume('(', "function parameters"); matchRecoveryStopOnToken[Lexeme::SkinnyArrow]++; diff --git a/Sources.cmake b/Sources.cmake index 14834b3a..a7153eb3 100644 --- a/Sources.cmake +++ b/Sources.cmake @@ -45,6 +45,7 @@ target_sources(Luau.Analysis PRIVATE Analysis/include/Luau/IostreamHelpers.h Analysis/include/Luau/JsonEncoder.h Analysis/include/Luau/Linter.h + Analysis/include/Luau/LValue.h Analysis/include/Luau/Module.h Analysis/include/Luau/ModuleResolver.h Analysis/include/Luau/Predicate.h @@ -80,8 +81,8 @@ target_sources(Luau.Analysis PRIVATE Analysis/src/IostreamHelpers.cpp Analysis/src/JsonEncoder.cpp Analysis/src/Linter.cpp + Analysis/src/LValue.cpp Analysis/src/Module.cpp - Analysis/src/Predicate.cpp Analysis/src/Quantify.cpp Analysis/src/RequireTracer.cpp Analysis/src/Scope.cpp @@ -194,10 +195,10 @@ if(TARGET Luau.UnitTest) tests/Frontend.test.cpp tests/JsonEncoder.test.cpp tests/Linter.test.cpp + tests/LValue.test.cpp tests/Module.test.cpp tests/NonstrictMode.test.cpp tests/Parser.test.cpp - tests/Predicate.test.cpp tests/RequireTracer.test.cpp tests/StringUtils.test.cpp tests/Symbol.test.cpp diff --git a/VM/src/lbaselib.cpp b/VM/src/lbaselib.cpp index 881c804d..988fd315 100644 --- a/VM/src/lbaselib.cpp +++ b/VM/src/lbaselib.cpp @@ -36,12 +36,14 @@ static int luaB_tonumber(lua_State* L) int base = luaL_optinteger(L, 2, 10); if (base == 10) { /* standard conversion */ - luaL_checkany(L, 1); - if (lua_isnumber(L, 1)) + int isnum = 0; + double n = lua_tonumberx(L, 1, &isnum); + if (isnum) { - lua_pushnumber(L, lua_tonumber(L, 1)); + lua_pushnumber(L, n); return 1; } + luaL_checkany(L, 1); /* error if we don't have any argument */ } else { diff --git a/tests/Autocomplete.test.cpp b/tests/Autocomplete.test.cpp index 62a9999b..8ca09c0e 100644 --- a/tests/Autocomplete.test.cpp +++ b/tests/Autocomplete.test.cpp @@ -1394,7 +1394,7 @@ TEST_CASE_FIXTURE(ACFixture, "type_correct_function_return_types") check(R"( local function target(a: number, b: string) return a + #b end local function bar1(a: number) return -a end -local function bar2(a: string) reutrn a .. 'x' end +local function bar2(a: string) return a .. 'x' end return target(b@1 )"); @@ -1422,7 +1422,7 @@ return target(bar1, b@1 check(R"( local function target(a: number, b: string) return a + #b end local function bar1(a: number): (...number) return -a, a end -local function bar2(a: string) reutrn a .. 'x' end +local function bar2(a: string) return a .. 'x' end return target(b@1 )"); @@ -1918,7 +1918,7 @@ TEST_CASE_FIXTURE(ACFixture, "type_correct_function_no_parenthesis") check(R"( local function target(a: (number) -> number) return a(4) end local function bar1(a: number) return -a end -local function bar2(a: string) reutrn a .. 'x' end +local function bar2(a: string) return a .. 'x' end return target(b@1 )"); diff --git a/tests/LValue.test.cpp b/tests/LValue.test.cpp new file mode 100644 index 00000000..8a092779 --- /dev/null +++ b/tests/LValue.test.cpp @@ -0,0 +1,198 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +#include "Luau/TypeInfer.h" + +#include "Fixture.h" +#include "ScopedFlags.h" + +#include "doctest.h" + +using namespace Luau; + +static void merge(TypeArena& arena, RefinementMap& l, const RefinementMap& r) +{ + Luau::merge(l, r, [&arena](TypeId a, TypeId b) -> TypeId { + // TODO: normalize here also. + std::unordered_set s; + + if (auto utv = get(follow(a))) + s.insert(begin(utv), end(utv)); + else + s.insert(a); + + if (auto utv = get(follow(b))) + s.insert(begin(utv), end(utv)); + else + s.insert(b); + + std::vector options(s.begin(), s.end()); + return options.size() == 1 ? options[0] : arena.addType(UnionTypeVar{std::move(options)}); + }); +} + +static LValue mkSymbol(const std::string& s) +{ + return Symbol{AstName{s.data()}}; +} + +TEST_SUITE_BEGIN("LValue"); + +TEST_CASE("Luau_merge_hashmap_order") +{ + ScopedFastFlag sff{"LuauLValueAsKey", true}; + + std::string a = "a"; + std::string b = "b"; + std::string c = "c"; + + RefinementMap m{{ + {mkSymbol(b), getSingletonTypes().stringType}, + {mkSymbol(c), getSingletonTypes().numberType}, + }}; + + RefinementMap other{{ + {mkSymbol(a), getSingletonTypes().stringType}, + {mkSymbol(b), getSingletonTypes().stringType}, + {mkSymbol(c), getSingletonTypes().booleanType}, + }}; + + TypeArena arena; + merge(arena, m, other); + + REQUIRE_EQ(3, m.NEW_refinements.size()); + REQUIRE(m.NEW_refinements.count(mkSymbol(a))); + REQUIRE(m.NEW_refinements.count(mkSymbol(b))); + REQUIRE(m.NEW_refinements.count(mkSymbol(c))); + + CHECK_EQ("string", toString(m.NEW_refinements[mkSymbol(a)])); + CHECK_EQ("string", toString(m.NEW_refinements[mkSymbol(b)])); + CHECK_EQ("boolean | number", toString(m.NEW_refinements[mkSymbol(c)])); +} + +TEST_CASE("Luau_merge_hashmap_order2") +{ + ScopedFastFlag sff{"LuauLValueAsKey", true}; + + std::string a = "a"; + std::string b = "b"; + std::string c = "c"; + + RefinementMap m{{ + {mkSymbol(a), getSingletonTypes().stringType}, + {mkSymbol(b), getSingletonTypes().stringType}, + {mkSymbol(c), getSingletonTypes().numberType}, + }}; + + RefinementMap other{{ + {mkSymbol(b), getSingletonTypes().stringType}, + {mkSymbol(c), getSingletonTypes().booleanType}, + }}; + + TypeArena arena; + merge(arena, m, other); + + REQUIRE_EQ(3, m.NEW_refinements.size()); + REQUIRE(m.NEW_refinements.count(mkSymbol(a))); + REQUIRE(m.NEW_refinements.count(mkSymbol(b))); + REQUIRE(m.NEW_refinements.count(mkSymbol(c))); + + CHECK_EQ("string", toString(m.NEW_refinements[mkSymbol(a)])); + CHECK_EQ("string", toString(m.NEW_refinements[mkSymbol(b)])); + CHECK_EQ("boolean | number", toString(m.NEW_refinements[mkSymbol(c)])); +} + +TEST_CASE("one_map_has_overlap_at_end_whereas_other_has_it_in_start") +{ + ScopedFastFlag sff{"LuauLValueAsKey", true}; + + std::string a = "a"; + std::string b = "b"; + std::string c = "c"; + std::string d = "d"; + std::string e = "e"; + + RefinementMap m{{ + {mkSymbol(a), getSingletonTypes().stringType}, + {mkSymbol(b), getSingletonTypes().numberType}, + {mkSymbol(c), getSingletonTypes().booleanType}, + }}; + + RefinementMap other{{ + {mkSymbol(c), getSingletonTypes().stringType}, + {mkSymbol(d), getSingletonTypes().numberType}, + {mkSymbol(e), getSingletonTypes().booleanType}, + }}; + + TypeArena arena; + merge(arena, m, other); + + REQUIRE_EQ(5, m.NEW_refinements.size()); + REQUIRE(m.NEW_refinements.count(mkSymbol(a))); + REQUIRE(m.NEW_refinements.count(mkSymbol(b))); + REQUIRE(m.NEW_refinements.count(mkSymbol(c))); + REQUIRE(m.NEW_refinements.count(mkSymbol(d))); + REQUIRE(m.NEW_refinements.count(mkSymbol(e))); + + CHECK_EQ("string", toString(m.NEW_refinements[mkSymbol(a)])); + CHECK_EQ("number", toString(m.NEW_refinements[mkSymbol(b)])); + CHECK_EQ("boolean | string", toString(m.NEW_refinements[mkSymbol(c)])); + CHECK_EQ("number", toString(m.NEW_refinements[mkSymbol(d)])); + CHECK_EQ("boolean", toString(m.NEW_refinements[mkSymbol(e)])); +} + +TEST_CASE("hashing_lvalue_global_prop_access") +{ + std::string t1 = "t"; + std::string x1 = "x"; + + LValue t_x1{Field{std::make_shared(Symbol{AstName{t1.data()}}), x1}}; + + std::string t2 = "t"; + std::string x2 = "x"; + + LValue t_x2{Field{std::make_shared(Symbol{AstName{t2.data()}}), x2}}; + + CHECK_EQ(t_x1, t_x1); + CHECK_EQ(t_x1, t_x2); + CHECK_EQ(t_x2, t_x2); + + CHECK_EQ(LValueHasher{}(t_x1), LValueHasher{}(t_x1)); + CHECK_EQ(LValueHasher{}(t_x1), LValueHasher{}(t_x2)); + CHECK_EQ(LValueHasher{}(t_x2), LValueHasher{}(t_x2)); + + NEW_RefinementMap m; + m[t_x1] = getSingletonTypes().stringType; + m[t_x2] = getSingletonTypes().numberType; + + CHECK_EQ(1, m.size()); +} + +TEST_CASE("hashing_lvalue_local_prop_access") +{ + std::string t1 = "t"; + std::string x1 = "x"; + + AstLocal localt1{AstName{t1.data()}, Location(), nullptr, 0, 0, nullptr}; + LValue t_x1{Field{std::make_shared(Symbol{&localt1}), x1}}; + + std::string t2 = "t"; + std::string x2 = "x"; + + AstLocal localt2{AstName{t2.data()}, Location(), &localt1, 0, 0, nullptr}; + LValue t_x2{Field{std::make_shared(Symbol{&localt2}), x2}}; + + CHECK_EQ(t_x1, t_x1); + CHECK_NE(t_x1, t_x2); + CHECK_EQ(t_x2, t_x2); + + CHECK_EQ(LValueHasher{}(t_x1), LValueHasher{}(t_x1)); + CHECK_NE(LValueHasher{}(t_x1), LValueHasher{}(t_x2)); + CHECK_EQ(LValueHasher{}(t_x2), LValueHasher{}(t_x2)); + + NEW_RefinementMap m; + m[t_x1] = getSingletonTypes().stringType; + m[t_x2] = getSingletonTypes().numberType; + + CHECK_EQ(2, m.size()); +} + +TEST_SUITE_END(); diff --git a/tests/Predicate.test.cpp b/tests/Predicate.test.cpp deleted file mode 100644 index 7081693e..00000000 --- a/tests/Predicate.test.cpp +++ /dev/null @@ -1,117 +0,0 @@ -// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details -#include "Luau/TypeInfer.h" - -#include "Fixture.h" -#include "ScopedFlags.h" - -#include "doctest.h" - -using namespace Luau; - -static void merge(TypeArena& arena, RefinementMap& l, const RefinementMap& r) -{ - Luau::merge(l, r, [&arena](TypeId a, TypeId b) -> TypeId { - // TODO: normalize here also. - std::unordered_set s; - - if (auto utv = get(follow(a))) - s.insert(begin(utv), end(utv)); - else - s.insert(a); - - if (auto utv = get(follow(b))) - s.insert(begin(utv), end(utv)); - else - s.insert(b); - - std::vector options(s.begin(), s.end()); - return options.size() == 1 ? options[0] : arena.addType(UnionTypeVar{std::move(options)}); - }); -} - -TEST_SUITE_BEGIN("Predicate"); - -TEST_CASE_FIXTURE(Fixture, "Luau_merge_hashmap_order") -{ - RefinementMap m{ - {"b", typeChecker.stringType}, - {"c", typeChecker.numberType}, - }; - - RefinementMap other{ - {"a", typeChecker.stringType}, - {"b", typeChecker.stringType}, - {"c", typeChecker.booleanType}, - }; - - TypeArena arena; - merge(arena, m, other); - - REQUIRE_EQ(3, m.size()); - REQUIRE(m.count("a")); - REQUIRE(m.count("b")); - REQUIRE(m.count("c")); - - CHECK_EQ("string", toString(m["a"])); - CHECK_EQ("string", toString(m["b"])); - CHECK_EQ("boolean | number", toString(m["c"])); -} - -TEST_CASE_FIXTURE(Fixture, "Luau_merge_hashmap_order2") -{ - RefinementMap m{ - {"a", typeChecker.stringType}, - {"b", typeChecker.stringType}, - {"c", typeChecker.numberType}, - }; - - RefinementMap other{ - {"b", typeChecker.stringType}, - {"c", typeChecker.booleanType}, - }; - - TypeArena arena; - merge(arena, m, other); - - REQUIRE_EQ(3, m.size()); - REQUIRE(m.count("a")); - REQUIRE(m.count("b")); - REQUIRE(m.count("c")); - - CHECK_EQ("string", toString(m["a"])); - CHECK_EQ("string", toString(m["b"])); - CHECK_EQ("boolean | number", toString(m["c"])); -} - -TEST_CASE_FIXTURE(Fixture, "one_map_has_overlap_at_end_whereas_other_has_it_in_start") -{ - RefinementMap m{ - {"a", typeChecker.stringType}, - {"b", typeChecker.numberType}, - {"c", typeChecker.booleanType}, - }; - - RefinementMap other{ - {"c", typeChecker.stringType}, - {"d", typeChecker.numberType}, - {"e", typeChecker.booleanType}, - }; - - TypeArena arena; - merge(arena, m, other); - - REQUIRE_EQ(5, m.size()); - REQUIRE(m.count("a")); - REQUIRE(m.count("b")); - REQUIRE(m.count("c")); - REQUIRE(m.count("d")); - REQUIRE(m.count("e")); - - CHECK_EQ("string", toString(m["a"])); - CHECK_EQ("number", toString(m["b"])); - CHECK_EQ("boolean | string", toString(m["c"])); - CHECK_EQ("number", toString(m["d"])); - CHECK_EQ("boolean", toString(m["e"])); -} - -TEST_SUITE_END(); diff --git a/tests/Symbol.test.cpp b/tests/Symbol.test.cpp index 44fe3a3c..e7d2973b 100644 --- a/tests/Symbol.test.cpp +++ b/tests/Symbol.test.cpp @@ -10,7 +10,7 @@ using namespace Luau; TEST_SUITE_BEGIN("SymbolTests"); -TEST_CASE("hashing") +TEST_CASE("hashing_globals") { std::string s1 = "name"; std::string s2 = "name"; @@ -31,10 +31,37 @@ TEST_CASE("hashing") CHECK_EQ(std::hash()(two), std::hash()(two)); std::unordered_map theMap; - theMap[AstName{s1.data()}] = 5; - theMap[AstName{s2.data()}] = 1; + theMap[n1] = 5; + theMap[n2] = 1; REQUIRE_EQ(1, theMap.size()); } +TEST_CASE("hashing_locals") +{ + std::string s1 = "name"; + std::string s2 = "name"; + + // These two names point to distinct memory areas. + AstLocal one{AstName{s1.data()}, Location(), nullptr, 0, 0, nullptr}; + AstLocal two{AstName{s2.data()}, Location(), &one, 0, 0, nullptr}; + + Symbol n1{&one}; + Symbol n2{&two}; + + CHECK(n1 == n1); + CHECK(n1 != n2); + CHECK(n2 == n2); + + CHECK_EQ(std::hash()(&one), std::hash()(&one)); + CHECK_NE(std::hash()(&one), std::hash()(&two)); + CHECK_EQ(std::hash()(&two), std::hash()(&two)); + + std::unordered_map theMap; + theMap[n1] = 5; + theMap[n2] = 1; + + REQUIRE_EQ(2, theMap.size()); +} + TEST_SUITE_END(); diff --git a/tests/Transpiler.test.cpp b/tests/Transpiler.test.cpp index 327fa0bb..47c3883c 100644 --- a/tests/Transpiler.test.cpp +++ b/tests/Transpiler.test.cpp @@ -555,8 +555,6 @@ TEST_CASE_FIXTURE(Fixture, "transpile_assign_multiple") TEST_CASE_FIXTURE(Fixture, "transpile_generic_function") { - ScopedFastFlag luauParseGenericFunctionTypeBegin("LuauParseGenericFunctionTypeBegin", true); - std::string code = R"( local function foo(a: T, ...: S...) return 1 end local f: (T, S...)->(number) = foo diff --git a/tests/TypeInfer.builtins.test.cpp b/tests/TypeInfer.builtins.test.cpp index 1d8135d4..506279b9 100644 --- a/tests/TypeInfer.builtins.test.cpp +++ b/tests/TypeInfer.builtins.test.cpp @@ -798,13 +798,14 @@ TEST_CASE_FIXTURE(Fixture, "string_format_report_all_type_errors_at_correct_posi { CheckResult result = check(R"( ("%s%d%s"):format(1, "hello", true) + string.format("%s%d%s", 1, "hello", true) )"); TypeId stringType = typeChecker.stringType; TypeId numberType = typeChecker.numberType; TypeId booleanType = typeChecker.booleanType; - LUAU_REQUIRE_ERROR_COUNT(3, result); + LUAU_REQUIRE_ERROR_COUNT(6, result); CHECK_EQ(Location(Position{1, 26}, Position{1, 27}), result.errors[0].location); CHECK_EQ(TypeErrorData(TypeMismatch{stringType, numberType}), result.errors[0].data); @@ -814,6 +815,15 @@ TEST_CASE_FIXTURE(Fixture, "string_format_report_all_type_errors_at_correct_posi CHECK_EQ(Location(Position{1, 38}, Position{1, 42}), result.errors[2].location); CHECK_EQ(TypeErrorData(TypeMismatch{stringType, booleanType}), result.errors[2].data); + + CHECK_EQ(Location(Position{2, 32}, Position{2, 33}), result.errors[3].location); + CHECK_EQ(TypeErrorData(TypeMismatch{stringType, numberType}), result.errors[3].data); + + CHECK_EQ(Location(Position{2, 35}, Position{2, 42}), result.errors[4].location); + CHECK_EQ(TypeErrorData(TypeMismatch{numberType, stringType}), result.errors[4].data); + + CHECK_EQ(Location(Position{2, 44}, Position{2, 48}), result.errors[5].location); + CHECK_EQ(TypeErrorData(TypeMismatch{stringType, booleanType}), result.errors[5].data); } TEST_CASE_FIXTURE(Fixture, "tonumber_returns_optional_number_type") diff --git a/tests/TypeInfer.classes.test.cpp b/tests/TypeInfer.classes.test.cpp index 1ff23fe6..0283ae19 100644 --- a/tests/TypeInfer.classes.test.cpp +++ b/tests/TypeInfer.classes.test.cpp @@ -449,8 +449,6 @@ b.X = 2 -- real Vector2.X is also read-only TEST_CASE_FIXTURE(ClassFixture, "detailed_class_unification_error") { - ScopedFastFlag luauExtendedClassMismatchError{"LuauExtendedClassMismatchError", true}; - CheckResult result = check(R"( local function foo(v) return v.X :: number + string.len(v.Y) diff --git a/tests/TypeInfer.intersectionTypes.test.cpp b/tests/TypeInfer.intersectionTypes.test.cpp index 893bc2b3..93c0baf6 100644 --- a/tests/TypeInfer.intersectionTypes.test.cpp +++ b/tests/TypeInfer.intersectionTypes.test.cpp @@ -343,8 +343,6 @@ TEST_CASE_FIXTURE(Fixture, "table_intersection_setmetatable") TEST_CASE_FIXTURE(Fixture, "error_detailed_intersection_part") { - ScopedFastFlag luauExtendedTypeMismatchError{"LuauExtendedTypeMismatchError", true}; - CheckResult result = check(R"( type X = { x: number } type Y = { y: number } @@ -363,8 +361,6 @@ caused by: TEST_CASE_FIXTURE(Fixture, "error_detailed_intersection_all") { - ScopedFastFlag luauExtendedTypeMismatchError{"LuauExtendedTypeMismatchError", true}; - CheckResult result = check(R"( type X = { x: number } type Y = { y: number } diff --git a/tests/TypeInfer.refinements.test.cpp b/tests/TypeInfer.refinements.test.cpp index 688680c1..503b613f 100644 --- a/tests/TypeInfer.refinements.test.cpp +++ b/tests/TypeInfer.refinements.test.cpp @@ -280,7 +280,6 @@ TEST_CASE_FIXTURE(Fixture, "assert_non_binary_expressions_actually_resolve_const TEST_CASE_FIXTURE(Fixture, "assign_table_with_refined_property_with_a_similar_type_is_illegal") { ScopedFastFlag LuauTableSubtypingVariance2{"LuauTableSubtypingVariance2", true}; - ScopedFastFlag luauExtendedTypeMismatchError{"LuauExtendedTypeMismatchError", true}; CheckResult result = check(R"( local t: {x: number?} = {x = nil} @@ -1085,6 +1084,41 @@ TEST_CASE_FIXTURE(Fixture, "type_comparison_ifelse_expression") CHECK_EQ("any", toString(requireTypeAtPosition({6, 66}))); } +TEST_CASE_FIXTURE(Fixture, "correctly_lookup_a_shadowed_local_that_which_was_previously_refined") +{ + ScopedFastFlag sff{"LuauLValueAsKey", true}; + + CheckResult result = check(R"( + local foo: string? = "hi" + assert(foo) + local foo: number = 5 + print(foo:sub(1, 1)) + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + CHECK_EQ("Type 'number' does not have key 'sub'", toString(result.errors[0])); +} + +TEST_CASE_FIXTURE(Fixture, "correctly_lookup_property_whose_base_was_previously_refined") +{ + ScopedFastFlag sff{"LuauLValueAsKey", true}; + + CheckResult result = check(R"( + type T = {x: string | number} + local t: T? = {x = "hi"} + if t then + if type(t.x) == "string" then + local foo = t.x + end + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ("string", toString(requireTypeAtPosition({5, 30}))); +} + TEST_CASE_FIXTURE(Fixture, "apply_refinements_on_astexprindexexpr_whose_subscript_expr_is_constant_string") { ScopedFastFlag sff{"LuauRefiLookupFromIndexExpr", true}; @@ -1092,6 +1126,7 @@ TEST_CASE_FIXTURE(Fixture, "apply_refinements_on_astexprindexexpr_whose_subscrip CheckResult result = check(R"( type T = { [string]: { prop: number }? } local t: T = {} + if t["hello"] then local foo = t["hello"].prop end diff --git a/tests/TypeInfer.singletons.test.cpp b/tests/TypeInfer.singletons.test.cpp index 1621ef32..68dc1b4f 100644 --- a/tests/TypeInfer.singletons.test.cpp +++ b/tests/TypeInfer.singletons.test.cpp @@ -202,7 +202,6 @@ TEST_CASE_FIXTURE(Fixture, "enums_using_singletons_mismatch") ScopedFastFlag sffs[] = { {"LuauSingletonTypes", true}, {"LuauParseSingletonTypes", true}, - {"LuauExtendedTypeMismatchError", true}, }; CheckResult result = check(R"( diff --git a/tests/TypeInfer.tables.test.cpp b/tests/TypeInfer.tables.test.cpp index 3ea9b80c..80f40407 100644 --- a/tests/TypeInfer.tables.test.cpp +++ b/tests/TypeInfer.tables.test.cpp @@ -1955,7 +1955,6 @@ TEST_CASE_FIXTURE(Fixture, "table_insert_should_cope_with_optional_properties_in TEST_CASE_FIXTURE(Fixture, "error_detailed_prop") { ScopedFastFlag LuauTableSubtypingVariance2{"LuauTableSubtypingVariance2", true}; // Only for new path - ScopedFastFlag luauExtendedTypeMismatchError{"LuauExtendedTypeMismatchError", true}; CheckResult result = check(R"( type A = { x: number, y: number } @@ -1974,7 +1973,6 @@ caused by: TEST_CASE_FIXTURE(Fixture, "error_detailed_prop_nested") { ScopedFastFlag LuauTableSubtypingVariance2{"LuauTableSubtypingVariance2", true}; // Only for new path - ScopedFastFlag luauExtendedTypeMismatchError{"LuauExtendedTypeMismatchError", true}; CheckResult result = check(R"( type AS = { x: number, y: number } @@ -1998,7 +1996,6 @@ caused by: TEST_CASE_FIXTURE(Fixture, "error_detailed_metatable_prop") { ScopedFastFlag LuauTableSubtypingVariance2{"LuauTableSubtypingVariance2", true}; // Only for new path - ScopedFastFlag luauExtendedTypeMismatchError{"LuauExtendedTypeMismatchError", true}; CheckResult result = check(R"( local a1 = setmetatable({ x = 2, y = 3 }, { __call = function(s) end }); @@ -2062,7 +2059,6 @@ TEST_CASE_FIXTURE(Fixture, "explicitly_typed_table_error") {"LuauPropertiesGetExpectedType", true}, {"LuauExpectedTypesOfProperties", true}, {"LuauTableSubtypingVariance2", true}, - {"LuauExtendedTypeMismatchError", true}, }; CheckResult result = check(R"( diff --git a/tests/TypeInfer.test.cpp b/tests/TypeInfer.test.cpp index ad9ea827..76324556 100644 --- a/tests/TypeInfer.test.cpp +++ b/tests/TypeInfer.test.cpp @@ -5013,6 +5013,19 @@ caused by: Return #2 type is not compatible. Type 'string' could not be converted into 'boolean')"); } +TEST_CASE_FIXTURE(Fixture, "prop_access_on_any_with_other_options") +{ + ScopedFastFlag sff{"LuauLValueAsKey", true}; + + CheckResult result = check(R"( + local function f(thing: any | string) + local foo = thing.SomeRandomKey + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + TEST_CASE_FIXTURE(Fixture, "table_function_check_use_after_free") { ScopedFastFlag luauUnifyFunctionCheckResult{"LuauUpdateFunctionNameBinding", true}; diff --git a/tests/TypeInfer.unionTypes.test.cpp b/tests/TypeInfer.unionTypes.test.cpp index b095a0db..2357869e 100644 --- a/tests/TypeInfer.unionTypes.test.cpp +++ b/tests/TypeInfer.unionTypes.test.cpp @@ -425,8 +425,6 @@ y = x TEST_CASE_FIXTURE(Fixture, "error_detailed_union_part") { - ScopedFastFlag luauExtendedTypeMismatchError{"LuauExtendedTypeMismatchError", true}; - CheckResult result = check(R"( type X = { x: number } type Y = { y: number } @@ -446,8 +444,6 @@ caused by: TEST_CASE_FIXTURE(Fixture, "error_detailed_union_all") { - ScopedFastFlag luauExtendedTypeMismatchError{"LuauExtendedTypeMismatchError", true}; - CheckResult result = check(R"( type X = { x: number } type Y = { y: number } diff --git a/tests/conformance/math.lua b/tests/conformance/math.lua index d5bca44f..bfea0e1f 100644 --- a/tests/conformance/math.lua +++ b/tests/conformance/math.lua @@ -26,6 +26,7 @@ function f(...) end end +assert(pcall(tonumber) == false) assert(tonumber{} == nil) assert(tonumber'+0.01' == 1/100 and tonumber'+.01' == 0.01 and tonumber'.01' == 0.01 and tonumber'-1.' == -1 and From d189bd9b1a3c6ecc882c793c7382a1c886635900 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Thu, 6 Jan 2022 17:37:50 -0800 Subject: [PATCH 117/399] Enable V2Read flag early --- VM/src/lvmload.cpp | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/VM/src/lvmload.cpp b/VM/src/lvmload.cpp index cdb276c0..7839c68c 100644 --- a/VM/src/lvmload.cpp +++ b/VM/src/lvmload.cpp @@ -13,7 +13,7 @@ #include -LUAU_FASTFLAGVARIABLE(LuauBytecodeV2Read, false) +LUAU_FASTFLAGVARIABLE(LuauBytecodeV2Read, true) LUAU_FASTFLAGVARIABLE(LuauBytecodeV2Force, false) // TODO: RAII deallocation doesn't work for longjmp builds if a memory error happens From d50b0793257b39cd70f7bbf0057f37f718b1beb9 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Thu, 6 Jan 2022 17:46:53 -0800 Subject: [PATCH 118/399] Sync to upstream/release/509 (#303) - Rework transaction log used for type checking which should result in more robust type checking internals with fewer bugs - Reduce the amount of memory consumed by type checker on large module graphs - Type checker now errors on attempts to change the type of imported module fields - The return type of newproxy is now any (fixes #296) - Implement new number printing algorithm (Schubfach) which makes tostring() produce precise (round-trippable) and short decimal output up to 10x faster - Fix lua_Debug::linedefined to point to the line with the function definition instead of the first statement (fixes #265) - Fix minor bugs in Tab completion in Repl - Repl now saves/restores command history in ~/.luau_history --- .gitignore | 18 +- Analysis/include/Luau/Frontend.h | 10 +- Analysis/include/Luau/TxnLog.h | 280 ++- Analysis/include/Luau/TypeInfer.h | 40 +- Analysis/include/Luau/TypePack.h | 8 + Analysis/include/Luau/TypeVar.h | 1 + Analysis/include/Luau/TypedAllocator.h | 23 +- Analysis/include/Luau/Unifier.h | 48 +- Analysis/src/Autocomplete.cpp | 33 +- Analysis/src/EmbeddedBuiltinDefinitions.cpp | 7 +- Analysis/src/Frontend.cpp | 28 +- Analysis/src/Module.cpp | 2 +- Analysis/src/Quantify.cpp | 14 +- Analysis/src/ToString.cpp | 48 +- Analysis/src/TxnLog.cpp | 295 ++- Analysis/src/TypeInfer.cpp | 863 ++++--- Analysis/src/TypePack.cpp | 51 +- Analysis/src/TypeVar.cpp | 19 +- Analysis/src/TypedAllocator.cpp | 1 + Analysis/src/Unifier.cpp | 2342 +++++++++++++------ Ast/include/Luau/Common.h | 8 +- CLI/Analyze.cpp | 2 +- CLI/Repl.cpp | 46 +- CLI/Web.cpp | 24 +- Compiler/include/Luau/Bytecode.h | 1 + Compiler/include/Luau/BytecodeBuilder.h | 2 + Compiler/src/BytecodeBuilder.cpp | 69 +- Compiler/src/Compiler.cpp | 7 +- Sources.cmake | 1 + VM/include/luaconf.h | 4 - VM/src/lapi.cpp | 34 +- VM/src/laux.cpp | 38 +- VM/src/lbitlib.cpp | 8 - VM/src/ldebug.cpp | 17 +- VM/src/ldo.cpp | 29 +- VM/src/lgc.cpp | 2 - VM/src/lgcdebug.cpp | 7 +- VM/src/lnumprint.cpp | 375 +++ VM/src/lnumutils.h | 7 +- VM/src/lobject.h | 1 + VM/src/ltable.cpp | 6 +- VM/src/lvmload.cpp | 28 +- VM/src/lvmutils.cpp | 7 +- fuzz/number.cpp | 35 + tests/AstQuery.test.cpp | 2 - tests/Autocomplete.test.cpp | 43 +- tests/Conformance.test.cpp | 31 +- tests/Fixture.cpp | 5 - tests/Fixture.h | 9 - tests/Frontend.test.cpp | 7 +- tests/TypeInfer.annotations.test.cpp | 2 +- tests/TypeInfer.builtins.test.cpp | 37 +- tests/TypeInfer.generics.test.cpp | 2 - tests/TypeInfer.test.cpp | 2 - tests/TypeInfer.tryUnify.test.cpp | 54 +- tests/TypeInfer.typePacks.cpp | 14 +- tests/TypeInfer.unionTypes.test.cpp | 10 +- tests/TypeVar.test.cpp | 4 +- tests/conformance/debug.lua | 9 + tests/conformance/strconv.lua | 51 + tests/main.cpp | 23 +- tools/numprint.py | 82 + 62 files changed, 3901 insertions(+), 1375 deletions(-) create mode 100644 VM/src/lnumprint.cpp create mode 100644 fuzz/number.cpp create mode 100644 tests/conformance/strconv.lua create mode 100644 tools/numprint.py diff --git a/.gitignore b/.gitignore index fa11b45b..5688dff5 100644 --- a/.gitignore +++ b/.gitignore @@ -1,8 +1,10 @@ -^build/ -^coverage/ -^fuzz/luau.pb.* -^crash-* -^default.prof* -^fuzz-* -^luau$ -/.vs +/build/ +/build[.-]*/ +/coverage/ +/.vs/ +/.vscode/ +/fuzz/luau.pb.* +/crash-* +/default.prof* +/fuzz-* +/luau diff --git a/Analysis/include/Luau/Frontend.h b/Analysis/include/Luau/Frontend.h index 07a0296a..1f64db30 100644 --- a/Analysis/include/Luau/Frontend.h +++ b/Analysis/include/Luau/Frontend.h @@ -68,7 +68,7 @@ struct FrontendOptions // is complete. bool retainFullTypeGraphs = false; - // When true, we run typechecking twice, one in the regular mode, ond once in strict mode + // When true, we run typechecking twice, once in the regular mode, and once in strict mode // in order to get more precise type information (e.g. for autocomplete). bool typecheckTwice = false; }; @@ -109,18 +109,18 @@ struct Frontend Frontend(FileResolver* fileResolver, ConfigResolver* configResolver, const FrontendOptions& options = {}); - CheckResult check(const ModuleName& name); // new shininess - LintResult lint(const ModuleName& name, std::optional enabledLintWarnings = {}); + CheckResult check(const ModuleName& name, std::optional optionOverride = {}); // new shininess + LintResult lint(const ModuleName& name, std::optional enabledLintWarnings = {}); /** Lint some code that has no associated DataModel object * * Since this source fragment has no name, we cannot cache its AST. Instead, * we return it to the caller to use as they wish. */ - std::pair lintFragment(std::string_view source, std::optional enabledLintWarnings = {}); + std::pair lintFragment(std::string_view source, std::optional enabledLintWarnings = {}); CheckResult check(const SourceModule& module); // OLD. TODO KILL - LintResult lint(const SourceModule& module, std::optional enabledLintWarnings = {}); + LintResult lint(const SourceModule& module, std::optional enabledLintWarnings = {}); bool isDirty(const ModuleName& name) const; void markDirty(const ModuleName& name, std::vector* markedDirty = nullptr); diff --git a/Analysis/include/Luau/TxnLog.h b/Analysis/include/Luau/TxnLog.h index 29988a3b..dc45bebf 100644 --- a/Analysis/include/Luau/TxnLog.h +++ b/Analysis/include/Luau/TxnLog.h @@ -1,7 +1,11 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #pragma once +#include +#include + #include "Luau/TypeVar.h" +#include "Luau/TypePack.h" LUAU_FASTFLAG(LuauShareTxnSeen); @@ -9,27 +13,28 @@ namespace Luau { // Log of where what TypeIds we are rebinding and what they used to be -struct TxnLog +// Remove with LuauUseCommitTxnLog +struct DEPRECATED_TxnLog { - TxnLog() + DEPRECATED_TxnLog() : originalSeenSize(0) , ownedSeen() , sharedSeen(&ownedSeen) { } - explicit TxnLog(std::vector>* sharedSeen) + explicit DEPRECATED_TxnLog(std::vector>* sharedSeen) : originalSeenSize(sharedSeen->size()) , ownedSeen() , sharedSeen(sharedSeen) { } - TxnLog(const TxnLog&) = delete; - TxnLog& operator=(const TxnLog&) = delete; + DEPRECATED_TxnLog(const DEPRECATED_TxnLog&) = delete; + DEPRECATED_TxnLog& operator=(const DEPRECATED_TxnLog&) = delete; - TxnLog(TxnLog&&) = default; - TxnLog& operator=(TxnLog&&) = default; + DEPRECATED_TxnLog(DEPRECATED_TxnLog&&) = default; + DEPRECATED_TxnLog& operator=(DEPRECATED_TxnLog&&) = default; void operator()(TypeId a); void operator()(TypePackId a); @@ -37,7 +42,7 @@ struct TxnLog void rollback(); - void concat(TxnLog rhs); + void concat(DEPRECATED_TxnLog rhs); bool haveSeen(TypeId lhs, TypeId rhs); void pushSeen(TypeId lhs, TypeId rhs); @@ -54,4 +59,263 @@ public: std::vector>* sharedSeen; // shared with all the descendent logs }; +// Pending state for a TypeVar. Generated by a TxnLog and committed via +// TxnLog::commit. +struct PendingType +{ + // The pending TypeVar state. + TypeVar pending; + + explicit PendingType(TypeVar state) + : pending(std::move(state)) + { + } +}; + +// Pending state for a TypePackVar. Generated by a TxnLog and committed via +// TxnLog::commit. +struct PendingTypePack +{ + // The pending TypePackVar state. + TypePackVar pending; + + explicit PendingTypePack(TypePackVar state) + : pending(std::move(state)) + { + } +}; + +template +T* getMutable(PendingType* pending) +{ + // We use getMutable here because this state is intended to be mutated freely. + return getMutable(&pending->pending); +} + +template +T* getMutable(PendingTypePack* pending) +{ + // We use getMutable here because this state is intended to be mutated freely. + return getMutable(&pending->pending); +} + +// Log of what TypeIds we are rebinding, to be committed later. +struct TxnLog +{ + TxnLog() + : ownedSeen() + , sharedSeen(&ownedSeen) + { + } + + explicit TxnLog(TxnLog* parent) + : parent(parent) + { + if (parent) + { + sharedSeen = parent->sharedSeen; + } + else + { + sharedSeen = &ownedSeen; + } + } + + explicit TxnLog(std::vector>* sharedSeen) + : sharedSeen(sharedSeen) + { + } + + TxnLog(TxnLog* parent, std::vector>* sharedSeen) + : parent(parent) + , sharedSeen(sharedSeen) + { + } + + TxnLog(const TxnLog&) = delete; + TxnLog& operator=(const TxnLog&) = delete; + + TxnLog(TxnLog&&) = default; + TxnLog& operator=(TxnLog&&) = default; + + // Gets an empty TxnLog pointer. This is useful for constructs that + // take a TxnLog, like TypePackIterator - use the empty log if you + // don't have a TxnLog to give it. + static const TxnLog* empty(); + + // Joins another TxnLog onto this one. You should use std::move to avoid + // copying the rhs TxnLog. + // + // If both logs talk about the same type, pack, or table, the rhs takes + // priority. + void concat(TxnLog rhs); + + // Commits the TxnLog, rebinding all type pointers to their pending states. + // Clears the TxnLog afterwards. + void commit(); + + // Clears the TxnLog without committing any pending changes. + void clear(); + + // Computes an inverse of this TxnLog at the current time. + // This method should be called before commit is called in order to give an + // accurate result. Committing the inverse of a TxnLog will undo the changes + // made by commit, assuming the inverse log is accurate. + TxnLog inverse(); + + bool haveSeen(TypeId lhs, TypeId rhs) const; + void pushSeen(TypeId lhs, TypeId rhs); + void popSeen(TypeId lhs, TypeId rhs); + + // Queues a type for modification. The original type will not change until commit + // is called. Use pending to get the pending state. + // + // The pointer returned lives until `commit` or `clear` is called. + PendingType* queue(TypeId ty); + + // Queues a type pack for modification. The original type pack will not change + // until commit is called. Use pending to get the pending state. + // + // The pointer returned lives until `commit` or `clear` is called. + PendingTypePack* queue(TypePackId tp); + + // Returns the pending state of a type, or nullptr if there isn't any. It is important + // to note that this pending state is not transitive: the pending state may reference + // non-pending types freely, so you may need to call pending multiple times to view the + // entire pending state of a type graph. + // + // The pointer returned lives until `commit` or `clear` is called. + PendingType* pending(TypeId ty) const; + + // Returns the pending state of a type pack, or nullptr if there isn't any. It is + // important to note that this pending state is not transitive: the pending state may + // reference non-pending types freely, so you may need to call pending multiple times + // to view the entire pending state of a type graph. + // + // The pointer returned lives until `commit` or `clear` is called. + PendingTypePack* pending(TypePackId tp) const; + + // Queues a replacement of a type with another type. + // + // The pointer returned lives until `commit` or `clear` is called. + PendingType* replace(TypeId ty, TypeVar replacement); + + // Queues a replacement of a type pack with another type pack. + // + // The pointer returned lives until `commit` or `clear` is called. + PendingTypePack* replace(TypePackId tp, TypePackVar replacement); + + // Queues a replacement of a table type with another table type that is bound + // to a specific value. + // + // The pointer returned lives until `commit` or `clear` is called. + PendingType* bindTable(TypeId ty, std::optional newBoundTo); + + // Queues a replacement of a type with a level with a duplicate of that type + // with a new type level. + // + // The pointer returned lives until `commit` or `clear` is called. + PendingType* changeLevel(TypeId ty, TypeLevel newLevel); + + // Queues a replacement of a type pack with a level with a duplicate of that + // type pack with a new type level. + // + // The pointer returned lives until `commit` or `clear` is called. + PendingTypePack* changeLevel(TypePackId tp, TypeLevel newLevel); + + // Queues a replacement of a table type with another table type with a new + // indexer. + // + // The pointer returned lives until `commit` or `clear` is called. + PendingType* changeIndexer(TypeId ty, std::optional indexer); + + // Returns the type level of the pending state of the type, or the level of that + // type, if no pending state exists. If the type doesn't have a notion of a level, + // returns nullopt. If the pending state doesn't have a notion of a level, but the + // original state does, returns nullopt. + std::optional getLevel(TypeId ty) const; + + // Follows a type, accounting for pending type states. The returned type may have + // pending state; you should use `pending` or `get` to find out. + TypeId follow(TypeId ty); + + // Follows a type pack, accounting for pending type states. The returned type pack + // may have pending state; you should use `pending` or `get` to find out. + TypePackId follow(TypePackId tp) const; + + // Replaces a given type's state with a new variant. Returns the new pending state + // of that type. + // + // The pointer returned lives until `commit` or `clear` is called. + template + PendingType* replace(TypeId ty, T replacement) + { + return replace(ty, TypeVar(replacement)); + } + + // Replaces a given type pack's state with a new variant. Returns the new + // pending state of that type pack. + // + // The pointer returned lives until `commit` or `clear` is called. + template + PendingTypePack* replace(TypePackId tp, T replacement) + { + return replace(tp, TypePackVar(replacement)); + } + + // Returns T if a given type or type pack is this variant, respecting the + // log's pending state. + // + // Do not retain this pointer; it has the potential to be invalidated when + // commit or clear is called. + template + T* getMutable(TID ty) const + { + auto* pendingTy = pending(ty); + if (pendingTy) + return Luau::getMutable(pendingTy); + + return Luau::getMutable(ty); + } + + // Returns whether a given type or type pack is a given state, respecting the + // log's pending state. + // + // This method will not assert if called on a BoundTypeVar or BoundTypePack. + template + bool is(TID ty) const + { + // We do not use getMutable here because this method can be called on + // BoundTypeVars, which triggers an assertion. + auto* pendingTy = pending(ty); + if (pendingTy) + return Luau::get_if(&pendingTy->pending.ty) != nullptr; + + return Luau::get_if(&ty->ty) != nullptr; + } + +private: + // unique_ptr is used to give us stable pointers across insertions into the + // map. Otherwise, it would be really easy to accidentally invalidate the + // pointers returned from queue/pending. + // + // We can't use a DenseHashMap here because we need a non-const iterator + // over the map when we concatenate. + std::unordered_map> typeVarChanges; + std::unordered_map> typePackChanges; + + TxnLog* parent = nullptr; + + // Owned version of sharedSeen. This should not be accessed directly in + // TxnLogs; use sharedSeen instead. This field exists because in the tree + // of TxnLogs, the root must own its seen set. In all descendant TxnLogs, + // this is an empty vector. + std::vector> ownedSeen; + +public: + // Used to avoid infinite recursion when types are cyclic. + // Shared with all the descendent TxnLogs. + std::vector>* sharedSeen; +}; + } // namespace Luau diff --git a/Analysis/include/Luau/TypeInfer.h b/Analysis/include/Luau/TypeInfer.h index 862f50d7..312283b0 100644 --- a/Analysis/include/Luau/TypeInfer.h +++ b/Analysis/include/Luau/TypeInfer.h @@ -198,32 +198,32 @@ struct TypeChecker */ TypeId anyIfNonstrict(TypeId ty) const; - /** Attempt to unify the types left and right. Treat any failures as type errors - * in the final typecheck report. + /** Attempt to unify the types. + * Treat any failures as type errors in the final typecheck report. */ - bool unify(TypeId left, TypeId right, const Location& location); - bool unify(TypePackId left, TypePackId right, const Location& location, CountMismatch::Context ctx = CountMismatch::Context::Arg); + bool unify(TypeId subTy, TypeId superTy, const Location& location); + bool unify(TypePackId subTy, TypePackId superTy, const Location& location, CountMismatch::Context ctx = CountMismatch::Context::Arg); - /** Attempt to unify the types left and right. - * If this fails, and the right type can be instantiated, do so and try unification again. + /** Attempt to unify the types. + * If this fails, and the subTy type can be instantiated, do so and try unification again. */ - bool unifyWithInstantiationIfNeeded(const ScopePtr& scope, TypeId left, TypeId right, const Location& location); - void unifyWithInstantiationIfNeeded(const ScopePtr& scope, TypeId left, TypeId right, Unifier& state); + bool unifyWithInstantiationIfNeeded(const ScopePtr& scope, TypeId subTy, TypeId superTy, const Location& location); + void unifyWithInstantiationIfNeeded(const ScopePtr& scope, TypeId subTy, TypeId superTy, Unifier& state); - /** Attempt to unify left with right. + /** Attempt to unify. * If there are errors, undo everything and return the errors. * If there are no errors, commit and return an empty error vector. */ - ErrorVec tryUnify(TypeId left, TypeId right, const Location& location); - ErrorVec tryUnify(TypePackId left, TypePackId right, const Location& location); + template + ErrorVec tryUnify_(Id subTy, Id superTy, const Location& location); + ErrorVec tryUnify(TypeId subTy, TypeId superTy, const Location& location); + ErrorVec tryUnify(TypePackId subTy, TypePackId superTy, const Location& location); // Test whether the two type vars unify. Never commits the result. - ErrorVec canUnify(TypeId superTy, TypeId subTy, const Location& location); - ErrorVec canUnify(TypePackId superTy, TypePackId subTy, const Location& location); - - // Variant that takes a preexisting 'seen' set. We need this in certain cases to avoid infinitely recursing - // into cyclic types. - ErrorVec canUnify(const std::vector>& seen, TypeId left, TypeId right, const Location& location); + template + ErrorVec canUnify_(Id subTy, Id superTy, const Location& location); + ErrorVec canUnify(TypeId subTy, TypeId superTy, const Location& location); + ErrorVec canUnify(TypePackId subTy, TypePackId superTy, const Location& location); std::optional findMetatableEntry(TypeId type, std::string entry, const Location& location); std::optional findTablePropertyRespectingMeta(TypeId lhsType, Name name, const Location& location); @@ -237,12 +237,6 @@ struct TypeChecker std::optional tryStripUnionFromNil(TypeId ty); TypeId stripFromNilAndReport(TypeId ty, const Location& location); - template - ErrorVec tryUnify_(Id left, Id right, const Location& location); - - template - ErrorVec canUnify_(Id left, Id right, const Location& location); - public: /* * Convert monotype into a a polytype, by replacing any metavariables in descendant scopes diff --git a/Analysis/include/Luau/TypePack.h b/Analysis/include/Luau/TypePack.h index e72808da..ca588ccb 100644 --- a/Analysis/include/Luau/TypePack.h +++ b/Analysis/include/Luau/TypePack.h @@ -18,6 +18,8 @@ struct VariadicTypePack; struct TypePackVar; +struct TxnLog; + using TypePackId = const TypePackVar*; using FreeTypePack = Unifiable::Free; using BoundTypePack = Unifiable::Bound; @@ -84,6 +86,7 @@ struct TypePackIterator TypePackIterator() = default; explicit TypePackIterator(TypePackId tp); + TypePackIterator(TypePackId tp, const TxnLog* log); TypePackIterator& operator++(); TypePackIterator operator++(int); @@ -104,9 +107,13 @@ private: TypePackId currentTypePack = nullptr; const TypePack* tp = nullptr; size_t currentIndex = 0; + + // Only used if LuauUseCommittingTxnLog is true. + const TxnLog* log; }; TypePackIterator begin(TypePackId tp); +TypePackIterator begin(TypePackId tp, TxnLog* log); TypePackIterator end(TypePackId tp); using SeenSet = std::set>; @@ -114,6 +121,7 @@ using SeenSet = std::set>; bool areEqual(SeenSet& seen, const TypePackVar& lhs, const TypePackVar& rhs); TypePackId follow(TypePackId tp); +TypePackId follow(TypePackId tp, std::function mapper); size_t size(TypePackId tp); bool finite(TypePackId tp); diff --git a/Analysis/include/Luau/TypeVar.h b/Analysis/include/Luau/TypeVar.h index f6829ec3..d6e17714 100644 --- a/Analysis/include/Luau/TypeVar.h +++ b/Analysis/include/Luau/TypeVar.h @@ -453,6 +453,7 @@ bool areEqual(SeenSet& seen, const TypeVar& lhs, const TypeVar& rhs); // Follow BoundTypeVars until we get to something real TypeId follow(TypeId t); +TypeId follow(TypeId t, std::function mapper); std::vector flattenIntersection(TypeId ty); diff --git a/Analysis/include/Luau/TypedAllocator.h b/Analysis/include/Luau/TypedAllocator.h index 0ded1489..64227e7c 100644 --- a/Analysis/include/Luau/TypedAllocator.h +++ b/Analysis/include/Luau/TypedAllocator.h @@ -6,6 +6,8 @@ #include #include +LUAU_FASTFLAG(LuauTypedAllocatorZeroStart) + namespace Luau { @@ -20,7 +22,10 @@ class TypedAllocator public: TypedAllocator() { - appendBlock(); + if (FFlag::LuauTypedAllocatorZeroStart) + currentBlockSize = kBlockSize; + else + appendBlock(); } ~TypedAllocator() @@ -59,12 +64,18 @@ public: bool empty() const { - return stuff.size() == 1 && currentBlockSize == 0; + if (FFlag::LuauTypedAllocatorZeroStart) + return stuff.empty(); + else + return stuff.size() == 1 && currentBlockSize == 0; } size_t size() const { - return kBlockSize * (stuff.size() - 1) + currentBlockSize; + if (FFlag::LuauTypedAllocatorZeroStart) + return stuff.empty() ? 0 : kBlockSize * (stuff.size() - 1) + currentBlockSize; + else + return kBlockSize * (stuff.size() - 1) + currentBlockSize; } void clear() @@ -72,7 +83,11 @@ public: if (frozen) unfreeze(); free(); - appendBlock(); + + if (FFlag::LuauTypedAllocatorZeroStart) + currentBlockSize = kBlockSize; + else + appendBlock(); } void freeze() diff --git a/Analysis/include/Luau/Unifier.h b/Analysis/include/Luau/Unifier.h index 7681b966..a3be739a 100644 --- a/Analysis/include/Luau/Unifier.h +++ b/Analysis/include/Luau/Unifier.h @@ -25,6 +25,7 @@ struct Unifier Mode mode; ScopePtr globalScope; // sigh. Needed solely to get at string's metatable. + DEPRECATED_TxnLog DEPRECATED_log; TxnLog log; ErrorVec errors; Location location; @@ -33,44 +34,45 @@ struct Unifier UnifierSharedState& sharedState; - Unifier(TypeArena* types, Mode mode, ScopePtr globalScope, const Location& location, Variance variance, UnifierSharedState& sharedState); + Unifier(TypeArena* types, Mode mode, ScopePtr globalScope, const Location& location, Variance variance, UnifierSharedState& sharedState, + TxnLog* parentLog = nullptr); Unifier(TypeArena* types, Mode mode, ScopePtr globalScope, std::vector>* sharedSeen, const Location& location, - Variance variance, UnifierSharedState& sharedState); + Variance variance, UnifierSharedState& sharedState, TxnLog* parentLog = nullptr); // Test whether the two type vars unify. Never commits the result. - ErrorVec canUnify(TypeId superTy, TypeId subTy); - ErrorVec canUnify(TypePackId superTy, TypePackId subTy, bool isFunctionCall = false); + ErrorVec canUnify(TypeId subTy, TypeId superTy); + ErrorVec canUnify(TypePackId subTy, TypePackId superTy, bool isFunctionCall = false); - /** Attempt to unify left with right. + /** Attempt to unify. * Populate the vector errors with any type errors that may arise. * Populate the transaction log with the set of TypeIds that need to be reset to undo the unification attempt. */ - void tryUnify(TypeId superTy, TypeId subTy, bool isFunctionCall = false, bool isIntersection = false); + void tryUnify(TypeId subTy, TypeId superTy, bool isFunctionCall = false, bool isIntersection = false); private: - void tryUnify_(TypeId superTy, TypeId subTy, bool isFunctionCall = false, bool isIntersection = false); - void tryUnifyPrimitives(TypeId superTy, TypeId subTy); - void tryUnifySingletons(TypeId superTy, TypeId subTy); - void tryUnifyFunctions(TypeId superTy, TypeId subTy, bool isFunctionCall = false); - void tryUnifyTables(TypeId left, TypeId right, bool isIntersection = false); - void DEPRECATED_tryUnifyTables(TypeId left, TypeId right, bool isIntersection = false); - void tryUnifyFreeTable(TypeId free, TypeId other); - void tryUnifySealedTables(TypeId left, TypeId right, bool isIntersection); - void tryUnifyWithMetatable(TypeId metatable, TypeId other, bool reversed); - void tryUnifyWithClass(TypeId superTy, TypeId subTy, bool reversed); - void tryUnify(const TableIndexer& superIndexer, const TableIndexer& subIndexer); + void tryUnify_(TypeId subTy, TypeId superTy, bool isFunctionCall = false, bool isIntersection = false); + void tryUnifyPrimitives(TypeId subTy, TypeId superTy); + void tryUnifySingletons(TypeId subTy, TypeId superTy); + void tryUnifyFunctions(TypeId subTy, TypeId superTy, bool isFunctionCall = false); + void tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection = false); + void DEPRECATED_tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection = false); + void tryUnifyFreeTable(TypeId subTy, TypeId superTy); + void tryUnifySealedTables(TypeId subTy, TypeId superTy, bool isIntersection); + void tryUnifyWithMetatable(TypeId subTy, TypeId superTy, bool reversed); + void tryUnifyWithClass(TypeId subTy, TypeId superTy, bool reversed); + void tryUnifyIndexer(const TableIndexer& subIndexer, const TableIndexer& superIndexer); TypeId deeplyOptional(TypeId ty, std::unordered_map seen = {}); - void cacheResult(TypeId superTy, TypeId subTy); + void cacheResult(TypeId subTy, TypeId superTy); public: - void tryUnify(TypePackId superTy, TypePackId subTy, bool isFunctionCall = false); + void tryUnify(TypePackId subTy, TypePackId superTy, bool isFunctionCall = false); private: - void tryUnify_(TypePackId superTy, TypePackId subTy, bool isFunctionCall = false); - void tryUnifyVariadics(TypePackId superTy, TypePackId subTy, bool reversed, int subOffset = 0); + void tryUnify_(TypePackId subTy, TypePackId superTy, bool isFunctionCall = false); + void tryUnifyVariadics(TypePackId subTy, TypePackId superTy, bool reversed, int subOffset = 0); - void tryUnifyWithAny(TypeId any, TypeId ty); - void tryUnifyWithAny(TypePackId any, TypePackId ty); + void tryUnifyWithAny(TypeId subTy, TypeId anyTy); + void tryUnifyWithAny(TypePackId subTy, TypePackId anyTp); std::optional findTablePropertyRespectingMeta(TypeId lhsType, Name name); diff --git a/Analysis/src/Autocomplete.cpp b/Analysis/src/Autocomplete.cpp index 4b583792..67ebd075 100644 --- a/Analysis/src/Autocomplete.cpp +++ b/Analysis/src/Autocomplete.cpp @@ -12,10 +12,12 @@ #include #include +LUAU_FASTFLAG(LuauUseCommittingTxnLog) LUAU_FASTFLAG(LuauIfElseExpressionAnalysisSupport) LUAU_FASTFLAGVARIABLE(LuauAutocompleteAvoidMutation, false); LUAU_FASTFLAGVARIABLE(LuauAutocompletePreferToCallFunctions, false); LUAU_FASTFLAGVARIABLE(LuauAutocompleteFirstArg, false); +LUAU_FASTFLAGVARIABLE(LuauCompleteBrokenStringParams, false); static const std::unordered_set kStatementStartingKeywords = { "while", "if", "local", "repeat", "function", "do", "for", "return", "break", "continue", "type", "export"}; @@ -236,28 +238,31 @@ static TypeCorrectKind checkTypeCorrectKind(const Module& module, TypeArena* typ { ty = follow(ty); - auto canUnify = [&typeArena, &module](TypeId expectedType, TypeId actualType) { + auto canUnify = [&typeArena, &module](TypeId subTy, TypeId superTy) { InternalErrorReporter iceReporter; UnifierSharedState unifierState(&iceReporter); Unifier unifier(typeArena, Mode::Strict, module.getModuleScope(), Location(), Variance::Covariant, unifierState); - if (FFlag::LuauAutocompleteAvoidMutation) + if (FFlag::LuauAutocompleteAvoidMutation && !FFlag::LuauUseCommittingTxnLog) { SeenTypes seenTypes; SeenTypePacks seenTypePacks; CloneState cloneState; - expectedType = clone(expectedType, *typeArena, seenTypes, seenTypePacks, cloneState); - actualType = clone(actualType, *typeArena, seenTypes, seenTypePacks, cloneState); + superTy = clone(superTy, *typeArena, seenTypes, seenTypePacks, cloneState); + subTy = clone(subTy, *typeArena, seenTypes, seenTypePacks, cloneState); - auto errors = unifier.canUnify(expectedType, actualType); + auto errors = unifier.canUnify(subTy, superTy); return errors.empty(); } else { - unifier.tryUnify(expectedType, actualType); + unifier.tryUnify(subTy, superTy); bool ok = unifier.errors.empty(); - unifier.log.rollback(); + + if (!FFlag::LuauUseCommittingTxnLog) + unifier.DEPRECATED_log.rollback(); + return ok; } }; @@ -293,22 +298,22 @@ static TypeCorrectKind checkTypeCorrectKind(const Module& module, TypeArena* typ { auto [retHead, retTail] = flatten(ftv->retType); - if (!retHead.empty() && canUnify(expectedType, retHead.front())) + if (!retHead.empty() && canUnify(retHead.front(), expectedType)) return TypeCorrectKind::CorrectFunctionResult; // We might only have a variadic tail pack, check if the element is compatible if (retTail) { - if (const VariadicTypePack* vtp = get(follow(*retTail)); vtp && canUnify(expectedType, vtp->ty)) + if (const VariadicTypePack* vtp = get(follow(*retTail)); vtp && canUnify(vtp->ty, expectedType)) return TypeCorrectKind::CorrectFunctionResult; } } - return canUnify(expectedType, ty) ? TypeCorrectKind::Correct : TypeCorrectKind::None; + return canUnify(ty, expectedType) ? TypeCorrectKind::Correct : TypeCorrectKind::None; } else { - if (canUnify(expectedType, ty)) + if (canUnify(ty, expectedType)) return TypeCorrectKind::Correct; // We also want to suggest functions that return compatible result @@ -320,13 +325,13 @@ static TypeCorrectKind checkTypeCorrectKind(const Module& module, TypeArena* typ auto [retHead, retTail] = flatten(ftv->retType); if (!retHead.empty()) - return canUnify(expectedType, retHead.front()) ? TypeCorrectKind::CorrectFunctionResult : TypeCorrectKind::None; + return canUnify(retHead.front(), expectedType) ? TypeCorrectKind::CorrectFunctionResult : TypeCorrectKind::None; // We might only have a variadic tail pack, check if the element is compatible if (retTail) { if (const VariadicTypePack* vtp = get(follow(*retTail))) - return canUnify(expectedType, vtp->ty) ? TypeCorrectKind::CorrectFunctionResult : TypeCorrectKind::None; + return canUnify(vtp->ty, expectedType) ? TypeCorrectKind::CorrectFunctionResult : TypeCorrectKind::None; } return TypeCorrectKind::None; @@ -1319,7 +1324,7 @@ static std::optional autocompleteStringParams(const Source return std::nullopt; } - if (!nodes.back()->is()) + if (!nodes.back()->is() && (!FFlag::LuauCompleteBrokenStringParams || !nodes.back()->is())) { return std::nullopt; } diff --git a/Analysis/src/EmbeddedBuiltinDefinitions.cpp b/Analysis/src/EmbeddedBuiltinDefinitions.cpp index d0afa742..24982506 100644 --- a/Analysis/src/EmbeddedBuiltinDefinitions.cpp +++ b/Analysis/src/EmbeddedBuiltinDefinitions.cpp @@ -138,12 +138,7 @@ declare function gcinfo(): number -- (nil, string). declare function loadstring(src: string, chunkname: string?): (((A...) -> any)?, string?) - -- a userdata object is "roughly" the same as a sealed empty table - -- except `type(newproxy(false))` evaluates to "userdata" so we may need another special type here too. - -- another important thing to note: the value passed in conditionally creates an empty metatable, and you have to use getmetatable, NOT - -- setmetatable. - -- FIXME: change this to something Luau can understand how to reject `setmetatable(newproxy(false or true), {})`. - declare function newproxy(mt: boolean?): {} + declare function newproxy(mt: boolean?): any declare coroutine: { create: ((A...) -> R...) -> thread, diff --git a/Analysis/src/Frontend.cpp b/Analysis/src/Frontend.cpp index e332f07d..fe4b6529 100644 --- a/Analysis/src/Frontend.cpp +++ b/Analysis/src/Frontend.cpp @@ -351,7 +351,7 @@ FrontendModuleResolver::FrontendModuleResolver(Frontend* frontend) { } -CheckResult Frontend::check(const ModuleName& name) +CheckResult Frontend::check(const ModuleName& name, std::optional optionOverride) { LUAU_TIMETRACE_SCOPE("Frontend::check", "Frontend"); LUAU_TIMETRACE_ARGUMENT("name", name.c_str()); @@ -372,6 +372,8 @@ CheckResult Frontend::check(const ModuleName& name) std::vector buildQueue; bool cycleDetected = parseGraph(buildQueue, checkResult, name); + FrontendOptions frontendOptions = optionOverride.value_or(options); + // Keep track of which AST nodes we've reported cycles in std::unordered_set reportedCycles; @@ -411,31 +413,11 @@ CheckResult Frontend::check(const ModuleName& name) // If we're typechecking twice, we do so. // The second typecheck is always in strict mode with DM awareness // to provide better typen information for IDE features. - if (options.typecheckTwice) + if (frontendOptions.typecheckTwice) { ModulePtr moduleForAutocomplete = typeCheckerForAutocomplete.check(sourceModule, Mode::Strict); moduleResolverForAutocomplete.modules[moduleName] = moduleForAutocomplete; } - else if (options.retainFullTypeGraphs && options.typecheckTwice && mode != Mode::Strict) - { - ModulePtr strictModule = typeChecker.check(sourceModule, Mode::Strict, environmentScope); - module->astTypes.clear(); - module->astOriginalCallTypes.clear(); - module->astExpectedTypes.clear(); - - SeenTypes seenTypes; - SeenTypePacks seenTypePacks; - CloneState cloneState; - - for (const auto& [expr, strictTy] : strictModule->astTypes) - module->astTypes[expr] = clone(strictTy, module->interfaceTypes, seenTypes, seenTypePacks, cloneState); - - for (const auto& [expr, strictTy] : strictModule->astOriginalCallTypes) - module->astOriginalCallTypes[expr] = clone(strictTy, module->interfaceTypes, seenTypes, seenTypePacks, cloneState); - - for (const auto& [expr, strictTy] : strictModule->astExpectedTypes) - module->astExpectedTypes[expr] = clone(strictTy, module->interfaceTypes, seenTypes, seenTypePacks, cloneState); - } stats.timeCheck += getTimestamp() - timestamp; stats.filesStrict += mode == Mode::Strict; @@ -444,7 +426,7 @@ CheckResult Frontend::check(const ModuleName& name) if (module == nullptr) throw std::runtime_error("Frontend::check produced a nullptr module for " + moduleName); - if (!options.retainFullTypeGraphs) + if (!frontendOptions.retainFullTypeGraphs) { // copyErrors needs to allocate into interfaceTypes as it copies // types out of internalTypes, so we unfreeze it here. diff --git a/Analysis/src/Module.cpp b/Analysis/src/Module.cpp index e1e53c97..cff85897 100644 --- a/Analysis/src/Module.cpp +++ b/Analysis/src/Module.cpp @@ -13,7 +13,7 @@ LUAU_FASTFLAGVARIABLE(DebugLuauFreezeArena, false) LUAU_FASTFLAGVARIABLE(DebugLuauTrackOwningArena, false) -LUAU_FASTINTVARIABLE(LuauTypeCloneRecursionLimit, 0) +LUAU_FASTINTVARIABLE(LuauTypeCloneRecursionLimit, 300) namespace Luau { diff --git a/Analysis/src/Quantify.cpp b/Analysis/src/Quantify.cpp index c773e208..04ebffc1 100644 --- a/Analysis/src/Quantify.cpp +++ b/Analysis/src/Quantify.cpp @@ -4,8 +4,6 @@ #include "Luau/VisitTypeVar.h" -LUAU_FASTFLAGVARIABLE(LuauQuantifyVisitOnce, false) - namespace Luau { @@ -81,16 +79,8 @@ struct Quantifier void quantify(ModulePtr module, TypeId ty, TypeLevel level) { Quantifier q{std::move(module), level}; - - if (FFlag::LuauQuantifyVisitOnce) - { - DenseHashSet seen{nullptr}; - visitTypeVarOnce(ty, q, seen); - } - else - { - visitTypeVar(ty, q); - } + DenseHashSet seen{nullptr}; + visitTypeVarOnce(ty, q, seen); FunctionTypeVar* ftv = getMutable(ty); LUAU_ASSERT(ftv); diff --git a/Analysis/src/ToString.cpp b/Analysis/src/ToString.cpp index a6be5348..889dd6dc 100644 --- a/Analysis/src/ToString.cpp +++ b/Analysis/src/ToString.cpp @@ -11,7 +11,6 @@ #include LUAU_FASTFLAG(LuauOccursCheckOkWithRecursiveFunctions) -LUAU_FASTFLAGVARIABLE(LuauFunctionArgumentNameSize, false) /* * Prefix generic typenames with gen- @@ -766,24 +765,12 @@ struct TypePackStringifier else state.emit(", "); - if (FFlag::LuauFunctionArgumentNameSize) + if (elemIndex < elemNames.size() && elemNames[elemIndex]) { - if (elemIndex < elemNames.size() && elemNames[elemIndex]) - { - state.emit(elemNames[elemIndex]->name); - state.emit(": "); - } + state.emit(elemNames[elemIndex]->name); + state.emit(": "); } - else - { - LUAU_ASSERT(elemNames.empty() || elemIndex < elemNames.size()); - if (!elemNames.empty() && elemNames[elemIndex]) - { - state.emit(elemNames[elemIndex]->name); - state.emit(": "); - } - } elemIndex++; stringify(typeId); @@ -1151,38 +1138,19 @@ std::string toStringNamedFunction(const std::string& prefix, const FunctionTypeV s += ", "; first = false; - if (FFlag::LuauFunctionArgumentNameSize) + // We don't currently respect opts.functionTypeArguments. I don't think this function should. + if (argNameIter != ftv.argNames.end()) { - // We don't currently respect opts.functionTypeArguments. I don't think this function should. - if (argNameIter != ftv.argNames.end()) - { - s += (*argNameIter ? (*argNameIter)->name : "_") + ": "; - ++argNameIter; - } - else - { - s += "_: "; - } + s += (*argNameIter ? (*argNameIter)->name : "_") + ": "; + ++argNameIter; } else { - // argNames is guaranteed to be equal to argTypes iff argNames is not empty. - // We don't currently respect opts.functionTypeArguments. I don't think this function should. - if (!ftv.argNames.empty()) - s += (*argNameIter ? (*argNameIter)->name : "_") + ": "; + s += "_: "; } s += toString_(*argPackIter); ++argPackIter; - - if (!FFlag::LuauFunctionArgumentNameSize) - { - if (!ftv.argNames.empty()) - { - LUAU_ASSERT(argNameIter != ftv.argNames.end()); - ++argNameIter; - } - } } if (argPackIter.tail()) diff --git a/Analysis/src/TxnLog.cpp b/Analysis/src/TxnLog.cpp index f6a61581..a46ac0c3 100644 --- a/Analysis/src/TxnLog.cpp +++ b/Analysis/src/TxnLog.cpp @@ -4,27 +4,34 @@ #include "Luau/TypePack.h" #include +#include + +LUAU_FASTFLAGVARIABLE(LuauUseCommittingTxnLog, false) namespace Luau { -void TxnLog::operator()(TypeId a) +void DEPRECATED_TxnLog::operator()(TypeId a) { + LUAU_ASSERT(!FFlag::LuauUseCommittingTxnLog); typeVarChanges.emplace_back(a, *a); } -void TxnLog::operator()(TypePackId a) +void DEPRECATED_TxnLog::operator()(TypePackId a) { + LUAU_ASSERT(!FFlag::LuauUseCommittingTxnLog); typePackChanges.emplace_back(a, *a); } -void TxnLog::operator()(TableTypeVar* a) +void DEPRECATED_TxnLog::operator()(TableTypeVar* a) { + LUAU_ASSERT(!FFlag::LuauUseCommittingTxnLog); tableChanges.emplace_back(a, a->boundTo); } -void TxnLog::rollback() +void DEPRECATED_TxnLog::rollback() { + LUAU_ASSERT(!FFlag::LuauUseCommittingTxnLog); for (auto it = typeVarChanges.rbegin(); it != typeVarChanges.rend(); ++it) std::swap(*asMutable(it->first), it->second); @@ -38,8 +45,9 @@ void TxnLog::rollback() sharedSeen->resize(originalSeenSize); } -void TxnLog::concat(TxnLog rhs) +void DEPRECATED_TxnLog::concat(DEPRECATED_TxnLog rhs) { + LUAU_ASSERT(!FFlag::LuauUseCommittingTxnLog); typeVarChanges.insert(typeVarChanges.end(), rhs.typeVarChanges.begin(), rhs.typeVarChanges.end()); rhs.typeVarChanges.clear(); @@ -50,23 +58,298 @@ void TxnLog::concat(TxnLog rhs) rhs.tableChanges.clear(); } -bool TxnLog::haveSeen(TypeId lhs, TypeId rhs) +bool DEPRECATED_TxnLog::haveSeen(TypeId lhs, TypeId rhs) { + LUAU_ASSERT(!FFlag::LuauUseCommittingTxnLog); const std::pair sortedPair = (lhs > rhs) ? std::make_pair(lhs, rhs) : std::make_pair(rhs, lhs); return (sharedSeen->end() != std::find(sharedSeen->begin(), sharedSeen->end(), sortedPair)); } +void DEPRECATED_TxnLog::pushSeen(TypeId lhs, TypeId rhs) +{ + LUAU_ASSERT(!FFlag::LuauUseCommittingTxnLog); + const std::pair sortedPair = (lhs > rhs) ? std::make_pair(lhs, rhs) : std::make_pair(rhs, lhs); + sharedSeen->push_back(sortedPair); +} + +void DEPRECATED_TxnLog::popSeen(TypeId lhs, TypeId rhs) +{ + LUAU_ASSERT(!FFlag::LuauUseCommittingTxnLog); + const std::pair sortedPair = (lhs > rhs) ? std::make_pair(lhs, rhs) : std::make_pair(rhs, lhs); + LUAU_ASSERT(sortedPair == sharedSeen->back()); + sharedSeen->pop_back(); +} + +static const TxnLog emptyLog; + +const TxnLog* TxnLog::empty() +{ + return &emptyLog; +} + +void TxnLog::concat(TxnLog rhs) +{ + LUAU_ASSERT(FFlag::LuauUseCommittingTxnLog); + + for (auto& [ty, rep] : rhs.typeVarChanges) + typeVarChanges[ty] = std::move(rep); + + for (auto& [tp, rep] : rhs.typePackChanges) + typePackChanges[tp] = std::move(rep); +} + +void TxnLog::commit() +{ + LUAU_ASSERT(FFlag::LuauUseCommittingTxnLog); + + for (auto& [ty, rep] : typeVarChanges) + *asMutable(ty) = rep.get()->pending; + + for (auto& [tp, rep] : typePackChanges) + *asMutable(tp) = rep.get()->pending; + + clear(); +} + +void TxnLog::clear() +{ + LUAU_ASSERT(FFlag::LuauUseCommittingTxnLog); + + typeVarChanges.clear(); + typePackChanges.clear(); +} + +TxnLog TxnLog::inverse() +{ + LUAU_ASSERT(FFlag::LuauUseCommittingTxnLog); + + TxnLog inversed(sharedSeen); + + for (auto& [ty, _rep] : typeVarChanges) + inversed.typeVarChanges[ty] = std::make_unique(*ty); + + for (auto& [tp, _rep] : typePackChanges) + inversed.typePackChanges[tp] = std::make_unique(*tp); + + return inversed; +} + +bool TxnLog::haveSeen(TypeId lhs, TypeId rhs) const +{ + LUAU_ASSERT(FFlag::LuauUseCommittingTxnLog); + + const std::pair sortedPair = (lhs > rhs) ? std::make_pair(lhs, rhs) : std::make_pair(rhs, lhs); + if (sharedSeen->end() != std::find(sharedSeen->begin(), sharedSeen->end(), sortedPair)) + { + return true; + } + + if (parent) + { + return parent->haveSeen(lhs, rhs); + } + + return false; +} + void TxnLog::pushSeen(TypeId lhs, TypeId rhs) { + LUAU_ASSERT(FFlag::LuauUseCommittingTxnLog); + const std::pair sortedPair = (lhs > rhs) ? std::make_pair(lhs, rhs) : std::make_pair(rhs, lhs); sharedSeen->push_back(sortedPair); } void TxnLog::popSeen(TypeId lhs, TypeId rhs) { + LUAU_ASSERT(FFlag::LuauUseCommittingTxnLog); + const std::pair sortedPair = (lhs > rhs) ? std::make_pair(lhs, rhs) : std::make_pair(rhs, lhs); LUAU_ASSERT(sortedPair == sharedSeen->back()); sharedSeen->pop_back(); } +PendingType* TxnLog::queue(TypeId ty) +{ + LUAU_ASSERT(FFlag::LuauUseCommittingTxnLog); + LUAU_ASSERT(!ty->persistent); + + // Explicitly don't look in ancestors. If we have discovered something new + // about this type, we don't want to mutate the parent's state. + auto& pending = typeVarChanges[ty]; + if (!pending) + pending = std::make_unique(*ty); + + return pending.get(); +} + +PendingTypePack* TxnLog::queue(TypePackId tp) +{ + LUAU_ASSERT(FFlag::LuauUseCommittingTxnLog); + LUAU_ASSERT(!tp->persistent); + + // Explicitly don't look in ancestors. If we have discovered something new + // about this type, we don't want to mutate the parent's state. + auto& pending = typePackChanges[tp]; + if (!pending) + pending = std::make_unique(*tp); + + return pending.get(); +} + +PendingType* TxnLog::pending(TypeId ty) const +{ + LUAU_ASSERT(FFlag::LuauUseCommittingTxnLog); + + for (const TxnLog* current = this; current; current = current->parent) + { + if (auto it = current->typeVarChanges.find(ty); it != current->typeVarChanges.end()) + return it->second.get(); + } + + return nullptr; +} + +PendingTypePack* TxnLog::pending(TypePackId tp) const +{ + LUAU_ASSERT(FFlag::LuauUseCommittingTxnLog); + + for (const TxnLog* current = this; current; current = current->parent) + { + if (auto it = current->typePackChanges.find(tp); it != current->typePackChanges.end()) + return it->second.get(); + } + + return nullptr; +} + +PendingType* TxnLog::replace(TypeId ty, TypeVar replacement) +{ + LUAU_ASSERT(FFlag::LuauUseCommittingTxnLog); + + PendingType* newTy = queue(ty); + newTy->pending = replacement; + return newTy; +} + +PendingTypePack* TxnLog::replace(TypePackId tp, TypePackVar replacement) +{ + LUAU_ASSERT(FFlag::LuauUseCommittingTxnLog); + + PendingTypePack* newTp = queue(tp); + newTp->pending = replacement; + return newTp; +} + +PendingType* TxnLog::bindTable(TypeId ty, std::optional newBoundTo) +{ + LUAU_ASSERT(FFlag::LuauUseCommittingTxnLog); + LUAU_ASSERT(get(ty)); + + PendingType* newTy = queue(ty); + if (TableTypeVar* ttv = Luau::getMutable(newTy)) + ttv->boundTo = newBoundTo; + + return newTy; +} + +PendingType* TxnLog::changeLevel(TypeId ty, TypeLevel newLevel) +{ + LUAU_ASSERT(FFlag::LuauUseCommittingTxnLog); + LUAU_ASSERT(get(ty) || get(ty) || get(ty)); + + PendingType* newTy = queue(ty); + if (FreeTypeVar* ftv = Luau::getMutable(newTy)) + { + ftv->level = newLevel; + } + else if (TableTypeVar* ttv = Luau::getMutable(newTy)) + { + LUAU_ASSERT(ttv->state == TableState::Free || ttv->state == TableState::Generic); + ttv->level = newLevel; + } + else if (FunctionTypeVar* ftv = Luau::getMutable(newTy)) + { + ftv->level = newLevel; + } + + return newTy; +} + +PendingTypePack* TxnLog::changeLevel(TypePackId tp, TypeLevel newLevel) +{ + LUAU_ASSERT(FFlag::LuauUseCommittingTxnLog); + LUAU_ASSERT(get(tp)); + + PendingTypePack* newTp = queue(tp); + if (FreeTypePack* ftp = Luau::getMutable(newTp)) + { + ftp->level = newLevel; + } + + return newTp; +} + +PendingType* TxnLog::changeIndexer(TypeId ty, std::optional indexer) +{ + LUAU_ASSERT(FFlag::LuauUseCommittingTxnLog); + LUAU_ASSERT(get(ty)); + + PendingType* newTy = queue(ty); + if (TableTypeVar* ttv = Luau::getMutable(newTy)) + { + ttv->indexer = indexer; + } + + return newTy; +} + +std::optional TxnLog::getLevel(TypeId ty) const +{ + LUAU_ASSERT(FFlag::LuauUseCommittingTxnLog); + + if (FreeTypeVar* ftv = getMutable(ty)) + return ftv->level; + else if (TableTypeVar* ttv = getMutable(ty); ttv && (ttv->state == TableState::Free || ttv->state == TableState::Generic)) + return ttv->level; + else if (FunctionTypeVar* ftv = getMutable(ty)) + return ftv->level; + + return std::nullopt; +} + +TypeId TxnLog::follow(TypeId ty) +{ + LUAU_ASSERT(FFlag::LuauUseCommittingTxnLog); + + return Luau::follow(ty, [this](TypeId ty) { + PendingType* state = this->pending(ty); + + if (state == nullptr) + return ty; + + // Ugly: Fabricate a TypeId that doesn't adhere to most of the invariants + // that normally apply. This is safe because follow will only call get<> + // on the returned pointer. + return const_cast(&state->pending); + }); +} + +TypePackId TxnLog::follow(TypePackId tp) const +{ + LUAU_ASSERT(FFlag::LuauUseCommittingTxnLog); + + return Luau::follow(tp, [this](TypePackId tp) { + PendingTypePack* state = this->pending(tp); + + if (state == nullptr) + return tp; + + // Ugly: Fabricate a TypePackId that doesn't adhere to most of the + // invariants that normally apply. This is safe because follow will + // only call get<> on the returned pointer. + return const_cast(&state->pending); + }); +} + } // namespace Luau diff --git a/Analysis/src/TypeInfer.cpp b/Analysis/src/TypeInfer.cpp index e29b6ec6..1689a5c3 100644 --- a/Analysis/src/TypeInfer.cpp +++ b/Analysis/src/TypeInfer.cpp @@ -27,15 +27,16 @@ LUAU_FASTFLAGVARIABLE(LuauEqConstraint, false) LUAU_FASTFLAGVARIABLE(LuauWeakEqConstraint, false) // Eventually removed as false. LUAU_FASTFLAGVARIABLE(LuauCloneCorrectlyBeforeMutatingTableType, false) LUAU_FASTFLAGVARIABLE(LuauStoreMatchingOverloadFnType, false) +LUAU_FASTFLAG(LuauUseCommittingTxnLog) +LUAU_FASTFLAGVARIABLE(DebugLuauFreezeDuringUnification, false) LUAU_FASTFLAGVARIABLE(LuauRecursiveTypeParameterRestriction, false) LUAU_FASTFLAGVARIABLE(LuauIfElseExpressionAnalysisSupport, false) LUAU_FASTFLAGVARIABLE(LuauQuantifyInPlace2, false) +LUAU_FASTFLAGVARIABLE(LuauSealExports, false) LUAU_FASTFLAGVARIABLE(LuauSingletonTypes, false) LUAU_FASTFLAGVARIABLE(LuauExpectedTypesOfProperties, false) LUAU_FASTFLAGVARIABLE(LuauErrorRecoveryType, false) LUAU_FASTFLAGVARIABLE(LuauPropertiesGetExpectedType, false) -LUAU_FASTFLAGVARIABLE(LuauTailArgumentTypeInfo, false) -LUAU_FASTFLAGVARIABLE(LuauModuleRequireErrorPack, false) LUAU_FASTFLAGVARIABLE(LuauLValueAsKey, false) LUAU_FASTFLAGVARIABLE(LuauRefiLookupFromIndexExpr, false) LUAU_FASTFLAGVARIABLE(LuauProperTypeLevels, false) @@ -450,7 +451,7 @@ void TypeChecker::checkBlock(const ScopePtr& scope, const AstStatBlock& block) ++subLevel; TypeId leftType = checkFunctionName(scope, *fun->name, funScope->level); - unify(leftType, funTy, fun->location); + unify(funTy, leftType, fun->location); } else if (auto fun = (*protoIter)->as()) { @@ -556,21 +557,21 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatIf& statement) } } -ErrorVec TypeChecker::canUnify(TypeId left, TypeId right, const Location& location) -{ - return canUnify_(left, right, location); -} - -ErrorVec TypeChecker::canUnify(TypePackId left, TypePackId right, const Location& location) -{ - return canUnify_(left, right, location); -} - template -ErrorVec TypeChecker::canUnify_(Id superTy, Id subTy, const Location& location) +ErrorVec TypeChecker::canUnify_(Id subTy, Id superTy, const Location& location) { Unifier state = mkUnifier(location); - return state.canUnify(superTy, subTy); + return state.canUnify(subTy, superTy); +} + +ErrorVec TypeChecker::canUnify(TypeId subTy, TypeId superTy, const Location& location) +{ + return canUnify_(subTy, superTy, location); +} + +ErrorVec TypeChecker::canUnify(TypePackId subTy, TypePackId superTy, const Location& location) +{ + return canUnify_(subTy, superTy, location); } void TypeChecker::check(const ScopePtr& scope, const AstStatWhile& statement) @@ -619,7 +620,7 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatReturn& return_) // start typechecking everything across module boundaries. if (isNonstrictMode() && follow(scope->returnType) == follow(currentModule->getModuleScope()->returnType)) { - ErrorVec errors = tryUnify(scope->returnType, retPack, return_.location); + ErrorVec errors = tryUnify(retPack, scope->returnType, return_.location); if (!errors.empty()) currentModule->getModuleScope()->returnType = addTypePack({anyType}); @@ -627,31 +628,41 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatReturn& return_) return; } - unify(scope->returnType, retPack, return_.location, CountMismatch::Context::Return); -} - -ErrorVec TypeChecker::tryUnify(TypeId left, TypeId right, const Location& location) -{ - return tryUnify_(left, right, location); -} - -ErrorVec TypeChecker::tryUnify(TypePackId left, TypePackId right, const Location& location) -{ - return tryUnify_(left, right, location); + unify(retPack, scope->returnType, return_.location, CountMismatch::Context::Return); } template -ErrorVec TypeChecker::tryUnify_(Id left, Id right, const Location& location) +ErrorVec TypeChecker::tryUnify_(Id subTy, Id superTy, const Location& location) { Unifier state = mkUnifier(location); - state.tryUnify(left, right); - if (!state.errors.empty()) - state.log.rollback(); + if (FFlag::LuauUseCommittingTxnLog && FFlag::DebugLuauFreezeDuringUnification) + freeze(currentModule->internalTypes); + + state.tryUnify(subTy, superTy); + + if (FFlag::LuauUseCommittingTxnLog && FFlag::DebugLuauFreezeDuringUnification) + unfreeze(currentModule->internalTypes); + + if (!state.errors.empty() && !FFlag::LuauUseCommittingTxnLog) + state.DEPRECATED_log.rollback(); + + if (state.errors.empty() && FFlag::LuauUseCommittingTxnLog) + state.log.commit(); return state.errors; } +ErrorVec TypeChecker::tryUnify(TypeId subTy, TypeId superTy, const Location& location) +{ + return tryUnify_(subTy, superTy, location); +} + +ErrorVec TypeChecker::tryUnify(TypePackId subTy, TypePackId superTy, const Location& location) +{ + return tryUnify_(subTy, superTy, location); +} + void TypeChecker::check(const ScopePtr& scope, const AstStatAssign& assign) { std::vector> expectedTypes; @@ -743,9 +754,9 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatAssign& assign) { // In nonstrict mode, any assignments where the lhs is free and rhs isn't a function, we give it any typevar. if (isNonstrictMode() && get(follow(left)) && !get(follow(right))) - unify(left, anyType, loc); + unify(anyType, left, loc); else - unify(left, right, loc); + unify(right, left, loc); } } } @@ -760,7 +771,7 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatCompoundAssign& assi TypeId result = checkBinaryOperation(scope, expr, left, right); - unify(left, result, assign.location); + unify(result, left, assign.location); } void TypeChecker::check(const ScopePtr& scope, const AstStatLocal& local) @@ -817,9 +828,12 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatLocal& local) Unifier state = mkUnifier(local.location); state.ctx = CountMismatch::Result; - state.tryUnify(variablePack, valuePack); + state.tryUnify(valuePack, variablePack); reportErrors(state.errors); + if (FFlag::LuauUseCommittingTxnLog) + state.log.commit(); + // In the code 'local T = {}', we wish to ascribe the name 'T' to the type of the table for error-reporting purposes. // We also want to do this for 'local T = setmetatable(...)'. if (local.vars.size == 1 && local.values.size == 1) @@ -889,7 +903,7 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatFor& expr) TypeId loopVarType = numberType; if (expr.var->annotation) - unify(resolveType(scope, *expr.var->annotation), loopVarType, expr.location); + unify(loopVarType, resolveType(scope, *expr.var->annotation), expr.location); loopScope->bindings[expr.var] = {loopVarType, expr.var->location}; @@ -899,11 +913,11 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatFor& expr) if (!expr.to) ice("Bad AstStatFor has no to expr"); - unify(loopVarType, checkExpr(loopScope, *expr.from).type, expr.from->location); - unify(loopVarType, checkExpr(loopScope, *expr.to).type, expr.to->location); + unify(checkExpr(loopScope, *expr.from).type, loopVarType, expr.from->location); + unify(checkExpr(loopScope, *expr.to).type, loopVarType, expr.to->location); if (expr.step) - unify(loopVarType, checkExpr(loopScope, *expr.step).type, expr.step->location); + unify(checkExpr(loopScope, *expr.step).type, loopVarType, expr.step->location); check(loopScope, *expr.body); } @@ -956,12 +970,12 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatForIn& forin) if (get(callRetPack)) { iterTy = freshType(scope); - unify(addTypePack({{iterTy}, freshTypePack(scope)}), callRetPack, forin.location); + unify(callRetPack, addTypePack({{iterTy}, freshTypePack(scope)}), forin.location); } else if (get(callRetPack) || !first(callRetPack)) { for (TypeId var : varTypes) - unify(var, errorRecoveryType(scope), forin.location); + unify(errorRecoveryType(scope), var, forin.location); return check(loopScope, *forin.body); } @@ -982,7 +996,7 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatForIn& forin) TypeId varTy = get(iterTy) ? anyType : errorRecoveryType(loopScope); for (TypeId var : varTypes) - unify(var, varTy, forin.location); + unify(varTy, var, forin.location); if (!get(iterTy) && !get(iterTy) && !get(iterTy)) reportError(TypeError{firstValue->location, TypeMismatch{globalScope->bindings[AstName{"next"}].typeId, iterTy}}); @@ -1010,6 +1024,9 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatForIn& forin) Unifier state = mkUnifier(firstValue->location); checkArgumentList(loopScope, state, argPack, iterFunc->argTypes, /*argLocations*/ {}); + if (FFlag::LuauUseCommittingTxnLog) + state.log.commit(); + reportErrors(state.errors); } @@ -1024,10 +1041,10 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatForIn& forin) AstExprCall exprCall{Location(start, end), firstValue, arguments, /* self= */ false, Location()}; TypePackId retPack = checkExprPack(scope, exprCall).type; - unify(varPack, retPack, forin.location); + unify(retPack, varPack, forin.location); } else - unify(varPack, iterFunc->retType, forin.location); + unify(iterFunc->retType, varPack, forin.location); check(loopScope, *forin.body); } @@ -1112,7 +1129,7 @@ void TypeChecker::check(const ScopePtr& scope, TypeId ty, const ScopePtr& funSco checkFunctionBody(funScope, ty, *function.func); - unify(leftType, ty, function.location); + unify(ty, leftType, function.location); if (FFlag::LuauUpdateFunctionNameBinding) { @@ -1242,7 +1259,7 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatTypeAlias& typealias else if (auto mtv = getMutable(follow(ty))) mtv->syntheticName = name; - unify(bindingsMap[name].type, ty, typealias.location); + unify(ty, bindingsMap[name].type, typealias.location); } } @@ -1526,7 +1543,7 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprCa { TypeId head = freshType(scope); TypePackId pack = addTypePack(TypePackVar{TypePack{{head}, freshTypePack(scope)}}); - unify(retPack, pack, expr.location); + unify(pack, retPack, expr.location); return {head, std::move(result.predicates)}; } if (get(retPack)) @@ -1598,7 +1615,7 @@ std::optional TypeChecker::getIndexTypeFromType( return it->second.type; else if (auto indexer = tableType->indexer) { - tryUnify(indexer->indexType, stringType, location); + tryUnify(stringType, indexer->indexType, location); return indexer->indexResultType; } else if (tableType->state == TableState::Free) @@ -1824,7 +1841,7 @@ TypeId TypeChecker::checkExprTable( indexer = expectedTable->indexer; if (indexer) - unify(indexer->indexResultType, valueType, value->location); + unify(valueType, indexer->indexResultType, value->location); else indexer = TableIndexer{numberType, anyIfNonstrict(valueType)}; } @@ -1842,13 +1859,13 @@ TypeId TypeChecker::checkExprTable( if (it != expectedTable->props.end()) { Property expectedProp = it->second; - ErrorVec errors = tryUnify(expectedProp.type, exprType, k->location); + ErrorVec errors = tryUnify(exprType, expectedProp.type, k->location); if (errors.empty()) exprType = expectedProp.type; } else if (expectedTable->indexer && isString(expectedTable->indexer->indexType)) { - ErrorVec errors = tryUnify(expectedTable->indexer->indexResultType, exprType, k->location); + ErrorVec errors = tryUnify(exprType, expectedTable->indexer->indexResultType, k->location); if (errors.empty()) exprType = expectedTable->indexer->indexResultType; } @@ -1863,8 +1880,8 @@ TypeId TypeChecker::checkExprTable( if (indexer) { - unify(indexer->indexType, keyType, k->location); - unify(indexer->indexResultType, valueType, value->location); + unify(keyType, indexer->indexType, k->location); + unify(valueType, indexer->indexResultType, value->location); } else if (isNonstrictMode()) { @@ -1992,7 +2009,10 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprUn TypeId expectedFunctionType = addType(FunctionTypeVar(scope->level, arguments, retTypePack)); Unifier state = mkUnifier(expr.location); - state.tryUnify(expectedFunctionType, actualFunctionType, /*isFunctionCall*/ true); + state.tryUnify(actualFunctionType, expectedFunctionType, /*isFunctionCall*/ true); + + if (FFlag::LuauUseCommittingTxnLog) + state.log.commit(); TypeId retType = first(retTypePack).value_or(nilType); if (!state.errors.empty()) @@ -2006,7 +2026,7 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprUn return {errorRecoveryType(scope)}; } - reportErrors(tryUnify(numberType, operandType, expr.location)); + reportErrors(tryUnify(operandType, numberType, expr.location)); return {numberType}; } case AstExprUnary::Len: @@ -2072,7 +2092,7 @@ TypeId TypeChecker::unionOfTypes(TypeId a, TypeId b, const Location& location, b { if (unifyFreeTypes && (get(a) || get(b))) { - if (unify(a, b, location)) + if (unify(b, a, location)) return a; return errorRecoveryType(anyType); @@ -2175,7 +2195,13 @@ TypeId TypeChecker::checkRelationalOperation( */ Unifier state = mkUnifier(expr.location); if (!isEquality) - state.tryUnify(lhsType, rhsType); + { + state.tryUnify(rhsType, lhsType); + + if (FFlag::LuauUseCommittingTxnLog) + state.log.commit(); + } + bool needsMetamethod = !isEquality; @@ -2216,13 +2242,16 @@ TypeId TypeChecker::checkRelationalOperation( if (isEquality) { Unifier state = mkUnifier(expr.location); - state.tryUnify(ftv->retType, addTypePack({booleanType})); + state.tryUnify(addTypePack({booleanType}), ftv->retType); if (!state.errors.empty()) { reportError(expr.location, GenericError{format("Metamethod '%s' must return type 'boolean'", metamethodName.c_str())}); return errorRecoveryType(booleanType); } + + if (FFlag::LuauUseCommittingTxnLog) + state.log.commit(); } } @@ -2230,7 +2259,10 @@ TypeId TypeChecker::checkRelationalOperation( TypeId actualFunctionType = addType(FunctionTypeVar(scope->level, addTypePack({lhsType, rhsType}), addTypePack({booleanType}))); state.tryUnify( - instantiate(scope, *metamethod, expr.location), instantiate(scope, actualFunctionType, expr.location), /*isFunctionCall*/ true); + instantiate(scope, actualFunctionType, expr.location), instantiate(scope, *metamethod, expr.location), /*isFunctionCall*/ true); + + if (FFlag::LuauUseCommittingTxnLog) + state.log.commit(); reportErrors(state.errors); return booleanType; @@ -2323,7 +2355,7 @@ TypeId TypeChecker::checkBinaryOperation( } if (get(rhsType)) - unify(lhsType, rhsType, expr.location); + unify(rhsType, lhsType, expr.location); if (typeCouldHaveMetatable(lhsType) || typeCouldHaveMetatable(rhsType)) { @@ -2334,7 +2366,7 @@ TypeId TypeChecker::checkBinaryOperation( TypeId expectedFunctionType = addType(FunctionTypeVar(scope->level, arguments, retTypePack)); Unifier state = mkUnifier(expr.location); - state.tryUnify(expectedFunctionType, actualFunctionType, /*isFunctionCall*/ true); + state.tryUnify(actualFunctionType, expectedFunctionType, /*isFunctionCall*/ true); reportErrors(state.errors); bool hasErrors = !state.errors.empty(); @@ -2345,11 +2377,28 @@ TypeId TypeChecker::checkBinaryOperation( // so we loosen the argument types to see if that helps. TypePackId fallbackArguments = freshTypePack(scope); TypeId fallbackFunctionType = addType(FunctionTypeVar(scope->level, fallbackArguments, retTypePack)); - state.log.rollback(); state.errors.clear(); - state.tryUnify(fallbackFunctionType, actualFunctionType, /*isFunctionCall*/ true); - if (!state.errors.empty()) - state.log.rollback(); + + if (FFlag::LuauUseCommittingTxnLog) + { + state.log.clear(); + } + else + { + state.DEPRECATED_log.rollback(); + } + + state.tryUnify(actualFunctionType, fallbackFunctionType, /*isFunctionCall*/ true); + + if (FFlag::LuauUseCommittingTxnLog && state.errors.empty()) + state.log.commit(); + else if (!state.errors.empty() && !FFlag::LuauUseCommittingTxnLog) + state.DEPRECATED_log.rollback(); + } + + if (FFlag::LuauUseCommittingTxnLog && !hasErrors) + { + state.log.commit(); } TypeId retType = first(retTypePack).value_or(nilType); @@ -2377,8 +2426,8 @@ TypeId TypeChecker::checkBinaryOperation( switch (expr.op) { case AstExprBinary::Concat: - reportErrors(tryUnify(addType(UnionTypeVar{{stringType, numberType}}), lhsType, expr.left->location)); - reportErrors(tryUnify(addType(UnionTypeVar{{stringType, numberType}}), rhsType, expr.right->location)); + reportErrors(tryUnify(lhsType, addType(UnionTypeVar{{stringType, numberType}}), expr.left->location)); + reportErrors(tryUnify(rhsType, addType(UnionTypeVar{{stringType, numberType}}), expr.right->location)); return stringType; case AstExprBinary::Add: case AstExprBinary::Sub: @@ -2386,8 +2435,8 @@ TypeId TypeChecker::checkBinaryOperation( case AstExprBinary::Div: case AstExprBinary::Mod: case AstExprBinary::Pow: - reportErrors(tryUnify(numberType, lhsType, expr.left->location)); - reportErrors(tryUnify(numberType, rhsType, expr.right->location)); + reportErrors(tryUnify(lhsType, numberType, expr.left->location)); + reportErrors(tryUnify(rhsType, numberType, expr.right->location)); return numberType; default: // These should have been handled with checkRelationalOperation @@ -2466,10 +2515,10 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprTy if (FFlag::LuauBidirectionalAsExpr) { // Note: As an optimization, we try 'number <: number | string' first, as that is the more likely case. - if (canUnify(result.type, annotationType, expr.location).empty()) + if (canUnify(annotationType, result.type, expr.location).empty()) return {annotationType, std::move(result.predicates)}; - if (canUnify(annotationType, result.type, expr.location).empty()) + if (canUnify(result.type, annotationType, expr.location).empty()) return {annotationType, std::move(result.predicates)}; reportError(expr.location, TypesAreUnrelated{result.type, annotationType}); @@ -2477,7 +2526,7 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprTy } else { - ErrorVec errorVec = canUnify(result.type, annotationType, expr.location); + ErrorVec errorVec = canUnify(annotationType, result.type, expr.location); reportErrors(errorVec); if (!errorVec.empty()) annotationType = errorRecoveryType(annotationType); @@ -2512,7 +2561,7 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprIf resolve(result.predicates, falseScope, false); ExprResult falseType = checkExpr(falseScope, *expr.falseExpr); - unify(trueType.type, falseType.type, expr.location); + unify(falseType.type, trueType.type, expr.location); // TODO: normalize(UnionTypeVar{{trueType, falseType}}) // For now both trueType and falseType must be the same type. @@ -2607,14 +2656,18 @@ std::pair TypeChecker::checkLValueBinding(const ScopePtr& scope else if (auto indexer = lhsTable->indexer) { Unifier state = mkUnifier(expr.location); - state.tryUnify(indexer->indexType, stringType); + state.tryUnify(stringType, indexer->indexType); TypeId retType = indexer->indexResultType; if (!state.errors.empty()) { - state.log.rollback(); + if (!FFlag::LuauUseCommittingTxnLog) + state.DEPRECATED_log.rollback(); + reportError(expr.location, UnknownProperty{lhs, name}); retType = errorRecoveryType(retType); } + else if (FFlag::LuauUseCommittingTxnLog) + state.log.commit(); return std::pair(retType, nullptr); } @@ -2713,7 +2766,7 @@ std::pair TypeChecker::checkLValueBinding(const ScopePtr& scope if (exprTable->indexer) { const TableIndexer& indexer = *exprTable->indexer; - unify(indexer.indexType, indexType, expr.index->location); + unify(indexType, indexer.indexType, expr.index->location); return std::pair(indexer.indexResultType, nullptr); } else if (exprTable->state == TableState::Unsealed || exprTable->state == TableState::Free) @@ -3106,204 +3159,402 @@ void TypeChecker::checkArgumentList( * A function requires parameters. * To call a function, you supply arguments. */ - TypePackIterator argIter = begin(argPack); - TypePackIterator paramIter = begin(paramPack); + TypePackIterator argIter = begin(argPack, &state.log); + TypePackIterator paramIter = begin(paramPack, &state.log); TypePackIterator endIter = end(argPack); // Important subtlety: All end TypePackIterators are equivalent size_t paramIndex = 0; size_t minParams = getMinParameterCount(paramPack); - while (true) + if (FFlag::LuauUseCommittingTxnLog) { - state.location = paramIndex < argLocations.size() ? argLocations[paramIndex] : state.location; - - if (argIter == endIter && paramIter == endIter) + while (true) { - std::optional argTail = argIter.tail(); - std::optional paramTail = paramIter.tail(); + state.location = paramIndex < argLocations.size() ? argLocations[paramIndex] : state.location; - // If we hit the end of both type packs simultaneously, then there are definitely no further type - // errors to report. All we need to do is tie up any free tails. - // - // If one side has a free tail and the other has none at all, we create an empty pack and bind the - // free tail to that. - - if (argTail) + if (argIter == endIter && paramIter == endIter) { - if (get(*argTail)) + std::optional argTail = argIter.tail(); + std::optional paramTail = paramIter.tail(); + + // If we hit the end of both type packs simultaneously, then there are definitely no further type + // errors to report. All we need to do is tie up any free tails. + // + // If one side has a free tail and the other has none at all, we create an empty pack and bind the + // free tail to that. + + if (argTail) { - if (paramTail) - state.tryUnify(*argTail, *paramTail); + if (state.log.getMutable(state.log.follow(*argTail))) + { + if (paramTail) + state.tryUnify(*paramTail, *argTail); + else + state.log.replace(*argTail, TypePackVar(TypePack{{}})); + } + } + else if (paramTail) + { + // argTail is definitely empty + if (state.log.getMutable(state.log.follow(*paramTail))) + state.log.replace(*paramTail, TypePackVar(TypePack{{}})); + } + + return; + } + else if (argIter == endIter) + { + // Not enough arguments. + + // Might be ok if we are forwarding a vararg along. This is a common thing to occur in nonstrict mode. + if (argIter.tail()) + { + TypePackId tail = *argIter.tail(); + if (state.log.getMutable(tail)) + { + // Unify remaining parameters so we don't leave any free-types hanging around. + while (paramIter != endIter) + { + state.tryUnify(errorRecoveryType(anyType), *paramIter); + ++paramIter; + } + return; + } + else if (auto vtp = state.log.getMutable(tail)) + { + while (paramIter != endIter) + { + state.tryUnify(vtp->ty, *paramIter); + ++paramIter; + } + + return; + } + else if (state.log.getMutable(tail)) + { + std::vector rest; + rest.reserve(std::distance(paramIter, endIter)); + while (paramIter != endIter) + { + rest.push_back(*paramIter); + ++paramIter; + } + + TypePackId varPack = addTypePack(TypePackVar{TypePack{rest, paramIter.tail()}}); + state.tryUnify(varPack, tail); + return; + } + } + + // If any remaining unfulfilled parameters are nonoptional, this is a problem. + while (paramIter != endIter) + { + TypeId t = state.log.follow(*paramIter); + if (isOptional(t)) + { + } // ok + else if (state.log.getMutable(t)) + { + } // ok + else if (isNonstrictMode() && state.log.getMutable(t)) + { + } // ok else { - state.log(*argTail); - *asMutable(*argTail) = TypePack{{}}; + state.errors.push_back(TypeError{state.location, CountMismatch{minParams, paramIndex}}); + return; + } + ++paramIter; + } + } + else if (paramIter == endIter) + { + // too many parameters passed + if (!paramIter.tail()) + { + while (argIter != endIter) + { + // The use of unify here is deliberate. We don't want this unification + // to be undoable. + unify(errorRecoveryType(scope), *argIter, state.location); + ++argIter; + } + // For this case, we want the error span to cover every errant extra parameter + Location location = state.location; + if (!argLocations.empty()) + location = {state.location.begin, argLocations.back().end}; + state.errors.push_back(TypeError{location, CountMismatch{minParams, std::distance(begin(argPack), end(argPack))}}); + return; + } + TypePackId tail = state.log.follow(*paramIter.tail()); + + if (state.log.getMutable(tail)) + { + // Function is variadic. Ok. + return; + } + else if (auto vtp = state.log.getMutable(tail)) + { + // Function is variadic and requires that all subsequent parameters + // be compatible with a type. + size_t argIndex = paramIndex; + while (argIter != endIter) + { + Location location = state.location; + + if (argIndex < argLocations.size()) + location = argLocations[argIndex]; + + unify(*argIter, vtp->ty, location); + ++argIter; + ++argIndex; + } + + return; + } + else if (state.log.getMutable(tail)) + { + // Create a type pack out of the remaining argument types + // and unify it with the tail. + std::vector rest; + rest.reserve(std::distance(argIter, endIter)); + while (argIter != endIter) + { + rest.push_back(*argIter); + ++argIter; + } + + TypePackId varPack = addTypePack(TypePackVar{TypePack{rest, argIter.tail()}}); + state.tryUnify(varPack, tail); + return; + } + else if (state.log.getMutable(tail)) + { + state.log.replace(tail, TypePackVar(TypePack{{}})); + return; + } + else if (state.log.getMutable(tail)) + { + // For this case, we want the error span to cover every errant extra parameter + Location location = state.location; + if (!argLocations.empty()) + location = {state.location.begin, argLocations.back().end}; + // TODO: Better error message? + state.errors.push_back(TypeError{location, CountMismatch{minParams, std::distance(begin(argPack), end(argPack))}}); + return; + } + } + else + { + unifyWithInstantiationIfNeeded(scope, *argIter, *paramIter, state); + ++argIter; + ++paramIter; + } + + ++paramIndex; + } + } + else + { + while (true) + { + state.location = paramIndex < argLocations.size() ? argLocations[paramIndex] : state.location; + + if (argIter == endIter && paramIter == endIter) + { + std::optional argTail = argIter.tail(); + std::optional paramTail = paramIter.tail(); + + // If we hit the end of both type packs simultaneously, then there are definitely no further type + // errors to report. All we need to do is tie up any free tails. + // + // If one side has a free tail and the other has none at all, we create an empty pack and bind the + // free tail to that. + + if (argTail) + { + if (get(*argTail)) + { + if (paramTail) + state.tryUnify(*paramTail, *argTail); + else + { + state.DEPRECATED_log(*argTail); + *asMutable(*argTail) = TypePack{{}}; + } } } - } - else if (paramTail) - { - // argTail is definitely empty - if (get(*paramTail)) + else if (paramTail) { - state.log(*paramTail); - *asMutable(*paramTail) = TypePack{{}}; + // argTail is definitely empty + if (get(*paramTail)) + { + state.DEPRECATED_log(*paramTail); + *asMutable(*paramTail) = TypePack{{}}; + } + } + + return; + } + else if (argIter == endIter) + { + // Not enough arguments. + + // Might be ok if we are forwarding a vararg along. This is a common thing to occur in nonstrict mode. + if (argIter.tail()) + { + TypePackId tail = *argIter.tail(); + if (get(tail)) + { + // Unify remaining parameters so we don't leave any free-types hanging around. + while (paramIter != endIter) + { + state.tryUnify(*paramIter, errorRecoveryType(anyType)); + ++paramIter; + } + return; + } + else if (auto vtp = get(tail)) + { + while (paramIter != endIter) + { + state.tryUnify(*paramIter, vtp->ty); + ++paramIter; + } + + return; + } + else if (get(tail)) + { + std::vector rest; + rest.reserve(std::distance(paramIter, endIter)); + while (paramIter != endIter) + { + rest.push_back(*paramIter); + ++paramIter; + } + + TypePackId varPack = addTypePack(TypePackVar{TypePack{rest, paramIter.tail()}}); + state.tryUnify(varPack, tail); + return; + } + } + + // If any remaining unfulfilled parameters are nonoptional, this is a problem. + while (paramIter != endIter) + { + TypeId t = follow(*paramIter); + if (isOptional(t)) + { + } // ok + else if (get(t)) + { + } // ok + else if (isNonstrictMode() && get(t)) + { + } // ok + else + { + state.errors.push_back(TypeError{state.location, CountMismatch{minParams, paramIndex}}); + return; + } + ++paramIter; } } - - return; - } - else if (argIter == endIter) - { - // Not enough arguments. - - // Might be ok if we are forwarding a vararg along. This is a common thing to occur in nonstrict mode. - if (argIter.tail()) + else if (paramIter == endIter) { - TypePackId tail = *argIter.tail(); + // too many parameters passed + if (!paramIter.tail()) + { + while (argIter != endIter) + { + unify(*argIter, errorRecoveryType(scope), state.location); + ++argIter; + } + // For this case, we want the error span to cover every errant extra parameter + Location location = state.location; + if (!argLocations.empty()) + location = {state.location.begin, argLocations.back().end}; + state.errors.push_back(TypeError{location, CountMismatch{minParams, std::distance(begin(argPack), end(argPack))}}); + return; + } + TypePackId tail = *paramIter.tail(); + if (get(tail)) { - // Unify remaining parameters so we don't leave any free-types hanging around. - while (paramIter != endIter) - { - state.tryUnify(*paramIter, errorRecoveryType(anyType)); - ++paramIter; - } + // Function is variadic. Ok. return; } else if (auto vtp = get(tail)) { - while (paramIter != endIter) + // Function is variadic and requires that all subsequent parameters + // be compatible with a type. + size_t argIndex = paramIndex; + while (argIter != endIter) { - state.tryUnify(*paramIter, vtp->ty); - ++paramIter; + Location location = state.location; + + if (argIndex < argLocations.size()) + location = argLocations[argIndex]; + + unify(*argIter, vtp->ty, location); + ++argIter; + ++argIndex; } return; } else if (get(tail)) { + // Create a type pack out of the remaining argument types + // and unify it with the tail. std::vector rest; - rest.reserve(std::distance(paramIter, endIter)); - while (paramIter != endIter) + rest.reserve(std::distance(argIter, endIter)); + while (argIter != endIter) { - rest.push_back(*paramIter); - ++paramIter; + rest.push_back(*argIter); + ++argIter; } - TypePackId varPack = addTypePack(TypePackVar{TypePack{rest, paramIter.tail()}}); + TypePackId varPack = addTypePack(TypePackVar{TypePack{rest, argIter.tail()}}); state.tryUnify(tail, varPack); return; } - } + else if (get(tail)) + { + if (FFlag::LuauUseCommittingTxnLog) + { + state.log.replace(tail, TypePackVar(TypePack{{}})); + } + else + { + state.DEPRECATED_log(tail); + *asMutable(tail) = TypePack{}; + } - // If any remaining unfulfilled parameters are nonoptional, this is a problem. - while (paramIter != endIter) - { - TypeId t = follow(*paramIter); - if (isOptional(t)) - { - } // ok - else if (get(t)) - { - } // ok - else if (isNonstrictMode() && get(t)) - { - } // ok - else - { - state.errors.push_back(TypeError{state.location, CountMismatch{minParams, paramIndex}}); return; } + else if (get(tail)) + { + // For this case, we want the error span to cover every errant extra parameter + Location location = state.location; + if (!argLocations.empty()) + location = {state.location.begin, argLocations.back().end}; + // TODO: Better error message? + state.errors.push_back(TypeError{location, CountMismatch{minParams, std::distance(begin(argPack), end(argPack))}}); + return; + } + } + else + { + unifyWithInstantiationIfNeeded(scope, *argIter, *paramIter, state); + ++argIter; ++paramIter; } + + ++paramIndex; } - else if (paramIter == endIter) - { - // too many parameters passed - if (!paramIter.tail()) - { - while (argIter != endIter) - { - unify(*argIter, errorRecoveryType(scope), state.location); - ++argIter; - } - // For this case, we want the error span to cover every errant extra parameter - Location location = state.location; - if (!argLocations.empty()) - location = {state.location.begin, argLocations.back().end}; - state.errors.push_back(TypeError{location, CountMismatch{minParams, std::distance(begin(argPack), end(argPack))}}); - return; - } - TypePackId tail = *paramIter.tail(); - - if (get(tail)) - { - // Function is variadic. Ok. - return; - } - else if (auto vtp = get(tail)) - { - // Function is variadic and requires that all subsequent parameters - // be compatible with a type. - size_t argIndex = paramIndex; - while (argIter != endIter) - { - Location location = state.location; - - if (argIndex < argLocations.size()) - location = argLocations[argIndex]; - - unify(vtp->ty, *argIter, location); - ++argIter; - ++argIndex; - } - - return; - } - else if (get(tail)) - { - // Create a type pack out of the remaining argument types - // and unify it with the tail. - std::vector rest; - rest.reserve(std::distance(argIter, endIter)); - while (argIter != endIter) - { - rest.push_back(*argIter); - ++argIter; - } - - TypePackId varPack = addTypePack(TypePackVar{TypePack{rest, argIter.tail()}}); - state.tryUnify(tail, varPack); - return; - } - else if (get(tail)) - { - state.log(tail); - *asMutable(tail) = TypePack{}; - - return; - } - else if (get(tail)) - { - // For this case, we want the error span to cover every errant extra parameter - Location location = state.location; - if (!argLocations.empty()) - location = {state.location.begin, argLocations.back().end}; - // TODO: Better error message? - state.errors.push_back(TypeError{location, CountMismatch{minParams, std::distance(begin(argPack), end(argPack))}}); - return; - } - } - else - { - unifyWithInstantiationIfNeeded(scope, *paramIter, *argIter, state); - ++argIter; - ++paramIter; - } - - ++paramIndex; } } @@ -3475,7 +3726,7 @@ std::optional> TypeChecker::checkCallOverload(const Scope if (get(fn)) { - unify(argPack, anyTypePack, expr.location); + unify(anyTypePack, argPack, expr.location); return {{anyTypePack}}; } @@ -3490,7 +3741,7 @@ std::optional> TypeChecker::checkCallOverload(const Scope // has been instantiated, so is a monotype. We can therefore // unify it with a monomorphic function. TypeId r = addType(FunctionTypeVar(scope->level, argPack, retPack)); - unify(r, fn, expr.location); + unify(fn, r, expr.location); return {{retPack}}; } @@ -3533,7 +3784,7 @@ std::optional> TypeChecker::checkCallOverload(const Scope if (!ftv) { reportError(TypeError{expr.func->location, CannotCallNonFunction{fn}}); - unify(retPack, errorRecoveryTypePack(scope), expr.func->location); + unify(errorRecoveryTypePack(scope), retPack, expr.func->location); return {{errorRecoveryTypePack(retPack)}}; } @@ -3552,7 +3803,9 @@ std::optional> TypeChecker::checkCallOverload(const Scope checkArgumentList(scope, state, retPack, ftv->retType, /*argLocations*/ {}); if (!state.errors.empty()) { - state.log.rollback(); + if (!FFlag::LuauUseCommittingTxnLog) + state.DEPRECATED_log.rollback(); + return {}; } @@ -3580,10 +3833,15 @@ std::optional> TypeChecker::checkCallOverload(const Scope overloadsThatDont.push_back(fn); errors.emplace_back(std::move(state.errors), args->head, ftv); - state.log.rollback(); + + if (!FFlag::LuauUseCommittingTxnLog) + state.DEPRECATED_log.rollback(); } else { + if (FFlag::LuauUseCommittingTxnLog) + state.log.commit(); + if (isNonstrictMode() && !expr.self && expr.func->is() && ftv->hasSelf) { // If we are running in nonstrict mode, passing fewer arguments than the function is declared to take AND @@ -3640,6 +3898,9 @@ bool TypeChecker::handleSelfCallMismatch(const ScopePtr& scope, const AstExprCal if (editedState.errors.empty()) { + if (FFlag::LuauUseCommittingTxnLog) + editedState.log.commit(); + reportError(TypeError{expr.location, FunctionDoesNotTakeSelf{}}); // This is a little bit suspect: If this overload would work with a . replaced by a : // we eagerly assume that that's what you actually meant and we commit to it. @@ -3648,8 +3909,8 @@ bool TypeChecker::handleSelfCallMismatch(const ScopePtr& scope, const AstExprCal // checkArgumentList(scope, editedState, retPack, ftv->retType, retLocations, CountMismatch::Return); return true; } - else - editedState.log.rollback(); + else if (!FFlag::LuauUseCommittingTxnLog) + editedState.DEPRECATED_log.rollback(); } else if (ftv->hasSelf) { @@ -3671,6 +3932,9 @@ bool TypeChecker::handleSelfCallMismatch(const ScopePtr& scope, const AstExprCal if (editedState.errors.empty()) { + if (FFlag::LuauUseCommittingTxnLog) + editedState.log.commit(); + reportError(TypeError{expr.location, FunctionRequiresSelf{}}); // This is a little bit suspect: If this overload would work with a : replaced by a . // we eagerly assume that that's what you actually meant and we commit to it. @@ -3679,8 +3943,8 @@ bool TypeChecker::handleSelfCallMismatch(const ScopePtr& scope, const AstExprCal // checkArgumentList(scope, editedState, retPack, ftv->retType, retLocations, CountMismatch::Return); return true; } - else - editedState.log.rollback(); + else if (!FFlag::LuauUseCommittingTxnLog) + editedState.DEPRECATED_log.rollback(); } } } @@ -3740,6 +4004,9 @@ void TypeChecker::reportOverloadResolutionError(const ScopePtr& scope, const Ast checkArgumentList(scope, state, argPack, ftv->argTypes, argLocations); } + if (FFlag::LuauUseCommittingTxnLog && state.errors.empty()) + state.log.commit(); + if (i > 0) s += "; "; @@ -3748,7 +4015,8 @@ void TypeChecker::reportOverloadResolutionError(const ScopePtr& scope, const Ast s += toString(overload); - state.log.rollback(); + if (!FFlag::LuauUseCommittingTxnLog) + state.DEPRECATED_log.rollback(); } if (overloadsThatMatchArgCount.size() == 0) @@ -3781,6 +4049,8 @@ ExprResult TypeChecker::checkExprList(const ScopePtr& scope, const L Unifier state = mkUnifier(location); + std::vector inverseLogs; + for (size_t i = 0; i < exprs.size; ++i) { AstExpr* expr = exprs.data[i]; @@ -3791,18 +4061,15 @@ ExprResult TypeChecker::checkExprList(const ScopePtr& scope, const L auto [typePack, exprPredicates] = checkExprPack(scope, *expr); insert(exprPredicates); - if (FFlag::LuauTailArgumentTypeInfo) + if (std::optional firstTy = first(typePack)) { - if (std::optional firstTy = first(typePack)) - { - if (!currentModule->astTypes.find(expr)) - currentModule->astTypes[expr] = follow(*firstTy); - } - - if (expectedType) - currentModule->astExpectedTypes[expr] = *expectedType; + if (!currentModule->astTypes.find(expr)) + currentModule->astTypes[expr] = follow(*firstTy); } + if (expectedType) + currentModule->astExpectedTypes[expr] = *expectedType; + tp->tail = typePack; } else @@ -3816,13 +4083,31 @@ ExprResult TypeChecker::checkExprList(const ScopePtr& scope, const L actualType = instantiate(scope, actualType, expr->location); if (expectedType) - state.tryUnify(*expectedType, actualType); + { + state.tryUnify(actualType, *expectedType); + + // Ugly: In future iterations of the loop, we might need the state of the unification we + // just performed. There's not a great way to pass that into checkExpr. Instead, we store + // the inverse of the current log, and commit it. When we're done, we'll commit all the + // inverses. This isn't optimal, and a better solution is welcome here. + if (FFlag::LuauUseCommittingTxnLog) + { + inverseLogs.push_back(state.log.inverse()); + state.log.commit(); + } + } tp->head.push_back(actualType); } } - state.log.rollback(); + if (FFlag::LuauUseCommittingTxnLog) + { + for (TxnLog& log : inverseLogs) + log.commit(); + } + else + state.DEPRECATED_log.rollback(); return {pack, predicates}; } @@ -3884,7 +4169,7 @@ TypeId TypeChecker::checkRequire(const ScopePtr& scope, const ModuleInfo& module TypePackId modulePack = module->getModuleScope()->returnType; - if (FFlag::LuauModuleRequireErrorPack && get(modulePack)) + if (get(modulePack)) return errorRecoveryType(scope); std::optional moduleType = first(modulePack); @@ -3917,72 +4202,94 @@ TypeId TypeChecker::anyIfNonstrict(TypeId ty) const return ty; } -bool TypeChecker::unify(TypeId left, TypeId right, const Location& location) +bool TypeChecker::unify(TypeId subTy, TypeId superTy, const Location& location) { Unifier state = mkUnifier(location); - state.tryUnify(left, right); + state.tryUnify(subTy, superTy); + + if (FFlag::LuauUseCommittingTxnLog) + state.log.commit(); reportErrors(state.errors); return state.errors.empty(); } -bool TypeChecker::unify(TypePackId left, TypePackId right, const Location& location, CountMismatch::Context ctx) +bool TypeChecker::unify(TypePackId subTy, TypePackId superTy, const Location& location, CountMismatch::Context ctx) { Unifier state = mkUnifier(location); state.ctx = ctx; - state.tryUnify(left, right); + state.tryUnify(subTy, superTy); + + if (FFlag::LuauUseCommittingTxnLog) + state.log.commit(); reportErrors(state.errors); return state.errors.empty(); } -bool TypeChecker::unifyWithInstantiationIfNeeded(const ScopePtr& scope, TypeId left, TypeId right, const Location& location) +bool TypeChecker::unifyWithInstantiationIfNeeded(const ScopePtr& scope, TypeId subTy, TypeId superTy, const Location& location) { Unifier state = mkUnifier(location); - unifyWithInstantiationIfNeeded(scope, left, right, state); + unifyWithInstantiationIfNeeded(scope, subTy, superTy, state); + + if (FFlag::LuauUseCommittingTxnLog) + state.log.commit(); reportErrors(state.errors); return state.errors.empty(); } -void TypeChecker::unifyWithInstantiationIfNeeded(const ScopePtr& scope, TypeId left, TypeId right, Unifier& state) +void TypeChecker::unifyWithInstantiationIfNeeded(const ScopePtr& scope, TypeId subTy, TypeId superTy, Unifier& state) { - if (!maybeGeneric(right)) + if (!maybeGeneric(subTy)) // Quick check to see if we definitely can't instantiate - state.tryUnify(left, right, /*isFunctionCall*/ false); - else if (!maybeGeneric(left) && isGeneric(right)) + state.tryUnify(subTy, superTy, /*isFunctionCall*/ false); + else if (!maybeGeneric(superTy) && isGeneric(subTy)) { // Quick check to see if we definitely have to instantiate - TypeId instantiated = instantiate(scope, right, state.location); - state.tryUnify(left, instantiated, /*isFunctionCall*/ false); + TypeId instantiated = instantiate(scope, subTy, state.location); + state.tryUnify(instantiated, superTy, /*isFunctionCall*/ false); } else { // First try unifying with the original uninstantiated type // but if that fails, try the instantiated one. Unifier child = state.makeChildUnifier(); - child.tryUnify(left, right, /*isFunctionCall*/ false); + child.tryUnify(subTy, superTy, /*isFunctionCall*/ false); if (!child.errors.empty()) { - TypeId instantiated = instantiate(scope, right, state.location); - if (right == instantiated) + TypeId instantiated = instantiate(scope, subTy, state.location); + if (subTy == instantiated) { // Instantiating the argument made no difference, so just report any child errors - state.log.concat(std::move(child.log)); + if (FFlag::LuauUseCommittingTxnLog) + state.log.concat(std::move(child.log)); + else + state.DEPRECATED_log.concat(std::move(child.DEPRECATED_log)); + state.errors.insert(state.errors.end(), child.errors.begin(), child.errors.end()); } else { - child.log.rollback(); - state.tryUnify(left, instantiated, /*isFunctionCall*/ false); + if (!FFlag::LuauUseCommittingTxnLog) + child.DEPRECATED_log.rollback(); + + state.tryUnify(instantiated, superTy, /*isFunctionCall*/ false); } } else { - state.log.concat(std::move(child.log)); + if (FFlag::LuauUseCommittingTxnLog) + { + state.log.concat(std::move(child.log)); + } + else + { + state.DEPRECATED_log.concat(std::move(child.DEPRECATED_log)); + } } } } @@ -4139,7 +4446,7 @@ TypePackId Quantification::clean(TypePackId tp) bool Anyification::isDirty(TypeId ty) { if (const TableTypeVar* ttv = get(ty)) - return (ttv->state == TableState::Free); + return (ttv->state == TableState::Free || (FFlag::LuauSealExports && ttv->state == TableState::Unsealed)); else if (get(ty)) return true; else @@ -4162,6 +4469,12 @@ TypeId Anyification::clean(TypeId ty) TableTypeVar clone = TableTypeVar{ttv->props, ttv->indexer, ttv->level, TableState::Sealed}; clone.methodDefinitionLocations = ttv->methodDefinitionLocations; clone.definitionModuleName = ttv->definitionModuleName; + if (FFlag::LuauSealExports) + { + clone.name = ttv->name; + clone.syntheticName = ttv->syntheticName; + clone.tags = ttv->tags; + } return addType(std::move(clone)); } else @@ -5194,8 +5507,8 @@ void TypeChecker::resolve(const IsAPredicate& isaP, ErrorVec& errVec, Refinement // This by itself is not truly enough to determine that A is stronger than B or vice versa. // The best unambiguous way about this would be to have a function that returns the relationship ordering of a pair. // i.e. TypeRelationship relationshipOf(TypeId superTy, TypeId subTy) - bool optionIsSubtype = canUnify(isaP.ty, option, isaP.location).empty(); - bool targetIsSubtype = canUnify(option, isaP.ty, isaP.location).empty(); + bool optionIsSubtype = canUnify(option, isaP.ty, isaP.location).empty(); + bool targetIsSubtype = canUnify(isaP.ty, option, isaP.location).empty(); // If A is a superset of B, then if sense is true, we promote A to B, otherwise we keep A. if (!optionIsSubtype && targetIsSubtype) @@ -5379,7 +5692,7 @@ void TypeChecker::resolve(const EqPredicate& eqP, ErrorVec& errVec, RefinementMa for (TypeId right : rhs) { // When singleton types arrive, `isNil` here probably should be replaced with `isLiteral`. - if (canUnify(left, right, eqP.location).empty() == sense || (!sense && !isNil(left))) + if (canUnify(right, left, eqP.location).empty() == sense || (!sense && !isNil(left))) set.insert(left); } } @@ -5406,7 +5719,7 @@ std::vector TypeChecker::unTypePack(const ScopePtr& scope, TypePackId tp for (size_t i = 0; i < expectedLength; ++i) expectedPack->head.push_back(freshType(scope)); - unify(expectedTypePack, tp, location); + unify(tp, expectedTypePack, location); for (TypeId& tp : expectedPack->head) tp = follow(tp); diff --git a/Analysis/src/TypePack.cpp b/Analysis/src/TypePack.cpp index d3221c73..b15548a8 100644 --- a/Analysis/src/TypePack.cpp +++ b/Analysis/src/TypePack.cpp @@ -1,8 +1,12 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #include "Luau/TypePack.h" +#include "Luau/TxnLog.h" + #include +LUAU_FASTFLAG(LuauUseCommittingTxnLog) + namespace Luau { @@ -35,14 +39,28 @@ TypePackVar& TypePackVar::operator=(TypePackVariant&& tp) } TypePackIterator::TypePackIterator(TypePackId typePack) + : TypePackIterator(typePack, TxnLog::empty()) +{ +} + +TypePackIterator::TypePackIterator(TypePackId typePack, const TxnLog* log) : currentTypePack(follow(typePack)) , tp(get(currentTypePack)) , currentIndex(0) + , log(log) { while (tp && tp->head.empty()) { - currentTypePack = tp->tail ? follow(*tp->tail) : nullptr; - tp = currentTypePack ? get(currentTypePack) : nullptr; + if (FFlag::LuauUseCommittingTxnLog) + { + currentTypePack = tp->tail ? log->follow(*tp->tail) : nullptr; + tp = currentTypePack ? log->getMutable(currentTypePack) : nullptr; + } + else + { + currentTypePack = tp->tail ? follow(*tp->tail) : nullptr; + tp = currentTypePack ? get(currentTypePack) : nullptr; + } } } @@ -53,8 +71,17 @@ TypePackIterator& TypePackIterator::operator++() ++currentIndex; while (tp && currentIndex >= tp->head.size()) { - currentTypePack = tp->tail ? follow(*tp->tail) : nullptr; - tp = currentTypePack ? get(currentTypePack) : nullptr; + if (FFlag::LuauUseCommittingTxnLog) + { + currentTypePack = tp->tail ? log->follow(*tp->tail) : nullptr; + tp = currentTypePack ? log->getMutable(currentTypePack) : nullptr; + } + else + { + currentTypePack = tp->tail ? follow(*tp->tail) : nullptr; + tp = currentTypePack ? get(currentTypePack) : nullptr; + } + currentIndex = 0; } @@ -95,6 +122,11 @@ TypePackIterator begin(TypePackId tp) return TypePackIterator{tp}; } +TypePackIterator begin(TypePackId tp, TxnLog* log) +{ + return TypePackIterator{tp, log}; +} + TypePackIterator end(TypePackId tp) { return TypePackIterator{}; @@ -160,8 +192,15 @@ bool areEqual(SeenSet& seen, const TypePackVar& lhs, const TypePackVar& rhs) TypePackId follow(TypePackId tp) { - auto advance = [](TypePackId ty) -> std::optional { - if (const Unifiable::Bound* btv = get>(ty)) + return follow(tp, [](TypePackId t) { + return t; + }); +} + +TypePackId follow(TypePackId tp, std::function mapper) +{ + auto advance = [&mapper](TypePackId ty) -> std::optional { + if (const Unifiable::Bound* btv = get>(mapper(ty))) return btv->boundTo; else return std::nullopt; diff --git a/Analysis/src/TypeVar.cpp b/Analysis/src/TypeVar.cpp index fb75aa02..4cab79c8 100644 --- a/Analysis/src/TypeVar.cpp +++ b/Analysis/src/TypeVar.cpp @@ -31,17 +31,24 @@ std::optional> magicFunctionFormat( TypeId follow(TypeId t) { - auto advance = [](TypeId ty) -> std::optional { - if (auto btv = get>(ty)) + return follow(t, [](TypeId t) { + return t; + }); +} + +TypeId follow(TypeId t, std::function mapper) +{ + auto advance = [&mapper](TypeId ty) -> std::optional { + if (auto btv = get>(mapper(ty))) return btv->boundTo; - else if (auto ttv = get(ty)) + else if (auto ttv = get(mapper(ty))) return ttv->boundTo; else return std::nullopt; }; - auto force = [](TypeId ty) { - if (auto ltv = get_if(&ty->ty)) + auto force = [&mapper](TypeId ty) { + if (auto ltv = get_if(&mapper(ty)->ty)) { TypeId res = ltv->thunk(); if (get(res)) @@ -1004,7 +1011,7 @@ std::optional> magicFunctionFormat( { Location location = expr.args.data[std::min(i + dataOffset, expr.args.size - 1)]->location; - typechecker.unify(expected[i], params[i + paramOffset], location); + typechecker.unify(params[i + paramOffset], expected[i], location); } // if we know the argument count or if we have too many arguments for sure, we can issue an error diff --git a/Analysis/src/TypedAllocator.cpp b/Analysis/src/TypedAllocator.cpp index f037351e..1f7ef8c2 100644 --- a/Analysis/src/TypedAllocator.cpp +++ b/Analysis/src/TypedAllocator.cpp @@ -20,6 +20,7 @@ const size_t kPageSize = sysconf(_SC_PAGESIZE); #include LUAU_FASTFLAG(DebugLuauFreezeArena) +LUAU_FASTFLAGVARIABLE(LuauTypedAllocatorZeroStart, false) namespace Luau { diff --git a/Analysis/src/Unifier.cpp b/Analysis/src/Unifier.cpp index 43ea37e7..393a84a7 100644 --- a/Analysis/src/Unifier.cpp +++ b/Analysis/src/Unifier.cpp @@ -13,6 +13,7 @@ LUAU_FASTINT(LuauTypeInferRecursionLimit); LUAU_FASTINT(LuauTypeInferTypePackLoopLimit); +LUAU_FASTFLAG(LuauUseCommittingTxnLog) LUAU_FASTINTVARIABLE(LuauTypeInferIterationLimit, 2000); LUAU_FASTFLAGVARIABLE(LuauTableSubtypingVariance2, false); LUAU_FASTFLAGVARIABLE(LuauUnionHeuristic, false) @@ -29,27 +30,39 @@ namespace Luau struct PromoteTypeLevels { + DEPRECATED_TxnLog& DEPRECATED_log; TxnLog& log; TypeLevel minLevel; - explicit PromoteTypeLevels(TxnLog& log, TypeLevel minLevel) - : log(log) + explicit PromoteTypeLevels(DEPRECATED_TxnLog& DEPRECATED_log, TxnLog& log, TypeLevel minLevel) + : DEPRECATED_log(DEPRECATED_log) + , log(log) , minLevel(minLevel) - {} + { + } - template + template void promote(TID ty, T* t) { LUAU_ASSERT(t); if (minLevel.subsumesStrict(t->level)) { - log(ty); - t->level = minLevel; + if (FFlag::LuauUseCommittingTxnLog) + { + log.changeLevel(ty, minLevel); + } + else + { + DEPRECATED_log(ty); + t->level = minLevel; + } } } template - void cycle(TID) {} + void cycle(TID) + { + } template bool operator()(TID, const T&) @@ -59,39 +72,47 @@ struct PromoteTypeLevels bool operator()(TypeId ty, const FreeTypeVar&) { - promote(ty, getMutable(ty)); + // Surprise, it's actually a BoundTypeVar that hasn't been committed yet. + // Calling getMutable on this will trigger an assertion. + if (FFlag::LuauUseCommittingTxnLog && !log.is(ty)) + return true; + + promote(ty, FFlag::LuauUseCommittingTxnLog ? log.getMutable(ty) : getMutable(ty)); return true; } bool operator()(TypeId ty, const FunctionTypeVar&) { - promote(ty, getMutable(ty)); + promote(ty, FFlag::LuauUseCommittingTxnLog ? log.getMutable(ty) : getMutable(ty)); return true; } - bool operator()(TypeId ty, const TableTypeVar&) + bool operator()(TypeId ty, const TableTypeVar& ttv) { - promote(ty, getMutable(ty)); + if (ttv.state != TableState::Free && ttv.state != TableState::Generic) + return true; + + promote(ty, FFlag::LuauUseCommittingTxnLog ? log.getMutable(ty) : getMutable(ty)); return true; } bool operator()(TypePackId tp, const FreeTypePack&) { - promote(tp, getMutable(tp)); + promote(tp, FFlag::LuauUseCommittingTxnLog ? log.getMutable(tp) : getMutable(tp)); return true; } }; -void promoteTypeLevels(TxnLog& log, TypeLevel minLevel, TypeId ty) +void promoteTypeLevels(DEPRECATED_TxnLog& DEPRECATED_log, TxnLog& log, TypeLevel minLevel, TypeId ty) { - PromoteTypeLevels ptl{log, minLevel}; + PromoteTypeLevels ptl{DEPRECATED_log, log, minLevel}; DenseHashSet seen{nullptr}; visitTypeVarOnce(ty, ptl, seen); } -void promoteTypeLevels(TxnLog& log, TypeLevel minLevel, TypePackId tp) +void promoteTypeLevels(DEPRECATED_TxnLog& DEPRECATED_log, TxnLog& log, TypeLevel minLevel, TypePackId tp) { - PromoteTypeLevels ptl{log, minLevel}; + PromoteTypeLevels ptl{DEPRECATED_log, log, minLevel}; DenseHashSet seen{nullptr}; visitTypeVarOnce(tp, ptl, seen); } @@ -221,10 +242,12 @@ static std::optional> getTableMat return std::nullopt; } -Unifier::Unifier(TypeArena* types, Mode mode, ScopePtr globalScope, const Location& location, Variance variance, UnifierSharedState& sharedState) +Unifier::Unifier(TypeArena* types, Mode mode, ScopePtr globalScope, const Location& location, Variance variance, UnifierSharedState& sharedState, + TxnLog* parentLog) : types(types) , mode(mode) , globalScope(std::move(globalScope)) + , log(parentLog) , location(location) , variance(variance) , sharedState(sharedState) @@ -233,11 +256,12 @@ Unifier::Unifier(TypeArena* types, Mode mode, ScopePtr globalScope, const Locati } Unifier::Unifier(TypeArena* types, Mode mode, ScopePtr globalScope, std::vector>* sharedSeen, const Location& location, - Variance variance, UnifierSharedState& sharedState) + Variance variance, UnifierSharedState& sharedState, TxnLog* parentLog) : types(types) , mode(mode) , globalScope(std::move(globalScope)) - , log(sharedSeen) + , DEPRECATED_log(sharedSeen) + , log(parentLog, sharedSeen) , location(location) , variance(variance) , sharedState(sharedState) @@ -245,14 +269,14 @@ Unifier::Unifier(TypeArena* types, Mode mode, ScopePtr globalScope, std::vector< LUAU_ASSERT(sharedState.iceHandler); } -void Unifier::tryUnify(TypeId superTy, TypeId subTy, bool isFunctionCall, bool isIntersection) +void Unifier::tryUnify(TypeId subTy, TypeId superTy, bool isFunctionCall, bool isIntersection) { sharedState.counters.iterationCount = 0; - tryUnify_(superTy, subTy, isFunctionCall, isIntersection); + tryUnify_(subTy, superTy, isFunctionCall, isIntersection); } -void Unifier::tryUnify_(TypeId superTy, TypeId subTy, bool isFunctionCall, bool isIntersection) +void Unifier::tryUnify_(TypeId subTy, TypeId superTy, bool isFunctionCall, bool isIntersection) { RecursionLimiter _ra(&sharedState.counters.recursionCount, FInt::LuauTypeInferRecursionLimit); @@ -264,55 +288,112 @@ void Unifier::tryUnify_(TypeId superTy, TypeId subTy, bool isFunctionCall, bool return; } - superTy = follow(superTy); - subTy = follow(subTy); + if (FFlag::LuauUseCommittingTxnLog) + { + superTy = log.follow(superTy); + subTy = log.follow(subTy); + } + else + { + superTy = follow(superTy); + subTy = follow(subTy); + } if (superTy == subTy) return; - auto l = getMutable(superTy); - auto r = getMutable(subTy); + auto superFree = getMutable(superTy); + auto subFree = getMutable(subTy); - if (l && r && l->level.subsumes(r->level)) + if (FFlag::LuauUseCommittingTxnLog) + { + superFree = log.getMutable(superTy); + subFree = log.getMutable(subTy); + } + + if (superFree && subFree && superFree->level.subsumes(subFree->level)) { occursCheck(subTy, superTy); // The occurrence check might have caused superTy no longer to be a free type - if (!get(subTy)) + bool occursFailed = false; + if (FFlag::LuauUseCommittingTxnLog) + occursFailed = bool(log.getMutable(subTy)); + else + occursFailed = bool(get(subTy)); + + if (!occursFailed) { - log(subTy); - *asMutable(subTy) = BoundTypeVar(superTy); + if (FFlag::LuauUseCommittingTxnLog) + { + log.replace(subTy, BoundTypeVar(superTy)); + } + else + { + DEPRECATED_log(subTy); + *asMutable(subTy) = BoundTypeVar(superTy); + } } return; } - else if (l && r) + else if (superFree && subFree) { - if (!FFlag::LuauErrorRecoveryType) - log(superTy); - occursCheck(superTy, subTy); - r->level = min(r->level, l->level); - - // The occurrence check might have caused superTy no longer to be a free type - if (!FFlag::LuauErrorRecoveryType) - *asMutable(superTy) = BoundTypeVar(subTy); - else if (!get(superTy)) + if (!FFlag::LuauErrorRecoveryType && !FFlag::LuauUseCommittingTxnLog) + { + DEPRECATED_log(superTy); + subFree->level = min(subFree->level, superFree->level); + } + + occursCheck(superTy, subTy); + + bool occursFailed = false; + if (FFlag::LuauUseCommittingTxnLog) + occursFailed = bool(log.getMutable(superTy)); + else + occursFailed = bool(get(superTy)); + + if (!FFlag::LuauErrorRecoveryType && !FFlag::LuauUseCommittingTxnLog) { - log(superTy); *asMutable(superTy) = BoundTypeVar(subTy); + return; + } + + if (!occursFailed) + { + if (FFlag::LuauUseCommittingTxnLog) + { + if (superFree->level.subsumes(subFree->level)) + { + log.changeLevel(subTy, superFree->level); + } + + log.replace(superTy, BoundTypeVar(subTy)); + } + else + { + DEPRECATED_log(superTy); + *asMutable(superTy) = BoundTypeVar(subTy); + subFree->level = min(subFree->level, superFree->level); + } } return; } - else if (l) + else if (superFree) { occursCheck(superTy, subTy); + bool occursFailed = false; + if (FFlag::LuauUseCommittingTxnLog) + occursFailed = bool(log.getMutable(superTy)); + else + occursFailed = bool(get(superTy)); - TypeLevel superLevel = l->level; + TypeLevel superLevel = superFree->level; // Unification can't change the level of a generic. - auto rightGeneric = get(subTy); - if (rightGeneric && !rightGeneric->level.subsumes(superLevel)) + auto subGeneric = FFlag::LuauUseCommittingTxnLog ? log.getMutable(subTy) : get(subTy); + if (subGeneric && !subGeneric->level.subsumes(superLevel)) { // TODO: a more informative error message? CLI-39912 errors.push_back(TypeError{location, GenericError{"Generic subtype escaping scope"}}); @@ -320,63 +401,83 @@ void Unifier::tryUnify_(TypeId superTy, TypeId subTy, bool isFunctionCall, bool } // The occurrence check might have caused superTy no longer to be a free type - if (!get(superTy)) + if (!occursFailed) { - if (FFlag::LuauProperTypeLevels) - promoteTypeLevels(log, superLevel, subTy); - else if (auto rightLevel = getMutableLevel(subTy)) + if (FFlag::LuauUseCommittingTxnLog) { - if (!rightLevel->subsumes(l->level)) - *rightLevel = l->level; + promoteTypeLevels(DEPRECATED_log, log, superLevel, subTy); + log.replace(superTy, BoundTypeVar(subTy)); } + else + { + if (FFlag::LuauProperTypeLevels) + promoteTypeLevels(DEPRECATED_log, log, superLevel, subTy); + else if (auto subLevel = getMutableLevel(subTy)) + { + if (!subLevel->subsumes(superFree->level)) + *subLevel = superFree->level; + } - log(superTy); - *asMutable(superTy) = BoundTypeVar(subTy); + DEPRECATED_log(superTy); + *asMutable(superTy) = BoundTypeVar(subTy); + } } return; } - else if (r) + else if (subFree) { - TypeLevel subLevel = r->level; + TypeLevel subLevel = subFree->level; occursCheck(subTy, superTy); + bool occursFailed = false; + if (FFlag::LuauUseCommittingTxnLog) + occursFailed = bool(log.getMutable(subTy)); + else + occursFailed = bool(get(subTy)); // Unification can't change the level of a generic. - auto leftGeneric = get(superTy); - if (leftGeneric && !leftGeneric->level.subsumes(r->level)) + auto superGeneric = FFlag::LuauUseCommittingTxnLog ? log.getMutable(superTy) : get(superTy); + if (superGeneric && !superGeneric->level.subsumes(subFree->level)) { // TODO: a more informative error message? CLI-39912 errors.push_back(TypeError{location, GenericError{"Generic supertype escaping scope"}}); return; } - if (!get(subTy)) + if (!occursFailed) { - if (FFlag::LuauProperTypeLevels) - promoteTypeLevels(log, subLevel, superTy); - - if (auto superLevel = getMutableLevel(superTy)) + if (FFlag::LuauUseCommittingTxnLog) { - if (!superLevel->subsumes(r->level)) - { - log(superTy); - *superLevel = r->level; - } + promoteTypeLevels(DEPRECATED_log, log, subLevel, superTy); + log.replace(subTy, BoundTypeVar(superTy)); } + else + { + if (FFlag::LuauProperTypeLevels) + promoteTypeLevels(DEPRECATED_log, log, subLevel, superTy); + else if (auto superLevel = getMutableLevel(superTy)) + { + if (!superLevel->subsumes(subFree->level)) + { + DEPRECATED_log(superTy); + *superLevel = subFree->level; + } + } - log(subTy); - *asMutable(subTy) = BoundTypeVar(superTy); + DEPRECATED_log(subTy); + *asMutable(subTy) = BoundTypeVar(superTy); + } } return; } if (get(superTy) || get(superTy)) - return tryUnifyWithAny(superTy, subTy); + return tryUnifyWithAny(subTy, superTy); if (get(subTy) || get(subTy)) - return tryUnifyWithAny(subTy, superTy); + return tryUnifyWithAny(superTy, subTy); bool cacheEnabled = !isFunctionCall && !isIntersection; auto& cache = sharedState.cachedUnify; @@ -389,12 +490,22 @@ void Unifier::tryUnify_(TypeId superTy, TypeId subTy, bool isFunctionCall, bool // Here, we assume that the types unify. If they do not, we will find out as we roll back // the stack. - if (log.haveSeen(superTy, subTy)) - return; + if (FFlag::LuauUseCommittingTxnLog) + { + if (log.haveSeen(superTy, subTy)) + return; - log.pushSeen(superTy, subTy); + log.pushSeen(superTy, subTy); + } + else + { + if (DEPRECATED_log.haveSeen(superTy, subTy)) + return; - if (const UnionTypeVar* uv = get(subTy)) + DEPRECATED_log.pushSeen(superTy, subTy); + } + + if (const UnionTypeVar* uv = FFlag::LuauUseCommittingTxnLog ? log.getMutable(subTy) : get(subTy)) { // A | B <: T if A <: T and B <: T bool failed = false; @@ -407,7 +518,7 @@ void Unifier::tryUnify_(TypeId superTy, TypeId subTy, bool isFunctionCall, bool for (TypeId type : uv->options) { Unifier innerState = makeChildUnifier(); - innerState.tryUnify_(superTy, type); + innerState.tryUnify_(type, superTy); if (auto e = hasUnificationTooComplex(innerState.errors)) unificationTooComplex = e; @@ -420,10 +531,24 @@ void Unifier::tryUnify_(TypeId superTy, TypeId subTy, bool isFunctionCall, bool failed = true; } - if (i != count - 1) - innerState.log.rollback(); + if (FFlag::LuauUseCommittingTxnLog) + { + if (i == count - 1) + { + log.concat(std::move(innerState.log)); + } + } else - log.concat(std::move(innerState.log)); + { + if (i != count - 1) + { + innerState.DEPRECATED_log.rollback(); + } + else + { + DEPRECATED_log.concat(std::move(innerState.DEPRECATED_log)); + } + } ++i; } @@ -438,7 +563,7 @@ void Unifier::tryUnify_(TypeId superTy, TypeId subTy, bool isFunctionCall, bool errors.push_back(TypeError{location, TypeMismatch{superTy, subTy}}); } } - else if (const UnionTypeVar* uv = get(superTy)) + else if (const UnionTypeVar* uv = FFlag::LuauUseCommittingTxnLog ? log.getMutable(superTy) : get(superTy)) { // T <: A | B if T <: A or T <: B bool found = false; @@ -502,12 +627,16 @@ void Unifier::tryUnify_(TypeId superTy, TypeId subTy, bool isFunctionCall, bool { TypeId type = uv->options[(i + startIndex) % uv->options.size()]; Unifier innerState = makeChildUnifier(); - innerState.tryUnify_(type, subTy, isFunctionCall); + innerState.tryUnify_(subTy, type, isFunctionCall); if (innerState.errors.empty()) { found = true; - log.concat(std::move(innerState.log)); + if (FFlag::LuauUseCommittingTxnLog) + log.concat(std::move(innerState.log)); + else + DEPRECATED_log.concat(std::move(innerState.DEPRECATED_log)); + break; } else if (auto e = hasUnificationTooComplex(innerState.errors)) @@ -522,7 +651,8 @@ void Unifier::tryUnify_(TypeId superTy, TypeId subTy, bool isFunctionCall, bool failedOption = {innerState.errors.front()}; } - innerState.log.rollback(); + if (!FFlag::LuauUseCommittingTxnLog) + innerState.DEPRECATED_log.rollback(); } if (unificationTooComplex) @@ -538,7 +668,8 @@ void Unifier::tryUnify_(TypeId superTy, TypeId subTy, bool isFunctionCall, bool errors.push_back(TypeError{location, TypeMismatch{superTy, subTy, "none of the union options are compatible"}}); } } - else if (const IntersectionTypeVar* uv = get(superTy)) + else if (const IntersectionTypeVar* uv = + FFlag::LuauUseCommittingTxnLog ? log.getMutable(superTy) : get(superTy)) { std::optional unificationTooComplex; std::optional firstFailedOption; @@ -547,7 +678,7 @@ void Unifier::tryUnify_(TypeId superTy, TypeId subTy, bool isFunctionCall, bool for (TypeId type : uv->parts) { Unifier innerState = makeChildUnifier(); - innerState.tryUnify_(type, subTy, /*isFunctionCall*/ false, /*isIntersection*/ true); + innerState.tryUnify_(subTy, type, /*isFunctionCall*/ false, /*isIntersection*/ true); if (auto e = hasUnificationTooComplex(innerState.errors)) unificationTooComplex = e; @@ -557,7 +688,10 @@ void Unifier::tryUnify_(TypeId superTy, TypeId subTy, bool isFunctionCall, bool firstFailedOption = {innerState.errors.front()}; } - log.concat(std::move(innerState.log)); + if (FFlag::LuauUseCommittingTxnLog) + log.concat(std::move(innerState.log)); + else + DEPRECATED_log.concat(std::move(innerState.DEPRECATED_log)); } if (unificationTooComplex) @@ -565,7 +699,8 @@ void Unifier::tryUnify_(TypeId superTy, TypeId subTy, bool isFunctionCall, bool else if (firstFailedOption) errors.push_back(TypeError{location, TypeMismatch{superTy, subTy, "Not all intersection parts are compatible.", *firstFailedOption}}); } - else if (const IntersectionTypeVar* uv = get(subTy)) + else if (const IntersectionTypeVar* uv = + FFlag::LuauUseCommittingTxnLog ? log.getMutable(subTy) : get(subTy)) { // A & B <: T if T <: A or T <: B bool found = false; @@ -591,12 +726,15 @@ void Unifier::tryUnify_(TypeId superTy, TypeId subTy, bool isFunctionCall, bool { TypeId type = uv->parts[(i + startIndex) % uv->parts.size()]; Unifier innerState = makeChildUnifier(); - innerState.tryUnify_(superTy, type, isFunctionCall); + innerState.tryUnify_(type, superTy, isFunctionCall); if (innerState.errors.empty()) { found = true; - log.concat(std::move(innerState.log)); + if (FFlag::LuauUseCommittingTxnLog) + log.concat(std::move(innerState.log)); + else + DEPRECATED_log.concat(std::move(innerState.DEPRECATED_log)); break; } else if (auto e = hasUnificationTooComplex(innerState.errors)) @@ -604,7 +742,8 @@ void Unifier::tryUnify_(TypeId superTy, TypeId subTy, bool isFunctionCall, bool unificationTooComplex = e; } - innerState.log.rollback(); + if (!FFlag::LuauUseCommittingTxnLog) + innerState.DEPRECATED_log.rollback(); } if (unificationTooComplex) @@ -614,44 +753,56 @@ void Unifier::tryUnify_(TypeId superTy, TypeId subTy, bool isFunctionCall, bool errors.push_back(TypeError{location, TypeMismatch{superTy, subTy, "none of the intersection parts are compatible"}}); } } - else if (get(superTy) && get(subTy)) - tryUnifyPrimitives(superTy, subTy); + else if ((FFlag::LuauUseCommittingTxnLog && log.getMutable(superTy) && log.getMutable(subTy)) || + (!FFlag::LuauUseCommittingTxnLog && get(superTy) && get(subTy))) + tryUnifyPrimitives(subTy, superTy); - else if (FFlag::LuauSingletonTypes && (get(superTy) || get(superTy)) && get(subTy)) - tryUnifySingletons(superTy, subTy); + else if (FFlag::LuauSingletonTypes && + ((FFlag::LuauUseCommittingTxnLog ? log.getMutable(superTy) : get(superTy)) || + (FFlag::LuauUseCommittingTxnLog ? log.getMutable(superTy) : get(superTy))) && + (FFlag::LuauUseCommittingTxnLog ? log.getMutable(subTy) : get(subTy))) + tryUnifySingletons(subTy, superTy); - else if (get(superTy) && get(subTy)) - tryUnifyFunctions(superTy, subTy, isFunctionCall); + else if ((FFlag::LuauUseCommittingTxnLog && log.getMutable(superTy) && log.getMutable(subTy)) || + (!FFlag::LuauUseCommittingTxnLog && get(superTy) && get(subTy))) + tryUnifyFunctions(subTy, superTy, isFunctionCall); - else if (get(superTy) && get(subTy)) + else if ((FFlag::LuauUseCommittingTxnLog && log.getMutable(superTy) && log.getMutable(subTy)) || + (!FFlag::LuauUseCommittingTxnLog && get(superTy) && get(subTy))) { - tryUnifyTables(superTy, subTy, isIntersection); + tryUnifyTables(subTy, superTy, isIntersection); if (cacheEnabled && errors.empty()) - cacheResult(superTy, subTy); + cacheResult(subTy, superTy); } // tryUnifyWithMetatable assumes its first argument is a MetatableTypeVar. The check is otherwise symmetrical. - else if (get(superTy)) - tryUnifyWithMetatable(superTy, subTy, /*reversed*/ false); - else if (get(subTy)) - tryUnifyWithMetatable(subTy, superTy, /*reversed*/ true); + else if ((FFlag::LuauUseCommittingTxnLog && log.getMutable(superTy)) || + (!FFlag::LuauUseCommittingTxnLog && get(superTy))) + tryUnifyWithMetatable(subTy, superTy, /*reversed*/ false); + else if ((FFlag::LuauUseCommittingTxnLog && log.getMutable(subTy)) || + (!FFlag::LuauUseCommittingTxnLog && get(subTy))) + tryUnifyWithMetatable(superTy, subTy, /*reversed*/ true); - else if (get(superTy)) - tryUnifyWithClass(superTy, subTy, /*reversed*/ false); + else if ((FFlag::LuauUseCommittingTxnLog && log.getMutable(superTy)) || + (!FFlag::LuauUseCommittingTxnLog && get(superTy))) + tryUnifyWithClass(subTy, superTy, /*reversed*/ false); // Unification of nonclasses with classes is almost, but not quite symmetrical. // The order in which we perform this test is significant in the case that both types are classes. - else if (get(subTy)) - tryUnifyWithClass(superTy, subTy, /*reversed*/ true); + else if ((FFlag::LuauUseCommittingTxnLog && log.getMutable(subTy)) || (!FFlag::LuauUseCommittingTxnLog && get(subTy))) + tryUnifyWithClass(subTy, superTy, /*reversed*/ true); else errors.push_back(TypeError{location, TypeMismatch{superTy, subTy}}); - log.popSeen(superTy, subTy); + if (FFlag::LuauUseCommittingTxnLog) + log.popSeen(superTy, subTy); + else + DEPRECATED_log.popSeen(superTy, subTy); } -void Unifier::cacheResult(TypeId superTy, TypeId subTy) +void Unifier::cacheResult(TypeId subTy, TypeId superTy) { bool* superTyInfo = sharedState.skipCacheForType.find(superTy); @@ -684,7 +835,7 @@ void Unifier::cacheResult(TypeId superTy, TypeId subTy) sharedState.cachedUnify.insert({subTy, superTy}); } -struct WeirdIter +struct DEPRECATED_WeirdIter { TypePackId packId; const TypePack* pack; @@ -692,7 +843,7 @@ struct WeirdIter bool growing; TypeLevel level; - WeirdIter(TypePackId packId) + DEPRECATED_WeirdIter(TypePackId packId) : packId(packId) , pack(get(packId)) , index(0) @@ -705,7 +856,7 @@ struct WeirdIter } } - WeirdIter(const WeirdIter&) = default; + DEPRECATED_WeirdIter(const DEPRECATED_WeirdIter&) = default; const TypeId& operator*() { @@ -756,34 +907,152 @@ struct WeirdIter } }; -ErrorVec Unifier::canUnify(TypeId superTy, TypeId subTy) +struct WeirdIter +{ + TypePackId packId; + TxnLog& log; + TypePack* pack; + size_t index; + bool growing; + TypeLevel level; + + WeirdIter(TypePackId packId, TxnLog& log) + : packId(packId) + , log(log) + , pack(log.getMutable(packId)) + , index(0) + , growing(false) + { + while (pack && pack->head.empty() && pack->tail) + { + packId = *pack->tail; + pack = log.getMutable(packId); + } + } + + WeirdIter(const WeirdIter&) = default; + + TypeId& operator*() + { + LUAU_ASSERT(good()); + return pack->head[index]; + } + + bool good() const + { + return pack != nullptr && index < pack->head.size(); + } + + bool advance() + { + if (!pack) + return good(); + + if (index < pack->head.size()) + ++index; + + if (growing || index < pack->head.size()) + return good(); + + if (pack->tail) + { + packId = log.follow(*pack->tail); + pack = log.getMutable(packId); + index = 0; + } + + return good(); + } + + bool canGrow() const + { + return nullptr != log.getMutable(packId); + } + + void grow(TypePackId newTail) + { + LUAU_ASSERT(canGrow()); + LUAU_ASSERT(log.getMutable(newTail)); + + level = log.getMutable(packId)->level; + log.replace(packId, Unifiable::Bound(newTail)); + packId = newTail; + pack = log.getMutable(newTail); + index = 0; + growing = true; + } + + void pushType(TypeId ty) + { + LUAU_ASSERT(pack); + PendingTypePack* pendingPack = log.queue(packId); + if (TypePack* pending = getMutable(pendingPack)) + { + pending->head.push_back(ty); + // We've potentially just replaced the TypePack* that we need to look + // in. We need to replace pack. + pack = pending; + } + else + { + LUAU_ASSERT(!"Pending state for this pack was not a TypePack"); + } + } +}; + +ErrorVec Unifier::canUnify(TypeId subTy, TypeId superTy) { Unifier s = makeChildUnifier(); - s.tryUnify_(superTy, subTy); - s.log.rollback(); + s.tryUnify_(subTy, superTy); + + if (!FFlag::LuauUseCommittingTxnLog) + s.DEPRECATED_log.rollback(); + return s.errors; } -ErrorVec Unifier::canUnify(TypePackId superTy, TypePackId subTy, bool isFunctionCall) +ErrorVec Unifier::canUnify(TypePackId subTy, TypePackId superTy, bool isFunctionCall) { Unifier s = makeChildUnifier(); - s.tryUnify_(superTy, subTy, isFunctionCall); - s.log.rollback(); + s.tryUnify_(subTy, superTy, isFunctionCall); + + if (!FFlag::LuauUseCommittingTxnLog) + s.DEPRECATED_log.rollback(); + return s.errors; } -void Unifier::tryUnify(TypePackId superTp, TypePackId subTp, bool isFunctionCall) +void Unifier::tryUnify(TypePackId subTp, TypePackId superTp, bool isFunctionCall) { sharedState.counters.iterationCount = 0; - tryUnify_(superTp, subTp, isFunctionCall); + tryUnify_(subTp, superTp, isFunctionCall); +} + +static std::pair, std::optional> logAwareFlatten(TypePackId tp, const TxnLog& log) +{ + tp = log.follow(tp); + + std::vector flattened; + std::optional tail = std::nullopt; + + TypePackIterator it(tp, &log); + + for (; it != end(tp); ++it) + { + flattened.push_back(*it); + } + + tail = it.tail(); + + return {flattened, tail}; } /* * This is quite tricky: we are walking two rope-like structures and unifying corresponding elements. * If one is longer than the other, but the short end is free, we grow it to the required length. */ -void Unifier::tryUnify_(TypePackId superTp, TypePackId subTp, bool isFunctionCall) +void Unifier::tryUnify_(TypePackId subTp, TypePackId superTp, bool isFunctionCall) { RecursionLimiter _ra(&sharedState.counters.recursionCount, FInt::LuauTypeInferRecursionLimit); @@ -795,252 +1064,458 @@ void Unifier::tryUnify_(TypePackId superTp, TypePackId subTp, bool isFunctionCal return; } - superTp = follow(superTp); - subTp = follow(subTp); - - while (auto r = get(subTp)) + if (FFlag::LuauUseCommittingTxnLog) { - if (r->head.empty() && r->tail) - subTp = follow(*r->tail); - else - break; - } + superTp = log.follow(superTp); + subTp = log.follow(subTp); - while (auto l = get(superTp)) - { - if (l->head.empty() && l->tail) - superTp = follow(*l->tail); - else - break; - } - - if (superTp == subTp) - return; - - if (get(superTp)) - { - occursCheck(superTp, subTp); - - // The occurrence check might have caused superTp no longer to be a free type - if (!get(superTp)) + while (auto tp = log.getMutable(subTp)) { - log(superTp); - *asMutable(superTp) = Unifiable::Bound(subTp); - } - } - else if (get(subTp)) - { - occursCheck(subTp, superTp); - - // The occurrence check might have caused superTp no longer to be a free type - if (!get(subTp)) - { - log(subTp); - *asMutable(subTp) = Unifiable::Bound(superTp); - } - } - - else if (get(superTp)) - tryUnifyWithAny(superTp, subTp); - - else if (get(subTp)) - tryUnifyWithAny(subTp, superTp); - - else if (get(superTp)) - tryUnifyVariadics(superTp, subTp, false); - else if (get(subTp)) - tryUnifyVariadics(subTp, superTp, true); - - else if (get(superTp) && get(subTp)) - { - auto l = get(superTp); - auto r = get(subTp); - - // If the size of two heads does not match, but both packs have free tail - // We set the sentinel variable to say so to avoid growing it forever. - auto [superTypes, superTail] = flatten(superTp); - auto [subTypes, subTail] = flatten(subTp); - - bool noInfiniteGrowth = - (superTypes.size() != subTypes.size()) && (superTail && get(*superTail)) && (subTail && get(*subTail)); - - auto superIter = WeirdIter{superTp}; - auto subIter = WeirdIter{subTp}; - - auto mkFreshType = [this](TypeLevel level) { - return types->freshType(level); - }; - - const TypePackId emptyTp = types->addTypePack(TypePack{{}, std::nullopt}); - - int loopCount = 0; - - do - { - if (FInt::LuauTypeInferTypePackLoopLimit > 0 && loopCount >= FInt::LuauTypeInferTypePackLoopLimit) - ice("Detected possibly infinite TypePack growth"); - - ++loopCount; - - if (superIter.good() && subIter.growing) - asMutable(subIter.pack)->head.push_back(mkFreshType(subIter.level)); - - if (subIter.good() && superIter.growing) - asMutable(superIter.pack)->head.push_back(mkFreshType(superIter.level)); - - if (superIter.good() && subIter.good()) - { - tryUnify_(*superIter, *subIter); - - if (FFlag::LuauExtendedFunctionMismatchError && !errors.empty() && !firstPackErrorPos) - firstPackErrorPos = loopCount; - - superIter.advance(); - subIter.advance(); - continue; - } - - // If both are at the end, we're done - if (!superIter.good() && !subIter.good()) - { - const bool lFreeTail = l->tail && get(follow(*l->tail)) != nullptr; - const bool rFreeTail = r->tail && get(follow(*r->tail)) != nullptr; - if (lFreeTail && rFreeTail) - tryUnify_(*l->tail, *r->tail); - else if (lFreeTail) - tryUnify_(*l->tail, emptyTp); - else if (rFreeTail) - tryUnify_(*r->tail, emptyTp); - - break; - } - - // If both tails are free, bind one to the other and call it a day - if (superIter.canGrow() && subIter.canGrow()) - return tryUnify_(*superIter.pack->tail, *subIter.pack->tail); - - // If just one side is free on its tail, grow it to fit the other side. - // FIXME: The tail-most tail of the growing pack should be the same as the tail-most tail of the non-growing pack. - if (superIter.canGrow()) - superIter.grow(types->addTypePack(TypePackVar(TypePack{}))); - - else if (subIter.canGrow()) - subIter.grow(types->addTypePack(TypePackVar(TypePack{}))); - + if (tp->head.empty() && tp->tail) + subTp = log.follow(*tp->tail); else + break; + } + + while (auto tp = log.getMutable(superTp)) + { + if (tp->head.empty() && tp->tail) + superTp = log.follow(*tp->tail); + else + break; + } + + if (superTp == subTp) + return; + + if (log.getMutable(superTp)) + { + occursCheck(superTp, subTp); + + if (!log.getMutable(superTp)) { - // A union type including nil marks an optional argument - if (superIter.good() && isOptional(*superIter)) - { - superIter.advance(); - continue; - } - else if (subIter.good() && isOptional(*subIter)) - { - subIter.advance(); - continue; - } - - // In nonstrict mode, any also marks an optional argument. - else if (superIter.good() && isNonstrictMode() && get(follow(*superIter))) - { - superIter.advance(); - continue; - } - - if (get(superIter.packId)) - { - tryUnifyVariadics(superIter.packId, subIter.packId, false, int(subIter.index)); - return; - } - - if (get(subIter.packId)) - { - tryUnifyVariadics(subIter.packId, superIter.packId, true, int(superIter.index)); - return; - } - - if (!isFunctionCall && subIter.good()) - { - // Sometimes it is ok to pass too many arguments - return; - } - - // This is a bit weird because we don't actually know expected vs actual. We just know - // subtype vs supertype. If we are checking the values returned by a function, we swap - // these to produce the expected error message. - size_t expectedSize = size(superTp); - size_t actualSize = size(subTp); - if (ctx == CountMismatch::Result) - std::swap(expectedSize, actualSize); - errors.push_back(TypeError{location, CountMismatch{expectedSize, actualSize, ctx}}); - - while (superIter.good()) - { - tryUnify_(getSingletonTypes().errorRecoveryType(), *superIter); - superIter.advance(); - } - - while (subIter.good()) - { - tryUnify_(getSingletonTypes().errorRecoveryType(), *subIter); - subIter.advance(); - } - - return; + log.replace(superTp, Unifiable::Bound(subTp)); } + } + else if (log.getMutable(subTp)) + { + occursCheck(subTp, superTp); - } while (!noInfiniteGrowth); + if (!log.getMutable(subTp)) + { + log.replace(subTp, Unifiable::Bound(superTp)); + } + } + else if (log.getMutable(superTp)) + tryUnifyWithAny(subTp, superTp); + else if (log.getMutable(subTp)) + tryUnifyWithAny(superTp, subTp); + else if (log.getMutable(superTp)) + tryUnifyVariadics(subTp, superTp, false); + else if (log.getMutable(subTp)) + tryUnifyVariadics(superTp, subTp, true); + else if (log.getMutable(superTp) && log.getMutable(subTp)) + { + auto superTpv = log.getMutable(superTp); + auto subTpv = log.getMutable(subTp); + + // If the size of two heads does not match, but both packs have free tail + // We set the sentinel variable to say so to avoid growing it forever. + auto [superTypes, superTail] = logAwareFlatten(superTp, log); + auto [subTypes, subTail] = logAwareFlatten(subTp, log); + + bool noInfiniteGrowth = + (superTypes.size() != subTypes.size()) && (superTail && get(*superTail)) && (subTail && get(*subTail)); + + auto superIter = WeirdIter(superTp, log); + auto subIter = WeirdIter(subTp, log); + + auto mkFreshType = [this](TypeLevel level) { + return types->freshType(level); + }; + + const TypePackId emptyTp = types->addTypePack(TypePack{{}, std::nullopt}); + + int loopCount = 0; + + do + { + if (FInt::LuauTypeInferTypePackLoopLimit > 0 && loopCount >= FInt::LuauTypeInferTypePackLoopLimit) + ice("Detected possibly infinite TypePack growth"); + + ++loopCount; + + if (superIter.good() && subIter.growing) + { + subIter.pushType(mkFreshType(subIter.level)); + } + + if (subIter.good() && superIter.growing) + { + superIter.pushType(mkFreshType(superIter.level)); + } + + if (superIter.good() && subIter.good()) + { + tryUnify_(*subIter, *superIter); + + if (FFlag::LuauExtendedFunctionMismatchError && !errors.empty() && !firstPackErrorPos) + firstPackErrorPos = loopCount; + + superIter.advance(); + subIter.advance(); + continue; + } + + // If both are at the end, we're done + if (!superIter.good() && !subIter.good()) + { + const bool lFreeTail = superTpv->tail && log.getMutable(log.follow(*superTpv->tail)) != nullptr; + const bool rFreeTail = subTpv->tail && log.getMutable(log.follow(*subTpv->tail)) != nullptr; + if (lFreeTail && rFreeTail) + tryUnify_(*subTpv->tail, *superTpv->tail); + else if (lFreeTail) + tryUnify_(emptyTp, *superTpv->tail); + else if (rFreeTail) + tryUnify_(emptyTp, *subTpv->tail); + + break; + } + + // If both tails are free, bind one to the other and call it a day + if (superIter.canGrow() && subIter.canGrow()) + return tryUnify_(*subIter.pack->tail, *superIter.pack->tail); + + // If just one side is free on its tail, grow it to fit the other side. + // FIXME: The tail-most tail of the growing pack should be the same as the tail-most tail of the non-growing pack. + if (superIter.canGrow()) + superIter.grow(types->addTypePack(TypePackVar(TypePack{}))); + else if (subIter.canGrow()) + subIter.grow(types->addTypePack(TypePackVar(TypePack{}))); + else + { + // A union type including nil marks an optional argument + if (superIter.good() && isOptional(*superIter)) + { + superIter.advance(); + continue; + } + else if (subIter.good() && isOptional(*subIter)) + { + subIter.advance(); + continue; + } + + // In nonstrict mode, any also marks an optional argument. + else if (superIter.good() && isNonstrictMode() && log.getMutable(log.follow(*superIter))) + { + superIter.advance(); + continue; + } + + if (log.getMutable(superIter.packId)) + { + tryUnifyVariadics(subIter.packId, superIter.packId, false, int(subIter.index)); + return; + } + + if (log.getMutable(subIter.packId)) + { + tryUnifyVariadics(superIter.packId, subIter.packId, true, int(superIter.index)); + return; + } + + if (!isFunctionCall && subIter.good()) + { + // Sometimes it is ok to pass too many arguments + return; + } + + // This is a bit weird because we don't actually know expected vs actual. We just know + // subtype vs supertype. If we are checking the values returned by a function, we swap + // these to produce the expected error message. + size_t expectedSize = size(superTp); + size_t actualSize = size(subTp); + if (ctx == CountMismatch::Result) + std::swap(expectedSize, actualSize); + errors.push_back(TypeError{location, CountMismatch{expectedSize, actualSize, ctx}}); + + while (superIter.good()) + { + tryUnify_(*superIter, getSingletonTypes().errorRecoveryType()); + superIter.advance(); + } + + while (subIter.good()) + { + tryUnify_(*subIter, getSingletonTypes().errorRecoveryType()); + subIter.advance(); + } + + return; + } + + } while (!noInfiniteGrowth); + } + else + { + errors.push_back(TypeError{location, GenericError{"Failed to unify type packs"}}); + } } else { - errors.push_back(TypeError{location, GenericError{"Failed to unify type packs"}}); + superTp = follow(superTp); + subTp = follow(subTp); + + while (auto tp = get(subTp)) + { + if (tp->head.empty() && tp->tail) + subTp = follow(*tp->tail); + else + break; + } + + while (auto tp = get(superTp)) + { + if (tp->head.empty() && tp->tail) + superTp = follow(*tp->tail); + else + break; + } + + if (superTp == subTp) + return; + + if (get(superTp)) + { + occursCheck(superTp, subTp); + + if (!get(superTp)) + { + DEPRECATED_log(superTp); + *asMutable(superTp) = Unifiable::Bound(subTp); + } + } + else if (get(subTp)) + { + occursCheck(subTp, superTp); + + if (!get(subTp)) + { + DEPRECATED_log(subTp); + *asMutable(subTp) = Unifiable::Bound(superTp); + } + } + + else if (get(superTp)) + tryUnifyWithAny(subTp, superTp); + + else if (get(subTp)) + tryUnifyWithAny(superTp, subTp); + + else if (get(superTp)) + tryUnifyVariadics(subTp, superTp, false); + else if (get(subTp)) + tryUnifyVariadics(superTp, subTp, true); + + else if (get(superTp) && get(subTp)) + { + auto superTpv = get(superTp); + auto subTpv = get(subTp); + + // If the size of two heads does not match, but both packs have free tail + // We set the sentinel variable to say so to avoid growing it forever. + auto [superTypes, superTail] = flatten(superTp); + auto [subTypes, subTail] = flatten(subTp); + + bool noInfiniteGrowth = + (superTypes.size() != subTypes.size()) && (superTail && get(*superTail)) && (subTail && get(*subTail)); + + auto superIter = DEPRECATED_WeirdIter{superTp}; + auto subIter = DEPRECATED_WeirdIter{subTp}; + + auto mkFreshType = [this](TypeLevel level) { + return types->freshType(level); + }; + + const TypePackId emptyTp = types->addTypePack(TypePack{{}, std::nullopt}); + + int loopCount = 0; + + do + { + if (FInt::LuauTypeInferTypePackLoopLimit > 0 && loopCount >= FInt::LuauTypeInferTypePackLoopLimit) + ice("Detected possibly infinite TypePack growth"); + + ++loopCount; + + if (superIter.good() && subIter.growing) + asMutable(subIter.pack)->head.push_back(mkFreshType(subIter.level)); + + if (subIter.good() && superIter.growing) + asMutable(superIter.pack)->head.push_back(mkFreshType(superIter.level)); + + if (superIter.good() && subIter.good()) + { + tryUnify_(*subIter, *superIter); + + if (FFlag::LuauExtendedFunctionMismatchError && !errors.empty() && !firstPackErrorPos) + firstPackErrorPos = loopCount; + + superIter.advance(); + subIter.advance(); + continue; + } + + // If both are at the end, we're done + if (!superIter.good() && !subIter.good()) + { + const bool lFreeTail = superTpv->tail && get(follow(*superTpv->tail)) != nullptr; + const bool rFreeTail = subTpv->tail && get(follow(*subTpv->tail)) != nullptr; + if (lFreeTail && rFreeTail) + tryUnify_(*subTpv->tail, *superTpv->tail); + else if (lFreeTail) + tryUnify_(emptyTp, *superTpv->tail); + else if (rFreeTail) + tryUnify_(emptyTp, *subTpv->tail); + + break; + } + + // If both tails are free, bind one to the other and call it a day + if (superIter.canGrow() && subIter.canGrow()) + return tryUnify_(*subIter.pack->tail, *superIter.pack->tail); + + // If just one side is free on its tail, grow it to fit the other side. + // FIXME: The tail-most tail of the growing pack should be the same as the tail-most tail of the non-growing pack. + if (superIter.canGrow()) + superIter.grow(types->addTypePack(TypePackVar(TypePack{}))); + + else if (subIter.canGrow()) + subIter.grow(types->addTypePack(TypePackVar(TypePack{}))); + + else + { + // A union type including nil marks an optional argument + if (superIter.good() && isOptional(*superIter)) + { + superIter.advance(); + continue; + } + else if (subIter.good() && isOptional(*subIter)) + { + subIter.advance(); + continue; + } + + // In nonstrict mode, any also marks an optional argument. + else if (superIter.good() && isNonstrictMode() && get(follow(*superIter))) + { + superIter.advance(); + continue; + } + + if (get(superIter.packId)) + { + tryUnifyVariadics(subIter.packId, superIter.packId, false, int(subIter.index)); + return; + } + + if (get(subIter.packId)) + { + tryUnifyVariadics(superIter.packId, subIter.packId, true, int(superIter.index)); + return; + } + + if (!isFunctionCall && subIter.good()) + { + // Sometimes it is ok to pass too many arguments + return; + } + + // This is a bit weird because we don't actually know expected vs actual. We just know + // subtype vs supertype. If we are checking the values returned by a function, we swap + // these to produce the expected error message. + size_t expectedSize = size(superTp); + size_t actualSize = size(subTp); + if (ctx == CountMismatch::Result) + std::swap(expectedSize, actualSize); + errors.push_back(TypeError{location, CountMismatch{expectedSize, actualSize, ctx}}); + + while (superIter.good()) + { + tryUnify_(*superIter, getSingletonTypes().errorRecoveryType()); + superIter.advance(); + } + + while (subIter.good()) + { + tryUnify_(*subIter, getSingletonTypes().errorRecoveryType()); + subIter.advance(); + } + + return; + } + + } while (!noInfiniteGrowth); + } + else + { + errors.push_back(TypeError{location, GenericError{"Failed to unify type packs"}}); + } } } -void Unifier::tryUnifyPrimitives(TypeId superTy, TypeId subTy) +void Unifier::tryUnifyPrimitives(TypeId subTy, TypeId superTy) { - const PrimitiveTypeVar* lp = get(superTy); - const PrimitiveTypeVar* rp = get(subTy); - if (!lp || !rp) + const PrimitiveTypeVar* superPrim = get(superTy); + const PrimitiveTypeVar* subPrim = get(subTy); + if (!superPrim || !subPrim) ice("passed non primitive types to unifyPrimitives"); - if (lp->type != rp->type) + if (superPrim->type != subPrim->type) errors.push_back(TypeError{location, TypeMismatch{superTy, subTy}}); } -void Unifier::tryUnifySingletons(TypeId superTy, TypeId subTy) +void Unifier::tryUnifySingletons(TypeId subTy, TypeId superTy) { - const PrimitiveTypeVar* lp = get(superTy); - const SingletonTypeVar* ls = get(superTy); - const SingletonTypeVar* rs = get(subTy); + const PrimitiveTypeVar* superPrim = get(superTy); + const SingletonTypeVar* superSingleton = get(superTy); + const SingletonTypeVar* subSingleton = get(subTy); - if ((!lp && !ls) || !rs) + if ((!superPrim && !superSingleton) || !subSingleton) ice("passed non singleton/primitive types to unifySingletons"); - if (ls && *ls == *rs) + if (superSingleton && *superSingleton == *subSingleton) return; - if (lp && lp->type == PrimitiveTypeVar::Boolean && get(rs) && variance == Covariant) + if (superPrim && superPrim->type == PrimitiveTypeVar::Boolean && get(subSingleton) && variance == Covariant) return; - if (lp && lp->type == PrimitiveTypeVar::String && get(rs) && variance == Covariant) + if (superPrim && superPrim->type == PrimitiveTypeVar::String && get(subSingleton) && variance == Covariant) return; errors.push_back(TypeError{location, TypeMismatch{superTy, subTy}}); } -void Unifier::tryUnifyFunctions(TypeId superTy, TypeId subTy, bool isFunctionCall) +void Unifier::tryUnifyFunctions(TypeId subTy, TypeId superTy, bool isFunctionCall) { - FunctionTypeVar* lf = getMutable(superTy); - FunctionTypeVar* rf = getMutable(subTy); - if (!lf || !rf) + FunctionTypeVar* superFunction = getMutable(superTy); + FunctionTypeVar* subFunction = getMutable(subTy); + + if (FFlag::LuauUseCommittingTxnLog) + { + superFunction = log.getMutable(superTy); + subFunction = log.getMutable(subTy); + } + + if (!superFunction || !subFunction) ice("passed non-function types to unifyFunction"); - size_t numGenerics = lf->generics.size(); - if (numGenerics != rf->generics.size()) + size_t numGenerics = superFunction->generics.size(); + if (numGenerics != subFunction->generics.size()) { - numGenerics = std::min(lf->generics.size(), rf->generics.size()); + numGenerics = std::min(superFunction->generics.size(), subFunction->generics.size()); if (FFlag::LuauExtendedFunctionMismatchError) errors.push_back(TypeError{location, TypeMismatch{superTy, subTy, "different number of generic type parameters"}}); @@ -1048,10 +1523,10 @@ void Unifier::tryUnifyFunctions(TypeId superTy, TypeId subTy, bool isFunctionCal errors.push_back(TypeError{location, TypeMismatch{superTy, subTy}}); } - size_t numGenericPacks = lf->genericPacks.size(); - if (numGenericPacks != rf->genericPacks.size()) + size_t numGenericPacks = superFunction->genericPacks.size(); + if (numGenericPacks != subFunction->genericPacks.size()) { - numGenericPacks = std::min(lf->genericPacks.size(), rf->genericPacks.size()); + numGenericPacks = std::min(superFunction->genericPacks.size(), subFunction->genericPacks.size()); if (FFlag::LuauExtendedFunctionMismatchError) errors.push_back(TypeError{location, TypeMismatch{superTy, subTy, "different number of generic type pack parameters"}}); @@ -1060,7 +1535,12 @@ void Unifier::tryUnifyFunctions(TypeId superTy, TypeId subTy, bool isFunctionCal } for (size_t i = 0; i < numGenerics; i++) - log.pushSeen(lf->generics[i], rf->generics[i]); + { + if (FFlag::LuauUseCommittingTxnLog) + log.pushSeen(superFunction->generics[i], subFunction->generics[i]); + else + DEPRECATED_log.pushSeen(superFunction->generics[i], subFunction->generics[i]); + } CountMismatch::Context context = ctx; @@ -1071,7 +1551,7 @@ void Unifier::tryUnifyFunctions(TypeId superTy, TypeId subTy, bool isFunctionCal if (FFlag::LuauExtendedFunctionMismatchError) { innerState.ctx = CountMismatch::Arg; - innerState.tryUnify_(rf->argTypes, lf->argTypes, isFunctionCall); + innerState.tryUnify_(superFunction->argTypes, subFunction->argTypes, isFunctionCall); bool reported = !innerState.errors.empty(); @@ -1085,13 +1565,13 @@ void Unifier::tryUnifyFunctions(TypeId superTy, TypeId subTy, bool isFunctionCal errors.push_back(TypeError{location, TypeMismatch{superTy, subTy, "", innerState.errors.front()}}); innerState.ctx = CountMismatch::Result; - innerState.tryUnify_(lf->retType, rf->retType); + innerState.tryUnify_(subFunction->retType, superFunction->retType); if (!reported) { if (auto e = hasUnificationTooComplex(innerState.errors)) errors.push_back(*e); - else if (!innerState.errors.empty() && size(lf->retType) == 1 && finite(lf->retType)) + else if (!innerState.errors.empty() && size(superFunction->retType) == 1 && finite(superFunction->retType)) errors.push_back(TypeError{location, TypeMismatch{superTy, subTy, "Return type is not compatible.", innerState.errors.front()}}); else if (!innerState.errors.empty() && innerState.firstPackErrorPos) errors.push_back( @@ -1104,38 +1584,70 @@ void Unifier::tryUnifyFunctions(TypeId superTy, TypeId subTy, bool isFunctionCal else { ctx = CountMismatch::Arg; - innerState.tryUnify_(rf->argTypes, lf->argTypes, isFunctionCall); + innerState.tryUnify_(superFunction->argTypes, subFunction->argTypes, isFunctionCall); ctx = CountMismatch::Result; - innerState.tryUnify_(lf->retType, rf->retType); + innerState.tryUnify_(subFunction->retType, superFunction->retType); checkChildUnifierTypeMismatch(innerState.errors, superTy, subTy); } - log.concat(std::move(innerState.log)); + if (FFlag::LuauUseCommittingTxnLog) + { + log.concat(std::move(innerState.log)); + } + else + { + DEPRECATED_log.concat(std::move(innerState.DEPRECATED_log)); + } } else { ctx = CountMismatch::Arg; - tryUnify_(rf->argTypes, lf->argTypes, isFunctionCall); + tryUnify_(superFunction->argTypes, subFunction->argTypes, isFunctionCall); ctx = CountMismatch::Result; - tryUnify_(lf->retType, rf->retType); + tryUnify_(subFunction->retType, superFunction->retType); } - if (lf->definition && !rf->definition && !subTy->persistent) + if (FFlag::LuauUseCommittingTxnLog) { - rf->definition = lf->definition; + if (superFunction->definition && !subFunction->definition && !subTy->persistent) + { + PendingType* newSubTy = log.queue(subTy); + FunctionTypeVar* newSubFtv = getMutable(newSubTy); + LUAU_ASSERT(newSubFtv); + newSubFtv->definition = superFunction->definition; + } + else if (!superFunction->definition && subFunction->definition && !superTy->persistent) + { + PendingType* newSuperTy = log.queue(superTy); + FunctionTypeVar* newSuperFtv = getMutable(newSuperTy); + LUAU_ASSERT(newSuperFtv); + newSuperFtv->definition = subFunction->definition; + } } - else if (!lf->definition && rf->definition && !superTy->persistent) + else { - lf->definition = rf->definition; + if (superFunction->definition && !subFunction->definition && !subTy->persistent) + { + subFunction->definition = superFunction->definition; + } + else if (!superFunction->definition && subFunction->definition && !superTy->persistent) + { + superFunction->definition = subFunction->definition; + } } ctx = context; for (int i = int(numGenerics) - 1; 0 <= i; i--) - log.popSeen(lf->generics[i], rf->generics[i]); + { + if (FFlag::LuauUseCommittingTxnLog) + log.popSeen(superFunction->generics[i], subFunction->generics[i]); + else + DEPRECATED_log.popSeen(superFunction->generics[i], subFunction->generics[i]); + } } namespace @@ -1160,77 +1672,84 @@ struct Resetter } // namespace -void Unifier::tryUnifyTables(TypeId left, TypeId right, bool isIntersection) +void Unifier::tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) { if (!FFlag::LuauTableSubtypingVariance2) - return DEPRECATED_tryUnifyTables(left, right, isIntersection); + return DEPRECATED_tryUnifyTables(subTy, superTy, isIntersection); - TableTypeVar* lt = getMutable(left); - TableTypeVar* rt = getMutable(right); - if (!lt || !rt) + TableTypeVar* superTable = getMutable(superTy); + TableTypeVar* subTable = getMutable(subTy); + if (!superTable || !subTable) ice("passed non-table types to unifyTables"); std::vector missingProperties; std::vector extraProperties; // Optimization: First test that the property sets are compatible without doing any recursive unification - if (FFlag::LuauTableUnificationEarlyTest && !rt->indexer && rt->state != TableState::Free) + if (FFlag::LuauTableUnificationEarlyTest && !subTable->indexer && subTable->state != TableState::Free) { - for (const auto& [propName, superProp] : lt->props) + for (const auto& [propName, superProp] : superTable->props) { - auto subIter = rt->props.find(propName); - if (subIter == rt->props.end() && !isOptional(superProp.type) && !get(follow(superProp.type))) + auto subIter = subTable->props.find(propName); + if (subIter == subTable->props.end() && !isOptional(superProp.type) && !get(follow(superProp.type))) missingProperties.push_back(propName); } if (!missingProperties.empty()) { - errors.push_back(TypeError{location, MissingProperties{left, right, std::move(missingProperties)}}); + errors.push_back(TypeError{location, MissingProperties{superTy, subTy, std::move(missingProperties)}}); return; } } // And vice versa if we're invariant - if (FFlag::LuauTableUnificationEarlyTest && variance == Invariant && !lt->indexer && lt->state != TableState::Unsealed && - lt->state != TableState::Free) + if (FFlag::LuauTableUnificationEarlyTest && variance == Invariant && !superTable->indexer && superTable->state != TableState::Unsealed && + superTable->state != TableState::Free) { - for (const auto& [propName, subProp] : rt->props) + for (const auto& [propName, subProp] : subTable->props) { - auto superIter = lt->props.find(propName); - if (superIter == lt->props.end() && !isOptional(subProp.type) && !get(follow(subProp.type))) + auto superIter = superTable->props.find(propName); + if (superIter == superTable->props.end() && !isOptional(subProp.type) && !get(follow(subProp.type))) extraProperties.push_back(propName); } if (!extraProperties.empty()) { - errors.push_back(TypeError{location, MissingProperties{left, right, std::move(extraProperties), MissingProperties::Extra}}); + errors.push_back(TypeError{location, MissingProperties{superTy, subTy, std::move(extraProperties), MissingProperties::Extra}}); return; } } - // Reminder: left is the supertype, right is the subtype. // Width subtyping: any property in the supertype must be in the subtype, // and the types must agree. - for (const auto& [name, prop] : lt->props) + for (const auto& [name, prop] : superTable->props) { - const auto& r = rt->props.find(name); - if (r != rt->props.end()) + const auto& r = subTable->props.find(name); + if (r != subTable->props.end()) { // TODO: read-only properties don't need invariance Resetter resetter{&variance}; variance = Invariant; Unifier innerState = makeChildUnifier(); - innerState.tryUnify_(prop.type, r->second.type); + innerState.tryUnify_(r->second.type, prop.type); - checkChildUnifierTypeMismatch(innerState.errors, name, left, right); + checkChildUnifierTypeMismatch(innerState.errors, name, superTy, subTy); - if (innerState.errors.empty()) - log.concat(std::move(innerState.log)); + if (FFlag::LuauUseCommittingTxnLog) + { + if (innerState.errors.empty()) + log.concat(std::move(innerState.log)); + } else - innerState.log.rollback(); + { + if (innerState.errors.empty()) + DEPRECATED_log.concat(std::move(innerState.DEPRECATED_log)); + else + innerState.DEPRECATED_log.rollback(); + } } - else if (rt->indexer && isString(rt->indexer->indexType)) + else if (subTable->indexer && isString(subTable->indexer->indexType)) { // TODO: read-only indexers don't need invariance // TODO: really we should only allow this if prop.type is optional. @@ -1238,37 +1757,55 @@ void Unifier::tryUnifyTables(TypeId left, TypeId right, bool isIntersection) variance = Invariant; Unifier innerState = makeChildUnifier(); - innerState.tryUnify_(prop.type, rt->indexer->indexResultType); + innerState.tryUnify_(subTable->indexer->indexResultType, prop.type); - checkChildUnifierTypeMismatch(innerState.errors, name, left, right); + checkChildUnifierTypeMismatch(innerState.errors, name, superTy, subTy); - if (innerState.errors.empty()) - log.concat(std::move(innerState.log)); + if (FFlag::LuauUseCommittingTxnLog) + { + if (innerState.errors.empty()) + log.concat(std::move(innerState.log)); + } else - innerState.log.rollback(); + { + if (innerState.errors.empty()) + DEPRECATED_log.concat(std::move(innerState.DEPRECATED_log)); + else + innerState.DEPRECATED_log.rollback(); + } } else if (isOptional(prop.type) || get(follow(prop.type))) // TODO: this case is unsound, but without it our test suite fails. CLI-46031 // TODO: should isOptional(anyType) be true? { } - else if (rt->state == TableState::Free) + else if (subTable->state == TableState::Free) { - log(rt); - rt->props[name] = prop; + if (FFlag::LuauUseCommittingTxnLog) + { + PendingType* pendingSub = log.queue(subTy); + TableTypeVar* ttv = getMutable(pendingSub); + LUAU_ASSERT(ttv); + ttv->props[name] = prop; + } + else + { + DEPRECATED_log(subTy); + subTable->props[name] = prop; + } } else missingProperties.push_back(name); } - for (const auto& [name, prop] : rt->props) + for (const auto& [name, prop] : subTable->props) { - if (lt->props.count(name)) + if (superTable->props.count(name)) { // If both lt and rt contain the property, then // we're done since we already unified them above } - else if (lt->indexer && isString(lt->indexer->indexType)) + else if (superTable->indexer && isString(superTable->indexer->indexType)) { // TODO: read-only indexers don't need invariance // TODO: really we should only allow this if prop.type is optional. @@ -1276,24 +1813,42 @@ void Unifier::tryUnifyTables(TypeId left, TypeId right, bool isIntersection) variance = Invariant; Unifier innerState = makeChildUnifier(); - innerState.tryUnify_(prop.type, lt->indexer->indexResultType); + innerState.tryUnify_(superTable->indexer->indexResultType, prop.type); - checkChildUnifierTypeMismatch(innerState.errors, name, left, right); + checkChildUnifierTypeMismatch(innerState.errors, name, superTy, subTy); - if (innerState.errors.empty()) - log.concat(std::move(innerState.log)); + if (FFlag::LuauUseCommittingTxnLog) + { + if (innerState.errors.empty()) + log.concat(std::move(innerState.log)); + } else - innerState.log.rollback(); + { + if (innerState.errors.empty()) + DEPRECATED_log.concat(std::move(innerState.DEPRECATED_log)); + else + innerState.DEPRECATED_log.rollback(); + } } - else if (lt->state == TableState::Unsealed) + else if (superTable->state == TableState::Unsealed) { // TODO: this case is unsound when variance is Invariant, but without it lua-apps fails to typecheck. // TODO: file a JIRA // TODO: hopefully readonly/writeonly properties will fix this. Property clone = prop; clone.type = deeplyOptional(clone.type); - log(left); - lt->props[name] = clone; + + if (FFlag::LuauUseCommittingTxnLog) + { + PendingType* pendingSuper = log.queue(superTy); + TableTypeVar* pendingSuperTtv = getMutable(pendingSuper); + pendingSuperTtv->props[name] = clone; + } + else + { + DEPRECATED_log(superTy); + superTable->props[name] = clone; + } } else if (variance == Covariant) { @@ -1303,61 +1858,93 @@ void Unifier::tryUnifyTables(TypeId left, TypeId right, bool isIntersection) // TODO: should isOptional(anyType) be true? { } - else if (lt->state == TableState::Free) + else if (superTable->state == TableState::Free) { - log(left); - lt->props[name] = prop; + if (FFlag::LuauUseCommittingTxnLog) + { + PendingType* pendingSuper = log.queue(superTy); + TableTypeVar* pendingSuperTtv = getMutable(pendingSuper); + pendingSuperTtv->props[name] = prop; + } + else + { + DEPRECATED_log(superTy); + superTable->props[name] = prop; + } } else extraProperties.push_back(name); } // Unify indexers - if (lt->indexer && rt->indexer) + if (superTable->indexer && subTable->indexer) { // TODO: read-only indexers don't need invariance Resetter resetter{&variance}; variance = Invariant; Unifier innerState = makeChildUnifier(); - innerState.tryUnify(*lt->indexer, *rt->indexer); - checkChildUnifierTypeMismatch(innerState.errors, left, right); - if (innerState.errors.empty()) - log.concat(std::move(innerState.log)); + innerState.tryUnifyIndexer(*subTable->indexer, *superTable->indexer); + checkChildUnifierTypeMismatch(innerState.errors, superTy, subTy); + + if (FFlag::LuauUseCommittingTxnLog) + { + if (innerState.errors.empty()) + log.concat(std::move(innerState.log)); + } else - innerState.log.rollback(); + { + if (innerState.errors.empty()) + DEPRECATED_log.concat(std::move(innerState.DEPRECATED_log)); + else + innerState.DEPRECATED_log.rollback(); + } } - else if (lt->indexer) + else if (superTable->indexer) { - if (rt->state == TableState::Unsealed || rt->state == TableState::Free) + if (subTable->state == TableState::Unsealed || subTable->state == TableState::Free) { // passing/assigning a table without an indexer to something that has one // e.g. table.insert(t, 1) where t is a non-sealed table and doesn't have an indexer. // TODO: we only need to do this if the supertype's indexer is read/write // since that can add indexed elements. - log(right); - rt->indexer = lt->indexer; + if (FFlag::LuauUseCommittingTxnLog) + { + log.changeIndexer(subTy, superTable->indexer); + } + else + { + DEPRECATED_log(subTy); + subTable->indexer = superTable->indexer; + } } } - else if (rt->indexer && variance == Invariant) + else if (subTable->indexer && variance == Invariant) { // Symmetric if we are invariant - if (lt->state == TableState::Unsealed || lt->state == TableState::Free) + if (superTable->state == TableState::Unsealed || superTable->state == TableState::Free) { - log(left); - lt->indexer = rt->indexer; + if (FFlag::LuauUseCommittingTxnLog) + { + log.changeIndexer(superTy, subTable->indexer); + } + else + { + DEPRECATED_log(superTy); + superTable->indexer = subTable->indexer; + } } } if (!missingProperties.empty()) { - errors.push_back(TypeError{location, MissingProperties{left, right, std::move(missingProperties)}}); + errors.push_back(TypeError{location, MissingProperties{superTy, subTy, std::move(missingProperties)}}); return; } if (!extraProperties.empty()) { - errors.push_back(TypeError{location, MissingProperties{left, right, std::move(extraProperties), MissingProperties::Extra}}); + errors.push_back(TypeError{location, MissingProperties{superTy, subTy, std::move(extraProperties), MissingProperties::Extra}}); return; } @@ -1369,18 +1956,32 @@ void Unifier::tryUnifyTables(TypeId left, TypeId right, bool isIntersection) * I believe this is guaranteed to terminate eventually because this will * only happen when a free table is bound to another table. */ - if (lt->boundTo || rt->boundTo) - return tryUnify_(left, right); + if (superTable->boundTo || subTable->boundTo) + return tryUnify_(subTy, superTy); - if (lt->state == TableState::Free) + if (superTable->state == TableState::Free) { - log(lt); - lt->boundTo = right; + if (FFlag::LuauUseCommittingTxnLog) + { + log.bindTable(superTy, subTy); + } + else + { + DEPRECATED_log(superTable); + superTable->boundTo = subTy; + } } - else if (rt->state == TableState::Free) + else if (subTable->state == TableState::Free) { - log(rt); - rt->boundTo = left; + if (FFlag::LuauUseCommittingTxnLog) + { + log.bindTable(subTy, superTy); + } + else + { + DEPRECATED_log(subTy); + subTable->boundTo = superTy; + } } } @@ -1406,99 +2007,129 @@ TypeId Unifier::deeplyOptional(TypeId ty, std::unordered_map see return types->addType(UnionTypeVar{{getSingletonTypes().nilType, ty}}); } -void Unifier::DEPRECATED_tryUnifyTables(TypeId left, TypeId right, bool isIntersection) +void Unifier::DEPRECATED_tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) { LUAU_ASSERT(!FFlag::LuauTableSubtypingVariance2); Resetter resetter{&variance}; variance = Invariant; - TableTypeVar* lt = getMutable(left); - TableTypeVar* rt = getMutable(right); - if (!lt || !rt) + TableTypeVar* superTable = getMutable(superTy); + TableTypeVar* subTable = getMutable(subTy); + + if (FFlag::LuauUseCommittingTxnLog) + { + superTable = log.getMutable(superTy); + subTable = log.getMutable(subTy); + } + + if (!superTable || !subTable) ice("passed non-table types to unifyTables"); - if (lt->state == TableState::Sealed && rt->state == TableState::Sealed) - return tryUnifySealedTables(left, right, isIntersection); - else if ((lt->state == TableState::Sealed && rt->state == TableState::Unsealed) || - (lt->state == TableState::Unsealed && rt->state == TableState::Sealed)) - return tryUnifySealedTables(left, right, isIntersection); - else if ((lt->state == TableState::Sealed && rt->state == TableState::Generic) || - (lt->state == TableState::Generic && rt->state == TableState::Sealed)) - errors.push_back(TypeError{location, TypeMismatch{left, right}}); - else if ((lt->state == TableState::Free) != (rt->state == TableState::Free)) // one table is free and the other is not + if (superTable->state == TableState::Sealed && subTable->state == TableState::Sealed) + return tryUnifySealedTables(subTy, superTy, isIntersection); + else if ((superTable->state == TableState::Sealed && subTable->state == TableState::Unsealed) || + (superTable->state == TableState::Unsealed && subTable->state == TableState::Sealed)) + return tryUnifySealedTables(subTy, superTy, isIntersection); + else if ((superTable->state == TableState::Sealed && subTable->state == TableState::Generic) || + (superTable->state == TableState::Generic && subTable->state == TableState::Sealed)) + errors.push_back(TypeError{location, TypeMismatch{superTy, subTy}}); + else if ((superTable->state == TableState::Free) != (subTable->state == TableState::Free)) // one table is free and the other is not { - TypeId freeTypeId = rt->state == TableState::Free ? right : left; - TypeId otherTypeId = rt->state == TableState::Free ? left : right; + TypeId freeTypeId = subTable->state == TableState::Free ? subTy : superTy; + TypeId otherTypeId = subTable->state == TableState::Free ? superTy : subTy; - return tryUnifyFreeTable(freeTypeId, otherTypeId); + return tryUnifyFreeTable(otherTypeId, freeTypeId); } - else if (lt->state == TableState::Free && rt->state == TableState::Free) + else if (superTable->state == TableState::Free && subTable->state == TableState::Free) { - tryUnifyFreeTable(left, right); + tryUnifyFreeTable(subTy, superTy); // avoid creating a cycle when the types are already pointing at each other - if (follow(left) != follow(right)) + if (follow(superTy) != follow(subTy)) { - log(lt); - lt->boundTo = right; + if (FFlag::LuauUseCommittingTxnLog) + { + log.bindTable(superTy, subTy); + } + else + { + DEPRECATED_log(superTable); + superTable->boundTo = subTy; + } } return; } - else if (lt->state != TableState::Sealed && rt->state != TableState::Sealed) + else if (superTable->state != TableState::Sealed && subTable->state != TableState::Sealed) { // All free tables are checked in one of the branches above - LUAU_ASSERT(lt->state != TableState::Free); - LUAU_ASSERT(rt->state != TableState::Free); + LUAU_ASSERT(superTable->state != TableState::Free); + LUAU_ASSERT(subTable->state != TableState::Free); // Tables must have exactly the same props and their types must all unify // I honestly have no idea if this is remotely close to reasonable. - for (const auto& [name, prop] : lt->props) + for (const auto& [name, prop] : superTable->props) { - const auto& r = rt->props.find(name); - if (r == rt->props.end()) - errors.push_back(TypeError{location, UnknownProperty{right, name}}); + const auto& r = subTable->props.find(name); + if (r == subTable->props.end()) + errors.push_back(TypeError{location, UnknownProperty{subTy, name}}); else - tryUnify_(prop.type, r->second.type); + tryUnify_(r->second.type, prop.type); } - if (lt->indexer && rt->indexer) - tryUnify(*lt->indexer, *rt->indexer); - else if (lt->indexer) + if (superTable->indexer && subTable->indexer) + tryUnifyIndexer(*subTable->indexer, *superTable->indexer); + else if (superTable->indexer) { // passing/assigning a table without an indexer to something that has one // e.g. table.insert(t, 1) where t is a non-sealed table and doesn't have an indexer. - if (rt->state == TableState::Unsealed) - rt->indexer = lt->indexer; + if (subTable->state == TableState::Unsealed) + { + if (FFlag::LuauUseCommittingTxnLog) + { + log.changeIndexer(subTy, superTable->indexer); + } + else + { + subTable->indexer = superTable->indexer; + } + } else - errors.push_back(TypeError{location, CannotExtendTable{right, CannotExtendTable::Indexer}}); + errors.push_back(TypeError{location, CannotExtendTable{subTy, CannotExtendTable::Indexer}}); } } - else if (lt->state == TableState::Sealed) + else if (superTable->state == TableState::Sealed) { // lt is sealed and so it must be possible for rt to have precisely the same shape // Verify that this is the case, then bind rt to lt. ice("unsealed tables are not working yet", location); } - else if (rt->state == TableState::Sealed) - return tryUnifyTables(right, left, isIntersection); + else if (subTable->state == TableState::Sealed) + return tryUnifyTables(superTy, subTy, isIntersection); else ice("tryUnifyTables"); } -void Unifier::tryUnifyFreeTable(TypeId freeTypeId, TypeId otherTypeId) +void Unifier::tryUnifyFreeTable(TypeId subTy, TypeId superTy) { - TableTypeVar* freeTable = getMutable(freeTypeId); - TableTypeVar* otherTable = getMutable(otherTypeId); - if (!freeTable || !otherTable) + TableTypeVar* freeTable = getMutable(superTy); + TableTypeVar* subTable = getMutable(subTy); + + if (FFlag::LuauUseCommittingTxnLog) + { + freeTable = log.getMutable(superTy); + subTable = log.getMutable(subTy); + } + + if (!freeTable || !subTable) ice("passed non-table types to tryUnifyFreeTable"); // Any properties in freeTable must unify with those in otherTable. // Then bind freeTable to otherTable. for (const auto& [freeName, freeProp] : freeTable->props) { - if (auto otherProp = findTablePropertyRespectingMeta(otherTypeId, freeName)) + if (auto subProp = findTablePropertyRespectingMeta(subTy, freeName)) { - tryUnify_(*otherProp, freeProp.type); + tryUnify_(freeProp.type, *subProp); /* * TypeVars are commonly cyclic, so it is entirely possible @@ -1508,84 +2139,133 @@ void Unifier::tryUnifyFreeTable(TypeId freeTypeId, TypeId otherTypeId) * I believe this is guaranteed to terminate eventually because this will * only happen when a free table is bound to another table. */ - if (!get(freeTypeId) || !get(otherTypeId)) - return tryUnify_(freeTypeId, otherTypeId); + if (FFlag::LuauUseCommittingTxnLog) + { + if (!log.getMutable(superTy) || !log.getMutable(subTy)) + return tryUnify_(subTy, superTy); - if (freeTable->boundTo) - return tryUnify_(freeTypeId, otherTypeId); + if (TableTypeVar* pendingFreeTtv = log.getMutable(superTy); pendingFreeTtv && pendingFreeTtv->boundTo) + return tryUnify_(subTy, superTy); + } + else + { + if (!get(superTy) || !get(subTy)) + return tryUnify_(subTy, superTy); + + if (freeTable->boundTo) + return tryUnify_(subTy, superTy); + } } else { // If the other table is also free, then we are learning that it has more // properties than we previously thought. Else, it is an error. - if (otherTable->state == TableState::Free) - otherTable->props.insert({freeName, freeProp}); + if (subTable->state == TableState::Free) + { + if (FFlag::LuauUseCommittingTxnLog) + { + PendingType* pendingSub = log.queue(subTy); + TableTypeVar* pendingSubTtv = getMutable(pendingSub); + LUAU_ASSERT(pendingSubTtv); + pendingSubTtv->props.insert({freeName, freeProp}); + } + else + { + subTable->props.insert({freeName, freeProp}); + } + } else - errors.push_back(TypeError{location, UnknownProperty{otherTypeId, freeName}}); + errors.push_back(TypeError{location, UnknownProperty{subTy, freeName}}); } } - if (freeTable->indexer && otherTable->indexer) + if (freeTable->indexer && subTable->indexer) { Unifier innerState = makeChildUnifier(); - innerState.tryUnify(*freeTable->indexer, *otherTable->indexer); + innerState.tryUnifyIndexer(*subTable->indexer, *freeTable->indexer); - checkChildUnifierTypeMismatch(innerState.errors, freeTypeId, otherTypeId); + checkChildUnifierTypeMismatch(innerState.errors, superTy, subTy); - log.concat(std::move(innerState.log)); + if (FFlag::LuauUseCommittingTxnLog) + log.concat(std::move(innerState.log)); + else + DEPRECATED_log.concat(std::move(innerState.DEPRECATED_log)); } - else if (otherTable->state == TableState::Free && freeTable->indexer) - freeTable->indexer = otherTable->indexer; - - if (!freeTable->boundTo && otherTable->state != TableState::Free) + else if (subTable->state == TableState::Free && freeTable->indexer) { - log(freeTable); - freeTable->boundTo = otherTypeId; + if (FFlag::LuauUseCommittingTxnLog) + { + log.changeIndexer(superTy, subTable->indexer); + } + else + { + freeTable->indexer = subTable->indexer; + } + } + + if (!freeTable->boundTo && subTable->state != TableState::Free) + { + if (FFlag::LuauUseCommittingTxnLog) + { + log.bindTable(superTy, subTy); + } + else + { + DEPRECATED_log(freeTable); + freeTable->boundTo = subTy; + } } } -void Unifier::tryUnifySealedTables(TypeId left, TypeId right, bool isIntersection) +void Unifier::tryUnifySealedTables(TypeId subTy, TypeId superTy, bool isIntersection) { - TableTypeVar* lt = getMutable(left); - TableTypeVar* rt = getMutable(right); - if (!lt || !rt) + TableTypeVar* superTable = getMutable(superTy); + TableTypeVar* subTable = getMutable(subTy); + + if (FFlag::LuauUseCommittingTxnLog) + { + superTable = log.getMutable(superTy); + subTable = log.getMutable(subTy); + } + + if (!superTable || !subTable) ice("passed non-table types to unifySealedTables"); Unifier innerState = makeChildUnifier(); std::vector missingPropertiesInSuper; - bool isUnnamedTable = rt->name == std::nullopt && rt->syntheticName == std::nullopt; + bool isUnnamedTable = subTable->name == std::nullopt && subTable->syntheticName == std::nullopt; bool errorReported = false; // Optimization: First test that the property sets are compatible without doing any recursive unification - if (FFlag::LuauTableUnificationEarlyTest && !rt->indexer) + if (FFlag::LuauTableUnificationEarlyTest && !subTable->indexer) { - for (const auto& [propName, superProp] : lt->props) + for (const auto& [propName, superProp] : superTable->props) { - auto subIter = rt->props.find(propName); - if (subIter == rt->props.end() && !isOptional(superProp.type)) + auto subIter = subTable->props.find(propName); + if (subIter == subTable->props.end() && !isOptional(superProp.type)) missingPropertiesInSuper.push_back(propName); } if (!missingPropertiesInSuper.empty()) { - errors.push_back(TypeError{location, MissingProperties{left, right, std::move(missingPropertiesInSuper)}}); + errors.push_back(TypeError{location, MissingProperties{superTy, subTy, std::move(missingPropertiesInSuper)}}); return; } } // Tables must have exactly the same props and their types must all unify - for (const auto& it : lt->props) + for (const auto& it : superTable->props) { - const auto& r = rt->props.find(it.first); - if (r == rt->props.end()) + const auto& r = subTable->props.find(it.first); + if (r == subTable->props.end()) { if (isOptional(it.second.type)) continue; missingPropertiesInSuper.push_back(it.first); - innerState.errors.push_back(TypeError{location, TypeMismatch{left, right}}); + innerState.errors.push_back(TypeError{location, TypeMismatch{superTy, subTy}}); } else { @@ -1594,7 +2274,7 @@ void Unifier::tryUnifySealedTables(TypeId left, TypeId right, bool isIntersectio size_t oldErrorSize = innerState.errors.size(); Location old = innerState.location; innerState.location = *r->second.location; - innerState.tryUnify_(it.second.type, r->second.type); + innerState.tryUnify_(r->second.type, it.second.type); innerState.location = old; if (oldErrorSize != innerState.errors.size() && !errorReported) @@ -1605,113 +2285,165 @@ void Unifier::tryUnifySealedTables(TypeId left, TypeId right, bool isIntersectio } else { - innerState.tryUnify_(it.second.type, r->second.type); + innerState.tryUnify_(r->second.type, it.second.type); } } } - if (lt->indexer || rt->indexer) + if (superTable->indexer || subTable->indexer) { - if (lt->indexer && rt->indexer) - innerState.tryUnify(*lt->indexer, *rt->indexer); - else if (rt->state == TableState::Unsealed) + if (FFlag::LuauUseCommittingTxnLog) { - if (lt->indexer && !rt->indexer) - rt->indexer = lt->indexer; - } - else if (lt->state == TableState::Unsealed) - { - if (rt->indexer && !lt->indexer) - lt->indexer = rt->indexer; - } - else if (lt->indexer) - { - innerState.tryUnify_(lt->indexer->indexType, getSingletonTypes().stringType); - // We already try to unify properties in both tables. - // Skip those and just look for the ones remaining and see if they fit into the indexer. - for (const auto& [name, type] : rt->props) + if (superTable->indexer && subTable->indexer) + innerState.tryUnifyIndexer(*subTable->indexer, *superTable->indexer); + else if (subTable->state == TableState::Unsealed) { - const auto& it = lt->props.find(name); - if (it == lt->props.end()) - innerState.tryUnify_(lt->indexer->indexResultType, type.type); + if (superTable->indexer && !subTable->indexer) + { + log.changeIndexer(subTy, superTable->indexer); + } } + else if (superTable->state == TableState::Unsealed) + { + if (subTable->indexer && !superTable->indexer) + { + log.changeIndexer(superTy, subTable->indexer); + } + } + else if (superTable->indexer) + { + innerState.tryUnify_(getSingletonTypes().stringType, superTable->indexer->indexType); + for (const auto& [name, type] : subTable->props) + { + const auto& it = superTable->props.find(name); + if (it == superTable->props.end()) + innerState.tryUnify_(type.type, superTable->indexer->indexResultType); + } + } + else + innerState.errors.push_back(TypeError{location, TypeMismatch{superTy, subTy}}); } else - innerState.errors.push_back(TypeError{location, TypeMismatch{left, right}}); + { + if (superTable->indexer && subTable->indexer) + innerState.tryUnifyIndexer(*subTable->indexer, *superTable->indexer); + else if (subTable->state == TableState::Unsealed) + { + if (superTable->indexer && !subTable->indexer) + subTable->indexer = superTable->indexer; + } + else if (superTable->state == TableState::Unsealed) + { + if (subTable->indexer && !superTable->indexer) + superTable->indexer = subTable->indexer; + } + else if (superTable->indexer) + { + innerState.tryUnify_(getSingletonTypes().stringType, superTable->indexer->indexType); + // We already try to unify properties in both tables. + // Skip those and just look for the ones remaining and see if they fit into the indexer. + for (const auto& [name, type] : subTable->props) + { + const auto& it = superTable->props.find(name); + if (it == superTable->props.end()) + innerState.tryUnify_(type.type, superTable->indexer->indexResultType); + } + } + else + innerState.errors.push_back(TypeError{location, TypeMismatch{superTy, subTy}}); + } } - log.concat(std::move(innerState.log)); + if (FFlag::LuauUseCommittingTxnLog) + { + if (!errorReported) + log.concat(std::move(innerState.log)); + } + else + DEPRECATED_log.concat(std::move(innerState.DEPRECATED_log)); if (errorReported) return; if (!missingPropertiesInSuper.empty()) { - errors.push_back(TypeError{location, MissingProperties{left, right, std::move(missingPropertiesInSuper)}}); + errors.push_back(TypeError{location, MissingProperties{superTy, subTy, std::move(missingPropertiesInSuper)}}); return; } - // If the superTy/left is an immediate part of an intersection type, do not do extra-property check. + // If the superTy is an immediate part of an intersection type, do not do extra-property check. // Otherwise, we would falsely generate an extra-property-error for 's' in this code: // local a: {n: number} & {s: string} = {n=1, s=""} // When checking against the table '{n: number}'. - if (!isIntersection && lt->state != TableState::Unsealed && !lt->indexer) + if (!isIntersection && superTable->state != TableState::Unsealed && !superTable->indexer) { // Check for extra properties in the subTy std::vector extraPropertiesInSub; - for (const auto& it : rt->props) + for (const auto& [subKey, subProp] : subTable->props) { - const auto& r = lt->props.find(it.first); - if (r == lt->props.end()) + const auto& superIt = superTable->props.find(subKey); + if (superIt == superTable->props.end()) { - if (isOptional(it.second.type)) + if (isOptional(subProp.type)) continue; - extraPropertiesInSub.push_back(it.first); + extraPropertiesInSub.push_back(subKey); } } if (!extraPropertiesInSub.empty()) { - errors.push_back(TypeError{location, MissingProperties{left, right, std::move(extraPropertiesInSub), MissingProperties::Extra}}); + errors.push_back(TypeError{location, MissingProperties{superTy, subTy, std::move(extraPropertiesInSub), MissingProperties::Extra}}); return; } } - checkChildUnifierTypeMismatch(innerState.errors, left, right); + checkChildUnifierTypeMismatch(innerState.errors, superTy, subTy); } -void Unifier::tryUnifyWithMetatable(TypeId metatable, TypeId other, bool reversed) +void Unifier::tryUnifyWithMetatable(TypeId subTy, TypeId superTy, bool reversed) { - const MetatableTypeVar* lhs = get(metatable); - if (!lhs) + const MetatableTypeVar* superMetatable = get(superTy); + if (!superMetatable) ice("tryUnifyMetatable invoked with non-metatable TypeVar"); - TypeError mismatchError = TypeError{location, TypeMismatch{reversed ? other : metatable, reversed ? metatable : other}}; + TypeError mismatchError = TypeError{location, TypeMismatch{reversed ? subTy : superTy, reversed ? superTy : subTy}}; - if (const MetatableTypeVar* rhs = get(other)) + if (const MetatableTypeVar* subMetatable = + FFlag::LuauUseCommittingTxnLog ? log.getMutable(subTy) : get(subTy)) { Unifier innerState = makeChildUnifier(); - innerState.tryUnify_(lhs->table, rhs->table); - innerState.tryUnify_(lhs->metatable, rhs->metatable); + innerState.tryUnify_(subMetatable->table, superMetatable->table); + innerState.tryUnify_(subMetatable->metatable, superMetatable->metatable); if (auto e = hasUnificationTooComplex(innerState.errors)) errors.push_back(*e); else if (!innerState.errors.empty()) errors.push_back( - TypeError{location, TypeMismatch{reversed ? other : metatable, reversed ? metatable : other, "", innerState.errors.front()}}); + TypeError{location, TypeMismatch{reversed ? subTy : superTy, reversed ? superTy : subTy, "", innerState.errors.front()}}); - log.concat(std::move(innerState.log)); + if (FFlag::LuauUseCommittingTxnLog) + log.concat(std::move(innerState.log)); + else + DEPRECATED_log.concat(std::move(innerState.DEPRECATED_log)); } - else if (TableTypeVar* rhs = getMutable(other)) + else if (TableTypeVar* subTable = FFlag::LuauUseCommittingTxnLog ? log.getMutable(subTy) : getMutable(subTy)) { - switch (rhs->state) + switch (subTable->state) { case TableState::Free: { - tryUnify_(lhs->table, other); - rhs->boundTo = metatable; + tryUnify_(subTy, superMetatable->table); + + if (FFlag::LuauUseCommittingTxnLog) + { + log.bindTable(subTy, superTy); + } + else + { + subTable->boundTo = superTy; + } break; } @@ -1722,7 +2454,8 @@ void Unifier::tryUnifyWithMetatable(TypeId metatable, TypeId other, bool reverse errors.push_back(mismatchError); } } - else if (get(other) || get(other)) + else if (FFlag::LuauUseCommittingTxnLog ? (log.getMutable(subTy) || log.getMutable(subTy)) + : (get(subTy) || get(subTy))) { } else @@ -1732,7 +2465,7 @@ void Unifier::tryUnifyWithMetatable(TypeId metatable, TypeId other, bool reverse } // Class unification is almost, but not quite symmetrical. We use the 'reversed' boolean to indicate which scenario we are evaluating. -void Unifier::tryUnifyWithClass(TypeId superTy, TypeId subTy, bool reversed) +void Unifier::tryUnifyWithClass(TypeId subTy, TypeId superTy, bool reversed) { if (reversed) std::swap(superTy, subTy); @@ -1763,7 +2496,7 @@ void Unifier::tryUnifyWithClass(TypeId superTy, TypeId subTy, bool reversed) } ice("Illegal variance setting!"); } - else if (TableTypeVar* table = getMutable(subTy)) + else if (TableTypeVar* subTable = getMutable(subTy)) { /** * A free table is something whose shape we do not exactly know yet. @@ -1775,12 +2508,12 @@ void Unifier::tryUnifyWithClass(TypeId superTy, TypeId subTy, bool reversed) * * Tables that are not free are known to be actual tables. */ - if (table->state != TableState::Free) + if (subTable->state != TableState::Free) return fail(); bool ok = true; - for (const auto& [propName, prop] : table->props) + for (const auto& [propName, prop] : subTable->props) { const Property* classProp = lookupClassProp(superClass, propName); if (!classProp) @@ -1791,23 +2524,37 @@ void Unifier::tryUnifyWithClass(TypeId superTy, TypeId subTy, bool reversed) else { Unifier innerState = makeChildUnifier(); - innerState.tryUnify_(prop.type, classProp->type); + innerState.tryUnify_(classProp->type, prop.type); checkChildUnifierTypeMismatch(innerState.errors, propName, reversed ? subTy : superTy, reversed ? superTy : subTy); - if (innerState.errors.empty()) + if (FFlag::LuauUseCommittingTxnLog) { - log.concat(std::move(innerState.log)); + if (innerState.errors.empty()) + { + log.concat(std::move(innerState.log)); + } + else + { + ok = false; + } } else { - ok = false; - innerState.log.rollback(); + if (innerState.errors.empty()) + { + DEPRECATED_log.concat(std::move(innerState.DEPRECATED_log)); + } + else + { + ok = false; + innerState.DEPRECATED_log.rollback(); + } } } } - if (table->indexer) + if (subTable->indexer) { ok = false; std::string msg = "Class " + superClass->name + " does not have an indexer"; @@ -1817,17 +2564,24 @@ void Unifier::tryUnifyWithClass(TypeId superTy, TypeId subTy, bool reversed) if (!ok) return; - log(table); - table->boundTo = superTy; + if (FFlag::LuauUseCommittingTxnLog) + { + log.bindTable(subTy, superTy); + } + else + { + DEPRECATED_log(subTable); + subTable->boundTo = superTy; + } } else return fail(); } -void Unifier::tryUnify(const TableIndexer& superIndexer, const TableIndexer& subIndexer) +void Unifier::tryUnifyIndexer(const TableIndexer& subIndexer, const TableIndexer& superIndexer) { - tryUnify_(superIndexer.indexType, subIndexer.indexType); - tryUnify_(superIndexer.indexResultType, subIndexer.indexResultType); + tryUnify_(subIndexer.indexType, superIndexer.indexType); + tryUnify_(subIndexer.indexResultType, superIndexer.indexResultType); } static void queueTypePack(std::vector& queue, DenseHashSet& seenTypePacks, Unifier& state, TypePackId a, TypePackId anyTypePack) @@ -1840,54 +2594,85 @@ static void queueTypePack(std::vector& queue, DenseHashSet& break; seenTypePacks.insert(a); - if (get(a)) + if (FFlag::LuauUseCommittingTxnLog) { - state.log(a); - *asMutable(a) = Unifiable::Bound{anyTypePack}; + if (state.log.getMutable(a)) + { + state.log.replace(a, Unifiable::Bound{anyTypePack}); + } + else if (auto tp = state.log.getMutable(a)) + { + queue.insert(queue.end(), tp->head.begin(), tp->head.end()); + if (tp->tail) + a = *tp->tail; + else + break; + } } - else if (auto tp = get(a)) + else { - queue.insert(queue.end(), tp->head.begin(), tp->head.end()); - if (tp->tail) - a = *tp->tail; - else - break; + if (get(a)) + { + state.DEPRECATED_log(a); + *asMutable(a) = Unifiable::Bound{anyTypePack}; + } + else if (auto tp = get(a)) + { + queue.insert(queue.end(), tp->head.begin(), tp->head.end()); + if (tp->tail) + a = *tp->tail; + else + break; + } } } } -void Unifier::tryUnifyVariadics(TypePackId superTp, TypePackId subTp, bool reversed, int subOffset) +void Unifier::tryUnifyVariadics(TypePackId subTp, TypePackId superTp, bool reversed, int subOffset) { - const VariadicTypePack* lv = get(superTp); - if (!lv) + const VariadicTypePack* superVariadic = get(superTp); + + if (FFlag::LuauUseCommittingTxnLog) + { + superVariadic = log.getMutable(superTp); + } + + if (!superVariadic) ice("passed non-variadic pack to tryUnifyVariadics"); - if (const VariadicTypePack* rv = get(subTp)) - tryUnify_(reversed ? rv->ty : lv->ty, reversed ? lv->ty : rv->ty); + if (const VariadicTypePack* subVariadic = get(subTp)) + tryUnify_(reversed ? superVariadic->ty : subVariadic->ty, reversed ? subVariadic->ty : superVariadic->ty); else if (get(subTp)) { - TypePackIterator rIter = begin(subTp); - TypePackIterator rEnd = end(subTp); + TypePackIterator subIter = begin(subTp, &log); + TypePackIterator subEnd = end(subTp); - std::advance(rIter, subOffset); + std::advance(subIter, subOffset); - while (rIter != rEnd) + while (subIter != subEnd) { - tryUnify_(reversed ? *rIter : lv->ty, reversed ? lv->ty : *rIter); - ++rIter; + tryUnify_(reversed ? superVariadic->ty : *subIter, reversed ? *subIter : superVariadic->ty); + ++subIter; } - if (std::optional maybeTail = rIter.tail()) + if (std::optional maybeTail = subIter.tail()) { TypePackId tail = follow(*maybeTail); if (get(tail)) { - log(tail); - *asMutable(tail) = BoundTypePack{superTp}; + if (FFlag::LuauUseCommittingTxnLog) + { + log.replace(tail, BoundTypePack(superTp)); + } + else + { + DEPRECATED_log(tail); + *asMutable(tail) = BoundTypePack{superTp}; + } } else if (const VariadicTypePack* vtp = get(tail)) { - tryUnify_(lv->ty, vtp->ty); + tryUnify_(vtp->ty, superVariadic->ty); } else if (get(tail)) { @@ -1914,65 +2699,113 @@ static void tryUnifyWithAny(std::vector& queue, Unifier& state, DenseHas { while (!queue.empty()) { - TypeId ty = follow(queue.back()); - queue.pop_back(); - if (seen.find(ty)) - continue; - seen.insert(ty); + if (FFlag::LuauUseCommittingTxnLog) + { + TypeId ty = state.log.follow(queue.back()); + queue.pop_back(); + if (seen.find(ty)) + continue; + seen.insert(ty); - if (get(ty)) - { - state.log(ty); - *asMutable(ty) = BoundTypeVar{anyType}; - } - else if (auto fun = get(ty)) - { - queueTypePack(queue, seenTypePacks, state, fun->argTypes, anyTypePack); - queueTypePack(queue, seenTypePacks, state, fun->retType, anyTypePack); - } - else if (auto table = get(ty)) - { - for (const auto& [_name, prop] : table->props) - queue.push_back(prop.type); - - if (table->indexer) + if (state.log.getMutable(ty)) { - queue.push_back(table->indexer->indexType); - queue.push_back(table->indexer->indexResultType); + state.log.replace(ty, BoundTypeVar{anyType}); } + else if (auto fun = state.log.getMutable(ty)) + { + queueTypePack(queue, seenTypePacks, state, fun->argTypes, anyTypePack); + queueTypePack(queue, seenTypePacks, state, fun->retType, anyTypePack); + } + else if (auto table = state.log.getMutable(ty)) + { + for (const auto& [_name, prop] : table->props) + queue.push_back(prop.type); + + if (table->indexer) + { + queue.push_back(table->indexer->indexType); + queue.push_back(table->indexer->indexResultType); + } + } + else if (auto mt = state.log.getMutable(ty)) + { + queue.push_back(mt->table); + queue.push_back(mt->metatable); + } + else if (state.log.getMutable(ty)) + { + // ClassTypeVars never contain free typevars. + } + else if (auto union_ = state.log.getMutable(ty)) + queue.insert(queue.end(), union_->options.begin(), union_->options.end()); + else if (auto intersection = state.log.getMutable(ty)) + queue.insert(queue.end(), intersection->parts.begin(), intersection->parts.end()); + else + { + } // Primitives, any, errors, and generics are left untouched. } - else if (auto mt = get(ty)) - { - queue.push_back(mt->table); - queue.push_back(mt->metatable); - } - else if (get(ty)) - { - // ClassTypeVars never contain free typevars. - } - else if (auto union_ = get(ty)) - queue.insert(queue.end(), union_->options.begin(), union_->options.end()); - else if (auto intersection = get(ty)) - queue.insert(queue.end(), intersection->parts.begin(), intersection->parts.end()); else { - } // Primitives, any, errors, and generics are left untouched. + TypeId ty = follow(queue.back()); + queue.pop_back(); + if (seen.find(ty)) + continue; + seen.insert(ty); + + if (get(ty)) + { + state.DEPRECATED_log(ty); + *asMutable(ty) = BoundTypeVar{anyType}; + } + else if (auto fun = get(ty)) + { + queueTypePack(queue, seenTypePacks, state, fun->argTypes, anyTypePack); + queueTypePack(queue, seenTypePacks, state, fun->retType, anyTypePack); + } + else if (auto table = get(ty)) + { + for (const auto& [_name, prop] : table->props) + queue.push_back(prop.type); + + if (table->indexer) + { + queue.push_back(table->indexer->indexType); + queue.push_back(table->indexer->indexResultType); + } + } + else if (auto mt = get(ty)) + { + queue.push_back(mt->table); + queue.push_back(mt->metatable); + } + else if (get(ty)) + { + // ClassTypeVars never contain free typevars. + } + else if (auto union_ = get(ty)) + queue.insert(queue.end(), union_->options.begin(), union_->options.end()); + else if (auto intersection = get(ty)) + queue.insert(queue.end(), intersection->parts.begin(), intersection->parts.end()); + else + { + } // Primitives, any, errors, and generics are left untouched. + } } } -void Unifier::tryUnifyWithAny(TypeId any, TypeId ty) +void Unifier::tryUnifyWithAny(TypeId subTy, TypeId anyTy) { - LUAU_ASSERT(get(any) || get(any)); + LUAU_ASSERT(get(anyTy) || get(anyTy)); // These types are not visited in general loop below - if (get(ty) || get(ty) || get(ty)) + if (get(subTy) || get(subTy) || get(subTy)) return; const TypePackId anyTypePack = types->addTypePack(TypePackVar{VariadicTypePack{getSingletonTypes().anyType}}); - const TypePackId anyTP = get(any) ? anyTypePack : types->addTypePack(TypePackVar{Unifiable::Error{}}); + const TypePackId anyTP = get(anyTy) ? anyTypePack : types->addTypePack(TypePackVar{Unifiable::Error{}}); - std::vector queue = {ty}; + std::vector queue = {subTy}; sharedState.tempSeenTy.clear(); sharedState.tempSeenTp.clear(); @@ -1980,9 +2813,9 @@ void Unifier::tryUnifyWithAny(TypeId any, TypeId ty) Luau::tryUnifyWithAny(queue, *this, sharedState.tempSeenTy, sharedState.tempSeenTp, getSingletonTypes().anyType, anyTP); } -void Unifier::tryUnifyWithAny(TypePackId any, TypePackId ty) +void Unifier::tryUnifyWithAny(TypePackId subTy, TypePackId anyTp) { - LUAU_ASSERT(get(any)); + LUAU_ASSERT(get(anyTp)); const TypeId anyTy = getSingletonTypes().errorRecoveryType(); @@ -1991,9 +2824,9 @@ void Unifier::tryUnifyWithAny(TypePackId any, TypePackId ty) sharedState.tempSeenTy.clear(); sharedState.tempSeenTp.clear(); - queueTypePack(queue, sharedState.tempSeenTp, *this, ty, any); + queueTypePack(queue, sharedState.tempSeenTp, *this, subTy, anyTp); - Luau::tryUnifyWithAny(queue, *this, sharedState.tempSeenTy, sharedState.tempSeenTp, anyTy, any); + Luau::tryUnifyWithAny(queue, *this, sharedState.tempSeenTy, sharedState.tempSeenTp, anyTy, anyTp); } std::optional Unifier::findTablePropertyRespectingMeta(TypeId lhsType, Name name) @@ -2012,54 +2845,105 @@ void Unifier::occursCheck(DenseHashSet& seen, TypeId needle, TypeId hays { RecursionLimiter _ra(&sharedState.counters.recursionCount, FInt::LuauTypeInferRecursionLimit); - needle = follow(needle); - haystack = follow(haystack); - - if (seen.find(haystack)) - return; - - seen.insert(haystack); - - if (get(needle)) - return; - - if (!get(needle)) - ice("Expected needle to be free"); - - if (needle == haystack) - { - errors.push_back(TypeError{location, OccursCheckFailed{}}); - log(needle); - *asMutable(needle) = *getSingletonTypes().errorRecoveryType(); - return; - } - auto check = [&](TypeId tv) { occursCheck(seen, needle, tv); }; - if (get(haystack)) - return; - else if (auto a = get(haystack)) + if (FFlag::LuauUseCommittingTxnLog) { - if (!FFlag::LuauOccursCheckOkWithRecursiveFunctions) - { - for (TypeId ty : a->argTypes) - check(ty); + needle = log.follow(needle); + haystack = log.follow(haystack); - for (TypeId ty : a->retType) + if (seen.find(haystack)) + return; + + seen.insert(haystack); + + if (log.getMutable(needle)) + return; + + if (!log.getMutable(needle)) + ice("Expected needle to be free"); + + if (needle == haystack) + { + errors.push_back(TypeError{location, OccursCheckFailed{}}); + log.replace(needle, *getSingletonTypes().errorRecoveryType()); + + return; + } + + if (log.getMutable(haystack)) + return; + else if (auto a = log.getMutable(haystack)) + { + if (!FFlag::LuauOccursCheckOkWithRecursiveFunctions) + { + for (TypePackIterator it(a->argTypes, &log); it != end(a->argTypes); ++it) + check(*it); + + for (TypePackIterator it(a->retType, &log); it != end(a->retType); ++it) + check(*it); + } + } + else if (auto a = log.getMutable(haystack)) + { + for (TypeId ty : a->options) + check(ty); + } + else if (auto a = log.getMutable(haystack)) + { + for (TypeId ty : a->parts) check(ty); } } - else if (auto a = get(haystack)) + else { - for (TypeId ty : a->options) - check(ty); - } - else if (auto a = get(haystack)) - { - for (TypeId ty : a->parts) - check(ty); + needle = follow(needle); + haystack = follow(haystack); + + if (seen.find(haystack)) + return; + + seen.insert(haystack); + + if (get(needle)) + return; + + if (!get(needle)) + ice("Expected needle to be free"); + + if (needle == haystack) + { + errors.push_back(TypeError{location, OccursCheckFailed{}}); + DEPRECATED_log(needle); + *asMutable(needle) = *getSingletonTypes().errorRecoveryType(); + return; + } + + if (get(haystack)) + return; + else if (auto a = get(haystack)) + { + if (!FFlag::LuauOccursCheckOkWithRecursiveFunctions) + { + for (TypeId ty : a->argTypes) + check(ty); + + for (TypeId ty : a->retType) + check(ty); + } + } + else if (auto a = get(haystack)) + { + for (TypeId ty : a->options) + check(ty); + } + else if (auto a = get(haystack)) + { + for (TypeId ty : a->parts) + check(ty); + } } } @@ -2072,59 +2956,115 @@ void Unifier::occursCheck(TypePackId needle, TypePackId haystack) void Unifier::occursCheck(DenseHashSet& seen, TypePackId needle, TypePackId haystack) { - needle = follow(needle); - haystack = follow(haystack); - - if (seen.find(haystack)) - return; - - seen.insert(haystack); - - if (get(needle)) - return; - - if (!get(needle)) - ice("Expected needle pack to be free"); - - RecursionLimiter _ra(&sharedState.counters.recursionCount, FInt::LuauTypeInferRecursionLimit); - - while (!get(haystack)) + if (FFlag::LuauUseCommittingTxnLog) { - if (needle == haystack) - { - errors.push_back(TypeError{location, OccursCheckFailed{}}); - log(needle); - *asMutable(needle) = *getSingletonTypes().errorRecoveryTypePack(); - return; - } + needle = log.follow(needle); + haystack = log.follow(haystack); - if (auto a = get(haystack)) + if (seen.find(haystack)) + return; + + seen.insert(haystack); + + if (log.getMutable(needle)) + return; + + if (!get(needle)) + ice("Expected needle pack to be free"); + + RecursionLimiter _ra(&sharedState.counters.recursionCount, FInt::LuauTypeInferRecursionLimit); + + while (!log.getMutable(haystack)) { - if (!FFlag::LuauOccursCheckOkWithRecursiveFunctions) + if (needle == haystack) + { + errors.push_back(TypeError{location, OccursCheckFailed{}}); + log.replace(needle, *getSingletonTypes().errorRecoveryTypePack()); + + return; + } + + if (auto a = get(haystack)) { for (const auto& ty : a->head) { - if (auto f = get(follow(ty))) + if (!FFlag::LuauOccursCheckOkWithRecursiveFunctions) { - occursCheck(seen, needle, f->argTypes); - occursCheck(seen, needle, f->retType); + if (auto f = log.getMutable(log.follow(ty))) + { + occursCheck(seen, needle, f->argTypes); + occursCheck(seen, needle, f->retType); + } } } + + if (a->tail) + { + haystack = follow(*a->tail); + continue; + } + } + break; + } + } + else + { + needle = follow(needle); + haystack = follow(haystack); + + if (seen.find(haystack)) + return; + + seen.insert(haystack); + + if (get(needle)) + return; + + if (!get(needle)) + ice("Expected needle pack to be free"); + + RecursionLimiter _ra(&sharedState.counters.recursionCount, FInt::LuauTypeInferRecursionLimit); + + while (!get(haystack)) + { + if (needle == haystack) + { + errors.push_back(TypeError{location, OccursCheckFailed{}}); + DEPRECATED_log(needle); + *asMutable(needle) = *getSingletonTypes().errorRecoveryTypePack(); } - if (a->tail) + if (auto a = get(haystack)) { - haystack = follow(*a->tail); - continue; + if (!FFlag::LuauOccursCheckOkWithRecursiveFunctions) + { + for (const auto& ty : a->head) + { + if (auto f = get(follow(ty))) + { + occursCheck(seen, needle, f->argTypes); + occursCheck(seen, needle, f->retType); + } + } + } + + if (a->tail) + { + haystack = follow(*a->tail); + continue; + } } + break; } - break; } } Unifier Unifier::makeChildUnifier() { - return Unifier{types, mode, globalScope, log.sharedSeen, location, variance, sharedState}; + if (FFlag::LuauUseCommittingTxnLog) + return Unifier{types, mode, globalScope, log.sharedSeen, location, variance, sharedState, &log}; + else + return Unifier{types, mode, globalScope, DEPRECATED_log.sharedSeen, location, variance, sharedState, &log}; } bool Unifier::isNonstrictMode() const diff --git a/Ast/include/Luau/Common.h b/Ast/include/Luau/Common.h index 63cd3df4..fbb03a9e 100644 --- a/Ast/include/Luau/Common.h +++ b/Ast/include/Luau/Common.h @@ -29,7 +29,7 @@ namespace Luau { -using AssertHandler = int (*)(const char* expression, const char* file, int line); +using AssertHandler = int (*)(const char* expression, const char* file, int line, const char* function); inline AssertHandler& assertHandler() { @@ -37,10 +37,10 @@ inline AssertHandler& assertHandler() return handler; } -inline int assertCallHandler(const char* expression, const char* file, int line) +inline int assertCallHandler(const char* expression, const char* file, int line, const char* function) { if (AssertHandler handler = assertHandler()) - return handler(expression, file, line); + return handler(expression, file, line, function); return 1; } @@ -48,7 +48,7 @@ inline int assertCallHandler(const char* expression, const char* file, int line) } // namespace Luau #if !defined(NDEBUG) || defined(LUAU_ENABLE_ASSERT) -#define LUAU_ASSERT(expr) ((void)(!!(expr) || (Luau::assertCallHandler(#expr, __FILE__, __LINE__) && (LUAU_DEBUGBREAK(), 0)))) +#define LUAU_ASSERT(expr) ((void)(!!(expr) || (Luau::assertCallHandler(#expr, __FILE__, __LINE__, __FUNCTION__) && (LUAU_DEBUGBREAK(), 0)))) #define LUAU_ASSERTENABLED #else #define LUAU_ASSERT(expr) (void)sizeof(!!(expr)) diff --git a/CLI/Analyze.cpp b/CLI/Analyze.cpp index 5318044b..1375f414 100644 --- a/CLI/Analyze.cpp +++ b/CLI/Analyze.cpp @@ -107,7 +107,7 @@ static void displayHelp(const char* argv0) printf(" --formatter=gnu: report analysis errors in GNU-compatible format\n"); } -static int assertionHandler(const char* expr, const char* file, int line) +static int assertionHandler(const char* expr, const char* file, int line, const char* function) { printf("%s(%d): ASSERTION FAILED: %s\n", file, line, expr); return 1; diff --git a/CLI/Repl.cpp b/CLI/Repl.cpp index 5d3324ab..35fbf309 100644 --- a/CLI/Repl.cpp +++ b/CLI/Repl.cpp @@ -235,11 +235,14 @@ static void completeIndexer(lua_State* L, const char* editBuffer, size_t start, while (lua_next(L, -2) != 0) { - // table, key, value - std::string_view key = lua_tostring(L, -2); + if (lua_type(L, -2) == LUA_TSTRING) + { + // table, key, value + std::string_view key = lua_tostring(L, -2); - if (!key.empty() && Luau::startsWith(key, prefix)) - completions.push_back(editBuffer + std::string(key.substr(prefix.size()))); + if (!key.empty() && Luau::startsWith(key, prefix)) + completions.push_back(editBuffer + std::string(key.substr(prefix.size()))); + } lua_pop(L, 1); } @@ -253,7 +256,7 @@ static void completeIndexer(lua_State* L, const char* editBuffer, size_t start, lua_rawget(L, -2); lua_remove(L, -2); - if (lua_isnil(L, -1)) + if (!lua_istable(L, -1)) break; lookup.remove_prefix(dot + 1); @@ -266,7 +269,7 @@ static void completeIndexer(lua_State* L, const char* editBuffer, size_t start, static void completeRepl(lua_State* L, const char* editBuffer, std::vector& completions) { size_t start = strlen(editBuffer); - while (start > 0 && (isalnum(editBuffer[start - 1]) || editBuffer[start - 1] == '.')) + while (start > 0 && (isalnum(editBuffer[start - 1]) || editBuffer[start - 1] == '.' || editBuffer[start - 1] == '_')) start--; // look the value up in current global table first @@ -278,6 +281,34 @@ static void completeRepl(lua_State* L, const char* editBuffer, std::vector globalState(luaL_newstate(), lua_close); @@ -292,6 +323,7 @@ static void runRepl() }); std::string buffer; + LinenoiseScopedHistory scopedHistory; for (;;) { @@ -457,7 +489,7 @@ static void displayHelp(const char* argv0) printf(" --coverage: collect code coverage while running the code and output results to coverage.out\n"); } -static int assertionHandler(const char* expr, const char* file, int line) +static int assertionHandler(const char* expr, const char* file, int line, const char* function) { printf("%s(%d): ASSERTION FAILED: %s\n", file, line, expr); return 1; diff --git a/CLI/Web.cpp b/CLI/Web.cpp index cf5c831e..416a79f2 100644 --- a/CLI/Web.cpp +++ b/CLI/Web.cpp @@ -53,30 +53,38 @@ static std::string runCode(lua_State* L, const std::string& source) lua_insert(T, 1); lua_pcall(T, n, 0, 0); } + + lua_pop(L, 1); // pop T + return std::string(); } else { std::string error; + lua_Debug ar; + if (lua_getinfo(L, 0, "sln", &ar)) + { + error += ar.short_src; + error += ':'; + error += std::to_string(ar.currentline); + error += ": "; + } + if (status == LUA_YIELD) { - error = "thread yielded unexpectedly"; + error += "thread yielded unexpectedly"; } else if (const char* str = lua_tostring(T, -1)) { - error = str; + error += str; } error += "\nstack backtrace:\n"; error += lua_debugtrace(T); - error = "Error:" + error; - - fprintf(stdout, "%s", error.c_str()); + lua_pop(L, 1); // pop T + return error; } - - lua_pop(L, 1); - return std::string(); } extern "C" const char* executeScript(const char* source) diff --git a/Compiler/include/Luau/Bytecode.h b/Compiler/include/Luau/Bytecode.h index 71631d10..d9694d7d 100644 --- a/Compiler/include/Luau/Bytecode.h +++ b/Compiler/include/Luau/Bytecode.h @@ -377,6 +377,7 @@ enum LuauBytecodeTag { // Bytecode version LBC_VERSION = 1, + LBC_VERSION_FUTURE = 2, // TODO: This will be removed in favor of LBC_VERSION with LuauBytecodeV2Force // Types of constant table entries LBC_CONSTANT_NIL = 0, LBC_CONSTANT_BOOLEAN, diff --git a/Compiler/include/Luau/BytecodeBuilder.h b/Compiler/include/Luau/BytecodeBuilder.h index d4ebad6b..287bf4ee 100644 --- a/Compiler/include/Luau/BytecodeBuilder.h +++ b/Compiler/include/Luau/BytecodeBuilder.h @@ -74,6 +74,7 @@ public: void expandJumps(); void setDebugFunctionName(StringRef name); + void setDebugFunctionLineDefined(int line); void setDebugLine(int line); void pushDebugLocal(StringRef name, uint8_t reg, uint32_t startpc, uint32_t endpc); void pushDebugUpval(StringRef name); @@ -162,6 +163,7 @@ private: bool isvararg = false; unsigned int debugname = 0; + int debuglinedefined = 0; std::string dump; std::string dumpname; diff --git a/Compiler/src/BytecodeBuilder.cpp b/Compiler/src/BytecodeBuilder.cpp index 3280c8a4..2d31c409 100644 --- a/Compiler/src/BytecodeBuilder.cpp +++ b/Compiler/src/BytecodeBuilder.cpp @@ -6,6 +6,8 @@ #include #include +LUAU_FASTFLAGVARIABLE(LuauBytecodeV2Write, false) + namespace Luau { @@ -81,6 +83,52 @@ static int getOpLength(LuauOpcode op) } } +inline bool isJumpD(LuauOpcode op) +{ + switch (op) + { + case LOP_JUMP: + case LOP_JUMPIF: + case LOP_JUMPIFNOT: + case LOP_JUMPIFEQ: + case LOP_JUMPIFLE: + case LOP_JUMPIFLT: + case LOP_JUMPIFNOTEQ: + case LOP_JUMPIFNOTLE: + case LOP_JUMPIFNOTLT: + case LOP_FORNPREP: + case LOP_FORNLOOP: + case LOP_FORGLOOP: + case LOP_FORGPREP_INEXT: + case LOP_FORGLOOP_INEXT: + case LOP_FORGPREP_NEXT: + case LOP_FORGLOOP_NEXT: + case LOP_JUMPBACK: + case LOP_JUMPIFEQK: + case LOP_JUMPIFNOTEQK: + return true; + + default: + return false; + } +} + +inline bool isSkipC(LuauOpcode op) +{ + switch (op) + { + case LOP_LOADB: + case LOP_FASTCALL: + case LOP_FASTCALL1: + case LOP_FASTCALL2: + case LOP_FASTCALL2K: + return true; + + default: + return false; + } +} + bool BytecodeBuilder::StringRef::operator==(const StringRef& other) const { return (data && other.data) ? (length == other.length && memcmp(data, other.data, length) == 0) : (data == other.data); @@ -365,13 +413,7 @@ bool BytecodeBuilder::patchJumpD(size_t jumpLabel, size_t targetLabel) unsigned int jumpInsn = insns[jumpLabel]; (void)jumpInsn; - LUAU_ASSERT(LUAU_INSN_OP(jumpInsn) == LOP_JUMP || LUAU_INSN_OP(jumpInsn) == LOP_JUMPIF || LUAU_INSN_OP(jumpInsn) == LOP_JUMPIFNOT || - LUAU_INSN_OP(jumpInsn) == LOP_JUMPIFEQ || LUAU_INSN_OP(jumpInsn) == LOP_JUMPIFLE || LUAU_INSN_OP(jumpInsn) == LOP_JUMPIFLT || - LUAU_INSN_OP(jumpInsn) == LOP_JUMPIFNOTEQ || LUAU_INSN_OP(jumpInsn) == LOP_JUMPIFNOTLE || LUAU_INSN_OP(jumpInsn) == LOP_JUMPIFNOTLT || - LUAU_INSN_OP(jumpInsn) == LOP_FORNPREP || LUAU_INSN_OP(jumpInsn) == LOP_FORNLOOP || LUAU_INSN_OP(jumpInsn) == LOP_FORGLOOP || - LUAU_INSN_OP(jumpInsn) == LOP_FORGPREP_INEXT || LUAU_INSN_OP(jumpInsn) == LOP_FORGLOOP_INEXT || - LUAU_INSN_OP(jumpInsn) == LOP_FORGPREP_NEXT || LUAU_INSN_OP(jumpInsn) == LOP_FORGLOOP_NEXT || - LUAU_INSN_OP(jumpInsn) == LOP_JUMPBACK || LUAU_INSN_OP(jumpInsn) == LOP_JUMPIFEQK || LUAU_INSN_OP(jumpInsn) == LOP_JUMPIFNOTEQK); + LUAU_ASSERT(isJumpD(LuauOpcode(LUAU_INSN_OP(jumpInsn)))); LUAU_ASSERT(LUAU_INSN_D(jumpInsn) == 0); LUAU_ASSERT(targetLabel <= insns.size()); @@ -403,8 +445,7 @@ bool BytecodeBuilder::patchSkipC(size_t jumpLabel, size_t targetLabel) unsigned int jumpInsn = insns[jumpLabel]; (void)jumpInsn; - LUAU_ASSERT(LUAU_INSN_OP(jumpInsn) == LOP_FASTCALL || LUAU_INSN_OP(jumpInsn) == LOP_FASTCALL1 || LUAU_INSN_OP(jumpInsn) == LOP_FASTCALL2 || - LUAU_INSN_OP(jumpInsn) == LOP_FASTCALL2K); + LUAU_ASSERT(isSkipC(LuauOpcode(LUAU_INSN_OP(jumpInsn)))); LUAU_ASSERT(LUAU_INSN_C(jumpInsn) == 0); int offset = int(targetLabel) - int(jumpLabel) - 1; @@ -428,6 +469,11 @@ void BytecodeBuilder::setDebugFunctionName(StringRef name) functions[currentFunction].dumpname = std::string(name.data, name.length); } +void BytecodeBuilder::setDebugFunctionLineDefined(int line) +{ + functions[currentFunction].debuglinedefined = line; +} + void BytecodeBuilder::setDebugLine(int line) { debugLine = line; @@ -464,7 +510,7 @@ uint32_t BytecodeBuilder::getDebugPC() const void BytecodeBuilder::finalize() { LUAU_ASSERT(bytecode.empty()); - bytecode = char(LBC_VERSION); + bytecode = char(FFlag::LuauBytecodeV2Write ? LBC_VERSION_FUTURE : LBC_VERSION); writeStringTable(bytecode); @@ -565,6 +611,9 @@ void BytecodeBuilder::writeFunction(std::string& ss, uint32_t id) const writeVarInt(ss, child); // debug info + if (FFlag::LuauBytecodeV2Write) + writeVarInt(ss, func.debuglinedefined); + writeVarInt(ss, func.debugname); bool hasLines = true; diff --git a/Compiler/src/Compiler.cpp b/Compiler/src/Compiler.cpp index 8f74ffed..6ae49027 100644 --- a/Compiler/src/Compiler.cpp +++ b/Compiler/src/Compiler.cpp @@ -11,7 +11,6 @@ #include LUAU_FASTFLAG(LuauIfElseExpressionBaseSupport) -LUAU_FASTFLAGVARIABLE(LuauBit32CountBuiltin, false) namespace Luau { @@ -179,6 +178,8 @@ struct Compiler if (options.optimizationLevel >= 1 && options.debugLevel >= 2) gatherConstUpvals(func); + bytecode.setDebugFunctionLineDefined(func->location.begin.line + 1); + if (options.debugLevel >= 1 && func->debugname.value) bytecode.setDebugFunctionName(sref(func->debugname)); @@ -3626,9 +3627,9 @@ struct Compiler return LBF_BIT32_RROTATE; if (builtin.method == "rshift") return LBF_BIT32_RSHIFT; - if (builtin.method == "countlz" && FFlag::LuauBit32CountBuiltin) + if (builtin.method == "countlz") return LBF_BIT32_COUNTLZ; - if (builtin.method == "countrz" && FFlag::LuauBit32CountBuiltin) + if (builtin.method == "countrz") return LBF_BIT32_COUNTRZ; } diff --git a/Sources.cmake b/Sources.cmake index a7153eb3..5dd486aa 100644 --- a/Sources.cmake +++ b/Sources.cmake @@ -125,6 +125,7 @@ target_sources(Luau.VM PRIVATE VM/src/linit.cpp VM/src/lmathlib.cpp VM/src/lmem.cpp + VM/src/lnumprint.cpp VM/src/lobject.cpp VM/src/loslib.cpp VM/src/lperf.cpp diff --git a/VM/include/luaconf.h b/VM/include/luaconf.h index a01a1481..7e0832e7 100644 --- a/VM/include/luaconf.h +++ b/VM/include/luaconf.h @@ -138,10 +138,6 @@ /* }================================================================== */ -/* Default number printing format and the string length limit */ -#define LUA_NUMBER_FMT "%.14g" -#define LUAI_MAXNUMBER2STR 32 /* 16 digits, sign, point, and \0 */ - /* @@ LUAI_USER_ALIGNMENT_T is a type that requires maximum alignment. ** CHANGE it if your system requires alignments larger than double. (For diff --git a/VM/src/lapi.cpp b/VM/src/lapi.cpp index a65b0325..c98b9590 100644 --- a/VM/src/lapi.cpp +++ b/VM/src/lapi.cpp @@ -14,8 +14,6 @@ #include -LUAU_FASTFLAG(LuauActivateBeforeExec) - const char* lua_ident = "$Lua: Lua 5.1.4 Copyright (C) 1994-2008 Lua.org, PUC-Rio $\n" "$Authors: R. Ierusalimschy, L. H. de Figueiredo & W. Celes $\n" "$URL: www.lua.org $\n"; @@ -939,21 +937,7 @@ void lua_call(lua_State* L, int nargs, int nresults) checkresults(L, nargs, nresults); func = L->top - (nargs + 1); - if (FFlag::LuauActivateBeforeExec) - { - luaD_call(L, func, nresults); - } - else - { - int oldactive = luaC_threadactive(L); - l_setbit(L->stackstate, THREAD_ACTIVEBIT); - luaC_checkthreadsleep(L); - - luaD_call(L, func, nresults); - - if (!oldactive) - resetbit(L->stackstate, THREAD_ACTIVEBIT); - } + luaD_call(L, func, nresults); adjustresults(L, nresults); return; @@ -994,21 +978,7 @@ int lua_pcall(lua_State* L, int nargs, int nresults, int errfunc) c.func = L->top - (nargs + 1); /* function to be called */ c.nresults = nresults; - if (FFlag::LuauActivateBeforeExec) - { - status = luaD_pcall(L, f_call, &c, savestack(L, c.func), func); - } - else - { - int oldactive = luaC_threadactive(L); - l_setbit(L->stackstate, THREAD_ACTIVEBIT); - luaC_checkthreadsleep(L); - - status = luaD_pcall(L, f_call, &c, savestack(L, c.func), func); - - if (!oldactive) - resetbit(L->stackstate, THREAD_ACTIVEBIT); - } + status = luaD_pcall(L, f_call, &c, savestack(L, c.func), func); adjustresults(L, nresults); return status; diff --git a/VM/src/laux.cpp b/VM/src/laux.cpp index 7ed2a62e..71975a52 100644 --- a/VM/src/laux.cpp +++ b/VM/src/laux.cpp @@ -7,9 +7,12 @@ #include "lstring.h" #include "lapi.h" #include "lgc.h" +#include "lnumutils.h" #include +LUAU_FASTFLAG(LuauSchubfach) + /* convert a stack index to positive */ #define abs_index(L, i) ((i) > 0 || (i) <= LUA_REGISTRYINDEX ? (i) : lua_gettop(L) + (i) + 1) @@ -477,7 +480,17 @@ const char* luaL_tolstring(lua_State* L, int idx, size_t* len) switch (lua_type(L, idx)) { case LUA_TNUMBER: - lua_pushstring(L, lua_tostring(L, idx)); + if (FFlag::LuauSchubfach) + { + double n = lua_tonumber(L, idx); + char s[LUAI_MAXNUM2STR]; + char* e = luai_num2str(s, n); + lua_pushlstring(L, s, e - s); + } + else + { + lua_pushstring(L, lua_tostring(L, idx)); + } break; case LUA_TSTRING: lua_pushvalue(L, idx); @@ -491,11 +504,30 @@ const char* luaL_tolstring(lua_State* L, int idx, size_t* len) case LUA_TVECTOR: { const float* v = lua_tovector(L, idx); + + if (FFlag::LuauSchubfach) + { + char s[LUAI_MAXNUM2STR * LUA_VECTOR_SIZE]; + char* e = s; + for (int i = 0; i < LUA_VECTOR_SIZE; ++i) + { + if (i != 0) + { + *e++ = ','; + *e++ = ' '; + } + e = luai_num2str(e, v[i]); + } + lua_pushlstring(L, s, e - s); + } + else + { #if LUA_VECTOR_SIZE == 4 - lua_pushfstring(L, LUA_NUMBER_FMT ", " LUA_NUMBER_FMT ", " LUA_NUMBER_FMT ", " LUA_NUMBER_FMT, v[0], v[1], v[2], v[3]); + lua_pushfstring(L, LUA_NUMBER_FMT ", " LUA_NUMBER_FMT ", " LUA_NUMBER_FMT ", " LUA_NUMBER_FMT, v[0], v[1], v[2], v[3]); #else - lua_pushfstring(L, LUA_NUMBER_FMT ", " LUA_NUMBER_FMT ", " LUA_NUMBER_FMT, v[0], v[1], v[2]); + lua_pushfstring(L, LUA_NUMBER_FMT ", " LUA_NUMBER_FMT ", " LUA_NUMBER_FMT, v[0], v[1], v[2]); #endif + } break; } default: diff --git a/VM/src/lbitlib.cpp b/VM/src/lbitlib.cpp index 8b511edf..093400f2 100644 --- a/VM/src/lbitlib.cpp +++ b/VM/src/lbitlib.cpp @@ -5,8 +5,6 @@ #include "lcommon.h" #include "lnumutils.h" -LUAU_FASTFLAGVARIABLE(LuauBit32Count, false) - #define ALLONES ~0u #define NBITS int(8 * sizeof(unsigned)) @@ -182,9 +180,6 @@ static int b_replace(lua_State* L) static int b_countlz(lua_State* L) { - if (!FFlag::LuauBit32Count) - luaL_error(L, "bit32.countlz isn't enabled"); - b_uint v = luaL_checkunsigned(L, 1); b_uint r = NBITS; @@ -201,9 +196,6 @@ static int b_countlz(lua_State* L) static int b_countrz(lua_State* L) { - if (!FFlag::LuauBit32Count) - luaL_error(L, "bit32.countrz isn't enabled"); - b_uint v = luaL_checkunsigned(L, 1); b_uint r = NBITS; diff --git a/VM/src/ldebug.cpp b/VM/src/ldebug.cpp index 9fe1885f..2b5382bb 100644 --- a/VM/src/ldebug.cpp +++ b/VM/src/ldebug.cpp @@ -12,6 +12,9 @@ #include #include +LUAU_FASTFLAG(LuauBytecodeV2Read) +LUAU_FASTFLAG(LuauBytecodeV2Force) + static const char* getfuncname(Closure* f); static int currentpc(lua_State* L, CallInfo* ci) @@ -89,6 +92,16 @@ const char* lua_setlocal(lua_State* L, int level, int n) return name; } +static int getlinedefined(Proto* p) +{ + if (FFlag::LuauBytecodeV2Force) + return p->linedefined; + else if (FFlag::LuauBytecodeV2Read && p->linedefined >= 0) + return p->linedefined; + else + return luaG_getline(p, 0); +} + static int auxgetinfo(lua_State* L, const char* what, lua_Debug* ar, Closure* f, CallInfo* ci) { int status = 1; @@ -108,7 +121,7 @@ static int auxgetinfo(lua_State* L, const char* what, lua_Debug* ar, Closure* f, { ar->source = getstr(f->l.p->source); ar->what = "Lua"; - ar->linedefined = luaG_getline(f->l.p, 0); + ar->linedefined = getlinedefined(f->l.p); } luaO_chunkid(ar->short_src, ar->source, LUA_IDSIZE); break; @@ -121,7 +134,7 @@ static int auxgetinfo(lua_State* L, const char* what, lua_Debug* ar, Closure* f, } else { - ar->currentline = f->isC ? -1 : luaG_getline(f->l.p, 0); + ar->currentline = f->isC ? -1 : getlinedefined(f->l.p); } break; diff --git a/VM/src/ldo.cpp b/VM/src/ldo.cpp index 43289ab4..eb47971a 100644 --- a/VM/src/ldo.cpp +++ b/VM/src/ldo.cpp @@ -19,7 +19,6 @@ LUAU_FASTFLAGVARIABLE(LuauCcallRestoreFix, false) LUAU_FASTFLAG(LuauCoroutineClose) -LUAU_FASTFLAGVARIABLE(LuauActivateBeforeExec, true) /* ** {====================================================== @@ -228,21 +227,14 @@ void luaD_call(lua_State* L, StkId func, int nResults) { /* is a Lua function? */ L->ci->flags |= LUA_CALLINFO_RETURN; /* luau_execute will stop after returning from the stack frame */ - if (FFlag::LuauActivateBeforeExec) - { - int oldactive = luaC_threadactive(L); - l_setbit(L->stackstate, THREAD_ACTIVEBIT); - luaC_checkthreadsleep(L); + int oldactive = luaC_threadactive(L); + l_setbit(L->stackstate, THREAD_ACTIVEBIT); + luaC_checkthreadsleep(L); - luau_execute(L); /* call it */ + luau_execute(L); /* call it */ - if (!oldactive) - resetbit(L->stackstate, THREAD_ACTIVEBIT); - } - else - { - luau_execute(L); /* call it */ - } + if (!oldactive) + resetbit(L->stackstate, THREAD_ACTIVEBIT); } L->nCcalls--; luaC_checkGC(L); @@ -549,12 +541,9 @@ int luaD_pcall(lua_State* L, Pfunc func, void* u, ptrdiff_t old_top, ptrdiff_t e status = LUA_ERRERR; } - if (FFlag::LuauActivateBeforeExec) - { - // since the call failed with an error, we might have to reset the 'active' thread state - if (!oldactive) - resetbit(L->stackstate, THREAD_ACTIVEBIT); - } + // since the call failed with an error, we might have to reset the 'active' thread state + if (!oldactive) + resetbit(L->stackstate, THREAD_ACTIVEBIT); if (FFlag::LuauCcallRestoreFix) { diff --git a/VM/src/lgc.cpp b/VM/src/lgc.cpp index 7393fc74..76ef7a06 100644 --- a/VM/src/lgc.cpp +++ b/VM/src/lgc.cpp @@ -12,8 +12,6 @@ #include -LUAU_FASTFLAG(LuauArrayBoundary) - #define GC_SWEEPMAX 40 #define GC_SWEEPCOST 10 diff --git a/VM/src/lgcdebug.cpp b/VM/src/lgcdebug.cpp index f6f7a878..c66de9c1 100644 --- a/VM/src/lgcdebug.cpp +++ b/VM/src/lgcdebug.cpp @@ -12,8 +12,6 @@ #include #include -LUAU_FASTFLAG(LuauArrayBoundary) - static void validateobjref(global_State* g, GCObject* f, GCObject* t) { LUAU_ASSERT(!isdead(g, t)); @@ -38,10 +36,7 @@ static void validatetable(global_State* g, Table* h) { int sizenode = 1 << h->lsizenode; - if (FFlag::LuauArrayBoundary) - LUAU_ASSERT(h->lastfree <= sizenode); - else - LUAU_ASSERT(h->lastfree >= 0 && h->lastfree <= sizenode); + LUAU_ASSERT(h->lastfree <= sizenode); if (h->metatable) validateobjref(g, obj2gco(h), obj2gco(h->metatable)); diff --git a/VM/src/lnumprint.cpp b/VM/src/lnumprint.cpp new file mode 100644 index 00000000..2fd0f1bb --- /dev/null +++ b/VM/src/lnumprint.cpp @@ -0,0 +1,375 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +// This code is based on Lua 5.x implementation licensed under MIT License; see lua_LICENSE.txt for details +#include "luaconf.h" +#include "lnumutils.h" + +#include "lcommon.h" + +#include +#include // TODO: Remove with LuauSchubfach + +#ifdef _MSC_VER +#include +#endif + +// This work is based on: +// Raffaello Giulietti. The Schubfach way to render doubles. 2021 +// https://drive.google.com/file/d/1IEeATSVnEE6TkrHlCYNY2GjaraBjOT4f/edit + +// The code uses the notation from the paper for local variables where appropriate, and refers to paper sections/figures/results. + +LUAU_FASTFLAGVARIABLE(LuauSchubfach, false) + +// 9.8.2. Precomputed table for 128-bit overestimates of powers of 10 (see figure 3 for table bounds) +// To avoid storing 616 128-bit numbers directly we use a technique inspired by Dragonbox implementation and store 16 consecutive +// powers using a 128-bit baseline and a bitvector with 1-bit scale and 3-bit offset for the delta between each entry and base*5^k +static const int kPow10TableMin = -292; +static const int kPow10TableMax = 324; + +// clang-format off +static const uint64_t kPow5Table[16] = { + 0x8000000000000000, 0xa000000000000000, 0xc800000000000000, 0xfa00000000000000, 0x9c40000000000000, 0xc350000000000000, + 0xf424000000000000, 0x9896800000000000, 0xbebc200000000000, 0xee6b280000000000, 0x9502f90000000000, 0xba43b74000000000, + 0xe8d4a51000000000, 0x9184e72a00000000, 0xb5e620f480000000, 0xe35fa931a0000000, +}; +static const uint64_t kPow10Table[(kPow10TableMax - kPow10TableMin + 1 + 15) / 16][3] = { + {0xff77b1fcbebcdc4f, 0x25e8e89c13bb0f7b, 0x333443443333443b}, {0x8dd01fad907ffc3b, 0xae3da7d97f6792e4, 0xbbb3ab3cb3ba3cbc}, + {0x9d71ac8fada6c9b5, 0x6f773fc3603db4aa, 0x4ba4bc4bb4bb4bcc}, {0xaecc49914078536d, 0x58fae9f773886e19, 0x3ba3bc33b43b43bb}, + {0xc21094364dfb5636, 0x985915fc12f542e5, 0x33b43b43a33b33cb}, {0xd77485cb25823ac7, 0x7d633293366b828c, 0x34b44c444343443c}, + {0xef340a98172aace4, 0x86fb897116c87c35, 0x333343333343334b}, {0x84c8d4dfd2c63f3b, 0x29ecd9f40041e074, 0xccaccbbcbcbb4bbc}, + {0x936b9fcebb25c995, 0xcab10dd900beec35, 0x3ab3ab3ab3bb3bbb}, {0xa3ab66580d5fdaf5, 0xc13e60d0d2e0ebbb, 0x4cc3dc4db4db4dbb}, + {0xb5b5ada8aaff80b8, 0x0d819992132456bb, 0x33b33a34c33b34ab}, {0xc9bcff6034c13052, 0xfc89b393dd02f0b6, 0x33c33b44b43c34bc}, + {0xdff9772470297ebd, 0x59787e2b93bc56f8, 0x43b444444443434c}, {0xf8a95fcf88747d94, 0x75a44c6397ce912b, 0x443334343443343b}, + {0x8a08f0f8bf0f156b, 0x1b8e9ecb641b5900, 0xbbabab3aa3ab4ccc}, {0x993fe2c6d07b7fab, 0xe546a8038efe402a, 0x4cb4bc4db4db4bcc}, + {0xaa242499697392d2, 0xdde50bd1d5d0b9ea, 0x3ba3ba3bb33b33bc}, {0xbce5086492111aea, 0x88f4bb1ca6bcf585, 0x44b44c44c44c43cb}, + {0xd1b71758e219652b, 0xd3c36113404ea4a9, 0x44c44c44c444443b}, {0xe8d4a51000000000, 0x0000000000000000, 0x444444444444444c}, + {0x813f3978f8940984, 0x4000000000000000, 0xcccccccccccccccc}, {0x8f7e32ce7bea5c6f, 0xe4820023a2000000, 0xbba3bc4cc4cc4ccc}, + {0x9f4f2726179a2245, 0x01d762422c946591, 0x4aa3bb3aa3ba3bab}, {0xb0de65388cc8ada8, 0x3b25a55f43294bcc, 0x3ca33b33b44b43bc}, + {0xc45d1df942711d9a, 0x3ba5d0bd324f8395, 0x44c44c34c44b44cb}, {0xda01ee641a708de9, 0xe80e6f4820cc9496, 0x33b33b343333333c}, + {0xf209787bb47d6b84, 0xc0678c5dbd23a49b, 0x443444444443443b}, {0x865b86925b9bc5c2, 0x0b8a2392ba45a9b3, 0xdbccbcccb4cb3bbb}, + {0x952ab45cfa97a0b2, 0xdd945a747bf26184, 0x3bc4bb4ab3ca3cbc}, {0xa59bc234db398c25, 0x43fab9837e699096, 0x3bb3ac3ab3bb33ac}, + {0xb7dcbf5354e9bece, 0x0c11ed6d538aeb30, 0x33b43b43b34c34dc}, {0xcc20ce9bd35c78a5, 0x31ec038df7b441f5, 0x34c44c43c44b44cb}, + {0xe2a0b5dc971f303a, 0x2e44ae64840fd61e, 0x333333333333333c}, {0xfb9b7cd9a4a7443c, 0x169840ef017da3b2, 0x433344443333344c}, + {0x8bab8eefb6409c1a, 0x1ad089b6c2f7548f, 0xdcbdcc3cc4cc4bcb}, {0x9b10a4e5e9913128, 0xca7cf2b4191c8327, 0x3ab3cb3bc3bb4bbb}, + {0xac2820d9623bf429, 0x546345fa9fbdcd45, 0x3bb3cc43c43c43cb}, {0xbf21e44003acdd2c, 0xe0470a63e6bd56c4, 0x44b34a43b44c44bc}, + {0xd433179d9c8cb841, 0x5fa60692a46151ec, 0x43a33a33a333333c}, +}; +// clang-format on + +static const char kDigitTable[] = "0001020304050607080910111213141516171819202122232425262728293031323334353637383940414243444546474849" + "5051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899"; + +// x*y => 128-bit product (lo+hi) +inline uint64_t mul128(uint64_t x, uint64_t y, uint64_t* hi) +{ +#if defined(_MSC_VER) && defined(_M_X64) + return _umul128(x, y, hi); +#elif defined(__SIZEOF_INT128__) + unsigned __int128 r = x; + r *= y; + *hi = uint64_t(r >> 64); + return uint64_t(r); +#else + uint32_t x0 = uint32_t(x), x1 = uint32_t(x >> 32); + uint32_t y0 = uint32_t(y), y1 = uint32_t(y >> 32); + uint64_t p11 = uint64_t(x1) * y1, p01 = uint64_t(x0) * y1; + uint64_t p10 = uint64_t(x1) * y0, p00 = uint64_t(x0) * y0; + uint64_t mid = p10 + (p00 >> 32) + uint32_t(p01); + uint64_t r0 = (mid << 32) | uint32_t(p00); + uint64_t r1 = p11 + (mid >> 32) + (p01 >> 32); + *hi = r1; + return r0; +#endif +} + +// (x*y)>>64 => 128-bit product (lo+hi) +inline uint64_t mul192hi(uint64_t xhi, uint64_t xlo, uint64_t y, uint64_t* hi) +{ + uint64_t z2; + uint64_t z1 = mul128(xhi, y, &z2); + + uint64_t z1c; + uint64_t z0 = mul128(xlo, y, &z1c); + (void)z0; + + z1 += z1c; + z2 += (z1 < z1c); + + *hi = z2; + return z1; +} + +// 9.3. Rounding to odd (+ figure 8 + result 23) +inline uint64_t roundodd(uint64_t ghi, uint64_t glo, uint64_t cp) +{ + uint64_t xhi; + uint64_t xlo = mul128(glo, cp, &xhi); + (void)xlo; + + uint64_t yhi; + uint64_t ylo = mul128(ghi, cp, &yhi); + + uint64_t z = ylo + xhi; + return (yhi + (z < xhi)) | (z > 1); +} + +struct Decimal +{ + uint64_t s; + int k; +}; + +static Decimal schubfach(int exponent, uint64_t fraction) +{ + // Extract c & q such that c*2^q == |v| + uint64_t c = fraction; + int q = exponent - 1023 - 51; + + if (exponent != 0) // normal numbers have implicit leading 1 + { + c |= (1ull << 52); + q--; + } + + // 8.3. Fast path for integers + if (unsigned(-q) < 53 && (c & ((1ull << (-q)) - 1)) == 0) + return {c >> (-q), 0}; + + // 5. Rounding interval + int irr = (c == (1ull << 52) && q != -1074); // Qmin + int out = int(c & 1); + + // 9.8.1. Boundaries for c + uint64_t cbl = 4 * c - 2 + irr; + uint64_t cb = 4 * c; + uint64_t cbr = 4 * c + 2; + + // 9.1. Computing k and h + const int Q = 20; + const int C = 315652; // floor(2^Q * log10(2)) + const int A = -131008; // floor(2^Q * log10(3/4)) + const int C2 = 3483294; // floor(2^Q * log2(10)) + int k = (q * C + (irr ? A : 0)) >> Q; + int h = q + ((-k * C2) >> Q) + 1; // see (9) in 9.9 + + // 9.8.2. Overestimates of powers of 10 + // Recover 10^-k fraction using compact tables generated by tools/numutils.py + // The 128-bit fraction is encoded as 128-bit baseline * power-of-5 * scale + offset + LUAU_ASSERT(-k >= kPow10TableMin && -k <= kPow10TableMax); + int gtoff = -k - kPow10TableMin; + const uint64_t* gt = kPow10Table[gtoff >> 4]; + + uint64_t ghi; + uint64_t glo = mul192hi(gt[0], gt[1], kPow5Table[gtoff & 15], &ghi); + + // Apply 1-bit scale + 3-bit offset; note, offset is intentionally applied without carry, numutils.py validates that this is sufficient + int gterr = (gt[2] >> ((gtoff & 15) * 4)) & 15; + int gtscale = gterr >> 3; + + ghi <<= gtscale; + ghi += (glo >> 63) & gtscale; + glo <<= gtscale; + glo -= (gterr & 7) - 4; + + // 9.9. Boundaries for v + uint64_t vbl = roundodd(ghi, glo, cbl << h); + uint64_t vb = roundodd(ghi, glo, cb << h); + uint64_t vbr = roundodd(ghi, glo, cbr << h); + + // Main algorithm; see figure 7 + figure 9 + uint64_t s = vb / 4; + + if (s >= 10) + { + uint64_t sp = s / 10; + + bool upin = vbl + out <= 40 * sp; + bool wpin = vbr >= 40 * sp + 40 + out; + + if (upin != wpin) + return {sp + wpin, k + 1}; + } + + // Figure 7 contains the algorithm to select between u (s) and w (s+1) + // rup computes the last 4 conditions in that algorithm + // rup is only used when uin == win, but since these branches predict poorly we use branchless selects + bool uin = vbl + out <= 4 * s; + bool win = 4 * s + 4 + out <= vbr; + bool rup = vb >= 4 * s + 2 + 1 - (s & 1); + + return {s + (uin != win ? win : rup), k}; +} + +static char* printspecial(char* buf, int sign, uint64_t fraction) +{ + if (fraction == 0) + { + memcpy(buf, ("-inf") + (1 - sign), 4); + return buf + 3 + sign; + } + else + { + memcpy(buf, "nan", 4); + return buf + 3; + } +} + +static char* printunsignedrev(char* end, uint64_t num) +{ + while (num >= 10000) + { + unsigned int tail = unsigned(num % 10000); + + memcpy(end - 4, &kDigitTable[int(tail / 100) * 2], 2); + memcpy(end - 2, &kDigitTable[int(tail % 100) * 2], 2); + num /= 10000; + end -= 4; + } + + unsigned int rest = unsigned(num); + + while (rest >= 10) + { + memcpy(end - 2, &kDigitTable[int(rest % 100) * 2], 2); + rest /= 100; + end -= 2; + } + + if (rest) + { + end[-1] = '0' + int(rest); + end -= 1; + } + + return end; +} + +static char* printexp(char* buf, int num) +{ + *buf++ = 'e'; + *buf++ = num < 0 ? '-' : '+'; + + int v = num < 0 ? -num : num; + + if (v >= 100) + { + *buf++ = '0' + (v / 100); + v %= 100; + } + + memcpy(buf, &kDigitTable[v * 2], 2); + return buf + 2; +} + +inline char* trimzero(char* end) +{ + while (end[-1] == '0') + end--; + + return end; +} + +// We use fixed-length memcpy/memset since they lower to fast SIMD+scalar writes; the target buffers should have padding space +#define fastmemcpy(dst, src, size, sizefast) check_exp((size) <= sizefast, memcpy(dst, src, sizefast)) +#define fastmemset(dst, val, size, sizefast) check_exp((size) <= sizefast, memset(dst, val, sizefast)) + +char* luai_num2str(char* buf, double n) +{ + if (!FFlag::LuauSchubfach) + { + snprintf(buf, LUAI_MAXNUM2STR, LUA_NUMBER_FMT, n); + return buf + strlen(buf); + } + + // IEEE-754 + union + { + double v; + uint64_t bits; + } v = {n}; + int sign = int(v.bits >> 63); + int exponent = int(v.bits >> 52) & 2047; + uint64_t fraction = v.bits & ((1ull << 52) - 1); + + // specials + if (LUAU_UNLIKELY(exponent == 0x7ff)) + return printspecial(buf, sign, fraction); + + // sign bit + *buf = '-'; + buf += sign; + + // zero + if (exponent == 0 && fraction == 0) + { + buf[0] = '0'; + return buf + 1; + } + + // convert binary to decimal using Schubfach + Decimal d = schubfach(exponent, fraction); + LUAU_ASSERT(d.s < uint64_t(1e17)); + + // print the decimal to a temporary buffer; we'll need to insert the decimal point and figure out the format + char decbuf[40]; + char* decend = decbuf + 20; // significand needs at most 17 digits; the rest of the buffer may be copied using fixed length memcpy + char* dec = printunsignedrev(decend, d.s); + + int declen = int(decend - dec); + LUAU_ASSERT(declen <= 17); + + int dot = declen + d.k; + + // the limits are somewhat arbitrary but changing them may require changing fastmemset/fastmemcpy sizes below + if (dot >= -5 && dot <= 21) + { + // fixed point format + if (dot <= 0) + { + buf[0] = '0'; + buf[1] = '.'; + + fastmemset(buf + 2, '0', -dot, 5); + fastmemcpy(buf + 2 + (-dot), dec, declen, 17); + + return trimzero(buf + 2 + (-dot) + declen); + } + else if (dot == declen) + { + // no dot + fastmemcpy(buf, dec, dot, 17); + + return buf + dot; + } + else if (dot < declen) + { + // dot in the middle + fastmemcpy(buf, dec, dot, 16); + + buf[dot] = '.'; + + fastmemcpy(buf + dot + 1, dec + dot, declen - dot, 16); + + return trimzero(buf + declen + 1); + } + else + { + // no dot, zero padding + fastmemcpy(buf, dec, declen, 17); + fastmemset(buf + declen, '0', dot - declen, 8); + + return buf + dot; + } + } + else + { + // scientific format + buf[0] = dec[0]; + buf[1] = '.'; + fastmemcpy(buf + 2, dec + 1, declen - 1, 16); + + char* exp = trimzero(buf + declen + 1); + + return printexp(exp, dot - 1); + } +} diff --git a/VM/src/lnumutils.h b/VM/src/lnumutils.h index 67f832dc..fba07bc3 100644 --- a/VM/src/lnumutils.h +++ b/VM/src/lnumutils.h @@ -3,7 +3,6 @@ #pragma once #include -#include #define luai_numadd(a, b) ((a) + (b)) #define luai_numsub(a, b) ((a) - (b)) @@ -56,5 +55,9 @@ LUAU_FASTMATH_END #define luai_num2unsigned(i, n) ((i) = (unsigned)(long long)(n)) #endif -#define luai_num2str(s, n) snprintf((s), sizeof(s), LUA_NUMBER_FMT, (n)) +#define LUA_NUMBER_FMT "%.14g" /* TODO: Remove with LuauSchubfach */ +#define LUAI_MAXNUM2STR 48 + +LUAI_FUNC char* luai_num2str(char* buf, double n); + #define luai_str2num(s, p) strtod((s), (p)) diff --git a/VM/src/lobject.h b/VM/src/lobject.h index fd0a15b7..b642cf78 100644 --- a/VM/src/lobject.h +++ b/VM/src/lobject.h @@ -289,6 +289,7 @@ typedef struct Proto int sizek; int sizelineinfo; int linegaplog2; + int linedefined; uint8_t nups; /* number of upvalues */ diff --git a/VM/src/ltable.cpp b/VM/src/ltable.cpp index 0b55fcea..83b59f3f 100644 --- a/VM/src/ltable.cpp +++ b/VM/src/ltable.cpp @@ -24,8 +24,6 @@ #include -LUAU_FASTFLAGVARIABLE(LuauArrayBoundary, false) - // max size of both array and hash part is 2^MAXBITS #define MAXBITS 26 #define MAXSIZE (1 << MAXBITS) @@ -222,7 +220,7 @@ int luaH_next(lua_State* L, Table* t, StkId key) #define maybesetaboundary(t, boundary) \ { \ - if (FFlag::LuauArrayBoundary && t->aboundary <= 0) \ + if (t->aboundary <= 0) \ t->aboundary = -int(boundary); \ } @@ -705,7 +703,7 @@ int luaH_getn(Table* t) { int boundary = getaboundary(t); - if (FFlag::LuauArrayBoundary && boundary > 0) + if (boundary > 0) { if (!ttisnil(&t->array[t->sizearray - 1]) && t->node == dummynode) return t->sizearray; /* fast-path: the end of the array in `t' already refers to a boundary */ diff --git a/VM/src/lvmload.cpp b/VM/src/lvmload.cpp index add3588d..7839c68c 100644 --- a/VM/src/lvmload.cpp +++ b/VM/src/lvmload.cpp @@ -13,6 +13,9 @@ #include +LUAU_FASTFLAGVARIABLE(LuauBytecodeV2Read, true) +LUAU_FASTFLAGVARIABLE(LuauBytecodeV2Force, false) + // TODO: RAII deallocation doesn't work for longjmp builds if a memory error happens template struct TempBuffer @@ -146,15 +149,19 @@ int luau_load(lua_State* L, const char* chunkname, const char* data, size_t size uint8_t version = read(data, size, offset); // 0 means the rest of the bytecode is the error message - if (version == 0 || version != LBC_VERSION) + if (version == 0) { char chunkid[LUA_IDSIZE]; luaO_chunkid(chunkid, chunkname, LUA_IDSIZE); + lua_pushfstring(L, "%s%.*s", chunkid, int(size - offset), data + offset); + return 1; + } - if (version == 0) - lua_pushfstring(L, "%s%.*s", chunkid, int(size - offset), data + offset); - else - lua_pushfstring(L, "%s: bytecode version mismatch", chunkid); + if (FFlag::LuauBytecodeV2Force ? (version != LBC_VERSION_FUTURE) : FFlag::LuauBytecodeV2Read ? (version != LBC_VERSION && version != LBC_VERSION_FUTURE) : (version != LBC_VERSION)) + { + char chunkid[LUA_IDSIZE]; + luaO_chunkid(chunkid, chunkname, LUA_IDSIZE); + lua_pushfstring(L, "%s: bytecode version mismatch (expected %d, got %d)", chunkid, FFlag::LuauBytecodeV2Force ? LBC_VERSION_FUTURE : LBC_VERSION, version); return 1; } @@ -285,6 +292,11 @@ int luau_load(lua_State* L, const char* chunkname, const char* data, size_t size p->p[j] = protos[fid]; } + if (FFlag::LuauBytecodeV2Force || (FFlag::LuauBytecodeV2Read && version == LBC_VERSION_FUTURE)) + p->linedefined = readVarInt(data, size, offset); + else + p->linedefined = -1; + p->debugname = readString(strings, data, size, offset); uint8_t lineinfo = read(data, size, offset); @@ -307,11 +319,11 @@ int luau_load(lua_State* L, const char* chunkname, const char* data, size_t size p->lineinfo[j] = lastoffset; } - int lastLine = 0; + int lastline = 0; for (int j = 0; j < intervals; ++j) { - lastLine += read(data, size, offset); - p->abslineinfo[j] = lastLine; + lastline += read(data, size, offset); + p->abslineinfo[j] = lastline; } } diff --git a/VM/src/lvmutils.cpp b/VM/src/lvmutils.cpp index 5d802277..31dd59c8 100644 --- a/VM/src/lvmutils.cpp +++ b/VM/src/lvmutils.cpp @@ -34,10 +34,11 @@ int luaV_tostring(lua_State* L, StkId obj) return 0; else { - char s[LUAI_MAXNUMBER2STR]; + char s[LUAI_MAXNUM2STR]; double n = nvalue(obj); - luai_num2str(s, n); - setsvalue2s(L, obj, luaS_new(L, s)); + char* e = luai_num2str(s, n); + LUAU_ASSERT(e < s + sizeof(s)); + setsvalue2s(L, obj, luaS_newlstr(L, s, e - s)); return 1; } } diff --git a/fuzz/number.cpp b/fuzz/number.cpp new file mode 100644 index 00000000..70447409 --- /dev/null +++ b/fuzz/number.cpp @@ -0,0 +1,35 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +#include "Luau/Common.h" + +#include +#include +#include +#include + +LUAU_FASTFLAG(LuauSchubfach); + +#define LUAI_MAXNUM2STR 48 + +char* luai_num2str(char* buf, double n); + +extern "C" int LLVMFuzzerTestOneInput(const uint8_t* Data, size_t Size) +{ + if (Size < 8) + return 0; + + FFlag::LuauSchubfach.value = true; + + double num; + memcpy(&num, Data, 8); + + char buf[LUAI_MAXNUM2STR]; + char* end = luai_num2str(buf, num); + LUAU_ASSERT(end < buf + sizeof(buf)); + + *end = 0; + + double rec = strtod(buf, nullptr); + + LUAU_ASSERT(rec == num || (rec != rec && num != num)); + return 0; +} diff --git a/tests/AstQuery.test.cpp b/tests/AstQuery.test.cpp index 2090b014..41b553b5 100644 --- a/tests/AstQuery.test.cpp +++ b/tests/AstQuery.test.cpp @@ -83,8 +83,6 @@ TEST_SUITE_BEGIN("AstQuery"); TEST_CASE_FIXTURE(Fixture, "last_argument_function_call_type") { - ScopedFastFlag luauTailArgumentTypeInfo{"LuauTailArgumentTypeInfo", true}; - check(R"( local function foo() return 2 end local function bar(a: number) return -a end diff --git a/tests/Autocomplete.test.cpp b/tests/Autocomplete.test.cpp index 8ca09c0e..210db7ee 100644 --- a/tests/Autocomplete.test.cpp +++ b/tests/Autocomplete.test.cpp @@ -15,6 +15,7 @@ LUAU_FASTFLAG(LuauTraceTypesInNonstrictMode2) LUAU_FASTFLAG(LuauSetMetatableDoesNotTimeTravel) +LUAU_FASTFLAG(LuauUseCommittingTxnLog) using namespace Luau; @@ -1911,11 +1912,14 @@ local bar: @1= foo CHECK(!ac.entryMap.count("foo")); } -TEST_CASE_FIXTURE(ACFixture, "type_correct_function_no_parenthesis") +// Switch back to TEST_CASE_FIXTURE with regular ACFixture when removing the +// LuauUseCommittingTxnLog flag. +TEST_CASE("type_correct_function_no_parenthesis") { - ScopedFastFlag luauAutocompleteAvoidMutation("LuauAutocompleteAvoidMutation", true); + ScopedFastFlag sff_LuauUseCommittingTxnLog = ScopedFastFlag("LuauUseCommittingTxnLog", true); + ACFixture fix; - check(R"( + fix.check(R"( local function target(a: (number) -> number) return a(4) end local function bar1(a: number) return -a end local function bar2(a: string) return a .. 'x' end @@ -1923,7 +1927,7 @@ local function bar2(a: string) return a .. 'x' end return target(b@1 )"); - auto ac = autocomplete('1'); + auto ac = fix.autocomplete('1'); CHECK(ac.entryMap.count("bar1")); CHECK(ac.entryMap["bar1"].typeCorrect == TypeCorrectKind::Correct); @@ -1976,11 +1980,14 @@ local fp: @1= f CHECK(ac.entryMap.count("({ x: number, y: number }) -> number")); } -TEST_CASE_FIXTURE(ACFixture, "type_correct_keywords") +// Switch back to TEST_CASE_FIXTURE with regular ACFixture when removing the +// LuauUseCommittingTxnLog flag. +TEST_CASE("type_correct_keywords") { - ScopedFastFlag luauAutocompleteAvoidMutation("LuauAutocompleteAvoidMutation", true); + ScopedFastFlag sff_LuauUseCommittingTxnLog = ScopedFastFlag("LuauUseCommittingTxnLog", true); + ACFixture fix; - check(R"( + fix.check(R"( local function a(x: boolean) end local function b(x: number?) end local function c(x: (number) -> string) end @@ -1997,26 +2004,26 @@ local dc = d(f@4) local ec = e(f@5) )"); - auto ac = autocomplete('1'); + auto ac = fix.autocomplete('1'); CHECK(ac.entryMap.count("tru")); CHECK(ac.entryMap["tru"].typeCorrect == TypeCorrectKind::None); CHECK(ac.entryMap["true"].typeCorrect == TypeCorrectKind::Correct); CHECK(ac.entryMap["false"].typeCorrect == TypeCorrectKind::Correct); - ac = autocomplete('2'); + ac = fix.autocomplete('2'); CHECK(ac.entryMap.count("ni")); CHECK(ac.entryMap["ni"].typeCorrect == TypeCorrectKind::None); CHECK(ac.entryMap["nil"].typeCorrect == TypeCorrectKind::Correct); - ac = autocomplete('3'); + ac = fix.autocomplete('3'); CHECK(ac.entryMap.count("false")); CHECK(ac.entryMap["false"].typeCorrect == TypeCorrectKind::None); CHECK(ac.entryMap["function"].typeCorrect == TypeCorrectKind::Correct); - ac = autocomplete('4'); + ac = fix.autocomplete('4'); CHECK(ac.entryMap["function"].typeCorrect == TypeCorrectKind::Correct); - ac = autocomplete('5'); + ac = fix.autocomplete('5'); CHECK(ac.entryMap["function"].typeCorrect == TypeCorrectKind::Correct); } @@ -2507,21 +2514,23 @@ local t = { CHECK(ac.entryMap.count("second")); } -TEST_CASE_FIXTURE(UnfrozenFixture, "autocomplete_documentation_symbols") +TEST_CASE("autocomplete_documentation_symbols") { - loadDefinition(R"( + Fixture fix(FFlag::LuauUseCommittingTxnLog); + + fix.loadDefinition(R"( declare y: { x: number, } )"); - fileResolver.source["Module/A"] = R"( + fix.fileResolver.source["Module/A"] = R"( local a = y. )"; - frontend.check("Module/A"); + fix.frontend.check("Module/A"); - auto ac = autocomplete(frontend, "Module/A", Position{1, 21}, nullCallback); + auto ac = autocomplete(fix.frontend, "Module/A", Position{1, 21}, nullCallback); REQUIRE(ac.entryMap.count("x")); CHECK_EQ(ac.entryMap["x"].documentationSymbol, "@test/global/y.x"); diff --git a/tests/Conformance.test.cpp b/tests/Conformance.test.cpp index b055a38e..663b329e 100644 --- a/tests/Conformance.test.cpp +++ b/tests/Conformance.test.cpp @@ -13,8 +13,11 @@ #include "ScopedFlags.h" #include +#include #include +extern bool verbose; + static int lua_collectgarbage(lua_State* L) { static const char* const opts[] = {"stop", "restart", "collect", "count", "isrunning", "step", "setgoal", "setstepmul", "setstepsize", nullptr}; @@ -146,15 +149,21 @@ static StateRef runConformance(const char* name, void (*setup)(lua_State* L) = n luaL_openlibs(L); // Register a few global functions for conformance tests - static const luaL_Reg funcs[] = { + std::vector funcs = { {"collectgarbage", lua_collectgarbage}, {"loadstring", lua_loadstring}, - {"print", lua_silence}, // Disable print() by default; comment this out to enable debug prints in tests - {nullptr, nullptr}, }; + if (!verbose) + { + funcs.push_back({"print", lua_silence}); + } + + // "null" terminate the list of functions to register + funcs.push_back({nullptr, nullptr}); + lua_pushvalue(L, LUA_GLOBALSINDEX); - luaL_register(L, nullptr, funcs); + luaL_register(L, nullptr, funcs.data()); lua_pop(L, 1); // In some configurations we have a larger C stack consumption which trips some conformance tests @@ -312,8 +321,6 @@ TEST_CASE("GC") TEST_CASE("Bitwise") { - ScopedFastFlag sff("LuauBit32Count", true); - runConformance("bitwise.lua"); } @@ -491,6 +498,9 @@ TEST_CASE("DateTime") TEST_CASE("Debug") { + ScopedFastFlag sffr("LuauBytecodeV2Read", true); + ScopedFastFlag sffw("LuauBytecodeV2Write", true); + runConformance("debug.lua"); } @@ -738,8 +748,6 @@ TEST_CASE("ApiFunctionCalls") // lua_equal with a sleeping thread wake up { - ScopedFastFlag luauActivateBeforeExec("LuauActivateBeforeExec", true); - lua_State* L2 = lua_newthread(L); lua_getfield(L2, LUA_GLOBALSINDEX, "create_with_tm"); @@ -913,4 +921,11 @@ TEST_CASE("Coverage") nullptr, nullptr, &copts); } +TEST_CASE("StringConversion") +{ + ScopedFastFlag sff{"LuauSchubfach", true}; + + runConformance("strconv.lua"); +} + TEST_SUITE_END(); diff --git a/tests/Fixture.cpp b/tests/Fixture.cpp index 36d6f561..ca4281a0 100644 --- a/tests/Fixture.cpp +++ b/tests/Fixture.cpp @@ -103,11 +103,6 @@ Fixture::~Fixture() Luau::resetPrintLine(); } -UnfrozenFixture::UnfrozenFixture() - : Fixture(false) -{ -} - AstStatBlock* Fixture::parse(const std::string& source, const ParseOptions& parseOptions) { sourceModule.reset(new SourceModule); diff --git a/tests/Fixture.h b/tests/Fixture.h index de2b7381..e01632ea 100644 --- a/tests/Fixture.h +++ b/tests/Fixture.h @@ -152,15 +152,6 @@ struct Fixture LoadDefinitionFileResult loadDefinition(const std::string& source); }; -// Disables arena freezing for a given test case. -// Do not use this in new tests. If you are running into access violations, you -// are violating Luau's memory model - the fix is not to use UnfrozenFixture. -// Related: CLI-45692 -struct UnfrozenFixture : Fixture -{ - UnfrozenFixture(); -}; - ModuleName fromString(std::string_view name); template diff --git a/tests/Frontend.test.cpp b/tests/Frontend.test.cpp index 51fcd3d6..405f26e0 100644 --- a/tests/Frontend.test.cpp +++ b/tests/Frontend.test.cpp @@ -914,6 +914,8 @@ TEST_CASE_FIXTURE(FrontendFixture, "typecheck_twice_for_ast_types") TEST_CASE_FIXTURE(FrontendFixture, "imported_table_modification_2") { + ScopedFastFlag sffs("LuauSealExports", true); + frontend.options.retainFullTypeGraphs = false; fileResolver.source["Module/A"] = R"( @@ -927,7 +929,7 @@ return a; --!nonstrict local a = require(script.Parent.A) local b = {} -function a:b() end -- this should error, but doesn't +function a:b() end -- this should error, since A doesn't define a:b() return b )"; @@ -942,8 +944,7 @@ a:b() -- this should error, since A doesn't define a:b() LUAU_REQUIRE_NO_ERRORS(resultA); CheckResult resultB = frontend.check("Module/B"); - // TODO (CLI-45592): this should error, since we shouldn't be adding properties to objects from other modules - LUAU_REQUIRE_NO_ERRORS(resultB); + LUAU_REQUIRE_ERRORS(resultB); CheckResult resultC = frontend.check("Module/C"); LUAU_REQUIRE_ERRORS(resultC); diff --git a/tests/TypeInfer.annotations.test.cpp b/tests/TypeInfer.annotations.test.cpp index 71ff4e1b..275782b3 100644 --- a/tests/TypeInfer.annotations.test.cpp +++ b/tests/TypeInfer.annotations.test.cpp @@ -620,7 +620,7 @@ struct AssertionCatcher { tripped = 0; oldhook = Luau::assertHandler(); - Luau::assertHandler() = [](const char* expr, const char* file, int line) -> int { + Luau::assertHandler() = [](const char* expr, const char* file, int line, const char* function) -> int { ++tripped; return 0; }; diff --git a/tests/TypeInfer.builtins.test.cpp b/tests/TypeInfer.builtins.test.cpp index 506279b9..5e08654a 100644 --- a/tests/TypeInfer.builtins.test.cpp +++ b/tests/TypeInfer.builtins.test.cpp @@ -11,6 +11,8 @@ LUAU_FASTFLAG(LuauFixTonumberReturnType) using namespace Luau; +LUAU_FASTFLAG(LuauUseCommittingTxnLog) + TEST_SUITE_BEGIN("BuiltinTests"); TEST_CASE_FIXTURE(Fixture, "math_things_are_defined") @@ -444,19 +446,28 @@ TEST_CASE_FIXTURE(Fixture, "os_time_takes_optional_date_table") CHECK_EQ(*typeChecker.numberType, *requireType("n3")); } -TEST_CASE_FIXTURE(Fixture, "thread_is_a_type") +// Switch back to TEST_CASE_FIXTURE with regular Fixture when removing the +// LuauUseCommittingTxnLog flag. +TEST_CASE("thread_is_a_type") { - CheckResult result = check(R"( + Fixture fix(FFlag::LuauUseCommittingTxnLog); + + CheckResult result = fix.check(R"( local co = coroutine.create(function() end) )"); - LUAU_REQUIRE_NO_ERRORS(result); - CHECK_EQ(*typeChecker.threadType, *requireType("co")); + // Replace with LUAU_REQUIRE_NO_ERRORS(result) when using TEST_CASE_FIXTURE. + CHECK(result.errors.size() == 0); + CHECK_EQ(*fix.typeChecker.threadType, *fix.requireType("co")); } -TEST_CASE_FIXTURE(Fixture, "coroutine_resume_anything_goes") +// Switch back to TEST_CASE_FIXTURE with regular Fixture when removing the +// LuauUseCommittingTxnLog flag. +TEST_CASE("coroutine_resume_anything_goes") { - CheckResult result = check(R"( + Fixture fix(FFlag::LuauUseCommittingTxnLog); + + CheckResult result = fix.check(R"( local function nifty(x, y) print(x, y) local z = coroutine.yield(1, 2) @@ -469,12 +480,17 @@ TEST_CASE_FIXTURE(Fixture, "coroutine_resume_anything_goes") local answer = coroutine.resume(co, 3) )"); - LUAU_REQUIRE_NO_ERRORS(result); + // Replace with LUAU_REQUIRE_NO_ERRORS(result) when using TEST_CASE_FIXTURE. + CHECK(result.errors.size() == 0); } -TEST_CASE_FIXTURE(Fixture, "coroutine_wrap_anything_goes") +// Switch back to TEST_CASE_FIXTURE with regular Fixture when removing the +// LuauUseCommittingTxnLog flag. +TEST_CASE("coroutine_wrap_anything_goes") { - CheckResult result = check(R"( + Fixture fix(FFlag::LuauUseCommittingTxnLog); + + CheckResult result = fix.check(R"( --!nonstrict local function nifty(x, y) print(x, y) @@ -488,7 +504,8 @@ TEST_CASE_FIXTURE(Fixture, "coroutine_wrap_anything_goes") local answer = f(3) )"); - LUAU_REQUIRE_NO_ERRORS(result); + // Replace with LUAU_REQUIRE_NO_ERRORS(result) when using TEST_CASE_FIXTURE. + CHECK(result.errors.size() == 0); } TEST_CASE_FIXTURE(Fixture, "setmetatable_should_not_mutate_persisted_types") diff --git a/tests/TypeInfer.generics.test.cpp b/tests/TypeInfer.generics.test.cpp index b62044fa..114679e3 100644 --- a/tests/TypeInfer.generics.test.cpp +++ b/tests/TypeInfer.generics.test.cpp @@ -629,8 +629,6 @@ return exports TEST_CASE_FIXTURE(Fixture, "instantiated_function_argument_names") { - ScopedFastFlag luauFunctionArgumentNameSize{"LuauFunctionArgumentNameSize", true}; - CheckResult result = check(R"( local function f(a: T, ...: U...) end diff --git a/tests/TypeInfer.test.cpp b/tests/TypeInfer.test.cpp index 76324556..f70f3b1c 100644 --- a/tests/TypeInfer.test.cpp +++ b/tests/TypeInfer.test.cpp @@ -4803,8 +4803,6 @@ local bar = foo.nutrition + 100 TEST_CASE_FIXTURE(Fixture, "require_failed_module") { - ScopedFastFlag luauModuleRequireErrorPack{"LuauModuleRequireErrorPack", true}; - fileResolver.source["game/A"] = R"( return unfortunately() )"; diff --git a/tests/TypeInfer.tryUnify.test.cpp b/tests/TypeInfer.tryUnify.test.cpp index f55b46a4..1e790eba 100644 --- a/tests/TypeInfer.tryUnify.test.cpp +++ b/tests/TypeInfer.tryUnify.test.cpp @@ -12,6 +12,8 @@ LUAU_FASTFLAG(LuauQuantifyInPlace2); using namespace Luau; +LUAU_FASTFLAG(LuauUseCommittingTxnLog) + struct TryUnifyFixture : Fixture { TypeArena arena; @@ -28,7 +30,7 @@ TEST_CASE_FIXTURE(TryUnifyFixture, "primitives_unify") TypeVar numberOne{TypeVariant{PrimitiveTypeVar{PrimitiveTypeVar::Number}}}; TypeVar numberTwo = numberOne; - state.tryUnify(&numberOne, &numberTwo); + state.tryUnify(&numberTwo, &numberOne); CHECK(state.errors.empty()); } @@ -41,9 +43,12 @@ TEST_CASE_FIXTURE(TryUnifyFixture, "compatible_functions_are_unified") TypeVar functionTwo{TypeVariant{ FunctionTypeVar(arena.addTypePack({arena.freshType(globalScope->level)}), arena.addTypePack({arena.freshType(globalScope->level)}))}}; - state.tryUnify(&functionOne, &functionTwo); + state.tryUnify(&functionTwo, &functionOne); CHECK(state.errors.empty()); + if (FFlag::LuauUseCommittingTxnLog) + state.log.commit(); + CHECK_EQ(functionOne, functionTwo); } @@ -61,7 +66,7 @@ TEST_CASE_FIXTURE(TryUnifyFixture, "incompatible_functions_are_preserved") TypeVar functionTwoSaved = functionTwo; - state.tryUnify(&functionOne, &functionTwo); + state.tryUnify(&functionTwo, &functionOne); CHECK(!state.errors.empty()); CHECK_EQ(functionOne, functionOneSaved); @@ -80,10 +85,13 @@ TEST_CASE_FIXTURE(TryUnifyFixture, "tables_can_be_unified") CHECK_NE(*getMutable(&tableOne)->props["foo"].type, *getMutable(&tableTwo)->props["foo"].type); - state.tryUnify(&tableOne, &tableTwo); + state.tryUnify(&tableTwo, &tableOne); CHECK(state.errors.empty()); + if (FFlag::LuauUseCommittingTxnLog) + state.log.commit(); + CHECK_EQ(*getMutable(&tableOne)->props["foo"].type, *getMutable(&tableTwo)->props["foo"].type); } @@ -101,11 +109,12 @@ TEST_CASE_FIXTURE(TryUnifyFixture, "incompatible_tables_are_preserved") CHECK_NE(*getMutable(&tableOne)->props["foo"].type, *getMutable(&tableTwo)->props["foo"].type); - state.tryUnify(&tableOne, &tableTwo); + state.tryUnify(&tableTwo, &tableOne); CHECK_EQ(1, state.errors.size()); - state.log.rollback(); + if (!FFlag::LuauUseCommittingTxnLog) + state.DEPRECATED_log.rollback(); CHECK_NE(*getMutable(&tableOne)->props["foo"].type, *getMutable(&tableTwo)->props["foo"].type); } @@ -170,7 +179,7 @@ TEST_CASE_FIXTURE(TryUnifyFixture, "variadic_type_pack_unification") TypePackVar testPack{TypePack{{typeChecker.numberType, typeChecker.stringType}, std::nullopt}}; TypePackVar variadicPack{VariadicTypePack{typeChecker.numberType}}; - state.tryUnify(&variadicPack, &testPack); + state.tryUnify(&testPack, &variadicPack); CHECK(!state.errors.empty()); } @@ -180,7 +189,7 @@ TEST_CASE_FIXTURE(TryUnifyFixture, "variadic_tails_respect_progress") TypePackVar a{TypePack{{typeChecker.numberType, typeChecker.stringType, typeChecker.booleanType, typeChecker.booleanType}}}; TypePackVar b{TypePack{{typeChecker.numberType, typeChecker.stringType}, &variadicPack}}; - state.tryUnify(&a, &b); + state.tryUnify(&b, &a); CHECK(state.errors.empty()); } @@ -214,32 +223,41 @@ TEST_CASE_FIXTURE(TryUnifyFixture, "cli_41095_concat_log_in_sealed_table_unifica CHECK_EQ(toString(result.errors[1]), "Available overloads: ({a}, a) -> (); and ({a}, number, a) -> ()"); } -TEST_CASE_FIXTURE(TryUnifyFixture, "undo_new_prop_on_unsealed_table") +TEST_CASE("undo_new_prop_on_unsealed_table") { ScopedFastFlag flags[] = { {"LuauTableSubtypingVariance2", true}, + // This test makes no sense with a committing TxnLog. + {"LuauUseCommittingTxnLog", false}, }; // I am not sure how to make this happen in Luau code. - TypeId unsealedTable = arena.addType(TableTypeVar{TableState::Unsealed, TypeLevel{}}); - TypeId sealedTable = arena.addType(TableTypeVar{ - {{"prop", Property{getSingletonTypes().numberType}}}, - std::nullopt, - TypeLevel{}, - TableState::Sealed - }); + TryUnifyFixture fix; + + TypeId unsealedTable = fix.arena.addType(TableTypeVar{TableState::Unsealed, TypeLevel{}}); + TypeId sealedTable = + fix.arena.addType(TableTypeVar{{{"prop", Property{getSingletonTypes().numberType}}}, std::nullopt, TypeLevel{}, TableState::Sealed}); const TableTypeVar* ttv = get(unsealedTable); REQUIRE(ttv); - state.tryUnify(unsealedTable, sealedTable); + fix.state.tryUnify(sealedTable, unsealedTable); // To be honest, it's really quite spooky here that we're amending an unsealed table in this case. CHECK(!ttv->props.empty()); - state.log.rollback(); + fix.state.DEPRECATED_log.rollback(); CHECK(ttv->props.empty()); } +TEST_CASE_FIXTURE(TryUnifyFixture, "free_tail_is_grown_properly") +{ + TypePackId threeNumbers = arena.addTypePack(TypePack{{typeChecker.numberType, typeChecker.numberType, typeChecker.numberType}, std::nullopt}); + TypePackId numberAndFreeTail = arena.addTypePack(TypePack{{typeChecker.numberType}, arena.addTypePack(TypePackVar{FreeTypePack{TypeLevel{}}})}); + + ErrorVec unifyErrors = state.canUnify(numberAndFreeTail, threeNumbers); + CHECK(unifyErrors.size() == 0); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.typePacks.cpp b/tests/TypeInfer.typePacks.cpp index 3f4420cd..5d37b032 100644 --- a/tests/TypeInfer.typePacks.cpp +++ b/tests/TypeInfer.typePacks.cpp @@ -10,6 +10,8 @@ using namespace Luau; +LUAU_FASTFLAG(LuauUseCommittingTxnLog) + TEST_SUITE_BEGIN("TypePackTests"); TEST_CASE_FIXTURE(Fixture, "infer_multi_return") @@ -263,10 +265,13 @@ TEST_CASE_FIXTURE(Fixture, "variadic_pack_syntax") CHECK_EQ(toString(requireType("foo")), "(...number) -> ()"); } -// CLI-45791 -TEST_CASE_FIXTURE(UnfrozenFixture, "type_pack_hidden_free_tail_infinite_growth") +// Switch back to TEST_CASE_FIXTURE with regular Fixture when removing the +// LuauUseCommittingTxnLog flag. +TEST_CASE("type_pack_hidden_free_tail_infinite_growth") { - CheckResult result = check(R"( + Fixture fix(FFlag::LuauUseCommittingTxnLog); + + CheckResult result = fix.check(R"( --!nonstrict if _ then _[function(l0)end],l0 = _ @@ -278,7 +283,8 @@ elseif _ then end )"); - LUAU_REQUIRE_ERRORS(result); + // Switch back to LUAU_REQUIRE_ERRORS(result) when using TEST_CASE_FIXTURE. + CHECK(result.errors.size() > 0); } TEST_CASE_FIXTURE(Fixture, "variadic_argument_tail") diff --git a/tests/TypeInfer.unionTypes.test.cpp b/tests/TypeInfer.unionTypes.test.cpp index 2357869e..b54ba996 100644 --- a/tests/TypeInfer.unionTypes.test.cpp +++ b/tests/TypeInfer.unionTypes.test.cpp @@ -8,6 +8,7 @@ #include "doctest.h" LUAU_FASTFLAG(LuauEqConstraint) +LUAU_FASTFLAG(LuauUseCommittingTxnLog) using namespace Luau; @@ -282,16 +283,19 @@ local c = b:foo(1, 2) CHECK_EQ("Value of type 'A?' could be nil", toString(result.errors[0])); } -TEST_CASE_FIXTURE(UnfrozenFixture, "optional_union_follow") +TEST_CASE("optional_union_follow") { - CheckResult result = check(R"( + Fixture fix(FFlag::LuauUseCommittingTxnLog); + + CheckResult result = fix.check(R"( local y: number? = 2 local x = y local function f(a: number, b: typeof(x), c: typeof(x)) return -a end return f() )"); - LUAU_REQUIRE_ERROR_COUNT(1, result); + REQUIRE_EQ(result.errors.size(), 1); + // LUAU_REQUIRE_ERROR_COUNT(1, result); auto acm = get(result.errors[0]); REQUIRE(acm); diff --git a/tests/TypeVar.test.cpp b/tests/TypeVar.test.cpp index 13db923e..2e0d149e 100644 --- a/tests/TypeVar.test.cpp +++ b/tests/TypeVar.test.cpp @@ -185,6 +185,8 @@ TEST_CASE_FIXTURE(Fixture, "UnionTypeVarIterator_with_empty_union") TEST_CASE_FIXTURE(Fixture, "substitution_skip_failure") { + ScopedFastFlag sff{"LuauSealExports", true}; + TypeVar ftv11{FreeTypeVar{TypeLevel{}}}; TypePackVar tp24{TypePack{{&ftv11}}}; @@ -261,7 +263,7 @@ TEST_CASE_FIXTURE(Fixture, "substitution_skip_failure") TypeId result = typeChecker.anyify(typeChecker.globalScope, root, Location{}); - CHECK_EQ("{ f: t1 } where t1 = () -> { f: () -> { f: ({ f: t1 }) -> (), signal: { f: (any) -> () } } }", toString(result)); + CHECK_EQ("{| f: t1 |} where t1 = () -> {| f: () -> {| f: ({| f: t1 |}) -> (), signal: {| f: (any) -> () |} |} |}", toString(result)); } TEST_CASE("tagging_tables") diff --git a/tests/conformance/debug.lua b/tests/conformance/debug.lua index 9cf3c742..8c96ab33 100644 --- a/tests/conformance/debug.lua +++ b/tests/conformance/debug.lua @@ -98,4 +98,13 @@ assert(quuz(function(...) end) == "0 true") assert(quuz(function(a, b) end) == "2 false") assert(quuz(function(a, b, ...) end) == "2 true") +-- info linedefined & line +function testlinedefined() + local line = debug.info(1, "l") + local linedefined = debug.info(testlinedefined, "l") + assert(linedefined + 1 == line) +end + +testlinedefined() + return 'OK' diff --git a/tests/conformance/strconv.lua b/tests/conformance/strconv.lua new file mode 100644 index 00000000..85ad0295 --- /dev/null +++ b/tests/conformance/strconv.lua @@ -0,0 +1,51 @@ +-- This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +-- This file is based on Lua 5.x tests -- https://github.com/lua/lua/tree/master/testes +print("testing string-number conversion") + +-- zero +assert(tostring(0) == "0") +assert(tostring(0/-1) == "-0") + +-- specials +assert(tostring(1/0) == "inf") +assert(tostring(-1/0) == "-inf") +assert(tostring(0/0) == "nan") + +-- integers +assert(tostring(1) == "1") +assert(tostring(42) == "42") +assert(tostring(-4294967296) == "-4294967296") +assert(tostring(9007199254740991) == "9007199254740991") + +-- decimals +assert(tostring(0.5) == "0.5") +assert(tostring(0.1) == "0.1") +assert(tostring(-0.17) == "-0.17") +assert(tostring(math.pi) == "3.141592653589793") + +-- fuzzing corpus +assert(tostring(5.4536123983019448e-311) == "5.453612398302e-311") +assert(tostring(5.4834368411298348e-311) == "5.48343684113e-311") +assert(tostring(4.4154895841930002e-305) == "4.415489584193e-305") +assert(tostring(1125968630513728) == "1125968630513728") +assert(tostring(3.3951932655938423e-313) == "3.3951932656e-313") +assert(tostring(1.625) == "1.625") +assert(tostring(4.9406564584124654e-324) == "5.e-324") +assert(tostring(2.0049288280105384) == "2.0049288280105384") +assert(tostring(3.0517578125e-05) == "0.000030517578125") +assert(tostring(1.383544921875) == "1.383544921875") +assert(tostring(3.0053350932691001) == "3.0053350932691") +assert(tostring(0.0001373291015625) == "0.0001373291015625") +assert(tostring(-1.9490628022799998e+289) == "-1.94906280228e+289") +assert(tostring(-0.00610404721867928) == "-0.00610404721867928") +assert(tostring(0.00014495849609375) == "0.00014495849609375") +assert(tostring(0.453125) == "0.453125") +assert(tostring(-4.2375343999999997e+73) == "-4.2375344e+73") +assert(tostring(1.3202313930270133e-192) == "1.3202313930270133e-192") +assert(tostring(3.6984408976312836e+19) == "36984408976312840000") +assert(tostring(2.0563000527063302) == "2.05630005270633") +assert(tostring(4.8970527433648997e-260) == "4.8970527433649e-260") +assert(tostring(1.62890625) == "1.62890625") +assert(tostring(1.1295093211933533e+65) == "1.1295093211933533e+65") + +return "OK" diff --git a/tests/main.cpp b/tests/main.cpp index ed17070c..cd24e100 100644 --- a/tests/main.cpp +++ b/tests/main.cpp @@ -2,6 +2,8 @@ #include "Luau/Common.h" #define DOCTEST_CONFIG_IMPLEMENT +// Our calls to parseOption/parseFlag don't provide a prefix so set the prefix to the empty string. +#define DOCTEST_CONFIG_OPTIONS_PREFIX "" #include "doctest.h" #ifdef _WIN32 @@ -18,6 +20,10 @@ #include +// Indicates if verbose output is enabled. +// Currently, this enables output from lua's 'print', but other verbose output could be enabled eventually. +bool verbose = false; + static bool skipFastFlag(const char* flagName) { if (strncmp(flagName, "Test", 4) == 0) @@ -46,7 +52,7 @@ static bool debuggerPresent() #endif } -static int assertionHandler(const char* expr, const char* file, int line) +static int assertionHandler(const char* expr, const char* file, int line, const char* function) { if (debuggerPresent()) LUAU_DEBUGBREAK(); @@ -235,6 +241,11 @@ int main(int argc, char** argv) return 0; } + if (doctest::parseFlag(argc, argv, "--verbose")) + { + verbose = true; + } + if (std::vector flags; doctest::parseCommaSepArgs(argc, argv, "--fflags=", flags)) setFastFlags(flags); @@ -261,7 +272,15 @@ int main(int argc, char** argv) } } - return context.run(); + int result = context.run(); + if (doctest::parseFlag(argc, argv, "--help") || doctest::parseFlag(argc, argv, "-h")) + { + printf("Additional command line options:\n"); + printf(" --verbose Enables verbose output (e.g. lua 'print' statements)\n"); + printf(" --fflags= Sets specified fast flags\n"); + printf(" --list-fflags List all fast flags\n"); + } + return result; } diff --git a/tools/numprint.py b/tools/numprint.py new file mode 100644 index 00000000..47ad36d9 --- /dev/null +++ b/tools/numprint.py @@ -0,0 +1,82 @@ +#!/usr/bin/python +# This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details + +# This code can be used to generate power tables for Schubfach algorithm (see lnumprint.cpp) + +import math +import sys + +(_, pow10min, pow10max, compact) = sys.argv +pow10min = int(pow10min) +pow10max = int(pow10max) +compact = compact == "True" + +# extract high 128 bits of the value +def high128(n, roundup): + L = math.ceil(math.log2(n)) + + r = 0 + for i in range(L - 128, L): + if i >= 0 and (n & (1 << i)) != 0: + r |= (1 << (i - L + 128)) + + return r + (1 if roundup else 0) + +def pow10approx(n): + if n == 0: + return 1 << 127 + elif n > 0: + return high128(10**n, 5**n >= 2**128) + else: + # 10^-n is a binary fraction that can't be represented in floating point + # we need to extract top 128 bits of the fraction starting from the first 1 + # to get there, we need to divide 2^k by 10^n for a sufficiently large k and repeat the extraction process + p = 10**-n + k = 2**128 * 16**-n # this guarantees that the fraction has more than 128 extra bits + return high128(k // p, True) + +def pow5_64(n): + assert(n >= 0) + if n == 0: + return 1 << 63 + else: + return high128(5**n, False) >> 64 + +if not compact: + print("// kPow10Table", pow10min, "..", pow10max) + print("{") + for p in range(pow10min, pow10max + 1): + h = hex(pow10approx(p))[2:] + assert(len(h) == 32) + print(" {0x%s, 0x%s}," % (h[0:16].upper(), h[16:32].upper())) + print("}") +else: + print("// kPow5Table") + print("{") + for i in range(16): + print(" " + hex(pow5_64(i)) + ",") + print("}") + print("// kPow10Table", pow10min, "..", pow10max) + print("{") + for p in range(pow10min, pow10max + 1, 16): + base = pow10approx(p) + errw = 0 + for i in range(16): + real = pow10approx(p + i) + appr = (base * pow5_64(i)) >> 64 + scale = 1 if appr < (1 << 127) else 0 # 1-bit scale + + offset = (appr << scale) - real + assert(offset >= -4 and offset <= 3) # 3-bit offset + assert((appr << scale) >> 64 == real >> 64) # offset only affects low half + assert((appr << scale) - offset == real) # validate full reconstruction + + err = (scale << 3) | (offset + 4) + errw |= err << (i * 4) + + hbase = hex(base)[2:] + assert(len(hbase) == 32) + assert(errw < 1 << 64) + + print(" {0x%s, 0x%s, 0x%16x}," % (hbase[0:16], hbase[16:32], errw)) + print("}") From b5e338325beee318e243387b1514e0137b19b277 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Fri, 7 Jan 2022 08:52:33 -0800 Subject: [PATCH 119/399] Mark coroutine.close RFC as implemented (#304) --- rfcs/function-coroutine-close.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/rfcs/function-coroutine-close.md b/rfcs/function-coroutine-close.md index 635050c4..6def1533 100644 --- a/rfcs/function-coroutine-close.md +++ b/rfcs/function-coroutine-close.md @@ -4,6 +4,8 @@ Add `coroutine.close` function from Lua 5.4 that takes a suspended coroutine and makes it "dead" (non-runnable). +**Status**: Implemented + ## Motivation When implementing various higher level objects on top of coroutines, such as promises, it can be useful to cancel the coroutine execution externally - when the caller is not From 287b46e6a7e4b85fd9388be8b6f7e1d5d3ced0bd Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Fri, 7 Jan 2022 11:07:22 -0800 Subject: [PATCH 120/399] Mark bidirectional ascription RFC as implemented (#305) --- rfcs/syntax-type-ascription-bidi.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/rfcs/syntax-type-ascription-bidi.md b/rfcs/syntax-type-ascription-bidi.md index 9d31939e..bf37eca2 100644 --- a/rfcs/syntax-type-ascription-bidi.md +++ b/rfcs/syntax-type-ascription-bidi.md @@ -4,6 +4,8 @@ The way `::` works today is really strange. The best solution we can come up with is to allow `::` to convert between any two related types. +**Status**: Implemented + ## Motivation Due to an accident of the implementation, the Luau `::` operator can only be used for downcasts and casts to `any`. From a23b46748557a1f224ac1d82692ad444dda48cda Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Fri, 7 Jan 2022 11:07:36 -0800 Subject: [PATCH 121/399] Add turbofish discussion to generic function RFC (#300) --- rfcs/generic-functions.md | 34 ++++++++++++++++++++++++++++++++++ 1 file changed, 34 insertions(+) diff --git a/rfcs/generic-functions.md b/rfcs/generic-functions.md index 59ec2d4a..3ac1bbba 100644 --- a/rfcs/generic-functions.md +++ b/rfcs/generic-functions.md @@ -32,6 +32,12 @@ Functions may also take generic type pack arguments for varargs, for instance: function compose(... : a...) -> (a...) return ... end ``` +Generic type and type pack parameters can also be used in function types, for instance: + +```lua +local id: (a)->a = function(x) return x end +``` + This change is *not* only syntax, as explicit type parameters need to be part of the semantics of types. For example, we can define a generic identity function ```lua @@ -86,6 +92,34 @@ Currently, Luau does not have explicit type binders, so `f` and `g` have the sam We propose supporting type parameters which can be instantiated with any type (jargon: Rank-N Types) but not type functions (jargon: Higher Kinded Types) or types with constraints (jargon: F-bounded polymorphism). +## Turbofish + +Note that this RFC proposes a syntax for adding generic parameters to functions, but it does *not* propose syntax for adding generic arguments to function call site. For example, for `id` function you *can* write: + +```lua + -- generic type gets inferred as a number in all these cases +local x = id(4) +local x = id(y) :: number +local x: number = id(y) +``` + +but you can *not* write `id(y)`. + +This syntax is difficult to parse as it's ambiguous wrt grammar for comparison, and disambiguating it requires being able to parse types in expression context which makes parsing slow and complicated. It's also worth noting that today there are programs with this syntax that are grammatically correct (eg `id('4')` parses as "compare variable `id` to variable `string`, and compare the result to string '4'"). The specific example with a single argument will always fail at runtime because booleans can't be compared with relational operators, but multi-argument cases such as `print(foo(4))` can execute without errors in certain cases. + +Note that in many cases the types can be inferred, whether through function arguments (`id(4)`) or through expected return type (`id(y) :: number`). It's also often possible to cast the function object to a given type, even though that can be unwieldy (`(id :: (number)->number)(y)`). Some languages don't have a way to specify the types at call site either, Swift being a prominent example. Thus it's not a given we need this feature in Luau. + +If we ever want to implement this though, we can use a solution inspired by Rust's turbofish and require an extra token before `<`. Rust uses `::<` but that doesn't work in Luau because as part of this RFC, `id::(a)->a` is a valid, if redundant, type ascription, so we need to choose a different prefix. + +The following two variants are grammatically unambiguous in expression context in Luau, and are a better parallel for Rust's turbofish (in Rust, `::` is more similar to Luau's `:` or `.` than `::`, which in Rust is called `as`): + +```lua +foo:() -- require : before <; this is only valid in Luau in variable declaration context, so it's safe to use in expression context +foo.() -- require . before <; this is currently never valid in Luau +``` + +This RFC doesn't propose using either of these options, but notes that either one of these options is possible to specify & implement in the future if we so desire. + ## Drawbacks This is a breaking change, in that examples like the unsound program above will no longer typecheck. From b7a7b92d12f81a1e5d1aba6f78b451400414979f Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Tue, 11 Jan 2022 13:24:56 -0800 Subject: [PATCH 122/399] Update syntax.md Remove confusing block because it reads as if we *do* support this syntax if you aren't reading carefully. --- docs/_pages/syntax.md | 12 ++---------- 1 file changed, 2 insertions(+), 10 deletions(-) diff --git a/docs/_pages/syntax.md b/docs/_pages/syntax.md index e5d5a0a7..e71dc1cc 100644 --- a/docs/_pages/syntax.md +++ b/docs/_pages/syntax.md @@ -21,21 +21,13 @@ local function tree_insert(tree, x) end ``` -Note that future versions of Lua extend the Lua 5.1 syntax with the following features; Luau does support string literal extensions but does not support other features from this list: - -- hexadecimal (`\x`), Unicode (`\u`) and `\z` escapes for string literals -- goto statements and labels -- bitwise operators -- floor division operator (`//`) -- `` and `` local attributes - -> For details please refer to [compatibility section](compatibility). +Note that future versions of Lua extend the Lua 5.1 syntax with more features; Luau does support string literal extensions but does not support other 5.x additions; for details please refer to [compatibility section](compatibility). The rest of this document documents additional syntax used in Luau. ## String literals -As noted above, Luau implements support for hexadecimal (`\0x`), Unicode (`\u`) and `\z` escapes for string literals. This syntax follows [Lua 5.3 syntax](https://www.lua.org/manual/5.3/manual.html#3.1): +Luau implements support for hexadecimal (`\0x`), Unicode (`\u`) and `\z` escapes for string literals. This syntax follows [Lua 5.3 syntax](https://www.lua.org/manual/5.3/manual.html#3.1): - `\xAB` inserts a character with the code 0xAB into the string - `\u{ABC}` inserts a UTF8 byte sequence that encodes U+0ABC character into the string (note that braces are mandatory) From b7f78f49974d8684ee7d89934de5ae911b25389d Mon Sep 17 00:00:00 2001 From: rafa_br34 <66086623+rafabr34@users.noreply.github.com> Date: Wed, 12 Jan 2022 15:05:31 -0300 Subject: [PATCH 123/399] MSVC warning C4244 fixes (#308) Co-authored-by: Arseny Kapoulkine --- Compiler/src/BytecodeBuilder.cpp | 6 +++--- Compiler/src/Compiler.cpp | 16 ++++++++-------- 2 files changed, 11 insertions(+), 11 deletions(-) diff --git a/Compiler/src/BytecodeBuilder.cpp b/Compiler/src/BytecodeBuilder.cpp index 2d31c409..e6d02454 100644 --- a/Compiler/src/BytecodeBuilder.cpp +++ b/Compiler/src/BytecodeBuilder.cpp @@ -714,7 +714,7 @@ void BytecodeBuilder::writeLineInfo(std::string& ss) const // third pass: write resulting data int logspan = log2(span); - writeByte(ss, logspan); + writeByte(ss, uint8_t(logspan)); uint8_t lastOffset = 0; @@ -723,8 +723,8 @@ void BytecodeBuilder::writeLineInfo(std::string& ss) const int delta = lines[i] - baseline[i >> logspan]; LUAU_ASSERT(delta >= 0 && delta <= 255); - writeByte(ss, delta - lastOffset); - lastOffset = delta; + writeByte(ss, uint8_t(delta) - lastOffset); + lastOffset = uint8_t(delta); } int lastLine = 0; diff --git a/Compiler/src/Compiler.cpp b/Compiler/src/Compiler.cpp index 6ae49027..0156fcf5 100644 --- a/Compiler/src/Compiler.cpp +++ b/Compiler/src/Compiler.cpp @@ -364,12 +364,12 @@ struct Compiler else { args[i] = uint8_t(regs + 1 + i); - compileExprTempTop(expr->args.data[i], args[i]); + compileExprTempTop(expr->args.data[i], uint8_t(args[i])); } } fastcallLabel = bytecode.emitLabel(); - bytecode.emitABC(opc, uint8_t(bfid), args[0], 0); + bytecode.emitABC(opc, uint8_t(bfid), uint8_t(args[0]), 0); if (opc != LOP_FASTCALL1) bytecode.emitAux(args[1]); @@ -385,7 +385,7 @@ struct Compiler } if (args[i] != regs + 1 + i) - bytecode.emitABC(LOP_MOVE, uint8_t(regs + 1 + i), args[i], 0); + bytecode.emitABC(LOP_MOVE, uint8_t(regs + 1 + i), uint8_t(args[i]), 0); } } else @@ -471,7 +471,7 @@ struct Compiler if (cid >= 0 && cid < 32768) { - bytecode.emitAD(LOP_DUPCLOSURE, target, cid); + bytecode.emitAD(LOP_DUPCLOSURE, target, int16_t(cid)); shared = true; } } @@ -493,7 +493,7 @@ struct Compiler // get local variable uint8_t reg = getLocal(uv); - bytecode.emitABC(LOP_CAPTURE, immutable ? LCT_VAL : LCT_REF, reg, 0); + bytecode.emitABC(LOP_CAPTURE, uint8_t(immutable ? LCT_VAL : LCT_REF), reg, 0); } else { @@ -635,7 +635,7 @@ struct Compiler if (expr->op == AstExprBinary::CompareGt || expr->op == AstExprBinary::CompareGe) { - bytecode.emitAD(opc, rr, 0); + bytecode.emitAD(opc, uint8_t(rr), 0); bytecode.emitAux(rl); } else @@ -1144,7 +1144,7 @@ struct Compiler } else { - bytecode.emitABC(LOP_NEWTABLE, reg, encodedHashSize, 0); + bytecode.emitABC(LOP_NEWTABLE, reg, uint8_t(encodedHashSize), 0); bytecode.emitAux(0); } } @@ -1157,7 +1157,7 @@ struct Compiler bool trailingVarargs = last && last->kind == AstExprTable::Item::List && last->value->is(); LUAU_ASSERT(!trailingVarargs || arraySize > 0); - bytecode.emitABC(LOP_NEWTABLE, reg, encodedHashSize, 0); + bytecode.emitABC(LOP_NEWTABLE, reg, uint8_t(encodedHashSize), 0); bytecode.emitAux(arraySize - trailingVarargs + indexSize); } From abf9fc2754d603e5a1ece15aba36a4e7afb58260 Mon Sep 17 00:00:00 2001 From: T 'Filtered' C Date: Wed, 12 Jan 2022 19:50:25 +0000 Subject: [PATCH 124/399] Update compatibility.md to split coroutine.close from lua_resetthread (#309) Co-authored-by: Arseny Kapoulkine --- docs/_pages/compatibility.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/_pages/compatibility.md b/docs/_pages/compatibility.md index 70ca77cc..a9cd9151 100644 --- a/docs/_pages/compatibility.md +++ b/docs/_pages/compatibility.md @@ -107,7 +107,7 @@ Floor division is less harmful, but it's used rarely enough that `math.floor(a/b | const variables | ❌ | while there's some demand for const variables, we'd never adopt this syntax | | new implementation for math.random | ✔️ | our RNG is based on PCG, unlike Lua 5.4 which uses Xoroshiro | | optional `init` argument to `string.gmatch` | 🤷‍♀️ | no strong use cases | -| new functions `lua_resetthread` and `coroutine.close` | 🤷‍ | not useful without to-be-closed variables | +| new functions `lua_resetthread` and `coroutine.close` | ✔️ || | coercions string-to-number moved to the string library | 😞 | we love this, but it breaks compatibility | | new format `%p` in `string.format` | 🤷‍♀️ | no strong use cases | | `utf8` library accepts codepoints up to 2^31 | 🤷‍♀️ | no strong use cases | From d6ba106be69091632a257b19bf29541b4542fd92 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Wed, 12 Jan 2022 11:56:46 -0800 Subject: [PATCH 125/399] Update compatibility.md Add a note about function identity --- docs/_pages/compatibility.md | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/_pages/compatibility.md b/docs/_pages/compatibility.md index a9cd9151..06d64b76 100644 --- a/docs/_pages/compatibility.md +++ b/docs/_pages/compatibility.md @@ -127,4 +127,5 @@ We have a few behavior deviations from Lua 5.x that come from either a different * Tail calls are not supported to simplify implementation, make debugging/stack traces more predictable and allow deep validation of caller identity for security * Order of table assignment in table literals follows program order in mixed tables (Lua 5.x assigns array elements first in some cases) * Equality comparisons call `__eq` metamethod even when objects are rawequal (which matches other metamethods like `<=` and facilitates NaN checking) +* `function()` expressions may reuse a previosly created closure in certain scenarios (when all upvalues captured are the same) for efficiency, which changes object identity but doesn't change call semantics -- this is different from Lua 5.1 but similar to Lua 5.2/5.3 * `os.time` returns UTC timestamp when called with a table for consistency From b2af550b08d6098b72b638908c4552dbe7cfa7a3 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Thu, 13 Jan 2022 15:23:18 -0800 Subject: [PATCH 126/399] Update grammar.md Add forgotten quotes around 'type'; reported by @Dionysusnu --- docs/_pages/grammar.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/_pages/grammar.md b/docs/_pages/grammar.md index d1767934..22b078a1 100644 --- a/docs/_pages/grammar.md +++ b/docs/_pages/grammar.md @@ -24,7 +24,7 @@ stat = varlist '=' explist | 'function' funcname funcbody | 'local' 'function' NAME funcbody | 'local' bindinglist ['=' explist] | - ['export'] type NAME ['<' GenericTypeList '>'] '=' Type + ['export'] 'type' NAME ['<' GenericTypeList '>'] '=' Type laststat = 'return' [explist] | 'break' | 'continue' From 80d5c0000ee34767f8fec7ec24c02a438174a667 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Fri, 14 Jan 2022 08:06:31 -0800 Subject: [PATCH 127/399] Sync to upstream/release/510 --- Analysis/include/Luau/TypeInfer.h | 12 +- Analysis/include/Luau/TypeVar.h | 20 +- Analysis/src/Autocomplete.cpp | 79 +-- Analysis/src/Error.cpp | 12 +- Analysis/src/IostreamHelpers.cpp | 8 +- Analysis/src/JsonEncoder.cpp | 38 ++ Analysis/src/Module.cpp | 26 +- Analysis/src/ToString.cpp | 107 +++- Analysis/src/Transpiler.cpp | 65 +- Analysis/src/TypeAttach.cpp | 12 +- Analysis/src/TypeInfer.cpp | 310 ++++++++-- Analysis/src/TypeVar.cpp | 4 + Analysis/src/Unifier.cpp | 17 +- Ast/include/Luau/Ast.h | 49 +- Ast/include/Luau/Parser.h | 6 +- Ast/src/Ast.cpp | 32 +- Ast/src/Parser.cpp | 128 ++-- CLI/Analyze.cpp | 18 +- CLI/Repl.cpp | 137 ++-- Compiler/src/Builtins.cpp | 197 ++++++ Compiler/src/Builtins.h | 41 ++ Compiler/src/BytecodeBuilder.cpp | 6 +- Compiler/src/Compiler.cpp | 894 ++------------------------- Compiler/src/ConstantFolding.cpp | 394 ++++++++++++ Compiler/src/ConstantFolding.h | 48 ++ Compiler/src/TableShape.cpp | 129 ++++ Compiler/src/TableShape.h | 21 + Compiler/src/ValueTracking.cpp | 103 +++ Compiler/src/ValueTracking.h | 42 ++ Sources.cmake | 8 + VM/src/ldo.cpp | 12 +- VM/src/lfunc.cpp | 8 +- VM/src/lstrlib.cpp | 20 +- bench/tests/sunspider/3d-cube.lua | 32 +- tests/Autocomplete.test.cpp | 61 +- tests/Compiler.test.cpp | 93 +-- tests/Conformance.test.cpp | 8 - tests/Fixture.cpp | 5 +- tests/Fixture.h | 2 +- tests/Parser.test.cpp | 53 +- tests/ToString.test.cpp | 22 +- tests/Transpiler.test.cpp | 14 +- tests/TypeInfer.annotations.test.cpp | 2 +- tests/TypeInfer.refinements.test.cpp | 28 +- tests/TypeInfer.singletons.test.cpp | 26 +- tests/TypeInfer.tables.test.cpp | 101 ++- tests/TypeInfer.test.cpp | 150 ++++- tests/TypeInfer.typePacks.cpp | 324 ++++++++++ 48 files changed, 2652 insertions(+), 1272 deletions(-) create mode 100644 Compiler/src/Builtins.cpp create mode 100644 Compiler/src/Builtins.h create mode 100644 Compiler/src/ConstantFolding.cpp create mode 100644 Compiler/src/ConstantFolding.h create mode 100644 Compiler/src/TableShape.cpp create mode 100644 Compiler/src/TableShape.h create mode 100644 Compiler/src/ValueTracking.cpp create mode 100644 Compiler/src/ValueTracking.h diff --git a/Analysis/include/Luau/TypeInfer.h b/Analysis/include/Luau/TypeInfer.h index 312283b0..aa090014 100644 --- a/Analysis/include/Luau/TypeInfer.h +++ b/Analysis/include/Luau/TypeInfer.h @@ -97,6 +97,12 @@ struct ApplyTypeFunction : Substitution TypePackId clean(TypePackId tp) override; }; +struct GenericTypeDefinitions +{ + std::vector genericTypes; + std::vector genericPacks; +}; + // All TypeVars are retained via Environment::typeVars. All TypeIds // within a program are borrowed pointers into this set. struct TypeChecker @@ -146,7 +152,7 @@ struct TypeChecker ExprResult checkExpr(const ScopePtr& scope, const AstExprBinary& expr); ExprResult checkExpr(const ScopePtr& scope, const AstExprTypeAssertion& expr); ExprResult checkExpr(const ScopePtr& scope, const AstExprError& expr); - ExprResult checkExpr(const ScopePtr& scope, const AstExprIfElse& expr); + ExprResult checkExpr(const ScopePtr& scope, const AstExprIfElse& expr, std::optional expectedType = std::nullopt); TypeId checkExprTable(const ScopePtr& scope, const AstExprTable& expr, const std::vector>& fieldTypes, std::optional expectedType); @@ -336,8 +342,8 @@ private: const std::vector& typePackParams, const Location& location); // Note: `scope` must be a fresh scope. - std::pair, std::vector> createGenericTypes(const ScopePtr& scope, std::optional levelOpt, - const AstNode& node, const AstArray& genericNames, const AstArray& genericPackNames); + GenericTypeDefinitions createGenericTypes(const ScopePtr& scope, std::optional levelOpt, const AstNode& node, + const AstArray& genericNames, const AstArray& genericPackNames); public: ErrorVec resolve(const PredicateVec& predicates, const ScopePtr& scope, bool sense); diff --git a/Analysis/include/Luau/TypeVar.h b/Analysis/include/Luau/TypeVar.h index d6e17714..fd2c2afa 100644 --- a/Analysis/include/Luau/TypeVar.h +++ b/Analysis/include/Luau/TypeVar.h @@ -181,6 +181,18 @@ const T* get(const SingletonTypeVar* stv) return nullptr; } +struct GenericTypeDefinition +{ + TypeId ty; + std::optional defaultValue; +}; + +struct GenericTypePackDefinition +{ + TypePackId tp; + std::optional defaultValue; +}; + struct FunctionArgument { Name name; @@ -358,8 +370,8 @@ struct ClassTypeVar struct TypeFun { // These should all be generic - std::vector typeParams; - std::vector typePackParams; + std::vector typeParams; + std::vector typePackParams; /** The underlying type. * @@ -369,13 +381,13 @@ struct TypeFun TypeId type; TypeFun() = default; - TypeFun(std::vector typeParams, TypeId type) + TypeFun(std::vector typeParams, TypeId type) : typeParams(std::move(typeParams)) , type(type) { } - TypeFun(std::vector typeParams, std::vector typePackParams, TypeId type) + TypeFun(std::vector typeParams, std::vector typePackParams, TypeId type) : typeParams(std::move(typeParams)) , typePackParams(std::move(typePackParams)) , type(type) diff --git a/Analysis/src/Autocomplete.cpp b/Analysis/src/Autocomplete.cpp index 67ebd075..7a801f97 100644 --- a/Analysis/src/Autocomplete.cpp +++ b/Analysis/src/Autocomplete.cpp @@ -13,11 +13,10 @@ #include LUAU_FASTFLAG(LuauUseCommittingTxnLog) -LUAU_FASTFLAG(LuauIfElseExpressionAnalysisSupport) LUAU_FASTFLAGVARIABLE(LuauAutocompleteAvoidMutation, false); -LUAU_FASTFLAGVARIABLE(LuauAutocompletePreferToCallFunctions, false); LUAU_FASTFLAGVARIABLE(LuauAutocompleteFirstArg, false); LUAU_FASTFLAGVARIABLE(LuauCompleteBrokenStringParams, false); +LUAU_FASTFLAGVARIABLE(LuauMissingFollowACMetatables, false); static const std::unordered_set kStatementStartingKeywords = { "while", "if", "local", "repeat", "function", "do", "for", "return", "break", "continue", "type", "export"}; @@ -291,51 +290,23 @@ static TypeCorrectKind checkTypeCorrectKind(const Module& module, TypeArena* typ expectedType = follow(*it); } - if (FFlag::LuauAutocompletePreferToCallFunctions) + // We also want to suggest functions that return compatible result + if (const FunctionTypeVar* ftv = get(ty)) { - // We also want to suggest functions that return compatible result - if (const FunctionTypeVar* ftv = get(ty)) - { - auto [retHead, retTail] = flatten(ftv->retType); - - if (!retHead.empty() && canUnify(retHead.front(), expectedType)) - return TypeCorrectKind::CorrectFunctionResult; - - // We might only have a variadic tail pack, check if the element is compatible - if (retTail) - { - if (const VariadicTypePack* vtp = get(follow(*retTail)); vtp && canUnify(vtp->ty, expectedType)) - return TypeCorrectKind::CorrectFunctionResult; - } - } - - return canUnify(ty, expectedType) ? TypeCorrectKind::Correct : TypeCorrectKind::None; - } - else - { - if (canUnify(ty, expectedType)) - return TypeCorrectKind::Correct; - - // We also want to suggest functions that return compatible result - const FunctionTypeVar* ftv = get(ty); - - if (!ftv) - return TypeCorrectKind::None; - auto [retHead, retTail] = flatten(ftv->retType); - if (!retHead.empty()) - return canUnify(retHead.front(), expectedType) ? TypeCorrectKind::CorrectFunctionResult : TypeCorrectKind::None; + if (!retHead.empty() && canUnify(retHead.front(), expectedType)) + return TypeCorrectKind::CorrectFunctionResult; // We might only have a variadic tail pack, check if the element is compatible if (retTail) { - if (const VariadicTypePack* vtp = get(follow(*retTail))) - return canUnify(vtp->ty, expectedType) ? TypeCorrectKind::CorrectFunctionResult : TypeCorrectKind::None; + if (const VariadicTypePack* vtp = get(follow(*retTail)); vtp && canUnify(vtp->ty, expectedType)) + return TypeCorrectKind::CorrectFunctionResult; } - - return TypeCorrectKind::None; } + + return canUnify(ty, expectedType) ? TypeCorrectKind::Correct : TypeCorrectKind::None; } enum class PropIndexType @@ -435,13 +406,28 @@ static void autocompleteProps(const Module& module, TypeArena* typeArena, TypeId auto indexIt = mtable->props.find("__index"); if (indexIt != mtable->props.end()) { - if (get(indexIt->second.type) || get(indexIt->second.type)) - autocompleteProps(module, typeArena, indexIt->second.type, indexType, nodes, result, seen); - else if (auto indexFunction = get(indexIt->second.type)) + if (FFlag::LuauMissingFollowACMetatables) { - std::optional indexFunctionResult = first(indexFunction->retType); - if (indexFunctionResult) - autocompleteProps(module, typeArena, *indexFunctionResult, indexType, nodes, result, seen); + TypeId followed = follow(indexIt->second.type); + if (get(followed) || get(followed)) + autocompleteProps(module, typeArena, followed, indexType, nodes, result, seen); + else if (auto indexFunction = get(followed)) + { + std::optional indexFunctionResult = first(indexFunction->retType); + if (indexFunctionResult) + autocompleteProps(module, typeArena, *indexFunctionResult, indexType, nodes, result, seen); + } + } + else + { + if (get(indexIt->second.type) || get(indexIt->second.type)) + autocompleteProps(module, typeArena, indexIt->second.type, indexType, nodes, result, seen); + else if (auto indexFunction = get(indexIt->second.type)) + { + std::optional indexFunctionResult = first(indexFunction->retType); + if (indexFunctionResult) + autocompleteProps(module, typeArena, *indexFunctionResult, indexType, nodes, result, seen); + } } } } @@ -1224,7 +1210,7 @@ static void autocompleteExpression(const SourceModule& sourceModule, const Modul if (auto it = module.astTypes.find(node->asExpr())) autocompleteProps(module, typeArena, *it, PropIndexType::Point, ancestry, result); } - else if (FFlag::LuauIfElseExpressionAnalysisSupport && autocompleteIfElseExpression(node, ancestry, position, result)) + else if (autocompleteIfElseExpression(node, ancestry, position, result)) return; else if (node->is()) return; @@ -1261,8 +1247,7 @@ static void autocompleteExpression(const SourceModule& sourceModule, const Modul TypeCorrectKind correctForFunction = functionIsExpectedAt(module, node, position).value_or(false) ? TypeCorrectKind::Correct : TypeCorrectKind::None; - if (FFlag::LuauIfElseExpressionAnalysisSupport) - result["if"] = {AutocompleteEntryKind::Keyword, std::nullopt, false, false}; + result["if"] = {AutocompleteEntryKind::Keyword, std::nullopt, false, false}; result["true"] = {AutocompleteEntryKind::Keyword, typeChecker.booleanType, false, false, correctForBoolean}; result["false"] = {AutocompleteEntryKind::Keyword, typeChecker.booleanType, false, false, correctForBoolean}; result["nil"] = {AutocompleteEntryKind::Keyword, typeChecker.nilType, false, false, correctForNil}; diff --git a/Analysis/src/Error.cpp b/Analysis/src/Error.cpp index ce832c6b..88069f1f 100644 --- a/Analysis/src/Error.cpp +++ b/Analysis/src/Error.cpp @@ -190,24 +190,24 @@ struct ErrorConverter { name += "<"; bool first = true; - for (TypeId t : e.typeFun.typeParams) + for (auto param : e.typeFun.typeParams) { if (first) first = false; else name += ", "; - name += toString(t); + name += toString(param.ty); } - for (TypePackId t : e.typeFun.typePackParams) + for (auto param : e.typeFun.typePackParams) { if (first) first = false; else name += ", "; - name += toString(t); + name += toString(param.tp); } name += ">"; @@ -544,13 +544,13 @@ bool IncorrectGenericParameterCount::operator==(const IncorrectGenericParameterC for (size_t i = 0; i < typeFun.typeParams.size(); ++i) { - if (typeFun.typeParams[i] != rhs.typeFun.typeParams[i]) + if (typeFun.typeParams[i].ty != rhs.typeFun.typeParams[i].ty) return false; } for (size_t i = 0; i < typeFun.typePackParams.size(); ++i) { - if (typeFun.typePackParams[i] != rhs.typeFun.typePackParams[i]) + if (typeFun.typePackParams[i].tp != rhs.typeFun.typePackParams[i].tp) return false; } diff --git a/Analysis/src/IostreamHelpers.cpp b/Analysis/src/IostreamHelpers.cpp index 5bc76ade..19c2ddab 100644 --- a/Analysis/src/IostreamHelpers.cpp +++ b/Analysis/src/IostreamHelpers.cpp @@ -96,24 +96,24 @@ std::ostream& operator<<(std::ostream& stream, const IncorrectGenericParameterCo { stream << "<"; bool first = true; - for (TypeId t : error.typeFun.typeParams) + for (auto param : error.typeFun.typeParams) { if (first) first = false; else stream << ", "; - stream << toString(t); + stream << toString(param.ty); } - for (TypePackId t : error.typeFun.typePackParams) + for (auto param : error.typeFun.typePackParams) { if (first) first = false; else stream << ", "; - stream << toString(t); + stream << toString(param.tp); } stream << ">"; diff --git a/Analysis/src/JsonEncoder.cpp b/Analysis/src/JsonEncoder.cpp index 23491a5a..8dd597e1 100644 --- a/Analysis/src/JsonEncoder.cpp +++ b/Analysis/src/JsonEncoder.cpp @@ -5,6 +5,8 @@ #include "Luau/StringUtils.h" #include "Luau/Common.h" +LUAU_FASTFLAG(LuauTypeAliasDefaults) + namespace Luau { @@ -337,6 +339,42 @@ struct AstJsonEncoder : public AstVisitor writeRaw("}"); } + void write(const AstGenericType& genericType) + { + if (FFlag::LuauTypeAliasDefaults) + { + writeRaw("{"); + bool c = pushComma(); + write("name", genericType.name); + if (genericType.defaultValue) + write("type", genericType.defaultValue); + popComma(c); + writeRaw("}"); + } + else + { + write(genericType.name); + } + } + + void write(const AstGenericTypePack& genericTypePack) + { + if (FFlag::LuauTypeAliasDefaults) + { + writeRaw("{"); + bool c = pushComma(); + write("name", genericTypePack.name); + if (genericTypePack.defaultValue) + write("type", genericTypePack.defaultValue); + popComma(c); + writeRaw("}"); + } + else + { + write(genericTypePack.name); + } + } + void write(AstExprTable::Item::Kind kind) { switch (kind) diff --git a/Analysis/src/Module.cpp b/Analysis/src/Module.cpp index cff85897..9f352f4b 100644 --- a/Analysis/src/Module.cpp +++ b/Analysis/src/Module.cpp @@ -14,6 +14,7 @@ LUAU_FASTFLAGVARIABLE(DebugLuauFreezeArena, false) LUAU_FASTFLAGVARIABLE(DebugLuauTrackOwningArena, false) LUAU_FASTINTVARIABLE(LuauTypeCloneRecursionLimit, 300) +LUAU_FASTFLAG(LuauTypeAliasDefaults) namespace Luau { @@ -447,11 +448,28 @@ TypeId clone(TypeId typeId, TypeArena& dest, SeenTypes& seenTypes, SeenTypePacks TypeFun clone(const TypeFun& typeFun, TypeArena& dest, SeenTypes& seenTypes, SeenTypePacks& seenTypePacks, CloneState& cloneState) { TypeFun result; - for (TypeId ty : typeFun.typeParams) - result.typeParams.push_back(clone(ty, dest, seenTypes, seenTypePacks, cloneState)); - for (TypePackId tp : typeFun.typePackParams) - result.typePackParams.push_back(clone(tp, dest, seenTypes, seenTypePacks, cloneState)); + for (auto param : typeFun.typeParams) + { + TypeId ty = clone(param.ty, dest, seenTypes, seenTypePacks, cloneState); + std::optional defaultValue; + + if (FFlag::LuauTypeAliasDefaults && param.defaultValue) + defaultValue = clone(*param.defaultValue, dest, seenTypes, seenTypePacks, cloneState); + + result.typeParams.push_back({ty, defaultValue}); + } + + for (auto param : typeFun.typePackParams) + { + TypePackId tp = clone(param.tp, dest, seenTypes, seenTypePacks, cloneState); + std::optional defaultValue; + + if (FFlag::LuauTypeAliasDefaults && param.defaultValue) + defaultValue = clone(*param.defaultValue, dest, seenTypes, seenTypePacks, cloneState); + + result.typePackParams.push_back({tp, defaultValue}); + } result.type = clone(typeFun.type, dest, seenTypes, seenTypePacks, cloneState); diff --git a/Analysis/src/ToString.cpp b/Analysis/src/ToString.cpp index 889dd6dc..4b898d3a 100644 --- a/Analysis/src/ToString.cpp +++ b/Analysis/src/ToString.cpp @@ -11,6 +11,7 @@ #include LUAU_FASTFLAG(LuauOccursCheckOkWithRecursiveFunctions) +LUAU_FASTFLAG(LuauTypeAliasDefaults) /* * Prefix generic typenames with gen- @@ -209,6 +210,14 @@ struct StringifierState result.name += s; } + + void emit(const char* s) + { + if (opts.maxTypeLength > 0 && result.name.length() > opts.maxTypeLength) + return; + + result.name += s; + } }; struct TypeVarStringifier @@ -280,13 +289,28 @@ struct TypeVarStringifier else first = false; - if (!singleTp) - state.emit("("); + if (FFlag::LuauTypeAliasDefaults) + { + bool wrap = !singleTp && get(follow(tp)); - stringify(tp); + if (wrap) + state.emit("("); - if (!singleTp) - state.emit(")"); + stringify(tp); + + if (wrap) + state.emit(")"); + } + else + { + if (!singleTp) + state.emit("("); + + stringify(tp); + + if (!singleTp) + state.emit(")"); + } } if (types.size() || typePacks.size()) @@ -1086,7 +1110,7 @@ std::string toString(const TypePackVar& tp, const ToStringOptions& opts) return toString(const_cast(&tp), std::move(opts)); } -std::string toStringNamedFunction(const std::string& prefix, const FunctionTypeVar& ftv, ToStringOptions opts) +std::string toStringNamedFunction_DEPRECATED(const std::string& prefix, const FunctionTypeVar& ftv, ToStringOptions opts) { std::string s = prefix; @@ -1175,6 +1199,77 @@ std::string toStringNamedFunction(const std::string& prefix, const FunctionTypeV return s; } +std::string toStringNamedFunction(const std::string& prefix, const FunctionTypeVar& ftv, ToStringOptions opts) +{ + if (!FFlag::LuauTypeAliasDefaults) + return toStringNamedFunction_DEPRECATED(prefix, ftv, opts); + + ToStringResult result; + StringifierState state(opts, result, opts.nameMap); + TypeVarStringifier tvs{state}; + + state.emit(prefix); + + if (!opts.hideNamedFunctionTypeParameters) + tvs.stringify(ftv.generics, ftv.genericPacks); + + state.emit("("); + + auto argPackIter = begin(ftv.argTypes); + auto argNameIter = ftv.argNames.begin(); + + bool first = true; + while (argPackIter != end(ftv.argTypes)) + { + if (!first) + state.emit(", "); + first = false; + + // We don't currently respect opts.functionTypeArguments. I don't think this function should. + if (argNameIter != ftv.argNames.end()) + { + state.emit((*argNameIter ? (*argNameIter)->name : "_") + ": "); + ++argNameIter; + } + else + { + state.emit("_: "); + } + + tvs.stringify(*argPackIter); + ++argPackIter; + } + + if (argPackIter.tail()) + { + if (!first) + state.emit(", "); + + state.emit("...: "); + + if (auto vtp = get(*argPackIter.tail())) + tvs.stringify(vtp->ty); + else + tvs.stringify(*argPackIter.tail()); + } + + state.emit("): "); + + size_t retSize = size(ftv.retType); + bool hasTail = !finite(ftv.retType); + bool wrap = get(follow(ftv.retType)) && (hasTail ? retSize != 0 : retSize != 1); + + if (wrap) + state.emit("("); + + tvs.stringify(ftv.retType); + + if (wrap) + state.emit(")"); + + return result.name; +} + std::string dump(TypeId ty) { ToStringOptions opts; diff --git a/Analysis/src/Transpiler.cpp b/Analysis/src/Transpiler.cpp index 8e13ea5b..f5908683 100644 --- a/Analysis/src/Transpiler.cpp +++ b/Analysis/src/Transpiler.cpp @@ -10,6 +10,8 @@ #include #include +LUAU_FASTFLAG(LuauTypeAliasDefaults) + namespace { bool isIdentifierStartChar(char c) @@ -793,14 +795,47 @@ struct Printer for (auto o : a->generics) { comma(); - writer.identifier(o.value); + + if (FFlag::LuauTypeAliasDefaults) + { + writer.advance(o.location.begin); + writer.identifier(o.name.value); + + if (o.defaultValue) + { + writer.maybeSpace(o.defaultValue->location.begin, 2); + writer.symbol("="); + visualizeTypeAnnotation(*o.defaultValue); + } + } + else + { + writer.identifier(o.name.value); + } } for (auto o : a->genericPacks) { comma(); - writer.identifier(o.value); - writer.symbol("..."); + + if (FFlag::LuauTypeAliasDefaults) + { + writer.advance(o.location.begin); + writer.identifier(o.name.value); + writer.symbol("..."); + + if (o.defaultValue) + { + writer.maybeSpace(o.defaultValue->location.begin, 2); + writer.symbol("="); + visualizeTypePackAnnotation(*o.defaultValue, false); + } + } + else + { + writer.identifier(o.name.value); + writer.symbol("..."); + } } writer.symbol(">"); @@ -846,12 +881,20 @@ struct Printer for (const auto& o : func.generics) { comma(); - writer.identifier(o.value); + + if (FFlag::LuauTypeAliasDefaults) + writer.advance(o.location.begin); + + writer.identifier(o.name.value); } for (const auto& o : func.genericPacks) { comma(); - writer.identifier(o.value); + + if (FFlag::LuauTypeAliasDefaults) + writer.advance(o.location.begin); + + writer.identifier(o.name.value); writer.symbol("..."); } writer.symbol(">"); @@ -979,12 +1022,20 @@ struct Printer for (const auto& o : a->generics) { comma(); - writer.identifier(o.value); + + if (FFlag::LuauTypeAliasDefaults) + writer.advance(o.location.begin); + + writer.identifier(o.name.value); } for (const auto& o : a->genericPacks) { comma(); - writer.identifier(o.value); + + if (FFlag::LuauTypeAliasDefaults) + writer.advance(o.location.begin); + + writer.identifier(o.name.value); writer.symbol("..."); } writer.symbol(">"); diff --git a/Analysis/src/TypeAttach.cpp b/Analysis/src/TypeAttach.cpp index 9e61c792..2ec02093 100644 --- a/Analysis/src/TypeAttach.cpp +++ b/Analysis/src/TypeAttach.cpp @@ -212,24 +212,24 @@ public: if (hasSeen(&ftv)) return allocator->alloc(Location(), std::nullopt, AstName("")); - AstArray generics; + AstArray generics; generics.size = ftv.generics.size(); - generics.data = static_cast(allocator->allocate(sizeof(AstName) * generics.size)); + generics.data = static_cast(allocator->allocate(sizeof(AstGenericType) * generics.size)); size_t numGenerics = 0; for (auto it = ftv.generics.begin(); it != ftv.generics.end(); ++it) { if (auto gtv = get(*it)) - generics.data[numGenerics++] = AstName(gtv->name.c_str()); + generics.data[numGenerics++] = {AstName(gtv->name.c_str()), Location(), nullptr}; } - AstArray genericPacks; + AstArray genericPacks; genericPacks.size = ftv.genericPacks.size(); - genericPacks.data = static_cast(allocator->allocate(sizeof(AstName) * genericPacks.size)); + genericPacks.data = static_cast(allocator->allocate(sizeof(AstGenericTypePack) * genericPacks.size)); size_t numGenericPacks = 0; for (auto it = ftv.genericPacks.begin(); it != ftv.genericPacks.end(); ++it) { if (auto gtv = get(*it)) - genericPacks.data[numGenericPacks++] = AstName(gtv->name.c_str()); + genericPacks.data[numGenericPacks++] = {AstName(gtv->name.c_str()), Location(), nullptr}; } AstArray argTypes; diff --git a/Analysis/src/TypeInfer.cpp b/Analysis/src/TypeInfer.cpp index 1689a5c3..bedcc022 100644 --- a/Analysis/src/TypeInfer.cpp +++ b/Analysis/src/TypeInfer.cpp @@ -15,8 +15,8 @@ #include "Luau/TypeVar.h" #include "Luau/TimeTrace.h" -#include #include +#include LUAU_FASTFLAGVARIABLE(DebugLuauMagicTypes, false) LUAU_FASTINTVARIABLE(LuauTypeInferRecursionLimit, 500) @@ -24,25 +24,30 @@ LUAU_FASTINTVARIABLE(LuauTypeInferTypePackLoopLimit, 5000) LUAU_FASTINTVARIABLE(LuauCheckRecursionLimit, 500) LUAU_FASTFLAG(LuauKnowsTheDataModel3) LUAU_FASTFLAGVARIABLE(LuauEqConstraint, false) +LUAU_FASTFLAGVARIABLE(LuauGroupExpectedType, false) LUAU_FASTFLAGVARIABLE(LuauWeakEqConstraint, false) // Eventually removed as false. LUAU_FASTFLAGVARIABLE(LuauCloneCorrectlyBeforeMutatingTableType, false) LUAU_FASTFLAGVARIABLE(LuauStoreMatchingOverloadFnType, false) LUAU_FASTFLAG(LuauUseCommittingTxnLog) LUAU_FASTFLAGVARIABLE(DebugLuauFreezeDuringUnification, false) LUAU_FASTFLAGVARIABLE(LuauRecursiveTypeParameterRestriction, false) -LUAU_FASTFLAGVARIABLE(LuauIfElseExpressionAnalysisSupport, false) +LUAU_FASTFLAGVARIABLE(LuauIfElseBranchTypeUnion, false) +LUAU_FASTFLAGVARIABLE(LuauIfElseExpectedType2, false) LUAU_FASTFLAGVARIABLE(LuauQuantifyInPlace2, false) LUAU_FASTFLAGVARIABLE(LuauSealExports, false) LUAU_FASTFLAGVARIABLE(LuauSingletonTypes, false) +LUAU_FASTFLAGVARIABLE(LuauTypeAliasDefaults, false) LUAU_FASTFLAGVARIABLE(LuauExpectedTypesOfProperties, false) LUAU_FASTFLAGVARIABLE(LuauErrorRecoveryType, false) LUAU_FASTFLAGVARIABLE(LuauPropertiesGetExpectedType, false) LUAU_FASTFLAGVARIABLE(LuauLValueAsKey, false) LUAU_FASTFLAGVARIABLE(LuauRefiLookupFromIndexExpr, false) +LUAU_FASTFLAGVARIABLE(LuauPerModuleUnificationCache, false) LUAU_FASTFLAGVARIABLE(LuauProperTypeLevels, false) LUAU_FASTFLAGVARIABLE(LuauAscribeCorrectLevelToInferredProperitesOfFreeTables, false) LUAU_FASTFLAGVARIABLE(LuauFixRecursiveMetatableCall, false) LUAU_FASTFLAGVARIABLE(LuauBidirectionalAsExpr, false) +LUAU_FASTFLAGVARIABLE(LuauUnsealedTableLiteral, false) LUAU_FASTFLAGVARIABLE(LuauUpdateFunctionNameBinding, false) namespace Luau @@ -279,6 +284,14 @@ ModulePtr TypeChecker::check(const SourceModule& module, Mode mode, std::optiona GenericError{"Free types leaked into this module's public interface. This is an internal Luau error; please report it."}}); } + if (FFlag::LuauPerModuleUnificationCache) + { + // Clear unifier cache since it's keyed off internal types that get deallocated + // This avoids fake cross-module cache hits and keeps cache size at bay when typechecking large module graphs. + unifierState.cachedUnify.clear(); + unifierState.skipCacheForType.clear(); + } + return std::move(currentModule); } @@ -1213,18 +1226,18 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatTypeAlias& typealias ScopePtr aliasScope = childScope(scope, typealias.location); aliasScope->level = scope->level.incr(); - for (TypeId ty : binding->typeParams) + for (auto param : binding->typeParams) { - auto generic = get(ty); + auto generic = get(param.ty); LUAU_ASSERT(generic); - aliasScope->privateTypeBindings[generic->name] = TypeFun{{}, ty}; + aliasScope->privateTypeBindings[generic->name] = TypeFun{{}, param.ty}; } - for (TypePackId tp : binding->typePackParams) + for (auto param : binding->typePackParams) { - auto generic = get(tp); + auto generic = get(param.tp); LUAU_ASSERT(generic); - aliasScope->privateTypePackBindings[generic->name] = tp; + aliasScope->privateTypePackBindings[generic->name] = param.tp; } TypeId ty = resolveType(aliasScope, *typealias.type); @@ -1233,9 +1246,17 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatTypeAlias& typealias // If the table is already named and we want to rename the type function, we have to bind new alias to a copy if (ttv->name) { + bool sameTys = std::equal(ttv->instantiatedTypeParams.begin(), ttv->instantiatedTypeParams.end(), binding->typeParams.begin(), + binding->typeParams.end(), [](auto&& itp, auto&& tp) { + return itp == tp.ty; + }); + bool sameTps = std::equal(ttv->instantiatedTypePackParams.begin(), ttv->instantiatedTypePackParams.end(), + binding->typePackParams.begin(), binding->typePackParams.end(), [](auto&& itpp, auto&& tpp) { + return itpp == tpp.tp; + }); + // Copy can be skipped if this is an identical alias - if (ttv->name != name || ttv->instantiatedTypeParams != binding->typeParams || - ttv->instantiatedTypePackParams != binding->typePackParams) + if (ttv->name != name || !sameTys || !sameTps) { // This is a shallow clone, original recursive links to self are not updated TableTypeVar clone = TableTypeVar{ttv->props, ttv->indexer, ttv->level, ttv->state}; @@ -1243,8 +1264,12 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatTypeAlias& typealias clone.methodDefinitionLocations = ttv->methodDefinitionLocations; clone.definitionModuleName = ttv->definitionModuleName; clone.name = name; - clone.instantiatedTypeParams = binding->typeParams; - clone.instantiatedTypePackParams = binding->typePackParams; + + for (auto param : binding->typeParams) + clone.instantiatedTypeParams.push_back(param.ty); + + for (auto param : binding->typePackParams) + clone.instantiatedTypePackParams.push_back(param.tp); ty = addType(std::move(clone)); } @@ -1252,8 +1277,14 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatTypeAlias& typealias else { ttv->name = name; - ttv->instantiatedTypeParams = binding->typeParams; - ttv->instantiatedTypePackParams = binding->typePackParams; + + ttv->instantiatedTypeParams.clear(); + for (auto param : binding->typeParams) + ttv->instantiatedTypeParams.push_back(param.ty); + + ttv->instantiatedTypePackParams.clear(); + for (auto param : binding->typePackParams) + ttv->instantiatedTypePackParams.push_back(param.tp); } } else if (auto mtv = getMutable(follow(ty))) @@ -1367,9 +1398,21 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatDeclareFunction& glo auto [generics, genericPacks] = createGenericTypes(funScope, std::nullopt, global, global.generics, global.genericPacks); + std::vector genericTys; + genericTys.reserve(generics.size()); + std::transform(generics.begin(), generics.end(), std::back_inserter(genericTys), [](auto&& el) { + return el.ty; + }); + + std::vector genericTps; + genericTps.reserve(genericPacks.size()); + std::transform(genericPacks.begin(), genericPacks.end(), std::back_inserter(genericTps), [](auto&& el) { + return el.tp; + }); + TypePackId argPack = resolveTypePack(funScope, global.params); TypePackId retPack = resolveTypePack(funScope, global.retTypes); - TypeId fnType = addType(FunctionTypeVar{funScope->level, generics, genericPacks, argPack, retPack}); + TypeId fnType = addType(FunctionTypeVar{funScope->level, std::move(genericTys), std::move(genericTps), argPack, retPack}); FunctionTypeVar* ftv = getMutable(fnType); ftv->argNames.reserve(global.paramNames.size); @@ -1394,7 +1437,7 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExpr& ExprResult result; if (auto a = expr.as()) - result = checkExpr(scope, *a->expr); + result = checkExpr(scope, *a->expr, FFlag::LuauGroupExpectedType ? expectedType : std::nullopt); else if (expr.is()) result = {nilType}; else if (const AstExprConstantBool* bexpr = expr.as()) @@ -1438,21 +1481,7 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExpr& else if (auto a = expr.as()) result = checkExpr(scope, *a); else if (auto a = expr.as()) - { - if (FFlag::LuauIfElseExpressionAnalysisSupport) - { - result = checkExpr(scope, *a); - } - else - { - // Note: When the fast flag is disabled we can't skip the handling of AstExprIfElse - // because we would generate an ICE. We also can't use the default value - // of result, because it will lead to a compiler crash. - // Note: LuauIfElseExpressionBaseSupport can be used to disable parser support - // for if-else expressions which will mean this node type is never created. - result = {anyType}; - } - } + result = checkExpr(scope, *a, FFlag::LuauIfElseExpectedType2 ? expectedType : std::nullopt); else ice("Unhandled AstExpr?"); @@ -1895,7 +1924,7 @@ TypeId TypeChecker::checkExprTable( } } - TableState state = (expr.items.size == 0 || isNonstrictMode()) ? TableState::Unsealed : TableState::Sealed; + TableState state = (expr.items.size == 0 || isNonstrictMode() || FFlag::LuauUnsealedTableLiteral) ? TableState::Unsealed : TableState::Sealed; TableTypeVar table = TableTypeVar{std::move(props), indexer, scope->level, state}; table.definitionModuleName = currentModuleName; return addType(table); @@ -2549,23 +2578,34 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprEr return {errorRecoveryType(scope)}; } -ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprIfElse& expr) +ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprIfElse& expr, std::optional expectedType) { ExprResult result = checkExpr(scope, *expr.condition); ScopePtr trueScope = childScope(scope, expr.trueExpr->location); reportErrors(resolve(result.predicates, trueScope, true)); - ExprResult trueType = checkExpr(trueScope, *expr.trueExpr); + ExprResult trueType = checkExpr(trueScope, *expr.trueExpr, expectedType); ScopePtr falseScope = childScope(scope, expr.falseExpr->location); // Don't report errors for this scope to avoid potentially duplicating errors reported for the first scope. resolve(result.predicates, falseScope, false); - ExprResult falseType = checkExpr(falseScope, *expr.falseExpr); + ExprResult falseType = checkExpr(falseScope, *expr.falseExpr, expectedType); - unify(falseType.type, trueType.type, expr.location); + if (FFlag::LuauIfElseBranchTypeUnion) + { + if (falseType.type == trueType.type) + return {trueType.type}; - // TODO: normalize(UnionTypeVar{{trueType, falseType}}) - // For now both trueType and falseType must be the same type. - return {trueType.type}; + std::vector types = reduceUnion({trueType.type, falseType.type}); + return {types.size() == 1 ? types[0] : addType(UnionTypeVar{std::move(types)})}; + } + else + { + unify(falseType.type, trueType.type, expr.location); + + // TODO: normalize(UnionTypeVar{{trueType, falseType}}) + // For now both trueType and falseType must be the same type. + return {trueType.type}; + } } TypeId TypeChecker::checkLValue(const ScopePtr& scope, const AstExpr& expr) @@ -3032,7 +3072,20 @@ std::pair TypeChecker::checkFunctionSignature( defn.varargLocation = expr.vararg ? std::make_optional(expr.varargLocation) : std::nullopt; defn.originalNameLocation = originalName.value_or(Location(expr.location.begin, 0)); - TypeId funTy = addType(FunctionTypeVar(funScope->level, generics, genericPacks, argPack, retPack, std::move(defn), bool(expr.self))); + std::vector genericTys; + genericTys.reserve(generics.size()); + std::transform(generics.begin(), generics.end(), std::back_inserter(genericTys), [](auto&& el) { + return el.ty; + }); + + std::vector genericTps; + genericTps.reserve(genericPacks.size()); + std::transform(genericPacks.begin(), genericPacks.end(), std::back_inserter(genericTps), [](auto&& el) { + return el.tp; + }); + + TypeId funTy = + addType(FunctionTypeVar(funScope->level, std::move(genericTys), std::move(genericTps), argPack, retPack, std::move(defn), bool(expr.self))); FunctionTypeVar* ftv = getMutable(funTy); @@ -4848,11 +4901,38 @@ TypeId TypeChecker::resolveType(const ScopePtr& scope, const AstType& annotation if (lit->parameters.size == 0 && tf->typeParams.empty() && tf->typePackParams.empty()) return tf->type; - if (!lit->hasParameterList && !tf->typePackParams.empty()) + bool hasDefaultTypes = false; + bool hasDefaultPacks = false; + bool parameterCountErrorReported = false; + + if (FFlag::LuauTypeAliasDefaults) { - reportError(TypeError{annotation.location, GenericError{"Type parameter list is required"}}); - if (!FFlag::LuauErrorRecoveryType) - return errorRecoveryType(scope); + hasDefaultTypes = std::any_of(tf->typeParams.begin(), tf->typeParams.end(), [](auto&& el) { + return el.defaultValue.has_value(); + }); + hasDefaultPacks = std::any_of(tf->typePackParams.begin(), tf->typePackParams.end(), [](auto&& el) { + return el.defaultValue.has_value(); + }); + + if (!lit->hasParameterList) + { + if ((!tf->typeParams.empty() && !hasDefaultTypes) || (!tf->typePackParams.empty() && !hasDefaultPacks)) + { + reportError(TypeError{annotation.location, GenericError{"Type parameter list is required"}}); + parameterCountErrorReported = true; + if (!FFlag::LuauErrorRecoveryType) + return errorRecoveryType(scope); + } + } + } + else + { + if (!lit->hasParameterList && !tf->typePackParams.empty()) + { + reportError(TypeError{annotation.location, GenericError{"Type parameter list is required"}}); + if (!FFlag::LuauErrorRecoveryType) + return errorRecoveryType(scope); + } } std::vector typeParams; @@ -4892,14 +4972,89 @@ TypeId TypeChecker::resolveType(const ScopePtr& scope, const AstType& annotation if (typePackParams.empty() && !extraTypes.empty()) typePackParams.push_back(addTypePack(extraTypes)); + if (FFlag::LuauTypeAliasDefaults) + { + size_t typesProvided = typeParams.size(); + size_t typesRequired = tf->typeParams.size(); + + size_t packsProvided = typePackParams.size(); + size_t packsRequired = tf->typePackParams.size(); + + bool notEnoughParameters = + (typesProvided < typesRequired && packsProvided == 0) || (typesProvided == typesRequired && packsProvided < packsRequired); + bool hasDefaultParameters = hasDefaultTypes || hasDefaultPacks; + + // Add default type and type pack parameters if that's required and it's possible + if (notEnoughParameters && hasDefaultParameters) + { + // 'applyTypeFunction' is used to substitute default types that reference previous generic types + applyTypeFunction.typeArguments.clear(); + applyTypeFunction.typePackArguments.clear(); + applyTypeFunction.currentModule = currentModule; + applyTypeFunction.level = scope->level; + applyTypeFunction.encounteredForwardedType = false; + + for (size_t i = 0; i < typesProvided; ++i) + applyTypeFunction.typeArguments[tf->typeParams[i].ty] = typeParams[i]; + + if (typesProvided < typesRequired) + { + for (size_t i = typesProvided; i < typesRequired; ++i) + { + TypeId defaultTy = tf->typeParams[i].defaultValue.value_or(nullptr); + + if (!defaultTy) + break; + + std::optional maybeInstantiated = applyTypeFunction.substitute(defaultTy); + + if (!maybeInstantiated.has_value()) + { + reportError(annotation.location, UnificationTooComplex{}); + maybeInstantiated = errorRecoveryType(scope); + } + + applyTypeFunction.typeArguments[tf->typeParams[i].ty] = *maybeInstantiated; + typeParams.push_back(*maybeInstantiated); + } + } + + for (size_t i = 0; i < packsProvided; ++i) + applyTypeFunction.typePackArguments[tf->typePackParams[i].tp] = typePackParams[i]; + + if (packsProvided < packsRequired) + { + for (size_t i = packsProvided; i < packsRequired; ++i) + { + TypePackId defaultTp = tf->typePackParams[i].defaultValue.value_or(nullptr); + + if (!defaultTp) + break; + + std::optional maybeInstantiated = applyTypeFunction.substitute(defaultTp); + + if (!maybeInstantiated.has_value()) + { + reportError(annotation.location, UnificationTooComplex{}); + maybeInstantiated = errorRecoveryTypePack(scope); + } + + applyTypeFunction.typePackArguments[tf->typePackParams[i].tp] = *maybeInstantiated; + typePackParams.push_back(*maybeInstantiated); + } + } + } + } + // If we didn't combine regular types into a type pack and we're still one type pack short, provide an empty type pack if (extraTypes.empty() && typePackParams.size() + 1 == tf->typePackParams.size()) typePackParams.push_back(addTypePack({})); if (typeParams.size() != tf->typeParams.size() || typePackParams.size() != tf->typePackParams.size()) { - reportError( - TypeError{annotation.location, IncorrectGenericParameterCount{lit->name.value, *tf, typeParams.size(), typePackParams.size()}}); + if (!parameterCountErrorReported) + reportError( + TypeError{annotation.location, IncorrectGenericParameterCount{lit->name.value, *tf, typeParams.size(), typePackParams.size()}}); if (FFlag::LuauErrorRecoveryType) { @@ -4913,11 +5068,20 @@ TypeId TypeChecker::resolveType(const ScopePtr& scope, const AstType& annotation return errorRecoveryType(scope); } - if (FFlag::LuauRecursiveTypeParameterRestriction && typeParams == tf->typeParams && typePackParams == tf->typePackParams) + if (FFlag::LuauRecursiveTypeParameterRestriction) { + bool sameTys = std::equal(typeParams.begin(), typeParams.end(), tf->typeParams.begin(), tf->typeParams.end(), [](auto&& itp, auto&& tp) { + return itp == tp.ty; + }); + bool sameTps = std::equal( + typePackParams.begin(), typePackParams.end(), tf->typePackParams.begin(), tf->typePackParams.end(), [](auto&& itpp, auto&& tpp) { + return itpp == tpp.tp; + }); + // If the generic parameters and the type arguments are the same, we are about to // perform an identity substitution, which we can just short-circuit. - return tf->type; + if (sameTys && sameTps) + return tf->type; } return instantiateTypeFun(scope, *tf, typeParams, typePackParams, annotation.location); @@ -4948,7 +5112,19 @@ TypeId TypeChecker::resolveType(const ScopePtr& scope, const AstType& annotation TypePackId argTypes = resolveTypePack(funcScope, func->argTypes); TypePackId retTypes = resolveTypePack(funcScope, func->returnTypes); - TypeId fnType = addType(FunctionTypeVar{funcScope->level, std::move(generics), std::move(genericPacks), argTypes, retTypes}); + std::vector genericTys; + genericTys.reserve(generics.size()); + std::transform(generics.begin(), generics.end(), std::back_inserter(genericTys), [](auto&& el) { + return el.ty; + }); + + std::vector genericTps; + genericTps.reserve(genericPacks.size()); + std::transform(genericPacks.begin(), genericPacks.end(), std::back_inserter(genericTps), [](auto&& el) { + return el.tp; + }); + + TypeId fnType = addType(FunctionTypeVar{funcScope->level, std::move(genericTys), std::move(genericTps), argTypes, retTypes}); FunctionTypeVar* ftv = getMutable(fnType); @@ -5137,11 +5313,11 @@ TypeId TypeChecker::instantiateTypeFun(const ScopePtr& scope, const TypeFun& tf, applyTypeFunction.typeArguments.clear(); for (size_t i = 0; i < tf.typeParams.size(); ++i) - applyTypeFunction.typeArguments[tf.typeParams[i]] = typeParams[i]; + applyTypeFunction.typeArguments[tf.typeParams[i].ty] = typeParams[i]; applyTypeFunction.typePackArguments.clear(); for (size_t i = 0; i < tf.typePackParams.size(); ++i) - applyTypeFunction.typePackArguments[tf.typePackParams[i]] = typePackParams[i]; + applyTypeFunction.typePackArguments[tf.typePackParams[i].tp] = typePackParams[i]; applyTypeFunction.currentModule = currentModule; applyTypeFunction.level = scope->level; @@ -5213,17 +5389,23 @@ TypeId TypeChecker::instantiateTypeFun(const ScopePtr& scope, const TypeFun& tf, return instantiated; } -std::pair, std::vector> TypeChecker::createGenericTypes(const ScopePtr& scope, std::optional levelOpt, - const AstNode& node, const AstArray& genericNames, const AstArray& genericPackNames) +GenericTypeDefinitions TypeChecker::createGenericTypes(const ScopePtr& scope, std::optional levelOpt, const AstNode& node, + const AstArray& genericNames, const AstArray& genericPackNames) { LUAU_ASSERT(scope->parent); const TypeLevel level = (FFlag::LuauQuantifyInPlace2 && levelOpt) ? *levelOpt : scope->level; - std::vector generics; - for (const AstName& generic : genericNames) + std::vector generics; + + for (const AstGenericType& generic : genericNames) { - Name n = generic.value; + std::optional defaultValue; + + if (FFlag::LuauTypeAliasDefaults && generic.defaultValue) + defaultValue = resolveType(scope, *generic.defaultValue); + + Name n = generic.name.value; // These generics are the only thing that will ever be added to scope, so we can be certain that // a collision can only occur when two generic typevars have the same name. @@ -5246,14 +5428,20 @@ std::pair, std::vector> TypeChecker::createGener g = addType(Unifiable::Generic{level, n}); } - generics.push_back(g); + generics.push_back({g, defaultValue}); scope->privateTypeBindings[n] = TypeFun{{}, g}; } - std::vector genericPacks; - for (const AstName& genericPack : genericPackNames) + std::vector genericPacks; + + for (const AstGenericTypePack& genericPack : genericPackNames) { - Name n = genericPack.value; + std::optional defaultValue; + + if (FFlag::LuauTypeAliasDefaults && genericPack.defaultValue) + defaultValue = resolveTypePack(scope, *genericPack.defaultValue); + + Name n = genericPack.name.value; // These generics are the only thing that will ever be added to scope, so we can be certain that // a collision can only occur when two generic typevars have the same name. @@ -5276,7 +5464,7 @@ std::pair, std::vector> TypeChecker::createGener g = addTypePack(TypePackVar{Unifiable::Generic{level, n}}); } - genericPacks.push_back(g); + genericPacks.push_back({g, defaultValue}); scope->privateTypePackBindings[n] = g; } diff --git a/Analysis/src/TypeVar.cpp b/Analysis/src/TypeVar.cpp index 4cab79c8..ac2b2541 100644 --- a/Analysis/src/TypeVar.cpp +++ b/Analysis/src/TypeVar.cpp @@ -19,6 +19,7 @@ LUAU_FASTINTVARIABLE(LuauTypeMaximumStringifierLength, 500) LUAU_FASTINTVARIABLE(LuauTableTypeMaximumStringifierLength, 0) +LUAU_FASTFLAGVARIABLE(LuauMetatableAreEqualRecursion, false) LUAU_FASTFLAGVARIABLE(LuauRefactorTagging, false) LUAU_FASTFLAG(LuauErrorRecoveryType) LUAU_FASTFLAG(DebugLuauFreezeArena) @@ -453,6 +454,9 @@ bool areEqual(SeenSet& seen, const TableTypeVar& lhs, const TableTypeVar& rhs) static bool areEqual(SeenSet& seen, const MetatableTypeVar& lhs, const MetatableTypeVar& rhs) { + if (FFlag::LuauMetatableAreEqualRecursion && areSeen(seen, &lhs, &rhs)) + return true; + return areEqual(seen, *lhs.table, *rhs.table) && areEqual(seen, *lhs.metatable, *rhs.metatable); } diff --git a/Analysis/src/Unifier.cpp b/Analysis/src/Unifier.cpp index 393a84a7..6873c657 100644 --- a/Analysis/src/Unifier.cpp +++ b/Analysis/src/Unifier.cpp @@ -22,6 +22,7 @@ LUAU_FASTFLAGVARIABLE(LuauOccursCheckOkWithRecursiveFunctions, false) LUAU_FASTFLAG(LuauSingletonTypes) LUAU_FASTFLAG(LuauErrorRecoveryType); LUAU_FASTFLAG(LuauProperTypeLevels); +LUAU_FASTFLAGVARIABLE(LuauUnifyPackTails, false) LUAU_FASTFLAGVARIABLE(LuauExtendedUnionMismatchError, false) LUAU_FASTFLAGVARIABLE(LuauExtendedFunctionMismatchError, false) @@ -1170,9 +1171,15 @@ void Unifier::tryUnify_(TypePackId subTp, TypePackId superTp, bool isFunctionCal // If both are at the end, we're done if (!superIter.good() && !subIter.good()) { + if (FFlag::LuauUnifyPackTails && subTpv->tail && superTpv->tail) + { + tryUnify_(*subTpv->tail, *superTpv->tail); + break; + } + const bool lFreeTail = superTpv->tail && log.getMutable(log.follow(*superTpv->tail)) != nullptr; const bool rFreeTail = subTpv->tail && log.getMutable(log.follow(*subTpv->tail)) != nullptr; - if (lFreeTail && rFreeTail) + if (!FFlag::LuauUnifyPackTails && lFreeTail && rFreeTail) tryUnify_(*subTpv->tail, *superTpv->tail); else if (lFreeTail) tryUnify_(emptyTp, *superTpv->tail); @@ -1370,9 +1377,15 @@ void Unifier::tryUnify_(TypePackId subTp, TypePackId superTp, bool isFunctionCal // If both are at the end, we're done if (!superIter.good() && !subIter.good()) { + if (FFlag::LuauUnifyPackTails && subTpv->tail && superTpv->tail) + { + tryUnify_(*subTpv->tail, *superTpv->tail); + break; + } + const bool lFreeTail = superTpv->tail && get(follow(*superTpv->tail)) != nullptr; const bool rFreeTail = subTpv->tail && get(follow(*subTpv->tail)) != nullptr; - if (lFreeTail && rFreeTail) + if (!FFlag::LuauUnifyPackTails && lFreeTail && rFreeTail) tryUnify_(*subTpv->tail, *superTpv->tail); else if (lFreeTail) tryUnify_(emptyTp, *superTpv->tail); diff --git a/Ast/include/Luau/Ast.h b/Ast/include/Luau/Ast.h index 5b4bfa03..573850a5 100644 --- a/Ast/include/Luau/Ast.h +++ b/Ast/include/Luau/Ast.h @@ -334,6 +334,20 @@ struct AstTypeList using AstArgumentName = std::pair; // TODO: remove and replace when we get a common struct for this pair instead of AstName +struct AstGenericType +{ + AstName name; + Location location; + AstType* defaultValue = nullptr; +}; + +struct AstGenericTypePack +{ + AstName name; + Location location; + AstTypePack* defaultValue = nullptr; +}; + extern int gAstRttiIndex; template @@ -569,15 +583,15 @@ class AstExprFunction : public AstExpr public: LUAU_RTTI(AstExprFunction) - AstExprFunction(const Location& location, const AstArray& generics, const AstArray& genericPacks, AstLocal* self, - const AstArray& args, std::optional vararg, AstStatBlock* body, size_t functionDepth, const AstName& debugname, - std::optional returnAnnotation = {}, AstTypePack* varargAnnotation = nullptr, bool hasEnd = false, + AstExprFunction(const Location& location, const AstArray& generics, const AstArray& genericPacks, + AstLocal* self, const AstArray& args, std::optional vararg, AstStatBlock* body, size_t functionDepth, + const AstName& debugname, std::optional returnAnnotation = {}, AstTypePack* varargAnnotation = nullptr, bool hasEnd = false, std::optional argLocation = std::nullopt); void visit(AstVisitor* visitor) override; - AstArray generics; - AstArray genericPacks; + AstArray generics; + AstArray genericPacks; AstLocal* self; AstArray args; bool hasReturnAnnotation; @@ -942,14 +956,14 @@ class AstStatTypeAlias : public AstStat public: LUAU_RTTI(AstStatTypeAlias) - AstStatTypeAlias(const Location& location, const AstName& name, const AstArray& generics, const AstArray& genericPacks, - AstType* type, bool exported); + AstStatTypeAlias(const Location& location, const AstName& name, const AstArray& generics, + const AstArray& genericPacks, AstType* type, bool exported); void visit(AstVisitor* visitor) override; AstName name; - AstArray generics; - AstArray genericPacks; + AstArray generics; + AstArray genericPacks; AstType* type; bool exported; }; @@ -972,14 +986,15 @@ class AstStatDeclareFunction : public AstStat public: LUAU_RTTI(AstStatDeclareFunction) - AstStatDeclareFunction(const Location& location, const AstName& name, const AstArray& generics, const AstArray& genericPacks, - const AstTypeList& params, const AstArray& paramNames, const AstTypeList& retTypes); + AstStatDeclareFunction(const Location& location, const AstName& name, const AstArray& generics, + const AstArray& genericPacks, const AstTypeList& params, const AstArray& paramNames, + const AstTypeList& retTypes); void visit(AstVisitor* visitor) override; AstName name; - AstArray generics; - AstArray genericPacks; + AstArray generics; + AstArray genericPacks; AstTypeList params; AstArray paramNames; AstTypeList retTypes; @@ -1077,13 +1092,13 @@ class AstTypeFunction : public AstType public: LUAU_RTTI(AstTypeFunction) - AstTypeFunction(const Location& location, const AstArray& generics, const AstArray& genericPacks, const AstTypeList& argTypes, - const AstArray>& argNames, const AstTypeList& returnTypes); + AstTypeFunction(const Location& location, const AstArray& generics, const AstArray& genericPacks, + const AstTypeList& argTypes, const AstArray>& argNames, const AstTypeList& returnTypes); void visit(AstVisitor* visitor) override; - AstArray generics; - AstArray genericPacks; + AstArray generics; + AstArray genericPacks; AstTypeList argTypes; AstArray> argNames; AstTypeList returnTypes; diff --git a/Ast/include/Luau/Parser.h b/Ast/include/Luau/Parser.h index 87ebc48b..40ecdcdd 100644 --- a/Ast/include/Luau/Parser.h +++ b/Ast/include/Luau/Parser.h @@ -219,7 +219,7 @@ private: AstTableIndexer* parseTableIndexerAnnotation(); AstTypeOrPack parseFunctionTypeAnnotation(bool allowPack); - AstType* parseFunctionTypeAnnotationTail(const Lexeme& begin, AstArray generics, AstArray genericPacks, + AstType* parseFunctionTypeAnnotationTail(const Lexeme& begin, AstArray generics, AstArray genericPacks, AstArray& params, AstArray>& paramNames, AstTypePack* varargAnnotation); AstType* parseTableTypeAnnotation(); @@ -281,7 +281,7 @@ private: Name parseIndexName(const char* context, const Position& previous); // `<' namelist `>' - std::pair, AstArray> parseGenericTypeList(); + std::pair, AstArray> parseGenericTypeList(bool withDefaultValues); // `<' typeAnnotation[, ...] `>' AstArray parseTypeParams(); @@ -418,6 +418,8 @@ private: std::vector scratchDeclaredClassProps; std::vector scratchItem; std::vector scratchArgName; + std::vector scratchGenericTypes; + std::vector scratchGenericTypePacks; std::vector> scratchOptArgName; std::string scratchData; }; diff --git a/Ast/src/Ast.cpp b/Ast/src/Ast.cpp index e709894d..9b5bc0c7 100644 --- a/Ast/src/Ast.cpp +++ b/Ast/src/Ast.cpp @@ -158,9 +158,10 @@ void AstExprIndexExpr::visit(AstVisitor* visitor) } } -AstExprFunction::AstExprFunction(const Location& location, const AstArray& generics, const AstArray& genericPacks, AstLocal* self, - const AstArray& args, std::optional vararg, AstStatBlock* body, size_t functionDepth, const AstName& debugname, - std::optional returnAnnotation, AstTypePack* varargAnnotation, bool hasEnd, std::optional argLocation) +AstExprFunction::AstExprFunction(const Location& location, const AstArray& generics, const AstArray& genericPacks, + AstLocal* self, const AstArray& args, std::optional vararg, AstStatBlock* body, size_t functionDepth, + const AstName& debugname, std::optional returnAnnotation, AstTypePack* varargAnnotation, bool hasEnd, + std::optional argLocation) : AstExpr(ClassIndex(), location) , generics(generics) , genericPacks(genericPacks) @@ -641,8 +642,8 @@ void AstStatLocalFunction::visit(AstVisitor* visitor) func->visit(visitor); } -AstStatTypeAlias::AstStatTypeAlias(const Location& location, const AstName& name, const AstArray& generics, - const AstArray& genericPacks, AstType* type, bool exported) +AstStatTypeAlias::AstStatTypeAlias(const Location& location, const AstName& name, const AstArray& generics, + const AstArray& genericPacks, AstType* type, bool exported) : AstStat(ClassIndex(), location) , name(name) , generics(generics) @@ -655,7 +656,21 @@ AstStatTypeAlias::AstStatTypeAlias(const Location& location, const AstName& name void AstStatTypeAlias::visit(AstVisitor* visitor) { if (visitor->visit(this)) + { + for (const AstGenericType& el : generics) + { + if (el.defaultValue) + el.defaultValue->visit(visitor); + } + + for (const AstGenericTypePack& el : genericPacks) + { + if (el.defaultValue) + el.defaultValue->visit(visitor); + } + type->visit(visitor); + } } AstStatDeclareGlobal::AstStatDeclareGlobal(const Location& location, const AstName& name, AstType* type) @@ -671,8 +686,9 @@ void AstStatDeclareGlobal::visit(AstVisitor* visitor) type->visit(visitor); } -AstStatDeclareFunction::AstStatDeclareFunction(const Location& location, const AstName& name, const AstArray& generics, - const AstArray& genericPacks, const AstTypeList& params, const AstArray& paramNames, const AstTypeList& retTypes) +AstStatDeclareFunction::AstStatDeclareFunction(const Location& location, const AstName& name, const AstArray& generics, + const AstArray& genericPacks, const AstTypeList& params, const AstArray& paramNames, + const AstTypeList& retTypes) : AstStat(ClassIndex(), location) , name(name) , generics(generics) @@ -778,7 +794,7 @@ void AstTypeTable::visit(AstVisitor* visitor) } } -AstTypeFunction::AstTypeFunction(const Location& location, const AstArray& generics, const AstArray& genericPacks, +AstTypeFunction::AstTypeFunction(const Location& location, const AstArray& generics, const AstArray& genericPacks, const AstTypeList& argTypes, const AstArray>& argNames, const AstTypeList& returnTypes) : AstType(ClassIndex(), location) , generics(generics) diff --git a/Ast/src/Parser.cpp b/Ast/src/Parser.cpp index 72f61649..77787cb1 100644 --- a/Ast/src/Parser.cpp +++ b/Ast/src/Parser.cpp @@ -10,10 +10,10 @@ // See docs/SyntaxChanges.md for an explanation. LUAU_FASTINTVARIABLE(LuauRecursionLimit, 1000) LUAU_FASTINTVARIABLE(LuauParseErrorLimit, 100) -LUAU_FASTFLAGVARIABLE(LuauIfElseExpressionBaseSupport, false) -LUAU_FASTFLAGVARIABLE(LuauIfStatementRecursionGuard, false) LUAU_FASTFLAGVARIABLE(LuauFixAmbiguousErrorRecoveryInAssign, false) LUAU_FASTFLAGVARIABLE(LuauParseSingletonTypes, false) +LUAU_FASTFLAGVARIABLE(LuauParseTypeAliasDefaults, false) +LUAU_FASTFLAGVARIABLE(LuauParseRecoverTypePackEllipsis, false) namespace Luau { @@ -394,23 +394,13 @@ AstStat* Parser::parseIf() if (lexer.current().type == Lexeme::ReservedElseif) { - if (FFlag::LuauIfStatementRecursionGuard) - { - unsigned int recursionCounterOld = recursionCounter; - incrementRecursionCounter("elseif"); - elseLocation = lexer.current().location; - elsebody = parseIf(); - end = elsebody->location; - hasEnd = elsebody->as()->hasEnd; - recursionCounter = recursionCounterOld; - } - else - { - elseLocation = lexer.current().location; - elsebody = parseIf(); - end = elsebody->location; - hasEnd = elsebody->as()->hasEnd; - } + unsigned int recursionCounterOld = recursionCounter; + incrementRecursionCounter("elseif"); + elseLocation = lexer.current().location; + elsebody = parseIf(); + end = elsebody->location; + hasEnd = elsebody->as()->hasEnd; + recursionCounter = recursionCounterOld; } else { @@ -772,7 +762,7 @@ AstStat* Parser::parseTypeAlias(const Location& start, bool exported) if (!name) name = Name(nameError, lexer.current().location); - auto [generics, genericPacks] = parseGenericTypeList(); + auto [generics, genericPacks] = parseGenericTypeList(/* withDefaultValues= */ FFlag::LuauParseTypeAliasDefaults); expectAndConsume('=', "type alias"); @@ -788,8 +778,8 @@ AstDeclaredClassProp Parser::parseDeclaredClassMethod() Name fnName = parseName("function name"); // TODO: generic method declarations CLI-39909 - AstArray generics; - AstArray genericPacks; + AstArray generics; + AstArray genericPacks; generics.size = 0; generics.data = nullptr; genericPacks.size = 0; @@ -849,7 +839,7 @@ AstStat* Parser::parseDeclaration(const Location& start) nextLexeme(); Name globalName = parseName("global function name"); - auto [generics, genericPacks] = parseGenericTypeList(); + auto [generics, genericPacks] = parseGenericTypeList(/* withDefaultValues= */ false); Lexeme matchParen = lexer.current(); @@ -991,7 +981,7 @@ std::pair Parser::parseFunctionBody( { Location start = matchFunction.location; - auto [generics, genericPacks] = parseGenericTypeList(); + auto [generics, genericPacks] = parseGenericTypeList(/* withDefaultValues= */ false); Lexeme matchParen = lexer.current(); expectAndConsume('(', "function"); @@ -1228,8 +1218,8 @@ std::pair Parser::parseReturnTypeAnnotation() return {location, AstTypeList{copy(result), varargAnnotation}}; } - AstArray generics{nullptr, 0}; - AstArray genericPacks{nullptr, 0}; + AstArray generics{nullptr, 0}; + AstArray genericPacks{nullptr, 0}; AstArray types = copy(result); AstArray> names = copy(resultNames); @@ -1363,7 +1353,7 @@ AstTypeOrPack Parser::parseFunctionTypeAnnotation(bool allowPack) Lexeme begin = lexer.current(); - auto [generics, genericPacks] = parseGenericTypeList(); + auto [generics, genericPacks] = parseGenericTypeList(/* withDefaultValues= */ false); Lexeme parameterStart = lexer.current(); @@ -1401,7 +1391,7 @@ AstTypeOrPack Parser::parseFunctionTypeAnnotation(bool allowPack) return {parseFunctionTypeAnnotationTail(begin, generics, genericPacks, paramTypes, paramNames, varargAnnotation), {}}; } -AstType* Parser::parseFunctionTypeAnnotationTail(const Lexeme& begin, AstArray generics, AstArray genericPacks, +AstType* Parser::parseFunctionTypeAnnotationTail(const Lexeme& begin, AstArray generics, AstArray genericPacks, AstArray& params, AstArray>& paramNames, AstTypePack* varargAnnotation) { @@ -1448,7 +1438,7 @@ AstType* Parser::parseTypeAnnotation(TempVector& parts, const Location if (c == '|') { nextLexeme(); - parts.push_back(parseSimpleTypeAnnotation(false).type); + parts.push_back(parseSimpleTypeAnnotation(/* allowPack= */ false).type); isUnion = true; } else if (c == '?') @@ -1461,7 +1451,7 @@ AstType* Parser::parseTypeAnnotation(TempVector& parts, const Location else if (c == '&') { nextLexeme(); - parts.push_back(parseSimpleTypeAnnotation(false).type); + parts.push_back(parseSimpleTypeAnnotation(/* allowPack= */ false).type); isIntersection = true; } else @@ -1498,7 +1488,7 @@ AstTypeOrPack Parser::parseTypeOrPackAnnotation() TempVector parts(scratchAnnotation); - auto [type, typePack] = parseSimpleTypeAnnotation(true); + auto [type, typePack] = parseSimpleTypeAnnotation(/* allowPack= */ true); if (typePack) { @@ -1521,7 +1511,7 @@ AstType* Parser::parseTypeAnnotation() Location begin = lexer.current().location; TempVector parts(scratchAnnotation); - parts.push_back(parseSimpleTypeAnnotation(false).type); + parts.push_back(parseSimpleTypeAnnotation(/* allowPack= */ false).type); recursionCounter = oldRecursionCount; @@ -2121,7 +2111,7 @@ AstExpr* Parser::parseSimpleExpr() { return parseTableConstructor(); } - else if (FFlag::LuauIfElseExpressionBaseSupport && lexer.current().type == Lexeme::ReservedIf) + else if (lexer.current().type == Lexeme::ReservedIf) { return parseIfElseExpr(); } @@ -2341,10 +2331,10 @@ Parser::Name Parser::parseIndexName(const char* context, const Position& previou return Name(nameError, location); } -std::pair, AstArray> Parser::parseGenericTypeList() +std::pair, AstArray> Parser::parseGenericTypeList(bool withDefaultValues) { - TempVector names{scratchName}; - TempVector namePacks{scratchPackName}; + TempVector names{scratchGenericTypes}; + TempVector namePacks{scratchGenericTypePacks}; if (lexer.current().type == '<') { @@ -2352,21 +2342,73 @@ std::pair, AstArray> Parser::parseGenericTypeList() nextLexeme(); bool seenPack = false; + bool seenDefault = false; + while (true) { + Location nameLocation = lexer.current().location; AstName name = parseName().name; - if (lexer.current().type == Lexeme::Dot3) + if (lexer.current().type == Lexeme::Dot3 || (FFlag::LuauParseRecoverTypePackEllipsis && seenPack)) { seenPack = true; - nextLexeme(); - namePacks.push_back(name); + + if (FFlag::LuauParseRecoverTypePackEllipsis && lexer.current().type != Lexeme::Dot3) + report(lexer.current().location, "Generic types come before generic type packs"); + else + nextLexeme(); + + if (withDefaultValues && lexer.current().type == '=') + { + seenDefault = true; + nextLexeme(); + + Lexeme packBegin = lexer.current(); + + if (shouldParseTypePackAnnotation(lexer)) + { + auto typePack = parseTypePackAnnotation(); + + namePacks.push_back({name, nameLocation, typePack}); + } + else if (lexer.current().type == '(') + { + auto [type, typePack] = parseTypeOrPackAnnotation(); + + if (type) + report(Location(packBegin.location.begin, lexer.previousLocation().end), "Expected type pack after '=', got type"); + + namePacks.push_back({name, nameLocation, typePack}); + } + } + else + { + if (seenDefault) + report(lexer.current().location, "Expected default type pack after type pack name"); + + namePacks.push_back({name, nameLocation, nullptr}); + } } else { - if (seenPack) + if (!FFlag::LuauParseRecoverTypePackEllipsis && seenPack) report(lexer.current().location, "Generic types come before generic type packs"); - names.push_back(name); + if (withDefaultValues && lexer.current().type == '=') + { + seenDefault = true; + nextLexeme(); + + AstType* defaultType = parseTypeAnnotation(); + + names.push_back({name, nameLocation, defaultType}); + } + else + { + if (seenDefault) + report(lexer.current().location, "Expected default type after type name"); + + names.push_back({name, nameLocation, nullptr}); + } } if (lexer.current().type == ',') @@ -2378,8 +2420,8 @@ std::pair, AstArray> Parser::parseGenericTypeList() expectMatchAndConsume('>', begin); } - AstArray generics = copy(names); - AstArray genericPacks = copy(namePacks); + AstArray generics = copy(names); + AstArray genericPacks = copy(namePacks); return {generics, genericPacks}; } diff --git a/CLI/Analyze.cpp b/CLI/Analyze.cpp index 54a9a26f..e0dc3e0f 100644 --- a/CLI/Analyze.cpp +++ b/CLI/Analyze.cpp @@ -8,6 +8,8 @@ #include "FileUtils.h" +LUAU_FASTFLAG(DebugLuauTimeTracing) + enum class ReportFormat { Default, @@ -105,6 +107,7 @@ static void displayHelp(const char* argv0) printf("Available options:\n"); printf(" --formatter=plain: report analysis errors in Luacheck-compatible format\n"); printf(" --formatter=gnu: report analysis errors in GNU-compatible format\n"); + printf(" --timetrace: record compiler time tracing information into trace.json\n"); } static int assertionHandler(const char* expr, const char* file, int line, const char* function) @@ -213,8 +216,18 @@ int main(int argc, char** argv) format = ReportFormat::Gnu; else if (strcmp(argv[i], "--annotate") == 0) annotate = true; + else if (strcmp(argv[i], "--timetrace") == 0) + FFlag::DebugLuauTimeTracing.value = true; } +#if !defined(LUAU_ENABLE_TIME_TRACE) + if (FFlag::DebugLuauTimeTracing) + { + printf("To run with --timetrace, Luau has to be built with LUAU_ENABLE_TIME_TRACE enabled\n"); + return 1; + } +#endif + Luau::FrontendOptions frontendOptions; frontendOptions.retainFullTypeGraphs = annotate; @@ -240,7 +253,10 @@ int main(int argc, char** argv) fprintf(stderr, "%s: %s\n", pair.first.c_str(), pair.second.c_str()); } - return (format == ReportFormat::Luacheck) ? 0 : failed; + if (format == ReportFormat::Luacheck) + return 0; + else + return failed ? 1 : 0; } diff --git a/CLI/Repl.cpp b/CLI/Repl.cpp index 26d4333a..36747f48 100644 --- a/CLI/Repl.cpp +++ b/CLI/Repl.cpp @@ -19,6 +19,16 @@ #include #endif +LUAU_FASTFLAG(DebugLuauTimeTracing) + +enum class CliMode +{ + Unknown, + Repl, + Compile, + RunSourceFiles +}; + enum class CompileFormat { Text, @@ -485,8 +495,10 @@ static void displayHelp(const char* argv0) printf(" --compile[=format]: compile input files and output resulting formatted bytecode (binary or text)\n"); printf("\n"); printf("Available options:\n"); + printf(" -h, --help: Display this usage message.\n"); printf(" --profile[=N]: profile the code using N Hz sampling (default 10000) and output results to profile.out\n"); printf(" --coverage: collect code coverage while running the code and output results to coverage.out\n"); + printf(" --timetrace: record compiler time tracing information into trace.json\n"); } static int assertionHandler(const char* expr, const char* file, int line, const char* function) @@ -503,71 +515,112 @@ int main(int argc, char** argv) if (strncmp(flag->name, "Luau", 4) == 0) flag->value = true; - if (argc == 1) - { - runRepl(); - return 0; - } - - if (argc >= 2 && strcmp(argv[1], "--help") == 0) - { - displayHelp(argv[0]); - return 0; - } - + CliMode mode = CliMode::Unknown; + CompileFormat compileFormat{}; + int profile = 0; + bool coverage = false; + // Set the mode if the user has explicitly specified one. + int argStart = 1; if (argc >= 2 && strncmp(argv[1], "--compile", strlen("--compile")) == 0) { - CompileFormat format = CompileFormat::Text; + argStart++; + mode = CliMode::Compile; + if (strcmp(argv[1], "--compile") == 0) + { + compileFormat = CompileFormat::Text; + } + else if (strcmp(argv[1], "--compile=binary") == 0) + { + compileFormat = CompileFormat::Binary; + } + else if (strcmp(argv[1], "--compile=text") == 0) + { + compileFormat = CompileFormat::Text; + } + else + { + fprintf(stdout, "Error: Unrecognized value for '--compile' specified.\n"); + return -1; + } + } - if (strcmp(argv[1], "--compile=binary") == 0) - format = CompileFormat::Binary; + for (int i = argStart; i < argc; i++) + { + if (strcmp(argv[i], "-h") == 0 || strcmp(argv[i], "--help") == 0) + { + displayHelp(argv[0]); + return 0; + } + else if (strcmp(argv[i], "--profile") == 0) + { + profile = 10000; // default to 10 KHz + } + else if (strncmp(argv[i], "--profile=", 10) == 0) + { + profile = atoi(argv[i] + 10); + } + else if (strcmp(argv[i], "--coverage") == 0) + { + coverage = true; + } + else if (strcmp(argv[i], "--timetrace") == 0) + { + FFlag::DebugLuauTimeTracing.value = true; +#if !defined(LUAU_ENABLE_TIME_TRACE) + printf("To run with --timetrace, Luau has to be built with LUAU_ENABLE_TIME_TRACE enabled\n"); + return 1; +#endif + } + else if (argv[i][0] == '-') + { + fprintf(stdout, "Error: Unrecognized option '%s'.\n\n", argv[i]); + displayHelp(argv[0]); + return 1; + } + } + + const std::vector files = getSourceFiles(argc, argv); + if (mode == CliMode::Unknown) + { + mode = files.empty() ? CliMode::Repl : CliMode::RunSourceFiles; + } + + switch (mode) + { + case CliMode::Compile: + { #ifdef _WIN32 - if (format == CompileFormat::Binary) + if (compileFormat == CompileFormat::Binary) _setmode(_fileno(stdout), _O_BINARY); #endif - std::vector files = getSourceFiles(argc, argv); - int failed = 0; for (const std::string& path : files) - failed += !compileFile(path.c_str(), format); + failed += !compileFile(path.c_str(), compileFormat); - return failed; + return failed ? 1 : 0; } - + case CliMode::Repl: + { + runRepl(); + return 0; + } + case CliMode::RunSourceFiles: { std::unique_ptr globalState(luaL_newstate(), lua_close); lua_State* L = globalState.get(); setupState(L); - int profile = 0; - bool coverage = false; - - for (int i = 1; i < argc; ++i) - { - if (argv[i][0] != '-') - continue; - - if (strcmp(argv[i], "--profile") == 0) - profile = 10000; // default to 10 KHz - else if (strncmp(argv[i], "--profile=", 10) == 0) - profile = atoi(argv[i] + 10); - else if (strcmp(argv[i], "--coverage") == 0) - coverage = true; - } - if (profile) profilerStart(L, profile); if (coverage) coverageInit(L); - std::vector files = getSourceFiles(argc, argv); - int failed = 0; for (const std::string& path : files) @@ -582,7 +635,11 @@ int main(int argc, char** argv) if (coverage) coverageDump("coverage.out"); - return failed; + return failed ? 1 : 0; + } + case CliMode::Unknown: + default: + LUAU_ASSERT(!"Unhandled cli mode."); } } diff --git a/Compiler/src/Builtins.cpp b/Compiler/src/Builtins.cpp new file mode 100644 index 00000000..e344eb91 --- /dev/null +++ b/Compiler/src/Builtins.cpp @@ -0,0 +1,197 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +#include "Builtins.h" + +#include "Luau/Bytecode.h" +#include "Luau/Compiler.h" + +namespace Luau +{ +namespace Compile +{ + +Builtin getBuiltin(AstExpr* node, const DenseHashMap& globals, const DenseHashMap& variables) +{ + if (AstExprLocal* expr = node->as()) + { + const Variable* v = variables.find(expr->local); + + return v && !v->written && v->init ? getBuiltin(v->init, globals, variables) : Builtin(); + } + else if (AstExprIndexName* expr = node->as()) + { + if (AstExprGlobal* object = expr->expr->as()) + { + return getGlobalState(globals, object->name) == Global::Default ? Builtin{object->name, expr->index} : Builtin(); + } + else + { + return Builtin(); + } + } + else if (AstExprGlobal* expr = node->as()) + { + return getGlobalState(globals, expr->name) == Global::Default ? Builtin{AstName(), expr->name} : Builtin(); + } + else + { + return Builtin(); + } +} + +int getBuiltinFunctionId(const Builtin& builtin, const CompileOptions& options) +{ + if (builtin.empty()) + return -1; + + if (builtin.isGlobal("assert")) + return LBF_ASSERT; + + if (builtin.isGlobal("type")) + return LBF_TYPE; + + if (builtin.isGlobal("typeof")) + return LBF_TYPEOF; + + if (builtin.isGlobal("rawset")) + return LBF_RAWSET; + if (builtin.isGlobal("rawget")) + return LBF_RAWGET; + if (builtin.isGlobal("rawequal")) + return LBF_RAWEQUAL; + + if (builtin.isGlobal("unpack")) + return LBF_TABLE_UNPACK; + + if (builtin.object == "math") + { + if (builtin.method == "abs") + return LBF_MATH_ABS; + if (builtin.method == "acos") + return LBF_MATH_ACOS; + if (builtin.method == "asin") + return LBF_MATH_ASIN; + if (builtin.method == "atan2") + return LBF_MATH_ATAN2; + if (builtin.method == "atan") + return LBF_MATH_ATAN; + if (builtin.method == "ceil") + return LBF_MATH_CEIL; + if (builtin.method == "cosh") + return LBF_MATH_COSH; + if (builtin.method == "cos") + return LBF_MATH_COS; + if (builtin.method == "deg") + return LBF_MATH_DEG; + if (builtin.method == "exp") + return LBF_MATH_EXP; + if (builtin.method == "floor") + return LBF_MATH_FLOOR; + if (builtin.method == "fmod") + return LBF_MATH_FMOD; + if (builtin.method == "frexp") + return LBF_MATH_FREXP; + if (builtin.method == "ldexp") + return LBF_MATH_LDEXP; + if (builtin.method == "log10") + return LBF_MATH_LOG10; + if (builtin.method == "log") + return LBF_MATH_LOG; + if (builtin.method == "max") + return LBF_MATH_MAX; + if (builtin.method == "min") + return LBF_MATH_MIN; + if (builtin.method == "modf") + return LBF_MATH_MODF; + if (builtin.method == "pow") + return LBF_MATH_POW; + if (builtin.method == "rad") + return LBF_MATH_RAD; + if (builtin.method == "sinh") + return LBF_MATH_SINH; + if (builtin.method == "sin") + return LBF_MATH_SIN; + if (builtin.method == "sqrt") + return LBF_MATH_SQRT; + if (builtin.method == "tanh") + return LBF_MATH_TANH; + if (builtin.method == "tan") + return LBF_MATH_TAN; + if (builtin.method == "clamp") + return LBF_MATH_CLAMP; + if (builtin.method == "sign") + return LBF_MATH_SIGN; + if (builtin.method == "round") + return LBF_MATH_ROUND; + } + + if (builtin.object == "bit32") + { + if (builtin.method == "arshift") + return LBF_BIT32_ARSHIFT; + if (builtin.method == "band") + return LBF_BIT32_BAND; + if (builtin.method == "bnot") + return LBF_BIT32_BNOT; + if (builtin.method == "bor") + return LBF_BIT32_BOR; + if (builtin.method == "bxor") + return LBF_BIT32_BXOR; + if (builtin.method == "btest") + return LBF_BIT32_BTEST; + if (builtin.method == "extract") + return LBF_BIT32_EXTRACT; + if (builtin.method == "lrotate") + return LBF_BIT32_LROTATE; + if (builtin.method == "lshift") + return LBF_BIT32_LSHIFT; + if (builtin.method == "replace") + return LBF_BIT32_REPLACE; + if (builtin.method == "rrotate") + return LBF_BIT32_RROTATE; + if (builtin.method == "rshift") + return LBF_BIT32_RSHIFT; + if (builtin.method == "countlz") + return LBF_BIT32_COUNTLZ; + if (builtin.method == "countrz") + return LBF_BIT32_COUNTRZ; + } + + if (builtin.object == "string") + { + if (builtin.method == "byte") + return LBF_STRING_BYTE; + if (builtin.method == "char") + return LBF_STRING_CHAR; + if (builtin.method == "len") + return LBF_STRING_LEN; + if (builtin.method == "sub") + return LBF_STRING_SUB; + } + + if (builtin.object == "table") + { + if (builtin.method == "insert") + return LBF_TABLE_INSERT; + if (builtin.method == "unpack") + return LBF_TABLE_UNPACK; + } + + if (options.vectorCtor) + { + if (options.vectorLib) + { + if (builtin.isMethod(options.vectorLib, options.vectorCtor)) + return LBF_VECTOR; + } + else + { + if (builtin.isGlobal(options.vectorCtor)) + return LBF_VECTOR; + } + } + + return -1; +} + +} // namespace Compile +} // namespace Luau diff --git a/Compiler/src/Builtins.h b/Compiler/src/Builtins.h new file mode 100644 index 00000000..60df53a1 --- /dev/null +++ b/Compiler/src/Builtins.h @@ -0,0 +1,41 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +#pragma once + +#include "ValueTracking.h" + +namespace Luau +{ +struct CompileOptions; +} + +namespace Luau +{ +namespace Compile +{ + +struct Builtin +{ + AstName object; + AstName method; + + bool empty() const + { + return object == AstName() && method == AstName(); + } + + bool isGlobal(const char* name) const + { + return object == AstName() && method == name; + } + + bool isMethod(const char* table, const char* name) const + { + return object == table && method == name; + } +}; + +Builtin getBuiltin(AstExpr* node, const DenseHashMap& globals, const DenseHashMap& variables); +int getBuiltinFunctionId(const Builtin& builtin, const CompileOptions& options); + +} // namespace Compile +} // namespace Luau diff --git a/Compiler/src/BytecodeBuilder.cpp b/Compiler/src/BytecodeBuilder.cpp index 2d31c409..e6d02454 100644 --- a/Compiler/src/BytecodeBuilder.cpp +++ b/Compiler/src/BytecodeBuilder.cpp @@ -714,7 +714,7 @@ void BytecodeBuilder::writeLineInfo(std::string& ss) const // third pass: write resulting data int logspan = log2(span); - writeByte(ss, logspan); + writeByte(ss, uint8_t(logspan)); uint8_t lastOffset = 0; @@ -723,8 +723,8 @@ void BytecodeBuilder::writeLineInfo(std::string& ss) const int delta = lines[i] - baseline[i >> logspan]; LUAU_ASSERT(delta >= 0 && delta <= 255); - writeByte(ss, delta - lastOffset); - lastOffset = delta; + writeByte(ss, uint8_t(delta) - lastOffset); + lastOffset = uint8_t(delta); } int lastLine = 0; diff --git a/Compiler/src/Compiler.cpp b/Compiler/src/Compiler.cpp index 6ae49027..9758c4a9 100644 --- a/Compiler/src/Compiler.cpp +++ b/Compiler/src/Compiler.cpp @@ -6,15 +6,20 @@ #include "Luau/Common.h" #include "Luau/TimeTrace.h" +#include "Builtins.h" +#include "ConstantFolding.h" +#include "TableShape.h" +#include "ValueTracking.h" + #include #include #include -LUAU_FASTFLAG(LuauIfElseExpressionBaseSupport) - namespace Luau { +using namespace Luau::Compile; + static const uint32_t kMaxRegisterCount = 255; static const uint32_t kMaxUpvalueCount = 200; static const uint32_t kMaxLocalCount = 200; @@ -62,7 +67,6 @@ static BytecodeBuilder::StringRef sref(AstArray data) struct Compiler { - struct Constant; struct RegScope; Compiler(BytecodeBuilder& bytecode, const CompileOptions& options) @@ -71,8 +75,9 @@ struct Compiler , functions(nullptr) , locals(nullptr) , globals(AstName()) + , variables(nullptr) , constants(nullptr) - , predictedTableSize(nullptr) + , tableShapes(nullptr) { } @@ -96,8 +101,10 @@ struct Compiler local->location, "Out of upvalue registers when trying to allocate %s: exceeded limit %d", local->name.value, kMaxUpvalueCount); // mark local as captured so that closeLocals emits LOP_CLOSEUPVALS accordingly - Local& l = locals[local]; - l.captured = true; + Variable* v = variables.find(local); + + if (v && v->written) + locals[local].captured = true; upvals.push_back(local); @@ -273,8 +280,8 @@ struct Compiler if (options.optimizationLevel >= 1) { - Builtin builtin = getBuiltin(expr->func); - bfid = getBuiltinFunctionId(builtin); + Builtin builtin = getBuiltin(expr->func, globals, variables); + bfid = getBuiltinFunctionId(builtin, options); } if (expr->self) @@ -364,12 +371,12 @@ struct Compiler else { args[i] = uint8_t(regs + 1 + i); - compileExprTempTop(expr->args.data[i], args[i]); + compileExprTempTop(expr->args.data[i], uint8_t(args[i])); } } fastcallLabel = bytecode.emitLabel(); - bytecode.emitABC(opc, uint8_t(bfid), args[0], 0); + bytecode.emitABC(opc, uint8_t(bfid), uint8_t(args[0]), 0); if (opc != LOP_FASTCALL1) bytecode.emitAux(args[1]); @@ -385,7 +392,7 @@ struct Compiler } if (args[i] != regs + 1 + i) - bytecode.emitABC(LOP_MOVE, uint8_t(regs + 1 + i), args[i], 0); + bytecode.emitABC(LOP_MOVE, uint8_t(regs + 1 + i), uint8_t(args[i]), 0); } } else @@ -424,8 +431,10 @@ struct Compiler for (AstLocal* uv : f->upvals) { - Local* ul = locals.find(uv); - LUAU_ASSERT(ul); + Variable* ul = variables.find(uv); + + if (!ul) + return false; if (ul->written) return false; @@ -437,10 +446,11 @@ struct Compiler // this will only deoptimize (outside of fenv changes) if top level code is executed twice with different results. if (uv->functionDepth != 0 || uv->loopDepth != 0) { - if (!ul->func) + AstExprFunction* uf = ul->init ? ul->init->as() : nullptr; + if (!uf) return false; - if (ul->func != func && !shouldShareClosure(ul->func)) + if (uf != func && !shouldShareClosure(uf)) return false; } } @@ -471,7 +481,7 @@ struct Compiler if (cid >= 0 && cid < 32768) { - bytecode.emitAD(LOP_DUPCLOSURE, target, cid); + bytecode.emitAD(LOP_DUPCLOSURE, target, int16_t(cid)); shared = true; } } @@ -483,17 +493,15 @@ struct Compiler { LUAU_ASSERT(uv->functionDepth < expr->functionDepth); - Local* ul = locals.find(uv); - LUAU_ASSERT(ul); - - bool immutable = !ul->written; + Variable* ul = variables.find(uv); + bool immutable = !ul || !ul->written; if (uv->functionDepth == expr->functionDepth - 1) { // get local variable uint8_t reg = getLocal(uv); - bytecode.emitABC(LOP_CAPTURE, immutable ? LCT_VAL : LCT_REF, reg, 0); + bytecode.emitABC(LOP_CAPTURE, uint8_t(immutable ? LCT_VAL : LCT_REF), reg, 0); } else { @@ -635,7 +643,7 @@ struct Compiler if (expr->op == AstExprBinary::CompareGt || expr->op == AstExprBinary::CompareGe) { - bytecode.emitAD(opc, rr, 0); + bytecode.emitAD(opc, uint8_t(rr), 0); bytecode.emitAux(rl); } else @@ -687,7 +695,7 @@ struct Compiler break; case Constant::Type_String: - cid = bytecode.addConstantString(sref(c->valueString)); + cid = bytecode.addConstantString(sref(c->getString())); break; default: @@ -1066,10 +1074,10 @@ struct Compiler // Optimization: if the table is empty, we can compute it directly into the target if (expr->items.size == 0) { - auto [hashSize, arraySize] = predictedTableSize[expr]; + TableShape shape = tableShapes[expr]; - bytecode.emitABC(LOP_NEWTABLE, target, encodeHashSize(hashSize), 0); - bytecode.emitAux(arraySize); + bytecode.emitABC(LOP_NEWTABLE, target, encodeHashSize(shape.hashSize), 0); + bytecode.emitAux(shape.arraySize); return; } @@ -1144,7 +1152,7 @@ struct Compiler } else { - bytecode.emitABC(LOP_NEWTABLE, reg, encodedHashSize, 0); + bytecode.emitABC(LOP_NEWTABLE, reg, uint8_t(encodedHashSize), 0); bytecode.emitAux(0); } } @@ -1157,7 +1165,7 @@ struct Compiler bool trailingVarargs = last && last->kind == AstExprTable::Item::List && last->value->is(); LUAU_ASSERT(!trailingVarargs || arraySize > 0); - bytecode.emitABC(LOP_NEWTABLE, reg, encodedHashSize, 0); + bytecode.emitABC(LOP_NEWTABLE, reg, uint8_t(encodedHashSize), 0); bytecode.emitAux(arraySize - trailingVarargs + indexSize); } @@ -1252,16 +1260,12 @@ struct Compiler bool canImport(AstExprGlobal* expr) { - const Global* global = globals.find(expr->name); - - return options.optimizationLevel >= 1 && (!global || !global->written); + return options.optimizationLevel >= 1 && getGlobalState(globals, expr->name) != Global::Written; } bool canImportChain(AstExprGlobal* expr) { - const Global* global = globals.find(expr->name); - - return options.optimizationLevel >= 1 && (!global || (!global->written && !global->writable)); + return options.optimizationLevel >= 1 && getGlobalState(globals, expr->name) == Global::Default; } void compileExprIndexName(AstExprIndexName* expr, uint8_t target) @@ -1341,7 +1345,7 @@ struct Compiler { uint8_t rt = compileExprAuto(expr->expr, rs); - BytecodeBuilder::StringRef iname = sref(cv->valueString); + BytecodeBuilder::StringRef iname = sref(cv->getString()); int32_t cid = bytecode.addConstantString(iname); if (cid < 0) CompileError::raise(expr->location, "Exceeded constant limit; simplify the code to compile"); @@ -1427,7 +1431,7 @@ struct Compiler case Constant::Type_String: { - int32_t cid = bytecode.addConstantString(sref(cv->valueString)); + int32_t cid = bytecode.addConstantString(sref(cv->getString())); if (cid < 0) CompileError::raise(node->location, "Exceeded constant limit; simplify the code to compile"); @@ -1546,7 +1550,7 @@ struct Compiler { compileExpr(expr->expr, target, targetTemp); } - else if (AstExprIfElse* expr = node->as(); FFlag::LuauIfElseExpressionBaseSupport && expr) + else if (AstExprIfElse* expr = node->as()) { compileExprIfElse(expr, target, targetTemp); } @@ -1711,7 +1715,7 @@ struct Compiler { LValue result = {LValue::Kind_IndexName}; result.reg = compileExprAuto(expr->expr, rs); - result.name = sref(cv->valueString); + result.name = sref(cv->getString()); result.location = node->location; return result; @@ -1796,9 +1800,8 @@ struct Compiler return false; Local* l = locals.find(le->local); - LUAU_ASSERT(l); - return l->allocated; + return l && l->allocated; } bool isStatBreak(AstStat* node) @@ -2040,9 +2043,9 @@ struct Compiler for (AstLocal* local : stat->vars) { - Local* l = locals.find(local); + Variable* v = variables.find(local); - if (!l || l->constant.type == Constant::Type_Unknown) + if (!v || !v->constant) return false; } @@ -2082,9 +2085,7 @@ struct Compiler // through) uint8_t varreg = regs + 2; - Local* il = locals.find(stat->var); - - if (il && il->written) + if (Variable* il = variables.find(stat->var); il && il->written) varreg = allocReg(stat, 1); compileExprTemp(stat->from, uint8_t(regs + 2)); @@ -2164,7 +2165,7 @@ struct Compiler { if (stat->values.size == 1 && stat->values.data[0]->is()) { - Builtin builtin = getBuiltin(stat->values.data[0]->as()->func); + Builtin builtin = getBuiltin(stat->values.data[0]->as()->func, globals, variables); if (builtin.isGlobal("ipairs")) // for .. in ipairs(t) { @@ -2179,7 +2180,7 @@ struct Compiler } else if (stat->values.size == 2) { - Builtin builtin = getBuiltin(stat->values.data[0]); + Builtin builtin = getBuiltin(stat->values.data[0], globals, variables); if (builtin.isGlobal("next")) // for .. in next,t { @@ -2594,7 +2595,7 @@ struct Compiler Local* l = locals.find(localStack[i]); LUAU_ASSERT(l); - if (l->captured && l->written) + if (l->captured) return true; } @@ -2613,7 +2614,7 @@ struct Compiler Local* l = locals.find(localStack[i]); LUAU_ASSERT(l); - if (l->captured && l->written) + if (l->captured) { captured = true; captureReg = std::min(captureReg, l->reg); @@ -2728,519 +2729,6 @@ struct Compiler return !node->is() && !node->is(); } - struct AssignmentVisitor : AstVisitor - { - struct Hasher - { - size_t operator()(const std::pair& p) const - { - return std::hash()(p.first) ^ std::hash()(p.second); - } - }; - - DenseHashMap localToTable; - DenseHashSet, Hasher> fields; - - AssignmentVisitor(Compiler* self) - : localToTable(nullptr) - , fields(std::pair()) - , self(self) - { - } - - void assignField(AstExpr* expr, AstName index) - { - if (AstExprLocal* lv = expr->as()) - { - if (AstExprTable** table = localToTable.find(lv->local)) - { - std::pair field = {*table, index}; - - if (!fields.contains(field)) - { - fields.insert(field); - self->predictedTableSize[*table].first += 1; - } - } - } - } - - void assignField(AstExpr* expr, AstExpr* index) - { - AstExprLocal* lv = expr->as(); - AstExprConstantNumber* number = index->as(); - - if (lv && number) - { - if (AstExprTable** table = localToTable.find(lv->local)) - { - unsigned int& arraySize = self->predictedTableSize[*table].second; - - if (number->value == double(arraySize + 1)) - arraySize += 1; - } - } - } - - void assign(AstExpr* var) - { - if (AstExprLocal* lv = var->as()) - { - self->locals[lv->local].written = true; - } - else if (AstExprGlobal* gv = var->as()) - { - self->globals[gv->name].written = true; - } - else if (AstExprIndexName* index = var->as()) - { - assignField(index->expr, index->index); - - var->visit(this); - } - else if (AstExprIndexExpr* index = var->as()) - { - assignField(index->expr, index->index); - - var->visit(this); - } - else - { - // we need to be able to track assignments in all expressions, including crazy ones like t[function() t = nil end] = 5 - var->visit(this); - } - } - - AstExprTable* getTableHint(AstExpr* expr) - { - // unadorned table literal - if (AstExprTable* table = expr->as()) - return table; - - // setmetatable(table literal, ...) - if (AstExprCall* call = expr->as(); call && !call->self && call->args.size == 2) - if (AstExprGlobal* func = call->func->as(); func && func->name == "setmetatable") - if (AstExprTable* table = call->args.data[0]->as()) - return table; - - return nullptr; - } - - bool visit(AstStatLocal* node) override - { - // track local -> table association so that we can update table size prediction in assignField - if (node->vars.size == 1 && node->values.size == 1) - if (AstExprTable* table = getTableHint(node->values.data[0]); table && table->items.size == 0) - localToTable[node->vars.data[0]] = table; - - return true; - } - - bool visit(AstStatAssign* node) override - { - for (size_t i = 0; i < node->vars.size; ++i) - assign(node->vars.data[i]); - - for (size_t i = 0; i < node->values.size; ++i) - node->values.data[i]->visit(this); - - return false; - } - - bool visit(AstStatCompoundAssign* node) override - { - assign(node->var); - node->value->visit(this); - - return false; - } - - bool visit(AstStatFunction* node) override - { - assign(node->name); - node->func->visit(this); - - return false; - } - - Compiler* self; - }; - - struct ConstantVisitor : AstVisitor - { - ConstantVisitor(Compiler* self) - : self(self) - { - } - - void analyzeUnary(Constant& result, AstExprUnary::Op op, const Constant& arg) - { - switch (op) - { - case AstExprUnary::Not: - if (arg.type != Constant::Type_Unknown) - { - result.type = Constant::Type_Boolean; - result.valueBoolean = !arg.isTruthful(); - } - break; - - case AstExprUnary::Minus: - if (arg.type == Constant::Type_Number) - { - result.type = Constant::Type_Number; - result.valueNumber = -arg.valueNumber; - } - break; - - case AstExprUnary::Len: - if (arg.type == Constant::Type_String) - { - result.type = Constant::Type_Number; - result.valueNumber = double(arg.valueString.size); - } - break; - - default: - LUAU_ASSERT(!"Unexpected unary operation"); - } - } - - bool constantsEqual(const Constant& la, const Constant& ra) - { - LUAU_ASSERT(la.type != Constant::Type_Unknown && ra.type != Constant::Type_Unknown); - - switch (la.type) - { - case Constant::Type_Nil: - return ra.type == Constant::Type_Nil; - - case Constant::Type_Boolean: - return ra.type == Constant::Type_Boolean && la.valueBoolean == ra.valueBoolean; - - case Constant::Type_Number: - return ra.type == Constant::Type_Number && la.valueNumber == ra.valueNumber; - - case Constant::Type_String: - return ra.type == Constant::Type_String && la.valueString.size == ra.valueString.size && - memcmp(la.valueString.data, ra.valueString.data, la.valueString.size) == 0; - - default: - LUAU_ASSERT(!"Unexpected constant type in comparison"); - return false; - } - } - - void analyzeBinary(Constant& result, AstExprBinary::Op op, const Constant& la, const Constant& ra) - { - switch (op) - { - case AstExprBinary::Add: - if (la.type == Constant::Type_Number && ra.type == Constant::Type_Number) - { - result.type = Constant::Type_Number; - result.valueNumber = la.valueNumber + ra.valueNumber; - } - break; - - case AstExprBinary::Sub: - if (la.type == Constant::Type_Number && ra.type == Constant::Type_Number) - { - result.type = Constant::Type_Number; - result.valueNumber = la.valueNumber - ra.valueNumber; - } - break; - - case AstExprBinary::Mul: - if (la.type == Constant::Type_Number && ra.type == Constant::Type_Number) - { - result.type = Constant::Type_Number; - result.valueNumber = la.valueNumber * ra.valueNumber; - } - break; - - case AstExprBinary::Div: - if (la.type == Constant::Type_Number && ra.type == Constant::Type_Number) - { - result.type = Constant::Type_Number; - result.valueNumber = la.valueNumber / ra.valueNumber; - } - break; - - case AstExprBinary::Mod: - if (la.type == Constant::Type_Number && ra.type == Constant::Type_Number) - { - result.type = Constant::Type_Number; - result.valueNumber = la.valueNumber - floor(la.valueNumber / ra.valueNumber) * ra.valueNumber; - } - break; - - case AstExprBinary::Pow: - if (la.type == Constant::Type_Number && ra.type == Constant::Type_Number) - { - result.type = Constant::Type_Number; - result.valueNumber = pow(la.valueNumber, ra.valueNumber); - } - break; - - case AstExprBinary::Concat: - break; - - case AstExprBinary::CompareNe: - if (la.type != Constant::Type_Unknown && ra.type != Constant::Type_Unknown) - { - result.type = Constant::Type_Boolean; - result.valueBoolean = !constantsEqual(la, ra); - } - break; - - case AstExprBinary::CompareEq: - if (la.type != Constant::Type_Unknown && ra.type != Constant::Type_Unknown) - { - result.type = Constant::Type_Boolean; - result.valueBoolean = constantsEqual(la, ra); - } - break; - - case AstExprBinary::CompareLt: - if (la.type == Constant::Type_Number && ra.type == Constant::Type_Number) - { - result.type = Constant::Type_Boolean; - result.valueBoolean = la.valueNumber < ra.valueNumber; - } - break; - - case AstExprBinary::CompareLe: - if (la.type == Constant::Type_Number && ra.type == Constant::Type_Number) - { - result.type = Constant::Type_Boolean; - result.valueBoolean = la.valueNumber <= ra.valueNumber; - } - break; - - case AstExprBinary::CompareGt: - if (la.type == Constant::Type_Number && ra.type == Constant::Type_Number) - { - result.type = Constant::Type_Boolean; - result.valueBoolean = la.valueNumber > ra.valueNumber; - } - break; - - case AstExprBinary::CompareGe: - if (la.type == Constant::Type_Number && ra.type == Constant::Type_Number) - { - result.type = Constant::Type_Boolean; - result.valueBoolean = la.valueNumber >= ra.valueNumber; - } - break; - - case AstExprBinary::And: - if (la.type != Constant::Type_Unknown) - { - result = la.isTruthful() ? ra : la; - } - break; - - case AstExprBinary::Or: - if (la.type != Constant::Type_Unknown) - { - result = la.isTruthful() ? la : ra; - } - break; - - default: - LUAU_ASSERT(!"Unexpected binary operation"); - } - } - - Constant analyze(AstExpr* node) - { - Constant result; - result.type = Constant::Type_Unknown; - - if (AstExprGroup* expr = node->as()) - { - result = analyze(expr->expr); - } - else if (node->is()) - { - result.type = Constant::Type_Nil; - } - else if (AstExprConstantBool* expr = node->as()) - { - result.type = Constant::Type_Boolean; - result.valueBoolean = expr->value; - } - else if (AstExprConstantNumber* expr = node->as()) - { - result.type = Constant::Type_Number; - result.valueNumber = expr->value; - } - else if (AstExprConstantString* expr = node->as()) - { - result.type = Constant::Type_String; - result.valueString = expr->value; - } - else if (AstExprLocal* expr = node->as()) - { - const Local* l = self->locals.find(expr->local); - - if (l && l->constant.type != Constant::Type_Unknown) - { - LUAU_ASSERT(!l->written); - result = l->constant; - } - } - else if (node->is()) - { - // nope - } - else if (node->is()) - { - // nope - } - else if (AstExprCall* expr = node->as()) - { - analyze(expr->func); - - for (size_t i = 0; i < expr->args.size; ++i) - analyze(expr->args.data[i]); - } - else if (AstExprIndexName* expr = node->as()) - { - analyze(expr->expr); - } - else if (AstExprIndexExpr* expr = node->as()) - { - analyze(expr->expr); - analyze(expr->index); - } - else if (AstExprFunction* expr = node->as()) - { - // this is necessary to propagate constant information in all child functions - expr->body->visit(this); - } - else if (AstExprTable* expr = node->as()) - { - for (size_t i = 0; i < expr->items.size; ++i) - { - const AstExprTable::Item& item = expr->items.data[i]; - - if (item.key) - analyze(item.key); - - analyze(item.value); - } - } - else if (AstExprUnary* expr = node->as()) - { - Constant arg = analyze(expr->expr); - - analyzeUnary(result, expr->op, arg); - } - else if (AstExprBinary* expr = node->as()) - { - Constant la = analyze(expr->left); - Constant ra = analyze(expr->right); - - analyzeBinary(result, expr->op, la, ra); - } - else if (AstExprTypeAssertion* expr = node->as()) - { - Constant arg = analyze(expr->expr); - - result = arg; - } - else if (AstExprIfElse* expr = node->as(); FFlag::LuauIfElseExpressionBaseSupport && expr) - { - Constant cond = analyze(expr->condition); - Constant trueExpr = analyze(expr->trueExpr); - Constant falseExpr = analyze(expr->falseExpr); - if (cond.type != Constant::Type_Unknown) - { - result = cond.isTruthful() ? trueExpr : falseExpr; - } - } - else - { - LUAU_ASSERT(!"Unknown expression type"); - } - - if (result.type != Constant::Type_Unknown) - self->constants[node] = result; - - return result; - } - - bool visit(AstExpr* node) override - { - // note: we short-circuit the visitor traversal through any expression trees by returning false - // recursive traversal is happening inside analyze() which makes it easier to get the resulting value of the subexpression - analyze(node); - - return false; - } - - bool visit(AstStatLocal* node) override - { - // for values that match 1-1 we record the initializing expression for future analysis - for (size_t i = 0; i < node->vars.size && i < node->values.size; ++i) - { - Local& l = self->locals[node->vars.data[i]]; - - l.init = node->values.data[i]; - } - - // all values that align wrt indexing are simple - we just match them 1-1 - for (size_t i = 0; i < node->vars.size && i < node->values.size; ++i) - { - Constant arg = analyze(node->values.data[i]); - - if (arg.type != Constant::Type_Unknown) - { - Local& l = self->locals[node->vars.data[i]]; - - // note: we rely on AssignmentVisitor to have been run before us - if (!l.written) - l.constant = arg; - } - } - - if (node->vars.size > node->values.size) - { - // if we have trailing variables, then depending on whether the last value is capable of returning multiple values - // (aka call or varargs), we either don't know anything about these vars, or we know they're nil - AstExpr* last = node->values.size ? node->values.data[node->values.size - 1] : nullptr; - bool multRet = last && (last->is() || last->is()); - - for (size_t i = node->values.size; i < node->vars.size; ++i) - { - if (!multRet) - { - Local& l = self->locals[node->vars.data[i]]; - - // note: we rely on AssignmentVisitor to have been run before us - if (!l.written) - { - l.constant.type = Constant::Type_Nil; - } - } - } - } - else - { - // we can have more values than variables; in this case we still need to analyze them to make sure we do constant propagation inside - // them - for (size_t i = node->vars.size; i < node->values.size; ++i) - analyze(node->values.data[i]); - } - - return false; - } - - Compiler* self; - }; - struct FenvVisitor : AstVisitor { bool& getfenvUsed; @@ -3283,14 +2771,6 @@ struct Compiler return false; } - - bool visit(AstStatLocalFunction* node) override - { - // record local->function association for some optimizations - self->locals[node->name].func = node->func; - - return true; - } }; struct UndefinedLocalVisitor : AstVisitor @@ -3397,70 +2877,12 @@ struct Compiler std::vector upvals; }; - struct Constant - { - enum Type - { - Type_Unknown, - Type_Nil, - Type_Boolean, - Type_Number, - Type_String, - }; - - Type type = Type_Unknown; - - union - { - bool valueBoolean; - double valueNumber; - AstArray valueString = {}; - }; - - bool isTruthful() const - { - LUAU_ASSERT(type != Type_Unknown); - return type != Type_Nil && !(type == Type_Boolean && valueBoolean == false); - } - }; - struct Local { uint8_t reg = 0; bool allocated = false; bool captured = false; - bool written = false; - AstExpr* init = nullptr; uint32_t debugpc = 0; - Constant constant; - AstExprFunction* func = nullptr; - }; - - struct Global - { - bool writable = false; - bool written = false; - }; - - struct Builtin - { - AstName object; - AstName method; - - bool empty() const - { - return object == AstName() && method == AstName(); - } - - bool isGlobal(const char* name) const - { - return object == AstName() && method == name; - } - - bool isMethod(const char* table, const char* name) const - { - return object == table && method == name; - } }; struct LoopJump @@ -3482,194 +2904,6 @@ struct Compiler AstExpr* untilCondition; }; - Builtin getBuiltin(AstExpr* node) - { - if (AstExprLocal* expr = node->as()) - { - Local* l = locals.find(expr->local); - - return l && !l->written && l->init ? getBuiltin(l->init) : Builtin(); - } - else if (AstExprIndexName* expr = node->as()) - { - if (AstExprGlobal* object = expr->expr->as()) - { - Global* g = globals.find(object->name); - - return !g || (!g->writable && !g->written) ? Builtin{object->name, expr->index} : Builtin(); - } - else - { - return Builtin(); - } - } - else if (AstExprGlobal* expr = node->as()) - { - Global* g = globals.find(expr->name); - - return !g || !g->written ? Builtin{AstName(), expr->name} : Builtin(); - } - else - { - return Builtin(); - } - } - - int getBuiltinFunctionId(const Builtin& builtin) - { - if (builtin.empty()) - return -1; - - if (builtin.isGlobal("assert")) - return LBF_ASSERT; - - if (builtin.isGlobal("type")) - return LBF_TYPE; - - if (builtin.isGlobal("typeof")) - return LBF_TYPEOF; - - if (builtin.isGlobal("rawset")) - return LBF_RAWSET; - if (builtin.isGlobal("rawget")) - return LBF_RAWGET; - if (builtin.isGlobal("rawequal")) - return LBF_RAWEQUAL; - - if (builtin.isGlobal("unpack")) - return LBF_TABLE_UNPACK; - - if (builtin.object == "math") - { - if (builtin.method == "abs") - return LBF_MATH_ABS; - if (builtin.method == "acos") - return LBF_MATH_ACOS; - if (builtin.method == "asin") - return LBF_MATH_ASIN; - if (builtin.method == "atan2") - return LBF_MATH_ATAN2; - if (builtin.method == "atan") - return LBF_MATH_ATAN; - if (builtin.method == "ceil") - return LBF_MATH_CEIL; - if (builtin.method == "cosh") - return LBF_MATH_COSH; - if (builtin.method == "cos") - return LBF_MATH_COS; - if (builtin.method == "deg") - return LBF_MATH_DEG; - if (builtin.method == "exp") - return LBF_MATH_EXP; - if (builtin.method == "floor") - return LBF_MATH_FLOOR; - if (builtin.method == "fmod") - return LBF_MATH_FMOD; - if (builtin.method == "frexp") - return LBF_MATH_FREXP; - if (builtin.method == "ldexp") - return LBF_MATH_LDEXP; - if (builtin.method == "log10") - return LBF_MATH_LOG10; - if (builtin.method == "log") - return LBF_MATH_LOG; - if (builtin.method == "max") - return LBF_MATH_MAX; - if (builtin.method == "min") - return LBF_MATH_MIN; - if (builtin.method == "modf") - return LBF_MATH_MODF; - if (builtin.method == "pow") - return LBF_MATH_POW; - if (builtin.method == "rad") - return LBF_MATH_RAD; - if (builtin.method == "sinh") - return LBF_MATH_SINH; - if (builtin.method == "sin") - return LBF_MATH_SIN; - if (builtin.method == "sqrt") - return LBF_MATH_SQRT; - if (builtin.method == "tanh") - return LBF_MATH_TANH; - if (builtin.method == "tan") - return LBF_MATH_TAN; - if (builtin.method == "clamp") - return LBF_MATH_CLAMP; - if (builtin.method == "sign") - return LBF_MATH_SIGN; - if (builtin.method == "round") - return LBF_MATH_ROUND; - } - - if (builtin.object == "bit32") - { - if (builtin.method == "arshift") - return LBF_BIT32_ARSHIFT; - if (builtin.method == "band") - return LBF_BIT32_BAND; - if (builtin.method == "bnot") - return LBF_BIT32_BNOT; - if (builtin.method == "bor") - return LBF_BIT32_BOR; - if (builtin.method == "bxor") - return LBF_BIT32_BXOR; - if (builtin.method == "btest") - return LBF_BIT32_BTEST; - if (builtin.method == "extract") - return LBF_BIT32_EXTRACT; - if (builtin.method == "lrotate") - return LBF_BIT32_LROTATE; - if (builtin.method == "lshift") - return LBF_BIT32_LSHIFT; - if (builtin.method == "replace") - return LBF_BIT32_REPLACE; - if (builtin.method == "rrotate") - return LBF_BIT32_RROTATE; - if (builtin.method == "rshift") - return LBF_BIT32_RSHIFT; - if (builtin.method == "countlz") - return LBF_BIT32_COUNTLZ; - if (builtin.method == "countrz") - return LBF_BIT32_COUNTRZ; - } - - if (builtin.object == "string") - { - if (builtin.method == "byte") - return LBF_STRING_BYTE; - if (builtin.method == "char") - return LBF_STRING_CHAR; - if (builtin.method == "len") - return LBF_STRING_LEN; - if (builtin.method == "sub") - return LBF_STRING_SUB; - } - - if (builtin.object == "table") - { - if (builtin.method == "insert") - return LBF_TABLE_INSERT; - if (builtin.method == "unpack") - return LBF_TABLE_UNPACK; - } - - if (options.vectorCtor) - { - if (options.vectorLib) - { - if (builtin.isMethod(options.vectorLib, options.vectorCtor)) - return LBF_VECTOR; - } - else - { - if (builtin.isGlobal(options.vectorCtor)) - return LBF_VECTOR; - } - } - - return -1; - } - BytecodeBuilder& bytecode; CompileOptions options; @@ -3677,8 +2911,9 @@ struct Compiler DenseHashMap functions; DenseHashMap locals; DenseHashMap globals; + DenseHashMap variables; DenseHashMap constants; - DenseHashMap> predictedTableSize; + DenseHashMap tableShapes; unsigned int regTop = 0; unsigned int stackSize = 0; @@ -3699,23 +2934,18 @@ void compileOrThrow(BytecodeBuilder& bytecode, AstStatBlock* root, const AstName Compiler compiler(bytecode, options); // since access to some global objects may result in values that change over time, we block imports from non-readonly tables - if (AstName name = names.get("_G"); name.value) - compiler.globals[name].writable = true; + assignMutable(compiler.globals, names, options.mutableGlobals); - if (options.mutableGlobals) - for (const char** ptr = options.mutableGlobals; *ptr; ++ptr) - if (AstName name = names.get(*ptr); name.value) - compiler.globals[name].writable = true; + // this pass analyzes mutability of locals/globals and associates locals with their initial values + trackValues(compiler.globals, compiler.variables, root); - // this visitor traverses the AST to analyze mutability of locals/globals, filling Local::written and Global::written - Compiler::AssignmentVisitor assignmentVisitor(&compiler); - root->visit(&assignmentVisitor); - - // this visitor traverses the AST to analyze constantness of expressions, filling constants[] and Local::constant/Local::init if (options.optimizationLevel >= 1) { - Compiler::ConstantVisitor constantVisitor(&compiler); - root->visit(&constantVisitor); + // this pass analyzes constantness of expressions + foldConstants(compiler.constants, compiler.variables, root); + + // this pass analyzes table assignments to estimate table shapes for initially empty tables + predictTableShapes(compiler.tableShapes, root); } // this visitor tracks calls to getfenv/setfenv and disables some optimizations when they are found @@ -3734,8 +2964,8 @@ void compileOrThrow(BytecodeBuilder& bytecode, AstStatBlock* root, const AstName for (AstExprFunction* expr : functions) compiler.compileFunction(expr); - AstExprFunction main(root->location, /*generics= */ AstArray(), /*genericPacks= */ AstArray(), /* self= */ nullptr, - AstArray(), /* vararg= */ Luau::Location(), root, /* functionDepth= */ 0, /* debugname= */ AstName()); + AstExprFunction main(root->location, /*generics= */ AstArray(), /*genericPacks= */ AstArray(), + /* self= */ nullptr, AstArray(), /* vararg= */ Luau::Location(), root, /* functionDepth= */ 0, /* debugname= */ AstName()); uint32_t mainid = compiler.compileFunction(&main); bytecode.setMainFunction(mainid); diff --git a/Compiler/src/ConstantFolding.cpp b/Compiler/src/ConstantFolding.cpp new file mode 100644 index 00000000..60a7c169 --- /dev/null +++ b/Compiler/src/ConstantFolding.cpp @@ -0,0 +1,394 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +#include "ConstantFolding.h" + +#include + +namespace Luau +{ +namespace Compile +{ + +static bool constantsEqual(const Constant& la, const Constant& ra) +{ + LUAU_ASSERT(la.type != Constant::Type_Unknown && ra.type != Constant::Type_Unknown); + + switch (la.type) + { + case Constant::Type_Nil: + return ra.type == Constant::Type_Nil; + + case Constant::Type_Boolean: + return ra.type == Constant::Type_Boolean && la.valueBoolean == ra.valueBoolean; + + case Constant::Type_Number: + return ra.type == Constant::Type_Number && la.valueNumber == ra.valueNumber; + + case Constant::Type_String: + return ra.type == Constant::Type_String && la.stringLength == ra.stringLength && memcmp(la.valueString, ra.valueString, la.stringLength) == 0; + + default: + LUAU_ASSERT(!"Unexpected constant type in comparison"); + return false; + } +} + +static void foldUnary(Constant& result, AstExprUnary::Op op, const Constant& arg) +{ + switch (op) + { + case AstExprUnary::Not: + if (arg.type != Constant::Type_Unknown) + { + result.type = Constant::Type_Boolean; + result.valueBoolean = !arg.isTruthful(); + } + break; + + case AstExprUnary::Minus: + if (arg.type == Constant::Type_Number) + { + result.type = Constant::Type_Number; + result.valueNumber = -arg.valueNumber; + } + break; + + case AstExprUnary::Len: + if (arg.type == Constant::Type_String) + { + result.type = Constant::Type_Number; + result.valueNumber = double(arg.stringLength); + } + break; + + default: + LUAU_ASSERT(!"Unexpected unary operation"); + } +} + +static void foldBinary(Constant& result, AstExprBinary::Op op, const Constant& la, const Constant& ra) +{ + switch (op) + { + case AstExprBinary::Add: + if (la.type == Constant::Type_Number && ra.type == Constant::Type_Number) + { + result.type = Constant::Type_Number; + result.valueNumber = la.valueNumber + ra.valueNumber; + } + break; + + case AstExprBinary::Sub: + if (la.type == Constant::Type_Number && ra.type == Constant::Type_Number) + { + result.type = Constant::Type_Number; + result.valueNumber = la.valueNumber - ra.valueNumber; + } + break; + + case AstExprBinary::Mul: + if (la.type == Constant::Type_Number && ra.type == Constant::Type_Number) + { + result.type = Constant::Type_Number; + result.valueNumber = la.valueNumber * ra.valueNumber; + } + break; + + case AstExprBinary::Div: + if (la.type == Constant::Type_Number && ra.type == Constant::Type_Number) + { + result.type = Constant::Type_Number; + result.valueNumber = la.valueNumber / ra.valueNumber; + } + break; + + case AstExprBinary::Mod: + if (la.type == Constant::Type_Number && ra.type == Constant::Type_Number) + { + result.type = Constant::Type_Number; + result.valueNumber = la.valueNumber - floor(la.valueNumber / ra.valueNumber) * ra.valueNumber; + } + break; + + case AstExprBinary::Pow: + if (la.type == Constant::Type_Number && ra.type == Constant::Type_Number) + { + result.type = Constant::Type_Number; + result.valueNumber = pow(la.valueNumber, ra.valueNumber); + } + break; + + case AstExprBinary::Concat: + break; + + case AstExprBinary::CompareNe: + if (la.type != Constant::Type_Unknown && ra.type != Constant::Type_Unknown) + { + result.type = Constant::Type_Boolean; + result.valueBoolean = !constantsEqual(la, ra); + } + break; + + case AstExprBinary::CompareEq: + if (la.type != Constant::Type_Unknown && ra.type != Constant::Type_Unknown) + { + result.type = Constant::Type_Boolean; + result.valueBoolean = constantsEqual(la, ra); + } + break; + + case AstExprBinary::CompareLt: + if (la.type == Constant::Type_Number && ra.type == Constant::Type_Number) + { + result.type = Constant::Type_Boolean; + result.valueBoolean = la.valueNumber < ra.valueNumber; + } + break; + + case AstExprBinary::CompareLe: + if (la.type == Constant::Type_Number && ra.type == Constant::Type_Number) + { + result.type = Constant::Type_Boolean; + result.valueBoolean = la.valueNumber <= ra.valueNumber; + } + break; + + case AstExprBinary::CompareGt: + if (la.type == Constant::Type_Number && ra.type == Constant::Type_Number) + { + result.type = Constant::Type_Boolean; + result.valueBoolean = la.valueNumber > ra.valueNumber; + } + break; + + case AstExprBinary::CompareGe: + if (la.type == Constant::Type_Number && ra.type == Constant::Type_Number) + { + result.type = Constant::Type_Boolean; + result.valueBoolean = la.valueNumber >= ra.valueNumber; + } + break; + + case AstExprBinary::And: + if (la.type != Constant::Type_Unknown) + { + result = la.isTruthful() ? ra : la; + } + break; + + case AstExprBinary::Or: + if (la.type != Constant::Type_Unknown) + { + result = la.isTruthful() ? la : ra; + } + break; + + default: + LUAU_ASSERT(!"Unexpected binary operation"); + } +} + +struct ConstantVisitor : AstVisitor +{ + DenseHashMap& constants; + DenseHashMap& variables; + + DenseHashMap locals; + + ConstantVisitor(DenseHashMap& constants, DenseHashMap& variables) + : constants(constants) + , variables(variables) + , locals(nullptr) + { + } + + Constant analyze(AstExpr* node) + { + Constant result; + result.type = Constant::Type_Unknown; + + if (AstExprGroup* expr = node->as()) + { + result = analyze(expr->expr); + } + else if (node->is()) + { + result.type = Constant::Type_Nil; + } + else if (AstExprConstantBool* expr = node->as()) + { + result.type = Constant::Type_Boolean; + result.valueBoolean = expr->value; + } + else if (AstExprConstantNumber* expr = node->as()) + { + result.type = Constant::Type_Number; + result.valueNumber = expr->value; + } + else if (AstExprConstantString* expr = node->as()) + { + result.type = Constant::Type_String; + result.valueString = expr->value.data; + result.stringLength = unsigned(expr->value.size); + } + else if (AstExprLocal* expr = node->as()) + { + const Constant* l = locals.find(expr->local); + + if (l) + result = *l; + } + else if (node->is()) + { + // nope + } + else if (node->is()) + { + // nope + } + else if (AstExprCall* expr = node->as()) + { + analyze(expr->func); + + for (size_t i = 0; i < expr->args.size; ++i) + analyze(expr->args.data[i]); + } + else if (AstExprIndexName* expr = node->as()) + { + analyze(expr->expr); + } + else if (AstExprIndexExpr* expr = node->as()) + { + analyze(expr->expr); + analyze(expr->index); + } + else if (AstExprFunction* expr = node->as()) + { + // this is necessary to propagate constant information in all child functions + expr->body->visit(this); + } + else if (AstExprTable* expr = node->as()) + { + for (size_t i = 0; i < expr->items.size; ++i) + { + const AstExprTable::Item& item = expr->items.data[i]; + + if (item.key) + analyze(item.key); + + analyze(item.value); + } + } + else if (AstExprUnary* expr = node->as()) + { + Constant arg = analyze(expr->expr); + + if (arg.type != Constant::Type_Unknown) + foldUnary(result, expr->op, arg); + } + else if (AstExprBinary* expr = node->as()) + { + Constant la = analyze(expr->left); + Constant ra = analyze(expr->right); + + if (la.type != Constant::Type_Unknown && ra.type != Constant::Type_Unknown) + foldBinary(result, expr->op, la, ra); + } + else if (AstExprTypeAssertion* expr = node->as()) + { + Constant arg = analyze(expr->expr); + + result = arg; + } + else if (AstExprIfElse* expr = node->as()) + { + Constant cond = analyze(expr->condition); + Constant trueExpr = analyze(expr->trueExpr); + Constant falseExpr = analyze(expr->falseExpr); + + if (cond.type != Constant::Type_Unknown) + result = cond.isTruthful() ? trueExpr : falseExpr; + } + else + { + LUAU_ASSERT(!"Unknown expression type"); + } + + if (result.type != Constant::Type_Unknown) + constants[node] = result; + + return result; + } + + bool visit(AstExpr* node) override + { + // note: we short-circuit the visitor traversal through any expression trees by returning false + // recursive traversal is happening inside analyze() which makes it easier to get the resulting value of the subexpression + analyze(node); + + return false; + } + + bool visit(AstStatLocal* node) override + { + // all values that align wrt indexing are simple - we just match them 1-1 + for (size_t i = 0; i < node->vars.size && i < node->values.size; ++i) + { + Constant arg = analyze(node->values.data[i]); + + if (arg.type != Constant::Type_Unknown) + { + // note: we rely on trackValues to have been run before us + Variable* v = variables.find(node->vars.data[i]); + LUAU_ASSERT(v); + + if (!v->written) + { + locals[node->vars.data[i]] = arg; + v->constant = true; + } + } + } + + if (node->vars.size > node->values.size) + { + // if we have trailing variables, then depending on whether the last value is capable of returning multiple values + // (aka call or varargs), we either don't know anything about these vars, or we know they're nil + AstExpr* last = node->values.size ? node->values.data[node->values.size - 1] : nullptr; + bool multRet = last && (last->is() || last->is()); + + if (!multRet) + { + for (size_t i = node->values.size; i < node->vars.size; ++i) + { + // note: we rely on trackValues to have been run before us + Variable* v = variables.find(node->vars.data[i]); + LUAU_ASSERT(v); + + if (!v->written) + { + locals[node->vars.data[i]].type = Constant::Type_Nil; + v->constant = true; + } + } + } + } + else + { + // we can have more values than variables; in this case we still need to analyze them to make sure we do constant propagation inside + // them + for (size_t i = node->vars.size; i < node->values.size; ++i) + analyze(node->values.data[i]); + } + + return false; + } +}; + +void foldConstants(DenseHashMap& constants, DenseHashMap& variables, AstNode* root) +{ + ConstantVisitor visitor{constants, variables}; + root->visit(&visitor); +} + +} // namespace Compile +} // namespace Luau diff --git a/Compiler/src/ConstantFolding.h b/Compiler/src/ConstantFolding.h new file mode 100644 index 00000000..c0e63539 --- /dev/null +++ b/Compiler/src/ConstantFolding.h @@ -0,0 +1,48 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +#pragma once + +#include "ValueTracking.h" + +namespace Luau +{ +namespace Compile +{ + +struct Constant +{ + enum Type + { + Type_Unknown, + Type_Nil, + Type_Boolean, + Type_Number, + Type_String, + }; + + Type type = Type_Unknown; + unsigned int stringLength = 0; + + union + { + bool valueBoolean; + double valueNumber; + char* valueString = nullptr; // length stored in stringLength + }; + + bool isTruthful() const + { + LUAU_ASSERT(type != Type_Unknown); + return type != Type_Nil && !(type == Type_Boolean && valueBoolean == false); + } + + AstArray getString() const + { + LUAU_ASSERT(type == Type_String); + return {valueString, stringLength}; + } +}; + +void foldConstants(DenseHashMap& constants, DenseHashMap& variables, AstNode* root); + +} // namespace Compile +} // namespace Luau diff --git a/Compiler/src/TableShape.cpp b/Compiler/src/TableShape.cpp new file mode 100644 index 00000000..7d99f222 --- /dev/null +++ b/Compiler/src/TableShape.cpp @@ -0,0 +1,129 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +#include "TableShape.h" + +namespace Luau +{ +namespace Compile +{ + +static AstExprTable* getTableHint(AstExpr* expr) +{ + // unadorned table literal + if (AstExprTable* table = expr->as()) + return table; + + // setmetatable(table literal, ...) + if (AstExprCall* call = expr->as(); call && !call->self && call->args.size == 2) + if (AstExprGlobal* func = call->func->as(); func && func->name == "setmetatable") + if (AstExprTable* table = call->args.data[0]->as()) + return table; + + return nullptr; +} + +struct ShapeVisitor : AstVisitor +{ + struct Hasher + { + size_t operator()(const std::pair& p) const + { + return std::hash()(p.first) ^ std::hash()(p.second); + } + }; + + DenseHashMap& shapes; + + DenseHashMap tables; + DenseHashSet, Hasher> fields; + + ShapeVisitor(DenseHashMap& shapes) + : shapes(shapes) + , tables(nullptr) + , fields(std::pair()) + { + } + + void assignField(AstExpr* expr, AstName index) + { + if (AstExprLocal* lv = expr->as()) + { + if (AstExprTable** table = tables.find(lv->local)) + { + std::pair field = {*table, index}; + + if (!fields.contains(field)) + { + fields.insert(field); + shapes[*table].hashSize += 1; + } + } + } + } + + void assignField(AstExpr* expr, AstExpr* index) + { + AstExprLocal* lv = expr->as(); + AstExprConstantNumber* number = index->as(); + + if (lv && number) + { + if (AstExprTable** table = tables.find(lv->local)) + { + TableShape& shape = shapes[*table]; + + if (number->value == double(shape.arraySize + 1)) + shape.arraySize += 1; + } + } + } + + void assign(AstExpr* var) + { + if (AstExprIndexName* index = var->as()) + { + assignField(index->expr, index->index); + } + else if (AstExprIndexExpr* index = var->as()) + { + assignField(index->expr, index->index); + } + } + + bool visit(AstStatLocal* node) override + { + // track local -> table association so that we can update table size prediction in assignField + if (node->vars.size == 1 && node->values.size == 1) + if (AstExprTable* table = getTableHint(node->values.data[0]); table && table->items.size == 0) + tables[node->vars.data[0]] = table; + + return true; + } + + bool visit(AstStatAssign* node) override + { + for (size_t i = 0; i < node->vars.size; ++i) + assign(node->vars.data[i]); + + for (size_t i = 0; i < node->values.size; ++i) + node->values.data[i]->visit(this); + + return false; + } + + bool visit(AstStatFunction* node) override + { + assign(node->name); + node->func->visit(this); + + return false; + } +}; + +void predictTableShapes(DenseHashMap& shapes, AstNode* root) +{ + ShapeVisitor visitor{shapes}; + root->visit(&visitor); +} + +} // namespace Compile +} // namespace Luau diff --git a/Compiler/src/TableShape.h b/Compiler/src/TableShape.h new file mode 100644 index 00000000..f30853a7 --- /dev/null +++ b/Compiler/src/TableShape.h @@ -0,0 +1,21 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +#pragma once + +#include "Luau/Ast.h" +#include "Luau/DenseHash.h" + +namespace Luau +{ +namespace Compile +{ + +struct TableShape +{ + unsigned int arraySize = 0; + unsigned int hashSize = 0; +}; + +void predictTableShapes(DenseHashMap& shapes, AstNode* root); + +} // namespace Compile +} // namespace Luau diff --git a/Compiler/src/ValueTracking.cpp b/Compiler/src/ValueTracking.cpp new file mode 100644 index 00000000..0bfaf9b3 --- /dev/null +++ b/Compiler/src/ValueTracking.cpp @@ -0,0 +1,103 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +#include "ValueTracking.h" + +#include "Luau/Lexer.h" + +namespace Luau +{ +namespace Compile +{ + +struct ValueVisitor : AstVisitor +{ + DenseHashMap& globals; + DenseHashMap& variables; + + ValueVisitor(DenseHashMap& globals, DenseHashMap& variables) + : globals(globals) + , variables(variables) + { + } + + void assign(AstExpr* var) + { + if (AstExprLocal* lv = var->as()) + { + variables[lv->local].written = true; + } + else if (AstExprGlobal* gv = var->as()) + { + globals[gv->name] = Global::Written; + } + else + { + // we need to be able to track assignments in all expressions, including crazy ones like t[function() t = nil end] = 5 + var->visit(this); + } + } + + bool visit(AstStatLocal* node) override + { + for (size_t i = 0; i < node->vars.size && i < node->values.size; ++i) + variables[node->vars.data[i]].init = node->values.data[i]; + + for (size_t i = node->values.size; i < node->vars.size; ++i) + variables[node->vars.data[i]].init = nullptr; + + return true; + } + + bool visit(AstStatAssign* node) override + { + for (size_t i = 0; i < node->vars.size; ++i) + assign(node->vars.data[i]); + + for (size_t i = 0; i < node->values.size; ++i) + node->values.data[i]->visit(this); + + return false; + } + + bool visit(AstStatCompoundAssign* node) override + { + assign(node->var); + node->value->visit(this); + + return false; + } + + bool visit(AstStatLocalFunction* node) override + { + variables[node->name].init = node->func; + + return true; + } + + bool visit(AstStatFunction* node) override + { + assign(node->name); + node->func->visit(this); + + return false; + } +}; + +void assignMutable(DenseHashMap& globals, const AstNameTable& names, const char** mutableGlobals) +{ + if (AstName name = names.get("_G"); name.value) + globals[name] = Global::Mutable; + + if (mutableGlobals) + for (const char** ptr = mutableGlobals; *ptr; ++ptr) + if (AstName name = names.get(*ptr); name.value) + globals[name] = Global::Mutable; +} + +void trackValues(DenseHashMap& globals, DenseHashMap& variables, AstNode* root) +{ + ValueVisitor visitor{globals, variables}; + root->visit(&visitor); +} + +} // namespace Compile +} // namespace Luau diff --git a/Compiler/src/ValueTracking.h b/Compiler/src/ValueTracking.h new file mode 100644 index 00000000..fc74c84a --- /dev/null +++ b/Compiler/src/ValueTracking.h @@ -0,0 +1,42 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +#pragma once + +#include "Luau/Ast.h" +#include "Luau/DenseHash.h" + +namespace Luau +{ +class AstNameTable; +} + +namespace Luau +{ +namespace Compile +{ + +enum class Global +{ + Default = 0, + Mutable, // builtin that has contents unknown at compile time, blocks GETIMPORT for chains + Written, // written in the code which means we can't reason about the value +}; + +struct Variable +{ + AstExpr* init = nullptr; // initial value of the variable; filled by trackValues + bool written = false; // is the variable ever assigned to? filled by trackValues + bool constant = false; // is the variable's value a compile-time constant? filled by constantFold +}; + +void assignMutable(DenseHashMap& globals, const AstNameTable& names, const char** mutableGlobals); +void trackValues(DenseHashMap& globals, DenseHashMap& variables, AstNode* root); + +inline Global getGlobalState(const DenseHashMap& globals, AstName name) +{ + const Global* it = globals.find(name); + + return it ? *it : Global::Default; +} + +} // namespace Compile +} // namespace Luau diff --git a/Sources.cmake b/Sources.cmake index 5dd486aa..bafe7594 100644 --- a/Sources.cmake +++ b/Sources.cmake @@ -29,7 +29,15 @@ target_sources(Luau.Compiler PRIVATE Compiler/src/BytecodeBuilder.cpp Compiler/src/Compiler.cpp + Compiler/src/Builtins.cpp + Compiler/src/ConstantFolding.cpp + Compiler/src/TableShape.cpp + Compiler/src/ValueTracking.cpp Compiler/src/lcode.cpp + Compiler/src/Builtins.h + Compiler/src/ConstantFolding.h + Compiler/src/TableShape.h + Compiler/src/ValueTracking.h ) # Luau.Analysis Sources diff --git a/VM/src/ldo.cpp b/VM/src/ldo.cpp index eb47971a..3cce7665 100644 --- a/VM/src/ldo.cpp +++ b/VM/src/ldo.cpp @@ -17,7 +17,6 @@ #include -LUAU_FASTFLAGVARIABLE(LuauCcallRestoreFix, false) LUAU_FASTFLAG(LuauCoroutineClose) /* @@ -545,11 +544,8 @@ int luaD_pcall(lua_State* L, Pfunc func, void* u, ptrdiff_t old_top, ptrdiff_t e if (!oldactive) resetbit(L->stackstate, THREAD_ACTIVEBIT); - if (FFlag::LuauCcallRestoreFix) - { - // Restore nCcalls before calling the debugprotectederror callback which may rely on the proper value to have been restored. - L->nCcalls = oldnCcalls; - } + // Restore nCcalls before calling the debugprotectederror callback which may rely on the proper value to have been restored. + L->nCcalls = oldnCcalls; // an error occurred, check if we have a protected error callback if (L->global->cb.debugprotectederror) @@ -564,10 +560,6 @@ int luaD_pcall(lua_State* L, Pfunc func, void* u, ptrdiff_t old_top, ptrdiff_t e StkId oldtop = restorestack(L, old_top); luaF_close(L, oldtop); /* close eventual pending closures */ seterrorobj(L, status, oldtop); - if (!FFlag::LuauCcallRestoreFix) - { - L->nCcalls = oldnCcalls; - } L->ci = restoreci(L, old_ci); L->base = L->ci->base; restore_stack_limit(L); diff --git a/VM/src/lfunc.cpp b/VM/src/lfunc.cpp index 64878569..4178eda4 100644 --- a/VM/src/lfunc.cpp +++ b/VM/src/lfunc.cpp @@ -6,6 +6,8 @@ #include "lmem.h" #include "lgc.h" +LUAU_FASTFLAGVARIABLE(LuauNoDirectUpvalRemoval, false) + Proto* luaF_newproto(lua_State* L) { Proto* f = luaM_new(L, Proto, sizeof(Proto), L->activememcat); @@ -113,14 +115,16 @@ void luaF_freeupval(lua_State* L, UpVal* uv) void luaF_close(lua_State* L, StkId level) { UpVal* uv; - global_State* g = L->global; + global_State* g = L->global; // TODO: remove with FFlagLuauNoDirectUpvalRemoval while (L->openupval != NULL && (uv = gco2uv(L->openupval))->v >= level) { GCObject* o = obj2gco(uv); LUAU_ASSERT(!isblack(o) && uv->v != &uv->u.value); L->openupval = uv->next; /* remove from `open' list */ - if (isdead(g, o)) + if (!FFlag::LuauNoDirectUpvalRemoval && isdead(g, o)) + { luaF_freeupval(L, uv); /* free upvalue */ + } else { unlinkupval(uv); diff --git a/VM/src/lstrlib.cpp b/VM/src/lstrlib.cpp index 0b3054ae..74a8aa8a 100644 --- a/VM/src/lstrlib.cpp +++ b/VM/src/lstrlib.cpp @@ -8,8 +8,6 @@ #include #include -LUAU_FASTFLAGVARIABLE(LuauStrPackUBCastFix, false) - /* macro to `unsign' a character */ #define uchar(c) ((unsigned char)(c)) @@ -1406,20 +1404,10 @@ static int str_pack(lua_State* L) } case Kuint: { /* unsigned integers */ - if (FFlag::LuauStrPackUBCastFix) - { - long long n = (long long)luaL_checknumber(L, arg); - if (size < SZINT) /* need overflow check? */ - luaL_argcheck(L, (unsigned long long)n < ((unsigned long long)1 << (size * NB)), arg, "unsigned overflow"); - packint(&b, (unsigned long long)n, h.islittle, size, 0); - } - else - { - unsigned long long n = (unsigned long long)luaL_checknumber(L, arg); - if (size < SZINT) /* need overflow check? */ - luaL_argcheck(L, n < ((unsigned long long)1 << (size * NB)), arg, "unsigned overflow"); - packint(&b, n, h.islittle, size, 0); - } + long long n = (long long)luaL_checknumber(L, arg); + if (size < SZINT) /* need overflow check? */ + luaL_argcheck(L, (unsigned long long)n < ((unsigned long long)1 << (size * NB)), arg, "unsigned overflow"); + packint(&b, (unsigned long long)n, h.islittle, size, 0); break; } case Kfloat: diff --git a/bench/tests/sunspider/3d-cube.lua b/bench/tests/sunspider/3d-cube.lua index 6d40406c..5d162ab9 100644 --- a/bench/tests/sunspider/3d-cube.lua +++ b/bench/tests/sunspider/3d-cube.lua @@ -111,15 +111,10 @@ end -- multiplies two matrices function MMulti(M1, M2) local M = {{},{},{},{}}; - local i = 1; - local j = 1; - while i <= 4 do - j = 1; - while j <= 4 do - M[i][j] = M1[i][1] * M2[1][j] + M1[i][2] * M2[2][j] + M1[i][3] * M2[3][j] + M1[i][4] * M2[4][j]; j = j + 1 + for i = 1,4 do + for j = 1,4 do + M[i][j] = M1[i][1] * M2[1][j] + M1[i][2] * M2[2][j] + M1[i][3] * M2[3][j] + M1[i][4] * M2[4][j]; end - - i = i + 1 end return M; end @@ -127,28 +122,27 @@ end -- multiplies matrix with vector function VMulti(M, V) local Vect = {}; - local i = 1; - while i <= 4 do Vect[i] = M[i][1] * V[1] + M[i][2] * V[2] + M[i][3] * V[3] + M[i][4] * V[4]; i = i + 1 end + for i = 1,4 do + Vect[i] = M[i][1] * V[1] + M[i][2] * V[2] + M[i][3] * V[3] + M[i][4] * V[4]; + end return Vect; end function VMulti2(M, V) local Vect = {}; - local i = 1; - while i < 4 do Vect[i] = M[i][1] * V[1] + M[i][2] * V[2] + M[i][3] * V[3]; i = i + 1 end + for i = 1,3 do + Vect[i] = M[i][1] * V[1] + M[i][2] * V[2] + M[i][3] * V[3]; + end return Vect; end -- add to matrices function MAdd(M1, M2) local M = {{},{},{},{}}; - local i = 1; - local j = 1; - while i <= 4 do - j = 1; - while j <= 4 do M[i][j] = M1[i][j] + M2[i][j]; j = j + 1 end - - i = i + 1 + for i = 1,4 do + for j = 1,4 do + M[i][j] = M1[i][j] + M2[i][j]; + end end return M; end diff --git a/tests/Autocomplete.test.cpp b/tests/Autocomplete.test.cpp index 210db7ee..211e1be1 100644 --- a/tests/Autocomplete.test.cpp +++ b/tests/Autocomplete.test.cpp @@ -1938,7 +1938,6 @@ return target(b@1 TEST_CASE_FIXTURE(ACFixture, "function_in_assignment_has_parentheses") { ScopedFastFlag luauAutocompleteAvoidMutation("LuauAutocompleteAvoidMutation", true); - ScopedFastFlag luauAutocompletePreferToCallFunctions("LuauAutocompletePreferToCallFunctions", true); check(R"( local function bar(a: number) return -a end @@ -1954,7 +1953,6 @@ local abc = b@1 TEST_CASE_FIXTURE(ACFixture, "function_result_passed_to_function_has_parentheses") { ScopedFastFlag luauAutocompleteAvoidMutation("LuauAutocompleteAvoidMutation", true); - ScopedFastFlag luauAutocompletePreferToCallFunctions("LuauAutocompletePreferToCallFunctions", true); check(R"( local function foo() return 1 end @@ -2538,10 +2536,6 @@ TEST_CASE("autocomplete_documentation_symbols") TEST_CASE_FIXTURE(ACFixture, "autocomplete_ifelse_expressions") { - ScopedFastFlag sff1{"LuauIfElseExpressionBaseSupport", true}; - ScopedFastFlag sff2{"LuauIfElseExpressionAnalysisSupport", true}; - - { check(R"( local temp = false local even = true; @@ -2614,7 +2608,6 @@ a = if temp then even elseif true then temp else e@9 CHECK(ac.entryMap.count("then") == 0); CHECK(ac.entryMap.count("else") == 0); CHECK(ac.entryMap.count("elseif") == 0); - } } TEST_CASE_FIXTURE(ACFixture, "autocomplete_explicit_type_pack") @@ -2681,4 +2674,58 @@ local r4 = t:bar1(@4) CHECK(ac.entryMap["foo2"].typeCorrect == TypeCorrectKind::None); } +TEST_CASE_FIXTURE(ACFixture, "autocomplete_default_type_parameters") +{ + ScopedFastFlag luauParseTypeAliasDefaults{"LuauParseTypeAliasDefaults", true}; + + check(R"( +type A = () -> T + )"); + + auto ac = autocomplete('1'); + + CHECK(ac.entryMap.count("number")); + CHECK(ac.entryMap.count("string")); +} + +TEST_CASE_FIXTURE(ACFixture, "autocomplete_default_type_pack_parameters") +{ + ScopedFastFlag luauParseTypeAliasDefaults{"LuauParseTypeAliasDefaults", true}; + + check(R"( +type A = () -> T + )"); + + auto ac = autocomplete('1'); + + CHECK(ac.entryMap.count("number")); + CHECK(ac.entryMap.count("string")); +} + +TEST_CASE_FIXTURE(ACFixture, "autocomplete_oop_implicit_self") +{ + ScopedFastFlag flag("LuauMissingFollowACMetatables", true); + check(R"( +--!strict +local Class = {} +Class.__index = Class +type Class = typeof(setmetatable({} :: { x: number }, Class)) +function Class.new(x: number): Class + return setmetatable({x = x}, Class) +end +function Class.getx(self: Class) + return self.x +end +function test() + local c = Class.new(42) + local n = c:@1 + print(n) +end + )"); + + auto ac = autocomplete('1'); + + CHECK(ac.entryMap.count("getx")); +} + TEST_SUITE_END(); diff --git a/tests/Compiler.test.cpp b/tests/Compiler.test.cpp index 95811b3f..8eed953f 100644 --- a/tests/Compiler.test.cpp +++ b/tests/Compiler.test.cpp @@ -603,9 +603,9 @@ RETURN R0 1 )"); } -TEST_CASE("EmptyTableHashSizePredictionOptimization") +TEST_CASE("TableSizePredictionBasic") { - const char* hashSizeSource = R"( + CHECK_EQ("\n" + compileFunction0(R"( local t = {} t.a = 1 t.b = 1 @@ -616,36 +616,8 @@ t.f = 1 t.g = 1 t.h = 1 t.i = 1 -)"; - - const char* hashSizeSource2 = R"( -local t = {} -t.x = 1 -t.x = 2 -t.x = 3 -t.x = 4 -t.x = 5 -t.x = 6 -t.x = 7 -t.x = 8 -t.x = 9 -)"; - - const char* arraySizeSource = R"( -local t = {} -t[1] = 1 -t[2] = 1 -t[3] = 1 -t[4] = 1 -t[5] = 1 -t[6] = 1 -t[7] = 1 -t[8] = 1 -t[9] = 1 -t[10] = 1 -)"; - - CHECK_EQ("\n" + compileFunction0(hashSizeSource), R"( +)"), + R"( NEWTABLE R0 16 0 LOADN R1 1 SETTABLEKS R1 R0 K0 @@ -668,7 +640,19 @@ SETTABLEKS R1 R0 K8 RETURN R0 0 )"); - CHECK_EQ("\n" + compileFunction0(hashSizeSource2), R"( + CHECK_EQ("\n" + compileFunction0(R"( +local t = {} +t.x = 1 +t.x = 2 +t.x = 3 +t.x = 4 +t.x = 5 +t.x = 6 +t.x = 7 +t.x = 8 +t.x = 9 +)"), + R"( NEWTABLE R0 1 0 LOADN R1 1 SETTABLEKS R1 R0 K0 @@ -691,7 +675,20 @@ SETTABLEKS R1 R0 K0 RETURN R0 0 )"); - CHECK_EQ("\n" + compileFunction0(arraySizeSource), R"( + CHECK_EQ("\n" + compileFunction0(R"( +local t = {} +t[1] = 1 +t[2] = 1 +t[3] = 1 +t[4] = 1 +t[5] = 1 +t[6] = 1 +t[7] = 1 +t[8] = 1 +t[9] = 1 +t[10] = 1 +)"), + R"( NEWTABLE R0 0 10 LOADN R1 1 SETTABLEN R1 R0 1 @@ -717,6 +714,27 @@ RETURN R0 0 )"); } +TEST_CASE("TableSizePredictionObject") +{ + CHECK_EQ("\n" + compileFunction(R"( +local t = {} +t.field = 1 +function t:getfield() + return self.field +end +return t +)", + 1), + R"( +NEWTABLE R0 2 0 +LOADN R1 1 +SETTABLEKS R1 R0 K0 +DUPCLOSURE R1 K1 +SETTABLEKS R1 R0 K2 +RETURN R0 1 +)"); +} + TEST_CASE("TableSizePredictionSetMetatable") { CHECK_EQ("\n" + compileFunction0(R"( @@ -1031,9 +1049,6 @@ RETURN R0 1 TEST_CASE("IfElseExpression") { - ScopedFastFlag sff1{"LuauIfElseExpressionBaseSupport", true}; - ScopedFastFlag sff2{"LuauIfElseExpressionAnalysisSupport", true}; - // codegen for a true constant condition CHECK_EQ("\n" + compileFunction0("return if true then 10 else 20"), R"( LOADN R0 10 @@ -3058,7 +3073,7 @@ RETURN R0 0 // table variants (indexed by string, number, variable) CHECK_EQ("\n" + compileFunction0("local a = {} a.foo += 5"), R"( -NEWTABLE R0 1 0 +NEWTABLE R0 0 0 GETTABLEKS R1 R0 K0 ADDK R1 R1 K1 SETTABLEKS R1 R0 K0 @@ -3066,7 +3081,7 @@ RETURN R0 0 )"); CHECK_EQ("\n" + compileFunction0("local a = {} a[1] += 5"), R"( -NEWTABLE R0 0 1 +NEWTABLE R0 0 0 GETTABLEN R1 R0 1 ADDK R1 R1 K0 SETTABLEN R1 R0 1 diff --git a/tests/Conformance.test.cpp b/tests/Conformance.test.cpp index 663b329e..5222af33 100644 --- a/tests/Conformance.test.cpp +++ b/tests/Conformance.test.cpp @@ -366,15 +366,11 @@ TEST_CASE("PCall") TEST_CASE("Pack") { - ScopedFastFlag sff{"LuauStrPackUBCastFix", true}; - runConformance("tpack.lua"); } TEST_CASE("Vector") { - ScopedFastFlag sff{"LuauIfElseExpressionBaseSupport", true}; - lua_CompileOptions copts = {}; copts.optimizationLevel = 1; copts.debugLevel = 1; @@ -861,15 +857,11 @@ TEST_CASE("ExceptionObject") TEST_CASE("IfElseExpression") { - ScopedFastFlag sff{"LuauIfElseExpressionBaseSupport", true}; - runConformance("ifelseexpr.lua"); } TEST_CASE("TagMethodError") { - ScopedFastFlag sff{"LuauCcallRestoreFix", true}; - runConformance("tmerror.lua", [](lua_State* L) { auto* cb = lua_callbacks(L); diff --git a/tests/Fixture.cpp b/tests/Fixture.cpp index ca4281a0..c74bfa27 100644 --- a/tests/Fixture.cpp +++ b/tests/Fixture.cpp @@ -191,7 +191,7 @@ ParseResult Fixture::tryParse(const std::string& source, const ParseOptions& par return result; } -ParseResult Fixture::matchParseError(const std::string& source, const std::string& message) +ParseResult Fixture::matchParseError(const std::string& source, const std::string& message, std::optional location) { ParseOptions options; options.allowDeclarationSyntax = true; @@ -203,6 +203,9 @@ ParseResult Fixture::matchParseError(const std::string& source, const std::strin CHECK_EQ(result.errors.front().getMessage(), message); + if (location) + CHECK_EQ(result.errors.front().getLocation(), *location); + return result; } diff --git a/tests/Fixture.h b/tests/Fixture.h index e01632ea..ab852ef6 100644 --- a/tests/Fixture.h +++ b/tests/Fixture.h @@ -106,7 +106,7 @@ struct Fixture /// Parse with all language extensions enabled ParseResult parseEx(const std::string& source, const ParseOptions& parseOptions = {}); ParseResult tryParse(const std::string& source, const ParseOptions& parseOptions = {}); - ParseResult matchParseError(const std::string& source, const std::string& message); + ParseResult matchParseError(const std::string& source, const std::string& message, std::optional location = std::nullopt); // Verify a parse error occurs and the parse error message has the specified prefix ParseResult matchParseErrorPrefix(const std::string& source, const std::string& prefix); diff --git a/tests/Parser.test.cpp b/tests/Parser.test.cpp index 5abcb09a..e9135651 100644 --- a/tests/Parser.test.cpp +++ b/tests/Parser.test.cpp @@ -1255,7 +1255,6 @@ TEST_CASE_FIXTURE(Fixture, "parse_error_with_too_many_nested_type_group") TEST_CASE_FIXTURE(Fixture, "parse_error_with_too_many_nested_if_statements") { ScopedFastInt sfis{"LuauRecursionLimit", 10}; - ScopedFastFlag sff{"LuauIfStatementRecursionGuard", true}; matchParseErrorPrefix( "function f() if true then if true then if true then if true then if true then if true then if true then if true then if true " @@ -1266,7 +1265,6 @@ TEST_CASE_FIXTURE(Fixture, "parse_error_with_too_many_nested_if_statements") TEST_CASE_FIXTURE(Fixture, "parse_error_with_too_many_changed_elseif_statements") { ScopedFastInt sfis{"LuauRecursionLimit", 10}; - ScopedFastFlag sff{"LuauIfStatementRecursionGuard", true}; matchParseErrorPrefix( "function f() if false then elseif false then elseif false then elseif false then elseif false then elseif false then elseif " @@ -1276,7 +1274,6 @@ TEST_CASE_FIXTURE(Fixture, "parse_error_with_too_many_changed_elseif_statements" TEST_CASE_FIXTURE(Fixture, "parse_error_with_too_many_nested_ifelse_expressions1") { - ScopedFastFlag sff{"LuauIfElseExpressionBaseSupport", true}; ScopedFastInt sfis{"LuauRecursionLimit", 10}; matchParseError("function f() return if true then 1 elseif true then 2 elseif true then 3 elseif true then 4 elseif true then 5 elseif true then " @@ -1286,7 +1283,6 @@ TEST_CASE_FIXTURE(Fixture, "parse_error_with_too_many_nested_ifelse_expressions1 TEST_CASE_FIXTURE(Fixture, "parse_error_with_too_many_nested_ifelse_expressions2") { - ScopedFastFlag sff{"LuauIfElseExpressionBaseSupport", true}; ScopedFastInt sfis{"LuauRecursionLimit", 10}; matchParseError( @@ -1962,6 +1958,37 @@ TEST_CASE_FIXTURE(Fixture, "function_type_named_arguments") matchParseError("type MyFunc = (number) -> (d: number) -> number", "Expected '->' when parsing function type, got '<'"); } +TEST_CASE_FIXTURE(Fixture, "function_type_matching_parenthesis") +{ + matchParseError("local a: (number -> string", "Expected ')' (to close '(' at column 13), got '->'"); +} + +TEST_CASE_FIXTURE(Fixture, "parse_type_alias_default_type") +{ + ScopedFastFlag luauParseTypeAliasDefaults{"LuauParseTypeAliasDefaults", true}; + + AstStat* stat = parse(R"( +type A = {} +type B = {} +type C = {} +type D = {} +type E = {} +type F = (T...) -> U... +type G = (U...) -> T... + )"); + + REQUIRE(stat != nullptr); +} + +TEST_CASE_FIXTURE(Fixture, "parse_type_alias_default_type_errors") +{ + ScopedFastFlag luauParseTypeAliasDefaults{"LuauParseTypeAliasDefaults", true}; + + matchParseError("type Y = {}", "Expected default type after type name", Location{{0, 20}, {0, 21}}); + matchParseError("type Y = {}", "Expected default type pack after type pack name", Location{{0, 29}, {0, 30}}); + matchParseError("type Y number> = {}", "Expected type pack after '=', got type", Location{{0, 14}, {0, 32}}); +} + TEST_SUITE_END(); TEST_SUITE_BEGIN("ParseErrorRecovery"); @@ -2455,10 +2482,19 @@ do end CHECK_EQ(1, result.errors.size()); } +TEST_CASE_FIXTURE(Fixture, "recover_expected_type_pack") +{ + ScopedFastFlag luauParseTypeAliasDefaults{"LuauParseTypeAliasDefaults", true}; + ScopedFastFlag luauParseRecoverTypePackEllipsis{"LuauParseRecoverTypePackEllipsis", true}; + + ParseResult result = tryParse(R"( +type Y = (T...) -> U... + )"); + CHECK_EQ(1, result.errors.size()); +} + TEST_CASE_FIXTURE(Fixture, "parse_if_else_expression") { - ScopedFastFlag sff{"LuauIfElseExpressionBaseSupport", true}; - { AstStat* stat = parse("return if true then 1 else 2"); @@ -2524,9 +2560,4 @@ type C = Packed<(number, X...)> REQUIRE(stat != nullptr); } -TEST_CASE_FIXTURE(Fixture, "function_type_matching_parenthesis") -{ - matchParseError("local a: (number -> string", "Expected ')' (to close '(' at column 13), got '->'"); -} - TEST_SUITE_END(); diff --git a/tests/ToString.test.cpp b/tests/ToString.test.cpp index 80a258f5..445ee532 100644 --- a/tests/ToString.test.cpp +++ b/tests/ToString.test.cpp @@ -338,6 +338,8 @@ TEST_CASE_FIXTURE(Fixture, "toStringDetailed") TEST_CASE_FIXTURE(Fixture, "toStringDetailed2") { + ScopedFastFlag sff{"LuauUnsealedTableLiteral", true}; + CheckResult result = check(R"( local base = {} function base:one() return 1 end @@ -353,7 +355,7 @@ TEST_CASE_FIXTURE(Fixture, "toStringDetailed2") TypeId tType = requireType("inst"); ToStringResult r = toStringDetailed(tType); - CHECK_EQ("{ @metatable {| __index: { @metatable {| __index: base |}, child } |}, inst }", r.name); + CHECK_EQ("{ @metatable { __index: { @metatable { __index: base }, child } }, inst }", r.name); CHECK_EQ(0, r.nameMap.typeVars.size()); ToStringOptions opts; @@ -500,6 +502,24 @@ TEST_CASE_FIXTURE(Fixture, "toStringNamedFunction_map") CHECK_EQ("map(arr: {a}, fn: (a) -> b): {b}", toStringNamedFunction("map", *ftv)); } +TEST_CASE_FIXTURE(Fixture, "toStringNamedFunction_generic_pack") +{ + ScopedFastFlag luauTypeAliasDefaults{"LuauTypeAliasDefaults", true}; + + CheckResult result = check(R"( + local function f(a: number, b: string) end + local function test(...: T...): U... + f(...) + return 1, 2, 3 + end + )"); + + TypeId ty = requireType("test"); + const FunctionTypeVar* ftv = get(follow(ty)); + + CHECK_EQ("test(...: T...): U...", toStringNamedFunction("test", *ftv)); +} + TEST_CASE("toStringNamedFunction_unit_f") { TypePackVar empty{TypePack{}}; diff --git a/tests/Transpiler.test.cpp b/tests/Transpiler.test.cpp index 47c3883c..ac5be859 100644 --- a/tests/Transpiler.test.cpp +++ b/tests/Transpiler.test.cpp @@ -421,8 +421,6 @@ TEST_CASE_FIXTURE(Fixture, "transpile_type_assertion") TEST_CASE_FIXTURE(Fixture, "transpile_if_then_else") { - ScopedFastFlag luauIfElseExpressionBaseSupport("LuauIfElseExpressionBaseSupport", true); - std::string code = "local a = if 1 then 2 else 3"; CHECK_EQ(code, transpile(code).code); @@ -641,4 +639,16 @@ TEST_CASE_FIXTURE(Fixture, "transpile_to_string") CHECK_EQ("'hello'", toString(expr)); } +TEST_CASE_FIXTURE(Fixture, "transpile_type_alias_default_type_parameters") +{ + ScopedFastFlag luauParseTypeAliasDefaults{"LuauParseTypeAliasDefaults", true}; + ScopedFastFlag luauTypeAliasDefaults{"LuauTypeAliasDefaults", true}; + + std::string code = R"( +type Packed = (T, U, V...)->(W...) +local a: Packed + )"; + + CHECK_EQ(code, transpile(code, {}, true).code); +} TEST_SUITE_END(); diff --git a/tests/TypeInfer.annotations.test.cpp b/tests/TypeInfer.annotations.test.cpp index 275782b3..86165814 100644 --- a/tests/TypeInfer.annotations.test.cpp +++ b/tests/TypeInfer.annotations.test.cpp @@ -497,7 +497,7 @@ TEST_CASE_FIXTURE(Fixture, "generic_aliases_are_cloned_properly") CHECK(arrayTable->indexer); CHECK(isInArena(array.type, mod.interfaceTypes)); - CHECK_EQ(array.typeParams[0], arrayTable->indexer->indexResultType); + CHECK_EQ(array.typeParams[0].ty, arrayTable->indexer->indexResultType); } TEST_CASE_FIXTURE(Fixture, "cloned_interface_maintains_pointers_between_definitions") diff --git a/tests/TypeInfer.refinements.test.cpp b/tests/TypeInfer.refinements.test.cpp index 503b613f..d76b920b 100644 --- a/tests/TypeInfer.refinements.test.cpp +++ b/tests/TypeInfer.refinements.test.cpp @@ -1031,9 +1031,6 @@ TEST_CASE_FIXTURE(Fixture, "refine_the_correct_types_opposite_of_when_a_is_not_n TEST_CASE_FIXTURE(Fixture, "is_truthy_constraint_ifelse_expression") { - ScopedFastFlag sff1{"LuauIfElseExpressionBaseSupport", true}; - ScopedFastFlag sff2{"LuauIfElseExpressionAnalysisSupport", true}; - CheckResult result = check(R"( function f(v:string?) return if v then v else tostring(v) @@ -1048,9 +1045,6 @@ TEST_CASE_FIXTURE(Fixture, "is_truthy_constraint_ifelse_expression") TEST_CASE_FIXTURE(Fixture, "invert_is_truthy_constraint_ifelse_expression") { - ScopedFastFlag sff1{"LuauIfElseExpressionBaseSupport", true}; - ScopedFastFlag sff2{"LuauIfElseExpressionAnalysisSupport", true}; - CheckResult result = check(R"( function f(v:string?) return if not v then tostring(v) else v @@ -1065,9 +1059,6 @@ TEST_CASE_FIXTURE(Fixture, "invert_is_truthy_constraint_ifelse_expression") TEST_CASE_FIXTURE(Fixture, "type_comparison_ifelse_expression") { - ScopedFastFlag sff1{"LuauIfElseExpressionBaseSupport", true}; - ScopedFastFlag sff2{"LuauIfElseExpressionAnalysisSupport", true}; - CheckResult result = check(R"( function returnOne(x) return 1 @@ -1119,6 +1110,25 @@ TEST_CASE_FIXTURE(Fixture, "correctly_lookup_property_whose_base_was_previously_ CHECK_EQ("string", toString(requireTypeAtPosition({5, 30}))); } +TEST_CASE_FIXTURE(Fixture, "correctly_lookup_property_whose_base_was_previously_refined2") +{ + ScopedFastFlag sff{"LuauLValueAsKey", true}; + + CheckResult result = check(R"( + type T = { x: { y: number }? } + + local function f(t: T?) + if t and t.x then + local foo = t.x.y + end + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ("number", toString(requireTypeAtPosition({5, 32}))); +} + TEST_CASE_FIXTURE(Fixture, "apply_refinements_on_astexprindexexpr_whose_subscript_expr_is_constant_string") { ScopedFastFlag sff{"LuauRefiLookupFromIndexExpr", true}; diff --git a/tests/TypeInfer.singletons.test.cpp b/tests/TypeInfer.singletons.test.cpp index 68dc1b4f..94cfb643 100644 --- a/tests/TypeInfer.singletons.test.cpp +++ b/tests/TypeInfer.singletons.test.cpp @@ -360,6 +360,7 @@ TEST_CASE_FIXTURE(Fixture, "table_properties_type_error_escapes") { ScopedFastFlag sffs[] = { {"LuauParseSingletonTypes", true}, + {"LuauUnsealedTableLiteral", true}, }; CheckResult result = check(R"( @@ -369,7 +370,7 @@ TEST_CASE_FIXTURE(Fixture, "table_properties_type_error_escapes") )"); LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK_EQ(R"(Table type '{| ["\n"]: number |}' not compatible with type '{| ["<>"]: number |}' because the former is missing field '<>')", + CHECK_EQ(R"(Table type '{ ["\n"]: number }' not compatible with type '{| ["<>"]: number |}' because the former is missing field '<>')", toString(result.errors[0])); } @@ -423,4 +424,27 @@ caused by: toString(result.errors[0])); } +TEST_CASE_FIXTURE(Fixture, "if_then_else_expression_singleton_options") +{ + ScopedFastFlag sffs[] = { + {"LuauSingletonTypes", true}, + {"LuauParseSingletonTypes", true}, + {"LuauUnionHeuristic", true}, + {"LuauExpectedTypesOfProperties", true}, + {"LuauExtendedUnionMismatchError", true}, + {"LuauIfElseExpectedType2", true}, + {"LuauIfElseBranchTypeUnion", true}, + }; + + CheckResult result = check(R"( +type Cat = { tag: 'cat', catfood: string } +type Dog = { tag: 'dog', dogfood: string } +type Animal = Cat | Dog + +local a: Animal = if true then { tag = 'cat', catfood = 'something' } else { tag = 'dog', dogfood = 'other' } + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.tables.test.cpp b/tests/TypeInfer.tables.test.cpp index 80f40407..27cda146 100644 --- a/tests/TypeInfer.tables.test.cpp +++ b/tests/TypeInfer.tables.test.cpp @@ -65,7 +65,7 @@ TEST_CASE_FIXTURE(Fixture, "augment_nested_table") TEST_CASE_FIXTURE(Fixture, "cannot_augment_sealed_table") { - CheckResult result = check("local t = {prop=999} t.foo = 'bar'"); + CheckResult result = check("function mkt() return {prop=999} end local t = mkt() t.foo = 'bar'"); LUAU_REQUIRE_ERROR_COUNT(1, result); TypeError& err = result.errors[0]; @@ -77,7 +77,7 @@ TEST_CASE_FIXTURE(Fixture, "cannot_augment_sealed_table") CHECK_EQ(s, "{| prop: number |}"); CHECK_EQ(error->prop, "foo"); CHECK_EQ(error->context, CannotExtendTable::Property); - CHECK_EQ(err.location, (Location{Position{0, 24}, Position{0, 29}})); + CHECK_EQ(err.location, (Location{Position{0, 59}, Position{0, 64}})); } TEST_CASE_FIXTURE(Fixture, "dont_seal_an_unsealed_table_by_passing_it_to_a_function_that_takes_a_sealed_table") @@ -1155,7 +1155,8 @@ TEST_CASE_FIXTURE(Fixture, "defining_a_self_method_for_a_builtin_sealed_table_mu TEST_CASE_FIXTURE(Fixture, "defining_a_method_for_a_local_sealed_table_must_fail") { CheckResult result = check(R"( - local t = {x = 1} + function mkt() return {x = 1} end + local t = mkt() function t.m() end )"); @@ -1165,13 +1166,38 @@ TEST_CASE_FIXTURE(Fixture, "defining_a_method_for_a_local_sealed_table_must_fail TEST_CASE_FIXTURE(Fixture, "defining_a_self_method_for_a_local_sealed_table_must_fail") { CheckResult result = check(R"( - local t = {x = 1} + function mkt() return {x = 1} end + local t = mkt() function t:m() end )"); LUAU_REQUIRE_ERROR_COUNT(1, result); } +TEST_CASE_FIXTURE(Fixture, "defining_a_method_for_a_local_unsealed_table_is_ok") +{ + ScopedFastFlag sff{"LuauUnsealedTableLiteral", true}; + + CheckResult result = check(R"( + local t = {x = 1} + function t.m() end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "defining_a_self_method_for_a_local_unsealed_table_is_ok") +{ + ScopedFastFlag sff{"LuauUnsealedTableLiteral", true}; + + CheckResult result = check(R"( + local t = {x = 1} + function t:m() end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + // This unit test could be flaky if the fix has regressed. TEST_CASE_FIXTURE(Fixture, "pass_incompatible_union_to_a_generic_table_without_crashing") { @@ -1439,8 +1465,13 @@ TEST_CASE_FIXTURE(Fixture, "right_table_missing_key2") CHECK_EQ("{| |}", toString(mp->subType)); } -TEST_CASE_FIXTURE(Fixture, "casting_tables_with_props_into_table_with_indexer") +TEST_CASE_FIXTURE(Fixture, "casting_unsealed_tables_with_props_into_table_with_indexer") { + ScopedFastFlag sff[]{ + {"LuauTableSubtypingVariance2", true}, + {"LuauUnsealedTableLiteral", true}, + }; + CheckResult result = check(R"( type StringToStringMap = { [string]: string } local rt: StringToStringMap = { ["foo"] = 1 } @@ -1448,6 +1479,25 @@ TEST_CASE_FIXTURE(Fixture, "casting_tables_with_props_into_table_with_indexer") LUAU_REQUIRE_ERROR_COUNT(1, result); + ToStringOptions o{/* exhaustive= */ true}; + TypeMismatch* tm = get(result.errors[0]); + REQUIRE(tm); + CHECK_EQ("{| [string]: string |}", toString(tm->wantedType, o)); + // Should t now have an indexer? + // It would if the assignment to rt was correctly typed. + CHECK_EQ("{ [string]: string, foo: number }", toString(tm->givenType, o)); +} + +TEST_CASE_FIXTURE(Fixture, "casting_sealed_tables_with_props_into_table_with_indexer") +{ + CheckResult result = check(R"( + type StringToStringMap = { [string]: string } + function mkrt() return { ["foo"] = 1 } end + local rt: StringToStringMap = mkrt() + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + ToStringOptions o{/* exhaustive= */ true}; TypeMismatch* tm = get(result.errors[0]); REQUIRE(tm); @@ -1467,7 +1517,10 @@ TEST_CASE_FIXTURE(Fixture, "casting_tables_with_props_into_table_with_indexer2") TEST_CASE_FIXTURE(Fixture, "casting_tables_with_props_into_table_with_indexer3") { - ScopedFastFlag sff{"LuauTableSubtypingVariance2", true}; + ScopedFastFlag sff[]{ + {"LuauTableSubtypingVariance2", true}, + {"LuauUnsealedTableLiteral", true}, + }; CheckResult result = check(R"( local function foo(a: {[string]: number, a: string}) end @@ -1480,7 +1533,7 @@ TEST_CASE_FIXTURE(Fixture, "casting_tables_with_props_into_table_with_indexer3") TypeMismatch* tm = get(result.errors[0]); REQUIRE(tm); CHECK_EQ("{| [string]: number, a: string |}", toString(tm->wantedType, o)); - CHECK_EQ("{| a: number |}", toString(tm->givenType, o)); + CHECK_EQ("{ a: number }", toString(tm->givenType, o)); } TEST_CASE_FIXTURE(Fixture, "casting_tables_with_props_into_table_with_indexer4") @@ -1536,8 +1589,11 @@ TEST_CASE_FIXTURE(Fixture, "table_subtyping_with_missing_props_dont_report_multi TEST_CASE_FIXTURE(Fixture, "table_subtyping_with_extra_props_dont_report_multiple_errors") { CheckResult result = check(R"( - local vec3 = {{x = 1, y = 2, z = 3}} - local vec1 = {{x = 1}} + function mkvec3() return {x = 1, y = 2, z = 3} end + function mkvec1() return {x = 1} end + + local vec3 = {mkvec3()} + local vec1 = {mkvec1()} vec1 = vec3 )"); @@ -1620,7 +1676,8 @@ TEST_CASE_FIXTURE(Fixture, "reasonable_error_when_adding_a_nonexistent_property_ { CheckResult result = check(R"( --!strict - local A = {"value"} + function mkA() return {"value"} end + local A = mkA() A.B = "Hello" )"); @@ -1668,7 +1725,8 @@ TEST_CASE_FIXTURE(Fixture, "hide_table_error_properties") --!strict local function f() - local t = { x = 1 } + local function mkt() return { x = 1 } end + local t = mkt() function t.a() end function t.b() end @@ -1995,7 +2053,10 @@ caused by: TEST_CASE_FIXTURE(Fixture, "error_detailed_metatable_prop") { - ScopedFastFlag LuauTableSubtypingVariance2{"LuauTableSubtypingVariance2", true}; // Only for new path + ScopedFastFlag sff[]{ + {"LuauTableSubtypingVariance2", true}, + {"LuauUnsealedTableLiteral", true}, + }; CheckResult result = check(R"( local a1 = setmetatable({ x = 2, y = 3 }, { __call = function(s) end }); @@ -2010,7 +2071,7 @@ local c2: typeof(a2) = b2 LUAU_REQUIRE_ERROR_COUNT(2, result); CHECK_EQ(toString(result.errors[0]), R"(Type 'b1' could not be converted into 'a1' caused by: - Type '{| x: number, y: string |}' could not be converted into '{| x: number, y: number |}' + Type '{ x: number, y: string }' could not be converted into '{ x: number, y: number }' caused by: Property 'y' is not compatible. Type 'string' could not be converted into 'number')"); @@ -2018,7 +2079,7 @@ caused by: { CHECK_EQ(toString(result.errors[1]), R"(Type 'b2' could not be converted into 'a2' caused by: - Type '{| __call: (a, b) -> () |}' could not be converted into '{| __call: (a) -> () |}' + Type '{ __call: (a, b) -> () }' could not be converted into '{ __call: (a) -> () }' caused by: Property '__call' is not compatible. Type '(a, b) -> ()' could not be converted into '(a) -> ()'; different number of generic type parameters)"); } @@ -2026,7 +2087,7 @@ caused by: { CHECK_EQ(toString(result.errors[1]), R"(Type 'b2' could not be converted into 'a2' caused by: - Type '{| __call: (a, b) -> () |}' could not be converted into '{| __call: (a) -> () |}' + Type '{ __call: (a, b) -> () }' could not be converted into '{ __call: (a) -> () }' caused by: Property '__call' is not compatible. Type '(a, b) -> ()' could not be converted into '(a) -> ()')"); } @@ -2059,6 +2120,7 @@ TEST_CASE_FIXTURE(Fixture, "explicitly_typed_table_error") {"LuauPropertiesGetExpectedType", true}, {"LuauExpectedTypesOfProperties", true}, {"LuauTableSubtypingVariance2", true}, + {"LuauUnsealedTableLiteral", true}, }; CheckResult result = check(R"( @@ -2077,7 +2139,7 @@ local y: number = tmp.p.y LUAU_REQUIRE_ERROR_COUNT(1, result); CHECK_EQ(toString(result.errors[0]), R"(Type 'tmp' could not be converted into 'HasSuper' caused by: - Property 'p' is not compatible. Table type '{| x: number, y: number |}' not compatible with type 'Super' because the former has extra field 'y')"); + Property 'p' is not compatible. Table type '{ x: number, y: number }' not compatible with type 'Super' because the former has extra field 'y')"); } TEST_CASE_FIXTURE(Fixture, "explicitly_typed_table_with_indexer") @@ -2103,7 +2165,10 @@ a.p = { x = 9 } TEST_CASE_FIXTURE(Fixture, "recursive_metatable_type_call") { - ScopedFastFlag luauFixRecursiveMetatableCall{"LuauFixRecursiveMetatableCall", true}; + ScopedFastFlag sff[]{ + {"LuauFixRecursiveMetatableCall", true}, + {"LuauUnsealedTableLiteral", true}, + }; CheckResult result = check(R"( local b @@ -2112,7 +2177,7 @@ b() )"); LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK_EQ(toString(result.errors[0]), R"(Cannot call non-function t1 where t1 = { @metatable {| __call: t1 |}, { } })"); + CHECK_EQ(toString(result.errors[0]), R"(Cannot call non-function t1 where t1 = { @metatable { __call: t1 }, { } })"); } TEST_SUITE_END(); diff --git a/tests/TypeInfer.test.cpp b/tests/TypeInfer.test.cpp index f70f3b1c..7a056af5 100644 --- a/tests/TypeInfer.test.cpp +++ b/tests/TypeInfer.test.cpp @@ -4525,7 +4525,9 @@ f(function(x) print(x) end) } TEST_CASE_FIXTURE(Fixture, "infer_generic_function_function_argument") -{ +{ + ScopedFastFlag sff{"LuauUnsealedTableLiteral", true}; + CheckResult result = check(R"( local function sum(x: a, y: a, f: (a, a) -> a) return f(x, y) end return sum(2, 3, function(a, b) return a + b end) @@ -4549,7 +4551,7 @@ local r = foldl(a, {s=0,c=0}, function(a, b) return {s = a.s + b, c = a.c + 1} e )"); LUAU_REQUIRE_NO_ERRORS(result); - REQUIRE_EQ("{| c: number, s: number |}", toString(requireType("r"))); + REQUIRE_EQ("{ c: number, s: number }", toString(requireType("r"))); } TEST_CASE_FIXTURE(Fixture, "infer_generic_function_function_argument_overloaded") @@ -4689,6 +4691,18 @@ a = setmetatable(a, { __call = function(x) end }) LUAU_REQUIRE_NO_ERRORS(result); } +TEST_CASE_FIXTURE(Fixture, "infer_through_group_expr") +{ + ScopedFastFlag luauGroupExpectedType{"LuauGroupExpectedType", true}; + + CheckResult result = check(R"( +local function f(a: (number, number) -> number) return a(1, 3) end +f(((function(a, b) return a + b end))) + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + TEST_CASE_FIXTURE(Fixture, "refine_and_or") { CheckResult result = check(R"( @@ -4743,46 +4757,75 @@ local c: X TEST_CASE_FIXTURE(Fixture, "tc_if_else_expressions1") { - ScopedFastFlag sff1{"LuauIfElseExpressionBaseSupport", true}; - ScopedFastFlag sff2{"LuauIfElseExpressionAnalysisSupport", true}; - - { - CheckResult result = check(R"(local a = if true then "true" else "false")"); - LUAU_REQUIRE_NO_ERRORS(result); - TypeId aType = requireType("a"); - CHECK_EQ(getPrimitiveType(aType), PrimitiveTypeVar::String); - } + CheckResult result = check(R"(local a = if true then "true" else "false")"); + LUAU_REQUIRE_NO_ERRORS(result); + TypeId aType = requireType("a"); + CHECK_EQ(getPrimitiveType(aType), PrimitiveTypeVar::String); } TEST_CASE_FIXTURE(Fixture, "tc_if_else_expressions2") { - ScopedFastFlag sff1{"LuauIfElseExpressionBaseSupport", true}; - ScopedFastFlag sff2{"LuauIfElseExpressionAnalysisSupport", true}; + // Test expression containing elseif + CheckResult result = check(R"( +local a = if false then "a" elseif false then "b" else "c" + )"); + LUAU_REQUIRE_NO_ERRORS(result); + TypeId aType = requireType("a"); + CHECK_EQ(getPrimitiveType(aType), PrimitiveTypeVar::String); +} + +TEST_CASE_FIXTURE(Fixture, "tc_if_else_expressions_type_union") +{ + ScopedFastFlag sff3{"LuauIfElseBranchTypeUnion", true}; { - // Test expression containing elseif - CheckResult result = check(R"( -local a = if false then "a" elseif false then "b" else "c" - )"); + CheckResult result = check(R"(local a: number? = if true then 42 else nil)"); + LUAU_REQUIRE_NO_ERRORS(result); - TypeId aType = requireType("a"); - CHECK_EQ(getPrimitiveType(aType), PrimitiveTypeVar::String); + CHECK_EQ(toString(requireType("a"), {true}), "number?"); } } -TEST_CASE_FIXTURE(Fixture, "tc_if_else_expressions3") +TEST_CASE_FIXTURE(Fixture, "tc_if_else_expressions_expected_type_1") { - ScopedFastFlag sff1{"LuauIfElseExpressionBaseSupport", true}; - ScopedFastFlag sff2{"LuauIfElseExpressionAnalysisSupport", true}; + ScopedFastFlag luauIfElseExpectedType2{"LuauIfElseExpectedType2", true}; + ScopedFastFlag luauIfElseBranchTypeUnion{"LuauIfElseBranchTypeUnion", true}; - { - CheckResult result = check(R"(local a = if true then "true" else 42)"); - // We currently require both true/false expressions to unify to the same type. However, we do intend to lift - // this restriction in the future. - LUAU_REQUIRE_ERROR_COUNT(1, result); - TypeId aType = requireType("a"); - CHECK_EQ(getPrimitiveType(aType), PrimitiveTypeVar::String); - } + CheckResult result = check(R"( +type X = {number | string} +local a: X = if true then {"1", 2, 3} else {4, 5, 6} +)"); + + LUAU_REQUIRE_NO_ERRORS(result); + CHECK_EQ(toString(requireType("a"), {true}), "{number | string}"); +} + +TEST_CASE_FIXTURE(Fixture, "tc_if_else_expressions_expected_type_2") +{ + ScopedFastFlag luauIfElseExpectedType2{"LuauIfElseExpectedType2", true}; + ScopedFastFlag luauIfElseBranchTypeUnion{ "LuauIfElseBranchTypeUnion", true }; + + CheckResult result = check(R"( +local a: number? = if true then 1 else nil +)"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "tc_if_else_expressions_expected_type_3") +{ + ScopedFastFlag luauIfElseExpectedType2{"LuauIfElseExpectedType2", true}; + + CheckResult result = check(R"( +local function times(n: any, f: () -> T) + local result: {T} = {} + local res = f() + table.insert(result, if true then res else n) + return result +end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); } TEST_CASE_FIXTURE(Fixture, "type_error_addition") @@ -5039,4 +5082,51 @@ end LUAU_REQUIRE_NO_ERRORS(result); } +TEST_CASE_FIXTURE(Fixture, "table_oop") +{ + CheckResult result = check(R"( + --!strict +local Class = {} +Class.__index = Class + +type Class = typeof(setmetatable({} :: { x: number }, Class)) + +function Class.new(x: number): Class + return setmetatable({x = x}, Class) +end + +function Class.getx(self: Class) + return self.x +end + +function test() + local c = Class.new(42) + local n = c:getx() + local nn = c.x + + print(string.format("%d %d", n, nn)) +end +)"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "recursive_metatable_crash") +{ + ScopedFastFlag luauMetatableAreEqualRecursion{"LuauMetatableAreEqualRecursion", true}; + + CheckResult result = check(R"( +local function getIt() + local y + y = setmetatable({}, y) + return y +end +local a = getIt() +local b = getIt() +local c = a or b + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.typePacks.cpp b/tests/TypeInfer.typePacks.cpp index 5d37b032..d4878d14 100644 --- a/tests/TypeInfer.typePacks.cpp +++ b/tests/TypeInfer.typePacks.cpp @@ -621,4 +621,328 @@ type Other = Packed CHECK_EQ(toString(result.errors[0]), "Generic type 'Packed' expects 2 type pack arguments, but only 1 is specified"); } +TEST_CASE_FIXTURE(Fixture, "type_alias_default_type_explicit") +{ + ScopedFastFlag luauParseTypeAliasDefaults{"LuauParseTypeAliasDefaults", true}; + ScopedFastFlag luauTypeAliasDefaults{"LuauTypeAliasDefaults", true}; + + CheckResult result = check(R"( +type Y = { a: T, b: U } + +local a: Y = { a = 2, b = 3 } +local b: Y = { a = 2, b = "s" } + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ(toString(requireType("a")), "Y"); + CHECK_EQ(toString(requireType("b")), "Y"); + + result = check(R"( +type Y = { a: T } + +local a: Y = { a = 2 } +local b: Y<> = { a = "s" } +local c: Y = { a = "s" } + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ(toString(requireType("a")), "Y"); + CHECK_EQ(toString(requireType("b")), "Y"); + CHECK_EQ(toString(requireType("c")), "Y"); +} + +TEST_CASE_FIXTURE(Fixture, "type_alias_default_type_self") +{ + ScopedFastFlag luauParseTypeAliasDefaults{"LuauParseTypeAliasDefaults", true}; + ScopedFastFlag luauTypeAliasDefaults{"LuauTypeAliasDefaults", true}; + + CheckResult result = check(R"( +type Y = { a: T, b: U } + +local a: Y = { a = 2, b = 3 } +local b: Y = { a = "h", b = "s" } + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ(toString(requireType("a")), "Y"); + CHECK_EQ(toString(requireType("b")), "Y"); + + result = check(R"( +type Y string> = { a: T, b: U } + +local a: Y + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ(toString(requireType("a")), "Y string>"); +} + +TEST_CASE_FIXTURE(Fixture, "type_alias_default_type_chained") +{ + ScopedFastFlag luauParseTypeAliasDefaults{"LuauParseTypeAliasDefaults", true}; + ScopedFastFlag luauTypeAliasDefaults{"LuauTypeAliasDefaults", true}; + + CheckResult result = check(R"( +type Y = { a: T, b: U, c: V } + +local a: Y +local b: Y + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ(toString(requireType("a")), "Y"); + CHECK_EQ(toString(requireType("b")), "Y"); +} + +TEST_CASE_FIXTURE(Fixture, "type_alias_default_type_pack_explicit") +{ + ScopedFastFlag luauParseTypeAliasDefaults{"LuauParseTypeAliasDefaults", true}; + ScopedFastFlag luauTypeAliasDefaults{"LuauTypeAliasDefaults", true}; + + CheckResult result = check(R"( +type Y = { a: (T...) -> () } +local a: Y<> + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ(toString(requireType("a")), "Y"); +} + +TEST_CASE_FIXTURE(Fixture, "type_alias_default_type_pack_self_ty") +{ + ScopedFastFlag luauParseTypeAliasDefaults{"LuauParseTypeAliasDefaults", true}; + ScopedFastFlag luauTypeAliasDefaults{"LuauTypeAliasDefaults", true}; + + CheckResult result = check(R"( +type Y = { a: T, b: (U...) -> T } + +local a: Y + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ(toString(requireType("a")), "Y"); +} + +TEST_CASE_FIXTURE(Fixture, "type_alias_default_type_pack_self_tp") +{ + ScopedFastFlag luauParseTypeAliasDefaults{"LuauParseTypeAliasDefaults", true}; + ScopedFastFlag luauTypeAliasDefaults{"LuauTypeAliasDefaults", true}; + + CheckResult result = check(R"( +type Y = { a: (T...) -> U... } +local a: Y + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ(toString(requireType("a")), "Y<(number, string), (number, string)>"); +} + +TEST_CASE_FIXTURE(Fixture, "type_alias_default_type_pack_self_chained_tp") +{ + ScopedFastFlag luauParseTypeAliasDefaults{"LuauParseTypeAliasDefaults", true}; + ScopedFastFlag luauTypeAliasDefaults{"LuauTypeAliasDefaults", true}; + + CheckResult result = check(R"( +type Y = { a: (T...) -> U..., b: (T...) -> V... } +local a: Y + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ(toString(requireType("a")), "Y<(number, string), (number, string), (number, string)>"); +} + +TEST_CASE_FIXTURE(Fixture, "type_alias_default_mixed_self") +{ + ScopedFastFlag luauParseTypeAliasDefaults{"LuauParseTypeAliasDefaults", true}; + ScopedFastFlag luauTypeAliasDefaults{"LuauTypeAliasDefaults", true}; + + CheckResult result = check(R"( +type Y = { a: (T, U, V...) -> W... } +local a: Y +local b: Y +local c: Y +local d: Y ()> + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ(toString(requireType("a")), "Y"); + CHECK_EQ(toString(requireType("b")), "Y"); + CHECK_EQ(toString(requireType("c")), "Y"); + CHECK_EQ(toString(requireType("d")), "Y ()>"); +} + +TEST_CASE_FIXTURE(Fixture, "type_alias_default_type_errors") +{ + ScopedFastFlag luauParseTypeAliasDefaults{"LuauParseTypeAliasDefaults", true}; + ScopedFastFlag luauTypeAliasDefaults{"LuauTypeAliasDefaults", true}; + + CheckResult result = check(R"( +type Y = { a: T } +local a: Y = { a = 2 } + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ(toString(result.errors[0]), "Unknown type 'T'"); + + result = check(R"( +type Y = { a: (T...) -> () } +local a: Y<> + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ(toString(result.errors[0]), "Unknown type 'T'"); + + result = check(R"( +type Y = { a: (T) -> U... } +local a: Y<...number> + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ(toString(result.errors[0]), "Generic type 'Y' expects at least 1 type argument, but none are specified"); + + result = check(R"( +type Packed = (T) -> T +local a: Packed + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ(toString(result.errors[0]), "Type parameter list is required"); + + result = check(R"( +type Y = { a: T } +local a: Y + )"); + + LUAU_REQUIRE_ERRORS(result); + + result = check(R"( +type Y = { a: T } +local a: Y<...number> + )"); + + LUAU_REQUIRE_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "type_alias_default_export") +{ + ScopedFastFlag luauParseTypeAliasDefaults{"LuauParseTypeAliasDefaults", true}; + ScopedFastFlag luauTypeAliasDefaults{"LuauTypeAliasDefaults", true}; + + fileResolver.source["Module/Types"] = R"( +export type A = { a: T, b: U } +export type B = { a: T, b: U } +export type C string> = { a: T, b: U } +export type D = { a: T, b: U, c: V } +export type E = { a: (T...) -> () } +export type F = { a: T, b: (U...) -> T } +export type G = { b: (U...) -> T... } +export type H = { b: (T...) -> T... } +return {} + )"; + + CheckResult resultTypes = frontend.check("Module/Types"); + LUAU_REQUIRE_NO_ERRORS(resultTypes); + + fileResolver.source["Module/Users"] = R"( +local Types = require(script.Parent.Types) + +local a: Types.A +local b: Types.B +local c: Types.C +local d: Types.D +local e: Types.E<> +local eVoid: Types.E<()> +local f: Types.F +local g: Types.G<...number> +local h: Types.H<> + )"; + + CheckResult resultUsers = frontend.check("Module/Users"); + LUAU_REQUIRE_NO_ERRORS(resultUsers); + + CHECK_EQ(toString(requireType("Module/Users", "a")), "A"); + CHECK_EQ(toString(requireType("Module/Users", "b")), "B"); + CHECK_EQ(toString(requireType("Module/Users", "c")), "C string>"); + CHECK_EQ(toString(requireType("Module/Users", "d")), "D"); + CHECK_EQ(toString(requireType("Module/Users", "e")), "E"); + CHECK_EQ(toString(requireType("Module/Users", "eVoid")), "E<>"); + CHECK_EQ(toString(requireType("Module/Users", "f")), "F"); + CHECK_EQ(toString(requireType("Module/Users", "g")), "G<...number, ()>"); + CHECK_EQ(toString(requireType("Module/Users", "h")), "H<>"); +} + +TEST_CASE_FIXTURE(Fixture, "type_alias_default_type_skip_brackets") +{ + ScopedFastFlag luauParseTypeAliasDefaults{"LuauParseTypeAliasDefaults", true}; + ScopedFastFlag luauTypeAliasDefaults{"LuauTypeAliasDefaults", true}; + + CheckResult result = check(R"( +type Y = (T...) -> number +local a: Y + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ(toString(requireType("a")), "(...string) -> number"); +} + +TEST_CASE_FIXTURE(Fixture, "type_alias_defaults_confusing_types") +{ + ScopedFastFlag luauParseTypeAliasDefaults{"LuauParseTypeAliasDefaults", true}; + ScopedFastFlag luauTypeAliasDefaults{"LuauTypeAliasDefaults", true}; + + CheckResult result = check(R"( +type A = (T, V...) -> (U, W...) +type B = A +type C = A + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ(toString(*lookupType("B"), {true}), "(string, ...any) -> (number, ...any)"); + CHECK_EQ(toString(*lookupType("C"), {true}), "(string, boolean) -> (number, boolean)"); +} + +TEST_CASE_FIXTURE(Fixture, "type_alias_defaults_recursive_type") +{ + ScopedFastFlag luauParseTypeAliasDefaults{"LuauParseTypeAliasDefaults", true}; + ScopedFastFlag luauTypeAliasDefaults{"LuauTypeAliasDefaults", true}; + + CheckResult result = check(R"( +type F ()> = (K) -> V +type R = { m: F } + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ(toString(*lookupType("R"), {true}), "t1 where t1 = {| m: (t1) -> (t1) -> () |}"); +} + +TEST_CASE_FIXTURE(Fixture, "pack_tail_unification_check") +{ + ScopedFastFlag luauUnifyPackTails{"LuauUnifyPackTails", true}; + ScopedFastFlag luauExtendedFunctionMismatchError{"LuauExtendedFunctionMismatchError", true}; + + CheckResult result = check(R"( +local a: () -> (number, ...string) +local b: () -> (number, ...boolean) +a = b + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ(toString(result.errors[0]), R"(Type '() -> (number, ...boolean)' could not be converted into '() -> (number, ...string)' +caused by: + Type 'boolean' could not be converted into 'string')"); +} + TEST_SUITE_END(); From 32c39e2162b7539fd3233215df3dea23421c0503 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Fri, 14 Jan 2022 08:20:09 -0800 Subject: [PATCH 128/399] Sync to upstream/release/510 (#313) --- Analysis/include/Luau/TypeInfer.h | 12 +- Analysis/include/Luau/TypeVar.h | 20 +- Analysis/src/Autocomplete.cpp | 79 +-- Analysis/src/Error.cpp | 12 +- Analysis/src/IostreamHelpers.cpp | 8 +- Analysis/src/JsonEncoder.cpp | 38 ++ Analysis/src/Module.cpp | 26 +- Analysis/src/ToString.cpp | 107 +++- Analysis/src/Transpiler.cpp | 65 +- Analysis/src/TypeAttach.cpp | 12 +- Analysis/src/TypeInfer.cpp | 310 ++++++++-- Analysis/src/TypeVar.cpp | 4 + Analysis/src/Unifier.cpp | 17 +- Ast/include/Luau/Ast.h | 49 +- Ast/include/Luau/Parser.h | 6 +- Ast/src/Ast.cpp | 32 +- Ast/src/Parser.cpp | 128 ++-- CLI/Analyze.cpp | 18 +- CLI/Repl.cpp | 137 +++-- Compiler/src/Builtins.cpp | 197 ++++++ Compiler/src/Builtins.h | 41 ++ Compiler/src/Compiler.cpp | 878 ++------------------------- Compiler/src/ConstantFolding.cpp | 394 ++++++++++++ Compiler/src/ConstantFolding.h | 48 ++ Compiler/src/TableShape.cpp | 129 ++++ Compiler/src/TableShape.h | 21 + Compiler/src/ValueTracking.cpp | 103 ++++ Compiler/src/ValueTracking.h | 42 ++ Sources.cmake | 8 + VM/src/ldo.cpp | 12 +- VM/src/lfunc.cpp | 8 +- VM/src/lstrlib.cpp | 20 +- bench/tests/sunspider/3d-cube.lua | 32 +- tests/Autocomplete.test.cpp | 61 +- tests/Compiler.test.cpp | 93 +-- tests/Conformance.test.cpp | 8 - tests/Fixture.cpp | 5 +- tests/Fixture.h | 2 +- tests/Parser.test.cpp | 53 +- tests/ToString.test.cpp | 22 +- tests/Transpiler.test.cpp | 14 +- tests/TypeInfer.annotations.test.cpp | 2 +- tests/TypeInfer.refinements.test.cpp | 28 +- tests/TypeInfer.singletons.test.cpp | 26 +- tests/TypeInfer.tables.test.cpp | 101 ++- tests/TypeInfer.test.cpp | 150 ++++- tests/TypeInfer.typePacks.cpp | 324 ++++++++++ 47 files changed, 2641 insertions(+), 1261 deletions(-) create mode 100644 Compiler/src/Builtins.cpp create mode 100644 Compiler/src/Builtins.h create mode 100644 Compiler/src/ConstantFolding.cpp create mode 100644 Compiler/src/ConstantFolding.h create mode 100644 Compiler/src/TableShape.cpp create mode 100644 Compiler/src/TableShape.h create mode 100644 Compiler/src/ValueTracking.cpp create mode 100644 Compiler/src/ValueTracking.h diff --git a/Analysis/include/Luau/TypeInfer.h b/Analysis/include/Luau/TypeInfer.h index 312283b0..aa090014 100644 --- a/Analysis/include/Luau/TypeInfer.h +++ b/Analysis/include/Luau/TypeInfer.h @@ -97,6 +97,12 @@ struct ApplyTypeFunction : Substitution TypePackId clean(TypePackId tp) override; }; +struct GenericTypeDefinitions +{ + std::vector genericTypes; + std::vector genericPacks; +}; + // All TypeVars are retained via Environment::typeVars. All TypeIds // within a program are borrowed pointers into this set. struct TypeChecker @@ -146,7 +152,7 @@ struct TypeChecker ExprResult checkExpr(const ScopePtr& scope, const AstExprBinary& expr); ExprResult checkExpr(const ScopePtr& scope, const AstExprTypeAssertion& expr); ExprResult checkExpr(const ScopePtr& scope, const AstExprError& expr); - ExprResult checkExpr(const ScopePtr& scope, const AstExprIfElse& expr); + ExprResult checkExpr(const ScopePtr& scope, const AstExprIfElse& expr, std::optional expectedType = std::nullopt); TypeId checkExprTable(const ScopePtr& scope, const AstExprTable& expr, const std::vector>& fieldTypes, std::optional expectedType); @@ -336,8 +342,8 @@ private: const std::vector& typePackParams, const Location& location); // Note: `scope` must be a fresh scope. - std::pair, std::vector> createGenericTypes(const ScopePtr& scope, std::optional levelOpt, - const AstNode& node, const AstArray& genericNames, const AstArray& genericPackNames); + GenericTypeDefinitions createGenericTypes(const ScopePtr& scope, std::optional levelOpt, const AstNode& node, + const AstArray& genericNames, const AstArray& genericPackNames); public: ErrorVec resolve(const PredicateVec& predicates, const ScopePtr& scope, bool sense); diff --git a/Analysis/include/Luau/TypeVar.h b/Analysis/include/Luau/TypeVar.h index d6e17714..fd2c2afa 100644 --- a/Analysis/include/Luau/TypeVar.h +++ b/Analysis/include/Luau/TypeVar.h @@ -181,6 +181,18 @@ const T* get(const SingletonTypeVar* stv) return nullptr; } +struct GenericTypeDefinition +{ + TypeId ty; + std::optional defaultValue; +}; + +struct GenericTypePackDefinition +{ + TypePackId tp; + std::optional defaultValue; +}; + struct FunctionArgument { Name name; @@ -358,8 +370,8 @@ struct ClassTypeVar struct TypeFun { // These should all be generic - std::vector typeParams; - std::vector typePackParams; + std::vector typeParams; + std::vector typePackParams; /** The underlying type. * @@ -369,13 +381,13 @@ struct TypeFun TypeId type; TypeFun() = default; - TypeFun(std::vector typeParams, TypeId type) + TypeFun(std::vector typeParams, TypeId type) : typeParams(std::move(typeParams)) , type(type) { } - TypeFun(std::vector typeParams, std::vector typePackParams, TypeId type) + TypeFun(std::vector typeParams, std::vector typePackParams, TypeId type) : typeParams(std::move(typeParams)) , typePackParams(std::move(typePackParams)) , type(type) diff --git a/Analysis/src/Autocomplete.cpp b/Analysis/src/Autocomplete.cpp index 67ebd075..7a801f97 100644 --- a/Analysis/src/Autocomplete.cpp +++ b/Analysis/src/Autocomplete.cpp @@ -13,11 +13,10 @@ #include LUAU_FASTFLAG(LuauUseCommittingTxnLog) -LUAU_FASTFLAG(LuauIfElseExpressionAnalysisSupport) LUAU_FASTFLAGVARIABLE(LuauAutocompleteAvoidMutation, false); -LUAU_FASTFLAGVARIABLE(LuauAutocompletePreferToCallFunctions, false); LUAU_FASTFLAGVARIABLE(LuauAutocompleteFirstArg, false); LUAU_FASTFLAGVARIABLE(LuauCompleteBrokenStringParams, false); +LUAU_FASTFLAGVARIABLE(LuauMissingFollowACMetatables, false); static const std::unordered_set kStatementStartingKeywords = { "while", "if", "local", "repeat", "function", "do", "for", "return", "break", "continue", "type", "export"}; @@ -291,51 +290,23 @@ static TypeCorrectKind checkTypeCorrectKind(const Module& module, TypeArena* typ expectedType = follow(*it); } - if (FFlag::LuauAutocompletePreferToCallFunctions) + // We also want to suggest functions that return compatible result + if (const FunctionTypeVar* ftv = get(ty)) { - // We also want to suggest functions that return compatible result - if (const FunctionTypeVar* ftv = get(ty)) - { - auto [retHead, retTail] = flatten(ftv->retType); - - if (!retHead.empty() && canUnify(retHead.front(), expectedType)) - return TypeCorrectKind::CorrectFunctionResult; - - // We might only have a variadic tail pack, check if the element is compatible - if (retTail) - { - if (const VariadicTypePack* vtp = get(follow(*retTail)); vtp && canUnify(vtp->ty, expectedType)) - return TypeCorrectKind::CorrectFunctionResult; - } - } - - return canUnify(ty, expectedType) ? TypeCorrectKind::Correct : TypeCorrectKind::None; - } - else - { - if (canUnify(ty, expectedType)) - return TypeCorrectKind::Correct; - - // We also want to suggest functions that return compatible result - const FunctionTypeVar* ftv = get(ty); - - if (!ftv) - return TypeCorrectKind::None; - auto [retHead, retTail] = flatten(ftv->retType); - if (!retHead.empty()) - return canUnify(retHead.front(), expectedType) ? TypeCorrectKind::CorrectFunctionResult : TypeCorrectKind::None; + if (!retHead.empty() && canUnify(retHead.front(), expectedType)) + return TypeCorrectKind::CorrectFunctionResult; // We might only have a variadic tail pack, check if the element is compatible if (retTail) { - if (const VariadicTypePack* vtp = get(follow(*retTail))) - return canUnify(vtp->ty, expectedType) ? TypeCorrectKind::CorrectFunctionResult : TypeCorrectKind::None; + if (const VariadicTypePack* vtp = get(follow(*retTail)); vtp && canUnify(vtp->ty, expectedType)) + return TypeCorrectKind::CorrectFunctionResult; } - - return TypeCorrectKind::None; } + + return canUnify(ty, expectedType) ? TypeCorrectKind::Correct : TypeCorrectKind::None; } enum class PropIndexType @@ -435,13 +406,28 @@ static void autocompleteProps(const Module& module, TypeArena* typeArena, TypeId auto indexIt = mtable->props.find("__index"); if (indexIt != mtable->props.end()) { - if (get(indexIt->second.type) || get(indexIt->second.type)) - autocompleteProps(module, typeArena, indexIt->second.type, indexType, nodes, result, seen); - else if (auto indexFunction = get(indexIt->second.type)) + if (FFlag::LuauMissingFollowACMetatables) { - std::optional indexFunctionResult = first(indexFunction->retType); - if (indexFunctionResult) - autocompleteProps(module, typeArena, *indexFunctionResult, indexType, nodes, result, seen); + TypeId followed = follow(indexIt->second.type); + if (get(followed) || get(followed)) + autocompleteProps(module, typeArena, followed, indexType, nodes, result, seen); + else if (auto indexFunction = get(followed)) + { + std::optional indexFunctionResult = first(indexFunction->retType); + if (indexFunctionResult) + autocompleteProps(module, typeArena, *indexFunctionResult, indexType, nodes, result, seen); + } + } + else + { + if (get(indexIt->second.type) || get(indexIt->second.type)) + autocompleteProps(module, typeArena, indexIt->second.type, indexType, nodes, result, seen); + else if (auto indexFunction = get(indexIt->second.type)) + { + std::optional indexFunctionResult = first(indexFunction->retType); + if (indexFunctionResult) + autocompleteProps(module, typeArena, *indexFunctionResult, indexType, nodes, result, seen); + } } } } @@ -1224,7 +1210,7 @@ static void autocompleteExpression(const SourceModule& sourceModule, const Modul if (auto it = module.astTypes.find(node->asExpr())) autocompleteProps(module, typeArena, *it, PropIndexType::Point, ancestry, result); } - else if (FFlag::LuauIfElseExpressionAnalysisSupport && autocompleteIfElseExpression(node, ancestry, position, result)) + else if (autocompleteIfElseExpression(node, ancestry, position, result)) return; else if (node->is()) return; @@ -1261,8 +1247,7 @@ static void autocompleteExpression(const SourceModule& sourceModule, const Modul TypeCorrectKind correctForFunction = functionIsExpectedAt(module, node, position).value_or(false) ? TypeCorrectKind::Correct : TypeCorrectKind::None; - if (FFlag::LuauIfElseExpressionAnalysisSupport) - result["if"] = {AutocompleteEntryKind::Keyword, std::nullopt, false, false}; + result["if"] = {AutocompleteEntryKind::Keyword, std::nullopt, false, false}; result["true"] = {AutocompleteEntryKind::Keyword, typeChecker.booleanType, false, false, correctForBoolean}; result["false"] = {AutocompleteEntryKind::Keyword, typeChecker.booleanType, false, false, correctForBoolean}; result["nil"] = {AutocompleteEntryKind::Keyword, typeChecker.nilType, false, false, correctForNil}; diff --git a/Analysis/src/Error.cpp b/Analysis/src/Error.cpp index ce832c6b..88069f1f 100644 --- a/Analysis/src/Error.cpp +++ b/Analysis/src/Error.cpp @@ -190,24 +190,24 @@ struct ErrorConverter { name += "<"; bool first = true; - for (TypeId t : e.typeFun.typeParams) + for (auto param : e.typeFun.typeParams) { if (first) first = false; else name += ", "; - name += toString(t); + name += toString(param.ty); } - for (TypePackId t : e.typeFun.typePackParams) + for (auto param : e.typeFun.typePackParams) { if (first) first = false; else name += ", "; - name += toString(t); + name += toString(param.tp); } name += ">"; @@ -544,13 +544,13 @@ bool IncorrectGenericParameterCount::operator==(const IncorrectGenericParameterC for (size_t i = 0; i < typeFun.typeParams.size(); ++i) { - if (typeFun.typeParams[i] != rhs.typeFun.typeParams[i]) + if (typeFun.typeParams[i].ty != rhs.typeFun.typeParams[i].ty) return false; } for (size_t i = 0; i < typeFun.typePackParams.size(); ++i) { - if (typeFun.typePackParams[i] != rhs.typeFun.typePackParams[i]) + if (typeFun.typePackParams[i].tp != rhs.typeFun.typePackParams[i].tp) return false; } diff --git a/Analysis/src/IostreamHelpers.cpp b/Analysis/src/IostreamHelpers.cpp index 5bc76ade..19c2ddab 100644 --- a/Analysis/src/IostreamHelpers.cpp +++ b/Analysis/src/IostreamHelpers.cpp @@ -96,24 +96,24 @@ std::ostream& operator<<(std::ostream& stream, const IncorrectGenericParameterCo { stream << "<"; bool first = true; - for (TypeId t : error.typeFun.typeParams) + for (auto param : error.typeFun.typeParams) { if (first) first = false; else stream << ", "; - stream << toString(t); + stream << toString(param.ty); } - for (TypePackId t : error.typeFun.typePackParams) + for (auto param : error.typeFun.typePackParams) { if (first) first = false; else stream << ", "; - stream << toString(t); + stream << toString(param.tp); } stream << ">"; diff --git a/Analysis/src/JsonEncoder.cpp b/Analysis/src/JsonEncoder.cpp index 23491a5a..8dd597e1 100644 --- a/Analysis/src/JsonEncoder.cpp +++ b/Analysis/src/JsonEncoder.cpp @@ -5,6 +5,8 @@ #include "Luau/StringUtils.h" #include "Luau/Common.h" +LUAU_FASTFLAG(LuauTypeAliasDefaults) + namespace Luau { @@ -337,6 +339,42 @@ struct AstJsonEncoder : public AstVisitor writeRaw("}"); } + void write(const AstGenericType& genericType) + { + if (FFlag::LuauTypeAliasDefaults) + { + writeRaw("{"); + bool c = pushComma(); + write("name", genericType.name); + if (genericType.defaultValue) + write("type", genericType.defaultValue); + popComma(c); + writeRaw("}"); + } + else + { + write(genericType.name); + } + } + + void write(const AstGenericTypePack& genericTypePack) + { + if (FFlag::LuauTypeAliasDefaults) + { + writeRaw("{"); + bool c = pushComma(); + write("name", genericTypePack.name); + if (genericTypePack.defaultValue) + write("type", genericTypePack.defaultValue); + popComma(c); + writeRaw("}"); + } + else + { + write(genericTypePack.name); + } + } + void write(AstExprTable::Item::Kind kind) { switch (kind) diff --git a/Analysis/src/Module.cpp b/Analysis/src/Module.cpp index cff85897..9f352f4b 100644 --- a/Analysis/src/Module.cpp +++ b/Analysis/src/Module.cpp @@ -14,6 +14,7 @@ LUAU_FASTFLAGVARIABLE(DebugLuauFreezeArena, false) LUAU_FASTFLAGVARIABLE(DebugLuauTrackOwningArena, false) LUAU_FASTINTVARIABLE(LuauTypeCloneRecursionLimit, 300) +LUAU_FASTFLAG(LuauTypeAliasDefaults) namespace Luau { @@ -447,11 +448,28 @@ TypeId clone(TypeId typeId, TypeArena& dest, SeenTypes& seenTypes, SeenTypePacks TypeFun clone(const TypeFun& typeFun, TypeArena& dest, SeenTypes& seenTypes, SeenTypePacks& seenTypePacks, CloneState& cloneState) { TypeFun result; - for (TypeId ty : typeFun.typeParams) - result.typeParams.push_back(clone(ty, dest, seenTypes, seenTypePacks, cloneState)); - for (TypePackId tp : typeFun.typePackParams) - result.typePackParams.push_back(clone(tp, dest, seenTypes, seenTypePacks, cloneState)); + for (auto param : typeFun.typeParams) + { + TypeId ty = clone(param.ty, dest, seenTypes, seenTypePacks, cloneState); + std::optional defaultValue; + + if (FFlag::LuauTypeAliasDefaults && param.defaultValue) + defaultValue = clone(*param.defaultValue, dest, seenTypes, seenTypePacks, cloneState); + + result.typeParams.push_back({ty, defaultValue}); + } + + for (auto param : typeFun.typePackParams) + { + TypePackId tp = clone(param.tp, dest, seenTypes, seenTypePacks, cloneState); + std::optional defaultValue; + + if (FFlag::LuauTypeAliasDefaults && param.defaultValue) + defaultValue = clone(*param.defaultValue, dest, seenTypes, seenTypePacks, cloneState); + + result.typePackParams.push_back({tp, defaultValue}); + } result.type = clone(typeFun.type, dest, seenTypes, seenTypePacks, cloneState); diff --git a/Analysis/src/ToString.cpp b/Analysis/src/ToString.cpp index 889dd6dc..4b898d3a 100644 --- a/Analysis/src/ToString.cpp +++ b/Analysis/src/ToString.cpp @@ -11,6 +11,7 @@ #include LUAU_FASTFLAG(LuauOccursCheckOkWithRecursiveFunctions) +LUAU_FASTFLAG(LuauTypeAliasDefaults) /* * Prefix generic typenames with gen- @@ -209,6 +210,14 @@ struct StringifierState result.name += s; } + + void emit(const char* s) + { + if (opts.maxTypeLength > 0 && result.name.length() > opts.maxTypeLength) + return; + + result.name += s; + } }; struct TypeVarStringifier @@ -280,13 +289,28 @@ struct TypeVarStringifier else first = false; - if (!singleTp) - state.emit("("); + if (FFlag::LuauTypeAliasDefaults) + { + bool wrap = !singleTp && get(follow(tp)); - stringify(tp); + if (wrap) + state.emit("("); - if (!singleTp) - state.emit(")"); + stringify(tp); + + if (wrap) + state.emit(")"); + } + else + { + if (!singleTp) + state.emit("("); + + stringify(tp); + + if (!singleTp) + state.emit(")"); + } } if (types.size() || typePacks.size()) @@ -1086,7 +1110,7 @@ std::string toString(const TypePackVar& tp, const ToStringOptions& opts) return toString(const_cast(&tp), std::move(opts)); } -std::string toStringNamedFunction(const std::string& prefix, const FunctionTypeVar& ftv, ToStringOptions opts) +std::string toStringNamedFunction_DEPRECATED(const std::string& prefix, const FunctionTypeVar& ftv, ToStringOptions opts) { std::string s = prefix; @@ -1175,6 +1199,77 @@ std::string toStringNamedFunction(const std::string& prefix, const FunctionTypeV return s; } +std::string toStringNamedFunction(const std::string& prefix, const FunctionTypeVar& ftv, ToStringOptions opts) +{ + if (!FFlag::LuauTypeAliasDefaults) + return toStringNamedFunction_DEPRECATED(prefix, ftv, opts); + + ToStringResult result; + StringifierState state(opts, result, opts.nameMap); + TypeVarStringifier tvs{state}; + + state.emit(prefix); + + if (!opts.hideNamedFunctionTypeParameters) + tvs.stringify(ftv.generics, ftv.genericPacks); + + state.emit("("); + + auto argPackIter = begin(ftv.argTypes); + auto argNameIter = ftv.argNames.begin(); + + bool first = true; + while (argPackIter != end(ftv.argTypes)) + { + if (!first) + state.emit(", "); + first = false; + + // We don't currently respect opts.functionTypeArguments. I don't think this function should. + if (argNameIter != ftv.argNames.end()) + { + state.emit((*argNameIter ? (*argNameIter)->name : "_") + ": "); + ++argNameIter; + } + else + { + state.emit("_: "); + } + + tvs.stringify(*argPackIter); + ++argPackIter; + } + + if (argPackIter.tail()) + { + if (!first) + state.emit(", "); + + state.emit("...: "); + + if (auto vtp = get(*argPackIter.tail())) + tvs.stringify(vtp->ty); + else + tvs.stringify(*argPackIter.tail()); + } + + state.emit("): "); + + size_t retSize = size(ftv.retType); + bool hasTail = !finite(ftv.retType); + bool wrap = get(follow(ftv.retType)) && (hasTail ? retSize != 0 : retSize != 1); + + if (wrap) + state.emit("("); + + tvs.stringify(ftv.retType); + + if (wrap) + state.emit(")"); + + return result.name; +} + std::string dump(TypeId ty) { ToStringOptions opts; diff --git a/Analysis/src/Transpiler.cpp b/Analysis/src/Transpiler.cpp index 8e13ea5b..f5908683 100644 --- a/Analysis/src/Transpiler.cpp +++ b/Analysis/src/Transpiler.cpp @@ -10,6 +10,8 @@ #include #include +LUAU_FASTFLAG(LuauTypeAliasDefaults) + namespace { bool isIdentifierStartChar(char c) @@ -793,14 +795,47 @@ struct Printer for (auto o : a->generics) { comma(); - writer.identifier(o.value); + + if (FFlag::LuauTypeAliasDefaults) + { + writer.advance(o.location.begin); + writer.identifier(o.name.value); + + if (o.defaultValue) + { + writer.maybeSpace(o.defaultValue->location.begin, 2); + writer.symbol("="); + visualizeTypeAnnotation(*o.defaultValue); + } + } + else + { + writer.identifier(o.name.value); + } } for (auto o : a->genericPacks) { comma(); - writer.identifier(o.value); - writer.symbol("..."); + + if (FFlag::LuauTypeAliasDefaults) + { + writer.advance(o.location.begin); + writer.identifier(o.name.value); + writer.symbol("..."); + + if (o.defaultValue) + { + writer.maybeSpace(o.defaultValue->location.begin, 2); + writer.symbol("="); + visualizeTypePackAnnotation(*o.defaultValue, false); + } + } + else + { + writer.identifier(o.name.value); + writer.symbol("..."); + } } writer.symbol(">"); @@ -846,12 +881,20 @@ struct Printer for (const auto& o : func.generics) { comma(); - writer.identifier(o.value); + + if (FFlag::LuauTypeAliasDefaults) + writer.advance(o.location.begin); + + writer.identifier(o.name.value); } for (const auto& o : func.genericPacks) { comma(); - writer.identifier(o.value); + + if (FFlag::LuauTypeAliasDefaults) + writer.advance(o.location.begin); + + writer.identifier(o.name.value); writer.symbol("..."); } writer.symbol(">"); @@ -979,12 +1022,20 @@ struct Printer for (const auto& o : a->generics) { comma(); - writer.identifier(o.value); + + if (FFlag::LuauTypeAliasDefaults) + writer.advance(o.location.begin); + + writer.identifier(o.name.value); } for (const auto& o : a->genericPacks) { comma(); - writer.identifier(o.value); + + if (FFlag::LuauTypeAliasDefaults) + writer.advance(o.location.begin); + + writer.identifier(o.name.value); writer.symbol("..."); } writer.symbol(">"); diff --git a/Analysis/src/TypeAttach.cpp b/Analysis/src/TypeAttach.cpp index 9e61c792..2ec02093 100644 --- a/Analysis/src/TypeAttach.cpp +++ b/Analysis/src/TypeAttach.cpp @@ -212,24 +212,24 @@ public: if (hasSeen(&ftv)) return allocator->alloc(Location(), std::nullopt, AstName("")); - AstArray generics; + AstArray generics; generics.size = ftv.generics.size(); - generics.data = static_cast(allocator->allocate(sizeof(AstName) * generics.size)); + generics.data = static_cast(allocator->allocate(sizeof(AstGenericType) * generics.size)); size_t numGenerics = 0; for (auto it = ftv.generics.begin(); it != ftv.generics.end(); ++it) { if (auto gtv = get(*it)) - generics.data[numGenerics++] = AstName(gtv->name.c_str()); + generics.data[numGenerics++] = {AstName(gtv->name.c_str()), Location(), nullptr}; } - AstArray genericPacks; + AstArray genericPacks; genericPacks.size = ftv.genericPacks.size(); - genericPacks.data = static_cast(allocator->allocate(sizeof(AstName) * genericPacks.size)); + genericPacks.data = static_cast(allocator->allocate(sizeof(AstGenericTypePack) * genericPacks.size)); size_t numGenericPacks = 0; for (auto it = ftv.genericPacks.begin(); it != ftv.genericPacks.end(); ++it) { if (auto gtv = get(*it)) - genericPacks.data[numGenericPacks++] = AstName(gtv->name.c_str()); + genericPacks.data[numGenericPacks++] = {AstName(gtv->name.c_str()), Location(), nullptr}; } AstArray argTypes; diff --git a/Analysis/src/TypeInfer.cpp b/Analysis/src/TypeInfer.cpp index 1689a5c3..bedcc022 100644 --- a/Analysis/src/TypeInfer.cpp +++ b/Analysis/src/TypeInfer.cpp @@ -15,8 +15,8 @@ #include "Luau/TypeVar.h" #include "Luau/TimeTrace.h" -#include #include +#include LUAU_FASTFLAGVARIABLE(DebugLuauMagicTypes, false) LUAU_FASTINTVARIABLE(LuauTypeInferRecursionLimit, 500) @@ -24,25 +24,30 @@ LUAU_FASTINTVARIABLE(LuauTypeInferTypePackLoopLimit, 5000) LUAU_FASTINTVARIABLE(LuauCheckRecursionLimit, 500) LUAU_FASTFLAG(LuauKnowsTheDataModel3) LUAU_FASTFLAGVARIABLE(LuauEqConstraint, false) +LUAU_FASTFLAGVARIABLE(LuauGroupExpectedType, false) LUAU_FASTFLAGVARIABLE(LuauWeakEqConstraint, false) // Eventually removed as false. LUAU_FASTFLAGVARIABLE(LuauCloneCorrectlyBeforeMutatingTableType, false) LUAU_FASTFLAGVARIABLE(LuauStoreMatchingOverloadFnType, false) LUAU_FASTFLAG(LuauUseCommittingTxnLog) LUAU_FASTFLAGVARIABLE(DebugLuauFreezeDuringUnification, false) LUAU_FASTFLAGVARIABLE(LuauRecursiveTypeParameterRestriction, false) -LUAU_FASTFLAGVARIABLE(LuauIfElseExpressionAnalysisSupport, false) +LUAU_FASTFLAGVARIABLE(LuauIfElseBranchTypeUnion, false) +LUAU_FASTFLAGVARIABLE(LuauIfElseExpectedType2, false) LUAU_FASTFLAGVARIABLE(LuauQuantifyInPlace2, false) LUAU_FASTFLAGVARIABLE(LuauSealExports, false) LUAU_FASTFLAGVARIABLE(LuauSingletonTypes, false) +LUAU_FASTFLAGVARIABLE(LuauTypeAliasDefaults, false) LUAU_FASTFLAGVARIABLE(LuauExpectedTypesOfProperties, false) LUAU_FASTFLAGVARIABLE(LuauErrorRecoveryType, false) LUAU_FASTFLAGVARIABLE(LuauPropertiesGetExpectedType, false) LUAU_FASTFLAGVARIABLE(LuauLValueAsKey, false) LUAU_FASTFLAGVARIABLE(LuauRefiLookupFromIndexExpr, false) +LUAU_FASTFLAGVARIABLE(LuauPerModuleUnificationCache, false) LUAU_FASTFLAGVARIABLE(LuauProperTypeLevels, false) LUAU_FASTFLAGVARIABLE(LuauAscribeCorrectLevelToInferredProperitesOfFreeTables, false) LUAU_FASTFLAGVARIABLE(LuauFixRecursiveMetatableCall, false) LUAU_FASTFLAGVARIABLE(LuauBidirectionalAsExpr, false) +LUAU_FASTFLAGVARIABLE(LuauUnsealedTableLiteral, false) LUAU_FASTFLAGVARIABLE(LuauUpdateFunctionNameBinding, false) namespace Luau @@ -279,6 +284,14 @@ ModulePtr TypeChecker::check(const SourceModule& module, Mode mode, std::optiona GenericError{"Free types leaked into this module's public interface. This is an internal Luau error; please report it."}}); } + if (FFlag::LuauPerModuleUnificationCache) + { + // Clear unifier cache since it's keyed off internal types that get deallocated + // This avoids fake cross-module cache hits and keeps cache size at bay when typechecking large module graphs. + unifierState.cachedUnify.clear(); + unifierState.skipCacheForType.clear(); + } + return std::move(currentModule); } @@ -1213,18 +1226,18 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatTypeAlias& typealias ScopePtr aliasScope = childScope(scope, typealias.location); aliasScope->level = scope->level.incr(); - for (TypeId ty : binding->typeParams) + for (auto param : binding->typeParams) { - auto generic = get(ty); + auto generic = get(param.ty); LUAU_ASSERT(generic); - aliasScope->privateTypeBindings[generic->name] = TypeFun{{}, ty}; + aliasScope->privateTypeBindings[generic->name] = TypeFun{{}, param.ty}; } - for (TypePackId tp : binding->typePackParams) + for (auto param : binding->typePackParams) { - auto generic = get(tp); + auto generic = get(param.tp); LUAU_ASSERT(generic); - aliasScope->privateTypePackBindings[generic->name] = tp; + aliasScope->privateTypePackBindings[generic->name] = param.tp; } TypeId ty = resolveType(aliasScope, *typealias.type); @@ -1233,9 +1246,17 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatTypeAlias& typealias // If the table is already named and we want to rename the type function, we have to bind new alias to a copy if (ttv->name) { + bool sameTys = std::equal(ttv->instantiatedTypeParams.begin(), ttv->instantiatedTypeParams.end(), binding->typeParams.begin(), + binding->typeParams.end(), [](auto&& itp, auto&& tp) { + return itp == tp.ty; + }); + bool sameTps = std::equal(ttv->instantiatedTypePackParams.begin(), ttv->instantiatedTypePackParams.end(), + binding->typePackParams.begin(), binding->typePackParams.end(), [](auto&& itpp, auto&& tpp) { + return itpp == tpp.tp; + }); + // Copy can be skipped if this is an identical alias - if (ttv->name != name || ttv->instantiatedTypeParams != binding->typeParams || - ttv->instantiatedTypePackParams != binding->typePackParams) + if (ttv->name != name || !sameTys || !sameTps) { // This is a shallow clone, original recursive links to self are not updated TableTypeVar clone = TableTypeVar{ttv->props, ttv->indexer, ttv->level, ttv->state}; @@ -1243,8 +1264,12 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatTypeAlias& typealias clone.methodDefinitionLocations = ttv->methodDefinitionLocations; clone.definitionModuleName = ttv->definitionModuleName; clone.name = name; - clone.instantiatedTypeParams = binding->typeParams; - clone.instantiatedTypePackParams = binding->typePackParams; + + for (auto param : binding->typeParams) + clone.instantiatedTypeParams.push_back(param.ty); + + for (auto param : binding->typePackParams) + clone.instantiatedTypePackParams.push_back(param.tp); ty = addType(std::move(clone)); } @@ -1252,8 +1277,14 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatTypeAlias& typealias else { ttv->name = name; - ttv->instantiatedTypeParams = binding->typeParams; - ttv->instantiatedTypePackParams = binding->typePackParams; + + ttv->instantiatedTypeParams.clear(); + for (auto param : binding->typeParams) + ttv->instantiatedTypeParams.push_back(param.ty); + + ttv->instantiatedTypePackParams.clear(); + for (auto param : binding->typePackParams) + ttv->instantiatedTypePackParams.push_back(param.tp); } } else if (auto mtv = getMutable(follow(ty))) @@ -1367,9 +1398,21 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatDeclareFunction& glo auto [generics, genericPacks] = createGenericTypes(funScope, std::nullopt, global, global.generics, global.genericPacks); + std::vector genericTys; + genericTys.reserve(generics.size()); + std::transform(generics.begin(), generics.end(), std::back_inserter(genericTys), [](auto&& el) { + return el.ty; + }); + + std::vector genericTps; + genericTps.reserve(genericPacks.size()); + std::transform(genericPacks.begin(), genericPacks.end(), std::back_inserter(genericTps), [](auto&& el) { + return el.tp; + }); + TypePackId argPack = resolveTypePack(funScope, global.params); TypePackId retPack = resolveTypePack(funScope, global.retTypes); - TypeId fnType = addType(FunctionTypeVar{funScope->level, generics, genericPacks, argPack, retPack}); + TypeId fnType = addType(FunctionTypeVar{funScope->level, std::move(genericTys), std::move(genericTps), argPack, retPack}); FunctionTypeVar* ftv = getMutable(fnType); ftv->argNames.reserve(global.paramNames.size); @@ -1394,7 +1437,7 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExpr& ExprResult result; if (auto a = expr.as()) - result = checkExpr(scope, *a->expr); + result = checkExpr(scope, *a->expr, FFlag::LuauGroupExpectedType ? expectedType : std::nullopt); else if (expr.is()) result = {nilType}; else if (const AstExprConstantBool* bexpr = expr.as()) @@ -1438,21 +1481,7 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExpr& else if (auto a = expr.as()) result = checkExpr(scope, *a); else if (auto a = expr.as()) - { - if (FFlag::LuauIfElseExpressionAnalysisSupport) - { - result = checkExpr(scope, *a); - } - else - { - // Note: When the fast flag is disabled we can't skip the handling of AstExprIfElse - // because we would generate an ICE. We also can't use the default value - // of result, because it will lead to a compiler crash. - // Note: LuauIfElseExpressionBaseSupport can be used to disable parser support - // for if-else expressions which will mean this node type is never created. - result = {anyType}; - } - } + result = checkExpr(scope, *a, FFlag::LuauIfElseExpectedType2 ? expectedType : std::nullopt); else ice("Unhandled AstExpr?"); @@ -1895,7 +1924,7 @@ TypeId TypeChecker::checkExprTable( } } - TableState state = (expr.items.size == 0 || isNonstrictMode()) ? TableState::Unsealed : TableState::Sealed; + TableState state = (expr.items.size == 0 || isNonstrictMode() || FFlag::LuauUnsealedTableLiteral) ? TableState::Unsealed : TableState::Sealed; TableTypeVar table = TableTypeVar{std::move(props), indexer, scope->level, state}; table.definitionModuleName = currentModuleName; return addType(table); @@ -2549,23 +2578,34 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprEr return {errorRecoveryType(scope)}; } -ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprIfElse& expr) +ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprIfElse& expr, std::optional expectedType) { ExprResult result = checkExpr(scope, *expr.condition); ScopePtr trueScope = childScope(scope, expr.trueExpr->location); reportErrors(resolve(result.predicates, trueScope, true)); - ExprResult trueType = checkExpr(trueScope, *expr.trueExpr); + ExprResult trueType = checkExpr(trueScope, *expr.trueExpr, expectedType); ScopePtr falseScope = childScope(scope, expr.falseExpr->location); // Don't report errors for this scope to avoid potentially duplicating errors reported for the first scope. resolve(result.predicates, falseScope, false); - ExprResult falseType = checkExpr(falseScope, *expr.falseExpr); + ExprResult falseType = checkExpr(falseScope, *expr.falseExpr, expectedType); - unify(falseType.type, trueType.type, expr.location); + if (FFlag::LuauIfElseBranchTypeUnion) + { + if (falseType.type == trueType.type) + return {trueType.type}; - // TODO: normalize(UnionTypeVar{{trueType, falseType}}) - // For now both trueType and falseType must be the same type. - return {trueType.type}; + std::vector types = reduceUnion({trueType.type, falseType.type}); + return {types.size() == 1 ? types[0] : addType(UnionTypeVar{std::move(types)})}; + } + else + { + unify(falseType.type, trueType.type, expr.location); + + // TODO: normalize(UnionTypeVar{{trueType, falseType}}) + // For now both trueType and falseType must be the same type. + return {trueType.type}; + } } TypeId TypeChecker::checkLValue(const ScopePtr& scope, const AstExpr& expr) @@ -3032,7 +3072,20 @@ std::pair TypeChecker::checkFunctionSignature( defn.varargLocation = expr.vararg ? std::make_optional(expr.varargLocation) : std::nullopt; defn.originalNameLocation = originalName.value_or(Location(expr.location.begin, 0)); - TypeId funTy = addType(FunctionTypeVar(funScope->level, generics, genericPacks, argPack, retPack, std::move(defn), bool(expr.self))); + std::vector genericTys; + genericTys.reserve(generics.size()); + std::transform(generics.begin(), generics.end(), std::back_inserter(genericTys), [](auto&& el) { + return el.ty; + }); + + std::vector genericTps; + genericTps.reserve(genericPacks.size()); + std::transform(genericPacks.begin(), genericPacks.end(), std::back_inserter(genericTps), [](auto&& el) { + return el.tp; + }); + + TypeId funTy = + addType(FunctionTypeVar(funScope->level, std::move(genericTys), std::move(genericTps), argPack, retPack, std::move(defn), bool(expr.self))); FunctionTypeVar* ftv = getMutable(funTy); @@ -4848,11 +4901,38 @@ TypeId TypeChecker::resolveType(const ScopePtr& scope, const AstType& annotation if (lit->parameters.size == 0 && tf->typeParams.empty() && tf->typePackParams.empty()) return tf->type; - if (!lit->hasParameterList && !tf->typePackParams.empty()) + bool hasDefaultTypes = false; + bool hasDefaultPacks = false; + bool parameterCountErrorReported = false; + + if (FFlag::LuauTypeAliasDefaults) { - reportError(TypeError{annotation.location, GenericError{"Type parameter list is required"}}); - if (!FFlag::LuauErrorRecoveryType) - return errorRecoveryType(scope); + hasDefaultTypes = std::any_of(tf->typeParams.begin(), tf->typeParams.end(), [](auto&& el) { + return el.defaultValue.has_value(); + }); + hasDefaultPacks = std::any_of(tf->typePackParams.begin(), tf->typePackParams.end(), [](auto&& el) { + return el.defaultValue.has_value(); + }); + + if (!lit->hasParameterList) + { + if ((!tf->typeParams.empty() && !hasDefaultTypes) || (!tf->typePackParams.empty() && !hasDefaultPacks)) + { + reportError(TypeError{annotation.location, GenericError{"Type parameter list is required"}}); + parameterCountErrorReported = true; + if (!FFlag::LuauErrorRecoveryType) + return errorRecoveryType(scope); + } + } + } + else + { + if (!lit->hasParameterList && !tf->typePackParams.empty()) + { + reportError(TypeError{annotation.location, GenericError{"Type parameter list is required"}}); + if (!FFlag::LuauErrorRecoveryType) + return errorRecoveryType(scope); + } } std::vector typeParams; @@ -4892,14 +4972,89 @@ TypeId TypeChecker::resolveType(const ScopePtr& scope, const AstType& annotation if (typePackParams.empty() && !extraTypes.empty()) typePackParams.push_back(addTypePack(extraTypes)); + if (FFlag::LuauTypeAliasDefaults) + { + size_t typesProvided = typeParams.size(); + size_t typesRequired = tf->typeParams.size(); + + size_t packsProvided = typePackParams.size(); + size_t packsRequired = tf->typePackParams.size(); + + bool notEnoughParameters = + (typesProvided < typesRequired && packsProvided == 0) || (typesProvided == typesRequired && packsProvided < packsRequired); + bool hasDefaultParameters = hasDefaultTypes || hasDefaultPacks; + + // Add default type and type pack parameters if that's required and it's possible + if (notEnoughParameters && hasDefaultParameters) + { + // 'applyTypeFunction' is used to substitute default types that reference previous generic types + applyTypeFunction.typeArguments.clear(); + applyTypeFunction.typePackArguments.clear(); + applyTypeFunction.currentModule = currentModule; + applyTypeFunction.level = scope->level; + applyTypeFunction.encounteredForwardedType = false; + + for (size_t i = 0; i < typesProvided; ++i) + applyTypeFunction.typeArguments[tf->typeParams[i].ty] = typeParams[i]; + + if (typesProvided < typesRequired) + { + for (size_t i = typesProvided; i < typesRequired; ++i) + { + TypeId defaultTy = tf->typeParams[i].defaultValue.value_or(nullptr); + + if (!defaultTy) + break; + + std::optional maybeInstantiated = applyTypeFunction.substitute(defaultTy); + + if (!maybeInstantiated.has_value()) + { + reportError(annotation.location, UnificationTooComplex{}); + maybeInstantiated = errorRecoveryType(scope); + } + + applyTypeFunction.typeArguments[tf->typeParams[i].ty] = *maybeInstantiated; + typeParams.push_back(*maybeInstantiated); + } + } + + for (size_t i = 0; i < packsProvided; ++i) + applyTypeFunction.typePackArguments[tf->typePackParams[i].tp] = typePackParams[i]; + + if (packsProvided < packsRequired) + { + for (size_t i = packsProvided; i < packsRequired; ++i) + { + TypePackId defaultTp = tf->typePackParams[i].defaultValue.value_or(nullptr); + + if (!defaultTp) + break; + + std::optional maybeInstantiated = applyTypeFunction.substitute(defaultTp); + + if (!maybeInstantiated.has_value()) + { + reportError(annotation.location, UnificationTooComplex{}); + maybeInstantiated = errorRecoveryTypePack(scope); + } + + applyTypeFunction.typePackArguments[tf->typePackParams[i].tp] = *maybeInstantiated; + typePackParams.push_back(*maybeInstantiated); + } + } + } + } + // If we didn't combine regular types into a type pack and we're still one type pack short, provide an empty type pack if (extraTypes.empty() && typePackParams.size() + 1 == tf->typePackParams.size()) typePackParams.push_back(addTypePack({})); if (typeParams.size() != tf->typeParams.size() || typePackParams.size() != tf->typePackParams.size()) { - reportError( - TypeError{annotation.location, IncorrectGenericParameterCount{lit->name.value, *tf, typeParams.size(), typePackParams.size()}}); + if (!parameterCountErrorReported) + reportError( + TypeError{annotation.location, IncorrectGenericParameterCount{lit->name.value, *tf, typeParams.size(), typePackParams.size()}}); if (FFlag::LuauErrorRecoveryType) { @@ -4913,11 +5068,20 @@ TypeId TypeChecker::resolveType(const ScopePtr& scope, const AstType& annotation return errorRecoveryType(scope); } - if (FFlag::LuauRecursiveTypeParameterRestriction && typeParams == tf->typeParams && typePackParams == tf->typePackParams) + if (FFlag::LuauRecursiveTypeParameterRestriction) { + bool sameTys = std::equal(typeParams.begin(), typeParams.end(), tf->typeParams.begin(), tf->typeParams.end(), [](auto&& itp, auto&& tp) { + return itp == tp.ty; + }); + bool sameTps = std::equal( + typePackParams.begin(), typePackParams.end(), tf->typePackParams.begin(), tf->typePackParams.end(), [](auto&& itpp, auto&& tpp) { + return itpp == tpp.tp; + }); + // If the generic parameters and the type arguments are the same, we are about to // perform an identity substitution, which we can just short-circuit. - return tf->type; + if (sameTys && sameTps) + return tf->type; } return instantiateTypeFun(scope, *tf, typeParams, typePackParams, annotation.location); @@ -4948,7 +5112,19 @@ TypeId TypeChecker::resolveType(const ScopePtr& scope, const AstType& annotation TypePackId argTypes = resolveTypePack(funcScope, func->argTypes); TypePackId retTypes = resolveTypePack(funcScope, func->returnTypes); - TypeId fnType = addType(FunctionTypeVar{funcScope->level, std::move(generics), std::move(genericPacks), argTypes, retTypes}); + std::vector genericTys; + genericTys.reserve(generics.size()); + std::transform(generics.begin(), generics.end(), std::back_inserter(genericTys), [](auto&& el) { + return el.ty; + }); + + std::vector genericTps; + genericTps.reserve(genericPacks.size()); + std::transform(genericPacks.begin(), genericPacks.end(), std::back_inserter(genericTps), [](auto&& el) { + return el.tp; + }); + + TypeId fnType = addType(FunctionTypeVar{funcScope->level, std::move(genericTys), std::move(genericTps), argTypes, retTypes}); FunctionTypeVar* ftv = getMutable(fnType); @@ -5137,11 +5313,11 @@ TypeId TypeChecker::instantiateTypeFun(const ScopePtr& scope, const TypeFun& tf, applyTypeFunction.typeArguments.clear(); for (size_t i = 0; i < tf.typeParams.size(); ++i) - applyTypeFunction.typeArguments[tf.typeParams[i]] = typeParams[i]; + applyTypeFunction.typeArguments[tf.typeParams[i].ty] = typeParams[i]; applyTypeFunction.typePackArguments.clear(); for (size_t i = 0; i < tf.typePackParams.size(); ++i) - applyTypeFunction.typePackArguments[tf.typePackParams[i]] = typePackParams[i]; + applyTypeFunction.typePackArguments[tf.typePackParams[i].tp] = typePackParams[i]; applyTypeFunction.currentModule = currentModule; applyTypeFunction.level = scope->level; @@ -5213,17 +5389,23 @@ TypeId TypeChecker::instantiateTypeFun(const ScopePtr& scope, const TypeFun& tf, return instantiated; } -std::pair, std::vector> TypeChecker::createGenericTypes(const ScopePtr& scope, std::optional levelOpt, - const AstNode& node, const AstArray& genericNames, const AstArray& genericPackNames) +GenericTypeDefinitions TypeChecker::createGenericTypes(const ScopePtr& scope, std::optional levelOpt, const AstNode& node, + const AstArray& genericNames, const AstArray& genericPackNames) { LUAU_ASSERT(scope->parent); const TypeLevel level = (FFlag::LuauQuantifyInPlace2 && levelOpt) ? *levelOpt : scope->level; - std::vector generics; - for (const AstName& generic : genericNames) + std::vector generics; + + for (const AstGenericType& generic : genericNames) { - Name n = generic.value; + std::optional defaultValue; + + if (FFlag::LuauTypeAliasDefaults && generic.defaultValue) + defaultValue = resolveType(scope, *generic.defaultValue); + + Name n = generic.name.value; // These generics are the only thing that will ever be added to scope, so we can be certain that // a collision can only occur when two generic typevars have the same name. @@ -5246,14 +5428,20 @@ std::pair, std::vector> TypeChecker::createGener g = addType(Unifiable::Generic{level, n}); } - generics.push_back(g); + generics.push_back({g, defaultValue}); scope->privateTypeBindings[n] = TypeFun{{}, g}; } - std::vector genericPacks; - for (const AstName& genericPack : genericPackNames) + std::vector genericPacks; + + for (const AstGenericTypePack& genericPack : genericPackNames) { - Name n = genericPack.value; + std::optional defaultValue; + + if (FFlag::LuauTypeAliasDefaults && genericPack.defaultValue) + defaultValue = resolveTypePack(scope, *genericPack.defaultValue); + + Name n = genericPack.name.value; // These generics are the only thing that will ever be added to scope, so we can be certain that // a collision can only occur when two generic typevars have the same name. @@ -5276,7 +5464,7 @@ std::pair, std::vector> TypeChecker::createGener g = addTypePack(TypePackVar{Unifiable::Generic{level, n}}); } - genericPacks.push_back(g); + genericPacks.push_back({g, defaultValue}); scope->privateTypePackBindings[n] = g; } diff --git a/Analysis/src/TypeVar.cpp b/Analysis/src/TypeVar.cpp index 4cab79c8..ac2b2541 100644 --- a/Analysis/src/TypeVar.cpp +++ b/Analysis/src/TypeVar.cpp @@ -19,6 +19,7 @@ LUAU_FASTINTVARIABLE(LuauTypeMaximumStringifierLength, 500) LUAU_FASTINTVARIABLE(LuauTableTypeMaximumStringifierLength, 0) +LUAU_FASTFLAGVARIABLE(LuauMetatableAreEqualRecursion, false) LUAU_FASTFLAGVARIABLE(LuauRefactorTagging, false) LUAU_FASTFLAG(LuauErrorRecoveryType) LUAU_FASTFLAG(DebugLuauFreezeArena) @@ -453,6 +454,9 @@ bool areEqual(SeenSet& seen, const TableTypeVar& lhs, const TableTypeVar& rhs) static bool areEqual(SeenSet& seen, const MetatableTypeVar& lhs, const MetatableTypeVar& rhs) { + if (FFlag::LuauMetatableAreEqualRecursion && areSeen(seen, &lhs, &rhs)) + return true; + return areEqual(seen, *lhs.table, *rhs.table) && areEqual(seen, *lhs.metatable, *rhs.metatable); } diff --git a/Analysis/src/Unifier.cpp b/Analysis/src/Unifier.cpp index 393a84a7..6873c657 100644 --- a/Analysis/src/Unifier.cpp +++ b/Analysis/src/Unifier.cpp @@ -22,6 +22,7 @@ LUAU_FASTFLAGVARIABLE(LuauOccursCheckOkWithRecursiveFunctions, false) LUAU_FASTFLAG(LuauSingletonTypes) LUAU_FASTFLAG(LuauErrorRecoveryType); LUAU_FASTFLAG(LuauProperTypeLevels); +LUAU_FASTFLAGVARIABLE(LuauUnifyPackTails, false) LUAU_FASTFLAGVARIABLE(LuauExtendedUnionMismatchError, false) LUAU_FASTFLAGVARIABLE(LuauExtendedFunctionMismatchError, false) @@ -1170,9 +1171,15 @@ void Unifier::tryUnify_(TypePackId subTp, TypePackId superTp, bool isFunctionCal // If both are at the end, we're done if (!superIter.good() && !subIter.good()) { + if (FFlag::LuauUnifyPackTails && subTpv->tail && superTpv->tail) + { + tryUnify_(*subTpv->tail, *superTpv->tail); + break; + } + const bool lFreeTail = superTpv->tail && log.getMutable(log.follow(*superTpv->tail)) != nullptr; const bool rFreeTail = subTpv->tail && log.getMutable(log.follow(*subTpv->tail)) != nullptr; - if (lFreeTail && rFreeTail) + if (!FFlag::LuauUnifyPackTails && lFreeTail && rFreeTail) tryUnify_(*subTpv->tail, *superTpv->tail); else if (lFreeTail) tryUnify_(emptyTp, *superTpv->tail); @@ -1370,9 +1377,15 @@ void Unifier::tryUnify_(TypePackId subTp, TypePackId superTp, bool isFunctionCal // If both are at the end, we're done if (!superIter.good() && !subIter.good()) { + if (FFlag::LuauUnifyPackTails && subTpv->tail && superTpv->tail) + { + tryUnify_(*subTpv->tail, *superTpv->tail); + break; + } + const bool lFreeTail = superTpv->tail && get(follow(*superTpv->tail)) != nullptr; const bool rFreeTail = subTpv->tail && get(follow(*subTpv->tail)) != nullptr; - if (lFreeTail && rFreeTail) + if (!FFlag::LuauUnifyPackTails && lFreeTail && rFreeTail) tryUnify_(*subTpv->tail, *superTpv->tail); else if (lFreeTail) tryUnify_(emptyTp, *superTpv->tail); diff --git a/Ast/include/Luau/Ast.h b/Ast/include/Luau/Ast.h index 5b4bfa03..573850a5 100644 --- a/Ast/include/Luau/Ast.h +++ b/Ast/include/Luau/Ast.h @@ -334,6 +334,20 @@ struct AstTypeList using AstArgumentName = std::pair; // TODO: remove and replace when we get a common struct for this pair instead of AstName +struct AstGenericType +{ + AstName name; + Location location; + AstType* defaultValue = nullptr; +}; + +struct AstGenericTypePack +{ + AstName name; + Location location; + AstTypePack* defaultValue = nullptr; +}; + extern int gAstRttiIndex; template @@ -569,15 +583,15 @@ class AstExprFunction : public AstExpr public: LUAU_RTTI(AstExprFunction) - AstExprFunction(const Location& location, const AstArray& generics, const AstArray& genericPacks, AstLocal* self, - const AstArray& args, std::optional vararg, AstStatBlock* body, size_t functionDepth, const AstName& debugname, - std::optional returnAnnotation = {}, AstTypePack* varargAnnotation = nullptr, bool hasEnd = false, + AstExprFunction(const Location& location, const AstArray& generics, const AstArray& genericPacks, + AstLocal* self, const AstArray& args, std::optional vararg, AstStatBlock* body, size_t functionDepth, + const AstName& debugname, std::optional returnAnnotation = {}, AstTypePack* varargAnnotation = nullptr, bool hasEnd = false, std::optional argLocation = std::nullopt); void visit(AstVisitor* visitor) override; - AstArray generics; - AstArray genericPacks; + AstArray generics; + AstArray genericPacks; AstLocal* self; AstArray args; bool hasReturnAnnotation; @@ -942,14 +956,14 @@ class AstStatTypeAlias : public AstStat public: LUAU_RTTI(AstStatTypeAlias) - AstStatTypeAlias(const Location& location, const AstName& name, const AstArray& generics, const AstArray& genericPacks, - AstType* type, bool exported); + AstStatTypeAlias(const Location& location, const AstName& name, const AstArray& generics, + const AstArray& genericPacks, AstType* type, bool exported); void visit(AstVisitor* visitor) override; AstName name; - AstArray generics; - AstArray genericPacks; + AstArray generics; + AstArray genericPacks; AstType* type; bool exported; }; @@ -972,14 +986,15 @@ class AstStatDeclareFunction : public AstStat public: LUAU_RTTI(AstStatDeclareFunction) - AstStatDeclareFunction(const Location& location, const AstName& name, const AstArray& generics, const AstArray& genericPacks, - const AstTypeList& params, const AstArray& paramNames, const AstTypeList& retTypes); + AstStatDeclareFunction(const Location& location, const AstName& name, const AstArray& generics, + const AstArray& genericPacks, const AstTypeList& params, const AstArray& paramNames, + const AstTypeList& retTypes); void visit(AstVisitor* visitor) override; AstName name; - AstArray generics; - AstArray genericPacks; + AstArray generics; + AstArray genericPacks; AstTypeList params; AstArray paramNames; AstTypeList retTypes; @@ -1077,13 +1092,13 @@ class AstTypeFunction : public AstType public: LUAU_RTTI(AstTypeFunction) - AstTypeFunction(const Location& location, const AstArray& generics, const AstArray& genericPacks, const AstTypeList& argTypes, - const AstArray>& argNames, const AstTypeList& returnTypes); + AstTypeFunction(const Location& location, const AstArray& generics, const AstArray& genericPacks, + const AstTypeList& argTypes, const AstArray>& argNames, const AstTypeList& returnTypes); void visit(AstVisitor* visitor) override; - AstArray generics; - AstArray genericPacks; + AstArray generics; + AstArray genericPacks; AstTypeList argTypes; AstArray> argNames; AstTypeList returnTypes; diff --git a/Ast/include/Luau/Parser.h b/Ast/include/Luau/Parser.h index 87ebc48b..40ecdcdd 100644 --- a/Ast/include/Luau/Parser.h +++ b/Ast/include/Luau/Parser.h @@ -219,7 +219,7 @@ private: AstTableIndexer* parseTableIndexerAnnotation(); AstTypeOrPack parseFunctionTypeAnnotation(bool allowPack); - AstType* parseFunctionTypeAnnotationTail(const Lexeme& begin, AstArray generics, AstArray genericPacks, + AstType* parseFunctionTypeAnnotationTail(const Lexeme& begin, AstArray generics, AstArray genericPacks, AstArray& params, AstArray>& paramNames, AstTypePack* varargAnnotation); AstType* parseTableTypeAnnotation(); @@ -281,7 +281,7 @@ private: Name parseIndexName(const char* context, const Position& previous); // `<' namelist `>' - std::pair, AstArray> parseGenericTypeList(); + std::pair, AstArray> parseGenericTypeList(bool withDefaultValues); // `<' typeAnnotation[, ...] `>' AstArray parseTypeParams(); @@ -418,6 +418,8 @@ private: std::vector scratchDeclaredClassProps; std::vector scratchItem; std::vector scratchArgName; + std::vector scratchGenericTypes; + std::vector scratchGenericTypePacks; std::vector> scratchOptArgName; std::string scratchData; }; diff --git a/Ast/src/Ast.cpp b/Ast/src/Ast.cpp index e709894d..9b5bc0c7 100644 --- a/Ast/src/Ast.cpp +++ b/Ast/src/Ast.cpp @@ -158,9 +158,10 @@ void AstExprIndexExpr::visit(AstVisitor* visitor) } } -AstExprFunction::AstExprFunction(const Location& location, const AstArray& generics, const AstArray& genericPacks, AstLocal* self, - const AstArray& args, std::optional vararg, AstStatBlock* body, size_t functionDepth, const AstName& debugname, - std::optional returnAnnotation, AstTypePack* varargAnnotation, bool hasEnd, std::optional argLocation) +AstExprFunction::AstExprFunction(const Location& location, const AstArray& generics, const AstArray& genericPacks, + AstLocal* self, const AstArray& args, std::optional vararg, AstStatBlock* body, size_t functionDepth, + const AstName& debugname, std::optional returnAnnotation, AstTypePack* varargAnnotation, bool hasEnd, + std::optional argLocation) : AstExpr(ClassIndex(), location) , generics(generics) , genericPacks(genericPacks) @@ -641,8 +642,8 @@ void AstStatLocalFunction::visit(AstVisitor* visitor) func->visit(visitor); } -AstStatTypeAlias::AstStatTypeAlias(const Location& location, const AstName& name, const AstArray& generics, - const AstArray& genericPacks, AstType* type, bool exported) +AstStatTypeAlias::AstStatTypeAlias(const Location& location, const AstName& name, const AstArray& generics, + const AstArray& genericPacks, AstType* type, bool exported) : AstStat(ClassIndex(), location) , name(name) , generics(generics) @@ -655,7 +656,21 @@ AstStatTypeAlias::AstStatTypeAlias(const Location& location, const AstName& name void AstStatTypeAlias::visit(AstVisitor* visitor) { if (visitor->visit(this)) + { + for (const AstGenericType& el : generics) + { + if (el.defaultValue) + el.defaultValue->visit(visitor); + } + + for (const AstGenericTypePack& el : genericPacks) + { + if (el.defaultValue) + el.defaultValue->visit(visitor); + } + type->visit(visitor); + } } AstStatDeclareGlobal::AstStatDeclareGlobal(const Location& location, const AstName& name, AstType* type) @@ -671,8 +686,9 @@ void AstStatDeclareGlobal::visit(AstVisitor* visitor) type->visit(visitor); } -AstStatDeclareFunction::AstStatDeclareFunction(const Location& location, const AstName& name, const AstArray& generics, - const AstArray& genericPacks, const AstTypeList& params, const AstArray& paramNames, const AstTypeList& retTypes) +AstStatDeclareFunction::AstStatDeclareFunction(const Location& location, const AstName& name, const AstArray& generics, + const AstArray& genericPacks, const AstTypeList& params, const AstArray& paramNames, + const AstTypeList& retTypes) : AstStat(ClassIndex(), location) , name(name) , generics(generics) @@ -778,7 +794,7 @@ void AstTypeTable::visit(AstVisitor* visitor) } } -AstTypeFunction::AstTypeFunction(const Location& location, const AstArray& generics, const AstArray& genericPacks, +AstTypeFunction::AstTypeFunction(const Location& location, const AstArray& generics, const AstArray& genericPacks, const AstTypeList& argTypes, const AstArray>& argNames, const AstTypeList& returnTypes) : AstType(ClassIndex(), location) , generics(generics) diff --git a/Ast/src/Parser.cpp b/Ast/src/Parser.cpp index 72f61649..77787cb1 100644 --- a/Ast/src/Parser.cpp +++ b/Ast/src/Parser.cpp @@ -10,10 +10,10 @@ // See docs/SyntaxChanges.md for an explanation. LUAU_FASTINTVARIABLE(LuauRecursionLimit, 1000) LUAU_FASTINTVARIABLE(LuauParseErrorLimit, 100) -LUAU_FASTFLAGVARIABLE(LuauIfElseExpressionBaseSupport, false) -LUAU_FASTFLAGVARIABLE(LuauIfStatementRecursionGuard, false) LUAU_FASTFLAGVARIABLE(LuauFixAmbiguousErrorRecoveryInAssign, false) LUAU_FASTFLAGVARIABLE(LuauParseSingletonTypes, false) +LUAU_FASTFLAGVARIABLE(LuauParseTypeAliasDefaults, false) +LUAU_FASTFLAGVARIABLE(LuauParseRecoverTypePackEllipsis, false) namespace Luau { @@ -394,23 +394,13 @@ AstStat* Parser::parseIf() if (lexer.current().type == Lexeme::ReservedElseif) { - if (FFlag::LuauIfStatementRecursionGuard) - { - unsigned int recursionCounterOld = recursionCounter; - incrementRecursionCounter("elseif"); - elseLocation = lexer.current().location; - elsebody = parseIf(); - end = elsebody->location; - hasEnd = elsebody->as()->hasEnd; - recursionCounter = recursionCounterOld; - } - else - { - elseLocation = lexer.current().location; - elsebody = parseIf(); - end = elsebody->location; - hasEnd = elsebody->as()->hasEnd; - } + unsigned int recursionCounterOld = recursionCounter; + incrementRecursionCounter("elseif"); + elseLocation = lexer.current().location; + elsebody = parseIf(); + end = elsebody->location; + hasEnd = elsebody->as()->hasEnd; + recursionCounter = recursionCounterOld; } else { @@ -772,7 +762,7 @@ AstStat* Parser::parseTypeAlias(const Location& start, bool exported) if (!name) name = Name(nameError, lexer.current().location); - auto [generics, genericPacks] = parseGenericTypeList(); + auto [generics, genericPacks] = parseGenericTypeList(/* withDefaultValues= */ FFlag::LuauParseTypeAliasDefaults); expectAndConsume('=', "type alias"); @@ -788,8 +778,8 @@ AstDeclaredClassProp Parser::parseDeclaredClassMethod() Name fnName = parseName("function name"); // TODO: generic method declarations CLI-39909 - AstArray generics; - AstArray genericPacks; + AstArray generics; + AstArray genericPacks; generics.size = 0; generics.data = nullptr; genericPacks.size = 0; @@ -849,7 +839,7 @@ AstStat* Parser::parseDeclaration(const Location& start) nextLexeme(); Name globalName = parseName("global function name"); - auto [generics, genericPacks] = parseGenericTypeList(); + auto [generics, genericPacks] = parseGenericTypeList(/* withDefaultValues= */ false); Lexeme matchParen = lexer.current(); @@ -991,7 +981,7 @@ std::pair Parser::parseFunctionBody( { Location start = matchFunction.location; - auto [generics, genericPacks] = parseGenericTypeList(); + auto [generics, genericPacks] = parseGenericTypeList(/* withDefaultValues= */ false); Lexeme matchParen = lexer.current(); expectAndConsume('(', "function"); @@ -1228,8 +1218,8 @@ std::pair Parser::parseReturnTypeAnnotation() return {location, AstTypeList{copy(result), varargAnnotation}}; } - AstArray generics{nullptr, 0}; - AstArray genericPacks{nullptr, 0}; + AstArray generics{nullptr, 0}; + AstArray genericPacks{nullptr, 0}; AstArray types = copy(result); AstArray> names = copy(resultNames); @@ -1363,7 +1353,7 @@ AstTypeOrPack Parser::parseFunctionTypeAnnotation(bool allowPack) Lexeme begin = lexer.current(); - auto [generics, genericPacks] = parseGenericTypeList(); + auto [generics, genericPacks] = parseGenericTypeList(/* withDefaultValues= */ false); Lexeme parameterStart = lexer.current(); @@ -1401,7 +1391,7 @@ AstTypeOrPack Parser::parseFunctionTypeAnnotation(bool allowPack) return {parseFunctionTypeAnnotationTail(begin, generics, genericPacks, paramTypes, paramNames, varargAnnotation), {}}; } -AstType* Parser::parseFunctionTypeAnnotationTail(const Lexeme& begin, AstArray generics, AstArray genericPacks, +AstType* Parser::parseFunctionTypeAnnotationTail(const Lexeme& begin, AstArray generics, AstArray genericPacks, AstArray& params, AstArray>& paramNames, AstTypePack* varargAnnotation) { @@ -1448,7 +1438,7 @@ AstType* Parser::parseTypeAnnotation(TempVector& parts, const Location if (c == '|') { nextLexeme(); - parts.push_back(parseSimpleTypeAnnotation(false).type); + parts.push_back(parseSimpleTypeAnnotation(/* allowPack= */ false).type); isUnion = true; } else if (c == '?') @@ -1461,7 +1451,7 @@ AstType* Parser::parseTypeAnnotation(TempVector& parts, const Location else if (c == '&') { nextLexeme(); - parts.push_back(parseSimpleTypeAnnotation(false).type); + parts.push_back(parseSimpleTypeAnnotation(/* allowPack= */ false).type); isIntersection = true; } else @@ -1498,7 +1488,7 @@ AstTypeOrPack Parser::parseTypeOrPackAnnotation() TempVector parts(scratchAnnotation); - auto [type, typePack] = parseSimpleTypeAnnotation(true); + auto [type, typePack] = parseSimpleTypeAnnotation(/* allowPack= */ true); if (typePack) { @@ -1521,7 +1511,7 @@ AstType* Parser::parseTypeAnnotation() Location begin = lexer.current().location; TempVector parts(scratchAnnotation); - parts.push_back(parseSimpleTypeAnnotation(false).type); + parts.push_back(parseSimpleTypeAnnotation(/* allowPack= */ false).type); recursionCounter = oldRecursionCount; @@ -2121,7 +2111,7 @@ AstExpr* Parser::parseSimpleExpr() { return parseTableConstructor(); } - else if (FFlag::LuauIfElseExpressionBaseSupport && lexer.current().type == Lexeme::ReservedIf) + else if (lexer.current().type == Lexeme::ReservedIf) { return parseIfElseExpr(); } @@ -2341,10 +2331,10 @@ Parser::Name Parser::parseIndexName(const char* context, const Position& previou return Name(nameError, location); } -std::pair, AstArray> Parser::parseGenericTypeList() +std::pair, AstArray> Parser::parseGenericTypeList(bool withDefaultValues) { - TempVector names{scratchName}; - TempVector namePacks{scratchPackName}; + TempVector names{scratchGenericTypes}; + TempVector namePacks{scratchGenericTypePacks}; if (lexer.current().type == '<') { @@ -2352,21 +2342,73 @@ std::pair, AstArray> Parser::parseGenericTypeList() nextLexeme(); bool seenPack = false; + bool seenDefault = false; + while (true) { + Location nameLocation = lexer.current().location; AstName name = parseName().name; - if (lexer.current().type == Lexeme::Dot3) + if (lexer.current().type == Lexeme::Dot3 || (FFlag::LuauParseRecoverTypePackEllipsis && seenPack)) { seenPack = true; - nextLexeme(); - namePacks.push_back(name); + + if (FFlag::LuauParseRecoverTypePackEllipsis && lexer.current().type != Lexeme::Dot3) + report(lexer.current().location, "Generic types come before generic type packs"); + else + nextLexeme(); + + if (withDefaultValues && lexer.current().type == '=') + { + seenDefault = true; + nextLexeme(); + + Lexeme packBegin = lexer.current(); + + if (shouldParseTypePackAnnotation(lexer)) + { + auto typePack = parseTypePackAnnotation(); + + namePacks.push_back({name, nameLocation, typePack}); + } + else if (lexer.current().type == '(') + { + auto [type, typePack] = parseTypeOrPackAnnotation(); + + if (type) + report(Location(packBegin.location.begin, lexer.previousLocation().end), "Expected type pack after '=', got type"); + + namePacks.push_back({name, nameLocation, typePack}); + } + } + else + { + if (seenDefault) + report(lexer.current().location, "Expected default type pack after type pack name"); + + namePacks.push_back({name, nameLocation, nullptr}); + } } else { - if (seenPack) + if (!FFlag::LuauParseRecoverTypePackEllipsis && seenPack) report(lexer.current().location, "Generic types come before generic type packs"); - names.push_back(name); + if (withDefaultValues && lexer.current().type == '=') + { + seenDefault = true; + nextLexeme(); + + AstType* defaultType = parseTypeAnnotation(); + + names.push_back({name, nameLocation, defaultType}); + } + else + { + if (seenDefault) + report(lexer.current().location, "Expected default type after type name"); + + names.push_back({name, nameLocation, nullptr}); + } } if (lexer.current().type == ',') @@ -2378,8 +2420,8 @@ std::pair, AstArray> Parser::parseGenericTypeList() expectMatchAndConsume('>', begin); } - AstArray generics = copy(names); - AstArray genericPacks = copy(namePacks); + AstArray generics = copy(names); + AstArray genericPacks = copy(namePacks); return {generics, genericPacks}; } diff --git a/CLI/Analyze.cpp b/CLI/Analyze.cpp index 1375f414..bf04281c 100644 --- a/CLI/Analyze.cpp +++ b/CLI/Analyze.cpp @@ -8,6 +8,8 @@ #include "FileUtils.h" +LUAU_FASTFLAG(DebugLuauTimeTracing) + enum class ReportFormat { Default, @@ -105,6 +107,7 @@ static void displayHelp(const char* argv0) printf("Available options:\n"); printf(" --formatter=plain: report analysis errors in Luacheck-compatible format\n"); printf(" --formatter=gnu: report analysis errors in GNU-compatible format\n"); + printf(" --timetrace: record compiler time tracing information into trace.json\n"); } static int assertionHandler(const char* expr, const char* file, int line, const char* function) @@ -213,8 +216,18 @@ int main(int argc, char** argv) format = ReportFormat::Gnu; else if (strcmp(argv[i], "--annotate") == 0) annotate = true; + else if (strcmp(argv[i], "--timetrace") == 0) + FFlag::DebugLuauTimeTracing.value = true; } +#if !defined(LUAU_ENABLE_TIME_TRACE) + if (FFlag::DebugLuauTimeTracing) + { + printf("To run with --timetrace, Luau has to be built with LUAU_ENABLE_TIME_TRACE enabled\n"); + return 1; + } +#endif + Luau::FrontendOptions frontendOptions; frontendOptions.retainFullTypeGraphs = annotate; @@ -240,5 +253,8 @@ int main(int argc, char** argv) fprintf(stderr, "%s: %s\n", pair.first.c_str(), pair.second.c_str()); } - return (format == ReportFormat::Luacheck) ? 0 : failed; + if (format == ReportFormat::Luacheck) + return 0; + else + return failed ? 1 : 0; } diff --git a/CLI/Repl.cpp b/CLI/Repl.cpp index 35fbf309..cf958939 100644 --- a/CLI/Repl.cpp +++ b/CLI/Repl.cpp @@ -19,6 +19,16 @@ #include #endif +LUAU_FASTFLAG(DebugLuauTimeTracing) + +enum class CliMode +{ + Unknown, + Repl, + Compile, + RunSourceFiles +}; + enum class CompileFormat { Text, @@ -485,8 +495,10 @@ static void displayHelp(const char* argv0) printf(" --compile[=format]: compile input files and output resulting formatted bytecode (binary or text)\n"); printf("\n"); printf("Available options:\n"); + printf(" -h, --help: Display this usage message.\n"); printf(" --profile[=N]: profile the code using N Hz sampling (default 10000) and output results to profile.out\n"); printf(" --coverage: collect code coverage while running the code and output results to coverage.out\n"); + printf(" --timetrace: record compiler time tracing information into trace.json\n"); } static int assertionHandler(const char* expr, const char* file, int line, const char* function) @@ -503,71 +515,112 @@ int main(int argc, char** argv) if (strncmp(flag->name, "Luau", 4) == 0) flag->value = true; - if (argc == 1) - { - runRepl(); - return 0; - } - - if (argc >= 2 && strcmp(argv[1], "--help") == 0) - { - displayHelp(argv[0]); - return 0; - } - + CliMode mode = CliMode::Unknown; + CompileFormat compileFormat{}; + int profile = 0; + bool coverage = false; + // Set the mode if the user has explicitly specified one. + int argStart = 1; if (argc >= 2 && strncmp(argv[1], "--compile", strlen("--compile")) == 0) { - CompileFormat format = CompileFormat::Text; + argStart++; + mode = CliMode::Compile; + if (strcmp(argv[1], "--compile") == 0) + { + compileFormat = CompileFormat::Text; + } + else if (strcmp(argv[1], "--compile=binary") == 0) + { + compileFormat = CompileFormat::Binary; + } + else if (strcmp(argv[1], "--compile=text") == 0) + { + compileFormat = CompileFormat::Text; + } + else + { + fprintf(stdout, "Error: Unrecognized value for '--compile' specified.\n"); + return -1; + } + } - if (strcmp(argv[1], "--compile=binary") == 0) - format = CompileFormat::Binary; + for (int i = argStart; i < argc; i++) + { + if (strcmp(argv[i], "-h") == 0 || strcmp(argv[i], "--help") == 0) + { + displayHelp(argv[0]); + return 0; + } + else if (strcmp(argv[i], "--profile") == 0) + { + profile = 10000; // default to 10 KHz + } + else if (strncmp(argv[i], "--profile=", 10) == 0) + { + profile = atoi(argv[i] + 10); + } + else if (strcmp(argv[i], "--coverage") == 0) + { + coverage = true; + } + else if (strcmp(argv[i], "--timetrace") == 0) + { + FFlag::DebugLuauTimeTracing.value = true; +#if !defined(LUAU_ENABLE_TIME_TRACE) + printf("To run with --timetrace, Luau has to be built with LUAU_ENABLE_TIME_TRACE enabled\n"); + return 1; +#endif + } + else if (argv[i][0] == '-') + { + fprintf(stdout, "Error: Unrecognized option '%s'.\n\n", argv[i]); + displayHelp(argv[0]); + return 1; + } + } + + const std::vector files = getSourceFiles(argc, argv); + if (mode == CliMode::Unknown) + { + mode = files.empty() ? CliMode::Repl : CliMode::RunSourceFiles; + } + + switch (mode) + { + case CliMode::Compile: + { #ifdef _WIN32 - if (format == CompileFormat::Binary) + if (compileFormat == CompileFormat::Binary) _setmode(_fileno(stdout), _O_BINARY); #endif - std::vector files = getSourceFiles(argc, argv); - int failed = 0; for (const std::string& path : files) - failed += !compileFile(path.c_str(), format); + failed += !compileFile(path.c_str(), compileFormat); - return failed; + return failed ? 1 : 0; } - + case CliMode::Repl: + { + runRepl(); + return 0; + } + case CliMode::RunSourceFiles: { std::unique_ptr globalState(luaL_newstate(), lua_close); lua_State* L = globalState.get(); setupState(L); - int profile = 0; - bool coverage = false; - - for (int i = 1; i < argc; ++i) - { - if (argv[i][0] != '-') - continue; - - if (strcmp(argv[i], "--profile") == 0) - profile = 10000; // default to 10 KHz - else if (strncmp(argv[i], "--profile=", 10) == 0) - profile = atoi(argv[i] + 10); - else if (strcmp(argv[i], "--coverage") == 0) - coverage = true; - } - if (profile) profilerStart(L, profile); if (coverage) coverageInit(L); - std::vector files = getSourceFiles(argc, argv); - int failed = 0; for (const std::string& path : files) @@ -582,6 +635,10 @@ int main(int argc, char** argv) if (coverage) coverageDump("coverage.out"); - return failed; + return failed ? 1 : 0; + } + case CliMode::Unknown: + default: + LUAU_ASSERT(!"Unhandled cli mode."); } } diff --git a/Compiler/src/Builtins.cpp b/Compiler/src/Builtins.cpp new file mode 100644 index 00000000..e344eb91 --- /dev/null +++ b/Compiler/src/Builtins.cpp @@ -0,0 +1,197 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +#include "Builtins.h" + +#include "Luau/Bytecode.h" +#include "Luau/Compiler.h" + +namespace Luau +{ +namespace Compile +{ + +Builtin getBuiltin(AstExpr* node, const DenseHashMap& globals, const DenseHashMap& variables) +{ + if (AstExprLocal* expr = node->as()) + { + const Variable* v = variables.find(expr->local); + + return v && !v->written && v->init ? getBuiltin(v->init, globals, variables) : Builtin(); + } + else if (AstExprIndexName* expr = node->as()) + { + if (AstExprGlobal* object = expr->expr->as()) + { + return getGlobalState(globals, object->name) == Global::Default ? Builtin{object->name, expr->index} : Builtin(); + } + else + { + return Builtin(); + } + } + else if (AstExprGlobal* expr = node->as()) + { + return getGlobalState(globals, expr->name) == Global::Default ? Builtin{AstName(), expr->name} : Builtin(); + } + else + { + return Builtin(); + } +} + +int getBuiltinFunctionId(const Builtin& builtin, const CompileOptions& options) +{ + if (builtin.empty()) + return -1; + + if (builtin.isGlobal("assert")) + return LBF_ASSERT; + + if (builtin.isGlobal("type")) + return LBF_TYPE; + + if (builtin.isGlobal("typeof")) + return LBF_TYPEOF; + + if (builtin.isGlobal("rawset")) + return LBF_RAWSET; + if (builtin.isGlobal("rawget")) + return LBF_RAWGET; + if (builtin.isGlobal("rawequal")) + return LBF_RAWEQUAL; + + if (builtin.isGlobal("unpack")) + return LBF_TABLE_UNPACK; + + if (builtin.object == "math") + { + if (builtin.method == "abs") + return LBF_MATH_ABS; + if (builtin.method == "acos") + return LBF_MATH_ACOS; + if (builtin.method == "asin") + return LBF_MATH_ASIN; + if (builtin.method == "atan2") + return LBF_MATH_ATAN2; + if (builtin.method == "atan") + return LBF_MATH_ATAN; + if (builtin.method == "ceil") + return LBF_MATH_CEIL; + if (builtin.method == "cosh") + return LBF_MATH_COSH; + if (builtin.method == "cos") + return LBF_MATH_COS; + if (builtin.method == "deg") + return LBF_MATH_DEG; + if (builtin.method == "exp") + return LBF_MATH_EXP; + if (builtin.method == "floor") + return LBF_MATH_FLOOR; + if (builtin.method == "fmod") + return LBF_MATH_FMOD; + if (builtin.method == "frexp") + return LBF_MATH_FREXP; + if (builtin.method == "ldexp") + return LBF_MATH_LDEXP; + if (builtin.method == "log10") + return LBF_MATH_LOG10; + if (builtin.method == "log") + return LBF_MATH_LOG; + if (builtin.method == "max") + return LBF_MATH_MAX; + if (builtin.method == "min") + return LBF_MATH_MIN; + if (builtin.method == "modf") + return LBF_MATH_MODF; + if (builtin.method == "pow") + return LBF_MATH_POW; + if (builtin.method == "rad") + return LBF_MATH_RAD; + if (builtin.method == "sinh") + return LBF_MATH_SINH; + if (builtin.method == "sin") + return LBF_MATH_SIN; + if (builtin.method == "sqrt") + return LBF_MATH_SQRT; + if (builtin.method == "tanh") + return LBF_MATH_TANH; + if (builtin.method == "tan") + return LBF_MATH_TAN; + if (builtin.method == "clamp") + return LBF_MATH_CLAMP; + if (builtin.method == "sign") + return LBF_MATH_SIGN; + if (builtin.method == "round") + return LBF_MATH_ROUND; + } + + if (builtin.object == "bit32") + { + if (builtin.method == "arshift") + return LBF_BIT32_ARSHIFT; + if (builtin.method == "band") + return LBF_BIT32_BAND; + if (builtin.method == "bnot") + return LBF_BIT32_BNOT; + if (builtin.method == "bor") + return LBF_BIT32_BOR; + if (builtin.method == "bxor") + return LBF_BIT32_BXOR; + if (builtin.method == "btest") + return LBF_BIT32_BTEST; + if (builtin.method == "extract") + return LBF_BIT32_EXTRACT; + if (builtin.method == "lrotate") + return LBF_BIT32_LROTATE; + if (builtin.method == "lshift") + return LBF_BIT32_LSHIFT; + if (builtin.method == "replace") + return LBF_BIT32_REPLACE; + if (builtin.method == "rrotate") + return LBF_BIT32_RROTATE; + if (builtin.method == "rshift") + return LBF_BIT32_RSHIFT; + if (builtin.method == "countlz") + return LBF_BIT32_COUNTLZ; + if (builtin.method == "countrz") + return LBF_BIT32_COUNTRZ; + } + + if (builtin.object == "string") + { + if (builtin.method == "byte") + return LBF_STRING_BYTE; + if (builtin.method == "char") + return LBF_STRING_CHAR; + if (builtin.method == "len") + return LBF_STRING_LEN; + if (builtin.method == "sub") + return LBF_STRING_SUB; + } + + if (builtin.object == "table") + { + if (builtin.method == "insert") + return LBF_TABLE_INSERT; + if (builtin.method == "unpack") + return LBF_TABLE_UNPACK; + } + + if (options.vectorCtor) + { + if (options.vectorLib) + { + if (builtin.isMethod(options.vectorLib, options.vectorCtor)) + return LBF_VECTOR; + } + else + { + if (builtin.isGlobal(options.vectorCtor)) + return LBF_VECTOR; + } + } + + return -1; +} + +} // namespace Compile +} // namespace Luau diff --git a/Compiler/src/Builtins.h b/Compiler/src/Builtins.h new file mode 100644 index 00000000..60df53a1 --- /dev/null +++ b/Compiler/src/Builtins.h @@ -0,0 +1,41 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +#pragma once + +#include "ValueTracking.h" + +namespace Luau +{ +struct CompileOptions; +} + +namespace Luau +{ +namespace Compile +{ + +struct Builtin +{ + AstName object; + AstName method; + + bool empty() const + { + return object == AstName() && method == AstName(); + } + + bool isGlobal(const char* name) const + { + return object == AstName() && method == name; + } + + bool isMethod(const char* table, const char* name) const + { + return object == table && method == name; + } +}; + +Builtin getBuiltin(AstExpr* node, const DenseHashMap& globals, const DenseHashMap& variables); +int getBuiltinFunctionId(const Builtin& builtin, const CompileOptions& options); + +} // namespace Compile +} // namespace Luau diff --git a/Compiler/src/Compiler.cpp b/Compiler/src/Compiler.cpp index 0156fcf5..9758c4a9 100644 --- a/Compiler/src/Compiler.cpp +++ b/Compiler/src/Compiler.cpp @@ -6,15 +6,20 @@ #include "Luau/Common.h" #include "Luau/TimeTrace.h" +#include "Builtins.h" +#include "ConstantFolding.h" +#include "TableShape.h" +#include "ValueTracking.h" + #include #include #include -LUAU_FASTFLAG(LuauIfElseExpressionBaseSupport) - namespace Luau { +using namespace Luau::Compile; + static const uint32_t kMaxRegisterCount = 255; static const uint32_t kMaxUpvalueCount = 200; static const uint32_t kMaxLocalCount = 200; @@ -62,7 +67,6 @@ static BytecodeBuilder::StringRef sref(AstArray data) struct Compiler { - struct Constant; struct RegScope; Compiler(BytecodeBuilder& bytecode, const CompileOptions& options) @@ -71,8 +75,9 @@ struct Compiler , functions(nullptr) , locals(nullptr) , globals(AstName()) + , variables(nullptr) , constants(nullptr) - , predictedTableSize(nullptr) + , tableShapes(nullptr) { } @@ -96,8 +101,10 @@ struct Compiler local->location, "Out of upvalue registers when trying to allocate %s: exceeded limit %d", local->name.value, kMaxUpvalueCount); // mark local as captured so that closeLocals emits LOP_CLOSEUPVALS accordingly - Local& l = locals[local]; - l.captured = true; + Variable* v = variables.find(local); + + if (v && v->written) + locals[local].captured = true; upvals.push_back(local); @@ -273,8 +280,8 @@ struct Compiler if (options.optimizationLevel >= 1) { - Builtin builtin = getBuiltin(expr->func); - bfid = getBuiltinFunctionId(builtin); + Builtin builtin = getBuiltin(expr->func, globals, variables); + bfid = getBuiltinFunctionId(builtin, options); } if (expr->self) @@ -424,8 +431,10 @@ struct Compiler for (AstLocal* uv : f->upvals) { - Local* ul = locals.find(uv); - LUAU_ASSERT(ul); + Variable* ul = variables.find(uv); + + if (!ul) + return false; if (ul->written) return false; @@ -437,10 +446,11 @@ struct Compiler // this will only deoptimize (outside of fenv changes) if top level code is executed twice with different results. if (uv->functionDepth != 0 || uv->loopDepth != 0) { - if (!ul->func) + AstExprFunction* uf = ul->init ? ul->init->as() : nullptr; + if (!uf) return false; - if (ul->func != func && !shouldShareClosure(ul->func)) + if (uf != func && !shouldShareClosure(uf)) return false; } } @@ -483,10 +493,8 @@ struct Compiler { LUAU_ASSERT(uv->functionDepth < expr->functionDepth); - Local* ul = locals.find(uv); - LUAU_ASSERT(ul); - - bool immutable = !ul->written; + Variable* ul = variables.find(uv); + bool immutable = !ul || !ul->written; if (uv->functionDepth == expr->functionDepth - 1) { @@ -687,7 +695,7 @@ struct Compiler break; case Constant::Type_String: - cid = bytecode.addConstantString(sref(c->valueString)); + cid = bytecode.addConstantString(sref(c->getString())); break; default: @@ -1066,10 +1074,10 @@ struct Compiler // Optimization: if the table is empty, we can compute it directly into the target if (expr->items.size == 0) { - auto [hashSize, arraySize] = predictedTableSize[expr]; + TableShape shape = tableShapes[expr]; - bytecode.emitABC(LOP_NEWTABLE, target, encodeHashSize(hashSize), 0); - bytecode.emitAux(arraySize); + bytecode.emitABC(LOP_NEWTABLE, target, encodeHashSize(shape.hashSize), 0); + bytecode.emitAux(shape.arraySize); return; } @@ -1252,16 +1260,12 @@ struct Compiler bool canImport(AstExprGlobal* expr) { - const Global* global = globals.find(expr->name); - - return options.optimizationLevel >= 1 && (!global || !global->written); + return options.optimizationLevel >= 1 && getGlobalState(globals, expr->name) != Global::Written; } bool canImportChain(AstExprGlobal* expr) { - const Global* global = globals.find(expr->name); - - return options.optimizationLevel >= 1 && (!global || (!global->written && !global->writable)); + return options.optimizationLevel >= 1 && getGlobalState(globals, expr->name) == Global::Default; } void compileExprIndexName(AstExprIndexName* expr, uint8_t target) @@ -1341,7 +1345,7 @@ struct Compiler { uint8_t rt = compileExprAuto(expr->expr, rs); - BytecodeBuilder::StringRef iname = sref(cv->valueString); + BytecodeBuilder::StringRef iname = sref(cv->getString()); int32_t cid = bytecode.addConstantString(iname); if (cid < 0) CompileError::raise(expr->location, "Exceeded constant limit; simplify the code to compile"); @@ -1427,7 +1431,7 @@ struct Compiler case Constant::Type_String: { - int32_t cid = bytecode.addConstantString(sref(cv->valueString)); + int32_t cid = bytecode.addConstantString(sref(cv->getString())); if (cid < 0) CompileError::raise(node->location, "Exceeded constant limit; simplify the code to compile"); @@ -1546,7 +1550,7 @@ struct Compiler { compileExpr(expr->expr, target, targetTemp); } - else if (AstExprIfElse* expr = node->as(); FFlag::LuauIfElseExpressionBaseSupport && expr) + else if (AstExprIfElse* expr = node->as()) { compileExprIfElse(expr, target, targetTemp); } @@ -1711,7 +1715,7 @@ struct Compiler { LValue result = {LValue::Kind_IndexName}; result.reg = compileExprAuto(expr->expr, rs); - result.name = sref(cv->valueString); + result.name = sref(cv->getString()); result.location = node->location; return result; @@ -1796,9 +1800,8 @@ struct Compiler return false; Local* l = locals.find(le->local); - LUAU_ASSERT(l); - return l->allocated; + return l && l->allocated; } bool isStatBreak(AstStat* node) @@ -2040,9 +2043,9 @@ struct Compiler for (AstLocal* local : stat->vars) { - Local* l = locals.find(local); + Variable* v = variables.find(local); - if (!l || l->constant.type == Constant::Type_Unknown) + if (!v || !v->constant) return false; } @@ -2082,9 +2085,7 @@ struct Compiler // through) uint8_t varreg = regs + 2; - Local* il = locals.find(stat->var); - - if (il && il->written) + if (Variable* il = variables.find(stat->var); il && il->written) varreg = allocReg(stat, 1); compileExprTemp(stat->from, uint8_t(regs + 2)); @@ -2164,7 +2165,7 @@ struct Compiler { if (stat->values.size == 1 && stat->values.data[0]->is()) { - Builtin builtin = getBuiltin(stat->values.data[0]->as()->func); + Builtin builtin = getBuiltin(stat->values.data[0]->as()->func, globals, variables); if (builtin.isGlobal("ipairs")) // for .. in ipairs(t) { @@ -2179,7 +2180,7 @@ struct Compiler } else if (stat->values.size == 2) { - Builtin builtin = getBuiltin(stat->values.data[0]); + Builtin builtin = getBuiltin(stat->values.data[0], globals, variables); if (builtin.isGlobal("next")) // for .. in next,t { @@ -2594,7 +2595,7 @@ struct Compiler Local* l = locals.find(localStack[i]); LUAU_ASSERT(l); - if (l->captured && l->written) + if (l->captured) return true; } @@ -2613,7 +2614,7 @@ struct Compiler Local* l = locals.find(localStack[i]); LUAU_ASSERT(l); - if (l->captured && l->written) + if (l->captured) { captured = true; captureReg = std::min(captureReg, l->reg); @@ -2728,519 +2729,6 @@ struct Compiler return !node->is() && !node->is(); } - struct AssignmentVisitor : AstVisitor - { - struct Hasher - { - size_t operator()(const std::pair& p) const - { - return std::hash()(p.first) ^ std::hash()(p.second); - } - }; - - DenseHashMap localToTable; - DenseHashSet, Hasher> fields; - - AssignmentVisitor(Compiler* self) - : localToTable(nullptr) - , fields(std::pair()) - , self(self) - { - } - - void assignField(AstExpr* expr, AstName index) - { - if (AstExprLocal* lv = expr->as()) - { - if (AstExprTable** table = localToTable.find(lv->local)) - { - std::pair field = {*table, index}; - - if (!fields.contains(field)) - { - fields.insert(field); - self->predictedTableSize[*table].first += 1; - } - } - } - } - - void assignField(AstExpr* expr, AstExpr* index) - { - AstExprLocal* lv = expr->as(); - AstExprConstantNumber* number = index->as(); - - if (lv && number) - { - if (AstExprTable** table = localToTable.find(lv->local)) - { - unsigned int& arraySize = self->predictedTableSize[*table].second; - - if (number->value == double(arraySize + 1)) - arraySize += 1; - } - } - } - - void assign(AstExpr* var) - { - if (AstExprLocal* lv = var->as()) - { - self->locals[lv->local].written = true; - } - else if (AstExprGlobal* gv = var->as()) - { - self->globals[gv->name].written = true; - } - else if (AstExprIndexName* index = var->as()) - { - assignField(index->expr, index->index); - - var->visit(this); - } - else if (AstExprIndexExpr* index = var->as()) - { - assignField(index->expr, index->index); - - var->visit(this); - } - else - { - // we need to be able to track assignments in all expressions, including crazy ones like t[function() t = nil end] = 5 - var->visit(this); - } - } - - AstExprTable* getTableHint(AstExpr* expr) - { - // unadorned table literal - if (AstExprTable* table = expr->as()) - return table; - - // setmetatable(table literal, ...) - if (AstExprCall* call = expr->as(); call && !call->self && call->args.size == 2) - if (AstExprGlobal* func = call->func->as(); func && func->name == "setmetatable") - if (AstExprTable* table = call->args.data[0]->as()) - return table; - - return nullptr; - } - - bool visit(AstStatLocal* node) override - { - // track local -> table association so that we can update table size prediction in assignField - if (node->vars.size == 1 && node->values.size == 1) - if (AstExprTable* table = getTableHint(node->values.data[0]); table && table->items.size == 0) - localToTable[node->vars.data[0]] = table; - - return true; - } - - bool visit(AstStatAssign* node) override - { - for (size_t i = 0; i < node->vars.size; ++i) - assign(node->vars.data[i]); - - for (size_t i = 0; i < node->values.size; ++i) - node->values.data[i]->visit(this); - - return false; - } - - bool visit(AstStatCompoundAssign* node) override - { - assign(node->var); - node->value->visit(this); - - return false; - } - - bool visit(AstStatFunction* node) override - { - assign(node->name); - node->func->visit(this); - - return false; - } - - Compiler* self; - }; - - struct ConstantVisitor : AstVisitor - { - ConstantVisitor(Compiler* self) - : self(self) - { - } - - void analyzeUnary(Constant& result, AstExprUnary::Op op, const Constant& arg) - { - switch (op) - { - case AstExprUnary::Not: - if (arg.type != Constant::Type_Unknown) - { - result.type = Constant::Type_Boolean; - result.valueBoolean = !arg.isTruthful(); - } - break; - - case AstExprUnary::Minus: - if (arg.type == Constant::Type_Number) - { - result.type = Constant::Type_Number; - result.valueNumber = -arg.valueNumber; - } - break; - - case AstExprUnary::Len: - if (arg.type == Constant::Type_String) - { - result.type = Constant::Type_Number; - result.valueNumber = double(arg.valueString.size); - } - break; - - default: - LUAU_ASSERT(!"Unexpected unary operation"); - } - } - - bool constantsEqual(const Constant& la, const Constant& ra) - { - LUAU_ASSERT(la.type != Constant::Type_Unknown && ra.type != Constant::Type_Unknown); - - switch (la.type) - { - case Constant::Type_Nil: - return ra.type == Constant::Type_Nil; - - case Constant::Type_Boolean: - return ra.type == Constant::Type_Boolean && la.valueBoolean == ra.valueBoolean; - - case Constant::Type_Number: - return ra.type == Constant::Type_Number && la.valueNumber == ra.valueNumber; - - case Constant::Type_String: - return ra.type == Constant::Type_String && la.valueString.size == ra.valueString.size && - memcmp(la.valueString.data, ra.valueString.data, la.valueString.size) == 0; - - default: - LUAU_ASSERT(!"Unexpected constant type in comparison"); - return false; - } - } - - void analyzeBinary(Constant& result, AstExprBinary::Op op, const Constant& la, const Constant& ra) - { - switch (op) - { - case AstExprBinary::Add: - if (la.type == Constant::Type_Number && ra.type == Constant::Type_Number) - { - result.type = Constant::Type_Number; - result.valueNumber = la.valueNumber + ra.valueNumber; - } - break; - - case AstExprBinary::Sub: - if (la.type == Constant::Type_Number && ra.type == Constant::Type_Number) - { - result.type = Constant::Type_Number; - result.valueNumber = la.valueNumber - ra.valueNumber; - } - break; - - case AstExprBinary::Mul: - if (la.type == Constant::Type_Number && ra.type == Constant::Type_Number) - { - result.type = Constant::Type_Number; - result.valueNumber = la.valueNumber * ra.valueNumber; - } - break; - - case AstExprBinary::Div: - if (la.type == Constant::Type_Number && ra.type == Constant::Type_Number) - { - result.type = Constant::Type_Number; - result.valueNumber = la.valueNumber / ra.valueNumber; - } - break; - - case AstExprBinary::Mod: - if (la.type == Constant::Type_Number && ra.type == Constant::Type_Number) - { - result.type = Constant::Type_Number; - result.valueNumber = la.valueNumber - floor(la.valueNumber / ra.valueNumber) * ra.valueNumber; - } - break; - - case AstExprBinary::Pow: - if (la.type == Constant::Type_Number && ra.type == Constant::Type_Number) - { - result.type = Constant::Type_Number; - result.valueNumber = pow(la.valueNumber, ra.valueNumber); - } - break; - - case AstExprBinary::Concat: - break; - - case AstExprBinary::CompareNe: - if (la.type != Constant::Type_Unknown && ra.type != Constant::Type_Unknown) - { - result.type = Constant::Type_Boolean; - result.valueBoolean = !constantsEqual(la, ra); - } - break; - - case AstExprBinary::CompareEq: - if (la.type != Constant::Type_Unknown && ra.type != Constant::Type_Unknown) - { - result.type = Constant::Type_Boolean; - result.valueBoolean = constantsEqual(la, ra); - } - break; - - case AstExprBinary::CompareLt: - if (la.type == Constant::Type_Number && ra.type == Constant::Type_Number) - { - result.type = Constant::Type_Boolean; - result.valueBoolean = la.valueNumber < ra.valueNumber; - } - break; - - case AstExprBinary::CompareLe: - if (la.type == Constant::Type_Number && ra.type == Constant::Type_Number) - { - result.type = Constant::Type_Boolean; - result.valueBoolean = la.valueNumber <= ra.valueNumber; - } - break; - - case AstExprBinary::CompareGt: - if (la.type == Constant::Type_Number && ra.type == Constant::Type_Number) - { - result.type = Constant::Type_Boolean; - result.valueBoolean = la.valueNumber > ra.valueNumber; - } - break; - - case AstExprBinary::CompareGe: - if (la.type == Constant::Type_Number && ra.type == Constant::Type_Number) - { - result.type = Constant::Type_Boolean; - result.valueBoolean = la.valueNumber >= ra.valueNumber; - } - break; - - case AstExprBinary::And: - if (la.type != Constant::Type_Unknown) - { - result = la.isTruthful() ? ra : la; - } - break; - - case AstExprBinary::Or: - if (la.type != Constant::Type_Unknown) - { - result = la.isTruthful() ? la : ra; - } - break; - - default: - LUAU_ASSERT(!"Unexpected binary operation"); - } - } - - Constant analyze(AstExpr* node) - { - Constant result; - result.type = Constant::Type_Unknown; - - if (AstExprGroup* expr = node->as()) - { - result = analyze(expr->expr); - } - else if (node->is()) - { - result.type = Constant::Type_Nil; - } - else if (AstExprConstantBool* expr = node->as()) - { - result.type = Constant::Type_Boolean; - result.valueBoolean = expr->value; - } - else if (AstExprConstantNumber* expr = node->as()) - { - result.type = Constant::Type_Number; - result.valueNumber = expr->value; - } - else if (AstExprConstantString* expr = node->as()) - { - result.type = Constant::Type_String; - result.valueString = expr->value; - } - else if (AstExprLocal* expr = node->as()) - { - const Local* l = self->locals.find(expr->local); - - if (l && l->constant.type != Constant::Type_Unknown) - { - LUAU_ASSERT(!l->written); - result = l->constant; - } - } - else if (node->is()) - { - // nope - } - else if (node->is()) - { - // nope - } - else if (AstExprCall* expr = node->as()) - { - analyze(expr->func); - - for (size_t i = 0; i < expr->args.size; ++i) - analyze(expr->args.data[i]); - } - else if (AstExprIndexName* expr = node->as()) - { - analyze(expr->expr); - } - else if (AstExprIndexExpr* expr = node->as()) - { - analyze(expr->expr); - analyze(expr->index); - } - else if (AstExprFunction* expr = node->as()) - { - // this is necessary to propagate constant information in all child functions - expr->body->visit(this); - } - else if (AstExprTable* expr = node->as()) - { - for (size_t i = 0; i < expr->items.size; ++i) - { - const AstExprTable::Item& item = expr->items.data[i]; - - if (item.key) - analyze(item.key); - - analyze(item.value); - } - } - else if (AstExprUnary* expr = node->as()) - { - Constant arg = analyze(expr->expr); - - analyzeUnary(result, expr->op, arg); - } - else if (AstExprBinary* expr = node->as()) - { - Constant la = analyze(expr->left); - Constant ra = analyze(expr->right); - - analyzeBinary(result, expr->op, la, ra); - } - else if (AstExprTypeAssertion* expr = node->as()) - { - Constant arg = analyze(expr->expr); - - result = arg; - } - else if (AstExprIfElse* expr = node->as(); FFlag::LuauIfElseExpressionBaseSupport && expr) - { - Constant cond = analyze(expr->condition); - Constant trueExpr = analyze(expr->trueExpr); - Constant falseExpr = analyze(expr->falseExpr); - if (cond.type != Constant::Type_Unknown) - { - result = cond.isTruthful() ? trueExpr : falseExpr; - } - } - else - { - LUAU_ASSERT(!"Unknown expression type"); - } - - if (result.type != Constant::Type_Unknown) - self->constants[node] = result; - - return result; - } - - bool visit(AstExpr* node) override - { - // note: we short-circuit the visitor traversal through any expression trees by returning false - // recursive traversal is happening inside analyze() which makes it easier to get the resulting value of the subexpression - analyze(node); - - return false; - } - - bool visit(AstStatLocal* node) override - { - // for values that match 1-1 we record the initializing expression for future analysis - for (size_t i = 0; i < node->vars.size && i < node->values.size; ++i) - { - Local& l = self->locals[node->vars.data[i]]; - - l.init = node->values.data[i]; - } - - // all values that align wrt indexing are simple - we just match them 1-1 - for (size_t i = 0; i < node->vars.size && i < node->values.size; ++i) - { - Constant arg = analyze(node->values.data[i]); - - if (arg.type != Constant::Type_Unknown) - { - Local& l = self->locals[node->vars.data[i]]; - - // note: we rely on AssignmentVisitor to have been run before us - if (!l.written) - l.constant = arg; - } - } - - if (node->vars.size > node->values.size) - { - // if we have trailing variables, then depending on whether the last value is capable of returning multiple values - // (aka call or varargs), we either don't know anything about these vars, or we know they're nil - AstExpr* last = node->values.size ? node->values.data[node->values.size - 1] : nullptr; - bool multRet = last && (last->is() || last->is()); - - for (size_t i = node->values.size; i < node->vars.size; ++i) - { - if (!multRet) - { - Local& l = self->locals[node->vars.data[i]]; - - // note: we rely on AssignmentVisitor to have been run before us - if (!l.written) - { - l.constant.type = Constant::Type_Nil; - } - } - } - } - else - { - // we can have more values than variables; in this case we still need to analyze them to make sure we do constant propagation inside - // them - for (size_t i = node->vars.size; i < node->values.size; ++i) - analyze(node->values.data[i]); - } - - return false; - } - - Compiler* self; - }; - struct FenvVisitor : AstVisitor { bool& getfenvUsed; @@ -3283,14 +2771,6 @@ struct Compiler return false; } - - bool visit(AstStatLocalFunction* node) override - { - // record local->function association for some optimizations - self->locals[node->name].func = node->func; - - return true; - } }; struct UndefinedLocalVisitor : AstVisitor @@ -3397,70 +2877,12 @@ struct Compiler std::vector upvals; }; - struct Constant - { - enum Type - { - Type_Unknown, - Type_Nil, - Type_Boolean, - Type_Number, - Type_String, - }; - - Type type = Type_Unknown; - - union - { - bool valueBoolean; - double valueNumber; - AstArray valueString = {}; - }; - - bool isTruthful() const - { - LUAU_ASSERT(type != Type_Unknown); - return type != Type_Nil && !(type == Type_Boolean && valueBoolean == false); - } - }; - struct Local { uint8_t reg = 0; bool allocated = false; bool captured = false; - bool written = false; - AstExpr* init = nullptr; uint32_t debugpc = 0; - Constant constant; - AstExprFunction* func = nullptr; - }; - - struct Global - { - bool writable = false; - bool written = false; - }; - - struct Builtin - { - AstName object; - AstName method; - - bool empty() const - { - return object == AstName() && method == AstName(); - } - - bool isGlobal(const char* name) const - { - return object == AstName() && method == name; - } - - bool isMethod(const char* table, const char* name) const - { - return object == table && method == name; - } }; struct LoopJump @@ -3482,194 +2904,6 @@ struct Compiler AstExpr* untilCondition; }; - Builtin getBuiltin(AstExpr* node) - { - if (AstExprLocal* expr = node->as()) - { - Local* l = locals.find(expr->local); - - return l && !l->written && l->init ? getBuiltin(l->init) : Builtin(); - } - else if (AstExprIndexName* expr = node->as()) - { - if (AstExprGlobal* object = expr->expr->as()) - { - Global* g = globals.find(object->name); - - return !g || (!g->writable && !g->written) ? Builtin{object->name, expr->index} : Builtin(); - } - else - { - return Builtin(); - } - } - else if (AstExprGlobal* expr = node->as()) - { - Global* g = globals.find(expr->name); - - return !g || !g->written ? Builtin{AstName(), expr->name} : Builtin(); - } - else - { - return Builtin(); - } - } - - int getBuiltinFunctionId(const Builtin& builtin) - { - if (builtin.empty()) - return -1; - - if (builtin.isGlobal("assert")) - return LBF_ASSERT; - - if (builtin.isGlobal("type")) - return LBF_TYPE; - - if (builtin.isGlobal("typeof")) - return LBF_TYPEOF; - - if (builtin.isGlobal("rawset")) - return LBF_RAWSET; - if (builtin.isGlobal("rawget")) - return LBF_RAWGET; - if (builtin.isGlobal("rawequal")) - return LBF_RAWEQUAL; - - if (builtin.isGlobal("unpack")) - return LBF_TABLE_UNPACK; - - if (builtin.object == "math") - { - if (builtin.method == "abs") - return LBF_MATH_ABS; - if (builtin.method == "acos") - return LBF_MATH_ACOS; - if (builtin.method == "asin") - return LBF_MATH_ASIN; - if (builtin.method == "atan2") - return LBF_MATH_ATAN2; - if (builtin.method == "atan") - return LBF_MATH_ATAN; - if (builtin.method == "ceil") - return LBF_MATH_CEIL; - if (builtin.method == "cosh") - return LBF_MATH_COSH; - if (builtin.method == "cos") - return LBF_MATH_COS; - if (builtin.method == "deg") - return LBF_MATH_DEG; - if (builtin.method == "exp") - return LBF_MATH_EXP; - if (builtin.method == "floor") - return LBF_MATH_FLOOR; - if (builtin.method == "fmod") - return LBF_MATH_FMOD; - if (builtin.method == "frexp") - return LBF_MATH_FREXP; - if (builtin.method == "ldexp") - return LBF_MATH_LDEXP; - if (builtin.method == "log10") - return LBF_MATH_LOG10; - if (builtin.method == "log") - return LBF_MATH_LOG; - if (builtin.method == "max") - return LBF_MATH_MAX; - if (builtin.method == "min") - return LBF_MATH_MIN; - if (builtin.method == "modf") - return LBF_MATH_MODF; - if (builtin.method == "pow") - return LBF_MATH_POW; - if (builtin.method == "rad") - return LBF_MATH_RAD; - if (builtin.method == "sinh") - return LBF_MATH_SINH; - if (builtin.method == "sin") - return LBF_MATH_SIN; - if (builtin.method == "sqrt") - return LBF_MATH_SQRT; - if (builtin.method == "tanh") - return LBF_MATH_TANH; - if (builtin.method == "tan") - return LBF_MATH_TAN; - if (builtin.method == "clamp") - return LBF_MATH_CLAMP; - if (builtin.method == "sign") - return LBF_MATH_SIGN; - if (builtin.method == "round") - return LBF_MATH_ROUND; - } - - if (builtin.object == "bit32") - { - if (builtin.method == "arshift") - return LBF_BIT32_ARSHIFT; - if (builtin.method == "band") - return LBF_BIT32_BAND; - if (builtin.method == "bnot") - return LBF_BIT32_BNOT; - if (builtin.method == "bor") - return LBF_BIT32_BOR; - if (builtin.method == "bxor") - return LBF_BIT32_BXOR; - if (builtin.method == "btest") - return LBF_BIT32_BTEST; - if (builtin.method == "extract") - return LBF_BIT32_EXTRACT; - if (builtin.method == "lrotate") - return LBF_BIT32_LROTATE; - if (builtin.method == "lshift") - return LBF_BIT32_LSHIFT; - if (builtin.method == "replace") - return LBF_BIT32_REPLACE; - if (builtin.method == "rrotate") - return LBF_BIT32_RROTATE; - if (builtin.method == "rshift") - return LBF_BIT32_RSHIFT; - if (builtin.method == "countlz") - return LBF_BIT32_COUNTLZ; - if (builtin.method == "countrz") - return LBF_BIT32_COUNTRZ; - } - - if (builtin.object == "string") - { - if (builtin.method == "byte") - return LBF_STRING_BYTE; - if (builtin.method == "char") - return LBF_STRING_CHAR; - if (builtin.method == "len") - return LBF_STRING_LEN; - if (builtin.method == "sub") - return LBF_STRING_SUB; - } - - if (builtin.object == "table") - { - if (builtin.method == "insert") - return LBF_TABLE_INSERT; - if (builtin.method == "unpack") - return LBF_TABLE_UNPACK; - } - - if (options.vectorCtor) - { - if (options.vectorLib) - { - if (builtin.isMethod(options.vectorLib, options.vectorCtor)) - return LBF_VECTOR; - } - else - { - if (builtin.isGlobal(options.vectorCtor)) - return LBF_VECTOR; - } - } - - return -1; - } - BytecodeBuilder& bytecode; CompileOptions options; @@ -3677,8 +2911,9 @@ struct Compiler DenseHashMap functions; DenseHashMap locals; DenseHashMap globals; + DenseHashMap variables; DenseHashMap constants; - DenseHashMap> predictedTableSize; + DenseHashMap tableShapes; unsigned int regTop = 0; unsigned int stackSize = 0; @@ -3699,23 +2934,18 @@ void compileOrThrow(BytecodeBuilder& bytecode, AstStatBlock* root, const AstName Compiler compiler(bytecode, options); // since access to some global objects may result in values that change over time, we block imports from non-readonly tables - if (AstName name = names.get("_G"); name.value) - compiler.globals[name].writable = true; + assignMutable(compiler.globals, names, options.mutableGlobals); - if (options.mutableGlobals) - for (const char** ptr = options.mutableGlobals; *ptr; ++ptr) - if (AstName name = names.get(*ptr); name.value) - compiler.globals[name].writable = true; + // this pass analyzes mutability of locals/globals and associates locals with their initial values + trackValues(compiler.globals, compiler.variables, root); - // this visitor traverses the AST to analyze mutability of locals/globals, filling Local::written and Global::written - Compiler::AssignmentVisitor assignmentVisitor(&compiler); - root->visit(&assignmentVisitor); - - // this visitor traverses the AST to analyze constantness of expressions, filling constants[] and Local::constant/Local::init if (options.optimizationLevel >= 1) { - Compiler::ConstantVisitor constantVisitor(&compiler); - root->visit(&constantVisitor); + // this pass analyzes constantness of expressions + foldConstants(compiler.constants, compiler.variables, root); + + // this pass analyzes table assignments to estimate table shapes for initially empty tables + predictTableShapes(compiler.tableShapes, root); } // this visitor tracks calls to getfenv/setfenv and disables some optimizations when they are found @@ -3734,8 +2964,8 @@ void compileOrThrow(BytecodeBuilder& bytecode, AstStatBlock* root, const AstName for (AstExprFunction* expr : functions) compiler.compileFunction(expr); - AstExprFunction main(root->location, /*generics= */ AstArray(), /*genericPacks= */ AstArray(), /* self= */ nullptr, - AstArray(), /* vararg= */ Luau::Location(), root, /* functionDepth= */ 0, /* debugname= */ AstName()); + AstExprFunction main(root->location, /*generics= */ AstArray(), /*genericPacks= */ AstArray(), + /* self= */ nullptr, AstArray(), /* vararg= */ Luau::Location(), root, /* functionDepth= */ 0, /* debugname= */ AstName()); uint32_t mainid = compiler.compileFunction(&main); bytecode.setMainFunction(mainid); diff --git a/Compiler/src/ConstantFolding.cpp b/Compiler/src/ConstantFolding.cpp new file mode 100644 index 00000000..60a7c169 --- /dev/null +++ b/Compiler/src/ConstantFolding.cpp @@ -0,0 +1,394 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +#include "ConstantFolding.h" + +#include + +namespace Luau +{ +namespace Compile +{ + +static bool constantsEqual(const Constant& la, const Constant& ra) +{ + LUAU_ASSERT(la.type != Constant::Type_Unknown && ra.type != Constant::Type_Unknown); + + switch (la.type) + { + case Constant::Type_Nil: + return ra.type == Constant::Type_Nil; + + case Constant::Type_Boolean: + return ra.type == Constant::Type_Boolean && la.valueBoolean == ra.valueBoolean; + + case Constant::Type_Number: + return ra.type == Constant::Type_Number && la.valueNumber == ra.valueNumber; + + case Constant::Type_String: + return ra.type == Constant::Type_String && la.stringLength == ra.stringLength && memcmp(la.valueString, ra.valueString, la.stringLength) == 0; + + default: + LUAU_ASSERT(!"Unexpected constant type in comparison"); + return false; + } +} + +static void foldUnary(Constant& result, AstExprUnary::Op op, const Constant& arg) +{ + switch (op) + { + case AstExprUnary::Not: + if (arg.type != Constant::Type_Unknown) + { + result.type = Constant::Type_Boolean; + result.valueBoolean = !arg.isTruthful(); + } + break; + + case AstExprUnary::Minus: + if (arg.type == Constant::Type_Number) + { + result.type = Constant::Type_Number; + result.valueNumber = -arg.valueNumber; + } + break; + + case AstExprUnary::Len: + if (arg.type == Constant::Type_String) + { + result.type = Constant::Type_Number; + result.valueNumber = double(arg.stringLength); + } + break; + + default: + LUAU_ASSERT(!"Unexpected unary operation"); + } +} + +static void foldBinary(Constant& result, AstExprBinary::Op op, const Constant& la, const Constant& ra) +{ + switch (op) + { + case AstExprBinary::Add: + if (la.type == Constant::Type_Number && ra.type == Constant::Type_Number) + { + result.type = Constant::Type_Number; + result.valueNumber = la.valueNumber + ra.valueNumber; + } + break; + + case AstExprBinary::Sub: + if (la.type == Constant::Type_Number && ra.type == Constant::Type_Number) + { + result.type = Constant::Type_Number; + result.valueNumber = la.valueNumber - ra.valueNumber; + } + break; + + case AstExprBinary::Mul: + if (la.type == Constant::Type_Number && ra.type == Constant::Type_Number) + { + result.type = Constant::Type_Number; + result.valueNumber = la.valueNumber * ra.valueNumber; + } + break; + + case AstExprBinary::Div: + if (la.type == Constant::Type_Number && ra.type == Constant::Type_Number) + { + result.type = Constant::Type_Number; + result.valueNumber = la.valueNumber / ra.valueNumber; + } + break; + + case AstExprBinary::Mod: + if (la.type == Constant::Type_Number && ra.type == Constant::Type_Number) + { + result.type = Constant::Type_Number; + result.valueNumber = la.valueNumber - floor(la.valueNumber / ra.valueNumber) * ra.valueNumber; + } + break; + + case AstExprBinary::Pow: + if (la.type == Constant::Type_Number && ra.type == Constant::Type_Number) + { + result.type = Constant::Type_Number; + result.valueNumber = pow(la.valueNumber, ra.valueNumber); + } + break; + + case AstExprBinary::Concat: + break; + + case AstExprBinary::CompareNe: + if (la.type != Constant::Type_Unknown && ra.type != Constant::Type_Unknown) + { + result.type = Constant::Type_Boolean; + result.valueBoolean = !constantsEqual(la, ra); + } + break; + + case AstExprBinary::CompareEq: + if (la.type != Constant::Type_Unknown && ra.type != Constant::Type_Unknown) + { + result.type = Constant::Type_Boolean; + result.valueBoolean = constantsEqual(la, ra); + } + break; + + case AstExprBinary::CompareLt: + if (la.type == Constant::Type_Number && ra.type == Constant::Type_Number) + { + result.type = Constant::Type_Boolean; + result.valueBoolean = la.valueNumber < ra.valueNumber; + } + break; + + case AstExprBinary::CompareLe: + if (la.type == Constant::Type_Number && ra.type == Constant::Type_Number) + { + result.type = Constant::Type_Boolean; + result.valueBoolean = la.valueNumber <= ra.valueNumber; + } + break; + + case AstExprBinary::CompareGt: + if (la.type == Constant::Type_Number && ra.type == Constant::Type_Number) + { + result.type = Constant::Type_Boolean; + result.valueBoolean = la.valueNumber > ra.valueNumber; + } + break; + + case AstExprBinary::CompareGe: + if (la.type == Constant::Type_Number && ra.type == Constant::Type_Number) + { + result.type = Constant::Type_Boolean; + result.valueBoolean = la.valueNumber >= ra.valueNumber; + } + break; + + case AstExprBinary::And: + if (la.type != Constant::Type_Unknown) + { + result = la.isTruthful() ? ra : la; + } + break; + + case AstExprBinary::Or: + if (la.type != Constant::Type_Unknown) + { + result = la.isTruthful() ? la : ra; + } + break; + + default: + LUAU_ASSERT(!"Unexpected binary operation"); + } +} + +struct ConstantVisitor : AstVisitor +{ + DenseHashMap& constants; + DenseHashMap& variables; + + DenseHashMap locals; + + ConstantVisitor(DenseHashMap& constants, DenseHashMap& variables) + : constants(constants) + , variables(variables) + , locals(nullptr) + { + } + + Constant analyze(AstExpr* node) + { + Constant result; + result.type = Constant::Type_Unknown; + + if (AstExprGroup* expr = node->as()) + { + result = analyze(expr->expr); + } + else if (node->is()) + { + result.type = Constant::Type_Nil; + } + else if (AstExprConstantBool* expr = node->as()) + { + result.type = Constant::Type_Boolean; + result.valueBoolean = expr->value; + } + else if (AstExprConstantNumber* expr = node->as()) + { + result.type = Constant::Type_Number; + result.valueNumber = expr->value; + } + else if (AstExprConstantString* expr = node->as()) + { + result.type = Constant::Type_String; + result.valueString = expr->value.data; + result.stringLength = unsigned(expr->value.size); + } + else if (AstExprLocal* expr = node->as()) + { + const Constant* l = locals.find(expr->local); + + if (l) + result = *l; + } + else if (node->is()) + { + // nope + } + else if (node->is()) + { + // nope + } + else if (AstExprCall* expr = node->as()) + { + analyze(expr->func); + + for (size_t i = 0; i < expr->args.size; ++i) + analyze(expr->args.data[i]); + } + else if (AstExprIndexName* expr = node->as()) + { + analyze(expr->expr); + } + else if (AstExprIndexExpr* expr = node->as()) + { + analyze(expr->expr); + analyze(expr->index); + } + else if (AstExprFunction* expr = node->as()) + { + // this is necessary to propagate constant information in all child functions + expr->body->visit(this); + } + else if (AstExprTable* expr = node->as()) + { + for (size_t i = 0; i < expr->items.size; ++i) + { + const AstExprTable::Item& item = expr->items.data[i]; + + if (item.key) + analyze(item.key); + + analyze(item.value); + } + } + else if (AstExprUnary* expr = node->as()) + { + Constant arg = analyze(expr->expr); + + if (arg.type != Constant::Type_Unknown) + foldUnary(result, expr->op, arg); + } + else if (AstExprBinary* expr = node->as()) + { + Constant la = analyze(expr->left); + Constant ra = analyze(expr->right); + + if (la.type != Constant::Type_Unknown && ra.type != Constant::Type_Unknown) + foldBinary(result, expr->op, la, ra); + } + else if (AstExprTypeAssertion* expr = node->as()) + { + Constant arg = analyze(expr->expr); + + result = arg; + } + else if (AstExprIfElse* expr = node->as()) + { + Constant cond = analyze(expr->condition); + Constant trueExpr = analyze(expr->trueExpr); + Constant falseExpr = analyze(expr->falseExpr); + + if (cond.type != Constant::Type_Unknown) + result = cond.isTruthful() ? trueExpr : falseExpr; + } + else + { + LUAU_ASSERT(!"Unknown expression type"); + } + + if (result.type != Constant::Type_Unknown) + constants[node] = result; + + return result; + } + + bool visit(AstExpr* node) override + { + // note: we short-circuit the visitor traversal through any expression trees by returning false + // recursive traversal is happening inside analyze() which makes it easier to get the resulting value of the subexpression + analyze(node); + + return false; + } + + bool visit(AstStatLocal* node) override + { + // all values that align wrt indexing are simple - we just match them 1-1 + for (size_t i = 0; i < node->vars.size && i < node->values.size; ++i) + { + Constant arg = analyze(node->values.data[i]); + + if (arg.type != Constant::Type_Unknown) + { + // note: we rely on trackValues to have been run before us + Variable* v = variables.find(node->vars.data[i]); + LUAU_ASSERT(v); + + if (!v->written) + { + locals[node->vars.data[i]] = arg; + v->constant = true; + } + } + } + + if (node->vars.size > node->values.size) + { + // if we have trailing variables, then depending on whether the last value is capable of returning multiple values + // (aka call or varargs), we either don't know anything about these vars, or we know they're nil + AstExpr* last = node->values.size ? node->values.data[node->values.size - 1] : nullptr; + bool multRet = last && (last->is() || last->is()); + + if (!multRet) + { + for (size_t i = node->values.size; i < node->vars.size; ++i) + { + // note: we rely on trackValues to have been run before us + Variable* v = variables.find(node->vars.data[i]); + LUAU_ASSERT(v); + + if (!v->written) + { + locals[node->vars.data[i]].type = Constant::Type_Nil; + v->constant = true; + } + } + } + } + else + { + // we can have more values than variables; in this case we still need to analyze them to make sure we do constant propagation inside + // them + for (size_t i = node->vars.size; i < node->values.size; ++i) + analyze(node->values.data[i]); + } + + return false; + } +}; + +void foldConstants(DenseHashMap& constants, DenseHashMap& variables, AstNode* root) +{ + ConstantVisitor visitor{constants, variables}; + root->visit(&visitor); +} + +} // namespace Compile +} // namespace Luau diff --git a/Compiler/src/ConstantFolding.h b/Compiler/src/ConstantFolding.h new file mode 100644 index 00000000..c0e63539 --- /dev/null +++ b/Compiler/src/ConstantFolding.h @@ -0,0 +1,48 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +#pragma once + +#include "ValueTracking.h" + +namespace Luau +{ +namespace Compile +{ + +struct Constant +{ + enum Type + { + Type_Unknown, + Type_Nil, + Type_Boolean, + Type_Number, + Type_String, + }; + + Type type = Type_Unknown; + unsigned int stringLength = 0; + + union + { + bool valueBoolean; + double valueNumber; + char* valueString = nullptr; // length stored in stringLength + }; + + bool isTruthful() const + { + LUAU_ASSERT(type != Type_Unknown); + return type != Type_Nil && !(type == Type_Boolean && valueBoolean == false); + } + + AstArray getString() const + { + LUAU_ASSERT(type == Type_String); + return {valueString, stringLength}; + } +}; + +void foldConstants(DenseHashMap& constants, DenseHashMap& variables, AstNode* root); + +} // namespace Compile +} // namespace Luau diff --git a/Compiler/src/TableShape.cpp b/Compiler/src/TableShape.cpp new file mode 100644 index 00000000..7d99f222 --- /dev/null +++ b/Compiler/src/TableShape.cpp @@ -0,0 +1,129 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +#include "TableShape.h" + +namespace Luau +{ +namespace Compile +{ + +static AstExprTable* getTableHint(AstExpr* expr) +{ + // unadorned table literal + if (AstExprTable* table = expr->as()) + return table; + + // setmetatable(table literal, ...) + if (AstExprCall* call = expr->as(); call && !call->self && call->args.size == 2) + if (AstExprGlobal* func = call->func->as(); func && func->name == "setmetatable") + if (AstExprTable* table = call->args.data[0]->as()) + return table; + + return nullptr; +} + +struct ShapeVisitor : AstVisitor +{ + struct Hasher + { + size_t operator()(const std::pair& p) const + { + return std::hash()(p.first) ^ std::hash()(p.second); + } + }; + + DenseHashMap& shapes; + + DenseHashMap tables; + DenseHashSet, Hasher> fields; + + ShapeVisitor(DenseHashMap& shapes) + : shapes(shapes) + , tables(nullptr) + , fields(std::pair()) + { + } + + void assignField(AstExpr* expr, AstName index) + { + if (AstExprLocal* lv = expr->as()) + { + if (AstExprTable** table = tables.find(lv->local)) + { + std::pair field = {*table, index}; + + if (!fields.contains(field)) + { + fields.insert(field); + shapes[*table].hashSize += 1; + } + } + } + } + + void assignField(AstExpr* expr, AstExpr* index) + { + AstExprLocal* lv = expr->as(); + AstExprConstantNumber* number = index->as(); + + if (lv && number) + { + if (AstExprTable** table = tables.find(lv->local)) + { + TableShape& shape = shapes[*table]; + + if (number->value == double(shape.arraySize + 1)) + shape.arraySize += 1; + } + } + } + + void assign(AstExpr* var) + { + if (AstExprIndexName* index = var->as()) + { + assignField(index->expr, index->index); + } + else if (AstExprIndexExpr* index = var->as()) + { + assignField(index->expr, index->index); + } + } + + bool visit(AstStatLocal* node) override + { + // track local -> table association so that we can update table size prediction in assignField + if (node->vars.size == 1 && node->values.size == 1) + if (AstExprTable* table = getTableHint(node->values.data[0]); table && table->items.size == 0) + tables[node->vars.data[0]] = table; + + return true; + } + + bool visit(AstStatAssign* node) override + { + for (size_t i = 0; i < node->vars.size; ++i) + assign(node->vars.data[i]); + + for (size_t i = 0; i < node->values.size; ++i) + node->values.data[i]->visit(this); + + return false; + } + + bool visit(AstStatFunction* node) override + { + assign(node->name); + node->func->visit(this); + + return false; + } +}; + +void predictTableShapes(DenseHashMap& shapes, AstNode* root) +{ + ShapeVisitor visitor{shapes}; + root->visit(&visitor); +} + +} // namespace Compile +} // namespace Luau diff --git a/Compiler/src/TableShape.h b/Compiler/src/TableShape.h new file mode 100644 index 00000000..f30853a7 --- /dev/null +++ b/Compiler/src/TableShape.h @@ -0,0 +1,21 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +#pragma once + +#include "Luau/Ast.h" +#include "Luau/DenseHash.h" + +namespace Luau +{ +namespace Compile +{ + +struct TableShape +{ + unsigned int arraySize = 0; + unsigned int hashSize = 0; +}; + +void predictTableShapes(DenseHashMap& shapes, AstNode* root); + +} // namespace Compile +} // namespace Luau diff --git a/Compiler/src/ValueTracking.cpp b/Compiler/src/ValueTracking.cpp new file mode 100644 index 00000000..0bfaf9b3 --- /dev/null +++ b/Compiler/src/ValueTracking.cpp @@ -0,0 +1,103 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +#include "ValueTracking.h" + +#include "Luau/Lexer.h" + +namespace Luau +{ +namespace Compile +{ + +struct ValueVisitor : AstVisitor +{ + DenseHashMap& globals; + DenseHashMap& variables; + + ValueVisitor(DenseHashMap& globals, DenseHashMap& variables) + : globals(globals) + , variables(variables) + { + } + + void assign(AstExpr* var) + { + if (AstExprLocal* lv = var->as()) + { + variables[lv->local].written = true; + } + else if (AstExprGlobal* gv = var->as()) + { + globals[gv->name] = Global::Written; + } + else + { + // we need to be able to track assignments in all expressions, including crazy ones like t[function() t = nil end] = 5 + var->visit(this); + } + } + + bool visit(AstStatLocal* node) override + { + for (size_t i = 0; i < node->vars.size && i < node->values.size; ++i) + variables[node->vars.data[i]].init = node->values.data[i]; + + for (size_t i = node->values.size; i < node->vars.size; ++i) + variables[node->vars.data[i]].init = nullptr; + + return true; + } + + bool visit(AstStatAssign* node) override + { + for (size_t i = 0; i < node->vars.size; ++i) + assign(node->vars.data[i]); + + for (size_t i = 0; i < node->values.size; ++i) + node->values.data[i]->visit(this); + + return false; + } + + bool visit(AstStatCompoundAssign* node) override + { + assign(node->var); + node->value->visit(this); + + return false; + } + + bool visit(AstStatLocalFunction* node) override + { + variables[node->name].init = node->func; + + return true; + } + + bool visit(AstStatFunction* node) override + { + assign(node->name); + node->func->visit(this); + + return false; + } +}; + +void assignMutable(DenseHashMap& globals, const AstNameTable& names, const char** mutableGlobals) +{ + if (AstName name = names.get("_G"); name.value) + globals[name] = Global::Mutable; + + if (mutableGlobals) + for (const char** ptr = mutableGlobals; *ptr; ++ptr) + if (AstName name = names.get(*ptr); name.value) + globals[name] = Global::Mutable; +} + +void trackValues(DenseHashMap& globals, DenseHashMap& variables, AstNode* root) +{ + ValueVisitor visitor{globals, variables}; + root->visit(&visitor); +} + +} // namespace Compile +} // namespace Luau diff --git a/Compiler/src/ValueTracking.h b/Compiler/src/ValueTracking.h new file mode 100644 index 00000000..fc74c84a --- /dev/null +++ b/Compiler/src/ValueTracking.h @@ -0,0 +1,42 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +#pragma once + +#include "Luau/Ast.h" +#include "Luau/DenseHash.h" + +namespace Luau +{ +class AstNameTable; +} + +namespace Luau +{ +namespace Compile +{ + +enum class Global +{ + Default = 0, + Mutable, // builtin that has contents unknown at compile time, blocks GETIMPORT for chains + Written, // written in the code which means we can't reason about the value +}; + +struct Variable +{ + AstExpr* init = nullptr; // initial value of the variable; filled by trackValues + bool written = false; // is the variable ever assigned to? filled by trackValues + bool constant = false; // is the variable's value a compile-time constant? filled by constantFold +}; + +void assignMutable(DenseHashMap& globals, const AstNameTable& names, const char** mutableGlobals); +void trackValues(DenseHashMap& globals, DenseHashMap& variables, AstNode* root); + +inline Global getGlobalState(const DenseHashMap& globals, AstName name) +{ + const Global* it = globals.find(name); + + return it ? *it : Global::Default; +} + +} // namespace Compile +} // namespace Luau diff --git a/Sources.cmake b/Sources.cmake index 5dd486aa..bafe7594 100644 --- a/Sources.cmake +++ b/Sources.cmake @@ -29,7 +29,15 @@ target_sources(Luau.Compiler PRIVATE Compiler/src/BytecodeBuilder.cpp Compiler/src/Compiler.cpp + Compiler/src/Builtins.cpp + Compiler/src/ConstantFolding.cpp + Compiler/src/TableShape.cpp + Compiler/src/ValueTracking.cpp Compiler/src/lcode.cpp + Compiler/src/Builtins.h + Compiler/src/ConstantFolding.h + Compiler/src/TableShape.h + Compiler/src/ValueTracking.h ) # Luau.Analysis Sources diff --git a/VM/src/ldo.cpp b/VM/src/ldo.cpp index eb47971a..3cce7665 100644 --- a/VM/src/ldo.cpp +++ b/VM/src/ldo.cpp @@ -17,7 +17,6 @@ #include -LUAU_FASTFLAGVARIABLE(LuauCcallRestoreFix, false) LUAU_FASTFLAG(LuauCoroutineClose) /* @@ -545,11 +544,8 @@ int luaD_pcall(lua_State* L, Pfunc func, void* u, ptrdiff_t old_top, ptrdiff_t e if (!oldactive) resetbit(L->stackstate, THREAD_ACTIVEBIT); - if (FFlag::LuauCcallRestoreFix) - { - // Restore nCcalls before calling the debugprotectederror callback which may rely on the proper value to have been restored. - L->nCcalls = oldnCcalls; - } + // Restore nCcalls before calling the debugprotectederror callback which may rely on the proper value to have been restored. + L->nCcalls = oldnCcalls; // an error occurred, check if we have a protected error callback if (L->global->cb.debugprotectederror) @@ -564,10 +560,6 @@ int luaD_pcall(lua_State* L, Pfunc func, void* u, ptrdiff_t old_top, ptrdiff_t e StkId oldtop = restorestack(L, old_top); luaF_close(L, oldtop); /* close eventual pending closures */ seterrorobj(L, status, oldtop); - if (!FFlag::LuauCcallRestoreFix) - { - L->nCcalls = oldnCcalls; - } L->ci = restoreci(L, old_ci); L->base = L->ci->base; restore_stack_limit(L); diff --git a/VM/src/lfunc.cpp b/VM/src/lfunc.cpp index 64878569..4178eda4 100644 --- a/VM/src/lfunc.cpp +++ b/VM/src/lfunc.cpp @@ -6,6 +6,8 @@ #include "lmem.h" #include "lgc.h" +LUAU_FASTFLAGVARIABLE(LuauNoDirectUpvalRemoval, false) + Proto* luaF_newproto(lua_State* L) { Proto* f = luaM_new(L, Proto, sizeof(Proto), L->activememcat); @@ -113,14 +115,16 @@ void luaF_freeupval(lua_State* L, UpVal* uv) void luaF_close(lua_State* L, StkId level) { UpVal* uv; - global_State* g = L->global; + global_State* g = L->global; // TODO: remove with FFlagLuauNoDirectUpvalRemoval while (L->openupval != NULL && (uv = gco2uv(L->openupval))->v >= level) { GCObject* o = obj2gco(uv); LUAU_ASSERT(!isblack(o) && uv->v != &uv->u.value); L->openupval = uv->next; /* remove from `open' list */ - if (isdead(g, o)) + if (!FFlag::LuauNoDirectUpvalRemoval && isdead(g, o)) + { luaF_freeupval(L, uv); /* free upvalue */ + } else { unlinkupval(uv); diff --git a/VM/src/lstrlib.cpp b/VM/src/lstrlib.cpp index 0b3054ae..74a8aa8a 100644 --- a/VM/src/lstrlib.cpp +++ b/VM/src/lstrlib.cpp @@ -8,8 +8,6 @@ #include #include -LUAU_FASTFLAGVARIABLE(LuauStrPackUBCastFix, false) - /* macro to `unsign' a character */ #define uchar(c) ((unsigned char)(c)) @@ -1406,20 +1404,10 @@ static int str_pack(lua_State* L) } case Kuint: { /* unsigned integers */ - if (FFlag::LuauStrPackUBCastFix) - { - long long n = (long long)luaL_checknumber(L, arg); - if (size < SZINT) /* need overflow check? */ - luaL_argcheck(L, (unsigned long long)n < ((unsigned long long)1 << (size * NB)), arg, "unsigned overflow"); - packint(&b, (unsigned long long)n, h.islittle, size, 0); - } - else - { - unsigned long long n = (unsigned long long)luaL_checknumber(L, arg); - if (size < SZINT) /* need overflow check? */ - luaL_argcheck(L, n < ((unsigned long long)1 << (size * NB)), arg, "unsigned overflow"); - packint(&b, n, h.islittle, size, 0); - } + long long n = (long long)luaL_checknumber(L, arg); + if (size < SZINT) /* need overflow check? */ + luaL_argcheck(L, (unsigned long long)n < ((unsigned long long)1 << (size * NB)), arg, "unsigned overflow"); + packint(&b, (unsigned long long)n, h.islittle, size, 0); break; } case Kfloat: diff --git a/bench/tests/sunspider/3d-cube.lua b/bench/tests/sunspider/3d-cube.lua index 6d40406c..5d162ab9 100644 --- a/bench/tests/sunspider/3d-cube.lua +++ b/bench/tests/sunspider/3d-cube.lua @@ -111,15 +111,10 @@ end -- multiplies two matrices function MMulti(M1, M2) local M = {{},{},{},{}}; - local i = 1; - local j = 1; - while i <= 4 do - j = 1; - while j <= 4 do - M[i][j] = M1[i][1] * M2[1][j] + M1[i][2] * M2[2][j] + M1[i][3] * M2[3][j] + M1[i][4] * M2[4][j]; j = j + 1 + for i = 1,4 do + for j = 1,4 do + M[i][j] = M1[i][1] * M2[1][j] + M1[i][2] * M2[2][j] + M1[i][3] * M2[3][j] + M1[i][4] * M2[4][j]; end - - i = i + 1 end return M; end @@ -127,28 +122,27 @@ end -- multiplies matrix with vector function VMulti(M, V) local Vect = {}; - local i = 1; - while i <= 4 do Vect[i] = M[i][1] * V[1] + M[i][2] * V[2] + M[i][3] * V[3] + M[i][4] * V[4]; i = i + 1 end + for i = 1,4 do + Vect[i] = M[i][1] * V[1] + M[i][2] * V[2] + M[i][3] * V[3] + M[i][4] * V[4]; + end return Vect; end function VMulti2(M, V) local Vect = {}; - local i = 1; - while i < 4 do Vect[i] = M[i][1] * V[1] + M[i][2] * V[2] + M[i][3] * V[3]; i = i + 1 end + for i = 1,3 do + Vect[i] = M[i][1] * V[1] + M[i][2] * V[2] + M[i][3] * V[3]; + end return Vect; end -- add to matrices function MAdd(M1, M2) local M = {{},{},{},{}}; - local i = 1; - local j = 1; - while i <= 4 do - j = 1; - while j <= 4 do M[i][j] = M1[i][j] + M2[i][j]; j = j + 1 end - - i = i + 1 + for i = 1,4 do + for j = 1,4 do + M[i][j] = M1[i][j] + M2[i][j]; + end end return M; end diff --git a/tests/Autocomplete.test.cpp b/tests/Autocomplete.test.cpp index 210db7ee..211e1be1 100644 --- a/tests/Autocomplete.test.cpp +++ b/tests/Autocomplete.test.cpp @@ -1938,7 +1938,6 @@ return target(b@1 TEST_CASE_FIXTURE(ACFixture, "function_in_assignment_has_parentheses") { ScopedFastFlag luauAutocompleteAvoidMutation("LuauAutocompleteAvoidMutation", true); - ScopedFastFlag luauAutocompletePreferToCallFunctions("LuauAutocompletePreferToCallFunctions", true); check(R"( local function bar(a: number) return -a end @@ -1954,7 +1953,6 @@ local abc = b@1 TEST_CASE_FIXTURE(ACFixture, "function_result_passed_to_function_has_parentheses") { ScopedFastFlag luauAutocompleteAvoidMutation("LuauAutocompleteAvoidMutation", true); - ScopedFastFlag luauAutocompletePreferToCallFunctions("LuauAutocompletePreferToCallFunctions", true); check(R"( local function foo() return 1 end @@ -2538,10 +2536,6 @@ TEST_CASE("autocomplete_documentation_symbols") TEST_CASE_FIXTURE(ACFixture, "autocomplete_ifelse_expressions") { - ScopedFastFlag sff1{"LuauIfElseExpressionBaseSupport", true}; - ScopedFastFlag sff2{"LuauIfElseExpressionAnalysisSupport", true}; - - { check(R"( local temp = false local even = true; @@ -2614,7 +2608,6 @@ a = if temp then even elseif true then temp else e@9 CHECK(ac.entryMap.count("then") == 0); CHECK(ac.entryMap.count("else") == 0); CHECK(ac.entryMap.count("elseif") == 0); - } } TEST_CASE_FIXTURE(ACFixture, "autocomplete_explicit_type_pack") @@ -2681,4 +2674,58 @@ local r4 = t:bar1(@4) CHECK(ac.entryMap["foo2"].typeCorrect == TypeCorrectKind::None); } +TEST_CASE_FIXTURE(ACFixture, "autocomplete_default_type_parameters") +{ + ScopedFastFlag luauParseTypeAliasDefaults{"LuauParseTypeAliasDefaults", true}; + + check(R"( +type A = () -> T + )"); + + auto ac = autocomplete('1'); + + CHECK(ac.entryMap.count("number")); + CHECK(ac.entryMap.count("string")); +} + +TEST_CASE_FIXTURE(ACFixture, "autocomplete_default_type_pack_parameters") +{ + ScopedFastFlag luauParseTypeAliasDefaults{"LuauParseTypeAliasDefaults", true}; + + check(R"( +type A = () -> T + )"); + + auto ac = autocomplete('1'); + + CHECK(ac.entryMap.count("number")); + CHECK(ac.entryMap.count("string")); +} + +TEST_CASE_FIXTURE(ACFixture, "autocomplete_oop_implicit_self") +{ + ScopedFastFlag flag("LuauMissingFollowACMetatables", true); + check(R"( +--!strict +local Class = {} +Class.__index = Class +type Class = typeof(setmetatable({} :: { x: number }, Class)) +function Class.new(x: number): Class + return setmetatable({x = x}, Class) +end +function Class.getx(self: Class) + return self.x +end +function test() + local c = Class.new(42) + local n = c:@1 + print(n) +end + )"); + + auto ac = autocomplete('1'); + + CHECK(ac.entryMap.count("getx")); +} + TEST_SUITE_END(); diff --git a/tests/Compiler.test.cpp b/tests/Compiler.test.cpp index 95811b3f..8eed953f 100644 --- a/tests/Compiler.test.cpp +++ b/tests/Compiler.test.cpp @@ -603,9 +603,9 @@ RETURN R0 1 )"); } -TEST_CASE("EmptyTableHashSizePredictionOptimization") +TEST_CASE("TableSizePredictionBasic") { - const char* hashSizeSource = R"( + CHECK_EQ("\n" + compileFunction0(R"( local t = {} t.a = 1 t.b = 1 @@ -616,36 +616,8 @@ t.f = 1 t.g = 1 t.h = 1 t.i = 1 -)"; - - const char* hashSizeSource2 = R"( -local t = {} -t.x = 1 -t.x = 2 -t.x = 3 -t.x = 4 -t.x = 5 -t.x = 6 -t.x = 7 -t.x = 8 -t.x = 9 -)"; - - const char* arraySizeSource = R"( -local t = {} -t[1] = 1 -t[2] = 1 -t[3] = 1 -t[4] = 1 -t[5] = 1 -t[6] = 1 -t[7] = 1 -t[8] = 1 -t[9] = 1 -t[10] = 1 -)"; - - CHECK_EQ("\n" + compileFunction0(hashSizeSource), R"( +)"), + R"( NEWTABLE R0 16 0 LOADN R1 1 SETTABLEKS R1 R0 K0 @@ -668,7 +640,19 @@ SETTABLEKS R1 R0 K8 RETURN R0 0 )"); - CHECK_EQ("\n" + compileFunction0(hashSizeSource2), R"( + CHECK_EQ("\n" + compileFunction0(R"( +local t = {} +t.x = 1 +t.x = 2 +t.x = 3 +t.x = 4 +t.x = 5 +t.x = 6 +t.x = 7 +t.x = 8 +t.x = 9 +)"), + R"( NEWTABLE R0 1 0 LOADN R1 1 SETTABLEKS R1 R0 K0 @@ -691,7 +675,20 @@ SETTABLEKS R1 R0 K0 RETURN R0 0 )"); - CHECK_EQ("\n" + compileFunction0(arraySizeSource), R"( + CHECK_EQ("\n" + compileFunction0(R"( +local t = {} +t[1] = 1 +t[2] = 1 +t[3] = 1 +t[4] = 1 +t[5] = 1 +t[6] = 1 +t[7] = 1 +t[8] = 1 +t[9] = 1 +t[10] = 1 +)"), + R"( NEWTABLE R0 0 10 LOADN R1 1 SETTABLEN R1 R0 1 @@ -717,6 +714,27 @@ RETURN R0 0 )"); } +TEST_CASE("TableSizePredictionObject") +{ + CHECK_EQ("\n" + compileFunction(R"( +local t = {} +t.field = 1 +function t:getfield() + return self.field +end +return t +)", + 1), + R"( +NEWTABLE R0 2 0 +LOADN R1 1 +SETTABLEKS R1 R0 K0 +DUPCLOSURE R1 K1 +SETTABLEKS R1 R0 K2 +RETURN R0 1 +)"); +} + TEST_CASE("TableSizePredictionSetMetatable") { CHECK_EQ("\n" + compileFunction0(R"( @@ -1031,9 +1049,6 @@ RETURN R0 1 TEST_CASE("IfElseExpression") { - ScopedFastFlag sff1{"LuauIfElseExpressionBaseSupport", true}; - ScopedFastFlag sff2{"LuauIfElseExpressionAnalysisSupport", true}; - // codegen for a true constant condition CHECK_EQ("\n" + compileFunction0("return if true then 10 else 20"), R"( LOADN R0 10 @@ -3058,7 +3073,7 @@ RETURN R0 0 // table variants (indexed by string, number, variable) CHECK_EQ("\n" + compileFunction0("local a = {} a.foo += 5"), R"( -NEWTABLE R0 1 0 +NEWTABLE R0 0 0 GETTABLEKS R1 R0 K0 ADDK R1 R1 K1 SETTABLEKS R1 R0 K0 @@ -3066,7 +3081,7 @@ RETURN R0 0 )"); CHECK_EQ("\n" + compileFunction0("local a = {} a[1] += 5"), R"( -NEWTABLE R0 0 1 +NEWTABLE R0 0 0 GETTABLEN R1 R0 1 ADDK R1 R1 K0 SETTABLEN R1 R0 1 diff --git a/tests/Conformance.test.cpp b/tests/Conformance.test.cpp index 663b329e..5222af33 100644 --- a/tests/Conformance.test.cpp +++ b/tests/Conformance.test.cpp @@ -366,15 +366,11 @@ TEST_CASE("PCall") TEST_CASE("Pack") { - ScopedFastFlag sff{"LuauStrPackUBCastFix", true}; - runConformance("tpack.lua"); } TEST_CASE("Vector") { - ScopedFastFlag sff{"LuauIfElseExpressionBaseSupport", true}; - lua_CompileOptions copts = {}; copts.optimizationLevel = 1; copts.debugLevel = 1; @@ -861,15 +857,11 @@ TEST_CASE("ExceptionObject") TEST_CASE("IfElseExpression") { - ScopedFastFlag sff{"LuauIfElseExpressionBaseSupport", true}; - runConformance("ifelseexpr.lua"); } TEST_CASE("TagMethodError") { - ScopedFastFlag sff{"LuauCcallRestoreFix", true}; - runConformance("tmerror.lua", [](lua_State* L) { auto* cb = lua_callbacks(L); diff --git a/tests/Fixture.cpp b/tests/Fixture.cpp index ca4281a0..c74bfa27 100644 --- a/tests/Fixture.cpp +++ b/tests/Fixture.cpp @@ -191,7 +191,7 @@ ParseResult Fixture::tryParse(const std::string& source, const ParseOptions& par return result; } -ParseResult Fixture::matchParseError(const std::string& source, const std::string& message) +ParseResult Fixture::matchParseError(const std::string& source, const std::string& message, std::optional location) { ParseOptions options; options.allowDeclarationSyntax = true; @@ -203,6 +203,9 @@ ParseResult Fixture::matchParseError(const std::string& source, const std::strin CHECK_EQ(result.errors.front().getMessage(), message); + if (location) + CHECK_EQ(result.errors.front().getLocation(), *location); + return result; } diff --git a/tests/Fixture.h b/tests/Fixture.h index e01632ea..ab852ef6 100644 --- a/tests/Fixture.h +++ b/tests/Fixture.h @@ -106,7 +106,7 @@ struct Fixture /// Parse with all language extensions enabled ParseResult parseEx(const std::string& source, const ParseOptions& parseOptions = {}); ParseResult tryParse(const std::string& source, const ParseOptions& parseOptions = {}); - ParseResult matchParseError(const std::string& source, const std::string& message); + ParseResult matchParseError(const std::string& source, const std::string& message, std::optional location = std::nullopt); // Verify a parse error occurs and the parse error message has the specified prefix ParseResult matchParseErrorPrefix(const std::string& source, const std::string& prefix); diff --git a/tests/Parser.test.cpp b/tests/Parser.test.cpp index 5abcb09a..e9135651 100644 --- a/tests/Parser.test.cpp +++ b/tests/Parser.test.cpp @@ -1255,7 +1255,6 @@ TEST_CASE_FIXTURE(Fixture, "parse_error_with_too_many_nested_type_group") TEST_CASE_FIXTURE(Fixture, "parse_error_with_too_many_nested_if_statements") { ScopedFastInt sfis{"LuauRecursionLimit", 10}; - ScopedFastFlag sff{"LuauIfStatementRecursionGuard", true}; matchParseErrorPrefix( "function f() if true then if true then if true then if true then if true then if true then if true then if true then if true " @@ -1266,7 +1265,6 @@ TEST_CASE_FIXTURE(Fixture, "parse_error_with_too_many_nested_if_statements") TEST_CASE_FIXTURE(Fixture, "parse_error_with_too_many_changed_elseif_statements") { ScopedFastInt sfis{"LuauRecursionLimit", 10}; - ScopedFastFlag sff{"LuauIfStatementRecursionGuard", true}; matchParseErrorPrefix( "function f() if false then elseif false then elseif false then elseif false then elseif false then elseif false then elseif " @@ -1276,7 +1274,6 @@ TEST_CASE_FIXTURE(Fixture, "parse_error_with_too_many_changed_elseif_statements" TEST_CASE_FIXTURE(Fixture, "parse_error_with_too_many_nested_ifelse_expressions1") { - ScopedFastFlag sff{"LuauIfElseExpressionBaseSupport", true}; ScopedFastInt sfis{"LuauRecursionLimit", 10}; matchParseError("function f() return if true then 1 elseif true then 2 elseif true then 3 elseif true then 4 elseif true then 5 elseif true then " @@ -1286,7 +1283,6 @@ TEST_CASE_FIXTURE(Fixture, "parse_error_with_too_many_nested_ifelse_expressions1 TEST_CASE_FIXTURE(Fixture, "parse_error_with_too_many_nested_ifelse_expressions2") { - ScopedFastFlag sff{"LuauIfElseExpressionBaseSupport", true}; ScopedFastInt sfis{"LuauRecursionLimit", 10}; matchParseError( @@ -1962,6 +1958,37 @@ TEST_CASE_FIXTURE(Fixture, "function_type_named_arguments") matchParseError("type MyFunc = (number) -> (d: number) -> number", "Expected '->' when parsing function type, got '<'"); } +TEST_CASE_FIXTURE(Fixture, "function_type_matching_parenthesis") +{ + matchParseError("local a: (number -> string", "Expected ')' (to close '(' at column 13), got '->'"); +} + +TEST_CASE_FIXTURE(Fixture, "parse_type_alias_default_type") +{ + ScopedFastFlag luauParseTypeAliasDefaults{"LuauParseTypeAliasDefaults", true}; + + AstStat* stat = parse(R"( +type A = {} +type B = {} +type C = {} +type D = {} +type E = {} +type F = (T...) -> U... +type G = (U...) -> T... + )"); + + REQUIRE(stat != nullptr); +} + +TEST_CASE_FIXTURE(Fixture, "parse_type_alias_default_type_errors") +{ + ScopedFastFlag luauParseTypeAliasDefaults{"LuauParseTypeAliasDefaults", true}; + + matchParseError("type Y = {}", "Expected default type after type name", Location{{0, 20}, {0, 21}}); + matchParseError("type Y = {}", "Expected default type pack after type pack name", Location{{0, 29}, {0, 30}}); + matchParseError("type Y number> = {}", "Expected type pack after '=', got type", Location{{0, 14}, {0, 32}}); +} + TEST_SUITE_END(); TEST_SUITE_BEGIN("ParseErrorRecovery"); @@ -2455,10 +2482,19 @@ do end CHECK_EQ(1, result.errors.size()); } +TEST_CASE_FIXTURE(Fixture, "recover_expected_type_pack") +{ + ScopedFastFlag luauParseTypeAliasDefaults{"LuauParseTypeAliasDefaults", true}; + ScopedFastFlag luauParseRecoverTypePackEllipsis{"LuauParseRecoverTypePackEllipsis", true}; + + ParseResult result = tryParse(R"( +type Y = (T...) -> U... + )"); + CHECK_EQ(1, result.errors.size()); +} + TEST_CASE_FIXTURE(Fixture, "parse_if_else_expression") { - ScopedFastFlag sff{"LuauIfElseExpressionBaseSupport", true}; - { AstStat* stat = parse("return if true then 1 else 2"); @@ -2524,9 +2560,4 @@ type C = Packed<(number, X...)> REQUIRE(stat != nullptr); } -TEST_CASE_FIXTURE(Fixture, "function_type_matching_parenthesis") -{ - matchParseError("local a: (number -> string", "Expected ')' (to close '(' at column 13), got '->'"); -} - TEST_SUITE_END(); diff --git a/tests/ToString.test.cpp b/tests/ToString.test.cpp index 80a258f5..445ee532 100644 --- a/tests/ToString.test.cpp +++ b/tests/ToString.test.cpp @@ -338,6 +338,8 @@ TEST_CASE_FIXTURE(Fixture, "toStringDetailed") TEST_CASE_FIXTURE(Fixture, "toStringDetailed2") { + ScopedFastFlag sff{"LuauUnsealedTableLiteral", true}; + CheckResult result = check(R"( local base = {} function base:one() return 1 end @@ -353,7 +355,7 @@ TEST_CASE_FIXTURE(Fixture, "toStringDetailed2") TypeId tType = requireType("inst"); ToStringResult r = toStringDetailed(tType); - CHECK_EQ("{ @metatable {| __index: { @metatable {| __index: base |}, child } |}, inst }", r.name); + CHECK_EQ("{ @metatable { __index: { @metatable { __index: base }, child } }, inst }", r.name); CHECK_EQ(0, r.nameMap.typeVars.size()); ToStringOptions opts; @@ -500,6 +502,24 @@ TEST_CASE_FIXTURE(Fixture, "toStringNamedFunction_map") CHECK_EQ("map(arr: {a}, fn: (a) -> b): {b}", toStringNamedFunction("map", *ftv)); } +TEST_CASE_FIXTURE(Fixture, "toStringNamedFunction_generic_pack") +{ + ScopedFastFlag luauTypeAliasDefaults{"LuauTypeAliasDefaults", true}; + + CheckResult result = check(R"( + local function f(a: number, b: string) end + local function test(...: T...): U... + f(...) + return 1, 2, 3 + end + )"); + + TypeId ty = requireType("test"); + const FunctionTypeVar* ftv = get(follow(ty)); + + CHECK_EQ("test(...: T...): U...", toStringNamedFunction("test", *ftv)); +} + TEST_CASE("toStringNamedFunction_unit_f") { TypePackVar empty{TypePack{}}; diff --git a/tests/Transpiler.test.cpp b/tests/Transpiler.test.cpp index 47c3883c..ac5be859 100644 --- a/tests/Transpiler.test.cpp +++ b/tests/Transpiler.test.cpp @@ -421,8 +421,6 @@ TEST_CASE_FIXTURE(Fixture, "transpile_type_assertion") TEST_CASE_FIXTURE(Fixture, "transpile_if_then_else") { - ScopedFastFlag luauIfElseExpressionBaseSupport("LuauIfElseExpressionBaseSupport", true); - std::string code = "local a = if 1 then 2 else 3"; CHECK_EQ(code, transpile(code).code); @@ -641,4 +639,16 @@ TEST_CASE_FIXTURE(Fixture, "transpile_to_string") CHECK_EQ("'hello'", toString(expr)); } +TEST_CASE_FIXTURE(Fixture, "transpile_type_alias_default_type_parameters") +{ + ScopedFastFlag luauParseTypeAliasDefaults{"LuauParseTypeAliasDefaults", true}; + ScopedFastFlag luauTypeAliasDefaults{"LuauTypeAliasDefaults", true}; + + std::string code = R"( +type Packed = (T, U, V...)->(W...) +local a: Packed + )"; + + CHECK_EQ(code, transpile(code, {}, true).code); +} TEST_SUITE_END(); diff --git a/tests/TypeInfer.annotations.test.cpp b/tests/TypeInfer.annotations.test.cpp index 275782b3..86165814 100644 --- a/tests/TypeInfer.annotations.test.cpp +++ b/tests/TypeInfer.annotations.test.cpp @@ -497,7 +497,7 @@ TEST_CASE_FIXTURE(Fixture, "generic_aliases_are_cloned_properly") CHECK(arrayTable->indexer); CHECK(isInArena(array.type, mod.interfaceTypes)); - CHECK_EQ(array.typeParams[0], arrayTable->indexer->indexResultType); + CHECK_EQ(array.typeParams[0].ty, arrayTable->indexer->indexResultType); } TEST_CASE_FIXTURE(Fixture, "cloned_interface_maintains_pointers_between_definitions") diff --git a/tests/TypeInfer.refinements.test.cpp b/tests/TypeInfer.refinements.test.cpp index 503b613f..d76b920b 100644 --- a/tests/TypeInfer.refinements.test.cpp +++ b/tests/TypeInfer.refinements.test.cpp @@ -1031,9 +1031,6 @@ TEST_CASE_FIXTURE(Fixture, "refine_the_correct_types_opposite_of_when_a_is_not_n TEST_CASE_FIXTURE(Fixture, "is_truthy_constraint_ifelse_expression") { - ScopedFastFlag sff1{"LuauIfElseExpressionBaseSupport", true}; - ScopedFastFlag sff2{"LuauIfElseExpressionAnalysisSupport", true}; - CheckResult result = check(R"( function f(v:string?) return if v then v else tostring(v) @@ -1048,9 +1045,6 @@ TEST_CASE_FIXTURE(Fixture, "is_truthy_constraint_ifelse_expression") TEST_CASE_FIXTURE(Fixture, "invert_is_truthy_constraint_ifelse_expression") { - ScopedFastFlag sff1{"LuauIfElseExpressionBaseSupport", true}; - ScopedFastFlag sff2{"LuauIfElseExpressionAnalysisSupport", true}; - CheckResult result = check(R"( function f(v:string?) return if not v then tostring(v) else v @@ -1065,9 +1059,6 @@ TEST_CASE_FIXTURE(Fixture, "invert_is_truthy_constraint_ifelse_expression") TEST_CASE_FIXTURE(Fixture, "type_comparison_ifelse_expression") { - ScopedFastFlag sff1{"LuauIfElseExpressionBaseSupport", true}; - ScopedFastFlag sff2{"LuauIfElseExpressionAnalysisSupport", true}; - CheckResult result = check(R"( function returnOne(x) return 1 @@ -1119,6 +1110,25 @@ TEST_CASE_FIXTURE(Fixture, "correctly_lookup_property_whose_base_was_previously_ CHECK_EQ("string", toString(requireTypeAtPosition({5, 30}))); } +TEST_CASE_FIXTURE(Fixture, "correctly_lookup_property_whose_base_was_previously_refined2") +{ + ScopedFastFlag sff{"LuauLValueAsKey", true}; + + CheckResult result = check(R"( + type T = { x: { y: number }? } + + local function f(t: T?) + if t and t.x then + local foo = t.x.y + end + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ("number", toString(requireTypeAtPosition({5, 32}))); +} + TEST_CASE_FIXTURE(Fixture, "apply_refinements_on_astexprindexexpr_whose_subscript_expr_is_constant_string") { ScopedFastFlag sff{"LuauRefiLookupFromIndexExpr", true}; diff --git a/tests/TypeInfer.singletons.test.cpp b/tests/TypeInfer.singletons.test.cpp index 68dc1b4f..94cfb643 100644 --- a/tests/TypeInfer.singletons.test.cpp +++ b/tests/TypeInfer.singletons.test.cpp @@ -360,6 +360,7 @@ TEST_CASE_FIXTURE(Fixture, "table_properties_type_error_escapes") { ScopedFastFlag sffs[] = { {"LuauParseSingletonTypes", true}, + {"LuauUnsealedTableLiteral", true}, }; CheckResult result = check(R"( @@ -369,7 +370,7 @@ TEST_CASE_FIXTURE(Fixture, "table_properties_type_error_escapes") )"); LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK_EQ(R"(Table type '{| ["\n"]: number |}' not compatible with type '{| ["<>"]: number |}' because the former is missing field '<>')", + CHECK_EQ(R"(Table type '{ ["\n"]: number }' not compatible with type '{| ["<>"]: number |}' because the former is missing field '<>')", toString(result.errors[0])); } @@ -423,4 +424,27 @@ caused by: toString(result.errors[0])); } +TEST_CASE_FIXTURE(Fixture, "if_then_else_expression_singleton_options") +{ + ScopedFastFlag sffs[] = { + {"LuauSingletonTypes", true}, + {"LuauParseSingletonTypes", true}, + {"LuauUnionHeuristic", true}, + {"LuauExpectedTypesOfProperties", true}, + {"LuauExtendedUnionMismatchError", true}, + {"LuauIfElseExpectedType2", true}, + {"LuauIfElseBranchTypeUnion", true}, + }; + + CheckResult result = check(R"( +type Cat = { tag: 'cat', catfood: string } +type Dog = { tag: 'dog', dogfood: string } +type Animal = Cat | Dog + +local a: Animal = if true then { tag = 'cat', catfood = 'something' } else { tag = 'dog', dogfood = 'other' } + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.tables.test.cpp b/tests/TypeInfer.tables.test.cpp index 80f40407..27cda146 100644 --- a/tests/TypeInfer.tables.test.cpp +++ b/tests/TypeInfer.tables.test.cpp @@ -65,7 +65,7 @@ TEST_CASE_FIXTURE(Fixture, "augment_nested_table") TEST_CASE_FIXTURE(Fixture, "cannot_augment_sealed_table") { - CheckResult result = check("local t = {prop=999} t.foo = 'bar'"); + CheckResult result = check("function mkt() return {prop=999} end local t = mkt() t.foo = 'bar'"); LUAU_REQUIRE_ERROR_COUNT(1, result); TypeError& err = result.errors[0]; @@ -77,7 +77,7 @@ TEST_CASE_FIXTURE(Fixture, "cannot_augment_sealed_table") CHECK_EQ(s, "{| prop: number |}"); CHECK_EQ(error->prop, "foo"); CHECK_EQ(error->context, CannotExtendTable::Property); - CHECK_EQ(err.location, (Location{Position{0, 24}, Position{0, 29}})); + CHECK_EQ(err.location, (Location{Position{0, 59}, Position{0, 64}})); } TEST_CASE_FIXTURE(Fixture, "dont_seal_an_unsealed_table_by_passing_it_to_a_function_that_takes_a_sealed_table") @@ -1155,7 +1155,8 @@ TEST_CASE_FIXTURE(Fixture, "defining_a_self_method_for_a_builtin_sealed_table_mu TEST_CASE_FIXTURE(Fixture, "defining_a_method_for_a_local_sealed_table_must_fail") { CheckResult result = check(R"( - local t = {x = 1} + function mkt() return {x = 1} end + local t = mkt() function t.m() end )"); @@ -1165,13 +1166,38 @@ TEST_CASE_FIXTURE(Fixture, "defining_a_method_for_a_local_sealed_table_must_fail TEST_CASE_FIXTURE(Fixture, "defining_a_self_method_for_a_local_sealed_table_must_fail") { CheckResult result = check(R"( - local t = {x = 1} + function mkt() return {x = 1} end + local t = mkt() function t:m() end )"); LUAU_REQUIRE_ERROR_COUNT(1, result); } +TEST_CASE_FIXTURE(Fixture, "defining_a_method_for_a_local_unsealed_table_is_ok") +{ + ScopedFastFlag sff{"LuauUnsealedTableLiteral", true}; + + CheckResult result = check(R"( + local t = {x = 1} + function t.m() end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "defining_a_self_method_for_a_local_unsealed_table_is_ok") +{ + ScopedFastFlag sff{"LuauUnsealedTableLiteral", true}; + + CheckResult result = check(R"( + local t = {x = 1} + function t:m() end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + // This unit test could be flaky if the fix has regressed. TEST_CASE_FIXTURE(Fixture, "pass_incompatible_union_to_a_generic_table_without_crashing") { @@ -1439,8 +1465,13 @@ TEST_CASE_FIXTURE(Fixture, "right_table_missing_key2") CHECK_EQ("{| |}", toString(mp->subType)); } -TEST_CASE_FIXTURE(Fixture, "casting_tables_with_props_into_table_with_indexer") +TEST_CASE_FIXTURE(Fixture, "casting_unsealed_tables_with_props_into_table_with_indexer") { + ScopedFastFlag sff[]{ + {"LuauTableSubtypingVariance2", true}, + {"LuauUnsealedTableLiteral", true}, + }; + CheckResult result = check(R"( type StringToStringMap = { [string]: string } local rt: StringToStringMap = { ["foo"] = 1 } @@ -1448,6 +1479,25 @@ TEST_CASE_FIXTURE(Fixture, "casting_tables_with_props_into_table_with_indexer") LUAU_REQUIRE_ERROR_COUNT(1, result); + ToStringOptions o{/* exhaustive= */ true}; + TypeMismatch* tm = get(result.errors[0]); + REQUIRE(tm); + CHECK_EQ("{| [string]: string |}", toString(tm->wantedType, o)); + // Should t now have an indexer? + // It would if the assignment to rt was correctly typed. + CHECK_EQ("{ [string]: string, foo: number }", toString(tm->givenType, o)); +} + +TEST_CASE_FIXTURE(Fixture, "casting_sealed_tables_with_props_into_table_with_indexer") +{ + CheckResult result = check(R"( + type StringToStringMap = { [string]: string } + function mkrt() return { ["foo"] = 1 } end + local rt: StringToStringMap = mkrt() + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + ToStringOptions o{/* exhaustive= */ true}; TypeMismatch* tm = get(result.errors[0]); REQUIRE(tm); @@ -1467,7 +1517,10 @@ TEST_CASE_FIXTURE(Fixture, "casting_tables_with_props_into_table_with_indexer2") TEST_CASE_FIXTURE(Fixture, "casting_tables_with_props_into_table_with_indexer3") { - ScopedFastFlag sff{"LuauTableSubtypingVariance2", true}; + ScopedFastFlag sff[]{ + {"LuauTableSubtypingVariance2", true}, + {"LuauUnsealedTableLiteral", true}, + }; CheckResult result = check(R"( local function foo(a: {[string]: number, a: string}) end @@ -1480,7 +1533,7 @@ TEST_CASE_FIXTURE(Fixture, "casting_tables_with_props_into_table_with_indexer3") TypeMismatch* tm = get(result.errors[0]); REQUIRE(tm); CHECK_EQ("{| [string]: number, a: string |}", toString(tm->wantedType, o)); - CHECK_EQ("{| a: number |}", toString(tm->givenType, o)); + CHECK_EQ("{ a: number }", toString(tm->givenType, o)); } TEST_CASE_FIXTURE(Fixture, "casting_tables_with_props_into_table_with_indexer4") @@ -1536,8 +1589,11 @@ TEST_CASE_FIXTURE(Fixture, "table_subtyping_with_missing_props_dont_report_multi TEST_CASE_FIXTURE(Fixture, "table_subtyping_with_extra_props_dont_report_multiple_errors") { CheckResult result = check(R"( - local vec3 = {{x = 1, y = 2, z = 3}} - local vec1 = {{x = 1}} + function mkvec3() return {x = 1, y = 2, z = 3} end + function mkvec1() return {x = 1} end + + local vec3 = {mkvec3()} + local vec1 = {mkvec1()} vec1 = vec3 )"); @@ -1620,7 +1676,8 @@ TEST_CASE_FIXTURE(Fixture, "reasonable_error_when_adding_a_nonexistent_property_ { CheckResult result = check(R"( --!strict - local A = {"value"} + function mkA() return {"value"} end + local A = mkA() A.B = "Hello" )"); @@ -1668,7 +1725,8 @@ TEST_CASE_FIXTURE(Fixture, "hide_table_error_properties") --!strict local function f() - local t = { x = 1 } + local function mkt() return { x = 1 } end + local t = mkt() function t.a() end function t.b() end @@ -1995,7 +2053,10 @@ caused by: TEST_CASE_FIXTURE(Fixture, "error_detailed_metatable_prop") { - ScopedFastFlag LuauTableSubtypingVariance2{"LuauTableSubtypingVariance2", true}; // Only for new path + ScopedFastFlag sff[]{ + {"LuauTableSubtypingVariance2", true}, + {"LuauUnsealedTableLiteral", true}, + }; CheckResult result = check(R"( local a1 = setmetatable({ x = 2, y = 3 }, { __call = function(s) end }); @@ -2010,7 +2071,7 @@ local c2: typeof(a2) = b2 LUAU_REQUIRE_ERROR_COUNT(2, result); CHECK_EQ(toString(result.errors[0]), R"(Type 'b1' could not be converted into 'a1' caused by: - Type '{| x: number, y: string |}' could not be converted into '{| x: number, y: number |}' + Type '{ x: number, y: string }' could not be converted into '{ x: number, y: number }' caused by: Property 'y' is not compatible. Type 'string' could not be converted into 'number')"); @@ -2018,7 +2079,7 @@ caused by: { CHECK_EQ(toString(result.errors[1]), R"(Type 'b2' could not be converted into 'a2' caused by: - Type '{| __call: (a, b) -> () |}' could not be converted into '{| __call: (a) -> () |}' + Type '{ __call: (a, b) -> () }' could not be converted into '{ __call: (a) -> () }' caused by: Property '__call' is not compatible. Type '(a, b) -> ()' could not be converted into '(a) -> ()'; different number of generic type parameters)"); } @@ -2026,7 +2087,7 @@ caused by: { CHECK_EQ(toString(result.errors[1]), R"(Type 'b2' could not be converted into 'a2' caused by: - Type '{| __call: (a, b) -> () |}' could not be converted into '{| __call: (a) -> () |}' + Type '{ __call: (a, b) -> () }' could not be converted into '{ __call: (a) -> () }' caused by: Property '__call' is not compatible. Type '(a, b) -> ()' could not be converted into '(a) -> ()')"); } @@ -2059,6 +2120,7 @@ TEST_CASE_FIXTURE(Fixture, "explicitly_typed_table_error") {"LuauPropertiesGetExpectedType", true}, {"LuauExpectedTypesOfProperties", true}, {"LuauTableSubtypingVariance2", true}, + {"LuauUnsealedTableLiteral", true}, }; CheckResult result = check(R"( @@ -2077,7 +2139,7 @@ local y: number = tmp.p.y LUAU_REQUIRE_ERROR_COUNT(1, result); CHECK_EQ(toString(result.errors[0]), R"(Type 'tmp' could not be converted into 'HasSuper' caused by: - Property 'p' is not compatible. Table type '{| x: number, y: number |}' not compatible with type 'Super' because the former has extra field 'y')"); + Property 'p' is not compatible. Table type '{ x: number, y: number }' not compatible with type 'Super' because the former has extra field 'y')"); } TEST_CASE_FIXTURE(Fixture, "explicitly_typed_table_with_indexer") @@ -2103,7 +2165,10 @@ a.p = { x = 9 } TEST_CASE_FIXTURE(Fixture, "recursive_metatable_type_call") { - ScopedFastFlag luauFixRecursiveMetatableCall{"LuauFixRecursiveMetatableCall", true}; + ScopedFastFlag sff[]{ + {"LuauFixRecursiveMetatableCall", true}, + {"LuauUnsealedTableLiteral", true}, + }; CheckResult result = check(R"( local b @@ -2112,7 +2177,7 @@ b() )"); LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK_EQ(toString(result.errors[0]), R"(Cannot call non-function t1 where t1 = { @metatable {| __call: t1 |}, { } })"); + CHECK_EQ(toString(result.errors[0]), R"(Cannot call non-function t1 where t1 = { @metatable { __call: t1 }, { } })"); } TEST_SUITE_END(); diff --git a/tests/TypeInfer.test.cpp b/tests/TypeInfer.test.cpp index f70f3b1c..7a056af5 100644 --- a/tests/TypeInfer.test.cpp +++ b/tests/TypeInfer.test.cpp @@ -4525,7 +4525,9 @@ f(function(x) print(x) end) } TEST_CASE_FIXTURE(Fixture, "infer_generic_function_function_argument") -{ +{ + ScopedFastFlag sff{"LuauUnsealedTableLiteral", true}; + CheckResult result = check(R"( local function sum(x: a, y: a, f: (a, a) -> a) return f(x, y) end return sum(2, 3, function(a, b) return a + b end) @@ -4549,7 +4551,7 @@ local r = foldl(a, {s=0,c=0}, function(a, b) return {s = a.s + b, c = a.c + 1} e )"); LUAU_REQUIRE_NO_ERRORS(result); - REQUIRE_EQ("{| c: number, s: number |}", toString(requireType("r"))); + REQUIRE_EQ("{ c: number, s: number }", toString(requireType("r"))); } TEST_CASE_FIXTURE(Fixture, "infer_generic_function_function_argument_overloaded") @@ -4689,6 +4691,18 @@ a = setmetatable(a, { __call = function(x) end }) LUAU_REQUIRE_NO_ERRORS(result); } +TEST_CASE_FIXTURE(Fixture, "infer_through_group_expr") +{ + ScopedFastFlag luauGroupExpectedType{"LuauGroupExpectedType", true}; + + CheckResult result = check(R"( +local function f(a: (number, number) -> number) return a(1, 3) end +f(((function(a, b) return a + b end))) + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + TEST_CASE_FIXTURE(Fixture, "refine_and_or") { CheckResult result = check(R"( @@ -4743,46 +4757,75 @@ local c: X TEST_CASE_FIXTURE(Fixture, "tc_if_else_expressions1") { - ScopedFastFlag sff1{"LuauIfElseExpressionBaseSupport", true}; - ScopedFastFlag sff2{"LuauIfElseExpressionAnalysisSupport", true}; - - { - CheckResult result = check(R"(local a = if true then "true" else "false")"); - LUAU_REQUIRE_NO_ERRORS(result); - TypeId aType = requireType("a"); - CHECK_EQ(getPrimitiveType(aType), PrimitiveTypeVar::String); - } + CheckResult result = check(R"(local a = if true then "true" else "false")"); + LUAU_REQUIRE_NO_ERRORS(result); + TypeId aType = requireType("a"); + CHECK_EQ(getPrimitiveType(aType), PrimitiveTypeVar::String); } TEST_CASE_FIXTURE(Fixture, "tc_if_else_expressions2") { - ScopedFastFlag sff1{"LuauIfElseExpressionBaseSupport", true}; - ScopedFastFlag sff2{"LuauIfElseExpressionAnalysisSupport", true}; + // Test expression containing elseif + CheckResult result = check(R"( +local a = if false then "a" elseif false then "b" else "c" + )"); + LUAU_REQUIRE_NO_ERRORS(result); + TypeId aType = requireType("a"); + CHECK_EQ(getPrimitiveType(aType), PrimitiveTypeVar::String); +} + +TEST_CASE_FIXTURE(Fixture, "tc_if_else_expressions_type_union") +{ + ScopedFastFlag sff3{"LuauIfElseBranchTypeUnion", true}; { - // Test expression containing elseif - CheckResult result = check(R"( -local a = if false then "a" elseif false then "b" else "c" - )"); + CheckResult result = check(R"(local a: number? = if true then 42 else nil)"); + LUAU_REQUIRE_NO_ERRORS(result); - TypeId aType = requireType("a"); - CHECK_EQ(getPrimitiveType(aType), PrimitiveTypeVar::String); + CHECK_EQ(toString(requireType("a"), {true}), "number?"); } } -TEST_CASE_FIXTURE(Fixture, "tc_if_else_expressions3") +TEST_CASE_FIXTURE(Fixture, "tc_if_else_expressions_expected_type_1") { - ScopedFastFlag sff1{"LuauIfElseExpressionBaseSupport", true}; - ScopedFastFlag sff2{"LuauIfElseExpressionAnalysisSupport", true}; + ScopedFastFlag luauIfElseExpectedType2{"LuauIfElseExpectedType2", true}; + ScopedFastFlag luauIfElseBranchTypeUnion{"LuauIfElseBranchTypeUnion", true}; - { - CheckResult result = check(R"(local a = if true then "true" else 42)"); - // We currently require both true/false expressions to unify to the same type. However, we do intend to lift - // this restriction in the future. - LUAU_REQUIRE_ERROR_COUNT(1, result); - TypeId aType = requireType("a"); - CHECK_EQ(getPrimitiveType(aType), PrimitiveTypeVar::String); - } + CheckResult result = check(R"( +type X = {number | string} +local a: X = if true then {"1", 2, 3} else {4, 5, 6} +)"); + + LUAU_REQUIRE_NO_ERRORS(result); + CHECK_EQ(toString(requireType("a"), {true}), "{number | string}"); +} + +TEST_CASE_FIXTURE(Fixture, "tc_if_else_expressions_expected_type_2") +{ + ScopedFastFlag luauIfElseExpectedType2{"LuauIfElseExpectedType2", true}; + ScopedFastFlag luauIfElseBranchTypeUnion{ "LuauIfElseBranchTypeUnion", true }; + + CheckResult result = check(R"( +local a: number? = if true then 1 else nil +)"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "tc_if_else_expressions_expected_type_3") +{ + ScopedFastFlag luauIfElseExpectedType2{"LuauIfElseExpectedType2", true}; + + CheckResult result = check(R"( +local function times(n: any, f: () -> T) + local result: {T} = {} + local res = f() + table.insert(result, if true then res else n) + return result +end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); } TEST_CASE_FIXTURE(Fixture, "type_error_addition") @@ -5039,4 +5082,51 @@ end LUAU_REQUIRE_NO_ERRORS(result); } +TEST_CASE_FIXTURE(Fixture, "table_oop") +{ + CheckResult result = check(R"( + --!strict +local Class = {} +Class.__index = Class + +type Class = typeof(setmetatable({} :: { x: number }, Class)) + +function Class.new(x: number): Class + return setmetatable({x = x}, Class) +end + +function Class.getx(self: Class) + return self.x +end + +function test() + local c = Class.new(42) + local n = c:getx() + local nn = c.x + + print(string.format("%d %d", n, nn)) +end +)"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "recursive_metatable_crash") +{ + ScopedFastFlag luauMetatableAreEqualRecursion{"LuauMetatableAreEqualRecursion", true}; + + CheckResult result = check(R"( +local function getIt() + local y + y = setmetatable({}, y) + return y +end +local a = getIt() +local b = getIt() +local c = a or b + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.typePacks.cpp b/tests/TypeInfer.typePacks.cpp index 5d37b032..d4878d14 100644 --- a/tests/TypeInfer.typePacks.cpp +++ b/tests/TypeInfer.typePacks.cpp @@ -621,4 +621,328 @@ type Other = Packed CHECK_EQ(toString(result.errors[0]), "Generic type 'Packed' expects 2 type pack arguments, but only 1 is specified"); } +TEST_CASE_FIXTURE(Fixture, "type_alias_default_type_explicit") +{ + ScopedFastFlag luauParseTypeAliasDefaults{"LuauParseTypeAliasDefaults", true}; + ScopedFastFlag luauTypeAliasDefaults{"LuauTypeAliasDefaults", true}; + + CheckResult result = check(R"( +type Y = { a: T, b: U } + +local a: Y = { a = 2, b = 3 } +local b: Y = { a = 2, b = "s" } + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ(toString(requireType("a")), "Y"); + CHECK_EQ(toString(requireType("b")), "Y"); + + result = check(R"( +type Y = { a: T } + +local a: Y = { a = 2 } +local b: Y<> = { a = "s" } +local c: Y = { a = "s" } + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ(toString(requireType("a")), "Y"); + CHECK_EQ(toString(requireType("b")), "Y"); + CHECK_EQ(toString(requireType("c")), "Y"); +} + +TEST_CASE_FIXTURE(Fixture, "type_alias_default_type_self") +{ + ScopedFastFlag luauParseTypeAliasDefaults{"LuauParseTypeAliasDefaults", true}; + ScopedFastFlag luauTypeAliasDefaults{"LuauTypeAliasDefaults", true}; + + CheckResult result = check(R"( +type Y = { a: T, b: U } + +local a: Y = { a = 2, b = 3 } +local b: Y = { a = "h", b = "s" } + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ(toString(requireType("a")), "Y"); + CHECK_EQ(toString(requireType("b")), "Y"); + + result = check(R"( +type Y string> = { a: T, b: U } + +local a: Y + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ(toString(requireType("a")), "Y string>"); +} + +TEST_CASE_FIXTURE(Fixture, "type_alias_default_type_chained") +{ + ScopedFastFlag luauParseTypeAliasDefaults{"LuauParseTypeAliasDefaults", true}; + ScopedFastFlag luauTypeAliasDefaults{"LuauTypeAliasDefaults", true}; + + CheckResult result = check(R"( +type Y = { a: T, b: U, c: V } + +local a: Y +local b: Y + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ(toString(requireType("a")), "Y"); + CHECK_EQ(toString(requireType("b")), "Y"); +} + +TEST_CASE_FIXTURE(Fixture, "type_alias_default_type_pack_explicit") +{ + ScopedFastFlag luauParseTypeAliasDefaults{"LuauParseTypeAliasDefaults", true}; + ScopedFastFlag luauTypeAliasDefaults{"LuauTypeAliasDefaults", true}; + + CheckResult result = check(R"( +type Y = { a: (T...) -> () } +local a: Y<> + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ(toString(requireType("a")), "Y"); +} + +TEST_CASE_FIXTURE(Fixture, "type_alias_default_type_pack_self_ty") +{ + ScopedFastFlag luauParseTypeAliasDefaults{"LuauParseTypeAliasDefaults", true}; + ScopedFastFlag luauTypeAliasDefaults{"LuauTypeAliasDefaults", true}; + + CheckResult result = check(R"( +type Y = { a: T, b: (U...) -> T } + +local a: Y + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ(toString(requireType("a")), "Y"); +} + +TEST_CASE_FIXTURE(Fixture, "type_alias_default_type_pack_self_tp") +{ + ScopedFastFlag luauParseTypeAliasDefaults{"LuauParseTypeAliasDefaults", true}; + ScopedFastFlag luauTypeAliasDefaults{"LuauTypeAliasDefaults", true}; + + CheckResult result = check(R"( +type Y = { a: (T...) -> U... } +local a: Y + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ(toString(requireType("a")), "Y<(number, string), (number, string)>"); +} + +TEST_CASE_FIXTURE(Fixture, "type_alias_default_type_pack_self_chained_tp") +{ + ScopedFastFlag luauParseTypeAliasDefaults{"LuauParseTypeAliasDefaults", true}; + ScopedFastFlag luauTypeAliasDefaults{"LuauTypeAliasDefaults", true}; + + CheckResult result = check(R"( +type Y = { a: (T...) -> U..., b: (T...) -> V... } +local a: Y + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ(toString(requireType("a")), "Y<(number, string), (number, string), (number, string)>"); +} + +TEST_CASE_FIXTURE(Fixture, "type_alias_default_mixed_self") +{ + ScopedFastFlag luauParseTypeAliasDefaults{"LuauParseTypeAliasDefaults", true}; + ScopedFastFlag luauTypeAliasDefaults{"LuauTypeAliasDefaults", true}; + + CheckResult result = check(R"( +type Y = { a: (T, U, V...) -> W... } +local a: Y +local b: Y +local c: Y +local d: Y ()> + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ(toString(requireType("a")), "Y"); + CHECK_EQ(toString(requireType("b")), "Y"); + CHECK_EQ(toString(requireType("c")), "Y"); + CHECK_EQ(toString(requireType("d")), "Y ()>"); +} + +TEST_CASE_FIXTURE(Fixture, "type_alias_default_type_errors") +{ + ScopedFastFlag luauParseTypeAliasDefaults{"LuauParseTypeAliasDefaults", true}; + ScopedFastFlag luauTypeAliasDefaults{"LuauTypeAliasDefaults", true}; + + CheckResult result = check(R"( +type Y = { a: T } +local a: Y = { a = 2 } + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ(toString(result.errors[0]), "Unknown type 'T'"); + + result = check(R"( +type Y = { a: (T...) -> () } +local a: Y<> + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ(toString(result.errors[0]), "Unknown type 'T'"); + + result = check(R"( +type Y = { a: (T) -> U... } +local a: Y<...number> + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ(toString(result.errors[0]), "Generic type 'Y' expects at least 1 type argument, but none are specified"); + + result = check(R"( +type Packed = (T) -> T +local a: Packed + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ(toString(result.errors[0]), "Type parameter list is required"); + + result = check(R"( +type Y = { a: T } +local a: Y + )"); + + LUAU_REQUIRE_ERRORS(result); + + result = check(R"( +type Y = { a: T } +local a: Y<...number> + )"); + + LUAU_REQUIRE_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "type_alias_default_export") +{ + ScopedFastFlag luauParseTypeAliasDefaults{"LuauParseTypeAliasDefaults", true}; + ScopedFastFlag luauTypeAliasDefaults{"LuauTypeAliasDefaults", true}; + + fileResolver.source["Module/Types"] = R"( +export type A = { a: T, b: U } +export type B = { a: T, b: U } +export type C string> = { a: T, b: U } +export type D = { a: T, b: U, c: V } +export type E = { a: (T...) -> () } +export type F = { a: T, b: (U...) -> T } +export type G = { b: (U...) -> T... } +export type H = { b: (T...) -> T... } +return {} + )"; + + CheckResult resultTypes = frontend.check("Module/Types"); + LUAU_REQUIRE_NO_ERRORS(resultTypes); + + fileResolver.source["Module/Users"] = R"( +local Types = require(script.Parent.Types) + +local a: Types.A +local b: Types.B +local c: Types.C +local d: Types.D +local e: Types.E<> +local eVoid: Types.E<()> +local f: Types.F +local g: Types.G<...number> +local h: Types.H<> + )"; + + CheckResult resultUsers = frontend.check("Module/Users"); + LUAU_REQUIRE_NO_ERRORS(resultUsers); + + CHECK_EQ(toString(requireType("Module/Users", "a")), "A"); + CHECK_EQ(toString(requireType("Module/Users", "b")), "B"); + CHECK_EQ(toString(requireType("Module/Users", "c")), "C string>"); + CHECK_EQ(toString(requireType("Module/Users", "d")), "D"); + CHECK_EQ(toString(requireType("Module/Users", "e")), "E"); + CHECK_EQ(toString(requireType("Module/Users", "eVoid")), "E<>"); + CHECK_EQ(toString(requireType("Module/Users", "f")), "F"); + CHECK_EQ(toString(requireType("Module/Users", "g")), "G<...number, ()>"); + CHECK_EQ(toString(requireType("Module/Users", "h")), "H<>"); +} + +TEST_CASE_FIXTURE(Fixture, "type_alias_default_type_skip_brackets") +{ + ScopedFastFlag luauParseTypeAliasDefaults{"LuauParseTypeAliasDefaults", true}; + ScopedFastFlag luauTypeAliasDefaults{"LuauTypeAliasDefaults", true}; + + CheckResult result = check(R"( +type Y = (T...) -> number +local a: Y + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ(toString(requireType("a")), "(...string) -> number"); +} + +TEST_CASE_FIXTURE(Fixture, "type_alias_defaults_confusing_types") +{ + ScopedFastFlag luauParseTypeAliasDefaults{"LuauParseTypeAliasDefaults", true}; + ScopedFastFlag luauTypeAliasDefaults{"LuauTypeAliasDefaults", true}; + + CheckResult result = check(R"( +type A = (T, V...) -> (U, W...) +type B = A +type C = A + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ(toString(*lookupType("B"), {true}), "(string, ...any) -> (number, ...any)"); + CHECK_EQ(toString(*lookupType("C"), {true}), "(string, boolean) -> (number, boolean)"); +} + +TEST_CASE_FIXTURE(Fixture, "type_alias_defaults_recursive_type") +{ + ScopedFastFlag luauParseTypeAliasDefaults{"LuauParseTypeAliasDefaults", true}; + ScopedFastFlag luauTypeAliasDefaults{"LuauTypeAliasDefaults", true}; + + CheckResult result = check(R"( +type F ()> = (K) -> V +type R = { m: F } + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ(toString(*lookupType("R"), {true}), "t1 where t1 = {| m: (t1) -> (t1) -> () |}"); +} + +TEST_CASE_FIXTURE(Fixture, "pack_tail_unification_check") +{ + ScopedFastFlag luauUnifyPackTails{"LuauUnifyPackTails", true}; + ScopedFastFlag luauExtendedFunctionMismatchError{"LuauExtendedFunctionMismatchError", true}; + + CheckResult result = check(R"( +local a: () -> (number, ...string) +local b: () -> (number, ...boolean) +a = b + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ(toString(result.errors[0]), R"(Type '() -> (number, ...boolean)' could not be converted into '() -> (number, ...string)' +caused by: + Type 'boolean' could not be converted into 'string')"); +} + TEST_SUITE_END(); From 497d625f7337fcfe829d5c8131d8b7d63ab16452 Mon Sep 17 00:00:00 2001 From: Halalaluyafail3 <55773281+Halalaluyafail3@users.noreply.github.com> Date: Fri, 14 Jan 2022 16:42:49 -0500 Subject: [PATCH 129/399] Fix some mistakes in the documentation (#314) --- docs/_pages/compatibility.md | 2 +- docs/_pages/lint.md | 2 +- docs/_pages/syntax.md | 4 ++-- 3 files changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/_pages/compatibility.md b/docs/_pages/compatibility.md index 06d64b76..cb97d114 100644 --- a/docs/_pages/compatibility.md +++ b/docs/_pages/compatibility.md @@ -116,7 +116,7 @@ Floor division is less harmful, but it's used rarely enough that `math.floor(a/b | The function print calls `__tostring` instead of tostring to format its arguments. | ✔️ | | | By default, the decoding functions in the utf8 library do not accept surrogates. | 😞 | breaks compatibility and doesn't seem very interesting otherwise | -Lua has a beautiful syntax and frankly we're disappointed in the ``/`` which takes away from that beauty. Taking syntax aside, `` isn't very useful in Luau - its dominant use case is for code that works with external resources like files or sockets, but we don't provide such APIs - and has a very large complexity cost, evidences by a lot of bug fixes since the initial implementation in 5.4 work versions. `` in Luau doesn't matter for performance - our multi-pass compiler is already able to analyze the usage of the variable to know if it's modified or not and extract all performance gains from it - so the only use here is for code readability, where the `` syntax is... suboptimal. +Lua has a beautiful syntax and frankly we're disappointed in the ``/`` which takes away from that beauty. Taking syntax aside, `` isn't very useful in Luau - its dominant use case is for code that works with external resources like files or sockets, but we don't provide such APIs - and has a very large complexity cost, evidences by a lot of bug fixes since the initial implementation in 5.4 work versions. `` in Luau doesn't matter for performance - our multi-pass compiler is already able to analyze the usage of the variable to know if it's modified or not and extract all performance gains from it - so the only use here is for code readability, where the `` syntax is... suboptimal. If we do end up introducing const variables, it would be through a `const var = value` syntax, which is backwards compatible through a context-sensitive keyword similar to `type`. diff --git a/docs/_pages/lint.md b/docs/_pages/lint.md index fd254b92..ff677829 100644 --- a/docs/_pages/lint.md +++ b/docs/_pages/lint.md @@ -145,7 +145,7 @@ In some cases the linter can detect code that is never executed, because all exe ```lua function cbrt(v) if v >= 0 then - return v ^ 1/3 + return v ^ (1/3) else error('cbrt expects a non-negative argument') end diff --git a/docs/_pages/syntax.md b/docs/_pages/syntax.md index e71dc1cc..4d39e462 100644 --- a/docs/_pages/syntax.md +++ b/docs/_pages/syntax.md @@ -27,7 +27,7 @@ The rest of this document documents additional syntax used in Luau. ## String literals -Luau implements support for hexadecimal (`\0x`), Unicode (`\u`) and `\z` escapes for string literals. This syntax follows [Lua 5.3 syntax](https://www.lua.org/manual/5.3/manual.html#3.1): +Luau implements support for hexadecimal (`\x`), Unicode (`\u`) and `\z` escapes for string literals. This syntax follows [Lua 5.3 syntax](https://www.lua.org/manual/5.3/manual.html#3.1): - `\xAB` inserts a character with the code 0xAB into the string - `\u{ABC}` inserts a UTF8 byte sequence that encodes U+0ABC character into the string (note that braces are mandatory) @@ -158,7 +158,7 @@ In addition to declaring types for a given value, Luau supports declaring type a ```lua type Point = { x: number, y: number } type Array = { [number]: T } -type Something = typeof(string.gmatch("", "\d")) +type Something = typeof(string.gmatch("", "%d")) ``` The right hand side of the type alias can be a type definition or a `typeof` expression; `typeof` expression doesn't evaluate its argument at runtime. From 49ce5096a427a952994ff5934963ca63d1075cf8 Mon Sep 17 00:00:00 2001 From: dcope-rbx <91100513+dcope-rbx@users.noreply.github.com> Date: Mon, 17 Jan 2022 09:44:31 -0800 Subject: [PATCH 130/399] Fixed a couple spelling mistakes in markdown files. (#316) --- docs/_pages/compatibility.md | 2 +- docs/_pages/profile.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/_pages/compatibility.md b/docs/_pages/compatibility.md index cb97d114..00d883e2 100644 --- a/docs/_pages/compatibility.md +++ b/docs/_pages/compatibility.md @@ -127,5 +127,5 @@ We have a few behavior deviations from Lua 5.x that come from either a different * Tail calls are not supported to simplify implementation, make debugging/stack traces more predictable and allow deep validation of caller identity for security * Order of table assignment in table literals follows program order in mixed tables (Lua 5.x assigns array elements first in some cases) * Equality comparisons call `__eq` metamethod even when objects are rawequal (which matches other metamethods like `<=` and facilitates NaN checking) -* `function()` expressions may reuse a previosly created closure in certain scenarios (when all upvalues captured are the same) for efficiency, which changes object identity but doesn't change call semantics -- this is different from Lua 5.1 but similar to Lua 5.2/5.3 +* `function()` expressions may reuse a previously created closure in certain scenarios (when all upvalues captured are the same) for efficiency, which changes object identity but doesn't change call semantics -- this is different from Lua 5.1 but similar to Lua 5.2/5.3 * `os.time` returns UTC timestamp when called with a table for consistency diff --git a/docs/_pages/profile.md b/docs/_pages/profile.md index 72d11546..be056841 100644 --- a/docs/_pages/profile.md +++ b/docs/_pages/profile.md @@ -8,7 +8,7 @@ One of main goals of Luau is to enable high performance code. To help with that code is in developers' hands, and is a combination of good algorithm design and implementation that adheres to the strengths of the language. To help write efficient code, Luau provides a built-in profiler that samples the execution of the program and outputs a profiler dump that can be converted to an interactive flamegraph. -To run the profiler, make sure you have an optimized build of the intepreter (otherwise profiling results are going to be very skewed) and run it with `--profile` argument: +To run the profiler, make sure you have an optimized build of the interpreter (otherwise profiling results are going to be very skewed) and run it with `--profile` argument: ``` $ luau --profile tests/chess.lua From 4e5ff995825e70415edaee4be4442d6e09e4640e Mon Sep 17 00:00:00 2001 From: JohnnyMorganz Date: Thu, 20 Jan 2022 16:27:19 +0000 Subject: [PATCH 131/399] Improve Grammar documentation (#315) --- docs/_pages/grammar.md | 20 +++++++++++++------- 1 file changed, 13 insertions(+), 7 deletions(-) diff --git a/docs/_pages/grammar.md b/docs/_pages/grammar.md index 22b078a1..585bac73 100644 --- a/docs/_pages/grammar.md +++ b/docs/_pages/grammar.md @@ -7,8 +7,6 @@ classes: wide This is the complete syntax grammar for Luau in EBNF. More information about the terminal nodes String and Number is available in the [syntax section](syntax). -> Note: this grammar is currently missing type pack syntax for generic arguments - ```ebnf chunk = block block = {stat [';']} [laststat [';']] @@ -24,12 +22,12 @@ stat = varlist '=' explist | 'function' funcname funcbody | 'local' 'function' NAME funcbody | 'local' bindinglist ['=' explist] | - ['export'] 'type' NAME ['<' GenericTypeList '>'] '=' Type + ['export'] 'type' NAME ['<' GenericTypeParameterList '>'] '=' Type laststat = 'return' [explist] | 'break' | 'continue' funcname = NAME {'.' NAME} [':' NAME] -funcbody = '(' [parlist] ')' [':' ReturnType] block 'end' +funcbody = ['<' GenericTypeParameterList '>'] '(' [parlist] ')' [':' ReturnType] block 'end' parlist = bindinglist [',' '...'] | '...' explist = {exp ','} exp @@ -60,19 +58,27 @@ unop = '-' | 'not' | '#' SimpleType = 'nil' | - NAME ['.' NAME] [ '<' TypeList '>' ] | + SingletonType | + NAME ['.' NAME] [ '<' [TypeParams] '>' ] | 'typeof' '(' exp ')' | TableType | FunctionType +SingletonType = STRING | 'true' | 'false' + Type = SimpleType ['?'] | SimpleType ['|' Type] | SimpleType ['&' Type] -GenericTypeList = NAME ['...'] {',' NAME ['...']} +GenericTypePackParameter = NAME '...' ['=' (TypePack | VariadicTypePack | GenericTypePack)] +GenericTypeParameterList = NAME ['=' Type] [',' GenericTypeParameterList] | GenericTypePackParameter {',' GenericTypePackParameter} TypeList = Type [',' TypeList] | '...' Type -ReturnType = Type | '(' TypeList ')' +TypeParams = (Type | TypePack | VariadicTypePack | GenericTypePack) [',' TypeParams] +TypePack = '(' [TypeList] ')' +GenericTypePack = NAME '...' +VariadicTypePack = '...' Type +ReturnType = Type | TypePack TableIndexer = '[' Type ']' ':' Type TableProp = NAME ':' Type TablePropOrIndexer = TableProp | TableIndexer From 478a3da63401c433f60128fe8c69814b187e8c32 Mon Sep 17 00:00:00 2001 From: Shiro <46553887+SnowyShiro@users.noreply.github.com> Date: Thu, 20 Jan 2022 09:42:49 -0800 Subject: [PATCH 132/399] Update copyright years. (#323) --- LICENSE.txt | 2 +- VM/include/lua.h | 2 +- VM/src/lapi.cpp | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/LICENSE.txt b/LICENSE.txt index 3f6c4670..40fc04c1 100644 --- a/LICENSE.txt +++ b/LICENSE.txt @@ -1,6 +1,6 @@ MIT License -Copyright (c) 2019-2021 Roblox Corporation +Copyright (c) 2019-2022 Roblox Corporation Copyright (c) 1994–2019 Lua.org, PUC-Rio. Permission is hereby granted, free of charge, to any person obtaining a copy of diff --git a/VM/include/lua.h b/VM/include/lua.h index 55902160..c5dcef25 100644 --- a/VM/include/lua.h +++ b/VM/include/lua.h @@ -382,7 +382,7 @@ typedef struct lua_Callbacks lua_Callbacks; LUA_API lua_Callbacks* lua_callbacks(lua_State* L); /****************************************************************************** - * Copyright (c) 2019-2021 Roblox Corporation + * Copyright (c) 2019-2022 Roblox Corporation * Copyright (C) 1994-2008 Lua.org, PUC-Rio. All rights reserved. * * Permission is hereby granted, free of charge, to any person obtaining diff --git a/VM/src/lapi.cpp b/VM/src/lapi.cpp index c98b9590..d5416285 100644 --- a/VM/src/lapi.cpp +++ b/VM/src/lapi.cpp @@ -18,7 +18,7 @@ const char* lua_ident = "$Lua: Lua 5.1.4 Copyright (C) 1994-2008 Lua.org, PUC-Ri "$Authors: R. Ierusalimschy, L. H. de Figueiredo & W. Celes $\n" "$URL: www.lua.org $\n"; -const char* luau_ident = "$Luau: Copyright (C) 2019-2021 Roblox Corporation $\n" +const char* luau_ident = "$Luau: Copyright (C) 2019-2022 Roblox Corporation $\n" "$URL: luau-lang.org $\n"; #define api_checknelems(L, n) api_check(L, (n) <= (L->top - L->base)) From d70a0788c5b7993cc535c19b2bb732d56b760663 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Fri, 21 Jan 2022 08:23:02 -0800 Subject: [PATCH 133/399] Sync to upstream/release/511 --- Analysis/include/Luau/TypeVar.h | 4 + Analysis/src/Linter.cpp | 15 ++ Analysis/src/TypeInfer.cpp | 29 ++- Analysis/src/TypeVar.cpp | 46 +++++ Analysis/src/Unifier.cpp | 6 + Ast/include/Luau/Ast.h | 3 +- Ast/include/Luau/DenseHash.h | 8 +- Ast/src/Parser.cpp | 20 +- CLI/Repl.cpp | 110 +++++++--- CMakeLists.txt | 6 + Compiler/src/TableShape.cpp | 49 ++++- LICENSE.txt | 2 +- VM/include/lua.h | 2 +- VM/src/lapi.cpp | 2 +- VM/src/ldo.cpp | 9 +- VM/src/lfunc.cpp | 96 ++++++--- VM/src/lfunc.h | 7 +- VM/src/lgc.cpp | 251 +++++++++++++++++++---- VM/src/lgc.h | 1 + VM/src/lgcdebug.cpp | 77 +++++-- VM/src/lmem.cpp | 347 +++++++++++++++++++++++++++++++- VM/src/lmem.h | 15 ++ VM/src/lobject.h | 16 +- VM/src/lstate.cpp | 37 +++- VM/src/lstate.h | 15 +- VM/src/lstring.cpp | 136 +++++++++---- VM/src/lstring.h | 2 +- VM/src/ltable.cpp | 8 +- VM/src/ltable.h | 2 +- VM/src/ludata.cpp | 6 +- VM/src/ludata.h | 2 +- VM/src/lvmexecute.cpp | 2 +- tests/Compiler.test.cpp | 24 +++ tests/Linter.test.cpp | 9 +- tests/Parser.test.cpp | 13 +- tests/TypeInfer.tables.test.cpp | 48 +++++ tests/TypeInfer.test.cpp | 29 +++ tests/conformance/closure.lua | 15 ++ tests/conformance/gc.lua | 28 +++ 39 files changed, 1278 insertions(+), 219 deletions(-) diff --git a/Analysis/include/Luau/TypeVar.h b/Analysis/include/Luau/TypeVar.h index fd2c2afa..3f5e26d6 100644 --- a/Analysis/include/Luau/TypeVar.h +++ b/Analysis/include/Luau/TypeVar.h @@ -1,6 +1,7 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #pragma once +#include "Luau/DenseHash.h" #include "Luau/Predicate.h" #include "Luau/Unifiable.h" #include "Luau/Variant.h" @@ -499,6 +500,9 @@ bool maybeGeneric(const TypeId ty); // Checks if a type is of the form T1|...|Tn where one of the Ti is a singleton bool maybeSingleton(TypeId ty); +// Checks if the length operator can be applied on the value of type +bool hasLength(TypeId ty, DenseHashSet& seen, int* recursionCount); + struct SingletonTypes { const TypeId nilType; diff --git a/Analysis/src/Linter.cpp b/Analysis/src/Linter.cpp index 1a5b24fe..905b70bf 100644 --- a/Analysis/src/Linter.cpp +++ b/Analysis/src/Linter.cpp @@ -12,6 +12,8 @@ #include #include +LUAU_FASTFLAGVARIABLE(LuauLintTableCreateTable, false) + namespace Luau { @@ -2153,6 +2155,19 @@ private: "table.move uses index 0 but arrays are 1-based; did you mean 1 instead?"); } + if (FFlag::LuauLintTableCreateTable && func->index == "create" && node->args.size == 2) + { + // table.create(n, {...}) + if (args[1]->is()) + emitWarning(*context, LintWarning::Code_TableOperations, args[1]->location, + "table.create with a table literal will reuse the same object for all elements; consider using a for loop instead"); + + // table.create(n, {...} :: ?) + if (AstExprTypeAssertion* as = args[1]->as(); as && as->expr->is()) + emitWarning(*context, LintWarning::Code_TableOperations, as->expr->location, + "table.create with a table literal will reuse the same object for all elements; consider using a for loop instead"); + } + return true; } diff --git a/Analysis/src/TypeInfer.cpp b/Analysis/src/TypeInfer.cpp index bedcc022..e2d8a4fb 100644 --- a/Analysis/src/TypeInfer.cpp +++ b/Analysis/src/TypeInfer.cpp @@ -33,6 +33,7 @@ LUAU_FASTFLAGVARIABLE(DebugLuauFreezeDuringUnification, false) LUAU_FASTFLAGVARIABLE(LuauRecursiveTypeParameterRestriction, false) LUAU_FASTFLAGVARIABLE(LuauIfElseBranchTypeUnion, false) LUAU_FASTFLAGVARIABLE(LuauIfElseExpectedType2, false) +LUAU_FASTFLAGVARIABLE(LuauLengthOnCompositeType, false) LUAU_FASTFLAGVARIABLE(LuauQuantifyInPlace2, false) LUAU_FASTFLAGVARIABLE(LuauSealExports, false) LUAU_FASTFLAGVARIABLE(LuauSingletonTypes, false) @@ -2066,17 +2067,27 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprUn if (get(operandType)) return {errorRecoveryType(scope)}; - if (get(operandType)) - return {numberType}; // Not strictly correct: metatables permit overriding this - - if (auto p = get(operandType)) + if (FFlag::LuauLengthOnCompositeType) { - if (p->type == PrimitiveTypeVar::String) - return {numberType}; - } + DenseHashSet seen{nullptr}; - if (!getTableType(operandType)) - reportError(TypeError{expr.location, NotATable{operandType}}); + if (!hasLength(operandType, seen, &recursionCount)) + reportError(TypeError{expr.location, NotATable{operandType}}); + } + else + { + if (get(operandType)) + return {numberType}; // Not strictly correct: metatables permit overriding this + + if (auto p = get(operandType)) + { + if (p->type == PrimitiveTypeVar::String) + return {numberType}; + } + + if (!getTableType(operandType)) + reportError(TypeError{expr.location, NotATable{operandType}}); + } return {numberType}; diff --git a/Analysis/src/TypeVar.cpp b/Analysis/src/TypeVar.cpp index ac2b2541..df5d76ed 100644 --- a/Analysis/src/TypeVar.cpp +++ b/Analysis/src/TypeVar.cpp @@ -5,6 +5,7 @@ #include "Luau/Common.h" #include "Luau/DenseHash.h" #include "Luau/Error.h" +#include "Luau/RecursionCounter.h" #include "Luau/StringUtils.h" #include "Luau/ToString.h" #include "Luau/TypeInfer.h" @@ -19,6 +20,8 @@ LUAU_FASTINTVARIABLE(LuauTypeMaximumStringifierLength, 500) LUAU_FASTINTVARIABLE(LuauTableTypeMaximumStringifierLength, 0) +LUAU_FASTINT(LuauTypeInferRecursionLimit) +LUAU_FASTFLAG(LuauLengthOnCompositeType) LUAU_FASTFLAGVARIABLE(LuauMetatableAreEqualRecursion, false) LUAU_FASTFLAGVARIABLE(LuauRefactorTagging, false) LUAU_FASTFLAG(LuauErrorRecoveryType) @@ -326,6 +329,49 @@ bool maybeSingleton(TypeId ty) return false; } +bool hasLength(TypeId ty, DenseHashSet& seen, int* recursionCount) +{ + LUAU_ASSERT(FFlag::LuauLengthOnCompositeType); + + RecursionLimiter _rl(recursionCount, FInt::LuauTypeInferRecursionLimit); + + ty = follow(ty); + + if (seen.contains(ty)) + return true; + + if (isPrim(ty, PrimitiveTypeVar::String) || get(ty) || get(ty) || get(ty)) + return true; + + if (auto uty = get(ty)) + { + seen.insert(ty); + + for (TypeId part : uty->options) + { + if (!hasLength(part, seen, recursionCount)) + return false; + } + + return true; + } + + if (auto ity = get(ty)) + { + seen.insert(ty); + + for (TypeId part : ity->parts) + { + if (hasLength(part, seen, recursionCount)) + return true; + } + + return false; + } + + return false; +} + FunctionTypeVar::FunctionTypeVar(TypePackId argTypes, TypePackId retType, std::optional defn, bool hasSelf) : argTypes(argTypes) , retType(retType) diff --git a/Analysis/src/Unifier.cpp b/Analysis/src/Unifier.cpp index 6873c657..2bd9cf83 100644 --- a/Analysis/src/Unifier.cpp +++ b/Analysis/src/Unifier.cpp @@ -13,6 +13,7 @@ LUAU_FASTINT(LuauTypeInferRecursionLimit); LUAU_FASTINT(LuauTypeInferTypePackLoopLimit); +LUAU_FASTFLAGVARIABLE(LuauCommittingTxnLogFreeTpPromote, false) LUAU_FASTFLAG(LuauUseCommittingTxnLog) LUAU_FASTINTVARIABLE(LuauTypeInferIterationLimit, 2000); LUAU_FASTFLAGVARIABLE(LuauTableSubtypingVariance2, false); @@ -99,6 +100,11 @@ struct PromoteTypeLevels bool operator()(TypePackId tp, const FreeTypePack&) { + // Surprise, it's actually a BoundTypePack that hasn't been committed yet. + // Calling getMutable on this will trigger an assertion. + if (FFlag::LuauCommittingTxnLogFreeTpPromote && FFlag::LuauUseCommittingTxnLog && !log.is(tp)) + return true; + promote(tp, FFlag::LuauUseCommittingTxnLog ? log.getMutable(tp) : getMutable(tp)); return true; } diff --git a/Ast/include/Luau/Ast.h b/Ast/include/Luau/Ast.h index 573850a5..ac5950c0 100644 --- a/Ast/include/Luau/Ast.h +++ b/Ast/include/Luau/Ast.h @@ -1265,7 +1265,8 @@ struct hash size_t operator()(const Luau::AstName& value) const { // note: since operator== uses pointer identity, hashing function uses it as well - return value.value ? std::hash()(value.value) : 0; + // the hasher is the same as DenseHashPointer (DenseHash.h) + return (uintptr_t(value.value) >> 4) ^ (uintptr_t(value.value) >> 9); } }; diff --git a/Ast/include/Luau/DenseHash.h b/Ast/include/Luau/DenseHash.h index a7b2515a..65939bee 100644 --- a/Ast/include/Luau/DenseHash.h +++ b/Ast/include/Luau/DenseHash.h @@ -12,10 +12,6 @@ namespace Luau { -// Internal implementation of DenseHashSet and DenseHashMap -namespace detail -{ - struct DenseHashPointer { size_t operator()(const void* key) const @@ -24,6 +20,10 @@ struct DenseHashPointer } }; +// Internal implementation of DenseHashSet and DenseHashMap +namespace detail +{ + template using DenseHashDefault = std::conditional_t, DenseHashPointer, std::hash>; diff --git a/Ast/src/Parser.cpp b/Ast/src/Parser.cpp index 77787cb1..3c607d24 100644 --- a/Ast/src/Parser.cpp +++ b/Ast/src/Parser.cpp @@ -14,6 +14,7 @@ LUAU_FASTFLAGVARIABLE(LuauFixAmbiguousErrorRecoveryInAssign, false) LUAU_FASTFLAGVARIABLE(LuauParseSingletonTypes, false) LUAU_FASTFLAGVARIABLE(LuauParseTypeAliasDefaults, false) LUAU_FASTFLAGVARIABLE(LuauParseRecoverTypePackEllipsis, false) +LUAU_FASTFLAGVARIABLE(LuauStartingBrokenComment, false) namespace Luau { @@ -174,10 +175,23 @@ ParseResult Parser::parse(const char* buffer, size_t bufferSize, AstNameTable& n const Lexeme::Type type = p.lexer.current().type; const Location loc = p.lexer.current().location; - p.lexer.next(); + if (FFlag::LuauStartingBrokenComment) + { + if (options.captureComments) + p.commentLocations.push_back(Comment{type, loc}); - if (options.captureComments) - p.commentLocations.push_back(Comment{type, loc}); + if (type == Lexeme::BrokenComment) + break; + + p.lexer.next(); + } + else + { + p.lexer.next(); + + if (options.captureComments) + p.commentLocations.push_back(Comment{type, loc}); + } } p.lexer.setSkipComments(true); diff --git a/CLI/Repl.cpp b/CLI/Repl.cpp index 36747f48..e5042152 100644 --- a/CLI/Repl.cpp +++ b/CLI/Repl.cpp @@ -35,10 +35,15 @@ enum class CompileFormat Binary }; +struct GlobalOptions +{ + int optimizationLevel = 1; +} globalOptions; + static Luau::CompileOptions copts() { Luau::CompileOptions result = {}; - result.optimizationLevel = 1; + result.optimizationLevel = globalOptions.optimizationLevel; result.debugLevel = 1; result.coverageLevel = coverageActive() ? 2 : 0; @@ -232,13 +237,14 @@ static std::string runCode(lua_State* L, const std::string& source) static void completeIndexer(lua_State* L, const char* editBuffer, size_t start, std::vector& completions) { std::string_view lookup = editBuffer + start; + char lastSep = 0; for (;;) { - size_t dot = lookup.find('.'); - std::string_view prefix = lookup.substr(0, dot); + size_t sep = lookup.find_first_of(".:"); + std::string_view prefix = lookup.substr(0, sep); - if (dot == std::string_view::npos) + if (sep == std::string_view::npos) { // table, key lua_pushnil(L); @@ -249,11 +255,22 @@ static void completeIndexer(lua_State* L, const char* editBuffer, size_t start, { // table, key, value std::string_view key = lua_tostring(L, -2); + int valueType = lua_type(L, -1); - if (!key.empty() && Luau::startsWith(key, prefix)) - completions.push_back(editBuffer + std::string(key.substr(prefix.size()))); + // If the last separator was a ':' (i.e. a method call) then only functions should be completed. + bool requiredValueType = (lastSep != ':' || valueType == LUA_TFUNCTION); + + if (!key.empty() && requiredValueType && Luau::startsWith(key, prefix)) + { + std::string completion(editBuffer + std::string(key.substr(prefix.size()))); + if (valueType == LUA_TFUNCTION) + { + // Add an opening paren for function calls by default. + completion += "("; + } + completions.push_back(completion); + } } - lua_pop(L, 1); } @@ -266,10 +283,21 @@ static void completeIndexer(lua_State* L, const char* editBuffer, size_t start, lua_rawget(L, -2); lua_remove(L, -2); - if (!lua_istable(L, -1)) + if (lua_type(L, -1) == LUA_TSTRING) + { + // Replace the string object with the string class to perform further lookups of string functions + // Note: We retrieve the string class from _G to prevent issues if the user assigns to `string`. + lua_getglobal(L, "_G"); + lua_pushlstring(L, "string", 6); + lua_rawget(L, -2); + lua_remove(L, -2); + LUAU_ASSERT(lua_istable(L, -1)); + } + else if (!lua_istable(L, -1)) break; - lookup.remove_prefix(dot + 1); + lastSep = lookup[sep]; + lookup.remove_prefix(sep + 1); } } @@ -279,7 +307,7 @@ static void completeIndexer(lua_State* L, const char* editBuffer, size_t start, static void completeRepl(lua_State* L, const char* editBuffer, std::vector& completions) { size_t start = strlen(editBuffer); - while (start > 0 && (isalnum(editBuffer[start - 1]) || editBuffer[start - 1] == '.' || editBuffer[start - 1] == '_')) + while (start > 0 && (isalnum(editBuffer[start - 1]) || editBuffer[start - 1] == '.' || editBuffer[start - 1] == ':' || editBuffer[start - 1] == '_')) start--; // look the value up in current global table first @@ -319,15 +347,8 @@ struct LinenoiseScopedHistory std::string historyFilepath; }; -static void runRepl() +static void runReplImpl(lua_State* L) { - std::unique_ptr globalState(luaL_newstate(), lua_close); - lua_State* L = globalState.get(); - - setupState(L); - - luaL_sandboxthread(L); - linenoise::SetCompletionCallback([L](const char* editBuffer, std::vector& completions) { completeRepl(L, editBuffer, completions); }); @@ -368,7 +389,18 @@ static void runRepl() } } -static bool runFile(const char* name, lua_State* GL) +static void runRepl() +{ + std::unique_ptr globalState(luaL_newstate(), lua_close); + lua_State* L = globalState.get(); + + setupState(L); + luaL_sandboxthread(L); + runReplImpl(L); +} + +// `repl` is used it indicate if a repl should be started after executing the file. +static bool runFile(const char* name, lua_State* GL, bool repl) { std::optional source = readFile(name); if (!source) @@ -419,6 +451,10 @@ static bool runFile(const char* name, lua_State* GL) fprintf(stderr, "%s", error.c_str()); } + if (repl) + { + runReplImpl(L); + } lua_pop(GL, 1); return status == 0; } @@ -457,7 +493,7 @@ static bool compileFile(const char* name, CompileFormat format) bcb.setDumpSource(*source); } - Luau::compileOrThrow(bcb, *source); + Luau::compileOrThrow(bcb, *source, copts()); switch (format) { @@ -495,9 +531,11 @@ static void displayHelp(const char* argv0) printf(" --compile[=format]: compile input files and output resulting formatted bytecode (binary or text)\n"); printf("\n"); printf("Available options:\n"); - printf(" -h, --help: Display this usage message.\n"); - printf(" --profile[=N]: profile the code using N Hz sampling (default 10000) and output results to profile.out\n"); printf(" --coverage: collect code coverage while running the code and output results to coverage.out\n"); + printf(" -h, --help: Display this usage message.\n"); + printf(" -i, --interactive: Run an interactive REPL after executing the last script specified.\n"); + printf(" -O: use compiler optimization level (n=0-2).\n"); + printf(" --profile[=N]: profile the code using N Hz sampling (default 10000) and output results to profile.out\n"); printf(" --timetrace: record compiler time tracing information into trace.json\n"); } @@ -519,6 +557,7 @@ int main(int argc, char** argv) CompileFormat compileFormat{}; int profile = 0; bool coverage = false; + bool interactive = false; // Set the mode if the user has explicitly specified one. int argStart = 1; @@ -540,8 +579,8 @@ int main(int argc, char** argv) } else { - fprintf(stdout, "Error: Unrecognized value for '--compile' specified.\n"); - return -1; + fprintf(stderr, "Error: Unrecognized value for '--compile' specified.\n"); + return 1; } } @@ -552,6 +591,20 @@ int main(int argc, char** argv) displayHelp(argv[0]); return 0; } + else if (strcmp(argv[i], "-i") == 0 || strcmp(argv[i], "--interactive") == 0) + { + interactive = true; + } + else if (strncmp(argv[i], "-O", 2) == 0) + { + int level = atoi(argv[i] + 2); + if (level < 0 || level > 2) + { + fprintf(stderr, "Error: Optimization level must be between 0 and 2 inclusive.\n"); + return 1; + } + globalOptions.optimizationLevel = level; + } else if (strcmp(argv[i], "--profile") == 0) { profile = 10000; // default to 10 KHz @@ -575,7 +628,7 @@ int main(int argc, char** argv) } else if (argv[i][0] == '-') { - fprintf(stdout, "Error: Unrecognized option '%s'.\n\n", argv[i]); + fprintf(stderr, "Error: Unrecognized option '%s'.\n\n", argv[i]); displayHelp(argv[0]); return 1; } @@ -623,8 +676,11 @@ int main(int argc, char** argv) int failed = 0; - for (const std::string& path : files) - failed += !runFile(path.c_str(), L); + for (size_t i = 0; i < files.size(); ++i) + { + bool isLastFile = i == files.size() - 1; + failed += !runFile(files[i].c_str(), L, interactive && isLastFile); + } if (profile) { diff --git a/CMakeLists.txt b/CMakeLists.txt index bafc59e5..77cf47e8 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -78,6 +78,12 @@ target_compile_options(Luau.Ast PRIVATE ${LUAU_OPTIONS}) target_compile_options(Luau.Analysis PRIVATE ${LUAU_OPTIONS}) target_compile_options(Luau.VM PRIVATE ${LUAU_OPTIONS}) +if (MSVC AND MSVC_VERSION GREATER_EQUAL 1924) + # disable partial redundancy elimination which regresses interpreter codegen substantially in VS2022: + # https://developercommunity.visualstudio.com/t/performance-regression-on-a-complex-interpreter-lo/1631863 + set_source_files_properties(VM/src/lvmexecute.cpp PROPERTIES COMPILE_FLAGS /d2ssa-pre-) +endif() + if(LUAU_BUILD_CLI) target_compile_options(Luau.Repl.CLI PRIVATE ${LUAU_OPTIONS}) target_compile_options(Luau.Analyze.CLI PRIVATE ${LUAU_OPTIONS}) diff --git a/Compiler/src/TableShape.cpp b/Compiler/src/TableShape.cpp index 7d99f222..9dc2f0a4 100644 --- a/Compiler/src/TableShape.cpp +++ b/Compiler/src/TableShape.cpp @@ -1,11 +1,16 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #include "TableShape.h" +LUAU_FASTFLAGVARIABLE(LuauPredictTableSizeLoop, false) + namespace Luau { namespace Compile { +// conservative limit for the loop bound that establishes table array size +static const int kMaxLoopBound = 16; + static AstExprTable* getTableHint(AstExpr* expr) { // unadorned table literal @@ -27,7 +32,7 @@ struct ShapeVisitor : AstVisitor { size_t operator()(const std::pair& p) const { - return std::hash()(p.first) ^ std::hash()(p.second); + return DenseHashPointer()(p.first) ^ std::hash()(p.second); } }; @@ -36,10 +41,13 @@ struct ShapeVisitor : AstVisitor DenseHashMap tables; DenseHashSet, Hasher> fields; + DenseHashMap loops; // iterator => upper bound for 1..k + ShapeVisitor(DenseHashMap& shapes) : shapes(shapes) , tables(nullptr) , fields(std::pair()) + , loops(nullptr) { } @@ -63,16 +71,31 @@ struct ShapeVisitor : AstVisitor void assignField(AstExpr* expr, AstExpr* index) { AstExprLocal* lv = expr->as(); - AstExprConstantNumber* number = index->as(); + if (!lv) + return; - if (lv && number) + AstExprTable** table = tables.find(lv->local); + if (!table) + return; + + if (AstExprConstantNumber* number = index->as()) { - if (AstExprTable** table = tables.find(lv->local)) + TableShape& shape = shapes[*table]; + + if (number->value == double(shape.arraySize + 1)) + shape.arraySize += 1; + } + else if (AstExprLocal* iter = index->as()) + { + if (!FFlag::LuauPredictTableSizeLoop) + return; + + if (const unsigned int* bound = loops.find(iter->local)) { TableShape& shape = shapes[*table]; - if (number->value == double(shape.arraySize + 1)) - shape.arraySize += 1; + if (shape.arraySize == 0) + shape.arraySize = *bound; } } } @@ -117,6 +140,20 @@ struct ShapeVisitor : AstVisitor return false; } + + bool visit(AstStatFor* node) override + { + if (!FFlag::LuauPredictTableSizeLoop) + return true; + + AstExprConstantNumber* from = node->from->as(); + AstExprConstantNumber* to = node->to->as(); + + if (from && to && from->value == 1.0 && to->value >= 1.0 && to->value <= double(kMaxLoopBound) && !node->step) + loops[node->var] = unsigned(to->value); + + return true; + } }; void predictTableShapes(DenseHashMap& shapes, AstNode* root) diff --git a/LICENSE.txt b/LICENSE.txt index d63e7299..fa9914d7 100644 --- a/LICENSE.txt +++ b/LICENSE.txt @@ -1,6 +1,6 @@ MIT License -Copyright (c) 2019-2021 Roblox Corporation +Copyright (c) 2019-2022 Roblox Corporation Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in diff --git a/VM/include/lua.h b/VM/include/lua.h index 55902160..c5dcef25 100644 --- a/VM/include/lua.h +++ b/VM/include/lua.h @@ -382,7 +382,7 @@ typedef struct lua_Callbacks lua_Callbacks; LUA_API lua_Callbacks* lua_callbacks(lua_State* L); /****************************************************************************** - * Copyright (c) 2019-2021 Roblox Corporation + * Copyright (c) 2019-2022 Roblox Corporation * Copyright (C) 1994-2008 Lua.org, PUC-Rio. All rights reserved. * * Permission is hereby granted, free of charge, to any person obtaining diff --git a/VM/src/lapi.cpp b/VM/src/lapi.cpp index c98b9590..d5416285 100644 --- a/VM/src/lapi.cpp +++ b/VM/src/lapi.cpp @@ -18,7 +18,7 @@ const char* lua_ident = "$Lua: Lua 5.1.4 Copyright (C) 1994-2008 Lua.org, PUC-Ri "$Authors: R. Ierusalimschy, L. H. de Figueiredo & W. Celes $\n" "$URL: www.lua.org $\n"; -const char* luau_ident = "$Luau: Copyright (C) 2019-2021 Roblox Corporation $\n" +const char* luau_ident = "$Luau: Copyright (C) 2019-2022 Roblox Corporation $\n" "$URL: luau-lang.org $\n"; #define api_checknelems(L, n) api_check(L, (n) <= (L->top - L->base)) diff --git a/VM/src/ldo.cpp b/VM/src/ldo.cpp index 3cce7665..581506a8 100644 --- a/VM/src/ldo.cpp +++ b/VM/src/ldo.cpp @@ -150,12 +150,11 @@ l_noret luaD_throw(lua_State* L, int errcode) static void correctstack(lua_State* L, TValue* oldstack) { - CallInfo* ci; - GCObject* up; L->top = (L->top - oldstack) + L->stack; - for (up = L->openupval; up != NULL; up = up->gch.next) - gco2uv(up)->v = (gco2uv(up)->v - oldstack) + L->stack; - for (ci = L->base_ci; ci <= L->ci; ci++) + // TODO (FFlagLuauGcPagedSweep): 'next' type will change after removal of the flag and the cast will not be required + for (UpVal* up = L->openupval; up != NULL; up = (UpVal*)up->next) + up->v = (up->v - oldstack) + L->stack; + for (CallInfo* ci = L->base_ci; ci <= L->ci; ci++) { ci->top = (ci->top - oldstack) + L->stack; ci->base = (ci->base - oldstack) + L->stack; diff --git a/VM/src/lfunc.cpp b/VM/src/lfunc.cpp index 4178eda4..6088f71c 100644 --- a/VM/src/lfunc.cpp +++ b/VM/src/lfunc.cpp @@ -7,10 +7,11 @@ #include "lgc.h" LUAU_FASTFLAGVARIABLE(LuauNoDirectUpvalRemoval, false) +LUAU_FASTFLAG(LuauGcPagedSweep) Proto* luaF_newproto(lua_State* L) { - Proto* f = luaM_new(L, Proto, sizeof(Proto), L->activememcat); + Proto* f = luaM_newgco(L, Proto, sizeof(Proto), L->activememcat); luaC_link(L, f, LUA_TPROTO); f->k = NULL; f->sizek = 0; @@ -38,7 +39,7 @@ Proto* luaF_newproto(lua_State* L) Closure* luaF_newLclosure(lua_State* L, int nelems, Table* e, Proto* p) { - Closure* c = luaM_new(L, Closure, sizeLclosure(nelems), L->activememcat); + Closure* c = luaM_newgco(L, Closure, sizeLclosure(nelems), L->activememcat); luaC_link(L, c, LUA_TFUNCTION); c->isC = 0; c->env = e; @@ -53,7 +54,7 @@ Closure* luaF_newLclosure(lua_State* L, int nelems, Table* e, Proto* p) Closure* luaF_newCclosure(lua_State* L, int nelems, Table* e) { - Closure* c = luaM_new(L, Closure, sizeCclosure(nelems), L->activememcat); + Closure* c = luaM_newgco(L, Closure, sizeCclosure(nelems), L->activememcat); luaC_link(L, c, LUA_TFUNCTION); c->isC = 1; c->env = e; @@ -69,10 +70,9 @@ Closure* luaF_newCclosure(lua_State* L, int nelems, Table* e) UpVal* luaF_findupval(lua_State* L, StkId level) { global_State* g = L->global; - GCObject** pp = &L->openupval; + UpVal** pp = &L->openupval; UpVal* p; - UpVal* uv; - while (*pp != NULL && (p = gco2uv(*pp))->v >= level) + while (*pp != NULL && (p = *pp)->v >= level) { LUAU_ASSERT(p->v != &p->u.value); if (p->v == level) @@ -81,53 +81,95 @@ UpVal* luaF_findupval(lua_State* L, StkId level) changewhite(obj2gco(p)); /* resurrect it */ return p; } - pp = &p->next; + + // TODO (FFlagLuauGcPagedSweep): 'next' type will change after removal of the flag and the cast will not be required + pp = (UpVal**)&p->next; } - uv = luaM_new(L, UpVal, sizeof(UpVal), L->activememcat); /* not found: create a new one */ + + UpVal* uv = luaM_newgco(L, UpVal, sizeof(UpVal), L->activememcat); /* not found: create a new one */ uv->tt = LUA_TUPVAL; uv->marked = luaC_white(g); uv->memcat = L->activememcat; uv->v = level; /* current value lives in the stack */ - uv->next = *pp; /* chain it in the proper position */ - *pp = obj2gco(uv); - uv->u.l.prev = &g->uvhead; /* double link it in `uvhead' list */ + + // chain the upvalue in the threads open upvalue list at the proper position + UpVal* next = *pp; + + // TODO (FFlagLuauGcPagedSweep): 'next' type will change after removal of the flag and the cast will not be required + uv->next = (GCObject*)next; + + if (FFlag::LuauGcPagedSweep) + { + uv->u.l.threadprev = pp; + if (next) + { + // TODO (FFlagLuauGcPagedSweep): 'next' type will change after removal of the flag and the cast will not be required + next->u.l.threadprev = (UpVal**)&uv->next; + } + } + + *pp = uv; + + // double link the upvalue in the global open upvalue list + uv->u.l.prev = &g->uvhead; uv->u.l.next = g->uvhead.u.l.next; uv->u.l.next->u.l.prev = uv; g->uvhead.u.l.next = uv; LUAU_ASSERT(uv->u.l.next->u.l.prev == uv && uv->u.l.prev->u.l.next == uv); return uv; } - -static void unlinkupval(UpVal* uv) +void luaF_unlinkupval(UpVal* uv) { + // unlink upvalue from the global open upvalue list LUAU_ASSERT(uv->u.l.next->u.l.prev == uv && uv->u.l.prev->u.l.next == uv); - uv->u.l.next->u.l.prev = uv->u.l.prev; /* remove from `uvhead' list */ + uv->u.l.next->u.l.prev = uv->u.l.prev; uv->u.l.prev->u.l.next = uv->u.l.next; + + if (FFlag::LuauGcPagedSweep) + { + // unlink upvalue from the thread open upvalue list + // TODO (FFlagLuauGcPagedSweep): 'next' type will change after removal of the flag and this and the following cast will not be required + *uv->u.l.threadprev = (UpVal*)uv->next; + + if (UpVal* next = (UpVal*)uv->next) + next->u.l.threadprev = uv->u.l.threadprev; + } } -void luaF_freeupval(lua_State* L, UpVal* uv) +void luaF_freeupval(lua_State* L, UpVal* uv, lua_Page* page) { if (uv->v != &uv->u.value) /* is it open? */ - unlinkupval(uv); /* remove from open list */ - luaM_free(L, uv, sizeof(UpVal), uv->memcat); /* free upvalue */ + luaF_unlinkupval(uv); /* remove from open list */ + luaM_freegco(L, uv, sizeof(UpVal), uv->memcat, page); /* free upvalue */ } void luaF_close(lua_State* L, StkId level) { - UpVal* uv; global_State* g = L->global; // TODO: remove with FFlagLuauNoDirectUpvalRemoval - while (L->openupval != NULL && (uv = gco2uv(L->openupval))->v >= level) + UpVal* uv; + while (L->openupval != NULL && (uv = L->openupval)->v >= level) { GCObject* o = obj2gco(uv); LUAU_ASSERT(!isblack(o) && uv->v != &uv->u.value); - L->openupval = uv->next; /* remove from `open' list */ - if (!FFlag::LuauNoDirectUpvalRemoval && isdead(g, o)) + + if (!FFlag::LuauGcPagedSweep) + L->openupval = (UpVal*)uv->next; /* remove from `open' list */ + + if (FFlag::LuauGcPagedSweep && isdead(g, o)) { - luaF_freeupval(L, uv); /* free upvalue */ + // by removing the upvalue from global/thread open upvalue lists, L->openupval will be pointing to the next upvalue + luaF_unlinkupval(uv); + // close the upvalue without copying the dead data so that luaF_freeupval will not unlink again + uv->v = &uv->u.value; + } + else if (!FFlag::LuauNoDirectUpvalRemoval && isdead(g, o)) + { + luaF_freeupval(L, uv, NULL); /* free upvalue */ } else { - unlinkupval(uv); + // by removing the upvalue from global/thread open upvalue lists, L->openupval will be pointing to the next upvalue + luaF_unlinkupval(uv); setobj(L, &uv->u.value, uv->v); uv->v = &uv->u.value; /* now current value lives here */ luaC_linkupval(L, uv); /* link upvalue into `gcroot' list */ @@ -135,7 +177,7 @@ void luaF_close(lua_State* L, StkId level) } } -void luaF_freeproto(lua_State* L, Proto* f) +void luaF_freeproto(lua_State* L, Proto* f, lua_Page* page) { luaM_freearray(L, f->code, f->sizecode, Instruction, f->memcat); luaM_freearray(L, f->p, f->sizep, Proto*, f->memcat); @@ -146,13 +188,13 @@ void luaF_freeproto(lua_State* L, Proto* f) luaM_freearray(L, f->upvalues, f->sizeupvalues, TString*, f->memcat); if (f->debuginsn) luaM_freearray(L, f->debuginsn, f->sizecode, uint8_t, f->memcat); - luaM_free(L, f, sizeof(Proto), f->memcat); + luaM_freegco(L, f, sizeof(Proto), f->memcat, page); } -void luaF_freeclosure(lua_State* L, Closure* c) +void luaF_freeclosure(lua_State* L, Closure* c, lua_Page* page) { int size = c->isC ? sizeCclosure(c->nupvalues) : sizeLclosure(c->nupvalues); - luaM_free(L, c, size, c->memcat); + luaM_freegco(L, c, size, c->memcat, page); } const LocVar* luaF_getlocal(const Proto* f, int local_number, int pc) diff --git a/VM/src/lfunc.h b/VM/src/lfunc.h index 4be23667..8047cebe 100644 --- a/VM/src/lfunc.h +++ b/VM/src/lfunc.h @@ -12,7 +12,8 @@ LUAI_FUNC Closure* luaF_newLclosure(lua_State* L, int nelems, Table* e, Proto* p LUAI_FUNC Closure* luaF_newCclosure(lua_State* L, int nelems, Table* e); LUAI_FUNC UpVal* luaF_findupval(lua_State* L, StkId level); LUAI_FUNC void luaF_close(lua_State* L, StkId level); -LUAI_FUNC void luaF_freeproto(lua_State* L, Proto* f); -LUAI_FUNC void luaF_freeclosure(lua_State* L, Closure* c); -LUAI_FUNC void luaF_freeupval(lua_State* L, UpVal* uv); +LUAI_FUNC void luaF_freeproto(lua_State* L, Proto* f, struct lua_Page* page); +LUAI_FUNC void luaF_freeclosure(lua_State* L, Closure* c, struct lua_Page* page); +void luaF_unlinkupval(UpVal* uv); +LUAI_FUNC void luaF_freeupval(lua_State* L, UpVal* uv, struct lua_Page* page); LUAI_FUNC const LocVar* luaF_getlocal(const Proto* func, int local_number, int pc); diff --git a/VM/src/lgc.cpp b/VM/src/lgc.cpp index 76ef7a06..50859b1e 100644 --- a/VM/src/lgc.cpp +++ b/VM/src/lgc.cpp @@ -8,12 +8,16 @@ #include "lfunc.h" #include "lstring.h" #include "ldo.h" +#include "lmem.h" #include "ludata.h" #include +LUAU_FASTFLAGVARIABLE(LuauGcPagedSweep, false) + #define GC_SWEEPMAX 40 #define GC_SWEEPCOST 10 +#define GC_SWEEPPAGESTEPCOST 4 #define GC_INTERRUPT(state) \ { \ @@ -457,31 +461,31 @@ static void shrinkstack(lua_State* L) condhardstacktests(luaD_reallocstack(L, s_used)); } -static void freeobj(lua_State* L, GCObject* o) +static void freeobj(lua_State* L, GCObject* o, lua_Page* page) { switch (o->gch.tt) { case LUA_TPROTO: - luaF_freeproto(L, gco2p(o)); + luaF_freeproto(L, gco2p(o), page); break; case LUA_TFUNCTION: - luaF_freeclosure(L, gco2cl(o)); + luaF_freeclosure(L, gco2cl(o), page); break; case LUA_TUPVAL: - luaF_freeupval(L, gco2uv(o)); + luaF_freeupval(L, gco2uv(o), page); break; case LUA_TTABLE: - luaH_free(L, gco2h(o)); + luaH_free(L, gco2h(o), page); break; case LUA_TTHREAD: LUAU_ASSERT(gco2th(o) != L && gco2th(o) != L->global->mainthread); - luaE_freethread(L, gco2th(o)); + luaE_freethread(L, gco2th(o), page); break; case LUA_TSTRING: - luaS_free(L, gco2ts(o)); + luaS_free(L, gco2ts(o), page); break; case LUA_TUSERDATA: - luaU_freeudata(L, gco2u(o)); + luaU_freeudata(L, gco2u(o), page); break; default: LUAU_ASSERT(0); @@ -492,6 +496,8 @@ static void freeobj(lua_State* L, GCObject* o) static GCObject** sweeplist(lua_State* L, GCObject** p, size_t count, size_t* traversedcount) { + LUAU_ASSERT(!FFlag::LuauGcPagedSweep); + GCObject* curr; global_State* g = L->global; int deadmask = otherwhite(g); @@ -502,7 +508,7 @@ static GCObject** sweeplist(lua_State* L, GCObject** p, size_t count, size_t* tr int alive = (curr->gch.marked ^ WHITEBITS) & deadmask; if (curr->gch.tt == LUA_TTHREAD) { - sweepwholelist(L, &gco2th(curr)->openupval, traversedcount); /* sweep open upvalues */ + sweepwholelist(L, (GCObject**)&gco2th(curr)->openupval, traversedcount); /* sweep open upvalues */ lua_State* th = gco2th(curr); @@ -524,7 +530,7 @@ static GCObject** sweeplist(lua_State* L, GCObject** p, size_t count, size_t* tr *p = curr->gch.next; if (curr == g->rootgc) /* is the first element of the list? */ g->rootgc = curr->gch.next; /* adjust first */ - freeobj(L, curr); + freeobj(L, curr, NULL); } } @@ -537,14 +543,16 @@ static GCObject** sweeplist(lua_State* L, GCObject** p, size_t count, size_t* tr static void deletelist(lua_State* L, GCObject** p, GCObject* limit) { + LUAU_ASSERT(!FFlag::LuauGcPagedSweep); + GCObject* curr; while ((curr = *p) != limit) { if (curr->gch.tt == LUA_TTHREAD) /* delete open upvalues of each thread */ - deletelist(L, &gco2th(curr)->openupval, NULL); + deletelist(L, (GCObject**)&gco2th(curr)->openupval, NULL); *p = curr->gch.next; - freeobj(L, curr); + freeobj(L, curr, NULL); } } @@ -567,23 +575,62 @@ static void shrinkbuffersfull(lua_State* L) luaS_resize(L, hashsize); /* table is too big */ } +static bool deletegco(void* context, lua_Page* page, GCObject* gco) +{ + LUAU_ASSERT(FFlag::LuauGcPagedSweep); + + // we are in the process of deleting everything + // threads with open upvalues will attempt to close them all on removal + // but those upvalues might point to stack values that were already deleted + if (gco->gch.tt == LUA_TTHREAD) + { + lua_State* th = gco2th(gco); + + while (UpVal* uv = th->openupval) + { + luaF_unlinkupval(uv); + // close the upvalue without copying the dead data so that luaF_freeupval will not unlink again + uv->v = &uv->u.value; + } + } + + lua_State* L = (lua_State*)context; + freeobj(L, gco, page); + return true; +} + void luaC_freeall(lua_State* L) { global_State* g = L->global; LUAU_ASSERT(L == g->mainthread); - LUAU_ASSERT(L->next == NULL); /* mainthread is at the end of rootgc list */ - deletelist(L, &g->rootgc, obj2gco(L)); + if (FFlag::LuauGcPagedSweep) + { + luaM_visitgco(L, L, deletegco); - for (int i = 0; i < g->strt.size; i++) /* free all string lists */ - deletelist(L, &g->strt.hash[i], NULL); + for (int i = 0; i < g->strt.size; i++) /* free all string lists */ + LUAU_ASSERT(g->strt.hash[i] == NULL); - LUAU_ASSERT(L->global->strt.nuse == 0); - deletelist(L, &g->strbufgc, NULL); - // unfortunately, when string objects are freed, the string table use count is decremented - // even when the string is a buffer that wasn't placed into the table - L->global->strt.nuse = 0; + LUAU_ASSERT(L->global->strt.nuse == 0); + LUAU_ASSERT(g->strbufgc == NULL); + } + else + { + LUAU_ASSERT(L->next == NULL); /* mainthread is at the end of rootgc list */ + + deletelist(L, &g->rootgc, obj2gco(L)); + + for (int i = 0; i < g->strt.size; i++) /* free all string lists */ + deletelist(L, (GCObject**)&g->strt.hash[i], NULL); + + LUAU_ASSERT(L->global->strt.nuse == 0); + deletelist(L, (GCObject**)&g->strbufgc, NULL); + + // unfortunately, when string objects are freed, the string table use count is decremented + // even when the string is a buffer that wasn't placed into the table + L->global->strt.nuse = 0; + } } static void markmt(global_State* g) @@ -648,12 +695,88 @@ static size_t atomic(lua_State* L) /* flip current white */ g->currentwhite = cast_byte(otherwhite(g)); g->sweepstrgc = 0; - g->sweepgc = &g->rootgc; - g->gcstate = GCSsweepstring; + + if (FFlag::LuauGcPagedSweep) + { + g->sweepgcopage = g->allgcopages; + g->gcstate = GCSsweep; + } + else + { + g->sweepgc = &g->rootgc; + g->gcstate = GCSsweepstring; + } return work; } +static bool sweepgco(lua_State* L, lua_Page* page, GCObject* gco) +{ + LUAU_ASSERT(FFlag::LuauGcPagedSweep); + + global_State* g = L->global; + + int deadmask = otherwhite(g); + LUAU_ASSERT(testbit(deadmask, FIXEDBIT)); // make sure we never sweep fixed objects + + int alive = (gco->gch.marked ^ WHITEBITS) & deadmask; + + g->gcstats.currcycle.sweepitems++; + + if (gco->gch.tt == LUA_TTHREAD) + { + lua_State* th = gco2th(gco); + + if (alive) + { + resetbit(th->stackstate, THREAD_SLEEPINGBIT); + shrinkstack(th); + } + } + + if (alive) + { + LUAU_ASSERT(!isdead(g, gco)); + makewhite(g, gco); // make it white (for next cycle) + return false; + } + + LUAU_ASSERT(isdead(g, gco)); + freeobj(L, gco, page); + return true; +} + +// a version of generic luaM_visitpage specialized for the main sweep stage +static int sweepgcopage(lua_State* L, lua_Page* page) +{ + LUAU_ASSERT(FFlag::LuauGcPagedSweep); + + char* start; + char* end; + int busyBlocks; + int blockSize; + luaM_getpagewalkinfo(page, &start, &end, &busyBlocks, &blockSize); + + for (char* pos = start; pos != end; pos += blockSize) + { + GCObject* gco = (GCObject*)pos; + + // skip memory blocks that are already freed + if (gco->gch.tt == LUA_TNIL) + continue; + + // when true is returned it means that the element was deleted + if (sweepgco(L, page, gco)) + { + // if the last block was removed, page would be removed as well + if (--busyBlocks == 0) + return (pos - start) / blockSize + 1; + } + } + + return (end - start) / blockSize; +} + static size_t gcstep(lua_State* L, size_t limit) { size_t cost = 0; @@ -706,15 +829,21 @@ static size_t gcstep(lua_State* L, size_t limit) g->gcstats.currcycle.atomicstarttotalsizebytes = g->totalbytes; cost = atomic(L); /* finish mark phase */ - LUAU_ASSERT(g->gcstate == GCSsweepstring); + + if (FFlag::LuauGcPagedSweep) + LUAU_ASSERT(g->gcstate == GCSsweep); + else + LUAU_ASSERT(g->gcstate == GCSsweepstring); break; } case GCSsweepstring: { + LUAU_ASSERT(!FFlag::LuauGcPagedSweep); + while (g->sweepstrgc < g->strt.size && cost < limit) { size_t traversedcount = 0; - sweepwholelist(L, &g->strt.hash[g->sweepstrgc++], &traversedcount); + sweepwholelist(L, (GCObject**)&g->strt.hash[g->sweepstrgc++], &traversedcount); g->gcstats.currcycle.sweepitems += traversedcount; cost += GC_SWEEPCOST; @@ -727,7 +856,7 @@ static size_t gcstep(lua_State* L, size_t limit) uint32_t nuse = L->global->strt.nuse; size_t traversedcount = 0; - sweepwholelist(L, &g->strbufgc, &traversedcount); + sweepwholelist(L, (GCObject**)&g->strbufgc, &traversedcount); L->global->strt.nuse = nuse; @@ -738,19 +867,44 @@ static size_t gcstep(lua_State* L, size_t limit) } case GCSsweep: { - while (*g->sweepgc && cost < limit) + if (FFlag::LuauGcPagedSweep) { - size_t traversedcount = 0; - g->sweepgc = sweeplist(L, g->sweepgc, GC_SWEEPMAX, &traversedcount); + while (g->sweepgcopage && cost < limit) + { + lua_Page* next = luaM_getnextgcopage(g->sweepgcopage); // page sweep might destroy the page - g->gcstats.currcycle.sweepitems += traversedcount; - cost += GC_SWEEPMAX * GC_SWEEPCOST; + int steps = sweepgcopage(L, g->sweepgcopage); + + g->sweepgcopage = next; + cost += steps * GC_SWEEPPAGESTEPCOST; + } + + // nothing more to sweep? + if (g->sweepgcopage == NULL) + { + // don't forget to visit main thread + sweepgco(L, NULL, obj2gco(g->mainthread)); + + shrinkbuffers(L); + g->gcstate = GCSpause; /* end collection */ + } } + else + { + while (*g->sweepgc && cost < limit) + { + size_t traversedcount = 0; + g->sweepgc = sweeplist(L, g->sweepgc, GC_SWEEPMAX, &traversedcount); - if (*g->sweepgc == NULL) - { /* nothing more to sweep? */ - shrinkbuffers(L); - g->gcstate = GCSpause; /* end collection */ + g->gcstats.currcycle.sweepitems += traversedcount; + cost += GC_SWEEPMAX * GC_SWEEPCOST; + } + + if (*g->sweepgc == NULL) + { /* nothing more to sweep? */ + shrinkbuffers(L); + g->gcstate = GCSpause; /* end collection */ + } } break; } @@ -877,12 +1031,19 @@ void luaC_fullgc(lua_State* L) { /* reset sweep marks to sweep all elements (returning them to white) */ g->sweepstrgc = 0; - g->sweepgc = &g->rootgc; + if (FFlag::LuauGcPagedSweep) + g->sweepgcopage = g->allgcopages; + else + g->sweepgc = &g->rootgc; /* reset other collector lists */ g->gray = NULL; g->grayagain = NULL; g->weak = NULL; - g->gcstate = GCSsweepstring; + + if (FFlag::LuauGcPagedSweep) + g->gcstate = GCSsweep; + else + g->gcstate = GCSsweepstring; } LUAU_ASSERT(g->gcstate == GCSsweepstring || g->gcstate == GCSsweep); /* finish any pending sweep phase */ @@ -979,8 +1140,11 @@ void luaC_barrierback(lua_State* L, Table* t) void luaC_linkobj(lua_State* L, GCObject* o, uint8_t tt) { global_State* g = L->global; - o->gch.next = g->rootgc; - g->rootgc = o; + if (!FFlag::LuauGcPagedSweep) + { + o->gch.next = g->rootgc; + g->rootgc = o; + } o->gch.marked = luaC_white(g); o->gch.tt = tt; o->gch.memcat = L->activememcat; @@ -990,8 +1154,13 @@ void luaC_linkupval(lua_State* L, UpVal* uv) { global_State* g = L->global; GCObject* o = obj2gco(uv); - o->gch.next = g->rootgc; /* link upvalue into `rootgc' list */ - g->rootgc = o; + + if (!FFlag::LuauGcPagedSweep) + { + o->gch.next = g->rootgc; /* link upvalue into `rootgc' list */ + g->rootgc = o; + } + if (isgray(o)) { if (keepinvariant(g)) diff --git a/VM/src/lgc.h b/VM/src/lgc.h index f434e506..4455fec5 100644 --- a/VM/src/lgc.h +++ b/VM/src/lgc.h @@ -13,6 +13,7 @@ #define GCSpropagate 1 #define GCSpropagateagain 2 #define GCSatomic 3 +// TODO: remove with FFlagLuauGcPagedSweep #define GCSsweepstring 4 #define GCSsweep 5 diff --git a/VM/src/lgcdebug.cpp b/VM/src/lgcdebug.cpp index c66de9c1..906fb0d0 100644 --- a/VM/src/lgcdebug.cpp +++ b/VM/src/lgcdebug.cpp @@ -2,16 +2,19 @@ // This code is based on Lua 5.x implementation licensed under MIT License; see lua_LICENSE.txt for details #include "lgc.h" +#include "lfunc.h" +#include "lmem.h" #include "lobject.h" #include "lstate.h" -#include "ltable.h" -#include "lfunc.h" #include "lstring.h" +#include "ltable.h" #include "ludata.h" #include #include +LUAU_FASTFLAG(LuauGcPagedSweep) + static void validateobjref(global_State* g, GCObject* f, GCObject* t) { LUAU_ASSERT(!isdead(g, t)); @@ -101,10 +104,11 @@ static void validatestack(global_State* g, lua_State* l) if (l->namecall) validateobjref(g, obj2gco(l), obj2gco(l->namecall)); - for (GCObject* uv = l->openupval; uv; uv = uv->gch.next) + // TODO (FFlagLuauGcPagedSweep): 'next' type will change after removal of the flag and the cast will not be required + for (UpVal* uv = l->openupval; uv; uv = (UpVal*)uv->next) { - LUAU_ASSERT(uv->gch.tt == LUA_TUPVAL); - LUAU_ASSERT(gco2uv(uv)->v != &gco2uv(uv)->u.value); + LUAU_ASSERT(uv->tt == LUA_TUPVAL); + LUAU_ASSERT(uv->v != &uv->u.value); } } @@ -178,6 +182,8 @@ static void validateobj(global_State* g, GCObject* o) static void validatelist(global_State* g, GCObject* o) { + LUAU_ASSERT(!FFlag::LuauGcPagedSweep); + while (o) { validateobj(g, o); @@ -216,6 +222,17 @@ static void validategraylist(global_State* g, GCObject* o) } } +static bool validategco(void* context, lua_Page* page, GCObject* gco) +{ + LUAU_ASSERT(FFlag::LuauGcPagedSweep); + + lua_State* L = (lua_State*)context; + global_State* g = L->global; + + validateobj(g, gco); + return false; +} + void luaC_validate(lua_State* L) { global_State* g = L->global; @@ -231,11 +248,18 @@ void luaC_validate(lua_State* L) validategraylist(g, g->gray); validategraylist(g, g->grayagain); - for (int i = 0; i < g->strt.size; ++i) - validatelist(g, g->strt.hash[i]); + if (FFlag::LuauGcPagedSweep) + { + luaM_visitgco(L, L, validategco); + } + else + { + for (int i = 0; i < g->strt.size; ++i) + validatelist(g, (GCObject*)(g->strt.hash[i])); - validatelist(g, g->rootgc); - validatelist(g, g->strbufgc); + validatelist(g, g->rootgc); + validatelist(g, (GCObject*)(g->strbufgc)); + } for (UpVal* uv = g->uvhead.u.l.next; uv != &g->uvhead; uv = uv->u.l.next) { @@ -499,6 +523,8 @@ static void dumpobj(FILE* f, GCObject* o) static void dumplist(FILE* f, GCObject* o) { + LUAU_ASSERT(!FFlag::LuauGcPagedSweep); + while (o) { dumpref(f, o); @@ -509,22 +535,45 @@ static void dumplist(FILE* f, GCObject* o) // thread has additional list containing collectable objects that are not present in rootgc if (o->gch.tt == LUA_TTHREAD) - dumplist(f, gco2th(o)->openupval); + dumplist(f, (GCObject*)gco2th(o)->openupval); o = o->gch.next; } } +static bool dumpgco(void* context, lua_Page* page, GCObject* gco) +{ + LUAU_ASSERT(FFlag::LuauGcPagedSweep); + + FILE* f = (FILE*)context; + + dumpref(f, gco); + fputc(':', f); + dumpobj(f, gco); + fputc(',', f); + fputc('\n', f); + + return false; +} + void luaC_dump(lua_State* L, void* file, const char* (*categoryName)(lua_State* L, uint8_t memcat)) { global_State* g = L->global; FILE* f = static_cast(file); fprintf(f, "{\"objects\":{\n"); - dumplist(f, g->rootgc); - dumplist(f, g->strbufgc); - for (int i = 0; i < g->strt.size; ++i) - dumplist(f, g->strt.hash[i]); + + if (FFlag::LuauGcPagedSweep) + { + luaM_visitgco(L, f, dumpgco); + } + else + { + dumplist(f, g->rootgc); + dumplist(f, (GCObject*)(g->strbufgc)); + for (int i = 0; i < g->strt.size; ++i) + dumplist(f, (GCObject*)(g->strt.hash[i])); + } fprintf(f, "\"0\":{\"type\":\"userdata\",\"cat\":0,\"size\":0}\n"); // to avoid issues with trailing , fprintf(f, "},\"roots\":{\n"); diff --git a/VM/src/lmem.cpp b/VM/src/lmem.cpp index 9f9d4a98..6d3b7772 100644 --- a/VM/src/lmem.cpp +++ b/VM/src/lmem.cpp @@ -8,6 +8,8 @@ #include +LUAU_FASTFLAG(LuauGcPagedSweep) + #ifndef __has_feature #define __has_feature(x) 0 #endif @@ -42,13 +44,21 @@ static_assert(sizeof(LuaNode) == ABISWITCH(32, 32, 32), "size mismatch for table #endif static_assert(offsetof(TString, data) == ABISWITCH(24, 20, 20), "size mismatch for string header"); +// TODO (FFlagLuauGcPagedSweep): this will become ABISWITCH(16, 16, 16) static_assert(offsetof(Udata, data) == ABISWITCH(24, 16, 16), "size mismatch for userdata header"); +// TODO (FFlagLuauGcPagedSweep): this will become ABISWITCH(48, 32, 32) static_assert(sizeof(Table) == ABISWITCH(56, 36, 36), "size mismatch for table header"); +// TODO (FFlagLuauGcPagedSweep): new code with old 'next' pointer requires that GCObject start at the same point as TString/UpVal +static_assert(offsetof(GCObject, uv) == 0, "UpVal data must be located at the start of the GCObject"); +static_assert(offsetof(GCObject, ts) == 0, "TString data must be located at the start of the GCObject"); + const size_t kSizeClasses = LUA_SIZECLASSES; const size_t kMaxSmallSize = 512; const size_t kPageSize = 16 * 1024 - 24; // slightly under 16KB since that results in less fragmentation due to heap metadata const size_t kBlockHeader = sizeof(double) > sizeof(void*) ? sizeof(double) : sizeof(void*); // suitable for aligning double & void* on all platforms +// TODO (FFlagLuauGcPagedSweep): when 'next' is removed, 'kBlockHeader' can be used unconditionally +const size_t kGCOHeader = sizeof(GCheader) > kBlockHeader ? sizeof(GCheader) : kBlockHeader; struct SizeClassConfig { @@ -96,6 +106,7 @@ const SizeClassConfig kSizeClassConfig; // metadata for a block is stored in the first pointer of the block #define metadata(block) (*(void**)(block)) +#define freegcolink(block) (*(void**)((char*)block + kGCOHeader)) /* ** About the realloc function: @@ -117,15 +128,22 @@ const SizeClassConfig kSizeClassConfig; struct lua_Page { + // list of pages with free blocks lua_Page* prev; lua_Page* next; + // list of all gco pages + lua_Page* gcolistprev; + lua_Page* gcolistnext; + int busyBlocks; int blockSize; void* freeList; int freeNext; + int pageSize; + union { char data[1]; @@ -141,6 +159,8 @@ l_noret luaM_toobig(lua_State* L) static lua_Page* luaM_newpage(lua_State* L, uint8_t sizeClass) { + LUAU_ASSERT(!FFlag::LuauGcPagedSweep); + global_State* g = L->global; lua_Page* page = (lua_Page*)(*g->frealloc)(L, g->ud, NULL, 0, kPageSize); if (!page) @@ -155,6 +175,9 @@ static lua_Page* luaM_newpage(lua_State* L, uint8_t sizeClass) page->prev = NULL; page->next = NULL; + page->gcolistprev = NULL; + page->gcolistnext = NULL; + page->busyBlocks = 0; page->blockSize = blockSize; @@ -171,8 +194,69 @@ static lua_Page* luaM_newpage(lua_State* L, uint8_t sizeClass) return page; } +static lua_Page* newpage(lua_State* L, lua_Page** gcopageset, int pageSize, int blockSize, int blockCount) +{ + LUAU_ASSERT(FFlag::LuauGcPagedSweep); + + global_State* g = L->global; + + LUAU_ASSERT(pageSize - offsetof(lua_Page, data) >= blockSize * blockCount); + + lua_Page* page = (lua_Page*)(*g->frealloc)(L, g->ud, NULL, 0, pageSize); + if (!page) + luaD_throw(L, LUA_ERRMEM); + + ASAN_POISON_MEMORY_REGION(page->data, blockSize * blockCount); + + // setup page header + page->prev = NULL; + page->next = NULL; + + page->gcolistprev = NULL; + page->gcolistnext = NULL; + + page->busyBlocks = 0; + page->blockSize = blockSize; + + // note: we start with the last block in the page and move downward + // either order would work, but that way we don't need to store the block count in the page + // additionally, GC stores objects in singly linked lists, and this way the GC lists end up in increasing pointer order + page->freeList = NULL; + page->freeNext = (blockCount - 1) * blockSize; + + page->pageSize = pageSize; + + if (gcopageset) + { + page->gcolistnext = *gcopageset; + if (page->gcolistnext) + page->gcolistnext->gcolistprev = page; + *gcopageset = page; + } + + return page; +} + +static lua_Page* newclasspage(lua_State* L, lua_Page** freepageset, lua_Page** gcopageset, uint8_t sizeClass, bool storeMetadata) +{ + LUAU_ASSERT(FFlag::LuauGcPagedSweep); + + int blockSize = kSizeClassConfig.sizeOfClass[sizeClass] + (storeMetadata ? kBlockHeader : 0); + int blockCount = (kPageSize - offsetof(lua_Page, data)) / blockSize; + + lua_Page* page = newpage(L, gcopageset, kPageSize, blockSize, blockCount); + + // prepend a page to page freelist (which is empty because we only ever allocate a new page when it is!) + LUAU_ASSERT(!freepageset[sizeClass]); + freepageset[sizeClass] = page; + + return page; +} + static void luaM_freepage(lua_State* L, lua_Page* page, uint8_t sizeClass) { + LUAU_ASSERT(!FFlag::LuauGcPagedSweep); + global_State* g = L->global; // remove page from freelist @@ -188,6 +272,44 @@ static void luaM_freepage(lua_State* L, lua_Page* page, uint8_t sizeClass) (*g->frealloc)(L, g->ud, page, kPageSize, 0); } +static void freepage(lua_State* L, lua_Page** gcopageset, lua_Page* page) +{ + LUAU_ASSERT(FFlag::LuauGcPagedSweep); + + global_State* g = L->global; + + if (gcopageset) + { + // remove page from alllist + if (page->gcolistnext) + page->gcolistnext->gcolistprev = page->gcolistprev; + + if (page->gcolistprev) + page->gcolistprev->gcolistnext = page->gcolistnext; + else if (*gcopageset == page) + *gcopageset = page->gcolistnext; + } + + // so long + (*g->frealloc)(L, g->ud, page, page->pageSize, 0); +} + +static void freeclasspage(lua_State* L, lua_Page** freepageset, lua_Page** gcopageset, lua_Page* page, uint8_t sizeClass) +{ + LUAU_ASSERT(FFlag::LuauGcPagedSweep); + + // remove page from freelist + if (page->next) + page->next->prev = page->prev; + + if (page->prev) + page->prev->next = page->next; + else if (freepageset[sizeClass] == page) + freepageset[sizeClass] = page->next; + + freepage(L, gcopageset, page); +} + static void* luaM_newblock(lua_State* L, int sizeClass) { global_State* g = L->global; @@ -195,7 +317,12 @@ static void* luaM_newblock(lua_State* L, int sizeClass) // slow path: no page in the freelist, allocate a new one if (!page) - page = luaM_newpage(L, sizeClass); + { + if (FFlag::LuauGcPagedSweep) + page = newclasspage(L, g->freepages, NULL, sizeClass, true); + else + page = luaM_newpage(L, sizeClass); + } LUAU_ASSERT(!page->prev); LUAU_ASSERT(page->freeList || page->freeNext >= 0); @@ -236,6 +363,55 @@ static void* luaM_newblock(lua_State* L, int sizeClass) return (char*)block + kBlockHeader; } +static void* luaM_newgcoblock(lua_State* L, int sizeClass) +{ + LUAU_ASSERT(FFlag::LuauGcPagedSweep); + + global_State* g = L->global; + lua_Page* page = g->freegcopages[sizeClass]; + + // slow path: no page in the freelist, allocate a new one + if (!page) + page = newclasspage(L, g->freegcopages, &g->allgcopages, sizeClass, false); + + LUAU_ASSERT(!page->prev); + LUAU_ASSERT(page->freeList || page->freeNext >= 0); + LUAU_ASSERT(size_t(page->blockSize) == kSizeClassConfig.sizeOfClass[sizeClass]); + + void* block; + + if (page->freeNext >= 0) + { + block = &page->data + page->freeNext; + ASAN_UNPOISON_MEMORY_REGION(block, page->blockSize); + + page->freeNext -= page->blockSize; + page->busyBlocks++; + } + else + { + // when separate block metadata is not used, free list link is stored inside the block data itself + block = (char*)page->freeList - kGCOHeader; + + ASAN_UNPOISON_MEMORY_REGION((char*)block + kGCOHeader, page->blockSize - kGCOHeader); + + page->freeList = freegcolink(block); + page->busyBlocks++; + } + + // if we allocate the last block out of a page, we need to remove it from free list + if (!page->freeList && page->freeNext < 0) + { + g->freegcopages[sizeClass] = page->next; + if (page->next) + page->next->prev = NULL; + page->next = NULL; + } + + // the user data is right after the metadata + return (char*)block; +} + static void luaM_freeblock(lua_State* L, int sizeClass, void* block) { global_State* g = L->global; @@ -270,12 +446,45 @@ static void luaM_freeblock(lua_State* L, int sizeClass, void* block) // if it's the last block in the page, we don't need the page if (page->busyBlocks == 0) - luaM_freepage(L, page, sizeClass); + { + if (FFlag::LuauGcPagedSweep) + freeclasspage(L, g->freepages, NULL, page, sizeClass); + else + luaM_freepage(L, page, sizeClass); + } +} + +static void luaM_freegcoblock(lua_State* L, int sizeClass, void* block, lua_Page* page) +{ + LUAU_ASSERT(FFlag::LuauGcPagedSweep); + + global_State* g = L->global; + + // if the page wasn't in the page free list, it should be now since it got a block! + if (!page->freeList && page->freeNext < 0) + { + LUAU_ASSERT(!page->prev); + LUAU_ASSERT(!page->next); + + page->next = g->freegcopages[sizeClass]; + if (page->next) + page->next->prev = page; + g->freegcopages[sizeClass] = page; + } + + // when separate block metadata is not used, free list link is stored inside the block data itself + freegcolink(block) = page->freeList; + page->freeList = (char*)block + kGCOHeader; + + ASAN_POISON_MEMORY_REGION((char*)block + kGCOHeader, page->blockSize - kGCOHeader); + + page->busyBlocks--; + + // if it's the last block in the page, we don't need the page + if (page->busyBlocks == 0) + freeclasspage(L, g->freegcopages, &g->allgcopages, page, sizeClass); } -/* -** generic allocation routines. -*/ void* luaM_new_(lua_State* L, size_t nsize, uint8_t memcat) { global_State* g = L->global; @@ -292,6 +501,43 @@ void* luaM_new_(lua_State* L, size_t nsize, uint8_t memcat) return block; } +GCObject* luaM_newgco_(lua_State* L, size_t nsize, uint8_t memcat) +{ + if (!FFlag::LuauGcPagedSweep) + return (GCObject*)luaM_new_(L, nsize, memcat); + + global_State* g = L->global; + + int nclass = sizeclass(nsize); + + void* block = NULL; + + if (nclass >= 0) + { + LUAU_ASSERT(nsize > 8); + + block = luaM_newgcoblock(L, nclass); + } + else + { + lua_Page* page = newpage(L, &g->allgcopages, offsetof(lua_Page, data) + nsize, nsize, 1); + + block = &page->data; + ASAN_UNPOISON_MEMORY_REGION(block, page->blockSize); + + page->freeNext -= page->blockSize; + page->busyBlocks++; + } + + if (block == NULL && nsize > 0) + luaD_throw(L, LUA_ERRMEM); + + g->totalbytes += nsize; + g->memcatbytes[memcat] += nsize; + + return (GCObject*)block; +} + void luaM_free_(lua_State* L, void* block, size_t osize, uint8_t memcat) { global_State* g = L->global; @@ -308,6 +554,36 @@ void luaM_free_(lua_State* L, void* block, size_t osize, uint8_t memcat) g->memcatbytes[memcat] -= osize; } +void luaM_freegco_(lua_State* L, GCObject* block, size_t osize, uint8_t memcat, lua_Page* page) +{ + if (!FFlag::LuauGcPagedSweep) + { + luaM_free_(L, block, osize, memcat); + return; + } + + global_State* g = L->global; + LUAU_ASSERT((osize == 0) == (block == NULL)); + + int oclass = sizeclass(osize); + + if (oclass >= 0) + { + block->gch.tt = LUA_TNIL; + + luaM_freegcoblock(L, oclass, block, page); + } + else + { + LUAU_ASSERT(page->busyBlocks == 1); + + freepage(L, &g->allgcopages, page); + } + + g->totalbytes -= osize; + g->memcatbytes[memcat] -= osize; +} + void* luaM_realloc_(lua_State* L, void* block, size_t osize, size_t nsize, uint8_t memcat) { global_State* g = L->global; @@ -344,3 +620,64 @@ void* luaM_realloc_(lua_State* L, void* block, size_t osize, size_t nsize, uint8 g->memcatbytes[memcat] += nsize - osize; return result; } + +void luaM_getpagewalkinfo(lua_Page* page, char** start, char** end, int* busyBlocks, int* blockSize) +{ + LUAU_ASSERT(FFlag::LuauGcPagedSweep); + + int blockCount = (page->pageSize - offsetof(lua_Page, data)) / page->blockSize; + + *start = page->data + page->freeNext + page->blockSize; + *end = page->data + blockCount * page->blockSize; + *busyBlocks = page->busyBlocks; + *blockSize = page->blockSize; +} + +lua_Page* luaM_getnextgcopage(lua_Page* page) +{ + return page->gcolistnext; +} + +void luaM_visitpage(lua_Page* page, void* context, bool (*visitor)(void* context, lua_Page* page, GCObject* gco)) +{ + LUAU_ASSERT(FFlag::LuauGcPagedSweep); + + char* start; + char* end; + int busyBlocks; + int blockSize; + luaM_getpagewalkinfo(page, &start, &end, &busyBlocks, &blockSize); + + for (char* pos = start; pos != end; pos += blockSize) + { + GCObject* gco = (GCObject*)pos; + + // skip memory blocks that are already freed + if (gco->gch.tt == LUA_TNIL) + continue; + + // when true is returned it means that the element was deleted + if (visitor(context, page, gco)) + { + // if the last block was removed, page would be removed as well + if (--busyBlocks == 0) + break; + } + } +} + +void luaM_visitgco(lua_State* L, void* context, bool (*visitor)(void* context, lua_Page* page, GCObject* gco)) +{ + LUAU_ASSERT(FFlag::LuauGcPagedSweep); + + global_State* g = L->global; + + for (lua_Page* curr = g->allgcopages; curr;) + { + lua_Page* next = curr->gcolistnext; // page blockvisit might destroy the page + + luaM_visitpage(curr, context, visitor); + + curr = next; + } +} diff --git a/VM/src/lmem.h b/VM/src/lmem.h index f526a1b6..1bfe48fa 100644 --- a/VM/src/lmem.h +++ b/VM/src/lmem.h @@ -4,8 +4,15 @@ #include "lua.h" +struct lua_Page; +union GCObject; + +// TODO: remove with FFlagLuauGcPagedSweep and rename luaM_newgco to luaM_new #define luaM_new(L, t, size, memcat) cast_to(t*, luaM_new_(L, size, memcat)) +#define luaM_newgco(L, t, size, memcat) cast_to(t*, luaM_newgco_(L, size, memcat)) +// TODO: remove with FFlagLuauGcPagedSweep and rename luaM_freegco to luaM_free #define luaM_free(L, p, size, memcat) luaM_free_(L, (p), size, memcat) +#define luaM_freegco(L, p, size, memcat, page) luaM_freegco_(L, obj2gco(p), size, memcat, page) #define luaM_arraysize_(n, e) ((cast_to(size_t, (n)) <= SIZE_MAX / (e)) ? (n) * (e) : (luaM_toobig(L), SIZE_MAX)) @@ -15,7 +22,15 @@ ((v) = cast_to(t*, luaM_realloc_(L, v, (oldn) * sizeof(t), luaM_arraysize_(n, sizeof(t)), memcat))) LUAI_FUNC void* luaM_new_(lua_State* L, size_t nsize, uint8_t memcat); +LUAI_FUNC GCObject* luaM_newgco_(lua_State* L, size_t nsize, uint8_t memcat); LUAI_FUNC void luaM_free_(lua_State* L, void* block, size_t osize, uint8_t memcat); +LUAI_FUNC void luaM_freegco_(lua_State* L, GCObject* block, size_t osize, uint8_t memcat, lua_Page* page); LUAI_FUNC void* luaM_realloc_(lua_State* L, void* block, size_t osize, size_t nsize, uint8_t memcat); LUAI_FUNC l_noret luaM_toobig(lua_State* L); + +LUAI_FUNC void luaM_getpagewalkinfo(lua_Page* page, char** start, char** end, int* busyBlocks, int* blockSize); +LUAI_FUNC lua_Page* luaM_getnextgcopage(lua_Page* page); + +LUAI_FUNC void luaM_visitpage(lua_Page* page, void* context, bool (*visitor)(void* context, lua_Page* page, GCObject* gco)); +LUAI_FUNC void luaM_visitgco(lua_State* L, void* context, bool (*visitor)(void* context, lua_Page* page, GCObject* gco)); diff --git a/VM/src/lobject.h b/VM/src/lobject.h index b642cf78..57ffd82a 100644 --- a/VM/src/lobject.h +++ b/VM/src/lobject.h @@ -11,12 +11,11 @@ typedef union GCObject GCObject; /* -** Common Header for all collectible objects (in macro form, to be -** included in other objects) +** Common Header for all collectible objects (in macro form, to be included in other objects) */ // clang-format off #define CommonHeader \ - GCObject* next; \ + GCObject* next; /* TODO: remove with FFlagLuauGcPagedSweep */ \ uint8_t tt; uint8_t marked; uint8_t memcat // clang-format on @@ -229,8 +228,10 @@ typedef TValue* StkId; /* index to stack elements */ typedef struct TString { CommonHeader; + // 1 byte padding int16_t atom; + // 2 byte padding unsigned int hash; unsigned int len; @@ -314,14 +315,21 @@ typedef struct LocVar typedef struct UpVal { CommonHeader; + // 1 (x86) or 5 (x64) byte padding TValue* v; /* points to stack or to its own value */ union { TValue value; /* the value (when closed) */ struct - { /* double linked list (when open) */ + { + /* global double linked list (when open) */ struct UpVal* prev; struct UpVal* next; + + /* thread double linked list (when open) */ + // TODO: when FFlagLuauGcPagedSweep is removed, old outer 'next' value will be placed here + /* note: this is the location of a pointer to this upvalue in the previous element that can be either an UpVal or a lua_State */ + struct UpVal** threadprev; } l; } u; } UpVal; diff --git a/VM/src/lstate.cpp b/VM/src/lstate.cpp index 24e97063..6762c638 100644 --- a/VM/src/lstate.cpp +++ b/VM/src/lstate.cpp @@ -10,6 +10,8 @@ #include "ldo.h" #include "ldebug.h" +LUAU_FASTFLAG(LuauGcPagedSweep) + /* ** Main thread combines a thread state and the global state */ @@ -86,14 +88,21 @@ static void close_state(lua_State* L) global_State* g = L->global; luaF_close(L, L->stack); /* close all upvalues for this thread */ luaC_freeall(L); /* collect all objects */ - LUAU_ASSERT(g->rootgc == obj2gco(L)); + if (!FFlag::LuauGcPagedSweep) + LUAU_ASSERT(g->rootgc == obj2gco(L)); LUAU_ASSERT(g->strbufgc == NULL); LUAU_ASSERT(g->strt.nuse == 0); luaM_freearray(L, L->global->strt.hash, L->global->strt.size, TString*, 0); freestack(L, L); - LUAU_ASSERT(g->totalbytes == sizeof(LG)); for (int i = 0; i < LUA_SIZECLASSES; i++) + { LUAU_ASSERT(g->freepages[i] == NULL); + if (FFlag::LuauGcPagedSweep) + LUAU_ASSERT(g->freegcopages[i] == NULL); + } + if (FFlag::LuauGcPagedSweep) + LUAU_ASSERT(g->allgcopages == NULL); + LUAU_ASSERT(g->totalbytes == sizeof(LG)); LUAU_ASSERT(g->memcatbytes[0] == sizeof(LG)); for (int i = 1; i < LUA_MEMORY_CATEGORIES; i++) LUAU_ASSERT(g->memcatbytes[i] == 0); @@ -102,7 +111,7 @@ static void close_state(lua_State* L) lua_State* luaE_newthread(lua_State* L) { - lua_State* L1 = luaM_new(L, lua_State, sizeof(lua_State), L->activememcat); + lua_State* L1 = luaM_newgco(L, lua_State, sizeof(lua_State), L->activememcat); luaC_link(L, L1, LUA_TTHREAD); preinit_state(L1, L->global); L1->activememcat = L->activememcat; // inherit the active memory category @@ -113,7 +122,7 @@ lua_State* luaE_newthread(lua_State* L) return L1; } -void luaE_freethread(lua_State* L, lua_State* L1) +void luaE_freethread(lua_State* L, lua_State* L1, lua_Page* page) { luaF_close(L1, L1->stack); /* close all upvalues for this thread */ LUAU_ASSERT(L1->openupval == NULL); @@ -121,7 +130,7 @@ void luaE_freethread(lua_State* L, lua_State* L1) if (g->cb.userthread) g->cb.userthread(NULL, L1); freestack(L, L1); - luaM_free(L, L1, sizeof(lua_State), L1->memcat); + luaM_freegco(L, L1, sizeof(lua_State), L1->memcat, page); } void lua_resetthread(lua_State* L) @@ -162,7 +171,8 @@ lua_State* lua_newstate(lua_Alloc f, void* ud) return NULL; L = (lua_State*)l; g = &((LG*)L)->g; - L->next = NULL; + if (!FFlag::LuauGcPagedSweep) + L->next = NULL; L->tt = LUA_TTHREAD; L->marked = g->currentwhite = bit2mask(WHITE0BIT, FIXEDBIT); L->memcat = 0; @@ -185,9 +195,11 @@ lua_State* lua_newstate(lua_Alloc f, void* ud) g->strt.hash = NULL; setnilvalue(registry(L)); g->gcstate = GCSpause; - g->rootgc = obj2gco(L); + if (!FFlag::LuauGcPagedSweep) + g->rootgc = obj2gco(L); g->sweepstrgc = 0; - g->sweepgc = &g->rootgc; + if (!FFlag::LuauGcPagedSweep) + g->sweepgc = &g->rootgc; g->gray = NULL; g->grayagain = NULL; g->weak = NULL; @@ -197,7 +209,16 @@ lua_State* lua_newstate(lua_Alloc f, void* ud) g->gcstepmul = LUAI_GCSTEPMUL; g->gcstepsize = LUAI_GCSTEPSIZE << 10; for (i = 0; i < LUA_SIZECLASSES; i++) + { g->freepages[i] = NULL; + if (FFlag::LuauGcPagedSweep) + g->freegcopages[i] = NULL; + } + if (FFlag::LuauGcPagedSweep) + { + g->allgcopages = NULL; + g->sweepgcopage = NULL; + } for (i = 0; i < LUA_T_COUNT; i++) g->mt[i] = NULL; for (i = 0; i < LUA_UTAG_LIMIT; i++) diff --git a/VM/src/lstate.h b/VM/src/lstate.h index 56379883..080f0024 100644 --- a/VM/src/lstate.h +++ b/VM/src/lstate.h @@ -22,7 +22,7 @@ typedef struct stringtable { - GCObject** hash; + TString** hash; uint32_t nuse; /* number of elements */ int size; } stringtable; @@ -149,13 +149,15 @@ typedef struct global_State int sweepstrgc; /* position of sweep in `strt' */ + // TODO: remove with FFlagLuauGcPagedSweep GCObject* rootgc; /* list of all collectable objects */ + // TODO: remove with FFlagLuauGcPagedSweep GCObject** sweepgc; /* position of sweep in `rootgc' */ GCObject* gray; /* list of gray objects */ GCObject* grayagain; /* list of objects to be traversed atomically */ GCObject* weak; /* list of weak tables (to be cleared) */ - GCObject* strbufgc; // list of all string buffer objects + TString* strbufgc; // list of all string buffer objects size_t GCthreshold; // when totalbytes > GCthreshold; run GC step @@ -164,7 +166,10 @@ typedef struct global_State int gcstepmul; // see LUAI_GCSTEPMUL int gcstepsize; // see LUAI_GCSTEPSIZE - struct lua_Page* freepages[LUA_SIZECLASSES]; /* free page linked list for each size class */ + struct lua_Page* freepages[LUA_SIZECLASSES]; // free page linked list for each size class for non-collectable objects + struct lua_Page* freegcopages[LUA_SIZECLASSES]; // free page linked list for each size class for collectable objects + struct lua_Page* allgcopages; // page linked list with all pages for all classes + struct lua_Page* sweepgcopage; // position of the sweep in `allgcopages' size_t memcatbytes[LUA_MEMORY_CATEGORIES]; /* total amount of memory used by each memory category */ @@ -231,7 +236,7 @@ struct lua_State TValue l_gt; /* table of globals */ TValue env; /* temporary place for environments */ - GCObject* openupval; /* list of open upvalues in this stack */ + UpVal* openupval; /* list of open upvalues in this stack */ GCObject* gclist; TString* namecall; /* when invoked from Luau using NAMECALL, what method do we need to invoke? */ @@ -268,4 +273,4 @@ union GCObject #define obj2gco(v) check_exp(iscollectable(v), cast_to(GCObject*, (v) + 0)) LUAI_FUNC lua_State* luaE_newthread(lua_State* L); -LUAI_FUNC void luaE_freethread(lua_State* L, lua_State* L1); +LUAI_FUNC void luaE_freethread(lua_State* L, lua_State* L1, struct lua_Page* page); diff --git a/VM/src/lstring.cpp b/VM/src/lstring.cpp index a9e90d17..cb22cc23 100644 --- a/VM/src/lstring.cpp +++ b/VM/src/lstring.cpp @@ -7,6 +7,8 @@ #include +LUAU_FASTFLAG(LuauGcPagedSweep) + unsigned int luaS_hash(const char* str, size_t len) { // Note that this hashing algorithm is replicated in BytecodeBuilder.cpp, BytecodeBuilder::getStringHash @@ -44,26 +46,25 @@ unsigned int luaS_hash(const char* str, size_t len) void luaS_resize(lua_State* L, int newsize) { - GCObject** newhash; - stringtable* tb; - int i; if (L->global->gcstate == GCSsweepstring) return; /* cannot resize during GC traverse */ - newhash = luaM_newarray(L, newsize, GCObject*, 0); - tb = &L->global->strt; - for (i = 0; i < newsize; i++) + TString** newhash = luaM_newarray(L, newsize, TString*, 0); + stringtable* tb = &L->global->strt; + for (int i = 0; i < newsize; i++) newhash[i] = NULL; /* rehash */ - for (i = 0; i < tb->size; i++) + for (int i = 0; i < tb->size; i++) { - GCObject* p = tb->hash[i]; + TString* p = tb->hash[i]; while (p) { /* for each node in the list */ - GCObject* next = p->gch.next; /* save next */ - unsigned int h = gco2ts(p)->hash; + // TODO (FFlagLuauGcPagedSweep): 'next' type will change after removal of the flag and the cast will not be required + TString* next = (TString*)p->next; /* save next */ + unsigned int h = p->hash; int h1 = lmod(h, newsize); /* new position */ LUAU_ASSERT(cast_int(h % newsize) == lmod(h, newsize)); - p->gch.next = newhash[h1]; /* chain it */ + // TODO (FFlagLuauGcPagedSweep): 'next' type will change after removal of the flag and the cast will not be required + p->next = (GCObject*)newhash[h1]; /* chain it */ newhash[h1] = p; p = next; } @@ -79,7 +80,7 @@ static TString* newlstr(lua_State* L, const char* str, size_t l, unsigned int h) stringtable* tb; if (l > MAXSSIZE) luaM_toobig(L); - ts = luaM_new(L, TString, sizestring(l), L->activememcat); + ts = luaM_newgco(L, TString, sizestring(l), L->activememcat); ts->len = unsigned(l); ts->hash = h; ts->marked = luaC_white(L->global); @@ -90,8 +91,9 @@ static TString* newlstr(lua_State* L, const char* str, size_t l, unsigned int h) ts->atom = L->global->cb.useratom ? L->global->cb.useratom(ts->data, l) : -1; tb = &L->global->strt; h = lmod(h, tb->size); - ts->next = tb->hash[h]; /* chain new entry */ - tb->hash[h] = obj2gco(ts); + // TODO (FFlagLuauGcPagedSweep): 'next' type will change after removal of the flag and the case will not be required + ts->next = (GCObject*)tb->hash[h]; /* chain new entry */ + tb->hash[h] = ts; tb->nuse++; if (tb->nuse > cast_to(uint32_t, tb->size) && tb->size <= INT_MAX / 2) luaS_resize(L, tb->size * 2); /* too crowded */ @@ -101,28 +103,41 @@ static TString* newlstr(lua_State* L, const char* str, size_t l, unsigned int h) static void linkstrbuf(lua_State* L, TString* ts) { global_State* g = L->global; - GCObject* o = obj2gco(ts); - o->gch.next = g->strbufgc; - g->strbufgc = o; - o->gch.marked = luaC_white(g); + + if (FFlag::LuauGcPagedSweep) + { + // TODO (FFlagLuauGcPagedSweep): 'next' type will change after removal of the flag and the cast will not be required + ts->next = (GCObject*)g->strbufgc; + g->strbufgc = ts; + ts->marked = luaC_white(g); + } + else + { + GCObject* o = obj2gco(ts); + o->gch.next = (GCObject*)g->strbufgc; + g->strbufgc = gco2ts(o); + o->gch.marked = luaC_white(g); + } } static void unlinkstrbuf(lua_State* L, TString* ts) { global_State* g = L->global; - GCObject** p = &g->strbufgc; + TString** p = &g->strbufgc; - while (GCObject* curr = *p) + while (TString* curr = *p) { - if (curr == obj2gco(ts)) + if (curr == ts) { - *p = curr->gch.next; + // TODO (FFlagLuauGcPagedSweep): 'next' type will change after removal of the flag and the cast will not be required + *p = (TString*)curr->next; return; } else { - p = &curr->gch.next; + // TODO (FFlagLuauGcPagedSweep): 'next' type will change after removal of the flag and the cast will not be required + p = (TString**)&curr->next; } } @@ -134,7 +149,7 @@ TString* luaS_bufstart(lua_State* L, size_t size) if (size > MAXSSIZE) luaM_toobig(L); - TString* ts = luaM_new(L, TString, sizestring(size), L->activememcat); + TString* ts = luaM_newgco(L, TString, sizestring(size), L->activememcat); ts->tt = LUA_TSTRING; ts->memcat = L->activememcat; @@ -152,15 +167,14 @@ TString* luaS_buffinish(lua_State* L, TString* ts) int bucket = lmod(h, tb->size); // search if we already have this string in the hash table - for (GCObject* o = tb->hash[bucket]; o != NULL; o = o->gch.next) + // TODO (FFlagLuauGcPagedSweep): 'next' type will change after removal of the flag and the cast will not be required + for (TString* el = tb->hash[bucket]; el != NULL; el = (TString*)el->next) { - TString* el = gco2ts(o); - if (el->len == ts->len && memcmp(el->data, ts->data, ts->len) == 0) { // string may be dead - if (isdead(L->global, o)) - changewhite(o); + if (isdead(L->global, obj2gco(el))) + changewhite(obj2gco(el)); return el; } @@ -173,8 +187,9 @@ TString* luaS_buffinish(lua_State* L, TString* ts) // Complete string object ts->atom = L->global->cb.useratom ? L->global->cb.useratom(ts->data, ts->len) : -1; - ts->next = tb->hash[bucket]; // chain new entry - tb->hash[bucket] = obj2gco(ts); + // TODO (FFlagLuauGcPagedSweep): 'next' type will change after removal of the flag and the cast will not be required + ts->next = (GCObject*)tb->hash[bucket]; // chain new entry + tb->hash[bucket] = ts; tb->nuse++; if (tb->nuse > cast_to(uint32_t, tb->size) && tb->size <= INT_MAX / 2) @@ -185,24 +200,63 @@ TString* luaS_buffinish(lua_State* L, TString* ts) TString* luaS_newlstr(lua_State* L, const char* str, size_t l) { - GCObject* o; unsigned int h = luaS_hash(str, l); - for (o = L->global->strt.hash[lmod(h, L->global->strt.size)]; o != NULL; o = o->gch.next) + // TODO (FFlagLuauGcPagedSweep): 'next' type will change after removal of the flag and the cast will not be required + for (TString* el = L->global->strt.hash[lmod(h, L->global->strt.size)]; el != NULL; el = (TString*)el->next) { - TString* ts = gco2ts(o); - if (ts->len == l && (memcmp(str, getstr(ts), l) == 0)) + if (el->len == l && (memcmp(str, getstr(el), l) == 0)) { /* string may be dead */ - if (isdead(L->global, o)) - changewhite(o); - return ts; + if (isdead(L->global, obj2gco(el))) + changewhite(obj2gco(el)); + return el; } } return newlstr(L, str, l, h); /* not found */ } -void luaS_free(lua_State* L, TString* ts) +static bool unlinkstr(lua_State* L, TString* ts) { - L->global->strt.nuse--; - luaM_free(L, ts, sizestring(ts->len), ts->memcat); + LUAU_ASSERT(FFlag::LuauGcPagedSweep); + + global_State* g = L->global; + + TString** p = &g->strt.hash[lmod(ts->hash, g->strt.size)]; + + while (TString* curr = *p) + { + if (curr == ts) + { + // TODO (FFlagLuauGcPagedSweep): 'next' type will change after removal of the flag and the cast will not be required + *p = (TString*)curr->next; + return true; + } + else + { + // TODO (FFlagLuauGcPagedSweep): 'next' type will change after removal of the flag and the cast will not be required + p = (TString**)&curr->next; + } + } + + return false; +} + +void luaS_free(lua_State* L, TString* ts, lua_Page* page) +{ + if (FFlag::LuauGcPagedSweep) + { + // Unchain from the string table + if (!unlinkstr(L, ts)) + unlinkstrbuf(L, ts); // An unlikely scenario when we have a string buffer on our hands + else + L->global->strt.nuse--; + + luaM_freegco(L, ts, sizestring(ts->len), ts->memcat, page); + } + else + { + L->global->strt.nuse--; + + luaM_free(L, ts, sizestring(ts->len), ts->memcat); + } } diff --git a/VM/src/lstring.h b/VM/src/lstring.h index 3fd0bd39..290b64d8 100644 --- a/VM/src/lstring.h +++ b/VM/src/lstring.h @@ -20,7 +20,7 @@ LUAI_FUNC unsigned int luaS_hash(const char* str, size_t len); LUAI_FUNC void luaS_resize(lua_State* L, int newsize); LUAI_FUNC TString* luaS_newlstr(lua_State* L, const char* str, size_t l); -LUAI_FUNC void luaS_free(lua_State* L, TString* ts); +LUAI_FUNC void luaS_free(lua_State* L, TString* ts, struct lua_Page* page); LUAI_FUNC TString* luaS_bufstart(lua_State* L, size_t size); LUAI_FUNC TString* luaS_buffinish(lua_State* L, TString* ts); diff --git a/VM/src/ltable.cpp b/VM/src/ltable.cpp index 83b59f3f..c57374e0 100644 --- a/VM/src/ltable.cpp +++ b/VM/src/ltable.cpp @@ -424,7 +424,7 @@ static void rehash(lua_State* L, Table* t, const TValue* ek) Table* luaH_new(lua_State* L, int narray, int nhash) { - Table* t = luaM_new(L, Table, sizeof(Table), L->activememcat); + Table* t = luaM_newgco(L, Table, sizeof(Table), L->activememcat); luaC_link(L, t, LUA_TTABLE); t->metatable = NULL; t->flags = cast_byte(~0); @@ -443,12 +443,12 @@ Table* luaH_new(lua_State* L, int narray, int nhash) return t; } -void luaH_free(lua_State* L, Table* t) +void luaH_free(lua_State* L, Table* t, lua_Page* page) { if (t->node != dummynode) luaM_freearray(L, t->node, sizenode(t), LuaNode, t->memcat); luaM_freearray(L, t->array, t->sizearray, TValue, t->memcat); - luaM_free(L, t, sizeof(Table), t->memcat); + luaM_freegco(L, t, sizeof(Table), t->memcat, page); } static LuaNode* getfreepos(Table* t) @@ -741,7 +741,7 @@ int luaH_getn(Table* t) Table* luaH_clone(lua_State* L, Table* tt) { - Table* t = luaM_new(L, Table, sizeof(Table), L->activememcat); + Table* t = luaM_newgco(L, Table, sizeof(Table), L->activememcat); luaC_link(L, t, LUA_TTABLE); t->metatable = tt->metatable; t->flags = tt->flags; diff --git a/VM/src/ltable.h b/VM/src/ltable.h index 45061443..e8413c85 100644 --- a/VM/src/ltable.h +++ b/VM/src/ltable.h @@ -20,7 +20,7 @@ LUAI_FUNC TValue* luaH_set(lua_State* L, Table* t, const TValue* key); LUAI_FUNC Table* luaH_new(lua_State* L, int narray, int lnhash); LUAI_FUNC void luaH_resizearray(lua_State* L, Table* t, int nasize); LUAI_FUNC void luaH_resizehash(lua_State* L, Table* t, int nhsize); -LUAI_FUNC void luaH_free(lua_State* L, Table* t); +LUAI_FUNC void luaH_free(lua_State* L, Table* t, struct lua_Page* page); LUAI_FUNC int luaH_next(lua_State* L, Table* t, StkId key); LUAI_FUNC int luaH_getn(Table* t); LUAI_FUNC Table* luaH_clone(lua_State* L, Table* tt); diff --git a/VM/src/ludata.cpp b/VM/src/ludata.cpp index d180c388..758a9bdb 100644 --- a/VM/src/ludata.cpp +++ b/VM/src/ludata.cpp @@ -11,7 +11,7 @@ Udata* luaU_newudata(lua_State* L, size_t s, int tag) { if (s > INT_MAX - sizeof(Udata)) luaM_toobig(L); - Udata* u = luaM_new(L, Udata, sizeudata(s), L->activememcat); + Udata* u = luaM_newgco(L, Udata, sizeudata(s), L->activememcat); luaC_link(L, u, LUA_TUSERDATA); u->len = int(s); u->metatable = NULL; @@ -20,7 +20,7 @@ Udata* luaU_newudata(lua_State* L, size_t s, int tag) return u; } -void luaU_freeudata(lua_State* L, Udata* u) +void luaU_freeudata(lua_State* L, Udata* u, lua_Page* page) { LUAU_ASSERT(u->tag < LUA_UTAG_LIMIT || u->tag == UTAG_IDTOR); @@ -33,5 +33,5 @@ void luaU_freeudata(lua_State* L, Udata* u) if (dtor) dtor(u->data); - luaM_free(L, u, sizeudata(u->len), u->memcat); + luaM_freegco(L, u, sizeudata(u->len), u->memcat, page); } diff --git a/VM/src/ludata.h b/VM/src/ludata.h index 59cb85bd..ec374c28 100644 --- a/VM/src/ludata.h +++ b/VM/src/ludata.h @@ -10,4 +10,4 @@ #define sizeudata(len) (offsetof(Udata, data) + len) LUAI_FUNC Udata* luaU_newudata(lua_State* L, size_t s, int tag); -LUAI_FUNC void luaU_freeudata(lua_State* L, Udata* u); +LUAI_FUNC void luaU_freeudata(lua_State* L, Udata* u, struct lua_Page* page); diff --git a/VM/src/lvmexecute.cpp b/VM/src/lvmexecute.cpp index cebeeb58..e58ff2a8 100644 --- a/VM/src/lvmexecute.cpp +++ b/VM/src/lvmexecute.cpp @@ -496,7 +496,7 @@ static void luau_execute(lua_State* L) Instruction insn = *pc++; StkId ra = VM_REG(LUAU_INSN_A(insn)); - if (L->openupval && gco2uv(L->openupval)->v >= ra) + if (L->openupval && L->openupval->v >= ra) luaF_close(L, ra); VM_NEXT(); } diff --git a/tests/Compiler.test.cpp b/tests/Compiler.test.cpp index 8eed953f..3b0d677d 100644 --- a/tests/Compiler.test.cpp +++ b/tests/Compiler.test.cpp @@ -756,6 +756,30 @@ RETURN R0 1 )"); } +TEST_CASE("TableSizePredictionLoop") +{ + ScopedFastFlag sff("LuauPredictTableSizeLoop", true); + + CHECK_EQ("\n" + compileFunction0(R"( +local t = {} +for i=1,4 do + t[i] = 0 +end +return t +)"), + R"( +NEWTABLE R0 0 4 +LOADN R3 1 +LOADN R1 4 +LOADN R2 1 +FORNPREP R1 +3 +LOADN R4 0 +SETTABLE R4 R0 R3 +FORNLOOP R1 -3 +RETURN R0 1 +)"); +} + TEST_CASE("ReflectionEnums") { CHECK_EQ("\n" + compileFunction0("return Enum.EasingStyle.Linear"), R"( diff --git a/tests/Linter.test.cpp b/tests/Linter.test.cpp index 1d13df28..5ad06f0d 100644 --- a/tests/Linter.test.cpp +++ b/tests/Linter.test.cpp @@ -1396,6 +1396,8 @@ end TEST_CASE_FIXTURE(Fixture, "TableOperations") { + ScopedFastFlag sff("LuauLintTableCreateTable", true); + LintResult result = lintTyped(R"( local t = {} local tt = {} @@ -1416,9 +1418,12 @@ table.insert(t, string.find("hello", "h")) table.move(t, 0, #t, 1, tt) table.move(t, 1, #t, 0, tt) + +table.create(42, {}) +table.create(42, {} :: {}) )"); - REQUIRE_EQ(result.warnings.size(), 8); + REQUIRE_EQ(result.warnings.size(), 10); CHECK_EQ(result.warnings[0].text, "table.insert will insert the value before the last element, which is likely a bug; consider removing the " "second argument or wrap it in parentheses to silence"); CHECK_EQ(result.warnings[1].text, "table.insert will append the value to the table; consider removing the second argument for efficiency"); @@ -1430,6 +1435,8 @@ table.move(t, 1, #t, 0, tt) "table.insert may change behavior if the call returns more than one result; consider adding parentheses around second argument"); CHECK_EQ(result.warnings[6].text, "table.move uses index 0 but arrays are 1-based; did you mean 1 instead?"); CHECK_EQ(result.warnings[7].text, "table.move uses index 0 but arrays are 1-based; did you mean 1 instead?"); + CHECK_EQ(result.warnings[8].text, "table.create with a table literal will reuse the same object for all elements; consider using a for loop instead"); + CHECK_EQ(result.warnings[9].text, "table.create with a table literal will reuse the same object for all elements; consider using a for loop instead"); } TEST_CASE_FIXTURE(Fixture, "DuplicateConditions") diff --git a/tests/Parser.test.cpp b/tests/Parser.test.cpp index e9135651..ac81005c 100644 --- a/tests/Parser.test.cpp +++ b/tests/Parser.test.cpp @@ -1498,6 +1498,17 @@ return CHECK_EQ(std::string(str->value.data, str->value.size), "\n"); } +TEST_CASE_FIXTURE(Fixture, "parse_error_broken_comment") +{ + ScopedFastFlag luauStartingBrokenComment{"LuauStartingBrokenComment", true}; + + const char* expected = "Expected identifier when parsing expression, got unfinished comment"; + + matchParseError("--[[unfinished work", expected); + matchParseError("--!strict\n--[[unfinished work", expected); + matchParseError("local x = 1 --[[unfinished work", expected); +} + TEST_CASE_FIXTURE(Fixture, "string_literals_escapes_broken") { const char* expected = "String literal contains malformed escape sequence"; @@ -2333,7 +2344,7 @@ TEST_CASE_FIXTURE(Fixture, "capture_broken_comment_at_the_start_of_the_file") ParseOptions options; options.captureComments = true; - ParseResult result = parseEx(R"( + ParseResult result = tryParse(R"( --[[ )", options); diff --git a/tests/TypeInfer.tables.test.cpp b/tests/TypeInfer.tables.test.cpp index 27cda146..644efed7 100644 --- a/tests/TypeInfer.tables.test.cpp +++ b/tests/TypeInfer.tables.test.cpp @@ -2180,4 +2180,52 @@ b() CHECK_EQ(toString(result.errors[0]), R"(Cannot call non-function t1 where t1 = { @metatable { __call: t1 }, { } })"); } +TEST_CASE_FIXTURE(Fixture, "length_operator_union") +{ + ScopedFastFlag luauLengthOnCompositeType{"LuauLengthOnCompositeType", true}; + + CheckResult result = check(R"( +local x: {number} | {string} +local y = #x + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "length_operator_intersection") +{ + ScopedFastFlag luauLengthOnCompositeType{"LuauLengthOnCompositeType", true}; + + CheckResult result = check(R"( +local x: {number} & {z:string} -- mixed tables are evil +local y = #x + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "length_operator_non_table_union") +{ + ScopedFastFlag luauLengthOnCompositeType{"LuauLengthOnCompositeType", true}; + + CheckResult result = check(R"( +local x: {number} | any | string +local y = #x + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "length_operator_union_errors") +{ + ScopedFastFlag luauLengthOnCompositeType{"LuauLengthOnCompositeType", true}; + + CheckResult result = check(R"( +local x: {number} | number | string +local y = #x + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.test.cpp b/tests/TypeInfer.test.cpp index 7a056af5..7ee5253c 100644 --- a/tests/TypeInfer.test.cpp +++ b/tests/TypeInfer.test.cpp @@ -5129,4 +5129,33 @@ local c = a or b LUAU_REQUIRE_NO_ERRORS(result); } +TEST_CASE_FIXTURE(Fixture, "bound_typepack_promote") +{ + ScopedFastFlag luauCommittingTxnLogFreeTpPromote{"LuauCommittingTxnLogFreeTpPromote", true}; + + // No assertions should trigger + check(R"( +local function p() + local this = {} + this.pf = foo() + function this:IsActive() end + function this:Start(o) end + return this +end + +local function h(tp, o) + ep = tp + tp:Start(o) + tp.pf.Connect(function() + ep:IsActive() + end) +end + +function on() + local t = p() + h(t) +end + )"); +} + TEST_SUITE_END(); diff --git a/tests/conformance/closure.lua b/tests/conformance/closure.lua index f32d5bdc..7b057354 100644 --- a/tests/conformance/closure.lua +++ b/tests/conformance/closure.lua @@ -419,5 +419,20 @@ co = coroutine.create(function () return loadstring("return a")() end) +-- large closure size +do + local a1, a2, a3, a4, a5, a6, a7, a8, a9, a0 + local b1, b2, b3, b4, b5, b6, b7, b8, b9, b0 + local c1, c2, c3, c4, c5, c6, c7, c8, c9, c0 + local d1, d2, d3, d4, d5, d6, d7, d8, d9, d0 + + local f = function() + return + a1 + a2 + a3 + a4 + a5 + a6 + a7 + a8 + a9 + a0 + + b1 + b2 + b3 + b4 + b5 + b6 + b7 + b8 + b9 + b0 + + c1 + c2 + c3 + c4 + c5 + c6 + c7 + c8 + c9 + c0 + + d1 + d2 + d3 + d4 + d5 + d6 + d7 + d8 + d9 + d0 + end +end return 'OK' diff --git a/tests/conformance/gc.lua b/tests/conformance/gc.lua index 6d9eb854..409cd224 100644 --- a/tests/conformance/gc.lua +++ b/tests/conformance/gc.lua @@ -291,4 +291,32 @@ do for i = 1,10 do table.insert(___Glob, newproxy(true)) end end +-- create threads that die together with their unmarked upvalues +do + local t = {} + + for i = 1,100 do + local c = coroutine.wrap(function() + local uv = {i + 1} + local function f() + return uv[1] * 10 + end + coroutine.yield(uv[1]) + uv = {i + 2} + coroutine.yield(f()) + end) + + assert(c() == i + 1) + table.insert(t, c) + end + + for i = 1,100 do + t[i] = nil + end + + collectgarbage() + +end + + return('OK') From 699660a4ebe33c582a71f73caacdc98440228b5d Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Fri, 21 Jan 2022 08:37:50 -0800 Subject: [PATCH 134/399] Fix MSVC warnings --- VM/src/lgc.cpp | 4 ++-- VM/src/lmem.cpp | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/VM/src/lgc.cpp b/VM/src/lgc.cpp index 50859b1e..9a8cb079 100644 --- a/VM/src/lgc.cpp +++ b/VM/src/lgc.cpp @@ -770,11 +770,11 @@ static int sweepgcopage(lua_State* L, lua_Page* page) { // if the last block was removed, page would be removed as well if (--busyBlocks == 0) - return (pos - start) / blockSize + 1; + return int(pos - start) / blockSize + 1; } } - return (end - start) / blockSize; + return int(end - start) / blockSize; } static size_t gcstep(lua_State* L, size_t limit) diff --git a/VM/src/lmem.cpp b/VM/src/lmem.cpp index 6d3b7772..7a31d6c8 100644 --- a/VM/src/lmem.cpp +++ b/VM/src/lmem.cpp @@ -520,7 +520,7 @@ GCObject* luaM_newgco_(lua_State* L, size_t nsize, uint8_t memcat) } else { - lua_Page* page = newpage(L, &g->allgcopages, offsetof(lua_Page, data) + nsize, nsize, 1); + lua_Page* page = newpage(L, &g->allgcopages, offsetof(lua_Page, data) + int(nsize), int(nsize), 1); block = &page->data; ASAN_UNPOISON_MEMORY_REGION(block, page->blockSize); From 0062000d4674c44bb145584b2baa531388c9f355 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Fri, 21 Jan 2022 08:43:41 -0800 Subject: [PATCH 135/399] One more --- VM/src/lmem.cpp | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/VM/src/lmem.cpp b/VM/src/lmem.cpp index 7a31d6c8..beacca65 100644 --- a/VM/src/lmem.cpp +++ b/VM/src/lmem.cpp @@ -200,7 +200,7 @@ static lua_Page* newpage(lua_State* L, lua_Page** gcopageset, int pageSize, int global_State* g = L->global; - LUAU_ASSERT(pageSize - offsetof(lua_Page, data) >= blockSize * blockCount); + LUAU_ASSERT(pageSize - int(offsetof(lua_Page, data)) >= blockSize * blockCount); lua_Page* page = (lua_Page*)(*g->frealloc)(L, g->ud, NULL, 0, pageSize); if (!page) From 9c15f6a6d79d769f4ac6cf80b1521826035f4d76 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Fri, 21 Jan 2022 08:52:48 -0800 Subject: [PATCH 136/399] And one more --- VM/src/lmem.cpp | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/VM/src/lmem.cpp b/VM/src/lmem.cpp index beacca65..e1dbce50 100644 --- a/VM/src/lmem.cpp +++ b/VM/src/lmem.cpp @@ -376,7 +376,7 @@ static void* luaM_newgcoblock(lua_State* L, int sizeClass) LUAU_ASSERT(!page->prev); LUAU_ASSERT(page->freeList || page->freeNext >= 0); - LUAU_ASSERT(size_t(page->blockSize) == kSizeClassConfig.sizeOfClass[sizeClass]); + LUAU_ASSERT(page->blockSize == kSizeClassConfig.sizeOfClass[sizeClass]); void* block; From 8fe95c99630063ead551303d3ecb6888baeb911f Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Fri, 21 Jan 2022 09:00:19 -0800 Subject: [PATCH 137/399] Sync to upstream/release/511 (#324) - TableOperations lint now includes a warning for table.create(N, {}) (which is likely a mistake since the table is shared by all entries) - Type checker now type checks #v when v is a union - Parser now rejects sources that consists of a single unfinished long comment - Work around significant MSVC 2022 performance regression, bringing it more or less in line with MSVC 2019 - Compiler now predicts array size for newly allocated tables when the table is filled in a short loop - Small improvements in compilation throughput (~2% faster) - Implement paged sweeper for GC which improves sweep throughput 2-3x and reduces memory consumption by 8 bytes per object (once it is stabilized we will see additional 8 bytes per object of savings) - Improve Repl Tab completion - Repl now supports -i (interactive mode to run code in context of a script's environment) and -On (to control optimization flags) --- Analysis/include/Luau/TypeVar.h | 4 + Analysis/src/Linter.cpp | 15 ++ Analysis/src/TypeInfer.cpp | 29 ++- Analysis/src/TypeVar.cpp | 46 +++++ Analysis/src/Unifier.cpp | 6 + Ast/include/Luau/Ast.h | 3 +- Ast/include/Luau/DenseHash.h | 8 +- Ast/src/Parser.cpp | 20 +- CLI/Repl.cpp | 110 +++++++--- CMakeLists.txt | 6 + Compiler/src/TableShape.cpp | 49 ++++- VM/src/ldo.cpp | 9 +- VM/src/lfunc.cpp | 96 ++++++--- VM/src/lfunc.h | 7 +- VM/src/lgc.cpp | 251 +++++++++++++++++++---- VM/src/lgc.h | 1 + VM/src/lgcdebug.cpp | 77 +++++-- VM/src/lmem.cpp | 347 +++++++++++++++++++++++++++++++- VM/src/lmem.h | 15 ++ VM/src/lobject.h | 16 +- VM/src/lstate.cpp | 37 +++- VM/src/lstate.h | 15 +- VM/src/lstring.cpp | 136 +++++++++---- VM/src/lstring.h | 2 +- VM/src/ltable.cpp | 8 +- VM/src/ltable.h | 2 +- VM/src/ludata.cpp | 6 +- VM/src/ludata.h | 2 +- VM/src/lvmexecute.cpp | 2 +- tests/Compiler.test.cpp | 24 +++ tests/Linter.test.cpp | 9 +- tests/Parser.test.cpp | 13 +- tests/TypeInfer.tables.test.cpp | 48 +++++ tests/TypeInfer.test.cpp | 29 +++ tests/conformance/closure.lua | 15 ++ tests/conformance/gc.lua | 28 +++ 36 files changed, 1275 insertions(+), 216 deletions(-) diff --git a/Analysis/include/Luau/TypeVar.h b/Analysis/include/Luau/TypeVar.h index fd2c2afa..3f5e26d6 100644 --- a/Analysis/include/Luau/TypeVar.h +++ b/Analysis/include/Luau/TypeVar.h @@ -1,6 +1,7 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #pragma once +#include "Luau/DenseHash.h" #include "Luau/Predicate.h" #include "Luau/Unifiable.h" #include "Luau/Variant.h" @@ -499,6 +500,9 @@ bool maybeGeneric(const TypeId ty); // Checks if a type is of the form T1|...|Tn where one of the Ti is a singleton bool maybeSingleton(TypeId ty); +// Checks if the length operator can be applied on the value of type +bool hasLength(TypeId ty, DenseHashSet& seen, int* recursionCount); + struct SingletonTypes { const TypeId nilType; diff --git a/Analysis/src/Linter.cpp b/Analysis/src/Linter.cpp index 1a5b24fe..905b70bf 100644 --- a/Analysis/src/Linter.cpp +++ b/Analysis/src/Linter.cpp @@ -12,6 +12,8 @@ #include #include +LUAU_FASTFLAGVARIABLE(LuauLintTableCreateTable, false) + namespace Luau { @@ -2153,6 +2155,19 @@ private: "table.move uses index 0 but arrays are 1-based; did you mean 1 instead?"); } + if (FFlag::LuauLintTableCreateTable && func->index == "create" && node->args.size == 2) + { + // table.create(n, {...}) + if (args[1]->is()) + emitWarning(*context, LintWarning::Code_TableOperations, args[1]->location, + "table.create with a table literal will reuse the same object for all elements; consider using a for loop instead"); + + // table.create(n, {...} :: ?) + if (AstExprTypeAssertion* as = args[1]->as(); as && as->expr->is()) + emitWarning(*context, LintWarning::Code_TableOperations, as->expr->location, + "table.create with a table literal will reuse the same object for all elements; consider using a for loop instead"); + } + return true; } diff --git a/Analysis/src/TypeInfer.cpp b/Analysis/src/TypeInfer.cpp index bedcc022..e2d8a4fb 100644 --- a/Analysis/src/TypeInfer.cpp +++ b/Analysis/src/TypeInfer.cpp @@ -33,6 +33,7 @@ LUAU_FASTFLAGVARIABLE(DebugLuauFreezeDuringUnification, false) LUAU_FASTFLAGVARIABLE(LuauRecursiveTypeParameterRestriction, false) LUAU_FASTFLAGVARIABLE(LuauIfElseBranchTypeUnion, false) LUAU_FASTFLAGVARIABLE(LuauIfElseExpectedType2, false) +LUAU_FASTFLAGVARIABLE(LuauLengthOnCompositeType, false) LUAU_FASTFLAGVARIABLE(LuauQuantifyInPlace2, false) LUAU_FASTFLAGVARIABLE(LuauSealExports, false) LUAU_FASTFLAGVARIABLE(LuauSingletonTypes, false) @@ -2066,17 +2067,27 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprUn if (get(operandType)) return {errorRecoveryType(scope)}; - if (get(operandType)) - return {numberType}; // Not strictly correct: metatables permit overriding this - - if (auto p = get(operandType)) + if (FFlag::LuauLengthOnCompositeType) { - if (p->type == PrimitiveTypeVar::String) - return {numberType}; - } + DenseHashSet seen{nullptr}; - if (!getTableType(operandType)) - reportError(TypeError{expr.location, NotATable{operandType}}); + if (!hasLength(operandType, seen, &recursionCount)) + reportError(TypeError{expr.location, NotATable{operandType}}); + } + else + { + if (get(operandType)) + return {numberType}; // Not strictly correct: metatables permit overriding this + + if (auto p = get(operandType)) + { + if (p->type == PrimitiveTypeVar::String) + return {numberType}; + } + + if (!getTableType(operandType)) + reportError(TypeError{expr.location, NotATable{operandType}}); + } return {numberType}; diff --git a/Analysis/src/TypeVar.cpp b/Analysis/src/TypeVar.cpp index ac2b2541..df5d76ed 100644 --- a/Analysis/src/TypeVar.cpp +++ b/Analysis/src/TypeVar.cpp @@ -5,6 +5,7 @@ #include "Luau/Common.h" #include "Luau/DenseHash.h" #include "Luau/Error.h" +#include "Luau/RecursionCounter.h" #include "Luau/StringUtils.h" #include "Luau/ToString.h" #include "Luau/TypeInfer.h" @@ -19,6 +20,8 @@ LUAU_FASTINTVARIABLE(LuauTypeMaximumStringifierLength, 500) LUAU_FASTINTVARIABLE(LuauTableTypeMaximumStringifierLength, 0) +LUAU_FASTINT(LuauTypeInferRecursionLimit) +LUAU_FASTFLAG(LuauLengthOnCompositeType) LUAU_FASTFLAGVARIABLE(LuauMetatableAreEqualRecursion, false) LUAU_FASTFLAGVARIABLE(LuauRefactorTagging, false) LUAU_FASTFLAG(LuauErrorRecoveryType) @@ -326,6 +329,49 @@ bool maybeSingleton(TypeId ty) return false; } +bool hasLength(TypeId ty, DenseHashSet& seen, int* recursionCount) +{ + LUAU_ASSERT(FFlag::LuauLengthOnCompositeType); + + RecursionLimiter _rl(recursionCount, FInt::LuauTypeInferRecursionLimit); + + ty = follow(ty); + + if (seen.contains(ty)) + return true; + + if (isPrim(ty, PrimitiveTypeVar::String) || get(ty) || get(ty) || get(ty)) + return true; + + if (auto uty = get(ty)) + { + seen.insert(ty); + + for (TypeId part : uty->options) + { + if (!hasLength(part, seen, recursionCount)) + return false; + } + + return true; + } + + if (auto ity = get(ty)) + { + seen.insert(ty); + + for (TypeId part : ity->parts) + { + if (hasLength(part, seen, recursionCount)) + return true; + } + + return false; + } + + return false; +} + FunctionTypeVar::FunctionTypeVar(TypePackId argTypes, TypePackId retType, std::optional defn, bool hasSelf) : argTypes(argTypes) , retType(retType) diff --git a/Analysis/src/Unifier.cpp b/Analysis/src/Unifier.cpp index 6873c657..2bd9cf83 100644 --- a/Analysis/src/Unifier.cpp +++ b/Analysis/src/Unifier.cpp @@ -13,6 +13,7 @@ LUAU_FASTINT(LuauTypeInferRecursionLimit); LUAU_FASTINT(LuauTypeInferTypePackLoopLimit); +LUAU_FASTFLAGVARIABLE(LuauCommittingTxnLogFreeTpPromote, false) LUAU_FASTFLAG(LuauUseCommittingTxnLog) LUAU_FASTINTVARIABLE(LuauTypeInferIterationLimit, 2000); LUAU_FASTFLAGVARIABLE(LuauTableSubtypingVariance2, false); @@ -99,6 +100,11 @@ struct PromoteTypeLevels bool operator()(TypePackId tp, const FreeTypePack&) { + // Surprise, it's actually a BoundTypePack that hasn't been committed yet. + // Calling getMutable on this will trigger an assertion. + if (FFlag::LuauCommittingTxnLogFreeTpPromote && FFlag::LuauUseCommittingTxnLog && !log.is(tp)) + return true; + promote(tp, FFlag::LuauUseCommittingTxnLog ? log.getMutable(tp) : getMutable(tp)); return true; } diff --git a/Ast/include/Luau/Ast.h b/Ast/include/Luau/Ast.h index 573850a5..ac5950c0 100644 --- a/Ast/include/Luau/Ast.h +++ b/Ast/include/Luau/Ast.h @@ -1265,7 +1265,8 @@ struct hash size_t operator()(const Luau::AstName& value) const { // note: since operator== uses pointer identity, hashing function uses it as well - return value.value ? std::hash()(value.value) : 0; + // the hasher is the same as DenseHashPointer (DenseHash.h) + return (uintptr_t(value.value) >> 4) ^ (uintptr_t(value.value) >> 9); } }; diff --git a/Ast/include/Luau/DenseHash.h b/Ast/include/Luau/DenseHash.h index a7b2515a..65939bee 100644 --- a/Ast/include/Luau/DenseHash.h +++ b/Ast/include/Luau/DenseHash.h @@ -12,10 +12,6 @@ namespace Luau { -// Internal implementation of DenseHashSet and DenseHashMap -namespace detail -{ - struct DenseHashPointer { size_t operator()(const void* key) const @@ -24,6 +20,10 @@ struct DenseHashPointer } }; +// Internal implementation of DenseHashSet and DenseHashMap +namespace detail +{ + template using DenseHashDefault = std::conditional_t, DenseHashPointer, std::hash>; diff --git a/Ast/src/Parser.cpp b/Ast/src/Parser.cpp index 77787cb1..3c607d24 100644 --- a/Ast/src/Parser.cpp +++ b/Ast/src/Parser.cpp @@ -14,6 +14,7 @@ LUAU_FASTFLAGVARIABLE(LuauFixAmbiguousErrorRecoveryInAssign, false) LUAU_FASTFLAGVARIABLE(LuauParseSingletonTypes, false) LUAU_FASTFLAGVARIABLE(LuauParseTypeAliasDefaults, false) LUAU_FASTFLAGVARIABLE(LuauParseRecoverTypePackEllipsis, false) +LUAU_FASTFLAGVARIABLE(LuauStartingBrokenComment, false) namespace Luau { @@ -174,10 +175,23 @@ ParseResult Parser::parse(const char* buffer, size_t bufferSize, AstNameTable& n const Lexeme::Type type = p.lexer.current().type; const Location loc = p.lexer.current().location; - p.lexer.next(); + if (FFlag::LuauStartingBrokenComment) + { + if (options.captureComments) + p.commentLocations.push_back(Comment{type, loc}); - if (options.captureComments) - p.commentLocations.push_back(Comment{type, loc}); + if (type == Lexeme::BrokenComment) + break; + + p.lexer.next(); + } + else + { + p.lexer.next(); + + if (options.captureComments) + p.commentLocations.push_back(Comment{type, loc}); + } } p.lexer.setSkipComments(true); diff --git a/CLI/Repl.cpp b/CLI/Repl.cpp index cf958939..ecf29fae 100644 --- a/CLI/Repl.cpp +++ b/CLI/Repl.cpp @@ -35,10 +35,15 @@ enum class CompileFormat Binary }; +struct GlobalOptions +{ + int optimizationLevel = 1; +} globalOptions; + static Luau::CompileOptions copts() { Luau::CompileOptions result = {}; - result.optimizationLevel = 1; + result.optimizationLevel = globalOptions.optimizationLevel; result.debugLevel = 1; result.coverageLevel = coverageActive() ? 2 : 0; @@ -232,13 +237,14 @@ static std::string runCode(lua_State* L, const std::string& source) static void completeIndexer(lua_State* L, const char* editBuffer, size_t start, std::vector& completions) { std::string_view lookup = editBuffer + start; + char lastSep = 0; for (;;) { - size_t dot = lookup.find('.'); - std::string_view prefix = lookup.substr(0, dot); + size_t sep = lookup.find_first_of(".:"); + std::string_view prefix = lookup.substr(0, sep); - if (dot == std::string_view::npos) + if (sep == std::string_view::npos) { // table, key lua_pushnil(L); @@ -249,11 +255,22 @@ static void completeIndexer(lua_State* L, const char* editBuffer, size_t start, { // table, key, value std::string_view key = lua_tostring(L, -2); + int valueType = lua_type(L, -1); - if (!key.empty() && Luau::startsWith(key, prefix)) - completions.push_back(editBuffer + std::string(key.substr(prefix.size()))); + // If the last separator was a ':' (i.e. a method call) then only functions should be completed. + bool requiredValueType = (lastSep != ':' || valueType == LUA_TFUNCTION); + + if (!key.empty() && requiredValueType && Luau::startsWith(key, prefix)) + { + std::string completion(editBuffer + std::string(key.substr(prefix.size()))); + if (valueType == LUA_TFUNCTION) + { + // Add an opening paren for function calls by default. + completion += "("; + } + completions.push_back(completion); + } } - lua_pop(L, 1); } @@ -266,10 +283,21 @@ static void completeIndexer(lua_State* L, const char* editBuffer, size_t start, lua_rawget(L, -2); lua_remove(L, -2); - if (!lua_istable(L, -1)) + if (lua_type(L, -1) == LUA_TSTRING) + { + // Replace the string object with the string class to perform further lookups of string functions + // Note: We retrieve the string class from _G to prevent issues if the user assigns to `string`. + lua_getglobal(L, "_G"); + lua_pushlstring(L, "string", 6); + lua_rawget(L, -2); + lua_remove(L, -2); + LUAU_ASSERT(lua_istable(L, -1)); + } + else if (!lua_istable(L, -1)) break; - lookup.remove_prefix(dot + 1); + lastSep = lookup[sep]; + lookup.remove_prefix(sep + 1); } } @@ -279,7 +307,7 @@ static void completeIndexer(lua_State* L, const char* editBuffer, size_t start, static void completeRepl(lua_State* L, const char* editBuffer, std::vector& completions) { size_t start = strlen(editBuffer); - while (start > 0 && (isalnum(editBuffer[start - 1]) || editBuffer[start - 1] == '.' || editBuffer[start - 1] == '_')) + while (start > 0 && (isalnum(editBuffer[start - 1]) || editBuffer[start - 1] == '.' || editBuffer[start - 1] == ':' || editBuffer[start - 1] == '_')) start--; // look the value up in current global table first @@ -319,15 +347,8 @@ struct LinenoiseScopedHistory std::string historyFilepath; }; -static void runRepl() +static void runReplImpl(lua_State* L) { - std::unique_ptr globalState(luaL_newstate(), lua_close); - lua_State* L = globalState.get(); - - setupState(L); - - luaL_sandboxthread(L); - linenoise::SetCompletionCallback([L](const char* editBuffer, std::vector& completions) { completeRepl(L, editBuffer, completions); }); @@ -368,7 +389,18 @@ static void runRepl() } } -static bool runFile(const char* name, lua_State* GL) +static void runRepl() +{ + std::unique_ptr globalState(luaL_newstate(), lua_close); + lua_State* L = globalState.get(); + + setupState(L); + luaL_sandboxthread(L); + runReplImpl(L); +} + +// `repl` is used it indicate if a repl should be started after executing the file. +static bool runFile(const char* name, lua_State* GL, bool repl) { std::optional source = readFile(name); if (!source) @@ -419,6 +451,10 @@ static bool runFile(const char* name, lua_State* GL) fprintf(stderr, "%s", error.c_str()); } + if (repl) + { + runReplImpl(L); + } lua_pop(GL, 1); return status == 0; } @@ -457,7 +493,7 @@ static bool compileFile(const char* name, CompileFormat format) bcb.setDumpSource(*source); } - Luau::compileOrThrow(bcb, *source); + Luau::compileOrThrow(bcb, *source, copts()); switch (format) { @@ -495,9 +531,11 @@ static void displayHelp(const char* argv0) printf(" --compile[=format]: compile input files and output resulting formatted bytecode (binary or text)\n"); printf("\n"); printf("Available options:\n"); - printf(" -h, --help: Display this usage message.\n"); - printf(" --profile[=N]: profile the code using N Hz sampling (default 10000) and output results to profile.out\n"); printf(" --coverage: collect code coverage while running the code and output results to coverage.out\n"); + printf(" -h, --help: Display this usage message.\n"); + printf(" -i, --interactive: Run an interactive REPL after executing the last script specified.\n"); + printf(" -O: use compiler optimization level (n=0-2).\n"); + printf(" --profile[=N]: profile the code using N Hz sampling (default 10000) and output results to profile.out\n"); printf(" --timetrace: record compiler time tracing information into trace.json\n"); } @@ -519,6 +557,7 @@ int main(int argc, char** argv) CompileFormat compileFormat{}; int profile = 0; bool coverage = false; + bool interactive = false; // Set the mode if the user has explicitly specified one. int argStart = 1; @@ -540,8 +579,8 @@ int main(int argc, char** argv) } else { - fprintf(stdout, "Error: Unrecognized value for '--compile' specified.\n"); - return -1; + fprintf(stderr, "Error: Unrecognized value for '--compile' specified.\n"); + return 1; } } @@ -552,6 +591,20 @@ int main(int argc, char** argv) displayHelp(argv[0]); return 0; } + else if (strcmp(argv[i], "-i") == 0 || strcmp(argv[i], "--interactive") == 0) + { + interactive = true; + } + else if (strncmp(argv[i], "-O", 2) == 0) + { + int level = atoi(argv[i] + 2); + if (level < 0 || level > 2) + { + fprintf(stderr, "Error: Optimization level must be between 0 and 2 inclusive.\n"); + return 1; + } + globalOptions.optimizationLevel = level; + } else if (strcmp(argv[i], "--profile") == 0) { profile = 10000; // default to 10 KHz @@ -575,7 +628,7 @@ int main(int argc, char** argv) } else if (argv[i][0] == '-') { - fprintf(stdout, "Error: Unrecognized option '%s'.\n\n", argv[i]); + fprintf(stderr, "Error: Unrecognized option '%s'.\n\n", argv[i]); displayHelp(argv[0]); return 1; } @@ -623,8 +676,11 @@ int main(int argc, char** argv) int failed = 0; - for (const std::string& path : files) - failed += !runFile(path.c_str(), L); + for (size_t i = 0; i < files.size(); ++i) + { + bool isLastFile = i == files.size() - 1; + failed += !runFile(files[i].c_str(), L, interactive && isLastFile); + } if (profile) { diff --git a/CMakeLists.txt b/CMakeLists.txt index bafc59e5..77cf47e8 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -78,6 +78,12 @@ target_compile_options(Luau.Ast PRIVATE ${LUAU_OPTIONS}) target_compile_options(Luau.Analysis PRIVATE ${LUAU_OPTIONS}) target_compile_options(Luau.VM PRIVATE ${LUAU_OPTIONS}) +if (MSVC AND MSVC_VERSION GREATER_EQUAL 1924) + # disable partial redundancy elimination which regresses interpreter codegen substantially in VS2022: + # https://developercommunity.visualstudio.com/t/performance-regression-on-a-complex-interpreter-lo/1631863 + set_source_files_properties(VM/src/lvmexecute.cpp PROPERTIES COMPILE_FLAGS /d2ssa-pre-) +endif() + if(LUAU_BUILD_CLI) target_compile_options(Luau.Repl.CLI PRIVATE ${LUAU_OPTIONS}) target_compile_options(Luau.Analyze.CLI PRIVATE ${LUAU_OPTIONS}) diff --git a/Compiler/src/TableShape.cpp b/Compiler/src/TableShape.cpp index 7d99f222..9dc2f0a4 100644 --- a/Compiler/src/TableShape.cpp +++ b/Compiler/src/TableShape.cpp @@ -1,11 +1,16 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #include "TableShape.h" +LUAU_FASTFLAGVARIABLE(LuauPredictTableSizeLoop, false) + namespace Luau { namespace Compile { +// conservative limit for the loop bound that establishes table array size +static const int kMaxLoopBound = 16; + static AstExprTable* getTableHint(AstExpr* expr) { // unadorned table literal @@ -27,7 +32,7 @@ struct ShapeVisitor : AstVisitor { size_t operator()(const std::pair& p) const { - return std::hash()(p.first) ^ std::hash()(p.second); + return DenseHashPointer()(p.first) ^ std::hash()(p.second); } }; @@ -36,10 +41,13 @@ struct ShapeVisitor : AstVisitor DenseHashMap tables; DenseHashSet, Hasher> fields; + DenseHashMap loops; // iterator => upper bound for 1..k + ShapeVisitor(DenseHashMap& shapes) : shapes(shapes) , tables(nullptr) , fields(std::pair()) + , loops(nullptr) { } @@ -63,16 +71,31 @@ struct ShapeVisitor : AstVisitor void assignField(AstExpr* expr, AstExpr* index) { AstExprLocal* lv = expr->as(); - AstExprConstantNumber* number = index->as(); + if (!lv) + return; - if (lv && number) + AstExprTable** table = tables.find(lv->local); + if (!table) + return; + + if (AstExprConstantNumber* number = index->as()) { - if (AstExprTable** table = tables.find(lv->local)) + TableShape& shape = shapes[*table]; + + if (number->value == double(shape.arraySize + 1)) + shape.arraySize += 1; + } + else if (AstExprLocal* iter = index->as()) + { + if (!FFlag::LuauPredictTableSizeLoop) + return; + + if (const unsigned int* bound = loops.find(iter->local)) { TableShape& shape = shapes[*table]; - if (number->value == double(shape.arraySize + 1)) - shape.arraySize += 1; + if (shape.arraySize == 0) + shape.arraySize = *bound; } } } @@ -117,6 +140,20 @@ struct ShapeVisitor : AstVisitor return false; } + + bool visit(AstStatFor* node) override + { + if (!FFlag::LuauPredictTableSizeLoop) + return true; + + AstExprConstantNumber* from = node->from->as(); + AstExprConstantNumber* to = node->to->as(); + + if (from && to && from->value == 1.0 && to->value >= 1.0 && to->value <= double(kMaxLoopBound) && !node->step) + loops[node->var] = unsigned(to->value); + + return true; + } }; void predictTableShapes(DenseHashMap& shapes, AstNode* root) diff --git a/VM/src/ldo.cpp b/VM/src/ldo.cpp index 3cce7665..581506a8 100644 --- a/VM/src/ldo.cpp +++ b/VM/src/ldo.cpp @@ -150,12 +150,11 @@ l_noret luaD_throw(lua_State* L, int errcode) static void correctstack(lua_State* L, TValue* oldstack) { - CallInfo* ci; - GCObject* up; L->top = (L->top - oldstack) + L->stack; - for (up = L->openupval; up != NULL; up = up->gch.next) - gco2uv(up)->v = (gco2uv(up)->v - oldstack) + L->stack; - for (ci = L->base_ci; ci <= L->ci; ci++) + // TODO (FFlagLuauGcPagedSweep): 'next' type will change after removal of the flag and the cast will not be required + for (UpVal* up = L->openupval; up != NULL; up = (UpVal*)up->next) + up->v = (up->v - oldstack) + L->stack; + for (CallInfo* ci = L->base_ci; ci <= L->ci; ci++) { ci->top = (ci->top - oldstack) + L->stack; ci->base = (ci->base - oldstack) + L->stack; diff --git a/VM/src/lfunc.cpp b/VM/src/lfunc.cpp index 4178eda4..6088f71c 100644 --- a/VM/src/lfunc.cpp +++ b/VM/src/lfunc.cpp @@ -7,10 +7,11 @@ #include "lgc.h" LUAU_FASTFLAGVARIABLE(LuauNoDirectUpvalRemoval, false) +LUAU_FASTFLAG(LuauGcPagedSweep) Proto* luaF_newproto(lua_State* L) { - Proto* f = luaM_new(L, Proto, sizeof(Proto), L->activememcat); + Proto* f = luaM_newgco(L, Proto, sizeof(Proto), L->activememcat); luaC_link(L, f, LUA_TPROTO); f->k = NULL; f->sizek = 0; @@ -38,7 +39,7 @@ Proto* luaF_newproto(lua_State* L) Closure* luaF_newLclosure(lua_State* L, int nelems, Table* e, Proto* p) { - Closure* c = luaM_new(L, Closure, sizeLclosure(nelems), L->activememcat); + Closure* c = luaM_newgco(L, Closure, sizeLclosure(nelems), L->activememcat); luaC_link(L, c, LUA_TFUNCTION); c->isC = 0; c->env = e; @@ -53,7 +54,7 @@ Closure* luaF_newLclosure(lua_State* L, int nelems, Table* e, Proto* p) Closure* luaF_newCclosure(lua_State* L, int nelems, Table* e) { - Closure* c = luaM_new(L, Closure, sizeCclosure(nelems), L->activememcat); + Closure* c = luaM_newgco(L, Closure, sizeCclosure(nelems), L->activememcat); luaC_link(L, c, LUA_TFUNCTION); c->isC = 1; c->env = e; @@ -69,10 +70,9 @@ Closure* luaF_newCclosure(lua_State* L, int nelems, Table* e) UpVal* luaF_findupval(lua_State* L, StkId level) { global_State* g = L->global; - GCObject** pp = &L->openupval; + UpVal** pp = &L->openupval; UpVal* p; - UpVal* uv; - while (*pp != NULL && (p = gco2uv(*pp))->v >= level) + while (*pp != NULL && (p = *pp)->v >= level) { LUAU_ASSERT(p->v != &p->u.value); if (p->v == level) @@ -81,53 +81,95 @@ UpVal* luaF_findupval(lua_State* L, StkId level) changewhite(obj2gco(p)); /* resurrect it */ return p; } - pp = &p->next; + + // TODO (FFlagLuauGcPagedSweep): 'next' type will change after removal of the flag and the cast will not be required + pp = (UpVal**)&p->next; } - uv = luaM_new(L, UpVal, sizeof(UpVal), L->activememcat); /* not found: create a new one */ + + UpVal* uv = luaM_newgco(L, UpVal, sizeof(UpVal), L->activememcat); /* not found: create a new one */ uv->tt = LUA_TUPVAL; uv->marked = luaC_white(g); uv->memcat = L->activememcat; uv->v = level; /* current value lives in the stack */ - uv->next = *pp; /* chain it in the proper position */ - *pp = obj2gco(uv); - uv->u.l.prev = &g->uvhead; /* double link it in `uvhead' list */ + + // chain the upvalue in the threads open upvalue list at the proper position + UpVal* next = *pp; + + // TODO (FFlagLuauGcPagedSweep): 'next' type will change after removal of the flag and the cast will not be required + uv->next = (GCObject*)next; + + if (FFlag::LuauGcPagedSweep) + { + uv->u.l.threadprev = pp; + if (next) + { + // TODO (FFlagLuauGcPagedSweep): 'next' type will change after removal of the flag and the cast will not be required + next->u.l.threadprev = (UpVal**)&uv->next; + } + } + + *pp = uv; + + // double link the upvalue in the global open upvalue list + uv->u.l.prev = &g->uvhead; uv->u.l.next = g->uvhead.u.l.next; uv->u.l.next->u.l.prev = uv; g->uvhead.u.l.next = uv; LUAU_ASSERT(uv->u.l.next->u.l.prev == uv && uv->u.l.prev->u.l.next == uv); return uv; } - -static void unlinkupval(UpVal* uv) +void luaF_unlinkupval(UpVal* uv) { + // unlink upvalue from the global open upvalue list LUAU_ASSERT(uv->u.l.next->u.l.prev == uv && uv->u.l.prev->u.l.next == uv); - uv->u.l.next->u.l.prev = uv->u.l.prev; /* remove from `uvhead' list */ + uv->u.l.next->u.l.prev = uv->u.l.prev; uv->u.l.prev->u.l.next = uv->u.l.next; + + if (FFlag::LuauGcPagedSweep) + { + // unlink upvalue from the thread open upvalue list + // TODO (FFlagLuauGcPagedSweep): 'next' type will change after removal of the flag and this and the following cast will not be required + *uv->u.l.threadprev = (UpVal*)uv->next; + + if (UpVal* next = (UpVal*)uv->next) + next->u.l.threadprev = uv->u.l.threadprev; + } } -void luaF_freeupval(lua_State* L, UpVal* uv) +void luaF_freeupval(lua_State* L, UpVal* uv, lua_Page* page) { if (uv->v != &uv->u.value) /* is it open? */ - unlinkupval(uv); /* remove from open list */ - luaM_free(L, uv, sizeof(UpVal), uv->memcat); /* free upvalue */ + luaF_unlinkupval(uv); /* remove from open list */ + luaM_freegco(L, uv, sizeof(UpVal), uv->memcat, page); /* free upvalue */ } void luaF_close(lua_State* L, StkId level) { - UpVal* uv; global_State* g = L->global; // TODO: remove with FFlagLuauNoDirectUpvalRemoval - while (L->openupval != NULL && (uv = gco2uv(L->openupval))->v >= level) + UpVal* uv; + while (L->openupval != NULL && (uv = L->openupval)->v >= level) { GCObject* o = obj2gco(uv); LUAU_ASSERT(!isblack(o) && uv->v != &uv->u.value); - L->openupval = uv->next; /* remove from `open' list */ - if (!FFlag::LuauNoDirectUpvalRemoval && isdead(g, o)) + + if (!FFlag::LuauGcPagedSweep) + L->openupval = (UpVal*)uv->next; /* remove from `open' list */ + + if (FFlag::LuauGcPagedSweep && isdead(g, o)) { - luaF_freeupval(L, uv); /* free upvalue */ + // by removing the upvalue from global/thread open upvalue lists, L->openupval will be pointing to the next upvalue + luaF_unlinkupval(uv); + // close the upvalue without copying the dead data so that luaF_freeupval will not unlink again + uv->v = &uv->u.value; + } + else if (!FFlag::LuauNoDirectUpvalRemoval && isdead(g, o)) + { + luaF_freeupval(L, uv, NULL); /* free upvalue */ } else { - unlinkupval(uv); + // by removing the upvalue from global/thread open upvalue lists, L->openupval will be pointing to the next upvalue + luaF_unlinkupval(uv); setobj(L, &uv->u.value, uv->v); uv->v = &uv->u.value; /* now current value lives here */ luaC_linkupval(L, uv); /* link upvalue into `gcroot' list */ @@ -135,7 +177,7 @@ void luaF_close(lua_State* L, StkId level) } } -void luaF_freeproto(lua_State* L, Proto* f) +void luaF_freeproto(lua_State* L, Proto* f, lua_Page* page) { luaM_freearray(L, f->code, f->sizecode, Instruction, f->memcat); luaM_freearray(L, f->p, f->sizep, Proto*, f->memcat); @@ -146,13 +188,13 @@ void luaF_freeproto(lua_State* L, Proto* f) luaM_freearray(L, f->upvalues, f->sizeupvalues, TString*, f->memcat); if (f->debuginsn) luaM_freearray(L, f->debuginsn, f->sizecode, uint8_t, f->memcat); - luaM_free(L, f, sizeof(Proto), f->memcat); + luaM_freegco(L, f, sizeof(Proto), f->memcat, page); } -void luaF_freeclosure(lua_State* L, Closure* c) +void luaF_freeclosure(lua_State* L, Closure* c, lua_Page* page) { int size = c->isC ? sizeCclosure(c->nupvalues) : sizeLclosure(c->nupvalues); - luaM_free(L, c, size, c->memcat); + luaM_freegco(L, c, size, c->memcat, page); } const LocVar* luaF_getlocal(const Proto* f, int local_number, int pc) diff --git a/VM/src/lfunc.h b/VM/src/lfunc.h index 4be23667..8047cebe 100644 --- a/VM/src/lfunc.h +++ b/VM/src/lfunc.h @@ -12,7 +12,8 @@ LUAI_FUNC Closure* luaF_newLclosure(lua_State* L, int nelems, Table* e, Proto* p LUAI_FUNC Closure* luaF_newCclosure(lua_State* L, int nelems, Table* e); LUAI_FUNC UpVal* luaF_findupval(lua_State* L, StkId level); LUAI_FUNC void luaF_close(lua_State* L, StkId level); -LUAI_FUNC void luaF_freeproto(lua_State* L, Proto* f); -LUAI_FUNC void luaF_freeclosure(lua_State* L, Closure* c); -LUAI_FUNC void luaF_freeupval(lua_State* L, UpVal* uv); +LUAI_FUNC void luaF_freeproto(lua_State* L, Proto* f, struct lua_Page* page); +LUAI_FUNC void luaF_freeclosure(lua_State* L, Closure* c, struct lua_Page* page); +void luaF_unlinkupval(UpVal* uv); +LUAI_FUNC void luaF_freeupval(lua_State* L, UpVal* uv, struct lua_Page* page); LUAI_FUNC const LocVar* luaF_getlocal(const Proto* func, int local_number, int pc); diff --git a/VM/src/lgc.cpp b/VM/src/lgc.cpp index 76ef7a06..9a8cb079 100644 --- a/VM/src/lgc.cpp +++ b/VM/src/lgc.cpp @@ -8,12 +8,16 @@ #include "lfunc.h" #include "lstring.h" #include "ldo.h" +#include "lmem.h" #include "ludata.h" #include +LUAU_FASTFLAGVARIABLE(LuauGcPagedSweep, false) + #define GC_SWEEPMAX 40 #define GC_SWEEPCOST 10 +#define GC_SWEEPPAGESTEPCOST 4 #define GC_INTERRUPT(state) \ { \ @@ -457,31 +461,31 @@ static void shrinkstack(lua_State* L) condhardstacktests(luaD_reallocstack(L, s_used)); } -static void freeobj(lua_State* L, GCObject* o) +static void freeobj(lua_State* L, GCObject* o, lua_Page* page) { switch (o->gch.tt) { case LUA_TPROTO: - luaF_freeproto(L, gco2p(o)); + luaF_freeproto(L, gco2p(o), page); break; case LUA_TFUNCTION: - luaF_freeclosure(L, gco2cl(o)); + luaF_freeclosure(L, gco2cl(o), page); break; case LUA_TUPVAL: - luaF_freeupval(L, gco2uv(o)); + luaF_freeupval(L, gco2uv(o), page); break; case LUA_TTABLE: - luaH_free(L, gco2h(o)); + luaH_free(L, gco2h(o), page); break; case LUA_TTHREAD: LUAU_ASSERT(gco2th(o) != L && gco2th(o) != L->global->mainthread); - luaE_freethread(L, gco2th(o)); + luaE_freethread(L, gco2th(o), page); break; case LUA_TSTRING: - luaS_free(L, gco2ts(o)); + luaS_free(L, gco2ts(o), page); break; case LUA_TUSERDATA: - luaU_freeudata(L, gco2u(o)); + luaU_freeudata(L, gco2u(o), page); break; default: LUAU_ASSERT(0); @@ -492,6 +496,8 @@ static void freeobj(lua_State* L, GCObject* o) static GCObject** sweeplist(lua_State* L, GCObject** p, size_t count, size_t* traversedcount) { + LUAU_ASSERT(!FFlag::LuauGcPagedSweep); + GCObject* curr; global_State* g = L->global; int deadmask = otherwhite(g); @@ -502,7 +508,7 @@ static GCObject** sweeplist(lua_State* L, GCObject** p, size_t count, size_t* tr int alive = (curr->gch.marked ^ WHITEBITS) & deadmask; if (curr->gch.tt == LUA_TTHREAD) { - sweepwholelist(L, &gco2th(curr)->openupval, traversedcount); /* sweep open upvalues */ + sweepwholelist(L, (GCObject**)&gco2th(curr)->openupval, traversedcount); /* sweep open upvalues */ lua_State* th = gco2th(curr); @@ -524,7 +530,7 @@ static GCObject** sweeplist(lua_State* L, GCObject** p, size_t count, size_t* tr *p = curr->gch.next; if (curr == g->rootgc) /* is the first element of the list? */ g->rootgc = curr->gch.next; /* adjust first */ - freeobj(L, curr); + freeobj(L, curr, NULL); } } @@ -537,14 +543,16 @@ static GCObject** sweeplist(lua_State* L, GCObject** p, size_t count, size_t* tr static void deletelist(lua_State* L, GCObject** p, GCObject* limit) { + LUAU_ASSERT(!FFlag::LuauGcPagedSweep); + GCObject* curr; while ((curr = *p) != limit) { if (curr->gch.tt == LUA_TTHREAD) /* delete open upvalues of each thread */ - deletelist(L, &gco2th(curr)->openupval, NULL); + deletelist(L, (GCObject**)&gco2th(curr)->openupval, NULL); *p = curr->gch.next; - freeobj(L, curr); + freeobj(L, curr, NULL); } } @@ -567,23 +575,62 @@ static void shrinkbuffersfull(lua_State* L) luaS_resize(L, hashsize); /* table is too big */ } +static bool deletegco(void* context, lua_Page* page, GCObject* gco) +{ + LUAU_ASSERT(FFlag::LuauGcPagedSweep); + + // we are in the process of deleting everything + // threads with open upvalues will attempt to close them all on removal + // but those upvalues might point to stack values that were already deleted + if (gco->gch.tt == LUA_TTHREAD) + { + lua_State* th = gco2th(gco); + + while (UpVal* uv = th->openupval) + { + luaF_unlinkupval(uv); + // close the upvalue without copying the dead data so that luaF_freeupval will not unlink again + uv->v = &uv->u.value; + } + } + + lua_State* L = (lua_State*)context; + freeobj(L, gco, page); + return true; +} + void luaC_freeall(lua_State* L) { global_State* g = L->global; LUAU_ASSERT(L == g->mainthread); - LUAU_ASSERT(L->next == NULL); /* mainthread is at the end of rootgc list */ - deletelist(L, &g->rootgc, obj2gco(L)); + if (FFlag::LuauGcPagedSweep) + { + luaM_visitgco(L, L, deletegco); - for (int i = 0; i < g->strt.size; i++) /* free all string lists */ - deletelist(L, &g->strt.hash[i], NULL); + for (int i = 0; i < g->strt.size; i++) /* free all string lists */ + LUAU_ASSERT(g->strt.hash[i] == NULL); - LUAU_ASSERT(L->global->strt.nuse == 0); - deletelist(L, &g->strbufgc, NULL); - // unfortunately, when string objects are freed, the string table use count is decremented - // even when the string is a buffer that wasn't placed into the table - L->global->strt.nuse = 0; + LUAU_ASSERT(L->global->strt.nuse == 0); + LUAU_ASSERT(g->strbufgc == NULL); + } + else + { + LUAU_ASSERT(L->next == NULL); /* mainthread is at the end of rootgc list */ + + deletelist(L, &g->rootgc, obj2gco(L)); + + for (int i = 0; i < g->strt.size; i++) /* free all string lists */ + deletelist(L, (GCObject**)&g->strt.hash[i], NULL); + + LUAU_ASSERT(L->global->strt.nuse == 0); + deletelist(L, (GCObject**)&g->strbufgc, NULL); + + // unfortunately, when string objects are freed, the string table use count is decremented + // even when the string is a buffer that wasn't placed into the table + L->global->strt.nuse = 0; + } } static void markmt(global_State* g) @@ -648,12 +695,88 @@ static size_t atomic(lua_State* L) /* flip current white */ g->currentwhite = cast_byte(otherwhite(g)); g->sweepstrgc = 0; - g->sweepgc = &g->rootgc; - g->gcstate = GCSsweepstring; + + if (FFlag::LuauGcPagedSweep) + { + g->sweepgcopage = g->allgcopages; + g->gcstate = GCSsweep; + } + else + { + g->sweepgc = &g->rootgc; + g->gcstate = GCSsweepstring; + } return work; } +static bool sweepgco(lua_State* L, lua_Page* page, GCObject* gco) +{ + LUAU_ASSERT(FFlag::LuauGcPagedSweep); + + global_State* g = L->global; + + int deadmask = otherwhite(g); + LUAU_ASSERT(testbit(deadmask, FIXEDBIT)); // make sure we never sweep fixed objects + + int alive = (gco->gch.marked ^ WHITEBITS) & deadmask; + + g->gcstats.currcycle.sweepitems++; + + if (gco->gch.tt == LUA_TTHREAD) + { + lua_State* th = gco2th(gco); + + if (alive) + { + resetbit(th->stackstate, THREAD_SLEEPINGBIT); + shrinkstack(th); + } + } + + if (alive) + { + LUAU_ASSERT(!isdead(g, gco)); + makewhite(g, gco); // make it white (for next cycle) + return false; + } + + LUAU_ASSERT(isdead(g, gco)); + freeobj(L, gco, page); + return true; +} + +// a version of generic luaM_visitpage specialized for the main sweep stage +static int sweepgcopage(lua_State* L, lua_Page* page) +{ + LUAU_ASSERT(FFlag::LuauGcPagedSweep); + + char* start; + char* end; + int busyBlocks; + int blockSize; + luaM_getpagewalkinfo(page, &start, &end, &busyBlocks, &blockSize); + + for (char* pos = start; pos != end; pos += blockSize) + { + GCObject* gco = (GCObject*)pos; + + // skip memory blocks that are already freed + if (gco->gch.tt == LUA_TNIL) + continue; + + // when true is returned it means that the element was deleted + if (sweepgco(L, page, gco)) + { + // if the last block was removed, page would be removed as well + if (--busyBlocks == 0) + return int(pos - start) / blockSize + 1; + } + } + + return int(end - start) / blockSize; +} + static size_t gcstep(lua_State* L, size_t limit) { size_t cost = 0; @@ -706,15 +829,21 @@ static size_t gcstep(lua_State* L, size_t limit) g->gcstats.currcycle.atomicstarttotalsizebytes = g->totalbytes; cost = atomic(L); /* finish mark phase */ - LUAU_ASSERT(g->gcstate == GCSsweepstring); + + if (FFlag::LuauGcPagedSweep) + LUAU_ASSERT(g->gcstate == GCSsweep); + else + LUAU_ASSERT(g->gcstate == GCSsweepstring); break; } case GCSsweepstring: { + LUAU_ASSERT(!FFlag::LuauGcPagedSweep); + while (g->sweepstrgc < g->strt.size && cost < limit) { size_t traversedcount = 0; - sweepwholelist(L, &g->strt.hash[g->sweepstrgc++], &traversedcount); + sweepwholelist(L, (GCObject**)&g->strt.hash[g->sweepstrgc++], &traversedcount); g->gcstats.currcycle.sweepitems += traversedcount; cost += GC_SWEEPCOST; @@ -727,7 +856,7 @@ static size_t gcstep(lua_State* L, size_t limit) uint32_t nuse = L->global->strt.nuse; size_t traversedcount = 0; - sweepwholelist(L, &g->strbufgc, &traversedcount); + sweepwholelist(L, (GCObject**)&g->strbufgc, &traversedcount); L->global->strt.nuse = nuse; @@ -738,19 +867,44 @@ static size_t gcstep(lua_State* L, size_t limit) } case GCSsweep: { - while (*g->sweepgc && cost < limit) + if (FFlag::LuauGcPagedSweep) { - size_t traversedcount = 0; - g->sweepgc = sweeplist(L, g->sweepgc, GC_SWEEPMAX, &traversedcount); + while (g->sweepgcopage && cost < limit) + { + lua_Page* next = luaM_getnextgcopage(g->sweepgcopage); // page sweep might destroy the page - g->gcstats.currcycle.sweepitems += traversedcount; - cost += GC_SWEEPMAX * GC_SWEEPCOST; + int steps = sweepgcopage(L, g->sweepgcopage); + + g->sweepgcopage = next; + cost += steps * GC_SWEEPPAGESTEPCOST; + } + + // nothing more to sweep? + if (g->sweepgcopage == NULL) + { + // don't forget to visit main thread + sweepgco(L, NULL, obj2gco(g->mainthread)); + + shrinkbuffers(L); + g->gcstate = GCSpause; /* end collection */ + } } + else + { + while (*g->sweepgc && cost < limit) + { + size_t traversedcount = 0; + g->sweepgc = sweeplist(L, g->sweepgc, GC_SWEEPMAX, &traversedcount); - if (*g->sweepgc == NULL) - { /* nothing more to sweep? */ - shrinkbuffers(L); - g->gcstate = GCSpause; /* end collection */ + g->gcstats.currcycle.sweepitems += traversedcount; + cost += GC_SWEEPMAX * GC_SWEEPCOST; + } + + if (*g->sweepgc == NULL) + { /* nothing more to sweep? */ + shrinkbuffers(L); + g->gcstate = GCSpause; /* end collection */ + } } break; } @@ -877,12 +1031,19 @@ void luaC_fullgc(lua_State* L) { /* reset sweep marks to sweep all elements (returning them to white) */ g->sweepstrgc = 0; - g->sweepgc = &g->rootgc; + if (FFlag::LuauGcPagedSweep) + g->sweepgcopage = g->allgcopages; + else + g->sweepgc = &g->rootgc; /* reset other collector lists */ g->gray = NULL; g->grayagain = NULL; g->weak = NULL; - g->gcstate = GCSsweepstring; + + if (FFlag::LuauGcPagedSweep) + g->gcstate = GCSsweep; + else + g->gcstate = GCSsweepstring; } LUAU_ASSERT(g->gcstate == GCSsweepstring || g->gcstate == GCSsweep); /* finish any pending sweep phase */ @@ -979,8 +1140,11 @@ void luaC_barrierback(lua_State* L, Table* t) void luaC_linkobj(lua_State* L, GCObject* o, uint8_t tt) { global_State* g = L->global; - o->gch.next = g->rootgc; - g->rootgc = o; + if (!FFlag::LuauGcPagedSweep) + { + o->gch.next = g->rootgc; + g->rootgc = o; + } o->gch.marked = luaC_white(g); o->gch.tt = tt; o->gch.memcat = L->activememcat; @@ -990,8 +1154,13 @@ void luaC_linkupval(lua_State* L, UpVal* uv) { global_State* g = L->global; GCObject* o = obj2gco(uv); - o->gch.next = g->rootgc; /* link upvalue into `rootgc' list */ - g->rootgc = o; + + if (!FFlag::LuauGcPagedSweep) + { + o->gch.next = g->rootgc; /* link upvalue into `rootgc' list */ + g->rootgc = o; + } + if (isgray(o)) { if (keepinvariant(g)) diff --git a/VM/src/lgc.h b/VM/src/lgc.h index f434e506..4455fec5 100644 --- a/VM/src/lgc.h +++ b/VM/src/lgc.h @@ -13,6 +13,7 @@ #define GCSpropagate 1 #define GCSpropagateagain 2 #define GCSatomic 3 +// TODO: remove with FFlagLuauGcPagedSweep #define GCSsweepstring 4 #define GCSsweep 5 diff --git a/VM/src/lgcdebug.cpp b/VM/src/lgcdebug.cpp index c66de9c1..906fb0d0 100644 --- a/VM/src/lgcdebug.cpp +++ b/VM/src/lgcdebug.cpp @@ -2,16 +2,19 @@ // This code is based on Lua 5.x implementation licensed under MIT License; see lua_LICENSE.txt for details #include "lgc.h" +#include "lfunc.h" +#include "lmem.h" #include "lobject.h" #include "lstate.h" -#include "ltable.h" -#include "lfunc.h" #include "lstring.h" +#include "ltable.h" #include "ludata.h" #include #include +LUAU_FASTFLAG(LuauGcPagedSweep) + static void validateobjref(global_State* g, GCObject* f, GCObject* t) { LUAU_ASSERT(!isdead(g, t)); @@ -101,10 +104,11 @@ static void validatestack(global_State* g, lua_State* l) if (l->namecall) validateobjref(g, obj2gco(l), obj2gco(l->namecall)); - for (GCObject* uv = l->openupval; uv; uv = uv->gch.next) + // TODO (FFlagLuauGcPagedSweep): 'next' type will change after removal of the flag and the cast will not be required + for (UpVal* uv = l->openupval; uv; uv = (UpVal*)uv->next) { - LUAU_ASSERT(uv->gch.tt == LUA_TUPVAL); - LUAU_ASSERT(gco2uv(uv)->v != &gco2uv(uv)->u.value); + LUAU_ASSERT(uv->tt == LUA_TUPVAL); + LUAU_ASSERT(uv->v != &uv->u.value); } } @@ -178,6 +182,8 @@ static void validateobj(global_State* g, GCObject* o) static void validatelist(global_State* g, GCObject* o) { + LUAU_ASSERT(!FFlag::LuauGcPagedSweep); + while (o) { validateobj(g, o); @@ -216,6 +222,17 @@ static void validategraylist(global_State* g, GCObject* o) } } +static bool validategco(void* context, lua_Page* page, GCObject* gco) +{ + LUAU_ASSERT(FFlag::LuauGcPagedSweep); + + lua_State* L = (lua_State*)context; + global_State* g = L->global; + + validateobj(g, gco); + return false; +} + void luaC_validate(lua_State* L) { global_State* g = L->global; @@ -231,11 +248,18 @@ void luaC_validate(lua_State* L) validategraylist(g, g->gray); validategraylist(g, g->grayagain); - for (int i = 0; i < g->strt.size; ++i) - validatelist(g, g->strt.hash[i]); + if (FFlag::LuauGcPagedSweep) + { + luaM_visitgco(L, L, validategco); + } + else + { + for (int i = 0; i < g->strt.size; ++i) + validatelist(g, (GCObject*)(g->strt.hash[i])); - validatelist(g, g->rootgc); - validatelist(g, g->strbufgc); + validatelist(g, g->rootgc); + validatelist(g, (GCObject*)(g->strbufgc)); + } for (UpVal* uv = g->uvhead.u.l.next; uv != &g->uvhead; uv = uv->u.l.next) { @@ -499,6 +523,8 @@ static void dumpobj(FILE* f, GCObject* o) static void dumplist(FILE* f, GCObject* o) { + LUAU_ASSERT(!FFlag::LuauGcPagedSweep); + while (o) { dumpref(f, o); @@ -509,22 +535,45 @@ static void dumplist(FILE* f, GCObject* o) // thread has additional list containing collectable objects that are not present in rootgc if (o->gch.tt == LUA_TTHREAD) - dumplist(f, gco2th(o)->openupval); + dumplist(f, (GCObject*)gco2th(o)->openupval); o = o->gch.next; } } +static bool dumpgco(void* context, lua_Page* page, GCObject* gco) +{ + LUAU_ASSERT(FFlag::LuauGcPagedSweep); + + FILE* f = (FILE*)context; + + dumpref(f, gco); + fputc(':', f); + dumpobj(f, gco); + fputc(',', f); + fputc('\n', f); + + return false; +} + void luaC_dump(lua_State* L, void* file, const char* (*categoryName)(lua_State* L, uint8_t memcat)) { global_State* g = L->global; FILE* f = static_cast(file); fprintf(f, "{\"objects\":{\n"); - dumplist(f, g->rootgc); - dumplist(f, g->strbufgc); - for (int i = 0; i < g->strt.size; ++i) - dumplist(f, g->strt.hash[i]); + + if (FFlag::LuauGcPagedSweep) + { + luaM_visitgco(L, f, dumpgco); + } + else + { + dumplist(f, g->rootgc); + dumplist(f, (GCObject*)(g->strbufgc)); + for (int i = 0; i < g->strt.size; ++i) + dumplist(f, (GCObject*)(g->strt.hash[i])); + } fprintf(f, "\"0\":{\"type\":\"userdata\",\"cat\":0,\"size\":0}\n"); // to avoid issues with trailing , fprintf(f, "},\"roots\":{\n"); diff --git a/VM/src/lmem.cpp b/VM/src/lmem.cpp index 9f9d4a98..e1dbce50 100644 --- a/VM/src/lmem.cpp +++ b/VM/src/lmem.cpp @@ -8,6 +8,8 @@ #include +LUAU_FASTFLAG(LuauGcPagedSweep) + #ifndef __has_feature #define __has_feature(x) 0 #endif @@ -42,13 +44,21 @@ static_assert(sizeof(LuaNode) == ABISWITCH(32, 32, 32), "size mismatch for table #endif static_assert(offsetof(TString, data) == ABISWITCH(24, 20, 20), "size mismatch for string header"); +// TODO (FFlagLuauGcPagedSweep): this will become ABISWITCH(16, 16, 16) static_assert(offsetof(Udata, data) == ABISWITCH(24, 16, 16), "size mismatch for userdata header"); +// TODO (FFlagLuauGcPagedSweep): this will become ABISWITCH(48, 32, 32) static_assert(sizeof(Table) == ABISWITCH(56, 36, 36), "size mismatch for table header"); +// TODO (FFlagLuauGcPagedSweep): new code with old 'next' pointer requires that GCObject start at the same point as TString/UpVal +static_assert(offsetof(GCObject, uv) == 0, "UpVal data must be located at the start of the GCObject"); +static_assert(offsetof(GCObject, ts) == 0, "TString data must be located at the start of the GCObject"); + const size_t kSizeClasses = LUA_SIZECLASSES; const size_t kMaxSmallSize = 512; const size_t kPageSize = 16 * 1024 - 24; // slightly under 16KB since that results in less fragmentation due to heap metadata const size_t kBlockHeader = sizeof(double) > sizeof(void*) ? sizeof(double) : sizeof(void*); // suitable for aligning double & void* on all platforms +// TODO (FFlagLuauGcPagedSweep): when 'next' is removed, 'kBlockHeader' can be used unconditionally +const size_t kGCOHeader = sizeof(GCheader) > kBlockHeader ? sizeof(GCheader) : kBlockHeader; struct SizeClassConfig { @@ -96,6 +106,7 @@ const SizeClassConfig kSizeClassConfig; // metadata for a block is stored in the first pointer of the block #define metadata(block) (*(void**)(block)) +#define freegcolink(block) (*(void**)((char*)block + kGCOHeader)) /* ** About the realloc function: @@ -117,15 +128,22 @@ const SizeClassConfig kSizeClassConfig; struct lua_Page { + // list of pages with free blocks lua_Page* prev; lua_Page* next; + // list of all gco pages + lua_Page* gcolistprev; + lua_Page* gcolistnext; + int busyBlocks; int blockSize; void* freeList; int freeNext; + int pageSize; + union { char data[1]; @@ -141,6 +159,8 @@ l_noret luaM_toobig(lua_State* L) static lua_Page* luaM_newpage(lua_State* L, uint8_t sizeClass) { + LUAU_ASSERT(!FFlag::LuauGcPagedSweep); + global_State* g = L->global; lua_Page* page = (lua_Page*)(*g->frealloc)(L, g->ud, NULL, 0, kPageSize); if (!page) @@ -155,6 +175,9 @@ static lua_Page* luaM_newpage(lua_State* L, uint8_t sizeClass) page->prev = NULL; page->next = NULL; + page->gcolistprev = NULL; + page->gcolistnext = NULL; + page->busyBlocks = 0; page->blockSize = blockSize; @@ -171,8 +194,69 @@ static lua_Page* luaM_newpage(lua_State* L, uint8_t sizeClass) return page; } +static lua_Page* newpage(lua_State* L, lua_Page** gcopageset, int pageSize, int blockSize, int blockCount) +{ + LUAU_ASSERT(FFlag::LuauGcPagedSweep); + + global_State* g = L->global; + + LUAU_ASSERT(pageSize - int(offsetof(lua_Page, data)) >= blockSize * blockCount); + + lua_Page* page = (lua_Page*)(*g->frealloc)(L, g->ud, NULL, 0, pageSize); + if (!page) + luaD_throw(L, LUA_ERRMEM); + + ASAN_POISON_MEMORY_REGION(page->data, blockSize * blockCount); + + // setup page header + page->prev = NULL; + page->next = NULL; + + page->gcolistprev = NULL; + page->gcolistnext = NULL; + + page->busyBlocks = 0; + page->blockSize = blockSize; + + // note: we start with the last block in the page and move downward + // either order would work, but that way we don't need to store the block count in the page + // additionally, GC stores objects in singly linked lists, and this way the GC lists end up in increasing pointer order + page->freeList = NULL; + page->freeNext = (blockCount - 1) * blockSize; + + page->pageSize = pageSize; + + if (gcopageset) + { + page->gcolistnext = *gcopageset; + if (page->gcolistnext) + page->gcolistnext->gcolistprev = page; + *gcopageset = page; + } + + return page; +} + +static lua_Page* newclasspage(lua_State* L, lua_Page** freepageset, lua_Page** gcopageset, uint8_t sizeClass, bool storeMetadata) +{ + LUAU_ASSERT(FFlag::LuauGcPagedSweep); + + int blockSize = kSizeClassConfig.sizeOfClass[sizeClass] + (storeMetadata ? kBlockHeader : 0); + int blockCount = (kPageSize - offsetof(lua_Page, data)) / blockSize; + + lua_Page* page = newpage(L, gcopageset, kPageSize, blockSize, blockCount); + + // prepend a page to page freelist (which is empty because we only ever allocate a new page when it is!) + LUAU_ASSERT(!freepageset[sizeClass]); + freepageset[sizeClass] = page; + + return page; +} + static void luaM_freepage(lua_State* L, lua_Page* page, uint8_t sizeClass) { + LUAU_ASSERT(!FFlag::LuauGcPagedSweep); + global_State* g = L->global; // remove page from freelist @@ -188,6 +272,44 @@ static void luaM_freepage(lua_State* L, lua_Page* page, uint8_t sizeClass) (*g->frealloc)(L, g->ud, page, kPageSize, 0); } +static void freepage(lua_State* L, lua_Page** gcopageset, lua_Page* page) +{ + LUAU_ASSERT(FFlag::LuauGcPagedSweep); + + global_State* g = L->global; + + if (gcopageset) + { + // remove page from alllist + if (page->gcolistnext) + page->gcolistnext->gcolistprev = page->gcolistprev; + + if (page->gcolistprev) + page->gcolistprev->gcolistnext = page->gcolistnext; + else if (*gcopageset == page) + *gcopageset = page->gcolistnext; + } + + // so long + (*g->frealloc)(L, g->ud, page, page->pageSize, 0); +} + +static void freeclasspage(lua_State* L, lua_Page** freepageset, lua_Page** gcopageset, lua_Page* page, uint8_t sizeClass) +{ + LUAU_ASSERT(FFlag::LuauGcPagedSweep); + + // remove page from freelist + if (page->next) + page->next->prev = page->prev; + + if (page->prev) + page->prev->next = page->next; + else if (freepageset[sizeClass] == page) + freepageset[sizeClass] = page->next; + + freepage(L, gcopageset, page); +} + static void* luaM_newblock(lua_State* L, int sizeClass) { global_State* g = L->global; @@ -195,7 +317,12 @@ static void* luaM_newblock(lua_State* L, int sizeClass) // slow path: no page in the freelist, allocate a new one if (!page) - page = luaM_newpage(L, sizeClass); + { + if (FFlag::LuauGcPagedSweep) + page = newclasspage(L, g->freepages, NULL, sizeClass, true); + else + page = luaM_newpage(L, sizeClass); + } LUAU_ASSERT(!page->prev); LUAU_ASSERT(page->freeList || page->freeNext >= 0); @@ -236,6 +363,55 @@ static void* luaM_newblock(lua_State* L, int sizeClass) return (char*)block + kBlockHeader; } +static void* luaM_newgcoblock(lua_State* L, int sizeClass) +{ + LUAU_ASSERT(FFlag::LuauGcPagedSweep); + + global_State* g = L->global; + lua_Page* page = g->freegcopages[sizeClass]; + + // slow path: no page in the freelist, allocate a new one + if (!page) + page = newclasspage(L, g->freegcopages, &g->allgcopages, sizeClass, false); + + LUAU_ASSERT(!page->prev); + LUAU_ASSERT(page->freeList || page->freeNext >= 0); + LUAU_ASSERT(page->blockSize == kSizeClassConfig.sizeOfClass[sizeClass]); + + void* block; + + if (page->freeNext >= 0) + { + block = &page->data + page->freeNext; + ASAN_UNPOISON_MEMORY_REGION(block, page->blockSize); + + page->freeNext -= page->blockSize; + page->busyBlocks++; + } + else + { + // when separate block metadata is not used, free list link is stored inside the block data itself + block = (char*)page->freeList - kGCOHeader; + + ASAN_UNPOISON_MEMORY_REGION((char*)block + kGCOHeader, page->blockSize - kGCOHeader); + + page->freeList = freegcolink(block); + page->busyBlocks++; + } + + // if we allocate the last block out of a page, we need to remove it from free list + if (!page->freeList && page->freeNext < 0) + { + g->freegcopages[sizeClass] = page->next; + if (page->next) + page->next->prev = NULL; + page->next = NULL; + } + + // the user data is right after the metadata + return (char*)block; +} + static void luaM_freeblock(lua_State* L, int sizeClass, void* block) { global_State* g = L->global; @@ -270,12 +446,45 @@ static void luaM_freeblock(lua_State* L, int sizeClass, void* block) // if it's the last block in the page, we don't need the page if (page->busyBlocks == 0) - luaM_freepage(L, page, sizeClass); + { + if (FFlag::LuauGcPagedSweep) + freeclasspage(L, g->freepages, NULL, page, sizeClass); + else + luaM_freepage(L, page, sizeClass); + } +} + +static void luaM_freegcoblock(lua_State* L, int sizeClass, void* block, lua_Page* page) +{ + LUAU_ASSERT(FFlag::LuauGcPagedSweep); + + global_State* g = L->global; + + // if the page wasn't in the page free list, it should be now since it got a block! + if (!page->freeList && page->freeNext < 0) + { + LUAU_ASSERT(!page->prev); + LUAU_ASSERT(!page->next); + + page->next = g->freegcopages[sizeClass]; + if (page->next) + page->next->prev = page; + g->freegcopages[sizeClass] = page; + } + + // when separate block metadata is not used, free list link is stored inside the block data itself + freegcolink(block) = page->freeList; + page->freeList = (char*)block + kGCOHeader; + + ASAN_POISON_MEMORY_REGION((char*)block + kGCOHeader, page->blockSize - kGCOHeader); + + page->busyBlocks--; + + // if it's the last block in the page, we don't need the page + if (page->busyBlocks == 0) + freeclasspage(L, g->freegcopages, &g->allgcopages, page, sizeClass); } -/* -** generic allocation routines. -*/ void* luaM_new_(lua_State* L, size_t nsize, uint8_t memcat) { global_State* g = L->global; @@ -292,6 +501,43 @@ void* luaM_new_(lua_State* L, size_t nsize, uint8_t memcat) return block; } +GCObject* luaM_newgco_(lua_State* L, size_t nsize, uint8_t memcat) +{ + if (!FFlag::LuauGcPagedSweep) + return (GCObject*)luaM_new_(L, nsize, memcat); + + global_State* g = L->global; + + int nclass = sizeclass(nsize); + + void* block = NULL; + + if (nclass >= 0) + { + LUAU_ASSERT(nsize > 8); + + block = luaM_newgcoblock(L, nclass); + } + else + { + lua_Page* page = newpage(L, &g->allgcopages, offsetof(lua_Page, data) + int(nsize), int(nsize), 1); + + block = &page->data; + ASAN_UNPOISON_MEMORY_REGION(block, page->blockSize); + + page->freeNext -= page->blockSize; + page->busyBlocks++; + } + + if (block == NULL && nsize > 0) + luaD_throw(L, LUA_ERRMEM); + + g->totalbytes += nsize; + g->memcatbytes[memcat] += nsize; + + return (GCObject*)block; +} + void luaM_free_(lua_State* L, void* block, size_t osize, uint8_t memcat) { global_State* g = L->global; @@ -308,6 +554,36 @@ void luaM_free_(lua_State* L, void* block, size_t osize, uint8_t memcat) g->memcatbytes[memcat] -= osize; } +void luaM_freegco_(lua_State* L, GCObject* block, size_t osize, uint8_t memcat, lua_Page* page) +{ + if (!FFlag::LuauGcPagedSweep) + { + luaM_free_(L, block, osize, memcat); + return; + } + + global_State* g = L->global; + LUAU_ASSERT((osize == 0) == (block == NULL)); + + int oclass = sizeclass(osize); + + if (oclass >= 0) + { + block->gch.tt = LUA_TNIL; + + luaM_freegcoblock(L, oclass, block, page); + } + else + { + LUAU_ASSERT(page->busyBlocks == 1); + + freepage(L, &g->allgcopages, page); + } + + g->totalbytes -= osize; + g->memcatbytes[memcat] -= osize; +} + void* luaM_realloc_(lua_State* L, void* block, size_t osize, size_t nsize, uint8_t memcat) { global_State* g = L->global; @@ -344,3 +620,64 @@ void* luaM_realloc_(lua_State* L, void* block, size_t osize, size_t nsize, uint8 g->memcatbytes[memcat] += nsize - osize; return result; } + +void luaM_getpagewalkinfo(lua_Page* page, char** start, char** end, int* busyBlocks, int* blockSize) +{ + LUAU_ASSERT(FFlag::LuauGcPagedSweep); + + int blockCount = (page->pageSize - offsetof(lua_Page, data)) / page->blockSize; + + *start = page->data + page->freeNext + page->blockSize; + *end = page->data + blockCount * page->blockSize; + *busyBlocks = page->busyBlocks; + *blockSize = page->blockSize; +} + +lua_Page* luaM_getnextgcopage(lua_Page* page) +{ + return page->gcolistnext; +} + +void luaM_visitpage(lua_Page* page, void* context, bool (*visitor)(void* context, lua_Page* page, GCObject* gco)) +{ + LUAU_ASSERT(FFlag::LuauGcPagedSweep); + + char* start; + char* end; + int busyBlocks; + int blockSize; + luaM_getpagewalkinfo(page, &start, &end, &busyBlocks, &blockSize); + + for (char* pos = start; pos != end; pos += blockSize) + { + GCObject* gco = (GCObject*)pos; + + // skip memory blocks that are already freed + if (gco->gch.tt == LUA_TNIL) + continue; + + // when true is returned it means that the element was deleted + if (visitor(context, page, gco)) + { + // if the last block was removed, page would be removed as well + if (--busyBlocks == 0) + break; + } + } +} + +void luaM_visitgco(lua_State* L, void* context, bool (*visitor)(void* context, lua_Page* page, GCObject* gco)) +{ + LUAU_ASSERT(FFlag::LuauGcPagedSweep); + + global_State* g = L->global; + + for (lua_Page* curr = g->allgcopages; curr;) + { + lua_Page* next = curr->gcolistnext; // page blockvisit might destroy the page + + luaM_visitpage(curr, context, visitor); + + curr = next; + } +} diff --git a/VM/src/lmem.h b/VM/src/lmem.h index f526a1b6..1bfe48fa 100644 --- a/VM/src/lmem.h +++ b/VM/src/lmem.h @@ -4,8 +4,15 @@ #include "lua.h" +struct lua_Page; +union GCObject; + +// TODO: remove with FFlagLuauGcPagedSweep and rename luaM_newgco to luaM_new #define luaM_new(L, t, size, memcat) cast_to(t*, luaM_new_(L, size, memcat)) +#define luaM_newgco(L, t, size, memcat) cast_to(t*, luaM_newgco_(L, size, memcat)) +// TODO: remove with FFlagLuauGcPagedSweep and rename luaM_freegco to luaM_free #define luaM_free(L, p, size, memcat) luaM_free_(L, (p), size, memcat) +#define luaM_freegco(L, p, size, memcat, page) luaM_freegco_(L, obj2gco(p), size, memcat, page) #define luaM_arraysize_(n, e) ((cast_to(size_t, (n)) <= SIZE_MAX / (e)) ? (n) * (e) : (luaM_toobig(L), SIZE_MAX)) @@ -15,7 +22,15 @@ ((v) = cast_to(t*, luaM_realloc_(L, v, (oldn) * sizeof(t), luaM_arraysize_(n, sizeof(t)), memcat))) LUAI_FUNC void* luaM_new_(lua_State* L, size_t nsize, uint8_t memcat); +LUAI_FUNC GCObject* luaM_newgco_(lua_State* L, size_t nsize, uint8_t memcat); LUAI_FUNC void luaM_free_(lua_State* L, void* block, size_t osize, uint8_t memcat); +LUAI_FUNC void luaM_freegco_(lua_State* L, GCObject* block, size_t osize, uint8_t memcat, lua_Page* page); LUAI_FUNC void* luaM_realloc_(lua_State* L, void* block, size_t osize, size_t nsize, uint8_t memcat); LUAI_FUNC l_noret luaM_toobig(lua_State* L); + +LUAI_FUNC void luaM_getpagewalkinfo(lua_Page* page, char** start, char** end, int* busyBlocks, int* blockSize); +LUAI_FUNC lua_Page* luaM_getnextgcopage(lua_Page* page); + +LUAI_FUNC void luaM_visitpage(lua_Page* page, void* context, bool (*visitor)(void* context, lua_Page* page, GCObject* gco)); +LUAI_FUNC void luaM_visitgco(lua_State* L, void* context, bool (*visitor)(void* context, lua_Page* page, GCObject* gco)); diff --git a/VM/src/lobject.h b/VM/src/lobject.h index b642cf78..57ffd82a 100644 --- a/VM/src/lobject.h +++ b/VM/src/lobject.h @@ -11,12 +11,11 @@ typedef union GCObject GCObject; /* -** Common Header for all collectible objects (in macro form, to be -** included in other objects) +** Common Header for all collectible objects (in macro form, to be included in other objects) */ // clang-format off #define CommonHeader \ - GCObject* next; \ + GCObject* next; /* TODO: remove with FFlagLuauGcPagedSweep */ \ uint8_t tt; uint8_t marked; uint8_t memcat // clang-format on @@ -229,8 +228,10 @@ typedef TValue* StkId; /* index to stack elements */ typedef struct TString { CommonHeader; + // 1 byte padding int16_t atom; + // 2 byte padding unsigned int hash; unsigned int len; @@ -314,14 +315,21 @@ typedef struct LocVar typedef struct UpVal { CommonHeader; + // 1 (x86) or 5 (x64) byte padding TValue* v; /* points to stack or to its own value */ union { TValue value; /* the value (when closed) */ struct - { /* double linked list (when open) */ + { + /* global double linked list (when open) */ struct UpVal* prev; struct UpVal* next; + + /* thread double linked list (when open) */ + // TODO: when FFlagLuauGcPagedSweep is removed, old outer 'next' value will be placed here + /* note: this is the location of a pointer to this upvalue in the previous element that can be either an UpVal or a lua_State */ + struct UpVal** threadprev; } l; } u; } UpVal; diff --git a/VM/src/lstate.cpp b/VM/src/lstate.cpp index 24e97063..6762c638 100644 --- a/VM/src/lstate.cpp +++ b/VM/src/lstate.cpp @@ -10,6 +10,8 @@ #include "ldo.h" #include "ldebug.h" +LUAU_FASTFLAG(LuauGcPagedSweep) + /* ** Main thread combines a thread state and the global state */ @@ -86,14 +88,21 @@ static void close_state(lua_State* L) global_State* g = L->global; luaF_close(L, L->stack); /* close all upvalues for this thread */ luaC_freeall(L); /* collect all objects */ - LUAU_ASSERT(g->rootgc == obj2gco(L)); + if (!FFlag::LuauGcPagedSweep) + LUAU_ASSERT(g->rootgc == obj2gco(L)); LUAU_ASSERT(g->strbufgc == NULL); LUAU_ASSERT(g->strt.nuse == 0); luaM_freearray(L, L->global->strt.hash, L->global->strt.size, TString*, 0); freestack(L, L); - LUAU_ASSERT(g->totalbytes == sizeof(LG)); for (int i = 0; i < LUA_SIZECLASSES; i++) + { LUAU_ASSERT(g->freepages[i] == NULL); + if (FFlag::LuauGcPagedSweep) + LUAU_ASSERT(g->freegcopages[i] == NULL); + } + if (FFlag::LuauGcPagedSweep) + LUAU_ASSERT(g->allgcopages == NULL); + LUAU_ASSERT(g->totalbytes == sizeof(LG)); LUAU_ASSERT(g->memcatbytes[0] == sizeof(LG)); for (int i = 1; i < LUA_MEMORY_CATEGORIES; i++) LUAU_ASSERT(g->memcatbytes[i] == 0); @@ -102,7 +111,7 @@ static void close_state(lua_State* L) lua_State* luaE_newthread(lua_State* L) { - lua_State* L1 = luaM_new(L, lua_State, sizeof(lua_State), L->activememcat); + lua_State* L1 = luaM_newgco(L, lua_State, sizeof(lua_State), L->activememcat); luaC_link(L, L1, LUA_TTHREAD); preinit_state(L1, L->global); L1->activememcat = L->activememcat; // inherit the active memory category @@ -113,7 +122,7 @@ lua_State* luaE_newthread(lua_State* L) return L1; } -void luaE_freethread(lua_State* L, lua_State* L1) +void luaE_freethread(lua_State* L, lua_State* L1, lua_Page* page) { luaF_close(L1, L1->stack); /* close all upvalues for this thread */ LUAU_ASSERT(L1->openupval == NULL); @@ -121,7 +130,7 @@ void luaE_freethread(lua_State* L, lua_State* L1) if (g->cb.userthread) g->cb.userthread(NULL, L1); freestack(L, L1); - luaM_free(L, L1, sizeof(lua_State), L1->memcat); + luaM_freegco(L, L1, sizeof(lua_State), L1->memcat, page); } void lua_resetthread(lua_State* L) @@ -162,7 +171,8 @@ lua_State* lua_newstate(lua_Alloc f, void* ud) return NULL; L = (lua_State*)l; g = &((LG*)L)->g; - L->next = NULL; + if (!FFlag::LuauGcPagedSweep) + L->next = NULL; L->tt = LUA_TTHREAD; L->marked = g->currentwhite = bit2mask(WHITE0BIT, FIXEDBIT); L->memcat = 0; @@ -185,9 +195,11 @@ lua_State* lua_newstate(lua_Alloc f, void* ud) g->strt.hash = NULL; setnilvalue(registry(L)); g->gcstate = GCSpause; - g->rootgc = obj2gco(L); + if (!FFlag::LuauGcPagedSweep) + g->rootgc = obj2gco(L); g->sweepstrgc = 0; - g->sweepgc = &g->rootgc; + if (!FFlag::LuauGcPagedSweep) + g->sweepgc = &g->rootgc; g->gray = NULL; g->grayagain = NULL; g->weak = NULL; @@ -197,7 +209,16 @@ lua_State* lua_newstate(lua_Alloc f, void* ud) g->gcstepmul = LUAI_GCSTEPMUL; g->gcstepsize = LUAI_GCSTEPSIZE << 10; for (i = 0; i < LUA_SIZECLASSES; i++) + { g->freepages[i] = NULL; + if (FFlag::LuauGcPagedSweep) + g->freegcopages[i] = NULL; + } + if (FFlag::LuauGcPagedSweep) + { + g->allgcopages = NULL; + g->sweepgcopage = NULL; + } for (i = 0; i < LUA_T_COUNT; i++) g->mt[i] = NULL; for (i = 0; i < LUA_UTAG_LIMIT; i++) diff --git a/VM/src/lstate.h b/VM/src/lstate.h index 56379883..080f0024 100644 --- a/VM/src/lstate.h +++ b/VM/src/lstate.h @@ -22,7 +22,7 @@ typedef struct stringtable { - GCObject** hash; + TString** hash; uint32_t nuse; /* number of elements */ int size; } stringtable; @@ -149,13 +149,15 @@ typedef struct global_State int sweepstrgc; /* position of sweep in `strt' */ + // TODO: remove with FFlagLuauGcPagedSweep GCObject* rootgc; /* list of all collectable objects */ + // TODO: remove with FFlagLuauGcPagedSweep GCObject** sweepgc; /* position of sweep in `rootgc' */ GCObject* gray; /* list of gray objects */ GCObject* grayagain; /* list of objects to be traversed atomically */ GCObject* weak; /* list of weak tables (to be cleared) */ - GCObject* strbufgc; // list of all string buffer objects + TString* strbufgc; // list of all string buffer objects size_t GCthreshold; // when totalbytes > GCthreshold; run GC step @@ -164,7 +166,10 @@ typedef struct global_State int gcstepmul; // see LUAI_GCSTEPMUL int gcstepsize; // see LUAI_GCSTEPSIZE - struct lua_Page* freepages[LUA_SIZECLASSES]; /* free page linked list for each size class */ + struct lua_Page* freepages[LUA_SIZECLASSES]; // free page linked list for each size class for non-collectable objects + struct lua_Page* freegcopages[LUA_SIZECLASSES]; // free page linked list for each size class for collectable objects + struct lua_Page* allgcopages; // page linked list with all pages for all classes + struct lua_Page* sweepgcopage; // position of the sweep in `allgcopages' size_t memcatbytes[LUA_MEMORY_CATEGORIES]; /* total amount of memory used by each memory category */ @@ -231,7 +236,7 @@ struct lua_State TValue l_gt; /* table of globals */ TValue env; /* temporary place for environments */ - GCObject* openupval; /* list of open upvalues in this stack */ + UpVal* openupval; /* list of open upvalues in this stack */ GCObject* gclist; TString* namecall; /* when invoked from Luau using NAMECALL, what method do we need to invoke? */ @@ -268,4 +273,4 @@ union GCObject #define obj2gco(v) check_exp(iscollectable(v), cast_to(GCObject*, (v) + 0)) LUAI_FUNC lua_State* luaE_newthread(lua_State* L); -LUAI_FUNC void luaE_freethread(lua_State* L, lua_State* L1); +LUAI_FUNC void luaE_freethread(lua_State* L, lua_State* L1, struct lua_Page* page); diff --git a/VM/src/lstring.cpp b/VM/src/lstring.cpp index a9e90d17..cb22cc23 100644 --- a/VM/src/lstring.cpp +++ b/VM/src/lstring.cpp @@ -7,6 +7,8 @@ #include +LUAU_FASTFLAG(LuauGcPagedSweep) + unsigned int luaS_hash(const char* str, size_t len) { // Note that this hashing algorithm is replicated in BytecodeBuilder.cpp, BytecodeBuilder::getStringHash @@ -44,26 +46,25 @@ unsigned int luaS_hash(const char* str, size_t len) void luaS_resize(lua_State* L, int newsize) { - GCObject** newhash; - stringtable* tb; - int i; if (L->global->gcstate == GCSsweepstring) return; /* cannot resize during GC traverse */ - newhash = luaM_newarray(L, newsize, GCObject*, 0); - tb = &L->global->strt; - for (i = 0; i < newsize; i++) + TString** newhash = luaM_newarray(L, newsize, TString*, 0); + stringtable* tb = &L->global->strt; + for (int i = 0; i < newsize; i++) newhash[i] = NULL; /* rehash */ - for (i = 0; i < tb->size; i++) + for (int i = 0; i < tb->size; i++) { - GCObject* p = tb->hash[i]; + TString* p = tb->hash[i]; while (p) { /* for each node in the list */ - GCObject* next = p->gch.next; /* save next */ - unsigned int h = gco2ts(p)->hash; + // TODO (FFlagLuauGcPagedSweep): 'next' type will change after removal of the flag and the cast will not be required + TString* next = (TString*)p->next; /* save next */ + unsigned int h = p->hash; int h1 = lmod(h, newsize); /* new position */ LUAU_ASSERT(cast_int(h % newsize) == lmod(h, newsize)); - p->gch.next = newhash[h1]; /* chain it */ + // TODO (FFlagLuauGcPagedSweep): 'next' type will change after removal of the flag and the cast will not be required + p->next = (GCObject*)newhash[h1]; /* chain it */ newhash[h1] = p; p = next; } @@ -79,7 +80,7 @@ static TString* newlstr(lua_State* L, const char* str, size_t l, unsigned int h) stringtable* tb; if (l > MAXSSIZE) luaM_toobig(L); - ts = luaM_new(L, TString, sizestring(l), L->activememcat); + ts = luaM_newgco(L, TString, sizestring(l), L->activememcat); ts->len = unsigned(l); ts->hash = h; ts->marked = luaC_white(L->global); @@ -90,8 +91,9 @@ static TString* newlstr(lua_State* L, const char* str, size_t l, unsigned int h) ts->atom = L->global->cb.useratom ? L->global->cb.useratom(ts->data, l) : -1; tb = &L->global->strt; h = lmod(h, tb->size); - ts->next = tb->hash[h]; /* chain new entry */ - tb->hash[h] = obj2gco(ts); + // TODO (FFlagLuauGcPagedSweep): 'next' type will change after removal of the flag and the case will not be required + ts->next = (GCObject*)tb->hash[h]; /* chain new entry */ + tb->hash[h] = ts; tb->nuse++; if (tb->nuse > cast_to(uint32_t, tb->size) && tb->size <= INT_MAX / 2) luaS_resize(L, tb->size * 2); /* too crowded */ @@ -101,28 +103,41 @@ static TString* newlstr(lua_State* L, const char* str, size_t l, unsigned int h) static void linkstrbuf(lua_State* L, TString* ts) { global_State* g = L->global; - GCObject* o = obj2gco(ts); - o->gch.next = g->strbufgc; - g->strbufgc = o; - o->gch.marked = luaC_white(g); + + if (FFlag::LuauGcPagedSweep) + { + // TODO (FFlagLuauGcPagedSweep): 'next' type will change after removal of the flag and the cast will not be required + ts->next = (GCObject*)g->strbufgc; + g->strbufgc = ts; + ts->marked = luaC_white(g); + } + else + { + GCObject* o = obj2gco(ts); + o->gch.next = (GCObject*)g->strbufgc; + g->strbufgc = gco2ts(o); + o->gch.marked = luaC_white(g); + } } static void unlinkstrbuf(lua_State* L, TString* ts) { global_State* g = L->global; - GCObject** p = &g->strbufgc; + TString** p = &g->strbufgc; - while (GCObject* curr = *p) + while (TString* curr = *p) { - if (curr == obj2gco(ts)) + if (curr == ts) { - *p = curr->gch.next; + // TODO (FFlagLuauGcPagedSweep): 'next' type will change after removal of the flag and the cast will not be required + *p = (TString*)curr->next; return; } else { - p = &curr->gch.next; + // TODO (FFlagLuauGcPagedSweep): 'next' type will change after removal of the flag and the cast will not be required + p = (TString**)&curr->next; } } @@ -134,7 +149,7 @@ TString* luaS_bufstart(lua_State* L, size_t size) if (size > MAXSSIZE) luaM_toobig(L); - TString* ts = luaM_new(L, TString, sizestring(size), L->activememcat); + TString* ts = luaM_newgco(L, TString, sizestring(size), L->activememcat); ts->tt = LUA_TSTRING; ts->memcat = L->activememcat; @@ -152,15 +167,14 @@ TString* luaS_buffinish(lua_State* L, TString* ts) int bucket = lmod(h, tb->size); // search if we already have this string in the hash table - for (GCObject* o = tb->hash[bucket]; o != NULL; o = o->gch.next) + // TODO (FFlagLuauGcPagedSweep): 'next' type will change after removal of the flag and the cast will not be required + for (TString* el = tb->hash[bucket]; el != NULL; el = (TString*)el->next) { - TString* el = gco2ts(o); - if (el->len == ts->len && memcmp(el->data, ts->data, ts->len) == 0) { // string may be dead - if (isdead(L->global, o)) - changewhite(o); + if (isdead(L->global, obj2gco(el))) + changewhite(obj2gco(el)); return el; } @@ -173,8 +187,9 @@ TString* luaS_buffinish(lua_State* L, TString* ts) // Complete string object ts->atom = L->global->cb.useratom ? L->global->cb.useratom(ts->data, ts->len) : -1; - ts->next = tb->hash[bucket]; // chain new entry - tb->hash[bucket] = obj2gco(ts); + // TODO (FFlagLuauGcPagedSweep): 'next' type will change after removal of the flag and the cast will not be required + ts->next = (GCObject*)tb->hash[bucket]; // chain new entry + tb->hash[bucket] = ts; tb->nuse++; if (tb->nuse > cast_to(uint32_t, tb->size) && tb->size <= INT_MAX / 2) @@ -185,24 +200,63 @@ TString* luaS_buffinish(lua_State* L, TString* ts) TString* luaS_newlstr(lua_State* L, const char* str, size_t l) { - GCObject* o; unsigned int h = luaS_hash(str, l); - for (o = L->global->strt.hash[lmod(h, L->global->strt.size)]; o != NULL; o = o->gch.next) + // TODO (FFlagLuauGcPagedSweep): 'next' type will change after removal of the flag and the cast will not be required + for (TString* el = L->global->strt.hash[lmod(h, L->global->strt.size)]; el != NULL; el = (TString*)el->next) { - TString* ts = gco2ts(o); - if (ts->len == l && (memcmp(str, getstr(ts), l) == 0)) + if (el->len == l && (memcmp(str, getstr(el), l) == 0)) { /* string may be dead */ - if (isdead(L->global, o)) - changewhite(o); - return ts; + if (isdead(L->global, obj2gco(el))) + changewhite(obj2gco(el)); + return el; } } return newlstr(L, str, l, h); /* not found */ } -void luaS_free(lua_State* L, TString* ts) +static bool unlinkstr(lua_State* L, TString* ts) { - L->global->strt.nuse--; - luaM_free(L, ts, sizestring(ts->len), ts->memcat); + LUAU_ASSERT(FFlag::LuauGcPagedSweep); + + global_State* g = L->global; + + TString** p = &g->strt.hash[lmod(ts->hash, g->strt.size)]; + + while (TString* curr = *p) + { + if (curr == ts) + { + // TODO (FFlagLuauGcPagedSweep): 'next' type will change after removal of the flag and the cast will not be required + *p = (TString*)curr->next; + return true; + } + else + { + // TODO (FFlagLuauGcPagedSweep): 'next' type will change after removal of the flag and the cast will not be required + p = (TString**)&curr->next; + } + } + + return false; +} + +void luaS_free(lua_State* L, TString* ts, lua_Page* page) +{ + if (FFlag::LuauGcPagedSweep) + { + // Unchain from the string table + if (!unlinkstr(L, ts)) + unlinkstrbuf(L, ts); // An unlikely scenario when we have a string buffer on our hands + else + L->global->strt.nuse--; + + luaM_freegco(L, ts, sizestring(ts->len), ts->memcat, page); + } + else + { + L->global->strt.nuse--; + + luaM_free(L, ts, sizestring(ts->len), ts->memcat); + } } diff --git a/VM/src/lstring.h b/VM/src/lstring.h index 3fd0bd39..290b64d8 100644 --- a/VM/src/lstring.h +++ b/VM/src/lstring.h @@ -20,7 +20,7 @@ LUAI_FUNC unsigned int luaS_hash(const char* str, size_t len); LUAI_FUNC void luaS_resize(lua_State* L, int newsize); LUAI_FUNC TString* luaS_newlstr(lua_State* L, const char* str, size_t l); -LUAI_FUNC void luaS_free(lua_State* L, TString* ts); +LUAI_FUNC void luaS_free(lua_State* L, TString* ts, struct lua_Page* page); LUAI_FUNC TString* luaS_bufstart(lua_State* L, size_t size); LUAI_FUNC TString* luaS_buffinish(lua_State* L, TString* ts); diff --git a/VM/src/ltable.cpp b/VM/src/ltable.cpp index 83b59f3f..c57374e0 100644 --- a/VM/src/ltable.cpp +++ b/VM/src/ltable.cpp @@ -424,7 +424,7 @@ static void rehash(lua_State* L, Table* t, const TValue* ek) Table* luaH_new(lua_State* L, int narray, int nhash) { - Table* t = luaM_new(L, Table, sizeof(Table), L->activememcat); + Table* t = luaM_newgco(L, Table, sizeof(Table), L->activememcat); luaC_link(L, t, LUA_TTABLE); t->metatable = NULL; t->flags = cast_byte(~0); @@ -443,12 +443,12 @@ Table* luaH_new(lua_State* L, int narray, int nhash) return t; } -void luaH_free(lua_State* L, Table* t) +void luaH_free(lua_State* L, Table* t, lua_Page* page) { if (t->node != dummynode) luaM_freearray(L, t->node, sizenode(t), LuaNode, t->memcat); luaM_freearray(L, t->array, t->sizearray, TValue, t->memcat); - luaM_free(L, t, sizeof(Table), t->memcat); + luaM_freegco(L, t, sizeof(Table), t->memcat, page); } static LuaNode* getfreepos(Table* t) @@ -741,7 +741,7 @@ int luaH_getn(Table* t) Table* luaH_clone(lua_State* L, Table* tt) { - Table* t = luaM_new(L, Table, sizeof(Table), L->activememcat); + Table* t = luaM_newgco(L, Table, sizeof(Table), L->activememcat); luaC_link(L, t, LUA_TTABLE); t->metatable = tt->metatable; t->flags = tt->flags; diff --git a/VM/src/ltable.h b/VM/src/ltable.h index 45061443..e8413c85 100644 --- a/VM/src/ltable.h +++ b/VM/src/ltable.h @@ -20,7 +20,7 @@ LUAI_FUNC TValue* luaH_set(lua_State* L, Table* t, const TValue* key); LUAI_FUNC Table* luaH_new(lua_State* L, int narray, int lnhash); LUAI_FUNC void luaH_resizearray(lua_State* L, Table* t, int nasize); LUAI_FUNC void luaH_resizehash(lua_State* L, Table* t, int nhsize); -LUAI_FUNC void luaH_free(lua_State* L, Table* t); +LUAI_FUNC void luaH_free(lua_State* L, Table* t, struct lua_Page* page); LUAI_FUNC int luaH_next(lua_State* L, Table* t, StkId key); LUAI_FUNC int luaH_getn(Table* t); LUAI_FUNC Table* luaH_clone(lua_State* L, Table* tt); diff --git a/VM/src/ludata.cpp b/VM/src/ludata.cpp index d180c388..758a9bdb 100644 --- a/VM/src/ludata.cpp +++ b/VM/src/ludata.cpp @@ -11,7 +11,7 @@ Udata* luaU_newudata(lua_State* L, size_t s, int tag) { if (s > INT_MAX - sizeof(Udata)) luaM_toobig(L); - Udata* u = luaM_new(L, Udata, sizeudata(s), L->activememcat); + Udata* u = luaM_newgco(L, Udata, sizeudata(s), L->activememcat); luaC_link(L, u, LUA_TUSERDATA); u->len = int(s); u->metatable = NULL; @@ -20,7 +20,7 @@ Udata* luaU_newudata(lua_State* L, size_t s, int tag) return u; } -void luaU_freeudata(lua_State* L, Udata* u) +void luaU_freeudata(lua_State* L, Udata* u, lua_Page* page) { LUAU_ASSERT(u->tag < LUA_UTAG_LIMIT || u->tag == UTAG_IDTOR); @@ -33,5 +33,5 @@ void luaU_freeudata(lua_State* L, Udata* u) if (dtor) dtor(u->data); - luaM_free(L, u, sizeudata(u->len), u->memcat); + luaM_freegco(L, u, sizeudata(u->len), u->memcat, page); } diff --git a/VM/src/ludata.h b/VM/src/ludata.h index 59cb85bd..ec374c28 100644 --- a/VM/src/ludata.h +++ b/VM/src/ludata.h @@ -10,4 +10,4 @@ #define sizeudata(len) (offsetof(Udata, data) + len) LUAI_FUNC Udata* luaU_newudata(lua_State* L, size_t s, int tag); -LUAI_FUNC void luaU_freeudata(lua_State* L, Udata* u); +LUAI_FUNC void luaU_freeudata(lua_State* L, Udata* u, struct lua_Page* page); diff --git a/VM/src/lvmexecute.cpp b/VM/src/lvmexecute.cpp index cebeeb58..e58ff2a8 100644 --- a/VM/src/lvmexecute.cpp +++ b/VM/src/lvmexecute.cpp @@ -496,7 +496,7 @@ static void luau_execute(lua_State* L) Instruction insn = *pc++; StkId ra = VM_REG(LUAU_INSN_A(insn)); - if (L->openupval && gco2uv(L->openupval)->v >= ra) + if (L->openupval && L->openupval->v >= ra) luaF_close(L, ra); VM_NEXT(); } diff --git a/tests/Compiler.test.cpp b/tests/Compiler.test.cpp index 8eed953f..3b0d677d 100644 --- a/tests/Compiler.test.cpp +++ b/tests/Compiler.test.cpp @@ -756,6 +756,30 @@ RETURN R0 1 )"); } +TEST_CASE("TableSizePredictionLoop") +{ + ScopedFastFlag sff("LuauPredictTableSizeLoop", true); + + CHECK_EQ("\n" + compileFunction0(R"( +local t = {} +for i=1,4 do + t[i] = 0 +end +return t +)"), + R"( +NEWTABLE R0 0 4 +LOADN R3 1 +LOADN R1 4 +LOADN R2 1 +FORNPREP R1 +3 +LOADN R4 0 +SETTABLE R4 R0 R3 +FORNLOOP R1 -3 +RETURN R0 1 +)"); +} + TEST_CASE("ReflectionEnums") { CHECK_EQ("\n" + compileFunction0("return Enum.EasingStyle.Linear"), R"( diff --git a/tests/Linter.test.cpp b/tests/Linter.test.cpp index 1d13df28..5ad06f0d 100644 --- a/tests/Linter.test.cpp +++ b/tests/Linter.test.cpp @@ -1396,6 +1396,8 @@ end TEST_CASE_FIXTURE(Fixture, "TableOperations") { + ScopedFastFlag sff("LuauLintTableCreateTable", true); + LintResult result = lintTyped(R"( local t = {} local tt = {} @@ -1416,9 +1418,12 @@ table.insert(t, string.find("hello", "h")) table.move(t, 0, #t, 1, tt) table.move(t, 1, #t, 0, tt) + +table.create(42, {}) +table.create(42, {} :: {}) )"); - REQUIRE_EQ(result.warnings.size(), 8); + REQUIRE_EQ(result.warnings.size(), 10); CHECK_EQ(result.warnings[0].text, "table.insert will insert the value before the last element, which is likely a bug; consider removing the " "second argument or wrap it in parentheses to silence"); CHECK_EQ(result.warnings[1].text, "table.insert will append the value to the table; consider removing the second argument for efficiency"); @@ -1430,6 +1435,8 @@ table.move(t, 1, #t, 0, tt) "table.insert may change behavior if the call returns more than one result; consider adding parentheses around second argument"); CHECK_EQ(result.warnings[6].text, "table.move uses index 0 but arrays are 1-based; did you mean 1 instead?"); CHECK_EQ(result.warnings[7].text, "table.move uses index 0 but arrays are 1-based; did you mean 1 instead?"); + CHECK_EQ(result.warnings[8].text, "table.create with a table literal will reuse the same object for all elements; consider using a for loop instead"); + CHECK_EQ(result.warnings[9].text, "table.create with a table literal will reuse the same object for all elements; consider using a for loop instead"); } TEST_CASE_FIXTURE(Fixture, "DuplicateConditions") diff --git a/tests/Parser.test.cpp b/tests/Parser.test.cpp index e9135651..ac81005c 100644 --- a/tests/Parser.test.cpp +++ b/tests/Parser.test.cpp @@ -1498,6 +1498,17 @@ return CHECK_EQ(std::string(str->value.data, str->value.size), "\n"); } +TEST_CASE_FIXTURE(Fixture, "parse_error_broken_comment") +{ + ScopedFastFlag luauStartingBrokenComment{"LuauStartingBrokenComment", true}; + + const char* expected = "Expected identifier when parsing expression, got unfinished comment"; + + matchParseError("--[[unfinished work", expected); + matchParseError("--!strict\n--[[unfinished work", expected); + matchParseError("local x = 1 --[[unfinished work", expected); +} + TEST_CASE_FIXTURE(Fixture, "string_literals_escapes_broken") { const char* expected = "String literal contains malformed escape sequence"; @@ -2333,7 +2344,7 @@ TEST_CASE_FIXTURE(Fixture, "capture_broken_comment_at_the_start_of_the_file") ParseOptions options; options.captureComments = true; - ParseResult result = parseEx(R"( + ParseResult result = tryParse(R"( --[[ )", options); diff --git a/tests/TypeInfer.tables.test.cpp b/tests/TypeInfer.tables.test.cpp index 27cda146..644efed7 100644 --- a/tests/TypeInfer.tables.test.cpp +++ b/tests/TypeInfer.tables.test.cpp @@ -2180,4 +2180,52 @@ b() CHECK_EQ(toString(result.errors[0]), R"(Cannot call non-function t1 where t1 = { @metatable { __call: t1 }, { } })"); } +TEST_CASE_FIXTURE(Fixture, "length_operator_union") +{ + ScopedFastFlag luauLengthOnCompositeType{"LuauLengthOnCompositeType", true}; + + CheckResult result = check(R"( +local x: {number} | {string} +local y = #x + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "length_operator_intersection") +{ + ScopedFastFlag luauLengthOnCompositeType{"LuauLengthOnCompositeType", true}; + + CheckResult result = check(R"( +local x: {number} & {z:string} -- mixed tables are evil +local y = #x + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "length_operator_non_table_union") +{ + ScopedFastFlag luauLengthOnCompositeType{"LuauLengthOnCompositeType", true}; + + CheckResult result = check(R"( +local x: {number} | any | string +local y = #x + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "length_operator_union_errors") +{ + ScopedFastFlag luauLengthOnCompositeType{"LuauLengthOnCompositeType", true}; + + CheckResult result = check(R"( +local x: {number} | number | string +local y = #x + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.test.cpp b/tests/TypeInfer.test.cpp index 7a056af5..7ee5253c 100644 --- a/tests/TypeInfer.test.cpp +++ b/tests/TypeInfer.test.cpp @@ -5129,4 +5129,33 @@ local c = a or b LUAU_REQUIRE_NO_ERRORS(result); } +TEST_CASE_FIXTURE(Fixture, "bound_typepack_promote") +{ + ScopedFastFlag luauCommittingTxnLogFreeTpPromote{"LuauCommittingTxnLogFreeTpPromote", true}; + + // No assertions should trigger + check(R"( +local function p() + local this = {} + this.pf = foo() + function this:IsActive() end + function this:Start(o) end + return this +end + +local function h(tp, o) + ep = tp + tp:Start(o) + tp.pf.Connect(function() + ep:IsActive() + end) +end + +function on() + local t = p() + h(t) +end + )"); +} + TEST_SUITE_END(); diff --git a/tests/conformance/closure.lua b/tests/conformance/closure.lua index f32d5bdc..7b057354 100644 --- a/tests/conformance/closure.lua +++ b/tests/conformance/closure.lua @@ -419,5 +419,20 @@ co = coroutine.create(function () return loadstring("return a")() end) +-- large closure size +do + local a1, a2, a3, a4, a5, a6, a7, a8, a9, a0 + local b1, b2, b3, b4, b5, b6, b7, b8, b9, b0 + local c1, c2, c3, c4, c5, c6, c7, c8, c9, c0 + local d1, d2, d3, d4, d5, d6, d7, d8, d9, d0 + + local f = function() + return + a1 + a2 + a3 + a4 + a5 + a6 + a7 + a8 + a9 + a0 + + b1 + b2 + b3 + b4 + b5 + b6 + b7 + b8 + b9 + b0 + + c1 + c2 + c3 + c4 + c5 + c6 + c7 + c8 + c9 + c0 + + d1 + d2 + d3 + d4 + d5 + d6 + d7 + d8 + d9 + d0 + end +end return 'OK' diff --git a/tests/conformance/gc.lua b/tests/conformance/gc.lua index 6d9eb854..409cd224 100644 --- a/tests/conformance/gc.lua +++ b/tests/conformance/gc.lua @@ -291,4 +291,32 @@ do for i = 1,10 do table.insert(___Glob, newproxy(true)) end end +-- create threads that die together with their unmarked upvalues +do + local t = {} + + for i = 1,100 do + local c = coroutine.wrap(function() + local uv = {i + 1} + local function f() + return uv[1] * 10 + end + coroutine.yield(uv[1]) + uv = {i + 2} + coroutine.yield(f()) + end) + + assert(c() == i + 1) + table.insert(t, c) + end + + for i = 1,100 do + t[i] = nil + end + + collectgarbage() + +end + + return('OK') From 4b96f7efc12f04fef2f9511811c807bb30dedaef Mon Sep 17 00:00:00 2001 From: Vlad Marica Date: Tue, 25 Jan 2022 08:25:01 -0800 Subject: [PATCH 138/399] luau-analyze: Add support for reading source code from stdin (#325) Co-authored-by: Arseny Kapoulkine --- CLI/Analyze.cpp | 40 +++++++++++++++++++++++++++++++--------- CLI/FileUtils.cpp | 26 ++++++++++++++++++++++++-- CLI/FileUtils.h | 1 + 3 files changed, 56 insertions(+), 11 deletions(-) diff --git a/CLI/Analyze.cpp b/CLI/Analyze.cpp index bf04281c..39fc78ef 100644 --- a/CLI/Analyze.cpp +++ b/CLI/Analyze.cpp @@ -43,14 +43,14 @@ static void report(ReportFormat format, const char* name, const Luau::Location& } } -static void reportError(ReportFormat format, const Luau::TypeError& error) +static void reportError(Luau::Frontend& frontend, ReportFormat format, const Luau::TypeError& error) { - const char* name = error.moduleName.c_str(); + std::string humanReadableName = frontend.fileResolver->getHumanReadableModuleName(error.moduleName); if (const Luau::SyntaxError* syntaxError = Luau::get_if(&error.data)) - report(format, name, error.location, "SyntaxError", syntaxError->message.c_str()); + report(format, humanReadableName.c_str(), error.location, "SyntaxError", syntaxError->message.c_str()); else - report(format, name, error.location, "TypeError", Luau::toString(error).c_str()); + report(format, humanReadableName.c_str(), error.location, "TypeError", Luau::toString(error).c_str()); } static void reportWarning(ReportFormat format, const char* name, const Luau::LintWarning& warning) @@ -72,14 +72,15 @@ static bool analyzeFile(Luau::Frontend& frontend, const char* name, ReportFormat } for (auto& error : cr.errors) - reportError(format, error); + reportError(frontend, format, error); Luau::LintResult lr = frontend.lint(name); + std::string humanReadableName = frontend.fileResolver->getHumanReadableModuleName(name); for (auto& error : lr.errors) - reportWarning(format, name, error); + reportWarning(format, humanReadableName.c_str(), error); for (auto& warning : lr.warnings) - reportWarning(format, name, warning); + reportWarning(format, humanReadableName.c_str(), warning); if (annotate) { @@ -120,11 +121,25 @@ struct CliFileResolver : Luau::FileResolver { std::optional readSource(const Luau::ModuleName& name) override { - std::optional source = readFile(name); + Luau::SourceCode::Type sourceType; + std::optional source = std::nullopt; + + // If the module name is "-", then read source from stdin + if (name == "-") + { + source = readStdin(); + sourceType = Luau::SourceCode::Script; + } + else + { + source = readFile(name); + sourceType = Luau::SourceCode::Module; + } + if (!source) return std::nullopt; - return Luau::SourceCode{*source, Luau::SourceCode::Module}; + return Luau::SourceCode{*source, sourceType}; } std::optional resolveModule(const Luau::ModuleInfo* context, Luau::AstExpr* node) override @@ -143,6 +158,13 @@ struct CliFileResolver : Luau::FileResolver return std::nullopt; } + + std::string getHumanReadableModuleName(const Luau::ModuleName& name) const override + { + if (name == "-") + return "stdin"; + return name; + } }; struct CliConfigResolver : Luau::ConfigResolver diff --git a/CLI/FileUtils.cpp b/CLI/FileUtils.cpp index cb993dfe..87decb51 100644 --- a/CLI/FileUtils.cpp +++ b/CLI/FileUtils.cpp @@ -15,6 +15,8 @@ #include +#define READ_BUFFER_SIZE 4096 + #ifdef _WIN32 static std::wstring fromUtf8(const std::string& path) { @@ -74,6 +76,21 @@ std::optional readFile(const std::string& name) return result; } +std::optional readStdin() +{ + std::string result; + char buffer[READ_BUFFER_SIZE] = { }; + + while (fgets(buffer, READ_BUFFER_SIZE, stdin) != nullptr) + result.append(buffer); + + // If eof was not reached for stdin, then a read error occurred + if (!feof(stdin)) + return std::nullopt; + + return result; +} + template static void joinPaths(std::basic_string& str, const Ch* lhs, const Ch* rhs) { @@ -190,7 +207,10 @@ bool traverseDirectory(const std::string& path, const std::function getSourceFiles(int argc, char** argv) for (int i = 1; i < argc; ++i) { - if (argv[i][0] == '-') + // Treat '-' as a special file whose source is read from stdin + // All other arguments that start with '-' are skipped + if (argv[i][0] == '-' && argv[i][1] != '\0') continue; if (isDirectory(argv[i])) diff --git a/CLI/FileUtils.h b/CLI/FileUtils.h index da11f512..97471cdc 100644 --- a/CLI/FileUtils.h +++ b/CLI/FileUtils.h @@ -7,6 +7,7 @@ #include std::optional readFile(const std::string& name); +std::optional readStdin(); bool isDirectory(const std::string& path); bool traverseDirectory(const std::string& path, const std::function& callback); From 6e1e277cb8f19f18740259c830ef49920f76043d Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Thu, 27 Jan 2022 13:29:34 -0800 Subject: [PATCH 139/399] Sync to upstream/release/512 --- Analysis/include/Luau/AstQuery.h | 15 + Analysis/include/Luau/Module.h | 8 + Analysis/include/Luau/TypeInfer.h | 21 +- Analysis/include/Luau/TypeVar.h | 8 +- Analysis/include/Luau/Unifier.h | 7 + Analysis/src/Autocomplete.cpp | 105 ++--- Analysis/src/Frontend.cpp | 7 +- Analysis/src/Module.cpp | 28 +- Analysis/src/ToString.cpp | 11 +- Analysis/src/TypeAttach.cpp | 2 +- Analysis/src/TypeInfer.cpp | 484 +++++++++++++--------- Analysis/src/TypeVar.cpp | 138 ++++--- Analysis/src/Unifier.cpp | 319 ++++++--------- CLI/Analyze.cpp | 40 +- CLI/FileUtils.cpp | 24 +- CLI/FileUtils.h | 1 + CLI/Repl.cpp | 17 +- CLI/Repl.h | 12 + CLI/ReplEntry.cpp | 10 + CMakeLists.txt | 12 + Compiler/include/Luau/Bytecode.h | 3 + Compiler/src/Builtins.cpp | 5 + Compiler/src/Compiler.cpp | 353 ++++++++++------ Makefile | 7 +- Sources.cmake | 18 +- VM/src/lapi.cpp | 13 +- VM/src/lbuiltins.cpp | 30 ++ VM/src/lcorolib.cpp | 5 - VM/src/ldo.cpp | 4 +- VM/src/lgc.cpp | 35 +- VM/src/lgc.h | 1 + VM/src/lmem.cpp | 6 +- VM/src/lstate.h | 3 - tests/AstQuery.test.cpp | 6 - tests/Autocomplete.test.cpp | 36 +- tests/Compiler.test.cpp | 131 ++++++ tests/Conformance.test.cpp | 2 - tests/Frontend.test.cpp | 1 - tests/Parser.test.cpp | 134 +++--- tests/Repl.test.cpp | 117 ++++++ tests/ToString.test.cpp | 4 - tests/TypeInfer.aliases.test.cpp | 4 - tests/TypeInfer.generics.test.cpp | 15 +- tests/TypeInfer.provisional.test.cpp | 28 +- tests/TypeInfer.refinements.test.cpp | 591 ++++++++++++++++----------- tests/TypeInfer.singletons.test.cpp | 6 - tests/TypeInfer.tables.test.cpp | 16 +- tests/TypeInfer.test.cpp | 64 +-- tests/TypeInfer.typePacks.cpp | 1 - tests/TypeInfer.unionTypes.test.cpp | 2 - tests/TypeVar.test.cpp | 72 +++- 51 files changed, 1836 insertions(+), 1146 deletions(-) create mode 100644 CLI/Repl.h create mode 100644 CLI/ReplEntry.cpp create mode 100644 tests/Repl.test.cpp diff --git a/Analysis/include/Luau/AstQuery.h b/Analysis/include/Luau/AstQuery.h index d38976ef..dfe373a5 100644 --- a/Analysis/include/Luau/AstQuery.h +++ b/Analysis/include/Luau/AstQuery.h @@ -42,6 +42,21 @@ struct ExprOrLocal { return expr ? expr->location : (local ? local->location : std::optional{}); } + std::optional getName() + { + if (expr) + { + if (AstName name = getIdentifier(expr); name.value) + { + return name; + } + } + else if (local) + { + return local->name; + } + return std::nullopt; + } private: AstExpr* expr = nullptr; diff --git a/Analysis/include/Luau/Module.h b/Analysis/include/Luau/Module.h index 2e41674b..1bf0473c 100644 --- a/Analysis/include/Luau/Module.h +++ b/Analysis/include/Luau/Module.h @@ -13,6 +13,8 @@ #include #include +LUAU_FASTFLAG(LuauPrepopulateUnionOptionsBeforeAllocation) + namespace Luau { @@ -58,6 +60,12 @@ struct TypeArena template TypeId addType(T tv) { + if (FFlag::LuauPrepopulateUnionOptionsBeforeAllocation) + { + if constexpr (std::is_same_v) + LUAU_ASSERT(tv.options.size() >= 2); + } + return addTV(TypeVar(std::move(tv))); } diff --git a/Analysis/include/Luau/TypeInfer.h b/Analysis/include/Luau/TypeInfer.h index aa090014..b843509d 100644 --- a/Analysis/include/Luau/TypeInfer.h +++ b/Analysis/include/Luau/TypeInfer.h @@ -135,7 +135,8 @@ struct TypeChecker void checkBlock(const ScopePtr& scope, const AstStatBlock& statement); void checkBlockTypeAliases(const ScopePtr& scope, std::vector& sorted); - ExprResult checkExpr(const ScopePtr& scope, const AstExpr& expr, std::optional expectedType = std::nullopt); + ExprResult checkExpr( + const ScopePtr& scope, const AstExpr& expr, std::optional expectedType = std::nullopt, bool forceSingleton = false); ExprResult checkExpr(const ScopePtr& scope, const AstExprLocal& expr); ExprResult checkExpr(const ScopePtr& scope, const AstExprGlobal& expr); ExprResult checkExpr(const ScopePtr& scope, const AstExprVarargs& expr); @@ -160,14 +161,12 @@ struct TypeChecker // Returns the type of the lvalue. TypeId checkLValue(const ScopePtr& scope, const AstExpr& expr); - // Returns both the type of the lvalue and its binding (if the caller wants to mutate the binding). - // Note: the binding may be null. - // TODO: remove second return value with FFlagLuauUpdateFunctionNameBinding - std::pair checkLValueBinding(const ScopePtr& scope, const AstExpr& expr); - std::pair checkLValueBinding(const ScopePtr& scope, const AstExprLocal& expr); - std::pair checkLValueBinding(const ScopePtr& scope, const AstExprGlobal& expr); - std::pair checkLValueBinding(const ScopePtr& scope, const AstExprIndexName& expr); - std::pair checkLValueBinding(const ScopePtr& scope, const AstExprIndexExpr& expr); + // Returns the type of the lvalue. + TypeId checkLValueBinding(const ScopePtr& scope, const AstExpr& expr); + TypeId checkLValueBinding(const ScopePtr& scope, const AstExprLocal& expr); + TypeId checkLValueBinding(const ScopePtr& scope, const AstExprGlobal& expr); + TypeId checkLValueBinding(const ScopePtr& scope, const AstExprIndexName& expr); + TypeId checkLValueBinding(const ScopePtr& scope, const AstExprIndexExpr& expr); TypeId checkFunctionName(const ScopePtr& scope, AstExpr& funName, TypeLevel level); std::pair checkFunctionSignature(const ScopePtr& scope, int subLevel, const AstExprFunction& expr, @@ -322,8 +321,6 @@ private: return addTV(TypeVar(tv)); } - TypeId addType(const UnionTypeVar& utv); - TypeId addTV(TypeVar&& tv); TypePackId addTypePack(TypePackVar&& tp); @@ -349,6 +346,8 @@ public: ErrorVec resolve(const PredicateVec& predicates, const ScopePtr& scope, bool sense); private: + void refineLValue(const LValue& lvalue, RefinementMap& refis, const ScopePtr& scope, TypeIdPredicate predicate); + std::optional resolveLValue(const ScopePtr& scope, const LValue& lvalue); std::optional DEPRECATED_resolveLValue(const ScopePtr& scope, const LValue& lvalue); std::optional resolveLValue(const RefinementMap& refis, const ScopePtr& scope, const LValue& lvalue); diff --git a/Analysis/include/Luau/TypeVar.h b/Analysis/include/Luau/TypeVar.h index 3f5e26d6..11dc9377 100644 --- a/Analysis/include/Luau/TypeVar.h +++ b/Analysis/include/Luau/TypeVar.h @@ -111,16 +111,16 @@ struct PrimitiveTypeVar // Singleton types https://github.com/Roblox/luau/blob/master/rfcs/syntax-singleton-types.md // Types for true and false -struct BoolSingleton +struct BooleanSingleton { bool value; - bool operator==(const BoolSingleton& rhs) const + bool operator==(const BooleanSingleton& rhs) const { return value == rhs.value; } - bool operator!=(const BoolSingleton& rhs) const + bool operator!=(const BooleanSingleton& rhs) const { return !(*this == rhs); } @@ -145,7 +145,7 @@ struct StringSingleton // No type for float singletons, partly because === isn't any equalivalence on floats // (NaN != NaN). -using SingletonVariant = Luau::Variant; +using SingletonVariant = Luau::Variant; struct SingletonTypeVar { diff --git a/Analysis/include/Luau/Unifier.h b/Analysis/include/Luau/Unifier.h index a3be739a..1b1671c0 100644 --- a/Analysis/include/Luau/Unifier.h +++ b/Analysis/include/Luau/Unifier.h @@ -85,6 +85,13 @@ public: Unifier makeChildUnifier(); + // A utility function that appends the given error to the unifier's error log. + // This allows setting a breakpoint wherever the unifier reports an error. + void reportError(TypeError error) + { + errors.push_back(error); + } + private: bool isNonstrictMode() const; diff --git a/Analysis/src/Autocomplete.cpp b/Analysis/src/Autocomplete.cpp index 7a801f97..85099e12 100644 --- a/Analysis/src/Autocomplete.cpp +++ b/Analysis/src/Autocomplete.cpp @@ -14,9 +14,9 @@ LUAU_FASTFLAG(LuauUseCommittingTxnLog) LUAU_FASTFLAGVARIABLE(LuauAutocompleteAvoidMutation, false); -LUAU_FASTFLAGVARIABLE(LuauAutocompleteFirstArg, false); LUAU_FASTFLAGVARIABLE(LuauCompleteBrokenStringParams, false); LUAU_FASTFLAGVARIABLE(LuauMissingFollowACMetatables, false); +LUAU_FASTFLAGVARIABLE(PreferToCallFunctionsForIntersects, false); static const std::unordered_set kStatementStartingKeywords = { "while", "if", "local", "repeat", "function", "do", "for", "return", "break", "continue", "type", "export"}; @@ -194,8 +194,6 @@ static ParenthesesRecommendation getParenRecommendation(TypeId id, const std::ve static std::optional findExpectedTypeAt(const Module& module, AstNode* node, Position position) { - LUAU_ASSERT(FFlag::LuauAutocompleteFirstArg); - auto expr = node->asExpr(); if (!expr) return std::nullopt; @@ -266,43 +264,63 @@ static TypeCorrectKind checkTypeCorrectKind(const Module& module, TypeArena* typ } }; - TypeId expectedType; + auto typeAtPosition = findExpectedTypeAt(module, node, position); - if (FFlag::LuauAutocompleteFirstArg) + if (!typeAtPosition) + return TypeCorrectKind::None; + + TypeId expectedType = follow(*typeAtPosition); + + if (FFlag::PreferToCallFunctionsForIntersects) { - auto typeAtPosition = findExpectedTypeAt(module, node, position); + auto checkFunctionType = [&canUnify, &expectedType](const FunctionTypeVar* ftv) { + auto [retHead, retTail] = flatten(ftv->retType); - if (!typeAtPosition) - return TypeCorrectKind::None; + if (!retHead.empty() && canUnify(retHead.front(), expectedType)) + return true; - expectedType = follow(*typeAtPosition); + // We might only have a variadic tail pack, check if the element is compatible + if (retTail) + { + if (const VariadicTypePack* vtp = get(follow(*retTail)); vtp && canUnify(vtp->ty, expectedType)) + return true; + } + + return false; + }; + + // We also want to suggest functions that return compatible result + if (const FunctionTypeVar* ftv = get(ty); ftv && checkFunctionType(ftv)) + { + return TypeCorrectKind::CorrectFunctionResult; + } + else if (const IntersectionTypeVar* itv = get(ty)) + { + for (TypeId id : itv->parts) + { + if (const FunctionTypeVar* ftv = get(id); ftv && checkFunctionType(ftv)) + { + return TypeCorrectKind::CorrectFunctionResult; + } + } + } } else { - auto expr = node->asExpr(); - if (!expr) - return TypeCorrectKind::None; - - auto it = module.astExpectedTypes.find(expr); - if (!it) - return TypeCorrectKind::None; - - expectedType = follow(*it); - } - - // We also want to suggest functions that return compatible result - if (const FunctionTypeVar* ftv = get(ty)) - { - auto [retHead, retTail] = flatten(ftv->retType); - - if (!retHead.empty() && canUnify(retHead.front(), expectedType)) - return TypeCorrectKind::CorrectFunctionResult; - - // We might only have a variadic tail pack, check if the element is compatible - if (retTail) + // We also want to suggest functions that return compatible result + if (const FunctionTypeVar* ftv = get(ty)) { - if (const VariadicTypePack* vtp = get(follow(*retTail)); vtp && canUnify(vtp->ty, expectedType)) + auto [retHead, retTail] = flatten(ftv->retType); + + if (!retHead.empty() && canUnify(retHead.front(), expectedType)) return TypeCorrectKind::CorrectFunctionResult; + + // We might only have a variadic tail pack, check if the element is compatible + if (retTail) + { + if (const VariadicTypePack* vtp = get(follow(*retTail)); vtp && canUnify(vtp->ty, expectedType)) + return TypeCorrectKind::CorrectFunctionResult; + } } } @@ -741,29 +759,12 @@ std::optional returnFirstNonnullOptionOfType(const UnionTypeVar* utv) static std::optional functionIsExpectedAt(const Module& module, AstNode* node, Position position) { - TypeId expectedType; + auto typeAtPosition = findExpectedTypeAt(module, node, position); - if (FFlag::LuauAutocompleteFirstArg) - { - auto typeAtPosition = findExpectedTypeAt(module, node, position); + if (!typeAtPosition) + return std::nullopt; - if (!typeAtPosition) - return std::nullopt; - - expectedType = follow(*typeAtPosition); - } - else - { - auto expr = node->asExpr(); - if (!expr) - return std::nullopt; - - auto it = module.astExpectedTypes.find(expr); - if (!it) - return std::nullopt; - - expectedType = follow(*it); - } + TypeId expectedType = follow(*typeAtPosition); if (get(expectedType)) return true; diff --git a/Analysis/src/Frontend.cpp b/Analysis/src/Frontend.cpp index fe4b6529..9001b19d 100644 --- a/Analysis/src/Frontend.cpp +++ b/Analysis/src/Frontend.cpp @@ -18,7 +18,6 @@ LUAU_FASTFLAG(LuauInferInNoCheckMode) LUAU_FASTFLAGVARIABLE(LuauTypeCheckTwice, false) LUAU_FASTFLAGVARIABLE(LuauKnowsTheDataModel3, false) -LUAU_FASTFLAGVARIABLE(LuauPersistDefinitionFileTypes, false) namespace Luau { @@ -102,8 +101,7 @@ LoadDefinitionFileResult loadDefinitionFile(TypeChecker& typeChecker, ScopePtr t generateDocumentationSymbols(globalTy, documentationSymbol); targetScope->bindings[typeChecker.globalNames.names->getOrAdd(name.c_str())] = {globalTy, Location(), false, {}, documentationSymbol}; - if (FFlag::LuauPersistDefinitionFileTypes) - persist(globalTy); + persist(globalTy); } for (const auto& [name, ty] : checkedModule->getModuleScope()->exportedTypeBindings) @@ -113,8 +111,7 @@ LoadDefinitionFileResult loadDefinitionFile(TypeChecker& typeChecker, ScopePtr t generateDocumentationSymbols(globalTy.type, documentationSymbol); targetScope->exportedTypeBindings[name] = globalTy; - if (FFlag::LuauPersistDefinitionFileTypes) - persist(globalTy.type); + persist(globalTy.type); } return LoadDefinitionFileResult{true, parseResult, checkedModule}; diff --git a/Analysis/src/Module.cpp b/Analysis/src/Module.cpp index 9f352f4b..4fdff8f7 100644 --- a/Analysis/src/Module.cpp +++ b/Analysis/src/Module.cpp @@ -16,6 +16,8 @@ LUAU_FASTFLAGVARIABLE(DebugLuauTrackOwningArena, false) LUAU_FASTINTVARIABLE(LuauTypeCloneRecursionLimit, 300) LUAU_FASTFLAG(LuauTypeAliasDefaults) +LUAU_FASTFLAGVARIABLE(LuauPrepopulateUnionOptionsBeforeAllocation, false) + namespace Luau { @@ -377,14 +379,28 @@ void TypeCloner::operator()(const AnyTypeVar& t) void TypeCloner::operator()(const UnionTypeVar& t) { - TypeId result = dest.addType(UnionTypeVar{}); - seenTypes[typeId] = result; + if (FFlag::LuauPrepopulateUnionOptionsBeforeAllocation) + { + std::vector options; + options.reserve(t.options.size()); - UnionTypeVar* option = getMutable(result); - LUAU_ASSERT(option != nullptr); + for (TypeId ty : t.options) + options.push_back(clone(ty, dest, seenTypes, seenTypePacks, cloneState)); - for (TypeId ty : t.options) - option->options.push_back(clone(ty, dest, seenTypes, seenTypePacks, cloneState)); + TypeId result = dest.addType(UnionTypeVar{std::move(options)}); + seenTypes[typeId] = result; + } + else + { + TypeId result = dest.addType(UnionTypeVar{}); + seenTypes[typeId] = result; + + UnionTypeVar* option = getMutable(result); + LUAU_ASSERT(option != nullptr); + + for (TypeId ty : t.options) + option->options.push_back(clone(ty, dest, seenTypes, seenTypePacks, cloneState)); + } } void TypeCloner::operator()(const IntersectionTypeVar& t) diff --git a/Analysis/src/ToString.cpp b/Analysis/src/ToString.cpp index 4b898d3a..5e79b841 100644 --- a/Analysis/src/ToString.cpp +++ b/Analysis/src/ToString.cpp @@ -10,7 +10,6 @@ #include #include -LUAU_FASTFLAG(LuauOccursCheckOkWithRecursiveFunctions) LUAU_FASTFLAG(LuauTypeAliasDefaults) /* @@ -374,7 +373,7 @@ struct TypeVarStringifier void operator()(TypeId, const SingletonTypeVar& stv) { - if (const BoolSingleton* bs = Luau::get(&stv)) + if (const BooleanSingleton* bs = Luau::get(&stv)) state.emit(bs->value ? "true" : "false"); else if (const StringSingleton* ss = Luau::get(&stv)) { @@ -617,9 +616,7 @@ struct TypeVarStringifier std::string saved = std::move(state.result.name); - bool needParens = FFlag::LuauOccursCheckOkWithRecursiveFunctions - ? !state.cycleNames.count(el) && (get(el) || get(el)) - : get(el) || get(el); + bool needParens = !state.cycleNames.count(el) && (get(el) || get(el)); if (needParens) state.emit("("); @@ -675,9 +672,7 @@ struct TypeVarStringifier std::string saved = std::move(state.result.name); - bool needParens = FFlag::LuauOccursCheckOkWithRecursiveFunctions - ? !state.cycleNames.count(el) && (get(el) || get(el)) - : get(el) || get(el); + bool needParens = !state.cycleNames.count(el) && (get(el) || get(el)); if (needParens) state.emit("("); diff --git a/Analysis/src/TypeAttach.cpp b/Analysis/src/TypeAttach.cpp index 2ec02093..2208213f 100644 --- a/Analysis/src/TypeAttach.cpp +++ b/Analysis/src/TypeAttach.cpp @@ -97,7 +97,7 @@ public: AstType* operator()(const SingletonTypeVar& stv) { - if (const BoolSingleton* bs = get(&stv)) + if (const BooleanSingleton* bs = get(&stv)) return allocator->alloc(Location(), bs->value); else if (const StringSingleton* ss = get(&stv)) { diff --git a/Analysis/src/TypeInfer.cpp b/Analysis/src/TypeInfer.cpp index e2d8a4fb..23fcc2d5 100644 --- a/Analysis/src/TypeInfer.cpp +++ b/Analysis/src/TypeInfer.cpp @@ -26,8 +26,6 @@ LUAU_FASTFLAG(LuauKnowsTheDataModel3) LUAU_FASTFLAGVARIABLE(LuauEqConstraint, false) LUAU_FASTFLAGVARIABLE(LuauGroupExpectedType, false) LUAU_FASTFLAGVARIABLE(LuauWeakEqConstraint, false) // Eventually removed as false. -LUAU_FASTFLAGVARIABLE(LuauCloneCorrectlyBeforeMutatingTableType, false) -LUAU_FASTFLAGVARIABLE(LuauStoreMatchingOverloadFnType, false) LUAU_FASTFLAG(LuauUseCommittingTxnLog) LUAU_FASTFLAGVARIABLE(DebugLuauFreezeDuringUnification, false) LUAU_FASTFLAGVARIABLE(LuauRecursiveTypeParameterRestriction, false) @@ -37,6 +35,7 @@ LUAU_FASTFLAGVARIABLE(LuauLengthOnCompositeType, false) LUAU_FASTFLAGVARIABLE(LuauQuantifyInPlace2, false) LUAU_FASTFLAGVARIABLE(LuauSealExports, false) LUAU_FASTFLAGVARIABLE(LuauSingletonTypes, false) +LUAU_FASTFLAGVARIABLE(LuauDiscriminableUnions, false) LUAU_FASTFLAGVARIABLE(LuauTypeAliasDefaults, false) LUAU_FASTFLAGVARIABLE(LuauExpectedTypesOfProperties, false) LUAU_FASTFLAGVARIABLE(LuauErrorRecoveryType, false) @@ -46,10 +45,8 @@ LUAU_FASTFLAGVARIABLE(LuauRefiLookupFromIndexExpr, false) LUAU_FASTFLAGVARIABLE(LuauPerModuleUnificationCache, false) LUAU_FASTFLAGVARIABLE(LuauProperTypeLevels, false) LUAU_FASTFLAGVARIABLE(LuauAscribeCorrectLevelToInferredProperitesOfFreeTables, false) -LUAU_FASTFLAGVARIABLE(LuauFixRecursiveMetatableCall, false) LUAU_FASTFLAGVARIABLE(LuauBidirectionalAsExpr, false) LUAU_FASTFLAGVARIABLE(LuauUnsealedTableLiteral, false) -LUAU_FASTFLAGVARIABLE(LuauUpdateFunctionNameBinding, false) namespace Luau { @@ -1139,33 +1136,25 @@ void TypeChecker::check(const ScopePtr& scope, TypeId ty, const ScopePtr& funSco } else { - auto [leftType, leftTypeBinding] = checkLValueBinding(scope, *function.name); + TypeId leftType = checkLValueBinding(scope, *function.name); checkFunctionBody(funScope, ty, *function.func); unify(ty, leftType, function.location); - if (FFlag::LuauUpdateFunctionNameBinding) - { - LUAU_ASSERT(function.name->is() || function.name->is()); + LUAU_ASSERT(function.name->is() || function.name->is()); - if (auto exprIndexName = function.name->as()) + if (auto exprIndexName = function.name->as()) + { + if (auto typeIt = currentModule->astTypes.find(exprIndexName->expr)) { - if (auto typeIt = currentModule->astTypes.find(exprIndexName->expr)) + if (auto ttv = getMutableTableType(*typeIt)) { - if (auto ttv = getMutableTableType(*typeIt)) - { - if (auto it = ttv->props.find(exprIndexName->index.value); it != ttv->props.end()) - it->second.type = follow(quantify(funScope, leftType, function.name->location)); - } + if (auto it = ttv->props.find(exprIndexName->index.value); it != ttv->props.end()) + it->second.type = follow(quantify(funScope, leftType, function.name->location)); } } } - else - { - if (leftTypeBinding) - *leftTypeBinding = follow(quantify(funScope, leftType, function.name->location)); - } } } @@ -1426,7 +1415,7 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatDeclareFunction& glo currentModule->getModuleScope()->bindings[global.name] = Binding{fnType, global.location}; } -ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExpr& expr, std::optional expectedType) +ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExpr& expr, std::optional expectedType, bool forceSingleton) { RecursionCounter _rc(&checkRecursionCount); if (FInt::LuauCheckRecursionLimit > 0 && checkRecursionCount >= FInt::LuauCheckRecursionLimit) @@ -1443,14 +1432,14 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExpr& result = {nilType}; else if (const AstExprConstantBool* bexpr = expr.as()) { - if (FFlag::LuauSingletonTypes && expectedType && maybeSingleton(*expectedType)) + if (FFlag::LuauSingletonTypes && (forceSingleton || (expectedType && maybeSingleton(*expectedType)))) result = {singletonType(bexpr->value)}; else result = {booleanType}; } else if (const AstExprConstantString* sexpr = expr.as()) { - if (FFlag::LuauSingletonTypes && expectedType && maybeSingleton(*expectedType)) + if (FFlag::LuauSingletonTypes && (forceSingleton || (expectedType && maybeSingleton(*expectedType)))) result = {singletonType(std::string(sexpr->value.data, sexpr->value.size))}; else result = {stringType}; @@ -1488,15 +1477,8 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExpr& result.type = follow(result.type); - if (FFlag::LuauStoreMatchingOverloadFnType) - { - if (!currentModule->astTypes.find(&expr)) - currentModule->astTypes[&expr] = result.type; - } - else - { + if (!currentModule->astTypes.find(&expr)) currentModule->astTypes[&expr] = result.type; - } if (expectedType) currentModule->astExpectedTypes[&expr] = *expectedType; @@ -2242,7 +2224,6 @@ TypeId TypeChecker::checkRelationalOperation( state.log.commit(); } - bool needsMetamethod = !isEquality; TypeId leftType = follow(lhsType); @@ -2250,10 +2231,11 @@ TypeId TypeChecker::checkRelationalOperation( { reportErrors(state.errors); - const PrimitiveTypeVar* ptv = get(leftType); - if (!isEquality && state.errors.empty() && (get(leftType) || (ptv && ptv->type == PrimitiveTypeVar::Boolean))) + if (!isEquality && state.errors.empty() && (get(leftType) || isBoolean(leftType))) + { reportError(expr.location, GenericError{format("Type '%s' cannot be compared with relational operator %s", toString(leftType).c_str(), toString(expr.op).c_str())}); + } return booleanType; } @@ -2501,7 +2483,8 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprBi ExprResult rhs = checkExpr(innerScope, *expr.right); - return {checkBinaryOperation(innerScope, expr, lhs.type, rhs.type), {AndPredicate{std::move(lhs.predicates), std::move(rhs.predicates)}}}; + return {checkBinaryOperation(FFlag::LuauDiscriminableUnions ? scope : innerScope, expr, lhs.type, rhs.type), + {AndPredicate{std::move(lhs.predicates), std::move(rhs.predicates)}}}; } else if (expr.op == AstExprBinary::Or) { @@ -2513,7 +2496,7 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprBi ExprResult rhs = checkExpr(innerScope, *expr.right); // Because of C++, I'm not sure if lhs.predicates was not moved out by the time we call checkBinaryOperation. - TypeId result = checkBinaryOperation(innerScope, expr, lhs.type, rhs.type, lhs.predicates); + TypeId result = checkBinaryOperation(FFlag::LuauDiscriminableUnions ? scope : innerScope, expr, lhs.type, rhs.type, lhs.predicates); return {result, {OrPredicate{std::move(lhs.predicates), std::move(rhs.predicates)}}}; } else if (expr.op == AstExprBinary::CompareEq || expr.op == AstExprBinary::CompareNe) @@ -2521,8 +2504,8 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprBi if (auto predicate = tryGetTypeGuardPredicate(expr)) return {booleanType, {std::move(*predicate)}}; - ExprResult lhs = checkExpr(scope, *expr.left); - ExprResult rhs = checkExpr(scope, *expr.right); + ExprResult lhs = checkExpr(scope, *expr.left, std::nullopt, /*forceSingleton=*/FFlag::LuauDiscriminableUnions); + ExprResult rhs = checkExpr(scope, *expr.right, std::nullopt, /*forceSingleton=*/FFlag::LuauDiscriminableUnions); PredicateVec predicates; @@ -2621,11 +2604,10 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprIf TypeId TypeChecker::checkLValue(const ScopePtr& scope, const AstExpr& expr) { - auto [ty, binding] = checkLValueBinding(scope, expr); - return ty; + return checkLValueBinding(scope, expr); } -std::pair TypeChecker::checkLValueBinding(const ScopePtr& scope, const AstExpr& expr) +TypeId TypeChecker::checkLValueBinding(const ScopePtr& scope, const AstExpr& expr) { if (auto a = expr.as()) return checkLValueBinding(scope, *a); @@ -2639,22 +2621,22 @@ std::pair TypeChecker::checkLValueBinding(const ScopePtr& scope { for (AstExpr* expr : a->expressions) checkExpr(scope, *expr); - return {errorRecoveryType(scope), nullptr}; + return errorRecoveryType(scope); } else ice("Unexpected AST node in checkLValue", expr.location); } -std::pair TypeChecker::checkLValueBinding(const ScopePtr& scope, const AstExprLocal& expr) +TypeId TypeChecker::checkLValueBinding(const ScopePtr& scope, const AstExprLocal& expr) { if (std::optional ty = scope->lookup(expr.local)) - return {*ty, nullptr}; + return *ty; reportError(expr.location, UnknownSymbol{expr.local->name.value, UnknownSymbol::Binding}); - return {errorRecoveryType(scope), nullptr}; + return errorRecoveryType(scope); } -std::pair TypeChecker::checkLValueBinding(const ScopePtr& scope, const AstExprGlobal& expr) +TypeId TypeChecker::checkLValueBinding(const ScopePtr& scope, const AstExprGlobal& expr) { Name name = expr.name.value; ScopePtr moduleScope = currentModule->getModuleScope(); @@ -2662,7 +2644,7 @@ std::pair TypeChecker::checkLValueBinding(const ScopePtr& scope const auto it = moduleScope->bindings.find(expr.name); if (it != moduleScope->bindings.end()) - return std::pair(it->second.typeId, &it->second.typeId); + return it->second.typeId; TypeId result = freshType(scope); Binding& binding = moduleScope->bindings[expr.name]; @@ -2673,15 +2655,15 @@ std::pair TypeChecker::checkLValueBinding(const ScopePtr& scope if (!isNonstrictMode()) reportError(TypeError{expr.location, UnknownSymbol{name, UnknownSymbol::Binding}}); - return std::pair(result, &binding.typeId); + return result; } -std::pair TypeChecker::checkLValueBinding(const ScopePtr& scope, const AstExprIndexName& expr) +TypeId TypeChecker::checkLValueBinding(const ScopePtr& scope, const AstExprIndexName& expr) { TypeId lhs = checkExpr(scope, *expr.expr).type; if (get(lhs) || get(lhs)) - return std::pair(lhs, nullptr); + return lhs; tablify(lhs); @@ -2694,7 +2676,7 @@ std::pair TypeChecker::checkLValueBinding(const ScopePtr& scope const auto& it = lhsTable->props.find(name); if (it != lhsTable->props.end()) { - return std::pair(it->second.type, &it->second.type); + return it->second.type; } else if (lhsTable->state == TableState::Unsealed || lhsTable->state == TableState::Free) { @@ -2702,7 +2684,7 @@ std::pair TypeChecker::checkLValueBinding(const ScopePtr& scope Property& property = lhsTable->props[name]; property.type = theType; property.location = expr.indexLocation; - return std::pair(theType, &property.type); + return theType; } else if (auto indexer = lhsTable->indexer) { @@ -2720,17 +2702,17 @@ std::pair TypeChecker::checkLValueBinding(const ScopePtr& scope else if (FFlag::LuauUseCommittingTxnLog) state.log.commit(); - return std::pair(retType, nullptr); + return retType; } else if (lhsTable->state == TableState::Sealed) { reportError(TypeError{expr.location, CannotExtendTable{lhs, CannotExtendTable::Property, name}}); - return std::pair(errorRecoveryType(scope), nullptr); + return errorRecoveryType(scope); } else { reportError(TypeError{expr.location, GenericError{"Internal error: generic tables are not lvalues"}}); - return std::pair(errorRecoveryType(scope), nullptr); + return errorRecoveryType(scope); } } else if (const ClassTypeVar* lhsClass = get(lhs)) @@ -2739,29 +2721,29 @@ std::pair TypeChecker::checkLValueBinding(const ScopePtr& scope if (!prop) { reportError(TypeError{expr.location, UnknownProperty{lhs, name}}); - return std::pair(errorRecoveryType(scope), nullptr); + return errorRecoveryType(scope); } - return std::pair(prop->type, nullptr); + return prop->type; } else if (get(lhs)) { if (std::optional ty = getIndexTypeFromType(scope, lhs, name, expr.location, false)) - return std::pair(*ty, nullptr); + return *ty; // If intersection has a table part, report that it cannot be extended just as a sealed table if (isTableIntersection(lhs)) { reportError(TypeError{expr.location, CannotExtendTable{lhs, CannotExtendTable::Property, name}}); - return std::pair(errorRecoveryType(scope), nullptr); + return errorRecoveryType(scope); } } reportError(TypeError{expr.location, NotATable{lhs}}); - return std::pair(errorRecoveryType(scope), nullptr); + return errorRecoveryType(scope); } -std::pair TypeChecker::checkLValueBinding(const ScopePtr& scope, const AstExprIndexExpr& expr) +TypeId TypeChecker::checkLValueBinding(const ScopePtr& scope, const AstExprIndexExpr& expr) { TypeId exprType = checkExpr(scope, *expr.expr).type; tablify(exprType); @@ -2771,7 +2753,7 @@ std::pair TypeChecker::checkLValueBinding(const ScopePtr& scope TypeId indexType = checkExpr(scope, *expr.index).type; if (get(exprType) || get(exprType)) - return std::pair(exprType, nullptr); + return exprType; AstExprConstantString* value = expr.index->as(); @@ -2783,9 +2765,9 @@ std::pair TypeChecker::checkLValueBinding(const ScopePtr& scope if (!prop) { reportError(TypeError{expr.location, UnknownProperty{exprType, value->value.data}}); - return std::pair(errorRecoveryType(scope), nullptr); + return errorRecoveryType(scope); } - return std::pair(prop->type, nullptr); + return prop->type; } } @@ -2794,7 +2776,7 @@ std::pair TypeChecker::checkLValueBinding(const ScopePtr& scope if (!exprTable) { reportError(TypeError{expr.expr->location, NotATable{exprType}}); - return std::pair(errorRecoveryType(scope), nullptr); + return errorRecoveryType(scope); } if (value) @@ -2802,7 +2784,7 @@ std::pair TypeChecker::checkLValueBinding(const ScopePtr& scope const auto& it = exprTable->props.find(value->value.data); if (it != exprTable->props.end()) { - return std::pair(it->second.type, &it->second.type); + return it->second.type; } else if (exprTable->state == TableState::Unsealed || exprTable->state == TableState::Free) { @@ -2810,7 +2792,7 @@ std::pair TypeChecker::checkLValueBinding(const ScopePtr& scope Property& property = exprTable->props[value->value.data]; property.type = resultType; property.location = expr.index->location; - return std::pair(resultType, &property.type); + return resultType; } } @@ -2818,18 +2800,18 @@ std::pair TypeChecker::checkLValueBinding(const ScopePtr& scope { const TableIndexer& indexer = *exprTable->indexer; unify(indexType, indexer.indexType, expr.index->location); - return std::pair(indexer.indexResultType, nullptr); + return indexer.indexResultType; } else if (exprTable->state == TableState::Unsealed || exprTable->state == TableState::Free) { TypeId resultType = freshType(scope); exprTable->indexer = TableIndexer{anyIfNonstrict(indexType), anyIfNonstrict(resultType)}; - return std::pair(resultType, nullptr); + return resultType; } else { TypeId resultType = freshType(scope); - return std::pair(resultType, nullptr); + return resultType; } } @@ -3326,7 +3308,7 @@ void TypeChecker::checkArgumentList( } // ok else { - state.errors.push_back(TypeError{state.location, CountMismatch{minParams, paramIndex}}); + state.reportError(TypeError{state.location, CountMismatch{minParams, paramIndex}}); return; } ++paramIter; @@ -3348,7 +3330,7 @@ void TypeChecker::checkArgumentList( Location location = state.location; if (!argLocations.empty()) location = {state.location.begin, argLocations.back().end}; - state.errors.push_back(TypeError{location, CountMismatch{minParams, std::distance(begin(argPack), end(argPack))}}); + state.reportError(TypeError{location, CountMismatch{minParams, std::distance(begin(argPack), end(argPack))}}); return; } TypePackId tail = state.log.follow(*paramIter.tail()); @@ -3405,7 +3387,7 @@ void TypeChecker::checkArgumentList( if (!argLocations.empty()) location = {state.location.begin, argLocations.back().end}; // TODO: Better error message? - state.errors.push_back(TypeError{location, CountMismatch{minParams, std::distance(begin(argPack), end(argPack))}}); + state.reportError(TypeError{location, CountMismatch{minParams, std::distance(begin(argPack), end(argPack))}}); return; } } @@ -3520,7 +3502,7 @@ void TypeChecker::checkArgumentList( } // ok else { - state.errors.push_back(TypeError{state.location, CountMismatch{minParams, paramIndex}}); + state.reportError(TypeError{state.location, CountMismatch{minParams, paramIndex}}); return; } ++paramIter; @@ -3540,7 +3522,7 @@ void TypeChecker::checkArgumentList( Location location = state.location; if (!argLocations.empty()) location = {state.location.begin, argLocations.back().end}; - state.errors.push_back(TypeError{location, CountMismatch{minParams, std::distance(begin(argPack), end(argPack))}}); + state.reportError(TypeError{location, CountMismatch{minParams, std::distance(begin(argPack), end(argPack))}}); return; } TypePackId tail = *paramIter.tail(); @@ -3606,7 +3588,7 @@ void TypeChecker::checkArgumentList( if (!argLocations.empty()) location = {state.location.begin, argLocations.back().end}; // TODO: Better error message? - state.errors.push_back(TypeError{location, CountMismatch{minParams, std::distance(begin(argPack), end(argPack))}}); + state.reportError(TypeError{location, CountMismatch{minParams, std::distance(begin(argPack), end(argPack))}}); return; } } @@ -3825,22 +3807,11 @@ std::optional> TypeChecker::checkCallOverload(const Scope metaArgLocations = *argLocations; metaArgLocations.insert(metaArgLocations.begin(), expr.func->location); - if (FFlag::LuauFixRecursiveMetatableCall) - { - fn = instantiate(scope, *ty, expr.func->location); + fn = instantiate(scope, *ty, expr.func->location); - argPack = metaCallArgPack; - args = metaCallArgs; - argLocations = &metaArgLocations; - } - else - { - TypeId fn = *ty; - fn = instantiate(scope, fn, expr.func->location); - - return checkCallOverload(scope, expr, fn, retPack, metaCallArgPack, metaCallArgs, &metaArgLocations, argListResult, - overloadsThatMatchArgCount, overloadsThatDont, errors); - } + argPack = metaCallArgPack; + args = metaCallArgs; + argLocations = &metaArgLocations; } } @@ -3932,8 +3903,7 @@ std::optional> TypeChecker::checkCallOverload(const Scope } } - if (FFlag::LuauStoreMatchingOverloadFnType) - currentModule->astOverloadResolvedTypes[&expr] = fn; + currentModule->astOverloadResolvedTypes[&expr] = fn; // We select this overload return {{retPack}}; @@ -4776,7 +4746,7 @@ TypeId TypeChecker::freshType(TypeLevel level) TypeId TypeChecker::singletonType(bool value) { // TODO: cache singleton types - return currentModule->internalTypes.addType(TypeVar(SingletonTypeVar(BoolSingleton{value}))); + return currentModule->internalTypes.addType(TypeVar(SingletonTypeVar(BooleanSingleton{value}))); } TypeId TypeChecker::singletonType(std::string value) @@ -4813,13 +4783,6 @@ std::optional TypeChecker::filterMap(TypeId type, TypeIdPredicate predic return std::nullopt; } -TypeId TypeChecker::addType(const UnionTypeVar& utv) -{ - LUAU_ASSERT(utv.options.size() > 1); - - return addTV(TypeVar(utv)); -} - TypeId TypeChecker::addTV(TypeVar&& tv) { return currentModule->internalTypes.addType(std::move(tv)); @@ -5347,54 +5310,35 @@ TypeId TypeChecker::instantiateTypeFun(const ScopePtr& scope, const TypeFun& tf, TypeId instantiated = *maybeInstantiated; - if (FFlag::LuauCloneCorrectlyBeforeMutatingTableType) + // TODO: CLI-46926 it's not a good idea to rename the type here + TypeId target = follow(instantiated); + bool needsClone = follow(tf.type) == target; + TableTypeVar* ttv = getMutableTableType(target); + + if (ttv && needsClone) { - // TODO: CLI-46926 it's not a good idea to rename the type here - TypeId target = follow(instantiated); - bool needsClone = follow(tf.type) == target; - TableTypeVar* ttv = getMutableTableType(target); - - if (ttv && needsClone) + // Substitution::clone is a shallow clone. If this is a metatable type, we + // want to mutate its table, so we need to explicitly clone that table as + // well. If we don't, we will mutate another module's type surface and cause + // a use-after-free. + if (get(target)) { - // Substitution::clone is a shallow clone. If this is a metatable type, we - // want to mutate its table, so we need to explicitly clone that table as - // well. If we don't, we will mutate another module's type surface and cause - // a use-after-free. - if (get(target)) - { - instantiated = applyTypeFunction.clone(tf.type); - MetatableTypeVar* mtv = getMutable(instantiated); - mtv->table = applyTypeFunction.clone(mtv->table); - ttv = getMutable(mtv->table); - } - if (get(target)) - { - instantiated = applyTypeFunction.clone(tf.type); - ttv = getMutable(instantiated); - } + instantiated = applyTypeFunction.clone(tf.type); + MetatableTypeVar* mtv = getMutable(instantiated); + mtv->table = applyTypeFunction.clone(mtv->table); + ttv = getMutable(mtv->table); } - - if (ttv) + if (get(target)) { - ttv->instantiatedTypeParams = typeParams; - ttv->instantiatedTypePackParams = typePackParams; + instantiated = applyTypeFunction.clone(tf.type); + ttv = getMutable(instantiated); } } - else - { - if (TableTypeVar* ttv = getMutableTableType(instantiated)) - { - if (follow(tf.type) == instantiated) - { - // This can happen if a type alias has generics that it does not use at all. - // ex type FooBar = { a: number } - instantiated = applyTypeFunction.clone(tf.type); - ttv = getMutableTableType(instantiated); - } - ttv->instantiatedTypeParams = typeParams; - ttv->instantiatedTypePackParams = typePackParams; - } + if (ttv) + { + ttv->instantiatedTypeParams = typeParams; + ttv->instantiatedTypePackParams = typePackParams; } return instantiated; @@ -5482,6 +5426,85 @@ GenericTypeDefinitions TypeChecker::createGenericTypes(const ScopePtr& scope, st return {generics, genericPacks}; } +void TypeChecker::refineLValue(const LValue& lvalue, RefinementMap& refis, const ScopePtr& scope, TypeIdPredicate predicate) +{ + LUAU_ASSERT(FFlag::LuauDiscriminableUnions); + + const LValue* target = &lvalue; + std::optional key; // If set, we know we took the base of the lvalue path and should be walking down each option of the base's type. + + auto ty = resolveLValue(scope, *target); + if (!ty) + return; // Do nothing. An error was already reported. + + // If the provided lvalue is a local or global, then that's without a doubt the target. + // However, if there is a base lvalue, then we'll want that to be the target iff the base is a union type. + if (auto base = baseof(lvalue)) + { + std::optional baseTy = resolveLValue(scope, *base); + if (baseTy && get(follow(*baseTy))) + { + ty = baseTy; + target = base; + key = lvalue; + } + } + + // If we do not have a key, it means we're not trying to discriminate anything, so it's a simple matter of just filtering for a subset. + if (!key) + { + if (std::optional result = filterMap(*ty, predicate)) + addRefinement(refis, *target, *result); + else + addRefinement(refis, *target, errorRecoveryType(scope)); + + return; + } + + // Otherwise, we'll want to walk each option of ty, get its index type, and filter that. + auto utv = get(follow(*ty)); + LUAU_ASSERT(utv); + + std::unordered_set viableTargetOptions; + std::unordered_set viableChildOptions; // There may be additional refinements that apply. We add those here too. + + for (TypeId option : utv) + { + std::optional discriminantTy; + if (auto field = Luau::get(*key)) // need to fully qualify Luau::get because of ADL. + discriminantTy = getIndexTypeFromType(scope, option, field->key, Location(), false); + else + LUAU_ASSERT(!"Unhandled LValue alternative?"); + + if (!discriminantTy) + return; // Do nothing. An error was already reported, as per usual. + + if (std::optional result = filterMap(*discriminantTy, predicate)) + { + viableTargetOptions.insert(option); + viableChildOptions.insert(*result); + } + } + + auto intoType = [this](const std::unordered_set& s) -> std::optional { + if (s.empty()) + return std::nullopt; + + // TODO: allocate UnionTypeVar and just normalize. + std::vector options(s.begin(), s.end()); + if (options.size() == 1) + return options[0]; + + return addType(UnionTypeVar{std::move(options)}); + }; + + if (std::optional viableTargetType = intoType(viableTargetOptions)) + addRefinement(refis, *target, *viableTargetType); + + if (std::optional viableChildType = intoType(viableChildOptions)) + addRefinement(refis, lvalue, *viableChildType); +} + std::optional TypeChecker::resolveLValue(const ScopePtr& scope, const LValue& lvalue) { if (!FFlag::LuauLValueAsKey) @@ -5645,18 +5668,29 @@ void TypeChecker::resolve(const TruthyPredicate& truthyP, ErrorVec& errVec, Refi return std::nullopt; }; - std::optional ty = resolveLValue(refis, scope, truthyP.lvalue); - if (!ty) - return; + if (FFlag::LuauDiscriminableUnions) + { + std::optional ty = resolveLValue(refis, scope, truthyP.lvalue); + if (ty && fromOr) + return addRefinement(refis, truthyP.lvalue, *ty); - // This is a hack. :( - // Without this, the expression 'a or b' might refine 'b' to be falsy. - // I'm not yet sure how else to get this to do the right thing without this hack, so we'll do this for now in the meantime. - if (fromOr) - return addRefinement(refis, truthyP.lvalue, *ty); + refineLValue(truthyP.lvalue, refis, scope, predicate); + } + else + { + std::optional ty = resolveLValue(refis, scope, truthyP.lvalue); + if (!ty) + return; - if (std::optional result = filterMap(*ty, predicate)) - addRefinement(refis, truthyP.lvalue, *result); + // This is a hack. :( + // Without this, the expression 'a or b' might refine 'b' to be falsy. + // I'm not yet sure how else to get this to do the right thing without this hack, so we'll do this for now in the meantime. + if (fromOr) + return addRefinement(refis, truthyP.lvalue, *ty); + + if (std::optional result = filterMap(*ty, predicate)) + addRefinement(refis, truthyP.lvalue, *result); + } } void TypeChecker::resolve(const AndPredicate& andP, ErrorVec& errVec, RefinementMap& refis, const ScopePtr& scope, bool sense) @@ -5746,16 +5780,23 @@ void TypeChecker::resolve(const IsAPredicate& isaP, ErrorVec& errVec, Refinement return res; }; - std::optional ty = resolveLValue(refis, scope, isaP.lvalue); - if (!ty) - return; - - if (std::optional result = filterMap(*ty, predicate)) - addRefinement(refis, isaP.lvalue, *result); + if (FFlag::LuauDiscriminableUnions) + { + refineLValue(isaP.lvalue, refis, scope, predicate); + } else { - addRefinement(refis, isaP.lvalue, errorRecoveryType(scope)); - errVec.push_back(TypeError{isaP.location, TypeMismatch{isaP.ty, *ty}}); + std::optional ty = resolveLValue(refis, scope, isaP.lvalue); + if (!ty) + return; + + if (std::optional result = filterMap(*ty, predicate)) + addRefinement(refis, isaP.lvalue, *result); + else + { + addRefinement(refis, isaP.lvalue, errorRecoveryType(scope)); + errVec.push_back(TypeError{isaP.location, TypeMismatch{isaP.ty, *ty}}); + } } } @@ -5814,21 +5855,30 @@ void TypeChecker::resolve(const TypeGuardPredicate& typeguardP, ErrorVec& errVec if (auto it = primitives.find(typeguardP.kind); it != primitives.end()) { - if (std::optional result = filterMap(*ty, it->second(sense))) - addRefinement(refis, typeguardP.lvalue, *result); + if (FFlag::LuauDiscriminableUnions) + { + refineLValue(typeguardP.lvalue, refis, scope, it->second(sense)); + return; + } else { - addRefinement(refis, typeguardP.lvalue, errorRecoveryType(scope)); - if (sense) - errVec.push_back( - TypeError{typeguardP.location, GenericError{"Type '" + toString(*ty) + "' has no overlap with '" + typeguardP.kind + "'"}}); - } + if (std::optional result = filterMap(*ty, it->second(sense))) + addRefinement(refis, typeguardP.lvalue, *result); + else + { + addRefinement(refis, typeguardP.lvalue, errorRecoveryType(scope)); + if (sense) + errVec.push_back( + TypeError{typeguardP.location, GenericError{"Type '" + toString(*ty) + "' has no overlap with '" + typeguardP.kind + "'"}}); + } - return; + return; + } } auto fail = [&](const TypeErrorData& err) { - errVec.push_back(TypeError{typeguardP.location, err}); + if (!FFlag::LuauDiscriminableUnions) + errVec.push_back(TypeError{typeguardP.location, err}); addRefinement(refis, typeguardP.lvalue, errorRecoveryType(scope)); }; @@ -5853,55 +5903,85 @@ void TypeChecker::resolve(const TypeGuardPredicate& typeguardP, ErrorVec& errVec void TypeChecker::resolve(const EqPredicate& eqP, ErrorVec& errVec, RefinementMap& refis, const ScopePtr& scope, bool sense) { // This refinement will require success typing to do everything correctly. For now, we can get most of the way there. - auto options = [](TypeId ty) -> std::vector { if (auto utv = get(follow(ty))) return std::vector(begin(utv), end(utv)); return {ty}; }; - if (FFlag::LuauWeakEqConstraint) + if (FFlag::LuauDiscriminableUnions) { - if (!sense && isNil(eqP.type)) - resolve(TruthyPredicate{std::move(eqP.lvalue), eqP.location}, errVec, refis, scope, true, /* fromOr= */ false); - - return; - } - - if (FFlag::LuauEqConstraint) - { - std::optional ty = resolveLValue(refis, scope, eqP.lvalue); - if (!ty) - return; - - std::vector lhs = options(*ty); std::vector rhs = options(eqP.type); - if (sense && std::any_of(lhs.begin(), lhs.end(), isUndecidable)) - { - addRefinement(refis, eqP.lvalue, eqP.type); - return; - } - else if (sense && std::any_of(rhs.begin(), rhs.end(), isUndecidable)) + if (sense && std::any_of(rhs.begin(), rhs.end(), isUndecidable)) return; // Optimization: the other side has unknown types, so there's probably an overlap. Refining is no-op here. - std::unordered_set set; - for (TypeId left : lhs) - { - for (TypeId right : rhs) + auto predicate = [&](TypeId option) -> std::optional { + if (sense && isUndecidable(option)) + return FFlag::LuauWeakEqConstraint ? option : eqP.type; + + if (!sense && isNil(eqP.type)) + return (isUndecidable(option) || !isNil(option)) ? std::optional(option) : std::nullopt; + + if (maybeSingleton(eqP.type)) { - // When singleton types arrive, `isNil` here probably should be replaced with `isLiteral`. - if (canUnify(right, left, eqP.location).empty() == sense || (!sense && !isNil(left))) - set.insert(left); + // Normally we'd write option <: eqP.type, but singletons are always the subtype, so we flip this. + if (!sense || canUnify(eqP.type, option, eqP.location).empty()) + return sense ? eqP.type : option; + + return std::nullopt; } + + return option; + }; + + refineLValue(eqP.lvalue, refis, scope, predicate); + } + else + { + if (FFlag::LuauWeakEqConstraint) + { + if (!sense && isNil(eqP.type)) + resolve(TruthyPredicate{std::move(eqP.lvalue), eqP.location}, errVec, refis, scope, true, /* fromOr= */ false); + + return; } - if (set.empty()) - return; + if (FFlag::LuauEqConstraint) + { + std::optional ty = resolveLValue(refis, scope, eqP.lvalue); + if (!ty) + return; - std::vector viable(set.begin(), set.end()); - TypeId result = viable.size() == 1 ? viable[0] : addType(UnionTypeVar{std::move(viable)}); - addRefinement(refis, eqP.lvalue, result); + std::vector lhs = options(*ty); + std::vector rhs = options(eqP.type); + + if (sense && std::any_of(lhs.begin(), lhs.end(), isUndecidable)) + { + addRefinement(refis, eqP.lvalue, eqP.type); + return; + } + else if (sense && std::any_of(rhs.begin(), rhs.end(), isUndecidable)) + return; // Optimization: the other side has unknown types, so there's probably an overlap. Refining is no-op here. + + std::unordered_set set; + for (TypeId left : lhs) + { + for (TypeId right : rhs) + { + // When singleton types arrive, `isNil` here probably should be replaced with `isLiteral`. + if (canUnify(right, left, eqP.location).empty() == sense || (!sense && !isNil(left))) + set.insert(left); + } + } + + if (set.empty()) + return; + + std::vector viable(set.begin(), set.end()); + TypeId result = viable.size() == 1 ? viable[0] : addType(UnionTypeVar{std::move(viable)}); + addRefinement(refis, eqP.lvalue, result); + } } } diff --git a/Analysis/src/TypeVar.cpp b/Analysis/src/TypeVar.cpp index df5d76ed..5b162b31 100644 --- a/Analysis/src/TypeVar.cpp +++ b/Analysis/src/TypeVar.cpp @@ -18,14 +18,15 @@ #include #include +LUAU_FASTFLAG(DebugLuauFreezeArena) + LUAU_FASTINTVARIABLE(LuauTypeMaximumStringifierLength, 500) LUAU_FASTINTVARIABLE(LuauTableTypeMaximumStringifierLength, 0) LUAU_FASTINT(LuauTypeInferRecursionLimit) LUAU_FASTFLAG(LuauLengthOnCompositeType) LUAU_FASTFLAGVARIABLE(LuauMetatableAreEqualRecursion, false) -LUAU_FASTFLAGVARIABLE(LuauRefactorTagging, false) +LUAU_FASTFLAGVARIABLE(LuauRefactorTypeVarQuestions, false) LUAU_FASTFLAG(LuauErrorRecoveryType) -LUAU_FASTFLAG(DebugLuauFreezeArena) namespace Luau { @@ -144,7 +145,20 @@ bool isNil(TypeId ty) bool isBoolean(TypeId ty) { - return isPrim(ty, PrimitiveTypeVar::Boolean); + if (FFlag::LuauRefactorTypeVarQuestions) + { + if (isPrim(ty, PrimitiveTypeVar::Boolean) || get(get(follow(ty)))) + return true; + + if (auto utv = get(follow(ty))) + return std::all_of(begin(utv), end(utv), isBoolean); + + return false; + } + else + { + return isPrim(ty, PrimitiveTypeVar::Boolean); + } } bool isNumber(TypeId ty) @@ -154,7 +168,20 @@ bool isNumber(TypeId ty) bool isString(TypeId ty) { - return isPrim(ty, PrimitiveTypeVar::String); + if (FFlag::LuauRefactorTypeVarQuestions) + { + if (isPrim(ty, PrimitiveTypeVar::String) || get(get(follow(ty)))) + return true; + + if (auto utv = get(follow(ty))) + return std::all_of(begin(utv), end(utv), isString); + + return false; + } + else + { + return isPrim(ty, PrimitiveTypeVar::String); + } } bool isThread(TypeId ty) @@ -167,37 +194,45 @@ bool isOptional(TypeId ty) if (isNil(ty)) return true; - if (!get(follow(ty))) - return false; - - std::unordered_set seen; - std::deque queue{ty}; - while (!queue.empty()) + if (FFlag::LuauRefactorTypeVarQuestions) { - TypeId current = follow(queue.front()); - queue.pop_front(); + auto utv = get(follow(ty)); + if (!utv) + return false; - if (seen.count(current)) - continue; - - seen.insert(current); - - if (isNil(current)) - return true; - - if (auto u = get(current)) + return std::any_of(begin(utv), end(utv), isNil); + } + else + { + std::unordered_set seen; + std::deque queue{ty}; + while (!queue.empty()) { - for (TypeId option : u->options) - { - if (isNil(option)) - return true; + TypeId current = follow(queue.front()); + queue.pop_front(); - queue.push_back(option); + if (seen.count(current)) + continue; + + seen.insert(current); + + if (isNil(current)) + return true; + + if (auto u = get(current)) + { + for (TypeId option : u->options) + { + if (isNil(option)) + return true; + + queue.push_back(option); + } } } - } - return false; + return false; + } } bool isTableIntersection(TypeId ty) @@ -228,13 +263,27 @@ std::optional getMetatable(TypeId type) return mtType->metatable; else if (const ClassTypeVar* classType = get(type)) return classType->metatable; - else if (const PrimitiveTypeVar* primitiveType = get(type); primitiveType && primitiveType->metatable) + else if (FFlag::LuauRefactorTypeVarQuestions) { - LUAU_ASSERT(primitiveType->type == PrimitiveTypeVar::String); - return primitiveType->metatable; + if (isString(type)) + { + auto ptv = get(getSingletonTypes().stringType); + LUAU_ASSERT(ptv && ptv->metatable); + return ptv->metatable; + } + else + return std::nullopt; } else - return std::nullopt; + { + if (const PrimitiveTypeVar* primitiveType = get(type); primitiveType && primitiveType->metatable) + { + LUAU_ASSERT(primitiveType->type == PrimitiveTypeVar::String); + return primitiveType->metatable; + } + else + return std::nullopt; + } } const TableTypeVar* getTableType(TypeId type) @@ -696,7 +745,7 @@ TypeId SingletonTypes::makeStringMetatable() {"reverse", {stringToStringType}}, {"sub", {makeFunction(*arena, stringType, {}, {}, {numberType, optionalNumber}, {}, {stringType})}}, {"upper", {stringToStringType}}, - {"split", {makeFunction(*arena, stringType, {}, {}, {stringType, optionalString}, {}, + {"split", {makeFunction(*arena, stringType, {}, {}, {optionalString}, {}, {arena->addType(TableTypeVar{{}, TableIndexer{numberType, stringType}, TypeLevel{}})})}}, {"pack", {arena->addType(FunctionTypeVar{ arena->addTypePack(TypePack{{stringType}, anyTypePack}), @@ -1108,30 +1157,14 @@ static Tags* getTags(TypeId ty) void attachTag(TypeId ty, const std::string& tagName) { - if (!FFlag::LuauRefactorTagging) - { - if (auto ftv = getMutable(ty)) - { - ftv->tags.emplace_back(tagName); - } - else - { - LUAU_ASSERT(!"Got a non functional type"); - } - } + if (auto tags = getTags(ty)) + tags->push_back(tagName); else - { - if (auto tags = getTags(ty)) - tags->push_back(tagName); - else - LUAU_ASSERT(!"This TypeId does not support tags"); - } + LUAU_ASSERT(!"This TypeId does not support tags"); } void attachTag(Property& prop, const std::string& tagName) { - LUAU_ASSERT(FFlag::LuauRefactorTagging); - prop.tags.push_back(tagName); } @@ -1140,7 +1173,6 @@ void attachTag(Property& prop, const std::string& tagName) // Unfortunately, there's already use cases that's hard to disentangle. For now, we expose it. bool hasTag(const Tags& tags, const std::string& tagName) { - LUAU_ASSERT(FFlag::LuauRefactorTagging); return std::find(tags.begin(), tags.end(), tagName) != tags.end(); } diff --git a/Analysis/src/Unifier.cpp b/Analysis/src/Unifier.cpp index 2bd9cf83..17d9bf58 100644 --- a/Analysis/src/Unifier.cpp +++ b/Analysis/src/Unifier.cpp @@ -17,15 +17,11 @@ LUAU_FASTFLAGVARIABLE(LuauCommittingTxnLogFreeTpPromote, false) LUAU_FASTFLAG(LuauUseCommittingTxnLog) LUAU_FASTINTVARIABLE(LuauTypeInferIterationLimit, 2000); LUAU_FASTFLAGVARIABLE(LuauTableSubtypingVariance2, false); -LUAU_FASTFLAGVARIABLE(LuauUnionHeuristic, false) LUAU_FASTFLAGVARIABLE(LuauTableUnificationEarlyTest, false) -LUAU_FASTFLAGVARIABLE(LuauOccursCheckOkWithRecursiveFunctions, false) LUAU_FASTFLAG(LuauSingletonTypes) LUAU_FASTFLAG(LuauErrorRecoveryType); LUAU_FASTFLAG(LuauProperTypeLevels); LUAU_FASTFLAGVARIABLE(LuauUnifyPackTails, false) -LUAU_FASTFLAGVARIABLE(LuauExtendedUnionMismatchError, false) -LUAU_FASTFLAGVARIABLE(LuauExtendedFunctionMismatchError, false) namespace Luau { @@ -229,8 +225,6 @@ static std::optional hasUnificationTooComplex(const ErrorVec& errors) // Used for tagged union matching heuristic, returns first singleton type field static std::optional> getTableMatchTag(TypeId type) { - LUAU_ASSERT(FFlag::LuauExtendedUnionMismatchError); - type = follow(type); if (auto ttv = get(type)) @@ -291,7 +285,7 @@ void Unifier::tryUnify_(TypeId subTy, TypeId superTy, bool isFunctionCall, bool if (FInt::LuauTypeInferIterationLimit > 0 && FInt::LuauTypeInferIterationLimit < sharedState.counters.iterationCount) { - errors.push_back(TypeError{location, UnificationTooComplex{}}); + reportError(TypeError{location, UnificationTooComplex{}}); return; } @@ -403,7 +397,7 @@ void Unifier::tryUnify_(TypeId subTy, TypeId superTy, bool isFunctionCall, bool if (subGeneric && !subGeneric->level.subsumes(superLevel)) { // TODO: a more informative error message? CLI-39912 - errors.push_back(TypeError{location, GenericError{"Generic subtype escaping scope"}}); + reportError(TypeError{location, GenericError{"Generic subtype escaping scope"}}); return; } @@ -448,7 +442,7 @@ void Unifier::tryUnify_(TypeId subTy, TypeId superTy, bool isFunctionCall, bool if (superGeneric && !superGeneric->level.subsumes(subFree->level)) { // TODO: a more informative error message? CLI-39912 - errors.push_back(TypeError{location, GenericError{"Generic supertype escaping scope"}}); + reportError(TypeError{location, GenericError{"Generic supertype escaping scope"}}); return; } @@ -561,13 +555,13 @@ void Unifier::tryUnify_(TypeId subTy, TypeId superTy, bool isFunctionCall, bool } if (unificationTooComplex) - errors.push_back(*unificationTooComplex); + reportError(*unificationTooComplex); else if (failed) { if (firstFailedOption) - errors.push_back(TypeError{location, TypeMismatch{superTy, subTy, "Not all union options are compatible.", *firstFailedOption}}); + reportError(TypeError{location, TypeMismatch{superTy, subTy, "Not all union options are compatible.", *firstFailedOption}}); else - errors.push_back(TypeError{location, TypeMismatch{superTy, subTy}}); + reportError(TypeError{location, TypeMismatch{superTy, subTy}}); } } else if (const UnionTypeVar* uv = FFlag::LuauUseCommittingTxnLog ? log.getMutable(superTy) : get(superTy)) @@ -582,50 +576,44 @@ void Unifier::tryUnify_(TypeId subTy, TypeId superTy, bool isFunctionCall, bool bool foundHeuristic = false; size_t startIndex = 0; - if (FFlag::LuauUnionHeuristic) + if (const std::string* subName = getName(subTy)) { - if (const std::string* subName = getName(subTy)) + for (size_t i = 0; i < uv->options.size(); ++i) { - for (size_t i = 0; i < uv->options.size(); ++i) + const std::string* optionName = getName(uv->options[i]); + if (optionName && *optionName == *subName) { - const std::string* optionName = getName(uv->options[i]); - if (optionName && *optionName == *subName) - { - foundHeuristic = true; - startIndex = i; - break; - } + foundHeuristic = true; + startIndex = i; + break; } } + } - if (FFlag::LuauExtendedUnionMismatchError) + if (auto subMatchTag = getTableMatchTag(subTy)) + { + for (size_t i = 0; i < uv->options.size(); ++i) { - if (auto subMatchTag = getTableMatchTag(subTy)) + auto optionMatchTag = getTableMatchTag(uv->options[i]); + if (optionMatchTag && optionMatchTag->first == subMatchTag->first && *optionMatchTag->second == *subMatchTag->second) { - for (size_t i = 0; i < uv->options.size(); ++i) - { - auto optionMatchTag = getTableMatchTag(uv->options[i]); - if (optionMatchTag && optionMatchTag->first == subMatchTag->first && *optionMatchTag->second == *subMatchTag->second) - { - foundHeuristic = true; - startIndex = i; - break; - } - } + foundHeuristic = true; + startIndex = i; + break; } } + } - if (!foundHeuristic && cacheEnabled) + if (!foundHeuristic && cacheEnabled) + { + for (size_t i = 0; i < uv->options.size(); ++i) { - for (size_t i = 0; i < uv->options.size(); ++i) - { - TypeId type = uv->options[i]; + TypeId type = uv->options[i]; - if (cache.contains({type, subTy}) && (variance == Covariant || cache.contains({subTy, type}))) - { - startIndex = i; - break; - } + if (cache.contains({type, subTy}) && (variance == Covariant || cache.contains({subTy, type}))) + { + startIndex = i; + break; } } } @@ -650,7 +638,7 @@ void Unifier::tryUnify_(TypeId subTy, TypeId superTy, bool isFunctionCall, bool { unificationTooComplex = e; } - else if (FFlag::LuauExtendedUnionMismatchError && !isNil(type)) + else if (!isNil(type)) { failedOptionCount++; @@ -664,15 +652,15 @@ void Unifier::tryUnify_(TypeId subTy, TypeId superTy, bool isFunctionCall, bool if (unificationTooComplex) { - errors.push_back(*unificationTooComplex); + reportError(*unificationTooComplex); } else if (!found) { - if (FFlag::LuauExtendedUnionMismatchError && (failedOptionCount == 1 || foundHeuristic) && failedOption) - errors.push_back( + if ((failedOptionCount == 1 || foundHeuristic) && failedOption) + reportError( TypeError{location, TypeMismatch{superTy, subTy, "None of the union options are compatible. For example:", *failedOption}}); else - errors.push_back(TypeError{location, TypeMismatch{superTy, subTy, "none of the union options are compatible"}}); + reportError(TypeError{location, TypeMismatch{superTy, subTy, "none of the union options are compatible"}}); } } else if (const IntersectionTypeVar* uv = @@ -702,9 +690,9 @@ void Unifier::tryUnify_(TypeId subTy, TypeId superTy, bool isFunctionCall, bool } if (unificationTooComplex) - errors.push_back(*unificationTooComplex); + reportError(*unificationTooComplex); else if (firstFailedOption) - errors.push_back(TypeError{location, TypeMismatch{superTy, subTy, "Not all intersection parts are compatible.", *firstFailedOption}}); + reportError(TypeError{location, TypeMismatch{superTy, subTy, "Not all intersection parts are compatible.", *firstFailedOption}}); } else if (const IntersectionTypeVar* uv = FFlag::LuauUseCommittingTxnLog ? log.getMutable(subTy) : get(subTy)) @@ -754,10 +742,10 @@ void Unifier::tryUnify_(TypeId subTy, TypeId superTy, bool isFunctionCall, bool } if (unificationTooComplex) - errors.push_back(*unificationTooComplex); + reportError(*unificationTooComplex); else if (!found) { - errors.push_back(TypeError{location, TypeMismatch{superTy, subTy, "none of the intersection parts are compatible"}}); + reportError(TypeError{location, TypeMismatch{superTy, subTy, "none of the intersection parts are compatible"}}); } } else if ((FFlag::LuauUseCommittingTxnLog && log.getMutable(superTy) && log.getMutable(subTy)) || @@ -801,7 +789,7 @@ void Unifier::tryUnify_(TypeId subTy, TypeId superTy, bool isFunctionCall, bool tryUnifyWithClass(subTy, superTy, /*reversed*/ true); else - errors.push_back(TypeError{location, TypeMismatch{superTy, subTy}}); + reportError(TypeError{location, TypeMismatch{superTy, subTy}}); if (FFlag::LuauUseCommittingTxnLog) log.popSeen(superTy, subTy); @@ -1067,7 +1055,7 @@ void Unifier::tryUnify_(TypePackId subTp, TypePackId superTp, bool isFunctionCal if (FInt::LuauTypeInferIterationLimit > 0 && FInt::LuauTypeInferIterationLimit < sharedState.counters.iterationCount) { - errors.push_back(TypeError{location, UnificationTooComplex{}}); + reportError(TypeError{location, UnificationTooComplex{}}); return; } @@ -1166,7 +1154,7 @@ void Unifier::tryUnify_(TypePackId subTp, TypePackId superTp, bool isFunctionCal { tryUnify_(*subIter, *superIter); - if (FFlag::LuauExtendedFunctionMismatchError && !errors.empty() && !firstPackErrorPos) + if (!errors.empty() && !firstPackErrorPos) firstPackErrorPos = loopCount; superIter.advance(); @@ -1251,7 +1239,7 @@ void Unifier::tryUnify_(TypePackId subTp, TypePackId superTp, bool isFunctionCal size_t actualSize = size(subTp); if (ctx == CountMismatch::Result) std::swap(expectedSize, actualSize); - errors.push_back(TypeError{location, CountMismatch{expectedSize, actualSize, ctx}}); + reportError(TypeError{location, CountMismatch{expectedSize, actualSize, ctx}}); while (superIter.good()) { @@ -1272,7 +1260,7 @@ void Unifier::tryUnify_(TypePackId subTp, TypePackId superTp, bool isFunctionCal } else { - errors.push_back(TypeError{location, GenericError{"Failed to unify type packs"}}); + reportError(TypeError{location, GenericError{"Failed to unify type packs"}}); } } else @@ -1372,7 +1360,7 @@ void Unifier::tryUnify_(TypePackId subTp, TypePackId superTp, bool isFunctionCal { tryUnify_(*subIter, *superIter); - if (FFlag::LuauExtendedFunctionMismatchError && !errors.empty() && !firstPackErrorPos) + if (!errors.empty() && !firstPackErrorPos) firstPackErrorPos = loopCount; superIter.advance(); @@ -1459,7 +1447,7 @@ void Unifier::tryUnify_(TypePackId subTp, TypePackId superTp, bool isFunctionCal size_t actualSize = size(subTp); if (ctx == CountMismatch::Result) std::swap(expectedSize, actualSize); - errors.push_back(TypeError{location, CountMismatch{expectedSize, actualSize, ctx}}); + reportError(TypeError{location, CountMismatch{expectedSize, actualSize, ctx}}); while (superIter.good()) { @@ -1480,7 +1468,7 @@ void Unifier::tryUnify_(TypePackId subTp, TypePackId superTp, bool isFunctionCal } else { - errors.push_back(TypeError{location, GenericError{"Failed to unify type packs"}}); + reportError(TypeError{location, GenericError{"Failed to unify type packs"}}); } } } @@ -1493,7 +1481,7 @@ void Unifier::tryUnifyPrimitives(TypeId subTy, TypeId superTy) ice("passed non primitive types to unifyPrimitives"); if (superPrim->type != subPrim->type) - errors.push_back(TypeError{location, TypeMismatch{superTy, subTy}}); + reportError(TypeError{location, TypeMismatch{superTy, subTy}}); } void Unifier::tryUnifySingletons(TypeId subTy, TypeId superTy) @@ -1508,13 +1496,13 @@ void Unifier::tryUnifySingletons(TypeId subTy, TypeId superTy) if (superSingleton && *superSingleton == *subSingleton) return; - if (superPrim && superPrim->type == PrimitiveTypeVar::Boolean && get(subSingleton) && variance == Covariant) + if (superPrim && superPrim->type == PrimitiveTypeVar::Boolean && get(subSingleton) && variance == Covariant) return; if (superPrim && superPrim->type == PrimitiveTypeVar::String && get(subSingleton) && variance == Covariant) return; - errors.push_back(TypeError{location, TypeMismatch{superTy, subTy}}); + reportError(TypeError{location, TypeMismatch{superTy, subTy}}); } void Unifier::tryUnifyFunctions(TypeId subTy, TypeId superTy, bool isFunctionCall) @@ -1536,10 +1524,7 @@ void Unifier::tryUnifyFunctions(TypeId subTy, TypeId superTy, bool isFunctionCal { numGenerics = std::min(superFunction->generics.size(), subFunction->generics.size()); - if (FFlag::LuauExtendedFunctionMismatchError) - errors.push_back(TypeError{location, TypeMismatch{superTy, subTy, "different number of generic type parameters"}}); - else - errors.push_back(TypeError{location, TypeMismatch{superTy, subTy}}); + reportError(TypeError{location, TypeMismatch{superTy, subTy, "different number of generic type parameters"}}); } size_t numGenericPacks = superFunction->genericPacks.size(); @@ -1547,10 +1532,7 @@ void Unifier::tryUnifyFunctions(TypeId subTy, TypeId superTy, bool isFunctionCal { numGenericPacks = std::min(superFunction->genericPacks.size(), subFunction->genericPacks.size()); - if (FFlag::LuauExtendedFunctionMismatchError) - errors.push_back(TypeError{location, TypeMismatch{superTy, subTy, "different number of generic type pack parameters"}}); - else - errors.push_back(TypeError{location, TypeMismatch{superTy, subTy}}); + reportError(TypeError{location, TypeMismatch{superTy, subTy, "different number of generic type pack parameters"}}); } for (size_t i = 0; i < numGenerics; i++) @@ -1567,48 +1549,35 @@ void Unifier::tryUnifyFunctions(TypeId subTy, TypeId superTy, bool isFunctionCal { Unifier innerState = makeChildUnifier(); - if (FFlag::LuauExtendedFunctionMismatchError) + innerState.ctx = CountMismatch::Arg; + innerState.tryUnify_(superFunction->argTypes, subFunction->argTypes, isFunctionCall); + + bool reported = !innerState.errors.empty(); + + if (auto e = hasUnificationTooComplex(innerState.errors)) + reportError(*e); + else if (!innerState.errors.empty() && innerState.firstPackErrorPos) + reportError( + TypeError{location, TypeMismatch{superTy, subTy, format("Argument #%d type is not compatible.", *innerState.firstPackErrorPos), + innerState.errors.front()}}); + else if (!innerState.errors.empty()) + reportError(TypeError{location, TypeMismatch{superTy, subTy, "", innerState.errors.front()}}); + + innerState.ctx = CountMismatch::Result; + innerState.tryUnify_(subFunction->retType, superFunction->retType); + + if (!reported) { - innerState.ctx = CountMismatch::Arg; - innerState.tryUnify_(superFunction->argTypes, subFunction->argTypes, isFunctionCall); - - bool reported = !innerState.errors.empty(); - if (auto e = hasUnificationTooComplex(innerState.errors)) - errors.push_back(*e); + reportError(*e); + else if (!innerState.errors.empty() && size(superFunction->retType) == 1 && finite(superFunction->retType)) + reportError(TypeError{location, TypeMismatch{superTy, subTy, "Return type is not compatible.", innerState.errors.front()}}); else if (!innerState.errors.empty() && innerState.firstPackErrorPos) - errors.push_back( - TypeError{location, TypeMismatch{superTy, subTy, format("Argument #%d type is not compatible.", *innerState.firstPackErrorPos), + reportError( + TypeError{location, TypeMismatch{superTy, subTy, format("Return #%d type is not compatible.", *innerState.firstPackErrorPos), innerState.errors.front()}}); else if (!innerState.errors.empty()) - errors.push_back(TypeError{location, TypeMismatch{superTy, subTy, "", innerState.errors.front()}}); - - innerState.ctx = CountMismatch::Result; - innerState.tryUnify_(subFunction->retType, superFunction->retType); - - if (!reported) - { - if (auto e = hasUnificationTooComplex(innerState.errors)) - errors.push_back(*e); - else if (!innerState.errors.empty() && size(superFunction->retType) == 1 && finite(superFunction->retType)) - errors.push_back(TypeError{location, TypeMismatch{superTy, subTy, "Return type is not compatible.", innerState.errors.front()}}); - else if (!innerState.errors.empty() && innerState.firstPackErrorPos) - errors.push_back( - TypeError{location, TypeMismatch{superTy, subTy, format("Return #%d type is not compatible.", *innerState.firstPackErrorPos), - innerState.errors.front()}}); - else if (!innerState.errors.empty()) - errors.push_back(TypeError{location, TypeMismatch{superTy, subTy, "", innerState.errors.front()}}); - } - } - else - { - ctx = CountMismatch::Arg; - innerState.tryUnify_(superFunction->argTypes, subFunction->argTypes, isFunctionCall); - - ctx = CountMismatch::Result; - innerState.tryUnify_(subFunction->retType, superFunction->retType); - - checkChildUnifierTypeMismatch(innerState.errors, superTy, subTy); + reportError(TypeError{location, TypeMismatch{superTy, subTy, "", innerState.errors.front()}}); } if (FFlag::LuauUseCommittingTxnLog) @@ -1716,7 +1685,7 @@ void Unifier::tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) if (!missingProperties.empty()) { - errors.push_back(TypeError{location, MissingProperties{superTy, subTy, std::move(missingProperties)}}); + reportError(TypeError{location, MissingProperties{superTy, subTy, std::move(missingProperties)}}); return; } } @@ -1734,7 +1703,7 @@ void Unifier::tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) if (!extraProperties.empty()) { - errors.push_back(TypeError{location, MissingProperties{superTy, subTy, std::move(extraProperties), MissingProperties::Extra}}); + reportError(TypeError{location, MissingProperties{superTy, subTy, std::move(extraProperties), MissingProperties::Extra}}); return; } } @@ -1957,13 +1926,13 @@ void Unifier::tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) if (!missingProperties.empty()) { - errors.push_back(TypeError{location, MissingProperties{superTy, subTy, std::move(missingProperties)}}); + reportError(TypeError{location, MissingProperties{superTy, subTy, std::move(missingProperties)}}); return; } if (!extraProperties.empty()) { - errors.push_back(TypeError{location, MissingProperties{superTy, subTy, std::move(extraProperties), MissingProperties::Extra}}); + reportError(TypeError{location, MissingProperties{superTy, subTy, std::move(extraProperties), MissingProperties::Extra}}); return; } @@ -2051,7 +2020,7 @@ void Unifier::DEPRECATED_tryUnifyTables(TypeId subTy, TypeId superTy, bool isInt return tryUnifySealedTables(subTy, superTy, isIntersection); else if ((superTable->state == TableState::Sealed && subTable->state == TableState::Generic) || (superTable->state == TableState::Generic && subTable->state == TableState::Sealed)) - errors.push_back(TypeError{location, TypeMismatch{superTy, subTy}}); + reportError(TypeError{location, TypeMismatch{superTy, subTy}}); else if ((superTable->state == TableState::Free) != (subTable->state == TableState::Free)) // one table is free and the other is not { TypeId freeTypeId = subTable->state == TableState::Free ? subTy : superTy; @@ -2090,7 +2059,7 @@ void Unifier::DEPRECATED_tryUnifyTables(TypeId subTy, TypeId superTy, bool isInt { const auto& r = subTable->props.find(name); if (r == subTable->props.end()) - errors.push_back(TypeError{location, UnknownProperty{subTy, name}}); + reportError(TypeError{location, UnknownProperty{subTy, name}}); else tryUnify_(r->second.type, prop.type); } @@ -2113,7 +2082,7 @@ void Unifier::DEPRECATED_tryUnifyTables(TypeId subTy, TypeId superTy, bool isInt } } else - errors.push_back(TypeError{location, CannotExtendTable{subTy, CannotExtendTable::Indexer}}); + reportError(TypeError{location, CannotExtendTable{subTy, CannotExtendTable::Indexer}}); } } else if (superTable->state == TableState::Sealed) @@ -2194,7 +2163,7 @@ void Unifier::tryUnifyFreeTable(TypeId subTy, TypeId superTy) } } else - errors.push_back(TypeError{location, UnknownProperty{subTy, freeName}}); + reportError(TypeError{location, UnknownProperty{subTy, freeName}}); } } @@ -2268,7 +2237,7 @@ void Unifier::tryUnifySealedTables(TypeId subTy, TypeId superTy, bool isIntersec if (!missingPropertiesInSuper.empty()) { - errors.push_back(TypeError{location, MissingProperties{superTy, subTy, std::move(missingPropertiesInSuper)}}); + reportError(TypeError{location, MissingProperties{superTy, subTy, std::move(missingPropertiesInSuper)}}); return; } } @@ -2284,7 +2253,7 @@ void Unifier::tryUnifySealedTables(TypeId subTy, TypeId superTy, bool isIntersec missingPropertiesInSuper.push_back(it.first); - innerState.errors.push_back(TypeError{location, TypeMismatch{superTy, subTy}}); + innerState.reportError(TypeError{location, TypeMismatch{superTy, subTy}}); } else { @@ -2299,7 +2268,7 @@ void Unifier::tryUnifySealedTables(TypeId subTy, TypeId superTy, bool isIntersec if (oldErrorSize != innerState.errors.size() && !errorReported) { errorReported = true; - errors.push_back(innerState.errors.back()); + reportError(innerState.errors.back()); } } else @@ -2340,7 +2309,7 @@ void Unifier::tryUnifySealedTables(TypeId subTy, TypeId superTy, bool isIntersec } } else - innerState.errors.push_back(TypeError{location, TypeMismatch{superTy, subTy}}); + innerState.reportError(TypeError{location, TypeMismatch{superTy, subTy}}); } else { @@ -2369,7 +2338,7 @@ void Unifier::tryUnifySealedTables(TypeId subTy, TypeId superTy, bool isIntersec } } else - innerState.errors.push_back(TypeError{location, TypeMismatch{superTy, subTy}}); + innerState.reportError(TypeError{location, TypeMismatch{superTy, subTy}}); } } @@ -2386,7 +2355,7 @@ void Unifier::tryUnifySealedTables(TypeId subTy, TypeId superTy, bool isIntersec if (!missingPropertiesInSuper.empty()) { - errors.push_back(TypeError{location, MissingProperties{superTy, subTy, std::move(missingPropertiesInSuper)}}); + reportError(TypeError{location, MissingProperties{superTy, subTy, std::move(missingPropertiesInSuper)}}); return; } @@ -2413,7 +2382,7 @@ void Unifier::tryUnifySealedTables(TypeId subTy, TypeId superTy, bool isIntersec if (!extraPropertiesInSub.empty()) { - errors.push_back(TypeError{location, MissingProperties{superTy, subTy, std::move(extraPropertiesInSub), MissingProperties::Extra}}); + reportError(TypeError{location, MissingProperties{superTy, subTy, std::move(extraPropertiesInSub), MissingProperties::Extra}}); return; } } @@ -2437,9 +2406,9 @@ void Unifier::tryUnifyWithMetatable(TypeId subTy, TypeId superTy, bool reversed) innerState.tryUnify_(subMetatable->metatable, superMetatable->metatable); if (auto e = hasUnificationTooComplex(innerState.errors)) - errors.push_back(*e); + reportError(*e); else if (!innerState.errors.empty()) - errors.push_back( + reportError( TypeError{location, TypeMismatch{reversed ? subTy : superTy, reversed ? superTy : subTy, "", innerState.errors.front()}}); if (FFlag::LuauUseCommittingTxnLog) @@ -2470,7 +2439,7 @@ void Unifier::tryUnifyWithMetatable(TypeId subTy, TypeId superTy, bool reversed) case TableState::Sealed: case TableState::Unsealed: case TableState::Generic: - errors.push_back(mismatchError); + reportError(mismatchError); } } else if (FFlag::LuauUseCommittingTxnLog ? (log.getMutable(subTy) || log.getMutable(subTy)) @@ -2479,7 +2448,7 @@ void Unifier::tryUnifyWithMetatable(TypeId subTy, TypeId superTy, bool reversed) } else { - errors.push_back(mismatchError); + reportError(mismatchError); } } @@ -2491,9 +2460,9 @@ void Unifier::tryUnifyWithClass(TypeId subTy, TypeId superTy, bool reversed) auto fail = [&]() { if (!reversed) - errors.push_back(TypeError{location, TypeMismatch{superTy, subTy}}); + reportError(TypeError{location, TypeMismatch{superTy, subTy}}); else - errors.push_back(TypeError{location, TypeMismatch{subTy, superTy}}); + reportError(TypeError{location, TypeMismatch{subTy, superTy}}); }; const ClassTypeVar* superClass = get(superTy); @@ -2538,7 +2507,7 @@ void Unifier::tryUnifyWithClass(TypeId subTy, TypeId superTy, bool reversed) if (!classProp) { ok = false; - errors.push_back(TypeError{location, UnknownProperty{superTy, propName}}); + reportError(TypeError{location, UnknownProperty{superTy, propName}}); } else { @@ -2577,7 +2546,7 @@ void Unifier::tryUnifyWithClass(TypeId subTy, TypeId superTy, bool reversed) { ok = false; std::string msg = "Class " + superClass->name + " does not have an indexer"; - errors.push_back(TypeError{location, GenericError{msg}}); + reportError(TypeError{location, GenericError{msg}}); } if (!ok) @@ -2695,7 +2664,7 @@ void Unifier::tryUnifyVariadics(TypePackId subTp, TypePackId superTp, bool rever } else if (get(tail)) { - errors.push_back(TypeError{location, GenericError{"Cannot unify variadic and generic packs"}}); + reportError(TypeError{location, GenericError{"Cannot unify variadic and generic packs"}}); } else if (get(tail)) { @@ -2709,7 +2678,7 @@ void Unifier::tryUnifyVariadics(TypePackId subTp, TypePackId superTp, bool rever } else { - errors.push_back(TypeError{location, GenericError{"Failed to unify variadic packs"}}); + reportError(TypeError{location, GenericError{"Failed to unify variadic packs"}}); } } @@ -2886,7 +2855,7 @@ void Unifier::occursCheck(DenseHashSet& seen, TypeId needle, TypeId hays if (needle == haystack) { - errors.push_back(TypeError{location, OccursCheckFailed{}}); + reportError(TypeError{location, OccursCheckFailed{}}); log.replace(needle, *getSingletonTypes().errorRecoveryType()); return; @@ -2894,17 +2863,6 @@ void Unifier::occursCheck(DenseHashSet& seen, TypeId needle, TypeId hays if (log.getMutable(haystack)) return; - else if (auto a = log.getMutable(haystack)) - { - if (!FFlag::LuauOccursCheckOkWithRecursiveFunctions) - { - for (TypePackIterator it(a->argTypes, &log); it != end(a->argTypes); ++it) - check(*it); - - for (TypePackIterator it(a->retType, &log); it != end(a->retType); ++it) - check(*it); - } - } else if (auto a = log.getMutable(haystack)) { for (TypeId ty : a->options) @@ -2934,7 +2892,7 @@ void Unifier::occursCheck(DenseHashSet& seen, TypeId needle, TypeId hays if (needle == haystack) { - errors.push_back(TypeError{location, OccursCheckFailed{}}); + reportError(TypeError{location, OccursCheckFailed{}}); DEPRECATED_log(needle); *asMutable(needle) = *getSingletonTypes().errorRecoveryType(); return; @@ -2942,17 +2900,6 @@ void Unifier::occursCheck(DenseHashSet& seen, TypeId needle, TypeId hays if (get(haystack)) return; - else if (auto a = get(haystack)) - { - if (!FFlag::LuauOccursCheckOkWithRecursiveFunctions) - { - for (TypeId ty : a->argTypes) - check(ty); - - for (TypeId ty : a->retType) - check(ty); - } - } else if (auto a = get(haystack)) { for (TypeId ty : a->options) @@ -2988,7 +2935,7 @@ void Unifier::occursCheck(DenseHashSet& seen, TypePackId needle, Typ if (log.getMutable(needle)) return; - if (!get(needle)) + if (!log.getMutable(needle)) ice("Expected needle pack to be free"); RecursionLimiter _ra(&sharedState.counters.recursionCount, FInt::LuauTypeInferRecursionLimit); @@ -2997,32 +2944,18 @@ void Unifier::occursCheck(DenseHashSet& seen, TypePackId needle, Typ { if (needle == haystack) { - errors.push_back(TypeError{location, OccursCheckFailed{}}); + reportError(TypeError{location, OccursCheckFailed{}}); log.replace(needle, *getSingletonTypes().errorRecoveryTypePack()); return; } - if (auto a = get(haystack)) + if (auto a = get(haystack); a && a->tail) { - for (const auto& ty : a->head) - { - if (!FFlag::LuauOccursCheckOkWithRecursiveFunctions) - { - if (auto f = log.getMutable(log.follow(ty))) - { - occursCheck(seen, needle, f->argTypes); - occursCheck(seen, needle, f->retType); - } - } - } - - if (a->tail) - { - haystack = follow(*a->tail); - continue; - } + haystack = log.follow(*a->tail); + continue; } + break; } } @@ -3048,31 +2981,17 @@ void Unifier::occursCheck(DenseHashSet& seen, TypePackId needle, Typ { if (needle == haystack) { - errors.push_back(TypeError{location, OccursCheckFailed{}}); + reportError(TypeError{location, OccursCheckFailed{}}); DEPRECATED_log(needle); *asMutable(needle) = *getSingletonTypes().errorRecoveryTypePack(); } - if (auto a = get(haystack)) + if (auto a = get(haystack); a && a->tail) { - if (!FFlag::LuauOccursCheckOkWithRecursiveFunctions) - { - for (const auto& ty : a->head) - { - if (auto f = get(follow(ty))) - { - occursCheck(seen, needle, f->argTypes); - occursCheck(seen, needle, f->retType); - } - } - } - - if (a->tail) - { - haystack = follow(*a->tail); - continue; - } + haystack = follow(*a->tail); + continue; } + break; } } @@ -3094,17 +3013,17 @@ bool Unifier::isNonstrictMode() const void Unifier::checkChildUnifierTypeMismatch(const ErrorVec& innerErrors, TypeId wantedType, TypeId givenType) { if (auto e = hasUnificationTooComplex(innerErrors)) - errors.push_back(*e); + reportError(*e); else if (!innerErrors.empty()) - errors.push_back(TypeError{location, TypeMismatch{wantedType, givenType}}); + reportError(TypeError{location, TypeMismatch{wantedType, givenType}}); } void Unifier::checkChildUnifierTypeMismatch(const ErrorVec& innerErrors, const std::string& prop, TypeId wantedType, TypeId givenType) { if (auto e = hasUnificationTooComplex(innerErrors)) - errors.push_back(*e); + reportError(*e); else if (!innerErrors.empty()) - errors.push_back( + reportError( TypeError{location, TypeMismatch{wantedType, givenType, format("Property '%s' is not compatible.", prop.c_str()), innerErrors.front()}}); } diff --git a/CLI/Analyze.cpp b/CLI/Analyze.cpp index e0dc3e0f..10cf17d2 100644 --- a/CLI/Analyze.cpp +++ b/CLI/Analyze.cpp @@ -43,14 +43,14 @@ static void report(ReportFormat format, const char* name, const Luau::Location& } } -static void reportError(ReportFormat format, const Luau::TypeError& error) +static void reportError(const Luau::Frontend& frontend, ReportFormat format, const Luau::TypeError& error) { - const char* name = error.moduleName.c_str(); + std::string humanReadableName = frontend.fileResolver->getHumanReadableModuleName(error.moduleName); if (const Luau::SyntaxError* syntaxError = Luau::get_if(&error.data)) - report(format, name, error.location, "SyntaxError", syntaxError->message.c_str()); + report(format, humanReadableName.c_str(), error.location, "SyntaxError", syntaxError->message.c_str()); else - report(format, name, error.location, "TypeError", Luau::toString(error).c_str()); + report(format, humanReadableName.c_str(), error.location, "TypeError", Luau::toString(error).c_str()); } static void reportWarning(ReportFormat format, const char* name, const Luau::LintWarning& warning) @@ -72,14 +72,15 @@ static bool analyzeFile(Luau::Frontend& frontend, const char* name, ReportFormat } for (auto& error : cr.errors) - reportError(format, error); + reportError(frontend, format, error); Luau::LintResult lr = frontend.lint(name); + std::string humanReadableName = frontend.fileResolver->getHumanReadableModuleName(name); for (auto& error : lr.errors) - reportWarning(format, name, error); + reportWarning(format, humanReadableName.c_str(), error); for (auto& warning : lr.warnings) - reportWarning(format, name, warning); + reportWarning(format, humanReadableName.c_str(), warning); if (annotate) { @@ -120,11 +121,25 @@ struct CliFileResolver : Luau::FileResolver { std::optional readSource(const Luau::ModuleName& name) override { - std::optional source = readFile(name); + Luau::SourceCode::Type sourceType; + std::optional source = std::nullopt; + + // If the module name is "-", then read source from stdin + if (name == "-") + { + source = readStdin(); + sourceType = Luau::SourceCode::Script; + } + else + { + source = readFile(name); + sourceType = Luau::SourceCode::Module; + } + if (!source) return std::nullopt; - return Luau::SourceCode{*source, Luau::SourceCode::Module}; + return Luau::SourceCode{*source, sourceType}; } std::optional resolveModule(const Luau::ModuleInfo* context, Luau::AstExpr* node) override @@ -143,6 +158,13 @@ struct CliFileResolver : Luau::FileResolver return std::nullopt; } + + std::string getHumanReadableModuleName(const Luau::ModuleName& name) const override + { + if (name == "-") + return "stdin"; + return name; + } }; struct CliConfigResolver : Luau::ConfigResolver diff --git a/CLI/FileUtils.cpp b/CLI/FileUtils.cpp index cb993dfe..c6807022 100644 --- a/CLI/FileUtils.cpp +++ b/CLI/FileUtils.cpp @@ -74,6 +74,21 @@ std::optional readFile(const std::string& name) return result; } +std::optional readStdin() +{ + std::string result; + char buffer[4096] = { }; + + while (fgets(buffer, sizeof(buffer), stdin) != nullptr) + result.append(buffer); + + // If eof was not reached for stdin, then a read error occurred + if (!feof(stdin)) + return std::nullopt; + + return result; +} + template static void joinPaths(std::basic_string& str, const Ch* lhs, const Ch* rhs) { @@ -190,7 +205,10 @@ bool traverseDirectory(const std::string& path, const std::function getSourceFiles(int argc, char** argv) for (int i = 1; i < argc; ++i) { - if (argv[i][0] == '-') + // Treat '-' as a special file whose source is read from stdin + // All other arguments that start with '-' are skipped + if (argv[i][0] == '-' && argv[i][1] != '\0') continue; if (isDirectory(argv[i])) diff --git a/CLI/FileUtils.h b/CLI/FileUtils.h index da11f512..97471cdc 100644 --- a/CLI/FileUtils.h +++ b/CLI/FileUtils.h @@ -7,6 +7,7 @@ #include std::optional readFile(const std::string& name); +std::optional readStdin(); bool isDirectory(const std::string& path); bool traverseDirectory(const std::string& path, const std::function& callback); diff --git a/CLI/Repl.cpp b/CLI/Repl.cpp index e5042152..ab0f0ed0 100644 --- a/CLI/Repl.cpp +++ b/CLI/Repl.cpp @@ -158,7 +158,7 @@ static int lua_collectgarbage(lua_State* L) luaL_error(L, "collectgarbage must be called with 'count' or 'collect'"); } -static void setupState(lua_State* L) +void setupState(lua_State* L) { luaL_openlibs(L); @@ -176,7 +176,7 @@ static void setupState(lua_State* L) luaL_sandbox(L); } -static std::string runCode(lua_State* L, const std::string& source) +std::string runCode(lua_State* L, const std::string& source) { std::string bytecode = Luau::compile(source, copts()); @@ -206,7 +206,13 @@ static std::string runCode(lua_State* L, const std::string& source) if (n) { luaL_checkstack(T, LUA_MINSTACK, "too many results to print"); - lua_getglobal(T, "print"); + lua_getglobal(T, "_PRETTYPRINT"); + // If _PRETTYPRINT is nil, then use the standard print function instead + if (lua_isnil(T, -1)) + { + lua_pop(T, 1); + lua_getglobal(T, "print"); + } lua_insert(T, 1); lua_pcall(T, n, 0, 0); } @@ -545,7 +551,7 @@ static int assertionHandler(const char* expr, const char* file, int line, const return 1; } -int main(int argc, char** argv) +int replMain(int argc, char** argv) { Luau::assertHandler() = assertionHandler; @@ -696,7 +702,6 @@ int main(int argc, char** argv) case CliMode::Unknown: default: LUAU_ASSERT(!"Unhandled cli mode."); + return 1; } } - - diff --git a/CLI/Repl.h b/CLI/Repl.h new file mode 100644 index 00000000..11a077ae --- /dev/null +++ b/CLI/Repl.h @@ -0,0 +1,12 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +#pragma once + +#include "lua.h" + +#include + +// Note: These are internal functions which are being exposed in a header +// so they can be included by unit tests. +int replMain(int argc, char** argv); +void setupState(lua_State* L); +std::string runCode(lua_State* L, const std::string& source); diff --git a/CLI/ReplEntry.cpp b/CLI/ReplEntry.cpp new file mode 100644 index 00000000..b3131712 --- /dev/null +++ b/CLI/ReplEntry.cpp @@ -0,0 +1,10 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details + +#include "Repl.h" + + + +int main(int argc, char** argv) +{ + return replMain(argc, argv); +} \ No newline at end of file diff --git a/CMakeLists.txt b/CMakeLists.txt index 77cf47e8..b9f7a9e1 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -29,6 +29,7 @@ endif() if(LUAU_BUILD_TESTS) add_executable(Luau.UnitTest) add_executable(Luau.Conformance) + add_executable(Luau.CLI.Test) endif() if(LUAU_BUILD_WEB) @@ -109,6 +110,17 @@ if(LUAU_BUILD_TESTS) target_compile_options(Luau.Conformance PRIVATE ${LUAU_OPTIONS}) target_include_directories(Luau.Conformance PRIVATE extern) target_link_libraries(Luau.Conformance PRIVATE Luau.Analysis Luau.Compiler Luau.VM) + + target_compile_options(Luau.CLI.Test PRIVATE ${LUAU_OPTIONS}) + target_include_directories(Luau.CLI.Test PRIVATE extern CLI) + target_link_libraries(Luau.CLI.Test PRIVATE Luau.Compiler Luau.VM) + if(UNIX) + find_library(LIBPTHREAD pthread) + if (LIBPTHREAD) + target_link_libraries(Luau.CLI.Test PRIVATE pthread) + endif() + endif() + endif() if(LUAU_BUILD_WEB) diff --git a/Compiler/include/Luau/Bytecode.h b/Compiler/include/Luau/Bytecode.h index d9694d7d..679712f6 100644 --- a/Compiler/include/Luau/Bytecode.h +++ b/Compiler/include/Luau/Bytecode.h @@ -472,6 +472,9 @@ enum LuauBuiltinFunction // bit32.count LBF_BIT32_COUNTLZ, LBF_BIT32_COUNTRZ, + + // select(_, ...) + LBF_SELECT_VARARG, }; // Capture type, used in LOP_CAPTURE diff --git a/Compiler/src/Builtins.cpp b/Compiler/src/Builtins.cpp index e344eb91..a907271c 100644 --- a/Compiler/src/Builtins.cpp +++ b/Compiler/src/Builtins.cpp @@ -4,6 +4,8 @@ #include "Luau/Bytecode.h" #include "Luau/Compiler.h" +LUAU_FASTFLAGVARIABLE(LuauCompileSelectBuiltin, false) + namespace Luau { namespace Compile @@ -62,6 +64,9 @@ int getBuiltinFunctionId(const Builtin& builtin, const CompileOptions& options) if (builtin.isGlobal("unpack")) return LBF_TABLE_UNPACK; + if (FFlag::LuauCompileSelectBuiltin && builtin.isGlobal("select")) + return LBF_SELECT_VARARG; + if (builtin.object == "math") { if (builtin.method == "abs") diff --git a/Compiler/src/Compiler.cpp b/Compiler/src/Compiler.cpp index 9758c4a9..7da85244 100644 --- a/Compiler/src/Compiler.cpp +++ b/Compiler/src/Compiler.cpp @@ -15,6 +15,9 @@ #include #include +LUAU_FASTFLAGVARIABLE(LuauCompileTableIndexOpt, false) +LUAU_FASTFLAG(LuauCompileSelectBuiltin) + namespace Luau { @@ -261,6 +264,122 @@ struct Compiler bytecode.emitABC(LOP_GETVARARGS, target, multRet ? 0 : uint8_t(targetCount + 1), 0); } + void compileExprSelectVararg(AstExprCall* expr, uint8_t target, uint8_t targetCount, bool targetTop, bool multRet, uint8_t regs) + { + LUAU_ASSERT(FFlag::LuauCompileSelectBuiltin); + LUAU_ASSERT(targetCount == 1); + LUAU_ASSERT(!expr->self); + LUAU_ASSERT(expr->args.size == 2 && expr->args.data[1]->is()); + + AstExpr* arg = expr->args.data[0]; + + uint8_t argreg; + + if (isExprLocalReg(arg)) + argreg = getLocal(arg->as()->local); + else + { + argreg = uint8_t(regs + 1); + compileExprTempTop(arg, argreg); + } + + size_t fastcallLabel = bytecode.emitLabel(); + + bytecode.emitABC(LOP_FASTCALL1, LBF_SELECT_VARARG, argreg, 0); + + // note, these instructions are normally not executed and are used as a fallback for FASTCALL + // we can't use TempTop variant here because we need to make sure the arguments we already computed aren't overwritten + compileExprTemp(expr->func, regs); + + bytecode.emitABC(LOP_GETVARARGS, uint8_t(regs + 2), 0, 0); + + size_t callLabel = bytecode.emitLabel(); + if (!bytecode.patchSkipC(fastcallLabel, callLabel)) + CompileError::raise(expr->func->location, "Exceeded jump distance limit; simplify the code to compile"); + + // note, this is always multCall (last argument is variadic) + bytecode.emitABC(LOP_CALL, regs, 0, multRet ? 0 : uint8_t(targetCount + 1)); + + // if we didn't output results directly to target, we need to move them + if (!targetTop) + { + for (size_t i = 0; i < targetCount; ++i) + bytecode.emitABC(LOP_MOVE, uint8_t(target + i), uint8_t(regs + i), 0); + } + } + + void compileExprFastcallN(AstExprCall* expr, uint8_t target, uint8_t targetCount, bool targetTop, bool multRet, uint8_t regs, int bfid) + { + LUAU_ASSERT(!expr->self); + LUAU_ASSERT(expr->args.size <= 2); + + LuauOpcode opc = expr->args.size == 1 ? LOP_FASTCALL1 : LOP_FASTCALL2; + + uint32_t args[2] = {}; + + for (size_t i = 0; i < expr->args.size; ++i) + { + if (i > 0) + { + if (int32_t cid = getConstantIndex(expr->args.data[i]); cid >= 0) + { + opc = LOP_FASTCALL2K; + args[i] = cid; + break; + } + } + + if (isExprLocalReg(expr->args.data[i])) + args[i] = getLocal(expr->args.data[i]->as()->local); + else + { + args[i] = uint8_t(regs + 1 + i); + compileExprTempTop(expr->args.data[i], uint8_t(args[i])); + } + } + + size_t fastcallLabel = bytecode.emitLabel(); + + bytecode.emitABC(opc, uint8_t(bfid), uint8_t(args[0]), 0); + if (opc != LOP_FASTCALL1) + bytecode.emitAux(args[1]); + + // Set up a traditional Lua stack for the subsequent LOP_CALL. + // Note, as with other instructions that immediately follow FASTCALL, these are normally not executed and are used as a fallback for + // these FASTCALL variants. + for (size_t i = 0; i < expr->args.size; ++i) + { + if (i > 0 && opc == LOP_FASTCALL2K) + { + emitLoadK(uint8_t(regs + 1 + i), args[i]); + break; + } + + if (args[i] != regs + 1 + i) + bytecode.emitABC(LOP_MOVE, uint8_t(regs + 1 + i), uint8_t(args[i]), 0); + } + + // note, these instructions are normally not executed and are used as a fallback for FASTCALL + // we can't use TempTop variant here because we need to make sure the arguments we already computed aren't overwritten + compileExprTemp(expr->func, regs); + + size_t callLabel = bytecode.emitLabel(); + + // FASTCALL will skip over the instructions needed to compute function and jump over CALL which must immediately follow the instruction + // sequence after FASTCALL + if (!bytecode.patchSkipC(fastcallLabel, callLabel)) + CompileError::raise(expr->func->location, "Exceeded jump distance limit; simplify the code to compile"); + + bytecode.emitABC(LOP_CALL, regs, uint8_t(expr->args.size + 1), multRet ? 0 : uint8_t(targetCount + 1)); + + // if we didn't output results directly to target, we need to move them + if (!targetTop) + { + for (size_t i = 0; i < targetCount; ++i) + bytecode.emitABC(LOP_MOVE, uint8_t(target + i), uint8_t(regs + i), 0); + } + } + void compileExprCall(AstExprCall* expr, uint8_t target, uint8_t targetCount, bool targetTop = false, bool multRet = false) { LUAU_ASSERT(!targetTop || unsigned(target + targetCount) == regTop); @@ -284,6 +403,25 @@ struct Compiler bfid = getBuiltinFunctionId(builtin, options); } + if (bfid == LBF_SELECT_VARARG) + { + LUAU_ASSERT(FFlag::LuauCompileSelectBuiltin); + // Optimization: compile select(_, ...) as FASTCALL1; the builtin will read variadic arguments directly + // note: for now we restrict this to single-return expressions since our runtime code doesn't deal with general cases + if (multRet == false && targetCount == 1 && expr->args.size == 2 && expr->args.data[1]->is()) + return compileExprSelectVararg(expr, target, targetCount, targetTop, multRet, regs); + else + bfid = -1; + } + + // Optimization: for 1/2 argument fast calls use specialized opcodes + if (!expr->self && bfid >= 0 && expr->args.size >= 1 && expr->args.size <= 2) + { + AstExpr* last = expr->args.data[expr->args.size - 1]; + if (!last->is() && !last->is()) + return compileExprFastcallN(expr, target, targetCount, targetTop, multRet, regs, bfid); + } + if (expr->self) { AstExprIndexName* fi = expr->func->as(); @@ -309,24 +447,13 @@ struct Compiler compileExprTempTop(expr->func, regs); } - // Note: if the last argument is ExprVararg or ExprCall, we need to route that directly to the called function preserving the # of args bool multCall = false; - bool skipArgs = false; - if (!expr->self && bfid >= 0 && expr->args.size >= 1 && expr->args.size <= 2) - { - AstExpr* last = expr->args.data[expr->args.size - 1]; - skipArgs = !(last->is() || last->is()); - } - - if (!skipArgs) - { - for (size_t i = 0; i < expr->args.size; ++i) - if (i + 1 == expr->args.size) - multCall = compileExprTempMultRet(expr->args.data[i], uint8_t(regs + 1 + expr->self + i)); - else - compileExprTempTop(expr->args.data[i], uint8_t(regs + 1 + expr->self + i)); - } + for (size_t i = 0; i < expr->args.size; ++i) + if (i + 1 == expr->args.size) + multCall = compileExprTempMultRet(expr->args.data[i], uint8_t(regs + 1 + expr->self + i)); + else + compileExprTempTop(expr->args.data[i], uint8_t(regs + 1 + expr->self + i)); setDebugLineEnd(expr->func); @@ -347,59 +474,8 @@ struct Compiler } else if (bfid >= 0) { - size_t fastcallLabel; - - if (skipArgs) - { - LuauOpcode opc = expr->args.size == 1 ? LOP_FASTCALL1 : LOP_FASTCALL2; - - uint32_t args[2] = {}; - for (size_t i = 0; i < expr->args.size; ++i) - { - if (i > 0) - { - if (int32_t cid = getConstantIndex(expr->args.data[i]); cid >= 0) - { - opc = LOP_FASTCALL2K; - args[i] = cid; - break; - } - } - - if (isExprLocalReg(expr->args.data[i])) - args[i] = getLocal(expr->args.data[i]->as()->local); - else - { - args[i] = uint8_t(regs + 1 + i); - compileExprTempTop(expr->args.data[i], uint8_t(args[i])); - } - } - - fastcallLabel = bytecode.emitLabel(); - bytecode.emitABC(opc, uint8_t(bfid), uint8_t(args[0]), 0); - if (opc != LOP_FASTCALL1) - bytecode.emitAux(args[1]); - - // Set up a traditional Lua stack for the subsequent LOP_CALL. - // Note, as with other instructions that immediately follow FASTCALL, these are normally not executed and are used as a fallback for - // these FASTCALL variants. - for (size_t i = 0; i < expr->args.size; ++i) - { - if (i > 0 && opc == LOP_FASTCALL2K) - { - emitLoadK(uint8_t(regs + 1 + i), args[i]); - break; - } - - if (args[i] != regs + 1 + i) - bytecode.emitABC(LOP_MOVE, uint8_t(regs + 1 + i), uint8_t(args[i]), 0); - } - } - else - { - fastcallLabel = bytecode.emitLabel(); - bytecode.emitABC(LOP_FASTCALL, uint8_t(bfid), 0, 0); - } + size_t fastcallLabel = bytecode.emitLabel(); + bytecode.emitABC(LOP_FASTCALL, uint8_t(bfid), 0, 0); // note, these instructions are normally not executed and are used as a fallback for FASTCALL // we can't use TempTop variant here because we need to make sure the arguments we already computed aren't overwritten @@ -1101,9 +1177,20 @@ struct Compiler for (size_t i = 0; i < expr->items.size; ++i) { const AstExprTable::Item& item = expr->items.data[i]; - AstExprConstantNumber* ckey = item.key->as(); + LUAU_ASSERT(item.key); // no list portion => all items have keys - indexSize += (ckey && ckey->value == double(indexSize + 1)); + if (FFlag::LuauCompileTableIndexOpt) + { + const Constant* ckey = constants.find(item.key); + + indexSize += (ckey && ckey->type == Constant::Type_Number && ckey->valueNumber == double(indexSize + 1)); + } + else + { + AstExprConstantNumber* ckey = item.key->as(); + + indexSize += (ckey && ckey->value == double(indexSize + 1)); + } } // we only perform the optimization if we don't have any other []-keys @@ -1200,37 +1287,47 @@ struct Compiler arrayChunkCurrent = 0; } - // items with a key are set one by one via SETTABLE/SETTABLEKS + // items with a key are set one by one via SETTABLE/SETTABLEKS/SETTABLEN if (key) { RegScope rsi(this); - // Optimization: use SETTABLEKS/SETTABLEN for literal keys, this happens often as part of usual table construction syntax - if (AstExprConstantString* ckey = key->as()) + if (FFlag::LuauCompileTableIndexOpt) { - BytecodeBuilder::StringRef cname = sref(ckey->value); - int32_t cid = bytecode.addConstantString(cname); - if (cid < 0) - CompileError::raise(expr->location, "Exceeded constant limit; simplify the code to compile"); - + LValue lv = compileLValueIndex(reg, key, rsi); uint8_t rv = compileExprAuto(value, rsi); - bytecode.emitABC(LOP_SETTABLEKS, rv, reg, uint8_t(BytecodeBuilder::getStringHash(cname))); - bytecode.emitAux(cid); - } - else if (AstExprConstantNumber* ckey = key->as(); - ckey && ckey->value >= 1 && ckey->value <= 256 && double(int(ckey->value)) == ckey->value) - { - uint8_t rv = compileExprAuto(value, rsi); - - bytecode.emitABC(LOP_SETTABLEN, rv, reg, uint8_t(int(ckey->value) - 1)); + compileAssign(lv, rv); } else { - uint8_t rk = compileExprAuto(key, rsi); - uint8_t rv = compileExprAuto(value, rsi); + // Optimization: use SETTABLEKS/SETTABLEN for literal keys, this happens often as part of usual table construction syntax + if (AstExprConstantString* ckey = key->as()) + { + BytecodeBuilder::StringRef cname = sref(ckey->value); + int32_t cid = bytecode.addConstantString(cname); + if (cid < 0) + CompileError::raise(expr->location, "Exceeded constant limit; simplify the code to compile"); - bytecode.emitABC(LOP_SETTABLE, rv, reg, rk); + uint8_t rv = compileExprAuto(value, rsi); + + bytecode.emitABC(LOP_SETTABLEKS, rv, reg, uint8_t(BytecodeBuilder::getStringHash(cname))); + bytecode.emitAux(cid); + } + else if (AstExprConstantNumber* ckey = key->as(); + ckey && ckey->value >= 1 && ckey->value <= 256 && double(int(ckey->value)) == ckey->value) + { + uint8_t rv = compileExprAuto(value, rsi); + + bytecode.emitABC(LOP_SETTABLEN, rv, reg, uint8_t(int(ckey->value) - 1)); + } + else + { + uint8_t rk = compileExprAuto(key, rsi); + uint8_t rv = compileExprAuto(value, rsi); + + bytecode.emitABC(LOP_SETTABLE, rv, reg, rk); + } } } // items without a key are set using SETLIST so that we can initialize large arrays quickly @@ -1339,6 +1436,9 @@ struct Compiler uint8_t rt = compileExprAuto(expr->expr, rs); uint8_t i = uint8_t(int(cv->valueNumber) - 1); + if (FFlag::LuauCompileTableIndexOpt) + setDebugLine(expr->index); + bytecode.emitABC(LOP_GETTABLEN, target, rt, i); } else if (cv && cv->type == Constant::Type_String) @@ -1350,6 +1450,9 @@ struct Compiler if (cid < 0) CompileError::raise(expr->location, "Exceeded constant limit; simplify the code to compile"); + if (FFlag::LuauCompileTableIndexOpt) + setDebugLine(expr->index); + bytecode.emitABC(LOP_GETTABLEKS, target, rt, uint8_t(BytecodeBuilder::getStringHash(iname))); bytecode.emitAux(cid); } @@ -1657,6 +1760,40 @@ struct Compiler Location location; }; + LValue compileLValueIndex(uint8_t reg, AstExpr* index, RegScope& rs) + { + const Constant* cv = constants.find(index); + + if (cv && cv->type == Constant::Type_Number && cv->valueNumber >= 1 && cv->valueNumber <= 256 && + double(int(cv->valueNumber)) == cv->valueNumber) + { + LValue result = {LValue::Kind_IndexNumber}; + result.reg = reg; + result.number = uint8_t(int(cv->valueNumber) - 1); + result.location = index->location; + + return result; + } + else if (cv && cv->type == Constant::Type_String) + { + LValue result = {LValue::Kind_IndexName}; + result.reg = reg; + result.name = sref(cv->getString()); + result.location = index->location; + + return result; + } + else + { + LValue result = {LValue::Kind_IndexExpr}; + result.reg = reg; + result.index = compileExprAuto(index, rs); + result.location = index->location; + + return result; + } + } + LValue compileLValue(AstExpr* node, RegScope& rs) { setDebugLine(node); @@ -1699,36 +1836,9 @@ struct Compiler } else if (AstExprIndexExpr* expr = node->as()) { - const Constant* cv = constants.find(expr->index); + uint8_t reg = compileExprAuto(expr->expr, rs); - if (cv && cv->type == Constant::Type_Number && cv->valueNumber >= 1 && cv->valueNumber <= 256 && - double(int(cv->valueNumber)) == cv->valueNumber) - { - LValue result = {LValue::Kind_IndexNumber}; - result.reg = compileExprAuto(expr->expr, rs); - result.number = uint8_t(int(cv->valueNumber) - 1); - result.location = node->location; - - return result; - } - else if (cv && cv->type == Constant::Type_String) - { - LValue result = {LValue::Kind_IndexName}; - result.reg = compileExprAuto(expr->expr, rs); - result.name = sref(cv->getString()); - result.location = node->location; - - return result; - } - else - { - LValue result = {LValue::Kind_IndexExpr}; - result.reg = compileExprAuto(expr->expr, rs); - result.index = compileExprAuto(expr->index, rs); - result.location = node->location; - - return result; - } + return compileLValueIndex(reg, expr->index, rs); } else { @@ -1740,6 +1850,9 @@ struct Compiler void compileLValueUse(const LValue& lv, uint8_t reg, bool set) { + if (FFlag::LuauCompileTableIndexOpt) + setDebugLine(lv.location); + switch (lv.kind) { case LValue::Kind_Local: diff --git a/Makefile b/Makefile index b144cac6..638c4c63 100644 --- a/Makefile +++ b/Makefile @@ -23,11 +23,11 @@ VM_SOURCES=$(wildcard VM/src/*.cpp) VM_OBJECTS=$(VM_SOURCES:%=$(BUILD)/%.o) VM_TARGET=$(BUILD)/libluauvm.a -TESTS_SOURCES=$(wildcard tests/*.cpp) +TESTS_SOURCES=$(wildcard tests/*.cpp) CLI/FileUtils.cpp CLI/Profiler.cpp CLI/Coverage.cpp CLI/Repl.cpp TESTS_OBJECTS=$(TESTS_SOURCES:%=$(BUILD)/%.o) TESTS_TARGET=$(BUILD)/luau-tests -REPL_CLI_SOURCES=CLI/FileUtils.cpp CLI/Profiler.cpp CLI/Coverage.cpp CLI/Repl.cpp +REPL_CLI_SOURCES=CLI/FileUtils.cpp CLI/Profiler.cpp CLI/Coverage.cpp CLI/Repl.cpp CLI/ReplEntry.cpp REPL_CLI_OBJECTS=$(REPL_CLI_SOURCES:%=$(BUILD)/%.o) REPL_CLI_TARGET=$(BUILD)/luau @@ -90,11 +90,12 @@ $(AST_OBJECTS): CXXFLAGS+=-std=c++17 -IAst/include $(COMPILER_OBJECTS): CXXFLAGS+=-std=c++17 -ICompiler/include -IAst/include $(ANALYSIS_OBJECTS): CXXFLAGS+=-std=c++17 -IAst/include -IAnalysis/include $(VM_OBJECTS): CXXFLAGS+=-std=c++11 -IVM/include -$(TESTS_OBJECTS): CXXFLAGS+=-std=c++17 -IAst/include -ICompiler/include -IAnalysis/include -IVM/include -Iextern +$(TESTS_OBJECTS): CXXFLAGS+=-std=c++17 -IAst/include -ICompiler/include -IAnalysis/include -IVM/include -ICLI -Iextern $(REPL_CLI_OBJECTS): CXXFLAGS+=-std=c++17 -IAst/include -ICompiler/include -IVM/include -Iextern $(ANALYZE_CLI_OBJECTS): CXXFLAGS+=-std=c++17 -IAst/include -IAnalysis/include -Iextern $(FUZZ_OBJECTS): CXXFLAGS+=-std=c++17 -IAst/include -ICompiler/include -IAnalysis/include -IVM/include +$(TESTS_TARGET): LDFLAGS+=-lpthread $(REPL_CLI_TARGET): LDFLAGS+=-lpthread fuzz-proto fuzz-prototest: LDFLAGS+=build/libprotobuf-mutator/src/libfuzzer/libprotobuf-mutator-libfuzzer.a build/libprotobuf-mutator/src/libprotobuf-mutator.a build/libprotobuf-mutator/external.protobuf/lib/libprotobuf.a diff --git a/Sources.cmake b/Sources.cmake index bafe7594..22e7af22 100644 --- a/Sources.cmake +++ b/Sources.cmake @@ -176,7 +176,8 @@ if(TARGET Luau.Repl.CLI) CLI/FileUtils.cpp CLI/Profiler.h CLI/Profiler.cpp - CLI/Repl.cpp) + CLI/Repl.cpp + CLI/ReplEntry.cpp) endif() if(TARGET Luau.Analyze.CLI) @@ -243,6 +244,21 @@ if(TARGET Luau.Conformance) tests/main.cpp) endif() +if(TARGET Luau.CLI.Test) + # Luau.CLI.Test Sources + target_sources(Luau.CLI.Test PRIVATE + CLI/Coverage.h + CLI/Coverage.cpp + CLI/FileUtils.h + CLI/FileUtils.cpp + CLI/Profiler.h + CLI/Profiler.cpp + CLI/Repl.cpp + + tests/Repl.test.cpp + tests/main.cpp) +endif() + if(TARGET Luau.Web) # Luau.Web Sources target_sources(Luau.Web PRIVATE diff --git a/VM/src/lapi.cpp b/VM/src/lapi.cpp index d5416285..5cffba63 100644 --- a/VM/src/lapi.cpp +++ b/VM/src/lapi.cpp @@ -14,6 +14,8 @@ #include +LUAU_FASTFLAGVARIABLE(LuauGcForwardMetatableBarrier, false) + const char* lua_ident = "$Lua: Lua 5.1.4 Copyright (C) 1994-2008 Lua.org, PUC-Rio $\n" "$Authors: R. Ierusalimschy, L. H. de Figueiredo & W. Celes $\n" "$URL: www.lua.org $\n"; @@ -869,7 +871,16 @@ int lua_setmetatable(lua_State* L, int objindex) luaG_runerror(L, "Attempt to modify a readonly table"); hvalue(obj)->metatable = mt; if (mt) - luaC_objbarriert(L, hvalue(obj), mt); + { + if (FFlag::LuauGcForwardMetatableBarrier) + { + luaC_objbarrier(L, hvalue(obj), mt); + } + else + { + luaC_objbarriert(L, hvalue(obj), mt); + } + } break; } case LUA_TUSERDATA: diff --git a/VM/src/lbuiltins.cpp b/VM/src/lbuiltins.cpp index 34e9ebc1..ecc14e87 100644 --- a/VM/src/lbuiltins.cpp +++ b/VM/src/lbuiltins.cpp @@ -1087,6 +1087,34 @@ static int luauF_countrz(lua_State* L, StkId res, TValue* arg0, int nresults, St return -1; } +static int luauF_select(lua_State* L, StkId res, TValue* arg0, int nresults, StkId args, int nparams) +{ + if (nparams == 1 && nresults == 1) + { + int n = cast_int(L->base - L->ci->func) - clvalue(L->ci->func)->l.p->numparams - 1; + + if (ttisnumber(arg0)) + { + int i = int(nvalue(arg0)); + + // i >= 1 && i <= n + if (unsigned(i - 1) <= unsigned(n)) + { + setobj2s(L, res, L->base - n + (i - 1)); + return 1; + } + // note: for now we don't handle negative case (wrap around) and defer to fallback + } + else if (ttisstring(arg0) && *svalue(arg0) == '#') + { + setnvalue(res, double(n)); + return 1; + } + } + + return -1; +} + luau_FastFunction luauF_table[256] = { NULL, luauF_assert, @@ -1156,4 +1184,6 @@ luau_FastFunction luauF_table[256] = { luauF_countlz, luauF_countrz, + + luauF_select, }; diff --git a/VM/src/lcorolib.cpp b/VM/src/lcorolib.cpp index abcde779..19222861 100644 --- a/VM/src/lcorolib.cpp +++ b/VM/src/lcorolib.cpp @@ -5,8 +5,6 @@ #include "lstate.h" #include "lvm.h" -LUAU_FASTFLAGVARIABLE(LuauCoroutineClose, false) - #define CO_RUN 0 /* running */ #define CO_SUS 1 /* suspended */ #define CO_NOR 2 /* 'normal' (it resumed another coroutine) */ @@ -235,9 +233,6 @@ static int coyieldable(lua_State* L) static int coclose(lua_State* L) { - if (!FFlag::LuauCoroutineClose) - luaL_error(L, "coroutine.close is not enabled"); - lua_State* co = lua_tothread(L, 1); luaL_argexpected(L, co, 1, "thread"); diff --git a/VM/src/ldo.cpp b/VM/src/ldo.cpp index 581506a8..a3982bc6 100644 --- a/VM/src/ldo.cpp +++ b/VM/src/ldo.cpp @@ -17,8 +17,6 @@ #include -LUAU_FASTFLAG(LuauCoroutineClose) - /* ** {====================================================== ** Error-recovery functions @@ -300,7 +298,7 @@ static void resume(lua_State* L, void* ud) { // start coroutine LUAU_ASSERT(L->ci == L->base_ci && firstArg >= L->base); - if (FFlag::LuauCoroutineClose && firstArg == L->base) + if (firstArg == L->base) luaG_runerror(L, "cannot resume dead coroutine"); if (luau_precall(L, firstArg - 1, LUA_MULTRET) != PCRLUA) diff --git a/VM/src/lgc.cpp b/VM/src/lgc.cpp index 50859b1e..82ac0009 100644 --- a/VM/src/lgc.cpp +++ b/VM/src/lgc.cpp @@ -93,10 +93,8 @@ static void finishGcCycleStats(global_State* g) g->gcstats.lastcycle = g->gcstats.currcycle; g->gcstats.currcycle = GCCycleStats(); - g->gcstats.cyclestatsacc.markitems += g->gcstats.lastcycle.markitems; g->gcstats.cyclestatsacc.marktime += g->gcstats.lastcycle.marktime; g->gcstats.cyclestatsacc.atomictime += g->gcstats.lastcycle.atomictime; - g->gcstats.cyclestatsacc.sweepitems += g->gcstats.lastcycle.sweepitems; g->gcstats.cyclestatsacc.sweeptime += g->gcstats.lastcycle.sweeptime; } @@ -492,23 +490,22 @@ static void freeobj(lua_State* L, GCObject* o, lua_Page* page) } } -#define sweepwholelist(L, p, tc) sweeplist(L, p, SIZE_MAX, tc) +#define sweepwholelist(L, p) sweeplist(L, p, SIZE_MAX) -static GCObject** sweeplist(lua_State* L, GCObject** p, size_t count, size_t* traversedcount) +static GCObject** sweeplist(lua_State* L, GCObject** p, size_t count) { LUAU_ASSERT(!FFlag::LuauGcPagedSweep); GCObject* curr; global_State* g = L->global; int deadmask = otherwhite(g); - size_t startcount = count; LUAU_ASSERT(testbit(deadmask, FIXEDBIT)); /* make sure we never sweep fixed objects */ while ((curr = *p) != NULL && count-- > 0) { int alive = (curr->gch.marked ^ WHITEBITS) & deadmask; if (curr->gch.tt == LUA_TTHREAD) { - sweepwholelist(L, (GCObject**)&gco2th(curr)->openupval, traversedcount); /* sweep open upvalues */ + sweepwholelist(L, (GCObject**)&gco2th(curr)->openupval); /* sweep open upvalues */ lua_State* th = gco2th(curr); @@ -534,10 +531,6 @@ static GCObject** sweeplist(lua_State* L, GCObject** p, size_t count, size_t* tr } } - // if we didn't reach the end of the list it means that we've stopped because the count dropped below zero - if (traversedcount) - *traversedcount += startcount - (curr ? count + 1 : count); - return p; } @@ -721,8 +714,6 @@ static bool sweepgco(lua_State* L, lua_Page* page, GCObject* gco) int alive = (gco->gch.marked ^ WHITEBITS) & deadmask; - g->gcstats.currcycle.sweepitems++; - if (gco->gch.tt == LUA_TTHREAD) { lua_State* th = gco2th(gco); @@ -770,11 +761,11 @@ static int sweepgcopage(lua_State* L, lua_Page* page) { // if the last block was removed, page would be removed as well if (--busyBlocks == 0) - return (pos - start) / blockSize + 1; + return int(pos - start) / blockSize + 1; } } - return (end - start) / blockSize; + return int(end - start) / blockSize; } static size_t gcstep(lua_State* L, size_t limit) @@ -793,8 +784,6 @@ static size_t gcstep(lua_State* L, size_t limit) { while (g->gray && cost < limit) { - g->gcstats.currcycle.markitems++; - cost += propagatemark(g); } @@ -812,8 +801,6 @@ static size_t gcstep(lua_State* L, size_t limit) { while (g->gray && cost < limit) { - g->gcstats.currcycle.markitems++; - cost += propagatemark(g); } @@ -842,10 +829,8 @@ static size_t gcstep(lua_State* L, size_t limit) while (g->sweepstrgc < g->strt.size && cost < limit) { - size_t traversedcount = 0; - sweepwholelist(L, (GCObject**)&g->strt.hash[g->sweepstrgc++], &traversedcount); + sweepwholelist(L, (GCObject**)&g->strt.hash[g->sweepstrgc++]); - g->gcstats.currcycle.sweepitems += traversedcount; cost += GC_SWEEPCOST; } @@ -855,12 +840,10 @@ static size_t gcstep(lua_State* L, size_t limit) // sweep string buffer list and preserve used string count uint32_t nuse = L->global->strt.nuse; - size_t traversedcount = 0; - sweepwholelist(L, (GCObject**)&g->strbufgc, &traversedcount); + sweepwholelist(L, (GCObject**)&g->strbufgc); L->global->strt.nuse = nuse; - g->gcstats.currcycle.sweepitems += traversedcount; g->gcstate = GCSsweep; // end sweep-string phase } break; @@ -893,10 +876,8 @@ static size_t gcstep(lua_State* L, size_t limit) { while (*g->sweepgc && cost < limit) { - size_t traversedcount = 0; - g->sweepgc = sweeplist(L, g->sweepgc, GC_SWEEPMAX, &traversedcount); + g->sweepgc = sweeplist(L, g->sweepgc, GC_SWEEPMAX); - g->gcstats.currcycle.sweepitems += traversedcount; cost += GC_SWEEPMAX * GC_SWEEPCOST; } diff --git a/VM/src/lgc.h b/VM/src/lgc.h index 4455fec5..528d0944 100644 --- a/VM/src/lgc.h +++ b/VM/src/lgc.h @@ -113,6 +113,7 @@ luaC_barrierf(L, obj2gco(p), obj2gco(o)); \ } +// TODO: remove with FFlagLuauGcForwardMetatableBarrier #define luaC_objbarriert(L, t, o) \ { \ if (isblack(obj2gco(t)) && iswhite(obj2gco(o))) \ diff --git a/VM/src/lmem.cpp b/VM/src/lmem.cpp index 6d3b7772..e1dbce50 100644 --- a/VM/src/lmem.cpp +++ b/VM/src/lmem.cpp @@ -200,7 +200,7 @@ static lua_Page* newpage(lua_State* L, lua_Page** gcopageset, int pageSize, int global_State* g = L->global; - LUAU_ASSERT(pageSize - offsetof(lua_Page, data) >= blockSize * blockCount); + LUAU_ASSERT(pageSize - int(offsetof(lua_Page, data)) >= blockSize * blockCount); lua_Page* page = (lua_Page*)(*g->frealloc)(L, g->ud, NULL, 0, pageSize); if (!page) @@ -376,7 +376,7 @@ static void* luaM_newgcoblock(lua_State* L, int sizeClass) LUAU_ASSERT(!page->prev); LUAU_ASSERT(page->freeList || page->freeNext >= 0); - LUAU_ASSERT(size_t(page->blockSize) == kSizeClassConfig.sizeOfClass[sizeClass]); + LUAU_ASSERT(page->blockSize == kSizeClassConfig.sizeOfClass[sizeClass]); void* block; @@ -520,7 +520,7 @@ GCObject* luaM_newgco_(lua_State* L, size_t nsize, uint8_t memcat) } else { - lua_Page* page = newpage(L, &g->allgcopages, offsetof(lua_Page, data) + nsize, nsize, 1); + lua_Page* page = newpage(L, &g->allgcopages, offsetof(lua_Page, data) + int(nsize), int(nsize), 1); block = &page->data; ASAN_UNPOISON_MEMORY_REGION(block, page->blockSize); diff --git a/VM/src/lstate.h b/VM/src/lstate.h index 080f0024..0708b71f 100644 --- a/VM/src/lstate.h +++ b/VM/src/lstate.h @@ -96,9 +96,6 @@ struct GCCycleStats double sweeptime = 0.0; - size_t markitems = 0; - size_t sweepitems = 0; - size_t assistwork = 0; size_t explicitwork = 0; diff --git a/tests/AstQuery.test.cpp b/tests/AstQuery.test.cpp index 41b553b5..292625b0 100644 --- a/tests/AstQuery.test.cpp +++ b/tests/AstQuery.test.cpp @@ -44,10 +44,6 @@ TEST_CASE_FIXTURE(DocumentationSymbolFixture, "prop") TEST_CASE_FIXTURE(DocumentationSymbolFixture, "event_callback_arg") { - ScopedFastFlag sffs[] = { - {"LuauPersistDefinitionFileTypes", true}, - }; - loadDefinition(R"( declare function Connect(fn: (string) -> ()) )"); @@ -63,8 +59,6 @@ TEST_CASE_FIXTURE(DocumentationSymbolFixture, "event_callback_arg") TEST_CASE_FIXTURE(DocumentationSymbolFixture, "overloaded_fn") { - ScopedFastFlag sffs{"LuauStoreMatchingOverloadFnType", true}; - loadDefinition(R"( declare foo: ((string) -> number) & ((number) -> string) )"); diff --git a/tests/Autocomplete.test.cpp b/tests/Autocomplete.test.cpp index 211e1be1..e8e3b315 100644 --- a/tests/Autocomplete.test.cpp +++ b/tests/Autocomplete.test.cpp @@ -2626,7 +2626,6 @@ local a: A<(number, s@1> TEST_CASE_FIXTURE(ACFixture, "autocomplete_first_function_arg_expected_type") { ScopedFastFlag luauAutocompleteAvoidMutation("LuauAutocompleteAvoidMutation", true); - ScopedFastFlag luauAutocompleteFirstArg("LuauAutocompleteFirstArg", true); check(R"( local function foo1() return 1 end @@ -2728,4 +2727,39 @@ end CHECK(ac.entryMap.count("getx")); } +TEST_CASE_FIXTURE(ACFixture, "autocomplete_on_string_singletons") +{ + ScopedFastFlag sffs[] = { + {"LuauParseSingletonTypes", true}, + {"LuauSingletonTypes", true}, + {"LuauRefactorTypeVarQuestions", true}, + }; + + check(R"( + --!strict + local foo: "hello" | "bye" = "hello" + foo:@1 + )"); + + auto ac = autocomplete('1'); + + CHECK(ac.entryMap.count("format")); +} + +TEST_CASE_FIXTURE(ACFixture, "function_in_assignment_has_parentheses_2") +{ + ScopedFastFlag luauAutocompleteAvoidMutation("LuauAutocompleteAvoidMutation", true); + ScopedFastFlag preferToCallFunctionsForIntersects("PreferToCallFunctionsForIntersects", true); + + check(R"( +local bar: ((number) -> number) & (number, number) -> number) +local abc = b@1 + )"); + + auto ac = autocomplete('1'); + + CHECK(ac.entryMap.count("bar")); + CHECK(ac.entryMap["bar"].parens == ParenthesesRecommendation::CursorInside); +} + TEST_SUITE_END(); diff --git a/tests/Compiler.test.cpp b/tests/Compiler.test.cpp index 3b0d677d..4a28bdde 100644 --- a/tests/Compiler.test.cpp +++ b/tests/Compiler.test.cpp @@ -603,6 +603,37 @@ RETURN R0 1 )"); } +TEST_CASE("TableLiteralsIndexConstant") +{ + ScopedFastFlag sff("LuauCompileTableIndexOpt", true); + + // validate that we use SETTTABLEKS for constant variable keys + CHECK_EQ("\n" + compileFunction0(R"( + local a, b = "key", "value" + return {[a] = 42, [b] = 0} +)"), R"( +NEWTABLE R0 2 0 +LOADN R1 42 +SETTABLEKS R1 R0 K0 +LOADN R1 0 +SETTABLEKS R1 R0 K1 +RETURN R0 1 +)"); + + // validate that we use SETTABLEN for constant variable keys *and* that we predict array size + CHECK_EQ("\n" + compileFunction0(R"( + local a, b = 1, 2 + return {[a] = 42, [b] = 0} +)"), R"( +NEWTABLE R0 0 2 +LOADN R1 42 +SETTABLEN R1 R0 1 +LOADN R1 0 +SETTABLEN R1 R0 2 +RETURN R0 1 +)"); +} + TEST_CASE("TableSizePredictionBasic") { CHECK_EQ("\n" + compileFunction0(R"( @@ -2450,6 +2481,37 @@ return )"); } +TEST_CASE("DebugLineInfoAssignment") +{ + ScopedFastFlag sff("LuauCompileTableIndexOpt", true); + + Luau::BytecodeBuilder bcb; + bcb.setDumpFlags(Luau::BytecodeBuilder::Dump_Code | Luau::BytecodeBuilder::Dump_Lines); + Luau::compileOrThrow(bcb, R"( + local a = { b = { c = { d = 3 } } } + +a +["b"] +["c"] +["d"] = 4 +)"); + + CHECK_EQ("\n" + bcb.dumpFunction(0), R"( +2: DUPTABLE R0 1 +2: DUPTABLE R1 3 +2: DUPTABLE R2 5 +2: LOADN R3 3 +2: SETTABLEKS R3 R2 K4 +2: SETTABLEKS R2 R1 K2 +2: SETTABLEKS R1 R0 K0 +5: GETTABLEKS R2 R0 K0 +6: GETTABLEKS R1 R2 K2 +7: LOADN R2 4 +7: SETTABLEKS R2 R1 K4 +8: RETURN R0 0 +)"); +} + TEST_CASE("DebugSource") { const char* source = R"( @@ -2763,6 +2825,75 @@ RETURN R1 -1 )"); } +TEST_CASE("FastcallSelect") +{ + ScopedFastFlag sff("LuauCompileSelectBuiltin", true); + + // select(_, ...) compiles to a builtin call + CHECK_EQ("\n" + compileFunction0("return (select('#', ...))"), R"( +LOADK R1 K0 +FASTCALL1 57 R1 +3 +GETIMPORT R0 2 +GETVARARGS R2 -1 +CALL R0 -1 1 +RETURN R0 1 +)"); + + // more complex example: select inside a for loop bound + select from a iterator + CHECK_EQ("\n" + compileFunction0(R"( +local sum = 0 +for i=1, select('#', ...) do + sum += select(i, ...) +end +return sum +)"), R"( +LOADN R0 0 +LOADN R3 1 +LOADK R5 K0 +FASTCALL1 57 R5 +3 +GETIMPORT R4 2 +GETVARARGS R6 -1 +CALL R4 -1 1 +MOVE R1 R4 +LOADN R2 1 +FORNPREP R1 +7 +FASTCALL1 57 R3 +3 +GETIMPORT R4 2 +GETVARARGS R6 -1 +CALL R4 -1 1 +ADD R0 R0 R4 +FORNLOOP R1 -7 +RETURN R0 1 +)"); + + // currently we assume a single value return to avoid dealing with stack resizing + CHECK_EQ("\n" + compileFunction0("return select('#', ...)"), R"( +GETIMPORT R0 1 +LOADK R1 K2 +GETVARARGS R2 -1 +CALL R0 -1 -1 +RETURN R0 -1 +)"); + + // note that select with a non-variadic second argument doesn't get optimized + CHECK_EQ("\n" + compileFunction0("return select('#')"), R"( +GETIMPORT R0 1 +LOADK R1 K2 +CALL R0 1 -1 +RETURN R0 -1 +)"); + + // note that select with a non-variadic second argument doesn't get optimized + CHECK_EQ("\n" + compileFunction0("return select('#', foo())"), R"( +GETIMPORT R0 1 +LOADK R1 K2 +GETIMPORT R2 4 +CALL R2 0 -1 +CALL R0 -1 -1 +RETURN R0 -1 +)"); +} + TEST_CASE("LotsOfParameters") { const char* source = R"( diff --git a/tests/Conformance.test.cpp b/tests/Conformance.test.cpp index 5222af33..914b881f 100644 --- a/tests/Conformance.test.cpp +++ b/tests/Conformance.test.cpp @@ -331,8 +331,6 @@ TEST_CASE("UTF8") TEST_CASE("Coroutine") { - ScopedFastFlag sff("LuauCoroutineClose", true); - runConformance("coroutine.lua"); } diff --git a/tests/Frontend.test.cpp b/tests/Frontend.test.cpp index 405f26e0..ea1a08fe 100644 --- a/tests/Frontend.test.cpp +++ b/tests/Frontend.test.cpp @@ -956,7 +956,6 @@ TEST_CASE("no_use_after_free_with_type_fun_instantiation") { // This flag forces this test to crash if there's a UAF in this code. ScopedFastFlag sff_DebugLuauFreezeArena("DebugLuauFreezeArena", true); - ScopedFastFlag sff_LuauCloneCorrectlyBeforeMutatingTableType("LuauCloneCorrectlyBeforeMutatingTableType", true); FrontendFixture fix; diff --git a/tests/Parser.test.cpp b/tests/Parser.test.cpp index ac81005c..90831ee9 100644 --- a/tests/Parser.test.cpp +++ b/tests/Parser.test.cpp @@ -2000,6 +2000,73 @@ TEST_CASE_FIXTURE(Fixture, "parse_type_alias_default_type_errors") matchParseError("type Y number> = {}", "Expected type pack after '=', got type", Location{{0, 14}, {0, 32}}); } +TEST_CASE_FIXTURE(Fixture, "parse_if_else_expression") +{ + { + AstStat* stat = parse("return if true then 1 else 2"); + + REQUIRE(stat != nullptr); + AstStatReturn* str = stat->as()->body.data[0]->as(); + REQUIRE(str != nullptr); + CHECK(str->list.size == 1); + auto* ifElseExpr = str->list.data[0]->as(); + REQUIRE(ifElseExpr != nullptr); + } + + { + AstStat* stat = parse("return if true then 1 elseif true then 2 else 3"); + + REQUIRE(stat != nullptr); + AstStatReturn* str = stat->as()->body.data[0]->as(); + REQUIRE(str != nullptr); + CHECK(str->list.size == 1); + auto* ifElseExpr1 = str->list.data[0]->as(); + REQUIRE(ifElseExpr1 != nullptr); + auto* ifElseExpr2 = ifElseExpr1->falseExpr->as(); + REQUIRE(ifElseExpr2 != nullptr); + } + + // Use "else if" as opposed to elseif + { + AstStat* stat = parse("return if true then 1 else if true then 2 else 3"); + + REQUIRE(stat != nullptr); + AstStatReturn* str = stat->as()->body.data[0]->as(); + REQUIRE(str != nullptr); + CHECK(str->list.size == 1); + auto* ifElseExpr1 = str->list.data[0]->as(); + REQUIRE(ifElseExpr1 != nullptr); + auto* ifElseExpr2 = ifElseExpr1->falseExpr->as(); + REQUIRE(ifElseExpr2 != nullptr); + } + + // Use an if-else expression as the conditional expression of an if-else expression + { + AstStat* stat = parse("return if if true then false else true then 1 else 2"); + + REQUIRE(stat != nullptr); + AstStatReturn* str = stat->as()->body.data[0]->as(); + REQUIRE(str != nullptr); + CHECK(str->list.size == 1); + auto* ifElseExpr = str->list.data[0]->as(); + REQUIRE(ifElseExpr != nullptr); + auto* nestedIfElseExpr = ifElseExpr->condition->as(); + REQUIRE(nestedIfElseExpr != nullptr); + } +} + +TEST_CASE_FIXTURE(Fixture, "parse_type_pack_type_parameters") +{ + AstStat* stat = parse(R"( +type Packed = () -> T... + +type A = Packed +type B = Packed<...number> +type C = Packed<(number, X...)> + )"); + REQUIRE(stat != nullptr); +} + TEST_SUITE_END(); TEST_SUITE_BEGIN("ParseErrorRecovery"); @@ -2504,71 +2571,4 @@ type Y = (T...) -> U... CHECK_EQ(1, result.errors.size()); } -TEST_CASE_FIXTURE(Fixture, "parse_if_else_expression") -{ - { - AstStat* stat = parse("return if true then 1 else 2"); - - REQUIRE(stat != nullptr); - AstStatReturn* str = stat->as()->body.data[0]->as(); - REQUIRE(str != nullptr); - CHECK(str->list.size == 1); - auto* ifElseExpr = str->list.data[0]->as(); - REQUIRE(ifElseExpr != nullptr); - } - - { - AstStat* stat = parse("return if true then 1 elseif true then 2 else 3"); - - REQUIRE(stat != nullptr); - AstStatReturn* str = stat->as()->body.data[0]->as(); - REQUIRE(str != nullptr); - CHECK(str->list.size == 1); - auto* ifElseExpr1 = str->list.data[0]->as(); - REQUIRE(ifElseExpr1 != nullptr); - auto* ifElseExpr2 = ifElseExpr1->falseExpr->as(); - REQUIRE(ifElseExpr2 != nullptr); - } - - // Use "else if" as opposed to elseif - { - AstStat* stat = parse("return if true then 1 else if true then 2 else 3"); - - REQUIRE(stat != nullptr); - AstStatReturn* str = stat->as()->body.data[0]->as(); - REQUIRE(str != nullptr); - CHECK(str->list.size == 1); - auto* ifElseExpr1 = str->list.data[0]->as(); - REQUIRE(ifElseExpr1 != nullptr); - auto* ifElseExpr2 = ifElseExpr1->falseExpr->as(); - REQUIRE(ifElseExpr2 != nullptr); - } - - // Use an if-else expression as the conditional expression of an if-else expression - { - AstStat* stat = parse("return if if true then false else true then 1 else 2"); - - REQUIRE(stat != nullptr); - AstStatReturn* str = stat->as()->body.data[0]->as(); - REQUIRE(str != nullptr); - CHECK(str->list.size == 1); - auto* ifElseExpr = str->list.data[0]->as(); - REQUIRE(ifElseExpr != nullptr); - auto* nestedIfElseExpr = ifElseExpr->condition->as(); - REQUIRE(nestedIfElseExpr != nullptr); - } -} - -TEST_CASE_FIXTURE(Fixture, "parse_type_pack_type_parameters") -{ - AstStat* stat = parse(R"( -type Packed = () -> T... - -type A = Packed -type B = Packed<...number> -type C = Packed<(number, X...)> - )"); - REQUIRE(stat != nullptr); -} - TEST_SUITE_END(); diff --git a/tests/Repl.test.cpp b/tests/Repl.test.cpp new file mode 100644 index 00000000..f660bcd3 --- /dev/null +++ b/tests/Repl.test.cpp @@ -0,0 +1,117 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +#include "lua.h" +#include "lualib.h" + +#include "Repl.h" + +#include "doctest.h" + +#include +#include +#include +#include + + +class ReplFixture +{ +public: + ReplFixture() + : luaState(luaL_newstate(), lua_close) + { + L = luaState.get(); + setupState(L); + luaL_sandboxthread(L); + + std::string result = runCode(L, prettyPrintSource); + } + + // Returns all of the output captured from the pretty printer + std::string getCapturedOutput() + { + lua_getglobal(L, "capturedoutput"); + const char* str = lua_tolstring(L, -1, nullptr); + std::string result(str); + lua_pop(L, 1); + return result; + } + lua_State* L; + +private: + std::unique_ptr luaState; + + // This is a simplicitic and incomplete pretty printer. + // It is included here to test that the pretty printer hook is being called. + // More elaborate tests to ensure correct output can be added if we introduce + // a more feature rich pretty printer. + std::string prettyPrintSource = R"( +-- Accumulate pretty printer output in `capturedoutput` +capturedoutput = "" + +function arraytostring(arr) + local strings = {} + table.foreachi(arr, function(k,v) table.insert(strings, pptostring(v)) end ) + return "{" .. table.concat(strings, ", ") .. "}" +end + +function pptostring(x) + if type(x) == "table" then + -- Just assume array-like tables for now. + return arraytostring(x) + elseif type(x) == "string" then + return '"' .. x .. '"' + else + return tostring(x) + end +end + +-- Note: Instead of calling print, the pretty printer just stores the output +-- in `capturedoutput` so we can check for the correct results. +function _PRETTYPRINT(...) + local args = table.pack(...) + local strings = {} + for i=1, args.n do + local item = args[i] + local str = pptostring(item, customoptions) + if i == 1 then + capturedoutput = capturedoutput .. str + else + capturedoutput = capturedoutput .. "\t" .. str + end + end +end +)"; +}; + +TEST_SUITE_BEGIN("ReplPrettyPrint"); + +TEST_CASE_FIXTURE(ReplFixture, "AdditionStatement") +{ + runCode(L, "return 30 + 12"); + CHECK(getCapturedOutput() == "42"); +} + +TEST_CASE_FIXTURE(ReplFixture, "TableLiteral") +{ + runCode(L, "return {1, 2, 3, 4}"); + CHECK(getCapturedOutput() == "{1, 2, 3, 4}"); +} + +TEST_CASE_FIXTURE(ReplFixture, "StringLiteral") +{ + runCode(L, "return 'str'"); + CHECK(getCapturedOutput() == "\"str\""); +} + +TEST_CASE_FIXTURE(ReplFixture, "TableWithStringLiterals") +{ + runCode(L, "return {1, 'two', 3, 'four'}"); + CHECK(getCapturedOutput() == "{1, \"two\", 3, \"four\"}"); +} + +TEST_CASE_FIXTURE(ReplFixture, "MultipleArguments") +{ + runCode(L, "return 3, 'three'"); + CHECK(getCapturedOutput() == "3\t\"three\""); +} + +TEST_SUITE_END(); diff --git a/tests/ToString.test.cpp b/tests/ToString.test.cpp index 445ee532..bbb26291 100644 --- a/tests/ToString.test.cpp +++ b/tests/ToString.test.cpp @@ -435,8 +435,6 @@ TEST_CASE_FIXTURE(Fixture, "toString_the_boundTo_table_type_contained_within_a_T TEST_CASE_FIXTURE(Fixture, "no_parentheses_around_cyclic_function_type_in_union") { - ScopedFastFlag sff{"LuauOccursCheckOkWithRecursiveFunctions", true}; - CheckResult result = check(R"( type F = ((() -> number)?) -> F? local function f(p) return f end @@ -450,8 +448,6 @@ TEST_CASE_FIXTURE(Fixture, "no_parentheses_around_cyclic_function_type_in_union" TEST_CASE_FIXTURE(Fixture, "no_parentheses_around_cyclic_function_type_in_intersection") { - ScopedFastFlag sff{"LuauOccursCheckOkWithRecursiveFunctions", true}; - CheckResult result = check(R"( function f() return f end local a: ((number) -> ()) & typeof(f) diff --git a/tests/TypeInfer.aliases.test.cpp b/tests/TypeInfer.aliases.test.cpp index 822bd727..76ab23b3 100644 --- a/tests/TypeInfer.aliases.test.cpp +++ b/tests/TypeInfer.aliases.test.cpp @@ -11,8 +11,6 @@ TEST_SUITE_BEGIN("TypeAliases"); TEST_CASE_FIXTURE(Fixture, "cyclic_function_type_in_type_alias") { - ScopedFastFlag sff{"LuauOccursCheckOkWithRecursiveFunctions", true}; - CheckResult result = check(R"( type F = () -> F? local function f() @@ -194,8 +192,6 @@ TEST_CASE_FIXTURE(Fixture, "corecursive_types_generic") TEST_CASE_FIXTURE(Fixture, "corecursive_function_types") { - ScopedFastFlag sff{"LuauOccursCheckOkWithRecursiveFunctions", true}; - CheckResult result = check(R"( type A = () -> (number, B) type B = () -> (string, A) diff --git a/tests/TypeInfer.generics.test.cpp b/tests/TypeInfer.generics.test.cpp index 114679e3..a7f27551 100644 --- a/tests/TypeInfer.generics.test.cpp +++ b/tests/TypeInfer.generics.test.cpp @@ -9,8 +9,6 @@ using namespace Luau; -LUAU_FASTFLAG(LuauExtendedFunctionMismatchError) - TEST_SUITE_BEGIN("GenericsTests"); TEST_CASE_FIXTURE(Fixture, "check_generic_function") @@ -656,11 +654,7 @@ local d: D = c LUAU_REQUIRE_ERROR_COUNT(1, result); - if (FFlag::LuauExtendedFunctionMismatchError) - CHECK_EQ( - toString(result.errors[0]), R"(Type '() -> ()' could not be converted into '() -> ()'; different number of generic type parameters)"); - else - CHECK_EQ(toString(result.errors[0]), R"(Type '() -> ()' could not be converted into '() -> ()')"); + CHECK_EQ(toString(result.errors[0]), R"(Type '() -> ()' could not be converted into '() -> ()'; different number of generic type parameters)"); } TEST_CASE_FIXTURE(Fixture, "error_detailed_function_mismatch_generic_pack") @@ -675,11 +669,8 @@ local d: D = c LUAU_REQUIRE_ERROR_COUNT(1, result); - if (FFlag::LuauExtendedFunctionMismatchError) - CHECK_EQ(toString(result.errors[0]), - R"(Type '() -> ()' could not be converted into '() -> ()'; different number of generic type pack parameters)"); - else - CHECK_EQ(toString(result.errors[0]), R"(Type '() -> ()' could not be converted into '() -> ()')"); + CHECK_EQ(toString(result.errors[0]), + R"(Type '() -> ()' could not be converted into '() -> ()'; different number of generic type pack parameters)"); } TEST_SUITE_END(); diff --git a/tests/TypeInfer.provisional.test.cpp b/tests/TypeInfer.provisional.test.cpp index e6d3d4d4..47c13be9 100644 --- a/tests/TypeInfer.provisional.test.cpp +++ b/tests/TypeInfer.provisional.test.cpp @@ -271,6 +271,32 @@ TEST_CASE_FIXTURE(Fixture, "lvalue_equals_another_lvalue_with_no_overlap") CHECK_EQ(toString(requireTypeAtPosition({5, 36})), "boolean?"); // a ~= b } +// Also belongs in TypeInfer.refinements.test.cpp. +// Just needs to fully support equality refinement. Which is annoying without type states. +TEST_CASE_FIXTURE(Fixture, "discriminate_from_x_not_equal_to_nil") +{ + ScopedFastFlag sff{"LuauDiscriminableUnions", true}; + + CheckResult result = check(R"( + type T = {x: string, y: number} | {x: nil, y: nil} + + local function f(t: T) + if t.x ~= nil then + local foo = t + else + local bar = t + end + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ("{| x: string, y: number |}", toString(requireTypeAtPosition({5, 28}))); + + // Should be {| x: nil, y: nil |} + CHECK_EQ("{| x: nil, y: nil |} | {| x: string, y: number |}", toString(requireTypeAtPosition({7, 28}))); +} + TEST_CASE_FIXTURE(Fixture, "bail_early_if_unification_is_too_complicated" * doctest::timeout(0.5)) { ScopedFastInt sffi{"LuauTarjanChildLimit", 1}; @@ -590,8 +616,6 @@ TEST_CASE_FIXTURE(Fixture, "invariant_table_properties_means_instantiating_table TEST_CASE_FIXTURE(Fixture, "self_recursive_instantiated_param") { - ScopedFastFlag luauCloneCorrectlyBeforeMutatingTableType{"LuauCloneCorrectlyBeforeMutatingTableType", true}; - // Mutability in type function application right now can create strange recursive types // TODO: instantiation right now is problematic, in this example should either leave the Table type alone // or it should rename the type to 'Self' so that the result will be 'Self
' diff --git a/tests/TypeInfer.refinements.test.cpp b/tests/TypeInfer.refinements.test.cpp index d76b920b..f346ddfd 100644 --- a/tests/TypeInfer.refinements.test.cpp +++ b/tests/TypeInfer.refinements.test.cpp @@ -6,11 +6,77 @@ #include "doctest.h" +LUAU_FASTFLAG(LuauDiscriminableUnions) LUAU_FASTFLAG(LuauWeakEqConstraint) LUAU_FASTFLAG(LuauQuantifyInPlace2) using namespace Luau; +namespace +{ +std::optional> magicFunctionInstanceIsA( + TypeChecker& typeChecker, const ScopePtr& scope, const AstExprCall& expr, ExprResult exprResult) +{ + if (expr.args.size != 1) + return std::nullopt; + + auto index = expr.func->as(); + auto str = expr.args.data[0]->as(); + if (!index || !str) + return std::nullopt; + + std::optional lvalue = tryGetLValue(*index->expr); + std::optional tfun = scope->lookupType(std::string(str->value.data, str->value.size)); + if (!lvalue || !tfun) + return std::nullopt; + + unfreeze(typeChecker.globalTypes); + TypePackId booleanPack = typeChecker.globalTypes.addTypePack({typeChecker.booleanType}); + freeze(typeChecker.globalTypes); + return ExprResult{booleanPack, {IsAPredicate{std::move(*lvalue), expr.location, tfun->type}}}; +} + +struct RefinementClassFixture : Fixture +{ + RefinementClassFixture() + { + TypeArena& arena = typeChecker.globalTypes; + + unfreeze(arena); + TypeId vec3 = arena.addType(ClassTypeVar{"Vector3", {}, std::nullopt, std::nullopt, {}, nullptr}); + getMutable(vec3)->props = { + {"X", Property{typeChecker.numberType}}, + {"Y", Property{typeChecker.numberType}}, + {"Z", Property{typeChecker.numberType}}, + }; + + TypeId inst = arena.addType(ClassTypeVar{"Instance", {}, std::nullopt, std::nullopt, {}, nullptr}); + + TypePackId isAParams = arena.addTypePack({inst, typeChecker.stringType}); + TypePackId isARets = arena.addTypePack({typeChecker.booleanType}); + TypeId isA = arena.addType(FunctionTypeVar{isAParams, isARets}); + getMutable(isA)->magicFunction = magicFunctionInstanceIsA; + + getMutable(inst)->props = { + {"Name", Property{typeChecker.stringType}}, + {"IsA", Property{isA}}, + }; + + TypeId folder = typeChecker.globalTypes.addType(ClassTypeVar{"Folder", {}, inst, std::nullopt, {}, nullptr}); + TypeId part = typeChecker.globalTypes.addType(ClassTypeVar{"Part", {}, inst, std::nullopt, {}, nullptr}); + getMutable(part)->props = { + {"Position", Property{vec3}}, + }; + + typeChecker.globalScope->exportedTypeBindings["Vector3"] = TypeFun{{}, vec3}; + typeChecker.globalScope->exportedTypeBindings["Instance"] = TypeFun{{}, inst}; + typeChecker.globalScope->exportedTypeBindings["Folder"] = TypeFun{{}, folder}; + typeChecker.globalScope->exportedTypeBindings["Part"] = TypeFun{{}, part}; + freeze(typeChecker.globalTypes); + } +}; +} // namespace + TEST_SUITE_BEGIN("RefinementTest"); TEST_CASE_FIXTURE(Fixture, "is_truthy_constraint") @@ -196,8 +262,18 @@ TEST_CASE_FIXTURE(Fixture, "typeguard_only_look_up_types_from_global_scope") end )"); - LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK_EQ("Type 'number' has no overlap with 'string'", toString(result.errors[0])); + if (FFlag::LuauDiscriminableUnions) + { + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ("*unknown*", toString(requireTypeAtPosition({8, 44}))); + CHECK_EQ("*unknown*", toString(requireTypeAtPosition({9, 38}))); + } + else + { + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ("Type 'number' has no overlap with 'string'", toString(result.errors[0])); + } } TEST_CASE_FIXTURE(Fixture, "call_a_more_specific_function_using_typeguard") @@ -237,7 +313,6 @@ TEST_CASE_FIXTURE(Fixture, "impossible_type_narrow_is_not_an_error") TEST_CASE_FIXTURE(Fixture, "truthy_constraint_on_properties") { - CheckResult result = check(R"( local t: {x: number?} = {x = 1} @@ -254,7 +329,6 @@ TEST_CASE_FIXTURE(Fixture, "truthy_constraint_on_properties") TEST_CASE_FIXTURE(Fixture, "index_on_a_refined_property") { - CheckResult result = check(R"( local t: {x: {y: string}?} = {x = {y = "hello!"}} @@ -360,7 +434,10 @@ TEST_CASE_FIXTURE(Fixture, "lvalue_is_equal_to_a_term") TEST_CASE_FIXTURE(Fixture, "term_is_equal_to_an_lvalue") { - ScopedFastFlag sff1{"LuauEqConstraint", true}; + ScopedFastFlag sff[] = { + {"LuauDiscriminableUnions", true}, + {"LuauSingletonTypes", true}, + }; CheckResult result = check(R"( local function f(a: (string | number)?) @@ -374,16 +451,8 @@ TEST_CASE_FIXTURE(Fixture, "term_is_equal_to_an_lvalue") LUAU_REQUIRE_NO_ERRORS(result); - if (FFlag::LuauWeakEqConstraint) - { - CHECK_EQ(toString(requireTypeAtPosition({3, 28})), "(number | string)?"); // a == "hello" - CHECK_EQ(toString(requireTypeAtPosition({5, 28})), "(number | string)?"); // a ~= "hello" - } - else - { - CHECK_EQ(toString(requireTypeAtPosition({3, 28})), "string"); // a == "hello" - CHECK_EQ(toString(requireTypeAtPosition({5, 28})), "(number | string)?"); // a ~= "hello" - } + CHECK_EQ(toString(requireTypeAtPosition({3, 28})), R"("hello")"); // a == "hello" + CHECK_EQ(toString(requireTypeAtPosition({5, 28})), "(number | string)?"); // a ~= "hello" } TEST_CASE_FIXTURE(Fixture, "lvalue_is_not_nil") @@ -416,7 +485,8 @@ TEST_CASE_FIXTURE(Fixture, "lvalue_is_not_nil") TEST_CASE_FIXTURE(Fixture, "free_type_is_equal_to_an_lvalue") { - ScopedFastFlag sff1{"LuauEqConstraint", true}; + ScopedFastFlag sff{"LuauDiscriminableUnions", true}; + ScopedFastFlag sff2{"LuauWeakEqConstraint", true}; CheckResult result = check(R"( local function f(a, b: string?) @@ -428,16 +498,8 @@ TEST_CASE_FIXTURE(Fixture, "free_type_is_equal_to_an_lvalue") LUAU_REQUIRE_NO_ERRORS(result); - if (FFlag::LuauWeakEqConstraint) - { - CHECK_EQ(toString(requireTypeAtPosition({3, 33})), "a"); // a == b - CHECK_EQ(toString(requireTypeAtPosition({3, 36})), "string?"); // a == b - } - else - { - CHECK_EQ(toString(requireTypeAtPosition({3, 33})), "string?"); // a == b - CHECK_EQ(toString(requireTypeAtPosition({3, 36})), "string?"); // a == b - } + CHECK_EQ(toString(requireTypeAtPosition({3, 33})), "a"); // a == b + CHECK_EQ(toString(requireTypeAtPosition({3, 36})), "string?"); // a == b } TEST_CASE_FIXTURE(Fixture, "unknown_lvalue_is_not_synonymous_with_other_on_not_equal") @@ -527,9 +589,17 @@ TEST_CASE_FIXTURE(Fixture, "type_narrow_to_vector") end )"); - // This is kinda weird to see, but this actually only happens in Luau without Roblox type bindings because we don't have a Vector3 type. - LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK_EQ("Unknown type 'Vector3'", toString(result.errors[0])); + if (FFlag::LuauDiscriminableUnions) + { + LUAU_REQUIRE_NO_ERRORS(result); + } + else + { + // This is kinda weird to see, but this actually only happens in Luau without Roblox type bindings because we don't have a Vector3 type. + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ("Unknown type 'Vector3'", toString(result.errors[0])); + } + CHECK_EQ("*unknown*", toString(requireTypeAtPosition({3, 28}))); } @@ -614,214 +684,6 @@ TEST_CASE_FIXTURE(Fixture, "typeguard_narrows_for_functions") CHECK_EQ("string", toString(requireTypeAtPosition({5, 28}))); // type(x) ~= "function" } -namespace -{ -std::optional> magicFunctionInstanceIsA( - TypeChecker& typeChecker, const ScopePtr& scope, const AstExprCall& expr, ExprResult exprResult) -{ - if (expr.args.size != 1) - return std::nullopt; - - auto index = expr.func->as(); - auto str = expr.args.data[0]->as(); - if (!index || !str) - return std::nullopt; - - std::optional lvalue = tryGetLValue(*index->expr); - std::optional tfun = scope->lookupType(std::string(str->value.data, str->value.size)); - if (!lvalue || !tfun) - return std::nullopt; - - unfreeze(typeChecker.globalTypes); - TypePackId booleanPack = typeChecker.globalTypes.addTypePack({typeChecker.booleanType}); - freeze(typeChecker.globalTypes); - return ExprResult{booleanPack, {IsAPredicate{std::move(*lvalue), expr.location, tfun->type}}}; -} - -struct RefinementClassFixture : Fixture -{ - RefinementClassFixture() - { - TypeArena& arena = typeChecker.globalTypes; - - unfreeze(arena); - TypeId vec3 = arena.addType(ClassTypeVar{"Vector3", {}, std::nullopt, std::nullopt, {}, nullptr}); - getMutable(vec3)->props = { - {"X", Property{typeChecker.numberType}}, - {"Y", Property{typeChecker.numberType}}, - {"Z", Property{typeChecker.numberType}}, - }; - - TypeId inst = arena.addType(ClassTypeVar{"Instance", {}, std::nullopt, std::nullopt, {}, nullptr}); - - TypePackId isAParams = arena.addTypePack({inst, typeChecker.stringType}); - TypePackId isARets = arena.addTypePack({typeChecker.booleanType}); - TypeId isA = arena.addType(FunctionTypeVar{isAParams, isARets}); - getMutable(isA)->magicFunction = magicFunctionInstanceIsA; - - getMutable(inst)->props = { - {"Name", Property{typeChecker.stringType}}, - {"IsA", Property{isA}}, - }; - - TypeId folder = typeChecker.globalTypes.addType(ClassTypeVar{"Folder", {}, inst, std::nullopt, {}, nullptr}); - TypeId part = typeChecker.globalTypes.addType(ClassTypeVar{"Part", {}, inst, std::nullopt, {}, nullptr}); - getMutable(part)->props = { - {"Position", Property{vec3}}, - }; - - typeChecker.globalScope->exportedTypeBindings["Vector3"] = TypeFun{{}, vec3}; - typeChecker.globalScope->exportedTypeBindings["Instance"] = TypeFun{{}, inst}; - typeChecker.globalScope->exportedTypeBindings["Folder"] = TypeFun{{}, folder}; - typeChecker.globalScope->exportedTypeBindings["Part"] = TypeFun{{}, part}; - freeze(typeChecker.globalTypes); - } -}; -} // namespace - -TEST_CASE_FIXTURE(RefinementClassFixture, "typeguard_cast_free_table_to_vector") -{ - CheckResult result = check(R"( - local function f(vec) - local X, Y, Z = vec.X, vec.Y, vec.Z - - if type(vec) == "vector" then - local foo = vec - elseif typeof(vec) == "Instance" then - local foo = vec - else - local foo = vec - end - end - )"); - - LUAU_REQUIRE_ERROR_COUNT(1, result); - - CHECK_EQ("Vector3", toString(requireTypeAtPosition({5, 28}))); // type(vec) == "vector" - - if (FFlag::LuauQuantifyInPlace2) - CHECK_EQ("Type '{+ X: a, Y: b, Z: c +}' could not be converted into 'Instance'", toString(result.errors[0])); - else - CHECK_EQ("Type '{- X: a, Y: b, Z: c -}' could not be converted into 'Instance'", toString(result.errors[0])); - - CHECK_EQ("*unknown*", toString(requireTypeAtPosition({7, 28}))); // typeof(vec) == "Instance" - - if (FFlag::LuauQuantifyInPlace2) - CHECK_EQ("{+ X: a, Y: b, Z: c +}", toString(requireTypeAtPosition({9, 28}))); // type(vec) ~= "vector" and typeof(vec) ~= "Instance" - else - CHECK_EQ("{- X: a, Y: b, Z: c -}", toString(requireTypeAtPosition({9, 28}))); // type(vec) ~= "vector" and typeof(vec) ~= "Instance" -} - -TEST_CASE_FIXTURE(RefinementClassFixture, "typeguard_cast_instance_or_vector3_to_vector") -{ - CheckResult result = check(R"( - local function f(x: Instance | Vector3) - if typeof(x) == "Vector3" then - local foo = x - else - local foo = x - end - end - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - - CHECK_EQ("Vector3", toString(requireTypeAtPosition({3, 28}))); - CHECK_EQ("Instance", toString(requireTypeAtPosition({5, 28}))); -} - -TEST_CASE_FIXTURE(RefinementClassFixture, "type_narrow_for_all_the_userdata") -{ - CheckResult result = check(R"( - local function f(x: string | number | Instance | Vector3) - if type(x) == "userdata" then - local foo = x - else - local foo = x - end - end - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - - CHECK_EQ("Instance | Vector3", toString(requireTypeAtPosition({3, 28}))); - CHECK_EQ("number | string", toString(requireTypeAtPosition({5, 28}))); -} - -TEST_CASE_FIXTURE(RefinementClassFixture, "eliminate_subclasses_of_instance") -{ - CheckResult result = check(R"( - local function f(x: Part | Folder | string) - if typeof(x) == "Instance" then - local foo = x - else - local foo = x - end - end - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - - CHECK_EQ("Folder | Part", toString(requireTypeAtPosition({3, 28}))); - CHECK_EQ("string", toString(requireTypeAtPosition({5, 28}))); -} - -TEST_CASE_FIXTURE(RefinementClassFixture, "narrow_this_large_union") -{ - CheckResult result = check(R"( - local function f(x: Part | Folder | Instance | string | Vector3 | any) - if typeof(x) == "Instance" then - local foo = x - else - local foo = x - end - end - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - - CHECK_EQ("Folder | Instance | Part", toString(requireTypeAtPosition({3, 28}))); - CHECK_EQ("Vector3 | any | string", toString(requireTypeAtPosition({5, 28}))); -} - -TEST_CASE_FIXTURE(RefinementClassFixture, "x_as_any_if_x_is_instance_elseif_x_is_table") -{ - CheckResult result = check(R"( - --!nonstrict - - local function f(x) - if typeof(x) == "Instance" and x:IsA("Folder") then - local foo = x - elseif typeof(x) == "table" then - local foo = x - end - end - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - - CHECK_EQ("Folder", toString(requireTypeAtPosition({5, 28}))); - CHECK_EQ("any", toString(requireTypeAtPosition({7, 28}))); -} - -TEST_CASE_FIXTURE(RefinementClassFixture, "x_is_not_instance_or_else_not_part") -{ - CheckResult result = check(R"( - local function f(x: Part | Folder | string) - if typeof(x) ~= "Instance" or not x:IsA("Part") then - local foo = x - else - local foo = x - end - end - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - - CHECK_EQ("Folder | string", toString(requireTypeAtPosition({3, 28}))); - CHECK_EQ("Part", toString(requireTypeAtPosition({5, 28}))); -} - TEST_CASE_FIXTURE(Fixture, "type_guard_can_filter_for_intersection_of_tables") { CheckResult result = check(R"( @@ -1145,4 +1007,259 @@ TEST_CASE_FIXTURE(Fixture, "apply_refinements_on_astexprindexexpr_whose_subscrip LUAU_REQUIRE_NO_ERRORS(result); } +TEST_CASE_FIXTURE(Fixture, "discriminate_from_truthiness_of_x") +{ + ScopedFastFlag sff[] = { + {"LuauDiscriminableUnions", true}, + {"LuauParseSingletonTypes", true}, + {"LuauSingletonTypes", true}, + }; + + CheckResult result = check(R"( + type T = {tag: "missing", x: nil} | {tag: "exists", x: string} + + local function f(t: T) + if t.x then + local foo = t + else + local bar = t + end + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ(R"({| tag: "exists", x: string |})", toString(requireTypeAtPosition({5, 28}))); + CHECK_EQ(R"({| tag: "missing", x: nil |})", toString(requireTypeAtPosition({7, 28}))); +} + +TEST_CASE_FIXTURE(Fixture, "discriminate_tag") +{ + ScopedFastFlag sff[] = { + {"LuauDiscriminableUnions", true}, + {"LuauParseSingletonTypes", true}, + {"LuauSingletonTypes", true}, + }; + + CheckResult result = check(R"( + type Cat = {tag: "Cat", name: string, catfood: string} + type Dog = {tag: "Dog", name: string, dogfood: string} + type Animal = Cat | Dog + + local function f(animal: Animal) + if animal.tag == "Cat" then + local cat: Cat = animal + elseif animal.tag == "Dog" then + local dog: Dog = animal + end + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ("Cat", toString(requireTypeAtPosition({7, 33}))); + CHECK_EQ("Dog", toString(requireTypeAtPosition({9, 33}))); +} + +TEST_CASE_FIXTURE(Fixture, "apply_refinements_on_astexprindexexpr_whose_subscript_expr_is_constant_string") +{ + ScopedFastFlag sff{"LuauRefiLookupFromIndexExpr", true}; + + CheckResult result = check(R"( + type T = { [string]: { prop: number }? } + local t: T = {} + + if t["hello"] then + local foo = t["hello"].prop + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "and_or_peephole_refinement") +{ + CheckResult result = check(R"( + local function len(a: {any}) + return a and #a or nil + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(RefinementClassFixture, "discriminate_from_isa_of_x") +{ + ScopedFastFlag sff[] = { + {"LuauDiscriminableUnions", true}, + {"LuauParseSingletonTypes", true}, + {"LuauSingletonTypes", true}, + }; + + CheckResult result = check(R"( + type T = {tag: "Part", x: Part} | {tag: "Folder", x: Folder} + + local function f(t: T) + if t.x:IsA("Part") then + local foo = t + else + local bar = t + end + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ(R"({| tag: "Part", x: Part |})", toString(requireTypeAtPosition({5, 28}))); + CHECK_EQ(R"({| tag: "Folder", x: Folder |})", toString(requireTypeAtPosition({7, 28}))); +} + +TEST_CASE_FIXTURE(RefinementClassFixture, "typeguard_cast_free_table_to_vector") +{ + CheckResult result = check(R"( + local function f(vec) + local X, Y, Z = vec.X, vec.Y, vec.Z + + if type(vec) == "vector" then + local foo = vec + elseif typeof(vec) == "Instance" then + local foo = vec + else + local foo = vec + end + end + )"); + + if (FFlag::LuauDiscriminableUnions) + LUAU_REQUIRE_NO_ERRORS(result); + else + { + LUAU_REQUIRE_ERROR_COUNT(1, result); + + if (FFlag::LuauQuantifyInPlace2) + CHECK_EQ("Type '{+ X: a, Y: b, Z: c +}' could not be converted into 'Instance'", toString(result.errors[0])); + else + CHECK_EQ("Type '{- X: a, Y: b, Z: c -}' could not be converted into 'Instance'", toString(result.errors[0])); + } + + CHECK_EQ("Vector3", toString(requireTypeAtPosition({5, 28}))); // type(vec) == "vector" + + CHECK_EQ("*unknown*", toString(requireTypeAtPosition({7, 28}))); // typeof(vec) == "Instance" + + if (FFlag::LuauQuantifyInPlace2) + CHECK_EQ("{+ X: a, Y: b, Z: c +}", toString(requireTypeAtPosition({9, 28}))); // type(vec) ~= "vector" and typeof(vec) ~= "Instance" + else + CHECK_EQ("{- X: a, Y: b, Z: c -}", toString(requireTypeAtPosition({9, 28}))); // type(vec) ~= "vector" and typeof(vec) ~= "Instance" +} + +TEST_CASE_FIXTURE(RefinementClassFixture, "typeguard_cast_instance_or_vector3_to_vector") +{ + CheckResult result = check(R"( + local function f(x: Instance | Vector3) + if typeof(x) == "Vector3" then + local foo = x + else + local foo = x + end + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ("Vector3", toString(requireTypeAtPosition({3, 28}))); + CHECK_EQ("Instance", toString(requireTypeAtPosition({5, 28}))); +} + +TEST_CASE_FIXTURE(RefinementClassFixture, "type_narrow_for_all_the_userdata") +{ + CheckResult result = check(R"( + local function f(x: string | number | Instance | Vector3) + if type(x) == "userdata" then + local foo = x + else + local foo = x + end + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ("Instance | Vector3", toString(requireTypeAtPosition({3, 28}))); + CHECK_EQ("number | string", toString(requireTypeAtPosition({5, 28}))); +} + +TEST_CASE_FIXTURE(RefinementClassFixture, "eliminate_subclasses_of_instance") +{ + CheckResult result = check(R"( + local function f(x: Part | Folder | string) + if typeof(x) == "Instance" then + local foo = x + else + local foo = x + end + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ("Folder | Part", toString(requireTypeAtPosition({3, 28}))); + CHECK_EQ("string", toString(requireTypeAtPosition({5, 28}))); +} + +TEST_CASE_FIXTURE(RefinementClassFixture, "narrow_this_large_union") +{ + CheckResult result = check(R"( + local function f(x: Part | Folder | Instance | string | Vector3 | any) + if typeof(x) == "Instance" then + local foo = x + else + local foo = x + end + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ("Folder | Instance | Part", toString(requireTypeAtPosition({3, 28}))); + CHECK_EQ("Vector3 | any | string", toString(requireTypeAtPosition({5, 28}))); +} + +TEST_CASE_FIXTURE(RefinementClassFixture, "x_as_any_if_x_is_instance_elseif_x_is_table") +{ + CheckResult result = check(R"( + --!nonstrict + + local function f(x) + if typeof(x) == "Instance" and x:IsA("Folder") then + local foo = x + elseif typeof(x) == "table" then + local foo = x + end + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ("Folder", toString(requireTypeAtPosition({5, 28}))); + CHECK_EQ("any", toString(requireTypeAtPosition({7, 28}))); +} + +TEST_CASE_FIXTURE(RefinementClassFixture, "x_is_not_instance_or_else_not_part") +{ + CheckResult result = check(R"( + local function f(x: Part | Folder | string) + if typeof(x) ~= "Instance" or not x:IsA("Part") then + local foo = x + else + local foo = x + end + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ("Folder | string", toString(requireTypeAtPosition({3, 28}))); + CHECK_EQ("Part", toString(requireTypeAtPosition({5, 28}))); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.singletons.test.cpp b/tests/TypeInfer.singletons.test.cpp index 94cfb643..df365fda 100644 --- a/tests/TypeInfer.singletons.test.cpp +++ b/tests/TypeInfer.singletons.test.cpp @@ -379,9 +379,7 @@ TEST_CASE_FIXTURE(Fixture, "error_detailed_tagged_union_mismatch_string") ScopedFastFlag sffs[] = { {"LuauSingletonTypes", true}, {"LuauParseSingletonTypes", true}, - {"LuauUnionHeuristic", true}, {"LuauExpectedTypesOfProperties", true}, - {"LuauExtendedUnionMismatchError", true}, }; CheckResult result = check(R"( @@ -404,9 +402,7 @@ TEST_CASE_FIXTURE(Fixture, "error_detailed_tagged_union_mismatch_bool") ScopedFastFlag sffs[] = { {"LuauSingletonTypes", true}, {"LuauParseSingletonTypes", true}, - {"LuauUnionHeuristic", true}, {"LuauExpectedTypesOfProperties", true}, - {"LuauExtendedUnionMismatchError", true}, }; CheckResult result = check(R"( @@ -429,9 +425,7 @@ TEST_CASE_FIXTURE(Fixture, "if_then_else_expression_singleton_options") ScopedFastFlag sffs[] = { {"LuauSingletonTypes", true}, {"LuauParseSingletonTypes", true}, - {"LuauUnionHeuristic", true}, {"LuauExpectedTypesOfProperties", true}, - {"LuauExtendedUnionMismatchError", true}, {"LuauIfElseExpectedType2", true}, {"LuauIfElseBranchTypeUnion", true}, }; diff --git a/tests/TypeInfer.tables.test.cpp b/tests/TypeInfer.tables.test.cpp index 644efed7..48310921 100644 --- a/tests/TypeInfer.tables.test.cpp +++ b/tests/TypeInfer.tables.test.cpp @@ -12,8 +12,6 @@ using namespace Luau; -LUAU_FASTFLAG(LuauExtendedFunctionMismatchError) - TEST_SUITE_BEGIN("TableTests"); TEST_CASE_FIXTURE(Fixture, "basic") @@ -2075,22 +2073,11 @@ caused by: caused by: Property 'y' is not compatible. Type 'string' could not be converted into 'number')"); - if (FFlag::LuauExtendedFunctionMismatchError) - { - CHECK_EQ(toString(result.errors[1]), R"(Type 'b2' could not be converted into 'a2' + CHECK_EQ(toString(result.errors[1]), R"(Type 'b2' could not be converted into 'a2' caused by: Type '{ __call: (a, b) -> () }' could not be converted into '{ __call: (a) -> () }' caused by: Property '__call' is not compatible. Type '(a, b) -> ()' could not be converted into '(a) -> ()'; different number of generic type parameters)"); - } - else - { - CHECK_EQ(toString(result.errors[1]), R"(Type 'b2' could not be converted into 'a2' -caused by: - Type '{ __call: (a, b) -> () }' could not be converted into '{ __call: (a) -> () }' -caused by: - Property '__call' is not compatible. Type '(a, b) -> ()' could not be converted into '(a) -> ()')"); - } } TEST_CASE_FIXTURE(Fixture, "explicitly_typed_table") @@ -2166,7 +2153,6 @@ a.p = { x = 9 } TEST_CASE_FIXTURE(Fixture, "recursive_metatable_type_call") { ScopedFastFlag sff[]{ - {"LuauFixRecursiveMetatableCall", true}, {"LuauUnsealedTableLiteral", true}, }; diff --git a/tests/TypeInfer.test.cpp b/tests/TypeInfer.test.cpp index 7ee5253c..c9b30e1a 100644 --- a/tests/TypeInfer.test.cpp +++ b/tests/TypeInfer.test.cpp @@ -16,7 +16,6 @@ LUAU_FASTFLAG(LuauFixLocationSpanTableIndexExpr) LUAU_FASTFLAG(LuauEqConstraint) -LUAU_FASTFLAG(LuauExtendedFunctionMismatchError) using namespace Luau; @@ -959,8 +958,6 @@ TEST_CASE_FIXTURE(Fixture, "another_recursive_local_function") TEST_CASE_FIXTURE(Fixture, "cyclic_function_type_in_rets") { - ScopedFastFlag sff{"LuauOccursCheckOkWithRecursiveFunctions", true}; - CheckResult result = check(R"( function f() return f @@ -973,8 +970,6 @@ TEST_CASE_FIXTURE(Fixture, "cyclic_function_type_in_rets") TEST_CASE_FIXTURE(Fixture, "cyclic_function_type_in_args") { - ScopedFastFlag sff{"LuauOccursCheckOkWithRecursiveFunctions", true}; - CheckResult result = check(R"( function f(g) return f(f) @@ -1699,8 +1694,6 @@ TEST_CASE_FIXTURE(Fixture, "first_argument_can_be_optional") TEST_CASE_FIXTURE(Fixture, "dont_ice_when_failing_the_occurs_check") { - ScopedFastFlag sff{"LuauOccursCheckOkWithRecursiveFunctions", true}; - CheckResult result = check(R"( --!strict local s @@ -1711,8 +1704,6 @@ TEST_CASE_FIXTURE(Fixture, "dont_ice_when_failing_the_occurs_check") TEST_CASE_FIXTURE(Fixture, "occurs_check_does_not_recurse_forever_if_asked_to_traverse_a_cyclic_type") { - ScopedFastFlag sff{"LuauOccursCheckOkWithRecursiveFunctions", true}; - CheckResult result = check(R"( --!strict function u(t, w) @@ -3326,11 +3317,12 @@ TEST_CASE_FIXTURE(Fixture, "unknown_type_in_comparison") LUAU_REQUIRE_NO_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "relation_op_on_any_lhs_where_rhs_maybe_has_metatable") +TEST_CASE_FIXTURE(Fixture, "concat_op_on_free_lhs_and_string_rhs") { CheckResult result = check(R"( - local x - print((x == true and (x .. "y")) .. 1) + local function f(x) + return x .. "y" + end )"); LUAU_REQUIRE_ERROR_COUNT(1, result); @@ -3340,13 +3332,14 @@ TEST_CASE_FIXTURE(Fixture, "relation_op_on_any_lhs_where_rhs_maybe_has_metatable TEST_CASE_FIXTURE(Fixture, "concat_op_on_string_lhs_and_free_rhs") { CheckResult result = check(R"( - local x - print("foo" .. x) + local function f(x) + return "foo" .. x + end )"); LUAU_REQUIRE_NO_ERRORS(result); - CHECK_EQ("string", toString(requireType("x"))); + CHECK_EQ("(string) -> string", toString(requireType("f"))); } TEST_CASE_FIXTURE(Fixture, "strict_binary_op_where_lhs_unknown") @@ -4374,8 +4367,6 @@ TEST_CASE_FIXTURE(Fixture, "recursive_types_restriction_not_ok") TEST_CASE_FIXTURE(Fixture, "record_matching_overload") { - ScopedFastFlag sffs("LuauStoreMatchingOverloadFnType", true); - CheckResult result = check(R"( type Overload = ((string) -> string) & ((number) -> number) local abc: Overload @@ -4475,17 +4466,10 @@ f(function(a, b, c, ...) return a + b end) LUAU_REQUIRE_ERRORS(result); - if (FFlag::LuauExtendedFunctionMismatchError) - { - CHECK_EQ(R"(Type '(number, number, a) -> number' could not be converted into '(number, number) -> number' + CHECK_EQ(R"(Type '(number, number, a) -> number' could not be converted into '(number, number) -> number' caused by: Argument count mismatch. Function expects 3 arguments, but only 2 are specified)", - toString(result.errors[0])); - } - else - { - CHECK_EQ(R"(Type '(number, number, a) -> number' could not be converted into '(number, number) -> number')", toString(result.errors[0])); - } + toString(result.errors[0])); // Infer from variadic packs into elements result = check(R"( @@ -4618,17 +4602,9 @@ local c = sumrec(function(x, y, f) return f(x, y) end) -- type binders are not i )"); LUAU_REQUIRE_ERRORS(result); - if (FFlag::LuauExtendedFunctionMismatchError) - { - CHECK_EQ( - "Type '(a, b, (a, b) -> (c...)) -> (c...)' could not be converted into '(a, a, (a, a) -> a) -> a'; different number of generic type " - "parameters", - toString(result.errors[0])); - } - else - { - CHECK_EQ("Type '(a, b, (a, b) -> (c...)) -> (c...)' could not be converted into '(a, a, (a, a) -> a) -> a'", toString(result.errors[0])); - } + CHECK_EQ("Type '(a, b, (a, b) -> (c...)) -> (c...)' could not be converted into '(a, a, (a, a) -> a) -> a'; different number of generic type " + "parameters", + toString(result.errors[0])); } TEST_CASE_FIXTURE(Fixture, "infer_return_value_type") @@ -4741,8 +4717,6 @@ TEST_CASE_FIXTURE(Fixture, "accidentally_checked_prop_in_opposite_branch") TEST_CASE_FIXTURE(Fixture, "substitution_with_bound_table") { - ScopedFastFlag luauCloneCorrectlyBeforeMutatingTableType{"LuauCloneCorrectlyBeforeMutatingTableType", true}; - CheckResult result = check(R"( type A = { x: number } local a: A = { x = 1 } @@ -4965,8 +4939,6 @@ TEST_CASE_FIXTURE(Fixture, "inferred_methods_of_free_tables_have_the_same_level_ TEST_CASE_FIXTURE(Fixture, "error_detailed_function_mismatch_arg_count") { - ScopedFastFlag luauExtendedFunctionMismatchError{"LuauExtendedFunctionMismatchError", true}; - CheckResult result = check(R"( type A = (number, number) -> string type B = (number) -> string @@ -4983,8 +4955,6 @@ caused by: TEST_CASE_FIXTURE(Fixture, "error_detailed_function_mismatch_arg") { - ScopedFastFlag luauExtendedFunctionMismatchError{"LuauExtendedFunctionMismatchError", true}; - CheckResult result = check(R"( type A = (number, number) -> string type B = (number, string) -> string @@ -5001,8 +4971,6 @@ caused by: TEST_CASE_FIXTURE(Fixture, "error_detailed_function_mismatch_ret_count") { - ScopedFastFlag luauExtendedFunctionMismatchError{"LuauExtendedFunctionMismatchError", true}; - CheckResult result = check(R"( type A = (number, number) -> (number) type B = (number, number) -> (number, boolean) @@ -5019,8 +4987,6 @@ caused by: TEST_CASE_FIXTURE(Fixture, "error_detailed_function_mismatch_ret") { - ScopedFastFlag luauExtendedFunctionMismatchError{"LuauExtendedFunctionMismatchError", true}; - CheckResult result = check(R"( type A = (number, number) -> string type B = (number, number) -> number @@ -5037,8 +5003,6 @@ caused by: TEST_CASE_FIXTURE(Fixture, "error_detailed_function_mismatch_ret_mult") { - ScopedFastFlag luauExtendedFunctionMismatchError{"LuauExtendedFunctionMismatchError", true}; - CheckResult result = check(R"( type A = (number, number) -> (number, string) type B = (number, number) -> (number, boolean) @@ -5069,8 +5033,6 @@ TEST_CASE_FIXTURE(Fixture, "prop_access_on_any_with_other_options") TEST_CASE_FIXTURE(Fixture, "table_function_check_use_after_free") { - ScopedFastFlag luauUnifyFunctionCheckResult{"LuauUpdateFunctionNameBinding", true}; - CheckResult result = check(R"( local t = {} diff --git a/tests/TypeInfer.typePacks.cpp b/tests/TypeInfer.typePacks.cpp index d4878d14..079870f5 100644 --- a/tests/TypeInfer.typePacks.cpp +++ b/tests/TypeInfer.typePacks.cpp @@ -931,7 +931,6 @@ type R = { m: F } TEST_CASE_FIXTURE(Fixture, "pack_tail_unification_check") { ScopedFastFlag luauUnifyPackTails{"LuauUnifyPackTails", true}; - ScopedFastFlag luauExtendedFunctionMismatchError{"LuauExtendedFunctionMismatchError", true}; CheckResult result = check(R"( local a: () -> (number, ...string) diff --git a/tests/TypeInfer.unionTypes.test.cpp b/tests/TypeInfer.unionTypes.test.cpp index b54ba996..759794e6 100644 --- a/tests/TypeInfer.unionTypes.test.cpp +++ b/tests/TypeInfer.unionTypes.test.cpp @@ -464,8 +464,6 @@ local a: XYZ = { w = 4 } TEST_CASE_FIXTURE(Fixture, "error_detailed_optional") { - ScopedFastFlag luauExtendedUnionMismatchError{"LuauExtendedUnionMismatchError", true}; - CheckResult result = check(R"( type X = { x: number } diff --git a/tests/TypeVar.test.cpp b/tests/TypeVar.test.cpp index 2e0d149e..329e7b1f 100644 --- a/tests/TypeVar.test.cpp +++ b/tests/TypeVar.test.cpp @@ -268,8 +268,6 @@ TEST_CASE_FIXTURE(Fixture, "substitution_skip_failure") TEST_CASE("tagging_tables") { - ScopedFastFlag sff{"LuauRefactorTagging", true}; - TypeVar ttv{TableTypeVar{}}; CHECK(!Luau::hasTag(&ttv, "foo")); Luau::attachTag(&ttv, "foo"); @@ -278,8 +276,6 @@ TEST_CASE("tagging_tables") TEST_CASE("tagging_classes") { - ScopedFastFlag sff{"LuauRefactorTagging", true}; - TypeVar base{ClassTypeVar{"Base", {}, std::nullopt, std::nullopt, {}, nullptr}}; CHECK(!Luau::hasTag(&base, "foo")); Luau::attachTag(&base, "foo"); @@ -288,8 +284,6 @@ TEST_CASE("tagging_classes") TEST_CASE("tagging_subclasses") { - ScopedFastFlag sff{"LuauRefactorTagging", true}; - TypeVar base{ClassTypeVar{"Base", {}, std::nullopt, std::nullopt, {}, nullptr}}; TypeVar derived{ClassTypeVar{"Derived", {}, &base, std::nullopt, {}, nullptr}}; @@ -307,8 +301,6 @@ TEST_CASE("tagging_subclasses") TEST_CASE("tagging_functions") { - ScopedFastFlag sff{"LuauRefactorTagging", true}; - TypePackVar empty{TypePack{}}; TypeVar ftv{FunctionTypeVar{&empty, &empty}}; CHECK(!Luau::hasTag(&ftv, "foo")); @@ -318,8 +310,6 @@ TEST_CASE("tagging_functions") TEST_CASE("tagging_props") { - ScopedFastFlag sff{"LuauRefactorTagging", true}; - Property prop{}; CHECK(!Luau::hasTag(prop, "foo")); Luau::attachTag(prop, "foo"); @@ -370,4 +360,66 @@ local b: (T, T, T) -> T CHECK_EQ(count, 1); } +TEST_CASE("isString_on_string_singletons") +{ + ScopedFastFlag sff{"LuauRefactorTypeVarQuestions", true}; + + TypeVar helloString{SingletonTypeVar{StringSingleton{"hello"}}}; + CHECK(isString(&helloString)); +} + +TEST_CASE("isString_on_unions_of_various_string_singletons") +{ + ScopedFastFlag sff{"LuauRefactorTypeVarQuestions", true}; + + TypeVar helloString{SingletonTypeVar{StringSingleton{"hello"}}}; + TypeVar byeString{SingletonTypeVar{StringSingleton{"bye"}}}; + TypeVar union_{UnionTypeVar{{&helloString, &byeString}}}; + + CHECK(isString(&union_)); +} + +TEST_CASE("proof_that_isString_uses_all_of") +{ + ScopedFastFlag sff{"LuauRefactorTypeVarQuestions", true}; + + TypeVar helloString{SingletonTypeVar{StringSingleton{"hello"}}}; + TypeVar byeString{SingletonTypeVar{StringSingleton{"bye"}}}; + TypeVar booleanType{PrimitiveTypeVar{PrimitiveTypeVar::Boolean}}; + TypeVar union_{UnionTypeVar{{&helloString, &byeString, &booleanType}}}; + + CHECK(!isString(&union_)); +} + +TEST_CASE("isBoolean_on_boolean_singletons") +{ + ScopedFastFlag sff{"LuauRefactorTypeVarQuestions", true}; + + TypeVar trueBool{SingletonTypeVar{BooleanSingleton{true}}}; + CHECK(isBoolean(&trueBool)); +} + +TEST_CASE("isBoolean_on_unions_of_true_or_false_singletons") +{ + ScopedFastFlag sff{"LuauRefactorTypeVarQuestions", true}; + + TypeVar trueBool{SingletonTypeVar{BooleanSingleton{true}}}; + TypeVar falseBool{SingletonTypeVar{BooleanSingleton{false}}}; + TypeVar union_{UnionTypeVar{{&trueBool, &falseBool}}}; + + CHECK(isBoolean(&union_)); +} + +TEST_CASE("proof_that_isBoolean_uses_all_of") +{ + ScopedFastFlag sff{"LuauRefactorTypeVarQuestions", true}; + + TypeVar trueBool{SingletonTypeVar{BooleanSingleton{true}}}; + TypeVar falseBool{SingletonTypeVar{BooleanSingleton{false}}}; + TypeVar stringType{PrimitiveTypeVar{PrimitiveTypeVar::String}}; + TypeVar union_{UnionTypeVar{{&trueBool, &falseBool, &stringType}}}; + + CHECK(!isBoolean(&union_)); +} + TEST_SUITE_END(); From 78039f45355c10064bfb635ca6b316b2e9303294 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Thu, 27 Jan 2022 13:52:56 -0800 Subject: [PATCH 140/399] Thanks gcc, we know you can't compile code. --- Analysis/src/TypeInfer.cpp | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/Analysis/src/TypeInfer.cpp b/Analysis/src/TypeInfer.cpp index 23fcc2d5..d6b3b5b3 100644 --- a/Analysis/src/TypeInfer.cpp +++ b/Analysis/src/TypeInfer.cpp @@ -5929,7 +5929,9 @@ void TypeChecker::resolve(const EqPredicate& eqP, ErrorVec& errVec, RefinementMa if (!sense || canUnify(eqP.type, option, eqP.location).empty()) return sense ? eqP.type : option; - return std::nullopt; + // local variable works around an odd gcc 9.3 warning: may be used uninitialized + std::optional res = std::nullopt; + return res; } return option; From 2f989fc049772f36de1a4281834c375858507bda Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Thu, 27 Jan 2022 15:46:05 -0800 Subject: [PATCH 141/399] Sync to upstream/release/512 (#330) - Improve refinement support for unions, in particular it's now possible to implement tagged unions as a union of tables where individual branches use a string literal type for one of the fields. - Fix `string.split` type information - Optimize `select(_, ...)` to run in constant time (~2.7x faster on VariadicSelect benchmark) - Improve debug line information for multi-line assignments - Improve compilation of table literals when table keys are constant expressions/variables - Use forward GC barrier for `setmetatable` which slightly accelerates GC progress --- Analysis/include/Luau/AstQuery.h | 15 + Analysis/include/Luau/Module.h | 8 + Analysis/include/Luau/TypeInfer.h | 21 +- Analysis/include/Luau/TypeVar.h | 8 +- Analysis/include/Luau/Unifier.h | 7 + Analysis/src/Autocomplete.cpp | 105 ++--- Analysis/src/Frontend.cpp | 7 +- Analysis/src/Module.cpp | 28 +- Analysis/src/ToString.cpp | 11 +- Analysis/src/TypeAttach.cpp | 2 +- Analysis/src/TypeInfer.cpp | 486 +++++++++++++--------- Analysis/src/TypeVar.cpp | 138 ++++--- Analysis/src/Unifier.cpp | 319 ++++++--------- CLI/Analyze.cpp | 2 +- CLI/FileUtils.cpp | 6 +- CLI/Repl.cpp | 15 +- CLI/Repl.h | 12 + CLI/ReplEntry.cpp | 10 + CMakeLists.txt | 12 + Compiler/include/Luau/Bytecode.h | 3 + Compiler/src/Builtins.cpp | 5 + Compiler/src/Compiler.cpp | 353 ++++++++++------ Makefile | 7 +- Sources.cmake | 18 +- VM/src/lapi.cpp | 13 +- VM/src/lbuiltins.cpp | 30 ++ VM/src/lcorolib.cpp | 5 - VM/src/ldo.cpp | 4 +- VM/src/lgc.cpp | 31 +- VM/src/lgc.h | 1 + VM/src/lstate.h | 3 - tests/AstQuery.test.cpp | 6 - tests/Autocomplete.test.cpp | 36 +- tests/Compiler.test.cpp | 131 ++++++ tests/Conformance.test.cpp | 2 - tests/Frontend.test.cpp | 1 - tests/Parser.test.cpp | 134 +++--- tests/Repl.test.cpp | 117 ++++++ tests/ToString.test.cpp | 4 - tests/TypeInfer.aliases.test.cpp | 4 - tests/TypeInfer.generics.test.cpp | 15 +- tests/TypeInfer.provisional.test.cpp | 28 +- tests/TypeInfer.refinements.test.cpp | 591 ++++++++++++++++----------- tests/TypeInfer.singletons.test.cpp | 6 - tests/TypeInfer.tables.test.cpp | 16 +- tests/TypeInfer.test.cpp | 64 +-- tests/TypeInfer.typePacks.cpp | 1 - tests/TypeInfer.unionTypes.test.cpp | 2 - tests/TypeVar.test.cpp | 72 +++- 49 files changed, 1782 insertions(+), 1133 deletions(-) create mode 100644 CLI/Repl.h create mode 100644 CLI/ReplEntry.cpp create mode 100644 tests/Repl.test.cpp diff --git a/Analysis/include/Luau/AstQuery.h b/Analysis/include/Luau/AstQuery.h index d38976ef..dfe373a5 100644 --- a/Analysis/include/Luau/AstQuery.h +++ b/Analysis/include/Luau/AstQuery.h @@ -42,6 +42,21 @@ struct ExprOrLocal { return expr ? expr->location : (local ? local->location : std::optional{}); } + std::optional getName() + { + if (expr) + { + if (AstName name = getIdentifier(expr); name.value) + { + return name; + } + } + else if (local) + { + return local->name; + } + return std::nullopt; + } private: AstExpr* expr = nullptr; diff --git a/Analysis/include/Luau/Module.h b/Analysis/include/Luau/Module.h index 2e41674b..1bf0473c 100644 --- a/Analysis/include/Luau/Module.h +++ b/Analysis/include/Luau/Module.h @@ -13,6 +13,8 @@ #include #include +LUAU_FASTFLAG(LuauPrepopulateUnionOptionsBeforeAllocation) + namespace Luau { @@ -58,6 +60,12 @@ struct TypeArena template TypeId addType(T tv) { + if (FFlag::LuauPrepopulateUnionOptionsBeforeAllocation) + { + if constexpr (std::is_same_v) + LUAU_ASSERT(tv.options.size() >= 2); + } + return addTV(TypeVar(std::move(tv))); } diff --git a/Analysis/include/Luau/TypeInfer.h b/Analysis/include/Luau/TypeInfer.h index aa090014..b843509d 100644 --- a/Analysis/include/Luau/TypeInfer.h +++ b/Analysis/include/Luau/TypeInfer.h @@ -135,7 +135,8 @@ struct TypeChecker void checkBlock(const ScopePtr& scope, const AstStatBlock& statement); void checkBlockTypeAliases(const ScopePtr& scope, std::vector& sorted); - ExprResult checkExpr(const ScopePtr& scope, const AstExpr& expr, std::optional expectedType = std::nullopt); + ExprResult checkExpr( + const ScopePtr& scope, const AstExpr& expr, std::optional expectedType = std::nullopt, bool forceSingleton = false); ExprResult checkExpr(const ScopePtr& scope, const AstExprLocal& expr); ExprResult checkExpr(const ScopePtr& scope, const AstExprGlobal& expr); ExprResult checkExpr(const ScopePtr& scope, const AstExprVarargs& expr); @@ -160,14 +161,12 @@ struct TypeChecker // Returns the type of the lvalue. TypeId checkLValue(const ScopePtr& scope, const AstExpr& expr); - // Returns both the type of the lvalue and its binding (if the caller wants to mutate the binding). - // Note: the binding may be null. - // TODO: remove second return value with FFlagLuauUpdateFunctionNameBinding - std::pair checkLValueBinding(const ScopePtr& scope, const AstExpr& expr); - std::pair checkLValueBinding(const ScopePtr& scope, const AstExprLocal& expr); - std::pair checkLValueBinding(const ScopePtr& scope, const AstExprGlobal& expr); - std::pair checkLValueBinding(const ScopePtr& scope, const AstExprIndexName& expr); - std::pair checkLValueBinding(const ScopePtr& scope, const AstExprIndexExpr& expr); + // Returns the type of the lvalue. + TypeId checkLValueBinding(const ScopePtr& scope, const AstExpr& expr); + TypeId checkLValueBinding(const ScopePtr& scope, const AstExprLocal& expr); + TypeId checkLValueBinding(const ScopePtr& scope, const AstExprGlobal& expr); + TypeId checkLValueBinding(const ScopePtr& scope, const AstExprIndexName& expr); + TypeId checkLValueBinding(const ScopePtr& scope, const AstExprIndexExpr& expr); TypeId checkFunctionName(const ScopePtr& scope, AstExpr& funName, TypeLevel level); std::pair checkFunctionSignature(const ScopePtr& scope, int subLevel, const AstExprFunction& expr, @@ -322,8 +321,6 @@ private: return addTV(TypeVar(tv)); } - TypeId addType(const UnionTypeVar& utv); - TypeId addTV(TypeVar&& tv); TypePackId addTypePack(TypePackVar&& tp); @@ -349,6 +346,8 @@ public: ErrorVec resolve(const PredicateVec& predicates, const ScopePtr& scope, bool sense); private: + void refineLValue(const LValue& lvalue, RefinementMap& refis, const ScopePtr& scope, TypeIdPredicate predicate); + std::optional resolveLValue(const ScopePtr& scope, const LValue& lvalue); std::optional DEPRECATED_resolveLValue(const ScopePtr& scope, const LValue& lvalue); std::optional resolveLValue(const RefinementMap& refis, const ScopePtr& scope, const LValue& lvalue); diff --git a/Analysis/include/Luau/TypeVar.h b/Analysis/include/Luau/TypeVar.h index 3f5e26d6..11dc9377 100644 --- a/Analysis/include/Luau/TypeVar.h +++ b/Analysis/include/Luau/TypeVar.h @@ -111,16 +111,16 @@ struct PrimitiveTypeVar // Singleton types https://github.com/Roblox/luau/blob/master/rfcs/syntax-singleton-types.md // Types for true and false -struct BoolSingleton +struct BooleanSingleton { bool value; - bool operator==(const BoolSingleton& rhs) const + bool operator==(const BooleanSingleton& rhs) const { return value == rhs.value; } - bool operator!=(const BoolSingleton& rhs) const + bool operator!=(const BooleanSingleton& rhs) const { return !(*this == rhs); } @@ -145,7 +145,7 @@ struct StringSingleton // No type for float singletons, partly because === isn't any equalivalence on floats // (NaN != NaN). -using SingletonVariant = Luau::Variant; +using SingletonVariant = Luau::Variant; struct SingletonTypeVar { diff --git a/Analysis/include/Luau/Unifier.h b/Analysis/include/Luau/Unifier.h index a3be739a..1b1671c0 100644 --- a/Analysis/include/Luau/Unifier.h +++ b/Analysis/include/Luau/Unifier.h @@ -85,6 +85,13 @@ public: Unifier makeChildUnifier(); + // A utility function that appends the given error to the unifier's error log. + // This allows setting a breakpoint wherever the unifier reports an error. + void reportError(TypeError error) + { + errors.push_back(error); + } + private: bool isNonstrictMode() const; diff --git a/Analysis/src/Autocomplete.cpp b/Analysis/src/Autocomplete.cpp index 7a801f97..85099e12 100644 --- a/Analysis/src/Autocomplete.cpp +++ b/Analysis/src/Autocomplete.cpp @@ -14,9 +14,9 @@ LUAU_FASTFLAG(LuauUseCommittingTxnLog) LUAU_FASTFLAGVARIABLE(LuauAutocompleteAvoidMutation, false); -LUAU_FASTFLAGVARIABLE(LuauAutocompleteFirstArg, false); LUAU_FASTFLAGVARIABLE(LuauCompleteBrokenStringParams, false); LUAU_FASTFLAGVARIABLE(LuauMissingFollowACMetatables, false); +LUAU_FASTFLAGVARIABLE(PreferToCallFunctionsForIntersects, false); static const std::unordered_set kStatementStartingKeywords = { "while", "if", "local", "repeat", "function", "do", "for", "return", "break", "continue", "type", "export"}; @@ -194,8 +194,6 @@ static ParenthesesRecommendation getParenRecommendation(TypeId id, const std::ve static std::optional findExpectedTypeAt(const Module& module, AstNode* node, Position position) { - LUAU_ASSERT(FFlag::LuauAutocompleteFirstArg); - auto expr = node->asExpr(); if (!expr) return std::nullopt; @@ -266,43 +264,63 @@ static TypeCorrectKind checkTypeCorrectKind(const Module& module, TypeArena* typ } }; - TypeId expectedType; + auto typeAtPosition = findExpectedTypeAt(module, node, position); - if (FFlag::LuauAutocompleteFirstArg) + if (!typeAtPosition) + return TypeCorrectKind::None; + + TypeId expectedType = follow(*typeAtPosition); + + if (FFlag::PreferToCallFunctionsForIntersects) { - auto typeAtPosition = findExpectedTypeAt(module, node, position); + auto checkFunctionType = [&canUnify, &expectedType](const FunctionTypeVar* ftv) { + auto [retHead, retTail] = flatten(ftv->retType); - if (!typeAtPosition) - return TypeCorrectKind::None; + if (!retHead.empty() && canUnify(retHead.front(), expectedType)) + return true; - expectedType = follow(*typeAtPosition); + // We might only have a variadic tail pack, check if the element is compatible + if (retTail) + { + if (const VariadicTypePack* vtp = get(follow(*retTail)); vtp && canUnify(vtp->ty, expectedType)) + return true; + } + + return false; + }; + + // We also want to suggest functions that return compatible result + if (const FunctionTypeVar* ftv = get(ty); ftv && checkFunctionType(ftv)) + { + return TypeCorrectKind::CorrectFunctionResult; + } + else if (const IntersectionTypeVar* itv = get(ty)) + { + for (TypeId id : itv->parts) + { + if (const FunctionTypeVar* ftv = get(id); ftv && checkFunctionType(ftv)) + { + return TypeCorrectKind::CorrectFunctionResult; + } + } + } } else { - auto expr = node->asExpr(); - if (!expr) - return TypeCorrectKind::None; - - auto it = module.astExpectedTypes.find(expr); - if (!it) - return TypeCorrectKind::None; - - expectedType = follow(*it); - } - - // We also want to suggest functions that return compatible result - if (const FunctionTypeVar* ftv = get(ty)) - { - auto [retHead, retTail] = flatten(ftv->retType); - - if (!retHead.empty() && canUnify(retHead.front(), expectedType)) - return TypeCorrectKind::CorrectFunctionResult; - - // We might only have a variadic tail pack, check if the element is compatible - if (retTail) + // We also want to suggest functions that return compatible result + if (const FunctionTypeVar* ftv = get(ty)) { - if (const VariadicTypePack* vtp = get(follow(*retTail)); vtp && canUnify(vtp->ty, expectedType)) + auto [retHead, retTail] = flatten(ftv->retType); + + if (!retHead.empty() && canUnify(retHead.front(), expectedType)) return TypeCorrectKind::CorrectFunctionResult; + + // We might only have a variadic tail pack, check if the element is compatible + if (retTail) + { + if (const VariadicTypePack* vtp = get(follow(*retTail)); vtp && canUnify(vtp->ty, expectedType)) + return TypeCorrectKind::CorrectFunctionResult; + } } } @@ -741,29 +759,12 @@ std::optional returnFirstNonnullOptionOfType(const UnionTypeVar* utv) static std::optional functionIsExpectedAt(const Module& module, AstNode* node, Position position) { - TypeId expectedType; + auto typeAtPosition = findExpectedTypeAt(module, node, position); - if (FFlag::LuauAutocompleteFirstArg) - { - auto typeAtPosition = findExpectedTypeAt(module, node, position); + if (!typeAtPosition) + return std::nullopt; - if (!typeAtPosition) - return std::nullopt; - - expectedType = follow(*typeAtPosition); - } - else - { - auto expr = node->asExpr(); - if (!expr) - return std::nullopt; - - auto it = module.astExpectedTypes.find(expr); - if (!it) - return std::nullopt; - - expectedType = follow(*it); - } + TypeId expectedType = follow(*typeAtPosition); if (get(expectedType)) return true; diff --git a/Analysis/src/Frontend.cpp b/Analysis/src/Frontend.cpp index fe4b6529..9001b19d 100644 --- a/Analysis/src/Frontend.cpp +++ b/Analysis/src/Frontend.cpp @@ -18,7 +18,6 @@ LUAU_FASTFLAG(LuauInferInNoCheckMode) LUAU_FASTFLAGVARIABLE(LuauTypeCheckTwice, false) LUAU_FASTFLAGVARIABLE(LuauKnowsTheDataModel3, false) -LUAU_FASTFLAGVARIABLE(LuauPersistDefinitionFileTypes, false) namespace Luau { @@ -102,8 +101,7 @@ LoadDefinitionFileResult loadDefinitionFile(TypeChecker& typeChecker, ScopePtr t generateDocumentationSymbols(globalTy, documentationSymbol); targetScope->bindings[typeChecker.globalNames.names->getOrAdd(name.c_str())] = {globalTy, Location(), false, {}, documentationSymbol}; - if (FFlag::LuauPersistDefinitionFileTypes) - persist(globalTy); + persist(globalTy); } for (const auto& [name, ty] : checkedModule->getModuleScope()->exportedTypeBindings) @@ -113,8 +111,7 @@ LoadDefinitionFileResult loadDefinitionFile(TypeChecker& typeChecker, ScopePtr t generateDocumentationSymbols(globalTy.type, documentationSymbol); targetScope->exportedTypeBindings[name] = globalTy; - if (FFlag::LuauPersistDefinitionFileTypes) - persist(globalTy.type); + persist(globalTy.type); } return LoadDefinitionFileResult{true, parseResult, checkedModule}; diff --git a/Analysis/src/Module.cpp b/Analysis/src/Module.cpp index 9f352f4b..4fdff8f7 100644 --- a/Analysis/src/Module.cpp +++ b/Analysis/src/Module.cpp @@ -16,6 +16,8 @@ LUAU_FASTFLAGVARIABLE(DebugLuauTrackOwningArena, false) LUAU_FASTINTVARIABLE(LuauTypeCloneRecursionLimit, 300) LUAU_FASTFLAG(LuauTypeAliasDefaults) +LUAU_FASTFLAGVARIABLE(LuauPrepopulateUnionOptionsBeforeAllocation, false) + namespace Luau { @@ -377,14 +379,28 @@ void TypeCloner::operator()(const AnyTypeVar& t) void TypeCloner::operator()(const UnionTypeVar& t) { - TypeId result = dest.addType(UnionTypeVar{}); - seenTypes[typeId] = result; + if (FFlag::LuauPrepopulateUnionOptionsBeforeAllocation) + { + std::vector options; + options.reserve(t.options.size()); - UnionTypeVar* option = getMutable(result); - LUAU_ASSERT(option != nullptr); + for (TypeId ty : t.options) + options.push_back(clone(ty, dest, seenTypes, seenTypePacks, cloneState)); - for (TypeId ty : t.options) - option->options.push_back(clone(ty, dest, seenTypes, seenTypePacks, cloneState)); + TypeId result = dest.addType(UnionTypeVar{std::move(options)}); + seenTypes[typeId] = result; + } + else + { + TypeId result = dest.addType(UnionTypeVar{}); + seenTypes[typeId] = result; + + UnionTypeVar* option = getMutable(result); + LUAU_ASSERT(option != nullptr); + + for (TypeId ty : t.options) + option->options.push_back(clone(ty, dest, seenTypes, seenTypePacks, cloneState)); + } } void TypeCloner::operator()(const IntersectionTypeVar& t) diff --git a/Analysis/src/ToString.cpp b/Analysis/src/ToString.cpp index 4b898d3a..5e79b841 100644 --- a/Analysis/src/ToString.cpp +++ b/Analysis/src/ToString.cpp @@ -10,7 +10,6 @@ #include #include -LUAU_FASTFLAG(LuauOccursCheckOkWithRecursiveFunctions) LUAU_FASTFLAG(LuauTypeAliasDefaults) /* @@ -374,7 +373,7 @@ struct TypeVarStringifier void operator()(TypeId, const SingletonTypeVar& stv) { - if (const BoolSingleton* bs = Luau::get(&stv)) + if (const BooleanSingleton* bs = Luau::get(&stv)) state.emit(bs->value ? "true" : "false"); else if (const StringSingleton* ss = Luau::get(&stv)) { @@ -617,9 +616,7 @@ struct TypeVarStringifier std::string saved = std::move(state.result.name); - bool needParens = FFlag::LuauOccursCheckOkWithRecursiveFunctions - ? !state.cycleNames.count(el) && (get(el) || get(el)) - : get(el) || get(el); + bool needParens = !state.cycleNames.count(el) && (get(el) || get(el)); if (needParens) state.emit("("); @@ -675,9 +672,7 @@ struct TypeVarStringifier std::string saved = std::move(state.result.name); - bool needParens = FFlag::LuauOccursCheckOkWithRecursiveFunctions - ? !state.cycleNames.count(el) && (get(el) || get(el)) - : get(el) || get(el); + bool needParens = !state.cycleNames.count(el) && (get(el) || get(el)); if (needParens) state.emit("("); diff --git a/Analysis/src/TypeAttach.cpp b/Analysis/src/TypeAttach.cpp index 2ec02093..2208213f 100644 --- a/Analysis/src/TypeAttach.cpp +++ b/Analysis/src/TypeAttach.cpp @@ -97,7 +97,7 @@ public: AstType* operator()(const SingletonTypeVar& stv) { - if (const BoolSingleton* bs = get(&stv)) + if (const BooleanSingleton* bs = get(&stv)) return allocator->alloc(Location(), bs->value); else if (const StringSingleton* ss = get(&stv)) { diff --git a/Analysis/src/TypeInfer.cpp b/Analysis/src/TypeInfer.cpp index e2d8a4fb..d6b3b5b3 100644 --- a/Analysis/src/TypeInfer.cpp +++ b/Analysis/src/TypeInfer.cpp @@ -26,8 +26,6 @@ LUAU_FASTFLAG(LuauKnowsTheDataModel3) LUAU_FASTFLAGVARIABLE(LuauEqConstraint, false) LUAU_FASTFLAGVARIABLE(LuauGroupExpectedType, false) LUAU_FASTFLAGVARIABLE(LuauWeakEqConstraint, false) // Eventually removed as false. -LUAU_FASTFLAGVARIABLE(LuauCloneCorrectlyBeforeMutatingTableType, false) -LUAU_FASTFLAGVARIABLE(LuauStoreMatchingOverloadFnType, false) LUAU_FASTFLAG(LuauUseCommittingTxnLog) LUAU_FASTFLAGVARIABLE(DebugLuauFreezeDuringUnification, false) LUAU_FASTFLAGVARIABLE(LuauRecursiveTypeParameterRestriction, false) @@ -37,6 +35,7 @@ LUAU_FASTFLAGVARIABLE(LuauLengthOnCompositeType, false) LUAU_FASTFLAGVARIABLE(LuauQuantifyInPlace2, false) LUAU_FASTFLAGVARIABLE(LuauSealExports, false) LUAU_FASTFLAGVARIABLE(LuauSingletonTypes, false) +LUAU_FASTFLAGVARIABLE(LuauDiscriminableUnions, false) LUAU_FASTFLAGVARIABLE(LuauTypeAliasDefaults, false) LUAU_FASTFLAGVARIABLE(LuauExpectedTypesOfProperties, false) LUAU_FASTFLAGVARIABLE(LuauErrorRecoveryType, false) @@ -46,10 +45,8 @@ LUAU_FASTFLAGVARIABLE(LuauRefiLookupFromIndexExpr, false) LUAU_FASTFLAGVARIABLE(LuauPerModuleUnificationCache, false) LUAU_FASTFLAGVARIABLE(LuauProperTypeLevels, false) LUAU_FASTFLAGVARIABLE(LuauAscribeCorrectLevelToInferredProperitesOfFreeTables, false) -LUAU_FASTFLAGVARIABLE(LuauFixRecursiveMetatableCall, false) LUAU_FASTFLAGVARIABLE(LuauBidirectionalAsExpr, false) LUAU_FASTFLAGVARIABLE(LuauUnsealedTableLiteral, false) -LUAU_FASTFLAGVARIABLE(LuauUpdateFunctionNameBinding, false) namespace Luau { @@ -1139,33 +1136,25 @@ void TypeChecker::check(const ScopePtr& scope, TypeId ty, const ScopePtr& funSco } else { - auto [leftType, leftTypeBinding] = checkLValueBinding(scope, *function.name); + TypeId leftType = checkLValueBinding(scope, *function.name); checkFunctionBody(funScope, ty, *function.func); unify(ty, leftType, function.location); - if (FFlag::LuauUpdateFunctionNameBinding) - { - LUAU_ASSERT(function.name->is() || function.name->is()); + LUAU_ASSERT(function.name->is() || function.name->is()); - if (auto exprIndexName = function.name->as()) + if (auto exprIndexName = function.name->as()) + { + if (auto typeIt = currentModule->astTypes.find(exprIndexName->expr)) { - if (auto typeIt = currentModule->astTypes.find(exprIndexName->expr)) + if (auto ttv = getMutableTableType(*typeIt)) { - if (auto ttv = getMutableTableType(*typeIt)) - { - if (auto it = ttv->props.find(exprIndexName->index.value); it != ttv->props.end()) - it->second.type = follow(quantify(funScope, leftType, function.name->location)); - } + if (auto it = ttv->props.find(exprIndexName->index.value); it != ttv->props.end()) + it->second.type = follow(quantify(funScope, leftType, function.name->location)); } } } - else - { - if (leftTypeBinding) - *leftTypeBinding = follow(quantify(funScope, leftType, function.name->location)); - } } } @@ -1426,7 +1415,7 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatDeclareFunction& glo currentModule->getModuleScope()->bindings[global.name] = Binding{fnType, global.location}; } -ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExpr& expr, std::optional expectedType) +ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExpr& expr, std::optional expectedType, bool forceSingleton) { RecursionCounter _rc(&checkRecursionCount); if (FInt::LuauCheckRecursionLimit > 0 && checkRecursionCount >= FInt::LuauCheckRecursionLimit) @@ -1443,14 +1432,14 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExpr& result = {nilType}; else if (const AstExprConstantBool* bexpr = expr.as()) { - if (FFlag::LuauSingletonTypes && expectedType && maybeSingleton(*expectedType)) + if (FFlag::LuauSingletonTypes && (forceSingleton || (expectedType && maybeSingleton(*expectedType)))) result = {singletonType(bexpr->value)}; else result = {booleanType}; } else if (const AstExprConstantString* sexpr = expr.as()) { - if (FFlag::LuauSingletonTypes && expectedType && maybeSingleton(*expectedType)) + if (FFlag::LuauSingletonTypes && (forceSingleton || (expectedType && maybeSingleton(*expectedType)))) result = {singletonType(std::string(sexpr->value.data, sexpr->value.size))}; else result = {stringType}; @@ -1488,15 +1477,8 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExpr& result.type = follow(result.type); - if (FFlag::LuauStoreMatchingOverloadFnType) - { - if (!currentModule->astTypes.find(&expr)) - currentModule->astTypes[&expr] = result.type; - } - else - { + if (!currentModule->astTypes.find(&expr)) currentModule->astTypes[&expr] = result.type; - } if (expectedType) currentModule->astExpectedTypes[&expr] = *expectedType; @@ -2242,7 +2224,6 @@ TypeId TypeChecker::checkRelationalOperation( state.log.commit(); } - bool needsMetamethod = !isEquality; TypeId leftType = follow(lhsType); @@ -2250,10 +2231,11 @@ TypeId TypeChecker::checkRelationalOperation( { reportErrors(state.errors); - const PrimitiveTypeVar* ptv = get(leftType); - if (!isEquality && state.errors.empty() && (get(leftType) || (ptv && ptv->type == PrimitiveTypeVar::Boolean))) + if (!isEquality && state.errors.empty() && (get(leftType) || isBoolean(leftType))) + { reportError(expr.location, GenericError{format("Type '%s' cannot be compared with relational operator %s", toString(leftType).c_str(), toString(expr.op).c_str())}); + } return booleanType; } @@ -2501,7 +2483,8 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprBi ExprResult rhs = checkExpr(innerScope, *expr.right); - return {checkBinaryOperation(innerScope, expr, lhs.type, rhs.type), {AndPredicate{std::move(lhs.predicates), std::move(rhs.predicates)}}}; + return {checkBinaryOperation(FFlag::LuauDiscriminableUnions ? scope : innerScope, expr, lhs.type, rhs.type), + {AndPredicate{std::move(lhs.predicates), std::move(rhs.predicates)}}}; } else if (expr.op == AstExprBinary::Or) { @@ -2513,7 +2496,7 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprBi ExprResult rhs = checkExpr(innerScope, *expr.right); // Because of C++, I'm not sure if lhs.predicates was not moved out by the time we call checkBinaryOperation. - TypeId result = checkBinaryOperation(innerScope, expr, lhs.type, rhs.type, lhs.predicates); + TypeId result = checkBinaryOperation(FFlag::LuauDiscriminableUnions ? scope : innerScope, expr, lhs.type, rhs.type, lhs.predicates); return {result, {OrPredicate{std::move(lhs.predicates), std::move(rhs.predicates)}}}; } else if (expr.op == AstExprBinary::CompareEq || expr.op == AstExprBinary::CompareNe) @@ -2521,8 +2504,8 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprBi if (auto predicate = tryGetTypeGuardPredicate(expr)) return {booleanType, {std::move(*predicate)}}; - ExprResult lhs = checkExpr(scope, *expr.left); - ExprResult rhs = checkExpr(scope, *expr.right); + ExprResult lhs = checkExpr(scope, *expr.left, std::nullopt, /*forceSingleton=*/FFlag::LuauDiscriminableUnions); + ExprResult rhs = checkExpr(scope, *expr.right, std::nullopt, /*forceSingleton=*/FFlag::LuauDiscriminableUnions); PredicateVec predicates; @@ -2621,11 +2604,10 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprIf TypeId TypeChecker::checkLValue(const ScopePtr& scope, const AstExpr& expr) { - auto [ty, binding] = checkLValueBinding(scope, expr); - return ty; + return checkLValueBinding(scope, expr); } -std::pair TypeChecker::checkLValueBinding(const ScopePtr& scope, const AstExpr& expr) +TypeId TypeChecker::checkLValueBinding(const ScopePtr& scope, const AstExpr& expr) { if (auto a = expr.as()) return checkLValueBinding(scope, *a); @@ -2639,22 +2621,22 @@ std::pair TypeChecker::checkLValueBinding(const ScopePtr& scope { for (AstExpr* expr : a->expressions) checkExpr(scope, *expr); - return {errorRecoveryType(scope), nullptr}; + return errorRecoveryType(scope); } else ice("Unexpected AST node in checkLValue", expr.location); } -std::pair TypeChecker::checkLValueBinding(const ScopePtr& scope, const AstExprLocal& expr) +TypeId TypeChecker::checkLValueBinding(const ScopePtr& scope, const AstExprLocal& expr) { if (std::optional ty = scope->lookup(expr.local)) - return {*ty, nullptr}; + return *ty; reportError(expr.location, UnknownSymbol{expr.local->name.value, UnknownSymbol::Binding}); - return {errorRecoveryType(scope), nullptr}; + return errorRecoveryType(scope); } -std::pair TypeChecker::checkLValueBinding(const ScopePtr& scope, const AstExprGlobal& expr) +TypeId TypeChecker::checkLValueBinding(const ScopePtr& scope, const AstExprGlobal& expr) { Name name = expr.name.value; ScopePtr moduleScope = currentModule->getModuleScope(); @@ -2662,7 +2644,7 @@ std::pair TypeChecker::checkLValueBinding(const ScopePtr& scope const auto it = moduleScope->bindings.find(expr.name); if (it != moduleScope->bindings.end()) - return std::pair(it->second.typeId, &it->second.typeId); + return it->second.typeId; TypeId result = freshType(scope); Binding& binding = moduleScope->bindings[expr.name]; @@ -2673,15 +2655,15 @@ std::pair TypeChecker::checkLValueBinding(const ScopePtr& scope if (!isNonstrictMode()) reportError(TypeError{expr.location, UnknownSymbol{name, UnknownSymbol::Binding}}); - return std::pair(result, &binding.typeId); + return result; } -std::pair TypeChecker::checkLValueBinding(const ScopePtr& scope, const AstExprIndexName& expr) +TypeId TypeChecker::checkLValueBinding(const ScopePtr& scope, const AstExprIndexName& expr) { TypeId lhs = checkExpr(scope, *expr.expr).type; if (get(lhs) || get(lhs)) - return std::pair(lhs, nullptr); + return lhs; tablify(lhs); @@ -2694,7 +2676,7 @@ std::pair TypeChecker::checkLValueBinding(const ScopePtr& scope const auto& it = lhsTable->props.find(name); if (it != lhsTable->props.end()) { - return std::pair(it->second.type, &it->second.type); + return it->second.type; } else if (lhsTable->state == TableState::Unsealed || lhsTable->state == TableState::Free) { @@ -2702,7 +2684,7 @@ std::pair TypeChecker::checkLValueBinding(const ScopePtr& scope Property& property = lhsTable->props[name]; property.type = theType; property.location = expr.indexLocation; - return std::pair(theType, &property.type); + return theType; } else if (auto indexer = lhsTable->indexer) { @@ -2720,17 +2702,17 @@ std::pair TypeChecker::checkLValueBinding(const ScopePtr& scope else if (FFlag::LuauUseCommittingTxnLog) state.log.commit(); - return std::pair(retType, nullptr); + return retType; } else if (lhsTable->state == TableState::Sealed) { reportError(TypeError{expr.location, CannotExtendTable{lhs, CannotExtendTable::Property, name}}); - return std::pair(errorRecoveryType(scope), nullptr); + return errorRecoveryType(scope); } else { reportError(TypeError{expr.location, GenericError{"Internal error: generic tables are not lvalues"}}); - return std::pair(errorRecoveryType(scope), nullptr); + return errorRecoveryType(scope); } } else if (const ClassTypeVar* lhsClass = get(lhs)) @@ -2739,29 +2721,29 @@ std::pair TypeChecker::checkLValueBinding(const ScopePtr& scope if (!prop) { reportError(TypeError{expr.location, UnknownProperty{lhs, name}}); - return std::pair(errorRecoveryType(scope), nullptr); + return errorRecoveryType(scope); } - return std::pair(prop->type, nullptr); + return prop->type; } else if (get(lhs)) { if (std::optional ty = getIndexTypeFromType(scope, lhs, name, expr.location, false)) - return std::pair(*ty, nullptr); + return *ty; // If intersection has a table part, report that it cannot be extended just as a sealed table if (isTableIntersection(lhs)) { reportError(TypeError{expr.location, CannotExtendTable{lhs, CannotExtendTable::Property, name}}); - return std::pair(errorRecoveryType(scope), nullptr); + return errorRecoveryType(scope); } } reportError(TypeError{expr.location, NotATable{lhs}}); - return std::pair(errorRecoveryType(scope), nullptr); + return errorRecoveryType(scope); } -std::pair TypeChecker::checkLValueBinding(const ScopePtr& scope, const AstExprIndexExpr& expr) +TypeId TypeChecker::checkLValueBinding(const ScopePtr& scope, const AstExprIndexExpr& expr) { TypeId exprType = checkExpr(scope, *expr.expr).type; tablify(exprType); @@ -2771,7 +2753,7 @@ std::pair TypeChecker::checkLValueBinding(const ScopePtr& scope TypeId indexType = checkExpr(scope, *expr.index).type; if (get(exprType) || get(exprType)) - return std::pair(exprType, nullptr); + return exprType; AstExprConstantString* value = expr.index->as(); @@ -2783,9 +2765,9 @@ std::pair TypeChecker::checkLValueBinding(const ScopePtr& scope if (!prop) { reportError(TypeError{expr.location, UnknownProperty{exprType, value->value.data}}); - return std::pair(errorRecoveryType(scope), nullptr); + return errorRecoveryType(scope); } - return std::pair(prop->type, nullptr); + return prop->type; } } @@ -2794,7 +2776,7 @@ std::pair TypeChecker::checkLValueBinding(const ScopePtr& scope if (!exprTable) { reportError(TypeError{expr.expr->location, NotATable{exprType}}); - return std::pair(errorRecoveryType(scope), nullptr); + return errorRecoveryType(scope); } if (value) @@ -2802,7 +2784,7 @@ std::pair TypeChecker::checkLValueBinding(const ScopePtr& scope const auto& it = exprTable->props.find(value->value.data); if (it != exprTable->props.end()) { - return std::pair(it->second.type, &it->second.type); + return it->second.type; } else if (exprTable->state == TableState::Unsealed || exprTable->state == TableState::Free) { @@ -2810,7 +2792,7 @@ std::pair TypeChecker::checkLValueBinding(const ScopePtr& scope Property& property = exprTable->props[value->value.data]; property.type = resultType; property.location = expr.index->location; - return std::pair(resultType, &property.type); + return resultType; } } @@ -2818,18 +2800,18 @@ std::pair TypeChecker::checkLValueBinding(const ScopePtr& scope { const TableIndexer& indexer = *exprTable->indexer; unify(indexType, indexer.indexType, expr.index->location); - return std::pair(indexer.indexResultType, nullptr); + return indexer.indexResultType; } else if (exprTable->state == TableState::Unsealed || exprTable->state == TableState::Free) { TypeId resultType = freshType(scope); exprTable->indexer = TableIndexer{anyIfNonstrict(indexType), anyIfNonstrict(resultType)}; - return std::pair(resultType, nullptr); + return resultType; } else { TypeId resultType = freshType(scope); - return std::pair(resultType, nullptr); + return resultType; } } @@ -3326,7 +3308,7 @@ void TypeChecker::checkArgumentList( } // ok else { - state.errors.push_back(TypeError{state.location, CountMismatch{minParams, paramIndex}}); + state.reportError(TypeError{state.location, CountMismatch{minParams, paramIndex}}); return; } ++paramIter; @@ -3348,7 +3330,7 @@ void TypeChecker::checkArgumentList( Location location = state.location; if (!argLocations.empty()) location = {state.location.begin, argLocations.back().end}; - state.errors.push_back(TypeError{location, CountMismatch{minParams, std::distance(begin(argPack), end(argPack))}}); + state.reportError(TypeError{location, CountMismatch{minParams, std::distance(begin(argPack), end(argPack))}}); return; } TypePackId tail = state.log.follow(*paramIter.tail()); @@ -3405,7 +3387,7 @@ void TypeChecker::checkArgumentList( if (!argLocations.empty()) location = {state.location.begin, argLocations.back().end}; // TODO: Better error message? - state.errors.push_back(TypeError{location, CountMismatch{minParams, std::distance(begin(argPack), end(argPack))}}); + state.reportError(TypeError{location, CountMismatch{minParams, std::distance(begin(argPack), end(argPack))}}); return; } } @@ -3520,7 +3502,7 @@ void TypeChecker::checkArgumentList( } // ok else { - state.errors.push_back(TypeError{state.location, CountMismatch{minParams, paramIndex}}); + state.reportError(TypeError{state.location, CountMismatch{minParams, paramIndex}}); return; } ++paramIter; @@ -3540,7 +3522,7 @@ void TypeChecker::checkArgumentList( Location location = state.location; if (!argLocations.empty()) location = {state.location.begin, argLocations.back().end}; - state.errors.push_back(TypeError{location, CountMismatch{minParams, std::distance(begin(argPack), end(argPack))}}); + state.reportError(TypeError{location, CountMismatch{minParams, std::distance(begin(argPack), end(argPack))}}); return; } TypePackId tail = *paramIter.tail(); @@ -3606,7 +3588,7 @@ void TypeChecker::checkArgumentList( if (!argLocations.empty()) location = {state.location.begin, argLocations.back().end}; // TODO: Better error message? - state.errors.push_back(TypeError{location, CountMismatch{minParams, std::distance(begin(argPack), end(argPack))}}); + state.reportError(TypeError{location, CountMismatch{minParams, std::distance(begin(argPack), end(argPack))}}); return; } } @@ -3825,22 +3807,11 @@ std::optional> TypeChecker::checkCallOverload(const Scope metaArgLocations = *argLocations; metaArgLocations.insert(metaArgLocations.begin(), expr.func->location); - if (FFlag::LuauFixRecursiveMetatableCall) - { - fn = instantiate(scope, *ty, expr.func->location); + fn = instantiate(scope, *ty, expr.func->location); - argPack = metaCallArgPack; - args = metaCallArgs; - argLocations = &metaArgLocations; - } - else - { - TypeId fn = *ty; - fn = instantiate(scope, fn, expr.func->location); - - return checkCallOverload(scope, expr, fn, retPack, metaCallArgPack, metaCallArgs, &metaArgLocations, argListResult, - overloadsThatMatchArgCount, overloadsThatDont, errors); - } + argPack = metaCallArgPack; + args = metaCallArgs; + argLocations = &metaArgLocations; } } @@ -3932,8 +3903,7 @@ std::optional> TypeChecker::checkCallOverload(const Scope } } - if (FFlag::LuauStoreMatchingOverloadFnType) - currentModule->astOverloadResolvedTypes[&expr] = fn; + currentModule->astOverloadResolvedTypes[&expr] = fn; // We select this overload return {{retPack}}; @@ -4776,7 +4746,7 @@ TypeId TypeChecker::freshType(TypeLevel level) TypeId TypeChecker::singletonType(bool value) { // TODO: cache singleton types - return currentModule->internalTypes.addType(TypeVar(SingletonTypeVar(BoolSingleton{value}))); + return currentModule->internalTypes.addType(TypeVar(SingletonTypeVar(BooleanSingleton{value}))); } TypeId TypeChecker::singletonType(std::string value) @@ -4813,13 +4783,6 @@ std::optional TypeChecker::filterMap(TypeId type, TypeIdPredicate predic return std::nullopt; } -TypeId TypeChecker::addType(const UnionTypeVar& utv) -{ - LUAU_ASSERT(utv.options.size() > 1); - - return addTV(TypeVar(utv)); -} - TypeId TypeChecker::addTV(TypeVar&& tv) { return currentModule->internalTypes.addType(std::move(tv)); @@ -5347,54 +5310,35 @@ TypeId TypeChecker::instantiateTypeFun(const ScopePtr& scope, const TypeFun& tf, TypeId instantiated = *maybeInstantiated; - if (FFlag::LuauCloneCorrectlyBeforeMutatingTableType) + // TODO: CLI-46926 it's not a good idea to rename the type here + TypeId target = follow(instantiated); + bool needsClone = follow(tf.type) == target; + TableTypeVar* ttv = getMutableTableType(target); + + if (ttv && needsClone) { - // TODO: CLI-46926 it's not a good idea to rename the type here - TypeId target = follow(instantiated); - bool needsClone = follow(tf.type) == target; - TableTypeVar* ttv = getMutableTableType(target); - - if (ttv && needsClone) + // Substitution::clone is a shallow clone. If this is a metatable type, we + // want to mutate its table, so we need to explicitly clone that table as + // well. If we don't, we will mutate another module's type surface and cause + // a use-after-free. + if (get(target)) { - // Substitution::clone is a shallow clone. If this is a metatable type, we - // want to mutate its table, so we need to explicitly clone that table as - // well. If we don't, we will mutate another module's type surface and cause - // a use-after-free. - if (get(target)) - { - instantiated = applyTypeFunction.clone(tf.type); - MetatableTypeVar* mtv = getMutable(instantiated); - mtv->table = applyTypeFunction.clone(mtv->table); - ttv = getMutable(mtv->table); - } - if (get(target)) - { - instantiated = applyTypeFunction.clone(tf.type); - ttv = getMutable(instantiated); - } + instantiated = applyTypeFunction.clone(tf.type); + MetatableTypeVar* mtv = getMutable(instantiated); + mtv->table = applyTypeFunction.clone(mtv->table); + ttv = getMutable(mtv->table); } - - if (ttv) + if (get(target)) { - ttv->instantiatedTypeParams = typeParams; - ttv->instantiatedTypePackParams = typePackParams; + instantiated = applyTypeFunction.clone(tf.type); + ttv = getMutable(instantiated); } } - else - { - if (TableTypeVar* ttv = getMutableTableType(instantiated)) - { - if (follow(tf.type) == instantiated) - { - // This can happen if a type alias has generics that it does not use at all. - // ex type FooBar = { a: number } - instantiated = applyTypeFunction.clone(tf.type); - ttv = getMutableTableType(instantiated); - } - ttv->instantiatedTypeParams = typeParams; - ttv->instantiatedTypePackParams = typePackParams; - } + if (ttv) + { + ttv->instantiatedTypeParams = typeParams; + ttv->instantiatedTypePackParams = typePackParams; } return instantiated; @@ -5482,6 +5426,85 @@ GenericTypeDefinitions TypeChecker::createGenericTypes(const ScopePtr& scope, st return {generics, genericPacks}; } +void TypeChecker::refineLValue(const LValue& lvalue, RefinementMap& refis, const ScopePtr& scope, TypeIdPredicate predicate) +{ + LUAU_ASSERT(FFlag::LuauDiscriminableUnions); + + const LValue* target = &lvalue; + std::optional key; // If set, we know we took the base of the lvalue path and should be walking down each option of the base's type. + + auto ty = resolveLValue(scope, *target); + if (!ty) + return; // Do nothing. An error was already reported. + + // If the provided lvalue is a local or global, then that's without a doubt the target. + // However, if there is a base lvalue, then we'll want that to be the target iff the base is a union type. + if (auto base = baseof(lvalue)) + { + std::optional baseTy = resolveLValue(scope, *base); + if (baseTy && get(follow(*baseTy))) + { + ty = baseTy; + target = base; + key = lvalue; + } + } + + // If we do not have a key, it means we're not trying to discriminate anything, so it's a simple matter of just filtering for a subset. + if (!key) + { + if (std::optional result = filterMap(*ty, predicate)) + addRefinement(refis, *target, *result); + else + addRefinement(refis, *target, errorRecoveryType(scope)); + + return; + } + + // Otherwise, we'll want to walk each option of ty, get its index type, and filter that. + auto utv = get(follow(*ty)); + LUAU_ASSERT(utv); + + std::unordered_set viableTargetOptions; + std::unordered_set viableChildOptions; // There may be additional refinements that apply. We add those here too. + + for (TypeId option : utv) + { + std::optional discriminantTy; + if (auto field = Luau::get(*key)) // need to fully qualify Luau::get because of ADL. + discriminantTy = getIndexTypeFromType(scope, option, field->key, Location(), false); + else + LUAU_ASSERT(!"Unhandled LValue alternative?"); + + if (!discriminantTy) + return; // Do nothing. An error was already reported, as per usual. + + if (std::optional result = filterMap(*discriminantTy, predicate)) + { + viableTargetOptions.insert(option); + viableChildOptions.insert(*result); + } + } + + auto intoType = [this](const std::unordered_set& s) -> std::optional { + if (s.empty()) + return std::nullopt; + + // TODO: allocate UnionTypeVar and just normalize. + std::vector options(s.begin(), s.end()); + if (options.size() == 1) + return options[0]; + + return addType(UnionTypeVar{std::move(options)}); + }; + + if (std::optional viableTargetType = intoType(viableTargetOptions)) + addRefinement(refis, *target, *viableTargetType); + + if (std::optional viableChildType = intoType(viableChildOptions)) + addRefinement(refis, lvalue, *viableChildType); +} + std::optional TypeChecker::resolveLValue(const ScopePtr& scope, const LValue& lvalue) { if (!FFlag::LuauLValueAsKey) @@ -5645,18 +5668,29 @@ void TypeChecker::resolve(const TruthyPredicate& truthyP, ErrorVec& errVec, Refi return std::nullopt; }; - std::optional ty = resolveLValue(refis, scope, truthyP.lvalue); - if (!ty) - return; + if (FFlag::LuauDiscriminableUnions) + { + std::optional ty = resolveLValue(refis, scope, truthyP.lvalue); + if (ty && fromOr) + return addRefinement(refis, truthyP.lvalue, *ty); - // This is a hack. :( - // Without this, the expression 'a or b' might refine 'b' to be falsy. - // I'm not yet sure how else to get this to do the right thing without this hack, so we'll do this for now in the meantime. - if (fromOr) - return addRefinement(refis, truthyP.lvalue, *ty); + refineLValue(truthyP.lvalue, refis, scope, predicate); + } + else + { + std::optional ty = resolveLValue(refis, scope, truthyP.lvalue); + if (!ty) + return; - if (std::optional result = filterMap(*ty, predicate)) - addRefinement(refis, truthyP.lvalue, *result); + // This is a hack. :( + // Without this, the expression 'a or b' might refine 'b' to be falsy. + // I'm not yet sure how else to get this to do the right thing without this hack, so we'll do this for now in the meantime. + if (fromOr) + return addRefinement(refis, truthyP.lvalue, *ty); + + if (std::optional result = filterMap(*ty, predicate)) + addRefinement(refis, truthyP.lvalue, *result); + } } void TypeChecker::resolve(const AndPredicate& andP, ErrorVec& errVec, RefinementMap& refis, const ScopePtr& scope, bool sense) @@ -5746,16 +5780,23 @@ void TypeChecker::resolve(const IsAPredicate& isaP, ErrorVec& errVec, Refinement return res; }; - std::optional ty = resolveLValue(refis, scope, isaP.lvalue); - if (!ty) - return; - - if (std::optional result = filterMap(*ty, predicate)) - addRefinement(refis, isaP.lvalue, *result); + if (FFlag::LuauDiscriminableUnions) + { + refineLValue(isaP.lvalue, refis, scope, predicate); + } else { - addRefinement(refis, isaP.lvalue, errorRecoveryType(scope)); - errVec.push_back(TypeError{isaP.location, TypeMismatch{isaP.ty, *ty}}); + std::optional ty = resolveLValue(refis, scope, isaP.lvalue); + if (!ty) + return; + + if (std::optional result = filterMap(*ty, predicate)) + addRefinement(refis, isaP.lvalue, *result); + else + { + addRefinement(refis, isaP.lvalue, errorRecoveryType(scope)); + errVec.push_back(TypeError{isaP.location, TypeMismatch{isaP.ty, *ty}}); + } } } @@ -5814,21 +5855,30 @@ void TypeChecker::resolve(const TypeGuardPredicate& typeguardP, ErrorVec& errVec if (auto it = primitives.find(typeguardP.kind); it != primitives.end()) { - if (std::optional result = filterMap(*ty, it->second(sense))) - addRefinement(refis, typeguardP.lvalue, *result); + if (FFlag::LuauDiscriminableUnions) + { + refineLValue(typeguardP.lvalue, refis, scope, it->second(sense)); + return; + } else { - addRefinement(refis, typeguardP.lvalue, errorRecoveryType(scope)); - if (sense) - errVec.push_back( - TypeError{typeguardP.location, GenericError{"Type '" + toString(*ty) + "' has no overlap with '" + typeguardP.kind + "'"}}); - } + if (std::optional result = filterMap(*ty, it->second(sense))) + addRefinement(refis, typeguardP.lvalue, *result); + else + { + addRefinement(refis, typeguardP.lvalue, errorRecoveryType(scope)); + if (sense) + errVec.push_back( + TypeError{typeguardP.location, GenericError{"Type '" + toString(*ty) + "' has no overlap with '" + typeguardP.kind + "'"}}); + } - return; + return; + } } auto fail = [&](const TypeErrorData& err) { - errVec.push_back(TypeError{typeguardP.location, err}); + if (!FFlag::LuauDiscriminableUnions) + errVec.push_back(TypeError{typeguardP.location, err}); addRefinement(refis, typeguardP.lvalue, errorRecoveryType(scope)); }; @@ -5853,55 +5903,87 @@ void TypeChecker::resolve(const TypeGuardPredicate& typeguardP, ErrorVec& errVec void TypeChecker::resolve(const EqPredicate& eqP, ErrorVec& errVec, RefinementMap& refis, const ScopePtr& scope, bool sense) { // This refinement will require success typing to do everything correctly. For now, we can get most of the way there. - auto options = [](TypeId ty) -> std::vector { if (auto utv = get(follow(ty))) return std::vector(begin(utv), end(utv)); return {ty}; }; - if (FFlag::LuauWeakEqConstraint) + if (FFlag::LuauDiscriminableUnions) { - if (!sense && isNil(eqP.type)) - resolve(TruthyPredicate{std::move(eqP.lvalue), eqP.location}, errVec, refis, scope, true, /* fromOr= */ false); - - return; - } - - if (FFlag::LuauEqConstraint) - { - std::optional ty = resolveLValue(refis, scope, eqP.lvalue); - if (!ty) - return; - - std::vector lhs = options(*ty); std::vector rhs = options(eqP.type); - if (sense && std::any_of(lhs.begin(), lhs.end(), isUndecidable)) - { - addRefinement(refis, eqP.lvalue, eqP.type); - return; - } - else if (sense && std::any_of(rhs.begin(), rhs.end(), isUndecidable)) + if (sense && std::any_of(rhs.begin(), rhs.end(), isUndecidable)) return; // Optimization: the other side has unknown types, so there's probably an overlap. Refining is no-op here. - std::unordered_set set; - for (TypeId left : lhs) - { - for (TypeId right : rhs) + auto predicate = [&](TypeId option) -> std::optional { + if (sense && isUndecidable(option)) + return FFlag::LuauWeakEqConstraint ? option : eqP.type; + + if (!sense && isNil(eqP.type)) + return (isUndecidable(option) || !isNil(option)) ? std::optional(option) : std::nullopt; + + if (maybeSingleton(eqP.type)) { - // When singleton types arrive, `isNil` here probably should be replaced with `isLiteral`. - if (canUnify(right, left, eqP.location).empty() == sense || (!sense && !isNil(left))) - set.insert(left); + // Normally we'd write option <: eqP.type, but singletons are always the subtype, so we flip this. + if (!sense || canUnify(eqP.type, option, eqP.location).empty()) + return sense ? eqP.type : option; + + // local variable works around an odd gcc 9.3 warning: may be used uninitialized + std::optional res = std::nullopt; + return res; } + + return option; + }; + + refineLValue(eqP.lvalue, refis, scope, predicate); + } + else + { + if (FFlag::LuauWeakEqConstraint) + { + if (!sense && isNil(eqP.type)) + resolve(TruthyPredicate{std::move(eqP.lvalue), eqP.location}, errVec, refis, scope, true, /* fromOr= */ false); + + return; } - if (set.empty()) - return; + if (FFlag::LuauEqConstraint) + { + std::optional ty = resolveLValue(refis, scope, eqP.lvalue); + if (!ty) + return; - std::vector viable(set.begin(), set.end()); - TypeId result = viable.size() == 1 ? viable[0] : addType(UnionTypeVar{std::move(viable)}); - addRefinement(refis, eqP.lvalue, result); + std::vector lhs = options(*ty); + std::vector rhs = options(eqP.type); + + if (sense && std::any_of(lhs.begin(), lhs.end(), isUndecidable)) + { + addRefinement(refis, eqP.lvalue, eqP.type); + return; + } + else if (sense && std::any_of(rhs.begin(), rhs.end(), isUndecidable)) + return; // Optimization: the other side has unknown types, so there's probably an overlap. Refining is no-op here. + + std::unordered_set set; + for (TypeId left : lhs) + { + for (TypeId right : rhs) + { + // When singleton types arrive, `isNil` here probably should be replaced with `isLiteral`. + if (canUnify(right, left, eqP.location).empty() == sense || (!sense && !isNil(left))) + set.insert(left); + } + } + + if (set.empty()) + return; + + std::vector viable(set.begin(), set.end()); + TypeId result = viable.size() == 1 ? viable[0] : addType(UnionTypeVar{std::move(viable)}); + addRefinement(refis, eqP.lvalue, result); + } } } diff --git a/Analysis/src/TypeVar.cpp b/Analysis/src/TypeVar.cpp index df5d76ed..5b162b31 100644 --- a/Analysis/src/TypeVar.cpp +++ b/Analysis/src/TypeVar.cpp @@ -18,14 +18,15 @@ #include #include +LUAU_FASTFLAG(DebugLuauFreezeArena) + LUAU_FASTINTVARIABLE(LuauTypeMaximumStringifierLength, 500) LUAU_FASTINTVARIABLE(LuauTableTypeMaximumStringifierLength, 0) LUAU_FASTINT(LuauTypeInferRecursionLimit) LUAU_FASTFLAG(LuauLengthOnCompositeType) LUAU_FASTFLAGVARIABLE(LuauMetatableAreEqualRecursion, false) -LUAU_FASTFLAGVARIABLE(LuauRefactorTagging, false) +LUAU_FASTFLAGVARIABLE(LuauRefactorTypeVarQuestions, false) LUAU_FASTFLAG(LuauErrorRecoveryType) -LUAU_FASTFLAG(DebugLuauFreezeArena) namespace Luau { @@ -144,7 +145,20 @@ bool isNil(TypeId ty) bool isBoolean(TypeId ty) { - return isPrim(ty, PrimitiveTypeVar::Boolean); + if (FFlag::LuauRefactorTypeVarQuestions) + { + if (isPrim(ty, PrimitiveTypeVar::Boolean) || get(get(follow(ty)))) + return true; + + if (auto utv = get(follow(ty))) + return std::all_of(begin(utv), end(utv), isBoolean); + + return false; + } + else + { + return isPrim(ty, PrimitiveTypeVar::Boolean); + } } bool isNumber(TypeId ty) @@ -154,7 +168,20 @@ bool isNumber(TypeId ty) bool isString(TypeId ty) { - return isPrim(ty, PrimitiveTypeVar::String); + if (FFlag::LuauRefactorTypeVarQuestions) + { + if (isPrim(ty, PrimitiveTypeVar::String) || get(get(follow(ty)))) + return true; + + if (auto utv = get(follow(ty))) + return std::all_of(begin(utv), end(utv), isString); + + return false; + } + else + { + return isPrim(ty, PrimitiveTypeVar::String); + } } bool isThread(TypeId ty) @@ -167,37 +194,45 @@ bool isOptional(TypeId ty) if (isNil(ty)) return true; - if (!get(follow(ty))) - return false; - - std::unordered_set seen; - std::deque queue{ty}; - while (!queue.empty()) + if (FFlag::LuauRefactorTypeVarQuestions) { - TypeId current = follow(queue.front()); - queue.pop_front(); + auto utv = get(follow(ty)); + if (!utv) + return false; - if (seen.count(current)) - continue; - - seen.insert(current); - - if (isNil(current)) - return true; - - if (auto u = get(current)) + return std::any_of(begin(utv), end(utv), isNil); + } + else + { + std::unordered_set seen; + std::deque queue{ty}; + while (!queue.empty()) { - for (TypeId option : u->options) - { - if (isNil(option)) - return true; + TypeId current = follow(queue.front()); + queue.pop_front(); - queue.push_back(option); + if (seen.count(current)) + continue; + + seen.insert(current); + + if (isNil(current)) + return true; + + if (auto u = get(current)) + { + for (TypeId option : u->options) + { + if (isNil(option)) + return true; + + queue.push_back(option); + } } } - } - return false; + return false; + } } bool isTableIntersection(TypeId ty) @@ -228,13 +263,27 @@ std::optional getMetatable(TypeId type) return mtType->metatable; else if (const ClassTypeVar* classType = get(type)) return classType->metatable; - else if (const PrimitiveTypeVar* primitiveType = get(type); primitiveType && primitiveType->metatable) + else if (FFlag::LuauRefactorTypeVarQuestions) { - LUAU_ASSERT(primitiveType->type == PrimitiveTypeVar::String); - return primitiveType->metatable; + if (isString(type)) + { + auto ptv = get(getSingletonTypes().stringType); + LUAU_ASSERT(ptv && ptv->metatable); + return ptv->metatable; + } + else + return std::nullopt; } else - return std::nullopt; + { + if (const PrimitiveTypeVar* primitiveType = get(type); primitiveType && primitiveType->metatable) + { + LUAU_ASSERT(primitiveType->type == PrimitiveTypeVar::String); + return primitiveType->metatable; + } + else + return std::nullopt; + } } const TableTypeVar* getTableType(TypeId type) @@ -696,7 +745,7 @@ TypeId SingletonTypes::makeStringMetatable() {"reverse", {stringToStringType}}, {"sub", {makeFunction(*arena, stringType, {}, {}, {numberType, optionalNumber}, {}, {stringType})}}, {"upper", {stringToStringType}}, - {"split", {makeFunction(*arena, stringType, {}, {}, {stringType, optionalString}, {}, + {"split", {makeFunction(*arena, stringType, {}, {}, {optionalString}, {}, {arena->addType(TableTypeVar{{}, TableIndexer{numberType, stringType}, TypeLevel{}})})}}, {"pack", {arena->addType(FunctionTypeVar{ arena->addTypePack(TypePack{{stringType}, anyTypePack}), @@ -1108,30 +1157,14 @@ static Tags* getTags(TypeId ty) void attachTag(TypeId ty, const std::string& tagName) { - if (!FFlag::LuauRefactorTagging) - { - if (auto ftv = getMutable(ty)) - { - ftv->tags.emplace_back(tagName); - } - else - { - LUAU_ASSERT(!"Got a non functional type"); - } - } + if (auto tags = getTags(ty)) + tags->push_back(tagName); else - { - if (auto tags = getTags(ty)) - tags->push_back(tagName); - else - LUAU_ASSERT(!"This TypeId does not support tags"); - } + LUAU_ASSERT(!"This TypeId does not support tags"); } void attachTag(Property& prop, const std::string& tagName) { - LUAU_ASSERT(FFlag::LuauRefactorTagging); - prop.tags.push_back(tagName); } @@ -1140,7 +1173,6 @@ void attachTag(Property& prop, const std::string& tagName) // Unfortunately, there's already use cases that's hard to disentangle. For now, we expose it. bool hasTag(const Tags& tags, const std::string& tagName) { - LUAU_ASSERT(FFlag::LuauRefactorTagging); return std::find(tags.begin(), tags.end(), tagName) != tags.end(); } diff --git a/Analysis/src/Unifier.cpp b/Analysis/src/Unifier.cpp index 2bd9cf83..17d9bf58 100644 --- a/Analysis/src/Unifier.cpp +++ b/Analysis/src/Unifier.cpp @@ -17,15 +17,11 @@ LUAU_FASTFLAGVARIABLE(LuauCommittingTxnLogFreeTpPromote, false) LUAU_FASTFLAG(LuauUseCommittingTxnLog) LUAU_FASTINTVARIABLE(LuauTypeInferIterationLimit, 2000); LUAU_FASTFLAGVARIABLE(LuauTableSubtypingVariance2, false); -LUAU_FASTFLAGVARIABLE(LuauUnionHeuristic, false) LUAU_FASTFLAGVARIABLE(LuauTableUnificationEarlyTest, false) -LUAU_FASTFLAGVARIABLE(LuauOccursCheckOkWithRecursiveFunctions, false) LUAU_FASTFLAG(LuauSingletonTypes) LUAU_FASTFLAG(LuauErrorRecoveryType); LUAU_FASTFLAG(LuauProperTypeLevels); LUAU_FASTFLAGVARIABLE(LuauUnifyPackTails, false) -LUAU_FASTFLAGVARIABLE(LuauExtendedUnionMismatchError, false) -LUAU_FASTFLAGVARIABLE(LuauExtendedFunctionMismatchError, false) namespace Luau { @@ -229,8 +225,6 @@ static std::optional hasUnificationTooComplex(const ErrorVec& errors) // Used for tagged union matching heuristic, returns first singleton type field static std::optional> getTableMatchTag(TypeId type) { - LUAU_ASSERT(FFlag::LuauExtendedUnionMismatchError); - type = follow(type); if (auto ttv = get(type)) @@ -291,7 +285,7 @@ void Unifier::tryUnify_(TypeId subTy, TypeId superTy, bool isFunctionCall, bool if (FInt::LuauTypeInferIterationLimit > 0 && FInt::LuauTypeInferIterationLimit < sharedState.counters.iterationCount) { - errors.push_back(TypeError{location, UnificationTooComplex{}}); + reportError(TypeError{location, UnificationTooComplex{}}); return; } @@ -403,7 +397,7 @@ void Unifier::tryUnify_(TypeId subTy, TypeId superTy, bool isFunctionCall, bool if (subGeneric && !subGeneric->level.subsumes(superLevel)) { // TODO: a more informative error message? CLI-39912 - errors.push_back(TypeError{location, GenericError{"Generic subtype escaping scope"}}); + reportError(TypeError{location, GenericError{"Generic subtype escaping scope"}}); return; } @@ -448,7 +442,7 @@ void Unifier::tryUnify_(TypeId subTy, TypeId superTy, bool isFunctionCall, bool if (superGeneric && !superGeneric->level.subsumes(subFree->level)) { // TODO: a more informative error message? CLI-39912 - errors.push_back(TypeError{location, GenericError{"Generic supertype escaping scope"}}); + reportError(TypeError{location, GenericError{"Generic supertype escaping scope"}}); return; } @@ -561,13 +555,13 @@ void Unifier::tryUnify_(TypeId subTy, TypeId superTy, bool isFunctionCall, bool } if (unificationTooComplex) - errors.push_back(*unificationTooComplex); + reportError(*unificationTooComplex); else if (failed) { if (firstFailedOption) - errors.push_back(TypeError{location, TypeMismatch{superTy, subTy, "Not all union options are compatible.", *firstFailedOption}}); + reportError(TypeError{location, TypeMismatch{superTy, subTy, "Not all union options are compatible.", *firstFailedOption}}); else - errors.push_back(TypeError{location, TypeMismatch{superTy, subTy}}); + reportError(TypeError{location, TypeMismatch{superTy, subTy}}); } } else if (const UnionTypeVar* uv = FFlag::LuauUseCommittingTxnLog ? log.getMutable(superTy) : get(superTy)) @@ -582,50 +576,44 @@ void Unifier::tryUnify_(TypeId subTy, TypeId superTy, bool isFunctionCall, bool bool foundHeuristic = false; size_t startIndex = 0; - if (FFlag::LuauUnionHeuristic) + if (const std::string* subName = getName(subTy)) { - if (const std::string* subName = getName(subTy)) + for (size_t i = 0; i < uv->options.size(); ++i) { - for (size_t i = 0; i < uv->options.size(); ++i) + const std::string* optionName = getName(uv->options[i]); + if (optionName && *optionName == *subName) { - const std::string* optionName = getName(uv->options[i]); - if (optionName && *optionName == *subName) - { - foundHeuristic = true; - startIndex = i; - break; - } + foundHeuristic = true; + startIndex = i; + break; } } + } - if (FFlag::LuauExtendedUnionMismatchError) + if (auto subMatchTag = getTableMatchTag(subTy)) + { + for (size_t i = 0; i < uv->options.size(); ++i) { - if (auto subMatchTag = getTableMatchTag(subTy)) + auto optionMatchTag = getTableMatchTag(uv->options[i]); + if (optionMatchTag && optionMatchTag->first == subMatchTag->first && *optionMatchTag->second == *subMatchTag->second) { - for (size_t i = 0; i < uv->options.size(); ++i) - { - auto optionMatchTag = getTableMatchTag(uv->options[i]); - if (optionMatchTag && optionMatchTag->first == subMatchTag->first && *optionMatchTag->second == *subMatchTag->second) - { - foundHeuristic = true; - startIndex = i; - break; - } - } + foundHeuristic = true; + startIndex = i; + break; } } + } - if (!foundHeuristic && cacheEnabled) + if (!foundHeuristic && cacheEnabled) + { + for (size_t i = 0; i < uv->options.size(); ++i) { - for (size_t i = 0; i < uv->options.size(); ++i) - { - TypeId type = uv->options[i]; + TypeId type = uv->options[i]; - if (cache.contains({type, subTy}) && (variance == Covariant || cache.contains({subTy, type}))) - { - startIndex = i; - break; - } + if (cache.contains({type, subTy}) && (variance == Covariant || cache.contains({subTy, type}))) + { + startIndex = i; + break; } } } @@ -650,7 +638,7 @@ void Unifier::tryUnify_(TypeId subTy, TypeId superTy, bool isFunctionCall, bool { unificationTooComplex = e; } - else if (FFlag::LuauExtendedUnionMismatchError && !isNil(type)) + else if (!isNil(type)) { failedOptionCount++; @@ -664,15 +652,15 @@ void Unifier::tryUnify_(TypeId subTy, TypeId superTy, bool isFunctionCall, bool if (unificationTooComplex) { - errors.push_back(*unificationTooComplex); + reportError(*unificationTooComplex); } else if (!found) { - if (FFlag::LuauExtendedUnionMismatchError && (failedOptionCount == 1 || foundHeuristic) && failedOption) - errors.push_back( + if ((failedOptionCount == 1 || foundHeuristic) && failedOption) + reportError( TypeError{location, TypeMismatch{superTy, subTy, "None of the union options are compatible. For example:", *failedOption}}); else - errors.push_back(TypeError{location, TypeMismatch{superTy, subTy, "none of the union options are compatible"}}); + reportError(TypeError{location, TypeMismatch{superTy, subTy, "none of the union options are compatible"}}); } } else if (const IntersectionTypeVar* uv = @@ -702,9 +690,9 @@ void Unifier::tryUnify_(TypeId subTy, TypeId superTy, bool isFunctionCall, bool } if (unificationTooComplex) - errors.push_back(*unificationTooComplex); + reportError(*unificationTooComplex); else if (firstFailedOption) - errors.push_back(TypeError{location, TypeMismatch{superTy, subTy, "Not all intersection parts are compatible.", *firstFailedOption}}); + reportError(TypeError{location, TypeMismatch{superTy, subTy, "Not all intersection parts are compatible.", *firstFailedOption}}); } else if (const IntersectionTypeVar* uv = FFlag::LuauUseCommittingTxnLog ? log.getMutable(subTy) : get(subTy)) @@ -754,10 +742,10 @@ void Unifier::tryUnify_(TypeId subTy, TypeId superTy, bool isFunctionCall, bool } if (unificationTooComplex) - errors.push_back(*unificationTooComplex); + reportError(*unificationTooComplex); else if (!found) { - errors.push_back(TypeError{location, TypeMismatch{superTy, subTy, "none of the intersection parts are compatible"}}); + reportError(TypeError{location, TypeMismatch{superTy, subTy, "none of the intersection parts are compatible"}}); } } else if ((FFlag::LuauUseCommittingTxnLog && log.getMutable(superTy) && log.getMutable(subTy)) || @@ -801,7 +789,7 @@ void Unifier::tryUnify_(TypeId subTy, TypeId superTy, bool isFunctionCall, bool tryUnifyWithClass(subTy, superTy, /*reversed*/ true); else - errors.push_back(TypeError{location, TypeMismatch{superTy, subTy}}); + reportError(TypeError{location, TypeMismatch{superTy, subTy}}); if (FFlag::LuauUseCommittingTxnLog) log.popSeen(superTy, subTy); @@ -1067,7 +1055,7 @@ void Unifier::tryUnify_(TypePackId subTp, TypePackId superTp, bool isFunctionCal if (FInt::LuauTypeInferIterationLimit > 0 && FInt::LuauTypeInferIterationLimit < sharedState.counters.iterationCount) { - errors.push_back(TypeError{location, UnificationTooComplex{}}); + reportError(TypeError{location, UnificationTooComplex{}}); return; } @@ -1166,7 +1154,7 @@ void Unifier::tryUnify_(TypePackId subTp, TypePackId superTp, bool isFunctionCal { tryUnify_(*subIter, *superIter); - if (FFlag::LuauExtendedFunctionMismatchError && !errors.empty() && !firstPackErrorPos) + if (!errors.empty() && !firstPackErrorPos) firstPackErrorPos = loopCount; superIter.advance(); @@ -1251,7 +1239,7 @@ void Unifier::tryUnify_(TypePackId subTp, TypePackId superTp, bool isFunctionCal size_t actualSize = size(subTp); if (ctx == CountMismatch::Result) std::swap(expectedSize, actualSize); - errors.push_back(TypeError{location, CountMismatch{expectedSize, actualSize, ctx}}); + reportError(TypeError{location, CountMismatch{expectedSize, actualSize, ctx}}); while (superIter.good()) { @@ -1272,7 +1260,7 @@ void Unifier::tryUnify_(TypePackId subTp, TypePackId superTp, bool isFunctionCal } else { - errors.push_back(TypeError{location, GenericError{"Failed to unify type packs"}}); + reportError(TypeError{location, GenericError{"Failed to unify type packs"}}); } } else @@ -1372,7 +1360,7 @@ void Unifier::tryUnify_(TypePackId subTp, TypePackId superTp, bool isFunctionCal { tryUnify_(*subIter, *superIter); - if (FFlag::LuauExtendedFunctionMismatchError && !errors.empty() && !firstPackErrorPos) + if (!errors.empty() && !firstPackErrorPos) firstPackErrorPos = loopCount; superIter.advance(); @@ -1459,7 +1447,7 @@ void Unifier::tryUnify_(TypePackId subTp, TypePackId superTp, bool isFunctionCal size_t actualSize = size(subTp); if (ctx == CountMismatch::Result) std::swap(expectedSize, actualSize); - errors.push_back(TypeError{location, CountMismatch{expectedSize, actualSize, ctx}}); + reportError(TypeError{location, CountMismatch{expectedSize, actualSize, ctx}}); while (superIter.good()) { @@ -1480,7 +1468,7 @@ void Unifier::tryUnify_(TypePackId subTp, TypePackId superTp, bool isFunctionCal } else { - errors.push_back(TypeError{location, GenericError{"Failed to unify type packs"}}); + reportError(TypeError{location, GenericError{"Failed to unify type packs"}}); } } } @@ -1493,7 +1481,7 @@ void Unifier::tryUnifyPrimitives(TypeId subTy, TypeId superTy) ice("passed non primitive types to unifyPrimitives"); if (superPrim->type != subPrim->type) - errors.push_back(TypeError{location, TypeMismatch{superTy, subTy}}); + reportError(TypeError{location, TypeMismatch{superTy, subTy}}); } void Unifier::tryUnifySingletons(TypeId subTy, TypeId superTy) @@ -1508,13 +1496,13 @@ void Unifier::tryUnifySingletons(TypeId subTy, TypeId superTy) if (superSingleton && *superSingleton == *subSingleton) return; - if (superPrim && superPrim->type == PrimitiveTypeVar::Boolean && get(subSingleton) && variance == Covariant) + if (superPrim && superPrim->type == PrimitiveTypeVar::Boolean && get(subSingleton) && variance == Covariant) return; if (superPrim && superPrim->type == PrimitiveTypeVar::String && get(subSingleton) && variance == Covariant) return; - errors.push_back(TypeError{location, TypeMismatch{superTy, subTy}}); + reportError(TypeError{location, TypeMismatch{superTy, subTy}}); } void Unifier::tryUnifyFunctions(TypeId subTy, TypeId superTy, bool isFunctionCall) @@ -1536,10 +1524,7 @@ void Unifier::tryUnifyFunctions(TypeId subTy, TypeId superTy, bool isFunctionCal { numGenerics = std::min(superFunction->generics.size(), subFunction->generics.size()); - if (FFlag::LuauExtendedFunctionMismatchError) - errors.push_back(TypeError{location, TypeMismatch{superTy, subTy, "different number of generic type parameters"}}); - else - errors.push_back(TypeError{location, TypeMismatch{superTy, subTy}}); + reportError(TypeError{location, TypeMismatch{superTy, subTy, "different number of generic type parameters"}}); } size_t numGenericPacks = superFunction->genericPacks.size(); @@ -1547,10 +1532,7 @@ void Unifier::tryUnifyFunctions(TypeId subTy, TypeId superTy, bool isFunctionCal { numGenericPacks = std::min(superFunction->genericPacks.size(), subFunction->genericPacks.size()); - if (FFlag::LuauExtendedFunctionMismatchError) - errors.push_back(TypeError{location, TypeMismatch{superTy, subTy, "different number of generic type pack parameters"}}); - else - errors.push_back(TypeError{location, TypeMismatch{superTy, subTy}}); + reportError(TypeError{location, TypeMismatch{superTy, subTy, "different number of generic type pack parameters"}}); } for (size_t i = 0; i < numGenerics; i++) @@ -1567,48 +1549,35 @@ void Unifier::tryUnifyFunctions(TypeId subTy, TypeId superTy, bool isFunctionCal { Unifier innerState = makeChildUnifier(); - if (FFlag::LuauExtendedFunctionMismatchError) + innerState.ctx = CountMismatch::Arg; + innerState.tryUnify_(superFunction->argTypes, subFunction->argTypes, isFunctionCall); + + bool reported = !innerState.errors.empty(); + + if (auto e = hasUnificationTooComplex(innerState.errors)) + reportError(*e); + else if (!innerState.errors.empty() && innerState.firstPackErrorPos) + reportError( + TypeError{location, TypeMismatch{superTy, subTy, format("Argument #%d type is not compatible.", *innerState.firstPackErrorPos), + innerState.errors.front()}}); + else if (!innerState.errors.empty()) + reportError(TypeError{location, TypeMismatch{superTy, subTy, "", innerState.errors.front()}}); + + innerState.ctx = CountMismatch::Result; + innerState.tryUnify_(subFunction->retType, superFunction->retType); + + if (!reported) { - innerState.ctx = CountMismatch::Arg; - innerState.tryUnify_(superFunction->argTypes, subFunction->argTypes, isFunctionCall); - - bool reported = !innerState.errors.empty(); - if (auto e = hasUnificationTooComplex(innerState.errors)) - errors.push_back(*e); + reportError(*e); + else if (!innerState.errors.empty() && size(superFunction->retType) == 1 && finite(superFunction->retType)) + reportError(TypeError{location, TypeMismatch{superTy, subTy, "Return type is not compatible.", innerState.errors.front()}}); else if (!innerState.errors.empty() && innerState.firstPackErrorPos) - errors.push_back( - TypeError{location, TypeMismatch{superTy, subTy, format("Argument #%d type is not compatible.", *innerState.firstPackErrorPos), + reportError( + TypeError{location, TypeMismatch{superTy, subTy, format("Return #%d type is not compatible.", *innerState.firstPackErrorPos), innerState.errors.front()}}); else if (!innerState.errors.empty()) - errors.push_back(TypeError{location, TypeMismatch{superTy, subTy, "", innerState.errors.front()}}); - - innerState.ctx = CountMismatch::Result; - innerState.tryUnify_(subFunction->retType, superFunction->retType); - - if (!reported) - { - if (auto e = hasUnificationTooComplex(innerState.errors)) - errors.push_back(*e); - else if (!innerState.errors.empty() && size(superFunction->retType) == 1 && finite(superFunction->retType)) - errors.push_back(TypeError{location, TypeMismatch{superTy, subTy, "Return type is not compatible.", innerState.errors.front()}}); - else if (!innerState.errors.empty() && innerState.firstPackErrorPos) - errors.push_back( - TypeError{location, TypeMismatch{superTy, subTy, format("Return #%d type is not compatible.", *innerState.firstPackErrorPos), - innerState.errors.front()}}); - else if (!innerState.errors.empty()) - errors.push_back(TypeError{location, TypeMismatch{superTy, subTy, "", innerState.errors.front()}}); - } - } - else - { - ctx = CountMismatch::Arg; - innerState.tryUnify_(superFunction->argTypes, subFunction->argTypes, isFunctionCall); - - ctx = CountMismatch::Result; - innerState.tryUnify_(subFunction->retType, superFunction->retType); - - checkChildUnifierTypeMismatch(innerState.errors, superTy, subTy); + reportError(TypeError{location, TypeMismatch{superTy, subTy, "", innerState.errors.front()}}); } if (FFlag::LuauUseCommittingTxnLog) @@ -1716,7 +1685,7 @@ void Unifier::tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) if (!missingProperties.empty()) { - errors.push_back(TypeError{location, MissingProperties{superTy, subTy, std::move(missingProperties)}}); + reportError(TypeError{location, MissingProperties{superTy, subTy, std::move(missingProperties)}}); return; } } @@ -1734,7 +1703,7 @@ void Unifier::tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) if (!extraProperties.empty()) { - errors.push_back(TypeError{location, MissingProperties{superTy, subTy, std::move(extraProperties), MissingProperties::Extra}}); + reportError(TypeError{location, MissingProperties{superTy, subTy, std::move(extraProperties), MissingProperties::Extra}}); return; } } @@ -1957,13 +1926,13 @@ void Unifier::tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) if (!missingProperties.empty()) { - errors.push_back(TypeError{location, MissingProperties{superTy, subTy, std::move(missingProperties)}}); + reportError(TypeError{location, MissingProperties{superTy, subTy, std::move(missingProperties)}}); return; } if (!extraProperties.empty()) { - errors.push_back(TypeError{location, MissingProperties{superTy, subTy, std::move(extraProperties), MissingProperties::Extra}}); + reportError(TypeError{location, MissingProperties{superTy, subTy, std::move(extraProperties), MissingProperties::Extra}}); return; } @@ -2051,7 +2020,7 @@ void Unifier::DEPRECATED_tryUnifyTables(TypeId subTy, TypeId superTy, bool isInt return tryUnifySealedTables(subTy, superTy, isIntersection); else if ((superTable->state == TableState::Sealed && subTable->state == TableState::Generic) || (superTable->state == TableState::Generic && subTable->state == TableState::Sealed)) - errors.push_back(TypeError{location, TypeMismatch{superTy, subTy}}); + reportError(TypeError{location, TypeMismatch{superTy, subTy}}); else if ((superTable->state == TableState::Free) != (subTable->state == TableState::Free)) // one table is free and the other is not { TypeId freeTypeId = subTable->state == TableState::Free ? subTy : superTy; @@ -2090,7 +2059,7 @@ void Unifier::DEPRECATED_tryUnifyTables(TypeId subTy, TypeId superTy, bool isInt { const auto& r = subTable->props.find(name); if (r == subTable->props.end()) - errors.push_back(TypeError{location, UnknownProperty{subTy, name}}); + reportError(TypeError{location, UnknownProperty{subTy, name}}); else tryUnify_(r->second.type, prop.type); } @@ -2113,7 +2082,7 @@ void Unifier::DEPRECATED_tryUnifyTables(TypeId subTy, TypeId superTy, bool isInt } } else - errors.push_back(TypeError{location, CannotExtendTable{subTy, CannotExtendTable::Indexer}}); + reportError(TypeError{location, CannotExtendTable{subTy, CannotExtendTable::Indexer}}); } } else if (superTable->state == TableState::Sealed) @@ -2194,7 +2163,7 @@ void Unifier::tryUnifyFreeTable(TypeId subTy, TypeId superTy) } } else - errors.push_back(TypeError{location, UnknownProperty{subTy, freeName}}); + reportError(TypeError{location, UnknownProperty{subTy, freeName}}); } } @@ -2268,7 +2237,7 @@ void Unifier::tryUnifySealedTables(TypeId subTy, TypeId superTy, bool isIntersec if (!missingPropertiesInSuper.empty()) { - errors.push_back(TypeError{location, MissingProperties{superTy, subTy, std::move(missingPropertiesInSuper)}}); + reportError(TypeError{location, MissingProperties{superTy, subTy, std::move(missingPropertiesInSuper)}}); return; } } @@ -2284,7 +2253,7 @@ void Unifier::tryUnifySealedTables(TypeId subTy, TypeId superTy, bool isIntersec missingPropertiesInSuper.push_back(it.first); - innerState.errors.push_back(TypeError{location, TypeMismatch{superTy, subTy}}); + innerState.reportError(TypeError{location, TypeMismatch{superTy, subTy}}); } else { @@ -2299,7 +2268,7 @@ void Unifier::tryUnifySealedTables(TypeId subTy, TypeId superTy, bool isIntersec if (oldErrorSize != innerState.errors.size() && !errorReported) { errorReported = true; - errors.push_back(innerState.errors.back()); + reportError(innerState.errors.back()); } } else @@ -2340,7 +2309,7 @@ void Unifier::tryUnifySealedTables(TypeId subTy, TypeId superTy, bool isIntersec } } else - innerState.errors.push_back(TypeError{location, TypeMismatch{superTy, subTy}}); + innerState.reportError(TypeError{location, TypeMismatch{superTy, subTy}}); } else { @@ -2369,7 +2338,7 @@ void Unifier::tryUnifySealedTables(TypeId subTy, TypeId superTy, bool isIntersec } } else - innerState.errors.push_back(TypeError{location, TypeMismatch{superTy, subTy}}); + innerState.reportError(TypeError{location, TypeMismatch{superTy, subTy}}); } } @@ -2386,7 +2355,7 @@ void Unifier::tryUnifySealedTables(TypeId subTy, TypeId superTy, bool isIntersec if (!missingPropertiesInSuper.empty()) { - errors.push_back(TypeError{location, MissingProperties{superTy, subTy, std::move(missingPropertiesInSuper)}}); + reportError(TypeError{location, MissingProperties{superTy, subTy, std::move(missingPropertiesInSuper)}}); return; } @@ -2413,7 +2382,7 @@ void Unifier::tryUnifySealedTables(TypeId subTy, TypeId superTy, bool isIntersec if (!extraPropertiesInSub.empty()) { - errors.push_back(TypeError{location, MissingProperties{superTy, subTy, std::move(extraPropertiesInSub), MissingProperties::Extra}}); + reportError(TypeError{location, MissingProperties{superTy, subTy, std::move(extraPropertiesInSub), MissingProperties::Extra}}); return; } } @@ -2437,9 +2406,9 @@ void Unifier::tryUnifyWithMetatable(TypeId subTy, TypeId superTy, bool reversed) innerState.tryUnify_(subMetatable->metatable, superMetatable->metatable); if (auto e = hasUnificationTooComplex(innerState.errors)) - errors.push_back(*e); + reportError(*e); else if (!innerState.errors.empty()) - errors.push_back( + reportError( TypeError{location, TypeMismatch{reversed ? subTy : superTy, reversed ? superTy : subTy, "", innerState.errors.front()}}); if (FFlag::LuauUseCommittingTxnLog) @@ -2470,7 +2439,7 @@ void Unifier::tryUnifyWithMetatable(TypeId subTy, TypeId superTy, bool reversed) case TableState::Sealed: case TableState::Unsealed: case TableState::Generic: - errors.push_back(mismatchError); + reportError(mismatchError); } } else if (FFlag::LuauUseCommittingTxnLog ? (log.getMutable(subTy) || log.getMutable(subTy)) @@ -2479,7 +2448,7 @@ void Unifier::tryUnifyWithMetatable(TypeId subTy, TypeId superTy, bool reversed) } else { - errors.push_back(mismatchError); + reportError(mismatchError); } } @@ -2491,9 +2460,9 @@ void Unifier::tryUnifyWithClass(TypeId subTy, TypeId superTy, bool reversed) auto fail = [&]() { if (!reversed) - errors.push_back(TypeError{location, TypeMismatch{superTy, subTy}}); + reportError(TypeError{location, TypeMismatch{superTy, subTy}}); else - errors.push_back(TypeError{location, TypeMismatch{subTy, superTy}}); + reportError(TypeError{location, TypeMismatch{subTy, superTy}}); }; const ClassTypeVar* superClass = get(superTy); @@ -2538,7 +2507,7 @@ void Unifier::tryUnifyWithClass(TypeId subTy, TypeId superTy, bool reversed) if (!classProp) { ok = false; - errors.push_back(TypeError{location, UnknownProperty{superTy, propName}}); + reportError(TypeError{location, UnknownProperty{superTy, propName}}); } else { @@ -2577,7 +2546,7 @@ void Unifier::tryUnifyWithClass(TypeId subTy, TypeId superTy, bool reversed) { ok = false; std::string msg = "Class " + superClass->name + " does not have an indexer"; - errors.push_back(TypeError{location, GenericError{msg}}); + reportError(TypeError{location, GenericError{msg}}); } if (!ok) @@ -2695,7 +2664,7 @@ void Unifier::tryUnifyVariadics(TypePackId subTp, TypePackId superTp, bool rever } else if (get(tail)) { - errors.push_back(TypeError{location, GenericError{"Cannot unify variadic and generic packs"}}); + reportError(TypeError{location, GenericError{"Cannot unify variadic and generic packs"}}); } else if (get(tail)) { @@ -2709,7 +2678,7 @@ void Unifier::tryUnifyVariadics(TypePackId subTp, TypePackId superTp, bool rever } else { - errors.push_back(TypeError{location, GenericError{"Failed to unify variadic packs"}}); + reportError(TypeError{location, GenericError{"Failed to unify variadic packs"}}); } } @@ -2886,7 +2855,7 @@ void Unifier::occursCheck(DenseHashSet& seen, TypeId needle, TypeId hays if (needle == haystack) { - errors.push_back(TypeError{location, OccursCheckFailed{}}); + reportError(TypeError{location, OccursCheckFailed{}}); log.replace(needle, *getSingletonTypes().errorRecoveryType()); return; @@ -2894,17 +2863,6 @@ void Unifier::occursCheck(DenseHashSet& seen, TypeId needle, TypeId hays if (log.getMutable(haystack)) return; - else if (auto a = log.getMutable(haystack)) - { - if (!FFlag::LuauOccursCheckOkWithRecursiveFunctions) - { - for (TypePackIterator it(a->argTypes, &log); it != end(a->argTypes); ++it) - check(*it); - - for (TypePackIterator it(a->retType, &log); it != end(a->retType); ++it) - check(*it); - } - } else if (auto a = log.getMutable(haystack)) { for (TypeId ty : a->options) @@ -2934,7 +2892,7 @@ void Unifier::occursCheck(DenseHashSet& seen, TypeId needle, TypeId hays if (needle == haystack) { - errors.push_back(TypeError{location, OccursCheckFailed{}}); + reportError(TypeError{location, OccursCheckFailed{}}); DEPRECATED_log(needle); *asMutable(needle) = *getSingletonTypes().errorRecoveryType(); return; @@ -2942,17 +2900,6 @@ void Unifier::occursCheck(DenseHashSet& seen, TypeId needle, TypeId hays if (get(haystack)) return; - else if (auto a = get(haystack)) - { - if (!FFlag::LuauOccursCheckOkWithRecursiveFunctions) - { - for (TypeId ty : a->argTypes) - check(ty); - - for (TypeId ty : a->retType) - check(ty); - } - } else if (auto a = get(haystack)) { for (TypeId ty : a->options) @@ -2988,7 +2935,7 @@ void Unifier::occursCheck(DenseHashSet& seen, TypePackId needle, Typ if (log.getMutable(needle)) return; - if (!get(needle)) + if (!log.getMutable(needle)) ice("Expected needle pack to be free"); RecursionLimiter _ra(&sharedState.counters.recursionCount, FInt::LuauTypeInferRecursionLimit); @@ -2997,32 +2944,18 @@ void Unifier::occursCheck(DenseHashSet& seen, TypePackId needle, Typ { if (needle == haystack) { - errors.push_back(TypeError{location, OccursCheckFailed{}}); + reportError(TypeError{location, OccursCheckFailed{}}); log.replace(needle, *getSingletonTypes().errorRecoveryTypePack()); return; } - if (auto a = get(haystack)) + if (auto a = get(haystack); a && a->tail) { - for (const auto& ty : a->head) - { - if (!FFlag::LuauOccursCheckOkWithRecursiveFunctions) - { - if (auto f = log.getMutable(log.follow(ty))) - { - occursCheck(seen, needle, f->argTypes); - occursCheck(seen, needle, f->retType); - } - } - } - - if (a->tail) - { - haystack = follow(*a->tail); - continue; - } + haystack = log.follow(*a->tail); + continue; } + break; } } @@ -3048,31 +2981,17 @@ void Unifier::occursCheck(DenseHashSet& seen, TypePackId needle, Typ { if (needle == haystack) { - errors.push_back(TypeError{location, OccursCheckFailed{}}); + reportError(TypeError{location, OccursCheckFailed{}}); DEPRECATED_log(needle); *asMutable(needle) = *getSingletonTypes().errorRecoveryTypePack(); } - if (auto a = get(haystack)) + if (auto a = get(haystack); a && a->tail) { - if (!FFlag::LuauOccursCheckOkWithRecursiveFunctions) - { - for (const auto& ty : a->head) - { - if (auto f = get(follow(ty))) - { - occursCheck(seen, needle, f->argTypes); - occursCheck(seen, needle, f->retType); - } - } - } - - if (a->tail) - { - haystack = follow(*a->tail); - continue; - } + haystack = follow(*a->tail); + continue; } + break; } } @@ -3094,17 +3013,17 @@ bool Unifier::isNonstrictMode() const void Unifier::checkChildUnifierTypeMismatch(const ErrorVec& innerErrors, TypeId wantedType, TypeId givenType) { if (auto e = hasUnificationTooComplex(innerErrors)) - errors.push_back(*e); + reportError(*e); else if (!innerErrors.empty()) - errors.push_back(TypeError{location, TypeMismatch{wantedType, givenType}}); + reportError(TypeError{location, TypeMismatch{wantedType, givenType}}); } void Unifier::checkChildUnifierTypeMismatch(const ErrorVec& innerErrors, const std::string& prop, TypeId wantedType, TypeId givenType) { if (auto e = hasUnificationTooComplex(innerErrors)) - errors.push_back(*e); + reportError(*e); else if (!innerErrors.empty()) - errors.push_back( + reportError( TypeError{location, TypeMismatch{wantedType, givenType, format("Property '%s' is not compatible.", prop.c_str()), innerErrors.front()}}); } diff --git a/CLI/Analyze.cpp b/CLI/Analyze.cpp index 39fc78ef..8b03ea1a 100644 --- a/CLI/Analyze.cpp +++ b/CLI/Analyze.cpp @@ -43,7 +43,7 @@ static void report(ReportFormat format, const char* name, const Luau::Location& } } -static void reportError(Luau::Frontend& frontend, ReportFormat format, const Luau::TypeError& error) +static void reportError(const Luau::Frontend& frontend, ReportFormat format, const Luau::TypeError& error) { std::string humanReadableName = frontend.fileResolver->getHumanReadableModuleName(error.moduleName); diff --git a/CLI/FileUtils.cpp b/CLI/FileUtils.cpp index 87decb51..c6807022 100644 --- a/CLI/FileUtils.cpp +++ b/CLI/FileUtils.cpp @@ -15,8 +15,6 @@ #include -#define READ_BUFFER_SIZE 4096 - #ifdef _WIN32 static std::wstring fromUtf8(const std::string& path) { @@ -79,9 +77,9 @@ std::optional readFile(const std::string& name) std::optional readStdin() { std::string result; - char buffer[READ_BUFFER_SIZE] = { }; + char buffer[4096] = { }; - while (fgets(buffer, READ_BUFFER_SIZE, stdin) != nullptr) + while (fgets(buffer, sizeof(buffer), stdin) != nullptr) result.append(buffer); // If eof was not reached for stdin, then a read error occurred diff --git a/CLI/Repl.cpp b/CLI/Repl.cpp index ecf29fae..ab0f0ed0 100644 --- a/CLI/Repl.cpp +++ b/CLI/Repl.cpp @@ -158,7 +158,7 @@ static int lua_collectgarbage(lua_State* L) luaL_error(L, "collectgarbage must be called with 'count' or 'collect'"); } -static void setupState(lua_State* L) +void setupState(lua_State* L) { luaL_openlibs(L); @@ -176,7 +176,7 @@ static void setupState(lua_State* L) luaL_sandbox(L); } -static std::string runCode(lua_State* L, const std::string& source) +std::string runCode(lua_State* L, const std::string& source) { std::string bytecode = Luau::compile(source, copts()); @@ -206,7 +206,13 @@ static std::string runCode(lua_State* L, const std::string& source) if (n) { luaL_checkstack(T, LUA_MINSTACK, "too many results to print"); - lua_getglobal(T, "print"); + lua_getglobal(T, "_PRETTYPRINT"); + // If _PRETTYPRINT is nil, then use the standard print function instead + if (lua_isnil(T, -1)) + { + lua_pop(T, 1); + lua_getglobal(T, "print"); + } lua_insert(T, 1); lua_pcall(T, n, 0, 0); } @@ -545,7 +551,7 @@ static int assertionHandler(const char* expr, const char* file, int line, const return 1; } -int main(int argc, char** argv) +int replMain(int argc, char** argv) { Luau::assertHandler() = assertionHandler; @@ -696,5 +702,6 @@ int main(int argc, char** argv) case CliMode::Unknown: default: LUAU_ASSERT(!"Unhandled cli mode."); + return 1; } } diff --git a/CLI/Repl.h b/CLI/Repl.h new file mode 100644 index 00000000..11a077ae --- /dev/null +++ b/CLI/Repl.h @@ -0,0 +1,12 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +#pragma once + +#include "lua.h" + +#include + +// Note: These are internal functions which are being exposed in a header +// so they can be included by unit tests. +int replMain(int argc, char** argv); +void setupState(lua_State* L); +std::string runCode(lua_State* L, const std::string& source); diff --git a/CLI/ReplEntry.cpp b/CLI/ReplEntry.cpp new file mode 100644 index 00000000..b3131712 --- /dev/null +++ b/CLI/ReplEntry.cpp @@ -0,0 +1,10 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details + +#include "Repl.h" + + + +int main(int argc, char** argv) +{ + return replMain(argc, argv); +} \ No newline at end of file diff --git a/CMakeLists.txt b/CMakeLists.txt index 77cf47e8..b9f7a9e1 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -29,6 +29,7 @@ endif() if(LUAU_BUILD_TESTS) add_executable(Luau.UnitTest) add_executable(Luau.Conformance) + add_executable(Luau.CLI.Test) endif() if(LUAU_BUILD_WEB) @@ -109,6 +110,17 @@ if(LUAU_BUILD_TESTS) target_compile_options(Luau.Conformance PRIVATE ${LUAU_OPTIONS}) target_include_directories(Luau.Conformance PRIVATE extern) target_link_libraries(Luau.Conformance PRIVATE Luau.Analysis Luau.Compiler Luau.VM) + + target_compile_options(Luau.CLI.Test PRIVATE ${LUAU_OPTIONS}) + target_include_directories(Luau.CLI.Test PRIVATE extern CLI) + target_link_libraries(Luau.CLI.Test PRIVATE Luau.Compiler Luau.VM) + if(UNIX) + find_library(LIBPTHREAD pthread) + if (LIBPTHREAD) + target_link_libraries(Luau.CLI.Test PRIVATE pthread) + endif() + endif() + endif() if(LUAU_BUILD_WEB) diff --git a/Compiler/include/Luau/Bytecode.h b/Compiler/include/Luau/Bytecode.h index d9694d7d..679712f6 100644 --- a/Compiler/include/Luau/Bytecode.h +++ b/Compiler/include/Luau/Bytecode.h @@ -472,6 +472,9 @@ enum LuauBuiltinFunction // bit32.count LBF_BIT32_COUNTLZ, LBF_BIT32_COUNTRZ, + + // select(_, ...) + LBF_SELECT_VARARG, }; // Capture type, used in LOP_CAPTURE diff --git a/Compiler/src/Builtins.cpp b/Compiler/src/Builtins.cpp index e344eb91..a907271c 100644 --- a/Compiler/src/Builtins.cpp +++ b/Compiler/src/Builtins.cpp @@ -4,6 +4,8 @@ #include "Luau/Bytecode.h" #include "Luau/Compiler.h" +LUAU_FASTFLAGVARIABLE(LuauCompileSelectBuiltin, false) + namespace Luau { namespace Compile @@ -62,6 +64,9 @@ int getBuiltinFunctionId(const Builtin& builtin, const CompileOptions& options) if (builtin.isGlobal("unpack")) return LBF_TABLE_UNPACK; + if (FFlag::LuauCompileSelectBuiltin && builtin.isGlobal("select")) + return LBF_SELECT_VARARG; + if (builtin.object == "math") { if (builtin.method == "abs") diff --git a/Compiler/src/Compiler.cpp b/Compiler/src/Compiler.cpp index 9758c4a9..7da85244 100644 --- a/Compiler/src/Compiler.cpp +++ b/Compiler/src/Compiler.cpp @@ -15,6 +15,9 @@ #include #include +LUAU_FASTFLAGVARIABLE(LuauCompileTableIndexOpt, false) +LUAU_FASTFLAG(LuauCompileSelectBuiltin) + namespace Luau { @@ -261,6 +264,122 @@ struct Compiler bytecode.emitABC(LOP_GETVARARGS, target, multRet ? 0 : uint8_t(targetCount + 1), 0); } + void compileExprSelectVararg(AstExprCall* expr, uint8_t target, uint8_t targetCount, bool targetTop, bool multRet, uint8_t regs) + { + LUAU_ASSERT(FFlag::LuauCompileSelectBuiltin); + LUAU_ASSERT(targetCount == 1); + LUAU_ASSERT(!expr->self); + LUAU_ASSERT(expr->args.size == 2 && expr->args.data[1]->is()); + + AstExpr* arg = expr->args.data[0]; + + uint8_t argreg; + + if (isExprLocalReg(arg)) + argreg = getLocal(arg->as()->local); + else + { + argreg = uint8_t(regs + 1); + compileExprTempTop(arg, argreg); + } + + size_t fastcallLabel = bytecode.emitLabel(); + + bytecode.emitABC(LOP_FASTCALL1, LBF_SELECT_VARARG, argreg, 0); + + // note, these instructions are normally not executed and are used as a fallback for FASTCALL + // we can't use TempTop variant here because we need to make sure the arguments we already computed aren't overwritten + compileExprTemp(expr->func, regs); + + bytecode.emitABC(LOP_GETVARARGS, uint8_t(regs + 2), 0, 0); + + size_t callLabel = bytecode.emitLabel(); + if (!bytecode.patchSkipC(fastcallLabel, callLabel)) + CompileError::raise(expr->func->location, "Exceeded jump distance limit; simplify the code to compile"); + + // note, this is always multCall (last argument is variadic) + bytecode.emitABC(LOP_CALL, regs, 0, multRet ? 0 : uint8_t(targetCount + 1)); + + // if we didn't output results directly to target, we need to move them + if (!targetTop) + { + for (size_t i = 0; i < targetCount; ++i) + bytecode.emitABC(LOP_MOVE, uint8_t(target + i), uint8_t(regs + i), 0); + } + } + + void compileExprFastcallN(AstExprCall* expr, uint8_t target, uint8_t targetCount, bool targetTop, bool multRet, uint8_t regs, int bfid) + { + LUAU_ASSERT(!expr->self); + LUAU_ASSERT(expr->args.size <= 2); + + LuauOpcode opc = expr->args.size == 1 ? LOP_FASTCALL1 : LOP_FASTCALL2; + + uint32_t args[2] = {}; + + for (size_t i = 0; i < expr->args.size; ++i) + { + if (i > 0) + { + if (int32_t cid = getConstantIndex(expr->args.data[i]); cid >= 0) + { + opc = LOP_FASTCALL2K; + args[i] = cid; + break; + } + } + + if (isExprLocalReg(expr->args.data[i])) + args[i] = getLocal(expr->args.data[i]->as()->local); + else + { + args[i] = uint8_t(regs + 1 + i); + compileExprTempTop(expr->args.data[i], uint8_t(args[i])); + } + } + + size_t fastcallLabel = bytecode.emitLabel(); + + bytecode.emitABC(opc, uint8_t(bfid), uint8_t(args[0]), 0); + if (opc != LOP_FASTCALL1) + bytecode.emitAux(args[1]); + + // Set up a traditional Lua stack for the subsequent LOP_CALL. + // Note, as with other instructions that immediately follow FASTCALL, these are normally not executed and are used as a fallback for + // these FASTCALL variants. + for (size_t i = 0; i < expr->args.size; ++i) + { + if (i > 0 && opc == LOP_FASTCALL2K) + { + emitLoadK(uint8_t(regs + 1 + i), args[i]); + break; + } + + if (args[i] != regs + 1 + i) + bytecode.emitABC(LOP_MOVE, uint8_t(regs + 1 + i), uint8_t(args[i]), 0); + } + + // note, these instructions are normally not executed and are used as a fallback for FASTCALL + // we can't use TempTop variant here because we need to make sure the arguments we already computed aren't overwritten + compileExprTemp(expr->func, regs); + + size_t callLabel = bytecode.emitLabel(); + + // FASTCALL will skip over the instructions needed to compute function and jump over CALL which must immediately follow the instruction + // sequence after FASTCALL + if (!bytecode.patchSkipC(fastcallLabel, callLabel)) + CompileError::raise(expr->func->location, "Exceeded jump distance limit; simplify the code to compile"); + + bytecode.emitABC(LOP_CALL, regs, uint8_t(expr->args.size + 1), multRet ? 0 : uint8_t(targetCount + 1)); + + // if we didn't output results directly to target, we need to move them + if (!targetTop) + { + for (size_t i = 0; i < targetCount; ++i) + bytecode.emitABC(LOP_MOVE, uint8_t(target + i), uint8_t(regs + i), 0); + } + } + void compileExprCall(AstExprCall* expr, uint8_t target, uint8_t targetCount, bool targetTop = false, bool multRet = false) { LUAU_ASSERT(!targetTop || unsigned(target + targetCount) == regTop); @@ -284,6 +403,25 @@ struct Compiler bfid = getBuiltinFunctionId(builtin, options); } + if (bfid == LBF_SELECT_VARARG) + { + LUAU_ASSERT(FFlag::LuauCompileSelectBuiltin); + // Optimization: compile select(_, ...) as FASTCALL1; the builtin will read variadic arguments directly + // note: for now we restrict this to single-return expressions since our runtime code doesn't deal with general cases + if (multRet == false && targetCount == 1 && expr->args.size == 2 && expr->args.data[1]->is()) + return compileExprSelectVararg(expr, target, targetCount, targetTop, multRet, regs); + else + bfid = -1; + } + + // Optimization: for 1/2 argument fast calls use specialized opcodes + if (!expr->self && bfid >= 0 && expr->args.size >= 1 && expr->args.size <= 2) + { + AstExpr* last = expr->args.data[expr->args.size - 1]; + if (!last->is() && !last->is()) + return compileExprFastcallN(expr, target, targetCount, targetTop, multRet, regs, bfid); + } + if (expr->self) { AstExprIndexName* fi = expr->func->as(); @@ -309,24 +447,13 @@ struct Compiler compileExprTempTop(expr->func, regs); } - // Note: if the last argument is ExprVararg or ExprCall, we need to route that directly to the called function preserving the # of args bool multCall = false; - bool skipArgs = false; - if (!expr->self && bfid >= 0 && expr->args.size >= 1 && expr->args.size <= 2) - { - AstExpr* last = expr->args.data[expr->args.size - 1]; - skipArgs = !(last->is() || last->is()); - } - - if (!skipArgs) - { - for (size_t i = 0; i < expr->args.size; ++i) - if (i + 1 == expr->args.size) - multCall = compileExprTempMultRet(expr->args.data[i], uint8_t(regs + 1 + expr->self + i)); - else - compileExprTempTop(expr->args.data[i], uint8_t(regs + 1 + expr->self + i)); - } + for (size_t i = 0; i < expr->args.size; ++i) + if (i + 1 == expr->args.size) + multCall = compileExprTempMultRet(expr->args.data[i], uint8_t(regs + 1 + expr->self + i)); + else + compileExprTempTop(expr->args.data[i], uint8_t(regs + 1 + expr->self + i)); setDebugLineEnd(expr->func); @@ -347,59 +474,8 @@ struct Compiler } else if (bfid >= 0) { - size_t fastcallLabel; - - if (skipArgs) - { - LuauOpcode opc = expr->args.size == 1 ? LOP_FASTCALL1 : LOP_FASTCALL2; - - uint32_t args[2] = {}; - for (size_t i = 0; i < expr->args.size; ++i) - { - if (i > 0) - { - if (int32_t cid = getConstantIndex(expr->args.data[i]); cid >= 0) - { - opc = LOP_FASTCALL2K; - args[i] = cid; - break; - } - } - - if (isExprLocalReg(expr->args.data[i])) - args[i] = getLocal(expr->args.data[i]->as()->local); - else - { - args[i] = uint8_t(regs + 1 + i); - compileExprTempTop(expr->args.data[i], uint8_t(args[i])); - } - } - - fastcallLabel = bytecode.emitLabel(); - bytecode.emitABC(opc, uint8_t(bfid), uint8_t(args[0]), 0); - if (opc != LOP_FASTCALL1) - bytecode.emitAux(args[1]); - - // Set up a traditional Lua stack for the subsequent LOP_CALL. - // Note, as with other instructions that immediately follow FASTCALL, these are normally not executed and are used as a fallback for - // these FASTCALL variants. - for (size_t i = 0; i < expr->args.size; ++i) - { - if (i > 0 && opc == LOP_FASTCALL2K) - { - emitLoadK(uint8_t(regs + 1 + i), args[i]); - break; - } - - if (args[i] != regs + 1 + i) - bytecode.emitABC(LOP_MOVE, uint8_t(regs + 1 + i), uint8_t(args[i]), 0); - } - } - else - { - fastcallLabel = bytecode.emitLabel(); - bytecode.emitABC(LOP_FASTCALL, uint8_t(bfid), 0, 0); - } + size_t fastcallLabel = bytecode.emitLabel(); + bytecode.emitABC(LOP_FASTCALL, uint8_t(bfid), 0, 0); // note, these instructions are normally not executed and are used as a fallback for FASTCALL // we can't use TempTop variant here because we need to make sure the arguments we already computed aren't overwritten @@ -1101,9 +1177,20 @@ struct Compiler for (size_t i = 0; i < expr->items.size; ++i) { const AstExprTable::Item& item = expr->items.data[i]; - AstExprConstantNumber* ckey = item.key->as(); + LUAU_ASSERT(item.key); // no list portion => all items have keys - indexSize += (ckey && ckey->value == double(indexSize + 1)); + if (FFlag::LuauCompileTableIndexOpt) + { + const Constant* ckey = constants.find(item.key); + + indexSize += (ckey && ckey->type == Constant::Type_Number && ckey->valueNumber == double(indexSize + 1)); + } + else + { + AstExprConstantNumber* ckey = item.key->as(); + + indexSize += (ckey && ckey->value == double(indexSize + 1)); + } } // we only perform the optimization if we don't have any other []-keys @@ -1200,37 +1287,47 @@ struct Compiler arrayChunkCurrent = 0; } - // items with a key are set one by one via SETTABLE/SETTABLEKS + // items with a key are set one by one via SETTABLE/SETTABLEKS/SETTABLEN if (key) { RegScope rsi(this); - // Optimization: use SETTABLEKS/SETTABLEN for literal keys, this happens often as part of usual table construction syntax - if (AstExprConstantString* ckey = key->as()) + if (FFlag::LuauCompileTableIndexOpt) { - BytecodeBuilder::StringRef cname = sref(ckey->value); - int32_t cid = bytecode.addConstantString(cname); - if (cid < 0) - CompileError::raise(expr->location, "Exceeded constant limit; simplify the code to compile"); - + LValue lv = compileLValueIndex(reg, key, rsi); uint8_t rv = compileExprAuto(value, rsi); - bytecode.emitABC(LOP_SETTABLEKS, rv, reg, uint8_t(BytecodeBuilder::getStringHash(cname))); - bytecode.emitAux(cid); - } - else if (AstExprConstantNumber* ckey = key->as(); - ckey && ckey->value >= 1 && ckey->value <= 256 && double(int(ckey->value)) == ckey->value) - { - uint8_t rv = compileExprAuto(value, rsi); - - bytecode.emitABC(LOP_SETTABLEN, rv, reg, uint8_t(int(ckey->value) - 1)); + compileAssign(lv, rv); } else { - uint8_t rk = compileExprAuto(key, rsi); - uint8_t rv = compileExprAuto(value, rsi); + // Optimization: use SETTABLEKS/SETTABLEN for literal keys, this happens often as part of usual table construction syntax + if (AstExprConstantString* ckey = key->as()) + { + BytecodeBuilder::StringRef cname = sref(ckey->value); + int32_t cid = bytecode.addConstantString(cname); + if (cid < 0) + CompileError::raise(expr->location, "Exceeded constant limit; simplify the code to compile"); - bytecode.emitABC(LOP_SETTABLE, rv, reg, rk); + uint8_t rv = compileExprAuto(value, rsi); + + bytecode.emitABC(LOP_SETTABLEKS, rv, reg, uint8_t(BytecodeBuilder::getStringHash(cname))); + bytecode.emitAux(cid); + } + else if (AstExprConstantNumber* ckey = key->as(); + ckey && ckey->value >= 1 && ckey->value <= 256 && double(int(ckey->value)) == ckey->value) + { + uint8_t rv = compileExprAuto(value, rsi); + + bytecode.emitABC(LOP_SETTABLEN, rv, reg, uint8_t(int(ckey->value) - 1)); + } + else + { + uint8_t rk = compileExprAuto(key, rsi); + uint8_t rv = compileExprAuto(value, rsi); + + bytecode.emitABC(LOP_SETTABLE, rv, reg, rk); + } } } // items without a key are set using SETLIST so that we can initialize large arrays quickly @@ -1339,6 +1436,9 @@ struct Compiler uint8_t rt = compileExprAuto(expr->expr, rs); uint8_t i = uint8_t(int(cv->valueNumber) - 1); + if (FFlag::LuauCompileTableIndexOpt) + setDebugLine(expr->index); + bytecode.emitABC(LOP_GETTABLEN, target, rt, i); } else if (cv && cv->type == Constant::Type_String) @@ -1350,6 +1450,9 @@ struct Compiler if (cid < 0) CompileError::raise(expr->location, "Exceeded constant limit; simplify the code to compile"); + if (FFlag::LuauCompileTableIndexOpt) + setDebugLine(expr->index); + bytecode.emitABC(LOP_GETTABLEKS, target, rt, uint8_t(BytecodeBuilder::getStringHash(iname))); bytecode.emitAux(cid); } @@ -1657,6 +1760,40 @@ struct Compiler Location location; }; + LValue compileLValueIndex(uint8_t reg, AstExpr* index, RegScope& rs) + { + const Constant* cv = constants.find(index); + + if (cv && cv->type == Constant::Type_Number && cv->valueNumber >= 1 && cv->valueNumber <= 256 && + double(int(cv->valueNumber)) == cv->valueNumber) + { + LValue result = {LValue::Kind_IndexNumber}; + result.reg = reg; + result.number = uint8_t(int(cv->valueNumber) - 1); + result.location = index->location; + + return result; + } + else if (cv && cv->type == Constant::Type_String) + { + LValue result = {LValue::Kind_IndexName}; + result.reg = reg; + result.name = sref(cv->getString()); + result.location = index->location; + + return result; + } + else + { + LValue result = {LValue::Kind_IndexExpr}; + result.reg = reg; + result.index = compileExprAuto(index, rs); + result.location = index->location; + + return result; + } + } + LValue compileLValue(AstExpr* node, RegScope& rs) { setDebugLine(node); @@ -1699,36 +1836,9 @@ struct Compiler } else if (AstExprIndexExpr* expr = node->as()) { - const Constant* cv = constants.find(expr->index); + uint8_t reg = compileExprAuto(expr->expr, rs); - if (cv && cv->type == Constant::Type_Number && cv->valueNumber >= 1 && cv->valueNumber <= 256 && - double(int(cv->valueNumber)) == cv->valueNumber) - { - LValue result = {LValue::Kind_IndexNumber}; - result.reg = compileExprAuto(expr->expr, rs); - result.number = uint8_t(int(cv->valueNumber) - 1); - result.location = node->location; - - return result; - } - else if (cv && cv->type == Constant::Type_String) - { - LValue result = {LValue::Kind_IndexName}; - result.reg = compileExprAuto(expr->expr, rs); - result.name = sref(cv->getString()); - result.location = node->location; - - return result; - } - else - { - LValue result = {LValue::Kind_IndexExpr}; - result.reg = compileExprAuto(expr->expr, rs); - result.index = compileExprAuto(expr->index, rs); - result.location = node->location; - - return result; - } + return compileLValueIndex(reg, expr->index, rs); } else { @@ -1740,6 +1850,9 @@ struct Compiler void compileLValueUse(const LValue& lv, uint8_t reg, bool set) { + if (FFlag::LuauCompileTableIndexOpt) + setDebugLine(lv.location); + switch (lv.kind) { case LValue::Kind_Local: diff --git a/Makefile b/Makefile index fe5a2d9c..697e34ed 100644 --- a/Makefile +++ b/Makefile @@ -23,11 +23,11 @@ VM_SOURCES=$(wildcard VM/src/*.cpp) VM_OBJECTS=$(VM_SOURCES:%=$(BUILD)/%.o) VM_TARGET=$(BUILD)/libluauvm.a -TESTS_SOURCES=$(wildcard tests/*.cpp) +TESTS_SOURCES=$(wildcard tests/*.cpp) CLI/FileUtils.cpp CLI/Profiler.cpp CLI/Coverage.cpp CLI/Repl.cpp TESTS_OBJECTS=$(TESTS_SOURCES:%=$(BUILD)/%.o) TESTS_TARGET=$(BUILD)/luau-tests -REPL_CLI_SOURCES=CLI/FileUtils.cpp CLI/Profiler.cpp CLI/Coverage.cpp CLI/Repl.cpp +REPL_CLI_SOURCES=CLI/FileUtils.cpp CLI/Profiler.cpp CLI/Coverage.cpp CLI/Repl.cpp CLI/ReplEntry.cpp REPL_CLI_OBJECTS=$(REPL_CLI_SOURCES:%=$(BUILD)/%.o) REPL_CLI_TARGET=$(BUILD)/luau @@ -90,11 +90,12 @@ $(AST_OBJECTS): CXXFLAGS+=-std=c++17 -IAst/include $(COMPILER_OBJECTS): CXXFLAGS+=-std=c++17 -ICompiler/include -IAst/include $(ANALYSIS_OBJECTS): CXXFLAGS+=-std=c++17 -IAst/include -IAnalysis/include $(VM_OBJECTS): CXXFLAGS+=-std=c++11 -IVM/include -$(TESTS_OBJECTS): CXXFLAGS+=-std=c++17 -IAst/include -ICompiler/include -IAnalysis/include -IVM/include -Iextern +$(TESTS_OBJECTS): CXXFLAGS+=-std=c++17 -IAst/include -ICompiler/include -IAnalysis/include -IVM/include -ICLI -Iextern $(REPL_CLI_OBJECTS): CXXFLAGS+=-std=c++17 -IAst/include -ICompiler/include -IVM/include -Iextern $(ANALYZE_CLI_OBJECTS): CXXFLAGS+=-std=c++17 -IAst/include -IAnalysis/include -Iextern $(FUZZ_OBJECTS): CXXFLAGS+=-std=c++17 -IAst/include -ICompiler/include -IAnalysis/include -IVM/include +$(TESTS_TARGET): LDFLAGS+=-lpthread $(REPL_CLI_TARGET): LDFLAGS+=-lpthread fuzz-proto fuzz-prototest: LDFLAGS+=build/libprotobuf-mutator/src/libfuzzer/libprotobuf-mutator-libfuzzer.a build/libprotobuf-mutator/src/libprotobuf-mutator.a build/libprotobuf-mutator/external.protobuf/lib/libprotobuf.a diff --git a/Sources.cmake b/Sources.cmake index bafe7594..22e7af22 100644 --- a/Sources.cmake +++ b/Sources.cmake @@ -176,7 +176,8 @@ if(TARGET Luau.Repl.CLI) CLI/FileUtils.cpp CLI/Profiler.h CLI/Profiler.cpp - CLI/Repl.cpp) + CLI/Repl.cpp + CLI/ReplEntry.cpp) endif() if(TARGET Luau.Analyze.CLI) @@ -243,6 +244,21 @@ if(TARGET Luau.Conformance) tests/main.cpp) endif() +if(TARGET Luau.CLI.Test) + # Luau.CLI.Test Sources + target_sources(Luau.CLI.Test PRIVATE + CLI/Coverage.h + CLI/Coverage.cpp + CLI/FileUtils.h + CLI/FileUtils.cpp + CLI/Profiler.h + CLI/Profiler.cpp + CLI/Repl.cpp + + tests/Repl.test.cpp + tests/main.cpp) +endif() + if(TARGET Luau.Web) # Luau.Web Sources target_sources(Luau.Web PRIVATE diff --git a/VM/src/lapi.cpp b/VM/src/lapi.cpp index d5416285..5cffba63 100644 --- a/VM/src/lapi.cpp +++ b/VM/src/lapi.cpp @@ -14,6 +14,8 @@ #include +LUAU_FASTFLAGVARIABLE(LuauGcForwardMetatableBarrier, false) + const char* lua_ident = "$Lua: Lua 5.1.4 Copyright (C) 1994-2008 Lua.org, PUC-Rio $\n" "$Authors: R. Ierusalimschy, L. H. de Figueiredo & W. Celes $\n" "$URL: www.lua.org $\n"; @@ -869,7 +871,16 @@ int lua_setmetatable(lua_State* L, int objindex) luaG_runerror(L, "Attempt to modify a readonly table"); hvalue(obj)->metatable = mt; if (mt) - luaC_objbarriert(L, hvalue(obj), mt); + { + if (FFlag::LuauGcForwardMetatableBarrier) + { + luaC_objbarrier(L, hvalue(obj), mt); + } + else + { + luaC_objbarriert(L, hvalue(obj), mt); + } + } break; } case LUA_TUSERDATA: diff --git a/VM/src/lbuiltins.cpp b/VM/src/lbuiltins.cpp index 34e9ebc1..ecc14e87 100644 --- a/VM/src/lbuiltins.cpp +++ b/VM/src/lbuiltins.cpp @@ -1087,6 +1087,34 @@ static int luauF_countrz(lua_State* L, StkId res, TValue* arg0, int nresults, St return -1; } +static int luauF_select(lua_State* L, StkId res, TValue* arg0, int nresults, StkId args, int nparams) +{ + if (nparams == 1 && nresults == 1) + { + int n = cast_int(L->base - L->ci->func) - clvalue(L->ci->func)->l.p->numparams - 1; + + if (ttisnumber(arg0)) + { + int i = int(nvalue(arg0)); + + // i >= 1 && i <= n + if (unsigned(i - 1) <= unsigned(n)) + { + setobj2s(L, res, L->base - n + (i - 1)); + return 1; + } + // note: for now we don't handle negative case (wrap around) and defer to fallback + } + else if (ttisstring(arg0) && *svalue(arg0) == '#') + { + setnvalue(res, double(n)); + return 1; + } + } + + return -1; +} + luau_FastFunction luauF_table[256] = { NULL, luauF_assert, @@ -1156,4 +1184,6 @@ luau_FastFunction luauF_table[256] = { luauF_countlz, luauF_countrz, + + luauF_select, }; diff --git a/VM/src/lcorolib.cpp b/VM/src/lcorolib.cpp index abcde779..19222861 100644 --- a/VM/src/lcorolib.cpp +++ b/VM/src/lcorolib.cpp @@ -5,8 +5,6 @@ #include "lstate.h" #include "lvm.h" -LUAU_FASTFLAGVARIABLE(LuauCoroutineClose, false) - #define CO_RUN 0 /* running */ #define CO_SUS 1 /* suspended */ #define CO_NOR 2 /* 'normal' (it resumed another coroutine) */ @@ -235,9 +233,6 @@ static int coyieldable(lua_State* L) static int coclose(lua_State* L) { - if (!FFlag::LuauCoroutineClose) - luaL_error(L, "coroutine.close is not enabled"); - lua_State* co = lua_tothread(L, 1); luaL_argexpected(L, co, 1, "thread"); diff --git a/VM/src/ldo.cpp b/VM/src/ldo.cpp index 581506a8..a3982bc6 100644 --- a/VM/src/ldo.cpp +++ b/VM/src/ldo.cpp @@ -17,8 +17,6 @@ #include -LUAU_FASTFLAG(LuauCoroutineClose) - /* ** {====================================================== ** Error-recovery functions @@ -300,7 +298,7 @@ static void resume(lua_State* L, void* ud) { // start coroutine LUAU_ASSERT(L->ci == L->base_ci && firstArg >= L->base); - if (FFlag::LuauCoroutineClose && firstArg == L->base) + if (firstArg == L->base) luaG_runerror(L, "cannot resume dead coroutine"); if (luau_precall(L, firstArg - 1, LUA_MULTRET) != PCRLUA) diff --git a/VM/src/lgc.cpp b/VM/src/lgc.cpp index 9a8cb079..82ac0009 100644 --- a/VM/src/lgc.cpp +++ b/VM/src/lgc.cpp @@ -93,10 +93,8 @@ static void finishGcCycleStats(global_State* g) g->gcstats.lastcycle = g->gcstats.currcycle; g->gcstats.currcycle = GCCycleStats(); - g->gcstats.cyclestatsacc.markitems += g->gcstats.lastcycle.markitems; g->gcstats.cyclestatsacc.marktime += g->gcstats.lastcycle.marktime; g->gcstats.cyclestatsacc.atomictime += g->gcstats.lastcycle.atomictime; - g->gcstats.cyclestatsacc.sweepitems += g->gcstats.lastcycle.sweepitems; g->gcstats.cyclestatsacc.sweeptime += g->gcstats.lastcycle.sweeptime; } @@ -492,23 +490,22 @@ static void freeobj(lua_State* L, GCObject* o, lua_Page* page) } } -#define sweepwholelist(L, p, tc) sweeplist(L, p, SIZE_MAX, tc) +#define sweepwholelist(L, p) sweeplist(L, p, SIZE_MAX) -static GCObject** sweeplist(lua_State* L, GCObject** p, size_t count, size_t* traversedcount) +static GCObject** sweeplist(lua_State* L, GCObject** p, size_t count) { LUAU_ASSERT(!FFlag::LuauGcPagedSweep); GCObject* curr; global_State* g = L->global; int deadmask = otherwhite(g); - size_t startcount = count; LUAU_ASSERT(testbit(deadmask, FIXEDBIT)); /* make sure we never sweep fixed objects */ while ((curr = *p) != NULL && count-- > 0) { int alive = (curr->gch.marked ^ WHITEBITS) & deadmask; if (curr->gch.tt == LUA_TTHREAD) { - sweepwholelist(L, (GCObject**)&gco2th(curr)->openupval, traversedcount); /* sweep open upvalues */ + sweepwholelist(L, (GCObject**)&gco2th(curr)->openupval); /* sweep open upvalues */ lua_State* th = gco2th(curr); @@ -534,10 +531,6 @@ static GCObject** sweeplist(lua_State* L, GCObject** p, size_t count, size_t* tr } } - // if we didn't reach the end of the list it means that we've stopped because the count dropped below zero - if (traversedcount) - *traversedcount += startcount - (curr ? count + 1 : count); - return p; } @@ -721,8 +714,6 @@ static bool sweepgco(lua_State* L, lua_Page* page, GCObject* gco) int alive = (gco->gch.marked ^ WHITEBITS) & deadmask; - g->gcstats.currcycle.sweepitems++; - if (gco->gch.tt == LUA_TTHREAD) { lua_State* th = gco2th(gco); @@ -793,8 +784,6 @@ static size_t gcstep(lua_State* L, size_t limit) { while (g->gray && cost < limit) { - g->gcstats.currcycle.markitems++; - cost += propagatemark(g); } @@ -812,8 +801,6 @@ static size_t gcstep(lua_State* L, size_t limit) { while (g->gray && cost < limit) { - g->gcstats.currcycle.markitems++; - cost += propagatemark(g); } @@ -842,10 +829,8 @@ static size_t gcstep(lua_State* L, size_t limit) while (g->sweepstrgc < g->strt.size && cost < limit) { - size_t traversedcount = 0; - sweepwholelist(L, (GCObject**)&g->strt.hash[g->sweepstrgc++], &traversedcount); + sweepwholelist(L, (GCObject**)&g->strt.hash[g->sweepstrgc++]); - g->gcstats.currcycle.sweepitems += traversedcount; cost += GC_SWEEPCOST; } @@ -855,12 +840,10 @@ static size_t gcstep(lua_State* L, size_t limit) // sweep string buffer list and preserve used string count uint32_t nuse = L->global->strt.nuse; - size_t traversedcount = 0; - sweepwholelist(L, (GCObject**)&g->strbufgc, &traversedcount); + sweepwholelist(L, (GCObject**)&g->strbufgc); L->global->strt.nuse = nuse; - g->gcstats.currcycle.sweepitems += traversedcount; g->gcstate = GCSsweep; // end sweep-string phase } break; @@ -893,10 +876,8 @@ static size_t gcstep(lua_State* L, size_t limit) { while (*g->sweepgc && cost < limit) { - size_t traversedcount = 0; - g->sweepgc = sweeplist(L, g->sweepgc, GC_SWEEPMAX, &traversedcount); + g->sweepgc = sweeplist(L, g->sweepgc, GC_SWEEPMAX); - g->gcstats.currcycle.sweepitems += traversedcount; cost += GC_SWEEPMAX * GC_SWEEPCOST; } diff --git a/VM/src/lgc.h b/VM/src/lgc.h index 4455fec5..528d0944 100644 --- a/VM/src/lgc.h +++ b/VM/src/lgc.h @@ -113,6 +113,7 @@ luaC_barrierf(L, obj2gco(p), obj2gco(o)); \ } +// TODO: remove with FFlagLuauGcForwardMetatableBarrier #define luaC_objbarriert(L, t, o) \ { \ if (isblack(obj2gco(t)) && iswhite(obj2gco(o))) \ diff --git a/VM/src/lstate.h b/VM/src/lstate.h index 080f0024..0708b71f 100644 --- a/VM/src/lstate.h +++ b/VM/src/lstate.h @@ -96,9 +96,6 @@ struct GCCycleStats double sweeptime = 0.0; - size_t markitems = 0; - size_t sweepitems = 0; - size_t assistwork = 0; size_t explicitwork = 0; diff --git a/tests/AstQuery.test.cpp b/tests/AstQuery.test.cpp index 41b553b5..292625b0 100644 --- a/tests/AstQuery.test.cpp +++ b/tests/AstQuery.test.cpp @@ -44,10 +44,6 @@ TEST_CASE_FIXTURE(DocumentationSymbolFixture, "prop") TEST_CASE_FIXTURE(DocumentationSymbolFixture, "event_callback_arg") { - ScopedFastFlag sffs[] = { - {"LuauPersistDefinitionFileTypes", true}, - }; - loadDefinition(R"( declare function Connect(fn: (string) -> ()) )"); @@ -63,8 +59,6 @@ TEST_CASE_FIXTURE(DocumentationSymbolFixture, "event_callback_arg") TEST_CASE_FIXTURE(DocumentationSymbolFixture, "overloaded_fn") { - ScopedFastFlag sffs{"LuauStoreMatchingOverloadFnType", true}; - loadDefinition(R"( declare foo: ((string) -> number) & ((number) -> string) )"); diff --git a/tests/Autocomplete.test.cpp b/tests/Autocomplete.test.cpp index 211e1be1..e8e3b315 100644 --- a/tests/Autocomplete.test.cpp +++ b/tests/Autocomplete.test.cpp @@ -2626,7 +2626,6 @@ local a: A<(number, s@1> TEST_CASE_FIXTURE(ACFixture, "autocomplete_first_function_arg_expected_type") { ScopedFastFlag luauAutocompleteAvoidMutation("LuauAutocompleteAvoidMutation", true); - ScopedFastFlag luauAutocompleteFirstArg("LuauAutocompleteFirstArg", true); check(R"( local function foo1() return 1 end @@ -2728,4 +2727,39 @@ end CHECK(ac.entryMap.count("getx")); } +TEST_CASE_FIXTURE(ACFixture, "autocomplete_on_string_singletons") +{ + ScopedFastFlag sffs[] = { + {"LuauParseSingletonTypes", true}, + {"LuauSingletonTypes", true}, + {"LuauRefactorTypeVarQuestions", true}, + }; + + check(R"( + --!strict + local foo: "hello" | "bye" = "hello" + foo:@1 + )"); + + auto ac = autocomplete('1'); + + CHECK(ac.entryMap.count("format")); +} + +TEST_CASE_FIXTURE(ACFixture, "function_in_assignment_has_parentheses_2") +{ + ScopedFastFlag luauAutocompleteAvoidMutation("LuauAutocompleteAvoidMutation", true); + ScopedFastFlag preferToCallFunctionsForIntersects("PreferToCallFunctionsForIntersects", true); + + check(R"( +local bar: ((number) -> number) & (number, number) -> number) +local abc = b@1 + )"); + + auto ac = autocomplete('1'); + + CHECK(ac.entryMap.count("bar")); + CHECK(ac.entryMap["bar"].parens == ParenthesesRecommendation::CursorInside); +} + TEST_SUITE_END(); diff --git a/tests/Compiler.test.cpp b/tests/Compiler.test.cpp index 3b0d677d..4a28bdde 100644 --- a/tests/Compiler.test.cpp +++ b/tests/Compiler.test.cpp @@ -603,6 +603,37 @@ RETURN R0 1 )"); } +TEST_CASE("TableLiteralsIndexConstant") +{ + ScopedFastFlag sff("LuauCompileTableIndexOpt", true); + + // validate that we use SETTTABLEKS for constant variable keys + CHECK_EQ("\n" + compileFunction0(R"( + local a, b = "key", "value" + return {[a] = 42, [b] = 0} +)"), R"( +NEWTABLE R0 2 0 +LOADN R1 42 +SETTABLEKS R1 R0 K0 +LOADN R1 0 +SETTABLEKS R1 R0 K1 +RETURN R0 1 +)"); + + // validate that we use SETTABLEN for constant variable keys *and* that we predict array size + CHECK_EQ("\n" + compileFunction0(R"( + local a, b = 1, 2 + return {[a] = 42, [b] = 0} +)"), R"( +NEWTABLE R0 0 2 +LOADN R1 42 +SETTABLEN R1 R0 1 +LOADN R1 0 +SETTABLEN R1 R0 2 +RETURN R0 1 +)"); +} + TEST_CASE("TableSizePredictionBasic") { CHECK_EQ("\n" + compileFunction0(R"( @@ -2450,6 +2481,37 @@ return )"); } +TEST_CASE("DebugLineInfoAssignment") +{ + ScopedFastFlag sff("LuauCompileTableIndexOpt", true); + + Luau::BytecodeBuilder bcb; + bcb.setDumpFlags(Luau::BytecodeBuilder::Dump_Code | Luau::BytecodeBuilder::Dump_Lines); + Luau::compileOrThrow(bcb, R"( + local a = { b = { c = { d = 3 } } } + +a +["b"] +["c"] +["d"] = 4 +)"); + + CHECK_EQ("\n" + bcb.dumpFunction(0), R"( +2: DUPTABLE R0 1 +2: DUPTABLE R1 3 +2: DUPTABLE R2 5 +2: LOADN R3 3 +2: SETTABLEKS R3 R2 K4 +2: SETTABLEKS R2 R1 K2 +2: SETTABLEKS R1 R0 K0 +5: GETTABLEKS R2 R0 K0 +6: GETTABLEKS R1 R2 K2 +7: LOADN R2 4 +7: SETTABLEKS R2 R1 K4 +8: RETURN R0 0 +)"); +} + TEST_CASE("DebugSource") { const char* source = R"( @@ -2763,6 +2825,75 @@ RETURN R1 -1 )"); } +TEST_CASE("FastcallSelect") +{ + ScopedFastFlag sff("LuauCompileSelectBuiltin", true); + + // select(_, ...) compiles to a builtin call + CHECK_EQ("\n" + compileFunction0("return (select('#', ...))"), R"( +LOADK R1 K0 +FASTCALL1 57 R1 +3 +GETIMPORT R0 2 +GETVARARGS R2 -1 +CALL R0 -1 1 +RETURN R0 1 +)"); + + // more complex example: select inside a for loop bound + select from a iterator + CHECK_EQ("\n" + compileFunction0(R"( +local sum = 0 +for i=1, select('#', ...) do + sum += select(i, ...) +end +return sum +)"), R"( +LOADN R0 0 +LOADN R3 1 +LOADK R5 K0 +FASTCALL1 57 R5 +3 +GETIMPORT R4 2 +GETVARARGS R6 -1 +CALL R4 -1 1 +MOVE R1 R4 +LOADN R2 1 +FORNPREP R1 +7 +FASTCALL1 57 R3 +3 +GETIMPORT R4 2 +GETVARARGS R6 -1 +CALL R4 -1 1 +ADD R0 R0 R4 +FORNLOOP R1 -7 +RETURN R0 1 +)"); + + // currently we assume a single value return to avoid dealing with stack resizing + CHECK_EQ("\n" + compileFunction0("return select('#', ...)"), R"( +GETIMPORT R0 1 +LOADK R1 K2 +GETVARARGS R2 -1 +CALL R0 -1 -1 +RETURN R0 -1 +)"); + + // note that select with a non-variadic second argument doesn't get optimized + CHECK_EQ("\n" + compileFunction0("return select('#')"), R"( +GETIMPORT R0 1 +LOADK R1 K2 +CALL R0 1 -1 +RETURN R0 -1 +)"); + + // note that select with a non-variadic second argument doesn't get optimized + CHECK_EQ("\n" + compileFunction0("return select('#', foo())"), R"( +GETIMPORT R0 1 +LOADK R1 K2 +GETIMPORT R2 4 +CALL R2 0 -1 +CALL R0 -1 -1 +RETURN R0 -1 +)"); +} + TEST_CASE("LotsOfParameters") { const char* source = R"( diff --git a/tests/Conformance.test.cpp b/tests/Conformance.test.cpp index 5222af33..914b881f 100644 --- a/tests/Conformance.test.cpp +++ b/tests/Conformance.test.cpp @@ -331,8 +331,6 @@ TEST_CASE("UTF8") TEST_CASE("Coroutine") { - ScopedFastFlag sff("LuauCoroutineClose", true); - runConformance("coroutine.lua"); } diff --git a/tests/Frontend.test.cpp b/tests/Frontend.test.cpp index 405f26e0..ea1a08fe 100644 --- a/tests/Frontend.test.cpp +++ b/tests/Frontend.test.cpp @@ -956,7 +956,6 @@ TEST_CASE("no_use_after_free_with_type_fun_instantiation") { // This flag forces this test to crash if there's a UAF in this code. ScopedFastFlag sff_DebugLuauFreezeArena("DebugLuauFreezeArena", true); - ScopedFastFlag sff_LuauCloneCorrectlyBeforeMutatingTableType("LuauCloneCorrectlyBeforeMutatingTableType", true); FrontendFixture fix; diff --git a/tests/Parser.test.cpp b/tests/Parser.test.cpp index ac81005c..90831ee9 100644 --- a/tests/Parser.test.cpp +++ b/tests/Parser.test.cpp @@ -2000,6 +2000,73 @@ TEST_CASE_FIXTURE(Fixture, "parse_type_alias_default_type_errors") matchParseError("type Y number> = {}", "Expected type pack after '=', got type", Location{{0, 14}, {0, 32}}); } +TEST_CASE_FIXTURE(Fixture, "parse_if_else_expression") +{ + { + AstStat* stat = parse("return if true then 1 else 2"); + + REQUIRE(stat != nullptr); + AstStatReturn* str = stat->as()->body.data[0]->as(); + REQUIRE(str != nullptr); + CHECK(str->list.size == 1); + auto* ifElseExpr = str->list.data[0]->as(); + REQUIRE(ifElseExpr != nullptr); + } + + { + AstStat* stat = parse("return if true then 1 elseif true then 2 else 3"); + + REQUIRE(stat != nullptr); + AstStatReturn* str = stat->as()->body.data[0]->as(); + REQUIRE(str != nullptr); + CHECK(str->list.size == 1); + auto* ifElseExpr1 = str->list.data[0]->as(); + REQUIRE(ifElseExpr1 != nullptr); + auto* ifElseExpr2 = ifElseExpr1->falseExpr->as(); + REQUIRE(ifElseExpr2 != nullptr); + } + + // Use "else if" as opposed to elseif + { + AstStat* stat = parse("return if true then 1 else if true then 2 else 3"); + + REQUIRE(stat != nullptr); + AstStatReturn* str = stat->as()->body.data[0]->as(); + REQUIRE(str != nullptr); + CHECK(str->list.size == 1); + auto* ifElseExpr1 = str->list.data[0]->as(); + REQUIRE(ifElseExpr1 != nullptr); + auto* ifElseExpr2 = ifElseExpr1->falseExpr->as(); + REQUIRE(ifElseExpr2 != nullptr); + } + + // Use an if-else expression as the conditional expression of an if-else expression + { + AstStat* stat = parse("return if if true then false else true then 1 else 2"); + + REQUIRE(stat != nullptr); + AstStatReturn* str = stat->as()->body.data[0]->as(); + REQUIRE(str != nullptr); + CHECK(str->list.size == 1); + auto* ifElseExpr = str->list.data[0]->as(); + REQUIRE(ifElseExpr != nullptr); + auto* nestedIfElseExpr = ifElseExpr->condition->as(); + REQUIRE(nestedIfElseExpr != nullptr); + } +} + +TEST_CASE_FIXTURE(Fixture, "parse_type_pack_type_parameters") +{ + AstStat* stat = parse(R"( +type Packed = () -> T... + +type A = Packed +type B = Packed<...number> +type C = Packed<(number, X...)> + )"); + REQUIRE(stat != nullptr); +} + TEST_SUITE_END(); TEST_SUITE_BEGIN("ParseErrorRecovery"); @@ -2504,71 +2571,4 @@ type Y = (T...) -> U... CHECK_EQ(1, result.errors.size()); } -TEST_CASE_FIXTURE(Fixture, "parse_if_else_expression") -{ - { - AstStat* stat = parse("return if true then 1 else 2"); - - REQUIRE(stat != nullptr); - AstStatReturn* str = stat->as()->body.data[0]->as(); - REQUIRE(str != nullptr); - CHECK(str->list.size == 1); - auto* ifElseExpr = str->list.data[0]->as(); - REQUIRE(ifElseExpr != nullptr); - } - - { - AstStat* stat = parse("return if true then 1 elseif true then 2 else 3"); - - REQUIRE(stat != nullptr); - AstStatReturn* str = stat->as()->body.data[0]->as(); - REQUIRE(str != nullptr); - CHECK(str->list.size == 1); - auto* ifElseExpr1 = str->list.data[0]->as(); - REQUIRE(ifElseExpr1 != nullptr); - auto* ifElseExpr2 = ifElseExpr1->falseExpr->as(); - REQUIRE(ifElseExpr2 != nullptr); - } - - // Use "else if" as opposed to elseif - { - AstStat* stat = parse("return if true then 1 else if true then 2 else 3"); - - REQUIRE(stat != nullptr); - AstStatReturn* str = stat->as()->body.data[0]->as(); - REQUIRE(str != nullptr); - CHECK(str->list.size == 1); - auto* ifElseExpr1 = str->list.data[0]->as(); - REQUIRE(ifElseExpr1 != nullptr); - auto* ifElseExpr2 = ifElseExpr1->falseExpr->as(); - REQUIRE(ifElseExpr2 != nullptr); - } - - // Use an if-else expression as the conditional expression of an if-else expression - { - AstStat* stat = parse("return if if true then false else true then 1 else 2"); - - REQUIRE(stat != nullptr); - AstStatReturn* str = stat->as()->body.data[0]->as(); - REQUIRE(str != nullptr); - CHECK(str->list.size == 1); - auto* ifElseExpr = str->list.data[0]->as(); - REQUIRE(ifElseExpr != nullptr); - auto* nestedIfElseExpr = ifElseExpr->condition->as(); - REQUIRE(nestedIfElseExpr != nullptr); - } -} - -TEST_CASE_FIXTURE(Fixture, "parse_type_pack_type_parameters") -{ - AstStat* stat = parse(R"( -type Packed = () -> T... - -type A = Packed -type B = Packed<...number> -type C = Packed<(number, X...)> - )"); - REQUIRE(stat != nullptr); -} - TEST_SUITE_END(); diff --git a/tests/Repl.test.cpp b/tests/Repl.test.cpp new file mode 100644 index 00000000..f660bcd3 --- /dev/null +++ b/tests/Repl.test.cpp @@ -0,0 +1,117 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +#include "lua.h" +#include "lualib.h" + +#include "Repl.h" + +#include "doctest.h" + +#include +#include +#include +#include + + +class ReplFixture +{ +public: + ReplFixture() + : luaState(luaL_newstate(), lua_close) + { + L = luaState.get(); + setupState(L); + luaL_sandboxthread(L); + + std::string result = runCode(L, prettyPrintSource); + } + + // Returns all of the output captured from the pretty printer + std::string getCapturedOutput() + { + lua_getglobal(L, "capturedoutput"); + const char* str = lua_tolstring(L, -1, nullptr); + std::string result(str); + lua_pop(L, 1); + return result; + } + lua_State* L; + +private: + std::unique_ptr luaState; + + // This is a simplicitic and incomplete pretty printer. + // It is included here to test that the pretty printer hook is being called. + // More elaborate tests to ensure correct output can be added if we introduce + // a more feature rich pretty printer. + std::string prettyPrintSource = R"( +-- Accumulate pretty printer output in `capturedoutput` +capturedoutput = "" + +function arraytostring(arr) + local strings = {} + table.foreachi(arr, function(k,v) table.insert(strings, pptostring(v)) end ) + return "{" .. table.concat(strings, ", ") .. "}" +end + +function pptostring(x) + if type(x) == "table" then + -- Just assume array-like tables for now. + return arraytostring(x) + elseif type(x) == "string" then + return '"' .. x .. '"' + else + return tostring(x) + end +end + +-- Note: Instead of calling print, the pretty printer just stores the output +-- in `capturedoutput` so we can check for the correct results. +function _PRETTYPRINT(...) + local args = table.pack(...) + local strings = {} + for i=1, args.n do + local item = args[i] + local str = pptostring(item, customoptions) + if i == 1 then + capturedoutput = capturedoutput .. str + else + capturedoutput = capturedoutput .. "\t" .. str + end + end +end +)"; +}; + +TEST_SUITE_BEGIN("ReplPrettyPrint"); + +TEST_CASE_FIXTURE(ReplFixture, "AdditionStatement") +{ + runCode(L, "return 30 + 12"); + CHECK(getCapturedOutput() == "42"); +} + +TEST_CASE_FIXTURE(ReplFixture, "TableLiteral") +{ + runCode(L, "return {1, 2, 3, 4}"); + CHECK(getCapturedOutput() == "{1, 2, 3, 4}"); +} + +TEST_CASE_FIXTURE(ReplFixture, "StringLiteral") +{ + runCode(L, "return 'str'"); + CHECK(getCapturedOutput() == "\"str\""); +} + +TEST_CASE_FIXTURE(ReplFixture, "TableWithStringLiterals") +{ + runCode(L, "return {1, 'two', 3, 'four'}"); + CHECK(getCapturedOutput() == "{1, \"two\", 3, \"four\"}"); +} + +TEST_CASE_FIXTURE(ReplFixture, "MultipleArguments") +{ + runCode(L, "return 3, 'three'"); + CHECK(getCapturedOutput() == "3\t\"three\""); +} + +TEST_SUITE_END(); diff --git a/tests/ToString.test.cpp b/tests/ToString.test.cpp index 445ee532..bbb26291 100644 --- a/tests/ToString.test.cpp +++ b/tests/ToString.test.cpp @@ -435,8 +435,6 @@ TEST_CASE_FIXTURE(Fixture, "toString_the_boundTo_table_type_contained_within_a_T TEST_CASE_FIXTURE(Fixture, "no_parentheses_around_cyclic_function_type_in_union") { - ScopedFastFlag sff{"LuauOccursCheckOkWithRecursiveFunctions", true}; - CheckResult result = check(R"( type F = ((() -> number)?) -> F? local function f(p) return f end @@ -450,8 +448,6 @@ TEST_CASE_FIXTURE(Fixture, "no_parentheses_around_cyclic_function_type_in_union" TEST_CASE_FIXTURE(Fixture, "no_parentheses_around_cyclic_function_type_in_intersection") { - ScopedFastFlag sff{"LuauOccursCheckOkWithRecursiveFunctions", true}; - CheckResult result = check(R"( function f() return f end local a: ((number) -> ()) & typeof(f) diff --git a/tests/TypeInfer.aliases.test.cpp b/tests/TypeInfer.aliases.test.cpp index 822bd727..76ab23b3 100644 --- a/tests/TypeInfer.aliases.test.cpp +++ b/tests/TypeInfer.aliases.test.cpp @@ -11,8 +11,6 @@ TEST_SUITE_BEGIN("TypeAliases"); TEST_CASE_FIXTURE(Fixture, "cyclic_function_type_in_type_alias") { - ScopedFastFlag sff{"LuauOccursCheckOkWithRecursiveFunctions", true}; - CheckResult result = check(R"( type F = () -> F? local function f() @@ -194,8 +192,6 @@ TEST_CASE_FIXTURE(Fixture, "corecursive_types_generic") TEST_CASE_FIXTURE(Fixture, "corecursive_function_types") { - ScopedFastFlag sff{"LuauOccursCheckOkWithRecursiveFunctions", true}; - CheckResult result = check(R"( type A = () -> (number, B) type B = () -> (string, A) diff --git a/tests/TypeInfer.generics.test.cpp b/tests/TypeInfer.generics.test.cpp index 114679e3..a7f27551 100644 --- a/tests/TypeInfer.generics.test.cpp +++ b/tests/TypeInfer.generics.test.cpp @@ -9,8 +9,6 @@ using namespace Luau; -LUAU_FASTFLAG(LuauExtendedFunctionMismatchError) - TEST_SUITE_BEGIN("GenericsTests"); TEST_CASE_FIXTURE(Fixture, "check_generic_function") @@ -656,11 +654,7 @@ local d: D = c LUAU_REQUIRE_ERROR_COUNT(1, result); - if (FFlag::LuauExtendedFunctionMismatchError) - CHECK_EQ( - toString(result.errors[0]), R"(Type '() -> ()' could not be converted into '() -> ()'; different number of generic type parameters)"); - else - CHECK_EQ(toString(result.errors[0]), R"(Type '() -> ()' could not be converted into '() -> ()')"); + CHECK_EQ(toString(result.errors[0]), R"(Type '() -> ()' could not be converted into '() -> ()'; different number of generic type parameters)"); } TEST_CASE_FIXTURE(Fixture, "error_detailed_function_mismatch_generic_pack") @@ -675,11 +669,8 @@ local d: D = c LUAU_REQUIRE_ERROR_COUNT(1, result); - if (FFlag::LuauExtendedFunctionMismatchError) - CHECK_EQ(toString(result.errors[0]), - R"(Type '() -> ()' could not be converted into '() -> ()'; different number of generic type pack parameters)"); - else - CHECK_EQ(toString(result.errors[0]), R"(Type '() -> ()' could not be converted into '() -> ()')"); + CHECK_EQ(toString(result.errors[0]), + R"(Type '() -> ()' could not be converted into '() -> ()'; different number of generic type pack parameters)"); } TEST_SUITE_END(); diff --git a/tests/TypeInfer.provisional.test.cpp b/tests/TypeInfer.provisional.test.cpp index e6d3d4d4..47c13be9 100644 --- a/tests/TypeInfer.provisional.test.cpp +++ b/tests/TypeInfer.provisional.test.cpp @@ -271,6 +271,32 @@ TEST_CASE_FIXTURE(Fixture, "lvalue_equals_another_lvalue_with_no_overlap") CHECK_EQ(toString(requireTypeAtPosition({5, 36})), "boolean?"); // a ~= b } +// Also belongs in TypeInfer.refinements.test.cpp. +// Just needs to fully support equality refinement. Which is annoying without type states. +TEST_CASE_FIXTURE(Fixture, "discriminate_from_x_not_equal_to_nil") +{ + ScopedFastFlag sff{"LuauDiscriminableUnions", true}; + + CheckResult result = check(R"( + type T = {x: string, y: number} | {x: nil, y: nil} + + local function f(t: T) + if t.x ~= nil then + local foo = t + else + local bar = t + end + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ("{| x: string, y: number |}", toString(requireTypeAtPosition({5, 28}))); + + // Should be {| x: nil, y: nil |} + CHECK_EQ("{| x: nil, y: nil |} | {| x: string, y: number |}", toString(requireTypeAtPosition({7, 28}))); +} + TEST_CASE_FIXTURE(Fixture, "bail_early_if_unification_is_too_complicated" * doctest::timeout(0.5)) { ScopedFastInt sffi{"LuauTarjanChildLimit", 1}; @@ -590,8 +616,6 @@ TEST_CASE_FIXTURE(Fixture, "invariant_table_properties_means_instantiating_table TEST_CASE_FIXTURE(Fixture, "self_recursive_instantiated_param") { - ScopedFastFlag luauCloneCorrectlyBeforeMutatingTableType{"LuauCloneCorrectlyBeforeMutatingTableType", true}; - // Mutability in type function application right now can create strange recursive types // TODO: instantiation right now is problematic, in this example should either leave the Table type alone // or it should rename the type to 'Self' so that the result will be 'Self
' diff --git a/tests/TypeInfer.refinements.test.cpp b/tests/TypeInfer.refinements.test.cpp index d76b920b..f346ddfd 100644 --- a/tests/TypeInfer.refinements.test.cpp +++ b/tests/TypeInfer.refinements.test.cpp @@ -6,11 +6,77 @@ #include "doctest.h" +LUAU_FASTFLAG(LuauDiscriminableUnions) LUAU_FASTFLAG(LuauWeakEqConstraint) LUAU_FASTFLAG(LuauQuantifyInPlace2) using namespace Luau; +namespace +{ +std::optional> magicFunctionInstanceIsA( + TypeChecker& typeChecker, const ScopePtr& scope, const AstExprCall& expr, ExprResult exprResult) +{ + if (expr.args.size != 1) + return std::nullopt; + + auto index = expr.func->as(); + auto str = expr.args.data[0]->as(); + if (!index || !str) + return std::nullopt; + + std::optional lvalue = tryGetLValue(*index->expr); + std::optional tfun = scope->lookupType(std::string(str->value.data, str->value.size)); + if (!lvalue || !tfun) + return std::nullopt; + + unfreeze(typeChecker.globalTypes); + TypePackId booleanPack = typeChecker.globalTypes.addTypePack({typeChecker.booleanType}); + freeze(typeChecker.globalTypes); + return ExprResult{booleanPack, {IsAPredicate{std::move(*lvalue), expr.location, tfun->type}}}; +} + +struct RefinementClassFixture : Fixture +{ + RefinementClassFixture() + { + TypeArena& arena = typeChecker.globalTypes; + + unfreeze(arena); + TypeId vec3 = arena.addType(ClassTypeVar{"Vector3", {}, std::nullopt, std::nullopt, {}, nullptr}); + getMutable(vec3)->props = { + {"X", Property{typeChecker.numberType}}, + {"Y", Property{typeChecker.numberType}}, + {"Z", Property{typeChecker.numberType}}, + }; + + TypeId inst = arena.addType(ClassTypeVar{"Instance", {}, std::nullopt, std::nullopt, {}, nullptr}); + + TypePackId isAParams = arena.addTypePack({inst, typeChecker.stringType}); + TypePackId isARets = arena.addTypePack({typeChecker.booleanType}); + TypeId isA = arena.addType(FunctionTypeVar{isAParams, isARets}); + getMutable(isA)->magicFunction = magicFunctionInstanceIsA; + + getMutable(inst)->props = { + {"Name", Property{typeChecker.stringType}}, + {"IsA", Property{isA}}, + }; + + TypeId folder = typeChecker.globalTypes.addType(ClassTypeVar{"Folder", {}, inst, std::nullopt, {}, nullptr}); + TypeId part = typeChecker.globalTypes.addType(ClassTypeVar{"Part", {}, inst, std::nullopt, {}, nullptr}); + getMutable(part)->props = { + {"Position", Property{vec3}}, + }; + + typeChecker.globalScope->exportedTypeBindings["Vector3"] = TypeFun{{}, vec3}; + typeChecker.globalScope->exportedTypeBindings["Instance"] = TypeFun{{}, inst}; + typeChecker.globalScope->exportedTypeBindings["Folder"] = TypeFun{{}, folder}; + typeChecker.globalScope->exportedTypeBindings["Part"] = TypeFun{{}, part}; + freeze(typeChecker.globalTypes); + } +}; +} // namespace + TEST_SUITE_BEGIN("RefinementTest"); TEST_CASE_FIXTURE(Fixture, "is_truthy_constraint") @@ -196,8 +262,18 @@ TEST_CASE_FIXTURE(Fixture, "typeguard_only_look_up_types_from_global_scope") end )"); - LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK_EQ("Type 'number' has no overlap with 'string'", toString(result.errors[0])); + if (FFlag::LuauDiscriminableUnions) + { + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ("*unknown*", toString(requireTypeAtPosition({8, 44}))); + CHECK_EQ("*unknown*", toString(requireTypeAtPosition({9, 38}))); + } + else + { + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ("Type 'number' has no overlap with 'string'", toString(result.errors[0])); + } } TEST_CASE_FIXTURE(Fixture, "call_a_more_specific_function_using_typeguard") @@ -237,7 +313,6 @@ TEST_CASE_FIXTURE(Fixture, "impossible_type_narrow_is_not_an_error") TEST_CASE_FIXTURE(Fixture, "truthy_constraint_on_properties") { - CheckResult result = check(R"( local t: {x: number?} = {x = 1} @@ -254,7 +329,6 @@ TEST_CASE_FIXTURE(Fixture, "truthy_constraint_on_properties") TEST_CASE_FIXTURE(Fixture, "index_on_a_refined_property") { - CheckResult result = check(R"( local t: {x: {y: string}?} = {x = {y = "hello!"}} @@ -360,7 +434,10 @@ TEST_CASE_FIXTURE(Fixture, "lvalue_is_equal_to_a_term") TEST_CASE_FIXTURE(Fixture, "term_is_equal_to_an_lvalue") { - ScopedFastFlag sff1{"LuauEqConstraint", true}; + ScopedFastFlag sff[] = { + {"LuauDiscriminableUnions", true}, + {"LuauSingletonTypes", true}, + }; CheckResult result = check(R"( local function f(a: (string | number)?) @@ -374,16 +451,8 @@ TEST_CASE_FIXTURE(Fixture, "term_is_equal_to_an_lvalue") LUAU_REQUIRE_NO_ERRORS(result); - if (FFlag::LuauWeakEqConstraint) - { - CHECK_EQ(toString(requireTypeAtPosition({3, 28})), "(number | string)?"); // a == "hello" - CHECK_EQ(toString(requireTypeAtPosition({5, 28})), "(number | string)?"); // a ~= "hello" - } - else - { - CHECK_EQ(toString(requireTypeAtPosition({3, 28})), "string"); // a == "hello" - CHECK_EQ(toString(requireTypeAtPosition({5, 28})), "(number | string)?"); // a ~= "hello" - } + CHECK_EQ(toString(requireTypeAtPosition({3, 28})), R"("hello")"); // a == "hello" + CHECK_EQ(toString(requireTypeAtPosition({5, 28})), "(number | string)?"); // a ~= "hello" } TEST_CASE_FIXTURE(Fixture, "lvalue_is_not_nil") @@ -416,7 +485,8 @@ TEST_CASE_FIXTURE(Fixture, "lvalue_is_not_nil") TEST_CASE_FIXTURE(Fixture, "free_type_is_equal_to_an_lvalue") { - ScopedFastFlag sff1{"LuauEqConstraint", true}; + ScopedFastFlag sff{"LuauDiscriminableUnions", true}; + ScopedFastFlag sff2{"LuauWeakEqConstraint", true}; CheckResult result = check(R"( local function f(a, b: string?) @@ -428,16 +498,8 @@ TEST_CASE_FIXTURE(Fixture, "free_type_is_equal_to_an_lvalue") LUAU_REQUIRE_NO_ERRORS(result); - if (FFlag::LuauWeakEqConstraint) - { - CHECK_EQ(toString(requireTypeAtPosition({3, 33})), "a"); // a == b - CHECK_EQ(toString(requireTypeAtPosition({3, 36})), "string?"); // a == b - } - else - { - CHECK_EQ(toString(requireTypeAtPosition({3, 33})), "string?"); // a == b - CHECK_EQ(toString(requireTypeAtPosition({3, 36})), "string?"); // a == b - } + CHECK_EQ(toString(requireTypeAtPosition({3, 33})), "a"); // a == b + CHECK_EQ(toString(requireTypeAtPosition({3, 36})), "string?"); // a == b } TEST_CASE_FIXTURE(Fixture, "unknown_lvalue_is_not_synonymous_with_other_on_not_equal") @@ -527,9 +589,17 @@ TEST_CASE_FIXTURE(Fixture, "type_narrow_to_vector") end )"); - // This is kinda weird to see, but this actually only happens in Luau without Roblox type bindings because we don't have a Vector3 type. - LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK_EQ("Unknown type 'Vector3'", toString(result.errors[0])); + if (FFlag::LuauDiscriminableUnions) + { + LUAU_REQUIRE_NO_ERRORS(result); + } + else + { + // This is kinda weird to see, but this actually only happens in Luau without Roblox type bindings because we don't have a Vector3 type. + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ("Unknown type 'Vector3'", toString(result.errors[0])); + } + CHECK_EQ("*unknown*", toString(requireTypeAtPosition({3, 28}))); } @@ -614,214 +684,6 @@ TEST_CASE_FIXTURE(Fixture, "typeguard_narrows_for_functions") CHECK_EQ("string", toString(requireTypeAtPosition({5, 28}))); // type(x) ~= "function" } -namespace -{ -std::optional> magicFunctionInstanceIsA( - TypeChecker& typeChecker, const ScopePtr& scope, const AstExprCall& expr, ExprResult exprResult) -{ - if (expr.args.size != 1) - return std::nullopt; - - auto index = expr.func->as(); - auto str = expr.args.data[0]->as(); - if (!index || !str) - return std::nullopt; - - std::optional lvalue = tryGetLValue(*index->expr); - std::optional tfun = scope->lookupType(std::string(str->value.data, str->value.size)); - if (!lvalue || !tfun) - return std::nullopt; - - unfreeze(typeChecker.globalTypes); - TypePackId booleanPack = typeChecker.globalTypes.addTypePack({typeChecker.booleanType}); - freeze(typeChecker.globalTypes); - return ExprResult{booleanPack, {IsAPredicate{std::move(*lvalue), expr.location, tfun->type}}}; -} - -struct RefinementClassFixture : Fixture -{ - RefinementClassFixture() - { - TypeArena& arena = typeChecker.globalTypes; - - unfreeze(arena); - TypeId vec3 = arena.addType(ClassTypeVar{"Vector3", {}, std::nullopt, std::nullopt, {}, nullptr}); - getMutable(vec3)->props = { - {"X", Property{typeChecker.numberType}}, - {"Y", Property{typeChecker.numberType}}, - {"Z", Property{typeChecker.numberType}}, - }; - - TypeId inst = arena.addType(ClassTypeVar{"Instance", {}, std::nullopt, std::nullopt, {}, nullptr}); - - TypePackId isAParams = arena.addTypePack({inst, typeChecker.stringType}); - TypePackId isARets = arena.addTypePack({typeChecker.booleanType}); - TypeId isA = arena.addType(FunctionTypeVar{isAParams, isARets}); - getMutable(isA)->magicFunction = magicFunctionInstanceIsA; - - getMutable(inst)->props = { - {"Name", Property{typeChecker.stringType}}, - {"IsA", Property{isA}}, - }; - - TypeId folder = typeChecker.globalTypes.addType(ClassTypeVar{"Folder", {}, inst, std::nullopt, {}, nullptr}); - TypeId part = typeChecker.globalTypes.addType(ClassTypeVar{"Part", {}, inst, std::nullopt, {}, nullptr}); - getMutable(part)->props = { - {"Position", Property{vec3}}, - }; - - typeChecker.globalScope->exportedTypeBindings["Vector3"] = TypeFun{{}, vec3}; - typeChecker.globalScope->exportedTypeBindings["Instance"] = TypeFun{{}, inst}; - typeChecker.globalScope->exportedTypeBindings["Folder"] = TypeFun{{}, folder}; - typeChecker.globalScope->exportedTypeBindings["Part"] = TypeFun{{}, part}; - freeze(typeChecker.globalTypes); - } -}; -} // namespace - -TEST_CASE_FIXTURE(RefinementClassFixture, "typeguard_cast_free_table_to_vector") -{ - CheckResult result = check(R"( - local function f(vec) - local X, Y, Z = vec.X, vec.Y, vec.Z - - if type(vec) == "vector" then - local foo = vec - elseif typeof(vec) == "Instance" then - local foo = vec - else - local foo = vec - end - end - )"); - - LUAU_REQUIRE_ERROR_COUNT(1, result); - - CHECK_EQ("Vector3", toString(requireTypeAtPosition({5, 28}))); // type(vec) == "vector" - - if (FFlag::LuauQuantifyInPlace2) - CHECK_EQ("Type '{+ X: a, Y: b, Z: c +}' could not be converted into 'Instance'", toString(result.errors[0])); - else - CHECK_EQ("Type '{- X: a, Y: b, Z: c -}' could not be converted into 'Instance'", toString(result.errors[0])); - - CHECK_EQ("*unknown*", toString(requireTypeAtPosition({7, 28}))); // typeof(vec) == "Instance" - - if (FFlag::LuauQuantifyInPlace2) - CHECK_EQ("{+ X: a, Y: b, Z: c +}", toString(requireTypeAtPosition({9, 28}))); // type(vec) ~= "vector" and typeof(vec) ~= "Instance" - else - CHECK_EQ("{- X: a, Y: b, Z: c -}", toString(requireTypeAtPosition({9, 28}))); // type(vec) ~= "vector" and typeof(vec) ~= "Instance" -} - -TEST_CASE_FIXTURE(RefinementClassFixture, "typeguard_cast_instance_or_vector3_to_vector") -{ - CheckResult result = check(R"( - local function f(x: Instance | Vector3) - if typeof(x) == "Vector3" then - local foo = x - else - local foo = x - end - end - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - - CHECK_EQ("Vector3", toString(requireTypeAtPosition({3, 28}))); - CHECK_EQ("Instance", toString(requireTypeAtPosition({5, 28}))); -} - -TEST_CASE_FIXTURE(RefinementClassFixture, "type_narrow_for_all_the_userdata") -{ - CheckResult result = check(R"( - local function f(x: string | number | Instance | Vector3) - if type(x) == "userdata" then - local foo = x - else - local foo = x - end - end - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - - CHECK_EQ("Instance | Vector3", toString(requireTypeAtPosition({3, 28}))); - CHECK_EQ("number | string", toString(requireTypeAtPosition({5, 28}))); -} - -TEST_CASE_FIXTURE(RefinementClassFixture, "eliminate_subclasses_of_instance") -{ - CheckResult result = check(R"( - local function f(x: Part | Folder | string) - if typeof(x) == "Instance" then - local foo = x - else - local foo = x - end - end - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - - CHECK_EQ("Folder | Part", toString(requireTypeAtPosition({3, 28}))); - CHECK_EQ("string", toString(requireTypeAtPosition({5, 28}))); -} - -TEST_CASE_FIXTURE(RefinementClassFixture, "narrow_this_large_union") -{ - CheckResult result = check(R"( - local function f(x: Part | Folder | Instance | string | Vector3 | any) - if typeof(x) == "Instance" then - local foo = x - else - local foo = x - end - end - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - - CHECK_EQ("Folder | Instance | Part", toString(requireTypeAtPosition({3, 28}))); - CHECK_EQ("Vector3 | any | string", toString(requireTypeAtPosition({5, 28}))); -} - -TEST_CASE_FIXTURE(RefinementClassFixture, "x_as_any_if_x_is_instance_elseif_x_is_table") -{ - CheckResult result = check(R"( - --!nonstrict - - local function f(x) - if typeof(x) == "Instance" and x:IsA("Folder") then - local foo = x - elseif typeof(x) == "table" then - local foo = x - end - end - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - - CHECK_EQ("Folder", toString(requireTypeAtPosition({5, 28}))); - CHECK_EQ("any", toString(requireTypeAtPosition({7, 28}))); -} - -TEST_CASE_FIXTURE(RefinementClassFixture, "x_is_not_instance_or_else_not_part") -{ - CheckResult result = check(R"( - local function f(x: Part | Folder | string) - if typeof(x) ~= "Instance" or not x:IsA("Part") then - local foo = x - else - local foo = x - end - end - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - - CHECK_EQ("Folder | string", toString(requireTypeAtPosition({3, 28}))); - CHECK_EQ("Part", toString(requireTypeAtPosition({5, 28}))); -} - TEST_CASE_FIXTURE(Fixture, "type_guard_can_filter_for_intersection_of_tables") { CheckResult result = check(R"( @@ -1145,4 +1007,259 @@ TEST_CASE_FIXTURE(Fixture, "apply_refinements_on_astexprindexexpr_whose_subscrip LUAU_REQUIRE_NO_ERRORS(result); } +TEST_CASE_FIXTURE(Fixture, "discriminate_from_truthiness_of_x") +{ + ScopedFastFlag sff[] = { + {"LuauDiscriminableUnions", true}, + {"LuauParseSingletonTypes", true}, + {"LuauSingletonTypes", true}, + }; + + CheckResult result = check(R"( + type T = {tag: "missing", x: nil} | {tag: "exists", x: string} + + local function f(t: T) + if t.x then + local foo = t + else + local bar = t + end + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ(R"({| tag: "exists", x: string |})", toString(requireTypeAtPosition({5, 28}))); + CHECK_EQ(R"({| tag: "missing", x: nil |})", toString(requireTypeAtPosition({7, 28}))); +} + +TEST_CASE_FIXTURE(Fixture, "discriminate_tag") +{ + ScopedFastFlag sff[] = { + {"LuauDiscriminableUnions", true}, + {"LuauParseSingletonTypes", true}, + {"LuauSingletonTypes", true}, + }; + + CheckResult result = check(R"( + type Cat = {tag: "Cat", name: string, catfood: string} + type Dog = {tag: "Dog", name: string, dogfood: string} + type Animal = Cat | Dog + + local function f(animal: Animal) + if animal.tag == "Cat" then + local cat: Cat = animal + elseif animal.tag == "Dog" then + local dog: Dog = animal + end + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ("Cat", toString(requireTypeAtPosition({7, 33}))); + CHECK_EQ("Dog", toString(requireTypeAtPosition({9, 33}))); +} + +TEST_CASE_FIXTURE(Fixture, "apply_refinements_on_astexprindexexpr_whose_subscript_expr_is_constant_string") +{ + ScopedFastFlag sff{"LuauRefiLookupFromIndexExpr", true}; + + CheckResult result = check(R"( + type T = { [string]: { prop: number }? } + local t: T = {} + + if t["hello"] then + local foo = t["hello"].prop + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "and_or_peephole_refinement") +{ + CheckResult result = check(R"( + local function len(a: {any}) + return a and #a or nil + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(RefinementClassFixture, "discriminate_from_isa_of_x") +{ + ScopedFastFlag sff[] = { + {"LuauDiscriminableUnions", true}, + {"LuauParseSingletonTypes", true}, + {"LuauSingletonTypes", true}, + }; + + CheckResult result = check(R"( + type T = {tag: "Part", x: Part} | {tag: "Folder", x: Folder} + + local function f(t: T) + if t.x:IsA("Part") then + local foo = t + else + local bar = t + end + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ(R"({| tag: "Part", x: Part |})", toString(requireTypeAtPosition({5, 28}))); + CHECK_EQ(R"({| tag: "Folder", x: Folder |})", toString(requireTypeAtPosition({7, 28}))); +} + +TEST_CASE_FIXTURE(RefinementClassFixture, "typeguard_cast_free_table_to_vector") +{ + CheckResult result = check(R"( + local function f(vec) + local X, Y, Z = vec.X, vec.Y, vec.Z + + if type(vec) == "vector" then + local foo = vec + elseif typeof(vec) == "Instance" then + local foo = vec + else + local foo = vec + end + end + )"); + + if (FFlag::LuauDiscriminableUnions) + LUAU_REQUIRE_NO_ERRORS(result); + else + { + LUAU_REQUIRE_ERROR_COUNT(1, result); + + if (FFlag::LuauQuantifyInPlace2) + CHECK_EQ("Type '{+ X: a, Y: b, Z: c +}' could not be converted into 'Instance'", toString(result.errors[0])); + else + CHECK_EQ("Type '{- X: a, Y: b, Z: c -}' could not be converted into 'Instance'", toString(result.errors[0])); + } + + CHECK_EQ("Vector3", toString(requireTypeAtPosition({5, 28}))); // type(vec) == "vector" + + CHECK_EQ("*unknown*", toString(requireTypeAtPosition({7, 28}))); // typeof(vec) == "Instance" + + if (FFlag::LuauQuantifyInPlace2) + CHECK_EQ("{+ X: a, Y: b, Z: c +}", toString(requireTypeAtPosition({9, 28}))); // type(vec) ~= "vector" and typeof(vec) ~= "Instance" + else + CHECK_EQ("{- X: a, Y: b, Z: c -}", toString(requireTypeAtPosition({9, 28}))); // type(vec) ~= "vector" and typeof(vec) ~= "Instance" +} + +TEST_CASE_FIXTURE(RefinementClassFixture, "typeguard_cast_instance_or_vector3_to_vector") +{ + CheckResult result = check(R"( + local function f(x: Instance | Vector3) + if typeof(x) == "Vector3" then + local foo = x + else + local foo = x + end + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ("Vector3", toString(requireTypeAtPosition({3, 28}))); + CHECK_EQ("Instance", toString(requireTypeAtPosition({5, 28}))); +} + +TEST_CASE_FIXTURE(RefinementClassFixture, "type_narrow_for_all_the_userdata") +{ + CheckResult result = check(R"( + local function f(x: string | number | Instance | Vector3) + if type(x) == "userdata" then + local foo = x + else + local foo = x + end + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ("Instance | Vector3", toString(requireTypeAtPosition({3, 28}))); + CHECK_EQ("number | string", toString(requireTypeAtPosition({5, 28}))); +} + +TEST_CASE_FIXTURE(RefinementClassFixture, "eliminate_subclasses_of_instance") +{ + CheckResult result = check(R"( + local function f(x: Part | Folder | string) + if typeof(x) == "Instance" then + local foo = x + else + local foo = x + end + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ("Folder | Part", toString(requireTypeAtPosition({3, 28}))); + CHECK_EQ("string", toString(requireTypeAtPosition({5, 28}))); +} + +TEST_CASE_FIXTURE(RefinementClassFixture, "narrow_this_large_union") +{ + CheckResult result = check(R"( + local function f(x: Part | Folder | Instance | string | Vector3 | any) + if typeof(x) == "Instance" then + local foo = x + else + local foo = x + end + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ("Folder | Instance | Part", toString(requireTypeAtPosition({3, 28}))); + CHECK_EQ("Vector3 | any | string", toString(requireTypeAtPosition({5, 28}))); +} + +TEST_CASE_FIXTURE(RefinementClassFixture, "x_as_any_if_x_is_instance_elseif_x_is_table") +{ + CheckResult result = check(R"( + --!nonstrict + + local function f(x) + if typeof(x) == "Instance" and x:IsA("Folder") then + local foo = x + elseif typeof(x) == "table" then + local foo = x + end + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ("Folder", toString(requireTypeAtPosition({5, 28}))); + CHECK_EQ("any", toString(requireTypeAtPosition({7, 28}))); +} + +TEST_CASE_FIXTURE(RefinementClassFixture, "x_is_not_instance_or_else_not_part") +{ + CheckResult result = check(R"( + local function f(x: Part | Folder | string) + if typeof(x) ~= "Instance" or not x:IsA("Part") then + local foo = x + else + local foo = x + end + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ("Folder | string", toString(requireTypeAtPosition({3, 28}))); + CHECK_EQ("Part", toString(requireTypeAtPosition({5, 28}))); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.singletons.test.cpp b/tests/TypeInfer.singletons.test.cpp index 94cfb643..df365fda 100644 --- a/tests/TypeInfer.singletons.test.cpp +++ b/tests/TypeInfer.singletons.test.cpp @@ -379,9 +379,7 @@ TEST_CASE_FIXTURE(Fixture, "error_detailed_tagged_union_mismatch_string") ScopedFastFlag sffs[] = { {"LuauSingletonTypes", true}, {"LuauParseSingletonTypes", true}, - {"LuauUnionHeuristic", true}, {"LuauExpectedTypesOfProperties", true}, - {"LuauExtendedUnionMismatchError", true}, }; CheckResult result = check(R"( @@ -404,9 +402,7 @@ TEST_CASE_FIXTURE(Fixture, "error_detailed_tagged_union_mismatch_bool") ScopedFastFlag sffs[] = { {"LuauSingletonTypes", true}, {"LuauParseSingletonTypes", true}, - {"LuauUnionHeuristic", true}, {"LuauExpectedTypesOfProperties", true}, - {"LuauExtendedUnionMismatchError", true}, }; CheckResult result = check(R"( @@ -429,9 +425,7 @@ TEST_CASE_FIXTURE(Fixture, "if_then_else_expression_singleton_options") ScopedFastFlag sffs[] = { {"LuauSingletonTypes", true}, {"LuauParseSingletonTypes", true}, - {"LuauUnionHeuristic", true}, {"LuauExpectedTypesOfProperties", true}, - {"LuauExtendedUnionMismatchError", true}, {"LuauIfElseExpectedType2", true}, {"LuauIfElseBranchTypeUnion", true}, }; diff --git a/tests/TypeInfer.tables.test.cpp b/tests/TypeInfer.tables.test.cpp index 644efed7..48310921 100644 --- a/tests/TypeInfer.tables.test.cpp +++ b/tests/TypeInfer.tables.test.cpp @@ -12,8 +12,6 @@ using namespace Luau; -LUAU_FASTFLAG(LuauExtendedFunctionMismatchError) - TEST_SUITE_BEGIN("TableTests"); TEST_CASE_FIXTURE(Fixture, "basic") @@ -2075,22 +2073,11 @@ caused by: caused by: Property 'y' is not compatible. Type 'string' could not be converted into 'number')"); - if (FFlag::LuauExtendedFunctionMismatchError) - { - CHECK_EQ(toString(result.errors[1]), R"(Type 'b2' could not be converted into 'a2' + CHECK_EQ(toString(result.errors[1]), R"(Type 'b2' could not be converted into 'a2' caused by: Type '{ __call: (a, b) -> () }' could not be converted into '{ __call: (a) -> () }' caused by: Property '__call' is not compatible. Type '(a, b) -> ()' could not be converted into '(a) -> ()'; different number of generic type parameters)"); - } - else - { - CHECK_EQ(toString(result.errors[1]), R"(Type 'b2' could not be converted into 'a2' -caused by: - Type '{ __call: (a, b) -> () }' could not be converted into '{ __call: (a) -> () }' -caused by: - Property '__call' is not compatible. Type '(a, b) -> ()' could not be converted into '(a) -> ()')"); - } } TEST_CASE_FIXTURE(Fixture, "explicitly_typed_table") @@ -2166,7 +2153,6 @@ a.p = { x = 9 } TEST_CASE_FIXTURE(Fixture, "recursive_metatable_type_call") { ScopedFastFlag sff[]{ - {"LuauFixRecursiveMetatableCall", true}, {"LuauUnsealedTableLiteral", true}, }; diff --git a/tests/TypeInfer.test.cpp b/tests/TypeInfer.test.cpp index 7ee5253c..c9b30e1a 100644 --- a/tests/TypeInfer.test.cpp +++ b/tests/TypeInfer.test.cpp @@ -16,7 +16,6 @@ LUAU_FASTFLAG(LuauFixLocationSpanTableIndexExpr) LUAU_FASTFLAG(LuauEqConstraint) -LUAU_FASTFLAG(LuauExtendedFunctionMismatchError) using namespace Luau; @@ -959,8 +958,6 @@ TEST_CASE_FIXTURE(Fixture, "another_recursive_local_function") TEST_CASE_FIXTURE(Fixture, "cyclic_function_type_in_rets") { - ScopedFastFlag sff{"LuauOccursCheckOkWithRecursiveFunctions", true}; - CheckResult result = check(R"( function f() return f @@ -973,8 +970,6 @@ TEST_CASE_FIXTURE(Fixture, "cyclic_function_type_in_rets") TEST_CASE_FIXTURE(Fixture, "cyclic_function_type_in_args") { - ScopedFastFlag sff{"LuauOccursCheckOkWithRecursiveFunctions", true}; - CheckResult result = check(R"( function f(g) return f(f) @@ -1699,8 +1694,6 @@ TEST_CASE_FIXTURE(Fixture, "first_argument_can_be_optional") TEST_CASE_FIXTURE(Fixture, "dont_ice_when_failing_the_occurs_check") { - ScopedFastFlag sff{"LuauOccursCheckOkWithRecursiveFunctions", true}; - CheckResult result = check(R"( --!strict local s @@ -1711,8 +1704,6 @@ TEST_CASE_FIXTURE(Fixture, "dont_ice_when_failing_the_occurs_check") TEST_CASE_FIXTURE(Fixture, "occurs_check_does_not_recurse_forever_if_asked_to_traverse_a_cyclic_type") { - ScopedFastFlag sff{"LuauOccursCheckOkWithRecursiveFunctions", true}; - CheckResult result = check(R"( --!strict function u(t, w) @@ -3326,11 +3317,12 @@ TEST_CASE_FIXTURE(Fixture, "unknown_type_in_comparison") LUAU_REQUIRE_NO_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "relation_op_on_any_lhs_where_rhs_maybe_has_metatable") +TEST_CASE_FIXTURE(Fixture, "concat_op_on_free_lhs_and_string_rhs") { CheckResult result = check(R"( - local x - print((x == true and (x .. "y")) .. 1) + local function f(x) + return x .. "y" + end )"); LUAU_REQUIRE_ERROR_COUNT(1, result); @@ -3340,13 +3332,14 @@ TEST_CASE_FIXTURE(Fixture, "relation_op_on_any_lhs_where_rhs_maybe_has_metatable TEST_CASE_FIXTURE(Fixture, "concat_op_on_string_lhs_and_free_rhs") { CheckResult result = check(R"( - local x - print("foo" .. x) + local function f(x) + return "foo" .. x + end )"); LUAU_REQUIRE_NO_ERRORS(result); - CHECK_EQ("string", toString(requireType("x"))); + CHECK_EQ("(string) -> string", toString(requireType("f"))); } TEST_CASE_FIXTURE(Fixture, "strict_binary_op_where_lhs_unknown") @@ -4374,8 +4367,6 @@ TEST_CASE_FIXTURE(Fixture, "recursive_types_restriction_not_ok") TEST_CASE_FIXTURE(Fixture, "record_matching_overload") { - ScopedFastFlag sffs("LuauStoreMatchingOverloadFnType", true); - CheckResult result = check(R"( type Overload = ((string) -> string) & ((number) -> number) local abc: Overload @@ -4475,17 +4466,10 @@ f(function(a, b, c, ...) return a + b end) LUAU_REQUIRE_ERRORS(result); - if (FFlag::LuauExtendedFunctionMismatchError) - { - CHECK_EQ(R"(Type '(number, number, a) -> number' could not be converted into '(number, number) -> number' + CHECK_EQ(R"(Type '(number, number, a) -> number' could not be converted into '(number, number) -> number' caused by: Argument count mismatch. Function expects 3 arguments, but only 2 are specified)", - toString(result.errors[0])); - } - else - { - CHECK_EQ(R"(Type '(number, number, a) -> number' could not be converted into '(number, number) -> number')", toString(result.errors[0])); - } + toString(result.errors[0])); // Infer from variadic packs into elements result = check(R"( @@ -4618,17 +4602,9 @@ local c = sumrec(function(x, y, f) return f(x, y) end) -- type binders are not i )"); LUAU_REQUIRE_ERRORS(result); - if (FFlag::LuauExtendedFunctionMismatchError) - { - CHECK_EQ( - "Type '(a, b, (a, b) -> (c...)) -> (c...)' could not be converted into '(a, a, (a, a) -> a) -> a'; different number of generic type " - "parameters", - toString(result.errors[0])); - } - else - { - CHECK_EQ("Type '(a, b, (a, b) -> (c...)) -> (c...)' could not be converted into '(a, a, (a, a) -> a) -> a'", toString(result.errors[0])); - } + CHECK_EQ("Type '(a, b, (a, b) -> (c...)) -> (c...)' could not be converted into '(a, a, (a, a) -> a) -> a'; different number of generic type " + "parameters", + toString(result.errors[0])); } TEST_CASE_FIXTURE(Fixture, "infer_return_value_type") @@ -4741,8 +4717,6 @@ TEST_CASE_FIXTURE(Fixture, "accidentally_checked_prop_in_opposite_branch") TEST_CASE_FIXTURE(Fixture, "substitution_with_bound_table") { - ScopedFastFlag luauCloneCorrectlyBeforeMutatingTableType{"LuauCloneCorrectlyBeforeMutatingTableType", true}; - CheckResult result = check(R"( type A = { x: number } local a: A = { x = 1 } @@ -4965,8 +4939,6 @@ TEST_CASE_FIXTURE(Fixture, "inferred_methods_of_free_tables_have_the_same_level_ TEST_CASE_FIXTURE(Fixture, "error_detailed_function_mismatch_arg_count") { - ScopedFastFlag luauExtendedFunctionMismatchError{"LuauExtendedFunctionMismatchError", true}; - CheckResult result = check(R"( type A = (number, number) -> string type B = (number) -> string @@ -4983,8 +4955,6 @@ caused by: TEST_CASE_FIXTURE(Fixture, "error_detailed_function_mismatch_arg") { - ScopedFastFlag luauExtendedFunctionMismatchError{"LuauExtendedFunctionMismatchError", true}; - CheckResult result = check(R"( type A = (number, number) -> string type B = (number, string) -> string @@ -5001,8 +4971,6 @@ caused by: TEST_CASE_FIXTURE(Fixture, "error_detailed_function_mismatch_ret_count") { - ScopedFastFlag luauExtendedFunctionMismatchError{"LuauExtendedFunctionMismatchError", true}; - CheckResult result = check(R"( type A = (number, number) -> (number) type B = (number, number) -> (number, boolean) @@ -5019,8 +4987,6 @@ caused by: TEST_CASE_FIXTURE(Fixture, "error_detailed_function_mismatch_ret") { - ScopedFastFlag luauExtendedFunctionMismatchError{"LuauExtendedFunctionMismatchError", true}; - CheckResult result = check(R"( type A = (number, number) -> string type B = (number, number) -> number @@ -5037,8 +5003,6 @@ caused by: TEST_CASE_FIXTURE(Fixture, "error_detailed_function_mismatch_ret_mult") { - ScopedFastFlag luauExtendedFunctionMismatchError{"LuauExtendedFunctionMismatchError", true}; - CheckResult result = check(R"( type A = (number, number) -> (number, string) type B = (number, number) -> (number, boolean) @@ -5069,8 +5033,6 @@ TEST_CASE_FIXTURE(Fixture, "prop_access_on_any_with_other_options") TEST_CASE_FIXTURE(Fixture, "table_function_check_use_after_free") { - ScopedFastFlag luauUnifyFunctionCheckResult{"LuauUpdateFunctionNameBinding", true}; - CheckResult result = check(R"( local t = {} diff --git a/tests/TypeInfer.typePacks.cpp b/tests/TypeInfer.typePacks.cpp index d4878d14..079870f5 100644 --- a/tests/TypeInfer.typePacks.cpp +++ b/tests/TypeInfer.typePacks.cpp @@ -931,7 +931,6 @@ type R = { m: F } TEST_CASE_FIXTURE(Fixture, "pack_tail_unification_check") { ScopedFastFlag luauUnifyPackTails{"LuauUnifyPackTails", true}; - ScopedFastFlag luauExtendedFunctionMismatchError{"LuauExtendedFunctionMismatchError", true}; CheckResult result = check(R"( local a: () -> (number, ...string) diff --git a/tests/TypeInfer.unionTypes.test.cpp b/tests/TypeInfer.unionTypes.test.cpp index b54ba996..759794e6 100644 --- a/tests/TypeInfer.unionTypes.test.cpp +++ b/tests/TypeInfer.unionTypes.test.cpp @@ -464,8 +464,6 @@ local a: XYZ = { w = 4 } TEST_CASE_FIXTURE(Fixture, "error_detailed_optional") { - ScopedFastFlag luauExtendedUnionMismatchError{"LuauExtendedUnionMismatchError", true}; - CheckResult result = check(R"( type X = { x: number } diff --git a/tests/TypeVar.test.cpp b/tests/TypeVar.test.cpp index 2e0d149e..329e7b1f 100644 --- a/tests/TypeVar.test.cpp +++ b/tests/TypeVar.test.cpp @@ -268,8 +268,6 @@ TEST_CASE_FIXTURE(Fixture, "substitution_skip_failure") TEST_CASE("tagging_tables") { - ScopedFastFlag sff{"LuauRefactorTagging", true}; - TypeVar ttv{TableTypeVar{}}; CHECK(!Luau::hasTag(&ttv, "foo")); Luau::attachTag(&ttv, "foo"); @@ -278,8 +276,6 @@ TEST_CASE("tagging_tables") TEST_CASE("tagging_classes") { - ScopedFastFlag sff{"LuauRefactorTagging", true}; - TypeVar base{ClassTypeVar{"Base", {}, std::nullopt, std::nullopt, {}, nullptr}}; CHECK(!Luau::hasTag(&base, "foo")); Luau::attachTag(&base, "foo"); @@ -288,8 +284,6 @@ TEST_CASE("tagging_classes") TEST_CASE("tagging_subclasses") { - ScopedFastFlag sff{"LuauRefactorTagging", true}; - TypeVar base{ClassTypeVar{"Base", {}, std::nullopt, std::nullopt, {}, nullptr}}; TypeVar derived{ClassTypeVar{"Derived", {}, &base, std::nullopt, {}, nullptr}}; @@ -307,8 +301,6 @@ TEST_CASE("tagging_subclasses") TEST_CASE("tagging_functions") { - ScopedFastFlag sff{"LuauRefactorTagging", true}; - TypePackVar empty{TypePack{}}; TypeVar ftv{FunctionTypeVar{&empty, &empty}}; CHECK(!Luau::hasTag(&ftv, "foo")); @@ -318,8 +310,6 @@ TEST_CASE("tagging_functions") TEST_CASE("tagging_props") { - ScopedFastFlag sff{"LuauRefactorTagging", true}; - Property prop{}; CHECK(!Luau::hasTag(prop, "foo")); Luau::attachTag(prop, "foo"); @@ -370,4 +360,66 @@ local b: (T, T, T) -> T CHECK_EQ(count, 1); } +TEST_CASE("isString_on_string_singletons") +{ + ScopedFastFlag sff{"LuauRefactorTypeVarQuestions", true}; + + TypeVar helloString{SingletonTypeVar{StringSingleton{"hello"}}}; + CHECK(isString(&helloString)); +} + +TEST_CASE("isString_on_unions_of_various_string_singletons") +{ + ScopedFastFlag sff{"LuauRefactorTypeVarQuestions", true}; + + TypeVar helloString{SingletonTypeVar{StringSingleton{"hello"}}}; + TypeVar byeString{SingletonTypeVar{StringSingleton{"bye"}}}; + TypeVar union_{UnionTypeVar{{&helloString, &byeString}}}; + + CHECK(isString(&union_)); +} + +TEST_CASE("proof_that_isString_uses_all_of") +{ + ScopedFastFlag sff{"LuauRefactorTypeVarQuestions", true}; + + TypeVar helloString{SingletonTypeVar{StringSingleton{"hello"}}}; + TypeVar byeString{SingletonTypeVar{StringSingleton{"bye"}}}; + TypeVar booleanType{PrimitiveTypeVar{PrimitiveTypeVar::Boolean}}; + TypeVar union_{UnionTypeVar{{&helloString, &byeString, &booleanType}}}; + + CHECK(!isString(&union_)); +} + +TEST_CASE("isBoolean_on_boolean_singletons") +{ + ScopedFastFlag sff{"LuauRefactorTypeVarQuestions", true}; + + TypeVar trueBool{SingletonTypeVar{BooleanSingleton{true}}}; + CHECK(isBoolean(&trueBool)); +} + +TEST_CASE("isBoolean_on_unions_of_true_or_false_singletons") +{ + ScopedFastFlag sff{"LuauRefactorTypeVarQuestions", true}; + + TypeVar trueBool{SingletonTypeVar{BooleanSingleton{true}}}; + TypeVar falseBool{SingletonTypeVar{BooleanSingleton{false}}}; + TypeVar union_{UnionTypeVar{{&trueBool, &falseBool}}}; + + CHECK(isBoolean(&union_)); +} + +TEST_CASE("proof_that_isBoolean_uses_all_of") +{ + ScopedFastFlag sff{"LuauRefactorTypeVarQuestions", true}; + + TypeVar trueBool{SingletonTypeVar{BooleanSingleton{true}}}; + TypeVar falseBool{SingletonTypeVar{BooleanSingleton{false}}}; + TypeVar stringType{PrimitiveTypeVar{PrimitiveTypeVar::String}}; + TypeVar union_{UnionTypeVar{{&trueBool, &falseBool, &stringType}}}; + + CHECK(!isBoolean(&union_)); +} + TEST_SUITE_END(); From c572f6944f130894ab3e9e8e2ff2b65ad4883076 Mon Sep 17 00:00:00 2001 From: Andy Friesen Date: Tue, 1 Feb 2022 14:49:55 -0800 Subject: [PATCH 142/399] January 2022 recap (#331) --- .../2022-01-27-luau-recap-january-2022.md | 122 ++++++++++++++++++ 1 file changed, 122 insertions(+) create mode 100644 docs/_posts/2022-01-27-luau-recap-january-2022.md diff --git a/docs/_posts/2022-01-27-luau-recap-january-2022.md b/docs/_posts/2022-01-27-luau-recap-january-2022.md new file mode 100644 index 00000000..5ee10022 --- /dev/null +++ b/docs/_posts/2022-01-27-luau-recap-january-2022.md @@ -0,0 +1,122 @@ +--- +layout: single +title: "Luau Recap: January 2022" +--- + +Luau is our programming language that you can read more about at [https://luau-lang.org](https://luau-lang.org). + +[Find us on GitHub](https://github.com/Roblox/luau)! + +[Cross-posted to the [Roblox Developer Forum](https://devforum.roblox.com/t/luau-recap-january-2022/).] + +## Performance improvements + +The implementation of `tostring` has been rewritten. This change replaces the default number->string conversion with a +new algorithm called Schubfach, which allows us to produce the shortest precise round-trippable representation of any +input number very quickly. + +While performance is not the main driving factor, this also happens to be significantly faster than our old +implementation (up to 10x depending on the number and the platform). + +--- + +Make `tonumber(x)` ~2x faster by avoiding reparsing string arguments. + +--- + +The Luau compiler now optimizes table literals where keys are constant variables the same way as if they were constants, eg + +```lua +local r, g, b = 1, 2, 3 +local col = { [r] = 255, [g] = 0, [b] = 255 } +``` + +## Improvements to type assertions + +The `::` type assertion operator can now be used to coerce a value between any two related types. Previously, it could +only be used for downcasts or casts to `any`. The following used to be invalid, but is now valid: + +```lua +local t = {x=0, y=0} +local a = t :: {x: number} +``` + +## Typechecking improvements + +An issue surrounding table literals and indexers has been fixed: + +```lua +type RecolorMap = {[string]: RecolorMap | Color3} + +local hatRecolorMap: RecolorMap = { + Brim = Color3.fromRGB(255, 0, 0), -- We used to report an error here + Top = Color3.fromRGB(255, 0, 0) +} +``` + +--- +Accessing a property whose base expression was previously refined will now return the correct result. + +## Linter improvements + +`table.create(N, {})` will now produce a static analysis warning since the element is going to be shared for all table entries. + +## Error reporting improvements + +When a type error involves a union (or an option), we now provide more context in the error message. + +For instance, given the following code: + +```lua +--!strict + +type T = {x: number} + +local x: T? = {w=4} +``` + +We now report the following: + +``` +Type 'x' could not be converted into 'T?' +caused by: + None of the union options are compatible. For example: Table type 'x' not compatible with type 'T' because the former is missing field 'x' +``` + +--- +Luau now gives up and reports an `*unknown*` type in far fewer cases when typechecking programs that have type errors. + +## New APIs + +We have brought in the [`coroutine.close`](https://luau-lang.org/library#coroutine-library) function from Lua 5.4. It accepts a suspended coroutine and marks it as non-runnable. In Roblox, this can be useful in combination with `task.defer` to implement cancellation. + +## REPL improvements + +The `luau` REPL application can be compiled from source or downloaded from [releases page](https://github.com/Roblox/luau/releases). It has grown some new features: + +* Added `--interactive` option to run the REPL after running the last script file. +* Allowed the compiler optimization level to be specified. +* Allowed methods to be tab completed +* Allowed methods on string instances to be completed +* Improved Luau REPL argument parsing and error reporting +* Input history is now saved/loaded + +## Thanks + +A special thanks to all the fine folks who contributed PRs over the last few months! + +* [Halalaluyafail3](https://github.com/Halalaluyafail3) +* [JohnnyMorganz](https://github.com/JohnnyMorganz) +* [Kampfkarren](https://github.com/Kampfkarren) +* [kunitoki](https://github.com/kunitoki) +* [MathematicalDessert](https://github.com/MathematicalDessert) +* [metatablecat](https://github.com/metatablecat) +* [petrihakkinen](https://github.com/petrihakkinen) +* [rafa_br34](https://github.com/rafa_br34) +* [Rerumu](https://github.com/Rerumu) +* [Slappy826](https://github.com/Slappy826) +* [SnowyShiro](https://github.com/SnowyShiro) +* [vladmarica](https://github.com/vladmarica) +* [xgladius](https://github.com/xgladius) + +[Contribution guide](https://github.com/Roblox/luau/blob/2f989fc049772f36de1a4281834c375858507bda/CONTRIBUTING.md) From f6b4cc9442f57db2ef9c186d7fea008dde8b2f68 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Thu, 3 Feb 2022 15:09:37 -0800 Subject: [PATCH 143/399] Sync to upstream/release/513 --- Analysis/include/Luau/Error.h | 12 +- Analysis/include/Luau/LValue.h | 14 +- Analysis/include/Luau/Substitution.h | 6 + Analysis/include/Luau/TxnLog.h | 8 +- Analysis/include/Luau/TypeInfer.h | 9 +- Analysis/include/Luau/TypedAllocator.h | 22 +- Analysis/include/Luau/Unifier.h | 4 + Analysis/src/BuiltinDefinitions.cpp | 41 +- Analysis/src/LValue.cpp | 52 +- Analysis/src/Linter.cpp | 4 +- Analysis/src/Substitution.cpp | 65 +- Analysis/src/TxnLog.cpp | 71 +- Analysis/src/TypeInfer.cpp | 244 +-- Analysis/src/TypeVar.cpp | 8 +- Analysis/src/TypedAllocator.cpp | 1 - Analysis/src/Unifier.cpp | 541 ++--- Ast/src/Parser.cpp | 3 +- CLI/Coverage.cpp | 2 +- CLI/FileUtils.cpp | 2 +- CLI/Repl.cpp | 55 +- CLI/ReplEntry.cpp | 3 +- CMakeLists.txt | 19 +- Compiler/src/Builtins.cpp | 4 +- Compiler/src/Compiler.cpp | 9 +- Compiler/src/TableShape.cpp | 8 - Makefile | 24 +- Sources.cmake | 5 + VM/include/luaconf.h | 4 +- VM/src/lcorolib.cpp | 2 +- VM/src/ldebug.cpp | 3 +- VM/src/lfunc.cpp | 6 +- VM/src/lgc.cpp | 2 + VM/src/lmem.cpp | 53 +- VM/src/lstring.cpp | 2 +- VM/src/lvmexecute.cpp | 11 +- VM/src/lvmload.cpp | 8 +- extern/isocline/.gitignore | 16 + extern/isocline/LICENSE | 21 + extern/isocline/include/isocline.h | 627 ++++++ extern/isocline/readme.md | 460 ++++ extern/isocline/src/attr.c | 294 +++ extern/isocline/src/attr.h | 70 + extern/isocline/src/bbcode.c | 842 +++++++ extern/isocline/src/bbcode.h | 37 + extern/isocline/src/bbcode_colors.c | 194 ++ extern/isocline/src/common.c | 347 +++ extern/isocline/src/common.h | 187 ++ extern/isocline/src/completers.c | 675 ++++++ extern/isocline/src/completions.c | 326 +++ extern/isocline/src/completions.h | 52 + extern/isocline/src/editline.c | 1142 ++++++++++ extern/isocline/src/editline_completion.c | 277 +++ extern/isocline/src/editline_help.c | 140 ++ extern/isocline/src/editline_history.c | 260 +++ extern/isocline/src/env.h | 60 + extern/isocline/src/highlight.c | 259 +++ extern/isocline/src/highlight.h | 24 + extern/isocline/src/history.c | 269 +++ extern/isocline/src/history.h | 38 + extern/isocline/src/isocline.c | 589 +++++ extern/isocline/src/stringbuf.c | 1038 +++++++++ extern/isocline/src/stringbuf.h | 121 ++ extern/isocline/src/term.c | 1124 ++++++++++ extern/isocline/src/term.h | 85 + extern/isocline/src/term_color.c | 371 ++++ extern/isocline/src/tty.c | 889 ++++++++ extern/isocline/src/tty.h | 160 ++ extern/isocline/src/tty_esc.c | 401 ++++ extern/isocline/src/undo.c | 67 + extern/isocline/src/undo.h | 24 + extern/isocline/src/wcwidth.c | 292 +++ extern/linenoise.hpp | 2415 --------------------- tests/Autocomplete.test.cpp | 100 +- tests/Compiler.test.cpp | 20 +- tests/Conformance.test.cpp | 1 - tests/LValue.test.cpp | 60 +- tests/Linter.test.cpp | 25 +- tests/Parser.test.cpp | 7 +- tests/TypeInfer.annotations.test.cpp | 3 - tests/TypeInfer.builtins.test.cpp | 51 + tests/TypeInfer.provisional.test.cpp | 13 - tests/TypeInfer.refinements.test.cpp | 65 +- tests/TypeInfer.tables.test.cpp | 8 +- tests/TypeInfer.test.cpp | 86 +- tests/TypeInfer.tryUnify.test.cpp | 13 + tests/conformance/basic.lua | 4 + tests/conformance/vararg.lua | 50 +- 87 files changed, 12814 insertions(+), 3212 deletions(-) create mode 100644 extern/isocline/.gitignore create mode 100644 extern/isocline/LICENSE create mode 100644 extern/isocline/include/isocline.h create mode 100644 extern/isocline/readme.md create mode 100644 extern/isocline/src/attr.c create mode 100644 extern/isocline/src/attr.h create mode 100644 extern/isocline/src/bbcode.c create mode 100644 extern/isocline/src/bbcode.h create mode 100644 extern/isocline/src/bbcode_colors.c create mode 100644 extern/isocline/src/common.c create mode 100644 extern/isocline/src/common.h create mode 100644 extern/isocline/src/completers.c create mode 100644 extern/isocline/src/completions.c create mode 100644 extern/isocline/src/completions.h create mode 100644 extern/isocline/src/editline.c create mode 100644 extern/isocline/src/editline_completion.c create mode 100644 extern/isocline/src/editline_help.c create mode 100644 extern/isocline/src/editline_history.c create mode 100644 extern/isocline/src/env.h create mode 100644 extern/isocline/src/highlight.c create mode 100644 extern/isocline/src/highlight.h create mode 100644 extern/isocline/src/history.c create mode 100644 extern/isocline/src/history.h create mode 100644 extern/isocline/src/isocline.c create mode 100644 extern/isocline/src/stringbuf.c create mode 100644 extern/isocline/src/stringbuf.h create mode 100644 extern/isocline/src/term.c create mode 100644 extern/isocline/src/term.h create mode 100644 extern/isocline/src/term_color.c create mode 100644 extern/isocline/src/tty.c create mode 100644 extern/isocline/src/tty.h create mode 100644 extern/isocline/src/tty_esc.c create mode 100644 extern/isocline/src/undo.c create mode 100644 extern/isocline/src/undo.h create mode 100644 extern/isocline/src/wcwidth.c delete mode 100644 extern/linenoise.hpp diff --git a/Analysis/include/Luau/Error.h b/Analysis/include/Luau/Error.h index aff3c4d9..a71e0224 100644 --- a/Analysis/include/Luau/Error.h +++ b/Analysis/include/Luau/Error.h @@ -285,12 +285,12 @@ struct TypesAreUnrelated bool operator==(const TypesAreUnrelated& rhs) const; }; -using TypeErrorData = Variant; +using TypeErrorData = + Variant; struct TypeError { diff --git a/Analysis/include/Luau/LValue.h b/Analysis/include/Luau/LValue.h index 8fd96f05..3d510d5f 100644 --- a/Analysis/include/Luau/LValue.h +++ b/Analysis/include/Luau/LValue.h @@ -4,7 +4,6 @@ #include "Luau/Variant.h" #include "Luau/Symbol.h" -#include // TODO: Kill with LuauLValueAsKey. #include #include @@ -38,24 +37,13 @@ std::optional tryGetLValue(const class AstExpr& expr); // Utility function: breaks down an LValue to get at the Symbol, and reverses the vector of keys. std::pair> getFullName(const LValue& lvalue); -// Kill with LuauLValueAsKey. -std::string toString(const LValue& lvalue); - template const T* get(const LValue& lvalue) { return get_if(&lvalue); } -using NEW_RefinementMap = std::unordered_map; -using DEPRECATED_RefinementMap = std::map; - -// Transient. Kill with LuauLValueAsKey. -struct RefinementMap -{ - NEW_RefinementMap NEW_refinements; - DEPRECATED_RefinementMap DEPRECATED_refinements; -}; +using RefinementMap = std::unordered_map; void merge(RefinementMap& l, const RefinementMap& r, std::function f); void addRefinement(RefinementMap& refis, const LValue& lvalue, TypeId ty); diff --git a/Analysis/include/Luau/Substitution.h b/Analysis/include/Luau/Substitution.h index 80a14e8f..4f3307cd 100644 --- a/Analysis/include/Luau/Substitution.h +++ b/Analysis/include/Luau/Substitution.h @@ -55,6 +55,8 @@ namespace Luau { +struct TxnLog; + enum class TarjanResult { TooManyChildren, @@ -89,6 +91,10 @@ struct Tarjan int childCount = 0; + // This should never be null; ensure you initialize it before calling + // substitution methods. + const TxnLog* log; + std::vector edgesTy; std::vector edgesTp; std::vector worklist; diff --git a/Analysis/include/Luau/TxnLog.h b/Analysis/include/Luau/TxnLog.h index dc45bebf..02b87374 100644 --- a/Analysis/include/Luau/TxnLog.h +++ b/Analysis/include/Luau/TxnLog.h @@ -72,6 +72,9 @@ struct PendingType } }; +std::string toString(PendingType* pending); +std::string dump(PendingType* pending); + // Pending state for a TypePackVar. Generated by a TxnLog and committed via // TxnLog::commit. struct PendingTypePack @@ -85,6 +88,9 @@ struct PendingTypePack } }; +std::string toString(PendingTypePack* pending); +std::string dump(PendingTypePack* pending); + template T* getMutable(PendingType* pending) { @@ -237,7 +243,7 @@ struct TxnLog // Follows a type, accounting for pending type states. The returned type may have // pending state; you should use `pending` or `get` to find out. - TypeId follow(TypeId ty); + TypeId follow(TypeId ty) const; // Follows a type pack, accounting for pending type states. The returned type pack // may have pending state; you should use `pending` or `get` to find out. diff --git a/Analysis/include/Luau/TypeInfer.h b/Analysis/include/Luau/TypeInfer.h index b843509d..90dc9f42 100644 --- a/Analysis/include/Luau/TypeInfer.h +++ b/Analysis/include/Luau/TypeInfer.h @@ -262,7 +262,7 @@ public: * {method: ({method: () -> a}) -> a} * */ - TypeId instantiate(const ScopePtr& scope, TypeId ty, Location location); + TypeId instantiate(const ScopePtr& scope, TypeId ty, Location location, const TxnLog* log = TxnLog::empty()); // Replace any free types or type packs by `any`. // This is used when exporting types from modules, to make sure free types don't leak. @@ -308,9 +308,15 @@ private: TypeId singletonType(bool value); TypeId singletonType(std::string value); + TypeIdPredicate mkTruthyPredicate(bool sense); + // Returns nullopt if the predicate filters down the TypeId to 0 options. std::optional filterMap(TypeId type, TypeIdPredicate predicate); +public: + std::optional pickTypesFromSense(TypeId type, bool sense); + +private: TypeId unionOfTypes(TypeId a, TypeId b, const Location& location, bool unifyFreeTypes = true); // ex @@ -349,7 +355,6 @@ private: void refineLValue(const LValue& lvalue, RefinementMap& refis, const ScopePtr& scope, TypeIdPredicate predicate); std::optional resolveLValue(const ScopePtr& scope, const LValue& lvalue); - std::optional DEPRECATED_resolveLValue(const ScopePtr& scope, const LValue& lvalue); std::optional resolveLValue(const RefinementMap& refis, const ScopePtr& scope, const LValue& lvalue); void resolve(const PredicateVec& predicates, ErrorVec& errVec, RefinementMap& refis, const ScopePtr& scope, bool sense, bool fromOr = false); diff --git a/Analysis/include/Luau/TypedAllocator.h b/Analysis/include/Luau/TypedAllocator.h index 64227e7c..c1c04d10 100644 --- a/Analysis/include/Luau/TypedAllocator.h +++ b/Analysis/include/Luau/TypedAllocator.h @@ -6,8 +6,6 @@ #include #include -LUAU_FASTFLAG(LuauTypedAllocatorZeroStart) - namespace Luau { @@ -22,10 +20,7 @@ class TypedAllocator public: TypedAllocator() { - if (FFlag::LuauTypedAllocatorZeroStart) - currentBlockSize = kBlockSize; - else - appendBlock(); + currentBlockSize = kBlockSize; } ~TypedAllocator() @@ -64,18 +59,12 @@ public: bool empty() const { - if (FFlag::LuauTypedAllocatorZeroStart) - return stuff.empty(); - else - return stuff.size() == 1 && currentBlockSize == 0; + return stuff.empty(); } size_t size() const { - if (FFlag::LuauTypedAllocatorZeroStart) - return stuff.empty() ? 0 : kBlockSize * (stuff.size() - 1) + currentBlockSize; - else - return kBlockSize * (stuff.size() - 1) + currentBlockSize; + return stuff.empty() ? 0 : kBlockSize * (stuff.size() - 1) + currentBlockSize; } void clear() @@ -84,10 +73,7 @@ public: unfreeze(); free(); - if (FFlag::LuauTypedAllocatorZeroStart) - currentBlockSize = kBlockSize; - else - appendBlock(); + currentBlockSize = kBlockSize; } void freeze() diff --git a/Analysis/include/Luau/Unifier.h b/Analysis/include/Luau/Unifier.h index 1b1671c0..9db4e22b 100644 --- a/Analysis/include/Luau/Unifier.h +++ b/Analysis/include/Luau/Unifier.h @@ -51,6 +51,10 @@ struct Unifier private: void tryUnify_(TypeId subTy, TypeId superTy, bool isFunctionCall = false, bool isIntersection = false); + void tryUnifyUnionWithType(TypeId subTy, const UnionTypeVar* uv, TypeId superTy); + void tryUnifyTypeWithUnion(TypeId subTy, TypeId superTy, const UnionTypeVar* uv, bool cacheEnabled, bool isFunctionCall); + void tryUnifyTypeWithIntersection(TypeId subTy, TypeId superTy, const IntersectionTypeVar* uv); + void tryUnifyIntersectionWithType(TypeId subTy, const IntersectionTypeVar* uv, TypeId superTy, bool cacheEnabled, bool isFunctionCall); void tryUnifyPrimitives(TypeId subTy, TypeId superTy); void tryUnifySingletons(TypeId subTy, TypeId superTy); void tryUnifyFunctions(TypeId subTy, TypeId superTy, bool isFunctionCall = false); diff --git a/Analysis/src/BuiltinDefinitions.cpp b/Analysis/src/BuiltinDefinitions.cpp index d527414a..d72422a5 100644 --- a/Analysis/src/BuiltinDefinitions.cpp +++ b/Analysis/src/BuiltinDefinitions.cpp @@ -8,6 +8,8 @@ #include +LUAU_FASTFLAG(LuauAssertStripsFalsyTypes) + /** FIXME: Many of these type definitions are not quite completely accurate. * * Some of them require richer generics than we have. For instance, we do not yet have a way to talk @@ -391,12 +393,41 @@ static std::optional> magicFunctionAssert( { auto [paramPack, predicates] = exprResult; - if (expr.args.size < 1) + if (FFlag::LuauAssertStripsFalsyTypes) + { + TypeArena& arena = typechecker.currentModule->internalTypes; + + auto [head, tail] = flatten(paramPack); + if (head.empty() && tail) + { + std::optional fst = first(*tail); + if (!fst) + return ExprResult{paramPack}; + head.push_back(*fst); + } + + typechecker.reportErrors(typechecker.resolve(predicates, scope, true)); + + if (head.size() > 0) + { + std::optional newhead = typechecker.pickTypesFromSense(head[0], true); + if (!newhead) + head = {typechecker.nilType}; + else + head[0] = *newhead; + } + + return ExprResult{arena.addTypePack(TypePack{std::move(head), tail})}; + } + else + { + if (expr.args.size < 1) + return ExprResult{paramPack}; + + typechecker.reportErrors(typechecker.resolve(predicates, scope, true)); + return ExprResult{paramPack}; - - typechecker.reportErrors(typechecker.resolve(predicates, scope, true)); - - return ExprResult{paramPack}; + } } static std::optional> magicFunctionPack( diff --git a/Analysis/src/LValue.cpp b/Analysis/src/LValue.cpp index da6804c6..c9466a40 100644 --- a/Analysis/src/LValue.cpp +++ b/Analysis/src/LValue.cpp @@ -5,8 +5,6 @@ #include -LUAU_FASTFLAG(LuauLValueAsKey) - namespace Luau { @@ -94,17 +92,7 @@ std::pair> getFullName(const LValue& lvalue) return {*symbol, std::vector(keys.rbegin(), keys.rend())}; } -// Kill with LuauLValueAsKey. -std::string toString(const LValue& lvalue) -{ - auto [symbol, keys] = getFullName(lvalue); - std::string s = toString(symbol); - for (std::string key : keys) - s += "." + key; - return s; -} - -static void merge(NEW_RefinementMap& l, const NEW_RefinementMap& r, std::function f) +void merge(RefinementMap& l, const RefinementMap& r, std::function f) { for (const auto& [k, a] : r) { @@ -115,45 +103,9 @@ static void merge(NEW_RefinementMap& l, const NEW_RefinementMap& r, std::functio } } -static void merge(DEPRECATED_RefinementMap& l, const DEPRECATED_RefinementMap& r, std::function f) -{ - auto itL = l.begin(); - auto itR = r.begin(); - while (itL != l.end() && itR != r.end()) - { - const auto& [k, a] = *itR; - if (itL->first == k) - { - l[k] = f(itL->second, a); - ++itL; - ++itR; - } - else if (itL->first < k) - ++itL; - else - { - l[k] = a; - ++itR; - } - } - - l.insert(itR, r.end()); -} - -void merge(RefinementMap& l, const RefinementMap& r, std::function f) -{ - if (FFlag::LuauLValueAsKey) - return merge(l.NEW_refinements, r.NEW_refinements, f); - else - return merge(l.DEPRECATED_refinements, r.DEPRECATED_refinements, f); -} - void addRefinement(RefinementMap& refis, const LValue& lvalue, TypeId ty) { - if (FFlag::LuauLValueAsKey) - refis.NEW_refinements[lvalue] = ty; - else - refis.DEPRECATED_refinements[toString(lvalue)] = ty; + refis[lvalue] = ty; } } // namespace Luau diff --git a/Analysis/src/Linter.cpp b/Analysis/src/Linter.cpp index 905b70bf..57a33e93 100644 --- a/Analysis/src/Linter.cpp +++ b/Analysis/src/Linter.cpp @@ -12,8 +12,6 @@ #include #include -LUAU_FASTFLAGVARIABLE(LuauLintTableCreateTable, false) - namespace Luau { @@ -2155,7 +2153,7 @@ private: "table.move uses index 0 but arrays are 1-based; did you mean 1 instead?"); } - if (FFlag::LuauLintTableCreateTable && func->index == "create" && node->args.size == 2) + if (func->index == "create" && node->args.size == 2) { // table.create(n, {...}) if (args[1]->is()) diff --git a/Analysis/src/Substitution.cpp b/Analysis/src/Substitution.cpp index 3d004bee..bacbca76 100644 --- a/Analysis/src/Substitution.cpp +++ b/Analysis/src/Substitution.cpp @@ -2,6 +2,7 @@ #include "Luau/Substitution.h" #include "Luau/Common.h" +#include "Luau/TxnLog.h" #include #include @@ -13,17 +14,17 @@ namespace Luau void Tarjan::visitChildren(TypeId ty, int index) { - ty = follow(ty); + ty = log->follow(ty); if (ignoreChildren(ty)) return; - if (const FunctionTypeVar* ftv = get(ty)) + if (const FunctionTypeVar* ftv = log->getMutable(ty)) { visitChild(ftv->argTypes); visitChild(ftv->retType); } - else if (const TableTypeVar* ttv = get(ty)) + else if (const TableTypeVar* ttv = log->getMutable(ty)) { LUAU_ASSERT(!ttv->boundTo); for (const auto& [name, prop] : ttv->props) @@ -40,17 +41,17 @@ void Tarjan::visitChildren(TypeId ty, int index) for (TypePackId itp : ttv->instantiatedTypePackParams) visitChild(itp); } - else if (const MetatableTypeVar* mtv = get(ty)) + else if (const MetatableTypeVar* mtv = log->getMutable(ty)) { visitChild(mtv->table); visitChild(mtv->metatable); } - else if (const UnionTypeVar* utv = get(ty)) + else if (const UnionTypeVar* utv = log->getMutable(ty)) { for (TypeId opt : utv->options) visitChild(opt); } - else if (const IntersectionTypeVar* itv = get(ty)) + else if (const IntersectionTypeVar* itv = log->getMutable(ty)) { for (TypeId part : itv->parts) visitChild(part); @@ -59,19 +60,19 @@ void Tarjan::visitChildren(TypeId ty, int index) void Tarjan::visitChildren(TypePackId tp, int index) { - tp = follow(tp); + tp = log->follow(tp); if (ignoreChildren(tp)) return; - if (const TypePack* tpp = get(tp)) + if (const TypePack* tpp = log->getMutable(tp)) { for (TypeId tv : tpp->head) visitChild(tv); if (tpp->tail) visitChild(*tpp->tail); } - else if (const VariadicTypePack* vtp = get(tp)) + else if (const VariadicTypePack* vtp = log->getMutable(tp)) { visitChild(vtp->ty); } @@ -79,7 +80,7 @@ void Tarjan::visitChildren(TypePackId tp, int index) std::pair Tarjan::indexify(TypeId ty) { - ty = follow(ty); + ty = log->follow(ty); bool fresh = !typeToIndex.contains(ty); int& index = typeToIndex[ty]; @@ -97,7 +98,7 @@ std::pair Tarjan::indexify(TypeId ty) std::pair Tarjan::indexify(TypePackId tp) { - tp = follow(tp); + tp = log->follow(tp); bool fresh = !packToIndex.contains(tp); int& index = packToIndex[tp]; @@ -115,7 +116,7 @@ std::pair Tarjan::indexify(TypePackId tp) void Tarjan::visitChild(TypeId ty) { - ty = follow(ty); + ty = log->follow(ty); edgesTy.push_back(ty); edgesTp.push_back(nullptr); @@ -123,7 +124,7 @@ void Tarjan::visitChild(TypeId ty) void Tarjan::visitChild(TypePackId tp) { - tp = follow(tp); + tp = log->follow(tp); edgesTy.push_back(nullptr); edgesTp.push_back(tp); @@ -243,7 +244,7 @@ void Tarjan::clear() TarjanResult Tarjan::visitRoot(TypeId ty) { childCount = 0; - ty = follow(ty); + ty = log->follow(ty); clear(); auto [index, fresh] = indexify(ty); @@ -254,7 +255,7 @@ TarjanResult Tarjan::visitRoot(TypeId ty) TarjanResult Tarjan::visitRoot(TypePackId tp) { childCount = 0; - tp = follow(tp); + tp = log->follow(tp); clear(); auto [index, fresh] = indexify(tp); @@ -325,7 +326,7 @@ TarjanResult FindDirty::findDirty(TypePackId tp) std::optional Substitution::substitute(TypeId ty) { - ty = follow(ty); + ty = log->follow(ty); newTypes.clear(); newPacks.clear(); @@ -345,7 +346,7 @@ std::optional Substitution::substitute(TypeId ty) std::optional Substitution::substitute(TypePackId tp) { - tp = follow(tp); + tp = log->follow(tp); newTypes.clear(); newPacks.clear(); @@ -365,11 +366,11 @@ std::optional Substitution::substitute(TypePackId tp) TypeId Substitution::clone(TypeId ty) { - ty = follow(ty); + ty = log->follow(ty); TypeId result = ty; - if (const FunctionTypeVar* ftv = get(ty)) + if (const FunctionTypeVar* ftv = log->getMutable(ty)) { FunctionTypeVar clone = FunctionTypeVar{ftv->level, ftv->argTypes, ftv->retType, ftv->definition, ftv->hasSelf}; clone.generics = ftv->generics; @@ -379,7 +380,7 @@ TypeId Substitution::clone(TypeId ty) clone.argNames = ftv->argNames; result = addType(std::move(clone)); } - else if (const TableTypeVar* ttv = get(ty)) + else if (const TableTypeVar* ttv = log->getMutable(ty)) { LUAU_ASSERT(!ttv->boundTo); TableTypeVar clone = TableTypeVar{ttv->props, ttv->indexer, ttv->level, ttv->state}; @@ -392,19 +393,19 @@ TypeId Substitution::clone(TypeId ty) clone.tags = ttv->tags; result = addType(std::move(clone)); } - else if (const MetatableTypeVar* mtv = get(ty)) + else if (const MetatableTypeVar* mtv = log->getMutable(ty)) { MetatableTypeVar clone = MetatableTypeVar{mtv->table, mtv->metatable}; clone.syntheticName = mtv->syntheticName; result = addType(std::move(clone)); } - else if (const UnionTypeVar* utv = get(ty)) + else if (const UnionTypeVar* utv = log->getMutable(ty)) { UnionTypeVar clone; clone.options = utv->options; result = addType(std::move(clone)); } - else if (const IntersectionTypeVar* itv = get(ty)) + else if (const IntersectionTypeVar* itv = log->getMutable(ty)) { IntersectionTypeVar clone; clone.parts = itv->parts; @@ -417,15 +418,15 @@ TypeId Substitution::clone(TypeId ty) TypePackId Substitution::clone(TypePackId tp) { - tp = follow(tp); - if (const TypePack* tpp = get(tp)) + tp = log->follow(tp); + if (const TypePack* tpp = log->getMutable(tp)) { TypePack clone; clone.head = tpp->head; clone.tail = tpp->tail; return addTypePack(std::move(clone)); } - else if (const VariadicTypePack* vtp = get(tp)) + else if (const VariadicTypePack* vtp = log->getMutable(tp)) { VariadicTypePack clone; clone.ty = vtp->ty; @@ -437,7 +438,7 @@ TypePackId Substitution::clone(TypePackId tp) void Substitution::foundDirty(TypeId ty) { - ty = follow(ty); + ty = log->follow(ty); if (isDirty(ty)) newTypes[ty] = clean(ty); else @@ -446,7 +447,7 @@ void Substitution::foundDirty(TypeId ty) void Substitution::foundDirty(TypePackId tp) { - tp = follow(tp); + tp = log->follow(tp); if (isDirty(tp)) newPacks[tp] = clean(tp); else @@ -455,7 +456,7 @@ void Substitution::foundDirty(TypePackId tp) TypeId Substitution::replace(TypeId ty) { - ty = follow(ty); + ty = log->follow(ty); if (TypeId* prevTy = newTypes.find(ty)) return *prevTy; else @@ -464,7 +465,7 @@ TypeId Substitution::replace(TypeId ty) TypePackId Substitution::replace(TypePackId tp) { - tp = follow(tp); + tp = log->follow(tp); if (TypePackId* prevTp = newPacks.find(tp)) return *prevTp; else @@ -473,7 +474,7 @@ TypePackId Substitution::replace(TypePackId tp) void Substitution::replaceChildren(TypeId ty) { - ty = follow(ty); + ty = log->follow(ty); if (ignoreChildren(ty)) return; @@ -519,7 +520,7 @@ void Substitution::replaceChildren(TypeId ty) void Substitution::replaceChildren(TypePackId tp) { - tp = follow(tp); + tp = log->follow(tp); if (ignoreChildren(tp)) return; diff --git a/Analysis/src/TxnLog.cpp b/Analysis/src/TxnLog.cpp index a46ac0c3..0968a4c1 100644 --- a/Analysis/src/TxnLog.cpp +++ b/Analysis/src/TxnLog.cpp @@ -1,6 +1,7 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #include "Luau/TxnLog.h" +#include "Luau/ToString.h" #include "Luau/TypePack.h" #include @@ -80,6 +81,56 @@ void DEPRECATED_TxnLog::popSeen(TypeId lhs, TypeId rhs) sharedSeen->pop_back(); } +const std::string nullPendingResult = ""; + +std::string toString(PendingType* pending) +{ + if (pending == nullptr) + return nullPendingResult; + + return toString(pending->pending); +} + +std::string dump(PendingType* pending) +{ + if (pending == nullptr) + { + printf("%s\n", nullPendingResult.c_str()); + return nullPendingResult; + } + + ToStringOptions opts; + opts.exhaustive = true; + opts.functionTypeArguments = true; + std::string result = toString(pending->pending, opts); + printf("%s\n", result.c_str()); + return result; +} + +std::string toString(PendingTypePack* pending) +{ + if (pending == nullptr) + return nullPendingResult; + + return toString(pending->pending); +} + +std::string dump(PendingTypePack* pending) +{ + if (pending == nullptr) + { + printf("%s\n", nullPendingResult.c_str()); + return nullPendingResult; + } + + ToStringOptions opts; + opts.exhaustive = true; + opts.functionTypeArguments = true; + std::string result = toString(pending->pending, opts); + printf("%s\n", result.c_str()); + return result; +} + static const TxnLog emptyLog; const TxnLog* TxnLog::empty() @@ -199,8 +250,6 @@ PendingTypePack* TxnLog::queue(TypePackId tp) PendingType* TxnLog::pending(TypeId ty) const { - LUAU_ASSERT(FFlag::LuauUseCommittingTxnLog); - for (const TxnLog* current = this; current; current = current->parent) { if (auto it = current->typeVarChanges.find(ty); it != current->typeVarChanges.end()) @@ -212,8 +261,6 @@ PendingType* TxnLog::pending(TypeId ty) const PendingTypePack* TxnLog::pending(TypePackId tp) const { - LUAU_ASSERT(FFlag::LuauUseCommittingTxnLog); - for (const TxnLog* current = this; current; current = current->parent) { if (auto it = current->typePackChanges.find(tp); it != current->typePackChanges.end()) @@ -225,8 +272,6 @@ PendingTypePack* TxnLog::pending(TypePackId tp) const PendingType* TxnLog::replace(TypeId ty, TypeVar replacement) { - LUAU_ASSERT(FFlag::LuauUseCommittingTxnLog); - PendingType* newTy = queue(ty); newTy->pending = replacement; return newTy; @@ -234,8 +279,6 @@ PendingType* TxnLog::replace(TypeId ty, TypeVar replacement) PendingTypePack* TxnLog::replace(TypePackId tp, TypePackVar replacement) { - LUAU_ASSERT(FFlag::LuauUseCommittingTxnLog); - PendingTypePack* newTp = queue(tp); newTp->pending = replacement; return newTp; @@ -243,7 +286,6 @@ PendingTypePack* TxnLog::replace(TypePackId tp, TypePackVar replacement) PendingType* TxnLog::bindTable(TypeId ty, std::optional newBoundTo) { - LUAU_ASSERT(FFlag::LuauUseCommittingTxnLog); LUAU_ASSERT(get(ty)); PendingType* newTy = queue(ty); @@ -255,7 +297,6 @@ PendingType* TxnLog::bindTable(TypeId ty, std::optional newBoundTo) PendingType* TxnLog::changeLevel(TypeId ty, TypeLevel newLevel) { - LUAU_ASSERT(FFlag::LuauUseCommittingTxnLog); LUAU_ASSERT(get(ty) || get(ty) || get(ty)); PendingType* newTy = queue(ty); @@ -278,7 +319,6 @@ PendingType* TxnLog::changeLevel(TypeId ty, TypeLevel newLevel) PendingTypePack* TxnLog::changeLevel(TypePackId tp, TypeLevel newLevel) { - LUAU_ASSERT(FFlag::LuauUseCommittingTxnLog); LUAU_ASSERT(get(tp)); PendingTypePack* newTp = queue(tp); @@ -292,7 +332,6 @@ PendingTypePack* TxnLog::changeLevel(TypePackId tp, TypeLevel newLevel) PendingType* TxnLog::changeIndexer(TypeId ty, std::optional indexer) { - LUAU_ASSERT(FFlag::LuauUseCommittingTxnLog); LUAU_ASSERT(get(ty)); PendingType* newTy = queue(ty); @@ -306,8 +345,6 @@ PendingType* TxnLog::changeIndexer(TypeId ty, std::optional indexe std::optional TxnLog::getLevel(TypeId ty) const { - LUAU_ASSERT(FFlag::LuauUseCommittingTxnLog); - if (FreeTypeVar* ftv = getMutable(ty)) return ftv->level; else if (TableTypeVar* ttv = getMutable(ty); ttv && (ttv->state == TableState::Free || ttv->state == TableState::Generic)) @@ -318,10 +355,8 @@ std::optional TxnLog::getLevel(TypeId ty) const return std::nullopt; } -TypeId TxnLog::follow(TypeId ty) +TypeId TxnLog::follow(TypeId ty) const { - LUAU_ASSERT(FFlag::LuauUseCommittingTxnLog); - return Luau::follow(ty, [this](TypeId ty) { PendingType* state = this->pending(ty); @@ -337,8 +372,6 @@ TypeId TxnLog::follow(TypeId ty) TypePackId TxnLog::follow(TypePackId tp) const { - LUAU_ASSERT(FFlag::LuauUseCommittingTxnLog); - return Luau::follow(tp, [this](TypePackId tp) { PendingTypePack* state = this->pending(tp); diff --git a/Analysis/src/TypeInfer.cpp b/Analysis/src/TypeInfer.cpp index 23fcc2d5..4d25fe2e 100644 --- a/Analysis/src/TypeInfer.cpp +++ b/Analysis/src/TypeInfer.cpp @@ -32,6 +32,7 @@ LUAU_FASTFLAGVARIABLE(LuauRecursiveTypeParameterRestriction, false) LUAU_FASTFLAGVARIABLE(LuauIfElseBranchTypeUnion, false) LUAU_FASTFLAGVARIABLE(LuauIfElseExpectedType2, false) LUAU_FASTFLAGVARIABLE(LuauLengthOnCompositeType, false) +LUAU_FASTFLAGVARIABLE(LuauNoSealedTypeMod, false) LUAU_FASTFLAGVARIABLE(LuauQuantifyInPlace2, false) LUAU_FASTFLAGVARIABLE(LuauSealExports, false) LUAU_FASTFLAGVARIABLE(LuauSingletonTypes, false) @@ -40,13 +41,12 @@ LUAU_FASTFLAGVARIABLE(LuauTypeAliasDefaults, false) LUAU_FASTFLAGVARIABLE(LuauExpectedTypesOfProperties, false) LUAU_FASTFLAGVARIABLE(LuauErrorRecoveryType, false) LUAU_FASTFLAGVARIABLE(LuauPropertiesGetExpectedType, false) -LUAU_FASTFLAGVARIABLE(LuauLValueAsKey, false) -LUAU_FASTFLAGVARIABLE(LuauRefiLookupFromIndexExpr, false) LUAU_FASTFLAGVARIABLE(LuauPerModuleUnificationCache, false) LUAU_FASTFLAGVARIABLE(LuauProperTypeLevels, false) LUAU_FASTFLAGVARIABLE(LuauAscribeCorrectLevelToInferredProperitesOfFreeTables, false) -LUAU_FASTFLAGVARIABLE(LuauBidirectionalAsExpr, false) +LUAU_FASTFLAG(LuauUnionTagMatchFix) LUAU_FASTFLAGVARIABLE(LuauUnsealedTableLiteral, false) +LUAU_FASTFLAGVARIABLE(LuauAssertStripsFalsyTypes, false) namespace Luau { @@ -1117,7 +1117,7 @@ void TypeChecker::check(const ScopePtr& scope, TypeId ty, const ScopePtr& funSco ty = follow(ty); - if (tableSelf && !selfTy->persistent) + if (tableSelf && (FFlag::LuauNoSealedTypeMod ? tableSelf->state != TableState::Sealed : !selfTy->persistent)) tableSelf->props[indexName->index.value] = {ty, /* deprecated */ false, {}, indexName->indexLocation}; const FunctionTypeVar* funTy = get(ty); @@ -1130,7 +1130,7 @@ void TypeChecker::check(const ScopePtr& scope, TypeId ty, const ScopePtr& funSco checkFunctionBody(funScope, ty, *function.func); - if (tableSelf && !selfTy->persistent) + if (tableSelf && (FFlag::LuauNoSealedTypeMod ? tableSelf->state != TableState::Sealed : !selfTy->persistent)) tableSelf->props[indexName->index.value] = { follow(quantify(funScope, ty, indexName->indexLocation)), /* deprecated */ false, {}, indexName->indexLocation}; } @@ -1657,7 +1657,7 @@ std::optional TypeChecker::getIndexTypeFromType( RecursionLimiter _rl(&recursionCount, FInt::LuauTypeInferRecursionLimit); // Not needed when we normalize types. - if (FFlag::LuauLValueAsKey && get(follow(t))) + if (get(follow(t))) return t; if (std::optional ty = getIndexTypeFromType(scope, t, name, location, false)) @@ -1802,12 +1802,9 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprIn { TypeId ty = checkLValue(scope, expr); - if (FFlag::LuauRefiLookupFromIndexExpr) - { - if (std::optional lvalue = tryGetLValue(expr)) - if (std::optional refiTy = resolveLValue(scope, *lvalue)) - return {*refiTy, {TruthyPredicate{std::move(*lvalue), expr.location}}}; - } + if (std::optional lvalue = tryGetLValue(expr)) + if (std::optional refiTy = resolveLValue(scope, *lvalue)) + return {*refiTy, {TruthyPredicate{std::move(*lvalue), expr.location}}}; return {ty}; } @@ -2471,33 +2468,28 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprBi { if (expr.op == AstExprBinary::And) { - ExprResult lhs = checkExpr(scope, *expr.left); + auto [lhsTy, lhsPredicates] = checkExpr(scope, *expr.left); - // We can't just report errors here. - // This function can be called from AstStatLocal or from AstStatIf, or even from AstExprBinary (and others). - // For now, ignore the errors returned by the predicate resolver. - // We may need an extra property for each predicate set that indicates it has been resolved. - // Requires a slight modification to the data structure. ScopePtr innerScope = childScope(scope, expr.location); - resolve(lhs.predicates, innerScope, true); + resolve(lhsPredicates, innerScope, true); - ExprResult rhs = checkExpr(innerScope, *expr.right); + auto [rhsTy, rhsPredicates] = checkExpr(innerScope, *expr.right); - return {checkBinaryOperation(FFlag::LuauDiscriminableUnions ? scope : innerScope, expr, lhs.type, rhs.type), - {AndPredicate{std::move(lhs.predicates), std::move(rhs.predicates)}}}; + return {checkBinaryOperation(FFlag::LuauDiscriminableUnions ? scope : innerScope, expr, lhsTy, rhsTy), + {AndPredicate{std::move(lhsPredicates), std::move(rhsPredicates)}}}; } else if (expr.op == AstExprBinary::Or) { - ExprResult lhs = checkExpr(scope, *expr.left); + auto [lhsTy, lhsPredicates] = checkExpr(scope, *expr.left); ScopePtr innerScope = childScope(scope, expr.location); - resolve(lhs.predicates, innerScope, false); + resolve(lhsPredicates, innerScope, false); - ExprResult rhs = checkExpr(innerScope, *expr.right); + auto [rhsTy, rhsPredicates] = checkExpr(innerScope, *expr.right); - // Because of C++, I'm not sure if lhs.predicates was not moved out by the time we call checkBinaryOperation. - TypeId result = checkBinaryOperation(FFlag::LuauDiscriminableUnions ? scope : innerScope, expr, lhs.type, rhs.type, lhs.predicates); - return {result, {OrPredicate{std::move(lhs.predicates), std::move(rhs.predicates)}}}; + // Because of C++, I'm not sure if lhsPredicates was not moved out by the time we call checkBinaryOperation. + TypeId result = checkBinaryOperation(FFlag::LuauDiscriminableUnions ? scope : innerScope, expr, lhsTy, rhsTy, lhsPredicates); + return {result, {OrPredicate{std::move(lhsPredicates), std::move(rhsPredicates)}}}; } else if (expr.op == AstExprBinary::CompareEq || expr.op == AstExprBinary::CompareNe) { @@ -2535,27 +2527,15 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprTy TypeId annotationType = resolveType(scope, *expr.annotation); ExprResult result = checkExpr(scope, *expr.expr, annotationType); - if (FFlag::LuauBidirectionalAsExpr) - { - // Note: As an optimization, we try 'number <: number | string' first, as that is the more likely case. - if (canUnify(annotationType, result.type, expr.location).empty()) - return {annotationType, std::move(result.predicates)}; - - if (canUnify(result.type, annotationType, expr.location).empty()) - return {annotationType, std::move(result.predicates)}; - - reportError(expr.location, TypesAreUnrelated{result.type, annotationType}); - return {errorRecoveryType(annotationType), std::move(result.predicates)}; - } - else - { - ErrorVec errorVec = canUnify(annotationType, result.type, expr.location); - reportErrors(errorVec); - if (!errorVec.empty()) - annotationType = errorRecoveryType(annotationType); - + // Note: As an optimization, we try 'number <: number | string' first, as that is the more likely case. + if (canUnify(annotationType, result.type, expr.location).empty()) return {annotationType, std::move(result.predicates)}; - } + + if (canUnify(result.type, annotationType, expr.location).empty()) + return {annotationType, std::move(result.predicates)}; + + reportError(expr.location, TypesAreUnrelated{result.type, annotationType}); + return {errorRecoveryType(annotationType), std::move(result.predicates)}; } ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprError& expr) @@ -4295,7 +4275,7 @@ void TypeChecker::unifyWithInstantiationIfNeeded(const ScopePtr& scope, TypeId s child.tryUnify(subTy, superTy, /*isFunctionCall*/ false); if (!child.errors.empty()) { - TypeId instantiated = instantiate(scope, subTy, state.location); + TypeId instantiated = instantiate(scope, subTy, state.location, &child.log); if (subTy == instantiated) { // Instantiating the argument made no difference, so just report any child errors @@ -4330,7 +4310,7 @@ void TypeChecker::unifyWithInstantiationIfNeeded(const ScopePtr& scope, TypeId s bool Instantiation::isDirty(TypeId ty) { - if (get(ty)) + if (log->getMutable(ty)) return true; else return false; @@ -4343,7 +4323,7 @@ bool Instantiation::isDirty(TypePackId tp) bool Instantiation::ignoreChildren(TypeId ty) { - if (get(ty)) + if (log->getMutable(ty)) return true; else return false; @@ -4351,7 +4331,7 @@ bool Instantiation::ignoreChildren(TypeId ty) TypeId Instantiation::clean(TypeId ty) { - const FunctionTypeVar* ftv = get(ty); + const FunctionTypeVar* ftv = log->getMutable(ty); LUAU_ASSERT(ftv); FunctionTypeVar clone = FunctionTypeVar{level, ftv->argTypes, ftv->retType, ftv->definition, ftv->hasSelf}; @@ -4362,6 +4342,7 @@ TypeId Instantiation::clean(TypeId ty) // Annoyingly, we have to do this even if there are no generics, // to replace any generic tables. + replaceGenerics.log = log; replaceGenerics.level = level; replaceGenerics.currentModule = currentModule; replaceGenerics.generics.assign(ftv->generics.begin(), ftv->generics.end()); @@ -4383,7 +4364,7 @@ TypePackId Instantiation::clean(TypePackId tp) bool ReplaceGenerics::ignoreChildren(TypeId ty) { - if (const FunctionTypeVar* ftv = get(ty)) + if (const FunctionTypeVar* ftv = log->getMutable(ty)) // We aren't recursing in the case of a generic function which // binds the same generics. This can happen if, for example, there's recursive types. // If T = (a,T)->T then instantiating T should produce T' = (X,T)->T not T' = (X,T')->T'. @@ -4396,9 +4377,9 @@ bool ReplaceGenerics::ignoreChildren(TypeId ty) bool ReplaceGenerics::isDirty(TypeId ty) { - if (const TableTypeVar* ttv = get(ty)) + if (const TableTypeVar* ttv = log->getMutable(ty)) return ttv->state == TableState::Generic; - else if (get(ty)) + else if (log->getMutable(ty)) return std::find(generics.begin(), generics.end(), ty) != generics.end(); else return false; @@ -4406,7 +4387,7 @@ bool ReplaceGenerics::isDirty(TypeId ty) bool ReplaceGenerics::isDirty(TypePackId tp) { - if (get(tp)) + if (log->getMutable(tp)) return std::find(genericPacks.begin(), genericPacks.end(), tp) != genericPacks.end(); else return false; @@ -4415,7 +4396,7 @@ bool ReplaceGenerics::isDirty(TypePackId tp) TypeId ReplaceGenerics::clean(TypeId ty) { LUAU_ASSERT(isDirty(ty)); - if (const TableTypeVar* ttv = get(ty)) + if (const TableTypeVar* ttv = log->getMutable(ty)) { TableTypeVar clone = TableTypeVar{ttv->props, ttv->indexer, level, TableState::Free}; clone.methodDefinitionLocations = ttv->methodDefinitionLocations; @@ -4434,9 +4415,9 @@ TypePackId ReplaceGenerics::clean(TypePackId tp) bool Quantification::isDirty(TypeId ty) { - if (const TableTypeVar* ttv = get(ty)) + if (const TableTypeVar* ttv = log->getMutable(ty)) return level.subsumes(ttv->level) && ((ttv->state == TableState::Free) || (ttv->state == TableState::Unsealed)); - else if (const FreeTypeVar* ftv = get(ty)) + else if (const FreeTypeVar* ftv = log->getMutable(ty)) return level.subsumes(ftv->level); else return false; @@ -4444,7 +4425,7 @@ bool Quantification::isDirty(TypeId ty) bool Quantification::isDirty(TypePackId tp) { - if (const FreeTypePack* ftv = get(tp)) + if (const FreeTypePack* ftv = log->getMutable(tp)) return level.subsumes(ftv->level); else return false; @@ -4453,7 +4434,7 @@ bool Quantification::isDirty(TypePackId tp) TypeId Quantification::clean(TypeId ty) { LUAU_ASSERT(isDirty(ty)); - if (const TableTypeVar* ttv = get(ty)) + if (const TableTypeVar* ttv = log->getMutable(ty)) { TableState state = (ttv->state == TableState::Unsealed ? TableState::Sealed : TableState::Generic); TableTypeVar clone = TableTypeVar{ttv->props, ttv->indexer, level, state}; @@ -4479,9 +4460,9 @@ TypePackId Quantification::clean(TypePackId tp) bool Anyification::isDirty(TypeId ty) { - if (const TableTypeVar* ttv = get(ty)) + if (const TableTypeVar* ttv = log->getMutable(ty)) return (ttv->state == TableState::Free || (FFlag::LuauSealExports && ttv->state == TableState::Unsealed)); - else if (get(ty)) + else if (log->getMutable(ty)) return true; else return false; @@ -4489,7 +4470,7 @@ bool Anyification::isDirty(TypeId ty) bool Anyification::isDirty(TypePackId tp) { - if (get(tp)) + if (log->getMutable(tp)) return true; else return false; @@ -4498,7 +4479,7 @@ bool Anyification::isDirty(TypePackId tp) TypeId Anyification::clean(TypeId ty) { LUAU_ASSERT(isDirty(ty)); - if (const TableTypeVar* ttv = get(ty)) + if (const TableTypeVar* ttv = log->getMutable(ty)) { TableTypeVar clone = TableTypeVar{ttv->props, ttv->indexer, ttv->level, TableState::Sealed}; clone.methodDefinitionLocations = ttv->methodDefinitionLocations; @@ -4535,6 +4516,7 @@ TypeId TypeChecker::quantify(const ScopePtr& scope, TypeId ty, Location location return ty; } + quantification.log = TxnLog::empty(); quantification.level = scope->level; quantification.generics.clear(); quantification.genericPacks.clear(); @@ -4558,8 +4540,11 @@ TypeId TypeChecker::quantify(const ScopePtr& scope, TypeId ty, Location location return *qty; } -TypeId TypeChecker::instantiate(const ScopePtr& scope, TypeId ty, Location location) +TypeId TypeChecker::instantiate(const ScopePtr& scope, TypeId ty, Location location, const TxnLog* log) { + LUAU_ASSERT(log != nullptr); + + instantiation.log = FFlag::LuauUseCommittingTxnLog ? log : TxnLog::empty(); instantiation.level = scope->level; instantiation.currentModule = currentModule; std::optional instantiated = instantiation.substitute(ty); @@ -4574,6 +4559,7 @@ TypeId TypeChecker::instantiate(const ScopePtr& scope, TypeId ty, Location locat TypeId TypeChecker::anyify(const ScopePtr& scope, TypeId ty, Location location) { + anyification.log = TxnLog::empty(); anyification.anyType = anyType; anyification.anyTypePack = anyTypePack; anyification.currentModule = currentModule; @@ -4589,6 +4575,7 @@ TypeId TypeChecker::anyify(const ScopePtr& scope, TypeId ty, Location location) TypePackId TypeChecker::anyify(const ScopePtr& scope, TypePackId ty, Location location) { + anyification.log = TxnLog::empty(); anyification.anyType = anyType; anyification.anyTypePack = anyTypePack; anyification.currentModule = currentModule; @@ -4660,7 +4647,7 @@ void TypeChecker::diagnoseMissingTableKey(UnknownProperty* utk, TypeErrorData& d } }; - if (auto ttv = getTableType(follow(utk->table))) + if (auto ttv = getTableType(FFlag::LuauUnionTagMatchFix ? utk->table : follow(utk->table))) accumulate(ttv->props); else if (auto ctv = get(follow(utk->table))) { @@ -4775,6 +4762,29 @@ TypePackId TypeChecker::errorRecoveryTypePack(TypePackId guess) return getSingletonTypes().errorRecoveryTypePack(guess); } +TypeIdPredicate TypeChecker::mkTruthyPredicate(bool sense) { + return [this, sense](TypeId ty) -> std::optional { + // any/error/free gets a special pass unconditionally because they can't be decided. + if (get(ty) || get(ty) || get(ty)) + return ty; + + // maps boolean primitive to the corresponding singleton equal to sense + if (isPrim(ty, PrimitiveTypeVar::Boolean)) + return singletonType(sense); + + // if we have boolean singleton, eliminate it if the sense doesn't match with that singleton + if (auto boolean = get(get(ty))) + return boolean->value == sense ? std::optional(ty) : std::nullopt; + + // if we have nil, eliminate it if sense is true, otherwise take it + if (isNil(ty)) + return sense ? std::nullopt : std::optional(ty); + + // at this point, anything else is kept if sense is true, or eliminated otherwise + return sense ? std::optional(ty) : std::nullopt; + }; +} + std::optional TypeChecker::filterMap(TypeId type, TypeIdPredicate predicate) { std::vector types = Luau::filterMap(type, predicate); @@ -4783,6 +4793,11 @@ std::optional TypeChecker::filterMap(TypeId type, TypeIdPredicate predic return std::nullopt; } +std::optional TypeChecker::pickTypesFromSense(TypeId type, bool sense) +{ + return filterMap(type, mkTruthyPredicate(sense)); +} + TypeId TypeChecker::addTV(TypeVar&& tv) { return currentModule->internalTypes.addType(std::move(tv)); @@ -5293,6 +5308,7 @@ TypeId TypeChecker::instantiateTypeFun(const ScopePtr& scope, const TypeFun& tf, for (size_t i = 0; i < tf.typePackParams.size(); ++i) applyTypeFunction.typePackArguments[tf.typePackParams[i].tp] = typePackParams[i]; + applyTypeFunction.log = TxnLog::empty(); applyTypeFunction.currentModule = currentModule; applyTypeFunction.level = scope->level; applyTypeFunction.encounteredForwardedType = false; @@ -5507,9 +5523,6 @@ void TypeChecker::refineLValue(const LValue& lvalue, RefinementMap& refis, const std::optional TypeChecker::resolveLValue(const ScopePtr& scope, const LValue& lvalue) { - if (!FFlag::LuauLValueAsKey) - return DEPRECATED_resolveLValue(scope, lvalue); - // We want to be walking the Scope parents. // We'll also want to walk up the LValue path. As we do this, we need to save each LValue because we must walk back. // For example: @@ -5529,7 +5542,7 @@ std::optional TypeChecker::resolveLValue(const ScopePtr& scope, const LV const LValue* currentLValue = &lvalue; while (currentLValue) { - if (auto it = currentScope->refinements.NEW_refinements.find(*currentLValue); it != currentScope->refinements.NEW_refinements.end()) + if (auto it = currentScope->refinements.find(*currentLValue); it != currentScope->refinements.end()) { found = it->second; break; @@ -5576,43 +5589,9 @@ std::optional TypeChecker::resolveLValue(const ScopePtr& scope, const LV return std::nullopt; } -std::optional TypeChecker::DEPRECATED_resolveLValue(const ScopePtr& scope, const LValue& lvalue) -{ - auto [symbol, keys] = getFullName(lvalue); - - ScopePtr currentScope = scope; - while (currentScope) - { - if (auto it = currentScope->refinements.DEPRECATED_refinements.find(toString(lvalue)); it != currentScope->refinements.DEPRECATED_refinements.end()) - return it->second; - - // Should not be using scope->lookup. This is already recursive. - if (auto it = currentScope->bindings.find(symbol); it != currentScope->bindings.end()) - { - std::optional currentTy = it->second.typeId; - - for (std::string key : keys) - { - // TODO: This function probably doesn't need Location at all, or at least should hide the argument. - currentTy = getIndexTypeFromType(scope, *currentTy, key, Location(), false); - if (!currentTy) - break; - } - - return currentTy; - } - - currentScope = currentScope->parent; - } - - return std::nullopt; -} - std::optional TypeChecker::resolveLValue(const RefinementMap& refis, const ScopePtr& scope, const LValue& lvalue) { - if (auto it = refis.DEPRECATED_refinements.find(toString(lvalue)); it != refis.DEPRECATED_refinements.end()) - return it->second; - else if (auto it = refis.NEW_refinements.find(lvalue); it != refis.NEW_refinements.end()) + if (auto it = refis.find(lvalue); it != refis.end()) return it->second; else return resolveLValue(scope, lvalue); @@ -5661,35 +5640,46 @@ void TypeChecker::resolve(const Predicate& predicate, ErrorVec& errVec, Refineme void TypeChecker::resolve(const TruthyPredicate& truthyP, ErrorVec& errVec, RefinementMap& refis, const ScopePtr& scope, bool sense, bool fromOr) { - auto predicate = [sense](TypeId option) -> std::optional { - if (isUndecidable(option) || isBoolean(option) || isNil(option) != sense) - return option; - - return std::nullopt; - }; - - if (FFlag::LuauDiscriminableUnions) + if (FFlag::LuauAssertStripsFalsyTypes) { std::optional ty = resolveLValue(refis, scope, truthyP.lvalue); if (ty && fromOr) return addRefinement(refis, truthyP.lvalue, *ty); - refineLValue(truthyP.lvalue, refis, scope, predicate); + refineLValue(truthyP.lvalue, refis, scope, mkTruthyPredicate(sense)); } else { - std::optional ty = resolveLValue(refis, scope, truthyP.lvalue); - if (!ty) - return; + auto predicate = [sense](TypeId option) -> std::optional { + if (isUndecidable(option) || isBoolean(option) || isNil(option) != sense) + return option; - // This is a hack. :( - // Without this, the expression 'a or b' might refine 'b' to be falsy. - // I'm not yet sure how else to get this to do the right thing without this hack, so we'll do this for now in the meantime. - if (fromOr) - return addRefinement(refis, truthyP.lvalue, *ty); + return std::nullopt; + }; - if (std::optional result = filterMap(*ty, predicate)) - addRefinement(refis, truthyP.lvalue, *result); + if (FFlag::LuauDiscriminableUnions) + { + std::optional ty = resolveLValue(refis, scope, truthyP.lvalue); + if (ty && fromOr) + return addRefinement(refis, truthyP.lvalue, *ty); + + refineLValue(truthyP.lvalue, refis, scope, predicate); + } + else + { + std::optional ty = resolveLValue(refis, scope, truthyP.lvalue); + if (!ty) + return; + + // This is a hack. :( + // Without this, the expression 'a or b' might refine 'b' to be falsy. + // I'm not yet sure how else to get this to do the right thing without this hack, so we'll do this for now in the meantime. + if (fromOr) + return addRefinement(refis, truthyP.lvalue, *ty); + + if (std::optional result = filterMap(*ty, predicate)) + addRefinement(refis, truthyP.lvalue, *result); + } } } @@ -5929,7 +5919,9 @@ void TypeChecker::resolve(const EqPredicate& eqP, ErrorVec& errVec, RefinementMa if (!sense || canUnify(eqP.type, option, eqP.location).empty()) return sense ? eqP.type : option; - return std::nullopt; + // local variable works around an odd gcc 9.3 warning: may be used uninitialized + std::optional res = std::nullopt; + return res; } return option; diff --git a/Analysis/src/TypeVar.cpp b/Analysis/src/TypeVar.cpp index 5b162b31..2321eafd 100644 --- a/Analysis/src/TypeVar.cpp +++ b/Analysis/src/TypeVar.cpp @@ -27,6 +27,7 @@ LUAU_FASTFLAG(LuauLengthOnCompositeType) LUAU_FASTFLAGVARIABLE(LuauMetatableAreEqualRecursion, false) LUAU_FASTFLAGVARIABLE(LuauRefactorTypeVarQuestions, false) LUAU_FASTFLAG(LuauErrorRecoveryType) +LUAU_FASTFLAG(LuauUnionTagMatchFix) namespace Luau { @@ -288,10 +289,13 @@ std::optional getMetatable(TypeId type) const TableTypeVar* getTableType(TypeId type) { + if (FFlag::LuauUnionTagMatchFix) + type = follow(type); + if (const TableTypeVar* ttv = get(type)) return ttv; else if (const MetatableTypeVar* mtv = get(type)) - return get(mtv->table); + return get(FFlag::LuauUnionTagMatchFix ? follow(mtv->table) : mtv->table); else return nullptr; } @@ -308,7 +312,7 @@ const std::string* getName(TypeId type) { if (mtv->syntheticName) return &*mtv->syntheticName; - type = mtv->table; + type = FFlag::LuauUnionTagMatchFix ? follow(mtv->table) : mtv->table; } if (auto ttv = get(type)) diff --git a/Analysis/src/TypedAllocator.cpp b/Analysis/src/TypedAllocator.cpp index 1f7ef8c2..f037351e 100644 --- a/Analysis/src/TypedAllocator.cpp +++ b/Analysis/src/TypedAllocator.cpp @@ -20,7 +20,6 @@ const size_t kPageSize = sysconf(_SC_PAGESIZE); #include LUAU_FASTFLAG(DebugLuauFreezeArena) -LUAU_FASTFLAGVARIABLE(LuauTypedAllocatorZeroStart, false) namespace Luau { diff --git a/Analysis/src/Unifier.cpp b/Analysis/src/Unifier.cpp index 17d9bf58..89e4ae23 100644 --- a/Analysis/src/Unifier.cpp +++ b/Analysis/src/Unifier.cpp @@ -8,6 +8,7 @@ #include "Luau/TypeUtils.h" #include "Luau/TimeTrace.h" #include "Luau/VisitTypeVar.h" +#include "Luau/ToString.h" #include @@ -22,6 +23,7 @@ LUAU_FASTFLAG(LuauSingletonTypes) LUAU_FASTFLAG(LuauErrorRecoveryType); LUAU_FASTFLAG(LuauProperTypeLevels); LUAU_FASTFLAGVARIABLE(LuauUnifyPackTails, false) +LUAU_FASTFLAGVARIABLE(LuauUnionTagMatchFix, false) namespace Luau { @@ -225,19 +227,33 @@ static std::optional hasUnificationTooComplex(const ErrorVec& errors) // Used for tagged union matching heuristic, returns first singleton type field static std::optional> getTableMatchTag(TypeId type) { - type = follow(type); - - if (auto ttv = get(type)) + if (FFlag::LuauUnionTagMatchFix) { - for (auto&& [name, prop] : ttv->props) + if (auto ttv = getTableType(type)) { - if (auto sing = get(follow(prop.type))) - return {{name, sing}}; + for (auto&& [name, prop] : ttv->props) + { + if (auto sing = get(follow(prop.type))) + return {{name, sing}}; + } } } - else if (auto mttv = get(type)) + else { - return getTableMatchTag(mttv->table); + type = follow(type); + + if (auto ttv = get(type)) + { + for (auto&& [name, prop] : ttv->props) + { + if (auto sing = get(follow(prop.type))) + return {{name, sing}}; + } + } + else if (auto mttv = get(type)) + { + return getTableMatchTag(mttv->table); + } } return std::nullopt; @@ -508,245 +524,21 @@ void Unifier::tryUnify_(TypeId subTy, TypeId superTy, bool isFunctionCall, bool if (const UnionTypeVar* uv = FFlag::LuauUseCommittingTxnLog ? log.getMutable(subTy) : get(subTy)) { - // A | B <: T if A <: T and B <: T - bool failed = false; - std::optional unificationTooComplex; - std::optional firstFailedOption; - - size_t count = uv->options.size(); - size_t i = 0; - - for (TypeId type : uv->options) - { - Unifier innerState = makeChildUnifier(); - innerState.tryUnify_(type, superTy); - - if (auto e = hasUnificationTooComplex(innerState.errors)) - unificationTooComplex = e; - else if (!innerState.errors.empty()) - { - // 'nil' option is skipped from extended report because we present the type in a special way - 'T?' - if (!firstFailedOption && !isNil(type)) - firstFailedOption = {innerState.errors.front()}; - - failed = true; - } - - if (FFlag::LuauUseCommittingTxnLog) - { - if (i == count - 1) - { - log.concat(std::move(innerState.log)); - } - } - else - { - if (i != count - 1) - { - innerState.DEPRECATED_log.rollback(); - } - else - { - DEPRECATED_log.concat(std::move(innerState.DEPRECATED_log)); - } - } - - ++i; - } - - if (unificationTooComplex) - reportError(*unificationTooComplex); - else if (failed) - { - if (firstFailedOption) - reportError(TypeError{location, TypeMismatch{superTy, subTy, "Not all union options are compatible.", *firstFailedOption}}); - else - reportError(TypeError{location, TypeMismatch{superTy, subTy}}); - } + tryUnifyUnionWithType(subTy, uv, superTy); } else if (const UnionTypeVar* uv = FFlag::LuauUseCommittingTxnLog ? log.getMutable(superTy) : get(superTy)) { - // T <: A | B if T <: A or T <: B - bool found = false; - std::optional unificationTooComplex; - - size_t failedOptionCount = 0; - std::optional failedOption; - - bool foundHeuristic = false; - size_t startIndex = 0; - - if (const std::string* subName = getName(subTy)) - { - for (size_t i = 0; i < uv->options.size(); ++i) - { - const std::string* optionName = getName(uv->options[i]); - if (optionName && *optionName == *subName) - { - foundHeuristic = true; - startIndex = i; - break; - } - } - } - - if (auto subMatchTag = getTableMatchTag(subTy)) - { - for (size_t i = 0; i < uv->options.size(); ++i) - { - auto optionMatchTag = getTableMatchTag(uv->options[i]); - if (optionMatchTag && optionMatchTag->first == subMatchTag->first && *optionMatchTag->second == *subMatchTag->second) - { - foundHeuristic = true; - startIndex = i; - break; - } - } - } - - if (!foundHeuristic && cacheEnabled) - { - for (size_t i = 0; i < uv->options.size(); ++i) - { - TypeId type = uv->options[i]; - - if (cache.contains({type, subTy}) && (variance == Covariant || cache.contains({subTy, type}))) - { - startIndex = i; - break; - } - } - } - - for (size_t i = 0; i < uv->options.size(); ++i) - { - TypeId type = uv->options[(i + startIndex) % uv->options.size()]; - Unifier innerState = makeChildUnifier(); - innerState.tryUnify_(subTy, type, isFunctionCall); - - if (innerState.errors.empty()) - { - found = true; - if (FFlag::LuauUseCommittingTxnLog) - log.concat(std::move(innerState.log)); - else - DEPRECATED_log.concat(std::move(innerState.DEPRECATED_log)); - - break; - } - else if (auto e = hasUnificationTooComplex(innerState.errors)) - { - unificationTooComplex = e; - } - else if (!isNil(type)) - { - failedOptionCount++; - - if (!failedOption) - failedOption = {innerState.errors.front()}; - } - - if (!FFlag::LuauUseCommittingTxnLog) - innerState.DEPRECATED_log.rollback(); - } - - if (unificationTooComplex) - { - reportError(*unificationTooComplex); - } - else if (!found) - { - if ((failedOptionCount == 1 || foundHeuristic) && failedOption) - reportError( - TypeError{location, TypeMismatch{superTy, subTy, "None of the union options are compatible. For example:", *failedOption}}); - else - reportError(TypeError{location, TypeMismatch{superTy, subTy, "none of the union options are compatible"}}); - } + tryUnifyTypeWithUnion(subTy, superTy, uv, cacheEnabled, isFunctionCall); } else if (const IntersectionTypeVar* uv = FFlag::LuauUseCommittingTxnLog ? log.getMutable(superTy) : get(superTy)) { - std::optional unificationTooComplex; - std::optional firstFailedOption; - - // T <: A & B if A <: T and B <: T - for (TypeId type : uv->parts) - { - Unifier innerState = makeChildUnifier(); - innerState.tryUnify_(subTy, type, /*isFunctionCall*/ false, /*isIntersection*/ true); - - if (auto e = hasUnificationTooComplex(innerState.errors)) - unificationTooComplex = e; - else if (!innerState.errors.empty()) - { - if (!firstFailedOption) - firstFailedOption = {innerState.errors.front()}; - } - - if (FFlag::LuauUseCommittingTxnLog) - log.concat(std::move(innerState.log)); - else - DEPRECATED_log.concat(std::move(innerState.DEPRECATED_log)); - } - - if (unificationTooComplex) - reportError(*unificationTooComplex); - else if (firstFailedOption) - reportError(TypeError{location, TypeMismatch{superTy, subTy, "Not all intersection parts are compatible.", *firstFailedOption}}); + tryUnifyTypeWithIntersection(subTy, superTy, uv); } else if (const IntersectionTypeVar* uv = FFlag::LuauUseCommittingTxnLog ? log.getMutable(subTy) : get(subTy)) { - // A & B <: T if T <: A or T <: B - bool found = false; - std::optional unificationTooComplex; - - size_t startIndex = 0; - - if (cacheEnabled) - { - for (size_t i = 0; i < uv->parts.size(); ++i) - { - TypeId type = uv->parts[i]; - - if (cache.contains({superTy, type}) && (variance == Covariant || cache.contains({type, superTy}))) - { - startIndex = i; - break; - } - } - } - - for (size_t i = 0; i < uv->parts.size(); ++i) - { - TypeId type = uv->parts[(i + startIndex) % uv->parts.size()]; - Unifier innerState = makeChildUnifier(); - innerState.tryUnify_(type, superTy, isFunctionCall); - - if (innerState.errors.empty()) - { - found = true; - if (FFlag::LuauUseCommittingTxnLog) - log.concat(std::move(innerState.log)); - else - DEPRECATED_log.concat(std::move(innerState.DEPRECATED_log)); - break; - } - else if (auto e = hasUnificationTooComplex(innerState.errors)) - { - unificationTooComplex = e; - } - - if (!FFlag::LuauUseCommittingTxnLog) - innerState.DEPRECATED_log.rollback(); - } - - if (unificationTooComplex) - reportError(*unificationTooComplex); - else if (!found) - { - reportError(TypeError{location, TypeMismatch{superTy, subTy, "none of the intersection parts are compatible"}}); - } + tryUnifyIntersectionWithType(subTy, uv, superTy, cacheEnabled, isFunctionCall); } else if ((FFlag::LuauUseCommittingTxnLog && log.getMutable(superTy) && log.getMutable(subTy)) || (!FFlag::LuauUseCommittingTxnLog && get(superTy) && get(subTy))) @@ -797,6 +589,253 @@ void Unifier::tryUnify_(TypeId subTy, TypeId superTy, bool isFunctionCall, bool DEPRECATED_log.popSeen(superTy, subTy); } +void Unifier::tryUnifyUnionWithType(TypeId subTy, const UnionTypeVar* uv, TypeId superTy) +{ + // A | B <: T if A <: T and B <: T + bool failed = false; + std::optional unificationTooComplex; + std::optional firstFailedOption; + + size_t count = uv->options.size(); + size_t i = 0; + + for (TypeId type : uv->options) + { + Unifier innerState = makeChildUnifier(); + innerState.tryUnify_(type, superTy); + + if (auto e = hasUnificationTooComplex(innerState.errors)) + unificationTooComplex = e; + else if (!innerState.errors.empty()) + { + // 'nil' option is skipped from extended report because we present the type in a special way - 'T?' + if (!firstFailedOption && !isNil(type)) + firstFailedOption = {innerState.errors.front()}; + + failed = true; + } + + if (FFlag::LuauUseCommittingTxnLog) + { + if (i == count - 1) + { + log.concat(std::move(innerState.log)); + } + } + else + { + if (i != count - 1) + { + innerState.DEPRECATED_log.rollback(); + } + else + { + DEPRECATED_log.concat(std::move(innerState.DEPRECATED_log)); + } + } + + ++i; + } + + if (unificationTooComplex) + reportError(*unificationTooComplex); + else if (failed) + { + if (firstFailedOption) + reportError(TypeError{location, TypeMismatch{superTy, subTy, "Not all union options are compatible.", *firstFailedOption}}); + else + reportError(TypeError{location, TypeMismatch{superTy, subTy}}); + } +} + +void Unifier::tryUnifyTypeWithUnion(TypeId subTy, TypeId superTy, const UnionTypeVar* uv, bool cacheEnabled, bool isFunctionCall) +{ + // T <: A | B if T <: A or T <: B + bool found = false; + std::optional unificationTooComplex; + + size_t failedOptionCount = 0; + std::optional failedOption; + + bool foundHeuristic = false; + size_t startIndex = 0; + + if (const std::string* subName = getName(subTy)) + { + for (size_t i = 0; i < uv->options.size(); ++i) + { + const std::string* optionName = getName(uv->options[i]); + if (optionName && *optionName == *subName) + { + foundHeuristic = true; + startIndex = i; + break; + } + } + } + + if (auto subMatchTag = getTableMatchTag(subTy)) + { + for (size_t i = 0; i < uv->options.size(); ++i) + { + auto optionMatchTag = getTableMatchTag(uv->options[i]); + if (optionMatchTag && optionMatchTag->first == subMatchTag->first && *optionMatchTag->second == *subMatchTag->second) + { + foundHeuristic = true; + startIndex = i; + break; + } + } + } + + if (!foundHeuristic && cacheEnabled) + { + auto& cache = sharedState.cachedUnify; + + for (size_t i = 0; i < uv->options.size(); ++i) + { + TypeId type = uv->options[i]; + + if (cache.contains({type, subTy}) && (variance == Covariant || cache.contains({subTy, type}))) + { + startIndex = i; + break; + } + } + } + + for (size_t i = 0; i < uv->options.size(); ++i) + { + TypeId type = uv->options[(i + startIndex) % uv->options.size()]; + Unifier innerState = makeChildUnifier(); + innerState.tryUnify_(subTy, type, isFunctionCall); + + if (innerState.errors.empty()) + { + found = true; + if (FFlag::LuauUseCommittingTxnLog) + log.concat(std::move(innerState.log)); + else + DEPRECATED_log.concat(std::move(innerState.DEPRECATED_log)); + + break; + } + else if (auto e = hasUnificationTooComplex(innerState.errors)) + { + unificationTooComplex = e; + } + else if (!isNil(type)) + { + failedOptionCount++; + + if (!failedOption) + failedOption = {innerState.errors.front()}; + } + + if (!FFlag::LuauUseCommittingTxnLog) + innerState.DEPRECATED_log.rollback(); + } + + if (unificationTooComplex) + { + reportError(*unificationTooComplex); + } + else if (!found) + { + if ((failedOptionCount == 1 || foundHeuristic) && failedOption) + reportError(TypeError{location, TypeMismatch{superTy, subTy, "None of the union options are compatible. For example:", *failedOption}}); + else + reportError(TypeError{location, TypeMismatch{superTy, subTy, "none of the union options are compatible"}}); + } +} + +void Unifier::tryUnifyTypeWithIntersection(TypeId subTy, TypeId superTy, const IntersectionTypeVar* uv) +{ + std::optional unificationTooComplex; + std::optional firstFailedOption; + + // T <: A & B if A <: T and B <: T + for (TypeId type : uv->parts) + { + Unifier innerState = makeChildUnifier(); + innerState.tryUnify_(subTy, type, /*isFunctionCall*/ false, /*isIntersection*/ true); + + if (auto e = hasUnificationTooComplex(innerState.errors)) + unificationTooComplex = e; + else if (!innerState.errors.empty()) + { + if (!firstFailedOption) + firstFailedOption = {innerState.errors.front()}; + } + + if (FFlag::LuauUseCommittingTxnLog) + log.concat(std::move(innerState.log)); + else + DEPRECATED_log.concat(std::move(innerState.DEPRECATED_log)); + } + + if (unificationTooComplex) + reportError(*unificationTooComplex); + else if (firstFailedOption) + reportError(TypeError{location, TypeMismatch{superTy, subTy, "Not all intersection parts are compatible.", *firstFailedOption}}); +} + +void Unifier::tryUnifyIntersectionWithType(TypeId subTy, const IntersectionTypeVar* uv, TypeId superTy, bool cacheEnabled, bool isFunctionCall) +{ + // A & B <: T if T <: A or T <: B + bool found = false; + std::optional unificationTooComplex; + + size_t startIndex = 0; + + if (cacheEnabled) + { + auto& cache = sharedState.cachedUnify; + + for (size_t i = 0; i < uv->parts.size(); ++i) + { + TypeId type = uv->parts[i]; + + if (cache.contains({superTy, type}) && (variance == Covariant || cache.contains({type, superTy}))) + { + startIndex = i; + break; + } + } + } + + for (size_t i = 0; i < uv->parts.size(); ++i) + { + TypeId type = uv->parts[(i + startIndex) % uv->parts.size()]; + Unifier innerState = makeChildUnifier(); + innerState.tryUnify_(type, superTy, isFunctionCall); + + if (innerState.errors.empty()) + { + found = true; + if (FFlag::LuauUseCommittingTxnLog) + log.concat(std::move(innerState.log)); + else + DEPRECATED_log.concat(std::move(innerState.DEPRECATED_log)); + break; + } + else if (auto e = hasUnificationTooComplex(innerState.errors)) + { + unificationTooComplex = e; + } + + if (!FFlag::LuauUseCommittingTxnLog) + innerState.DEPRECATED_log.rollback(); + } + + if (unificationTooComplex) + reportError(*unificationTooComplex); + else if (!found) + { + reportError(TypeError{location, TypeMismatch{superTy, subTy, "none of the intersection parts are compatible"}}); + } +} + void Unifier::cacheResult(TypeId subTy, TypeId superTy) { bool* superTyInfo = sharedState.skipCacheForType.find(superTy); @@ -1119,8 +1158,8 @@ void Unifier::tryUnify_(TypePackId subTp, TypePackId superTp, bool isFunctionCal auto [superTypes, superTail] = logAwareFlatten(superTp, log); auto [subTypes, subTail] = logAwareFlatten(subTp, log); - bool noInfiniteGrowth = - (superTypes.size() != subTypes.size()) && (superTail && get(*superTail)) && (subTail && get(*subTail)); + bool noInfiniteGrowth = (superTypes.size() != subTypes.size()) && (superTail && log.getMutable(*superTail)) && + (subTail && log.getMutable(*subTail)); auto superIter = WeirdIter(superTp, log); auto subIter = WeirdIter(subTp, log); @@ -1667,6 +1706,13 @@ void Unifier::tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) TableTypeVar* superTable = getMutable(superTy); TableTypeVar* subTable = getMutable(subTy); + + if (FFlag::LuauUseCommittingTxnLog) + { + superTable = log.getMutable(superTy); + subTable = log.getMutable(subTy); + } + if (!superTable || !subTable) ice("passed non-table types to unifyTables"); @@ -1679,7 +1725,11 @@ void Unifier::tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) for (const auto& [propName, superProp] : superTable->props) { auto subIter = subTable->props.find(propName); - if (subIter == subTable->props.end() && !isOptional(superProp.type) && !get(follow(superProp.type))) + + bool isAny = + FFlag::LuauUseCommittingTxnLog ? log.getMutable(log.follow(superProp.type)) : get(follow(superProp.type)); + + if (subIter == subTable->props.end() && !isOptional(superProp.type) && !isAny) missingProperties.push_back(propName); } @@ -1697,7 +1747,10 @@ void Unifier::tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) for (const auto& [propName, subProp] : subTable->props) { auto superIter = superTable->props.find(propName); - if (superIter == superTable->props.end() && !isOptional(subProp.type) && !get(follow(subProp.type))) + + bool isAny = + FFlag::LuauUseCommittingTxnLog ? log.getMutable(log.follow(subProp.type)) : get(follow(subProp.type)); + if (superIter == superTable->props.end() && !isOptional(subProp.type) && !isAny) extraProperties.push_back(propName); } @@ -1775,6 +1828,7 @@ void Unifier::tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) TableTypeVar* ttv = getMutable(pendingSub); LUAU_ASSERT(ttv); ttv->props[name] = prop; + subTable = ttv; } else { @@ -1831,6 +1885,7 @@ void Unifier::tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) PendingType* pendingSuper = log.queue(superTy); TableTypeVar* pendingSuperTtv = getMutable(pendingSuper); pendingSuperTtv->props[name] = clone; + superTable = pendingSuperTtv; } else { @@ -1853,6 +1908,7 @@ void Unifier::tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) PendingType* pendingSuper = log.queue(superTy); TableTypeVar* pendingSuperTtv = getMutable(pendingSuper); pendingSuperTtv->props[name] = prop; + superTable = pendingSuperTtv; } else { @@ -1967,7 +2023,7 @@ void Unifier::tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) } else { - DEPRECATED_log(subTy); + DEPRECATED_log(subTable); subTable->boundTo = superTy; } } @@ -2408,8 +2464,7 @@ void Unifier::tryUnifyWithMetatable(TypeId subTy, TypeId superTy, bool reversed) if (auto e = hasUnificationTooComplex(innerState.errors)) reportError(*e); else if (!innerState.errors.empty()) - reportError( - TypeError{location, TypeMismatch{reversed ? subTy : superTy, reversed ? superTy : subTy, "", innerState.errors.front()}}); + reportError(TypeError{location, TypeMismatch{reversed ? subTy : superTy, reversed ? superTy : subTy, "", innerState.errors.front()}}); if (FFlag::LuauUseCommittingTxnLog) log.concat(std::move(innerState.log)); diff --git a/Ast/src/Parser.cpp b/Ast/src/Parser.cpp index 3c607d24..f559e2e0 100644 --- a/Ast/src/Parser.cpp +++ b/Ast/src/Parser.cpp @@ -10,7 +10,6 @@ // See docs/SyntaxChanges.md for an explanation. LUAU_FASTINTVARIABLE(LuauRecursionLimit, 1000) LUAU_FASTINTVARIABLE(LuauParseErrorLimit, 100) -LUAU_FASTFLAGVARIABLE(LuauFixAmbiguousErrorRecoveryInAssign, false) LUAU_FASTFLAGVARIABLE(LuauParseSingletonTypes, false) LUAU_FASTFLAGVARIABLE(LuauParseTypeAliasDefaults, false) LUAU_FASTFLAGVARIABLE(LuauParseRecoverTypePackEllipsis, false) @@ -957,7 +956,7 @@ AstStat* Parser::parseAssignment(AstExpr* initial) { nextLexeme(); - AstExpr* expr = parsePrimaryExpr(/* asStatement= */ FFlag::LuauFixAmbiguousErrorRecoveryInAssign); + AstExpr* expr = parsePrimaryExpr(/* asStatement= */ true); if (!isExprLValue(expr)) expr = reportExprError(expr->location, copy({expr}), "Assigned expression must be a variable or a field"); diff --git a/CLI/Coverage.cpp b/CLI/Coverage.cpp index 254df3f0..a509ab89 100644 --- a/CLI/Coverage.cpp +++ b/CLI/Coverage.cpp @@ -68,7 +68,7 @@ void coverageDump(const char* path) fprintf(f, "TN:\n"); - for (int fref: gCoverage.functions) + for (int fref : gCoverage.functions) { lua_getref(L, fref); diff --git a/CLI/FileUtils.cpp b/CLI/FileUtils.cpp index c6807022..fe005aec 100644 --- a/CLI/FileUtils.cpp +++ b/CLI/FileUtils.cpp @@ -77,7 +77,7 @@ std::optional readFile(const std::string& name) std::optional readStdin() { std::string result; - char buffer[4096] = { }; + char buffer[4096] = {}; while (fgets(buffer, sizeof(buffer), stdin) != nullptr) result.append(buffer); diff --git a/CLI/Repl.cpp b/CLI/Repl.cpp index ab0f0ed0..5af6b508 100644 --- a/CLI/Repl.cpp +++ b/CLI/Repl.cpp @@ -10,7 +10,7 @@ #include "Profiler.h" #include "Coverage.h" -#include "linenoise.hpp" +#include "isocline.h" #include @@ -240,9 +240,10 @@ std::string runCode(lua_State* L, const std::string& source) return std::string(); } -static void completeIndexer(lua_State* L, const char* editBuffer, size_t start, std::vector& completions) +static void completeIndexer(ic_completion_env_t* cenv, const char* editBuffer) { - std::string_view lookup = editBuffer + start; + auto* L = reinterpret_cast(ic_completion_arg(cenv)); + std::string_view lookup = editBuffer; char lastSep = 0; for (;;) @@ -268,13 +269,14 @@ static void completeIndexer(lua_State* L, const char* editBuffer, size_t start, if (!key.empty() && requiredValueType && Luau::startsWith(key, prefix)) { - std::string completion(editBuffer + std::string(key.substr(prefix.size()))); + std::string completedComponent(key.substr(prefix.size())); + std::string completion(editBuffer + completedComponent); if (valueType == LUA_TFUNCTION) { // Add an opening paren for function calls by default. completion += "("; } - completions.push_back(completion); + ic_add_completion_ex(cenv, completion.data(), key.data(), nullptr); } } lua_pop(L, 1); @@ -310,19 +312,23 @@ static void completeIndexer(lua_State* L, const char* editBuffer, size_t start, lua_pop(L, 1); } -static void completeRepl(lua_State* L, const char* editBuffer, std::vector& completions) +static bool isMethodOrFunctionChar(const char* s, long len) { - size_t start = strlen(editBuffer); - while (start > 0 && (isalnum(editBuffer[start - 1]) || editBuffer[start - 1] == '.' || editBuffer[start - 1] == ':' || editBuffer[start - 1] == '_')) - start--; + char c = *s; + return len == 1 && (isalnum(c) || c == '.' || c == ':' || c == '_'); +} + +static void completeRepl(ic_completion_env_t* cenv, const char* editBuffer) +{ + auto* L = reinterpret_cast(ic_completion_arg(cenv)); // look the value up in current global table first lua_pushvalue(L, LUA_GLOBALSINDEX); - completeIndexer(L, editBuffer, start, completions); + ic_complete_word(cenv, editBuffer, completeIndexer, isMethodOrFunctionChar); // and in actual global table after that lua_getglobal(L, "_G"); - completeIndexer(L, editBuffer, start, completions); + ic_complete_word(cenv, editBuffer, completeIndexer, isMethodOrFunctionChar); } struct LinenoiseScopedHistory @@ -341,13 +347,11 @@ struct LinenoiseScopedHistory } if (!historyFilepath.empty()) - linenoise::LoadHistory(historyFilepath.c_str()); + ic_set_history(historyFilepath.c_str(), -1 /* default entries (= 200) */); } ~LinenoiseScopedHistory() { - if (!historyFilepath.empty()) - linenoise::SaveHistory(historyFilepath.c_str()); } std::string historyFilepath; @@ -355,28 +359,32 @@ struct LinenoiseScopedHistory static void runReplImpl(lua_State* L) { - linenoise::SetCompletionCallback([L](const char* editBuffer, std::vector& completions) { - completeRepl(L, editBuffer, completions); - }); + ic_set_default_completer(completeRepl, L); + + // Make brace matching easier to see + ic_style_def("ic-bracematch", "teal"); + + // Prevent auto insertion of braces + ic_enable_brace_insertion(false); std::string buffer; LinenoiseScopedHistory scopedHistory; for (;;) { - bool quit = false; - std::string line = linenoise::Readline(buffer.empty() ? "> " : ">> ", quit); - if (quit) + const char* line = ic_readline(buffer.empty() ? "" : ">"); + if (!line) break; if (buffer.empty() && runCode(L, std::string("return ") + line) == std::string()) { - linenoise::AddHistory(line.c_str()); + ic_history_add(line); continue; } + if (!buffer.empty()) + buffer += "\n"; buffer += line; - buffer += " "; // linenoise doesn't work very well with multiline history entries std::string error = runCode(L, buffer); @@ -390,8 +398,9 @@ static void runReplImpl(lua_State* L) fprintf(stdout, "%s\n", error.c_str()); } - linenoise::AddHistory(buffer.c_str()); + ic_history_add(buffer.c_str()); buffer.clear(); + free((void*)line); } } diff --git a/CLI/ReplEntry.cpp b/CLI/ReplEntry.cpp index b3131712..75995e6a 100644 --- a/CLI/ReplEntry.cpp +++ b/CLI/ReplEntry.cpp @@ -1,5 +1,4 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details - #include "Repl.h" @@ -7,4 +6,4 @@ int main(int argc, char** argv) { return replMain(argc, argv); -} \ No newline at end of file +} diff --git a/CMakeLists.txt b/CMakeLists.txt index b9f7a9e1..881d3c3f 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -5,7 +5,7 @@ if(EXT_PLATFORM_STRING) endif() cmake_minimum_required(VERSION 3.0) -project(Luau LANGUAGES CXX) +project(Luau LANGUAGES CXX C) option(LUAU_BUILD_CLI "Build CLI" ON) option(LUAU_BUILD_TESTS "Build tests" ON) @@ -16,6 +16,7 @@ add_library(Luau.Ast STATIC) add_library(Luau.Compiler STATIC) add_library(Luau.Analysis STATIC) add_library(Luau.VM STATIC) +add_library(isocline STATIC) if(LUAU_BUILD_CLI) add_executable(Luau.Repl.CLI) @@ -52,6 +53,8 @@ target_link_libraries(Luau.Analysis PUBLIC Luau.Ast) target_compile_features(Luau.VM PRIVATE cxx_std_11) target_include_directories(Luau.VM PUBLIC VM/include) +target_include_directories(isocline PUBLIC extern/isocline/include) + set(LUAU_OPTIONS) if(MSVC) @@ -75,9 +78,16 @@ if(LUAU_BUILD_WEB) list(APPEND LUAU_OPTIONS -fexceptions) endif() +set(ISOCLINE_OPTIONS) + +if (NOT MSVC) + list(APPEND ISOCLINE_OPTIONS -Wno-unused-function) +endif() + target_compile_options(Luau.Ast PRIVATE ${LUAU_OPTIONS}) target_compile_options(Luau.Analysis PRIVATE ${LUAU_OPTIONS}) target_compile_options(Luau.VM PRIVATE ${LUAU_OPTIONS}) +target_compile_options(isocline PRIVATE ${LUAU_OPTIONS} ${ISOCLINE_OPTIONS}) if (MSVC AND MSVC_VERSION GREATER_EQUAL 1924) # disable partial redundancy elimination which regresses interpreter codegen substantially in VS2022: @@ -89,8 +99,9 @@ if(LUAU_BUILD_CLI) target_compile_options(Luau.Repl.CLI PRIVATE ${LUAU_OPTIONS}) target_compile_options(Luau.Analyze.CLI PRIVATE ${LUAU_OPTIONS}) - target_include_directories(Luau.Repl.CLI PRIVATE extern) - target_link_libraries(Luau.Repl.CLI PRIVATE Luau.Compiler Luau.VM) + target_include_directories(Luau.Repl.CLI PRIVATE extern extern/isocline/include) + + target_link_libraries(Luau.Repl.CLI PRIVATE Luau.Compiler Luau.VM isocline) if(UNIX) find_library(LIBPTHREAD pthread) @@ -113,7 +124,7 @@ if(LUAU_BUILD_TESTS) target_compile_options(Luau.CLI.Test PRIVATE ${LUAU_OPTIONS}) target_include_directories(Luau.CLI.Test PRIVATE extern CLI) - target_link_libraries(Luau.CLI.Test PRIVATE Luau.Compiler Luau.VM) + target_link_libraries(Luau.CLI.Test PRIVATE Luau.Compiler Luau.VM isocline) if(UNIX) find_library(LIBPTHREAD pthread) if (LIBPTHREAD) diff --git a/Compiler/src/Builtins.cpp b/Compiler/src/Builtins.cpp index a907271c..26360c49 100644 --- a/Compiler/src/Builtins.cpp +++ b/Compiler/src/Builtins.cpp @@ -4,7 +4,7 @@ #include "Luau/Bytecode.h" #include "Luau/Compiler.h" -LUAU_FASTFLAGVARIABLE(LuauCompileSelectBuiltin, false) +LUAU_FASTFLAGVARIABLE(LuauCompileSelectBuiltin2, false) namespace Luau { @@ -64,7 +64,7 @@ int getBuiltinFunctionId(const Builtin& builtin, const CompileOptions& options) if (builtin.isGlobal("unpack")) return LBF_TABLE_UNPACK; - if (FFlag::LuauCompileSelectBuiltin && builtin.isGlobal("select")) + if (FFlag::LuauCompileSelectBuiltin2 && builtin.isGlobal("select")) return LBF_SELECT_VARARG; if (builtin.object == "math") diff --git a/Compiler/src/Compiler.cpp b/Compiler/src/Compiler.cpp index 7da85244..e4253adc 100644 --- a/Compiler/src/Compiler.cpp +++ b/Compiler/src/Compiler.cpp @@ -16,7 +16,7 @@ #include LUAU_FASTFLAGVARIABLE(LuauCompileTableIndexOpt, false) -LUAU_FASTFLAG(LuauCompileSelectBuiltin) +LUAU_FASTFLAG(LuauCompileSelectBuiltin2) namespace Luau { @@ -266,7 +266,7 @@ struct Compiler void compileExprSelectVararg(AstExprCall* expr, uint8_t target, uint8_t targetCount, bool targetTop, bool multRet, uint8_t regs) { - LUAU_ASSERT(FFlag::LuauCompileSelectBuiltin); + LUAU_ASSERT(FFlag::LuauCompileSelectBuiltin2); LUAU_ASSERT(targetCount == 1); LUAU_ASSERT(!expr->self); LUAU_ASSERT(expr->args.size == 2 && expr->args.data[1]->is()); @@ -291,6 +291,9 @@ struct Compiler // we can't use TempTop variant here because we need to make sure the arguments we already computed aren't overwritten compileExprTemp(expr->func, regs); + if (argreg != regs + 1) + bytecode.emitABC(LOP_MOVE, uint8_t(regs + 1), argreg, 0); + bytecode.emitABC(LOP_GETVARARGS, uint8_t(regs + 2), 0, 0); size_t callLabel = bytecode.emitLabel(); @@ -405,7 +408,7 @@ struct Compiler if (bfid == LBF_SELECT_VARARG) { - LUAU_ASSERT(FFlag::LuauCompileSelectBuiltin); + LUAU_ASSERT(FFlag::LuauCompileSelectBuiltin2); // Optimization: compile select(_, ...) as FASTCALL1; the builtin will read variadic arguments directly // note: for now we restrict this to single-return expressions since our runtime code doesn't deal with general cases if (multRet == false && targetCount == 1 && expr->args.size == 2 && expr->args.data[1]->is()) diff --git a/Compiler/src/TableShape.cpp b/Compiler/src/TableShape.cpp index 9dc2f0a4..5a866e87 100644 --- a/Compiler/src/TableShape.cpp +++ b/Compiler/src/TableShape.cpp @@ -1,8 +1,6 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #include "TableShape.h" -LUAU_FASTFLAGVARIABLE(LuauPredictTableSizeLoop, false) - namespace Luau { namespace Compile @@ -87,9 +85,6 @@ struct ShapeVisitor : AstVisitor } else if (AstExprLocal* iter = index->as()) { - if (!FFlag::LuauPredictTableSizeLoop) - return; - if (const unsigned int* bound = loops.find(iter->local)) { TableShape& shape = shapes[*table]; @@ -143,9 +138,6 @@ struct ShapeVisitor : AstVisitor bool visit(AstStatFor* node) override { - if (!FFlag::LuauPredictTableSizeLoop) - return true; - AstExprConstantNumber* from = node->from->as(); AstExprConstantNumber* to = node->to->as(); diff --git a/Makefile b/Makefile index 638c4c63..80eff018 100644 --- a/Makefile +++ b/Makefile @@ -23,6 +23,10 @@ VM_SOURCES=$(wildcard VM/src/*.cpp) VM_OBJECTS=$(VM_SOURCES:%=$(BUILD)/%.o) VM_TARGET=$(BUILD)/libluauvm.a +ISOCLINE_SOURCES=extern/isocline/src/isocline.c +ISOCLINE_OBJECTS=$(ISOCLINE_SOURCES:%=$(BUILD)/%.o) +ISOCLINE_TARGET=$(BUILD)/libisocline.a + TESTS_SOURCES=$(wildcard tests/*.cpp) CLI/FileUtils.cpp CLI/Profiler.cpp CLI/Coverage.cpp CLI/Repl.cpp TESTS_OBJECTS=$(TESTS_SOURCES:%=$(BUILD)/%.o) TESTS_TARGET=$(BUILD)/luau-tests @@ -43,7 +47,7 @@ ifneq ($(flags),) TESTS_ARGS+=--fflags=$(flags) endif -OBJECTS=$(AST_OBJECTS) $(COMPILER_OBJECTS) $(ANALYSIS_OBJECTS) $(VM_OBJECTS) $(TESTS_OBJECTS) $(CLI_OBJECTS) $(FUZZ_OBJECTS) +OBJECTS=$(AST_OBJECTS) $(COMPILER_OBJECTS) $(ANALYSIS_OBJECTS) $(VM_OBJECTS) $(ISOCLINE_OBJECTS) $(TESTS_OBJECTS) $(CLI_OBJECTS) $(FUZZ_OBJECTS) # common flags CXXFLAGS=-g -Wall @@ -90,8 +94,9 @@ $(AST_OBJECTS): CXXFLAGS+=-std=c++17 -IAst/include $(COMPILER_OBJECTS): CXXFLAGS+=-std=c++17 -ICompiler/include -IAst/include $(ANALYSIS_OBJECTS): CXXFLAGS+=-std=c++17 -IAst/include -IAnalysis/include $(VM_OBJECTS): CXXFLAGS+=-std=c++11 -IVM/include +$(ISOCLINE_OBJECTS): CXXFLAGS+=-Wno-unused-function -Iextern/isocline/include $(TESTS_OBJECTS): CXXFLAGS+=-std=c++17 -IAst/include -ICompiler/include -IAnalysis/include -IVM/include -ICLI -Iextern -$(REPL_CLI_OBJECTS): CXXFLAGS+=-std=c++17 -IAst/include -ICompiler/include -IVM/include -Iextern +$(REPL_CLI_OBJECTS): CXXFLAGS+=-std=c++17 -IAst/include -ICompiler/include -IVM/include -Iextern -Iextern/isocline/include $(ANALYZE_CLI_OBJECTS): CXXFLAGS+=-std=c++17 -IAst/include -IAnalysis/include -Iextern $(FUZZ_OBJECTS): CXXFLAGS+=-std=c++17 -IAst/include -ICompiler/include -IAnalysis/include -IVM/include @@ -116,8 +121,8 @@ coverage: $(TESTS_TARGET) $(TESTS_TARGET) llvm-profdata merge default.profraw default-flags.profraw -o default.profdata rm default.profraw default-flags.profraw - llvm-cov show -format=html -show-instantiations=false -show-line-counts=true -show-region-summary=false -ignore-filename-regex=\(tests\|extern\)/.* -output-dir=coverage --instr-profile default.profdata build/coverage/luau-tests - llvm-cov report -ignore-filename-regex=\(tests\|extern\)/.* -show-region-summary=false --instr-profile default.profdata build/coverage/luau-tests + llvm-cov show -format=html -show-instantiations=false -show-line-counts=true -show-region-summary=false -ignore-filename-regex=\(tests\|extern\|CLI\)/.* -output-dir=coverage --instr-profile default.profdata build/coverage/luau-tests + llvm-cov report -ignore-filename-regex=\(tests\|extern\|CLI\)/.* -show-region-summary=false --instr-profile default.profdata build/coverage/luau-tests llvm-cov export -format lcov --instr-profile default.profdata build/coverage/luau-tests >coverage.info format: @@ -135,8 +140,8 @@ luau-analyze: $(ANALYZE_CLI_TARGET) ln -fs $^ $@ # executable targets -$(TESTS_TARGET): $(TESTS_OBJECTS) $(ANALYSIS_TARGET) $(COMPILER_TARGET) $(AST_TARGET) $(VM_TARGET) -$(REPL_CLI_TARGET): $(REPL_CLI_OBJECTS) $(COMPILER_TARGET) $(AST_TARGET) $(VM_TARGET) +$(TESTS_TARGET): $(TESTS_OBJECTS) $(ANALYSIS_TARGET) $(COMPILER_TARGET) $(AST_TARGET) $(VM_TARGET) $(ISOCLINE_TARGET) +$(REPL_CLI_TARGET): $(REPL_CLI_OBJECTS) $(COMPILER_TARGET) $(AST_TARGET) $(VM_TARGET) $(ISOCLINE_TARGET) $(ANALYZE_CLI_TARGET): $(ANALYZE_CLI_OBJECTS) $(ANALYSIS_TARGET) $(AST_TARGET) $(TESTS_TARGET) $(REPL_CLI_TARGET) $(ANALYZE_CLI_TARGET): @@ -154,8 +159,9 @@ $(AST_TARGET): $(AST_OBJECTS) $(COMPILER_TARGET): $(COMPILER_OBJECTS) $(ANALYSIS_TARGET): $(ANALYSIS_OBJECTS) $(VM_TARGET): $(VM_OBJECTS) +$(ISOCLINE_TARGET): $(ISOCLINE_OBJECTS) -$(AST_TARGET) $(COMPILER_TARGET) $(ANALYSIS_TARGET) $(VM_TARGET): +$(AST_TARGET) $(COMPILER_TARGET) $(ANALYSIS_TARGET) $(VM_TARGET) $(ISOCLINE_TARGET): ar rcs $@ $^ # object file targets @@ -163,6 +169,10 @@ $(BUILD)/%.cpp.o: %.cpp @mkdir -p $(dir $@) $(CXX) $< $(CXXFLAGS) -c -MMD -MP -o $@ +$(BUILD)/%.c.o: %.c + @mkdir -p $(dir $@) + $(CXX) -x c $< $(CXXFLAGS) -c -MMD -MP -o $@ + # protobuf fuzzer setup fuzz/luau.pb.cpp: fuzz/luau.proto build/libprotobuf-mutator cd fuzz && ../build/libprotobuf-mutator/external.protobuf/bin/protoc luau.proto --cpp_out=. diff --git a/Sources.cmake b/Sources.cmake index 22e7af22..b36b6db5 100644 --- a/Sources.cmake +++ b/Sources.cmake @@ -167,6 +167,11 @@ target_sources(Luau.VM PRIVATE VM/src/lvm.h ) +target_sources(isocline PRIVATE + extern/isocline/include/isocline.h + extern/isocline/src/isocline.c +) + if(TARGET Luau.Repl.CLI) # Luau.Repl.CLI Sources target_sources(Luau.Repl.CLI PRIVATE diff --git a/VM/include/luaconf.h b/VM/include/luaconf.h index 7e0832e7..c5bf1c18 100644 --- a/VM/include/luaconf.h +++ b/VM/include/luaconf.h @@ -83,7 +83,7 @@ #endif #ifndef LUAI_GCSTEPSIZE -#define LUAI_GCSTEPSIZE 1 /* GC runs every KB of memory allocation */ +#define LUAI_GCSTEPSIZE 1 /* GC runs every KB of memory allocation */ #endif /* LUA_MINSTACK is the guaranteed number of Lua stack slots available to a C function */ @@ -153,6 +153,6 @@ long l; \ } -#define LUA_VECTOR_SIZE 3 /* must be 3 or 4 */ +#define LUA_VECTOR_SIZE 3 /* must be 3 or 4 */ #define LUA_EXTRA_SIZE LUA_VECTOR_SIZE - 2 diff --git a/VM/src/lcorolib.cpp b/VM/src/lcorolib.cpp index 19222861..7592a14c 100644 --- a/VM/src/lcorolib.cpp +++ b/VM/src/lcorolib.cpp @@ -250,7 +250,7 @@ static int coclose(lua_State* L) { lua_pushboolean(L, false); if (lua_gettop(co)) - lua_xmove(co, L, 1); /* move error message */ + lua_xmove(co, L, 1); /* move error message */ lua_resetthread(co); return 2; } diff --git a/VM/src/ldebug.cpp b/VM/src/ldebug.cpp index 2b5382bb..e9930f7a 100644 --- a/VM/src/ldebug.cpp +++ b/VM/src/ldebug.cpp @@ -12,7 +12,6 @@ #include #include -LUAU_FASTFLAG(LuauBytecodeV2Read) LUAU_FASTFLAG(LuauBytecodeV2Force) static const char* getfuncname(Closure* f); @@ -96,7 +95,7 @@ static int getlinedefined(Proto* p) { if (FFlag::LuauBytecodeV2Force) return p->linedefined; - else if (FFlag::LuauBytecodeV2Read && p->linedefined >= 0) + else if (p->linedefined >= 0) return p->linedefined; else return luaG_getline(p, 0); diff --git a/VM/src/lfunc.cpp b/VM/src/lfunc.cpp index 6088f71c..582d4627 100644 --- a/VM/src/lfunc.cpp +++ b/VM/src/lfunc.cpp @@ -90,7 +90,7 @@ UpVal* luaF_findupval(lua_State* L, StkId level) uv->tt = LUA_TUPVAL; uv->marked = luaC_white(g); uv->memcat = L->activememcat; - uv->v = level; /* current value lives in the stack */ + uv->v = level; /* current value lives in the stack */ // chain the upvalue in the threads open upvalue list at the proper position UpVal* next = *pp; @@ -138,8 +138,8 @@ void luaF_unlinkupval(UpVal* uv) void luaF_freeupval(lua_State* L, UpVal* uv, lua_Page* page) { - if (uv->v != &uv->u.value) /* is it open? */ - luaF_unlinkupval(uv); /* remove from open list */ + if (uv->v != &uv->u.value) /* is it open? */ + luaF_unlinkupval(uv); /* remove from open list */ luaM_freegco(L, uv, sizeof(UpVal), uv->memcat, page); /* free upvalue */ } diff --git a/VM/src/lgc.cpp b/VM/src/lgc.cpp index 82ac0009..835572fa 100644 --- a/VM/src/lgc.cpp +++ b/VM/src/lgc.cpp @@ -759,6 +759,8 @@ static int sweepgcopage(lua_State* L, lua_Page* page) // when true is returned it means that the element was deleted if (sweepgco(L, page, gco)) { + LUAU_ASSERT(busyBlocks > 0); + // if the last block was removed, page would be removed as well if (--busyBlocks == 0) return int(pos - start) / blockSize + 1; diff --git a/VM/src/lmem.cpp b/VM/src/lmem.cpp index e1dbce50..de85cf59 100644 --- a/VM/src/lmem.cpp +++ b/VM/src/lmem.cpp @@ -57,8 +57,7 @@ const size_t kSizeClasses = LUA_SIZECLASSES; const size_t kMaxSmallSize = 512; const size_t kPageSize = 16 * 1024 - 24; // slightly under 16KB since that results in less fragmentation due to heap metadata const size_t kBlockHeader = sizeof(double) > sizeof(void*) ? sizeof(double) : sizeof(void*); // suitable for aligning double & void* on all platforms -// TODO (FFlagLuauGcPagedSweep): when 'next' is removed, 'kBlockHeader' can be used unconditionally -const size_t kGCOHeader = sizeof(GCheader) > kBlockHeader ? sizeof(GCheader) : kBlockHeader; +const size_t kGCOLinkOffset = (sizeof(GCheader) + sizeof(void*) - 1) & ~(sizeof(void*) - 1); // GCO pages contain freelist links after the GC header struct SizeClassConfig { @@ -101,12 +100,12 @@ struct SizeClassConfig const SizeClassConfig kSizeClassConfig; -// size class for a block of size sz +// size class for a block of size sz; returns -1 for size=0 because empty allocations take no space #define sizeclass(sz) (size_t((sz)-1) < kMaxSmallSize ? kSizeClassConfig.classForSize[sz] : -1) // metadata for a block is stored in the first pointer of the block #define metadata(block) (*(void**)(block)) -#define freegcolink(block) (*(void**)((char*)block + kGCOHeader)) +#define freegcolink(block) (*(void**)((char*)block + kGCOLinkOffset)) /* ** About the realloc function: @@ -157,7 +156,7 @@ l_noret luaM_toobig(lua_State* L) luaG_runerror(L, "memory allocation error: block too big"); } -static lua_Page* luaM_newpage(lua_State* L, uint8_t sizeClass) +static lua_Page* newpageold(lua_State* L, uint8_t sizeClass) { LUAU_ASSERT(!FFlag::LuauGcPagedSweep); @@ -253,7 +252,7 @@ static lua_Page* newclasspage(lua_State* L, lua_Page** freepageset, lua_Page** g return page; } -static void luaM_freepage(lua_State* L, lua_Page* page, uint8_t sizeClass) +static void freepageold(lua_State* L, lua_Page* page, uint8_t sizeClass) { LUAU_ASSERT(!FFlag::LuauGcPagedSweep); @@ -310,7 +309,7 @@ static void freeclasspage(lua_State* L, lua_Page** freepageset, lua_Page** gcopa freepage(L, gcopageset, page); } -static void* luaM_newblock(lua_State* L, int sizeClass) +static void* newblock(lua_State* L, int sizeClass) { global_State* g = L->global; lua_Page* page = g->freepages[sizeClass]; @@ -321,7 +320,7 @@ static void* luaM_newblock(lua_State* L, int sizeClass) if (FFlag::LuauGcPagedSweep) page = newclasspage(L, g->freepages, NULL, sizeClass, true); else - page = luaM_newpage(L, sizeClass); + page = newpageold(L, sizeClass); } LUAU_ASSERT(!page->prev); @@ -363,7 +362,7 @@ static void* luaM_newblock(lua_State* L, int sizeClass) return (char*)block + kBlockHeader; } -static void* luaM_newgcoblock(lua_State* L, int sizeClass) +static void* newgcoblock(lua_State* L, int sizeClass) { LUAU_ASSERT(FFlag::LuauGcPagedSweep); @@ -390,11 +389,10 @@ static void* luaM_newgcoblock(lua_State* L, int sizeClass) } else { + block = page->freeList; + ASAN_UNPOISON_MEMORY_REGION((char*)block + sizeof(GCheader), page->blockSize - sizeof(GCheader)); + // when separate block metadata is not used, free list link is stored inside the block data itself - block = (char*)page->freeList - kGCOHeader; - - ASAN_UNPOISON_MEMORY_REGION((char*)block + kGCOHeader, page->blockSize - kGCOHeader); - page->freeList = freegcolink(block); page->busyBlocks++; } @@ -412,7 +410,7 @@ static void* luaM_newgcoblock(lua_State* L, int sizeClass) return (char*)block; } -static void luaM_freeblock(lua_State* L, int sizeClass, void* block) +static void freeblock(lua_State* L, int sizeClass, void* block) { global_State* g = L->global; @@ -450,11 +448,11 @@ static void luaM_freeblock(lua_State* L, int sizeClass, void* block) if (FFlag::LuauGcPagedSweep) freeclasspage(L, g->freepages, NULL, page, sizeClass); else - luaM_freepage(L, page, sizeClass); + freepageold(L, page, sizeClass); } } -static void luaM_freegcoblock(lua_State* L, int sizeClass, void* block, lua_Page* page) +static void freegcoblock(lua_State* L, int sizeClass, void* block, lua_Page* page) { LUAU_ASSERT(FFlag::LuauGcPagedSweep); @@ -474,9 +472,9 @@ static void luaM_freegcoblock(lua_State* L, int sizeClass, void* block, lua_Page // when separate block metadata is not used, free list link is stored inside the block data itself freegcolink(block) = page->freeList; - page->freeList = (char*)block + kGCOHeader; + page->freeList = block; - ASAN_POISON_MEMORY_REGION((char*)block + kGCOHeader, page->blockSize - kGCOHeader); + ASAN_POISON_MEMORY_REGION((char*)block + sizeof(GCheader), page->blockSize - sizeof(GCheader)); page->busyBlocks--; @@ -491,7 +489,7 @@ void* luaM_new_(lua_State* L, size_t nsize, uint8_t memcat) int nclass = sizeclass(nsize); - void* block = nclass >= 0 ? luaM_newblock(L, nclass) : (*g->frealloc)(L, g->ud, NULL, 0, nsize); + void* block = nclass >= 0 ? newblock(L, nclass) : (*g->frealloc)(L, g->ud, NULL, 0, nsize); if (block == NULL && nsize > 0) luaD_throw(L, LUA_ERRMEM); @@ -506,6 +504,9 @@ GCObject* luaM_newgco_(lua_State* L, size_t nsize, uint8_t memcat) if (!FFlag::LuauGcPagedSweep) return (GCObject*)luaM_new_(L, nsize, memcat); + // we need to accommodate space for link for free blocks (freegcolink) + LUAU_ASSERT(nsize >= kGCOLinkOffset + sizeof(void*)); + global_State* g = L->global; int nclass = sizeclass(nsize); @@ -514,9 +515,7 @@ GCObject* luaM_newgco_(lua_State* L, size_t nsize, uint8_t memcat) if (nclass >= 0) { - LUAU_ASSERT(nsize > 8); - - block = luaM_newgcoblock(L, nclass); + block = newgcoblock(L, nclass); } else { @@ -546,7 +545,7 @@ void luaM_free_(lua_State* L, void* block, size_t osize, uint8_t memcat) int oclass = sizeclass(osize); if (oclass >= 0) - luaM_freeblock(L, oclass, block); + freeblock(L, oclass, block); else (*g->frealloc)(L, g->ud, block, osize, 0); @@ -571,7 +570,7 @@ void luaM_freegco_(lua_State* L, GCObject* block, size_t osize, uint8_t memcat, { block->gch.tt = LUA_TNIL; - luaM_freegcoblock(L, oclass, block, page); + freegcoblock(L, oclass, block, page); } else { @@ -596,7 +595,7 @@ void* luaM_realloc_(lua_State* L, void* block, size_t osize, size_t nsize, uint8 // if either block needs to be allocated using a block allocator, we can't use realloc directly if (nclass >= 0 || oclass >= 0) { - result = nclass >= 0 ? luaM_newblock(L, nclass) : (*g->frealloc)(L, g->ud, NULL, 0, nsize); + result = nclass >= 0 ? newblock(L, nclass) : (*g->frealloc)(L, g->ud, NULL, 0, nsize); if (result == NULL && nsize > 0) luaD_throw(L, LUA_ERRMEM); @@ -604,7 +603,7 @@ void* luaM_realloc_(lua_State* L, void* block, size_t osize, size_t nsize, uint8 memcpy(result, block, osize < nsize ? osize : nsize); if (oclass >= 0) - luaM_freeblock(L, oclass, block); + freeblock(L, oclass, block); else (*g->frealloc)(L, g->ud, block, osize, 0); } @@ -659,6 +658,8 @@ void luaM_visitpage(lua_Page* page, void* context, bool (*visitor)(void* context // when true is returned it means that the element was deleted if (visitor(context, page, gco)) { + LUAU_ASSERT(busyBlocks > 0); + // if the last block was removed, page would be removed as well if (--busyBlocks == 0) break; diff --git a/VM/src/lstring.cpp b/VM/src/lstring.cpp index cb22cc23..9bbc43de 100644 --- a/VM/src/lstring.cpp +++ b/VM/src/lstring.cpp @@ -57,7 +57,7 @@ void luaS_resize(lua_State* L, int newsize) { TString* p = tb->hash[i]; while (p) - { /* for each node in the list */ + { /* for each node in the list */ // TODO (FFlagLuauGcPagedSweep): 'next' type will change after removal of the flag and the cast will not be required TString* next = (TString*)p->next; /* save next */ unsigned int h = p->hash; diff --git a/VM/src/lvmexecute.cpp b/VM/src/lvmexecute.cpp index e58ff2a8..cba3670a 100644 --- a/VM/src/lvmexecute.cpp +++ b/VM/src/lvmexecute.cpp @@ -676,14 +676,9 @@ static void luau_execute(lua_State* L) VM_PROTECT_PC(); // set may fail TValue* res = luaH_setstr(L, h, tsvalue(kv)); - - if (res != luaO_nilobject) - { - int cachedslot = gval2slot(h, res); - // save cachedslot to accelerate future lookups; patches currently executing instruction since pc-2 rolls back two pc++ - VM_PATCH_C(pc - 2, cachedslot); - } - + int cachedslot = gval2slot(h, res); + // save cachedslot to accelerate future lookups; patches currently executing instruction since pc-2 rolls back two pc++ + VM_PATCH_C(pc - 2, cachedslot); setobj(L, res, ra); luaC_barriert(L, h, ra); VM_NEXT(); diff --git a/VM/src/lvmload.cpp b/VM/src/lvmload.cpp index cdb276c0..2472cd90 100644 --- a/VM/src/lvmload.cpp +++ b/VM/src/lvmload.cpp @@ -13,7 +13,6 @@ #include -LUAU_FASTFLAGVARIABLE(LuauBytecodeV2Read, false) LUAU_FASTFLAGVARIABLE(LuauBytecodeV2Force, false) // TODO: RAII deallocation doesn't work for longjmp builds if a memory error happens @@ -157,11 +156,12 @@ int luau_load(lua_State* L, const char* chunkname, const char* data, size_t size return 1; } - if (FFlag::LuauBytecodeV2Force ? (version != LBC_VERSION_FUTURE) : FFlag::LuauBytecodeV2Read ? (version != LBC_VERSION && version != LBC_VERSION_FUTURE) : (version != LBC_VERSION)) + if (FFlag::LuauBytecodeV2Force ? (version != LBC_VERSION_FUTURE) : (version != LBC_VERSION && version != LBC_VERSION_FUTURE)) { char chunkid[LUA_IDSIZE]; luaO_chunkid(chunkid, chunkname, LUA_IDSIZE); - lua_pushfstring(L, "%s: bytecode version mismatch (expected %d, got %d)", chunkid, FFlag::LuauBytecodeV2Force ? LBC_VERSION_FUTURE : LBC_VERSION, version); + lua_pushfstring(L, "%s: bytecode version mismatch (expected %d, got %d)", chunkid, + FFlag::LuauBytecodeV2Force ? LBC_VERSION_FUTURE : LBC_VERSION, version); return 1; } @@ -292,7 +292,7 @@ int luau_load(lua_State* L, const char* chunkname, const char* data, size_t size p->p[j] = protos[fid]; } - if (FFlag::LuauBytecodeV2Force || (FFlag::LuauBytecodeV2Read && version == LBC_VERSION_FUTURE)) + if (FFlag::LuauBytecodeV2Force || version == LBC_VERSION_FUTURE) p->linedefined = readVarInt(data, size, offset); else p->linedefined = -1; diff --git a/extern/isocline/.gitignore b/extern/isocline/.gitignore new file mode 100644 index 00000000..470cc813 --- /dev/null +++ b/extern/isocline/.gitignore @@ -0,0 +1,16 @@ +out/ +build/ +dist/ +doc/html/ +.vs/ +.vscode/ +.stack-work/ +.DS_Store +*.user +*.exe +*.hi +*.o +*_stub.h +*.lock +history.txt +isocline.debug.txt \ No newline at end of file diff --git a/extern/isocline/LICENSE b/extern/isocline/LICENSE new file mode 100644 index 00000000..7ac3104b --- /dev/null +++ b/extern/isocline/LICENSE @@ -0,0 +1,21 @@ +MIT License + +Copyright (c) 2021 Daan Leijen + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. diff --git a/extern/isocline/include/isocline.h b/extern/isocline/include/isocline.h new file mode 100644 index 00000000..0d46cf3f --- /dev/null +++ b/extern/isocline/include/isocline.h @@ -0,0 +1,627 @@ +/* ---------------------------------------------------------------------------- + Copyright (c) 2021, Daan Leijen + This is free software; you can redistribute it and/or modify it + under the terms of the MIT License. A copy of the license can be + found in the "LICENSE" file at the root of this distribution. +-----------------------------------------------------------------------------*/ +#pragma once +#ifndef IC_ISOCLINE_H +#define IC_ISOCLINE_H + +#ifdef __cplusplus +extern "C" { +#endif + +#include // size_t +#include // bool +#include // uint32_t +#include // term_vprintf + + +/*! \mainpage +Isocline C API reference. + +Isocline is a pure C library that can be used as an alternative to the GNU readline library. + +See the [Github repository](https://github.com/daanx/isocline#readme) +for general information and building the library. + +Contents: +- \ref readline +- \ref bbcode +- \ref history +- \ref completion +- \ref highlight +- \ref options +- \ref helper +- \ref completex +- \ref term +- \ref async +- \ref alloc +*/ + +/// \defgroup readline Readline +/// The basic readline interface. +/// \{ + +/// Isocline version: 102 = 1.0.2. +#define IC_VERSION (104) + + +/// Read input from the user using rich editing abilities. +/// @param prompt_text The prompt text, can be NULL for the default (""). +/// The displayed prompt becomes `prompt_text` followed by the `prompt_marker` ("> "). +/// @returns the heap allocated input on succes, which should be `free`d by the caller. +/// Returns NULL on error, or if the user typed ctrl+d or ctrl+c. +/// +/// If the standard input (`stdin`) has no editing capability +/// (like a dumb terminal (e.g. `TERM`=`dumb`), running in a debuggen, a pipe or redirected file, etc.) +/// the input is read directly from the input stream up to the +/// next line without editing capability. +/// See also \a ic_set_prompt_marker(), \a ic_style_def() +/// +/// @see ic_set_prompt_marker(), ic_style_def() +char* ic_readline(const char* prompt_text); + +/// \} + + +//-------------------------------------------------------------- +/// \defgroup bbcode Formatted Text +/// Formatted text using [bbcode markup](https://github.com/daanx/isocline#bbcode-format). +/// \{ + +/// Print to the terminal while respection bbcode markup. +/// Any unclosed tags are closed automatically at the end of the print. +/// For example: +/// ``` +/// ic_print("[b]bold, [i]bold and italic[/i], [red]red and bold[/][/b] default."); +/// ic_print("[b]bold[/], [i b]bold and italic[/], [yellow on blue]yellow on blue background"); +/// ic_style_add("em","i color=#888800"); +/// ic_print("[em]emphasis"); +/// ``` +/// Properties that can be assigned are: +/// * `color=` _clr_, `bgcolor=` _clr_: where _clr_ is either a hex value `#`RRGGBB or `#`RGB, a +/// standard HTML color name, or an ANSI palette name, like `ansi-maroon`, `ansi-default`, etc. +/// * `bold`,`italic`,`reverse`,`underline`: can be `on` or `off`. +/// * everything else is a style; all HTML and ANSI color names are also a style (so we can just use `red` +/// instead of `color=red`, or `on red` instead of `bgcolor=red`), and there are +/// the `b`, `i`, `u`, and `r` styles for bold, italic, underline, and reverse. +/// +/// See [here](https://github.com/daanx/isocline#bbcode-format) for a description of the full bbcode format. +void ic_print( const char* s ); + +/// Print with bbcode markup ending with a newline. +/// @see ic_print() +void ic_println( const char* s ); + +/// Print formatted with bbcode markup. +/// @see ic_print() +void ic_printf(const char* fmt, ...); + +/// Print formatted with bbcode markup. +/// @see ic_print +void ic_vprintf(const char* fmt, va_list args); + +/// Define or redefine a style. +/// @param style_name The name of the style. +/// @param fmt The `fmt` string is the content of a tag and can contain +/// other styles. This is very useful to theme the output of a program +/// by assigning standard styles like `em` or `warning` etc. +void ic_style_def( const char* style_name, const char* fmt ); + +/// Start a global style that is only reset when calling a matching ic_style_close(). +void ic_style_open( const char* fmt ); + +/// End a global style. +void ic_style_close(void); + +/// \} + + +//-------------------------------------------------------------- +// History +//-------------------------------------------------------------- +/// \defgroup history History +/// Readline input history. +/// \{ + +/// Enable history. +/// Use a \a NULL filename to not persist the history. Use -1 for max_entries to get the default (200). +void ic_set_history(const char* fname, long max_entries ); + +/// Remove the last entry in the history. +/// The last returned input from ic_readline() is automatically added to the history; this function removes it. +void ic_history_remove_last(void); + +/// Clear the history. +void ic_history_clear(void); + +/// Add an entry to the history +void ic_history_add( const char* entry ); + +/// \} + +//-------------------------------------------------------------- +// Basic Completion +//-------------------------------------------------------------- + +/// \defgroup completion Completion +/// Basic word completion. +/// \{ + +/// A completion environment +struct ic_completion_env_s; + +/// A completion environment +typedef struct ic_completion_env_s ic_completion_env_t; + +/// A completion callback that is called by isocline when tab is pressed. +/// It is passed a completion environment (containing the current input and the current cursor position), +/// the current input up-to the cursor (`prefix`) +/// and the user given argument when the callback was set. +/// When using completion transformers, like `ic_complete_quoted_word` the `prefix` contains the +/// the word to be completed without escape characters or quotes. +typedef void (ic_completer_fun_t)(ic_completion_env_t* cenv, const char* prefix ); + +/// Set the default completion handler. +/// @param completer The completion function +/// @param arg Argument passed to the \a completer. +/// There can only be one default completion function, setting it again disables the previous one. +/// The initial completer use `ic_complete_filename`. +void ic_set_default_completer( ic_completer_fun_t* completer, void* arg); + + +/// In a completion callback (usually from ic_complete_word()), use this function to add a completion. +/// (the completion string is copied by isocline and do not need to be preserved or allocated). +/// +/// Returns `true` if the callback should continue trying to find more possible completions. +/// If `false` is returned, the callback should try to return and not add more completions (for improved latency). +bool ic_add_completion(ic_completion_env_t* cenv, const char* completion); + +/// In a completion callback (usually from ic_complete_word()), use this function to add a completion. +/// The `display` is used to display the completion in the completion menu, and `help` is +/// displayed for hints for example. Both can be `NULL` for the default. +/// (all are copied by isocline and do not need to be preserved or allocated). +/// +/// Returns `true` if the callback should continue trying to find more possible completions. +/// If `false` is returned, the callback should try to return and not add more completions (for improved latency). +bool ic_add_completion_ex( ic_completion_env_t* cenv, const char* completion, const char* display, const char* help ); + +/// In a completion callback (usually from ic_complete_word()), use this function to add completions. +/// The `completions` array should be terminated with a NULL element, and all elements +/// are added as completions if they start with `prefix`. +/// +/// Returns `true` if the callback should continue trying to find more possible completions. +/// If `false` is returned, the callback should try to return and not add more completions (for improved latency). +bool ic_add_completions(ic_completion_env_t* cenv, const char* prefix, const char** completions); + +/// Complete a filename. +/// Complete a filename given a semi-colon separated list of root directories `roots` and +/// semi-colon separated list of possible extensions (excluding directories). +/// If `roots` is NULL, the current directory is the root ("."). +/// If `extensions` is NULL, any extension will match. +/// Each root directory should _not_ end with a directory separator. +/// If a directory is completed, the `dir_separator` is added at the end if it is not `0`. +/// Usually the `dir_separator` is `/` but it can be set to `\\` on Windows systems. +/// For example: +/// ``` +/// /ho --> /home/ +/// /home/.ba --> /home/.bashrc +/// ``` +/// (This already uses ic_complete_quoted_word() so do not call it from inside a word handler). +void ic_complete_filename( ic_completion_env_t* cenv, const char* prefix, char dir_separator, const char* roots, const char* extensions ); + + + +/// Function that returns whether a (utf8) character (of length `len`) is in a certain character class +/// @see ic_char_is_separator() etc. +typedef bool (ic_is_char_class_fun_t)(const char* s, long len); + + +/// Complete a _word_ (i.e. _token_). +/// Calls the user provided function `fun` to complete on the +/// current _word_. Almost all user provided completers should use this function. +/// If `is_word_char` is NULL, the default `&ic_char_is_nonseparator` is used. +/// The `prefix` passed to `fun` is modified to only contain the current word, and +/// any results from `ic_add_completion` are automatically adjusted to replace that part. +/// For example, on the input "hello w", a the user `fun` only gets `w` and can just complete +/// with "world" resulting in "hello world" without needing to consider `delete_before` etc. +/// @see ic_complete_qword() for completing quoted and escaped tokens. +void ic_complete_word(ic_completion_env_t* cenv, const char* prefix, ic_completer_fun_t* fun, ic_is_char_class_fun_t* is_word_char); + + +/// Complete a quoted _word_. +/// Calls the user provided function `fun` to complete while taking +/// care of quotes and escape characters. Almost all user provided completers should use +/// this function. The `prefix` passed to `fun` is modified to be unquoted and unescaped, and +/// any results from `ic_add_completion` are automatically quoted and escaped again. +/// For example, completing `hello world`, the `fun` always just completes `hel` or `hello w` to `hello world`, +/// but depending on user input, it will complete as: +/// ``` +/// hel --> hello\ world +/// hello\ w --> hello\ world +/// hello w --> # no completion, the word is just 'w'> +/// "hel --> "hello world" +/// "hello w --> "hello world" +/// ``` +/// with proper quotes and escapes. +/// If `is_word_char` is NULL, the default `&ic_char_is_nonseparator` is used. +/// @see ic_complete_quoted_word() to customize the word boundary, quotes etc. +void ic_complete_qword( ic_completion_env_t* cenv, const char* prefix, ic_completer_fun_t* fun, ic_is_char_class_fun_t* is_word_char ); + + + +/// Complete a _word_. +/// Calls the user provided function `fun` to complete while taking +/// care of quotes and escape characters. Almost all user provided completers should use this function. +/// The `is_word_char` is a set of characters that are part of a "word". Use NULL for the default (`&ic_char_is_nonseparator`). +/// The `escape_char` is the escaping character, usually `\` but use 0 to not have escape characters. +/// The `quote_chars` define the quotes, use NULL for the default `"\'\""` quotes. +/// @see ic_complete_word() which uses the default values for `non_word_chars`, `quote_chars` and `\` for escape characters. +void ic_complete_qword_ex( ic_completion_env_t* cenv, const char* prefix, ic_completer_fun_t fun, + ic_is_char_class_fun_t* is_word_char, char escape_char, const char* quote_chars ); + +/// \} + +//-------------------------------------------------------------- +/// \defgroup highlight Syntax Highlighting +/// Basic syntax highlighting. +/// \{ + +/// A syntax highlight environment +struct ic_highlight_env_s; +typedef struct ic_highlight_env_s ic_highlight_env_t; + +/// A syntax highlighter callback that is called by readline to syntax highlight user input. +typedef void (ic_highlight_fun_t)(ic_highlight_env_t* henv, const char* input, void* arg); + +/// Set a syntax highlighter. +/// There can only be one highlight function, setting it again disables the previous one. +void ic_set_default_highlighter(ic_highlight_fun_t* highlighter, void* arg); + +/// Set the style of characters starting at position `pos`. +void ic_highlight(ic_highlight_env_t* henv, long pos, long count, const char* style ); + +/// Experimental: Convenience callback for a function that highlights `s` using bbcode's. +/// The returned string should be allocated and is free'd by the caller. +typedef char* (ic_highlight_format_fun_t)(const char* s, void* arg); + +/// Experimental: Convenience function for highlighting with bbcodes. +/// Can be called in a `ic_highlight_fun_t` callback to colorize the `input` using the +/// the provided `formatted` input that is the styled `input` with bbcodes. The +/// content of `formatted` without bbcode tags should match `input` exactly. +void ic_highlight_formatted(ic_highlight_env_t* henv, const char* input, const char* formatted); + +/// \} + +//-------------------------------------------------------------- +// Readline with a specific completer and highlighter +//-------------------------------------------------------------- + +/// \defgroup readline +/// \{ + +/// Read input from the user using rich editing abilities, +/// using a particular completion function and highlighter for this call only. +/// both can be NULL in which case the defaults are used. +/// @see ic_readline(), ic_set_prompt_marker(), ic_set_default_completer(), ic_set_default_highlighter(). +char* ic_readline_ex(const char* prompt_text, ic_completer_fun_t* completer, void* completer_arg, + ic_highlight_fun_t* highlighter, void* highlighter_arg); + +/// \} + + +//-------------------------------------------------------------- +// Options +//-------------------------------------------------------------- + +/// \defgroup options Options +/// \{ + +/// Set a prompt marker and a potential marker for extra lines with multiline input. +/// Pass \a NULL for the `prompt_marker` for the default marker (`"> "`). +/// Pass \a NULL for continuation prompt marker to make it equal to the `prompt_marker`. +void ic_set_prompt_marker( const char* prompt_marker, const char* continuation_prompt_marker ); + +/// Get the current prompt marker. +const char* ic_get_prompt_marker(void); + +/// Get the current continuation prompt marker. +const char* ic_get_continuation_prompt_marker(void); + +/// Disable or enable multi-line input (enabled by default). +/// Returns the previous setting. +bool ic_enable_multiline( bool enable ); + +/// Disable or enable sound (enabled by default). +/// A beep is used when tab cannot find any completion for example. +/// Returns the previous setting. +bool ic_enable_beep( bool enable ); + +/// Disable or enable color output (enabled by default). +/// Returns the previous setting. +bool ic_enable_color( bool enable ); + +/// Disable or enable duplicate entries in the history (disabled by default). +/// Returns the previous setting. +bool ic_enable_history_duplicates( bool enable ); + +/// Disable or enable automatic tab completion after a completion +/// to expand as far as possible if the completions are unique. (disabled by default). +/// Returns the previous setting. +bool ic_enable_auto_tab( bool enable ); + +/// Disable or enable preview of a completion selection (enabled by default) +/// Returns the previous setting. +bool ic_enable_completion_preview( bool enable ); + +/// Disable or enable automatic identation of continuation lines in multiline +/// input so it aligns with the initial prompt. +/// Returns the previous setting. +bool ic_enable_multiline_indent(bool enable); + +/// Disable or enable display of short help messages for history search etc. +/// (full help is always dispayed when pressing F1 regardless of this setting) +/// @returns the previous setting. +bool ic_enable_inline_help(bool enable); + +/// Disable or enable hinting (enabled by default) +/// Shows a hint inline when there is a single possible completion. +/// @returns the previous setting. +bool ic_enable_hint(bool enable); + +/// Set millisecond delay before a hint is displayed. Can be zero. (500ms by default). +long ic_set_hint_delay(long delay_ms); + +/// Disable or enable syntax highlighting (enabled by default). +/// This applies regardless whether a syntax highlighter callback was set (`ic_set_highlighter`) +/// Returns the previous setting. +bool ic_enable_highlight(bool enable); + + +/// Set millisecond delay for reading escape sequences in order to distinguish +/// a lone ESC from the start of a escape sequence. The defaults are 100ms and 10ms, +/// but it may be increased if working with very slow terminals. +void ic_set_tty_esc_delay(long initial_delay_ms, long followup_delay_ms); + +/// Enable highlighting of matching braces (and error highlight unmatched braces).` +bool ic_enable_brace_matching(bool enable); + +/// Set matching brace pairs. +/// Pass \a NULL for the default `"()[]{}"`. +void ic_set_matching_braces(const char* brace_pairs); + +/// Enable automatic brace insertion (enabled by default). +bool ic_enable_brace_insertion(bool enable); + +/// Set matching brace pairs for automatic insertion. +/// Pass \a NULL for the default `()[]{}\"\"''` +void ic_set_insertion_braces(const char* brace_pairs); + +/// \} + + +//-------------------------------------------------------------- +// Advanced Completion +//-------------------------------------------------------------- + +/// \defgroup completex Advanced Completion +/// \{ + +/// Get the raw current input (and cursor position if `cursor` != NULL) for the completion. +/// Usually completer functions should look at their `prefix` though as transformers +/// like `ic_complete_word` may modify the prefix (for example, unescape it). +const char* ic_completion_input( ic_completion_env_t* cenv, long* cursor ); + +/// Get the completion argument passed to `ic_set_completer`. +void* ic_completion_arg( const ic_completion_env_t* cenv ); + +/// Do we have already some completions? +bool ic_has_completions( const ic_completion_env_t* cenv ); + +/// Do we already have enough completions and should we return if possible? (for improved latency) +bool ic_stop_completing( const ic_completion_env_t* cenv); + + +/// Primitive completion, cannot be used with most transformers (like `ic_complete_word` and `ic_complete_qword`). +/// When completed, `delete_before` _bytes_ are deleted before the cursor position, +/// `delete_after` _bytes_ are deleted after the cursor, and finally `completion` is inserted. +/// The `display` is used to display the completion in the completion menu, and `help` is displayed +/// with hinting. Both `display` and `help` can be NULL. +/// (all are copied by isocline and do not need to be preserved or allocated). +/// +/// Returns `true` if the callback should continue trying to find more possible completions. +/// If `false` is returned, the callback should try to return and not add more completions (for improved latency). +bool ic_add_completion_prim( ic_completion_env_t* cenv, const char* completion, + const char* display, const char* help, + long delete_before, long delete_after); + +/// \} + +//-------------------------------------------------------------- +/// \defgroup helper Character Classes. +/// Convenience functions for character classes, highlighting and completion. +/// \{ + +/// Convenience: return the position of a previous code point in a UTF-8 string `s` from postion `pos`. +/// Returns `-1` if `pos <= 0` or `pos > strlen(s)` (or other errors). +long ic_prev_char( const char* s, long pos ); + +/// Convenience: return the position of the next code point in a UTF-8 string `s` from postion `pos`. +/// Returns `-1` if `pos < 0` or `pos >= strlen(s)` (or other errors). +long ic_next_char( const char* s, long pos ); + +/// Convenience: does a string `s` starts with a given `prefix` ? +bool ic_starts_with( const char* s, const char* prefix ); + +/// Convenience: does a string `s` starts with a given `prefix` ignoring (ascii) case? +bool ic_istarts_with( const char* s, const char* prefix ); + + +/// Convenience: character class for whitespace `[ \t\r\n]`. +bool ic_char_is_white(const char* s, long len); + +/// Convenience: character class for non-whitespace `[^ \t\r\n]`. +bool ic_char_is_nonwhite(const char* s, long len); + +/// Convenience: character class for separators. +/// (``[ \t\r\n,.;:/\\(){}\[\]]``.) +/// This is used for word boundaries in isocline. +bool ic_char_is_separator(const char* s, long len); + +/// Convenience: character class for non-separators. +bool ic_char_is_nonseparator(const char* s, long len); + +/// Convenience: character class for letters (`[A-Za-z]` and any unicode > 0x80). +bool ic_char_is_letter(const char* s, long len); + +/// Convenience: character class for digits (`[0-9]`). +bool ic_char_is_digit(const char* s, long len); + +/// Convenience: character class for hexadecimal digits (`[A-Fa-f0-9]`). +bool ic_char_is_hexdigit(const char* s, long len); + +/// Convenience: character class for identifier letters (`[A-Za-z0-9_-]` and any unicode > 0x80). +bool ic_char_is_idletter(const char* s, long len); + +/// Convenience: character class for filename letters (_not in_ " \t\r\n`@$><=;|&\{\}\(\)\[\]]"). +bool ic_char_is_filename_letter(const char* s, long len); + + +/// Convenience: If this is a token start, return the length. Otherwise return 0. +long ic_is_token(const char* s, long pos, ic_is_char_class_fun_t* is_token_char); + +/// Convenience: Does this match the specified token? +/// Ensures not to match prefixes or suffixes, and returns the length of the match (in bytes). +/// E.g. `ic_match_token("function",0,&ic_char_is_letter,"fun")` returns 0. +/// while `ic_match_token("fun x",0,&ic_char_is_letter,"fun"})` returns 3. +long ic_match_token(const char* s, long pos, ic_is_char_class_fun_t* is_token_char, const char* token); + + +/// Convenience: Do any of the specified tokens match? +/// Ensures not to match prefixes or suffixes, and returns the length of the match (in bytes). +/// E.g. `ic_match_any_token("function",0,&ic_char_is_letter,{"fun","func",NULL})` returns 0. +/// while `ic_match_any_token("func x",0,&ic_char_is_letter,{"fun","func",NULL})` returns 4. +long ic_match_any_token(const char* s, long pos, ic_is_char_class_fun_t* is_token_char, const char** tokens); + +/// \} + +//-------------------------------------------------------------- +/// \defgroup term Terminal +/// +/// Experimental: Low level terminal output. +/// Ensures basic ANSI SGR escape sequences are processed +/// in a portable way (e.g. on Windows) +/// \{ + +/// Initialize for terminal output. +/// Call this before using the terminal write functions (`ic_term_write`) +/// Does nothing on most platforms but on Windows it sets the console to UTF8 output and possible +/// enables virtual terminal processing. +void ic_term_init(void); + +/// Call this when done with the terminal functions. +void ic_term_done(void); + +/// Flush the terminal output. +/// (happens automatically on newline characters ('\n') as well). +void ic_term_flush(void); + +/// Write a string to the console (and process CSI escape sequences). +void ic_term_write(const char* s); + +/// Write a string to the console and end with a newline +/// (and process CSI escape sequences). +void ic_term_writeln(const char* s); + +/// Write a formatted string to the console. +/// (and process CSI escape sequences) +void ic_term_writef(const char* fmt, ...); + +/// Write a formatted string to the console. +void ic_term_vwritef(const char* fmt, va_list args); + +/// Set text attributes from a style. +void ic_term_style( const char* style ); + +/// Set text attribute to bold. +void ic_term_bold(bool enable); + +/// Set text attribute to underline. +void ic_term_underline(bool enable); + +/// Set text attribute to italic. +void ic_term_italic(bool enable); + +/// Set text attribute to reverse video. +void ic_term_reverse(bool enable); + +/// Set text attribute to ansi color palette index between 0 and 255 (or 256 for the ANSI "default" color). +/// (auto matched to smaller palette if not supported) +void ic_term_color_ansi(bool foreground, int color); + +/// Set text attribute to 24-bit RGB color (between `0x000000` and `0xFFFFFF`). +/// (auto matched to smaller palette if not supported) +void ic_term_color_rgb(bool foreground, uint32_t color ); + +/// Reset the text attributes. +void ic_term_reset( void ); + +/// Get the palette used by the terminal: +/// This is usually initialized from the COLORTERM environment variable. The +/// possible values of COLORTERM for each palette are given in parenthesis. +/// +/// - 1: monochrome (`monochrome`) +/// - 3: old ANSI terminal with 8 colors, using bold for bright (`8color`/`3bit`) +/// - 4: regular ANSI terminal with 16 colors. (`16color`/`4bit`) +/// - 8: terminal with ANSI 256 color palette. (`256color`/`8bit`) +/// - 24: true-color terminal with full RGB colors. (`truecolor`/`24bit`/`direct`) +int ic_term_get_color_bits( void ); + +/// \} + +//-------------------------------------------------------------- +/// \defgroup async ASync +/// Async support +/// \{ + +/// Thread-safe way to asynchronously unblock a readline. +/// Behaves as if the user pressed the `ctrl-C` character +/// (resulting in returning NULL from `ic_readline`). +/// Returns `true` if the event was successfully delivered. +/// (This may not be supported on all platforms, but it is +/// functional on Linux, macOS and Windows). +bool ic_async_stop(void); + +/// \} + +//-------------------------------------------------------------- +/// \defgroup alloc Custom Allocation +/// Register allocation functions for custom allocators +/// \{ + +typedef void* (ic_malloc_fun_t)( size_t size ); +typedef void* (ic_realloc_fun_t)( void* p, size_t newsize ); +typedef void (ic_free_fun_t)( void* p ); + +/// Initialize with custom allocation functions. +/// This must be called as the first function in a program! +void ic_init_custom_alloc( ic_malloc_fun_t* _malloc, ic_realloc_fun_t* _realloc, ic_free_fun_t* _free ); + +/// Free a potentially custom alloc'd pointer (in particular, the result returned from `ic_readline`) +void ic_free( void* p ); + +/// Allocate using the current memory allocator. +void* ic_malloc(size_t sz); + +/// Duplicate a string using the current memory allocator. +const char* ic_strdup( const char* s ); + +/// \} + +#ifdef __cplusplus +} +#endif + +#endif /// IC_ISOCLINE_H diff --git a/extern/isocline/readme.md b/extern/isocline/readme.md new file mode 100644 index 00000000..1f4709bd --- /dev/null +++ b/extern/isocline/readme.md @@ -0,0 +1,460 @@ + + + + +# Isocline: a portable readline alternative. + +Isocline is a pure C library that can be used as an alternative to the GNU readline library (latest release v1.0.9, 2022-01-15). + +- Small: less than 8k lines and can be compiled as a single C file without + any dependencies or configuration (e.g. `gcc -c src/isocline.c`). + +- Portable: works on Unix, Windows, and macOS, and uses a minimal + subset of ANSI escape sequences. + +- Features: extensive multi-line editing mode (`shift-tab`), (24-bit) color, history, completion, unicode, + undo/redo, incremental history search, inline hints, syntax highlighting, brace matching, + closing brace insertion, auto indentation, graceful fallback, support for custom allocators, etc. + +- License: MIT. + +- Comes with a Haskell binding ([`System.Console.Isocline`][hdoc]. + +Enjoy, + Daan + + + +# Demo + +![recording](doc/record-macos.svg) + +Shows in order: unicode, syntax highlighting, brace matching, jump to matching brace, auto indent, multiline editing, 24-bit colors, inline hinting, filename completion, and incremental history search. +(screen capture was made with [termtosvg] by Nicolas Bedos) + +# Usage + +Include the isocline header in your C or C++ source: +```C +#include +``` + +and call `ic_readline` to get user input with rich editing abilities: +```C +char* input; +while( (input = ic_readline("prompt")) != NULL ) { // ctrl+d/c or errors return NULL + printf("you typed:\n%s\n", input); // use the input + free(input); +} +``` + +See the [example] for a full example with completion, syntax highligting, history, etc. + +# Run the Example + +You can compile and run the [example] as: +``` +$ gcc -o example -Iinclude test/example.c src/isocline.c +$ ./example +``` + +or, the Haskell [example][HaskellExample]: +``` +$ ghc -ihaskell test/Example.hs src/isocline.c +$ ./test/Example +``` + + +# Editing with Isocline + +Isocline tries to be as compatible as possible with standard [GNU Readline] key bindings. + +### Overview: +```apl + home/ctrl-a cursor end/ctrl-e + ┌─────────────────┼───────────────┐ (navigate) + │ ctrl-left │ ctrl-right │ + │ ┌───────┼──────┐ │ ctrl-r : search history + ▼ ▼ ▼ ▼ ▼ tab : complete word + prompt> it is the quintessential language shift-tab: insert new line + ▲ ▲ ▲ ▲ esc : delete input, done + │ └──────────────┘ │ ctrl-z : undo + │ alt-backsp alt-d │ + └─────────────────────────────────┘ (delete) + ctrl-u ctrl-k +``` + +Note: on macOS, the meta (alt) key is not directly available in most terminals. +Terminal/iTerm2 users can activate the meta key through +`Terminal` → `Preferences` → `Settings` → `Use option as meta key`. + +### Key Bindings + +These are also shown when pressing `F1` on a Isocline prompt. We use `^` as a shorthand for `ctrl-`: + +| Navigation | | +|-------------------|-------------------------------------------------| +| `left`,`^b` | go one character to the left | +| `right`,`^f ` | go one character to the right | +| `up ` | go one row up, or back in the history | +| `down ` | go one row down, or forward in the history | +| `^left ` | go to the start of the previous word | +| `^right ` | go to the end the current word | +| `home`,`^a ` | go to the start of the current line | +| `end`,`^e ` | go to the end of the current line | +| `pgup`,`^home ` | go to the start of the current input | +| `pgdn`,`^end ` | go to the end of the current input | +| `alt-m ` | jump to matching brace | +| `^p ` | go back in the history | +| `^n ` | go forward in the history | +| `^r`,`^s ` | search the history starting with the current word | + + +| Deletion | | +|-------------------|-------------------------------------------------| +| `del`,`^d ` | delete the current character | +| `backsp`,`^h ` | delete the previous character | +| `^w ` | delete to preceding white space | +| `alt-backsp ` | delete to the start of the current word | +| `alt-d ` | delete to the end of the current word | +| `^u ` | delete to the start of the current line | +| `^k ` | delete to the end of the current line | +| `esc ` | delete the current input, or done with empty input | + + +| Editing | | +|-------------------|-------------------------------------------------| +| `enter ` | accept current input | +| `^enter`,`^j`,`shift-tab` | create a new line for multi-line input | +| `^l ` | clear screen | +| `^t ` | swap with previous character (move character backward) | +| `^z`,`^_ ` | undo | +| `^y ` | redo | +| `tab ` | try to complete the current input | + + +| Completion menu | | +|-------------------|-------------------------------------------------| +| `enter`,`left` | use the currently selected completion | +| `1` - `9` | use completion N from the menu | +| `tab, down ` | select the next completion | +| `shift-tab, up` | select the previous completion | +| `esc ` | exit menu without completing | +| `pgdn`,`^enter`,`^j` | show all further possible completions | + + +| Incremental history search | | +|-------------------|-------------------------------------------------| +| `enter ` | use the currently found history entry | +| `backsp`,`^z ` | go back to the previous match (undo) | +| `tab`,`^r`,`up` | find the next match | +| `shift-tab`,`^s`,`down` | find an earlier match | +| `esc ` | exit search | + + +# Build the Library + +### Build as a Single Source + +Copy the sources (in `include` and `src`) into your project, or add the library as a [submodule]: +``` +$ git submodule add https://github.com/daanx/isocline +``` +and add `isocline/src/isocline.c` to your build rules -- no configuration is needed. + +### Build with CMake + +Clone the repository and run cmake to build a static library (`.a`/`.lib`): +``` +$ git clone https://github.com/daanx/isocline +$ cd isocline +$ mkdir -p build/release +$ cd build/release +$ cmake ../.. +$ cmake --build . +``` +This builds a static library `libisocline.a` (or `isocline.lib` on Windows) +and the example program: +``` +$ ./example +``` + +### Build the Haskell Library + +See the Haskell [readme][Haskell] for instructions to build and use the Haskell library. + + +# API Reference + +* See the [C API reference][docapi] and the [example] for example usage of history, completion, etc. + +* See the [Haskell API reference][hdoc] on Hackage and the Haskell [example][HaskellExample]. + + +# Motivation + +Isocline was created for use in the [Koka] interactive compiler. +This required: pure C (no dependency on a C++ runtime or other libraries), +portable (across Linux, macOS, and Windows), unicode support, +a BSD-style license, and good functionality for completion and multi-line editing. + +Some other excellent libraries that we considered: +[GNU readline], +[editline](https://github.com/troglobit/editline), +[linenoise](https://github.com/antirez/linenoise), +[replxx](https://github.com/AmokHuginnsson/replxx), and +[Haskeline](https://github.com/judah/haskeline). + + +# Formatted Output + +Isocline also exposes functions for rich terminal output +as `ic_print` (and `ic_println` and `ic_printf`). +Inspired by the (Python) [Rich][RichBBcode] library, +this supports a form of [bbcode]'s to format the output: +```c +ic_println( "[b]bold [red]and red[/red][/b]" ); +``` +Each print automatically closes any open tags that were +not yet closed. Also, you can use a general close +tag as `[/]` to close the innermost tag, so the +following print is equivalent to the earlier one: +```c +ic_println( "[b]bold [red]and red[/]" ); +``` +There can be multiple styles in one tag +(where the first name is used for the closing tag): +```c +ic_println( "[u #FFD700]underlined gold[/]" ); +``` + +Sometimes, you need to display arbitrary messages +that may contain sequences that you would not like +to be interpreted as bbcode tags. One way to do +this is the `[!`_tag_`]` which ignores formatting +up to a close tag of the form `[/`_tag_`]`. +```c +ic_printf( "[red]red? [!pre]%s[/pre].\n", "[blue]not blue!" ); +``` + +Predefined styles include `b` (bold), +`u` (underline), `i` (italic), and `r` (reverse video), but +you can (re)define any style yourself as: +```c +ic_style_def("warning", "crimson u"); +``` + +and use them like any builtin style or property: +```c +ic_println( "[warning]this is a warning![/]" ); +``` +which is great for adding themes to your application. + +Each `ic_print` function always closes any unclosed tags automatically. +To open a style persistently, use `ic_style_open` with a matching +`ic_style_close` which scopes over any `ic_print` statements in between. +```c +ic_style_open("warning"); +ic_println("[b]crimson underlined and bold[/]"); +ic_style_close(); +``` + +# Advanced + + +## BBCode Format + +An open tag can have multiple white space separated +entries that are +either a _style name_, or a primitive _property_[`=`_value_]. + +### Styles + +Isocline provides the following builtin styles as property shorthands: +`b` (bold), `u` (underline), `i` (italic), `r` (reverse video), +and some builtin styles for syntax highlighting: +`keyword`, `control` (control-flow keywords), `string`, +`comment`, `number`, `type`, `constant`. + +Predefined styles used by Isocline itself are: + +- `ic-prompt`: prompt style, e.g. `ic_style_def("ic-prompt", "yellow on blue")`. +- `ic-info`: information (like the numbers in a completion menu). +- `ic-diminish`: dim text (used for example in history search). +- `ic-emphasis`: emphasized text (also used in history search). +- `ic-hint`: color of an inline hint. +- `ic-error`: error color (like an unmatched brace). +- `ic-bracematch`: color of matching parenthesis. + +### Properties + +Boolean properties are by default `on`: + +- `bold` [`=`(`on`|`off`)] +- `italic` [`=`(`on`|`off`)] +- `underline` [`=`(`on`|`off`)] +- `reverse` [`=`(`on`|`off`)] + +Color properties can be assigned a _color_: + +- `color=`_color_ +- `bgcolor=`_color_ +- _color_: equivalent to `color=`_color_. +- `on` _color_: equivalent to `bgcolor=`_color_. + +A color value can be specified in many ways: + +- any standard HTML [color name][htmlcolors]. +- any of the 16 standard ANSI [color names][ansicolors] by prefixing `ansi-` + (like `ansi-black` or `ansi-maroon`). + The actual color value of these depend on the a terminal theme. +- `#`_rrggbb_ or `#`_rgb_ for a specific 24-bit color. +- `ansi-color=`_idx_: where 0 <= _idx_ <= 256 specifies an entry in the + standard ANSI 256 [color palette][ansicolor256], where 256 is used for the ANSI + default color. + + +## Environment Variables + +- `NO_COLOR`: if present no colors are displayed. +- `CLICOLOR=1`: if set, the `LSCOLORS` or `LS_COLORS` environment variables are used to colorize + filename completions. +- `COLORTERM=`(`truecolor`|`256color`|`16color`|`8color`|`monochrome`): enable a certain color palette, see the next section. +- `TERM`: used on some systems to determine the color + +## Colors + +Isocline supports 24-bit colors and any RGB colors are automatically +mapped to a reduced palette on older terminals if these do not +support true color. Detection of full color support +is not always possible to do automatically and you can +set the `COLORTERM` environment variable expicitly to force Isocline to use +a specific palette: +- `COLORTERM=truecolor`: use 24-bit colors. + +- `COLORTERM=256color`: use the ANSI 256 color palette. + +- `COLORTERM=16color` : use the regular ANSI 16 color + palette (8 normal and 8 bright colors). + +- `COLORTERM=8color`: use bold for bright colors. +- `COLORTERM=monochrome`: use no color. + +The above screenshots are made with the +[`test_colors.c`](https://github.com/daanx/isocline/blob/main/test/test_colors.c) program. You can test your own +terminal as: +``` +$ gcc -o test_colors -Iinclude test/test_colors.c src/isocline.c +$ ./test_colors +$ COLORTERM=truecolor ./test_colors +$ COLORTERM=16color ./test_colors +``` + +## ANSI Escape Sequences + +Isocline uses just few ANSI escape sequences that are widely +supported: +- `ESC[`_n_`A`, `ESC[`_n_`B`, `ESC[`_n_`C`, and `ESC[`_n_`D`, + for moving the cursor _n_ places up, down, right, and left. +- `ESC[K` to clear the line from the cursor. +- `ESC[`_n_`m` for colors, with _n_ one of: 0 (reset), 1,22 (bold), 3,23 (italic), + 4,24 (underline), 7,27 (reverse), 30-37,40-47,90-97,100-107 (color), + and 39,49 (select default color). +- `ESC[38;5;`_n_`m`, `ESC[48;5;`_n_`m`, `ESC[38;2;`_r_`;`_g_`;`_b_`m`, `ESC[48;2;`_r_`;`_g_`;`_b_`m`: + on terminals that support it, select + entry _n_ from the + 256 color ANSI palette (used with `XTERM=xterm-256color` for example), or directly specify + any 24-bit _rgb_ color (used with `COLORTERM=truecolor`) for the foreground or background. + +On Windows the above functionality is implemented using the Windows console API +(except if running in the new Windows Terminal which supports these escape +sequences natively). + +## Async and Threads + +Isocline is _not_ thread-safe and `ic_readline`_xxx_ and `ic_print`_xxx_ should +be used from one thread only. + +The best way to use `ic_readline` asynchronously is +to run it in a (blocking) dedicated thread and deliver +results from there to the async event loop. Isocline has the +```C +bool ic_async_stop(void) +``` +function that is thread-safe and can deliver an +asynchronous event to Isocline that unblocks a current +`ic_readline` and makes it behave as if the user pressed +`ctrl-c` (which returns NULL from the read line call). + +## Color Mapping + +To map full RGB colors to an ANSI 256 or 16-color palette +Isocline finds a palette color with the minimal "color distance" to +the original color. There are various +ways of calculating this: one way is to take the euclidean distance +in the sRGB space (_simple-rgb_), a slightly better way is to +take a weighted distance where the weight distribution is adjusted +according to how big the red component is ([redmean](https://en.wikipedia.org/wiki/Color_difference), +denoted as _delta-rgb_ in the figure), +this is used by Isocline), +and finally, we can first translate into a perceptually uniform color space +(CIElab) and calculate the distance there using the [CIEDE2000](https://en.wikipedia.org/wiki/Color_difference) +algorithm (_ciede2000_). Here are these three methods compared on +some colors: + +![color space comparison](doc/color/colorspace-map.png) + +Each top row is the true 24-bit RGB color. Surprisingly, +the sophisticated CIEDE2000 distance seems less good here compared to the +simpler methods (as in the upper left block for example) +(perhaps because this algorithm was created to find close +perceptual colors in images where lightness differences may be given +less weight?). CIEDE2000 also leads to more "outliers", for example as seen +in column 5. Given these results, Isocline uses _redmean_ for +color mapping. We also add a gray correction that makes it less +likely to substitute a color for a gray value (and the other way +around). + + +## Possible Future Extensions + +- Vi key bindings. +- kill buffer. +- make the `ic_print`_xxx_ functions thread-safe. +- extended low-level terminal functions. +- status and progress bars. +- prompt variants: confirm, etc. +- ... + +Contact me if you are interested in doing any of these :-) + + +# Releases + +* `2022-01-15`: v1.0.9: fix missing `ic_completion_arg` (issue #6), + fix null ptr check in ic_print (issue #7), fix crash when using /dev/null as both input and output. +* `2021-09-05`: v1.0.5: use our own wcwidth for consistency; + thanks to Hans-Georg Breunig for helping with testing on NetBSD. +* `2021-08-28`: v1.0.4: fix color query on Ubuntu/Gnome +* `2021-08-27`: v1.0.3: fix duplicates in completions +* `2021-08-23`: v1.0.2: fix windows eol wrapping +* `2021-08-21`: v1.0.1: fix line-buffering +* `2021-08-20`: v1.0.0: initial release + + + +[GNU readline]: https://tiswww.case.edu/php/chet/readline/rltop.html +[koka]: http://www.koka-lang.org +[submodule]: https://git-scm.com/book/en/v2/Git-Tools-Submodules +[Haskell]: https://github.com/daanx/isocline/tree/main/haskell +[HaskellExample]: https://github.com/daanx/isocline/blob/main/test/Example.hs +[example]: https://github.com/daanx/isocline/blob/main/test/example.c +[termtosvg]: https://github.com/nbedos/termtosvg +[Rich]: https://github.com/willmcgugan/rich +[RichBBcode]: https://rich.readthedocs.io/en/latest/markup.html +[bbcode]: https://en.wikipedia.org/wiki/BBCode +[htmlcolors]: https://en.wikipedia.org/wiki/Web_colors#HTML_color_names +[ansicolors]: https://en.wikipedia.org/wiki/Web_colors#Basic_colors +[ansicolor256]: https://en.wikipedia.org/wiki/ANSI_escape_code#8-bit +[docapi]: https://daanx.github.io/isocline +[hdoc]: https://hackage.haskell.org/package/isocline/docs/System-Console-Isocline.html diff --git a/extern/isocline/src/attr.c b/extern/isocline/src/attr.c new file mode 100644 index 00000000..b5ad78f8 --- /dev/null +++ b/extern/isocline/src/attr.c @@ -0,0 +1,294 @@ +/* ---------------------------------------------------------------------------- + Copyright (c) 2021, Daan Leijen + This is free software; you can redistribute it and/or modify it + under the terms of the MIT License. A copy of the license can be + found in the "LICENSE" file at the root of this distribution. +-----------------------------------------------------------------------------*/ +#include + +#include "common.h" +#include "stringbuf.h" // str_next_ofs +#include "attr.h" +#include "term.h" // color_from_ansi256 + +//------------------------------------------------------------- +// Attributes +//------------------------------------------------------------- + +ic_private attr_t attr_none(void) { + attr_t attr; + attr.value = 0; + return attr; +} + +ic_private attr_t attr_default(void) { + attr_t attr = attr_none(); + attr.x.color = IC_ANSI_DEFAULT; + attr.x.bgcolor = IC_ANSI_DEFAULT; + attr.x.bold = IC_OFF; + attr.x.underline = IC_OFF; + attr.x.reverse = IC_OFF; + attr.x.italic = IC_OFF; + return attr; +} + +ic_private bool attr_is_none(attr_t attr) { + return (attr.value == 0); +} + +ic_private bool attr_is_eq(attr_t attr1, attr_t attr2) { + return (attr1.value == attr2.value); +} + +ic_private attr_t attr_from_color( ic_color_t color ) { + attr_t attr = attr_none(); + attr.x.color = color; + return attr; +} + + +ic_private attr_t attr_update_with( attr_t oldattr, attr_t newattr ) { + attr_t attr = oldattr; + if (newattr.x.color != IC_COLOR_NONE) { attr.x.color = newattr.x.color; } + if (newattr.x.bgcolor != IC_COLOR_NONE) { attr.x.bgcolor = newattr.x.bgcolor; } + if (newattr.x.bold != IC_NONE) { attr.x.bold = newattr.x.bold; } + if (newattr.x.italic != IC_NONE) { attr.x.italic = newattr.x.italic; } + if (newattr.x.reverse != IC_NONE) { attr.x.reverse = newattr.x.reverse; } + if (newattr.x.underline != IC_NONE) { attr.x.underline = newattr.x.underline; } + return attr; +} + +static bool sgr_is_digit(char c) { + return (c >= '0' && c <= '9'); +} + +static bool sgr_is_sep( char c ) { + return (c==';' || c==':'); +} + +static bool sgr_next_par(const char* s, ssize_t* pi, ssize_t* par) { + const ssize_t i = *pi; + ssize_t n = 0; + while( sgr_is_digit(s[i+n])) { + n++; + } + if (n==0) { + *par = 0; + return true; + } + else { + *pi = i+n; + return ic_atoz(s+i, par); + } +} + +static bool sgr_next_par3(const char* s, ssize_t* pi, ssize_t* p1, ssize_t* p2, ssize_t* p3) { + bool ok = false; + ssize_t i = *pi; + if (sgr_next_par(s,&i,p1) && sgr_is_sep(s[i])) { + i++; + if (sgr_next_par(s,&i,p2) && sgr_is_sep(s[i])) { + i++; + if (sgr_next_par(s,&i,p3)) { + ok = true; + }; + } + } + *pi = i; + return ok; +} + +ic_private attr_t attr_from_sgr( const char* s, ssize_t len) { + attr_t attr = attr_none(); + for( ssize_t i = 0; i < len && s[i] != 0; i++) { + ssize_t cmd = 0; + if (!sgr_next_par(s,&i,&cmd)) continue; + switch(cmd) { + case 0: attr = attr_default(); break; + case 1: attr.x.bold = IC_ON; break; + case 3: attr.x.italic = IC_ON; break; + case 4: attr.x.underline = IC_ON; break; + case 7: attr.x.reverse = IC_ON; break; + case 22: attr.x.bold = IC_OFF; break; + case 23: attr.x.italic = IC_OFF; break; + case 24: attr.x.underline = IC_OFF; break; + case 27: attr.x.reverse = IC_OFF; break; + case 39: attr.x.color = IC_ANSI_DEFAULT; break; + case 49: attr.x.bgcolor = IC_ANSI_DEFAULT; break; + default: { + if (cmd >= 30 && cmd <= 37) { + attr.x.color = IC_ANSI_BLACK + (unsigned)(cmd - 30); + } + else if (cmd >= 40 && cmd <= 47) { + attr.x.bgcolor = IC_ANSI_BLACK + (unsigned)(cmd - 40); + } + else if (cmd >= 90 && cmd <= 97) { + attr.x.color = IC_ANSI_DARKGRAY + (unsigned)(cmd - 90); + } + else if (cmd >= 100 && cmd <= 107) { + attr.x.bgcolor = IC_ANSI_DARKGRAY + (unsigned)(cmd - 100); + } + else if ((cmd == 38 || cmd == 48) && sgr_is_sep(s[i])) { + // non-associative SGR :-( + ssize_t par = 0; + i++; + if (sgr_next_par(s, &i, &par)) { + if (par==5 && sgr_is_sep(s[i])) { + // ansi 256 index + i++; + if (sgr_next_par(s, &i, &par) && par >= 0 && par <= 0xFF) { + ic_color_t color = color_from_ansi256(par); + if (cmd==38) { attr.x.color = color; } + else { attr.x.bgcolor = color; } + } + } + else if (par == 2 && sgr_is_sep(s[i])) { + // rgb value + i++; + ssize_t r,g,b; + if (sgr_next_par3(s, &i, &r,&g,&b)) { + ic_color_t color = ic_rgbx(r,g,b); + if (cmd==38) { attr.x.color = color; } + else { attr.x.bgcolor = color; } + } + } + } + } + else { + debug_msg("attr: unknow ANSI SGR code: %zd\n", cmd ); + } + } + } + } + return attr; +} + +ic_private attr_t attr_from_esc_sgr( const char* s, ssize_t len) { + if (len <= 2 || s[0] != '\x1B' || s[1] != '[' || s[len-1] != 'm') return attr_none(); + return attr_from_sgr(s+2, len-2); +} + + +//------------------------------------------------------------- +// Attribute buffer +//------------------------------------------------------------- +struct attrbuf_s { + attr_t* attrs; + ssize_t capacity; + ssize_t count; + alloc_t* mem; +}; + +static bool attrbuf_ensure_capacity( attrbuf_t* ab, ssize_t needed ) { + if (needed <= ab->capacity) return true; + ssize_t newcap = (ab->capacity <= 0 ? 240 : (ab->capacity > 1000 ? ab->capacity + 1000 : 2*ab->capacity)); + if (needed > newcap) { newcap = needed; } + attr_t* newattrs = mem_realloc_tp( ab->mem, attr_t, ab->attrs, newcap ); + if (newattrs == NULL) return false; + ab->attrs = newattrs; + ab->capacity = newcap; + assert(needed <= ab->capacity); + return true; +} + +static bool attrbuf_ensure_extra( attrbuf_t* ab, ssize_t extra ) { + const ssize_t needed = ab->count + extra; + return attrbuf_ensure_capacity( ab, needed ); +} + + +ic_private attrbuf_t* attrbuf_new( alloc_t* mem ) { + attrbuf_t* ab = mem_zalloc_tp(mem,attrbuf_t); + if (ab == NULL) return NULL; + ab->mem = mem; + attrbuf_ensure_extra(ab,1); + return ab; +} + +ic_private void attrbuf_free( attrbuf_t* ab ) { + if (ab==NULL) return; + mem_free(ab->mem, ab->attrs); + mem_free(ab->mem, ab); +} + +ic_private void attrbuf_clear(attrbuf_t* ab) { + if (ab == NULL) return; + ab->count = 0; +} + +ic_private ssize_t attrbuf_len( attrbuf_t* ab ) { + return (ab==NULL ? 0 : ab->count); +} + +ic_private const attr_t* attrbuf_attrs( attrbuf_t* ab, ssize_t expected_len ) { + assert(expected_len <= ab->count ); + // expand if needed + if (ab->count < expected_len) { + if (!attrbuf_ensure_capacity(ab,expected_len)) return NULL; + for(ssize_t i = ab->count; i < expected_len; i++) { + ab->attrs[i] = attr_none(); + } + ab->count = expected_len; + } + return ab->attrs; +} + + + +static void attrbuf_update_set_at( attrbuf_t* ab, ssize_t pos, ssize_t count, attr_t attr, bool update ) { + const ssize_t end = pos + count; + if (!attrbuf_ensure_capacity(ab, end)) return; + ssize_t i; + // initialize if end is beyond the count (todo: avoid duplicate init and set if update==false?) + if (ab->count < end) { + for(i = ab->count; i < end; i++) { + ab->attrs[i] = attr_none(); + } + ab->count = end; + } + // fill pos to end with attr + for(i = pos; i < end; i++) { + ab->attrs[i] = (update ? attr_update_with(ab->attrs[i],attr) : attr); + } +} + +ic_private void attrbuf_set_at( attrbuf_t* ab, ssize_t pos, ssize_t count, attr_t attr ) { + attrbuf_update_set_at(ab, pos, count, attr, false); +} + +ic_private void attrbuf_update_at( attrbuf_t* ab, ssize_t pos, ssize_t count, attr_t attr ) { + attrbuf_update_set_at(ab, pos, count, attr, true); +} + +ic_private void attrbuf_insert_at( attrbuf_t* ab, ssize_t pos, ssize_t count, attr_t attr ) { + if (pos < 0 || pos > ab->count || count <= 0) return; + if (!attrbuf_ensure_extra(ab,count)) return; + ic_memmove( ab->attrs + pos + count, ab->attrs + pos, (ab->count - pos)*ssizeof(attr_t) ); + ab->count += count; + attrbuf_set_at( ab, pos, count, attr ); +} + + +// note: must allow ab == NULL! +ic_private ssize_t attrbuf_append_n( stringbuf_t* sb, attrbuf_t* ab, const char* s, ssize_t len, attr_t attr ) { + if (s == NULL || len == 0) return sbuf_len(sb); + if (ab != NULL) { + if (!attrbuf_ensure_extra(ab,len)) return sbuf_len(sb); + attrbuf_set_at(ab, ab->count, len, attr); + } + return sbuf_append_n(sb,s,len); +} + +ic_private attr_t attrbuf_attr_at( attrbuf_t* ab, ssize_t pos ) { + if (ab==NULL || pos < 0 || pos > ab->count) return attr_none(); + return ab->attrs[pos]; +} + +ic_private void attrbuf_delete_at( attrbuf_t* ab, ssize_t pos, ssize_t count ) { + if (ab==NULL || pos < 0 || pos > ab->count) return; + if (pos + count > ab->count) { count = ab->count - pos; } + if (count == 0) return; + assert(pos + count <= ab->count); + ic_memmove( ab->attrs + pos, ab->attrs + pos + count, ab->count - (pos + count) ); + ab->count -= count; +} diff --git a/extern/isocline/src/attr.h b/extern/isocline/src/attr.h new file mode 100644 index 00000000..8f37d050 --- /dev/null +++ b/extern/isocline/src/attr.h @@ -0,0 +1,70 @@ +/* ---------------------------------------------------------------------------- + Copyright (c) 2021, Daan Leijen + This is free software; you can redistribute it and/or modify it + under the terms of the MIT License. A copy of the license can be + found in the "LICENSE" file at the root of this distribution. +-----------------------------------------------------------------------------*/ +#pragma once +#ifndef IC_ATTR_H +#define IC_ATTR_H + +#include "common.h" +#include "stringbuf.h" + +//------------------------------------------------------------- +// text attributes +//------------------------------------------------------------- + +#define IC_ON (1) +#define IC_OFF (-1) +#define IC_NONE (0) + +// try to fit in 64 bits +// note: order is important for some compilers +// note: each color can actually be 25 bits +typedef union attr_s { + struct { + unsigned int color:28; + signed int bold:2; + signed int reverse:2; + unsigned int bgcolor:28; + signed int underline:2; + signed int italic:2; + } x; + uint64_t value; +} attr_t; + +ic_private attr_t attr_none(void); +ic_private attr_t attr_default(void); +ic_private attr_t attr_from_color( ic_color_t color ); + +ic_private bool attr_is_none(attr_t attr); +ic_private bool attr_is_eq(attr_t attr1, attr_t attr2); + +ic_private attr_t attr_update_with( attr_t attr, attr_t newattr ); + +ic_private attr_t attr_from_sgr( const char* s, ssize_t len); +ic_private attr_t attr_from_esc_sgr( const char* s, ssize_t len); + +//------------------------------------------------------------- +// attribute buffer used for rich rendering +//------------------------------------------------------------- + +struct attrbuf_s; +typedef struct attrbuf_s attrbuf_t; + +ic_private attrbuf_t* attrbuf_new( alloc_t* mem ); +ic_private void attrbuf_free( attrbuf_t* ab ); // ab can be NULL +ic_private void attrbuf_clear( attrbuf_t* ab ); // ab can be NULL +ic_private ssize_t attrbuf_len( attrbuf_t* ab); // ab can be NULL +ic_private const attr_t* attrbuf_attrs( attrbuf_t* ab, ssize_t expected_len ); +ic_private ssize_t attrbuf_append_n( stringbuf_t* sb, attrbuf_t* ab, const char* s, ssize_t len, attr_t attr ); + +ic_private void attrbuf_set_at( attrbuf_t* ab, ssize_t pos, ssize_t count, attr_t attr ); +ic_private void attrbuf_update_at( attrbuf_t* ab, ssize_t pos, ssize_t count, attr_t attr ); +ic_private void attrbuf_insert_at( attrbuf_t* ab, ssize_t pos, ssize_t count, attr_t attr ); + +ic_private attr_t attrbuf_attr_at( attrbuf_t* ab, ssize_t pos ); +ic_private void attrbuf_delete_at( attrbuf_t* ab, ssize_t pos, ssize_t count ); + +#endif // IC_ATTR_H diff --git a/extern/isocline/src/bbcode.c b/extern/isocline/src/bbcode.c new file mode 100644 index 00000000..4d11ac38 --- /dev/null +++ b/extern/isocline/src/bbcode.c @@ -0,0 +1,842 @@ +/* ---------------------------------------------------------------------------- + Copyright (c) 2021, Daan Leijen + This is free software; you can redistribute it and/or modify it + under the terms of the MIT License. A copy of the license can be + found in the "LICENSE" file at the root of this distribution. +-----------------------------------------------------------------------------*/ +#include +#include +#include +#include + +#include "common.h" +#include "attr.h" +#include "term.h" +#include "bbcode.h" + +//------------------------------------------------------------- +// HTML color table +//------------------------------------------------------------- + +#include "bbcode_colors.c" + +//------------------------------------------------------------- +// Types +//------------------------------------------------------------- + +typedef struct style_s { + const char* name; // name of the style + attr_t attr; // attribute to apply +} style_t; + +typedef enum align_e { + IC_ALIGN_LEFT, + IC_ALIGN_CENTER, + IC_ALIGN_RIGHT +} align_t; + +typedef struct width_s { + ssize_t w; // > 0 + align_t align; + bool dots; // "..." (e.g. "sentence...") + char fill; // " " (e.g. "hello ") +} width_t; + +typedef struct tag_s { + const char* name; // tag open name + attr_t attr; // the saved attribute before applying the style + width_t width; // start sequence of at most "width" columns + ssize_t pos; // start position in the output (used for width restriction) +} tag_t; + + +static void tag_init(tag_t* tag) { + memset(tag,0,sizeof(*tag)); +} + +struct bbcode_s { + tag_t* tags; // stack of tags; one entry for each open tag + ssize_t tags_capacity; + ssize_t tags_nesting; + style_t* styles; // list of used defined styles + ssize_t styles_capacity; + ssize_t styles_count; + term_t* term; // terminal + alloc_t* mem; // allocator + // caches + stringbuf_t* out; // print buffer + attrbuf_t* out_attrs; + stringbuf_t* vout; // vprintf buffer +}; + + +//------------------------------------------------------------- +// Create, helpers +//------------------------------------------------------------- + +ic_private bbcode_t* bbcode_new( alloc_t* mem, term_t* term ) { + bbcode_t* bb = mem_zalloc_tp(mem,bbcode_t); + if (bb==NULL) return NULL; + bb->mem = mem; + bb->term = term; + bb->out = sbuf_new(mem); + bb->out_attrs = attrbuf_new(mem); + bb->vout = sbuf_new(mem); + return bb; +} + +ic_private void bbcode_free( bbcode_t* bb ) { + for(ssize_t i = 0; i < bb->styles_count; i++) { + mem_free(bb->mem, bb->styles[i].name); + } + mem_free(bb->mem, bb->tags); + mem_free(bb->mem, bb->styles); + sbuf_free(bb->vout); + sbuf_free(bb->out); + attrbuf_free(bb->out_attrs); + mem_free(bb->mem, bb); +} + +ic_private void bbcode_style_add( bbcode_t* bb, const char* style_name, attr_t attr ) { + if (bb->styles_count >= bb->styles_capacity) { + ssize_t newlen = bb->styles_capacity + 32; + style_t* p = mem_realloc_tp( bb->mem, style_t, bb->styles, newlen ); + if (p == NULL) return; + bb->styles = p; + bb->styles_capacity = newlen; + } + assert(bb->styles_count < bb->styles_capacity); + bb->styles[bb->styles_count].name = mem_strdup( bb->mem, style_name ); + bb->styles[bb->styles_count].attr = attr; + bb->styles_count++; +} + +static ssize_t bbcode_tag_push( bbcode_t* bb, const tag_t* tag ) { + if (bb->tags_nesting >= bb->tags_capacity) { + ssize_t newcap = bb->tags_capacity + 32; + tag_t* p = mem_realloc_tp( bb->mem, tag_t, bb->tags, newcap ); + if (p == NULL) return -1; + bb->tags = p; + bb->tags_capacity = newcap; + } + assert(bb->tags_nesting < bb->tags_capacity); + bb->tags[bb->tags_nesting] = *tag; + bb->tags_nesting++; + return (bb->tags_nesting-1); +} + +static void bbcode_tag_pop( bbcode_t* bb, tag_t* tag ) { + if (bb->tags_nesting <= 0) { + if (tag != NULL) { tag_init(tag); } + } + else { + bb->tags_nesting--; + if (tag != NULL) { + *tag = bb->tags[bb->tags_nesting]; + } + } +} + +//------------------------------------------------------------- +// Invalid parse/values/balance +//------------------------------------------------------------- + +static void bbcode_invalid(const char* fmt, ... ) { + if (getenv("ISOCLINE_BBCODE_DEBUG") != NULL) { + va_list args; + va_start(args,fmt); + vfprintf(stderr,fmt,args); + va_end(args); + } +} + +//------------------------------------------------------------- +// Set attributes +//------------------------------------------------------------- + + +static attr_t bbcode_open( bbcode_t* bb, ssize_t out_pos, const tag_t* tag, attr_t current ) { + // save current and set + tag_t cur; + tag_init(&cur); + cur.name = tag->name; + cur.attr = current; + cur.width = tag->width; + cur.pos = out_pos; + bbcode_tag_push(bb,&cur); + return attr_update_with( current, tag->attr ); +} + +static bool bbcode_close( bbcode_t* bb, ssize_t base, const char* name, tag_t* pprev ) { + // pop until match + while (bb->tags_nesting > base) { + tag_t prev; + bbcode_tag_pop(bb,&prev); + if (name==NULL || prev.name==NULL || ic_stricmp(prev.name,name) == 0) { + // matched + if (pprev != NULL) { *pprev = prev; } + return true; + } + else { + // unbalanced: we either continue popping or restore the tags depending if there is a matching open tag in our tags. + bool has_open_tag = false; + if (name != NULL) { + for( ssize_t i = bb->tags_nesting - 1; i > base; i--) { + if (bb->tags[i].name != NULL && ic_stricmp(bb->tags[i].name, name) == 0) { + has_open_tag = true; + break; + } + } + } + bbcode_invalid("bbcode: unbalanced tags: open [%s], close [/%s]\n", prev.name, name); + if (!has_open_tag) { + bbcode_tag_push( bb, &prev ); // restore the tags and ignore this close tag + break; + } + else { + // continue until we hit our open tag + } + } + } + if (pprev != NULL) { memset(pprev,0,sizeof(*pprev)); } + return false; +} + +//------------------------------------------------------------- +// Update attributes +//------------------------------------------------------------- + +static const char* attr_update_bool( const char* fname, signed int* field, const char* value ) { + if (value == NULL || value[0] == 0 || strcmp(value,"on") || strcmp(value,"true") || strcmp(value,"1")) { + *field = IC_ON; + } + else if (strcmp(value,"off") || strcmp(value,"false") || strcmp(value,"0")) { + *field = IC_OFF; + } + else { + bbcode_invalid("bbcode: invalid %s value: %s\n", fname, value ); + } + return fname; +} + +static const char* attr_update_color( const char* fname, ic_color_t* field, const char* value ) { + if (value == NULL || value[0] == 0 || strcmp(value,"none") == 0) { + *field = IC_COLOR_NONE; + return fname; + } + + // hex value + if (value[0] == '#') { + uint32_t rgb = 0; + if (sscanf(value,"#%x",&rgb) == 1) { + *field = ic_rgb(rgb); + } + else { + bbcode_invalid("bbcode: invalid color code: %s\n", value); + } + return fname; + } + + // search color names + ssize_t lo = 0; + ssize_t hi = IC_HTML_COLOR_COUNT-1; + while( lo <= hi ) { + ssize_t mid = (lo + hi) / 2; + style_color_t* info = &html_colors[mid]; + int cmp = strcmp(info->name,value); + if (cmp < 0) { + lo = mid+1; + } + else if (cmp > 0) { + hi = mid-1; + } + else { + *field = info->color; + return fname; + } + } + bbcode_invalid("bbcode: unknown %s: %s\n", fname, value); + *field = IC_COLOR_NONE; + return fname; +} + +static const char* attr_update_sgr( const char* fname, attr_t* attr, const char* value ) { + *attr = attr_update_with(*attr, attr_from_sgr(value, ic_strlen(value))); + return fname; +} + +static void attr_update_width( width_t* pwidth, char default_fill, const char* value ) { + // parse width value: ;;; + width_t width; + memset(&width, 0, sizeof(width)); + width.fill = default_fill; // use 0 for no-fill (as for max-width) + if (ic_atoz(value, &width.w)) { + ssize_t i = 0; + while( value[i] != ';' && value[i] != 0 ) { i++; } + if (value[i] == ';') { + i++; + ssize_t len = 0; + while( value[i+len] != ';' && value[i+len] != 0 ) { len++; } + if (len == 4 && ic_istarts_with(value+i,"left")) { + width.align = IC_ALIGN_LEFT; + } + else if (len == 5 && ic_istarts_with(value+i,"right")) { + width.align = IC_ALIGN_RIGHT; + } + else if (len == 6 && ic_istarts_with(value+i,"center")) { + width.align = IC_ALIGN_CENTER; + } + i += len; + if (value[i] == ';') { + i++; len = 0; + while( value[i+len] != ';' && value[i+len] != 0 ) { len++; } + if (len == 1) { width.fill = value[i]; } + i+= len; + if (value[i] == ';') { + i++; len = 0; + while( value[i+len] != ';' && value[i+len] != 0 ) { len++; } + if ((len == 2 && ic_istarts_with(value+i,"on")) || (len == 1 && value[i] == '1')) { width.dots = true; } + i += len; + } + } + } + } + else { + bbcode_invalid("bbcode: illegal width: %s\n", value); + } + *pwidth = width; +} + +static const char* attr_update_ansi_color( const char* fname, ic_color_t* color, const char* value ) { + ssize_t num = 0; + if (ic_atoz(value, &num) && num >= 0 && num <= 256) { + *color = color_from_ansi256(num); + } + return fname; +} + + +static const char* attr_update_property( tag_t* tag, const char* attr_name, const char* value ) { + const char* fname = NULL; + fname = "bold"; + if (strcmp(attr_name,fname) == 0) { + signed int b = IC_NONE; + attr_update_bool(fname,&b, value); + if (b != IC_NONE) { tag->attr.x.bold = b; } + return fname; + } + fname = "italic"; + if (strcmp(attr_name,fname) == 0) { + signed int b = IC_NONE; + attr_update_bool(fname,&b, value); + if (b != IC_NONE) { tag->attr.x.italic = b; } + return fname; + } + fname = "underline"; + if (strcmp(attr_name,fname) == 0) { + signed int b = IC_NONE; + attr_update_bool(fname,&b, value); + if (b != IC_NONE) { tag->attr.x.underline = b; } + return fname; + } + fname = "reverse"; + if (strcmp(attr_name,fname) == 0) { + signed int b = IC_NONE; + attr_update_bool(fname,&b, value); + if (b != IC_NONE) { tag->attr.x.reverse = b; } + return fname; + } + fname = "color"; + if (strcmp(attr_name,fname) == 0) { + unsigned int color = IC_COLOR_NONE; + attr_update_color(fname, &color, value); + if (color != IC_COLOR_NONE) { tag->attr.x.color = color; } + return fname; + } + fname = "bgcolor"; + if (strcmp(attr_name,fname) == 0) { + unsigned int color = IC_COLOR_NONE; + attr_update_color(fname, &color, value); + if (color != IC_COLOR_NONE) { tag->attr.x.bgcolor = color; } + return fname; + } + fname = "ansi-sgr"; + if (strcmp(attr_name,fname) == 0) { + attr_update_sgr(fname, &tag->attr, value); + return fname; + } + fname = "ansi-color"; + if (strcmp(attr_name,fname) == 0) { + ic_color_t color = IC_COLOR_NONE;; + attr_update_ansi_color(fname, &color, value); + if (color != IC_COLOR_NONE) { tag->attr.x.color = color; } + return fname; + } + fname = "ansi-bgcolor"; + if (strcmp(attr_name,fname) == 0) { + ic_color_t color = IC_COLOR_NONE;; + attr_update_ansi_color(fname, &color, value); + if (color != IC_COLOR_NONE) { tag->attr.x.bgcolor = color; } + return fname; + } + fname = "width"; + if (strcmp(attr_name,fname) == 0) { + attr_update_width(&tag->width, ' ', value); + return fname; + } + fname = "max-width"; + if (strcmp(attr_name,fname) == 0) { + attr_update_width(&tag->width, 0, value); + return "width"; + } + else { + return NULL; + } +} + +static const style_t builtin_styles[] = { + { "b", { { IC_COLOR_NONE, IC_ON , IC_NONE, IC_COLOR_NONE, IC_NONE, IC_NONE } } }, + { "r", { { IC_COLOR_NONE, IC_NONE, IC_ON , IC_COLOR_NONE, IC_NONE, IC_NONE } } }, + { "u", { { IC_COLOR_NONE, IC_NONE, IC_NONE, IC_COLOR_NONE, IC_ON , IC_NONE } } }, + { "i", { { IC_COLOR_NONE, IC_NONE, IC_NONE, IC_COLOR_NONE, IC_NONE, IC_ON } } }, + { "em", { { IC_COLOR_NONE, IC_ON , IC_NONE, IC_COLOR_NONE, IC_NONE, IC_NONE } } }, // bold + { "url",{ { IC_COLOR_NONE, IC_NONE, IC_NONE, IC_COLOR_NONE, IC_ON, IC_NONE } } }, // underline + { NULL, { { IC_COLOR_NONE, IC_NONE, IC_NONE, IC_COLOR_NONE, IC_NONE, IC_NONE } } } +}; + +static void attr_update_with_styles( tag_t* tag, const char* attr_name, const char* value, + bool usebgcolor, const style_t* styles, ssize_t count ) +{ + // direct hex color? + if (attr_name[0] == '#' && (value == NULL || value[0]==0)) { + value = attr_name; + attr_name = (usebgcolor ? "bgcolor" : "color"); + } + // first try if it is a builtin property + const char* name; + if ((name = attr_update_property(tag,attr_name,value)) != NULL) { + if (tag->name != NULL) tag->name = name; + return; + } + // then check all styles + while( count-- > 0 ) { + const style_t* style = styles + count; + if (strcmp(style->name,attr_name) == 0) { + tag->attr = attr_update_with(tag->attr,style->attr); + if (tag->name != NULL) tag->name = style->name; + return; + } + } + // check builtin styles; todo: binary search? + for( const style_t* style = builtin_styles; style->name != NULL; style++) { + if (strcmp(style->name,attr_name) == 0) { + tag->attr = attr_update_with(tag->attr,style->attr); + if (tag->name != NULL) tag->name = style->name; + return; + } + } + // check colors as a style + ssize_t lo = 0; + ssize_t hi = IC_HTML_COLOR_COUNT-1; + while( lo <= hi ) { + ssize_t mid = (lo + hi) / 2; + style_color_t* info = &html_colors[mid]; + int cmp = strcmp(info->name,attr_name); + if (cmp < 0) { + lo = mid+1; + } + else if (cmp > 0) { + hi = mid-1; + } + else { + attr_t cattr = attr_none(); + if (usebgcolor) { cattr.x.bgcolor = info->color; } + else { cattr.x.color = info->color; } + tag->attr = attr_update_with(tag->attr,cattr); + if (tag->name != NULL) tag->name = info->name; + return; + } + } + // not found + bbcode_invalid("bbcode: unknown style: %s\n", attr_name); +} + + +ic_private attr_t bbcode_style( bbcode_t* bb, const char* style_name ) { + tag_t tag; + tag_init(&tag); + attr_update_with_styles( &tag, style_name, NULL, false, bb->styles, bb->styles_count ); + return tag.attr; +} + +//------------------------------------------------------------- +// Parse tags +//------------------------------------------------------------- + +ic_private const char* parse_skip_white(const char* s) { + while( *s != 0 && *s != ']') { + if (!(*s == ' ' || *s == '\t' || *s == '\n' || *s == '\r')) break; + s++; + } + return s; +} + +ic_private const char* parse_skip_to_white(const char* s) { + while( *s != 0 && *s != ']') { + if (*s == ' ' || *s == '\t' || *s == '\n' || *s == '\r') break; + s++; + } + return parse_skip_white(s); +} + +ic_private const char* parse_skip_to_end(const char* s) { + while( *s != 0 && *s != ']' ) { s++; } + return s; +} + +ic_private const char* parse_attr_name(const char* s) { + if (*s == '#') { + s++; // hex rgb color as id + while( *s != 0 && *s != ']') { + if (!((*s >= 'a' && *s <= 'f') || (*s >= 'A' && *s <= 'Z') || (*s >= '0' && *s <= '9'))) break; + s++; + } + } + else { + while( *s != 0 && *s != ']') { + if (!((*s >= 'a' && *s <= 'z') || (*s >= 'A' && *s <= 'Z') || + (*s >= '0' && *s <= '9') || *s == '_' || *s == '-')) break; + s++; + } + } + return s; +} + +ic_private const char* parse_value(const char* s, const char** start, const char** end) { + if (*s == '"') { + s++; + *start = s; + while( *s != 0 ) { + if (*s == '"') break; + s++; + } + *end = s; + if (*s == '"') { s++; } + } + else if (*s == '#') { + *start = s; + s++; + while( *s != 0 ) { + if (!((*s >= 'a' && *s <= 'f') || (*s >= 'A' && *s <= 'Z') || (*s >= '0' && *s <= '9'))) break; + s++; + } + *end = s; + } + else { + *start = s; + while( *s != 0 ) { + if (!((*s >= 'a' && *s <= 'z') || (*s >= 'A' && *s <= 'F') || (*s >= '0' && *s <= '9') || *s == '-' || *s == '_')) break; + s++; + } + *end = s; + } + return s; +} + +ic_private const char* parse_tag_value( tag_t* tag, char* idbuf, const char* s, const style_t* styles, ssize_t scount ) { + // parse: \s*[\w-]+\s*(=\s*) + bool usebgcolor = false; + const char* id = s; + const char* idend = parse_attr_name(id); + const char* val = NULL; + const char* valend = NULL; + if (id == idend) { + bbcode_invalid("bbcode: empty identifier? %.10s...\n", id ); + return parse_skip_to_white(id); + } + // "on" bgcolor? + s = parse_skip_white(idend); + if (idend - id == 2 && ic_strnicmp(id,"on",2) == 0 && *s != '=') { + usebgcolor = true; + id = s; + idend = parse_attr_name(id); + if (id == idend) { + bbcode_invalid("bbcode: empty identifier follows 'on'? %.10s...\n", id ); + return parse_skip_to_white(id); + } + s = parse_skip_white(idend); + } + // value + if (*s == '=') { + s++; + s = parse_skip_white(s); + s = parse_value(s, &val, &valend); + s = parse_skip_white(s); + } + // limit name and attr to 128 bytes + char valbuf[128]; + ic_strncpy( idbuf, 128, id, idend - id); + ic_strncpy( valbuf, 128, val, valend - val); + ic_str_tolower(idbuf); + ic_str_tolower(valbuf); + attr_update_with_styles( tag, idbuf, valbuf, usebgcolor, styles, scount ); + return s; +} + +static const char* parse_tag_values( tag_t* tag, char* idbuf, const char* s, const style_t* styles, ssize_t scount ) { + s = parse_skip_white(s); + idbuf[0] = 0; + ssize_t count = 0; + while( *s != 0 && *s != ']') { + char idbuf_next[128]; + s = parse_tag_value(tag, (count==0 ? idbuf : idbuf_next), s, styles, scount); + count++; + } + if (*s == ']') { s++; } + return s; +} + +static const char* parse_tag( tag_t* tag, char* idbuf, bool* open, bool* pre, const char* s, const style_t* styles, ssize_t scount ) { + *open = true; + *pre = false; + if (*s != '[') return s; + s = parse_skip_white(s+1); + if (*s == '!') { // pre + *pre = true; + s = parse_skip_white(s+1); + } + else if (*s == '/') { + *open = false; + s = parse_skip_white(s+1); + }; + s = parse_tag_values( tag, idbuf, s, styles, scount); + return s; +} + + +//--------------------------------------------------------- +// Styles +//--------------------------------------------------------- + +static void bbcode_parse_tag_content( bbcode_t* bb, const char* s, tag_t* tag ) { + tag_init(tag); + if (s != NULL) { + char idbuf[128]; + parse_tag_values(tag, idbuf, s, bb->styles, bb->styles_count); + } +} + +ic_private void bbcode_style_def( bbcode_t* bb, const char* style_name, const char* s ) { + tag_t tag; + bbcode_parse_tag_content( bb, s, &tag); + bbcode_style_add(bb, style_name, tag.attr); +} + +ic_private void bbcode_style_open( bbcode_t* bb, const char* fmt ) { + tag_t tag; + bbcode_parse_tag_content(bb, fmt, &tag); + term_set_attr( bb->term, bbcode_open(bb, 0, &tag, term_get_attr(bb->term)) ); +} + +ic_private void bbcode_style_close( bbcode_t* bb, const char* fmt ) { + const ssize_t base = bb->tags_nesting - 1; // as we end a style + tag_t tag; + bbcode_parse_tag_content(bb, fmt, &tag); + tag_t prev; + if (bbcode_close(bb, base, tag.name, &prev)) { + term_set_attr( bb->term, prev.attr ); + } +} + +//--------------------------------------------------------- +// Restrict to width +//--------------------------------------------------------- + +static void bbcode_restrict_width( ssize_t start, width_t width, stringbuf_t* out, attrbuf_t* attr_out ) { + if (width.w <= 0) return; + assert(start <= sbuf_len(out)); + assert(attr_out == NULL || sbuf_len(out) == attrbuf_len(attr_out)); + const char* s = sbuf_string(out) + start; + const ssize_t len = sbuf_len(out) - start; + const ssize_t w = str_column_width(s); + if (w == width.w) return; // fits exactly + if (w > width.w) { + // too large + ssize_t innerw = (width.dots && width.w > 3 ? width.w-3 : width.w); + if (width.align == IC_ALIGN_RIGHT) { + // right align + const ssize_t ndel = str_skip_until_fit( s, innerw ); + sbuf_delete_at( out, start, ndel ); + attrbuf_delete_at( attr_out, start, ndel ); + if (innerw < width.w) { + // add dots + sbuf_insert_at( out, "...", start ); + attr_t attr = attrbuf_attr_at(attr_out, start); + attrbuf_insert_at( attr_out, start, 3, attr); + } + } + else { + // left or center align + ssize_t count = str_take_while_fit( s, innerw ); + sbuf_delete_at( out, start + count, len - count ); + attrbuf_delete_at( attr_out, start + count, len - count ); + if (innerw < width.w) { + // add dots + attr_t attr = attrbuf_attr_at(attr_out,start); + attrbuf_append_n( out, attr_out, "...", 3, attr ); + } + } + } + else { + // too short, pad to width + const ssize_t diff = (width.w - w); + const ssize_t pad_left = (width.align == IC_ALIGN_RIGHT ? diff : (width.align == IC_ALIGN_LEFT ? 0 : diff / 2)); + const ssize_t pad_right = (width.align == IC_ALIGN_LEFT ? diff : (width.align == IC_ALIGN_RIGHT ? 0 : diff - pad_left)); + if (width.fill != 0 && pad_left > 0) { + const attr_t attr = attrbuf_attr_at(attr_out,start); + for( ssize_t i = 0; i < pad_left; i++) { // todo: optimize + sbuf_insert_char_at(out, width.fill, start); + } + attrbuf_insert_at( attr_out, start, pad_left, attr ); + } + if (width.fill != 0 && pad_right > 0) { + const attr_t attr = attrbuf_attr_at(attr_out,sbuf_len(out) - 1); + char buf[2]; + buf[0] = width.fill; + buf[1] = 0; + for( ssize_t i = 0; i < pad_right; i++) { // todo: optimize + attrbuf_append_n( out, attr_out, buf, 1, attr ); + } + } + } +} + +//--------------------------------------------------------- +// Print +//--------------------------------------------------------- + +ic_private ssize_t bbcode_process_tag( bbcode_t* bb, const char* s, const ssize_t nesting_base, + stringbuf_t* out, attrbuf_t* attr_out, attr_t* cur_attr ) { + assert(*s == '['); + tag_t tag; + tag_init(&tag); + bool open = true; + bool ispre = false; + char idbuf[128]; + const char* end = parse_tag( &tag, idbuf, &open, &ispre, s, bb->styles, bb->styles_count ); // todo: styles + assert(end > s); + if (open) { + if (!ispre) { + // open tag + *cur_attr = bbcode_open( bb, sbuf_len(out), &tag, *cur_attr ); + } + else { + // scan pre to end tag + attr_t attr = attr_update_with(*cur_attr, tag.attr); + char pre[132]; + if (snprintf(pre, 132, "[/%s]", idbuf) < ssizeof(pre)) { + const char* etag = strstr(end,pre); + if (etag == NULL) { + const ssize_t len = ic_strlen(end); + attrbuf_append_n(out, attr_out, end, len, attr); + end += len; + } + else { + attrbuf_append_n(out, attr_out, end, (etag - end), attr); + end = etag + ic_strlen(pre); + } + } + } + } + else { + // pop the tag + tag_t prev; + if (bbcode_close( bb, nesting_base, tag.name, &prev)) { + *cur_attr = prev.attr; + if (prev.width.w > 0) { + // closed a width tag; restrict the output to width + bbcode_restrict_width( prev.pos, prev.width, out, attr_out); + } + } + } + return (end - s); +} + +ic_private void bbcode_append( bbcode_t* bb, const char* s, stringbuf_t* out, attrbuf_t* attr_out ) { + if (bb == NULL || s == NULL) return; + attr_t attr = attr_none(); + const ssize_t base = bb->tags_nesting; // base; will not be popped + ssize_t i = 0; + while( s[i] != 0 ) { + // handle no tags in bulk + ssize_t nobb = 0; + char c; + while( (c = s[i+nobb]) != 0) { + if (c == '[' || c == '\\') { break; } + if (c == '\x1B' && s[i+nobb+1] == '[') { + nobb++; // don't count 'ESC[' as a tag opener + } + nobb++; + } + if (nobb > 0) { attrbuf_append_n(out, attr_out, s+i, nobb, attr); } + i += nobb; + // tag + if (s[i] == '[') { + i += bbcode_process_tag(bb, s+i, base, out, attr_out, &attr); + } + else if (s[i] == '\\') { + if (s[i+1] == '\\' || s[i+1] == '[') { + attrbuf_append_n(out, attr_out, s+i+1, 1, attr); // escape '\[' and '\\' + i += 2; + } + else { + attrbuf_append_n(out, attr_out, s+i, 1, attr); // pass '\\' as is + i++; + } + } + } + // pop unclosed openings + assert(bb->tags_nesting >= base); + while( bb->tags_nesting > base ) { + bbcode_tag_pop(bb,NULL); + }; +} + +ic_private void bbcode_print( bbcode_t* bb, const char* s ) { + if (bb->out == NULL || bb->out_attrs == NULL || s == NULL) return; + assert(sbuf_len(bb->out) == 0 && attrbuf_len(bb->out_attrs) == 0); + bbcode_append( bb, s, bb->out, bb->out_attrs ); + term_write_formatted( bb->term, sbuf_string(bb->out), attrbuf_attrs(bb->out_attrs,sbuf_len(bb->out)) ); + attrbuf_clear(bb->out_attrs); + sbuf_clear(bb->out); +} + +ic_private void bbcode_println( bbcode_t* bb, const char* s ) { + bbcode_print(bb,s); + term_writeln(bb->term, ""); +} + +ic_private void bbcode_vprintf( bbcode_t* bb, const char* fmt, va_list args ) { + if (bb->vout == NULL || fmt == NULL) return; + assert(sbuf_len(bb->vout) == 0); + sbuf_append_vprintf(bb->vout,fmt,args); + bbcode_print(bb, sbuf_string(bb->vout)); + sbuf_clear(bb->vout); +} + +ic_private void bbcode_printf( bbcode_t* bb, const char* fmt, ... ) { + va_list args; + va_start(args,fmt); + bbcode_vprintf(bb,fmt,args); + va_end(args); +} + +ic_private ssize_t bbcode_column_width( bbcode_t* bb, const char* s ) { + if (s==NULL || s[0] == 0) return 0; + if (bb->vout == NULL) { return str_column_width(s); } + assert(sbuf_len(bb->vout) == 0); + bbcode_append( bb, s, bb->vout, NULL); + const ssize_t w = str_column_width(sbuf_string(bb->vout)); + sbuf_clear(bb->vout); + return w; +} diff --git a/extern/isocline/src/bbcode.h b/extern/isocline/src/bbcode.h new file mode 100644 index 00000000..be96bfe2 --- /dev/null +++ b/extern/isocline/src/bbcode.h @@ -0,0 +1,37 @@ +/* ---------------------------------------------------------------------------- + Copyright (c) 2021, Daan Leijen + This is free software; you can redistribute it and/or modify it + under the terms of the MIT License. A copy of the license can be + found in the "LICENSE" file at the root of this distribution. +-----------------------------------------------------------------------------*/ +#pragma once +#ifndef IC_BBCODE_H +#define IC_BBCODE_H + +#include +#include "common.h" +#include "term.h" + +struct bbcode_s; +typedef struct bbcode_s bbcode_t; + +ic_private bbcode_t* bbcode_new( alloc_t* mem, term_t* term ); +ic_private void bbcode_free( bbcode_t* bb ); + +ic_private void bbcode_style_add( bbcode_t* bb, const char* style_name, attr_t attr ); +ic_private void bbcode_style_def( bbcode_t* bb, const char* style_name, const char* s ); +ic_private void bbcode_style_open( bbcode_t* bb, const char* fmt ); +ic_private void bbcode_style_close( bbcode_t* bb, const char* fmt ); +ic_private attr_t bbcode_style( bbcode_t* bb, const char* style_name ); + +ic_private void bbcode_print( bbcode_t* bb, const char* s ); +ic_private void bbcode_println( bbcode_t* bb, const char* s ); +ic_private void bbcode_printf( bbcode_t* bb, const char* fmt, ... ); +ic_private void bbcode_vprintf( bbcode_t* bb, const char* fmt, va_list args ); + +ic_private ssize_t bbcode_column_width( bbcode_t* bb, const char* s ); + +// allows `attr_out == NULL`. +ic_private void bbcode_append( bbcode_t* bb, const char* s, stringbuf_t* out, attrbuf_t* attr_out ); + +#endif // IC_BBCODE_H diff --git a/extern/isocline/src/bbcode_colors.c b/extern/isocline/src/bbcode_colors.c new file mode 100644 index 00000000..245cd3de --- /dev/null +++ b/extern/isocline/src/bbcode_colors.c @@ -0,0 +1,194 @@ +/* ---------------------------------------------------------------------------- + Copyright (c) 2021, Daan Leijen + This is free software; you can redistribute it and/or modify it + under the terms of the MIT License. A copy of the license can be + found in the "LICENSE" file at the root of this distribution. +-----------------------------------------------------------------------------*/ + +// This file is included from "bbcode.c" and contains html color names + +#include "common.h" + +typedef struct style_color_s { + const char* name; + ic_color_t color; +} style_color_t; + +#define IC_HTML_COLOR_COUNT (172) + +// ordered list of HTML color names (so we can use binary search) +static style_color_t html_colors[IC_HTML_COLOR_COUNT+1] = { + { "aliceblue", IC_RGB(0xf0f8ff) }, + { "ansi-aqua", IC_ANSI_AQUA }, + { "ansi-black", IC_ANSI_BLACK }, + { "ansi-blue", IC_ANSI_BLUE }, + { "ansi-cyan", IC_ANSI_CYAN }, + { "ansi-darkgray", IC_ANSI_DARKGRAY }, + { "ansi-darkgrey", IC_ANSI_DARKGRAY }, + { "ansi-default", IC_ANSI_DEFAULT }, + { "ansi-fuchsia", IC_ANSI_FUCHSIA }, + { "ansi-gray", IC_ANSI_GRAY }, + { "ansi-green", IC_ANSI_GREEN }, + { "ansi-grey", IC_ANSI_GRAY }, + { "ansi-lightgray", IC_ANSI_LIGHTGRAY }, + { "ansi-lightgrey", IC_ANSI_LIGHTGRAY }, + { "ansi-lime" , IC_ANSI_LIME }, + { "ansi-magenta", IC_ANSI_MAGENTA }, + { "ansi-maroon", IC_ANSI_MAROON }, + { "ansi-navy", IC_ANSI_NAVY }, + { "ansi-olive", IC_ANSI_OLIVE }, + { "ansi-purple", IC_ANSI_PURPLE }, + { "ansi-red", IC_ANSI_RED }, + { "ansi-silver", IC_ANSI_SILVER }, + { "ansi-teal", IC_ANSI_TEAL }, + { "ansi-white", IC_ANSI_WHITE }, + { "ansi-yellow", IC_ANSI_YELLOW }, + { "antiquewhite", IC_RGB(0xfaebd7) }, + { "aqua", IC_RGB(0x00ffff) }, + { "aquamarine", IC_RGB(0x7fffd4) }, + { "azure", IC_RGB(0xf0ffff) }, + { "beige", IC_RGB(0xf5f5dc) }, + { "bisque", IC_RGB(0xffe4c4) }, + { "black", IC_RGB(0x000000) }, + { "blanchedalmond", IC_RGB(0xffebcd) }, + { "blue", IC_RGB(0x0000ff) }, + { "blueviolet", IC_RGB(0x8a2be2) }, + { "brown", IC_RGB(0xa52a2a) }, + { "burlywood", IC_RGB(0xdeb887) }, + { "cadetblue", IC_RGB(0x5f9ea0) }, + { "chartreuse", IC_RGB(0x7fff00) }, + { "chocolate", IC_RGB(0xd2691e) }, + { "coral", IC_RGB(0xff7f50) }, + { "cornflowerblue", IC_RGB(0x6495ed) }, + { "cornsilk", IC_RGB(0xfff8dc) }, + { "crimson", IC_RGB(0xdc143c) }, + { "cyan", IC_RGB(0x00ffff) }, + { "darkblue", IC_RGB(0x00008b) }, + { "darkcyan", IC_RGB(0x008b8b) }, + { "darkgoldenrod", IC_RGB(0xb8860b) }, + { "darkgray", IC_RGB(0xa9a9a9) }, + { "darkgreen", IC_RGB(0x006400) }, + { "darkgrey", IC_RGB(0xa9a9a9) }, + { "darkkhaki", IC_RGB(0xbdb76b) }, + { "darkmagenta", IC_RGB(0x8b008b) }, + { "darkolivegreen", IC_RGB(0x556b2f) }, + { "darkorange", IC_RGB(0xff8c00) }, + { "darkorchid", IC_RGB(0x9932cc) }, + { "darkred", IC_RGB(0x8b0000) }, + { "darksalmon", IC_RGB(0xe9967a) }, + { "darkseagreen", IC_RGB(0x8fbc8f) }, + { "darkslateblue", IC_RGB(0x483d8b) }, + { "darkslategray", IC_RGB(0x2f4f4f) }, + { "darkslategrey", IC_RGB(0x2f4f4f) }, + { "darkturquoise", IC_RGB(0x00ced1) }, + { "darkviolet", IC_RGB(0x9400d3) }, + { "deeppink", IC_RGB(0xff1493) }, + { "deepskyblue", IC_RGB(0x00bfff) }, + { "dimgray", IC_RGB(0x696969) }, + { "dimgrey", IC_RGB(0x696969) }, + { "dodgerblue", IC_RGB(0x1e90ff) }, + { "firebrick", IC_RGB(0xb22222) }, + { "floralwhite", IC_RGB(0xfffaf0) }, + { "forestgreen", IC_RGB(0x228b22) }, + { "fuchsia", IC_RGB(0xff00ff) }, + { "gainsboro", IC_RGB(0xdcdcdc) }, + { "ghostwhite", IC_RGB(0xf8f8ff) }, + { "gold", IC_RGB(0xffd700) }, + { "goldenrod", IC_RGB(0xdaa520) }, + { "gray", IC_RGB(0x808080) }, + { "green", IC_RGB(0x008000) }, + { "greenyellow", IC_RGB(0xadff2f) }, + { "grey", IC_RGB(0x808080) }, + { "honeydew", IC_RGB(0xf0fff0) }, + { "hotpink", IC_RGB(0xff69b4) }, + { "indianred", IC_RGB(0xcd5c5c) }, + { "indigo", IC_RGB(0x4b0082) }, + { "ivory", IC_RGB(0xfffff0) }, + { "khaki", IC_RGB(0xf0e68c) }, + { "lavender", IC_RGB(0xe6e6fa) }, + { "lavenderblush", IC_RGB(0xfff0f5) }, + { "lawngreen", IC_RGB(0x7cfc00) }, + { "lemonchiffon", IC_RGB(0xfffacd) }, + { "lightblue", IC_RGB(0xadd8e6) }, + { "lightcoral", IC_RGB(0xf08080) }, + { "lightcyan", IC_RGB(0xe0ffff) }, + { "lightgoldenrodyellow", IC_RGB(0xfafad2) }, + { "lightgray", IC_RGB(0xd3d3d3) }, + { "lightgreen", IC_RGB(0x90ee90) }, + { "lightgrey", IC_RGB(0xd3d3d3) }, + { "lightpink", IC_RGB(0xffb6c1) }, + { "lightsalmon", IC_RGB(0xffa07a) }, + { "lightseagreen", IC_RGB(0x20b2aa) }, + { "lightskyblue", IC_RGB(0x87cefa) }, + { "lightslategray", IC_RGB(0x778899) }, + { "lightslategrey", IC_RGB(0x778899) }, + { "lightsteelblue", IC_RGB(0xb0c4de) }, + { "lightyellow", IC_RGB(0xffffe0) }, + { "lime", IC_RGB(0x00ff00) }, + { "limegreen", IC_RGB(0x32cd32) }, + { "linen", IC_RGB(0xfaf0e6) }, + { "magenta", IC_RGB(0xff00ff) }, + { "maroon", IC_RGB(0x800000) }, + { "mediumaquamarine", IC_RGB(0x66cdaa) }, + { "mediumblue", IC_RGB(0x0000cd) }, + { "mediumorchid", IC_RGB(0xba55d3) }, + { "mediumpurple", IC_RGB(0x9370db) }, + { "mediumseagreen", IC_RGB(0x3cb371) }, + { "mediumslateblue", IC_RGB(0x7b68ee) }, + { "mediumspringgreen", IC_RGB(0x00fa9a) }, + { "mediumturquoise", IC_RGB(0x48d1cc) }, + { "mediumvioletred", IC_RGB(0xc71585) }, + { "midnightblue", IC_RGB(0x191970) }, + { "mintcream", IC_RGB(0xf5fffa) }, + { "mistyrose", IC_RGB(0xffe4e1) }, + { "moccasin", IC_RGB(0xffe4b5) }, + { "navajowhite", IC_RGB(0xffdead) }, + { "navy", IC_RGB(0x000080) }, + { "oldlace", IC_RGB(0xfdf5e6) }, + { "olive", IC_RGB(0x808000) }, + { "olivedrab", IC_RGB(0x6b8e23) }, + { "orange", IC_RGB(0xffa500) }, + { "orangered", IC_RGB(0xff4500) }, + { "orchid", IC_RGB(0xda70d6) }, + { "palegoldenrod", IC_RGB(0xeee8aa) }, + { "palegreen", IC_RGB(0x98fb98) }, + { "paleturquoise", IC_RGB(0xafeeee) }, + { "palevioletred", IC_RGB(0xdb7093) }, + { "papayawhip", IC_RGB(0xffefd5) }, + { "peachpuff", IC_RGB(0xffdab9) }, + { "peru", IC_RGB(0xcd853f) }, + { "pink", IC_RGB(0xffc0cb) }, + { "plum", IC_RGB(0xdda0dd) }, + { "powderblue", IC_RGB(0xb0e0e6) }, + { "purple", IC_RGB(0x800080) }, + { "rebeccapurple", IC_RGB(0x663399) }, + { "red", IC_RGB(0xff0000) }, + { "rosybrown", IC_RGB(0xbc8f8f) }, + { "royalblue", IC_RGB(0x4169e1) }, + { "saddlebrown", IC_RGB(0x8b4513) }, + { "salmon", IC_RGB(0xfa8072) }, + { "sandybrown", IC_RGB(0xf4a460) }, + { "seagreen", IC_RGB(0x2e8b57) }, + { "seashell", IC_RGB(0xfff5ee) }, + { "sienna", IC_RGB(0xa0522d) }, + { "silver", IC_RGB(0xc0c0c0) }, + { "skyblue", IC_RGB(0x87ceeb) }, + { "slateblue", IC_RGB(0x6a5acd) }, + { "slategray", IC_RGB(0x708090) }, + { "slategrey", IC_RGB(0x708090) }, + { "snow", IC_RGB(0xfffafa) }, + { "springgreen", IC_RGB(0x00ff7f) }, + { "steelblue", IC_RGB(0x4682b4) }, + { "tan", IC_RGB(0xd2b48c) }, + { "teal", IC_RGB(0x008080) }, + { "thistle", IC_RGB(0xd8bfd8) }, + { "tomato", IC_RGB(0xff6347) }, + { "turquoise", IC_RGB(0x40e0d0) }, + { "violet", IC_RGB(0xee82ee) }, + { "wheat", IC_RGB(0xf5deb3) }, + { "white", IC_RGB(0xffffff) }, + { "whitesmoke", IC_RGB(0xf5f5f5) }, + { "yellow", IC_RGB(0xffff00) }, + { "yellowgreen", IC_RGB(0x9acd32) }, + {NULL, 0} +}; diff --git a/extern/isocline/src/common.c b/extern/isocline/src/common.c new file mode 100644 index 00000000..1d9fb566 --- /dev/null +++ b/extern/isocline/src/common.c @@ -0,0 +1,347 @@ +/* ---------------------------------------------------------------------------- + Copyright (c) 2021, Daan Leijen + This is free software; you can redistribute it and/or modify it + under the terms of the MIT License. A copy of the license can be + found in the "LICENSE" file at the root of this distribution. +-----------------------------------------------------------------------------*/ + +#include +#include +#include +#include +#include "common.h" + + +//------------------------------------------------------------- +// String wrappers for ssize_t +//------------------------------------------------------------- + +ic_private ssize_t ic_strlen( const char* s ) { + if (s==NULL) return 0; + return to_ssize_t(strlen(s)); +} + +ic_private void ic_memmove( void* dest, const void* src, ssize_t n ) { + assert(dest!=NULL && src != NULL); + if (n <= 0) return; + memmove(dest,src,to_size_t(n)); +} + + +ic_private void ic_memcpy( void* dest, const void* src, ssize_t n ) { + assert(dest!=NULL && src != NULL); + if (dest == NULL || src == NULL || n <= 0) return; + memcpy(dest,src,to_size_t(n)); +} + +ic_private void ic_memset(void* dest, uint8_t value, ssize_t n) { + assert(dest!=NULL); + if (dest == NULL || n <= 0) return; + memset(dest,(int8_t)value,to_size_t(n)); +} + +ic_private bool ic_memnmove( void* dest, ssize_t dest_size, const void* src, ssize_t n ) { + assert(dest!=NULL && src != NULL); + if (n <= 0) return true; + if (dest_size < n) { assert(false); return false; } + memmove(dest,src,to_size_t(n)); + return true; +} + +ic_private bool ic_strcpy( char* dest, ssize_t dest_size /* including 0 */, const char* src) { + assert(dest!=NULL && src != NULL); + if (dest == NULL || dest_size <= 0) return false; + ssize_t slen = ic_strlen(src); + if (slen >= dest_size) return false; + strcpy(dest,src); + assert(dest[slen] == 0); + return true; +} + + +ic_private bool ic_strncpy( char* dest, ssize_t dest_size /* including 0 */, const char* src, ssize_t n) { + assert(dest!=NULL && n < dest_size); + if (dest == NULL || dest_size <= 0) return false; + if (n >= dest_size) return false; + if (src==NULL || n <= 0) { + dest[0] = 0; + } + else { + strncpy(dest,src,to_size_t(n)); + dest[n] = 0; + } + return true; +} + +//------------------------------------------------------------- +// String matching +//------------------------------------------------------------- + +ic_public bool ic_starts_with( const char* s, const char* prefix ) { + if (s==prefix) return true; + if (prefix==NULL) return true; + if (s==NULL) return false; + + ssize_t i; + for( i = 0; s[i] != 0 && prefix[i] != 0; i++) { + if (s[i] != prefix[i]) return false; + } + return (prefix[i] == 0); +} + +ic_private char ic_tolower( char c ) { + return (c >= 'A' && c <= 'Z' ? c - 'A' + 'a' : c); +} + +ic_private void ic_str_tolower(char* s) { + while(*s != 0) { + *s = ic_tolower(*s); + s++; + } +} + +ic_public bool ic_istarts_with( const char* s, const char* prefix ) { + if (s==prefix) return true; + if (prefix==NULL) return true; + if (s==NULL) return false; + + ssize_t i; + for( i = 0; s[i] != 0 && prefix[i] != 0; i++) { + if (ic_tolower(s[i]) != ic_tolower(prefix[i])) return false; + } + return (prefix[i] == 0); +} + + +ic_private int ic_strnicmp(const char* s1, const char* s2, ssize_t n) { + if (s1 == NULL && s2 == NULL) return 0; + if (s1 == NULL) return -1; + if (s2 == NULL) return 1; + ssize_t i; + for (i = 0; s1[i] != 0 && i < n; i++) { // note: if s2[i] == 0 the loop will stop as c1 != c2 + char c1 = ic_tolower(s1[i]); + char c2 = ic_tolower(s2[i]); + if (c1 < c2) return -1; + if (c1 > c2) return 1; + } + return ((i >= n || s2[i] == 0) ? 0 : -1); +} + +ic_private int ic_stricmp(const char* s1, const char* s2) { + ssize_t len1 = ic_strlen(s1); + ssize_t len2 = ic_strlen(s2); + if (len1 < len2) return -1; + if (len1 > len2) return 1; + return (ic_strnicmp(s1, s2, (len1 >= len2 ? len1 : len2))); +} + + +static const char* ic_stristr(const char* s, const char* pat) { + if (s==NULL) return NULL; + if (pat==NULL || pat[0] == 0) return s; + ssize_t patlen = ic_strlen(pat); + for (ssize_t i = 0; s[i] != 0; i++) { + if (ic_strnicmp(s + i, pat, patlen) == 0) return (s+i); + } + return NULL; +} + +ic_private bool ic_contains(const char* big, const char* s) { + if (big == NULL) return false; + if (s == NULL) return true; + return (strstr(big,s) != NULL); +} + +ic_private bool ic_icontains(const char* big, const char* s) { + if (big == NULL) return false; + if (s == NULL) return true; + return (ic_stristr(big,s) != NULL); +} + + +//------------------------------------------------------------- +// Unicode +// QUTF-8: See +// Raw bytes are code points 0xEE000 - 0xEE0FF +//------------------------------------------------------------- +#define IC_UNICODE_RAW ((unicode_t)(0xEE000U)) + +ic_private unicode_t unicode_from_raw(uint8_t c) { + return (IC_UNICODE_RAW + c); +} + +ic_private bool unicode_is_raw(unicode_t u, uint8_t* c) { + if (u >= IC_UNICODE_RAW && u <= IC_UNICODE_RAW + 0xFF) { + *c = (uint8_t)(u - IC_UNICODE_RAW); + return true; + } + else { + return false; + } +} + +ic_private void unicode_to_qutf8(unicode_t u, uint8_t buf[5]) { + memset(buf, 0, 5); + if (u <= 0x7F) { + buf[0] = (uint8_t)u; + } + else if (u <= 0x07FF) { + buf[0] = (0xC0 | ((uint8_t)(u >> 6))); + buf[1] = (0x80 | (((uint8_t)u) & 0x3F)); + } + else if (u <= 0xFFFF) { + buf[0] = (0xE0 | ((uint8_t)(u >> 12))); + buf[1] = (0x80 | (((uint8_t)(u >> 6)) & 0x3F)); + buf[2] = (0x80 | (((uint8_t)u) & 0x3F)); + } + else if (u <= 0x10FFFF) { + if (unicode_is_raw(u, &buf[0])) { + buf[1] = 0; + } + else { + buf[0] = (0xF0 | ((uint8_t)(u >> 18))); + buf[1] = (0x80 | (((uint8_t)(u >> 12)) & 0x3F)); + buf[2] = (0x80 | (((uint8_t)(u >> 6)) & 0x3F)); + buf[3] = (0x80 | (((uint8_t)u) & 0x3F)); + } + } +} + +// is this a utf8 continuation byte? +ic_private bool utf8_is_cont(uint8_t c) { + return ((c & 0xC0) == 0x80); +} + +ic_private unicode_t unicode_from_qutf8(const uint8_t* s, ssize_t len, ssize_t* count) { + unicode_t c0 = 0; + if (len <= 0 || s == NULL) { + goto fail; + } + // 1 byte + c0 = s[0]; + if (c0 <= 0x7F && len >= 1) { + if (count != NULL) *count = 1; + return c0; + } + else if (c0 <= 0xC1) { // invalid continuation byte or invalid 0xC0, 0xC1 + goto fail; + } + // 2 bytes + else if (c0 <= 0xDF && len >= 2 && utf8_is_cont(s[1])) { + if (count != NULL) *count = 2; + return (((c0 & 0x1F) << 6) | (s[1] & 0x3F)); + } + // 3 bytes: reject overlong and surrogate halves + else if (len >= 3 && + ((c0 == 0xE0 && s[1] >= 0xA0 && s[1] <= 0xBF && utf8_is_cont(s[2])) || + (c0 >= 0xE1 && c0 <= 0xEC && utf8_is_cont(s[1]) && utf8_is_cont(s[2])) + )) + { + if (count != NULL) *count = 3; + return (((c0 & 0x0F) << 12) | ((unicode_t)(s[1] & 0x3F) << 6) | (s[2] & 0x3F)); + } + // 4 bytes: reject overlong + else if (len >= 4 && + (((c0 == 0xF0 && s[1] >= 0x90 && s[1] <= 0xBF && utf8_is_cont(s[2]) && utf8_is_cont(s[3])) || + (c0 >= 0xF1 && c0 <= 0xF3 && utf8_is_cont(s[1]) && utf8_is_cont(s[2]) && utf8_is_cont(s[3])) || + (c0 == 0xF4 && s[1] >= 0x80 && s[1] <= 0x8F && utf8_is_cont(s[2]) && utf8_is_cont(s[3]))) + )) + { + if (count != NULL) *count = 4; + return (((c0 & 0x07) << 18) | ((unicode_t)(s[1] & 0x3F) << 12) | ((unicode_t)(s[2] & 0x3F) << 6) | (s[3] & 0x3F)); + } +fail: + if (count != NULL) *count = 1; + return unicode_from_raw(s[0]); +} + + +//------------------------------------------------------------- +// Debug +//------------------------------------------------------------- + +#if defined(IC_NO_DEBUG_MSG) +// nothing +#elif !defined(IC_DEBUG_TO_FILE) +ic_private void debug_msg(const char* fmt, ...) { + if (getenv("ISOCLINE_DEBUG")) { + va_list args; + va_start(args, fmt); + vfprintf(stderr, fmt, args); + va_end(args); + } +} +#else +ic_private void debug_msg(const char* fmt, ...) { + static int debug_init; + static const char* debug_fname = "isocline.debug.txt"; + // initialize? + if (debug_init==0) { + debug_init = -1; + const char* rdebug = getenv("ISOCLINE_DEBUG"); + if (rdebug!=NULL && strcmp(rdebug,"1") == 0) { + FILE* fdbg = fopen(debug_fname, "w"); + if (fdbg!=NULL) { + debug_init = 1; + fclose(fdbg); + } + } + } + if (debug_init <= 0) return; + + // write debug messages + FILE* fdbg = fopen(debug_fname, "a"); + if (fdbg==NULL) return; + va_list args; + va_start(args, fmt); + vfprintf(fdbg, fmt, args); + fclose(fdbg); + va_end(args); +} +#endif + + +//------------------------------------------------------------- +// Allocation +//------------------------------------------------------------- + +ic_private void* mem_malloc(alloc_t* mem, ssize_t sz) { + return mem->malloc(to_size_t(sz)); +} + +ic_private void* mem_zalloc(alloc_t* mem, ssize_t sz) { + void* p = mem_malloc(mem, sz); + if (p != NULL) memset(p, 0, to_size_t(sz)); + return p; +} + +ic_private void* mem_realloc(alloc_t* mem, void* p, ssize_t newsz) { + return mem->realloc(p, to_size_t(newsz)); +} + +ic_private void mem_free(alloc_t* mem, const void* p) { + mem->free((void*)p); +} + +ic_private char* mem_strdup(alloc_t* mem, const char* s) { + if (s==NULL) return NULL; + ssize_t n = ic_strlen(s); + char* p = mem_malloc_tp_n(mem, char, n+1); + if (p == NULL) return NULL; + ic_memcpy(p, s, n+1); + return p; +} + +ic_private char* mem_strndup(alloc_t* mem, const char* s, ssize_t n) { + if (s==NULL || n < 0) return NULL; + char* p = mem_malloc_tp_n(mem, char, n+1); + if (p == NULL) return NULL; + ssize_t i; + for (i = 0; i < n && s[i] != 0; i++) { + p[i] = s[i]; + } + assert(i <= n); + p[i] = 0; + return p; +} + diff --git a/extern/isocline/src/common.h b/extern/isocline/src/common.h new file mode 100644 index 00000000..dd5b2569 --- /dev/null +++ b/extern/isocline/src/common.h @@ -0,0 +1,187 @@ +/* ---------------------------------------------------------------------------- + Copyright (c) 2021, Daan Leijen + This is free software; you can redistribute it and/or modify it + under the terms of the MIT License. A copy of the license can be + found in the "LICENSE" file at the root of this distribution. +-----------------------------------------------------------------------------*/ + +#pragma once +#ifndef IC_COMMON_H +#define IC_COMMON_H + +//------------------------------------------------------------- +// Headers and defines +//------------------------------------------------------------- + +#include // ssize_t +#include +#include +#include +#include +#include +#include "../include/isocline.h" // ic_malloc_fun_t, ic_color_t etc. + +# ifdef __cplusplus +# define ic_extern_c extern "C" +# else +# define ic_extern_c +# endif + +#if defined(IC_SEPARATE_OBJS) +# define ic_public ic_extern_c +# if defined(__GNUC__) // includes clang and icc +# define ic_private __attribute__((visibility("hidden"))) +# else +# define ic_private +# endif +#else +# define ic_private static +# define ic_public ic_extern_c +#endif + +#define ic_unused(x) (void)(x) + + +//------------------------------------------------------------- +// ssize_t +//------------------------------------------------------------- + +#if defined(_MSC_VER) +typedef intptr_t ssize_t; +#endif + +#define ssizeof(tp) (ssize_t)(sizeof(tp)) +static inline size_t to_size_t(ssize_t sz) { return (sz >= 0 ? (size_t)sz : 0); } +static inline ssize_t to_ssize_t(size_t sz) { return (sz <= SIZE_MAX/2 ? (ssize_t)sz : 0); } + +ic_private void ic_memmove(void* dest, const void* src, ssize_t n); +ic_private void ic_memcpy(void* dest, const void* src, ssize_t n); +ic_private void ic_memset(void* dest, uint8_t value, ssize_t n); +ic_private bool ic_memnmove(void* dest, ssize_t dest_size, const void* src, ssize_t n); + +ic_private ssize_t ic_strlen(const char* s); +ic_private bool ic_strcpy(char* dest, ssize_t dest_size /* including 0 */, const char* src); +ic_private bool ic_strncpy(char* dest, ssize_t dest_size /* including 0 */, const char* src, ssize_t n); + +ic_private bool ic_contains(const char* big, const char* s); +ic_private bool ic_icontains(const char* big, const char* s); +ic_private char ic_tolower(char c); +ic_private void ic_str_tolower(char* s); +ic_private int ic_stricmp(const char* s1, const char* s2); +ic_private int ic_strnicmp(const char* s1, const char* s2, ssize_t n); + + + +//--------------------------------------------------------------------- +// Unicode +// +// We use "qutf-8" (quite like utf-8) encoding and decoding. +// Internally we always use valid utf-8. If we encounter invalid +// utf-8 bytes (or bytes >= 0x80 from any other encoding) we encode +// these as special code points in the "raw plane" (0xEE000 - 0xEE0FF). +// When decoding we are then able to restore such raw bytes as-is. +// See +//--------------------------------------------------------------------- + +typedef uint32_t unicode_t; + +ic_private void unicode_to_qutf8(unicode_t u, uint8_t buf[5]); +ic_private unicode_t unicode_from_qutf8(const uint8_t* s, ssize_t len, ssize_t* nread); // validating + +ic_private unicode_t unicode_from_raw(uint8_t c); +ic_private bool unicode_is_raw(unicode_t u, uint8_t* c); + +ic_private bool utf8_is_cont(uint8_t c); + + +//------------------------------------------------------------- +// Colors +//------------------------------------------------------------- + +// A color is either RGB or an ANSI code. +// (RGB colors have bit 24 set to distinguish them from the ANSI color palette colors.) +// (Isocline will automatically convert from RGB on terminals that do not support full colors) +typedef uint32_t ic_color_t; + +// Create a color from a 24-bit color value. +ic_private ic_color_t ic_rgb(uint32_t hex); + +// Create a color from a 8-bit red/green/blue components. +// The value of each component is capped between 0 and 255. +ic_private ic_color_t ic_rgbx(ssize_t r, ssize_t g, ssize_t b); + +#define IC_COLOR_NONE (0) +#define IC_RGB(rgb) (0x1000000 | (uint32_t)(rgb)) // ic_rgb(rgb) // define to it can be used as a constant + +// ANSI colors. +// The actual colors used is usually determined by the terminal theme +// See +#define IC_ANSI_BLACK (30) +#define IC_ANSI_MAROON (31) +#define IC_ANSI_GREEN (32) +#define IC_ANSI_OLIVE (33) +#define IC_ANSI_NAVY (34) +#define IC_ANSI_PURPLE (35) +#define IC_ANSI_TEAL (36) +#define IC_ANSI_SILVER (37) +#define IC_ANSI_DEFAULT (39) + +#define IC_ANSI_GRAY (90) +#define IC_ANSI_RED (91) +#define IC_ANSI_LIME (92) +#define IC_ANSI_YELLOW (93) +#define IC_ANSI_BLUE (94) +#define IC_ANSI_FUCHSIA (95) +#define IC_ANSI_AQUA (96) +#define IC_ANSI_WHITE (97) + +#define IC_ANSI_DARKGRAY IC_ANSI_GRAY +#define IC_ANSI_LIGHTGRAY IC_ANSI_SILVER +#define IC_ANSI_MAGENTA IC_ANSI_FUCHSIA +#define IC_ANSI_CYAN IC_ANSI_AQUA + + + +//------------------------------------------------------------- +// Debug +//------------------------------------------------------------- + +#if defined(IC_NO_DEBUG_MSG) +#define debug_msg(fmt,...) (void)(0) +#else +ic_private void debug_msg( const char* fmt, ... ); +#endif + + +//------------------------------------------------------------- +// Abstract environment +//------------------------------------------------------------- +struct ic_env_s; +typedef struct ic_env_s ic_env_t; + + +//------------------------------------------------------------- +// Allocation +//------------------------------------------------------------- + +typedef struct alloc_s { + ic_malloc_fun_t* malloc; + ic_realloc_fun_t* realloc; + ic_free_fun_t* free; +} alloc_t; + + +ic_private void* mem_malloc( alloc_t* mem, ssize_t sz ); +ic_private void* mem_zalloc( alloc_t* mem, ssize_t sz ); +ic_private void* mem_realloc( alloc_t* mem, void* p, ssize_t newsz ); +ic_private void mem_free( alloc_t* mem, const void* p ); +ic_private char* mem_strdup( alloc_t* mem, const char* s); +ic_private char* mem_strndup( alloc_t* mem, const char* s, ssize_t n); + +#define mem_zalloc_tp(mem,tp) (tp*)mem_zalloc(mem,ssizeof(tp)) +#define mem_malloc_tp_n(mem,tp,n) (tp*)mem_malloc(mem,(n)*ssizeof(tp)) +#define mem_zalloc_tp_n(mem,tp,n) (tp*)mem_zalloc(mem,(n)*ssizeof(tp)) +#define mem_realloc_tp(mem,tp,p,n) (tp*)mem_realloc(mem,p,(n)*ssizeof(tp)) + + +#endif // IC_COMMON_H diff --git a/extern/isocline/src/completers.c b/extern/isocline/src/completers.c new file mode 100644 index 00000000..e9701c16 --- /dev/null +++ b/extern/isocline/src/completers.c @@ -0,0 +1,675 @@ +/* ---------------------------------------------------------------------------- + Copyright (c) 2021, Daan Leijen + This is free software; you can redistribute it and/or modify it + under the terms of the MIT License. A copy of the license can be + found in the "LICENSE" file at the root of this distribution. +-----------------------------------------------------------------------------*/ +#include +#include + +#include "../include/isocline.h" +#include "common.h" +#include "env.h" +#include "stringbuf.h" +#include "completions.h" + + + +//------------------------------------------------------------- +// Word completion +//------------------------------------------------------------- + +// free variables for word completion +typedef struct word_closure_s { + long delete_before_adjust; + void* prev_env; + ic_completion_fun_t* prev_complete; +} word_closure_t; + + +// word completion callback +static bool token_add_completion_ex(ic_env_t* env, void* closure, const char* replacement, const char* display, const char* help, long delete_before, long delete_after) { + word_closure_t* wenv = (word_closure_t*)(closure); + // call the previous completer with an adjusted delete-before + return (*wenv->prev_complete)(env, wenv->prev_env, replacement, display, help, wenv->delete_before_adjust + delete_before, delete_after); +} + + +ic_public void ic_complete_word(ic_completion_env_t* cenv, const char* prefix, ic_completer_fun_t* fun, + ic_is_char_class_fun_t* is_word_char) +{ + if (is_word_char == NULL) is_word_char = &ic_char_is_nonseparator; + + ssize_t len = ic_strlen(prefix); + ssize_t pos = len; // will be start of the 'word' (excluding a potential start quote) + while (pos > 0) { + // go back one code point + ssize_t ofs = str_prev_ofs(prefix, pos, NULL); + if (ofs <= 0) break; + if (!(*is_word_char)(prefix + (pos - ofs), (long)ofs)) { + break; + } + pos -= ofs; + } + if (pos < 0) { pos = 0; } + + // stop if empty word + // if (len == pos) return; + + // set up the closure + word_closure_t wenv; + wenv.delete_before_adjust = (long)(len - pos); + wenv.prev_complete = cenv->complete; + wenv.prev_env = cenv->env; + cenv->complete = &token_add_completion_ex; + cenv->closure = &wenv; + + // and call the user completion routine + (*fun)(cenv, prefix + pos); + + // restore the original environment + cenv->complete = wenv.prev_complete; + cenv->closure = wenv.prev_env; +} + + +//------------------------------------------------------------- +// Quoted word completion (with escape characters) +//------------------------------------------------------------- + +// free variables for word completion +typedef struct qword_closure_s { + char escape_char; + char quote; + long delete_before_adjust; + stringbuf_t* sbuf; + void* prev_env; + ic_is_char_class_fun_t* is_word_char; + ic_completion_fun_t* prev_complete; +} qword_closure_t; + + +// word completion callback +static bool qword_add_completion_ex(ic_env_t* env, void* closure, const char* replacement, const char* display, const char* help, + long delete_before, long delete_after) { + qword_closure_t* wenv = (qword_closure_t*)(closure); + sbuf_replace( wenv->sbuf, replacement ); + if (wenv->quote != 0) { + // add end quote + sbuf_append_char( wenv->sbuf, wenv->quote); + } + else { + // escape non-word characters if it was not quoted + ssize_t pos = 0; + ssize_t next; + while ( (next = sbuf_next_ofs(wenv->sbuf, pos, NULL)) > 0 ) + { + if (!(*wenv->is_word_char)(sbuf_string(wenv->sbuf) + pos, (long)next)) { // strchr(wenv->non_word_char, sbuf_char_at( wenv->sbuf, pos )) != NULL) { + sbuf_insert_char_at( wenv->sbuf, wenv->escape_char, pos); + pos++; + } + pos += next; + } + } + // and call the previous completion function + return (*wenv->prev_complete)( env, wenv->prev_env, sbuf_string(wenv->sbuf), display, help, wenv->delete_before_adjust + delete_before, delete_after ); +} + + +ic_public void ic_complete_qword( ic_completion_env_t* cenv, const char* prefix, ic_completer_fun_t* fun, ic_is_char_class_fun_t* is_word_char ) { + ic_complete_qword_ex( cenv, prefix, fun, is_word_char, '\\', NULL); +} + + +ic_public void ic_complete_qword_ex( ic_completion_env_t* cenv, const char* prefix, ic_completer_fun_t* fun, + ic_is_char_class_fun_t* is_word_char, char escape_char, const char* quote_chars ) { + if (is_word_char == NULL) is_word_char = &ic_char_is_nonseparator ; + if (quote_chars == NULL) quote_chars = "'\""; + + ssize_t len = ic_strlen(prefix); + ssize_t pos; // will be start of the 'word' (excluding a potential start quote) + char quote = 0; + ssize_t quote_len = 0; + + // 1. look for a starting quote + if (quote_chars[0] != 0) { + // we go forward and count all quotes; if it is uneven, we need to complete quoted. + ssize_t qpos_open = -1; + ssize_t qpos_close = -1; + ssize_t qcount = 0; + pos = 0; + while(pos < len) { + if (prefix[pos] == escape_char && prefix[pos+1] != 0 && + !(*is_word_char)(prefix + pos + 1, 1)) // strchr(non_word_char, prefix[pos+1]) != NULL + { + pos++; // skip escape and next char + } + else if (qcount % 2 == 0 && strchr(quote_chars, prefix[pos]) != NULL) { + // open quote + qpos_open = pos; + quote = prefix[pos]; + qcount++; + } + else if (qcount % 2 == 1 && prefix[pos] == quote) { + // close quote + qpos_close = pos; + qcount++; + } + else if (!(*is_word_char)(prefix + pos, 1)) { // strchr(non_word_char, prefix[pos]) != NULL) { + qpos_close = -1; + } + ssize_t ofs = str_next_ofs( prefix, len, pos, NULL ); + if (ofs <= 0) break; + pos += ofs; + } + if ((qcount % 2 == 0 && qpos_close >= 0) || // if the last quote is only followed by word chars, we still complete it + (qcount % 2 == 1)) // opening quote found + { + quote_len = (len - qpos_open - 1); + pos = qpos_open + 1; // pos points to the word start just after the quote. + } + else { + quote = 0; + } + } + + // 2. if we did not find a quoted word, look for non-word-chars + if (quote == 0) { + pos = len; + while(pos > 0) { + // go back one code point + ssize_t ofs = str_prev_ofs(prefix, pos, NULL ); + if (ofs <= 0) break; + if (!(*is_word_char)(prefix + (pos - ofs), (long)ofs)) { // strchr(non_word_char, prefix[pos - ofs]) != NULL) { + // non word char, break if it is not escaped + if (pos <= ofs || prefix[pos - ofs - 1] != escape_char) break; + // otherwise go on + pos--; // skip escaped char + } + pos -= ofs; + } + } + + // stop if empty word + // if (len == pos) return; + + // allocate new unescaped word prefix + char* word = mem_strndup( cenv->env->mem, prefix + pos, (quote==0 ? len - pos : quote_len)); + if (word == NULL) return; + + if (quote == 0) { + // unescape prefix + ssize_t wlen = len - pos; + ssize_t wpos = 0; + while (wpos < wlen) { + ssize_t ofs = str_next_ofs(word, wlen, wpos, NULL); + if (ofs <= 0) break; + if (word[wpos] == escape_char && word[wpos+1] != 0 && + !(*is_word_char)(word + wpos + 1, (long)ofs)) // strchr(non_word_char, word[wpos+1]) != NULL) { + { + ic_memmove(word + wpos, word + wpos + 1, wlen - wpos /* including 0 */); + } + wpos += ofs; + } + } + #ifdef _WIN32 + else { + // remove inner quote: "c:\Program Files\"Win + ssize_t wlen = len - pos; + ssize_t wpos = 0; + while (wpos < wlen) { + ssize_t ofs = str_next_ofs(word, wlen, wpos, NULL); + if (ofs <= 0) break; + if (word[wpos] == escape_char && word[wpos+1] == quote) { + word[wpos+1] = escape_char; + ic_memmove(word + wpos, word + wpos + 1, wlen - wpos /* including 0 */); + } + wpos += ofs; + } + } + #endif + + // set up the closure + qword_closure_t wenv; + wenv.quote = quote; + wenv.is_word_char = is_word_char; + wenv.escape_char = escape_char; + wenv.delete_before_adjust = (long)(len - pos); + wenv.prev_complete = cenv->complete; + wenv.prev_env = cenv->env; + wenv.sbuf = sbuf_new(cenv->env->mem); + if (wenv.sbuf == NULL) { mem_free(cenv->env->mem, word); return; } + cenv->complete = &qword_add_completion_ex; + cenv->closure = &wenv; + + // and call the user completion routine + (*fun)( cenv, word ); + + // restore the original environment + cenv->complete = wenv.prev_complete; + cenv->closure = wenv.prev_env; + + sbuf_free(wenv.sbuf); + mem_free(cenv->env->mem, word); +} + + + + +//------------------------------------------------------------- +// Complete file names +// Listing files +//------------------------------------------------------------- +#include + +typedef enum file_type_e { + // must follow BSD style LSCOLORS order + FT_DEFAULT = 0, + FT_DIR, + FT_SYM, + FT_SOCK, + FT_PIPE, + FT_BLOCK, + FT_CHAR, + FT_SETUID, + FT_SETGID, + FT_DIR_OW_STICKY, + FT_DIR_OW, + FT_DIR_STICKY, + FT_EXE, + FT_LAST +} file_type_t; + +static int cli_color; // 1 enabled, 0 not initialized, -1 disabled +static const char* lscolors = "exfxcxdxbxegedabagacad"; // default BSD setting +static const char* ls_colors; +static const char* ls_colors_names[] = { "no=","di=","ln=","so=","pi=","bd=","cd=","su=","sg=","tw=","ow=","st=","ex=", NULL }; + +static bool ls_colors_init(void) { + if (cli_color != 0) return (cli_color >= 1); + // colors enabled? + const char* s = getenv("CLICOLOR"); + if (s==NULL || (strcmp(s, "1")!=0 && strcmp(s, "") != 0)) { + cli_color = -1; + return false; + } + cli_color = 1; + s = getenv("LS_COLORS"); + if (s != NULL) { ls_colors = s; } + s = getenv("LSCOLORS"); + if (s != NULL) { lscolors = s; } + return true; +} + +static bool ls_valid_esc(ssize_t c) { + return ((c==0 || c==1 || c==4 || c==7 || c==22 || c==24 || c==27) || + (c >= 30 && c <= 37) || (c >= 40 && c <= 47) || + (c >= 90 && c <= 97) || (c >= 100 && c <= 107)); +} + +static bool ls_colors_from_key(stringbuf_t* sb, const char* key) { + // find key + ssize_t keylen = ic_strlen(key); + if (keylen <= 0) return false; + const char* p = strstr(ls_colors, key); + if (p == NULL) return false; + p += keylen; + if (key[keylen-1] != '=') { + if (*p != '=') return false; + p++; + } + ssize_t len = 0; + while (p[len] != 0 && p[len] != ':') { + len++; + } + if (len <= 0) return false; + sbuf_append(sb, "[ansi-sgr=\"" ); + sbuf_append_n(sb, p, len ); + sbuf_append(sb, "\"]"); + return true; +} + +static int ls_colors_from_char(char c) { + if (c >= 'a' && c <= 'h') { return (c - 'a'); } + else if (c >= 'A' && c <= 'H') { return (c - 'A') + 8; } + else if (c == 'x') { return 256; } + else return 256; // default +} + +static bool ls_colors_append(stringbuf_t* sb, file_type_t ft, const char* ext) { + if (!ls_colors_init()) return false; + if (ls_colors != NULL) { + // GNU style + if (ft == FT_DEFAULT && ext != NULL) { + // first try extension match + if (ls_colors_from_key(sb, ext)) return true; + } + if (ft >= FT_DEFAULT && ft < FT_LAST) { + // then a filetype match + const char* key = ls_colors_names[ft]; + if (ls_colors_from_key(sb, key)) return true; + } + } + else if (lscolors != NULL) { + // BSD style + char fg = 'x'; + char bg = 'x'; + if (ic_strlen(lscolors) > (2*(ssize_t)ft)+1) { + fg = lscolors[2*ft]; + bg = lscolors[2*ft + 1]; + } + sbuf_appendf(sb, "[ansi-color=%d ansi-bgcolor=%d]", ls_colors_from_char(fg), ls_colors_from_char(bg) ); + return true; + } + return false; +} + +static void ls_colorize(bool no_lscolor, stringbuf_t* sb, file_type_t ft, const char* name, const char* ext, char dirsep) { + bool close = (no_lscolor ? false : ls_colors_append( sb, ft, ext)); + sbuf_append(sb, "[!pre]" ); + sbuf_append(sb, name); + if (dirsep != 0) sbuf_append_char(sb, dirsep); + sbuf_append(sb,"[/pre]" ); + if (close) { sbuf_append(sb, "[/]"); } +} + +#if defined(_WIN32) +#include +#include + +static bool os_is_dir(const char* cpath) { + struct _stat64 st = { 0 }; + _stat64(cpath, &st); + return ((st.st_mode & _S_IFDIR) != 0); +} + +static file_type_t os_get_filetype(const char* cpath) { + struct _stat64 st = { 0 }; + _stat64(cpath, &st); + if (((st.st_mode) & _S_IFDIR) != 0) return FT_DIR; + if (((st.st_mode) & _S_IFCHR) != 0) return FT_CHAR; + if (((st.st_mode) & _S_IFIFO) != 0) return FT_PIPE; + if (((st.st_mode) & _S_IEXEC) != 0) return FT_EXE; + return FT_DEFAULT; +} + + +#define dir_cursor intptr_t +#define dir_entry struct __finddata64_t + +static bool os_findfirst(alloc_t* mem, const char* path, dir_cursor* d, dir_entry* entry) { + stringbuf_t* spath = sbuf_new(mem); + if (spath == NULL) return false; + sbuf_append(spath, path); + sbuf_append(spath, "\\*"); + *d = _findfirsti64(sbuf_string(spath), entry); + mem_free(mem,spath); + return (*d != -1); +} + +static bool os_findnext(dir_cursor d, dir_entry* entry) { + return (_findnexti64(d, entry) == 0); +} + +static void os_findclose(dir_cursor d) { + _findclose(d); +} + +static const char* os_direntry_name(dir_entry* entry) { + return entry->name; +} + +static bool os_path_is_absolute( const char* path ) { + if (path != NULL && path[0] != 0 && path[1] == ':' && (path[2] == '\\' || path[2] == '/' || path[2] == 0)) { + char drive = path[0]; + return ((drive >= 'A' && drive <= 'Z') || (drive >= 'a' && drive <= 'z')); + } + else return false; +} + +ic_private char ic_dirsep(void) { + return '\\'; +} +#else + +#include +#include +#include +#include + +static bool os_is_dir(const char* cpath) { + struct stat st; + memset(&st, 0, sizeof(st)); + stat(cpath, &st); + return (S_ISDIR(st.st_mode)); +} + +static file_type_t os_get_filetype(const char* cpath) { + struct stat st; + memset(&st, 0, sizeof(st)); + lstat(cpath, &st); + switch ((st.st_mode)&S_IFMT) { + case S_IFSOCK: return FT_SOCK; + case S_IFLNK: { + return FT_SYM; + } + case S_IFIFO: return FT_PIPE; + case S_IFCHR: return FT_CHAR; + case S_IFBLK: return FT_BLOCK; + case S_IFDIR: { + if ((st.st_mode & S_ISUID) != 0) return FT_SETUID; + if ((st.st_mode & S_ISGID) != 0) return FT_SETGID; + if ((st.st_mode & S_IWGRP) != 0 && (st.st_mode & S_ISVTX) != 0) return FT_DIR_OW_STICKY; + if ((st.st_mode & S_IWGRP)) return FT_DIR_OW; + if ((st.st_mode & S_ISVTX)) return FT_DIR_STICKY; + return FT_DIR; + } + case S_IFREG: + default: { + if ((st.st_mode & S_IXUSR) != 0) return FT_EXE; + return FT_DEFAULT; + } + } +} + + +#define dir_cursor DIR* +#define dir_entry struct dirent* + +static bool os_findnext(dir_cursor d, dir_entry* entry) { + *entry = readdir(d); + return (*entry != NULL); +} + +static bool os_findfirst(alloc_t* mem, const char* cpath, dir_cursor* d, dir_entry* entry) { + ic_unused(mem); + *d = opendir(cpath); + if (*d == NULL) { + return false; + } + else { + return os_findnext(*d, entry); + } +} + +static void os_findclose(dir_cursor d) { + closedir(d); +} + +static const char* os_direntry_name(dir_entry* entry) { + return (*entry)->d_name; +} + +static bool os_path_is_absolute( const char* path ) { + return (path != NULL && path[0] == '/'); +} + +ic_private char ic_dirsep(void) { + return '/'; +} +#endif + + + +//------------------------------------------------------------- +// File completion +//------------------------------------------------------------- + +static bool ends_with_n(const char* name, ssize_t name_len, const char* ending, ssize_t len) { + if (name_len < len) return false; + if (ending == NULL || len <= 0) return true; + for (ssize_t i = 1; i <= len; i++) { + char c1 = name[name_len - i]; + char c2 = ending[len - i]; + #ifdef _WIN32 + if (ic_tolower(c1) != ic_tolower(c2)) return false; + #else + if (c1 != c2) return false; + #endif + } + return true; +} + +static bool match_extension(const char* name, const char* extensions) { + if (extensions == NULL || extensions[0] == 0) return true; + if (name == NULL) return false; + ssize_t name_len = ic_strlen(name); + ssize_t len = ic_strlen(extensions); + ssize_t cur = 0; + //debug_msg("match extensions: %s ~ %s", name, extensions); + for (ssize_t end = 0; end <= len; end++) { + if (extensions[end] == ';' || extensions[end] == 0) { + if (ends_with_n(name, name_len, extensions+cur, (end - cur))) { + return true; + } + cur = end+1; + } + } + return false; +} + +static bool filename_complete_indir( ic_completion_env_t* cenv, stringbuf_t* dir, + stringbuf_t* dir_prefix, stringbuf_t* display, + const char* base_prefix, + char dir_sep, const char* extensions ) +{ + dir_cursor d = 0; + dir_entry entry; + bool cont = true; + if (os_findfirst(cenv->env->mem, sbuf_string(dir), &d, &entry)) { + do { + const char* name = os_direntry_name(&entry); + if (name != NULL && strcmp(name, ".") != 0 && strcmp(name, "..") != 0 && + ic_istarts_with(name, base_prefix)) + { + // possible match, first check if it is a directory + file_type_t ft; + bool isdir; + const ssize_t plen = sbuf_len(dir_prefix); + sbuf_append(dir_prefix, name); + { // check directory and potentially add a dirsep to the dir_prefix + const ssize_t dlen = sbuf_len(dir); + sbuf_append_char(dir,ic_dirsep()); + sbuf_append(dir,name); + ft = os_get_filetype(sbuf_string(dir)); + isdir = os_is_dir(sbuf_string(dir)); + if (isdir && dir_sep != 0) { + sbuf_append_char(dir_prefix,dir_sep); + } + sbuf_delete_from(dir,dlen); // restore dir + } + if (isdir || match_extension(name, extensions)) { + // add completion + sbuf_clear(display); + ls_colorize(cenv->env->no_lscolors, display, ft, name, NULL, (isdir ? dir_sep : 0)); + cont = ic_add_completion_ex(cenv, sbuf_string(dir_prefix), sbuf_string(display), NULL); + } + sbuf_delete_from( dir_prefix, plen ); // restore dir_prefix + } + } while (cont && os_findnext(d, &entry)); + os_findclose(d); + } + return cont; +} + +typedef struct filename_closure_s { + const char* roots; + const char* extensions; + char dir_sep; +} filename_closure_t; + +static void filename_completer( ic_completion_env_t* cenv, const char* prefix ) { + if (prefix == NULL) return; + filename_closure_t* fclosure = (filename_closure_t*)cenv->arg; + stringbuf_t* root_dir = sbuf_new(cenv->env->mem); + stringbuf_t* dir_prefix = sbuf_new(cenv->env->mem); + stringbuf_t* display = sbuf_new(cenv->env->mem); + if (root_dir!=NULL && dir_prefix != NULL && display != NULL) + { + // split prefix in dir_prefix / base. + const char* base = strrchr(prefix,'/'); + #ifdef _WIN32 + const char* base2 = strrchr(prefix,'\\'); + if (base == NULL || base2 > base) base = base2; + #endif + if (base != NULL) { + base++; + sbuf_append_n(dir_prefix, prefix, base - prefix ); // includes dir separator + } + + // absolute path + if (os_path_is_absolute(prefix)) { + // do not use roots but try to complete directly + if (base != NULL) { + sbuf_append_n( root_dir, prefix, (base - prefix)); // include dir separator + } + filename_complete_indir( cenv, root_dir, dir_prefix, display, + (base != NULL ? base : prefix), + fclosure->dir_sep, fclosure->extensions ); + } + else { + // relative path, complete with respect to every root. + const char* next; + const char* root = fclosure->roots; + while ( root != NULL ) { + // create full root in `root_dir` + sbuf_clear(root_dir); + next = strchr(root,';'); + if (next == NULL) { + sbuf_append( root_dir, root ); + root = NULL; + } + else { + sbuf_append_n( root_dir, root, next - root ); + root = next + 1; + } + sbuf_append_char( root_dir, ic_dirsep()); + + // add the dir_prefix to the root + if (base != NULL) { + sbuf_append_n( root_dir, prefix, (base - prefix) - 1); + } + + // and complete in this directory + filename_complete_indir( cenv, root_dir, dir_prefix, display, + (base != NULL ? base : prefix), + fclosure->dir_sep, fclosure->extensions); + } + } + } + sbuf_free(display); + sbuf_free(root_dir); + sbuf_free(dir_prefix); +} + +ic_public void ic_complete_filename( ic_completion_env_t* cenv, const char* prefix, char dir_sep, const char* roots, const char* extensions ) { + if (roots == NULL) roots = "."; + if (extensions == NULL) extensions = ""; + if (dir_sep == 0) dir_sep = ic_dirsep(); + filename_closure_t fclosure; + fclosure.dir_sep = dir_sep; + fclosure.roots = roots; + fclosure.extensions = extensions; + cenv->arg = &fclosure; + ic_complete_qword_ex( cenv, prefix, &filename_completer, &ic_char_is_filename_letter, '\\', "'\""); +} diff --git a/extern/isocline/src/completions.c b/extern/isocline/src/completions.c new file mode 100644 index 00000000..01453efc --- /dev/null +++ b/extern/isocline/src/completions.c @@ -0,0 +1,326 @@ +/* ---------------------------------------------------------------------------- + Copyright (c) 2021, Daan Leijen + This is free software; you can redistribute it and/or modify it + under the terms of the MIT License. A copy of the license can be + found in the "LICENSE" file at the root of this distribution. +-----------------------------------------------------------------------------*/ +#include +#include +#include + +#include "../include/isocline.h" +#include "common.h" +#include "env.h" +#include "stringbuf.h" +#include "completions.h" + + +//------------------------------------------------------------- +// Completions +//------------------------------------------------------------- + +typedef struct completion_s { + const char* replacement; + const char* display; + const char* help; + ssize_t delete_before; + ssize_t delete_after; +} completion_t; + +struct completions_s { + ic_completer_fun_t* completer; + void* completer_arg; + ssize_t completer_max; + ssize_t count; + ssize_t len; + completion_t* elems; + alloc_t* mem; +}; + +static void default_filename_completer( ic_completion_env_t* cenv, const char* prefix ); + +ic_private completions_t* completions_new(alloc_t* mem) { + completions_t* cms = mem_zalloc_tp(mem, completions_t); + if (cms == NULL) return NULL; + cms->mem = mem; + cms->completer = &default_filename_completer; + return cms; +} + +ic_private void completions_free(completions_t* cms) { + if (cms == NULL) return; + completions_clear(cms); + if (cms->elems != NULL) { + mem_free(cms->mem, cms->elems); + cms->elems = NULL; + cms->count = 0; + cms->len = 0; + } + mem_free(cms->mem, cms); // free ourselves +} + + +ic_private void completions_clear(completions_t* cms) { + while (cms->count > 0) { + completion_t* cm = cms->elems + cms->count - 1; + mem_free( cms->mem, cm->display); + mem_free( cms->mem, cm->replacement); + mem_free( cms->mem, cm->help); + memset(cm,0,sizeof(*cm)); + cms->count--; + } +} + +static void completions_push(completions_t* cms, const char* replacement, const char* display, const char* help, ssize_t delete_before, ssize_t delete_after) +{ + if (cms->count >= cms->len) { + ssize_t newlen = (cms->len <= 0 ? 32 : cms->len*2); + completion_t* newelems = mem_realloc_tp(cms->mem, completion_t, cms->elems, newlen ); + if (newelems == NULL) return; + cms->elems = newelems; + cms->len = newlen; + } + assert(cms->count < cms->len); + completion_t* cm = cms->elems + cms->count; + cm->replacement = mem_strdup(cms->mem,replacement); + cm->display = mem_strdup(cms->mem,display); + cm->help = mem_strdup(cms->mem,help); + cm->delete_before = delete_before; + cm->delete_after = delete_after; + cms->count++; +} + +ic_private ssize_t completions_count(completions_t* cms) { + return cms->count; +} + +static bool completions_contains(completions_t* cms, const char* replacement) { + for( ssize_t i = 0; i < cms->count; i++ ) { + const completion_t* c = cms->elems + i; + if (strcmp(replacement,c->replacement) == 0) { return true; } + } + return false; +} + +ic_private bool completions_add(completions_t* cms, const char* replacement, const char* display, const char* help, ssize_t delete_before, ssize_t delete_after) { + if (cms->completer_max <= 0) return false; + cms->completer_max--; + //debug_msg("completion: add: %d,%d, %s\n", delete_before, delete_after, replacement); + if (!completions_contains(cms,replacement)) { + completions_push(cms, replacement, display, help, delete_before, delete_after); + } + return true; +} + +static completion_t* completions_get(completions_t* cms, ssize_t index) { + if (index < 0 || cms->count <= 0 || index >= cms->count) return NULL; + return &cms->elems[index]; +} + +ic_private const char* completions_get_display( completions_t* cms, ssize_t index, const char** help ) { + if (help != NULL) { *help = NULL; } + completion_t* cm = completions_get(cms, index); + if (cm == NULL) return NULL; + if (help != NULL) { *help = cm->help; } + return (cm->display != NULL ? cm->display : cm->replacement); +} + +ic_private const char* completions_get_help( completions_t* cms, ssize_t index ) { + completion_t* cm = completions_get(cms, index); + if (cm == NULL) return NULL; + return cm->help; +} + +ic_private const char* completions_get_hint(completions_t* cms, ssize_t index, const char** help) { + if (help != NULL) { *help = NULL; } + completion_t* cm = completions_get(cms, index); + if (cm == NULL) return NULL; + ssize_t len = ic_strlen(cm->replacement); + if (len < cm->delete_before) return NULL; + const char* hint = (cm->replacement + cm->delete_before); + if (*hint == 0 || utf8_is_cont((uint8_t)(*hint))) return NULL; // utf8 boundary? + if (help != NULL) { *help = cm->help; } + return hint; +} + +ic_private void completions_set_completer(completions_t* cms, ic_completer_fun_t* completer, void* arg) { + cms->completer = completer; + cms->completer_arg = arg; +} + +ic_private void completions_get_completer(completions_t* cms, ic_completer_fun_t** completer, void** arg) { + *completer = cms->completer; + *arg = cms->completer_arg; +} + + +ic_public void* ic_completion_arg( const ic_completion_env_t* cenv ) { + return (cenv == NULL ? NULL : cenv->env->completions->completer_arg); +} + +ic_public bool ic_has_completions( const ic_completion_env_t* cenv ) { + return (cenv == NULL ? false : cenv->env->completions->count > 0); +} + +ic_public bool ic_stop_completing( const ic_completion_env_t* cenv) { + return (cenv == NULL ? true : cenv->env->completions->completer_max <= 0); +} + + +static ssize_t completion_apply( completion_t* cm, stringbuf_t* sbuf, ssize_t pos ) { + if (cm == NULL) return -1; + debug_msg( "completion: apply: %s at %zd\n", cm->replacement, pos); + ssize_t start = pos - cm->delete_before; + if (start < 0) start = 0; + ssize_t n = cm->delete_before + cm->delete_after; + if (ic_strlen(cm->replacement) == n && strncmp(sbuf_string_at(sbuf,start), cm->replacement, to_size_t(n)) == 0) { + // no changes + return -1; + } + else { + sbuf_delete_from_to( sbuf, start, pos + cm->delete_after ); + return sbuf_insert_at(sbuf, cm->replacement, start); + } +} + +ic_private ssize_t completions_apply( completions_t* cms, ssize_t index, stringbuf_t* sbuf, ssize_t pos ) { + completion_t* cm = completions_get(cms, index); + return completion_apply( cm, sbuf, pos ); +} + + +static int completion_compare(const void* p1, const void* p2) { + if (p1 == NULL || p2 == NULL) return 0; + const completion_t* cm1 = (const completion_t*)p1; + const completion_t* cm2 = (const completion_t*)p2; + return ic_stricmp(cm1->replacement, cm2->replacement); +} + +ic_private void completions_sort(completions_t* cms) { + if (cms->count <= 0) return; + qsort(cms->elems, to_size_t(cms->count), sizeof(cms->elems[0]), &completion_compare); +} + +#define IC_MAX_PREFIX (256) + +// find longest common prefix and complete with that. +ic_private ssize_t completions_apply_longest_prefix(completions_t* cms, stringbuf_t* sbuf, ssize_t pos) { + if (cms->count <= 1) { + return completions_apply(cms,0,sbuf,pos); + } + + // set initial prefix to the first entry + completion_t* cm = completions_get(cms, 0); + if (cm == NULL) return -1; + + char prefix[IC_MAX_PREFIX+1]; + ssize_t delete_before = cm->delete_before; + ic_strncpy( prefix, IC_MAX_PREFIX+1, cm->replacement, IC_MAX_PREFIX ); + prefix[IC_MAX_PREFIX] = 0; + + // and visit all others to find the longest common prefix + for(ssize_t i = 1; i < cms->count; i++) { + cm = completions_get(cms,i); + if (cm->delete_before != delete_before) { // deletions must match delete_before + prefix[0] = 0; + break; + } + // check if it is still a prefix + const char* r = cm->replacement; + ssize_t j; + for(j = 0; prefix[j] != 0 && r[j] != 0; j++) { + if (prefix[j] != r[j]) break; + } + prefix[j] = 0; + if (j <= 0) break; + } + + // check the length + ssize_t len = ic_strlen(prefix); + if (len <= 0 || len < delete_before) return -1; + + // we found a prefix :-) + completion_t cprefix; + memset(&cprefix,0,sizeof(cprefix)); + cprefix.delete_before = delete_before; + cprefix.replacement = prefix; + ssize_t newpos = completion_apply( &cprefix, sbuf, pos); + if (newpos < 0) return newpos; + + // adjust all delete_before for the new replacement + for( ssize_t i = 0; i < cms->count; i++) { + cm = completions_get(cms,i); + cm->delete_before = len; + } + + return newpos; +} + + +//------------------------------------------------------------- +// Completer functions +//------------------------------------------------------------- + +ic_public bool ic_add_completions(ic_completion_env_t* cenv, const char* prefix, const char** completions) { + for (const char** pc = completions; *pc != NULL; pc++) { + if (ic_istarts_with(*pc, prefix)) { + if (!ic_add_completion_ex(cenv, *pc, NULL, NULL)) return false; + } + } + return true; +} + +ic_public bool ic_add_completion(ic_completion_env_t* cenv, const char* replacement) { + return ic_add_completion_ex(cenv, replacement, NULL, NULL); +} + +ic_public bool ic_add_completion_ex( ic_completion_env_t* cenv, const char* replacement, const char* display, const char* help ) { + return ic_add_completion_prim(cenv,replacement,display,help,0,0); +} + +ic_public bool ic_add_completion_prim(ic_completion_env_t* cenv, const char* replacement, const char* display, const char* help, long delete_before, long delete_after) { + return (*cenv->complete)(cenv->env, cenv->closure, replacement, display, help, delete_before, delete_after ); +} + +static bool prim_add_completion(ic_env_t* env, void* funenv, const char* replacement, const char* display, const char* help, long delete_before, long delete_after) { + ic_unused(funenv); + return completions_add(env->completions, replacement, display, help, delete_before, delete_after); +} + +ic_public void ic_set_default_completer(ic_completer_fun_t* completer, void* arg) { + ic_env_t* env = ic_get_env(); if (env == NULL) return; + completions_set_completer(env->completions, completer, arg); +} + +ic_private ssize_t completions_generate(struct ic_env_s* env, completions_t* cms, const char* input, ssize_t pos, ssize_t max) { + completions_clear(cms); + if (cms->completer == NULL || input == NULL || ic_strlen(input) < pos) return 0; + + // set up env + ic_completion_env_t cenv; + cenv.env = env; + cenv.input = input, + cenv.cursor = (long)pos; + cenv.arg = cms->completer_arg; + cenv.complete = &prim_add_completion; + cenv.closure = NULL; + const char* prefix = mem_strndup(cms->mem, input, pos); + cms->completer_max = max; + + // and complete + cms->completer(&cenv,prefix); + + // restore + mem_free(cms->mem,prefix); + return completions_count(cms); +} + +// The default completer is no completion is set +static void default_filename_completer( ic_completion_env_t* cenv, const char* prefix ) { + #ifdef _WIN32 + const char sep = '\\'; + #else + const char sep = '/'; + #endif + ic_complete_filename( cenv, prefix, sep, ".", NULL); +} diff --git a/extern/isocline/src/completions.h b/extern/isocline/src/completions.h new file mode 100644 index 00000000..8361d507 --- /dev/null +++ b/extern/isocline/src/completions.h @@ -0,0 +1,52 @@ +/* ---------------------------------------------------------------------------- + Copyright (c) 2021, Daan Leijen + This is free software; you can redistribute it and/or modify it + under the terms of the MIT License. A copy of the license can be + found in the "LICENSE" file at the root of this distribution. +-----------------------------------------------------------------------------*/ +#pragma once +#ifndef IC_COMPLETIONS_H +#define IC_COMPLETIONS_H + +#include "common.h" +#include "stringbuf.h" + + +//------------------------------------------------------------- +// Completions +//------------------------------------------------------------- +#define IC_MAX_COMPLETIONS_TO_SHOW (1000) +#define IC_MAX_COMPLETIONS_TO_TRY (IC_MAX_COMPLETIONS_TO_SHOW/4) + +typedef struct completions_s completions_t; + +ic_private completions_t* completions_new(alloc_t* mem); +ic_private void completions_free(completions_t* cms); +ic_private void completions_clear(completions_t* cms); +ic_private bool completions_add(completions_t* cms , const char* replacement, const char* display, const char* help, ssize_t delete_before, ssize_t delete_after); +ic_private ssize_t completions_count(completions_t* cms); +ic_private ssize_t completions_generate(struct ic_env_s* env, completions_t* cms , const char* input, ssize_t pos, ssize_t max); +ic_private void completions_sort(completions_t* cms); +ic_private void completions_set_completer(completions_t* cms, ic_completer_fun_t* completer, void* arg); +ic_private const char* completions_get_display(completions_t* cms , ssize_t index, const char** help); +ic_private const char* completions_get_hint(completions_t* cms, ssize_t index, const char** help); +ic_private void completions_get_completer(completions_t* cms, ic_completer_fun_t** completer, void** arg); + +ic_private ssize_t completions_apply(completions_t* cms, ssize_t index, stringbuf_t* sbuf, ssize_t pos); +ic_private ssize_t completions_apply_longest_prefix(completions_t* cms, stringbuf_t* sbuf, ssize_t pos); + +//------------------------------------------------------------- +// Completion environment +//------------------------------------------------------------- +typedef bool (ic_completion_fun_t)( ic_env_t* env, void* funenv, const char* replacement, const char* display, const char* help, long delete_before, long delete_after ); + +struct ic_completion_env_s { + ic_env_t* env; // the isocline environment + const char* input; // current full input + long cursor; // current cursor position + void* arg; // argument given to `ic_set_completer` + void* closure; // free variables for function composition + ic_completion_fun_t* complete; // function that adds a completion +}; + +#endif // IC_COMPLETIONS_H diff --git a/extern/isocline/src/editline.c b/extern/isocline/src/editline.c new file mode 100644 index 00000000..270c42d9 --- /dev/null +++ b/extern/isocline/src/editline.c @@ -0,0 +1,1142 @@ +/* ---------------------------------------------------------------------------- + Copyright (c) 2021, Daan Leijen + This is free software; you can redistribute it and/or modify it + under the terms of the MIT License. A copy of the license can be + found in the "LICENSE" file at the root of this distribution. +-----------------------------------------------------------------------------*/ +#include +#include + +#include "common.h" +#include "term.h" +#include "tty.h" +#include "env.h" +#include "stringbuf.h" +#include "history.h" +#include "completions.h" +#include "undo.h" +#include "highlight.h" + +//------------------------------------------------------------- +// The editor state +//------------------------------------------------------------- + + + +// editor state +typedef struct editor_s { + stringbuf_t* input; // current user input + stringbuf_t* extra; // extra displayed info (for completion menu etc) + stringbuf_t* hint; // hint displayed as part of the input + stringbuf_t* hint_help; // help for a hint. + ssize_t pos; // current cursor position in the input + ssize_t cur_rows; // current used rows to display our content (including extra content) + ssize_t cur_row; // current row that has the cursor (0 based, relative to the prompt) + ssize_t termw; + bool modified; // has a modification happened? (used for history navigation for example) + bool disable_undo; // temporarily disable auto undo (for history search) + ssize_t history_idx; // current index in the history + editstate_t* undo; // undo buffer + editstate_t* redo; // redo buffer + const char* prompt_text; // text of the prompt before the prompt marker + alloc_t* mem; // allocator + // caches + attrbuf_t* attrs; // reuse attribute buffers + attrbuf_t* attrs_extra; +} editor_t; + + + + + +//------------------------------------------------------------- +// Main edit line +//------------------------------------------------------------- +static char* edit_line( ic_env_t* env, const char* prompt_text ); // defined at bottom +static void edit_refresh(ic_env_t* env, editor_t* eb); + +ic_private char* ic_editline(ic_env_t* env, const char* prompt_text) { + tty_start_raw(env->tty); + term_start_raw(env->term); + char* line = edit_line(env,prompt_text); + term_end_raw(env->term,false); + tty_end_raw(env->tty); + term_writeln(env->term,""); + term_flush(env->term); + return line; +} + + +//------------------------------------------------------------- +// Undo/Redo +//------------------------------------------------------------- + +// capture the current edit state +static void editor_capture(editor_t* eb, editstate_t** es ) { + if (!eb->disable_undo) { + editstate_capture( eb->mem, es, sbuf_string(eb->input), eb->pos ); + } +} + +static void editor_undo_capture(editor_t* eb ) { + editor_capture(eb, &eb->undo ); +} + +static void editor_undo_forget(editor_t* eb) { + if (eb->disable_undo) return; + const char* input = NULL; + ssize_t pos = 0; + editstate_restore(eb->mem, &eb->undo, &input, &pos); + mem_free(eb->mem, input); +} + +static void editor_restore(editor_t* eb, editstate_t** from, editstate_t** to ) { + if (eb->disable_undo) return; + if (*from == NULL) return; + const char* input; + if (to != NULL) { editor_capture( eb, to ); } + if (!editstate_restore( eb->mem, from, &input, &eb->pos )) return; + sbuf_replace( eb->input, input ); + mem_free(eb->mem, input); + eb->modified = false; +} + +static void editor_undo_restore(editor_t* eb, bool with_redo ) { + editor_restore(eb, &eb->undo, (with_redo ? &eb->redo : NULL)); +} + +static void editor_redo_restore(editor_t* eb ) { + editor_restore(eb, &eb->redo, &eb->undo); + eb->modified = false; +} + +static void editor_start_modify(editor_t* eb ) { + editor_undo_capture(eb); + editstate_done(eb->mem, &eb->redo); // clear redo + eb->modified = true; +} + + + +static bool editor_pos_is_at_end(editor_t* eb ) { + return (eb->pos == sbuf_len(eb->input)); +} + +//------------------------------------------------------------- +// Row/Column width and positioning +//------------------------------------------------------------- + + +static void edit_get_prompt_width( ic_env_t* env, editor_t* eb, bool in_extra, ssize_t* promptw, ssize_t* cpromptw ) { + if (in_extra) { + *promptw = 0; + *cpromptw = 0; + } + else { + // todo: cache prompt widths + ssize_t textw = bbcode_column_width(env->bbcode, eb->prompt_text); + ssize_t markerw = bbcode_column_width(env->bbcode, env->prompt_marker); + ssize_t cmarkerw = bbcode_column_width(env->bbcode, env->cprompt_marker); + *promptw = markerw + textw; + *cpromptw = (env->no_multiline_indent || *promptw < cmarkerw ? cmarkerw : *promptw); + } +} + +static ssize_t edit_get_rowcol( ic_env_t* env, editor_t* eb, rowcol_t* rc ) { + ssize_t promptw, cpromptw; + edit_get_prompt_width(env, eb, false, &promptw, &cpromptw); + return sbuf_get_rc_at_pos( eb->input, eb->termw, promptw, cpromptw, eb->pos, rc ); +} + +static void edit_set_pos_at_rowcol( ic_env_t* env, editor_t* eb, ssize_t row, ssize_t col ) { + ssize_t promptw, cpromptw; + edit_get_prompt_width(env, eb, false, &promptw, &cpromptw); + ssize_t pos = sbuf_get_pos_at_rc( eb->input, eb->termw, promptw, cpromptw, row, col ); + if (pos < 0) return; + eb->pos = pos; + edit_refresh(env, eb); +} + +static bool edit_pos_is_at_row_end( ic_env_t* env, editor_t* eb ) { + rowcol_t rc; + edit_get_rowcol( env, eb, &rc ); + return rc.last_on_row; +} + +static void edit_write_prompt( ic_env_t* env, editor_t* eb, ssize_t row, bool in_extra ) { + if (in_extra) return; + bbcode_style_open(env->bbcode, "ic-prompt"); + if (row==0) { + // regular prompt text + bbcode_print( env->bbcode, eb->prompt_text ); + } + else if (!env->no_multiline_indent) { + // multiline continuation indentation + // todo: cache prompt widths + ssize_t textw = bbcode_column_width(env->bbcode, eb->prompt_text ); + ssize_t markerw = bbcode_column_width(env->bbcode, env->prompt_marker); + ssize_t cmarkerw = bbcode_column_width(env->bbcode, env->cprompt_marker); + if (cmarkerw < markerw + textw) { + term_write_repeat(env->term, " ", markerw + textw - cmarkerw ); + } + } + // the marker + bbcode_print(env->bbcode, (row == 0 ? env->prompt_marker : env->cprompt_marker )); + bbcode_style_close(env->bbcode,NULL); +} + +//------------------------------------------------------------- +// Refresh +//------------------------------------------------------------- + +typedef struct refresh_info_s { + ic_env_t* env; + editor_t* eb; + attrbuf_t* attrs; + bool in_extra; + ssize_t first_row; + ssize_t last_row; +} refresh_info_t; + +static bool edit_refresh_rows_iter( + const char* s, + ssize_t row, ssize_t row_start, ssize_t row_len, + ssize_t startw, bool is_wrap, const void* arg, void* res) +{ + ic_unused(res); ic_unused(startw); + const refresh_info_t* info = (const refresh_info_t*)(arg); + term_t* term = info->env->term; + + // debug_msg("edit: line refresh: row %zd, len: %zd\n", row, row_len); + if (row < info->first_row) return false; + if (row > info->last_row) return true; // should not occur + + // term_clear_line(term); + edit_write_prompt(info->env, info->eb, row, info->in_extra); + + //' write output + if (info->attrs == NULL || (info->env->no_highlight && info->env->no_bracematch)) { + term_write_n( term, s + row_start, row_len ); + } + else { + term_write_formatted_n( term, s + row_start, attrbuf_attrs(info->attrs, row_start + row_len) + row_start, row_len ); + } + + // write line ending + if (row < info->last_row) { + if (is_wrap && tty_is_utf8(info->env->tty)) { + #ifndef __APPLE__ + bbcode_print( info->env->bbcode, "[ic-dim]\xE2\x86\x90"); // left arrow + #else + bbcode_print( info->env->bbcode, "[ic-dim]\xE2\x86\xB5" ); // return symbol + #endif + } + term_clear_to_end_of_line(term); + term_writeln(term, ""); + } + else { + term_clear_to_end_of_line(term); + } + return (row >= info->last_row); +} + +static void edit_refresh_rows(ic_env_t* env, editor_t* eb, stringbuf_t* input, attrbuf_t* attrs, + ssize_t promptw, ssize_t cpromptw, bool in_extra, + ssize_t first_row, ssize_t last_row) +{ + if (input == NULL) return; + refresh_info_t info; + info.env = env; + info.eb = eb; + info.attrs = attrs; + info.in_extra = in_extra; + info.first_row = first_row; + info.last_row = last_row; + sbuf_for_each_row( input, eb->termw, promptw, cpromptw, &edit_refresh_rows_iter, &info, NULL); +} + + +static void edit_refresh(ic_env_t* env, editor_t* eb) +{ + // calculate the new cursor row and total rows needed + ssize_t promptw, cpromptw; + edit_get_prompt_width( env, eb, false, &promptw, &cpromptw ); + + if (eb->attrs != NULL) { + highlight( env->mem, env->bbcode, sbuf_string(eb->input), eb->attrs, + (env->no_highlight ? NULL : env->highlighter), env->highlighter_arg ); + } + + // highlight matching braces + if (eb->attrs != NULL && !env->no_bracematch) { + highlight_match_braces(sbuf_string(eb->input), eb->attrs, eb->pos, ic_env_get_match_braces(env), + bbcode_style(env->bbcode,"ic-bracematch"), bbcode_style(env->bbcode,"ic-error")); + } + + // insert hint + if (sbuf_len(eb->hint) > 0) { + if (eb->attrs != NULL) { + attrbuf_insert_at( eb->attrs, eb->pos, sbuf_len(eb->hint), bbcode_style(env->bbcode, "ic-hint") ); + } + sbuf_insert_at(eb->input, sbuf_string(eb->hint), eb->pos ); + } + + // render extra (like a completion menu) + stringbuf_t* extra = NULL; + if (sbuf_len(eb->extra) > 0) { + extra = sbuf_new(eb->mem); + if (extra != NULL) { + if (sbuf_len(eb->hint_help) > 0) { + bbcode_append(env->bbcode, sbuf_string(eb->hint_help), extra, eb->attrs_extra); + } + bbcode_append(env->bbcode, sbuf_string(eb->extra), extra, eb->attrs_extra); + } + } + + // calculate rows and row/col position + rowcol_t rc = { 0 }; + const ssize_t rows_input = sbuf_get_rc_at_pos( eb->input, eb->termw, promptw, cpromptw, eb->pos, &rc ); + rowcol_t rc_extra = { 0 }; + ssize_t rows_extra = 0; + if (extra != NULL) { + rows_extra = sbuf_get_rc_at_pos( extra, eb->termw, 0, 0, 0 /*pos*/, &rc_extra ); + } + const ssize_t rows = rows_input + rows_extra; + debug_msg("edit: refresh: rows %zd, cursor: %zd,%zd (previous rows %zd, cursor row %zd)\n", rows, rc.row, rc.col, eb->cur_rows, eb->cur_row); + + // only render at most terminal height rows + const ssize_t termh = term_get_height(env->term); + ssize_t first_row = 0; // first visible row + ssize_t last_row = rows - 1; // last visible row + if (rows > termh) { + first_row = rc.row - termh + 1; // ensure cursor is visible + if (first_row < 0) first_row = 0; + last_row = first_row + termh - 1; + } + assert(last_row - first_row < termh); + + // reduce flicker + buffer_mode_t bmode = term_set_buffer_mode(env->term, BUFFERED); + + // back up to the first line + term_start_of_line(env->term); + term_up(env->term, (eb->cur_row >= termh ? termh-1 : eb->cur_row) ); + // term_clear_lines_to_end(env->term); // gives flicker in old Windows cmd prompt + + // render rows + edit_refresh_rows( env, eb, eb->input, eb->attrs, promptw, cpromptw, false, first_row, last_row ); + if (rows_extra > 0) { + assert(extra != NULL); + const ssize_t first_rowx = (first_row > rows_input ? first_row - rows_input : 0); + const ssize_t last_rowx = last_row - rows_input; assert(last_rowx >= 0); + edit_refresh_rows(env, eb, extra, eb->attrs_extra, 0, 0, true, first_rowx, last_rowx); + } + + // overwrite trailing rows we do not use anymore + ssize_t rrows = last_row - first_row + 1; // rendered rows + if (rrows < termh && rows < eb->cur_rows) { + ssize_t clear = eb->cur_rows - rows; + while (rrows < termh && clear > 0) { + clear--; + rrows++; + term_writeln(env->term,""); + term_clear_line(env->term); + } + } + + // move cursor back to edit position + term_start_of_line(env->term); + term_up(env->term, first_row + rrows - 1 - rc.row ); + term_right(env->term, rc.col + (rc.row == 0 ? promptw : cpromptw)); + + // and refresh + term_flush(env->term); + + // stop buffering + term_set_buffer_mode(env->term, bmode); + + // restore input by removing the hint + sbuf_delete_at(eb->input, eb->pos, sbuf_len(eb->hint)); + sbuf_delete_at(eb->extra, 0, sbuf_len(eb->hint_help)); + attrbuf_clear(eb->attrs); + attrbuf_clear(eb->attrs_extra); + sbuf_free(extra); + + // update previous + eb->cur_rows = rows; + eb->cur_row = rc.row; +} + +// clear current output +static void edit_clear(ic_env_t* env, editor_t* eb ) { + term_attr_reset(env->term); + term_up(env->term, eb->cur_row); + + // overwrite all rows + for( ssize_t i = 0; i < eb->cur_rows; i++) { + term_clear_line(env->term); + term_writeln(env->term, ""); + } + + // move cursor back + term_up(env->term, eb->cur_rows - eb->cur_row ); +} + + +// clear screen and refresh +static void edit_clear_screen(ic_env_t* env, editor_t* eb ) { + ssize_t cur_rows = eb->cur_rows; + eb->cur_rows = term_get_height(env->term) - 1; + edit_clear(env,eb); + eb->cur_rows = cur_rows; + edit_refresh(env,eb); +} + + +// refresh after a terminal window resized (but before doing further edit operations!) +static bool edit_resize(ic_env_t* env, editor_t* eb ) { + // update dimensions + term_update_dim(env->term); + ssize_t newtermw = term_get_width(env->term); + if (eb->termw == newtermw) return false; + + // recalculate the row layout assuming the hardwrapping for the new terminal width + ssize_t promptw, cpromptw; + edit_get_prompt_width( env, eb, false, &promptw, &cpromptw ); + sbuf_insert_at(eb->input, sbuf_string(eb->hint), eb->pos); // insert used hint + + // render extra (like a completion menu) + stringbuf_t* extra = NULL; + if (sbuf_len(eb->extra) > 0) { + extra = sbuf_new(eb->mem); + if (extra != NULL) { + if (sbuf_len(eb->hint_help) > 0) { + bbcode_append(env->bbcode, sbuf_string(eb->hint_help), extra, NULL); + } + bbcode_append(env->bbcode, sbuf_string(eb->extra), extra, NULL); + } + } + rowcol_t rc = { 0 }; + const ssize_t rows_input = sbuf_get_wrapped_rc_at_pos( eb->input, eb->termw, newtermw, promptw, cpromptw, eb->pos, &rc ); + rowcol_t rc_extra = { 0 }; + ssize_t rows_extra = 0; + if (extra != NULL) { + rows_extra = sbuf_get_wrapped_rc_at_pos(extra, eb->termw, newtermw, 0, 0, 0 /*pos*/, &rc_extra); + } + ssize_t rows = rows_input + rows_extra; + debug_msg("edit: resize: new rows: %zd, cursor row: %zd (previous: rows: %zd, cursor row %zd)\n", rows, rc.row, eb->cur_rows, eb->cur_row); + + // update the newly calculated row and rows + eb->cur_row = rc.row; + if (rows > eb->cur_rows) { + eb->cur_rows = rows; + } + eb->termw = newtermw; + edit_refresh(env,eb); + + // remove hint again + sbuf_delete_at(eb->input, eb->pos, sbuf_len(eb->hint)); + sbuf_free(extra); + return true; +} + +static void editor_append_hint_help(editor_t* eb, const char* help) { + sbuf_clear(eb->hint_help); + if (help != NULL) { + sbuf_replace(eb->hint_help, "[ic-info]"); + sbuf_append(eb->hint_help, help); + sbuf_append(eb->hint_help, "[/ic-info]\n"); + } +} + +// refresh with possible hint +static void edit_refresh_hint(ic_env_t* env, editor_t* eb) { + if (env->no_hint || env->hint_delay > 0) { + // refresh without hint first + edit_refresh(env, eb); + if (env->no_hint) return; + } + + // and see if we can construct a hint (displayed after a delay) + ssize_t count = completions_generate(env, env->completions, sbuf_string(eb->input), eb->pos, 2); + if (count == 1) { + const char* help = NULL; + const char* hint = completions_get_hint(env->completions, 0, &help); + if (hint != NULL) { + sbuf_replace(eb->hint, hint); + editor_append_hint_help(eb, help); + // do auto-tabbing? + if (env->complete_autotab) { + stringbuf_t* sb = sbuf_new(env->mem); // temporary buffer for completion + if (sb != NULL) { + sbuf_replace( sb, sbuf_string(eb->input) ); + ssize_t pos = eb->pos; + const char* extra_hint = hint; + do { + ssize_t newpos = sbuf_insert_at( sb, extra_hint, pos ); + if (newpos <= pos) break; + pos = newpos; + count = completions_generate(env, env->completions, sbuf_string(sb), pos, 2); + if (count == 1) { + const char* extra_help = NULL; + extra_hint = completions_get_hint(env->completions, 0, &extra_help); + if (extra_hint != NULL) { + editor_append_hint_help(eb, extra_help); + sbuf_append(eb->hint, extra_hint); + } + } + } + while(count == 1); + sbuf_free(sb); + } + } + } + } + + if (env->hint_delay <= 0) { + // refresh with hint directly + edit_refresh(env, eb); + } +} + +//------------------------------------------------------------- +// Edit operations +//------------------------------------------------------------- + +static void edit_history_prev(ic_env_t* env, editor_t* eb); +static void edit_history_next(ic_env_t* env, editor_t* eb); + +static void edit_undo_restore(ic_env_t* env, editor_t* eb) { + editor_undo_restore(eb, true); + edit_refresh(env,eb); +} + +static void edit_redo_restore(ic_env_t* env, editor_t* eb) { + editor_redo_restore(eb); + edit_refresh(env,eb); +} + +static void edit_cursor_left(ic_env_t* env, editor_t* eb) { + ssize_t cwidth = 1; + ssize_t prev = sbuf_prev(eb->input,eb->pos,&cwidth); + if (prev < 0) return; + rowcol_t rc; + edit_get_rowcol( env, eb, &rc); + eb->pos = prev; + edit_refresh(env,eb); +} + +static void edit_cursor_right(ic_env_t* env, editor_t* eb) { + ssize_t cwidth = 1; + ssize_t next = sbuf_next(eb->input,eb->pos,&cwidth); + if (next < 0) return; + rowcol_t rc; + edit_get_rowcol( env, eb, &rc); + eb->pos = next; + edit_refresh(env,eb); +} + +static void edit_cursor_line_end(ic_env_t* env, editor_t* eb) { + ssize_t end = sbuf_find_line_end(eb->input,eb->pos); + if (end < 0) return; + eb->pos = end; + edit_refresh(env,eb); +} + +static void edit_cursor_line_start(ic_env_t* env, editor_t* eb) { + ssize_t start = sbuf_find_line_start(eb->input,eb->pos); + if (start < 0) return; + eb->pos = start; + edit_refresh(env,eb); +} + +static void edit_cursor_next_word(ic_env_t* env, editor_t* eb) { + ssize_t end = sbuf_find_word_end(eb->input,eb->pos); + if (end < 0) return; + eb->pos = end; + edit_refresh(env,eb); +} + +static void edit_cursor_prev_word(ic_env_t* env, editor_t* eb) { + ssize_t start = sbuf_find_word_start(eb->input,eb->pos); + if (start < 0) return; + eb->pos = start; + edit_refresh(env,eb); +} + +static void edit_cursor_next_ws_word(ic_env_t* env, editor_t* eb) { + ssize_t end = sbuf_find_ws_word_end(eb->input, eb->pos); + if (end < 0) return; + eb->pos = end; + edit_refresh(env, eb); +} + +static void edit_cursor_prev_ws_word(ic_env_t* env, editor_t* eb) { + ssize_t start = sbuf_find_ws_word_start(eb->input, eb->pos); + if (start < 0) return; + eb->pos = start; + edit_refresh(env, eb); +} + +static void edit_cursor_to_start(ic_env_t* env, editor_t* eb) { + eb->pos = 0; + edit_refresh(env,eb); +} + +static void edit_cursor_to_end(ic_env_t* env, editor_t* eb) { + eb->pos = sbuf_len(eb->input); + edit_refresh(env,eb); +} + + +static void edit_cursor_row_up(ic_env_t* env, editor_t* eb) { + rowcol_t rc; + edit_get_rowcol( env, eb, &rc); + if (rc.row == 0) { + edit_history_prev(env,eb); + } + else { + edit_set_pos_at_rowcol( env, eb, rc.row - 1, rc.col ); + } +} + +static void edit_cursor_row_down(ic_env_t* env, editor_t* eb) { + rowcol_t rc; + ssize_t rows = edit_get_rowcol( env, eb, &rc); + if (rc.row + 1 >= rows) { + edit_history_next(env,eb); + } + else { + edit_set_pos_at_rowcol( env, eb, rc.row + 1, rc.col ); + } +} + + +static void edit_cursor_match_brace(ic_env_t* env, editor_t* eb) { + ssize_t match = find_matching_brace( sbuf_string(eb->input), eb->pos, ic_env_get_match_braces(env), NULL ); + if (match < 0) return; + eb->pos = match; + edit_refresh(env,eb); +} + +static void edit_backspace(ic_env_t* env, editor_t* eb) { + if (eb->pos <= 0) return; + editor_start_modify(eb); + eb->pos = sbuf_delete_char_before(eb->input,eb->pos); + edit_refresh(env,eb); +} + +static void edit_delete_char(ic_env_t* env, editor_t* eb) { + if (eb->pos >= sbuf_len(eb->input)) return; + editor_start_modify(eb); + sbuf_delete_char_at(eb->input,eb->pos); + edit_refresh(env,eb); +} + +static void edit_delete_all(ic_env_t* env, editor_t* eb) { + if (sbuf_len(eb->input) <= 0) return; + editor_start_modify(eb); + sbuf_clear(eb->input); + eb->pos = 0; + edit_refresh(env,eb); +} + +static void edit_delete_to_end_of_line(ic_env_t* env, editor_t* eb) { + ssize_t start = sbuf_find_line_start(eb->input,eb->pos); + if (start < 0) return; + ssize_t end = sbuf_find_line_end(eb->input,eb->pos); + if (end < 0) return; + editor_start_modify(eb); + // if on an empty line, remove it completely + if (start == end && sbuf_char_at(eb->input,end) == '\n') { + end++; + } + else if (start == end && sbuf_char_at(eb->input,start - 1) == '\n') { + eb->pos--; + } + sbuf_delete_from_to( eb->input, eb->pos, end ); + edit_refresh(env,eb); +} + +static void edit_delete_to_start_of_line(ic_env_t* env, editor_t* eb) { + ssize_t start = sbuf_find_line_start(eb->input,eb->pos); + if (start < 0) return; + ssize_t end = sbuf_find_line_end(eb->input,eb->pos); + if (end < 0) return; + editor_start_modify(eb); + // delete start newline if it was an empty line + bool goright = false; + if (start > 0 && sbuf_char_at(eb->input,start-1) == '\n' && start == end) { + // if it is an empty line remove it + start--; + // afterwards, move to start of next line if it exists (so the cursor stays on the same row) + goright = true; + } + sbuf_delete_from_to( eb->input, start, eb->pos ); + eb->pos = start; + if (goright) edit_cursor_right(env,eb); + edit_refresh(env,eb); +} + +static void edit_delete_line(ic_env_t* env, editor_t* eb) { + ssize_t start = sbuf_find_line_start(eb->input,eb->pos); + if (start < 0) return; + ssize_t end = sbuf_find_line_end(eb->input,eb->pos); + if (end < 0) return; + editor_start_modify(eb); + // delete newline as well so no empty line is left; + bool goright = false; + if (start > 0 && sbuf_char_at(eb->input,start-1) == '\n') { + start--; + // afterwards, move to start of next line if it exists (so the cursor stays on the same row) + goright = true; + } + else if (sbuf_char_at(eb->input,end) == '\n') { + end++; + } + sbuf_delete_from_to(eb->input,start,end); + eb->pos = start; + if (goright) edit_cursor_right(env,eb); + edit_refresh(env,eb); +} + +static void edit_delete_to_start_of_word(ic_env_t* env, editor_t* eb) { + ssize_t start = sbuf_find_word_start(eb->input,eb->pos); + if (start < 0) return; + editor_start_modify(eb); + sbuf_delete_from_to( eb->input, start, eb->pos ); + eb->pos = start; + edit_refresh(env,eb); +} + +static void edit_delete_to_end_of_word(ic_env_t* env, editor_t* eb) { + ssize_t end = sbuf_find_word_end(eb->input,eb->pos); + if (end < 0) return; + editor_start_modify(eb); + sbuf_delete_from_to( eb->input, eb->pos, end ); + edit_refresh(env,eb); +} + +static void edit_delete_to_start_of_ws_word(ic_env_t* env, editor_t* eb) { + ssize_t start = sbuf_find_ws_word_start(eb->input, eb->pos); + if (start < 0) return; + editor_start_modify(eb); + sbuf_delete_from_to(eb->input, start, eb->pos); + eb->pos = start; + edit_refresh(env, eb); +} + +static void edit_delete_to_end_of_ws_word(ic_env_t* env, editor_t* eb) { + ssize_t end = sbuf_find_ws_word_end(eb->input, eb->pos); + if (end < 0) return; + editor_start_modify(eb); + sbuf_delete_from_to(eb->input, eb->pos, end); + edit_refresh(env, eb); +} + + +static void edit_delete_word(ic_env_t* env, editor_t* eb) { + ssize_t start = sbuf_find_word_start(eb->input,eb->pos); + if (start < 0) return; + ssize_t end = sbuf_find_word_end(eb->input,eb->pos); + if (end < 0) return; + editor_start_modify(eb); + sbuf_delete_from_to(eb->input,start,end); + eb->pos = start; + edit_refresh(env,eb); +} + +static void edit_swap_char( ic_env_t* env, editor_t* eb ) { + if (eb->pos <= 0 || eb->pos == sbuf_len(eb->input)) return; + editor_start_modify(eb); + eb->pos = sbuf_swap_char(eb->input,eb->pos); + edit_refresh(env,eb); +} + +static void edit_multiline_eol(ic_env_t* env, editor_t* eb) { + if (eb->pos <= 0) return; + if (sbuf_string(eb->input)[eb->pos-1] != env->multiline_eol) return; + editor_start_modify(eb); + // replace line continuation with a real newline + sbuf_delete_at( eb->input, eb->pos-1, 1); + sbuf_insert_at( eb->input, "\n", eb->pos-1); + edit_refresh(env,eb); +} + +static void edit_insert_unicode(ic_env_t* env, editor_t* eb, unicode_t u) { + editor_start_modify(eb); + ssize_t nextpos = sbuf_insert_unicode_at(eb->input, u, eb->pos); + if (nextpos >= 0) eb->pos = nextpos; + edit_refresh_hint(env, eb); +} + +static void edit_auto_brace(ic_env_t* env, editor_t* eb, char c) { + if (env->no_autobrace) return; + const char* braces = ic_env_get_auto_braces(env); + for (const char* b = braces; *b != 0; b += 2) { + if (*b == c) { + const char close = b[1]; + //if (sbuf_char_at(eb->input, eb->pos) != close) { + sbuf_insert_char_at(eb->input, close, eb->pos); + bool balanced = false; + find_matching_brace(sbuf_string(eb->input), eb->pos, braces, &balanced ); + if (!balanced) { + // don't insert if it leads to an unbalanced expression. + sbuf_delete_char_at(eb->input, eb->pos); + } + //} + return; + } + else if (b[1] == c) { + // close brace, check if there we don't overwrite to the right + if (sbuf_char_at(eb->input, eb->pos) == c) { + sbuf_delete_char_at(eb->input, eb->pos); + } + return; + } + } +} + +static void editor_auto_indent(editor_t* eb, const char* pre, const char* post ) { + assert(eb->pos > 0 && sbuf_char_at(eb->input,eb->pos-1) == '\n'); + ssize_t prelen = ic_strlen(pre); + if (prelen > 0) { + if (eb->pos - 1 < prelen) return; + if (!ic_starts_with(sbuf_string(eb->input) + eb->pos - 1 - prelen, pre)) return; + if (!ic_starts_with(sbuf_string(eb->input) + eb->pos, post)) return; + eb->pos = sbuf_insert_at(eb->input, " ", eb->pos); + sbuf_insert_char_at(eb->input, '\n', eb->pos); + } +} + +static void edit_insert_char(ic_env_t* env, editor_t* eb, char c) { + editor_start_modify(eb); + ssize_t nextpos = sbuf_insert_char_at( eb->input, c, eb->pos ); + if (nextpos >= 0) eb->pos = nextpos; + edit_auto_brace(env, eb, c); + if (c=='\n') { + editor_auto_indent(eb, "{", "}"); // todo: custom auto indent tokens? + } + edit_refresh_hint(env,eb); +} + +//------------------------------------------------------------- +// Help +//------------------------------------------------------------- + +#include "editline_help.c" + +//------------------------------------------------------------- +// History +//------------------------------------------------------------- + +#include "editline_history.c" + +//------------------------------------------------------------- +// Completion +//------------------------------------------------------------- + +#include "editline_completion.c" + + +//------------------------------------------------------------- +// Edit line: main edit loop +//------------------------------------------------------------- + +static char* edit_line( ic_env_t* env, const char* prompt_text ) +{ + // set up an edit buffer + editor_t eb; + memset(&eb, 0, sizeof(eb)); + eb.mem = env->mem; + eb.input = sbuf_new(env->mem); + eb.extra = sbuf_new(env->mem); + eb.hint = sbuf_new(env->mem); + eb.hint_help= sbuf_new(env->mem); + eb.termw = term_get_width(env->term); + eb.pos = 0; + eb.cur_rows = 1; + eb.cur_row = 0; + eb.modified = false; + eb.prompt_text = (prompt_text != NULL ? prompt_text : ""); + eb.history_idx = 0; + editstate_init(&eb.undo); + editstate_init(&eb.redo); + if (eb.input==NULL || eb.extra==NULL || eb.hint==NULL || eb.hint_help==NULL) { + return NULL; + } + + // caching + if (!(env->no_highlight && env->no_bracematch)) { + eb.attrs = attrbuf_new(env->mem); + eb.attrs_extra = attrbuf_new(env->mem); + } + + // show prompt + edit_write_prompt(env, &eb, 0, false); + + // always a history entry for the current input + history_push(env->history, ""); + + // process keys + code_t c; // current key code + while(true) { + // read a character + term_flush(env->term); + if (env->hint_delay <= 0 || sbuf_len(eb.hint) == 0) { + // blocking read + c = tty_read(env->tty); + } + else { + // timeout to display hint + if (!tty_read_timeout(env->tty, env->hint_delay, &c)) { + // timed-out + if (sbuf_len(eb.hint) > 0) { + // display hint + edit_refresh(env, &eb); + } + c = tty_read(env->tty); + } + else { + // clear the pending hint if we got input before the delay expired + sbuf_clear(eb.hint); + sbuf_clear(eb.hint_help); + } + } + + // update terminal in case of a resize + if (tty_term_resize_event(env->tty)) { + edit_resize(env,&eb); + } + + // clear hint only after a potential resize (so resize row calculations are correct) + const bool had_hint = (sbuf_len(eb.hint) > 0); + sbuf_clear(eb.hint); + sbuf_clear(eb.hint_help); + + // if the user tries to move into a hint with left-cursor or end, we complete it first + if ((c == KEY_RIGHT || c == KEY_END) && had_hint) { + edit_generate_completions(env, &eb, true); + c = KEY_NONE; + } + + // Operations that may return + if (c == KEY_ENTER) { + if (!env->singleline_only && eb.pos > 0 && + sbuf_string(eb.input)[eb.pos-1] == env->multiline_eol && + edit_pos_is_at_row_end(env,&eb)) + { + // replace line-continuation with newline + edit_multiline_eol(env,&eb); + } + else { + // otherwise done + break; + } + } + else if (c == KEY_CTRL_D) { + if (eb.pos == 0 && editor_pos_is_at_end(&eb)) break; // ctrl+D on empty quits with NULL + edit_delete_char(env,&eb); // otherwise it is like delete + } + else if (c == KEY_CTRL_C || c == KEY_EVENT_STOP) { + break; // ctrl+C or STOP event quits with NULL + } + else if (c == KEY_ESC) { + if (eb.pos == 0 && editor_pos_is_at_end(&eb)) break; // ESC on empty input returns with empty input + edit_delete_all(env,&eb); // otherwise delete the current input + // edit_delete_line(env,&eb); // otherwise delete the current line + } + else if (c == KEY_BELL /* ^G */) { + edit_delete_all(env,&eb); + break; // ctrl+G cancels (and returns empty input) + } + + // Editing Operations + else switch(c) { + // events + case KEY_EVENT_RESIZE: // not used + edit_resize(env,&eb); + break; + case KEY_EVENT_AUTOTAB: + edit_generate_completions(env, &eb, true); + break; + + // completion, history, help, undo + case KEY_TAB: + case WITH_ALT('?'): + edit_generate_completions(env,&eb,false); + break; + case KEY_CTRL_R: + case KEY_CTRL_S: + edit_history_search_with_current_word(env,&eb); + break; + case KEY_CTRL_P: + edit_history_prev(env, &eb); + break; + case KEY_CTRL_N: + edit_history_next(env, &eb); + break; + case KEY_CTRL_L: + edit_clear_screen(env, &eb); + break; + case KEY_CTRL_Z: + case WITH_CTRL('_'): + edit_undo_restore(env, &eb); + break; + case KEY_CTRL_Y: + edit_redo_restore(env, &eb); + break; + case KEY_F1: + edit_show_help(env, &eb); + break; + + // navigation + case KEY_LEFT: + case KEY_CTRL_B: + edit_cursor_left(env,&eb); + break; + case KEY_RIGHT: + case KEY_CTRL_F: + if (eb.pos == sbuf_len(eb.input)) { + edit_generate_completions( env, &eb, false ); + } + else { + edit_cursor_right(env,&eb); + } + break; + case KEY_UP: + edit_cursor_row_up(env,&eb); + break; + case KEY_DOWN: + edit_cursor_row_down(env,&eb); + break; + case KEY_HOME: + case KEY_CTRL_A: + edit_cursor_line_start(env,&eb); + break; + case KEY_END: + case KEY_CTRL_E: + edit_cursor_line_end(env,&eb); + break; + case KEY_CTRL_LEFT: + case WITH_SHIFT(KEY_LEFT): + case WITH_ALT('b'): + edit_cursor_prev_word(env,&eb); + break; + case KEY_CTRL_RIGHT: + case WITH_SHIFT(KEY_RIGHT): + case WITH_ALT('f'): + if (eb.pos == sbuf_len(eb.input)) { + edit_generate_completions( env, &eb, false ); + } + else { + edit_cursor_next_word(env,&eb); + } + break; + case KEY_CTRL_HOME: + case WITH_SHIFT(KEY_HOME): + case KEY_PAGEUP: + case WITH_ALT('<'): + edit_cursor_to_start(env,&eb); + break; + case KEY_CTRL_END: + case WITH_SHIFT(KEY_END): + case KEY_PAGEDOWN: + case WITH_ALT('>'): + edit_cursor_to_end(env,&eb); + break; + case WITH_ALT('m'): + edit_cursor_match_brace(env,&eb); + break; + + // deletion + case KEY_BACKSP: + edit_backspace(env,&eb); + break; + case KEY_DEL: + edit_delete_char(env,&eb); + break; + case WITH_ALT('d'): + edit_delete_to_end_of_word(env,&eb); + break; + case KEY_CTRL_W: + edit_delete_to_start_of_ws_word(env, &eb); + break; + case WITH_ALT(KEY_DEL): + case WITH_ALT(KEY_BACKSP): + edit_delete_to_start_of_word(env,&eb); + break; + case KEY_CTRL_U: + edit_delete_to_start_of_line(env,&eb); + break; + case KEY_CTRL_K: + edit_delete_to_end_of_line(env,&eb); + break; + case KEY_CTRL_T: + edit_swap_char(env,&eb); + break; + + // Editing + case KEY_SHIFT_TAB: + case KEY_LINEFEED: // '\n' (ctrl+J, shift+enter) + if (!env->singleline_only) { + edit_insert_char(env, &eb, '\n'); + } + break; + default: { + char chr; + unicode_t uchr; + if (code_is_ascii_char(c,&chr)) { + edit_insert_char(env,&eb,chr); + } + else if (code_is_unicode(c, &uchr)) { + edit_insert_unicode(env,&eb, uchr); + } + else { + debug_msg( "edit: ignore code: 0x%04x\n", c); + } + break; + } + } + + } + + // goto end + eb.pos = sbuf_len(eb.input); + + // refresh once more but without brace matching + bool bm = env->no_bracematch; + env->no_bracematch = true; + edit_refresh(env,&eb); + env->no_bracematch = bm; + + // save result + char* res; + if ((c == KEY_CTRL_D && sbuf_len(eb.input) == 0) || c == KEY_CTRL_C || c == KEY_EVENT_STOP) { + res = NULL; + } + else if (!tty_is_utf8(env->tty)) { + res = sbuf_strdup_from_utf8(eb.input); + } + else { + res = sbuf_strdup(eb.input); + } + + // update history + history_update(env->history, sbuf_string(eb.input)); + if (res == NULL || sbuf_len(eb.input) <= 1) { ic_history_remove_last(); } // no empty or single-char entries + history_save(env->history); + + // free resources + editstate_done(env->mem, &eb.undo); + editstate_done(env->mem, &eb.redo); + attrbuf_free(eb.attrs); + attrbuf_free(eb.attrs_extra); + sbuf_free(eb.input); + sbuf_free(eb.extra); + sbuf_free(eb.hint); + sbuf_free(eb.hint_help); + + return res; +} + diff --git a/extern/isocline/src/editline_completion.c b/extern/isocline/src/editline_completion.c new file mode 100644 index 00000000..1734ef34 --- /dev/null +++ b/extern/isocline/src/editline_completion.c @@ -0,0 +1,277 @@ +/* ---------------------------------------------------------------------------- + Copyright (c) 2021, Daan Leijen + This is free software; you can redistribute it and/or modify it + under the terms of the MIT License. A copy of the license can be + found in the "LICENSE" file at the root of this distribution. +-----------------------------------------------------------------------------*/ + +//------------------------------------------------------------- +// Completion menu: this file is included in editline.c +//------------------------------------------------------------- + +// return true if anything changed +static bool edit_complete(ic_env_t* env, editor_t* eb, ssize_t idx) { + editor_start_modify(eb); + ssize_t newpos = completions_apply(env->completions, idx, eb->input, eb->pos); + if (newpos < 0) { + editor_undo_restore(eb,false); + return false; + } + eb->pos = newpos; + edit_refresh(env,eb); + return true; +} + +static bool edit_complete_longest_prefix(ic_env_t* env, editor_t* eb ) { + editor_start_modify(eb); + ssize_t newpos = completions_apply_longest_prefix( env->completions, eb->input, eb->pos ); + if (newpos < 0) { + editor_undo_restore(eb,false); + return false; + } + eb->pos = newpos; + edit_refresh(env,eb); + return true; +} + +ic_private void sbuf_append_tagged( stringbuf_t* sb, const char* tag, const char* content ) { + sbuf_appendf(sb, "[%s]", tag); + sbuf_append(sb,content); + sbuf_append(sb,"[/]"); +} + +static void editor_append_completion(ic_env_t* env, editor_t* eb, ssize_t idx, ssize_t width, bool numbered, bool selected ) { + const char* help = NULL; + const char* display = completions_get_display(env->completions, idx, &help); + if (display == NULL) return; + if (numbered) { + sbuf_appendf(eb->extra, "[ic-info]%s%zd [/]", (selected ? (tty_is_utf8(env->tty) ? "\xE2\x86\x92" : "*") : " "), 1 + idx); + width -= 3; + } + + if (width > 0) { + sbuf_appendf(eb->extra, "[width=\"%zd;left; ;on\"]", width ); + } + if (selected) { + sbuf_append(eb->extra, "[ic-emphasis]"); + } + sbuf_append(eb->extra, display); + if (selected) { sbuf_append(eb->extra,"[/ic-emphasis]"); } + if (help != NULL) { + sbuf_append(eb->extra, " "); + sbuf_append_tagged(eb->extra, "ic-info", help ); + } + if (width > 0) { sbuf_append(eb->extra,"[/width]"); } +} + +// 2 and 3 column output up to 80 wide +#define IC_DISPLAY2_MAX 34 +#define IC_DISPLAY2_COL (3+IC_DISPLAY2_MAX) +#define IC_DISPLAY2_WIDTH (2*IC_DISPLAY2_COL + 2) // 75 + +#define IC_DISPLAY3_MAX 21 +#define IC_DISPLAY3_COL (3+IC_DISPLAY3_MAX) +#define IC_DISPLAY3_WIDTH (3*IC_DISPLAY3_COL + 2*2) // 76 + +static void editor_append_completion2(ic_env_t* env, editor_t* eb, ssize_t col_width, ssize_t idx1, ssize_t idx2, ssize_t selected ) { + editor_append_completion(env, eb, idx1, col_width, true, (idx1 == selected) ); + sbuf_append( eb->extra, " "); + editor_append_completion(env, eb, idx2, col_width, true, (idx2 == selected) ); +} + +static void editor_append_completion3(ic_env_t* env, editor_t* eb, ssize_t col_width, ssize_t idx1, ssize_t idx2, ssize_t idx3, ssize_t selected ) { + editor_append_completion(env, eb, idx1, col_width, true, (idx1 == selected) ); + sbuf_append( eb->extra, " "); + editor_append_completion(env, eb, idx2, col_width, true, (idx2 == selected)); + sbuf_append( eb->extra, " "); + editor_append_completion(env, eb, idx3, col_width, true, (idx3 == selected) ); +} + +static ssize_t edit_completions_max_width( ic_env_t* env, ssize_t count ) { + ssize_t max_width = 0; + for( ssize_t i = 0; i < count; i++) { + const char* help = NULL; + ssize_t w = bbcode_column_width(env->bbcode, completions_get_display(env->completions, i, &help)); + if (help != NULL) { + w += 2 + bbcode_column_width(env->bbcode, help); + } + if (w > max_width) { + max_width = w; + } + } + return max_width; +} + +static void edit_completion_menu(ic_env_t* env, editor_t* eb, bool more_available) { + ssize_t count = completions_count(env->completions); + ssize_t count_displayed = count; + assert(count > 1); + ssize_t selected = (env->complete_nopreview ? 0 : -1); // select first or none + ssize_t percolumn = count; + +again: + // show first 9 (or 8) completions + sbuf_clear(eb->extra); + ssize_t twidth = term_get_width(env->term) - 1; + ssize_t colwidth; + if (count > 3 && ((colwidth = 3 + edit_completions_max_width(env, 9))*3 + 2*2) < twidth) { + // display as a 3 column block + count_displayed = (count > 9 ? 9 : count); + percolumn = 3; + for (ssize_t rw = 0; rw < percolumn; rw++) { + if (rw > 0) sbuf_append(eb->extra, "\n"); + editor_append_completion3(env, eb, colwidth, rw, percolumn+rw, (2*percolumn)+rw, selected); + } + } + else if (count > 4 && ((colwidth = 3 + edit_completions_max_width(env, 8))*2 + 2) < twidth) { + // display as a 2 column block if some entries are too wide for three columns + count_displayed = (count > 8 ? 8 : count); + percolumn = (count_displayed <= 6 ? 3 : 4); + for (ssize_t rw = 0; rw < percolumn; rw++) { + if (rw > 0) sbuf_append(eb->extra, "\n"); + editor_append_completion2(env, eb, colwidth, rw, percolumn+rw, selected); + } + } + else { + // display as a list + count_displayed = (count > 9 ? 9 : count); + percolumn = count_displayed; + for (ssize_t i = 0; i < count_displayed; i++) { + if (i > 0) sbuf_append(eb->extra, "\n"); + editor_append_completion(env, eb, i, -1, true /* numbered */, selected == i); + } + } + if (count > count_displayed) { + if (more_available) { + sbuf_append(eb->extra, "\n[ic-info](press page-down (or ctrl-j) to see all further completions)[/]"); + } + else { + sbuf_appendf(eb->extra, "\n[ic-info](press page-down (or ctrl-j) to see all %zd completions)[/]", count ); + } + } + if (!env->complete_nopreview && selected >= 0 && selected <= count_displayed) { + edit_complete(env,eb,selected); + editor_undo_restore(eb,false); + } + else { + edit_refresh(env, eb); + } + + // read here; if not a valid key, push it back and return to main event loop + code_t c = tty_read(env->tty); + if (tty_term_resize_event(env->tty)) { + edit_resize(env, eb); + } + sbuf_clear(eb->extra); + + // direct selection? + if (c >= '1' && c <= '9') { + ssize_t i = (c - '1'); + if (i < count) { + selected = i; + c = KEY_ENTER; + } + } + + // process commands + if (c == KEY_DOWN || c == KEY_TAB) { + selected++; + if (selected >= count_displayed) { + //term_beep(env->term); + selected = 0; + } + goto again; + } + else if (c == KEY_UP || c == KEY_SHIFT_TAB) { + selected--; + if (selected < 0) { + selected = count_displayed - 1; + //term_beep(env->term); + } + goto again; + } + else if (c == KEY_F1) { + edit_show_help(env, eb); + goto again; + } + else if (c == KEY_ESC) { + completions_clear(env->completions); + edit_refresh(env,eb); + c = 0; // ignore and return + } + else if (selected >= 0 && (c == KEY_ENTER || c == KEY_RIGHT || c == KEY_END)) /* || c == KEY_TAB*/ { + // select the current entry + assert(selected < count); + c = 0; + edit_complete(env, eb, selected); + if (env->complete_autotab) { + tty_code_pushback(env->tty,KEY_EVENT_AUTOTAB); // immediately try to complete again + } + } + else if (!env->complete_nopreview && !code_is_virt_key(c)) { + // if in preview mode, select the current entry and exit the menu + assert(selected < count); + edit_complete(env, eb, selected); + } + else if ((c == KEY_PAGEDOWN || c == KEY_LINEFEED) && count > 9) { + // show all completions + c = 0; + if (more_available) { + // generate all entries (up to the max (= 1000)) + count = completions_generate(env, env->completions, sbuf_string(eb->input), eb->pos, IC_MAX_COMPLETIONS_TO_SHOW); + } + rowcol_t rc; + edit_get_rowcol(env,eb,&rc); + edit_clear(env,eb); + edit_write_prompt(env,eb,0,false); + term_writeln(env->term, ""); + for(ssize_t i = 0; i < count; i++) { + const char* display = completions_get_display(env->completions, i, NULL); + if (display != NULL) { + bbcode_println(env->bbcode, display); + } + } + if (count >= IC_MAX_COMPLETIONS_TO_SHOW) { + bbcode_println(env->bbcode, "[ic-info]... and more.[/]"); + } + else { + bbcode_printf(env->bbcode, "[ic-info](%zd possible completions)[/]\n", count ); + } + for(ssize_t i = 0; i < rc.row+1; i++) { + term_write(env->term, " \n"); + } + eb->cur_rows = 0; + edit_refresh(env,eb); + } + else { + edit_refresh(env,eb); + } + // done + completions_clear(env->completions); + if (c != 0) tty_code_pushback(env->tty,c); +} + +static void edit_generate_completions(ic_env_t* env, editor_t* eb, bool autotab) { + debug_msg( "edit: complete: %zd: %s\n", eb->pos, sbuf_string(eb->input) ); + if (eb->pos < 0) return; + ssize_t count = completions_generate(env, env->completions, sbuf_string(eb->input), eb->pos, IC_MAX_COMPLETIONS_TO_TRY); + bool more_available = (count >= IC_MAX_COMPLETIONS_TO_TRY); + if (count <= 0) { + // no completions + if (!autotab) { term_beep(env->term); } + } + else if (count == 1) { + // complete if only one match + if (edit_complete(env,eb,0 /*idx*/) && env->complete_autotab) { + tty_code_pushback(env->tty,KEY_EVENT_AUTOTAB); + } + } + else { + //term_beep(env->term); + if (!more_available) { + edit_complete_longest_prefix(env,eb); + } + completions_sort(env->completions); + edit_completion_menu( env, eb, more_available); + } +} diff --git a/extern/isocline/src/editline_help.c b/extern/isocline/src/editline_help.c new file mode 100644 index 00000000..fa07d1db --- /dev/null +++ b/extern/isocline/src/editline_help.c @@ -0,0 +1,140 @@ +/* ---------------------------------------------------------------------------- + Copyright (c) 2021, Daan Leijen + This is free software; you can redistribute it and/or modify it + under the terms of the MIT License. A copy of the license can be + found in the "LICENSE" file at the root of this distribution. +-----------------------------------------------------------------------------*/ + +//------------------------------------------------------------- +// Help: this is included into editline.c +//------------------------------------------------------------- + +static const char* help[] = { + "","Navigation:", + "left," + "^b", "go one character to the left", + "right," + "^f", "go one character to the right", + "up", "go one row up, or back in the history", + "down", "go one row down, or forward in the history", + #ifdef __APPLE__ + "shift-left", + #else + "^left", + #endif + "go to the start of the previous word", + #ifdef __APPLE__ + "shift-right", + #else + "^right", + #endif + "go to the end the current word", + "home," + "^a", "go to the start of the current line", + "end," + "^e", "go to the end of the current line", + "pgup," + "^home", "go to the start of the current input", + "pgdn," + "^end", "go to the end of the current input", + "alt-m", "jump to matching brace", + "^p", "go back in the history", + "^n", "go forward in the history", + "^r,^s", "search the history starting with the current word", + "","", + + "", "Deletion:", + "del,^d", "delete the current character", + "backsp,^h", "delete the previous character", + "^w", "delete to preceding white space", + "alt-backsp", "delete to the start of the current word", + "alt-d", "delete to the end of the current word", + "^u", "delete to the start of the current line", + "^k", "delete to the end of the current line", + "esc", "delete the current input, or done with empty input", + "","", + + "", "Editing:", + "enter", "accept current input", + #ifndef __APPLE__ + "^enter, ^j", "", + "shift-tab", + #else + "shift-tab,^j", + #endif + "create a new line for multi-line input", + //" ", "(or type '\\' followed by enter)", + "^l", "clear screen", + "^t", "swap with previous character (move character backward)", + "^z,^_", "undo", + "^y", "redo", + //"^C", "done with empty input", + //"F1", "show this help", + "tab", "try to complete the current input", + "","", + "","In the completion menu:", + "enter,left", "use the currently selected completion", + "1 - 9", "use completion N from the menu", + "tab,down", "select the next completion", + "shift-tab,up","select the previous completion", + "esc", "exit menu without completing", + "pgdn,^j", "show all further possible completions", + "","", + "","In incremental history search:", + "enter", "use the currently found history entry", + "backsp," + "^z", "go back to the previous match (undo)", + "tab," + "^r", "find the next match", + "shift-tab," + "^s", "find an earlier match", + "esc", "exit search", + " ","", + NULL, NULL +}; + +static const char* help_initial = + "[ic-info]" + "Isocline v1.0, copyright (c) 2021 Daan Leijen.\n" + "This is free software; you can redistribute it and/or\n" + "modify it under the terms of the MIT License.\n" + "See <[url]https://github.com/daanx/isocline[/url]> for further information.\n" + "We use ^ as a shorthand for ctrl-.\n" + "\n" + "Overview:\n" + "\n[ansi-lightgray]" + " home,ctrl-a cursor end,ctrl-e\n" + " ┌────────────────┼───────────────┐ (navigate)\n" + //" │ │ │\n" + #ifndef __APPLE__ + " │ ctrl-left │ ctrl-right │\n" + #else + " │ alt-left │ alt-right │\n" + #endif + " │ ┌───────┼──────┐ │ ctrl-r : search history\n" + " ▼ ▼ ▼ ▼ ▼ tab : complete word\n" + " prompt> [ansi-darkgray]it's the quintessential language[/] shift-tab: insert new line\n" + " ▲ ▲ ▲ ▲ esc : delete input, done\n" + " │ └──────────────┘ │ ctrl-z : undo\n" + " │ alt-backsp alt-d │\n" + //" │ │ │\n" + " └────────────────────────────────┘ (delete)\n" + " ctrl-u ctrl-k\n" + "[/ansi-lightgray][/ic-info]\n"; + +static void edit_show_help(ic_env_t* env, editor_t* eb) { + edit_clear(env, eb); + bbcode_println(env->bbcode, help_initial); + for (ssize_t i = 0; help[i] != NULL && help[i+1] != NULL; i += 2) { + if (help[i][0] == 0) { + bbcode_printf(env->bbcode, "[ic-info]%s[/]\n", help[i+1]); + } + else { + bbcode_printf(env->bbcode, " [ic-emphasis]%-13s[/][ansi-lightgray]%s%s[/]\n", help[i], (help[i+1][0] == 0 ? "" : ": "), help[i+1]); + } + } + + eb->cur_rows = 0; + eb->cur_row = 0; + edit_refresh(env, eb); +} diff --git a/extern/isocline/src/editline_history.c b/extern/isocline/src/editline_history.c new file mode 100644 index 00000000..2a0afa1c --- /dev/null +++ b/extern/isocline/src/editline_history.c @@ -0,0 +1,260 @@ +/* ---------------------------------------------------------------------------- + Copyright (c) 2021, Daan Leijen + This is free software; you can redistribute it and/or modify it + under the terms of the MIT License. A copy of the license can be + found in the "LICENSE" file at the root of this distribution. +-----------------------------------------------------------------------------*/ + +//------------------------------------------------------------- +// History search: this file is included in editline.c +//------------------------------------------------------------- + +static void edit_history_at(ic_env_t* env, editor_t* eb, int ofs ) +{ + if (eb->modified) { + history_update(env->history, sbuf_string(eb->input)); // update first entry if modified + eb->history_idx = 0; // and start again + eb->modified = false; + } + const char* entry = history_get(env->history,eb->history_idx + ofs); + // debug_msg( "edit: history: at: %d + %d, found: %s\n", eb->history_idx, ofs, entry); + if (entry == NULL) { + term_beep(env->term); + } + else { + eb->history_idx += ofs; + sbuf_replace(eb->input, entry); + if (ofs > 0) { + // at end of first line when scrolling up + ssize_t end = sbuf_find_line_end(eb->input,0); + eb->pos = (end < 0 ? 0 : end); + } + else { + eb->pos = sbuf_len(eb->input); // at end of last line when scrolling down + } + edit_refresh(env, eb); + } +} + +static void edit_history_prev(ic_env_t* env, editor_t* eb) { + edit_history_at(env,eb, 1 ); +} + +static void edit_history_next(ic_env_t* env, editor_t* eb) { + edit_history_at(env,eb, -1 ); +} + +typedef struct hsearch_s { + struct hsearch_s* next; + ssize_t hidx; + ssize_t match_pos; + ssize_t match_len; + bool cinsert; +} hsearch_t; + +static void hsearch_push( alloc_t* mem, hsearch_t** hs, ssize_t hidx, ssize_t mpos, ssize_t mlen, bool cinsert ) { + hsearch_t* h = mem_zalloc_tp( mem, hsearch_t ); + if (h == NULL) return; + h->hidx = hidx; + h->match_pos = mpos; + h->match_len = mlen; + h->cinsert = cinsert; + h->next = *hs; + *hs = h; +} + +static bool hsearch_pop( alloc_t* mem, hsearch_t** hs, ssize_t* hidx, ssize_t* match_pos, ssize_t* match_len, bool* cinsert ) { + hsearch_t* h = *hs; + if (h == NULL) return false; + *hs = h->next; + if (hidx != NULL) *hidx = h->hidx; + if (match_pos != NULL) *match_pos = h->match_pos; + if (match_len != NULL) *match_len = h->match_len; + if (cinsert != NULL) *cinsert = h->cinsert; + mem_free(mem, h); + return true; +} + +static void hsearch_done( alloc_t* mem, hsearch_t* hs ) { + while (hs != NULL) { + hsearch_t* next = hs->next; + mem_free(mem, hs); + hs = next; + } +} + +static void edit_history_search(ic_env_t* env, editor_t* eb, char* initial ) { + if (history_count( env->history ) <= 0) { + term_beep(env->term); + return; + } + + // update history + if (eb->modified) { + history_update(env->history, sbuf_string(eb->input)); // update first entry if modified + eb->history_idx = 0; // and start again + eb->modified = false; + } + + // set a search prompt and remember the previous state + editor_undo_capture(eb); + eb->disable_undo = true; + bool old_hint = ic_enable_hint(false); + const char* prompt_text = eb->prompt_text; + eb->prompt_text = "history search"; + + // search state + hsearch_t* hs = NULL; // search undo + ssize_t hidx = 1; // current history entry + ssize_t match_pos = 0; // current matched position + ssize_t match_len = 0; // length of the match + const char* hentry = NULL; // current history entry + + // Simulate per character searches for each letter in `initial` (so backspace works) + if (initial != NULL) { + const ssize_t initial_len = ic_strlen(initial); + ssize_t ipos = 0; + while( ipos < initial_len ) { + ssize_t next = str_next_ofs( initial, initial_len, ipos, NULL ); + if (next < 0) break; + hsearch_push( eb->mem, &hs, hidx, match_pos, match_len, true); + char c = initial[ipos + next]; // terminate temporarily + initial[ipos + next] = 0; + if (history_search( env->history, hidx, initial, true, &hidx, &match_pos )) { + match_len = ipos + next; + } + else if (ipos + next >= initial_len) { + term_beep(env->term); + } + initial[ipos + next] = c; // restore + ipos += next; + } + sbuf_replace( eb->input, initial); + eb->pos = ipos; + } + else { + sbuf_clear( eb->input ); + eb->pos = 0; + } + + // Incremental search +again: + hentry = history_get(env->history,hidx); + if (hentry != NULL) { + sbuf_appendf(eb->extra, "[ic-info]%zd. [/][ic-diminish][!pre]", hidx); + sbuf_append_n( eb->extra, hentry, match_pos ); + sbuf_append(eb->extra, "[/pre][u ic-emphasis][!pre]" ); + sbuf_append_n( eb->extra, hentry + match_pos, match_len ); + sbuf_append(eb->extra, "[/pre][/u][!pre]" ); + sbuf_append(eb->extra, hentry + match_pos + match_len ); + sbuf_append(eb->extra, "[/pre][/ic-diminish]"); + if (!env->no_help) { + sbuf_append(eb->extra, "\n[ic-info](use tab for the next match)[/]"); + } + sbuf_append(eb->extra, "\n" ); + } + edit_refresh(env, eb); + + // Wait for input + code_t c = (hentry == NULL ? KEY_ESC : tty_read(env->tty)); + if (tty_term_resize_event(env->tty)) { + edit_resize(env, eb); + } + sbuf_clear(eb->extra); + + // Process commands + if (c == KEY_ESC || c == KEY_BELL /* ^G */ || c == KEY_CTRL_C) { + c = 0; + eb->disable_undo = false; + editor_undo_restore(eb, false); + } + else if (c == KEY_ENTER) { + c = 0; + editor_undo_forget(eb); + sbuf_replace( eb->input, hentry ); + eb->pos = sbuf_len(eb->input); + eb->modified = false; + eb->history_idx = hidx; + } + else if (c == KEY_BACKSP || c == KEY_CTRL_Z) { + // undo last search action + bool cinsert; + if (hsearch_pop(env->mem,&hs, &hidx, &match_pos, &match_len, &cinsert)) { + if (cinsert) edit_backspace(env,eb); + } + goto again; + } + else if (c == KEY_CTRL_R || c == KEY_TAB || c == KEY_UP) { + // search backward + hsearch_push(env->mem, &hs, hidx, match_pos, match_len, false); + if (!history_search( env->history, hidx+1, sbuf_string(eb->input), true, &hidx, &match_pos )) { + hsearch_pop(env->mem,&hs,NULL,NULL,NULL,NULL); + term_beep(env->term); + }; + goto again; + } + else if (c == KEY_CTRL_S || c == KEY_SHIFT_TAB || c == KEY_DOWN) { + // search forward + hsearch_push(env->mem, &hs, hidx, match_pos, match_len, false); + if (!history_search( env->history, hidx-1, sbuf_string(eb->input), false, &hidx, &match_pos )) { + hsearch_pop(env->mem, &hs,NULL,NULL,NULL,NULL); + term_beep(env->term); + }; + goto again; + } + else if (c == KEY_F1) { + edit_show_help(env, eb); + goto again; + } + else { + // insert character and search further backward + char chr; + unicode_t uchr; + if (code_is_ascii_char(c,&chr)) { + hsearch_push(env->mem, &hs, hidx, match_pos, match_len, true); + edit_insert_char(env,eb,chr); + } + else if (code_is_unicode(c,&uchr)) { + hsearch_push(env->mem, &hs, hidx, match_pos, match_len, true); + edit_insert_unicode(env,eb,uchr); + } + else { + // ignore command + term_beep(env->term); + goto again; + } + // search for the new input + if (history_search( env->history, hidx, sbuf_string(eb->input), true, &hidx, &match_pos )) { + match_len = sbuf_len(eb->input); + } + else { + term_beep(env->term); + }; + goto again; + } + + // done + eb->disable_undo = false; + hsearch_done(env->mem,hs); + eb->prompt_text = prompt_text; + ic_enable_hint(old_hint); + edit_refresh(env,eb); + if (c != 0) tty_code_pushback(env->tty, c); +} + +// Start an incremental search with the current word +static void edit_history_search_with_current_word(ic_env_t* env, editor_t* eb) { + char* initial = NULL; + ssize_t start = sbuf_find_word_start( eb->input, eb->pos ); + if (start >= 0) { + const ssize_t next = sbuf_next(eb->input, start, NULL); + if (!ic_char_is_idletter(sbuf_string(eb->input) + start, (long)(next - start))) { + start = next; + } + if (start >= 0 && start < eb->pos) { + initial = mem_strndup(eb->mem, sbuf_string(eb->input) + start, eb->pos - start); + } + } + edit_history_search( env, eb, initial); + mem_free(env->mem, initial); +} diff --git a/extern/isocline/src/env.h b/extern/isocline/src/env.h new file mode 100644 index 00000000..edfc1003 --- /dev/null +++ b/extern/isocline/src/env.h @@ -0,0 +1,60 @@ +/* ---------------------------------------------------------------------------- + Copyright (c) 2021, Daan Leijen + This is free software; you can redistribute it and/or modify it + under the terms of the MIT License. A copy of the license can be + found in the "LICENSE" file at the root of this distribution. +-----------------------------------------------------------------------------*/ +#pragma once +#ifndef IC_ENV_H +#define IC_ENV_H + +#include "../include/isocline.h" +#include "common.h" +#include "term.h" +#include "tty.h" +#include "stringbuf.h" +#include "history.h" +#include "completions.h" +#include "bbcode.h" + +//------------------------------------------------------------- +// Environment +//------------------------------------------------------------- + +struct ic_env_s { + alloc_t* mem; // potential custom allocator + ic_env_t* next; // next environment (used for proper deallocation) + term_t* term; // terminal + tty_t* tty; // keyboard (NULL if stdin is a pipe, file, etc) + completions_t* completions; // current completions + history_t* history; // edit history + bbcode_t* bbcode; // print with bbcodes + const char* prompt_marker; // the prompt marker (defaults to "> ") + const char* cprompt_marker; // prompt marker for continuation lines (defaults to `prompt_marker`) + ic_highlight_fun_t* highlighter; // highlight callback + void* highlighter_arg; // user state for the highlighter. + const char* match_braces; // matching braces, e.g "()[]{}" + const char* auto_braces; // auto insertion braces, e.g "()[]{}\"\"''" + char multiline_eol; // character used for multiline input ("\") (set to 0 to disable) + bool initialized; // are we initialized? + bool noedit; // is rich editing possible (tty != NULL) + bool singleline_only; // allow only single line editing? + bool complete_nopreview; // do not show completion preview for each selection in the completion menu? + bool complete_autotab; // try to keep completing after a completion? + bool no_multiline_indent; // indent continuation lines to line up under the initial prompt + bool no_help; // show short help line for history search etc. + bool no_hint; // allow hinting? + bool no_highlight; // enable highlighting? + bool no_bracematch; // enable brace matching? + bool no_autobrace; // enable automatic brace insertion? + bool no_lscolors; // use LSCOLORS/LS_COLORS to colorize file name completions? + long hint_delay; // delay before displaying a hint in milliseconds +}; + +ic_private char* ic_editline(ic_env_t* env, const char* prompt_text); + +ic_private ic_env_t* ic_get_env(void); +ic_private const char* ic_env_get_auto_braces(ic_env_t* env); +ic_private const char* ic_env_get_match_braces(ic_env_t* env); + +#endif // IC_ENV_H diff --git a/extern/isocline/src/highlight.c b/extern/isocline/src/highlight.c new file mode 100644 index 00000000..59c7255c --- /dev/null +++ b/extern/isocline/src/highlight.c @@ -0,0 +1,259 @@ +/* ---------------------------------------------------------------------------- + Copyright (c) 2021, Daan Leijen + This is free software; you can redistribute it and/or modify it + under the terms of the MIT License. A copy of the license can be + found in the "LICENSE" file at the root of this distribution. +-----------------------------------------------------------------------------*/ + +#include +#include "common.h" +#include "term.h" +#include "stringbuf.h" +#include "attr.h" +#include "bbcode.h" + +//------------------------------------------------------------- +// Syntax highlighting +//------------------------------------------------------------- + +struct ic_highlight_env_s { + attrbuf_t* attrs; + const char* input; + ssize_t input_len; + bbcode_t* bbcode; + alloc_t* mem; + ssize_t cached_upos; // cached unicode position + ssize_t cached_cpos; // corresponding utf-8 byte position +}; + + +ic_private void highlight( alloc_t* mem, bbcode_t* bb, const char* s, attrbuf_t* attrs, ic_highlight_fun_t* highlighter, void* arg ) { + const ssize_t len = ic_strlen(s); + if (len <= 0) return; + attrbuf_set_at(attrs,0,len,attr_none()); // fill to length of s + if (highlighter != NULL) { + ic_highlight_env_t henv; + henv.attrs = attrs; + henv.input = s; + henv.input_len = len; + henv.bbcode = bb; + henv.mem = mem; + henv.cached_cpos = 0; + henv.cached_upos = 0; + (*highlighter)( &henv, s, arg ); + } +} + + +//------------------------------------------------------------- +// Client interface +//------------------------------------------------------------- + +static void pos_adjust( ic_highlight_env_t* henv, ssize_t* ppos, ssize_t* plen ) { + ssize_t pos = *ppos; + ssize_t len = *plen; + if (pos >= henv->input_len) return; + if (pos >= 0 && len >= 0) return; // already character positions + if (henv->input == NULL) return; + + if (pos < 0) { + // negative `pos` is used as the unicode character position (for easy interfacing with Haskell) + ssize_t upos = -pos; + ssize_t cpos = 0; + ssize_t ucount = 0; + if (henv->cached_upos <= upos) { // if we have a cached position, start from there + ucount = henv->cached_upos; + cpos = henv->cached_cpos; + } + while ( ucount < upos ) { + ssize_t next = str_next_ofs(henv->input, henv->input_len, cpos, NULL); + if (next <= 0) return; + ucount++; + cpos += next; + } + *ppos = pos = cpos; + // and cache it to avoid quadratic behavior + henv->cached_upos = upos; + henv->cached_cpos = cpos; + } + if (len < 0) { + // negative `len` is used as a unicode character length + len = -len; + ssize_t ucount = 0; + ssize_t clen = 0; + while (ucount < len) { + ssize_t next = str_next_ofs(henv->input, henv->input_len, pos + clen, NULL); + if (next <= 0) return; + ucount++; + clen += next; + } + *plen = len = clen; + // and update cache if possible + if (henv->cached_cpos == pos) { + henv->cached_upos += ucount; + henv->cached_cpos += clen; + } + } +} + +static void highlight_attr(ic_highlight_env_t* henv, ssize_t pos, ssize_t count, attr_t attr ) { + if (henv==NULL) return; + pos_adjust(henv,&pos,&count); + if (pos < 0 || count <= 0) return; + attrbuf_update_at(henv->attrs, pos, count, attr); +} + +ic_public void ic_highlight(ic_highlight_env_t* henv, long pos, long count, const char* style ) { + if (henv == NULL || style==NULL || style[0]==0 || pos < 0) return; + highlight_attr(henv,pos,count,bbcode_style( henv->bbcode, style )); +} + +ic_public void ic_highlight_formatted(ic_highlight_env_t* henv, const char* s, const char* fmt) { + if (s==NULL || s[0] == 0 || fmt==NULL) return; + attrbuf_t* attrs = attrbuf_new(henv->mem); + stringbuf_t* out = sbuf_new(henv->mem); // todo: avoid allocating out? + if (attrs!=NULL && out != NULL) { + bbcode_append( henv->bbcode, fmt, out, attrs); + const ssize_t len = ic_strlen(s); + if (sbuf_len(out) != len) { + debug_msg("highlight: formatted string content differs from the original input:\n original: %s\n formatted: %s\n", s, fmt); + } + for( ssize_t i = 0; i < len; i++) { + attrbuf_update_at(henv->attrs, i, 1, attrbuf_attr_at(attrs,i)); + } + } + sbuf_free(out); + attrbuf_free(attrs); +} + +//------------------------------------------------------------- +// Brace matching +//------------------------------------------------------------- +#define MAX_NESTING (64) + +typedef struct brace_s { + char close; + bool at_cursor; + ssize_t pos; +} brace_t; + +ic_private void highlight_match_braces(const char* s, attrbuf_t* attrs, ssize_t cursor_pos, const char* braces, attr_t match_attr, attr_t error_attr) +{ + brace_t open[MAX_NESTING+1]; + ssize_t nesting = 0; + const ssize_t brace_len = ic_strlen(braces); + for (long i = 0; i < ic_strlen(s); i++) { + const char c = s[i]; + // push open brace + bool found_open = false; + for (ssize_t b = 0; b < brace_len; b += 2) { + if (c == braces[b]) { + // open brace + if (nesting >= MAX_NESTING) return; // give up + open[nesting].close = braces[b+1]; + open[nesting].pos = i; + open[nesting].at_cursor = (i == cursor_pos - 1); + nesting++; + found_open = true; + break; + } + } + if (found_open) continue; + + // pop to closing brace and potentially highlight + for (ssize_t b = 1; b < brace_len; b += 2) { + if (c == braces[b]) { + // close brace + if (nesting <= 0) { + // unmatched close brace + attrbuf_update_at( attrs, i, 1, error_attr); + } + else { + // can we fix an unmatched brace where we can match by popping just one? + if (open[nesting-1].close != c && nesting > 1 && open[nesting-2].close == c) { + // assume previous open brace was wrong + attrbuf_update_at(attrs, open[nesting-1].pos, 1, error_attr); + nesting--; + } + if (open[nesting-1].close != c) { + // unmatched open brace + attrbuf_update_at( attrs, i, 1, error_attr); + } + else { + // matching brace + nesting--; + if (i == cursor_pos - 1 || (open[nesting].at_cursor && open[nesting].pos != i - 1)) { + // highlight matching brace + attrbuf_update_at(attrs, open[nesting].pos, 1, match_attr); + attrbuf_update_at(attrs, i, 1, match_attr); + } + } + } + break; + } + } + } + // note: don't mark further unmatched open braces as in error +} + + +ic_private ssize_t find_matching_brace(const char* s, ssize_t cursor_pos, const char* braces, bool* is_balanced) +{ + if (is_balanced != NULL) { *is_balanced = false; } + bool balanced = true; + ssize_t match = -1; + brace_t open[MAX_NESTING+1]; + ssize_t nesting = 0; + const ssize_t brace_len = ic_strlen(braces); + for (long i = 0; i < ic_strlen(s); i++) { + const char c = s[i]; + // push open brace + bool found_open = false; + for (ssize_t b = 0; b < brace_len; b += 2) { + if (c == braces[b]) { + // open brace + if (nesting >= MAX_NESTING) return -1; // give up + open[nesting].close = braces[b+1]; + open[nesting].pos = i; + open[nesting].at_cursor = (i == cursor_pos - 1); + nesting++; + found_open = true; + break; + } + } + if (found_open) continue; + + // pop to closing brace + for (ssize_t b = 1; b < brace_len; b += 2) { + if (c == braces[b]) { + // close brace + if (nesting <= 0) { + // unmatched close brace + balanced = false; + } + else { + if (open[nesting-1].close != c) { + // unmatched open brace + balanced = false; + } + else { + // matching brace + nesting--; + if (i == cursor_pos - 1) { + // found matching open brace + match = open[nesting].pos + 1; + } + else if (open[nesting].at_cursor) { + // found matching close brace + match = i + 1; + } + } + } + break; + } + } + } + if (nesting != 0) { balanced = false; } + if (is_balanced != NULL) { *is_balanced = balanced; } + return match; +} diff --git a/extern/isocline/src/highlight.h b/extern/isocline/src/highlight.h new file mode 100644 index 00000000..67da02ff --- /dev/null +++ b/extern/isocline/src/highlight.h @@ -0,0 +1,24 @@ +/* ---------------------------------------------------------------------------- + Copyright (c) 2021, Daan Leijen + This is free software; you can redistribute it and/or modify it + under the terms of the MIT License. A copy of the license can be + found in the "LICENSE" file at the root of this distribution. +-----------------------------------------------------------------------------*/ +#pragma once +#ifndef IC_HIGHLIGHT_H +#define IC_HIGHLIGHT_H + +#include "common.h" +#include "attr.h" +#include "term.h" +#include "bbcode.h" + +//------------------------------------------------------------- +// Syntax highlighting +//------------------------------------------------------------- + +ic_private void highlight( alloc_t* mem, bbcode_t* bb, const char* s, attrbuf_t* attrs, ic_highlight_fun_t* highlighter, void* arg ); +ic_private void highlight_match_braces(const char* s, attrbuf_t* attrs, ssize_t cursor_pos, const char* braces, attr_t match_attr, attr_t error_attr); +ic_private ssize_t find_matching_brace(const char* s, ssize_t cursor_pos, const char* braces, bool* is_balanced); + +#endif // IC_HIGHLIGHT_H diff --git a/extern/isocline/src/history.c b/extern/isocline/src/history.c new file mode 100644 index 00000000..440976aa --- /dev/null +++ b/extern/isocline/src/history.c @@ -0,0 +1,269 @@ +/* ---------------------------------------------------------------------------- + Copyright (c) 2021, Daan Leijen + This is free software; you can redistribute it and/or modify it + under the terms of the MIT License. A copy of the license can be + found in the "LICENSE" file at the root of this distribution. +-----------------------------------------------------------------------------*/ +#include +#include +#include + +#include "../include/isocline.h" +#include "common.h" +#include "history.h" +#include "stringbuf.h" + +#define IC_MAX_HISTORY (200) + +struct history_s { + ssize_t count; // current number of entries in use + ssize_t len; // size of elems + const char** elems; // history items (up to count) + const char* fname; // history file + alloc_t* mem; + bool allow_duplicates; // allow duplicate entries? +}; + +ic_private history_t* history_new(alloc_t* mem) { + history_t* h = mem_zalloc_tp(mem,history_t); + h->mem = mem; + return h; +} + +ic_private void history_free(history_t* h) { + if (h == NULL) return; + history_clear(h); + if (h->len > 0) { + mem_free( h->mem, h->elems ); + h->elems = NULL; + h->len = 0; + } + mem_free(h->mem, h->fname); + h->fname = NULL; + mem_free(h->mem, h); // free ourselves +} + +ic_private bool history_enable_duplicates( history_t* h, bool enable ) { + bool prev = h->allow_duplicates; + h->allow_duplicates = enable; + return prev; +} + +ic_private ssize_t history_count(const history_t* h) { + return h->count; +} + +//------------------------------------------------------------- +// push/clear +//------------------------------------------------------------- + +ic_private bool history_update( history_t* h, const char* entry ) { + if (entry==NULL) return false; + history_remove_last(h); + history_push(h,entry); + //debug_msg("history: update: with %s; now at %s\n", entry, history_get(h,0)); + return true; +} + +static void history_delete_at( history_t* h, ssize_t idx ) { + if (idx < 0 || idx >= h->count) return; + mem_free(h->mem, h->elems[idx]); + for(ssize_t i = idx+1; i < h->count; i++) { + h->elems[i-1] = h->elems[i]; + } + h->count--; +} + +ic_private bool history_push( history_t* h, const char* entry ) { + if (h->len <= 0 || entry==NULL) return false; + // remove any older duplicate + if (!h->allow_duplicates) { + for( int i = 0; i < h->count; i++) { + if (strcmp(h->elems[i],entry) == 0) { + history_delete_at(h,i); + } + } + } + // insert at front + if (h->count == h->len) { + // delete oldest entry + history_delete_at(h,0); + } + assert(h->count < h->len); + h->elems[h->count] = mem_strdup(h->mem,entry); + h->count++; + return true; +} + + +static void history_remove_last_n( history_t* h, ssize_t n ) { + if (n <= 0) return; + if (n > h->count) n = h->count; + for( ssize_t i = h->count - n; i < h->count; i++) { + mem_free( h->mem, h->elems[i] ); + } + h->count -= n; + assert(h->count >= 0); +} + +ic_private void history_remove_last(history_t* h) { + history_remove_last_n(h,1); +} + +ic_private void history_clear(history_t* h) { + history_remove_last_n( h, h->count ); +} + +ic_private const char* history_get( const history_t* h, ssize_t n ) { + if (n < 0 || n >= h->count) return NULL; + return h->elems[h->count - n - 1]; +} + +ic_private bool history_search( const history_t* h, ssize_t from /*including*/, const char* search, bool backward, ssize_t* hidx, ssize_t* hpos ) { + const char* p = NULL; + ssize_t i; + if (backward) { + for( i = from; i < h->count; i++ ) { + p = strstr( history_get(h,i), search); + if (p != NULL) break; + } + } + else { + for( i = from; i >= 0; i-- ) { + p = strstr( history_get(h,i), search); + if (p != NULL) break; + } + } + if (p == NULL) return false; + if (hidx != NULL) *hidx = i; + if (hpos != NULL) *hpos = (p - history_get(h,i)); + return true; +} + +//------------------------------------------------------------- +// +//------------------------------------------------------------- + +ic_private void history_load_from(history_t* h, const char* fname, long max_entries ) { + history_clear(h); + h->fname = mem_strdup(h->mem,fname); + if (max_entries == 0) { + assert(h->elems == NULL); + return; + } + if (max_entries < 0 || max_entries > IC_MAX_HISTORY) max_entries = IC_MAX_HISTORY; + h->elems = (const char**)mem_zalloc_tp_n(h->mem, char*, max_entries ); + if (h->elems == NULL) return; + h->len = max_entries; + history_load(h); +} + + + + +//------------------------------------------------------------- +// save/load history to file +//------------------------------------------------------------- + +static char from_xdigit( int c ) { + if (c >= '0' && c <= '9') return (char)(c - '0'); + if (c >= 'A' && c <= 'F') return (char)(10 + (c - 'A')); + if (c >= 'a' && c <= 'f') return (char)(10 + (c - 'a')); + return 0; +} + +static char to_xdigit( uint8_t c ) { + if (c <= 9) return ((char)c + '0'); + if (c >= 10 && c <= 15) return ((char)c - 10 + 'A'); + return '0'; +} + +static bool ic_isxdigit( int c ) { + return ((c >= 'a' && c <= 'f') || (c >= 'A' && c <= 'F') || (c >= '0' && c <= '9')); +} + +static bool history_read_entry( history_t* h, FILE* f, stringbuf_t* sbuf ) { + sbuf_clear(sbuf); + while( !feof(f)) { + int c = fgetc(f); + if (c == EOF || c == '\n') break; + if (c == '\\') { + c = fgetc(f); + if (c == 'n') { sbuf_append(sbuf,"\n"); } + else if (c == 'r') { /* ignore */ } // sbuf_append(sbuf,"\r"); + else if (c == 't') { sbuf_append(sbuf,"\t"); } + else if (c == '\\') { sbuf_append(sbuf,"\\"); } + else if (c == 'x') { + int c1 = fgetc(f); + int c2 = fgetc(f); + if (ic_isxdigit(c1) && ic_isxdigit(c2)) { + char chr = from_xdigit(c1)*16 + from_xdigit(c2); + sbuf_append_char(sbuf,chr); + } + else return false; + } + else return false; + } + else sbuf_append_char(sbuf,(char)c); + } + if (sbuf_len(sbuf)==0 || sbuf_string(sbuf)[0] == '#') return true; + return history_push(h, sbuf_string(sbuf)); +} + +static bool history_write_entry( const char* entry, FILE* f, stringbuf_t* sbuf ) { + sbuf_clear(sbuf); + //debug_msg("history: write: %s\n", entry); + while( entry != NULL && *entry != 0 ) { + char c = *entry++; + if (c == '\\') { sbuf_append(sbuf,"\\\\"); } + else if (c == '\n') { sbuf_append(sbuf,"\\n"); } + else if (c == '\r') { /* ignore */ } // sbuf_append(sbuf,"\\r"); } + else if (c == '\t') { sbuf_append(sbuf,"\\t"); } + else if (c < ' ' || c > '~' || c == '#') { + char c1 = to_xdigit( (uint8_t)c / 16 ); + char c2 = to_xdigit( (uint8_t)c % 16 ); + sbuf_append(sbuf,"\\x"); + sbuf_append_char(sbuf,c1); + sbuf_append_char(sbuf,c2); + } + else sbuf_append_char(sbuf,c); + } + //debug_msg("history: write buf: %s\n", sbuf_string(sbuf)); + + if (sbuf_len(sbuf) > 0) { + sbuf_append(sbuf,"\n"); + fputs(sbuf_string(sbuf),f); + } + return true; +} + +ic_private void history_load( history_t* h ) { + if (h->fname == NULL) return; + FILE* f = fopen(h->fname, "r"); + if (f == NULL) return; + stringbuf_t* sbuf = sbuf_new(h->mem); + if (sbuf != NULL) { + while (!feof(f)) { + if (!history_read_entry(h,f,sbuf)) break; // error + } + sbuf_free(sbuf); + } + fclose(f); +} + +ic_private void history_save( const history_t* h ) { + if (h->fname == NULL) return; + FILE* f = fopen(h->fname, "w"); + if (f == NULL) return; + #ifndef _WIN32 + chmod(h->fname,S_IRUSR|S_IWUSR); + #endif + stringbuf_t* sbuf = sbuf_new(h->mem); + if (sbuf != NULL) { + for( int i = 0; i < h->count; i++ ) { + if (!history_write_entry(h->elems[i],f,sbuf)) break; // error + } + sbuf_free(sbuf); + } + fclose(f); +} diff --git a/extern/isocline/src/history.h b/extern/isocline/src/history.h new file mode 100644 index 00000000..76a37160 --- /dev/null +++ b/extern/isocline/src/history.h @@ -0,0 +1,38 @@ +/* ---------------------------------------------------------------------------- + Copyright (c) 2021, Daan Leijen + This is free software; you can redistribute it and/or modify it + under the terms of the MIT License. A copy of the license can be + found in the "LICENSE" file at the root of this distribution. +-----------------------------------------------------------------------------*/ +#pragma once +#ifndef IC_HISTORY_H +#define IC_HISTORY_H + +#include "common.h" + +//------------------------------------------------------------- +// History +//------------------------------------------------------------- + +struct history_s; +typedef struct history_s history_t; + +ic_private history_t* history_new(alloc_t* mem); +ic_private void history_free(history_t* h); +ic_private void history_clear(history_t* h); +ic_private bool history_enable_duplicates( history_t* h, bool enable ); +ic_private ssize_t history_count(const history_t* h); + +ic_private void history_load_from(history_t* h, const char* fname, long max_entries); +ic_private void history_load( history_t* h ); +ic_private void history_save( const history_t* h ); + +ic_private bool history_push( history_t* h, const char* entry ); +ic_private bool history_update( history_t* h, const char* entry ); +ic_private const char* history_get( const history_t* h, ssize_t n ); +ic_private void history_remove_last(history_t* h); + +ic_private bool history_search( const history_t* h, ssize_t from, const char* search, bool backward, ssize_t* hidx, ssize_t* hpos); + + +#endif // IC_HISTORY_H diff --git a/extern/isocline/src/isocline.c b/extern/isocline/src/isocline.c new file mode 100644 index 00000000..13278062 --- /dev/null +++ b/extern/isocline/src/isocline.c @@ -0,0 +1,589 @@ +/* ---------------------------------------------------------------------------- + Copyright (c) 2021, Daan Leijen + This is free software; you can redistribute it and/or modify it + under the terms of the MIT License. A copy of the license can be + found in the "LICENSE" file at the root of this distribution. +-----------------------------------------------------------------------------*/ + +//------------------------------------------------------------- +// Usually we include all sources one file so no internal +// symbols are public in the libray. +// +// You can compile the entire library just as: +// $ gcc -c src/isocline.c +//------------------------------------------------------------- +#if !defined(IC_SEPARATE_OBJS) +# define _CRT_SECURE_NO_WARNINGS // for msvc +# define _XOPEN_SOURCE 700 // for wcwidth +# define _DEFAULT_SOURCE // ensure usleep stays visible with _XOPEN_SOURCE >= 700 +# include "attr.c" +# include "bbcode.c" +# include "editline.c" +# include "highlight.c" +# include "undo.c" +# include "history.c" +# include "completers.c" +# include "completions.c" +# include "term.c" +# include "tty_esc.c" +# include "tty.c" +# include "stringbuf.c" +# include "common.c" +#endif + +//------------------------------------------------------------- +// includes +//------------------------------------------------------------- +#include +#include +#include +#include + +#include "../include/isocline.h" +#include "common.h" +#include "env.h" + + +//------------------------------------------------------------- +// Readline +//------------------------------------------------------------- + +static char* ic_getline( alloc_t* mem ); + +ic_public char* ic_readline(const char* prompt_text) +{ + ic_env_t* env = ic_get_env(); + if (env == NULL) return NULL; + if (!env->noedit) { + // terminal editing enabled + return ic_editline(env, prompt_text); // in editline.c + } + else { + // no editing capability (pipe, dumb terminal, etc) + if (env->tty != NULL && env->term != NULL) { + // if the terminal is not interactive, but we are reading from the tty (keyboard), we display a prompt + term_start_raw(env->term); // set utf8 mode on windows + if (prompt_text != NULL) { + term_write(env->term, prompt_text); + } + term_write(env->term, env->prompt_marker); + term_end_raw(env->term, false); + } + // read directly from stdin + return ic_getline(env->mem); + } +} + + +//------------------------------------------------------------- +// Read a line from the stdin stream if there is no editing +// support (like from a pipe, file, or dumb terminal). +//------------------------------------------------------------- + +static char* ic_getline(alloc_t* mem) +{ + // read until eof or newline + stringbuf_t* sb = sbuf_new(mem); + int c; + while (true) { + c = fgetc(stdin); + if (c==EOF || c=='\n') { + break; + } + else { + sbuf_append_char(sb, (char)c); + } + } + return sbuf_free_dup(sb); +} + + +//------------------------------------------------------------- +// Formatted output +//------------------------------------------------------------- + + +ic_public void ic_printf(const char* fmt, ...) { + va_list ap; + va_start(ap, fmt); + ic_vprintf(fmt, ap); + va_end(ap); +} + +ic_public void ic_vprintf(const char* fmt, va_list args) { + ic_env_t* env = ic_get_env(); if (env==NULL || env->bbcode == NULL) return; + bbcode_vprintf(env->bbcode, fmt, args); +} + +ic_public void ic_print(const char* s) { + ic_env_t* env = ic_get_env(); if (env==NULL || env->bbcode==NULL) return; + bbcode_print(env->bbcode, s); +} + +ic_public void ic_println(const char* s) { + ic_env_t* env = ic_get_env(); if (env==NULL || env->bbcode==NULL) return; + bbcode_println(env->bbcode, s); +} + +void ic_style_def(const char* name, const char* fmt) { + ic_env_t* env = ic_get_env(); if (env==NULL || env->bbcode==NULL) return; + bbcode_style_def(env->bbcode, name, fmt); +} + +void ic_style_open(const char* fmt) { + ic_env_t* env = ic_get_env(); if (env==NULL || env->bbcode==NULL) return; + bbcode_style_open(env->bbcode, fmt); +} + +void ic_style_close(void) { + ic_env_t* env = ic_get_env(); if (env==NULL || env->bbcode==NULL) return; + bbcode_style_close(env->bbcode, NULL); +} + + +//------------------------------------------------------------- +// Interface +//------------------------------------------------------------- + +ic_public bool ic_async_stop(void) { + ic_env_t* env = ic_get_env(); if (env==NULL) return false; + if (env->tty==NULL) return false; + return tty_async_stop(env->tty); +} + +static void set_prompt_marker(ic_env_t* env, const char* prompt_marker, const char* cprompt_marker) { + if (prompt_marker == NULL) prompt_marker = "> "; + if (cprompt_marker == NULL) cprompt_marker = prompt_marker; + mem_free(env->mem, env->prompt_marker); + mem_free(env->mem, env->cprompt_marker); + env->prompt_marker = mem_strdup(env->mem, prompt_marker); + env->cprompt_marker = mem_strdup(env->mem, cprompt_marker); +} + +ic_public const char* ic_get_prompt_marker(void) { + ic_env_t* env = ic_get_env(); if (env==NULL) return NULL; + return env->prompt_marker; +} + +ic_public const char* ic_get_continuation_prompt_marker(void) { + ic_env_t* env = ic_get_env(); if (env==NULL) return NULL; + return env->cprompt_marker; +} + +ic_public void ic_set_prompt_marker( const char* prompt_marker, const char* cprompt_marker ) { + ic_env_t* env = ic_get_env(); if (env==NULL) return; + set_prompt_marker(env, prompt_marker, cprompt_marker); +} + +ic_public bool ic_enable_multiline( bool enable ) { + ic_env_t* env = ic_get_env(); if (env==NULL) return false; + bool prev = env->singleline_only; + env->singleline_only = !enable; + return !prev; +} + +ic_public bool ic_enable_beep( bool enable ) { + ic_env_t* env = ic_get_env(); if (env==NULL) return false; + return term_enable_beep(env->term, enable); +} + +ic_public bool ic_enable_color( bool enable ) { + ic_env_t* env = ic_get_env(); if (env==NULL) return false; + return term_enable_color( env->term, enable ); +} + +ic_public bool ic_enable_history_duplicates( bool enable ) { + ic_env_t* env = ic_get_env(); if (env==NULL) return false; + return history_enable_duplicates(env->history, enable); +} + +ic_public void ic_set_history(const char* fname, long max_entries ) { + ic_env_t* env = ic_get_env(); if (env==NULL) return; + history_load_from(env->history, fname, max_entries ); +} + +ic_public void ic_history_remove_last(void) { + ic_env_t* env = ic_get_env(); if (env==NULL) return; + history_remove_last(env->history); +} + +ic_public void ic_history_add( const char* entry ) { + ic_env_t* env = ic_get_env(); if (env==NULL) return; + history_push( env->history, entry ); +} + +ic_public void ic_history_clear(void) { + ic_env_t* env = ic_get_env(); if (env==NULL) return; + history_clear(env->history); +} + +ic_public bool ic_enable_auto_tab( bool enable ) { + ic_env_t* env = ic_get_env(); if (env==NULL) return false; + bool prev = env->complete_autotab; + env->complete_autotab = enable; + return prev; +} + +ic_public bool ic_enable_completion_preview( bool enable ) { + ic_env_t* env = ic_get_env(); if (env==NULL) return false; + bool prev = env->complete_nopreview; + env->complete_nopreview = !enable; + return !prev; +} + +ic_public bool ic_enable_multiline_indent(bool enable) { + ic_env_t* env = ic_get_env(); if (env==NULL) return false; + bool prev = env->no_multiline_indent; + env->no_multiline_indent = !enable; + return !prev; +} + +ic_public bool ic_enable_hint(bool enable) { + ic_env_t* env = ic_get_env(); if (env==NULL) return false; + bool prev = env->no_hint; + env->no_hint = !enable; + return !prev; +} + +ic_public long ic_set_hint_delay(long delay_ms) { + ic_env_t* env = ic_get_env(); if (env==NULL) return false; + long prev = env->hint_delay; + env->hint_delay = (delay_ms < 0 ? 0 : (delay_ms > 5000 ? 5000 : delay_ms)); + return prev; +} + +ic_public void ic_set_tty_esc_delay(long initial_delay_ms, long followup_delay_ms ) { + ic_env_t* env = ic_get_env(); if (env==NULL) return; + if (env->tty == NULL) return; + tty_set_esc_delay(env->tty, initial_delay_ms, followup_delay_ms); +} + + +ic_public bool ic_enable_highlight(bool enable) { + ic_env_t* env = ic_get_env(); if (env==NULL) return false; + bool prev = env->no_highlight; + env->no_highlight = !enable; + return !prev; +} + +ic_public bool ic_enable_inline_help(bool enable) { + ic_env_t* env = ic_get_env(); if (env==NULL) return false; + bool prev = env->no_help; + env->no_help = !enable; + return !prev; +} + +ic_public bool ic_enable_brace_matching(bool enable) { + ic_env_t* env = ic_get_env(); if (env==NULL) return false; + bool prev = env->no_bracematch; + env->no_bracematch = !enable; + return !prev; +} + +ic_public void ic_set_matching_braces(const char* brace_pairs) { + ic_env_t* env = ic_get_env(); if (env==NULL) return; + mem_free(env->mem, env->match_braces); + env->match_braces = NULL; + if (brace_pairs != NULL) { + ssize_t len = ic_strlen(brace_pairs); + if (len > 0 && (len % 2) == 0) { + env->match_braces = mem_strdup(env->mem, brace_pairs); + } + } +} + +ic_public bool ic_enable_brace_insertion(bool enable) { + ic_env_t* env = ic_get_env(); if (env==NULL) return false; + bool prev = env->no_autobrace; + env->no_autobrace = !enable; + return !prev; +} + +ic_public void ic_set_insertion_braces(const char* brace_pairs) { + ic_env_t* env = ic_get_env(); if (env==NULL) return; + mem_free(env->mem, env->auto_braces); + env->auto_braces = NULL; + if (brace_pairs != NULL) { + ssize_t len = ic_strlen(brace_pairs); + if (len > 0 && (len % 2) == 0) { + env->auto_braces = mem_strdup(env->mem, brace_pairs); + } + } +} + +ic_private const char* ic_env_get_match_braces(ic_env_t* env) { + return (env->match_braces == NULL ? "()[]{}" : env->match_braces); +} + +ic_private const char* ic_env_get_auto_braces(ic_env_t* env) { + return (env->auto_braces == NULL ? "()[]{}\"\"''" : env->auto_braces); +} + +ic_public void ic_set_default_highlighter(ic_highlight_fun_t* highlighter, void* arg) { + ic_env_t* env = ic_get_env(); if (env==NULL) return; + env->highlighter = highlighter; + env->highlighter_arg = arg; +} + + +ic_public void ic_free( void* p ) { + ic_env_t* env = ic_get_env(); if (env==NULL) return; + mem_free(env->mem, p); +} + +ic_public void* ic_malloc(size_t sz) { + ic_env_t* env = ic_get_env(); if (env==NULL) return NULL; + return mem_malloc(env->mem, to_ssize_t(sz)); +} + +ic_public const char* ic_strdup( const char* s ) { + if (s==NULL) return NULL; + ic_env_t* env = ic_get_env(); if (env==NULL) return NULL; + ssize_t len = ic_strlen(s); + char* p = mem_malloc_tp_n( env->mem, char, len + 1 ); + if (p == NULL) return NULL; + ic_memcpy( p, s, len ); + p[len] = 0; + return p; +} + +//------------------------------------------------------------- +// Terminal +//------------------------------------------------------------- + +ic_public void ic_term_init(void) { + ic_env_t* env = ic_get_env(); if (env==NULL) return; + if (env->term==NULL) return; + term_start_raw(env->term); +} + +ic_public void ic_term_done(void) { + ic_env_t* env = ic_get_env(); if (env==NULL) return; + if (env->term==NULL) return; + term_end_raw(env->term,false); +} + +ic_public void ic_term_flush(void) { + ic_env_t* env = ic_get_env(); if (env==NULL) return; + if (env->term==NULL) return; + term_flush(env->term); +} + +ic_public void ic_term_write(const char* s) { + ic_env_t* env = ic_get_env(); if (env==NULL) return; + if (env->term == NULL) return; + term_write(env->term, s); +} + +ic_public void ic_term_writeln(const char* s) { + ic_env_t* env = ic_get_env(); if (env==NULL) return; + if (env->term == NULL) return; + term_writeln(env->term, s); +} + +ic_public void ic_term_writef(const char* fmt, ...) { + va_list ap; + va_start(ap, fmt); + ic_term_vwritef(fmt, ap); + va_end(ap); +} + +ic_public void ic_term_vwritef(const char* fmt, va_list args) { + ic_env_t* env = ic_get_env(); if (env==NULL) return; + if (env->term == NULL) return; + term_vwritef(env->term, fmt, args); +} + +ic_public void ic_term_reset( void ) { + ic_env_t* env = ic_get_env(); if (env==NULL) return; + if (env->term == NULL) return; + term_attr_reset(env->term); +} + +ic_public void ic_term_style( const char* style ) { + ic_env_t* env = ic_get_env(); if (env==NULL) return; + if (env->term == NULL || env->bbcode == NULL) return; + term_set_attr( env->term, bbcode_style(env->bbcode, style)); +} + +ic_public int ic_term_get_color_bits(void) { + ic_env_t* env = ic_get_env(); + if (env==NULL || env->term==NULL) return 4; + return term_get_color_bits(env->term); +} + +ic_public void ic_term_bold(bool enable) { + ic_env_t* env = ic_get_env(); if (env==NULL || env->term==NULL) return; + term_bold(env->term, enable); +} + +ic_public void ic_term_underline(bool enable) { + ic_env_t* env = ic_get_env(); if (env==NULL || env->term==NULL) return; + term_underline(env->term, enable); +} + +ic_public void ic_term_italic(bool enable) { + ic_env_t* env = ic_get_env(); if (env==NULL || env->term==NULL) return; + term_italic(env->term, enable); +} + +ic_public void ic_term_reverse(bool enable) { + ic_env_t* env = ic_get_env(); if (env==NULL || env->term==NULL) return; + term_reverse(env->term, enable); +} + +ic_public void ic_term_color_ansi(bool foreground, int ansi_color) { + ic_env_t* env = ic_get_env(); if (env==NULL || env->term==NULL) return; + ic_color_t color = color_from_ansi256(ansi_color); + if (foreground) { term_color(env->term, color); } + else { term_bgcolor(env->term, color); } +} + +ic_public void ic_term_color_rgb(bool foreground, uint32_t hcolor) { + ic_env_t* env = ic_get_env(); if (env==NULL || env->term==NULL) return; + ic_color_t color = ic_rgb(hcolor); + if (foreground) { term_color(env->term, color); } + else { term_bgcolor(env->term, color); } +} + + +//------------------------------------------------------------- +// Readline with temporary completer and highlighter +//------------------------------------------------------------- + +ic_public char* ic_readline_ex(const char* prompt_text, + ic_completer_fun_t* completer, void* completer_arg, + ic_highlight_fun_t* highlighter, void* highlighter_arg ) +{ + ic_env_t* env = ic_get_env(); if (env == NULL) return NULL; + // save previous + ic_completer_fun_t* prev_completer; + void* prev_completer_arg; + completions_get_completer(env->completions, &prev_completer, &prev_completer_arg); + ic_highlight_fun_t* prev_highlighter = env->highlighter; + void* prev_highlighter_arg = env->highlighter_arg; + // call with current + if (completer != NULL) { ic_set_default_completer(completer, completer_arg); } + if (highlighter != NULL) { ic_set_default_highlighter(highlighter, highlighter_arg); } + char* res = ic_readline(prompt_text); + // restore previous + ic_set_default_completer(prev_completer, prev_completer_arg); + ic_set_default_highlighter(prev_highlighter, prev_highlighter_arg); + return res; +} + + +//------------------------------------------------------------- +// Initialize +//------------------------------------------------------------- + +static void ic_atexit(void); + +static void ic_env_free(ic_env_t* env) { + if (env == NULL) return; + history_save(env->history); + history_free(env->history); + completions_free(env->completions); + bbcode_free(env->bbcode); + term_free(env->term); + tty_free(env->tty); + mem_free(env->mem, env->cprompt_marker); + mem_free(env->mem,env->prompt_marker); + mem_free(env->mem, env->match_braces); + mem_free(env->mem, env->auto_braces); + env->prompt_marker = NULL; + + // and deallocate ourselves + alloc_t* mem = env->mem; + mem_free(mem, env); + + // and finally the custom memory allocation structure + mem_free(mem, mem); +} + + +static ic_env_t* ic_env_create( ic_malloc_fun_t* _malloc, ic_realloc_fun_t* _realloc, ic_free_fun_t* _free ) +{ + if (_malloc == NULL) _malloc = &malloc; + if (_realloc == NULL) _realloc = &realloc; + if (_free == NULL) _free = &free; + // allocate + alloc_t* mem = (alloc_t*)_malloc(sizeof(alloc_t)); + if (mem == NULL) return NULL; + mem->malloc = _malloc; + mem->realloc = _realloc; + mem->free = _free; + ic_env_t* env = mem_zalloc_tp(mem, ic_env_t); + if (env==NULL) { + mem->free(mem); + return NULL; + } + env->mem = mem; + + // Initialize + env->tty = tty_new(env->mem, -1); // can return NULL + env->term = term_new(env->mem, env->tty, false, false, -1 ); + env->history = history_new(env->mem); + env->completions = completions_new(env->mem); + env->bbcode = bbcode_new(env->mem, env->term); + env->hint_delay = 400; + + if (env->tty == NULL || env->term==NULL || + env->completions == NULL || env->history == NULL || env->bbcode == NULL || + !term_is_interactive(env->term)) + { + env->noedit = true; + } + env->multiline_eol = '\\'; + + bbcode_style_def(env->bbcode, "ic-prompt", "ansi-green" ); + bbcode_style_def(env->bbcode, "ic-info", "ansi-darkgray" ); + bbcode_style_def(env->bbcode, "ic-diminish", "ansi-lightgray" ); + bbcode_style_def(env->bbcode, "ic-emphasis", "#ffffd7" ); + bbcode_style_def(env->bbcode, "ic-hint", "ansi-darkgray" ); + bbcode_style_def(env->bbcode, "ic-error", "#d70000" ); + bbcode_style_def(env->bbcode, "ic-bracematch","ansi-white"); // color = #F7DC6F" ); + + bbcode_style_def(env->bbcode, "keyword", "#569cd6" ); + bbcode_style_def(env->bbcode, "control", "#c586c0" ); + bbcode_style_def(env->bbcode, "number", "#b5cea8" ); + bbcode_style_def(env->bbcode, "string", "#ce9178" ); + bbcode_style_def(env->bbcode, "comment", "#6A9955" ); + bbcode_style_def(env->bbcode, "type", "darkcyan" ); + bbcode_style_def(env->bbcode, "constant", "#569cd6" ); + + set_prompt_marker(env, NULL, NULL); + return env; +} + +static ic_env_t* rpenv; + +static void ic_atexit(void) { + if (rpenv != NULL) { + ic_env_free(rpenv); + rpenv = NULL; + } +} + +ic_private ic_env_t* ic_get_env(void) { + if (rpenv==NULL) { + rpenv = ic_env_create( NULL, NULL, NULL ); + if (rpenv != NULL) { atexit( &ic_atexit ); } + } + return rpenv; +} + +ic_public void ic_init_custom_malloc( ic_malloc_fun_t* _malloc, ic_realloc_fun_t* _realloc, ic_free_fun_t* _free ) { + assert(rpenv == NULL); + if (rpenv != NULL) { + ic_env_free(rpenv); + rpenv = ic_env_create( _malloc, _realloc, _free ); + } + else { + rpenv = ic_env_create( _malloc, _realloc, _free ); + if (rpenv != NULL) { + atexit( &ic_atexit ); + } + } +} + diff --git a/extern/isocline/src/stringbuf.c b/extern/isocline/src/stringbuf.c new file mode 100644 index 00000000..7bbfad04 --- /dev/null +++ b/extern/isocline/src/stringbuf.c @@ -0,0 +1,1038 @@ +/* ---------------------------------------------------------------------------- + Copyright (c) 2021, Daan Leijen + This is free software; you can redistribute it and/or modify it + under the terms of the MIT License. A copy of the license can be + found in the "LICENSE" file at the root of this distribution. +-----------------------------------------------------------------------------*/ + +// get `wcwidth` for the column width of unicode characters +// note: for now the OS provided one is unused as we see quite a bit of variation +// among platforms and including our own seems more reliable. +/* +#if defined(__linux__) || defined(__freebsd__) +// use the system supplied one +#if !defined(_XOPEN_SOURCE) +#define _XOPEN_SOURCE 700 // so wcwidth is visible +#endif +#include +#else +*/ +// use our own (also on APPLE as that fails within vscode) +#define wcwidth(c) mk_wcwidth(c) +#include "wcwidth.c" +// #endif + +#include +#include +#include + +#include "common.h" +#include "stringbuf.h" + +//------------------------------------------------------------- +// In place growable utf-8 strings +//------------------------------------------------------------- + +struct stringbuf_s { + char* buf; + ssize_t buflen; + ssize_t count; + alloc_t* mem; +}; + + +//------------------------------------------------------------- +// String column width +//------------------------------------------------------------- + +// column width of a utf8 single character sequence. +static ssize_t utf8_char_width( const char* s, ssize_t n ) { + if (n <= 0) return 0; + + uint8_t b = (uint8_t)s[0]; + int32_t c; + if (b < ' ') { + return 0; + } + else if (b <= 0x7F) { + return 1; + } + else if (b <= 0xC1) { // invalid continuation byte or invalid 0xC0, 0xC1 (check is strictly not necessary as we don't validate..) + return 1; + } + else if (b <= 0xDF && n >= 2) { // b >= 0xC2 // 2 bytes + c = (((b & 0x1F) << 6) | (s[1] & 0x3F)); + assert(c < 0xD800 || c > 0xDFFF); + int w = wcwidth(c); + return w; + } + else if (b <= 0xEF && n >= 3) { // b >= 0xE0 // 3 bytes + c = (((b & 0x0F) << 12) | ((s[1] & 0x3F) << 6) | (s[2] & 0x3F)); + return wcwidth(c); + } + else if (b <= 0xF4 && n >= 4) { // b >= 0xF0 // 4 bytes + c = (((b & 0x07) << 18) | ((s[1] & 0x3F) << 12) | ((s[2] & 0x3F) << 6) | (s[3] & 0x3F)); + return wcwidth(c); + } + else { + // failed + return 1; + } +} + + +// The column width of a codepoint (0, 1, or 2) +static ssize_t char_column_width( const char* s, ssize_t n ) { + if (s == NULL || n <= 0) return 0; + else if ((uint8_t)(*s) < ' ') return 0; // also for CSI escape sequences + else { + ssize_t w = utf8_char_width(s, n); + #ifdef _WIN32 + return (w <= 0 ? 1 : w); // windows console seems to use at least one column + #else + return w; + #endif + } +} + +static ssize_t str_column_width_n( const char* s, ssize_t len ) { + if (s == NULL || len <= 0) return 0; + ssize_t pos = 0; + ssize_t cwidth = 0; + ssize_t cw; + ssize_t ofs; + while (s[pos] != 0 && (ofs = str_next_ofs(s, len, pos, &cw)) > 0) { + cwidth += cw; + pos += ofs; + } + return cwidth; +} + +ic_private ssize_t str_column_width( const char* s ) { + return str_column_width_n( s, ic_strlen(s) ); +} + +ic_private ssize_t str_skip_until_fit( const char* s, ssize_t max_width ) { + if (s == NULL) return 0; + ssize_t cwidth = str_column_width(s); + ssize_t len = ic_strlen(s); + ssize_t pos = 0; + ssize_t next; + ssize_t cw; + while (cwidth > max_width && (next = str_next_ofs(s, len, pos, &cw)) > 0) { + cwidth -= cw; + pos += next; + } + return pos; +} + +ic_private ssize_t str_take_while_fit( const char* s, ssize_t max_width) { + if (s == NULL) return 0; + const ssize_t len = ic_strlen(s); + ssize_t pos = 0; + ssize_t next; + ssize_t cw; + ssize_t cwidth = 0; + while ((next = str_next_ofs(s, len, pos, &cw)) > 0) { + if (cwidth + cw > max_width) break; + cwidth += cw; + pos += next; + } + return pos; +} + + +//------------------------------------------------------------- +// String navigation +//------------------------------------------------------------- + +// get offset of the previous codepoint. does not skip back over CSI sequences. +ic_private ssize_t str_prev_ofs( const char* s, ssize_t pos, ssize_t* width ) { + ssize_t ofs = 0; + if (s != NULL && pos > 0) { + ofs = 1; + while (pos > ofs) { + uint8_t u = (uint8_t)s[pos - ofs]; + if (u < 0x80 || u > 0xBF) break; // continue while follower + ofs++; + } + } + if (width != NULL) *width = char_column_width( s+(pos-ofs), ofs ); + return ofs; +} + +// skip an escape sequence +// +ic_private bool skip_esc( const char* s, ssize_t len, ssize_t* esclen ) { + if (s == NULL || len <= 1 || s[0] != '\x1B') return false; + if (esclen != NULL) *esclen = 0; + if (strchr("[PX^_]",s[1]) != NULL) { + // CSI (ESC [), DCS (ESC P), SOS (ESC X), PM (ESC ^), APC (ESC _), and OSC (ESC ]): terminated with a special sequence + bool finalCSI = (s[1] == '['); // CSI terminates with 0x40-0x7F; otherwise ST (bell or ESC \) + ssize_t n = 2; + while (len > n) { + char c = s[n++]; + if ((finalCSI && (uint8_t)c >= 0x40 && (uint8_t)c <= 0x7F) || // terminating byte: @A–Z[\]^_`a–z{|}~ + (!finalCSI && c == '\x07') || // bell + (c == '\x02')) // STX terminates as well + { + if (esclen != NULL) *esclen = n; + return true; + } + else if (!finalCSI && c == '\x1B' && len > n && s[n] == '\\') { // ST (ESC \) + n++; + if (esclen != NULL) *esclen = n; + return true; + } + } + } + if (strchr(" #%()*+",s[1]) != NULL) { + // assume escape sequence of length 3 (like ESC % G) + if (esclen != NULL) *esclen = 2; + return true; + } + else { + // assume single character escape code (like ESC 7) + if (esclen != NULL) *esclen = 2; + return true; + } + return false; +} + +// Offset to the next codepoint, treats CSI escape sequences as a single code point. +ic_private ssize_t str_next_ofs( const char* s, ssize_t len, ssize_t pos, ssize_t* cwidth ) { + ssize_t ofs = 0; + if (s != NULL && len > pos) { + if (skip_esc(s+pos,len-pos,&ofs)) { + // skip escape sequence + } + else { + ofs = 1; + // utf8 extended character? + while(len > pos + ofs) { + uint8_t u = (uint8_t)s[pos + ofs]; + if (u < 0x80 || u > 0xBF) break; // break if not a follower + ofs++; + } + } + } + if (cwidth != NULL) *cwidth = char_column_width( s+pos, ofs ); + return ofs; +} + +static ssize_t str_limit_to_length( const char* s, ssize_t n ) { + ssize_t i; + for(i = 0; i < n && s[i] != 0; i++) { /* nothing */ } + return i; +} + + +//------------------------------------------------------------- +// String searching prev/next word, line, ws_word +//------------------------------------------------------------- + + +static ssize_t str_find_backward( const char* s, ssize_t len, ssize_t pos, ic_is_char_class_fun_t* match, bool skip_immediate_matches ) { + if (pos > len) pos = len; + if (pos < 0) pos = 0; + ssize_t i = pos; + // skip matching first (say, whitespace in case of the previous start-of-word) + if (skip_immediate_matches) { + do { + ssize_t prev = str_prev_ofs(s, i, NULL); + if (prev <= 0) break; + assert(i - prev >= 0); + if (!match(s + i - prev, (long)prev)) break; + i -= prev; + } while (i > 0); + } + // find match + do { + ssize_t prev = str_prev_ofs(s, i, NULL); + if (prev <= 0) break; + assert(i - prev >= 0); + if (match(s + i - prev, (long)prev)) { + return i; // found; + } + i -= prev; + } while (i > 0); + return -1; // not found +} + +static ssize_t str_find_forward( const char* s, ssize_t len, ssize_t pos, ic_is_char_class_fun_t* match, bool skip_immediate_matches ) { + if (s == NULL || len < 0) return -1; + if (pos > len) pos = len; + if (pos < 0) pos = 0; + ssize_t i = pos; + ssize_t next; + // skip matching first (say, whitespace in case of the next end-of-word) + if (skip_immediate_matches) { + do { + next = str_next_ofs(s, len, i, NULL); + if (next <= 0) break; + assert( i + next <= len); + if (!match(s + i, (long)next)) break; + i += next; + } while (i < len); + } + // and then look + do { + next = str_next_ofs(s, len, i, NULL); + if (next <= 0) break; + assert( i + next <= len); + if (match(s + i, (long)next)) { + return i; // found + } + i += next; + } while (i < len); + return -1; +} + +static bool char_is_linefeed( const char* s, long n ) { + return (n == 1 && (*s == '\n' || *s == 0)); +} + +static ssize_t str_find_line_start( const char* s, ssize_t len, ssize_t pos) { + ssize_t start = str_find_backward(s,len,pos,&char_is_linefeed,false /* don't skip immediate matches */); + return (start < 0 ? 0 : start); +} + +static ssize_t str_find_line_end( const char* s, ssize_t len, ssize_t pos) { + ssize_t end = str_find_forward(s,len,pos, &char_is_linefeed, false); + return (end < 0 ? len : end); +} + +static ssize_t str_find_word_start( const char* s, ssize_t len, ssize_t pos) { + ssize_t start = str_find_backward(s,len,pos, &ic_char_is_idletter,true /* skip immediate matches */); + return (start < 0 ? 0 : start); +} + +static ssize_t str_find_word_end( const char* s, ssize_t len, ssize_t pos) { + ssize_t end = str_find_forward(s,len,pos,&ic_char_is_idletter,true /* skip immediate matches */); + return (end < 0 ? len : end); +} + +static ssize_t str_find_ws_word_start( const char* s, ssize_t len, ssize_t pos) { + ssize_t start = str_find_backward(s,len,pos,&ic_char_is_white,true /* skip immediate matches */); + return (start < 0 ? 0 : start); +} + +static ssize_t str_find_ws_word_end( const char* s, ssize_t len, ssize_t pos) { + ssize_t end = str_find_forward(s,len,pos,&ic_char_is_white,true /* skip immediate matches */); + return (end < 0 ? len : end); +} + + +//------------------------------------------------------------- +// String row/column iteration +//------------------------------------------------------------- + +// invoke a function for each terminal row; returns total row count. +static ssize_t str_for_each_row( const char* s, ssize_t len, ssize_t termw, ssize_t promptw, ssize_t cpromptw, + row_fun_t* fun, const void* arg, void* res ) +{ + if (s == NULL) s = ""; + ssize_t i; + ssize_t rcount = 0; + ssize_t rcol = 0; + ssize_t rstart = 0; + ssize_t startw = promptw; + for(i = 0; i < len; ) { + ssize_t w; + ssize_t next = str_next_ofs(s, len, i, &w); + if (next <= 0) { + debug_msg("str: foreach row: next<=0: len %zd, i %zd, w %zd, buf %s\n", len, i, w, s ); + assert(false); + break; + } + startw = (rcount == 0 ? promptw : cpromptw); + ssize_t termcol = rcol + w + startw + 1 /* for the cursor */; + if (termw != 0 && i != 0 && termcol >= termw) { + // wrap + if (fun != NULL) { + if (fun(s,rcount,rstart,i - rstart,startw,true,arg,res)) return rcount; + } + rcount++; + rstart = i; + rcol = 0; + } + if (s[i] == '\n') { + // newline + if (fun != NULL) { + if (fun(s,rcount,rstart,i - rstart,startw,false,arg,res)) return rcount; + } + rcount++; + rstart = i+1; + rcol = 0; + } + assert (s[i] != 0); + i += next; + rcol += w; + } + if (fun != NULL) { + if (fun(s,rcount,rstart,i - rstart,startw,false,arg,res)) return rcount; + } + return rcount+1; +} + +//------------------------------------------------------------- +// String: get row/column position +//------------------------------------------------------------- + + +static bool str_get_current_pos_iter( + const char* s, + ssize_t row, ssize_t row_start, ssize_t row_len, + ssize_t startw, bool is_wrap, const void* arg, void* res) +{ + ic_unused(is_wrap); ic_unused(startw); + rowcol_t* rc = (rowcol_t*)res; + ssize_t pos = *((ssize_t*)arg); + + if (pos >= row_start && pos <= (row_start + row_len)) { + // found the cursor row + rc->row_start = row_start; + rc->row_len = row_len; + rc->row = row; + rc->col = str_column_width_n( s + row_start, pos - row_start ); + rc->first_on_row = (pos == row_start); + if (is_wrap) { + // if wrapped, we check if the next character is at row_len + ssize_t next = str_next_ofs(s, row_start + row_len, pos, NULL); + rc->last_on_row = (pos + next >= row_start + row_len); + } + else { + // normal last position is right after the last character + rc->last_on_row = (pos >= row_start + row_len); + } + // debug_msg("edit; pos iter: pos: %zd (%c), row_start: %zd, rowlen: %zd\n", pos, s[pos], row_start, row_len); + } + return false; // always continue to count all rows +} + +static ssize_t str_get_rc_at_pos(const char* s, ssize_t len, ssize_t termw, ssize_t promptw, ssize_t cpromptw, ssize_t pos, rowcol_t* rc) { + memset(rc, 0, sizeof(*rc)); + ssize_t rows = str_for_each_row(s, len, termw, promptw, cpromptw, &str_get_current_pos_iter, &pos, rc); + // debug_msg("edit: current pos: (%d, %d) %s %s\n", rc->row, rc->col, rc->first_on_row ? "first" : "", rc->last_on_row ? "last" : ""); + return rows; +} + + + +//------------------------------------------------------------- +// String: get row/column position for a resized terminal +// with potentially "hard-wrapped" rows +//------------------------------------------------------------- +typedef struct wrapped_arg_s { + ssize_t pos; + ssize_t newtermw; +} wrapped_arg_t; + +typedef struct wrowcol_s { + rowcol_t rc; + ssize_t hrows; // count of hard-wrapped extra rows +} wrowcol_t; + +static bool str_get_current_wrapped_pos_iter( + const char* s, + ssize_t row, ssize_t row_start, ssize_t row_len, + ssize_t startw, bool is_wrap, const void* arg, void* res) +{ + ic_unused(is_wrap); + wrowcol_t* wrc = (wrowcol_t*)res; + const wrapped_arg_t* warg = (const wrapped_arg_t*)arg; + + // iterate through the row and record the postion and hard-wraps + ssize_t hwidth = startw; + ssize_t i = 0; + while( i <= row_len ) { // include rowlen as the cursor position can be just after the last character + // get next position and column width + ssize_t cw; + ssize_t next; + bool is_cursor = (warg->pos == row_start+i); + if (i < row_len) { + next = str_next_ofs(s + row_start, row_len, i, &cw); + } + else { + // end of row: take wrap or cursor into account + // (wrap has width 2 as it displays a back-arrow but also has an invisible newline that wraps) + cw = (is_wrap ? 2 : (is_cursor ? 1 : 0)); + next = 1; + } + + if (next > 0) { + if (hwidth + cw > warg->newtermw) { + // hardwrap + hwidth = 0; + wrc->hrows++; + debug_msg("str: found hardwrap: row: %zd, hrows: %zd\n", row, wrc->hrows); + } + } + else { + next++; // ensure we terminate (as we go up to rowlen) + } + + // did we find our position? + if (is_cursor) { + debug_msg("str: found position: row: %zd, hrows: %zd\n", row, wrc->hrows); + wrc->rc.row_start = row_start; + wrc->rc.row_len = row_len; + wrc->rc.row = wrc->hrows + row; + wrc->rc.col = hwidth; + wrc->rc.first_on_row = (i==0); + wrc->rc.last_on_row = (i+next >= row_len - (is_wrap ? 1 : 0)); + } + + // advance + hwidth += cw; + i += next; + } + return false; // always continue to count all rows +} + + +static ssize_t str_get_wrapped_rc_at_pos(const char* s, ssize_t len, ssize_t termw, ssize_t newtermw, ssize_t promptw, ssize_t cpromptw, ssize_t pos, rowcol_t* rc) { + wrapped_arg_t warg; + warg.pos = pos; + warg.newtermw = newtermw; + wrowcol_t wrc; + memset(&wrc,0,sizeof(wrc)); + ssize_t rows = str_for_each_row(s, len, termw, promptw, cpromptw, &str_get_current_wrapped_pos_iter, &warg, &wrc); + debug_msg("edit: wrapped pos: (%zd,%zd) rows %zd %s %s, hrows: %zd\n", wrc.rc.row, wrc.rc.col, rows, wrc.rc.first_on_row ? "first" : "", wrc.rc.last_on_row ? "last" : "", wrc.hrows); + *rc = wrc.rc; + return (rows + wrc.hrows); +} + + +//------------------------------------------------------------- +// Set position +//------------------------------------------------------------- + +static bool str_set_pos_iter( + const char* s, + ssize_t row, ssize_t row_start, ssize_t row_len, + ssize_t startw, bool is_wrap, const void* arg, void* res) +{ + ic_unused(arg); ic_unused(is_wrap); ic_unused(startw); + rowcol_t* rc = (rowcol_t*)arg; + if (rc->row != row) return false; // keep searching + // we found our row + ssize_t col = 0; + ssize_t i = row_start; + ssize_t end = row_start + row_len; + while (col < rc->col && i < end) { + ssize_t cw; + ssize_t next = str_next_ofs(s, row_start + row_len, i, &cw); + if (next <= 0) break; + i += next; + col += cw; + } + *((ssize_t*)res) = i; + return true; // stop iteration +} + +static ssize_t str_get_pos_at_rc(const char* s, ssize_t len, ssize_t termw, ssize_t promptw, ssize_t cpromptw, ssize_t row, ssize_t col /* without prompt */) { + rowcol_t rc; + memset(&rc,0,ssizeof(rc)); + rc.row = row; + rc.col = col; + ssize_t pos = -1; + str_for_each_row(s,len,termw,promptw,cpromptw,&str_set_pos_iter,&rc,&pos); + return pos; +} + + +//------------------------------------------------------------- +// String buffer +//------------------------------------------------------------- +static bool sbuf_ensure_extra(stringbuf_t* s, ssize_t extra) +{ + if (s->buflen >= s->count + extra) return true; + // reallocate; pick good initial size and multiples to increase reuse on allocation + ssize_t newlen = (s->buflen <= 0 ? 120 : (s->buflen > 1000 ? s->buflen + 1000 : 2*s->buflen)); + if (newlen < s->count + extra) newlen = s->count + extra; + if (s->buflen > 0) { + debug_msg("stringbuf: reallocate: old %zd, new %zd\n", s->buflen, newlen); + } + char* newbuf = mem_realloc_tp(s->mem, char, s->buf, newlen+1); // one more for terminating zero + if (newbuf == NULL) { + assert(false); + return false; + } + s->buf = newbuf; + s->buflen = newlen; + s->buf[s->count] = s->buf[s->buflen] = 0; + assert(s->buflen >= s->count + extra); + return true; +} + +static void sbuf_init( stringbuf_t* sbuf, alloc_t* mem ) { + sbuf->mem = mem; + sbuf->buf = NULL; + sbuf->buflen = 0; + sbuf->count = 0; +} + +static void sbuf_done( stringbuf_t* sbuf ) { + mem_free( sbuf->mem, sbuf->buf ); + sbuf->buf = NULL; + sbuf->buflen = 0; + sbuf->count = 0; +} + + +ic_private void sbuf_free( stringbuf_t* sbuf ) { + if (sbuf==NULL) return; + sbuf_done(sbuf); + mem_free(sbuf->mem, sbuf); +} + +ic_private stringbuf_t* sbuf_new( alloc_t* mem ) { + stringbuf_t* sbuf = mem_zalloc_tp(mem,stringbuf_t); + if (sbuf == NULL) return NULL; + sbuf_init(sbuf,mem); + return sbuf; +} + +// free the sbuf and return the current string buffer as the result +ic_private char* sbuf_free_dup(stringbuf_t* sbuf) { + if (sbuf == NULL) return NULL; + char* s = NULL; + if (sbuf->buf != NULL) { + s = mem_realloc_tp(sbuf->mem, char, sbuf->buf, sbuf_len(sbuf)+1); + if (s == NULL) { s = sbuf->buf; } + sbuf->buf = 0; + sbuf->buflen = 0; + sbuf->count = 0; + } + sbuf_free(sbuf); + return s; +} + +ic_private const char* sbuf_string_at( stringbuf_t* sbuf, ssize_t pos ) { + if (pos < 0 || sbuf->count < pos) return NULL; + if (sbuf->buf == NULL) return ""; + assert(sbuf->buf[sbuf->count] == 0); + return sbuf->buf + pos; +} + +ic_private const char* sbuf_string( stringbuf_t* sbuf ) { + return sbuf_string_at( sbuf, 0 ); +} + +ic_private char sbuf_char_at(stringbuf_t* sbuf, ssize_t pos) { + if (sbuf->buf == NULL || pos < 0 || sbuf->count < pos) return 0; + return sbuf->buf[pos]; +} + +ic_private char* sbuf_strdup_at( stringbuf_t* sbuf, ssize_t pos ) { + return mem_strdup(sbuf->mem, sbuf_string_at(sbuf,pos)); +} + +ic_private char* sbuf_strdup( stringbuf_t* sbuf ) { + return mem_strdup(sbuf->mem, sbuf_string(sbuf)); +} + +ic_private ssize_t sbuf_len(const stringbuf_t* s) { + if (s == NULL) return 0; + return s->count; +} + +ic_private ssize_t sbuf_append_vprintf(stringbuf_t* sb, const char* fmt, va_list args) { + const ssize_t min_needed = ic_strlen(fmt); + if (!sbuf_ensure_extra(sb,min_needed + 16)) return sb->count; + ssize_t avail = sb->buflen - sb->count; + va_list args0; + va_copy(args0, args); + ssize_t needed = vsnprintf(sb->buf + sb->count, to_size_t(avail), fmt, args0); + if (needed > avail) { + sb->buf[sb->count] = 0; + if (!sbuf_ensure_extra(sb, needed)) return sb->count; + avail = sb->buflen - sb->count; + needed = vsnprintf(sb->buf + sb->count, to_size_t(avail), fmt, args); + } + assert(needed <= avail); + sb->count += (needed > avail ? avail : (needed >= 0 ? needed : 0)); + assert(sb->count <= sb->buflen); + sb->buf[sb->count] = 0; + return sb->count; +} + +ic_private ssize_t sbuf_appendf(stringbuf_t* sb, const char* fmt, ...) { + va_list args; + va_start( args, fmt); + ssize_t res = sbuf_append_vprintf( sb, fmt, args ); + va_end(args); + return res; +} + + +ic_private ssize_t sbuf_insert_at_n(stringbuf_t* sbuf, const char* s, ssize_t n, ssize_t pos ) { + if (pos < 0 || pos > sbuf->count || s == NULL) return pos; + n = str_limit_to_length(s,n); + if (n <= 0 || !sbuf_ensure_extra(sbuf,n)) return pos; + ic_memmove(sbuf->buf + pos + n, sbuf->buf + pos, sbuf->count - pos); + ic_memcpy(sbuf->buf + pos, s, n); + sbuf->count += n; + sbuf->buf[sbuf->count] = 0; + return (pos + n); +} + +ic_private stringbuf_t* sbuf_split_at( stringbuf_t* sb, ssize_t pos ) { + stringbuf_t* res = sbuf_new(sb->mem); + if (res==NULL || pos < 0) return NULL; + if (pos < sb->count) { + sbuf_append_n(res, sb->buf + pos, sb->count - pos); + sb->count = pos; + } + return res; +} + +ic_private ssize_t sbuf_insert_at(stringbuf_t* sbuf, const char* s, ssize_t pos ) { + return sbuf_insert_at_n( sbuf, s, ic_strlen(s), pos ); +} + +ic_private ssize_t sbuf_insert_char_at(stringbuf_t* sbuf, char c, ssize_t pos ) { + char s[2]; + s[0] = c; + s[1] = 0; + return sbuf_insert_at_n( sbuf, s, 1, pos); +} + +ic_private ssize_t sbuf_insert_unicode_at(stringbuf_t* sbuf, unicode_t u, ssize_t pos) { + uint8_t s[5]; + unicode_to_qutf8(u, s); + return sbuf_insert_at(sbuf, (const char*)s, pos); +} + + + +ic_private void sbuf_delete_at( stringbuf_t* sbuf, ssize_t pos, ssize_t count ) { + if (pos < 0 || pos >= sbuf->count) return; + if (pos + count > sbuf->count) count = sbuf->count - pos; + ic_memmove(sbuf->buf + pos, sbuf->buf + pos + count, sbuf->count - pos - count); + sbuf->count -= count; + sbuf->buf[sbuf->count] = 0; +} + +ic_private void sbuf_delete_from_to( stringbuf_t* sbuf, ssize_t pos, ssize_t end ) { + if (end <= pos) return; + sbuf_delete_at( sbuf, pos, end - pos); +} + +ic_private void sbuf_delete_from(stringbuf_t* sbuf, ssize_t pos ) { + sbuf_delete_at(sbuf, pos, sbuf_len(sbuf) - pos ); +} + + +ic_private void sbuf_clear( stringbuf_t* sbuf ) { + sbuf_delete_at(sbuf, 0, sbuf_len(sbuf)); +} + +ic_private ssize_t sbuf_append_n( stringbuf_t* sbuf, const char* s, ssize_t n ) { + return sbuf_insert_at_n( sbuf, s, n, sbuf_len(sbuf)); +} + +ic_private ssize_t sbuf_append( stringbuf_t* sbuf, const char* s ) { + return sbuf_insert_at( sbuf, s, sbuf_len(sbuf)); +} + +ic_private ssize_t sbuf_append_char( stringbuf_t* sbuf, char c ) { + char buf[2]; + buf[0] = c; + buf[1] = 0; + return sbuf_append( sbuf, buf ); +} + +ic_private void sbuf_replace(stringbuf_t* sbuf, const char* s) { + sbuf_clear(sbuf); + sbuf_append(sbuf,s); +} + +ic_private ssize_t sbuf_next_ofs( stringbuf_t* sbuf, ssize_t pos, ssize_t* cwidth ) { + return str_next_ofs( sbuf->buf, sbuf->count, pos, cwidth); +} + +ic_private ssize_t sbuf_prev_ofs( stringbuf_t* sbuf, ssize_t pos, ssize_t* cwidth ) { + return str_prev_ofs( sbuf->buf, pos, cwidth); +} + +ic_private ssize_t sbuf_next( stringbuf_t* sbuf, ssize_t pos, ssize_t* cwidth) { + ssize_t ofs = sbuf_next_ofs(sbuf,pos,cwidth); + if (ofs <= 0) return -1; + assert(pos + ofs <= sbuf->count); + return pos + ofs; +} + +ic_private ssize_t sbuf_prev( stringbuf_t* sbuf, ssize_t pos, ssize_t* cwidth) { + ssize_t ofs = sbuf_prev_ofs(sbuf,pos,cwidth); + if (ofs <= 0) return -1; + assert(pos - ofs >= 0); + return pos - ofs; +} + +ic_private ssize_t sbuf_delete_char_before( stringbuf_t* sbuf, ssize_t pos ) { + ssize_t n = sbuf_prev_ofs(sbuf, pos, NULL); + if (n <= 0) return 0; + assert( pos - n >= 0 ); + sbuf_delete_at(sbuf, pos - n, n); + return pos - n; +} + +ic_private void sbuf_delete_char_at( stringbuf_t* sbuf, ssize_t pos ) { + ssize_t n = sbuf_next_ofs(sbuf, pos, NULL); + if (n <= 0) return; + assert( pos + n <= sbuf->count ); + sbuf_delete_at(sbuf, pos, n); + return; +} + +ic_private ssize_t sbuf_swap_char( stringbuf_t* sbuf, ssize_t pos ) { + ssize_t next = sbuf_next_ofs(sbuf, pos, NULL); + if (next <= 0) return 0; + ssize_t prev = sbuf_prev_ofs(sbuf, pos, NULL); + if (prev <= 0) return 0; + char buf[64]; + if (prev >= 63) return 0; + ic_memcpy(buf, sbuf->buf + pos - prev, prev ); + ic_memmove(sbuf->buf + pos - prev, sbuf->buf + pos, next); + ic_memmove(sbuf->buf + pos - prev + next, buf, prev); + return pos - prev; +} + +ic_private ssize_t sbuf_find_line_start( stringbuf_t* sbuf, ssize_t pos ) { + return str_find_line_start( sbuf->buf, sbuf->count, pos); +} + +ic_private ssize_t sbuf_find_line_end( stringbuf_t* sbuf, ssize_t pos ) { + return str_find_line_end( sbuf->buf, sbuf->count, pos); +} + +ic_private ssize_t sbuf_find_word_start( stringbuf_t* sbuf, ssize_t pos ) { + return str_find_word_start( sbuf->buf, sbuf->count, pos); +} + +ic_private ssize_t sbuf_find_word_end( stringbuf_t* sbuf, ssize_t pos ) { + return str_find_word_end( sbuf->buf, sbuf->count, pos); +} + +ic_private ssize_t sbuf_find_ws_word_start( stringbuf_t* sbuf, ssize_t pos ) { + return str_find_ws_word_start( sbuf->buf, sbuf->count, pos); +} + +ic_private ssize_t sbuf_find_ws_word_end( stringbuf_t* sbuf, ssize_t pos ) { + return str_find_ws_word_end( sbuf->buf, sbuf->count, pos); +} + +// find row/col position +ic_private ssize_t sbuf_get_pos_at_rc( stringbuf_t* sbuf, ssize_t termw, ssize_t promptw, ssize_t cpromptw, ssize_t row, ssize_t col ) { + return str_get_pos_at_rc( sbuf->buf, sbuf->count, termw, promptw, cpromptw, row, col); +} + +// get row/col for a given position +ic_private ssize_t sbuf_get_rc_at_pos( stringbuf_t* sbuf, ssize_t termw, ssize_t promptw, ssize_t cpromptw, ssize_t pos, rowcol_t* rc ) { + return str_get_rc_at_pos( sbuf->buf, sbuf->count, termw, promptw, cpromptw, pos, rc); +} + +ic_private ssize_t sbuf_get_wrapped_rc_at_pos( stringbuf_t* sbuf, ssize_t termw, ssize_t newtermw, ssize_t promptw, ssize_t cpromptw, ssize_t pos, rowcol_t* rc ) { + return str_get_wrapped_rc_at_pos( sbuf->buf, sbuf->count, termw, newtermw, promptw, cpromptw, pos, rc); +} + +ic_private ssize_t sbuf_for_each_row( stringbuf_t* sbuf, ssize_t termw, ssize_t promptw, ssize_t cpromptw, row_fun_t* fun, void* arg, void* res ) { + if (sbuf == NULL) return 0; + return str_for_each_row( sbuf->buf, sbuf->count, termw, promptw, cpromptw, fun, arg, res); +} + + +// Duplicate and decode from utf-8 (for non-utf8 terminals) +ic_private char* sbuf_strdup_from_utf8(stringbuf_t* sbuf) { + ssize_t len = sbuf_len(sbuf); + if (sbuf == NULL || len <= 0) return NULL; + char* s = mem_zalloc_tp_n(sbuf->mem, char, len); + if (s == NULL) return NULL; + ssize_t dest = 0; + for (ssize_t i = 0; i < len; ) { + ssize_t ofs = sbuf_next_ofs(sbuf, i, NULL); + if (ofs <= 0) { + // invalid input + break; + } + else if (ofs == 1) { + // regular character + s[dest++] = sbuf->buf[i]; + } + else if (sbuf->buf[i] == '\x1B') { + // skip escape sequences + } + else { + // decode unicode + ssize_t nread; + unicode_t uchr = unicode_from_qutf8( (const uint8_t*)(sbuf->buf + i), ofs, &nread); + uint8_t c; + if (unicode_is_raw(uchr, &c)) { + // raw byte, output as is (this will take care of locale specific input) + s[dest++] = (char)c; + } + else if (uchr <= 0x7F) { + // allow ascii + s[dest++] = (char)uchr; + } + else { + // skip unknown unicode characters.. + // todo: convert according to locale? + } + } + i += ofs; + } + assert(dest <= len); + s[dest] = 0; + return s; +} + +//------------------------------------------------------------- +// String helpers +//------------------------------------------------------------- + +ic_public long ic_prev_char( const char* s, long pos ) { + ssize_t len = ic_strlen(s); + if (pos < 0 || pos > len) return -1; + ssize_t ofs = str_prev_ofs( s, pos, NULL ); + if (ofs <= 0) return -1; + return (long)(pos - ofs); +} + +ic_public long ic_next_char( const char* s, long pos ) { + ssize_t len = ic_strlen(s); + if (pos < 0 || pos > len) return -1; + ssize_t ofs = str_next_ofs( s, len, pos, NULL ); + if (ofs <= 0) return -1; + return (long)(pos + ofs); +} + + +// parse a decimal (leave pi unchanged on error) +ic_private bool ic_atoz(const char* s, ssize_t* pi) { + return (sscanf(s, "%zd", pi) == 1); +} + +// parse two decimals separated by a semicolon +ic_private bool ic_atoz2(const char* s, ssize_t* pi, ssize_t* pj) { + return (sscanf(s, "%zd;%zd", pi, pj) == 2); +} + +// parse unsigned 32-bit (leave pu unchanged on error) +ic_private bool ic_atou32(const char* s, uint32_t* pu) { + return (sscanf(s, "%" SCNu32, pu) == 1); +} + + +// Convenience: character class for whitespace `[ \t\r\n]`. +ic_public bool ic_char_is_white(const char* s, long len) { + if (s == NULL || len != 1) return false; + const char c = *s; + return (c==' ' || c == '\t' || c == '\n' || c == '\r'); +} + +// Convenience: character class for non-whitespace `[^ \t\r\n]`. +ic_public bool ic_char_is_nonwhite(const char* s, long len) { + return !ic_char_is_white(s, len); +} + +// Convenience: character class for separators `[ \t\r\n,.;:/\\\(\)\{\}\[\]]`. +ic_public bool ic_char_is_separator(const char* s, long len) { + if (s == NULL || len != 1) return false; + const char c = *s; + return (strchr(" \t\r\n,.;:/\\(){}[]", c) != NULL); +} + +// Convenience: character class for non-separators. +ic_public bool ic_char_is_nonseparator(const char* s, long len) { + return !ic_char_is_separator(s, len); +} + + +// Convenience: character class for digits (`[0-9]`). +ic_public bool ic_char_is_digit(const char* s, long len) { + if (s == NULL || len != 1) return false; + const char c = *s; + return (c >= '0' && c <= '9'); +} + +// Convenience: character class for hexadecimal digits (`[A-Fa-f0-9]`). +ic_public bool ic_char_is_hexdigit(const char* s, long len) { + if (s == NULL || len != 1) return false; + const char c = *s; + return ((c >= '0' && c <= '9') || (c >= 'a' && c <= 'f') || (c >= 'A' && c <= 'F')); +} + +// Convenience: character class for letters (`[A-Za-z]` and any unicode > 0x80). +ic_public bool ic_char_is_letter(const char* s, long len) { + if (s == NULL || len <= 0) return false; + const char c = *s; + return ((uint8_t)c >= 0x80 || (c >= 'A' && c <= 'Z') || (c >= 'a' && c <= 'z')); +} + +// Convenience: character class for identifier letters (`[A-Za-z0-9_-]` and any unicode > 0x80). +ic_public bool ic_char_is_idletter(const char* s, long len) { + if (s == NULL || len <= 0) return false; + const char c = *s; + return ((uint8_t)c >= 0x80 || (c >= 'A' && c <= 'Z') || (c >= 'a' && c <= 'z') || (c >= '0' && c <= '9') || (c == '_') || (c == '-')); +} + +// Convenience: character class for filename letters (`[^ \t\r\n`@$><=;|&{(]`). +ic_public bool ic_char_is_filename_letter(const char* s, long len) { + if (s == NULL || len <= 0) return false; + const char c = *s; + return ((uint8_t)c >= 0x80 || (strchr(" \t\r\n`@$><=;|&{}()[]", c) == NULL)); +} + +// Convenience: If this is a token start, returns the length (or <= 0 if not found). +ic_public long ic_is_token(const char* s, long pos, ic_is_char_class_fun_t* is_token_char) { + if (s == NULL || pos < 0 || is_token_char == NULL) return -1; + ssize_t len = ic_strlen(s); + if (pos >= len) return -1; + if (pos > 0 && is_token_char(s + pos -1, 1)) return -1; // token start? + ssize_t i = pos; + while ( i < len ) { + ssize_t next = str_next_ofs(s, len, i, NULL); + if (next <= 0) return -1; + if (!is_token_char(s + i, (long)next)) break; + i += next; + } + return (long)(i - pos); +} + + +static int ic_strncmp(const char* s1, const char* s2, ssize_t n) { + return strncmp(s1, s2, to_size_t(n)); +} + +// Convenience: Does this match the specified token? +// Ensures not to match prefixes or suffixes, and returns the length of the match (in bytes). +// E.g. `ic_match_token("function",0,&ic_char_is_letter,"fun")` returns 0. +ic_public long ic_match_token(const char* s, long pos, ic_is_char_class_fun_t* is_token_char, const char* token) { + long n = ic_is_token(s, pos, is_token_char); + if (n > 0 && token != NULL && n == ic_strlen(token) && ic_strncmp(s + pos, token, n) == 0) { + return n; + } + else { + return 0; + } +} + + +// Convenience: Do any of the specified tokens match? +// Ensures not to match prefixes or suffixes, and returns the length of the match (in bytes). +// Ensures not to match prefixes or suffixes. +// E.g. `ic_match_any_token("function",0,&ic_char_is_letter,{"fun","func",NULL})` returns 0. +ic_public long ic_match_any_token(const char* s, long pos, ic_is_char_class_fun_t* is_token_char, const char** tokens) { + long n = ic_is_token(s, pos, is_token_char); + if (n <= 0 || tokens == NULL) return 0; + for (const char** token = tokens; *token != NULL; token++) { + if (n == ic_strlen(*token) && ic_strncmp(s + pos, *token, n) == 0) { + return n; + } + } + return 0; +} + diff --git a/extern/isocline/src/stringbuf.h b/extern/isocline/src/stringbuf.h new file mode 100644 index 00000000..39b21ea4 --- /dev/null +++ b/extern/isocline/src/stringbuf.h @@ -0,0 +1,121 @@ +/* ---------------------------------------------------------------------------- + Copyright (c) 2021, Daan Leijen + This is free software; you can redistribute it and/or modify it + under the terms of the MIT License. A copy of the license can be + found in the "LICENSE" file at the root of this distribution. +-----------------------------------------------------------------------------*/ +#pragma once +#ifndef IC_STRINGBUF_H +#define IC_STRINGBUF_H + +#include +#include "common.h" + +//------------------------------------------------------------- +// string buffer +// in-place modified buffer with edit operations +// that grows on demand. +//------------------------------------------------------------- + +// abstract string buffer +struct stringbuf_s; +typedef struct stringbuf_s stringbuf_t; + +ic_private stringbuf_t* sbuf_new( alloc_t* mem ); +ic_private void sbuf_free( stringbuf_t* sbuf ); +ic_private char* sbuf_free_dup(stringbuf_t* sbuf); +ic_private ssize_t sbuf_len(const stringbuf_t* s); + +ic_private const char* sbuf_string_at( stringbuf_t* sbuf, ssize_t pos ); +ic_private const char* sbuf_string( stringbuf_t* sbuf ); +ic_private char sbuf_char_at(stringbuf_t* sbuf, ssize_t pos); +ic_private char* sbuf_strdup_at( stringbuf_t* sbuf, ssize_t pos ); +ic_private char* sbuf_strdup( stringbuf_t* sbuf ); +ic_private char* sbuf_strdup_from_utf8(stringbuf_t* sbuf); // decode to locale + + +ic_private ssize_t sbuf_appendf(stringbuf_t* sb, const char* fmt, ...); +ic_private ssize_t sbuf_append_vprintf(stringbuf_t* sb, const char* fmt, va_list args); + +ic_private stringbuf_t* sbuf_split_at( stringbuf_t* sb, ssize_t pos ); + +// primitive edit operations (inserts return the new position) +ic_private void sbuf_clear(stringbuf_t* sbuf); +ic_private void sbuf_replace(stringbuf_t* sbuf, const char* s); +ic_private void sbuf_delete_at(stringbuf_t* sbuf, ssize_t pos, ssize_t count); +ic_private void sbuf_delete_from_to(stringbuf_t* sbuf, ssize_t pos, ssize_t end); +ic_private void sbuf_delete_from(stringbuf_t* sbuf, ssize_t pos ); +ic_private ssize_t sbuf_insert_at_n(stringbuf_t* sbuf, const char* s, ssize_t n, ssize_t pos ); +ic_private ssize_t sbuf_insert_at(stringbuf_t* sbuf, const char* s, ssize_t pos ); +ic_private ssize_t sbuf_insert_char_at(stringbuf_t* sbuf, char c, ssize_t pos ); +ic_private ssize_t sbuf_insert_unicode_at(stringbuf_t* sbuf, unicode_t u, ssize_t pos); +ic_private ssize_t sbuf_append_n(stringbuf_t* sbuf, const char* s, ssize_t n); +ic_private ssize_t sbuf_append(stringbuf_t* sbuf, const char* s); +ic_private ssize_t sbuf_append_char(stringbuf_t* sbuf, char c); + +// high level edit operations (return the new position) +ic_private ssize_t sbuf_next( stringbuf_t* sbuf, ssize_t pos, ssize_t* cwidth ); +ic_private ssize_t sbuf_prev( stringbuf_t* sbuf, ssize_t pos, ssize_t* cwidth ); +ic_private ssize_t sbuf_next_ofs(stringbuf_t* sbuf, ssize_t pos, ssize_t* cwidth); + +ic_private ssize_t sbuf_delete_char_before( stringbuf_t* sbuf, ssize_t pos ); +ic_private void sbuf_delete_char_at( stringbuf_t* sbuf, ssize_t pos ); +ic_private ssize_t sbuf_swap_char( stringbuf_t* sbuf, ssize_t pos ); + +ic_private ssize_t sbuf_find_line_start( stringbuf_t* sbuf, ssize_t pos ); +ic_private ssize_t sbuf_find_line_end( stringbuf_t* sbuf, ssize_t pos ); +ic_private ssize_t sbuf_find_word_start( stringbuf_t* sbuf, ssize_t pos ); +ic_private ssize_t sbuf_find_word_end( stringbuf_t* sbuf, ssize_t pos ); +ic_private ssize_t sbuf_find_ws_word_start( stringbuf_t* sbuf, ssize_t pos ); +ic_private ssize_t sbuf_find_ws_word_end( stringbuf_t* sbuf, ssize_t pos ); + +// parse a decimal +ic_private bool ic_atoz(const char* s, ssize_t* i); +// parse two decimals separated by a semicolon +ic_private bool ic_atoz2(const char* s, ssize_t* i, ssize_t* j); +ic_private bool ic_atou32(const char* s, uint32_t* pu); + +// row/column info +typedef struct rowcol_s { + ssize_t row; + ssize_t col; + ssize_t row_start; + ssize_t row_len; + bool first_on_row; + bool last_on_row; +} rowcol_t; + +// find row/col position +ic_private ssize_t sbuf_get_pos_at_rc( stringbuf_t* sbuf, ssize_t termw, ssize_t promptw, ssize_t cpromptw, + ssize_t row, ssize_t col ); +// get row/col for a given position +ic_private ssize_t sbuf_get_rc_at_pos( stringbuf_t* sbuf, ssize_t termw, ssize_t promptw, ssize_t cpromptw, + ssize_t pos, rowcol_t* rc ); + +ic_private ssize_t sbuf_get_wrapped_rc_at_pos( stringbuf_t* sbuf, ssize_t termw, ssize_t newtermw, ssize_t promptw, ssize_t cpromptw, + ssize_t pos, rowcol_t* rc ); + +// row iteration +typedef bool (row_fun_t)(const char* s, + ssize_t row, ssize_t row_start, ssize_t row_len, + ssize_t startw, // prompt width + bool is_wrap, const void* arg, void* res); + +ic_private ssize_t sbuf_for_each_row( stringbuf_t* sbuf, ssize_t termw, ssize_t promptw, ssize_t cpromptw, + row_fun_t* fun, void* arg, void* res ); + + +//------------------------------------------------------------- +// Strings +//------------------------------------------------------------- + +// skip a single CSI sequence (ESC [ ...) +ic_private bool skip_csi_esc( const char* s, ssize_t len, ssize_t* esclen ); // used in term.c + +ic_private ssize_t str_column_width( const char* s ); +ic_private ssize_t str_prev_ofs( const char* s, ssize_t pos, ssize_t* cwidth ); +ic_private ssize_t str_next_ofs( const char* s, ssize_t len, ssize_t pos, ssize_t* cwidth ); +ic_private ssize_t str_skip_until_fit( const char* s, ssize_t max_width); // tail that fits +ic_private ssize_t str_take_while_fit( const char* s, ssize_t max_width); // prefix that fits + +#endif // IC_STRINGBUF_H diff --git a/extern/isocline/src/term.c b/extern/isocline/src/term.c new file mode 100644 index 00000000..c55d9ae5 --- /dev/null +++ b/extern/isocline/src/term.c @@ -0,0 +1,1124 @@ +/* ---------------------------------------------------------------------------- + Copyright (c) 2021, Daan Leijen + This is free software; you can redistribute it and/or modify it + under the terms of the MIT License. A copy of the license can be + found in the "LICENSE" file at the root of this distribution. +-----------------------------------------------------------------------------*/ +#include +#include +#include +#include // getenv +#include + +#include "common.h" +#include "tty.h" +#include "term.h" +#include "stringbuf.h" // str_next_ofs + +#if defined(_WIN32) +#include +#define STDOUT_FILENO 1 +#else +#include +#include +#include +#if defined(__linux__) +#include +#endif +#endif + +#define IC_CSI "\x1B[" + +// color support; colors are auto mapped smaller palettes if needed. (see `term_color.c`) +typedef enum palette_e { + MONOCHROME, // no color + ANSI8, // only basic 8 ANSI color (ESC[m, idx: 30-37, +10 for background) + ANSI16, // basic + bright ANSI colors (ESC[m, idx: 30-37, 90-97, +10 for background) + ANSI256, // ANSI 256 color palette (ESC[38;5;m, idx: 0-15 standard color, 16-231 6x6x6 rbg colors, 232-255 gray shades) + ANSIRGB // direct rgb colors supported (ESC[38;2;;;m) +} palette_t; + +// The terminal screen +struct term_s { + int fd_out; // output handle + ssize_t width; // screen column width + ssize_t height; // screen row height + ssize_t raw_enabled; // is raw mode active? counted by start/end pairs + bool nocolor; // show colors? + bool silent; // enable beep? + bool is_utf8; // utf-8 output? determined by the tty + attr_t attr; // current text attributes + palette_t palette; // color support + buffer_mode_t bufmode; // buffer mode + stringbuf_t* buf; // buffer for buffered output + tty_t* tty; // used on posix to get the cursor position + alloc_t* mem; // allocator + #ifdef _WIN32 + HANDLE hcon; // output console handler + WORD hcon_default_attr; // default text attributes + WORD hcon_orig_attr; // original text attributes + DWORD hcon_orig_mode; // original console mode + DWORD hcon_mode; // used console mode + UINT hcon_orig_cp; // original console code-page (locale) + COORD hcon_save_cursor; // saved cursor position (for escape sequence emulation) + #endif +}; + +static bool term_write_direct(term_t* term, const char* s, ssize_t n ); +static void term_append_buf(term_t* term, const char* s, ssize_t n); + +//------------------------------------------------------------- +// Colors +//------------------------------------------------------------- + +#include "term_color.c" + +//------------------------------------------------------------- +// Helpers +//------------------------------------------------------------- + +ic_private void term_left(term_t* term, ssize_t n) { + if (n <= 0) return; + term_writef( term, IC_CSI "%zdD", n ); +} + +ic_private void term_right(term_t* term, ssize_t n) { + if (n <= 0) return; + term_writef( term, IC_CSI "%zdC", n ); +} + +ic_private void term_up(term_t* term, ssize_t n) { + if (n <= 0) return; + term_writef( term, IC_CSI "%zdA", n ); +} + +ic_private void term_down(term_t* term, ssize_t n) { + if (n <= 0) return; + term_writef( term, IC_CSI "%zdB", n ); +} + +ic_private void term_clear_line(term_t* term) { + term_write( term, "\r" IC_CSI "K"); +} + +ic_private void term_clear_to_end_of_line(term_t* term) { + term_write(term, IC_CSI "K"); +} + +ic_private void term_start_of_line(term_t* term) { + term_write( term, "\r" ); +} + +ic_private ssize_t term_get_width(term_t* term) { + return term->width; +} + +ic_private ssize_t term_get_height(term_t* term) { + return term->height; +} + +ic_private void term_attr_reset(term_t* term) { + term_write(term, IC_CSI "m" ); +} + +ic_private void term_underline(term_t* term, bool on) { + term_write(term, on ? IC_CSI "4m" : IC_CSI "24m" ); +} + +ic_private void term_reverse(term_t* term, bool on) { + term_write(term, on ? IC_CSI "7m" : IC_CSI "27m"); +} + +ic_private void term_bold(term_t* term, bool on) { + term_write(term, on ? IC_CSI "1m" : IC_CSI "22m" ); +} + +ic_private void term_italic(term_t* term, bool on) { + term_write(term, on ? IC_CSI "3m" : IC_CSI "23m" ); +} + +ic_private void term_writeln(term_t* term, const char* s) { + term_write(term,s); + term_write(term,"\n"); +} + +ic_private void term_write_char(term_t* term, char c) { + char buf[2]; + buf[0] = c; + buf[1] = 0; + term_write_n(term, buf, 1 ); +} + +ic_private attr_t term_get_attr( const term_t* term ) { + return term->attr; +} + +ic_private void term_set_attr( term_t* term, attr_t attr ) { + if (term->nocolor) return; + if (attr.x.color != term->attr.x.color && attr.x.color != IC_COLOR_NONE) { + term_color(term,attr.x.color); + if (term->palette < ANSIRGB && color_is_rgb(attr.x.color)) { + term->attr.x.color = attr.x.color; // actual color may have been approximated but we keep the actual color to avoid updating every time + } + } + if (attr.x.bgcolor != term->attr.x.bgcolor && attr.x.bgcolor != IC_COLOR_NONE) { + term_bgcolor(term,attr.x.bgcolor); + if (term->palette < ANSIRGB && color_is_rgb(attr.x.bgcolor)) { + term->attr.x.bgcolor = attr.x.bgcolor; + } + } + if (attr.x.bold != term->attr.x.bold && attr.x.bold != IC_NONE) { + term_bold(term,attr.x.bold == IC_ON); + } + if (attr.x.underline != term->attr.x.underline && attr.x.underline != IC_NONE) { + term_underline(term,attr.x.underline == IC_ON); + } + if (attr.x.reverse != term->attr.x.reverse && attr.x.reverse != IC_NONE) { + term_reverse(term,attr.x.reverse == IC_ON); + } + if (attr.x.italic != term->attr.x.italic && attr.x.italic != IC_NONE) { + term_italic(term,attr.x.italic == IC_ON); + } + assert(attr.x.color == term->attr.x.color || attr.x.color == IC_COLOR_NONE); + assert(attr.x.bgcolor == term->attr.x.bgcolor || attr.x.bgcolor == IC_COLOR_NONE); + assert(attr.x.bold == term->attr.x.bold || attr.x.bold == IC_NONE); + assert(attr.x.reverse == term->attr.x.reverse || attr.x.reverse == IC_NONE); + assert(attr.x.underline == term->attr.x.underline || attr.x.underline == IC_NONE); + assert(attr.x.italic == term->attr.x.italic || attr.x.italic == IC_NONE); +} + + +/* +ic_private void term_clear_lines_to_end(term_t* term) { + term_write(term, "\r" IC_CSI "J"); +} + +ic_private void term_show_cursor(term_t* term, bool on) { + term_write(term, on ? IC_CSI "?25h" : IC_CSI "?25l"); +} +*/ + +//------------------------------------------------------------- +// Formatted output +//------------------------------------------------------------- + +ic_private void term_writef(term_t* term, const char* fmt, ...) { + va_list ap; + va_start(ap, fmt); + term_vwritef(term,fmt,ap); + va_end(ap); +} + +ic_private void term_vwritef(term_t* term, const char* fmt, va_list args ) { + sbuf_append_vprintf(term->buf, fmt, args); +} + +ic_private void term_write_formatted( term_t* term, const char* s, const attr_t* attrs ) { + term_write_formatted_n( term, s, attrs, ic_strlen(s)); +} + +ic_private void term_write_formatted_n( term_t* term, const char* s, const attr_t* attrs, ssize_t len ) { + if (attrs == NULL) { + // write directly + term_write(term,s); + } + else { + // ensure raw mode from now on + if (term->raw_enabled <= 0) { + term_start_raw(term); + } + // and output with text attributes + const attr_t default_attr = term_get_attr(term); + attr_t attr = attr_none(); + ssize_t i = 0; + ssize_t n = 0; + while( i+n < len && s[i+n] != 0 ) { + if (!attr_is_eq(attr,attrs[i+n])) { + if (n > 0) { + term_write_n( term, s+i, n ); + i += n; + n = 0; + } + attr = attrs[i]; + term_set_attr( term, attr_update_with(default_attr,attr) ); + } + n++; + } + if (n > 0) { + term_write_n( term, s+i, n ); + i += n; + n = 0; + } + assert(s[i] != 0 || i == len); + term_set_attr(term, default_attr); + } +} + +//------------------------------------------------------------- +// Write to the terminal +// The buffered functions are used to reduce cursor flicker +// during refresh +//------------------------------------------------------------- + +ic_private void term_beep(term_t* term) { + if (term->silent) return; + fprintf(stderr,"\x7"); + fflush(stderr); +} + +ic_private void term_write_repeat(term_t* term, const char* s, ssize_t count) { + for (; count > 0; count--) { + term_write(term, s); + } +} + +ic_private void term_write(term_t* term, const char* s) { + if (s == NULL || s[0] == 0) return; + ssize_t n = ic_strlen(s); + term_write_n(term,s,n); +} + +// Primitive terminal write; all writes go through here +ic_private void term_write_n(term_t* term, const char* s, ssize_t n) { + if (s == NULL || n <= 0) return; + // write to buffer to reduce flicker and to process escape sequences (this may flush too) + term_append_buf(term, s, n); +} + + +//------------------------------------------------------------- +// Buffering +//------------------------------------------------------------- + + +ic_private void term_flush(term_t* term) { + if (sbuf_len(term->buf) > 0) { + //term_show_cursor(term,false); + term_write_direct(term, sbuf_string(term->buf), sbuf_len(term->buf)); + //term_show_cursor(term,true); + sbuf_clear(term->buf); + } +} + +ic_private buffer_mode_t term_set_buffer_mode(term_t* term, buffer_mode_t mode) { + buffer_mode_t oldmode = term->bufmode; + if (oldmode != mode) { + if (mode == UNBUFFERED) { + term_flush(term); + } + term->bufmode = mode; + } + return oldmode; +} + +static void term_check_flush(term_t* term, bool contains_nl) { + if (term->bufmode == UNBUFFERED || + sbuf_len(term->buf) > 4000 || + (term->bufmode == LINEBUFFERED && contains_nl)) + { + term_flush(term); + } +} + +//------------------------------------------------------------- +// Init +//------------------------------------------------------------- + +static void term_init_raw(term_t* term); + +ic_private term_t* term_new(alloc_t* mem, tty_t* tty, bool nocolor, bool silent, int fd_out ) +{ + term_t* term = mem_zalloc_tp(mem, term_t); + if (term == NULL) return NULL; + + term->fd_out = (fd_out < 0 ? STDOUT_FILENO : fd_out); + term->nocolor = nocolor || (isatty(term->fd_out) == 0); + term->silent = silent; + term->mem = mem; + term->tty = tty; // can be NULL + term->width = 80; + term->height = 25; + term->is_utf8 = tty_is_utf8(tty); + term->palette = ANSI16; // almost universally supported + term->buf = sbuf_new(mem); + term->bufmode = LINEBUFFERED; + term->attr = attr_default(); + + // respect NO_COLOR + if (getenv("NO_COLOR") != NULL) { + term->nocolor = true; + } + if (!term->nocolor) { + // detect color palette + // COLORTERM takes precedence + const char* colorterm = getenv("COLORTERM"); + const char* eterm = getenv("TERM"); + if (ic_contains(colorterm,"24bit") || ic_contains(colorterm,"truecolor") || ic_contains(colorterm,"direct")) { + term->palette = ANSIRGB; + } + else if (ic_contains(colorterm,"8bit") || ic_contains(colorterm,"256color")) { term->palette = ANSI256; } + else if (ic_contains(colorterm,"4bit") || ic_contains(colorterm,"16color")) { term->palette = ANSI16; } + else if (ic_contains(colorterm,"3bit") || ic_contains(colorterm,"8color")) { term->palette = ANSI8; } + else if (ic_contains(colorterm,"1bit") || ic_contains(colorterm,"nocolor") || ic_contains(colorterm,"monochrome")) { + term->palette = MONOCHROME; + } + // otherwise check for some specific terminals + else if (getenv("WT_SESSION") != NULL) { term->palette = ANSIRGB; } // Windows terminal + else if (getenv("ITERM_SESSION_ID") != NULL) { term->palette = ANSIRGB; } // iTerm2 terminal + else if (getenv("VSCODE_PID") != NULL) { term->palette = ANSIRGB; } // vscode terminal + else { + // and otherwise fall back to checking TERM + if (ic_contains(eterm,"truecolor") || ic_contains(eterm,"direct") || ic_contains(colorterm,"24bit")) { + term->palette = ANSIRGB; + } + else if (ic_contains(eterm,"alacritty") || ic_contains(eterm,"kitty")) { + term->palette = ANSIRGB; + } + else if (ic_contains(eterm,"256color") || ic_contains(eterm,"gnome")) { + term->palette = ANSI256; + } + else if (ic_contains(eterm,"16color")){ term->palette = ANSI16; } + else if (ic_contains(eterm,"8color")) { term->palette = ANSI8; } + else if (ic_contains(eterm,"monochrome") || ic_contains(eterm,"nocolor") || ic_contains(eterm,"dumb")) { + term->palette = MONOCHROME; + } + } + debug_msg("term: color-bits: %d (COLORTERM=%s, TERM=%s)\n", term_get_color_bits(term), colorterm, eterm); + } + + // read COLUMS/LINES from the environment for a better initial guess. + const char* env_columns = getenv("COLUMNS"); + if (env_columns != NULL) { ic_atoz(env_columns, &term->width); } + const char* env_lines = getenv("LINES"); + if (env_lines != NULL) { ic_atoz(env_lines, &term->height); } + + // initialize raw terminal output and terminal dimensions + term_init_raw(term); + term_update_dim(term); + term_attr_reset(term); // ensure we are at default settings + + return term; +} + +ic_private bool term_is_interactive(const term_t* term) { + ic_unused(term); + // check dimensions (0 is used for debuggers) + // if (term->width <= 0) return false; + + // check editing support + const char* eterm = getenv("TERM"); + debug_msg("term: TERM=%s\n", eterm); + if (eterm != NULL && + (strstr("dumb|DUMB|cons25|CONS25|emacs|EMACS",eterm) != NULL)) { + return false; + } + + return true; +} + +ic_private bool term_enable_beep(term_t* term, bool enable) { + bool prev = term->silent; + term->silent = !enable; + return prev; +} + +ic_private bool term_enable_color(term_t* term, bool enable) { + bool prev = !term->nocolor; + term->nocolor = !enable; + return prev; +} + +ic_private void term_free(term_t* term) { + if (term == NULL) return; + term_flush(term); + term_end_raw(term, true); + sbuf_free(term->buf); term->buf = NULL; + mem_free(term->mem, term); +} + +//------------------------------------------------------------- +// For best portability and applications inserting CSI SGR (ESC[ .. m) +// codes themselves in strings, we interpret these at the +// lowest level so we can have a `term_get_attr` function which +// is needed for bracketed styles etc. +//------------------------------------------------------------- + +static void term_append_esc(term_t* term, const char* const s, ssize_t len) { + if (s[1]=='[' && s[len-1] == 'm') { + // it is a CSI SGR sequence: ESC[ ... m + if (term->nocolor) return; // ignore escape sequences if nocolor is set + term->attr = attr_update_with(term->attr, attr_from_esc_sgr(s,len)); + } + // and write out the escape sequence as-is + sbuf_append_n(term->buf, s, len); +} + + +static void term_append_utf8(term_t* term, const char* s, ssize_t len) { + ssize_t nread; + unicode_t uchr = unicode_from_qutf8((const uint8_t*)s, len, &nread); + uint8_t c; + if (unicode_is_raw(uchr, &c)) { + // write bytes as is; this also ensure that on non-utf8 terminals characters between 0x80-0xFF + // go through _as is_ due to the qutf8 encoding. + sbuf_append_char(term->buf,(char)c); + } + else if (!term->is_utf8) { + // on non-utf8 terminals still send utf-8 and hope for the best + // todo: we could try to convert to the locale first? + sbuf_append_n(term->buf, s, len); + // sbuf_appendf(term->buf, "\x1B[%" PRIu32 "u", uchr); // unicode escape code + } + else { + // write utf-8 as is + sbuf_append_n(term->buf, s, len); + } +} + +static void term_append_buf( term_t* term, const char* s, ssize_t len ) { + ssize_t pos = 0; + bool newline = false; + while (pos < len) { + // handle ascii sequences in bulk + ssize_t ascii = 0; + ssize_t next; + while ((next = str_next_ofs(s, len, pos+ascii, NULL)) > 0 && + (uint8_t)s[pos + ascii] > '\x1B' && (uint8_t)s[pos + ascii] <= 0x7F ) + { + ascii += next; + } + if (ascii > 0) { + sbuf_append_n(term->buf, s+pos, ascii); + pos += ascii; + } + if (next <= 0) break; + + const uint8_t c = (uint8_t)s[pos]; + // handle utf8 sequences (for non-utf8 terminals) + if (c >= 0x80) { + term_append_utf8(term, s+pos, next); + } + // handle escape sequence (note: str_next_ofs considers whole CSI escape sequences at a time) + else if (next > 1 && c == '\x1B') { + term_append_esc(term, s+pos, next); + } + else if (c < ' ' && c != 0 && (c < '\x07' || c > '\x0D')) { + // ignore control characters except \a, \b, \t, \n, \r, and form-feed and vertical tab. + } + else { + if (c == '\n') { newline = true; } + sbuf_append_n(term->buf, s+pos, next); + } + pos += next; + } + // possibly flush + term_check_flush(term, newline); +} + +//------------------------------------------------------------- +// Platform dependent: Write directly to the terminal +//------------------------------------------------------------- + +#if !defined(_WIN32) + +// write to the console without further processing +static bool term_write_direct(term_t* term, const char* s, ssize_t n) { + ssize_t count = 0; + while( count < n ) { + ssize_t nwritten = write(term->fd_out, s + count, to_size_t(n - count)); + if (nwritten > 0) { + count += nwritten; + } + else if (errno != EINTR && errno != EAGAIN) { + debug_msg("term: write failed: length %i, errno %i: \"%s\"\n", n, errno, s); + return false; + } + } + return true; +} + +#else + +//---------------------------------------------------------------------------------- +// On windows we use the new virtual terminal processing if it is available (Windows Terminal) +// but fall back to ansi escape emulation on older systems but also for example +// the PS terminal +// +// note: we use row/col as 1-based ANSI escape while windows X/Y coords are 0-based. +//----------------------------------------------------------------------------------- + +#if !defined(ENABLE_VIRTUAL_TERMINAL_PROCESSING) +#define ENABLE_VIRTUAL_TERMINAL_PROCESSING (0) +#endif +#if !defined(ENABLE_LVB_GRID_WORLDWIDE) +#define ENABLE_LVB_GRID_WORLDWIDE (0) +#endif + +// direct write to the console without further processing +static bool term_write_console(term_t* term, const char* s, ssize_t n ) { + DWORD written; + // WriteConsoleA(term->hcon, s, (DWORD)(to_size_t(n)), &written, NULL); + WriteFile(term->hcon, s, (DWORD)(to_size_t(n)), &written, NULL); // so it can be redirected + return (written == (DWORD)(to_size_t(n))); +} + +static bool term_get_cursor_pos( term_t* term, ssize_t* row, ssize_t* col) { + *row = 0; + *col = 0; + CONSOLE_SCREEN_BUFFER_INFO info; + if (!GetConsoleScreenBufferInfo(term->hcon, &info)) return false; + *row = (ssize_t)info.dwCursorPosition.Y + 1; + *col = (ssize_t)info.dwCursorPosition.X + 1; + return true; +} + +static void term_move_cursor_to( term_t* term, ssize_t row, ssize_t col ) { + CONSOLE_SCREEN_BUFFER_INFO info; + if (!GetConsoleScreenBufferInfo( term->hcon, &info )) return; + if (col > info.dwSize.X) col = info.dwSize.X; + if (row > info.dwSize.Y) row = info.dwSize.Y; + if (col <= 0) col = 1; + if (row <= 0) row = 1; + COORD coord; + coord.X = (SHORT)col - 1; + coord.Y = (SHORT)row - 1; + SetConsoleCursorPosition( term->hcon, coord); +} + +static void term_cursor_save(term_t* term) { + memset(&term->hcon_save_cursor, 0, sizeof(term->hcon_save_cursor)); + CONSOLE_SCREEN_BUFFER_INFO info; + if (!GetConsoleScreenBufferInfo(term->hcon, &info)) return; + term->hcon_save_cursor = info.dwCursorPosition; +} + +static void term_cursor_restore(term_t* term) { + if (term->hcon_save_cursor.X == 0) return; + SetConsoleCursorPosition(term->hcon, term->hcon_save_cursor); +} + +static void term_move_cursor( term_t* term, ssize_t drow, ssize_t dcol, ssize_t n ) { + CONSOLE_SCREEN_BUFFER_INFO info; + if (!GetConsoleScreenBufferInfo( term->hcon, &info )) return; + COORD cur = info.dwCursorPosition; + ssize_t col = (ssize_t)cur.X + 1 + n*dcol; + ssize_t row = (ssize_t)cur.Y + 1 + n*drow; + term_move_cursor_to( term, row, col ); +} + +static void term_cursor_visible( term_t* term, bool visible ) { + CONSOLE_CURSOR_INFO info; + if (!GetConsoleCursorInfo(term->hcon,&info)) return; + info.bVisible = visible; + SetConsoleCursorInfo(term->hcon,&info); +} + +static void term_erase_line( term_t* term, ssize_t mode ) { + CONSOLE_SCREEN_BUFFER_INFO info; + if (!GetConsoleScreenBufferInfo( term->hcon, &info )) return; + DWORD written; + COORD start; + ssize_t length; + if (mode == 2) { + // entire line + start.X = 0; + start.Y = info.dwCursorPosition.Y; + length = (ssize_t)info.srWindow.Right + 1; + } + else if (mode == 1) { + // to start of line + start.X = 0; + start.Y = info.dwCursorPosition.Y; + length = info.dwCursorPosition.X; + } + else { + // to end of line + length = (ssize_t)info.srWindow.Right - info.dwCursorPosition.X + 1; + start = info.dwCursorPosition; + } + FillConsoleOutputAttribute( term->hcon, term->hcon_default_attr, (DWORD)length, start, &written ); + FillConsoleOutputCharacterA( term->hcon, ' ', (DWORD)length, start, &written ); +} + +static void term_clear_screen(term_t* term, ssize_t mode) { + CONSOLE_SCREEN_BUFFER_INFO info; + if (!GetConsoleScreenBufferInfo(term->hcon, &info)) return; + COORD start; + start.X = 0; + start.Y = 0; + ssize_t length; + ssize_t width = (ssize_t)info.dwSize.X; + if (mode == 2) { + // entire screen + length = width * info.dwSize.Y; + } + else if (mode == 1) { + // to cursor + length = (width * ((ssize_t)info.dwCursorPosition.Y - 1)) + info.dwCursorPosition.X; + } + else { + // from cursor + start = info.dwCursorPosition; + length = (width * ((ssize_t)info.dwSize.Y - info.dwCursorPosition.Y)) + (width - info.dwCursorPosition.X + 1); + } + DWORD written; + FillConsoleOutputAttribute(term->hcon, term->hcon_default_attr, (DWORD)length, start, &written); + FillConsoleOutputCharacterA(term->hcon, ' ', (DWORD)length, start, &written); +} + +static WORD attr_color[8] = { + 0, // black + FOREGROUND_RED, // maroon + FOREGROUND_GREEN, // green + FOREGROUND_RED | FOREGROUND_GREEN, // orange + FOREGROUND_BLUE, // navy + FOREGROUND_RED | FOREGROUND_BLUE, // purple + FOREGROUND_GREEN | FOREGROUND_BLUE, // teal + FOREGROUND_RED | FOREGROUND_GREEN | FOREGROUND_BLUE, // light gray +}; + +static void term_set_win_attr( term_t* term, attr_t ta ) { + WORD def_attr = term->hcon_default_attr; + CONSOLE_SCREEN_BUFFER_INFO info; + if (!GetConsoleScreenBufferInfo( term->hcon, &info )) return; + WORD cur_attr = info.wAttributes; + WORD attr = cur_attr; + if (ta.x.color != IC_COLOR_NONE) { + if (ta.x.color >= IC_ANSI_BLACK && ta.x.color <= IC_ANSI_SILVER) { + attr = (attr & 0xFFF0) | attr_color[ta.x.color - IC_ANSI_BLACK]; + } + else if (ta.x.color >= IC_ANSI_GRAY && ta.x.color <= IC_ANSI_WHITE) { + attr = (attr & 0xFFF0) | attr_color[ta.x.color - IC_ANSI_GRAY] | FOREGROUND_INTENSITY; + } + else if (ta.x.color == IC_ANSI_DEFAULT) { + attr = (attr & 0xFFF0) | (def_attr & 0x000F); + } + } + if (ta.x.bgcolor != IC_COLOR_NONE) { + if (ta.x.bgcolor >= IC_ANSI_BLACK && ta.x.bgcolor <= IC_ANSI_SILVER) { + attr = (attr & 0xFF0F) | (WORD)(attr_color[ta.x.bgcolor - IC_ANSI_BLACK] << 4); + } + else if (ta.x.bgcolor >= IC_ANSI_GRAY && ta.x.bgcolor <= IC_ANSI_WHITE) { + attr = (attr & 0xFF0F) | (WORD)(attr_color[ta.x.bgcolor - IC_ANSI_GRAY] << 4) | BACKGROUND_INTENSITY; + } + else if (ta.x.bgcolor == IC_ANSI_DEFAULT) { + attr = (attr & 0xFF0F) | (def_attr & 0x00F0); + } + } + if (ta.x.underline != IC_NONE) { + attr = (attr & ~COMMON_LVB_UNDERSCORE) | (ta.x.underline == IC_ON ? COMMON_LVB_UNDERSCORE : 0); + } + if (ta.x.reverse != IC_NONE) { + attr = (attr & ~COMMON_LVB_REVERSE_VIDEO) | (ta.x.reverse == IC_ON ? COMMON_LVB_REVERSE_VIDEO : 0); + } + if (attr != cur_attr) { + SetConsoleTextAttribute(term->hcon, attr); + } +} + +static ssize_t esc_param( const char* s, ssize_t def ) { + if (*s == '?') s++; + ssize_t n = def; + ic_atoz(s, &n); + return n; +} + +static void esc_param2( const char* s, ssize_t* p1, ssize_t* p2, ssize_t def ) { + if (*s == '?') s++; + *p1 = def; + *p2 = def; + ic_atoz2(s, p1, p2); +} + +// Emulate escape sequences on older windows. +static void term_write_esc( term_t* term, const char* s, ssize_t len ) { + ssize_t row; + ssize_t col; + + if (s[1] == '[') { + switch (s[len-1]) { + case 'A': + term_move_cursor(term, -1, 0, esc_param(s+2, 1)); + break; + case 'B': + term_move_cursor(term, 1, 0, esc_param(s+2, 1)); + break; + case 'C': + term_move_cursor(term, 0, 1, esc_param(s+2, 1)); + break; + case 'D': + term_move_cursor(term, 0, -1, esc_param(s+2, 1)); + break; + case 'H': + esc_param2(s+2, &row, &col, 1); + term_move_cursor_to(term, row, col); + break; + case 'K': + term_erase_line(term, esc_param(s+2, 0)); + break; + case 'm': + term_set_win_attr( term, attr_from_esc_sgr(s,len) ); + break; + + // support some less standard escape codes (currently not used by isocline) + case 'E': // line down + term_get_cursor_pos(term, &row, &col); + row += esc_param(s+2, 1); + term_move_cursor_to(term, row, 1); + break; + case 'F': // line up + term_get_cursor_pos(term, &row, &col); + row -= esc_param(s+2, 1); + term_move_cursor_to(term, row, 1); + break; + case 'G': // absolute column + term_get_cursor_pos(term, &row, &col); + col = esc_param(s+2, 1); + term_move_cursor_to(term, row, col); + break; + case 'J': + term_clear_screen(term, esc_param(s+2, 0)); + break; + case 'h': + if (strncmp(s+2, "?25h", 4) == 0) { + term_cursor_visible(term, true); + } + break; + case 'l': + if (strncmp(s+2, "?25l", 4) == 0) { + term_cursor_visible(term, false); + } + break; + case 's': + term_cursor_save(term); + break; + case 'u': + term_cursor_restore(term); + break; + // otherwise ignore + } + } + else if (s[1] == '7') { + term_cursor_save(term); + } + else if (s[1] == '8') { + term_cursor_restore(term); + } + else { + // otherwise ignore + } +} + +static bool term_write_direct(term_t* term, const char* s, ssize_t len ) { + term_cursor_visible(term,false); // reduce flicker + ssize_t pos = 0; + if ((term->hcon_mode & ENABLE_VIRTUAL_TERMINAL_PROCESSING) != 0) { + // use the builtin virtual terminal processing. (enables truecolor for example) + term_write_console(term, s, len); + pos = len; + } + else { + // emulate escape sequences + while( pos < len ) { + // handle non-control in bulk (including utf-8 sequences) + // (We don't need to handle utf-8 separately as we set the codepage to always be in utf-8 mode) + ssize_t nonctrl = 0; + ssize_t next; + while( (next = str_next_ofs( s, len, pos+nonctrl, NULL )) > 0 && + (uint8_t)s[pos + nonctrl] >= ' ' && (uint8_t)s[pos + nonctrl] <= 0x7F) { + nonctrl += next; + } + if (nonctrl > 0) { + term_write_console(term, s+pos, nonctrl); + pos += nonctrl; + } + if (next <= 0) break; + + if ((uint8_t)s[pos] >= 0x80) { + // utf8 is already processed + term_write_console(term, s+pos, next); + } + else if (next > 1 && s[pos] == '\x1B') { + // handle control (note: str_next_ofs considers whole CSI escape sequences at a time) + term_write_esc(term, s+pos, next); + } + else if (next == 1 && (s[pos] == '\r' || s[pos] == '\n' || s[pos] == '\t' || s[pos] == '\b')) { + term_write_console( term, s+pos, next); + } + else { + // ignore + } + pos += next; + } + } + term_cursor_visible(term,true); + assert(pos == len); + return (pos == len); + +} +#endif + + + +//------------------------------------------------------------- +// Update terminal dimensions +//------------------------------------------------------------- + +#if !defined(_WIN32) + +// send escape query that may return a response on the tty +static bool term_esc_query_raw( term_t* term, const char* query, char* buf, ssize_t buflen ) +{ + if (buf==NULL || buflen <= 0 || query[0] == 0) return false; + bool osc = (query[1] == ']'); + if (!term_write_direct(term, query, ic_strlen(query))) return false; + debug_msg("term: read tty query response to: ESC %s\n", query + 1); + return tty_read_esc_response( term->tty, query[1], osc, buf, buflen ); +} + +static bool term_esc_query( term_t* term, const char* query, char* buf, ssize_t buflen ) +{ + if (!tty_start_raw(term->tty)) return false; + bool ok = term_esc_query_raw(term,query,buf,buflen); + tty_end_raw(term->tty); + return ok; +} + +// get the cursor position via an ESC[6n +static bool term_get_cursor_pos( term_t* term, ssize_t* row, ssize_t* col) +{ + // send escape query + char buf[128]; + if (!term_esc_query(term,"\x1B[6n",buf,128)) return false; + if (!ic_atoz2(buf,row,col)) return false; + return true; +} + +static void term_set_cursor_pos( term_t* term, ssize_t row, ssize_t col ) { + term_writef( term, IC_CSI "%zd;%zdH", row, col ); +} + +ic_private bool term_update_dim(term_t* term) { + ssize_t cols = 0; + ssize_t rows = 0; + struct winsize ws; + if (ioctl(term->fd_out, TIOCGWINSZ, &ws) >= 0) { + // ioctl succeeded + cols = ws.ws_col; // debuggers return 0 for the column + rows = ws.ws_row; + } + else { + // determine width by querying the cursor position + debug_msg("term: ioctl term-size failed: %d,%d\n", ws.ws_row, ws.ws_col); + ssize_t col0 = 0; + ssize_t row0 = 0; + if (term_get_cursor_pos(term,&row0,&col0)) { + term_set_cursor_pos(term,999,999); + ssize_t col1 = 0; + ssize_t row1 = 0; + if (term_get_cursor_pos(term,&row1,&col1)) { + cols = col1; + rows = row1; + } + term_set_cursor_pos(term,row0,col0); + } + else { + // cannot query position + // return 0 column + } + } + + // update width and return whether it changed. + bool changed = (term->width != cols || term->height != rows); + debug_msg("terminal dim: %zd,%zd: %s\n", rows, cols, changed ? "changed" : "unchanged"); + if (cols > 0) { + term->width = cols; + term->height = rows; + } + return changed; +} + +#else + +ic_private bool term_update_dim(term_t* term) { + if (term->hcon == 0) { + term->hcon = GetConsoleWindow(); + } + ssize_t rows = 0; + ssize_t cols = 0; + CONSOLE_SCREEN_BUFFER_INFO sbinfo; + if (GetConsoleScreenBufferInfo(term->hcon, &sbinfo)) { + cols = (ssize_t)sbinfo.srWindow.Right - (ssize_t)sbinfo.srWindow.Left + 1; + rows = (ssize_t)sbinfo.srWindow.Bottom - (ssize_t)sbinfo.srWindow.Top + 1; + } + bool changed = (term->width != cols || term->height != rows); + term->width = cols; + term->height = rows; + debug_msg("term: update dim: %zd, %zd\n", term->height, term->width ); + return changed; +} + +#endif + + + +//------------------------------------------------------------- +// Enable/disable terminal raw mode +//------------------------------------------------------------- + +#if !defined(_WIN32) + +// On non-windows, the terminal is set in raw mode by the tty. + +ic_private void term_start_raw(term_t* term) { + term->raw_enabled++; +} + +ic_private void term_end_raw(term_t* term, bool force) { + if (term->raw_enabled <= 0) return; + if (!force) { + term->raw_enabled--; + } + else { + term->raw_enabled = 0; + } +} + +static bool term_esc_query_color_raw(term_t* term, int color_idx, uint32_t* color ) { + char buf[128+1]; + snprintf(buf,128,"\x1B]4;%d;?\x1B\\", color_idx); + if (!term_esc_query_raw( term, buf, buf, 128 )) { + debug_msg("esc query response not received\n"); + return false; + } + if (buf[0] != '4') return false; + const char* rgb = strchr(buf,':'); + if (rgb==NULL) return false; + rgb++; // skip ':' + unsigned int r,g,b; + if (sscanf(rgb,"%x/%x/%x",&r,&g,&b) != 3) return false; + if (rgb[2]!='/') { // 48-bit rgb, hexadecimal round to 24-bit + r = (r+0x7F)/0x100; // note: can "overflow", e.g. 0xFFFF -> 0x100. (and we need `ic_cap8` to convert.) + g = (g+0x7F)/0x100; + b = (b+0x7F)/0x100; + } + *color = (ic_cap8(r)<<16) | (ic_cap8(g)<<8) | ic_cap8(b); + debug_msg("color query: %02x,%02x,%02x: %06x\n", r, g, b, *color); + return true; +} + +// update ansi 16 color palette for better color approximation +static void term_update_ansi16(term_t* term) { + debug_msg("update ansi colors\n"); + #if defined(GIO_CMAP) + // try ioctl first (on Linux) + uint8_t cmap[48]; + memset(cmap,0,48); + if (ioctl(term->fd_out,GIO_CMAP,&cmap) >= 0) { + // success + for(ssize_t i = 0; i < 48; i+=3) { + uint32_t color = ((uint32_t)(cmap[i]) << 16) | ((uint32_t)(cmap[i+1]) << 8) | cmap[i+2]; + debug_msg("term (ioctl) ansi color %d: 0x%06x\n", i, color); + ansi256[i] = color; + } + return; + } + else { + debug_msg("ioctl GIO_CMAP failed: entry 1: 0x%02x%02x%02x\n", cmap[3], cmap[4], cmap[5]); + } + #endif + // this seems to be unreliable on some systems (Ubuntu+Gnome terminal) so only enable when known ok. + #if __APPLE__ + // otherwise use OSC 4 escape sequence query + if (tty_start_raw(term->tty)) { + for(ssize_t i = 0; i < 16; i++) { + uint32_t color; + if (!term_esc_query_color_raw(term, i, &color)) break; + debug_msg("term ansi color %d: 0x%06x\n", i, color); + ansi256[i] = color; + } + tty_end_raw(term->tty); + } + #endif +} + +static void term_init_raw(term_t* term) { + if (term->palette < ANSIRGB) { + term_update_ansi16(term); + } +} + +#else + +ic_private void term_start_raw(term_t* term) { + if (term->raw_enabled++ > 0) return; + CONSOLE_SCREEN_BUFFER_INFO info; + if (GetConsoleScreenBufferInfo(term->hcon, &info)) { + term->hcon_orig_attr = info.wAttributes; + } + term->hcon_orig_cp = GetConsoleOutputCP(); + SetConsoleOutputCP(CP_UTF8); + if (term->hcon_mode == 0) { + // first time initialization + DWORD mode = ENABLE_PROCESSED_OUTPUT | ENABLE_WRAP_AT_EOL_OUTPUT | ENABLE_LVB_GRID_WORLDWIDE; // for \r \n and \b + // use escape sequence handling if available and the terminal supports it (so we can use rgb colors in Windows terminal) + // Unfortunately, in plain powershell, we can successfully enable terminal processing + // but it still fails to render correctly; so we require the palette be large enough (like in Windows Terminal) + if (term->palette >= ANSI256 && SetConsoleMode(term->hcon, mode | ENABLE_VIRTUAL_TERMINAL_PROCESSING)) { + term->hcon_mode = mode | ENABLE_VIRTUAL_TERMINAL_PROCESSING; + debug_msg("term: console mode: virtual terminal processing enabled\n"); + } + // no virtual terminal processing, emulate instead + else if (SetConsoleMode(term->hcon, mode)) { + term->hcon_mode = mode; + term->palette = ANSI16; + } + GetConsoleMode(term->hcon, &mode); + debug_msg("term: console mode: orig: 0x%x, new: 0x%x, current 0x%x\n", term->hcon_orig_mode, term->hcon_mode, mode); + } + else { + SetConsoleMode(term->hcon, term->hcon_mode); + } +} + +ic_private void term_end_raw(term_t* term, bool force) { + if (term->raw_enabled <= 0) return; + if (!force && term->raw_enabled > 1) { + term->raw_enabled--; + } + else { + term->raw_enabled = 0; + SetConsoleMode(term->hcon, term->hcon_orig_mode); + SetConsoleOutputCP(term->hcon_orig_cp); + SetConsoleTextAttribute(term->hcon, term->hcon_orig_attr); + } +} + +static void term_init_raw(term_t* term) { + term->hcon = GetStdHandle(STD_OUTPUT_HANDLE); + GetConsoleMode(term->hcon, &term->hcon_orig_mode); + CONSOLE_SCREEN_BUFFER_INFOEX info; + memset(&info, 0, sizeof(info)); + info.cbSize = sizeof(info); + if (GetConsoleScreenBufferInfoEx(term->hcon, &info)) { + // store default attributes + term->hcon_default_attr = info.wAttributes; + // update our color table with the actual colors used. + for (unsigned i = 0; i < 16; i++) { + COLORREF cr = info.ColorTable[i]; + uint32_t color = (ic_cap8(GetRValue(cr))<<16) | (ic_cap8(GetGValue(cr))<<8) | ic_cap8(GetBValue(cr)); // COLORREF = BGR + // index is also in reverse in the bits 0 and 2 + unsigned j = (i&0x08) | ((i&0x04)>>2) | (i&0x02) | (i&0x01)<<2; + debug_msg("term: ansi color %d is 0x%06x\n", j, color); + ansi256[j] = color; + } + } + else { + DWORD err = GetLastError(); + debug_msg("term: cannot get console screen buffer: %d %x", err, err); + } + term_start_raw(term); // initialize the hcon_mode + term_end_raw(term,false); +} + +#endif diff --git a/extern/isocline/src/term.h b/extern/isocline/src/term.h new file mode 100644 index 00000000..50bfd968 --- /dev/null +++ b/extern/isocline/src/term.h @@ -0,0 +1,85 @@ +/* ---------------------------------------------------------------------------- + Copyright (c) 2021, Daan Leijen + This is free software; you can redistribute it and/or modify it + under the terms of the MIT License. A copy of the license can be + found in the "LICENSE" file at the root of this distribution. +-----------------------------------------------------------------------------*/ +#pragma once +#ifndef IC_TERM_H +#define IC_TERM_H + +#include "common.h" +#include "tty.h" +#include "stringbuf.h" +#include "attr.h" + +struct term_s; +typedef struct term_s term_t; + +typedef enum buffer_mode_e { + UNBUFFERED, + LINEBUFFERED, + BUFFERED, +} buffer_mode_t; + +// Primitives +ic_private term_t* term_new(alloc_t* mem, tty_t* tty, bool nocolor, bool silent, int fd_out); +ic_private void term_free(term_t* term); + +ic_private bool term_is_interactive(const term_t* term); +ic_private void term_start_raw(term_t* term); +ic_private void term_end_raw(term_t* term, bool force); + +ic_private bool term_enable_beep(term_t* term, bool enable); +ic_private bool term_enable_color(term_t* term, bool enable); + +ic_private void term_flush(term_t* term); +ic_private buffer_mode_t term_set_buffer_mode(term_t* term, buffer_mode_t mode); + +ic_private void term_write_n(term_t* term, const char* s, ssize_t n); +ic_private void term_write(term_t* term, const char* s); +ic_private void term_writeln(term_t* term, const char* s); +ic_private void term_write_char(term_t* term, char c); + +ic_private void term_write_repeat(term_t* term, const char* s, ssize_t count ); +ic_private void term_beep(term_t* term); + +ic_private bool term_update_dim(term_t* term); + +ic_private ssize_t term_get_width(term_t* term); +ic_private ssize_t term_get_height(term_t* term); +ic_private int term_get_color_bits(term_t* term); + +// Helpers +ic_private void term_writef(term_t* term, const char* fmt, ...); +ic_private void term_vwritef(term_t* term, const char* fmt, va_list args); + +ic_private void term_left(term_t* term, ssize_t n); +ic_private void term_right(term_t* term, ssize_t n); +ic_private void term_up(term_t* term, ssize_t n); +ic_private void term_down(term_t* term, ssize_t n); +ic_private void term_start_of_line(term_t* term ); +ic_private void term_clear_line(term_t* term); +ic_private void term_clear_to_end_of_line(term_t* term); +// ic_private void term_clear_lines_to_end(term_t* term); + + +ic_private void term_attr_reset(term_t* term); +ic_private void term_underline(term_t* term, bool on); +ic_private void term_reverse(term_t* term, bool on); +ic_private void term_bold(term_t* term, bool on); +ic_private void term_italic(term_t* term, bool on); + +ic_private void term_color(term_t* term, ic_color_t color); +ic_private void term_bgcolor(term_t* term, ic_color_t color); + +// Formatted output + +ic_private attr_t term_get_attr( const term_t* term ); +ic_private void term_set_attr( term_t* term, attr_t attr ); +ic_private void term_write_formatted( term_t* term, const char* s, const attr_t* attrs ); +ic_private void term_write_formatted_n( term_t* term, const char* s, const attr_t* attrs, ssize_t n ); + +ic_private ic_color_t color_from_ansi256(ssize_t i); + +#endif // IC_TERM_H diff --git a/extern/isocline/src/term_color.c b/extern/isocline/src/term_color.c new file mode 100644 index 00000000..98af3cf4 --- /dev/null +++ b/extern/isocline/src/term_color.c @@ -0,0 +1,371 @@ +/* ---------------------------------------------------------------------------- + Copyright (c) 2021, Daan Leijen + This is free software; you can redistribute it and/or modify it + under the terms of the MIT License. A copy of the license can be + found in the "LICENSE" file at the root of this distribution. +-----------------------------------------------------------------------------*/ + +// This file is included in "term.c" + +//------------------------------------------------------------- +// Standard ANSI palette for 256 colors +//------------------------------------------------------------- + +static uint32_t ansi256[256] = { + // not const as on some platforms (e.g. Windows, xterm) we update the first 16 entries with the actual used colors. + // 0, standard ANSI + 0x000000, 0x800000, 0x008000, 0x808000, 0x000080, 0x800080, + 0x008080, 0xc0c0c0, + // 8, bright ANSI + 0x808080, 0xff0000, 0x00ff00, 0xffff00, 0x0000ff, 0xff00ff, + 0x00ffff, 0xffffff, + // 6x6x6 RGB colors + // 16 + 0x000000, 0x00005f, 0x000087, 0x0000af, 0x0000d7, 0x0000ff, + 0x005f00, 0x005f5f, 0x005f87, 0x005faf, 0x005fd7, 0x005fff, + 0x008700, 0x00875f, 0x008787, 0x0087af, 0x0087d7, 0x0087ff, + 0x00af00, 0x00af5f, 0x00af87, 0x00afaf, 0x00afd7, 0x00afff, + 0x00d700, 0x00d75f, 0x00d787, 0x00d7af, 0x00d7d7, 0x00d7ff, + 0x00ff00, 0x00ff5f, 0x00ff87, 0x00ffaf, 0x00ffd7, 0x00ffff, + // 52 + 0x5f0000, 0x5f005f, 0x5f0087, 0x5f00af, 0x5f00d7, 0x5f00ff, + 0x5f5f00, 0x5f5f5f, 0x5f5f87, 0x5f5faf, 0x5f5fd7, 0x5f5fff, + 0x5f8700, 0x5f875f, 0x5f8787, 0x5f87af, 0x5f87d7, 0x5f87ff, + 0x5faf00, 0x5faf5f, 0x5faf87, 0x5fafaf, 0x5fafd7, 0x5fafff, + 0x5fd700, 0x5fd75f, 0x5fd787, 0x5fd7af, 0x5fd7d7, 0x5fd7ff, + 0x5fff00, 0x5fff5f, 0x5fff87, 0x5fffaf, 0x5fffd7, 0x5fffff, + // 88 + 0x870000, 0x87005f, 0x870087, 0x8700af, 0x8700d7, 0x8700ff, + 0x875f00, 0x875f5f, 0x875f87, 0x875faf, 0x875fd7, 0x875fff, + 0x878700, 0x87875f, 0x878787, 0x8787af, 0x8787d7, 0x8787ff, + 0x87af00, 0x87af5f, 0x87af87, 0x87afaf, 0x87afd7, 0x87afff, + 0x87d700, 0x87d75f, 0x87d787, 0x87d7af, 0x87d7d7, 0x87d7ff, + 0x87ff00, 0x87ff5f, 0x87ff87, 0x87ffaf, 0x87ffd7, 0x87ffff, + // 124 + 0xaf0000, 0xaf005f, 0xaf0087, 0xaf00af, 0xaf00d7, 0xaf00ff, + 0xaf5f00, 0xaf5f5f, 0xaf5f87, 0xaf5faf, 0xaf5fd7, 0xaf5fff, + 0xaf8700, 0xaf875f, 0xaf8787, 0xaf87af, 0xaf87d7, 0xaf87ff, + 0xafaf00, 0xafaf5f, 0xafaf87, 0xafafaf, 0xafafd7, 0xafafff, + 0xafd700, 0xafd75f, 0xafd787, 0xafd7af, 0xafd7d7, 0xafd7ff, + 0xafff00, 0xafff5f, 0xafff87, 0xafffaf, 0xafffd7, 0xafffff, + // 160 + 0xd70000, 0xd7005f, 0xd70087, 0xd700af, 0xd700d7, 0xd700ff, + 0xd75f00, 0xd75f5f, 0xd75f87, 0xd75faf, 0xd75fd7, 0xd75fff, + 0xd78700, 0xd7875f, 0xd78787, 0xd787af, 0xd787d7, 0xd787ff, + 0xd7af00, 0xd7af5f, 0xd7af87, 0xd7afaf, 0xd7afd7, 0xd7afff, + 0xd7d700, 0xd7d75f, 0xd7d787, 0xd7d7af, 0xd7d7d7, 0xd7d7ff, + 0xd7ff00, 0xd7ff5f, 0xd7ff87, 0xd7ffaf, 0xd7ffd7, 0xd7ffff, + // 196 + 0xff0000, 0xff005f, 0xff0087, 0xff00af, 0xff00d7, 0xff00ff, + 0xff5f00, 0xff5f5f, 0xff5f87, 0xff5faf, 0xff5fd7, 0xff5fff, + 0xff8700, 0xff875f, 0xff8787, 0xff87af, 0xff87d7, 0xff87ff, + 0xffaf00, 0xffaf5f, 0xffaf87, 0xffafaf, 0xffafd7, 0xffafff, + 0xffd700, 0xffd75f, 0xffd787, 0xffd7af, 0xffd7d7, 0xffd7ff, + 0xffff00, 0xffff5f, 0xffff87, 0xffffaf, 0xffffd7, 0xffffff, + // 232, gray scale + 0x080808, 0x121212, 0x1c1c1c, 0x262626, 0x303030, 0x3a3a3a, + 0x444444, 0x4e4e4e, 0x585858, 0x626262, 0x6c6c6c, 0x767676, + 0x808080, 0x8a8a8a, 0x949494, 0x9e9e9e, 0xa8a8a8, 0xb2b2b2, + 0xbcbcbc, 0xc6c6c6, 0xd0d0d0, 0xdadada, 0xe4e4e4, 0xeeeeee +}; + + +//------------------------------------------------------------- +// Create colors +//------------------------------------------------------------- + +// Create a color from a 24-bit color value. +ic_private ic_color_t ic_rgb(uint32_t hex) { + return (ic_color_t)(0x1000000 | (hex & 0xFFFFFF)); +} + +// Limit an int to values between 0 and 255. +static uint32_t ic_cap8(ssize_t i) { + return (i < 0 ? 0 : (i > 255 ? 255 : (uint32_t)i)); +} + +// Create a color from a 24-bit color value. +ic_private ic_color_t ic_rgbx(ssize_t r, ssize_t g, ssize_t b) { + return ic_rgb( (ic_cap8(r)<<16) | (ic_cap8(g)<<8) | ic_cap8(b) ); +} + + +//------------------------------------------------------------- +// Match an rgb color to a ansi8, ansi16, or ansi256 +//------------------------------------------------------------- + +static bool color_is_rgb( ic_color_t color ) { + return (color >= IC_RGB(0)); // bit 24 is set for rgb colors +} + +static void color_to_rgb(ic_color_t color, int* r, int* g, int* b) { + assert(color_is_rgb(color)); + *r = ((color >> 16) & 0xFF); + *g = ((color >> 8) & 0xFF); + *b = (color & 0xFF); +} + +ic_private ic_color_t color_from_ansi256(ssize_t i) { + if (i >= 0 && i < 8) { + return (IC_ANSI_BLACK + (uint32_t)i); + } + else if (i >= 8 && i < 16) { + return (IC_ANSI_DARKGRAY + (uint32_t)(i - 8)); + } + else if (i >= 16 && i <= 255) { + return ic_rgb( ansi256[i] ); + } + else if (i == 256) { + return IC_ANSI_DEFAULT; + } + else { + return IC_ANSI_DEFAULT; + } +} + +static bool is_grayish(int r, int g, int b) { + return (abs(r-g) <= 4) && (abs((r+g)/2 - b) <= 4); +} + +static bool is_grayish_color( uint32_t rgb ) { + int r, g, b; + color_to_rgb(IC_RGB(rgb),&r,&g,&b); + return is_grayish(r,g,b); +} + +static int_least32_t sqr(int_least32_t x) { + return x*x; +} + +// Approximation to delta-E CIE color distance using much +// simpler calculations. See . +// This is essentialy weighted euclidean distance but the weight distribution +// depends on how big the "red" component of the color is. +// We do not take the square root as we only need to find +// the minimal distance (and multiply by 256 to increase precision). +// Needs at least 28-bit signed integers to avoid overflow. +static int_least32_t rgb_distance_rmean( uint32_t color, int r2, int g2, int b2 ) { + int r1, g1, b1; + color_to_rgb(IC_RGB(color),&r1,&g1,&b1); + int_least32_t rmean = (r1 + r2) / 2; + int_least32_t dr2 = sqr(r1 - r2); + int_least32_t dg2 = sqr(g1 - g2); + int_least32_t db2 = sqr(b1 - b2); + int_least32_t dist = ((512+rmean)*dr2) + 1024*dg2 + ((767-rmean)*db2); + return dist; +} + +// Another approximation to delta-E CIE color distance using +// simpler calculations. Similar to `rmean` but adds an adjustment factor +// based on the "red/blue" difference. +static int_least32_t rgb_distance_rbmean( uint32_t color, int r2, int g2, int b2 ) { + int r1, g1, b1; + color_to_rgb(IC_RGB(color),&r1,&g1,&b1); + int_least32_t rmean = (r1 + r2) / 2; + int_least32_t dr2 = sqr(r1 - r2); + int_least32_t dg2 = sqr(g1 - g2); + int_least32_t db2 = sqr(b1 - b2); + int_least32_t dist = 2*dr2 + 4*dg2 + 3*db2 + ((rmean*(dr2 - db2))/256); + return dist; +} + + +// Maintain a small cache of recently used colors. Should be short enough to be effectively constant time. +// If we ever use a more expensive color distance method, we may increase the size a bit (64?) +// (Initial zero initialized cache is valid.) +#define RGB_CACHE_LEN (16) +typedef struct rgb_cache_s { + int last; + int indices[RGB_CACHE_LEN]; + ic_color_t colors[RGB_CACHE_LEN]; +} rgb_cache_t; + +// remember a color in the LRU cache +void rgb_remember( rgb_cache_t* cache, ic_color_t color, int idx ) { + if (cache == NULL) return; + cache->colors[cache->last] = color; + cache->indices[cache->last] = idx; + cache->last++; + if (cache->last >= RGB_CACHE_LEN) { cache->last = 0; } +} + +// quick lookup in cache; -1 on failure +int rgb_lookup( const rgb_cache_t* cache, ic_color_t color ) { + if (cache != NULL) { + for(int i = 0; i < RGB_CACHE_LEN; i++) { + if (cache->colors[i] == color) return cache->indices[i]; + } + } + return -1; +} + +// return the index of the closest matching color +static int rgb_match( uint32_t* palette, int start, int len, rgb_cache_t* cache, ic_color_t color ) { + assert(color_is_rgb(color)); + // in cache? + int min = rgb_lookup(cache,color); + if (min >= 0) { + return min; + } + // otherwise find closest color match in the palette + int r, g, b; + color_to_rgb(color,&r,&g,&b); + min = start; + int_least32_t mindist = (INT_LEAST32_MAX)/4; + for(int i = start; i < len; i++) { + //int_least32_t dist = rgb_distance_rbmean(palette[i],r,g,b); + int_least32_t dist = rgb_distance_rmean(palette[i],r,g,b); + if (is_grayish_color(palette[i]) != is_grayish(r, g, b)) { + // with few colors, make it less eager to substitute a gray for a non-gray (or the other way around) + if (len <= 16) { + dist *= 4; + } + else { + dist = (dist/4)*5; + } + } + if (dist < mindist) { + min = i; + mindist = dist; + } + } + rgb_remember(cache,color,min); + return min; +} + + +// Match RGB to an index in the ANSI 256 color table +static int rgb_to_ansi256(ic_color_t color) { + static rgb_cache_t ansi256_cache; + int c = rgb_match(ansi256, 16, 256, &ansi256_cache, color); // not the first 16 ANSI colors as those may be different + //debug_msg("term: rgb %x -> ansi 256: %d\n", color, c ); + return c; +} + +// Match RGB to an ANSI 16 color code (30-37, 90-97) +static int color_to_ansi16(ic_color_t color) { + if (!color_is_rgb(color)) { + return (int)color; + } + else { + static rgb_cache_t ansi16_cache; + int c = rgb_match(ansi256, 0, 16, &ansi16_cache, color); + //debug_msg("term: rgb %x -> ansi 16: %d\n", color, c ); + return (c < 8 ? 30 + c : 90 + c - 8); + } +} + +// Match RGB to an ANSI 16 color code (30-37, 90-97) +// but assuming the bright colors are simulated using 'bold'. +static int color_to_ansi8(ic_color_t color) { + if (!color_is_rgb(color)) { + return (int)color; + } + else { + // match to basic 8 colors first + static rgb_cache_t ansi8_cache; + int c = 30 + rgb_match(ansi256, 0, 8, &ansi8_cache, color); + // and then adjust for brightness + int r, g, b; + color_to_rgb(color,&r,&g,&b); + if (r>=196 || g>=196 || b>=196) c += 60; + //debug_msg("term: rgb %x -> ansi 8: %d\n", color, c ); + return c; + } +} + + +//------------------------------------------------------------- +// Emit color escape codes based on the terminal capability +//------------------------------------------------------------- + +static void fmt_color_ansi8( char* buf, ssize_t len, ic_color_t color, bool bg ) { + int c = color_to_ansi8(color) + (bg ? 10 : 0); + if (c >= 90) { + snprintf(buf, to_size_t(len), IC_CSI "1;%dm", c - 60); + } + else { + snprintf(buf, to_size_t(len), IC_CSI "22;%dm", c ); + } +} + +static void fmt_color_ansi16( char* buf, ssize_t len, ic_color_t color, bool bg ) { + snprintf( buf, to_size_t(len), IC_CSI "%dm", color_to_ansi16(color) + (bg ? 10 : 0) ); +} + +static void fmt_color_ansi256( char* buf, ssize_t len, ic_color_t color, bool bg ) { + if (!color_is_rgb(color)) { + fmt_color_ansi16(buf,len,color,bg); + } + else { + snprintf( buf, to_size_t(len), IC_CSI "%d;5;%dm", (bg ? 48 : 38), rgb_to_ansi256(color) ); + } +} + +static void fmt_color_rgb( char* buf, ssize_t len, ic_color_t color, bool bg ) { + if (!color_is_rgb(color)) { + fmt_color_ansi16(buf,len,color,bg); + } + else { + int r,g,b; + color_to_rgb(color, &r,&g,&b); + snprintf( buf, to_size_t(len), IC_CSI "%d;2;%d;%d;%dm", (bg ? 48 : 38), r, g, b ); + } +} + +static void fmt_color_ex(char* buf, ssize_t len, palette_t palette, ic_color_t color, bool bg) { + if (color == IC_COLOR_NONE || palette == MONOCHROME) return; + if (palette == ANSI8) { + fmt_color_ansi8(buf,len,color,bg); + } + else if (!color_is_rgb(color) || palette == ANSI16) { + fmt_color_ansi16(buf,len,color,bg); + } + else if (palette == ANSI256) { + fmt_color_ansi256(buf,len,color,bg); + } + else { + fmt_color_rgb(buf,len,color,bg); + } +} + +static void term_color_ex(term_t* term, ic_color_t color, bool bg) { + char buf[128+1]; + fmt_color_ex(buf,128,term->palette,color,bg); + term_write(term,buf); +} + +//------------------------------------------------------------- +// Main API functions +//------------------------------------------------------------- + +ic_private void term_color(term_t* term, ic_color_t color) { + term_color_ex(term,color,false); +} + +ic_private void term_bgcolor(term_t* term, ic_color_t color) { + term_color_ex(term,color,true); +} + +ic_private void term_append_color(term_t* term, stringbuf_t* sbuf, ic_color_t color) { + char buf[128+1]; + fmt_color_ex(buf,128,term->palette,color,false); + sbuf_append(sbuf,buf); +} + +ic_private void term_append_bgcolor(term_t* term, stringbuf_t* sbuf, ic_color_t color) { + char buf[128+1]; + fmt_color_ex(buf, 128, term->palette, color, true); + sbuf_append(sbuf, buf); +} + +ic_private int term_get_color_bits(term_t* term) { + switch (term->palette) { + case MONOCHROME: return 1; + case ANSI8: return 3; + case ANSI16: return 4; + case ANSI256: return 8; + case ANSIRGB: return 24; + default: return 4; + } +} diff --git a/extern/isocline/src/tty.c b/extern/isocline/src/tty.c new file mode 100644 index 00000000..09f7aedd --- /dev/null +++ b/extern/isocline/src/tty.c @@ -0,0 +1,889 @@ +/* ---------------------------------------------------------------------------- + Copyright (c) 2021, Daan Leijen + This is free software; you can redistribute it and/or modify it + under the terms of the MIT License. A copy of the license can be + found in the "LICENSE" file at the root of this distribution. +-----------------------------------------------------------------------------*/ +#include +#include +#include +#include +#include + +#include "tty.h" + +#if defined(_WIN32) +#include +#include +#define isatty(fd) _isatty(fd) +#define read(fd,s,n) _read(fd,s,n) +#define STDIN_FILENO 0 +#if (_WIN32_WINNT < 0x0600) +WINBASEAPI ULONGLONG WINAPI GetTickCount64(VOID); +#endif +#else +#include +#include +#include +#include +#include +#include +#if !defined(FIONREAD) +#include +#endif +#endif + +#define TTY_PUSH_MAX (32) + +struct tty_s { + int fd_in; // input handle + bool raw_enabled; // is raw mode enabled? + bool is_utf8; // is the input stream in utf-8 mode? + bool has_term_resize_event; // are resize events generated? + bool term_resize_event; // did a term resize happen? + alloc_t* mem; // memory allocator + code_t pushbuf[TTY_PUSH_MAX]; // push back buffer for full key codes + ssize_t push_count; + uint8_t cpushbuf[TTY_PUSH_MAX]; // low level push back buffer for bytes + ssize_t cpush_count; + long esc_initial_timeout; // initial ms wait to see if ESC starts an escape sequence + long esc_timeout; // follow up delay for characters in an escape sequence + #if defined(_WIN32) + HANDLE hcon; // console input handle + DWORD hcon_orig_mode; // original console mode + #else + struct termios orig_ios; // original terminal settings + struct termios raw_ios; // raw terminal settings + #endif +}; + + +//------------------------------------------------------------- +// Forward declarations of platform dependent primitives below +//------------------------------------------------------------- + +ic_private bool tty_readc_noblock(tty_t* tty, uint8_t* c, long timeout_ms); // does not modify `c` when no input (false is returned) + +//------------------------------------------------------------- +// Key code helpers +//------------------------------------------------------------- + +ic_private bool code_is_ascii_char(code_t c, char* chr ) { + if (c >= ' ' && c <= 0x7F) { + if (chr != NULL) *chr = (char)c; + return true; + } + else { + if (chr != NULL) *chr = 0; + return false; + } +} + +ic_private bool code_is_unicode(code_t c, unicode_t* uchr) { + if (c <= KEY_UNICODE_MAX) { + if (uchr != NULL) *uchr = c; + return true; + } + else { + if (uchr != NULL) *uchr = 0; + return false; + } +} + +ic_private bool code_is_virt_key(code_t c ) { + return (KEY_NO_MODS(c) <= 0x20 || KEY_NO_MODS(c) >= KEY_VIRT); +} + + +//------------------------------------------------------------- +// Read a key code +//------------------------------------------------------------- +static code_t modify_code( code_t code ); + +static code_t tty_read_utf8( tty_t* tty, uint8_t c0 ) { + uint8_t buf[5]; + memset(buf, 0, 5); + + // try to read as many bytes as potentially needed + buf[0] = c0; + ssize_t count = 1; + if (c0 > 0x7F) { + if (tty_readc_noblock(tty, buf+count, tty->esc_timeout)) { + count++; + if (c0 > 0xDF) { + if (tty_readc_noblock(tty, buf+count, tty->esc_timeout)) { + count++; + if (c0 > 0xEF) { + if (tty_readc_noblock(tty, buf+count, tty->esc_timeout)) { + count++; + } + } + } + } + } + } + + buf[count] = 0; + debug_msg("tty: read utf8: count: %zd: %02x,%02x,%02x,%02x", count, buf[0], buf[1], buf[2], buf[3]); + + // decode the utf8 to unicode + ssize_t read = 0; + code_t code = key_unicode(unicode_from_qutf8(buf, count, &read)); + + // push back unused bytes (in the case of invalid utf8) + while (count > read) { + count--; + if (count >= 0 && count <= 4) { // to help the static analyzer + tty_cpush_char(tty, buf[count]); + } + } + return code; +} + +// pop a code from the pushback buffer. +static bool tty_code_pop(tty_t* tty, code_t* code); + + +// read a single char/key +ic_private bool tty_read_timeout(tty_t* tty, long timeout_ms, code_t* code) +{ + // is there a push_count back code? + if (tty_code_pop(tty,code)) { + return code; + } + + // read a single char/byte from a character stream + uint8_t c; + if (!tty_readc_noblock(tty, &c, timeout_ms)) return false; + + if (c == KEY_ESC) { + // escape sequence? + *code = tty_read_esc(tty, tty->esc_initial_timeout, tty->esc_timeout); + } + else if (c <= 0x7F) { + // ascii + *code = key_unicode(c); + } + else if (tty->is_utf8) { + // utf8 sequence + *code = tty_read_utf8(tty,c); + } + else { + // c >= 0x80 but tty is not utf8; use raw plane so we can translate it back in the end + *code = key_unicode( unicode_from_raw(c) ); + } + + *code = modify_code(*code); + return true; +} + +// Transform virtual keys to be more portable across platforms +static code_t modify_code( code_t code ) { + code_t key = KEY_NO_MODS(code); + code_t mods = KEY_MODS(code); + debug_msg( "tty: readc %s%s%s 0x%03x ('%c')\n", + mods&KEY_MOD_SHIFT ? "shift+" : "", mods&KEY_MOD_CTRL ? "ctrl+" : "", mods&KEY_MOD_ALT ? "alt+" : "", + key, (key >= ' ' && key <= '~' ? key : ' ')); + + // treat KEY_RUBOUT (0x7F) as KEY_BACKSP + if (key == KEY_RUBOUT) { + code = KEY_BACKSP | mods; + } + // ctrl+'_' is translated to '\x1F' on Linux, translate it back + else if (key == key_char('\x1F') && (mods & KEY_MOD_ALT) == 0) { + key = '_'; + code = WITH_CTRL(key_char('_')); + } + // treat ctrl/shift + enter always as KEY_LINEFEED for portability + else if (key == KEY_ENTER && (mods == KEY_MOD_SHIFT || mods == KEY_MOD_ALT || mods == KEY_MOD_CTRL)) { + code = KEY_LINEFEED; + } + // treat ctrl+tab always as shift+tab for portability + else if (code == WITH_CTRL(KEY_TAB)) { + code = KEY_SHIFT_TAB; + } + // treat ctrl+end/alt+>/alt-down and ctrl+home/alt+') || code == WITH_CTRL(KEY_END)) { + code = KEY_PAGEDOWN; + } + else if (code == WITH_ALT(KEY_UP) || code == WITH_ALT('<') || code == WITH_CTRL(KEY_HOME)) { + code = KEY_PAGEUP; + } + + // treat C0 codes without KEY_MOD_CTRL + if (key < ' ' && (mods&KEY_MOD_CTRL) != 0) { + code &= ~KEY_MOD_CTRL; + } + + return code; +} + + +// read a single char/key +ic_private code_t tty_read(tty_t* tty) +{ + code_t code; + if (!tty_read_timeout(tty, -1, &code)) return KEY_NONE; + return code; +} + +//------------------------------------------------------------- +// Read back an ANSI query response +//------------------------------------------------------------- + +ic_private bool tty_read_esc_response(tty_t* tty, char esc_start, bool final_st, char* buf, ssize_t buflen ) +{ + buf[0] = 0; + ssize_t len = 0; + uint8_t c = 0; + if (!tty_readc_noblock(tty, &c, 2*tty->esc_initial_timeout) || c != '\x1B') { + debug_msg("initial esc response failed: 0x%02x\n", c); + return false; + } + if (!tty_readc_noblock(tty, &c, tty->esc_timeout) || (c != esc_start)) return false; + while( len < buflen ) { + if (!tty_readc_noblock(tty, &c, tty->esc_timeout)) return false; + if (final_st) { + // OSC is terminated by BELL, or ESC \ (ST) (and STX) + if (c=='\x07' || c=='\x02') { + break; + } + else if (c=='\x1B') { + uint8_t c1; + if (!tty_readc_noblock(tty, &c1, tty->esc_timeout)) return false; + if (c1=='\\') break; + tty_cpush_char(tty,c1); + } + } + else { + if (c == '\x02') { // STX + break; + } + else if (!((c >= '0' && c <= '9') || strchr("<=>?;:",c) != NULL)) { + buf[len++] = (char)c; // for non-OSC save the terminating character + break; + } + } + buf[len++] = (char)c; + } + buf[len] = 0; + debug_msg("tty: escape query response: %s\n", buf); + return true; +} + +//------------------------------------------------------------- +// High level code pushback +//------------------------------------------------------------- + +static bool tty_code_pop( tty_t* tty, code_t* code ) { + if (tty->push_count <= 0) return false; + tty->push_count--; + *code = tty->pushbuf[tty->push_count]; + return true; +} + +ic_private void tty_code_pushback( tty_t* tty, code_t c ) { + // note: must be signal safe + if (tty->push_count >= TTY_PUSH_MAX) return; + tty->pushbuf[tty->push_count] = c; + tty->push_count++; +} + + +//------------------------------------------------------------- +// low-level character pushback (for escape sequences and windows) +//------------------------------------------------------------- + +ic_private bool tty_cpop(tty_t* tty, uint8_t* c) { + if (tty->cpush_count <= 0) { // do not modify c on failure (see `tty_decode_unicode`) + return false; + } + else { + tty->cpush_count--; + *c = tty->cpushbuf[tty->cpush_count]; + return true; + } +} + +static void tty_cpush(tty_t* tty, const char* s) { + ssize_t len = ic_strlen(s); + if (tty->push_count + len > TTY_PUSH_MAX) { + debug_msg("tty: cpush buffer full! (pushing %s)\n", s); + assert(false); + return; + } + for (ssize_t i = 0; i < len; i++) { + tty->cpushbuf[tty->cpush_count + i] = (uint8_t)( s[len - i - 1] ); + } + tty->cpush_count += len; + return; +} + +// convenience function for small sequences +static void tty_cpushf(tty_t* tty, const char* fmt, ...) { + va_list args; + va_start(args,fmt); + char buf[TTY_PUSH_MAX+1]; + vsnprintf(buf,TTY_PUSH_MAX,fmt,args); + buf[TTY_PUSH_MAX] = 0; + tty_cpush(tty,buf); + va_end(args); + return; +} + +ic_private void tty_cpush_char(tty_t* tty, uint8_t c) { + uint8_t buf[2]; + buf[0] = c; + buf[1] = 0; + tty_cpush(tty, (const char*)buf); +} + + +//------------------------------------------------------------- +// Push escape codes (used on Windows to insert keys) +//------------------------------------------------------------- + +static unsigned csi_mods(code_t mods) { + unsigned m = 1; + if (mods&KEY_MOD_SHIFT) m += 1; + if (mods&KEY_MOD_ALT) m += 2; + if (mods&KEY_MOD_CTRL) m += 4; + return m; +} + +// Push ESC [ ; ~ +static void tty_cpush_csi_vt( tty_t* tty, code_t mods, uint32_t vtcode ) { + tty_cpushf(tty,"\x1B[%u;%u~", vtcode, csi_mods(mods) ); +} + +// push ESC [ 1 ; +static void tty_cpush_csi_xterm( tty_t* tty, code_t mods, char xcode ) { + tty_cpushf(tty,"\x1B[1;%u%c", csi_mods(mods), xcode ); +} + +// push ESC [ ; u +static void tty_cpush_csi_unicode( tty_t* tty, code_t mods, uint32_t unicode ) { + if ((unicode < 0x80 && mods == 0) || + (mods == KEY_MOD_CTRL && unicode < ' ' && unicode != KEY_TAB && unicode != KEY_ENTER + && unicode != KEY_LINEFEED && unicode != KEY_BACKSP) || + (mods == KEY_MOD_SHIFT && unicode >= ' ' && unicode <= KEY_RUBOUT)) { + tty_cpush_char(tty,(uint8_t)unicode); + } + else { + tty_cpushf(tty,"\x1B[%u;%uu", unicode, csi_mods(mods) ); + } +} + +//------------------------------------------------------------- +// Init +//------------------------------------------------------------- + +static bool tty_init_raw(tty_t* tty); +static void tty_done_raw(tty_t* tty); + +static bool tty_init_utf8(tty_t* tty) { + #ifdef _WIN32 + tty->is_utf8 = true; + #else + const char* loc = setlocale(LC_ALL,""); + tty->is_utf8 = (ic_icontains(loc,"UTF-8") || ic_icontains(loc,"utf8") || ic_stricmp(loc,"C") == 0); + debug_msg("tty: utf8: %s (loc=%s)\n", tty->is_utf8 ? "true" : "false", loc); + #endif + return true; +} + +ic_private tty_t* tty_new(alloc_t* mem, int fd_in) +{ + tty_t* tty = mem_zalloc_tp(mem, tty_t); + tty->mem = mem; + tty->fd_in = (fd_in < 0 ? STDIN_FILENO : fd_in); + #if defined(__APPLE__) + tty->esc_initial_timeout = 200; // apple use ESC+ for alt- + #else + tty->esc_initial_timeout = 100; + #endif + tty->esc_timeout = 10; + if (!(isatty(tty->fd_in) && tty_init_raw(tty) && tty_init_utf8(tty))) { + tty_free(tty); + return NULL; + } + return tty; +} + +ic_private void tty_free(tty_t* tty) { + if (tty==NULL) return; + tty_end_raw(tty); + tty_done_raw(tty); + mem_free(tty->mem,tty); +} + +ic_private bool tty_is_utf8(const tty_t* tty) { + if (tty == NULL) return true; + return (tty->is_utf8); +} + +ic_private bool tty_term_resize_event(tty_t* tty) { + if (tty == NULL) return true; + if (tty->has_term_resize_event) { + if (!tty->term_resize_event) return false; + tty->term_resize_event = false; // reset. + } + return true; // always return true on systems without a resize event (more expensive but still ok) +} + +ic_private void tty_set_esc_delay(tty_t* tty, long initial_delay_ms, long followup_delay_ms) { + tty->esc_initial_timeout = (initial_delay_ms < 0 ? 0 : (initial_delay_ms > 1000 ? 1000 : initial_delay_ms)); + tty->esc_timeout = (followup_delay_ms < 0 ? 0 : (followup_delay_ms > 1000 ? 1000 : followup_delay_ms)); +} + +//------------------------------------------------------------- +// Unix +//------------------------------------------------------------- +#if !defined(_WIN32) + +static bool tty_readc_blocking(tty_t* tty, uint8_t* c) { + if (tty_cpop(tty,c)) return true; + *c = 0; + ssize_t nread = read(tty->fd_in, (char*)c, 1); + if (nread < 0 && errno == EINTR) { + // can happen on SIGWINCH signal for terminal resize + } + return (nread == 1); +} + + +// non blocking read -- with a small timeout used for reading escape sequences. +ic_private bool tty_readc_noblock(tty_t* tty, uint8_t* c, long timeout_ms) +{ + // in our pushback buffer? + if (tty_cpop(tty, c)) return true; + + // blocking read? + if (timeout_ms < 0) { + return tty_readc_blocking(tty,c); + } + + // if supported, peek first if any char is available. + #if defined(FIONREAD) + { int navail = 0; + if (ioctl(0, FIONREAD, &navail) == 0) { + if (navail >= 1) { + return tty_readc_blocking(tty, c); + } + else if (timeout_ms == 0) { + return false; // return early if there is no input available (with a zero timeout) + } + } + } + #endif + + // otherwise block for at most timeout milliseconds + #if defined(FD_SET) + // we can use select to detect when input becomes available + fd_set readset; + struct timeval time; + FD_ZERO(&readset); + FD_SET(tty->fd_in, &readset); + time.tv_sec = (timeout_ms > 0 ? timeout_ms / 1000 : 0); + time.tv_usec = (timeout_ms > 0 ? 1000*(timeout_ms % 1000) : 0); + if (select(tty->fd_in + 1, &readset, NULL, NULL, &time) == 1) { + // input available + return tty_readc_blocking(tty, c); + } + #else + // no select, we cannot timeout; use usleeps :-( + // todo: this seems very rare nowadays; should be even support this? + do { + // peek ahead if possible + #if defined(FIONREAD) + int navail = 0; + if (ioctl(0, FIONREAD, &navail) == 0 && navail >= 1) { + return tty_readc_blocking(tty, c); + } + #elif defined(O_NONBLOCK) + // use a temporary non-blocking read mode + int fstatus = fcntl(tty->fd_in, F_GETFL, 0); + if (fstatus != -1) { + if (fcntl(tty->fd_in, F_SETFL, (fstatus | O_NONBLOCK)) != -1) { + char buf[2] = { 0, 0 }; + ssize_t nread = read(tty->fd_in, buf, 1); + fcntl(tty->fd_in, F_SETFL, fstatus); + if (nread >= 1) { + *c = (uint8_t)buf[0]; + return true; + } + } + } + #else + #error "define an nonblocking read for this platform" + #endif + // and sleep a bit + if (timeout_ms > 0) { + usleep(50*1000L); // sleep at most 0.05s at a time + timeout_ms -= 100; + if (timeout_ms < 0) { timeout_ms = 0; } + } + } + while (timeout_ms > 0); + #endif + return false; +} + +#if defined(TIOCSTI) +ic_private bool tty_async_stop(const tty_t* tty) { + // insert ^C in the input stream + char c = KEY_CTRL_C; + return (ioctl(tty->fd_in, TIOCSTI, &c) >= 0); +} +#else +ic_private bool tty_async_stop(const tty_t* tty) { + return false; +} +#endif + +// We install various signal handlers to restore the terminal settings +// in case of a terminating signal. This is also used to catch terminal window resizes. +// This is not strictly needed so this can be disabled on +// (older) platforms that do not support signal handling well. +#if defined(SIGWINCH) && defined(SA_RESTART) // ensure basic signal functionality is defined + +// store the tty in a global so we access it on unexpected termination +static tty_t* sig_tty; // = NULL + +// Catch all termination signals (and SIGWINCH) +typedef struct signal_handler_s { + int signum; + union { + int _avoid_warning; + struct sigaction previous; + } action; +} signal_handler_t; + +static signal_handler_t sighandlers[] = { + { SIGWINCH, {0} }, + { SIGTERM , {0} }, + { SIGINT , {0} }, + { SIGQUIT , {0} }, + { SIGHUP , {0} }, + { SIGSEGV , {0} }, + { SIGTRAP , {0} }, + { SIGBUS , {0} }, + { SIGTSTP , {0} }, + { SIGTTIN , {0} }, + { SIGTTOU , {0} }, + { 0 , {0} } +}; + +static bool sigaction_is_valid( struct sigaction* sa ) { + return (sa->sa_sigaction != NULL && sa->sa_handler != SIG_DFL && sa->sa_handler != SIG_IGN); +} + +// Generic signal handler +static void sig_handler(int signum, siginfo_t* siginfo, void* uap ) { + if (signum == SIGWINCH) { + if (sig_tty != NULL) { + sig_tty->term_resize_event = true; + } + } + else { + // the rest are termination signals; restore the terminal mode. (`tcsetattr` is signal-safe) + if (sig_tty != NULL && sig_tty->raw_enabled) { + tcsetattr(sig_tty->fd_in, TCSAFLUSH, &sig_tty->orig_ios); + sig_tty->raw_enabled = false; + } + } + // call previous handler + signal_handler_t* sh = sighandlers; + while( sh->signum != 0 && sh->signum != signum) { sh++; } + if (sh->signum == signum) { + if (sigaction_is_valid(&sh->action.previous)) { + (sh->action.previous.sa_sigaction)(signum, siginfo, uap); + } + } +} + +static void signals_install(tty_t* tty) { + sig_tty = tty; + // generic signal handler + struct sigaction handler; + memset(&handler,0,sizeof(handler)); + sigemptyset(&handler.sa_mask); + handler.sa_sigaction = &sig_handler; + handler.sa_flags = SA_RESTART; + // install for all signals + for( signal_handler_t* sh = sighandlers; sh->signum != 0; sh++ ) { + if (sigaction( sh->signum, NULL, &sh->action.previous) == 0) { // get previous + if (sh->action.previous.sa_handler != SIG_IGN) { // if not to be ignored + if (sigaction( sh->signum, &handler, &sh->action.previous ) < 0) { // install our handler + sh->action.previous.sa_sigaction = NULL; // do not restore on error + } + else if (sh->signum == SIGWINCH) { + sig_tty->has_term_resize_event = true; + }; + } + } + } +} + +static void signals_restore(void) { + // restore all signal handlers + for( signal_handler_t* sh = sighandlers; sh->signum != 0; sh++ ) { + if (sigaction_is_valid(&sh->action.previous)) { + sigaction( sh->signum, &sh->action.previous, NULL ); + }; + } + sig_tty = NULL; +} + +#else +static void signals_install(tty_t* tty) { + ic_unused(tty); + // nothing +} +static void signals_restore(void) { + // nothing +} + +#endif + +ic_private bool tty_start_raw(tty_t* tty) { + if (tty == NULL) return false; + if (tty->raw_enabled) return true; + if (tcsetattr(tty->fd_in,TCSAFLUSH,&tty->raw_ios) < 0) return false; + tty->raw_enabled = true; + return true; +} + +ic_private void tty_end_raw(tty_t* tty) { + if (tty == NULL) return; + if (!tty->raw_enabled) return; + tty->cpush_count = 0; + if (tcsetattr(tty->fd_in,TCSAFLUSH,&tty->orig_ios) < 0) return; + tty->raw_enabled = false; +} + +static bool tty_init_raw(tty_t* tty) +{ + // Set input to raw mode. See . + if (tcgetattr(tty->fd_in,&tty->orig_ios) == -1) return false; + tty->raw_ios = tty->orig_ios; + // input: no break signal, no \r to \n, no parity check, no 8-bit to 7-bit, no flow control + tty->raw_ios.c_iflag &= ~(unsigned long)(BRKINT | ICRNL | INPCK | ISTRIP | IXON); + // control: allow 8-bit + tty->raw_ios.c_cflag |= CS8; + // local: no echo, no line-by-line (canonical), no extended input processing, no signals for ^z,^c + tty->raw_ios.c_lflag &= ~(unsigned long)(ECHO | ICANON | IEXTEN | ISIG); + // 1 byte at a time, no delay + tty->raw_ios.c_cc[VTIME] = 0; + tty->raw_ios.c_cc[VMIN] = 1; + + // store in global so our signal handlers can restore the terminal mode + signals_install(tty); + + return true; +} + +static void tty_done_raw(tty_t* tty) { + ic_unused(tty); + signals_restore(); +} + + +#else + +//------------------------------------------------------------- +// Windows +// For best portability we push CSI escape sequences directly +// to the character stream (instead of returning key codes). +//------------------------------------------------------------- + +static void tty_waitc_console(tty_t* tty, long timeout_ms); + +ic_private bool tty_readc_noblock(tty_t* tty, uint8_t* c, long timeout_ms) { // don't modify `c` if there is no input + // in our pushback buffer? + if (tty_cpop(tty, c)) return true; + // any events in the input queue? + tty_waitc_console(tty, timeout_ms); + return tty_cpop(tty, c); +} + +// Read from the console input events and push escape codes into the tty cbuffer. +static void tty_waitc_console(tty_t* tty, long timeout_ms) +{ + // wait for a key down event + INPUT_RECORD inp; + DWORD count; + uint32_t surrogate_hi = 0; + while (true) { + // check if there are events if in non-blocking timeout mode + if (timeout_ms >= 0) { + // first peek ahead + if (!GetNumberOfConsoleInputEvents(tty->hcon, &count)) return; + if (count == 0) { + if (timeout_ms == 0) { + // out of time + return; + } + else { + // wait for input events for at most timeout milli seconds + ULONGLONG start_ms = GetTickCount64(); + DWORD res = WaitForSingleObject(tty->hcon, (DWORD)timeout_ms); + switch (res) { + case WAIT_OBJECT_0: { + // input is available, decrease our timeout + ULONGLONG waited_ms = (GetTickCount64() - start_ms); + timeout_ms -= (long)waited_ms; + if (timeout_ms < 0) { + timeout_ms = 0; + } + break; + } + case WAIT_TIMEOUT: + case WAIT_ABANDONED: + case WAIT_FAILED: + default: + return; + } + } + } + } + + // (blocking) Read from the input + if (!ReadConsoleInputW(tty->hcon, &inp, 1, &count)) return; + if (count != 1) return; + + // resize event? + if (inp.EventType == WINDOW_BUFFER_SIZE_EVENT) { + tty->term_resize_event = true; + continue; + } + + // wait for key down events + if (inp.EventType != KEY_EVENT) continue; + + // the modifier state + DWORD modstate = inp.Event.KeyEvent.dwControlKeyState; + + // we need to handle shift up events separately + if (!inp.Event.KeyEvent.bKeyDown && inp.Event.KeyEvent.wVirtualKeyCode == VK_SHIFT) { + modstate &= (DWORD)~SHIFT_PRESSED; + } + + // ignore AltGr + DWORD altgr = LEFT_CTRL_PRESSED | RIGHT_ALT_PRESSED; + if ((modstate & altgr) == altgr) { modstate &= ~altgr; } + + + // get modifiers + code_t mods = 0; + if ((modstate & ( RIGHT_CTRL_PRESSED | LEFT_CTRL_PRESSED )) != 0) mods |= KEY_MOD_CTRL; + if ((modstate & ( RIGHT_ALT_PRESSED | LEFT_ALT_PRESSED )) != 0) mods |= KEY_MOD_ALT; + if ((modstate & SHIFT_PRESSED) != 0) mods |= KEY_MOD_SHIFT; + + // virtual keys + uint32_t chr = (uint32_t)inp.Event.KeyEvent.uChar.UnicodeChar; + WORD virt = inp.Event.KeyEvent.wVirtualKeyCode; + debug_msg("tty: console %s: %s%s%s virt 0x%04x, chr 0x%04x ('%c')\n", inp.Event.KeyEvent.bKeyDown ? "down" : "up", mods&KEY_MOD_CTRL ? "ctrl-" : "", mods&KEY_MOD_ALT ? "alt-" : "", mods&KEY_MOD_SHIFT ? "shift-" : "", virt, chr, chr); + + // only process keydown events (except for Alt-up which is used for unicode pasting...) + if (!inp.Event.KeyEvent.bKeyDown && virt != VK_MENU) { + continue; + } + + if (chr == 0) { + switch (virt) { + case VK_UP: tty_cpush_csi_xterm(tty, mods, 'A'); return; + case VK_DOWN: tty_cpush_csi_xterm(tty, mods, 'B'); return; + case VK_RIGHT: tty_cpush_csi_xterm(tty, mods, 'C'); return; + case VK_LEFT: tty_cpush_csi_xterm(tty, mods, 'D'); return; + case VK_END: tty_cpush_csi_xterm(tty, mods, 'F'); return; + case VK_HOME: tty_cpush_csi_xterm(tty, mods, 'H'); return; + case VK_DELETE: tty_cpush_csi_vt(tty,mods,3); return; + case VK_PRIOR: tty_cpush_csi_vt(tty,mods,5); return; //page up + case VK_NEXT: tty_cpush_csi_vt(tty,mods,6); return; //page down + case VK_TAB: tty_cpush_csi_unicode(tty,mods,9); return; + case VK_RETURN: tty_cpush_csi_unicode(tty,mods,13); return; + default: { + uint32_t vtcode = 0; + if (virt >= VK_F1 && virt <= VK_F5) { + vtcode = 10 + (virt - VK_F1); + } + else if (virt >= VK_F6 && virt <= VK_F10) { + vtcode = 17 + (virt - VK_F6); + } + else if (virt >= VK_F11 && virt <= VK_F12) { + vtcode = 13 + (virt - VK_F11); + } + if (vtcode > 0) { + tty_cpush_csi_vt(tty,mods,vtcode); + return; + } + } + } + // ignore other control keys (shift etc). + } + // high surrogate pair + else if (chr >= 0xD800 && chr <= 0xDBFF) { + surrogate_hi = (chr - 0xD800); + } + // low surrogate pair + else if (chr >= 0xDC00 && chr <= 0xDFFF) { + chr = ((surrogate_hi << 10) + (chr - 0xDC00) + 0x10000); + tty_cpush_csi_unicode(tty,mods,chr); + surrogate_hi = 0; + return; + } + // regular character + else { + tty_cpush_csi_unicode(tty,mods,chr); + return; + } + } +} + +ic_private bool tty_async_stop(const tty_t* tty) { + // send ^c + INPUT_RECORD events[2]; + memset(events, 0, 2*sizeof(INPUT_RECORD)); + events[0].EventType = KEY_EVENT; + events[0].Event.KeyEvent.bKeyDown = TRUE; + events[0].Event.KeyEvent.uChar.AsciiChar = KEY_CTRL_C; + events[1] = events[0]; + events[1].Event.KeyEvent.bKeyDown = FALSE; + DWORD nwritten = 0; + WriteConsoleInput(tty->hcon, events, 2, &nwritten); + return (nwritten == 2); +} + +ic_private bool tty_start_raw(tty_t* tty) { + if (tty->raw_enabled) return true; + GetConsoleMode(tty->hcon,&tty->hcon_orig_mode); + DWORD mode = ENABLE_QUICK_EDIT_MODE // cut&paste allowed + | ENABLE_WINDOW_INPUT // to catch resize events + // | ENABLE_VIRTUAL_TERMINAL_INPUT + // | ENABLE_PROCESSED_INPUT + ; + SetConsoleMode(tty->hcon, mode ); + tty->raw_enabled = true; + return true; +} + +ic_private void tty_end_raw(tty_t* tty) { + if (!tty->raw_enabled) return; + SetConsoleMode(tty->hcon, tty->hcon_orig_mode ); + tty->raw_enabled = false; +} + +static bool tty_init_raw(tty_t* tty) { + tty->hcon = GetStdHandle( STD_INPUT_HANDLE ); + tty->has_term_resize_event = true; + return true; +} + +static void tty_done_raw(tty_t* tty) { + ic_unused(tty); +} + +#endif + + diff --git a/extern/isocline/src/tty.h b/extern/isocline/src/tty.h new file mode 100644 index 00000000..a0062bf3 --- /dev/null +++ b/extern/isocline/src/tty.h @@ -0,0 +1,160 @@ +/* ---------------------------------------------------------------------------- + Copyright (c) 2021, Daan Leijen + This is free software; you can redistribute it and/or modify it + under the terms of the MIT License. A copy of the license can be + found in the "LICENSE" file at the root of this distribution. +-----------------------------------------------------------------------------*/ +#pragma once +#ifndef IC_TTY_H +#define IC_TTY_H + +#include "common.h" + +//------------------------------------------------------------- +// TTY/Keyboard input +//------------------------------------------------------------- + +// Key code +typedef uint32_t code_t; + +// TTY interface +struct tty_s; +typedef struct tty_s tty_t; + + +ic_private tty_t* tty_new(alloc_t* mem, int fd_in); +ic_private void tty_free(tty_t* tty); + +ic_private bool tty_is_utf8(const tty_t* tty); +ic_private bool tty_start_raw(tty_t* tty); +ic_private void tty_end_raw(tty_t* tty); +ic_private code_t tty_read(tty_t* tty); +ic_private bool tty_read_timeout(tty_t* tty, long timeout_ms, code_t* c ); + +ic_private void tty_code_pushback( tty_t* tty, code_t c ); +ic_private bool code_is_ascii_char(code_t c, char* chr ); +ic_private bool code_is_unicode(code_t c, unicode_t* uchr); +ic_private bool code_is_virt_key(code_t c ); + +ic_private bool tty_term_resize_event(tty_t* tty); // did the terminal resize? +ic_private bool tty_async_stop(const tty_t* tty); // unblock the read asynchronously +ic_private void tty_set_esc_delay(tty_t* tty, long initial_delay_ms, long followup_delay_ms); + +// shared between tty.c and tty_esc.c: low level character push +ic_private void tty_cpush_char(tty_t* tty, uint8_t c); +ic_private bool tty_cpop(tty_t* tty, uint8_t* c); +ic_private bool tty_readc_noblock(tty_t* tty, uint8_t* c, long timeout_ms); +ic_private code_t tty_read_esc(tty_t* tty, long esc_initial_timeout, long esc_timeout); // in tty_esc.c + +// used by term.c to read back ANSI escape responses +ic_private bool tty_read_esc_response(tty_t* tty, char esc_start, bool final_st, char* buf, ssize_t buflen ); + + +//------------------------------------------------------------- +// Key codes: a code_t is 32 bits. +// we use the bottom 24 (nah, 21) bits for unicode (up to x0010FFFF) +// The codes after x01000000 are for virtual keys +// and events use x02000000. +// The top 4 bits are used for modifiers. +//------------------------------------------------------------- + +static inline code_t key_char( char c ) { + // careful about signed character conversion (negative char ~> 0x80 - 0xFF) + return ((uint8_t)c); +} + +static inline code_t key_unicode( unicode_t u ) { + return u; +} + + +#define KEY_MOD_SHIFT (0x10000000U) +#define KEY_MOD_ALT (0x20000000U) +#define KEY_MOD_CTRL (0x40000000U) + +#define KEY_NO_MODS(k) (k & 0x0FFFFFFFU) +#define KEY_MODS(k) (k & 0xF0000000U) + +#define WITH_SHIFT(x) (x | KEY_MOD_SHIFT) +#define WITH_ALT(x) (x | KEY_MOD_ALT) +#define WITH_CTRL(x) (x | KEY_MOD_CTRL) + +#define KEY_NONE (0) +#define KEY_CTRL_A (1) +#define KEY_CTRL_B (2) +#define KEY_CTRL_C (3) +#define KEY_CTRL_D (4) +#define KEY_CTRL_E (5) +#define KEY_CTRL_F (6) +#define KEY_BELL (7) +#define KEY_BACKSP (8) +#define KEY_TAB (9) +#define KEY_LINEFEED (10) // ctrl/shift + enter is considered KEY_LINEFEED +#define KEY_CTRL_K (11) +#define KEY_CTRL_L (12) +#define KEY_ENTER (13) +#define KEY_CTRL_N (14) +#define KEY_CTRL_O (15) +#define KEY_CTRL_P (16) +#define KEY_CTRL_Q (17) +#define KEY_CTRL_R (18) +#define KEY_CTRL_S (19) +#define KEY_CTRL_T (20) +#define KEY_CTRL_U (21) +#define KEY_CTRL_V (22) +#define KEY_CTRL_W (23) +#define KEY_CTRL_X (24) +#define KEY_CTRL_Y (25) +#define KEY_CTRL_Z (26) +#define KEY_ESC (27) +#define KEY_SPACE (32) +#define KEY_RUBOUT (127) // always translated to KEY_BACKSP +#define KEY_UNICODE_MAX (0x0010FFFFU) + + +#define KEY_VIRT (0x01000000U) +#define KEY_UP (KEY_VIRT+0) +#define KEY_DOWN (KEY_VIRT+1) +#define KEY_LEFT (KEY_VIRT+2) +#define KEY_RIGHT (KEY_VIRT+3) +#define KEY_HOME (KEY_VIRT+4) +#define KEY_END (KEY_VIRT+5) +#define KEY_DEL (KEY_VIRT+6) +#define KEY_PAGEUP (KEY_VIRT+7) +#define KEY_PAGEDOWN (KEY_VIRT+8) +#define KEY_INS (KEY_VIRT+9) + +#define KEY_F1 (KEY_VIRT+11) +#define KEY_F2 (KEY_VIRT+12) +#define KEY_F3 (KEY_VIRT+13) +#define KEY_F4 (KEY_VIRT+14) +#define KEY_F5 (KEY_VIRT+15) +#define KEY_F6 (KEY_VIRT+16) +#define KEY_F7 (KEY_VIRT+17) +#define KEY_F8 (KEY_VIRT+18) +#define KEY_F9 (KEY_VIRT+19) +#define KEY_F10 (KEY_VIRT+20) +#define KEY_F11 (KEY_VIRT+21) +#define KEY_F12 (KEY_VIRT+22) +#define KEY_F(n) (KEY_F1 + (n) - 1) + +#define KEY_EVENT_BASE (0x02000000U) +#define KEY_EVENT_RESIZE (KEY_EVENT_BASE+1) +#define KEY_EVENT_AUTOTAB (KEY_EVENT_BASE+2) +#define KEY_EVENT_STOP (KEY_EVENT_BASE+3) + +// Convenience +#define KEY_CTRL_UP (WITH_CTRL(KEY_UP)) +#define KEY_CTRL_DOWN (WITH_CTRL(KEY_DOWN)) +#define KEY_CTRL_LEFT (WITH_CTRL(KEY_LEFT)) +#define KEY_CTRL_RIGHT (WITH_CTRL(KEY_RIGHT)) +#define KEY_CTRL_HOME (WITH_CTRL(KEY_HOME)) +#define KEY_CTRL_END (WITH_CTRL(KEY_END)) +#define KEY_CTRL_DEL (WITH_CTRL(KEY_DEL)) +#define KEY_CTRL_PAGEUP (WITH_CTRL(KEY_PAGEUP)) +#define KEY_CTRL_PAGEDOWN (WITH_CTRL(KEY_PAGEDOWN))) +#define KEY_CTRL_INS (WITH_CTRL(KEY_INS)) + +#define KEY_SHIFT_TAB (WITH_SHIFT(KEY_TAB)) + +#endif // IC_TTY_H diff --git a/extern/isocline/src/tty_esc.c b/extern/isocline/src/tty_esc.c new file mode 100644 index 00000000..0ac8761d --- /dev/null +++ b/extern/isocline/src/tty_esc.c @@ -0,0 +1,401 @@ +/* ---------------------------------------------------------------------------- + Copyright (c) 2021, Daan Leijen + This is free software; you can redistribute it and/or modify it + under the terms of the MIT License. A copy of the license can be + found in the "LICENSE" file at the root of this distribution. +-----------------------------------------------------------------------------*/ +#include +#include "tty.h" + +/*------------------------------------------------------------- +Decoding escape sequences to key codes. +This is a bit tricky there are many variants to encode keys as escape sequences, see for example: +- . +- +- +- +- + +Generally, for our purposes we accept a subset of escape sequences as: + + escseq ::= ESC + | ESC char + | ESC start special? (number (';' modifiers)?)? final + +where: + char ::= [\x00-\xFF] # any character + special ::= [:<=>?] + number ::= [0-9+] + modifiers ::= [1-9] + intermediate ::= [\x20-\x2F] # !"#$%&'()*+,-./ + final ::= [\x40-\x7F] # @A–Z[\]^_`a–z{|}~ + ESC ::= '\x1B' + CSI ::= ESC '[' + SS3 ::= ESC 'O' + +In ECMA48 `special? (number (';' modifiers)?)?` is the more liberal `[\x30-\x3F]*` +but that seems never used for key codes. If the number (vtcode or unicode) or the +modifiers are not given, we assume these are '1'. +We then accept the following key sequences: + + key ::= ESC # lone ESC + | ESC char # Alt+char + | ESC '[' special? vtcode ';' modifiers '~' # vt100 codes + | ESC '[' special? '1' ';' modifiers [A-Z] # xterm codes + | ESC 'O' special? '1' ';' modifiers [A-Za-z] # SS3 codes + | ESC '[' special? unicode ';' modifiers 'u' # direct unicode code + +Moreover, we translate the following special cases that do not fit into the above grammar. +First we translate away special starter sequences: +--------------------------------------------------------------------- + ESC '[' '[' .. ~> ESC '[' .. # Linux sometimes uses extra '[' for CSI + ESC '[' 'O' .. ~> ESC 'O' .. # Linux sometimes uses extra '[' for SS3 + ESC 'o' .. ~> ESC 'O' .. # Eterm: ctrl + SS3 + ESC '?' .. ~> ESC 'O' .. # vt52 treated as SS3 + +And then translate the following special cases into a standard form: +--------------------------------------------------------------------- + ESC '[' .. '@' ~> ESC '[' '3' '~' # Del on Mach + ESC '[' .. '9' ~> ESC '[' '2' '~' # Ins on Mach + ESC .. [^@$] ~> ESC .. '~' # ETerm,xrvt,urxt: ^ = ctrl, $ = shift, @ = alt + ESC '[' [a-d] ~> ESC '[' '1' ';' '2' [A-D] # Eterm shift+ + ESC 'O' [1-9] final ~> ESC 'O' '1' ';' [1-9] final # modifiers as parameter 1 (like on Haiku) + ESC '[' [1-9] [^~u] ~> ESC 'O' '1' ';' [1-9] final # modifiers as parameter 1 + +The modifier keys are encoded as "(modifiers-1) & mask" where the +shift mask is 0x01, alt 0x02 and ctrl 0x04. Therefore: +------------------------------------------------------------ + 1: - 5: ctrl 9: alt (for minicom) + 2: shift 6: shift+ctrl + 3: alt 7: alt+ctrl + 4: shift+alt 8: shift+alt+ctrl + +The different encodings fox vt100, xterm, and SS3 are: + +vt100: ESC [ vtcode ';' modifiers '~' +-------------------------------------- + 1: Home 10-15: F1-F5 + 2: Ins 16 : F5 + 3: Del 17-21: F6-F10 + 4: End 23-26: F11-F14 + 5: PageUp 28 : F15 + 6: PageDn 29 : F16 + 7: Home 31-34: F17-F20 + 8: End + +xterm: ESC [ 1 ';' modifiers [A-Z] +----------------------------------- + A: Up N: F2 + B: Down O: F3 + C: Right P: F4 + D: Left Q: F5 + E: '5' R: F6 + F: End S: F7 + G: T: F8 + H: Home U: PageDn + I: PageUp V: PageUp + J: W: F11 + K: X: F12 + L: Ins Y: End + M: F1 Z: shift+Tab + +SS3: ESC 'O' 1 ';' modifiers [A-Za-z] +--------------------------------------- + (normal) (numpad) + A: Up N: a: Up n: + B: Down O: b: Down o: + C: Right P: F1 c: Right p: Ins + D: Left Q: F2 d: Left q: End + E: '5' R: F3 e: r: Down + F: End S: F4 f: s: PageDn + G: T: F5 g: t: Left + H: Home U: F6 h: u: '5' + I: Tab V: F7 i: v: Right + J: W: F8 j: '*' w: Home + K: X: F9 k: '+' x: Up + L: Y: F10 l: ',' y: PageUp + M: \x0A '\n' Z: shift+Tab m: '-' z: + +-------------------------------------------------------------*/ + +//------------------------------------------------------------- +// Decode escape sequences +//------------------------------------------------------------- + +static code_t esc_decode_vt(uint32_t vt_code ) { + switch(vt_code) { + case 1: return KEY_HOME; + case 2: return KEY_INS; + case 3: return KEY_DEL; + case 4: return KEY_END; + case 5: return KEY_PAGEUP; + case 6: return KEY_PAGEDOWN; + case 7: return KEY_HOME; + case 8: return KEY_END; + default: + if (vt_code >= 10 && vt_code <= 15) return KEY_F(1 + (vt_code - 10)); + if (vt_code == 16) return KEY_F5; // minicom + if (vt_code >= 17 && vt_code <= 21) return KEY_F(6 + (vt_code - 17)); + if (vt_code >= 23 && vt_code <= 26) return KEY_F(11 + (vt_code - 23)); + if (vt_code >= 28 && vt_code <= 29) return KEY_F(15 + (vt_code - 28)); + if (vt_code >= 31 && vt_code <= 34) return KEY_F(17 + (vt_code - 31)); + } + return KEY_NONE; +} + +static code_t esc_decode_xterm( uint8_t xcode ) { + // ESC [ + switch(xcode) { + case 'A': return KEY_UP; + case 'B': return KEY_DOWN; + case 'C': return KEY_RIGHT; + case 'D': return KEY_LEFT; + case 'E': return '5'; // numpad 5 + case 'F': return KEY_END; + case 'H': return KEY_HOME; + case 'Z': return KEY_TAB | KEY_MOD_SHIFT; + // Freebsd: + case 'I': return KEY_PAGEUP; + case 'L': return KEY_INS; + case 'M': return KEY_F1; + case 'N': return KEY_F2; + case 'O': return KEY_F3; + case 'P': return KEY_F4; // note: differs from + case 'Q': return KEY_F5; + case 'R': return KEY_F6; + case 'S': return KEY_F7; + case 'T': return KEY_F8; + case 'U': return KEY_PAGEDOWN; // Mach + case 'V': return KEY_PAGEUP; // Mach + case 'W': return KEY_F11; + case 'X': return KEY_F12; + case 'Y': return KEY_END; // Mach + } + return KEY_NONE; +} + +static code_t esc_decode_ss3( uint8_t ss3_code ) { + // ESC O + switch(ss3_code) { + case 'A': return KEY_UP; + case 'B': return KEY_DOWN; + case 'C': return KEY_RIGHT; + case 'D': return KEY_LEFT; + case 'E': return '5'; // numpad 5 + case 'F': return KEY_END; + case 'H': return KEY_HOME; + case 'I': return KEY_TAB; + case 'Z': return KEY_TAB | KEY_MOD_SHIFT; + case 'M': return KEY_LINEFEED; + case 'P': return KEY_F1; + case 'Q': return KEY_F2; + case 'R': return KEY_F3; + case 'S': return KEY_F4; + // on Mach + case 'T': return KEY_F5; + case 'U': return KEY_F6; + case 'V': return KEY_F7; + case 'W': return KEY_F8; + case 'X': return KEY_F9; // '=' on vt220 + case 'Y': return KEY_F10; + // numpad + case 'a': return KEY_UP; + case 'b': return KEY_DOWN; + case 'c': return KEY_RIGHT; + case 'd': return KEY_LEFT; + case 'j': return '*'; + case 'k': return '+'; + case 'l': return ','; + case 'm': return '-'; + case 'n': return KEY_DEL; // '.' + case 'o': return '/'; + case 'p': return KEY_INS; + case 'q': return KEY_END; + case 'r': return KEY_DOWN; + case 's': return KEY_PAGEDOWN; + case 't': return KEY_LEFT; + case 'u': return '5'; + case 'v': return KEY_RIGHT; + case 'w': return KEY_HOME; + case 'x': return KEY_UP; + case 'y': return KEY_PAGEUP; + } + return KEY_NONE; +} + +static void tty_read_csi_num(tty_t* tty, uint8_t* ppeek, uint32_t* num, long esc_timeout) { + *num = 1; // default + ssize_t count = 0; + uint32_t i = 0; + while (*ppeek >= '0' && *ppeek <= '9' && count < 16) { + uint8_t digit = *ppeek - '0'; + if (!tty_readc_noblock(tty,ppeek,esc_timeout)) break; // peek is not modified in this case + count++; + i = 10*i + digit; + } + if (count > 0) *num = i; +} + +static code_t tty_read_csi(tty_t* tty, uint8_t c1, uint8_t peek, code_t mods0, long esc_timeout) { + // CSI starts with 0x9b (c1=='[') | ESC [ (c1=='[') | ESC [Oo?] (c1 == 'O') /* = SS3 */ + + // check for extra starter '[' (Linux sends ESC [ [ 15 ~ for F5 for example) + if (c1 == '[' && strchr("[Oo", (char)peek) != NULL) { + uint8_t cx = peek; + if (tty_readc_noblock(tty,&peek,esc_timeout)) { + c1 = cx; + } + } + + // "special" characters ('?' is used for private sequences) + uint8_t special = 0; + if (strchr(":<=>?",(char)peek) != NULL) { + special = peek; + if (!tty_readc_noblock(tty,&peek,esc_timeout)) { + tty_cpush_char(tty,special); // recover + return (key_unicode(c1) | KEY_MOD_ALT); // Alt+ + } + } + + // up to 2 parameters that default to 1 + uint32_t num1 = 1; + uint32_t num2 = 1; + tty_read_csi_num(tty,&peek,&num1,esc_timeout); + if (peek == ';') { + if (!tty_readc_noblock(tty,&peek,esc_timeout)) return KEY_NONE; + tty_read_csi_num(tty,&peek,&num2,esc_timeout); + } + + // the final character (we do not allow 'intermediate characters') + uint8_t final = peek; + code_t modifiers = mods0; + + debug_msg("tty: escape sequence: ESC %c %c %d;%d %c\n", c1, (special == 0 ? '_' : special), num1, num2, final); + + // Adjust special cases into standard ones. + if ((final == '@' || final == '9') && c1 == '[' && num1 == 1) { + // ESC [ @, ESC [ 9 : on Mach + if (final == '@') num1 = 3; // DEL + else if (final == '9') num1 = 2; // INS + final = '~'; + } + else if (final == '^' || final == '$' || final == '@') { + // Eterm/rxvt/urxt + if (final=='^') modifiers |= KEY_MOD_CTRL; + if (final=='$') modifiers |= KEY_MOD_SHIFT; + if (final=='@') modifiers |= KEY_MOD_SHIFT | KEY_MOD_CTRL; + final = '~'; + } + else if (c1 == '[' && final >= 'a' && final <= 'd') { // note: do not catch ESC [ .. u (for unicode) + // ESC [ [a-d] : on Eterm for shift+ cursor + modifiers |= KEY_MOD_SHIFT; + final = 'A' + (final - 'a'); + } + + if (((c1 == 'O') || (c1=='[' && final != '~' && final != 'u')) && + (num2 == 1 && num1 > 1 && num1 <= 8)) + { + // on haiku the modifier can be parameter 1, make it parameter 2 instead + num2 = num1; + num1 = 1; + } + + // parameter 2 determines the modifiers + if (num2 > 1 && num2 <= 9) { + if (num2 == 9) num2 = 3; // iTerm2 in xterm mode + num2--; + if (num2 & 0x1) modifiers |= KEY_MOD_SHIFT; + if (num2 & 0x2) modifiers |= KEY_MOD_ALT; + if (num2 & 0x4) modifiers |= KEY_MOD_CTRL; + } + + // and translate + code_t code = KEY_NONE; + if (final == '~') { + // vt codes + code = esc_decode_vt(num1); + } + else if (c1 == '[' && final == 'u') { + // unicode + code = key_unicode(num1); + } + else if (c1 == 'O' && ((final >= 'A' && final <= 'Z') || (final >= 'a' && final <= 'z'))) { + // ss3 + code = esc_decode_ss3(final); + } + else if (num1 == 1 && final >= 'A' && final <= 'Z') { + // xterm + code = esc_decode_xterm(final); + } + else if (c1 == '[' && final == 'R') { + // cursor position + code = KEY_NONE; + } + + if (code == KEY_NONE && final != 'R') { + debug_msg("tty: ignore escape sequence: ESC %c %zu;%zu %c\n", c1, num1, num2, final); + } + return (code != KEY_NONE ? (code | modifiers) : KEY_NONE); +} + +static code_t tty_read_osc( tty_t* tty, uint8_t* ppeek, long esc_timeout ) { + debug_msg("discard OSC response..\n"); + // keep reading until termination: OSC is terminated by BELL, or ESC \ (ST) (and STX) + while (true) { + uint8_t c = *ppeek; + if (c <= '\x07') { // BELL and anything below (STX, ^C, ^D) + if (c != '\x07') { tty_cpush_char( tty, c ); } + break; + } + else if (c=='\x1B') { + uint8_t c1; + if (!tty_readc_noblock(tty, &c1, esc_timeout)) break; + if (c1=='\\') break; + tty_cpush_char(tty,c1); + } + if (!tty_readc_noblock(tty, ppeek, esc_timeout)) break; + } + return KEY_NONE; +} + +ic_private code_t tty_read_esc(tty_t* tty, long esc_initial_timeout, long esc_timeout) { + code_t mods = 0; + uint8_t peek = 0; + + // lone ESC? + if (!tty_readc_noblock(tty, &peek, esc_initial_timeout)) return KEY_ESC; + + // treat ESC ESC as Alt modifier (macOS sends ESC ESC [ [A-D] for alt-) + if (peek == KEY_ESC) { + if (!tty_readc_noblock(tty, &peek, esc_timeout)) goto alt; + mods |= KEY_MOD_ALT; + } + + // CSI ? + if (peek == '[') { + if (!tty_readc_noblock(tty, &peek, esc_timeout)) goto alt; + return tty_read_csi(tty, '[', peek, mods, esc_timeout); // ESC [ ... + } + + // SS3? + if (peek == 'O' || peek == 'o' || peek == '?' /*vt52*/) { + uint8_t c1 = peek; + if (!tty_readc_noblock(tty, &peek, esc_timeout)) goto alt; + if (c1 == 'o') { + // ETerm uses this for ctrl+ + mods |= KEY_MOD_CTRL; + } + // treat all as standard SS3 'O' + return tty_read_csi(tty,'O',peek,mods, esc_timeout); // ESC [Oo?] ... + } + + // OSC: we may get a delayed query response; ensure it is ignored + if (peek == ']') { + if (!tty_readc_noblock(tty, &peek, esc_timeout)) goto alt; + return tty_read_osc(tty, &peek, esc_timeout); // ESC ] ... + } + +alt: + // Alt+ + return (key_unicode(peek) | KEY_MOD_ALT); // ESC +} diff --git a/extern/isocline/src/undo.c b/extern/isocline/src/undo.c new file mode 100644 index 00000000..eefc318d --- /dev/null +++ b/extern/isocline/src/undo.c @@ -0,0 +1,67 @@ +/* ---------------------------------------------------------------------------- + Copyright (c) 2021, Daan Leijen + This is free software; you can redistribute it and/or modify it + under the terms of the MIT License. A copy of the license can be + found in the "LICENSE" file at the root of this distribution. +-----------------------------------------------------------------------------*/ +#include +#include + +#include "../include/isocline.h" +#include "common.h" +#include "env.h" +#include "stringbuf.h" +#include "completions.h" +#include "undo.h" + + + +//------------------------------------------------------------- +// edit state +//------------------------------------------------------------- +struct editstate_s { + struct editstate_s* next; + const char* input; // input + ssize_t pos; // cursor position +}; + +ic_private void editstate_init( editstate_t** es ) { + *es = NULL; +} + +ic_private void editstate_done( alloc_t* mem, editstate_t** es ) { + while (*es != NULL) { + editstate_t* next = (*es)->next; + mem_free(mem, (*es)->input); + mem_free(mem, *es ); + *es = next; + } + *es = NULL; +} + +ic_private void editstate_capture( alloc_t* mem, editstate_t** es, const char* input, ssize_t pos) { + if (input==NULL) input = ""; + // alloc + editstate_t* entry = mem_zalloc_tp(mem, editstate_t); + if (entry == NULL) return; + // initialize + entry->input = mem_strdup( mem, input); + entry->pos = pos; + if (entry->input == NULL) { mem_free(mem, entry); return; } + // and push + entry->next = *es; + *es = entry; +} + +// caller should free *input +ic_private bool editstate_restore( alloc_t* mem, editstate_t** es, const char** input, ssize_t* pos ) { + if (*es == NULL) return false; + // pop + editstate_t* entry = *es; + *es = entry->next; + *input = entry->input; + *pos = entry->pos; + mem_free(mem, entry); + return true; +} + diff --git a/extern/isocline/src/undo.h b/extern/isocline/src/undo.h new file mode 100644 index 00000000..576cf977 --- /dev/null +++ b/extern/isocline/src/undo.h @@ -0,0 +1,24 @@ +/* ---------------------------------------------------------------------------- + Copyright (c) 2021, Daan Leijen + This is free software; you can redistribute it and/or modify it + under the terms of the MIT License. A copy of the license can be + found in the "LICENSE" file at the root of this distribution. +-----------------------------------------------------------------------------*/ +#pragma once +#ifndef IC_UNDO_H +#define IC_UNDO_H + +#include "common.h" + +//------------------------------------------------------------- +// Edit state +//------------------------------------------------------------- +struct editstate_s; +typedef struct editstate_s editstate_t; + +ic_private void editstate_init( editstate_t** es ); +ic_private void editstate_done( alloc_t* mem, editstate_t** es ); +ic_private void editstate_capture( alloc_t* mem, editstate_t** es, const char* input, ssize_t pos); +ic_private bool editstate_restore( alloc_t* mem, editstate_t** es, const char** input, ssize_t* pos ); // caller needs to free input + +#endif // IC_UNDO_H diff --git a/extern/isocline/src/wcwidth.c b/extern/isocline/src/wcwidth.c new file mode 100644 index 00000000..85187d41 --- /dev/null +++ b/extern/isocline/src/wcwidth.c @@ -0,0 +1,292 @@ +// include in "stringbuf.c" +/* + * This is an implementation of wcwidth() and wcswidth() (defined in + * IEEE Std 1002.1-2001) for Unicode. + * + * http://www.opengroup.org/onlinepubs/007904975/functions/wcwidth.html + * http://www.opengroup.org/onlinepubs/007904975/functions/wcswidth.html + * + * In fixed-width output devices, Latin characters all occupy a single + * "cell" position of equal width, whereas ideographic CJK characters + * occupy two such cells. Interoperability between terminal-line + * applications and (teletype-style) character terminals using the + * UTF-8 encoding requires agreement on which character should advance + * the cursor by how many cell positions. No established formal + * standards exist at present on which Unicode character shall occupy + * how many cell positions on character terminals. These routines are + * a first attempt of defining such behavior based on simple rules + * applied to data provided by the Unicode Consortium. + * + * For some graphical characters, the Unicode standard explicitly + * defines a character-cell width via the definition of the East Asian + * FullWidth (F), Wide (W), Half-width (H), and Narrow (Na) classes. + * In all these cases, there is no ambiguity about which width a + * terminal shall use. For characters in the East Asian Ambiguous (A) + * class, the width choice depends purely on a preference of backward + * compatibility with either historic CJK or Western practice. + * Choosing single-width for these characters is easy to justify as + * the appropriate long-term solution, as the CJK practice of + * displaying these characters as double-width comes from historic + * implementation simplicity (8-bit encoded characters were displayed + * single-width and 16-bit ones double-width, even for Greek, + * Cyrillic, etc.) and not any typographic considerations. + * + * Much less clear is the choice of width for the Not East Asian + * (Neutral) class. Existing practice does not dictate a width for any + * of these characters. It would nevertheless make sense + * typographically to allocate two character cells to characters such + * as for instance EM SPACE or VOLUME INTEGRAL, which cannot be + * represented adequately with a single-width glyph. The following + * routines at present merely assign a single-cell width to all + * neutral characters, in the interest of simplicity. This is not + * entirely satisfactory and should be reconsidered before + * establishing a formal standard in this area. At the moment, the + * decision which Not East Asian (Neutral) characters should be + * represented by double-width glyphs cannot yet be answered by + * applying a simple rule from the Unicode database content. Setting + * up a proper standard for the behavior of UTF-8 character terminals + * will require a careful analysis not only of each Unicode character, + * but also of each presentation form, something the author of these + * routines has avoided to do so far. + * + * http://www.unicode.org/unicode/reports/tr11/ + * + * Markus Kuhn -- 2007-05-26 (Unicode 5.0) + * + * Permission to use, copy, modify, and distribute this software + * for any purpose and without fee is hereby granted. The author + * disclaims all warranties with regard to this software. + * + * Latest version: http://www.cl.cam.ac.uk/~mgk25/ucs/wcwidth.c + */ + +#include +#include + +struct interval { + int32_t first; + int32_t last; +}; + +/* auxiliary function for binary search in interval table */ +static int bisearch(int32_t ucs, const struct interval *table, int max) { + int min = 0; + int mid; + + if (ucs < table[0].first || ucs > table[max].last) + return 0; + while (max >= min) { + mid = (min + max) / 2; + if (ucs > table[mid].last) + min = mid + 1; + else if (ucs < table[mid].first) + max = mid - 1; + else + return 1; + } + + return 0; +} + + +/* The following two functions define the column width of an ISO 10646 + * character as follows: + * + * - The null character (U+0000) has a column width of 0. + * + * - Other C0/C1 control characters and DEL will lead to a return + * value of -1. + * + * - Non-spacing and enclosing combining characters (general + * category code Mn or Me in the Unicode database) have a + * column width of 0. + * + * - SOFT HYPHEN (U+00AD) has a column width of 1. + * + * - Other format characters (general category code Cf in the Unicode + * database) and ZERO WIDTH SPACE (U+200B) have a column width of 0. + * + * - Hangul Jamo medial vowels and final consonants (U+1160-U+11FF) + * have a column width of 0. + * + * - Spacing characters in the East Asian Wide (W) or East Asian + * Full-width (F) category as defined in Unicode Technical + * Report #11 have a column width of 2. + * + * - All remaining characters (including all printable + * ISO 8859-1 and WGL4 characters, Unicode control characters, + * etc.) have a column width of 1. + * + * This implementation assumes that wchar_t characters are encoded + * in ISO 10646. + */ + +static int mk_is_wide_char(int32_t ucs) { + static const struct interval wide[] = { + {0x1100, 0x115f}, {0x231a, 0x231b}, {0x2329, 0x232a}, + {0x23e9, 0x23ec}, {0x23f0, 0x23f0}, {0x23f3, 0x23f3}, + {0x25fd, 0x25fe}, {0x2614, 0x2615}, {0x2648, 0x2653}, + {0x267f, 0x267f}, {0x2693, 0x2693}, {0x26a1, 0x26a1}, + {0x26aa, 0x26ab}, {0x26bd, 0x26be}, {0x26c4, 0x26c5}, + {0x26ce, 0x26ce}, {0x26d4, 0x26d4}, {0x26ea, 0x26ea}, + {0x26f2, 0x26f3}, {0x26f5, 0x26f5}, {0x26fa, 0x26fa}, + {0x26fd, 0x26fd}, {0x2705, 0x2705}, {0x270a, 0x270b}, + {0x2728, 0x2728}, {0x274c, 0x274c}, {0x274e, 0x274e}, + {0x2753, 0x2755}, {0x2757, 0x2757}, {0x2795, 0x2797}, + {0x27b0, 0x27b0}, {0x27bf, 0x27bf}, {0x2b1b, 0x2b1c}, + {0x2b50, 0x2b50}, {0x2b55, 0x2b55}, {0x2e80, 0x2fdf}, + {0x2ff0, 0x303e}, {0x3040, 0x3247}, {0x3250, 0x4dbf}, + {0x4e00, 0xa4cf}, {0xa960, 0xa97f}, {0xac00, 0xd7a3}, + {0xf900, 0xfaff}, {0xfe10, 0xfe19}, {0xfe30, 0xfe6f}, + {0xff01, 0xff60}, {0xffe0, 0xffe6}, {0x16fe0, 0x16fe1}, + {0x17000, 0x18aff}, {0x1b000, 0x1b12f}, {0x1b170, 0x1b2ff}, + {0x1f004, 0x1f004}, {0x1f0cf, 0x1f0cf}, {0x1f18e, 0x1f18e}, + {0x1f191, 0x1f19a}, {0x1f200, 0x1f202}, {0x1f210, 0x1f23b}, + {0x1f240, 0x1f248}, {0x1f250, 0x1f251}, {0x1f260, 0x1f265}, + {0x1f300, 0x1f320}, {0x1f32d, 0x1f335}, {0x1f337, 0x1f37c}, + {0x1f37e, 0x1f393}, {0x1f3a0, 0x1f3ca}, {0x1f3cf, 0x1f3d3}, + {0x1f3e0, 0x1f3f0}, {0x1f3f4, 0x1f3f4}, {0x1f3f8, 0x1f43e}, + {0x1f440, 0x1f440}, {0x1f442, 0x1f4fc}, {0x1f4ff, 0x1f53d}, + {0x1f54b, 0x1f54e}, {0x1f550, 0x1f567}, {0x1f57a, 0x1f57a}, + {0x1f595, 0x1f596}, {0x1f5a4, 0x1f5a4}, {0x1f5fb, 0x1f64f}, + {0x1f680, 0x1f6c5}, {0x1f6cc, 0x1f6cc}, {0x1f6d0, 0x1f6d2}, + {0x1f6eb, 0x1f6ec}, {0x1f6f4, 0x1f6f8}, {0x1f910, 0x1f93e}, + {0x1f940, 0x1f94c}, {0x1f950, 0x1f96b}, {0x1f980, 0x1f997}, + {0x1f9c0, 0x1f9c0}, {0x1f9d0, 0x1f9e6}, {0x20000, 0x2fffd}, + {0x30000, 0x3fffd}, + }; + + if ( bisearch(ucs, wide, sizeof(wide) / sizeof(struct interval) - 1) ) { + return 1; + } + + return 0; +} + +static int mk_wcwidth(int32_t ucs) { + /* sorted list of non-overlapping intervals of non-spacing characters */ + /* generated by "uniset +cat=Me +cat=Mn +cat=Cf -00AD +1160-11FF +200B c" */ + static const struct interval combining[] = { + {0x00ad, 0x00ad}, {0x0300, 0x036f}, {0x0483, 0x0489}, + {0x0591, 0x05bd}, {0x05bf, 0x05bf}, {0x05c1, 0x05c2}, + {0x05c4, 0x05c5}, {0x05c7, 0x05c7}, {0x0610, 0x061a}, + {0x061c, 0x061c}, {0x064b, 0x065f}, {0x0670, 0x0670}, + {0x06d6, 0x06dc}, {0x06df, 0x06e4}, {0x06e7, 0x06e8}, + {0x06ea, 0x06ed}, {0x0711, 0x0711}, {0x0730, 0x074a}, + {0x07a6, 0x07b0}, {0x07eb, 0x07f3}, {0x0816, 0x0819}, + {0x081b, 0x0823}, {0x0825, 0x0827}, {0x0829, 0x082d}, + {0x0859, 0x085b}, {0x08d4, 0x08e1}, {0x08e3, 0x0902}, + {0x093a, 0x093a}, {0x093c, 0x093c}, {0x0941, 0x0948}, + {0x094d, 0x094d}, {0x0951, 0x0957}, {0x0962, 0x0963}, + {0x0981, 0x0981}, {0x09bc, 0x09bc}, {0x09c1, 0x09c4}, + {0x09cd, 0x09cd}, {0x09e2, 0x09e3}, {0x0a01, 0x0a02}, + {0x0a3c, 0x0a3c}, {0x0a41, 0x0a42}, {0x0a47, 0x0a48}, + {0x0a4b, 0x0a4d}, {0x0a51, 0x0a51}, {0x0a70, 0x0a71}, + {0x0a75, 0x0a75}, {0x0a81, 0x0a82}, {0x0abc, 0x0abc}, + {0x0ac1, 0x0ac5}, {0x0ac7, 0x0ac8}, {0x0acd, 0x0acd}, + {0x0ae2, 0x0ae3}, {0x0afa, 0x0aff}, {0x0b01, 0x0b01}, + {0x0b3c, 0x0b3c}, {0x0b3f, 0x0b3f}, {0x0b41, 0x0b44}, + {0x0b4d, 0x0b4d}, {0x0b56, 0x0b56}, {0x0b62, 0x0b63}, + {0x0b82, 0x0b82}, {0x0bc0, 0x0bc0}, {0x0bcd, 0x0bcd}, + {0x0c00, 0x0c00}, {0x0c3e, 0x0c40}, {0x0c46, 0x0c48}, + {0x0c4a, 0x0c4d}, {0x0c55, 0x0c56}, {0x0c62, 0x0c63}, + {0x0c81, 0x0c81}, {0x0cbc, 0x0cbc}, {0x0cbf, 0x0cbf}, + {0x0cc6, 0x0cc6}, {0x0ccc, 0x0ccd}, {0x0ce2, 0x0ce3}, + {0x0d00, 0x0d01}, {0x0d3b, 0x0d3c}, {0x0d41, 0x0d44}, + {0x0d4d, 0x0d4d}, {0x0d62, 0x0d63}, {0x0dca, 0x0dca}, + {0x0dd2, 0x0dd4}, {0x0dd6, 0x0dd6}, {0x0e31, 0x0e31}, + {0x0e34, 0x0e3a}, {0x0e47, 0x0e4e}, {0x0eb1, 0x0eb1}, + {0x0eb4, 0x0eb9}, {0x0ebb, 0x0ebc}, {0x0ec8, 0x0ecd}, + {0x0f18, 0x0f19}, {0x0f35, 0x0f35}, {0x0f37, 0x0f37}, + {0x0f39, 0x0f39}, {0x0f71, 0x0f7e}, {0x0f80, 0x0f84}, + {0x0f86, 0x0f87}, {0x0f8d, 0x0f97}, {0x0f99, 0x0fbc}, + {0x0fc6, 0x0fc6}, {0x102d, 0x1030}, {0x1032, 0x1037}, + {0x1039, 0x103a}, {0x103d, 0x103e}, {0x1058, 0x1059}, + {0x105e, 0x1060}, {0x1071, 0x1074}, {0x1082, 0x1082}, + {0x1085, 0x1086}, {0x108d, 0x108d}, {0x109d, 0x109d}, + {0x1160, 0x11ff}, {0x135d, 0x135f}, {0x1712, 0x1714}, + {0x1732, 0x1734}, {0x1752, 0x1753}, {0x1772, 0x1773}, + {0x17b4, 0x17b5}, {0x17b7, 0x17bd}, {0x17c6, 0x17c6}, + {0x17c9, 0x17d3}, {0x17dd, 0x17dd}, {0x180b, 0x180e}, + {0x1885, 0x1886}, {0x18a9, 0x18a9}, {0x1920, 0x1922}, + {0x1927, 0x1928}, {0x1932, 0x1932}, {0x1939, 0x193b}, + {0x1a17, 0x1a18}, {0x1a1b, 0x1a1b}, {0x1a56, 0x1a56}, + {0x1a58, 0x1a5e}, {0x1a60, 0x1a60}, {0x1a62, 0x1a62}, + {0x1a65, 0x1a6c}, {0x1a73, 0x1a7c}, {0x1a7f, 0x1a7f}, + {0x1ab0, 0x1abe}, {0x1b00, 0x1b03}, {0x1b34, 0x1b34}, + {0x1b36, 0x1b3a}, {0x1b3c, 0x1b3c}, {0x1b42, 0x1b42}, + {0x1b6b, 0x1b73}, {0x1b80, 0x1b81}, {0x1ba2, 0x1ba5}, + {0x1ba8, 0x1ba9}, {0x1bab, 0x1bad}, {0x1be6, 0x1be6}, + {0x1be8, 0x1be9}, {0x1bed, 0x1bed}, {0x1bef, 0x1bf1}, + {0x1c2c, 0x1c33}, {0x1c36, 0x1c37}, {0x1cd0, 0x1cd2}, + {0x1cd4, 0x1ce0}, {0x1ce2, 0x1ce8}, {0x1ced, 0x1ced}, + {0x1cf4, 0x1cf4}, {0x1cf8, 0x1cf9}, {0x1dc0, 0x1df9}, + {0x1dfb, 0x1dff}, {0x200b, 0x200f}, {0x202a, 0x202e}, + {0x2060, 0x2064}, {0x2066, 0x206f}, {0x20d0, 0x20f0}, + {0x2cef, 0x2cf1}, {0x2d7f, 0x2d7f}, {0x2de0, 0x2dff}, + {0x302a, 0x302d}, {0x3099, 0x309a}, {0xa66f, 0xa672}, + {0xa674, 0xa67d}, {0xa69e, 0xa69f}, {0xa6f0, 0xa6f1}, + {0xa802, 0xa802}, {0xa806, 0xa806}, {0xa80b, 0xa80b}, + {0xa825, 0xa826}, {0xa8c4, 0xa8c5}, {0xa8e0, 0xa8f1}, + {0xa926, 0xa92d}, {0xa947, 0xa951}, {0xa980, 0xa982}, + {0xa9b3, 0xa9b3}, {0xa9b6, 0xa9b9}, {0xa9bc, 0xa9bc}, + {0xa9e5, 0xa9e5}, {0xaa29, 0xaa2e}, {0xaa31, 0xaa32}, + {0xaa35, 0xaa36}, {0xaa43, 0xaa43}, {0xaa4c, 0xaa4c}, + {0xaa7c, 0xaa7c}, {0xaab0, 0xaab0}, {0xaab2, 0xaab4}, + {0xaab7, 0xaab8}, {0xaabe, 0xaabf}, {0xaac1, 0xaac1}, + {0xaaec, 0xaaed}, {0xaaf6, 0xaaf6}, {0xabe5, 0xabe5}, + {0xabe8, 0xabe8}, {0xabed, 0xabed}, {0xfb1e, 0xfb1e}, + {0xfe00, 0xfe0f}, {0xfe20, 0xfe2f}, {0xfeff, 0xfeff}, + {0xfff9, 0xfffb}, {0x101fd, 0x101fd}, {0x102e0, 0x102e0}, + {0x10376, 0x1037a}, {0x10a01, 0x10a03}, {0x10a05, 0x10a06}, + {0x10a0c, 0x10a0f}, {0x10a38, 0x10a3a}, {0x10a3f, 0x10a3f}, + {0x10ae5, 0x10ae6}, {0x11001, 0x11001}, {0x11038, 0x11046}, + {0x1107f, 0x11081}, {0x110b3, 0x110b6}, {0x110b9, 0x110ba}, + {0x11100, 0x11102}, {0x11127, 0x1112b}, {0x1112d, 0x11134}, + {0x11173, 0x11173}, {0x11180, 0x11181}, {0x111b6, 0x111be}, + {0x111ca, 0x111cc}, {0x1122f, 0x11231}, {0x11234, 0x11234}, + {0x11236, 0x11237}, {0x1123e, 0x1123e}, {0x112df, 0x112df}, + {0x112e3, 0x112ea}, {0x11300, 0x11301}, {0x1133c, 0x1133c}, + {0x11340, 0x11340}, {0x11366, 0x1136c}, {0x11370, 0x11374}, + {0x11438, 0x1143f}, {0x11442, 0x11444}, {0x11446, 0x11446}, + {0x114b3, 0x114b8}, {0x114ba, 0x114ba}, {0x114bf, 0x114c0}, + {0x114c2, 0x114c3}, {0x115b2, 0x115b5}, {0x115bc, 0x115bd}, + {0x115bf, 0x115c0}, {0x115dc, 0x115dd}, {0x11633, 0x1163a}, + {0x1163d, 0x1163d}, {0x1163f, 0x11640}, {0x116ab, 0x116ab}, + {0x116ad, 0x116ad}, {0x116b0, 0x116b5}, {0x116b7, 0x116b7}, + {0x1171d, 0x1171f}, {0x11722, 0x11725}, {0x11727, 0x1172b}, + {0x11a01, 0x11a06}, {0x11a09, 0x11a0a}, {0x11a33, 0x11a38}, + {0x11a3b, 0x11a3e}, {0x11a47, 0x11a47}, {0x11a51, 0x11a56}, + {0x11a59, 0x11a5b}, {0x11a8a, 0x11a96}, {0x11a98, 0x11a99}, + {0x11c30, 0x11c36}, {0x11c38, 0x11c3d}, {0x11c3f, 0x11c3f}, + {0x11c92, 0x11ca7}, {0x11caa, 0x11cb0}, {0x11cb2, 0x11cb3}, + {0x11cb5, 0x11cb6}, {0x11d31, 0x11d36}, {0x11d3a, 0x11d3a}, + {0x11d3c, 0x11d3d}, {0x11d3f, 0x11d45}, {0x11d47, 0x11d47}, + {0x16af0, 0x16af4}, {0x16b30, 0x16b36}, {0x16f8f, 0x16f92}, + {0x1bc9d, 0x1bc9e}, {0x1bca0, 0x1bca3}, {0x1d167, 0x1d169}, + {0x1d173, 0x1d182}, {0x1d185, 0x1d18b}, {0x1d1aa, 0x1d1ad}, + {0x1d242, 0x1d244}, {0x1da00, 0x1da36}, {0x1da3b, 0x1da6c}, + {0x1da75, 0x1da75}, {0x1da84, 0x1da84}, {0x1da9b, 0x1da9f}, + {0x1daa1, 0x1daaf}, {0x1e000, 0x1e006}, {0x1e008, 0x1e018}, + {0x1e01b, 0x1e021}, {0x1e023, 0x1e024}, {0x1e026, 0x1e02a}, + {0x1e8d0, 0x1e8d6}, {0x1e944, 0x1e94a}, {0xe0001, 0xe0001}, + {0xe0020, 0xe007f}, {0xe0100, 0xe01ef}, + }; + + /* test for 8-bit control characters */ + if ( ucs == 0 ) { + return 0; + } + if ( ( ucs < 32 ) || ( ( ucs >= 0x7f ) && ( ucs < 0xa0 ) ) ) { + return -1; + } + + /* binary search in table of non-spacing characters */ + if ( bisearch( ucs, combining, sizeof( combining ) / sizeof( struct interval ) - 1 ) ) { + return 0; + } + + /* if we arrive here, ucs is not a combining or C0/C1 control character */ + return ( mk_is_wide_char( ucs ) ? 2 : 1 ); +} + diff --git a/extern/linenoise.hpp b/extern/linenoise.hpp deleted file mode 100644 index ae36eb0b..00000000 --- a/extern/linenoise.hpp +++ /dev/null @@ -1,2415 +0,0 @@ -/* - * linenoise.hpp -- Multi-platfrom C++ header-only linenoise library. - * - * All credits and commendations have to go to the authors of the - * following excellent libraries. - * - * - linenoise.h and linenose.c (https://github.com/antirez/linenoise) - * - ANSI.c (https://github.com/adoxa/ansicon) - * - Win32_ANSI.h and Win32_ANSI.c (https://github.com/MSOpenTech/redis) - * - * ------------------------------------------------------------------------ - * - * Copyright (c) 2015 yhirose - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * 1. Redistributions of source code must retain the above copyright notice, this - * list of conditions and the following disclaimer. - * 2. Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR - * ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES - * (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; - * LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND - * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT - * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS - * SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - */ - -/* linenoise.h -- guerrilla line editing library against the idea that a - * line editing lib needs to be 20,000 lines of C code. - * - * See linenoise.c for more information. - * - * ------------------------------------------------------------------------ - * - * Copyright (c) 2010, Salvatore Sanfilippo - * Copyright (c) 2010, Pieter Noordhuis - * - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are - * met: - * - * * Redistributions of source code must retain the above copyright - * notice, this list of conditions and the following disclaimer. - * - * * Redistributions in binary form must reproduce the above copyright - * notice, this list of conditions and the following disclaimer in the - * documentation and/or other materials provided with the distribution. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS - * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT - * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR - * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT - * HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, - * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT - * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, - * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY - * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT - * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - */ - -/* - * ANSI.c - ANSI escape sequence console driver. - * - * Copyright (C) 2005-2014 Jason Hood - * This software is provided 'as-is', without any express or implied - * warranty. In no event will the author be held liable for any damages - * arising from the use of this software. - * - * Permission is granted to anyone to use this software for any purpose, - * including commercial applications, and to alter it and redistribute it - * freely, subject to the following restrictions: - * - * 1. The origin of this software must not be misrepresented; you must not - * claim that you wrote the original software. If you use this software - * in a product, an acknowledgment in the product documentation would be - * appreciated but is not required. - * 2. Altered source versions must be plainly marked as such, and must not be - * misrepresented as being the original software. - * 3. This notice may not be removed or altered from any source distribution. - * - * Jason Hood - * jadoxa@yahoo.com.au - */ - -/* - * Win32_ANSI.h and Win32_ANSI.c - * - * Derived from ANSI.c by Jason Hood, from his ansicon project (https://github.com/adoxa/ansicon), with modifications. - * - * Copyright (c), Microsoft Open Technologies, Inc. - * All rights reserved. - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * - Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" - * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE - * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - */ - -#ifndef LINENOISE_HPP -#define LINENOISE_HPP - -#ifndef _WIN32 -#include -#include -#include -#else -#ifndef NOMINMAX -#define NOMINMAX -#endif -#include -#include -#ifndef STDIN_FILENO -#define STDIN_FILENO (_fileno(stdin)) -#endif -#ifndef STDOUT_FILENO -#define STDOUT_FILENO 1 -#endif -#define isatty _isatty -#define write win32_write -#define read _read -#pragma warning(push) -#pragma warning(disable : 4996) -#endif -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include - -namespace linenoise { - -typedef std::function&)> CompletionCallback; - -#ifdef _WIN32 - -namespace ansi { - -#define lenof(array) (sizeof(array)/sizeof(*(array))) - -typedef struct -{ - BYTE foreground; // ANSI base color (0 to 7; add 30) - BYTE background; // ANSI base color (0 to 7; add 40) - BYTE bold; // console FOREGROUND_INTENSITY bit - BYTE underline; // console BACKGROUND_INTENSITY bit - BYTE rvideo; // swap foreground/bold & background/underline - BYTE concealed; // set foreground/bold to background/underline - BYTE reverse; // swap console foreground & background attributes -} GRM, *PGRM; // Graphic Rendition Mode - - -inline bool is_digit(char c) { return '0' <= c && c <= '9'; } - -// ========== Global variables and constants - -HANDLE hConOut; // handle to CONOUT$ - -const char ESC = '\x1B'; // ESCape character -const char BEL = '\x07'; -const char SO = '\x0E'; // Shift Out -const char SI = '\x0F'; // Shift In - -const int MAX_ARG = 16; // max number of args in an escape sequence -int state; // automata state -WCHAR prefix; // escape sequence prefix ( '[', ']' or '(' ); -WCHAR prefix2; // secondary prefix ( '?' or '>' ); -WCHAR suffix; // escape sequence suffix -int es_argc; // escape sequence args count -int es_argv[MAX_ARG]; // escape sequence args -WCHAR Pt_arg[MAX_PATH * 2]; // text parameter for Operating System Command -int Pt_len; -BOOL shifted; - - -// DEC Special Graphics Character Set from -// http://vt100.net/docs/vt220-rm/table2-4.html -// Some of these may not look right, depending on the font and code page (in -// particular, the Control Pictures probably won't work at all). -const WCHAR G1[] = -{ - ' ', // _ - blank - L'\x2666', // ` - Black Diamond Suit - L'\x2592', // a - Medium Shade - L'\x2409', // b - HT - L'\x240c', // c - FF - L'\x240d', // d - CR - L'\x240a', // e - LF - L'\x00b0', // f - Degree Sign - L'\x00b1', // g - Plus-Minus Sign - L'\x2424', // h - NL - L'\x240b', // i - VT - L'\x2518', // j - Box Drawings Light Up And Left - L'\x2510', // k - Box Drawings Light Down And Left - L'\x250c', // l - Box Drawings Light Down And Right - L'\x2514', // m - Box Drawings Light Up And Right - L'\x253c', // n - Box Drawings Light Vertical And Horizontal - L'\x00af', // o - SCAN 1 - Macron - L'\x25ac', // p - SCAN 3 - Black Rectangle - L'\x2500', // q - SCAN 5 - Box Drawings Light Horizontal - L'_', // r - SCAN 7 - Low Line - L'_', // s - SCAN 9 - Low Line - L'\x251c', // t - Box Drawings Light Vertical And Right - L'\x2524', // u - Box Drawings Light Vertical And Left - L'\x2534', // v - Box Drawings Light Up And Horizontal - L'\x252c', // w - Box Drawings Light Down And Horizontal - L'\x2502', // x - Box Drawings Light Vertical - L'\x2264', // y - Less-Than Or Equal To - L'\x2265', // z - Greater-Than Or Equal To - L'\x03c0', // { - Greek Small Letter Pi - L'\x2260', // | - Not Equal To - L'\x00a3', // } - Pound Sign - L'\x00b7', // ~ - Middle Dot -}; - -#define FIRST_G1 '_' -#define LAST_G1 '~' - - -// color constants - -#define FOREGROUND_BLACK 0 -#define FOREGROUND_WHITE FOREGROUND_RED|FOREGROUND_GREEN|FOREGROUND_BLUE - -#define BACKGROUND_BLACK 0 -#define BACKGROUND_WHITE BACKGROUND_RED|BACKGROUND_GREEN|BACKGROUND_BLUE - -const BYTE foregroundcolor[8] = - { - FOREGROUND_BLACK, // black foreground - FOREGROUND_RED, // red foreground - FOREGROUND_GREEN, // green foreground - FOREGROUND_RED | FOREGROUND_GREEN, // yellow foreground - FOREGROUND_BLUE, // blue foreground - FOREGROUND_BLUE | FOREGROUND_RED, // magenta foreground - FOREGROUND_BLUE | FOREGROUND_GREEN, // cyan foreground - FOREGROUND_WHITE // white foreground - }; - -const BYTE backgroundcolor[8] = - { - BACKGROUND_BLACK, // black background - BACKGROUND_RED, // red background - BACKGROUND_GREEN, // green background - BACKGROUND_RED | BACKGROUND_GREEN, // yellow background - BACKGROUND_BLUE, // blue background - BACKGROUND_BLUE | BACKGROUND_RED, // magenta background - BACKGROUND_BLUE | BACKGROUND_GREEN, // cyan background - BACKGROUND_WHITE, // white background - }; - -const BYTE attr2ansi[8] = // map console attribute to ANSI number -{ - 0, // black - 4, // blue - 2, // green - 6, // cyan - 1, // red - 5, // magenta - 3, // yellow - 7 // white -}; - -GRM grm; - -// saved cursor position -COORD SavePos; - -// ========== Print Buffer functions - -#define BUFFER_SIZE 2048 - -int nCharInBuffer; -WCHAR ChBuffer[BUFFER_SIZE]; - -//----------------------------------------------------------------------------- -// FlushBuffer() -// Writes the buffer to the console and empties it. -//----------------------------------------------------------------------------- - -inline void FlushBuffer(void) -{ - DWORD nWritten; - if (nCharInBuffer <= 0) return; - WriteConsoleW(hConOut, ChBuffer, nCharInBuffer, &nWritten, NULL); - nCharInBuffer = 0; -} - -//----------------------------------------------------------------------------- -// PushBuffer( WCHAR c ) -// Adds a character in the buffer. -//----------------------------------------------------------------------------- - -inline void PushBuffer(WCHAR c) -{ - if (shifted && c >= FIRST_G1 && c <= LAST_G1) - c = G1[c - FIRST_G1]; - ChBuffer[nCharInBuffer] = c; - if (++nCharInBuffer == BUFFER_SIZE) - FlushBuffer(); -} - -//----------------------------------------------------------------------------- -// SendSequence( LPCWSTR seq ) -// Send the string to the input buffer. -//----------------------------------------------------------------------------- - -inline void SendSequence(LPCWSTR seq) -{ - DWORD out; - INPUT_RECORD in; - HANDLE hStdIn = GetStdHandle(STD_INPUT_HANDLE); - - in.EventType = KEY_EVENT; - in.Event.KeyEvent.bKeyDown = TRUE; - in.Event.KeyEvent.wRepeatCount = 1; - in.Event.KeyEvent.wVirtualKeyCode = 0; - in.Event.KeyEvent.wVirtualScanCode = 0; - in.Event.KeyEvent.dwControlKeyState = 0; - for (; *seq; ++seq) - { - in.Event.KeyEvent.uChar.UnicodeChar = *seq; - WriteConsoleInput(hStdIn, &in, 1, &out); - } -} - -// ========== Print functions - -//----------------------------------------------------------------------------- -// InterpretEscSeq() -// Interprets the last escape sequence scanned by ParseAndPrintANSIString -// prefix escape sequence prefix -// es_argc escape sequence args count -// es_argv[] escape sequence args array -// suffix escape sequence suffix -// -// for instance, with \e[33;45;1m we have -// prefix = '[', -// es_argc = 3, es_argv[0] = 33, es_argv[1] = 45, es_argv[2] = 1 -// suffix = 'm' -//----------------------------------------------------------------------------- - -inline void InterpretEscSeq(void) -{ - int i; - WORD attribut; - CONSOLE_SCREEN_BUFFER_INFO Info; - CONSOLE_CURSOR_INFO CursInfo; - DWORD len, NumberOfCharsWritten; - COORD Pos; - SMALL_RECT Rect; - CHAR_INFO CharInfo; - - if (prefix == '[') - { - if (prefix2 == '?' && (suffix == 'h' || suffix == 'l')) - { - if (es_argc == 1 && es_argv[0] == 25) - { - GetConsoleCursorInfo(hConOut, &CursInfo); - CursInfo.bVisible = (suffix == 'h'); - SetConsoleCursorInfo(hConOut, &CursInfo); - return; - } - } - // Ignore any other \e[? or \e[> sequences. - if (prefix2 != 0) - return; - - GetConsoleScreenBufferInfo(hConOut, &Info); - switch (suffix) - { - case 'm': - if (es_argc == 0) es_argv[es_argc++] = 0; - for (i = 0; i < es_argc; i++) - { - if (30 <= es_argv[i] && es_argv[i] <= 37) - grm.foreground = es_argv[i] - 30; - else if (40 <= es_argv[i] && es_argv[i] <= 47) - grm.background = es_argv[i] - 40; - else switch (es_argv[i]) - { - case 0: - case 39: - case 49: - { - WCHAR def[4]; - int a; - *def = '7'; def[1] = '\0'; - GetEnvironmentVariableW(L"ANSICON_DEF", def, lenof(def)); - a = wcstol(def, NULL, 16); - grm.reverse = FALSE; - if (a < 0) - { - grm.reverse = TRUE; - a = -a; - } - if (es_argv[i] != 49) - grm.foreground = attr2ansi[a & 7]; - if (es_argv[i] != 39) - grm.background = attr2ansi[(a >> 4) & 7]; - if (es_argv[i] == 0) - { - if (es_argc == 1) - { - grm.bold = a & FOREGROUND_INTENSITY; - grm.underline = a & BACKGROUND_INTENSITY; - } - else - { - grm.bold = 0; - grm.underline = 0; - } - grm.rvideo = 0; - grm.concealed = 0; - } - } - break; - - case 1: grm.bold = FOREGROUND_INTENSITY; break; - case 5: // blink - case 4: grm.underline = BACKGROUND_INTENSITY; break; - case 7: grm.rvideo = 1; break; - case 8: grm.concealed = 1; break; - case 21: // oops, this actually turns on double underline - case 22: grm.bold = 0; break; - case 25: - case 24: grm.underline = 0; break; - case 27: grm.rvideo = 0; break; - case 28: grm.concealed = 0; break; - } - } - if (grm.concealed) - { - if (grm.rvideo) - { - attribut = foregroundcolor[grm.foreground] - | backgroundcolor[grm.foreground]; - if (grm.bold) - attribut |= FOREGROUND_INTENSITY | BACKGROUND_INTENSITY; - } - else - { - attribut = foregroundcolor[grm.background] - | backgroundcolor[grm.background]; - if (grm.underline) - attribut |= FOREGROUND_INTENSITY | BACKGROUND_INTENSITY; - } - } - else if (grm.rvideo) - { - attribut = foregroundcolor[grm.background] - | backgroundcolor[grm.foreground]; - if (grm.bold) - attribut |= BACKGROUND_INTENSITY; - if (grm.underline) - attribut |= FOREGROUND_INTENSITY; - } - else - attribut = foregroundcolor[grm.foreground] | grm.bold - | backgroundcolor[grm.background] | grm.underline; - if (grm.reverse) - attribut = ((attribut >> 4) & 15) | ((attribut & 15) << 4); - SetConsoleTextAttribute(hConOut, attribut); - return; - - case 'J': - if (es_argc == 0) es_argv[es_argc++] = 0; // ESC[J == ESC[0J - if (es_argc != 1) return; - switch (es_argv[0]) - { - case 0: // ESC[0J erase from cursor to end of display - len = (Info.dwSize.Y - Info.dwCursorPosition.Y - 1) * Info.dwSize.X - + Info.dwSize.X - Info.dwCursorPosition.X - 1; - FillConsoleOutputCharacter(hConOut, ' ', len, - Info.dwCursorPosition, - &NumberOfCharsWritten); - FillConsoleOutputAttribute(hConOut, Info.wAttributes, len, - Info.dwCursorPosition, - &NumberOfCharsWritten); - return; - - case 1: // ESC[1J erase from start to cursor. - Pos.X = 0; - Pos.Y = 0; - len = Info.dwCursorPosition.Y * Info.dwSize.X - + Info.dwCursorPosition.X + 1; - FillConsoleOutputCharacter(hConOut, ' ', len, Pos, - &NumberOfCharsWritten); - FillConsoleOutputAttribute(hConOut, Info.wAttributes, len, Pos, - &NumberOfCharsWritten); - return; - - case 2: // ESC[2J Clear screen and home cursor - Pos.X = 0; - Pos.Y = 0; - len = Info.dwSize.X * Info.dwSize.Y; - FillConsoleOutputCharacter(hConOut, ' ', len, Pos, - &NumberOfCharsWritten); - FillConsoleOutputAttribute(hConOut, Info.wAttributes, len, Pos, - &NumberOfCharsWritten); - SetConsoleCursorPosition(hConOut, Pos); - return; - - default: - return; - } - - case 'K': - if (es_argc == 0) es_argv[es_argc++] = 0; // ESC[K == ESC[0K - if (es_argc != 1) return; - switch (es_argv[0]) - { - case 0: // ESC[0K Clear to end of line - len = Info.dwSize.X - Info.dwCursorPosition.X + 1; - FillConsoleOutputCharacter(hConOut, ' ', len, - Info.dwCursorPosition, - &NumberOfCharsWritten); - FillConsoleOutputAttribute(hConOut, Info.wAttributes, len, - Info.dwCursorPosition, - &NumberOfCharsWritten); - return; - - case 1: // ESC[1K Clear from start of line to cursor - Pos.X = 0; - Pos.Y = Info.dwCursorPosition.Y; - FillConsoleOutputCharacter(hConOut, ' ', - Info.dwCursorPosition.X + 1, Pos, - &NumberOfCharsWritten); - FillConsoleOutputAttribute(hConOut, Info.wAttributes, - Info.dwCursorPosition.X + 1, Pos, - &NumberOfCharsWritten); - return; - - case 2: // ESC[2K Clear whole line. - Pos.X = 0; - Pos.Y = Info.dwCursorPosition.Y; - FillConsoleOutputCharacter(hConOut, ' ', Info.dwSize.X, Pos, - &NumberOfCharsWritten); - FillConsoleOutputAttribute(hConOut, Info.wAttributes, - Info.dwSize.X, Pos, - &NumberOfCharsWritten); - return; - - default: - return; - } - - case 'X': // ESC[#X Erase # characters. - if (es_argc == 0) es_argv[es_argc++] = 1; // ESC[X == ESC[1X - if (es_argc != 1) return; - FillConsoleOutputCharacter(hConOut, ' ', es_argv[0], - Info.dwCursorPosition, - &NumberOfCharsWritten); - FillConsoleOutputAttribute(hConOut, Info.wAttributes, es_argv[0], - Info.dwCursorPosition, - &NumberOfCharsWritten); - return; - - case 'L': // ESC[#L Insert # blank lines. - if (es_argc == 0) es_argv[es_argc++] = 1; // ESC[L == ESC[1L - if (es_argc != 1) return; - Rect.Left = 0; - Rect.Top = Info.dwCursorPosition.Y; - Rect.Right = Info.dwSize.X - 1; - Rect.Bottom = Info.dwSize.Y - 1; - Pos.X = 0; - Pos.Y = Info.dwCursorPosition.Y + es_argv[0]; - CharInfo.Char.UnicodeChar = ' '; - CharInfo.Attributes = Info.wAttributes; - ScrollConsoleScreenBuffer(hConOut, &Rect, NULL, Pos, &CharInfo); - return; - - case 'M': // ESC[#M Delete # lines. - if (es_argc == 0) es_argv[es_argc++] = 1; // ESC[M == ESC[1M - if (es_argc != 1) return; - if (es_argv[0] > Info.dwSize.Y - Info.dwCursorPosition.Y) - es_argv[0] = Info.dwSize.Y - Info.dwCursorPosition.Y; - Rect.Left = 0; - Rect.Top = Info.dwCursorPosition.Y + es_argv[0]; - Rect.Right = Info.dwSize.X - 1; - Rect.Bottom = Info.dwSize.Y - 1; - Pos.X = 0; - Pos.Y = Info.dwCursorPosition.Y; - CharInfo.Char.UnicodeChar = ' '; - CharInfo.Attributes = Info.wAttributes; - ScrollConsoleScreenBuffer(hConOut, &Rect, NULL, Pos, &CharInfo); - return; - - case 'P': // ESC[#P Delete # characters. - if (es_argc == 0) es_argv[es_argc++] = 1; // ESC[P == ESC[1P - if (es_argc != 1) return; - if (Info.dwCursorPosition.X + es_argv[0] > Info.dwSize.X - 1) - es_argv[0] = Info.dwSize.X - Info.dwCursorPosition.X; - Rect.Left = Info.dwCursorPosition.X + es_argv[0]; - Rect.Top = Info.dwCursorPosition.Y; - Rect.Right = Info.dwSize.X - 1; - Rect.Bottom = Info.dwCursorPosition.Y; - CharInfo.Char.UnicodeChar = ' '; - CharInfo.Attributes = Info.wAttributes; - ScrollConsoleScreenBuffer(hConOut, &Rect, NULL, Info.dwCursorPosition, - &CharInfo); - return; - - case '@': // ESC[#@ Insert # blank characters. - if (es_argc == 0) es_argv[es_argc++] = 1; // ESC[@ == ESC[1@ - if (es_argc != 1) return; - if (Info.dwCursorPosition.X + es_argv[0] > Info.dwSize.X - 1) - es_argv[0] = Info.dwSize.X - Info.dwCursorPosition.X; - Rect.Left = Info.dwCursorPosition.X; - Rect.Top = Info.dwCursorPosition.Y; - Rect.Right = Info.dwSize.X - 1 - es_argv[0]; - Rect.Bottom = Info.dwCursorPosition.Y; - Pos.X = Info.dwCursorPosition.X + es_argv[0]; - Pos.Y = Info.dwCursorPosition.Y; - CharInfo.Char.UnicodeChar = ' '; - CharInfo.Attributes = Info.wAttributes; - ScrollConsoleScreenBuffer(hConOut, &Rect, NULL, Pos, &CharInfo); - return; - - case 'k': // ESC[#k - case 'A': // ESC[#A Moves cursor up # lines - if (es_argc == 0) es_argv[es_argc++] = 1; // ESC[A == ESC[1A - if (es_argc != 1) return; - Pos.Y = Info.dwCursorPosition.Y - es_argv[0]; - if (Pos.Y < 0) Pos.Y = 0; - Pos.X = Info.dwCursorPosition.X; - SetConsoleCursorPosition(hConOut, Pos); - return; - - case 'e': // ESC[#e - case 'B': // ESC[#B Moves cursor down # lines - if (es_argc == 0) es_argv[es_argc++] = 1; // ESC[B == ESC[1B - if (es_argc != 1) return; - Pos.Y = Info.dwCursorPosition.Y + es_argv[0]; - if (Pos.Y >= Info.dwSize.Y) Pos.Y = Info.dwSize.Y - 1; - Pos.X = Info.dwCursorPosition.X; - SetConsoleCursorPosition(hConOut, Pos); - return; - - case 'a': // ESC[#a - case 'C': // ESC[#C Moves cursor forward # spaces - if (es_argc == 0) es_argv[es_argc++] = 1; // ESC[C == ESC[1C - if (es_argc != 1) return; - Pos.X = Info.dwCursorPosition.X + es_argv[0]; - if (Pos.X >= Info.dwSize.X) Pos.X = Info.dwSize.X - 1; - Pos.Y = Info.dwCursorPosition.Y; - SetConsoleCursorPosition(hConOut, Pos); - return; - - case 'j': // ESC[#j - case 'D': // ESC[#D Moves cursor back # spaces - if (es_argc == 0) es_argv[es_argc++] = 1; // ESC[D == ESC[1D - if (es_argc != 1) return; - Pos.X = Info.dwCursorPosition.X - es_argv[0]; - if (Pos.X < 0) Pos.X = 0; - Pos.Y = Info.dwCursorPosition.Y; - SetConsoleCursorPosition(hConOut, Pos); - return; - - case 'E': // ESC[#E Moves cursor down # lines, column 1. - if (es_argc == 0) es_argv[es_argc++] = 1; // ESC[E == ESC[1E - if (es_argc != 1) return; - Pos.Y = Info.dwCursorPosition.Y + es_argv[0]; - if (Pos.Y >= Info.dwSize.Y) Pos.Y = Info.dwSize.Y - 1; - Pos.X = 0; - SetConsoleCursorPosition(hConOut, Pos); - return; - - case 'F': // ESC[#F Moves cursor up # lines, column 1. - if (es_argc == 0) es_argv[es_argc++] = 1; // ESC[F == ESC[1F - if (es_argc != 1) return; - Pos.Y = Info.dwCursorPosition.Y - es_argv[0]; - if (Pos.Y < 0) Pos.Y = 0; - Pos.X = 0; - SetConsoleCursorPosition(hConOut, Pos); - return; - - case '`': // ESC[#` - case 'G': // ESC[#G Moves cursor column # in current row. - if (es_argc == 0) es_argv[es_argc++] = 1; // ESC[G == ESC[1G - if (es_argc != 1) return; - Pos.X = es_argv[0] - 1; - if (Pos.X >= Info.dwSize.X) Pos.X = Info.dwSize.X - 1; - if (Pos.X < 0) Pos.X = 0; - Pos.Y = Info.dwCursorPosition.Y; - SetConsoleCursorPosition(hConOut, Pos); - return; - - case 'd': // ESC[#d Moves cursor row #, current column. - if (es_argc == 0) es_argv[es_argc++] = 1; // ESC[d == ESC[1d - if (es_argc != 1) return; - Pos.Y = es_argv[0] - 1; - if (Pos.Y < 0) Pos.Y = 0; - if (Pos.Y >= Info.dwSize.Y) Pos.Y = Info.dwSize.Y - 1; - SetConsoleCursorPosition(hConOut, Pos); - return; - - case 'f': // ESC[#;#f - case 'H': // ESC[#;#H Moves cursor to line #, column # - if (es_argc == 0) - es_argv[es_argc++] = 1; // ESC[H == ESC[1;1H - if (es_argc == 1) - es_argv[es_argc++] = 1; // ESC[#H == ESC[#;1H - if (es_argc > 2) return; - Pos.X = es_argv[1] - 1; - if (Pos.X < 0) Pos.X = 0; - if (Pos.X >= Info.dwSize.X) Pos.X = Info.dwSize.X - 1; - Pos.Y = es_argv[0] - 1; - if (Pos.Y < 0) Pos.Y = 0; - if (Pos.Y >= Info.dwSize.Y) Pos.Y = Info.dwSize.Y - 1; - SetConsoleCursorPosition(hConOut, Pos); - return; - - case 's': // ESC[s Saves cursor position for recall later - if (es_argc != 0) return; - SavePos = Info.dwCursorPosition; - return; - - case 'u': // ESC[u Return to saved cursor position - if (es_argc != 0) return; - SetConsoleCursorPosition(hConOut, SavePos); - return; - - case 'n': // ESC[#n Device status report - if (es_argc != 1) return; // ESC[n == ESC[0n -> ignored - switch (es_argv[0]) - { - case 5: // ESC[5n Report status - SendSequence(L"\33[0n"); // "OK" - return; - - case 6: // ESC[6n Report cursor position - { - WCHAR buf[32]; - swprintf(buf, 32, L"\33[%d;%dR", Info.dwCursorPosition.Y + 1, - Info.dwCursorPosition.X + 1); - SendSequence(buf); - } - return; - - default: - return; - } - - case 't': // ESC[#t Window manipulation - if (es_argc != 1) return; - if (es_argv[0] == 21) // ESC[21t Report xterm window's title - { - WCHAR buf[MAX_PATH * 2]; - DWORD len = GetConsoleTitleW(buf + 3, lenof(buf) - 3 - 2); - // Too bad if it's too big or fails. - buf[0] = ESC; - buf[1] = ']'; - buf[2] = 'l'; - buf[3 + len] = ESC; - buf[3 + len + 1] = '\\'; - buf[3 + len + 2] = '\0'; - SendSequence(buf); - } - return; - - default: - return; - } - } - else // (prefix == ']') - { - // Ignore any \e]? or \e]> sequences. - if (prefix2 != 0) - return; - - if (es_argc == 1 && es_argv[0] == 0) // ESC]0;titleST - { - SetConsoleTitleW(Pt_arg); - } - } -} - -//----------------------------------------------------------------------------- -// ParseAndPrintANSIString(hDev, lpBuffer, nNumberOfBytesToWrite) -// Parses the string lpBuffer, interprets the escapes sequences and prints the -// characters in the device hDev (console). -// The lexer is a three states automata. -// If the number of arguments es_argc > MAX_ARG, only the MAX_ARG-1 firsts and -// the last arguments are processed (no es_argv[] overflow). -//----------------------------------------------------------------------------- - -inline BOOL ParseAndPrintANSIString(HANDLE hDev, LPCVOID lpBuffer, DWORD nNumberOfBytesToWrite, LPDWORD lpNumberOfBytesWritten) -{ - DWORD i; - LPCSTR s; - - if (hDev != hConOut) // reinit if device has changed - { - hConOut = hDev; - state = 1; - shifted = FALSE; - } - for (i = nNumberOfBytesToWrite, s = (LPCSTR)lpBuffer; i > 0; i--, s++) - { - if (state == 1) - { - if (*s == ESC) state = 2; - else if (*s == SO) shifted = TRUE; - else if (*s == SI) shifted = FALSE; - else PushBuffer(*s); - } - else if (state == 2) - { - if (*s == ESC); // \e\e...\e == \e - else if ((*s == '[') || (*s == ']')) - { - FlushBuffer(); - prefix = *s; - prefix2 = 0; - state = 3; - Pt_len = 0; - *Pt_arg = '\0'; - } - else if (*s == ')' || *s == '(') state = 6; - else state = 1; - } - else if (state == 3) - { - if (is_digit(*s)) - { - es_argc = 0; - es_argv[0] = *s - '0'; - state = 4; - } - else if (*s == ';') - { - es_argc = 1; - es_argv[0] = 0; - es_argv[1] = 0; - state = 4; - } - else if (*s == '?' || *s == '>') - { - prefix2 = *s; - } - else - { - es_argc = 0; - suffix = *s; - InterpretEscSeq(); - state = 1; - } - } - else if (state == 4) - { - if (is_digit(*s)) - { - es_argv[es_argc] = 10 * es_argv[es_argc] + (*s - '0'); - } - else if (*s == ';') - { - if (es_argc < MAX_ARG - 1) es_argc++; - es_argv[es_argc] = 0; - if (prefix == ']') - state = 5; - } - else - { - es_argc++; - suffix = *s; - InterpretEscSeq(); - state = 1; - } - } - else if (state == 5) - { - if (*s == BEL) - { - Pt_arg[Pt_len] = '\0'; - InterpretEscSeq(); - state = 1; - } - else if (*s == '\\' && Pt_len > 0 && Pt_arg[Pt_len - 1] == ESC) - { - Pt_arg[--Pt_len] = '\0'; - InterpretEscSeq(); - state = 1; - } - else if (Pt_len < lenof(Pt_arg) - 1) - Pt_arg[Pt_len++] = *s; - } - else if (state == 6) - { - // Ignore it (ESC ) 0 is implicit; nothing else is supported). - state = 1; - } - } - FlushBuffer(); - if (lpNumberOfBytesWritten != NULL) - *lpNumberOfBytesWritten = nNumberOfBytesToWrite - i; - return (i == 0); -} - -} // namespace ansi - -HANDLE hOut; -HANDLE hIn; -DWORD consolemodeIn = 0; - -inline int win32read(int *c) { - DWORD foo; - INPUT_RECORD b; - KEY_EVENT_RECORD e; - BOOL altgr; - - while (1) { - if (!ReadConsoleInput(hIn, &b, 1, &foo)) return 0; - if (!foo) return 0; - - if (b.EventType == KEY_EVENT && b.Event.KeyEvent.bKeyDown) { - - e = b.Event.KeyEvent; - *c = b.Event.KeyEvent.uChar.AsciiChar; - - altgr = e.dwControlKeyState & (LEFT_CTRL_PRESSED | RIGHT_ALT_PRESSED); - - if (e.dwControlKeyState & (LEFT_CTRL_PRESSED | RIGHT_CTRL_PRESSED) && !altgr) { - - /* Ctrl+Key */ - switch (*c) { - case 'D': - *c = 4; - return 1; - case 'C': - *c = 3; - return 1; - case 'H': - *c = 8; - return 1; - case 'T': - *c = 20; - return 1; - case 'B': /* ctrl-b, left_arrow */ - *c = 2; - return 1; - case 'F': /* ctrl-f right_arrow*/ - *c = 6; - return 1; - case 'P': /* ctrl-p up_arrow*/ - *c = 16; - return 1; - case 'N': /* ctrl-n down_arrow*/ - *c = 14; - return 1; - case 'U': /* Ctrl+u, delete the whole line. */ - *c = 21; - return 1; - case 'K': /* Ctrl+k, delete from current to end of line. */ - *c = 11; - return 1; - case 'A': /* Ctrl+a, go to the start of the line */ - *c = 1; - return 1; - case 'E': /* ctrl+e, go to the end of the line */ - *c = 5; - return 1; - } - - /* Other Ctrl+KEYs ignored */ - } else { - - switch (e.wVirtualKeyCode) { - - case VK_ESCAPE: /* ignore - send ctrl-c, will return -1 */ - *c = 3; - return 1; - case VK_RETURN: /* enter */ - *c = 13; - return 1; - case VK_LEFT: /* left */ - *c = 2; - return 1; - case VK_RIGHT: /* right */ - *c = 6; - return 1; - case VK_UP: /* up */ - *c = 16; - return 1; - case VK_DOWN: /* down */ - *c = 14; - return 1; - case VK_HOME: - *c = 1; - return 1; - case VK_END: - *c = 5; - return 1; - case VK_BACK: - *c = 8; - return 1; - case VK_DELETE: - *c = 4; /* same as Ctrl+D above */ - return 1; - default: - if (*c) return 1; - } - } - } - } - - return -1; /* Makes compiler happy */ -} - -inline int win32_write(int fd, const void *buffer, unsigned int count) { - if (fd == _fileno(stdout)) { - DWORD bytesWritten = 0; - if (FALSE != ansi::ParseAndPrintANSIString(GetStdHandle(STD_OUTPUT_HANDLE), buffer, (DWORD)count, &bytesWritten)) { - return (int)bytesWritten; - } else { - errno = GetLastError(); - return 0; - } - } else if (fd == _fileno(stderr)) { - DWORD bytesWritten = 0; - if (FALSE != ansi::ParseAndPrintANSIString(GetStdHandle(STD_ERROR_HANDLE), buffer, (DWORD)count, &bytesWritten)) { - return (int)bytesWritten; - } else { - errno = GetLastError(); - return 0; - } - } else { - return _write(fd, buffer, count); - } -} -#endif // _WIN32 - -#define LINENOISE_DEFAULT_HISTORY_MAX_LEN 100 -#define LINENOISE_MAX_LINE 4096 -static const char *unsupported_term[] = {"dumb","cons25","emacs",NULL}; -static CompletionCallback completionCallback; - -#ifndef _WIN32 -static struct termios orig_termios; /* In order to restore at exit.*/ -#endif -static bool rawmode = false; /* For atexit() function to check if restore is needed*/ -static bool mlmode = false; /* Multi line mode. Default is single line. */ -static bool atexit_registered = false; /* Register atexit just 1 time. */ -static size_t history_max_len = LINENOISE_DEFAULT_HISTORY_MAX_LEN; -static std::vector history; - -/* The linenoiseState structure represents the state during line editing. - * We pass this state to functions implementing specific editing - * functionalities. */ -struct linenoiseState { - int ifd; /* Terminal stdin file descriptor. */ - int ofd; /* Terminal stdout file descriptor. */ - char *buf; /* Edited line buffer. */ - int buflen; /* Edited line buffer size. */ - std::string prompt; /* Prompt to display. */ - int pos; /* Current cursor position. */ - int oldcolpos; /* Previous refresh cursor column position. */ - int len; /* Current edited line length. */ - int cols; /* Number of columns in terminal. */ - int maxrows; /* Maximum num of rows used so far (multiline mode) */ - int history_index; /* The history index we are currently editing. */ -}; - -enum KEY_ACTION { - KEY_NULL = 0, /* NULL */ - CTRL_A = 1, /* Ctrl+a */ - CTRL_B = 2, /* Ctrl-b */ - CTRL_C = 3, /* Ctrl-c */ - CTRL_D = 4, /* Ctrl-d */ - CTRL_E = 5, /* Ctrl-e */ - CTRL_F = 6, /* Ctrl-f */ - CTRL_H = 8, /* Ctrl-h */ - TAB = 9, /* Tab */ - CTRL_K = 11, /* Ctrl+k */ - CTRL_L = 12, /* Ctrl+l */ - ENTER = 13, /* Enter */ - CTRL_N = 14, /* Ctrl-n */ - CTRL_P = 16, /* Ctrl-p */ - CTRL_T = 20, /* Ctrl-t */ - CTRL_U = 21, /* Ctrl+u */ - CTRL_W = 23, /* Ctrl+w */ - ESC = 27, /* Escape */ - BACKSPACE = 127 /* Backspace */ -}; - -void linenoiseAtExit(void); -bool AddHistory(const char *line); -void refreshLine(struct linenoiseState *l); - -/* ============================ UTF8 utilities ============================== */ - -static unsigned long unicodeWideCharTable[][2] = { - { 0x1100, 0x115F }, { 0x2329, 0x232A }, { 0x2E80, 0x2E99, }, { 0x2E9B, 0x2EF3, }, - { 0x2F00, 0x2FD5, }, { 0x2FF0, 0x2FFB, }, { 0x3000, 0x303E, }, { 0x3041, 0x3096, }, - { 0x3099, 0x30FF, }, { 0x3105, 0x312D, }, { 0x3131, 0x318E, }, { 0x3190, 0x31BA, }, - { 0x31C0, 0x31E3, }, { 0x31F0, 0x321E, }, { 0x3220, 0x3247, }, { 0x3250, 0x4DBF, }, - { 0x4E00, 0xA48C, }, { 0xA490, 0xA4C6, }, { 0xA960, 0xA97C, }, { 0xAC00, 0xD7A3, }, - { 0xF900, 0xFAFF, }, { 0xFE10, 0xFE19, }, { 0xFE30, 0xFE52, }, { 0xFE54, 0xFE66, }, - { 0xFE68, 0xFE6B, }, { 0xFF01, 0xFFE6, }, - { 0x1B000, 0x1B001, }, { 0x1F200, 0x1F202, }, { 0x1F210, 0x1F23A, }, - { 0x1F240, 0x1F248, }, { 0x1F250, 0x1F251, }, { 0x20000, 0x3FFFD, }, -}; - -static int unicodeWideCharTableSize = sizeof(unicodeWideCharTable) / sizeof(unicodeWideCharTable[0]); - -static int unicodeIsWideChar(unsigned long cp) -{ - int i; - for (i = 0; i < unicodeWideCharTableSize; i++) { - if (unicodeWideCharTable[i][0] <= cp && cp <= unicodeWideCharTable[i][1]) { - return 1; - } - } - return 0; -} - -static unsigned long unicodeCombiningCharTable[] = { - 0x0300,0x0301,0x0302,0x0303,0x0304,0x0305,0x0306,0x0307, - 0x0308,0x0309,0x030A,0x030B,0x030C,0x030D,0x030E,0x030F, - 0x0310,0x0311,0x0312,0x0313,0x0314,0x0315,0x0316,0x0317, - 0x0318,0x0319,0x031A,0x031B,0x031C,0x031D,0x031E,0x031F, - 0x0320,0x0321,0x0322,0x0323,0x0324,0x0325,0x0326,0x0327, - 0x0328,0x0329,0x032A,0x032B,0x032C,0x032D,0x032E,0x032F, - 0x0330,0x0331,0x0332,0x0333,0x0334,0x0335,0x0336,0x0337, - 0x0338,0x0339,0x033A,0x033B,0x033C,0x033D,0x033E,0x033F, - 0x0340,0x0341,0x0342,0x0343,0x0344,0x0345,0x0346,0x0347, - 0x0348,0x0349,0x034A,0x034B,0x034C,0x034D,0x034E,0x034F, - 0x0350,0x0351,0x0352,0x0353,0x0354,0x0355,0x0356,0x0357, - 0x0358,0x0359,0x035A,0x035B,0x035C,0x035D,0x035E,0x035F, - 0x0360,0x0361,0x0362,0x0363,0x0364,0x0365,0x0366,0x0367, - 0x0368,0x0369,0x036A,0x036B,0x036C,0x036D,0x036E,0x036F, - 0x0483,0x0484,0x0485,0x0486,0x0487,0x0591,0x0592,0x0593, - 0x0594,0x0595,0x0596,0x0597,0x0598,0x0599,0x059A,0x059B, - 0x059C,0x059D,0x059E,0x059F,0x05A0,0x05A1,0x05A2,0x05A3, - 0x05A4,0x05A5,0x05A6,0x05A7,0x05A8,0x05A9,0x05AA,0x05AB, - 0x05AC,0x05AD,0x05AE,0x05AF,0x05B0,0x05B1,0x05B2,0x05B3, - 0x05B4,0x05B5,0x05B6,0x05B7,0x05B8,0x05B9,0x05BA,0x05BB, - 0x05BC,0x05BD,0x05BF,0x05C1,0x05C2,0x05C4,0x05C5,0x05C7, - 0x0610,0x0611,0x0612,0x0613,0x0614,0x0615,0x0616,0x0617, - 0x0618,0x0619,0x061A,0x064B,0x064C,0x064D,0x064E,0x064F, - 0x0650,0x0651,0x0652,0x0653,0x0654,0x0655,0x0656,0x0657, - 0x0658,0x0659,0x065A,0x065B,0x065C,0x065D,0x065E,0x065F, - 0x0670,0x06D6,0x06D7,0x06D8,0x06D9,0x06DA,0x06DB,0x06DC, - 0x06DF,0x06E0,0x06E1,0x06E2,0x06E3,0x06E4,0x06E7,0x06E8, - 0x06EA,0x06EB,0x06EC,0x06ED,0x0711,0x0730,0x0731,0x0732, - 0x0733,0x0734,0x0735,0x0736,0x0737,0x0738,0x0739,0x073A, - 0x073B,0x073C,0x073D,0x073E,0x073F,0x0740,0x0741,0x0742, - 0x0743,0x0744,0x0745,0x0746,0x0747,0x0748,0x0749,0x074A, - 0x07A6,0x07A7,0x07A8,0x07A9,0x07AA,0x07AB,0x07AC,0x07AD, - 0x07AE,0x07AF,0x07B0,0x07EB,0x07EC,0x07ED,0x07EE,0x07EF, - 0x07F0,0x07F1,0x07F2,0x07F3,0x0816,0x0817,0x0818,0x0819, - 0x081B,0x081C,0x081D,0x081E,0x081F,0x0820,0x0821,0x0822, - 0x0823,0x0825,0x0826,0x0827,0x0829,0x082A,0x082B,0x082C, - 0x082D,0x0859,0x085A,0x085B,0x08E3,0x08E4,0x08E5,0x08E6, - 0x08E7,0x08E8,0x08E9,0x08EA,0x08EB,0x08EC,0x08ED,0x08EE, - 0x08EF,0x08F0,0x08F1,0x08F2,0x08F3,0x08F4,0x08F5,0x08F6, - 0x08F7,0x08F8,0x08F9,0x08FA,0x08FB,0x08FC,0x08FD,0x08FE, - 0x08FF,0x0900,0x0901,0x0902,0x093A,0x093C,0x0941,0x0942, - 0x0943,0x0944,0x0945,0x0946,0x0947,0x0948,0x094D,0x0951, - 0x0952,0x0953,0x0954,0x0955,0x0956,0x0957,0x0962,0x0963, - 0x0981,0x09BC,0x09C1,0x09C2,0x09C3,0x09C4,0x09CD,0x09E2, - 0x09E3,0x0A01,0x0A02,0x0A3C,0x0A41,0x0A42,0x0A47,0x0A48, - 0x0A4B,0x0A4C,0x0A4D,0x0A51,0x0A70,0x0A71,0x0A75,0x0A81, - 0x0A82,0x0ABC,0x0AC1,0x0AC2,0x0AC3,0x0AC4,0x0AC5,0x0AC7, - 0x0AC8,0x0ACD,0x0AE2,0x0AE3,0x0B01,0x0B3C,0x0B3F,0x0B41, - 0x0B42,0x0B43,0x0B44,0x0B4D,0x0B56,0x0B62,0x0B63,0x0B82, - 0x0BC0,0x0BCD,0x0C00,0x0C3E,0x0C3F,0x0C40,0x0C46,0x0C47, - 0x0C48,0x0C4A,0x0C4B,0x0C4C,0x0C4D,0x0C55,0x0C56,0x0C62, - 0x0C63,0x0C81,0x0CBC,0x0CBF,0x0CC6,0x0CCC,0x0CCD,0x0CE2, - 0x0CE3,0x0D01,0x0D41,0x0D42,0x0D43,0x0D44,0x0D4D,0x0D62, - 0x0D63,0x0DCA,0x0DD2,0x0DD3,0x0DD4,0x0DD6,0x0E31,0x0E34, - 0x0E35,0x0E36,0x0E37,0x0E38,0x0E39,0x0E3A,0x0E47,0x0E48, - 0x0E49,0x0E4A,0x0E4B,0x0E4C,0x0E4D,0x0E4E,0x0EB1,0x0EB4, - 0x0EB5,0x0EB6,0x0EB7,0x0EB8,0x0EB9,0x0EBB,0x0EBC,0x0EC8, - 0x0EC9,0x0ECA,0x0ECB,0x0ECC,0x0ECD,0x0F18,0x0F19,0x0F35, - 0x0F37,0x0F39,0x0F71,0x0F72,0x0F73,0x0F74,0x0F75,0x0F76, - 0x0F77,0x0F78,0x0F79,0x0F7A,0x0F7B,0x0F7C,0x0F7D,0x0F7E, - 0x0F80,0x0F81,0x0F82,0x0F83,0x0F84,0x0F86,0x0F87,0x0F8D, - 0x0F8E,0x0F8F,0x0F90,0x0F91,0x0F92,0x0F93,0x0F94,0x0F95, - 0x0F96,0x0F97,0x0F99,0x0F9A,0x0F9B,0x0F9C,0x0F9D,0x0F9E, - 0x0F9F,0x0FA0,0x0FA1,0x0FA2,0x0FA3,0x0FA4,0x0FA5,0x0FA6, - 0x0FA7,0x0FA8,0x0FA9,0x0FAA,0x0FAB,0x0FAC,0x0FAD,0x0FAE, - 0x0FAF,0x0FB0,0x0FB1,0x0FB2,0x0FB3,0x0FB4,0x0FB5,0x0FB6, - 0x0FB7,0x0FB8,0x0FB9,0x0FBA,0x0FBB,0x0FBC,0x0FC6,0x102D, - 0x102E,0x102F,0x1030,0x1032,0x1033,0x1034,0x1035,0x1036, - 0x1037,0x1039,0x103A,0x103D,0x103E,0x1058,0x1059,0x105E, - 0x105F,0x1060,0x1071,0x1072,0x1073,0x1074,0x1082,0x1085, - 0x1086,0x108D,0x109D,0x135D,0x135E,0x135F,0x1712,0x1713, - 0x1714,0x1732,0x1733,0x1734,0x1752,0x1753,0x1772,0x1773, - 0x17B4,0x17B5,0x17B7,0x17B8,0x17B9,0x17BA,0x17BB,0x17BC, - 0x17BD,0x17C6,0x17C9,0x17CA,0x17CB,0x17CC,0x17CD,0x17CE, - 0x17CF,0x17D0,0x17D1,0x17D2,0x17D3,0x17DD,0x180B,0x180C, - 0x180D,0x18A9,0x1920,0x1921,0x1922,0x1927,0x1928,0x1932, - 0x1939,0x193A,0x193B,0x1A17,0x1A18,0x1A1B,0x1A56,0x1A58, - 0x1A59,0x1A5A,0x1A5B,0x1A5C,0x1A5D,0x1A5E,0x1A60,0x1A62, - 0x1A65,0x1A66,0x1A67,0x1A68,0x1A69,0x1A6A,0x1A6B,0x1A6C, - 0x1A73,0x1A74,0x1A75,0x1A76,0x1A77,0x1A78,0x1A79,0x1A7A, - 0x1A7B,0x1A7C,0x1A7F,0x1AB0,0x1AB1,0x1AB2,0x1AB3,0x1AB4, - 0x1AB5,0x1AB6,0x1AB7,0x1AB8,0x1AB9,0x1ABA,0x1ABB,0x1ABC, - 0x1ABD,0x1B00,0x1B01,0x1B02,0x1B03,0x1B34,0x1B36,0x1B37, - 0x1B38,0x1B39,0x1B3A,0x1B3C,0x1B42,0x1B6B,0x1B6C,0x1B6D, - 0x1B6E,0x1B6F,0x1B70,0x1B71,0x1B72,0x1B73,0x1B80,0x1B81, - 0x1BA2,0x1BA3,0x1BA4,0x1BA5,0x1BA8,0x1BA9,0x1BAB,0x1BAC, - 0x1BAD,0x1BE6,0x1BE8,0x1BE9,0x1BED,0x1BEF,0x1BF0,0x1BF1, - 0x1C2C,0x1C2D,0x1C2E,0x1C2F,0x1C30,0x1C31,0x1C32,0x1C33, - 0x1C36,0x1C37,0x1CD0,0x1CD1,0x1CD2,0x1CD4,0x1CD5,0x1CD6, - 0x1CD7,0x1CD8,0x1CD9,0x1CDA,0x1CDB,0x1CDC,0x1CDD,0x1CDE, - 0x1CDF,0x1CE0,0x1CE2,0x1CE3,0x1CE4,0x1CE5,0x1CE6,0x1CE7, - 0x1CE8,0x1CED,0x1CF4,0x1CF8,0x1CF9,0x1DC0,0x1DC1,0x1DC2, - 0x1DC3,0x1DC4,0x1DC5,0x1DC6,0x1DC7,0x1DC8,0x1DC9,0x1DCA, - 0x1DCB,0x1DCC,0x1DCD,0x1DCE,0x1DCF,0x1DD0,0x1DD1,0x1DD2, - 0x1DD3,0x1DD4,0x1DD5,0x1DD6,0x1DD7,0x1DD8,0x1DD9,0x1DDA, - 0x1DDB,0x1DDC,0x1DDD,0x1DDE,0x1DDF,0x1DE0,0x1DE1,0x1DE2, - 0x1DE3,0x1DE4,0x1DE5,0x1DE6,0x1DE7,0x1DE8,0x1DE9,0x1DEA, - 0x1DEB,0x1DEC,0x1DED,0x1DEE,0x1DEF,0x1DF0,0x1DF1,0x1DF2, - 0x1DF3,0x1DF4,0x1DF5,0x1DFC,0x1DFD,0x1DFE,0x1DFF,0x20D0, - 0x20D1,0x20D2,0x20D3,0x20D4,0x20D5,0x20D6,0x20D7,0x20D8, - 0x20D9,0x20DA,0x20DB,0x20DC,0x20E1,0x20E5,0x20E6,0x20E7, - 0x20E8,0x20E9,0x20EA,0x20EB,0x20EC,0x20ED,0x20EE,0x20EF, - 0x20F0,0x2CEF,0x2CF0,0x2CF1,0x2D7F,0x2DE0,0x2DE1,0x2DE2, - 0x2DE3,0x2DE4,0x2DE5,0x2DE6,0x2DE7,0x2DE8,0x2DE9,0x2DEA, - 0x2DEB,0x2DEC,0x2DED,0x2DEE,0x2DEF,0x2DF0,0x2DF1,0x2DF2, - 0x2DF3,0x2DF4,0x2DF5,0x2DF6,0x2DF7,0x2DF8,0x2DF9,0x2DFA, - 0x2DFB,0x2DFC,0x2DFD,0x2DFE,0x2DFF,0x302A,0x302B,0x302C, - 0x302D,0x3099,0x309A,0xA66F,0xA674,0xA675,0xA676,0xA677, - 0xA678,0xA679,0xA67A,0xA67B,0xA67C,0xA67D,0xA69E,0xA69F, - 0xA6F0,0xA6F1,0xA802,0xA806,0xA80B,0xA825,0xA826,0xA8C4, - 0xA8E0,0xA8E1,0xA8E2,0xA8E3,0xA8E4,0xA8E5,0xA8E6,0xA8E7, - 0xA8E8,0xA8E9,0xA8EA,0xA8EB,0xA8EC,0xA8ED,0xA8EE,0xA8EF, - 0xA8F0,0xA8F1,0xA926,0xA927,0xA928,0xA929,0xA92A,0xA92B, - 0xA92C,0xA92D,0xA947,0xA948,0xA949,0xA94A,0xA94B,0xA94C, - 0xA94D,0xA94E,0xA94F,0xA950,0xA951,0xA980,0xA981,0xA982, - 0xA9B3,0xA9B6,0xA9B7,0xA9B8,0xA9B9,0xA9BC,0xA9E5,0xAA29, - 0xAA2A,0xAA2B,0xAA2C,0xAA2D,0xAA2E,0xAA31,0xAA32,0xAA35, - 0xAA36,0xAA43,0xAA4C,0xAA7C,0xAAB0,0xAAB2,0xAAB3,0xAAB4, - 0xAAB7,0xAAB8,0xAABE,0xAABF,0xAAC1,0xAAEC,0xAAED,0xAAF6, - 0xABE5,0xABE8,0xABED,0xFB1E,0xFE00,0xFE01,0xFE02,0xFE03, - 0xFE04,0xFE05,0xFE06,0xFE07,0xFE08,0xFE09,0xFE0A,0xFE0B, - 0xFE0C,0xFE0D,0xFE0E,0xFE0F,0xFE20,0xFE21,0xFE22,0xFE23, - 0xFE24,0xFE25,0xFE26,0xFE27,0xFE28,0xFE29,0xFE2A,0xFE2B, - 0xFE2C,0xFE2D,0xFE2E,0xFE2F, - 0x101FD,0x102E0,0x10376,0x10377,0x10378,0x10379,0x1037A,0x10A01, - 0x10A02,0x10A03,0x10A05,0x10A06,0x10A0C,0x10A0D,0x10A0E,0x10A0F, - 0x10A38,0x10A39,0x10A3A,0x10A3F,0x10AE5,0x10AE6,0x11001,0x11038, - 0x11039,0x1103A,0x1103B,0x1103C,0x1103D,0x1103E,0x1103F,0x11040, - 0x11041,0x11042,0x11043,0x11044,0x11045,0x11046,0x1107F,0x11080, - 0x11081,0x110B3,0x110B4,0x110B5,0x110B6,0x110B9,0x110BA,0x11100, - 0x11101,0x11102,0x11127,0x11128,0x11129,0x1112A,0x1112B,0x1112D, - 0x1112E,0x1112F,0x11130,0x11131,0x11132,0x11133,0x11134,0x11173, - 0x11180,0x11181,0x111B6,0x111B7,0x111B8,0x111B9,0x111BA,0x111BB, - 0x111BC,0x111BD,0x111BE,0x111CA,0x111CB,0x111CC,0x1122F,0x11230, - 0x11231,0x11234,0x11236,0x11237,0x112DF,0x112E3,0x112E4,0x112E5, - 0x112E6,0x112E7,0x112E8,0x112E9,0x112EA,0x11300,0x11301,0x1133C, - 0x11340,0x11366,0x11367,0x11368,0x11369,0x1136A,0x1136B,0x1136C, - 0x11370,0x11371,0x11372,0x11373,0x11374,0x114B3,0x114B4,0x114B5, - 0x114B6,0x114B7,0x114B8,0x114BA,0x114BF,0x114C0,0x114C2,0x114C3, - 0x115B2,0x115B3,0x115B4,0x115B5,0x115BC,0x115BD,0x115BF,0x115C0, - 0x115DC,0x115DD,0x11633,0x11634,0x11635,0x11636,0x11637,0x11638, - 0x11639,0x1163A,0x1163D,0x1163F,0x11640,0x116AB,0x116AD,0x116B0, - 0x116B1,0x116B2,0x116B3,0x116B4,0x116B5,0x116B7,0x1171D,0x1171E, - 0x1171F,0x11722,0x11723,0x11724,0x11725,0x11727,0x11728,0x11729, - 0x1172A,0x1172B,0x16AF0,0x16AF1,0x16AF2,0x16AF3,0x16AF4,0x16B30, - 0x16B31,0x16B32,0x16B33,0x16B34,0x16B35,0x16B36,0x16F8F,0x16F90, - 0x16F91,0x16F92,0x1BC9D,0x1BC9E,0x1D167,0x1D168,0x1D169,0x1D17B, - 0x1D17C,0x1D17D,0x1D17E,0x1D17F,0x1D180,0x1D181,0x1D182,0x1D185, - 0x1D186,0x1D187,0x1D188,0x1D189,0x1D18A,0x1D18B,0x1D1AA,0x1D1AB, - 0x1D1AC,0x1D1AD,0x1D242,0x1D243,0x1D244,0x1DA00,0x1DA01,0x1DA02, - 0x1DA03,0x1DA04,0x1DA05,0x1DA06,0x1DA07,0x1DA08,0x1DA09,0x1DA0A, - 0x1DA0B,0x1DA0C,0x1DA0D,0x1DA0E,0x1DA0F,0x1DA10,0x1DA11,0x1DA12, - 0x1DA13,0x1DA14,0x1DA15,0x1DA16,0x1DA17,0x1DA18,0x1DA19,0x1DA1A, - 0x1DA1B,0x1DA1C,0x1DA1D,0x1DA1E,0x1DA1F,0x1DA20,0x1DA21,0x1DA22, - 0x1DA23,0x1DA24,0x1DA25,0x1DA26,0x1DA27,0x1DA28,0x1DA29,0x1DA2A, - 0x1DA2B,0x1DA2C,0x1DA2D,0x1DA2E,0x1DA2F,0x1DA30,0x1DA31,0x1DA32, - 0x1DA33,0x1DA34,0x1DA35,0x1DA36,0x1DA3B,0x1DA3C,0x1DA3D,0x1DA3E, - 0x1DA3F,0x1DA40,0x1DA41,0x1DA42,0x1DA43,0x1DA44,0x1DA45,0x1DA46, - 0x1DA47,0x1DA48,0x1DA49,0x1DA4A,0x1DA4B,0x1DA4C,0x1DA4D,0x1DA4E, - 0x1DA4F,0x1DA50,0x1DA51,0x1DA52,0x1DA53,0x1DA54,0x1DA55,0x1DA56, - 0x1DA57,0x1DA58,0x1DA59,0x1DA5A,0x1DA5B,0x1DA5C,0x1DA5D,0x1DA5E, - 0x1DA5F,0x1DA60,0x1DA61,0x1DA62,0x1DA63,0x1DA64,0x1DA65,0x1DA66, - 0x1DA67,0x1DA68,0x1DA69,0x1DA6A,0x1DA6B,0x1DA6C,0x1DA75,0x1DA84, - 0x1DA9B,0x1DA9C,0x1DA9D,0x1DA9E,0x1DA9F,0x1DAA1,0x1DAA2,0x1DAA3, - 0x1DAA4,0x1DAA5,0x1DAA6,0x1DAA7,0x1DAA8,0x1DAA9,0x1DAAA,0x1DAAB, - 0x1DAAC,0x1DAAD,0x1DAAE,0x1DAAF,0x1E8D0,0x1E8D1,0x1E8D2,0x1E8D3, - 0x1E8D4,0x1E8D5,0x1E8D6,0xE0100,0xE0101,0xE0102,0xE0103,0xE0104, - 0xE0105,0xE0106,0xE0107,0xE0108,0xE0109,0xE010A,0xE010B,0xE010C, - 0xE010D,0xE010E,0xE010F,0xE0110,0xE0111,0xE0112,0xE0113,0xE0114, - 0xE0115,0xE0116,0xE0117,0xE0118,0xE0119,0xE011A,0xE011B,0xE011C, - 0xE011D,0xE011E,0xE011F,0xE0120,0xE0121,0xE0122,0xE0123,0xE0124, - 0xE0125,0xE0126,0xE0127,0xE0128,0xE0129,0xE012A,0xE012B,0xE012C, - 0xE012D,0xE012E,0xE012F,0xE0130,0xE0131,0xE0132,0xE0133,0xE0134, - 0xE0135,0xE0136,0xE0137,0xE0138,0xE0139,0xE013A,0xE013B,0xE013C, - 0xE013D,0xE013E,0xE013F,0xE0140,0xE0141,0xE0142,0xE0143,0xE0144, - 0xE0145,0xE0146,0xE0147,0xE0148,0xE0149,0xE014A,0xE014B,0xE014C, - 0xE014D,0xE014E,0xE014F,0xE0150,0xE0151,0xE0152,0xE0153,0xE0154, - 0xE0155,0xE0156,0xE0157,0xE0158,0xE0159,0xE015A,0xE015B,0xE015C, - 0xE015D,0xE015E,0xE015F,0xE0160,0xE0161,0xE0162,0xE0163,0xE0164, - 0xE0165,0xE0166,0xE0167,0xE0168,0xE0169,0xE016A,0xE016B,0xE016C, - 0xE016D,0xE016E,0xE016F,0xE0170,0xE0171,0xE0172,0xE0173,0xE0174, - 0xE0175,0xE0176,0xE0177,0xE0178,0xE0179,0xE017A,0xE017B,0xE017C, - 0xE017D,0xE017E,0xE017F,0xE0180,0xE0181,0xE0182,0xE0183,0xE0184, - 0xE0185,0xE0186,0xE0187,0xE0188,0xE0189,0xE018A,0xE018B,0xE018C, - 0xE018D,0xE018E,0xE018F,0xE0190,0xE0191,0xE0192,0xE0193,0xE0194, - 0xE0195,0xE0196,0xE0197,0xE0198,0xE0199,0xE019A,0xE019B,0xE019C, - 0xE019D,0xE019E,0xE019F,0xE01A0,0xE01A1,0xE01A2,0xE01A3,0xE01A4, - 0xE01A5,0xE01A6,0xE01A7,0xE01A8,0xE01A9,0xE01AA,0xE01AB,0xE01AC, - 0xE01AD,0xE01AE,0xE01AF,0xE01B0,0xE01B1,0xE01B2,0xE01B3,0xE01B4, - 0xE01B5,0xE01B6,0xE01B7,0xE01B8,0xE01B9,0xE01BA,0xE01BB,0xE01BC, - 0xE01BD,0xE01BE,0xE01BF,0xE01C0,0xE01C1,0xE01C2,0xE01C3,0xE01C4, - 0xE01C5,0xE01C6,0xE01C7,0xE01C8,0xE01C9,0xE01CA,0xE01CB,0xE01CC, - 0xE01CD,0xE01CE,0xE01CF,0xE01D0,0xE01D1,0xE01D2,0xE01D3,0xE01D4, - 0xE01D5,0xE01D6,0xE01D7,0xE01D8,0xE01D9,0xE01DA,0xE01DB,0xE01DC, - 0xE01DD,0xE01DE,0xE01DF,0xE01E0,0xE01E1,0xE01E2,0xE01E3,0xE01E4, - 0xE01E5,0xE01E6,0xE01E7,0xE01E8,0xE01E9,0xE01EA,0xE01EB,0xE01EC, - 0xE01ED,0xE01EE,0xE01EF, -}; - -static int unicodeCombiningCharTableSize = sizeof(unicodeCombiningCharTable) / sizeof(unicodeCombiningCharTable[0]); - -inline int unicodeIsCombiningChar(unsigned long cp) -{ - int i; - for (i = 0; i < unicodeCombiningCharTableSize; i++) { - if (unicodeCombiningCharTable[i] == cp) { - return 1; - } - } - return 0; -} - -/* Get length of previous UTF8 character - */ -inline int unicodePrevUTF8CharLen(char* buf, int pos) -{ - int end = pos--; - while (pos >= 0 && ((unsigned char)buf[pos] & 0xC0) == 0x80) { - pos--; - } - return end - pos; -} - -/* Get length of previous UTF8 character - */ -inline int unicodeUTF8CharLen(char* buf, int buf_len, int pos) -{ - if (pos == buf_len) { return 0; } - unsigned char ch = buf[pos]; - if (ch < 0x80) { return 1; } - else if (ch < 0xE0) { return 2; } - else if (ch < 0xF0) { return 3; } - else { return 4; } -} - -/* Convert UTF8 to Unicode code point - */ -inline int unicodeUTF8CharToCodePoint( - const char* buf, - int len, - int* cp) -{ - if (len) { - unsigned char byte = buf[0]; - if ((byte & 0x80) == 0) { - *cp = byte; - return 1; - } else if ((byte & 0xE0) == 0xC0) { - if (len >= 2) { - *cp = (((unsigned long)(buf[0] & 0x1F)) << 6) | - ((unsigned long)(buf[1] & 0x3F)); - return 2; - } - } else if ((byte & 0xF0) == 0xE0) { - if (len >= 3) { - *cp = (((unsigned long)(buf[0] & 0x0F)) << 12) | - (((unsigned long)(buf[1] & 0x3F)) << 6) | - ((unsigned long)(buf[2] & 0x3F)); - return 3; - } - } else if ((byte & 0xF8) == 0xF0) { - if (len >= 4) { - *cp = (((unsigned long)(buf[0] & 0x07)) << 18) | - (((unsigned long)(buf[1] & 0x3F)) << 12) | - (((unsigned long)(buf[2] & 0x3F)) << 6) | - ((unsigned long)(buf[3] & 0x3F)); - return 4; - } - } - } - return 0; -} - -/* Get length of grapheme - */ -inline int unicodeGraphemeLen(char* buf, int buf_len, int pos) -{ - if (pos == buf_len) { - return 0; - } - int beg = pos; - pos += unicodeUTF8CharLen(buf, buf_len, pos); - while (pos < buf_len) { - int len = unicodeUTF8CharLen(buf, buf_len, pos); - int cp = 0; - unicodeUTF8CharToCodePoint(buf + pos, len, &cp); - if (!unicodeIsCombiningChar(cp)) { - return pos - beg; - } - pos += len; - } - return pos - beg; -} - -/* Get length of previous grapheme - */ -inline int unicodePrevGraphemeLen(char* buf, int pos) -{ - if (pos == 0) { - return 0; - } - int end = pos; - while (pos > 0) { - int len = unicodePrevUTF8CharLen(buf, pos); - pos -= len; - int cp = 0; - unicodeUTF8CharToCodePoint(buf + pos, len, &cp); - if (!unicodeIsCombiningChar(cp)) { - return end - pos; - } - } - return 0; -} - -inline int isAnsiEscape(const char* buf, int buf_len, int* len) -{ - if (buf_len > 2 && !memcmp("\033[", buf, 2)) { - int off = 2; - while (off < buf_len) { - switch (buf[off++]) { - case 'A': case 'B': case 'C': case 'D': - case 'E': case 'F': case 'G': case 'H': - case 'J': case 'K': case 'S': case 'T': - case 'f': case 'm': - *len = off; - return 1; - } - } - } - return 0; -} - -/* Get column position for the single line mode. - */ -inline int unicodeColumnPos(const char* buf, int buf_len) -{ - int ret = 0; - - int off = 0; - while (off < buf_len) { - int len; - if (isAnsiEscape(buf + off, buf_len - off, &len)) { - off += len; - continue; - } - - int cp = 0; - len = unicodeUTF8CharToCodePoint(buf + off, buf_len - off, &cp); - - if (!unicodeIsCombiningChar(cp)) { - ret += unicodeIsWideChar(cp) ? 2 : 1; - } - - off += len; - } - - return ret; -} - -/* Get column position for the multi line mode. - */ -inline int unicodeColumnPosForMultiLine(char* buf, int buf_len, int pos, int cols, int ini_pos) -{ - int ret = 0; - int colwid = ini_pos; - - int off = 0; - while (off < buf_len) { - int cp = 0; - int len = unicodeUTF8CharToCodePoint(buf + off, buf_len - off, &cp); - - int wid = 0; - if (!unicodeIsCombiningChar(cp)) { - wid = unicodeIsWideChar(cp) ? 2 : 1; - } - - int dif = (int)(colwid + wid) - (int)cols; - if (dif > 0) { - ret += dif; - colwid = wid; - } else if (dif == 0) { - colwid = 0; - } else { - colwid += wid; - } - - if (off >= pos) { - break; - } - - off += len; - ret += wid; - } - - return ret; -} - -/* Read UTF8 character from file. - */ -inline int unicodeReadUTF8Char(int fd, char* buf, int* cp) -{ - int nread = read(fd,&buf[0],1); - - if (nread <= 0) { return nread; } - - unsigned char byte = buf[0]; - - if ((byte & 0x80) == 0) { - ; - } else if ((byte & 0xE0) == 0xC0) { - nread = read(fd,&buf[1],1); - if (nread <= 0) { return nread; } - } else if ((byte & 0xF0) == 0xE0) { - nread = read(fd,&buf[1],2); - if (nread <= 0) { return nread; } - } else if ((byte & 0xF8) == 0xF0) { - nread = read(fd,&buf[1],3); - if (nread <= 0) { return nread; } - } else { - return -1; - } - - return unicodeUTF8CharToCodePoint(buf, 4, cp); -} - -/* ======================= Low level terminal handling ====================== */ - -/* Set if to use or not the multi line mode. */ -inline void SetMultiLine(bool ml) { - mlmode = ml; -} - -/* Return true if the terminal name is in the list of terminals we know are - * not able to understand basic escape sequences. */ -inline bool isUnsupportedTerm(void) { -#ifndef _WIN32 - char *term = getenv("TERM"); - int j; - - if (term == NULL) return false; - for (j = 0; unsupported_term[j]; j++) - if (!strcasecmp(term,unsupported_term[j])) return true; -#endif - return false; -} - -/* Raw mode: 1960 magic shit. */ -inline bool enableRawMode(int fd) { -#ifndef _WIN32 - struct termios raw; - - if (!isatty(STDIN_FILENO)) goto fatal; - if (!atexit_registered) { - atexit(linenoiseAtExit); - atexit_registered = true; - } - if (tcgetattr(fd,&orig_termios) == -1) goto fatal; - - raw = orig_termios; /* modify the original mode */ - /* input modes: no break, no CR to NL, no parity check, no strip char, - * no start/stop output control. */ - raw.c_iflag &= ~(BRKINT | ICRNL | INPCK | ISTRIP | IXON); - /* output modes - disable post processing */ - // NOTE: Multithreaded issue #20 (https://github.com/yhirose/cpp-linenoise/issues/20) - // raw.c_oflag &= ~(OPOST); - /* control modes - set 8 bit chars */ - raw.c_cflag |= (CS8); - /* local modes - echoing off, canonical off, no extended functions, - * no signal chars (^Z,^C) */ - raw.c_lflag &= ~(ECHO | ICANON | IEXTEN | ISIG); - /* control chars - set return condition: min number of bytes and timer. - * We want read to return every single byte, without timeout. */ - raw.c_cc[VMIN] = 1; raw.c_cc[VTIME] = 0; /* 1 byte, no timer */ - - /* put terminal in raw mode after flushing */ - if (tcsetattr(fd,TCSAFLUSH,&raw) < 0) goto fatal; - rawmode = true; -#else - if (!atexit_registered) { - /* Cleanup them at exit */ - atexit(linenoiseAtExit); - atexit_registered = true; - - /* Init windows console handles only once */ - hOut = GetStdHandle(STD_OUTPUT_HANDLE); - if (hOut==INVALID_HANDLE_VALUE) goto fatal; - } - - DWORD consolemodeOut; - if (!GetConsoleMode(hOut, &consolemodeOut)) { - CloseHandle(hOut); - errno = ENOTTY; - return false; - }; - - hIn = GetStdHandle(STD_INPUT_HANDLE); - if (hIn == INVALID_HANDLE_VALUE) { - CloseHandle(hOut); - errno = ENOTTY; - return false; - } - - GetConsoleMode(hIn, &consolemodeIn); - /* Enable raw mode */ - SetConsoleMode(hIn, consolemodeIn & ~ENABLE_PROCESSED_INPUT); - - rawmode = true; -#endif - return true; - -fatal: - errno = ENOTTY; - return false; -} - -inline void disableRawMode(int fd) { -#ifdef _WIN32 - if (consolemodeIn) { - SetConsoleMode(hIn, consolemodeIn); - consolemodeIn = 0; - } - rawmode = false; -#else - /* Don't even check the return value as it's too late. */ - if (rawmode && tcsetattr(fd,TCSAFLUSH,&orig_termios) != -1) - rawmode = false; -#endif -} - -/* Use the ESC [6n escape sequence to query the horizontal cursor position - * and return it. On error -1 is returned, on success the position of the - * cursor. */ -inline int getCursorPosition(int ifd, int ofd) { - char buf[32]; - int cols, rows; - unsigned int i = 0; - - /* Report cursor location */ - if (write(ofd, "\x1b[6n", 4) != 4) return -1; - - /* Read the response: ESC [ rows ; cols R */ - while (i < sizeof(buf)-1) { - if (read(ifd,buf+i,1) != 1) break; - if (buf[i] == 'R') break; - i++; - } - buf[i] = '\0'; - - /* Parse it. */ - if (buf[0] != ESC || buf[1] != '[') return -1; - if (sscanf(buf+2,"%d;%d",&rows,&cols) != 2) return -1; - return cols; -} - -/* Try to get the number of columns in the current terminal, or assume 80 - * if it fails. */ -inline int getColumns(int ifd, int ofd) { -#ifdef _WIN32 - CONSOLE_SCREEN_BUFFER_INFO b; - - if (!GetConsoleScreenBufferInfo(hOut, &b)) return 80; - return b.srWindow.Right - b.srWindow.Left; -#else - struct winsize ws; - - if (ioctl(1, TIOCGWINSZ, &ws) == -1 || ws.ws_col == 0) { - /* ioctl() failed. Try to query the terminal itself. */ - int start, cols; - - /* Get the initial position so we can restore it later. */ - start = getCursorPosition(ifd,ofd); - if (start == -1) goto failed; - - /* Go to right margin and get position. */ - if (write(ofd,"\x1b[999C",6) != 6) goto failed; - cols = getCursorPosition(ifd,ofd); - if (cols == -1) goto failed; - - /* Restore position. */ - if (cols > start) { - char seq[32]; - snprintf(seq,32,"\x1b[%dD",cols-start); - if (write(ofd,seq,strlen(seq)) == -1) { - /* Can't recover... */ - } - } - return cols; - } else { - return ws.ws_col; - } - -failed: - return 80; -#endif -} - -/* Clear the screen. Used to handle ctrl+l */ -inline void linenoiseClearScreen(void) { - if (write(STDOUT_FILENO,"\x1b[H\x1b[2J",7) <= 0) { - /* nothing to do, just to avoid warning. */ - } -} - -/* Beep, used for completion when there is nothing to complete or when all - * the choices were already shown. */ -inline void linenoiseBeep(void) { - fprintf(stderr, "\x7"); - fflush(stderr); -} - -/* ============================== Completion ================================ */ - -/* This is an helper function for linenoiseEdit() and is called when the - * user types the key in order to complete the string currently in the - * input. - * - * The state of the editing is encapsulated into the pointed linenoiseState - * structure as described in the structure definition. */ -inline int completeLine(struct linenoiseState *ls, char *cbuf, int *c) { - std::vector lc; - int nread = 0, nwritten; - *c = 0; - - completionCallback(ls->buf,lc); - if (lc.empty()) { - linenoiseBeep(); - } else { - int stop = 0, i = 0; - - while(!stop) { - /* Show completion or original buffer */ - if (i < static_cast(lc.size())) { - struct linenoiseState saved = *ls; - - ls->len = ls->pos = static_cast(lc[i].size()); - ls->buf = &lc[i][0]; - refreshLine(ls); - ls->len = saved.len; - ls->pos = saved.pos; - ls->buf = saved.buf; - } else { - refreshLine(ls); - } - - //nread = read(ls->ifd,&c,1); -#ifdef _WIN32 - nread = win32read(c); - if (nread == 1) { - cbuf[0] = *c; - } -#else - nread = unicodeReadUTF8Char(ls->ifd,cbuf,c); -#endif - if (nread <= 0) { - *c = -1; - return nread; - } - - switch(*c) { - case 9: /* tab */ - i = (i+1) % (lc.size()+1); - if (i == static_cast(lc.size())) linenoiseBeep(); - break; - case 27: /* escape */ - /* Re-show original buffer */ - if (i < static_cast(lc.size())) refreshLine(ls); - stop = 1; - break; - default: - /* Update buffer and return */ - if (i < static_cast(lc.size())) { - nwritten = snprintf(ls->buf,ls->buflen,"%s",&lc[i][0]); - ls->len = ls->pos = nwritten; - } - stop = 1; - break; - } - } - } - - return nread; -} - -/* Register a callback function to be called for tab-completion. */ -inline void SetCompletionCallback(CompletionCallback fn) { - completionCallback = fn; -} - -/* =========================== Line editing ================================= */ - -/* Single line low level line refresh. - * - * Rewrite the currently edited line accordingly to the buffer content, - * cursor position, and number of columns of the terminal. */ -inline void refreshSingleLine(struct linenoiseState *l) { - char seq[64]; - int pcolwid = unicodeColumnPos(l->prompt.c_str(), static_cast(l->prompt.length())); - int fd = l->ofd; - char *buf = l->buf; - int len = l->len; - int pos = l->pos; - std::string ab; - - while((pcolwid+unicodeColumnPos(buf, pos)) >= l->cols) { - int glen = unicodeGraphemeLen(buf, len, 0); - buf += glen; - len -= glen; - pos -= glen; - } - while (pcolwid+unicodeColumnPos(buf, len) > l->cols) { - len -= unicodePrevGraphemeLen(buf, len); - } - - /* Cursor to left edge */ - snprintf(seq,64,"\r"); - ab += seq; - /* Write the prompt and the current buffer content */ - ab += l->prompt; - ab.append(buf, len); - /* Erase to right */ - snprintf(seq,64,"\x1b[0K"); - ab += seq; - /* Move cursor to original position. */ - snprintf(seq,64,"\r\x1b[%dC", (int)(unicodeColumnPos(buf, pos)+pcolwid)); - ab += seq; - if (write(fd,ab.c_str(), static_cast(ab.length())) == -1) {} /* Can't recover from write error. */ -} - -/* Multi line low level line refresh. - * - * Rewrite the currently edited line accordingly to the buffer content, - * cursor position, and number of columns of the terminal. */ -inline void refreshMultiLine(struct linenoiseState *l) { - char seq[64]; - int pcolwid = unicodeColumnPos(l->prompt.c_str(), static_cast(l->prompt.length())); - int colpos = unicodeColumnPosForMultiLine(l->buf, l->len, l->len, l->cols, pcolwid); - int colpos2; /* cursor column position. */ - int rows = (pcolwid+colpos+l->cols-1)/l->cols; /* rows used by current buf. */ - int rpos = (pcolwid+l->oldcolpos+l->cols)/l->cols; /* cursor relative row. */ - int rpos2; /* rpos after refresh. */ - int col; /* colum position, zero-based. */ - int old_rows = (int)l->maxrows; - int fd = l->ofd, j; - std::string ab; - - /* Update maxrows if needed. */ - if (rows > (int)l->maxrows) l->maxrows = rows; - - /* First step: clear all the lines used before. To do so start by - * going to the last row. */ - if (old_rows-rpos > 0) { - snprintf(seq,64,"\x1b[%dB", old_rows-rpos); - ab += seq; - } - - /* Now for every row clear it, go up. */ - for (j = 0; j < old_rows-1; j++) { - snprintf(seq,64,"\r\x1b[0K\x1b[1A"); - ab += seq; - } - - /* Clean the top line. */ - snprintf(seq,64,"\r\x1b[0K"); - ab += seq; - - /* Write the prompt and the current buffer content */ - ab += l->prompt; - ab.append(l->buf, l->len); - - /* Get text width to cursor position */ - colpos2 = unicodeColumnPosForMultiLine(l->buf, l->len, l->pos, l->cols, pcolwid); - - /* If we are at the very end of the screen with our prompt, we need to - * emit a newline and move the prompt to the first column. */ - if (l->pos && - l->pos == l->len && - (colpos2+pcolwid) % l->cols == 0) - { - ab += "\n"; - snprintf(seq,64,"\r"); - ab += seq; - rows++; - if (rows > (int)l->maxrows) l->maxrows = rows; - } - - /* Move cursor to right position. */ - rpos2 = (pcolwid+colpos2+l->cols)/l->cols; /* current cursor relative row. */ - - /* Go up till we reach the expected positon. */ - if (rows-rpos2 > 0) { - snprintf(seq,64,"\x1b[%dA", rows-rpos2); - ab += seq; - } - - /* Set column. */ - col = (pcolwid + colpos2) % l->cols; - if (col) - snprintf(seq,64,"\r\x1b[%dC", col); - else - snprintf(seq,64,"\r"); - ab += seq; - - l->oldcolpos = colpos2; - - if (write(fd,ab.c_str(), static_cast(ab.length())) == -1) {} /* Can't recover from write error. */ -} - -/* Calls the two low level functions refreshSingleLine() or - * refreshMultiLine() according to the selected mode. */ -inline void refreshLine(struct linenoiseState *l) { - if (mlmode) - refreshMultiLine(l); - else - refreshSingleLine(l); -} - -/* Insert the character 'c' at cursor current position. - * - * On error writing to the terminal -1 is returned, otherwise 0. */ -inline int linenoiseEditInsert(struct linenoiseState *l, const char* cbuf, int clen) { - if (l->len < l->buflen) { - if (l->len == l->pos) { - memcpy(&l->buf[l->pos],cbuf,clen); - l->pos+=clen; - l->len+=clen;; - l->buf[l->len] = '\0'; - if ((!mlmode && unicodeColumnPos(l->prompt.c_str(), static_cast(l->prompt.length()))+unicodeColumnPos(l->buf,l->len) < l->cols) /* || mlmode */) { - /* Avoid a full update of the line in the - * trivial case. */ - if (write(l->ofd,cbuf,clen) == -1) return -1; - } else { - refreshLine(l); - } - } else { - memmove(l->buf+l->pos+clen,l->buf+l->pos,l->len-l->pos); - memcpy(&l->buf[l->pos],cbuf,clen); - l->pos+=clen; - l->len+=clen; - l->buf[l->len] = '\0'; - refreshLine(l); - } - } - return 0; -} - -/* Move cursor on the left. */ -inline void linenoiseEditMoveLeft(struct linenoiseState *l) { - if (l->pos > 0) { - l->pos -= unicodePrevGraphemeLen(l->buf, l->pos); - refreshLine(l); - } -} - -/* Move cursor on the right. */ -inline void linenoiseEditMoveRight(struct linenoiseState *l) { - if (l->pos != l->len) { - l->pos += unicodeGraphemeLen(l->buf, l->len, l->pos); - refreshLine(l); - } -} - -/* Move cursor to the start of the line. */ -inline void linenoiseEditMoveHome(struct linenoiseState *l) { - if (l->pos != 0) { - l->pos = 0; - refreshLine(l); - } -} - -/* Move cursor to the end of the line. */ -inline void linenoiseEditMoveEnd(struct linenoiseState *l) { - if (l->pos != l->len) { - l->pos = l->len; - refreshLine(l); - } -} - -/* Substitute the currently edited line with the next or previous history - * entry as specified by 'dir'. */ -#define LINENOISE_HISTORY_NEXT 0 -#define LINENOISE_HISTORY_PREV 1 -inline void linenoiseEditHistoryNext(struct linenoiseState *l, int dir) { - if (history.size() > 1) { - /* Update the current history entry before to - * overwrite it with the next one. */ - history[history.size() - 1 - l->history_index] = l->buf; - /* Show the new entry */ - l->history_index += (dir == LINENOISE_HISTORY_PREV) ? 1 : -1; - if (l->history_index < 0) { - l->history_index = 0; - return; - } else if (l->history_index >= (int)history.size()) { - l->history_index = static_cast(history.size())-1; - return; - } - memset(l->buf, 0, l->buflen); - strcpy(l->buf,history[history.size() - 1 - l->history_index].c_str()); - l->len = l->pos = static_cast(strlen(l->buf)); - refreshLine(l); - } -} - -/* Delete the character at the right of the cursor without altering the cursor - * position. Basically this is what happens with the "Delete" keyboard key. */ -inline void linenoiseEditDelete(struct linenoiseState *l) { - if (l->len > 0 && l->pos < l->len) { - int glen = unicodeGraphemeLen(l->buf,l->len,l->pos); - memmove(l->buf+l->pos,l->buf+l->pos+glen,l->len-l->pos-glen); - l->len-=glen; - l->buf[l->len] = '\0'; - refreshLine(l); - } -} - -/* Backspace implementation. */ -inline void linenoiseEditBackspace(struct linenoiseState *l) { - if (l->pos > 0 && l->len > 0) { - int glen = unicodePrevGraphemeLen(l->buf,l->pos); - memmove(l->buf+l->pos-glen,l->buf+l->pos,l->len-l->pos); - l->pos-=glen; - l->len-=glen; - l->buf[l->len] = '\0'; - refreshLine(l); - } -} - -/* Delete the previosu word, maintaining the cursor at the start of the - * current word. */ -inline void linenoiseEditDeletePrevWord(struct linenoiseState *l) { - int old_pos = l->pos; - int diff; - - while (l->pos > 0 && l->buf[l->pos-1] == ' ') - l->pos--; - while (l->pos > 0 && l->buf[l->pos-1] != ' ') - l->pos--; - diff = old_pos - l->pos; - memmove(l->buf+l->pos,l->buf+old_pos,l->len-old_pos+1); - l->len -= diff; - refreshLine(l); -} - -/* This function is the core of the line editing capability of linenoise. - * It expects 'fd' to be already in "raw mode" so that every key pressed - * will be returned ASAP to read(). - * - * The resulting string is put into 'buf' when the user type enter, or - * when ctrl+d is typed. - * - * The function returns the length of the current buffer. */ -inline int linenoiseEdit(int stdin_fd, int stdout_fd, char *buf, int buflen, const char *prompt) -{ - struct linenoiseState l; - - /* Populate the linenoise state that we pass to functions implementing - * specific editing functionalities. */ - l.ifd = stdin_fd; - l.ofd = stdout_fd; - l.buf = buf; - l.buflen = buflen; - l.prompt = prompt; - l.oldcolpos = l.pos = 0; - l.len = 0; - l.cols = getColumns(stdin_fd, stdout_fd); - l.maxrows = 0; - l.history_index = 0; - - /* Buffer starts empty. */ - l.buf[0] = '\0'; - l.buflen--; /* Make sure there is always space for the nulterm */ - - /* The latest history entry is always our current buffer, that - * initially is just an empty string. */ - AddHistory(""); - - if (write(l.ofd,prompt, static_cast(l.prompt.length())) == -1) return -1; - while(1) { - int c; - char cbuf[4]; - int nread; - char seq[3]; - -#ifdef _WIN32 - nread = win32read(&c); - if (nread == 1) { - cbuf[0] = c; - } -#else - nread = unicodeReadUTF8Char(l.ifd,cbuf,&c); -#endif - if (nread <= 0) return (int)l.len; - - /* Only autocomplete when the callback is set. It returns < 0 when - * there was an error reading from fd. Otherwise it will return the - * character that should be handled next. */ - if (c == 9 && completionCallback != NULL) { - nread = completeLine(&l,cbuf,&c); - /* Return on errors */ - if (c < 0) return l.len; - /* Read next character when 0 */ - if (c == 0) continue; - } - - switch(c) { - case ENTER: /* enter */ - if (!history.empty()) history.pop_back(); - if (mlmode) linenoiseEditMoveEnd(&l); - return (int)l.len; - case CTRL_C: /* ctrl-c */ - errno = EAGAIN; - return -1; - case BACKSPACE: /* backspace */ - case 8: /* ctrl-h */ - linenoiseEditBackspace(&l); - break; - case CTRL_D: /* ctrl-d, remove char at right of cursor, or if the - line is empty, act as end-of-file. */ - if (l.len > 0) { - linenoiseEditDelete(&l); - } else { - history.pop_back(); - return -1; - } - break; - case CTRL_T: /* ctrl-t, swaps current character with previous. */ - if (l.pos > 0 && l.pos < l.len) { - char aux = buf[l.pos-1]; - buf[l.pos-1] = buf[l.pos]; - buf[l.pos] = aux; - if (l.pos != l.len-1) l.pos++; - refreshLine(&l); - } - break; - case CTRL_B: /* ctrl-b */ - linenoiseEditMoveLeft(&l); - break; - case CTRL_F: /* ctrl-f */ - linenoiseEditMoveRight(&l); - break; - case CTRL_P: /* ctrl-p */ - linenoiseEditHistoryNext(&l, LINENOISE_HISTORY_PREV); - break; - case CTRL_N: /* ctrl-n */ - linenoiseEditHistoryNext(&l, LINENOISE_HISTORY_NEXT); - break; - case ESC: /* escape sequence */ - /* Read the next two bytes representing the escape sequence. - * Use two calls to handle slow terminals returning the two - * chars at different times. */ - if (read(l.ifd,seq,1) == -1) break; - if (read(l.ifd,seq+1,1) == -1) break; - - /* ESC [ sequences. */ - if (seq[0] == '[') { - if (seq[1] >= '0' && seq[1] <= '9') { - /* Extended escape, read additional byte. */ - if (read(l.ifd,seq+2,1) == -1) break; - if (seq[2] == '~') { - switch(seq[1]) { - case '3': /* Delete key. */ - linenoiseEditDelete(&l); - break; - } - } - } else { - switch(seq[1]) { - case 'A': /* Up */ - linenoiseEditHistoryNext(&l, LINENOISE_HISTORY_PREV); - break; - case 'B': /* Down */ - linenoiseEditHistoryNext(&l, LINENOISE_HISTORY_NEXT); - break; - case 'C': /* Right */ - linenoiseEditMoveRight(&l); - break; - case 'D': /* Left */ - linenoiseEditMoveLeft(&l); - break; - case 'H': /* Home */ - linenoiseEditMoveHome(&l); - break; - case 'F': /* End*/ - linenoiseEditMoveEnd(&l); - break; - } - } - } - - /* ESC O sequences. */ - else if (seq[0] == 'O') { - switch(seq[1]) { - case 'H': /* Home */ - linenoiseEditMoveHome(&l); - break; - case 'F': /* End*/ - linenoiseEditMoveEnd(&l); - break; - } - } - break; - default: - if (linenoiseEditInsert(&l,cbuf,nread)) return -1; - break; - case CTRL_U: /* Ctrl+u, delete the whole line. */ - buf[0] = '\0'; - l.pos = l.len = 0; - refreshLine(&l); - break; - case CTRL_K: /* Ctrl+k, delete from current to end of line. */ - buf[l.pos] = '\0'; - l.len = l.pos; - refreshLine(&l); - break; - case CTRL_A: /* Ctrl+a, go to the start of the line */ - linenoiseEditMoveHome(&l); - break; - case CTRL_E: /* ctrl+e, go to the end of the line */ - linenoiseEditMoveEnd(&l); - break; - case CTRL_L: /* ctrl+l, clear screen */ - linenoiseClearScreen(); - refreshLine(&l); - break; - case CTRL_W: /* ctrl+w, delete previous word */ - linenoiseEditDeletePrevWord(&l); - break; - } - } - return l.len; -} - -/* This function calls the line editing function linenoiseEdit() using - * the STDIN file descriptor set in raw mode. */ -inline bool linenoiseRaw(const char *prompt, std::string& line) { - bool quit = false; - - if (!isatty(STDIN_FILENO)) { - /* Not a tty: read from file / pipe. */ - std::getline(std::cin, line); - } else { - /* Interactive editing. */ - if (enableRawMode(STDIN_FILENO) == false) { - return quit; - } - - char buf[LINENOISE_MAX_LINE]; - auto count = linenoiseEdit(STDIN_FILENO, STDOUT_FILENO, buf, LINENOISE_MAX_LINE, prompt); - if (count == -1) { - quit = true; - } else { - line.assign(buf, count); - } - - disableRawMode(STDIN_FILENO); - printf("\n"); - } - return quit; -} - -/* The high level function that is the main API of the linenoise library. - * This function checks if the terminal has basic capabilities, just checking - * for a blacklist of stupid terminals, and later either calls the line - * editing function or uses dummy fgets() so that you will be able to type - * something even in the most desperate of the conditions. */ -inline bool Readline(const char *prompt, std::string& line) { - if (isUnsupportedTerm()) { - printf("%s",prompt); - fflush(stdout); - std::getline(std::cin, line); - return false; - } else { - return linenoiseRaw(prompt, line); - } -} - -inline std::string Readline(const char *prompt, bool& quit) { - std::string line; - quit = Readline(prompt, line); - return line; -} - -inline std::string Readline(const char *prompt) { - bool quit; // dummy - return Readline(prompt, quit); -} - -/* ================================ History ================================= */ - -/* At exit we'll try to fix the terminal to the initial conditions. */ -inline void linenoiseAtExit(void) { - disableRawMode(STDIN_FILENO); -} - -/* This is the API call to add a new entry in the linenoise history. - * It uses a fixed array of char pointers that are shifted (memmoved) - * when the history max length is reached in order to remove the older - * entry and make room for the new one, so it is not exactly suitable for huge - * histories, but will work well for a few hundred of entries. - * - * Using a circular buffer is smarter, but a bit more complex to handle. */ -inline bool AddHistory(const char* line) { - if (history_max_len == 0) return false; - - /* Don't add duplicated lines. */ - if (!history.empty() && history.back() == line) return false; - - /* If we reached the max length, remove the older line. */ - if (history.size() == history_max_len) { - history.erase(history.begin()); - } - history.push_back(line); - - return true; -} - -/* Set the maximum length for the history. This function can be called even - * if there is already some history, the function will make sure to retain - * just the latest 'len' elements if the new history length value is smaller - * than the amount of items already inside the history. */ -inline bool SetHistoryMaxLen(size_t len) { - if (len < 1) return false; - history_max_len = len; - if (len < history.size()) { - history.resize(len); - } - return true; -} - -/* Save the history in the specified file. On success *true* is returned - * otherwise *false* is returned. */ -inline bool SaveHistory(const char* path) { - std::ofstream f(path); // TODO: need 'std::ios::binary'? - if (!f) return false; - for (const auto& h: history) { - f << h << std::endl; - } - return true; -} - -/* Load the history from the specified file. If the file does not exist - * zero is returned and no operation is performed. - * - * If the file exists and the operation succeeded *true* is returned, otherwise - * on error *false* is returned. */ -inline bool LoadHistory(const char* path) { - std::ifstream f(path); - if (!f) return false; - std::string line; - while (std::getline(f, line)) { - AddHistory(line.c_str()); - } - return true; -} - -inline const std::vector& GetHistory() { - return history; -} - -} // namespace linenoise - -#ifdef _WIN32 -#undef isatty -#undef write -#undef read -#pragma warning(pop) -#endif - -#endif /* __LINENOISE_HPP */ diff --git a/tests/Autocomplete.test.cpp b/tests/Autocomplete.test.cpp index e8e3b315..59e12574 100644 --- a/tests/Autocomplete.test.cpp +++ b/tests/Autocomplete.test.cpp @@ -2536,7 +2536,7 @@ TEST_CASE("autocomplete_documentation_symbols") TEST_CASE_FIXTURE(ACFixture, "autocomplete_ifelse_expressions") { - check(R"( + check(R"( local temp = false local even = true; local a = true @@ -2551,63 +2551,63 @@ a = if temp then even elseif true then temp e@8 a = if temp then even elseif true then temp else e@9 )"); - auto ac = autocomplete('1'); - CHECK(ac.entryMap.count("temp")); - CHECK(ac.entryMap.count("true")); - CHECK(ac.entryMap.count("then") == 0); - CHECK(ac.entryMap.count("else") == 0); - CHECK(ac.entryMap.count("elseif") == 0); + auto ac = autocomplete('1'); + CHECK(ac.entryMap.count("temp")); + CHECK(ac.entryMap.count("true")); + CHECK(ac.entryMap.count("then") == 0); + CHECK(ac.entryMap.count("else") == 0); + CHECK(ac.entryMap.count("elseif") == 0); - ac = autocomplete('2'); - CHECK(ac.entryMap.count("temp") == 0); - CHECK(ac.entryMap.count("true") == 0); - CHECK(ac.entryMap.count("then")); - CHECK(ac.entryMap.count("else") == 0); - CHECK(ac.entryMap.count("elseif") == 0); + ac = autocomplete('2'); + CHECK(ac.entryMap.count("temp") == 0); + CHECK(ac.entryMap.count("true") == 0); + CHECK(ac.entryMap.count("then")); + CHECK(ac.entryMap.count("else") == 0); + CHECK(ac.entryMap.count("elseif") == 0); - ac = autocomplete('3'); - CHECK(ac.entryMap.count("even")); - CHECK(ac.entryMap.count("then") == 0); - CHECK(ac.entryMap.count("else") == 0); - CHECK(ac.entryMap.count("elseif") == 0); + ac = autocomplete('3'); + CHECK(ac.entryMap.count("even")); + CHECK(ac.entryMap.count("then") == 0); + CHECK(ac.entryMap.count("else") == 0); + CHECK(ac.entryMap.count("elseif") == 0); - ac = autocomplete('4'); - CHECK(ac.entryMap.count("even") == 0); - CHECK(ac.entryMap.count("then") == 0); - CHECK(ac.entryMap.count("else")); - CHECK(ac.entryMap.count("elseif")); + ac = autocomplete('4'); + CHECK(ac.entryMap.count("even") == 0); + CHECK(ac.entryMap.count("then") == 0); + CHECK(ac.entryMap.count("else")); + CHECK(ac.entryMap.count("elseif")); - ac = autocomplete('5'); - CHECK(ac.entryMap.count("temp")); - CHECK(ac.entryMap.count("true")); - CHECK(ac.entryMap.count("then") == 0); - CHECK(ac.entryMap.count("else") == 0); - CHECK(ac.entryMap.count("elseif") == 0); + ac = autocomplete('5'); + CHECK(ac.entryMap.count("temp")); + CHECK(ac.entryMap.count("true")); + CHECK(ac.entryMap.count("then") == 0); + CHECK(ac.entryMap.count("else") == 0); + CHECK(ac.entryMap.count("elseif") == 0); - ac = autocomplete('6'); - CHECK(ac.entryMap.count("temp") == 0); - CHECK(ac.entryMap.count("true") == 0); - CHECK(ac.entryMap.count("then")); - CHECK(ac.entryMap.count("else") == 0); - CHECK(ac.entryMap.count("elseif") == 0); + ac = autocomplete('6'); + CHECK(ac.entryMap.count("temp") == 0); + CHECK(ac.entryMap.count("true") == 0); + CHECK(ac.entryMap.count("then")); + CHECK(ac.entryMap.count("else") == 0); + CHECK(ac.entryMap.count("elseif") == 0); - ac = autocomplete('7'); - CHECK(ac.entryMap.count("temp")); - CHECK(ac.entryMap.count("true")); - CHECK(ac.entryMap.count("then") == 0); - CHECK(ac.entryMap.count("else") == 0); - CHECK(ac.entryMap.count("elseif") == 0); + ac = autocomplete('7'); + CHECK(ac.entryMap.count("temp")); + CHECK(ac.entryMap.count("true")); + CHECK(ac.entryMap.count("then") == 0); + CHECK(ac.entryMap.count("else") == 0); + CHECK(ac.entryMap.count("elseif") == 0); - ac = autocomplete('8'); - CHECK(ac.entryMap.count("even") == 0); - CHECK(ac.entryMap.count("then") == 0); - CHECK(ac.entryMap.count("else")); - CHECK(ac.entryMap.count("elseif")); + ac = autocomplete('8'); + CHECK(ac.entryMap.count("even") == 0); + CHECK(ac.entryMap.count("then") == 0); + CHECK(ac.entryMap.count("else")); + CHECK(ac.entryMap.count("elseif")); - ac = autocomplete('9'); - CHECK(ac.entryMap.count("then") == 0); - CHECK(ac.entryMap.count("else") == 0); - CHECK(ac.entryMap.count("elseif") == 0); + ac = autocomplete('9'); + CHECK(ac.entryMap.count("then") == 0); + CHECK(ac.entryMap.count("else") == 0); + CHECK(ac.entryMap.count("elseif") == 0); } TEST_CASE_FIXTURE(ACFixture, "autocomplete_explicit_type_pack") diff --git a/tests/Compiler.test.cpp b/tests/Compiler.test.cpp index 4a28bdde..d8af94db 100644 --- a/tests/Compiler.test.cpp +++ b/tests/Compiler.test.cpp @@ -611,7 +611,8 @@ TEST_CASE("TableLiteralsIndexConstant") CHECK_EQ("\n" + compileFunction0(R"( local a, b = "key", "value" return {[a] = 42, [b] = 0} -)"), R"( +)"), + R"( NEWTABLE R0 2 0 LOADN R1 42 SETTABLEKS R1 R0 K0 @@ -624,7 +625,8 @@ RETURN R0 1 CHECK_EQ("\n" + compileFunction0(R"( local a, b = 1, 2 return {[a] = 42, [b] = 0} -)"), R"( +)"), + R"( NEWTABLE R0 0 2 LOADN R1 42 SETTABLEN R1 R0 1 @@ -789,8 +791,6 @@ RETURN R0 1 TEST_CASE("TableSizePredictionLoop") { - ScopedFastFlag sff("LuauPredictTableSizeLoop", true); - CHECK_EQ("\n" + compileFunction0(R"( local t = {} for i=1,4 do @@ -2827,7 +2827,7 @@ RETURN R1 -1 TEST_CASE("FastcallSelect") { - ScopedFastFlag sff("LuauCompileSelectBuiltin", true); + ScopedFastFlag sff("LuauCompileSelectBuiltin2", true); // select(_, ...) compiles to a builtin call CHECK_EQ("\n" + compileFunction0("return (select('#', ...))"), R"( @@ -2846,7 +2846,8 @@ for i=1, select('#', ...) do sum += select(i, ...) end return sum -)"), R"( +)"), + R"( LOADN R0 0 LOADN R3 1 LOADK R5 K0 @@ -2856,13 +2857,14 @@ GETVARARGS R6 -1 CALL R4 -1 1 MOVE R1 R4 LOADN R2 1 -FORNPREP R1 +7 -FASTCALL1 57 R3 +3 +FORNPREP R1 +8 +FASTCALL1 57 R3 +4 GETIMPORT R4 2 +MOVE R5 R3 GETVARARGS R6 -1 CALL R4 -1 1 ADD R0 R0 R4 -FORNLOOP R1 -7 +FORNLOOP R1 -8 RETURN R0 1 )"); diff --git a/tests/Conformance.test.cpp b/tests/Conformance.test.cpp index 914b881f..e580949f 100644 --- a/tests/Conformance.test.cpp +++ b/tests/Conformance.test.cpp @@ -492,7 +492,6 @@ TEST_CASE("DateTime") TEST_CASE("Debug") { - ScopedFastFlag sffr("LuauBytecodeV2Read", true); ScopedFastFlag sffw("LuauBytecodeV2Write", true); runConformance("debug.lua"); diff --git a/tests/LValue.test.cpp b/tests/LValue.test.cpp index 8a092779..606f6de3 100644 --- a/tests/LValue.test.cpp +++ b/tests/LValue.test.cpp @@ -38,8 +38,6 @@ TEST_SUITE_BEGIN("LValue"); TEST_CASE("Luau_merge_hashmap_order") { - ScopedFastFlag sff{"LuauLValueAsKey", true}; - std::string a = "a"; std::string b = "b"; std::string c = "c"; @@ -58,20 +56,18 @@ TEST_CASE("Luau_merge_hashmap_order") TypeArena arena; merge(arena, m, other); - REQUIRE_EQ(3, m.NEW_refinements.size()); - REQUIRE(m.NEW_refinements.count(mkSymbol(a))); - REQUIRE(m.NEW_refinements.count(mkSymbol(b))); - REQUIRE(m.NEW_refinements.count(mkSymbol(c))); + REQUIRE_EQ(3, m.size()); + REQUIRE(m.count(mkSymbol(a))); + REQUIRE(m.count(mkSymbol(b))); + REQUIRE(m.count(mkSymbol(c))); - CHECK_EQ("string", toString(m.NEW_refinements[mkSymbol(a)])); - CHECK_EQ("string", toString(m.NEW_refinements[mkSymbol(b)])); - CHECK_EQ("boolean | number", toString(m.NEW_refinements[mkSymbol(c)])); + CHECK_EQ("string", toString(m[mkSymbol(a)])); + CHECK_EQ("string", toString(m[mkSymbol(b)])); + CHECK_EQ("boolean | number", toString(m[mkSymbol(c)])); } TEST_CASE("Luau_merge_hashmap_order2") { - ScopedFastFlag sff{"LuauLValueAsKey", true}; - std::string a = "a"; std::string b = "b"; std::string c = "c"; @@ -90,20 +86,18 @@ TEST_CASE("Luau_merge_hashmap_order2") TypeArena arena; merge(arena, m, other); - REQUIRE_EQ(3, m.NEW_refinements.size()); - REQUIRE(m.NEW_refinements.count(mkSymbol(a))); - REQUIRE(m.NEW_refinements.count(mkSymbol(b))); - REQUIRE(m.NEW_refinements.count(mkSymbol(c))); + REQUIRE_EQ(3, m.size()); + REQUIRE(m.count(mkSymbol(a))); + REQUIRE(m.count(mkSymbol(b))); + REQUIRE(m.count(mkSymbol(c))); - CHECK_EQ("string", toString(m.NEW_refinements[mkSymbol(a)])); - CHECK_EQ("string", toString(m.NEW_refinements[mkSymbol(b)])); - CHECK_EQ("boolean | number", toString(m.NEW_refinements[mkSymbol(c)])); + CHECK_EQ("string", toString(m[mkSymbol(a)])); + CHECK_EQ("string", toString(m[mkSymbol(b)])); + CHECK_EQ("boolean | number", toString(m[mkSymbol(c)])); } TEST_CASE("one_map_has_overlap_at_end_whereas_other_has_it_in_start") { - ScopedFastFlag sff{"LuauLValueAsKey", true}; - std::string a = "a"; std::string b = "b"; std::string c = "c"; @@ -125,18 +119,18 @@ TEST_CASE("one_map_has_overlap_at_end_whereas_other_has_it_in_start") TypeArena arena; merge(arena, m, other); - REQUIRE_EQ(5, m.NEW_refinements.size()); - REQUIRE(m.NEW_refinements.count(mkSymbol(a))); - REQUIRE(m.NEW_refinements.count(mkSymbol(b))); - REQUIRE(m.NEW_refinements.count(mkSymbol(c))); - REQUIRE(m.NEW_refinements.count(mkSymbol(d))); - REQUIRE(m.NEW_refinements.count(mkSymbol(e))); + REQUIRE_EQ(5, m.size()); + REQUIRE(m.count(mkSymbol(a))); + REQUIRE(m.count(mkSymbol(b))); + REQUIRE(m.count(mkSymbol(c))); + REQUIRE(m.count(mkSymbol(d))); + REQUIRE(m.count(mkSymbol(e))); - CHECK_EQ("string", toString(m.NEW_refinements[mkSymbol(a)])); - CHECK_EQ("number", toString(m.NEW_refinements[mkSymbol(b)])); - CHECK_EQ("boolean | string", toString(m.NEW_refinements[mkSymbol(c)])); - CHECK_EQ("number", toString(m.NEW_refinements[mkSymbol(d)])); - CHECK_EQ("boolean", toString(m.NEW_refinements[mkSymbol(e)])); + CHECK_EQ("string", toString(m[mkSymbol(a)])); + CHECK_EQ("number", toString(m[mkSymbol(b)])); + CHECK_EQ("boolean | string", toString(m[mkSymbol(c)])); + CHECK_EQ("number", toString(m[mkSymbol(d)])); + CHECK_EQ("boolean", toString(m[mkSymbol(e)])); } TEST_CASE("hashing_lvalue_global_prop_access") @@ -159,7 +153,7 @@ TEST_CASE("hashing_lvalue_global_prop_access") CHECK_EQ(LValueHasher{}(t_x1), LValueHasher{}(t_x2)); CHECK_EQ(LValueHasher{}(t_x2), LValueHasher{}(t_x2)); - NEW_RefinementMap m; + RefinementMap m; m[t_x1] = getSingletonTypes().stringType; m[t_x2] = getSingletonTypes().numberType; @@ -188,7 +182,7 @@ TEST_CASE("hashing_lvalue_local_prop_access") CHECK_NE(LValueHasher{}(t_x1), LValueHasher{}(t_x2)); CHECK_EQ(LValueHasher{}(t_x2), LValueHasher{}(t_x2)); - NEW_RefinementMap m; + RefinementMap m; m[t_x1] = getSingletonTypes().stringType; m[t_x2] = getSingletonTypes().numberType; diff --git a/tests/Linter.test.cpp b/tests/Linter.test.cpp index 5ad06f0d..d1cc49b2 100644 --- a/tests/Linter.test.cpp +++ b/tests/Linter.test.cpp @@ -54,6 +54,17 @@ return _ CHECK_EQ(result.warnings[0].text, "Placeholder value '_' is read here; consider using a named variable"); } +TEST_CASE_FIXTURE(Fixture, "PlaceholderReadGlobal") +{ + LintResult result = lint(R"( +_ = 5 +print(_) +)"); + + CHECK_EQ(result.warnings.size(), 1); + CHECK_EQ(result.warnings[0].text, "Placeholder value '_' is read here; consider using a named variable"); +} + TEST_CASE_FIXTURE(Fixture, "PlaceholderWrite") { LintResult result = lint(R"( @@ -853,7 +864,7 @@ string.format("%Y") local _ = ("%"):format() -- correct format strings, just to uh make sure -string.format("hello %d %f", 4, 5) +string.format("hello %+10d %.02f %%", 4, 5) )"); CHECK_EQ(result.warnings.size(), 4); @@ -1078,16 +1089,18 @@ TEST_CASE_FIXTURE(Fixture, "FormatStringDate") os.date("%") os.date("%L") os.date("%?") +os.date("\0") -- correct formats os.date("it's %c now") os.date("!*t") )"); - CHECK_EQ(result.warnings.size(), 3); + CHECK_EQ(result.warnings.size(), 4); CHECK_EQ(result.warnings[0].text, "Invalid date format: unfinished replacement"); CHECK_EQ(result.warnings[1].text, "Invalid date format: unexpected replacement character; must be a date format specifier or %"); CHECK_EQ(result.warnings[2].text, "Invalid date format: unexpected replacement character; must be a date format specifier or %"); + CHECK_EQ(result.warnings[3].text, "Invalid date format: date format can not contain null characters"); } TEST_CASE_FIXTURE(Fixture, "FormatStringTyped") @@ -1396,8 +1409,6 @@ end TEST_CASE_FIXTURE(Fixture, "TableOperations") { - ScopedFastFlag sff("LuauLintTableCreateTable", true); - LintResult result = lintTyped(R"( local t = {} local tt = {} @@ -1435,8 +1446,10 @@ table.create(42, {} :: {}) "table.insert may change behavior if the call returns more than one result; consider adding parentheses around second argument"); CHECK_EQ(result.warnings[6].text, "table.move uses index 0 but arrays are 1-based; did you mean 1 instead?"); CHECK_EQ(result.warnings[7].text, "table.move uses index 0 but arrays are 1-based; did you mean 1 instead?"); - CHECK_EQ(result.warnings[8].text, "table.create with a table literal will reuse the same object for all elements; consider using a for loop instead"); - CHECK_EQ(result.warnings[9].text, "table.create with a table literal will reuse the same object for all elements; consider using a for loop instead"); + CHECK_EQ( + result.warnings[8].text, "table.create with a table literal will reuse the same object for all elements; consider using a for loop instead"); + CHECK_EQ( + result.warnings[9].text, "table.create with a table literal will reuse the same object for all elements; consider using a for loop instead"); } TEST_CASE_FIXTURE(Fixture, "DuplicateConditions") diff --git a/tests/Parser.test.cpp b/tests/Parser.test.cpp index 90831ee9..c1a8887b 100644 --- a/tests/Parser.test.cpp +++ b/tests/Parser.test.cpp @@ -7,8 +7,6 @@ #include "doctest.h" -LUAU_FASTFLAG(LuauFixAmbiguousErrorRecoveryInAssign) - using namespace Luau; namespace @@ -1639,10 +1637,7 @@ TEST_CASE_FIXTURE(Fixture, "parse_error_confusing_function_call") "Ambiguous syntax: this looks like an argument list for a function call, but could also be a start of new statement; use ';' to separate " "statements"); - if (FFlag::LuauFixAmbiguousErrorRecoveryInAssign) - CHECK(result4.errors.size() == 1); - else - CHECK(result4.errors.size() == 5); + CHECK(result4.errors.size() == 1); } TEST_CASE_FIXTURE(Fixture, "parse_error_varargs") diff --git a/tests/TypeInfer.annotations.test.cpp b/tests/TypeInfer.annotations.test.cpp index 86165814..572b882d 100644 --- a/tests/TypeInfer.annotations.test.cpp +++ b/tests/TypeInfer.annotations.test.cpp @@ -209,8 +209,6 @@ TEST_CASE_FIXTURE(Fixture, "as_expr_does_not_propagate_type_info") TEST_CASE_FIXTURE(Fixture, "as_expr_is_bidirectional") { - ScopedFastFlag sff{"LuauBidirectionalAsExpr", true}; - CheckResult result = check(R"( local a = 55 :: number? local b = a :: number @@ -224,7 +222,6 @@ TEST_CASE_FIXTURE(Fixture, "as_expr_is_bidirectional") TEST_CASE_FIXTURE(Fixture, "as_expr_warns_on_unrelated_cast") { - ScopedFastFlag sff{"LuauBidirectionalAsExpr", true}; ScopedFastFlag sff2{"LuauErrorRecoveryType", true}; CheckResult result = check(R"( diff --git a/tests/TypeInfer.builtins.test.cpp b/tests/TypeInfer.builtins.test.cpp index 5e08654a..6730bedb 100644 --- a/tests/TypeInfer.builtins.test.cpp +++ b/tests/TypeInfer.builtins.test.cpp @@ -889,4 +889,55 @@ TEST_CASE_FIXTURE(Fixture, "dont_add_definitions_to_persistent_types") REQUIRE(gtv->definition); } +TEST_CASE_FIXTURE(Fixture, "assert_removes_falsy_types") +{ + ScopedFastFlag sff[]{ + {"LuauAssertStripsFalsyTypes", true}, + {"LuauDiscriminableUnions", true}, + }; + + CheckResult result = check(R"( + local function f(x: (number | boolean)?) + return assert(x) + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + CHECK_EQ("((boolean | number)?) -> number | true", toString(requireType("f"))); +} + +TEST_CASE_FIXTURE(Fixture, "assert_removes_falsy_types_even_from_type_pack_tail_but_only_for_the_first_type") +{ + ScopedFastFlag sff[]{ + {"LuauAssertStripsFalsyTypes", true}, + {"LuauDiscriminableUnions", true}, + }; + + CheckResult result = check(R"( + local function f(...: number?) + return assert(...) + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + CHECK_EQ("(...number?) -> (number, ...number?)", toString(requireType("f"))); +} + +TEST_CASE_FIXTURE(Fixture, "assert_returns_false_and_string_iff_it_knows_the_first_argument_cannot_be_truthy") +{ + ScopedFastFlag sff[]{ + {"LuauAssertStripsFalsyTypes", true}, + {"LuauDiscriminableUnions", true}, + }; + + CheckResult result = check(R"( + local function f(x: nil) + return assert(x, "hmm") + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + CHECK_EQ("(nil) -> nil", toString(requireType("f"))); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.provisional.test.cpp b/tests/TypeInfer.provisional.test.cpp index 47c13be9..e5eb0dca 100644 --- a/tests/TypeInfer.provisional.test.cpp +++ b/tests/TypeInfer.provisional.test.cpp @@ -176,19 +176,6 @@ TEST_CASE_FIXTURE(Fixture, "pass_a_union_of_tables_to_a_function_that_requires_a REQUIRE_EQ("{| [any]: any, x: number, y: number |}", toString(requireType("b"))); } -TEST_CASE_FIXTURE(Fixture, "normal_conditional_expression_has_refinements") -{ - CheckResult result = check(R"( - local foo: {x: number}? = nil - local bar = foo and foo.x -- TODO: Geez. We are inferring the wrong types here. Should be 'number?'. - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - - // Binary and/or return types are straight up wrong. JIRA: CLI-40300 - CHECK_EQ("boolean | number", toString(requireType("bar"))); -} - // Luau currently doesn't yet know how to allow assignments when the binding was refined. TEST_CASE_FIXTURE(Fixture, "while_body_are_also_refined") { diff --git a/tests/TypeInfer.refinements.test.cpp b/tests/TypeInfer.refinements.test.cpp index f346ddfd..3a610c3a 100644 --- a/tests/TypeInfer.refinements.test.cpp +++ b/tests/TypeInfer.refinements.test.cpp @@ -939,8 +939,6 @@ TEST_CASE_FIXTURE(Fixture, "type_comparison_ifelse_expression") TEST_CASE_FIXTURE(Fixture, "correctly_lookup_a_shadowed_local_that_which_was_previously_refined") { - ScopedFastFlag sff{"LuauLValueAsKey", true}; - CheckResult result = check(R"( local foo: string? = "hi" assert(foo) @@ -955,8 +953,6 @@ TEST_CASE_FIXTURE(Fixture, "correctly_lookup_a_shadowed_local_that_which_was_pre TEST_CASE_FIXTURE(Fixture, "correctly_lookup_property_whose_base_was_previously_refined") { - ScopedFastFlag sff{"LuauLValueAsKey", true}; - CheckResult result = check(R"( type T = {x: string | number} local t: T? = {x = "hi"} @@ -974,8 +970,6 @@ TEST_CASE_FIXTURE(Fixture, "correctly_lookup_property_whose_base_was_previously_ TEST_CASE_FIXTURE(Fixture, "correctly_lookup_property_whose_base_was_previously_refined2") { - ScopedFastFlag sff{"LuauLValueAsKey", true}; - CheckResult result = check(R"( type T = { x: { y: number }? } @@ -993,8 +987,6 @@ TEST_CASE_FIXTURE(Fixture, "correctly_lookup_property_whose_base_was_previously_ TEST_CASE_FIXTURE(Fixture, "apply_refinements_on_astexprindexexpr_whose_subscript_expr_is_constant_string") { - ScopedFastFlag sff{"LuauRefiLookupFromIndexExpr", true}; - CheckResult result = check(R"( type T = { [string]: { prop: number }? } local t: T = {} @@ -1061,27 +1053,62 @@ TEST_CASE_FIXTURE(Fixture, "discriminate_tag") CHECK_EQ("Dog", toString(requireTypeAtPosition({9, 33}))); } -TEST_CASE_FIXTURE(Fixture, "apply_refinements_on_astexprindexexpr_whose_subscript_expr_is_constant_string") +TEST_CASE_FIXTURE(Fixture, "and_or_peephole_refinement") { - ScopedFastFlag sff{"LuauRefiLookupFromIndexExpr", true}; - CheckResult result = check(R"( - type T = { [string]: { prop: number }? } - local t: T = {} - - if t["hello"] then - local foo = t["hello"].prop + local function len(a: {any}) + return a and #a or nil end )"); LUAU_REQUIRE_NO_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "and_or_peephole_refinement") +TEST_CASE_FIXTURE(Fixture, "narrow_boolean_to_true_or_false") { + ScopedFastFlag sff[]{ + {"LuauParseSingletonTypes", true}, + {"LuauSingletonTypes", true}, + {"LuauDiscriminableUnions", true}, + {"LuauAssertStripsFalsyTypes", true}, + }; + CheckResult result = check(R"( - local function len(a: {any}) - return a and #a or nil + local function is_true(b: true) end + local function is_false(b: false) end + + local function f(x: boolean) + if x then + is_true(x) + else + is_false(x) + end + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "discriminate_on_properties_of_disjoint_tables_where_that_property_is_true_or_false") +{ + ScopedFastFlag sff[]{ + {"LuauParseSingletonTypes", true}, + {"LuauSingletonTypes", true}, + {"LuauDiscriminableUnions", true}, + {"LuauAssertStripsFalsyTypes", true}, + }; + + CheckResult result = check(R"( + type Ok = { ok: true, value: T } + type Err = { ok: false, error: E } + type Result = Ok | Err + + local function apply(t: Result, f: (T) -> (), g: (E) -> ()) + if t.ok then + f(t.value) + else + g(t.error) + end end )"); diff --git a/tests/TypeInfer.tables.test.cpp b/tests/TypeInfer.tables.test.cpp index 48310921..f19cb618 100644 --- a/tests/TypeInfer.tables.test.cpp +++ b/tests/TypeInfer.tables.test.cpp @@ -1482,7 +1482,7 @@ TEST_CASE_FIXTURE(Fixture, "casting_unsealed_tables_with_props_into_table_with_i REQUIRE(tm); CHECK_EQ("{| [string]: string |}", toString(tm->wantedType, o)); // Should t now have an indexer? - // It would if the assignment to rt was correctly typed. + // It would if the assignment to rt was correctly typed. CHECK_EQ("{ [string]: string, foo: number }", toString(tm->givenType, o)); } @@ -2082,7 +2082,7 @@ caused by: TEST_CASE_FIXTURE(Fixture, "explicitly_typed_table") { - ScopedFastFlag sffs[] { + ScopedFastFlag sffs[]{ {"LuauPropertiesGetExpectedType", true}, {"LuauExpectedTypesOfProperties", true}, {"LuauTableSubtypingVariance2", true}, @@ -2103,7 +2103,7 @@ a.p = { x = 9 } TEST_CASE_FIXTURE(Fixture, "explicitly_typed_table_error") { - ScopedFastFlag sffs[] { + ScopedFastFlag sffs[]{ {"LuauPropertiesGetExpectedType", true}, {"LuauExpectedTypesOfProperties", true}, {"LuauTableSubtypingVariance2", true}, @@ -2131,7 +2131,7 @@ caused by: TEST_CASE_FIXTURE(Fixture, "explicitly_typed_table_with_indexer") { - ScopedFastFlag sffs[] { + ScopedFastFlag sffs[]{ {"LuauPropertiesGetExpectedType", true}, {"LuauExpectedTypesOfProperties", true}, {"LuauTableSubtypingVariance2", true}, diff --git a/tests/TypeInfer.test.cpp b/tests/TypeInfer.test.cpp index c9b30e1a..ead3d762 100644 --- a/tests/TypeInfer.test.cpp +++ b/tests/TypeInfer.test.cpp @@ -2429,21 +2429,6 @@ TEST_CASE_FIXTURE(Fixture, "should_be_able_to_infer_this_without_stack_overflowi LUAU_REQUIRE_NO_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "x_or_y_forces_both_x_and_y_to_be_of_same_type_if_either_is_free") -{ - CheckResult result = check(R"( - local function f(x, y) return x or y end - - local x = f(1, 2) - local y = f(3, "foo") - )"); - - LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK_EQ(*requireType("x"), *typeChecker.numberType); - - CHECK_EQ(result.errors[0], (TypeError{Location{{4, 23}, {4, 28}}, TypeMismatch{typeChecker.numberType, typeChecker.stringType}})); -} - TEST_CASE_FIXTURE(Fixture, "inferring_hundreds_of_self_calls_should_not_suffocate_memory") { CheckResult result = check(R"( @@ -4509,7 +4494,7 @@ f(function(x) print(x) end) } TEST_CASE_FIXTURE(Fixture, "infer_generic_function_function_argument") -{ +{ ScopedFastFlag sff{"LuauUnsealedTableLiteral", true}; CheckResult result = check(R"( @@ -4777,7 +4762,7 @@ local a: X = if true then {"1", 2, 3} else {4, 5, 6} TEST_CASE_FIXTURE(Fixture, "tc_if_else_expressions_expected_type_2") { ScopedFastFlag luauIfElseExpectedType2{"LuauIfElseExpectedType2", true}; - ScopedFastFlag luauIfElseBranchTypeUnion{ "LuauIfElseBranchTypeUnion", true }; + ScopedFastFlag luauIfElseBranchTypeUnion{"LuauIfElseBranchTypeUnion", true}; CheckResult result = check(R"( local a: number? = if true then 1 else nil @@ -5012,16 +4997,14 @@ local b: B = a )"); LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK_EQ( - toString(result.errors[0]), R"(Type '(number, number) -> (number, string)' could not be converted into '(number, number) -> (number, boolean)' + CHECK_EQ(toString(result.errors[0]), + R"(Type '(number, number) -> (number, string)' could not be converted into '(number, number) -> (number, boolean)' caused by: Return #2 type is not compatible. Type 'string' could not be converted into 'boolean')"); } TEST_CASE_FIXTURE(Fixture, "prop_access_on_any_with_other_options") { - ScopedFastFlag sff{"LuauLValueAsKey", true}; - CheckResult result = check(R"( local function f(thing: any | string) local foo = thing.SomeRandomKey @@ -5120,4 +5103,65 @@ end )"); } +TEST_CASE_FIXTURE(Fixture, "cli_50041_committing_txnlog_in_apollo_client_error") +{ + ScopedFastFlag committingTxnLog{"LuauUseCommittingTxnLog", true}; + ScopedFastFlag subtypingVariance{"LuauTableSubtypingVariance2", true}; + + CheckResult result = check(R"( + --!strict + --!nolint + + type FieldSpecifier = { + fieldName: string, + } + + type ReadFieldOptions = FieldSpecifier & { from: number? } + + type Policies = { + getStoreFieldName: (self: Policies, fieldSpec: FieldSpecifier) -> string, + } + + local Policies = {} + + local function foo(p: Policies) + end + + function Policies:getStoreFieldName(specifier: FieldSpecifier): string + return "" + end + + function Policies:readField(options: ReadFieldOptions) + local _ = self:getStoreFieldName(options) + --[[ + Type error: + TypeError { "MainModule", Location { { line = 25, col = 16 }, { line = 25, col = 20 } }, TypeMismatch { Policies, {- getStoreFieldName: (tp1) -> (a, b...) -} } } + ]] + foo(self) + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "do_not_modify_imported_types") +{ + ScopedFastFlag noSealedTypeMod{"LuauNoSealedTypeMod", true}; + + fileResolver.source["game/A"] = R"( +export type Type = { unrelated: boolean } +return {} + )"; + + fileResolver.source["game/B"] = R"( +local types = require(game.A) +type Type = types.Type +local x: Type = {} +function x:Destroy(): () end + )"; + + CheckResult result = frontend.check("game/B"); + LUAU_REQUIRE_ERRORS(result); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.tryUnify.test.cpp b/tests/TypeInfer.tryUnify.test.cpp index 1e790eba..0aeca096 100644 --- a/tests/TypeInfer.tryUnify.test.cpp +++ b/tests/TypeInfer.tryUnify.test.cpp @@ -260,4 +260,17 @@ TEST_CASE_FIXTURE(TryUnifyFixture, "free_tail_is_grown_properly") CHECK(unifyErrors.size() == 0); } +TEST_CASE_FIXTURE(TryUnifyFixture, "recursive_metatable_getmatchtag") +{ + ScopedFastFlag luauUnionTagMatchFix{"LuauUnionTagMatchFix", true}; + + TypeVar redirect{FreeTypeVar{TypeLevel{}}}; + TypeVar table{TableTypeVar{}}; + TypeVar metatable{MetatableTypeVar{&redirect, &table}}; + redirect = BoundTypeVar{&metatable}; // Now we have a metatable that is recursive on the table type + TypeVar variant{UnionTypeVar{{&metatable, typeChecker.numberType}}}; + + state.tryUnify(&metatable, &variant); +} + TEST_SUITE_END(); diff --git a/tests/conformance/basic.lua b/tests/conformance/basic.lua index 188b8ebc..de091632 100644 --- a/tests/conformance/basic.lua +++ b/tests/conformance/basic.lua @@ -118,6 +118,10 @@ assert((function() return #_G end)() == 0) assert((function() return #{1,2} end)() == 2) assert((function() return #'g' end)() == 1) +local ud = newproxy(true) +getmetatable(ud).__len = function() return 42 end +assert((function() return #ud end)() == 42) + assert((function() local a = 1 a = -a return a end)() == -1) -- while/repeat diff --git a/tests/conformance/vararg.lua b/tests/conformance/vararg.lua index 5aaa422b..d05f9577 100644 --- a/tests/conformance/vararg.lua +++ b/tests/conformance/vararg.lua @@ -105,6 +105,45 @@ assert(a==5 and b==4 and c==3 and d==2 and e==1) a,b,c,d,e = f(4) assert(a==nil and b==nil and c==nil and d==nil and e==nil) +-- select tests +a = {select(3, unpack{10,20,30,40})} +assert(table.getn(a) == 2 and a[1] == 30 and a[2] == 40) +a = {select(1)} +assert(next(a) == nil) +a = {select(-1, 3, 5, 7)} +assert(a[1] == 7 and a[2] == nil) +a = {select(-2, 3, 5, 7)} +assert(a[1] == 5 and a[2] == 7 and a[3] == nil) +pcall(select, 10000) +pcall(select, -10000) + +-- select(_, ...) has special optimizations so it needs extra testing +function selectone(n, ...) + local e = select(n, ...) + return e +end + +function selectmany(n, ...) + return table.concat({select(n, ...)}, ',') +end + +assert(selectone('#') == 0) +assert(selectmany('#') == "0") + +assert(selectone('#', 10, 20, 30) == 3) +assert(selectmany('#', 10, 20, 30) == "3") + +assert(selectone(1, 10, 20, 30) == 10) +assert(selectmany(1, 10, 20, 30) == "10,20,30") + +assert(selectone(2, 10, 20, 30) == 20) +assert(selectmany(2, 10, 20, 30) == "20,30") + +assert(selectone(-2, 10, 20, 30) == 20) +assert(selectmany(-2, 10, 20, 30) == "20,30") + +assert(selectone('3', 10, 20, 30) == 30) +assert(selectmany('3', 10, 20, 30) == "30") -- varargs for main chunks f = loadstring[[ return {...} ]] @@ -122,16 +161,5 @@ f = loadstring[[ assert(f("a", "b", nil, {}, assert)) assert(f()) -a = {select(3, unpack{10,20,30,40})} -assert(table.getn(a) == 2 and a[1] == 30 and a[2] == 40) -a = {select(1)} -assert(next(a) == nil) -a = {select(-1, 3, 5, 7)} -assert(a[1] == 7 and a[2] == nil) -a = {select(-2, 3, 5, 7)} -assert(a[1] == 5 and a[2] == 7 and a[3] == nil) -pcall(select, 10000) -pcall(select, -10000) - return('OK') From 4e60eec1fc97132bee2f5bb09664b08f235f2c55 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Thu, 3 Feb 2022 16:31:50 -0800 Subject: [PATCH 144/399] Apply fix to the crash --- Analysis/src/TypeInfer.cpp | 1 + 1 file changed, 1 insertion(+) diff --git a/Analysis/src/TypeInfer.cpp b/Analysis/src/TypeInfer.cpp index 4d25fe2e..b9096d2e 100644 --- a/Analysis/src/TypeInfer.cpp +++ b/Analysis/src/TypeInfer.cpp @@ -4977,6 +4977,7 @@ TypeId TypeChecker::resolveType(const ScopePtr& scope, const AstType& annotation if (notEnoughParameters && hasDefaultParameters) { // 'applyTypeFunction' is used to substitute default types that reference previous generic types + applyTypeFunction.log = TxnLog::empty(); applyTypeFunction.typeArguments.clear(); applyTypeFunction.typePackArguments.clear(); applyTypeFunction.currentModule = currentModule; From 4748777ce850813d639e97338679357171d98289 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Thu, 3 Feb 2022 16:43:44 -0800 Subject: [PATCH 145/399] Fix isocline warnings --- extern/isocline/src/isocline.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/extern/isocline/src/isocline.c b/extern/isocline/src/isocline.c index 13278062..8b6055cf 100644 --- a/extern/isocline/src/isocline.c +++ b/extern/isocline/src/isocline.c @@ -13,7 +13,12 @@ // $ gcc -c src/isocline.c //------------------------------------------------------------- #if !defined(IC_SEPARATE_OBJS) -# define _CRT_SECURE_NO_WARNINGS // for msvc +# ifndef _CRT_NONSTDC_NO_WARNINGS +# define _CRT_NONSTDC_NO_WARNINGS // for msvc +# endif +# ifndef _CRT_SECURE_NO_WARNINGS +# define _CRT_SECURE_NO_WARNINGS // for msvc +# endif # define _XOPEN_SOURCE 700 // for wcwidth # define _DEFAULT_SOURCE // ensure usleep stays visible with _XOPEN_SOURCE >= 700 # include "attr.c" From d58e70b8c181aa6f352c696448b8dceb77ecae51 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Fri, 4 Feb 2022 08:45:57 -0800 Subject: [PATCH 146/399] Sync to upstream/release/513 (#340) --- Analysis/include/Luau/Error.h | 12 +- Analysis/include/Luau/LValue.h | 14 +- Analysis/include/Luau/Substitution.h | 6 + Analysis/include/Luau/TxnLog.h | 8 +- Analysis/include/Luau/TypeInfer.h | 9 +- Analysis/include/Luau/TypedAllocator.h | 22 +- Analysis/include/Luau/Unifier.h | 4 + Analysis/src/BuiltinDefinitions.cpp | 41 +- Analysis/src/LValue.cpp | 52 +- Analysis/src/Linter.cpp | 4 +- Analysis/src/Substitution.cpp | 65 +- Analysis/src/TxnLog.cpp | 71 +- Analysis/src/TypeInfer.cpp | 241 +- Analysis/src/TypeVar.cpp | 8 +- Analysis/src/TypedAllocator.cpp | 1 - Analysis/src/Unifier.cpp | 541 ++--- Ast/src/Parser.cpp | 3 +- CLI/Coverage.cpp | 2 +- CLI/FileUtils.cpp | 2 +- CLI/Repl.cpp | 55 +- CLI/ReplEntry.cpp | 3 +- CMakeLists.txt | 19 +- Compiler/src/Builtins.cpp | 4 +- Compiler/src/Compiler.cpp | 9 +- Compiler/src/TableShape.cpp | 8 - Makefile | 26 +- Sources.cmake | 5 + VM/include/luaconf.h | 4 +- VM/src/lcorolib.cpp | 2 +- VM/src/ldebug.cpp | 3 +- VM/src/lfunc.cpp | 6 +- VM/src/lgc.cpp | 2 + VM/src/lmem.cpp | 53 +- VM/src/lstring.cpp | 2 +- VM/src/lvmexecute.cpp | 11 +- VM/src/lvmload.cpp | 8 +- extern/isocline/.gitignore | 16 + extern/isocline/LICENSE | 21 + extern/isocline/include/isocline.h | 627 ++++++ extern/isocline/readme.md | 460 ++++ extern/isocline/src/attr.c | 294 +++ extern/isocline/src/attr.h | 70 + extern/isocline/src/bbcode.c | 842 +++++++ extern/isocline/src/bbcode.h | 37 + extern/isocline/src/bbcode_colors.c | 194 ++ extern/isocline/src/common.c | 347 +++ extern/isocline/src/common.h | 187 ++ extern/isocline/src/completers.c | 675 ++++++ extern/isocline/src/completions.c | 326 +++ extern/isocline/src/completions.h | 52 + extern/isocline/src/editline.c | 1142 ++++++++++ extern/isocline/src/editline_completion.c | 277 +++ extern/isocline/src/editline_help.c | 140 ++ extern/isocline/src/editline_history.c | 260 +++ extern/isocline/src/env.h | 60 + extern/isocline/src/highlight.c | 259 +++ extern/isocline/src/highlight.h | 24 + extern/isocline/src/history.c | 269 +++ extern/isocline/src/history.h | 38 + extern/isocline/src/isocline.c | 594 +++++ extern/isocline/src/stringbuf.c | 1038 +++++++++ extern/isocline/src/stringbuf.h | 121 ++ extern/isocline/src/term.c | 1124 ++++++++++ extern/isocline/src/term.h | 85 + extern/isocline/src/term_color.c | 371 ++++ extern/isocline/src/tty.c | 889 ++++++++ extern/isocline/src/tty.h | 160 ++ extern/isocline/src/tty_esc.c | 401 ++++ extern/isocline/src/undo.c | 67 + extern/isocline/src/undo.h | 24 + extern/isocline/src/wcwidth.c | 292 +++ extern/linenoise.hpp | 2415 --------------------- tests/Autocomplete.test.cpp | 100 +- tests/Compiler.test.cpp | 20 +- tests/Conformance.test.cpp | 1 - tests/LValue.test.cpp | 60 +- tests/Linter.test.cpp | 25 +- tests/Parser.test.cpp | 7 +- tests/TypeInfer.annotations.test.cpp | 3 - tests/TypeInfer.builtins.test.cpp | 51 + tests/TypeInfer.provisional.test.cpp | 13 - tests/TypeInfer.refinements.test.cpp | 65 +- tests/TypeInfer.tables.test.cpp | 8 +- tests/TypeInfer.test.cpp | 86 +- tests/TypeInfer.tryUnify.test.cpp | 13 + tests/conformance/basic.lua | 4 + tests/conformance/vararg.lua | 50 +- 87 files changed, 12818 insertions(+), 3212 deletions(-) create mode 100644 extern/isocline/.gitignore create mode 100644 extern/isocline/LICENSE create mode 100644 extern/isocline/include/isocline.h create mode 100644 extern/isocline/readme.md create mode 100644 extern/isocline/src/attr.c create mode 100644 extern/isocline/src/attr.h create mode 100644 extern/isocline/src/bbcode.c create mode 100644 extern/isocline/src/bbcode.h create mode 100644 extern/isocline/src/bbcode_colors.c create mode 100644 extern/isocline/src/common.c create mode 100644 extern/isocline/src/common.h create mode 100644 extern/isocline/src/completers.c create mode 100644 extern/isocline/src/completions.c create mode 100644 extern/isocline/src/completions.h create mode 100644 extern/isocline/src/editline.c create mode 100644 extern/isocline/src/editline_completion.c create mode 100644 extern/isocline/src/editline_help.c create mode 100644 extern/isocline/src/editline_history.c create mode 100644 extern/isocline/src/env.h create mode 100644 extern/isocline/src/highlight.c create mode 100644 extern/isocline/src/highlight.h create mode 100644 extern/isocline/src/history.c create mode 100644 extern/isocline/src/history.h create mode 100644 extern/isocline/src/isocline.c create mode 100644 extern/isocline/src/stringbuf.c create mode 100644 extern/isocline/src/stringbuf.h create mode 100644 extern/isocline/src/term.c create mode 100644 extern/isocline/src/term.h create mode 100644 extern/isocline/src/term_color.c create mode 100644 extern/isocline/src/tty.c create mode 100644 extern/isocline/src/tty.h create mode 100644 extern/isocline/src/tty_esc.c create mode 100644 extern/isocline/src/undo.c create mode 100644 extern/isocline/src/undo.h create mode 100644 extern/isocline/src/wcwidth.c delete mode 100644 extern/linenoise.hpp diff --git a/Analysis/include/Luau/Error.h b/Analysis/include/Luau/Error.h index aff3c4d9..a71e0224 100644 --- a/Analysis/include/Luau/Error.h +++ b/Analysis/include/Luau/Error.h @@ -285,12 +285,12 @@ struct TypesAreUnrelated bool operator==(const TypesAreUnrelated& rhs) const; }; -using TypeErrorData = Variant; +using TypeErrorData = + Variant; struct TypeError { diff --git a/Analysis/include/Luau/LValue.h b/Analysis/include/Luau/LValue.h index 8fd96f05..3d510d5f 100644 --- a/Analysis/include/Luau/LValue.h +++ b/Analysis/include/Luau/LValue.h @@ -4,7 +4,6 @@ #include "Luau/Variant.h" #include "Luau/Symbol.h" -#include // TODO: Kill with LuauLValueAsKey. #include #include @@ -38,24 +37,13 @@ std::optional tryGetLValue(const class AstExpr& expr); // Utility function: breaks down an LValue to get at the Symbol, and reverses the vector of keys. std::pair> getFullName(const LValue& lvalue); -// Kill with LuauLValueAsKey. -std::string toString(const LValue& lvalue); - template const T* get(const LValue& lvalue) { return get_if(&lvalue); } -using NEW_RefinementMap = std::unordered_map; -using DEPRECATED_RefinementMap = std::map; - -// Transient. Kill with LuauLValueAsKey. -struct RefinementMap -{ - NEW_RefinementMap NEW_refinements; - DEPRECATED_RefinementMap DEPRECATED_refinements; -}; +using RefinementMap = std::unordered_map; void merge(RefinementMap& l, const RefinementMap& r, std::function f); void addRefinement(RefinementMap& refis, const LValue& lvalue, TypeId ty); diff --git a/Analysis/include/Luau/Substitution.h b/Analysis/include/Luau/Substitution.h index 80a14e8f..4f3307cd 100644 --- a/Analysis/include/Luau/Substitution.h +++ b/Analysis/include/Luau/Substitution.h @@ -55,6 +55,8 @@ namespace Luau { +struct TxnLog; + enum class TarjanResult { TooManyChildren, @@ -89,6 +91,10 @@ struct Tarjan int childCount = 0; + // This should never be null; ensure you initialize it before calling + // substitution methods. + const TxnLog* log; + std::vector edgesTy; std::vector edgesTp; std::vector worklist; diff --git a/Analysis/include/Luau/TxnLog.h b/Analysis/include/Luau/TxnLog.h index dc45bebf..02b87374 100644 --- a/Analysis/include/Luau/TxnLog.h +++ b/Analysis/include/Luau/TxnLog.h @@ -72,6 +72,9 @@ struct PendingType } }; +std::string toString(PendingType* pending); +std::string dump(PendingType* pending); + // Pending state for a TypePackVar. Generated by a TxnLog and committed via // TxnLog::commit. struct PendingTypePack @@ -85,6 +88,9 @@ struct PendingTypePack } }; +std::string toString(PendingTypePack* pending); +std::string dump(PendingTypePack* pending); + template T* getMutable(PendingType* pending) { @@ -237,7 +243,7 @@ struct TxnLog // Follows a type, accounting for pending type states. The returned type may have // pending state; you should use `pending` or `get` to find out. - TypeId follow(TypeId ty); + TypeId follow(TypeId ty) const; // Follows a type pack, accounting for pending type states. The returned type pack // may have pending state; you should use `pending` or `get` to find out. diff --git a/Analysis/include/Luau/TypeInfer.h b/Analysis/include/Luau/TypeInfer.h index b843509d..90dc9f42 100644 --- a/Analysis/include/Luau/TypeInfer.h +++ b/Analysis/include/Luau/TypeInfer.h @@ -262,7 +262,7 @@ public: * {method: ({method: () -> a}) -> a} * */ - TypeId instantiate(const ScopePtr& scope, TypeId ty, Location location); + TypeId instantiate(const ScopePtr& scope, TypeId ty, Location location, const TxnLog* log = TxnLog::empty()); // Replace any free types or type packs by `any`. // This is used when exporting types from modules, to make sure free types don't leak. @@ -308,9 +308,15 @@ private: TypeId singletonType(bool value); TypeId singletonType(std::string value); + TypeIdPredicate mkTruthyPredicate(bool sense); + // Returns nullopt if the predicate filters down the TypeId to 0 options. std::optional filterMap(TypeId type, TypeIdPredicate predicate); +public: + std::optional pickTypesFromSense(TypeId type, bool sense); + +private: TypeId unionOfTypes(TypeId a, TypeId b, const Location& location, bool unifyFreeTypes = true); // ex @@ -349,7 +355,6 @@ private: void refineLValue(const LValue& lvalue, RefinementMap& refis, const ScopePtr& scope, TypeIdPredicate predicate); std::optional resolveLValue(const ScopePtr& scope, const LValue& lvalue); - std::optional DEPRECATED_resolveLValue(const ScopePtr& scope, const LValue& lvalue); std::optional resolveLValue(const RefinementMap& refis, const ScopePtr& scope, const LValue& lvalue); void resolve(const PredicateVec& predicates, ErrorVec& errVec, RefinementMap& refis, const ScopePtr& scope, bool sense, bool fromOr = false); diff --git a/Analysis/include/Luau/TypedAllocator.h b/Analysis/include/Luau/TypedAllocator.h index 64227e7c..c1c04d10 100644 --- a/Analysis/include/Luau/TypedAllocator.h +++ b/Analysis/include/Luau/TypedAllocator.h @@ -6,8 +6,6 @@ #include #include -LUAU_FASTFLAG(LuauTypedAllocatorZeroStart) - namespace Luau { @@ -22,10 +20,7 @@ class TypedAllocator public: TypedAllocator() { - if (FFlag::LuauTypedAllocatorZeroStart) - currentBlockSize = kBlockSize; - else - appendBlock(); + currentBlockSize = kBlockSize; } ~TypedAllocator() @@ -64,18 +59,12 @@ public: bool empty() const { - if (FFlag::LuauTypedAllocatorZeroStart) - return stuff.empty(); - else - return stuff.size() == 1 && currentBlockSize == 0; + return stuff.empty(); } size_t size() const { - if (FFlag::LuauTypedAllocatorZeroStart) - return stuff.empty() ? 0 : kBlockSize * (stuff.size() - 1) + currentBlockSize; - else - return kBlockSize * (stuff.size() - 1) + currentBlockSize; + return stuff.empty() ? 0 : kBlockSize * (stuff.size() - 1) + currentBlockSize; } void clear() @@ -84,10 +73,7 @@ public: unfreeze(); free(); - if (FFlag::LuauTypedAllocatorZeroStart) - currentBlockSize = kBlockSize; - else - appendBlock(); + currentBlockSize = kBlockSize; } void freeze() diff --git a/Analysis/include/Luau/Unifier.h b/Analysis/include/Luau/Unifier.h index 1b1671c0..9db4e22b 100644 --- a/Analysis/include/Luau/Unifier.h +++ b/Analysis/include/Luau/Unifier.h @@ -51,6 +51,10 @@ struct Unifier private: void tryUnify_(TypeId subTy, TypeId superTy, bool isFunctionCall = false, bool isIntersection = false); + void tryUnifyUnionWithType(TypeId subTy, const UnionTypeVar* uv, TypeId superTy); + void tryUnifyTypeWithUnion(TypeId subTy, TypeId superTy, const UnionTypeVar* uv, bool cacheEnabled, bool isFunctionCall); + void tryUnifyTypeWithIntersection(TypeId subTy, TypeId superTy, const IntersectionTypeVar* uv); + void tryUnifyIntersectionWithType(TypeId subTy, const IntersectionTypeVar* uv, TypeId superTy, bool cacheEnabled, bool isFunctionCall); void tryUnifyPrimitives(TypeId subTy, TypeId superTy); void tryUnifySingletons(TypeId subTy, TypeId superTy); void tryUnifyFunctions(TypeId subTy, TypeId superTy, bool isFunctionCall = false); diff --git a/Analysis/src/BuiltinDefinitions.cpp b/Analysis/src/BuiltinDefinitions.cpp index d527414a..d72422a5 100644 --- a/Analysis/src/BuiltinDefinitions.cpp +++ b/Analysis/src/BuiltinDefinitions.cpp @@ -8,6 +8,8 @@ #include +LUAU_FASTFLAG(LuauAssertStripsFalsyTypes) + /** FIXME: Many of these type definitions are not quite completely accurate. * * Some of them require richer generics than we have. For instance, we do not yet have a way to talk @@ -391,12 +393,41 @@ static std::optional> magicFunctionAssert( { auto [paramPack, predicates] = exprResult; - if (expr.args.size < 1) + if (FFlag::LuauAssertStripsFalsyTypes) + { + TypeArena& arena = typechecker.currentModule->internalTypes; + + auto [head, tail] = flatten(paramPack); + if (head.empty() && tail) + { + std::optional fst = first(*tail); + if (!fst) + return ExprResult{paramPack}; + head.push_back(*fst); + } + + typechecker.reportErrors(typechecker.resolve(predicates, scope, true)); + + if (head.size() > 0) + { + std::optional newhead = typechecker.pickTypesFromSense(head[0], true); + if (!newhead) + head = {typechecker.nilType}; + else + head[0] = *newhead; + } + + return ExprResult{arena.addTypePack(TypePack{std::move(head), tail})}; + } + else + { + if (expr.args.size < 1) + return ExprResult{paramPack}; + + typechecker.reportErrors(typechecker.resolve(predicates, scope, true)); + return ExprResult{paramPack}; - - typechecker.reportErrors(typechecker.resolve(predicates, scope, true)); - - return ExprResult{paramPack}; + } } static std::optional> magicFunctionPack( diff --git a/Analysis/src/LValue.cpp b/Analysis/src/LValue.cpp index da6804c6..c9466a40 100644 --- a/Analysis/src/LValue.cpp +++ b/Analysis/src/LValue.cpp @@ -5,8 +5,6 @@ #include -LUAU_FASTFLAG(LuauLValueAsKey) - namespace Luau { @@ -94,17 +92,7 @@ std::pair> getFullName(const LValue& lvalue) return {*symbol, std::vector(keys.rbegin(), keys.rend())}; } -// Kill with LuauLValueAsKey. -std::string toString(const LValue& lvalue) -{ - auto [symbol, keys] = getFullName(lvalue); - std::string s = toString(symbol); - for (std::string key : keys) - s += "." + key; - return s; -} - -static void merge(NEW_RefinementMap& l, const NEW_RefinementMap& r, std::function f) +void merge(RefinementMap& l, const RefinementMap& r, std::function f) { for (const auto& [k, a] : r) { @@ -115,45 +103,9 @@ static void merge(NEW_RefinementMap& l, const NEW_RefinementMap& r, std::functio } } -static void merge(DEPRECATED_RefinementMap& l, const DEPRECATED_RefinementMap& r, std::function f) -{ - auto itL = l.begin(); - auto itR = r.begin(); - while (itL != l.end() && itR != r.end()) - { - const auto& [k, a] = *itR; - if (itL->first == k) - { - l[k] = f(itL->second, a); - ++itL; - ++itR; - } - else if (itL->first < k) - ++itL; - else - { - l[k] = a; - ++itR; - } - } - - l.insert(itR, r.end()); -} - -void merge(RefinementMap& l, const RefinementMap& r, std::function f) -{ - if (FFlag::LuauLValueAsKey) - return merge(l.NEW_refinements, r.NEW_refinements, f); - else - return merge(l.DEPRECATED_refinements, r.DEPRECATED_refinements, f); -} - void addRefinement(RefinementMap& refis, const LValue& lvalue, TypeId ty) { - if (FFlag::LuauLValueAsKey) - refis.NEW_refinements[lvalue] = ty; - else - refis.DEPRECATED_refinements[toString(lvalue)] = ty; + refis[lvalue] = ty; } } // namespace Luau diff --git a/Analysis/src/Linter.cpp b/Analysis/src/Linter.cpp index 905b70bf..57a33e93 100644 --- a/Analysis/src/Linter.cpp +++ b/Analysis/src/Linter.cpp @@ -12,8 +12,6 @@ #include #include -LUAU_FASTFLAGVARIABLE(LuauLintTableCreateTable, false) - namespace Luau { @@ -2155,7 +2153,7 @@ private: "table.move uses index 0 but arrays are 1-based; did you mean 1 instead?"); } - if (FFlag::LuauLintTableCreateTable && func->index == "create" && node->args.size == 2) + if (func->index == "create" && node->args.size == 2) { // table.create(n, {...}) if (args[1]->is()) diff --git a/Analysis/src/Substitution.cpp b/Analysis/src/Substitution.cpp index 3d004bee..bacbca76 100644 --- a/Analysis/src/Substitution.cpp +++ b/Analysis/src/Substitution.cpp @@ -2,6 +2,7 @@ #include "Luau/Substitution.h" #include "Luau/Common.h" +#include "Luau/TxnLog.h" #include #include @@ -13,17 +14,17 @@ namespace Luau void Tarjan::visitChildren(TypeId ty, int index) { - ty = follow(ty); + ty = log->follow(ty); if (ignoreChildren(ty)) return; - if (const FunctionTypeVar* ftv = get(ty)) + if (const FunctionTypeVar* ftv = log->getMutable(ty)) { visitChild(ftv->argTypes); visitChild(ftv->retType); } - else if (const TableTypeVar* ttv = get(ty)) + else if (const TableTypeVar* ttv = log->getMutable(ty)) { LUAU_ASSERT(!ttv->boundTo); for (const auto& [name, prop] : ttv->props) @@ -40,17 +41,17 @@ void Tarjan::visitChildren(TypeId ty, int index) for (TypePackId itp : ttv->instantiatedTypePackParams) visitChild(itp); } - else if (const MetatableTypeVar* mtv = get(ty)) + else if (const MetatableTypeVar* mtv = log->getMutable(ty)) { visitChild(mtv->table); visitChild(mtv->metatable); } - else if (const UnionTypeVar* utv = get(ty)) + else if (const UnionTypeVar* utv = log->getMutable(ty)) { for (TypeId opt : utv->options) visitChild(opt); } - else if (const IntersectionTypeVar* itv = get(ty)) + else if (const IntersectionTypeVar* itv = log->getMutable(ty)) { for (TypeId part : itv->parts) visitChild(part); @@ -59,19 +60,19 @@ void Tarjan::visitChildren(TypeId ty, int index) void Tarjan::visitChildren(TypePackId tp, int index) { - tp = follow(tp); + tp = log->follow(tp); if (ignoreChildren(tp)) return; - if (const TypePack* tpp = get(tp)) + if (const TypePack* tpp = log->getMutable(tp)) { for (TypeId tv : tpp->head) visitChild(tv); if (tpp->tail) visitChild(*tpp->tail); } - else if (const VariadicTypePack* vtp = get(tp)) + else if (const VariadicTypePack* vtp = log->getMutable(tp)) { visitChild(vtp->ty); } @@ -79,7 +80,7 @@ void Tarjan::visitChildren(TypePackId tp, int index) std::pair Tarjan::indexify(TypeId ty) { - ty = follow(ty); + ty = log->follow(ty); bool fresh = !typeToIndex.contains(ty); int& index = typeToIndex[ty]; @@ -97,7 +98,7 @@ std::pair Tarjan::indexify(TypeId ty) std::pair Tarjan::indexify(TypePackId tp) { - tp = follow(tp); + tp = log->follow(tp); bool fresh = !packToIndex.contains(tp); int& index = packToIndex[tp]; @@ -115,7 +116,7 @@ std::pair Tarjan::indexify(TypePackId tp) void Tarjan::visitChild(TypeId ty) { - ty = follow(ty); + ty = log->follow(ty); edgesTy.push_back(ty); edgesTp.push_back(nullptr); @@ -123,7 +124,7 @@ void Tarjan::visitChild(TypeId ty) void Tarjan::visitChild(TypePackId tp) { - tp = follow(tp); + tp = log->follow(tp); edgesTy.push_back(nullptr); edgesTp.push_back(tp); @@ -243,7 +244,7 @@ void Tarjan::clear() TarjanResult Tarjan::visitRoot(TypeId ty) { childCount = 0; - ty = follow(ty); + ty = log->follow(ty); clear(); auto [index, fresh] = indexify(ty); @@ -254,7 +255,7 @@ TarjanResult Tarjan::visitRoot(TypeId ty) TarjanResult Tarjan::visitRoot(TypePackId tp) { childCount = 0; - tp = follow(tp); + tp = log->follow(tp); clear(); auto [index, fresh] = indexify(tp); @@ -325,7 +326,7 @@ TarjanResult FindDirty::findDirty(TypePackId tp) std::optional Substitution::substitute(TypeId ty) { - ty = follow(ty); + ty = log->follow(ty); newTypes.clear(); newPacks.clear(); @@ -345,7 +346,7 @@ std::optional Substitution::substitute(TypeId ty) std::optional Substitution::substitute(TypePackId tp) { - tp = follow(tp); + tp = log->follow(tp); newTypes.clear(); newPacks.clear(); @@ -365,11 +366,11 @@ std::optional Substitution::substitute(TypePackId tp) TypeId Substitution::clone(TypeId ty) { - ty = follow(ty); + ty = log->follow(ty); TypeId result = ty; - if (const FunctionTypeVar* ftv = get(ty)) + if (const FunctionTypeVar* ftv = log->getMutable(ty)) { FunctionTypeVar clone = FunctionTypeVar{ftv->level, ftv->argTypes, ftv->retType, ftv->definition, ftv->hasSelf}; clone.generics = ftv->generics; @@ -379,7 +380,7 @@ TypeId Substitution::clone(TypeId ty) clone.argNames = ftv->argNames; result = addType(std::move(clone)); } - else if (const TableTypeVar* ttv = get(ty)) + else if (const TableTypeVar* ttv = log->getMutable(ty)) { LUAU_ASSERT(!ttv->boundTo); TableTypeVar clone = TableTypeVar{ttv->props, ttv->indexer, ttv->level, ttv->state}; @@ -392,19 +393,19 @@ TypeId Substitution::clone(TypeId ty) clone.tags = ttv->tags; result = addType(std::move(clone)); } - else if (const MetatableTypeVar* mtv = get(ty)) + else if (const MetatableTypeVar* mtv = log->getMutable(ty)) { MetatableTypeVar clone = MetatableTypeVar{mtv->table, mtv->metatable}; clone.syntheticName = mtv->syntheticName; result = addType(std::move(clone)); } - else if (const UnionTypeVar* utv = get(ty)) + else if (const UnionTypeVar* utv = log->getMutable(ty)) { UnionTypeVar clone; clone.options = utv->options; result = addType(std::move(clone)); } - else if (const IntersectionTypeVar* itv = get(ty)) + else if (const IntersectionTypeVar* itv = log->getMutable(ty)) { IntersectionTypeVar clone; clone.parts = itv->parts; @@ -417,15 +418,15 @@ TypeId Substitution::clone(TypeId ty) TypePackId Substitution::clone(TypePackId tp) { - tp = follow(tp); - if (const TypePack* tpp = get(tp)) + tp = log->follow(tp); + if (const TypePack* tpp = log->getMutable(tp)) { TypePack clone; clone.head = tpp->head; clone.tail = tpp->tail; return addTypePack(std::move(clone)); } - else if (const VariadicTypePack* vtp = get(tp)) + else if (const VariadicTypePack* vtp = log->getMutable(tp)) { VariadicTypePack clone; clone.ty = vtp->ty; @@ -437,7 +438,7 @@ TypePackId Substitution::clone(TypePackId tp) void Substitution::foundDirty(TypeId ty) { - ty = follow(ty); + ty = log->follow(ty); if (isDirty(ty)) newTypes[ty] = clean(ty); else @@ -446,7 +447,7 @@ void Substitution::foundDirty(TypeId ty) void Substitution::foundDirty(TypePackId tp) { - tp = follow(tp); + tp = log->follow(tp); if (isDirty(tp)) newPacks[tp] = clean(tp); else @@ -455,7 +456,7 @@ void Substitution::foundDirty(TypePackId tp) TypeId Substitution::replace(TypeId ty) { - ty = follow(ty); + ty = log->follow(ty); if (TypeId* prevTy = newTypes.find(ty)) return *prevTy; else @@ -464,7 +465,7 @@ TypeId Substitution::replace(TypeId ty) TypePackId Substitution::replace(TypePackId tp) { - tp = follow(tp); + tp = log->follow(tp); if (TypePackId* prevTp = newPacks.find(tp)) return *prevTp; else @@ -473,7 +474,7 @@ TypePackId Substitution::replace(TypePackId tp) void Substitution::replaceChildren(TypeId ty) { - ty = follow(ty); + ty = log->follow(ty); if (ignoreChildren(ty)) return; @@ -519,7 +520,7 @@ void Substitution::replaceChildren(TypeId ty) void Substitution::replaceChildren(TypePackId tp) { - tp = follow(tp); + tp = log->follow(tp); if (ignoreChildren(tp)) return; diff --git a/Analysis/src/TxnLog.cpp b/Analysis/src/TxnLog.cpp index a46ac0c3..0968a4c1 100644 --- a/Analysis/src/TxnLog.cpp +++ b/Analysis/src/TxnLog.cpp @@ -1,6 +1,7 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #include "Luau/TxnLog.h" +#include "Luau/ToString.h" #include "Luau/TypePack.h" #include @@ -80,6 +81,56 @@ void DEPRECATED_TxnLog::popSeen(TypeId lhs, TypeId rhs) sharedSeen->pop_back(); } +const std::string nullPendingResult = ""; + +std::string toString(PendingType* pending) +{ + if (pending == nullptr) + return nullPendingResult; + + return toString(pending->pending); +} + +std::string dump(PendingType* pending) +{ + if (pending == nullptr) + { + printf("%s\n", nullPendingResult.c_str()); + return nullPendingResult; + } + + ToStringOptions opts; + opts.exhaustive = true; + opts.functionTypeArguments = true; + std::string result = toString(pending->pending, opts); + printf("%s\n", result.c_str()); + return result; +} + +std::string toString(PendingTypePack* pending) +{ + if (pending == nullptr) + return nullPendingResult; + + return toString(pending->pending); +} + +std::string dump(PendingTypePack* pending) +{ + if (pending == nullptr) + { + printf("%s\n", nullPendingResult.c_str()); + return nullPendingResult; + } + + ToStringOptions opts; + opts.exhaustive = true; + opts.functionTypeArguments = true; + std::string result = toString(pending->pending, opts); + printf("%s\n", result.c_str()); + return result; +} + static const TxnLog emptyLog; const TxnLog* TxnLog::empty() @@ -199,8 +250,6 @@ PendingTypePack* TxnLog::queue(TypePackId tp) PendingType* TxnLog::pending(TypeId ty) const { - LUAU_ASSERT(FFlag::LuauUseCommittingTxnLog); - for (const TxnLog* current = this; current; current = current->parent) { if (auto it = current->typeVarChanges.find(ty); it != current->typeVarChanges.end()) @@ -212,8 +261,6 @@ PendingType* TxnLog::pending(TypeId ty) const PendingTypePack* TxnLog::pending(TypePackId tp) const { - LUAU_ASSERT(FFlag::LuauUseCommittingTxnLog); - for (const TxnLog* current = this; current; current = current->parent) { if (auto it = current->typePackChanges.find(tp); it != current->typePackChanges.end()) @@ -225,8 +272,6 @@ PendingTypePack* TxnLog::pending(TypePackId tp) const PendingType* TxnLog::replace(TypeId ty, TypeVar replacement) { - LUAU_ASSERT(FFlag::LuauUseCommittingTxnLog); - PendingType* newTy = queue(ty); newTy->pending = replacement; return newTy; @@ -234,8 +279,6 @@ PendingType* TxnLog::replace(TypeId ty, TypeVar replacement) PendingTypePack* TxnLog::replace(TypePackId tp, TypePackVar replacement) { - LUAU_ASSERT(FFlag::LuauUseCommittingTxnLog); - PendingTypePack* newTp = queue(tp); newTp->pending = replacement; return newTp; @@ -243,7 +286,6 @@ PendingTypePack* TxnLog::replace(TypePackId tp, TypePackVar replacement) PendingType* TxnLog::bindTable(TypeId ty, std::optional newBoundTo) { - LUAU_ASSERT(FFlag::LuauUseCommittingTxnLog); LUAU_ASSERT(get(ty)); PendingType* newTy = queue(ty); @@ -255,7 +297,6 @@ PendingType* TxnLog::bindTable(TypeId ty, std::optional newBoundTo) PendingType* TxnLog::changeLevel(TypeId ty, TypeLevel newLevel) { - LUAU_ASSERT(FFlag::LuauUseCommittingTxnLog); LUAU_ASSERT(get(ty) || get(ty) || get(ty)); PendingType* newTy = queue(ty); @@ -278,7 +319,6 @@ PendingType* TxnLog::changeLevel(TypeId ty, TypeLevel newLevel) PendingTypePack* TxnLog::changeLevel(TypePackId tp, TypeLevel newLevel) { - LUAU_ASSERT(FFlag::LuauUseCommittingTxnLog); LUAU_ASSERT(get(tp)); PendingTypePack* newTp = queue(tp); @@ -292,7 +332,6 @@ PendingTypePack* TxnLog::changeLevel(TypePackId tp, TypeLevel newLevel) PendingType* TxnLog::changeIndexer(TypeId ty, std::optional indexer) { - LUAU_ASSERT(FFlag::LuauUseCommittingTxnLog); LUAU_ASSERT(get(ty)); PendingType* newTy = queue(ty); @@ -306,8 +345,6 @@ PendingType* TxnLog::changeIndexer(TypeId ty, std::optional indexe std::optional TxnLog::getLevel(TypeId ty) const { - LUAU_ASSERT(FFlag::LuauUseCommittingTxnLog); - if (FreeTypeVar* ftv = getMutable(ty)) return ftv->level; else if (TableTypeVar* ttv = getMutable(ty); ttv && (ttv->state == TableState::Free || ttv->state == TableState::Generic)) @@ -318,10 +355,8 @@ std::optional TxnLog::getLevel(TypeId ty) const return std::nullopt; } -TypeId TxnLog::follow(TypeId ty) +TypeId TxnLog::follow(TypeId ty) const { - LUAU_ASSERT(FFlag::LuauUseCommittingTxnLog); - return Luau::follow(ty, [this](TypeId ty) { PendingType* state = this->pending(ty); @@ -337,8 +372,6 @@ TypeId TxnLog::follow(TypeId ty) TypePackId TxnLog::follow(TypePackId tp) const { - LUAU_ASSERT(FFlag::LuauUseCommittingTxnLog); - return Luau::follow(tp, [this](TypePackId tp) { PendingTypePack* state = this->pending(tp); diff --git a/Analysis/src/TypeInfer.cpp b/Analysis/src/TypeInfer.cpp index d6b3b5b3..b9096d2e 100644 --- a/Analysis/src/TypeInfer.cpp +++ b/Analysis/src/TypeInfer.cpp @@ -32,6 +32,7 @@ LUAU_FASTFLAGVARIABLE(LuauRecursiveTypeParameterRestriction, false) LUAU_FASTFLAGVARIABLE(LuauIfElseBranchTypeUnion, false) LUAU_FASTFLAGVARIABLE(LuauIfElseExpectedType2, false) LUAU_FASTFLAGVARIABLE(LuauLengthOnCompositeType, false) +LUAU_FASTFLAGVARIABLE(LuauNoSealedTypeMod, false) LUAU_FASTFLAGVARIABLE(LuauQuantifyInPlace2, false) LUAU_FASTFLAGVARIABLE(LuauSealExports, false) LUAU_FASTFLAGVARIABLE(LuauSingletonTypes, false) @@ -40,13 +41,12 @@ LUAU_FASTFLAGVARIABLE(LuauTypeAliasDefaults, false) LUAU_FASTFLAGVARIABLE(LuauExpectedTypesOfProperties, false) LUAU_FASTFLAGVARIABLE(LuauErrorRecoveryType, false) LUAU_FASTFLAGVARIABLE(LuauPropertiesGetExpectedType, false) -LUAU_FASTFLAGVARIABLE(LuauLValueAsKey, false) -LUAU_FASTFLAGVARIABLE(LuauRefiLookupFromIndexExpr, false) LUAU_FASTFLAGVARIABLE(LuauPerModuleUnificationCache, false) LUAU_FASTFLAGVARIABLE(LuauProperTypeLevels, false) LUAU_FASTFLAGVARIABLE(LuauAscribeCorrectLevelToInferredProperitesOfFreeTables, false) -LUAU_FASTFLAGVARIABLE(LuauBidirectionalAsExpr, false) +LUAU_FASTFLAG(LuauUnionTagMatchFix) LUAU_FASTFLAGVARIABLE(LuauUnsealedTableLiteral, false) +LUAU_FASTFLAGVARIABLE(LuauAssertStripsFalsyTypes, false) namespace Luau { @@ -1117,7 +1117,7 @@ void TypeChecker::check(const ScopePtr& scope, TypeId ty, const ScopePtr& funSco ty = follow(ty); - if (tableSelf && !selfTy->persistent) + if (tableSelf && (FFlag::LuauNoSealedTypeMod ? tableSelf->state != TableState::Sealed : !selfTy->persistent)) tableSelf->props[indexName->index.value] = {ty, /* deprecated */ false, {}, indexName->indexLocation}; const FunctionTypeVar* funTy = get(ty); @@ -1130,7 +1130,7 @@ void TypeChecker::check(const ScopePtr& scope, TypeId ty, const ScopePtr& funSco checkFunctionBody(funScope, ty, *function.func); - if (tableSelf && !selfTy->persistent) + if (tableSelf && (FFlag::LuauNoSealedTypeMod ? tableSelf->state != TableState::Sealed : !selfTy->persistent)) tableSelf->props[indexName->index.value] = { follow(quantify(funScope, ty, indexName->indexLocation)), /* deprecated */ false, {}, indexName->indexLocation}; } @@ -1657,7 +1657,7 @@ std::optional TypeChecker::getIndexTypeFromType( RecursionLimiter _rl(&recursionCount, FInt::LuauTypeInferRecursionLimit); // Not needed when we normalize types. - if (FFlag::LuauLValueAsKey && get(follow(t))) + if (get(follow(t))) return t; if (std::optional ty = getIndexTypeFromType(scope, t, name, location, false)) @@ -1802,12 +1802,9 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprIn { TypeId ty = checkLValue(scope, expr); - if (FFlag::LuauRefiLookupFromIndexExpr) - { - if (std::optional lvalue = tryGetLValue(expr)) - if (std::optional refiTy = resolveLValue(scope, *lvalue)) - return {*refiTy, {TruthyPredicate{std::move(*lvalue), expr.location}}}; - } + if (std::optional lvalue = tryGetLValue(expr)) + if (std::optional refiTy = resolveLValue(scope, *lvalue)) + return {*refiTy, {TruthyPredicate{std::move(*lvalue), expr.location}}}; return {ty}; } @@ -2471,33 +2468,28 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprBi { if (expr.op == AstExprBinary::And) { - ExprResult lhs = checkExpr(scope, *expr.left); + auto [lhsTy, lhsPredicates] = checkExpr(scope, *expr.left); - // We can't just report errors here. - // This function can be called from AstStatLocal or from AstStatIf, or even from AstExprBinary (and others). - // For now, ignore the errors returned by the predicate resolver. - // We may need an extra property for each predicate set that indicates it has been resolved. - // Requires a slight modification to the data structure. ScopePtr innerScope = childScope(scope, expr.location); - resolve(lhs.predicates, innerScope, true); + resolve(lhsPredicates, innerScope, true); - ExprResult rhs = checkExpr(innerScope, *expr.right); + auto [rhsTy, rhsPredicates] = checkExpr(innerScope, *expr.right); - return {checkBinaryOperation(FFlag::LuauDiscriminableUnions ? scope : innerScope, expr, lhs.type, rhs.type), - {AndPredicate{std::move(lhs.predicates), std::move(rhs.predicates)}}}; + return {checkBinaryOperation(FFlag::LuauDiscriminableUnions ? scope : innerScope, expr, lhsTy, rhsTy), + {AndPredicate{std::move(lhsPredicates), std::move(rhsPredicates)}}}; } else if (expr.op == AstExprBinary::Or) { - ExprResult lhs = checkExpr(scope, *expr.left); + auto [lhsTy, lhsPredicates] = checkExpr(scope, *expr.left); ScopePtr innerScope = childScope(scope, expr.location); - resolve(lhs.predicates, innerScope, false); + resolve(lhsPredicates, innerScope, false); - ExprResult rhs = checkExpr(innerScope, *expr.right); + auto [rhsTy, rhsPredicates] = checkExpr(innerScope, *expr.right); - // Because of C++, I'm not sure if lhs.predicates was not moved out by the time we call checkBinaryOperation. - TypeId result = checkBinaryOperation(FFlag::LuauDiscriminableUnions ? scope : innerScope, expr, lhs.type, rhs.type, lhs.predicates); - return {result, {OrPredicate{std::move(lhs.predicates), std::move(rhs.predicates)}}}; + // Because of C++, I'm not sure if lhsPredicates was not moved out by the time we call checkBinaryOperation. + TypeId result = checkBinaryOperation(FFlag::LuauDiscriminableUnions ? scope : innerScope, expr, lhsTy, rhsTy, lhsPredicates); + return {result, {OrPredicate{std::move(lhsPredicates), std::move(rhsPredicates)}}}; } else if (expr.op == AstExprBinary::CompareEq || expr.op == AstExprBinary::CompareNe) { @@ -2535,27 +2527,15 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprTy TypeId annotationType = resolveType(scope, *expr.annotation); ExprResult result = checkExpr(scope, *expr.expr, annotationType); - if (FFlag::LuauBidirectionalAsExpr) - { - // Note: As an optimization, we try 'number <: number | string' first, as that is the more likely case. - if (canUnify(annotationType, result.type, expr.location).empty()) - return {annotationType, std::move(result.predicates)}; - - if (canUnify(result.type, annotationType, expr.location).empty()) - return {annotationType, std::move(result.predicates)}; - - reportError(expr.location, TypesAreUnrelated{result.type, annotationType}); - return {errorRecoveryType(annotationType), std::move(result.predicates)}; - } - else - { - ErrorVec errorVec = canUnify(annotationType, result.type, expr.location); - reportErrors(errorVec); - if (!errorVec.empty()) - annotationType = errorRecoveryType(annotationType); - + // Note: As an optimization, we try 'number <: number | string' first, as that is the more likely case. + if (canUnify(annotationType, result.type, expr.location).empty()) return {annotationType, std::move(result.predicates)}; - } + + if (canUnify(result.type, annotationType, expr.location).empty()) + return {annotationType, std::move(result.predicates)}; + + reportError(expr.location, TypesAreUnrelated{result.type, annotationType}); + return {errorRecoveryType(annotationType), std::move(result.predicates)}; } ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprError& expr) @@ -4295,7 +4275,7 @@ void TypeChecker::unifyWithInstantiationIfNeeded(const ScopePtr& scope, TypeId s child.tryUnify(subTy, superTy, /*isFunctionCall*/ false); if (!child.errors.empty()) { - TypeId instantiated = instantiate(scope, subTy, state.location); + TypeId instantiated = instantiate(scope, subTy, state.location, &child.log); if (subTy == instantiated) { // Instantiating the argument made no difference, so just report any child errors @@ -4330,7 +4310,7 @@ void TypeChecker::unifyWithInstantiationIfNeeded(const ScopePtr& scope, TypeId s bool Instantiation::isDirty(TypeId ty) { - if (get(ty)) + if (log->getMutable(ty)) return true; else return false; @@ -4343,7 +4323,7 @@ bool Instantiation::isDirty(TypePackId tp) bool Instantiation::ignoreChildren(TypeId ty) { - if (get(ty)) + if (log->getMutable(ty)) return true; else return false; @@ -4351,7 +4331,7 @@ bool Instantiation::ignoreChildren(TypeId ty) TypeId Instantiation::clean(TypeId ty) { - const FunctionTypeVar* ftv = get(ty); + const FunctionTypeVar* ftv = log->getMutable(ty); LUAU_ASSERT(ftv); FunctionTypeVar clone = FunctionTypeVar{level, ftv->argTypes, ftv->retType, ftv->definition, ftv->hasSelf}; @@ -4362,6 +4342,7 @@ TypeId Instantiation::clean(TypeId ty) // Annoyingly, we have to do this even if there are no generics, // to replace any generic tables. + replaceGenerics.log = log; replaceGenerics.level = level; replaceGenerics.currentModule = currentModule; replaceGenerics.generics.assign(ftv->generics.begin(), ftv->generics.end()); @@ -4383,7 +4364,7 @@ TypePackId Instantiation::clean(TypePackId tp) bool ReplaceGenerics::ignoreChildren(TypeId ty) { - if (const FunctionTypeVar* ftv = get(ty)) + if (const FunctionTypeVar* ftv = log->getMutable(ty)) // We aren't recursing in the case of a generic function which // binds the same generics. This can happen if, for example, there's recursive types. // If T = (a,T)->T then instantiating T should produce T' = (X,T)->T not T' = (X,T')->T'. @@ -4396,9 +4377,9 @@ bool ReplaceGenerics::ignoreChildren(TypeId ty) bool ReplaceGenerics::isDirty(TypeId ty) { - if (const TableTypeVar* ttv = get(ty)) + if (const TableTypeVar* ttv = log->getMutable(ty)) return ttv->state == TableState::Generic; - else if (get(ty)) + else if (log->getMutable(ty)) return std::find(generics.begin(), generics.end(), ty) != generics.end(); else return false; @@ -4406,7 +4387,7 @@ bool ReplaceGenerics::isDirty(TypeId ty) bool ReplaceGenerics::isDirty(TypePackId tp) { - if (get(tp)) + if (log->getMutable(tp)) return std::find(genericPacks.begin(), genericPacks.end(), tp) != genericPacks.end(); else return false; @@ -4415,7 +4396,7 @@ bool ReplaceGenerics::isDirty(TypePackId tp) TypeId ReplaceGenerics::clean(TypeId ty) { LUAU_ASSERT(isDirty(ty)); - if (const TableTypeVar* ttv = get(ty)) + if (const TableTypeVar* ttv = log->getMutable(ty)) { TableTypeVar clone = TableTypeVar{ttv->props, ttv->indexer, level, TableState::Free}; clone.methodDefinitionLocations = ttv->methodDefinitionLocations; @@ -4434,9 +4415,9 @@ TypePackId ReplaceGenerics::clean(TypePackId tp) bool Quantification::isDirty(TypeId ty) { - if (const TableTypeVar* ttv = get(ty)) + if (const TableTypeVar* ttv = log->getMutable(ty)) return level.subsumes(ttv->level) && ((ttv->state == TableState::Free) || (ttv->state == TableState::Unsealed)); - else if (const FreeTypeVar* ftv = get(ty)) + else if (const FreeTypeVar* ftv = log->getMutable(ty)) return level.subsumes(ftv->level); else return false; @@ -4444,7 +4425,7 @@ bool Quantification::isDirty(TypeId ty) bool Quantification::isDirty(TypePackId tp) { - if (const FreeTypePack* ftv = get(tp)) + if (const FreeTypePack* ftv = log->getMutable(tp)) return level.subsumes(ftv->level); else return false; @@ -4453,7 +4434,7 @@ bool Quantification::isDirty(TypePackId tp) TypeId Quantification::clean(TypeId ty) { LUAU_ASSERT(isDirty(ty)); - if (const TableTypeVar* ttv = get(ty)) + if (const TableTypeVar* ttv = log->getMutable(ty)) { TableState state = (ttv->state == TableState::Unsealed ? TableState::Sealed : TableState::Generic); TableTypeVar clone = TableTypeVar{ttv->props, ttv->indexer, level, state}; @@ -4479,9 +4460,9 @@ TypePackId Quantification::clean(TypePackId tp) bool Anyification::isDirty(TypeId ty) { - if (const TableTypeVar* ttv = get(ty)) + if (const TableTypeVar* ttv = log->getMutable(ty)) return (ttv->state == TableState::Free || (FFlag::LuauSealExports && ttv->state == TableState::Unsealed)); - else if (get(ty)) + else if (log->getMutable(ty)) return true; else return false; @@ -4489,7 +4470,7 @@ bool Anyification::isDirty(TypeId ty) bool Anyification::isDirty(TypePackId tp) { - if (get(tp)) + if (log->getMutable(tp)) return true; else return false; @@ -4498,7 +4479,7 @@ bool Anyification::isDirty(TypePackId tp) TypeId Anyification::clean(TypeId ty) { LUAU_ASSERT(isDirty(ty)); - if (const TableTypeVar* ttv = get(ty)) + if (const TableTypeVar* ttv = log->getMutable(ty)) { TableTypeVar clone = TableTypeVar{ttv->props, ttv->indexer, ttv->level, TableState::Sealed}; clone.methodDefinitionLocations = ttv->methodDefinitionLocations; @@ -4535,6 +4516,7 @@ TypeId TypeChecker::quantify(const ScopePtr& scope, TypeId ty, Location location return ty; } + quantification.log = TxnLog::empty(); quantification.level = scope->level; quantification.generics.clear(); quantification.genericPacks.clear(); @@ -4558,8 +4540,11 @@ TypeId TypeChecker::quantify(const ScopePtr& scope, TypeId ty, Location location return *qty; } -TypeId TypeChecker::instantiate(const ScopePtr& scope, TypeId ty, Location location) +TypeId TypeChecker::instantiate(const ScopePtr& scope, TypeId ty, Location location, const TxnLog* log) { + LUAU_ASSERT(log != nullptr); + + instantiation.log = FFlag::LuauUseCommittingTxnLog ? log : TxnLog::empty(); instantiation.level = scope->level; instantiation.currentModule = currentModule; std::optional instantiated = instantiation.substitute(ty); @@ -4574,6 +4559,7 @@ TypeId TypeChecker::instantiate(const ScopePtr& scope, TypeId ty, Location locat TypeId TypeChecker::anyify(const ScopePtr& scope, TypeId ty, Location location) { + anyification.log = TxnLog::empty(); anyification.anyType = anyType; anyification.anyTypePack = anyTypePack; anyification.currentModule = currentModule; @@ -4589,6 +4575,7 @@ TypeId TypeChecker::anyify(const ScopePtr& scope, TypeId ty, Location location) TypePackId TypeChecker::anyify(const ScopePtr& scope, TypePackId ty, Location location) { + anyification.log = TxnLog::empty(); anyification.anyType = anyType; anyification.anyTypePack = anyTypePack; anyification.currentModule = currentModule; @@ -4660,7 +4647,7 @@ void TypeChecker::diagnoseMissingTableKey(UnknownProperty* utk, TypeErrorData& d } }; - if (auto ttv = getTableType(follow(utk->table))) + if (auto ttv = getTableType(FFlag::LuauUnionTagMatchFix ? utk->table : follow(utk->table))) accumulate(ttv->props); else if (auto ctv = get(follow(utk->table))) { @@ -4775,6 +4762,29 @@ TypePackId TypeChecker::errorRecoveryTypePack(TypePackId guess) return getSingletonTypes().errorRecoveryTypePack(guess); } +TypeIdPredicate TypeChecker::mkTruthyPredicate(bool sense) { + return [this, sense](TypeId ty) -> std::optional { + // any/error/free gets a special pass unconditionally because they can't be decided. + if (get(ty) || get(ty) || get(ty)) + return ty; + + // maps boolean primitive to the corresponding singleton equal to sense + if (isPrim(ty, PrimitiveTypeVar::Boolean)) + return singletonType(sense); + + // if we have boolean singleton, eliminate it if the sense doesn't match with that singleton + if (auto boolean = get(get(ty))) + return boolean->value == sense ? std::optional(ty) : std::nullopt; + + // if we have nil, eliminate it if sense is true, otherwise take it + if (isNil(ty)) + return sense ? std::nullopt : std::optional(ty); + + // at this point, anything else is kept if sense is true, or eliminated otherwise + return sense ? std::optional(ty) : std::nullopt; + }; +} + std::optional TypeChecker::filterMap(TypeId type, TypeIdPredicate predicate) { std::vector types = Luau::filterMap(type, predicate); @@ -4783,6 +4793,11 @@ std::optional TypeChecker::filterMap(TypeId type, TypeIdPredicate predic return std::nullopt; } +std::optional TypeChecker::pickTypesFromSense(TypeId type, bool sense) +{ + return filterMap(type, mkTruthyPredicate(sense)); +} + TypeId TypeChecker::addTV(TypeVar&& tv) { return currentModule->internalTypes.addType(std::move(tv)); @@ -4962,6 +4977,7 @@ TypeId TypeChecker::resolveType(const ScopePtr& scope, const AstType& annotation if (notEnoughParameters && hasDefaultParameters) { // 'applyTypeFunction' is used to substitute default types that reference previous generic types + applyTypeFunction.log = TxnLog::empty(); applyTypeFunction.typeArguments.clear(); applyTypeFunction.typePackArguments.clear(); applyTypeFunction.currentModule = currentModule; @@ -5293,6 +5309,7 @@ TypeId TypeChecker::instantiateTypeFun(const ScopePtr& scope, const TypeFun& tf, for (size_t i = 0; i < tf.typePackParams.size(); ++i) applyTypeFunction.typePackArguments[tf.typePackParams[i].tp] = typePackParams[i]; + applyTypeFunction.log = TxnLog::empty(); applyTypeFunction.currentModule = currentModule; applyTypeFunction.level = scope->level; applyTypeFunction.encounteredForwardedType = false; @@ -5507,9 +5524,6 @@ void TypeChecker::refineLValue(const LValue& lvalue, RefinementMap& refis, const std::optional TypeChecker::resolveLValue(const ScopePtr& scope, const LValue& lvalue) { - if (!FFlag::LuauLValueAsKey) - return DEPRECATED_resolveLValue(scope, lvalue); - // We want to be walking the Scope parents. // We'll also want to walk up the LValue path. As we do this, we need to save each LValue because we must walk back. // For example: @@ -5529,7 +5543,7 @@ std::optional TypeChecker::resolveLValue(const ScopePtr& scope, const LV const LValue* currentLValue = &lvalue; while (currentLValue) { - if (auto it = currentScope->refinements.NEW_refinements.find(*currentLValue); it != currentScope->refinements.NEW_refinements.end()) + if (auto it = currentScope->refinements.find(*currentLValue); it != currentScope->refinements.end()) { found = it->second; break; @@ -5576,43 +5590,9 @@ std::optional TypeChecker::resolveLValue(const ScopePtr& scope, const LV return std::nullopt; } -std::optional TypeChecker::DEPRECATED_resolveLValue(const ScopePtr& scope, const LValue& lvalue) -{ - auto [symbol, keys] = getFullName(lvalue); - - ScopePtr currentScope = scope; - while (currentScope) - { - if (auto it = currentScope->refinements.DEPRECATED_refinements.find(toString(lvalue)); it != currentScope->refinements.DEPRECATED_refinements.end()) - return it->second; - - // Should not be using scope->lookup. This is already recursive. - if (auto it = currentScope->bindings.find(symbol); it != currentScope->bindings.end()) - { - std::optional currentTy = it->second.typeId; - - for (std::string key : keys) - { - // TODO: This function probably doesn't need Location at all, or at least should hide the argument. - currentTy = getIndexTypeFromType(scope, *currentTy, key, Location(), false); - if (!currentTy) - break; - } - - return currentTy; - } - - currentScope = currentScope->parent; - } - - return std::nullopt; -} - std::optional TypeChecker::resolveLValue(const RefinementMap& refis, const ScopePtr& scope, const LValue& lvalue) { - if (auto it = refis.DEPRECATED_refinements.find(toString(lvalue)); it != refis.DEPRECATED_refinements.end()) - return it->second; - else if (auto it = refis.NEW_refinements.find(lvalue); it != refis.NEW_refinements.end()) + if (auto it = refis.find(lvalue); it != refis.end()) return it->second; else return resolveLValue(scope, lvalue); @@ -5661,35 +5641,46 @@ void TypeChecker::resolve(const Predicate& predicate, ErrorVec& errVec, Refineme void TypeChecker::resolve(const TruthyPredicate& truthyP, ErrorVec& errVec, RefinementMap& refis, const ScopePtr& scope, bool sense, bool fromOr) { - auto predicate = [sense](TypeId option) -> std::optional { - if (isUndecidable(option) || isBoolean(option) || isNil(option) != sense) - return option; - - return std::nullopt; - }; - - if (FFlag::LuauDiscriminableUnions) + if (FFlag::LuauAssertStripsFalsyTypes) { std::optional ty = resolveLValue(refis, scope, truthyP.lvalue); if (ty && fromOr) return addRefinement(refis, truthyP.lvalue, *ty); - refineLValue(truthyP.lvalue, refis, scope, predicate); + refineLValue(truthyP.lvalue, refis, scope, mkTruthyPredicate(sense)); } else { - std::optional ty = resolveLValue(refis, scope, truthyP.lvalue); - if (!ty) - return; + auto predicate = [sense](TypeId option) -> std::optional { + if (isUndecidable(option) || isBoolean(option) || isNil(option) != sense) + return option; - // This is a hack. :( - // Without this, the expression 'a or b' might refine 'b' to be falsy. - // I'm not yet sure how else to get this to do the right thing without this hack, so we'll do this for now in the meantime. - if (fromOr) - return addRefinement(refis, truthyP.lvalue, *ty); + return std::nullopt; + }; - if (std::optional result = filterMap(*ty, predicate)) - addRefinement(refis, truthyP.lvalue, *result); + if (FFlag::LuauDiscriminableUnions) + { + std::optional ty = resolveLValue(refis, scope, truthyP.lvalue); + if (ty && fromOr) + return addRefinement(refis, truthyP.lvalue, *ty); + + refineLValue(truthyP.lvalue, refis, scope, predicate); + } + else + { + std::optional ty = resolveLValue(refis, scope, truthyP.lvalue); + if (!ty) + return; + + // This is a hack. :( + // Without this, the expression 'a or b' might refine 'b' to be falsy. + // I'm not yet sure how else to get this to do the right thing without this hack, so we'll do this for now in the meantime. + if (fromOr) + return addRefinement(refis, truthyP.lvalue, *ty); + + if (std::optional result = filterMap(*ty, predicate)) + addRefinement(refis, truthyP.lvalue, *result); + } } } diff --git a/Analysis/src/TypeVar.cpp b/Analysis/src/TypeVar.cpp index 5b162b31..2321eafd 100644 --- a/Analysis/src/TypeVar.cpp +++ b/Analysis/src/TypeVar.cpp @@ -27,6 +27,7 @@ LUAU_FASTFLAG(LuauLengthOnCompositeType) LUAU_FASTFLAGVARIABLE(LuauMetatableAreEqualRecursion, false) LUAU_FASTFLAGVARIABLE(LuauRefactorTypeVarQuestions, false) LUAU_FASTFLAG(LuauErrorRecoveryType) +LUAU_FASTFLAG(LuauUnionTagMatchFix) namespace Luau { @@ -288,10 +289,13 @@ std::optional getMetatable(TypeId type) const TableTypeVar* getTableType(TypeId type) { + if (FFlag::LuauUnionTagMatchFix) + type = follow(type); + if (const TableTypeVar* ttv = get(type)) return ttv; else if (const MetatableTypeVar* mtv = get(type)) - return get(mtv->table); + return get(FFlag::LuauUnionTagMatchFix ? follow(mtv->table) : mtv->table); else return nullptr; } @@ -308,7 +312,7 @@ const std::string* getName(TypeId type) { if (mtv->syntheticName) return &*mtv->syntheticName; - type = mtv->table; + type = FFlag::LuauUnionTagMatchFix ? follow(mtv->table) : mtv->table; } if (auto ttv = get(type)) diff --git a/Analysis/src/TypedAllocator.cpp b/Analysis/src/TypedAllocator.cpp index 1f7ef8c2..f037351e 100644 --- a/Analysis/src/TypedAllocator.cpp +++ b/Analysis/src/TypedAllocator.cpp @@ -20,7 +20,6 @@ const size_t kPageSize = sysconf(_SC_PAGESIZE); #include LUAU_FASTFLAG(DebugLuauFreezeArena) -LUAU_FASTFLAGVARIABLE(LuauTypedAllocatorZeroStart, false) namespace Luau { diff --git a/Analysis/src/Unifier.cpp b/Analysis/src/Unifier.cpp index 17d9bf58..89e4ae23 100644 --- a/Analysis/src/Unifier.cpp +++ b/Analysis/src/Unifier.cpp @@ -8,6 +8,7 @@ #include "Luau/TypeUtils.h" #include "Luau/TimeTrace.h" #include "Luau/VisitTypeVar.h" +#include "Luau/ToString.h" #include @@ -22,6 +23,7 @@ LUAU_FASTFLAG(LuauSingletonTypes) LUAU_FASTFLAG(LuauErrorRecoveryType); LUAU_FASTFLAG(LuauProperTypeLevels); LUAU_FASTFLAGVARIABLE(LuauUnifyPackTails, false) +LUAU_FASTFLAGVARIABLE(LuauUnionTagMatchFix, false) namespace Luau { @@ -225,19 +227,33 @@ static std::optional hasUnificationTooComplex(const ErrorVec& errors) // Used for tagged union matching heuristic, returns first singleton type field static std::optional> getTableMatchTag(TypeId type) { - type = follow(type); - - if (auto ttv = get(type)) + if (FFlag::LuauUnionTagMatchFix) { - for (auto&& [name, prop] : ttv->props) + if (auto ttv = getTableType(type)) { - if (auto sing = get(follow(prop.type))) - return {{name, sing}}; + for (auto&& [name, prop] : ttv->props) + { + if (auto sing = get(follow(prop.type))) + return {{name, sing}}; + } } } - else if (auto mttv = get(type)) + else { - return getTableMatchTag(mttv->table); + type = follow(type); + + if (auto ttv = get(type)) + { + for (auto&& [name, prop] : ttv->props) + { + if (auto sing = get(follow(prop.type))) + return {{name, sing}}; + } + } + else if (auto mttv = get(type)) + { + return getTableMatchTag(mttv->table); + } } return std::nullopt; @@ -508,245 +524,21 @@ void Unifier::tryUnify_(TypeId subTy, TypeId superTy, bool isFunctionCall, bool if (const UnionTypeVar* uv = FFlag::LuauUseCommittingTxnLog ? log.getMutable(subTy) : get(subTy)) { - // A | B <: T if A <: T and B <: T - bool failed = false; - std::optional unificationTooComplex; - std::optional firstFailedOption; - - size_t count = uv->options.size(); - size_t i = 0; - - for (TypeId type : uv->options) - { - Unifier innerState = makeChildUnifier(); - innerState.tryUnify_(type, superTy); - - if (auto e = hasUnificationTooComplex(innerState.errors)) - unificationTooComplex = e; - else if (!innerState.errors.empty()) - { - // 'nil' option is skipped from extended report because we present the type in a special way - 'T?' - if (!firstFailedOption && !isNil(type)) - firstFailedOption = {innerState.errors.front()}; - - failed = true; - } - - if (FFlag::LuauUseCommittingTxnLog) - { - if (i == count - 1) - { - log.concat(std::move(innerState.log)); - } - } - else - { - if (i != count - 1) - { - innerState.DEPRECATED_log.rollback(); - } - else - { - DEPRECATED_log.concat(std::move(innerState.DEPRECATED_log)); - } - } - - ++i; - } - - if (unificationTooComplex) - reportError(*unificationTooComplex); - else if (failed) - { - if (firstFailedOption) - reportError(TypeError{location, TypeMismatch{superTy, subTy, "Not all union options are compatible.", *firstFailedOption}}); - else - reportError(TypeError{location, TypeMismatch{superTy, subTy}}); - } + tryUnifyUnionWithType(subTy, uv, superTy); } else if (const UnionTypeVar* uv = FFlag::LuauUseCommittingTxnLog ? log.getMutable(superTy) : get(superTy)) { - // T <: A | B if T <: A or T <: B - bool found = false; - std::optional unificationTooComplex; - - size_t failedOptionCount = 0; - std::optional failedOption; - - bool foundHeuristic = false; - size_t startIndex = 0; - - if (const std::string* subName = getName(subTy)) - { - for (size_t i = 0; i < uv->options.size(); ++i) - { - const std::string* optionName = getName(uv->options[i]); - if (optionName && *optionName == *subName) - { - foundHeuristic = true; - startIndex = i; - break; - } - } - } - - if (auto subMatchTag = getTableMatchTag(subTy)) - { - for (size_t i = 0; i < uv->options.size(); ++i) - { - auto optionMatchTag = getTableMatchTag(uv->options[i]); - if (optionMatchTag && optionMatchTag->first == subMatchTag->first && *optionMatchTag->second == *subMatchTag->second) - { - foundHeuristic = true; - startIndex = i; - break; - } - } - } - - if (!foundHeuristic && cacheEnabled) - { - for (size_t i = 0; i < uv->options.size(); ++i) - { - TypeId type = uv->options[i]; - - if (cache.contains({type, subTy}) && (variance == Covariant || cache.contains({subTy, type}))) - { - startIndex = i; - break; - } - } - } - - for (size_t i = 0; i < uv->options.size(); ++i) - { - TypeId type = uv->options[(i + startIndex) % uv->options.size()]; - Unifier innerState = makeChildUnifier(); - innerState.tryUnify_(subTy, type, isFunctionCall); - - if (innerState.errors.empty()) - { - found = true; - if (FFlag::LuauUseCommittingTxnLog) - log.concat(std::move(innerState.log)); - else - DEPRECATED_log.concat(std::move(innerState.DEPRECATED_log)); - - break; - } - else if (auto e = hasUnificationTooComplex(innerState.errors)) - { - unificationTooComplex = e; - } - else if (!isNil(type)) - { - failedOptionCount++; - - if (!failedOption) - failedOption = {innerState.errors.front()}; - } - - if (!FFlag::LuauUseCommittingTxnLog) - innerState.DEPRECATED_log.rollback(); - } - - if (unificationTooComplex) - { - reportError(*unificationTooComplex); - } - else if (!found) - { - if ((failedOptionCount == 1 || foundHeuristic) && failedOption) - reportError( - TypeError{location, TypeMismatch{superTy, subTy, "None of the union options are compatible. For example:", *failedOption}}); - else - reportError(TypeError{location, TypeMismatch{superTy, subTy, "none of the union options are compatible"}}); - } + tryUnifyTypeWithUnion(subTy, superTy, uv, cacheEnabled, isFunctionCall); } else if (const IntersectionTypeVar* uv = FFlag::LuauUseCommittingTxnLog ? log.getMutable(superTy) : get(superTy)) { - std::optional unificationTooComplex; - std::optional firstFailedOption; - - // T <: A & B if A <: T and B <: T - for (TypeId type : uv->parts) - { - Unifier innerState = makeChildUnifier(); - innerState.tryUnify_(subTy, type, /*isFunctionCall*/ false, /*isIntersection*/ true); - - if (auto e = hasUnificationTooComplex(innerState.errors)) - unificationTooComplex = e; - else if (!innerState.errors.empty()) - { - if (!firstFailedOption) - firstFailedOption = {innerState.errors.front()}; - } - - if (FFlag::LuauUseCommittingTxnLog) - log.concat(std::move(innerState.log)); - else - DEPRECATED_log.concat(std::move(innerState.DEPRECATED_log)); - } - - if (unificationTooComplex) - reportError(*unificationTooComplex); - else if (firstFailedOption) - reportError(TypeError{location, TypeMismatch{superTy, subTy, "Not all intersection parts are compatible.", *firstFailedOption}}); + tryUnifyTypeWithIntersection(subTy, superTy, uv); } else if (const IntersectionTypeVar* uv = FFlag::LuauUseCommittingTxnLog ? log.getMutable(subTy) : get(subTy)) { - // A & B <: T if T <: A or T <: B - bool found = false; - std::optional unificationTooComplex; - - size_t startIndex = 0; - - if (cacheEnabled) - { - for (size_t i = 0; i < uv->parts.size(); ++i) - { - TypeId type = uv->parts[i]; - - if (cache.contains({superTy, type}) && (variance == Covariant || cache.contains({type, superTy}))) - { - startIndex = i; - break; - } - } - } - - for (size_t i = 0; i < uv->parts.size(); ++i) - { - TypeId type = uv->parts[(i + startIndex) % uv->parts.size()]; - Unifier innerState = makeChildUnifier(); - innerState.tryUnify_(type, superTy, isFunctionCall); - - if (innerState.errors.empty()) - { - found = true; - if (FFlag::LuauUseCommittingTxnLog) - log.concat(std::move(innerState.log)); - else - DEPRECATED_log.concat(std::move(innerState.DEPRECATED_log)); - break; - } - else if (auto e = hasUnificationTooComplex(innerState.errors)) - { - unificationTooComplex = e; - } - - if (!FFlag::LuauUseCommittingTxnLog) - innerState.DEPRECATED_log.rollback(); - } - - if (unificationTooComplex) - reportError(*unificationTooComplex); - else if (!found) - { - reportError(TypeError{location, TypeMismatch{superTy, subTy, "none of the intersection parts are compatible"}}); - } + tryUnifyIntersectionWithType(subTy, uv, superTy, cacheEnabled, isFunctionCall); } else if ((FFlag::LuauUseCommittingTxnLog && log.getMutable(superTy) && log.getMutable(subTy)) || (!FFlag::LuauUseCommittingTxnLog && get(superTy) && get(subTy))) @@ -797,6 +589,253 @@ void Unifier::tryUnify_(TypeId subTy, TypeId superTy, bool isFunctionCall, bool DEPRECATED_log.popSeen(superTy, subTy); } +void Unifier::tryUnifyUnionWithType(TypeId subTy, const UnionTypeVar* uv, TypeId superTy) +{ + // A | B <: T if A <: T and B <: T + bool failed = false; + std::optional unificationTooComplex; + std::optional firstFailedOption; + + size_t count = uv->options.size(); + size_t i = 0; + + for (TypeId type : uv->options) + { + Unifier innerState = makeChildUnifier(); + innerState.tryUnify_(type, superTy); + + if (auto e = hasUnificationTooComplex(innerState.errors)) + unificationTooComplex = e; + else if (!innerState.errors.empty()) + { + // 'nil' option is skipped from extended report because we present the type in a special way - 'T?' + if (!firstFailedOption && !isNil(type)) + firstFailedOption = {innerState.errors.front()}; + + failed = true; + } + + if (FFlag::LuauUseCommittingTxnLog) + { + if (i == count - 1) + { + log.concat(std::move(innerState.log)); + } + } + else + { + if (i != count - 1) + { + innerState.DEPRECATED_log.rollback(); + } + else + { + DEPRECATED_log.concat(std::move(innerState.DEPRECATED_log)); + } + } + + ++i; + } + + if (unificationTooComplex) + reportError(*unificationTooComplex); + else if (failed) + { + if (firstFailedOption) + reportError(TypeError{location, TypeMismatch{superTy, subTy, "Not all union options are compatible.", *firstFailedOption}}); + else + reportError(TypeError{location, TypeMismatch{superTy, subTy}}); + } +} + +void Unifier::tryUnifyTypeWithUnion(TypeId subTy, TypeId superTy, const UnionTypeVar* uv, bool cacheEnabled, bool isFunctionCall) +{ + // T <: A | B if T <: A or T <: B + bool found = false; + std::optional unificationTooComplex; + + size_t failedOptionCount = 0; + std::optional failedOption; + + bool foundHeuristic = false; + size_t startIndex = 0; + + if (const std::string* subName = getName(subTy)) + { + for (size_t i = 0; i < uv->options.size(); ++i) + { + const std::string* optionName = getName(uv->options[i]); + if (optionName && *optionName == *subName) + { + foundHeuristic = true; + startIndex = i; + break; + } + } + } + + if (auto subMatchTag = getTableMatchTag(subTy)) + { + for (size_t i = 0; i < uv->options.size(); ++i) + { + auto optionMatchTag = getTableMatchTag(uv->options[i]); + if (optionMatchTag && optionMatchTag->first == subMatchTag->first && *optionMatchTag->second == *subMatchTag->second) + { + foundHeuristic = true; + startIndex = i; + break; + } + } + } + + if (!foundHeuristic && cacheEnabled) + { + auto& cache = sharedState.cachedUnify; + + for (size_t i = 0; i < uv->options.size(); ++i) + { + TypeId type = uv->options[i]; + + if (cache.contains({type, subTy}) && (variance == Covariant || cache.contains({subTy, type}))) + { + startIndex = i; + break; + } + } + } + + for (size_t i = 0; i < uv->options.size(); ++i) + { + TypeId type = uv->options[(i + startIndex) % uv->options.size()]; + Unifier innerState = makeChildUnifier(); + innerState.tryUnify_(subTy, type, isFunctionCall); + + if (innerState.errors.empty()) + { + found = true; + if (FFlag::LuauUseCommittingTxnLog) + log.concat(std::move(innerState.log)); + else + DEPRECATED_log.concat(std::move(innerState.DEPRECATED_log)); + + break; + } + else if (auto e = hasUnificationTooComplex(innerState.errors)) + { + unificationTooComplex = e; + } + else if (!isNil(type)) + { + failedOptionCount++; + + if (!failedOption) + failedOption = {innerState.errors.front()}; + } + + if (!FFlag::LuauUseCommittingTxnLog) + innerState.DEPRECATED_log.rollback(); + } + + if (unificationTooComplex) + { + reportError(*unificationTooComplex); + } + else if (!found) + { + if ((failedOptionCount == 1 || foundHeuristic) && failedOption) + reportError(TypeError{location, TypeMismatch{superTy, subTy, "None of the union options are compatible. For example:", *failedOption}}); + else + reportError(TypeError{location, TypeMismatch{superTy, subTy, "none of the union options are compatible"}}); + } +} + +void Unifier::tryUnifyTypeWithIntersection(TypeId subTy, TypeId superTy, const IntersectionTypeVar* uv) +{ + std::optional unificationTooComplex; + std::optional firstFailedOption; + + // T <: A & B if A <: T and B <: T + for (TypeId type : uv->parts) + { + Unifier innerState = makeChildUnifier(); + innerState.tryUnify_(subTy, type, /*isFunctionCall*/ false, /*isIntersection*/ true); + + if (auto e = hasUnificationTooComplex(innerState.errors)) + unificationTooComplex = e; + else if (!innerState.errors.empty()) + { + if (!firstFailedOption) + firstFailedOption = {innerState.errors.front()}; + } + + if (FFlag::LuauUseCommittingTxnLog) + log.concat(std::move(innerState.log)); + else + DEPRECATED_log.concat(std::move(innerState.DEPRECATED_log)); + } + + if (unificationTooComplex) + reportError(*unificationTooComplex); + else if (firstFailedOption) + reportError(TypeError{location, TypeMismatch{superTy, subTy, "Not all intersection parts are compatible.", *firstFailedOption}}); +} + +void Unifier::tryUnifyIntersectionWithType(TypeId subTy, const IntersectionTypeVar* uv, TypeId superTy, bool cacheEnabled, bool isFunctionCall) +{ + // A & B <: T if T <: A or T <: B + bool found = false; + std::optional unificationTooComplex; + + size_t startIndex = 0; + + if (cacheEnabled) + { + auto& cache = sharedState.cachedUnify; + + for (size_t i = 0; i < uv->parts.size(); ++i) + { + TypeId type = uv->parts[i]; + + if (cache.contains({superTy, type}) && (variance == Covariant || cache.contains({type, superTy}))) + { + startIndex = i; + break; + } + } + } + + for (size_t i = 0; i < uv->parts.size(); ++i) + { + TypeId type = uv->parts[(i + startIndex) % uv->parts.size()]; + Unifier innerState = makeChildUnifier(); + innerState.tryUnify_(type, superTy, isFunctionCall); + + if (innerState.errors.empty()) + { + found = true; + if (FFlag::LuauUseCommittingTxnLog) + log.concat(std::move(innerState.log)); + else + DEPRECATED_log.concat(std::move(innerState.DEPRECATED_log)); + break; + } + else if (auto e = hasUnificationTooComplex(innerState.errors)) + { + unificationTooComplex = e; + } + + if (!FFlag::LuauUseCommittingTxnLog) + innerState.DEPRECATED_log.rollback(); + } + + if (unificationTooComplex) + reportError(*unificationTooComplex); + else if (!found) + { + reportError(TypeError{location, TypeMismatch{superTy, subTy, "none of the intersection parts are compatible"}}); + } +} + void Unifier::cacheResult(TypeId subTy, TypeId superTy) { bool* superTyInfo = sharedState.skipCacheForType.find(superTy); @@ -1119,8 +1158,8 @@ void Unifier::tryUnify_(TypePackId subTp, TypePackId superTp, bool isFunctionCal auto [superTypes, superTail] = logAwareFlatten(superTp, log); auto [subTypes, subTail] = logAwareFlatten(subTp, log); - bool noInfiniteGrowth = - (superTypes.size() != subTypes.size()) && (superTail && get(*superTail)) && (subTail && get(*subTail)); + bool noInfiniteGrowth = (superTypes.size() != subTypes.size()) && (superTail && log.getMutable(*superTail)) && + (subTail && log.getMutable(*subTail)); auto superIter = WeirdIter(superTp, log); auto subIter = WeirdIter(subTp, log); @@ -1667,6 +1706,13 @@ void Unifier::tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) TableTypeVar* superTable = getMutable(superTy); TableTypeVar* subTable = getMutable(subTy); + + if (FFlag::LuauUseCommittingTxnLog) + { + superTable = log.getMutable(superTy); + subTable = log.getMutable(subTy); + } + if (!superTable || !subTable) ice("passed non-table types to unifyTables"); @@ -1679,7 +1725,11 @@ void Unifier::tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) for (const auto& [propName, superProp] : superTable->props) { auto subIter = subTable->props.find(propName); - if (subIter == subTable->props.end() && !isOptional(superProp.type) && !get(follow(superProp.type))) + + bool isAny = + FFlag::LuauUseCommittingTxnLog ? log.getMutable(log.follow(superProp.type)) : get(follow(superProp.type)); + + if (subIter == subTable->props.end() && !isOptional(superProp.type) && !isAny) missingProperties.push_back(propName); } @@ -1697,7 +1747,10 @@ void Unifier::tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) for (const auto& [propName, subProp] : subTable->props) { auto superIter = superTable->props.find(propName); - if (superIter == superTable->props.end() && !isOptional(subProp.type) && !get(follow(subProp.type))) + + bool isAny = + FFlag::LuauUseCommittingTxnLog ? log.getMutable(log.follow(subProp.type)) : get(follow(subProp.type)); + if (superIter == superTable->props.end() && !isOptional(subProp.type) && !isAny) extraProperties.push_back(propName); } @@ -1775,6 +1828,7 @@ void Unifier::tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) TableTypeVar* ttv = getMutable(pendingSub); LUAU_ASSERT(ttv); ttv->props[name] = prop; + subTable = ttv; } else { @@ -1831,6 +1885,7 @@ void Unifier::tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) PendingType* pendingSuper = log.queue(superTy); TableTypeVar* pendingSuperTtv = getMutable(pendingSuper); pendingSuperTtv->props[name] = clone; + superTable = pendingSuperTtv; } else { @@ -1853,6 +1908,7 @@ void Unifier::tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) PendingType* pendingSuper = log.queue(superTy); TableTypeVar* pendingSuperTtv = getMutable(pendingSuper); pendingSuperTtv->props[name] = prop; + superTable = pendingSuperTtv; } else { @@ -1967,7 +2023,7 @@ void Unifier::tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) } else { - DEPRECATED_log(subTy); + DEPRECATED_log(subTable); subTable->boundTo = superTy; } } @@ -2408,8 +2464,7 @@ void Unifier::tryUnifyWithMetatable(TypeId subTy, TypeId superTy, bool reversed) if (auto e = hasUnificationTooComplex(innerState.errors)) reportError(*e); else if (!innerState.errors.empty()) - reportError( - TypeError{location, TypeMismatch{reversed ? subTy : superTy, reversed ? superTy : subTy, "", innerState.errors.front()}}); + reportError(TypeError{location, TypeMismatch{reversed ? subTy : superTy, reversed ? superTy : subTy, "", innerState.errors.front()}}); if (FFlag::LuauUseCommittingTxnLog) log.concat(std::move(innerState.log)); diff --git a/Ast/src/Parser.cpp b/Ast/src/Parser.cpp index 3c607d24..f559e2e0 100644 --- a/Ast/src/Parser.cpp +++ b/Ast/src/Parser.cpp @@ -10,7 +10,6 @@ // See docs/SyntaxChanges.md for an explanation. LUAU_FASTINTVARIABLE(LuauRecursionLimit, 1000) LUAU_FASTINTVARIABLE(LuauParseErrorLimit, 100) -LUAU_FASTFLAGVARIABLE(LuauFixAmbiguousErrorRecoveryInAssign, false) LUAU_FASTFLAGVARIABLE(LuauParseSingletonTypes, false) LUAU_FASTFLAGVARIABLE(LuauParseTypeAliasDefaults, false) LUAU_FASTFLAGVARIABLE(LuauParseRecoverTypePackEllipsis, false) @@ -957,7 +956,7 @@ AstStat* Parser::parseAssignment(AstExpr* initial) { nextLexeme(); - AstExpr* expr = parsePrimaryExpr(/* asStatement= */ FFlag::LuauFixAmbiguousErrorRecoveryInAssign); + AstExpr* expr = parsePrimaryExpr(/* asStatement= */ true); if (!isExprLValue(expr)) expr = reportExprError(expr->location, copy({expr}), "Assigned expression must be a variable or a field"); diff --git a/CLI/Coverage.cpp b/CLI/Coverage.cpp index 254df3f0..a509ab89 100644 --- a/CLI/Coverage.cpp +++ b/CLI/Coverage.cpp @@ -68,7 +68,7 @@ void coverageDump(const char* path) fprintf(f, "TN:\n"); - for (int fref: gCoverage.functions) + for (int fref : gCoverage.functions) { lua_getref(L, fref); diff --git a/CLI/FileUtils.cpp b/CLI/FileUtils.cpp index c6807022..fe005aec 100644 --- a/CLI/FileUtils.cpp +++ b/CLI/FileUtils.cpp @@ -77,7 +77,7 @@ std::optional readFile(const std::string& name) std::optional readStdin() { std::string result; - char buffer[4096] = { }; + char buffer[4096] = {}; while (fgets(buffer, sizeof(buffer), stdin) != nullptr) result.append(buffer); diff --git a/CLI/Repl.cpp b/CLI/Repl.cpp index ab0f0ed0..5af6b508 100644 --- a/CLI/Repl.cpp +++ b/CLI/Repl.cpp @@ -10,7 +10,7 @@ #include "Profiler.h" #include "Coverage.h" -#include "linenoise.hpp" +#include "isocline.h" #include @@ -240,9 +240,10 @@ std::string runCode(lua_State* L, const std::string& source) return std::string(); } -static void completeIndexer(lua_State* L, const char* editBuffer, size_t start, std::vector& completions) +static void completeIndexer(ic_completion_env_t* cenv, const char* editBuffer) { - std::string_view lookup = editBuffer + start; + auto* L = reinterpret_cast(ic_completion_arg(cenv)); + std::string_view lookup = editBuffer; char lastSep = 0; for (;;) @@ -268,13 +269,14 @@ static void completeIndexer(lua_State* L, const char* editBuffer, size_t start, if (!key.empty() && requiredValueType && Luau::startsWith(key, prefix)) { - std::string completion(editBuffer + std::string(key.substr(prefix.size()))); + std::string completedComponent(key.substr(prefix.size())); + std::string completion(editBuffer + completedComponent); if (valueType == LUA_TFUNCTION) { // Add an opening paren for function calls by default. completion += "("; } - completions.push_back(completion); + ic_add_completion_ex(cenv, completion.data(), key.data(), nullptr); } } lua_pop(L, 1); @@ -310,19 +312,23 @@ static void completeIndexer(lua_State* L, const char* editBuffer, size_t start, lua_pop(L, 1); } -static void completeRepl(lua_State* L, const char* editBuffer, std::vector& completions) +static bool isMethodOrFunctionChar(const char* s, long len) { - size_t start = strlen(editBuffer); - while (start > 0 && (isalnum(editBuffer[start - 1]) || editBuffer[start - 1] == '.' || editBuffer[start - 1] == ':' || editBuffer[start - 1] == '_')) - start--; + char c = *s; + return len == 1 && (isalnum(c) || c == '.' || c == ':' || c == '_'); +} + +static void completeRepl(ic_completion_env_t* cenv, const char* editBuffer) +{ + auto* L = reinterpret_cast(ic_completion_arg(cenv)); // look the value up in current global table first lua_pushvalue(L, LUA_GLOBALSINDEX); - completeIndexer(L, editBuffer, start, completions); + ic_complete_word(cenv, editBuffer, completeIndexer, isMethodOrFunctionChar); // and in actual global table after that lua_getglobal(L, "_G"); - completeIndexer(L, editBuffer, start, completions); + ic_complete_word(cenv, editBuffer, completeIndexer, isMethodOrFunctionChar); } struct LinenoiseScopedHistory @@ -341,13 +347,11 @@ struct LinenoiseScopedHistory } if (!historyFilepath.empty()) - linenoise::LoadHistory(historyFilepath.c_str()); + ic_set_history(historyFilepath.c_str(), -1 /* default entries (= 200) */); } ~LinenoiseScopedHistory() { - if (!historyFilepath.empty()) - linenoise::SaveHistory(historyFilepath.c_str()); } std::string historyFilepath; @@ -355,28 +359,32 @@ struct LinenoiseScopedHistory static void runReplImpl(lua_State* L) { - linenoise::SetCompletionCallback([L](const char* editBuffer, std::vector& completions) { - completeRepl(L, editBuffer, completions); - }); + ic_set_default_completer(completeRepl, L); + + // Make brace matching easier to see + ic_style_def("ic-bracematch", "teal"); + + // Prevent auto insertion of braces + ic_enable_brace_insertion(false); std::string buffer; LinenoiseScopedHistory scopedHistory; for (;;) { - bool quit = false; - std::string line = linenoise::Readline(buffer.empty() ? "> " : ">> ", quit); - if (quit) + const char* line = ic_readline(buffer.empty() ? "" : ">"); + if (!line) break; if (buffer.empty() && runCode(L, std::string("return ") + line) == std::string()) { - linenoise::AddHistory(line.c_str()); + ic_history_add(line); continue; } + if (!buffer.empty()) + buffer += "\n"; buffer += line; - buffer += " "; // linenoise doesn't work very well with multiline history entries std::string error = runCode(L, buffer); @@ -390,8 +398,9 @@ static void runReplImpl(lua_State* L) fprintf(stdout, "%s\n", error.c_str()); } - linenoise::AddHistory(buffer.c_str()); + ic_history_add(buffer.c_str()); buffer.clear(); + free((void*)line); } } diff --git a/CLI/ReplEntry.cpp b/CLI/ReplEntry.cpp index b3131712..75995e6a 100644 --- a/CLI/ReplEntry.cpp +++ b/CLI/ReplEntry.cpp @@ -1,5 +1,4 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details - #include "Repl.h" @@ -7,4 +6,4 @@ int main(int argc, char** argv) { return replMain(argc, argv); -} \ No newline at end of file +} diff --git a/CMakeLists.txt b/CMakeLists.txt index b9f7a9e1..881d3c3f 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -5,7 +5,7 @@ if(EXT_PLATFORM_STRING) endif() cmake_minimum_required(VERSION 3.0) -project(Luau LANGUAGES CXX) +project(Luau LANGUAGES CXX C) option(LUAU_BUILD_CLI "Build CLI" ON) option(LUAU_BUILD_TESTS "Build tests" ON) @@ -16,6 +16,7 @@ add_library(Luau.Ast STATIC) add_library(Luau.Compiler STATIC) add_library(Luau.Analysis STATIC) add_library(Luau.VM STATIC) +add_library(isocline STATIC) if(LUAU_BUILD_CLI) add_executable(Luau.Repl.CLI) @@ -52,6 +53,8 @@ target_link_libraries(Luau.Analysis PUBLIC Luau.Ast) target_compile_features(Luau.VM PRIVATE cxx_std_11) target_include_directories(Luau.VM PUBLIC VM/include) +target_include_directories(isocline PUBLIC extern/isocline/include) + set(LUAU_OPTIONS) if(MSVC) @@ -75,9 +78,16 @@ if(LUAU_BUILD_WEB) list(APPEND LUAU_OPTIONS -fexceptions) endif() +set(ISOCLINE_OPTIONS) + +if (NOT MSVC) + list(APPEND ISOCLINE_OPTIONS -Wno-unused-function) +endif() + target_compile_options(Luau.Ast PRIVATE ${LUAU_OPTIONS}) target_compile_options(Luau.Analysis PRIVATE ${LUAU_OPTIONS}) target_compile_options(Luau.VM PRIVATE ${LUAU_OPTIONS}) +target_compile_options(isocline PRIVATE ${LUAU_OPTIONS} ${ISOCLINE_OPTIONS}) if (MSVC AND MSVC_VERSION GREATER_EQUAL 1924) # disable partial redundancy elimination which regresses interpreter codegen substantially in VS2022: @@ -89,8 +99,9 @@ if(LUAU_BUILD_CLI) target_compile_options(Luau.Repl.CLI PRIVATE ${LUAU_OPTIONS}) target_compile_options(Luau.Analyze.CLI PRIVATE ${LUAU_OPTIONS}) - target_include_directories(Luau.Repl.CLI PRIVATE extern) - target_link_libraries(Luau.Repl.CLI PRIVATE Luau.Compiler Luau.VM) + target_include_directories(Luau.Repl.CLI PRIVATE extern extern/isocline/include) + + target_link_libraries(Luau.Repl.CLI PRIVATE Luau.Compiler Luau.VM isocline) if(UNIX) find_library(LIBPTHREAD pthread) @@ -113,7 +124,7 @@ if(LUAU_BUILD_TESTS) target_compile_options(Luau.CLI.Test PRIVATE ${LUAU_OPTIONS}) target_include_directories(Luau.CLI.Test PRIVATE extern CLI) - target_link_libraries(Luau.CLI.Test PRIVATE Luau.Compiler Luau.VM) + target_link_libraries(Luau.CLI.Test PRIVATE Luau.Compiler Luau.VM isocline) if(UNIX) find_library(LIBPTHREAD pthread) if (LIBPTHREAD) diff --git a/Compiler/src/Builtins.cpp b/Compiler/src/Builtins.cpp index a907271c..26360c49 100644 --- a/Compiler/src/Builtins.cpp +++ b/Compiler/src/Builtins.cpp @@ -4,7 +4,7 @@ #include "Luau/Bytecode.h" #include "Luau/Compiler.h" -LUAU_FASTFLAGVARIABLE(LuauCompileSelectBuiltin, false) +LUAU_FASTFLAGVARIABLE(LuauCompileSelectBuiltin2, false) namespace Luau { @@ -64,7 +64,7 @@ int getBuiltinFunctionId(const Builtin& builtin, const CompileOptions& options) if (builtin.isGlobal("unpack")) return LBF_TABLE_UNPACK; - if (FFlag::LuauCompileSelectBuiltin && builtin.isGlobal("select")) + if (FFlag::LuauCompileSelectBuiltin2 && builtin.isGlobal("select")) return LBF_SELECT_VARARG; if (builtin.object == "math") diff --git a/Compiler/src/Compiler.cpp b/Compiler/src/Compiler.cpp index 7da85244..e4253adc 100644 --- a/Compiler/src/Compiler.cpp +++ b/Compiler/src/Compiler.cpp @@ -16,7 +16,7 @@ #include LUAU_FASTFLAGVARIABLE(LuauCompileTableIndexOpt, false) -LUAU_FASTFLAG(LuauCompileSelectBuiltin) +LUAU_FASTFLAG(LuauCompileSelectBuiltin2) namespace Luau { @@ -266,7 +266,7 @@ struct Compiler void compileExprSelectVararg(AstExprCall* expr, uint8_t target, uint8_t targetCount, bool targetTop, bool multRet, uint8_t regs) { - LUAU_ASSERT(FFlag::LuauCompileSelectBuiltin); + LUAU_ASSERT(FFlag::LuauCompileSelectBuiltin2); LUAU_ASSERT(targetCount == 1); LUAU_ASSERT(!expr->self); LUAU_ASSERT(expr->args.size == 2 && expr->args.data[1]->is()); @@ -291,6 +291,9 @@ struct Compiler // we can't use TempTop variant here because we need to make sure the arguments we already computed aren't overwritten compileExprTemp(expr->func, regs); + if (argreg != regs + 1) + bytecode.emitABC(LOP_MOVE, uint8_t(regs + 1), argreg, 0); + bytecode.emitABC(LOP_GETVARARGS, uint8_t(regs + 2), 0, 0); size_t callLabel = bytecode.emitLabel(); @@ -405,7 +408,7 @@ struct Compiler if (bfid == LBF_SELECT_VARARG) { - LUAU_ASSERT(FFlag::LuauCompileSelectBuiltin); + LUAU_ASSERT(FFlag::LuauCompileSelectBuiltin2); // Optimization: compile select(_, ...) as FASTCALL1; the builtin will read variadic arguments directly // note: for now we restrict this to single-return expressions since our runtime code doesn't deal with general cases if (multRet == false && targetCount == 1 && expr->args.size == 2 && expr->args.data[1]->is()) diff --git a/Compiler/src/TableShape.cpp b/Compiler/src/TableShape.cpp index 9dc2f0a4..5a866e87 100644 --- a/Compiler/src/TableShape.cpp +++ b/Compiler/src/TableShape.cpp @@ -1,8 +1,6 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #include "TableShape.h" -LUAU_FASTFLAGVARIABLE(LuauPredictTableSizeLoop, false) - namespace Luau { namespace Compile @@ -87,9 +85,6 @@ struct ShapeVisitor : AstVisitor } else if (AstExprLocal* iter = index->as()) { - if (!FFlag::LuauPredictTableSizeLoop) - return; - if (const unsigned int* bound = loops.find(iter->local)) { TableShape& shape = shapes[*table]; @@ -143,9 +138,6 @@ struct ShapeVisitor : AstVisitor bool visit(AstStatFor* node) override { - if (!FFlag::LuauPredictTableSizeLoop) - return true; - AstExprConstantNumber* from = node->from->as(); AstExprConstantNumber* to = node->to->as(); diff --git a/Makefile b/Makefile index 697e34ed..d90dad87 100644 --- a/Makefile +++ b/Makefile @@ -23,6 +23,10 @@ VM_SOURCES=$(wildcard VM/src/*.cpp) VM_OBJECTS=$(VM_SOURCES:%=$(BUILD)/%.o) VM_TARGET=$(BUILD)/libluauvm.a +ISOCLINE_SOURCES=extern/isocline/src/isocline.c +ISOCLINE_OBJECTS=$(ISOCLINE_SOURCES:%=$(BUILD)/%.o) +ISOCLINE_TARGET=$(BUILD)/libisocline.a + TESTS_SOURCES=$(wildcard tests/*.cpp) CLI/FileUtils.cpp CLI/Profiler.cpp CLI/Coverage.cpp CLI/Repl.cpp TESTS_OBJECTS=$(TESTS_SOURCES:%=$(BUILD)/%.o) TESTS_TARGET=$(BUILD)/luau-tests @@ -43,7 +47,7 @@ ifneq ($(flags),) TESTS_ARGS+=--fflags=$(flags) endif -OBJECTS=$(AST_OBJECTS) $(COMPILER_OBJECTS) $(ANALYSIS_OBJECTS) $(VM_OBJECTS) $(TESTS_OBJECTS) $(CLI_OBJECTS) $(FUZZ_OBJECTS) +OBJECTS=$(AST_OBJECTS) $(COMPILER_OBJECTS) $(ANALYSIS_OBJECTS) $(VM_OBJECTS) $(ISOCLINE_OBJECTS) $(TESTS_OBJECTS) $(CLI_OBJECTS) $(FUZZ_OBJECTS) # common flags CXXFLAGS=-g -Wall @@ -90,8 +94,9 @@ $(AST_OBJECTS): CXXFLAGS+=-std=c++17 -IAst/include $(COMPILER_OBJECTS): CXXFLAGS+=-std=c++17 -ICompiler/include -IAst/include $(ANALYSIS_OBJECTS): CXXFLAGS+=-std=c++17 -IAst/include -IAnalysis/include $(VM_OBJECTS): CXXFLAGS+=-std=c++11 -IVM/include +$(ISOCLINE_OBJECTS): CXXFLAGS+=-Wno-unused-function -Iextern/isocline/include $(TESTS_OBJECTS): CXXFLAGS+=-std=c++17 -IAst/include -ICompiler/include -IAnalysis/include -IVM/include -ICLI -Iextern -$(REPL_CLI_OBJECTS): CXXFLAGS+=-std=c++17 -IAst/include -ICompiler/include -IVM/include -Iextern +$(REPL_CLI_OBJECTS): CXXFLAGS+=-std=c++17 -IAst/include -ICompiler/include -IVM/include -Iextern -Iextern/isocline/include $(ANALYZE_CLI_OBJECTS): CXXFLAGS+=-std=c++17 -IAst/include -IAnalysis/include -Iextern $(FUZZ_OBJECTS): CXXFLAGS+=-std=c++17 -IAst/include -ICompiler/include -IAnalysis/include -IVM/include @@ -116,9 +121,9 @@ coverage: $(TESTS_TARGET) $(TESTS_TARGET) llvm-profdata merge default.profraw default-flags.profraw -o default.profdata rm default.profraw default-flags.profraw - llvm-cov show -format=html -show-instantiations=false -show-line-counts=true -show-region-summary=false -ignore-filename-regex=\(tests\|extern\)/.* -output-dir=coverage --instr-profile default.profdata build/coverage/luau-tests - llvm-cov report -ignore-filename-regex=\(tests\|extern\)/.* -show-region-summary=false --instr-profile default.profdata build/coverage/luau-tests - llvm-cov export -ignore-filename-regex=\(tests\|extern\)/.* -format lcov --instr-profile default.profdata build/coverage/luau-tests >coverage.info + llvm-cov show -format=html -show-instantiations=false -show-line-counts=true -show-region-summary=false -ignore-filename-regex=\(tests\|extern\|CLI\)/.* -output-dir=coverage --instr-profile default.profdata build/coverage/luau-tests + llvm-cov report -ignore-filename-regex=\(tests\|extern\|CLI\)/.* -show-region-summary=false --instr-profile default.profdata build/coverage/luau-tests + llvm-cov export -ignore-filename-regex=\(tests\|extern\|CLI\)/.* -format lcov --instr-profile default.profdata build/coverage/luau-tests >coverage.info format: find . -name '*.h' -or -name '*.cpp' | xargs clang-format -i @@ -135,8 +140,8 @@ luau-analyze: $(ANALYZE_CLI_TARGET) ln -fs $^ $@ # executable targets -$(TESTS_TARGET): $(TESTS_OBJECTS) $(ANALYSIS_TARGET) $(COMPILER_TARGET) $(AST_TARGET) $(VM_TARGET) -$(REPL_CLI_TARGET): $(REPL_CLI_OBJECTS) $(COMPILER_TARGET) $(AST_TARGET) $(VM_TARGET) +$(TESTS_TARGET): $(TESTS_OBJECTS) $(ANALYSIS_TARGET) $(COMPILER_TARGET) $(AST_TARGET) $(VM_TARGET) $(ISOCLINE_TARGET) +$(REPL_CLI_TARGET): $(REPL_CLI_OBJECTS) $(COMPILER_TARGET) $(AST_TARGET) $(VM_TARGET) $(ISOCLINE_TARGET) $(ANALYZE_CLI_TARGET): $(ANALYZE_CLI_OBJECTS) $(ANALYSIS_TARGET) $(AST_TARGET) $(TESTS_TARGET) $(REPL_CLI_TARGET) $(ANALYZE_CLI_TARGET): @@ -154,8 +159,9 @@ $(AST_TARGET): $(AST_OBJECTS) $(COMPILER_TARGET): $(COMPILER_OBJECTS) $(ANALYSIS_TARGET): $(ANALYSIS_OBJECTS) $(VM_TARGET): $(VM_OBJECTS) +$(ISOCLINE_TARGET): $(ISOCLINE_OBJECTS) -$(AST_TARGET) $(COMPILER_TARGET) $(ANALYSIS_TARGET) $(VM_TARGET): +$(AST_TARGET) $(COMPILER_TARGET) $(ANALYSIS_TARGET) $(VM_TARGET) $(ISOCLINE_TARGET): ar rcs $@ $^ # object file targets @@ -163,6 +169,10 @@ $(BUILD)/%.cpp.o: %.cpp @mkdir -p $(dir $@) $(CXX) $< $(CXXFLAGS) -c -MMD -MP -o $@ +$(BUILD)/%.c.o: %.c + @mkdir -p $(dir $@) + $(CXX) -x c $< $(CXXFLAGS) -c -MMD -MP -o $@ + # protobuf fuzzer setup fuzz/luau.pb.cpp: fuzz/luau.proto build/libprotobuf-mutator cd fuzz && ../build/libprotobuf-mutator/external.protobuf/bin/protoc luau.proto --cpp_out=. diff --git a/Sources.cmake b/Sources.cmake index 22e7af22..b36b6db5 100644 --- a/Sources.cmake +++ b/Sources.cmake @@ -167,6 +167,11 @@ target_sources(Luau.VM PRIVATE VM/src/lvm.h ) +target_sources(isocline PRIVATE + extern/isocline/include/isocline.h + extern/isocline/src/isocline.c +) + if(TARGET Luau.Repl.CLI) # Luau.Repl.CLI Sources target_sources(Luau.Repl.CLI PRIVATE diff --git a/VM/include/luaconf.h b/VM/include/luaconf.h index 7e0832e7..c5bf1c18 100644 --- a/VM/include/luaconf.h +++ b/VM/include/luaconf.h @@ -83,7 +83,7 @@ #endif #ifndef LUAI_GCSTEPSIZE -#define LUAI_GCSTEPSIZE 1 /* GC runs every KB of memory allocation */ +#define LUAI_GCSTEPSIZE 1 /* GC runs every KB of memory allocation */ #endif /* LUA_MINSTACK is the guaranteed number of Lua stack slots available to a C function */ @@ -153,6 +153,6 @@ long l; \ } -#define LUA_VECTOR_SIZE 3 /* must be 3 or 4 */ +#define LUA_VECTOR_SIZE 3 /* must be 3 or 4 */ #define LUA_EXTRA_SIZE LUA_VECTOR_SIZE - 2 diff --git a/VM/src/lcorolib.cpp b/VM/src/lcorolib.cpp index 19222861..7592a14c 100644 --- a/VM/src/lcorolib.cpp +++ b/VM/src/lcorolib.cpp @@ -250,7 +250,7 @@ static int coclose(lua_State* L) { lua_pushboolean(L, false); if (lua_gettop(co)) - lua_xmove(co, L, 1); /* move error message */ + lua_xmove(co, L, 1); /* move error message */ lua_resetthread(co); return 2; } diff --git a/VM/src/ldebug.cpp b/VM/src/ldebug.cpp index 2b5382bb..e9930f7a 100644 --- a/VM/src/ldebug.cpp +++ b/VM/src/ldebug.cpp @@ -12,7 +12,6 @@ #include #include -LUAU_FASTFLAG(LuauBytecodeV2Read) LUAU_FASTFLAG(LuauBytecodeV2Force) static const char* getfuncname(Closure* f); @@ -96,7 +95,7 @@ static int getlinedefined(Proto* p) { if (FFlag::LuauBytecodeV2Force) return p->linedefined; - else if (FFlag::LuauBytecodeV2Read && p->linedefined >= 0) + else if (p->linedefined >= 0) return p->linedefined; else return luaG_getline(p, 0); diff --git a/VM/src/lfunc.cpp b/VM/src/lfunc.cpp index 6088f71c..582d4627 100644 --- a/VM/src/lfunc.cpp +++ b/VM/src/lfunc.cpp @@ -90,7 +90,7 @@ UpVal* luaF_findupval(lua_State* L, StkId level) uv->tt = LUA_TUPVAL; uv->marked = luaC_white(g); uv->memcat = L->activememcat; - uv->v = level; /* current value lives in the stack */ + uv->v = level; /* current value lives in the stack */ // chain the upvalue in the threads open upvalue list at the proper position UpVal* next = *pp; @@ -138,8 +138,8 @@ void luaF_unlinkupval(UpVal* uv) void luaF_freeupval(lua_State* L, UpVal* uv, lua_Page* page) { - if (uv->v != &uv->u.value) /* is it open? */ - luaF_unlinkupval(uv); /* remove from open list */ + if (uv->v != &uv->u.value) /* is it open? */ + luaF_unlinkupval(uv); /* remove from open list */ luaM_freegco(L, uv, sizeof(UpVal), uv->memcat, page); /* free upvalue */ } diff --git a/VM/src/lgc.cpp b/VM/src/lgc.cpp index 82ac0009..835572fa 100644 --- a/VM/src/lgc.cpp +++ b/VM/src/lgc.cpp @@ -759,6 +759,8 @@ static int sweepgcopage(lua_State* L, lua_Page* page) // when true is returned it means that the element was deleted if (sweepgco(L, page, gco)) { + LUAU_ASSERT(busyBlocks > 0); + // if the last block was removed, page would be removed as well if (--busyBlocks == 0) return int(pos - start) / blockSize + 1; diff --git a/VM/src/lmem.cpp b/VM/src/lmem.cpp index e1dbce50..de85cf59 100644 --- a/VM/src/lmem.cpp +++ b/VM/src/lmem.cpp @@ -57,8 +57,7 @@ const size_t kSizeClasses = LUA_SIZECLASSES; const size_t kMaxSmallSize = 512; const size_t kPageSize = 16 * 1024 - 24; // slightly under 16KB since that results in less fragmentation due to heap metadata const size_t kBlockHeader = sizeof(double) > sizeof(void*) ? sizeof(double) : sizeof(void*); // suitable for aligning double & void* on all platforms -// TODO (FFlagLuauGcPagedSweep): when 'next' is removed, 'kBlockHeader' can be used unconditionally -const size_t kGCOHeader = sizeof(GCheader) > kBlockHeader ? sizeof(GCheader) : kBlockHeader; +const size_t kGCOLinkOffset = (sizeof(GCheader) + sizeof(void*) - 1) & ~(sizeof(void*) - 1); // GCO pages contain freelist links after the GC header struct SizeClassConfig { @@ -101,12 +100,12 @@ struct SizeClassConfig const SizeClassConfig kSizeClassConfig; -// size class for a block of size sz +// size class for a block of size sz; returns -1 for size=0 because empty allocations take no space #define sizeclass(sz) (size_t((sz)-1) < kMaxSmallSize ? kSizeClassConfig.classForSize[sz] : -1) // metadata for a block is stored in the first pointer of the block #define metadata(block) (*(void**)(block)) -#define freegcolink(block) (*(void**)((char*)block + kGCOHeader)) +#define freegcolink(block) (*(void**)((char*)block + kGCOLinkOffset)) /* ** About the realloc function: @@ -157,7 +156,7 @@ l_noret luaM_toobig(lua_State* L) luaG_runerror(L, "memory allocation error: block too big"); } -static lua_Page* luaM_newpage(lua_State* L, uint8_t sizeClass) +static lua_Page* newpageold(lua_State* L, uint8_t sizeClass) { LUAU_ASSERT(!FFlag::LuauGcPagedSweep); @@ -253,7 +252,7 @@ static lua_Page* newclasspage(lua_State* L, lua_Page** freepageset, lua_Page** g return page; } -static void luaM_freepage(lua_State* L, lua_Page* page, uint8_t sizeClass) +static void freepageold(lua_State* L, lua_Page* page, uint8_t sizeClass) { LUAU_ASSERT(!FFlag::LuauGcPagedSweep); @@ -310,7 +309,7 @@ static void freeclasspage(lua_State* L, lua_Page** freepageset, lua_Page** gcopa freepage(L, gcopageset, page); } -static void* luaM_newblock(lua_State* L, int sizeClass) +static void* newblock(lua_State* L, int sizeClass) { global_State* g = L->global; lua_Page* page = g->freepages[sizeClass]; @@ -321,7 +320,7 @@ static void* luaM_newblock(lua_State* L, int sizeClass) if (FFlag::LuauGcPagedSweep) page = newclasspage(L, g->freepages, NULL, sizeClass, true); else - page = luaM_newpage(L, sizeClass); + page = newpageold(L, sizeClass); } LUAU_ASSERT(!page->prev); @@ -363,7 +362,7 @@ static void* luaM_newblock(lua_State* L, int sizeClass) return (char*)block + kBlockHeader; } -static void* luaM_newgcoblock(lua_State* L, int sizeClass) +static void* newgcoblock(lua_State* L, int sizeClass) { LUAU_ASSERT(FFlag::LuauGcPagedSweep); @@ -390,11 +389,10 @@ static void* luaM_newgcoblock(lua_State* L, int sizeClass) } else { + block = page->freeList; + ASAN_UNPOISON_MEMORY_REGION((char*)block + sizeof(GCheader), page->blockSize - sizeof(GCheader)); + // when separate block metadata is not used, free list link is stored inside the block data itself - block = (char*)page->freeList - kGCOHeader; - - ASAN_UNPOISON_MEMORY_REGION((char*)block + kGCOHeader, page->blockSize - kGCOHeader); - page->freeList = freegcolink(block); page->busyBlocks++; } @@ -412,7 +410,7 @@ static void* luaM_newgcoblock(lua_State* L, int sizeClass) return (char*)block; } -static void luaM_freeblock(lua_State* L, int sizeClass, void* block) +static void freeblock(lua_State* L, int sizeClass, void* block) { global_State* g = L->global; @@ -450,11 +448,11 @@ static void luaM_freeblock(lua_State* L, int sizeClass, void* block) if (FFlag::LuauGcPagedSweep) freeclasspage(L, g->freepages, NULL, page, sizeClass); else - luaM_freepage(L, page, sizeClass); + freepageold(L, page, sizeClass); } } -static void luaM_freegcoblock(lua_State* L, int sizeClass, void* block, lua_Page* page) +static void freegcoblock(lua_State* L, int sizeClass, void* block, lua_Page* page) { LUAU_ASSERT(FFlag::LuauGcPagedSweep); @@ -474,9 +472,9 @@ static void luaM_freegcoblock(lua_State* L, int sizeClass, void* block, lua_Page // when separate block metadata is not used, free list link is stored inside the block data itself freegcolink(block) = page->freeList; - page->freeList = (char*)block + kGCOHeader; + page->freeList = block; - ASAN_POISON_MEMORY_REGION((char*)block + kGCOHeader, page->blockSize - kGCOHeader); + ASAN_POISON_MEMORY_REGION((char*)block + sizeof(GCheader), page->blockSize - sizeof(GCheader)); page->busyBlocks--; @@ -491,7 +489,7 @@ void* luaM_new_(lua_State* L, size_t nsize, uint8_t memcat) int nclass = sizeclass(nsize); - void* block = nclass >= 0 ? luaM_newblock(L, nclass) : (*g->frealloc)(L, g->ud, NULL, 0, nsize); + void* block = nclass >= 0 ? newblock(L, nclass) : (*g->frealloc)(L, g->ud, NULL, 0, nsize); if (block == NULL && nsize > 0) luaD_throw(L, LUA_ERRMEM); @@ -506,6 +504,9 @@ GCObject* luaM_newgco_(lua_State* L, size_t nsize, uint8_t memcat) if (!FFlag::LuauGcPagedSweep) return (GCObject*)luaM_new_(L, nsize, memcat); + // we need to accommodate space for link for free blocks (freegcolink) + LUAU_ASSERT(nsize >= kGCOLinkOffset + sizeof(void*)); + global_State* g = L->global; int nclass = sizeclass(nsize); @@ -514,9 +515,7 @@ GCObject* luaM_newgco_(lua_State* L, size_t nsize, uint8_t memcat) if (nclass >= 0) { - LUAU_ASSERT(nsize > 8); - - block = luaM_newgcoblock(L, nclass); + block = newgcoblock(L, nclass); } else { @@ -546,7 +545,7 @@ void luaM_free_(lua_State* L, void* block, size_t osize, uint8_t memcat) int oclass = sizeclass(osize); if (oclass >= 0) - luaM_freeblock(L, oclass, block); + freeblock(L, oclass, block); else (*g->frealloc)(L, g->ud, block, osize, 0); @@ -571,7 +570,7 @@ void luaM_freegco_(lua_State* L, GCObject* block, size_t osize, uint8_t memcat, { block->gch.tt = LUA_TNIL; - luaM_freegcoblock(L, oclass, block, page); + freegcoblock(L, oclass, block, page); } else { @@ -596,7 +595,7 @@ void* luaM_realloc_(lua_State* L, void* block, size_t osize, size_t nsize, uint8 // if either block needs to be allocated using a block allocator, we can't use realloc directly if (nclass >= 0 || oclass >= 0) { - result = nclass >= 0 ? luaM_newblock(L, nclass) : (*g->frealloc)(L, g->ud, NULL, 0, nsize); + result = nclass >= 0 ? newblock(L, nclass) : (*g->frealloc)(L, g->ud, NULL, 0, nsize); if (result == NULL && nsize > 0) luaD_throw(L, LUA_ERRMEM); @@ -604,7 +603,7 @@ void* luaM_realloc_(lua_State* L, void* block, size_t osize, size_t nsize, uint8 memcpy(result, block, osize < nsize ? osize : nsize); if (oclass >= 0) - luaM_freeblock(L, oclass, block); + freeblock(L, oclass, block); else (*g->frealloc)(L, g->ud, block, osize, 0); } @@ -659,6 +658,8 @@ void luaM_visitpage(lua_Page* page, void* context, bool (*visitor)(void* context // when true is returned it means that the element was deleted if (visitor(context, page, gco)) { + LUAU_ASSERT(busyBlocks > 0); + // if the last block was removed, page would be removed as well if (--busyBlocks == 0) break; diff --git a/VM/src/lstring.cpp b/VM/src/lstring.cpp index cb22cc23..9bbc43de 100644 --- a/VM/src/lstring.cpp +++ b/VM/src/lstring.cpp @@ -57,7 +57,7 @@ void luaS_resize(lua_State* L, int newsize) { TString* p = tb->hash[i]; while (p) - { /* for each node in the list */ + { /* for each node in the list */ // TODO (FFlagLuauGcPagedSweep): 'next' type will change after removal of the flag and the cast will not be required TString* next = (TString*)p->next; /* save next */ unsigned int h = p->hash; diff --git a/VM/src/lvmexecute.cpp b/VM/src/lvmexecute.cpp index e58ff2a8..cba3670a 100644 --- a/VM/src/lvmexecute.cpp +++ b/VM/src/lvmexecute.cpp @@ -676,14 +676,9 @@ static void luau_execute(lua_State* L) VM_PROTECT_PC(); // set may fail TValue* res = luaH_setstr(L, h, tsvalue(kv)); - - if (res != luaO_nilobject) - { - int cachedslot = gval2slot(h, res); - // save cachedslot to accelerate future lookups; patches currently executing instruction since pc-2 rolls back two pc++ - VM_PATCH_C(pc - 2, cachedslot); - } - + int cachedslot = gval2slot(h, res); + // save cachedslot to accelerate future lookups; patches currently executing instruction since pc-2 rolls back two pc++ + VM_PATCH_C(pc - 2, cachedslot); setobj(L, res, ra); luaC_barriert(L, h, ra); VM_NEXT(); diff --git a/VM/src/lvmload.cpp b/VM/src/lvmload.cpp index 7839c68c..2472cd90 100644 --- a/VM/src/lvmload.cpp +++ b/VM/src/lvmload.cpp @@ -13,7 +13,6 @@ #include -LUAU_FASTFLAGVARIABLE(LuauBytecodeV2Read, true) LUAU_FASTFLAGVARIABLE(LuauBytecodeV2Force, false) // TODO: RAII deallocation doesn't work for longjmp builds if a memory error happens @@ -157,11 +156,12 @@ int luau_load(lua_State* L, const char* chunkname, const char* data, size_t size return 1; } - if (FFlag::LuauBytecodeV2Force ? (version != LBC_VERSION_FUTURE) : FFlag::LuauBytecodeV2Read ? (version != LBC_VERSION && version != LBC_VERSION_FUTURE) : (version != LBC_VERSION)) + if (FFlag::LuauBytecodeV2Force ? (version != LBC_VERSION_FUTURE) : (version != LBC_VERSION && version != LBC_VERSION_FUTURE)) { char chunkid[LUA_IDSIZE]; luaO_chunkid(chunkid, chunkname, LUA_IDSIZE); - lua_pushfstring(L, "%s: bytecode version mismatch (expected %d, got %d)", chunkid, FFlag::LuauBytecodeV2Force ? LBC_VERSION_FUTURE : LBC_VERSION, version); + lua_pushfstring(L, "%s: bytecode version mismatch (expected %d, got %d)", chunkid, + FFlag::LuauBytecodeV2Force ? LBC_VERSION_FUTURE : LBC_VERSION, version); return 1; } @@ -292,7 +292,7 @@ int luau_load(lua_State* L, const char* chunkname, const char* data, size_t size p->p[j] = protos[fid]; } - if (FFlag::LuauBytecodeV2Force || (FFlag::LuauBytecodeV2Read && version == LBC_VERSION_FUTURE)) + if (FFlag::LuauBytecodeV2Force || version == LBC_VERSION_FUTURE) p->linedefined = readVarInt(data, size, offset); else p->linedefined = -1; diff --git a/extern/isocline/.gitignore b/extern/isocline/.gitignore new file mode 100644 index 00000000..470cc813 --- /dev/null +++ b/extern/isocline/.gitignore @@ -0,0 +1,16 @@ +out/ +build/ +dist/ +doc/html/ +.vs/ +.vscode/ +.stack-work/ +.DS_Store +*.user +*.exe +*.hi +*.o +*_stub.h +*.lock +history.txt +isocline.debug.txt \ No newline at end of file diff --git a/extern/isocline/LICENSE b/extern/isocline/LICENSE new file mode 100644 index 00000000..7ac3104b --- /dev/null +++ b/extern/isocline/LICENSE @@ -0,0 +1,21 @@ +MIT License + +Copyright (c) 2021 Daan Leijen + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. diff --git a/extern/isocline/include/isocline.h b/extern/isocline/include/isocline.h new file mode 100644 index 00000000..0d46cf3f --- /dev/null +++ b/extern/isocline/include/isocline.h @@ -0,0 +1,627 @@ +/* ---------------------------------------------------------------------------- + Copyright (c) 2021, Daan Leijen + This is free software; you can redistribute it and/or modify it + under the terms of the MIT License. A copy of the license can be + found in the "LICENSE" file at the root of this distribution. +-----------------------------------------------------------------------------*/ +#pragma once +#ifndef IC_ISOCLINE_H +#define IC_ISOCLINE_H + +#ifdef __cplusplus +extern "C" { +#endif + +#include // size_t +#include // bool +#include // uint32_t +#include // term_vprintf + + +/*! \mainpage +Isocline C API reference. + +Isocline is a pure C library that can be used as an alternative to the GNU readline library. + +See the [Github repository](https://github.com/daanx/isocline#readme) +for general information and building the library. + +Contents: +- \ref readline +- \ref bbcode +- \ref history +- \ref completion +- \ref highlight +- \ref options +- \ref helper +- \ref completex +- \ref term +- \ref async +- \ref alloc +*/ + +/// \defgroup readline Readline +/// The basic readline interface. +/// \{ + +/// Isocline version: 102 = 1.0.2. +#define IC_VERSION (104) + + +/// Read input from the user using rich editing abilities. +/// @param prompt_text The prompt text, can be NULL for the default (""). +/// The displayed prompt becomes `prompt_text` followed by the `prompt_marker` ("> "). +/// @returns the heap allocated input on succes, which should be `free`d by the caller. +/// Returns NULL on error, or if the user typed ctrl+d or ctrl+c. +/// +/// If the standard input (`stdin`) has no editing capability +/// (like a dumb terminal (e.g. `TERM`=`dumb`), running in a debuggen, a pipe or redirected file, etc.) +/// the input is read directly from the input stream up to the +/// next line without editing capability. +/// See also \a ic_set_prompt_marker(), \a ic_style_def() +/// +/// @see ic_set_prompt_marker(), ic_style_def() +char* ic_readline(const char* prompt_text); + +/// \} + + +//-------------------------------------------------------------- +/// \defgroup bbcode Formatted Text +/// Formatted text using [bbcode markup](https://github.com/daanx/isocline#bbcode-format). +/// \{ + +/// Print to the terminal while respection bbcode markup. +/// Any unclosed tags are closed automatically at the end of the print. +/// For example: +/// ``` +/// ic_print("[b]bold, [i]bold and italic[/i], [red]red and bold[/][/b] default."); +/// ic_print("[b]bold[/], [i b]bold and italic[/], [yellow on blue]yellow on blue background"); +/// ic_style_add("em","i color=#888800"); +/// ic_print("[em]emphasis"); +/// ``` +/// Properties that can be assigned are: +/// * `color=` _clr_, `bgcolor=` _clr_: where _clr_ is either a hex value `#`RRGGBB or `#`RGB, a +/// standard HTML color name, or an ANSI palette name, like `ansi-maroon`, `ansi-default`, etc. +/// * `bold`,`italic`,`reverse`,`underline`: can be `on` or `off`. +/// * everything else is a style; all HTML and ANSI color names are also a style (so we can just use `red` +/// instead of `color=red`, or `on red` instead of `bgcolor=red`), and there are +/// the `b`, `i`, `u`, and `r` styles for bold, italic, underline, and reverse. +/// +/// See [here](https://github.com/daanx/isocline#bbcode-format) for a description of the full bbcode format. +void ic_print( const char* s ); + +/// Print with bbcode markup ending with a newline. +/// @see ic_print() +void ic_println( const char* s ); + +/// Print formatted with bbcode markup. +/// @see ic_print() +void ic_printf(const char* fmt, ...); + +/// Print formatted with bbcode markup. +/// @see ic_print +void ic_vprintf(const char* fmt, va_list args); + +/// Define or redefine a style. +/// @param style_name The name of the style. +/// @param fmt The `fmt` string is the content of a tag and can contain +/// other styles. This is very useful to theme the output of a program +/// by assigning standard styles like `em` or `warning` etc. +void ic_style_def( const char* style_name, const char* fmt ); + +/// Start a global style that is only reset when calling a matching ic_style_close(). +void ic_style_open( const char* fmt ); + +/// End a global style. +void ic_style_close(void); + +/// \} + + +//-------------------------------------------------------------- +// History +//-------------------------------------------------------------- +/// \defgroup history History +/// Readline input history. +/// \{ + +/// Enable history. +/// Use a \a NULL filename to not persist the history. Use -1 for max_entries to get the default (200). +void ic_set_history(const char* fname, long max_entries ); + +/// Remove the last entry in the history. +/// The last returned input from ic_readline() is automatically added to the history; this function removes it. +void ic_history_remove_last(void); + +/// Clear the history. +void ic_history_clear(void); + +/// Add an entry to the history +void ic_history_add( const char* entry ); + +/// \} + +//-------------------------------------------------------------- +// Basic Completion +//-------------------------------------------------------------- + +/// \defgroup completion Completion +/// Basic word completion. +/// \{ + +/// A completion environment +struct ic_completion_env_s; + +/// A completion environment +typedef struct ic_completion_env_s ic_completion_env_t; + +/// A completion callback that is called by isocline when tab is pressed. +/// It is passed a completion environment (containing the current input and the current cursor position), +/// the current input up-to the cursor (`prefix`) +/// and the user given argument when the callback was set. +/// When using completion transformers, like `ic_complete_quoted_word` the `prefix` contains the +/// the word to be completed without escape characters or quotes. +typedef void (ic_completer_fun_t)(ic_completion_env_t* cenv, const char* prefix ); + +/// Set the default completion handler. +/// @param completer The completion function +/// @param arg Argument passed to the \a completer. +/// There can only be one default completion function, setting it again disables the previous one. +/// The initial completer use `ic_complete_filename`. +void ic_set_default_completer( ic_completer_fun_t* completer, void* arg); + + +/// In a completion callback (usually from ic_complete_word()), use this function to add a completion. +/// (the completion string is copied by isocline and do not need to be preserved or allocated). +/// +/// Returns `true` if the callback should continue trying to find more possible completions. +/// If `false` is returned, the callback should try to return and not add more completions (for improved latency). +bool ic_add_completion(ic_completion_env_t* cenv, const char* completion); + +/// In a completion callback (usually from ic_complete_word()), use this function to add a completion. +/// The `display` is used to display the completion in the completion menu, and `help` is +/// displayed for hints for example. Both can be `NULL` for the default. +/// (all are copied by isocline and do not need to be preserved or allocated). +/// +/// Returns `true` if the callback should continue trying to find more possible completions. +/// If `false` is returned, the callback should try to return and not add more completions (for improved latency). +bool ic_add_completion_ex( ic_completion_env_t* cenv, const char* completion, const char* display, const char* help ); + +/// In a completion callback (usually from ic_complete_word()), use this function to add completions. +/// The `completions` array should be terminated with a NULL element, and all elements +/// are added as completions if they start with `prefix`. +/// +/// Returns `true` if the callback should continue trying to find more possible completions. +/// If `false` is returned, the callback should try to return and not add more completions (for improved latency). +bool ic_add_completions(ic_completion_env_t* cenv, const char* prefix, const char** completions); + +/// Complete a filename. +/// Complete a filename given a semi-colon separated list of root directories `roots` and +/// semi-colon separated list of possible extensions (excluding directories). +/// If `roots` is NULL, the current directory is the root ("."). +/// If `extensions` is NULL, any extension will match. +/// Each root directory should _not_ end with a directory separator. +/// If a directory is completed, the `dir_separator` is added at the end if it is not `0`. +/// Usually the `dir_separator` is `/` but it can be set to `\\` on Windows systems. +/// For example: +/// ``` +/// /ho --> /home/ +/// /home/.ba --> /home/.bashrc +/// ``` +/// (This already uses ic_complete_quoted_word() so do not call it from inside a word handler). +void ic_complete_filename( ic_completion_env_t* cenv, const char* prefix, char dir_separator, const char* roots, const char* extensions ); + + + +/// Function that returns whether a (utf8) character (of length `len`) is in a certain character class +/// @see ic_char_is_separator() etc. +typedef bool (ic_is_char_class_fun_t)(const char* s, long len); + + +/// Complete a _word_ (i.e. _token_). +/// Calls the user provided function `fun` to complete on the +/// current _word_. Almost all user provided completers should use this function. +/// If `is_word_char` is NULL, the default `&ic_char_is_nonseparator` is used. +/// The `prefix` passed to `fun` is modified to only contain the current word, and +/// any results from `ic_add_completion` are automatically adjusted to replace that part. +/// For example, on the input "hello w", a the user `fun` only gets `w` and can just complete +/// with "world" resulting in "hello world" without needing to consider `delete_before` etc. +/// @see ic_complete_qword() for completing quoted and escaped tokens. +void ic_complete_word(ic_completion_env_t* cenv, const char* prefix, ic_completer_fun_t* fun, ic_is_char_class_fun_t* is_word_char); + + +/// Complete a quoted _word_. +/// Calls the user provided function `fun` to complete while taking +/// care of quotes and escape characters. Almost all user provided completers should use +/// this function. The `prefix` passed to `fun` is modified to be unquoted and unescaped, and +/// any results from `ic_add_completion` are automatically quoted and escaped again. +/// For example, completing `hello world`, the `fun` always just completes `hel` or `hello w` to `hello world`, +/// but depending on user input, it will complete as: +/// ``` +/// hel --> hello\ world +/// hello\ w --> hello\ world +/// hello w --> # no completion, the word is just 'w'> +/// "hel --> "hello world" +/// "hello w --> "hello world" +/// ``` +/// with proper quotes and escapes. +/// If `is_word_char` is NULL, the default `&ic_char_is_nonseparator` is used. +/// @see ic_complete_quoted_word() to customize the word boundary, quotes etc. +void ic_complete_qword( ic_completion_env_t* cenv, const char* prefix, ic_completer_fun_t* fun, ic_is_char_class_fun_t* is_word_char ); + + + +/// Complete a _word_. +/// Calls the user provided function `fun` to complete while taking +/// care of quotes and escape characters. Almost all user provided completers should use this function. +/// The `is_word_char` is a set of characters that are part of a "word". Use NULL for the default (`&ic_char_is_nonseparator`). +/// The `escape_char` is the escaping character, usually `\` but use 0 to not have escape characters. +/// The `quote_chars` define the quotes, use NULL for the default `"\'\""` quotes. +/// @see ic_complete_word() which uses the default values for `non_word_chars`, `quote_chars` and `\` for escape characters. +void ic_complete_qword_ex( ic_completion_env_t* cenv, const char* prefix, ic_completer_fun_t fun, + ic_is_char_class_fun_t* is_word_char, char escape_char, const char* quote_chars ); + +/// \} + +//-------------------------------------------------------------- +/// \defgroup highlight Syntax Highlighting +/// Basic syntax highlighting. +/// \{ + +/// A syntax highlight environment +struct ic_highlight_env_s; +typedef struct ic_highlight_env_s ic_highlight_env_t; + +/// A syntax highlighter callback that is called by readline to syntax highlight user input. +typedef void (ic_highlight_fun_t)(ic_highlight_env_t* henv, const char* input, void* arg); + +/// Set a syntax highlighter. +/// There can only be one highlight function, setting it again disables the previous one. +void ic_set_default_highlighter(ic_highlight_fun_t* highlighter, void* arg); + +/// Set the style of characters starting at position `pos`. +void ic_highlight(ic_highlight_env_t* henv, long pos, long count, const char* style ); + +/// Experimental: Convenience callback for a function that highlights `s` using bbcode's. +/// The returned string should be allocated and is free'd by the caller. +typedef char* (ic_highlight_format_fun_t)(const char* s, void* arg); + +/// Experimental: Convenience function for highlighting with bbcodes. +/// Can be called in a `ic_highlight_fun_t` callback to colorize the `input` using the +/// the provided `formatted` input that is the styled `input` with bbcodes. The +/// content of `formatted` without bbcode tags should match `input` exactly. +void ic_highlight_formatted(ic_highlight_env_t* henv, const char* input, const char* formatted); + +/// \} + +//-------------------------------------------------------------- +// Readline with a specific completer and highlighter +//-------------------------------------------------------------- + +/// \defgroup readline +/// \{ + +/// Read input from the user using rich editing abilities, +/// using a particular completion function and highlighter for this call only. +/// both can be NULL in which case the defaults are used. +/// @see ic_readline(), ic_set_prompt_marker(), ic_set_default_completer(), ic_set_default_highlighter(). +char* ic_readline_ex(const char* prompt_text, ic_completer_fun_t* completer, void* completer_arg, + ic_highlight_fun_t* highlighter, void* highlighter_arg); + +/// \} + + +//-------------------------------------------------------------- +// Options +//-------------------------------------------------------------- + +/// \defgroup options Options +/// \{ + +/// Set a prompt marker and a potential marker for extra lines with multiline input. +/// Pass \a NULL for the `prompt_marker` for the default marker (`"> "`). +/// Pass \a NULL for continuation prompt marker to make it equal to the `prompt_marker`. +void ic_set_prompt_marker( const char* prompt_marker, const char* continuation_prompt_marker ); + +/// Get the current prompt marker. +const char* ic_get_prompt_marker(void); + +/// Get the current continuation prompt marker. +const char* ic_get_continuation_prompt_marker(void); + +/// Disable or enable multi-line input (enabled by default). +/// Returns the previous setting. +bool ic_enable_multiline( bool enable ); + +/// Disable or enable sound (enabled by default). +/// A beep is used when tab cannot find any completion for example. +/// Returns the previous setting. +bool ic_enable_beep( bool enable ); + +/// Disable or enable color output (enabled by default). +/// Returns the previous setting. +bool ic_enable_color( bool enable ); + +/// Disable or enable duplicate entries in the history (disabled by default). +/// Returns the previous setting. +bool ic_enable_history_duplicates( bool enable ); + +/// Disable or enable automatic tab completion after a completion +/// to expand as far as possible if the completions are unique. (disabled by default). +/// Returns the previous setting. +bool ic_enable_auto_tab( bool enable ); + +/// Disable or enable preview of a completion selection (enabled by default) +/// Returns the previous setting. +bool ic_enable_completion_preview( bool enable ); + +/// Disable or enable automatic identation of continuation lines in multiline +/// input so it aligns with the initial prompt. +/// Returns the previous setting. +bool ic_enable_multiline_indent(bool enable); + +/// Disable or enable display of short help messages for history search etc. +/// (full help is always dispayed when pressing F1 regardless of this setting) +/// @returns the previous setting. +bool ic_enable_inline_help(bool enable); + +/// Disable or enable hinting (enabled by default) +/// Shows a hint inline when there is a single possible completion. +/// @returns the previous setting. +bool ic_enable_hint(bool enable); + +/// Set millisecond delay before a hint is displayed. Can be zero. (500ms by default). +long ic_set_hint_delay(long delay_ms); + +/// Disable or enable syntax highlighting (enabled by default). +/// This applies regardless whether a syntax highlighter callback was set (`ic_set_highlighter`) +/// Returns the previous setting. +bool ic_enable_highlight(bool enable); + + +/// Set millisecond delay for reading escape sequences in order to distinguish +/// a lone ESC from the start of a escape sequence. The defaults are 100ms and 10ms, +/// but it may be increased if working with very slow terminals. +void ic_set_tty_esc_delay(long initial_delay_ms, long followup_delay_ms); + +/// Enable highlighting of matching braces (and error highlight unmatched braces).` +bool ic_enable_brace_matching(bool enable); + +/// Set matching brace pairs. +/// Pass \a NULL for the default `"()[]{}"`. +void ic_set_matching_braces(const char* brace_pairs); + +/// Enable automatic brace insertion (enabled by default). +bool ic_enable_brace_insertion(bool enable); + +/// Set matching brace pairs for automatic insertion. +/// Pass \a NULL for the default `()[]{}\"\"''` +void ic_set_insertion_braces(const char* brace_pairs); + +/// \} + + +//-------------------------------------------------------------- +// Advanced Completion +//-------------------------------------------------------------- + +/// \defgroup completex Advanced Completion +/// \{ + +/// Get the raw current input (and cursor position if `cursor` != NULL) for the completion. +/// Usually completer functions should look at their `prefix` though as transformers +/// like `ic_complete_word` may modify the prefix (for example, unescape it). +const char* ic_completion_input( ic_completion_env_t* cenv, long* cursor ); + +/// Get the completion argument passed to `ic_set_completer`. +void* ic_completion_arg( const ic_completion_env_t* cenv ); + +/// Do we have already some completions? +bool ic_has_completions( const ic_completion_env_t* cenv ); + +/// Do we already have enough completions and should we return if possible? (for improved latency) +bool ic_stop_completing( const ic_completion_env_t* cenv); + + +/// Primitive completion, cannot be used with most transformers (like `ic_complete_word` and `ic_complete_qword`). +/// When completed, `delete_before` _bytes_ are deleted before the cursor position, +/// `delete_after` _bytes_ are deleted after the cursor, and finally `completion` is inserted. +/// The `display` is used to display the completion in the completion menu, and `help` is displayed +/// with hinting. Both `display` and `help` can be NULL. +/// (all are copied by isocline and do not need to be preserved or allocated). +/// +/// Returns `true` if the callback should continue trying to find more possible completions. +/// If `false` is returned, the callback should try to return and not add more completions (for improved latency). +bool ic_add_completion_prim( ic_completion_env_t* cenv, const char* completion, + const char* display, const char* help, + long delete_before, long delete_after); + +/// \} + +//-------------------------------------------------------------- +/// \defgroup helper Character Classes. +/// Convenience functions for character classes, highlighting and completion. +/// \{ + +/// Convenience: return the position of a previous code point in a UTF-8 string `s` from postion `pos`. +/// Returns `-1` if `pos <= 0` or `pos > strlen(s)` (or other errors). +long ic_prev_char( const char* s, long pos ); + +/// Convenience: return the position of the next code point in a UTF-8 string `s` from postion `pos`. +/// Returns `-1` if `pos < 0` or `pos >= strlen(s)` (or other errors). +long ic_next_char( const char* s, long pos ); + +/// Convenience: does a string `s` starts with a given `prefix` ? +bool ic_starts_with( const char* s, const char* prefix ); + +/// Convenience: does a string `s` starts with a given `prefix` ignoring (ascii) case? +bool ic_istarts_with( const char* s, const char* prefix ); + + +/// Convenience: character class for whitespace `[ \t\r\n]`. +bool ic_char_is_white(const char* s, long len); + +/// Convenience: character class for non-whitespace `[^ \t\r\n]`. +bool ic_char_is_nonwhite(const char* s, long len); + +/// Convenience: character class for separators. +/// (``[ \t\r\n,.;:/\\(){}\[\]]``.) +/// This is used for word boundaries in isocline. +bool ic_char_is_separator(const char* s, long len); + +/// Convenience: character class for non-separators. +bool ic_char_is_nonseparator(const char* s, long len); + +/// Convenience: character class for letters (`[A-Za-z]` and any unicode > 0x80). +bool ic_char_is_letter(const char* s, long len); + +/// Convenience: character class for digits (`[0-9]`). +bool ic_char_is_digit(const char* s, long len); + +/// Convenience: character class for hexadecimal digits (`[A-Fa-f0-9]`). +bool ic_char_is_hexdigit(const char* s, long len); + +/// Convenience: character class for identifier letters (`[A-Za-z0-9_-]` and any unicode > 0x80). +bool ic_char_is_idletter(const char* s, long len); + +/// Convenience: character class for filename letters (_not in_ " \t\r\n`@$><=;|&\{\}\(\)\[\]]"). +bool ic_char_is_filename_letter(const char* s, long len); + + +/// Convenience: If this is a token start, return the length. Otherwise return 0. +long ic_is_token(const char* s, long pos, ic_is_char_class_fun_t* is_token_char); + +/// Convenience: Does this match the specified token? +/// Ensures not to match prefixes or suffixes, and returns the length of the match (in bytes). +/// E.g. `ic_match_token("function",0,&ic_char_is_letter,"fun")` returns 0. +/// while `ic_match_token("fun x",0,&ic_char_is_letter,"fun"})` returns 3. +long ic_match_token(const char* s, long pos, ic_is_char_class_fun_t* is_token_char, const char* token); + + +/// Convenience: Do any of the specified tokens match? +/// Ensures not to match prefixes or suffixes, and returns the length of the match (in bytes). +/// E.g. `ic_match_any_token("function",0,&ic_char_is_letter,{"fun","func",NULL})` returns 0. +/// while `ic_match_any_token("func x",0,&ic_char_is_letter,{"fun","func",NULL})` returns 4. +long ic_match_any_token(const char* s, long pos, ic_is_char_class_fun_t* is_token_char, const char** tokens); + +/// \} + +//-------------------------------------------------------------- +/// \defgroup term Terminal +/// +/// Experimental: Low level terminal output. +/// Ensures basic ANSI SGR escape sequences are processed +/// in a portable way (e.g. on Windows) +/// \{ + +/// Initialize for terminal output. +/// Call this before using the terminal write functions (`ic_term_write`) +/// Does nothing on most platforms but on Windows it sets the console to UTF8 output and possible +/// enables virtual terminal processing. +void ic_term_init(void); + +/// Call this when done with the terminal functions. +void ic_term_done(void); + +/// Flush the terminal output. +/// (happens automatically on newline characters ('\n') as well). +void ic_term_flush(void); + +/// Write a string to the console (and process CSI escape sequences). +void ic_term_write(const char* s); + +/// Write a string to the console and end with a newline +/// (and process CSI escape sequences). +void ic_term_writeln(const char* s); + +/// Write a formatted string to the console. +/// (and process CSI escape sequences) +void ic_term_writef(const char* fmt, ...); + +/// Write a formatted string to the console. +void ic_term_vwritef(const char* fmt, va_list args); + +/// Set text attributes from a style. +void ic_term_style( const char* style ); + +/// Set text attribute to bold. +void ic_term_bold(bool enable); + +/// Set text attribute to underline. +void ic_term_underline(bool enable); + +/// Set text attribute to italic. +void ic_term_italic(bool enable); + +/// Set text attribute to reverse video. +void ic_term_reverse(bool enable); + +/// Set text attribute to ansi color palette index between 0 and 255 (or 256 for the ANSI "default" color). +/// (auto matched to smaller palette if not supported) +void ic_term_color_ansi(bool foreground, int color); + +/// Set text attribute to 24-bit RGB color (between `0x000000` and `0xFFFFFF`). +/// (auto matched to smaller palette if not supported) +void ic_term_color_rgb(bool foreground, uint32_t color ); + +/// Reset the text attributes. +void ic_term_reset( void ); + +/// Get the palette used by the terminal: +/// This is usually initialized from the COLORTERM environment variable. The +/// possible values of COLORTERM for each palette are given in parenthesis. +/// +/// - 1: monochrome (`monochrome`) +/// - 3: old ANSI terminal with 8 colors, using bold for bright (`8color`/`3bit`) +/// - 4: regular ANSI terminal with 16 colors. (`16color`/`4bit`) +/// - 8: terminal with ANSI 256 color palette. (`256color`/`8bit`) +/// - 24: true-color terminal with full RGB colors. (`truecolor`/`24bit`/`direct`) +int ic_term_get_color_bits( void ); + +/// \} + +//-------------------------------------------------------------- +/// \defgroup async ASync +/// Async support +/// \{ + +/// Thread-safe way to asynchronously unblock a readline. +/// Behaves as if the user pressed the `ctrl-C` character +/// (resulting in returning NULL from `ic_readline`). +/// Returns `true` if the event was successfully delivered. +/// (This may not be supported on all platforms, but it is +/// functional on Linux, macOS and Windows). +bool ic_async_stop(void); + +/// \} + +//-------------------------------------------------------------- +/// \defgroup alloc Custom Allocation +/// Register allocation functions for custom allocators +/// \{ + +typedef void* (ic_malloc_fun_t)( size_t size ); +typedef void* (ic_realloc_fun_t)( void* p, size_t newsize ); +typedef void (ic_free_fun_t)( void* p ); + +/// Initialize with custom allocation functions. +/// This must be called as the first function in a program! +void ic_init_custom_alloc( ic_malloc_fun_t* _malloc, ic_realloc_fun_t* _realloc, ic_free_fun_t* _free ); + +/// Free a potentially custom alloc'd pointer (in particular, the result returned from `ic_readline`) +void ic_free( void* p ); + +/// Allocate using the current memory allocator. +void* ic_malloc(size_t sz); + +/// Duplicate a string using the current memory allocator. +const char* ic_strdup( const char* s ); + +/// \} + +#ifdef __cplusplus +} +#endif + +#endif /// IC_ISOCLINE_H diff --git a/extern/isocline/readme.md b/extern/isocline/readme.md new file mode 100644 index 00000000..1f4709bd --- /dev/null +++ b/extern/isocline/readme.md @@ -0,0 +1,460 @@ + + + + +# Isocline: a portable readline alternative. + +Isocline is a pure C library that can be used as an alternative to the GNU readline library (latest release v1.0.9, 2022-01-15). + +- Small: less than 8k lines and can be compiled as a single C file without + any dependencies or configuration (e.g. `gcc -c src/isocline.c`). + +- Portable: works on Unix, Windows, and macOS, and uses a minimal + subset of ANSI escape sequences. + +- Features: extensive multi-line editing mode (`shift-tab`), (24-bit) color, history, completion, unicode, + undo/redo, incremental history search, inline hints, syntax highlighting, brace matching, + closing brace insertion, auto indentation, graceful fallback, support for custom allocators, etc. + +- License: MIT. + +- Comes with a Haskell binding ([`System.Console.Isocline`][hdoc]. + +Enjoy, + Daan + + + +# Demo + +![recording](doc/record-macos.svg) + +Shows in order: unicode, syntax highlighting, brace matching, jump to matching brace, auto indent, multiline editing, 24-bit colors, inline hinting, filename completion, and incremental history search. +(screen capture was made with [termtosvg] by Nicolas Bedos) + +# Usage + +Include the isocline header in your C or C++ source: +```C +#include +``` + +and call `ic_readline` to get user input with rich editing abilities: +```C +char* input; +while( (input = ic_readline("prompt")) != NULL ) { // ctrl+d/c or errors return NULL + printf("you typed:\n%s\n", input); // use the input + free(input); +} +``` + +See the [example] for a full example with completion, syntax highligting, history, etc. + +# Run the Example + +You can compile and run the [example] as: +``` +$ gcc -o example -Iinclude test/example.c src/isocline.c +$ ./example +``` + +or, the Haskell [example][HaskellExample]: +``` +$ ghc -ihaskell test/Example.hs src/isocline.c +$ ./test/Example +``` + + +# Editing with Isocline + +Isocline tries to be as compatible as possible with standard [GNU Readline] key bindings. + +### Overview: +```apl + home/ctrl-a cursor end/ctrl-e + ┌─────────────────┼───────────────┐ (navigate) + │ ctrl-left │ ctrl-right │ + │ ┌───────┼──────┐ │ ctrl-r : search history + ▼ ▼ ▼ ▼ ▼ tab : complete word + prompt> it is the quintessential language shift-tab: insert new line + ▲ ▲ ▲ ▲ esc : delete input, done + │ └──────────────┘ │ ctrl-z : undo + │ alt-backsp alt-d │ + └─────────────────────────────────┘ (delete) + ctrl-u ctrl-k +``` + +Note: on macOS, the meta (alt) key is not directly available in most terminals. +Terminal/iTerm2 users can activate the meta key through +`Terminal` → `Preferences` → `Settings` → `Use option as meta key`. + +### Key Bindings + +These are also shown when pressing `F1` on a Isocline prompt. We use `^` as a shorthand for `ctrl-`: + +| Navigation | | +|-------------------|-------------------------------------------------| +| `left`,`^b` | go one character to the left | +| `right`,`^f ` | go one character to the right | +| `up ` | go one row up, or back in the history | +| `down ` | go one row down, or forward in the history | +| `^left ` | go to the start of the previous word | +| `^right ` | go to the end the current word | +| `home`,`^a ` | go to the start of the current line | +| `end`,`^e ` | go to the end of the current line | +| `pgup`,`^home ` | go to the start of the current input | +| `pgdn`,`^end ` | go to the end of the current input | +| `alt-m ` | jump to matching brace | +| `^p ` | go back in the history | +| `^n ` | go forward in the history | +| `^r`,`^s ` | search the history starting with the current word | + + +| Deletion | | +|-------------------|-------------------------------------------------| +| `del`,`^d ` | delete the current character | +| `backsp`,`^h ` | delete the previous character | +| `^w ` | delete to preceding white space | +| `alt-backsp ` | delete to the start of the current word | +| `alt-d ` | delete to the end of the current word | +| `^u ` | delete to the start of the current line | +| `^k ` | delete to the end of the current line | +| `esc ` | delete the current input, or done with empty input | + + +| Editing | | +|-------------------|-------------------------------------------------| +| `enter ` | accept current input | +| `^enter`,`^j`,`shift-tab` | create a new line for multi-line input | +| `^l ` | clear screen | +| `^t ` | swap with previous character (move character backward) | +| `^z`,`^_ ` | undo | +| `^y ` | redo | +| `tab ` | try to complete the current input | + + +| Completion menu | | +|-------------------|-------------------------------------------------| +| `enter`,`left` | use the currently selected completion | +| `1` - `9` | use completion N from the menu | +| `tab, down ` | select the next completion | +| `shift-tab, up` | select the previous completion | +| `esc ` | exit menu without completing | +| `pgdn`,`^enter`,`^j` | show all further possible completions | + + +| Incremental history search | | +|-------------------|-------------------------------------------------| +| `enter ` | use the currently found history entry | +| `backsp`,`^z ` | go back to the previous match (undo) | +| `tab`,`^r`,`up` | find the next match | +| `shift-tab`,`^s`,`down` | find an earlier match | +| `esc ` | exit search | + + +# Build the Library + +### Build as a Single Source + +Copy the sources (in `include` and `src`) into your project, or add the library as a [submodule]: +``` +$ git submodule add https://github.com/daanx/isocline +``` +and add `isocline/src/isocline.c` to your build rules -- no configuration is needed. + +### Build with CMake + +Clone the repository and run cmake to build a static library (`.a`/`.lib`): +``` +$ git clone https://github.com/daanx/isocline +$ cd isocline +$ mkdir -p build/release +$ cd build/release +$ cmake ../.. +$ cmake --build . +``` +This builds a static library `libisocline.a` (or `isocline.lib` on Windows) +and the example program: +``` +$ ./example +``` + +### Build the Haskell Library + +See the Haskell [readme][Haskell] for instructions to build and use the Haskell library. + + +# API Reference + +* See the [C API reference][docapi] and the [example] for example usage of history, completion, etc. + +* See the [Haskell API reference][hdoc] on Hackage and the Haskell [example][HaskellExample]. + + +# Motivation + +Isocline was created for use in the [Koka] interactive compiler. +This required: pure C (no dependency on a C++ runtime or other libraries), +portable (across Linux, macOS, and Windows), unicode support, +a BSD-style license, and good functionality for completion and multi-line editing. + +Some other excellent libraries that we considered: +[GNU readline], +[editline](https://github.com/troglobit/editline), +[linenoise](https://github.com/antirez/linenoise), +[replxx](https://github.com/AmokHuginnsson/replxx), and +[Haskeline](https://github.com/judah/haskeline). + + +# Formatted Output + +Isocline also exposes functions for rich terminal output +as `ic_print` (and `ic_println` and `ic_printf`). +Inspired by the (Python) [Rich][RichBBcode] library, +this supports a form of [bbcode]'s to format the output: +```c +ic_println( "[b]bold [red]and red[/red][/b]" ); +``` +Each print automatically closes any open tags that were +not yet closed. Also, you can use a general close +tag as `[/]` to close the innermost tag, so the +following print is equivalent to the earlier one: +```c +ic_println( "[b]bold [red]and red[/]" ); +``` +There can be multiple styles in one tag +(where the first name is used for the closing tag): +```c +ic_println( "[u #FFD700]underlined gold[/]" ); +``` + +Sometimes, you need to display arbitrary messages +that may contain sequences that you would not like +to be interpreted as bbcode tags. One way to do +this is the `[!`_tag_`]` which ignores formatting +up to a close tag of the form `[/`_tag_`]`. +```c +ic_printf( "[red]red? [!pre]%s[/pre].\n", "[blue]not blue!" ); +``` + +Predefined styles include `b` (bold), +`u` (underline), `i` (italic), and `r` (reverse video), but +you can (re)define any style yourself as: +```c +ic_style_def("warning", "crimson u"); +``` + +and use them like any builtin style or property: +```c +ic_println( "[warning]this is a warning![/]" ); +``` +which is great for adding themes to your application. + +Each `ic_print` function always closes any unclosed tags automatically. +To open a style persistently, use `ic_style_open` with a matching +`ic_style_close` which scopes over any `ic_print` statements in between. +```c +ic_style_open("warning"); +ic_println("[b]crimson underlined and bold[/]"); +ic_style_close(); +``` + +# Advanced + + +## BBCode Format + +An open tag can have multiple white space separated +entries that are +either a _style name_, or a primitive _property_[`=`_value_]. + +### Styles + +Isocline provides the following builtin styles as property shorthands: +`b` (bold), `u` (underline), `i` (italic), `r` (reverse video), +and some builtin styles for syntax highlighting: +`keyword`, `control` (control-flow keywords), `string`, +`comment`, `number`, `type`, `constant`. + +Predefined styles used by Isocline itself are: + +- `ic-prompt`: prompt style, e.g. `ic_style_def("ic-prompt", "yellow on blue")`. +- `ic-info`: information (like the numbers in a completion menu). +- `ic-diminish`: dim text (used for example in history search). +- `ic-emphasis`: emphasized text (also used in history search). +- `ic-hint`: color of an inline hint. +- `ic-error`: error color (like an unmatched brace). +- `ic-bracematch`: color of matching parenthesis. + +### Properties + +Boolean properties are by default `on`: + +- `bold` [`=`(`on`|`off`)] +- `italic` [`=`(`on`|`off`)] +- `underline` [`=`(`on`|`off`)] +- `reverse` [`=`(`on`|`off`)] + +Color properties can be assigned a _color_: + +- `color=`_color_ +- `bgcolor=`_color_ +- _color_: equivalent to `color=`_color_. +- `on` _color_: equivalent to `bgcolor=`_color_. + +A color value can be specified in many ways: + +- any standard HTML [color name][htmlcolors]. +- any of the 16 standard ANSI [color names][ansicolors] by prefixing `ansi-` + (like `ansi-black` or `ansi-maroon`). + The actual color value of these depend on the a terminal theme. +- `#`_rrggbb_ or `#`_rgb_ for a specific 24-bit color. +- `ansi-color=`_idx_: where 0 <= _idx_ <= 256 specifies an entry in the + standard ANSI 256 [color palette][ansicolor256], where 256 is used for the ANSI + default color. + + +## Environment Variables + +- `NO_COLOR`: if present no colors are displayed. +- `CLICOLOR=1`: if set, the `LSCOLORS` or `LS_COLORS` environment variables are used to colorize + filename completions. +- `COLORTERM=`(`truecolor`|`256color`|`16color`|`8color`|`monochrome`): enable a certain color palette, see the next section. +- `TERM`: used on some systems to determine the color + +## Colors + +Isocline supports 24-bit colors and any RGB colors are automatically +mapped to a reduced palette on older terminals if these do not +support true color. Detection of full color support +is not always possible to do automatically and you can +set the `COLORTERM` environment variable expicitly to force Isocline to use +a specific palette: +- `COLORTERM=truecolor`: use 24-bit colors. + +- `COLORTERM=256color`: use the ANSI 256 color palette. + +- `COLORTERM=16color` : use the regular ANSI 16 color + palette (8 normal and 8 bright colors). + +- `COLORTERM=8color`: use bold for bright colors. +- `COLORTERM=monochrome`: use no color. + +The above screenshots are made with the +[`test_colors.c`](https://github.com/daanx/isocline/blob/main/test/test_colors.c) program. You can test your own +terminal as: +``` +$ gcc -o test_colors -Iinclude test/test_colors.c src/isocline.c +$ ./test_colors +$ COLORTERM=truecolor ./test_colors +$ COLORTERM=16color ./test_colors +``` + +## ANSI Escape Sequences + +Isocline uses just few ANSI escape sequences that are widely +supported: +- `ESC[`_n_`A`, `ESC[`_n_`B`, `ESC[`_n_`C`, and `ESC[`_n_`D`, + for moving the cursor _n_ places up, down, right, and left. +- `ESC[K` to clear the line from the cursor. +- `ESC[`_n_`m` for colors, with _n_ one of: 0 (reset), 1,22 (bold), 3,23 (italic), + 4,24 (underline), 7,27 (reverse), 30-37,40-47,90-97,100-107 (color), + and 39,49 (select default color). +- `ESC[38;5;`_n_`m`, `ESC[48;5;`_n_`m`, `ESC[38;2;`_r_`;`_g_`;`_b_`m`, `ESC[48;2;`_r_`;`_g_`;`_b_`m`: + on terminals that support it, select + entry _n_ from the + 256 color ANSI palette (used with `XTERM=xterm-256color` for example), or directly specify + any 24-bit _rgb_ color (used with `COLORTERM=truecolor`) for the foreground or background. + +On Windows the above functionality is implemented using the Windows console API +(except if running in the new Windows Terminal which supports these escape +sequences natively). + +## Async and Threads + +Isocline is _not_ thread-safe and `ic_readline`_xxx_ and `ic_print`_xxx_ should +be used from one thread only. + +The best way to use `ic_readline` asynchronously is +to run it in a (blocking) dedicated thread and deliver +results from there to the async event loop. Isocline has the +```C +bool ic_async_stop(void) +``` +function that is thread-safe and can deliver an +asynchronous event to Isocline that unblocks a current +`ic_readline` and makes it behave as if the user pressed +`ctrl-c` (which returns NULL from the read line call). + +## Color Mapping + +To map full RGB colors to an ANSI 256 or 16-color palette +Isocline finds a palette color with the minimal "color distance" to +the original color. There are various +ways of calculating this: one way is to take the euclidean distance +in the sRGB space (_simple-rgb_), a slightly better way is to +take a weighted distance where the weight distribution is adjusted +according to how big the red component is ([redmean](https://en.wikipedia.org/wiki/Color_difference), +denoted as _delta-rgb_ in the figure), +this is used by Isocline), +and finally, we can first translate into a perceptually uniform color space +(CIElab) and calculate the distance there using the [CIEDE2000](https://en.wikipedia.org/wiki/Color_difference) +algorithm (_ciede2000_). Here are these three methods compared on +some colors: + +![color space comparison](doc/color/colorspace-map.png) + +Each top row is the true 24-bit RGB color. Surprisingly, +the sophisticated CIEDE2000 distance seems less good here compared to the +simpler methods (as in the upper left block for example) +(perhaps because this algorithm was created to find close +perceptual colors in images where lightness differences may be given +less weight?). CIEDE2000 also leads to more "outliers", for example as seen +in column 5. Given these results, Isocline uses _redmean_ for +color mapping. We also add a gray correction that makes it less +likely to substitute a color for a gray value (and the other way +around). + + +## Possible Future Extensions + +- Vi key bindings. +- kill buffer. +- make the `ic_print`_xxx_ functions thread-safe. +- extended low-level terminal functions. +- status and progress bars. +- prompt variants: confirm, etc. +- ... + +Contact me if you are interested in doing any of these :-) + + +# Releases + +* `2022-01-15`: v1.0.9: fix missing `ic_completion_arg` (issue #6), + fix null ptr check in ic_print (issue #7), fix crash when using /dev/null as both input and output. +* `2021-09-05`: v1.0.5: use our own wcwidth for consistency; + thanks to Hans-Georg Breunig for helping with testing on NetBSD. +* `2021-08-28`: v1.0.4: fix color query on Ubuntu/Gnome +* `2021-08-27`: v1.0.3: fix duplicates in completions +* `2021-08-23`: v1.0.2: fix windows eol wrapping +* `2021-08-21`: v1.0.1: fix line-buffering +* `2021-08-20`: v1.0.0: initial release + + + +[GNU readline]: https://tiswww.case.edu/php/chet/readline/rltop.html +[koka]: http://www.koka-lang.org +[submodule]: https://git-scm.com/book/en/v2/Git-Tools-Submodules +[Haskell]: https://github.com/daanx/isocline/tree/main/haskell +[HaskellExample]: https://github.com/daanx/isocline/blob/main/test/Example.hs +[example]: https://github.com/daanx/isocline/blob/main/test/example.c +[termtosvg]: https://github.com/nbedos/termtosvg +[Rich]: https://github.com/willmcgugan/rich +[RichBBcode]: https://rich.readthedocs.io/en/latest/markup.html +[bbcode]: https://en.wikipedia.org/wiki/BBCode +[htmlcolors]: https://en.wikipedia.org/wiki/Web_colors#HTML_color_names +[ansicolors]: https://en.wikipedia.org/wiki/Web_colors#Basic_colors +[ansicolor256]: https://en.wikipedia.org/wiki/ANSI_escape_code#8-bit +[docapi]: https://daanx.github.io/isocline +[hdoc]: https://hackage.haskell.org/package/isocline/docs/System-Console-Isocline.html diff --git a/extern/isocline/src/attr.c b/extern/isocline/src/attr.c new file mode 100644 index 00000000..b5ad78f8 --- /dev/null +++ b/extern/isocline/src/attr.c @@ -0,0 +1,294 @@ +/* ---------------------------------------------------------------------------- + Copyright (c) 2021, Daan Leijen + This is free software; you can redistribute it and/or modify it + under the terms of the MIT License. A copy of the license can be + found in the "LICENSE" file at the root of this distribution. +-----------------------------------------------------------------------------*/ +#include + +#include "common.h" +#include "stringbuf.h" // str_next_ofs +#include "attr.h" +#include "term.h" // color_from_ansi256 + +//------------------------------------------------------------- +// Attributes +//------------------------------------------------------------- + +ic_private attr_t attr_none(void) { + attr_t attr; + attr.value = 0; + return attr; +} + +ic_private attr_t attr_default(void) { + attr_t attr = attr_none(); + attr.x.color = IC_ANSI_DEFAULT; + attr.x.bgcolor = IC_ANSI_DEFAULT; + attr.x.bold = IC_OFF; + attr.x.underline = IC_OFF; + attr.x.reverse = IC_OFF; + attr.x.italic = IC_OFF; + return attr; +} + +ic_private bool attr_is_none(attr_t attr) { + return (attr.value == 0); +} + +ic_private bool attr_is_eq(attr_t attr1, attr_t attr2) { + return (attr1.value == attr2.value); +} + +ic_private attr_t attr_from_color( ic_color_t color ) { + attr_t attr = attr_none(); + attr.x.color = color; + return attr; +} + + +ic_private attr_t attr_update_with( attr_t oldattr, attr_t newattr ) { + attr_t attr = oldattr; + if (newattr.x.color != IC_COLOR_NONE) { attr.x.color = newattr.x.color; } + if (newattr.x.bgcolor != IC_COLOR_NONE) { attr.x.bgcolor = newattr.x.bgcolor; } + if (newattr.x.bold != IC_NONE) { attr.x.bold = newattr.x.bold; } + if (newattr.x.italic != IC_NONE) { attr.x.italic = newattr.x.italic; } + if (newattr.x.reverse != IC_NONE) { attr.x.reverse = newattr.x.reverse; } + if (newattr.x.underline != IC_NONE) { attr.x.underline = newattr.x.underline; } + return attr; +} + +static bool sgr_is_digit(char c) { + return (c >= '0' && c <= '9'); +} + +static bool sgr_is_sep( char c ) { + return (c==';' || c==':'); +} + +static bool sgr_next_par(const char* s, ssize_t* pi, ssize_t* par) { + const ssize_t i = *pi; + ssize_t n = 0; + while( sgr_is_digit(s[i+n])) { + n++; + } + if (n==0) { + *par = 0; + return true; + } + else { + *pi = i+n; + return ic_atoz(s+i, par); + } +} + +static bool sgr_next_par3(const char* s, ssize_t* pi, ssize_t* p1, ssize_t* p2, ssize_t* p3) { + bool ok = false; + ssize_t i = *pi; + if (sgr_next_par(s,&i,p1) && sgr_is_sep(s[i])) { + i++; + if (sgr_next_par(s,&i,p2) && sgr_is_sep(s[i])) { + i++; + if (sgr_next_par(s,&i,p3)) { + ok = true; + }; + } + } + *pi = i; + return ok; +} + +ic_private attr_t attr_from_sgr( const char* s, ssize_t len) { + attr_t attr = attr_none(); + for( ssize_t i = 0; i < len && s[i] != 0; i++) { + ssize_t cmd = 0; + if (!sgr_next_par(s,&i,&cmd)) continue; + switch(cmd) { + case 0: attr = attr_default(); break; + case 1: attr.x.bold = IC_ON; break; + case 3: attr.x.italic = IC_ON; break; + case 4: attr.x.underline = IC_ON; break; + case 7: attr.x.reverse = IC_ON; break; + case 22: attr.x.bold = IC_OFF; break; + case 23: attr.x.italic = IC_OFF; break; + case 24: attr.x.underline = IC_OFF; break; + case 27: attr.x.reverse = IC_OFF; break; + case 39: attr.x.color = IC_ANSI_DEFAULT; break; + case 49: attr.x.bgcolor = IC_ANSI_DEFAULT; break; + default: { + if (cmd >= 30 && cmd <= 37) { + attr.x.color = IC_ANSI_BLACK + (unsigned)(cmd - 30); + } + else if (cmd >= 40 && cmd <= 47) { + attr.x.bgcolor = IC_ANSI_BLACK + (unsigned)(cmd - 40); + } + else if (cmd >= 90 && cmd <= 97) { + attr.x.color = IC_ANSI_DARKGRAY + (unsigned)(cmd - 90); + } + else if (cmd >= 100 && cmd <= 107) { + attr.x.bgcolor = IC_ANSI_DARKGRAY + (unsigned)(cmd - 100); + } + else if ((cmd == 38 || cmd == 48) && sgr_is_sep(s[i])) { + // non-associative SGR :-( + ssize_t par = 0; + i++; + if (sgr_next_par(s, &i, &par)) { + if (par==5 && sgr_is_sep(s[i])) { + // ansi 256 index + i++; + if (sgr_next_par(s, &i, &par) && par >= 0 && par <= 0xFF) { + ic_color_t color = color_from_ansi256(par); + if (cmd==38) { attr.x.color = color; } + else { attr.x.bgcolor = color; } + } + } + else if (par == 2 && sgr_is_sep(s[i])) { + // rgb value + i++; + ssize_t r,g,b; + if (sgr_next_par3(s, &i, &r,&g,&b)) { + ic_color_t color = ic_rgbx(r,g,b); + if (cmd==38) { attr.x.color = color; } + else { attr.x.bgcolor = color; } + } + } + } + } + else { + debug_msg("attr: unknow ANSI SGR code: %zd\n", cmd ); + } + } + } + } + return attr; +} + +ic_private attr_t attr_from_esc_sgr( const char* s, ssize_t len) { + if (len <= 2 || s[0] != '\x1B' || s[1] != '[' || s[len-1] != 'm') return attr_none(); + return attr_from_sgr(s+2, len-2); +} + + +//------------------------------------------------------------- +// Attribute buffer +//------------------------------------------------------------- +struct attrbuf_s { + attr_t* attrs; + ssize_t capacity; + ssize_t count; + alloc_t* mem; +}; + +static bool attrbuf_ensure_capacity( attrbuf_t* ab, ssize_t needed ) { + if (needed <= ab->capacity) return true; + ssize_t newcap = (ab->capacity <= 0 ? 240 : (ab->capacity > 1000 ? ab->capacity + 1000 : 2*ab->capacity)); + if (needed > newcap) { newcap = needed; } + attr_t* newattrs = mem_realloc_tp( ab->mem, attr_t, ab->attrs, newcap ); + if (newattrs == NULL) return false; + ab->attrs = newattrs; + ab->capacity = newcap; + assert(needed <= ab->capacity); + return true; +} + +static bool attrbuf_ensure_extra( attrbuf_t* ab, ssize_t extra ) { + const ssize_t needed = ab->count + extra; + return attrbuf_ensure_capacity( ab, needed ); +} + + +ic_private attrbuf_t* attrbuf_new( alloc_t* mem ) { + attrbuf_t* ab = mem_zalloc_tp(mem,attrbuf_t); + if (ab == NULL) return NULL; + ab->mem = mem; + attrbuf_ensure_extra(ab,1); + return ab; +} + +ic_private void attrbuf_free( attrbuf_t* ab ) { + if (ab==NULL) return; + mem_free(ab->mem, ab->attrs); + mem_free(ab->mem, ab); +} + +ic_private void attrbuf_clear(attrbuf_t* ab) { + if (ab == NULL) return; + ab->count = 0; +} + +ic_private ssize_t attrbuf_len( attrbuf_t* ab ) { + return (ab==NULL ? 0 : ab->count); +} + +ic_private const attr_t* attrbuf_attrs( attrbuf_t* ab, ssize_t expected_len ) { + assert(expected_len <= ab->count ); + // expand if needed + if (ab->count < expected_len) { + if (!attrbuf_ensure_capacity(ab,expected_len)) return NULL; + for(ssize_t i = ab->count; i < expected_len; i++) { + ab->attrs[i] = attr_none(); + } + ab->count = expected_len; + } + return ab->attrs; +} + + + +static void attrbuf_update_set_at( attrbuf_t* ab, ssize_t pos, ssize_t count, attr_t attr, bool update ) { + const ssize_t end = pos + count; + if (!attrbuf_ensure_capacity(ab, end)) return; + ssize_t i; + // initialize if end is beyond the count (todo: avoid duplicate init and set if update==false?) + if (ab->count < end) { + for(i = ab->count; i < end; i++) { + ab->attrs[i] = attr_none(); + } + ab->count = end; + } + // fill pos to end with attr + for(i = pos; i < end; i++) { + ab->attrs[i] = (update ? attr_update_with(ab->attrs[i],attr) : attr); + } +} + +ic_private void attrbuf_set_at( attrbuf_t* ab, ssize_t pos, ssize_t count, attr_t attr ) { + attrbuf_update_set_at(ab, pos, count, attr, false); +} + +ic_private void attrbuf_update_at( attrbuf_t* ab, ssize_t pos, ssize_t count, attr_t attr ) { + attrbuf_update_set_at(ab, pos, count, attr, true); +} + +ic_private void attrbuf_insert_at( attrbuf_t* ab, ssize_t pos, ssize_t count, attr_t attr ) { + if (pos < 0 || pos > ab->count || count <= 0) return; + if (!attrbuf_ensure_extra(ab,count)) return; + ic_memmove( ab->attrs + pos + count, ab->attrs + pos, (ab->count - pos)*ssizeof(attr_t) ); + ab->count += count; + attrbuf_set_at( ab, pos, count, attr ); +} + + +// note: must allow ab == NULL! +ic_private ssize_t attrbuf_append_n( stringbuf_t* sb, attrbuf_t* ab, const char* s, ssize_t len, attr_t attr ) { + if (s == NULL || len == 0) return sbuf_len(sb); + if (ab != NULL) { + if (!attrbuf_ensure_extra(ab,len)) return sbuf_len(sb); + attrbuf_set_at(ab, ab->count, len, attr); + } + return sbuf_append_n(sb,s,len); +} + +ic_private attr_t attrbuf_attr_at( attrbuf_t* ab, ssize_t pos ) { + if (ab==NULL || pos < 0 || pos > ab->count) return attr_none(); + return ab->attrs[pos]; +} + +ic_private void attrbuf_delete_at( attrbuf_t* ab, ssize_t pos, ssize_t count ) { + if (ab==NULL || pos < 0 || pos > ab->count) return; + if (pos + count > ab->count) { count = ab->count - pos; } + if (count == 0) return; + assert(pos + count <= ab->count); + ic_memmove( ab->attrs + pos, ab->attrs + pos + count, ab->count - (pos + count) ); + ab->count -= count; +} diff --git a/extern/isocline/src/attr.h b/extern/isocline/src/attr.h new file mode 100644 index 00000000..8f37d050 --- /dev/null +++ b/extern/isocline/src/attr.h @@ -0,0 +1,70 @@ +/* ---------------------------------------------------------------------------- + Copyright (c) 2021, Daan Leijen + This is free software; you can redistribute it and/or modify it + under the terms of the MIT License. A copy of the license can be + found in the "LICENSE" file at the root of this distribution. +-----------------------------------------------------------------------------*/ +#pragma once +#ifndef IC_ATTR_H +#define IC_ATTR_H + +#include "common.h" +#include "stringbuf.h" + +//------------------------------------------------------------- +// text attributes +//------------------------------------------------------------- + +#define IC_ON (1) +#define IC_OFF (-1) +#define IC_NONE (0) + +// try to fit in 64 bits +// note: order is important for some compilers +// note: each color can actually be 25 bits +typedef union attr_s { + struct { + unsigned int color:28; + signed int bold:2; + signed int reverse:2; + unsigned int bgcolor:28; + signed int underline:2; + signed int italic:2; + } x; + uint64_t value; +} attr_t; + +ic_private attr_t attr_none(void); +ic_private attr_t attr_default(void); +ic_private attr_t attr_from_color( ic_color_t color ); + +ic_private bool attr_is_none(attr_t attr); +ic_private bool attr_is_eq(attr_t attr1, attr_t attr2); + +ic_private attr_t attr_update_with( attr_t attr, attr_t newattr ); + +ic_private attr_t attr_from_sgr( const char* s, ssize_t len); +ic_private attr_t attr_from_esc_sgr( const char* s, ssize_t len); + +//------------------------------------------------------------- +// attribute buffer used for rich rendering +//------------------------------------------------------------- + +struct attrbuf_s; +typedef struct attrbuf_s attrbuf_t; + +ic_private attrbuf_t* attrbuf_new( alloc_t* mem ); +ic_private void attrbuf_free( attrbuf_t* ab ); // ab can be NULL +ic_private void attrbuf_clear( attrbuf_t* ab ); // ab can be NULL +ic_private ssize_t attrbuf_len( attrbuf_t* ab); // ab can be NULL +ic_private const attr_t* attrbuf_attrs( attrbuf_t* ab, ssize_t expected_len ); +ic_private ssize_t attrbuf_append_n( stringbuf_t* sb, attrbuf_t* ab, const char* s, ssize_t len, attr_t attr ); + +ic_private void attrbuf_set_at( attrbuf_t* ab, ssize_t pos, ssize_t count, attr_t attr ); +ic_private void attrbuf_update_at( attrbuf_t* ab, ssize_t pos, ssize_t count, attr_t attr ); +ic_private void attrbuf_insert_at( attrbuf_t* ab, ssize_t pos, ssize_t count, attr_t attr ); + +ic_private attr_t attrbuf_attr_at( attrbuf_t* ab, ssize_t pos ); +ic_private void attrbuf_delete_at( attrbuf_t* ab, ssize_t pos, ssize_t count ); + +#endif // IC_ATTR_H diff --git a/extern/isocline/src/bbcode.c b/extern/isocline/src/bbcode.c new file mode 100644 index 00000000..4d11ac38 --- /dev/null +++ b/extern/isocline/src/bbcode.c @@ -0,0 +1,842 @@ +/* ---------------------------------------------------------------------------- + Copyright (c) 2021, Daan Leijen + This is free software; you can redistribute it and/or modify it + under the terms of the MIT License. A copy of the license can be + found in the "LICENSE" file at the root of this distribution. +-----------------------------------------------------------------------------*/ +#include +#include +#include +#include + +#include "common.h" +#include "attr.h" +#include "term.h" +#include "bbcode.h" + +//------------------------------------------------------------- +// HTML color table +//------------------------------------------------------------- + +#include "bbcode_colors.c" + +//------------------------------------------------------------- +// Types +//------------------------------------------------------------- + +typedef struct style_s { + const char* name; // name of the style + attr_t attr; // attribute to apply +} style_t; + +typedef enum align_e { + IC_ALIGN_LEFT, + IC_ALIGN_CENTER, + IC_ALIGN_RIGHT +} align_t; + +typedef struct width_s { + ssize_t w; // > 0 + align_t align; + bool dots; // "..." (e.g. "sentence...") + char fill; // " " (e.g. "hello ") +} width_t; + +typedef struct tag_s { + const char* name; // tag open name + attr_t attr; // the saved attribute before applying the style + width_t width; // start sequence of at most "width" columns + ssize_t pos; // start position in the output (used for width restriction) +} tag_t; + + +static void tag_init(tag_t* tag) { + memset(tag,0,sizeof(*tag)); +} + +struct bbcode_s { + tag_t* tags; // stack of tags; one entry for each open tag + ssize_t tags_capacity; + ssize_t tags_nesting; + style_t* styles; // list of used defined styles + ssize_t styles_capacity; + ssize_t styles_count; + term_t* term; // terminal + alloc_t* mem; // allocator + // caches + stringbuf_t* out; // print buffer + attrbuf_t* out_attrs; + stringbuf_t* vout; // vprintf buffer +}; + + +//------------------------------------------------------------- +// Create, helpers +//------------------------------------------------------------- + +ic_private bbcode_t* bbcode_new( alloc_t* mem, term_t* term ) { + bbcode_t* bb = mem_zalloc_tp(mem,bbcode_t); + if (bb==NULL) return NULL; + bb->mem = mem; + bb->term = term; + bb->out = sbuf_new(mem); + bb->out_attrs = attrbuf_new(mem); + bb->vout = sbuf_new(mem); + return bb; +} + +ic_private void bbcode_free( bbcode_t* bb ) { + for(ssize_t i = 0; i < bb->styles_count; i++) { + mem_free(bb->mem, bb->styles[i].name); + } + mem_free(bb->mem, bb->tags); + mem_free(bb->mem, bb->styles); + sbuf_free(bb->vout); + sbuf_free(bb->out); + attrbuf_free(bb->out_attrs); + mem_free(bb->mem, bb); +} + +ic_private void bbcode_style_add( bbcode_t* bb, const char* style_name, attr_t attr ) { + if (bb->styles_count >= bb->styles_capacity) { + ssize_t newlen = bb->styles_capacity + 32; + style_t* p = mem_realloc_tp( bb->mem, style_t, bb->styles, newlen ); + if (p == NULL) return; + bb->styles = p; + bb->styles_capacity = newlen; + } + assert(bb->styles_count < bb->styles_capacity); + bb->styles[bb->styles_count].name = mem_strdup( bb->mem, style_name ); + bb->styles[bb->styles_count].attr = attr; + bb->styles_count++; +} + +static ssize_t bbcode_tag_push( bbcode_t* bb, const tag_t* tag ) { + if (bb->tags_nesting >= bb->tags_capacity) { + ssize_t newcap = bb->tags_capacity + 32; + tag_t* p = mem_realloc_tp( bb->mem, tag_t, bb->tags, newcap ); + if (p == NULL) return -1; + bb->tags = p; + bb->tags_capacity = newcap; + } + assert(bb->tags_nesting < bb->tags_capacity); + bb->tags[bb->tags_nesting] = *tag; + bb->tags_nesting++; + return (bb->tags_nesting-1); +} + +static void bbcode_tag_pop( bbcode_t* bb, tag_t* tag ) { + if (bb->tags_nesting <= 0) { + if (tag != NULL) { tag_init(tag); } + } + else { + bb->tags_nesting--; + if (tag != NULL) { + *tag = bb->tags[bb->tags_nesting]; + } + } +} + +//------------------------------------------------------------- +// Invalid parse/values/balance +//------------------------------------------------------------- + +static void bbcode_invalid(const char* fmt, ... ) { + if (getenv("ISOCLINE_BBCODE_DEBUG") != NULL) { + va_list args; + va_start(args,fmt); + vfprintf(stderr,fmt,args); + va_end(args); + } +} + +//------------------------------------------------------------- +// Set attributes +//------------------------------------------------------------- + + +static attr_t bbcode_open( bbcode_t* bb, ssize_t out_pos, const tag_t* tag, attr_t current ) { + // save current and set + tag_t cur; + tag_init(&cur); + cur.name = tag->name; + cur.attr = current; + cur.width = tag->width; + cur.pos = out_pos; + bbcode_tag_push(bb,&cur); + return attr_update_with( current, tag->attr ); +} + +static bool bbcode_close( bbcode_t* bb, ssize_t base, const char* name, tag_t* pprev ) { + // pop until match + while (bb->tags_nesting > base) { + tag_t prev; + bbcode_tag_pop(bb,&prev); + if (name==NULL || prev.name==NULL || ic_stricmp(prev.name,name) == 0) { + // matched + if (pprev != NULL) { *pprev = prev; } + return true; + } + else { + // unbalanced: we either continue popping or restore the tags depending if there is a matching open tag in our tags. + bool has_open_tag = false; + if (name != NULL) { + for( ssize_t i = bb->tags_nesting - 1; i > base; i--) { + if (bb->tags[i].name != NULL && ic_stricmp(bb->tags[i].name, name) == 0) { + has_open_tag = true; + break; + } + } + } + bbcode_invalid("bbcode: unbalanced tags: open [%s], close [/%s]\n", prev.name, name); + if (!has_open_tag) { + bbcode_tag_push( bb, &prev ); // restore the tags and ignore this close tag + break; + } + else { + // continue until we hit our open tag + } + } + } + if (pprev != NULL) { memset(pprev,0,sizeof(*pprev)); } + return false; +} + +//------------------------------------------------------------- +// Update attributes +//------------------------------------------------------------- + +static const char* attr_update_bool( const char* fname, signed int* field, const char* value ) { + if (value == NULL || value[0] == 0 || strcmp(value,"on") || strcmp(value,"true") || strcmp(value,"1")) { + *field = IC_ON; + } + else if (strcmp(value,"off") || strcmp(value,"false") || strcmp(value,"0")) { + *field = IC_OFF; + } + else { + bbcode_invalid("bbcode: invalid %s value: %s\n", fname, value ); + } + return fname; +} + +static const char* attr_update_color( const char* fname, ic_color_t* field, const char* value ) { + if (value == NULL || value[0] == 0 || strcmp(value,"none") == 0) { + *field = IC_COLOR_NONE; + return fname; + } + + // hex value + if (value[0] == '#') { + uint32_t rgb = 0; + if (sscanf(value,"#%x",&rgb) == 1) { + *field = ic_rgb(rgb); + } + else { + bbcode_invalid("bbcode: invalid color code: %s\n", value); + } + return fname; + } + + // search color names + ssize_t lo = 0; + ssize_t hi = IC_HTML_COLOR_COUNT-1; + while( lo <= hi ) { + ssize_t mid = (lo + hi) / 2; + style_color_t* info = &html_colors[mid]; + int cmp = strcmp(info->name,value); + if (cmp < 0) { + lo = mid+1; + } + else if (cmp > 0) { + hi = mid-1; + } + else { + *field = info->color; + return fname; + } + } + bbcode_invalid("bbcode: unknown %s: %s\n", fname, value); + *field = IC_COLOR_NONE; + return fname; +} + +static const char* attr_update_sgr( const char* fname, attr_t* attr, const char* value ) { + *attr = attr_update_with(*attr, attr_from_sgr(value, ic_strlen(value))); + return fname; +} + +static void attr_update_width( width_t* pwidth, char default_fill, const char* value ) { + // parse width value: ;;; + width_t width; + memset(&width, 0, sizeof(width)); + width.fill = default_fill; // use 0 for no-fill (as for max-width) + if (ic_atoz(value, &width.w)) { + ssize_t i = 0; + while( value[i] != ';' && value[i] != 0 ) { i++; } + if (value[i] == ';') { + i++; + ssize_t len = 0; + while( value[i+len] != ';' && value[i+len] != 0 ) { len++; } + if (len == 4 && ic_istarts_with(value+i,"left")) { + width.align = IC_ALIGN_LEFT; + } + else if (len == 5 && ic_istarts_with(value+i,"right")) { + width.align = IC_ALIGN_RIGHT; + } + else if (len == 6 && ic_istarts_with(value+i,"center")) { + width.align = IC_ALIGN_CENTER; + } + i += len; + if (value[i] == ';') { + i++; len = 0; + while( value[i+len] != ';' && value[i+len] != 0 ) { len++; } + if (len == 1) { width.fill = value[i]; } + i+= len; + if (value[i] == ';') { + i++; len = 0; + while( value[i+len] != ';' && value[i+len] != 0 ) { len++; } + if ((len == 2 && ic_istarts_with(value+i,"on")) || (len == 1 && value[i] == '1')) { width.dots = true; } + i += len; + } + } + } + } + else { + bbcode_invalid("bbcode: illegal width: %s\n", value); + } + *pwidth = width; +} + +static const char* attr_update_ansi_color( const char* fname, ic_color_t* color, const char* value ) { + ssize_t num = 0; + if (ic_atoz(value, &num) && num >= 0 && num <= 256) { + *color = color_from_ansi256(num); + } + return fname; +} + + +static const char* attr_update_property( tag_t* tag, const char* attr_name, const char* value ) { + const char* fname = NULL; + fname = "bold"; + if (strcmp(attr_name,fname) == 0) { + signed int b = IC_NONE; + attr_update_bool(fname,&b, value); + if (b != IC_NONE) { tag->attr.x.bold = b; } + return fname; + } + fname = "italic"; + if (strcmp(attr_name,fname) == 0) { + signed int b = IC_NONE; + attr_update_bool(fname,&b, value); + if (b != IC_NONE) { tag->attr.x.italic = b; } + return fname; + } + fname = "underline"; + if (strcmp(attr_name,fname) == 0) { + signed int b = IC_NONE; + attr_update_bool(fname,&b, value); + if (b != IC_NONE) { tag->attr.x.underline = b; } + return fname; + } + fname = "reverse"; + if (strcmp(attr_name,fname) == 0) { + signed int b = IC_NONE; + attr_update_bool(fname,&b, value); + if (b != IC_NONE) { tag->attr.x.reverse = b; } + return fname; + } + fname = "color"; + if (strcmp(attr_name,fname) == 0) { + unsigned int color = IC_COLOR_NONE; + attr_update_color(fname, &color, value); + if (color != IC_COLOR_NONE) { tag->attr.x.color = color; } + return fname; + } + fname = "bgcolor"; + if (strcmp(attr_name,fname) == 0) { + unsigned int color = IC_COLOR_NONE; + attr_update_color(fname, &color, value); + if (color != IC_COLOR_NONE) { tag->attr.x.bgcolor = color; } + return fname; + } + fname = "ansi-sgr"; + if (strcmp(attr_name,fname) == 0) { + attr_update_sgr(fname, &tag->attr, value); + return fname; + } + fname = "ansi-color"; + if (strcmp(attr_name,fname) == 0) { + ic_color_t color = IC_COLOR_NONE;; + attr_update_ansi_color(fname, &color, value); + if (color != IC_COLOR_NONE) { tag->attr.x.color = color; } + return fname; + } + fname = "ansi-bgcolor"; + if (strcmp(attr_name,fname) == 0) { + ic_color_t color = IC_COLOR_NONE;; + attr_update_ansi_color(fname, &color, value); + if (color != IC_COLOR_NONE) { tag->attr.x.bgcolor = color; } + return fname; + } + fname = "width"; + if (strcmp(attr_name,fname) == 0) { + attr_update_width(&tag->width, ' ', value); + return fname; + } + fname = "max-width"; + if (strcmp(attr_name,fname) == 0) { + attr_update_width(&tag->width, 0, value); + return "width"; + } + else { + return NULL; + } +} + +static const style_t builtin_styles[] = { + { "b", { { IC_COLOR_NONE, IC_ON , IC_NONE, IC_COLOR_NONE, IC_NONE, IC_NONE } } }, + { "r", { { IC_COLOR_NONE, IC_NONE, IC_ON , IC_COLOR_NONE, IC_NONE, IC_NONE } } }, + { "u", { { IC_COLOR_NONE, IC_NONE, IC_NONE, IC_COLOR_NONE, IC_ON , IC_NONE } } }, + { "i", { { IC_COLOR_NONE, IC_NONE, IC_NONE, IC_COLOR_NONE, IC_NONE, IC_ON } } }, + { "em", { { IC_COLOR_NONE, IC_ON , IC_NONE, IC_COLOR_NONE, IC_NONE, IC_NONE } } }, // bold + { "url",{ { IC_COLOR_NONE, IC_NONE, IC_NONE, IC_COLOR_NONE, IC_ON, IC_NONE } } }, // underline + { NULL, { { IC_COLOR_NONE, IC_NONE, IC_NONE, IC_COLOR_NONE, IC_NONE, IC_NONE } } } +}; + +static void attr_update_with_styles( tag_t* tag, const char* attr_name, const char* value, + bool usebgcolor, const style_t* styles, ssize_t count ) +{ + // direct hex color? + if (attr_name[0] == '#' && (value == NULL || value[0]==0)) { + value = attr_name; + attr_name = (usebgcolor ? "bgcolor" : "color"); + } + // first try if it is a builtin property + const char* name; + if ((name = attr_update_property(tag,attr_name,value)) != NULL) { + if (tag->name != NULL) tag->name = name; + return; + } + // then check all styles + while( count-- > 0 ) { + const style_t* style = styles + count; + if (strcmp(style->name,attr_name) == 0) { + tag->attr = attr_update_with(tag->attr,style->attr); + if (tag->name != NULL) tag->name = style->name; + return; + } + } + // check builtin styles; todo: binary search? + for( const style_t* style = builtin_styles; style->name != NULL; style++) { + if (strcmp(style->name,attr_name) == 0) { + tag->attr = attr_update_with(tag->attr,style->attr); + if (tag->name != NULL) tag->name = style->name; + return; + } + } + // check colors as a style + ssize_t lo = 0; + ssize_t hi = IC_HTML_COLOR_COUNT-1; + while( lo <= hi ) { + ssize_t mid = (lo + hi) / 2; + style_color_t* info = &html_colors[mid]; + int cmp = strcmp(info->name,attr_name); + if (cmp < 0) { + lo = mid+1; + } + else if (cmp > 0) { + hi = mid-1; + } + else { + attr_t cattr = attr_none(); + if (usebgcolor) { cattr.x.bgcolor = info->color; } + else { cattr.x.color = info->color; } + tag->attr = attr_update_with(tag->attr,cattr); + if (tag->name != NULL) tag->name = info->name; + return; + } + } + // not found + bbcode_invalid("bbcode: unknown style: %s\n", attr_name); +} + + +ic_private attr_t bbcode_style( bbcode_t* bb, const char* style_name ) { + tag_t tag; + tag_init(&tag); + attr_update_with_styles( &tag, style_name, NULL, false, bb->styles, bb->styles_count ); + return tag.attr; +} + +//------------------------------------------------------------- +// Parse tags +//------------------------------------------------------------- + +ic_private const char* parse_skip_white(const char* s) { + while( *s != 0 && *s != ']') { + if (!(*s == ' ' || *s == '\t' || *s == '\n' || *s == '\r')) break; + s++; + } + return s; +} + +ic_private const char* parse_skip_to_white(const char* s) { + while( *s != 0 && *s != ']') { + if (*s == ' ' || *s == '\t' || *s == '\n' || *s == '\r') break; + s++; + } + return parse_skip_white(s); +} + +ic_private const char* parse_skip_to_end(const char* s) { + while( *s != 0 && *s != ']' ) { s++; } + return s; +} + +ic_private const char* parse_attr_name(const char* s) { + if (*s == '#') { + s++; // hex rgb color as id + while( *s != 0 && *s != ']') { + if (!((*s >= 'a' && *s <= 'f') || (*s >= 'A' && *s <= 'Z') || (*s >= '0' && *s <= '9'))) break; + s++; + } + } + else { + while( *s != 0 && *s != ']') { + if (!((*s >= 'a' && *s <= 'z') || (*s >= 'A' && *s <= 'Z') || + (*s >= '0' && *s <= '9') || *s == '_' || *s == '-')) break; + s++; + } + } + return s; +} + +ic_private const char* parse_value(const char* s, const char** start, const char** end) { + if (*s == '"') { + s++; + *start = s; + while( *s != 0 ) { + if (*s == '"') break; + s++; + } + *end = s; + if (*s == '"') { s++; } + } + else if (*s == '#') { + *start = s; + s++; + while( *s != 0 ) { + if (!((*s >= 'a' && *s <= 'f') || (*s >= 'A' && *s <= 'Z') || (*s >= '0' && *s <= '9'))) break; + s++; + } + *end = s; + } + else { + *start = s; + while( *s != 0 ) { + if (!((*s >= 'a' && *s <= 'z') || (*s >= 'A' && *s <= 'F') || (*s >= '0' && *s <= '9') || *s == '-' || *s == '_')) break; + s++; + } + *end = s; + } + return s; +} + +ic_private const char* parse_tag_value( tag_t* tag, char* idbuf, const char* s, const style_t* styles, ssize_t scount ) { + // parse: \s*[\w-]+\s*(=\s*) + bool usebgcolor = false; + const char* id = s; + const char* idend = parse_attr_name(id); + const char* val = NULL; + const char* valend = NULL; + if (id == idend) { + bbcode_invalid("bbcode: empty identifier? %.10s...\n", id ); + return parse_skip_to_white(id); + } + // "on" bgcolor? + s = parse_skip_white(idend); + if (idend - id == 2 && ic_strnicmp(id,"on",2) == 0 && *s != '=') { + usebgcolor = true; + id = s; + idend = parse_attr_name(id); + if (id == idend) { + bbcode_invalid("bbcode: empty identifier follows 'on'? %.10s...\n", id ); + return parse_skip_to_white(id); + } + s = parse_skip_white(idend); + } + // value + if (*s == '=') { + s++; + s = parse_skip_white(s); + s = parse_value(s, &val, &valend); + s = parse_skip_white(s); + } + // limit name and attr to 128 bytes + char valbuf[128]; + ic_strncpy( idbuf, 128, id, idend - id); + ic_strncpy( valbuf, 128, val, valend - val); + ic_str_tolower(idbuf); + ic_str_tolower(valbuf); + attr_update_with_styles( tag, idbuf, valbuf, usebgcolor, styles, scount ); + return s; +} + +static const char* parse_tag_values( tag_t* tag, char* idbuf, const char* s, const style_t* styles, ssize_t scount ) { + s = parse_skip_white(s); + idbuf[0] = 0; + ssize_t count = 0; + while( *s != 0 && *s != ']') { + char idbuf_next[128]; + s = parse_tag_value(tag, (count==0 ? idbuf : idbuf_next), s, styles, scount); + count++; + } + if (*s == ']') { s++; } + return s; +} + +static const char* parse_tag( tag_t* tag, char* idbuf, bool* open, bool* pre, const char* s, const style_t* styles, ssize_t scount ) { + *open = true; + *pre = false; + if (*s != '[') return s; + s = parse_skip_white(s+1); + if (*s == '!') { // pre + *pre = true; + s = parse_skip_white(s+1); + } + else if (*s == '/') { + *open = false; + s = parse_skip_white(s+1); + }; + s = parse_tag_values( tag, idbuf, s, styles, scount); + return s; +} + + +//--------------------------------------------------------- +// Styles +//--------------------------------------------------------- + +static void bbcode_parse_tag_content( bbcode_t* bb, const char* s, tag_t* tag ) { + tag_init(tag); + if (s != NULL) { + char idbuf[128]; + parse_tag_values(tag, idbuf, s, bb->styles, bb->styles_count); + } +} + +ic_private void bbcode_style_def( bbcode_t* bb, const char* style_name, const char* s ) { + tag_t tag; + bbcode_parse_tag_content( bb, s, &tag); + bbcode_style_add(bb, style_name, tag.attr); +} + +ic_private void bbcode_style_open( bbcode_t* bb, const char* fmt ) { + tag_t tag; + bbcode_parse_tag_content(bb, fmt, &tag); + term_set_attr( bb->term, bbcode_open(bb, 0, &tag, term_get_attr(bb->term)) ); +} + +ic_private void bbcode_style_close( bbcode_t* bb, const char* fmt ) { + const ssize_t base = bb->tags_nesting - 1; // as we end a style + tag_t tag; + bbcode_parse_tag_content(bb, fmt, &tag); + tag_t prev; + if (bbcode_close(bb, base, tag.name, &prev)) { + term_set_attr( bb->term, prev.attr ); + } +} + +//--------------------------------------------------------- +// Restrict to width +//--------------------------------------------------------- + +static void bbcode_restrict_width( ssize_t start, width_t width, stringbuf_t* out, attrbuf_t* attr_out ) { + if (width.w <= 0) return; + assert(start <= sbuf_len(out)); + assert(attr_out == NULL || sbuf_len(out) == attrbuf_len(attr_out)); + const char* s = sbuf_string(out) + start; + const ssize_t len = sbuf_len(out) - start; + const ssize_t w = str_column_width(s); + if (w == width.w) return; // fits exactly + if (w > width.w) { + // too large + ssize_t innerw = (width.dots && width.w > 3 ? width.w-3 : width.w); + if (width.align == IC_ALIGN_RIGHT) { + // right align + const ssize_t ndel = str_skip_until_fit( s, innerw ); + sbuf_delete_at( out, start, ndel ); + attrbuf_delete_at( attr_out, start, ndel ); + if (innerw < width.w) { + // add dots + sbuf_insert_at( out, "...", start ); + attr_t attr = attrbuf_attr_at(attr_out, start); + attrbuf_insert_at( attr_out, start, 3, attr); + } + } + else { + // left or center align + ssize_t count = str_take_while_fit( s, innerw ); + sbuf_delete_at( out, start + count, len - count ); + attrbuf_delete_at( attr_out, start + count, len - count ); + if (innerw < width.w) { + // add dots + attr_t attr = attrbuf_attr_at(attr_out,start); + attrbuf_append_n( out, attr_out, "...", 3, attr ); + } + } + } + else { + // too short, pad to width + const ssize_t diff = (width.w - w); + const ssize_t pad_left = (width.align == IC_ALIGN_RIGHT ? diff : (width.align == IC_ALIGN_LEFT ? 0 : diff / 2)); + const ssize_t pad_right = (width.align == IC_ALIGN_LEFT ? diff : (width.align == IC_ALIGN_RIGHT ? 0 : diff - pad_left)); + if (width.fill != 0 && pad_left > 0) { + const attr_t attr = attrbuf_attr_at(attr_out,start); + for( ssize_t i = 0; i < pad_left; i++) { // todo: optimize + sbuf_insert_char_at(out, width.fill, start); + } + attrbuf_insert_at( attr_out, start, pad_left, attr ); + } + if (width.fill != 0 && pad_right > 0) { + const attr_t attr = attrbuf_attr_at(attr_out,sbuf_len(out) - 1); + char buf[2]; + buf[0] = width.fill; + buf[1] = 0; + for( ssize_t i = 0; i < pad_right; i++) { // todo: optimize + attrbuf_append_n( out, attr_out, buf, 1, attr ); + } + } + } +} + +//--------------------------------------------------------- +// Print +//--------------------------------------------------------- + +ic_private ssize_t bbcode_process_tag( bbcode_t* bb, const char* s, const ssize_t nesting_base, + stringbuf_t* out, attrbuf_t* attr_out, attr_t* cur_attr ) { + assert(*s == '['); + tag_t tag; + tag_init(&tag); + bool open = true; + bool ispre = false; + char idbuf[128]; + const char* end = parse_tag( &tag, idbuf, &open, &ispre, s, bb->styles, bb->styles_count ); // todo: styles + assert(end > s); + if (open) { + if (!ispre) { + // open tag + *cur_attr = bbcode_open( bb, sbuf_len(out), &tag, *cur_attr ); + } + else { + // scan pre to end tag + attr_t attr = attr_update_with(*cur_attr, tag.attr); + char pre[132]; + if (snprintf(pre, 132, "[/%s]", idbuf) < ssizeof(pre)) { + const char* etag = strstr(end,pre); + if (etag == NULL) { + const ssize_t len = ic_strlen(end); + attrbuf_append_n(out, attr_out, end, len, attr); + end += len; + } + else { + attrbuf_append_n(out, attr_out, end, (etag - end), attr); + end = etag + ic_strlen(pre); + } + } + } + } + else { + // pop the tag + tag_t prev; + if (bbcode_close( bb, nesting_base, tag.name, &prev)) { + *cur_attr = prev.attr; + if (prev.width.w > 0) { + // closed a width tag; restrict the output to width + bbcode_restrict_width( prev.pos, prev.width, out, attr_out); + } + } + } + return (end - s); +} + +ic_private void bbcode_append( bbcode_t* bb, const char* s, stringbuf_t* out, attrbuf_t* attr_out ) { + if (bb == NULL || s == NULL) return; + attr_t attr = attr_none(); + const ssize_t base = bb->tags_nesting; // base; will not be popped + ssize_t i = 0; + while( s[i] != 0 ) { + // handle no tags in bulk + ssize_t nobb = 0; + char c; + while( (c = s[i+nobb]) != 0) { + if (c == '[' || c == '\\') { break; } + if (c == '\x1B' && s[i+nobb+1] == '[') { + nobb++; // don't count 'ESC[' as a tag opener + } + nobb++; + } + if (nobb > 0) { attrbuf_append_n(out, attr_out, s+i, nobb, attr); } + i += nobb; + // tag + if (s[i] == '[') { + i += bbcode_process_tag(bb, s+i, base, out, attr_out, &attr); + } + else if (s[i] == '\\') { + if (s[i+1] == '\\' || s[i+1] == '[') { + attrbuf_append_n(out, attr_out, s+i+1, 1, attr); // escape '\[' and '\\' + i += 2; + } + else { + attrbuf_append_n(out, attr_out, s+i, 1, attr); // pass '\\' as is + i++; + } + } + } + // pop unclosed openings + assert(bb->tags_nesting >= base); + while( bb->tags_nesting > base ) { + bbcode_tag_pop(bb,NULL); + }; +} + +ic_private void bbcode_print( bbcode_t* bb, const char* s ) { + if (bb->out == NULL || bb->out_attrs == NULL || s == NULL) return; + assert(sbuf_len(bb->out) == 0 && attrbuf_len(bb->out_attrs) == 0); + bbcode_append( bb, s, bb->out, bb->out_attrs ); + term_write_formatted( bb->term, sbuf_string(bb->out), attrbuf_attrs(bb->out_attrs,sbuf_len(bb->out)) ); + attrbuf_clear(bb->out_attrs); + sbuf_clear(bb->out); +} + +ic_private void bbcode_println( bbcode_t* bb, const char* s ) { + bbcode_print(bb,s); + term_writeln(bb->term, ""); +} + +ic_private void bbcode_vprintf( bbcode_t* bb, const char* fmt, va_list args ) { + if (bb->vout == NULL || fmt == NULL) return; + assert(sbuf_len(bb->vout) == 0); + sbuf_append_vprintf(bb->vout,fmt,args); + bbcode_print(bb, sbuf_string(bb->vout)); + sbuf_clear(bb->vout); +} + +ic_private void bbcode_printf( bbcode_t* bb, const char* fmt, ... ) { + va_list args; + va_start(args,fmt); + bbcode_vprintf(bb,fmt,args); + va_end(args); +} + +ic_private ssize_t bbcode_column_width( bbcode_t* bb, const char* s ) { + if (s==NULL || s[0] == 0) return 0; + if (bb->vout == NULL) { return str_column_width(s); } + assert(sbuf_len(bb->vout) == 0); + bbcode_append( bb, s, bb->vout, NULL); + const ssize_t w = str_column_width(sbuf_string(bb->vout)); + sbuf_clear(bb->vout); + return w; +} diff --git a/extern/isocline/src/bbcode.h b/extern/isocline/src/bbcode.h new file mode 100644 index 00000000..be96bfe2 --- /dev/null +++ b/extern/isocline/src/bbcode.h @@ -0,0 +1,37 @@ +/* ---------------------------------------------------------------------------- + Copyright (c) 2021, Daan Leijen + This is free software; you can redistribute it and/or modify it + under the terms of the MIT License. A copy of the license can be + found in the "LICENSE" file at the root of this distribution. +-----------------------------------------------------------------------------*/ +#pragma once +#ifndef IC_BBCODE_H +#define IC_BBCODE_H + +#include +#include "common.h" +#include "term.h" + +struct bbcode_s; +typedef struct bbcode_s bbcode_t; + +ic_private bbcode_t* bbcode_new( alloc_t* mem, term_t* term ); +ic_private void bbcode_free( bbcode_t* bb ); + +ic_private void bbcode_style_add( bbcode_t* bb, const char* style_name, attr_t attr ); +ic_private void bbcode_style_def( bbcode_t* bb, const char* style_name, const char* s ); +ic_private void bbcode_style_open( bbcode_t* bb, const char* fmt ); +ic_private void bbcode_style_close( bbcode_t* bb, const char* fmt ); +ic_private attr_t bbcode_style( bbcode_t* bb, const char* style_name ); + +ic_private void bbcode_print( bbcode_t* bb, const char* s ); +ic_private void bbcode_println( bbcode_t* bb, const char* s ); +ic_private void bbcode_printf( bbcode_t* bb, const char* fmt, ... ); +ic_private void bbcode_vprintf( bbcode_t* bb, const char* fmt, va_list args ); + +ic_private ssize_t bbcode_column_width( bbcode_t* bb, const char* s ); + +// allows `attr_out == NULL`. +ic_private void bbcode_append( bbcode_t* bb, const char* s, stringbuf_t* out, attrbuf_t* attr_out ); + +#endif // IC_BBCODE_H diff --git a/extern/isocline/src/bbcode_colors.c b/extern/isocline/src/bbcode_colors.c new file mode 100644 index 00000000..245cd3de --- /dev/null +++ b/extern/isocline/src/bbcode_colors.c @@ -0,0 +1,194 @@ +/* ---------------------------------------------------------------------------- + Copyright (c) 2021, Daan Leijen + This is free software; you can redistribute it and/or modify it + under the terms of the MIT License. A copy of the license can be + found in the "LICENSE" file at the root of this distribution. +-----------------------------------------------------------------------------*/ + +// This file is included from "bbcode.c" and contains html color names + +#include "common.h" + +typedef struct style_color_s { + const char* name; + ic_color_t color; +} style_color_t; + +#define IC_HTML_COLOR_COUNT (172) + +// ordered list of HTML color names (so we can use binary search) +static style_color_t html_colors[IC_HTML_COLOR_COUNT+1] = { + { "aliceblue", IC_RGB(0xf0f8ff) }, + { "ansi-aqua", IC_ANSI_AQUA }, + { "ansi-black", IC_ANSI_BLACK }, + { "ansi-blue", IC_ANSI_BLUE }, + { "ansi-cyan", IC_ANSI_CYAN }, + { "ansi-darkgray", IC_ANSI_DARKGRAY }, + { "ansi-darkgrey", IC_ANSI_DARKGRAY }, + { "ansi-default", IC_ANSI_DEFAULT }, + { "ansi-fuchsia", IC_ANSI_FUCHSIA }, + { "ansi-gray", IC_ANSI_GRAY }, + { "ansi-green", IC_ANSI_GREEN }, + { "ansi-grey", IC_ANSI_GRAY }, + { "ansi-lightgray", IC_ANSI_LIGHTGRAY }, + { "ansi-lightgrey", IC_ANSI_LIGHTGRAY }, + { "ansi-lime" , IC_ANSI_LIME }, + { "ansi-magenta", IC_ANSI_MAGENTA }, + { "ansi-maroon", IC_ANSI_MAROON }, + { "ansi-navy", IC_ANSI_NAVY }, + { "ansi-olive", IC_ANSI_OLIVE }, + { "ansi-purple", IC_ANSI_PURPLE }, + { "ansi-red", IC_ANSI_RED }, + { "ansi-silver", IC_ANSI_SILVER }, + { "ansi-teal", IC_ANSI_TEAL }, + { "ansi-white", IC_ANSI_WHITE }, + { "ansi-yellow", IC_ANSI_YELLOW }, + { "antiquewhite", IC_RGB(0xfaebd7) }, + { "aqua", IC_RGB(0x00ffff) }, + { "aquamarine", IC_RGB(0x7fffd4) }, + { "azure", IC_RGB(0xf0ffff) }, + { "beige", IC_RGB(0xf5f5dc) }, + { "bisque", IC_RGB(0xffe4c4) }, + { "black", IC_RGB(0x000000) }, + { "blanchedalmond", IC_RGB(0xffebcd) }, + { "blue", IC_RGB(0x0000ff) }, + { "blueviolet", IC_RGB(0x8a2be2) }, + { "brown", IC_RGB(0xa52a2a) }, + { "burlywood", IC_RGB(0xdeb887) }, + { "cadetblue", IC_RGB(0x5f9ea0) }, + { "chartreuse", IC_RGB(0x7fff00) }, + { "chocolate", IC_RGB(0xd2691e) }, + { "coral", IC_RGB(0xff7f50) }, + { "cornflowerblue", IC_RGB(0x6495ed) }, + { "cornsilk", IC_RGB(0xfff8dc) }, + { "crimson", IC_RGB(0xdc143c) }, + { "cyan", IC_RGB(0x00ffff) }, + { "darkblue", IC_RGB(0x00008b) }, + { "darkcyan", IC_RGB(0x008b8b) }, + { "darkgoldenrod", IC_RGB(0xb8860b) }, + { "darkgray", IC_RGB(0xa9a9a9) }, + { "darkgreen", IC_RGB(0x006400) }, + { "darkgrey", IC_RGB(0xa9a9a9) }, + { "darkkhaki", IC_RGB(0xbdb76b) }, + { "darkmagenta", IC_RGB(0x8b008b) }, + { "darkolivegreen", IC_RGB(0x556b2f) }, + { "darkorange", IC_RGB(0xff8c00) }, + { "darkorchid", IC_RGB(0x9932cc) }, + { "darkred", IC_RGB(0x8b0000) }, + { "darksalmon", IC_RGB(0xe9967a) }, + { "darkseagreen", IC_RGB(0x8fbc8f) }, + { "darkslateblue", IC_RGB(0x483d8b) }, + { "darkslategray", IC_RGB(0x2f4f4f) }, + { "darkslategrey", IC_RGB(0x2f4f4f) }, + { "darkturquoise", IC_RGB(0x00ced1) }, + { "darkviolet", IC_RGB(0x9400d3) }, + { "deeppink", IC_RGB(0xff1493) }, + { "deepskyblue", IC_RGB(0x00bfff) }, + { "dimgray", IC_RGB(0x696969) }, + { "dimgrey", IC_RGB(0x696969) }, + { "dodgerblue", IC_RGB(0x1e90ff) }, + { "firebrick", IC_RGB(0xb22222) }, + { "floralwhite", IC_RGB(0xfffaf0) }, + { "forestgreen", IC_RGB(0x228b22) }, + { "fuchsia", IC_RGB(0xff00ff) }, + { "gainsboro", IC_RGB(0xdcdcdc) }, + { "ghostwhite", IC_RGB(0xf8f8ff) }, + { "gold", IC_RGB(0xffd700) }, + { "goldenrod", IC_RGB(0xdaa520) }, + { "gray", IC_RGB(0x808080) }, + { "green", IC_RGB(0x008000) }, + { "greenyellow", IC_RGB(0xadff2f) }, + { "grey", IC_RGB(0x808080) }, + { "honeydew", IC_RGB(0xf0fff0) }, + { "hotpink", IC_RGB(0xff69b4) }, + { "indianred", IC_RGB(0xcd5c5c) }, + { "indigo", IC_RGB(0x4b0082) }, + { "ivory", IC_RGB(0xfffff0) }, + { "khaki", IC_RGB(0xf0e68c) }, + { "lavender", IC_RGB(0xe6e6fa) }, + { "lavenderblush", IC_RGB(0xfff0f5) }, + { "lawngreen", IC_RGB(0x7cfc00) }, + { "lemonchiffon", IC_RGB(0xfffacd) }, + { "lightblue", IC_RGB(0xadd8e6) }, + { "lightcoral", IC_RGB(0xf08080) }, + { "lightcyan", IC_RGB(0xe0ffff) }, + { "lightgoldenrodyellow", IC_RGB(0xfafad2) }, + { "lightgray", IC_RGB(0xd3d3d3) }, + { "lightgreen", IC_RGB(0x90ee90) }, + { "lightgrey", IC_RGB(0xd3d3d3) }, + { "lightpink", IC_RGB(0xffb6c1) }, + { "lightsalmon", IC_RGB(0xffa07a) }, + { "lightseagreen", IC_RGB(0x20b2aa) }, + { "lightskyblue", IC_RGB(0x87cefa) }, + { "lightslategray", IC_RGB(0x778899) }, + { "lightslategrey", IC_RGB(0x778899) }, + { "lightsteelblue", IC_RGB(0xb0c4de) }, + { "lightyellow", IC_RGB(0xffffe0) }, + { "lime", IC_RGB(0x00ff00) }, + { "limegreen", IC_RGB(0x32cd32) }, + { "linen", IC_RGB(0xfaf0e6) }, + { "magenta", IC_RGB(0xff00ff) }, + { "maroon", IC_RGB(0x800000) }, + { "mediumaquamarine", IC_RGB(0x66cdaa) }, + { "mediumblue", IC_RGB(0x0000cd) }, + { "mediumorchid", IC_RGB(0xba55d3) }, + { "mediumpurple", IC_RGB(0x9370db) }, + { "mediumseagreen", IC_RGB(0x3cb371) }, + { "mediumslateblue", IC_RGB(0x7b68ee) }, + { "mediumspringgreen", IC_RGB(0x00fa9a) }, + { "mediumturquoise", IC_RGB(0x48d1cc) }, + { "mediumvioletred", IC_RGB(0xc71585) }, + { "midnightblue", IC_RGB(0x191970) }, + { "mintcream", IC_RGB(0xf5fffa) }, + { "mistyrose", IC_RGB(0xffe4e1) }, + { "moccasin", IC_RGB(0xffe4b5) }, + { "navajowhite", IC_RGB(0xffdead) }, + { "navy", IC_RGB(0x000080) }, + { "oldlace", IC_RGB(0xfdf5e6) }, + { "olive", IC_RGB(0x808000) }, + { "olivedrab", IC_RGB(0x6b8e23) }, + { "orange", IC_RGB(0xffa500) }, + { "orangered", IC_RGB(0xff4500) }, + { "orchid", IC_RGB(0xda70d6) }, + { "palegoldenrod", IC_RGB(0xeee8aa) }, + { "palegreen", IC_RGB(0x98fb98) }, + { "paleturquoise", IC_RGB(0xafeeee) }, + { "palevioletred", IC_RGB(0xdb7093) }, + { "papayawhip", IC_RGB(0xffefd5) }, + { "peachpuff", IC_RGB(0xffdab9) }, + { "peru", IC_RGB(0xcd853f) }, + { "pink", IC_RGB(0xffc0cb) }, + { "plum", IC_RGB(0xdda0dd) }, + { "powderblue", IC_RGB(0xb0e0e6) }, + { "purple", IC_RGB(0x800080) }, + { "rebeccapurple", IC_RGB(0x663399) }, + { "red", IC_RGB(0xff0000) }, + { "rosybrown", IC_RGB(0xbc8f8f) }, + { "royalblue", IC_RGB(0x4169e1) }, + { "saddlebrown", IC_RGB(0x8b4513) }, + { "salmon", IC_RGB(0xfa8072) }, + { "sandybrown", IC_RGB(0xf4a460) }, + { "seagreen", IC_RGB(0x2e8b57) }, + { "seashell", IC_RGB(0xfff5ee) }, + { "sienna", IC_RGB(0xa0522d) }, + { "silver", IC_RGB(0xc0c0c0) }, + { "skyblue", IC_RGB(0x87ceeb) }, + { "slateblue", IC_RGB(0x6a5acd) }, + { "slategray", IC_RGB(0x708090) }, + { "slategrey", IC_RGB(0x708090) }, + { "snow", IC_RGB(0xfffafa) }, + { "springgreen", IC_RGB(0x00ff7f) }, + { "steelblue", IC_RGB(0x4682b4) }, + { "tan", IC_RGB(0xd2b48c) }, + { "teal", IC_RGB(0x008080) }, + { "thistle", IC_RGB(0xd8bfd8) }, + { "tomato", IC_RGB(0xff6347) }, + { "turquoise", IC_RGB(0x40e0d0) }, + { "violet", IC_RGB(0xee82ee) }, + { "wheat", IC_RGB(0xf5deb3) }, + { "white", IC_RGB(0xffffff) }, + { "whitesmoke", IC_RGB(0xf5f5f5) }, + { "yellow", IC_RGB(0xffff00) }, + { "yellowgreen", IC_RGB(0x9acd32) }, + {NULL, 0} +}; diff --git a/extern/isocline/src/common.c b/extern/isocline/src/common.c new file mode 100644 index 00000000..1d9fb566 --- /dev/null +++ b/extern/isocline/src/common.c @@ -0,0 +1,347 @@ +/* ---------------------------------------------------------------------------- + Copyright (c) 2021, Daan Leijen + This is free software; you can redistribute it and/or modify it + under the terms of the MIT License. A copy of the license can be + found in the "LICENSE" file at the root of this distribution. +-----------------------------------------------------------------------------*/ + +#include +#include +#include +#include +#include "common.h" + + +//------------------------------------------------------------- +// String wrappers for ssize_t +//------------------------------------------------------------- + +ic_private ssize_t ic_strlen( const char* s ) { + if (s==NULL) return 0; + return to_ssize_t(strlen(s)); +} + +ic_private void ic_memmove( void* dest, const void* src, ssize_t n ) { + assert(dest!=NULL && src != NULL); + if (n <= 0) return; + memmove(dest,src,to_size_t(n)); +} + + +ic_private void ic_memcpy( void* dest, const void* src, ssize_t n ) { + assert(dest!=NULL && src != NULL); + if (dest == NULL || src == NULL || n <= 0) return; + memcpy(dest,src,to_size_t(n)); +} + +ic_private void ic_memset(void* dest, uint8_t value, ssize_t n) { + assert(dest!=NULL); + if (dest == NULL || n <= 0) return; + memset(dest,(int8_t)value,to_size_t(n)); +} + +ic_private bool ic_memnmove( void* dest, ssize_t dest_size, const void* src, ssize_t n ) { + assert(dest!=NULL && src != NULL); + if (n <= 0) return true; + if (dest_size < n) { assert(false); return false; } + memmove(dest,src,to_size_t(n)); + return true; +} + +ic_private bool ic_strcpy( char* dest, ssize_t dest_size /* including 0 */, const char* src) { + assert(dest!=NULL && src != NULL); + if (dest == NULL || dest_size <= 0) return false; + ssize_t slen = ic_strlen(src); + if (slen >= dest_size) return false; + strcpy(dest,src); + assert(dest[slen] == 0); + return true; +} + + +ic_private bool ic_strncpy( char* dest, ssize_t dest_size /* including 0 */, const char* src, ssize_t n) { + assert(dest!=NULL && n < dest_size); + if (dest == NULL || dest_size <= 0) return false; + if (n >= dest_size) return false; + if (src==NULL || n <= 0) { + dest[0] = 0; + } + else { + strncpy(dest,src,to_size_t(n)); + dest[n] = 0; + } + return true; +} + +//------------------------------------------------------------- +// String matching +//------------------------------------------------------------- + +ic_public bool ic_starts_with( const char* s, const char* prefix ) { + if (s==prefix) return true; + if (prefix==NULL) return true; + if (s==NULL) return false; + + ssize_t i; + for( i = 0; s[i] != 0 && prefix[i] != 0; i++) { + if (s[i] != prefix[i]) return false; + } + return (prefix[i] == 0); +} + +ic_private char ic_tolower( char c ) { + return (c >= 'A' && c <= 'Z' ? c - 'A' + 'a' : c); +} + +ic_private void ic_str_tolower(char* s) { + while(*s != 0) { + *s = ic_tolower(*s); + s++; + } +} + +ic_public bool ic_istarts_with( const char* s, const char* prefix ) { + if (s==prefix) return true; + if (prefix==NULL) return true; + if (s==NULL) return false; + + ssize_t i; + for( i = 0; s[i] != 0 && prefix[i] != 0; i++) { + if (ic_tolower(s[i]) != ic_tolower(prefix[i])) return false; + } + return (prefix[i] == 0); +} + + +ic_private int ic_strnicmp(const char* s1, const char* s2, ssize_t n) { + if (s1 == NULL && s2 == NULL) return 0; + if (s1 == NULL) return -1; + if (s2 == NULL) return 1; + ssize_t i; + for (i = 0; s1[i] != 0 && i < n; i++) { // note: if s2[i] == 0 the loop will stop as c1 != c2 + char c1 = ic_tolower(s1[i]); + char c2 = ic_tolower(s2[i]); + if (c1 < c2) return -1; + if (c1 > c2) return 1; + } + return ((i >= n || s2[i] == 0) ? 0 : -1); +} + +ic_private int ic_stricmp(const char* s1, const char* s2) { + ssize_t len1 = ic_strlen(s1); + ssize_t len2 = ic_strlen(s2); + if (len1 < len2) return -1; + if (len1 > len2) return 1; + return (ic_strnicmp(s1, s2, (len1 >= len2 ? len1 : len2))); +} + + +static const char* ic_stristr(const char* s, const char* pat) { + if (s==NULL) return NULL; + if (pat==NULL || pat[0] == 0) return s; + ssize_t patlen = ic_strlen(pat); + for (ssize_t i = 0; s[i] != 0; i++) { + if (ic_strnicmp(s + i, pat, patlen) == 0) return (s+i); + } + return NULL; +} + +ic_private bool ic_contains(const char* big, const char* s) { + if (big == NULL) return false; + if (s == NULL) return true; + return (strstr(big,s) != NULL); +} + +ic_private bool ic_icontains(const char* big, const char* s) { + if (big == NULL) return false; + if (s == NULL) return true; + return (ic_stristr(big,s) != NULL); +} + + +//------------------------------------------------------------- +// Unicode +// QUTF-8: See +// Raw bytes are code points 0xEE000 - 0xEE0FF +//------------------------------------------------------------- +#define IC_UNICODE_RAW ((unicode_t)(0xEE000U)) + +ic_private unicode_t unicode_from_raw(uint8_t c) { + return (IC_UNICODE_RAW + c); +} + +ic_private bool unicode_is_raw(unicode_t u, uint8_t* c) { + if (u >= IC_UNICODE_RAW && u <= IC_UNICODE_RAW + 0xFF) { + *c = (uint8_t)(u - IC_UNICODE_RAW); + return true; + } + else { + return false; + } +} + +ic_private void unicode_to_qutf8(unicode_t u, uint8_t buf[5]) { + memset(buf, 0, 5); + if (u <= 0x7F) { + buf[0] = (uint8_t)u; + } + else if (u <= 0x07FF) { + buf[0] = (0xC0 | ((uint8_t)(u >> 6))); + buf[1] = (0x80 | (((uint8_t)u) & 0x3F)); + } + else if (u <= 0xFFFF) { + buf[0] = (0xE0 | ((uint8_t)(u >> 12))); + buf[1] = (0x80 | (((uint8_t)(u >> 6)) & 0x3F)); + buf[2] = (0x80 | (((uint8_t)u) & 0x3F)); + } + else if (u <= 0x10FFFF) { + if (unicode_is_raw(u, &buf[0])) { + buf[1] = 0; + } + else { + buf[0] = (0xF0 | ((uint8_t)(u >> 18))); + buf[1] = (0x80 | (((uint8_t)(u >> 12)) & 0x3F)); + buf[2] = (0x80 | (((uint8_t)(u >> 6)) & 0x3F)); + buf[3] = (0x80 | (((uint8_t)u) & 0x3F)); + } + } +} + +// is this a utf8 continuation byte? +ic_private bool utf8_is_cont(uint8_t c) { + return ((c & 0xC0) == 0x80); +} + +ic_private unicode_t unicode_from_qutf8(const uint8_t* s, ssize_t len, ssize_t* count) { + unicode_t c0 = 0; + if (len <= 0 || s == NULL) { + goto fail; + } + // 1 byte + c0 = s[0]; + if (c0 <= 0x7F && len >= 1) { + if (count != NULL) *count = 1; + return c0; + } + else if (c0 <= 0xC1) { // invalid continuation byte or invalid 0xC0, 0xC1 + goto fail; + } + // 2 bytes + else if (c0 <= 0xDF && len >= 2 && utf8_is_cont(s[1])) { + if (count != NULL) *count = 2; + return (((c0 & 0x1F) << 6) | (s[1] & 0x3F)); + } + // 3 bytes: reject overlong and surrogate halves + else if (len >= 3 && + ((c0 == 0xE0 && s[1] >= 0xA0 && s[1] <= 0xBF && utf8_is_cont(s[2])) || + (c0 >= 0xE1 && c0 <= 0xEC && utf8_is_cont(s[1]) && utf8_is_cont(s[2])) + )) + { + if (count != NULL) *count = 3; + return (((c0 & 0x0F) << 12) | ((unicode_t)(s[1] & 0x3F) << 6) | (s[2] & 0x3F)); + } + // 4 bytes: reject overlong + else if (len >= 4 && + (((c0 == 0xF0 && s[1] >= 0x90 && s[1] <= 0xBF && utf8_is_cont(s[2]) && utf8_is_cont(s[3])) || + (c0 >= 0xF1 && c0 <= 0xF3 && utf8_is_cont(s[1]) && utf8_is_cont(s[2]) && utf8_is_cont(s[3])) || + (c0 == 0xF4 && s[1] >= 0x80 && s[1] <= 0x8F && utf8_is_cont(s[2]) && utf8_is_cont(s[3]))) + )) + { + if (count != NULL) *count = 4; + return (((c0 & 0x07) << 18) | ((unicode_t)(s[1] & 0x3F) << 12) | ((unicode_t)(s[2] & 0x3F) << 6) | (s[3] & 0x3F)); + } +fail: + if (count != NULL) *count = 1; + return unicode_from_raw(s[0]); +} + + +//------------------------------------------------------------- +// Debug +//------------------------------------------------------------- + +#if defined(IC_NO_DEBUG_MSG) +// nothing +#elif !defined(IC_DEBUG_TO_FILE) +ic_private void debug_msg(const char* fmt, ...) { + if (getenv("ISOCLINE_DEBUG")) { + va_list args; + va_start(args, fmt); + vfprintf(stderr, fmt, args); + va_end(args); + } +} +#else +ic_private void debug_msg(const char* fmt, ...) { + static int debug_init; + static const char* debug_fname = "isocline.debug.txt"; + // initialize? + if (debug_init==0) { + debug_init = -1; + const char* rdebug = getenv("ISOCLINE_DEBUG"); + if (rdebug!=NULL && strcmp(rdebug,"1") == 0) { + FILE* fdbg = fopen(debug_fname, "w"); + if (fdbg!=NULL) { + debug_init = 1; + fclose(fdbg); + } + } + } + if (debug_init <= 0) return; + + // write debug messages + FILE* fdbg = fopen(debug_fname, "a"); + if (fdbg==NULL) return; + va_list args; + va_start(args, fmt); + vfprintf(fdbg, fmt, args); + fclose(fdbg); + va_end(args); +} +#endif + + +//------------------------------------------------------------- +// Allocation +//------------------------------------------------------------- + +ic_private void* mem_malloc(alloc_t* mem, ssize_t sz) { + return mem->malloc(to_size_t(sz)); +} + +ic_private void* mem_zalloc(alloc_t* mem, ssize_t sz) { + void* p = mem_malloc(mem, sz); + if (p != NULL) memset(p, 0, to_size_t(sz)); + return p; +} + +ic_private void* mem_realloc(alloc_t* mem, void* p, ssize_t newsz) { + return mem->realloc(p, to_size_t(newsz)); +} + +ic_private void mem_free(alloc_t* mem, const void* p) { + mem->free((void*)p); +} + +ic_private char* mem_strdup(alloc_t* mem, const char* s) { + if (s==NULL) return NULL; + ssize_t n = ic_strlen(s); + char* p = mem_malloc_tp_n(mem, char, n+1); + if (p == NULL) return NULL; + ic_memcpy(p, s, n+1); + return p; +} + +ic_private char* mem_strndup(alloc_t* mem, const char* s, ssize_t n) { + if (s==NULL || n < 0) return NULL; + char* p = mem_malloc_tp_n(mem, char, n+1); + if (p == NULL) return NULL; + ssize_t i; + for (i = 0; i < n && s[i] != 0; i++) { + p[i] = s[i]; + } + assert(i <= n); + p[i] = 0; + return p; +} + diff --git a/extern/isocline/src/common.h b/extern/isocline/src/common.h new file mode 100644 index 00000000..dd5b2569 --- /dev/null +++ b/extern/isocline/src/common.h @@ -0,0 +1,187 @@ +/* ---------------------------------------------------------------------------- + Copyright (c) 2021, Daan Leijen + This is free software; you can redistribute it and/or modify it + under the terms of the MIT License. A copy of the license can be + found in the "LICENSE" file at the root of this distribution. +-----------------------------------------------------------------------------*/ + +#pragma once +#ifndef IC_COMMON_H +#define IC_COMMON_H + +//------------------------------------------------------------- +// Headers and defines +//------------------------------------------------------------- + +#include // ssize_t +#include +#include +#include +#include +#include +#include "../include/isocline.h" // ic_malloc_fun_t, ic_color_t etc. + +# ifdef __cplusplus +# define ic_extern_c extern "C" +# else +# define ic_extern_c +# endif + +#if defined(IC_SEPARATE_OBJS) +# define ic_public ic_extern_c +# if defined(__GNUC__) // includes clang and icc +# define ic_private __attribute__((visibility("hidden"))) +# else +# define ic_private +# endif +#else +# define ic_private static +# define ic_public ic_extern_c +#endif + +#define ic_unused(x) (void)(x) + + +//------------------------------------------------------------- +// ssize_t +//------------------------------------------------------------- + +#if defined(_MSC_VER) +typedef intptr_t ssize_t; +#endif + +#define ssizeof(tp) (ssize_t)(sizeof(tp)) +static inline size_t to_size_t(ssize_t sz) { return (sz >= 0 ? (size_t)sz : 0); } +static inline ssize_t to_ssize_t(size_t sz) { return (sz <= SIZE_MAX/2 ? (ssize_t)sz : 0); } + +ic_private void ic_memmove(void* dest, const void* src, ssize_t n); +ic_private void ic_memcpy(void* dest, const void* src, ssize_t n); +ic_private void ic_memset(void* dest, uint8_t value, ssize_t n); +ic_private bool ic_memnmove(void* dest, ssize_t dest_size, const void* src, ssize_t n); + +ic_private ssize_t ic_strlen(const char* s); +ic_private bool ic_strcpy(char* dest, ssize_t dest_size /* including 0 */, const char* src); +ic_private bool ic_strncpy(char* dest, ssize_t dest_size /* including 0 */, const char* src, ssize_t n); + +ic_private bool ic_contains(const char* big, const char* s); +ic_private bool ic_icontains(const char* big, const char* s); +ic_private char ic_tolower(char c); +ic_private void ic_str_tolower(char* s); +ic_private int ic_stricmp(const char* s1, const char* s2); +ic_private int ic_strnicmp(const char* s1, const char* s2, ssize_t n); + + + +//--------------------------------------------------------------------- +// Unicode +// +// We use "qutf-8" (quite like utf-8) encoding and decoding. +// Internally we always use valid utf-8. If we encounter invalid +// utf-8 bytes (or bytes >= 0x80 from any other encoding) we encode +// these as special code points in the "raw plane" (0xEE000 - 0xEE0FF). +// When decoding we are then able to restore such raw bytes as-is. +// See +//--------------------------------------------------------------------- + +typedef uint32_t unicode_t; + +ic_private void unicode_to_qutf8(unicode_t u, uint8_t buf[5]); +ic_private unicode_t unicode_from_qutf8(const uint8_t* s, ssize_t len, ssize_t* nread); // validating + +ic_private unicode_t unicode_from_raw(uint8_t c); +ic_private bool unicode_is_raw(unicode_t u, uint8_t* c); + +ic_private bool utf8_is_cont(uint8_t c); + + +//------------------------------------------------------------- +// Colors +//------------------------------------------------------------- + +// A color is either RGB or an ANSI code. +// (RGB colors have bit 24 set to distinguish them from the ANSI color palette colors.) +// (Isocline will automatically convert from RGB on terminals that do not support full colors) +typedef uint32_t ic_color_t; + +// Create a color from a 24-bit color value. +ic_private ic_color_t ic_rgb(uint32_t hex); + +// Create a color from a 8-bit red/green/blue components. +// The value of each component is capped between 0 and 255. +ic_private ic_color_t ic_rgbx(ssize_t r, ssize_t g, ssize_t b); + +#define IC_COLOR_NONE (0) +#define IC_RGB(rgb) (0x1000000 | (uint32_t)(rgb)) // ic_rgb(rgb) // define to it can be used as a constant + +// ANSI colors. +// The actual colors used is usually determined by the terminal theme +// See +#define IC_ANSI_BLACK (30) +#define IC_ANSI_MAROON (31) +#define IC_ANSI_GREEN (32) +#define IC_ANSI_OLIVE (33) +#define IC_ANSI_NAVY (34) +#define IC_ANSI_PURPLE (35) +#define IC_ANSI_TEAL (36) +#define IC_ANSI_SILVER (37) +#define IC_ANSI_DEFAULT (39) + +#define IC_ANSI_GRAY (90) +#define IC_ANSI_RED (91) +#define IC_ANSI_LIME (92) +#define IC_ANSI_YELLOW (93) +#define IC_ANSI_BLUE (94) +#define IC_ANSI_FUCHSIA (95) +#define IC_ANSI_AQUA (96) +#define IC_ANSI_WHITE (97) + +#define IC_ANSI_DARKGRAY IC_ANSI_GRAY +#define IC_ANSI_LIGHTGRAY IC_ANSI_SILVER +#define IC_ANSI_MAGENTA IC_ANSI_FUCHSIA +#define IC_ANSI_CYAN IC_ANSI_AQUA + + + +//------------------------------------------------------------- +// Debug +//------------------------------------------------------------- + +#if defined(IC_NO_DEBUG_MSG) +#define debug_msg(fmt,...) (void)(0) +#else +ic_private void debug_msg( const char* fmt, ... ); +#endif + + +//------------------------------------------------------------- +// Abstract environment +//------------------------------------------------------------- +struct ic_env_s; +typedef struct ic_env_s ic_env_t; + + +//------------------------------------------------------------- +// Allocation +//------------------------------------------------------------- + +typedef struct alloc_s { + ic_malloc_fun_t* malloc; + ic_realloc_fun_t* realloc; + ic_free_fun_t* free; +} alloc_t; + + +ic_private void* mem_malloc( alloc_t* mem, ssize_t sz ); +ic_private void* mem_zalloc( alloc_t* mem, ssize_t sz ); +ic_private void* mem_realloc( alloc_t* mem, void* p, ssize_t newsz ); +ic_private void mem_free( alloc_t* mem, const void* p ); +ic_private char* mem_strdup( alloc_t* mem, const char* s); +ic_private char* mem_strndup( alloc_t* mem, const char* s, ssize_t n); + +#define mem_zalloc_tp(mem,tp) (tp*)mem_zalloc(mem,ssizeof(tp)) +#define mem_malloc_tp_n(mem,tp,n) (tp*)mem_malloc(mem,(n)*ssizeof(tp)) +#define mem_zalloc_tp_n(mem,tp,n) (tp*)mem_zalloc(mem,(n)*ssizeof(tp)) +#define mem_realloc_tp(mem,tp,p,n) (tp*)mem_realloc(mem,p,(n)*ssizeof(tp)) + + +#endif // IC_COMMON_H diff --git a/extern/isocline/src/completers.c b/extern/isocline/src/completers.c new file mode 100644 index 00000000..e9701c16 --- /dev/null +++ b/extern/isocline/src/completers.c @@ -0,0 +1,675 @@ +/* ---------------------------------------------------------------------------- + Copyright (c) 2021, Daan Leijen + This is free software; you can redistribute it and/or modify it + under the terms of the MIT License. A copy of the license can be + found in the "LICENSE" file at the root of this distribution. +-----------------------------------------------------------------------------*/ +#include +#include + +#include "../include/isocline.h" +#include "common.h" +#include "env.h" +#include "stringbuf.h" +#include "completions.h" + + + +//------------------------------------------------------------- +// Word completion +//------------------------------------------------------------- + +// free variables for word completion +typedef struct word_closure_s { + long delete_before_adjust; + void* prev_env; + ic_completion_fun_t* prev_complete; +} word_closure_t; + + +// word completion callback +static bool token_add_completion_ex(ic_env_t* env, void* closure, const char* replacement, const char* display, const char* help, long delete_before, long delete_after) { + word_closure_t* wenv = (word_closure_t*)(closure); + // call the previous completer with an adjusted delete-before + return (*wenv->prev_complete)(env, wenv->prev_env, replacement, display, help, wenv->delete_before_adjust + delete_before, delete_after); +} + + +ic_public void ic_complete_word(ic_completion_env_t* cenv, const char* prefix, ic_completer_fun_t* fun, + ic_is_char_class_fun_t* is_word_char) +{ + if (is_word_char == NULL) is_word_char = &ic_char_is_nonseparator; + + ssize_t len = ic_strlen(prefix); + ssize_t pos = len; // will be start of the 'word' (excluding a potential start quote) + while (pos > 0) { + // go back one code point + ssize_t ofs = str_prev_ofs(prefix, pos, NULL); + if (ofs <= 0) break; + if (!(*is_word_char)(prefix + (pos - ofs), (long)ofs)) { + break; + } + pos -= ofs; + } + if (pos < 0) { pos = 0; } + + // stop if empty word + // if (len == pos) return; + + // set up the closure + word_closure_t wenv; + wenv.delete_before_adjust = (long)(len - pos); + wenv.prev_complete = cenv->complete; + wenv.prev_env = cenv->env; + cenv->complete = &token_add_completion_ex; + cenv->closure = &wenv; + + // and call the user completion routine + (*fun)(cenv, prefix + pos); + + // restore the original environment + cenv->complete = wenv.prev_complete; + cenv->closure = wenv.prev_env; +} + + +//------------------------------------------------------------- +// Quoted word completion (with escape characters) +//------------------------------------------------------------- + +// free variables for word completion +typedef struct qword_closure_s { + char escape_char; + char quote; + long delete_before_adjust; + stringbuf_t* sbuf; + void* prev_env; + ic_is_char_class_fun_t* is_word_char; + ic_completion_fun_t* prev_complete; +} qword_closure_t; + + +// word completion callback +static bool qword_add_completion_ex(ic_env_t* env, void* closure, const char* replacement, const char* display, const char* help, + long delete_before, long delete_after) { + qword_closure_t* wenv = (qword_closure_t*)(closure); + sbuf_replace( wenv->sbuf, replacement ); + if (wenv->quote != 0) { + // add end quote + sbuf_append_char( wenv->sbuf, wenv->quote); + } + else { + // escape non-word characters if it was not quoted + ssize_t pos = 0; + ssize_t next; + while ( (next = sbuf_next_ofs(wenv->sbuf, pos, NULL)) > 0 ) + { + if (!(*wenv->is_word_char)(sbuf_string(wenv->sbuf) + pos, (long)next)) { // strchr(wenv->non_word_char, sbuf_char_at( wenv->sbuf, pos )) != NULL) { + sbuf_insert_char_at( wenv->sbuf, wenv->escape_char, pos); + pos++; + } + pos += next; + } + } + // and call the previous completion function + return (*wenv->prev_complete)( env, wenv->prev_env, sbuf_string(wenv->sbuf), display, help, wenv->delete_before_adjust + delete_before, delete_after ); +} + + +ic_public void ic_complete_qword( ic_completion_env_t* cenv, const char* prefix, ic_completer_fun_t* fun, ic_is_char_class_fun_t* is_word_char ) { + ic_complete_qword_ex( cenv, prefix, fun, is_word_char, '\\', NULL); +} + + +ic_public void ic_complete_qword_ex( ic_completion_env_t* cenv, const char* prefix, ic_completer_fun_t* fun, + ic_is_char_class_fun_t* is_word_char, char escape_char, const char* quote_chars ) { + if (is_word_char == NULL) is_word_char = &ic_char_is_nonseparator ; + if (quote_chars == NULL) quote_chars = "'\""; + + ssize_t len = ic_strlen(prefix); + ssize_t pos; // will be start of the 'word' (excluding a potential start quote) + char quote = 0; + ssize_t quote_len = 0; + + // 1. look for a starting quote + if (quote_chars[0] != 0) { + // we go forward and count all quotes; if it is uneven, we need to complete quoted. + ssize_t qpos_open = -1; + ssize_t qpos_close = -1; + ssize_t qcount = 0; + pos = 0; + while(pos < len) { + if (prefix[pos] == escape_char && prefix[pos+1] != 0 && + !(*is_word_char)(prefix + pos + 1, 1)) // strchr(non_word_char, prefix[pos+1]) != NULL + { + pos++; // skip escape and next char + } + else if (qcount % 2 == 0 && strchr(quote_chars, prefix[pos]) != NULL) { + // open quote + qpos_open = pos; + quote = prefix[pos]; + qcount++; + } + else if (qcount % 2 == 1 && prefix[pos] == quote) { + // close quote + qpos_close = pos; + qcount++; + } + else if (!(*is_word_char)(prefix + pos, 1)) { // strchr(non_word_char, prefix[pos]) != NULL) { + qpos_close = -1; + } + ssize_t ofs = str_next_ofs( prefix, len, pos, NULL ); + if (ofs <= 0) break; + pos += ofs; + } + if ((qcount % 2 == 0 && qpos_close >= 0) || // if the last quote is only followed by word chars, we still complete it + (qcount % 2 == 1)) // opening quote found + { + quote_len = (len - qpos_open - 1); + pos = qpos_open + 1; // pos points to the word start just after the quote. + } + else { + quote = 0; + } + } + + // 2. if we did not find a quoted word, look for non-word-chars + if (quote == 0) { + pos = len; + while(pos > 0) { + // go back one code point + ssize_t ofs = str_prev_ofs(prefix, pos, NULL ); + if (ofs <= 0) break; + if (!(*is_word_char)(prefix + (pos - ofs), (long)ofs)) { // strchr(non_word_char, prefix[pos - ofs]) != NULL) { + // non word char, break if it is not escaped + if (pos <= ofs || prefix[pos - ofs - 1] != escape_char) break; + // otherwise go on + pos--; // skip escaped char + } + pos -= ofs; + } + } + + // stop if empty word + // if (len == pos) return; + + // allocate new unescaped word prefix + char* word = mem_strndup( cenv->env->mem, prefix + pos, (quote==0 ? len - pos : quote_len)); + if (word == NULL) return; + + if (quote == 0) { + // unescape prefix + ssize_t wlen = len - pos; + ssize_t wpos = 0; + while (wpos < wlen) { + ssize_t ofs = str_next_ofs(word, wlen, wpos, NULL); + if (ofs <= 0) break; + if (word[wpos] == escape_char && word[wpos+1] != 0 && + !(*is_word_char)(word + wpos + 1, (long)ofs)) // strchr(non_word_char, word[wpos+1]) != NULL) { + { + ic_memmove(word + wpos, word + wpos + 1, wlen - wpos /* including 0 */); + } + wpos += ofs; + } + } + #ifdef _WIN32 + else { + // remove inner quote: "c:\Program Files\"Win + ssize_t wlen = len - pos; + ssize_t wpos = 0; + while (wpos < wlen) { + ssize_t ofs = str_next_ofs(word, wlen, wpos, NULL); + if (ofs <= 0) break; + if (word[wpos] == escape_char && word[wpos+1] == quote) { + word[wpos+1] = escape_char; + ic_memmove(word + wpos, word + wpos + 1, wlen - wpos /* including 0 */); + } + wpos += ofs; + } + } + #endif + + // set up the closure + qword_closure_t wenv; + wenv.quote = quote; + wenv.is_word_char = is_word_char; + wenv.escape_char = escape_char; + wenv.delete_before_adjust = (long)(len - pos); + wenv.prev_complete = cenv->complete; + wenv.prev_env = cenv->env; + wenv.sbuf = sbuf_new(cenv->env->mem); + if (wenv.sbuf == NULL) { mem_free(cenv->env->mem, word); return; } + cenv->complete = &qword_add_completion_ex; + cenv->closure = &wenv; + + // and call the user completion routine + (*fun)( cenv, word ); + + // restore the original environment + cenv->complete = wenv.prev_complete; + cenv->closure = wenv.prev_env; + + sbuf_free(wenv.sbuf); + mem_free(cenv->env->mem, word); +} + + + + +//------------------------------------------------------------- +// Complete file names +// Listing files +//------------------------------------------------------------- +#include + +typedef enum file_type_e { + // must follow BSD style LSCOLORS order + FT_DEFAULT = 0, + FT_DIR, + FT_SYM, + FT_SOCK, + FT_PIPE, + FT_BLOCK, + FT_CHAR, + FT_SETUID, + FT_SETGID, + FT_DIR_OW_STICKY, + FT_DIR_OW, + FT_DIR_STICKY, + FT_EXE, + FT_LAST +} file_type_t; + +static int cli_color; // 1 enabled, 0 not initialized, -1 disabled +static const char* lscolors = "exfxcxdxbxegedabagacad"; // default BSD setting +static const char* ls_colors; +static const char* ls_colors_names[] = { "no=","di=","ln=","so=","pi=","bd=","cd=","su=","sg=","tw=","ow=","st=","ex=", NULL }; + +static bool ls_colors_init(void) { + if (cli_color != 0) return (cli_color >= 1); + // colors enabled? + const char* s = getenv("CLICOLOR"); + if (s==NULL || (strcmp(s, "1")!=0 && strcmp(s, "") != 0)) { + cli_color = -1; + return false; + } + cli_color = 1; + s = getenv("LS_COLORS"); + if (s != NULL) { ls_colors = s; } + s = getenv("LSCOLORS"); + if (s != NULL) { lscolors = s; } + return true; +} + +static bool ls_valid_esc(ssize_t c) { + return ((c==0 || c==1 || c==4 || c==7 || c==22 || c==24 || c==27) || + (c >= 30 && c <= 37) || (c >= 40 && c <= 47) || + (c >= 90 && c <= 97) || (c >= 100 && c <= 107)); +} + +static bool ls_colors_from_key(stringbuf_t* sb, const char* key) { + // find key + ssize_t keylen = ic_strlen(key); + if (keylen <= 0) return false; + const char* p = strstr(ls_colors, key); + if (p == NULL) return false; + p += keylen; + if (key[keylen-1] != '=') { + if (*p != '=') return false; + p++; + } + ssize_t len = 0; + while (p[len] != 0 && p[len] != ':') { + len++; + } + if (len <= 0) return false; + sbuf_append(sb, "[ansi-sgr=\"" ); + sbuf_append_n(sb, p, len ); + sbuf_append(sb, "\"]"); + return true; +} + +static int ls_colors_from_char(char c) { + if (c >= 'a' && c <= 'h') { return (c - 'a'); } + else if (c >= 'A' && c <= 'H') { return (c - 'A') + 8; } + else if (c == 'x') { return 256; } + else return 256; // default +} + +static bool ls_colors_append(stringbuf_t* sb, file_type_t ft, const char* ext) { + if (!ls_colors_init()) return false; + if (ls_colors != NULL) { + // GNU style + if (ft == FT_DEFAULT && ext != NULL) { + // first try extension match + if (ls_colors_from_key(sb, ext)) return true; + } + if (ft >= FT_DEFAULT && ft < FT_LAST) { + // then a filetype match + const char* key = ls_colors_names[ft]; + if (ls_colors_from_key(sb, key)) return true; + } + } + else if (lscolors != NULL) { + // BSD style + char fg = 'x'; + char bg = 'x'; + if (ic_strlen(lscolors) > (2*(ssize_t)ft)+1) { + fg = lscolors[2*ft]; + bg = lscolors[2*ft + 1]; + } + sbuf_appendf(sb, "[ansi-color=%d ansi-bgcolor=%d]", ls_colors_from_char(fg), ls_colors_from_char(bg) ); + return true; + } + return false; +} + +static void ls_colorize(bool no_lscolor, stringbuf_t* sb, file_type_t ft, const char* name, const char* ext, char dirsep) { + bool close = (no_lscolor ? false : ls_colors_append( sb, ft, ext)); + sbuf_append(sb, "[!pre]" ); + sbuf_append(sb, name); + if (dirsep != 0) sbuf_append_char(sb, dirsep); + sbuf_append(sb,"[/pre]" ); + if (close) { sbuf_append(sb, "[/]"); } +} + +#if defined(_WIN32) +#include +#include + +static bool os_is_dir(const char* cpath) { + struct _stat64 st = { 0 }; + _stat64(cpath, &st); + return ((st.st_mode & _S_IFDIR) != 0); +} + +static file_type_t os_get_filetype(const char* cpath) { + struct _stat64 st = { 0 }; + _stat64(cpath, &st); + if (((st.st_mode) & _S_IFDIR) != 0) return FT_DIR; + if (((st.st_mode) & _S_IFCHR) != 0) return FT_CHAR; + if (((st.st_mode) & _S_IFIFO) != 0) return FT_PIPE; + if (((st.st_mode) & _S_IEXEC) != 0) return FT_EXE; + return FT_DEFAULT; +} + + +#define dir_cursor intptr_t +#define dir_entry struct __finddata64_t + +static bool os_findfirst(alloc_t* mem, const char* path, dir_cursor* d, dir_entry* entry) { + stringbuf_t* spath = sbuf_new(mem); + if (spath == NULL) return false; + sbuf_append(spath, path); + sbuf_append(spath, "\\*"); + *d = _findfirsti64(sbuf_string(spath), entry); + mem_free(mem,spath); + return (*d != -1); +} + +static bool os_findnext(dir_cursor d, dir_entry* entry) { + return (_findnexti64(d, entry) == 0); +} + +static void os_findclose(dir_cursor d) { + _findclose(d); +} + +static const char* os_direntry_name(dir_entry* entry) { + return entry->name; +} + +static bool os_path_is_absolute( const char* path ) { + if (path != NULL && path[0] != 0 && path[1] == ':' && (path[2] == '\\' || path[2] == '/' || path[2] == 0)) { + char drive = path[0]; + return ((drive >= 'A' && drive <= 'Z') || (drive >= 'a' && drive <= 'z')); + } + else return false; +} + +ic_private char ic_dirsep(void) { + return '\\'; +} +#else + +#include +#include +#include +#include + +static bool os_is_dir(const char* cpath) { + struct stat st; + memset(&st, 0, sizeof(st)); + stat(cpath, &st); + return (S_ISDIR(st.st_mode)); +} + +static file_type_t os_get_filetype(const char* cpath) { + struct stat st; + memset(&st, 0, sizeof(st)); + lstat(cpath, &st); + switch ((st.st_mode)&S_IFMT) { + case S_IFSOCK: return FT_SOCK; + case S_IFLNK: { + return FT_SYM; + } + case S_IFIFO: return FT_PIPE; + case S_IFCHR: return FT_CHAR; + case S_IFBLK: return FT_BLOCK; + case S_IFDIR: { + if ((st.st_mode & S_ISUID) != 0) return FT_SETUID; + if ((st.st_mode & S_ISGID) != 0) return FT_SETGID; + if ((st.st_mode & S_IWGRP) != 0 && (st.st_mode & S_ISVTX) != 0) return FT_DIR_OW_STICKY; + if ((st.st_mode & S_IWGRP)) return FT_DIR_OW; + if ((st.st_mode & S_ISVTX)) return FT_DIR_STICKY; + return FT_DIR; + } + case S_IFREG: + default: { + if ((st.st_mode & S_IXUSR) != 0) return FT_EXE; + return FT_DEFAULT; + } + } +} + + +#define dir_cursor DIR* +#define dir_entry struct dirent* + +static bool os_findnext(dir_cursor d, dir_entry* entry) { + *entry = readdir(d); + return (*entry != NULL); +} + +static bool os_findfirst(alloc_t* mem, const char* cpath, dir_cursor* d, dir_entry* entry) { + ic_unused(mem); + *d = opendir(cpath); + if (*d == NULL) { + return false; + } + else { + return os_findnext(*d, entry); + } +} + +static void os_findclose(dir_cursor d) { + closedir(d); +} + +static const char* os_direntry_name(dir_entry* entry) { + return (*entry)->d_name; +} + +static bool os_path_is_absolute( const char* path ) { + return (path != NULL && path[0] == '/'); +} + +ic_private char ic_dirsep(void) { + return '/'; +} +#endif + + + +//------------------------------------------------------------- +// File completion +//------------------------------------------------------------- + +static bool ends_with_n(const char* name, ssize_t name_len, const char* ending, ssize_t len) { + if (name_len < len) return false; + if (ending == NULL || len <= 0) return true; + for (ssize_t i = 1; i <= len; i++) { + char c1 = name[name_len - i]; + char c2 = ending[len - i]; + #ifdef _WIN32 + if (ic_tolower(c1) != ic_tolower(c2)) return false; + #else + if (c1 != c2) return false; + #endif + } + return true; +} + +static bool match_extension(const char* name, const char* extensions) { + if (extensions == NULL || extensions[0] == 0) return true; + if (name == NULL) return false; + ssize_t name_len = ic_strlen(name); + ssize_t len = ic_strlen(extensions); + ssize_t cur = 0; + //debug_msg("match extensions: %s ~ %s", name, extensions); + for (ssize_t end = 0; end <= len; end++) { + if (extensions[end] == ';' || extensions[end] == 0) { + if (ends_with_n(name, name_len, extensions+cur, (end - cur))) { + return true; + } + cur = end+1; + } + } + return false; +} + +static bool filename_complete_indir( ic_completion_env_t* cenv, stringbuf_t* dir, + stringbuf_t* dir_prefix, stringbuf_t* display, + const char* base_prefix, + char dir_sep, const char* extensions ) +{ + dir_cursor d = 0; + dir_entry entry; + bool cont = true; + if (os_findfirst(cenv->env->mem, sbuf_string(dir), &d, &entry)) { + do { + const char* name = os_direntry_name(&entry); + if (name != NULL && strcmp(name, ".") != 0 && strcmp(name, "..") != 0 && + ic_istarts_with(name, base_prefix)) + { + // possible match, first check if it is a directory + file_type_t ft; + bool isdir; + const ssize_t plen = sbuf_len(dir_prefix); + sbuf_append(dir_prefix, name); + { // check directory and potentially add a dirsep to the dir_prefix + const ssize_t dlen = sbuf_len(dir); + sbuf_append_char(dir,ic_dirsep()); + sbuf_append(dir,name); + ft = os_get_filetype(sbuf_string(dir)); + isdir = os_is_dir(sbuf_string(dir)); + if (isdir && dir_sep != 0) { + sbuf_append_char(dir_prefix,dir_sep); + } + sbuf_delete_from(dir,dlen); // restore dir + } + if (isdir || match_extension(name, extensions)) { + // add completion + sbuf_clear(display); + ls_colorize(cenv->env->no_lscolors, display, ft, name, NULL, (isdir ? dir_sep : 0)); + cont = ic_add_completion_ex(cenv, sbuf_string(dir_prefix), sbuf_string(display), NULL); + } + sbuf_delete_from( dir_prefix, plen ); // restore dir_prefix + } + } while (cont && os_findnext(d, &entry)); + os_findclose(d); + } + return cont; +} + +typedef struct filename_closure_s { + const char* roots; + const char* extensions; + char dir_sep; +} filename_closure_t; + +static void filename_completer( ic_completion_env_t* cenv, const char* prefix ) { + if (prefix == NULL) return; + filename_closure_t* fclosure = (filename_closure_t*)cenv->arg; + stringbuf_t* root_dir = sbuf_new(cenv->env->mem); + stringbuf_t* dir_prefix = sbuf_new(cenv->env->mem); + stringbuf_t* display = sbuf_new(cenv->env->mem); + if (root_dir!=NULL && dir_prefix != NULL && display != NULL) + { + // split prefix in dir_prefix / base. + const char* base = strrchr(prefix,'/'); + #ifdef _WIN32 + const char* base2 = strrchr(prefix,'\\'); + if (base == NULL || base2 > base) base = base2; + #endif + if (base != NULL) { + base++; + sbuf_append_n(dir_prefix, prefix, base - prefix ); // includes dir separator + } + + // absolute path + if (os_path_is_absolute(prefix)) { + // do not use roots but try to complete directly + if (base != NULL) { + sbuf_append_n( root_dir, prefix, (base - prefix)); // include dir separator + } + filename_complete_indir( cenv, root_dir, dir_prefix, display, + (base != NULL ? base : prefix), + fclosure->dir_sep, fclosure->extensions ); + } + else { + // relative path, complete with respect to every root. + const char* next; + const char* root = fclosure->roots; + while ( root != NULL ) { + // create full root in `root_dir` + sbuf_clear(root_dir); + next = strchr(root,';'); + if (next == NULL) { + sbuf_append( root_dir, root ); + root = NULL; + } + else { + sbuf_append_n( root_dir, root, next - root ); + root = next + 1; + } + sbuf_append_char( root_dir, ic_dirsep()); + + // add the dir_prefix to the root + if (base != NULL) { + sbuf_append_n( root_dir, prefix, (base - prefix) - 1); + } + + // and complete in this directory + filename_complete_indir( cenv, root_dir, dir_prefix, display, + (base != NULL ? base : prefix), + fclosure->dir_sep, fclosure->extensions); + } + } + } + sbuf_free(display); + sbuf_free(root_dir); + sbuf_free(dir_prefix); +} + +ic_public void ic_complete_filename( ic_completion_env_t* cenv, const char* prefix, char dir_sep, const char* roots, const char* extensions ) { + if (roots == NULL) roots = "."; + if (extensions == NULL) extensions = ""; + if (dir_sep == 0) dir_sep = ic_dirsep(); + filename_closure_t fclosure; + fclosure.dir_sep = dir_sep; + fclosure.roots = roots; + fclosure.extensions = extensions; + cenv->arg = &fclosure; + ic_complete_qword_ex( cenv, prefix, &filename_completer, &ic_char_is_filename_letter, '\\', "'\""); +} diff --git a/extern/isocline/src/completions.c b/extern/isocline/src/completions.c new file mode 100644 index 00000000..01453efc --- /dev/null +++ b/extern/isocline/src/completions.c @@ -0,0 +1,326 @@ +/* ---------------------------------------------------------------------------- + Copyright (c) 2021, Daan Leijen + This is free software; you can redistribute it and/or modify it + under the terms of the MIT License. A copy of the license can be + found in the "LICENSE" file at the root of this distribution. +-----------------------------------------------------------------------------*/ +#include +#include +#include + +#include "../include/isocline.h" +#include "common.h" +#include "env.h" +#include "stringbuf.h" +#include "completions.h" + + +//------------------------------------------------------------- +// Completions +//------------------------------------------------------------- + +typedef struct completion_s { + const char* replacement; + const char* display; + const char* help; + ssize_t delete_before; + ssize_t delete_after; +} completion_t; + +struct completions_s { + ic_completer_fun_t* completer; + void* completer_arg; + ssize_t completer_max; + ssize_t count; + ssize_t len; + completion_t* elems; + alloc_t* mem; +}; + +static void default_filename_completer( ic_completion_env_t* cenv, const char* prefix ); + +ic_private completions_t* completions_new(alloc_t* mem) { + completions_t* cms = mem_zalloc_tp(mem, completions_t); + if (cms == NULL) return NULL; + cms->mem = mem; + cms->completer = &default_filename_completer; + return cms; +} + +ic_private void completions_free(completions_t* cms) { + if (cms == NULL) return; + completions_clear(cms); + if (cms->elems != NULL) { + mem_free(cms->mem, cms->elems); + cms->elems = NULL; + cms->count = 0; + cms->len = 0; + } + mem_free(cms->mem, cms); // free ourselves +} + + +ic_private void completions_clear(completions_t* cms) { + while (cms->count > 0) { + completion_t* cm = cms->elems + cms->count - 1; + mem_free( cms->mem, cm->display); + mem_free( cms->mem, cm->replacement); + mem_free( cms->mem, cm->help); + memset(cm,0,sizeof(*cm)); + cms->count--; + } +} + +static void completions_push(completions_t* cms, const char* replacement, const char* display, const char* help, ssize_t delete_before, ssize_t delete_after) +{ + if (cms->count >= cms->len) { + ssize_t newlen = (cms->len <= 0 ? 32 : cms->len*2); + completion_t* newelems = mem_realloc_tp(cms->mem, completion_t, cms->elems, newlen ); + if (newelems == NULL) return; + cms->elems = newelems; + cms->len = newlen; + } + assert(cms->count < cms->len); + completion_t* cm = cms->elems + cms->count; + cm->replacement = mem_strdup(cms->mem,replacement); + cm->display = mem_strdup(cms->mem,display); + cm->help = mem_strdup(cms->mem,help); + cm->delete_before = delete_before; + cm->delete_after = delete_after; + cms->count++; +} + +ic_private ssize_t completions_count(completions_t* cms) { + return cms->count; +} + +static bool completions_contains(completions_t* cms, const char* replacement) { + for( ssize_t i = 0; i < cms->count; i++ ) { + const completion_t* c = cms->elems + i; + if (strcmp(replacement,c->replacement) == 0) { return true; } + } + return false; +} + +ic_private bool completions_add(completions_t* cms, const char* replacement, const char* display, const char* help, ssize_t delete_before, ssize_t delete_after) { + if (cms->completer_max <= 0) return false; + cms->completer_max--; + //debug_msg("completion: add: %d,%d, %s\n", delete_before, delete_after, replacement); + if (!completions_contains(cms,replacement)) { + completions_push(cms, replacement, display, help, delete_before, delete_after); + } + return true; +} + +static completion_t* completions_get(completions_t* cms, ssize_t index) { + if (index < 0 || cms->count <= 0 || index >= cms->count) return NULL; + return &cms->elems[index]; +} + +ic_private const char* completions_get_display( completions_t* cms, ssize_t index, const char** help ) { + if (help != NULL) { *help = NULL; } + completion_t* cm = completions_get(cms, index); + if (cm == NULL) return NULL; + if (help != NULL) { *help = cm->help; } + return (cm->display != NULL ? cm->display : cm->replacement); +} + +ic_private const char* completions_get_help( completions_t* cms, ssize_t index ) { + completion_t* cm = completions_get(cms, index); + if (cm == NULL) return NULL; + return cm->help; +} + +ic_private const char* completions_get_hint(completions_t* cms, ssize_t index, const char** help) { + if (help != NULL) { *help = NULL; } + completion_t* cm = completions_get(cms, index); + if (cm == NULL) return NULL; + ssize_t len = ic_strlen(cm->replacement); + if (len < cm->delete_before) return NULL; + const char* hint = (cm->replacement + cm->delete_before); + if (*hint == 0 || utf8_is_cont((uint8_t)(*hint))) return NULL; // utf8 boundary? + if (help != NULL) { *help = cm->help; } + return hint; +} + +ic_private void completions_set_completer(completions_t* cms, ic_completer_fun_t* completer, void* arg) { + cms->completer = completer; + cms->completer_arg = arg; +} + +ic_private void completions_get_completer(completions_t* cms, ic_completer_fun_t** completer, void** arg) { + *completer = cms->completer; + *arg = cms->completer_arg; +} + + +ic_public void* ic_completion_arg( const ic_completion_env_t* cenv ) { + return (cenv == NULL ? NULL : cenv->env->completions->completer_arg); +} + +ic_public bool ic_has_completions( const ic_completion_env_t* cenv ) { + return (cenv == NULL ? false : cenv->env->completions->count > 0); +} + +ic_public bool ic_stop_completing( const ic_completion_env_t* cenv) { + return (cenv == NULL ? true : cenv->env->completions->completer_max <= 0); +} + + +static ssize_t completion_apply( completion_t* cm, stringbuf_t* sbuf, ssize_t pos ) { + if (cm == NULL) return -1; + debug_msg( "completion: apply: %s at %zd\n", cm->replacement, pos); + ssize_t start = pos - cm->delete_before; + if (start < 0) start = 0; + ssize_t n = cm->delete_before + cm->delete_after; + if (ic_strlen(cm->replacement) == n && strncmp(sbuf_string_at(sbuf,start), cm->replacement, to_size_t(n)) == 0) { + // no changes + return -1; + } + else { + sbuf_delete_from_to( sbuf, start, pos + cm->delete_after ); + return sbuf_insert_at(sbuf, cm->replacement, start); + } +} + +ic_private ssize_t completions_apply( completions_t* cms, ssize_t index, stringbuf_t* sbuf, ssize_t pos ) { + completion_t* cm = completions_get(cms, index); + return completion_apply( cm, sbuf, pos ); +} + + +static int completion_compare(const void* p1, const void* p2) { + if (p1 == NULL || p2 == NULL) return 0; + const completion_t* cm1 = (const completion_t*)p1; + const completion_t* cm2 = (const completion_t*)p2; + return ic_stricmp(cm1->replacement, cm2->replacement); +} + +ic_private void completions_sort(completions_t* cms) { + if (cms->count <= 0) return; + qsort(cms->elems, to_size_t(cms->count), sizeof(cms->elems[0]), &completion_compare); +} + +#define IC_MAX_PREFIX (256) + +// find longest common prefix and complete with that. +ic_private ssize_t completions_apply_longest_prefix(completions_t* cms, stringbuf_t* sbuf, ssize_t pos) { + if (cms->count <= 1) { + return completions_apply(cms,0,sbuf,pos); + } + + // set initial prefix to the first entry + completion_t* cm = completions_get(cms, 0); + if (cm == NULL) return -1; + + char prefix[IC_MAX_PREFIX+1]; + ssize_t delete_before = cm->delete_before; + ic_strncpy( prefix, IC_MAX_PREFIX+1, cm->replacement, IC_MAX_PREFIX ); + prefix[IC_MAX_PREFIX] = 0; + + // and visit all others to find the longest common prefix + for(ssize_t i = 1; i < cms->count; i++) { + cm = completions_get(cms,i); + if (cm->delete_before != delete_before) { // deletions must match delete_before + prefix[0] = 0; + break; + } + // check if it is still a prefix + const char* r = cm->replacement; + ssize_t j; + for(j = 0; prefix[j] != 0 && r[j] != 0; j++) { + if (prefix[j] != r[j]) break; + } + prefix[j] = 0; + if (j <= 0) break; + } + + // check the length + ssize_t len = ic_strlen(prefix); + if (len <= 0 || len < delete_before) return -1; + + // we found a prefix :-) + completion_t cprefix; + memset(&cprefix,0,sizeof(cprefix)); + cprefix.delete_before = delete_before; + cprefix.replacement = prefix; + ssize_t newpos = completion_apply( &cprefix, sbuf, pos); + if (newpos < 0) return newpos; + + // adjust all delete_before for the new replacement + for( ssize_t i = 0; i < cms->count; i++) { + cm = completions_get(cms,i); + cm->delete_before = len; + } + + return newpos; +} + + +//------------------------------------------------------------- +// Completer functions +//------------------------------------------------------------- + +ic_public bool ic_add_completions(ic_completion_env_t* cenv, const char* prefix, const char** completions) { + for (const char** pc = completions; *pc != NULL; pc++) { + if (ic_istarts_with(*pc, prefix)) { + if (!ic_add_completion_ex(cenv, *pc, NULL, NULL)) return false; + } + } + return true; +} + +ic_public bool ic_add_completion(ic_completion_env_t* cenv, const char* replacement) { + return ic_add_completion_ex(cenv, replacement, NULL, NULL); +} + +ic_public bool ic_add_completion_ex( ic_completion_env_t* cenv, const char* replacement, const char* display, const char* help ) { + return ic_add_completion_prim(cenv,replacement,display,help,0,0); +} + +ic_public bool ic_add_completion_prim(ic_completion_env_t* cenv, const char* replacement, const char* display, const char* help, long delete_before, long delete_after) { + return (*cenv->complete)(cenv->env, cenv->closure, replacement, display, help, delete_before, delete_after ); +} + +static bool prim_add_completion(ic_env_t* env, void* funenv, const char* replacement, const char* display, const char* help, long delete_before, long delete_after) { + ic_unused(funenv); + return completions_add(env->completions, replacement, display, help, delete_before, delete_after); +} + +ic_public void ic_set_default_completer(ic_completer_fun_t* completer, void* arg) { + ic_env_t* env = ic_get_env(); if (env == NULL) return; + completions_set_completer(env->completions, completer, arg); +} + +ic_private ssize_t completions_generate(struct ic_env_s* env, completions_t* cms, const char* input, ssize_t pos, ssize_t max) { + completions_clear(cms); + if (cms->completer == NULL || input == NULL || ic_strlen(input) < pos) return 0; + + // set up env + ic_completion_env_t cenv; + cenv.env = env; + cenv.input = input, + cenv.cursor = (long)pos; + cenv.arg = cms->completer_arg; + cenv.complete = &prim_add_completion; + cenv.closure = NULL; + const char* prefix = mem_strndup(cms->mem, input, pos); + cms->completer_max = max; + + // and complete + cms->completer(&cenv,prefix); + + // restore + mem_free(cms->mem,prefix); + return completions_count(cms); +} + +// The default completer is no completion is set +static void default_filename_completer( ic_completion_env_t* cenv, const char* prefix ) { + #ifdef _WIN32 + const char sep = '\\'; + #else + const char sep = '/'; + #endif + ic_complete_filename( cenv, prefix, sep, ".", NULL); +} diff --git a/extern/isocline/src/completions.h b/extern/isocline/src/completions.h new file mode 100644 index 00000000..8361d507 --- /dev/null +++ b/extern/isocline/src/completions.h @@ -0,0 +1,52 @@ +/* ---------------------------------------------------------------------------- + Copyright (c) 2021, Daan Leijen + This is free software; you can redistribute it and/or modify it + under the terms of the MIT License. A copy of the license can be + found in the "LICENSE" file at the root of this distribution. +-----------------------------------------------------------------------------*/ +#pragma once +#ifndef IC_COMPLETIONS_H +#define IC_COMPLETIONS_H + +#include "common.h" +#include "stringbuf.h" + + +//------------------------------------------------------------- +// Completions +//------------------------------------------------------------- +#define IC_MAX_COMPLETIONS_TO_SHOW (1000) +#define IC_MAX_COMPLETIONS_TO_TRY (IC_MAX_COMPLETIONS_TO_SHOW/4) + +typedef struct completions_s completions_t; + +ic_private completions_t* completions_new(alloc_t* mem); +ic_private void completions_free(completions_t* cms); +ic_private void completions_clear(completions_t* cms); +ic_private bool completions_add(completions_t* cms , const char* replacement, const char* display, const char* help, ssize_t delete_before, ssize_t delete_after); +ic_private ssize_t completions_count(completions_t* cms); +ic_private ssize_t completions_generate(struct ic_env_s* env, completions_t* cms , const char* input, ssize_t pos, ssize_t max); +ic_private void completions_sort(completions_t* cms); +ic_private void completions_set_completer(completions_t* cms, ic_completer_fun_t* completer, void* arg); +ic_private const char* completions_get_display(completions_t* cms , ssize_t index, const char** help); +ic_private const char* completions_get_hint(completions_t* cms, ssize_t index, const char** help); +ic_private void completions_get_completer(completions_t* cms, ic_completer_fun_t** completer, void** arg); + +ic_private ssize_t completions_apply(completions_t* cms, ssize_t index, stringbuf_t* sbuf, ssize_t pos); +ic_private ssize_t completions_apply_longest_prefix(completions_t* cms, stringbuf_t* sbuf, ssize_t pos); + +//------------------------------------------------------------- +// Completion environment +//------------------------------------------------------------- +typedef bool (ic_completion_fun_t)( ic_env_t* env, void* funenv, const char* replacement, const char* display, const char* help, long delete_before, long delete_after ); + +struct ic_completion_env_s { + ic_env_t* env; // the isocline environment + const char* input; // current full input + long cursor; // current cursor position + void* arg; // argument given to `ic_set_completer` + void* closure; // free variables for function composition + ic_completion_fun_t* complete; // function that adds a completion +}; + +#endif // IC_COMPLETIONS_H diff --git a/extern/isocline/src/editline.c b/extern/isocline/src/editline.c new file mode 100644 index 00000000..270c42d9 --- /dev/null +++ b/extern/isocline/src/editline.c @@ -0,0 +1,1142 @@ +/* ---------------------------------------------------------------------------- + Copyright (c) 2021, Daan Leijen + This is free software; you can redistribute it and/or modify it + under the terms of the MIT License. A copy of the license can be + found in the "LICENSE" file at the root of this distribution. +-----------------------------------------------------------------------------*/ +#include +#include + +#include "common.h" +#include "term.h" +#include "tty.h" +#include "env.h" +#include "stringbuf.h" +#include "history.h" +#include "completions.h" +#include "undo.h" +#include "highlight.h" + +//------------------------------------------------------------- +// The editor state +//------------------------------------------------------------- + + + +// editor state +typedef struct editor_s { + stringbuf_t* input; // current user input + stringbuf_t* extra; // extra displayed info (for completion menu etc) + stringbuf_t* hint; // hint displayed as part of the input + stringbuf_t* hint_help; // help for a hint. + ssize_t pos; // current cursor position in the input + ssize_t cur_rows; // current used rows to display our content (including extra content) + ssize_t cur_row; // current row that has the cursor (0 based, relative to the prompt) + ssize_t termw; + bool modified; // has a modification happened? (used for history navigation for example) + bool disable_undo; // temporarily disable auto undo (for history search) + ssize_t history_idx; // current index in the history + editstate_t* undo; // undo buffer + editstate_t* redo; // redo buffer + const char* prompt_text; // text of the prompt before the prompt marker + alloc_t* mem; // allocator + // caches + attrbuf_t* attrs; // reuse attribute buffers + attrbuf_t* attrs_extra; +} editor_t; + + + + + +//------------------------------------------------------------- +// Main edit line +//------------------------------------------------------------- +static char* edit_line( ic_env_t* env, const char* prompt_text ); // defined at bottom +static void edit_refresh(ic_env_t* env, editor_t* eb); + +ic_private char* ic_editline(ic_env_t* env, const char* prompt_text) { + tty_start_raw(env->tty); + term_start_raw(env->term); + char* line = edit_line(env,prompt_text); + term_end_raw(env->term,false); + tty_end_raw(env->tty); + term_writeln(env->term,""); + term_flush(env->term); + return line; +} + + +//------------------------------------------------------------- +// Undo/Redo +//------------------------------------------------------------- + +// capture the current edit state +static void editor_capture(editor_t* eb, editstate_t** es ) { + if (!eb->disable_undo) { + editstate_capture( eb->mem, es, sbuf_string(eb->input), eb->pos ); + } +} + +static void editor_undo_capture(editor_t* eb ) { + editor_capture(eb, &eb->undo ); +} + +static void editor_undo_forget(editor_t* eb) { + if (eb->disable_undo) return; + const char* input = NULL; + ssize_t pos = 0; + editstate_restore(eb->mem, &eb->undo, &input, &pos); + mem_free(eb->mem, input); +} + +static void editor_restore(editor_t* eb, editstate_t** from, editstate_t** to ) { + if (eb->disable_undo) return; + if (*from == NULL) return; + const char* input; + if (to != NULL) { editor_capture( eb, to ); } + if (!editstate_restore( eb->mem, from, &input, &eb->pos )) return; + sbuf_replace( eb->input, input ); + mem_free(eb->mem, input); + eb->modified = false; +} + +static void editor_undo_restore(editor_t* eb, bool with_redo ) { + editor_restore(eb, &eb->undo, (with_redo ? &eb->redo : NULL)); +} + +static void editor_redo_restore(editor_t* eb ) { + editor_restore(eb, &eb->redo, &eb->undo); + eb->modified = false; +} + +static void editor_start_modify(editor_t* eb ) { + editor_undo_capture(eb); + editstate_done(eb->mem, &eb->redo); // clear redo + eb->modified = true; +} + + + +static bool editor_pos_is_at_end(editor_t* eb ) { + return (eb->pos == sbuf_len(eb->input)); +} + +//------------------------------------------------------------- +// Row/Column width and positioning +//------------------------------------------------------------- + + +static void edit_get_prompt_width( ic_env_t* env, editor_t* eb, bool in_extra, ssize_t* promptw, ssize_t* cpromptw ) { + if (in_extra) { + *promptw = 0; + *cpromptw = 0; + } + else { + // todo: cache prompt widths + ssize_t textw = bbcode_column_width(env->bbcode, eb->prompt_text); + ssize_t markerw = bbcode_column_width(env->bbcode, env->prompt_marker); + ssize_t cmarkerw = bbcode_column_width(env->bbcode, env->cprompt_marker); + *promptw = markerw + textw; + *cpromptw = (env->no_multiline_indent || *promptw < cmarkerw ? cmarkerw : *promptw); + } +} + +static ssize_t edit_get_rowcol( ic_env_t* env, editor_t* eb, rowcol_t* rc ) { + ssize_t promptw, cpromptw; + edit_get_prompt_width(env, eb, false, &promptw, &cpromptw); + return sbuf_get_rc_at_pos( eb->input, eb->termw, promptw, cpromptw, eb->pos, rc ); +} + +static void edit_set_pos_at_rowcol( ic_env_t* env, editor_t* eb, ssize_t row, ssize_t col ) { + ssize_t promptw, cpromptw; + edit_get_prompt_width(env, eb, false, &promptw, &cpromptw); + ssize_t pos = sbuf_get_pos_at_rc( eb->input, eb->termw, promptw, cpromptw, row, col ); + if (pos < 0) return; + eb->pos = pos; + edit_refresh(env, eb); +} + +static bool edit_pos_is_at_row_end( ic_env_t* env, editor_t* eb ) { + rowcol_t rc; + edit_get_rowcol( env, eb, &rc ); + return rc.last_on_row; +} + +static void edit_write_prompt( ic_env_t* env, editor_t* eb, ssize_t row, bool in_extra ) { + if (in_extra) return; + bbcode_style_open(env->bbcode, "ic-prompt"); + if (row==0) { + // regular prompt text + bbcode_print( env->bbcode, eb->prompt_text ); + } + else if (!env->no_multiline_indent) { + // multiline continuation indentation + // todo: cache prompt widths + ssize_t textw = bbcode_column_width(env->bbcode, eb->prompt_text ); + ssize_t markerw = bbcode_column_width(env->bbcode, env->prompt_marker); + ssize_t cmarkerw = bbcode_column_width(env->bbcode, env->cprompt_marker); + if (cmarkerw < markerw + textw) { + term_write_repeat(env->term, " ", markerw + textw - cmarkerw ); + } + } + // the marker + bbcode_print(env->bbcode, (row == 0 ? env->prompt_marker : env->cprompt_marker )); + bbcode_style_close(env->bbcode,NULL); +} + +//------------------------------------------------------------- +// Refresh +//------------------------------------------------------------- + +typedef struct refresh_info_s { + ic_env_t* env; + editor_t* eb; + attrbuf_t* attrs; + bool in_extra; + ssize_t first_row; + ssize_t last_row; +} refresh_info_t; + +static bool edit_refresh_rows_iter( + const char* s, + ssize_t row, ssize_t row_start, ssize_t row_len, + ssize_t startw, bool is_wrap, const void* arg, void* res) +{ + ic_unused(res); ic_unused(startw); + const refresh_info_t* info = (const refresh_info_t*)(arg); + term_t* term = info->env->term; + + // debug_msg("edit: line refresh: row %zd, len: %zd\n", row, row_len); + if (row < info->first_row) return false; + if (row > info->last_row) return true; // should not occur + + // term_clear_line(term); + edit_write_prompt(info->env, info->eb, row, info->in_extra); + + //' write output + if (info->attrs == NULL || (info->env->no_highlight && info->env->no_bracematch)) { + term_write_n( term, s + row_start, row_len ); + } + else { + term_write_formatted_n( term, s + row_start, attrbuf_attrs(info->attrs, row_start + row_len) + row_start, row_len ); + } + + // write line ending + if (row < info->last_row) { + if (is_wrap && tty_is_utf8(info->env->tty)) { + #ifndef __APPLE__ + bbcode_print( info->env->bbcode, "[ic-dim]\xE2\x86\x90"); // left arrow + #else + bbcode_print( info->env->bbcode, "[ic-dim]\xE2\x86\xB5" ); // return symbol + #endif + } + term_clear_to_end_of_line(term); + term_writeln(term, ""); + } + else { + term_clear_to_end_of_line(term); + } + return (row >= info->last_row); +} + +static void edit_refresh_rows(ic_env_t* env, editor_t* eb, stringbuf_t* input, attrbuf_t* attrs, + ssize_t promptw, ssize_t cpromptw, bool in_extra, + ssize_t first_row, ssize_t last_row) +{ + if (input == NULL) return; + refresh_info_t info; + info.env = env; + info.eb = eb; + info.attrs = attrs; + info.in_extra = in_extra; + info.first_row = first_row; + info.last_row = last_row; + sbuf_for_each_row( input, eb->termw, promptw, cpromptw, &edit_refresh_rows_iter, &info, NULL); +} + + +static void edit_refresh(ic_env_t* env, editor_t* eb) +{ + // calculate the new cursor row and total rows needed + ssize_t promptw, cpromptw; + edit_get_prompt_width( env, eb, false, &promptw, &cpromptw ); + + if (eb->attrs != NULL) { + highlight( env->mem, env->bbcode, sbuf_string(eb->input), eb->attrs, + (env->no_highlight ? NULL : env->highlighter), env->highlighter_arg ); + } + + // highlight matching braces + if (eb->attrs != NULL && !env->no_bracematch) { + highlight_match_braces(sbuf_string(eb->input), eb->attrs, eb->pos, ic_env_get_match_braces(env), + bbcode_style(env->bbcode,"ic-bracematch"), bbcode_style(env->bbcode,"ic-error")); + } + + // insert hint + if (sbuf_len(eb->hint) > 0) { + if (eb->attrs != NULL) { + attrbuf_insert_at( eb->attrs, eb->pos, sbuf_len(eb->hint), bbcode_style(env->bbcode, "ic-hint") ); + } + sbuf_insert_at(eb->input, sbuf_string(eb->hint), eb->pos ); + } + + // render extra (like a completion menu) + stringbuf_t* extra = NULL; + if (sbuf_len(eb->extra) > 0) { + extra = sbuf_new(eb->mem); + if (extra != NULL) { + if (sbuf_len(eb->hint_help) > 0) { + bbcode_append(env->bbcode, sbuf_string(eb->hint_help), extra, eb->attrs_extra); + } + bbcode_append(env->bbcode, sbuf_string(eb->extra), extra, eb->attrs_extra); + } + } + + // calculate rows and row/col position + rowcol_t rc = { 0 }; + const ssize_t rows_input = sbuf_get_rc_at_pos( eb->input, eb->termw, promptw, cpromptw, eb->pos, &rc ); + rowcol_t rc_extra = { 0 }; + ssize_t rows_extra = 0; + if (extra != NULL) { + rows_extra = sbuf_get_rc_at_pos( extra, eb->termw, 0, 0, 0 /*pos*/, &rc_extra ); + } + const ssize_t rows = rows_input + rows_extra; + debug_msg("edit: refresh: rows %zd, cursor: %zd,%zd (previous rows %zd, cursor row %zd)\n", rows, rc.row, rc.col, eb->cur_rows, eb->cur_row); + + // only render at most terminal height rows + const ssize_t termh = term_get_height(env->term); + ssize_t first_row = 0; // first visible row + ssize_t last_row = rows - 1; // last visible row + if (rows > termh) { + first_row = rc.row - termh + 1; // ensure cursor is visible + if (first_row < 0) first_row = 0; + last_row = first_row + termh - 1; + } + assert(last_row - first_row < termh); + + // reduce flicker + buffer_mode_t bmode = term_set_buffer_mode(env->term, BUFFERED); + + // back up to the first line + term_start_of_line(env->term); + term_up(env->term, (eb->cur_row >= termh ? termh-1 : eb->cur_row) ); + // term_clear_lines_to_end(env->term); // gives flicker in old Windows cmd prompt + + // render rows + edit_refresh_rows( env, eb, eb->input, eb->attrs, promptw, cpromptw, false, first_row, last_row ); + if (rows_extra > 0) { + assert(extra != NULL); + const ssize_t first_rowx = (first_row > rows_input ? first_row - rows_input : 0); + const ssize_t last_rowx = last_row - rows_input; assert(last_rowx >= 0); + edit_refresh_rows(env, eb, extra, eb->attrs_extra, 0, 0, true, first_rowx, last_rowx); + } + + // overwrite trailing rows we do not use anymore + ssize_t rrows = last_row - first_row + 1; // rendered rows + if (rrows < termh && rows < eb->cur_rows) { + ssize_t clear = eb->cur_rows - rows; + while (rrows < termh && clear > 0) { + clear--; + rrows++; + term_writeln(env->term,""); + term_clear_line(env->term); + } + } + + // move cursor back to edit position + term_start_of_line(env->term); + term_up(env->term, first_row + rrows - 1 - rc.row ); + term_right(env->term, rc.col + (rc.row == 0 ? promptw : cpromptw)); + + // and refresh + term_flush(env->term); + + // stop buffering + term_set_buffer_mode(env->term, bmode); + + // restore input by removing the hint + sbuf_delete_at(eb->input, eb->pos, sbuf_len(eb->hint)); + sbuf_delete_at(eb->extra, 0, sbuf_len(eb->hint_help)); + attrbuf_clear(eb->attrs); + attrbuf_clear(eb->attrs_extra); + sbuf_free(extra); + + // update previous + eb->cur_rows = rows; + eb->cur_row = rc.row; +} + +// clear current output +static void edit_clear(ic_env_t* env, editor_t* eb ) { + term_attr_reset(env->term); + term_up(env->term, eb->cur_row); + + // overwrite all rows + for( ssize_t i = 0; i < eb->cur_rows; i++) { + term_clear_line(env->term); + term_writeln(env->term, ""); + } + + // move cursor back + term_up(env->term, eb->cur_rows - eb->cur_row ); +} + + +// clear screen and refresh +static void edit_clear_screen(ic_env_t* env, editor_t* eb ) { + ssize_t cur_rows = eb->cur_rows; + eb->cur_rows = term_get_height(env->term) - 1; + edit_clear(env,eb); + eb->cur_rows = cur_rows; + edit_refresh(env,eb); +} + + +// refresh after a terminal window resized (but before doing further edit operations!) +static bool edit_resize(ic_env_t* env, editor_t* eb ) { + // update dimensions + term_update_dim(env->term); + ssize_t newtermw = term_get_width(env->term); + if (eb->termw == newtermw) return false; + + // recalculate the row layout assuming the hardwrapping for the new terminal width + ssize_t promptw, cpromptw; + edit_get_prompt_width( env, eb, false, &promptw, &cpromptw ); + sbuf_insert_at(eb->input, sbuf_string(eb->hint), eb->pos); // insert used hint + + // render extra (like a completion menu) + stringbuf_t* extra = NULL; + if (sbuf_len(eb->extra) > 0) { + extra = sbuf_new(eb->mem); + if (extra != NULL) { + if (sbuf_len(eb->hint_help) > 0) { + bbcode_append(env->bbcode, sbuf_string(eb->hint_help), extra, NULL); + } + bbcode_append(env->bbcode, sbuf_string(eb->extra), extra, NULL); + } + } + rowcol_t rc = { 0 }; + const ssize_t rows_input = sbuf_get_wrapped_rc_at_pos( eb->input, eb->termw, newtermw, promptw, cpromptw, eb->pos, &rc ); + rowcol_t rc_extra = { 0 }; + ssize_t rows_extra = 0; + if (extra != NULL) { + rows_extra = sbuf_get_wrapped_rc_at_pos(extra, eb->termw, newtermw, 0, 0, 0 /*pos*/, &rc_extra); + } + ssize_t rows = rows_input + rows_extra; + debug_msg("edit: resize: new rows: %zd, cursor row: %zd (previous: rows: %zd, cursor row %zd)\n", rows, rc.row, eb->cur_rows, eb->cur_row); + + // update the newly calculated row and rows + eb->cur_row = rc.row; + if (rows > eb->cur_rows) { + eb->cur_rows = rows; + } + eb->termw = newtermw; + edit_refresh(env,eb); + + // remove hint again + sbuf_delete_at(eb->input, eb->pos, sbuf_len(eb->hint)); + sbuf_free(extra); + return true; +} + +static void editor_append_hint_help(editor_t* eb, const char* help) { + sbuf_clear(eb->hint_help); + if (help != NULL) { + sbuf_replace(eb->hint_help, "[ic-info]"); + sbuf_append(eb->hint_help, help); + sbuf_append(eb->hint_help, "[/ic-info]\n"); + } +} + +// refresh with possible hint +static void edit_refresh_hint(ic_env_t* env, editor_t* eb) { + if (env->no_hint || env->hint_delay > 0) { + // refresh without hint first + edit_refresh(env, eb); + if (env->no_hint) return; + } + + // and see if we can construct a hint (displayed after a delay) + ssize_t count = completions_generate(env, env->completions, sbuf_string(eb->input), eb->pos, 2); + if (count == 1) { + const char* help = NULL; + const char* hint = completions_get_hint(env->completions, 0, &help); + if (hint != NULL) { + sbuf_replace(eb->hint, hint); + editor_append_hint_help(eb, help); + // do auto-tabbing? + if (env->complete_autotab) { + stringbuf_t* sb = sbuf_new(env->mem); // temporary buffer for completion + if (sb != NULL) { + sbuf_replace( sb, sbuf_string(eb->input) ); + ssize_t pos = eb->pos; + const char* extra_hint = hint; + do { + ssize_t newpos = sbuf_insert_at( sb, extra_hint, pos ); + if (newpos <= pos) break; + pos = newpos; + count = completions_generate(env, env->completions, sbuf_string(sb), pos, 2); + if (count == 1) { + const char* extra_help = NULL; + extra_hint = completions_get_hint(env->completions, 0, &extra_help); + if (extra_hint != NULL) { + editor_append_hint_help(eb, extra_help); + sbuf_append(eb->hint, extra_hint); + } + } + } + while(count == 1); + sbuf_free(sb); + } + } + } + } + + if (env->hint_delay <= 0) { + // refresh with hint directly + edit_refresh(env, eb); + } +} + +//------------------------------------------------------------- +// Edit operations +//------------------------------------------------------------- + +static void edit_history_prev(ic_env_t* env, editor_t* eb); +static void edit_history_next(ic_env_t* env, editor_t* eb); + +static void edit_undo_restore(ic_env_t* env, editor_t* eb) { + editor_undo_restore(eb, true); + edit_refresh(env,eb); +} + +static void edit_redo_restore(ic_env_t* env, editor_t* eb) { + editor_redo_restore(eb); + edit_refresh(env,eb); +} + +static void edit_cursor_left(ic_env_t* env, editor_t* eb) { + ssize_t cwidth = 1; + ssize_t prev = sbuf_prev(eb->input,eb->pos,&cwidth); + if (prev < 0) return; + rowcol_t rc; + edit_get_rowcol( env, eb, &rc); + eb->pos = prev; + edit_refresh(env,eb); +} + +static void edit_cursor_right(ic_env_t* env, editor_t* eb) { + ssize_t cwidth = 1; + ssize_t next = sbuf_next(eb->input,eb->pos,&cwidth); + if (next < 0) return; + rowcol_t rc; + edit_get_rowcol( env, eb, &rc); + eb->pos = next; + edit_refresh(env,eb); +} + +static void edit_cursor_line_end(ic_env_t* env, editor_t* eb) { + ssize_t end = sbuf_find_line_end(eb->input,eb->pos); + if (end < 0) return; + eb->pos = end; + edit_refresh(env,eb); +} + +static void edit_cursor_line_start(ic_env_t* env, editor_t* eb) { + ssize_t start = sbuf_find_line_start(eb->input,eb->pos); + if (start < 0) return; + eb->pos = start; + edit_refresh(env,eb); +} + +static void edit_cursor_next_word(ic_env_t* env, editor_t* eb) { + ssize_t end = sbuf_find_word_end(eb->input,eb->pos); + if (end < 0) return; + eb->pos = end; + edit_refresh(env,eb); +} + +static void edit_cursor_prev_word(ic_env_t* env, editor_t* eb) { + ssize_t start = sbuf_find_word_start(eb->input,eb->pos); + if (start < 0) return; + eb->pos = start; + edit_refresh(env,eb); +} + +static void edit_cursor_next_ws_word(ic_env_t* env, editor_t* eb) { + ssize_t end = sbuf_find_ws_word_end(eb->input, eb->pos); + if (end < 0) return; + eb->pos = end; + edit_refresh(env, eb); +} + +static void edit_cursor_prev_ws_word(ic_env_t* env, editor_t* eb) { + ssize_t start = sbuf_find_ws_word_start(eb->input, eb->pos); + if (start < 0) return; + eb->pos = start; + edit_refresh(env, eb); +} + +static void edit_cursor_to_start(ic_env_t* env, editor_t* eb) { + eb->pos = 0; + edit_refresh(env,eb); +} + +static void edit_cursor_to_end(ic_env_t* env, editor_t* eb) { + eb->pos = sbuf_len(eb->input); + edit_refresh(env,eb); +} + + +static void edit_cursor_row_up(ic_env_t* env, editor_t* eb) { + rowcol_t rc; + edit_get_rowcol( env, eb, &rc); + if (rc.row == 0) { + edit_history_prev(env,eb); + } + else { + edit_set_pos_at_rowcol( env, eb, rc.row - 1, rc.col ); + } +} + +static void edit_cursor_row_down(ic_env_t* env, editor_t* eb) { + rowcol_t rc; + ssize_t rows = edit_get_rowcol( env, eb, &rc); + if (rc.row + 1 >= rows) { + edit_history_next(env,eb); + } + else { + edit_set_pos_at_rowcol( env, eb, rc.row + 1, rc.col ); + } +} + + +static void edit_cursor_match_brace(ic_env_t* env, editor_t* eb) { + ssize_t match = find_matching_brace( sbuf_string(eb->input), eb->pos, ic_env_get_match_braces(env), NULL ); + if (match < 0) return; + eb->pos = match; + edit_refresh(env,eb); +} + +static void edit_backspace(ic_env_t* env, editor_t* eb) { + if (eb->pos <= 0) return; + editor_start_modify(eb); + eb->pos = sbuf_delete_char_before(eb->input,eb->pos); + edit_refresh(env,eb); +} + +static void edit_delete_char(ic_env_t* env, editor_t* eb) { + if (eb->pos >= sbuf_len(eb->input)) return; + editor_start_modify(eb); + sbuf_delete_char_at(eb->input,eb->pos); + edit_refresh(env,eb); +} + +static void edit_delete_all(ic_env_t* env, editor_t* eb) { + if (sbuf_len(eb->input) <= 0) return; + editor_start_modify(eb); + sbuf_clear(eb->input); + eb->pos = 0; + edit_refresh(env,eb); +} + +static void edit_delete_to_end_of_line(ic_env_t* env, editor_t* eb) { + ssize_t start = sbuf_find_line_start(eb->input,eb->pos); + if (start < 0) return; + ssize_t end = sbuf_find_line_end(eb->input,eb->pos); + if (end < 0) return; + editor_start_modify(eb); + // if on an empty line, remove it completely + if (start == end && sbuf_char_at(eb->input,end) == '\n') { + end++; + } + else if (start == end && sbuf_char_at(eb->input,start - 1) == '\n') { + eb->pos--; + } + sbuf_delete_from_to( eb->input, eb->pos, end ); + edit_refresh(env,eb); +} + +static void edit_delete_to_start_of_line(ic_env_t* env, editor_t* eb) { + ssize_t start = sbuf_find_line_start(eb->input,eb->pos); + if (start < 0) return; + ssize_t end = sbuf_find_line_end(eb->input,eb->pos); + if (end < 0) return; + editor_start_modify(eb); + // delete start newline if it was an empty line + bool goright = false; + if (start > 0 && sbuf_char_at(eb->input,start-1) == '\n' && start == end) { + // if it is an empty line remove it + start--; + // afterwards, move to start of next line if it exists (so the cursor stays on the same row) + goright = true; + } + sbuf_delete_from_to( eb->input, start, eb->pos ); + eb->pos = start; + if (goright) edit_cursor_right(env,eb); + edit_refresh(env,eb); +} + +static void edit_delete_line(ic_env_t* env, editor_t* eb) { + ssize_t start = sbuf_find_line_start(eb->input,eb->pos); + if (start < 0) return; + ssize_t end = sbuf_find_line_end(eb->input,eb->pos); + if (end < 0) return; + editor_start_modify(eb); + // delete newline as well so no empty line is left; + bool goright = false; + if (start > 0 && sbuf_char_at(eb->input,start-1) == '\n') { + start--; + // afterwards, move to start of next line if it exists (so the cursor stays on the same row) + goright = true; + } + else if (sbuf_char_at(eb->input,end) == '\n') { + end++; + } + sbuf_delete_from_to(eb->input,start,end); + eb->pos = start; + if (goright) edit_cursor_right(env,eb); + edit_refresh(env,eb); +} + +static void edit_delete_to_start_of_word(ic_env_t* env, editor_t* eb) { + ssize_t start = sbuf_find_word_start(eb->input,eb->pos); + if (start < 0) return; + editor_start_modify(eb); + sbuf_delete_from_to( eb->input, start, eb->pos ); + eb->pos = start; + edit_refresh(env,eb); +} + +static void edit_delete_to_end_of_word(ic_env_t* env, editor_t* eb) { + ssize_t end = sbuf_find_word_end(eb->input,eb->pos); + if (end < 0) return; + editor_start_modify(eb); + sbuf_delete_from_to( eb->input, eb->pos, end ); + edit_refresh(env,eb); +} + +static void edit_delete_to_start_of_ws_word(ic_env_t* env, editor_t* eb) { + ssize_t start = sbuf_find_ws_word_start(eb->input, eb->pos); + if (start < 0) return; + editor_start_modify(eb); + sbuf_delete_from_to(eb->input, start, eb->pos); + eb->pos = start; + edit_refresh(env, eb); +} + +static void edit_delete_to_end_of_ws_word(ic_env_t* env, editor_t* eb) { + ssize_t end = sbuf_find_ws_word_end(eb->input, eb->pos); + if (end < 0) return; + editor_start_modify(eb); + sbuf_delete_from_to(eb->input, eb->pos, end); + edit_refresh(env, eb); +} + + +static void edit_delete_word(ic_env_t* env, editor_t* eb) { + ssize_t start = sbuf_find_word_start(eb->input,eb->pos); + if (start < 0) return; + ssize_t end = sbuf_find_word_end(eb->input,eb->pos); + if (end < 0) return; + editor_start_modify(eb); + sbuf_delete_from_to(eb->input,start,end); + eb->pos = start; + edit_refresh(env,eb); +} + +static void edit_swap_char( ic_env_t* env, editor_t* eb ) { + if (eb->pos <= 0 || eb->pos == sbuf_len(eb->input)) return; + editor_start_modify(eb); + eb->pos = sbuf_swap_char(eb->input,eb->pos); + edit_refresh(env,eb); +} + +static void edit_multiline_eol(ic_env_t* env, editor_t* eb) { + if (eb->pos <= 0) return; + if (sbuf_string(eb->input)[eb->pos-1] != env->multiline_eol) return; + editor_start_modify(eb); + // replace line continuation with a real newline + sbuf_delete_at( eb->input, eb->pos-1, 1); + sbuf_insert_at( eb->input, "\n", eb->pos-1); + edit_refresh(env,eb); +} + +static void edit_insert_unicode(ic_env_t* env, editor_t* eb, unicode_t u) { + editor_start_modify(eb); + ssize_t nextpos = sbuf_insert_unicode_at(eb->input, u, eb->pos); + if (nextpos >= 0) eb->pos = nextpos; + edit_refresh_hint(env, eb); +} + +static void edit_auto_brace(ic_env_t* env, editor_t* eb, char c) { + if (env->no_autobrace) return; + const char* braces = ic_env_get_auto_braces(env); + for (const char* b = braces; *b != 0; b += 2) { + if (*b == c) { + const char close = b[1]; + //if (sbuf_char_at(eb->input, eb->pos) != close) { + sbuf_insert_char_at(eb->input, close, eb->pos); + bool balanced = false; + find_matching_brace(sbuf_string(eb->input), eb->pos, braces, &balanced ); + if (!balanced) { + // don't insert if it leads to an unbalanced expression. + sbuf_delete_char_at(eb->input, eb->pos); + } + //} + return; + } + else if (b[1] == c) { + // close brace, check if there we don't overwrite to the right + if (sbuf_char_at(eb->input, eb->pos) == c) { + sbuf_delete_char_at(eb->input, eb->pos); + } + return; + } + } +} + +static void editor_auto_indent(editor_t* eb, const char* pre, const char* post ) { + assert(eb->pos > 0 && sbuf_char_at(eb->input,eb->pos-1) == '\n'); + ssize_t prelen = ic_strlen(pre); + if (prelen > 0) { + if (eb->pos - 1 < prelen) return; + if (!ic_starts_with(sbuf_string(eb->input) + eb->pos - 1 - prelen, pre)) return; + if (!ic_starts_with(sbuf_string(eb->input) + eb->pos, post)) return; + eb->pos = sbuf_insert_at(eb->input, " ", eb->pos); + sbuf_insert_char_at(eb->input, '\n', eb->pos); + } +} + +static void edit_insert_char(ic_env_t* env, editor_t* eb, char c) { + editor_start_modify(eb); + ssize_t nextpos = sbuf_insert_char_at( eb->input, c, eb->pos ); + if (nextpos >= 0) eb->pos = nextpos; + edit_auto_brace(env, eb, c); + if (c=='\n') { + editor_auto_indent(eb, "{", "}"); // todo: custom auto indent tokens? + } + edit_refresh_hint(env,eb); +} + +//------------------------------------------------------------- +// Help +//------------------------------------------------------------- + +#include "editline_help.c" + +//------------------------------------------------------------- +// History +//------------------------------------------------------------- + +#include "editline_history.c" + +//------------------------------------------------------------- +// Completion +//------------------------------------------------------------- + +#include "editline_completion.c" + + +//------------------------------------------------------------- +// Edit line: main edit loop +//------------------------------------------------------------- + +static char* edit_line( ic_env_t* env, const char* prompt_text ) +{ + // set up an edit buffer + editor_t eb; + memset(&eb, 0, sizeof(eb)); + eb.mem = env->mem; + eb.input = sbuf_new(env->mem); + eb.extra = sbuf_new(env->mem); + eb.hint = sbuf_new(env->mem); + eb.hint_help= sbuf_new(env->mem); + eb.termw = term_get_width(env->term); + eb.pos = 0; + eb.cur_rows = 1; + eb.cur_row = 0; + eb.modified = false; + eb.prompt_text = (prompt_text != NULL ? prompt_text : ""); + eb.history_idx = 0; + editstate_init(&eb.undo); + editstate_init(&eb.redo); + if (eb.input==NULL || eb.extra==NULL || eb.hint==NULL || eb.hint_help==NULL) { + return NULL; + } + + // caching + if (!(env->no_highlight && env->no_bracematch)) { + eb.attrs = attrbuf_new(env->mem); + eb.attrs_extra = attrbuf_new(env->mem); + } + + // show prompt + edit_write_prompt(env, &eb, 0, false); + + // always a history entry for the current input + history_push(env->history, ""); + + // process keys + code_t c; // current key code + while(true) { + // read a character + term_flush(env->term); + if (env->hint_delay <= 0 || sbuf_len(eb.hint) == 0) { + // blocking read + c = tty_read(env->tty); + } + else { + // timeout to display hint + if (!tty_read_timeout(env->tty, env->hint_delay, &c)) { + // timed-out + if (sbuf_len(eb.hint) > 0) { + // display hint + edit_refresh(env, &eb); + } + c = tty_read(env->tty); + } + else { + // clear the pending hint if we got input before the delay expired + sbuf_clear(eb.hint); + sbuf_clear(eb.hint_help); + } + } + + // update terminal in case of a resize + if (tty_term_resize_event(env->tty)) { + edit_resize(env,&eb); + } + + // clear hint only after a potential resize (so resize row calculations are correct) + const bool had_hint = (sbuf_len(eb.hint) > 0); + sbuf_clear(eb.hint); + sbuf_clear(eb.hint_help); + + // if the user tries to move into a hint with left-cursor or end, we complete it first + if ((c == KEY_RIGHT || c == KEY_END) && had_hint) { + edit_generate_completions(env, &eb, true); + c = KEY_NONE; + } + + // Operations that may return + if (c == KEY_ENTER) { + if (!env->singleline_only && eb.pos > 0 && + sbuf_string(eb.input)[eb.pos-1] == env->multiline_eol && + edit_pos_is_at_row_end(env,&eb)) + { + // replace line-continuation with newline + edit_multiline_eol(env,&eb); + } + else { + // otherwise done + break; + } + } + else if (c == KEY_CTRL_D) { + if (eb.pos == 0 && editor_pos_is_at_end(&eb)) break; // ctrl+D on empty quits with NULL + edit_delete_char(env,&eb); // otherwise it is like delete + } + else if (c == KEY_CTRL_C || c == KEY_EVENT_STOP) { + break; // ctrl+C or STOP event quits with NULL + } + else if (c == KEY_ESC) { + if (eb.pos == 0 && editor_pos_is_at_end(&eb)) break; // ESC on empty input returns with empty input + edit_delete_all(env,&eb); // otherwise delete the current input + // edit_delete_line(env,&eb); // otherwise delete the current line + } + else if (c == KEY_BELL /* ^G */) { + edit_delete_all(env,&eb); + break; // ctrl+G cancels (and returns empty input) + } + + // Editing Operations + else switch(c) { + // events + case KEY_EVENT_RESIZE: // not used + edit_resize(env,&eb); + break; + case KEY_EVENT_AUTOTAB: + edit_generate_completions(env, &eb, true); + break; + + // completion, history, help, undo + case KEY_TAB: + case WITH_ALT('?'): + edit_generate_completions(env,&eb,false); + break; + case KEY_CTRL_R: + case KEY_CTRL_S: + edit_history_search_with_current_word(env,&eb); + break; + case KEY_CTRL_P: + edit_history_prev(env, &eb); + break; + case KEY_CTRL_N: + edit_history_next(env, &eb); + break; + case KEY_CTRL_L: + edit_clear_screen(env, &eb); + break; + case KEY_CTRL_Z: + case WITH_CTRL('_'): + edit_undo_restore(env, &eb); + break; + case KEY_CTRL_Y: + edit_redo_restore(env, &eb); + break; + case KEY_F1: + edit_show_help(env, &eb); + break; + + // navigation + case KEY_LEFT: + case KEY_CTRL_B: + edit_cursor_left(env,&eb); + break; + case KEY_RIGHT: + case KEY_CTRL_F: + if (eb.pos == sbuf_len(eb.input)) { + edit_generate_completions( env, &eb, false ); + } + else { + edit_cursor_right(env,&eb); + } + break; + case KEY_UP: + edit_cursor_row_up(env,&eb); + break; + case KEY_DOWN: + edit_cursor_row_down(env,&eb); + break; + case KEY_HOME: + case KEY_CTRL_A: + edit_cursor_line_start(env,&eb); + break; + case KEY_END: + case KEY_CTRL_E: + edit_cursor_line_end(env,&eb); + break; + case KEY_CTRL_LEFT: + case WITH_SHIFT(KEY_LEFT): + case WITH_ALT('b'): + edit_cursor_prev_word(env,&eb); + break; + case KEY_CTRL_RIGHT: + case WITH_SHIFT(KEY_RIGHT): + case WITH_ALT('f'): + if (eb.pos == sbuf_len(eb.input)) { + edit_generate_completions( env, &eb, false ); + } + else { + edit_cursor_next_word(env,&eb); + } + break; + case KEY_CTRL_HOME: + case WITH_SHIFT(KEY_HOME): + case KEY_PAGEUP: + case WITH_ALT('<'): + edit_cursor_to_start(env,&eb); + break; + case KEY_CTRL_END: + case WITH_SHIFT(KEY_END): + case KEY_PAGEDOWN: + case WITH_ALT('>'): + edit_cursor_to_end(env,&eb); + break; + case WITH_ALT('m'): + edit_cursor_match_brace(env,&eb); + break; + + // deletion + case KEY_BACKSP: + edit_backspace(env,&eb); + break; + case KEY_DEL: + edit_delete_char(env,&eb); + break; + case WITH_ALT('d'): + edit_delete_to_end_of_word(env,&eb); + break; + case KEY_CTRL_W: + edit_delete_to_start_of_ws_word(env, &eb); + break; + case WITH_ALT(KEY_DEL): + case WITH_ALT(KEY_BACKSP): + edit_delete_to_start_of_word(env,&eb); + break; + case KEY_CTRL_U: + edit_delete_to_start_of_line(env,&eb); + break; + case KEY_CTRL_K: + edit_delete_to_end_of_line(env,&eb); + break; + case KEY_CTRL_T: + edit_swap_char(env,&eb); + break; + + // Editing + case KEY_SHIFT_TAB: + case KEY_LINEFEED: // '\n' (ctrl+J, shift+enter) + if (!env->singleline_only) { + edit_insert_char(env, &eb, '\n'); + } + break; + default: { + char chr; + unicode_t uchr; + if (code_is_ascii_char(c,&chr)) { + edit_insert_char(env,&eb,chr); + } + else if (code_is_unicode(c, &uchr)) { + edit_insert_unicode(env,&eb, uchr); + } + else { + debug_msg( "edit: ignore code: 0x%04x\n", c); + } + break; + } + } + + } + + // goto end + eb.pos = sbuf_len(eb.input); + + // refresh once more but without brace matching + bool bm = env->no_bracematch; + env->no_bracematch = true; + edit_refresh(env,&eb); + env->no_bracematch = bm; + + // save result + char* res; + if ((c == KEY_CTRL_D && sbuf_len(eb.input) == 0) || c == KEY_CTRL_C || c == KEY_EVENT_STOP) { + res = NULL; + } + else if (!tty_is_utf8(env->tty)) { + res = sbuf_strdup_from_utf8(eb.input); + } + else { + res = sbuf_strdup(eb.input); + } + + // update history + history_update(env->history, sbuf_string(eb.input)); + if (res == NULL || sbuf_len(eb.input) <= 1) { ic_history_remove_last(); } // no empty or single-char entries + history_save(env->history); + + // free resources + editstate_done(env->mem, &eb.undo); + editstate_done(env->mem, &eb.redo); + attrbuf_free(eb.attrs); + attrbuf_free(eb.attrs_extra); + sbuf_free(eb.input); + sbuf_free(eb.extra); + sbuf_free(eb.hint); + sbuf_free(eb.hint_help); + + return res; +} + diff --git a/extern/isocline/src/editline_completion.c b/extern/isocline/src/editline_completion.c new file mode 100644 index 00000000..1734ef34 --- /dev/null +++ b/extern/isocline/src/editline_completion.c @@ -0,0 +1,277 @@ +/* ---------------------------------------------------------------------------- + Copyright (c) 2021, Daan Leijen + This is free software; you can redistribute it and/or modify it + under the terms of the MIT License. A copy of the license can be + found in the "LICENSE" file at the root of this distribution. +-----------------------------------------------------------------------------*/ + +//------------------------------------------------------------- +// Completion menu: this file is included in editline.c +//------------------------------------------------------------- + +// return true if anything changed +static bool edit_complete(ic_env_t* env, editor_t* eb, ssize_t idx) { + editor_start_modify(eb); + ssize_t newpos = completions_apply(env->completions, idx, eb->input, eb->pos); + if (newpos < 0) { + editor_undo_restore(eb,false); + return false; + } + eb->pos = newpos; + edit_refresh(env,eb); + return true; +} + +static bool edit_complete_longest_prefix(ic_env_t* env, editor_t* eb ) { + editor_start_modify(eb); + ssize_t newpos = completions_apply_longest_prefix( env->completions, eb->input, eb->pos ); + if (newpos < 0) { + editor_undo_restore(eb,false); + return false; + } + eb->pos = newpos; + edit_refresh(env,eb); + return true; +} + +ic_private void sbuf_append_tagged( stringbuf_t* sb, const char* tag, const char* content ) { + sbuf_appendf(sb, "[%s]", tag); + sbuf_append(sb,content); + sbuf_append(sb,"[/]"); +} + +static void editor_append_completion(ic_env_t* env, editor_t* eb, ssize_t idx, ssize_t width, bool numbered, bool selected ) { + const char* help = NULL; + const char* display = completions_get_display(env->completions, idx, &help); + if (display == NULL) return; + if (numbered) { + sbuf_appendf(eb->extra, "[ic-info]%s%zd [/]", (selected ? (tty_is_utf8(env->tty) ? "\xE2\x86\x92" : "*") : " "), 1 + idx); + width -= 3; + } + + if (width > 0) { + sbuf_appendf(eb->extra, "[width=\"%zd;left; ;on\"]", width ); + } + if (selected) { + sbuf_append(eb->extra, "[ic-emphasis]"); + } + sbuf_append(eb->extra, display); + if (selected) { sbuf_append(eb->extra,"[/ic-emphasis]"); } + if (help != NULL) { + sbuf_append(eb->extra, " "); + sbuf_append_tagged(eb->extra, "ic-info", help ); + } + if (width > 0) { sbuf_append(eb->extra,"[/width]"); } +} + +// 2 and 3 column output up to 80 wide +#define IC_DISPLAY2_MAX 34 +#define IC_DISPLAY2_COL (3+IC_DISPLAY2_MAX) +#define IC_DISPLAY2_WIDTH (2*IC_DISPLAY2_COL + 2) // 75 + +#define IC_DISPLAY3_MAX 21 +#define IC_DISPLAY3_COL (3+IC_DISPLAY3_MAX) +#define IC_DISPLAY3_WIDTH (3*IC_DISPLAY3_COL + 2*2) // 76 + +static void editor_append_completion2(ic_env_t* env, editor_t* eb, ssize_t col_width, ssize_t idx1, ssize_t idx2, ssize_t selected ) { + editor_append_completion(env, eb, idx1, col_width, true, (idx1 == selected) ); + sbuf_append( eb->extra, " "); + editor_append_completion(env, eb, idx2, col_width, true, (idx2 == selected) ); +} + +static void editor_append_completion3(ic_env_t* env, editor_t* eb, ssize_t col_width, ssize_t idx1, ssize_t idx2, ssize_t idx3, ssize_t selected ) { + editor_append_completion(env, eb, idx1, col_width, true, (idx1 == selected) ); + sbuf_append( eb->extra, " "); + editor_append_completion(env, eb, idx2, col_width, true, (idx2 == selected)); + sbuf_append( eb->extra, " "); + editor_append_completion(env, eb, idx3, col_width, true, (idx3 == selected) ); +} + +static ssize_t edit_completions_max_width( ic_env_t* env, ssize_t count ) { + ssize_t max_width = 0; + for( ssize_t i = 0; i < count; i++) { + const char* help = NULL; + ssize_t w = bbcode_column_width(env->bbcode, completions_get_display(env->completions, i, &help)); + if (help != NULL) { + w += 2 + bbcode_column_width(env->bbcode, help); + } + if (w > max_width) { + max_width = w; + } + } + return max_width; +} + +static void edit_completion_menu(ic_env_t* env, editor_t* eb, bool more_available) { + ssize_t count = completions_count(env->completions); + ssize_t count_displayed = count; + assert(count > 1); + ssize_t selected = (env->complete_nopreview ? 0 : -1); // select first or none + ssize_t percolumn = count; + +again: + // show first 9 (or 8) completions + sbuf_clear(eb->extra); + ssize_t twidth = term_get_width(env->term) - 1; + ssize_t colwidth; + if (count > 3 && ((colwidth = 3 + edit_completions_max_width(env, 9))*3 + 2*2) < twidth) { + // display as a 3 column block + count_displayed = (count > 9 ? 9 : count); + percolumn = 3; + for (ssize_t rw = 0; rw < percolumn; rw++) { + if (rw > 0) sbuf_append(eb->extra, "\n"); + editor_append_completion3(env, eb, colwidth, rw, percolumn+rw, (2*percolumn)+rw, selected); + } + } + else if (count > 4 && ((colwidth = 3 + edit_completions_max_width(env, 8))*2 + 2) < twidth) { + // display as a 2 column block if some entries are too wide for three columns + count_displayed = (count > 8 ? 8 : count); + percolumn = (count_displayed <= 6 ? 3 : 4); + for (ssize_t rw = 0; rw < percolumn; rw++) { + if (rw > 0) sbuf_append(eb->extra, "\n"); + editor_append_completion2(env, eb, colwidth, rw, percolumn+rw, selected); + } + } + else { + // display as a list + count_displayed = (count > 9 ? 9 : count); + percolumn = count_displayed; + for (ssize_t i = 0; i < count_displayed; i++) { + if (i > 0) sbuf_append(eb->extra, "\n"); + editor_append_completion(env, eb, i, -1, true /* numbered */, selected == i); + } + } + if (count > count_displayed) { + if (more_available) { + sbuf_append(eb->extra, "\n[ic-info](press page-down (or ctrl-j) to see all further completions)[/]"); + } + else { + sbuf_appendf(eb->extra, "\n[ic-info](press page-down (or ctrl-j) to see all %zd completions)[/]", count ); + } + } + if (!env->complete_nopreview && selected >= 0 && selected <= count_displayed) { + edit_complete(env,eb,selected); + editor_undo_restore(eb,false); + } + else { + edit_refresh(env, eb); + } + + // read here; if not a valid key, push it back and return to main event loop + code_t c = tty_read(env->tty); + if (tty_term_resize_event(env->tty)) { + edit_resize(env, eb); + } + sbuf_clear(eb->extra); + + // direct selection? + if (c >= '1' && c <= '9') { + ssize_t i = (c - '1'); + if (i < count) { + selected = i; + c = KEY_ENTER; + } + } + + // process commands + if (c == KEY_DOWN || c == KEY_TAB) { + selected++; + if (selected >= count_displayed) { + //term_beep(env->term); + selected = 0; + } + goto again; + } + else if (c == KEY_UP || c == KEY_SHIFT_TAB) { + selected--; + if (selected < 0) { + selected = count_displayed - 1; + //term_beep(env->term); + } + goto again; + } + else if (c == KEY_F1) { + edit_show_help(env, eb); + goto again; + } + else if (c == KEY_ESC) { + completions_clear(env->completions); + edit_refresh(env,eb); + c = 0; // ignore and return + } + else if (selected >= 0 && (c == KEY_ENTER || c == KEY_RIGHT || c == KEY_END)) /* || c == KEY_TAB*/ { + // select the current entry + assert(selected < count); + c = 0; + edit_complete(env, eb, selected); + if (env->complete_autotab) { + tty_code_pushback(env->tty,KEY_EVENT_AUTOTAB); // immediately try to complete again + } + } + else if (!env->complete_nopreview && !code_is_virt_key(c)) { + // if in preview mode, select the current entry and exit the menu + assert(selected < count); + edit_complete(env, eb, selected); + } + else if ((c == KEY_PAGEDOWN || c == KEY_LINEFEED) && count > 9) { + // show all completions + c = 0; + if (more_available) { + // generate all entries (up to the max (= 1000)) + count = completions_generate(env, env->completions, sbuf_string(eb->input), eb->pos, IC_MAX_COMPLETIONS_TO_SHOW); + } + rowcol_t rc; + edit_get_rowcol(env,eb,&rc); + edit_clear(env,eb); + edit_write_prompt(env,eb,0,false); + term_writeln(env->term, ""); + for(ssize_t i = 0; i < count; i++) { + const char* display = completions_get_display(env->completions, i, NULL); + if (display != NULL) { + bbcode_println(env->bbcode, display); + } + } + if (count >= IC_MAX_COMPLETIONS_TO_SHOW) { + bbcode_println(env->bbcode, "[ic-info]... and more.[/]"); + } + else { + bbcode_printf(env->bbcode, "[ic-info](%zd possible completions)[/]\n", count ); + } + for(ssize_t i = 0; i < rc.row+1; i++) { + term_write(env->term, " \n"); + } + eb->cur_rows = 0; + edit_refresh(env,eb); + } + else { + edit_refresh(env,eb); + } + // done + completions_clear(env->completions); + if (c != 0) tty_code_pushback(env->tty,c); +} + +static void edit_generate_completions(ic_env_t* env, editor_t* eb, bool autotab) { + debug_msg( "edit: complete: %zd: %s\n", eb->pos, sbuf_string(eb->input) ); + if (eb->pos < 0) return; + ssize_t count = completions_generate(env, env->completions, sbuf_string(eb->input), eb->pos, IC_MAX_COMPLETIONS_TO_TRY); + bool more_available = (count >= IC_MAX_COMPLETIONS_TO_TRY); + if (count <= 0) { + // no completions + if (!autotab) { term_beep(env->term); } + } + else if (count == 1) { + // complete if only one match + if (edit_complete(env,eb,0 /*idx*/) && env->complete_autotab) { + tty_code_pushback(env->tty,KEY_EVENT_AUTOTAB); + } + } + else { + //term_beep(env->term); + if (!more_available) { + edit_complete_longest_prefix(env,eb); + } + completions_sort(env->completions); + edit_completion_menu( env, eb, more_available); + } +} diff --git a/extern/isocline/src/editline_help.c b/extern/isocline/src/editline_help.c new file mode 100644 index 00000000..fa07d1db --- /dev/null +++ b/extern/isocline/src/editline_help.c @@ -0,0 +1,140 @@ +/* ---------------------------------------------------------------------------- + Copyright (c) 2021, Daan Leijen + This is free software; you can redistribute it and/or modify it + under the terms of the MIT License. A copy of the license can be + found in the "LICENSE" file at the root of this distribution. +-----------------------------------------------------------------------------*/ + +//------------------------------------------------------------- +// Help: this is included into editline.c +//------------------------------------------------------------- + +static const char* help[] = { + "","Navigation:", + "left," + "^b", "go one character to the left", + "right," + "^f", "go one character to the right", + "up", "go one row up, or back in the history", + "down", "go one row down, or forward in the history", + #ifdef __APPLE__ + "shift-left", + #else + "^left", + #endif + "go to the start of the previous word", + #ifdef __APPLE__ + "shift-right", + #else + "^right", + #endif + "go to the end the current word", + "home," + "^a", "go to the start of the current line", + "end," + "^e", "go to the end of the current line", + "pgup," + "^home", "go to the start of the current input", + "pgdn," + "^end", "go to the end of the current input", + "alt-m", "jump to matching brace", + "^p", "go back in the history", + "^n", "go forward in the history", + "^r,^s", "search the history starting with the current word", + "","", + + "", "Deletion:", + "del,^d", "delete the current character", + "backsp,^h", "delete the previous character", + "^w", "delete to preceding white space", + "alt-backsp", "delete to the start of the current word", + "alt-d", "delete to the end of the current word", + "^u", "delete to the start of the current line", + "^k", "delete to the end of the current line", + "esc", "delete the current input, or done with empty input", + "","", + + "", "Editing:", + "enter", "accept current input", + #ifndef __APPLE__ + "^enter, ^j", "", + "shift-tab", + #else + "shift-tab,^j", + #endif + "create a new line for multi-line input", + //" ", "(or type '\\' followed by enter)", + "^l", "clear screen", + "^t", "swap with previous character (move character backward)", + "^z,^_", "undo", + "^y", "redo", + //"^C", "done with empty input", + //"F1", "show this help", + "tab", "try to complete the current input", + "","", + "","In the completion menu:", + "enter,left", "use the currently selected completion", + "1 - 9", "use completion N from the menu", + "tab,down", "select the next completion", + "shift-tab,up","select the previous completion", + "esc", "exit menu without completing", + "pgdn,^j", "show all further possible completions", + "","", + "","In incremental history search:", + "enter", "use the currently found history entry", + "backsp," + "^z", "go back to the previous match (undo)", + "tab," + "^r", "find the next match", + "shift-tab," + "^s", "find an earlier match", + "esc", "exit search", + " ","", + NULL, NULL +}; + +static const char* help_initial = + "[ic-info]" + "Isocline v1.0, copyright (c) 2021 Daan Leijen.\n" + "This is free software; you can redistribute it and/or\n" + "modify it under the terms of the MIT License.\n" + "See <[url]https://github.com/daanx/isocline[/url]> for further information.\n" + "We use ^ as a shorthand for ctrl-.\n" + "\n" + "Overview:\n" + "\n[ansi-lightgray]" + " home,ctrl-a cursor end,ctrl-e\n" + " ┌────────────────┼───────────────┐ (navigate)\n" + //" │ │ │\n" + #ifndef __APPLE__ + " │ ctrl-left │ ctrl-right │\n" + #else + " │ alt-left │ alt-right │\n" + #endif + " │ ┌───────┼──────┐ │ ctrl-r : search history\n" + " ▼ ▼ ▼ ▼ ▼ tab : complete word\n" + " prompt> [ansi-darkgray]it's the quintessential language[/] shift-tab: insert new line\n" + " ▲ ▲ ▲ ▲ esc : delete input, done\n" + " │ └──────────────┘ │ ctrl-z : undo\n" + " │ alt-backsp alt-d │\n" + //" │ │ │\n" + " └────────────────────────────────┘ (delete)\n" + " ctrl-u ctrl-k\n" + "[/ansi-lightgray][/ic-info]\n"; + +static void edit_show_help(ic_env_t* env, editor_t* eb) { + edit_clear(env, eb); + bbcode_println(env->bbcode, help_initial); + for (ssize_t i = 0; help[i] != NULL && help[i+1] != NULL; i += 2) { + if (help[i][0] == 0) { + bbcode_printf(env->bbcode, "[ic-info]%s[/]\n", help[i+1]); + } + else { + bbcode_printf(env->bbcode, " [ic-emphasis]%-13s[/][ansi-lightgray]%s%s[/]\n", help[i], (help[i+1][0] == 0 ? "" : ": "), help[i+1]); + } + } + + eb->cur_rows = 0; + eb->cur_row = 0; + edit_refresh(env, eb); +} diff --git a/extern/isocline/src/editline_history.c b/extern/isocline/src/editline_history.c new file mode 100644 index 00000000..2a0afa1c --- /dev/null +++ b/extern/isocline/src/editline_history.c @@ -0,0 +1,260 @@ +/* ---------------------------------------------------------------------------- + Copyright (c) 2021, Daan Leijen + This is free software; you can redistribute it and/or modify it + under the terms of the MIT License. A copy of the license can be + found in the "LICENSE" file at the root of this distribution. +-----------------------------------------------------------------------------*/ + +//------------------------------------------------------------- +// History search: this file is included in editline.c +//------------------------------------------------------------- + +static void edit_history_at(ic_env_t* env, editor_t* eb, int ofs ) +{ + if (eb->modified) { + history_update(env->history, sbuf_string(eb->input)); // update first entry if modified + eb->history_idx = 0; // and start again + eb->modified = false; + } + const char* entry = history_get(env->history,eb->history_idx + ofs); + // debug_msg( "edit: history: at: %d + %d, found: %s\n", eb->history_idx, ofs, entry); + if (entry == NULL) { + term_beep(env->term); + } + else { + eb->history_idx += ofs; + sbuf_replace(eb->input, entry); + if (ofs > 0) { + // at end of first line when scrolling up + ssize_t end = sbuf_find_line_end(eb->input,0); + eb->pos = (end < 0 ? 0 : end); + } + else { + eb->pos = sbuf_len(eb->input); // at end of last line when scrolling down + } + edit_refresh(env, eb); + } +} + +static void edit_history_prev(ic_env_t* env, editor_t* eb) { + edit_history_at(env,eb, 1 ); +} + +static void edit_history_next(ic_env_t* env, editor_t* eb) { + edit_history_at(env,eb, -1 ); +} + +typedef struct hsearch_s { + struct hsearch_s* next; + ssize_t hidx; + ssize_t match_pos; + ssize_t match_len; + bool cinsert; +} hsearch_t; + +static void hsearch_push( alloc_t* mem, hsearch_t** hs, ssize_t hidx, ssize_t mpos, ssize_t mlen, bool cinsert ) { + hsearch_t* h = mem_zalloc_tp( mem, hsearch_t ); + if (h == NULL) return; + h->hidx = hidx; + h->match_pos = mpos; + h->match_len = mlen; + h->cinsert = cinsert; + h->next = *hs; + *hs = h; +} + +static bool hsearch_pop( alloc_t* mem, hsearch_t** hs, ssize_t* hidx, ssize_t* match_pos, ssize_t* match_len, bool* cinsert ) { + hsearch_t* h = *hs; + if (h == NULL) return false; + *hs = h->next; + if (hidx != NULL) *hidx = h->hidx; + if (match_pos != NULL) *match_pos = h->match_pos; + if (match_len != NULL) *match_len = h->match_len; + if (cinsert != NULL) *cinsert = h->cinsert; + mem_free(mem, h); + return true; +} + +static void hsearch_done( alloc_t* mem, hsearch_t* hs ) { + while (hs != NULL) { + hsearch_t* next = hs->next; + mem_free(mem, hs); + hs = next; + } +} + +static void edit_history_search(ic_env_t* env, editor_t* eb, char* initial ) { + if (history_count( env->history ) <= 0) { + term_beep(env->term); + return; + } + + // update history + if (eb->modified) { + history_update(env->history, sbuf_string(eb->input)); // update first entry if modified + eb->history_idx = 0; // and start again + eb->modified = false; + } + + // set a search prompt and remember the previous state + editor_undo_capture(eb); + eb->disable_undo = true; + bool old_hint = ic_enable_hint(false); + const char* prompt_text = eb->prompt_text; + eb->prompt_text = "history search"; + + // search state + hsearch_t* hs = NULL; // search undo + ssize_t hidx = 1; // current history entry + ssize_t match_pos = 0; // current matched position + ssize_t match_len = 0; // length of the match + const char* hentry = NULL; // current history entry + + // Simulate per character searches for each letter in `initial` (so backspace works) + if (initial != NULL) { + const ssize_t initial_len = ic_strlen(initial); + ssize_t ipos = 0; + while( ipos < initial_len ) { + ssize_t next = str_next_ofs( initial, initial_len, ipos, NULL ); + if (next < 0) break; + hsearch_push( eb->mem, &hs, hidx, match_pos, match_len, true); + char c = initial[ipos + next]; // terminate temporarily + initial[ipos + next] = 0; + if (history_search( env->history, hidx, initial, true, &hidx, &match_pos )) { + match_len = ipos + next; + } + else if (ipos + next >= initial_len) { + term_beep(env->term); + } + initial[ipos + next] = c; // restore + ipos += next; + } + sbuf_replace( eb->input, initial); + eb->pos = ipos; + } + else { + sbuf_clear( eb->input ); + eb->pos = 0; + } + + // Incremental search +again: + hentry = history_get(env->history,hidx); + if (hentry != NULL) { + sbuf_appendf(eb->extra, "[ic-info]%zd. [/][ic-diminish][!pre]", hidx); + sbuf_append_n( eb->extra, hentry, match_pos ); + sbuf_append(eb->extra, "[/pre][u ic-emphasis][!pre]" ); + sbuf_append_n( eb->extra, hentry + match_pos, match_len ); + sbuf_append(eb->extra, "[/pre][/u][!pre]" ); + sbuf_append(eb->extra, hentry + match_pos + match_len ); + sbuf_append(eb->extra, "[/pre][/ic-diminish]"); + if (!env->no_help) { + sbuf_append(eb->extra, "\n[ic-info](use tab for the next match)[/]"); + } + sbuf_append(eb->extra, "\n" ); + } + edit_refresh(env, eb); + + // Wait for input + code_t c = (hentry == NULL ? KEY_ESC : tty_read(env->tty)); + if (tty_term_resize_event(env->tty)) { + edit_resize(env, eb); + } + sbuf_clear(eb->extra); + + // Process commands + if (c == KEY_ESC || c == KEY_BELL /* ^G */ || c == KEY_CTRL_C) { + c = 0; + eb->disable_undo = false; + editor_undo_restore(eb, false); + } + else if (c == KEY_ENTER) { + c = 0; + editor_undo_forget(eb); + sbuf_replace( eb->input, hentry ); + eb->pos = sbuf_len(eb->input); + eb->modified = false; + eb->history_idx = hidx; + } + else if (c == KEY_BACKSP || c == KEY_CTRL_Z) { + // undo last search action + bool cinsert; + if (hsearch_pop(env->mem,&hs, &hidx, &match_pos, &match_len, &cinsert)) { + if (cinsert) edit_backspace(env,eb); + } + goto again; + } + else if (c == KEY_CTRL_R || c == KEY_TAB || c == KEY_UP) { + // search backward + hsearch_push(env->mem, &hs, hidx, match_pos, match_len, false); + if (!history_search( env->history, hidx+1, sbuf_string(eb->input), true, &hidx, &match_pos )) { + hsearch_pop(env->mem,&hs,NULL,NULL,NULL,NULL); + term_beep(env->term); + }; + goto again; + } + else if (c == KEY_CTRL_S || c == KEY_SHIFT_TAB || c == KEY_DOWN) { + // search forward + hsearch_push(env->mem, &hs, hidx, match_pos, match_len, false); + if (!history_search( env->history, hidx-1, sbuf_string(eb->input), false, &hidx, &match_pos )) { + hsearch_pop(env->mem, &hs,NULL,NULL,NULL,NULL); + term_beep(env->term); + }; + goto again; + } + else if (c == KEY_F1) { + edit_show_help(env, eb); + goto again; + } + else { + // insert character and search further backward + char chr; + unicode_t uchr; + if (code_is_ascii_char(c,&chr)) { + hsearch_push(env->mem, &hs, hidx, match_pos, match_len, true); + edit_insert_char(env,eb,chr); + } + else if (code_is_unicode(c,&uchr)) { + hsearch_push(env->mem, &hs, hidx, match_pos, match_len, true); + edit_insert_unicode(env,eb,uchr); + } + else { + // ignore command + term_beep(env->term); + goto again; + } + // search for the new input + if (history_search( env->history, hidx, sbuf_string(eb->input), true, &hidx, &match_pos )) { + match_len = sbuf_len(eb->input); + } + else { + term_beep(env->term); + }; + goto again; + } + + // done + eb->disable_undo = false; + hsearch_done(env->mem,hs); + eb->prompt_text = prompt_text; + ic_enable_hint(old_hint); + edit_refresh(env,eb); + if (c != 0) tty_code_pushback(env->tty, c); +} + +// Start an incremental search with the current word +static void edit_history_search_with_current_word(ic_env_t* env, editor_t* eb) { + char* initial = NULL; + ssize_t start = sbuf_find_word_start( eb->input, eb->pos ); + if (start >= 0) { + const ssize_t next = sbuf_next(eb->input, start, NULL); + if (!ic_char_is_idletter(sbuf_string(eb->input) + start, (long)(next - start))) { + start = next; + } + if (start >= 0 && start < eb->pos) { + initial = mem_strndup(eb->mem, sbuf_string(eb->input) + start, eb->pos - start); + } + } + edit_history_search( env, eb, initial); + mem_free(env->mem, initial); +} diff --git a/extern/isocline/src/env.h b/extern/isocline/src/env.h new file mode 100644 index 00000000..edfc1003 --- /dev/null +++ b/extern/isocline/src/env.h @@ -0,0 +1,60 @@ +/* ---------------------------------------------------------------------------- + Copyright (c) 2021, Daan Leijen + This is free software; you can redistribute it and/or modify it + under the terms of the MIT License. A copy of the license can be + found in the "LICENSE" file at the root of this distribution. +-----------------------------------------------------------------------------*/ +#pragma once +#ifndef IC_ENV_H +#define IC_ENV_H + +#include "../include/isocline.h" +#include "common.h" +#include "term.h" +#include "tty.h" +#include "stringbuf.h" +#include "history.h" +#include "completions.h" +#include "bbcode.h" + +//------------------------------------------------------------- +// Environment +//------------------------------------------------------------- + +struct ic_env_s { + alloc_t* mem; // potential custom allocator + ic_env_t* next; // next environment (used for proper deallocation) + term_t* term; // terminal + tty_t* tty; // keyboard (NULL if stdin is a pipe, file, etc) + completions_t* completions; // current completions + history_t* history; // edit history + bbcode_t* bbcode; // print with bbcodes + const char* prompt_marker; // the prompt marker (defaults to "> ") + const char* cprompt_marker; // prompt marker for continuation lines (defaults to `prompt_marker`) + ic_highlight_fun_t* highlighter; // highlight callback + void* highlighter_arg; // user state for the highlighter. + const char* match_braces; // matching braces, e.g "()[]{}" + const char* auto_braces; // auto insertion braces, e.g "()[]{}\"\"''" + char multiline_eol; // character used for multiline input ("\") (set to 0 to disable) + bool initialized; // are we initialized? + bool noedit; // is rich editing possible (tty != NULL) + bool singleline_only; // allow only single line editing? + bool complete_nopreview; // do not show completion preview for each selection in the completion menu? + bool complete_autotab; // try to keep completing after a completion? + bool no_multiline_indent; // indent continuation lines to line up under the initial prompt + bool no_help; // show short help line for history search etc. + bool no_hint; // allow hinting? + bool no_highlight; // enable highlighting? + bool no_bracematch; // enable brace matching? + bool no_autobrace; // enable automatic brace insertion? + bool no_lscolors; // use LSCOLORS/LS_COLORS to colorize file name completions? + long hint_delay; // delay before displaying a hint in milliseconds +}; + +ic_private char* ic_editline(ic_env_t* env, const char* prompt_text); + +ic_private ic_env_t* ic_get_env(void); +ic_private const char* ic_env_get_auto_braces(ic_env_t* env); +ic_private const char* ic_env_get_match_braces(ic_env_t* env); + +#endif // IC_ENV_H diff --git a/extern/isocline/src/highlight.c b/extern/isocline/src/highlight.c new file mode 100644 index 00000000..59c7255c --- /dev/null +++ b/extern/isocline/src/highlight.c @@ -0,0 +1,259 @@ +/* ---------------------------------------------------------------------------- + Copyright (c) 2021, Daan Leijen + This is free software; you can redistribute it and/or modify it + under the terms of the MIT License. A copy of the license can be + found in the "LICENSE" file at the root of this distribution. +-----------------------------------------------------------------------------*/ + +#include +#include "common.h" +#include "term.h" +#include "stringbuf.h" +#include "attr.h" +#include "bbcode.h" + +//------------------------------------------------------------- +// Syntax highlighting +//------------------------------------------------------------- + +struct ic_highlight_env_s { + attrbuf_t* attrs; + const char* input; + ssize_t input_len; + bbcode_t* bbcode; + alloc_t* mem; + ssize_t cached_upos; // cached unicode position + ssize_t cached_cpos; // corresponding utf-8 byte position +}; + + +ic_private void highlight( alloc_t* mem, bbcode_t* bb, const char* s, attrbuf_t* attrs, ic_highlight_fun_t* highlighter, void* arg ) { + const ssize_t len = ic_strlen(s); + if (len <= 0) return; + attrbuf_set_at(attrs,0,len,attr_none()); // fill to length of s + if (highlighter != NULL) { + ic_highlight_env_t henv; + henv.attrs = attrs; + henv.input = s; + henv.input_len = len; + henv.bbcode = bb; + henv.mem = mem; + henv.cached_cpos = 0; + henv.cached_upos = 0; + (*highlighter)( &henv, s, arg ); + } +} + + +//------------------------------------------------------------- +// Client interface +//------------------------------------------------------------- + +static void pos_adjust( ic_highlight_env_t* henv, ssize_t* ppos, ssize_t* plen ) { + ssize_t pos = *ppos; + ssize_t len = *plen; + if (pos >= henv->input_len) return; + if (pos >= 0 && len >= 0) return; // already character positions + if (henv->input == NULL) return; + + if (pos < 0) { + // negative `pos` is used as the unicode character position (for easy interfacing with Haskell) + ssize_t upos = -pos; + ssize_t cpos = 0; + ssize_t ucount = 0; + if (henv->cached_upos <= upos) { // if we have a cached position, start from there + ucount = henv->cached_upos; + cpos = henv->cached_cpos; + } + while ( ucount < upos ) { + ssize_t next = str_next_ofs(henv->input, henv->input_len, cpos, NULL); + if (next <= 0) return; + ucount++; + cpos += next; + } + *ppos = pos = cpos; + // and cache it to avoid quadratic behavior + henv->cached_upos = upos; + henv->cached_cpos = cpos; + } + if (len < 0) { + // negative `len` is used as a unicode character length + len = -len; + ssize_t ucount = 0; + ssize_t clen = 0; + while (ucount < len) { + ssize_t next = str_next_ofs(henv->input, henv->input_len, pos + clen, NULL); + if (next <= 0) return; + ucount++; + clen += next; + } + *plen = len = clen; + // and update cache if possible + if (henv->cached_cpos == pos) { + henv->cached_upos += ucount; + henv->cached_cpos += clen; + } + } +} + +static void highlight_attr(ic_highlight_env_t* henv, ssize_t pos, ssize_t count, attr_t attr ) { + if (henv==NULL) return; + pos_adjust(henv,&pos,&count); + if (pos < 0 || count <= 0) return; + attrbuf_update_at(henv->attrs, pos, count, attr); +} + +ic_public void ic_highlight(ic_highlight_env_t* henv, long pos, long count, const char* style ) { + if (henv == NULL || style==NULL || style[0]==0 || pos < 0) return; + highlight_attr(henv,pos,count,bbcode_style( henv->bbcode, style )); +} + +ic_public void ic_highlight_formatted(ic_highlight_env_t* henv, const char* s, const char* fmt) { + if (s==NULL || s[0] == 0 || fmt==NULL) return; + attrbuf_t* attrs = attrbuf_new(henv->mem); + stringbuf_t* out = sbuf_new(henv->mem); // todo: avoid allocating out? + if (attrs!=NULL && out != NULL) { + bbcode_append( henv->bbcode, fmt, out, attrs); + const ssize_t len = ic_strlen(s); + if (sbuf_len(out) != len) { + debug_msg("highlight: formatted string content differs from the original input:\n original: %s\n formatted: %s\n", s, fmt); + } + for( ssize_t i = 0; i < len; i++) { + attrbuf_update_at(henv->attrs, i, 1, attrbuf_attr_at(attrs,i)); + } + } + sbuf_free(out); + attrbuf_free(attrs); +} + +//------------------------------------------------------------- +// Brace matching +//------------------------------------------------------------- +#define MAX_NESTING (64) + +typedef struct brace_s { + char close; + bool at_cursor; + ssize_t pos; +} brace_t; + +ic_private void highlight_match_braces(const char* s, attrbuf_t* attrs, ssize_t cursor_pos, const char* braces, attr_t match_attr, attr_t error_attr) +{ + brace_t open[MAX_NESTING+1]; + ssize_t nesting = 0; + const ssize_t brace_len = ic_strlen(braces); + for (long i = 0; i < ic_strlen(s); i++) { + const char c = s[i]; + // push open brace + bool found_open = false; + for (ssize_t b = 0; b < brace_len; b += 2) { + if (c == braces[b]) { + // open brace + if (nesting >= MAX_NESTING) return; // give up + open[nesting].close = braces[b+1]; + open[nesting].pos = i; + open[nesting].at_cursor = (i == cursor_pos - 1); + nesting++; + found_open = true; + break; + } + } + if (found_open) continue; + + // pop to closing brace and potentially highlight + for (ssize_t b = 1; b < brace_len; b += 2) { + if (c == braces[b]) { + // close brace + if (nesting <= 0) { + // unmatched close brace + attrbuf_update_at( attrs, i, 1, error_attr); + } + else { + // can we fix an unmatched brace where we can match by popping just one? + if (open[nesting-1].close != c && nesting > 1 && open[nesting-2].close == c) { + // assume previous open brace was wrong + attrbuf_update_at(attrs, open[nesting-1].pos, 1, error_attr); + nesting--; + } + if (open[nesting-1].close != c) { + // unmatched open brace + attrbuf_update_at( attrs, i, 1, error_attr); + } + else { + // matching brace + nesting--; + if (i == cursor_pos - 1 || (open[nesting].at_cursor && open[nesting].pos != i - 1)) { + // highlight matching brace + attrbuf_update_at(attrs, open[nesting].pos, 1, match_attr); + attrbuf_update_at(attrs, i, 1, match_attr); + } + } + } + break; + } + } + } + // note: don't mark further unmatched open braces as in error +} + + +ic_private ssize_t find_matching_brace(const char* s, ssize_t cursor_pos, const char* braces, bool* is_balanced) +{ + if (is_balanced != NULL) { *is_balanced = false; } + bool balanced = true; + ssize_t match = -1; + brace_t open[MAX_NESTING+1]; + ssize_t nesting = 0; + const ssize_t brace_len = ic_strlen(braces); + for (long i = 0; i < ic_strlen(s); i++) { + const char c = s[i]; + // push open brace + bool found_open = false; + for (ssize_t b = 0; b < brace_len; b += 2) { + if (c == braces[b]) { + // open brace + if (nesting >= MAX_NESTING) return -1; // give up + open[nesting].close = braces[b+1]; + open[nesting].pos = i; + open[nesting].at_cursor = (i == cursor_pos - 1); + nesting++; + found_open = true; + break; + } + } + if (found_open) continue; + + // pop to closing brace + for (ssize_t b = 1; b < brace_len; b += 2) { + if (c == braces[b]) { + // close brace + if (nesting <= 0) { + // unmatched close brace + balanced = false; + } + else { + if (open[nesting-1].close != c) { + // unmatched open brace + balanced = false; + } + else { + // matching brace + nesting--; + if (i == cursor_pos - 1) { + // found matching open brace + match = open[nesting].pos + 1; + } + else if (open[nesting].at_cursor) { + // found matching close brace + match = i + 1; + } + } + } + break; + } + } + } + if (nesting != 0) { balanced = false; } + if (is_balanced != NULL) { *is_balanced = balanced; } + return match; +} diff --git a/extern/isocline/src/highlight.h b/extern/isocline/src/highlight.h new file mode 100644 index 00000000..67da02ff --- /dev/null +++ b/extern/isocline/src/highlight.h @@ -0,0 +1,24 @@ +/* ---------------------------------------------------------------------------- + Copyright (c) 2021, Daan Leijen + This is free software; you can redistribute it and/or modify it + under the terms of the MIT License. A copy of the license can be + found in the "LICENSE" file at the root of this distribution. +-----------------------------------------------------------------------------*/ +#pragma once +#ifndef IC_HIGHLIGHT_H +#define IC_HIGHLIGHT_H + +#include "common.h" +#include "attr.h" +#include "term.h" +#include "bbcode.h" + +//------------------------------------------------------------- +// Syntax highlighting +//------------------------------------------------------------- + +ic_private void highlight( alloc_t* mem, bbcode_t* bb, const char* s, attrbuf_t* attrs, ic_highlight_fun_t* highlighter, void* arg ); +ic_private void highlight_match_braces(const char* s, attrbuf_t* attrs, ssize_t cursor_pos, const char* braces, attr_t match_attr, attr_t error_attr); +ic_private ssize_t find_matching_brace(const char* s, ssize_t cursor_pos, const char* braces, bool* is_balanced); + +#endif // IC_HIGHLIGHT_H diff --git a/extern/isocline/src/history.c b/extern/isocline/src/history.c new file mode 100644 index 00000000..440976aa --- /dev/null +++ b/extern/isocline/src/history.c @@ -0,0 +1,269 @@ +/* ---------------------------------------------------------------------------- + Copyright (c) 2021, Daan Leijen + This is free software; you can redistribute it and/or modify it + under the terms of the MIT License. A copy of the license can be + found in the "LICENSE" file at the root of this distribution. +-----------------------------------------------------------------------------*/ +#include +#include +#include + +#include "../include/isocline.h" +#include "common.h" +#include "history.h" +#include "stringbuf.h" + +#define IC_MAX_HISTORY (200) + +struct history_s { + ssize_t count; // current number of entries in use + ssize_t len; // size of elems + const char** elems; // history items (up to count) + const char* fname; // history file + alloc_t* mem; + bool allow_duplicates; // allow duplicate entries? +}; + +ic_private history_t* history_new(alloc_t* mem) { + history_t* h = mem_zalloc_tp(mem,history_t); + h->mem = mem; + return h; +} + +ic_private void history_free(history_t* h) { + if (h == NULL) return; + history_clear(h); + if (h->len > 0) { + mem_free( h->mem, h->elems ); + h->elems = NULL; + h->len = 0; + } + mem_free(h->mem, h->fname); + h->fname = NULL; + mem_free(h->mem, h); // free ourselves +} + +ic_private bool history_enable_duplicates( history_t* h, bool enable ) { + bool prev = h->allow_duplicates; + h->allow_duplicates = enable; + return prev; +} + +ic_private ssize_t history_count(const history_t* h) { + return h->count; +} + +//------------------------------------------------------------- +// push/clear +//------------------------------------------------------------- + +ic_private bool history_update( history_t* h, const char* entry ) { + if (entry==NULL) return false; + history_remove_last(h); + history_push(h,entry); + //debug_msg("history: update: with %s; now at %s\n", entry, history_get(h,0)); + return true; +} + +static void history_delete_at( history_t* h, ssize_t idx ) { + if (idx < 0 || idx >= h->count) return; + mem_free(h->mem, h->elems[idx]); + for(ssize_t i = idx+1; i < h->count; i++) { + h->elems[i-1] = h->elems[i]; + } + h->count--; +} + +ic_private bool history_push( history_t* h, const char* entry ) { + if (h->len <= 0 || entry==NULL) return false; + // remove any older duplicate + if (!h->allow_duplicates) { + for( int i = 0; i < h->count; i++) { + if (strcmp(h->elems[i],entry) == 0) { + history_delete_at(h,i); + } + } + } + // insert at front + if (h->count == h->len) { + // delete oldest entry + history_delete_at(h,0); + } + assert(h->count < h->len); + h->elems[h->count] = mem_strdup(h->mem,entry); + h->count++; + return true; +} + + +static void history_remove_last_n( history_t* h, ssize_t n ) { + if (n <= 0) return; + if (n > h->count) n = h->count; + for( ssize_t i = h->count - n; i < h->count; i++) { + mem_free( h->mem, h->elems[i] ); + } + h->count -= n; + assert(h->count >= 0); +} + +ic_private void history_remove_last(history_t* h) { + history_remove_last_n(h,1); +} + +ic_private void history_clear(history_t* h) { + history_remove_last_n( h, h->count ); +} + +ic_private const char* history_get( const history_t* h, ssize_t n ) { + if (n < 0 || n >= h->count) return NULL; + return h->elems[h->count - n - 1]; +} + +ic_private bool history_search( const history_t* h, ssize_t from /*including*/, const char* search, bool backward, ssize_t* hidx, ssize_t* hpos ) { + const char* p = NULL; + ssize_t i; + if (backward) { + for( i = from; i < h->count; i++ ) { + p = strstr( history_get(h,i), search); + if (p != NULL) break; + } + } + else { + for( i = from; i >= 0; i-- ) { + p = strstr( history_get(h,i), search); + if (p != NULL) break; + } + } + if (p == NULL) return false; + if (hidx != NULL) *hidx = i; + if (hpos != NULL) *hpos = (p - history_get(h,i)); + return true; +} + +//------------------------------------------------------------- +// +//------------------------------------------------------------- + +ic_private void history_load_from(history_t* h, const char* fname, long max_entries ) { + history_clear(h); + h->fname = mem_strdup(h->mem,fname); + if (max_entries == 0) { + assert(h->elems == NULL); + return; + } + if (max_entries < 0 || max_entries > IC_MAX_HISTORY) max_entries = IC_MAX_HISTORY; + h->elems = (const char**)mem_zalloc_tp_n(h->mem, char*, max_entries ); + if (h->elems == NULL) return; + h->len = max_entries; + history_load(h); +} + + + + +//------------------------------------------------------------- +// save/load history to file +//------------------------------------------------------------- + +static char from_xdigit( int c ) { + if (c >= '0' && c <= '9') return (char)(c - '0'); + if (c >= 'A' && c <= 'F') return (char)(10 + (c - 'A')); + if (c >= 'a' && c <= 'f') return (char)(10 + (c - 'a')); + return 0; +} + +static char to_xdigit( uint8_t c ) { + if (c <= 9) return ((char)c + '0'); + if (c >= 10 && c <= 15) return ((char)c - 10 + 'A'); + return '0'; +} + +static bool ic_isxdigit( int c ) { + return ((c >= 'a' && c <= 'f') || (c >= 'A' && c <= 'F') || (c >= '0' && c <= '9')); +} + +static bool history_read_entry( history_t* h, FILE* f, stringbuf_t* sbuf ) { + sbuf_clear(sbuf); + while( !feof(f)) { + int c = fgetc(f); + if (c == EOF || c == '\n') break; + if (c == '\\') { + c = fgetc(f); + if (c == 'n') { sbuf_append(sbuf,"\n"); } + else if (c == 'r') { /* ignore */ } // sbuf_append(sbuf,"\r"); + else if (c == 't') { sbuf_append(sbuf,"\t"); } + else if (c == '\\') { sbuf_append(sbuf,"\\"); } + else if (c == 'x') { + int c1 = fgetc(f); + int c2 = fgetc(f); + if (ic_isxdigit(c1) && ic_isxdigit(c2)) { + char chr = from_xdigit(c1)*16 + from_xdigit(c2); + sbuf_append_char(sbuf,chr); + } + else return false; + } + else return false; + } + else sbuf_append_char(sbuf,(char)c); + } + if (sbuf_len(sbuf)==0 || sbuf_string(sbuf)[0] == '#') return true; + return history_push(h, sbuf_string(sbuf)); +} + +static bool history_write_entry( const char* entry, FILE* f, stringbuf_t* sbuf ) { + sbuf_clear(sbuf); + //debug_msg("history: write: %s\n", entry); + while( entry != NULL && *entry != 0 ) { + char c = *entry++; + if (c == '\\') { sbuf_append(sbuf,"\\\\"); } + else if (c == '\n') { sbuf_append(sbuf,"\\n"); } + else if (c == '\r') { /* ignore */ } // sbuf_append(sbuf,"\\r"); } + else if (c == '\t') { sbuf_append(sbuf,"\\t"); } + else if (c < ' ' || c > '~' || c == '#') { + char c1 = to_xdigit( (uint8_t)c / 16 ); + char c2 = to_xdigit( (uint8_t)c % 16 ); + sbuf_append(sbuf,"\\x"); + sbuf_append_char(sbuf,c1); + sbuf_append_char(sbuf,c2); + } + else sbuf_append_char(sbuf,c); + } + //debug_msg("history: write buf: %s\n", sbuf_string(sbuf)); + + if (sbuf_len(sbuf) > 0) { + sbuf_append(sbuf,"\n"); + fputs(sbuf_string(sbuf),f); + } + return true; +} + +ic_private void history_load( history_t* h ) { + if (h->fname == NULL) return; + FILE* f = fopen(h->fname, "r"); + if (f == NULL) return; + stringbuf_t* sbuf = sbuf_new(h->mem); + if (sbuf != NULL) { + while (!feof(f)) { + if (!history_read_entry(h,f,sbuf)) break; // error + } + sbuf_free(sbuf); + } + fclose(f); +} + +ic_private void history_save( const history_t* h ) { + if (h->fname == NULL) return; + FILE* f = fopen(h->fname, "w"); + if (f == NULL) return; + #ifndef _WIN32 + chmod(h->fname,S_IRUSR|S_IWUSR); + #endif + stringbuf_t* sbuf = sbuf_new(h->mem); + if (sbuf != NULL) { + for( int i = 0; i < h->count; i++ ) { + if (!history_write_entry(h->elems[i],f,sbuf)) break; // error + } + sbuf_free(sbuf); + } + fclose(f); +} diff --git a/extern/isocline/src/history.h b/extern/isocline/src/history.h new file mode 100644 index 00000000..76a37160 --- /dev/null +++ b/extern/isocline/src/history.h @@ -0,0 +1,38 @@ +/* ---------------------------------------------------------------------------- + Copyright (c) 2021, Daan Leijen + This is free software; you can redistribute it and/or modify it + under the terms of the MIT License. A copy of the license can be + found in the "LICENSE" file at the root of this distribution. +-----------------------------------------------------------------------------*/ +#pragma once +#ifndef IC_HISTORY_H +#define IC_HISTORY_H + +#include "common.h" + +//------------------------------------------------------------- +// History +//------------------------------------------------------------- + +struct history_s; +typedef struct history_s history_t; + +ic_private history_t* history_new(alloc_t* mem); +ic_private void history_free(history_t* h); +ic_private void history_clear(history_t* h); +ic_private bool history_enable_duplicates( history_t* h, bool enable ); +ic_private ssize_t history_count(const history_t* h); + +ic_private void history_load_from(history_t* h, const char* fname, long max_entries); +ic_private void history_load( history_t* h ); +ic_private void history_save( const history_t* h ); + +ic_private bool history_push( history_t* h, const char* entry ); +ic_private bool history_update( history_t* h, const char* entry ); +ic_private const char* history_get( const history_t* h, ssize_t n ); +ic_private void history_remove_last(history_t* h); + +ic_private bool history_search( const history_t* h, ssize_t from, const char* search, bool backward, ssize_t* hidx, ssize_t* hpos); + + +#endif // IC_HISTORY_H diff --git a/extern/isocline/src/isocline.c b/extern/isocline/src/isocline.c new file mode 100644 index 00000000..8b6055cf --- /dev/null +++ b/extern/isocline/src/isocline.c @@ -0,0 +1,594 @@ +/* ---------------------------------------------------------------------------- + Copyright (c) 2021, Daan Leijen + This is free software; you can redistribute it and/or modify it + under the terms of the MIT License. A copy of the license can be + found in the "LICENSE" file at the root of this distribution. +-----------------------------------------------------------------------------*/ + +//------------------------------------------------------------- +// Usually we include all sources one file so no internal +// symbols are public in the libray. +// +// You can compile the entire library just as: +// $ gcc -c src/isocline.c +//------------------------------------------------------------- +#if !defined(IC_SEPARATE_OBJS) +# ifndef _CRT_NONSTDC_NO_WARNINGS +# define _CRT_NONSTDC_NO_WARNINGS // for msvc +# endif +# ifndef _CRT_SECURE_NO_WARNINGS +# define _CRT_SECURE_NO_WARNINGS // for msvc +# endif +# define _XOPEN_SOURCE 700 // for wcwidth +# define _DEFAULT_SOURCE // ensure usleep stays visible with _XOPEN_SOURCE >= 700 +# include "attr.c" +# include "bbcode.c" +# include "editline.c" +# include "highlight.c" +# include "undo.c" +# include "history.c" +# include "completers.c" +# include "completions.c" +# include "term.c" +# include "tty_esc.c" +# include "tty.c" +# include "stringbuf.c" +# include "common.c" +#endif + +//------------------------------------------------------------- +// includes +//------------------------------------------------------------- +#include +#include +#include +#include + +#include "../include/isocline.h" +#include "common.h" +#include "env.h" + + +//------------------------------------------------------------- +// Readline +//------------------------------------------------------------- + +static char* ic_getline( alloc_t* mem ); + +ic_public char* ic_readline(const char* prompt_text) +{ + ic_env_t* env = ic_get_env(); + if (env == NULL) return NULL; + if (!env->noedit) { + // terminal editing enabled + return ic_editline(env, prompt_text); // in editline.c + } + else { + // no editing capability (pipe, dumb terminal, etc) + if (env->tty != NULL && env->term != NULL) { + // if the terminal is not interactive, but we are reading from the tty (keyboard), we display a prompt + term_start_raw(env->term); // set utf8 mode on windows + if (prompt_text != NULL) { + term_write(env->term, prompt_text); + } + term_write(env->term, env->prompt_marker); + term_end_raw(env->term, false); + } + // read directly from stdin + return ic_getline(env->mem); + } +} + + +//------------------------------------------------------------- +// Read a line from the stdin stream if there is no editing +// support (like from a pipe, file, or dumb terminal). +//------------------------------------------------------------- + +static char* ic_getline(alloc_t* mem) +{ + // read until eof or newline + stringbuf_t* sb = sbuf_new(mem); + int c; + while (true) { + c = fgetc(stdin); + if (c==EOF || c=='\n') { + break; + } + else { + sbuf_append_char(sb, (char)c); + } + } + return sbuf_free_dup(sb); +} + + +//------------------------------------------------------------- +// Formatted output +//------------------------------------------------------------- + + +ic_public void ic_printf(const char* fmt, ...) { + va_list ap; + va_start(ap, fmt); + ic_vprintf(fmt, ap); + va_end(ap); +} + +ic_public void ic_vprintf(const char* fmt, va_list args) { + ic_env_t* env = ic_get_env(); if (env==NULL || env->bbcode == NULL) return; + bbcode_vprintf(env->bbcode, fmt, args); +} + +ic_public void ic_print(const char* s) { + ic_env_t* env = ic_get_env(); if (env==NULL || env->bbcode==NULL) return; + bbcode_print(env->bbcode, s); +} + +ic_public void ic_println(const char* s) { + ic_env_t* env = ic_get_env(); if (env==NULL || env->bbcode==NULL) return; + bbcode_println(env->bbcode, s); +} + +void ic_style_def(const char* name, const char* fmt) { + ic_env_t* env = ic_get_env(); if (env==NULL || env->bbcode==NULL) return; + bbcode_style_def(env->bbcode, name, fmt); +} + +void ic_style_open(const char* fmt) { + ic_env_t* env = ic_get_env(); if (env==NULL || env->bbcode==NULL) return; + bbcode_style_open(env->bbcode, fmt); +} + +void ic_style_close(void) { + ic_env_t* env = ic_get_env(); if (env==NULL || env->bbcode==NULL) return; + bbcode_style_close(env->bbcode, NULL); +} + + +//------------------------------------------------------------- +// Interface +//------------------------------------------------------------- + +ic_public bool ic_async_stop(void) { + ic_env_t* env = ic_get_env(); if (env==NULL) return false; + if (env->tty==NULL) return false; + return tty_async_stop(env->tty); +} + +static void set_prompt_marker(ic_env_t* env, const char* prompt_marker, const char* cprompt_marker) { + if (prompt_marker == NULL) prompt_marker = "> "; + if (cprompt_marker == NULL) cprompt_marker = prompt_marker; + mem_free(env->mem, env->prompt_marker); + mem_free(env->mem, env->cprompt_marker); + env->prompt_marker = mem_strdup(env->mem, prompt_marker); + env->cprompt_marker = mem_strdup(env->mem, cprompt_marker); +} + +ic_public const char* ic_get_prompt_marker(void) { + ic_env_t* env = ic_get_env(); if (env==NULL) return NULL; + return env->prompt_marker; +} + +ic_public const char* ic_get_continuation_prompt_marker(void) { + ic_env_t* env = ic_get_env(); if (env==NULL) return NULL; + return env->cprompt_marker; +} + +ic_public void ic_set_prompt_marker( const char* prompt_marker, const char* cprompt_marker ) { + ic_env_t* env = ic_get_env(); if (env==NULL) return; + set_prompt_marker(env, prompt_marker, cprompt_marker); +} + +ic_public bool ic_enable_multiline( bool enable ) { + ic_env_t* env = ic_get_env(); if (env==NULL) return false; + bool prev = env->singleline_only; + env->singleline_only = !enable; + return !prev; +} + +ic_public bool ic_enable_beep( bool enable ) { + ic_env_t* env = ic_get_env(); if (env==NULL) return false; + return term_enable_beep(env->term, enable); +} + +ic_public bool ic_enable_color( bool enable ) { + ic_env_t* env = ic_get_env(); if (env==NULL) return false; + return term_enable_color( env->term, enable ); +} + +ic_public bool ic_enable_history_duplicates( bool enable ) { + ic_env_t* env = ic_get_env(); if (env==NULL) return false; + return history_enable_duplicates(env->history, enable); +} + +ic_public void ic_set_history(const char* fname, long max_entries ) { + ic_env_t* env = ic_get_env(); if (env==NULL) return; + history_load_from(env->history, fname, max_entries ); +} + +ic_public void ic_history_remove_last(void) { + ic_env_t* env = ic_get_env(); if (env==NULL) return; + history_remove_last(env->history); +} + +ic_public void ic_history_add( const char* entry ) { + ic_env_t* env = ic_get_env(); if (env==NULL) return; + history_push( env->history, entry ); +} + +ic_public void ic_history_clear(void) { + ic_env_t* env = ic_get_env(); if (env==NULL) return; + history_clear(env->history); +} + +ic_public bool ic_enable_auto_tab( bool enable ) { + ic_env_t* env = ic_get_env(); if (env==NULL) return false; + bool prev = env->complete_autotab; + env->complete_autotab = enable; + return prev; +} + +ic_public bool ic_enable_completion_preview( bool enable ) { + ic_env_t* env = ic_get_env(); if (env==NULL) return false; + bool prev = env->complete_nopreview; + env->complete_nopreview = !enable; + return !prev; +} + +ic_public bool ic_enable_multiline_indent(bool enable) { + ic_env_t* env = ic_get_env(); if (env==NULL) return false; + bool prev = env->no_multiline_indent; + env->no_multiline_indent = !enable; + return !prev; +} + +ic_public bool ic_enable_hint(bool enable) { + ic_env_t* env = ic_get_env(); if (env==NULL) return false; + bool prev = env->no_hint; + env->no_hint = !enable; + return !prev; +} + +ic_public long ic_set_hint_delay(long delay_ms) { + ic_env_t* env = ic_get_env(); if (env==NULL) return false; + long prev = env->hint_delay; + env->hint_delay = (delay_ms < 0 ? 0 : (delay_ms > 5000 ? 5000 : delay_ms)); + return prev; +} + +ic_public void ic_set_tty_esc_delay(long initial_delay_ms, long followup_delay_ms ) { + ic_env_t* env = ic_get_env(); if (env==NULL) return; + if (env->tty == NULL) return; + tty_set_esc_delay(env->tty, initial_delay_ms, followup_delay_ms); +} + + +ic_public bool ic_enable_highlight(bool enable) { + ic_env_t* env = ic_get_env(); if (env==NULL) return false; + bool prev = env->no_highlight; + env->no_highlight = !enable; + return !prev; +} + +ic_public bool ic_enable_inline_help(bool enable) { + ic_env_t* env = ic_get_env(); if (env==NULL) return false; + bool prev = env->no_help; + env->no_help = !enable; + return !prev; +} + +ic_public bool ic_enable_brace_matching(bool enable) { + ic_env_t* env = ic_get_env(); if (env==NULL) return false; + bool prev = env->no_bracematch; + env->no_bracematch = !enable; + return !prev; +} + +ic_public void ic_set_matching_braces(const char* brace_pairs) { + ic_env_t* env = ic_get_env(); if (env==NULL) return; + mem_free(env->mem, env->match_braces); + env->match_braces = NULL; + if (brace_pairs != NULL) { + ssize_t len = ic_strlen(brace_pairs); + if (len > 0 && (len % 2) == 0) { + env->match_braces = mem_strdup(env->mem, brace_pairs); + } + } +} + +ic_public bool ic_enable_brace_insertion(bool enable) { + ic_env_t* env = ic_get_env(); if (env==NULL) return false; + bool prev = env->no_autobrace; + env->no_autobrace = !enable; + return !prev; +} + +ic_public void ic_set_insertion_braces(const char* brace_pairs) { + ic_env_t* env = ic_get_env(); if (env==NULL) return; + mem_free(env->mem, env->auto_braces); + env->auto_braces = NULL; + if (brace_pairs != NULL) { + ssize_t len = ic_strlen(brace_pairs); + if (len > 0 && (len % 2) == 0) { + env->auto_braces = mem_strdup(env->mem, brace_pairs); + } + } +} + +ic_private const char* ic_env_get_match_braces(ic_env_t* env) { + return (env->match_braces == NULL ? "()[]{}" : env->match_braces); +} + +ic_private const char* ic_env_get_auto_braces(ic_env_t* env) { + return (env->auto_braces == NULL ? "()[]{}\"\"''" : env->auto_braces); +} + +ic_public void ic_set_default_highlighter(ic_highlight_fun_t* highlighter, void* arg) { + ic_env_t* env = ic_get_env(); if (env==NULL) return; + env->highlighter = highlighter; + env->highlighter_arg = arg; +} + + +ic_public void ic_free( void* p ) { + ic_env_t* env = ic_get_env(); if (env==NULL) return; + mem_free(env->mem, p); +} + +ic_public void* ic_malloc(size_t sz) { + ic_env_t* env = ic_get_env(); if (env==NULL) return NULL; + return mem_malloc(env->mem, to_ssize_t(sz)); +} + +ic_public const char* ic_strdup( const char* s ) { + if (s==NULL) return NULL; + ic_env_t* env = ic_get_env(); if (env==NULL) return NULL; + ssize_t len = ic_strlen(s); + char* p = mem_malloc_tp_n( env->mem, char, len + 1 ); + if (p == NULL) return NULL; + ic_memcpy( p, s, len ); + p[len] = 0; + return p; +} + +//------------------------------------------------------------- +// Terminal +//------------------------------------------------------------- + +ic_public void ic_term_init(void) { + ic_env_t* env = ic_get_env(); if (env==NULL) return; + if (env->term==NULL) return; + term_start_raw(env->term); +} + +ic_public void ic_term_done(void) { + ic_env_t* env = ic_get_env(); if (env==NULL) return; + if (env->term==NULL) return; + term_end_raw(env->term,false); +} + +ic_public void ic_term_flush(void) { + ic_env_t* env = ic_get_env(); if (env==NULL) return; + if (env->term==NULL) return; + term_flush(env->term); +} + +ic_public void ic_term_write(const char* s) { + ic_env_t* env = ic_get_env(); if (env==NULL) return; + if (env->term == NULL) return; + term_write(env->term, s); +} + +ic_public void ic_term_writeln(const char* s) { + ic_env_t* env = ic_get_env(); if (env==NULL) return; + if (env->term == NULL) return; + term_writeln(env->term, s); +} + +ic_public void ic_term_writef(const char* fmt, ...) { + va_list ap; + va_start(ap, fmt); + ic_term_vwritef(fmt, ap); + va_end(ap); +} + +ic_public void ic_term_vwritef(const char* fmt, va_list args) { + ic_env_t* env = ic_get_env(); if (env==NULL) return; + if (env->term == NULL) return; + term_vwritef(env->term, fmt, args); +} + +ic_public void ic_term_reset( void ) { + ic_env_t* env = ic_get_env(); if (env==NULL) return; + if (env->term == NULL) return; + term_attr_reset(env->term); +} + +ic_public void ic_term_style( const char* style ) { + ic_env_t* env = ic_get_env(); if (env==NULL) return; + if (env->term == NULL || env->bbcode == NULL) return; + term_set_attr( env->term, bbcode_style(env->bbcode, style)); +} + +ic_public int ic_term_get_color_bits(void) { + ic_env_t* env = ic_get_env(); + if (env==NULL || env->term==NULL) return 4; + return term_get_color_bits(env->term); +} + +ic_public void ic_term_bold(bool enable) { + ic_env_t* env = ic_get_env(); if (env==NULL || env->term==NULL) return; + term_bold(env->term, enable); +} + +ic_public void ic_term_underline(bool enable) { + ic_env_t* env = ic_get_env(); if (env==NULL || env->term==NULL) return; + term_underline(env->term, enable); +} + +ic_public void ic_term_italic(bool enable) { + ic_env_t* env = ic_get_env(); if (env==NULL || env->term==NULL) return; + term_italic(env->term, enable); +} + +ic_public void ic_term_reverse(bool enable) { + ic_env_t* env = ic_get_env(); if (env==NULL || env->term==NULL) return; + term_reverse(env->term, enable); +} + +ic_public void ic_term_color_ansi(bool foreground, int ansi_color) { + ic_env_t* env = ic_get_env(); if (env==NULL || env->term==NULL) return; + ic_color_t color = color_from_ansi256(ansi_color); + if (foreground) { term_color(env->term, color); } + else { term_bgcolor(env->term, color); } +} + +ic_public void ic_term_color_rgb(bool foreground, uint32_t hcolor) { + ic_env_t* env = ic_get_env(); if (env==NULL || env->term==NULL) return; + ic_color_t color = ic_rgb(hcolor); + if (foreground) { term_color(env->term, color); } + else { term_bgcolor(env->term, color); } +} + + +//------------------------------------------------------------- +// Readline with temporary completer and highlighter +//------------------------------------------------------------- + +ic_public char* ic_readline_ex(const char* prompt_text, + ic_completer_fun_t* completer, void* completer_arg, + ic_highlight_fun_t* highlighter, void* highlighter_arg ) +{ + ic_env_t* env = ic_get_env(); if (env == NULL) return NULL; + // save previous + ic_completer_fun_t* prev_completer; + void* prev_completer_arg; + completions_get_completer(env->completions, &prev_completer, &prev_completer_arg); + ic_highlight_fun_t* prev_highlighter = env->highlighter; + void* prev_highlighter_arg = env->highlighter_arg; + // call with current + if (completer != NULL) { ic_set_default_completer(completer, completer_arg); } + if (highlighter != NULL) { ic_set_default_highlighter(highlighter, highlighter_arg); } + char* res = ic_readline(prompt_text); + // restore previous + ic_set_default_completer(prev_completer, prev_completer_arg); + ic_set_default_highlighter(prev_highlighter, prev_highlighter_arg); + return res; +} + + +//------------------------------------------------------------- +// Initialize +//------------------------------------------------------------- + +static void ic_atexit(void); + +static void ic_env_free(ic_env_t* env) { + if (env == NULL) return; + history_save(env->history); + history_free(env->history); + completions_free(env->completions); + bbcode_free(env->bbcode); + term_free(env->term); + tty_free(env->tty); + mem_free(env->mem, env->cprompt_marker); + mem_free(env->mem,env->prompt_marker); + mem_free(env->mem, env->match_braces); + mem_free(env->mem, env->auto_braces); + env->prompt_marker = NULL; + + // and deallocate ourselves + alloc_t* mem = env->mem; + mem_free(mem, env); + + // and finally the custom memory allocation structure + mem_free(mem, mem); +} + + +static ic_env_t* ic_env_create( ic_malloc_fun_t* _malloc, ic_realloc_fun_t* _realloc, ic_free_fun_t* _free ) +{ + if (_malloc == NULL) _malloc = &malloc; + if (_realloc == NULL) _realloc = &realloc; + if (_free == NULL) _free = &free; + // allocate + alloc_t* mem = (alloc_t*)_malloc(sizeof(alloc_t)); + if (mem == NULL) return NULL; + mem->malloc = _malloc; + mem->realloc = _realloc; + mem->free = _free; + ic_env_t* env = mem_zalloc_tp(mem, ic_env_t); + if (env==NULL) { + mem->free(mem); + return NULL; + } + env->mem = mem; + + // Initialize + env->tty = tty_new(env->mem, -1); // can return NULL + env->term = term_new(env->mem, env->tty, false, false, -1 ); + env->history = history_new(env->mem); + env->completions = completions_new(env->mem); + env->bbcode = bbcode_new(env->mem, env->term); + env->hint_delay = 400; + + if (env->tty == NULL || env->term==NULL || + env->completions == NULL || env->history == NULL || env->bbcode == NULL || + !term_is_interactive(env->term)) + { + env->noedit = true; + } + env->multiline_eol = '\\'; + + bbcode_style_def(env->bbcode, "ic-prompt", "ansi-green" ); + bbcode_style_def(env->bbcode, "ic-info", "ansi-darkgray" ); + bbcode_style_def(env->bbcode, "ic-diminish", "ansi-lightgray" ); + bbcode_style_def(env->bbcode, "ic-emphasis", "#ffffd7" ); + bbcode_style_def(env->bbcode, "ic-hint", "ansi-darkgray" ); + bbcode_style_def(env->bbcode, "ic-error", "#d70000" ); + bbcode_style_def(env->bbcode, "ic-bracematch","ansi-white"); // color = #F7DC6F" ); + + bbcode_style_def(env->bbcode, "keyword", "#569cd6" ); + bbcode_style_def(env->bbcode, "control", "#c586c0" ); + bbcode_style_def(env->bbcode, "number", "#b5cea8" ); + bbcode_style_def(env->bbcode, "string", "#ce9178" ); + bbcode_style_def(env->bbcode, "comment", "#6A9955" ); + bbcode_style_def(env->bbcode, "type", "darkcyan" ); + bbcode_style_def(env->bbcode, "constant", "#569cd6" ); + + set_prompt_marker(env, NULL, NULL); + return env; +} + +static ic_env_t* rpenv; + +static void ic_atexit(void) { + if (rpenv != NULL) { + ic_env_free(rpenv); + rpenv = NULL; + } +} + +ic_private ic_env_t* ic_get_env(void) { + if (rpenv==NULL) { + rpenv = ic_env_create( NULL, NULL, NULL ); + if (rpenv != NULL) { atexit( &ic_atexit ); } + } + return rpenv; +} + +ic_public void ic_init_custom_malloc( ic_malloc_fun_t* _malloc, ic_realloc_fun_t* _realloc, ic_free_fun_t* _free ) { + assert(rpenv == NULL); + if (rpenv != NULL) { + ic_env_free(rpenv); + rpenv = ic_env_create( _malloc, _realloc, _free ); + } + else { + rpenv = ic_env_create( _malloc, _realloc, _free ); + if (rpenv != NULL) { + atexit( &ic_atexit ); + } + } +} + diff --git a/extern/isocline/src/stringbuf.c b/extern/isocline/src/stringbuf.c new file mode 100644 index 00000000..7bbfad04 --- /dev/null +++ b/extern/isocline/src/stringbuf.c @@ -0,0 +1,1038 @@ +/* ---------------------------------------------------------------------------- + Copyright (c) 2021, Daan Leijen + This is free software; you can redistribute it and/or modify it + under the terms of the MIT License. A copy of the license can be + found in the "LICENSE" file at the root of this distribution. +-----------------------------------------------------------------------------*/ + +// get `wcwidth` for the column width of unicode characters +// note: for now the OS provided one is unused as we see quite a bit of variation +// among platforms and including our own seems more reliable. +/* +#if defined(__linux__) || defined(__freebsd__) +// use the system supplied one +#if !defined(_XOPEN_SOURCE) +#define _XOPEN_SOURCE 700 // so wcwidth is visible +#endif +#include +#else +*/ +// use our own (also on APPLE as that fails within vscode) +#define wcwidth(c) mk_wcwidth(c) +#include "wcwidth.c" +// #endif + +#include +#include +#include + +#include "common.h" +#include "stringbuf.h" + +//------------------------------------------------------------- +// In place growable utf-8 strings +//------------------------------------------------------------- + +struct stringbuf_s { + char* buf; + ssize_t buflen; + ssize_t count; + alloc_t* mem; +}; + + +//------------------------------------------------------------- +// String column width +//------------------------------------------------------------- + +// column width of a utf8 single character sequence. +static ssize_t utf8_char_width( const char* s, ssize_t n ) { + if (n <= 0) return 0; + + uint8_t b = (uint8_t)s[0]; + int32_t c; + if (b < ' ') { + return 0; + } + else if (b <= 0x7F) { + return 1; + } + else if (b <= 0xC1) { // invalid continuation byte or invalid 0xC0, 0xC1 (check is strictly not necessary as we don't validate..) + return 1; + } + else if (b <= 0xDF && n >= 2) { // b >= 0xC2 // 2 bytes + c = (((b & 0x1F) << 6) | (s[1] & 0x3F)); + assert(c < 0xD800 || c > 0xDFFF); + int w = wcwidth(c); + return w; + } + else if (b <= 0xEF && n >= 3) { // b >= 0xE0 // 3 bytes + c = (((b & 0x0F) << 12) | ((s[1] & 0x3F) << 6) | (s[2] & 0x3F)); + return wcwidth(c); + } + else if (b <= 0xF4 && n >= 4) { // b >= 0xF0 // 4 bytes + c = (((b & 0x07) << 18) | ((s[1] & 0x3F) << 12) | ((s[2] & 0x3F) << 6) | (s[3] & 0x3F)); + return wcwidth(c); + } + else { + // failed + return 1; + } +} + + +// The column width of a codepoint (0, 1, or 2) +static ssize_t char_column_width( const char* s, ssize_t n ) { + if (s == NULL || n <= 0) return 0; + else if ((uint8_t)(*s) < ' ') return 0; // also for CSI escape sequences + else { + ssize_t w = utf8_char_width(s, n); + #ifdef _WIN32 + return (w <= 0 ? 1 : w); // windows console seems to use at least one column + #else + return w; + #endif + } +} + +static ssize_t str_column_width_n( const char* s, ssize_t len ) { + if (s == NULL || len <= 0) return 0; + ssize_t pos = 0; + ssize_t cwidth = 0; + ssize_t cw; + ssize_t ofs; + while (s[pos] != 0 && (ofs = str_next_ofs(s, len, pos, &cw)) > 0) { + cwidth += cw; + pos += ofs; + } + return cwidth; +} + +ic_private ssize_t str_column_width( const char* s ) { + return str_column_width_n( s, ic_strlen(s) ); +} + +ic_private ssize_t str_skip_until_fit( const char* s, ssize_t max_width ) { + if (s == NULL) return 0; + ssize_t cwidth = str_column_width(s); + ssize_t len = ic_strlen(s); + ssize_t pos = 0; + ssize_t next; + ssize_t cw; + while (cwidth > max_width && (next = str_next_ofs(s, len, pos, &cw)) > 0) { + cwidth -= cw; + pos += next; + } + return pos; +} + +ic_private ssize_t str_take_while_fit( const char* s, ssize_t max_width) { + if (s == NULL) return 0; + const ssize_t len = ic_strlen(s); + ssize_t pos = 0; + ssize_t next; + ssize_t cw; + ssize_t cwidth = 0; + while ((next = str_next_ofs(s, len, pos, &cw)) > 0) { + if (cwidth + cw > max_width) break; + cwidth += cw; + pos += next; + } + return pos; +} + + +//------------------------------------------------------------- +// String navigation +//------------------------------------------------------------- + +// get offset of the previous codepoint. does not skip back over CSI sequences. +ic_private ssize_t str_prev_ofs( const char* s, ssize_t pos, ssize_t* width ) { + ssize_t ofs = 0; + if (s != NULL && pos > 0) { + ofs = 1; + while (pos > ofs) { + uint8_t u = (uint8_t)s[pos - ofs]; + if (u < 0x80 || u > 0xBF) break; // continue while follower + ofs++; + } + } + if (width != NULL) *width = char_column_width( s+(pos-ofs), ofs ); + return ofs; +} + +// skip an escape sequence +// +ic_private bool skip_esc( const char* s, ssize_t len, ssize_t* esclen ) { + if (s == NULL || len <= 1 || s[0] != '\x1B') return false; + if (esclen != NULL) *esclen = 0; + if (strchr("[PX^_]",s[1]) != NULL) { + // CSI (ESC [), DCS (ESC P), SOS (ESC X), PM (ESC ^), APC (ESC _), and OSC (ESC ]): terminated with a special sequence + bool finalCSI = (s[1] == '['); // CSI terminates with 0x40-0x7F; otherwise ST (bell or ESC \) + ssize_t n = 2; + while (len > n) { + char c = s[n++]; + if ((finalCSI && (uint8_t)c >= 0x40 && (uint8_t)c <= 0x7F) || // terminating byte: @A–Z[\]^_`a–z{|}~ + (!finalCSI && c == '\x07') || // bell + (c == '\x02')) // STX terminates as well + { + if (esclen != NULL) *esclen = n; + return true; + } + else if (!finalCSI && c == '\x1B' && len > n && s[n] == '\\') { // ST (ESC \) + n++; + if (esclen != NULL) *esclen = n; + return true; + } + } + } + if (strchr(" #%()*+",s[1]) != NULL) { + // assume escape sequence of length 3 (like ESC % G) + if (esclen != NULL) *esclen = 2; + return true; + } + else { + // assume single character escape code (like ESC 7) + if (esclen != NULL) *esclen = 2; + return true; + } + return false; +} + +// Offset to the next codepoint, treats CSI escape sequences as a single code point. +ic_private ssize_t str_next_ofs( const char* s, ssize_t len, ssize_t pos, ssize_t* cwidth ) { + ssize_t ofs = 0; + if (s != NULL && len > pos) { + if (skip_esc(s+pos,len-pos,&ofs)) { + // skip escape sequence + } + else { + ofs = 1; + // utf8 extended character? + while(len > pos + ofs) { + uint8_t u = (uint8_t)s[pos + ofs]; + if (u < 0x80 || u > 0xBF) break; // break if not a follower + ofs++; + } + } + } + if (cwidth != NULL) *cwidth = char_column_width( s+pos, ofs ); + return ofs; +} + +static ssize_t str_limit_to_length( const char* s, ssize_t n ) { + ssize_t i; + for(i = 0; i < n && s[i] != 0; i++) { /* nothing */ } + return i; +} + + +//------------------------------------------------------------- +// String searching prev/next word, line, ws_word +//------------------------------------------------------------- + + +static ssize_t str_find_backward( const char* s, ssize_t len, ssize_t pos, ic_is_char_class_fun_t* match, bool skip_immediate_matches ) { + if (pos > len) pos = len; + if (pos < 0) pos = 0; + ssize_t i = pos; + // skip matching first (say, whitespace in case of the previous start-of-word) + if (skip_immediate_matches) { + do { + ssize_t prev = str_prev_ofs(s, i, NULL); + if (prev <= 0) break; + assert(i - prev >= 0); + if (!match(s + i - prev, (long)prev)) break; + i -= prev; + } while (i > 0); + } + // find match + do { + ssize_t prev = str_prev_ofs(s, i, NULL); + if (prev <= 0) break; + assert(i - prev >= 0); + if (match(s + i - prev, (long)prev)) { + return i; // found; + } + i -= prev; + } while (i > 0); + return -1; // not found +} + +static ssize_t str_find_forward( const char* s, ssize_t len, ssize_t pos, ic_is_char_class_fun_t* match, bool skip_immediate_matches ) { + if (s == NULL || len < 0) return -1; + if (pos > len) pos = len; + if (pos < 0) pos = 0; + ssize_t i = pos; + ssize_t next; + // skip matching first (say, whitespace in case of the next end-of-word) + if (skip_immediate_matches) { + do { + next = str_next_ofs(s, len, i, NULL); + if (next <= 0) break; + assert( i + next <= len); + if (!match(s + i, (long)next)) break; + i += next; + } while (i < len); + } + // and then look + do { + next = str_next_ofs(s, len, i, NULL); + if (next <= 0) break; + assert( i + next <= len); + if (match(s + i, (long)next)) { + return i; // found + } + i += next; + } while (i < len); + return -1; +} + +static bool char_is_linefeed( const char* s, long n ) { + return (n == 1 && (*s == '\n' || *s == 0)); +} + +static ssize_t str_find_line_start( const char* s, ssize_t len, ssize_t pos) { + ssize_t start = str_find_backward(s,len,pos,&char_is_linefeed,false /* don't skip immediate matches */); + return (start < 0 ? 0 : start); +} + +static ssize_t str_find_line_end( const char* s, ssize_t len, ssize_t pos) { + ssize_t end = str_find_forward(s,len,pos, &char_is_linefeed, false); + return (end < 0 ? len : end); +} + +static ssize_t str_find_word_start( const char* s, ssize_t len, ssize_t pos) { + ssize_t start = str_find_backward(s,len,pos, &ic_char_is_idletter,true /* skip immediate matches */); + return (start < 0 ? 0 : start); +} + +static ssize_t str_find_word_end( const char* s, ssize_t len, ssize_t pos) { + ssize_t end = str_find_forward(s,len,pos,&ic_char_is_idletter,true /* skip immediate matches */); + return (end < 0 ? len : end); +} + +static ssize_t str_find_ws_word_start( const char* s, ssize_t len, ssize_t pos) { + ssize_t start = str_find_backward(s,len,pos,&ic_char_is_white,true /* skip immediate matches */); + return (start < 0 ? 0 : start); +} + +static ssize_t str_find_ws_word_end( const char* s, ssize_t len, ssize_t pos) { + ssize_t end = str_find_forward(s,len,pos,&ic_char_is_white,true /* skip immediate matches */); + return (end < 0 ? len : end); +} + + +//------------------------------------------------------------- +// String row/column iteration +//------------------------------------------------------------- + +// invoke a function for each terminal row; returns total row count. +static ssize_t str_for_each_row( const char* s, ssize_t len, ssize_t termw, ssize_t promptw, ssize_t cpromptw, + row_fun_t* fun, const void* arg, void* res ) +{ + if (s == NULL) s = ""; + ssize_t i; + ssize_t rcount = 0; + ssize_t rcol = 0; + ssize_t rstart = 0; + ssize_t startw = promptw; + for(i = 0; i < len; ) { + ssize_t w; + ssize_t next = str_next_ofs(s, len, i, &w); + if (next <= 0) { + debug_msg("str: foreach row: next<=0: len %zd, i %zd, w %zd, buf %s\n", len, i, w, s ); + assert(false); + break; + } + startw = (rcount == 0 ? promptw : cpromptw); + ssize_t termcol = rcol + w + startw + 1 /* for the cursor */; + if (termw != 0 && i != 0 && termcol >= termw) { + // wrap + if (fun != NULL) { + if (fun(s,rcount,rstart,i - rstart,startw,true,arg,res)) return rcount; + } + rcount++; + rstart = i; + rcol = 0; + } + if (s[i] == '\n') { + // newline + if (fun != NULL) { + if (fun(s,rcount,rstart,i - rstart,startw,false,arg,res)) return rcount; + } + rcount++; + rstart = i+1; + rcol = 0; + } + assert (s[i] != 0); + i += next; + rcol += w; + } + if (fun != NULL) { + if (fun(s,rcount,rstart,i - rstart,startw,false,arg,res)) return rcount; + } + return rcount+1; +} + +//------------------------------------------------------------- +// String: get row/column position +//------------------------------------------------------------- + + +static bool str_get_current_pos_iter( + const char* s, + ssize_t row, ssize_t row_start, ssize_t row_len, + ssize_t startw, bool is_wrap, const void* arg, void* res) +{ + ic_unused(is_wrap); ic_unused(startw); + rowcol_t* rc = (rowcol_t*)res; + ssize_t pos = *((ssize_t*)arg); + + if (pos >= row_start && pos <= (row_start + row_len)) { + // found the cursor row + rc->row_start = row_start; + rc->row_len = row_len; + rc->row = row; + rc->col = str_column_width_n( s + row_start, pos - row_start ); + rc->first_on_row = (pos == row_start); + if (is_wrap) { + // if wrapped, we check if the next character is at row_len + ssize_t next = str_next_ofs(s, row_start + row_len, pos, NULL); + rc->last_on_row = (pos + next >= row_start + row_len); + } + else { + // normal last position is right after the last character + rc->last_on_row = (pos >= row_start + row_len); + } + // debug_msg("edit; pos iter: pos: %zd (%c), row_start: %zd, rowlen: %zd\n", pos, s[pos], row_start, row_len); + } + return false; // always continue to count all rows +} + +static ssize_t str_get_rc_at_pos(const char* s, ssize_t len, ssize_t termw, ssize_t promptw, ssize_t cpromptw, ssize_t pos, rowcol_t* rc) { + memset(rc, 0, sizeof(*rc)); + ssize_t rows = str_for_each_row(s, len, termw, promptw, cpromptw, &str_get_current_pos_iter, &pos, rc); + // debug_msg("edit: current pos: (%d, %d) %s %s\n", rc->row, rc->col, rc->first_on_row ? "first" : "", rc->last_on_row ? "last" : ""); + return rows; +} + + + +//------------------------------------------------------------- +// String: get row/column position for a resized terminal +// with potentially "hard-wrapped" rows +//------------------------------------------------------------- +typedef struct wrapped_arg_s { + ssize_t pos; + ssize_t newtermw; +} wrapped_arg_t; + +typedef struct wrowcol_s { + rowcol_t rc; + ssize_t hrows; // count of hard-wrapped extra rows +} wrowcol_t; + +static bool str_get_current_wrapped_pos_iter( + const char* s, + ssize_t row, ssize_t row_start, ssize_t row_len, + ssize_t startw, bool is_wrap, const void* arg, void* res) +{ + ic_unused(is_wrap); + wrowcol_t* wrc = (wrowcol_t*)res; + const wrapped_arg_t* warg = (const wrapped_arg_t*)arg; + + // iterate through the row and record the postion and hard-wraps + ssize_t hwidth = startw; + ssize_t i = 0; + while( i <= row_len ) { // include rowlen as the cursor position can be just after the last character + // get next position and column width + ssize_t cw; + ssize_t next; + bool is_cursor = (warg->pos == row_start+i); + if (i < row_len) { + next = str_next_ofs(s + row_start, row_len, i, &cw); + } + else { + // end of row: take wrap or cursor into account + // (wrap has width 2 as it displays a back-arrow but also has an invisible newline that wraps) + cw = (is_wrap ? 2 : (is_cursor ? 1 : 0)); + next = 1; + } + + if (next > 0) { + if (hwidth + cw > warg->newtermw) { + // hardwrap + hwidth = 0; + wrc->hrows++; + debug_msg("str: found hardwrap: row: %zd, hrows: %zd\n", row, wrc->hrows); + } + } + else { + next++; // ensure we terminate (as we go up to rowlen) + } + + // did we find our position? + if (is_cursor) { + debug_msg("str: found position: row: %zd, hrows: %zd\n", row, wrc->hrows); + wrc->rc.row_start = row_start; + wrc->rc.row_len = row_len; + wrc->rc.row = wrc->hrows + row; + wrc->rc.col = hwidth; + wrc->rc.first_on_row = (i==0); + wrc->rc.last_on_row = (i+next >= row_len - (is_wrap ? 1 : 0)); + } + + // advance + hwidth += cw; + i += next; + } + return false; // always continue to count all rows +} + + +static ssize_t str_get_wrapped_rc_at_pos(const char* s, ssize_t len, ssize_t termw, ssize_t newtermw, ssize_t promptw, ssize_t cpromptw, ssize_t pos, rowcol_t* rc) { + wrapped_arg_t warg; + warg.pos = pos; + warg.newtermw = newtermw; + wrowcol_t wrc; + memset(&wrc,0,sizeof(wrc)); + ssize_t rows = str_for_each_row(s, len, termw, promptw, cpromptw, &str_get_current_wrapped_pos_iter, &warg, &wrc); + debug_msg("edit: wrapped pos: (%zd,%zd) rows %zd %s %s, hrows: %zd\n", wrc.rc.row, wrc.rc.col, rows, wrc.rc.first_on_row ? "first" : "", wrc.rc.last_on_row ? "last" : "", wrc.hrows); + *rc = wrc.rc; + return (rows + wrc.hrows); +} + + +//------------------------------------------------------------- +// Set position +//------------------------------------------------------------- + +static bool str_set_pos_iter( + const char* s, + ssize_t row, ssize_t row_start, ssize_t row_len, + ssize_t startw, bool is_wrap, const void* arg, void* res) +{ + ic_unused(arg); ic_unused(is_wrap); ic_unused(startw); + rowcol_t* rc = (rowcol_t*)arg; + if (rc->row != row) return false; // keep searching + // we found our row + ssize_t col = 0; + ssize_t i = row_start; + ssize_t end = row_start + row_len; + while (col < rc->col && i < end) { + ssize_t cw; + ssize_t next = str_next_ofs(s, row_start + row_len, i, &cw); + if (next <= 0) break; + i += next; + col += cw; + } + *((ssize_t*)res) = i; + return true; // stop iteration +} + +static ssize_t str_get_pos_at_rc(const char* s, ssize_t len, ssize_t termw, ssize_t promptw, ssize_t cpromptw, ssize_t row, ssize_t col /* without prompt */) { + rowcol_t rc; + memset(&rc,0,ssizeof(rc)); + rc.row = row; + rc.col = col; + ssize_t pos = -1; + str_for_each_row(s,len,termw,promptw,cpromptw,&str_set_pos_iter,&rc,&pos); + return pos; +} + + +//------------------------------------------------------------- +// String buffer +//------------------------------------------------------------- +static bool sbuf_ensure_extra(stringbuf_t* s, ssize_t extra) +{ + if (s->buflen >= s->count + extra) return true; + // reallocate; pick good initial size and multiples to increase reuse on allocation + ssize_t newlen = (s->buflen <= 0 ? 120 : (s->buflen > 1000 ? s->buflen + 1000 : 2*s->buflen)); + if (newlen < s->count + extra) newlen = s->count + extra; + if (s->buflen > 0) { + debug_msg("stringbuf: reallocate: old %zd, new %zd\n", s->buflen, newlen); + } + char* newbuf = mem_realloc_tp(s->mem, char, s->buf, newlen+1); // one more for terminating zero + if (newbuf == NULL) { + assert(false); + return false; + } + s->buf = newbuf; + s->buflen = newlen; + s->buf[s->count] = s->buf[s->buflen] = 0; + assert(s->buflen >= s->count + extra); + return true; +} + +static void sbuf_init( stringbuf_t* sbuf, alloc_t* mem ) { + sbuf->mem = mem; + sbuf->buf = NULL; + sbuf->buflen = 0; + sbuf->count = 0; +} + +static void sbuf_done( stringbuf_t* sbuf ) { + mem_free( sbuf->mem, sbuf->buf ); + sbuf->buf = NULL; + sbuf->buflen = 0; + sbuf->count = 0; +} + + +ic_private void sbuf_free( stringbuf_t* sbuf ) { + if (sbuf==NULL) return; + sbuf_done(sbuf); + mem_free(sbuf->mem, sbuf); +} + +ic_private stringbuf_t* sbuf_new( alloc_t* mem ) { + stringbuf_t* sbuf = mem_zalloc_tp(mem,stringbuf_t); + if (sbuf == NULL) return NULL; + sbuf_init(sbuf,mem); + return sbuf; +} + +// free the sbuf and return the current string buffer as the result +ic_private char* sbuf_free_dup(stringbuf_t* sbuf) { + if (sbuf == NULL) return NULL; + char* s = NULL; + if (sbuf->buf != NULL) { + s = mem_realloc_tp(sbuf->mem, char, sbuf->buf, sbuf_len(sbuf)+1); + if (s == NULL) { s = sbuf->buf; } + sbuf->buf = 0; + sbuf->buflen = 0; + sbuf->count = 0; + } + sbuf_free(sbuf); + return s; +} + +ic_private const char* sbuf_string_at( stringbuf_t* sbuf, ssize_t pos ) { + if (pos < 0 || sbuf->count < pos) return NULL; + if (sbuf->buf == NULL) return ""; + assert(sbuf->buf[sbuf->count] == 0); + return sbuf->buf + pos; +} + +ic_private const char* sbuf_string( stringbuf_t* sbuf ) { + return sbuf_string_at( sbuf, 0 ); +} + +ic_private char sbuf_char_at(stringbuf_t* sbuf, ssize_t pos) { + if (sbuf->buf == NULL || pos < 0 || sbuf->count < pos) return 0; + return sbuf->buf[pos]; +} + +ic_private char* sbuf_strdup_at( stringbuf_t* sbuf, ssize_t pos ) { + return mem_strdup(sbuf->mem, sbuf_string_at(sbuf,pos)); +} + +ic_private char* sbuf_strdup( stringbuf_t* sbuf ) { + return mem_strdup(sbuf->mem, sbuf_string(sbuf)); +} + +ic_private ssize_t sbuf_len(const stringbuf_t* s) { + if (s == NULL) return 0; + return s->count; +} + +ic_private ssize_t sbuf_append_vprintf(stringbuf_t* sb, const char* fmt, va_list args) { + const ssize_t min_needed = ic_strlen(fmt); + if (!sbuf_ensure_extra(sb,min_needed + 16)) return sb->count; + ssize_t avail = sb->buflen - sb->count; + va_list args0; + va_copy(args0, args); + ssize_t needed = vsnprintf(sb->buf + sb->count, to_size_t(avail), fmt, args0); + if (needed > avail) { + sb->buf[sb->count] = 0; + if (!sbuf_ensure_extra(sb, needed)) return sb->count; + avail = sb->buflen - sb->count; + needed = vsnprintf(sb->buf + sb->count, to_size_t(avail), fmt, args); + } + assert(needed <= avail); + sb->count += (needed > avail ? avail : (needed >= 0 ? needed : 0)); + assert(sb->count <= sb->buflen); + sb->buf[sb->count] = 0; + return sb->count; +} + +ic_private ssize_t sbuf_appendf(stringbuf_t* sb, const char* fmt, ...) { + va_list args; + va_start( args, fmt); + ssize_t res = sbuf_append_vprintf( sb, fmt, args ); + va_end(args); + return res; +} + + +ic_private ssize_t sbuf_insert_at_n(stringbuf_t* sbuf, const char* s, ssize_t n, ssize_t pos ) { + if (pos < 0 || pos > sbuf->count || s == NULL) return pos; + n = str_limit_to_length(s,n); + if (n <= 0 || !sbuf_ensure_extra(sbuf,n)) return pos; + ic_memmove(sbuf->buf + pos + n, sbuf->buf + pos, sbuf->count - pos); + ic_memcpy(sbuf->buf + pos, s, n); + sbuf->count += n; + sbuf->buf[sbuf->count] = 0; + return (pos + n); +} + +ic_private stringbuf_t* sbuf_split_at( stringbuf_t* sb, ssize_t pos ) { + stringbuf_t* res = sbuf_new(sb->mem); + if (res==NULL || pos < 0) return NULL; + if (pos < sb->count) { + sbuf_append_n(res, sb->buf + pos, sb->count - pos); + sb->count = pos; + } + return res; +} + +ic_private ssize_t sbuf_insert_at(stringbuf_t* sbuf, const char* s, ssize_t pos ) { + return sbuf_insert_at_n( sbuf, s, ic_strlen(s), pos ); +} + +ic_private ssize_t sbuf_insert_char_at(stringbuf_t* sbuf, char c, ssize_t pos ) { + char s[2]; + s[0] = c; + s[1] = 0; + return sbuf_insert_at_n( sbuf, s, 1, pos); +} + +ic_private ssize_t sbuf_insert_unicode_at(stringbuf_t* sbuf, unicode_t u, ssize_t pos) { + uint8_t s[5]; + unicode_to_qutf8(u, s); + return sbuf_insert_at(sbuf, (const char*)s, pos); +} + + + +ic_private void sbuf_delete_at( stringbuf_t* sbuf, ssize_t pos, ssize_t count ) { + if (pos < 0 || pos >= sbuf->count) return; + if (pos + count > sbuf->count) count = sbuf->count - pos; + ic_memmove(sbuf->buf + pos, sbuf->buf + pos + count, sbuf->count - pos - count); + sbuf->count -= count; + sbuf->buf[sbuf->count] = 0; +} + +ic_private void sbuf_delete_from_to( stringbuf_t* sbuf, ssize_t pos, ssize_t end ) { + if (end <= pos) return; + sbuf_delete_at( sbuf, pos, end - pos); +} + +ic_private void sbuf_delete_from(stringbuf_t* sbuf, ssize_t pos ) { + sbuf_delete_at(sbuf, pos, sbuf_len(sbuf) - pos ); +} + + +ic_private void sbuf_clear( stringbuf_t* sbuf ) { + sbuf_delete_at(sbuf, 0, sbuf_len(sbuf)); +} + +ic_private ssize_t sbuf_append_n( stringbuf_t* sbuf, const char* s, ssize_t n ) { + return sbuf_insert_at_n( sbuf, s, n, sbuf_len(sbuf)); +} + +ic_private ssize_t sbuf_append( stringbuf_t* sbuf, const char* s ) { + return sbuf_insert_at( sbuf, s, sbuf_len(sbuf)); +} + +ic_private ssize_t sbuf_append_char( stringbuf_t* sbuf, char c ) { + char buf[2]; + buf[0] = c; + buf[1] = 0; + return sbuf_append( sbuf, buf ); +} + +ic_private void sbuf_replace(stringbuf_t* sbuf, const char* s) { + sbuf_clear(sbuf); + sbuf_append(sbuf,s); +} + +ic_private ssize_t sbuf_next_ofs( stringbuf_t* sbuf, ssize_t pos, ssize_t* cwidth ) { + return str_next_ofs( sbuf->buf, sbuf->count, pos, cwidth); +} + +ic_private ssize_t sbuf_prev_ofs( stringbuf_t* sbuf, ssize_t pos, ssize_t* cwidth ) { + return str_prev_ofs( sbuf->buf, pos, cwidth); +} + +ic_private ssize_t sbuf_next( stringbuf_t* sbuf, ssize_t pos, ssize_t* cwidth) { + ssize_t ofs = sbuf_next_ofs(sbuf,pos,cwidth); + if (ofs <= 0) return -1; + assert(pos + ofs <= sbuf->count); + return pos + ofs; +} + +ic_private ssize_t sbuf_prev( stringbuf_t* sbuf, ssize_t pos, ssize_t* cwidth) { + ssize_t ofs = sbuf_prev_ofs(sbuf,pos,cwidth); + if (ofs <= 0) return -1; + assert(pos - ofs >= 0); + return pos - ofs; +} + +ic_private ssize_t sbuf_delete_char_before( stringbuf_t* sbuf, ssize_t pos ) { + ssize_t n = sbuf_prev_ofs(sbuf, pos, NULL); + if (n <= 0) return 0; + assert( pos - n >= 0 ); + sbuf_delete_at(sbuf, pos - n, n); + return pos - n; +} + +ic_private void sbuf_delete_char_at( stringbuf_t* sbuf, ssize_t pos ) { + ssize_t n = sbuf_next_ofs(sbuf, pos, NULL); + if (n <= 0) return; + assert( pos + n <= sbuf->count ); + sbuf_delete_at(sbuf, pos, n); + return; +} + +ic_private ssize_t sbuf_swap_char( stringbuf_t* sbuf, ssize_t pos ) { + ssize_t next = sbuf_next_ofs(sbuf, pos, NULL); + if (next <= 0) return 0; + ssize_t prev = sbuf_prev_ofs(sbuf, pos, NULL); + if (prev <= 0) return 0; + char buf[64]; + if (prev >= 63) return 0; + ic_memcpy(buf, sbuf->buf + pos - prev, prev ); + ic_memmove(sbuf->buf + pos - prev, sbuf->buf + pos, next); + ic_memmove(sbuf->buf + pos - prev + next, buf, prev); + return pos - prev; +} + +ic_private ssize_t sbuf_find_line_start( stringbuf_t* sbuf, ssize_t pos ) { + return str_find_line_start( sbuf->buf, sbuf->count, pos); +} + +ic_private ssize_t sbuf_find_line_end( stringbuf_t* sbuf, ssize_t pos ) { + return str_find_line_end( sbuf->buf, sbuf->count, pos); +} + +ic_private ssize_t sbuf_find_word_start( stringbuf_t* sbuf, ssize_t pos ) { + return str_find_word_start( sbuf->buf, sbuf->count, pos); +} + +ic_private ssize_t sbuf_find_word_end( stringbuf_t* sbuf, ssize_t pos ) { + return str_find_word_end( sbuf->buf, sbuf->count, pos); +} + +ic_private ssize_t sbuf_find_ws_word_start( stringbuf_t* sbuf, ssize_t pos ) { + return str_find_ws_word_start( sbuf->buf, sbuf->count, pos); +} + +ic_private ssize_t sbuf_find_ws_word_end( stringbuf_t* sbuf, ssize_t pos ) { + return str_find_ws_word_end( sbuf->buf, sbuf->count, pos); +} + +// find row/col position +ic_private ssize_t sbuf_get_pos_at_rc( stringbuf_t* sbuf, ssize_t termw, ssize_t promptw, ssize_t cpromptw, ssize_t row, ssize_t col ) { + return str_get_pos_at_rc( sbuf->buf, sbuf->count, termw, promptw, cpromptw, row, col); +} + +// get row/col for a given position +ic_private ssize_t sbuf_get_rc_at_pos( stringbuf_t* sbuf, ssize_t termw, ssize_t promptw, ssize_t cpromptw, ssize_t pos, rowcol_t* rc ) { + return str_get_rc_at_pos( sbuf->buf, sbuf->count, termw, promptw, cpromptw, pos, rc); +} + +ic_private ssize_t sbuf_get_wrapped_rc_at_pos( stringbuf_t* sbuf, ssize_t termw, ssize_t newtermw, ssize_t promptw, ssize_t cpromptw, ssize_t pos, rowcol_t* rc ) { + return str_get_wrapped_rc_at_pos( sbuf->buf, sbuf->count, termw, newtermw, promptw, cpromptw, pos, rc); +} + +ic_private ssize_t sbuf_for_each_row( stringbuf_t* sbuf, ssize_t termw, ssize_t promptw, ssize_t cpromptw, row_fun_t* fun, void* arg, void* res ) { + if (sbuf == NULL) return 0; + return str_for_each_row( sbuf->buf, sbuf->count, termw, promptw, cpromptw, fun, arg, res); +} + + +// Duplicate and decode from utf-8 (for non-utf8 terminals) +ic_private char* sbuf_strdup_from_utf8(stringbuf_t* sbuf) { + ssize_t len = sbuf_len(sbuf); + if (sbuf == NULL || len <= 0) return NULL; + char* s = mem_zalloc_tp_n(sbuf->mem, char, len); + if (s == NULL) return NULL; + ssize_t dest = 0; + for (ssize_t i = 0; i < len; ) { + ssize_t ofs = sbuf_next_ofs(sbuf, i, NULL); + if (ofs <= 0) { + // invalid input + break; + } + else if (ofs == 1) { + // regular character + s[dest++] = sbuf->buf[i]; + } + else if (sbuf->buf[i] == '\x1B') { + // skip escape sequences + } + else { + // decode unicode + ssize_t nread; + unicode_t uchr = unicode_from_qutf8( (const uint8_t*)(sbuf->buf + i), ofs, &nread); + uint8_t c; + if (unicode_is_raw(uchr, &c)) { + // raw byte, output as is (this will take care of locale specific input) + s[dest++] = (char)c; + } + else if (uchr <= 0x7F) { + // allow ascii + s[dest++] = (char)uchr; + } + else { + // skip unknown unicode characters.. + // todo: convert according to locale? + } + } + i += ofs; + } + assert(dest <= len); + s[dest] = 0; + return s; +} + +//------------------------------------------------------------- +// String helpers +//------------------------------------------------------------- + +ic_public long ic_prev_char( const char* s, long pos ) { + ssize_t len = ic_strlen(s); + if (pos < 0 || pos > len) return -1; + ssize_t ofs = str_prev_ofs( s, pos, NULL ); + if (ofs <= 0) return -1; + return (long)(pos - ofs); +} + +ic_public long ic_next_char( const char* s, long pos ) { + ssize_t len = ic_strlen(s); + if (pos < 0 || pos > len) return -1; + ssize_t ofs = str_next_ofs( s, len, pos, NULL ); + if (ofs <= 0) return -1; + return (long)(pos + ofs); +} + + +// parse a decimal (leave pi unchanged on error) +ic_private bool ic_atoz(const char* s, ssize_t* pi) { + return (sscanf(s, "%zd", pi) == 1); +} + +// parse two decimals separated by a semicolon +ic_private bool ic_atoz2(const char* s, ssize_t* pi, ssize_t* pj) { + return (sscanf(s, "%zd;%zd", pi, pj) == 2); +} + +// parse unsigned 32-bit (leave pu unchanged on error) +ic_private bool ic_atou32(const char* s, uint32_t* pu) { + return (sscanf(s, "%" SCNu32, pu) == 1); +} + + +// Convenience: character class for whitespace `[ \t\r\n]`. +ic_public bool ic_char_is_white(const char* s, long len) { + if (s == NULL || len != 1) return false; + const char c = *s; + return (c==' ' || c == '\t' || c == '\n' || c == '\r'); +} + +// Convenience: character class for non-whitespace `[^ \t\r\n]`. +ic_public bool ic_char_is_nonwhite(const char* s, long len) { + return !ic_char_is_white(s, len); +} + +// Convenience: character class for separators `[ \t\r\n,.;:/\\\(\)\{\}\[\]]`. +ic_public bool ic_char_is_separator(const char* s, long len) { + if (s == NULL || len != 1) return false; + const char c = *s; + return (strchr(" \t\r\n,.;:/\\(){}[]", c) != NULL); +} + +// Convenience: character class for non-separators. +ic_public bool ic_char_is_nonseparator(const char* s, long len) { + return !ic_char_is_separator(s, len); +} + + +// Convenience: character class for digits (`[0-9]`). +ic_public bool ic_char_is_digit(const char* s, long len) { + if (s == NULL || len != 1) return false; + const char c = *s; + return (c >= '0' && c <= '9'); +} + +// Convenience: character class for hexadecimal digits (`[A-Fa-f0-9]`). +ic_public bool ic_char_is_hexdigit(const char* s, long len) { + if (s == NULL || len != 1) return false; + const char c = *s; + return ((c >= '0' && c <= '9') || (c >= 'a' && c <= 'f') || (c >= 'A' && c <= 'F')); +} + +// Convenience: character class for letters (`[A-Za-z]` and any unicode > 0x80). +ic_public bool ic_char_is_letter(const char* s, long len) { + if (s == NULL || len <= 0) return false; + const char c = *s; + return ((uint8_t)c >= 0x80 || (c >= 'A' && c <= 'Z') || (c >= 'a' && c <= 'z')); +} + +// Convenience: character class for identifier letters (`[A-Za-z0-9_-]` and any unicode > 0x80). +ic_public bool ic_char_is_idletter(const char* s, long len) { + if (s == NULL || len <= 0) return false; + const char c = *s; + return ((uint8_t)c >= 0x80 || (c >= 'A' && c <= 'Z') || (c >= 'a' && c <= 'z') || (c >= '0' && c <= '9') || (c == '_') || (c == '-')); +} + +// Convenience: character class for filename letters (`[^ \t\r\n`@$><=;|&{(]`). +ic_public bool ic_char_is_filename_letter(const char* s, long len) { + if (s == NULL || len <= 0) return false; + const char c = *s; + return ((uint8_t)c >= 0x80 || (strchr(" \t\r\n`@$><=;|&{}()[]", c) == NULL)); +} + +// Convenience: If this is a token start, returns the length (or <= 0 if not found). +ic_public long ic_is_token(const char* s, long pos, ic_is_char_class_fun_t* is_token_char) { + if (s == NULL || pos < 0 || is_token_char == NULL) return -1; + ssize_t len = ic_strlen(s); + if (pos >= len) return -1; + if (pos > 0 && is_token_char(s + pos -1, 1)) return -1; // token start? + ssize_t i = pos; + while ( i < len ) { + ssize_t next = str_next_ofs(s, len, i, NULL); + if (next <= 0) return -1; + if (!is_token_char(s + i, (long)next)) break; + i += next; + } + return (long)(i - pos); +} + + +static int ic_strncmp(const char* s1, const char* s2, ssize_t n) { + return strncmp(s1, s2, to_size_t(n)); +} + +// Convenience: Does this match the specified token? +// Ensures not to match prefixes or suffixes, and returns the length of the match (in bytes). +// E.g. `ic_match_token("function",0,&ic_char_is_letter,"fun")` returns 0. +ic_public long ic_match_token(const char* s, long pos, ic_is_char_class_fun_t* is_token_char, const char* token) { + long n = ic_is_token(s, pos, is_token_char); + if (n > 0 && token != NULL && n == ic_strlen(token) && ic_strncmp(s + pos, token, n) == 0) { + return n; + } + else { + return 0; + } +} + + +// Convenience: Do any of the specified tokens match? +// Ensures not to match prefixes or suffixes, and returns the length of the match (in bytes). +// Ensures not to match prefixes or suffixes. +// E.g. `ic_match_any_token("function",0,&ic_char_is_letter,{"fun","func",NULL})` returns 0. +ic_public long ic_match_any_token(const char* s, long pos, ic_is_char_class_fun_t* is_token_char, const char** tokens) { + long n = ic_is_token(s, pos, is_token_char); + if (n <= 0 || tokens == NULL) return 0; + for (const char** token = tokens; *token != NULL; token++) { + if (n == ic_strlen(*token) && ic_strncmp(s + pos, *token, n) == 0) { + return n; + } + } + return 0; +} + diff --git a/extern/isocline/src/stringbuf.h b/extern/isocline/src/stringbuf.h new file mode 100644 index 00000000..39b21ea4 --- /dev/null +++ b/extern/isocline/src/stringbuf.h @@ -0,0 +1,121 @@ +/* ---------------------------------------------------------------------------- + Copyright (c) 2021, Daan Leijen + This is free software; you can redistribute it and/or modify it + under the terms of the MIT License. A copy of the license can be + found in the "LICENSE" file at the root of this distribution. +-----------------------------------------------------------------------------*/ +#pragma once +#ifndef IC_STRINGBUF_H +#define IC_STRINGBUF_H + +#include +#include "common.h" + +//------------------------------------------------------------- +// string buffer +// in-place modified buffer with edit operations +// that grows on demand. +//------------------------------------------------------------- + +// abstract string buffer +struct stringbuf_s; +typedef struct stringbuf_s stringbuf_t; + +ic_private stringbuf_t* sbuf_new( alloc_t* mem ); +ic_private void sbuf_free( stringbuf_t* sbuf ); +ic_private char* sbuf_free_dup(stringbuf_t* sbuf); +ic_private ssize_t sbuf_len(const stringbuf_t* s); + +ic_private const char* sbuf_string_at( stringbuf_t* sbuf, ssize_t pos ); +ic_private const char* sbuf_string( stringbuf_t* sbuf ); +ic_private char sbuf_char_at(stringbuf_t* sbuf, ssize_t pos); +ic_private char* sbuf_strdup_at( stringbuf_t* sbuf, ssize_t pos ); +ic_private char* sbuf_strdup( stringbuf_t* sbuf ); +ic_private char* sbuf_strdup_from_utf8(stringbuf_t* sbuf); // decode to locale + + +ic_private ssize_t sbuf_appendf(stringbuf_t* sb, const char* fmt, ...); +ic_private ssize_t sbuf_append_vprintf(stringbuf_t* sb, const char* fmt, va_list args); + +ic_private stringbuf_t* sbuf_split_at( stringbuf_t* sb, ssize_t pos ); + +// primitive edit operations (inserts return the new position) +ic_private void sbuf_clear(stringbuf_t* sbuf); +ic_private void sbuf_replace(stringbuf_t* sbuf, const char* s); +ic_private void sbuf_delete_at(stringbuf_t* sbuf, ssize_t pos, ssize_t count); +ic_private void sbuf_delete_from_to(stringbuf_t* sbuf, ssize_t pos, ssize_t end); +ic_private void sbuf_delete_from(stringbuf_t* sbuf, ssize_t pos ); +ic_private ssize_t sbuf_insert_at_n(stringbuf_t* sbuf, const char* s, ssize_t n, ssize_t pos ); +ic_private ssize_t sbuf_insert_at(stringbuf_t* sbuf, const char* s, ssize_t pos ); +ic_private ssize_t sbuf_insert_char_at(stringbuf_t* sbuf, char c, ssize_t pos ); +ic_private ssize_t sbuf_insert_unicode_at(stringbuf_t* sbuf, unicode_t u, ssize_t pos); +ic_private ssize_t sbuf_append_n(stringbuf_t* sbuf, const char* s, ssize_t n); +ic_private ssize_t sbuf_append(stringbuf_t* sbuf, const char* s); +ic_private ssize_t sbuf_append_char(stringbuf_t* sbuf, char c); + +// high level edit operations (return the new position) +ic_private ssize_t sbuf_next( stringbuf_t* sbuf, ssize_t pos, ssize_t* cwidth ); +ic_private ssize_t sbuf_prev( stringbuf_t* sbuf, ssize_t pos, ssize_t* cwidth ); +ic_private ssize_t sbuf_next_ofs(stringbuf_t* sbuf, ssize_t pos, ssize_t* cwidth); + +ic_private ssize_t sbuf_delete_char_before( stringbuf_t* sbuf, ssize_t pos ); +ic_private void sbuf_delete_char_at( stringbuf_t* sbuf, ssize_t pos ); +ic_private ssize_t sbuf_swap_char( stringbuf_t* sbuf, ssize_t pos ); + +ic_private ssize_t sbuf_find_line_start( stringbuf_t* sbuf, ssize_t pos ); +ic_private ssize_t sbuf_find_line_end( stringbuf_t* sbuf, ssize_t pos ); +ic_private ssize_t sbuf_find_word_start( stringbuf_t* sbuf, ssize_t pos ); +ic_private ssize_t sbuf_find_word_end( stringbuf_t* sbuf, ssize_t pos ); +ic_private ssize_t sbuf_find_ws_word_start( stringbuf_t* sbuf, ssize_t pos ); +ic_private ssize_t sbuf_find_ws_word_end( stringbuf_t* sbuf, ssize_t pos ); + +// parse a decimal +ic_private bool ic_atoz(const char* s, ssize_t* i); +// parse two decimals separated by a semicolon +ic_private bool ic_atoz2(const char* s, ssize_t* i, ssize_t* j); +ic_private bool ic_atou32(const char* s, uint32_t* pu); + +// row/column info +typedef struct rowcol_s { + ssize_t row; + ssize_t col; + ssize_t row_start; + ssize_t row_len; + bool first_on_row; + bool last_on_row; +} rowcol_t; + +// find row/col position +ic_private ssize_t sbuf_get_pos_at_rc( stringbuf_t* sbuf, ssize_t termw, ssize_t promptw, ssize_t cpromptw, + ssize_t row, ssize_t col ); +// get row/col for a given position +ic_private ssize_t sbuf_get_rc_at_pos( stringbuf_t* sbuf, ssize_t termw, ssize_t promptw, ssize_t cpromptw, + ssize_t pos, rowcol_t* rc ); + +ic_private ssize_t sbuf_get_wrapped_rc_at_pos( stringbuf_t* sbuf, ssize_t termw, ssize_t newtermw, ssize_t promptw, ssize_t cpromptw, + ssize_t pos, rowcol_t* rc ); + +// row iteration +typedef bool (row_fun_t)(const char* s, + ssize_t row, ssize_t row_start, ssize_t row_len, + ssize_t startw, // prompt width + bool is_wrap, const void* arg, void* res); + +ic_private ssize_t sbuf_for_each_row( stringbuf_t* sbuf, ssize_t termw, ssize_t promptw, ssize_t cpromptw, + row_fun_t* fun, void* arg, void* res ); + + +//------------------------------------------------------------- +// Strings +//------------------------------------------------------------- + +// skip a single CSI sequence (ESC [ ...) +ic_private bool skip_csi_esc( const char* s, ssize_t len, ssize_t* esclen ); // used in term.c + +ic_private ssize_t str_column_width( const char* s ); +ic_private ssize_t str_prev_ofs( const char* s, ssize_t pos, ssize_t* cwidth ); +ic_private ssize_t str_next_ofs( const char* s, ssize_t len, ssize_t pos, ssize_t* cwidth ); +ic_private ssize_t str_skip_until_fit( const char* s, ssize_t max_width); // tail that fits +ic_private ssize_t str_take_while_fit( const char* s, ssize_t max_width); // prefix that fits + +#endif // IC_STRINGBUF_H diff --git a/extern/isocline/src/term.c b/extern/isocline/src/term.c new file mode 100644 index 00000000..c55d9ae5 --- /dev/null +++ b/extern/isocline/src/term.c @@ -0,0 +1,1124 @@ +/* ---------------------------------------------------------------------------- + Copyright (c) 2021, Daan Leijen + This is free software; you can redistribute it and/or modify it + under the terms of the MIT License. A copy of the license can be + found in the "LICENSE" file at the root of this distribution. +-----------------------------------------------------------------------------*/ +#include +#include +#include +#include // getenv +#include + +#include "common.h" +#include "tty.h" +#include "term.h" +#include "stringbuf.h" // str_next_ofs + +#if defined(_WIN32) +#include +#define STDOUT_FILENO 1 +#else +#include +#include +#include +#if defined(__linux__) +#include +#endif +#endif + +#define IC_CSI "\x1B[" + +// color support; colors are auto mapped smaller palettes if needed. (see `term_color.c`) +typedef enum palette_e { + MONOCHROME, // no color + ANSI8, // only basic 8 ANSI color (ESC[m, idx: 30-37, +10 for background) + ANSI16, // basic + bright ANSI colors (ESC[m, idx: 30-37, 90-97, +10 for background) + ANSI256, // ANSI 256 color palette (ESC[38;5;m, idx: 0-15 standard color, 16-231 6x6x6 rbg colors, 232-255 gray shades) + ANSIRGB // direct rgb colors supported (ESC[38;2;;;m) +} palette_t; + +// The terminal screen +struct term_s { + int fd_out; // output handle + ssize_t width; // screen column width + ssize_t height; // screen row height + ssize_t raw_enabled; // is raw mode active? counted by start/end pairs + bool nocolor; // show colors? + bool silent; // enable beep? + bool is_utf8; // utf-8 output? determined by the tty + attr_t attr; // current text attributes + palette_t palette; // color support + buffer_mode_t bufmode; // buffer mode + stringbuf_t* buf; // buffer for buffered output + tty_t* tty; // used on posix to get the cursor position + alloc_t* mem; // allocator + #ifdef _WIN32 + HANDLE hcon; // output console handler + WORD hcon_default_attr; // default text attributes + WORD hcon_orig_attr; // original text attributes + DWORD hcon_orig_mode; // original console mode + DWORD hcon_mode; // used console mode + UINT hcon_orig_cp; // original console code-page (locale) + COORD hcon_save_cursor; // saved cursor position (for escape sequence emulation) + #endif +}; + +static bool term_write_direct(term_t* term, const char* s, ssize_t n ); +static void term_append_buf(term_t* term, const char* s, ssize_t n); + +//------------------------------------------------------------- +// Colors +//------------------------------------------------------------- + +#include "term_color.c" + +//------------------------------------------------------------- +// Helpers +//------------------------------------------------------------- + +ic_private void term_left(term_t* term, ssize_t n) { + if (n <= 0) return; + term_writef( term, IC_CSI "%zdD", n ); +} + +ic_private void term_right(term_t* term, ssize_t n) { + if (n <= 0) return; + term_writef( term, IC_CSI "%zdC", n ); +} + +ic_private void term_up(term_t* term, ssize_t n) { + if (n <= 0) return; + term_writef( term, IC_CSI "%zdA", n ); +} + +ic_private void term_down(term_t* term, ssize_t n) { + if (n <= 0) return; + term_writef( term, IC_CSI "%zdB", n ); +} + +ic_private void term_clear_line(term_t* term) { + term_write( term, "\r" IC_CSI "K"); +} + +ic_private void term_clear_to_end_of_line(term_t* term) { + term_write(term, IC_CSI "K"); +} + +ic_private void term_start_of_line(term_t* term) { + term_write( term, "\r" ); +} + +ic_private ssize_t term_get_width(term_t* term) { + return term->width; +} + +ic_private ssize_t term_get_height(term_t* term) { + return term->height; +} + +ic_private void term_attr_reset(term_t* term) { + term_write(term, IC_CSI "m" ); +} + +ic_private void term_underline(term_t* term, bool on) { + term_write(term, on ? IC_CSI "4m" : IC_CSI "24m" ); +} + +ic_private void term_reverse(term_t* term, bool on) { + term_write(term, on ? IC_CSI "7m" : IC_CSI "27m"); +} + +ic_private void term_bold(term_t* term, bool on) { + term_write(term, on ? IC_CSI "1m" : IC_CSI "22m" ); +} + +ic_private void term_italic(term_t* term, bool on) { + term_write(term, on ? IC_CSI "3m" : IC_CSI "23m" ); +} + +ic_private void term_writeln(term_t* term, const char* s) { + term_write(term,s); + term_write(term,"\n"); +} + +ic_private void term_write_char(term_t* term, char c) { + char buf[2]; + buf[0] = c; + buf[1] = 0; + term_write_n(term, buf, 1 ); +} + +ic_private attr_t term_get_attr( const term_t* term ) { + return term->attr; +} + +ic_private void term_set_attr( term_t* term, attr_t attr ) { + if (term->nocolor) return; + if (attr.x.color != term->attr.x.color && attr.x.color != IC_COLOR_NONE) { + term_color(term,attr.x.color); + if (term->palette < ANSIRGB && color_is_rgb(attr.x.color)) { + term->attr.x.color = attr.x.color; // actual color may have been approximated but we keep the actual color to avoid updating every time + } + } + if (attr.x.bgcolor != term->attr.x.bgcolor && attr.x.bgcolor != IC_COLOR_NONE) { + term_bgcolor(term,attr.x.bgcolor); + if (term->palette < ANSIRGB && color_is_rgb(attr.x.bgcolor)) { + term->attr.x.bgcolor = attr.x.bgcolor; + } + } + if (attr.x.bold != term->attr.x.bold && attr.x.bold != IC_NONE) { + term_bold(term,attr.x.bold == IC_ON); + } + if (attr.x.underline != term->attr.x.underline && attr.x.underline != IC_NONE) { + term_underline(term,attr.x.underline == IC_ON); + } + if (attr.x.reverse != term->attr.x.reverse && attr.x.reverse != IC_NONE) { + term_reverse(term,attr.x.reverse == IC_ON); + } + if (attr.x.italic != term->attr.x.italic && attr.x.italic != IC_NONE) { + term_italic(term,attr.x.italic == IC_ON); + } + assert(attr.x.color == term->attr.x.color || attr.x.color == IC_COLOR_NONE); + assert(attr.x.bgcolor == term->attr.x.bgcolor || attr.x.bgcolor == IC_COLOR_NONE); + assert(attr.x.bold == term->attr.x.bold || attr.x.bold == IC_NONE); + assert(attr.x.reverse == term->attr.x.reverse || attr.x.reverse == IC_NONE); + assert(attr.x.underline == term->attr.x.underline || attr.x.underline == IC_NONE); + assert(attr.x.italic == term->attr.x.italic || attr.x.italic == IC_NONE); +} + + +/* +ic_private void term_clear_lines_to_end(term_t* term) { + term_write(term, "\r" IC_CSI "J"); +} + +ic_private void term_show_cursor(term_t* term, bool on) { + term_write(term, on ? IC_CSI "?25h" : IC_CSI "?25l"); +} +*/ + +//------------------------------------------------------------- +// Formatted output +//------------------------------------------------------------- + +ic_private void term_writef(term_t* term, const char* fmt, ...) { + va_list ap; + va_start(ap, fmt); + term_vwritef(term,fmt,ap); + va_end(ap); +} + +ic_private void term_vwritef(term_t* term, const char* fmt, va_list args ) { + sbuf_append_vprintf(term->buf, fmt, args); +} + +ic_private void term_write_formatted( term_t* term, const char* s, const attr_t* attrs ) { + term_write_formatted_n( term, s, attrs, ic_strlen(s)); +} + +ic_private void term_write_formatted_n( term_t* term, const char* s, const attr_t* attrs, ssize_t len ) { + if (attrs == NULL) { + // write directly + term_write(term,s); + } + else { + // ensure raw mode from now on + if (term->raw_enabled <= 0) { + term_start_raw(term); + } + // and output with text attributes + const attr_t default_attr = term_get_attr(term); + attr_t attr = attr_none(); + ssize_t i = 0; + ssize_t n = 0; + while( i+n < len && s[i+n] != 0 ) { + if (!attr_is_eq(attr,attrs[i+n])) { + if (n > 0) { + term_write_n( term, s+i, n ); + i += n; + n = 0; + } + attr = attrs[i]; + term_set_attr( term, attr_update_with(default_attr,attr) ); + } + n++; + } + if (n > 0) { + term_write_n( term, s+i, n ); + i += n; + n = 0; + } + assert(s[i] != 0 || i == len); + term_set_attr(term, default_attr); + } +} + +//------------------------------------------------------------- +// Write to the terminal +// The buffered functions are used to reduce cursor flicker +// during refresh +//------------------------------------------------------------- + +ic_private void term_beep(term_t* term) { + if (term->silent) return; + fprintf(stderr,"\x7"); + fflush(stderr); +} + +ic_private void term_write_repeat(term_t* term, const char* s, ssize_t count) { + for (; count > 0; count--) { + term_write(term, s); + } +} + +ic_private void term_write(term_t* term, const char* s) { + if (s == NULL || s[0] == 0) return; + ssize_t n = ic_strlen(s); + term_write_n(term,s,n); +} + +// Primitive terminal write; all writes go through here +ic_private void term_write_n(term_t* term, const char* s, ssize_t n) { + if (s == NULL || n <= 0) return; + // write to buffer to reduce flicker and to process escape sequences (this may flush too) + term_append_buf(term, s, n); +} + + +//------------------------------------------------------------- +// Buffering +//------------------------------------------------------------- + + +ic_private void term_flush(term_t* term) { + if (sbuf_len(term->buf) > 0) { + //term_show_cursor(term,false); + term_write_direct(term, sbuf_string(term->buf), sbuf_len(term->buf)); + //term_show_cursor(term,true); + sbuf_clear(term->buf); + } +} + +ic_private buffer_mode_t term_set_buffer_mode(term_t* term, buffer_mode_t mode) { + buffer_mode_t oldmode = term->bufmode; + if (oldmode != mode) { + if (mode == UNBUFFERED) { + term_flush(term); + } + term->bufmode = mode; + } + return oldmode; +} + +static void term_check_flush(term_t* term, bool contains_nl) { + if (term->bufmode == UNBUFFERED || + sbuf_len(term->buf) > 4000 || + (term->bufmode == LINEBUFFERED && contains_nl)) + { + term_flush(term); + } +} + +//------------------------------------------------------------- +// Init +//------------------------------------------------------------- + +static void term_init_raw(term_t* term); + +ic_private term_t* term_new(alloc_t* mem, tty_t* tty, bool nocolor, bool silent, int fd_out ) +{ + term_t* term = mem_zalloc_tp(mem, term_t); + if (term == NULL) return NULL; + + term->fd_out = (fd_out < 0 ? STDOUT_FILENO : fd_out); + term->nocolor = nocolor || (isatty(term->fd_out) == 0); + term->silent = silent; + term->mem = mem; + term->tty = tty; // can be NULL + term->width = 80; + term->height = 25; + term->is_utf8 = tty_is_utf8(tty); + term->palette = ANSI16; // almost universally supported + term->buf = sbuf_new(mem); + term->bufmode = LINEBUFFERED; + term->attr = attr_default(); + + // respect NO_COLOR + if (getenv("NO_COLOR") != NULL) { + term->nocolor = true; + } + if (!term->nocolor) { + // detect color palette + // COLORTERM takes precedence + const char* colorterm = getenv("COLORTERM"); + const char* eterm = getenv("TERM"); + if (ic_contains(colorterm,"24bit") || ic_contains(colorterm,"truecolor") || ic_contains(colorterm,"direct")) { + term->palette = ANSIRGB; + } + else if (ic_contains(colorterm,"8bit") || ic_contains(colorterm,"256color")) { term->palette = ANSI256; } + else if (ic_contains(colorterm,"4bit") || ic_contains(colorterm,"16color")) { term->palette = ANSI16; } + else if (ic_contains(colorterm,"3bit") || ic_contains(colorterm,"8color")) { term->palette = ANSI8; } + else if (ic_contains(colorterm,"1bit") || ic_contains(colorterm,"nocolor") || ic_contains(colorterm,"monochrome")) { + term->palette = MONOCHROME; + } + // otherwise check for some specific terminals + else if (getenv("WT_SESSION") != NULL) { term->palette = ANSIRGB; } // Windows terminal + else if (getenv("ITERM_SESSION_ID") != NULL) { term->palette = ANSIRGB; } // iTerm2 terminal + else if (getenv("VSCODE_PID") != NULL) { term->palette = ANSIRGB; } // vscode terminal + else { + // and otherwise fall back to checking TERM + if (ic_contains(eterm,"truecolor") || ic_contains(eterm,"direct") || ic_contains(colorterm,"24bit")) { + term->palette = ANSIRGB; + } + else if (ic_contains(eterm,"alacritty") || ic_contains(eterm,"kitty")) { + term->palette = ANSIRGB; + } + else if (ic_contains(eterm,"256color") || ic_contains(eterm,"gnome")) { + term->palette = ANSI256; + } + else if (ic_contains(eterm,"16color")){ term->palette = ANSI16; } + else if (ic_contains(eterm,"8color")) { term->palette = ANSI8; } + else if (ic_contains(eterm,"monochrome") || ic_contains(eterm,"nocolor") || ic_contains(eterm,"dumb")) { + term->palette = MONOCHROME; + } + } + debug_msg("term: color-bits: %d (COLORTERM=%s, TERM=%s)\n", term_get_color_bits(term), colorterm, eterm); + } + + // read COLUMS/LINES from the environment for a better initial guess. + const char* env_columns = getenv("COLUMNS"); + if (env_columns != NULL) { ic_atoz(env_columns, &term->width); } + const char* env_lines = getenv("LINES"); + if (env_lines != NULL) { ic_atoz(env_lines, &term->height); } + + // initialize raw terminal output and terminal dimensions + term_init_raw(term); + term_update_dim(term); + term_attr_reset(term); // ensure we are at default settings + + return term; +} + +ic_private bool term_is_interactive(const term_t* term) { + ic_unused(term); + // check dimensions (0 is used for debuggers) + // if (term->width <= 0) return false; + + // check editing support + const char* eterm = getenv("TERM"); + debug_msg("term: TERM=%s\n", eterm); + if (eterm != NULL && + (strstr("dumb|DUMB|cons25|CONS25|emacs|EMACS",eterm) != NULL)) { + return false; + } + + return true; +} + +ic_private bool term_enable_beep(term_t* term, bool enable) { + bool prev = term->silent; + term->silent = !enable; + return prev; +} + +ic_private bool term_enable_color(term_t* term, bool enable) { + bool prev = !term->nocolor; + term->nocolor = !enable; + return prev; +} + +ic_private void term_free(term_t* term) { + if (term == NULL) return; + term_flush(term); + term_end_raw(term, true); + sbuf_free(term->buf); term->buf = NULL; + mem_free(term->mem, term); +} + +//------------------------------------------------------------- +// For best portability and applications inserting CSI SGR (ESC[ .. m) +// codes themselves in strings, we interpret these at the +// lowest level so we can have a `term_get_attr` function which +// is needed for bracketed styles etc. +//------------------------------------------------------------- + +static void term_append_esc(term_t* term, const char* const s, ssize_t len) { + if (s[1]=='[' && s[len-1] == 'm') { + // it is a CSI SGR sequence: ESC[ ... m + if (term->nocolor) return; // ignore escape sequences if nocolor is set + term->attr = attr_update_with(term->attr, attr_from_esc_sgr(s,len)); + } + // and write out the escape sequence as-is + sbuf_append_n(term->buf, s, len); +} + + +static void term_append_utf8(term_t* term, const char* s, ssize_t len) { + ssize_t nread; + unicode_t uchr = unicode_from_qutf8((const uint8_t*)s, len, &nread); + uint8_t c; + if (unicode_is_raw(uchr, &c)) { + // write bytes as is; this also ensure that on non-utf8 terminals characters between 0x80-0xFF + // go through _as is_ due to the qutf8 encoding. + sbuf_append_char(term->buf,(char)c); + } + else if (!term->is_utf8) { + // on non-utf8 terminals still send utf-8 and hope for the best + // todo: we could try to convert to the locale first? + sbuf_append_n(term->buf, s, len); + // sbuf_appendf(term->buf, "\x1B[%" PRIu32 "u", uchr); // unicode escape code + } + else { + // write utf-8 as is + sbuf_append_n(term->buf, s, len); + } +} + +static void term_append_buf( term_t* term, const char* s, ssize_t len ) { + ssize_t pos = 0; + bool newline = false; + while (pos < len) { + // handle ascii sequences in bulk + ssize_t ascii = 0; + ssize_t next; + while ((next = str_next_ofs(s, len, pos+ascii, NULL)) > 0 && + (uint8_t)s[pos + ascii] > '\x1B' && (uint8_t)s[pos + ascii] <= 0x7F ) + { + ascii += next; + } + if (ascii > 0) { + sbuf_append_n(term->buf, s+pos, ascii); + pos += ascii; + } + if (next <= 0) break; + + const uint8_t c = (uint8_t)s[pos]; + // handle utf8 sequences (for non-utf8 terminals) + if (c >= 0x80) { + term_append_utf8(term, s+pos, next); + } + // handle escape sequence (note: str_next_ofs considers whole CSI escape sequences at a time) + else if (next > 1 && c == '\x1B') { + term_append_esc(term, s+pos, next); + } + else if (c < ' ' && c != 0 && (c < '\x07' || c > '\x0D')) { + // ignore control characters except \a, \b, \t, \n, \r, and form-feed and vertical tab. + } + else { + if (c == '\n') { newline = true; } + sbuf_append_n(term->buf, s+pos, next); + } + pos += next; + } + // possibly flush + term_check_flush(term, newline); +} + +//------------------------------------------------------------- +// Platform dependent: Write directly to the terminal +//------------------------------------------------------------- + +#if !defined(_WIN32) + +// write to the console without further processing +static bool term_write_direct(term_t* term, const char* s, ssize_t n) { + ssize_t count = 0; + while( count < n ) { + ssize_t nwritten = write(term->fd_out, s + count, to_size_t(n - count)); + if (nwritten > 0) { + count += nwritten; + } + else if (errno != EINTR && errno != EAGAIN) { + debug_msg("term: write failed: length %i, errno %i: \"%s\"\n", n, errno, s); + return false; + } + } + return true; +} + +#else + +//---------------------------------------------------------------------------------- +// On windows we use the new virtual terminal processing if it is available (Windows Terminal) +// but fall back to ansi escape emulation on older systems but also for example +// the PS terminal +// +// note: we use row/col as 1-based ANSI escape while windows X/Y coords are 0-based. +//----------------------------------------------------------------------------------- + +#if !defined(ENABLE_VIRTUAL_TERMINAL_PROCESSING) +#define ENABLE_VIRTUAL_TERMINAL_PROCESSING (0) +#endif +#if !defined(ENABLE_LVB_GRID_WORLDWIDE) +#define ENABLE_LVB_GRID_WORLDWIDE (0) +#endif + +// direct write to the console without further processing +static bool term_write_console(term_t* term, const char* s, ssize_t n ) { + DWORD written; + // WriteConsoleA(term->hcon, s, (DWORD)(to_size_t(n)), &written, NULL); + WriteFile(term->hcon, s, (DWORD)(to_size_t(n)), &written, NULL); // so it can be redirected + return (written == (DWORD)(to_size_t(n))); +} + +static bool term_get_cursor_pos( term_t* term, ssize_t* row, ssize_t* col) { + *row = 0; + *col = 0; + CONSOLE_SCREEN_BUFFER_INFO info; + if (!GetConsoleScreenBufferInfo(term->hcon, &info)) return false; + *row = (ssize_t)info.dwCursorPosition.Y + 1; + *col = (ssize_t)info.dwCursorPosition.X + 1; + return true; +} + +static void term_move_cursor_to( term_t* term, ssize_t row, ssize_t col ) { + CONSOLE_SCREEN_BUFFER_INFO info; + if (!GetConsoleScreenBufferInfo( term->hcon, &info )) return; + if (col > info.dwSize.X) col = info.dwSize.X; + if (row > info.dwSize.Y) row = info.dwSize.Y; + if (col <= 0) col = 1; + if (row <= 0) row = 1; + COORD coord; + coord.X = (SHORT)col - 1; + coord.Y = (SHORT)row - 1; + SetConsoleCursorPosition( term->hcon, coord); +} + +static void term_cursor_save(term_t* term) { + memset(&term->hcon_save_cursor, 0, sizeof(term->hcon_save_cursor)); + CONSOLE_SCREEN_BUFFER_INFO info; + if (!GetConsoleScreenBufferInfo(term->hcon, &info)) return; + term->hcon_save_cursor = info.dwCursorPosition; +} + +static void term_cursor_restore(term_t* term) { + if (term->hcon_save_cursor.X == 0) return; + SetConsoleCursorPosition(term->hcon, term->hcon_save_cursor); +} + +static void term_move_cursor( term_t* term, ssize_t drow, ssize_t dcol, ssize_t n ) { + CONSOLE_SCREEN_BUFFER_INFO info; + if (!GetConsoleScreenBufferInfo( term->hcon, &info )) return; + COORD cur = info.dwCursorPosition; + ssize_t col = (ssize_t)cur.X + 1 + n*dcol; + ssize_t row = (ssize_t)cur.Y + 1 + n*drow; + term_move_cursor_to( term, row, col ); +} + +static void term_cursor_visible( term_t* term, bool visible ) { + CONSOLE_CURSOR_INFO info; + if (!GetConsoleCursorInfo(term->hcon,&info)) return; + info.bVisible = visible; + SetConsoleCursorInfo(term->hcon,&info); +} + +static void term_erase_line( term_t* term, ssize_t mode ) { + CONSOLE_SCREEN_BUFFER_INFO info; + if (!GetConsoleScreenBufferInfo( term->hcon, &info )) return; + DWORD written; + COORD start; + ssize_t length; + if (mode == 2) { + // entire line + start.X = 0; + start.Y = info.dwCursorPosition.Y; + length = (ssize_t)info.srWindow.Right + 1; + } + else if (mode == 1) { + // to start of line + start.X = 0; + start.Y = info.dwCursorPosition.Y; + length = info.dwCursorPosition.X; + } + else { + // to end of line + length = (ssize_t)info.srWindow.Right - info.dwCursorPosition.X + 1; + start = info.dwCursorPosition; + } + FillConsoleOutputAttribute( term->hcon, term->hcon_default_attr, (DWORD)length, start, &written ); + FillConsoleOutputCharacterA( term->hcon, ' ', (DWORD)length, start, &written ); +} + +static void term_clear_screen(term_t* term, ssize_t mode) { + CONSOLE_SCREEN_BUFFER_INFO info; + if (!GetConsoleScreenBufferInfo(term->hcon, &info)) return; + COORD start; + start.X = 0; + start.Y = 0; + ssize_t length; + ssize_t width = (ssize_t)info.dwSize.X; + if (mode == 2) { + // entire screen + length = width * info.dwSize.Y; + } + else if (mode == 1) { + // to cursor + length = (width * ((ssize_t)info.dwCursorPosition.Y - 1)) + info.dwCursorPosition.X; + } + else { + // from cursor + start = info.dwCursorPosition; + length = (width * ((ssize_t)info.dwSize.Y - info.dwCursorPosition.Y)) + (width - info.dwCursorPosition.X + 1); + } + DWORD written; + FillConsoleOutputAttribute(term->hcon, term->hcon_default_attr, (DWORD)length, start, &written); + FillConsoleOutputCharacterA(term->hcon, ' ', (DWORD)length, start, &written); +} + +static WORD attr_color[8] = { + 0, // black + FOREGROUND_RED, // maroon + FOREGROUND_GREEN, // green + FOREGROUND_RED | FOREGROUND_GREEN, // orange + FOREGROUND_BLUE, // navy + FOREGROUND_RED | FOREGROUND_BLUE, // purple + FOREGROUND_GREEN | FOREGROUND_BLUE, // teal + FOREGROUND_RED | FOREGROUND_GREEN | FOREGROUND_BLUE, // light gray +}; + +static void term_set_win_attr( term_t* term, attr_t ta ) { + WORD def_attr = term->hcon_default_attr; + CONSOLE_SCREEN_BUFFER_INFO info; + if (!GetConsoleScreenBufferInfo( term->hcon, &info )) return; + WORD cur_attr = info.wAttributes; + WORD attr = cur_attr; + if (ta.x.color != IC_COLOR_NONE) { + if (ta.x.color >= IC_ANSI_BLACK && ta.x.color <= IC_ANSI_SILVER) { + attr = (attr & 0xFFF0) | attr_color[ta.x.color - IC_ANSI_BLACK]; + } + else if (ta.x.color >= IC_ANSI_GRAY && ta.x.color <= IC_ANSI_WHITE) { + attr = (attr & 0xFFF0) | attr_color[ta.x.color - IC_ANSI_GRAY] | FOREGROUND_INTENSITY; + } + else if (ta.x.color == IC_ANSI_DEFAULT) { + attr = (attr & 0xFFF0) | (def_attr & 0x000F); + } + } + if (ta.x.bgcolor != IC_COLOR_NONE) { + if (ta.x.bgcolor >= IC_ANSI_BLACK && ta.x.bgcolor <= IC_ANSI_SILVER) { + attr = (attr & 0xFF0F) | (WORD)(attr_color[ta.x.bgcolor - IC_ANSI_BLACK] << 4); + } + else if (ta.x.bgcolor >= IC_ANSI_GRAY && ta.x.bgcolor <= IC_ANSI_WHITE) { + attr = (attr & 0xFF0F) | (WORD)(attr_color[ta.x.bgcolor - IC_ANSI_GRAY] << 4) | BACKGROUND_INTENSITY; + } + else if (ta.x.bgcolor == IC_ANSI_DEFAULT) { + attr = (attr & 0xFF0F) | (def_attr & 0x00F0); + } + } + if (ta.x.underline != IC_NONE) { + attr = (attr & ~COMMON_LVB_UNDERSCORE) | (ta.x.underline == IC_ON ? COMMON_LVB_UNDERSCORE : 0); + } + if (ta.x.reverse != IC_NONE) { + attr = (attr & ~COMMON_LVB_REVERSE_VIDEO) | (ta.x.reverse == IC_ON ? COMMON_LVB_REVERSE_VIDEO : 0); + } + if (attr != cur_attr) { + SetConsoleTextAttribute(term->hcon, attr); + } +} + +static ssize_t esc_param( const char* s, ssize_t def ) { + if (*s == '?') s++; + ssize_t n = def; + ic_atoz(s, &n); + return n; +} + +static void esc_param2( const char* s, ssize_t* p1, ssize_t* p2, ssize_t def ) { + if (*s == '?') s++; + *p1 = def; + *p2 = def; + ic_atoz2(s, p1, p2); +} + +// Emulate escape sequences on older windows. +static void term_write_esc( term_t* term, const char* s, ssize_t len ) { + ssize_t row; + ssize_t col; + + if (s[1] == '[') { + switch (s[len-1]) { + case 'A': + term_move_cursor(term, -1, 0, esc_param(s+2, 1)); + break; + case 'B': + term_move_cursor(term, 1, 0, esc_param(s+2, 1)); + break; + case 'C': + term_move_cursor(term, 0, 1, esc_param(s+2, 1)); + break; + case 'D': + term_move_cursor(term, 0, -1, esc_param(s+2, 1)); + break; + case 'H': + esc_param2(s+2, &row, &col, 1); + term_move_cursor_to(term, row, col); + break; + case 'K': + term_erase_line(term, esc_param(s+2, 0)); + break; + case 'm': + term_set_win_attr( term, attr_from_esc_sgr(s,len) ); + break; + + // support some less standard escape codes (currently not used by isocline) + case 'E': // line down + term_get_cursor_pos(term, &row, &col); + row += esc_param(s+2, 1); + term_move_cursor_to(term, row, 1); + break; + case 'F': // line up + term_get_cursor_pos(term, &row, &col); + row -= esc_param(s+2, 1); + term_move_cursor_to(term, row, 1); + break; + case 'G': // absolute column + term_get_cursor_pos(term, &row, &col); + col = esc_param(s+2, 1); + term_move_cursor_to(term, row, col); + break; + case 'J': + term_clear_screen(term, esc_param(s+2, 0)); + break; + case 'h': + if (strncmp(s+2, "?25h", 4) == 0) { + term_cursor_visible(term, true); + } + break; + case 'l': + if (strncmp(s+2, "?25l", 4) == 0) { + term_cursor_visible(term, false); + } + break; + case 's': + term_cursor_save(term); + break; + case 'u': + term_cursor_restore(term); + break; + // otherwise ignore + } + } + else if (s[1] == '7') { + term_cursor_save(term); + } + else if (s[1] == '8') { + term_cursor_restore(term); + } + else { + // otherwise ignore + } +} + +static bool term_write_direct(term_t* term, const char* s, ssize_t len ) { + term_cursor_visible(term,false); // reduce flicker + ssize_t pos = 0; + if ((term->hcon_mode & ENABLE_VIRTUAL_TERMINAL_PROCESSING) != 0) { + // use the builtin virtual terminal processing. (enables truecolor for example) + term_write_console(term, s, len); + pos = len; + } + else { + // emulate escape sequences + while( pos < len ) { + // handle non-control in bulk (including utf-8 sequences) + // (We don't need to handle utf-8 separately as we set the codepage to always be in utf-8 mode) + ssize_t nonctrl = 0; + ssize_t next; + while( (next = str_next_ofs( s, len, pos+nonctrl, NULL )) > 0 && + (uint8_t)s[pos + nonctrl] >= ' ' && (uint8_t)s[pos + nonctrl] <= 0x7F) { + nonctrl += next; + } + if (nonctrl > 0) { + term_write_console(term, s+pos, nonctrl); + pos += nonctrl; + } + if (next <= 0) break; + + if ((uint8_t)s[pos] >= 0x80) { + // utf8 is already processed + term_write_console(term, s+pos, next); + } + else if (next > 1 && s[pos] == '\x1B') { + // handle control (note: str_next_ofs considers whole CSI escape sequences at a time) + term_write_esc(term, s+pos, next); + } + else if (next == 1 && (s[pos] == '\r' || s[pos] == '\n' || s[pos] == '\t' || s[pos] == '\b')) { + term_write_console( term, s+pos, next); + } + else { + // ignore + } + pos += next; + } + } + term_cursor_visible(term,true); + assert(pos == len); + return (pos == len); + +} +#endif + + + +//------------------------------------------------------------- +// Update terminal dimensions +//------------------------------------------------------------- + +#if !defined(_WIN32) + +// send escape query that may return a response on the tty +static bool term_esc_query_raw( term_t* term, const char* query, char* buf, ssize_t buflen ) +{ + if (buf==NULL || buflen <= 0 || query[0] == 0) return false; + bool osc = (query[1] == ']'); + if (!term_write_direct(term, query, ic_strlen(query))) return false; + debug_msg("term: read tty query response to: ESC %s\n", query + 1); + return tty_read_esc_response( term->tty, query[1], osc, buf, buflen ); +} + +static bool term_esc_query( term_t* term, const char* query, char* buf, ssize_t buflen ) +{ + if (!tty_start_raw(term->tty)) return false; + bool ok = term_esc_query_raw(term,query,buf,buflen); + tty_end_raw(term->tty); + return ok; +} + +// get the cursor position via an ESC[6n +static bool term_get_cursor_pos( term_t* term, ssize_t* row, ssize_t* col) +{ + // send escape query + char buf[128]; + if (!term_esc_query(term,"\x1B[6n",buf,128)) return false; + if (!ic_atoz2(buf,row,col)) return false; + return true; +} + +static void term_set_cursor_pos( term_t* term, ssize_t row, ssize_t col ) { + term_writef( term, IC_CSI "%zd;%zdH", row, col ); +} + +ic_private bool term_update_dim(term_t* term) { + ssize_t cols = 0; + ssize_t rows = 0; + struct winsize ws; + if (ioctl(term->fd_out, TIOCGWINSZ, &ws) >= 0) { + // ioctl succeeded + cols = ws.ws_col; // debuggers return 0 for the column + rows = ws.ws_row; + } + else { + // determine width by querying the cursor position + debug_msg("term: ioctl term-size failed: %d,%d\n", ws.ws_row, ws.ws_col); + ssize_t col0 = 0; + ssize_t row0 = 0; + if (term_get_cursor_pos(term,&row0,&col0)) { + term_set_cursor_pos(term,999,999); + ssize_t col1 = 0; + ssize_t row1 = 0; + if (term_get_cursor_pos(term,&row1,&col1)) { + cols = col1; + rows = row1; + } + term_set_cursor_pos(term,row0,col0); + } + else { + // cannot query position + // return 0 column + } + } + + // update width and return whether it changed. + bool changed = (term->width != cols || term->height != rows); + debug_msg("terminal dim: %zd,%zd: %s\n", rows, cols, changed ? "changed" : "unchanged"); + if (cols > 0) { + term->width = cols; + term->height = rows; + } + return changed; +} + +#else + +ic_private bool term_update_dim(term_t* term) { + if (term->hcon == 0) { + term->hcon = GetConsoleWindow(); + } + ssize_t rows = 0; + ssize_t cols = 0; + CONSOLE_SCREEN_BUFFER_INFO sbinfo; + if (GetConsoleScreenBufferInfo(term->hcon, &sbinfo)) { + cols = (ssize_t)sbinfo.srWindow.Right - (ssize_t)sbinfo.srWindow.Left + 1; + rows = (ssize_t)sbinfo.srWindow.Bottom - (ssize_t)sbinfo.srWindow.Top + 1; + } + bool changed = (term->width != cols || term->height != rows); + term->width = cols; + term->height = rows; + debug_msg("term: update dim: %zd, %zd\n", term->height, term->width ); + return changed; +} + +#endif + + + +//------------------------------------------------------------- +// Enable/disable terminal raw mode +//------------------------------------------------------------- + +#if !defined(_WIN32) + +// On non-windows, the terminal is set in raw mode by the tty. + +ic_private void term_start_raw(term_t* term) { + term->raw_enabled++; +} + +ic_private void term_end_raw(term_t* term, bool force) { + if (term->raw_enabled <= 0) return; + if (!force) { + term->raw_enabled--; + } + else { + term->raw_enabled = 0; + } +} + +static bool term_esc_query_color_raw(term_t* term, int color_idx, uint32_t* color ) { + char buf[128+1]; + snprintf(buf,128,"\x1B]4;%d;?\x1B\\", color_idx); + if (!term_esc_query_raw( term, buf, buf, 128 )) { + debug_msg("esc query response not received\n"); + return false; + } + if (buf[0] != '4') return false; + const char* rgb = strchr(buf,':'); + if (rgb==NULL) return false; + rgb++; // skip ':' + unsigned int r,g,b; + if (sscanf(rgb,"%x/%x/%x",&r,&g,&b) != 3) return false; + if (rgb[2]!='/') { // 48-bit rgb, hexadecimal round to 24-bit + r = (r+0x7F)/0x100; // note: can "overflow", e.g. 0xFFFF -> 0x100. (and we need `ic_cap8` to convert.) + g = (g+0x7F)/0x100; + b = (b+0x7F)/0x100; + } + *color = (ic_cap8(r)<<16) | (ic_cap8(g)<<8) | ic_cap8(b); + debug_msg("color query: %02x,%02x,%02x: %06x\n", r, g, b, *color); + return true; +} + +// update ansi 16 color palette for better color approximation +static void term_update_ansi16(term_t* term) { + debug_msg("update ansi colors\n"); + #if defined(GIO_CMAP) + // try ioctl first (on Linux) + uint8_t cmap[48]; + memset(cmap,0,48); + if (ioctl(term->fd_out,GIO_CMAP,&cmap) >= 0) { + // success + for(ssize_t i = 0; i < 48; i+=3) { + uint32_t color = ((uint32_t)(cmap[i]) << 16) | ((uint32_t)(cmap[i+1]) << 8) | cmap[i+2]; + debug_msg("term (ioctl) ansi color %d: 0x%06x\n", i, color); + ansi256[i] = color; + } + return; + } + else { + debug_msg("ioctl GIO_CMAP failed: entry 1: 0x%02x%02x%02x\n", cmap[3], cmap[4], cmap[5]); + } + #endif + // this seems to be unreliable on some systems (Ubuntu+Gnome terminal) so only enable when known ok. + #if __APPLE__ + // otherwise use OSC 4 escape sequence query + if (tty_start_raw(term->tty)) { + for(ssize_t i = 0; i < 16; i++) { + uint32_t color; + if (!term_esc_query_color_raw(term, i, &color)) break; + debug_msg("term ansi color %d: 0x%06x\n", i, color); + ansi256[i] = color; + } + tty_end_raw(term->tty); + } + #endif +} + +static void term_init_raw(term_t* term) { + if (term->palette < ANSIRGB) { + term_update_ansi16(term); + } +} + +#else + +ic_private void term_start_raw(term_t* term) { + if (term->raw_enabled++ > 0) return; + CONSOLE_SCREEN_BUFFER_INFO info; + if (GetConsoleScreenBufferInfo(term->hcon, &info)) { + term->hcon_orig_attr = info.wAttributes; + } + term->hcon_orig_cp = GetConsoleOutputCP(); + SetConsoleOutputCP(CP_UTF8); + if (term->hcon_mode == 0) { + // first time initialization + DWORD mode = ENABLE_PROCESSED_OUTPUT | ENABLE_WRAP_AT_EOL_OUTPUT | ENABLE_LVB_GRID_WORLDWIDE; // for \r \n and \b + // use escape sequence handling if available and the terminal supports it (so we can use rgb colors in Windows terminal) + // Unfortunately, in plain powershell, we can successfully enable terminal processing + // but it still fails to render correctly; so we require the palette be large enough (like in Windows Terminal) + if (term->palette >= ANSI256 && SetConsoleMode(term->hcon, mode | ENABLE_VIRTUAL_TERMINAL_PROCESSING)) { + term->hcon_mode = mode | ENABLE_VIRTUAL_TERMINAL_PROCESSING; + debug_msg("term: console mode: virtual terminal processing enabled\n"); + } + // no virtual terminal processing, emulate instead + else if (SetConsoleMode(term->hcon, mode)) { + term->hcon_mode = mode; + term->palette = ANSI16; + } + GetConsoleMode(term->hcon, &mode); + debug_msg("term: console mode: orig: 0x%x, new: 0x%x, current 0x%x\n", term->hcon_orig_mode, term->hcon_mode, mode); + } + else { + SetConsoleMode(term->hcon, term->hcon_mode); + } +} + +ic_private void term_end_raw(term_t* term, bool force) { + if (term->raw_enabled <= 0) return; + if (!force && term->raw_enabled > 1) { + term->raw_enabled--; + } + else { + term->raw_enabled = 0; + SetConsoleMode(term->hcon, term->hcon_orig_mode); + SetConsoleOutputCP(term->hcon_orig_cp); + SetConsoleTextAttribute(term->hcon, term->hcon_orig_attr); + } +} + +static void term_init_raw(term_t* term) { + term->hcon = GetStdHandle(STD_OUTPUT_HANDLE); + GetConsoleMode(term->hcon, &term->hcon_orig_mode); + CONSOLE_SCREEN_BUFFER_INFOEX info; + memset(&info, 0, sizeof(info)); + info.cbSize = sizeof(info); + if (GetConsoleScreenBufferInfoEx(term->hcon, &info)) { + // store default attributes + term->hcon_default_attr = info.wAttributes; + // update our color table with the actual colors used. + for (unsigned i = 0; i < 16; i++) { + COLORREF cr = info.ColorTable[i]; + uint32_t color = (ic_cap8(GetRValue(cr))<<16) | (ic_cap8(GetGValue(cr))<<8) | ic_cap8(GetBValue(cr)); // COLORREF = BGR + // index is also in reverse in the bits 0 and 2 + unsigned j = (i&0x08) | ((i&0x04)>>2) | (i&0x02) | (i&0x01)<<2; + debug_msg("term: ansi color %d is 0x%06x\n", j, color); + ansi256[j] = color; + } + } + else { + DWORD err = GetLastError(); + debug_msg("term: cannot get console screen buffer: %d %x", err, err); + } + term_start_raw(term); // initialize the hcon_mode + term_end_raw(term,false); +} + +#endif diff --git a/extern/isocline/src/term.h b/extern/isocline/src/term.h new file mode 100644 index 00000000..50bfd968 --- /dev/null +++ b/extern/isocline/src/term.h @@ -0,0 +1,85 @@ +/* ---------------------------------------------------------------------------- + Copyright (c) 2021, Daan Leijen + This is free software; you can redistribute it and/or modify it + under the terms of the MIT License. A copy of the license can be + found in the "LICENSE" file at the root of this distribution. +-----------------------------------------------------------------------------*/ +#pragma once +#ifndef IC_TERM_H +#define IC_TERM_H + +#include "common.h" +#include "tty.h" +#include "stringbuf.h" +#include "attr.h" + +struct term_s; +typedef struct term_s term_t; + +typedef enum buffer_mode_e { + UNBUFFERED, + LINEBUFFERED, + BUFFERED, +} buffer_mode_t; + +// Primitives +ic_private term_t* term_new(alloc_t* mem, tty_t* tty, bool nocolor, bool silent, int fd_out); +ic_private void term_free(term_t* term); + +ic_private bool term_is_interactive(const term_t* term); +ic_private void term_start_raw(term_t* term); +ic_private void term_end_raw(term_t* term, bool force); + +ic_private bool term_enable_beep(term_t* term, bool enable); +ic_private bool term_enable_color(term_t* term, bool enable); + +ic_private void term_flush(term_t* term); +ic_private buffer_mode_t term_set_buffer_mode(term_t* term, buffer_mode_t mode); + +ic_private void term_write_n(term_t* term, const char* s, ssize_t n); +ic_private void term_write(term_t* term, const char* s); +ic_private void term_writeln(term_t* term, const char* s); +ic_private void term_write_char(term_t* term, char c); + +ic_private void term_write_repeat(term_t* term, const char* s, ssize_t count ); +ic_private void term_beep(term_t* term); + +ic_private bool term_update_dim(term_t* term); + +ic_private ssize_t term_get_width(term_t* term); +ic_private ssize_t term_get_height(term_t* term); +ic_private int term_get_color_bits(term_t* term); + +// Helpers +ic_private void term_writef(term_t* term, const char* fmt, ...); +ic_private void term_vwritef(term_t* term, const char* fmt, va_list args); + +ic_private void term_left(term_t* term, ssize_t n); +ic_private void term_right(term_t* term, ssize_t n); +ic_private void term_up(term_t* term, ssize_t n); +ic_private void term_down(term_t* term, ssize_t n); +ic_private void term_start_of_line(term_t* term ); +ic_private void term_clear_line(term_t* term); +ic_private void term_clear_to_end_of_line(term_t* term); +// ic_private void term_clear_lines_to_end(term_t* term); + + +ic_private void term_attr_reset(term_t* term); +ic_private void term_underline(term_t* term, bool on); +ic_private void term_reverse(term_t* term, bool on); +ic_private void term_bold(term_t* term, bool on); +ic_private void term_italic(term_t* term, bool on); + +ic_private void term_color(term_t* term, ic_color_t color); +ic_private void term_bgcolor(term_t* term, ic_color_t color); + +// Formatted output + +ic_private attr_t term_get_attr( const term_t* term ); +ic_private void term_set_attr( term_t* term, attr_t attr ); +ic_private void term_write_formatted( term_t* term, const char* s, const attr_t* attrs ); +ic_private void term_write_formatted_n( term_t* term, const char* s, const attr_t* attrs, ssize_t n ); + +ic_private ic_color_t color_from_ansi256(ssize_t i); + +#endif // IC_TERM_H diff --git a/extern/isocline/src/term_color.c b/extern/isocline/src/term_color.c new file mode 100644 index 00000000..98af3cf4 --- /dev/null +++ b/extern/isocline/src/term_color.c @@ -0,0 +1,371 @@ +/* ---------------------------------------------------------------------------- + Copyright (c) 2021, Daan Leijen + This is free software; you can redistribute it and/or modify it + under the terms of the MIT License. A copy of the license can be + found in the "LICENSE" file at the root of this distribution. +-----------------------------------------------------------------------------*/ + +// This file is included in "term.c" + +//------------------------------------------------------------- +// Standard ANSI palette for 256 colors +//------------------------------------------------------------- + +static uint32_t ansi256[256] = { + // not const as on some platforms (e.g. Windows, xterm) we update the first 16 entries with the actual used colors. + // 0, standard ANSI + 0x000000, 0x800000, 0x008000, 0x808000, 0x000080, 0x800080, + 0x008080, 0xc0c0c0, + // 8, bright ANSI + 0x808080, 0xff0000, 0x00ff00, 0xffff00, 0x0000ff, 0xff00ff, + 0x00ffff, 0xffffff, + // 6x6x6 RGB colors + // 16 + 0x000000, 0x00005f, 0x000087, 0x0000af, 0x0000d7, 0x0000ff, + 0x005f00, 0x005f5f, 0x005f87, 0x005faf, 0x005fd7, 0x005fff, + 0x008700, 0x00875f, 0x008787, 0x0087af, 0x0087d7, 0x0087ff, + 0x00af00, 0x00af5f, 0x00af87, 0x00afaf, 0x00afd7, 0x00afff, + 0x00d700, 0x00d75f, 0x00d787, 0x00d7af, 0x00d7d7, 0x00d7ff, + 0x00ff00, 0x00ff5f, 0x00ff87, 0x00ffaf, 0x00ffd7, 0x00ffff, + // 52 + 0x5f0000, 0x5f005f, 0x5f0087, 0x5f00af, 0x5f00d7, 0x5f00ff, + 0x5f5f00, 0x5f5f5f, 0x5f5f87, 0x5f5faf, 0x5f5fd7, 0x5f5fff, + 0x5f8700, 0x5f875f, 0x5f8787, 0x5f87af, 0x5f87d7, 0x5f87ff, + 0x5faf00, 0x5faf5f, 0x5faf87, 0x5fafaf, 0x5fafd7, 0x5fafff, + 0x5fd700, 0x5fd75f, 0x5fd787, 0x5fd7af, 0x5fd7d7, 0x5fd7ff, + 0x5fff00, 0x5fff5f, 0x5fff87, 0x5fffaf, 0x5fffd7, 0x5fffff, + // 88 + 0x870000, 0x87005f, 0x870087, 0x8700af, 0x8700d7, 0x8700ff, + 0x875f00, 0x875f5f, 0x875f87, 0x875faf, 0x875fd7, 0x875fff, + 0x878700, 0x87875f, 0x878787, 0x8787af, 0x8787d7, 0x8787ff, + 0x87af00, 0x87af5f, 0x87af87, 0x87afaf, 0x87afd7, 0x87afff, + 0x87d700, 0x87d75f, 0x87d787, 0x87d7af, 0x87d7d7, 0x87d7ff, + 0x87ff00, 0x87ff5f, 0x87ff87, 0x87ffaf, 0x87ffd7, 0x87ffff, + // 124 + 0xaf0000, 0xaf005f, 0xaf0087, 0xaf00af, 0xaf00d7, 0xaf00ff, + 0xaf5f00, 0xaf5f5f, 0xaf5f87, 0xaf5faf, 0xaf5fd7, 0xaf5fff, + 0xaf8700, 0xaf875f, 0xaf8787, 0xaf87af, 0xaf87d7, 0xaf87ff, + 0xafaf00, 0xafaf5f, 0xafaf87, 0xafafaf, 0xafafd7, 0xafafff, + 0xafd700, 0xafd75f, 0xafd787, 0xafd7af, 0xafd7d7, 0xafd7ff, + 0xafff00, 0xafff5f, 0xafff87, 0xafffaf, 0xafffd7, 0xafffff, + // 160 + 0xd70000, 0xd7005f, 0xd70087, 0xd700af, 0xd700d7, 0xd700ff, + 0xd75f00, 0xd75f5f, 0xd75f87, 0xd75faf, 0xd75fd7, 0xd75fff, + 0xd78700, 0xd7875f, 0xd78787, 0xd787af, 0xd787d7, 0xd787ff, + 0xd7af00, 0xd7af5f, 0xd7af87, 0xd7afaf, 0xd7afd7, 0xd7afff, + 0xd7d700, 0xd7d75f, 0xd7d787, 0xd7d7af, 0xd7d7d7, 0xd7d7ff, + 0xd7ff00, 0xd7ff5f, 0xd7ff87, 0xd7ffaf, 0xd7ffd7, 0xd7ffff, + // 196 + 0xff0000, 0xff005f, 0xff0087, 0xff00af, 0xff00d7, 0xff00ff, + 0xff5f00, 0xff5f5f, 0xff5f87, 0xff5faf, 0xff5fd7, 0xff5fff, + 0xff8700, 0xff875f, 0xff8787, 0xff87af, 0xff87d7, 0xff87ff, + 0xffaf00, 0xffaf5f, 0xffaf87, 0xffafaf, 0xffafd7, 0xffafff, + 0xffd700, 0xffd75f, 0xffd787, 0xffd7af, 0xffd7d7, 0xffd7ff, + 0xffff00, 0xffff5f, 0xffff87, 0xffffaf, 0xffffd7, 0xffffff, + // 232, gray scale + 0x080808, 0x121212, 0x1c1c1c, 0x262626, 0x303030, 0x3a3a3a, + 0x444444, 0x4e4e4e, 0x585858, 0x626262, 0x6c6c6c, 0x767676, + 0x808080, 0x8a8a8a, 0x949494, 0x9e9e9e, 0xa8a8a8, 0xb2b2b2, + 0xbcbcbc, 0xc6c6c6, 0xd0d0d0, 0xdadada, 0xe4e4e4, 0xeeeeee +}; + + +//------------------------------------------------------------- +// Create colors +//------------------------------------------------------------- + +// Create a color from a 24-bit color value. +ic_private ic_color_t ic_rgb(uint32_t hex) { + return (ic_color_t)(0x1000000 | (hex & 0xFFFFFF)); +} + +// Limit an int to values between 0 and 255. +static uint32_t ic_cap8(ssize_t i) { + return (i < 0 ? 0 : (i > 255 ? 255 : (uint32_t)i)); +} + +// Create a color from a 24-bit color value. +ic_private ic_color_t ic_rgbx(ssize_t r, ssize_t g, ssize_t b) { + return ic_rgb( (ic_cap8(r)<<16) | (ic_cap8(g)<<8) | ic_cap8(b) ); +} + + +//------------------------------------------------------------- +// Match an rgb color to a ansi8, ansi16, or ansi256 +//------------------------------------------------------------- + +static bool color_is_rgb( ic_color_t color ) { + return (color >= IC_RGB(0)); // bit 24 is set for rgb colors +} + +static void color_to_rgb(ic_color_t color, int* r, int* g, int* b) { + assert(color_is_rgb(color)); + *r = ((color >> 16) & 0xFF); + *g = ((color >> 8) & 0xFF); + *b = (color & 0xFF); +} + +ic_private ic_color_t color_from_ansi256(ssize_t i) { + if (i >= 0 && i < 8) { + return (IC_ANSI_BLACK + (uint32_t)i); + } + else if (i >= 8 && i < 16) { + return (IC_ANSI_DARKGRAY + (uint32_t)(i - 8)); + } + else if (i >= 16 && i <= 255) { + return ic_rgb( ansi256[i] ); + } + else if (i == 256) { + return IC_ANSI_DEFAULT; + } + else { + return IC_ANSI_DEFAULT; + } +} + +static bool is_grayish(int r, int g, int b) { + return (abs(r-g) <= 4) && (abs((r+g)/2 - b) <= 4); +} + +static bool is_grayish_color( uint32_t rgb ) { + int r, g, b; + color_to_rgb(IC_RGB(rgb),&r,&g,&b); + return is_grayish(r,g,b); +} + +static int_least32_t sqr(int_least32_t x) { + return x*x; +} + +// Approximation to delta-E CIE color distance using much +// simpler calculations. See . +// This is essentialy weighted euclidean distance but the weight distribution +// depends on how big the "red" component of the color is. +// We do not take the square root as we only need to find +// the minimal distance (and multiply by 256 to increase precision). +// Needs at least 28-bit signed integers to avoid overflow. +static int_least32_t rgb_distance_rmean( uint32_t color, int r2, int g2, int b2 ) { + int r1, g1, b1; + color_to_rgb(IC_RGB(color),&r1,&g1,&b1); + int_least32_t rmean = (r1 + r2) / 2; + int_least32_t dr2 = sqr(r1 - r2); + int_least32_t dg2 = sqr(g1 - g2); + int_least32_t db2 = sqr(b1 - b2); + int_least32_t dist = ((512+rmean)*dr2) + 1024*dg2 + ((767-rmean)*db2); + return dist; +} + +// Another approximation to delta-E CIE color distance using +// simpler calculations. Similar to `rmean` but adds an adjustment factor +// based on the "red/blue" difference. +static int_least32_t rgb_distance_rbmean( uint32_t color, int r2, int g2, int b2 ) { + int r1, g1, b1; + color_to_rgb(IC_RGB(color),&r1,&g1,&b1); + int_least32_t rmean = (r1 + r2) / 2; + int_least32_t dr2 = sqr(r1 - r2); + int_least32_t dg2 = sqr(g1 - g2); + int_least32_t db2 = sqr(b1 - b2); + int_least32_t dist = 2*dr2 + 4*dg2 + 3*db2 + ((rmean*(dr2 - db2))/256); + return dist; +} + + +// Maintain a small cache of recently used colors. Should be short enough to be effectively constant time. +// If we ever use a more expensive color distance method, we may increase the size a bit (64?) +// (Initial zero initialized cache is valid.) +#define RGB_CACHE_LEN (16) +typedef struct rgb_cache_s { + int last; + int indices[RGB_CACHE_LEN]; + ic_color_t colors[RGB_CACHE_LEN]; +} rgb_cache_t; + +// remember a color in the LRU cache +void rgb_remember( rgb_cache_t* cache, ic_color_t color, int idx ) { + if (cache == NULL) return; + cache->colors[cache->last] = color; + cache->indices[cache->last] = idx; + cache->last++; + if (cache->last >= RGB_CACHE_LEN) { cache->last = 0; } +} + +// quick lookup in cache; -1 on failure +int rgb_lookup( const rgb_cache_t* cache, ic_color_t color ) { + if (cache != NULL) { + for(int i = 0; i < RGB_CACHE_LEN; i++) { + if (cache->colors[i] == color) return cache->indices[i]; + } + } + return -1; +} + +// return the index of the closest matching color +static int rgb_match( uint32_t* palette, int start, int len, rgb_cache_t* cache, ic_color_t color ) { + assert(color_is_rgb(color)); + // in cache? + int min = rgb_lookup(cache,color); + if (min >= 0) { + return min; + } + // otherwise find closest color match in the palette + int r, g, b; + color_to_rgb(color,&r,&g,&b); + min = start; + int_least32_t mindist = (INT_LEAST32_MAX)/4; + for(int i = start; i < len; i++) { + //int_least32_t dist = rgb_distance_rbmean(palette[i],r,g,b); + int_least32_t dist = rgb_distance_rmean(palette[i],r,g,b); + if (is_grayish_color(palette[i]) != is_grayish(r, g, b)) { + // with few colors, make it less eager to substitute a gray for a non-gray (or the other way around) + if (len <= 16) { + dist *= 4; + } + else { + dist = (dist/4)*5; + } + } + if (dist < mindist) { + min = i; + mindist = dist; + } + } + rgb_remember(cache,color,min); + return min; +} + + +// Match RGB to an index in the ANSI 256 color table +static int rgb_to_ansi256(ic_color_t color) { + static rgb_cache_t ansi256_cache; + int c = rgb_match(ansi256, 16, 256, &ansi256_cache, color); // not the first 16 ANSI colors as those may be different + //debug_msg("term: rgb %x -> ansi 256: %d\n", color, c ); + return c; +} + +// Match RGB to an ANSI 16 color code (30-37, 90-97) +static int color_to_ansi16(ic_color_t color) { + if (!color_is_rgb(color)) { + return (int)color; + } + else { + static rgb_cache_t ansi16_cache; + int c = rgb_match(ansi256, 0, 16, &ansi16_cache, color); + //debug_msg("term: rgb %x -> ansi 16: %d\n", color, c ); + return (c < 8 ? 30 + c : 90 + c - 8); + } +} + +// Match RGB to an ANSI 16 color code (30-37, 90-97) +// but assuming the bright colors are simulated using 'bold'. +static int color_to_ansi8(ic_color_t color) { + if (!color_is_rgb(color)) { + return (int)color; + } + else { + // match to basic 8 colors first + static rgb_cache_t ansi8_cache; + int c = 30 + rgb_match(ansi256, 0, 8, &ansi8_cache, color); + // and then adjust for brightness + int r, g, b; + color_to_rgb(color,&r,&g,&b); + if (r>=196 || g>=196 || b>=196) c += 60; + //debug_msg("term: rgb %x -> ansi 8: %d\n", color, c ); + return c; + } +} + + +//------------------------------------------------------------- +// Emit color escape codes based on the terminal capability +//------------------------------------------------------------- + +static void fmt_color_ansi8( char* buf, ssize_t len, ic_color_t color, bool bg ) { + int c = color_to_ansi8(color) + (bg ? 10 : 0); + if (c >= 90) { + snprintf(buf, to_size_t(len), IC_CSI "1;%dm", c - 60); + } + else { + snprintf(buf, to_size_t(len), IC_CSI "22;%dm", c ); + } +} + +static void fmt_color_ansi16( char* buf, ssize_t len, ic_color_t color, bool bg ) { + snprintf( buf, to_size_t(len), IC_CSI "%dm", color_to_ansi16(color) + (bg ? 10 : 0) ); +} + +static void fmt_color_ansi256( char* buf, ssize_t len, ic_color_t color, bool bg ) { + if (!color_is_rgb(color)) { + fmt_color_ansi16(buf,len,color,bg); + } + else { + snprintf( buf, to_size_t(len), IC_CSI "%d;5;%dm", (bg ? 48 : 38), rgb_to_ansi256(color) ); + } +} + +static void fmt_color_rgb( char* buf, ssize_t len, ic_color_t color, bool bg ) { + if (!color_is_rgb(color)) { + fmt_color_ansi16(buf,len,color,bg); + } + else { + int r,g,b; + color_to_rgb(color, &r,&g,&b); + snprintf( buf, to_size_t(len), IC_CSI "%d;2;%d;%d;%dm", (bg ? 48 : 38), r, g, b ); + } +} + +static void fmt_color_ex(char* buf, ssize_t len, palette_t palette, ic_color_t color, bool bg) { + if (color == IC_COLOR_NONE || palette == MONOCHROME) return; + if (palette == ANSI8) { + fmt_color_ansi8(buf,len,color,bg); + } + else if (!color_is_rgb(color) || palette == ANSI16) { + fmt_color_ansi16(buf,len,color,bg); + } + else if (palette == ANSI256) { + fmt_color_ansi256(buf,len,color,bg); + } + else { + fmt_color_rgb(buf,len,color,bg); + } +} + +static void term_color_ex(term_t* term, ic_color_t color, bool bg) { + char buf[128+1]; + fmt_color_ex(buf,128,term->palette,color,bg); + term_write(term,buf); +} + +//------------------------------------------------------------- +// Main API functions +//------------------------------------------------------------- + +ic_private void term_color(term_t* term, ic_color_t color) { + term_color_ex(term,color,false); +} + +ic_private void term_bgcolor(term_t* term, ic_color_t color) { + term_color_ex(term,color,true); +} + +ic_private void term_append_color(term_t* term, stringbuf_t* sbuf, ic_color_t color) { + char buf[128+1]; + fmt_color_ex(buf,128,term->palette,color,false); + sbuf_append(sbuf,buf); +} + +ic_private void term_append_bgcolor(term_t* term, stringbuf_t* sbuf, ic_color_t color) { + char buf[128+1]; + fmt_color_ex(buf, 128, term->palette, color, true); + sbuf_append(sbuf, buf); +} + +ic_private int term_get_color_bits(term_t* term) { + switch (term->palette) { + case MONOCHROME: return 1; + case ANSI8: return 3; + case ANSI16: return 4; + case ANSI256: return 8; + case ANSIRGB: return 24; + default: return 4; + } +} diff --git a/extern/isocline/src/tty.c b/extern/isocline/src/tty.c new file mode 100644 index 00000000..09f7aedd --- /dev/null +++ b/extern/isocline/src/tty.c @@ -0,0 +1,889 @@ +/* ---------------------------------------------------------------------------- + Copyright (c) 2021, Daan Leijen + This is free software; you can redistribute it and/or modify it + under the terms of the MIT License. A copy of the license can be + found in the "LICENSE" file at the root of this distribution. +-----------------------------------------------------------------------------*/ +#include +#include +#include +#include +#include + +#include "tty.h" + +#if defined(_WIN32) +#include +#include +#define isatty(fd) _isatty(fd) +#define read(fd,s,n) _read(fd,s,n) +#define STDIN_FILENO 0 +#if (_WIN32_WINNT < 0x0600) +WINBASEAPI ULONGLONG WINAPI GetTickCount64(VOID); +#endif +#else +#include +#include +#include +#include +#include +#include +#if !defined(FIONREAD) +#include +#endif +#endif + +#define TTY_PUSH_MAX (32) + +struct tty_s { + int fd_in; // input handle + bool raw_enabled; // is raw mode enabled? + bool is_utf8; // is the input stream in utf-8 mode? + bool has_term_resize_event; // are resize events generated? + bool term_resize_event; // did a term resize happen? + alloc_t* mem; // memory allocator + code_t pushbuf[TTY_PUSH_MAX]; // push back buffer for full key codes + ssize_t push_count; + uint8_t cpushbuf[TTY_PUSH_MAX]; // low level push back buffer for bytes + ssize_t cpush_count; + long esc_initial_timeout; // initial ms wait to see if ESC starts an escape sequence + long esc_timeout; // follow up delay for characters in an escape sequence + #if defined(_WIN32) + HANDLE hcon; // console input handle + DWORD hcon_orig_mode; // original console mode + #else + struct termios orig_ios; // original terminal settings + struct termios raw_ios; // raw terminal settings + #endif +}; + + +//------------------------------------------------------------- +// Forward declarations of platform dependent primitives below +//------------------------------------------------------------- + +ic_private bool tty_readc_noblock(tty_t* tty, uint8_t* c, long timeout_ms); // does not modify `c` when no input (false is returned) + +//------------------------------------------------------------- +// Key code helpers +//------------------------------------------------------------- + +ic_private bool code_is_ascii_char(code_t c, char* chr ) { + if (c >= ' ' && c <= 0x7F) { + if (chr != NULL) *chr = (char)c; + return true; + } + else { + if (chr != NULL) *chr = 0; + return false; + } +} + +ic_private bool code_is_unicode(code_t c, unicode_t* uchr) { + if (c <= KEY_UNICODE_MAX) { + if (uchr != NULL) *uchr = c; + return true; + } + else { + if (uchr != NULL) *uchr = 0; + return false; + } +} + +ic_private bool code_is_virt_key(code_t c ) { + return (KEY_NO_MODS(c) <= 0x20 || KEY_NO_MODS(c) >= KEY_VIRT); +} + + +//------------------------------------------------------------- +// Read a key code +//------------------------------------------------------------- +static code_t modify_code( code_t code ); + +static code_t tty_read_utf8( tty_t* tty, uint8_t c0 ) { + uint8_t buf[5]; + memset(buf, 0, 5); + + // try to read as many bytes as potentially needed + buf[0] = c0; + ssize_t count = 1; + if (c0 > 0x7F) { + if (tty_readc_noblock(tty, buf+count, tty->esc_timeout)) { + count++; + if (c0 > 0xDF) { + if (tty_readc_noblock(tty, buf+count, tty->esc_timeout)) { + count++; + if (c0 > 0xEF) { + if (tty_readc_noblock(tty, buf+count, tty->esc_timeout)) { + count++; + } + } + } + } + } + } + + buf[count] = 0; + debug_msg("tty: read utf8: count: %zd: %02x,%02x,%02x,%02x", count, buf[0], buf[1], buf[2], buf[3]); + + // decode the utf8 to unicode + ssize_t read = 0; + code_t code = key_unicode(unicode_from_qutf8(buf, count, &read)); + + // push back unused bytes (in the case of invalid utf8) + while (count > read) { + count--; + if (count >= 0 && count <= 4) { // to help the static analyzer + tty_cpush_char(tty, buf[count]); + } + } + return code; +} + +// pop a code from the pushback buffer. +static bool tty_code_pop(tty_t* tty, code_t* code); + + +// read a single char/key +ic_private bool tty_read_timeout(tty_t* tty, long timeout_ms, code_t* code) +{ + // is there a push_count back code? + if (tty_code_pop(tty,code)) { + return code; + } + + // read a single char/byte from a character stream + uint8_t c; + if (!tty_readc_noblock(tty, &c, timeout_ms)) return false; + + if (c == KEY_ESC) { + // escape sequence? + *code = tty_read_esc(tty, tty->esc_initial_timeout, tty->esc_timeout); + } + else if (c <= 0x7F) { + // ascii + *code = key_unicode(c); + } + else if (tty->is_utf8) { + // utf8 sequence + *code = tty_read_utf8(tty,c); + } + else { + // c >= 0x80 but tty is not utf8; use raw plane so we can translate it back in the end + *code = key_unicode( unicode_from_raw(c) ); + } + + *code = modify_code(*code); + return true; +} + +// Transform virtual keys to be more portable across platforms +static code_t modify_code( code_t code ) { + code_t key = KEY_NO_MODS(code); + code_t mods = KEY_MODS(code); + debug_msg( "tty: readc %s%s%s 0x%03x ('%c')\n", + mods&KEY_MOD_SHIFT ? "shift+" : "", mods&KEY_MOD_CTRL ? "ctrl+" : "", mods&KEY_MOD_ALT ? "alt+" : "", + key, (key >= ' ' && key <= '~' ? key : ' ')); + + // treat KEY_RUBOUT (0x7F) as KEY_BACKSP + if (key == KEY_RUBOUT) { + code = KEY_BACKSP | mods; + } + // ctrl+'_' is translated to '\x1F' on Linux, translate it back + else if (key == key_char('\x1F') && (mods & KEY_MOD_ALT) == 0) { + key = '_'; + code = WITH_CTRL(key_char('_')); + } + // treat ctrl/shift + enter always as KEY_LINEFEED for portability + else if (key == KEY_ENTER && (mods == KEY_MOD_SHIFT || mods == KEY_MOD_ALT || mods == KEY_MOD_CTRL)) { + code = KEY_LINEFEED; + } + // treat ctrl+tab always as shift+tab for portability + else if (code == WITH_CTRL(KEY_TAB)) { + code = KEY_SHIFT_TAB; + } + // treat ctrl+end/alt+>/alt-down and ctrl+home/alt+') || code == WITH_CTRL(KEY_END)) { + code = KEY_PAGEDOWN; + } + else if (code == WITH_ALT(KEY_UP) || code == WITH_ALT('<') || code == WITH_CTRL(KEY_HOME)) { + code = KEY_PAGEUP; + } + + // treat C0 codes without KEY_MOD_CTRL + if (key < ' ' && (mods&KEY_MOD_CTRL) != 0) { + code &= ~KEY_MOD_CTRL; + } + + return code; +} + + +// read a single char/key +ic_private code_t tty_read(tty_t* tty) +{ + code_t code; + if (!tty_read_timeout(tty, -1, &code)) return KEY_NONE; + return code; +} + +//------------------------------------------------------------- +// Read back an ANSI query response +//------------------------------------------------------------- + +ic_private bool tty_read_esc_response(tty_t* tty, char esc_start, bool final_st, char* buf, ssize_t buflen ) +{ + buf[0] = 0; + ssize_t len = 0; + uint8_t c = 0; + if (!tty_readc_noblock(tty, &c, 2*tty->esc_initial_timeout) || c != '\x1B') { + debug_msg("initial esc response failed: 0x%02x\n", c); + return false; + } + if (!tty_readc_noblock(tty, &c, tty->esc_timeout) || (c != esc_start)) return false; + while( len < buflen ) { + if (!tty_readc_noblock(tty, &c, tty->esc_timeout)) return false; + if (final_st) { + // OSC is terminated by BELL, or ESC \ (ST) (and STX) + if (c=='\x07' || c=='\x02') { + break; + } + else if (c=='\x1B') { + uint8_t c1; + if (!tty_readc_noblock(tty, &c1, tty->esc_timeout)) return false; + if (c1=='\\') break; + tty_cpush_char(tty,c1); + } + } + else { + if (c == '\x02') { // STX + break; + } + else if (!((c >= '0' && c <= '9') || strchr("<=>?;:",c) != NULL)) { + buf[len++] = (char)c; // for non-OSC save the terminating character + break; + } + } + buf[len++] = (char)c; + } + buf[len] = 0; + debug_msg("tty: escape query response: %s\n", buf); + return true; +} + +//------------------------------------------------------------- +// High level code pushback +//------------------------------------------------------------- + +static bool tty_code_pop( tty_t* tty, code_t* code ) { + if (tty->push_count <= 0) return false; + tty->push_count--; + *code = tty->pushbuf[tty->push_count]; + return true; +} + +ic_private void tty_code_pushback( tty_t* tty, code_t c ) { + // note: must be signal safe + if (tty->push_count >= TTY_PUSH_MAX) return; + tty->pushbuf[tty->push_count] = c; + tty->push_count++; +} + + +//------------------------------------------------------------- +// low-level character pushback (for escape sequences and windows) +//------------------------------------------------------------- + +ic_private bool tty_cpop(tty_t* tty, uint8_t* c) { + if (tty->cpush_count <= 0) { // do not modify c on failure (see `tty_decode_unicode`) + return false; + } + else { + tty->cpush_count--; + *c = tty->cpushbuf[tty->cpush_count]; + return true; + } +} + +static void tty_cpush(tty_t* tty, const char* s) { + ssize_t len = ic_strlen(s); + if (tty->push_count + len > TTY_PUSH_MAX) { + debug_msg("tty: cpush buffer full! (pushing %s)\n", s); + assert(false); + return; + } + for (ssize_t i = 0; i < len; i++) { + tty->cpushbuf[tty->cpush_count + i] = (uint8_t)( s[len - i - 1] ); + } + tty->cpush_count += len; + return; +} + +// convenience function for small sequences +static void tty_cpushf(tty_t* tty, const char* fmt, ...) { + va_list args; + va_start(args,fmt); + char buf[TTY_PUSH_MAX+1]; + vsnprintf(buf,TTY_PUSH_MAX,fmt,args); + buf[TTY_PUSH_MAX] = 0; + tty_cpush(tty,buf); + va_end(args); + return; +} + +ic_private void tty_cpush_char(tty_t* tty, uint8_t c) { + uint8_t buf[2]; + buf[0] = c; + buf[1] = 0; + tty_cpush(tty, (const char*)buf); +} + + +//------------------------------------------------------------- +// Push escape codes (used on Windows to insert keys) +//------------------------------------------------------------- + +static unsigned csi_mods(code_t mods) { + unsigned m = 1; + if (mods&KEY_MOD_SHIFT) m += 1; + if (mods&KEY_MOD_ALT) m += 2; + if (mods&KEY_MOD_CTRL) m += 4; + return m; +} + +// Push ESC [ ; ~ +static void tty_cpush_csi_vt( tty_t* tty, code_t mods, uint32_t vtcode ) { + tty_cpushf(tty,"\x1B[%u;%u~", vtcode, csi_mods(mods) ); +} + +// push ESC [ 1 ; +static void tty_cpush_csi_xterm( tty_t* tty, code_t mods, char xcode ) { + tty_cpushf(tty,"\x1B[1;%u%c", csi_mods(mods), xcode ); +} + +// push ESC [ ; u +static void tty_cpush_csi_unicode( tty_t* tty, code_t mods, uint32_t unicode ) { + if ((unicode < 0x80 && mods == 0) || + (mods == KEY_MOD_CTRL && unicode < ' ' && unicode != KEY_TAB && unicode != KEY_ENTER + && unicode != KEY_LINEFEED && unicode != KEY_BACKSP) || + (mods == KEY_MOD_SHIFT && unicode >= ' ' && unicode <= KEY_RUBOUT)) { + tty_cpush_char(tty,(uint8_t)unicode); + } + else { + tty_cpushf(tty,"\x1B[%u;%uu", unicode, csi_mods(mods) ); + } +} + +//------------------------------------------------------------- +// Init +//------------------------------------------------------------- + +static bool tty_init_raw(tty_t* tty); +static void tty_done_raw(tty_t* tty); + +static bool tty_init_utf8(tty_t* tty) { + #ifdef _WIN32 + tty->is_utf8 = true; + #else + const char* loc = setlocale(LC_ALL,""); + tty->is_utf8 = (ic_icontains(loc,"UTF-8") || ic_icontains(loc,"utf8") || ic_stricmp(loc,"C") == 0); + debug_msg("tty: utf8: %s (loc=%s)\n", tty->is_utf8 ? "true" : "false", loc); + #endif + return true; +} + +ic_private tty_t* tty_new(alloc_t* mem, int fd_in) +{ + tty_t* tty = mem_zalloc_tp(mem, tty_t); + tty->mem = mem; + tty->fd_in = (fd_in < 0 ? STDIN_FILENO : fd_in); + #if defined(__APPLE__) + tty->esc_initial_timeout = 200; // apple use ESC+ for alt- + #else + tty->esc_initial_timeout = 100; + #endif + tty->esc_timeout = 10; + if (!(isatty(tty->fd_in) && tty_init_raw(tty) && tty_init_utf8(tty))) { + tty_free(tty); + return NULL; + } + return tty; +} + +ic_private void tty_free(tty_t* tty) { + if (tty==NULL) return; + tty_end_raw(tty); + tty_done_raw(tty); + mem_free(tty->mem,tty); +} + +ic_private bool tty_is_utf8(const tty_t* tty) { + if (tty == NULL) return true; + return (tty->is_utf8); +} + +ic_private bool tty_term_resize_event(tty_t* tty) { + if (tty == NULL) return true; + if (tty->has_term_resize_event) { + if (!tty->term_resize_event) return false; + tty->term_resize_event = false; // reset. + } + return true; // always return true on systems without a resize event (more expensive but still ok) +} + +ic_private void tty_set_esc_delay(tty_t* tty, long initial_delay_ms, long followup_delay_ms) { + tty->esc_initial_timeout = (initial_delay_ms < 0 ? 0 : (initial_delay_ms > 1000 ? 1000 : initial_delay_ms)); + tty->esc_timeout = (followup_delay_ms < 0 ? 0 : (followup_delay_ms > 1000 ? 1000 : followup_delay_ms)); +} + +//------------------------------------------------------------- +// Unix +//------------------------------------------------------------- +#if !defined(_WIN32) + +static bool tty_readc_blocking(tty_t* tty, uint8_t* c) { + if (tty_cpop(tty,c)) return true; + *c = 0; + ssize_t nread = read(tty->fd_in, (char*)c, 1); + if (nread < 0 && errno == EINTR) { + // can happen on SIGWINCH signal for terminal resize + } + return (nread == 1); +} + + +// non blocking read -- with a small timeout used for reading escape sequences. +ic_private bool tty_readc_noblock(tty_t* tty, uint8_t* c, long timeout_ms) +{ + // in our pushback buffer? + if (tty_cpop(tty, c)) return true; + + // blocking read? + if (timeout_ms < 0) { + return tty_readc_blocking(tty,c); + } + + // if supported, peek first if any char is available. + #if defined(FIONREAD) + { int navail = 0; + if (ioctl(0, FIONREAD, &navail) == 0) { + if (navail >= 1) { + return tty_readc_blocking(tty, c); + } + else if (timeout_ms == 0) { + return false; // return early if there is no input available (with a zero timeout) + } + } + } + #endif + + // otherwise block for at most timeout milliseconds + #if defined(FD_SET) + // we can use select to detect when input becomes available + fd_set readset; + struct timeval time; + FD_ZERO(&readset); + FD_SET(tty->fd_in, &readset); + time.tv_sec = (timeout_ms > 0 ? timeout_ms / 1000 : 0); + time.tv_usec = (timeout_ms > 0 ? 1000*(timeout_ms % 1000) : 0); + if (select(tty->fd_in + 1, &readset, NULL, NULL, &time) == 1) { + // input available + return tty_readc_blocking(tty, c); + } + #else + // no select, we cannot timeout; use usleeps :-( + // todo: this seems very rare nowadays; should be even support this? + do { + // peek ahead if possible + #if defined(FIONREAD) + int navail = 0; + if (ioctl(0, FIONREAD, &navail) == 0 && navail >= 1) { + return tty_readc_blocking(tty, c); + } + #elif defined(O_NONBLOCK) + // use a temporary non-blocking read mode + int fstatus = fcntl(tty->fd_in, F_GETFL, 0); + if (fstatus != -1) { + if (fcntl(tty->fd_in, F_SETFL, (fstatus | O_NONBLOCK)) != -1) { + char buf[2] = { 0, 0 }; + ssize_t nread = read(tty->fd_in, buf, 1); + fcntl(tty->fd_in, F_SETFL, fstatus); + if (nread >= 1) { + *c = (uint8_t)buf[0]; + return true; + } + } + } + #else + #error "define an nonblocking read for this platform" + #endif + // and sleep a bit + if (timeout_ms > 0) { + usleep(50*1000L); // sleep at most 0.05s at a time + timeout_ms -= 100; + if (timeout_ms < 0) { timeout_ms = 0; } + } + } + while (timeout_ms > 0); + #endif + return false; +} + +#if defined(TIOCSTI) +ic_private bool tty_async_stop(const tty_t* tty) { + // insert ^C in the input stream + char c = KEY_CTRL_C; + return (ioctl(tty->fd_in, TIOCSTI, &c) >= 0); +} +#else +ic_private bool tty_async_stop(const tty_t* tty) { + return false; +} +#endif + +// We install various signal handlers to restore the terminal settings +// in case of a terminating signal. This is also used to catch terminal window resizes. +// This is not strictly needed so this can be disabled on +// (older) platforms that do not support signal handling well. +#if defined(SIGWINCH) && defined(SA_RESTART) // ensure basic signal functionality is defined + +// store the tty in a global so we access it on unexpected termination +static tty_t* sig_tty; // = NULL + +// Catch all termination signals (and SIGWINCH) +typedef struct signal_handler_s { + int signum; + union { + int _avoid_warning; + struct sigaction previous; + } action; +} signal_handler_t; + +static signal_handler_t sighandlers[] = { + { SIGWINCH, {0} }, + { SIGTERM , {0} }, + { SIGINT , {0} }, + { SIGQUIT , {0} }, + { SIGHUP , {0} }, + { SIGSEGV , {0} }, + { SIGTRAP , {0} }, + { SIGBUS , {0} }, + { SIGTSTP , {0} }, + { SIGTTIN , {0} }, + { SIGTTOU , {0} }, + { 0 , {0} } +}; + +static bool sigaction_is_valid( struct sigaction* sa ) { + return (sa->sa_sigaction != NULL && sa->sa_handler != SIG_DFL && sa->sa_handler != SIG_IGN); +} + +// Generic signal handler +static void sig_handler(int signum, siginfo_t* siginfo, void* uap ) { + if (signum == SIGWINCH) { + if (sig_tty != NULL) { + sig_tty->term_resize_event = true; + } + } + else { + // the rest are termination signals; restore the terminal mode. (`tcsetattr` is signal-safe) + if (sig_tty != NULL && sig_tty->raw_enabled) { + tcsetattr(sig_tty->fd_in, TCSAFLUSH, &sig_tty->orig_ios); + sig_tty->raw_enabled = false; + } + } + // call previous handler + signal_handler_t* sh = sighandlers; + while( sh->signum != 0 && sh->signum != signum) { sh++; } + if (sh->signum == signum) { + if (sigaction_is_valid(&sh->action.previous)) { + (sh->action.previous.sa_sigaction)(signum, siginfo, uap); + } + } +} + +static void signals_install(tty_t* tty) { + sig_tty = tty; + // generic signal handler + struct sigaction handler; + memset(&handler,0,sizeof(handler)); + sigemptyset(&handler.sa_mask); + handler.sa_sigaction = &sig_handler; + handler.sa_flags = SA_RESTART; + // install for all signals + for( signal_handler_t* sh = sighandlers; sh->signum != 0; sh++ ) { + if (sigaction( sh->signum, NULL, &sh->action.previous) == 0) { // get previous + if (sh->action.previous.sa_handler != SIG_IGN) { // if not to be ignored + if (sigaction( sh->signum, &handler, &sh->action.previous ) < 0) { // install our handler + sh->action.previous.sa_sigaction = NULL; // do not restore on error + } + else if (sh->signum == SIGWINCH) { + sig_tty->has_term_resize_event = true; + }; + } + } + } +} + +static void signals_restore(void) { + // restore all signal handlers + for( signal_handler_t* sh = sighandlers; sh->signum != 0; sh++ ) { + if (sigaction_is_valid(&sh->action.previous)) { + sigaction( sh->signum, &sh->action.previous, NULL ); + }; + } + sig_tty = NULL; +} + +#else +static void signals_install(tty_t* tty) { + ic_unused(tty); + // nothing +} +static void signals_restore(void) { + // nothing +} + +#endif + +ic_private bool tty_start_raw(tty_t* tty) { + if (tty == NULL) return false; + if (tty->raw_enabled) return true; + if (tcsetattr(tty->fd_in,TCSAFLUSH,&tty->raw_ios) < 0) return false; + tty->raw_enabled = true; + return true; +} + +ic_private void tty_end_raw(tty_t* tty) { + if (tty == NULL) return; + if (!tty->raw_enabled) return; + tty->cpush_count = 0; + if (tcsetattr(tty->fd_in,TCSAFLUSH,&tty->orig_ios) < 0) return; + tty->raw_enabled = false; +} + +static bool tty_init_raw(tty_t* tty) +{ + // Set input to raw mode. See . + if (tcgetattr(tty->fd_in,&tty->orig_ios) == -1) return false; + tty->raw_ios = tty->orig_ios; + // input: no break signal, no \r to \n, no parity check, no 8-bit to 7-bit, no flow control + tty->raw_ios.c_iflag &= ~(unsigned long)(BRKINT | ICRNL | INPCK | ISTRIP | IXON); + // control: allow 8-bit + tty->raw_ios.c_cflag |= CS8; + // local: no echo, no line-by-line (canonical), no extended input processing, no signals for ^z,^c + tty->raw_ios.c_lflag &= ~(unsigned long)(ECHO | ICANON | IEXTEN | ISIG); + // 1 byte at a time, no delay + tty->raw_ios.c_cc[VTIME] = 0; + tty->raw_ios.c_cc[VMIN] = 1; + + // store in global so our signal handlers can restore the terminal mode + signals_install(tty); + + return true; +} + +static void tty_done_raw(tty_t* tty) { + ic_unused(tty); + signals_restore(); +} + + +#else + +//------------------------------------------------------------- +// Windows +// For best portability we push CSI escape sequences directly +// to the character stream (instead of returning key codes). +//------------------------------------------------------------- + +static void tty_waitc_console(tty_t* tty, long timeout_ms); + +ic_private bool tty_readc_noblock(tty_t* tty, uint8_t* c, long timeout_ms) { // don't modify `c` if there is no input + // in our pushback buffer? + if (tty_cpop(tty, c)) return true; + // any events in the input queue? + tty_waitc_console(tty, timeout_ms); + return tty_cpop(tty, c); +} + +// Read from the console input events and push escape codes into the tty cbuffer. +static void tty_waitc_console(tty_t* tty, long timeout_ms) +{ + // wait for a key down event + INPUT_RECORD inp; + DWORD count; + uint32_t surrogate_hi = 0; + while (true) { + // check if there are events if in non-blocking timeout mode + if (timeout_ms >= 0) { + // first peek ahead + if (!GetNumberOfConsoleInputEvents(tty->hcon, &count)) return; + if (count == 0) { + if (timeout_ms == 0) { + // out of time + return; + } + else { + // wait for input events for at most timeout milli seconds + ULONGLONG start_ms = GetTickCount64(); + DWORD res = WaitForSingleObject(tty->hcon, (DWORD)timeout_ms); + switch (res) { + case WAIT_OBJECT_0: { + // input is available, decrease our timeout + ULONGLONG waited_ms = (GetTickCount64() - start_ms); + timeout_ms -= (long)waited_ms; + if (timeout_ms < 0) { + timeout_ms = 0; + } + break; + } + case WAIT_TIMEOUT: + case WAIT_ABANDONED: + case WAIT_FAILED: + default: + return; + } + } + } + } + + // (blocking) Read from the input + if (!ReadConsoleInputW(tty->hcon, &inp, 1, &count)) return; + if (count != 1) return; + + // resize event? + if (inp.EventType == WINDOW_BUFFER_SIZE_EVENT) { + tty->term_resize_event = true; + continue; + } + + // wait for key down events + if (inp.EventType != KEY_EVENT) continue; + + // the modifier state + DWORD modstate = inp.Event.KeyEvent.dwControlKeyState; + + // we need to handle shift up events separately + if (!inp.Event.KeyEvent.bKeyDown && inp.Event.KeyEvent.wVirtualKeyCode == VK_SHIFT) { + modstate &= (DWORD)~SHIFT_PRESSED; + } + + // ignore AltGr + DWORD altgr = LEFT_CTRL_PRESSED | RIGHT_ALT_PRESSED; + if ((modstate & altgr) == altgr) { modstate &= ~altgr; } + + + // get modifiers + code_t mods = 0; + if ((modstate & ( RIGHT_CTRL_PRESSED | LEFT_CTRL_PRESSED )) != 0) mods |= KEY_MOD_CTRL; + if ((modstate & ( RIGHT_ALT_PRESSED | LEFT_ALT_PRESSED )) != 0) mods |= KEY_MOD_ALT; + if ((modstate & SHIFT_PRESSED) != 0) mods |= KEY_MOD_SHIFT; + + // virtual keys + uint32_t chr = (uint32_t)inp.Event.KeyEvent.uChar.UnicodeChar; + WORD virt = inp.Event.KeyEvent.wVirtualKeyCode; + debug_msg("tty: console %s: %s%s%s virt 0x%04x, chr 0x%04x ('%c')\n", inp.Event.KeyEvent.bKeyDown ? "down" : "up", mods&KEY_MOD_CTRL ? "ctrl-" : "", mods&KEY_MOD_ALT ? "alt-" : "", mods&KEY_MOD_SHIFT ? "shift-" : "", virt, chr, chr); + + // only process keydown events (except for Alt-up which is used for unicode pasting...) + if (!inp.Event.KeyEvent.bKeyDown && virt != VK_MENU) { + continue; + } + + if (chr == 0) { + switch (virt) { + case VK_UP: tty_cpush_csi_xterm(tty, mods, 'A'); return; + case VK_DOWN: tty_cpush_csi_xterm(tty, mods, 'B'); return; + case VK_RIGHT: tty_cpush_csi_xterm(tty, mods, 'C'); return; + case VK_LEFT: tty_cpush_csi_xterm(tty, mods, 'D'); return; + case VK_END: tty_cpush_csi_xterm(tty, mods, 'F'); return; + case VK_HOME: tty_cpush_csi_xterm(tty, mods, 'H'); return; + case VK_DELETE: tty_cpush_csi_vt(tty,mods,3); return; + case VK_PRIOR: tty_cpush_csi_vt(tty,mods,5); return; //page up + case VK_NEXT: tty_cpush_csi_vt(tty,mods,6); return; //page down + case VK_TAB: tty_cpush_csi_unicode(tty,mods,9); return; + case VK_RETURN: tty_cpush_csi_unicode(tty,mods,13); return; + default: { + uint32_t vtcode = 0; + if (virt >= VK_F1 && virt <= VK_F5) { + vtcode = 10 + (virt - VK_F1); + } + else if (virt >= VK_F6 && virt <= VK_F10) { + vtcode = 17 + (virt - VK_F6); + } + else if (virt >= VK_F11 && virt <= VK_F12) { + vtcode = 13 + (virt - VK_F11); + } + if (vtcode > 0) { + tty_cpush_csi_vt(tty,mods,vtcode); + return; + } + } + } + // ignore other control keys (shift etc). + } + // high surrogate pair + else if (chr >= 0xD800 && chr <= 0xDBFF) { + surrogate_hi = (chr - 0xD800); + } + // low surrogate pair + else if (chr >= 0xDC00 && chr <= 0xDFFF) { + chr = ((surrogate_hi << 10) + (chr - 0xDC00) + 0x10000); + tty_cpush_csi_unicode(tty,mods,chr); + surrogate_hi = 0; + return; + } + // regular character + else { + tty_cpush_csi_unicode(tty,mods,chr); + return; + } + } +} + +ic_private bool tty_async_stop(const tty_t* tty) { + // send ^c + INPUT_RECORD events[2]; + memset(events, 0, 2*sizeof(INPUT_RECORD)); + events[0].EventType = KEY_EVENT; + events[0].Event.KeyEvent.bKeyDown = TRUE; + events[0].Event.KeyEvent.uChar.AsciiChar = KEY_CTRL_C; + events[1] = events[0]; + events[1].Event.KeyEvent.bKeyDown = FALSE; + DWORD nwritten = 0; + WriteConsoleInput(tty->hcon, events, 2, &nwritten); + return (nwritten == 2); +} + +ic_private bool tty_start_raw(tty_t* tty) { + if (tty->raw_enabled) return true; + GetConsoleMode(tty->hcon,&tty->hcon_orig_mode); + DWORD mode = ENABLE_QUICK_EDIT_MODE // cut&paste allowed + | ENABLE_WINDOW_INPUT // to catch resize events + // | ENABLE_VIRTUAL_TERMINAL_INPUT + // | ENABLE_PROCESSED_INPUT + ; + SetConsoleMode(tty->hcon, mode ); + tty->raw_enabled = true; + return true; +} + +ic_private void tty_end_raw(tty_t* tty) { + if (!tty->raw_enabled) return; + SetConsoleMode(tty->hcon, tty->hcon_orig_mode ); + tty->raw_enabled = false; +} + +static bool tty_init_raw(tty_t* tty) { + tty->hcon = GetStdHandle( STD_INPUT_HANDLE ); + tty->has_term_resize_event = true; + return true; +} + +static void tty_done_raw(tty_t* tty) { + ic_unused(tty); +} + +#endif + + diff --git a/extern/isocline/src/tty.h b/extern/isocline/src/tty.h new file mode 100644 index 00000000..a0062bf3 --- /dev/null +++ b/extern/isocline/src/tty.h @@ -0,0 +1,160 @@ +/* ---------------------------------------------------------------------------- + Copyright (c) 2021, Daan Leijen + This is free software; you can redistribute it and/or modify it + under the terms of the MIT License. A copy of the license can be + found in the "LICENSE" file at the root of this distribution. +-----------------------------------------------------------------------------*/ +#pragma once +#ifndef IC_TTY_H +#define IC_TTY_H + +#include "common.h" + +//------------------------------------------------------------- +// TTY/Keyboard input +//------------------------------------------------------------- + +// Key code +typedef uint32_t code_t; + +// TTY interface +struct tty_s; +typedef struct tty_s tty_t; + + +ic_private tty_t* tty_new(alloc_t* mem, int fd_in); +ic_private void tty_free(tty_t* tty); + +ic_private bool tty_is_utf8(const tty_t* tty); +ic_private bool tty_start_raw(tty_t* tty); +ic_private void tty_end_raw(tty_t* tty); +ic_private code_t tty_read(tty_t* tty); +ic_private bool tty_read_timeout(tty_t* tty, long timeout_ms, code_t* c ); + +ic_private void tty_code_pushback( tty_t* tty, code_t c ); +ic_private bool code_is_ascii_char(code_t c, char* chr ); +ic_private bool code_is_unicode(code_t c, unicode_t* uchr); +ic_private bool code_is_virt_key(code_t c ); + +ic_private bool tty_term_resize_event(tty_t* tty); // did the terminal resize? +ic_private bool tty_async_stop(const tty_t* tty); // unblock the read asynchronously +ic_private void tty_set_esc_delay(tty_t* tty, long initial_delay_ms, long followup_delay_ms); + +// shared between tty.c and tty_esc.c: low level character push +ic_private void tty_cpush_char(tty_t* tty, uint8_t c); +ic_private bool tty_cpop(tty_t* tty, uint8_t* c); +ic_private bool tty_readc_noblock(tty_t* tty, uint8_t* c, long timeout_ms); +ic_private code_t tty_read_esc(tty_t* tty, long esc_initial_timeout, long esc_timeout); // in tty_esc.c + +// used by term.c to read back ANSI escape responses +ic_private bool tty_read_esc_response(tty_t* tty, char esc_start, bool final_st, char* buf, ssize_t buflen ); + + +//------------------------------------------------------------- +// Key codes: a code_t is 32 bits. +// we use the bottom 24 (nah, 21) bits for unicode (up to x0010FFFF) +// The codes after x01000000 are for virtual keys +// and events use x02000000. +// The top 4 bits are used for modifiers. +//------------------------------------------------------------- + +static inline code_t key_char( char c ) { + // careful about signed character conversion (negative char ~> 0x80 - 0xFF) + return ((uint8_t)c); +} + +static inline code_t key_unicode( unicode_t u ) { + return u; +} + + +#define KEY_MOD_SHIFT (0x10000000U) +#define KEY_MOD_ALT (0x20000000U) +#define KEY_MOD_CTRL (0x40000000U) + +#define KEY_NO_MODS(k) (k & 0x0FFFFFFFU) +#define KEY_MODS(k) (k & 0xF0000000U) + +#define WITH_SHIFT(x) (x | KEY_MOD_SHIFT) +#define WITH_ALT(x) (x | KEY_MOD_ALT) +#define WITH_CTRL(x) (x | KEY_MOD_CTRL) + +#define KEY_NONE (0) +#define KEY_CTRL_A (1) +#define KEY_CTRL_B (2) +#define KEY_CTRL_C (3) +#define KEY_CTRL_D (4) +#define KEY_CTRL_E (5) +#define KEY_CTRL_F (6) +#define KEY_BELL (7) +#define KEY_BACKSP (8) +#define KEY_TAB (9) +#define KEY_LINEFEED (10) // ctrl/shift + enter is considered KEY_LINEFEED +#define KEY_CTRL_K (11) +#define KEY_CTRL_L (12) +#define KEY_ENTER (13) +#define KEY_CTRL_N (14) +#define KEY_CTRL_O (15) +#define KEY_CTRL_P (16) +#define KEY_CTRL_Q (17) +#define KEY_CTRL_R (18) +#define KEY_CTRL_S (19) +#define KEY_CTRL_T (20) +#define KEY_CTRL_U (21) +#define KEY_CTRL_V (22) +#define KEY_CTRL_W (23) +#define KEY_CTRL_X (24) +#define KEY_CTRL_Y (25) +#define KEY_CTRL_Z (26) +#define KEY_ESC (27) +#define KEY_SPACE (32) +#define KEY_RUBOUT (127) // always translated to KEY_BACKSP +#define KEY_UNICODE_MAX (0x0010FFFFU) + + +#define KEY_VIRT (0x01000000U) +#define KEY_UP (KEY_VIRT+0) +#define KEY_DOWN (KEY_VIRT+1) +#define KEY_LEFT (KEY_VIRT+2) +#define KEY_RIGHT (KEY_VIRT+3) +#define KEY_HOME (KEY_VIRT+4) +#define KEY_END (KEY_VIRT+5) +#define KEY_DEL (KEY_VIRT+6) +#define KEY_PAGEUP (KEY_VIRT+7) +#define KEY_PAGEDOWN (KEY_VIRT+8) +#define KEY_INS (KEY_VIRT+9) + +#define KEY_F1 (KEY_VIRT+11) +#define KEY_F2 (KEY_VIRT+12) +#define KEY_F3 (KEY_VIRT+13) +#define KEY_F4 (KEY_VIRT+14) +#define KEY_F5 (KEY_VIRT+15) +#define KEY_F6 (KEY_VIRT+16) +#define KEY_F7 (KEY_VIRT+17) +#define KEY_F8 (KEY_VIRT+18) +#define KEY_F9 (KEY_VIRT+19) +#define KEY_F10 (KEY_VIRT+20) +#define KEY_F11 (KEY_VIRT+21) +#define KEY_F12 (KEY_VIRT+22) +#define KEY_F(n) (KEY_F1 + (n) - 1) + +#define KEY_EVENT_BASE (0x02000000U) +#define KEY_EVENT_RESIZE (KEY_EVENT_BASE+1) +#define KEY_EVENT_AUTOTAB (KEY_EVENT_BASE+2) +#define KEY_EVENT_STOP (KEY_EVENT_BASE+3) + +// Convenience +#define KEY_CTRL_UP (WITH_CTRL(KEY_UP)) +#define KEY_CTRL_DOWN (WITH_CTRL(KEY_DOWN)) +#define KEY_CTRL_LEFT (WITH_CTRL(KEY_LEFT)) +#define KEY_CTRL_RIGHT (WITH_CTRL(KEY_RIGHT)) +#define KEY_CTRL_HOME (WITH_CTRL(KEY_HOME)) +#define KEY_CTRL_END (WITH_CTRL(KEY_END)) +#define KEY_CTRL_DEL (WITH_CTRL(KEY_DEL)) +#define KEY_CTRL_PAGEUP (WITH_CTRL(KEY_PAGEUP)) +#define KEY_CTRL_PAGEDOWN (WITH_CTRL(KEY_PAGEDOWN))) +#define KEY_CTRL_INS (WITH_CTRL(KEY_INS)) + +#define KEY_SHIFT_TAB (WITH_SHIFT(KEY_TAB)) + +#endif // IC_TTY_H diff --git a/extern/isocline/src/tty_esc.c b/extern/isocline/src/tty_esc.c new file mode 100644 index 00000000..0ac8761d --- /dev/null +++ b/extern/isocline/src/tty_esc.c @@ -0,0 +1,401 @@ +/* ---------------------------------------------------------------------------- + Copyright (c) 2021, Daan Leijen + This is free software; you can redistribute it and/or modify it + under the terms of the MIT License. A copy of the license can be + found in the "LICENSE" file at the root of this distribution. +-----------------------------------------------------------------------------*/ +#include +#include "tty.h" + +/*------------------------------------------------------------- +Decoding escape sequences to key codes. +This is a bit tricky there are many variants to encode keys as escape sequences, see for example: +- . +- +- +- +- + +Generally, for our purposes we accept a subset of escape sequences as: + + escseq ::= ESC + | ESC char + | ESC start special? (number (';' modifiers)?)? final + +where: + char ::= [\x00-\xFF] # any character + special ::= [:<=>?] + number ::= [0-9+] + modifiers ::= [1-9] + intermediate ::= [\x20-\x2F] # !"#$%&'()*+,-./ + final ::= [\x40-\x7F] # @A–Z[\]^_`a–z{|}~ + ESC ::= '\x1B' + CSI ::= ESC '[' + SS3 ::= ESC 'O' + +In ECMA48 `special? (number (';' modifiers)?)?` is the more liberal `[\x30-\x3F]*` +but that seems never used for key codes. If the number (vtcode or unicode) or the +modifiers are not given, we assume these are '1'. +We then accept the following key sequences: + + key ::= ESC # lone ESC + | ESC char # Alt+char + | ESC '[' special? vtcode ';' modifiers '~' # vt100 codes + | ESC '[' special? '1' ';' modifiers [A-Z] # xterm codes + | ESC 'O' special? '1' ';' modifiers [A-Za-z] # SS3 codes + | ESC '[' special? unicode ';' modifiers 'u' # direct unicode code + +Moreover, we translate the following special cases that do not fit into the above grammar. +First we translate away special starter sequences: +--------------------------------------------------------------------- + ESC '[' '[' .. ~> ESC '[' .. # Linux sometimes uses extra '[' for CSI + ESC '[' 'O' .. ~> ESC 'O' .. # Linux sometimes uses extra '[' for SS3 + ESC 'o' .. ~> ESC 'O' .. # Eterm: ctrl + SS3 + ESC '?' .. ~> ESC 'O' .. # vt52 treated as SS3 + +And then translate the following special cases into a standard form: +--------------------------------------------------------------------- + ESC '[' .. '@' ~> ESC '[' '3' '~' # Del on Mach + ESC '[' .. '9' ~> ESC '[' '2' '~' # Ins on Mach + ESC .. [^@$] ~> ESC .. '~' # ETerm,xrvt,urxt: ^ = ctrl, $ = shift, @ = alt + ESC '[' [a-d] ~> ESC '[' '1' ';' '2' [A-D] # Eterm shift+ + ESC 'O' [1-9] final ~> ESC 'O' '1' ';' [1-9] final # modifiers as parameter 1 (like on Haiku) + ESC '[' [1-9] [^~u] ~> ESC 'O' '1' ';' [1-9] final # modifiers as parameter 1 + +The modifier keys are encoded as "(modifiers-1) & mask" where the +shift mask is 0x01, alt 0x02 and ctrl 0x04. Therefore: +------------------------------------------------------------ + 1: - 5: ctrl 9: alt (for minicom) + 2: shift 6: shift+ctrl + 3: alt 7: alt+ctrl + 4: shift+alt 8: shift+alt+ctrl + +The different encodings fox vt100, xterm, and SS3 are: + +vt100: ESC [ vtcode ';' modifiers '~' +-------------------------------------- + 1: Home 10-15: F1-F5 + 2: Ins 16 : F5 + 3: Del 17-21: F6-F10 + 4: End 23-26: F11-F14 + 5: PageUp 28 : F15 + 6: PageDn 29 : F16 + 7: Home 31-34: F17-F20 + 8: End + +xterm: ESC [ 1 ';' modifiers [A-Z] +----------------------------------- + A: Up N: F2 + B: Down O: F3 + C: Right P: F4 + D: Left Q: F5 + E: '5' R: F6 + F: End S: F7 + G: T: F8 + H: Home U: PageDn + I: PageUp V: PageUp + J: W: F11 + K: X: F12 + L: Ins Y: End + M: F1 Z: shift+Tab + +SS3: ESC 'O' 1 ';' modifiers [A-Za-z] +--------------------------------------- + (normal) (numpad) + A: Up N: a: Up n: + B: Down O: b: Down o: + C: Right P: F1 c: Right p: Ins + D: Left Q: F2 d: Left q: End + E: '5' R: F3 e: r: Down + F: End S: F4 f: s: PageDn + G: T: F5 g: t: Left + H: Home U: F6 h: u: '5' + I: Tab V: F7 i: v: Right + J: W: F8 j: '*' w: Home + K: X: F9 k: '+' x: Up + L: Y: F10 l: ',' y: PageUp + M: \x0A '\n' Z: shift+Tab m: '-' z: + +-------------------------------------------------------------*/ + +//------------------------------------------------------------- +// Decode escape sequences +//------------------------------------------------------------- + +static code_t esc_decode_vt(uint32_t vt_code ) { + switch(vt_code) { + case 1: return KEY_HOME; + case 2: return KEY_INS; + case 3: return KEY_DEL; + case 4: return KEY_END; + case 5: return KEY_PAGEUP; + case 6: return KEY_PAGEDOWN; + case 7: return KEY_HOME; + case 8: return KEY_END; + default: + if (vt_code >= 10 && vt_code <= 15) return KEY_F(1 + (vt_code - 10)); + if (vt_code == 16) return KEY_F5; // minicom + if (vt_code >= 17 && vt_code <= 21) return KEY_F(6 + (vt_code - 17)); + if (vt_code >= 23 && vt_code <= 26) return KEY_F(11 + (vt_code - 23)); + if (vt_code >= 28 && vt_code <= 29) return KEY_F(15 + (vt_code - 28)); + if (vt_code >= 31 && vt_code <= 34) return KEY_F(17 + (vt_code - 31)); + } + return KEY_NONE; +} + +static code_t esc_decode_xterm( uint8_t xcode ) { + // ESC [ + switch(xcode) { + case 'A': return KEY_UP; + case 'B': return KEY_DOWN; + case 'C': return KEY_RIGHT; + case 'D': return KEY_LEFT; + case 'E': return '5'; // numpad 5 + case 'F': return KEY_END; + case 'H': return KEY_HOME; + case 'Z': return KEY_TAB | KEY_MOD_SHIFT; + // Freebsd: + case 'I': return KEY_PAGEUP; + case 'L': return KEY_INS; + case 'M': return KEY_F1; + case 'N': return KEY_F2; + case 'O': return KEY_F3; + case 'P': return KEY_F4; // note: differs from + case 'Q': return KEY_F5; + case 'R': return KEY_F6; + case 'S': return KEY_F7; + case 'T': return KEY_F8; + case 'U': return KEY_PAGEDOWN; // Mach + case 'V': return KEY_PAGEUP; // Mach + case 'W': return KEY_F11; + case 'X': return KEY_F12; + case 'Y': return KEY_END; // Mach + } + return KEY_NONE; +} + +static code_t esc_decode_ss3( uint8_t ss3_code ) { + // ESC O + switch(ss3_code) { + case 'A': return KEY_UP; + case 'B': return KEY_DOWN; + case 'C': return KEY_RIGHT; + case 'D': return KEY_LEFT; + case 'E': return '5'; // numpad 5 + case 'F': return KEY_END; + case 'H': return KEY_HOME; + case 'I': return KEY_TAB; + case 'Z': return KEY_TAB | KEY_MOD_SHIFT; + case 'M': return KEY_LINEFEED; + case 'P': return KEY_F1; + case 'Q': return KEY_F2; + case 'R': return KEY_F3; + case 'S': return KEY_F4; + // on Mach + case 'T': return KEY_F5; + case 'U': return KEY_F6; + case 'V': return KEY_F7; + case 'W': return KEY_F8; + case 'X': return KEY_F9; // '=' on vt220 + case 'Y': return KEY_F10; + // numpad + case 'a': return KEY_UP; + case 'b': return KEY_DOWN; + case 'c': return KEY_RIGHT; + case 'd': return KEY_LEFT; + case 'j': return '*'; + case 'k': return '+'; + case 'l': return ','; + case 'm': return '-'; + case 'n': return KEY_DEL; // '.' + case 'o': return '/'; + case 'p': return KEY_INS; + case 'q': return KEY_END; + case 'r': return KEY_DOWN; + case 's': return KEY_PAGEDOWN; + case 't': return KEY_LEFT; + case 'u': return '5'; + case 'v': return KEY_RIGHT; + case 'w': return KEY_HOME; + case 'x': return KEY_UP; + case 'y': return KEY_PAGEUP; + } + return KEY_NONE; +} + +static void tty_read_csi_num(tty_t* tty, uint8_t* ppeek, uint32_t* num, long esc_timeout) { + *num = 1; // default + ssize_t count = 0; + uint32_t i = 0; + while (*ppeek >= '0' && *ppeek <= '9' && count < 16) { + uint8_t digit = *ppeek - '0'; + if (!tty_readc_noblock(tty,ppeek,esc_timeout)) break; // peek is not modified in this case + count++; + i = 10*i + digit; + } + if (count > 0) *num = i; +} + +static code_t tty_read_csi(tty_t* tty, uint8_t c1, uint8_t peek, code_t mods0, long esc_timeout) { + // CSI starts with 0x9b (c1=='[') | ESC [ (c1=='[') | ESC [Oo?] (c1 == 'O') /* = SS3 */ + + // check for extra starter '[' (Linux sends ESC [ [ 15 ~ for F5 for example) + if (c1 == '[' && strchr("[Oo", (char)peek) != NULL) { + uint8_t cx = peek; + if (tty_readc_noblock(tty,&peek,esc_timeout)) { + c1 = cx; + } + } + + // "special" characters ('?' is used for private sequences) + uint8_t special = 0; + if (strchr(":<=>?",(char)peek) != NULL) { + special = peek; + if (!tty_readc_noblock(tty,&peek,esc_timeout)) { + tty_cpush_char(tty,special); // recover + return (key_unicode(c1) | KEY_MOD_ALT); // Alt+ + } + } + + // up to 2 parameters that default to 1 + uint32_t num1 = 1; + uint32_t num2 = 1; + tty_read_csi_num(tty,&peek,&num1,esc_timeout); + if (peek == ';') { + if (!tty_readc_noblock(tty,&peek,esc_timeout)) return KEY_NONE; + tty_read_csi_num(tty,&peek,&num2,esc_timeout); + } + + // the final character (we do not allow 'intermediate characters') + uint8_t final = peek; + code_t modifiers = mods0; + + debug_msg("tty: escape sequence: ESC %c %c %d;%d %c\n", c1, (special == 0 ? '_' : special), num1, num2, final); + + // Adjust special cases into standard ones. + if ((final == '@' || final == '9') && c1 == '[' && num1 == 1) { + // ESC [ @, ESC [ 9 : on Mach + if (final == '@') num1 = 3; // DEL + else if (final == '9') num1 = 2; // INS + final = '~'; + } + else if (final == '^' || final == '$' || final == '@') { + // Eterm/rxvt/urxt + if (final=='^') modifiers |= KEY_MOD_CTRL; + if (final=='$') modifiers |= KEY_MOD_SHIFT; + if (final=='@') modifiers |= KEY_MOD_SHIFT | KEY_MOD_CTRL; + final = '~'; + } + else if (c1 == '[' && final >= 'a' && final <= 'd') { // note: do not catch ESC [ .. u (for unicode) + // ESC [ [a-d] : on Eterm for shift+ cursor + modifiers |= KEY_MOD_SHIFT; + final = 'A' + (final - 'a'); + } + + if (((c1 == 'O') || (c1=='[' && final != '~' && final != 'u')) && + (num2 == 1 && num1 > 1 && num1 <= 8)) + { + // on haiku the modifier can be parameter 1, make it parameter 2 instead + num2 = num1; + num1 = 1; + } + + // parameter 2 determines the modifiers + if (num2 > 1 && num2 <= 9) { + if (num2 == 9) num2 = 3; // iTerm2 in xterm mode + num2--; + if (num2 & 0x1) modifiers |= KEY_MOD_SHIFT; + if (num2 & 0x2) modifiers |= KEY_MOD_ALT; + if (num2 & 0x4) modifiers |= KEY_MOD_CTRL; + } + + // and translate + code_t code = KEY_NONE; + if (final == '~') { + // vt codes + code = esc_decode_vt(num1); + } + else if (c1 == '[' && final == 'u') { + // unicode + code = key_unicode(num1); + } + else if (c1 == 'O' && ((final >= 'A' && final <= 'Z') || (final >= 'a' && final <= 'z'))) { + // ss3 + code = esc_decode_ss3(final); + } + else if (num1 == 1 && final >= 'A' && final <= 'Z') { + // xterm + code = esc_decode_xterm(final); + } + else if (c1 == '[' && final == 'R') { + // cursor position + code = KEY_NONE; + } + + if (code == KEY_NONE && final != 'R') { + debug_msg("tty: ignore escape sequence: ESC %c %zu;%zu %c\n", c1, num1, num2, final); + } + return (code != KEY_NONE ? (code | modifiers) : KEY_NONE); +} + +static code_t tty_read_osc( tty_t* tty, uint8_t* ppeek, long esc_timeout ) { + debug_msg("discard OSC response..\n"); + // keep reading until termination: OSC is terminated by BELL, or ESC \ (ST) (and STX) + while (true) { + uint8_t c = *ppeek; + if (c <= '\x07') { // BELL and anything below (STX, ^C, ^D) + if (c != '\x07') { tty_cpush_char( tty, c ); } + break; + } + else if (c=='\x1B') { + uint8_t c1; + if (!tty_readc_noblock(tty, &c1, esc_timeout)) break; + if (c1=='\\') break; + tty_cpush_char(tty,c1); + } + if (!tty_readc_noblock(tty, ppeek, esc_timeout)) break; + } + return KEY_NONE; +} + +ic_private code_t tty_read_esc(tty_t* tty, long esc_initial_timeout, long esc_timeout) { + code_t mods = 0; + uint8_t peek = 0; + + // lone ESC? + if (!tty_readc_noblock(tty, &peek, esc_initial_timeout)) return KEY_ESC; + + // treat ESC ESC as Alt modifier (macOS sends ESC ESC [ [A-D] for alt-) + if (peek == KEY_ESC) { + if (!tty_readc_noblock(tty, &peek, esc_timeout)) goto alt; + mods |= KEY_MOD_ALT; + } + + // CSI ? + if (peek == '[') { + if (!tty_readc_noblock(tty, &peek, esc_timeout)) goto alt; + return tty_read_csi(tty, '[', peek, mods, esc_timeout); // ESC [ ... + } + + // SS3? + if (peek == 'O' || peek == 'o' || peek == '?' /*vt52*/) { + uint8_t c1 = peek; + if (!tty_readc_noblock(tty, &peek, esc_timeout)) goto alt; + if (c1 == 'o') { + // ETerm uses this for ctrl+ + mods |= KEY_MOD_CTRL; + } + // treat all as standard SS3 'O' + return tty_read_csi(tty,'O',peek,mods, esc_timeout); // ESC [Oo?] ... + } + + // OSC: we may get a delayed query response; ensure it is ignored + if (peek == ']') { + if (!tty_readc_noblock(tty, &peek, esc_timeout)) goto alt; + return tty_read_osc(tty, &peek, esc_timeout); // ESC ] ... + } + +alt: + // Alt+ + return (key_unicode(peek) | KEY_MOD_ALT); // ESC +} diff --git a/extern/isocline/src/undo.c b/extern/isocline/src/undo.c new file mode 100644 index 00000000..eefc318d --- /dev/null +++ b/extern/isocline/src/undo.c @@ -0,0 +1,67 @@ +/* ---------------------------------------------------------------------------- + Copyright (c) 2021, Daan Leijen + This is free software; you can redistribute it and/or modify it + under the terms of the MIT License. A copy of the license can be + found in the "LICENSE" file at the root of this distribution. +-----------------------------------------------------------------------------*/ +#include +#include + +#include "../include/isocline.h" +#include "common.h" +#include "env.h" +#include "stringbuf.h" +#include "completions.h" +#include "undo.h" + + + +//------------------------------------------------------------- +// edit state +//------------------------------------------------------------- +struct editstate_s { + struct editstate_s* next; + const char* input; // input + ssize_t pos; // cursor position +}; + +ic_private void editstate_init( editstate_t** es ) { + *es = NULL; +} + +ic_private void editstate_done( alloc_t* mem, editstate_t** es ) { + while (*es != NULL) { + editstate_t* next = (*es)->next; + mem_free(mem, (*es)->input); + mem_free(mem, *es ); + *es = next; + } + *es = NULL; +} + +ic_private void editstate_capture( alloc_t* mem, editstate_t** es, const char* input, ssize_t pos) { + if (input==NULL) input = ""; + // alloc + editstate_t* entry = mem_zalloc_tp(mem, editstate_t); + if (entry == NULL) return; + // initialize + entry->input = mem_strdup( mem, input); + entry->pos = pos; + if (entry->input == NULL) { mem_free(mem, entry); return; } + // and push + entry->next = *es; + *es = entry; +} + +// caller should free *input +ic_private bool editstate_restore( alloc_t* mem, editstate_t** es, const char** input, ssize_t* pos ) { + if (*es == NULL) return false; + // pop + editstate_t* entry = *es; + *es = entry->next; + *input = entry->input; + *pos = entry->pos; + mem_free(mem, entry); + return true; +} + diff --git a/extern/isocline/src/undo.h b/extern/isocline/src/undo.h new file mode 100644 index 00000000..576cf977 --- /dev/null +++ b/extern/isocline/src/undo.h @@ -0,0 +1,24 @@ +/* ---------------------------------------------------------------------------- + Copyright (c) 2021, Daan Leijen + This is free software; you can redistribute it and/or modify it + under the terms of the MIT License. A copy of the license can be + found in the "LICENSE" file at the root of this distribution. +-----------------------------------------------------------------------------*/ +#pragma once +#ifndef IC_UNDO_H +#define IC_UNDO_H + +#include "common.h" + +//------------------------------------------------------------- +// Edit state +//------------------------------------------------------------- +struct editstate_s; +typedef struct editstate_s editstate_t; + +ic_private void editstate_init( editstate_t** es ); +ic_private void editstate_done( alloc_t* mem, editstate_t** es ); +ic_private void editstate_capture( alloc_t* mem, editstate_t** es, const char* input, ssize_t pos); +ic_private bool editstate_restore( alloc_t* mem, editstate_t** es, const char** input, ssize_t* pos ); // caller needs to free input + +#endif // IC_UNDO_H diff --git a/extern/isocline/src/wcwidth.c b/extern/isocline/src/wcwidth.c new file mode 100644 index 00000000..85187d41 --- /dev/null +++ b/extern/isocline/src/wcwidth.c @@ -0,0 +1,292 @@ +// include in "stringbuf.c" +/* + * This is an implementation of wcwidth() and wcswidth() (defined in + * IEEE Std 1002.1-2001) for Unicode. + * + * http://www.opengroup.org/onlinepubs/007904975/functions/wcwidth.html + * http://www.opengroup.org/onlinepubs/007904975/functions/wcswidth.html + * + * In fixed-width output devices, Latin characters all occupy a single + * "cell" position of equal width, whereas ideographic CJK characters + * occupy two such cells. Interoperability between terminal-line + * applications and (teletype-style) character terminals using the + * UTF-8 encoding requires agreement on which character should advance + * the cursor by how many cell positions. No established formal + * standards exist at present on which Unicode character shall occupy + * how many cell positions on character terminals. These routines are + * a first attempt of defining such behavior based on simple rules + * applied to data provided by the Unicode Consortium. + * + * For some graphical characters, the Unicode standard explicitly + * defines a character-cell width via the definition of the East Asian + * FullWidth (F), Wide (W), Half-width (H), and Narrow (Na) classes. + * In all these cases, there is no ambiguity about which width a + * terminal shall use. For characters in the East Asian Ambiguous (A) + * class, the width choice depends purely on a preference of backward + * compatibility with either historic CJK or Western practice. + * Choosing single-width for these characters is easy to justify as + * the appropriate long-term solution, as the CJK practice of + * displaying these characters as double-width comes from historic + * implementation simplicity (8-bit encoded characters were displayed + * single-width and 16-bit ones double-width, even for Greek, + * Cyrillic, etc.) and not any typographic considerations. + * + * Much less clear is the choice of width for the Not East Asian + * (Neutral) class. Existing practice does not dictate a width for any + * of these characters. It would nevertheless make sense + * typographically to allocate two character cells to characters such + * as for instance EM SPACE or VOLUME INTEGRAL, which cannot be + * represented adequately with a single-width glyph. The following + * routines at present merely assign a single-cell width to all + * neutral characters, in the interest of simplicity. This is not + * entirely satisfactory and should be reconsidered before + * establishing a formal standard in this area. At the moment, the + * decision which Not East Asian (Neutral) characters should be + * represented by double-width glyphs cannot yet be answered by + * applying a simple rule from the Unicode database content. Setting + * up a proper standard for the behavior of UTF-8 character terminals + * will require a careful analysis not only of each Unicode character, + * but also of each presentation form, something the author of these + * routines has avoided to do so far. + * + * http://www.unicode.org/unicode/reports/tr11/ + * + * Markus Kuhn -- 2007-05-26 (Unicode 5.0) + * + * Permission to use, copy, modify, and distribute this software + * for any purpose and without fee is hereby granted. The author + * disclaims all warranties with regard to this software. + * + * Latest version: http://www.cl.cam.ac.uk/~mgk25/ucs/wcwidth.c + */ + +#include +#include + +struct interval { + int32_t first; + int32_t last; +}; + +/* auxiliary function for binary search in interval table */ +static int bisearch(int32_t ucs, const struct interval *table, int max) { + int min = 0; + int mid; + + if (ucs < table[0].first || ucs > table[max].last) + return 0; + while (max >= min) { + mid = (min + max) / 2; + if (ucs > table[mid].last) + min = mid + 1; + else if (ucs < table[mid].first) + max = mid - 1; + else + return 1; + } + + return 0; +} + + +/* The following two functions define the column width of an ISO 10646 + * character as follows: + * + * - The null character (U+0000) has a column width of 0. + * + * - Other C0/C1 control characters and DEL will lead to a return + * value of -1. + * + * - Non-spacing and enclosing combining characters (general + * category code Mn or Me in the Unicode database) have a + * column width of 0. + * + * - SOFT HYPHEN (U+00AD) has a column width of 1. + * + * - Other format characters (general category code Cf in the Unicode + * database) and ZERO WIDTH SPACE (U+200B) have a column width of 0. + * + * - Hangul Jamo medial vowels and final consonants (U+1160-U+11FF) + * have a column width of 0. + * + * - Spacing characters in the East Asian Wide (W) or East Asian + * Full-width (F) category as defined in Unicode Technical + * Report #11 have a column width of 2. + * + * - All remaining characters (including all printable + * ISO 8859-1 and WGL4 characters, Unicode control characters, + * etc.) have a column width of 1. + * + * This implementation assumes that wchar_t characters are encoded + * in ISO 10646. + */ + +static int mk_is_wide_char(int32_t ucs) { + static const struct interval wide[] = { + {0x1100, 0x115f}, {0x231a, 0x231b}, {0x2329, 0x232a}, + {0x23e9, 0x23ec}, {0x23f0, 0x23f0}, {0x23f3, 0x23f3}, + {0x25fd, 0x25fe}, {0x2614, 0x2615}, {0x2648, 0x2653}, + {0x267f, 0x267f}, {0x2693, 0x2693}, {0x26a1, 0x26a1}, + {0x26aa, 0x26ab}, {0x26bd, 0x26be}, {0x26c4, 0x26c5}, + {0x26ce, 0x26ce}, {0x26d4, 0x26d4}, {0x26ea, 0x26ea}, + {0x26f2, 0x26f3}, {0x26f5, 0x26f5}, {0x26fa, 0x26fa}, + {0x26fd, 0x26fd}, {0x2705, 0x2705}, {0x270a, 0x270b}, + {0x2728, 0x2728}, {0x274c, 0x274c}, {0x274e, 0x274e}, + {0x2753, 0x2755}, {0x2757, 0x2757}, {0x2795, 0x2797}, + {0x27b0, 0x27b0}, {0x27bf, 0x27bf}, {0x2b1b, 0x2b1c}, + {0x2b50, 0x2b50}, {0x2b55, 0x2b55}, {0x2e80, 0x2fdf}, + {0x2ff0, 0x303e}, {0x3040, 0x3247}, {0x3250, 0x4dbf}, + {0x4e00, 0xa4cf}, {0xa960, 0xa97f}, {0xac00, 0xd7a3}, + {0xf900, 0xfaff}, {0xfe10, 0xfe19}, {0xfe30, 0xfe6f}, + {0xff01, 0xff60}, {0xffe0, 0xffe6}, {0x16fe0, 0x16fe1}, + {0x17000, 0x18aff}, {0x1b000, 0x1b12f}, {0x1b170, 0x1b2ff}, + {0x1f004, 0x1f004}, {0x1f0cf, 0x1f0cf}, {0x1f18e, 0x1f18e}, + {0x1f191, 0x1f19a}, {0x1f200, 0x1f202}, {0x1f210, 0x1f23b}, + {0x1f240, 0x1f248}, {0x1f250, 0x1f251}, {0x1f260, 0x1f265}, + {0x1f300, 0x1f320}, {0x1f32d, 0x1f335}, {0x1f337, 0x1f37c}, + {0x1f37e, 0x1f393}, {0x1f3a0, 0x1f3ca}, {0x1f3cf, 0x1f3d3}, + {0x1f3e0, 0x1f3f0}, {0x1f3f4, 0x1f3f4}, {0x1f3f8, 0x1f43e}, + {0x1f440, 0x1f440}, {0x1f442, 0x1f4fc}, {0x1f4ff, 0x1f53d}, + {0x1f54b, 0x1f54e}, {0x1f550, 0x1f567}, {0x1f57a, 0x1f57a}, + {0x1f595, 0x1f596}, {0x1f5a4, 0x1f5a4}, {0x1f5fb, 0x1f64f}, + {0x1f680, 0x1f6c5}, {0x1f6cc, 0x1f6cc}, {0x1f6d0, 0x1f6d2}, + {0x1f6eb, 0x1f6ec}, {0x1f6f4, 0x1f6f8}, {0x1f910, 0x1f93e}, + {0x1f940, 0x1f94c}, {0x1f950, 0x1f96b}, {0x1f980, 0x1f997}, + {0x1f9c0, 0x1f9c0}, {0x1f9d0, 0x1f9e6}, {0x20000, 0x2fffd}, + {0x30000, 0x3fffd}, + }; + + if ( bisearch(ucs, wide, sizeof(wide) / sizeof(struct interval) - 1) ) { + return 1; + } + + return 0; +} + +static int mk_wcwidth(int32_t ucs) { + /* sorted list of non-overlapping intervals of non-spacing characters */ + /* generated by "uniset +cat=Me +cat=Mn +cat=Cf -00AD +1160-11FF +200B c" */ + static const struct interval combining[] = { + {0x00ad, 0x00ad}, {0x0300, 0x036f}, {0x0483, 0x0489}, + {0x0591, 0x05bd}, {0x05bf, 0x05bf}, {0x05c1, 0x05c2}, + {0x05c4, 0x05c5}, {0x05c7, 0x05c7}, {0x0610, 0x061a}, + {0x061c, 0x061c}, {0x064b, 0x065f}, {0x0670, 0x0670}, + {0x06d6, 0x06dc}, {0x06df, 0x06e4}, {0x06e7, 0x06e8}, + {0x06ea, 0x06ed}, {0x0711, 0x0711}, {0x0730, 0x074a}, + {0x07a6, 0x07b0}, {0x07eb, 0x07f3}, {0x0816, 0x0819}, + {0x081b, 0x0823}, {0x0825, 0x0827}, {0x0829, 0x082d}, + {0x0859, 0x085b}, {0x08d4, 0x08e1}, {0x08e3, 0x0902}, + {0x093a, 0x093a}, {0x093c, 0x093c}, {0x0941, 0x0948}, + {0x094d, 0x094d}, {0x0951, 0x0957}, {0x0962, 0x0963}, + {0x0981, 0x0981}, {0x09bc, 0x09bc}, {0x09c1, 0x09c4}, + {0x09cd, 0x09cd}, {0x09e2, 0x09e3}, {0x0a01, 0x0a02}, + {0x0a3c, 0x0a3c}, {0x0a41, 0x0a42}, {0x0a47, 0x0a48}, + {0x0a4b, 0x0a4d}, {0x0a51, 0x0a51}, {0x0a70, 0x0a71}, + {0x0a75, 0x0a75}, {0x0a81, 0x0a82}, {0x0abc, 0x0abc}, + {0x0ac1, 0x0ac5}, {0x0ac7, 0x0ac8}, {0x0acd, 0x0acd}, + {0x0ae2, 0x0ae3}, {0x0afa, 0x0aff}, {0x0b01, 0x0b01}, + {0x0b3c, 0x0b3c}, {0x0b3f, 0x0b3f}, {0x0b41, 0x0b44}, + {0x0b4d, 0x0b4d}, {0x0b56, 0x0b56}, {0x0b62, 0x0b63}, + {0x0b82, 0x0b82}, {0x0bc0, 0x0bc0}, {0x0bcd, 0x0bcd}, + {0x0c00, 0x0c00}, {0x0c3e, 0x0c40}, {0x0c46, 0x0c48}, + {0x0c4a, 0x0c4d}, {0x0c55, 0x0c56}, {0x0c62, 0x0c63}, + {0x0c81, 0x0c81}, {0x0cbc, 0x0cbc}, {0x0cbf, 0x0cbf}, + {0x0cc6, 0x0cc6}, {0x0ccc, 0x0ccd}, {0x0ce2, 0x0ce3}, + {0x0d00, 0x0d01}, {0x0d3b, 0x0d3c}, {0x0d41, 0x0d44}, + {0x0d4d, 0x0d4d}, {0x0d62, 0x0d63}, {0x0dca, 0x0dca}, + {0x0dd2, 0x0dd4}, {0x0dd6, 0x0dd6}, {0x0e31, 0x0e31}, + {0x0e34, 0x0e3a}, {0x0e47, 0x0e4e}, {0x0eb1, 0x0eb1}, + {0x0eb4, 0x0eb9}, {0x0ebb, 0x0ebc}, {0x0ec8, 0x0ecd}, + {0x0f18, 0x0f19}, {0x0f35, 0x0f35}, {0x0f37, 0x0f37}, + {0x0f39, 0x0f39}, {0x0f71, 0x0f7e}, {0x0f80, 0x0f84}, + {0x0f86, 0x0f87}, {0x0f8d, 0x0f97}, {0x0f99, 0x0fbc}, + {0x0fc6, 0x0fc6}, {0x102d, 0x1030}, {0x1032, 0x1037}, + {0x1039, 0x103a}, {0x103d, 0x103e}, {0x1058, 0x1059}, + {0x105e, 0x1060}, {0x1071, 0x1074}, {0x1082, 0x1082}, + {0x1085, 0x1086}, {0x108d, 0x108d}, {0x109d, 0x109d}, + {0x1160, 0x11ff}, {0x135d, 0x135f}, {0x1712, 0x1714}, + {0x1732, 0x1734}, {0x1752, 0x1753}, {0x1772, 0x1773}, + {0x17b4, 0x17b5}, {0x17b7, 0x17bd}, {0x17c6, 0x17c6}, + {0x17c9, 0x17d3}, {0x17dd, 0x17dd}, {0x180b, 0x180e}, + {0x1885, 0x1886}, {0x18a9, 0x18a9}, {0x1920, 0x1922}, + {0x1927, 0x1928}, {0x1932, 0x1932}, {0x1939, 0x193b}, + {0x1a17, 0x1a18}, {0x1a1b, 0x1a1b}, {0x1a56, 0x1a56}, + {0x1a58, 0x1a5e}, {0x1a60, 0x1a60}, {0x1a62, 0x1a62}, + {0x1a65, 0x1a6c}, {0x1a73, 0x1a7c}, {0x1a7f, 0x1a7f}, + {0x1ab0, 0x1abe}, {0x1b00, 0x1b03}, {0x1b34, 0x1b34}, + {0x1b36, 0x1b3a}, {0x1b3c, 0x1b3c}, {0x1b42, 0x1b42}, + {0x1b6b, 0x1b73}, {0x1b80, 0x1b81}, {0x1ba2, 0x1ba5}, + {0x1ba8, 0x1ba9}, {0x1bab, 0x1bad}, {0x1be6, 0x1be6}, + {0x1be8, 0x1be9}, {0x1bed, 0x1bed}, {0x1bef, 0x1bf1}, + {0x1c2c, 0x1c33}, {0x1c36, 0x1c37}, {0x1cd0, 0x1cd2}, + {0x1cd4, 0x1ce0}, {0x1ce2, 0x1ce8}, {0x1ced, 0x1ced}, + {0x1cf4, 0x1cf4}, {0x1cf8, 0x1cf9}, {0x1dc0, 0x1df9}, + {0x1dfb, 0x1dff}, {0x200b, 0x200f}, {0x202a, 0x202e}, + {0x2060, 0x2064}, {0x2066, 0x206f}, {0x20d0, 0x20f0}, + {0x2cef, 0x2cf1}, {0x2d7f, 0x2d7f}, {0x2de0, 0x2dff}, + {0x302a, 0x302d}, {0x3099, 0x309a}, {0xa66f, 0xa672}, + {0xa674, 0xa67d}, {0xa69e, 0xa69f}, {0xa6f0, 0xa6f1}, + {0xa802, 0xa802}, {0xa806, 0xa806}, {0xa80b, 0xa80b}, + {0xa825, 0xa826}, {0xa8c4, 0xa8c5}, {0xa8e0, 0xa8f1}, + {0xa926, 0xa92d}, {0xa947, 0xa951}, {0xa980, 0xa982}, + {0xa9b3, 0xa9b3}, {0xa9b6, 0xa9b9}, {0xa9bc, 0xa9bc}, + {0xa9e5, 0xa9e5}, {0xaa29, 0xaa2e}, {0xaa31, 0xaa32}, + {0xaa35, 0xaa36}, {0xaa43, 0xaa43}, {0xaa4c, 0xaa4c}, + {0xaa7c, 0xaa7c}, {0xaab0, 0xaab0}, {0xaab2, 0xaab4}, + {0xaab7, 0xaab8}, {0xaabe, 0xaabf}, {0xaac1, 0xaac1}, + {0xaaec, 0xaaed}, {0xaaf6, 0xaaf6}, {0xabe5, 0xabe5}, + {0xabe8, 0xabe8}, {0xabed, 0xabed}, {0xfb1e, 0xfb1e}, + {0xfe00, 0xfe0f}, {0xfe20, 0xfe2f}, {0xfeff, 0xfeff}, + {0xfff9, 0xfffb}, {0x101fd, 0x101fd}, {0x102e0, 0x102e0}, + {0x10376, 0x1037a}, {0x10a01, 0x10a03}, {0x10a05, 0x10a06}, + {0x10a0c, 0x10a0f}, {0x10a38, 0x10a3a}, {0x10a3f, 0x10a3f}, + {0x10ae5, 0x10ae6}, {0x11001, 0x11001}, {0x11038, 0x11046}, + {0x1107f, 0x11081}, {0x110b3, 0x110b6}, {0x110b9, 0x110ba}, + {0x11100, 0x11102}, {0x11127, 0x1112b}, {0x1112d, 0x11134}, + {0x11173, 0x11173}, {0x11180, 0x11181}, {0x111b6, 0x111be}, + {0x111ca, 0x111cc}, {0x1122f, 0x11231}, {0x11234, 0x11234}, + {0x11236, 0x11237}, {0x1123e, 0x1123e}, {0x112df, 0x112df}, + {0x112e3, 0x112ea}, {0x11300, 0x11301}, {0x1133c, 0x1133c}, + {0x11340, 0x11340}, {0x11366, 0x1136c}, {0x11370, 0x11374}, + {0x11438, 0x1143f}, {0x11442, 0x11444}, {0x11446, 0x11446}, + {0x114b3, 0x114b8}, {0x114ba, 0x114ba}, {0x114bf, 0x114c0}, + {0x114c2, 0x114c3}, {0x115b2, 0x115b5}, {0x115bc, 0x115bd}, + {0x115bf, 0x115c0}, {0x115dc, 0x115dd}, {0x11633, 0x1163a}, + {0x1163d, 0x1163d}, {0x1163f, 0x11640}, {0x116ab, 0x116ab}, + {0x116ad, 0x116ad}, {0x116b0, 0x116b5}, {0x116b7, 0x116b7}, + {0x1171d, 0x1171f}, {0x11722, 0x11725}, {0x11727, 0x1172b}, + {0x11a01, 0x11a06}, {0x11a09, 0x11a0a}, {0x11a33, 0x11a38}, + {0x11a3b, 0x11a3e}, {0x11a47, 0x11a47}, {0x11a51, 0x11a56}, + {0x11a59, 0x11a5b}, {0x11a8a, 0x11a96}, {0x11a98, 0x11a99}, + {0x11c30, 0x11c36}, {0x11c38, 0x11c3d}, {0x11c3f, 0x11c3f}, + {0x11c92, 0x11ca7}, {0x11caa, 0x11cb0}, {0x11cb2, 0x11cb3}, + {0x11cb5, 0x11cb6}, {0x11d31, 0x11d36}, {0x11d3a, 0x11d3a}, + {0x11d3c, 0x11d3d}, {0x11d3f, 0x11d45}, {0x11d47, 0x11d47}, + {0x16af0, 0x16af4}, {0x16b30, 0x16b36}, {0x16f8f, 0x16f92}, + {0x1bc9d, 0x1bc9e}, {0x1bca0, 0x1bca3}, {0x1d167, 0x1d169}, + {0x1d173, 0x1d182}, {0x1d185, 0x1d18b}, {0x1d1aa, 0x1d1ad}, + {0x1d242, 0x1d244}, {0x1da00, 0x1da36}, {0x1da3b, 0x1da6c}, + {0x1da75, 0x1da75}, {0x1da84, 0x1da84}, {0x1da9b, 0x1da9f}, + {0x1daa1, 0x1daaf}, {0x1e000, 0x1e006}, {0x1e008, 0x1e018}, + {0x1e01b, 0x1e021}, {0x1e023, 0x1e024}, {0x1e026, 0x1e02a}, + {0x1e8d0, 0x1e8d6}, {0x1e944, 0x1e94a}, {0xe0001, 0xe0001}, + {0xe0020, 0xe007f}, {0xe0100, 0xe01ef}, + }; + + /* test for 8-bit control characters */ + if ( ucs == 0 ) { + return 0; + } + if ( ( ucs < 32 ) || ( ( ucs >= 0x7f ) && ( ucs < 0xa0 ) ) ) { + return -1; + } + + /* binary search in table of non-spacing characters */ + if ( bisearch( ucs, combining, sizeof( combining ) / sizeof( struct interval ) - 1 ) ) { + return 0; + } + + /* if we arrive here, ucs is not a combining or C0/C1 control character */ + return ( mk_is_wide_char( ucs ) ? 2 : 1 ); +} + diff --git a/extern/linenoise.hpp b/extern/linenoise.hpp deleted file mode 100644 index ae36eb0b..00000000 --- a/extern/linenoise.hpp +++ /dev/null @@ -1,2415 +0,0 @@ -/* - * linenoise.hpp -- Multi-platfrom C++ header-only linenoise library. - * - * All credits and commendations have to go to the authors of the - * following excellent libraries. - * - * - linenoise.h and linenose.c (https://github.com/antirez/linenoise) - * - ANSI.c (https://github.com/adoxa/ansicon) - * - Win32_ANSI.h and Win32_ANSI.c (https://github.com/MSOpenTech/redis) - * - * ------------------------------------------------------------------------ - * - * Copyright (c) 2015 yhirose - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - * 1. Redistributions of source code must retain the above copyright notice, this - * list of conditions and the following disclaimer. - * 2. Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND - * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED - * WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR - * ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES - * (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; - * LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND - * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT - * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS - * SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - */ - -/* linenoise.h -- guerrilla line editing library against the idea that a - * line editing lib needs to be 20,000 lines of C code. - * - * See linenoise.c for more information. - * - * ------------------------------------------------------------------------ - * - * Copyright (c) 2010, Salvatore Sanfilippo - * Copyright (c) 2010, Pieter Noordhuis - * - * All rights reserved. - * - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are - * met: - * - * * Redistributions of source code must retain the above copyright - * notice, this list of conditions and the following disclaimer. - * - * * Redistributions in binary form must reproduce the above copyright - * notice, this list of conditions and the following disclaimer in the - * documentation and/or other materials provided with the distribution. - * - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS - * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT - * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR - * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT - * HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, - * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT - * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, - * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY - * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT - * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - */ - -/* - * ANSI.c - ANSI escape sequence console driver. - * - * Copyright (C) 2005-2014 Jason Hood - * This software is provided 'as-is', without any express or implied - * warranty. In no event will the author be held liable for any damages - * arising from the use of this software. - * - * Permission is granted to anyone to use this software for any purpose, - * including commercial applications, and to alter it and redistribute it - * freely, subject to the following restrictions: - * - * 1. The origin of this software must not be misrepresented; you must not - * claim that you wrote the original software. If you use this software - * in a product, an acknowledgment in the product documentation would be - * appreciated but is not required. - * 2. Altered source versions must be plainly marked as such, and must not be - * misrepresented as being the original software. - * 3. This notice may not be removed or altered from any source distribution. - * - * Jason Hood - * jadoxa@yahoo.com.au - */ - -/* - * Win32_ANSI.h and Win32_ANSI.c - * - * Derived from ANSI.c by Jason Hood, from his ansicon project (https://github.com/adoxa/ansicon), with modifications. - * - * Copyright (c), Microsoft Open Technologies, Inc. - * All rights reserved. - * Redistribution and use in source and binary forms, with or without - * modification, are permitted provided that the following conditions are met: - * - Redistributions of source code must retain the above copyright notice, - * this list of conditions and the following disclaimer. - * - Redistributions in binary form must reproduce the above copyright notice, - * this list of conditions and the following disclaimer in the documentation - * and/or other materials provided with the distribution. - * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" - * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE - * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE - * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE - * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL - * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR - * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER - * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, - * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE - * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. - */ - -#ifndef LINENOISE_HPP -#define LINENOISE_HPP - -#ifndef _WIN32 -#include -#include -#include -#else -#ifndef NOMINMAX -#define NOMINMAX -#endif -#include -#include -#ifndef STDIN_FILENO -#define STDIN_FILENO (_fileno(stdin)) -#endif -#ifndef STDOUT_FILENO -#define STDOUT_FILENO 1 -#endif -#define isatty _isatty -#define write win32_write -#define read _read -#pragma warning(push) -#pragma warning(disable : 4996) -#endif -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include -#include - -namespace linenoise { - -typedef std::function&)> CompletionCallback; - -#ifdef _WIN32 - -namespace ansi { - -#define lenof(array) (sizeof(array)/sizeof(*(array))) - -typedef struct -{ - BYTE foreground; // ANSI base color (0 to 7; add 30) - BYTE background; // ANSI base color (0 to 7; add 40) - BYTE bold; // console FOREGROUND_INTENSITY bit - BYTE underline; // console BACKGROUND_INTENSITY bit - BYTE rvideo; // swap foreground/bold & background/underline - BYTE concealed; // set foreground/bold to background/underline - BYTE reverse; // swap console foreground & background attributes -} GRM, *PGRM; // Graphic Rendition Mode - - -inline bool is_digit(char c) { return '0' <= c && c <= '9'; } - -// ========== Global variables and constants - -HANDLE hConOut; // handle to CONOUT$ - -const char ESC = '\x1B'; // ESCape character -const char BEL = '\x07'; -const char SO = '\x0E'; // Shift Out -const char SI = '\x0F'; // Shift In - -const int MAX_ARG = 16; // max number of args in an escape sequence -int state; // automata state -WCHAR prefix; // escape sequence prefix ( '[', ']' or '(' ); -WCHAR prefix2; // secondary prefix ( '?' or '>' ); -WCHAR suffix; // escape sequence suffix -int es_argc; // escape sequence args count -int es_argv[MAX_ARG]; // escape sequence args -WCHAR Pt_arg[MAX_PATH * 2]; // text parameter for Operating System Command -int Pt_len; -BOOL shifted; - - -// DEC Special Graphics Character Set from -// http://vt100.net/docs/vt220-rm/table2-4.html -// Some of these may not look right, depending on the font and code page (in -// particular, the Control Pictures probably won't work at all). -const WCHAR G1[] = -{ - ' ', // _ - blank - L'\x2666', // ` - Black Diamond Suit - L'\x2592', // a - Medium Shade - L'\x2409', // b - HT - L'\x240c', // c - FF - L'\x240d', // d - CR - L'\x240a', // e - LF - L'\x00b0', // f - Degree Sign - L'\x00b1', // g - Plus-Minus Sign - L'\x2424', // h - NL - L'\x240b', // i - VT - L'\x2518', // j - Box Drawings Light Up And Left - L'\x2510', // k - Box Drawings Light Down And Left - L'\x250c', // l - Box Drawings Light Down And Right - L'\x2514', // m - Box Drawings Light Up And Right - L'\x253c', // n - Box Drawings Light Vertical And Horizontal - L'\x00af', // o - SCAN 1 - Macron - L'\x25ac', // p - SCAN 3 - Black Rectangle - L'\x2500', // q - SCAN 5 - Box Drawings Light Horizontal - L'_', // r - SCAN 7 - Low Line - L'_', // s - SCAN 9 - Low Line - L'\x251c', // t - Box Drawings Light Vertical And Right - L'\x2524', // u - Box Drawings Light Vertical And Left - L'\x2534', // v - Box Drawings Light Up And Horizontal - L'\x252c', // w - Box Drawings Light Down And Horizontal - L'\x2502', // x - Box Drawings Light Vertical - L'\x2264', // y - Less-Than Or Equal To - L'\x2265', // z - Greater-Than Or Equal To - L'\x03c0', // { - Greek Small Letter Pi - L'\x2260', // | - Not Equal To - L'\x00a3', // } - Pound Sign - L'\x00b7', // ~ - Middle Dot -}; - -#define FIRST_G1 '_' -#define LAST_G1 '~' - - -// color constants - -#define FOREGROUND_BLACK 0 -#define FOREGROUND_WHITE FOREGROUND_RED|FOREGROUND_GREEN|FOREGROUND_BLUE - -#define BACKGROUND_BLACK 0 -#define BACKGROUND_WHITE BACKGROUND_RED|BACKGROUND_GREEN|BACKGROUND_BLUE - -const BYTE foregroundcolor[8] = - { - FOREGROUND_BLACK, // black foreground - FOREGROUND_RED, // red foreground - FOREGROUND_GREEN, // green foreground - FOREGROUND_RED | FOREGROUND_GREEN, // yellow foreground - FOREGROUND_BLUE, // blue foreground - FOREGROUND_BLUE | FOREGROUND_RED, // magenta foreground - FOREGROUND_BLUE | FOREGROUND_GREEN, // cyan foreground - FOREGROUND_WHITE // white foreground - }; - -const BYTE backgroundcolor[8] = - { - BACKGROUND_BLACK, // black background - BACKGROUND_RED, // red background - BACKGROUND_GREEN, // green background - BACKGROUND_RED | BACKGROUND_GREEN, // yellow background - BACKGROUND_BLUE, // blue background - BACKGROUND_BLUE | BACKGROUND_RED, // magenta background - BACKGROUND_BLUE | BACKGROUND_GREEN, // cyan background - BACKGROUND_WHITE, // white background - }; - -const BYTE attr2ansi[8] = // map console attribute to ANSI number -{ - 0, // black - 4, // blue - 2, // green - 6, // cyan - 1, // red - 5, // magenta - 3, // yellow - 7 // white -}; - -GRM grm; - -// saved cursor position -COORD SavePos; - -// ========== Print Buffer functions - -#define BUFFER_SIZE 2048 - -int nCharInBuffer; -WCHAR ChBuffer[BUFFER_SIZE]; - -//----------------------------------------------------------------------------- -// FlushBuffer() -// Writes the buffer to the console and empties it. -//----------------------------------------------------------------------------- - -inline void FlushBuffer(void) -{ - DWORD nWritten; - if (nCharInBuffer <= 0) return; - WriteConsoleW(hConOut, ChBuffer, nCharInBuffer, &nWritten, NULL); - nCharInBuffer = 0; -} - -//----------------------------------------------------------------------------- -// PushBuffer( WCHAR c ) -// Adds a character in the buffer. -//----------------------------------------------------------------------------- - -inline void PushBuffer(WCHAR c) -{ - if (shifted && c >= FIRST_G1 && c <= LAST_G1) - c = G1[c - FIRST_G1]; - ChBuffer[nCharInBuffer] = c; - if (++nCharInBuffer == BUFFER_SIZE) - FlushBuffer(); -} - -//----------------------------------------------------------------------------- -// SendSequence( LPCWSTR seq ) -// Send the string to the input buffer. -//----------------------------------------------------------------------------- - -inline void SendSequence(LPCWSTR seq) -{ - DWORD out; - INPUT_RECORD in; - HANDLE hStdIn = GetStdHandle(STD_INPUT_HANDLE); - - in.EventType = KEY_EVENT; - in.Event.KeyEvent.bKeyDown = TRUE; - in.Event.KeyEvent.wRepeatCount = 1; - in.Event.KeyEvent.wVirtualKeyCode = 0; - in.Event.KeyEvent.wVirtualScanCode = 0; - in.Event.KeyEvent.dwControlKeyState = 0; - for (; *seq; ++seq) - { - in.Event.KeyEvent.uChar.UnicodeChar = *seq; - WriteConsoleInput(hStdIn, &in, 1, &out); - } -} - -// ========== Print functions - -//----------------------------------------------------------------------------- -// InterpretEscSeq() -// Interprets the last escape sequence scanned by ParseAndPrintANSIString -// prefix escape sequence prefix -// es_argc escape sequence args count -// es_argv[] escape sequence args array -// suffix escape sequence suffix -// -// for instance, with \e[33;45;1m we have -// prefix = '[', -// es_argc = 3, es_argv[0] = 33, es_argv[1] = 45, es_argv[2] = 1 -// suffix = 'm' -//----------------------------------------------------------------------------- - -inline void InterpretEscSeq(void) -{ - int i; - WORD attribut; - CONSOLE_SCREEN_BUFFER_INFO Info; - CONSOLE_CURSOR_INFO CursInfo; - DWORD len, NumberOfCharsWritten; - COORD Pos; - SMALL_RECT Rect; - CHAR_INFO CharInfo; - - if (prefix == '[') - { - if (prefix2 == '?' && (suffix == 'h' || suffix == 'l')) - { - if (es_argc == 1 && es_argv[0] == 25) - { - GetConsoleCursorInfo(hConOut, &CursInfo); - CursInfo.bVisible = (suffix == 'h'); - SetConsoleCursorInfo(hConOut, &CursInfo); - return; - } - } - // Ignore any other \e[? or \e[> sequences. - if (prefix2 != 0) - return; - - GetConsoleScreenBufferInfo(hConOut, &Info); - switch (suffix) - { - case 'm': - if (es_argc == 0) es_argv[es_argc++] = 0; - for (i = 0; i < es_argc; i++) - { - if (30 <= es_argv[i] && es_argv[i] <= 37) - grm.foreground = es_argv[i] - 30; - else if (40 <= es_argv[i] && es_argv[i] <= 47) - grm.background = es_argv[i] - 40; - else switch (es_argv[i]) - { - case 0: - case 39: - case 49: - { - WCHAR def[4]; - int a; - *def = '7'; def[1] = '\0'; - GetEnvironmentVariableW(L"ANSICON_DEF", def, lenof(def)); - a = wcstol(def, NULL, 16); - grm.reverse = FALSE; - if (a < 0) - { - grm.reverse = TRUE; - a = -a; - } - if (es_argv[i] != 49) - grm.foreground = attr2ansi[a & 7]; - if (es_argv[i] != 39) - grm.background = attr2ansi[(a >> 4) & 7]; - if (es_argv[i] == 0) - { - if (es_argc == 1) - { - grm.bold = a & FOREGROUND_INTENSITY; - grm.underline = a & BACKGROUND_INTENSITY; - } - else - { - grm.bold = 0; - grm.underline = 0; - } - grm.rvideo = 0; - grm.concealed = 0; - } - } - break; - - case 1: grm.bold = FOREGROUND_INTENSITY; break; - case 5: // blink - case 4: grm.underline = BACKGROUND_INTENSITY; break; - case 7: grm.rvideo = 1; break; - case 8: grm.concealed = 1; break; - case 21: // oops, this actually turns on double underline - case 22: grm.bold = 0; break; - case 25: - case 24: grm.underline = 0; break; - case 27: grm.rvideo = 0; break; - case 28: grm.concealed = 0; break; - } - } - if (grm.concealed) - { - if (grm.rvideo) - { - attribut = foregroundcolor[grm.foreground] - | backgroundcolor[grm.foreground]; - if (grm.bold) - attribut |= FOREGROUND_INTENSITY | BACKGROUND_INTENSITY; - } - else - { - attribut = foregroundcolor[grm.background] - | backgroundcolor[grm.background]; - if (grm.underline) - attribut |= FOREGROUND_INTENSITY | BACKGROUND_INTENSITY; - } - } - else if (grm.rvideo) - { - attribut = foregroundcolor[grm.background] - | backgroundcolor[grm.foreground]; - if (grm.bold) - attribut |= BACKGROUND_INTENSITY; - if (grm.underline) - attribut |= FOREGROUND_INTENSITY; - } - else - attribut = foregroundcolor[grm.foreground] | grm.bold - | backgroundcolor[grm.background] | grm.underline; - if (grm.reverse) - attribut = ((attribut >> 4) & 15) | ((attribut & 15) << 4); - SetConsoleTextAttribute(hConOut, attribut); - return; - - case 'J': - if (es_argc == 0) es_argv[es_argc++] = 0; // ESC[J == ESC[0J - if (es_argc != 1) return; - switch (es_argv[0]) - { - case 0: // ESC[0J erase from cursor to end of display - len = (Info.dwSize.Y - Info.dwCursorPosition.Y - 1) * Info.dwSize.X - + Info.dwSize.X - Info.dwCursorPosition.X - 1; - FillConsoleOutputCharacter(hConOut, ' ', len, - Info.dwCursorPosition, - &NumberOfCharsWritten); - FillConsoleOutputAttribute(hConOut, Info.wAttributes, len, - Info.dwCursorPosition, - &NumberOfCharsWritten); - return; - - case 1: // ESC[1J erase from start to cursor. - Pos.X = 0; - Pos.Y = 0; - len = Info.dwCursorPosition.Y * Info.dwSize.X - + Info.dwCursorPosition.X + 1; - FillConsoleOutputCharacter(hConOut, ' ', len, Pos, - &NumberOfCharsWritten); - FillConsoleOutputAttribute(hConOut, Info.wAttributes, len, Pos, - &NumberOfCharsWritten); - return; - - case 2: // ESC[2J Clear screen and home cursor - Pos.X = 0; - Pos.Y = 0; - len = Info.dwSize.X * Info.dwSize.Y; - FillConsoleOutputCharacter(hConOut, ' ', len, Pos, - &NumberOfCharsWritten); - FillConsoleOutputAttribute(hConOut, Info.wAttributes, len, Pos, - &NumberOfCharsWritten); - SetConsoleCursorPosition(hConOut, Pos); - return; - - default: - return; - } - - case 'K': - if (es_argc == 0) es_argv[es_argc++] = 0; // ESC[K == ESC[0K - if (es_argc != 1) return; - switch (es_argv[0]) - { - case 0: // ESC[0K Clear to end of line - len = Info.dwSize.X - Info.dwCursorPosition.X + 1; - FillConsoleOutputCharacter(hConOut, ' ', len, - Info.dwCursorPosition, - &NumberOfCharsWritten); - FillConsoleOutputAttribute(hConOut, Info.wAttributes, len, - Info.dwCursorPosition, - &NumberOfCharsWritten); - return; - - case 1: // ESC[1K Clear from start of line to cursor - Pos.X = 0; - Pos.Y = Info.dwCursorPosition.Y; - FillConsoleOutputCharacter(hConOut, ' ', - Info.dwCursorPosition.X + 1, Pos, - &NumberOfCharsWritten); - FillConsoleOutputAttribute(hConOut, Info.wAttributes, - Info.dwCursorPosition.X + 1, Pos, - &NumberOfCharsWritten); - return; - - case 2: // ESC[2K Clear whole line. - Pos.X = 0; - Pos.Y = Info.dwCursorPosition.Y; - FillConsoleOutputCharacter(hConOut, ' ', Info.dwSize.X, Pos, - &NumberOfCharsWritten); - FillConsoleOutputAttribute(hConOut, Info.wAttributes, - Info.dwSize.X, Pos, - &NumberOfCharsWritten); - return; - - default: - return; - } - - case 'X': // ESC[#X Erase # characters. - if (es_argc == 0) es_argv[es_argc++] = 1; // ESC[X == ESC[1X - if (es_argc != 1) return; - FillConsoleOutputCharacter(hConOut, ' ', es_argv[0], - Info.dwCursorPosition, - &NumberOfCharsWritten); - FillConsoleOutputAttribute(hConOut, Info.wAttributes, es_argv[0], - Info.dwCursorPosition, - &NumberOfCharsWritten); - return; - - case 'L': // ESC[#L Insert # blank lines. - if (es_argc == 0) es_argv[es_argc++] = 1; // ESC[L == ESC[1L - if (es_argc != 1) return; - Rect.Left = 0; - Rect.Top = Info.dwCursorPosition.Y; - Rect.Right = Info.dwSize.X - 1; - Rect.Bottom = Info.dwSize.Y - 1; - Pos.X = 0; - Pos.Y = Info.dwCursorPosition.Y + es_argv[0]; - CharInfo.Char.UnicodeChar = ' '; - CharInfo.Attributes = Info.wAttributes; - ScrollConsoleScreenBuffer(hConOut, &Rect, NULL, Pos, &CharInfo); - return; - - case 'M': // ESC[#M Delete # lines. - if (es_argc == 0) es_argv[es_argc++] = 1; // ESC[M == ESC[1M - if (es_argc != 1) return; - if (es_argv[0] > Info.dwSize.Y - Info.dwCursorPosition.Y) - es_argv[0] = Info.dwSize.Y - Info.dwCursorPosition.Y; - Rect.Left = 0; - Rect.Top = Info.dwCursorPosition.Y + es_argv[0]; - Rect.Right = Info.dwSize.X - 1; - Rect.Bottom = Info.dwSize.Y - 1; - Pos.X = 0; - Pos.Y = Info.dwCursorPosition.Y; - CharInfo.Char.UnicodeChar = ' '; - CharInfo.Attributes = Info.wAttributes; - ScrollConsoleScreenBuffer(hConOut, &Rect, NULL, Pos, &CharInfo); - return; - - case 'P': // ESC[#P Delete # characters. - if (es_argc == 0) es_argv[es_argc++] = 1; // ESC[P == ESC[1P - if (es_argc != 1) return; - if (Info.dwCursorPosition.X + es_argv[0] > Info.dwSize.X - 1) - es_argv[0] = Info.dwSize.X - Info.dwCursorPosition.X; - Rect.Left = Info.dwCursorPosition.X + es_argv[0]; - Rect.Top = Info.dwCursorPosition.Y; - Rect.Right = Info.dwSize.X - 1; - Rect.Bottom = Info.dwCursorPosition.Y; - CharInfo.Char.UnicodeChar = ' '; - CharInfo.Attributes = Info.wAttributes; - ScrollConsoleScreenBuffer(hConOut, &Rect, NULL, Info.dwCursorPosition, - &CharInfo); - return; - - case '@': // ESC[#@ Insert # blank characters. - if (es_argc == 0) es_argv[es_argc++] = 1; // ESC[@ == ESC[1@ - if (es_argc != 1) return; - if (Info.dwCursorPosition.X + es_argv[0] > Info.dwSize.X - 1) - es_argv[0] = Info.dwSize.X - Info.dwCursorPosition.X; - Rect.Left = Info.dwCursorPosition.X; - Rect.Top = Info.dwCursorPosition.Y; - Rect.Right = Info.dwSize.X - 1 - es_argv[0]; - Rect.Bottom = Info.dwCursorPosition.Y; - Pos.X = Info.dwCursorPosition.X + es_argv[0]; - Pos.Y = Info.dwCursorPosition.Y; - CharInfo.Char.UnicodeChar = ' '; - CharInfo.Attributes = Info.wAttributes; - ScrollConsoleScreenBuffer(hConOut, &Rect, NULL, Pos, &CharInfo); - return; - - case 'k': // ESC[#k - case 'A': // ESC[#A Moves cursor up # lines - if (es_argc == 0) es_argv[es_argc++] = 1; // ESC[A == ESC[1A - if (es_argc != 1) return; - Pos.Y = Info.dwCursorPosition.Y - es_argv[0]; - if (Pos.Y < 0) Pos.Y = 0; - Pos.X = Info.dwCursorPosition.X; - SetConsoleCursorPosition(hConOut, Pos); - return; - - case 'e': // ESC[#e - case 'B': // ESC[#B Moves cursor down # lines - if (es_argc == 0) es_argv[es_argc++] = 1; // ESC[B == ESC[1B - if (es_argc != 1) return; - Pos.Y = Info.dwCursorPosition.Y + es_argv[0]; - if (Pos.Y >= Info.dwSize.Y) Pos.Y = Info.dwSize.Y - 1; - Pos.X = Info.dwCursorPosition.X; - SetConsoleCursorPosition(hConOut, Pos); - return; - - case 'a': // ESC[#a - case 'C': // ESC[#C Moves cursor forward # spaces - if (es_argc == 0) es_argv[es_argc++] = 1; // ESC[C == ESC[1C - if (es_argc != 1) return; - Pos.X = Info.dwCursorPosition.X + es_argv[0]; - if (Pos.X >= Info.dwSize.X) Pos.X = Info.dwSize.X - 1; - Pos.Y = Info.dwCursorPosition.Y; - SetConsoleCursorPosition(hConOut, Pos); - return; - - case 'j': // ESC[#j - case 'D': // ESC[#D Moves cursor back # spaces - if (es_argc == 0) es_argv[es_argc++] = 1; // ESC[D == ESC[1D - if (es_argc != 1) return; - Pos.X = Info.dwCursorPosition.X - es_argv[0]; - if (Pos.X < 0) Pos.X = 0; - Pos.Y = Info.dwCursorPosition.Y; - SetConsoleCursorPosition(hConOut, Pos); - return; - - case 'E': // ESC[#E Moves cursor down # lines, column 1. - if (es_argc == 0) es_argv[es_argc++] = 1; // ESC[E == ESC[1E - if (es_argc != 1) return; - Pos.Y = Info.dwCursorPosition.Y + es_argv[0]; - if (Pos.Y >= Info.dwSize.Y) Pos.Y = Info.dwSize.Y - 1; - Pos.X = 0; - SetConsoleCursorPosition(hConOut, Pos); - return; - - case 'F': // ESC[#F Moves cursor up # lines, column 1. - if (es_argc == 0) es_argv[es_argc++] = 1; // ESC[F == ESC[1F - if (es_argc != 1) return; - Pos.Y = Info.dwCursorPosition.Y - es_argv[0]; - if (Pos.Y < 0) Pos.Y = 0; - Pos.X = 0; - SetConsoleCursorPosition(hConOut, Pos); - return; - - case '`': // ESC[#` - case 'G': // ESC[#G Moves cursor column # in current row. - if (es_argc == 0) es_argv[es_argc++] = 1; // ESC[G == ESC[1G - if (es_argc != 1) return; - Pos.X = es_argv[0] - 1; - if (Pos.X >= Info.dwSize.X) Pos.X = Info.dwSize.X - 1; - if (Pos.X < 0) Pos.X = 0; - Pos.Y = Info.dwCursorPosition.Y; - SetConsoleCursorPosition(hConOut, Pos); - return; - - case 'd': // ESC[#d Moves cursor row #, current column. - if (es_argc == 0) es_argv[es_argc++] = 1; // ESC[d == ESC[1d - if (es_argc != 1) return; - Pos.Y = es_argv[0] - 1; - if (Pos.Y < 0) Pos.Y = 0; - if (Pos.Y >= Info.dwSize.Y) Pos.Y = Info.dwSize.Y - 1; - SetConsoleCursorPosition(hConOut, Pos); - return; - - case 'f': // ESC[#;#f - case 'H': // ESC[#;#H Moves cursor to line #, column # - if (es_argc == 0) - es_argv[es_argc++] = 1; // ESC[H == ESC[1;1H - if (es_argc == 1) - es_argv[es_argc++] = 1; // ESC[#H == ESC[#;1H - if (es_argc > 2) return; - Pos.X = es_argv[1] - 1; - if (Pos.X < 0) Pos.X = 0; - if (Pos.X >= Info.dwSize.X) Pos.X = Info.dwSize.X - 1; - Pos.Y = es_argv[0] - 1; - if (Pos.Y < 0) Pos.Y = 0; - if (Pos.Y >= Info.dwSize.Y) Pos.Y = Info.dwSize.Y - 1; - SetConsoleCursorPosition(hConOut, Pos); - return; - - case 's': // ESC[s Saves cursor position for recall later - if (es_argc != 0) return; - SavePos = Info.dwCursorPosition; - return; - - case 'u': // ESC[u Return to saved cursor position - if (es_argc != 0) return; - SetConsoleCursorPosition(hConOut, SavePos); - return; - - case 'n': // ESC[#n Device status report - if (es_argc != 1) return; // ESC[n == ESC[0n -> ignored - switch (es_argv[0]) - { - case 5: // ESC[5n Report status - SendSequence(L"\33[0n"); // "OK" - return; - - case 6: // ESC[6n Report cursor position - { - WCHAR buf[32]; - swprintf(buf, 32, L"\33[%d;%dR", Info.dwCursorPosition.Y + 1, - Info.dwCursorPosition.X + 1); - SendSequence(buf); - } - return; - - default: - return; - } - - case 't': // ESC[#t Window manipulation - if (es_argc != 1) return; - if (es_argv[0] == 21) // ESC[21t Report xterm window's title - { - WCHAR buf[MAX_PATH * 2]; - DWORD len = GetConsoleTitleW(buf + 3, lenof(buf) - 3 - 2); - // Too bad if it's too big or fails. - buf[0] = ESC; - buf[1] = ']'; - buf[2] = 'l'; - buf[3 + len] = ESC; - buf[3 + len + 1] = '\\'; - buf[3 + len + 2] = '\0'; - SendSequence(buf); - } - return; - - default: - return; - } - } - else // (prefix == ']') - { - // Ignore any \e]? or \e]> sequences. - if (prefix2 != 0) - return; - - if (es_argc == 1 && es_argv[0] == 0) // ESC]0;titleST - { - SetConsoleTitleW(Pt_arg); - } - } -} - -//----------------------------------------------------------------------------- -// ParseAndPrintANSIString(hDev, lpBuffer, nNumberOfBytesToWrite) -// Parses the string lpBuffer, interprets the escapes sequences and prints the -// characters in the device hDev (console). -// The lexer is a three states automata. -// If the number of arguments es_argc > MAX_ARG, only the MAX_ARG-1 firsts and -// the last arguments are processed (no es_argv[] overflow). -//----------------------------------------------------------------------------- - -inline BOOL ParseAndPrintANSIString(HANDLE hDev, LPCVOID lpBuffer, DWORD nNumberOfBytesToWrite, LPDWORD lpNumberOfBytesWritten) -{ - DWORD i; - LPCSTR s; - - if (hDev != hConOut) // reinit if device has changed - { - hConOut = hDev; - state = 1; - shifted = FALSE; - } - for (i = nNumberOfBytesToWrite, s = (LPCSTR)lpBuffer; i > 0; i--, s++) - { - if (state == 1) - { - if (*s == ESC) state = 2; - else if (*s == SO) shifted = TRUE; - else if (*s == SI) shifted = FALSE; - else PushBuffer(*s); - } - else if (state == 2) - { - if (*s == ESC); // \e\e...\e == \e - else if ((*s == '[') || (*s == ']')) - { - FlushBuffer(); - prefix = *s; - prefix2 = 0; - state = 3; - Pt_len = 0; - *Pt_arg = '\0'; - } - else if (*s == ')' || *s == '(') state = 6; - else state = 1; - } - else if (state == 3) - { - if (is_digit(*s)) - { - es_argc = 0; - es_argv[0] = *s - '0'; - state = 4; - } - else if (*s == ';') - { - es_argc = 1; - es_argv[0] = 0; - es_argv[1] = 0; - state = 4; - } - else if (*s == '?' || *s == '>') - { - prefix2 = *s; - } - else - { - es_argc = 0; - suffix = *s; - InterpretEscSeq(); - state = 1; - } - } - else if (state == 4) - { - if (is_digit(*s)) - { - es_argv[es_argc] = 10 * es_argv[es_argc] + (*s - '0'); - } - else if (*s == ';') - { - if (es_argc < MAX_ARG - 1) es_argc++; - es_argv[es_argc] = 0; - if (prefix == ']') - state = 5; - } - else - { - es_argc++; - suffix = *s; - InterpretEscSeq(); - state = 1; - } - } - else if (state == 5) - { - if (*s == BEL) - { - Pt_arg[Pt_len] = '\0'; - InterpretEscSeq(); - state = 1; - } - else if (*s == '\\' && Pt_len > 0 && Pt_arg[Pt_len - 1] == ESC) - { - Pt_arg[--Pt_len] = '\0'; - InterpretEscSeq(); - state = 1; - } - else if (Pt_len < lenof(Pt_arg) - 1) - Pt_arg[Pt_len++] = *s; - } - else if (state == 6) - { - // Ignore it (ESC ) 0 is implicit; nothing else is supported). - state = 1; - } - } - FlushBuffer(); - if (lpNumberOfBytesWritten != NULL) - *lpNumberOfBytesWritten = nNumberOfBytesToWrite - i; - return (i == 0); -} - -} // namespace ansi - -HANDLE hOut; -HANDLE hIn; -DWORD consolemodeIn = 0; - -inline int win32read(int *c) { - DWORD foo; - INPUT_RECORD b; - KEY_EVENT_RECORD e; - BOOL altgr; - - while (1) { - if (!ReadConsoleInput(hIn, &b, 1, &foo)) return 0; - if (!foo) return 0; - - if (b.EventType == KEY_EVENT && b.Event.KeyEvent.bKeyDown) { - - e = b.Event.KeyEvent; - *c = b.Event.KeyEvent.uChar.AsciiChar; - - altgr = e.dwControlKeyState & (LEFT_CTRL_PRESSED | RIGHT_ALT_PRESSED); - - if (e.dwControlKeyState & (LEFT_CTRL_PRESSED | RIGHT_CTRL_PRESSED) && !altgr) { - - /* Ctrl+Key */ - switch (*c) { - case 'D': - *c = 4; - return 1; - case 'C': - *c = 3; - return 1; - case 'H': - *c = 8; - return 1; - case 'T': - *c = 20; - return 1; - case 'B': /* ctrl-b, left_arrow */ - *c = 2; - return 1; - case 'F': /* ctrl-f right_arrow*/ - *c = 6; - return 1; - case 'P': /* ctrl-p up_arrow*/ - *c = 16; - return 1; - case 'N': /* ctrl-n down_arrow*/ - *c = 14; - return 1; - case 'U': /* Ctrl+u, delete the whole line. */ - *c = 21; - return 1; - case 'K': /* Ctrl+k, delete from current to end of line. */ - *c = 11; - return 1; - case 'A': /* Ctrl+a, go to the start of the line */ - *c = 1; - return 1; - case 'E': /* ctrl+e, go to the end of the line */ - *c = 5; - return 1; - } - - /* Other Ctrl+KEYs ignored */ - } else { - - switch (e.wVirtualKeyCode) { - - case VK_ESCAPE: /* ignore - send ctrl-c, will return -1 */ - *c = 3; - return 1; - case VK_RETURN: /* enter */ - *c = 13; - return 1; - case VK_LEFT: /* left */ - *c = 2; - return 1; - case VK_RIGHT: /* right */ - *c = 6; - return 1; - case VK_UP: /* up */ - *c = 16; - return 1; - case VK_DOWN: /* down */ - *c = 14; - return 1; - case VK_HOME: - *c = 1; - return 1; - case VK_END: - *c = 5; - return 1; - case VK_BACK: - *c = 8; - return 1; - case VK_DELETE: - *c = 4; /* same as Ctrl+D above */ - return 1; - default: - if (*c) return 1; - } - } - } - } - - return -1; /* Makes compiler happy */ -} - -inline int win32_write(int fd, const void *buffer, unsigned int count) { - if (fd == _fileno(stdout)) { - DWORD bytesWritten = 0; - if (FALSE != ansi::ParseAndPrintANSIString(GetStdHandle(STD_OUTPUT_HANDLE), buffer, (DWORD)count, &bytesWritten)) { - return (int)bytesWritten; - } else { - errno = GetLastError(); - return 0; - } - } else if (fd == _fileno(stderr)) { - DWORD bytesWritten = 0; - if (FALSE != ansi::ParseAndPrintANSIString(GetStdHandle(STD_ERROR_HANDLE), buffer, (DWORD)count, &bytesWritten)) { - return (int)bytesWritten; - } else { - errno = GetLastError(); - return 0; - } - } else { - return _write(fd, buffer, count); - } -} -#endif // _WIN32 - -#define LINENOISE_DEFAULT_HISTORY_MAX_LEN 100 -#define LINENOISE_MAX_LINE 4096 -static const char *unsupported_term[] = {"dumb","cons25","emacs",NULL}; -static CompletionCallback completionCallback; - -#ifndef _WIN32 -static struct termios orig_termios; /* In order to restore at exit.*/ -#endif -static bool rawmode = false; /* For atexit() function to check if restore is needed*/ -static bool mlmode = false; /* Multi line mode. Default is single line. */ -static bool atexit_registered = false; /* Register atexit just 1 time. */ -static size_t history_max_len = LINENOISE_DEFAULT_HISTORY_MAX_LEN; -static std::vector history; - -/* The linenoiseState structure represents the state during line editing. - * We pass this state to functions implementing specific editing - * functionalities. */ -struct linenoiseState { - int ifd; /* Terminal stdin file descriptor. */ - int ofd; /* Terminal stdout file descriptor. */ - char *buf; /* Edited line buffer. */ - int buflen; /* Edited line buffer size. */ - std::string prompt; /* Prompt to display. */ - int pos; /* Current cursor position. */ - int oldcolpos; /* Previous refresh cursor column position. */ - int len; /* Current edited line length. */ - int cols; /* Number of columns in terminal. */ - int maxrows; /* Maximum num of rows used so far (multiline mode) */ - int history_index; /* The history index we are currently editing. */ -}; - -enum KEY_ACTION { - KEY_NULL = 0, /* NULL */ - CTRL_A = 1, /* Ctrl+a */ - CTRL_B = 2, /* Ctrl-b */ - CTRL_C = 3, /* Ctrl-c */ - CTRL_D = 4, /* Ctrl-d */ - CTRL_E = 5, /* Ctrl-e */ - CTRL_F = 6, /* Ctrl-f */ - CTRL_H = 8, /* Ctrl-h */ - TAB = 9, /* Tab */ - CTRL_K = 11, /* Ctrl+k */ - CTRL_L = 12, /* Ctrl+l */ - ENTER = 13, /* Enter */ - CTRL_N = 14, /* Ctrl-n */ - CTRL_P = 16, /* Ctrl-p */ - CTRL_T = 20, /* Ctrl-t */ - CTRL_U = 21, /* Ctrl+u */ - CTRL_W = 23, /* Ctrl+w */ - ESC = 27, /* Escape */ - BACKSPACE = 127 /* Backspace */ -}; - -void linenoiseAtExit(void); -bool AddHistory(const char *line); -void refreshLine(struct linenoiseState *l); - -/* ============================ UTF8 utilities ============================== */ - -static unsigned long unicodeWideCharTable[][2] = { - { 0x1100, 0x115F }, { 0x2329, 0x232A }, { 0x2E80, 0x2E99, }, { 0x2E9B, 0x2EF3, }, - { 0x2F00, 0x2FD5, }, { 0x2FF0, 0x2FFB, }, { 0x3000, 0x303E, }, { 0x3041, 0x3096, }, - { 0x3099, 0x30FF, }, { 0x3105, 0x312D, }, { 0x3131, 0x318E, }, { 0x3190, 0x31BA, }, - { 0x31C0, 0x31E3, }, { 0x31F0, 0x321E, }, { 0x3220, 0x3247, }, { 0x3250, 0x4DBF, }, - { 0x4E00, 0xA48C, }, { 0xA490, 0xA4C6, }, { 0xA960, 0xA97C, }, { 0xAC00, 0xD7A3, }, - { 0xF900, 0xFAFF, }, { 0xFE10, 0xFE19, }, { 0xFE30, 0xFE52, }, { 0xFE54, 0xFE66, }, - { 0xFE68, 0xFE6B, }, { 0xFF01, 0xFFE6, }, - { 0x1B000, 0x1B001, }, { 0x1F200, 0x1F202, }, { 0x1F210, 0x1F23A, }, - { 0x1F240, 0x1F248, }, { 0x1F250, 0x1F251, }, { 0x20000, 0x3FFFD, }, -}; - -static int unicodeWideCharTableSize = sizeof(unicodeWideCharTable) / sizeof(unicodeWideCharTable[0]); - -static int unicodeIsWideChar(unsigned long cp) -{ - int i; - for (i = 0; i < unicodeWideCharTableSize; i++) { - if (unicodeWideCharTable[i][0] <= cp && cp <= unicodeWideCharTable[i][1]) { - return 1; - } - } - return 0; -} - -static unsigned long unicodeCombiningCharTable[] = { - 0x0300,0x0301,0x0302,0x0303,0x0304,0x0305,0x0306,0x0307, - 0x0308,0x0309,0x030A,0x030B,0x030C,0x030D,0x030E,0x030F, - 0x0310,0x0311,0x0312,0x0313,0x0314,0x0315,0x0316,0x0317, - 0x0318,0x0319,0x031A,0x031B,0x031C,0x031D,0x031E,0x031F, - 0x0320,0x0321,0x0322,0x0323,0x0324,0x0325,0x0326,0x0327, - 0x0328,0x0329,0x032A,0x032B,0x032C,0x032D,0x032E,0x032F, - 0x0330,0x0331,0x0332,0x0333,0x0334,0x0335,0x0336,0x0337, - 0x0338,0x0339,0x033A,0x033B,0x033C,0x033D,0x033E,0x033F, - 0x0340,0x0341,0x0342,0x0343,0x0344,0x0345,0x0346,0x0347, - 0x0348,0x0349,0x034A,0x034B,0x034C,0x034D,0x034E,0x034F, - 0x0350,0x0351,0x0352,0x0353,0x0354,0x0355,0x0356,0x0357, - 0x0358,0x0359,0x035A,0x035B,0x035C,0x035D,0x035E,0x035F, - 0x0360,0x0361,0x0362,0x0363,0x0364,0x0365,0x0366,0x0367, - 0x0368,0x0369,0x036A,0x036B,0x036C,0x036D,0x036E,0x036F, - 0x0483,0x0484,0x0485,0x0486,0x0487,0x0591,0x0592,0x0593, - 0x0594,0x0595,0x0596,0x0597,0x0598,0x0599,0x059A,0x059B, - 0x059C,0x059D,0x059E,0x059F,0x05A0,0x05A1,0x05A2,0x05A3, - 0x05A4,0x05A5,0x05A6,0x05A7,0x05A8,0x05A9,0x05AA,0x05AB, - 0x05AC,0x05AD,0x05AE,0x05AF,0x05B0,0x05B1,0x05B2,0x05B3, - 0x05B4,0x05B5,0x05B6,0x05B7,0x05B8,0x05B9,0x05BA,0x05BB, - 0x05BC,0x05BD,0x05BF,0x05C1,0x05C2,0x05C4,0x05C5,0x05C7, - 0x0610,0x0611,0x0612,0x0613,0x0614,0x0615,0x0616,0x0617, - 0x0618,0x0619,0x061A,0x064B,0x064C,0x064D,0x064E,0x064F, - 0x0650,0x0651,0x0652,0x0653,0x0654,0x0655,0x0656,0x0657, - 0x0658,0x0659,0x065A,0x065B,0x065C,0x065D,0x065E,0x065F, - 0x0670,0x06D6,0x06D7,0x06D8,0x06D9,0x06DA,0x06DB,0x06DC, - 0x06DF,0x06E0,0x06E1,0x06E2,0x06E3,0x06E4,0x06E7,0x06E8, - 0x06EA,0x06EB,0x06EC,0x06ED,0x0711,0x0730,0x0731,0x0732, - 0x0733,0x0734,0x0735,0x0736,0x0737,0x0738,0x0739,0x073A, - 0x073B,0x073C,0x073D,0x073E,0x073F,0x0740,0x0741,0x0742, - 0x0743,0x0744,0x0745,0x0746,0x0747,0x0748,0x0749,0x074A, - 0x07A6,0x07A7,0x07A8,0x07A9,0x07AA,0x07AB,0x07AC,0x07AD, - 0x07AE,0x07AF,0x07B0,0x07EB,0x07EC,0x07ED,0x07EE,0x07EF, - 0x07F0,0x07F1,0x07F2,0x07F3,0x0816,0x0817,0x0818,0x0819, - 0x081B,0x081C,0x081D,0x081E,0x081F,0x0820,0x0821,0x0822, - 0x0823,0x0825,0x0826,0x0827,0x0829,0x082A,0x082B,0x082C, - 0x082D,0x0859,0x085A,0x085B,0x08E3,0x08E4,0x08E5,0x08E6, - 0x08E7,0x08E8,0x08E9,0x08EA,0x08EB,0x08EC,0x08ED,0x08EE, - 0x08EF,0x08F0,0x08F1,0x08F2,0x08F3,0x08F4,0x08F5,0x08F6, - 0x08F7,0x08F8,0x08F9,0x08FA,0x08FB,0x08FC,0x08FD,0x08FE, - 0x08FF,0x0900,0x0901,0x0902,0x093A,0x093C,0x0941,0x0942, - 0x0943,0x0944,0x0945,0x0946,0x0947,0x0948,0x094D,0x0951, - 0x0952,0x0953,0x0954,0x0955,0x0956,0x0957,0x0962,0x0963, - 0x0981,0x09BC,0x09C1,0x09C2,0x09C3,0x09C4,0x09CD,0x09E2, - 0x09E3,0x0A01,0x0A02,0x0A3C,0x0A41,0x0A42,0x0A47,0x0A48, - 0x0A4B,0x0A4C,0x0A4D,0x0A51,0x0A70,0x0A71,0x0A75,0x0A81, - 0x0A82,0x0ABC,0x0AC1,0x0AC2,0x0AC3,0x0AC4,0x0AC5,0x0AC7, - 0x0AC8,0x0ACD,0x0AE2,0x0AE3,0x0B01,0x0B3C,0x0B3F,0x0B41, - 0x0B42,0x0B43,0x0B44,0x0B4D,0x0B56,0x0B62,0x0B63,0x0B82, - 0x0BC0,0x0BCD,0x0C00,0x0C3E,0x0C3F,0x0C40,0x0C46,0x0C47, - 0x0C48,0x0C4A,0x0C4B,0x0C4C,0x0C4D,0x0C55,0x0C56,0x0C62, - 0x0C63,0x0C81,0x0CBC,0x0CBF,0x0CC6,0x0CCC,0x0CCD,0x0CE2, - 0x0CE3,0x0D01,0x0D41,0x0D42,0x0D43,0x0D44,0x0D4D,0x0D62, - 0x0D63,0x0DCA,0x0DD2,0x0DD3,0x0DD4,0x0DD6,0x0E31,0x0E34, - 0x0E35,0x0E36,0x0E37,0x0E38,0x0E39,0x0E3A,0x0E47,0x0E48, - 0x0E49,0x0E4A,0x0E4B,0x0E4C,0x0E4D,0x0E4E,0x0EB1,0x0EB4, - 0x0EB5,0x0EB6,0x0EB7,0x0EB8,0x0EB9,0x0EBB,0x0EBC,0x0EC8, - 0x0EC9,0x0ECA,0x0ECB,0x0ECC,0x0ECD,0x0F18,0x0F19,0x0F35, - 0x0F37,0x0F39,0x0F71,0x0F72,0x0F73,0x0F74,0x0F75,0x0F76, - 0x0F77,0x0F78,0x0F79,0x0F7A,0x0F7B,0x0F7C,0x0F7D,0x0F7E, - 0x0F80,0x0F81,0x0F82,0x0F83,0x0F84,0x0F86,0x0F87,0x0F8D, - 0x0F8E,0x0F8F,0x0F90,0x0F91,0x0F92,0x0F93,0x0F94,0x0F95, - 0x0F96,0x0F97,0x0F99,0x0F9A,0x0F9B,0x0F9C,0x0F9D,0x0F9E, - 0x0F9F,0x0FA0,0x0FA1,0x0FA2,0x0FA3,0x0FA4,0x0FA5,0x0FA6, - 0x0FA7,0x0FA8,0x0FA9,0x0FAA,0x0FAB,0x0FAC,0x0FAD,0x0FAE, - 0x0FAF,0x0FB0,0x0FB1,0x0FB2,0x0FB3,0x0FB4,0x0FB5,0x0FB6, - 0x0FB7,0x0FB8,0x0FB9,0x0FBA,0x0FBB,0x0FBC,0x0FC6,0x102D, - 0x102E,0x102F,0x1030,0x1032,0x1033,0x1034,0x1035,0x1036, - 0x1037,0x1039,0x103A,0x103D,0x103E,0x1058,0x1059,0x105E, - 0x105F,0x1060,0x1071,0x1072,0x1073,0x1074,0x1082,0x1085, - 0x1086,0x108D,0x109D,0x135D,0x135E,0x135F,0x1712,0x1713, - 0x1714,0x1732,0x1733,0x1734,0x1752,0x1753,0x1772,0x1773, - 0x17B4,0x17B5,0x17B7,0x17B8,0x17B9,0x17BA,0x17BB,0x17BC, - 0x17BD,0x17C6,0x17C9,0x17CA,0x17CB,0x17CC,0x17CD,0x17CE, - 0x17CF,0x17D0,0x17D1,0x17D2,0x17D3,0x17DD,0x180B,0x180C, - 0x180D,0x18A9,0x1920,0x1921,0x1922,0x1927,0x1928,0x1932, - 0x1939,0x193A,0x193B,0x1A17,0x1A18,0x1A1B,0x1A56,0x1A58, - 0x1A59,0x1A5A,0x1A5B,0x1A5C,0x1A5D,0x1A5E,0x1A60,0x1A62, - 0x1A65,0x1A66,0x1A67,0x1A68,0x1A69,0x1A6A,0x1A6B,0x1A6C, - 0x1A73,0x1A74,0x1A75,0x1A76,0x1A77,0x1A78,0x1A79,0x1A7A, - 0x1A7B,0x1A7C,0x1A7F,0x1AB0,0x1AB1,0x1AB2,0x1AB3,0x1AB4, - 0x1AB5,0x1AB6,0x1AB7,0x1AB8,0x1AB9,0x1ABA,0x1ABB,0x1ABC, - 0x1ABD,0x1B00,0x1B01,0x1B02,0x1B03,0x1B34,0x1B36,0x1B37, - 0x1B38,0x1B39,0x1B3A,0x1B3C,0x1B42,0x1B6B,0x1B6C,0x1B6D, - 0x1B6E,0x1B6F,0x1B70,0x1B71,0x1B72,0x1B73,0x1B80,0x1B81, - 0x1BA2,0x1BA3,0x1BA4,0x1BA5,0x1BA8,0x1BA9,0x1BAB,0x1BAC, - 0x1BAD,0x1BE6,0x1BE8,0x1BE9,0x1BED,0x1BEF,0x1BF0,0x1BF1, - 0x1C2C,0x1C2D,0x1C2E,0x1C2F,0x1C30,0x1C31,0x1C32,0x1C33, - 0x1C36,0x1C37,0x1CD0,0x1CD1,0x1CD2,0x1CD4,0x1CD5,0x1CD6, - 0x1CD7,0x1CD8,0x1CD9,0x1CDA,0x1CDB,0x1CDC,0x1CDD,0x1CDE, - 0x1CDF,0x1CE0,0x1CE2,0x1CE3,0x1CE4,0x1CE5,0x1CE6,0x1CE7, - 0x1CE8,0x1CED,0x1CF4,0x1CF8,0x1CF9,0x1DC0,0x1DC1,0x1DC2, - 0x1DC3,0x1DC4,0x1DC5,0x1DC6,0x1DC7,0x1DC8,0x1DC9,0x1DCA, - 0x1DCB,0x1DCC,0x1DCD,0x1DCE,0x1DCF,0x1DD0,0x1DD1,0x1DD2, - 0x1DD3,0x1DD4,0x1DD5,0x1DD6,0x1DD7,0x1DD8,0x1DD9,0x1DDA, - 0x1DDB,0x1DDC,0x1DDD,0x1DDE,0x1DDF,0x1DE0,0x1DE1,0x1DE2, - 0x1DE3,0x1DE4,0x1DE5,0x1DE6,0x1DE7,0x1DE8,0x1DE9,0x1DEA, - 0x1DEB,0x1DEC,0x1DED,0x1DEE,0x1DEF,0x1DF0,0x1DF1,0x1DF2, - 0x1DF3,0x1DF4,0x1DF5,0x1DFC,0x1DFD,0x1DFE,0x1DFF,0x20D0, - 0x20D1,0x20D2,0x20D3,0x20D4,0x20D5,0x20D6,0x20D7,0x20D8, - 0x20D9,0x20DA,0x20DB,0x20DC,0x20E1,0x20E5,0x20E6,0x20E7, - 0x20E8,0x20E9,0x20EA,0x20EB,0x20EC,0x20ED,0x20EE,0x20EF, - 0x20F0,0x2CEF,0x2CF0,0x2CF1,0x2D7F,0x2DE0,0x2DE1,0x2DE2, - 0x2DE3,0x2DE4,0x2DE5,0x2DE6,0x2DE7,0x2DE8,0x2DE9,0x2DEA, - 0x2DEB,0x2DEC,0x2DED,0x2DEE,0x2DEF,0x2DF0,0x2DF1,0x2DF2, - 0x2DF3,0x2DF4,0x2DF5,0x2DF6,0x2DF7,0x2DF8,0x2DF9,0x2DFA, - 0x2DFB,0x2DFC,0x2DFD,0x2DFE,0x2DFF,0x302A,0x302B,0x302C, - 0x302D,0x3099,0x309A,0xA66F,0xA674,0xA675,0xA676,0xA677, - 0xA678,0xA679,0xA67A,0xA67B,0xA67C,0xA67D,0xA69E,0xA69F, - 0xA6F0,0xA6F1,0xA802,0xA806,0xA80B,0xA825,0xA826,0xA8C4, - 0xA8E0,0xA8E1,0xA8E2,0xA8E3,0xA8E4,0xA8E5,0xA8E6,0xA8E7, - 0xA8E8,0xA8E9,0xA8EA,0xA8EB,0xA8EC,0xA8ED,0xA8EE,0xA8EF, - 0xA8F0,0xA8F1,0xA926,0xA927,0xA928,0xA929,0xA92A,0xA92B, - 0xA92C,0xA92D,0xA947,0xA948,0xA949,0xA94A,0xA94B,0xA94C, - 0xA94D,0xA94E,0xA94F,0xA950,0xA951,0xA980,0xA981,0xA982, - 0xA9B3,0xA9B6,0xA9B7,0xA9B8,0xA9B9,0xA9BC,0xA9E5,0xAA29, - 0xAA2A,0xAA2B,0xAA2C,0xAA2D,0xAA2E,0xAA31,0xAA32,0xAA35, - 0xAA36,0xAA43,0xAA4C,0xAA7C,0xAAB0,0xAAB2,0xAAB3,0xAAB4, - 0xAAB7,0xAAB8,0xAABE,0xAABF,0xAAC1,0xAAEC,0xAAED,0xAAF6, - 0xABE5,0xABE8,0xABED,0xFB1E,0xFE00,0xFE01,0xFE02,0xFE03, - 0xFE04,0xFE05,0xFE06,0xFE07,0xFE08,0xFE09,0xFE0A,0xFE0B, - 0xFE0C,0xFE0D,0xFE0E,0xFE0F,0xFE20,0xFE21,0xFE22,0xFE23, - 0xFE24,0xFE25,0xFE26,0xFE27,0xFE28,0xFE29,0xFE2A,0xFE2B, - 0xFE2C,0xFE2D,0xFE2E,0xFE2F, - 0x101FD,0x102E0,0x10376,0x10377,0x10378,0x10379,0x1037A,0x10A01, - 0x10A02,0x10A03,0x10A05,0x10A06,0x10A0C,0x10A0D,0x10A0E,0x10A0F, - 0x10A38,0x10A39,0x10A3A,0x10A3F,0x10AE5,0x10AE6,0x11001,0x11038, - 0x11039,0x1103A,0x1103B,0x1103C,0x1103D,0x1103E,0x1103F,0x11040, - 0x11041,0x11042,0x11043,0x11044,0x11045,0x11046,0x1107F,0x11080, - 0x11081,0x110B3,0x110B4,0x110B5,0x110B6,0x110B9,0x110BA,0x11100, - 0x11101,0x11102,0x11127,0x11128,0x11129,0x1112A,0x1112B,0x1112D, - 0x1112E,0x1112F,0x11130,0x11131,0x11132,0x11133,0x11134,0x11173, - 0x11180,0x11181,0x111B6,0x111B7,0x111B8,0x111B9,0x111BA,0x111BB, - 0x111BC,0x111BD,0x111BE,0x111CA,0x111CB,0x111CC,0x1122F,0x11230, - 0x11231,0x11234,0x11236,0x11237,0x112DF,0x112E3,0x112E4,0x112E5, - 0x112E6,0x112E7,0x112E8,0x112E9,0x112EA,0x11300,0x11301,0x1133C, - 0x11340,0x11366,0x11367,0x11368,0x11369,0x1136A,0x1136B,0x1136C, - 0x11370,0x11371,0x11372,0x11373,0x11374,0x114B3,0x114B4,0x114B5, - 0x114B6,0x114B7,0x114B8,0x114BA,0x114BF,0x114C0,0x114C2,0x114C3, - 0x115B2,0x115B3,0x115B4,0x115B5,0x115BC,0x115BD,0x115BF,0x115C0, - 0x115DC,0x115DD,0x11633,0x11634,0x11635,0x11636,0x11637,0x11638, - 0x11639,0x1163A,0x1163D,0x1163F,0x11640,0x116AB,0x116AD,0x116B0, - 0x116B1,0x116B2,0x116B3,0x116B4,0x116B5,0x116B7,0x1171D,0x1171E, - 0x1171F,0x11722,0x11723,0x11724,0x11725,0x11727,0x11728,0x11729, - 0x1172A,0x1172B,0x16AF0,0x16AF1,0x16AF2,0x16AF3,0x16AF4,0x16B30, - 0x16B31,0x16B32,0x16B33,0x16B34,0x16B35,0x16B36,0x16F8F,0x16F90, - 0x16F91,0x16F92,0x1BC9D,0x1BC9E,0x1D167,0x1D168,0x1D169,0x1D17B, - 0x1D17C,0x1D17D,0x1D17E,0x1D17F,0x1D180,0x1D181,0x1D182,0x1D185, - 0x1D186,0x1D187,0x1D188,0x1D189,0x1D18A,0x1D18B,0x1D1AA,0x1D1AB, - 0x1D1AC,0x1D1AD,0x1D242,0x1D243,0x1D244,0x1DA00,0x1DA01,0x1DA02, - 0x1DA03,0x1DA04,0x1DA05,0x1DA06,0x1DA07,0x1DA08,0x1DA09,0x1DA0A, - 0x1DA0B,0x1DA0C,0x1DA0D,0x1DA0E,0x1DA0F,0x1DA10,0x1DA11,0x1DA12, - 0x1DA13,0x1DA14,0x1DA15,0x1DA16,0x1DA17,0x1DA18,0x1DA19,0x1DA1A, - 0x1DA1B,0x1DA1C,0x1DA1D,0x1DA1E,0x1DA1F,0x1DA20,0x1DA21,0x1DA22, - 0x1DA23,0x1DA24,0x1DA25,0x1DA26,0x1DA27,0x1DA28,0x1DA29,0x1DA2A, - 0x1DA2B,0x1DA2C,0x1DA2D,0x1DA2E,0x1DA2F,0x1DA30,0x1DA31,0x1DA32, - 0x1DA33,0x1DA34,0x1DA35,0x1DA36,0x1DA3B,0x1DA3C,0x1DA3D,0x1DA3E, - 0x1DA3F,0x1DA40,0x1DA41,0x1DA42,0x1DA43,0x1DA44,0x1DA45,0x1DA46, - 0x1DA47,0x1DA48,0x1DA49,0x1DA4A,0x1DA4B,0x1DA4C,0x1DA4D,0x1DA4E, - 0x1DA4F,0x1DA50,0x1DA51,0x1DA52,0x1DA53,0x1DA54,0x1DA55,0x1DA56, - 0x1DA57,0x1DA58,0x1DA59,0x1DA5A,0x1DA5B,0x1DA5C,0x1DA5D,0x1DA5E, - 0x1DA5F,0x1DA60,0x1DA61,0x1DA62,0x1DA63,0x1DA64,0x1DA65,0x1DA66, - 0x1DA67,0x1DA68,0x1DA69,0x1DA6A,0x1DA6B,0x1DA6C,0x1DA75,0x1DA84, - 0x1DA9B,0x1DA9C,0x1DA9D,0x1DA9E,0x1DA9F,0x1DAA1,0x1DAA2,0x1DAA3, - 0x1DAA4,0x1DAA5,0x1DAA6,0x1DAA7,0x1DAA8,0x1DAA9,0x1DAAA,0x1DAAB, - 0x1DAAC,0x1DAAD,0x1DAAE,0x1DAAF,0x1E8D0,0x1E8D1,0x1E8D2,0x1E8D3, - 0x1E8D4,0x1E8D5,0x1E8D6,0xE0100,0xE0101,0xE0102,0xE0103,0xE0104, - 0xE0105,0xE0106,0xE0107,0xE0108,0xE0109,0xE010A,0xE010B,0xE010C, - 0xE010D,0xE010E,0xE010F,0xE0110,0xE0111,0xE0112,0xE0113,0xE0114, - 0xE0115,0xE0116,0xE0117,0xE0118,0xE0119,0xE011A,0xE011B,0xE011C, - 0xE011D,0xE011E,0xE011F,0xE0120,0xE0121,0xE0122,0xE0123,0xE0124, - 0xE0125,0xE0126,0xE0127,0xE0128,0xE0129,0xE012A,0xE012B,0xE012C, - 0xE012D,0xE012E,0xE012F,0xE0130,0xE0131,0xE0132,0xE0133,0xE0134, - 0xE0135,0xE0136,0xE0137,0xE0138,0xE0139,0xE013A,0xE013B,0xE013C, - 0xE013D,0xE013E,0xE013F,0xE0140,0xE0141,0xE0142,0xE0143,0xE0144, - 0xE0145,0xE0146,0xE0147,0xE0148,0xE0149,0xE014A,0xE014B,0xE014C, - 0xE014D,0xE014E,0xE014F,0xE0150,0xE0151,0xE0152,0xE0153,0xE0154, - 0xE0155,0xE0156,0xE0157,0xE0158,0xE0159,0xE015A,0xE015B,0xE015C, - 0xE015D,0xE015E,0xE015F,0xE0160,0xE0161,0xE0162,0xE0163,0xE0164, - 0xE0165,0xE0166,0xE0167,0xE0168,0xE0169,0xE016A,0xE016B,0xE016C, - 0xE016D,0xE016E,0xE016F,0xE0170,0xE0171,0xE0172,0xE0173,0xE0174, - 0xE0175,0xE0176,0xE0177,0xE0178,0xE0179,0xE017A,0xE017B,0xE017C, - 0xE017D,0xE017E,0xE017F,0xE0180,0xE0181,0xE0182,0xE0183,0xE0184, - 0xE0185,0xE0186,0xE0187,0xE0188,0xE0189,0xE018A,0xE018B,0xE018C, - 0xE018D,0xE018E,0xE018F,0xE0190,0xE0191,0xE0192,0xE0193,0xE0194, - 0xE0195,0xE0196,0xE0197,0xE0198,0xE0199,0xE019A,0xE019B,0xE019C, - 0xE019D,0xE019E,0xE019F,0xE01A0,0xE01A1,0xE01A2,0xE01A3,0xE01A4, - 0xE01A5,0xE01A6,0xE01A7,0xE01A8,0xE01A9,0xE01AA,0xE01AB,0xE01AC, - 0xE01AD,0xE01AE,0xE01AF,0xE01B0,0xE01B1,0xE01B2,0xE01B3,0xE01B4, - 0xE01B5,0xE01B6,0xE01B7,0xE01B8,0xE01B9,0xE01BA,0xE01BB,0xE01BC, - 0xE01BD,0xE01BE,0xE01BF,0xE01C0,0xE01C1,0xE01C2,0xE01C3,0xE01C4, - 0xE01C5,0xE01C6,0xE01C7,0xE01C8,0xE01C9,0xE01CA,0xE01CB,0xE01CC, - 0xE01CD,0xE01CE,0xE01CF,0xE01D0,0xE01D1,0xE01D2,0xE01D3,0xE01D4, - 0xE01D5,0xE01D6,0xE01D7,0xE01D8,0xE01D9,0xE01DA,0xE01DB,0xE01DC, - 0xE01DD,0xE01DE,0xE01DF,0xE01E0,0xE01E1,0xE01E2,0xE01E3,0xE01E4, - 0xE01E5,0xE01E6,0xE01E7,0xE01E8,0xE01E9,0xE01EA,0xE01EB,0xE01EC, - 0xE01ED,0xE01EE,0xE01EF, -}; - -static int unicodeCombiningCharTableSize = sizeof(unicodeCombiningCharTable) / sizeof(unicodeCombiningCharTable[0]); - -inline int unicodeIsCombiningChar(unsigned long cp) -{ - int i; - for (i = 0; i < unicodeCombiningCharTableSize; i++) { - if (unicodeCombiningCharTable[i] == cp) { - return 1; - } - } - return 0; -} - -/* Get length of previous UTF8 character - */ -inline int unicodePrevUTF8CharLen(char* buf, int pos) -{ - int end = pos--; - while (pos >= 0 && ((unsigned char)buf[pos] & 0xC0) == 0x80) { - pos--; - } - return end - pos; -} - -/* Get length of previous UTF8 character - */ -inline int unicodeUTF8CharLen(char* buf, int buf_len, int pos) -{ - if (pos == buf_len) { return 0; } - unsigned char ch = buf[pos]; - if (ch < 0x80) { return 1; } - else if (ch < 0xE0) { return 2; } - else if (ch < 0xF0) { return 3; } - else { return 4; } -} - -/* Convert UTF8 to Unicode code point - */ -inline int unicodeUTF8CharToCodePoint( - const char* buf, - int len, - int* cp) -{ - if (len) { - unsigned char byte = buf[0]; - if ((byte & 0x80) == 0) { - *cp = byte; - return 1; - } else if ((byte & 0xE0) == 0xC0) { - if (len >= 2) { - *cp = (((unsigned long)(buf[0] & 0x1F)) << 6) | - ((unsigned long)(buf[1] & 0x3F)); - return 2; - } - } else if ((byte & 0xF0) == 0xE0) { - if (len >= 3) { - *cp = (((unsigned long)(buf[0] & 0x0F)) << 12) | - (((unsigned long)(buf[1] & 0x3F)) << 6) | - ((unsigned long)(buf[2] & 0x3F)); - return 3; - } - } else if ((byte & 0xF8) == 0xF0) { - if (len >= 4) { - *cp = (((unsigned long)(buf[0] & 0x07)) << 18) | - (((unsigned long)(buf[1] & 0x3F)) << 12) | - (((unsigned long)(buf[2] & 0x3F)) << 6) | - ((unsigned long)(buf[3] & 0x3F)); - return 4; - } - } - } - return 0; -} - -/* Get length of grapheme - */ -inline int unicodeGraphemeLen(char* buf, int buf_len, int pos) -{ - if (pos == buf_len) { - return 0; - } - int beg = pos; - pos += unicodeUTF8CharLen(buf, buf_len, pos); - while (pos < buf_len) { - int len = unicodeUTF8CharLen(buf, buf_len, pos); - int cp = 0; - unicodeUTF8CharToCodePoint(buf + pos, len, &cp); - if (!unicodeIsCombiningChar(cp)) { - return pos - beg; - } - pos += len; - } - return pos - beg; -} - -/* Get length of previous grapheme - */ -inline int unicodePrevGraphemeLen(char* buf, int pos) -{ - if (pos == 0) { - return 0; - } - int end = pos; - while (pos > 0) { - int len = unicodePrevUTF8CharLen(buf, pos); - pos -= len; - int cp = 0; - unicodeUTF8CharToCodePoint(buf + pos, len, &cp); - if (!unicodeIsCombiningChar(cp)) { - return end - pos; - } - } - return 0; -} - -inline int isAnsiEscape(const char* buf, int buf_len, int* len) -{ - if (buf_len > 2 && !memcmp("\033[", buf, 2)) { - int off = 2; - while (off < buf_len) { - switch (buf[off++]) { - case 'A': case 'B': case 'C': case 'D': - case 'E': case 'F': case 'G': case 'H': - case 'J': case 'K': case 'S': case 'T': - case 'f': case 'm': - *len = off; - return 1; - } - } - } - return 0; -} - -/* Get column position for the single line mode. - */ -inline int unicodeColumnPos(const char* buf, int buf_len) -{ - int ret = 0; - - int off = 0; - while (off < buf_len) { - int len; - if (isAnsiEscape(buf + off, buf_len - off, &len)) { - off += len; - continue; - } - - int cp = 0; - len = unicodeUTF8CharToCodePoint(buf + off, buf_len - off, &cp); - - if (!unicodeIsCombiningChar(cp)) { - ret += unicodeIsWideChar(cp) ? 2 : 1; - } - - off += len; - } - - return ret; -} - -/* Get column position for the multi line mode. - */ -inline int unicodeColumnPosForMultiLine(char* buf, int buf_len, int pos, int cols, int ini_pos) -{ - int ret = 0; - int colwid = ini_pos; - - int off = 0; - while (off < buf_len) { - int cp = 0; - int len = unicodeUTF8CharToCodePoint(buf + off, buf_len - off, &cp); - - int wid = 0; - if (!unicodeIsCombiningChar(cp)) { - wid = unicodeIsWideChar(cp) ? 2 : 1; - } - - int dif = (int)(colwid + wid) - (int)cols; - if (dif > 0) { - ret += dif; - colwid = wid; - } else if (dif == 0) { - colwid = 0; - } else { - colwid += wid; - } - - if (off >= pos) { - break; - } - - off += len; - ret += wid; - } - - return ret; -} - -/* Read UTF8 character from file. - */ -inline int unicodeReadUTF8Char(int fd, char* buf, int* cp) -{ - int nread = read(fd,&buf[0],1); - - if (nread <= 0) { return nread; } - - unsigned char byte = buf[0]; - - if ((byte & 0x80) == 0) { - ; - } else if ((byte & 0xE0) == 0xC0) { - nread = read(fd,&buf[1],1); - if (nread <= 0) { return nread; } - } else if ((byte & 0xF0) == 0xE0) { - nread = read(fd,&buf[1],2); - if (nread <= 0) { return nread; } - } else if ((byte & 0xF8) == 0xF0) { - nread = read(fd,&buf[1],3); - if (nread <= 0) { return nread; } - } else { - return -1; - } - - return unicodeUTF8CharToCodePoint(buf, 4, cp); -} - -/* ======================= Low level terminal handling ====================== */ - -/* Set if to use or not the multi line mode. */ -inline void SetMultiLine(bool ml) { - mlmode = ml; -} - -/* Return true if the terminal name is in the list of terminals we know are - * not able to understand basic escape sequences. */ -inline bool isUnsupportedTerm(void) { -#ifndef _WIN32 - char *term = getenv("TERM"); - int j; - - if (term == NULL) return false; - for (j = 0; unsupported_term[j]; j++) - if (!strcasecmp(term,unsupported_term[j])) return true; -#endif - return false; -} - -/* Raw mode: 1960 magic shit. */ -inline bool enableRawMode(int fd) { -#ifndef _WIN32 - struct termios raw; - - if (!isatty(STDIN_FILENO)) goto fatal; - if (!atexit_registered) { - atexit(linenoiseAtExit); - atexit_registered = true; - } - if (tcgetattr(fd,&orig_termios) == -1) goto fatal; - - raw = orig_termios; /* modify the original mode */ - /* input modes: no break, no CR to NL, no parity check, no strip char, - * no start/stop output control. */ - raw.c_iflag &= ~(BRKINT | ICRNL | INPCK | ISTRIP | IXON); - /* output modes - disable post processing */ - // NOTE: Multithreaded issue #20 (https://github.com/yhirose/cpp-linenoise/issues/20) - // raw.c_oflag &= ~(OPOST); - /* control modes - set 8 bit chars */ - raw.c_cflag |= (CS8); - /* local modes - echoing off, canonical off, no extended functions, - * no signal chars (^Z,^C) */ - raw.c_lflag &= ~(ECHO | ICANON | IEXTEN | ISIG); - /* control chars - set return condition: min number of bytes and timer. - * We want read to return every single byte, without timeout. */ - raw.c_cc[VMIN] = 1; raw.c_cc[VTIME] = 0; /* 1 byte, no timer */ - - /* put terminal in raw mode after flushing */ - if (tcsetattr(fd,TCSAFLUSH,&raw) < 0) goto fatal; - rawmode = true; -#else - if (!atexit_registered) { - /* Cleanup them at exit */ - atexit(linenoiseAtExit); - atexit_registered = true; - - /* Init windows console handles only once */ - hOut = GetStdHandle(STD_OUTPUT_HANDLE); - if (hOut==INVALID_HANDLE_VALUE) goto fatal; - } - - DWORD consolemodeOut; - if (!GetConsoleMode(hOut, &consolemodeOut)) { - CloseHandle(hOut); - errno = ENOTTY; - return false; - }; - - hIn = GetStdHandle(STD_INPUT_HANDLE); - if (hIn == INVALID_HANDLE_VALUE) { - CloseHandle(hOut); - errno = ENOTTY; - return false; - } - - GetConsoleMode(hIn, &consolemodeIn); - /* Enable raw mode */ - SetConsoleMode(hIn, consolemodeIn & ~ENABLE_PROCESSED_INPUT); - - rawmode = true; -#endif - return true; - -fatal: - errno = ENOTTY; - return false; -} - -inline void disableRawMode(int fd) { -#ifdef _WIN32 - if (consolemodeIn) { - SetConsoleMode(hIn, consolemodeIn); - consolemodeIn = 0; - } - rawmode = false; -#else - /* Don't even check the return value as it's too late. */ - if (rawmode && tcsetattr(fd,TCSAFLUSH,&orig_termios) != -1) - rawmode = false; -#endif -} - -/* Use the ESC [6n escape sequence to query the horizontal cursor position - * and return it. On error -1 is returned, on success the position of the - * cursor. */ -inline int getCursorPosition(int ifd, int ofd) { - char buf[32]; - int cols, rows; - unsigned int i = 0; - - /* Report cursor location */ - if (write(ofd, "\x1b[6n", 4) != 4) return -1; - - /* Read the response: ESC [ rows ; cols R */ - while (i < sizeof(buf)-1) { - if (read(ifd,buf+i,1) != 1) break; - if (buf[i] == 'R') break; - i++; - } - buf[i] = '\0'; - - /* Parse it. */ - if (buf[0] != ESC || buf[1] != '[') return -1; - if (sscanf(buf+2,"%d;%d",&rows,&cols) != 2) return -1; - return cols; -} - -/* Try to get the number of columns in the current terminal, or assume 80 - * if it fails. */ -inline int getColumns(int ifd, int ofd) { -#ifdef _WIN32 - CONSOLE_SCREEN_BUFFER_INFO b; - - if (!GetConsoleScreenBufferInfo(hOut, &b)) return 80; - return b.srWindow.Right - b.srWindow.Left; -#else - struct winsize ws; - - if (ioctl(1, TIOCGWINSZ, &ws) == -1 || ws.ws_col == 0) { - /* ioctl() failed. Try to query the terminal itself. */ - int start, cols; - - /* Get the initial position so we can restore it later. */ - start = getCursorPosition(ifd,ofd); - if (start == -1) goto failed; - - /* Go to right margin and get position. */ - if (write(ofd,"\x1b[999C",6) != 6) goto failed; - cols = getCursorPosition(ifd,ofd); - if (cols == -1) goto failed; - - /* Restore position. */ - if (cols > start) { - char seq[32]; - snprintf(seq,32,"\x1b[%dD",cols-start); - if (write(ofd,seq,strlen(seq)) == -1) { - /* Can't recover... */ - } - } - return cols; - } else { - return ws.ws_col; - } - -failed: - return 80; -#endif -} - -/* Clear the screen. Used to handle ctrl+l */ -inline void linenoiseClearScreen(void) { - if (write(STDOUT_FILENO,"\x1b[H\x1b[2J",7) <= 0) { - /* nothing to do, just to avoid warning. */ - } -} - -/* Beep, used for completion when there is nothing to complete or when all - * the choices were already shown. */ -inline void linenoiseBeep(void) { - fprintf(stderr, "\x7"); - fflush(stderr); -} - -/* ============================== Completion ================================ */ - -/* This is an helper function for linenoiseEdit() and is called when the - * user types the key in order to complete the string currently in the - * input. - * - * The state of the editing is encapsulated into the pointed linenoiseState - * structure as described in the structure definition. */ -inline int completeLine(struct linenoiseState *ls, char *cbuf, int *c) { - std::vector lc; - int nread = 0, nwritten; - *c = 0; - - completionCallback(ls->buf,lc); - if (lc.empty()) { - linenoiseBeep(); - } else { - int stop = 0, i = 0; - - while(!stop) { - /* Show completion or original buffer */ - if (i < static_cast(lc.size())) { - struct linenoiseState saved = *ls; - - ls->len = ls->pos = static_cast(lc[i].size()); - ls->buf = &lc[i][0]; - refreshLine(ls); - ls->len = saved.len; - ls->pos = saved.pos; - ls->buf = saved.buf; - } else { - refreshLine(ls); - } - - //nread = read(ls->ifd,&c,1); -#ifdef _WIN32 - nread = win32read(c); - if (nread == 1) { - cbuf[0] = *c; - } -#else - nread = unicodeReadUTF8Char(ls->ifd,cbuf,c); -#endif - if (nread <= 0) { - *c = -1; - return nread; - } - - switch(*c) { - case 9: /* tab */ - i = (i+1) % (lc.size()+1); - if (i == static_cast(lc.size())) linenoiseBeep(); - break; - case 27: /* escape */ - /* Re-show original buffer */ - if (i < static_cast(lc.size())) refreshLine(ls); - stop = 1; - break; - default: - /* Update buffer and return */ - if (i < static_cast(lc.size())) { - nwritten = snprintf(ls->buf,ls->buflen,"%s",&lc[i][0]); - ls->len = ls->pos = nwritten; - } - stop = 1; - break; - } - } - } - - return nread; -} - -/* Register a callback function to be called for tab-completion. */ -inline void SetCompletionCallback(CompletionCallback fn) { - completionCallback = fn; -} - -/* =========================== Line editing ================================= */ - -/* Single line low level line refresh. - * - * Rewrite the currently edited line accordingly to the buffer content, - * cursor position, and number of columns of the terminal. */ -inline void refreshSingleLine(struct linenoiseState *l) { - char seq[64]; - int pcolwid = unicodeColumnPos(l->prompt.c_str(), static_cast(l->prompt.length())); - int fd = l->ofd; - char *buf = l->buf; - int len = l->len; - int pos = l->pos; - std::string ab; - - while((pcolwid+unicodeColumnPos(buf, pos)) >= l->cols) { - int glen = unicodeGraphemeLen(buf, len, 0); - buf += glen; - len -= glen; - pos -= glen; - } - while (pcolwid+unicodeColumnPos(buf, len) > l->cols) { - len -= unicodePrevGraphemeLen(buf, len); - } - - /* Cursor to left edge */ - snprintf(seq,64,"\r"); - ab += seq; - /* Write the prompt and the current buffer content */ - ab += l->prompt; - ab.append(buf, len); - /* Erase to right */ - snprintf(seq,64,"\x1b[0K"); - ab += seq; - /* Move cursor to original position. */ - snprintf(seq,64,"\r\x1b[%dC", (int)(unicodeColumnPos(buf, pos)+pcolwid)); - ab += seq; - if (write(fd,ab.c_str(), static_cast(ab.length())) == -1) {} /* Can't recover from write error. */ -} - -/* Multi line low level line refresh. - * - * Rewrite the currently edited line accordingly to the buffer content, - * cursor position, and number of columns of the terminal. */ -inline void refreshMultiLine(struct linenoiseState *l) { - char seq[64]; - int pcolwid = unicodeColumnPos(l->prompt.c_str(), static_cast(l->prompt.length())); - int colpos = unicodeColumnPosForMultiLine(l->buf, l->len, l->len, l->cols, pcolwid); - int colpos2; /* cursor column position. */ - int rows = (pcolwid+colpos+l->cols-1)/l->cols; /* rows used by current buf. */ - int rpos = (pcolwid+l->oldcolpos+l->cols)/l->cols; /* cursor relative row. */ - int rpos2; /* rpos after refresh. */ - int col; /* colum position, zero-based. */ - int old_rows = (int)l->maxrows; - int fd = l->ofd, j; - std::string ab; - - /* Update maxrows if needed. */ - if (rows > (int)l->maxrows) l->maxrows = rows; - - /* First step: clear all the lines used before. To do so start by - * going to the last row. */ - if (old_rows-rpos > 0) { - snprintf(seq,64,"\x1b[%dB", old_rows-rpos); - ab += seq; - } - - /* Now for every row clear it, go up. */ - for (j = 0; j < old_rows-1; j++) { - snprintf(seq,64,"\r\x1b[0K\x1b[1A"); - ab += seq; - } - - /* Clean the top line. */ - snprintf(seq,64,"\r\x1b[0K"); - ab += seq; - - /* Write the prompt and the current buffer content */ - ab += l->prompt; - ab.append(l->buf, l->len); - - /* Get text width to cursor position */ - colpos2 = unicodeColumnPosForMultiLine(l->buf, l->len, l->pos, l->cols, pcolwid); - - /* If we are at the very end of the screen with our prompt, we need to - * emit a newline and move the prompt to the first column. */ - if (l->pos && - l->pos == l->len && - (colpos2+pcolwid) % l->cols == 0) - { - ab += "\n"; - snprintf(seq,64,"\r"); - ab += seq; - rows++; - if (rows > (int)l->maxrows) l->maxrows = rows; - } - - /* Move cursor to right position. */ - rpos2 = (pcolwid+colpos2+l->cols)/l->cols; /* current cursor relative row. */ - - /* Go up till we reach the expected positon. */ - if (rows-rpos2 > 0) { - snprintf(seq,64,"\x1b[%dA", rows-rpos2); - ab += seq; - } - - /* Set column. */ - col = (pcolwid + colpos2) % l->cols; - if (col) - snprintf(seq,64,"\r\x1b[%dC", col); - else - snprintf(seq,64,"\r"); - ab += seq; - - l->oldcolpos = colpos2; - - if (write(fd,ab.c_str(), static_cast(ab.length())) == -1) {} /* Can't recover from write error. */ -} - -/* Calls the two low level functions refreshSingleLine() or - * refreshMultiLine() according to the selected mode. */ -inline void refreshLine(struct linenoiseState *l) { - if (mlmode) - refreshMultiLine(l); - else - refreshSingleLine(l); -} - -/* Insert the character 'c' at cursor current position. - * - * On error writing to the terminal -1 is returned, otherwise 0. */ -inline int linenoiseEditInsert(struct linenoiseState *l, const char* cbuf, int clen) { - if (l->len < l->buflen) { - if (l->len == l->pos) { - memcpy(&l->buf[l->pos],cbuf,clen); - l->pos+=clen; - l->len+=clen;; - l->buf[l->len] = '\0'; - if ((!mlmode && unicodeColumnPos(l->prompt.c_str(), static_cast(l->prompt.length()))+unicodeColumnPos(l->buf,l->len) < l->cols) /* || mlmode */) { - /* Avoid a full update of the line in the - * trivial case. */ - if (write(l->ofd,cbuf,clen) == -1) return -1; - } else { - refreshLine(l); - } - } else { - memmove(l->buf+l->pos+clen,l->buf+l->pos,l->len-l->pos); - memcpy(&l->buf[l->pos],cbuf,clen); - l->pos+=clen; - l->len+=clen; - l->buf[l->len] = '\0'; - refreshLine(l); - } - } - return 0; -} - -/* Move cursor on the left. */ -inline void linenoiseEditMoveLeft(struct linenoiseState *l) { - if (l->pos > 0) { - l->pos -= unicodePrevGraphemeLen(l->buf, l->pos); - refreshLine(l); - } -} - -/* Move cursor on the right. */ -inline void linenoiseEditMoveRight(struct linenoiseState *l) { - if (l->pos != l->len) { - l->pos += unicodeGraphemeLen(l->buf, l->len, l->pos); - refreshLine(l); - } -} - -/* Move cursor to the start of the line. */ -inline void linenoiseEditMoveHome(struct linenoiseState *l) { - if (l->pos != 0) { - l->pos = 0; - refreshLine(l); - } -} - -/* Move cursor to the end of the line. */ -inline void linenoiseEditMoveEnd(struct linenoiseState *l) { - if (l->pos != l->len) { - l->pos = l->len; - refreshLine(l); - } -} - -/* Substitute the currently edited line with the next or previous history - * entry as specified by 'dir'. */ -#define LINENOISE_HISTORY_NEXT 0 -#define LINENOISE_HISTORY_PREV 1 -inline void linenoiseEditHistoryNext(struct linenoiseState *l, int dir) { - if (history.size() > 1) { - /* Update the current history entry before to - * overwrite it with the next one. */ - history[history.size() - 1 - l->history_index] = l->buf; - /* Show the new entry */ - l->history_index += (dir == LINENOISE_HISTORY_PREV) ? 1 : -1; - if (l->history_index < 0) { - l->history_index = 0; - return; - } else if (l->history_index >= (int)history.size()) { - l->history_index = static_cast(history.size())-1; - return; - } - memset(l->buf, 0, l->buflen); - strcpy(l->buf,history[history.size() - 1 - l->history_index].c_str()); - l->len = l->pos = static_cast(strlen(l->buf)); - refreshLine(l); - } -} - -/* Delete the character at the right of the cursor without altering the cursor - * position. Basically this is what happens with the "Delete" keyboard key. */ -inline void linenoiseEditDelete(struct linenoiseState *l) { - if (l->len > 0 && l->pos < l->len) { - int glen = unicodeGraphemeLen(l->buf,l->len,l->pos); - memmove(l->buf+l->pos,l->buf+l->pos+glen,l->len-l->pos-glen); - l->len-=glen; - l->buf[l->len] = '\0'; - refreshLine(l); - } -} - -/* Backspace implementation. */ -inline void linenoiseEditBackspace(struct linenoiseState *l) { - if (l->pos > 0 && l->len > 0) { - int glen = unicodePrevGraphemeLen(l->buf,l->pos); - memmove(l->buf+l->pos-glen,l->buf+l->pos,l->len-l->pos); - l->pos-=glen; - l->len-=glen; - l->buf[l->len] = '\0'; - refreshLine(l); - } -} - -/* Delete the previosu word, maintaining the cursor at the start of the - * current word. */ -inline void linenoiseEditDeletePrevWord(struct linenoiseState *l) { - int old_pos = l->pos; - int diff; - - while (l->pos > 0 && l->buf[l->pos-1] == ' ') - l->pos--; - while (l->pos > 0 && l->buf[l->pos-1] != ' ') - l->pos--; - diff = old_pos - l->pos; - memmove(l->buf+l->pos,l->buf+old_pos,l->len-old_pos+1); - l->len -= diff; - refreshLine(l); -} - -/* This function is the core of the line editing capability of linenoise. - * It expects 'fd' to be already in "raw mode" so that every key pressed - * will be returned ASAP to read(). - * - * The resulting string is put into 'buf' when the user type enter, or - * when ctrl+d is typed. - * - * The function returns the length of the current buffer. */ -inline int linenoiseEdit(int stdin_fd, int stdout_fd, char *buf, int buflen, const char *prompt) -{ - struct linenoiseState l; - - /* Populate the linenoise state that we pass to functions implementing - * specific editing functionalities. */ - l.ifd = stdin_fd; - l.ofd = stdout_fd; - l.buf = buf; - l.buflen = buflen; - l.prompt = prompt; - l.oldcolpos = l.pos = 0; - l.len = 0; - l.cols = getColumns(stdin_fd, stdout_fd); - l.maxrows = 0; - l.history_index = 0; - - /* Buffer starts empty. */ - l.buf[0] = '\0'; - l.buflen--; /* Make sure there is always space for the nulterm */ - - /* The latest history entry is always our current buffer, that - * initially is just an empty string. */ - AddHistory(""); - - if (write(l.ofd,prompt, static_cast(l.prompt.length())) == -1) return -1; - while(1) { - int c; - char cbuf[4]; - int nread; - char seq[3]; - -#ifdef _WIN32 - nread = win32read(&c); - if (nread == 1) { - cbuf[0] = c; - } -#else - nread = unicodeReadUTF8Char(l.ifd,cbuf,&c); -#endif - if (nread <= 0) return (int)l.len; - - /* Only autocomplete when the callback is set. It returns < 0 when - * there was an error reading from fd. Otherwise it will return the - * character that should be handled next. */ - if (c == 9 && completionCallback != NULL) { - nread = completeLine(&l,cbuf,&c); - /* Return on errors */ - if (c < 0) return l.len; - /* Read next character when 0 */ - if (c == 0) continue; - } - - switch(c) { - case ENTER: /* enter */ - if (!history.empty()) history.pop_back(); - if (mlmode) linenoiseEditMoveEnd(&l); - return (int)l.len; - case CTRL_C: /* ctrl-c */ - errno = EAGAIN; - return -1; - case BACKSPACE: /* backspace */ - case 8: /* ctrl-h */ - linenoiseEditBackspace(&l); - break; - case CTRL_D: /* ctrl-d, remove char at right of cursor, or if the - line is empty, act as end-of-file. */ - if (l.len > 0) { - linenoiseEditDelete(&l); - } else { - history.pop_back(); - return -1; - } - break; - case CTRL_T: /* ctrl-t, swaps current character with previous. */ - if (l.pos > 0 && l.pos < l.len) { - char aux = buf[l.pos-1]; - buf[l.pos-1] = buf[l.pos]; - buf[l.pos] = aux; - if (l.pos != l.len-1) l.pos++; - refreshLine(&l); - } - break; - case CTRL_B: /* ctrl-b */ - linenoiseEditMoveLeft(&l); - break; - case CTRL_F: /* ctrl-f */ - linenoiseEditMoveRight(&l); - break; - case CTRL_P: /* ctrl-p */ - linenoiseEditHistoryNext(&l, LINENOISE_HISTORY_PREV); - break; - case CTRL_N: /* ctrl-n */ - linenoiseEditHistoryNext(&l, LINENOISE_HISTORY_NEXT); - break; - case ESC: /* escape sequence */ - /* Read the next two bytes representing the escape sequence. - * Use two calls to handle slow terminals returning the two - * chars at different times. */ - if (read(l.ifd,seq,1) == -1) break; - if (read(l.ifd,seq+1,1) == -1) break; - - /* ESC [ sequences. */ - if (seq[0] == '[') { - if (seq[1] >= '0' && seq[1] <= '9') { - /* Extended escape, read additional byte. */ - if (read(l.ifd,seq+2,1) == -1) break; - if (seq[2] == '~') { - switch(seq[1]) { - case '3': /* Delete key. */ - linenoiseEditDelete(&l); - break; - } - } - } else { - switch(seq[1]) { - case 'A': /* Up */ - linenoiseEditHistoryNext(&l, LINENOISE_HISTORY_PREV); - break; - case 'B': /* Down */ - linenoiseEditHistoryNext(&l, LINENOISE_HISTORY_NEXT); - break; - case 'C': /* Right */ - linenoiseEditMoveRight(&l); - break; - case 'D': /* Left */ - linenoiseEditMoveLeft(&l); - break; - case 'H': /* Home */ - linenoiseEditMoveHome(&l); - break; - case 'F': /* End*/ - linenoiseEditMoveEnd(&l); - break; - } - } - } - - /* ESC O sequences. */ - else if (seq[0] == 'O') { - switch(seq[1]) { - case 'H': /* Home */ - linenoiseEditMoveHome(&l); - break; - case 'F': /* End*/ - linenoiseEditMoveEnd(&l); - break; - } - } - break; - default: - if (linenoiseEditInsert(&l,cbuf,nread)) return -1; - break; - case CTRL_U: /* Ctrl+u, delete the whole line. */ - buf[0] = '\0'; - l.pos = l.len = 0; - refreshLine(&l); - break; - case CTRL_K: /* Ctrl+k, delete from current to end of line. */ - buf[l.pos] = '\0'; - l.len = l.pos; - refreshLine(&l); - break; - case CTRL_A: /* Ctrl+a, go to the start of the line */ - linenoiseEditMoveHome(&l); - break; - case CTRL_E: /* ctrl+e, go to the end of the line */ - linenoiseEditMoveEnd(&l); - break; - case CTRL_L: /* ctrl+l, clear screen */ - linenoiseClearScreen(); - refreshLine(&l); - break; - case CTRL_W: /* ctrl+w, delete previous word */ - linenoiseEditDeletePrevWord(&l); - break; - } - } - return l.len; -} - -/* This function calls the line editing function linenoiseEdit() using - * the STDIN file descriptor set in raw mode. */ -inline bool linenoiseRaw(const char *prompt, std::string& line) { - bool quit = false; - - if (!isatty(STDIN_FILENO)) { - /* Not a tty: read from file / pipe. */ - std::getline(std::cin, line); - } else { - /* Interactive editing. */ - if (enableRawMode(STDIN_FILENO) == false) { - return quit; - } - - char buf[LINENOISE_MAX_LINE]; - auto count = linenoiseEdit(STDIN_FILENO, STDOUT_FILENO, buf, LINENOISE_MAX_LINE, prompt); - if (count == -1) { - quit = true; - } else { - line.assign(buf, count); - } - - disableRawMode(STDIN_FILENO); - printf("\n"); - } - return quit; -} - -/* The high level function that is the main API of the linenoise library. - * This function checks if the terminal has basic capabilities, just checking - * for a blacklist of stupid terminals, and later either calls the line - * editing function or uses dummy fgets() so that you will be able to type - * something even in the most desperate of the conditions. */ -inline bool Readline(const char *prompt, std::string& line) { - if (isUnsupportedTerm()) { - printf("%s",prompt); - fflush(stdout); - std::getline(std::cin, line); - return false; - } else { - return linenoiseRaw(prompt, line); - } -} - -inline std::string Readline(const char *prompt, bool& quit) { - std::string line; - quit = Readline(prompt, line); - return line; -} - -inline std::string Readline(const char *prompt) { - bool quit; // dummy - return Readline(prompt, quit); -} - -/* ================================ History ================================= */ - -/* At exit we'll try to fix the terminal to the initial conditions. */ -inline void linenoiseAtExit(void) { - disableRawMode(STDIN_FILENO); -} - -/* This is the API call to add a new entry in the linenoise history. - * It uses a fixed array of char pointers that are shifted (memmoved) - * when the history max length is reached in order to remove the older - * entry and make room for the new one, so it is not exactly suitable for huge - * histories, but will work well for a few hundred of entries. - * - * Using a circular buffer is smarter, but a bit more complex to handle. */ -inline bool AddHistory(const char* line) { - if (history_max_len == 0) return false; - - /* Don't add duplicated lines. */ - if (!history.empty() && history.back() == line) return false; - - /* If we reached the max length, remove the older line. */ - if (history.size() == history_max_len) { - history.erase(history.begin()); - } - history.push_back(line); - - return true; -} - -/* Set the maximum length for the history. This function can be called even - * if there is already some history, the function will make sure to retain - * just the latest 'len' elements if the new history length value is smaller - * than the amount of items already inside the history. */ -inline bool SetHistoryMaxLen(size_t len) { - if (len < 1) return false; - history_max_len = len; - if (len < history.size()) { - history.resize(len); - } - return true; -} - -/* Save the history in the specified file. On success *true* is returned - * otherwise *false* is returned. */ -inline bool SaveHistory(const char* path) { - std::ofstream f(path); // TODO: need 'std::ios::binary'? - if (!f) return false; - for (const auto& h: history) { - f << h << std::endl; - } - return true; -} - -/* Load the history from the specified file. If the file does not exist - * zero is returned and no operation is performed. - * - * If the file exists and the operation succeeded *true* is returned, otherwise - * on error *false* is returned. */ -inline bool LoadHistory(const char* path) { - std::ifstream f(path); - if (!f) return false; - std::string line; - while (std::getline(f, line)) { - AddHistory(line.c_str()); - } - return true; -} - -inline const std::vector& GetHistory() { - return history; -} - -} // namespace linenoise - -#ifdef _WIN32 -#undef isatty -#undef write -#undef read -#pragma warning(pop) -#endif - -#endif /* __LINENOISE_HPP */ diff --git a/tests/Autocomplete.test.cpp b/tests/Autocomplete.test.cpp index e8e3b315..59e12574 100644 --- a/tests/Autocomplete.test.cpp +++ b/tests/Autocomplete.test.cpp @@ -2536,7 +2536,7 @@ TEST_CASE("autocomplete_documentation_symbols") TEST_CASE_FIXTURE(ACFixture, "autocomplete_ifelse_expressions") { - check(R"( + check(R"( local temp = false local even = true; local a = true @@ -2551,63 +2551,63 @@ a = if temp then even elseif true then temp e@8 a = if temp then even elseif true then temp else e@9 )"); - auto ac = autocomplete('1'); - CHECK(ac.entryMap.count("temp")); - CHECK(ac.entryMap.count("true")); - CHECK(ac.entryMap.count("then") == 0); - CHECK(ac.entryMap.count("else") == 0); - CHECK(ac.entryMap.count("elseif") == 0); + auto ac = autocomplete('1'); + CHECK(ac.entryMap.count("temp")); + CHECK(ac.entryMap.count("true")); + CHECK(ac.entryMap.count("then") == 0); + CHECK(ac.entryMap.count("else") == 0); + CHECK(ac.entryMap.count("elseif") == 0); - ac = autocomplete('2'); - CHECK(ac.entryMap.count("temp") == 0); - CHECK(ac.entryMap.count("true") == 0); - CHECK(ac.entryMap.count("then")); - CHECK(ac.entryMap.count("else") == 0); - CHECK(ac.entryMap.count("elseif") == 0); + ac = autocomplete('2'); + CHECK(ac.entryMap.count("temp") == 0); + CHECK(ac.entryMap.count("true") == 0); + CHECK(ac.entryMap.count("then")); + CHECK(ac.entryMap.count("else") == 0); + CHECK(ac.entryMap.count("elseif") == 0); - ac = autocomplete('3'); - CHECK(ac.entryMap.count("even")); - CHECK(ac.entryMap.count("then") == 0); - CHECK(ac.entryMap.count("else") == 0); - CHECK(ac.entryMap.count("elseif") == 0); + ac = autocomplete('3'); + CHECK(ac.entryMap.count("even")); + CHECK(ac.entryMap.count("then") == 0); + CHECK(ac.entryMap.count("else") == 0); + CHECK(ac.entryMap.count("elseif") == 0); - ac = autocomplete('4'); - CHECK(ac.entryMap.count("even") == 0); - CHECK(ac.entryMap.count("then") == 0); - CHECK(ac.entryMap.count("else")); - CHECK(ac.entryMap.count("elseif")); + ac = autocomplete('4'); + CHECK(ac.entryMap.count("even") == 0); + CHECK(ac.entryMap.count("then") == 0); + CHECK(ac.entryMap.count("else")); + CHECK(ac.entryMap.count("elseif")); - ac = autocomplete('5'); - CHECK(ac.entryMap.count("temp")); - CHECK(ac.entryMap.count("true")); - CHECK(ac.entryMap.count("then") == 0); - CHECK(ac.entryMap.count("else") == 0); - CHECK(ac.entryMap.count("elseif") == 0); + ac = autocomplete('5'); + CHECK(ac.entryMap.count("temp")); + CHECK(ac.entryMap.count("true")); + CHECK(ac.entryMap.count("then") == 0); + CHECK(ac.entryMap.count("else") == 0); + CHECK(ac.entryMap.count("elseif") == 0); - ac = autocomplete('6'); - CHECK(ac.entryMap.count("temp") == 0); - CHECK(ac.entryMap.count("true") == 0); - CHECK(ac.entryMap.count("then")); - CHECK(ac.entryMap.count("else") == 0); - CHECK(ac.entryMap.count("elseif") == 0); + ac = autocomplete('6'); + CHECK(ac.entryMap.count("temp") == 0); + CHECK(ac.entryMap.count("true") == 0); + CHECK(ac.entryMap.count("then")); + CHECK(ac.entryMap.count("else") == 0); + CHECK(ac.entryMap.count("elseif") == 0); - ac = autocomplete('7'); - CHECK(ac.entryMap.count("temp")); - CHECK(ac.entryMap.count("true")); - CHECK(ac.entryMap.count("then") == 0); - CHECK(ac.entryMap.count("else") == 0); - CHECK(ac.entryMap.count("elseif") == 0); + ac = autocomplete('7'); + CHECK(ac.entryMap.count("temp")); + CHECK(ac.entryMap.count("true")); + CHECK(ac.entryMap.count("then") == 0); + CHECK(ac.entryMap.count("else") == 0); + CHECK(ac.entryMap.count("elseif") == 0); - ac = autocomplete('8'); - CHECK(ac.entryMap.count("even") == 0); - CHECK(ac.entryMap.count("then") == 0); - CHECK(ac.entryMap.count("else")); - CHECK(ac.entryMap.count("elseif")); + ac = autocomplete('8'); + CHECK(ac.entryMap.count("even") == 0); + CHECK(ac.entryMap.count("then") == 0); + CHECK(ac.entryMap.count("else")); + CHECK(ac.entryMap.count("elseif")); - ac = autocomplete('9'); - CHECK(ac.entryMap.count("then") == 0); - CHECK(ac.entryMap.count("else") == 0); - CHECK(ac.entryMap.count("elseif") == 0); + ac = autocomplete('9'); + CHECK(ac.entryMap.count("then") == 0); + CHECK(ac.entryMap.count("else") == 0); + CHECK(ac.entryMap.count("elseif") == 0); } TEST_CASE_FIXTURE(ACFixture, "autocomplete_explicit_type_pack") diff --git a/tests/Compiler.test.cpp b/tests/Compiler.test.cpp index 4a28bdde..d8af94db 100644 --- a/tests/Compiler.test.cpp +++ b/tests/Compiler.test.cpp @@ -611,7 +611,8 @@ TEST_CASE("TableLiteralsIndexConstant") CHECK_EQ("\n" + compileFunction0(R"( local a, b = "key", "value" return {[a] = 42, [b] = 0} -)"), R"( +)"), + R"( NEWTABLE R0 2 0 LOADN R1 42 SETTABLEKS R1 R0 K0 @@ -624,7 +625,8 @@ RETURN R0 1 CHECK_EQ("\n" + compileFunction0(R"( local a, b = 1, 2 return {[a] = 42, [b] = 0} -)"), R"( +)"), + R"( NEWTABLE R0 0 2 LOADN R1 42 SETTABLEN R1 R0 1 @@ -789,8 +791,6 @@ RETURN R0 1 TEST_CASE("TableSizePredictionLoop") { - ScopedFastFlag sff("LuauPredictTableSizeLoop", true); - CHECK_EQ("\n" + compileFunction0(R"( local t = {} for i=1,4 do @@ -2827,7 +2827,7 @@ RETURN R1 -1 TEST_CASE("FastcallSelect") { - ScopedFastFlag sff("LuauCompileSelectBuiltin", true); + ScopedFastFlag sff("LuauCompileSelectBuiltin2", true); // select(_, ...) compiles to a builtin call CHECK_EQ("\n" + compileFunction0("return (select('#', ...))"), R"( @@ -2846,7 +2846,8 @@ for i=1, select('#', ...) do sum += select(i, ...) end return sum -)"), R"( +)"), + R"( LOADN R0 0 LOADN R3 1 LOADK R5 K0 @@ -2856,13 +2857,14 @@ GETVARARGS R6 -1 CALL R4 -1 1 MOVE R1 R4 LOADN R2 1 -FORNPREP R1 +7 -FASTCALL1 57 R3 +3 +FORNPREP R1 +8 +FASTCALL1 57 R3 +4 GETIMPORT R4 2 +MOVE R5 R3 GETVARARGS R6 -1 CALL R4 -1 1 ADD R0 R0 R4 -FORNLOOP R1 -7 +FORNLOOP R1 -8 RETURN R0 1 )"); diff --git a/tests/Conformance.test.cpp b/tests/Conformance.test.cpp index 914b881f..e580949f 100644 --- a/tests/Conformance.test.cpp +++ b/tests/Conformance.test.cpp @@ -492,7 +492,6 @@ TEST_CASE("DateTime") TEST_CASE("Debug") { - ScopedFastFlag sffr("LuauBytecodeV2Read", true); ScopedFastFlag sffw("LuauBytecodeV2Write", true); runConformance("debug.lua"); diff --git a/tests/LValue.test.cpp b/tests/LValue.test.cpp index 8a092779..606f6de3 100644 --- a/tests/LValue.test.cpp +++ b/tests/LValue.test.cpp @@ -38,8 +38,6 @@ TEST_SUITE_BEGIN("LValue"); TEST_CASE("Luau_merge_hashmap_order") { - ScopedFastFlag sff{"LuauLValueAsKey", true}; - std::string a = "a"; std::string b = "b"; std::string c = "c"; @@ -58,20 +56,18 @@ TEST_CASE("Luau_merge_hashmap_order") TypeArena arena; merge(arena, m, other); - REQUIRE_EQ(3, m.NEW_refinements.size()); - REQUIRE(m.NEW_refinements.count(mkSymbol(a))); - REQUIRE(m.NEW_refinements.count(mkSymbol(b))); - REQUIRE(m.NEW_refinements.count(mkSymbol(c))); + REQUIRE_EQ(3, m.size()); + REQUIRE(m.count(mkSymbol(a))); + REQUIRE(m.count(mkSymbol(b))); + REQUIRE(m.count(mkSymbol(c))); - CHECK_EQ("string", toString(m.NEW_refinements[mkSymbol(a)])); - CHECK_EQ("string", toString(m.NEW_refinements[mkSymbol(b)])); - CHECK_EQ("boolean | number", toString(m.NEW_refinements[mkSymbol(c)])); + CHECK_EQ("string", toString(m[mkSymbol(a)])); + CHECK_EQ("string", toString(m[mkSymbol(b)])); + CHECK_EQ("boolean | number", toString(m[mkSymbol(c)])); } TEST_CASE("Luau_merge_hashmap_order2") { - ScopedFastFlag sff{"LuauLValueAsKey", true}; - std::string a = "a"; std::string b = "b"; std::string c = "c"; @@ -90,20 +86,18 @@ TEST_CASE("Luau_merge_hashmap_order2") TypeArena arena; merge(arena, m, other); - REQUIRE_EQ(3, m.NEW_refinements.size()); - REQUIRE(m.NEW_refinements.count(mkSymbol(a))); - REQUIRE(m.NEW_refinements.count(mkSymbol(b))); - REQUIRE(m.NEW_refinements.count(mkSymbol(c))); + REQUIRE_EQ(3, m.size()); + REQUIRE(m.count(mkSymbol(a))); + REQUIRE(m.count(mkSymbol(b))); + REQUIRE(m.count(mkSymbol(c))); - CHECK_EQ("string", toString(m.NEW_refinements[mkSymbol(a)])); - CHECK_EQ("string", toString(m.NEW_refinements[mkSymbol(b)])); - CHECK_EQ("boolean | number", toString(m.NEW_refinements[mkSymbol(c)])); + CHECK_EQ("string", toString(m[mkSymbol(a)])); + CHECK_EQ("string", toString(m[mkSymbol(b)])); + CHECK_EQ("boolean | number", toString(m[mkSymbol(c)])); } TEST_CASE("one_map_has_overlap_at_end_whereas_other_has_it_in_start") { - ScopedFastFlag sff{"LuauLValueAsKey", true}; - std::string a = "a"; std::string b = "b"; std::string c = "c"; @@ -125,18 +119,18 @@ TEST_CASE("one_map_has_overlap_at_end_whereas_other_has_it_in_start") TypeArena arena; merge(arena, m, other); - REQUIRE_EQ(5, m.NEW_refinements.size()); - REQUIRE(m.NEW_refinements.count(mkSymbol(a))); - REQUIRE(m.NEW_refinements.count(mkSymbol(b))); - REQUIRE(m.NEW_refinements.count(mkSymbol(c))); - REQUIRE(m.NEW_refinements.count(mkSymbol(d))); - REQUIRE(m.NEW_refinements.count(mkSymbol(e))); + REQUIRE_EQ(5, m.size()); + REQUIRE(m.count(mkSymbol(a))); + REQUIRE(m.count(mkSymbol(b))); + REQUIRE(m.count(mkSymbol(c))); + REQUIRE(m.count(mkSymbol(d))); + REQUIRE(m.count(mkSymbol(e))); - CHECK_EQ("string", toString(m.NEW_refinements[mkSymbol(a)])); - CHECK_EQ("number", toString(m.NEW_refinements[mkSymbol(b)])); - CHECK_EQ("boolean | string", toString(m.NEW_refinements[mkSymbol(c)])); - CHECK_EQ("number", toString(m.NEW_refinements[mkSymbol(d)])); - CHECK_EQ("boolean", toString(m.NEW_refinements[mkSymbol(e)])); + CHECK_EQ("string", toString(m[mkSymbol(a)])); + CHECK_EQ("number", toString(m[mkSymbol(b)])); + CHECK_EQ("boolean | string", toString(m[mkSymbol(c)])); + CHECK_EQ("number", toString(m[mkSymbol(d)])); + CHECK_EQ("boolean", toString(m[mkSymbol(e)])); } TEST_CASE("hashing_lvalue_global_prop_access") @@ -159,7 +153,7 @@ TEST_CASE("hashing_lvalue_global_prop_access") CHECK_EQ(LValueHasher{}(t_x1), LValueHasher{}(t_x2)); CHECK_EQ(LValueHasher{}(t_x2), LValueHasher{}(t_x2)); - NEW_RefinementMap m; + RefinementMap m; m[t_x1] = getSingletonTypes().stringType; m[t_x2] = getSingletonTypes().numberType; @@ -188,7 +182,7 @@ TEST_CASE("hashing_lvalue_local_prop_access") CHECK_NE(LValueHasher{}(t_x1), LValueHasher{}(t_x2)); CHECK_EQ(LValueHasher{}(t_x2), LValueHasher{}(t_x2)); - NEW_RefinementMap m; + RefinementMap m; m[t_x1] = getSingletonTypes().stringType; m[t_x2] = getSingletonTypes().numberType; diff --git a/tests/Linter.test.cpp b/tests/Linter.test.cpp index 5ad06f0d..d1cc49b2 100644 --- a/tests/Linter.test.cpp +++ b/tests/Linter.test.cpp @@ -54,6 +54,17 @@ return _ CHECK_EQ(result.warnings[0].text, "Placeholder value '_' is read here; consider using a named variable"); } +TEST_CASE_FIXTURE(Fixture, "PlaceholderReadGlobal") +{ + LintResult result = lint(R"( +_ = 5 +print(_) +)"); + + CHECK_EQ(result.warnings.size(), 1); + CHECK_EQ(result.warnings[0].text, "Placeholder value '_' is read here; consider using a named variable"); +} + TEST_CASE_FIXTURE(Fixture, "PlaceholderWrite") { LintResult result = lint(R"( @@ -853,7 +864,7 @@ string.format("%Y") local _ = ("%"):format() -- correct format strings, just to uh make sure -string.format("hello %d %f", 4, 5) +string.format("hello %+10d %.02f %%", 4, 5) )"); CHECK_EQ(result.warnings.size(), 4); @@ -1078,16 +1089,18 @@ TEST_CASE_FIXTURE(Fixture, "FormatStringDate") os.date("%") os.date("%L") os.date("%?") +os.date("\0") -- correct formats os.date("it's %c now") os.date("!*t") )"); - CHECK_EQ(result.warnings.size(), 3); + CHECK_EQ(result.warnings.size(), 4); CHECK_EQ(result.warnings[0].text, "Invalid date format: unfinished replacement"); CHECK_EQ(result.warnings[1].text, "Invalid date format: unexpected replacement character; must be a date format specifier or %"); CHECK_EQ(result.warnings[2].text, "Invalid date format: unexpected replacement character; must be a date format specifier or %"); + CHECK_EQ(result.warnings[3].text, "Invalid date format: date format can not contain null characters"); } TEST_CASE_FIXTURE(Fixture, "FormatStringTyped") @@ -1396,8 +1409,6 @@ end TEST_CASE_FIXTURE(Fixture, "TableOperations") { - ScopedFastFlag sff("LuauLintTableCreateTable", true); - LintResult result = lintTyped(R"( local t = {} local tt = {} @@ -1435,8 +1446,10 @@ table.create(42, {} :: {}) "table.insert may change behavior if the call returns more than one result; consider adding parentheses around second argument"); CHECK_EQ(result.warnings[6].text, "table.move uses index 0 but arrays are 1-based; did you mean 1 instead?"); CHECK_EQ(result.warnings[7].text, "table.move uses index 0 but arrays are 1-based; did you mean 1 instead?"); - CHECK_EQ(result.warnings[8].text, "table.create with a table literal will reuse the same object for all elements; consider using a for loop instead"); - CHECK_EQ(result.warnings[9].text, "table.create with a table literal will reuse the same object for all elements; consider using a for loop instead"); + CHECK_EQ( + result.warnings[8].text, "table.create with a table literal will reuse the same object for all elements; consider using a for loop instead"); + CHECK_EQ( + result.warnings[9].text, "table.create with a table literal will reuse the same object for all elements; consider using a for loop instead"); } TEST_CASE_FIXTURE(Fixture, "DuplicateConditions") diff --git a/tests/Parser.test.cpp b/tests/Parser.test.cpp index 90831ee9..c1a8887b 100644 --- a/tests/Parser.test.cpp +++ b/tests/Parser.test.cpp @@ -7,8 +7,6 @@ #include "doctest.h" -LUAU_FASTFLAG(LuauFixAmbiguousErrorRecoveryInAssign) - using namespace Luau; namespace @@ -1639,10 +1637,7 @@ TEST_CASE_FIXTURE(Fixture, "parse_error_confusing_function_call") "Ambiguous syntax: this looks like an argument list for a function call, but could also be a start of new statement; use ';' to separate " "statements"); - if (FFlag::LuauFixAmbiguousErrorRecoveryInAssign) - CHECK(result4.errors.size() == 1); - else - CHECK(result4.errors.size() == 5); + CHECK(result4.errors.size() == 1); } TEST_CASE_FIXTURE(Fixture, "parse_error_varargs") diff --git a/tests/TypeInfer.annotations.test.cpp b/tests/TypeInfer.annotations.test.cpp index 86165814..572b882d 100644 --- a/tests/TypeInfer.annotations.test.cpp +++ b/tests/TypeInfer.annotations.test.cpp @@ -209,8 +209,6 @@ TEST_CASE_FIXTURE(Fixture, "as_expr_does_not_propagate_type_info") TEST_CASE_FIXTURE(Fixture, "as_expr_is_bidirectional") { - ScopedFastFlag sff{"LuauBidirectionalAsExpr", true}; - CheckResult result = check(R"( local a = 55 :: number? local b = a :: number @@ -224,7 +222,6 @@ TEST_CASE_FIXTURE(Fixture, "as_expr_is_bidirectional") TEST_CASE_FIXTURE(Fixture, "as_expr_warns_on_unrelated_cast") { - ScopedFastFlag sff{"LuauBidirectionalAsExpr", true}; ScopedFastFlag sff2{"LuauErrorRecoveryType", true}; CheckResult result = check(R"( diff --git a/tests/TypeInfer.builtins.test.cpp b/tests/TypeInfer.builtins.test.cpp index 5e08654a..6730bedb 100644 --- a/tests/TypeInfer.builtins.test.cpp +++ b/tests/TypeInfer.builtins.test.cpp @@ -889,4 +889,55 @@ TEST_CASE_FIXTURE(Fixture, "dont_add_definitions_to_persistent_types") REQUIRE(gtv->definition); } +TEST_CASE_FIXTURE(Fixture, "assert_removes_falsy_types") +{ + ScopedFastFlag sff[]{ + {"LuauAssertStripsFalsyTypes", true}, + {"LuauDiscriminableUnions", true}, + }; + + CheckResult result = check(R"( + local function f(x: (number | boolean)?) + return assert(x) + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + CHECK_EQ("((boolean | number)?) -> number | true", toString(requireType("f"))); +} + +TEST_CASE_FIXTURE(Fixture, "assert_removes_falsy_types_even_from_type_pack_tail_but_only_for_the_first_type") +{ + ScopedFastFlag sff[]{ + {"LuauAssertStripsFalsyTypes", true}, + {"LuauDiscriminableUnions", true}, + }; + + CheckResult result = check(R"( + local function f(...: number?) + return assert(...) + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + CHECK_EQ("(...number?) -> (number, ...number?)", toString(requireType("f"))); +} + +TEST_CASE_FIXTURE(Fixture, "assert_returns_false_and_string_iff_it_knows_the_first_argument_cannot_be_truthy") +{ + ScopedFastFlag sff[]{ + {"LuauAssertStripsFalsyTypes", true}, + {"LuauDiscriminableUnions", true}, + }; + + CheckResult result = check(R"( + local function f(x: nil) + return assert(x, "hmm") + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + CHECK_EQ("(nil) -> nil", toString(requireType("f"))); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.provisional.test.cpp b/tests/TypeInfer.provisional.test.cpp index 47c13be9..e5eb0dca 100644 --- a/tests/TypeInfer.provisional.test.cpp +++ b/tests/TypeInfer.provisional.test.cpp @@ -176,19 +176,6 @@ TEST_CASE_FIXTURE(Fixture, "pass_a_union_of_tables_to_a_function_that_requires_a REQUIRE_EQ("{| [any]: any, x: number, y: number |}", toString(requireType("b"))); } -TEST_CASE_FIXTURE(Fixture, "normal_conditional_expression_has_refinements") -{ - CheckResult result = check(R"( - local foo: {x: number}? = nil - local bar = foo and foo.x -- TODO: Geez. We are inferring the wrong types here. Should be 'number?'. - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - - // Binary and/or return types are straight up wrong. JIRA: CLI-40300 - CHECK_EQ("boolean | number", toString(requireType("bar"))); -} - // Luau currently doesn't yet know how to allow assignments when the binding was refined. TEST_CASE_FIXTURE(Fixture, "while_body_are_also_refined") { diff --git a/tests/TypeInfer.refinements.test.cpp b/tests/TypeInfer.refinements.test.cpp index f346ddfd..3a610c3a 100644 --- a/tests/TypeInfer.refinements.test.cpp +++ b/tests/TypeInfer.refinements.test.cpp @@ -939,8 +939,6 @@ TEST_CASE_FIXTURE(Fixture, "type_comparison_ifelse_expression") TEST_CASE_FIXTURE(Fixture, "correctly_lookup_a_shadowed_local_that_which_was_previously_refined") { - ScopedFastFlag sff{"LuauLValueAsKey", true}; - CheckResult result = check(R"( local foo: string? = "hi" assert(foo) @@ -955,8 +953,6 @@ TEST_CASE_FIXTURE(Fixture, "correctly_lookup_a_shadowed_local_that_which_was_pre TEST_CASE_FIXTURE(Fixture, "correctly_lookup_property_whose_base_was_previously_refined") { - ScopedFastFlag sff{"LuauLValueAsKey", true}; - CheckResult result = check(R"( type T = {x: string | number} local t: T? = {x = "hi"} @@ -974,8 +970,6 @@ TEST_CASE_FIXTURE(Fixture, "correctly_lookup_property_whose_base_was_previously_ TEST_CASE_FIXTURE(Fixture, "correctly_lookup_property_whose_base_was_previously_refined2") { - ScopedFastFlag sff{"LuauLValueAsKey", true}; - CheckResult result = check(R"( type T = { x: { y: number }? } @@ -993,8 +987,6 @@ TEST_CASE_FIXTURE(Fixture, "correctly_lookup_property_whose_base_was_previously_ TEST_CASE_FIXTURE(Fixture, "apply_refinements_on_astexprindexexpr_whose_subscript_expr_is_constant_string") { - ScopedFastFlag sff{"LuauRefiLookupFromIndexExpr", true}; - CheckResult result = check(R"( type T = { [string]: { prop: number }? } local t: T = {} @@ -1061,27 +1053,62 @@ TEST_CASE_FIXTURE(Fixture, "discriminate_tag") CHECK_EQ("Dog", toString(requireTypeAtPosition({9, 33}))); } -TEST_CASE_FIXTURE(Fixture, "apply_refinements_on_astexprindexexpr_whose_subscript_expr_is_constant_string") +TEST_CASE_FIXTURE(Fixture, "and_or_peephole_refinement") { - ScopedFastFlag sff{"LuauRefiLookupFromIndexExpr", true}; - CheckResult result = check(R"( - type T = { [string]: { prop: number }? } - local t: T = {} - - if t["hello"] then - local foo = t["hello"].prop + local function len(a: {any}) + return a and #a or nil end )"); LUAU_REQUIRE_NO_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "and_or_peephole_refinement") +TEST_CASE_FIXTURE(Fixture, "narrow_boolean_to_true_or_false") { + ScopedFastFlag sff[]{ + {"LuauParseSingletonTypes", true}, + {"LuauSingletonTypes", true}, + {"LuauDiscriminableUnions", true}, + {"LuauAssertStripsFalsyTypes", true}, + }; + CheckResult result = check(R"( - local function len(a: {any}) - return a and #a or nil + local function is_true(b: true) end + local function is_false(b: false) end + + local function f(x: boolean) + if x then + is_true(x) + else + is_false(x) + end + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "discriminate_on_properties_of_disjoint_tables_where_that_property_is_true_or_false") +{ + ScopedFastFlag sff[]{ + {"LuauParseSingletonTypes", true}, + {"LuauSingletonTypes", true}, + {"LuauDiscriminableUnions", true}, + {"LuauAssertStripsFalsyTypes", true}, + }; + + CheckResult result = check(R"( + type Ok = { ok: true, value: T } + type Err = { ok: false, error: E } + type Result = Ok | Err + + local function apply(t: Result, f: (T) -> (), g: (E) -> ()) + if t.ok then + f(t.value) + else + g(t.error) + end end )"); diff --git a/tests/TypeInfer.tables.test.cpp b/tests/TypeInfer.tables.test.cpp index 48310921..f19cb618 100644 --- a/tests/TypeInfer.tables.test.cpp +++ b/tests/TypeInfer.tables.test.cpp @@ -1482,7 +1482,7 @@ TEST_CASE_FIXTURE(Fixture, "casting_unsealed_tables_with_props_into_table_with_i REQUIRE(tm); CHECK_EQ("{| [string]: string |}", toString(tm->wantedType, o)); // Should t now have an indexer? - // It would if the assignment to rt was correctly typed. + // It would if the assignment to rt was correctly typed. CHECK_EQ("{ [string]: string, foo: number }", toString(tm->givenType, o)); } @@ -2082,7 +2082,7 @@ caused by: TEST_CASE_FIXTURE(Fixture, "explicitly_typed_table") { - ScopedFastFlag sffs[] { + ScopedFastFlag sffs[]{ {"LuauPropertiesGetExpectedType", true}, {"LuauExpectedTypesOfProperties", true}, {"LuauTableSubtypingVariance2", true}, @@ -2103,7 +2103,7 @@ a.p = { x = 9 } TEST_CASE_FIXTURE(Fixture, "explicitly_typed_table_error") { - ScopedFastFlag sffs[] { + ScopedFastFlag sffs[]{ {"LuauPropertiesGetExpectedType", true}, {"LuauExpectedTypesOfProperties", true}, {"LuauTableSubtypingVariance2", true}, @@ -2131,7 +2131,7 @@ caused by: TEST_CASE_FIXTURE(Fixture, "explicitly_typed_table_with_indexer") { - ScopedFastFlag sffs[] { + ScopedFastFlag sffs[]{ {"LuauPropertiesGetExpectedType", true}, {"LuauExpectedTypesOfProperties", true}, {"LuauTableSubtypingVariance2", true}, diff --git a/tests/TypeInfer.test.cpp b/tests/TypeInfer.test.cpp index c9b30e1a..ead3d762 100644 --- a/tests/TypeInfer.test.cpp +++ b/tests/TypeInfer.test.cpp @@ -2429,21 +2429,6 @@ TEST_CASE_FIXTURE(Fixture, "should_be_able_to_infer_this_without_stack_overflowi LUAU_REQUIRE_NO_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "x_or_y_forces_both_x_and_y_to_be_of_same_type_if_either_is_free") -{ - CheckResult result = check(R"( - local function f(x, y) return x or y end - - local x = f(1, 2) - local y = f(3, "foo") - )"); - - LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK_EQ(*requireType("x"), *typeChecker.numberType); - - CHECK_EQ(result.errors[0], (TypeError{Location{{4, 23}, {4, 28}}, TypeMismatch{typeChecker.numberType, typeChecker.stringType}})); -} - TEST_CASE_FIXTURE(Fixture, "inferring_hundreds_of_self_calls_should_not_suffocate_memory") { CheckResult result = check(R"( @@ -4509,7 +4494,7 @@ f(function(x) print(x) end) } TEST_CASE_FIXTURE(Fixture, "infer_generic_function_function_argument") -{ +{ ScopedFastFlag sff{"LuauUnsealedTableLiteral", true}; CheckResult result = check(R"( @@ -4777,7 +4762,7 @@ local a: X = if true then {"1", 2, 3} else {4, 5, 6} TEST_CASE_FIXTURE(Fixture, "tc_if_else_expressions_expected_type_2") { ScopedFastFlag luauIfElseExpectedType2{"LuauIfElseExpectedType2", true}; - ScopedFastFlag luauIfElseBranchTypeUnion{ "LuauIfElseBranchTypeUnion", true }; + ScopedFastFlag luauIfElseBranchTypeUnion{"LuauIfElseBranchTypeUnion", true}; CheckResult result = check(R"( local a: number? = if true then 1 else nil @@ -5012,16 +4997,14 @@ local b: B = a )"); LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK_EQ( - toString(result.errors[0]), R"(Type '(number, number) -> (number, string)' could not be converted into '(number, number) -> (number, boolean)' + CHECK_EQ(toString(result.errors[0]), + R"(Type '(number, number) -> (number, string)' could not be converted into '(number, number) -> (number, boolean)' caused by: Return #2 type is not compatible. Type 'string' could not be converted into 'boolean')"); } TEST_CASE_FIXTURE(Fixture, "prop_access_on_any_with_other_options") { - ScopedFastFlag sff{"LuauLValueAsKey", true}; - CheckResult result = check(R"( local function f(thing: any | string) local foo = thing.SomeRandomKey @@ -5120,4 +5103,65 @@ end )"); } +TEST_CASE_FIXTURE(Fixture, "cli_50041_committing_txnlog_in_apollo_client_error") +{ + ScopedFastFlag committingTxnLog{"LuauUseCommittingTxnLog", true}; + ScopedFastFlag subtypingVariance{"LuauTableSubtypingVariance2", true}; + + CheckResult result = check(R"( + --!strict + --!nolint + + type FieldSpecifier = { + fieldName: string, + } + + type ReadFieldOptions = FieldSpecifier & { from: number? } + + type Policies = { + getStoreFieldName: (self: Policies, fieldSpec: FieldSpecifier) -> string, + } + + local Policies = {} + + local function foo(p: Policies) + end + + function Policies:getStoreFieldName(specifier: FieldSpecifier): string + return "" + end + + function Policies:readField(options: ReadFieldOptions) + local _ = self:getStoreFieldName(options) + --[[ + Type error: + TypeError { "MainModule", Location { { line = 25, col = 16 }, { line = 25, col = 20 } }, TypeMismatch { Policies, {- getStoreFieldName: (tp1) -> (a, b...) -} } } + ]] + foo(self) + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "do_not_modify_imported_types") +{ + ScopedFastFlag noSealedTypeMod{"LuauNoSealedTypeMod", true}; + + fileResolver.source["game/A"] = R"( +export type Type = { unrelated: boolean } +return {} + )"; + + fileResolver.source["game/B"] = R"( +local types = require(game.A) +type Type = types.Type +local x: Type = {} +function x:Destroy(): () end + )"; + + CheckResult result = frontend.check("game/B"); + LUAU_REQUIRE_ERRORS(result); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.tryUnify.test.cpp b/tests/TypeInfer.tryUnify.test.cpp index 1e790eba..0aeca096 100644 --- a/tests/TypeInfer.tryUnify.test.cpp +++ b/tests/TypeInfer.tryUnify.test.cpp @@ -260,4 +260,17 @@ TEST_CASE_FIXTURE(TryUnifyFixture, "free_tail_is_grown_properly") CHECK(unifyErrors.size() == 0); } +TEST_CASE_FIXTURE(TryUnifyFixture, "recursive_metatable_getmatchtag") +{ + ScopedFastFlag luauUnionTagMatchFix{"LuauUnionTagMatchFix", true}; + + TypeVar redirect{FreeTypeVar{TypeLevel{}}}; + TypeVar table{TableTypeVar{}}; + TypeVar metatable{MetatableTypeVar{&redirect, &table}}; + redirect = BoundTypeVar{&metatable}; // Now we have a metatable that is recursive on the table type + TypeVar variant{UnionTypeVar{{&metatable, typeChecker.numberType}}}; + + state.tryUnify(&metatable, &variant); +} + TEST_SUITE_END(); diff --git a/tests/conformance/basic.lua b/tests/conformance/basic.lua index 188b8ebc..de091632 100644 --- a/tests/conformance/basic.lua +++ b/tests/conformance/basic.lua @@ -118,6 +118,10 @@ assert((function() return #_G end)() == 0) assert((function() return #{1,2} end)() == 2) assert((function() return #'g' end)() == 1) +local ud = newproxy(true) +getmetatable(ud).__len = function() return 42 end +assert((function() return #ud end)() == 42) + assert((function() local a = 1 a = -a return a end)() == -1) -- while/repeat diff --git a/tests/conformance/vararg.lua b/tests/conformance/vararg.lua index 5aaa422b..d05f9577 100644 --- a/tests/conformance/vararg.lua +++ b/tests/conformance/vararg.lua @@ -105,6 +105,45 @@ assert(a==5 and b==4 and c==3 and d==2 and e==1) a,b,c,d,e = f(4) assert(a==nil and b==nil and c==nil and d==nil and e==nil) +-- select tests +a = {select(3, unpack{10,20,30,40})} +assert(table.getn(a) == 2 and a[1] == 30 and a[2] == 40) +a = {select(1)} +assert(next(a) == nil) +a = {select(-1, 3, 5, 7)} +assert(a[1] == 7 and a[2] == nil) +a = {select(-2, 3, 5, 7)} +assert(a[1] == 5 and a[2] == 7 and a[3] == nil) +pcall(select, 10000) +pcall(select, -10000) + +-- select(_, ...) has special optimizations so it needs extra testing +function selectone(n, ...) + local e = select(n, ...) + return e +end + +function selectmany(n, ...) + return table.concat({select(n, ...)}, ',') +end + +assert(selectone('#') == 0) +assert(selectmany('#') == "0") + +assert(selectone('#', 10, 20, 30) == 3) +assert(selectmany('#', 10, 20, 30) == "3") + +assert(selectone(1, 10, 20, 30) == 10) +assert(selectmany(1, 10, 20, 30) == "10,20,30") + +assert(selectone(2, 10, 20, 30) == 20) +assert(selectmany(2, 10, 20, 30) == "20,30") + +assert(selectone(-2, 10, 20, 30) == 20) +assert(selectmany(-2, 10, 20, 30) == "20,30") + +assert(selectone('3', 10, 20, 30) == 30) +assert(selectmany('3', 10, 20, 30) == "30") -- varargs for main chunks f = loadstring[[ return {...} ]] @@ -122,16 +161,5 @@ f = loadstring[[ assert(f("a", "b", nil, {}, assert)) assert(f()) -a = {select(3, unpack{10,20,30,40})} -assert(table.getn(a) == 2 and a[1] == 30 and a[2] == 40) -a = {select(1)} -assert(next(a) == nil) -a = {select(-1, 3, 5, 7)} -assert(a[1] == 7 and a[2] == nil) -a = {select(-2, 3, 5, 7)} -assert(a[1] == 5 and a[2] == 7 and a[3] == nil) -pcall(select, 10000) -pcall(select, -10000) - return('OK') From bbae46600635e43d875cc95392a1e8481f445524 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Fri, 4 Feb 2022 12:31:19 -0800 Subject: [PATCH 147/399] Sync to upstream/release/513 This takes the extra bug fix for generic name confusion --- Analysis/include/Luau/TypeInfer.h | 3 ++- Analysis/src/TypeInfer.cpp | 7 ++++--- tests/TypeInfer.generics.test.cpp | 25 +++++++++++++++++++++++++ 3 files changed, 31 insertions(+), 4 deletions(-) diff --git a/Analysis/include/Luau/TypeInfer.h b/Analysis/include/Luau/TypeInfer.h index 90dc9f42..f61ecbf5 100644 --- a/Analysis/include/Luau/TypeInfer.h +++ b/Analysis/include/Luau/TypeInfer.h @@ -346,7 +346,8 @@ private: // Note: `scope` must be a fresh scope. GenericTypeDefinitions createGenericTypes(const ScopePtr& scope, std::optional levelOpt, const AstNode& node, - const AstArray& genericNames, const AstArray& genericPackNames); + const AstArray& genericNames, const AstArray& genericPackNames, + bool useCache = false); public: ErrorVec resolve(const PredicateVec& predicates, const ScopePtr& scope, bool sense); diff --git a/Analysis/src/TypeInfer.cpp b/Analysis/src/TypeInfer.cpp index 4d25fe2e..e1987937 100644 --- a/Analysis/src/TypeInfer.cpp +++ b/Analysis/src/TypeInfer.cpp @@ -29,6 +29,7 @@ LUAU_FASTFLAGVARIABLE(LuauWeakEqConstraint, false) // Eventually removed as fals LUAU_FASTFLAG(LuauUseCommittingTxnLog) LUAU_FASTFLAGVARIABLE(DebugLuauFreezeDuringUnification, false) LUAU_FASTFLAGVARIABLE(LuauRecursiveTypeParameterRestriction, false) +LUAU_FASTFLAGVARIABLE(LuauGenericFunctionsDontCacheTypeParams, false) LUAU_FASTFLAGVARIABLE(LuauIfElseBranchTypeUnion, false) LUAU_FASTFLAGVARIABLE(LuauIfElseExpectedType2, false) LUAU_FASTFLAGVARIABLE(LuauLengthOnCompositeType, false) @@ -1199,7 +1200,7 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatTypeAlias& typealias if (FFlag::LuauProperTypeLevels) aliasScope->level.subLevel = subLevel; - auto [generics, genericPacks] = createGenericTypes(aliasScope, scope->level, typealias, typealias.generics, typealias.genericPacks); + auto [generics, genericPacks] = createGenericTypes(aliasScope, scope->level, typealias, typealias.generics, typealias.genericPacks, /* useCache = */ true); TypeId ty = freshType(aliasScope); FreeTypeVar* ftv = getMutable(ty); @@ -5361,7 +5362,7 @@ TypeId TypeChecker::instantiateTypeFun(const ScopePtr& scope, const TypeFun& tf, } GenericTypeDefinitions TypeChecker::createGenericTypes(const ScopePtr& scope, std::optional levelOpt, const AstNode& node, - const AstArray& genericNames, const AstArray& genericPackNames) + const AstArray& genericNames, const AstArray& genericPackNames, bool useCache) { LUAU_ASSERT(scope->parent); @@ -5387,7 +5388,7 @@ GenericTypeDefinitions TypeChecker::createGenericTypes(const ScopePtr& scope, st } TypeId g; - if (FFlag::LuauRecursiveTypeParameterRestriction) + if (FFlag::LuauRecursiveTypeParameterRestriction && (!FFlag::LuauGenericFunctionsDontCacheTypeParams || useCache)) { TypeId& cached = scope->parent->typeAliasTypeParameters[n]; if (!cached) diff --git a/tests/TypeInfer.generics.test.cpp b/tests/TypeInfer.generics.test.cpp index a7f27551..8a2c6f27 100644 --- a/tests/TypeInfer.generics.test.cpp +++ b/tests/TypeInfer.generics.test.cpp @@ -673,4 +673,29 @@ local d: D = c R"(Type '() -> ()' could not be converted into '() -> ()'; different number of generic type pack parameters)"); } +TEST_CASE_FIXTURE(Fixture, "generic_functions_dont_cache_type_parameters") +{ + ScopedFastFlag sff{"LuauGenericFunctionsDontCacheTypeParams", true}; + + CheckResult result = check(R"( +-- See https://github.com/Roblox/luau/issues/332 +-- This function has a type parameter with the same name as clones, +-- so if we cache type parameter names for functions these get confused. +-- function id(x : Z) : Z +function id(x : X) : X + return x +end + +function clone(dict: {[X]:Y}): {[X]:Y} + local copy = {} + for k, v in pairs(dict) do + copy[k] = v + end + return copy +end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + TEST_SUITE_END(); From e51ff38d194d31cbdf272b5d116b9a6a04b1b5d8 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Fri, 4 Feb 2022 12:46:08 -0800 Subject: [PATCH 148/399] Sync to upstream/release/513 (#342) --- Analysis/include/Luau/TypeInfer.h | 3 ++- Analysis/src/TypeInfer.cpp | 7 ++++--- tests/TypeInfer.generics.test.cpp | 25 +++++++++++++++++++++++++ 3 files changed, 31 insertions(+), 4 deletions(-) diff --git a/Analysis/include/Luau/TypeInfer.h b/Analysis/include/Luau/TypeInfer.h index 90dc9f42..f61ecbf5 100644 --- a/Analysis/include/Luau/TypeInfer.h +++ b/Analysis/include/Luau/TypeInfer.h @@ -346,7 +346,8 @@ private: // Note: `scope` must be a fresh scope. GenericTypeDefinitions createGenericTypes(const ScopePtr& scope, std::optional levelOpt, const AstNode& node, - const AstArray& genericNames, const AstArray& genericPackNames); + const AstArray& genericNames, const AstArray& genericPackNames, + bool useCache = false); public: ErrorVec resolve(const PredicateVec& predicates, const ScopePtr& scope, bool sense); diff --git a/Analysis/src/TypeInfer.cpp b/Analysis/src/TypeInfer.cpp index b9096d2e..99398f74 100644 --- a/Analysis/src/TypeInfer.cpp +++ b/Analysis/src/TypeInfer.cpp @@ -29,6 +29,7 @@ LUAU_FASTFLAGVARIABLE(LuauWeakEqConstraint, false) // Eventually removed as fals LUAU_FASTFLAG(LuauUseCommittingTxnLog) LUAU_FASTFLAGVARIABLE(DebugLuauFreezeDuringUnification, false) LUAU_FASTFLAGVARIABLE(LuauRecursiveTypeParameterRestriction, false) +LUAU_FASTFLAGVARIABLE(LuauGenericFunctionsDontCacheTypeParams, false) LUAU_FASTFLAGVARIABLE(LuauIfElseBranchTypeUnion, false) LUAU_FASTFLAGVARIABLE(LuauIfElseExpectedType2, false) LUAU_FASTFLAGVARIABLE(LuauLengthOnCompositeType, false) @@ -1199,7 +1200,7 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatTypeAlias& typealias if (FFlag::LuauProperTypeLevels) aliasScope->level.subLevel = subLevel; - auto [generics, genericPacks] = createGenericTypes(aliasScope, scope->level, typealias, typealias.generics, typealias.genericPacks); + auto [generics, genericPacks] = createGenericTypes(aliasScope, scope->level, typealias, typealias.generics, typealias.genericPacks, /* useCache = */ true); TypeId ty = freshType(aliasScope); FreeTypeVar* ftv = getMutable(ty); @@ -5362,7 +5363,7 @@ TypeId TypeChecker::instantiateTypeFun(const ScopePtr& scope, const TypeFun& tf, } GenericTypeDefinitions TypeChecker::createGenericTypes(const ScopePtr& scope, std::optional levelOpt, const AstNode& node, - const AstArray& genericNames, const AstArray& genericPackNames) + const AstArray& genericNames, const AstArray& genericPackNames, bool useCache) { LUAU_ASSERT(scope->parent); @@ -5388,7 +5389,7 @@ GenericTypeDefinitions TypeChecker::createGenericTypes(const ScopePtr& scope, st } TypeId g; - if (FFlag::LuauRecursiveTypeParameterRestriction) + if (FFlag::LuauRecursiveTypeParameterRestriction && (!FFlag::LuauGenericFunctionsDontCacheTypeParams || useCache)) { TypeId& cached = scope->parent->typeAliasTypeParameters[n]; if (!cached) diff --git a/tests/TypeInfer.generics.test.cpp b/tests/TypeInfer.generics.test.cpp index a7f27551..8a2c6f27 100644 --- a/tests/TypeInfer.generics.test.cpp +++ b/tests/TypeInfer.generics.test.cpp @@ -673,4 +673,29 @@ local d: D = c R"(Type '() -> ()' could not be converted into '() -> ()'; different number of generic type pack parameters)"); } +TEST_CASE_FIXTURE(Fixture, "generic_functions_dont_cache_type_parameters") +{ + ScopedFastFlag sff{"LuauGenericFunctionsDontCacheTypeParams", true}; + + CheckResult result = check(R"( +-- See https://github.com/Roblox/luau/issues/332 +-- This function has a type parameter with the same name as clones, +-- so if we cache type parameter names for functions these get confused. +-- function id(x : Z) : Z +function id(x : X) : X + return x +end + +function clone(dict: {[X]:Y}): {[X]:Y} + local copy = {} + for k, v in pairs(dict) do + copy[k] = v + end + return copy +end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + TEST_SUITE_END(); From 7ffb5fc4fdafd37c1bc03ab24c2b7083d4424162 Mon Sep 17 00:00:00 2001 From: Lily Brown Date: Mon, 7 Feb 2022 12:08:43 -0800 Subject: [PATCH 149/399] Add Luau.Ast.CLI target (#345) Adds a `luau-ast` CLI that dumps Luau source to JSON. @asajeffrey and I are planning to use this functionality to construct an Agda model of the Luau type system/operational semantics, to allow formally proving properties of Luau's type systems. --- CLI/Ast.cpp | 84 ++++++++++++++++++++++++++++++++++++++++++++++++++ CMakeLists.txt | 5 +++ Sources.cmake | 7 +++++ 3 files changed, 96 insertions(+) create mode 100644 CLI/Ast.cpp diff --git a/CLI/Ast.cpp b/CLI/Ast.cpp new file mode 100644 index 00000000..4ea46236 --- /dev/null +++ b/CLI/Ast.cpp @@ -0,0 +1,84 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +#include + +#include "Luau/Common.h" +#include "Luau/Ast.h" +#include "Luau/JsonEncoder.h" +#include "Luau/Parser.h" +#include "Luau/ParseOptions.h" + +#include "FileUtils.h" + +static void displayHelp(const char* argv0) +{ + printf("Usage: %s [file]\n", argv0); +} + +static int assertionHandler(const char* expr, const char* file, int line, const char* function) +{ + printf("%s(%d): ASSERTION FAILED: %s\n", file, line, expr); + return 1; +} + +int main(int argc, char** argv) +{ + Luau::assertHandler() = assertionHandler; + + for (Luau::FValue* flag = Luau::FValue::list; flag; flag = flag->next) + if (strncmp(flag->name, "Luau", 4) == 0) + flag->value = true; + + if (argc >= 2 && strcmp(argv[1], "--help") == 0) + { + displayHelp(argv[0]); + return 0; + } + else if (argc < 2) + { + displayHelp(argv[0]); + return 1; + } + + const char* name = argv[1]; + std::optional maybeSource = std::nullopt; + if (strcmp(name, "-") == 0) + { + maybeSource = readStdin(); + } + else + { + maybeSource = readFile(name); + } + + if (!maybeSource) + { + fprintf(stderr, "Couldn't read source %s\n", name); + return 1; + } + + std::string source = *maybeSource; + + Luau::Allocator allocator; + Luau::AstNameTable names(allocator); + + Luau::ParseOptions options; + options.supportContinueStatement = true; + options.allowTypeAnnotations = true; + options.allowDeclarationSyntax = true; + + Luau::ParseResult parseResult = Luau::Parser::parse(source.data(), source.size(), names, allocator, options); + + if (parseResult.errors.size() > 0) + { + fprintf(stderr, "Parse errors were encountered:\n"); + for (const Luau::ParseError& error : parseResult.errors) + { + fprintf(stderr, " %s - %s\n", toString(error.getLocation()).c_str(), error.getMessage().c_str()); + } + fprintf(stderr, "\n"); + } + + printf("%s", Luau::toJson(parseResult.root).c_str()); + + return parseResult.errors.size() > 0 ? 1 : 0; +} diff --git a/CMakeLists.txt b/CMakeLists.txt index 881d3c3f..efef6e3a 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -21,10 +21,12 @@ add_library(isocline STATIC) if(LUAU_BUILD_CLI) add_executable(Luau.Repl.CLI) add_executable(Luau.Analyze.CLI) + add_executable(Luau.Ast.CLI) # This also adds target `name` on Linux/macOS and `name.exe` on Windows set_target_properties(Luau.Repl.CLI PROPERTIES OUTPUT_NAME luau) set_target_properties(Luau.Analyze.CLI PROPERTIES OUTPUT_NAME luau-analyze) + set_target_properties(Luau.Ast.CLI PROPERTIES OUTPUT_NAME luau-ast) endif() if(LUAU_BUILD_TESTS) @@ -98,6 +100,7 @@ endif() if(LUAU_BUILD_CLI) target_compile_options(Luau.Repl.CLI PRIVATE ${LUAU_OPTIONS}) target_compile_options(Luau.Analyze.CLI PRIVATE ${LUAU_OPTIONS}) + target_compile_options(Luau.Ast.CLI PRIVATE ${LUAU_OPTIONS}) target_include_directories(Luau.Repl.CLI PRIVATE extern extern/isocline/include) @@ -111,6 +114,8 @@ if(LUAU_BUILD_CLI) endif() target_link_libraries(Luau.Analyze.CLI PRIVATE Luau.Analysis) + + target_link_libraries(Luau.Ast.CLI PRIVATE Luau.Ast Luau.Analysis) endif() if(LUAU_BUILD_TESTS) diff --git a/Sources.cmake b/Sources.cmake index b36b6db5..4ab1d98e 100644 --- a/Sources.cmake +++ b/Sources.cmake @@ -193,6 +193,13 @@ if(TARGET Luau.Analyze.CLI) CLI/Analyze.cpp) endif() +if (TARGET Luau.Ast.CLI) + target_sources(Luau.Ast.CLI PRIVATE + CLI/FileUtils.h + CLI/FileUtils.cpp + CLI/Ast.cpp) +endif() + if(TARGET Luau.UnitTest) # Luau.UnitTest Sources target_sources(Luau.UnitTest PRIVATE From 324bc4b01ddb655f3526042426f21c382339f8ea Mon Sep 17 00:00:00 2001 From: Michael Savage Date: Mon, 7 Feb 2022 23:51:57 +0200 Subject: [PATCH 150/399] Add a build option to link with the static CRT (#343) LUAU_STATIC_CRT CMake option can be set externally (requires CMake 3.15+) --- CMakeLists.txt | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/CMakeLists.txt b/CMakeLists.txt index efef6e3a..c19d2b40 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -5,13 +5,20 @@ if(EXT_PLATFORM_STRING) endif() cmake_minimum_required(VERSION 3.0) -project(Luau LANGUAGES CXX C) option(LUAU_BUILD_CLI "Build CLI" ON) option(LUAU_BUILD_TESTS "Build tests" ON) option(LUAU_BUILD_WEB "Build Web module" OFF) option(LUAU_WERROR "Warnings as errors" OFF) +option(LUAU_STATIC_CRT "Link with the static CRT (/MT)" OFF) +if(LUAU_STATIC_CRT) + cmake_minimum_required(VERSION 3.15) + cmake_policy(SET CMP0091 NEW) + set(CMAKE_MSVC_RUNTIME_LIBRARY "MultiThreaded$<$:Debug>") +endif() + +project(Luau LANGUAGES CXX C) add_library(Luau.Ast STATIC) add_library(Luau.Compiler STATIC) add_library(Luau.Analysis STATIC) From 041838a9422829225afbda9a29e6db4fefa0acea Mon Sep 17 00:00:00 2001 From: Alan Jeffrey <403333+asajeffrey@users.noreply.github.com> Date: Tue, 8 Feb 2022 18:26:58 -0600 Subject: [PATCH 151/399] Prototyping a small subset of Luau in Agda (#350) * First cut reading JSON into an Agda representation of Luau syntax --- .github/workflows/prototyping.yml | 55 ++++++++++ prototyping/.gitignore | 6 ++ prototyping/Examples.agda | 4 + prototyping/Examples/SmokeTest.lua | 12 +++ prototyping/Examples/Syntax.agda | 24 +++++ prototyping/FFI/Data/Aeson.agda | 50 ++++++++++ prototyping/FFI/Data/Bool.agda | 8 ++ prototyping/FFI/Data/ByteString.agda | 7 ++ prototyping/FFI/Data/Either.agda | 16 +++ prototyping/FFI/Data/HaskellString.agda | 16 +++ prototyping/FFI/Data/Maybe.agda | 8 ++ prototyping/FFI/Data/Scientific.agda | 6 ++ prototyping/FFI/Data/String.agda | 8 ++ prototyping/FFI/Data/Text/Encoding.agda | 10 ++ prototyping/FFI/Data/Vector.agda | 31 ++++++ prototyping/FFI/IO.agda | 34 +++++++ prototyping/FFI/System/Exit.agda | 29 ++++++ prototyping/Luau/Syntax.agda | 35 +++++++ prototyping/Luau/Syntax/FromJSON.agda | 127 ++++++++++++++++++++++++ prototyping/Luau/Syntax/ToString.agda | 41 ++++++++ prototyping/PrettyPrinter.agda | 33 ++++++ prototyping/README.md | 27 +++++ 22 files changed, 587 insertions(+) create mode 100644 .github/workflows/prototyping.yml create mode 100644 prototyping/.gitignore create mode 100644 prototyping/Examples.agda create mode 100644 prototyping/Examples/SmokeTest.lua create mode 100644 prototyping/Examples/Syntax.agda create mode 100644 prototyping/FFI/Data/Aeson.agda create mode 100644 prototyping/FFI/Data/Bool.agda create mode 100644 prototyping/FFI/Data/ByteString.agda create mode 100644 prototyping/FFI/Data/Either.agda create mode 100644 prototyping/FFI/Data/HaskellString.agda create mode 100644 prototyping/FFI/Data/Maybe.agda create mode 100644 prototyping/FFI/Data/Scientific.agda create mode 100644 prototyping/FFI/Data/String.agda create mode 100644 prototyping/FFI/Data/Text/Encoding.agda create mode 100644 prototyping/FFI/Data/Vector.agda create mode 100644 prototyping/FFI/IO.agda create mode 100644 prototyping/FFI/System/Exit.agda create mode 100644 prototyping/Luau/Syntax.agda create mode 100644 prototyping/Luau/Syntax/FromJSON.agda create mode 100644 prototyping/Luau/Syntax/ToString.agda create mode 100644 prototyping/PrettyPrinter.agda create mode 100644 prototyping/README.md diff --git a/.github/workflows/prototyping.yml b/.github/workflows/prototyping.yml new file mode 100644 index 00000000..1d5a36f5 --- /dev/null +++ b/.github/workflows/prototyping.yml @@ -0,0 +1,55 @@ +name: prototyping + +on: + push: + branches: + - 'master' + paths: + - '.github/workflows/**' + - 'prototyping/**' + pull_request: + paths: + - '.github/workflows/**' + - 'prototyping/**' + +jobs: + linux: + strategy: + matrix: + agda: [2.6.2.1] + name: prototyping + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v1 + - uses: actions/cache@v2 + with: + path: ~/.cabal/store + key: prototyping-${{ runner.os }}-${{ matrix.agda }} + - name: install cabal + run: sudo apt-get install -y cabal-install + - name: cabal update + working-directory: prototyping + run: cabal update + - name: cabal install + working-directory: prototyping + run: | + cabal install Agda-${{ matrix.agda }} + cabal install --lib scientific --package-env . + cabal install --lib vector --package-env . + cabal install --lib aeson --package-env . + - name: check examples + working-directory: prototyping + run: ~/.cabal/bin/agda Examples.agda + - name: build PrettyPrinter + working-directory: prototyping + run: ~/.cabal/bin/agda --compile --ghc-flag=-v PrettyPrinter.agda + - name: cmake configure + run: cmake . + - name: cmake build luau-ast + run: cmake --build . --target Luau.Ast.CLI + - name: run smoketest + working-directory: prototyping + run: ../luau-ast Examples/SmokeTest.lua | ./PrettyPrinter > Examples/SmokeTestOutput.lua + - name: diff smoketest + working-directory: prototyping + run: diff Examples/SmokeTest.lua Examples/SmokeTestOutput.lua diff --git a/prototyping/.gitignore b/prototyping/.gitignore new file mode 100644 index 00000000..cca4f464 --- /dev/null +++ b/prototyping/.gitignore @@ -0,0 +1,6 @@ +*~ +*.agdai +Main +MAlonzo +PrettyPrinter +.ghc.* diff --git a/prototyping/Examples.agda b/prototyping/Examples.agda new file mode 100644 index 00000000..4b576377 --- /dev/null +++ b/prototyping/Examples.agda @@ -0,0 +1,4 @@ +module Examples where + +import Examples.Syntax + diff --git a/prototyping/Examples/SmokeTest.lua b/prototyping/Examples/SmokeTest.lua new file mode 100644 index 00000000..b1b91e19 --- /dev/null +++ b/prototyping/Examples/SmokeTest.lua @@ -0,0 +1,12 @@ +local function id(x) + return x +end +local function comp(f) + return function(g) + return function(x) + return f(g(x)) + end + end +end +local id2 = id(id) +local nil2 = id2(nil) diff --git a/prototyping/Examples/Syntax.agda b/prototyping/Examples/Syntax.agda new file mode 100644 index 00000000..d16dfebb --- /dev/null +++ b/prototyping/Examples/Syntax.agda @@ -0,0 +1,24 @@ +module Examples.Syntax where + +open import Agda.Builtin.Equality using (_≡_; refl) +open import FFI.Data.String using (_++_) +open import Luau.Syntax using (var; _$_; return; nil; function_⟨_⟩_end; _∙; _∙_) +open import Luau.Syntax.ToString using (exprToString; blockToString) + +f = var "f" +x = var "x" + +ex1 : exprToString(f $ x) ≡ + "f(x)" +ex1 = refl + +ex2 : blockToString(return nil ∙) ≡ + "return nil" +ex2 = refl + +ex3 : blockToString(function "f" ⟨ "x" ⟩ return x ∙ end ∙ return f ∙) ≡ + "local function f(x)\n" ++ + " return x\n" ++ + "end\n" ++ + "return f" +ex3 = refl diff --git a/prototyping/FFI/Data/Aeson.agda b/prototyping/FFI/Data/Aeson.agda new file mode 100644 index 00000000..a9082d31 --- /dev/null +++ b/prototyping/FFI/Data/Aeson.agda @@ -0,0 +1,50 @@ +module FFI.Data.Aeson where + +open import Agda.Builtin.Bool using (Bool) +open import Agda.Builtin.String using (String) + +open import FFI.Data.ByteString using (ByteString) +open import FFI.Data.HaskellString using (HaskellString; pack) +open import FFI.Data.Maybe using (Maybe) +open import FFI.Data.Either using (Either; mapLeft) +open import FFI.Data.Scientific using (Scientific) +open import FFI.Data.Vector using (Vector) + +{-# FOREIGN GHC import qualified Data.Aeson #-} +{-# FOREIGN GHC import qualified Data.Aeson.Key #-} +{-# FOREIGN GHC import qualified Data.Aeson.KeyMap #-} + +postulate + KeyMap : Set → Set + Key : Set + fromString : String → Key + toString : Key → String + lookup : ∀ {A} → Key -> KeyMap A -> Maybe A +{-# POLARITY KeyMap ++ #-} +{-# COMPILE GHC KeyMap = type Data.Aeson.KeyMap.KeyMap #-} +{-# COMPILE GHC Key = type Data.Aeson.Key.Key #-} +{-# COMPILE GHC fromString = Data.Aeson.Key.fromText #-} +{-# COMPILE GHC toString = Data.Aeson.Key.toText #-} +{-# COMPILE GHC lookup = \_ -> Data.Aeson.KeyMap.lookup #-} + +data Value : Set where + object : KeyMap Value → Value + array : Vector Value → Value + string : String → Value + number : Scientific → Value + bool : Bool → Value + null : Value +{-# COMPILE GHC Value = data Data.Aeson.Value (Data.Aeson.Object|Data.Aeson.Array|Data.Aeson.String|Data.Aeson.Number|Data.Aeson.Bool|Data.Aeson.Null) #-} + +Object = KeyMap Value +Array = Vector Value + +postulate + decode : ByteString → Maybe Value + eitherHDecode : ByteString → Either HaskellString Value +{-# COMPILE GHC decode = Data.Aeson.decodeStrict #-} +{-# COMPILE GHC eitherHDecode = Data.Aeson.eitherDecodeStrict #-} + +eitherDecode : ByteString → Either String Value +eitherDecode bytes = mapLeft pack (eitherHDecode bytes) + diff --git a/prototyping/FFI/Data/Bool.agda b/prototyping/FFI/Data/Bool.agda new file mode 100644 index 00000000..4b04b033 --- /dev/null +++ b/prototyping/FFI/Data/Bool.agda @@ -0,0 +1,8 @@ +module FFI.Data.Bool where + +{-# FOREIGN GHC import qualified Data.Bool #-} + +data Bool : Set where + false : Bool + true : Bool +{-# COMPILE GHC Bool = data Data.Bool.Bool (Data.Bool.False|Data.Bool.True) #-} diff --git a/prototyping/FFI/Data/ByteString.agda b/prototyping/FFI/Data/ByteString.agda new file mode 100644 index 00000000..670c5234 --- /dev/null +++ b/prototyping/FFI/Data/ByteString.agda @@ -0,0 +1,7 @@ +module FFI.Data.ByteString where + +{-# FOREIGN GHC import qualified Data.ByteString #-} + +postulate ByteString : Set +{-# COMPILE GHC ByteString = type Data.ByteString.ByteString #-} + diff --git a/prototyping/FFI/Data/Either.agda b/prototyping/FFI/Data/Either.agda new file mode 100644 index 00000000..d024c695 --- /dev/null +++ b/prototyping/FFI/Data/Either.agda @@ -0,0 +1,16 @@ +module FFI.Data.Either where + +{-# FOREIGN GHC import qualified Data.Either #-} + +data Either (A B : Set) : Set where + Left : A → Either A B + Right : B → Either A B +{-# COMPILE GHC Either = data Data.Either.Either (Data.Either.Left|Data.Either.Right) #-} + +mapLeft : ∀ {A B C} → (A → B) → (Either A C) → (Either B C) +mapLeft f (Left x) = Left (f x) +mapLeft f (Right x) = Right x + +mapRight : ∀ {A B C} → (B → C) → (Either A B) → (Either A C) +mapRight f (Left x) = Left x +mapRight f (Right x) = Right (f x) diff --git a/prototyping/FFI/Data/HaskellString.agda b/prototyping/FFI/Data/HaskellString.agda new file mode 100644 index 00000000..cb4ace37 --- /dev/null +++ b/prototyping/FFI/Data/HaskellString.agda @@ -0,0 +1,16 @@ +module FFI.Data.HaskellString where + +open import Agda.Builtin.String using (String) + +{-# FOREIGN GHC import qualified Data.String #-} +{-# FOREIGN GHC import qualified Data.Text #-} + +postulate HaskellString : Set +{-# COMPILE GHC HaskellString = type Data.String.String #-} + +postulate pack : HaskellString → String +{-# COMPILE GHC pack = Data.Text.pack #-} + +postulate unpack : String → HaskellString +{-# COMPILE GHC unpack = Data.Text.unpack #-} + diff --git a/prototyping/FFI/Data/Maybe.agda b/prototyping/FFI/Data/Maybe.agda new file mode 100644 index 00000000..b16fa810 --- /dev/null +++ b/prototyping/FFI/Data/Maybe.agda @@ -0,0 +1,8 @@ +module FFI.Data.Maybe where + +{-# FOREIGN GHC import qualified Data.Maybe #-} + +data Maybe (A : Set) : Set where + nothing : Maybe A + just : A → Maybe A +{-# COMPILE GHC Maybe = data Data.Maybe.Maybe (Data.Maybe.Nothing|Data.Maybe.Just) #-} diff --git a/prototyping/FFI/Data/Scientific.agda b/prototyping/FFI/Data/Scientific.agda new file mode 100644 index 00000000..8a5be39e --- /dev/null +++ b/prototyping/FFI/Data/Scientific.agda @@ -0,0 +1,6 @@ +module FFI.Data.Scientific where + +{-# FOREIGN GHC import qualified Data.Scientific #-} + +postulate Scientific : Set +{-# COMPILE GHC Scientific = type Data.Scientific.Scientific #-} diff --git a/prototyping/FFI/Data/String.agda b/prototyping/FFI/Data/String.agda new file mode 100644 index 00000000..5bb315cf --- /dev/null +++ b/prototyping/FFI/Data/String.agda @@ -0,0 +1,8 @@ +module FFI.Data.String where + +import Agda.Builtin.String + +String = Agda.Builtin.String.String + +infixr 5 _++_ +_++_ = Agda.Builtin.String.primStringAppend diff --git a/prototyping/FFI/Data/Text/Encoding.agda b/prototyping/FFI/Data/Text/Encoding.agda new file mode 100644 index 00000000..54f32486 --- /dev/null +++ b/prototyping/FFI/Data/Text/Encoding.agda @@ -0,0 +1,10 @@ +module FFI.Data.Text.Encoding where + +open import Agda.Builtin.String using (String) + +open import FFI.Data.ByteString using (ByteString) + +{-# FOREIGN GHC import qualified Data.Text.Encoding #-} + +postulate encodeUtf8 : String → ByteString +{-# COMPILE GHC encodeUtf8 = Data.Text.Encoding.encodeUtf8 #-} diff --git a/prototyping/FFI/Data/Vector.agda b/prototyping/FFI/Data/Vector.agda new file mode 100644 index 00000000..8ef8d63c --- /dev/null +++ b/prototyping/FFI/Data/Vector.agda @@ -0,0 +1,31 @@ +module FFI.Data.Vector where + +open import FFI.Data.Bool using (Bool; false; true) +open import FFI.Data.Maybe using (Maybe; just; nothing) + +{-# FOREIGN GHC import qualified Data.Vector #-} + +postulate Vector : Set → Set +{-# POLARITY Vector ++ #-} +{-# COMPILE GHC Vector = type Data.Vector.Vector #-} + +postulate + empty : ∀ {A} → (Vector A) + null : ∀ {A} → (Vector A) → Bool + unsafeHead : ∀ {A} → (Vector A) → A + unsafeTail : ∀ {A} → (Vector A) → (Vector A) +{-# COMPILE GHC empty = \_ -> Data.Vector.empty #-} +{-# COMPILE GHC null = \_ -> Data.Vector.null #-} +{-# COMPILE GHC unsafeHead = \_ -> Data.Vector.unsafeHead #-} +{-# COMPILE GHC unsafeTail = \_ -> Data.Vector.unsafeTail #-} + +head : ∀ {A} → (Vector A) → (Maybe A) +head vec with null vec +head vec | false = just (unsafeHead vec) +head vec | true = nothing + +tail : ∀ {A} → (Vector A) → Vector A +tail vec with null vec +tail vec | false = unsafeTail vec +tail vec | true = empty + diff --git a/prototyping/FFI/IO.agda b/prototyping/FFI/IO.agda new file mode 100644 index 00000000..825a788f --- /dev/null +++ b/prototyping/FFI/IO.agda @@ -0,0 +1,34 @@ +module FFI.IO where + +open import Agda.Builtin.IO using (IO) +open import Agda.Builtin.Unit using (⊤) +open import Agda.Builtin.String using (String) + +open import FFI.Data.HaskellString using (HaskellString; pack ; unpack) + +infixl 1 _>>=_ +infixl 1 _>>_ + +postulate + return : ∀ {a} {A : Set a} → A → IO A + _>>=_ : ∀ {a b} {A : Set a} {B : Set b} → IO A → (A → IO B) → IO B + fmap : ∀ {a b} {A : Set a} {B : Set b} → (A → B) → IO A → IO B + +{-# COMPILE GHC return = \_ _ -> return #-} +{-# COMPILE GHC _>>=_ = \_ _ _ _ -> (>>=) #-} +{-# COMPILE GHC fmap = \_ _ _ _ -> fmap #-} + +postulate getHContents : IO HaskellString +{-# COMPILE GHC getHContents = getContents #-} + +postulate putHStrLn : HaskellString → IO ⊤ +{-# COMPILE GHC putHStrLn = putStrLn #-} + +getContents : IO String +getContents = fmap pack getHContents + +putStrLn : String → IO ⊤ +putStrLn txt = putHStrLn (unpack txt) + +_>>_ : ∀ {a} {A : Set a} → IO ⊤ → IO A → IO A +a >> b = a >>= (λ _ → b ) diff --git a/prototyping/FFI/System/Exit.agda b/prototyping/FFI/System/Exit.agda new file mode 100644 index 00000000..fcf01395 --- /dev/null +++ b/prototyping/FFI/System/Exit.agda @@ -0,0 +1,29 @@ +module FFI.System.Exit where + +open import Agda.Builtin.Int using (Int) +open import Agda.Builtin.IO using (IO) +open import Agda.Builtin.Unit using (⊤) + +data ExitCode : Set where + ExitSuccess : ExitCode + ExitFailure : Int → ExitCode + +{-# FOREIGN GHC data AgdaExitCode = AgdaExitSuccess | AgdaExitFailure Integer #-} +{-# COMPILE GHC ExitCode = data AgdaExitCode (AgdaExitSuccess | AgdaExitFailure) #-} + +{-# FOREIGN GHC import qualified System.Exit #-} + +{-# FOREIGN GHC +toExitCode :: AgdaExitCode -> System.Exit.ExitCode +toExitCode AgdaExitSuccess = System.Exit.ExitSuccess +toExitCode (AgdaExitFailure n) = System.Exit.ExitFailure (fromIntegral n) + +fromExitCode :: System.Exit.ExitCode -> AgdaExitCode +fromExitCode System.Exit.ExitSuccess = AgdaExitSuccess +fromExitCode (System.Exit.ExitFailure n) = AgdaExitFailure (fromIntegral n) +#-} + +postulate + exitWith : ExitCode → IO ⊤ + +{-# COMPILE GHC exitWith = System.Exit.exitWith . toExitCode #-} diff --git a/prototyping/Luau/Syntax.agda b/prototyping/Luau/Syntax.agda new file mode 100644 index 00000000..c1d9c951 --- /dev/null +++ b/prototyping/Luau/Syntax.agda @@ -0,0 +1,35 @@ +module Luau.Syntax where + +open import Agda.Builtin.String using (String) + +infixr 5 _∙_ + +data Type : Set where + nil : Type + _⇒_ : Type → Type → Type + none : Type + any : Type + _∪_ : Type → Type → Type + _∩_ : Type → Type → Type + +Var : Set +Var = String + +data Block : Set +data Stat : Set +data Expr : Set + +data Block where + _∙_ : Stat → Block → Block + _∙ : Stat → Block + +data Stat where + function_⟨_⟩_end : Var → Var → Block → Stat + local_←_ : Var → Expr → Stat + return : Expr → Stat + +data Expr where + nil : Expr + var : Var → Expr + _$_ : Expr → Expr → Expr + function⟨_⟩_end : Var → Block → Expr diff --git a/prototyping/Luau/Syntax/FromJSON.agda b/prototyping/Luau/Syntax/FromJSON.agda new file mode 100644 index 00000000..36432118 --- /dev/null +++ b/prototyping/Luau/Syntax/FromJSON.agda @@ -0,0 +1,127 @@ +module Luau.Syntax.FromJSON where + +open import Luau.Syntax using (Type; Block; Stat ; Expr; nil; _$_; var; function⟨_⟩_end; local_←_; function_⟨_⟩_end; return; _∙; _∙_) + +open import Agda.Builtin.List using (List; _∷_; []) + +open import FFI.Data.Aeson using (Value; Array; Object; object; array; string; fromString; lookup) +open import FFI.Data.Bool using (true; false) +open import FFI.Data.Either using (Either; Left; Right) +open import FFI.Data.Maybe using (Maybe; nothing; just) +open import FFI.Data.String using (String; _++_) +open import FFI.Data.Vector using (head; tail; null; empty) + +args = fromString "args" +body = fromString "body" +func = fromString "func" +lokal = fromString "local" +list = fromString "list" +name = fromString "name" +type = fromString "type" +values = fromString "values" +vars = fromString "vars" + +data Lookup : Set where + _,_ : String → Value → Lookup + nil : Lookup + +lookupIn : List String → Object → Lookup +lookupIn [] obj = nil +lookupIn (key ∷ keys) obj with lookup (fromString key) obj +lookupIn (key ∷ keys) obj | nothing = lookupIn keys obj +lookupIn (key ∷ keys) obj | just value = (key , value) + +exprFromJSON : Value → Either String Expr +exprFromObject : Object → Either String Expr +statFromJSON : Value → Either String Stat +statFromObject : Object → Either String Stat +blockFromJSON : Value → Either String Block +blockFromArray : Array → Either String Block + +exprFromJSON (object obj) = exprFromObject obj +exprFromJSON val = Left "AstExpr not an object" + +exprFromObject obj with lookup type obj +exprFromObject obj | just (string "AstExprCall") with lookup func obj | lookup args obj +exprFromObject obj | just (string "AstExprCall") | just value | just (array arr) with head arr +exprFromObject obj | just (string "AstExprCall") | just value | just (array arr) | just value2 with exprFromJSON value | exprFromJSON value2 +exprFromObject obj | just (string "AstExprCall") | just value | just (array arr) | just value2 | Right fun | Right arg = Right (fun $ arg) +exprFromObject obj | just (string "AstExprCall") | just value | just (array arr) | just value2 | Left err | _ = Left err +exprFromObject obj | just (string "AstExprCall") | just value | just (array arr) | just value2 | _ | Left err = Left err +exprFromObject obj | just (string "AstExprCall") | just value | just (array arr) | nothing = Left ("AstExprCall empty args") +exprFromObject obj | just (string "AstExprCall") | just value | just _ = Left ("AstExprCall args not an array") +exprFromObject obj | just (string "AstExprCall") | nothing | _ = Left ("AstExprCall missing func") +exprFromObject obj | just (string "AstExprCall") | _ | nothing = Left ("AstExprCall missing args") +exprFromObject obj | just (string "AstExprConstantNil") = Right nil +exprFromObject obj | just (string "AstExprFunction") with lookup args obj | lookup body obj +exprFromObject obj | just (string "AstExprFunction") | just (array arr) | just value with head arr | blockFromJSON value +exprFromObject obj | just (string "AstExprFunction") | just (array arr) | just value | just (string x) | Right B = Right (function⟨ x ⟩ B end) +exprFromObject obj | just (string "AstExprFunction") | just (array arr) | just value | just _ | Right B = Left "AstExprFunction args not a string array" +exprFromObject obj | just (string "AstExprFunction") | just (array arr) | just value | nothing | Right B = Left "Unsupported AstExprFunction empty args" +exprFromObject obj | just (string "AstExprFunction") | just (array arr) | just value | _ | Left err = Left err +exprFromObject obj | just (string "AstExprFunction") | just _ | just _ = Left "AstExprFunction args not an array" +exprFromObject obj | just (string "AstExprFunction") | nothing | _ = Left "AstExprFunction missing args" +exprFromObject obj | just (string "AstExprFunction") | _ | nothing = Left "AstExprFunction missing body" +exprFromObject obj | just (string "AstExprLocal") with lookup lokal obj +exprFromObject obj | just (string "AstExprLocal") | just (string x) = Right (var x) +exprFromObject obj | just (string "AstExprLocal") | just (_) = Left "AstExprLocal local not a string" +exprFromObject obj | just (string "AstExprLocal") | nothing = Left "AstExprLocal missing local" +exprFromObject obj | just (string ty) = Left ("TODO: Unsupported AstExpr " ++ ty) +exprFromObject obj | just _ = Left "AstExpr type not a string" +exprFromObject obj | nothing = Left "AstExpr missing type" + +{-# NON_TERMINATING #-} +statFromJSON (object obj) = statFromObject obj +statFromJSON _ = Left "AstStat not an object" + +statFromObject obj with lookup type obj +statFromObject obj | just(string "AstStatLocal") with lookup vars obj | lookup values obj +statFromObject obj | just(string "AstStatLocal") | just(array arr1) | just(array arr2) with head(arr1) | head(arr2) +statFromObject obj | just(string "AstStatLocal") | just(array arr1) | just(array arr2) | just(string x) | just(value) with exprFromJSON(value) +statFromObject obj | just(string "AstStatLocal") | just(array arr1) | just(array arr2) | just(string x) | just(value) | Right M = Right (local x ← M) +statFromObject obj | just(string "AstStatLocal") | just(array arr1) | just(array arr2) | just(string x) | just(value) | Left err = Left err +statFromObject obj | just(string "AstStatLocal") | just(array arr1) | just(array arr2) | just(string x) | nothing = Left "AstStatLocal empty values" +statFromObject obj | just(string "AstStatLocal") | just(array arr1) | just(array arr2) | just(_) | _ = Left "AstStatLocal vars not a string array" +statFromObject obj | just(string "AstStatLocal") | just(array arr1) | just(array arr2) | nothing | _ = Left "AstStatLocal empty vars" +statFromObject obj | just(string "AstStatLocal") | just(array arr1) | just(_) = Left "AstStatLocal values not an array" +statFromObject obj | just(string "AstStatLocal") | just(_) | just(_) = Left "AstStatLocal vars not an array" +statFromObject obj | just(string "AstStatLocal") | just(_) | nothing = Left "AstStatLocal missing values" +statFromObject obj | just(string "AstStatLocal") | nothing | _ = Left "AstStatLocal missing vars" +statFromObject obj | just(string "AstStatLocalFunction") with lookup name obj | lookup func obj +statFromObject obj | just(string "AstStatLocalFunction") | just (string f) | just value with exprFromJSON value +statFromObject obj | just(string "AstStatLocalFunction") | just (string f) | just value | Right (function⟨ x ⟩ B end) = Right (function f ⟨ x ⟩ B end) +statFromObject obj | just(string "AstStatLocalFunction") | just (string f) | just value | Left err = Left err +statFromObject obj | just(string "AstStatLocalFunction") | just _ | just _ | Right _ = Left "AstStatLocalFunction func is not an AstExprFunction" +statFromObject obj | just(string "AstStatLocalFunction") | just _ | just _ = Left "AstStatLocalFunction name is not a string" +statFromObject obj | just(string "AstStatLocalFunction") | nothing | _ = Left "AstStatFunction missing name" +statFromObject obj | just(string "AstStatLocalFunction") | _ | nothing = Left "AstStatFunction missing func" +statFromObject obj | just(string "AstStatReturn") with lookup list obj +statFromObject obj | just(string "AstStatReturn") | just(array arr) with head arr +statFromObject obj | just(string "AstStatReturn") | just(array arr) | just value with exprFromJSON value +statFromObject obj | just(string "AstStatReturn") | just(array arr) | just value | Right M = Right (return M) +statFromObject obj | just(string "AstStatReturn") | just(array arr) | just value | Left err = Left err +statFromObject obj | just(string "AstStatReturn") | just(array arr) | nothing = Left "AstStatReturn empty list" +statFromObject obj | just(string "AstStatReturn") | just(_) = Left "AstStatReturn list not an array" +statFromObject obj | just(string "AstStatReturn") | nothing = Left "AstStatReturn missing list" +statFromObject obj | just (string ty) = Left ("TODO: Unsupported AstStat " ++ ty) +statFromObject obj | just _ = Left "AstStat type not a string" +statFromObject obj | nothing = Left "AstStat missing type" + +blockFromJSON (array arr) = blockFromArray arr +blockFromJSON (object obj) with lookup type obj | lookup body obj +blockFromJSON (object obj) | just (string "AstStatBlock") | just value = blockFromJSON value +blockFromJSON (object obj) | just (string "AstStatBlock") | nothing = Left "AstStatBlock missing body" +blockFromJSON (object obj) | just (string ty) | _ = Left ("Unsupported AstBlock " ++ ty) +blockFromJSON (object obj) | just _ | _ = Left "AstStatBlock type not a string" +blockFromJSON (object obj) | nothing | _ = Left "AstStatBlock missing type" +blockFromJSON _ = Left "AstBlock not an array or AstStatBlock object" + +blockFromArray arr with head arr +blockFromArray arr | nothing = Left "Block should be a non-empty array" +blockFromArray arr | just value with statFromJSON value +blockFromArray arr | just value | Left err = Left err +blockFromArray arr | just value | Right S with null (tail arr) +blockFromArray arr | just value | Right S | true = Right (S ∙) +blockFromArray arr | just value | Right S | false with blockFromArray(tail arr) +blockFromArray arr | just value | Right S | false | Left err = Left (err) +blockFromArray arr | just value | Right S | false | Right B = Right (S ∙ B) diff --git a/prototyping/Luau/Syntax/ToString.agda b/prototyping/Luau/Syntax/ToString.agda new file mode 100644 index 00000000..87876565 --- /dev/null +++ b/prototyping/Luau/Syntax/ToString.agda @@ -0,0 +1,41 @@ +module Luau.Syntax.ToString where + +open import Luau.Syntax using (Type; Block; Stat; Expr; nil; var; _$_; function⟨_⟩_end; return; function_⟨_⟩_end ;local_←_; _∙_; _∙) + +open import FFI.Data.String using (String; _++_) + +exprToString′ : String → Expr → String +statToString′ : String → Stat → String +blockToString′ : String → Block → String + +exprToString′ lb nil = + "nil" +exprToString′ lb (var x) = + x +exprToString′ lb (M $ N) = + (exprToString′ lb M) ++ "(" ++ (exprToString′ lb N) ++ ")" +exprToString′ lb (function⟨ x ⟩ B end) = + "function(" ++ x ++ ")" ++ lb ++ + " " ++ (blockToString′ (lb ++ " ") B) ++ lb ++ + "end" + +statToString′ lb (function f ⟨ x ⟩ B end) = + "local function " ++ f ++ "(" ++ x ++ ")" ++ lb ++ + " " ++ (blockToString′ (lb ++ " ") B) ++ lb ++ + "end" +statToString′ lb (local x ← M) = + "local " ++ x ++ " = " ++ (exprToString′ lb M) +statToString′ lb (return M) = + "return " ++ (exprToString′ lb M) + +blockToString′ lb (S ∙ B) = statToString′ lb S ++ lb ++ blockToString′ lb B +blockToString′ lb (S ∙) = statToString′ lb S + +exprToString : Expr → String +exprToString = exprToString′ "\n" + +statToString : Stat → String +statToString = statToString′ "\n" + +blockToString : Block → String +blockToString = blockToString′ "\n" diff --git a/prototyping/PrettyPrinter.agda b/prototyping/PrettyPrinter.agda new file mode 100644 index 00000000..dfb2c053 --- /dev/null +++ b/prototyping/PrettyPrinter.agda @@ -0,0 +1,33 @@ +module PrettyPrinter where + +open import Agda.Builtin.IO using (IO) +open import Agda.Builtin.Int using (pos) +open import Agda.Builtin.Unit using (⊤) + +open import FFI.IO using (getContents; putStrLn; _>>=_; _>>_) +open import FFI.Data.Aeson using (Value; eitherDecode) +open import FFI.Data.Either using (Left; Right) +open import FFI.Data.String using (String; _++_) +open import FFI.Data.Text.Encoding using (encodeUtf8) +open import FFI.System.Exit using (exitWith; ExitFailure) + +open import Luau.Syntax using (Block) +open import Luau.Syntax.FromJSON using (blockFromJSON) +open import Luau.Syntax.ToString using (blockToString) + +runBlock : Block → IO ⊤ +runBlock block = putStrLn (blockToString block) + +runJSON : Value → IO ⊤ +runJSON value with blockFromJSON(value) +runJSON value | (Left err) = putStrLn ("Luau error: " ++ err) >> exitWith (ExitFailure (pos 1)) +runJSON value | (Right block) = runBlock block + +runString : String → IO ⊤ +runString txt with eitherDecode (encodeUtf8 txt) +runString txt | (Left err) = putStrLn ("JSON error: " ++ err) >> exitWith (ExitFailure (pos 1)) +runString txt | (Right value) = runJSON value + +main : IO ⊤ +main = getContents >>= runString + diff --git a/prototyping/README.md b/prototyping/README.md new file mode 100644 index 00000000..f1761d1c --- /dev/null +++ b/prototyping/README.md @@ -0,0 +1,27 @@ +# Prototyping Luau + +![prototyping workflow](https://github.com/Roblox/luau/actions/workflows/prototyping.yml/badge.svg) + +An experimental prototyping system for the Luau type system. This is +intended to allow core language features to be tested quickly, without +having to interact with all the features of production Lua. + +## Building + +First install Haskell and Agda. + +Install dependencies: +``` + cabal update + cabal install --lib aeson scientific vector +``` + +Then compile +``` + agda --compile PrettyPrinter.agda +``` + +and run! +``` + luau-ast Examples/SmokeTest.lua | ./PrettyPrinter +``` From ec481695a3bc82c66477ee74206d77e8bc871f9b Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Wed, 9 Feb 2022 09:19:50 -0800 Subject: [PATCH 152/399] Update library.md (#352) Clarify the relationship between `typeof` and `newproxy`. As a sandboxing measure, `typeof` only uses `__type` on host-defined userdata. Fixes #351. --- docs/_pages/library.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/docs/_pages/library.md b/docs/_pages/library.md index 834bd73b..7695762d 100644 --- a/docs/_pages/library.md +++ b/docs/_pages/library.md @@ -128,7 +128,8 @@ Returns the type of the object, which is one of `"nil"`, `"boolean"`, `"number"` function typeof(obj: any): string ``` -Returns the type of the object; for userdata objects that have a metatable with the `__type` field, returns the value for that key. +Returns the type of the object; for userdata objects that have a metatable with the `__type` field *and* are defined by the host with an internal tag, returns the value for that key. +For custom userdata objects, such as ones returned by `newproxy`, this function returns `"userdata"` to make sure host-defined types can not be spoofed. ``` function ipairs(t: table): From abe3f87b483b323e04a378c56b95a3757b8853e7 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Wed, 9 Feb 2022 09:27:01 -0800 Subject: [PATCH 153/399] docs: Add documentation for upcoming MisleadingAndOr lint (#349) This is going to be part of Luau 0.514 --- docs/_pages/lint.md | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/docs/_pages/lint.md b/docs/_pages/lint.md index ff677829..de4d247e 100644 --- a/docs/_pages/lint.md +++ b/docs/_pages/lint.md @@ -294,6 +294,7 @@ table.insert(t, 0, 42) -- table.insert uses index 0 but arrays are 1-based; did table.insert(t, #t+1, 42) -- table.insert will append the value to the table; consider removing the second argument for efficiency ``` + ## DuplicateCondition (24) When checking multiple conditions via `and/or` or `if/elseif`, a copy & paste error may result in checking the same condition redundantly. This almost always indicates a bug, so a warning is emitted when use of a duplicate condition is detected. @@ -301,3 +302,18 @@ When checking multiple conditions via `and/or` or `if/elseif`, a copy & paste er ```lua assert(self._adorns[normID1] and self._adorns[normID1]) -- Condition has already been checked on column 8 ``` + +## MisleadingAndOr (25) + +In Lua, there is no first-class ternary operator but it can be emulated via `a and b or c` pattern. However, due to how boolean evaluation works, if `b` is `false` or `nil`, the resulting expression evaluates to `c` regardless of the value of `a`. Luau solves this problem with the `if a then b else c` expression; a warning is emitted for and-or expressions where the first alternative is `false` or `nil` because it's almost always a bug. + +```lua +-- The and-or expression always evaluates to the second alternative because the first alternative is false; consider using if-then-else expression instead +local x = flag and false or true +``` + +The code above can be rewritten as follows to avoid the warning and the associated bug: + +```lua +local x = if flag then false else true +``` From 5187e64f88953f34785ffe58acd0610ee5041f5f Mon Sep 17 00:00:00 2001 From: Alan Jeffrey <403333+asajeffrey@users.noreply.github.com> Date: Wed, 9 Feb 2022 17:14:29 -0600 Subject: [PATCH 154/399] Implement a prototype interpreter (#353) * First cut interpreter --- .github/workflows/prototyping.yml | 12 ++- prototyping/Examples.agda | 3 + prototyping/Examples/OpSem.agda | 11 +++ prototyping/Examples/Run.agda | 18 ++++ prototyping/Examples/SmokeTest.lua | 3 +- prototyping/Examples/SmokeTestOutput.lua | 12 +++ prototyping/Examples/Syntax.agda | 6 +- prototyping/FFI/Data/HaskellInt.agda | 14 ++++ prototyping/FFI/Data/Vector.agda | 15 +++- prototyping/Interpreter.agda | 39 +++++++++ prototyping/Luau/Addr.agda | 17 ++++ prototyping/Luau/Addr/ToString.agda | 8 ++ prototyping/Luau/Heap.agda | 48 +++++++++++ prototyping/Luau/OpSem.agda | 92 +++++++++++++++++++++ prototyping/Luau/Run.agda | 28 +++++++ prototyping/Luau/RuntimeError.agda | 21 +++++ prototyping/Luau/RuntimeError/ToString.agda | 18 ++++ prototyping/Luau/Substitution.agda | 30 +++++++ prototyping/Luau/Syntax.agda | 18 ++-- prototyping/Luau/Syntax/FromJSON.agda | 12 ++- prototyping/Luau/Syntax/ToString.agda | 16 +++- prototyping/Luau/Type.agda | 10 +++ prototyping/Luau/Value.agda | 15 ++++ prototyping/Luau/Value/ToString.agda | 10 +++ prototyping/Luau/Var.agda | 16 ++++ prototyping/Luau/Var/ToString.agda | 8 ++ prototyping/Properties.agda | 3 + prototyping/Properties/Dec.agda | 8 ++ prototyping/Properties/Remember.agda | 9 ++ prototyping/Properties/Step.agda | 58 +++++++++++++ 30 files changed, 546 insertions(+), 32 deletions(-) create mode 100644 prototyping/Examples/OpSem.agda create mode 100644 prototyping/Examples/Run.agda create mode 100644 prototyping/Examples/SmokeTestOutput.lua create mode 100644 prototyping/FFI/Data/HaskellInt.agda create mode 100644 prototyping/Interpreter.agda create mode 100644 prototyping/Luau/Addr.agda create mode 100644 prototyping/Luau/Addr/ToString.agda create mode 100644 prototyping/Luau/Heap.agda create mode 100644 prototyping/Luau/OpSem.agda create mode 100644 prototyping/Luau/Run.agda create mode 100644 prototyping/Luau/RuntimeError.agda create mode 100644 prototyping/Luau/RuntimeError/ToString.agda create mode 100644 prototyping/Luau/Substitution.agda create mode 100644 prototyping/Luau/Type.agda create mode 100644 prototyping/Luau/Value.agda create mode 100644 prototyping/Luau/Value/ToString.agda create mode 100644 prototyping/Luau/Var.agda create mode 100644 prototyping/Luau/Var/ToString.agda create mode 100644 prototyping/Properties.agda create mode 100644 prototyping/Properties/Dec.agda create mode 100644 prototyping/Properties/Remember.agda create mode 100644 prototyping/Properties/Step.agda diff --git a/.github/workflows/prototyping.yml b/.github/workflows/prototyping.yml index 1d5a36f5..02e021ee 100644 --- a/.github/workflows/prototyping.yml +++ b/.github/workflows/prototyping.yml @@ -7,10 +7,12 @@ on: paths: - '.github/workflows/**' - 'prototyping/**' + - 'Analysis/src/JsonEncoder.cpp' pull_request: paths: - '.github/workflows/**' - 'prototyping/**' + - 'Analysis/src/JsonEncoder.cpp' jobs: linux: @@ -40,16 +42,20 @@ jobs: - name: check examples working-directory: prototyping run: ~/.cabal/bin/agda Examples.agda - - name: build PrettyPrinter + - name: build executables working-directory: prototyping - run: ~/.cabal/bin/agda --compile --ghc-flag=-v PrettyPrinter.agda + run: | + ~/.cabal/bin/agda --compile PrettyPrinter.agda + ~/.cabal/bin/agda --compile Interpreter.agda - name: cmake configure run: cmake . - name: cmake build luau-ast run: cmake --build . --target Luau.Ast.CLI - name: run smoketest working-directory: prototyping - run: ../luau-ast Examples/SmokeTest.lua | ./PrettyPrinter > Examples/SmokeTestOutput.lua + run: | + ../luau-ast Examples/SmokeTest.lua | ./PrettyPrinter > Examples/SmokeTestOutput.lua + ../luau-ast Examples/SmokeTest.lua | ./Interpreter - name: diff smoketest working-directory: prototyping run: diff Examples/SmokeTest.lua Examples/SmokeTestOutput.lua diff --git a/prototyping/Examples.agda b/prototyping/Examples.agda index 4b576377..fe396eff 100644 --- a/prototyping/Examples.agda +++ b/prototyping/Examples.agda @@ -1,4 +1,7 @@ +{-# OPTIONS --rewriting #-} module Examples where import Examples.Syntax +import Examples.OpSem +import Examples.Run diff --git a/prototyping/Examples/OpSem.agda b/prototyping/Examples/OpSem.agda new file mode 100644 index 00000000..4043869e --- /dev/null +++ b/prototyping/Examples/OpSem.agda @@ -0,0 +1,11 @@ +module Examples.OpSem where + +open import Luau.OpSem using (_⊢_⟶ᴱ_⊣_; _⊢_⟶ᴮ_⊣_; subst) +open import Luau.Syntax using (var; nil; local_←_; _∙_; done; return) +open import Luau.Heap using (emp) + +x = var "x" + +ex1 : emp ⊢ (local "x" ← nil ∙ return x ∙ done) ⟶ᴮ (return nil ∙ done) ⊣ emp +ex1 = subst + diff --git a/prototyping/Examples/Run.agda b/prototyping/Examples/Run.agda new file mode 100644 index 00000000..bfa79839 --- /dev/null +++ b/prototyping/Examples/Run.agda @@ -0,0 +1,18 @@ +{-# OPTIONS --rewriting #-} + +module Examples.Run where + +open import Agda.Builtin.Equality using (_≡_; refl) +open import Luau.Syntax using (nil; var; _$_; function_⟨_⟩_end; return; _∙_; done) +open import Luau.Value using (nil) +open import Luau.Run using (run; return) +open import Luau.Heap using (emp; lookup-next; next-emp; lookup-next-emp) + +import Agda.Builtin.Equality.Rewrite +{-# REWRITE lookup-next next-emp lookup-next-emp #-} + +x = var "x" +id = var "id" + +ex1 : (run (function "id" ⟨ "x" ⟩ return x ∙ done end ∙ return (id $ nil) ∙ done) ≡ return nil _) +ex1 = refl diff --git a/prototyping/Examples/SmokeTest.lua b/prototyping/Examples/SmokeTest.lua index b1b91e19..e1e37e32 100644 --- a/prototyping/Examples/SmokeTest.lua +++ b/prototyping/Examples/SmokeTest.lua @@ -8,5 +8,6 @@ local function comp(f) end end end -local id2 = id(id) +local id2 = comp(id)(id) local nil2 = id2(nil) +return id2(nil2) diff --git a/prototyping/Examples/SmokeTestOutput.lua b/prototyping/Examples/SmokeTestOutput.lua new file mode 100644 index 00000000..b1b91e19 --- /dev/null +++ b/prototyping/Examples/SmokeTestOutput.lua @@ -0,0 +1,12 @@ +local function id(x) + return x +end +local function comp(f) + return function(g) + return function(x) + return f(g(x)) + end + end +end +local id2 = id(id) +local nil2 = id2(nil) diff --git a/prototyping/Examples/Syntax.agda b/prototyping/Examples/Syntax.agda index d16dfebb..8af9bca8 100644 --- a/prototyping/Examples/Syntax.agda +++ b/prototyping/Examples/Syntax.agda @@ -2,7 +2,7 @@ module Examples.Syntax where open import Agda.Builtin.Equality using (_≡_; refl) open import FFI.Data.String using (_++_) -open import Luau.Syntax using (var; _$_; return; nil; function_⟨_⟩_end; _∙; _∙_) +open import Luau.Syntax using (var; _$_; return; nil; function_⟨_⟩_end; done; _∙_) open import Luau.Syntax.ToString using (exprToString; blockToString) f = var "f" @@ -12,11 +12,11 @@ ex1 : exprToString(f $ x) ≡ "f(x)" ex1 = refl -ex2 : blockToString(return nil ∙) ≡ +ex2 : blockToString(return nil ∙ done) ≡ "return nil" ex2 = refl -ex3 : blockToString(function "f" ⟨ "x" ⟩ return x ∙ end ∙ return f ∙) ≡ +ex3 : blockToString(function "f" ⟨ "x" ⟩ return x ∙ done end ∙ return f ∙ done) ≡ "local function f(x)\n" ++ " return x\n" ++ "end\n" ++ diff --git a/prototyping/FFI/Data/HaskellInt.agda b/prototyping/FFI/Data/HaskellInt.agda new file mode 100644 index 00000000..9ab0868e --- /dev/null +++ b/prototyping/FFI/Data/HaskellInt.agda @@ -0,0 +1,14 @@ +module FFI.Data.HaskellInt where + +open import Agda.Builtin.Int using (Int) + +{-# FOREIGN GHC import qualified Data.Int #-} + +postulate HaskellInt : Set +{-# COMPILE GHC HaskellInt = type Data.Int.Int #-} + +postulate + intToHaskellInt : Int → HaskellInt + haskellIntToInt : HaskellInt → Int +{-# COMPILE GHC intToHaskellInt = fromIntegral #-} +{-# COMPILE GHC haskellIntToInt = fromIntegral #-} diff --git a/prototyping/FFI/Data/Vector.agda b/prototyping/FFI/Data/Vector.agda index 8ef8d63c..2c5d1925 100644 --- a/prototyping/FFI/Data/Vector.agda +++ b/prototyping/FFI/Data/Vector.agda @@ -1,6 +1,10 @@ module FFI.Data.Vector where +open import Agda.Builtin.Equality using (_≡_) +open import Agda.Builtin.Int using (Int; pos; negsuc) +open import Agda.Builtin.Nat using (Nat) open import FFI.Data.Bool using (Bool; false; true) +open import FFI.Data.HaskellInt using (HaskellInt; haskellIntToInt; intToHaskellInt) open import FFI.Data.Maybe using (Maybe; just; nothing) {-# FOREIGN GHC import qualified Data.Vector #-} @@ -14,10 +18,20 @@ postulate null : ∀ {A} → (Vector A) → Bool unsafeHead : ∀ {A} → (Vector A) → A unsafeTail : ∀ {A} → (Vector A) → (Vector A) + length : ∀ {A} → (Vector A) → Nat + lookup : ∀ {A} → (Vector A) → Nat → (Maybe A) + snoc : ∀ {A} → (Vector A) → A → (Vector A) {-# COMPILE GHC empty = \_ -> Data.Vector.empty #-} {-# COMPILE GHC null = \_ -> Data.Vector.null #-} {-# COMPILE GHC unsafeHead = \_ -> Data.Vector.unsafeHead #-} {-# COMPILE GHC unsafeTail = \_ -> Data.Vector.unsafeTail #-} +{-# COMPILE GHC length = \_ -> (fromIntegral . Data.Vector.length) #-} +{-# COMPILE GHC lookup = \_ v -> ((v Data.Vector.!?) . fromIntegral) #-} +{-# COMPILE GHC snoc = \_ -> Data.Vector.snoc #-} + +postulate length-empty : ∀ {A} → (length (empty {A}) ≡ 0) +postulate lookup-snoc : ∀ {A} (x : A) (v : Vector A) → (lookup (snoc v x) (length v) ≡ just x) +postulate lookup-snoc-empty : ∀ {A} (x : A) → (lookup (snoc empty x) 0 ≡ just x) head : ∀ {A} → (Vector A) → (Maybe A) head vec with null vec @@ -28,4 +42,3 @@ tail : ∀ {A} → (Vector A) → Vector A tail vec with null vec tail vec | false = unsafeTail vec tail vec | true = empty - diff --git a/prototyping/Interpreter.agda b/prototyping/Interpreter.agda new file mode 100644 index 00000000..9184235e --- /dev/null +++ b/prototyping/Interpreter.agda @@ -0,0 +1,39 @@ +module Interpreter where + +open import Agda.Builtin.IO using (IO) +open import Agda.Builtin.Int using (pos) +open import Agda.Builtin.Unit using (⊤) + +open import FFI.IO using (getContents; putStrLn; _>>=_; _>>_) +open import FFI.Data.Aeson using (Value; eitherDecode) +open import FFI.Data.Either using (Left; Right) +open import FFI.Data.String using (String; _++_) +open import FFI.Data.Text.Encoding using (encodeUtf8) +open import FFI.System.Exit using (exitWith; ExitFailure) + +open import Luau.Syntax using (Block) +open import Luau.Syntax.FromJSON using (blockFromJSON) +open import Luau.Syntax.ToString using (blockToString) +open import Luau.Run using (run; return; done; error) +open import Luau.RuntimeError.ToString using (errToStringᴮ) +open import Luau.Value.ToString using (valueToString) + +runBlock : Block → IO ⊤ +runBlock block with run block +runBlock block | return V D = putStrLn (valueToString V) +runBlock block | done D = putStrLn "nil" +runBlock block | error E D = putStrLn (errToStringᴮ E) + +runJSON : Value → IO ⊤ +runJSON value with blockFromJSON(value) +runJSON value | (Left err) = putStrLn ("Luau error: " ++ err) >> exitWith (ExitFailure (pos 1)) +runJSON value | (Right block) = runBlock block + +runString : String → IO ⊤ +runString txt with eitherDecode (encodeUtf8 txt) +runString txt | (Left err) = putStrLn ("JSON error: " ++ err) >> exitWith (ExitFailure (pos 1)) +runString txt | (Right value) = runJSON value + +main : IO ⊤ +main = getContents >>= runString + diff --git a/prototyping/Luau/Addr.agda b/prototyping/Luau/Addr.agda new file mode 100644 index 00000000..c1978047 --- /dev/null +++ b/prototyping/Luau/Addr.agda @@ -0,0 +1,17 @@ +module Luau.Addr where + +open import Agda.Builtin.Bool using (true; false) +open import Agda.Builtin.Equality using (_≡_) +open import Agda.Builtin.Nat using (Nat; _==_) +open import Agda.Builtin.String using (String) +open import Agda.Builtin.TrustMe using (primTrustMe) +open import Properties.Dec using (Dec; yes; no; ⊥) + +Addr : Set +Addr = Nat + +_≡ᴬ_ : (a b : Addr) → Dec (a ≡ b) +a ≡ᴬ b with a == b +a ≡ᴬ b | false = no p where postulate p : (a ≡ b) → ⊥ +a ≡ᴬ b | true = yes primTrustMe + diff --git a/prototyping/Luau/Addr/ToString.agda b/prototyping/Luau/Addr/ToString.agda new file mode 100644 index 00000000..2fc38335 --- /dev/null +++ b/prototyping/Luau/Addr/ToString.agda @@ -0,0 +1,8 @@ +module Luau.Addr.ToString where + +open import Agda.Builtin.String using (String; primStringAppend) +open import Luau.Addr using (Addr) +open import Agda.Builtin.Int using (Int; primShowInteger; pos) + +addrToString : Addr → String +addrToString a = primStringAppend "a" (primShowInteger (pos a)) diff --git a/prototyping/Luau/Heap.agda b/prototyping/Luau/Heap.agda new file mode 100644 index 00000000..1a0416e4 --- /dev/null +++ b/prototyping/Luau/Heap.agda @@ -0,0 +1,48 @@ +module Luau.Heap where + +open import Agda.Builtin.Equality using (_≡_) +open import FFI.Data.Maybe using (Maybe; just) +open import FFI.Data.Vector using (Vector; length; snoc; empty) +open import Luau.Addr using (Addr) +open import Luau.Var using (Var) +open import Luau.Syntax using (Block; Expr; nil; addr; function⟨_⟩_end) + +data HeapValue : Set where + function_⟨_⟩_end : Var → Var → Block → HeapValue + +Heap = Vector HeapValue + +data _≡_⊕_↦_ : Heap → Heap → Addr → HeapValue → Set where + + defn : ∀ {H val} → + + ----------------------------------- + (snoc H val) ≡ H ⊕ (length H) ↦ val + +lookup : Heap → Addr → Maybe HeapValue +lookup = FFI.Data.Vector.lookup + +emp : Heap +emp = empty + +data AllocResult (H : Heap) (V : HeapValue) : Set where + ok : ∀ a H′ → (H′ ≡ H ⊕ a ↦ V) → AllocResult H V + +alloc : ∀ H V → AllocResult H V +alloc H V = ok (length H) (snoc H V) defn + +next : Heap → Addr +next = length + +allocated : Heap → HeapValue → Heap +allocated = snoc + +-- next-emp : (length empty ≡ 0) +next-emp = FFI.Data.Vector.length-empty + +-- lookup-next : ∀ V H → (lookup (allocated H V) (next H) ≡ just V) +lookup-next = FFI.Data.Vector.lookup-snoc + +-- lookup-next-emp : ∀ V → (lookup (allocated emp V) 0 ≡ just V) +lookup-next-emp = FFI.Data.Vector.lookup-snoc-empty + diff --git a/prototyping/Luau/OpSem.agda b/prototyping/Luau/OpSem.agda new file mode 100644 index 00000000..dcd474bf --- /dev/null +++ b/prototyping/Luau/OpSem.agda @@ -0,0 +1,92 @@ +module Luau.OpSem where + +open import Agda.Builtin.Equality using (_≡_) +open import FFI.Data.Maybe using (just) +open import Luau.Heap using (Heap; _≡_⊕_↦_; lookup; function_⟨_⟩_end) +open import Luau.Substitution using (_[_/_]ᴮ) +open import Luau.Syntax using (Expr; Stat; Block; nil; addr; var; function⟨_⟩_end; _$_; block_is_end; local_←_; _∙_; done; function_⟨_⟩_end; return) +open import Luau.Value using (addr; val) + +data _⊢_⟶ᴮ_⊣_ : Heap → Block → Block → Heap → Set +data _⊢_⟶ᴱ_⊣_ : Heap → Expr → Expr → Heap → Set + +data _⊢_⟶ᴱ_⊣_ where + + nil : ∀ {H} → + + ------------------- + H ⊢ nil ⟶ᴱ nil ⊣ H + + function : ∀ {H H′ a x B} → + + H′ ≡ H ⊕ a ↦ (function "anon" ⟨ x ⟩ B end) → + ------------------------------------------- + H ⊢ (function⟨ x ⟩ B end) ⟶ᴱ (addr a) ⊣ H′ + + app : ∀ {H H′ M M′ N} → + + H ⊢ M ⟶ᴱ M′ ⊣ H′ → + ----------------------------- + H ⊢ (M $ N) ⟶ᴱ (M′ $ N) ⊣ H′ + + beta : ∀ {H M a f x B} → + + (lookup H a) ≡ just(function f ⟨ x ⟩ B end) → + ----------------------------------------------------- + H ⊢ (addr a $ M) ⟶ᴱ (block f is local x ← M ∙ B end) ⊣ H + + block : ∀ {H H′ B B′ b} → + + H ⊢ B ⟶ᴮ B′ ⊣ H′ → + ---------------------------------------------------- + H ⊢ (block b is B end) ⟶ᴱ (block b is B′ end) ⊣ H′ + + return : ∀ {H V B b} → + + -------------------------------------------------------- + H ⊢ (block b is return (val V) ∙ B end) ⟶ᴱ (val V) ⊣ H + + done : ∀ {H b} → + + --------------------------------- + H ⊢ (block b is done end) ⟶ᴱ nil ⊣ H + +data _⊢_⟶ᴮ_⊣_ where + + local : ∀ {H H′ x M M′ B} → + + H ⊢ M ⟶ᴱ M′ ⊣ H′ → + ------------------------------------------------- + H ⊢ (local x ← M ∙ B) ⟶ᴮ (local x ← M′ ∙ B) ⊣ H′ + + subst : ∀ {H x v B} → + + ------------------------------------------------- + H ⊢ (local x ← val v ∙ B) ⟶ᴮ (B [ v / x ]ᴮ) ⊣ H + + function : ∀ {H H′ a f x B C} → + + H′ ≡ H ⊕ a ↦ (function f ⟨ x ⟩ C end) → + -------------------------------------------------------------- + H ⊢ (function f ⟨ x ⟩ C end ∙ B) ⟶ᴮ (B [ addr a / f ]ᴮ) ⊣ H′ + + return : ∀ {H H′ M M′ B} → + + H ⊢ M ⟶ᴱ M′ ⊣ H′ → + -------------------------------------------- + H ⊢ (return M ∙ B) ⟶ᴮ (return M′ ∙ B) ⊣ H′ + +data _⊢_⟶*_⊣_ : Heap → Block → Block → Heap → Set where + + refl : ∀ {H B} → + + ---------------- + H ⊢ B ⟶* B ⊣ H + + step : ∀ {H H′ H″ B B′ B″} → + H ⊢ B ⟶ᴮ B′ ⊣ H′ → + H′ ⊢ B′ ⟶* B″ ⊣ H″ → + ------------------ + H ⊢ B ⟶* B″ ⊣ H″ + + diff --git a/prototyping/Luau/Run.agda b/prototyping/Luau/Run.agda new file mode 100644 index 00000000..29709ceb --- /dev/null +++ b/prototyping/Luau/Run.agda @@ -0,0 +1,28 @@ +module Luau.Run where + +open import Agda.Builtin.Equality using (_≡_; refl) +open import Luau.Heap using (Heap; emp) +open import Luau.Syntax using (Block; return; _∙_; done) +open import Luau.OpSem using (_⊢_⟶*_⊣_; refl; step) +open import Luau.Value using (val) +open import Properties.Step using (stepᴮ; step; return; done; error) +open import Luau.RuntimeError using (RuntimeErrorᴮ) + +data RunResult (H : Heap) (B : Block) : Set where + return : ∀ V {B′ H′} → (H ⊢ B ⟶* (return (val V) ∙ B′) ⊣ H′) → RunResult H B + done : ∀ {H′} → (H ⊢ B ⟶* done ⊣ H′) → RunResult H B + error : ∀ {B′ H′} → (RuntimeErrorᴮ H′ B′) → (H ⊢ B ⟶* B′ ⊣ H′) → RunResult H B + +{-# TERMINATING #-} +run′ : ∀ H B → RunResult H B +run′ H B with stepᴮ H B +run′ H B | step H′ B′ D with run′ H′ B′ +run′ H B | step H′ B′ D | return V D′ = return V (step D D′) +run′ H B | step H′ B′ D | done D′ = done (step D D′) +run′ H B | step H′ B′ D | error E D′ = error E (step D D′) +run′ H _ | return V refl = return V refl +run′ H _ | done refl = done refl +run′ H B | error E = error E refl + +run : ∀ B → RunResult emp B +run = run′ emp diff --git a/prototyping/Luau/RuntimeError.agda b/prototyping/Luau/RuntimeError.agda new file mode 100644 index 00000000..86e7cf21 --- /dev/null +++ b/prototyping/Luau/RuntimeError.agda @@ -0,0 +1,21 @@ +module Luau.RuntimeError where + +open import Agda.Builtin.Equality using (_≡_) +open import Luau.Heap using (Heap; lookup) +open import FFI.Data.Maybe using (just; nothing) +open import Luau.Syntax using (Block; Expr; nil; var; addr; function⟨_⟩_end; block_is_end; _$_; local_←_; function_⟨_⟩_end; return; done; _∙_) + +data RuntimeErrorᴮ (H : Heap) : Block → Set +data RuntimeErrorᴱ (H : Heap) : Expr → Set + +data RuntimeErrorᴱ H where + NilIsNotAFunction : ∀ {M} → RuntimeErrorᴱ H (nil $ M) + UnboundVariable : ∀ x → RuntimeErrorᴱ H (var x) + SEGV : ∀ a → (lookup H a ≡ nothing) → RuntimeErrorᴱ H (addr a) + app : ∀ {M N} → RuntimeErrorᴱ H M → RuntimeErrorᴱ H (M $ N) + block : ∀ b {B} → RuntimeErrorᴮ H B → RuntimeErrorᴱ H (block b is B end) + +data RuntimeErrorᴮ H where + local : ∀ x {M B} → RuntimeErrorᴱ H M → RuntimeErrorᴮ H (local x ← M ∙ B) + return : ∀ {M B} → RuntimeErrorᴱ H M → RuntimeErrorᴮ H (return M ∙ B) + diff --git a/prototyping/Luau/RuntimeError/ToString.agda b/prototyping/Luau/RuntimeError/ToString.agda new file mode 100644 index 00000000..f11756f7 --- /dev/null +++ b/prototyping/Luau/RuntimeError/ToString.agda @@ -0,0 +1,18 @@ +module Luau.RuntimeError.ToString where + +open import FFI.Data.String using (String; _++_) +open import Luau.RuntimeError using (RuntimeErrorᴮ; RuntimeErrorᴱ; local; return; NilIsNotAFunction; UnboundVariable; SEGV; app; block) +open import Luau.Addr.ToString using (addrToString) +open import Luau.Var.ToString using (varToString) + +errToStringᴱ : ∀ {H B} → RuntimeErrorᴱ H B → String +errToStringᴮ : ∀ {H B} → RuntimeErrorᴮ H B → String + +errToStringᴱ NilIsNotAFunction = "nil is not a function" +errToStringᴱ (UnboundVariable x) = "variable " ++ varToString x ++ " is unbound" +errToStringᴱ (SEGV a x) = "address " ++ addrToString a ++ " is unallocated" +errToStringᴱ (app E) = errToStringᴱ E +errToStringᴱ (block b E) = errToStringᴮ E ++ "\n in call of function " ++ varToString b + +errToStringᴮ (local x E) = errToStringᴱ E ++ "\n in definition of " ++ varToString x +errToStringᴮ (return E) = errToStringᴱ E ++ "\n in return statement" diff --git a/prototyping/Luau/Substitution.agda b/prototyping/Luau/Substitution.agda new file mode 100644 index 00000000..1a361174 --- /dev/null +++ b/prototyping/Luau/Substitution.agda @@ -0,0 +1,30 @@ +module Luau.Substitution where + +open import Luau.Syntax using (Expr; Stat; Block; nil; addr; var; function⟨_⟩_end; _$_; block_is_end; local_←_; _∙_; done; function_⟨_⟩_end; return) +open import Luau.Value using (Value; val) +open import Luau.Var using (Var; _≡ⱽ_) +open import Properties.Dec using (Dec; yes; no) + +_[_/_]ᴱ : Expr → Value → Var → Expr +_[_/_]ᴮ : Block → Value → Var → Block +var_[_/_]ᴱwhenever_ : ∀ {P} → Var → Value → Var → (Dec P) → Expr +_[_/_]ᴮunless_ : ∀ {P} → Block → Value → Var → (Dec P) → Block + +nil [ v / x ]ᴱ = nil +var y [ v / x ]ᴱ = var y [ v / x ]ᴱwhenever (x ≡ⱽ y) +addr a [ v / x ]ᴱ = addr a +(M $ N) [ v / x ]ᴱ = (M [ v / x ]ᴱ) $ (N [ v / x ]ᴱ) +function⟨ y ⟩ C end [ v / x ]ᴱ = function⟨ y ⟩ C [ v / x ]ᴮunless (x ≡ⱽ y) end +block b is C end [ v / x ]ᴱ = block b is C [ v / x ]ᴮ end + +(function f ⟨ y ⟩ C end ∙ B) [ v / x ]ᴮ = function f ⟨ y ⟩ (C [ v / x ]ᴮunless (x ≡ⱽ y)) end ∙ (B [ v / x ]ᴮunless (x ≡ⱽ f)) +(local y ← M ∙ B) [ v / x ]ᴮ = local y ← (M [ v / x ]ᴱ) ∙ (B [ v / x ]ᴮunless (x ≡ⱽ y)) +(return M ∙ B) [ v / x ]ᴮ = return (M [ v / x ]ᴱ) ∙ (B [ v / x ]ᴮ) +done [ v / x ]ᴮ = done + +var y [ v / x ]ᴱwhenever yes p = val v +var y [ v / x ]ᴱwhenever no p = var y + +B [ v / x ]ᴮunless yes p = B +B [ v / x ]ᴮunless no p = B [ v / x ]ᴮ + diff --git a/prototyping/Luau/Syntax.agda b/prototyping/Luau/Syntax.agda index c1d9c951..3fe05bc8 100644 --- a/prototyping/Luau/Syntax.agda +++ b/prototyping/Luau/Syntax.agda @@ -1,27 +1,17 @@ module Luau.Syntax where -open import Agda.Builtin.String using (String) +open import Luau.Var using (Var) +open import Luau.Addr using (Addr) infixr 5 _∙_ -data Type : Set where - nil : Type - _⇒_ : Type → Type → Type - none : Type - any : Type - _∪_ : Type → Type → Type - _∩_ : Type → Type → Type - -Var : Set -Var = String - data Block : Set data Stat : Set data Expr : Set data Block where _∙_ : Stat → Block → Block - _∙ : Stat → Block + done : Block data Stat where function_⟨_⟩_end : Var → Var → Block → Stat @@ -31,5 +21,7 @@ data Stat where data Expr where nil : Expr var : Var → Expr + addr : Addr → Expr _$_ : Expr → Expr → Expr function⟨_⟩_end : Var → Block → Expr + block_is_end : Var → Block → Expr diff --git a/prototyping/Luau/Syntax/FromJSON.agda b/prototyping/Luau/Syntax/FromJSON.agda index 36432118..0a4164fb 100644 --- a/prototyping/Luau/Syntax/FromJSON.agda +++ b/prototyping/Luau/Syntax/FromJSON.agda @@ -1,6 +1,6 @@ module Luau.Syntax.FromJSON where -open import Luau.Syntax using (Type; Block; Stat ; Expr; nil; _$_; var; function⟨_⟩_end; local_←_; function_⟨_⟩_end; return; _∙; _∙_) +open import Luau.Syntax using (Block; Stat ; Expr; nil; _$_; var; function⟨_⟩_end; local_←_; function_⟨_⟩_end; return; done; _∙_) open import Agda.Builtin.List using (List; _∷_; []) @@ -117,11 +117,9 @@ blockFromJSON (object obj) | nothing | _ = Left "AstStatBlock missing type" blockFromJSON _ = Left "AstBlock not an array or AstStatBlock object" blockFromArray arr with head arr -blockFromArray arr | nothing = Left "Block should be a non-empty array" +blockFromArray arr | nothing = Right done blockFromArray arr | just value with statFromJSON value blockFromArray arr | just value | Left err = Left err -blockFromArray arr | just value | Right S with null (tail arr) -blockFromArray arr | just value | Right S | true = Right (S ∙) -blockFromArray arr | just value | Right S | false with blockFromArray(tail arr) -blockFromArray arr | just value | Right S | false | Left err = Left (err) -blockFromArray arr | just value | Right S | false | Right B = Right (S ∙ B) +blockFromArray arr | just value | Right S with blockFromArray(tail arr) +blockFromArray arr | just value | Right S | Left err = Left (err) +blockFromArray arr | just value | Right S | Right B = Right (S ∙ B) diff --git a/prototyping/Luau/Syntax/ToString.agda b/prototyping/Luau/Syntax/ToString.agda index 87876565..afec0935 100644 --- a/prototyping/Luau/Syntax/ToString.agda +++ b/prototyping/Luau/Syntax/ToString.agda @@ -1,8 +1,9 @@ module Luau.Syntax.ToString where -open import Luau.Syntax using (Type; Block; Stat; Expr; nil; var; _$_; function⟨_⟩_end; return; function_⟨_⟩_end ;local_←_; _∙_; _∙) - +open import Luau.Syntax using (Block; Stat; Expr; nil; var; addr; _$_; function⟨_⟩_end; return; function_⟨_⟩_end ;local_←_; _∙_; done; block_is_end) open import FFI.Data.String using (String; _++_) +open import Luau.Addr.ToString using (addrToString) +open import Luau.Var.ToString using (varToString) exprToString′ : String → Expr → String statToString′ : String → Stat → String @@ -10,14 +11,20 @@ blockToString′ : String → Block → String exprToString′ lb nil = "nil" +exprToString′ lb (addr a) = + addrToString(a) exprToString′ lb (var x) = - x + varToString(x) exprToString′ lb (M $ N) = (exprToString′ lb M) ++ "(" ++ (exprToString′ lb N) ++ ")" exprToString′ lb (function⟨ x ⟩ B end) = "function(" ++ x ++ ")" ++ lb ++ " " ++ (blockToString′ (lb ++ " ") B) ++ lb ++ "end" +exprToString′ lb (block b is B end) = + "(function " ++ b ++ "()" ++ lb ++ + " " ++ (blockToString′ (lb ++ " ") B) ++ lb ++ + "end)()" statToString′ lb (function f ⟨ x ⟩ B end) = "local function " ++ f ++ "(" ++ x ++ ")" ++ lb ++ @@ -28,8 +35,9 @@ statToString′ lb (local x ← M) = statToString′ lb (return M) = "return " ++ (exprToString′ lb M) +blockToString′ lb (S ∙ done) = statToString′ lb S blockToString′ lb (S ∙ B) = statToString′ lb S ++ lb ++ blockToString′ lb B -blockToString′ lb (S ∙) = statToString′ lb S +blockToString′ lb (done) = "" exprToString : Expr → String exprToString = exprToString′ "\n" diff --git a/prototyping/Luau/Type.agda b/prototyping/Luau/Type.agda new file mode 100644 index 00000000..8da3a985 --- /dev/null +++ b/prototyping/Luau/Type.agda @@ -0,0 +1,10 @@ +module Luau.Type where + +data Type : Set where + nil : Type + _⇒_ : Type → Type → Type + none : Type + any : Type + _∪_ : Type → Type → Type + _∩_ : Type → Type → Type + diff --git a/prototyping/Luau/Value.agda b/prototyping/Luau/Value.agda new file mode 100644 index 00000000..855e663e --- /dev/null +++ b/prototyping/Luau/Value.agda @@ -0,0 +1,15 @@ +module Luau.Value where + +open import Luau.Addr using (Addr) +open import Luau.Syntax using (Block; Expr; nil; addr; function⟨_⟩_end) +open import Luau.Var using (Var) + +data Value : Set where + nil : Value + addr : Addr → Value + +val : Value → Expr +val nil = nil +val (addr a) = addr a + + diff --git a/prototyping/Luau/Value/ToString.agda b/prototyping/Luau/Value/ToString.agda new file mode 100644 index 00000000..3cac3ee7 --- /dev/null +++ b/prototyping/Luau/Value/ToString.agda @@ -0,0 +1,10 @@ +module Luau.Value.ToString where + +open import Agda.Builtin.String using (String) +open import Luau.Value using (Value; nil; addr) +open import Luau.Addr.ToString using (addrToString) + +valueToString : Value → String +valueToString nil = "nil" +valueToString (addr a) = addrToString a + diff --git a/prototyping/Luau/Var.agda b/prototyping/Luau/Var.agda new file mode 100644 index 00000000..091ce125 --- /dev/null +++ b/prototyping/Luau/Var.agda @@ -0,0 +1,16 @@ +module Luau.Var where + +open import Agda.Builtin.Bool using (true; false) +open import Agda.Builtin.Equality using (_≡_) +open import Agda.Builtin.String using (String; primStringEquality) +open import Agda.Builtin.TrustMe using (primTrustMe) +open import Properties.Dec using (Dec; yes; no; ⊥) + +Var : Set +Var = String + +_≡ⱽ_ : (a b : Var) → Dec (a ≡ b) +a ≡ⱽ b with primStringEquality a b +a ≡ⱽ b | false = no p where postulate p : (a ≡ b) → ⊥ +a ≡ⱽ b | true = yes primTrustMe + diff --git a/prototyping/Luau/Var/ToString.agda b/prototyping/Luau/Var/ToString.agda new file mode 100644 index 00000000..10cd915b --- /dev/null +++ b/prototyping/Luau/Var/ToString.agda @@ -0,0 +1,8 @@ +module Luau.Var.ToString where + +open import Agda.Builtin.String using (String) +open import Luau.Var using (Var) + +varToString : Var → String +varToString x = x + diff --git a/prototyping/Properties.agda b/prototyping/Properties.agda new file mode 100644 index 00000000..a3c685d1 --- /dev/null +++ b/prototyping/Properties.agda @@ -0,0 +1,3 @@ +module Properties where + +import Properties.Dec diff --git a/prototyping/Properties/Dec.agda b/prototyping/Properties/Dec.agda new file mode 100644 index 00000000..d89fd754 --- /dev/null +++ b/prototyping/Properties/Dec.agda @@ -0,0 +1,8 @@ +module Properties.Dec where + +data ⊥ : Set where + +data Dec(A : Set) : Set where + yes : A → Dec A + no : (A → ⊥) → Dec A + diff --git a/prototyping/Properties/Remember.agda b/prototyping/Properties/Remember.agda new file mode 100644 index 00000000..5058d594 --- /dev/null +++ b/prototyping/Properties/Remember.agda @@ -0,0 +1,9 @@ +module Properties.Remember where + +open import Agda.Builtin.Equality using (_≡_; refl) + +data Remember {A : Set} (a : A) : Set where + _,_ : ∀ b → (a ≡ b) → Remember(a) + +remember : ∀ {A} (a : A) → Remember(a) +remember a = (a , refl) diff --git a/prototyping/Properties/Step.agda b/prototyping/Properties/Step.agda new file mode 100644 index 00000000..0eddd9fe --- /dev/null +++ b/prototyping/Properties/Step.agda @@ -0,0 +1,58 @@ +module Properties.Step where + +open import Agda.Builtin.Equality using (_≡_; refl) +open import FFI.Data.Maybe using (just; nothing) +open import Luau.Heap using (Heap; lookup; alloc; ok; function_⟨_⟩_end) +open import Luau.Syntax using (Block; Expr; nil; var; addr; function⟨_⟩_end; block_is_end; _$_; local_←_; function_⟨_⟩_end; return; done; _∙_) +open import Luau.OpSem using (_⊢_⟶ᴱ_⊣_; _⊢_⟶ᴮ_⊣_; app ; beta; function; block; return; done; local; subst) +open import Luau.RuntimeError using (RuntimeErrorᴱ; RuntimeErrorᴮ; NilIsNotAFunction; UnboundVariable; SEGV; app; block; local; return) +open import Luau.Substitution using (_[_/_]ᴮ) +open import Luau.Value using (nil; addr; val) +open import Properties.Remember using (remember; _,_) + +data StepResultᴮ (H : Heap) (B : Block) : Set +data StepResultᴱ (H : Heap) (M : Expr) : Set + +data StepResultᴮ H B where + step : ∀ H′ B′ → (H ⊢ B ⟶ᴮ B′ ⊣ H′) → StepResultᴮ H B + return : ∀ V {B′} → (B ≡ (return (val V) ∙ B′)) → StepResultᴮ H B + done : (B ≡ done) → StepResultᴮ H B + error : (RuntimeErrorᴮ H B) → StepResultᴮ H B + +data StepResultᴱ H M where + step : ∀ H′ M′ → (H ⊢ M ⟶ᴱ M′ ⊣ H′) → StepResultᴱ H M + value : ∀ V → (M ≡ val V) → StepResultᴱ H M + error : (RuntimeErrorᴱ H M) → StepResultᴱ H M + +stepᴱ : ∀ H M → StepResultᴱ H M +stepᴮ : ∀ H B → StepResultᴮ H B + +stepᴱ H nil = value nil refl +stepᴱ H (var x) = error (UnboundVariable x) +stepᴱ H (addr a) = value (addr a) refl +stepᴱ H (M $ N) with stepᴱ H M +stepᴱ H (M $ N) | step H′ M′ D = step H′ (M′ $ N) (app D) +stepᴱ H (nil $ N) | value nil refl = error NilIsNotAFunction +stepᴱ H (addr a $ N) | value (addr a) refl with remember (lookup H a) +stepᴱ H (addr a $ N) | value (addr a) refl | (nothing , p) = error (app (SEGV a p)) +stepᴱ H (addr a $ N) | value (addr a) refl | (just(function f ⟨ x ⟩ B end) , p) = step H (block f is local x ← N ∙ B end) (beta p) +stepᴱ H (M $ N) | error E = error (app E) +stepᴱ H (function⟨ x ⟩ B end) with alloc H (function "anon" ⟨ x ⟩ B end) +stepᴱ H (function⟨ x ⟩ B end) | ok a H′ p = step H′ (addr a) (function p) +stepᴱ H (block b is B end) with stepᴮ H B +stepᴱ H (block b is B end) | step H′ B′ D = step H′ (block b is B′ end) (block D) +stepᴱ H (block b is (return _ ∙ B′) end) | return V refl = step H (val V) return +stepᴱ H (block b is done end) | done refl = step H nil done +stepᴱ H (block b is B end) | error E = error (block b E) + +stepᴮ H (function f ⟨ x ⟩ C end ∙ B) with alloc H (function f ⟨ x ⟩ C end) +stepᴮ H (function f ⟨ x ⟩ C end ∙ B) | ok a H′ p = step H′ (B [ addr a / f ]ᴮ) (function p) +stepᴮ H (local x ← M ∙ B) with stepᴱ H M +stepᴮ H (local x ← M ∙ B) | step H′ M′ D = step H′ (local x ← M′ ∙ B) (local D) +stepᴮ H (local x ← _ ∙ B) | value V refl = step H (B [ V / x ]ᴮ) subst +stepᴮ H (local x ← M ∙ B) | error E = error (local x E) +stepᴮ H (return M ∙ B) with stepᴱ H M +stepᴮ H (return M ∙ B) | step H′ M′ D = step H′ (return M′ ∙ B) (return D) +stepᴮ H (return _ ∙ B) | value V refl = return V refl +stepᴮ H (return M ∙ B) | error E = error (return E) +stepᴮ H done = done refl From aecd60371be8e54470adeebb7476370b7541c7e0 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Fri, 11 Feb 2022 09:13:27 -0800 Subject: [PATCH 155/399] Update performance.md (#355) Document weak table shrinking and paged sweeper --- docs/_pages/performance.md | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/docs/_pages/performance.md b/docs/_pages/performance.md index 4e87ec5b..b4fd3a7b 100644 --- a/docs/_pages/performance.md +++ b/docs/_pages/performance.md @@ -167,3 +167,11 @@ While Luau uses an incremental garbage collector, once per each collector cycle Normally objects that have been modified after the GC marked them in an incremental mark phase need to be rescanned during atomic phase, so frequent modifications of existing tables may result in a slow atomic step. To address this, we run a "remark" step where we traverse objects that have been modified after being marked once more (incrementally); additionally, the write barrier that triggers for object modifications changes the transition logic during remark phase to reduce the probability that the object will need to be rescanned. Another source of scalability challenges is coroutines. Writes to coroutine stacks don't use a write barrier, since that's prohibitively expensive as they are too frequent. This means that coroutine stacks need to be traversed during atomic step, so applications with many coroutines suffer large atomic pauses. To address this, we implement incremental marking of coroutines: marking a coroutine makes it "inactive" and resuming a coroutine (or pushing extra objects on the coroutine stack via C API) makes it "active". Atomic step only needs to traverse active coroutines again, which reduces the cost of atomic step by effectively making coroutine collection incremental as well. + +While large tables can be a problem for incremental GC in general since currently marking a single object is indivisible, large weak tables are a unique challenge because they also need to be processed during atomic phase, and the main use case for weak tables - object caches - may result in tables with large capacity but few live objects in long-running applications that exhibit bursts of activity. To address this, weak tables in Luau can be marked as "shrinkable" by including `s` as part of `__mode` string, which results in weak tables being resized to the optimal capacity during GC. This option may result in missing keys during table iteration if the table is resized while iteration is in progress and as such is only recommended for use in specific circumstances. + +## Optimized garbage collector sweeping + +The incremental garbage collector in Luau runs three phases for each cycle: mark, atomic and sweep. Mark incrementally traverses all live objects, atomic finishes various operations that need to happen without mutator intervention (see previous section), and sweep traverses all objects in the heap, reclaiming memory used by dead objects and performing minor fixup for live objects. While objects allocated during the mark phase are traversed in the same cycle and thus may get reclaimed, objects allocated during the sweep phase are considered live. Because of this, the faster the sweep phase completes, the less garbage will accumulate; and, of course, the less time sweeping takes the less overhead there is from this phase of garbage collection on the process. + +Since sweeping traverses the whole heap, we maximize the efficiency of this traversal by allocating garbage-collected objects of the same size in 16 KB pages, and traversing each page at a time, which is otherwise known as a paged sweeper. This ensures good locality of reference as consecutively swept objects are contiugous in memory, and allows us to spend no memory for each object on sweep-related data or allocation metadata, since paged sweeper doesn't need to be able to free objects without knowing which page they are in. Compared to linked list based sweeping that Lua/LuaJIT implement, paged sweeper is 2-3x faster, and saves 16 bytes per object on 64-bit platforms. From e9bf182585e4cfc3bdf9bf71fc74ca947027b8dd Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Fri, 11 Feb 2022 10:43:14 -0800 Subject: [PATCH 156/399] Sync to upstream/release/514 --- Analysis/include/Luau/Autocomplete.h | 2 + Analysis/include/Luau/Linter.h | 1 + Analysis/include/Luau/Substitution.h | 2 +- Analysis/include/Luau/TxnLog.h | 4 +- Analysis/include/Luau/TypeInfer.h | 11 ++ Analysis/include/Luau/TypePack.h | 3 - Analysis/include/Luau/TypeVar.h | 3 - Analysis/src/EmbeddedBuiltinDefinitions.cpp | 12 +- Analysis/src/Linter.cpp | 103 ++++++++++++-- Analysis/src/Module.cpp | 28 ++-- Analysis/src/Scope.cpp | 4 + Analysis/src/TxnLog.cpp | 8 ++ Analysis/src/TypeInfer.cpp | 101 +++++++++++--- Analysis/src/TypeVar.cpp | 4 +- Analysis/src/Unifier.cpp | 125 +++++++++++------ Ast/src/Parser.cpp | 2 +- CLI/Ast.cpp | 86 ++++++++++++ CLI/Repl.cpp | 68 ++++++--- CLI/Repl.h | 7 +- CMakeLists.txt | 14 +- Compiler/src/BytecodeBuilder.cpp | 8 +- Compiler/src/Compiler.cpp | 62 ++------- Sources.cmake | 8 ++ VM/src/lbuiltins.cpp | 2 +- VM/src/lgcdebug.cpp | 4 + VM/src/lmem.cpp | 127 ++++++++++++----- VM/src/lobject.cpp | 4 +- VM/src/lvmexecute.cpp | 3 +- extern/isocline/src/isocline.c | 7 +- tests/Compiler.test.cpp | 4 - tests/Conformance.test.cpp | 2 - tests/Linter.test.cpp | 36 ++++- tests/Repl.test.cpp | 92 ++++++++++++ tests/TypeInfer.aliases.test.cpp | 61 ++++++++ tests/TypeInfer.builtins.test.cpp | 15 +- tests/TypeInfer.provisional.test.cpp | 74 +++++++++- tests/TypeInfer.refinements.test.cpp | 22 +-- tests/TypeInfer.test.cpp | 147 ++++++++++++++++++++ tests/TypeInfer.tryUnify.test.cpp | 17 +++ tests/conformance/basic.lua | 10 +- tests/conformance/debug.lua | 1 + tests/conformance/errors.lua | 16 ++- tests/conformance/gc.lua | 7 +- tests/conformance/math.lua | 1 + tests/conformance/vararg.lua | 6 + tests/conformance/vector.lua | 9 ++ tools/heapgraph.py | 33 +++-- tools/svg.py | 2 +- 48 files changed, 1100 insertions(+), 268 deletions(-) create mode 100644 CLI/Ast.cpp diff --git a/Analysis/include/Luau/Autocomplete.h b/Analysis/include/Luau/Autocomplete.h index 58534293..65b788d3 100644 --- a/Analysis/include/Luau/Autocomplete.h +++ b/Analysis/include/Luau/Autocomplete.h @@ -86,6 +86,8 @@ struct OwningAutocompleteResult }; AutocompleteResult autocomplete(Frontend& frontend, const ModuleName& moduleName, Position position, StringCompletionCallback callback); + +// Deprecated, do not use in new work. OwningAutocompleteResult autocompleteSource(Frontend& frontend, std::string_view source, Position position, StringCompletionCallback callback); } // namespace Luau diff --git a/Analysis/include/Luau/Linter.h b/Analysis/include/Luau/Linter.h index 1f7f7f9d..ec3c124d 100644 --- a/Analysis/include/Luau/Linter.h +++ b/Analysis/include/Luau/Linter.h @@ -49,6 +49,7 @@ struct LintWarning Code_DeprecatedApi = 22, Code_TableOperations = 23, Code_DuplicateCondition = 24, + Code_MisleadingAndOr = 25, Code__Count }; diff --git a/Analysis/include/Luau/Substitution.h b/Analysis/include/Luau/Substitution.h index 4f3307cd..f85b4269 100644 --- a/Analysis/include/Luau/Substitution.h +++ b/Analysis/include/Luau/Substitution.h @@ -93,7 +93,7 @@ struct Tarjan // This should never be null; ensure you initialize it before calling // substitution methods. - const TxnLog* log; + const TxnLog* log = nullptr; std::vector edgesTy; std::vector edgesTp; diff --git a/Analysis/include/Luau/TxnLog.h b/Analysis/include/Luau/TxnLog.h index 02b87374..f238e258 100644 --- a/Analysis/include/Luau/TxnLog.h +++ b/Analysis/include/Luau/TxnLog.h @@ -307,8 +307,8 @@ private: // // We can't use a DenseHashMap here because we need a non-const iterator // over the map when we concatenate. - std::unordered_map> typeVarChanges; - std::unordered_map> typePackChanges; + std::unordered_map, DenseHashPointer> typeVarChanges; + std::unordered_map, DenseHashPointer> typePackChanges; TxnLog* parent = nullptr; diff --git a/Analysis/include/Luau/TypeInfer.h b/Analysis/include/Luau/TypeInfer.h index f61ecbf5..5592fa1f 100644 --- a/Analysis/include/Luau/TypeInfer.h +++ b/Analysis/include/Luau/TypeInfer.h @@ -103,6 +103,11 @@ struct GenericTypeDefinitions std::vector genericPacks; }; +struct HashBoolNamePair +{ + size_t operator()(const std::pair& pair) const; +}; + // All TypeVars are retained via Environment::typeVars. All TypeIds // within a program are borrowed pointers into this set. struct TypeChecker @@ -411,6 +416,12 @@ public: private: int checkRecursionCount = 0; int recursionCount = 0; + + /** + * We use this to avoid doing second-pass analysis of type aliases that are duplicates. We record a pair + * (exported, name) to properly deal with the case where the two duplicates do not have the same export status. + */ + DenseHashSet, HashBoolNamePair> duplicateTypeAliases; }; // Unit test hook diff --git a/Analysis/include/Luau/TypePack.h b/Analysis/include/Luau/TypePack.h index ca588ccb..c74bad11 100644 --- a/Analysis/include/Luau/TypePack.h +++ b/Analysis/include/Luau/TypePack.h @@ -54,9 +54,6 @@ struct TypePackVar bool persistent = false; // Pointer to the type arena that allocated this type. - // Do not depend on the value of this under any circumstances. This is for - // debugging purposes only. This is only set in debug builds; it is nullptr - // in all other environments. TypeArena* owningArena = nullptr; }; diff --git a/Analysis/include/Luau/TypeVar.h b/Analysis/include/Luau/TypeVar.h index 11dc9377..8d1a9fa6 100644 --- a/Analysis/include/Luau/TypeVar.h +++ b/Analysis/include/Luau/TypeVar.h @@ -449,9 +449,6 @@ struct TypeVar final std::optional documentationSymbol; // Pointer to the type arena that allocated this type. - // Do not depend on the value of this under any circumstances. This is for - // debugging purposes only. This is only set in debug builds; it is nullptr - // in all other environments. TypeArena* owningArena = nullptr; bool operator==(const TypeVar& rhs) const; diff --git a/Analysis/src/EmbeddedBuiltinDefinitions.cpp b/Analysis/src/EmbeddedBuiltinDefinitions.cpp index 24982506..f3ef88fc 100644 --- a/Analysis/src/EmbeddedBuiltinDefinitions.cpp +++ b/Analysis/src/EmbeddedBuiltinDefinitions.cpp @@ -1,8 +1,6 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #include "Luau/BuiltinDefinitions.h" -LUAU_FASTFLAGVARIABLE(LuauFixTonumberReturnType, false) - namespace Luau { @@ -115,6 +113,7 @@ declare function gcinfo(): number declare function error(message: T, level: number?) declare function tostring(value: T): string + declare function tonumber(value: T, radix: number?): number? declare function rawequal(a: T1, b: T2): boolean declare function rawget(tab: {[K]: V}, k: K): V @@ -200,14 +199,7 @@ declare function gcinfo(): number std::string getBuiltinDefinitionSource() { - std::string result = kBuiltinDefinitionLuaSrc; - - if (FFlag::LuauFixTonumberReturnType) - result += "declare function tonumber(value: T, radix: number?): number?\n"; - else - result += "declare function tonumber(value: T, radix: number?): number\n"; - - return result; + return kBuiltinDefinitionLuaSrc; } } // namespace Luau diff --git a/Analysis/src/Linter.cpp b/Analysis/src/Linter.cpp index 57a33e93..2ba6a0fc 100644 --- a/Analysis/src/Linter.cpp +++ b/Analysis/src/Linter.cpp @@ -43,6 +43,7 @@ static const char* kWarningNames[] = { "DeprecatedApi", "TableOperations", "DuplicateCondition", + "MisleadingAndOr", }; // clang-format on @@ -2040,18 +2041,28 @@ private: const Property* prop = lookupClassProp(cty, node->index.value); if (prop && prop->deprecated) - { - if (!prop->deprecatedSuggestion.empty()) - emitWarning(*context, LintWarning::Code_DeprecatedApi, node->location, "Member '%s.%s' is deprecated, use '%s' instead", - cty->name.c_str(), node->index.value, prop->deprecatedSuggestion.c_str()); - else - emitWarning(*context, LintWarning::Code_DeprecatedApi, node->location, "Member '%s.%s' is deprecated", cty->name.c_str(), - node->index.value); - } + report(node->location, *prop, cty->name.c_str(), node->index.value); + } + else if (const TableTypeVar* tty = get(follow(*ty))) + { + auto prop = tty->props.find(node->index.value); + + if (prop != tty->props.end() && prop->second.deprecated) + report(node->location, prop->second, tty->name ? tty->name->c_str() : nullptr, node->index.value); } return true; } + + void report(const Location& location, const Property& prop, const char* container, const char* field) + { + std::string suggestion = prop.deprecatedSuggestion.empty() ? "" : format(", use '%s' instead", prop.deprecatedSuggestion.c_str()); + + if (container) + emitWarning(*context, LintWarning::Code_DeprecatedApi, location, "Member '%s.%s' is deprecated%s", container, field, suggestion.c_str()); + else + emitWarning(*context, LintWarning::Code_DeprecatedApi, location, "Member '%s' is deprecated%s", field, suggestion.c_str()); + } }; class LintTableOperations : AstVisitor @@ -2257,6 +2268,39 @@ private: return false; } + bool visit(AstExprIfElse* expr) override + { + if (!expr->falseExpr->is()) + return true; + + // if..elseif chain detected, we need to unroll it + std::vector conditions; + conditions.reserve(2); + + AstExprIfElse* head = expr; + while (head) + { + head->condition->visit(this); + head->trueExpr->visit(this); + + conditions.push_back(head->condition); + + if (head->falseExpr->is()) + { + head = head->falseExpr->as(); + continue; + } + + head->falseExpr->visit(this); + break; + } + + detectDuplicates(conditions); + + // block recursive visits so that we only analyze each chain once + return false; + } + bool visit(AstExprBinary* expr) override { if (expr->op != AstExprBinary::And && expr->op != AstExprBinary::Or) @@ -2418,6 +2462,46 @@ private: } }; +class LintMisleadingAndOr : AstVisitor +{ +public: + LUAU_NOINLINE static void process(LintContext& context) + { + LintMisleadingAndOr pass; + pass.context = &context; + + context.root->visit(&pass); + } + +private: + LintContext* context; + + bool visit(AstExprBinary* node) override + { + if (node->op != AstExprBinary::Or) + return true; + + AstExprBinary* and_ = node->left->as(); + if (!and_ || and_->op != AstExprBinary::And) + return true; + + const char* alt = nullptr; + + if (and_->right->is()) + alt = "nil"; + else if (AstExprConstantBool* c = and_->right->as(); c && c->value == false) + alt = "false"; + + if (alt) + emitWarning(*context, LintWarning::Code_MisleadingAndOr, node->location, + "The and-or expression always evaluates to the second alternative because the first alternative is %s; consider using if-then-else " + "expression instead", + alt); + + return true; + } +}; + static void fillBuiltinGlobals(LintContext& context, const AstNameTable& names, const ScopePtr& env) { ScopePtr current = env; @@ -2522,6 +2606,9 @@ std::vector lint(AstStat* root, const AstNameTable& names, const Sc if (context.warningEnabled(LintWarning::Code_DuplicateLocal)) LintDuplicateLocal::process(context); + if (context.warningEnabled(LintWarning::Code_MisleadingAndOr)) + LintMisleadingAndOr::process(context); + std::sort(context.result.begin(), context.result.end(), WarningComparator()); return context.result; diff --git a/Analysis/src/Module.cpp b/Analysis/src/Module.cpp index 4fdff8f7..817a33e9 100644 --- a/Analysis/src/Module.cpp +++ b/Analysis/src/Module.cpp @@ -12,10 +12,10 @@ #include LUAU_FASTFLAGVARIABLE(DebugLuauFreezeArena, false) -LUAU_FASTFLAGVARIABLE(DebugLuauTrackOwningArena, false) +LUAU_FASTFLAGVARIABLE(DebugLuauTrackOwningArena, false) // Remove with FFlagLuauImmutableTypes LUAU_FASTINTVARIABLE(LuauTypeCloneRecursionLimit, 300) LUAU_FASTFLAG(LuauTypeAliasDefaults) - +LUAU_FASTFLAG(LuauImmutableTypes) LUAU_FASTFLAGVARIABLE(LuauPrepopulateUnionOptionsBeforeAllocation, false) namespace Luau @@ -66,7 +66,7 @@ TypeId TypeArena::addTV(TypeVar&& tv) { TypeId allocated = typeVars.allocate(std::move(tv)); - if (FFlag::DebugLuauTrackOwningArena) + if (FFlag::DebugLuauTrackOwningArena || FFlag::LuauImmutableTypes) asMutable(allocated)->owningArena = this; return allocated; @@ -76,7 +76,7 @@ TypeId TypeArena::freshType(TypeLevel level) { TypeId allocated = typeVars.allocate(FreeTypeVar{level}); - if (FFlag::DebugLuauTrackOwningArena) + if (FFlag::DebugLuauTrackOwningArena || FFlag::LuauImmutableTypes) asMutable(allocated)->owningArena = this; return allocated; @@ -86,7 +86,7 @@ TypePackId TypeArena::addTypePack(std::initializer_list types) { TypePackId allocated = typePacks.allocate(TypePack{std::move(types)}); - if (FFlag::DebugLuauTrackOwningArena) + if (FFlag::DebugLuauTrackOwningArena || FFlag::LuauImmutableTypes) asMutable(allocated)->owningArena = this; return allocated; @@ -96,7 +96,7 @@ TypePackId TypeArena::addTypePack(std::vector types) { TypePackId allocated = typePacks.allocate(TypePack{std::move(types)}); - if (FFlag::DebugLuauTrackOwningArena) + if (FFlag::DebugLuauTrackOwningArena || FFlag::LuauImmutableTypes) asMutable(allocated)->owningArena = this; return allocated; @@ -106,7 +106,7 @@ TypePackId TypeArena::addTypePack(TypePack tp) { TypePackId allocated = typePacks.allocate(std::move(tp)); - if (FFlag::DebugLuauTrackOwningArena) + if (FFlag::DebugLuauTrackOwningArena || FFlag::LuauImmutableTypes) asMutable(allocated)->owningArena = this; return allocated; @@ -116,7 +116,7 @@ TypePackId TypeArena::addTypePack(TypePackVar tp) { TypePackId allocated = typePacks.allocate(std::move(tp)); - if (FFlag::DebugLuauTrackOwningArena) + if (FFlag::DebugLuauTrackOwningArena || FFlag::LuauImmutableTypes) asMutable(allocated)->owningArena = this; return allocated; @@ -454,8 +454,16 @@ TypeId clone(TypeId typeId, TypeArena& dest, SeenTypes& seenTypes, SeenTypePacks TypeCloner cloner{dest, typeId, seenTypes, seenTypePacks, cloneState}; Luau::visit(cloner, typeId->ty); // Mutates the storage that 'res' points into. - // TODO: Make this work when the arena of 'res' might be frozen - asMutable(res)->documentationSymbol = typeId->documentationSymbol; + if (FFlag::LuauImmutableTypes) + { + // Persistent types are not being cloned and we get the original type back which might be read-only + if (!res->persistent) + asMutable(res)->documentationSymbol = typeId->documentationSymbol; + } + else + { + asMutable(res)->documentationSymbol = typeId->documentationSymbol; + } } return res; diff --git a/Analysis/src/Scope.cpp b/Analysis/src/Scope.cpp index c30db9c2..0a362a5e 100644 --- a/Analysis/src/Scope.cpp +++ b/Analysis/src/Scope.cpp @@ -2,6 +2,8 @@ #include "Luau/Scope.h" +LUAU_FASTFLAG(LuauTwoPassAliasDefinitionFix); + namespace Luau { @@ -17,6 +19,8 @@ Scope::Scope(const ScopePtr& parent, int subLevel) , returnType(parent->returnType) , level(parent->level.incr()) { + if (FFlag::LuauTwoPassAliasDefinitionFix) + level = level.incr(); level.subLevel = subLevel; } diff --git a/Analysis/src/TxnLog.cpp b/Analysis/src/TxnLog.cpp index 0968a4c1..00067bdd 100644 --- a/Analysis/src/TxnLog.cpp +++ b/Analysis/src/TxnLog.cpp @@ -250,6 +250,10 @@ PendingTypePack* TxnLog::queue(TypePackId tp) PendingType* TxnLog::pending(TypeId ty) const { + // This function will technically work if `this` is nullptr, but this + // indicates a bug, so we explicitly assert. + LUAU_ASSERT(static_cast(this) != nullptr); + for (const TxnLog* current = this; current; current = current->parent) { if (auto it = current->typeVarChanges.find(ty); it != current->typeVarChanges.end()) @@ -261,6 +265,10 @@ PendingType* TxnLog::pending(TypeId ty) const PendingTypePack* TxnLog::pending(TypePackId tp) const { + // This function will technically work if `this` is nullptr, but this + // indicates a bug, so we explicitly assert. + LUAU_ASSERT(static_cast(this) != nullptr); + for (const TxnLog* current = this; current; current = current->parent) { if (auto it = current->typePackChanges.find(tp); it != current->typePackChanges.end()) diff --git a/Analysis/src/TypeInfer.cpp b/Analysis/src/TypeInfer.cpp index e1987937..f1c314cd 100644 --- a/Analysis/src/TypeInfer.cpp +++ b/Analysis/src/TypeInfer.cpp @@ -32,12 +32,13 @@ LUAU_FASTFLAGVARIABLE(LuauRecursiveTypeParameterRestriction, false) LUAU_FASTFLAGVARIABLE(LuauGenericFunctionsDontCacheTypeParams, false) LUAU_FASTFLAGVARIABLE(LuauIfElseBranchTypeUnion, false) LUAU_FASTFLAGVARIABLE(LuauIfElseExpectedType2, false) +LUAU_FASTFLAGVARIABLE(LuauImmutableTypes, false) LUAU_FASTFLAGVARIABLE(LuauLengthOnCompositeType, false) LUAU_FASTFLAGVARIABLE(LuauNoSealedTypeMod, false) LUAU_FASTFLAGVARIABLE(LuauQuantifyInPlace2, false) LUAU_FASTFLAGVARIABLE(LuauSealExports, false) LUAU_FASTFLAGVARIABLE(LuauSingletonTypes, false) -LUAU_FASTFLAGVARIABLE(LuauDiscriminableUnions, false) +LUAU_FASTFLAGVARIABLE(LuauDiscriminableUnions2, false) LUAU_FASTFLAGVARIABLE(LuauTypeAliasDefaults, false) LUAU_FASTFLAGVARIABLE(LuauExpectedTypesOfProperties, false) LUAU_FASTFLAGVARIABLE(LuauErrorRecoveryType, false) @@ -47,7 +48,10 @@ LUAU_FASTFLAGVARIABLE(LuauProperTypeLevels, false) LUAU_FASTFLAGVARIABLE(LuauAscribeCorrectLevelToInferredProperitesOfFreeTables, false) LUAU_FASTFLAG(LuauUnionTagMatchFix) LUAU_FASTFLAGVARIABLE(LuauUnsealedTableLiteral, false) +LUAU_FASTFLAGVARIABLE(LuauTwoPassAliasDefinitionFix, false) LUAU_FASTFLAGVARIABLE(LuauAssertStripsFalsyTypes, false) +LUAU_FASTFLAGVARIABLE(LuauReturnAnyInsteadOfICE, false) // Eventually removed as false. +LUAU_FASTFLAGVARIABLE(LuauAnotherTypeLevelFix, false) namespace Luau { @@ -213,6 +217,11 @@ static bool isMetamethod(const Name& name) name == "__metatable" || name == "__eq" || name == "__lt" || name == "__le" || name == "__mode"; } +size_t HashBoolNamePair::operator()(const std::pair& pair) const +{ + return std::hash()(pair.first) ^ std::hash()(pair.second); +} + TypeChecker::TypeChecker(ModuleResolver* resolver, InternalErrorReporter* iceHandler) : resolver(resolver) , iceHandler(iceHandler) @@ -225,6 +234,7 @@ TypeChecker::TypeChecker(ModuleResolver* resolver, InternalErrorReporter* iceHan , anyType(getSingletonTypes().anyType) , optionalNumberType(getSingletonTypes().optionalNumberType) , anyTypePack(getSingletonTypes().anyTypePack) + , duplicateTypeAliases{{false, {}}} { globalScope = std::make_shared(globalTypes.addTypePack(TypePackVar{FreeTypePack{TypeLevel{}}})); @@ -291,6 +301,9 @@ ModulePtr TypeChecker::check(const SourceModule& module, Mode mode, std::optiona unifierState.skipCacheForType.clear(); } + if (FFlag::LuauTwoPassAliasDefinitionFix) + duplicateTypeAliases.clear(); + return std::move(currentModule); } @@ -496,6 +509,9 @@ LUAU_NOINLINE void TypeChecker::checkBlockTypeAliases(const ScopePtr& scope, std { if (const auto& typealias = stat->as()) { + if (FFlag::LuauTwoPassAliasDefinitionFix && typealias->name == Parser::errorName) + continue; + auto& bindings = typealias->exported ? scope->exportedTypeBindings : scope->privateTypeBindings; Name name = typealias->name.value; @@ -1176,6 +1192,10 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatTypeAlias& typealias // Once with forwardDeclare, and once without. Name name = typealias.name.value; + // If the alias is missing a name, we can't do anything with it. Ignore it. + if (FFlag::LuauTwoPassAliasDefinitionFix && name == Parser::errorName) + return; + std::optional binding; if (auto it = scope->exportedTypeBindings.find(name); it != scope->exportedTypeBindings.end()) binding = it->second; @@ -1192,6 +1212,8 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatTypeAlias& typealias reportError(TypeError{typealias.location, DuplicateTypeDefinition{name, location}}); bindingsMap[name] = TypeFun{binding->typeParams, binding->typePackParams, errorRecoveryType(anyType)}; + if (FFlag::LuauTwoPassAliasDefinitionFix) + duplicateTypeAliases.insert({typealias.exported, name}); } else { @@ -1211,6 +1233,11 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatTypeAlias& typealias } else { + // If the first pass failed (this should mean a duplicate definition), the second pass isn't going to be + // interesting. + if (FFlag::LuauTwoPassAliasDefinitionFix && duplicateTypeAliases.find({typealias.exported, name})) + return; + if (!binding) ice("Not predeclared"); @@ -1235,7 +1262,8 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatTypeAlias& typealias if (auto ttv = getMutable(follow(ty))) { // If the table is already named and we want to rename the type function, we have to bind new alias to a copy - if (ttv->name) + // Additionally, we can't modify types that come from other modules + if (ttv->name || (FFlag::LuauImmutableTypes && follow(ty)->owningArena != ¤tModule->internalTypes)) { bool sameTys = std::equal(ttv->instantiatedTypeParams.begin(), ttv->instantiatedTypeParams.end(), binding->typeParams.begin(), binding->typeParams.end(), [](auto&& itp, auto&& tp) { @@ -1247,7 +1275,7 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatTypeAlias& typealias }); // Copy can be skipped if this is an identical alias - if (ttv->name != name || !sameTys || !sameTps) + if ((FFlag::LuauImmutableTypes && !ttv->name) || ttv->name != name || !sameTys || !sameTps) { // This is a shallow clone, original recursive links to self are not updated TableTypeVar clone = TableTypeVar{ttv->props, ttv->indexer, ttv->level, ttv->state}; @@ -1279,9 +1307,17 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatTypeAlias& typealias } } else if (auto mtv = getMutable(follow(ty))) - mtv->syntheticName = name; + { + // We can't modify types that come from other modules + if (!FFlag::LuauImmutableTypes || follow(ty)->owningArena == ¤tModule->internalTypes) + mtv->syntheticName = name; + } - unify(ty, bindingsMap[name].type, typealias.location); + TypeId& bindingType = bindingsMap[name].type; + bool ok = unify(ty, bindingType, typealias.location); + + if (FFlag::LuauTwoPassAliasDefinitionFix && ok) + bindingType = ty; } } @@ -1564,7 +1600,12 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprCa else if (auto vtp = get(retPack)) return {vtp->ty, std::move(result.predicates)}; else if (get(retPack)) - ice("Unexpected abstract type pack!", expr.location); + { + if (FFlag::LuauReturnAnyInsteadOfICE) + return {anyType, std::move(result.predicates)}; + else + ice("Unexpected abstract type pack!", expr.location); + } else ice("Unknown TypePack type!", expr.location); } @@ -1614,11 +1655,23 @@ std::optional TypeChecker::getIndexTypeFromType( tablify(type); - const PrimitiveTypeVar* primitiveType = get(type); - if (primitiveType && primitiveType->type == PrimitiveTypeVar::String) + if (FFlag::LuauDiscriminableUnions2) { - if (std::optional mtIndex = findMetatableEntry(type, "__index", location)) + if (isString(type)) + { + std::optional mtIndex = findMetatableEntry(stringType, "__index", location); + LUAU_ASSERT(mtIndex); type = *mtIndex; + } + } + else + { + const PrimitiveTypeVar* primitiveType = get(type); + if (primitiveType && primitiveType->type == PrimitiveTypeVar::String) + { + if (std::optional mtIndex = findMetatableEntry(type, "__index", location)) + type = *mtIndex; + } } if (TableTypeVar* tableType = getMutableTableType(type)) @@ -2476,7 +2529,7 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprBi auto [rhsTy, rhsPredicates] = checkExpr(innerScope, *expr.right); - return {checkBinaryOperation(FFlag::LuauDiscriminableUnions ? scope : innerScope, expr, lhsTy, rhsTy), + return {checkBinaryOperation(FFlag::LuauDiscriminableUnions2 ? scope : innerScope, expr, lhsTy, rhsTy), {AndPredicate{std::move(lhsPredicates), std::move(rhsPredicates)}}}; } else if (expr.op == AstExprBinary::Or) @@ -2489,7 +2542,7 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprBi auto [rhsTy, rhsPredicates] = checkExpr(innerScope, *expr.right); // Because of C++, I'm not sure if lhsPredicates was not moved out by the time we call checkBinaryOperation. - TypeId result = checkBinaryOperation(FFlag::LuauDiscriminableUnions ? scope : innerScope, expr, lhsTy, rhsTy, lhsPredicates); + TypeId result = checkBinaryOperation(FFlag::LuauDiscriminableUnions2 ? scope : innerScope, expr, lhsTy, rhsTy, lhsPredicates); return {result, {OrPredicate{std::move(lhsPredicates), std::move(rhsPredicates)}}}; } else if (expr.op == AstExprBinary::CompareEq || expr.op == AstExprBinary::CompareNe) @@ -2497,8 +2550,8 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprBi if (auto predicate = tryGetTypeGuardPredicate(expr)) return {booleanType, {std::move(*predicate)}}; - ExprResult lhs = checkExpr(scope, *expr.left, std::nullopt, /*forceSingleton=*/FFlag::LuauDiscriminableUnions); - ExprResult rhs = checkExpr(scope, *expr.right, std::nullopt, /*forceSingleton=*/FFlag::LuauDiscriminableUnions); + ExprResult lhs = checkExpr(scope, *expr.left, std::nullopt, /*forceSingleton=*/FFlag::LuauDiscriminableUnions2); + ExprResult rhs = checkExpr(scope, *expr.right, std::nullopt, /*forceSingleton=*/FFlag::LuauDiscriminableUnions2); PredicateVec predicates; @@ -2785,12 +2838,16 @@ TypeId TypeChecker::checkLValueBinding(const ScopePtr& scope, const AstExprIndex } else if (exprTable->state == TableState::Unsealed || exprTable->state == TableState::Free) { - TypeId resultType = freshType(scope); + TypeId resultType = freshType(FFlag::LuauAnotherTypeLevelFix ? exprTable->level : scope->level); exprTable->indexer = TableIndexer{anyIfNonstrict(indexType), anyIfNonstrict(resultType)}; return resultType; } else { + /* + * If we use [] indexing to fetch a property from a sealed table that has no indexer, we have no idea if it will + * work, so we just mint a fresh type, return that, and hope for the best. + */ TypeId resultType = freshType(scope); return resultType; } @@ -4195,6 +4252,9 @@ TypeId TypeChecker::checkRequire(const ScopePtr& scope, const ModuleInfo& module return errorRecoveryType(scope); } + if (FFlag::LuauImmutableTypes) + return *moduleType; + SeenTypes seenTypes; SeenTypePacks seenTypePacks; CloneState cloneState; @@ -4978,6 +5038,7 @@ TypeId TypeChecker::resolveType(const ScopePtr& scope, const AstType& annotation if (notEnoughParameters && hasDefaultParameters) { // 'applyTypeFunction' is used to substitute default types that reference previous generic types + applyTypeFunction.log = TxnLog::empty(); applyTypeFunction.typeArguments.clear(); applyTypeFunction.typePackArguments.clear(); applyTypeFunction.currentModule = currentModule; @@ -5445,7 +5506,7 @@ GenericTypeDefinitions TypeChecker::createGenericTypes(const ScopePtr& scope, st void TypeChecker::refineLValue(const LValue& lvalue, RefinementMap& refis, const ScopePtr& scope, TypeIdPredicate predicate) { - LUAU_ASSERT(FFlag::LuauDiscriminableUnions); + LUAU_ASSERT(FFlag::LuauDiscriminableUnions2); const LValue* target = &lvalue; std::optional key; // If set, we know we took the base of the lvalue path and should be walking down each option of the base's type. @@ -5658,7 +5719,7 @@ void TypeChecker::resolve(const TruthyPredicate& truthyP, ErrorVec& errVec, Refi return std::nullopt; }; - if (FFlag::LuauDiscriminableUnions) + if (FFlag::LuauDiscriminableUnions2) { std::optional ty = resolveLValue(refis, scope, truthyP.lvalue); if (ty && fromOr) @@ -5771,7 +5832,7 @@ void TypeChecker::resolve(const IsAPredicate& isaP, ErrorVec& errVec, Refinement return res; }; - if (FFlag::LuauDiscriminableUnions) + if (FFlag::LuauDiscriminableUnions2) { refineLValue(isaP.lvalue, refis, scope, predicate); } @@ -5846,7 +5907,7 @@ void TypeChecker::resolve(const TypeGuardPredicate& typeguardP, ErrorVec& errVec if (auto it = primitives.find(typeguardP.kind); it != primitives.end()) { - if (FFlag::LuauDiscriminableUnions) + if (FFlag::LuauDiscriminableUnions2) { refineLValue(typeguardP.lvalue, refis, scope, it->second(sense)); return; @@ -5868,7 +5929,7 @@ void TypeChecker::resolve(const TypeGuardPredicate& typeguardP, ErrorVec& errVec } auto fail = [&](const TypeErrorData& err) { - if (!FFlag::LuauDiscriminableUnions) + if (!FFlag::LuauDiscriminableUnions2) errVec.push_back(TypeError{typeguardP.location, err}); addRefinement(refis, typeguardP.lvalue, errorRecoveryType(scope)); }; @@ -5900,7 +5961,7 @@ void TypeChecker::resolve(const EqPredicate& eqP, ErrorVec& errVec, RefinementMa return {ty}; }; - if (FFlag::LuauDiscriminableUnions) + if (FFlag::LuauDiscriminableUnions2) { std::vector rhs = options(eqP.type); diff --git a/Analysis/src/TypeVar.cpp b/Analysis/src/TypeVar.cpp index 2321eafd..7e438e31 100644 --- a/Analysis/src/TypeVar.cpp +++ b/Analysis/src/TypeVar.cpp @@ -28,6 +28,7 @@ LUAU_FASTFLAGVARIABLE(LuauMetatableAreEqualRecursion, false) LUAU_FASTFLAGVARIABLE(LuauRefactorTypeVarQuestions, false) LUAU_FASTFLAG(LuauErrorRecoveryType) LUAU_FASTFLAG(LuauUnionTagMatchFix) +LUAU_FASTFLAG(LuauDiscriminableUnions2) namespace Luau { @@ -393,7 +394,8 @@ bool hasLength(TypeId ty, DenseHashSet& seen, int* recursionCount) if (seen.contains(ty)) return true; - if (isPrim(ty, PrimitiveTypeVar::String) || get(ty) || get(ty) || get(ty)) + bool isStr = FFlag::LuauDiscriminableUnions2 ? isString(ty) : isPrim(ty, PrimitiveTypeVar::String); + if (isStr || get(ty) || get(ty) || get(ty)) return true; if (auto uty = get(ty)) diff --git a/Analysis/src/Unifier.cpp b/Analysis/src/Unifier.cpp index 89e4ae23..a8ad5159 100644 --- a/Analysis/src/Unifier.cpp +++ b/Analysis/src/Unifier.cpp @@ -15,6 +15,7 @@ LUAU_FASTINT(LuauTypeInferRecursionLimit); LUAU_FASTINT(LuauTypeInferTypePackLoopLimit); LUAU_FASTFLAGVARIABLE(LuauCommittingTxnLogFreeTpPromote, false) +LUAU_FASTFLAG(LuauImmutableTypes) LUAU_FASTFLAG(LuauUseCommittingTxnLog) LUAU_FASTINTVARIABLE(LuauTypeInferIterationLimit, 2000); LUAU_FASTFLAGVARIABLE(LuauTableSubtypingVariance2, false); @@ -24,6 +25,7 @@ LUAU_FASTFLAG(LuauErrorRecoveryType); LUAU_FASTFLAG(LuauProperTypeLevels); LUAU_FASTFLAGVARIABLE(LuauUnifyPackTails, false) LUAU_FASTFLAGVARIABLE(LuauUnionTagMatchFix, false) +LUAU_FASTFLAGVARIABLE(LuauFollowWithCommittingTxnLogInAnyUnification, false) namespace Luau { @@ -32,11 +34,13 @@ struct PromoteTypeLevels { DEPRECATED_TxnLog& DEPRECATED_log; TxnLog& log; + const TypeArena* typeArena = nullptr; TypeLevel minLevel; - explicit PromoteTypeLevels(DEPRECATED_TxnLog& DEPRECATED_log, TxnLog& log, TypeLevel minLevel) + explicit PromoteTypeLevels(DEPRECATED_TxnLog& DEPRECATED_log, TxnLog& log, const TypeArena* typeArena, TypeLevel minLevel) : DEPRECATED_log(DEPRECATED_log) , log(log) + , typeArena(typeArena) , minLevel(minLevel) { } @@ -65,8 +69,12 @@ struct PromoteTypeLevels } template - bool operator()(TID, const T&) + bool operator()(TID ty, const T&) { + // Type levels of types from other modules are already global, so we don't need to promote anything inside + if (FFlag::LuauImmutableTypes && ty->owningArena != typeArena) + return false; + return true; } @@ -83,12 +91,20 @@ struct PromoteTypeLevels bool operator()(TypeId ty, const FunctionTypeVar&) { + // Type levels of types from other modules are already global, so we don't need to promote anything inside + if (FFlag::LuauImmutableTypes && ty->owningArena != typeArena) + return false; + promote(ty, FFlag::LuauUseCommittingTxnLog ? log.getMutable(ty) : getMutable(ty)); return true; } bool operator()(TypeId ty, const TableTypeVar& ttv) { + // Type levels of types from other modules are already global, so we don't need to promote anything inside + if (FFlag::LuauImmutableTypes && ty->owningArena != typeArena) + return false; + if (ttv.state != TableState::Free && ttv.state != TableState::Generic) return true; @@ -108,24 +124,33 @@ struct PromoteTypeLevels } }; -void promoteTypeLevels(DEPRECATED_TxnLog& DEPRECATED_log, TxnLog& log, TypeLevel minLevel, TypeId ty) +void promoteTypeLevels(DEPRECATED_TxnLog& DEPRECATED_log, TxnLog& log, const TypeArena* typeArena, TypeLevel minLevel, TypeId ty) { - PromoteTypeLevels ptl{DEPRECATED_log, log, minLevel}; + // Type levels of types from other modules are already global, so we don't need to promote anything inside + if (FFlag::LuauImmutableTypes && ty->owningArena != typeArena) + return; + + PromoteTypeLevels ptl{DEPRECATED_log, log, typeArena, minLevel}; DenseHashSet seen{nullptr}; visitTypeVarOnce(ty, ptl, seen); } -void promoteTypeLevels(DEPRECATED_TxnLog& DEPRECATED_log, TxnLog& log, TypeLevel minLevel, TypePackId tp) +void promoteTypeLevels(DEPRECATED_TxnLog& DEPRECATED_log, TxnLog& log, const TypeArena* typeArena, TypeLevel minLevel, TypePackId tp) { - PromoteTypeLevels ptl{DEPRECATED_log, log, minLevel}; + // Type levels of types from other modules are already global, so we don't need to promote anything inside + if (FFlag::LuauImmutableTypes && tp->owningArena != typeArena) + return; + + PromoteTypeLevels ptl{DEPRECATED_log, log, typeArena, minLevel}; DenseHashSet seen{nullptr}; visitTypeVarOnce(tp, ptl, seen); } struct SkipCacheForType { - SkipCacheForType(const DenseHashMap& skipCacheForType) + SkipCacheForType(const DenseHashMap& skipCacheForType, const TypeArena* typeArena) : skipCacheForType(skipCacheForType) + , typeArena(typeArena) { } @@ -152,6 +177,10 @@ struct SkipCacheForType bool operator()(TypeId ty, const TableTypeVar&) { + // Types from other modules don't contain mutable elements and are ok to cache + if (FFlag::LuauImmutableTypes && ty->owningArena != typeArena) + return false; + TableTypeVar& ttv = *getMutable(ty); if (ttv.boundTo) @@ -172,6 +201,10 @@ struct SkipCacheForType template bool operator()(TypeId ty, const T& t) { + // Types from other modules don't contain mutable elements and are ok to cache + if (FFlag::LuauImmutableTypes && ty->owningArena != typeArena) + return false; + const bool* prev = skipCacheForType.find(ty); if (prev && *prev) @@ -184,8 +217,12 @@ struct SkipCacheForType } template - bool operator()(TypePackId, const T&) + bool operator()(TypePackId tp, const T&) { + // Types from other modules don't contain mutable elements and are ok to cache + if (FFlag::LuauImmutableTypes && tp->owningArena != typeArena) + return false; + return true; } @@ -208,6 +245,7 @@ struct SkipCacheForType } const DenseHashMap& skipCacheForType; + const TypeArena* typeArena = nullptr; bool result = false; }; @@ -422,13 +460,13 @@ void Unifier::tryUnify_(TypeId subTy, TypeId superTy, bool isFunctionCall, bool { if (FFlag::LuauUseCommittingTxnLog) { - promoteTypeLevels(DEPRECATED_log, log, superLevel, subTy); + promoteTypeLevels(DEPRECATED_log, log, types, superLevel, subTy); log.replace(superTy, BoundTypeVar(subTy)); } else { if (FFlag::LuauProperTypeLevels) - promoteTypeLevels(DEPRECATED_log, log, superLevel, subTy); + promoteTypeLevels(DEPRECATED_log, log, types, superLevel, subTy); else if (auto subLevel = getMutableLevel(subTy)) { if (!subLevel->subsumes(superFree->level)) @@ -466,13 +504,13 @@ void Unifier::tryUnify_(TypeId subTy, TypeId superTy, bool isFunctionCall, bool { if (FFlag::LuauUseCommittingTxnLog) { - promoteTypeLevels(DEPRECATED_log, log, subLevel, superTy); + promoteTypeLevels(DEPRECATED_log, log, types, subLevel, superTy); log.replace(subTy, BoundTypeVar(superTy)); } else { if (FFlag::LuauProperTypeLevels) - promoteTypeLevels(DEPRECATED_log, log, subLevel, superTy); + promoteTypeLevels(DEPRECATED_log, log, types, subLevel, superTy); else if (auto superLevel = getMutableLevel(superTy)) { if (!superLevel->subsumes(subFree->level)) @@ -849,7 +887,7 @@ void Unifier::cacheResult(TypeId subTy, TypeId superTy) return; auto skipCacheFor = [this](TypeId ty) { - SkipCacheForType visitor{sharedState.skipCacheForType}; + SkipCacheForType visitor{sharedState.skipCacheForType, types}; visitTypeVarOnce(ty, visitor, sharedState.seenAny); sharedState.skipCacheForType[ty] = visitor.result; @@ -1637,32 +1675,35 @@ void Unifier::tryUnifyFunctions(TypeId subTy, TypeId superTy, bool isFunctionCal tryUnify_(subFunction->retType, superFunction->retType); } - if (FFlag::LuauUseCommittingTxnLog) + if (!FFlag::LuauImmutableTypes) { - if (superFunction->definition && !subFunction->definition && !subTy->persistent) + if (FFlag::LuauUseCommittingTxnLog) { - PendingType* newSubTy = log.queue(subTy); - FunctionTypeVar* newSubFtv = getMutable(newSubTy); - LUAU_ASSERT(newSubFtv); - newSubFtv->definition = superFunction->definition; + if (superFunction->definition && !subFunction->definition && !subTy->persistent) + { + PendingType* newSubTy = log.queue(subTy); + FunctionTypeVar* newSubFtv = getMutable(newSubTy); + LUAU_ASSERT(newSubFtv); + newSubFtv->definition = superFunction->definition; + } + else if (!superFunction->definition && subFunction->definition && !superTy->persistent) + { + PendingType* newSuperTy = log.queue(superTy); + FunctionTypeVar* newSuperFtv = getMutable(newSuperTy); + LUAU_ASSERT(newSuperFtv); + newSuperFtv->definition = subFunction->definition; + } } - else if (!superFunction->definition && subFunction->definition && !superTy->persistent) + else { - PendingType* newSuperTy = log.queue(superTy); - FunctionTypeVar* newSuperFtv = getMutable(newSuperTy); - LUAU_ASSERT(newSuperFtv); - newSuperFtv->definition = subFunction->definition; - } - } - else - { - if (superFunction->definition && !subFunction->definition && !subTy->persistent) - { - subFunction->definition = superFunction->definition; - } - else if (!superFunction->definition && subFunction->definition && !superTy->persistent) - { - superFunction->definition = subFunction->definition; + if (superFunction->definition && !subFunction->definition && !subTy->persistent) + { + subFunction->definition = superFunction->definition; + } + else if (!superFunction->definition && subFunction->definition && !superTy->persistent) + { + superFunction->definition = subFunction->definition; + } } } @@ -2631,7 +2672,7 @@ static void queueTypePack(std::vector& queue, DenseHashSet& { while (true) { - a = follow(a); + a = FFlag::LuauFollowWithCommittingTxnLogInAnyUnification ? state.log.follow(a) : follow(a); if (seenTypePacks.find(a)) break; @@ -2738,7 +2779,7 @@ void Unifier::tryUnifyVariadics(TypePackId subTp, TypePackId superTp, bool rever } static void tryUnifyWithAny(std::vector& queue, Unifier& state, DenseHashSet& seen, DenseHashSet& seenTypePacks, - TypeId anyType, TypePackId anyTypePack) + const TypeArena* typeArena, TypeId anyType, TypePackId anyTypePack) { while (!queue.empty()) { @@ -2746,8 +2787,14 @@ static void tryUnifyWithAny(std::vector& queue, Unifier& state, DenseHas { TypeId ty = state.log.follow(queue.back()); queue.pop_back(); + + // Types from other modules don't have free types + if (FFlag::LuauImmutableTypes && ty->owningArena != typeArena) + continue; + if (seen.find(ty)) continue; + seen.insert(ty); if (state.log.getMutable(ty)) @@ -2853,7 +2900,7 @@ void Unifier::tryUnifyWithAny(TypeId subTy, TypeId anyTy) sharedState.tempSeenTy.clear(); sharedState.tempSeenTp.clear(); - Luau::tryUnifyWithAny(queue, *this, sharedState.tempSeenTy, sharedState.tempSeenTp, getSingletonTypes().anyType, anyTP); + Luau::tryUnifyWithAny(queue, *this, sharedState.tempSeenTy, sharedState.tempSeenTp, types, getSingletonTypes().anyType, anyTP); } void Unifier::tryUnifyWithAny(TypePackId subTy, TypePackId anyTp) @@ -2869,7 +2916,7 @@ void Unifier::tryUnifyWithAny(TypePackId subTy, TypePackId anyTp) queueTypePack(queue, sharedState.tempSeenTp, *this, subTy, anyTp); - Luau::tryUnifyWithAny(queue, *this, sharedState.tempSeenTy, sharedState.tempSeenTp, anyTy, anyTp); + Luau::tryUnifyWithAny(queue, *this, sharedState.tempSeenTy, sharedState.tempSeenTp, types, anyTy, anyTp); } std::optional Unifier::findTablePropertyRespectingMeta(TypeId lhsType, Name name) diff --git a/Ast/src/Parser.cpp b/Ast/src/Parser.cpp index f559e2e0..30b32f91 100644 --- a/Ast/src/Parser.cpp +++ b/Ast/src/Parser.cpp @@ -1133,7 +1133,7 @@ AstTypePack* Parser::parseTypeList(TempVector& result, TempVector + +#include "Luau/Common.h" +#include "Luau/Ast.h" +#include "Luau/JsonEncoder.h" +#include "Luau/Parser.h" +#include "Luau/ParseOptions.h" + +#include "FileUtils.h" + +static void displayHelp(const char* argv0) +{ + printf("Usage: %s [file]\n", argv0); +} + +static int assertionHandler(const char* expr, const char* file, int line, const char* function) +{ + printf("%s(%d): ASSERTION FAILED: %s\n", file, line, expr); + return 1; +} + +int main(int argc, char** argv) +{ + Luau::assertHandler() = assertionHandler; + + for (Luau::FValue* flag = Luau::FValue::list; flag; flag = flag->next) + if (strncmp(flag->name, "Luau", 4) == 0) + flag->value = true; + + if (argc >= 2 && strcmp(argv[1], "--help") == 0) + { + displayHelp(argv[0]); + return 0; + } + else if (argc < 2) + { + displayHelp(argv[0]); + return 1; + } + + const char* name = argv[1]; + std::optional maybeSource = std::nullopt; + if (strcmp(name, "-") == 0) + { + maybeSource = readStdin(); + } + else + { + maybeSource = readFile(name); + } + + if (!maybeSource) + { + fprintf(stderr, "Couldn't read source %s\n", name); + return 1; + } + + std::string source = *maybeSource; + + Luau::Allocator allocator; + Luau::AstNameTable names(allocator); + + Luau::ParseOptions options; + options.supportContinueStatement = true; + options.allowTypeAnnotations = true; + options.allowDeclarationSyntax = true; + + Luau::ParseResult parseResult = Luau::Parser::parse(source.data(), source.size(), names, allocator, options); + + if (parseResult.errors.size() > 0) + { + fprintf(stderr, "Parse errors were encountered:\n"); + for (const Luau::ParseError& error : parseResult.errors) + { + fprintf(stderr, " %s - %s\n", toString(error.getLocation()).c_str(), error.getMessage().c_str()); + } + fprintf(stderr, "\n"); + } + + printf("%s", Luau::toJson(parseResult.root).c_str()); + + return parseResult.errors.size() > 0 ? 1 : 0; +} + + diff --git a/CLI/Repl.cpp b/CLI/Repl.cpp index 5af6b508..9a6e25c2 100644 --- a/CLI/Repl.cpp +++ b/CLI/Repl.cpp @@ -1,4 +1,6 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +#include "Repl.h" + #include "lua.h" #include "lualib.h" @@ -38,13 +40,14 @@ enum class CompileFormat struct GlobalOptions { int optimizationLevel = 1; + int debugLevel = 1; } globalOptions; static Luau::CompileOptions copts() { Luau::CompileOptions result = {}; result.optimizationLevel = globalOptions.optimizationLevel; - result.debugLevel = 1; + result.debugLevel = globalOptions.debugLevel; result.coverageLevel = coverageActive() ? 2 : 0; return result; @@ -240,9 +243,8 @@ std::string runCode(lua_State* L, const std::string& source) return std::string(); } -static void completeIndexer(ic_completion_env_t* cenv, const char* editBuffer) +static void completeIndexer(lua_State* L, const std::string& editBuffer, const AddCompletionCallback& addCompletionCallback) { - auto* L = reinterpret_cast(ic_completion_arg(cenv)); std::string_view lookup = editBuffer; char lastSep = 0; @@ -276,7 +278,7 @@ static void completeIndexer(ic_completion_env_t* cenv, const char* editBuffer) // Add an opening paren for function calls by default. completion += "("; } - ic_add_completion_ex(cenv, completion.data(), key.data(), nullptr); + addCompletionCallback(completion, std::string(key)); } } lua_pop(L, 1); @@ -295,10 +297,11 @@ static void completeIndexer(ic_completion_env_t* cenv, const char* editBuffer) { // Replace the string object with the string class to perform further lookups of string functions // Note: We retrieve the string class from _G to prevent issues if the user assigns to `string`. + lua_pop(L, 1); // Pop the string instance lua_getglobal(L, "_G"); lua_pushlstring(L, "string", 6); lua_rawget(L, -2); - lua_remove(L, -2); + lua_remove(L, -2); // Remove the global table LUAU_ASSERT(lua_istable(L, -1)); } else if (!lua_istable(L, -1)) @@ -312,6 +315,26 @@ static void completeIndexer(ic_completion_env_t* cenv, const char* editBuffer) lua_pop(L, 1); } +void getCompletions(lua_State* L, const std::string& editBuffer, const AddCompletionCallback& addCompletionCallback) +{ + // look the value up in current global table first + lua_pushvalue(L, LUA_GLOBALSINDEX); + completeIndexer(L, editBuffer, addCompletionCallback); + + // and in actual global table after that + lua_getglobal(L, "_G"); + completeIndexer(L, editBuffer, addCompletionCallback); +} + +static void icGetCompletions(ic_completion_env_t* cenv, const char* editBuffer) +{ + auto* L = reinterpret_cast(ic_completion_arg(cenv)); + + getCompletions(L, std::string(editBuffer), [cenv](const std::string& completion, const std::string& display) { + ic_add_completion_ex(cenv, completion.data(), display.data(), nullptr); + }); +} + static bool isMethodOrFunctionChar(const char* s, long len) { char c = *s; @@ -320,15 +343,7 @@ static bool isMethodOrFunctionChar(const char* s, long len) static void completeRepl(ic_completion_env_t* cenv, const char* editBuffer) { - auto* L = reinterpret_cast(ic_completion_arg(cenv)); - - // look the value up in current global table first - lua_pushvalue(L, LUA_GLOBALSINDEX); - ic_complete_word(cenv, editBuffer, completeIndexer, isMethodOrFunctionChar); - - // and in actual global table after that - lua_getglobal(L, "_G"); - ic_complete_word(cenv, editBuffer, completeIndexer, isMethodOrFunctionChar); + ic_complete_word(cenv, editBuffer, icGetCompletions, isMethodOrFunctionChar); } struct LinenoiseScopedHistory @@ -372,19 +387,20 @@ static void runReplImpl(lua_State* L) for (;;) { - const char* line = ic_readline(buffer.empty() ? "" : ">"); + const char* prompt = buffer.empty() ? "" : ">"; + std::unique_ptr line(ic_readline(prompt), free); if (!line) break; - if (buffer.empty() && runCode(L, std::string("return ") + line) == std::string()) + if (buffer.empty() && runCode(L, std::string("return ") + line.get()) == std::string()) { - ic_history_add(line); + ic_history_add(line.get()); continue; } if (!buffer.empty()) buffer += "\n"; - buffer += line; + buffer += line.get(); std::string error = runCode(L, buffer); @@ -400,7 +416,6 @@ static void runReplImpl(lua_State* L) ic_history_add(buffer.c_str()); buffer.clear(); - free((void*)line); } } @@ -504,7 +519,7 @@ static bool compileFile(const char* name, CompileFormat format) if (format == CompileFormat::Text) { - bcb.setDumpFlags(Luau::BytecodeBuilder::Dump_Code | Luau::BytecodeBuilder::Dump_Source); + bcb.setDumpFlags(Luau::BytecodeBuilder::Dump_Code | Luau::BytecodeBuilder::Dump_Source | Luau::BytecodeBuilder::Dump_Locals); bcb.setDumpSource(*source); } @@ -549,7 +564,8 @@ static void displayHelp(const char* argv0) printf(" --coverage: collect code coverage while running the code and output results to coverage.out\n"); printf(" -h, --help: Display this usage message.\n"); printf(" -i, --interactive: Run an interactive REPL after executing the last script specified.\n"); - printf(" -O: use compiler optimization level (n=0-2).\n"); + printf(" -O: compile with optimization level n (default 1, n should be between 0 and 2).\n"); + printf(" -g: compile with debug level n (default 1, n should be between 0 and 2).\n"); printf(" --profile[=N]: profile the code using N Hz sampling (default 10000) and output results to profile.out\n"); printf(" --timetrace: record compiler time tracing information into trace.json\n"); } @@ -620,6 +636,16 @@ int replMain(int argc, char** argv) } globalOptions.optimizationLevel = level; } + else if (strncmp(argv[i], "-g", 2) == 0) + { + int level = atoi(argv[i] + 2); + if (level < 0 || level > 2) + { + fprintf(stderr, "Error: Debug level must be between 0 and 2 inclusive.\n"); + return 1; + } + globalOptions.debugLevel = level; + } else if (strcmp(argv[i], "--profile") == 0) { profile = 10000; // default to 10 KHz diff --git a/CLI/Repl.h b/CLI/Repl.h index 11a077ae..cd54b7e0 100644 --- a/CLI/Repl.h +++ b/CLI/Repl.h @@ -3,10 +3,15 @@ #include "lua.h" +#include #include +using AddCompletionCallback = std::function; + // Note: These are internal functions which are being exposed in a header // so they can be included by unit tests. -int replMain(int argc, char** argv); void setupState(lua_State* L); std::string runCode(lua_State* L, const std::string& source); +void getCompletions(lua_State* L, const std::string& editBuffer, const AddCompletionCallback& addCompletionCallback); + +int replMain(int argc, char** argv); diff --git a/CMakeLists.txt b/CMakeLists.txt index 881d3c3f..c19d2b40 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -5,13 +5,20 @@ if(EXT_PLATFORM_STRING) endif() cmake_minimum_required(VERSION 3.0) -project(Luau LANGUAGES CXX C) option(LUAU_BUILD_CLI "Build CLI" ON) option(LUAU_BUILD_TESTS "Build tests" ON) option(LUAU_BUILD_WEB "Build Web module" OFF) option(LUAU_WERROR "Warnings as errors" OFF) +option(LUAU_STATIC_CRT "Link with the static CRT (/MT)" OFF) +if(LUAU_STATIC_CRT) + cmake_minimum_required(VERSION 3.15) + cmake_policy(SET CMP0091 NEW) + set(CMAKE_MSVC_RUNTIME_LIBRARY "MultiThreaded$<$:Debug>") +endif() + +project(Luau LANGUAGES CXX C) add_library(Luau.Ast STATIC) add_library(Luau.Compiler STATIC) add_library(Luau.Analysis STATIC) @@ -21,10 +28,12 @@ add_library(isocline STATIC) if(LUAU_BUILD_CLI) add_executable(Luau.Repl.CLI) add_executable(Luau.Analyze.CLI) + add_executable(Luau.Ast.CLI) # This also adds target `name` on Linux/macOS and `name.exe` on Windows set_target_properties(Luau.Repl.CLI PROPERTIES OUTPUT_NAME luau) set_target_properties(Luau.Analyze.CLI PROPERTIES OUTPUT_NAME luau-analyze) + set_target_properties(Luau.Ast.CLI PROPERTIES OUTPUT_NAME luau-ast) endif() if(LUAU_BUILD_TESTS) @@ -98,6 +107,7 @@ endif() if(LUAU_BUILD_CLI) target_compile_options(Luau.Repl.CLI PRIVATE ${LUAU_OPTIONS}) target_compile_options(Luau.Analyze.CLI PRIVATE ${LUAU_OPTIONS}) + target_compile_options(Luau.Ast.CLI PRIVATE ${LUAU_OPTIONS}) target_include_directories(Luau.Repl.CLI PRIVATE extern extern/isocline/include) @@ -111,6 +121,8 @@ if(LUAU_BUILD_CLI) endif() target_link_libraries(Luau.Analyze.CLI PRIVATE Luau.Analysis) + + target_link_libraries(Luau.Ast.CLI PRIVATE Luau.Ast Luau.Analysis) endif() if(LUAU_BUILD_TESTS) diff --git a/Compiler/src/BytecodeBuilder.cpp b/Compiler/src/BytecodeBuilder.cpp index e6d02454..09f06b68 100644 --- a/Compiler/src/BytecodeBuilder.cpp +++ b/Compiler/src/BytecodeBuilder.cpp @@ -6,8 +6,6 @@ #include #include -LUAU_FASTFLAGVARIABLE(LuauBytecodeV2Write, false) - namespace Luau { @@ -510,7 +508,7 @@ uint32_t BytecodeBuilder::getDebugPC() const void BytecodeBuilder::finalize() { LUAU_ASSERT(bytecode.empty()); - bytecode = char(FFlag::LuauBytecodeV2Write ? LBC_VERSION_FUTURE : LBC_VERSION); + bytecode = char(LBC_VERSION_FUTURE); writeStringTable(bytecode); @@ -611,9 +609,7 @@ void BytecodeBuilder::writeFunction(std::string& ss, uint32_t id) const writeVarInt(ss, child); // debug info - if (FFlag::LuauBytecodeV2Write) - writeVarInt(ss, func.debuglinedefined); - + writeVarInt(ss, func.debuglinedefined); writeVarInt(ss, func.debugname); bool hasLines = true; diff --git a/Compiler/src/Compiler.cpp b/Compiler/src/Compiler.cpp index e4253adc..656a9926 100644 --- a/Compiler/src/Compiler.cpp +++ b/Compiler/src/Compiler.cpp @@ -15,7 +15,6 @@ #include #include -LUAU_FASTFLAGVARIABLE(LuauCompileTableIndexOpt, false) LUAU_FASTFLAG(LuauCompileSelectBuiltin2) namespace Luau @@ -1182,18 +1181,9 @@ struct Compiler const AstExprTable::Item& item = expr->items.data[i]; LUAU_ASSERT(item.key); // no list portion => all items have keys - if (FFlag::LuauCompileTableIndexOpt) - { - const Constant* ckey = constants.find(item.key); + const Constant* ckey = constants.find(item.key); - indexSize += (ckey && ckey->type == Constant::Type_Number && ckey->valueNumber == double(indexSize + 1)); - } - else - { - AstExprConstantNumber* ckey = item.key->as(); - - indexSize += (ckey && ckey->value == double(indexSize + 1)); - } + indexSize += (ckey && ckey->type == Constant::Type_Number && ckey->valueNumber == double(indexSize + 1)); } // we only perform the optimization if we don't have any other []-keys @@ -1295,43 +1285,10 @@ struct Compiler { RegScope rsi(this); - if (FFlag::LuauCompileTableIndexOpt) - { - LValue lv = compileLValueIndex(reg, key, rsi); - uint8_t rv = compileExprAuto(value, rsi); + LValue lv = compileLValueIndex(reg, key, rsi); + uint8_t rv = compileExprAuto(value, rsi); - compileAssign(lv, rv); - } - else - { - // Optimization: use SETTABLEKS/SETTABLEN for literal keys, this happens often as part of usual table construction syntax - if (AstExprConstantString* ckey = key->as()) - { - BytecodeBuilder::StringRef cname = sref(ckey->value); - int32_t cid = bytecode.addConstantString(cname); - if (cid < 0) - CompileError::raise(expr->location, "Exceeded constant limit; simplify the code to compile"); - - uint8_t rv = compileExprAuto(value, rsi); - - bytecode.emitABC(LOP_SETTABLEKS, rv, reg, uint8_t(BytecodeBuilder::getStringHash(cname))); - bytecode.emitAux(cid); - } - else if (AstExprConstantNumber* ckey = key->as(); - ckey && ckey->value >= 1 && ckey->value <= 256 && double(int(ckey->value)) == ckey->value) - { - uint8_t rv = compileExprAuto(value, rsi); - - bytecode.emitABC(LOP_SETTABLEN, rv, reg, uint8_t(int(ckey->value) - 1)); - } - else - { - uint8_t rk = compileExprAuto(key, rsi); - uint8_t rv = compileExprAuto(value, rsi); - - bytecode.emitABC(LOP_SETTABLE, rv, reg, rk); - } - } + compileAssign(lv, rv); } // items without a key are set using SETLIST so that we can initialize large arrays quickly else @@ -1439,8 +1396,7 @@ struct Compiler uint8_t rt = compileExprAuto(expr->expr, rs); uint8_t i = uint8_t(int(cv->valueNumber) - 1); - if (FFlag::LuauCompileTableIndexOpt) - setDebugLine(expr->index); + setDebugLine(expr->index); bytecode.emitABC(LOP_GETTABLEN, target, rt, i); } @@ -1453,8 +1409,7 @@ struct Compiler if (cid < 0) CompileError::raise(expr->location, "Exceeded constant limit; simplify the code to compile"); - if (FFlag::LuauCompileTableIndexOpt) - setDebugLine(expr->index); + setDebugLine(expr->index); bytecode.emitABC(LOP_GETTABLEKS, target, rt, uint8_t(BytecodeBuilder::getStringHash(iname))); bytecode.emitAux(cid); @@ -1853,8 +1808,7 @@ struct Compiler void compileLValueUse(const LValue& lv, uint8_t reg, bool set) { - if (FFlag::LuauCompileTableIndexOpt) - setDebugLine(lv.location); + setDebugLine(lv.location); switch (lv.kind) { diff --git a/Sources.cmake b/Sources.cmake index b36b6db5..773f6f35 100644 --- a/Sources.cmake +++ b/Sources.cmake @@ -193,6 +193,14 @@ if(TARGET Luau.Analyze.CLI) CLI/Analyze.cpp) endif() +if(TARGET Luau.Ast.CLI) + target_sources(Luau.Ast.CLI PRIVATE + CLI/Ast.cpp + CLI/FileUtils.h + CLI/FileUtils.cpp + ) +endif() + if(TARGET Luau.UnitTest) # Luau.UnitTest Sources target_sources(Luau.UnitTest PRIVATE diff --git a/VM/src/lbuiltins.cpp b/VM/src/lbuiltins.cpp index ecc14e87..718d387d 100644 --- a/VM/src/lbuiltins.cpp +++ b/VM/src/lbuiltins.cpp @@ -1098,7 +1098,7 @@ static int luauF_select(lua_State* L, StkId res, TValue* arg0, int nresults, Stk int i = int(nvalue(arg0)); // i >= 1 && i <= n - if (unsigned(i - 1) <= unsigned(n)) + if (unsigned(i - 1) < unsigned(n)) { setobj2s(L, res, L->base - n + (i - 1)); return 1; diff --git a/VM/src/lgcdebug.cpp b/VM/src/lgcdebug.cpp index 906fb0d0..ce196520 100644 --- a/VM/src/lgcdebug.cpp +++ b/VM/src/lgcdebug.cpp @@ -250,6 +250,8 @@ void luaC_validate(lua_State* L) if (FFlag::LuauGcPagedSweep) { + validategco(L, NULL, obj2gco(g->mainthread)); + luaM_visitgco(L, L, validategco); } else @@ -565,6 +567,8 @@ void luaC_dump(lua_State* L, void* file, const char* (*categoryName)(lua_State* if (FFlag::LuauGcPagedSweep) { + dumpgco(f, NULL, obj2gco(g->mainthread)); + luaM_visitgco(L, f, dumpgco); } else diff --git a/VM/src/lmem.cpp b/VM/src/lmem.cpp index de85cf59..19617b8c 100644 --- a/VM/src/lmem.cpp +++ b/VM/src/lmem.cpp @@ -8,6 +8,76 @@ #include +/* + * Luau heap uses a size-segregated page structure, with individual pages and large allocations + * allocated using system heap (via frealloc callback). + * + * frealloc callback serves as a general, if slow, allocation callback that can allocate, free or + * resize allocations: + * + * void* frealloc(void* ud, void* ptr, size_t oldsize, size_t newsize); + * + * frealloc(ud, NULL, 0, x) creates a new block of size x + * frealloc(ud, p, x, 0) frees the block p (must return NULL) + * frealloc(ud, NULL, 0, 0) does nothing, equivalent to free(NULL) + * + * frealloc returns NULL if it cannot create or reallocate the area + * (any reallocation to an equal or smaller size cannot fail!) + * + * On top of this, Luau implements heap storage which is split into two types of allocations: + * + * - GCO, short for "garbage collected objects" + * - other objects (for example, arrays stored inside table objects) + * + * The heap layout for these two allocation types is a bit different. + * + * All GCO are allocated in pages, which is a block of memory of ~16K in size that has a page header + * (lua_Page). Each page contains 1..N blocks of the same size, where N is selected to fill the page + * completely. This amortizes the allocation cost and increases locality. Each GCO block starts with + * the GC header (GCheader) which contains the object type, mark bits and other GC metadata. If the + * GCO block is free (not used), then it must have the type set to TNIL; in this case the block can + * be part of the per-page free list, the link for that list is stored after the header (freegcolink). + * + * Importantly, the GCO block doesn't have any back references to the page it's allocated in, so it's + * impossible to free it in isolation - GCO blocks are freed by sweeping the pages they belong to, + * using luaM_freegco which must specify the page; this is called by page sweeper that traverses the + * entire page's worth of objects. For this reason it's also important that freed GCO blocks keep the + * GC header intact and accessible (with type = NIL) so that the sweeper can access it. + * + * Some GCOs are too large to fit in a 16K page without excessive fragmentation (the size threshold is + * currently 512 bytes); in this case, we allocate a dedicated small page with just a single block's worth + * storage space, but that requires allocating an extra page header. In effect large GCOs are a little bit + * less memory efficient, but this allows us to uniformly sweep small and large GCOs using page lists. + * + * All GCO pages are linked in a large intrusive linked list (global_State::allgcopages). Additionally, + * for each block size there's a page free list that contains pages that have at least one free block + * (global_State::freegcopages). This free list is used to make sure object allocation is O(1). + * + * Compared to GCOs, regular allocations have two important differences: they can be freed in isolation, + * and they don't start with a GC header. Because of this, each allocation is prefixed with block metadata, + * which contains the pointer to the page for allocated blocks, and the pointer to the next free block + * inside the page for freed blocks. + * For regular allocations that are too large to fit in a page (using the same threshold of 512 bytes), + * we don't allocate a separate page, instead simply using frealloc to allocate a vanilla block of memory. + * + * Just like GCO pages, we store a page free list (global_State::freepages) that allows O(1) allocation; + * there is no global list for non-GCO pages since we never need to traverse them directly. + * + * In both cases, we pick the page by computing the size class from the block size which rounds the block + * size up to reduce the chance that we'll allocate pages that have very few allocated blocks. The size + * class strategy is determined by SizeClassConfig constructor. + * + * Note that when the last block in a page is freed, we immediately free the page with frealloc - the + * memory manager doesn't currently attempt to keep unused memory around. This can result in excessive + * allocation traffic and can be mitigated by adding a page cache in the future. + * + * For both GCO and non-GCO pages, the per-page block allocation combines bump pointer style allocation + * (lua_Page::freeNext) and per-page free list (lua_Page::freeList). We use the bump allocator to allocate + * the contents of the page, and the free list for further reuse; this allows shorter page setup times + * which results in less variance between allocation cost, as well as tighter sweep bounds for newly + * allocated pages. + */ + LUAU_FASTFLAG(LuauGcPagedSweep) #ifndef __has_feature @@ -56,6 +126,7 @@ static_assert(offsetof(GCObject, ts) == 0, "TString data must be located at the const size_t kSizeClasses = LUA_SIZECLASSES; const size_t kMaxSmallSize = 512; const size_t kPageSize = 16 * 1024 - 24; // slightly under 16KB since that results in less fragmentation due to heap metadata + const size_t kBlockHeader = sizeof(double) > sizeof(void*) ? sizeof(double) : sizeof(void*); // suitable for aligning double & void* on all platforms const size_t kGCOLinkOffset = (sizeof(GCheader) + sizeof(void*) - 1) & ~(sizeof(void*) - 1); // GCO pages contain freelist links after the GC header @@ -107,24 +178,6 @@ const SizeClassConfig kSizeClassConfig; #define metadata(block) (*(void**)(block)) #define freegcolink(block) (*(void**)((char*)block + kGCOLinkOffset)) -/* -** About the realloc function: -** void * frealloc (void *ud, void *ptr, size_t osize, size_t nsize); -** (`osize' is the old size, `nsize' is the new size) -** -** Lua ensures that (ptr == NULL) iff (osize == 0). -** -** * frealloc(ud, NULL, 0, x) creates a new block of size `x' -** -** * frealloc(ud, p, x, 0) frees the block `p' -** (in this specific case, frealloc must return NULL). -** particularly, frealloc(ud, NULL, 0, 0) does nothing -** (which is equivalent to free(NULL) in ANSI C) -** -** frealloc returns NULL if it cannot create or reallocate the area -** (any reallocation to an equal or smaller size cannot fail!) -*/ - struct lua_Page { // list of pages with free blocks @@ -135,13 +188,12 @@ struct lua_Page lua_Page* gcolistprev; lua_Page* gcolistnext; - int busyBlocks; - int blockSize; + int pageSize; // page size in bytes, including page header + int blockSize; // block size in bytes, including block header (for non-GCO) - void* freeList; - int freeNext; - - int pageSize; + void* freeList; // next free block in this page; linked with metadata()/freegcolink() + int freeNext; // next free block offset in this page, in bytes; when negative, freeList is used instead + int busyBlocks; // number of blocks allocated out of this page union { @@ -177,7 +229,7 @@ static lua_Page* newpageold(lua_State* L, uint8_t sizeClass) page->gcolistprev = NULL; page->gcolistnext = NULL; - page->busyBlocks = 0; + page->pageSize = kPageSize; page->blockSize = blockSize; // note: we start with the last block in the page and move downward @@ -185,6 +237,7 @@ static lua_Page* newpageold(lua_State* L, uint8_t sizeClass) // additionally, GC stores objects in singly linked lists, and this way the GC lists end up in increasing pointer order page->freeList = NULL; page->freeNext = (blockCount - 1) * blockSize; + page->busyBlocks = 0; // prepend a page to page freelist (which is empty because we only ever allocate a new page when it is!) LUAU_ASSERT(!g->freepages[sizeClass]); @@ -214,7 +267,7 @@ static lua_Page* newpage(lua_State* L, lua_Page** gcopageset, int pageSize, int page->gcolistprev = NULL; page->gcolistnext = NULL; - page->busyBlocks = 0; + page->pageSize = pageSize; page->blockSize = blockSize; // note: we start with the last block in the page and move downward @@ -222,8 +275,7 @@ static lua_Page* newpage(lua_State* L, lua_Page** gcopageset, int pageSize, int // additionally, GC stores objects in singly linked lists, and this way the GC lists end up in increasing pointer order page->freeList = NULL; page->freeNext = (blockCount - 1) * blockSize; - - page->pageSize = pageSize; + page->busyBlocks = 0; if (gcopageset) { @@ -406,8 +458,7 @@ static void* newgcoblock(lua_State* L, int sizeClass) page->next = NULL; } - // the user data is right after the metadata - return (char*)block; + return block; } static void freeblock(lua_State* L, int sizeClass, void* block) @@ -421,6 +472,7 @@ static void freeblock(lua_State* L, int sizeClass, void* block) lua_Page* page = (lua_Page*)metadata(block); LUAU_ASSERT(page && page->busyBlocks > 0); LUAU_ASSERT(size_t(page->blockSize) == kSizeClassConfig.sizeOfClass[sizeClass] + kBlockHeader); + LUAU_ASSERT(block >= page->data && block < (char*)page + page->pageSize); // if the page wasn't in the page free list, it should be now since it got a block! if (!page->freeList && page->freeNext < 0) @@ -455,6 +507,9 @@ static void freeblock(lua_State* L, int sizeClass, void* block) static void freegcoblock(lua_State* L, int sizeClass, void* block, lua_Page* page) { LUAU_ASSERT(FFlag::LuauGcPagedSweep); + LUAU_ASSERT(page && page->busyBlocks > 0); + LUAU_ASSERT(page->blockSize == kSizeClassConfig.sizeOfClass[sizeClass]); + LUAU_ASSERT(block >= page->data && block < (char*)page + page->pageSize); global_State* g = L->global; @@ -575,6 +630,8 @@ void luaM_freegco_(lua_State* L, GCObject* block, size_t osize, uint8_t memcat, else { LUAU_ASSERT(page->busyBlocks == 1); + LUAU_ASSERT(size_t(page->blockSize) == osize); + LUAU_ASSERT((void*)block == page->data); freepage(L, &g->allgcopages, page); } @@ -626,8 +683,12 @@ void luaM_getpagewalkinfo(lua_Page* page, char** start, char** end, int* busyBlo int blockCount = (page->pageSize - offsetof(lua_Page, data)) / page->blockSize; - *start = page->data + page->freeNext + page->blockSize; - *end = page->data + blockCount * page->blockSize; + LUAU_ASSERT(page->freeNext >= -page->blockSize && page->freeNext <= (blockCount - 1) * page->blockSize); + + char* data = page->data; // silences ubsan when indexing page->data + + *start = data + page->freeNext + page->blockSize; + *end = data + blockCount * page->blockSize; *busyBlocks = page->busyBlocks; *blockSize = page->blockSize; } @@ -675,7 +736,7 @@ void luaM_visitgco(lua_State* L, void* context, bool (*visitor)(void* context, l for (lua_Page* curr = g->allgcopages; curr;) { - lua_Page* next = curr->gcolistnext; // page blockvisit might destroy the page + lua_Page* next = curr->gcolistnext; // block visit might destroy the page luaM_visitpage(curr, context, visitor); diff --git a/VM/src/lobject.cpp b/VM/src/lobject.cpp index 370c7b28..d5bd76a8 100644 --- a/VM/src/lobject.cpp +++ b/VM/src/lobject.cpp @@ -131,7 +131,7 @@ void luaO_chunkid(char* out, const char* source, size_t bufflen) { size_t l; source++; /* skip the `@' */ - bufflen -= sizeof(" '...' "); + bufflen -= sizeof("..."); l = strlen(source); strcpy(out, ""); if (l > bufflen) @@ -144,7 +144,7 @@ void luaO_chunkid(char* out, const char* source, size_t bufflen) else { /* out = [string "string"] */ size_t len = strcspn(source, "\n\r"); /* stop at first newline */ - bufflen -= sizeof(" [string \"...\"] "); + bufflen -= sizeof("[string \"...\"]"); if (len > bufflen) len = bufflen; strcpy(out, "[string \""); diff --git a/VM/src/lvmexecute.cpp b/VM/src/lvmexecute.cpp index cba3670a..c3b662a2 100644 --- a/VM/src/lvmexecute.cpp +++ b/VM/src/lvmexecute.cpp @@ -609,7 +609,8 @@ static void luau_execute(lua_State* L) if (unsigned(ic) < LUA_VECTOR_SIZE && name[1] == '\0') { - setnvalue(ra, rb->value.v[ic]); + const float* v = rb->value.v; // silences ubsan when indexing v[] + setnvalue(ra, v[ic]); VM_NEXT(); } diff --git a/extern/isocline/src/isocline.c b/extern/isocline/src/isocline.c index 13278062..8b6055cf 100644 --- a/extern/isocline/src/isocline.c +++ b/extern/isocline/src/isocline.c @@ -13,7 +13,12 @@ // $ gcc -c src/isocline.c //------------------------------------------------------------- #if !defined(IC_SEPARATE_OBJS) -# define _CRT_SECURE_NO_WARNINGS // for msvc +# ifndef _CRT_NONSTDC_NO_WARNINGS +# define _CRT_NONSTDC_NO_WARNINGS // for msvc +# endif +# ifndef _CRT_SECURE_NO_WARNINGS +# define _CRT_SECURE_NO_WARNINGS // for msvc +# endif # define _XOPEN_SOURCE 700 // for wcwidth # define _DEFAULT_SOURCE // ensure usleep stays visible with _XOPEN_SOURCE >= 700 # include "attr.c" diff --git a/tests/Compiler.test.cpp b/tests/Compiler.test.cpp index d8af94db..cd7a21d8 100644 --- a/tests/Compiler.test.cpp +++ b/tests/Compiler.test.cpp @@ -605,8 +605,6 @@ RETURN R0 1 TEST_CASE("TableLiteralsIndexConstant") { - ScopedFastFlag sff("LuauCompileTableIndexOpt", true); - // validate that we use SETTTABLEKS for constant variable keys CHECK_EQ("\n" + compileFunction0(R"( local a, b = "key", "value" @@ -2483,8 +2481,6 @@ return TEST_CASE("DebugLineInfoAssignment") { - ScopedFastFlag sff("LuauCompileTableIndexOpt", true); - Luau::BytecodeBuilder bcb; bcb.setDumpFlags(Luau::BytecodeBuilder::Dump_Code | Luau::BytecodeBuilder::Dump_Lines); Luau::compileOrThrow(bcb, R"( diff --git a/tests/Conformance.test.cpp b/tests/Conformance.test.cpp index e580949f..8b58d2ce 100644 --- a/tests/Conformance.test.cpp +++ b/tests/Conformance.test.cpp @@ -492,8 +492,6 @@ TEST_CASE("DateTime") TEST_CASE("Debug") { - ScopedFastFlag sffw("LuauBytecodeV2Write", true); - runConformance("debug.lua"); } diff --git a/tests/Linter.test.cpp b/tests/Linter.test.cpp index d1cc49b2..577415fc 100644 --- a/tests/Linter.test.cpp +++ b/tests/Linter.test.cpp @@ -1392,19 +1392,31 @@ TEST_CASE_FIXTURE(Fixture, "DeprecatedApi") {"DataCost", {typeChecker.numberType, /* deprecated= */ true}}, {"Wait", {typeChecker.anyType, /* deprecated= */ true}}, }; + + TypeId colorType = typeChecker.globalTypes.addType(TableTypeVar{{}, std::nullopt, typeChecker.globalScope->level, Luau::TableState::Sealed}); + + getMutable(colorType)->props = { + {"toHSV", {typeChecker.anyType, /* deprecated= */ true, "Color3:ToHSV"} } + }; + + addGlobalBinding(typeChecker, "Color3", Binding{colorType, {}}); + freeze(typeChecker.globalTypes); LintResult result = lintTyped(R"( return function (i: Instance) i:Wait(1.0) print(i.Name) + print(Color3.toHSV()) + print(Color3.doesntexist, i.doesntexist) -- type error, but this verifies we correctly handle non-existent members return i.DataCost end )"); - REQUIRE_EQ(result.warnings.size(), 2); + REQUIRE_EQ(result.warnings.size(), 3); CHECK_EQ(result.warnings[0].text, "Member 'Instance.Wait' is deprecated"); - CHECK_EQ(result.warnings[1].text, "Member 'Instance.DataCost' is deprecated"); + CHECK_EQ(result.warnings[1].text, "Member 'toHSV' is deprecated, use 'Color3:ToHSV' instead"); + CHECK_EQ(result.warnings[2].text, "Member 'Instance.DataCost' is deprecated"); } TEST_CASE_FIXTURE(Fixture, "TableOperations") @@ -1475,9 +1487,11 @@ _ = (true and true) or true _ = (true and false) and (42 and false) _ = true and true or false -- no warning since this is is a common pattern used as a ternary replacement + +_ = if true then 1 elseif true then 2 else 3 )"); - REQUIRE_EQ(result.warnings.size(), 7); + REQUIRE_EQ(result.warnings.size(), 8); CHECK_EQ(result.warnings[0].text, "Condition has already been checked on line 2"); CHECK_EQ(result.warnings[0].location.begin.line + 1, 4); CHECK_EQ(result.warnings[1].text, "Condition has already been checked on column 5"); @@ -1487,6 +1501,7 @@ _ = true and true or false -- no warning since this is is a common pattern used CHECK_EQ(result.warnings[5].text, "Condition has already been checked on column 6"); CHECK_EQ(result.warnings[6].text, "Condition has already been checked on column 15"); CHECK_EQ(result.warnings[6].location.begin.line + 1, 19); + CHECK_EQ(result.warnings[7].text, "Condition has already been checked on column 8"); } TEST_CASE_FIXTURE(Fixture, "DuplicateConditionsExpr") @@ -1528,4 +1543,19 @@ return foo, moo, a1, a2 CHECK_EQ(result.warnings[3].text, "Function parameter 'self' already defined implicitly"); } +TEST_CASE_FIXTURE(Fixture, "MisleadingAndOr") +{ + LintResult result = lint(R"( +_ = math.random() < 0.5 and true or 42 +_ = math.random() < 0.5 and false or 42 -- misleading +_ = math.random() < 0.5 and nil or 42 -- misleading +_ = math.random() < 0.5 and 0 or 42 +_ = (math.random() < 0.5 and false) or 42 -- currently ignored +)"); + + REQUIRE_EQ(result.warnings.size(), 2); + CHECK_EQ(result.warnings[0].text, "The and-or expression always evaluates to the second alternative because the first alternative is false; consider using if-then-else expression instead"); + CHECK_EQ(result.warnings[1].text, "The and-or expression always evaluates to the second alternative because the first alternative is nil; consider using if-then-else expression instead"); +} + TEST_SUITE_END(); diff --git a/tests/Repl.test.cpp b/tests/Repl.test.cpp index f660bcd3..1f9c9739 100644 --- a/tests/Repl.test.cpp +++ b/tests/Repl.test.cpp @@ -8,9 +8,22 @@ #include #include +#include #include #include +struct Completion +{ + std::string completion; + std::string display; + + bool operator<(Completion const& other) const + { + return std::tie(completion, display) < std::tie(other.completion, other.display); + } +}; + +using CompletionSet = std::set; class ReplFixture { @@ -34,6 +47,27 @@ public: lua_pop(L, 1); return result; } + + CompletionSet getCompletionSet(const char* inputPrefix) + { + CompletionSet result; + int top = lua_gettop(L); + getCompletions(L, inputPrefix, [&result](const std::string& completion, const std::string& display) { + result.insert(Completion{completion, display}); + }); + // Ensure that generating completions doesn't change the position of luau's stack top. + CHECK(top == lua_gettop(L)); + + return result; + } + + bool checkCompletion(const CompletionSet& completions, const std::string& prefix, const std::string& expected) + { + std::string expectedDisplay(expected.substr(0, expected.find_first_of('('))); + Completion expectedCompletion{prefix + expected, expectedDisplay}; + return completions.count(expectedCompletion) == 1; + } + lua_State* L; private: @@ -115,3 +149,61 @@ TEST_CASE_FIXTURE(ReplFixture, "MultipleArguments") } TEST_SUITE_END(); + +TEST_SUITE_BEGIN("ReplCodeCompletion"); + +TEST_CASE_FIXTURE(ReplFixture, "CompleteGlobalVariables") +{ + runCode(L, R"( + myvariable1 = 5 + myvariable2 = 5 +)"); + CompletionSet completions = getCompletionSet("myvar"); + + std::string prefix = ""; + CHECK(completions.size() == 2); + CHECK(checkCompletion(completions, prefix, "myvariable1")); + CHECK(checkCompletion(completions, prefix, "myvariable2")); +} + +TEST_CASE_FIXTURE(ReplFixture, "CompleteTableKeys") +{ + runCode(L, R"( + t = { color = "red", size = 1, shape = "circle" } +)"); + { + CompletionSet completions = getCompletionSet("t."); + + std::string prefix = "t."; + CHECK(completions.size() == 3); + CHECK(checkCompletion(completions, prefix, "color")); + CHECK(checkCompletion(completions, prefix, "size")); + CHECK(checkCompletion(completions, prefix, "shape")); + } + + { + CompletionSet completions = getCompletionSet("t.s"); + + std::string prefix = "t."; + CHECK(completions.size() == 2); + CHECK(checkCompletion(completions, prefix, "size")); + CHECK(checkCompletion(completions, prefix, "shape")); + } +} + +TEST_CASE_FIXTURE(ReplFixture, "StringMethods") +{ + runCode(L, R"( + s = "" +)"); + { + CompletionSet completions = getCompletionSet("s:l"); + + std::string prefix = "s:"; + CHECK(completions.size() == 2); + CHECK(checkCompletion(completions, prefix, "len(")); + CHECK(checkCompletion(completions, prefix, "lower(")); + } +} + +TEST_SUITE_END(); diff --git a/tests/TypeInfer.aliases.test.cpp b/tests/TypeInfer.aliases.test.cpp index 76ab23b3..a8729268 100644 --- a/tests/TypeInfer.aliases.test.cpp +++ b/tests/TypeInfer.aliases.test.cpp @@ -595,4 +595,65 @@ TEST_CASE_FIXTURE(Fixture, "generic_typevars_are_not_considered_to_escape_their_ LUAU_REQUIRE_NO_ERRORS(result); } +/* + * The two-pass alias definition system starts by ascribing a free TypeVar to each alias. It then + * circles back to fill in the actual type later on. + * + * If this free type is unified with something degenerate like `any`, we need to take extra care + * to ensure that the alias actually binds to the type that the user expected. + */ +TEST_CASE_FIXTURE(Fixture, "forward_declared_alias_is_not_clobbered_by_prior_unification_with_any") +{ + ScopedFastFlag sff[] = { + {"LuauTwoPassAliasDefinitionFix", true} + }; + + CheckResult result = check(R"( + local function x() + local y: FutureType = {}::any + return 1 + end + type FutureType = { foo: typeof(x()) } + local d: FutureType = { smth = true } -- missing error, 'd' is resolved to 'any' + )"); + + CHECK_EQ("{| foo: number |}", toString(requireType("d"), {true})); + + LUAU_REQUIRE_ERROR_COUNT(1, result); +} + +TEST_CASE_FIXTURE(Fixture, "forward_declared_alias_is_not_clobbered_by_prior_unification_with_any_2") +{ + ScopedFastFlag sff[] = { + {"LuauTwoPassAliasDefinitionFix", true}, + + // We also force these two flags because this surfaced an unfortunate interaction. + {"LuauErrorRecoveryType", true}, + {"LuauQuantifyInPlace2", true}, + }; + + CheckResult result = check(R"( + local B = {} + B.bar = 4 + + function B:smth1() + local self: FutureIntersection = self + self.foo = 4 + return 4 + end + + function B:smth2() + local self: FutureIntersection = self + self.bar = 5 -- error, even though we should have B part with bar + end + + type A = { foo: typeof(B.smth1({foo=3})) } -- trick toposort into sorting functions before types + type B = typeof(B) + + type FutureIntersection = A & B + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.builtins.test.cpp b/tests/TypeInfer.builtins.test.cpp index 6730bedb..df06884d 100644 --- a/tests/TypeInfer.builtins.test.cpp +++ b/tests/TypeInfer.builtins.test.cpp @@ -7,8 +7,6 @@ #include "doctest.h" -LUAU_FASTFLAG(LuauFixTonumberReturnType) - using namespace Luau; LUAU_FASTFLAG(LuauUseCommittingTxnLog) @@ -850,11 +848,8 @@ TEST_CASE_FIXTURE(Fixture, "tonumber_returns_optional_number_type") local b: number = tonumber('asdf') )"); - if (FFlag::LuauFixTonumberReturnType) - { - LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK_EQ("Type 'number?' could not be converted into 'number'", toString(result.errors[0])); - } + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ("Type 'number?' could not be converted into 'number'", toString(result.errors[0])); } TEST_CASE_FIXTURE(Fixture, "tonumber_returns_optional_number_type2") @@ -893,7 +888,7 @@ TEST_CASE_FIXTURE(Fixture, "assert_removes_falsy_types") { ScopedFastFlag sff[]{ {"LuauAssertStripsFalsyTypes", true}, - {"LuauDiscriminableUnions", true}, + {"LuauDiscriminableUnions2", true}, }; CheckResult result = check(R"( @@ -910,7 +905,7 @@ TEST_CASE_FIXTURE(Fixture, "assert_removes_falsy_types_even_from_type_pack_tail_ { ScopedFastFlag sff[]{ {"LuauAssertStripsFalsyTypes", true}, - {"LuauDiscriminableUnions", true}, + {"LuauDiscriminableUnions2", true}, }; CheckResult result = check(R"( @@ -927,7 +922,7 @@ TEST_CASE_FIXTURE(Fixture, "assert_returns_false_and_string_iff_it_knows_the_fir { ScopedFastFlag sff[]{ {"LuauAssertStripsFalsyTypes", true}, - {"LuauDiscriminableUnions", true}, + {"LuauDiscriminableUnions2", true}, }; CheckResult result = check(R"( diff --git a/tests/TypeInfer.provisional.test.cpp b/tests/TypeInfer.provisional.test.cpp index e5eb0dca..2bcd840c 100644 --- a/tests/TypeInfer.provisional.test.cpp +++ b/tests/TypeInfer.provisional.test.cpp @@ -262,7 +262,7 @@ TEST_CASE_FIXTURE(Fixture, "lvalue_equals_another_lvalue_with_no_overlap") // Just needs to fully support equality refinement. Which is annoying without type states. TEST_CASE_FIXTURE(Fixture, "discriminate_from_x_not_equal_to_nil") { - ScopedFastFlag sff{"LuauDiscriminableUnions", true}; + ScopedFastFlag sff{"LuauDiscriminableUnions2", true}; CheckResult result = check(R"( type T = {x: string, y: number} | {x: nil, y: nil} @@ -616,4 +616,76 @@ local a: Self
CHECK_EQ(toString(requireType("a")), "Table
"); } +TEST_CASE_FIXTURE(Fixture, "do_not_ice_when_trying_to_pick_first_of_generic_type_pack") +{ + ScopedFastFlag sff[]{ + {"LuauQuantifyInPlace2", true}, + {"LuauReturnAnyInsteadOfICE", true}, + }; + + // In-place quantification causes these types to have the wrong types but only because of nasty interaction with prototyping. + // The type of f is initially () -> free1... + // Then the prototype iterator advances, and checks the function expression assigned to g, which has the type () -> free2... + // In the body it calls f and returns what f() returns. This binds free2... with free1..., causing f and g to have same types. + // We then quantify g, leaving it with the final type () -> a... + // Because free1... and free2... were bound, in combination with in-place quantification, f's return type was also turned into a... + // Then the check iterator catches up, and checks the body of f, and attempts to quantify it too. + // Alas, one of the requirements for quantification is that a type must contain free types. () -> a... has no free types. + // Thus the quantification for f was no-op, which explains why f does not have any type parameters. + // Calling f() will attempt to instantiate the function type, which turns generics in type binders into to free types. + // However, instantiations only converts generics contained within the type binders of a function, so instantiation was also no-op. + // Which means that calling f() simply returned a... rather than an instantiation of it. And since the call site was not in tail position, + // picking first element in a... triggers an ICE because calls returning generic packs are unexpected. + CheckResult result = check(R"( + local function f() end + + local g = function() return f() end + + local x = (f()) -- should error: no return values to assign from the call to f + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + // f and g should have the type () -> () + CHECK_EQ("() -> (a...)", toString(requireType("f"))); + CHECK_EQ("() -> (a...)", toString(requireType("g"))); + CHECK_EQ("any", toString(requireType("x"))); // any is returned instead of ICE for now +} + +TEST_CASE_FIXTURE(Fixture, "specialization_binds_with_prototypes_too_early") +{ + CheckResult result = check(R"( + local function id(x) return x end + local n2n: (number) -> number = id + local s2s: (string) -> string = id + )"); + + LUAU_REQUIRE_ERRORS(result); // Should not have any errors. +} + +TEST_CASE_FIXTURE(Fixture, "weird_fail_to_unify_type_pack") +{ + ScopedFastFlag sff{"LuauQuantifyInPlace2", true}; + + CheckResult result = check(R"( + local function f() return end + local g = function() return f() end + )"); + + LUAU_REQUIRE_ERRORS(result); // Should not have any errors. +} + +TEST_CASE_FIXTURE(Fixture, "weird_fail_to_unify_variadic_pack") +{ + ScopedFastFlag sff{"LuauQuantifyInPlace2", true}; + + CheckResult result = check(R"( + --!strict + local function f(...) return ... end + local g = function(...) return f(...) end + )"); + + LUAU_REQUIRE_ERRORS(result); // Should not have any errors. +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.refinements.test.cpp b/tests/TypeInfer.refinements.test.cpp index 3a610c3a..48e6be6a 100644 --- a/tests/TypeInfer.refinements.test.cpp +++ b/tests/TypeInfer.refinements.test.cpp @@ -6,7 +6,7 @@ #include "doctest.h" -LUAU_FASTFLAG(LuauDiscriminableUnions) +LUAU_FASTFLAG(LuauDiscriminableUnions2) LUAU_FASTFLAG(LuauWeakEqConstraint) LUAU_FASTFLAG(LuauQuantifyInPlace2) @@ -262,7 +262,7 @@ TEST_CASE_FIXTURE(Fixture, "typeguard_only_look_up_types_from_global_scope") end )"); - if (FFlag::LuauDiscriminableUnions) + if (FFlag::LuauDiscriminableUnions2) { LUAU_REQUIRE_NO_ERRORS(result); @@ -435,7 +435,7 @@ TEST_CASE_FIXTURE(Fixture, "lvalue_is_equal_to_a_term") TEST_CASE_FIXTURE(Fixture, "term_is_equal_to_an_lvalue") { ScopedFastFlag sff[] = { - {"LuauDiscriminableUnions", true}, + {"LuauDiscriminableUnions2", true}, {"LuauSingletonTypes", true}, }; @@ -485,7 +485,7 @@ TEST_CASE_FIXTURE(Fixture, "lvalue_is_not_nil") TEST_CASE_FIXTURE(Fixture, "free_type_is_equal_to_an_lvalue") { - ScopedFastFlag sff{"LuauDiscriminableUnions", true}; + ScopedFastFlag sff{"LuauDiscriminableUnions2", true}; ScopedFastFlag sff2{"LuauWeakEqConstraint", true}; CheckResult result = check(R"( @@ -589,7 +589,7 @@ TEST_CASE_FIXTURE(Fixture, "type_narrow_to_vector") end )"); - if (FFlag::LuauDiscriminableUnions) + if (FFlag::LuauDiscriminableUnions2) { LUAU_REQUIRE_NO_ERRORS(result); } @@ -1002,7 +1002,7 @@ TEST_CASE_FIXTURE(Fixture, "apply_refinements_on_astexprindexexpr_whose_subscrip TEST_CASE_FIXTURE(Fixture, "discriminate_from_truthiness_of_x") { ScopedFastFlag sff[] = { - {"LuauDiscriminableUnions", true}, + {"LuauDiscriminableUnions2", true}, {"LuauParseSingletonTypes", true}, {"LuauSingletonTypes", true}, }; @@ -1028,7 +1028,7 @@ TEST_CASE_FIXTURE(Fixture, "discriminate_from_truthiness_of_x") TEST_CASE_FIXTURE(Fixture, "discriminate_tag") { ScopedFastFlag sff[] = { - {"LuauDiscriminableUnions", true}, + {"LuauDiscriminableUnions2", true}, {"LuauParseSingletonTypes", true}, {"LuauSingletonTypes", true}, }; @@ -1069,7 +1069,7 @@ TEST_CASE_FIXTURE(Fixture, "narrow_boolean_to_true_or_false") ScopedFastFlag sff[]{ {"LuauParseSingletonTypes", true}, {"LuauSingletonTypes", true}, - {"LuauDiscriminableUnions", true}, + {"LuauDiscriminableUnions2", true}, {"LuauAssertStripsFalsyTypes", true}, }; @@ -1094,7 +1094,7 @@ TEST_CASE_FIXTURE(Fixture, "discriminate_on_properties_of_disjoint_tables_where_ ScopedFastFlag sff[]{ {"LuauParseSingletonTypes", true}, {"LuauSingletonTypes", true}, - {"LuauDiscriminableUnions", true}, + {"LuauDiscriminableUnions2", true}, {"LuauAssertStripsFalsyTypes", true}, }; @@ -1118,7 +1118,7 @@ TEST_CASE_FIXTURE(Fixture, "discriminate_on_properties_of_disjoint_tables_where_ TEST_CASE_FIXTURE(RefinementClassFixture, "discriminate_from_isa_of_x") { ScopedFastFlag sff[] = { - {"LuauDiscriminableUnions", true}, + {"LuauDiscriminableUnions2", true}, {"LuauParseSingletonTypes", true}, {"LuauSingletonTypes", true}, }; @@ -1157,7 +1157,7 @@ TEST_CASE_FIXTURE(RefinementClassFixture, "typeguard_cast_free_table_to_vector") end )"); - if (FFlag::LuauDiscriminableUnions) + if (FFlag::LuauDiscriminableUnions2) LUAU_REQUIRE_NO_ERRORS(result); else { diff --git a/tests/TypeInfer.test.cpp b/tests/TypeInfer.test.cpp index ead3d762..531a382f 100644 --- a/tests/TypeInfer.test.cpp +++ b/tests/TypeInfer.test.cpp @@ -5164,4 +5164,151 @@ function x:Destroy(): () end LUAU_REQUIRE_ERRORS(result); } +TEST_CASE_FIXTURE(Fixture, "do_not_modify_imported_types_2") +{ + ScopedFastFlag immutableTypes{"LuauImmutableTypes", true}; + + fileResolver.source["game/A"] = R"( +export type Type = { x: { a: number } } +return {} + )"; + + fileResolver.source["game/B"] = R"( +local types = require(game.A) +type Type = types.Type +local x: Type = { x = { a = 2 } } +type Rename = typeof(x.x) + )"; + + CheckResult result = frontend.check("game/B"); + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "do_not_modify_imported_types_3") +{ + ScopedFastFlag immutableTypes{"LuauImmutableTypes", true}; + + fileResolver.source["game/A"] = R"( +local y = setmetatable({}, {}) +export type Type = { x: typeof(y) } +return { x = y } + )"; + + fileResolver.source["game/B"] = R"( +local types = require(game.A) +type Type = types.Type +local x: Type = types +type Rename = typeof(x.x) + )"; + + CheckResult result = frontend.check("game/B"); + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "indexing_on_string_singletons") +{ + ScopedFastFlag sff[]{ + {"LuauDiscriminableUnions2", true}, + {"LuauRefactorTypeVarQuestions", true}, + {"LuauSingletonTypes", true}, + }; + + CheckResult result = check(R"( + local a: string = "hi" + if a == "hi" then + local x = a:byte() + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ(R"("hi")", toString(requireTypeAtPosition({3, 22}))); +} + +TEST_CASE_FIXTURE(Fixture, "indexing_on_union_of_string_singletons") +{ + ScopedFastFlag sff[]{ + {"LuauDiscriminableUnions2", true}, + {"LuauRefactorTypeVarQuestions", true}, + {"LuauSingletonTypes", true}, + }; + + CheckResult result = check(R"( + local a: string = "hi" + if a == "hi" or a == "bye" then + local x = a:byte() + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ(R"("bye" | "hi")", toString(requireTypeAtPosition({3, 22}))); +} + +TEST_CASE_FIXTURE(Fixture, "taking_the_length_of_string_singleton") +{ + ScopedFastFlag sff[]{ + {"LuauDiscriminableUnions2", true}, + {"LuauRefactorTypeVarQuestions", true}, + {"LuauSingletonTypes", true}, + {"LuauLengthOnCompositeType", true}, + }; + + CheckResult result = check(R"( + local a: string = "hi" + if a == "hi" then + local x = #a + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ(R"("hi")", toString(requireTypeAtPosition({3, 23}))); +} + +TEST_CASE_FIXTURE(Fixture, "taking_the_length_of_union_of_string_singleton") +{ + ScopedFastFlag sff[]{ + {"LuauDiscriminableUnions2", true}, + {"LuauRefactorTypeVarQuestions", true}, + {"LuauSingletonTypes", true}, + {"LuauLengthOnCompositeType", true}, + }; + + CheckResult result = check(R"( + local a: string = "hi" + if a == "hi" or a == "bye" then + local x = #a + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ(R"("bye" | "hi")", toString(requireTypeAtPosition({3, 23}))); +} + +/* + * When we add new properties to an unsealed table, we should do a level check and promote the property type to be at + * the level of the table. + */ +TEST_CASE_FIXTURE(Fixture, "inferred_properties_of_a_table_should_start_with_the_same_TypeLevel_of_that_table") +{ + CheckResult result = check(R"( + --!strict + local T = {} + + local function f(prop) + T[1] = { + prop = prop, + } + end + + local function g() + local l = T[1].prop + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.tryUnify.test.cpp b/tests/TypeInfer.tryUnify.test.cpp index 0aeca096..8c7fb79a 100644 --- a/tests/TypeInfer.tryUnify.test.cpp +++ b/tests/TypeInfer.tryUnify.test.cpp @@ -273,4 +273,21 @@ TEST_CASE_FIXTURE(TryUnifyFixture, "recursive_metatable_getmatchtag") state.tryUnify(&metatable, &variant); } +TEST_CASE_FIXTURE(TryUnifyFixture, "cli_50320_follow_in_any_unification") +{ + ScopedFastFlag sffs[] = { + {"LuauUseCommittingTxnLog", true}, + {"LuauFollowWithCommittingTxnLogInAnyUnification", true}, + }; + + TypePackVar free{FreeTypePack{TypeLevel{}}}; + TypePackVar target{TypePack{}}; + + TypeVar func{FunctionTypeVar{&free, &free}}; + + state.tryUnify(&free, &target); + // Shouldn't assert or error. + state.tryUnify(&func, typeChecker.anyType); +} + TEST_SUITE_END(); diff --git a/tests/conformance/basic.lua b/tests/conformance/basic.lua index de091632..78d90077 100644 --- a/tests/conformance/basic.lua +++ b/tests/conformance/basic.lua @@ -118,9 +118,7 @@ assert((function() return #_G end)() == 0) assert((function() return #{1,2} end)() == 2) assert((function() return #'g' end)() == 1) -local ud = newproxy(true) -getmetatable(ud).__len = function() return 42 end -assert((function() return #ud end)() == 42) +assert((function() local ud = newproxy(true) getmetatable(ud).__len = function() return 42 end return #ud end)() == 42) assert((function() local a = 1 a = -a return a end)() == -1) @@ -325,6 +323,10 @@ assert((function() local t = {6, 9, 7} t[4.5] = 10 return t[4.5] end)() == 10) assert((function() local t = {6, 9, 7} t['a'] = 11 return t['a'] end)() == 11) assert((function() local t = {6, 9, 7} setmetatable(t, { __newindex = function(t,i,v) rawset(t, i * 10, v) end }) t[1] = 17 t[5] = 1 return concat(t[1],t[5],t[50]) end)() == "17,nil,1") +-- userdata access +assert((function() local ud = newproxy(true) getmetatable(ud).__index = function(ud,i) return i * 10 end return ud[2] end)() == 20) +assert((function() local ud = newproxy(true) getmetatable(ud).__index = function() return function(self, i) return i * 10 end end return ud:meow(2) end)() == 20) + -- and/or -- rhs is a constant assert((function() local a = 1 a = a and 2 return a end)() == 2) @@ -462,7 +464,7 @@ assert((function() a = {} b = {} mt = { __eq = function(l, r) return #l == #r en -- metatable ops local function vec3t(x, y, z) - return setmetatable({ x=x, y=y, z=z}, { + return setmetatable({x=x, y=y, z=z}, { __add = function(l, r) return vec3t(l.x + r.x, l.y + r.y, l.z + r.z) end, __sub = function(l, r) return vec3t(l.x - r.x, l.y - r.y, l.z - r.z) end, __mul = function(l, r) return type(r) == "number" and vec3t(l.x * r, l.y * r, l.z * r) or vec3t(l.x * r.x, l.y * r.y, l.z * r.z) end, diff --git a/tests/conformance/debug.lua b/tests/conformance/debug.lua index 8c96ab33..0e410000 100644 --- a/tests/conformance/debug.lua +++ b/tests/conformance/debug.lua @@ -37,6 +37,7 @@ coroutine.resume(co2, 0 / 0, 42) assert(debug.traceback(co2) == "debug.lua:31 function halp\n") assert(debug.info(co2, 0, "l") == 31) +assert(debug.info(co2, 0, "f") == halp) -- info errors function qux(...) diff --git a/tests/conformance/errors.lua b/tests/conformance/errors.lua index d5ff215b..751188be 100644 --- a/tests/conformance/errors.lua +++ b/tests/conformance/errors.lua @@ -260,8 +260,7 @@ local a,b = loadstring(s) assert(not a) --assert(string.find(b, "line 2")) --- Test for CLI-28786 --- The xpcall is intentially going to cause an exception +-- The xpcall is intentionally going to cause an exception -- followed by a forced exception in the error handler. -- If the secondary handler isn't trapped, it will cause -- the unit test to fail. If the xpcall captures the @@ -281,6 +280,19 @@ coroutine.wrap(function() assert(not pcall(debug.getinfo, coroutine.running(), 0, ">")) end)() +-- loadstring chunk truncation +local a,b = loadstring("nope", "@short") +assert(not a and b:match('[^ ]+') == "short:1:") + +local a,b = loadstring("nope", "@" .. string.rep("thisisaverylongstringitssolongthatitwontfitintotheinternalbufferprovidedtovariousdebugfacilities", 10)) +assert(not a and b:match('[^ ]+') == "...wontfitintotheinternalbufferprovidedtovariousdebugfacilitiesthisisaverylongstringitssolongthatitwontfitintotheinternalbufferprovidedtovariousdebugfacilitiesthisisaverylongstringitssolongthatitwontfitintotheinternalbufferprovidedtovariousdebugfacilities:1:") + +local a,b = loadstring("nope", "=short") +assert(not a and b:match('[^ ]+') == "short:1:") + +local a,b = loadstring("nope", "=" .. string.rep("thisisaverylongstringitssolongthatitwontfitintotheinternalbufferprovidedtovariousdebugfacilities", 10)) +assert(not a and b:match('[^ ]+') == "thisisaverylongstringitssolongthatitwontfitintotheinternalbufferprovidedtovariousdebugfacilitiesthisisaverylongstringitssolongthatitwontfitintotheinternalbufferprovidedtovariousdebugfacilitiesthisisaverylongstringitssolongthatitwontfitintotheinternalbuffe:1:") + -- arith errors function ecall(fn, ...) local ok, err = pcall(fn, ...) diff --git a/tests/conformance/gc.lua b/tests/conformance/gc.lua index 409cd224..5804ea7f 100644 --- a/tests/conformance/gc.lua +++ b/tests/conformance/gc.lua @@ -180,6 +180,11 @@ x,y,z=nil collectgarbage() assert(next(a) == string.rep('$', 11)) +-- shrinking tables reduce their capacity; confirming the shrinking is difficult but we can at least test the surface level behavior +a = {}; setmetatable(a, {__mode = 'ks'}) +for i=1,lim do a[{}] = i end +collectgarbage() +assert(next(a) == nil) -- testing userdata collectgarbage("stop") -- stop collection @@ -315,8 +320,6 @@ do end collectgarbage() - end - return('OK') diff --git a/tests/conformance/math.lua b/tests/conformance/math.lua index bfea0e1f..79ea0fb6 100644 --- a/tests/conformance/math.lua +++ b/tests/conformance/math.lua @@ -289,6 +289,7 @@ assert(math.sqrt("4") == 2) assert(math.tanh("0") == 0) assert(math.tan("0") == 0) assert(math.clamp("0", 2, 3) == 2) +assert(math.clamp("4", 2, 3) == 3) assert(math.sign("2") == 1) assert(math.sign("-2") == -1) assert(math.sign("0") == 0) diff --git a/tests/conformance/vararg.lua b/tests/conformance/vararg.lua index d05f9577..178c56b8 100644 --- a/tests/conformance/vararg.lua +++ b/tests/conformance/vararg.lua @@ -139,6 +139,12 @@ assert(selectmany(1, 10, 20, 30) == "10,20,30") assert(selectone(2, 10, 20, 30) == 20) assert(selectmany(2, 10, 20, 30) == "20,30") +assert(selectone(3, 10, 20, 30) == 30) +assert(selectmany(3, 10, 20, 30) == "30") + +assert(selectone(4, 10, 20, 30) == nil) +assert(selectmany(4, 10, 20, 30) == "") + assert(selectone(-2, 10, 20, 30) == 20) assert(selectmany(-2, 10, 20, 30) == "20,30") diff --git a/tests/conformance/vector.lua b/tests/conformance/vector.lua index 7d18bda3..22d6adfc 100644 --- a/tests/conformance/vector.lua +++ b/tests/conformance/vector.lua @@ -87,9 +87,18 @@ assert(pcall(function() local t = {} rawset(t, vector(0/0, 2, 3), 1) end) == fal -- make sure we cover both builtin and C impl assert(vector(1, 2, 4) == vector("1", "2", "4")) +-- validate component access (both cases) +assert(vector(1, 2, 3).x == 1) +assert(vector(1, 2, 3).X == 1) +assert(vector(1, 2, 3).y == 2) +assert(vector(1, 2, 3).Y == 2) +assert(vector(1, 2, 3).z == 3) +assert(vector(1, 2, 3).Z == 3) + -- additional checks for 4-component vectors if vector_size == 4 then assert(vector(1, 2, 3, 4).w == 4) + assert(vector(1, 2, 3, 4).W == 4) end return 'OK' diff --git a/tools/heapgraph.py b/tools/heapgraph.py index 106db549..d4d29af1 100644 --- a/tools/heapgraph.py +++ b/tools/heapgraph.py @@ -7,10 +7,20 @@ # The result of analysis is a .svg file which can be viewed in a browser # To generate these dumps, use luaC_dump, ideally preceded by luaC_fullgc +import argparse import json import sys import svg +argumentParser = argparse.ArgumentParser(description='Luau heap snapshot analyzer') + +argumentParser.add_argument('--split', dest = 'split', type = str, default = 'none', help = 'Perform additional root split using memory categories', choices = ['none', 'custom', 'all']) + +argumentParser.add_argument('snapshot') +argumentParser.add_argument('snapshotnew', nargs='?') + +arguments = argumentParser.parse_args() + class Node(svg.Node): def __init__(self): svg.Node.__init__(self) @@ -30,14 +40,14 @@ class Node(svg.Node): return "{} ({:,} bytes, {:.1%}); self: {:,} bytes in {:,} objects".format(self.name, self.width, self.width / root.width, self.size, self.count) # load files -if len(sys.argv) == 2: +if arguments.snapshotnew == None: dumpold = None - with open(sys.argv[1]) as f: + with open(arguments.snapshot) as f: dump = json.load(f) else: - with open(sys.argv[1]) as f: + with open(arguments.snapshot) as f: dumpold = json.load(f) - with open(sys.argv[2]) as f: + with open(arguments.snapshotnew) as f: dump = json.load(f) # reachability analysis: how much of the heap is reachable from roots? @@ -111,12 +121,15 @@ while offset < len(queue): if "object" in obj: queue.append((obj["object"], node)) -def annotateContainedCategories(node): +def annotateContainedCategories(node, start): for obj in node.objects: + if obj["cat"] < start: + obj["cat"] = 0 + node.categories.add(obj["cat"]) for child in node.children.values(): - annotateContainedCategories(child) + annotateContainedCategories(child, start) for cat in child.categories: node.categories.add(cat) @@ -172,9 +185,11 @@ def splitIntoCategories(root): return result -# temporarily disabled because it makes FG harder to read, maybe this should be a separate command line option? -if dump["stats"].get("categories") and False: - annotateContainedCategories(root) +if dump["stats"].get("categories") and arguments.split != 'none': + if arguments.split == 'custom': + annotateContainedCategories(root, 128) + else: + annotateContainedCategories(root, 0) root = splitIntoCategories(root) diff --git a/tools/svg.py b/tools/svg.py index 99853fb6..21200eeb 100644 --- a/tools/svg.py +++ b/tools/svg.py @@ -452,7 +452,7 @@ def display(root, title, colors, flip = False): .replace("$gradient-start", gradient_start) .replace("$gradient-end", gradient_end) .replace("$height", str(svgheight)) - .replace("$status", str(svgheight - 16 + 3)) + .replace("$status", str((svgheight - 16 + 3 if flip else 3 * 16 - 3))) .replace("$flip", str(int(flip))) ) From 63d5423bbb283e602dd396a693a90318f4902d23 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Fri, 11 Feb 2022 11:02:09 -0800 Subject: [PATCH 157/399] Sync to upstream/release/514 (#357) --- Analysis/include/Luau/Autocomplete.h | 2 + Analysis/include/Luau/Linter.h | 1 + Analysis/include/Luau/Substitution.h | 2 +- Analysis/include/Luau/TxnLog.h | 4 +- Analysis/include/Luau/TypeInfer.h | 11 ++ Analysis/include/Luau/TypePack.h | 3 - Analysis/include/Luau/TypeVar.h | 3 - Analysis/src/EmbeddedBuiltinDefinitions.cpp | 12 +- Analysis/src/Linter.cpp | 103 ++++++++++++-- Analysis/src/Module.cpp | 28 ++-- Analysis/src/Scope.cpp | 4 + Analysis/src/TxnLog.cpp | 8 ++ Analysis/src/TypeInfer.cpp | 100 ++++++++++--- Analysis/src/TypeVar.cpp | 4 +- Analysis/src/Unifier.cpp | 125 +++++++++++------ Ast/src/Parser.cpp | 2 +- CLI/Repl.cpp | 68 ++++++--- CLI/Repl.h | 7 +- Compiler/src/BytecodeBuilder.cpp | 8 +- Compiler/src/Compiler.cpp | 62 ++------- Sources.cmake | 5 +- VM/src/lbuiltins.cpp | 2 +- VM/src/lgcdebug.cpp | 4 + VM/src/lmem.cpp | 127 ++++++++++++----- VM/src/lobject.cpp | 4 +- VM/src/lvmexecute.cpp | 3 +- tests/Compiler.test.cpp | 4 - tests/Conformance.test.cpp | 2 - tests/Linter.test.cpp | 36 ++++- tests/Repl.test.cpp | 92 ++++++++++++ tests/TypeInfer.aliases.test.cpp | 61 ++++++++ tests/TypeInfer.builtins.test.cpp | 15 +- tests/TypeInfer.provisional.test.cpp | 74 +++++++++- tests/TypeInfer.refinements.test.cpp | 22 +-- tests/TypeInfer.test.cpp | 147 ++++++++++++++++++++ tests/TypeInfer.tryUnify.test.cpp | 17 +++ tests/conformance/basic.lua | 10 +- tests/conformance/debug.lua | 1 + tests/conformance/errors.lua | 16 ++- tests/conformance/gc.lua | 7 +- tests/conformance/math.lua | 1 + tests/conformance/vararg.lua | 6 + tests/conformance/vector.lua | 9 ++ tools/heapgraph.py | 33 +++-- tools/svg.py | 2 +- 45 files changed, 989 insertions(+), 268 deletions(-) diff --git a/Analysis/include/Luau/Autocomplete.h b/Analysis/include/Luau/Autocomplete.h index 58534293..65b788d3 100644 --- a/Analysis/include/Luau/Autocomplete.h +++ b/Analysis/include/Luau/Autocomplete.h @@ -86,6 +86,8 @@ struct OwningAutocompleteResult }; AutocompleteResult autocomplete(Frontend& frontend, const ModuleName& moduleName, Position position, StringCompletionCallback callback); + +// Deprecated, do not use in new work. OwningAutocompleteResult autocompleteSource(Frontend& frontend, std::string_view source, Position position, StringCompletionCallback callback); } // namespace Luau diff --git a/Analysis/include/Luau/Linter.h b/Analysis/include/Luau/Linter.h index 1f7f7f9d..ec3c124d 100644 --- a/Analysis/include/Luau/Linter.h +++ b/Analysis/include/Luau/Linter.h @@ -49,6 +49,7 @@ struct LintWarning Code_DeprecatedApi = 22, Code_TableOperations = 23, Code_DuplicateCondition = 24, + Code_MisleadingAndOr = 25, Code__Count }; diff --git a/Analysis/include/Luau/Substitution.h b/Analysis/include/Luau/Substitution.h index 4f3307cd..f85b4269 100644 --- a/Analysis/include/Luau/Substitution.h +++ b/Analysis/include/Luau/Substitution.h @@ -93,7 +93,7 @@ struct Tarjan // This should never be null; ensure you initialize it before calling // substitution methods. - const TxnLog* log; + const TxnLog* log = nullptr; std::vector edgesTy; std::vector edgesTp; diff --git a/Analysis/include/Luau/TxnLog.h b/Analysis/include/Luau/TxnLog.h index 02b87374..f238e258 100644 --- a/Analysis/include/Luau/TxnLog.h +++ b/Analysis/include/Luau/TxnLog.h @@ -307,8 +307,8 @@ private: // // We can't use a DenseHashMap here because we need a non-const iterator // over the map when we concatenate. - std::unordered_map> typeVarChanges; - std::unordered_map> typePackChanges; + std::unordered_map, DenseHashPointer> typeVarChanges; + std::unordered_map, DenseHashPointer> typePackChanges; TxnLog* parent = nullptr; diff --git a/Analysis/include/Luau/TypeInfer.h b/Analysis/include/Luau/TypeInfer.h index f61ecbf5..5592fa1f 100644 --- a/Analysis/include/Luau/TypeInfer.h +++ b/Analysis/include/Luau/TypeInfer.h @@ -103,6 +103,11 @@ struct GenericTypeDefinitions std::vector genericPacks; }; +struct HashBoolNamePair +{ + size_t operator()(const std::pair& pair) const; +}; + // All TypeVars are retained via Environment::typeVars. All TypeIds // within a program are borrowed pointers into this set. struct TypeChecker @@ -411,6 +416,12 @@ public: private: int checkRecursionCount = 0; int recursionCount = 0; + + /** + * We use this to avoid doing second-pass analysis of type aliases that are duplicates. We record a pair + * (exported, name) to properly deal with the case where the two duplicates do not have the same export status. + */ + DenseHashSet, HashBoolNamePair> duplicateTypeAliases; }; // Unit test hook diff --git a/Analysis/include/Luau/TypePack.h b/Analysis/include/Luau/TypePack.h index ca588ccb..c74bad11 100644 --- a/Analysis/include/Luau/TypePack.h +++ b/Analysis/include/Luau/TypePack.h @@ -54,9 +54,6 @@ struct TypePackVar bool persistent = false; // Pointer to the type arena that allocated this type. - // Do not depend on the value of this under any circumstances. This is for - // debugging purposes only. This is only set in debug builds; it is nullptr - // in all other environments. TypeArena* owningArena = nullptr; }; diff --git a/Analysis/include/Luau/TypeVar.h b/Analysis/include/Luau/TypeVar.h index 11dc9377..8d1a9fa6 100644 --- a/Analysis/include/Luau/TypeVar.h +++ b/Analysis/include/Luau/TypeVar.h @@ -449,9 +449,6 @@ struct TypeVar final std::optional documentationSymbol; // Pointer to the type arena that allocated this type. - // Do not depend on the value of this under any circumstances. This is for - // debugging purposes only. This is only set in debug builds; it is nullptr - // in all other environments. TypeArena* owningArena = nullptr; bool operator==(const TypeVar& rhs) const; diff --git a/Analysis/src/EmbeddedBuiltinDefinitions.cpp b/Analysis/src/EmbeddedBuiltinDefinitions.cpp index 24982506..f3ef88fc 100644 --- a/Analysis/src/EmbeddedBuiltinDefinitions.cpp +++ b/Analysis/src/EmbeddedBuiltinDefinitions.cpp @@ -1,8 +1,6 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #include "Luau/BuiltinDefinitions.h" -LUAU_FASTFLAGVARIABLE(LuauFixTonumberReturnType, false) - namespace Luau { @@ -115,6 +113,7 @@ declare function gcinfo(): number declare function error(message: T, level: number?) declare function tostring(value: T): string + declare function tonumber(value: T, radix: number?): number? declare function rawequal(a: T1, b: T2): boolean declare function rawget(tab: {[K]: V}, k: K): V @@ -200,14 +199,7 @@ declare function gcinfo(): number std::string getBuiltinDefinitionSource() { - std::string result = kBuiltinDefinitionLuaSrc; - - if (FFlag::LuauFixTonumberReturnType) - result += "declare function tonumber(value: T, radix: number?): number?\n"; - else - result += "declare function tonumber(value: T, radix: number?): number\n"; - - return result; + return kBuiltinDefinitionLuaSrc; } } // namespace Luau diff --git a/Analysis/src/Linter.cpp b/Analysis/src/Linter.cpp index 57a33e93..2ba6a0fc 100644 --- a/Analysis/src/Linter.cpp +++ b/Analysis/src/Linter.cpp @@ -43,6 +43,7 @@ static const char* kWarningNames[] = { "DeprecatedApi", "TableOperations", "DuplicateCondition", + "MisleadingAndOr", }; // clang-format on @@ -2040,18 +2041,28 @@ private: const Property* prop = lookupClassProp(cty, node->index.value); if (prop && prop->deprecated) - { - if (!prop->deprecatedSuggestion.empty()) - emitWarning(*context, LintWarning::Code_DeprecatedApi, node->location, "Member '%s.%s' is deprecated, use '%s' instead", - cty->name.c_str(), node->index.value, prop->deprecatedSuggestion.c_str()); - else - emitWarning(*context, LintWarning::Code_DeprecatedApi, node->location, "Member '%s.%s' is deprecated", cty->name.c_str(), - node->index.value); - } + report(node->location, *prop, cty->name.c_str(), node->index.value); + } + else if (const TableTypeVar* tty = get(follow(*ty))) + { + auto prop = tty->props.find(node->index.value); + + if (prop != tty->props.end() && prop->second.deprecated) + report(node->location, prop->second, tty->name ? tty->name->c_str() : nullptr, node->index.value); } return true; } + + void report(const Location& location, const Property& prop, const char* container, const char* field) + { + std::string suggestion = prop.deprecatedSuggestion.empty() ? "" : format(", use '%s' instead", prop.deprecatedSuggestion.c_str()); + + if (container) + emitWarning(*context, LintWarning::Code_DeprecatedApi, location, "Member '%s.%s' is deprecated%s", container, field, suggestion.c_str()); + else + emitWarning(*context, LintWarning::Code_DeprecatedApi, location, "Member '%s' is deprecated%s", field, suggestion.c_str()); + } }; class LintTableOperations : AstVisitor @@ -2257,6 +2268,39 @@ private: return false; } + bool visit(AstExprIfElse* expr) override + { + if (!expr->falseExpr->is()) + return true; + + // if..elseif chain detected, we need to unroll it + std::vector conditions; + conditions.reserve(2); + + AstExprIfElse* head = expr; + while (head) + { + head->condition->visit(this); + head->trueExpr->visit(this); + + conditions.push_back(head->condition); + + if (head->falseExpr->is()) + { + head = head->falseExpr->as(); + continue; + } + + head->falseExpr->visit(this); + break; + } + + detectDuplicates(conditions); + + // block recursive visits so that we only analyze each chain once + return false; + } + bool visit(AstExprBinary* expr) override { if (expr->op != AstExprBinary::And && expr->op != AstExprBinary::Or) @@ -2418,6 +2462,46 @@ private: } }; +class LintMisleadingAndOr : AstVisitor +{ +public: + LUAU_NOINLINE static void process(LintContext& context) + { + LintMisleadingAndOr pass; + pass.context = &context; + + context.root->visit(&pass); + } + +private: + LintContext* context; + + bool visit(AstExprBinary* node) override + { + if (node->op != AstExprBinary::Or) + return true; + + AstExprBinary* and_ = node->left->as(); + if (!and_ || and_->op != AstExprBinary::And) + return true; + + const char* alt = nullptr; + + if (and_->right->is()) + alt = "nil"; + else if (AstExprConstantBool* c = and_->right->as(); c && c->value == false) + alt = "false"; + + if (alt) + emitWarning(*context, LintWarning::Code_MisleadingAndOr, node->location, + "The and-or expression always evaluates to the second alternative because the first alternative is %s; consider using if-then-else " + "expression instead", + alt); + + return true; + } +}; + static void fillBuiltinGlobals(LintContext& context, const AstNameTable& names, const ScopePtr& env) { ScopePtr current = env; @@ -2522,6 +2606,9 @@ std::vector lint(AstStat* root, const AstNameTable& names, const Sc if (context.warningEnabled(LintWarning::Code_DuplicateLocal)) LintDuplicateLocal::process(context); + if (context.warningEnabled(LintWarning::Code_MisleadingAndOr)) + LintMisleadingAndOr::process(context); + std::sort(context.result.begin(), context.result.end(), WarningComparator()); return context.result; diff --git a/Analysis/src/Module.cpp b/Analysis/src/Module.cpp index 4fdff8f7..817a33e9 100644 --- a/Analysis/src/Module.cpp +++ b/Analysis/src/Module.cpp @@ -12,10 +12,10 @@ #include LUAU_FASTFLAGVARIABLE(DebugLuauFreezeArena, false) -LUAU_FASTFLAGVARIABLE(DebugLuauTrackOwningArena, false) +LUAU_FASTFLAGVARIABLE(DebugLuauTrackOwningArena, false) // Remove with FFlagLuauImmutableTypes LUAU_FASTINTVARIABLE(LuauTypeCloneRecursionLimit, 300) LUAU_FASTFLAG(LuauTypeAliasDefaults) - +LUAU_FASTFLAG(LuauImmutableTypes) LUAU_FASTFLAGVARIABLE(LuauPrepopulateUnionOptionsBeforeAllocation, false) namespace Luau @@ -66,7 +66,7 @@ TypeId TypeArena::addTV(TypeVar&& tv) { TypeId allocated = typeVars.allocate(std::move(tv)); - if (FFlag::DebugLuauTrackOwningArena) + if (FFlag::DebugLuauTrackOwningArena || FFlag::LuauImmutableTypes) asMutable(allocated)->owningArena = this; return allocated; @@ -76,7 +76,7 @@ TypeId TypeArena::freshType(TypeLevel level) { TypeId allocated = typeVars.allocate(FreeTypeVar{level}); - if (FFlag::DebugLuauTrackOwningArena) + if (FFlag::DebugLuauTrackOwningArena || FFlag::LuauImmutableTypes) asMutable(allocated)->owningArena = this; return allocated; @@ -86,7 +86,7 @@ TypePackId TypeArena::addTypePack(std::initializer_list types) { TypePackId allocated = typePacks.allocate(TypePack{std::move(types)}); - if (FFlag::DebugLuauTrackOwningArena) + if (FFlag::DebugLuauTrackOwningArena || FFlag::LuauImmutableTypes) asMutable(allocated)->owningArena = this; return allocated; @@ -96,7 +96,7 @@ TypePackId TypeArena::addTypePack(std::vector types) { TypePackId allocated = typePacks.allocate(TypePack{std::move(types)}); - if (FFlag::DebugLuauTrackOwningArena) + if (FFlag::DebugLuauTrackOwningArena || FFlag::LuauImmutableTypes) asMutable(allocated)->owningArena = this; return allocated; @@ -106,7 +106,7 @@ TypePackId TypeArena::addTypePack(TypePack tp) { TypePackId allocated = typePacks.allocate(std::move(tp)); - if (FFlag::DebugLuauTrackOwningArena) + if (FFlag::DebugLuauTrackOwningArena || FFlag::LuauImmutableTypes) asMutable(allocated)->owningArena = this; return allocated; @@ -116,7 +116,7 @@ TypePackId TypeArena::addTypePack(TypePackVar tp) { TypePackId allocated = typePacks.allocate(std::move(tp)); - if (FFlag::DebugLuauTrackOwningArena) + if (FFlag::DebugLuauTrackOwningArena || FFlag::LuauImmutableTypes) asMutable(allocated)->owningArena = this; return allocated; @@ -454,8 +454,16 @@ TypeId clone(TypeId typeId, TypeArena& dest, SeenTypes& seenTypes, SeenTypePacks TypeCloner cloner{dest, typeId, seenTypes, seenTypePacks, cloneState}; Luau::visit(cloner, typeId->ty); // Mutates the storage that 'res' points into. - // TODO: Make this work when the arena of 'res' might be frozen - asMutable(res)->documentationSymbol = typeId->documentationSymbol; + if (FFlag::LuauImmutableTypes) + { + // Persistent types are not being cloned and we get the original type back which might be read-only + if (!res->persistent) + asMutable(res)->documentationSymbol = typeId->documentationSymbol; + } + else + { + asMutable(res)->documentationSymbol = typeId->documentationSymbol; + } } return res; diff --git a/Analysis/src/Scope.cpp b/Analysis/src/Scope.cpp index c30db9c2..0a362a5e 100644 --- a/Analysis/src/Scope.cpp +++ b/Analysis/src/Scope.cpp @@ -2,6 +2,8 @@ #include "Luau/Scope.h" +LUAU_FASTFLAG(LuauTwoPassAliasDefinitionFix); + namespace Luau { @@ -17,6 +19,8 @@ Scope::Scope(const ScopePtr& parent, int subLevel) , returnType(parent->returnType) , level(parent->level.incr()) { + if (FFlag::LuauTwoPassAliasDefinitionFix) + level = level.incr(); level.subLevel = subLevel; } diff --git a/Analysis/src/TxnLog.cpp b/Analysis/src/TxnLog.cpp index 0968a4c1..00067bdd 100644 --- a/Analysis/src/TxnLog.cpp +++ b/Analysis/src/TxnLog.cpp @@ -250,6 +250,10 @@ PendingTypePack* TxnLog::queue(TypePackId tp) PendingType* TxnLog::pending(TypeId ty) const { + // This function will technically work if `this` is nullptr, but this + // indicates a bug, so we explicitly assert. + LUAU_ASSERT(static_cast(this) != nullptr); + for (const TxnLog* current = this; current; current = current->parent) { if (auto it = current->typeVarChanges.find(ty); it != current->typeVarChanges.end()) @@ -261,6 +265,10 @@ PendingType* TxnLog::pending(TypeId ty) const PendingTypePack* TxnLog::pending(TypePackId tp) const { + // This function will technically work if `this` is nullptr, but this + // indicates a bug, so we explicitly assert. + LUAU_ASSERT(static_cast(this) != nullptr); + for (const TxnLog* current = this; current; current = current->parent) { if (auto it = current->typePackChanges.find(tp); it != current->typePackChanges.end()) diff --git a/Analysis/src/TypeInfer.cpp b/Analysis/src/TypeInfer.cpp index 99398f74..f1c314cd 100644 --- a/Analysis/src/TypeInfer.cpp +++ b/Analysis/src/TypeInfer.cpp @@ -32,12 +32,13 @@ LUAU_FASTFLAGVARIABLE(LuauRecursiveTypeParameterRestriction, false) LUAU_FASTFLAGVARIABLE(LuauGenericFunctionsDontCacheTypeParams, false) LUAU_FASTFLAGVARIABLE(LuauIfElseBranchTypeUnion, false) LUAU_FASTFLAGVARIABLE(LuauIfElseExpectedType2, false) +LUAU_FASTFLAGVARIABLE(LuauImmutableTypes, false) LUAU_FASTFLAGVARIABLE(LuauLengthOnCompositeType, false) LUAU_FASTFLAGVARIABLE(LuauNoSealedTypeMod, false) LUAU_FASTFLAGVARIABLE(LuauQuantifyInPlace2, false) LUAU_FASTFLAGVARIABLE(LuauSealExports, false) LUAU_FASTFLAGVARIABLE(LuauSingletonTypes, false) -LUAU_FASTFLAGVARIABLE(LuauDiscriminableUnions, false) +LUAU_FASTFLAGVARIABLE(LuauDiscriminableUnions2, false) LUAU_FASTFLAGVARIABLE(LuauTypeAliasDefaults, false) LUAU_FASTFLAGVARIABLE(LuauExpectedTypesOfProperties, false) LUAU_FASTFLAGVARIABLE(LuauErrorRecoveryType, false) @@ -47,7 +48,10 @@ LUAU_FASTFLAGVARIABLE(LuauProperTypeLevels, false) LUAU_FASTFLAGVARIABLE(LuauAscribeCorrectLevelToInferredProperitesOfFreeTables, false) LUAU_FASTFLAG(LuauUnionTagMatchFix) LUAU_FASTFLAGVARIABLE(LuauUnsealedTableLiteral, false) +LUAU_FASTFLAGVARIABLE(LuauTwoPassAliasDefinitionFix, false) LUAU_FASTFLAGVARIABLE(LuauAssertStripsFalsyTypes, false) +LUAU_FASTFLAGVARIABLE(LuauReturnAnyInsteadOfICE, false) // Eventually removed as false. +LUAU_FASTFLAGVARIABLE(LuauAnotherTypeLevelFix, false) namespace Luau { @@ -213,6 +217,11 @@ static bool isMetamethod(const Name& name) name == "__metatable" || name == "__eq" || name == "__lt" || name == "__le" || name == "__mode"; } +size_t HashBoolNamePair::operator()(const std::pair& pair) const +{ + return std::hash()(pair.first) ^ std::hash()(pair.second); +} + TypeChecker::TypeChecker(ModuleResolver* resolver, InternalErrorReporter* iceHandler) : resolver(resolver) , iceHandler(iceHandler) @@ -225,6 +234,7 @@ TypeChecker::TypeChecker(ModuleResolver* resolver, InternalErrorReporter* iceHan , anyType(getSingletonTypes().anyType) , optionalNumberType(getSingletonTypes().optionalNumberType) , anyTypePack(getSingletonTypes().anyTypePack) + , duplicateTypeAliases{{false, {}}} { globalScope = std::make_shared(globalTypes.addTypePack(TypePackVar{FreeTypePack{TypeLevel{}}})); @@ -291,6 +301,9 @@ ModulePtr TypeChecker::check(const SourceModule& module, Mode mode, std::optiona unifierState.skipCacheForType.clear(); } + if (FFlag::LuauTwoPassAliasDefinitionFix) + duplicateTypeAliases.clear(); + return std::move(currentModule); } @@ -496,6 +509,9 @@ LUAU_NOINLINE void TypeChecker::checkBlockTypeAliases(const ScopePtr& scope, std { if (const auto& typealias = stat->as()) { + if (FFlag::LuauTwoPassAliasDefinitionFix && typealias->name == Parser::errorName) + continue; + auto& bindings = typealias->exported ? scope->exportedTypeBindings : scope->privateTypeBindings; Name name = typealias->name.value; @@ -1176,6 +1192,10 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatTypeAlias& typealias // Once with forwardDeclare, and once without. Name name = typealias.name.value; + // If the alias is missing a name, we can't do anything with it. Ignore it. + if (FFlag::LuauTwoPassAliasDefinitionFix && name == Parser::errorName) + return; + std::optional binding; if (auto it = scope->exportedTypeBindings.find(name); it != scope->exportedTypeBindings.end()) binding = it->second; @@ -1192,6 +1212,8 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatTypeAlias& typealias reportError(TypeError{typealias.location, DuplicateTypeDefinition{name, location}}); bindingsMap[name] = TypeFun{binding->typeParams, binding->typePackParams, errorRecoveryType(anyType)}; + if (FFlag::LuauTwoPassAliasDefinitionFix) + duplicateTypeAliases.insert({typealias.exported, name}); } else { @@ -1211,6 +1233,11 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatTypeAlias& typealias } else { + // If the first pass failed (this should mean a duplicate definition), the second pass isn't going to be + // interesting. + if (FFlag::LuauTwoPassAliasDefinitionFix && duplicateTypeAliases.find({typealias.exported, name})) + return; + if (!binding) ice("Not predeclared"); @@ -1235,7 +1262,8 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatTypeAlias& typealias if (auto ttv = getMutable(follow(ty))) { // If the table is already named and we want to rename the type function, we have to bind new alias to a copy - if (ttv->name) + // Additionally, we can't modify types that come from other modules + if (ttv->name || (FFlag::LuauImmutableTypes && follow(ty)->owningArena != ¤tModule->internalTypes)) { bool sameTys = std::equal(ttv->instantiatedTypeParams.begin(), ttv->instantiatedTypeParams.end(), binding->typeParams.begin(), binding->typeParams.end(), [](auto&& itp, auto&& tp) { @@ -1247,7 +1275,7 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatTypeAlias& typealias }); // Copy can be skipped if this is an identical alias - if (ttv->name != name || !sameTys || !sameTps) + if ((FFlag::LuauImmutableTypes && !ttv->name) || ttv->name != name || !sameTys || !sameTps) { // This is a shallow clone, original recursive links to self are not updated TableTypeVar clone = TableTypeVar{ttv->props, ttv->indexer, ttv->level, ttv->state}; @@ -1279,9 +1307,17 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatTypeAlias& typealias } } else if (auto mtv = getMutable(follow(ty))) - mtv->syntheticName = name; + { + // We can't modify types that come from other modules + if (!FFlag::LuauImmutableTypes || follow(ty)->owningArena == ¤tModule->internalTypes) + mtv->syntheticName = name; + } - unify(ty, bindingsMap[name].type, typealias.location); + TypeId& bindingType = bindingsMap[name].type; + bool ok = unify(ty, bindingType, typealias.location); + + if (FFlag::LuauTwoPassAliasDefinitionFix && ok) + bindingType = ty; } } @@ -1564,7 +1600,12 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprCa else if (auto vtp = get(retPack)) return {vtp->ty, std::move(result.predicates)}; else if (get(retPack)) - ice("Unexpected abstract type pack!", expr.location); + { + if (FFlag::LuauReturnAnyInsteadOfICE) + return {anyType, std::move(result.predicates)}; + else + ice("Unexpected abstract type pack!", expr.location); + } else ice("Unknown TypePack type!", expr.location); } @@ -1614,11 +1655,23 @@ std::optional TypeChecker::getIndexTypeFromType( tablify(type); - const PrimitiveTypeVar* primitiveType = get(type); - if (primitiveType && primitiveType->type == PrimitiveTypeVar::String) + if (FFlag::LuauDiscriminableUnions2) { - if (std::optional mtIndex = findMetatableEntry(type, "__index", location)) + if (isString(type)) + { + std::optional mtIndex = findMetatableEntry(stringType, "__index", location); + LUAU_ASSERT(mtIndex); type = *mtIndex; + } + } + else + { + const PrimitiveTypeVar* primitiveType = get(type); + if (primitiveType && primitiveType->type == PrimitiveTypeVar::String) + { + if (std::optional mtIndex = findMetatableEntry(type, "__index", location)) + type = *mtIndex; + } } if (TableTypeVar* tableType = getMutableTableType(type)) @@ -2476,7 +2529,7 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprBi auto [rhsTy, rhsPredicates] = checkExpr(innerScope, *expr.right); - return {checkBinaryOperation(FFlag::LuauDiscriminableUnions ? scope : innerScope, expr, lhsTy, rhsTy), + return {checkBinaryOperation(FFlag::LuauDiscriminableUnions2 ? scope : innerScope, expr, lhsTy, rhsTy), {AndPredicate{std::move(lhsPredicates), std::move(rhsPredicates)}}}; } else if (expr.op == AstExprBinary::Or) @@ -2489,7 +2542,7 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprBi auto [rhsTy, rhsPredicates] = checkExpr(innerScope, *expr.right); // Because of C++, I'm not sure if lhsPredicates was not moved out by the time we call checkBinaryOperation. - TypeId result = checkBinaryOperation(FFlag::LuauDiscriminableUnions ? scope : innerScope, expr, lhsTy, rhsTy, lhsPredicates); + TypeId result = checkBinaryOperation(FFlag::LuauDiscriminableUnions2 ? scope : innerScope, expr, lhsTy, rhsTy, lhsPredicates); return {result, {OrPredicate{std::move(lhsPredicates), std::move(rhsPredicates)}}}; } else if (expr.op == AstExprBinary::CompareEq || expr.op == AstExprBinary::CompareNe) @@ -2497,8 +2550,8 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprBi if (auto predicate = tryGetTypeGuardPredicate(expr)) return {booleanType, {std::move(*predicate)}}; - ExprResult lhs = checkExpr(scope, *expr.left, std::nullopt, /*forceSingleton=*/FFlag::LuauDiscriminableUnions); - ExprResult rhs = checkExpr(scope, *expr.right, std::nullopt, /*forceSingleton=*/FFlag::LuauDiscriminableUnions); + ExprResult lhs = checkExpr(scope, *expr.left, std::nullopt, /*forceSingleton=*/FFlag::LuauDiscriminableUnions2); + ExprResult rhs = checkExpr(scope, *expr.right, std::nullopt, /*forceSingleton=*/FFlag::LuauDiscriminableUnions2); PredicateVec predicates; @@ -2785,12 +2838,16 @@ TypeId TypeChecker::checkLValueBinding(const ScopePtr& scope, const AstExprIndex } else if (exprTable->state == TableState::Unsealed || exprTable->state == TableState::Free) { - TypeId resultType = freshType(scope); + TypeId resultType = freshType(FFlag::LuauAnotherTypeLevelFix ? exprTable->level : scope->level); exprTable->indexer = TableIndexer{anyIfNonstrict(indexType), anyIfNonstrict(resultType)}; return resultType; } else { + /* + * If we use [] indexing to fetch a property from a sealed table that has no indexer, we have no idea if it will + * work, so we just mint a fresh type, return that, and hope for the best. + */ TypeId resultType = freshType(scope); return resultType; } @@ -4195,6 +4252,9 @@ TypeId TypeChecker::checkRequire(const ScopePtr& scope, const ModuleInfo& module return errorRecoveryType(scope); } + if (FFlag::LuauImmutableTypes) + return *moduleType; + SeenTypes seenTypes; SeenTypePacks seenTypePacks; CloneState cloneState; @@ -5446,7 +5506,7 @@ GenericTypeDefinitions TypeChecker::createGenericTypes(const ScopePtr& scope, st void TypeChecker::refineLValue(const LValue& lvalue, RefinementMap& refis, const ScopePtr& scope, TypeIdPredicate predicate) { - LUAU_ASSERT(FFlag::LuauDiscriminableUnions); + LUAU_ASSERT(FFlag::LuauDiscriminableUnions2); const LValue* target = &lvalue; std::optional key; // If set, we know we took the base of the lvalue path and should be walking down each option of the base's type. @@ -5659,7 +5719,7 @@ void TypeChecker::resolve(const TruthyPredicate& truthyP, ErrorVec& errVec, Refi return std::nullopt; }; - if (FFlag::LuauDiscriminableUnions) + if (FFlag::LuauDiscriminableUnions2) { std::optional ty = resolveLValue(refis, scope, truthyP.lvalue); if (ty && fromOr) @@ -5772,7 +5832,7 @@ void TypeChecker::resolve(const IsAPredicate& isaP, ErrorVec& errVec, Refinement return res; }; - if (FFlag::LuauDiscriminableUnions) + if (FFlag::LuauDiscriminableUnions2) { refineLValue(isaP.lvalue, refis, scope, predicate); } @@ -5847,7 +5907,7 @@ void TypeChecker::resolve(const TypeGuardPredicate& typeguardP, ErrorVec& errVec if (auto it = primitives.find(typeguardP.kind); it != primitives.end()) { - if (FFlag::LuauDiscriminableUnions) + if (FFlag::LuauDiscriminableUnions2) { refineLValue(typeguardP.lvalue, refis, scope, it->second(sense)); return; @@ -5869,7 +5929,7 @@ void TypeChecker::resolve(const TypeGuardPredicate& typeguardP, ErrorVec& errVec } auto fail = [&](const TypeErrorData& err) { - if (!FFlag::LuauDiscriminableUnions) + if (!FFlag::LuauDiscriminableUnions2) errVec.push_back(TypeError{typeguardP.location, err}); addRefinement(refis, typeguardP.lvalue, errorRecoveryType(scope)); }; @@ -5901,7 +5961,7 @@ void TypeChecker::resolve(const EqPredicate& eqP, ErrorVec& errVec, RefinementMa return {ty}; }; - if (FFlag::LuauDiscriminableUnions) + if (FFlag::LuauDiscriminableUnions2) { std::vector rhs = options(eqP.type); diff --git a/Analysis/src/TypeVar.cpp b/Analysis/src/TypeVar.cpp index 2321eafd..7e438e31 100644 --- a/Analysis/src/TypeVar.cpp +++ b/Analysis/src/TypeVar.cpp @@ -28,6 +28,7 @@ LUAU_FASTFLAGVARIABLE(LuauMetatableAreEqualRecursion, false) LUAU_FASTFLAGVARIABLE(LuauRefactorTypeVarQuestions, false) LUAU_FASTFLAG(LuauErrorRecoveryType) LUAU_FASTFLAG(LuauUnionTagMatchFix) +LUAU_FASTFLAG(LuauDiscriminableUnions2) namespace Luau { @@ -393,7 +394,8 @@ bool hasLength(TypeId ty, DenseHashSet& seen, int* recursionCount) if (seen.contains(ty)) return true; - if (isPrim(ty, PrimitiveTypeVar::String) || get(ty) || get(ty) || get(ty)) + bool isStr = FFlag::LuauDiscriminableUnions2 ? isString(ty) : isPrim(ty, PrimitiveTypeVar::String); + if (isStr || get(ty) || get(ty) || get(ty)) return true; if (auto uty = get(ty)) diff --git a/Analysis/src/Unifier.cpp b/Analysis/src/Unifier.cpp index 89e4ae23..a8ad5159 100644 --- a/Analysis/src/Unifier.cpp +++ b/Analysis/src/Unifier.cpp @@ -15,6 +15,7 @@ LUAU_FASTINT(LuauTypeInferRecursionLimit); LUAU_FASTINT(LuauTypeInferTypePackLoopLimit); LUAU_FASTFLAGVARIABLE(LuauCommittingTxnLogFreeTpPromote, false) +LUAU_FASTFLAG(LuauImmutableTypes) LUAU_FASTFLAG(LuauUseCommittingTxnLog) LUAU_FASTINTVARIABLE(LuauTypeInferIterationLimit, 2000); LUAU_FASTFLAGVARIABLE(LuauTableSubtypingVariance2, false); @@ -24,6 +25,7 @@ LUAU_FASTFLAG(LuauErrorRecoveryType); LUAU_FASTFLAG(LuauProperTypeLevels); LUAU_FASTFLAGVARIABLE(LuauUnifyPackTails, false) LUAU_FASTFLAGVARIABLE(LuauUnionTagMatchFix, false) +LUAU_FASTFLAGVARIABLE(LuauFollowWithCommittingTxnLogInAnyUnification, false) namespace Luau { @@ -32,11 +34,13 @@ struct PromoteTypeLevels { DEPRECATED_TxnLog& DEPRECATED_log; TxnLog& log; + const TypeArena* typeArena = nullptr; TypeLevel minLevel; - explicit PromoteTypeLevels(DEPRECATED_TxnLog& DEPRECATED_log, TxnLog& log, TypeLevel minLevel) + explicit PromoteTypeLevels(DEPRECATED_TxnLog& DEPRECATED_log, TxnLog& log, const TypeArena* typeArena, TypeLevel minLevel) : DEPRECATED_log(DEPRECATED_log) , log(log) + , typeArena(typeArena) , minLevel(minLevel) { } @@ -65,8 +69,12 @@ struct PromoteTypeLevels } template - bool operator()(TID, const T&) + bool operator()(TID ty, const T&) { + // Type levels of types from other modules are already global, so we don't need to promote anything inside + if (FFlag::LuauImmutableTypes && ty->owningArena != typeArena) + return false; + return true; } @@ -83,12 +91,20 @@ struct PromoteTypeLevels bool operator()(TypeId ty, const FunctionTypeVar&) { + // Type levels of types from other modules are already global, so we don't need to promote anything inside + if (FFlag::LuauImmutableTypes && ty->owningArena != typeArena) + return false; + promote(ty, FFlag::LuauUseCommittingTxnLog ? log.getMutable(ty) : getMutable(ty)); return true; } bool operator()(TypeId ty, const TableTypeVar& ttv) { + // Type levels of types from other modules are already global, so we don't need to promote anything inside + if (FFlag::LuauImmutableTypes && ty->owningArena != typeArena) + return false; + if (ttv.state != TableState::Free && ttv.state != TableState::Generic) return true; @@ -108,24 +124,33 @@ struct PromoteTypeLevels } }; -void promoteTypeLevels(DEPRECATED_TxnLog& DEPRECATED_log, TxnLog& log, TypeLevel minLevel, TypeId ty) +void promoteTypeLevels(DEPRECATED_TxnLog& DEPRECATED_log, TxnLog& log, const TypeArena* typeArena, TypeLevel minLevel, TypeId ty) { - PromoteTypeLevels ptl{DEPRECATED_log, log, minLevel}; + // Type levels of types from other modules are already global, so we don't need to promote anything inside + if (FFlag::LuauImmutableTypes && ty->owningArena != typeArena) + return; + + PromoteTypeLevels ptl{DEPRECATED_log, log, typeArena, minLevel}; DenseHashSet seen{nullptr}; visitTypeVarOnce(ty, ptl, seen); } -void promoteTypeLevels(DEPRECATED_TxnLog& DEPRECATED_log, TxnLog& log, TypeLevel minLevel, TypePackId tp) +void promoteTypeLevels(DEPRECATED_TxnLog& DEPRECATED_log, TxnLog& log, const TypeArena* typeArena, TypeLevel minLevel, TypePackId tp) { - PromoteTypeLevels ptl{DEPRECATED_log, log, minLevel}; + // Type levels of types from other modules are already global, so we don't need to promote anything inside + if (FFlag::LuauImmutableTypes && tp->owningArena != typeArena) + return; + + PromoteTypeLevels ptl{DEPRECATED_log, log, typeArena, minLevel}; DenseHashSet seen{nullptr}; visitTypeVarOnce(tp, ptl, seen); } struct SkipCacheForType { - SkipCacheForType(const DenseHashMap& skipCacheForType) + SkipCacheForType(const DenseHashMap& skipCacheForType, const TypeArena* typeArena) : skipCacheForType(skipCacheForType) + , typeArena(typeArena) { } @@ -152,6 +177,10 @@ struct SkipCacheForType bool operator()(TypeId ty, const TableTypeVar&) { + // Types from other modules don't contain mutable elements and are ok to cache + if (FFlag::LuauImmutableTypes && ty->owningArena != typeArena) + return false; + TableTypeVar& ttv = *getMutable(ty); if (ttv.boundTo) @@ -172,6 +201,10 @@ struct SkipCacheForType template bool operator()(TypeId ty, const T& t) { + // Types from other modules don't contain mutable elements and are ok to cache + if (FFlag::LuauImmutableTypes && ty->owningArena != typeArena) + return false; + const bool* prev = skipCacheForType.find(ty); if (prev && *prev) @@ -184,8 +217,12 @@ struct SkipCacheForType } template - bool operator()(TypePackId, const T&) + bool operator()(TypePackId tp, const T&) { + // Types from other modules don't contain mutable elements and are ok to cache + if (FFlag::LuauImmutableTypes && tp->owningArena != typeArena) + return false; + return true; } @@ -208,6 +245,7 @@ struct SkipCacheForType } const DenseHashMap& skipCacheForType; + const TypeArena* typeArena = nullptr; bool result = false; }; @@ -422,13 +460,13 @@ void Unifier::tryUnify_(TypeId subTy, TypeId superTy, bool isFunctionCall, bool { if (FFlag::LuauUseCommittingTxnLog) { - promoteTypeLevels(DEPRECATED_log, log, superLevel, subTy); + promoteTypeLevels(DEPRECATED_log, log, types, superLevel, subTy); log.replace(superTy, BoundTypeVar(subTy)); } else { if (FFlag::LuauProperTypeLevels) - promoteTypeLevels(DEPRECATED_log, log, superLevel, subTy); + promoteTypeLevels(DEPRECATED_log, log, types, superLevel, subTy); else if (auto subLevel = getMutableLevel(subTy)) { if (!subLevel->subsumes(superFree->level)) @@ -466,13 +504,13 @@ void Unifier::tryUnify_(TypeId subTy, TypeId superTy, bool isFunctionCall, bool { if (FFlag::LuauUseCommittingTxnLog) { - promoteTypeLevels(DEPRECATED_log, log, subLevel, superTy); + promoteTypeLevels(DEPRECATED_log, log, types, subLevel, superTy); log.replace(subTy, BoundTypeVar(superTy)); } else { if (FFlag::LuauProperTypeLevels) - promoteTypeLevels(DEPRECATED_log, log, subLevel, superTy); + promoteTypeLevels(DEPRECATED_log, log, types, subLevel, superTy); else if (auto superLevel = getMutableLevel(superTy)) { if (!superLevel->subsumes(subFree->level)) @@ -849,7 +887,7 @@ void Unifier::cacheResult(TypeId subTy, TypeId superTy) return; auto skipCacheFor = [this](TypeId ty) { - SkipCacheForType visitor{sharedState.skipCacheForType}; + SkipCacheForType visitor{sharedState.skipCacheForType, types}; visitTypeVarOnce(ty, visitor, sharedState.seenAny); sharedState.skipCacheForType[ty] = visitor.result; @@ -1637,32 +1675,35 @@ void Unifier::tryUnifyFunctions(TypeId subTy, TypeId superTy, bool isFunctionCal tryUnify_(subFunction->retType, superFunction->retType); } - if (FFlag::LuauUseCommittingTxnLog) + if (!FFlag::LuauImmutableTypes) { - if (superFunction->definition && !subFunction->definition && !subTy->persistent) + if (FFlag::LuauUseCommittingTxnLog) { - PendingType* newSubTy = log.queue(subTy); - FunctionTypeVar* newSubFtv = getMutable(newSubTy); - LUAU_ASSERT(newSubFtv); - newSubFtv->definition = superFunction->definition; + if (superFunction->definition && !subFunction->definition && !subTy->persistent) + { + PendingType* newSubTy = log.queue(subTy); + FunctionTypeVar* newSubFtv = getMutable(newSubTy); + LUAU_ASSERT(newSubFtv); + newSubFtv->definition = superFunction->definition; + } + else if (!superFunction->definition && subFunction->definition && !superTy->persistent) + { + PendingType* newSuperTy = log.queue(superTy); + FunctionTypeVar* newSuperFtv = getMutable(newSuperTy); + LUAU_ASSERT(newSuperFtv); + newSuperFtv->definition = subFunction->definition; + } } - else if (!superFunction->definition && subFunction->definition && !superTy->persistent) + else { - PendingType* newSuperTy = log.queue(superTy); - FunctionTypeVar* newSuperFtv = getMutable(newSuperTy); - LUAU_ASSERT(newSuperFtv); - newSuperFtv->definition = subFunction->definition; - } - } - else - { - if (superFunction->definition && !subFunction->definition && !subTy->persistent) - { - subFunction->definition = superFunction->definition; - } - else if (!superFunction->definition && subFunction->definition && !superTy->persistent) - { - superFunction->definition = subFunction->definition; + if (superFunction->definition && !subFunction->definition && !subTy->persistent) + { + subFunction->definition = superFunction->definition; + } + else if (!superFunction->definition && subFunction->definition && !superTy->persistent) + { + superFunction->definition = subFunction->definition; + } } } @@ -2631,7 +2672,7 @@ static void queueTypePack(std::vector& queue, DenseHashSet& { while (true) { - a = follow(a); + a = FFlag::LuauFollowWithCommittingTxnLogInAnyUnification ? state.log.follow(a) : follow(a); if (seenTypePacks.find(a)) break; @@ -2738,7 +2779,7 @@ void Unifier::tryUnifyVariadics(TypePackId subTp, TypePackId superTp, bool rever } static void tryUnifyWithAny(std::vector& queue, Unifier& state, DenseHashSet& seen, DenseHashSet& seenTypePacks, - TypeId anyType, TypePackId anyTypePack) + const TypeArena* typeArena, TypeId anyType, TypePackId anyTypePack) { while (!queue.empty()) { @@ -2746,8 +2787,14 @@ static void tryUnifyWithAny(std::vector& queue, Unifier& state, DenseHas { TypeId ty = state.log.follow(queue.back()); queue.pop_back(); + + // Types from other modules don't have free types + if (FFlag::LuauImmutableTypes && ty->owningArena != typeArena) + continue; + if (seen.find(ty)) continue; + seen.insert(ty); if (state.log.getMutable(ty)) @@ -2853,7 +2900,7 @@ void Unifier::tryUnifyWithAny(TypeId subTy, TypeId anyTy) sharedState.tempSeenTy.clear(); sharedState.tempSeenTp.clear(); - Luau::tryUnifyWithAny(queue, *this, sharedState.tempSeenTy, sharedState.tempSeenTp, getSingletonTypes().anyType, anyTP); + Luau::tryUnifyWithAny(queue, *this, sharedState.tempSeenTy, sharedState.tempSeenTp, types, getSingletonTypes().anyType, anyTP); } void Unifier::tryUnifyWithAny(TypePackId subTy, TypePackId anyTp) @@ -2869,7 +2916,7 @@ void Unifier::tryUnifyWithAny(TypePackId subTy, TypePackId anyTp) queueTypePack(queue, sharedState.tempSeenTp, *this, subTy, anyTp); - Luau::tryUnifyWithAny(queue, *this, sharedState.tempSeenTy, sharedState.tempSeenTp, anyTy, anyTp); + Luau::tryUnifyWithAny(queue, *this, sharedState.tempSeenTy, sharedState.tempSeenTp, types, anyTy, anyTp); } std::optional Unifier::findTablePropertyRespectingMeta(TypeId lhsType, Name name) diff --git a/Ast/src/Parser.cpp b/Ast/src/Parser.cpp index f559e2e0..30b32f91 100644 --- a/Ast/src/Parser.cpp +++ b/Ast/src/Parser.cpp @@ -1133,7 +1133,7 @@ AstTypePack* Parser::parseTypeList(TempVector& result, TempVector(ic_completion_arg(cenv)); std::string_view lookup = editBuffer; char lastSep = 0; @@ -276,7 +278,7 @@ static void completeIndexer(ic_completion_env_t* cenv, const char* editBuffer) // Add an opening paren for function calls by default. completion += "("; } - ic_add_completion_ex(cenv, completion.data(), key.data(), nullptr); + addCompletionCallback(completion, std::string(key)); } } lua_pop(L, 1); @@ -295,10 +297,11 @@ static void completeIndexer(ic_completion_env_t* cenv, const char* editBuffer) { // Replace the string object with the string class to perform further lookups of string functions // Note: We retrieve the string class from _G to prevent issues if the user assigns to `string`. + lua_pop(L, 1); // Pop the string instance lua_getglobal(L, "_G"); lua_pushlstring(L, "string", 6); lua_rawget(L, -2); - lua_remove(L, -2); + lua_remove(L, -2); // Remove the global table LUAU_ASSERT(lua_istable(L, -1)); } else if (!lua_istable(L, -1)) @@ -312,6 +315,26 @@ static void completeIndexer(ic_completion_env_t* cenv, const char* editBuffer) lua_pop(L, 1); } +void getCompletions(lua_State* L, const std::string& editBuffer, const AddCompletionCallback& addCompletionCallback) +{ + // look the value up in current global table first + lua_pushvalue(L, LUA_GLOBALSINDEX); + completeIndexer(L, editBuffer, addCompletionCallback); + + // and in actual global table after that + lua_getglobal(L, "_G"); + completeIndexer(L, editBuffer, addCompletionCallback); +} + +static void icGetCompletions(ic_completion_env_t* cenv, const char* editBuffer) +{ + auto* L = reinterpret_cast(ic_completion_arg(cenv)); + + getCompletions(L, std::string(editBuffer), [cenv](const std::string& completion, const std::string& display) { + ic_add_completion_ex(cenv, completion.data(), display.data(), nullptr); + }); +} + static bool isMethodOrFunctionChar(const char* s, long len) { char c = *s; @@ -320,15 +343,7 @@ static bool isMethodOrFunctionChar(const char* s, long len) static void completeRepl(ic_completion_env_t* cenv, const char* editBuffer) { - auto* L = reinterpret_cast(ic_completion_arg(cenv)); - - // look the value up in current global table first - lua_pushvalue(L, LUA_GLOBALSINDEX); - ic_complete_word(cenv, editBuffer, completeIndexer, isMethodOrFunctionChar); - - // and in actual global table after that - lua_getglobal(L, "_G"); - ic_complete_word(cenv, editBuffer, completeIndexer, isMethodOrFunctionChar); + ic_complete_word(cenv, editBuffer, icGetCompletions, isMethodOrFunctionChar); } struct LinenoiseScopedHistory @@ -372,19 +387,20 @@ static void runReplImpl(lua_State* L) for (;;) { - const char* line = ic_readline(buffer.empty() ? "" : ">"); + const char* prompt = buffer.empty() ? "" : ">"; + std::unique_ptr line(ic_readline(prompt), free); if (!line) break; - if (buffer.empty() && runCode(L, std::string("return ") + line) == std::string()) + if (buffer.empty() && runCode(L, std::string("return ") + line.get()) == std::string()) { - ic_history_add(line); + ic_history_add(line.get()); continue; } if (!buffer.empty()) buffer += "\n"; - buffer += line; + buffer += line.get(); std::string error = runCode(L, buffer); @@ -400,7 +416,6 @@ static void runReplImpl(lua_State* L) ic_history_add(buffer.c_str()); buffer.clear(); - free((void*)line); } } @@ -504,7 +519,7 @@ static bool compileFile(const char* name, CompileFormat format) if (format == CompileFormat::Text) { - bcb.setDumpFlags(Luau::BytecodeBuilder::Dump_Code | Luau::BytecodeBuilder::Dump_Source); + bcb.setDumpFlags(Luau::BytecodeBuilder::Dump_Code | Luau::BytecodeBuilder::Dump_Source | Luau::BytecodeBuilder::Dump_Locals); bcb.setDumpSource(*source); } @@ -549,7 +564,8 @@ static void displayHelp(const char* argv0) printf(" --coverage: collect code coverage while running the code and output results to coverage.out\n"); printf(" -h, --help: Display this usage message.\n"); printf(" -i, --interactive: Run an interactive REPL after executing the last script specified.\n"); - printf(" -O: use compiler optimization level (n=0-2).\n"); + printf(" -O: compile with optimization level n (default 1, n should be between 0 and 2).\n"); + printf(" -g: compile with debug level n (default 1, n should be between 0 and 2).\n"); printf(" --profile[=N]: profile the code using N Hz sampling (default 10000) and output results to profile.out\n"); printf(" --timetrace: record compiler time tracing information into trace.json\n"); } @@ -620,6 +636,16 @@ int replMain(int argc, char** argv) } globalOptions.optimizationLevel = level; } + else if (strncmp(argv[i], "-g", 2) == 0) + { + int level = atoi(argv[i] + 2); + if (level < 0 || level > 2) + { + fprintf(stderr, "Error: Debug level must be between 0 and 2 inclusive.\n"); + return 1; + } + globalOptions.debugLevel = level; + } else if (strcmp(argv[i], "--profile") == 0) { profile = 10000; // default to 10 KHz diff --git a/CLI/Repl.h b/CLI/Repl.h index 11a077ae..cd54b7e0 100644 --- a/CLI/Repl.h +++ b/CLI/Repl.h @@ -3,10 +3,15 @@ #include "lua.h" +#include #include +using AddCompletionCallback = std::function; + // Note: These are internal functions which are being exposed in a header // so they can be included by unit tests. -int replMain(int argc, char** argv); void setupState(lua_State* L); std::string runCode(lua_State* L, const std::string& source); +void getCompletions(lua_State* L, const std::string& editBuffer, const AddCompletionCallback& addCompletionCallback); + +int replMain(int argc, char** argv); diff --git a/Compiler/src/BytecodeBuilder.cpp b/Compiler/src/BytecodeBuilder.cpp index e6d02454..09f06b68 100644 --- a/Compiler/src/BytecodeBuilder.cpp +++ b/Compiler/src/BytecodeBuilder.cpp @@ -6,8 +6,6 @@ #include #include -LUAU_FASTFLAGVARIABLE(LuauBytecodeV2Write, false) - namespace Luau { @@ -510,7 +508,7 @@ uint32_t BytecodeBuilder::getDebugPC() const void BytecodeBuilder::finalize() { LUAU_ASSERT(bytecode.empty()); - bytecode = char(FFlag::LuauBytecodeV2Write ? LBC_VERSION_FUTURE : LBC_VERSION); + bytecode = char(LBC_VERSION_FUTURE); writeStringTable(bytecode); @@ -611,9 +609,7 @@ void BytecodeBuilder::writeFunction(std::string& ss, uint32_t id) const writeVarInt(ss, child); // debug info - if (FFlag::LuauBytecodeV2Write) - writeVarInt(ss, func.debuglinedefined); - + writeVarInt(ss, func.debuglinedefined); writeVarInt(ss, func.debugname); bool hasLines = true; diff --git a/Compiler/src/Compiler.cpp b/Compiler/src/Compiler.cpp index e4253adc..656a9926 100644 --- a/Compiler/src/Compiler.cpp +++ b/Compiler/src/Compiler.cpp @@ -15,7 +15,6 @@ #include #include -LUAU_FASTFLAGVARIABLE(LuauCompileTableIndexOpt, false) LUAU_FASTFLAG(LuauCompileSelectBuiltin2) namespace Luau @@ -1182,18 +1181,9 @@ struct Compiler const AstExprTable::Item& item = expr->items.data[i]; LUAU_ASSERT(item.key); // no list portion => all items have keys - if (FFlag::LuauCompileTableIndexOpt) - { - const Constant* ckey = constants.find(item.key); + const Constant* ckey = constants.find(item.key); - indexSize += (ckey && ckey->type == Constant::Type_Number && ckey->valueNumber == double(indexSize + 1)); - } - else - { - AstExprConstantNumber* ckey = item.key->as(); - - indexSize += (ckey && ckey->value == double(indexSize + 1)); - } + indexSize += (ckey && ckey->type == Constant::Type_Number && ckey->valueNumber == double(indexSize + 1)); } // we only perform the optimization if we don't have any other []-keys @@ -1295,43 +1285,10 @@ struct Compiler { RegScope rsi(this); - if (FFlag::LuauCompileTableIndexOpt) - { - LValue lv = compileLValueIndex(reg, key, rsi); - uint8_t rv = compileExprAuto(value, rsi); + LValue lv = compileLValueIndex(reg, key, rsi); + uint8_t rv = compileExprAuto(value, rsi); - compileAssign(lv, rv); - } - else - { - // Optimization: use SETTABLEKS/SETTABLEN for literal keys, this happens often as part of usual table construction syntax - if (AstExprConstantString* ckey = key->as()) - { - BytecodeBuilder::StringRef cname = sref(ckey->value); - int32_t cid = bytecode.addConstantString(cname); - if (cid < 0) - CompileError::raise(expr->location, "Exceeded constant limit; simplify the code to compile"); - - uint8_t rv = compileExprAuto(value, rsi); - - bytecode.emitABC(LOP_SETTABLEKS, rv, reg, uint8_t(BytecodeBuilder::getStringHash(cname))); - bytecode.emitAux(cid); - } - else if (AstExprConstantNumber* ckey = key->as(); - ckey && ckey->value >= 1 && ckey->value <= 256 && double(int(ckey->value)) == ckey->value) - { - uint8_t rv = compileExprAuto(value, rsi); - - bytecode.emitABC(LOP_SETTABLEN, rv, reg, uint8_t(int(ckey->value) - 1)); - } - else - { - uint8_t rk = compileExprAuto(key, rsi); - uint8_t rv = compileExprAuto(value, rsi); - - bytecode.emitABC(LOP_SETTABLE, rv, reg, rk); - } - } + compileAssign(lv, rv); } // items without a key are set using SETLIST so that we can initialize large arrays quickly else @@ -1439,8 +1396,7 @@ struct Compiler uint8_t rt = compileExprAuto(expr->expr, rs); uint8_t i = uint8_t(int(cv->valueNumber) - 1); - if (FFlag::LuauCompileTableIndexOpt) - setDebugLine(expr->index); + setDebugLine(expr->index); bytecode.emitABC(LOP_GETTABLEN, target, rt, i); } @@ -1453,8 +1409,7 @@ struct Compiler if (cid < 0) CompileError::raise(expr->location, "Exceeded constant limit; simplify the code to compile"); - if (FFlag::LuauCompileTableIndexOpt) - setDebugLine(expr->index); + setDebugLine(expr->index); bytecode.emitABC(LOP_GETTABLEKS, target, rt, uint8_t(BytecodeBuilder::getStringHash(iname))); bytecode.emitAux(cid); @@ -1853,8 +1808,7 @@ struct Compiler void compileLValueUse(const LValue& lv, uint8_t reg, bool set) { - if (FFlag::LuauCompileTableIndexOpt) - setDebugLine(lv.location); + setDebugLine(lv.location); switch (lv.kind) { diff --git a/Sources.cmake b/Sources.cmake index 4ab1d98e..773f6f35 100644 --- a/Sources.cmake +++ b/Sources.cmake @@ -193,11 +193,12 @@ if(TARGET Luau.Analyze.CLI) CLI/Analyze.cpp) endif() -if (TARGET Luau.Ast.CLI) +if(TARGET Luau.Ast.CLI) target_sources(Luau.Ast.CLI PRIVATE + CLI/Ast.cpp CLI/FileUtils.h CLI/FileUtils.cpp - CLI/Ast.cpp) + ) endif() if(TARGET Luau.UnitTest) diff --git a/VM/src/lbuiltins.cpp b/VM/src/lbuiltins.cpp index ecc14e87..718d387d 100644 --- a/VM/src/lbuiltins.cpp +++ b/VM/src/lbuiltins.cpp @@ -1098,7 +1098,7 @@ static int luauF_select(lua_State* L, StkId res, TValue* arg0, int nresults, Stk int i = int(nvalue(arg0)); // i >= 1 && i <= n - if (unsigned(i - 1) <= unsigned(n)) + if (unsigned(i - 1) < unsigned(n)) { setobj2s(L, res, L->base - n + (i - 1)); return 1; diff --git a/VM/src/lgcdebug.cpp b/VM/src/lgcdebug.cpp index 906fb0d0..ce196520 100644 --- a/VM/src/lgcdebug.cpp +++ b/VM/src/lgcdebug.cpp @@ -250,6 +250,8 @@ void luaC_validate(lua_State* L) if (FFlag::LuauGcPagedSweep) { + validategco(L, NULL, obj2gco(g->mainthread)); + luaM_visitgco(L, L, validategco); } else @@ -565,6 +567,8 @@ void luaC_dump(lua_State* L, void* file, const char* (*categoryName)(lua_State* if (FFlag::LuauGcPagedSweep) { + dumpgco(f, NULL, obj2gco(g->mainthread)); + luaM_visitgco(L, f, dumpgco); } else diff --git a/VM/src/lmem.cpp b/VM/src/lmem.cpp index de85cf59..19617b8c 100644 --- a/VM/src/lmem.cpp +++ b/VM/src/lmem.cpp @@ -8,6 +8,76 @@ #include +/* + * Luau heap uses a size-segregated page structure, with individual pages and large allocations + * allocated using system heap (via frealloc callback). + * + * frealloc callback serves as a general, if slow, allocation callback that can allocate, free or + * resize allocations: + * + * void* frealloc(void* ud, void* ptr, size_t oldsize, size_t newsize); + * + * frealloc(ud, NULL, 0, x) creates a new block of size x + * frealloc(ud, p, x, 0) frees the block p (must return NULL) + * frealloc(ud, NULL, 0, 0) does nothing, equivalent to free(NULL) + * + * frealloc returns NULL if it cannot create or reallocate the area + * (any reallocation to an equal or smaller size cannot fail!) + * + * On top of this, Luau implements heap storage which is split into two types of allocations: + * + * - GCO, short for "garbage collected objects" + * - other objects (for example, arrays stored inside table objects) + * + * The heap layout for these two allocation types is a bit different. + * + * All GCO are allocated in pages, which is a block of memory of ~16K in size that has a page header + * (lua_Page). Each page contains 1..N blocks of the same size, where N is selected to fill the page + * completely. This amortizes the allocation cost and increases locality. Each GCO block starts with + * the GC header (GCheader) which contains the object type, mark bits and other GC metadata. If the + * GCO block is free (not used), then it must have the type set to TNIL; in this case the block can + * be part of the per-page free list, the link for that list is stored after the header (freegcolink). + * + * Importantly, the GCO block doesn't have any back references to the page it's allocated in, so it's + * impossible to free it in isolation - GCO blocks are freed by sweeping the pages they belong to, + * using luaM_freegco which must specify the page; this is called by page sweeper that traverses the + * entire page's worth of objects. For this reason it's also important that freed GCO blocks keep the + * GC header intact and accessible (with type = NIL) so that the sweeper can access it. + * + * Some GCOs are too large to fit in a 16K page without excessive fragmentation (the size threshold is + * currently 512 bytes); in this case, we allocate a dedicated small page with just a single block's worth + * storage space, but that requires allocating an extra page header. In effect large GCOs are a little bit + * less memory efficient, but this allows us to uniformly sweep small and large GCOs using page lists. + * + * All GCO pages are linked in a large intrusive linked list (global_State::allgcopages). Additionally, + * for each block size there's a page free list that contains pages that have at least one free block + * (global_State::freegcopages). This free list is used to make sure object allocation is O(1). + * + * Compared to GCOs, regular allocations have two important differences: they can be freed in isolation, + * and they don't start with a GC header. Because of this, each allocation is prefixed with block metadata, + * which contains the pointer to the page for allocated blocks, and the pointer to the next free block + * inside the page for freed blocks. + * For regular allocations that are too large to fit in a page (using the same threshold of 512 bytes), + * we don't allocate a separate page, instead simply using frealloc to allocate a vanilla block of memory. + * + * Just like GCO pages, we store a page free list (global_State::freepages) that allows O(1) allocation; + * there is no global list for non-GCO pages since we never need to traverse them directly. + * + * In both cases, we pick the page by computing the size class from the block size which rounds the block + * size up to reduce the chance that we'll allocate pages that have very few allocated blocks. The size + * class strategy is determined by SizeClassConfig constructor. + * + * Note that when the last block in a page is freed, we immediately free the page with frealloc - the + * memory manager doesn't currently attempt to keep unused memory around. This can result in excessive + * allocation traffic and can be mitigated by adding a page cache in the future. + * + * For both GCO and non-GCO pages, the per-page block allocation combines bump pointer style allocation + * (lua_Page::freeNext) and per-page free list (lua_Page::freeList). We use the bump allocator to allocate + * the contents of the page, and the free list for further reuse; this allows shorter page setup times + * which results in less variance between allocation cost, as well as tighter sweep bounds for newly + * allocated pages. + */ + LUAU_FASTFLAG(LuauGcPagedSweep) #ifndef __has_feature @@ -56,6 +126,7 @@ static_assert(offsetof(GCObject, ts) == 0, "TString data must be located at the const size_t kSizeClasses = LUA_SIZECLASSES; const size_t kMaxSmallSize = 512; const size_t kPageSize = 16 * 1024 - 24; // slightly under 16KB since that results in less fragmentation due to heap metadata + const size_t kBlockHeader = sizeof(double) > sizeof(void*) ? sizeof(double) : sizeof(void*); // suitable for aligning double & void* on all platforms const size_t kGCOLinkOffset = (sizeof(GCheader) + sizeof(void*) - 1) & ~(sizeof(void*) - 1); // GCO pages contain freelist links after the GC header @@ -107,24 +178,6 @@ const SizeClassConfig kSizeClassConfig; #define metadata(block) (*(void**)(block)) #define freegcolink(block) (*(void**)((char*)block + kGCOLinkOffset)) -/* -** About the realloc function: -** void * frealloc (void *ud, void *ptr, size_t osize, size_t nsize); -** (`osize' is the old size, `nsize' is the new size) -** -** Lua ensures that (ptr == NULL) iff (osize == 0). -** -** * frealloc(ud, NULL, 0, x) creates a new block of size `x' -** -** * frealloc(ud, p, x, 0) frees the block `p' -** (in this specific case, frealloc must return NULL). -** particularly, frealloc(ud, NULL, 0, 0) does nothing -** (which is equivalent to free(NULL) in ANSI C) -** -** frealloc returns NULL if it cannot create or reallocate the area -** (any reallocation to an equal or smaller size cannot fail!) -*/ - struct lua_Page { // list of pages with free blocks @@ -135,13 +188,12 @@ struct lua_Page lua_Page* gcolistprev; lua_Page* gcolistnext; - int busyBlocks; - int blockSize; + int pageSize; // page size in bytes, including page header + int blockSize; // block size in bytes, including block header (for non-GCO) - void* freeList; - int freeNext; - - int pageSize; + void* freeList; // next free block in this page; linked with metadata()/freegcolink() + int freeNext; // next free block offset in this page, in bytes; when negative, freeList is used instead + int busyBlocks; // number of blocks allocated out of this page union { @@ -177,7 +229,7 @@ static lua_Page* newpageold(lua_State* L, uint8_t sizeClass) page->gcolistprev = NULL; page->gcolistnext = NULL; - page->busyBlocks = 0; + page->pageSize = kPageSize; page->blockSize = blockSize; // note: we start with the last block in the page and move downward @@ -185,6 +237,7 @@ static lua_Page* newpageold(lua_State* L, uint8_t sizeClass) // additionally, GC stores objects in singly linked lists, and this way the GC lists end up in increasing pointer order page->freeList = NULL; page->freeNext = (blockCount - 1) * blockSize; + page->busyBlocks = 0; // prepend a page to page freelist (which is empty because we only ever allocate a new page when it is!) LUAU_ASSERT(!g->freepages[sizeClass]); @@ -214,7 +267,7 @@ static lua_Page* newpage(lua_State* L, lua_Page** gcopageset, int pageSize, int page->gcolistprev = NULL; page->gcolistnext = NULL; - page->busyBlocks = 0; + page->pageSize = pageSize; page->blockSize = blockSize; // note: we start with the last block in the page and move downward @@ -222,8 +275,7 @@ static lua_Page* newpage(lua_State* L, lua_Page** gcopageset, int pageSize, int // additionally, GC stores objects in singly linked lists, and this way the GC lists end up in increasing pointer order page->freeList = NULL; page->freeNext = (blockCount - 1) * blockSize; - - page->pageSize = pageSize; + page->busyBlocks = 0; if (gcopageset) { @@ -406,8 +458,7 @@ static void* newgcoblock(lua_State* L, int sizeClass) page->next = NULL; } - // the user data is right after the metadata - return (char*)block; + return block; } static void freeblock(lua_State* L, int sizeClass, void* block) @@ -421,6 +472,7 @@ static void freeblock(lua_State* L, int sizeClass, void* block) lua_Page* page = (lua_Page*)metadata(block); LUAU_ASSERT(page && page->busyBlocks > 0); LUAU_ASSERT(size_t(page->blockSize) == kSizeClassConfig.sizeOfClass[sizeClass] + kBlockHeader); + LUAU_ASSERT(block >= page->data && block < (char*)page + page->pageSize); // if the page wasn't in the page free list, it should be now since it got a block! if (!page->freeList && page->freeNext < 0) @@ -455,6 +507,9 @@ static void freeblock(lua_State* L, int sizeClass, void* block) static void freegcoblock(lua_State* L, int sizeClass, void* block, lua_Page* page) { LUAU_ASSERT(FFlag::LuauGcPagedSweep); + LUAU_ASSERT(page && page->busyBlocks > 0); + LUAU_ASSERT(page->blockSize == kSizeClassConfig.sizeOfClass[sizeClass]); + LUAU_ASSERT(block >= page->data && block < (char*)page + page->pageSize); global_State* g = L->global; @@ -575,6 +630,8 @@ void luaM_freegco_(lua_State* L, GCObject* block, size_t osize, uint8_t memcat, else { LUAU_ASSERT(page->busyBlocks == 1); + LUAU_ASSERT(size_t(page->blockSize) == osize); + LUAU_ASSERT((void*)block == page->data); freepage(L, &g->allgcopages, page); } @@ -626,8 +683,12 @@ void luaM_getpagewalkinfo(lua_Page* page, char** start, char** end, int* busyBlo int blockCount = (page->pageSize - offsetof(lua_Page, data)) / page->blockSize; - *start = page->data + page->freeNext + page->blockSize; - *end = page->data + blockCount * page->blockSize; + LUAU_ASSERT(page->freeNext >= -page->blockSize && page->freeNext <= (blockCount - 1) * page->blockSize); + + char* data = page->data; // silences ubsan when indexing page->data + + *start = data + page->freeNext + page->blockSize; + *end = data + blockCount * page->blockSize; *busyBlocks = page->busyBlocks; *blockSize = page->blockSize; } @@ -675,7 +736,7 @@ void luaM_visitgco(lua_State* L, void* context, bool (*visitor)(void* context, l for (lua_Page* curr = g->allgcopages; curr;) { - lua_Page* next = curr->gcolistnext; // page blockvisit might destroy the page + lua_Page* next = curr->gcolistnext; // block visit might destroy the page luaM_visitpage(curr, context, visitor); diff --git a/VM/src/lobject.cpp b/VM/src/lobject.cpp index 370c7b28..d5bd76a8 100644 --- a/VM/src/lobject.cpp +++ b/VM/src/lobject.cpp @@ -131,7 +131,7 @@ void luaO_chunkid(char* out, const char* source, size_t bufflen) { size_t l; source++; /* skip the `@' */ - bufflen -= sizeof(" '...' "); + bufflen -= sizeof("..."); l = strlen(source); strcpy(out, ""); if (l > bufflen) @@ -144,7 +144,7 @@ void luaO_chunkid(char* out, const char* source, size_t bufflen) else { /* out = [string "string"] */ size_t len = strcspn(source, "\n\r"); /* stop at first newline */ - bufflen -= sizeof(" [string \"...\"] "); + bufflen -= sizeof("[string \"...\"]"); if (len > bufflen) len = bufflen; strcpy(out, "[string \""); diff --git a/VM/src/lvmexecute.cpp b/VM/src/lvmexecute.cpp index cba3670a..c3b662a2 100644 --- a/VM/src/lvmexecute.cpp +++ b/VM/src/lvmexecute.cpp @@ -609,7 +609,8 @@ static void luau_execute(lua_State* L) if (unsigned(ic) < LUA_VECTOR_SIZE && name[1] == '\0') { - setnvalue(ra, rb->value.v[ic]); + const float* v = rb->value.v; // silences ubsan when indexing v[] + setnvalue(ra, v[ic]); VM_NEXT(); } diff --git a/tests/Compiler.test.cpp b/tests/Compiler.test.cpp index d8af94db..cd7a21d8 100644 --- a/tests/Compiler.test.cpp +++ b/tests/Compiler.test.cpp @@ -605,8 +605,6 @@ RETURN R0 1 TEST_CASE("TableLiteralsIndexConstant") { - ScopedFastFlag sff("LuauCompileTableIndexOpt", true); - // validate that we use SETTTABLEKS for constant variable keys CHECK_EQ("\n" + compileFunction0(R"( local a, b = "key", "value" @@ -2483,8 +2481,6 @@ return TEST_CASE("DebugLineInfoAssignment") { - ScopedFastFlag sff("LuauCompileTableIndexOpt", true); - Luau::BytecodeBuilder bcb; bcb.setDumpFlags(Luau::BytecodeBuilder::Dump_Code | Luau::BytecodeBuilder::Dump_Lines); Luau::compileOrThrow(bcb, R"( diff --git a/tests/Conformance.test.cpp b/tests/Conformance.test.cpp index e580949f..8b58d2ce 100644 --- a/tests/Conformance.test.cpp +++ b/tests/Conformance.test.cpp @@ -492,8 +492,6 @@ TEST_CASE("DateTime") TEST_CASE("Debug") { - ScopedFastFlag sffw("LuauBytecodeV2Write", true); - runConformance("debug.lua"); } diff --git a/tests/Linter.test.cpp b/tests/Linter.test.cpp index d1cc49b2..577415fc 100644 --- a/tests/Linter.test.cpp +++ b/tests/Linter.test.cpp @@ -1392,19 +1392,31 @@ TEST_CASE_FIXTURE(Fixture, "DeprecatedApi") {"DataCost", {typeChecker.numberType, /* deprecated= */ true}}, {"Wait", {typeChecker.anyType, /* deprecated= */ true}}, }; + + TypeId colorType = typeChecker.globalTypes.addType(TableTypeVar{{}, std::nullopt, typeChecker.globalScope->level, Luau::TableState::Sealed}); + + getMutable(colorType)->props = { + {"toHSV", {typeChecker.anyType, /* deprecated= */ true, "Color3:ToHSV"} } + }; + + addGlobalBinding(typeChecker, "Color3", Binding{colorType, {}}); + freeze(typeChecker.globalTypes); LintResult result = lintTyped(R"( return function (i: Instance) i:Wait(1.0) print(i.Name) + print(Color3.toHSV()) + print(Color3.doesntexist, i.doesntexist) -- type error, but this verifies we correctly handle non-existent members return i.DataCost end )"); - REQUIRE_EQ(result.warnings.size(), 2); + REQUIRE_EQ(result.warnings.size(), 3); CHECK_EQ(result.warnings[0].text, "Member 'Instance.Wait' is deprecated"); - CHECK_EQ(result.warnings[1].text, "Member 'Instance.DataCost' is deprecated"); + CHECK_EQ(result.warnings[1].text, "Member 'toHSV' is deprecated, use 'Color3:ToHSV' instead"); + CHECK_EQ(result.warnings[2].text, "Member 'Instance.DataCost' is deprecated"); } TEST_CASE_FIXTURE(Fixture, "TableOperations") @@ -1475,9 +1487,11 @@ _ = (true and true) or true _ = (true and false) and (42 and false) _ = true and true or false -- no warning since this is is a common pattern used as a ternary replacement + +_ = if true then 1 elseif true then 2 else 3 )"); - REQUIRE_EQ(result.warnings.size(), 7); + REQUIRE_EQ(result.warnings.size(), 8); CHECK_EQ(result.warnings[0].text, "Condition has already been checked on line 2"); CHECK_EQ(result.warnings[0].location.begin.line + 1, 4); CHECK_EQ(result.warnings[1].text, "Condition has already been checked on column 5"); @@ -1487,6 +1501,7 @@ _ = true and true or false -- no warning since this is is a common pattern used CHECK_EQ(result.warnings[5].text, "Condition has already been checked on column 6"); CHECK_EQ(result.warnings[6].text, "Condition has already been checked on column 15"); CHECK_EQ(result.warnings[6].location.begin.line + 1, 19); + CHECK_EQ(result.warnings[7].text, "Condition has already been checked on column 8"); } TEST_CASE_FIXTURE(Fixture, "DuplicateConditionsExpr") @@ -1528,4 +1543,19 @@ return foo, moo, a1, a2 CHECK_EQ(result.warnings[3].text, "Function parameter 'self' already defined implicitly"); } +TEST_CASE_FIXTURE(Fixture, "MisleadingAndOr") +{ + LintResult result = lint(R"( +_ = math.random() < 0.5 and true or 42 +_ = math.random() < 0.5 and false or 42 -- misleading +_ = math.random() < 0.5 and nil or 42 -- misleading +_ = math.random() < 0.5 and 0 or 42 +_ = (math.random() < 0.5 and false) or 42 -- currently ignored +)"); + + REQUIRE_EQ(result.warnings.size(), 2); + CHECK_EQ(result.warnings[0].text, "The and-or expression always evaluates to the second alternative because the first alternative is false; consider using if-then-else expression instead"); + CHECK_EQ(result.warnings[1].text, "The and-or expression always evaluates to the second alternative because the first alternative is nil; consider using if-then-else expression instead"); +} + TEST_SUITE_END(); diff --git a/tests/Repl.test.cpp b/tests/Repl.test.cpp index f660bcd3..1f9c9739 100644 --- a/tests/Repl.test.cpp +++ b/tests/Repl.test.cpp @@ -8,9 +8,22 @@ #include #include +#include #include #include +struct Completion +{ + std::string completion; + std::string display; + + bool operator<(Completion const& other) const + { + return std::tie(completion, display) < std::tie(other.completion, other.display); + } +}; + +using CompletionSet = std::set; class ReplFixture { @@ -34,6 +47,27 @@ public: lua_pop(L, 1); return result; } + + CompletionSet getCompletionSet(const char* inputPrefix) + { + CompletionSet result; + int top = lua_gettop(L); + getCompletions(L, inputPrefix, [&result](const std::string& completion, const std::string& display) { + result.insert(Completion{completion, display}); + }); + // Ensure that generating completions doesn't change the position of luau's stack top. + CHECK(top == lua_gettop(L)); + + return result; + } + + bool checkCompletion(const CompletionSet& completions, const std::string& prefix, const std::string& expected) + { + std::string expectedDisplay(expected.substr(0, expected.find_first_of('('))); + Completion expectedCompletion{prefix + expected, expectedDisplay}; + return completions.count(expectedCompletion) == 1; + } + lua_State* L; private: @@ -115,3 +149,61 @@ TEST_CASE_FIXTURE(ReplFixture, "MultipleArguments") } TEST_SUITE_END(); + +TEST_SUITE_BEGIN("ReplCodeCompletion"); + +TEST_CASE_FIXTURE(ReplFixture, "CompleteGlobalVariables") +{ + runCode(L, R"( + myvariable1 = 5 + myvariable2 = 5 +)"); + CompletionSet completions = getCompletionSet("myvar"); + + std::string prefix = ""; + CHECK(completions.size() == 2); + CHECK(checkCompletion(completions, prefix, "myvariable1")); + CHECK(checkCompletion(completions, prefix, "myvariable2")); +} + +TEST_CASE_FIXTURE(ReplFixture, "CompleteTableKeys") +{ + runCode(L, R"( + t = { color = "red", size = 1, shape = "circle" } +)"); + { + CompletionSet completions = getCompletionSet("t."); + + std::string prefix = "t."; + CHECK(completions.size() == 3); + CHECK(checkCompletion(completions, prefix, "color")); + CHECK(checkCompletion(completions, prefix, "size")); + CHECK(checkCompletion(completions, prefix, "shape")); + } + + { + CompletionSet completions = getCompletionSet("t.s"); + + std::string prefix = "t."; + CHECK(completions.size() == 2); + CHECK(checkCompletion(completions, prefix, "size")); + CHECK(checkCompletion(completions, prefix, "shape")); + } +} + +TEST_CASE_FIXTURE(ReplFixture, "StringMethods") +{ + runCode(L, R"( + s = "" +)"); + { + CompletionSet completions = getCompletionSet("s:l"); + + std::string prefix = "s:"; + CHECK(completions.size() == 2); + CHECK(checkCompletion(completions, prefix, "len(")); + CHECK(checkCompletion(completions, prefix, "lower(")); + } +} + +TEST_SUITE_END(); diff --git a/tests/TypeInfer.aliases.test.cpp b/tests/TypeInfer.aliases.test.cpp index 76ab23b3..a8729268 100644 --- a/tests/TypeInfer.aliases.test.cpp +++ b/tests/TypeInfer.aliases.test.cpp @@ -595,4 +595,65 @@ TEST_CASE_FIXTURE(Fixture, "generic_typevars_are_not_considered_to_escape_their_ LUAU_REQUIRE_NO_ERRORS(result); } +/* + * The two-pass alias definition system starts by ascribing a free TypeVar to each alias. It then + * circles back to fill in the actual type later on. + * + * If this free type is unified with something degenerate like `any`, we need to take extra care + * to ensure that the alias actually binds to the type that the user expected. + */ +TEST_CASE_FIXTURE(Fixture, "forward_declared_alias_is_not_clobbered_by_prior_unification_with_any") +{ + ScopedFastFlag sff[] = { + {"LuauTwoPassAliasDefinitionFix", true} + }; + + CheckResult result = check(R"( + local function x() + local y: FutureType = {}::any + return 1 + end + type FutureType = { foo: typeof(x()) } + local d: FutureType = { smth = true } -- missing error, 'd' is resolved to 'any' + )"); + + CHECK_EQ("{| foo: number |}", toString(requireType("d"), {true})); + + LUAU_REQUIRE_ERROR_COUNT(1, result); +} + +TEST_CASE_FIXTURE(Fixture, "forward_declared_alias_is_not_clobbered_by_prior_unification_with_any_2") +{ + ScopedFastFlag sff[] = { + {"LuauTwoPassAliasDefinitionFix", true}, + + // We also force these two flags because this surfaced an unfortunate interaction. + {"LuauErrorRecoveryType", true}, + {"LuauQuantifyInPlace2", true}, + }; + + CheckResult result = check(R"( + local B = {} + B.bar = 4 + + function B:smth1() + local self: FutureIntersection = self + self.foo = 4 + return 4 + end + + function B:smth2() + local self: FutureIntersection = self + self.bar = 5 -- error, even though we should have B part with bar + end + + type A = { foo: typeof(B.smth1({foo=3})) } -- trick toposort into sorting functions before types + type B = typeof(B) + + type FutureIntersection = A & B + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.builtins.test.cpp b/tests/TypeInfer.builtins.test.cpp index 6730bedb..df06884d 100644 --- a/tests/TypeInfer.builtins.test.cpp +++ b/tests/TypeInfer.builtins.test.cpp @@ -7,8 +7,6 @@ #include "doctest.h" -LUAU_FASTFLAG(LuauFixTonumberReturnType) - using namespace Luau; LUAU_FASTFLAG(LuauUseCommittingTxnLog) @@ -850,11 +848,8 @@ TEST_CASE_FIXTURE(Fixture, "tonumber_returns_optional_number_type") local b: number = tonumber('asdf') )"); - if (FFlag::LuauFixTonumberReturnType) - { - LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK_EQ("Type 'number?' could not be converted into 'number'", toString(result.errors[0])); - } + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ("Type 'number?' could not be converted into 'number'", toString(result.errors[0])); } TEST_CASE_FIXTURE(Fixture, "tonumber_returns_optional_number_type2") @@ -893,7 +888,7 @@ TEST_CASE_FIXTURE(Fixture, "assert_removes_falsy_types") { ScopedFastFlag sff[]{ {"LuauAssertStripsFalsyTypes", true}, - {"LuauDiscriminableUnions", true}, + {"LuauDiscriminableUnions2", true}, }; CheckResult result = check(R"( @@ -910,7 +905,7 @@ TEST_CASE_FIXTURE(Fixture, "assert_removes_falsy_types_even_from_type_pack_tail_ { ScopedFastFlag sff[]{ {"LuauAssertStripsFalsyTypes", true}, - {"LuauDiscriminableUnions", true}, + {"LuauDiscriminableUnions2", true}, }; CheckResult result = check(R"( @@ -927,7 +922,7 @@ TEST_CASE_FIXTURE(Fixture, "assert_returns_false_and_string_iff_it_knows_the_fir { ScopedFastFlag sff[]{ {"LuauAssertStripsFalsyTypes", true}, - {"LuauDiscriminableUnions", true}, + {"LuauDiscriminableUnions2", true}, }; CheckResult result = check(R"( diff --git a/tests/TypeInfer.provisional.test.cpp b/tests/TypeInfer.provisional.test.cpp index e5eb0dca..2bcd840c 100644 --- a/tests/TypeInfer.provisional.test.cpp +++ b/tests/TypeInfer.provisional.test.cpp @@ -262,7 +262,7 @@ TEST_CASE_FIXTURE(Fixture, "lvalue_equals_another_lvalue_with_no_overlap") // Just needs to fully support equality refinement. Which is annoying without type states. TEST_CASE_FIXTURE(Fixture, "discriminate_from_x_not_equal_to_nil") { - ScopedFastFlag sff{"LuauDiscriminableUnions", true}; + ScopedFastFlag sff{"LuauDiscriminableUnions2", true}; CheckResult result = check(R"( type T = {x: string, y: number} | {x: nil, y: nil} @@ -616,4 +616,76 @@ local a: Self
CHECK_EQ(toString(requireType("a")), "Table
"); } +TEST_CASE_FIXTURE(Fixture, "do_not_ice_when_trying_to_pick_first_of_generic_type_pack") +{ + ScopedFastFlag sff[]{ + {"LuauQuantifyInPlace2", true}, + {"LuauReturnAnyInsteadOfICE", true}, + }; + + // In-place quantification causes these types to have the wrong types but only because of nasty interaction with prototyping. + // The type of f is initially () -> free1... + // Then the prototype iterator advances, and checks the function expression assigned to g, which has the type () -> free2... + // In the body it calls f and returns what f() returns. This binds free2... with free1..., causing f and g to have same types. + // We then quantify g, leaving it with the final type () -> a... + // Because free1... and free2... were bound, in combination with in-place quantification, f's return type was also turned into a... + // Then the check iterator catches up, and checks the body of f, and attempts to quantify it too. + // Alas, one of the requirements for quantification is that a type must contain free types. () -> a... has no free types. + // Thus the quantification for f was no-op, which explains why f does not have any type parameters. + // Calling f() will attempt to instantiate the function type, which turns generics in type binders into to free types. + // However, instantiations only converts generics contained within the type binders of a function, so instantiation was also no-op. + // Which means that calling f() simply returned a... rather than an instantiation of it. And since the call site was not in tail position, + // picking first element in a... triggers an ICE because calls returning generic packs are unexpected. + CheckResult result = check(R"( + local function f() end + + local g = function() return f() end + + local x = (f()) -- should error: no return values to assign from the call to f + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + // f and g should have the type () -> () + CHECK_EQ("() -> (a...)", toString(requireType("f"))); + CHECK_EQ("() -> (a...)", toString(requireType("g"))); + CHECK_EQ("any", toString(requireType("x"))); // any is returned instead of ICE for now +} + +TEST_CASE_FIXTURE(Fixture, "specialization_binds_with_prototypes_too_early") +{ + CheckResult result = check(R"( + local function id(x) return x end + local n2n: (number) -> number = id + local s2s: (string) -> string = id + )"); + + LUAU_REQUIRE_ERRORS(result); // Should not have any errors. +} + +TEST_CASE_FIXTURE(Fixture, "weird_fail_to_unify_type_pack") +{ + ScopedFastFlag sff{"LuauQuantifyInPlace2", true}; + + CheckResult result = check(R"( + local function f() return end + local g = function() return f() end + )"); + + LUAU_REQUIRE_ERRORS(result); // Should not have any errors. +} + +TEST_CASE_FIXTURE(Fixture, "weird_fail_to_unify_variadic_pack") +{ + ScopedFastFlag sff{"LuauQuantifyInPlace2", true}; + + CheckResult result = check(R"( + --!strict + local function f(...) return ... end + local g = function(...) return f(...) end + )"); + + LUAU_REQUIRE_ERRORS(result); // Should not have any errors. +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.refinements.test.cpp b/tests/TypeInfer.refinements.test.cpp index 3a610c3a..48e6be6a 100644 --- a/tests/TypeInfer.refinements.test.cpp +++ b/tests/TypeInfer.refinements.test.cpp @@ -6,7 +6,7 @@ #include "doctest.h" -LUAU_FASTFLAG(LuauDiscriminableUnions) +LUAU_FASTFLAG(LuauDiscriminableUnions2) LUAU_FASTFLAG(LuauWeakEqConstraint) LUAU_FASTFLAG(LuauQuantifyInPlace2) @@ -262,7 +262,7 @@ TEST_CASE_FIXTURE(Fixture, "typeguard_only_look_up_types_from_global_scope") end )"); - if (FFlag::LuauDiscriminableUnions) + if (FFlag::LuauDiscriminableUnions2) { LUAU_REQUIRE_NO_ERRORS(result); @@ -435,7 +435,7 @@ TEST_CASE_FIXTURE(Fixture, "lvalue_is_equal_to_a_term") TEST_CASE_FIXTURE(Fixture, "term_is_equal_to_an_lvalue") { ScopedFastFlag sff[] = { - {"LuauDiscriminableUnions", true}, + {"LuauDiscriminableUnions2", true}, {"LuauSingletonTypes", true}, }; @@ -485,7 +485,7 @@ TEST_CASE_FIXTURE(Fixture, "lvalue_is_not_nil") TEST_CASE_FIXTURE(Fixture, "free_type_is_equal_to_an_lvalue") { - ScopedFastFlag sff{"LuauDiscriminableUnions", true}; + ScopedFastFlag sff{"LuauDiscriminableUnions2", true}; ScopedFastFlag sff2{"LuauWeakEqConstraint", true}; CheckResult result = check(R"( @@ -589,7 +589,7 @@ TEST_CASE_FIXTURE(Fixture, "type_narrow_to_vector") end )"); - if (FFlag::LuauDiscriminableUnions) + if (FFlag::LuauDiscriminableUnions2) { LUAU_REQUIRE_NO_ERRORS(result); } @@ -1002,7 +1002,7 @@ TEST_CASE_FIXTURE(Fixture, "apply_refinements_on_astexprindexexpr_whose_subscrip TEST_CASE_FIXTURE(Fixture, "discriminate_from_truthiness_of_x") { ScopedFastFlag sff[] = { - {"LuauDiscriminableUnions", true}, + {"LuauDiscriminableUnions2", true}, {"LuauParseSingletonTypes", true}, {"LuauSingletonTypes", true}, }; @@ -1028,7 +1028,7 @@ TEST_CASE_FIXTURE(Fixture, "discriminate_from_truthiness_of_x") TEST_CASE_FIXTURE(Fixture, "discriminate_tag") { ScopedFastFlag sff[] = { - {"LuauDiscriminableUnions", true}, + {"LuauDiscriminableUnions2", true}, {"LuauParseSingletonTypes", true}, {"LuauSingletonTypes", true}, }; @@ -1069,7 +1069,7 @@ TEST_CASE_FIXTURE(Fixture, "narrow_boolean_to_true_or_false") ScopedFastFlag sff[]{ {"LuauParseSingletonTypes", true}, {"LuauSingletonTypes", true}, - {"LuauDiscriminableUnions", true}, + {"LuauDiscriminableUnions2", true}, {"LuauAssertStripsFalsyTypes", true}, }; @@ -1094,7 +1094,7 @@ TEST_CASE_FIXTURE(Fixture, "discriminate_on_properties_of_disjoint_tables_where_ ScopedFastFlag sff[]{ {"LuauParseSingletonTypes", true}, {"LuauSingletonTypes", true}, - {"LuauDiscriminableUnions", true}, + {"LuauDiscriminableUnions2", true}, {"LuauAssertStripsFalsyTypes", true}, }; @@ -1118,7 +1118,7 @@ TEST_CASE_FIXTURE(Fixture, "discriminate_on_properties_of_disjoint_tables_where_ TEST_CASE_FIXTURE(RefinementClassFixture, "discriminate_from_isa_of_x") { ScopedFastFlag sff[] = { - {"LuauDiscriminableUnions", true}, + {"LuauDiscriminableUnions2", true}, {"LuauParseSingletonTypes", true}, {"LuauSingletonTypes", true}, }; @@ -1157,7 +1157,7 @@ TEST_CASE_FIXTURE(RefinementClassFixture, "typeguard_cast_free_table_to_vector") end )"); - if (FFlag::LuauDiscriminableUnions) + if (FFlag::LuauDiscriminableUnions2) LUAU_REQUIRE_NO_ERRORS(result); else { diff --git a/tests/TypeInfer.test.cpp b/tests/TypeInfer.test.cpp index ead3d762..531a382f 100644 --- a/tests/TypeInfer.test.cpp +++ b/tests/TypeInfer.test.cpp @@ -5164,4 +5164,151 @@ function x:Destroy(): () end LUAU_REQUIRE_ERRORS(result); } +TEST_CASE_FIXTURE(Fixture, "do_not_modify_imported_types_2") +{ + ScopedFastFlag immutableTypes{"LuauImmutableTypes", true}; + + fileResolver.source["game/A"] = R"( +export type Type = { x: { a: number } } +return {} + )"; + + fileResolver.source["game/B"] = R"( +local types = require(game.A) +type Type = types.Type +local x: Type = { x = { a = 2 } } +type Rename = typeof(x.x) + )"; + + CheckResult result = frontend.check("game/B"); + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "do_not_modify_imported_types_3") +{ + ScopedFastFlag immutableTypes{"LuauImmutableTypes", true}; + + fileResolver.source["game/A"] = R"( +local y = setmetatable({}, {}) +export type Type = { x: typeof(y) } +return { x = y } + )"; + + fileResolver.source["game/B"] = R"( +local types = require(game.A) +type Type = types.Type +local x: Type = types +type Rename = typeof(x.x) + )"; + + CheckResult result = frontend.check("game/B"); + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "indexing_on_string_singletons") +{ + ScopedFastFlag sff[]{ + {"LuauDiscriminableUnions2", true}, + {"LuauRefactorTypeVarQuestions", true}, + {"LuauSingletonTypes", true}, + }; + + CheckResult result = check(R"( + local a: string = "hi" + if a == "hi" then + local x = a:byte() + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ(R"("hi")", toString(requireTypeAtPosition({3, 22}))); +} + +TEST_CASE_FIXTURE(Fixture, "indexing_on_union_of_string_singletons") +{ + ScopedFastFlag sff[]{ + {"LuauDiscriminableUnions2", true}, + {"LuauRefactorTypeVarQuestions", true}, + {"LuauSingletonTypes", true}, + }; + + CheckResult result = check(R"( + local a: string = "hi" + if a == "hi" or a == "bye" then + local x = a:byte() + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ(R"("bye" | "hi")", toString(requireTypeAtPosition({3, 22}))); +} + +TEST_CASE_FIXTURE(Fixture, "taking_the_length_of_string_singleton") +{ + ScopedFastFlag sff[]{ + {"LuauDiscriminableUnions2", true}, + {"LuauRefactorTypeVarQuestions", true}, + {"LuauSingletonTypes", true}, + {"LuauLengthOnCompositeType", true}, + }; + + CheckResult result = check(R"( + local a: string = "hi" + if a == "hi" then + local x = #a + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ(R"("hi")", toString(requireTypeAtPosition({3, 23}))); +} + +TEST_CASE_FIXTURE(Fixture, "taking_the_length_of_union_of_string_singleton") +{ + ScopedFastFlag sff[]{ + {"LuauDiscriminableUnions2", true}, + {"LuauRefactorTypeVarQuestions", true}, + {"LuauSingletonTypes", true}, + {"LuauLengthOnCompositeType", true}, + }; + + CheckResult result = check(R"( + local a: string = "hi" + if a == "hi" or a == "bye" then + local x = #a + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ(R"("bye" | "hi")", toString(requireTypeAtPosition({3, 23}))); +} + +/* + * When we add new properties to an unsealed table, we should do a level check and promote the property type to be at + * the level of the table. + */ +TEST_CASE_FIXTURE(Fixture, "inferred_properties_of_a_table_should_start_with_the_same_TypeLevel_of_that_table") +{ + CheckResult result = check(R"( + --!strict + local T = {} + + local function f(prop) + T[1] = { + prop = prop, + } + end + + local function g() + local l = T[1].prop + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.tryUnify.test.cpp b/tests/TypeInfer.tryUnify.test.cpp index 0aeca096..8c7fb79a 100644 --- a/tests/TypeInfer.tryUnify.test.cpp +++ b/tests/TypeInfer.tryUnify.test.cpp @@ -273,4 +273,21 @@ TEST_CASE_FIXTURE(TryUnifyFixture, "recursive_metatable_getmatchtag") state.tryUnify(&metatable, &variant); } +TEST_CASE_FIXTURE(TryUnifyFixture, "cli_50320_follow_in_any_unification") +{ + ScopedFastFlag sffs[] = { + {"LuauUseCommittingTxnLog", true}, + {"LuauFollowWithCommittingTxnLogInAnyUnification", true}, + }; + + TypePackVar free{FreeTypePack{TypeLevel{}}}; + TypePackVar target{TypePack{}}; + + TypeVar func{FunctionTypeVar{&free, &free}}; + + state.tryUnify(&free, &target); + // Shouldn't assert or error. + state.tryUnify(&func, typeChecker.anyType); +} + TEST_SUITE_END(); diff --git a/tests/conformance/basic.lua b/tests/conformance/basic.lua index de091632..78d90077 100644 --- a/tests/conformance/basic.lua +++ b/tests/conformance/basic.lua @@ -118,9 +118,7 @@ assert((function() return #_G end)() == 0) assert((function() return #{1,2} end)() == 2) assert((function() return #'g' end)() == 1) -local ud = newproxy(true) -getmetatable(ud).__len = function() return 42 end -assert((function() return #ud end)() == 42) +assert((function() local ud = newproxy(true) getmetatable(ud).__len = function() return 42 end return #ud end)() == 42) assert((function() local a = 1 a = -a return a end)() == -1) @@ -325,6 +323,10 @@ assert((function() local t = {6, 9, 7} t[4.5] = 10 return t[4.5] end)() == 10) assert((function() local t = {6, 9, 7} t['a'] = 11 return t['a'] end)() == 11) assert((function() local t = {6, 9, 7} setmetatable(t, { __newindex = function(t,i,v) rawset(t, i * 10, v) end }) t[1] = 17 t[5] = 1 return concat(t[1],t[5],t[50]) end)() == "17,nil,1") +-- userdata access +assert((function() local ud = newproxy(true) getmetatable(ud).__index = function(ud,i) return i * 10 end return ud[2] end)() == 20) +assert((function() local ud = newproxy(true) getmetatable(ud).__index = function() return function(self, i) return i * 10 end end return ud:meow(2) end)() == 20) + -- and/or -- rhs is a constant assert((function() local a = 1 a = a and 2 return a end)() == 2) @@ -462,7 +464,7 @@ assert((function() a = {} b = {} mt = { __eq = function(l, r) return #l == #r en -- metatable ops local function vec3t(x, y, z) - return setmetatable({ x=x, y=y, z=z}, { + return setmetatable({x=x, y=y, z=z}, { __add = function(l, r) return vec3t(l.x + r.x, l.y + r.y, l.z + r.z) end, __sub = function(l, r) return vec3t(l.x - r.x, l.y - r.y, l.z - r.z) end, __mul = function(l, r) return type(r) == "number" and vec3t(l.x * r, l.y * r, l.z * r) or vec3t(l.x * r.x, l.y * r.y, l.z * r.z) end, diff --git a/tests/conformance/debug.lua b/tests/conformance/debug.lua index 8c96ab33..0e410000 100644 --- a/tests/conformance/debug.lua +++ b/tests/conformance/debug.lua @@ -37,6 +37,7 @@ coroutine.resume(co2, 0 / 0, 42) assert(debug.traceback(co2) == "debug.lua:31 function halp\n") assert(debug.info(co2, 0, "l") == 31) +assert(debug.info(co2, 0, "f") == halp) -- info errors function qux(...) diff --git a/tests/conformance/errors.lua b/tests/conformance/errors.lua index d5ff215b..751188be 100644 --- a/tests/conformance/errors.lua +++ b/tests/conformance/errors.lua @@ -260,8 +260,7 @@ local a,b = loadstring(s) assert(not a) --assert(string.find(b, "line 2")) --- Test for CLI-28786 --- The xpcall is intentially going to cause an exception +-- The xpcall is intentionally going to cause an exception -- followed by a forced exception in the error handler. -- If the secondary handler isn't trapped, it will cause -- the unit test to fail. If the xpcall captures the @@ -281,6 +280,19 @@ coroutine.wrap(function() assert(not pcall(debug.getinfo, coroutine.running(), 0, ">")) end)() +-- loadstring chunk truncation +local a,b = loadstring("nope", "@short") +assert(not a and b:match('[^ ]+') == "short:1:") + +local a,b = loadstring("nope", "@" .. string.rep("thisisaverylongstringitssolongthatitwontfitintotheinternalbufferprovidedtovariousdebugfacilities", 10)) +assert(not a and b:match('[^ ]+') == "...wontfitintotheinternalbufferprovidedtovariousdebugfacilitiesthisisaverylongstringitssolongthatitwontfitintotheinternalbufferprovidedtovariousdebugfacilitiesthisisaverylongstringitssolongthatitwontfitintotheinternalbufferprovidedtovariousdebugfacilities:1:") + +local a,b = loadstring("nope", "=short") +assert(not a and b:match('[^ ]+') == "short:1:") + +local a,b = loadstring("nope", "=" .. string.rep("thisisaverylongstringitssolongthatitwontfitintotheinternalbufferprovidedtovariousdebugfacilities", 10)) +assert(not a and b:match('[^ ]+') == "thisisaverylongstringitssolongthatitwontfitintotheinternalbufferprovidedtovariousdebugfacilitiesthisisaverylongstringitssolongthatitwontfitintotheinternalbufferprovidedtovariousdebugfacilitiesthisisaverylongstringitssolongthatitwontfitintotheinternalbuffe:1:") + -- arith errors function ecall(fn, ...) local ok, err = pcall(fn, ...) diff --git a/tests/conformance/gc.lua b/tests/conformance/gc.lua index 409cd224..5804ea7f 100644 --- a/tests/conformance/gc.lua +++ b/tests/conformance/gc.lua @@ -180,6 +180,11 @@ x,y,z=nil collectgarbage() assert(next(a) == string.rep('$', 11)) +-- shrinking tables reduce their capacity; confirming the shrinking is difficult but we can at least test the surface level behavior +a = {}; setmetatable(a, {__mode = 'ks'}) +for i=1,lim do a[{}] = i end +collectgarbage() +assert(next(a) == nil) -- testing userdata collectgarbage("stop") -- stop collection @@ -315,8 +320,6 @@ do end collectgarbage() - end - return('OK') diff --git a/tests/conformance/math.lua b/tests/conformance/math.lua index bfea0e1f..79ea0fb6 100644 --- a/tests/conformance/math.lua +++ b/tests/conformance/math.lua @@ -289,6 +289,7 @@ assert(math.sqrt("4") == 2) assert(math.tanh("0") == 0) assert(math.tan("0") == 0) assert(math.clamp("0", 2, 3) == 2) +assert(math.clamp("4", 2, 3) == 3) assert(math.sign("2") == 1) assert(math.sign("-2") == -1) assert(math.sign("0") == 0) diff --git a/tests/conformance/vararg.lua b/tests/conformance/vararg.lua index d05f9577..178c56b8 100644 --- a/tests/conformance/vararg.lua +++ b/tests/conformance/vararg.lua @@ -139,6 +139,12 @@ assert(selectmany(1, 10, 20, 30) == "10,20,30") assert(selectone(2, 10, 20, 30) == 20) assert(selectmany(2, 10, 20, 30) == "20,30") +assert(selectone(3, 10, 20, 30) == 30) +assert(selectmany(3, 10, 20, 30) == "30") + +assert(selectone(4, 10, 20, 30) == nil) +assert(selectmany(4, 10, 20, 30) == "") + assert(selectone(-2, 10, 20, 30) == 20) assert(selectmany(-2, 10, 20, 30) == "20,30") diff --git a/tests/conformance/vector.lua b/tests/conformance/vector.lua index 7d18bda3..22d6adfc 100644 --- a/tests/conformance/vector.lua +++ b/tests/conformance/vector.lua @@ -87,9 +87,18 @@ assert(pcall(function() local t = {} rawset(t, vector(0/0, 2, 3), 1) end) == fal -- make sure we cover both builtin and C impl assert(vector(1, 2, 4) == vector("1", "2", "4")) +-- validate component access (both cases) +assert(vector(1, 2, 3).x == 1) +assert(vector(1, 2, 3).X == 1) +assert(vector(1, 2, 3).y == 2) +assert(vector(1, 2, 3).Y == 2) +assert(vector(1, 2, 3).z == 3) +assert(vector(1, 2, 3).Z == 3) + -- additional checks for 4-component vectors if vector_size == 4 then assert(vector(1, 2, 3, 4).w == 4) + assert(vector(1, 2, 3, 4).W == 4) end return 'OK' diff --git a/tools/heapgraph.py b/tools/heapgraph.py index 106db549..d4d29af1 100644 --- a/tools/heapgraph.py +++ b/tools/heapgraph.py @@ -7,10 +7,20 @@ # The result of analysis is a .svg file which can be viewed in a browser # To generate these dumps, use luaC_dump, ideally preceded by luaC_fullgc +import argparse import json import sys import svg +argumentParser = argparse.ArgumentParser(description='Luau heap snapshot analyzer') + +argumentParser.add_argument('--split', dest = 'split', type = str, default = 'none', help = 'Perform additional root split using memory categories', choices = ['none', 'custom', 'all']) + +argumentParser.add_argument('snapshot') +argumentParser.add_argument('snapshotnew', nargs='?') + +arguments = argumentParser.parse_args() + class Node(svg.Node): def __init__(self): svg.Node.__init__(self) @@ -30,14 +40,14 @@ class Node(svg.Node): return "{} ({:,} bytes, {:.1%}); self: {:,} bytes in {:,} objects".format(self.name, self.width, self.width / root.width, self.size, self.count) # load files -if len(sys.argv) == 2: +if arguments.snapshotnew == None: dumpold = None - with open(sys.argv[1]) as f: + with open(arguments.snapshot) as f: dump = json.load(f) else: - with open(sys.argv[1]) as f: + with open(arguments.snapshot) as f: dumpold = json.load(f) - with open(sys.argv[2]) as f: + with open(arguments.snapshotnew) as f: dump = json.load(f) # reachability analysis: how much of the heap is reachable from roots? @@ -111,12 +121,15 @@ while offset < len(queue): if "object" in obj: queue.append((obj["object"], node)) -def annotateContainedCategories(node): +def annotateContainedCategories(node, start): for obj in node.objects: + if obj["cat"] < start: + obj["cat"] = 0 + node.categories.add(obj["cat"]) for child in node.children.values(): - annotateContainedCategories(child) + annotateContainedCategories(child, start) for cat in child.categories: node.categories.add(cat) @@ -172,9 +185,11 @@ def splitIntoCategories(root): return result -# temporarily disabled because it makes FG harder to read, maybe this should be a separate command line option? -if dump["stats"].get("categories") and False: - annotateContainedCategories(root) +if dump["stats"].get("categories") and arguments.split != 'none': + if arguments.split == 'custom': + annotateContainedCategories(root, 128) + else: + annotateContainedCategories(root, 0) root = splitIntoCategories(root) diff --git a/tools/svg.py b/tools/svg.py index 99853fb6..21200eeb 100644 --- a/tools/svg.py +++ b/tools/svg.py @@ -452,7 +452,7 @@ def display(root, title, colors, flip = False): .replace("$gradient-start", gradient_start) .replace("$gradient-end", gradient_end) .replace("$height", str(svgheight)) - .replace("$status", str(svgheight - 16 + 3)) + .replace("$status", str((svgheight - 16 + 3 if flip else 3 * 16 - 3))) .replace("$flip", str(int(flip))) ) From db90c7da48f30a833802f23e0f41daab1ab6c316 Mon Sep 17 00:00:00 2001 From: Alan Jeffrey <403333+asajeffrey@users.noreply.github.com> Date: Fri, 11 Feb 2022 14:38:35 -0600 Subject: [PATCH 158/399] Add a typeToString function to the prototype (#354) * Added Luau.Type.ToString --- .github/workflows/prototyping.yml | 1 + prototyping/Examples.agda | 2 +- prototyping/Examples/Type.agda | 27 +++++++++++++++++++++++ prototyping/Luau/Type.agda | 33 +++++++++++++++++++++++++++++ prototyping/Luau/Type/ToString.agda | 26 +++++++++++++++++++++++ prototyping/Properties.agda | 2 ++ 6 files changed, 90 insertions(+), 1 deletion(-) create mode 100644 prototyping/Examples/Type.agda create mode 100644 prototyping/Luau/Type/ToString.agda diff --git a/.github/workflows/prototyping.yml b/.github/workflows/prototyping.yml index 02e021ee..76c1e3c1 100644 --- a/.github/workflows/prototyping.yml +++ b/.github/workflows/prototyping.yml @@ -4,6 +4,7 @@ on: push: branches: - 'master' + - 'prototyping-*' paths: - '.github/workflows/**' - 'prototyping/**' diff --git a/prototyping/Examples.agda b/prototyping/Examples.agda index fe396eff..212067b7 100644 --- a/prototyping/Examples.agda +++ b/prototyping/Examples.agda @@ -4,4 +4,4 @@ module Examples where import Examples.Syntax import Examples.OpSem import Examples.Run - +import Examples.Type diff --git a/prototyping/Examples/Type.agda b/prototyping/Examples/Type.agda new file mode 100644 index 00000000..859227ee --- /dev/null +++ b/prototyping/Examples/Type.agda @@ -0,0 +1,27 @@ +module Examples.Type where + +open import Agda.Builtin.Equality using (_≡_; refl) +open import FFI.Data.String using (_++_) +open import Luau.Type using (nil; _∪_; _∩_; _⇒_) +open import Luau.Type.ToString using (typeToString) + +ex1 : typeToString(nil) ≡ "nil" +ex1 = refl + +ex2 : typeToString(nil ⇒ nil) ≡ "(nil) -> nil" +ex2 = refl + +ex3 : typeToString(nil ⇒ (nil ⇒ nil)) ≡ "(nil) -> (nil) -> nil" +ex3 = refl + +ex4 : typeToString(nil ∪ (nil ⇒ (nil ⇒ nil))) ≡ "((nil) -> (nil) -> nil)?" +ex4 = refl + +ex5 : typeToString(nil ⇒ ((nil ⇒ nil) ∪ nil)) ≡ "(nil) -> ((nil) -> nil)?" +ex5 = refl + +ex6 : typeToString((nil ⇒ nil) ∪ (nil ⇒ (nil ⇒ nil))) ≡ "((nil) -> nil | (nil) -> (nil) -> nil)" +ex6 = refl + +ex7 : typeToString((nil ⇒ nil) ∪ ((nil ⇒ (nil ⇒ nil)) ∪ nil)) ≡ "((nil) -> nil | (nil) -> (nil) -> nil)?" +ex7 = refl diff --git a/prototyping/Luau/Type.agda b/prototyping/Luau/Type.agda index 8da3a985..6e384c3b 100644 --- a/prototyping/Luau/Type.agda +++ b/prototyping/Luau/Type.agda @@ -1,5 +1,7 @@ module Luau.Type where +open import FFI.Data.Maybe using (Maybe; just; nothing) + data Type : Set where nil : Type _⇒_ : Type → Type → Type @@ -8,3 +10,34 @@ data Type : Set where _∪_ : Type → Type → Type _∩_ : Type → Type → Type +src : Type → Type +src nil = none +src (S ⇒ T) = S +src none = none +src any = any +src (S ∪ T) = (src S) ∪ (src T) +src (S ∩ T) = (src S) ∩ (src T) + +tgt : Type → Type +tgt nil = none +tgt (S ⇒ T) = T +tgt none = none +tgt any = any +tgt (S ∪ T) = (tgt S) ∪ (tgt T) +tgt (S ∩ T) = (tgt S) ∩ (tgt T) + +optional : Type → Type +optional nil = nil +optional (T ∪ nil) = (T ∪ nil) +optional T = (T ∪ nil) + +normalizeOptional : Type → Type +normalizeOptional (S ∪ T) with normalizeOptional S | normalizeOptional T +normalizeOptional (S ∪ T) | (S′ ∪ nil) | (T′ ∪ nil) = (S′ ∪ T′) ∪ nil +normalizeOptional (S ∪ T) | S′ | (T′ ∪ nil) = (S′ ∪ T′) ∪ nil +normalizeOptional (S ∪ T) | (S′ ∪ nil) | T′ = (S′ ∪ T′) ∪ nil +normalizeOptional (S ∪ T) | S′ | nil = optional S′ +normalizeOptional (S ∪ T) | nil | T′ = optional T′ +normalizeOptional (S ∪ T) | S′ | T′ = S′ ∪ T′ +normalizeOptional T = T + diff --git a/prototyping/Luau/Type/ToString.agda b/prototyping/Luau/Type/ToString.agda new file mode 100644 index 00000000..698d6e8e --- /dev/null +++ b/prototyping/Luau/Type/ToString.agda @@ -0,0 +1,26 @@ +module Luau.Type.ToString where + +open import FFI.Data.String using (String; _++_) +open import Luau.Type using (Type; nil; _⇒_; none; any; _∪_; _∩_; normalizeOptional) + +{-# TERMINATING #-} +typeToString : Type → String +typeToStringᵁ : Type → String +typeToStringᴵ : Type → String + +typeToString nil = "nil" +typeToString (S ⇒ T) = "(" ++ (typeToString S) ++ ") -> " ++ (typeToString T) +typeToString none = "none" +typeToString any = "any" +typeToString (S ∪ T) with normalizeOptional(S ∪ T) +typeToString (S ∪ T) | ((S′ ⇒ T′) ∪ nil) = "(" ++ typeToString (S′ ⇒ T′) ++ ")?" +typeToString (S ∪ T) | (S′ ∪ nil) = typeToString S′ ++ "?" +typeToString (S ∪ T) | (S′ ∪ T′) = "(" ++ typeToStringᵁ (S ∪ T) ++ ")" +typeToString (S ∪ T) | T′ = typeToString T′ +typeToString (S ∩ T) = "(" ++ typeToStringᴵ (S ∩ T) ++ ")" + +typeToStringᵁ (S ∪ T) = (typeToStringᵁ S) ++ " | " ++ (typeToStringᵁ T) +typeToStringᵁ T = typeToString T + +typeToStringᴵ (S ∩ T) = (typeToStringᴵ S) ++ " & " ++ (typeToStringᴵ T) +typeToStringᴵ T = typeToString T diff --git a/prototyping/Properties.agda b/prototyping/Properties.agda index a3c685d1..b08a3a81 100644 --- a/prototyping/Properties.agda +++ b/prototyping/Properties.agda @@ -1,3 +1,5 @@ module Properties where import Properties.Dec +import Properties.Step +import Properties.Remember From e0a9bc191aef922dbc019ff08c4a3125ce3ac749 Mon Sep 17 00:00:00 2001 From: Alan Jeffrey <403333+asajeffrey@users.noreply.github.com> Date: Fri, 11 Feb 2022 19:03:26 -0600 Subject: [PATCH 159/399] Prototype: added syntax for optional type annotations (#358) --- prototyping/Examples/OpSem.agda | 8 +-- prototyping/Examples/Run.agda | 9 +-- prototyping/Examples/Syntax.agda | 18 +++--- prototyping/Interpreter.agda | 2 +- prototyping/Luau/Heap.agda | 31 ++++++----- prototyping/Luau/OpSem.agda | 32 +++++------ prototyping/Luau/Run.agda | 10 ++-- prototyping/Luau/RuntimeError.agda | 10 ++-- prototyping/Luau/RuntimeError/ToString.agda | 7 ++- prototyping/Luau/Substitution.agda | 16 +++--- prototyping/Luau/Syntax.agda | 62 +++++++++++++++------ prototyping/Luau/Syntax/FromJSON.agda | 20 +++---- prototyping/Luau/Syntax/ToString.agda | 37 +++++++----- prototyping/Luau/Value.agda | 4 +- prototyping/PrettyPrinter.agda | 2 +- prototyping/Properties/Step.agda | 26 ++++----- 16 files changed, 165 insertions(+), 129 deletions(-) diff --git a/prototyping/Examples/OpSem.agda b/prototyping/Examples/OpSem.agda index 4043869e..ec8bce7b 100644 --- a/prototyping/Examples/OpSem.agda +++ b/prototyping/Examples/OpSem.agda @@ -1,11 +1,9 @@ module Examples.OpSem where open import Luau.OpSem using (_⊢_⟶ᴱ_⊣_; _⊢_⟶ᴮ_⊣_; subst) -open import Luau.Syntax using (var; nil; local_←_; _∙_; done; return) -open import Luau.Heap using (emp) +open import Luau.Syntax using (Block; var; nil; local_←_; _∙_; done; return; block_is_end) +open import Luau.Heap using (∅) -x = var "x" - -ex1 : emp ⊢ (local "x" ← nil ∙ return x ∙ done) ⟶ᴮ (return nil ∙ done) ⊣ emp +ex1 : ∅ ⊢ (local (var "x") ← nil ∙ return (var "x") ∙ done) ⟶ᴮ (return nil ∙ done) ⊣ ∅ ex1 = subst diff --git a/prototyping/Examples/Run.agda b/prototyping/Examples/Run.agda index bfa79839..88ca39b6 100644 --- a/prototyping/Examples/Run.agda +++ b/prototyping/Examples/Run.agda @@ -3,16 +3,13 @@ module Examples.Run where open import Agda.Builtin.Equality using (_≡_; refl) -open import Luau.Syntax using (nil; var; _$_; function_⟨_⟩_end; return; _∙_; done) +open import Luau.Syntax using (nil; var; _$_; function_is_end; return; _∙_; done; _⟨_⟩) open import Luau.Value using (nil) open import Luau.Run using (run; return) -open import Luau.Heap using (emp; lookup-next; next-emp; lookup-next-emp) +open import Luau.Heap using (lookup-next; next-emp; lookup-next-emp) import Agda.Builtin.Equality.Rewrite {-# REWRITE lookup-next next-emp lookup-next-emp #-} -x = var "x" -id = var "id" - -ex1 : (run (function "id" ⟨ "x" ⟩ return x ∙ done end ∙ return (id $ nil) ∙ done) ≡ return nil _) +ex1 : (run (function "id" ⟨ var "x" ⟩ is return (var "x") ∙ done end ∙ return (var "id" $ nil) ∙ done) ≡ return nil _) ex1 = refl diff --git a/prototyping/Examples/Syntax.agda b/prototyping/Examples/Syntax.agda index 8af9bca8..4ffcf4c5 100644 --- a/prototyping/Examples/Syntax.agda +++ b/prototyping/Examples/Syntax.agda @@ -2,21 +2,21 @@ module Examples.Syntax where open import Agda.Builtin.Equality using (_≡_; refl) open import FFI.Data.String using (_++_) -open import Luau.Syntax using (var; _$_; return; nil; function_⟨_⟩_end; done; _∙_) +open import Luau.Syntax using (var; _$_; return; nil; function_is_end; local_←_; done; _∙_; _⟨_⟩) open import Luau.Syntax.ToString using (exprToString; blockToString) -f = var "f" -x = var "x" - -ex1 : exprToString(f $ x) ≡ - "f(x)" +ex1 : exprToString(function "" ⟨ var "x" ⟩ is return (var "f" $ var "x") ∙ done end) ≡ + "function(x)\n" ++ + " return f(x)\n" ++ + "end" ex1 = refl -ex2 : blockToString(return nil ∙ done) ≡ - "return nil" +ex2 : blockToString(local var "x" ← nil ∙ return (var "x") ∙ done) ≡ + "local x = nil\n" ++ + "return x" ex2 = refl -ex3 : blockToString(function "f" ⟨ "x" ⟩ return x ∙ done end ∙ return f ∙ done) ≡ +ex3 : blockToString(function "f" ⟨ var "x" ⟩ is return (var "x") ∙ done end ∙ return (var "f") ∙ done) ≡ "local function f(x)\n" ++ " return x\n" ++ "end\n" ++ diff --git a/prototyping/Interpreter.agda b/prototyping/Interpreter.agda index 9184235e..3ec5b8d1 100644 --- a/prototyping/Interpreter.agda +++ b/prototyping/Interpreter.agda @@ -18,7 +18,7 @@ open import Luau.Run using (run; return; done; error) open import Luau.RuntimeError.ToString using (errToStringᴮ) open import Luau.Value.ToString using (valueToString) -runBlock : Block → IO ⊤ +runBlock : ∀ {a} → Block a → IO ⊤ runBlock block with run block runBlock block | return V D = putStrLn (valueToString V) runBlock block | done D = putStrLn "nil" diff --git a/prototyping/Luau/Heap.agda b/prototyping/Luau/Heap.agda index 1a0416e4..12f8dab0 100644 --- a/prototyping/Luau/Heap.agda +++ b/prototyping/Luau/Heap.agda @@ -5,39 +5,40 @@ open import FFI.Data.Maybe using (Maybe; just) open import FFI.Data.Vector using (Vector; length; snoc; empty) open import Luau.Addr using (Addr) open import Luau.Var using (Var) -open import Luau.Syntax using (Block; Expr; nil; addr; function⟨_⟩_end) +open import Luau.Syntax using (Block; Expr; Annotated; FunDec; nil; addr; function_is_end) -data HeapValue : Set where - function_⟨_⟩_end : Var → Var → Block → HeapValue +data HeapValue (a : Annotated) : Set where + function_is_end : FunDec a → Block a → HeapValue a -Heap = Vector HeapValue +Heap : Annotated → Set +Heap a = Vector (HeapValue a) -data _≡_⊕_↦_ : Heap → Heap → Addr → HeapValue → Set where +data _≡_⊕_↦_ {a} : Heap a → Heap a → Addr → HeapValue a → Set where defn : ∀ {H val} → ----------------------------------- (snoc H val) ≡ H ⊕ (length H) ↦ val -lookup : Heap → Addr → Maybe HeapValue -lookup = FFI.Data.Vector.lookup +_[_] : ∀ {a} → Heap a → Addr → Maybe (HeapValue a) +_[_] = FFI.Data.Vector.lookup -emp : Heap -emp = empty +∅ : ∀ {a} → Heap a +∅ = empty -data AllocResult (H : Heap) (V : HeapValue) : Set where - ok : ∀ a H′ → (H′ ≡ H ⊕ a ↦ V) → AllocResult H V +data AllocResult a (H : Heap a) (V : HeapValue a) : Set where + ok : ∀ b H′ → (H′ ≡ H ⊕ b ↦ V) → AllocResult a H V -alloc : ∀ H V → AllocResult H V +alloc : ∀ {a} H V → AllocResult a H V alloc H V = ok (length H) (snoc H V) defn -next : Heap → Addr +next : ∀ {a} → Heap a → Addr next = length -allocated : Heap → HeapValue → Heap +allocated : ∀ {a} → Heap a → HeapValue a → Heap a allocated = snoc --- next-emp : (length empty ≡ 0) +-- next-emp : (length ∅ ≡ 0) next-emp = FFI.Data.Vector.length-empty -- lookup-next : ∀ V H → (lookup (allocated H V) (next H) ≡ just V) diff --git a/prototyping/Luau/OpSem.agda b/prototyping/Luau/OpSem.agda index dcd474bf..878b8bd0 100644 --- a/prototyping/Luau/OpSem.agda +++ b/prototyping/Luau/OpSem.agda @@ -2,13 +2,13 @@ module Luau.OpSem where open import Agda.Builtin.Equality using (_≡_) open import FFI.Data.Maybe using (just) -open import Luau.Heap using (Heap; _≡_⊕_↦_; lookup; function_⟨_⟩_end) +open import Luau.Heap using (Heap; _≡_⊕_↦_; _[_]; function_is_end) open import Luau.Substitution using (_[_/_]ᴮ) -open import Luau.Syntax using (Expr; Stat; Block; nil; addr; var; function⟨_⟩_end; _$_; block_is_end; local_←_; _∙_; done; function_⟨_⟩_end; return) +open import Luau.Syntax using (Expr; Stat; Block; nil; addr; var; function_is_end; _$_; block_is_end; local_←_; _∙_; done; return; name; fun; arg) open import Luau.Value using (addr; val) -data _⊢_⟶ᴮ_⊣_ : Heap → Block → Block → Heap → Set -data _⊢_⟶ᴱ_⊣_ : Heap → Expr → Expr → Heap → Set +data _⊢_⟶ᴮ_⊣_ {a} : Heap a → Block a → Block a → Heap a → Set +data _⊢_⟶ᴱ_⊣_ {a} : Heap a → Expr a → Expr a → Heap a → Set data _⊢_⟶ᴱ_⊣_ where @@ -17,11 +17,11 @@ data _⊢_⟶ᴱ_⊣_ where ------------------- H ⊢ nil ⟶ᴱ nil ⊣ H - function : ∀ {H H′ a x B} → + function : ∀ {H H′ a F B} → - H′ ≡ H ⊕ a ↦ (function "anon" ⟨ x ⟩ B end) → + H′ ≡ H ⊕ a ↦ (function F is B end) → ------------------------------------------- - H ⊢ (function⟨ x ⟩ B end) ⟶ᴱ (addr a) ⊣ H′ + H ⊢ (function F is B end) ⟶ᴱ (addr a) ⊣ H′ app : ∀ {H H′ M M′ N} → @@ -29,11 +29,11 @@ data _⊢_⟶ᴱ_⊣_ where ----------------------------- H ⊢ (M $ N) ⟶ᴱ (M′ $ N) ⊣ H′ - beta : ∀ {H M a f x B} → + beta : ∀ {H M a F B} → - (lookup H a) ≡ just(function f ⟨ x ⟩ B end) → + H [ a ] ≡ just(function F is B end) → ----------------------------------------------------- - H ⊢ (addr a $ M) ⟶ᴱ (block f is local x ← M ∙ B end) ⊣ H + H ⊢ (addr a $ M) ⟶ᴱ (block (fun F) is local (arg F) ← M ∙ B end) ⊣ H block : ∀ {H H′ B B′ b} → @@ -61,14 +61,14 @@ data _⊢_⟶ᴮ_⊣_ where subst : ∀ {H x v B} → - ------------------------------------------------- - H ⊢ (local x ← val v ∙ B) ⟶ᴮ (B [ v / x ]ᴮ) ⊣ H + ------------------------------------------------------ + H ⊢ (local x ← val v ∙ B) ⟶ᴮ (B [ v / name x ]ᴮ) ⊣ H - function : ∀ {H H′ a f x B C} → + function : ∀ {H H′ a F B C} → - H′ ≡ H ⊕ a ↦ (function f ⟨ x ⟩ C end) → + H′ ≡ H ⊕ a ↦ (function F is C end) → -------------------------------------------------------------- - H ⊢ (function f ⟨ x ⟩ C end ∙ B) ⟶ᴮ (B [ addr a / f ]ᴮ) ⊣ H′ + H ⊢ (function F is C end ∙ B) ⟶ᴮ (B [ addr a / fun F ]ᴮ) ⊣ H′ return : ∀ {H H′ M M′ B} → @@ -76,7 +76,7 @@ data _⊢_⟶ᴮ_⊣_ where -------------------------------------------- H ⊢ (return M ∙ B) ⟶ᴮ (return M′ ∙ B) ⊣ H′ -data _⊢_⟶*_⊣_ : Heap → Block → Block → Heap → Set where +data _⊢_⟶*_⊣_ {a} : Heap a → Block a → Block a → Heap a → Set where refl : ∀ {H B} → diff --git a/prototyping/Luau/Run.agda b/prototyping/Luau/Run.agda index 29709ceb..d84f75de 100644 --- a/prototyping/Luau/Run.agda +++ b/prototyping/Luau/Run.agda @@ -1,20 +1,20 @@ module Luau.Run where open import Agda.Builtin.Equality using (_≡_; refl) -open import Luau.Heap using (Heap; emp) +open import Luau.Heap using (Heap; ∅) open import Luau.Syntax using (Block; return; _∙_; done) open import Luau.OpSem using (_⊢_⟶*_⊣_; refl; step) open import Luau.Value using (val) open import Properties.Step using (stepᴮ; step; return; done; error) open import Luau.RuntimeError using (RuntimeErrorᴮ) -data RunResult (H : Heap) (B : Block) : Set where +data RunResult {a} (H : Heap a) (B : Block a) : Set where return : ∀ V {B′ H′} → (H ⊢ B ⟶* (return (val V) ∙ B′) ⊣ H′) → RunResult H B done : ∀ {H′} → (H ⊢ B ⟶* done ⊣ H′) → RunResult H B error : ∀ {B′ H′} → (RuntimeErrorᴮ H′ B′) → (H ⊢ B ⟶* B′ ⊣ H′) → RunResult H B {-# TERMINATING #-} -run′ : ∀ H B → RunResult H B +run′ : ∀ {a} H B → RunResult {a} H B run′ H B with stepᴮ H B run′ H B | step H′ B′ D with run′ H′ B′ run′ H B | step H′ B′ D | return V D′ = return V (step D D′) @@ -24,5 +24,5 @@ run′ H _ | return V refl = return V refl run′ H _ | done refl = done refl run′ H B | error E = error E refl -run : ∀ B → RunResult emp B -run = run′ emp +run : ∀ {a} B → RunResult {a} ∅ B +run = run′ ∅ diff --git a/prototyping/Luau/RuntimeError.agda b/prototyping/Luau/RuntimeError.agda index 86e7cf21..e514dc9d 100644 --- a/prototyping/Luau/RuntimeError.agda +++ b/prototyping/Luau/RuntimeError.agda @@ -1,17 +1,17 @@ module Luau.RuntimeError where open import Agda.Builtin.Equality using (_≡_) -open import Luau.Heap using (Heap; lookup) +open import Luau.Heap using (Heap; _[_]) open import FFI.Data.Maybe using (just; nothing) -open import Luau.Syntax using (Block; Expr; nil; var; addr; function⟨_⟩_end; block_is_end; _$_; local_←_; function_⟨_⟩_end; return; done; _∙_) +open import Luau.Syntax using (Block; Expr; nil; var; addr; block_is_end; _$_; local_←_; return; done; _∙_) -data RuntimeErrorᴮ (H : Heap) : Block → Set -data RuntimeErrorᴱ (H : Heap) : Expr → Set +data RuntimeErrorᴮ {a} (H : Heap a) : Block a → Set +data RuntimeErrorᴱ {a} (H : Heap a) : Expr a → Set data RuntimeErrorᴱ H where NilIsNotAFunction : ∀ {M} → RuntimeErrorᴱ H (nil $ M) UnboundVariable : ∀ x → RuntimeErrorᴱ H (var x) - SEGV : ∀ a → (lookup H a ≡ nothing) → RuntimeErrorᴱ H (addr a) + SEGV : ∀ a → (H [ a ] ≡ nothing) → RuntimeErrorᴱ H (addr a) app : ∀ {M N} → RuntimeErrorᴱ H M → RuntimeErrorᴱ H (M $ N) block : ∀ b {B} → RuntimeErrorᴮ H B → RuntimeErrorᴱ H (block b is B end) diff --git a/prototyping/Luau/RuntimeError/ToString.agda b/prototyping/Luau/RuntimeError/ToString.agda index f11756f7..e8287157 100644 --- a/prototyping/Luau/RuntimeError/ToString.agda +++ b/prototyping/Luau/RuntimeError/ToString.agda @@ -4,9 +4,10 @@ open import FFI.Data.String using (String; _++_) open import Luau.RuntimeError using (RuntimeErrorᴮ; RuntimeErrorᴱ; local; return; NilIsNotAFunction; UnboundVariable; SEGV; app; block) open import Luau.Addr.ToString using (addrToString) open import Luau.Var.ToString using (varToString) +open import Luau.Syntax using (name) -errToStringᴱ : ∀ {H B} → RuntimeErrorᴱ H B → String -errToStringᴮ : ∀ {H B} → RuntimeErrorᴮ H B → String +errToStringᴱ : ∀ {a H B} → RuntimeErrorᴱ {a} H B → String +errToStringᴮ : ∀ {a H B} → RuntimeErrorᴮ {a} H B → String errToStringᴱ NilIsNotAFunction = "nil is not a function" errToStringᴱ (UnboundVariable x) = "variable " ++ varToString x ++ " is unbound" @@ -14,5 +15,5 @@ errToStringᴱ (SEGV a x) = "address " ++ addrToString a ++ " is unallocated" errToStringᴱ (app E) = errToStringᴱ E errToStringᴱ (block b E) = errToStringᴮ E ++ "\n in call of function " ++ varToString b -errToStringᴮ (local x E) = errToStringᴱ E ++ "\n in definition of " ++ varToString x +errToStringᴮ (local x E) = errToStringᴱ E ++ "\n in definition of " ++ varToString (name x) errToStringᴮ (return E) = errToStringᴱ E ++ "\n in return statement" diff --git a/prototyping/Luau/Substitution.agda b/prototyping/Luau/Substitution.agda index 1a361174..b956aeae 100644 --- a/prototyping/Luau/Substitution.agda +++ b/prototyping/Luau/Substitution.agda @@ -1,24 +1,24 @@ module Luau.Substitution where -open import Luau.Syntax using (Expr; Stat; Block; nil; addr; var; function⟨_⟩_end; _$_; block_is_end; local_←_; _∙_; done; function_⟨_⟩_end; return) +open import Luau.Syntax using (Expr; Stat; Block; nil; addr; var; function_is_end; _$_; block_is_end; local_←_; _∙_; done; return; _⟨_⟩ ; name; fun; arg) open import Luau.Value using (Value; val) open import Luau.Var using (Var; _≡ⱽ_) open import Properties.Dec using (Dec; yes; no) -_[_/_]ᴱ : Expr → Value → Var → Expr -_[_/_]ᴮ : Block → Value → Var → Block -var_[_/_]ᴱwhenever_ : ∀ {P} → Var → Value → Var → (Dec P) → Expr -_[_/_]ᴮunless_ : ∀ {P} → Block → Value → Var → (Dec P) → Block +_[_/_]ᴱ : ∀ {a} → Expr a → Value → Var → Expr a +_[_/_]ᴮ : ∀ {a} → Block a → Value → Var → Block a +var_[_/_]ᴱwhenever_ : ∀ {a P} → Var → Value → Var → (Dec P) → Expr a +_[_/_]ᴮunless_ : ∀ {a P} → Block a → Value → Var → (Dec P) → Block a nil [ v / x ]ᴱ = nil var y [ v / x ]ᴱ = var y [ v / x ]ᴱwhenever (x ≡ⱽ y) addr a [ v / x ]ᴱ = addr a (M $ N) [ v / x ]ᴱ = (M [ v / x ]ᴱ) $ (N [ v / x ]ᴱ) -function⟨ y ⟩ C end [ v / x ]ᴱ = function⟨ y ⟩ C [ v / x ]ᴮunless (x ≡ⱽ y) end +function F is C end [ v / x ]ᴱ = function F is C [ v / x ]ᴮunless (x ≡ⱽ name(arg F)) end block b is C end [ v / x ]ᴱ = block b is C [ v / x ]ᴮ end -(function f ⟨ y ⟩ C end ∙ B) [ v / x ]ᴮ = function f ⟨ y ⟩ (C [ v / x ]ᴮunless (x ≡ⱽ y)) end ∙ (B [ v / x ]ᴮunless (x ≡ⱽ f)) -(local y ← M ∙ B) [ v / x ]ᴮ = local y ← (M [ v / x ]ᴱ) ∙ (B [ v / x ]ᴮunless (x ≡ⱽ y)) +(function F is C end ∙ B) [ v / x ]ᴮ = function F is (C [ v / x ]ᴮunless (x ≡ⱽ name(arg F))) end ∙ (B [ v / x ]ᴮunless (x ≡ⱽ fun F)) +(local y ← M ∙ B) [ v / x ]ᴮ = local y ← (M [ v / x ]ᴱ) ∙ (B [ v / x ]ᴮunless (x ≡ⱽ name y)) (return M ∙ B) [ v / x ]ᴮ = return (M [ v / x ]ᴱ) ∙ (B [ v / x ]ᴮ) done [ v / x ]ᴮ = done diff --git a/prototyping/Luau/Syntax.agda b/prototyping/Luau/Syntax.agda index 3fe05bc8..04970b62 100644 --- a/prototyping/Luau/Syntax.agda +++ b/prototyping/Luau/Syntax.agda @@ -1,27 +1,55 @@ module Luau.Syntax where +open import Agda.Builtin.Equality using (_≡_) +open import Properties.Dec using (⊥) open import Luau.Var using (Var) open import Luau.Addr using (Addr) +open import Luau.Type using (Type) infixr 5 _∙_ -data Block : Set -data Stat : Set -data Expr : Set +data Annotated : Set where + maybe : Annotated + yes : Annotated -data Block where - _∙_ : Stat → Block → Block - done : Block +data VarDec : Annotated → Set where + var : Var → VarDec maybe + var_∈_ : ∀ {a} → Var → Type → VarDec a -data Stat where - function_⟨_⟩_end : Var → Var → Block → Stat - local_←_ : Var → Expr → Stat - return : Expr → Stat +name : ∀ {a} → VarDec a → Var +name (var x) = x +name (var x ∈ T) = x + +data FunDec : Annotated → Set where + _⟨_⟩∈_ : ∀ {a} → Var → VarDec a → Type → FunDec a + _⟨_⟩ : Var → VarDec maybe → FunDec maybe + +fun : ∀ {a} → FunDec a → Var +fun (f ⟨ x ⟩∈ T) = f +fun (f ⟨ x ⟩) = f + +arg : ∀ {a} → FunDec a → VarDec a +arg (f ⟨ x ⟩∈ T) = x +arg (f ⟨ x ⟩) = x + +data Block (a : Annotated) : Set +data Stat (a : Annotated) : Set +data Expr (a : Annotated) : Set + +data Block a where + _∙_ : Stat a → Block a → Block a + done : Block a + +data Stat a where + function_is_end : FunDec a → Block a → Stat a + local_←_ : VarDec a → Expr a → Stat a + return : Expr a → Stat a + +data Expr a where + nil : Expr a + var : Var → Expr a + addr : Addr → Expr a + _$_ : Expr a → Expr a → Expr a + function_is_end : FunDec a → Block a → Expr a + block_is_end : Var → Block a → Expr a -data Expr where - nil : Expr - var : Var → Expr - addr : Addr → Expr - _$_ : Expr → Expr → Expr - function⟨_⟩_end : Var → Block → Expr - block_is_end : Var → Block → Expr diff --git a/prototyping/Luau/Syntax/FromJSON.agda b/prototyping/Luau/Syntax/FromJSON.agda index 0a4164fb..b081d156 100644 --- a/prototyping/Luau/Syntax/FromJSON.agda +++ b/prototyping/Luau/Syntax/FromJSON.agda @@ -1,6 +1,6 @@ module Luau.Syntax.FromJSON where -open import Luau.Syntax using (Block; Stat ; Expr; nil; _$_; var; function⟨_⟩_end; local_←_; function_⟨_⟩_end; return; done; _∙_) +open import Luau.Syntax using (Block; Stat ; Expr; nil; _$_; var; function_is_end; _⟨_⟩; local_←_; return; done; _∙_; maybe) open import Agda.Builtin.List using (List; _∷_; []) @@ -31,12 +31,12 @@ lookupIn (key ∷ keys) obj with lookup (fromString key) obj lookupIn (key ∷ keys) obj | nothing = lookupIn keys obj lookupIn (key ∷ keys) obj | just value = (key , value) -exprFromJSON : Value → Either String Expr -exprFromObject : Object → Either String Expr -statFromJSON : Value → Either String Stat -statFromObject : Object → Either String Stat -blockFromJSON : Value → Either String Block -blockFromArray : Array → Either String Block +exprFromJSON : Value → Either String (Expr maybe) +exprFromObject : Object → Either String (Expr maybe) +statFromJSON : Value → Either String (Stat maybe) +statFromObject : Object → Either String (Stat maybe) +blockFromJSON : Value → Either String (Block maybe) +blockFromArray : Array → Either String (Block maybe) exprFromJSON (object obj) = exprFromObject obj exprFromJSON val = Left "AstExpr not an object" @@ -55,7 +55,7 @@ exprFromObject obj | just (string "AstExprCall") | _ | nothing = Left ("AstExpr exprFromObject obj | just (string "AstExprConstantNil") = Right nil exprFromObject obj | just (string "AstExprFunction") with lookup args obj | lookup body obj exprFromObject obj | just (string "AstExprFunction") | just (array arr) | just value with head arr | blockFromJSON value -exprFromObject obj | just (string "AstExprFunction") | just (array arr) | just value | just (string x) | Right B = Right (function⟨ x ⟩ B end) +exprFromObject obj | just (string "AstExprFunction") | just (array arr) | just value | just (string x) | Right B = Right (function "" ⟨ var x ⟩ is B end) exprFromObject obj | just (string "AstExprFunction") | just (array arr) | just value | just _ | Right B = Left "AstExprFunction args not a string array" exprFromObject obj | just (string "AstExprFunction") | just (array arr) | just value | nothing | Right B = Left "Unsupported AstExprFunction empty args" exprFromObject obj | just (string "AstExprFunction") | just (array arr) | just value | _ | Left err = Left err @@ -78,7 +78,7 @@ statFromObject obj with lookup type obj statFromObject obj | just(string "AstStatLocal") with lookup vars obj | lookup values obj statFromObject obj | just(string "AstStatLocal") | just(array arr1) | just(array arr2) with head(arr1) | head(arr2) statFromObject obj | just(string "AstStatLocal") | just(array arr1) | just(array arr2) | just(string x) | just(value) with exprFromJSON(value) -statFromObject obj | just(string "AstStatLocal") | just(array arr1) | just(array arr2) | just(string x) | just(value) | Right M = Right (local x ← M) +statFromObject obj | just(string "AstStatLocal") | just(array arr1) | just(array arr2) | just(string x) | just(value) | Right M = Right (local (var x) ← M) statFromObject obj | just(string "AstStatLocal") | just(array arr1) | just(array arr2) | just(string x) | just(value) | Left err = Left err statFromObject obj | just(string "AstStatLocal") | just(array arr1) | just(array arr2) | just(string x) | nothing = Left "AstStatLocal empty values" statFromObject obj | just(string "AstStatLocal") | just(array arr1) | just(array arr2) | just(_) | _ = Left "AstStatLocal vars not a string array" @@ -89,7 +89,7 @@ statFromObject obj | just(string "AstStatLocal") | just(_) | nothing = Left "Ast statFromObject obj | just(string "AstStatLocal") | nothing | _ = Left "AstStatLocal missing vars" statFromObject obj | just(string "AstStatLocalFunction") with lookup name obj | lookup func obj statFromObject obj | just(string "AstStatLocalFunction") | just (string f) | just value with exprFromJSON value -statFromObject obj | just(string "AstStatLocalFunction") | just (string f) | just value | Right (function⟨ x ⟩ B end) = Right (function f ⟨ x ⟩ B end) +statFromObject obj | just(string "AstStatLocalFunction") | just (string f) | just value | Right (function "" ⟨ x ⟩ is B end) = Right (function f ⟨ x ⟩ is B end) statFromObject obj | just(string "AstStatLocalFunction") | just (string f) | just value | Left err = Left err statFromObject obj | just(string "AstStatLocalFunction") | just _ | just _ | Right _ = Left "AstStatLocalFunction func is not an AstExprFunction" statFromObject obj | just(string "AstStatLocalFunction") | just _ | just _ = Left "AstStatLocalFunction name is not a string" diff --git a/prototyping/Luau/Syntax/ToString.agda b/prototyping/Luau/Syntax/ToString.agda index afec0935..18d36175 100644 --- a/prototyping/Luau/Syntax/ToString.agda +++ b/prototyping/Luau/Syntax/ToString.agda @@ -1,13 +1,24 @@ module Luau.Syntax.ToString where -open import Luau.Syntax using (Block; Stat; Expr; nil; var; addr; _$_; function⟨_⟩_end; return; function_⟨_⟩_end ;local_←_; _∙_; done; block_is_end) +open import Luau.Syntax using (Block; Stat; Expr; VarDec; FunDec; nil; var; var_∈_; addr; _$_; function_is_end; return; local_←_; _∙_; done; block_is_end; _⟨_⟩; _⟨_⟩∈_) open import FFI.Data.String using (String; _++_) open import Luau.Addr.ToString using (addrToString) +open import Luau.Type.ToString using (typeToString) open import Luau.Var.ToString using (varToString) -exprToString′ : String → Expr → String -statToString′ : String → Stat → String -blockToString′ : String → Block → String +varDecToString : ∀ {a} → VarDec a → String +varDecToString (var x) = varToString x +varDecToString (var x ∈ T) = varToString x ++ " : " ++ typeToString T + +funDecToString : ∀ {a} → FunDec a → String +funDecToString ("" ⟨ x ⟩∈ T) = "function(" ++ varDecToString x ++ "): " ++ typeToString T +funDecToString ("" ⟨ x ⟩) = "function(" ++ varDecToString x ++ ")" +funDecToString (f ⟨ x ⟩∈ T) = "function " ++ varToString f ++ "(" ++ varDecToString x ++ "): " ++ typeToString T +funDecToString (f ⟨ x ⟩) = "function " ++ varToString f ++ "(" ++ varDecToString x ++ ")" + +exprToString′ : ∀ {a} → String → Expr a → String +statToString′ : ∀ {a} → String → Stat a → String +blockToString′ : ∀ {a} → String → Block a → String exprToString′ lb nil = "nil" @@ -17,21 +28,21 @@ exprToString′ lb (var x) = varToString(x) exprToString′ lb (M $ N) = (exprToString′ lb M) ++ "(" ++ (exprToString′ lb N) ++ ")" -exprToString′ lb (function⟨ x ⟩ B end) = - "function(" ++ x ++ ")" ++ lb ++ +exprToString′ lb (function F is B end) = + funDecToString F ++ lb ++ " " ++ (blockToString′ (lb ++ " ") B) ++ lb ++ "end" exprToString′ lb (block b is B end) = - "(function " ++ b ++ "()" ++ lb ++ + "(" ++ b ++ "()" ++ lb ++ " " ++ (blockToString′ (lb ++ " ") B) ++ lb ++ "end)()" -statToString′ lb (function f ⟨ x ⟩ B end) = - "local function " ++ f ++ "(" ++ x ++ ")" ++ lb ++ +statToString′ lb (function F is B end) = + "local " ++ funDecToString F ++ lb ++ " " ++ (blockToString′ (lb ++ " ") B) ++ lb ++ "end" statToString′ lb (local x ← M) = - "local " ++ x ++ " = " ++ (exprToString′ lb M) + "local " ++ varDecToString x ++ " = " ++ (exprToString′ lb M) statToString′ lb (return M) = "return " ++ (exprToString′ lb M) @@ -39,11 +50,11 @@ blockToString′ lb (S ∙ done) = statToString′ lb S blockToString′ lb (S ∙ B) = statToString′ lb S ++ lb ++ blockToString′ lb B blockToString′ lb (done) = "" -exprToString : Expr → String +exprToString : ∀ {a} → Expr a → String exprToString = exprToString′ "\n" -statToString : Stat → String +statToString : ∀ {a} → Stat a → String statToString = statToString′ "\n" -blockToString : Block → String +blockToString : ∀ {a} → Block a → String blockToString = blockToString′ "\n" diff --git a/prototyping/Luau/Value.agda b/prototyping/Luau/Value.agda index 855e663e..4768f859 100644 --- a/prototyping/Luau/Value.agda +++ b/prototyping/Luau/Value.agda @@ -1,14 +1,14 @@ module Luau.Value where open import Luau.Addr using (Addr) -open import Luau.Syntax using (Block; Expr; nil; addr; function⟨_⟩_end) +open import Luau.Syntax using (Block; Expr; nil; addr) open import Luau.Var using (Var) data Value : Set where nil : Value addr : Addr → Value -val : Value → Expr +val : ∀ {a} → Value → Expr a val nil = nil val (addr a) = addr a diff --git a/prototyping/PrettyPrinter.agda b/prototyping/PrettyPrinter.agda index dfb2c053..4a6c7dd6 100644 --- a/prototyping/PrettyPrinter.agda +++ b/prototyping/PrettyPrinter.agda @@ -15,7 +15,7 @@ open import Luau.Syntax using (Block) open import Luau.Syntax.FromJSON using (blockFromJSON) open import Luau.Syntax.ToString using (blockToString) -runBlock : Block → IO ⊤ +runBlock : ∀ {a} → Block a → IO ⊤ runBlock block = putStrLn (blockToString block) runJSON : Value → IO ⊤ diff --git a/prototyping/Properties/Step.agda b/prototyping/Properties/Step.agda index 0eddd9fe..eeda4ef2 100644 --- a/prototyping/Properties/Step.agda +++ b/prototyping/Properties/Step.agda @@ -2,16 +2,16 @@ module Properties.Step where open import Agda.Builtin.Equality using (_≡_; refl) open import FFI.Data.Maybe using (just; nothing) -open import Luau.Heap using (Heap; lookup; alloc; ok; function_⟨_⟩_end) -open import Luau.Syntax using (Block; Expr; nil; var; addr; function⟨_⟩_end; block_is_end; _$_; local_←_; function_⟨_⟩_end; return; done; _∙_) +open import Luau.Heap using (Heap; _[_]; alloc; ok; function_is_end) +open import Luau.Syntax using (Block; Expr; nil; var; addr; function_is_end; block_is_end; _$_; local_←_; return; done; _∙_; name; fun; arg) open import Luau.OpSem using (_⊢_⟶ᴱ_⊣_; _⊢_⟶ᴮ_⊣_; app ; beta; function; block; return; done; local; subst) open import Luau.RuntimeError using (RuntimeErrorᴱ; RuntimeErrorᴮ; NilIsNotAFunction; UnboundVariable; SEGV; app; block; local; return) open import Luau.Substitution using (_[_/_]ᴮ) open import Luau.Value using (nil; addr; val) open import Properties.Remember using (remember; _,_) -data StepResultᴮ (H : Heap) (B : Block) : Set -data StepResultᴱ (H : Heap) (M : Expr) : Set +data StepResultᴮ {a} (H : Heap a) (B : Block a) : Set +data StepResultᴱ {a} (H : Heap a) (M : Expr a) : Set data StepResultᴮ H B where step : ∀ H′ B′ → (H ⊢ B ⟶ᴮ B′ ⊣ H′) → StepResultᴮ H B @@ -24,8 +24,8 @@ data StepResultᴱ H M where value : ∀ V → (M ≡ val V) → StepResultᴱ H M error : (RuntimeErrorᴱ H M) → StepResultᴱ H M -stepᴱ : ∀ H M → StepResultᴱ H M -stepᴮ : ∀ H B → StepResultᴮ H B +stepᴱ : ∀ {a} H M → StepResultᴱ {a} H M +stepᴮ : ∀ {a} H B → StepResultᴮ {a} H B stepᴱ H nil = value nil refl stepᴱ H (var x) = error (UnboundVariable x) @@ -33,23 +33,23 @@ stepᴱ H (addr a) = value (addr a) refl stepᴱ H (M $ N) with stepᴱ H M stepᴱ H (M $ N) | step H′ M′ D = step H′ (M′ $ N) (app D) stepᴱ H (nil $ N) | value nil refl = error NilIsNotAFunction -stepᴱ H (addr a $ N) | value (addr a) refl with remember (lookup H a) +stepᴱ H (addr a $ N) | value (addr a) refl with remember (H [ a ]) stepᴱ H (addr a $ N) | value (addr a) refl | (nothing , p) = error (app (SEGV a p)) -stepᴱ H (addr a $ N) | value (addr a) refl | (just(function f ⟨ x ⟩ B end) , p) = step H (block f is local x ← N ∙ B end) (beta p) +stepᴱ H (addr a $ N) | value (addr a) refl | (just(function F is B end) , p) = step H (block fun F is (local arg F ← N) ∙ B end) (beta p) stepᴱ H (M $ N) | error E = error (app E) -stepᴱ H (function⟨ x ⟩ B end) with alloc H (function "anon" ⟨ x ⟩ B end) -stepᴱ H (function⟨ x ⟩ B end) | ok a H′ p = step H′ (addr a) (function p) stepᴱ H (block b is B end) with stepᴮ H B stepᴱ H (block b is B end) | step H′ B′ D = step H′ (block b is B′ end) (block D) stepᴱ H (block b is (return _ ∙ B′) end) | return V refl = step H (val V) return stepᴱ H (block b is done end) | done refl = step H nil done stepᴱ H (block b is B end) | error E = error (block b E) +stepᴱ H (function F is C end) with alloc H (function F is C end) +stepᴱ H function F is C end | ok a H′ p = step H′ (addr a) (function p) -stepᴮ H (function f ⟨ x ⟩ C end ∙ B) with alloc H (function f ⟨ x ⟩ C end) -stepᴮ H (function f ⟨ x ⟩ C end ∙ B) | ok a H′ p = step H′ (B [ addr a / f ]ᴮ) (function p) +stepᴮ H (function F is C end ∙ B) with alloc H (function F is C end) +stepᴮ H (function F is C end ∙ B) | ok a H′ p = step H′ (B [ addr a / fun F ]ᴮ) (function p) stepᴮ H (local x ← M ∙ B) with stepᴱ H M stepᴮ H (local x ← M ∙ B) | step H′ M′ D = step H′ (local x ← M′ ∙ B) (local D) -stepᴮ H (local x ← _ ∙ B) | value V refl = step H (B [ V / x ]ᴮ) subst +stepᴮ H (local x ← _ ∙ B) | value V refl = step H (B [ V / name x ]ᴮ) subst stepᴮ H (local x ← M ∙ B) | error E = error (local x E) stepᴮ H (return M ∙ B) with stepᴱ H M stepᴮ H (return M ∙ B) | step H′ M′ D = step H′ (return M′ ∙ B) (return D) From 7d679b317fea4db3bd979c682c1a3be1f1c092a5 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Mon, 14 Feb 2022 10:04:07 -0800 Subject: [PATCH 160/399] RFC: Generalized iteration (#335) Co-authored-by: dcope-rbx <91100513+dcope-rbx@users.noreply.github.com> Co-authored-by: vegorov-rbx <75688451+vegorov-rbx@users.noreply.github.com> --- rfcs/generalized-iteration.md | 122 ++++++++++++++++++++++++++++++++++ 1 file changed, 122 insertions(+) create mode 100644 rfcs/generalized-iteration.md diff --git a/rfcs/generalized-iteration.md b/rfcs/generalized-iteration.md new file mode 100644 index 00000000..72bdd69e --- /dev/null +++ b/rfcs/generalized-iteration.md @@ -0,0 +1,122 @@ +# Generalized iteration + +## Summary + +Introduce support for iterating over tables without using `pairs`/`ipairs` as well as a generic customization point for iteration via `__iter` metamethod. + +## Motivation + +Today there are many different ways to iterate through various containers that are syntactically incompatible. + +To iterate over arrays, you need to use `ipairs`: `for i, v in ipairs(t) do`. The traversal goes over a sequence `1..k` of numeric keys until `t[k] == nil`, preserving order. + +To iterate over dictionaries, you need to use `pairs`: `for k, v in pairs(t) do`. The traversal goes over all keys, numeric and otherwise, but doesn't guarantee an order; when iterating over arrays this may happen to work but is not guaranteed to work, as it depends on how keys are distributed between array and hash portion. + +To iterate over custom objects, whether they are represented as tables (user-specified) or userdata (host-specified), you need to expose special iteration methods, for example `for k, v in obj:Iterator() do`. + +All of these rely on the standard Lua iteration protocol, but it's impossible to trigger them in a generic fashion. Additionally, you *must* use one of `pairs`/`ipairs`/`next` to iterate over tables, which is easy to forget - a naive `for k, v in tab do` doesn't work and produces a hard-to-understand error `attempt to call a table value`. + +This proposal solves all of these by providing a way to implement uniform iteration with self-iterating objects by allowing to iterate over objects and tables directly via convenient `for k, v in obj do` syntax, and specifies the default iteration behavior for tables, thus mostly rendering `pairs`/`ipairs` obsolete - making Luau easier to use and teach. + +## Design + +In Lua, `for vars in iter do` has the following semantics (otherwise known as the iteration protocol): `iter` is expanded into three variables, `gen`, `state` and `index` (using `nil` if `iter` evaluates to fewer than 3 results); after this the loop is converted to the following pseudocode: + +```lua +while true do + vars... = gen(state, index) + index = vars... -- copy the first variable into the index + if index == nil then break end + + -- loop body goes here +end +``` + +This is a general mechanism that can support iteration through many containers, especially if `gen` is allowed to mutate state. Importantly, the *first* returned variable (which is exposed to the user) is used to continue the process on the next iteration - this can be limiting because it may require `gen` or `state` to carry extra internal iteration data for efficiency. To work around this for table iteration to avoid repeated calls to `next`, Luau compiler produces a special instruction sequence that recognizes `pairs`/`ipairs` iterators and stores the iteration index separately. + +Thus, today the loop `for k, v in tab do` effectively executes `k, v = tab()` on the first iteration, which is why it yields `attempt to call a table value`. If the object defines `__call` metamethod then it can act as a self-iterating method, but this is not idiomatic, not efficient and not pure/clean. + +This proposal comes in two parts: general support for `__iter` metamethod and default implementation for tables without one. With both of these in place, there's going to be a single, idiomatic, general and performant way to iterate through the object of any type: + +```lua +for k, v in obj do +... +end +``` + +### __iter + +To support self-iterating objects, we modify the iteration protocol as follows: instead of simply expanding the result of expression `iter` into three variables (`gen`, `state` and `index`), we check if the first result has an `__iter` metamethod (which can be the case if it's a table, userdata or another composite object (e.g. a record in the future). If it does, the metamethod is called with `gen` as the first argument, and the returned three values replace `gen`/`state`/`index`. This happens *before* the loop: + +```lua +if getmetatable(gen) and getmetatable(gen).__iter then + gen, state, index = getmetatable(gen).__iter(gen) +end +``` + +This check is comparatively trivial: usually `gen` is a function, and functions don't have metatables; as such we can simply check the type of `gen` and if it's a table/userdata, we can check if it has a metamethod `__iter`. Due to tag-method cache, this check is also very cheap if the metamethod is absent. + +This allows objects to provide a custom function that guides the iteration. Since the function is called once, it is easy to reuse other functions in the implementation, for example here's a node object that exposes iteration through its children: + +```lua +local Node = {} +Node.__index = Node + +function Node.new(children) + return setmetatable({ children = children }, Node) +end + +function Node:__iter() + return next, self.children +end +``` + +Luau compiler already emits a bytecode instruction, FORGPREP*, to perform initial loop setup - this is where we can evaluate `__iter` as well. + +Naturally, this means that if the table has `__iter` metamethod and you need to iterate through the table fields instead of using the provided metamethod, you can't rely on the general iteration scheme and need to use `pairs`. This is similar to other parts of the language, like `t[k]` vs `rawget(t, 'k')`, where the default behavior is overrideable but a library function can help peek behind the curtain. + +### Default table iteration + +If the argument is a table and it does not implement `__iter` metamethod, we treat this as an attempt to iterate through the table using the builtin iteration order. + +> Note: we also check if the table implements `__call`; if it does, we fall back to the default handling. We may be able to remove this check in the future, but we will need this initially to preserve backwards compatibility with custom table-driven iterator objects that implement `__call`. In either case, we will be able to collect detailed analytics about the use of `__call` in iteration, and if neither is present we can emit a specialized error message such as `object X is not iteratable`. + +To have a single, unified, iteration scheme over tables regardless of whether they are arrays or dictionaries, we establish the following semantics: + +- First, the traversal goes over numeric keys in range `1..k` up until reaching the first `k` such that `t[k] == nil` +- Then, the traversal goes over the remaining keys (with non-nil values), numeric and otherwise, in unspecified order. + +For arrays with gaps, this iterates until the first gap in order, and the remaining order is not specified. + +> Note: This behavior is similar to what `pairs` happens to provide today, but `pairs` doesn't give any guarantees, and it doesn't always provide this behavior in practice. + +To ensure that this traversal is performant, the actual implementation of the traversal involves going over the array part (in index order) and then over the hash part (in hash order). For that implementation to satisfy the criteria above, we need to make two additional changes to table insertion/rehash: + +- When inserting key `k` in the table when `k == t->sizearray + 1`, we force the table to rehash (resize its array portion). Today this is only performed if the hash portion is full, as such sometimes numeric keys can end up in the hash part. +- When rehashing the table, we ensure that the hash part doesn't contain the key `newsizearray + 1`. This requires checking if the table has this key, which may require an additional hash lookup but we only need to do this in rare cases based on the analysis of power-of-two key buckets that we already collect during rehash. + +These changes guarantee that the order observed via standard traversal with `next`/`pairs` matches the guarantee above, which is nice because it means we can minimize the complexity cost of this change by reusing the traversal code, including VM optimizations. They also mean that the array boundary (aka `#t`) can *always* be computed from just the array portion, which simplifies the table length computation and may slightly speed it up. + +## Drawbacks + +This makes `for` desugaring and implementation a little more complicated; it's not a large complexity factor in Luau because we already have special handling for `for` loops in the VM, but it's something to keep in mind. + +While the proposed iteration scheme should be a superset to both `pairs` and `ipairs` for tables, for arrays `ipairs` may in some cases be faster because it stops at the first `nil`, whereas the proposed new scheme (like `pairs`) needs to iterate through the rest of the table's array storage. This may be fixable in the future, if we replace our cached table length (`aboundary`) with Lua 5.4's `alimit`, which maintains the invariant that all values after `alimit` in the array are `nil`. This would make default table iteration maximally performant as well as help us accelerate GC in some cases, but will require extra checks during table assignments which is a cost we may not be willing to pay. Thus it is theoretically possible that we will end up with `ipairs` being a slightly faster equivalent for array iteration forever. + +The resulting iteration behavior, while powerful, increases the divergence between Luau and Lua, making more programs that are written for Luau not runnable in Lua. Luau language in general does not consider this type of compatibility essential, but this is noted for posterity. + +The changes in insertion behavior that facilitate single iteration order may have a small cost; that said, they are currently understood to belong to paths that are already slow and the added cost is minimal. + +The extra semantics will make inferring the types of the variables in a for loop more difficult - if we know the type of the expression that is being iterated through it probably is not a problem though. + +## Alternatives + +Other major designs have been considered. + +A minor variation of the proposal involves having `__iter` be called on every iteration instead of at loop startup, effectively having `__iter` work as an alternative to `__call`. The issue with this variant is that while it's a little simpler to specify and implement, it restricts the options when implementing custom iteratable objects, because it would be difficult for iteratable objects to store custom iteration state elsewhere since `__iter` method would effectively need to be pure, as it can't modify the object itself as more than one concurrent iteration needs to be supported. + +A major variation of the proposal involves instead supporting `__pairs` from Lua 5.2. The issue with this variant is that it still requires the use of a library method, `pairs`, to work, which doesn't make the language simpler as far as table iteration, which is the 95% case, is concerned. Additionally, with some rare exceptions metamethods today extend the *language* behavior, not the *library* behavior, and extending extra library functions with metamethods does not seem true to the core of the language. Finally, this only works if the user uses `pairs` to iterate and doesn't work with `ipairs`/`next`. + +Another variation involves using a new pseudo-keyword, `foreach`, instead of overloading existing `for`, and only using the new `__iter` semantics there. This can more cleanly separate behavior, requiring the object to have an `__iter` metamethod (or be a table) in `foreach` - which also avoids having to deal with `__call` - but it also requires teaching the users a new keyword which fragments the iteration space a little bit more. Compared to that, the main proposal doesn't introduce new divergent syntax, and merely tweaks existing behavior to be more general, thus making an existing construct easier to use. + +Finally, the author also considered and rejected extending the iteration protocol as part of this change. One problem with the current protocol is that the iterator requires an allocation (per loop execution) to keep extra state that isn't exposed to the user. The builtin iterators like `pairs`/`ipairs` work around this by feeding the user-visible index back to the search function, but that's not always practical. That said, having a different iteration protocol in effect only when `__iter` is used makes the language more complicated for unclear efficiency gains, thus this design doesn't suggest a new core protocol to favor simplicity. From 4b7e06e14f9968a26e105590fafddeebea04e109 Mon Sep 17 00:00:00 2001 From: Lily Brown Date: Mon, 14 Feb 2022 14:14:11 -0800 Subject: [PATCH 161/399] Emit more information for AstLocals in JSON encoder (#364) We don't emit type annotations right now for `AstLocal`s in the JSON encoder. This makes it really hard to surface annotations in the Agda implementation. This PR changes it to emit location and type annotations, if present. --- Analysis/src/JsonEncoder.cpp | 15 ++++++++- prototyping/Luau/Syntax/FromJSON.agda | 48 +++++++++++++++++---------- tests/JsonEncoder.test.cpp | 2 +- 3 files changed, 45 insertions(+), 20 deletions(-) diff --git a/Analysis/src/JsonEncoder.cpp b/Analysis/src/JsonEncoder.cpp index 8dd597e1..0d48f2ca 100644 --- a/Analysis/src/JsonEncoder.cpp +++ b/Analysis/src/JsonEncoder.cpp @@ -150,6 +150,10 @@ struct AstJsonEncoder : public AstVisitor { writeRaw(std::to_string(i)); } + void write(std::nullptr_t) + { + writeRaw("null"); + } void write(std::string_view str) { writeString(str); @@ -177,7 +181,16 @@ struct AstJsonEncoder : public AstVisitor void write(AstLocal* local) { - write(local->name); + writeRaw("{"); + bool c = pushComma(); + if (local->annotation != nullptr) + write("type", local->annotation); + else + write("type", nullptr); + write("name", local->name); + write("location", local->location); + popComma(c); + writeRaw("}"); } void writeNode(AstNode* node) diff --git a/prototyping/Luau/Syntax/FromJSON.agda b/prototyping/Luau/Syntax/FromJSON.agda index b081d156..82f95193 100644 --- a/prototyping/Luau/Syntax/FromJSON.agda +++ b/prototyping/Luau/Syntax/FromJSON.agda @@ -1,6 +1,6 @@ module Luau.Syntax.FromJSON where -open import Luau.Syntax using (Block; Stat ; Expr; nil; _$_; var; function_is_end; _⟨_⟩; local_←_; return; done; _∙_; maybe) +open import Luau.Syntax using (Block; Stat ; Expr; nil; _$_; var; function_is_end; _⟨_⟩; local_←_; return; done; _∙_; maybe; VarDec) open import Agda.Builtin.List using (List; _∷_; []) @@ -31,6 +31,8 @@ lookupIn (key ∷ keys) obj with lookup (fromString key) obj lookupIn (key ∷ keys) obj | nothing = lookupIn keys obj lookupIn (key ∷ keys) obj | just value = (key , value) +varDecFromJSON : Value → Either String (VarDec maybe) +varDecFromObject : Object → Either String (VarDec maybe) exprFromJSON : Value → Either String (Expr maybe) exprFromObject : Object → Either String (Expr maybe) statFromJSON : Value → Either String (Stat maybe) @@ -38,6 +40,14 @@ statFromObject : Object → Either String (Stat maybe) blockFromJSON : Value → Either String (Block maybe) blockFromArray : Array → Either String (Block maybe) +varDecFromJSON (object arg) = varDecFromObject arg +varDecFromJSON val = Left "VarDec not an object" + +varDecFromObject obj with lookup name obj +varDecFromObject obj | just (string name) = Right (var name) +varDecFromObject obj | just _ = Left "AstLocal name is not a string" +varDecFromObject obj | nothing = Left "AstLocal missing name" + exprFromJSON (object obj) = exprFromObject obj exprFromJSON val = Left "AstExpr not an object" @@ -54,17 +64,19 @@ exprFromObject obj | just (string "AstExprCall") | nothing | _ = Left ("AstExpr exprFromObject obj | just (string "AstExprCall") | _ | nothing = Left ("AstExprCall missing args") exprFromObject obj | just (string "AstExprConstantNil") = Right nil exprFromObject obj | just (string "AstExprFunction") with lookup args obj | lookup body obj -exprFromObject obj | just (string "AstExprFunction") | just (array arr) | just value with head arr | blockFromJSON value -exprFromObject obj | just (string "AstExprFunction") | just (array arr) | just value | just (string x) | Right B = Right (function "" ⟨ var x ⟩ is B end) -exprFromObject obj | just (string "AstExprFunction") | just (array arr) | just value | just _ | Right B = Left "AstExprFunction args not a string array" -exprFromObject obj | just (string "AstExprFunction") | just (array arr) | just value | nothing | Right B = Left "Unsupported AstExprFunction empty args" -exprFromObject obj | just (string "AstExprFunction") | just (array arr) | just value | _ | Left err = Left err +exprFromObject obj | just (string "AstExprFunction") | just (array arr) | just blockValue with head arr | blockFromJSON blockValue +exprFromObject obj | just (string "AstExprFunction") | just (array arr) | just blockValue | just argValue | Right B with varDecFromJSON argValue +exprFromObject obj | just (string "AstExprFunction") | just (array arr) | just blockValue | just argValue | Right B | Right arg = Right (function "" ⟨ arg ⟩ is B end) +exprFromObject obj | just (string "AstExprFunction") | just (array arr) | just blockValue | just argValue | Right B | Left err = Left err +exprFromObject obj | just (string "AstExprFunction") | just (array arr) | just blockValue | nothing | Right B = Left "Unsupported AstExprFunction empty args" +exprFromObject obj | just (string "AstExprFunction") | just (array arr) | just blockValue | _ | Left err = Left err exprFromObject obj | just (string "AstExprFunction") | just _ | just _ = Left "AstExprFunction args not an array" exprFromObject obj | just (string "AstExprFunction") | nothing | _ = Left "AstExprFunction missing args" exprFromObject obj | just (string "AstExprFunction") | _ | nothing = Left "AstExprFunction missing body" exprFromObject obj | just (string "AstExprLocal") with lookup lokal obj -exprFromObject obj | just (string "AstExprLocal") | just (string x) = Right (var x) -exprFromObject obj | just (string "AstExprLocal") | just (_) = Left "AstExprLocal local not a string" +exprFromObject obj | just (string "AstExprLocal") | just x with varDecFromJSON x +exprFromObject obj | just (string "AstExprLocal") | just x | Right x′ = Right (var (Luau.Syntax.name x′)) +exprFromObject obj | just (string "AstExprLocal") | just x | Left err = Left err exprFromObject obj | just (string "AstExprLocal") | nothing = Left "AstExprLocal missing local" exprFromObject obj | just (string ty) = Left ("TODO: Unsupported AstExpr " ++ ty) exprFromObject obj | just _ = Left "AstExpr type not a string" @@ -77,22 +89,22 @@ statFromJSON _ = Left "AstStat not an object" statFromObject obj with lookup type obj statFromObject obj | just(string "AstStatLocal") with lookup vars obj | lookup values obj statFromObject obj | just(string "AstStatLocal") | just(array arr1) | just(array arr2) with head(arr1) | head(arr2) -statFromObject obj | just(string "AstStatLocal") | just(array arr1) | just(array arr2) | just(string x) | just(value) with exprFromJSON(value) -statFromObject obj | just(string "AstStatLocal") | just(array arr1) | just(array arr2) | just(string x) | just(value) | Right M = Right (local (var x) ← M) -statFromObject obj | just(string "AstStatLocal") | just(array arr1) | just(array arr2) | just(string x) | just(value) | Left err = Left err -statFromObject obj | just(string "AstStatLocal") | just(array arr1) | just(array arr2) | just(string x) | nothing = Left "AstStatLocal empty values" -statFromObject obj | just(string "AstStatLocal") | just(array arr1) | just(array arr2) | just(_) | _ = Left "AstStatLocal vars not a string array" +statFromObject obj | just(string "AstStatLocal") | just(array arr1) | just(array arr2) | just(x) | just(value) with varDecFromJSON(x) | exprFromJSON(value) +statFromObject obj | just(string "AstStatLocal") | just(array arr1) | just(array arr2) | just(x) | just(value) | Right x′ | Right M = Right (local x′ ← M) +statFromObject obj | just(string "AstStatLocal") | just(array arr1) | just(array arr2) | just(x) | just(value) | Left err | _ = Left err +statFromObject obj | just(string "AstStatLocal") | just(array arr1) | just(array arr2) | just(x) | just(value) | _ | Left err = Left err +statFromObject obj | just(string "AstStatLocal") | just(array arr1) | just(array arr2) | just(x) | nothing = Left "AstStatLocal empty values" statFromObject obj | just(string "AstStatLocal") | just(array arr1) | just(array arr2) | nothing | _ = Left "AstStatLocal empty vars" statFromObject obj | just(string "AstStatLocal") | just(array arr1) | just(_) = Left "AstStatLocal values not an array" statFromObject obj | just(string "AstStatLocal") | just(_) | just(_) = Left "AstStatLocal vars not an array" statFromObject obj | just(string "AstStatLocal") | just(_) | nothing = Left "AstStatLocal missing values" statFromObject obj | just(string "AstStatLocal") | nothing | _ = Left "AstStatLocal missing vars" statFromObject obj | just(string "AstStatLocalFunction") with lookup name obj | lookup func obj -statFromObject obj | just(string "AstStatLocalFunction") | just (string f) | just value with exprFromJSON value -statFromObject obj | just(string "AstStatLocalFunction") | just (string f) | just value | Right (function "" ⟨ x ⟩ is B end) = Right (function f ⟨ x ⟩ is B end) -statFromObject obj | just(string "AstStatLocalFunction") | just (string f) | just value | Left err = Left err -statFromObject obj | just(string "AstStatLocalFunction") | just _ | just _ | Right _ = Left "AstStatLocalFunction func is not an AstExprFunction" -statFromObject obj | just(string "AstStatLocalFunction") | just _ | just _ = Left "AstStatLocalFunction name is not a string" +statFromObject obj | just(string "AstStatLocalFunction") | just fnName | just value with varDecFromJSON fnName | exprFromJSON value +statFromObject obj | just(string "AstStatLocalFunction") | just fnName | just value | Right fnVar | Right (function "" ⟨ x ⟩ is B end) = Right (function (Luau.Syntax.name fnVar) ⟨ x ⟩ is B end) +statFromObject obj | just(string "AstStatLocalFunction") | just fnName | just value | Left err | _ = Left err +statFromObject obj | just(string "AstStatLocalFunction") | just fnName | just value | _ | Left err = Left err +statFromObject obj | just(string "AstStatLocalFunction") | just _ | just _ | Right _ | Right _ = Left "AstStatLocalFunction func is not an AstExprFunction" statFromObject obj | just(string "AstStatLocalFunction") | nothing | _ = Left "AstStatFunction missing name" statFromObject obj | just(string "AstStatLocalFunction") | _ | nothing = Left "AstStatFunction missing func" statFromObject obj | just(string "AstStatReturn") with lookup list obj diff --git a/tests/JsonEncoder.test.cpp b/tests/JsonEncoder.test.cpp index 4a717275..cb508072 100644 --- a/tests/JsonEncoder.test.cpp +++ b/tests/JsonEncoder.test.cpp @@ -46,7 +46,7 @@ TEST_CASE("encode_AstStatBlock") AstStatBlock block{Location(), bodyArray}; CHECK_EQ( - (R"({"type":"AstStatBlock","location":"0,0 - 0,0","body":[{"type":"AstStatLocal","location":"0,0 - 0,0","vars":["a_local"],"values":[]}]})"), + (R"({"type":"AstStatBlock","location":"0,0 - 0,0","body":[{"type":"AstStatLocal","location":"0,0 - 0,0","vars":[{"type":null,"name":"a_local","location":"0,0 - 0,0"}],"values":[]}]})"), toJson(&block)); } From f0c9d84461339fa8bcd2d8bf7d6f5be5be029f46 Mon Sep 17 00:00:00 2001 From: Lily Brown Date: Tue, 15 Feb 2022 14:10:43 -0800 Subject: [PATCH 162/399] Prototyping: Parse type annotations (#366) Parses type annotations from the JSON output of `luau-ast`. --- prototyping/Examples/SmokeTest.lua | 5 ++ prototyping/Examples/SmokeTestOutput.lua | 8 ++- prototyping/Luau/Syntax/FromJSON.agda | 15 ++++-- prototyping/Luau/Type/FromJSON.agda | 66 ++++++++++++++++++++++++ 4 files changed, 88 insertions(+), 6 deletions(-) create mode 100644 prototyping/Luau/Type/FromJSON.agda diff --git a/prototyping/Examples/SmokeTest.lua b/prototyping/Examples/SmokeTest.lua index e1e37e32..e663534a 100644 --- a/prototyping/Examples/SmokeTest.lua +++ b/prototyping/Examples/SmokeTest.lua @@ -10,4 +10,9 @@ local function comp(f) end local id2 = comp(id)(id) local nil2 = id2(nil) +local a : any = nil +local b : nil = nil +local c : (nil) -> nil = nil +local d : (any & nil) = nil +local e : any? = nil return id2(nil2) diff --git a/prototyping/Examples/SmokeTestOutput.lua b/prototyping/Examples/SmokeTestOutput.lua index b1b91e19..e663534a 100644 --- a/prototyping/Examples/SmokeTestOutput.lua +++ b/prototyping/Examples/SmokeTestOutput.lua @@ -8,5 +8,11 @@ local function comp(f) end end end -local id2 = id(id) +local id2 = comp(id)(id) local nil2 = id2(nil) +local a : any = nil +local b : nil = nil +local c : (nil) -> nil = nil +local d : (any & nil) = nil +local e : any? = nil +return id2(nil2) diff --git a/prototyping/Luau/Syntax/FromJSON.agda b/prototyping/Luau/Syntax/FromJSON.agda index 82f95193..8ae620bb 100644 --- a/prototyping/Luau/Syntax/FromJSON.agda +++ b/prototyping/Luau/Syntax/FromJSON.agda @@ -1,6 +1,7 @@ module Luau.Syntax.FromJSON where -open import Luau.Syntax using (Block; Stat ; Expr; nil; _$_; var; function_is_end; _⟨_⟩; local_←_; return; done; _∙_; maybe; VarDec) +open import Luau.Syntax using (Block; Stat ; Expr; nil; _$_; var; var_∈_; function_is_end; _⟨_⟩; local_←_; return; done; _∙_; maybe; VarDec) +open import Luau.Type.FromJSON using (typeFromJSON) open import Agda.Builtin.List using (List; _∷_; []) @@ -43,10 +44,14 @@ blockFromArray : Array → Either String (Block maybe) varDecFromJSON (object arg) = varDecFromObject arg varDecFromJSON val = Left "VarDec not an object" -varDecFromObject obj with lookup name obj -varDecFromObject obj | just (string name) = Right (var name) -varDecFromObject obj | just _ = Left "AstLocal name is not a string" -varDecFromObject obj | nothing = Left "AstLocal missing name" +varDecFromObject obj with lookup name obj | lookup type obj +varDecFromObject obj | just (string name) | nothing = Right (var name) +varDecFromObject obj | just (string name) | just Value.null = Right (var name) +varDecFromObject obj | just (string name) | just tyValue with typeFromJSON tyValue +varDecFromObject obj | just (string name) | just tyValue | Right ty = Right (var name ∈ ty) +varDecFromObject obj | just (string name) | just tyValue | Left err = Left err +varDecFromObject obj | just _ | _ = Left "AstLocal name is not a string" +varDecFromObject obj | nothing | _ = Left "AstLocal missing name" exprFromJSON (object obj) = exprFromObject obj exprFromJSON val = Left "AstExpr not an object" diff --git a/prototyping/Luau/Type/FromJSON.agda b/prototyping/Luau/Type/FromJSON.agda new file mode 100644 index 00000000..45bda5f3 --- /dev/null +++ b/prototyping/Luau/Type/FromJSON.agda @@ -0,0 +1,66 @@ +module Luau.Type.FromJSON where + +open import Luau.Type using (Type; nil; _⇒_; _∪_; _∩_; any) + +open import Agda.Builtin.List using (List; _∷_; []) + +open import FFI.Data.Aeson using (Value; Array; Object; object; array; string; fromString; lookup) +open import FFI.Data.Bool using (true; false) +open import FFI.Data.Either using (Either; Left; Right) +open import FFI.Data.Maybe using (Maybe; nothing; just) +open import FFI.Data.String using (String; _++_) +open import FFI.Data.Vector using (head; tail; null; empty) + +name = fromString "name" +type = fromString "type" +argTypes = fromString "argTypes" +returnTypes = fromString "returnTypes" +types = fromString "types" + +{-# TERMINATING #-} +typeFromJSON : Value → Either String Type +compoundFromArray : (Type → Type → Type) → Array → Either String Type + +typeFromJSON (object o) with lookup type o +typeFromJSON (object o) | just (string "AstTypeFunction") with lookup argTypes o | lookup returnTypes o +typeFromJSON (object o) | just (string "AstTypeFunction") | just (object argsSet) | just (object retsSet) with lookup types argsSet | lookup types retsSet +typeFromJSON (object o) | just (string "AstTypeFunction") | just (object argsSet) | just (object retsSet) | just (array args) | just (array rets) with head args | head rets +typeFromJSON (object o) | just (string "AstTypeFunction") | just (object argsSet) | just (object retsSet) | just (array args) | just (array rets) | just argValue | just retValue with typeFromJSON argValue | typeFromJSON retValue +typeFromJSON (object o) | just (string "AstTypeFunction") | just (object argsSet) | just (object retsSet) | just (array args) | just (array rets) | just argValue | just retValue | Right arg | Right ret = Right (arg ⇒ ret) +typeFromJSON (object o) | just (string "AstTypeFunction") | just (object argsSet) | just (object retsSet) | just (array args) | just (array rets) | just argValue | just retValue | Left err | _ = Left err +typeFromJSON (object o) | just (string "AstTypeFunction") | just (object argsSet) | just (object retsSet) | just (array args) | just (array rets) | just argValue | just retValue | _ | Left err = Left err +typeFromJSON (object o) | just (string "AstTypeFunction") | just (object argsSet) | just (object retsSet) | just (array args) | just (array rets) | _ | nothing = Left "No return type" +typeFromJSON (object o) | just (string "AstTypeFunction") | just (object argsSet) | just (object retsSet) | just (array args) | just (array rets) | nothing | _ = Left "No argument type" +typeFromJSON (object o) | just (string "AstTypeFunction") | just (object argsSet) | just (object retsSet) | just _ | _ = Left "argTypes.types is not an array" +typeFromJSON (object o) | just (string "AstTypeFunction") | just (object argsSet) | just (object retsSet) | _ | just _ = Left "retTypes.types is not an array" +typeFromJSON (object o) | just (string "AstTypeFunction") | just (object argsSet) | just (object retsSet) | nothing | _ = Left "argTypes.types does not exist" +typeFromJSON (object o) | just (string "AstTypeFunction") | _ | just _ = Left "argTypes is not an object" +typeFromJSON (object o) | just (string "AstTypeFunction") | just _ | _ = Left "returnTypes is not an object" +typeFromJSON (object o) | just (string "AstTypeFunction") | nothing | nothing = Left "Missing argTypes and returnTypes" + +typeFromJSON (object o) | just (string "AstTypeReference") with lookup name o +typeFromJSON (object o) | just (string "AstTypeReference") | just (string "nil") = Right nil +typeFromJSON (object o) | just (string "AstTypeReference") | just (string "any") = Right any +typeFromJSON (object o) | just (string "AstTypeReference") | _ = Left "Unknown referenced type" + +typeFromJSON (object o) | just (string "AstTypeUnion") with lookup types o +typeFromJSON (object o) | just (string "AstTypeUnion") | just (array types) = compoundFromArray _∪_ types +typeFromJSON (object o) | just (string "AstTypeUnion") | _ = Left "`types` field must be an array" + +typeFromJSON (object o) | just (string "AstTypeIntersection") with lookup types o +typeFromJSON (object o) | just (string "AstTypeIntersection") | just (array types) = compoundFromArray _∩_ types +typeFromJSON (object o) | just (string "AstTypeIntersection") | _ = Left "`types` field must be an array" + +typeFromJSON (object o) | just (string ty) = Left ("Unsupported type " ++ ty) +typeFromJSON (object o) | just _ = Left "`type` field must be a string" +typeFromJSON (object o) | nothing = Left "No `type` field" +typeFromJSON _ = Left "Unsupported JSON type" + +compoundFromArray ctor ts with head ts | tail ts +compoundFromArray ctor ts | just hd | tl with null tl +compoundFromArray ctor ts | just hd | tl | true = typeFromJSON hd +compoundFromArray ctor ts | just hd | tl | false with typeFromJSON hd | compoundFromArray ctor tl +compoundFromArray ctor ts | just hd | tl | false | Right hdTy | Right tlTy = Right (ctor hdTy tlTy) +compoundFromArray ctor ts | just hd | tl | false | Left err | _ = Left err +compoundFromArray ctor ts | just hd | tl | false | _ | Left Err = Left Err +compoundFromArray ctor ts | nothing | empty = Left "Empty types array?" From c8d6dc2758ef6369010eb7e23e368d478790aa94 Mon Sep 17 00:00:00 2001 From: Lily Brown Date: Tue, 15 Feb 2022 14:24:51 -0800 Subject: [PATCH 163/399] Revise GHA workflows for prototyping (#367) Changed the GHA workflows to: - Not run `build` and `release` workflows for PRs that only affect `prototyping/` - Run `prototyping` workflow when PRs affect `Analysis/**`, `Ast/**`, or the `luau-ast` source files --- .github/workflows/build.yml | 2 ++ .github/workflows/prototyping.yml | 10 ++++++++-- .github/workflows/release.yml | 1 + 3 files changed, 11 insertions(+), 2 deletions(-) diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml index b797e91b..93b92645 100644 --- a/.github/workflows/build.yml +++ b/.github/workflows/build.yml @@ -9,12 +9,14 @@ on: - 'papers/**' - 'rfcs/**' - '*.md' + - 'prototyping/**' pull_request: paths-ignore: - 'docs/**' - 'papers/**' - 'rfcs/**' - '*.md' + - 'prototyping/**' jobs: unix: diff --git a/.github/workflows/prototyping.yml b/.github/workflows/prototyping.yml index 76c1e3c1..4a1b8e08 100644 --- a/.github/workflows/prototyping.yml +++ b/.github/workflows/prototyping.yml @@ -8,12 +8,18 @@ on: paths: - '.github/workflows/**' - 'prototyping/**' - - 'Analysis/src/JsonEncoder.cpp' + - 'Analysis/**' + - 'Ast/**' + - 'CLI/Ast.cpp' + - 'CLI/FileUtils.*' pull_request: paths: - '.github/workflows/**' - 'prototyping/**' - - 'Analysis/src/JsonEncoder.cpp' + - 'Analysis/**' + - 'Ast/**' + - 'CLI/Ast.cpp' + - 'CLI/FileUtils.*' jobs: linux: diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index e5cd952b..2bf796a2 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -9,6 +9,7 @@ on: - 'papers/**' - 'rfcs/**' - '*.md' + - 'prototyping/**' jobs: build: From e541e19f440b4eaefceb122c65b66103460973f9 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Tue, 15 Feb 2022 18:37:02 -0800 Subject: [PATCH 164/399] Create RFC status tracking document (#363) This tracks status of all unimplemented RFCs in one central place. Hopefully we won't forget to update this when new RFCs are added! --- rfcs/STATUS.md | 64 ++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 64 insertions(+) create mode 100644 rfcs/STATUS.md diff --git a/rfcs/STATUS.md b/rfcs/STATUS.md new file mode 100644 index 00000000..9a45afe0 --- /dev/null +++ b/rfcs/STATUS.md @@ -0,0 +1,64 @@ +This document tracks unimplemented RFCs. + +## Deprecate getfenv/setfenv + +[RFC: Deprecate getfenv/setfenv](https://github.com/Roblox/luau/blob/master/rfcs/deprecate-getfenv-setfenv.md) + +**Status**: Needs implementation. + +**Notes**: Implementing this RFC triggers warnings across the board in the apps ecosystem, in particular in testing libraries. Pending code changes / decisions. + +## Read-only and write-only properties + +[RFC: Read-only properties](https://github.com/Roblox/luau/blob/master/rfcs/property-readonly.md) | +[RFC: Write-only properties](https://github.com/Roblox/luau/blob/master/rfcs/property-writeonly.md) + +**Status**: Needs implementation + +## Sealed/unsealed typing changes + +[RFC: Sealed table subtyping](https://github.com/Roblox/luau/blob/master/rfcs/sealed-table-subtyping.md) | +[RFC: Unsealed table literals](https://github.com/Roblox/luau/blob/master/rfcs/unsealed-table-literals.md) | +[RFC: Only strip optional properties from unsealed tables during subtyping](https://github.com/Roblox/luau/blob/master/rfcs/unsealed-table-subtyping-strips-optional-properties.md) + +**Status**: Implemented but depends on new transaction log implementation that is not fully live yet. + +## Default type parameters + +[RFC: Default type alias type parameters](https://github.com/Roblox/luau/blob/master/rfcs/syntax-default-type-alias-type-parameters.md) + +**Status**: Implemented but not fully rolled out yet. + +## Singleton types + +[RFC: Singleton types](https://github.com/Roblox/luau/blob/master/rfcs/syntax-singleton-types.md) + +**Status**: Implemented but not fully rolled out yet. + +## Nil-forgiving operator + +[RFC: nil-forgiving postfix operator !](https://github.com/Roblox/luau/blob/master/rfcs/syntax-nil-forgiving-operator.md) + +**Status**: Needs implementation. + +**Notes**: Do we need to reevaluate the necessity given `assert` and improved refinements? + +## Safe navigation operator + +[RFC: Safe navigation postfix operator (?)](https://github.com/Roblox/luau/blob/master/rfcs/syntax-safe-navigation-operator.md) + +**Status**: Needs implementation. + +**Notes**: We have unresolved issues with interaction between this feature and Roblox instance hierarchy. This may affect the viability of this proposal. + +## String interpolation + +[RFC: String interpolation](https://github.com/Roblox/luau/blob/master/rfcs/syntax-string-interpolation.md) + +**Status**: Needs implementation + +## Generalized iteration + +[RFC: Generalized iteration](https://github.com/Roblox/luau/blob/master/rfcs/generalized-iteration.md) + +**Status**: Needs implementation From e49a0fd4cd785462ca5b8b2b4a29d0684f50e2ac Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Thu, 17 Feb 2022 16:14:35 -0800 Subject: [PATCH 165/399] Mark default type parameters RFC as implemented (#369) --- rfcs/STATUS.md | 6 ------ rfcs/syntax-default-type-alias-type-parameters.md | 2 ++ 2 files changed, 2 insertions(+), 6 deletions(-) diff --git a/rfcs/STATUS.md b/rfcs/STATUS.md index 9a45afe0..72db2003 100644 --- a/rfcs/STATUS.md +++ b/rfcs/STATUS.md @@ -23,12 +23,6 @@ This document tracks unimplemented RFCs. **Status**: Implemented but depends on new transaction log implementation that is not fully live yet. -## Default type parameters - -[RFC: Default type alias type parameters](https://github.com/Roblox/luau/blob/master/rfcs/syntax-default-type-alias-type-parameters.md) - -**Status**: Implemented but not fully rolled out yet. - ## Singleton types [RFC: Singleton types](https://github.com/Roblox/luau/blob/master/rfcs/syntax-singleton-types.md) diff --git a/rfcs/syntax-default-type-alias-type-parameters.md b/rfcs/syntax-default-type-alias-type-parameters.md index 15d23c81..443bbac3 100644 --- a/rfcs/syntax-default-type-alias-type-parameters.md +++ b/rfcs/syntax-default-type-alias-type-parameters.md @@ -1,5 +1,7 @@ # Default type alias type parameters +**Status**: Implemented + ## Summary Introduce syntax to provide default type values inside the type alias type parameter list. From 731e197757ead8908e558cfc63f9d518f53940d5 Mon Sep 17 00:00:00 2001 From: Alexander McCord <11488393+alexmccord@users.noreply.github.com> Date: Thu, 17 Feb 2022 16:16:31 -0800 Subject: [PATCH 166/399] Mark singleton types RFC as implemented (#370) --- rfcs/STATUS.md | 6 ------ rfcs/syntax-singleton-types.md | 2 ++ 2 files changed, 2 insertions(+), 6 deletions(-) diff --git a/rfcs/STATUS.md b/rfcs/STATUS.md index 72db2003..742cb501 100644 --- a/rfcs/STATUS.md +++ b/rfcs/STATUS.md @@ -23,12 +23,6 @@ This document tracks unimplemented RFCs. **Status**: Implemented but depends on new transaction log implementation that is not fully live yet. -## Singleton types - -[RFC: Singleton types](https://github.com/Roblox/luau/blob/master/rfcs/syntax-singleton-types.md) - -**Status**: Implemented but not fully rolled out yet. - ## Nil-forgiving operator [RFC: nil-forgiving postfix operator !](https://github.com/Roblox/luau/blob/master/rfcs/syntax-nil-forgiving-operator.md) diff --git a/rfcs/syntax-singleton-types.md b/rfcs/syntax-singleton-types.md index 26ea3028..2c1f5442 100644 --- a/rfcs/syntax-singleton-types.md +++ b/rfcs/syntax-singleton-types.md @@ -2,6 +2,8 @@ > Note: this RFC was adapted from an internal proposal that predates RFC process +**Status**: Implemented + ## Summary Introduce a new kind of type variable, called singleton types. They are just like normal types but has the capability to represent a constant runtime value as a type. From 49304095161c4532a236b8141faf1e0c7dda498f Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Thu, 17 Feb 2022 16:41:20 -0800 Subject: [PATCH 167/399] Sync to upstream/release/515 --- Analysis/include/Luau/Documentation.h | 3 + Analysis/include/Luau/Frontend.h | 3 +- Analysis/include/Luau/Linter.h | 7 +- Analysis/include/Luau/Module.h | 4 +- Analysis/include/Luau/Quantify.h | 5 +- Analysis/include/Luau/Substitution.h | 20 +- Analysis/include/Luau/TypeInfer.h | 45 +++- Analysis/src/Autocomplete.cpp | 45 ++-- Analysis/src/Config.cpp | 2 +- Analysis/src/EmbeddedBuiltinDefinitions.cpp | 2 +- Analysis/src/Frontend.cpp | 35 ++- Analysis/src/JsonEncoder.cpp | 36 ++- Analysis/src/Linter.cpp | 126 ++++++++- Analysis/src/Quantify.cpp | 10 +- Analysis/src/Substitution.cpp | 23 -- Analysis/src/TopoSortStatements.cpp | 3 +- Analysis/src/Transpiler.cpp | 8 +- Analysis/src/TypeAttach.cpp | 4 +- Analysis/src/TypeInfer.cpp | 142 +++-------- Analysis/src/TypeUtils.cpp | 12 + Analysis/src/TypeVar.cpp | 6 +- Analysis/src/TypedAllocator.cpp | 3 + Analysis/src/Unifier.cpp | 16 +- Ast/include/Luau/Ast.h | 17 +- Ast/include/Luau/Lexer.h | 5 + Ast/include/Luau/ParseResult.h | 69 +++++ Ast/include/Luau/Parser.h | 56 +--- Ast/src/Ast.cpp | 29 +-- Ast/src/Lexer.cpp | 12 +- Ast/src/Parser.cpp | 167 +++++++----- Ast/src/TimeTrace.cpp | 6 + CLI/FileUtils.cpp | 7 +- CLI/Repl.cpp | 171 +++++++++---- CMakeLists.txt | 14 + Sources.cmake | 1 + VM/src/lapi.cpp | 25 +- VM/src/ldebug.cpp | 2 +- VM/src/ldo.cpp | 13 +- VM/src/ldo.h | 2 +- VM/src/lgc.cpp | 4 +- VM/src/lgc.h | 2 +- VM/src/lgcdebug.cpp | 10 +- VM/src/lperf.cpp | 6 + VM/src/lstate.cpp | 35 ++- VM/src/lstate.h | 8 +- VM/src/lvmexecute.cpp | 2 +- VM/src/lvmload.cpp | 6 +- VM/src/lvmutils.cpp | 4 +- fuzz/linter.cpp | 2 +- fuzz/proto.cpp | 2 +- tests/Autocomplete.test.cpp | 22 +- tests/Compiler.test.cpp | 14 +- tests/Conformance.test.cpp | 8 + tests/Fixture.cpp | 4 +- tests/Fixture.h | 1 - tests/Frontend.test.cpp | 3 - tests/JsonEncoder.test.cpp | 2 +- tests/Linter.test.cpp | 48 +++- tests/NonstrictMode.test.cpp | 1 - tests/Parser.test.cpp | 51 ++-- tests/Repl.test.cpp | 209 ++++++++++++++- tests/RequireTracer.test.cpp | 2 +- tests/TypeInfer.aliases.test.cpp | 6 +- tests/TypeInfer.annotations.test.cpp | 1 - tests/TypeInfer.builtins.test.cpp | 1 - tests/TypeInfer.classes.test.cpp | 1 - tests/TypeInfer.definitions.test.cpp | 1 - tests/TypeInfer.generics.test.cpp | 1 - tests/TypeInfer.intersectionTypes.test.cpp | 1 - tests/TypeInfer.provisional.test.cpp | 1 - tests/TypeInfer.singletons.test.cpp | 2 - tests/TypeInfer.tables.test.cpp | 43 +++- tests/TypeInfer.test.cpp | 27 +- tests/TypeInfer.tryUnify.test.cpp | 1 - tests/TypeInfer.typePacks.cpp | 3 - tests/TypeInfer.unionTypes.test.cpp | 1 - tests/TypePack.test.cpp | 1 - tests/TypeVar.test.cpp | 1 - tests/conformance/coroutine.lua | 9 + tests/conformance/coverage.lua | 8 + tests/conformance/debug.lua | 3 + tests/main.cpp | 7 +- tools/natvis/Analysis.natvis | 78 ++++++ tools/natvis/Ast.natvis | 25 ++ tools/natvis/VM.natvis | 269 ++++++++++++++++++++ 85 files changed, 1512 insertions(+), 581 deletions(-) create mode 100644 Ast/include/Luau/ParseResult.h create mode 100644 tools/natvis/Analysis.natvis create mode 100644 tools/natvis/Ast.natvis create mode 100644 tools/natvis/VM.natvis diff --git a/Analysis/include/Luau/Documentation.h b/Analysis/include/Luau/Documentation.h index 68ff3a7c..7a2b56ff 100644 --- a/Analysis/include/Luau/Documentation.h +++ b/Analysis/include/Luau/Documentation.h @@ -21,6 +21,7 @@ struct BasicDocumentation { std::string documentation; std::string learnMoreLink; + std::string codeSample; }; struct FunctionParameterDocumentation @@ -37,6 +38,7 @@ struct FunctionDocumentation std::vector parameters; std::vector returns; std::string learnMoreLink; + std::string codeSample; }; struct OverloadedFunctionDocumentation @@ -52,6 +54,7 @@ struct TableDocumentation std::string documentation; Luau::DenseHashMap keys; std::string learnMoreLink; + std::string codeSample; }; using DocumentationDatabase = Luau::DenseHashMap; diff --git a/Analysis/include/Luau/Frontend.h b/Analysis/include/Luau/Frontend.h index 1f64db30..0bf8f362 100644 --- a/Analysis/include/Luau/Frontend.h +++ b/Analysis/include/Luau/Frontend.h @@ -24,6 +24,7 @@ struct TypeChecker; struct FileResolver; struct ModuleResolver; struct ParseResult; +struct HotComment; struct LoadDefinitionFileResult { @@ -35,7 +36,7 @@ struct LoadDefinitionFileResult LoadDefinitionFileResult loadDefinitionFile( TypeChecker& typeChecker, ScopePtr targetScope, std::string_view definition, const std::string& packageName); -std::optional parseMode(const std::vector& hotcomments); +std::optional parseMode(const std::vector& hotcomments); std::vector parsePathExpr(const AstExpr& pathExpr); diff --git a/Analysis/include/Luau/Linter.h b/Analysis/include/Luau/Linter.h index ec3c124d..6c7ce47f 100644 --- a/Analysis/include/Luau/Linter.h +++ b/Analysis/include/Luau/Linter.h @@ -14,6 +14,7 @@ class AstStat; class AstNameTable; struct TypeChecker; struct Module; +struct HotComment; using ScopePtr = std::shared_ptr; @@ -50,6 +51,7 @@ struct LintWarning Code_TableOperations = 23, Code_DuplicateCondition = 24, Code_MisleadingAndOr = 25, + Code_CommentDirective = 26, Code__Count }; @@ -60,7 +62,7 @@ struct LintWarning static const char* getName(Code code); static Code parseName(const char* name); - static uint64_t parseMask(const std::vector& hotcomments); + static uint64_t parseMask(const std::vector& hotcomments); }; struct LintResult @@ -90,7 +92,8 @@ struct LintOptions void setDefaults(); }; -std::vector lint(AstStat* root, const AstNameTable& names, const ScopePtr& env, const Module* module, const LintOptions& options); +std::vector lint(AstStat* root, const AstNameTable& names, const ScopePtr& env, const Module* module, + const std::vector& hotcomments, const LintOptions& options); std::vector getDeprecatedGlobals(const AstNameTable& names); diff --git a/Analysis/include/Luau/Module.h b/Analysis/include/Luau/Module.h index 1bf0473c..61200771 100644 --- a/Analysis/include/Luau/Module.h +++ b/Analysis/include/Luau/Module.h @@ -6,7 +6,7 @@ #include "Luau/TypedAllocator.h" #include "Luau/ParseOptions.h" #include "Luau/Error.h" -#include "Luau/Parser.h" +#include "Luau/ParseResult.h" #include #include @@ -37,8 +37,8 @@ struct SourceModule AstStatBlock* root = nullptr; std::optional mode; - uint64_t ignoreLints = 0; + std::vector hotcomments; std::vector commentLocations; SourceModule() diff --git a/Analysis/include/Luau/Quantify.h b/Analysis/include/Luau/Quantify.h index f46df146..e48cad40 100644 --- a/Analysis/include/Luau/Quantify.h +++ b/Analysis/include/Luau/Quantify.h @@ -6,9 +6,6 @@ namespace Luau { -struct Module; -using ModulePtr = std::shared_ptr; - -void quantify(ModulePtr module, TypeId ty, TypeLevel level); +void quantify(TypeId ty, TypeLevel level); } // namespace Luau diff --git a/Analysis/include/Luau/Substitution.h b/Analysis/include/Luau/Substitution.h index f85b4269..9662d5b3 100644 --- a/Analysis/include/Luau/Substitution.h +++ b/Analysis/include/Luau/Substitution.h @@ -101,9 +101,6 @@ struct Tarjan // This is hot code, so we optimize recursion to a stack. TarjanResult loop(); - // Clear the state - void clear(); - // Find or create the index for a vertex. // Return a boolean which is `true` if it's a freshly created index. std::pair indexify(TypeId ty); @@ -166,7 +163,17 @@ struct FindDirty : Tarjan // and replaces them with clean ones. struct Substitution : FindDirty { - ModulePtr currentModule; +protected: + Substitution(const TxnLog* log_, TypeArena* arena) + : arena(arena) + { + log = log_; + LUAU_ASSERT(log); + LUAU_ASSERT(arena); + } + +public: + TypeArena* arena; DenseHashMap newTypes{nullptr}; DenseHashMap newPacks{nullptr}; @@ -192,12 +199,13 @@ struct Substitution : FindDirty template TypeId addType(const T& tv) { - return currentModule->internalTypes.addType(tv); + return arena->addType(tv); } + template TypePackId addTypePack(const T& tp) { - return currentModule->internalTypes.addTypePack(TypePackVar{tp}); + return arena->addTypePack(TypePackVar{tp}); } }; diff --git a/Analysis/include/Luau/TypeInfer.h b/Analysis/include/Luau/TypeInfer.h index 5592fa1f..3c5ded3c 100644 --- a/Analysis/include/Luau/TypeInfer.h +++ b/Analysis/include/Luau/TypeInfer.h @@ -5,7 +5,6 @@ #include "Luau/Error.h" #include "Luau/Module.h" #include "Luau/Symbol.h" -#include "Luau/Parser.h" #include "Luau/Substitution.h" #include "Luau/TxnLog.h" #include "Luau/TypePack.h" @@ -37,6 +36,15 @@ struct Unifier; // A substitution which replaces generic types in a given set by free types. struct ReplaceGenerics : Substitution { + ReplaceGenerics( + const TxnLog* log, TypeArena* arena, TypeLevel level, const std::vector& generics, const std::vector& genericPacks) + : Substitution(log, arena) + , level(level) + , generics(generics) + , genericPacks(genericPacks) + { + } + TypeLevel level; std::vector generics; std::vector genericPacks; @@ -50,8 +58,13 @@ struct ReplaceGenerics : Substitution // A substitution which replaces generic functions by monomorphic functions struct Instantiation : Substitution { + Instantiation(const TxnLog* log, TypeArena* arena, TypeLevel level) + : Substitution(log, arena) + , level(level) + { + } + TypeLevel level; - ReplaceGenerics replaceGenerics; bool ignoreChildren(TypeId ty) override; bool isDirty(TypeId ty) override; bool isDirty(TypePackId tp) override; @@ -62,6 +75,12 @@ struct Instantiation : Substitution // A substitution which replaces free types by generic types. struct Quantification : Substitution { + Quantification(TypeArena* arena, TypeLevel level) + : Substitution(TxnLog::empty(), arena) + , level(level) + { + } + TypeLevel level; std::vector generics; std::vector genericPacks; @@ -74,6 +93,13 @@ struct Quantification : Substitution // A substitution which replaces free types by any struct Anyification : Substitution { + Anyification(TypeArena* arena, TypeId anyType, TypePackId anyTypePack) + : Substitution(TxnLog::empty(), arena) + , anyType(anyType) + , anyTypePack(anyTypePack) + { + } + TypeId anyType; TypePackId anyTypePack; bool isDirty(TypeId ty) override; @@ -85,6 +111,13 @@ struct Anyification : Substitution // A substitution which replaces the type parameters of a type function by arguments struct ApplyTypeFunction : Substitution { + ApplyTypeFunction(TypeArena* arena, TypeLevel level) + : Substitution(TxnLog::empty(), arena) + , level(level) + , encounteredForwardedType(false) + { + } + TypeLevel level; bool encounteredForwardedType; std::unordered_map typeArguments; @@ -351,8 +384,7 @@ private: // Note: `scope` must be a fresh scope. GenericTypeDefinitions createGenericTypes(const ScopePtr& scope, std::optional levelOpt, const AstNode& node, - const AstArray& genericNames, const AstArray& genericPackNames, - bool useCache = false); + const AstArray& genericNames, const AstArray& genericPackNames, bool useCache = false); public: ErrorVec resolve(const PredicateVec& predicates, const ScopePtr& scope, bool sense); @@ -392,11 +424,6 @@ public: ModulePtr currentModule; ModuleName currentModuleName; - Instantiation instantiation; - Quantification quantification; - Anyification anyification; - ApplyTypeFunction applyTypeFunction; - std::function prepareModuleScope; InternalErrorReporter* iceHandler; diff --git a/Analysis/src/Autocomplete.cpp b/Analysis/src/Autocomplete.cpp index 85099e12..5a1ae397 100644 --- a/Analysis/src/Autocomplete.cpp +++ b/Analysis/src/Autocomplete.cpp @@ -7,6 +7,7 @@ #include "Luau/ToString.h" #include "Luau/TypeInfer.h" #include "Luau/TypePack.h" +#include "Luau/Parser.h" // TODO: only needed for autocompleteSource which is deprecated #include #include @@ -14,9 +15,9 @@ LUAU_FASTFLAG(LuauUseCommittingTxnLog) LUAU_FASTFLAGVARIABLE(LuauAutocompleteAvoidMutation, false); -LUAU_FASTFLAGVARIABLE(LuauCompleteBrokenStringParams, false); LUAU_FASTFLAGVARIABLE(LuauMissingFollowACMetatables, false); LUAU_FASTFLAGVARIABLE(PreferToCallFunctionsForIntersects, false); +LUAU_FASTFLAGVARIABLE(LuauIfElseExprFixCompletionIssue, false); static const std::unordered_set kStatementStartingKeywords = { "while", "if", "local", "repeat", "function", "do", "for", "return", "break", "continue", "type", "export"}; @@ -380,7 +381,7 @@ static void autocompleteProps(const Module& module, TypeArena* typeArena, TypeId { // We are walking up the class hierarchy, so if we encounter a property that we have // already populated, it takes precedence over the property we found just now. - if (result.count(name) == 0 && name != Parser::errorName) + if (result.count(name) == 0 && name != kParseNameError) { Luau::TypeId type = Luau::follow(prop.type); TypeCorrectKind typeCorrect = indexType == PropIndexType::Key ? TypeCorrectKind::Correct @@ -948,9 +949,12 @@ AutocompleteEntryMap autocompleteTypeNames(const Module& module, Position positi } } - for (size_t i = 0; i < node->returnAnnotation.types.size; i++) + if (!node->returnAnnotation) + return result; + + for (size_t i = 0; i < node->returnAnnotation->types.size; i++) { - AstType* ret = node->returnAnnotation.types.data[i]; + AstType* ret = node->returnAnnotation->types.data[i]; if (ret->location.containsClosed(position)) { @@ -965,7 +969,7 @@ AutocompleteEntryMap autocompleteTypeNames(const Module& module, Position positi } } - if (AstTypePack* retTp = node->returnAnnotation.tailType) + if (AstTypePack* retTp = node->returnAnnotation->tailType) { if (auto variadic = retTp->as()) { @@ -1136,7 +1140,7 @@ static AutocompleteEntryMap autocompleteStatement( AstNode* parent = ancestry.rbegin()[1]; if (AstStatIf* statIf = parent->as()) { - if (!statIf->elsebody || (statIf->hasElse && statIf->elseLocation.containsClosed(position))) + if (!statIf->elsebody || (statIf->elseLocation && statIf->elseLocation->containsClosed(position))) { result.emplace("else", AutocompleteEntry{AutocompleteEntryKind::Keyword}); result.emplace("elseif", AutocompleteEntry{AutocompleteEntryKind::Keyword}); @@ -1164,8 +1168,7 @@ static AutocompleteEntryMap autocompleteStatement( return result; } -// Returns true if completions were generated (completions will be inserted into 'outResult') -// Returns false if no completions were generated +// Returns true iff `node` was handled by this function (completions, if any, are returned in `outResult`) static bool autocompleteIfElseExpression( const AstNode* node, const std::vector& ancestry, const Position& position, AutocompleteEntryMap& outResult) { @@ -1173,6 +1176,13 @@ static bool autocompleteIfElseExpression( if (!parent) return false; + if (FFlag::LuauIfElseExprFixCompletionIssue && node->is()) + { + // Don't try to complete when the current node is an if-else expression (i.e. only try to complete when the node is a child of an if-else + // expression. + return true; + } + AstExprIfElse* ifElseExpr = parent->as(); if (!ifElseExpr || ifElseExpr->condition->location.containsClosed(position)) { @@ -1310,7 +1320,7 @@ static std::optional autocompleteStringParams(const Source return std::nullopt; } - if (!nodes.back()->is() && (!FFlag::LuauCompleteBrokenStringParams || !nodes.back()->is())) + if (!nodes.back()->is() && !nodes.back()->is()) { return std::nullopt; } @@ -1408,8 +1418,8 @@ static AutocompleteResult autocomplete(const SourceModule& sourceModule, const M } else if (auto typeReference = node->as()) { - if (typeReference->hasPrefix) - return {autocompleteModuleTypes(*module, position, typeReference->prefix.value), finder.ancestry}; + if (typeReference->prefix) + return {autocompleteModuleTypes(*module, position, typeReference->prefix->value), finder.ancestry}; else return {autocompleteTypeNames(*module, position, finder.ancestry), finder.ancestry}; } @@ -1419,9 +1429,9 @@ static AutocompleteResult autocomplete(const SourceModule& sourceModule, const M } else if (AstStatLocal* statLocal = node->as()) { - if (statLocal->vars.size == 1 && (!statLocal->hasEqualsSign || position < statLocal->equalsSignLocation.begin)) + if (statLocal->vars.size == 1 && (!statLocal->equalsSignLocation || position < statLocal->equalsSignLocation->begin)) return {{{"function", AutocompleteEntry{AutocompleteEntryKind::Keyword}}}, finder.ancestry}; - else if (statLocal->hasEqualsSign && position >= statLocal->equalsSignLocation.end) + else if (statLocal->equalsSignLocation && position >= statLocal->equalsSignLocation->end) return {autocompleteExpression(sourceModule, *module, typeChecker, typeArena, finder.ancestry, position), finder.ancestry}; else return {}; @@ -1449,7 +1459,7 @@ static AutocompleteResult autocomplete(const SourceModule& sourceModule, const M if (!statForIn->hasIn || position <= statForIn->inLocation.begin) { AstLocal* lastName = statForIn->vars.data[statForIn->vars.size - 1]; - if (lastName->name == Parser::errorName || lastName->location.containsClosed(position)) + if (lastName->name == kParseNameError || lastName->location.containsClosed(position)) { // Here we are either working with a missing binding (as would be the case in a bare "for" keyword) or // the cursor is still touching a binding name. The user is still typing a new name, so we should not offer @@ -1499,7 +1509,7 @@ static AutocompleteResult autocomplete(const SourceModule& sourceModule, const M else if (AstStatWhile* statWhile = extractStat(finder.ancestry); statWhile && !statWhile->hasDo) return {{{"do", AutocompleteEntry{AutocompleteEntryKind::Keyword}}}, finder.ancestry}; - else if (AstStatIf* statIf = node->as(); statIf && !statIf->hasElse) + else if (AstStatIf* statIf = node->as(); statIf && !statIf->elseLocation.has_value()) { return {{{"else", AutocompleteEntry{AutocompleteEntryKind::Keyword}}, {"elseif", AutocompleteEntry{AutocompleteEntryKind::Keyword}}}, finder.ancestry}; @@ -1508,11 +1518,11 @@ static AutocompleteResult autocomplete(const SourceModule& sourceModule, const M { if (statIf->condition->is()) return {autocompleteExpression(sourceModule, *module, typeChecker, typeArena, finder.ancestry, position), finder.ancestry}; - else if (!statIf->hasThen || statIf->thenLocation.containsClosed(position)) + else if (!statIf->thenLocation || statIf->thenLocation->containsClosed(position)) return {{{"then", AutocompleteEntry{AutocompleteEntryKind::Keyword}}}, finder.ancestry}; } else if (AstStatIf* statIf = extractStat(finder.ancestry); - statIf && (!statIf->hasThen || statIf->thenLocation.containsClosed(position))) + statIf && (!statIf->thenLocation || statIf->thenLocation->containsClosed(position))) return {{{"then", AutocompleteEntry{AutocompleteEntryKind::Keyword}}}, finder.ancestry}; else if (AstStatRepeat* statRepeat = node->as(); statRepeat && statRepeat->condition->is()) return {autocompleteExpression(sourceModule, *module, typeChecker, typeArena, finder.ancestry, position), finder.ancestry}; @@ -1612,6 +1622,7 @@ AutocompleteResult autocomplete(Frontend& frontend, const ModuleName& moduleName OwningAutocompleteResult autocompleteSource(Frontend& frontend, std::string_view source, Position position, StringCompletionCallback callback) { + // TODO: Remove #include "Luau/Parser.h" with this function auto sourceModule = std::make_unique(); ParseOptions parseOptions; parseOptions.captureComments = true; diff --git a/Analysis/src/Config.cpp b/Analysis/src/Config.cpp index d9fc44f8..35a2259d 100644 --- a/Analysis/src/Config.cpp +++ b/Analysis/src/Config.cpp @@ -1,7 +1,7 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #include "Luau/Config.h" -#include "Luau/Parser.h" +#include "Luau/Lexer.h" #include "Luau/StringUtils.h" namespace diff --git a/Analysis/src/EmbeddedBuiltinDefinitions.cpp b/Analysis/src/EmbeddedBuiltinDefinitions.cpp index f3ef88fc..bf6e1193 100644 --- a/Analysis/src/EmbeddedBuiltinDefinitions.cpp +++ b/Analysis/src/EmbeddedBuiltinDefinitions.cpp @@ -167,7 +167,7 @@ declare function gcinfo(): number foreach: ({[K]: V}, (K, V) -> ()) -> (), foreachi: ({V}, (number, V) -> ()) -> (), - move: ({V}, number, number, number, {V}?) -> (), + move: ({V}, number, number, number, {V}?) -> {V}, clear: ({[K]: V}) -> (), freeze: ({[K]: V}) -> {[K]: V}, diff --git a/Analysis/src/Frontend.cpp b/Analysis/src/Frontend.cpp index 9001b19d..d8906f6e 100644 --- a/Analysis/src/Frontend.cpp +++ b/Analysis/src/Frontend.cpp @@ -4,6 +4,7 @@ #include "Luau/Common.h" #include "Luau/Config.h" #include "Luau/FileResolver.h" +#include "Luau/Parser.h" #include "Luau/Scope.h" #include "Luau/StringUtils.h" #include "Luau/TimeTrace.h" @@ -16,23 +17,25 @@ #include LUAU_FASTFLAG(LuauInferInNoCheckMode) -LUAU_FASTFLAGVARIABLE(LuauTypeCheckTwice, false) LUAU_FASTFLAGVARIABLE(LuauKnowsTheDataModel3, false) namespace Luau { -std::optional parseMode(const std::vector& hotcomments) +std::optional parseMode(const std::vector& hotcomments) { - for (const std::string& hc : hotcomments) + for (const HotComment& hc : hotcomments) { - if (hc == "nocheck") + if (!hc.header) + continue; + + if (hc.content == "nocheck") return Mode::NoCheck; - if (hc == "nonstrict") + if (hc.content == "nonstrict") return Mode::Nonstrict; - if (hc == "strict") + if (hc.content == "strict") return Mode::Strict; } @@ -607,13 +610,15 @@ std::pair Frontend::lintFragment(std::string_view sour SourceModule sourceModule = parse(ModuleName{}, source, config.parseOptions); + uint64_t ignoreLints = LintWarning::parseMask(sourceModule.hotcomments); + Luau::LintOptions lintOptions = enabledLintWarnings.value_or(config.enabledLint); - lintOptions.warningMask &= sourceModule.ignoreLints; + lintOptions.warningMask &= ~ignoreLints; double timestamp = getTimestamp(); - std::vector warnings = - Luau::lint(sourceModule.root, *sourceModule.names.get(), typeChecker.globalScope, nullptr, enabledLintWarnings.value_or(config.enabledLint)); + std::vector warnings = Luau::lint(sourceModule.root, *sourceModule.names.get(), typeChecker.globalScope, nullptr, + sourceModule.hotcomments, enabledLintWarnings.value_or(config.enabledLint)); stats.timeLint += getTimestamp() - timestamp; @@ -651,8 +656,10 @@ LintResult Frontend::lint(const SourceModule& module, std::optionalgetConfig(module.name); + uint64_t ignoreLints = LintWarning::parseMask(module.hotcomments); + LintOptions options = enabledLintWarnings.value_or(config.enabledLint); - options.warningMask &= ~module.ignoreLints; + options.warningMask &= ~ignoreLints; Mode mode = module.mode.value_or(config.mode); if (mode != Mode::NoCheck) @@ -671,7 +678,7 @@ LintResult Frontend::lint(const SourceModule& module, std::optional warnings = Luau::lint(module.root, *module.names, environmentScope, modulePtr.get(), options); + std::vector warnings = Luau::lint(module.root, *module.names, environmentScope, modulePtr.get(), module.hotcomments, options); stats.timeLint += getTimestamp() - timestamp; @@ -839,7 +846,6 @@ SourceModule Frontend::parse(const ModuleName& name, std::string_view src, const { sourceModule.root = parseResult.root; sourceModule.mode = parseMode(parseResult.hotcomments); - sourceModule.ignoreLints = LintWarning::parseMask(parseResult.hotcomments); } else { @@ -848,8 +854,13 @@ SourceModule Frontend::parse(const ModuleName& name, std::string_view src, const } sourceModule.name = name; + if (parseOptions.captureComments) + { sourceModule.commentLocations = std::move(parseResult.commentLocations); + sourceModule.hotcomments = std::move(parseResult.hotcomments); + } + return sourceModule; } diff --git a/Analysis/src/JsonEncoder.cpp b/Analysis/src/JsonEncoder.cpp index 8dd597e1..ec399158 100644 --- a/Analysis/src/JsonEncoder.cpp +++ b/Analysis/src/JsonEncoder.cpp @@ -150,10 +150,21 @@ struct AstJsonEncoder : public AstVisitor { writeRaw(std::to_string(i)); } + void write(std::nullptr_t) + { + writeRaw("null"); + } void write(std::string_view str) { writeString(str); } + void write(std::optional name) + { + if (name) + write(*name); + else + writeRaw("null"); + } void write(AstName name) { writeString(name.value ? name.value : ""); @@ -177,7 +188,16 @@ struct AstJsonEncoder : public AstVisitor void write(AstLocal* local) { - write(local->name); + writeRaw("{"); + bool c = pushComma(); + if (local->annotation != nullptr) + write("type", local->annotation); + else + write("type", nullptr); + write("name", local->name); + write("location", local->location); + popComma(c); + writeRaw("}"); } void writeNode(AstNode* node) @@ -314,7 +334,7 @@ struct AstJsonEncoder : public AstVisitor if (node->self) PROP(self); PROP(args); - if (node->hasReturnAnnotation) + if (node->returnAnnotation) PROP(returnAnnotation); PROP(vararg); PROP(varargLocation); @@ -328,6 +348,14 @@ struct AstJsonEncoder : public AstVisitor }); } + void write(const std::optional& typeList) + { + if (typeList) + write(*typeList); + else + writeRaw("null"); + } + void write(const AstTypeList& typeList) { writeRaw("{"); @@ -531,7 +559,7 @@ struct AstJsonEncoder : public AstVisitor PROP(thenbody); if (node->elsebody) PROP(elsebody); - PROP(hasThen); + write("hasThen", node->thenLocation.has_value()); PROP(hasEnd); }); } @@ -715,7 +743,7 @@ struct AstJsonEncoder : public AstVisitor void write(class AstTypeReference* node) { writeNode(node, "AstTypeReference", [&]() { - if (node->hasPrefix) + if (node->prefix) PROP(prefix); PROP(name); PROP(parameters); diff --git a/Analysis/src/Linter.cpp b/Analysis/src/Linter.cpp index 2ba6a0fc..8d7d2d97 100644 --- a/Analysis/src/Linter.cpp +++ b/Analysis/src/Linter.cpp @@ -12,6 +12,8 @@ #include #include +LUAU_FASTINTVARIABLE(LuauSuggestionDistance, 4) + namespace Luau { @@ -44,6 +46,7 @@ static const char* kWarningNames[] = { "TableOperations", "DuplicateCondition", "MisleadingAndOr", + "CommentDirective", }; // clang-format on @@ -732,13 +735,13 @@ private: bool visit(AstTypeReference* node) override { - if (!node->hasPrefix) + if (!node->prefix) return true; - if (!imports.contains(node->prefix)) + if (!imports.contains(*node->prefix)) return true; - AstLocal* astLocal = imports[node->prefix]; + AstLocal* astLocal = imports[*node->prefix]; Local& local = locals[astLocal]; LUAU_ASSERT(local.import); local.used = true; @@ -2527,13 +2530,108 @@ static void fillBuiltinGlobals(LintContext& context, const AstNameTable& names, } } +static const char* fuzzyMatch(std::string_view str, const char** array, size_t size) +{ + if (FInt::LuauSuggestionDistance == 0) + return nullptr; + + size_t bestDistance = FInt::LuauSuggestionDistance; + size_t bestMatch = size; + + for (size_t i = 0; i < size; ++i) + { + size_t ed = editDistance(str, array[i]); + + if (ed <= bestDistance) + { + bestDistance = ed; + bestMatch = i; + } + } + + return bestMatch < size ? array[bestMatch] : nullptr; +} + +static void lintComments(LintContext& context, const std::vector& hotcomments) +{ + bool seenMode = false; + + for (const HotComment& hc : hotcomments) + { + // We reserve --! for various informational (non-directive) comments + if (hc.content.empty() || hc.content[0] == ' ' || hc.content[0] == '\t') + continue; + + if (!hc.header) + { + emitWarning(context, LintWarning::Code_CommentDirective, hc.location, + "Comment directive is ignored because it is placed after the first non-comment token"); + } + else + { + std::string::size_type space = hc.content.find_first_of(" \t"); + std::string_view first = std::string_view(hc.content).substr(0, space); + + if (first == "nolint") + { + std::string::size_type notspace = hc.content.find_first_not_of(" \t", space); + + if (space == std::string::npos || notspace == std::string::npos) + { + // disables all lints + } + else if (LintWarning::parseName(hc.content.c_str() + notspace) == LintWarning::Code_Unknown) + { + const char* rule = hc.content.c_str() + notspace; + + // skip Unknown + if (const char* suggestion = fuzzyMatch(rule, kWarningNames + 1, LintWarning::Code__Count - 1)) + emitWarning(context, LintWarning::Code_CommentDirective, hc.location, + "nolint directive refers to unknown lint rule '%s'; did you mean '%s'?", rule, suggestion); + else + emitWarning( + context, LintWarning::Code_CommentDirective, hc.location, "nolint directive refers to unknown lint rule '%s'", rule); + } + } + else if (first == "nocheck" || first == "nonstrict" || first == "strict") + { + if (space != std::string::npos) + emitWarning(context, LintWarning::Code_CommentDirective, hc.location, + "Comment directive with the type checking mode has extra symbols at the end of the line"); + else if (seenMode) + emitWarning(context, LintWarning::Code_CommentDirective, hc.location, + "Comment directive with the type checking mode has already been used"); + else + seenMode = true; + } + else + { + static const char* kHotComments[] = { + "nolint", + "nocheck", + "nonstrict", + "strict", + }; + + if (const char* suggestion = fuzzyMatch(first, kHotComments, std::size(kHotComments))) + emitWarning(context, LintWarning::Code_CommentDirective, hc.location, "Unknown comment directive '%.*s'; did you mean '%s'?", + int(first.size()), first.data(), suggestion); + else + emitWarning(context, LintWarning::Code_CommentDirective, hc.location, "Unknown comment directive '%.*s'", int(first.size()), + first.data()); + } + } + } +} + void LintOptions::setDefaults() { // By default, we enable all warnings warningMask = ~0ull; } -std::vector lint(AstStat* root, const AstNameTable& names, const ScopePtr& env, const Module* module, const LintOptions& options) +std::vector lint(AstStat* root, const AstNameTable& names, const ScopePtr& env, const Module* module, + const std::vector& hotcomments, const LintOptions& options) { LintContext context; @@ -2609,6 +2707,9 @@ std::vector lint(AstStat* root, const AstNameTable& names, const Sc if (context.warningEnabled(LintWarning::Code_MisleadingAndOr)) LintMisleadingAndOr::process(context); + if (context.warningEnabled(LintWarning::Code_CommentDirective)) + lintComments(context, hotcomments); + std::sort(context.result.begin(), context.result.end(), WarningComparator()); return context.result; @@ -2630,23 +2731,30 @@ LintWarning::Code LintWarning::parseName(const char* name) return Code_Unknown; } -uint64_t LintWarning::parseMask(const std::vector& hotcomments) +uint64_t LintWarning::parseMask(const std::vector& hotcomments) { uint64_t result = 0; - for (const std::string& hc : hotcomments) + for (const HotComment& hc : hotcomments) { - if (hc.compare(0, 6, "nolint") != 0) + if (!hc.header) continue; - std::string::size_type name = hc.find_first_not_of(" \t", 6); + if (hc.content.compare(0, 6, "nolint") != 0) + continue; + + std::string::size_type name = hc.content.find_first_not_of(" \t", 6); // --!nolint disables everything if (name == std::string::npos) return ~0ull; + // --!nolint needs to be followed by a whitespace character + if (name == 6) + continue; + // --!nolint name disables the specific lint - LintWarning::Code code = LintWarning::parseName(hc.c_str() + name); + LintWarning::Code code = LintWarning::parseName(hc.content.c_str() + name); if (code != LintWarning::Code_Unknown) result |= 1ull << int(code); diff --git a/Analysis/src/Quantify.cpp b/Analysis/src/Quantify.cpp index 04ebffc1..94e169f1 100644 --- a/Analysis/src/Quantify.cpp +++ b/Analysis/src/Quantify.cpp @@ -9,14 +9,12 @@ namespace Luau struct Quantifier { - ModulePtr module; TypeLevel level; std::vector generics; std::vector genericPacks; - Quantifier(ModulePtr module, TypeLevel level) - : module(module) - , level(level) + Quantifier(TypeLevel level) + : level(level) { } @@ -76,9 +74,9 @@ struct Quantifier } }; -void quantify(ModulePtr module, TypeId ty, TypeLevel level) +void quantify(TypeId ty, TypeLevel level) { - Quantifier q{std::move(module), level}; + Quantifier q{level}; DenseHashSet seen{nullptr}; visitTypeVarOnce(ty, q, seen); diff --git a/Analysis/src/Substitution.cpp b/Analysis/src/Substitution.cpp index bacbca76..770c7a47 100644 --- a/Analysis/src/Substitution.cpp +++ b/Analysis/src/Substitution.cpp @@ -226,27 +226,11 @@ TarjanResult Tarjan::loop() return TarjanResult::Ok; } -void Tarjan::clear() -{ - typeToIndex.clear(); - indexToType.clear(); - packToIndex.clear(); - indexToPack.clear(); - lowlink.clear(); - stack.clear(); - onStack.clear(); - - edgesTy.clear(); - edgesTp.clear(); - worklist.clear(); -} - TarjanResult Tarjan::visitRoot(TypeId ty) { childCount = 0; ty = log->follow(ty); - clear(); auto [index, fresh] = indexify(ty); worklist.push_back({index, -1, -1}); return loop(); @@ -257,7 +241,6 @@ TarjanResult Tarjan::visitRoot(TypePackId tp) childCount = 0; tp = log->follow(tp); - clear(); auto [index, fresh] = indexify(tp); worklist.push_back({index, -1, -1}); return loop(); @@ -314,21 +297,17 @@ void FindDirty::visitSCC(int index) TarjanResult FindDirty::findDirty(TypeId ty) { - dirty.clear(); return visitRoot(ty); } TarjanResult FindDirty::findDirty(TypePackId tp) { - dirty.clear(); return visitRoot(tp); } std::optional Substitution::substitute(TypeId ty) { ty = log->follow(ty); - newTypes.clear(); - newPacks.clear(); auto result = findDirty(ty); if (result != TarjanResult::Ok) @@ -347,8 +326,6 @@ std::optional Substitution::substitute(TypeId ty) std::optional Substitution::substitute(TypePackId tp) { tp = log->follow(tp); - newTypes.clear(); - newPacks.clear(); auto result = findDirty(tp); if (result != TarjanResult::Ok) diff --git a/Analysis/src/TopoSortStatements.cpp b/Analysis/src/TopoSortStatements.cpp index dba694be..678001bf 100644 --- a/Analysis/src/TopoSortStatements.cpp +++ b/Analysis/src/TopoSortStatements.cpp @@ -26,9 +26,10 @@ * 3. Cyclic dependencies can be resolved by picking an arbitrary statement to check first. */ -#include "Luau/Parser.h" +#include "Luau/Ast.h" #include "Luau/DenseHash.h" #include "Luau/Common.h" +#include "Luau/StringUtils.h" #include #include diff --git a/Analysis/src/Transpiler.cpp b/Analysis/src/Transpiler.cpp index f5908683..54bd0d5e 100644 --- a/Analysis/src/Transpiler.cpp +++ b/Analysis/src/Transpiler.cpp @@ -933,12 +933,12 @@ struct Printer writer.symbol(")"); - if (writeTypes && func.hasReturnAnnotation) + if (writeTypes && func.returnAnnotation) { writer.symbol(":"); writer.space(); - visualizeTypeList(func.returnAnnotation, false); + visualizeTypeList(*func.returnAnnotation, false); } visualizeBlock(*func.body); @@ -989,9 +989,9 @@ struct Printer advance(typeAnnotation.location.begin); if (const auto& a = typeAnnotation.as()) { - if (a->hasPrefix) + if (a->prefix) { - writer.write(a->prefix.value); + writer.write(a->prefix->value); writer.symbol("."); } diff --git a/Analysis/src/TypeAttach.cpp b/Analysis/src/TypeAttach.cpp index 2208213f..d575e023 100644 --- a/Analysis/src/TypeAttach.cpp +++ b/Analysis/src/TypeAttach.cpp @@ -3,7 +3,6 @@ #include "Luau/Error.h" #include "Luau/Module.h" -#include "Luau/Parser.h" #include "Luau/RecursionCounter.h" #include "Luau/Scope.h" #include "Luau/ToString.h" @@ -476,12 +475,11 @@ public: visitLocal(arg); } - if (!fn->hasReturnAnnotation) + if (!fn->returnAnnotation) { if (auto result = getScope(fn->body->location)) { TypePackId ret = result->returnType; - fn->hasReturnAnnotation = true; AstTypePack* variadicAnnotation = nullptr; const auto& [v, tail] = flatten(ret); diff --git a/Analysis/src/TypeInfer.cpp b/Analysis/src/TypeInfer.cpp index f1c314cd..c29699b7 100644 --- a/Analysis/src/TypeInfer.cpp +++ b/Analysis/src/TypeInfer.cpp @@ -3,7 +3,6 @@ #include "Luau/Common.h" #include "Luau/ModuleResolver.h" -#include "Luau/Parser.h" #include "Luau/Quantify.h" #include "Luau/RecursionCounter.h" #include "Luau/Scope.h" @@ -24,16 +23,12 @@ LUAU_FASTINTVARIABLE(LuauTypeInferTypePackLoopLimit, 5000) LUAU_FASTINTVARIABLE(LuauCheckRecursionLimit, 500) LUAU_FASTFLAG(LuauKnowsTheDataModel3) LUAU_FASTFLAGVARIABLE(LuauEqConstraint, false) -LUAU_FASTFLAGVARIABLE(LuauGroupExpectedType, false) LUAU_FASTFLAGVARIABLE(LuauWeakEqConstraint, false) // Eventually removed as false. LUAU_FASTFLAG(LuauUseCommittingTxnLog) LUAU_FASTFLAGVARIABLE(DebugLuauFreezeDuringUnification, false) LUAU_FASTFLAGVARIABLE(LuauRecursiveTypeParameterRestriction, false) LUAU_FASTFLAGVARIABLE(LuauGenericFunctionsDontCacheTypeParams, false) -LUAU_FASTFLAGVARIABLE(LuauIfElseBranchTypeUnion, false) -LUAU_FASTFLAGVARIABLE(LuauIfElseExpectedType2, false) LUAU_FASTFLAGVARIABLE(LuauImmutableTypes, false) -LUAU_FASTFLAGVARIABLE(LuauLengthOnCompositeType, false) LUAU_FASTFLAGVARIABLE(LuauNoSealedTypeMod, false) LUAU_FASTFLAGVARIABLE(LuauQuantifyInPlace2, false) LUAU_FASTFLAGVARIABLE(LuauSealExports, false) @@ -43,7 +38,6 @@ LUAU_FASTFLAGVARIABLE(LuauTypeAliasDefaults, false) LUAU_FASTFLAGVARIABLE(LuauExpectedTypesOfProperties, false) LUAU_FASTFLAGVARIABLE(LuauErrorRecoveryType, false) LUAU_FASTFLAGVARIABLE(LuauPropertiesGetExpectedType, false) -LUAU_FASTFLAGVARIABLE(LuauPerModuleUnificationCache, false) LUAU_FASTFLAGVARIABLE(LuauProperTypeLevels, false) LUAU_FASTFLAGVARIABLE(LuauAscribeCorrectLevelToInferredProperitesOfFreeTables, false) LUAU_FASTFLAG(LuauUnionTagMatchFix) @@ -293,13 +287,10 @@ ModulePtr TypeChecker::check(const SourceModule& module, Mode mode, std::optiona GenericError{"Free types leaked into this module's public interface. This is an internal Luau error; please report it."}}); } - if (FFlag::LuauPerModuleUnificationCache) - { - // Clear unifier cache since it's keyed off internal types that get deallocated - // This avoids fake cross-module cache hits and keeps cache size at bay when typechecking large module graphs. - unifierState.cachedUnify.clear(); - unifierState.skipCacheForType.clear(); - } + // Clear unifier cache since it's keyed off internal types that get deallocated + // This avoids fake cross-module cache hits and keeps cache size at bay when typechecking large module graphs. + unifierState.cachedUnify.clear(); + unifierState.skipCacheForType.clear(); if (FFlag::LuauTwoPassAliasDefinitionFix) duplicateTypeAliases.clear(); @@ -509,7 +500,7 @@ LUAU_NOINLINE void TypeChecker::checkBlockTypeAliases(const ScopePtr& scope, std { if (const auto& typealias = stat->as()) { - if (FFlag::LuauTwoPassAliasDefinitionFix && typealias->name == Parser::errorName) + if (FFlag::LuauTwoPassAliasDefinitionFix && typealias->name == kParseNameError) continue; auto& bindings = typealias->exported ? scope->exportedTypeBindings : scope->privateTypeBindings; @@ -1193,7 +1184,7 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatTypeAlias& typealias Name name = typealias.name.value; // If the alias is missing a name, we can't do anything with it. Ignore it. - if (FFlag::LuauTwoPassAliasDefinitionFix && name == Parser::errorName) + if (FFlag::LuauTwoPassAliasDefinitionFix && name == kParseNameError) return; std::optional binding; @@ -1222,7 +1213,8 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatTypeAlias& typealias if (FFlag::LuauProperTypeLevels) aliasScope->level.subLevel = subLevel; - auto [generics, genericPacks] = createGenericTypes(aliasScope, scope->level, typealias, typealias.generics, typealias.genericPacks, /* useCache = */ true); + auto [generics, genericPacks] = + createGenericTypes(aliasScope, scope->level, typealias, typealias.generics, typealias.genericPacks, /* useCache = */ true); TypeId ty = freshType(aliasScope); FreeTypeVar* ftv = getMutable(ty); @@ -1464,7 +1456,7 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExpr& ExprResult result; if (auto a = expr.as()) - result = checkExpr(scope, *a->expr, FFlag::LuauGroupExpectedType ? expectedType : std::nullopt); + result = checkExpr(scope, *a->expr, expectedType); else if (expr.is()) result = {nilType}; else if (const AstExprConstantBool* bexpr = expr.as()) @@ -1508,7 +1500,7 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExpr& else if (auto a = expr.as()) result = checkExpr(scope, *a); else if (auto a = expr.as()) - result = checkExpr(scope, *a, FFlag::LuauIfElseExpectedType2 ? expectedType : std::nullopt); + result = checkExpr(scope, *a, expectedType); else ice("Unhandled AstExpr?"); @@ -2093,6 +2085,7 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprUn return {numberType}; } case AstExprUnary::Len: + { tablify(operandType); operandType = stripFromNilAndReport(operandType, expr.location); @@ -2100,30 +2093,13 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprUn if (get(operandType)) return {errorRecoveryType(scope)}; - if (FFlag::LuauLengthOnCompositeType) - { - DenseHashSet seen{nullptr}; + DenseHashSet seen{nullptr}; - if (!hasLength(operandType, seen, &recursionCount)) - reportError(TypeError{expr.location, NotATable{operandType}}); - } - else - { - if (get(operandType)) - return {numberType}; // Not strictly correct: metatables permit overriding this - - if (auto p = get(operandType)) - { - if (p->type == PrimitiveTypeVar::String) - return {numberType}; - } - - if (!getTableType(operandType)) - reportError(TypeError{expr.location, NotATable{operandType}}); - } + if (!hasLength(operandType, seen, &recursionCount)) + reportError(TypeError{expr.location, NotATable{operandType}}); return {numberType}; - + } default: ice("Unknown AstExprUnary " + std::to_string(int(expr.op))); } @@ -2618,22 +2594,11 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprIf resolve(result.predicates, falseScope, false); ExprResult falseType = checkExpr(falseScope, *expr.falseExpr, expectedType); - if (FFlag::LuauIfElseBranchTypeUnion) - { - if (falseType.type == trueType.type) - return {trueType.type}; - - std::vector types = reduceUnion({trueType.type, falseType.type}); - return {types.size() == 1 ? types[0] : addType(UnionTypeVar{std::move(types)})}; - } - else - { - unify(falseType.type, trueType.type, expr.location); - - // TODO: normalize(UnionTypeVar{{trueType, falseType}}) - // For now both trueType and falseType must be the same type. + if (falseType.type == trueType.type) return {trueType.type}; - } + + std::vector types = reduceUnion({trueType.type, falseType.type}); + return {types.size() == 1 ? types[0] : addType(UnionTypeVar{std::move(types)})}; } TypeId TypeChecker::checkLValue(const ScopePtr& scope, const AstExpr& expr) @@ -2986,8 +2951,8 @@ std::pair TypeChecker::checkFunctionSignature( auto [generics, genericPacks] = createGenericTypes(funScope, std::nullopt, expr, expr.generics, expr.genericPacks); TypePackId retPack; - if (expr.hasReturnAnnotation) - retPack = resolveTypePack(funScope, expr.returnAnnotation); + if (expr.returnAnnotation) + retPack = resolveTypePack(funScope, *expr.returnAnnotation); else if (isNonstrictMode()) retPack = anyTypePack; else if (expectedFunctionType) @@ -3181,7 +3146,7 @@ void TypeChecker::checkFunctionBody(const ScopePtr& scope, TypeId ty, const AstE // If we're in nonstrict mode we want to only report this missing return // statement if there are type annotations on the function. In strict mode // we report it regardless. - if (!isNonstrictMode() || function.hasReturnAnnotation) + if (!isNonstrictMode() || function.returnAnnotation) { reportError(getEndLocation(function), FunctionExitsWithoutReturning{funTy->retType}); } @@ -4403,11 +4368,7 @@ TypeId Instantiation::clean(TypeId ty) // Annoyingly, we have to do this even if there are no generics, // to replace any generic tables. - replaceGenerics.log = log; - replaceGenerics.level = level; - replaceGenerics.currentModule = currentModule; - replaceGenerics.generics.assign(ftv->generics.begin(), ftv->generics.end()); - replaceGenerics.genericPacks.assign(ftv->genericPacks.begin(), ftv->genericPacks.end()); + ReplaceGenerics replaceGenerics{log, arena, level, ftv->generics, ftv->genericPacks}; // TODO: What to do if this returns nullopt? // We don't have access to the error-reporting machinery @@ -4573,16 +4534,11 @@ TypeId TypeChecker::quantify(const ScopePtr& scope, TypeId ty, Location location if (FFlag::LuauQuantifyInPlace2) { - Luau::quantify(currentModule, ty, scope->level); + Luau::quantify(ty, scope->level); return ty; } - quantification.log = TxnLog::empty(); - quantification.level = scope->level; - quantification.generics.clear(); - quantification.genericPacks.clear(); - quantification.currentModule = currentModule; - + Quantification quantification{¤tModule->internalTypes, scope->level}; std::optional qty = quantification.substitute(ty); if (!qty.has_value()) @@ -4596,18 +4552,14 @@ TypeId TypeChecker::quantify(const ScopePtr& scope, TypeId ty, Location location FunctionTypeVar* qftv = getMutable(*qty); LUAU_ASSERT(qftv); - qftv->generics = quantification.generics; - qftv->genericPacks = quantification.genericPacks; + qftv->generics = std::move(quantification.generics); + qftv->genericPacks = std::move(quantification.genericPacks); return *qty; } TypeId TypeChecker::instantiate(const ScopePtr& scope, TypeId ty, Location location, const TxnLog* log) { - LUAU_ASSERT(log != nullptr); - - instantiation.log = FFlag::LuauUseCommittingTxnLog ? log : TxnLog::empty(); - instantiation.level = scope->level; - instantiation.currentModule = currentModule; + Instantiation instantiation{FFlag::LuauUseCommittingTxnLog ? log : TxnLog::empty(), ¤tModule->internalTypes, scope->level}; std::optional instantiated = instantiation.substitute(ty); if (instantiated.has_value()) return *instantiated; @@ -4620,10 +4572,7 @@ TypeId TypeChecker::instantiate(const ScopePtr& scope, TypeId ty, Location locat TypeId TypeChecker::anyify(const ScopePtr& scope, TypeId ty, Location location) { - anyification.log = TxnLog::empty(); - anyification.anyType = anyType; - anyification.anyTypePack = anyTypePack; - anyification.currentModule = currentModule; + Anyification anyification{¤tModule->internalTypes, anyType, anyTypePack}; std::optional any = anyification.substitute(ty); if (any.has_value()) return *any; @@ -4636,10 +4585,7 @@ TypeId TypeChecker::anyify(const ScopePtr& scope, TypeId ty, Location location) TypePackId TypeChecker::anyify(const ScopePtr& scope, TypePackId ty, Location location) { - anyification.log = TxnLog::empty(); - anyification.anyType = anyType; - anyification.anyTypePack = anyTypePack; - anyification.currentModule = currentModule; + Anyification anyification{¤tModule->internalTypes, anyType, anyTypePack}; std::optional any = anyification.substitute(ty); if (any.has_value()) return *any; @@ -4823,7 +4769,8 @@ TypePackId TypeChecker::errorRecoveryTypePack(TypePackId guess) return getSingletonTypes().errorRecoveryTypePack(guess); } -TypeIdPredicate TypeChecker::mkTruthyPredicate(bool sense) { +TypeIdPredicate TypeChecker::mkTruthyPredicate(bool sense) +{ return [this, sense](TypeId ty) -> std::optional { // any/error/free gets a special pass unconditionally because they can't be decided. if (get(ty) || get(ty) || get(ty)) @@ -4904,8 +4851,8 @@ TypeId TypeChecker::resolveType(const ScopePtr& scope, const AstType& annotation if (const auto& lit = annotation.as()) { std::optional tf; - if (lit->hasPrefix) - tf = scope->lookupImportedType(lit->prefix.value, lit->name.value); + if (lit->prefix) + tf = scope->lookupImportedType(lit->prefix->value, lit->name.value); else if (FFlag::DebugLuauMagicTypes && lit->name == "_luau_ice") ice("_luau_ice encountered", lit->location); @@ -4932,12 +4879,12 @@ TypeId TypeChecker::resolveType(const ScopePtr& scope, const AstType& annotation if (!tf) { - if (lit->name == Parser::errorName) + if (lit->name == kParseNameError) return errorRecoveryType(scope); std::string typeName; - if (lit->hasPrefix) - typeName = std::string(lit->prefix.value) + "."; + if (lit->prefix) + typeName = std::string(lit->prefix->value) + "."; typeName += lit->name.value; if (scope->lookupPack(typeName)) @@ -5038,12 +4985,7 @@ TypeId TypeChecker::resolveType(const ScopePtr& scope, const AstType& annotation if (notEnoughParameters && hasDefaultParameters) { // 'applyTypeFunction' is used to substitute default types that reference previous generic types - applyTypeFunction.log = TxnLog::empty(); - applyTypeFunction.typeArguments.clear(); - applyTypeFunction.typePackArguments.clear(); - applyTypeFunction.currentModule = currentModule; - applyTypeFunction.level = scope->level; - applyTypeFunction.encounteredForwardedType = false; + ApplyTypeFunction applyTypeFunction{¤tModule->internalTypes, scope->level}; for (size_t i = 0; i < typesProvided; ++i) applyTypeFunction.typeArguments[tf->typeParams[i].ty] = typeParams[i]; @@ -5362,18 +5304,14 @@ TypeId TypeChecker::instantiateTypeFun(const ScopePtr& scope, const TypeFun& tf, if (tf.typeParams.empty() && tf.typePackParams.empty()) return tf.type; - applyTypeFunction.typeArguments.clear(); + ApplyTypeFunction applyTypeFunction{¤tModule->internalTypes, scope->level}; + for (size_t i = 0; i < tf.typeParams.size(); ++i) applyTypeFunction.typeArguments[tf.typeParams[i].ty] = typeParams[i]; - applyTypeFunction.typePackArguments.clear(); for (size_t i = 0; i < tf.typePackParams.size(); ++i) applyTypeFunction.typePackArguments[tf.typePackParams[i].tp] = typePackParams[i]; - applyTypeFunction.log = TxnLog::empty(); - applyTypeFunction.currentModule = currentModule; - applyTypeFunction.level = scope->level; - applyTypeFunction.encounteredForwardedType = false; std::optional maybeInstantiated = applyTypeFunction.substitute(tf.type); if (!maybeInstantiated.has_value()) { diff --git a/Analysis/src/TypeUtils.cpp b/Analysis/src/TypeUtils.cpp index 8c6d5e49..593b54c8 100644 --- a/Analysis/src/TypeUtils.cpp +++ b/Analysis/src/TypeUtils.cpp @@ -5,6 +5,8 @@ #include "Luau/ToString.h" #include "Luau/TypeInfer.h" +LUAU_FASTFLAGVARIABLE(LuauTerminateCyclicMetatableIndexLookup, false) + namespace Luau { @@ -48,9 +50,19 @@ std::optional findTablePropertyRespectingMeta(ErrorVec& errors, const Sc } std::optional mtIndex = findMetatableEntry(errors, globalScope, ty, "__index", location); + int count = 0; while (mtIndex) { TypeId index = follow(*mtIndex); + + if (FFlag::LuauTerminateCyclicMetatableIndexLookup) + { + if (count >= 100) + return std::nullopt; + + ++count; + } + if (const auto& itt = getTableType(index)) { const auto& fit = itt->props.find(name); diff --git a/Analysis/src/TypeVar.cpp b/Analysis/src/TypeVar.cpp index 7e438e31..b2358c27 100644 --- a/Analysis/src/TypeVar.cpp +++ b/Analysis/src/TypeVar.cpp @@ -23,8 +23,6 @@ LUAU_FASTFLAG(DebugLuauFreezeArena) LUAU_FASTINTVARIABLE(LuauTypeMaximumStringifierLength, 500) LUAU_FASTINTVARIABLE(LuauTableTypeMaximumStringifierLength, 0) LUAU_FASTINT(LuauTypeInferRecursionLimit) -LUAU_FASTFLAG(LuauLengthOnCompositeType) -LUAU_FASTFLAGVARIABLE(LuauMetatableAreEqualRecursion, false) LUAU_FASTFLAGVARIABLE(LuauRefactorTypeVarQuestions, false) LUAU_FASTFLAG(LuauErrorRecoveryType) LUAU_FASTFLAG(LuauUnionTagMatchFix) @@ -385,8 +383,6 @@ bool maybeSingleton(TypeId ty) bool hasLength(TypeId ty, DenseHashSet& seen, int* recursionCount) { - LUAU_ASSERT(FFlag::LuauLengthOnCompositeType); - RecursionLimiter _rl(recursionCount, FInt::LuauTypeInferRecursionLimit); ty = follow(ty); @@ -555,7 +551,7 @@ bool areEqual(SeenSet& seen, const TableTypeVar& lhs, const TableTypeVar& rhs) static bool areEqual(SeenSet& seen, const MetatableTypeVar& lhs, const MetatableTypeVar& rhs) { - if (FFlag::LuauMetatableAreEqualRecursion && areSeen(seen, &lhs, &rhs)) + if (areSeen(seen, &lhs, &rhs)) return true; return areEqual(seen, *lhs.table, *rhs.table) && areEqual(seen, *lhs.metatable, *rhs.metatable); diff --git a/Analysis/src/TypedAllocator.cpp b/Analysis/src/TypedAllocator.cpp index f037351e..c7f31822 100644 --- a/Analysis/src/TypedAllocator.cpp +++ b/Analysis/src/TypedAllocator.cpp @@ -7,6 +7,9 @@ #ifndef WIN32_LEAN_AND_MEAN #define WIN32_LEAN_AND_MEAN #endif +#ifndef NOMINMAX +#define NOMINMAX +#endif #include const size_t kPageSize = 4096; diff --git a/Analysis/src/Unifier.cpp b/Analysis/src/Unifier.cpp index a8ad5159..322f6ebf 100644 --- a/Analysis/src/Unifier.cpp +++ b/Analysis/src/Unifier.cpp @@ -14,7 +14,6 @@ LUAU_FASTINT(LuauTypeInferRecursionLimit); LUAU_FASTINT(LuauTypeInferTypePackLoopLimit); -LUAU_FASTFLAGVARIABLE(LuauCommittingTxnLogFreeTpPromote, false) LUAU_FASTFLAG(LuauImmutableTypes) LUAU_FASTFLAG(LuauUseCommittingTxnLog) LUAU_FASTINTVARIABLE(LuauTypeInferIterationLimit, 2000); @@ -23,7 +22,6 @@ LUAU_FASTFLAGVARIABLE(LuauTableUnificationEarlyTest, false) LUAU_FASTFLAG(LuauSingletonTypes) LUAU_FASTFLAG(LuauErrorRecoveryType); LUAU_FASTFLAG(LuauProperTypeLevels); -LUAU_FASTFLAGVARIABLE(LuauUnifyPackTails, false) LUAU_FASTFLAGVARIABLE(LuauUnionTagMatchFix, false) LUAU_FASTFLAGVARIABLE(LuauFollowWithCommittingTxnLogInAnyUnification, false) @@ -116,7 +114,7 @@ struct PromoteTypeLevels { // Surprise, it's actually a BoundTypePack that hasn't been committed yet. // Calling getMutable on this will trigger an assertion. - if (FFlag::LuauCommittingTxnLogFreeTpPromote && FFlag::LuauUseCommittingTxnLog && !log.is(tp)) + if (FFlag::LuauUseCommittingTxnLog && !log.is(tp)) return true; promote(tp, FFlag::LuauUseCommittingTxnLog ? log.getMutable(tp) : getMutable(tp)); @@ -1242,7 +1240,7 @@ void Unifier::tryUnify_(TypePackId subTp, TypePackId superTp, bool isFunctionCal // If both are at the end, we're done if (!superIter.good() && !subIter.good()) { - if (FFlag::LuauUnifyPackTails && subTpv->tail && superTpv->tail) + if (subTpv->tail && superTpv->tail) { tryUnify_(*subTpv->tail, *superTpv->tail); break; @@ -1250,9 +1248,7 @@ void Unifier::tryUnify_(TypePackId subTp, TypePackId superTp, bool isFunctionCal const bool lFreeTail = superTpv->tail && log.getMutable(log.follow(*superTpv->tail)) != nullptr; const bool rFreeTail = subTpv->tail && log.getMutable(log.follow(*subTpv->tail)) != nullptr; - if (!FFlag::LuauUnifyPackTails && lFreeTail && rFreeTail) - tryUnify_(*subTpv->tail, *superTpv->tail); - else if (lFreeTail) + if (lFreeTail) tryUnify_(emptyTp, *superTpv->tail); else if (rFreeTail) tryUnify_(emptyTp, *subTpv->tail); @@ -1448,7 +1444,7 @@ void Unifier::tryUnify_(TypePackId subTp, TypePackId superTp, bool isFunctionCal // If both are at the end, we're done if (!superIter.good() && !subIter.good()) { - if (FFlag::LuauUnifyPackTails && subTpv->tail && superTpv->tail) + if (subTpv->tail && superTpv->tail) { tryUnify_(*subTpv->tail, *superTpv->tail); break; @@ -1456,9 +1452,7 @@ void Unifier::tryUnify_(TypePackId subTp, TypePackId superTp, bool isFunctionCal const bool lFreeTail = superTpv->tail && get(follow(*superTpv->tail)) != nullptr; const bool rFreeTail = subTpv->tail && get(follow(*subTpv->tail)) != nullptr; - if (!FFlag::LuauUnifyPackTails && lFreeTail && rFreeTail) - tryUnify_(*subTpv->tail, *superTpv->tail); - else if (lFreeTail) + if (lFreeTail) tryUnify_(emptyTp, *superTpv->tail); else if (rFreeTail) tryUnify_(emptyTp, *subTpv->tail); diff --git a/Ast/include/Luau/Ast.h b/Ast/include/Luau/Ast.h index ac5950c0..31cd01cc 100644 --- a/Ast/include/Luau/Ast.h +++ b/Ast/include/Luau/Ast.h @@ -594,8 +594,7 @@ public: AstArray genericPacks; AstLocal* self; AstArray args; - bool hasReturnAnnotation; - AstTypeList returnAnnotation; + std::optional returnAnnotation; bool vararg = false; Location varargLocation; AstTypePack* varargAnnotation; @@ -740,7 +739,7 @@ class AstStatIf : public AstStat public: LUAU_RTTI(AstStatIf) - AstStatIf(const Location& location, AstExpr* condition, AstStatBlock* thenbody, AstStat* elsebody, bool hasThen, const Location& thenLocation, + AstStatIf(const Location& location, AstExpr* condition, AstStatBlock* thenbody, AstStat* elsebody, const std::optional& thenLocation, const std::optional& elseLocation, bool hasEnd); void visit(AstVisitor* visitor) override; @@ -749,12 +748,10 @@ public: AstStatBlock* thenbody; AstStat* elsebody; - bool hasThen = false; - Location thenLocation; + std::optional thenLocation; // Active for 'elseif' as well - bool hasElse = false; - Location elseLocation; + std::optional elseLocation; bool hasEnd = false; }; @@ -849,8 +846,7 @@ public: AstArray vars; AstArray values; - bool hasEqualsSign = false; - Location equalsSignLocation; + std::optional equalsSignLocation; }; class AstStatFor : public AstStat @@ -1053,9 +1049,8 @@ public: void visit(AstVisitor* visitor) override; - bool hasPrefix; bool hasParameterList; - AstName prefix; + std::optional prefix; AstName name; AstArray parameters; }; diff --git a/Ast/include/Luau/Lexer.h b/Ast/include/Luau/Lexer.h index 460ef056..d7d867f4 100644 --- a/Ast/include/Luau/Lexer.h +++ b/Ast/include/Luau/Lexer.h @@ -233,4 +233,9 @@ private: bool readNames; }; +inline bool isSpace(char ch) +{ + return ch == ' ' || ch == '\t' || ch == '\r' || ch == '\n' || ch == '\v' || ch == '\f'; +} + } // namespace Luau diff --git a/Ast/include/Luau/ParseResult.h b/Ast/include/Luau/ParseResult.h new file mode 100644 index 00000000..17ce2e3b --- /dev/null +++ b/Ast/include/Luau/ParseResult.h @@ -0,0 +1,69 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +#pragma once + +#include "Luau/Common.h" +#include "Luau/Location.h" +#include "Luau/Lexer.h" +#include "Luau/StringUtils.h" + +namespace Luau +{ + +class AstStatBlock; + +class ParseError : public std::exception +{ +public: + ParseError(const Location& location, const std::string& message); + + virtual const char* what() const throw(); + + const Location& getLocation() const; + const std::string& getMessage() const; + + static LUAU_NORETURN void raise(const Location& location, const char* format, ...) LUAU_PRINTF_ATTR(2, 3); + +private: + Location location; + std::string message; +}; + +class ParseErrors : public std::exception +{ +public: + ParseErrors(std::vector errors); + + virtual const char* what() const throw(); + + const std::vector& getErrors() const; + +private: + std::vector errors; + std::string message; +}; + +struct HotComment +{ + bool header; + Location location; + std::string content; +}; + +struct Comment +{ + Lexeme::Type type; // Comment, BlockComment, or BrokenComment + Location location; +}; + +struct ParseResult +{ + AstStatBlock* root; + std::vector hotcomments; + std::vector errors; + + std::vector commentLocations; +}; + +static constexpr const char* kParseNameError = "%error-id%"; + +} // namespace Luau diff --git a/Ast/include/Luau/Parser.h b/Ast/include/Luau/Parser.h index 40ecdcdd..4b5ae315 100644 --- a/Ast/include/Luau/Parser.h +++ b/Ast/include/Luau/Parser.h @@ -4,6 +4,7 @@ #include "Luau/Ast.h" #include "Luau/Lexer.h" #include "Luau/ParseOptions.h" +#include "Luau/ParseResult.h" #include "Luau/StringUtils.h" #include "Luau/DenseHash.h" #include "Luau/Common.h" @@ -14,37 +15,6 @@ namespace Luau { -class ParseError : public std::exception -{ -public: - ParseError(const Location& location, const std::string& message); - - virtual const char* what() const throw(); - - const Location& getLocation() const; - const std::string& getMessage() const; - - static LUAU_NORETURN void raise(const Location& location, const char* format, ...) LUAU_PRINTF_ATTR(2, 3); - -private: - Location location; - std::string message; -}; - -class ParseErrors : public std::exception -{ -public: - ParseErrors(std::vector errors); - - virtual const char* what() const throw(); - - const std::vector& getErrors() const; - -private: - std::vector errors; - std::string message; -}; - template class TempVector { @@ -80,34 +50,17 @@ private: size_t size_; }; -struct Comment -{ - Lexeme::Type type; // Comment, BlockComment, or BrokenComment - Location location; -}; - -struct ParseResult -{ - AstStatBlock* root; - std::vector hotcomments; - std::vector errors; - - std::vector commentLocations; -}; - class Parser { public: static ParseResult parse( const char* buffer, std::size_t bufferSize, AstNameTable& names, Allocator& allocator, ParseOptions options = ParseOptions()); - static constexpr const char* errorName = "%error-id%"; - private: struct Name; struct Binding; - Parser(const char* buffer, std::size_t bufferSize, AstNameTable& names, Allocator& allocator); + Parser(const char* buffer, std::size_t bufferSize, AstNameTable& names, Allocator& allocator, const ParseOptions& options); bool blockFollow(const Lexeme& l); @@ -330,7 +283,7 @@ private: AstTypeError* reportTypeAnnotationError(const Location& location, const AstArray& types, bool isMissing, const char* format, ...) LUAU_PRINTF_ATTR(5, 6); - const Lexeme& nextLexeme(); + void nextLexeme(); struct Function { @@ -386,6 +339,9 @@ private: Allocator& allocator; std::vector commentLocations; + std::vector hotcomments; + + bool hotcommentHeader = true; unsigned int recursionCounter; diff --git a/Ast/src/Ast.cpp b/Ast/src/Ast.cpp index 9b5bc0c7..24a280da 100644 --- a/Ast/src/Ast.cpp +++ b/Ast/src/Ast.cpp @@ -167,8 +167,7 @@ AstExprFunction::AstExprFunction(const Location& location, const AstArrayreturnAnnotation = *returnAnnotation; } void AstExprFunction::visit(AstVisitor* visitor) @@ -195,8 +192,8 @@ void AstExprFunction::visit(AstVisitor* visitor) if (varargAnnotation) varargAnnotation->visit(visitor); - if (hasReturnAnnotation) - visitTypeList(visitor, returnAnnotation); + if (returnAnnotation) + visitTypeList(visitor, *returnAnnotation); body->visit(visitor); } @@ -375,21 +372,16 @@ void AstStatBlock::visit(AstVisitor* visitor) } } -AstStatIf::AstStatIf(const Location& location, AstExpr* condition, AstStatBlock* thenbody, AstStat* elsebody, bool hasThen, - const Location& thenLocation, const std::optional& elseLocation, bool hasEnd) +AstStatIf::AstStatIf(const Location& location, AstExpr* condition, AstStatBlock* thenbody, AstStat* elsebody, + const std::optional& thenLocation, const std::optional& elseLocation, bool hasEnd) : AstStat(ClassIndex(), location) , condition(condition) , thenbody(thenbody) , elsebody(elsebody) - , hasThen(hasThen) , thenLocation(thenLocation) + , elseLocation(elseLocation) , hasEnd(hasEnd) { - if (bool(elseLocation)) - { - hasElse = true; - this->elseLocation = *elseLocation; - } } void AstStatIf::visit(AstVisitor* visitor) @@ -492,12 +484,8 @@ AstStatLocal::AstStatLocal( : AstStat(ClassIndex(), location) , vars(vars) , values(values) + , equalsSignLocation(equalsSignLocation) { - if (bool(equalsSignLocation)) - { - hasEqualsSign = true; - this->equalsSignLocation = *equalsSignLocation; - } } void AstStatLocal::visit(AstVisitor* visitor) @@ -750,9 +738,8 @@ void AstStatError::visit(AstVisitor* visitor) AstTypeReference::AstTypeReference( const Location& location, std::optional prefix, AstName name, bool hasParameterList, const AstArray& parameters) : AstType(ClassIndex(), location) - , hasPrefix(bool(prefix)) , hasParameterList(hasParameterList) - , prefix(prefix ? *prefix : AstName()) + , prefix(prefix) , name(name) , parameters(parameters) { diff --git a/Ast/src/Lexer.cpp b/Ast/src/Lexer.cpp index a7aa24ca..d56c8860 100644 --- a/Ast/src/Lexer.cpp +++ b/Ast/src/Lexer.cpp @@ -101,11 +101,6 @@ Lexeme::Lexeme(const Location& location, Type type, const char* name) LUAU_ASSERT(type == Name || (type >= Reserved_BEGIN && type < Lexeme::Reserved_END)); } -static bool isComment(const Lexeme& lexeme) -{ - return lexeme.type == Lexeme::Comment || lexeme.type == Lexeme::BlockComment; -} - static const char* kReserved[] = {"and", "break", "do", "else", "elseif", "end", "false", "for", "function", "if", "in", "local", "nil", "not", "or", "repeat", "return", "then", "true", "until", "while"}; @@ -282,11 +277,6 @@ AstName AstNameTable::get(const char* name) const return getWithType(name, strlen(name)).first; } -inline bool isSpace(char ch) -{ - return ch == ' ' || ch == '\t' || ch == '\r' || ch == '\n' || ch == '\v' || ch == '\f'; -} - inline bool isAlpha(char ch) { // use or trick to convert to lower case and unsigned comparison to do range check @@ -372,7 +362,7 @@ const Lexeme& Lexer::next(bool skipComments) prevLocation = lexeme.location; lexeme = readNext(); - } while (skipComments && isComment(lexeme)); + } while (skipComments && (lexeme.type == Lexeme::Comment || lexeme.type == Lexeme::BlockComment)); return lexeme; } diff --git a/Ast/src/Parser.cpp b/Ast/src/Parser.cpp index 30b32f91..235d6349 100644 --- a/Ast/src/Parser.cpp +++ b/Ast/src/Parser.cpp @@ -13,18 +13,15 @@ LUAU_FASTINTVARIABLE(LuauParseErrorLimit, 100) LUAU_FASTFLAGVARIABLE(LuauParseSingletonTypes, false) LUAU_FASTFLAGVARIABLE(LuauParseTypeAliasDefaults, false) LUAU_FASTFLAGVARIABLE(LuauParseRecoverTypePackEllipsis, false) -LUAU_FASTFLAGVARIABLE(LuauStartingBrokenComment, false) +LUAU_FASTFLAGVARIABLE(LuauParseAllHotComments, false) +LUAU_FASTFLAGVARIABLE(LuauTableFieldFunctionDebugname, false) namespace Luau { -inline bool isSpace(char ch) -{ - return ch == ' ' || ch == '\t' || ch == '\r' || ch == '\n' || ch == '\v' || ch == '\f'; -} - static bool isComment(const Lexeme& lexeme) { + LUAU_ASSERT(!FFlag::LuauParseAllHotComments); return lexeme.type == Lexeme::Comment || lexeme.type == Lexeme::BlockComment; } @@ -151,31 +148,37 @@ ParseResult Parser::parse(const char* buffer, size_t bufferSize, AstNameTable& n { LUAU_TIMETRACE_SCOPE("Parser::parse", "Parser"); - Parser p(buffer, bufferSize, names, allocator); + Parser p(buffer, bufferSize, names, allocator, FFlag::LuauParseAllHotComments ? options : ParseOptions()); try { - std::vector hotcomments; - - while (isComment(p.lexer.current()) || p.lexer.current().type == Lexeme::BrokenComment) + if (FFlag::LuauParseAllHotComments) { - const char* text = p.lexer.current().data; - unsigned int length = p.lexer.current().length; + AstStatBlock* root = p.parseChunk(); - if (length && text[0] == '!') + return ParseResult{root, std::move(p.hotcomments), std::move(p.parseErrors), std::move(p.commentLocations)}; + } + else + { + std::vector hotcomments; + + while (isComment(p.lexer.current()) || p.lexer.current().type == Lexeme::BrokenComment) { - unsigned int end = length; - while (end > 0 && isSpace(text[end - 1])) - --end; + const char* text = p.lexer.current().data; + unsigned int length = p.lexer.current().length; - hotcomments.push_back(std::string(text + 1, text + end)); - } + if (length && text[0] == '!') + { + unsigned int end = length; + while (end > 0 && isSpace(text[end - 1])) + --end; - const Lexeme::Type type = p.lexer.current().type; - const Location loc = p.lexer.current().location; + hotcomments.push_back({true, p.lexer.current().location, std::string(text + 1, text + end)}); + } + + const Lexeme::Type type = p.lexer.current().type; + const Location loc = p.lexer.current().location; - if (FFlag::LuauStartingBrokenComment) - { if (options.captureComments) p.commentLocations.push_back(Comment{type, loc}); @@ -184,22 +187,15 @@ ParseResult Parser::parse(const char* buffer, size_t bufferSize, AstNameTable& n p.lexer.next(); } - else - { - p.lexer.next(); - if (options.captureComments) - p.commentLocations.push_back(Comment{type, loc}); - } + p.lexer.setSkipComments(true); + + p.options = options; + + AstStatBlock* root = p.parseChunk(); + + return ParseResult{root, hotcomments, p.parseErrors, std::move(p.commentLocations)}; } - - p.lexer.setSkipComments(true); - - p.options = options; - - AstStatBlock* root = p.parseChunk(); - - return ParseResult{root, hotcomments, p.parseErrors, std::move(p.commentLocations)}; } catch (ParseError& err) { @@ -210,8 +206,9 @@ ParseResult Parser::parse(const char* buffer, size_t bufferSize, AstNameTable& n } } -Parser::Parser(const char* buffer, size_t bufferSize, AstNameTable& names, Allocator& allocator) - : lexer(buffer, bufferSize, names) +Parser::Parser(const char* buffer, size_t bufferSize, AstNameTable& names, Allocator& allocator, const ParseOptions& options) + : options(options) + , lexer(buffer, bufferSize, names) , allocator(allocator) , recursionCounter(0) , endMismatchSuspect(Location(), Lexeme::Eof) @@ -224,14 +221,20 @@ Parser::Parser(const char* buffer, size_t bufferSize, AstNameTable& names, Alloc nameSelf = names.addStatic("self"); nameNumber = names.addStatic("number"); - nameError = names.addStatic(errorName); + nameError = names.addStatic(kParseNameError); nameNil = names.getOrAdd("nil"); // nil is a reserved keyword matchRecoveryStopOnToken.assign(Lexeme::Type::Reserved_END, 0); matchRecoveryStopOnToken[Lexeme::Type::Eof] = 1; + if (FFlag::LuauParseAllHotComments) + lexer.setSkipComments(true); + // read first lexeme nextLexeme(); + + // all hot comments parsed after the first non-comment lexeme are special in that they don't affect type checking / linting mode + hotcommentHeader = false; } bool Parser::blockFollow(const Lexeme& l) @@ -396,7 +399,9 @@ AstStat* Parser::parseIf() AstExpr* cond = parseExpr(); Lexeme matchThen = lexer.current(); - bool hasThen = expectAndConsume(Lexeme::ReservedThen, "if statement"); + std::optional thenLocation; + if (expectAndConsume(Lexeme::ReservedThen, "if statement")) + thenLocation = matchThen.location; AstStatBlock* thenbody = parseBlock(); @@ -434,7 +439,7 @@ AstStat* Parser::parseIf() hasEnd = expectMatchEndAndConsume(Lexeme::ReservedEnd, matchThenElse); } - return allocator.alloc(Location(start, end), cond, thenbody, elsebody, hasThen, matchThen.location, elseLocation, hasEnd); + return allocator.alloc(Location(start, end), cond, thenbody, elsebody, thenLocation, elseLocation, hasEnd); } // while exp do block end @@ -769,7 +774,7 @@ AstStat* Parser::parseTypeAlias(const Location& start, bool exported) { // note: `type` token is already parsed for us, so we just need to parse the rest - auto name = parseNameOpt("type name"); + std::optional name = parseNameOpt("type name"); // Use error name if the name is missing if (!name) @@ -925,7 +930,7 @@ AstStat* Parser::parseDeclaration(const Location& start) return allocator.alloc(Location(classStart, classEnd), className.name, superName, copy(props)); } - else if (auto globalName = parseNameOpt("global variable name")) + else if (std::optional globalName = parseNameOpt("global variable name")) { expectAndConsume(':', "global variable declaration"); @@ -1066,7 +1071,7 @@ void Parser::parseExprList(TempVector& result) Parser::Binding Parser::parseBinding() { - auto name = parseNameOpt("variable name"); + std::optional name = parseNameOpt("variable name"); // Use placeholder if the name is missing if (!name) @@ -1325,7 +1330,7 @@ AstType* Parser::parseTableTypeAnnotation() } else { - auto name = parseNameOpt("table field"); + std::optional name = parseNameOpt("table field"); if (!name) break; @@ -1422,7 +1427,7 @@ AstType* Parser::parseFunctionTypeAnnotationTail(const Lexeme& begin, AstArray(Location(begin.location, endLocation), generics, genericPacks, paramTypes, paramNames, returnTypeList); @@ -1869,7 +1874,7 @@ AstExpr* Parser::parseExpr(unsigned int limit) // NAME AstExpr* Parser::parseNameExpr(const char* context) { - auto name = parseNameOpt(context); + std::optional name = parseNameOpt(context); if (!name) return allocator.alloc(lexer.current().location, copy({}), unsigned(parseErrors.size() - 1)); @@ -2233,6 +2238,12 @@ AstExpr* Parser::parseTableConstructor() AstExpr* key = allocator.alloc(name.location, nameString); AstExpr* value = parseExpr(); + if (FFlag::LuauTableFieldFunctionDebugname) + { + if (AstExprFunction* func = value->as()) + func->debugname = name.name; + } + items.push_back({AstExprTable::Item::Record, key, value}); } else @@ -2313,7 +2324,7 @@ std::optional Parser::parseNameOpt(const char* context) Parser::Name Parser::parseName(const char* context) { - if (auto name = parseNameOpt(context)) + if (std::optional name = parseNameOpt(context)) return *name; Location location = lexer.current().location; @@ -2324,7 +2335,7 @@ Parser::Name Parser::parseName(const char* context) Parser::Name Parser::parseIndexName(const char* context, const Position& previous) { - if (auto name = parseNameOpt(context)) + if (std::optional name = parseNameOpt(context)) return *name; // If we have a reserved keyword next at the same line, assume it's an incomplete name @@ -2379,7 +2390,7 @@ std::pair, AstArray> Parser::parseG if (shouldParseTypePackAnnotation(lexer)) { - auto typePack = parseTypePackAnnotation(); + AstTypePack* typePack = parseTypePackAnnotation(); namePacks.push_back({name, nameLocation, typePack}); } @@ -2451,7 +2462,7 @@ AstArray Parser::parseTypeParams() { if (shouldParseTypePackAnnotation(lexer)) { - auto typePack = parseTypePackAnnotation(); + AstTypePack* typePack = parseTypePackAnnotation(); parameters.push_back({{}, typePack}); } @@ -2821,25 +2832,57 @@ AstTypeError* Parser::reportTypeAnnotationError(const Location& location, const return allocator.alloc(location, types, isMissing, unsigned(parseErrors.size() - 1)); } -const Lexeme& Parser::nextLexeme() +void Parser::nextLexeme() { if (options.captureComments) { - while (true) + if (FFlag::LuauParseAllHotComments) { - const Lexeme& lexeme = lexer.next(/*skipComments*/ false); - // Subtlety: Broken comments are weird because we record them as comments AND pass them to the parser as a lexeme. - // The parser will turn this into a proper syntax error. - if (lexeme.type == Lexeme::BrokenComment) + Lexeme::Type type = lexer.next(/* skipComments= */ false).type; + + while (type == Lexeme::BrokenComment || type == Lexeme::Comment || type == Lexeme::BlockComment) + { + const Lexeme& lexeme = lexer.current(); commentLocations.push_back(Comment{lexeme.type, lexeme.location}); - if (isComment(lexeme)) - commentLocations.push_back(Comment{lexeme.type, lexeme.location}); - else - return lexeme; + + // Subtlety: Broken comments are weird because we record them as comments AND pass them to the parser as a lexeme. + // The parser will turn this into a proper syntax error. + if (lexeme.type == Lexeme::BrokenComment) + return; + + // Comments starting with ! are called "hot comments" and contain directives for type checking / linting + if (lexeme.type == Lexeme::Comment && lexeme.length && lexeme.data[0] == '!') + { + const char* text = lexeme.data; + + unsigned int end = lexeme.length; + while (end > 0 && isSpace(text[end - 1])) + --end; + + hotcomments.push_back({hotcommentHeader, lexeme.location, std::string(text + 1, text + end)}); + } + + type = lexer.next(/* skipComments= */ false).type; + } + } + else + { + while (true) + { + const Lexeme& lexeme = lexer.next(/*skipComments*/ false); + // Subtlety: Broken comments are weird because we record them as comments AND pass them to the parser as a lexeme. + // The parser will turn this into a proper syntax error. + if (lexeme.type == Lexeme::BrokenComment) + commentLocations.push_back(Comment{lexeme.type, lexeme.location}); + if (isComment(lexeme)) + commentLocations.push_back(Comment{lexeme.type, lexeme.location}); + else + return; + } } } else - return lexer.next(); + lexer.next(); } } // namespace Luau diff --git a/Ast/src/TimeTrace.cpp b/Ast/src/TimeTrace.cpp index ded50e53..8079830b 100644 --- a/Ast/src/TimeTrace.cpp +++ b/Ast/src/TimeTrace.cpp @@ -9,6 +9,12 @@ #include #ifdef _WIN32 +#ifndef WIN32_LEAN_AND_MEAN +#define WIN32_LEAN_AND_MEAN +#endif +#ifndef NOMINMAX +#define NOMINMAX +#endif #include #endif diff --git a/CLI/FileUtils.cpp b/CLI/FileUtils.cpp index fe005aec..fb6ac373 100644 --- a/CLI/FileUtils.cpp +++ b/CLI/FileUtils.cpp @@ -4,8 +4,13 @@ #include "Luau/Common.h" #ifdef _WIN32 +#ifndef WIN32_LEAN_AND_MEAN #define WIN32_LEAN_AND_MEAN -#include +#endif +#ifndef NOMINMAX +#define NOMINMAX +#endif +#include #else #include #include diff --git a/CLI/Repl.cpp b/CLI/Repl.cpp index 9a6e25c2..13304d57 100644 --- a/CLI/Repl.cpp +++ b/CLI/Repl.cpp @@ -37,6 +37,8 @@ enum class CompileFormat Binary }; +constexpr int MaxTraversalLimit = 50; + struct GlobalOptions { int optimizationLevel = 1; @@ -243,10 +245,115 @@ std::string runCode(lua_State* L, const std::string& source) return std::string(); } +// Replaces the top of the lua stack with the metatable __index for the value +// if it exists. Returns true iff __index exists. +static bool tryReplaceTopWithIndex(lua_State* L) +{ + if (luaL_getmetafield(L, -1, "__index")) + { + // Remove the table leaving __index on the top of stack + lua_remove(L, -2); + return true; + } + return false; +} + + +// This function is similar to lua_gettable, but it avoids calling any +// lua callback functions (e.g. __index) which might modify the Lua VM state. +static void safeGetTable(lua_State* L, int tableIndex) +{ + lua_pushvalue(L, tableIndex); // Duplicate the table + + // The loop invariant is that the table to search is at -1 + // and the key is at -2. + for (int loopCount = 0;; loopCount++) + { + lua_pushvalue(L, -2); // Duplicate the key + lua_rawget(L, -2); // Try to find the key + if (!lua_isnil(L, -1) || loopCount >= MaxTraversalLimit) + { + // Either the key has been found, and/or we have reached the max traversal limit + break; + } + else + { + lua_pop(L, 1); // Pop the nil result + if (!luaL_getmetafield(L, -1, "__index")) + { + lua_pushnil(L); + break; + } + else if (lua_istable(L, -1)) + { + // Replace the current table being searched with __index table + lua_replace(L, -2); + } + else + { + lua_pop(L, 1); // Pop the value + lua_pushnil(L); + break; + } + } + } + + lua_remove(L, -2); // Remove the table + lua_remove(L, -2); // Remove the original key +} + +// completePartialMatches finds keys that match the specified 'prefix' +// Note: the table/object to be searched must be on the top of the Lua stack +static void completePartialMatches(lua_State* L, bool completeOnlyFunctions, const std::string& editBuffer, std::string_view prefix, + const AddCompletionCallback& addCompletionCallback) +{ + for (int i = 0; i < MaxTraversalLimit && lua_istable(L, -1); i++) + { + // table, key + lua_pushnil(L); + + // Loop over all the keys in the current table + while (lua_next(L, -2) != 0) + { + if (lua_type(L, -2) == LUA_TSTRING) + { + // table, key, value + std::string_view key = lua_tostring(L, -2); + int valueType = lua_type(L, -1); + + // If the last separator was a ':' (i.e. a method call) then only functions should be completed. + bool requiredValueType = (!completeOnlyFunctions || valueType == LUA_TFUNCTION); + + if (!key.empty() && requiredValueType && Luau::startsWith(key, prefix)) + { + std::string completedComponent(key.substr(prefix.size())); + std::string completion(editBuffer + completedComponent); + if (valueType == LUA_TFUNCTION) + { + // Add an opening paren for function calls by default. + completion += "("; + } + addCompletionCallback(completion, std::string(key)); + } + } + lua_pop(L, 1); + } + + // Replace the current table being searched with an __index table if one exists + if (!tryReplaceTopWithIndex(L)) + { + break; + } + } +} + static void completeIndexer(lua_State* L, const std::string& editBuffer, const AddCompletionCallback& addCompletionCallback) { std::string_view lookup = editBuffer; - char lastSep = 0; + bool completeOnlyFunctions = false; + + // Push the global variable table to begin the search + lua_pushvalue(L, LUA_GLOBALSINDEX); for (;;) { @@ -255,60 +362,26 @@ static void completeIndexer(lua_State* L, const std::string& editBuffer, const A if (sep == std::string_view::npos) { - // table, key - lua_pushnil(L); - - while (lua_next(L, -2) != 0) - { - if (lua_type(L, -2) == LUA_TSTRING) - { - // table, key, value - std::string_view key = lua_tostring(L, -2); - int valueType = lua_type(L, -1); - - // If the last separator was a ':' (i.e. a method call) then only functions should be completed. - bool requiredValueType = (lastSep != ':' || valueType == LUA_TFUNCTION); - - if (!key.empty() && requiredValueType && Luau::startsWith(key, prefix)) - { - std::string completedComponent(key.substr(prefix.size())); - std::string completion(editBuffer + completedComponent); - if (valueType == LUA_TFUNCTION) - { - // Add an opening paren for function calls by default. - completion += "("; - } - addCompletionCallback(completion, std::string(key)); - } - } - lua_pop(L, 1); - } - + completePartialMatches(L, completeOnlyFunctions, editBuffer, prefix, addCompletionCallback); break; } else { // find the key in the table lua_pushlstring(L, prefix.data(), prefix.size()); - lua_rawget(L, -2); + safeGetTable(L, -2); lua_remove(L, -2); - if (lua_type(L, -1) == LUA_TSTRING) + if (lua_istable(L, -1) || tryReplaceTopWithIndex(L)) { - // Replace the string object with the string class to perform further lookups of string functions - // Note: We retrieve the string class from _G to prevent issues if the user assigns to `string`. - lua_pop(L, 1); // Pop the string instance - lua_getglobal(L, "_G"); - lua_pushlstring(L, "string", 6); - lua_rawget(L, -2); - lua_remove(L, -2); // Remove the global table - LUAU_ASSERT(lua_istable(L, -1)); + completeOnlyFunctions = lookup[sep] == ':'; + lookup.remove_prefix(sep + 1); } - else if (!lua_istable(L, -1)) + else + { + // Unable to search for keys, so stop searching break; - - lastSep = lookup[sep]; - lookup.remove_prefix(sep + 1); + } } } @@ -317,12 +390,6 @@ static void completeIndexer(lua_State* L, const std::string& editBuffer, const A void getCompletions(lua_State* L, const std::string& editBuffer, const AddCompletionCallback& addCompletionCallback) { - // look the value up in current global table first - lua_pushvalue(L, LUA_GLOBALSINDEX); - completeIndexer(L, editBuffer, addCompletionCallback); - - // and in actual global table after that - lua_getglobal(L, "_G"); completeIndexer(L, editBuffer, addCompletionCallback); } @@ -365,9 +432,7 @@ struct LinenoiseScopedHistory ic_set_history(historyFilepath.c_str(), -1 /* default entries (= 200) */); } - ~LinenoiseScopedHistory() - { - } + ~LinenoiseScopedHistory() {} std::string historyFilepath; }; diff --git a/CMakeLists.txt b/CMakeLists.txt index c19d2b40..c6ccebc5 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -104,6 +104,20 @@ if (MSVC AND MSVC_VERSION GREATER_EQUAL 1924) set_source_files_properties(VM/src/lvmexecute.cpp PROPERTIES COMPILE_FLAGS /d2ssa-pre-) endif() +# embed .natvis inside the library debug information +if(MSVC) + target_link_options(Luau.Ast INTERFACE /NATVIS:${CMAKE_CURRENT_SOURCE_DIR}/tools/natvis/Ast.natvis) + target_link_options(Luau.Analysis INTERFACE /NATVIS:${CMAKE_CURRENT_SOURCE_DIR}/tools/natvis/Analysis.natvis) + target_link_options(Luau.VM INTERFACE /NATVIS:${CMAKE_CURRENT_SOURCE_DIR}/tools/natvis/VM.natvis) +endif() + +# make .natvis visible inside the solution +if(MSVC_IDE) + target_sources(Luau.Ast PRIVATE tools/natvis/Ast.natvis) + target_sources(Luau.Analysis PRIVATE tools/natvis/Analysis.natvis) + target_sources(Luau.VM PRIVATE tools/natvis/VM.natvis) +endif() + if(LUAU_BUILD_CLI) target_compile_options(Luau.Repl.CLI PRIVATE ${LUAU_OPTIONS}) target_compile_options(Luau.Analyze.CLI PRIVATE ${LUAU_OPTIONS}) diff --git a/Sources.cmake b/Sources.cmake index 773f6f35..615641eb 100644 --- a/Sources.cmake +++ b/Sources.cmake @@ -8,6 +8,7 @@ target_sources(Luau.Ast PRIVATE Ast/include/Luau/Location.h Ast/include/Luau/ParseOptions.h Ast/include/Luau/Parser.h + Ast/include/Luau/ParseResult.h Ast/include/Luau/StringUtils.h Ast/include/Luau/TimeTrace.h diff --git a/VM/src/lapi.cpp b/VM/src/lapi.cpp index 5cffba63..29d5f397 100644 --- a/VM/src/lapi.cpp +++ b/VM/src/lapi.cpp @@ -36,12 +36,9 @@ const char* luau_ident = "$Luau: Copyright (C) 2019-2022 Roblox Corporation $\n" static Table* getcurrenv(lua_State* L) { if (L->ci == L->base_ci) /* no enclosing function? */ - return hvalue(gt(L)); /* use global table as environment */ + return L->gt; /* use global table as environment */ else - { - Closure* func = curr_func(L); - return func->env; - } + return curr_func(L)->env; } static LUAU_NOINLINE TValue* pseudo2addr(lua_State* L, int idx) @@ -53,11 +50,14 @@ static LUAU_NOINLINE TValue* pseudo2addr(lua_State* L, int idx) return registry(L); case LUA_ENVIRONINDEX: { - sethvalue(L, &L->env, getcurrenv(L)); - return &L->env; + sethvalue(L, &L->global->pseudotemp, getcurrenv(L)); + return &L->global->pseudotemp; } case LUA_GLOBALSINDEX: - return gt(L); + { + sethvalue(L, &L->global->pseudotemp, L->gt); + return &L->global->pseudotemp; + } default: { Closure* func = curr_func(L); @@ -237,6 +237,11 @@ void lua_replace(lua_State* L, int idx) func->env = hvalue(L->top - 1); luaC_barrier(L, func, L->top - 1); } + else if (idx == LUA_GLOBALSINDEX) + { + api_check(L, ttistable(L->top - 1)); + L->gt = hvalue(L->top - 1); + } else { setobj(L, o, L->top - 1); @@ -783,7 +788,7 @@ void lua_getfenv(lua_State* L, int idx) sethvalue(L, L->top, clvalue(o)->env); break; case LUA_TTHREAD: - setobj2s(L, L->top, gt(thvalue(o))); + sethvalue(L, L->top, thvalue(o)->gt); break; default: setnilvalue(L->top); @@ -914,7 +919,7 @@ int lua_setfenv(lua_State* L, int idx) clvalue(o)->env = hvalue(L->top - 1); break; case LUA_TTHREAD: - sethvalue(L, gt(thvalue(o)), hvalue(L->top - 1)); + thvalue(o)->gt = hvalue(L->top - 1); break; default: res = 0; diff --git a/VM/src/ldebug.cpp b/VM/src/ldebug.cpp index e9930f7a..a4f93c62 100644 --- a/VM/src/ldebug.cpp +++ b/VM/src/ldebug.cpp @@ -419,7 +419,7 @@ static void getcoverage(Proto* p, int depth, int* buffer, size_t size, void* con } const char* debugname = p->debugname ? getstr(p->debugname) : NULL; - int linedefined = luaG_getline(p, 0); + int linedefined = getlinedefined(p); callback(context, debugname, linedefined, depth, buffer, size); diff --git a/VM/src/ldo.cpp b/VM/src/ldo.cpp index a3982bc6..d87f0661 100644 --- a/VM/src/ldo.cpp +++ b/VM/src/ldo.cpp @@ -17,6 +17,8 @@ #include +LUAU_FASTFLAG(LuauReduceStackReallocs) + /* ** {====================================================== ** Error-recovery functions @@ -164,13 +166,14 @@ static void correctstack(lua_State* L, TValue* oldstack) void luaD_reallocstack(lua_State* L, int newsize) { TValue* oldstack = L->stack; - int realsize = newsize + 1 + EXTRA_STACK; - LUAU_ASSERT(L->stack_last - L->stack == L->stacksize - EXTRA_STACK - 1); + int realsize = newsize + (FFlag::LuauReduceStackReallocs ? EXTRA_STACK : 1 + EXTRA_STACK); + LUAU_ASSERT(L->stack_last - L->stack == L->stacksize - (FFlag::LuauReduceStackReallocs ? EXTRA_STACK : 1 + EXTRA_STACK)); luaM_reallocarray(L, L->stack, L->stacksize, realsize, TValue, L->memcat); + TValue* newstack = L->stack; for (int i = L->stacksize; i < realsize; i++) - setnilvalue(L->stack + i); /* erase new segment */ + setnilvalue(newstack + i); /* erase new segment */ L->stacksize = realsize; - L->stack_last = L->stack + newsize; + L->stack_last = newstack + newsize; correctstack(L, oldstack); } @@ -512,7 +515,7 @@ static void callerrfunc(lua_State* L, void* ud) static void restore_stack_limit(lua_State* L) { - LUAU_ASSERT(L->stack_last - L->stack == L->stacksize - EXTRA_STACK - 1); + LUAU_ASSERT(L->stack_last - L->stack == L->stacksize - (FFlag::LuauReduceStackReallocs ? EXTRA_STACK : 1 + EXTRA_STACK)); if (L->size_ci > LUAI_MAXCALLS) { /* there was an overflow? */ int inuse = cast_int(L->ci - L->base_ci); diff --git a/VM/src/ldo.h b/VM/src/ldo.h index 72807f0f..1c1480d6 100644 --- a/VM/src/ldo.h +++ b/VM/src/ldo.h @@ -11,7 +11,7 @@ if ((char*)L->stack_last - (char*)L->top <= (n) * (int)sizeof(TValue)) \ luaD_growstack(L, n); \ else \ - condhardstacktests(luaD_reallocstack(L, L->stacksize - EXTRA_STACK - 1)); + condhardstacktests(luaD_reallocstack(L, L->stacksize - (FFlag::LuauReduceStackReallocs ? EXTRA_STACK : 1 + EXTRA_STACK))); #define incr_top(L) \ { \ diff --git a/VM/src/lgc.cpp b/VM/src/lgc.cpp index 835572fa..724b24b2 100644 --- a/VM/src/lgc.cpp +++ b/VM/src/lgc.cpp @@ -268,7 +268,7 @@ static void traverseclosure(global_State* g, Closure* cl) static void traversestack(global_State* g, lua_State* l, bool clearstack) { - markvalue(g, gt(l)); + markobject(g, l->gt); if (l->namecall) stringmark(l->namecall); for (StkId o = l->stack; o < l->top; o++) @@ -643,7 +643,7 @@ static void markroot(lua_State* L) g->weak = NULL; markobject(g, g->mainthread); /* make global table be traversed before main stack */ - markvalue(g, gt(g->mainthread)); + markobject(g, g->mainthread->gt); markvalue(g, registry(L)); markmt(g); g->gcstate = GCSpropagate; diff --git a/VM/src/lgc.h b/VM/src/lgc.h index 528d0944..2acb5d8a 100644 --- a/VM/src/lgc.h +++ b/VM/src/lgc.h @@ -77,7 +77,7 @@ #define luaC_checkGC(L) \ { \ - condhardstacktests(luaD_reallocstack(L, L->stacksize - EXTRA_STACK - 1)); \ + condhardstacktests(luaD_reallocstack(L, L->stacksize - (FFlag::LuauReduceStackReallocs ? EXTRA_STACK : 1 + EXTRA_STACK))); \ if (L->global->totalbytes >= L->global->GCthreshold) \ { \ condhardmemtests(luaC_validate(L), 1); \ diff --git a/VM/src/lgcdebug.cpp b/VM/src/lgcdebug.cpp index ce196520..30242e52 100644 --- a/VM/src/lgcdebug.cpp +++ b/VM/src/lgcdebug.cpp @@ -88,7 +88,7 @@ static void validateclosure(global_State* g, Closure* cl) static void validatestack(global_State* g, lua_State* l) { - validateref(g, obj2gco(l), gt(l)); + validateobjref(g, obj2gco(l), obj2gco(l->gt)); for (CallInfo* ci = l->base_ci; ci <= l->ci; ++ci) { @@ -370,6 +370,7 @@ static void dumpclosure(FILE* f, Closure* cl) fprintf(f, ",\"env\":"); dumpref(f, obj2gco(cl->env)); + if (cl->isC) { if (cl->nupvalues) @@ -411,11 +412,8 @@ static void dumpthread(FILE* f, lua_State* th) fprintf(f, "{\"type\":\"thread\",\"cat\":%d,\"size\":%d", th->memcat, int(size)); - if (iscollectable(&th->l_gt)) - { - fprintf(f, ",\"env\":"); - dumpref(f, gcvalue(&th->l_gt)); - } + fprintf(f, ",\"env\":"); + dumpref(f, obj2gco(th->gt)); Closure* tcl = 0; for (CallInfo* ci = th->base_ci; ci <= th->ci; ++ci) diff --git a/VM/src/lperf.cpp b/VM/src/lperf.cpp index 2f6c7297..da68e376 100644 --- a/VM/src/lperf.cpp +++ b/VM/src/lperf.cpp @@ -3,6 +3,12 @@ #include "lua.h" #ifdef _WIN32 +#ifndef WIN32_LEAN_AND_MEAN +#define WIN32_LEAN_AND_MEAN +#endif +#ifndef NOMINMAX +#define NOMINMAX +#endif #include #endif diff --git a/VM/src/lstate.cpp b/VM/src/lstate.cpp index 6762c638..d6d127c0 100644 --- a/VM/src/lstate.cpp +++ b/VM/src/lstate.cpp @@ -11,6 +11,7 @@ #include "ldebug.h" LUAU_FASTFLAG(LuauGcPagedSweep) +LUAU_FASTFLAGVARIABLE(LuauReduceStackReallocs, false) /* ** Main thread combines a thread state and the global state @@ -31,10 +32,11 @@ static void stack_init(lua_State* L1, lua_State* L) /* initialize stack array */ L1->stack = luaM_newarray(L, BASIC_STACK_SIZE + EXTRA_STACK, TValue, L1->memcat); L1->stacksize = BASIC_STACK_SIZE + EXTRA_STACK; + TValue* stack = L1->stack; for (int i = 0; i < BASIC_STACK_SIZE + EXTRA_STACK; i++) - setnilvalue(L1->stack + i); /* erase new stack */ - L1->top = L1->stack; - L1->stack_last = L1->stack + (L1->stacksize - EXTRA_STACK) - 1; + setnilvalue(stack + i); /* erase new stack */ + L1->top = stack; + L1->stack_last = stack + (L1->stacksize - (FFlag::LuauReduceStackReallocs ? EXTRA_STACK : 1 + EXTRA_STACK)); /* initialize first ci */ L1->ci->func = L1->top; setnilvalue(L1->top++); /* `function' entry for this `ci' */ @@ -55,7 +57,7 @@ static void f_luaopen(lua_State* L, void* ud) { global_State* g = L->global; stack_init(L, L); /* init stack */ - sethvalue(L, gt(L), luaH_new(L, 0, 2)); /* table of globals */ + L->gt = luaH_new(L, 0, 2); /* table of globals */ sethvalue(L, registry(L), luaH_new(L, 0, 2)); /* registry */ luaS_resize(L, LUA_MINSTRTABSIZE); /* initial size of string table */ luaT_init(L); @@ -69,6 +71,7 @@ static void preinit_state(lua_State* L, global_State* g) L->global = g; L->stack = NULL; L->stacksize = 0; + L->gt = NULL; L->openupval = NULL; L->size_ci = 0; L->nCcalls = L->baseCcalls = 0; @@ -80,7 +83,6 @@ static void preinit_state(lua_State* L, global_State* g) L->stackstate = 0; L->activememcat = 0; L->userdata = NULL; - setnilvalue(gt(L)); } static void close_state(lua_State* L) @@ -116,7 +118,7 @@ lua_State* luaE_newthread(lua_State* L) preinit_state(L1, L->global); L1->activememcat = L->activememcat; // inherit the active memory category stack_init(L1, L); /* init stack */ - setobj2n(L, gt(L1), gt(L)); /* share table of globals */ + L1->gt = L->gt; /* share table of globals */ L1->singlestep = L->singlestep; LUAU_ASSERT(iswhite(obj2gco(L1))); return L1; @@ -144,14 +146,30 @@ void lua_resetthread(lua_State* L) ci->top = ci->base + LUA_MINSTACK; setnilvalue(ci->func); L->ci = ci; - luaD_reallocCI(L, BASIC_CI_SIZE); + if (FFlag::LuauReduceStackReallocs) + { + if (L->size_ci != BASIC_CI_SIZE) + luaD_reallocCI(L, BASIC_CI_SIZE); + } + else + { + luaD_reallocCI(L, BASIC_CI_SIZE); + } /* clear thread state */ L->status = LUA_OK; L->base = L->ci->base; L->top = L->ci->base; L->nCcalls = L->baseCcalls = 0; /* clear thread stack */ - luaD_reallocstack(L, BASIC_STACK_SIZE); + if (FFlag::LuauReduceStackReallocs) + { + if (L->stacksize != BASIC_STACK_SIZE + EXTRA_STACK) + luaD_reallocstack(L, BASIC_STACK_SIZE); + } + else + { + luaD_reallocstack(L, BASIC_STACK_SIZE); + } for (int i = 0; i < L->stacksize; i++) setnilvalue(L->stack + i); } @@ -193,6 +211,7 @@ lua_State* lua_newstate(lua_Alloc f, void* ud) g->strt.size = 0; g->strt.nuse = 0; g->strt.hash = NULL; + setnilvalue(&g->pseudotemp); setnilvalue(registry(L)); g->gcstate = GCSpause; if (!FFlag::LuauGcPagedSweep) diff --git a/VM/src/lstate.h b/VM/src/lstate.h index 0708b71f..6dd89138 100644 --- a/VM/src/lstate.h +++ b/VM/src/lstate.h @@ -5,9 +5,6 @@ #include "lobject.h" #include "ltm.h" -/* table of globals */ -#define gt(L) (&L->l_gt) - /* registry */ #define registry(L) (&L->global->registry) @@ -177,6 +174,8 @@ typedef struct global_State TString* ttname[LUA_T_COUNT]; /* names for basic types */ TString* tmname[TM_N]; /* array with tag-method names */ + TValue pseudotemp; /* storage for temporary values used in pseudo2addr */ + TValue registry; /* registry table, used by lua_ref and LUA_REGISTRYINDEX */ int registryfree; /* next free slot in registry */ @@ -231,8 +230,7 @@ struct lua_State int cachedslot; /* when table operations or INDEX/NEWINDEX is invoked from Luau, what is the expected slot for lookup? */ - TValue l_gt; /* table of globals */ - TValue env; /* temporary place for environments */ + Table* gt; /* table of globals */ UpVal* openupval; /* list of open upvalues in this stack */ GCObject* gclist; diff --git a/VM/src/lvmexecute.cpp b/VM/src/lvmexecute.cpp index c3b662a2..6c31d36f 100644 --- a/VM/src/lvmexecute.cpp +++ b/VM/src/lvmexecute.cpp @@ -39,7 +39,7 @@ // When calling luau_callTM, we usually push the arguments to the top of the stack. // This is safe to do for complicated reasons: -// - stack guarantees 1 + EXTRA_STACK room beyond stack_last (see luaD_reallocstack) +// - stack guarantees EXTRA_STACK room beyond stack_last (see luaD_reallocstack) // - stack reallocation copies values past stack_last // All external function calls that can cause stack realloc or Lua calls have to be wrapped in VM_PROTECT diff --git a/VM/src/lvmload.cpp b/VM/src/lvmload.cpp index 2472cd90..4e5435b7 100644 --- a/VM/src/lvmload.cpp +++ b/VM/src/lvmload.cpp @@ -116,12 +116,12 @@ static void resolveImportSafe(lua_State* L, Table* env, TValue* k, uint32_t id) // note: we call getimport with nil propagation which means that accesses to table chains like A.B.C will resolve in nil // this is technically not necessary but it reduces the number of exceptions when loading scripts that rely on getfenv/setfenv for global // injection - luaV_getimport(L, hvalue(gt(L)), self->k, self->id, /* propagatenil= */ true); + luaV_getimport(L, L->gt, self->k, self->id, /* propagatenil= */ true); } }; ResolveImport ri = {k, id}; - if (hvalue(gt(L))->safeenv) + if (L->gt->safeenv) { // luaD_pcall will make sure that if any C/Lua calls during import resolution fail, the thread state is restored back int oldTop = lua_gettop(L); @@ -171,7 +171,7 @@ int luau_load(lua_State* L, const char* chunkname, const char* data, size_t size L->global->GCthreshold = SIZE_MAX; // env is 0 for current environment and a stack index otherwise - Table* envt = (env == 0) ? hvalue(gt(L)) : hvalue(luaA_toobject(L, env)); + Table* envt = (env == 0) ? L->gt : hvalue(luaA_toobject(L, env)); TString* source = luaS_new(L, chunkname); diff --git a/VM/src/lvmutils.cpp b/VM/src/lvmutils.cpp index 31dd59c8..8a18a4d4 100644 --- a/VM/src/lvmutils.cpp +++ b/VM/src/lvmutils.cpp @@ -55,7 +55,7 @@ static void callTMres(lua_State* L, StkId res, const TValue* f, const TValue* p1 { ptrdiff_t result = savestack(L, res); // using stack room beyond top is technically safe here, but for very complicated reasons: - // * The stack guarantees 1 + EXTRA_STACK room beyond stack_last (see luaD_reallocstack) will be allocated + // * The stack guarantees EXTRA_STACK room beyond stack_last (see luaD_reallocstack) will be allocated // * we cannot move luaD_checkstack above because the arguments are *sometimes* pointers to the lua // stack and checkstack may invalidate those pointers // * we cannot use savestack/restorestack because the arguments are sometimes on the C++ stack @@ -76,7 +76,7 @@ static void callTMres(lua_State* L, StkId res, const TValue* f, const TValue* p1 static void callTM(lua_State* L, const TValue* f, const TValue* p1, const TValue* p2, const TValue* p3) { // using stack room beyond top is technically safe here, but for very complicated reasons: - // * The stack guarantees 1 + EXTRA_STACK room beyond stack_last (see luaD_reallocstack) will be allocated + // * The stack guarantees EXTRA_STACK room beyond stack_last (see luaD_reallocstack) will be allocated // * we cannot move luaD_checkstack above because the arguments are *sometimes* pointers to the lua // stack and checkstack may invalidate those pointers // * we cannot use savestack/restorestack because the arguments are sometimes on the C++ stack diff --git a/fuzz/linter.cpp b/fuzz/linter.cpp index 55e0888b..04638d23 100644 --- a/fuzz/linter.cpp +++ b/fuzz/linter.cpp @@ -32,7 +32,7 @@ extern "C" int LLVMFuzzerTestOneInput(const uint8_t* Data, size_t Size) Luau::LintOptions lintOptions; lintOptions.warningMask = ~0ull; - Luau::lint(parseResult.root, names, typeck.globalScope, nullptr, lintOptions); + Luau::lint(parseResult.root, names, typeck.globalScope, nullptr, {}, lintOptions); } return 0; diff --git a/fuzz/proto.cpp b/fuzz/proto.cpp index 27e53492..f407248a 100644 --- a/fuzz/proto.cpp +++ b/fuzz/proto.cpp @@ -227,7 +227,7 @@ DEFINE_PROTO_FUZZER(const luau::StatBlock& message) if (kFuzzLinter) { Luau::LintOptions lintOptions = {~0u}; - Luau::lint(parseResult.root, names, sharedEnv.globalScope, module.get(), lintOptions); + Luau::lint(parseResult.root, names, sharedEnv.globalScope, module.get(), {}, lintOptions); } } diff --git a/tests/Autocomplete.test.cpp b/tests/Autocomplete.test.cpp index 59e12574..1978a0d3 100644 --- a/tests/Autocomplete.test.cpp +++ b/tests/Autocomplete.test.cpp @@ -1,7 +1,6 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #include "Luau/Autocomplete.h" #include "Luau/BuiltinDefinitions.h" -#include "Luau/Parser.h" #include "Luau/TypeInfer.h" #include "Luau/TypeVar.h" #include "Luau/VisitTypeVar.h" @@ -2610,6 +2609,27 @@ a = if temp then even elseif true then temp else e@9 CHECK(ac.entryMap.count("elseif") == 0); } +TEST_CASE_FIXTURE(ACFixture, "autocomplete_if_else_regression") +{ + ScopedFastFlag FFlagLuauIfElseExprFixCompletionIssue("LuauIfElseExprFixCompletionIssue", true); + check(R"( +local abcdef = 0; +local temp = false +local even = true; +local a +a = if temp then even else@1 +a = if temp then even else @2 +a = if temp then even else abc@3 + )"); + + auto ac = autocomplete('1'); + CHECK(ac.entryMap.count("else") == 0); + ac = autocomplete('2'); + CHECK(ac.entryMap.count("else") == 0); + ac = autocomplete('3'); + CHECK(ac.entryMap.count("abcdef")); +} + TEST_CASE_FIXTURE(ACFixture, "autocomplete_explicit_type_pack") { check(R"( diff --git a/tests/Compiler.test.cpp b/tests/Compiler.test.cpp index cd7a21d8..f982c86f 100644 --- a/tests/Compiler.test.cpp +++ b/tests/Compiler.test.cpp @@ -10,6 +10,11 @@ #include #include +namespace Luau +{ +std::string rep(const std::string& s, size_t n); +} + using namespace Luau; static std::string compileFunction(const char* source, uint32_t id) @@ -1960,15 +1965,6 @@ RETURN R8 -1 )"); } -static std::string rep(const std::string& s, size_t n) -{ - std::string r; - r.reserve(s.length() * n); - for (size_t i = 0; i < n; ++i) - r += s; - return r; -} - TEST_CASE("RecursionParse") { // The test forcibly pushes the stack limit during compilation; in NoOpt, the stack consumption is much larger so we need to reduce the limit to diff --git a/tests/Conformance.test.cpp b/tests/Conformance.test.cpp index 8b58d2ce..b09c1efb 100644 --- a/tests/Conformance.test.cpp +++ b/tests/Conformance.test.cpp @@ -492,6 +492,8 @@ TEST_CASE("DateTime") TEST_CASE("Debug") { + ScopedFastFlag luauTableFieldFunctionDebugname{"LuauTableFieldFunctionDebugname", true}; + runConformance("debug.lua"); } @@ -890,6 +892,12 @@ TEST_CASE("Coverage") lua_pushstring(L, function); lua_setfield(L, -2, "name"); + lua_pushinteger(L, linedefined); + lua_setfield(L, -2, "linedefined"); + + lua_pushinteger(L, depth); + lua_setfield(L, -2, "depth"); + for (size_t i = 0; i < size; ++i) if (hits[i] != -1) { diff --git a/tests/Fixture.cpp b/tests/Fixture.cpp index c74bfa27..dbdd06a4 100644 --- a/tests/Fixture.cpp +++ b/tests/Fixture.cpp @@ -2,6 +2,7 @@ #include "Fixture.h" #include "Luau/AstQuery.h" +#include "Luau/Parser.h" #include "Luau/TypeVar.h" #include "Luau/TypeAttach.h" #include "Luau/Transpiler.h" @@ -112,7 +113,7 @@ AstStatBlock* Fixture::parse(const std::string& source, const ParseOptions& pars sourceModule->name = fromString(mainModuleName); sourceModule->root = result.root; sourceModule->mode = parseMode(result.hotcomments); - sourceModule->ignoreLints = LintWarning::parseMask(result.hotcomments); + sourceModule->hotcomments = std::move(result.hotcomments); if (!result.errors.empty()) { @@ -157,6 +158,7 @@ CheckResult Fixture::check(const std::string& source) LintResult Fixture::lint(const std::string& source, const std::optional& lintOptions) { ParseOptions parseOptions; + parseOptions.captureComments = true; configResolver.defaultConfig.mode = Mode::Nonstrict; parse(source, parseOptions); diff --git a/tests/Fixture.h b/tests/Fixture.h index ab852ef6..4e45a952 100644 --- a/tests/Fixture.h +++ b/tests/Fixture.h @@ -8,7 +8,6 @@ #include "Luau/Linter.h" #include "Luau/Location.h" #include "Luau/ModuleResolver.h" -#include "Luau/Parser.h" #include "Luau/ToString.h" #include "Luau/TypeInfer.h" #include "Luau/TypeVar.h" diff --git a/tests/Frontend.test.cpp b/tests/Frontend.test.cpp index ea1a08fe..8a59acd1 100644 --- a/tests/Frontend.test.cpp +++ b/tests/Frontend.test.cpp @@ -2,7 +2,6 @@ #include "Luau/AstQuery.h" #include "Luau/BuiltinDefinitions.h" #include "Luau/Frontend.h" -#include "Luau/Parser.h" #include "Luau/RequireTracer.h" #include "Fixture.h" @@ -897,8 +896,6 @@ TEST_CASE_FIXTURE(FrontendFixture, "clearStats") TEST_CASE_FIXTURE(FrontendFixture, "typecheck_twice_for_ast_types") { - ScopedFastFlag sffs("LuauTypeCheckTwice", true); - fileResolver.source["Module/A"] = R"( local a = 1 )"; diff --git a/tests/JsonEncoder.test.cpp b/tests/JsonEncoder.test.cpp index 4a717275..cb508072 100644 --- a/tests/JsonEncoder.test.cpp +++ b/tests/JsonEncoder.test.cpp @@ -46,7 +46,7 @@ TEST_CASE("encode_AstStatBlock") AstStatBlock block{Location(), bodyArray}; CHECK_EQ( - (R"({"type":"AstStatBlock","location":"0,0 - 0,0","body":[{"type":"AstStatLocal","location":"0,0 - 0,0","vars":["a_local"],"values":[]}]})"), + (R"({"type":"AstStatBlock","location":"0,0 - 0,0","body":[{"type":"AstStatLocal","location":"0,0 - 0,0","vars":[{"type":null,"name":"a_local","location":"0,0 - 0,0"}],"values":[]}]})"), toJson(&block)); } diff --git a/tests/Linter.test.cpp b/tests/Linter.test.cpp index 577415fc..d4b97360 100644 --- a/tests/Linter.test.cpp +++ b/tests/Linter.test.cpp @@ -1395,12 +1395,10 @@ TEST_CASE_FIXTURE(Fixture, "DeprecatedApi") TypeId colorType = typeChecker.globalTypes.addType(TableTypeVar{{}, std::nullopt, typeChecker.globalScope->level, Luau::TableState::Sealed}); - getMutable(colorType)->props = { - {"toHSV", {typeChecker.anyType, /* deprecated= */ true, "Color3:ToHSV"} } - }; + getMutable(colorType)->props = {{"toHSV", {typeChecker.anyType, /* deprecated= */ true, "Color3:ToHSV"}}}; addGlobalBinding(typeChecker, "Color3", Binding{colorType, {}}); - + freeze(typeChecker.globalTypes); LintResult result = lintTyped(R"( @@ -1554,8 +1552,46 @@ _ = (math.random() < 0.5 and false) or 42 -- currently ignored )"); REQUIRE_EQ(result.warnings.size(), 2); - CHECK_EQ(result.warnings[0].text, "The and-or expression always evaluates to the second alternative because the first alternative is false; consider using if-then-else expression instead"); - CHECK_EQ(result.warnings[1].text, "The and-or expression always evaluates to the second alternative because the first alternative is nil; consider using if-then-else expression instead"); + CHECK_EQ(result.warnings[0].text, "The and-or expression always evaluates to the second alternative because the first alternative is false; " + "consider using if-then-else expression instead"); + CHECK_EQ(result.warnings[1].text, "The and-or expression always evaluates to the second alternative because the first alternative is nil; " + "consider using if-then-else expression instead"); +} + +TEST_CASE_FIXTURE(Fixture, "WrongComment") +{ + ScopedFastFlag sff("LuauParseAllHotComments", true); + + LintResult result = lint(R"( +--!strict +--!struct +--!nolintGlobal +--!nolint Global +--!nolint KnownGlobal +--!nolint UnknownGlobal +--! no more lint +--!strict here +do end +--!nolint +)"); + + REQUIRE_EQ(result.warnings.size(), 6); + CHECK_EQ(result.warnings[0].text, "Unknown comment directive 'struct'; did you mean 'strict'?"); + CHECK_EQ(result.warnings[1].text, "Unknown comment directive 'nolintGlobal'"); + CHECK_EQ(result.warnings[2].text, "nolint directive refers to unknown lint rule 'Global'"); + CHECK_EQ(result.warnings[3].text, "nolint directive refers to unknown lint rule 'KnownGlobal'; did you mean 'UnknownGlobal'?"); + CHECK_EQ(result.warnings[4].text, "Comment directive with the type checking mode has extra symbols at the end of the line"); + CHECK_EQ(result.warnings[5].text, "Comment directive is ignored because it is placed after the first non-comment token"); +} + +TEST_CASE_FIXTURE(Fixture, "WrongCommentMuteSelf") +{ + LintResult result = lint(R"( +--!nolint +--!struct +)"); + + REQUIRE_EQ(result.warnings.size(), 0); // --!nolint disables WrongComment lint :) } TEST_SUITE_END(); diff --git a/tests/NonstrictMode.test.cpp b/tests/NonstrictMode.test.cpp index 931a8403..5bad9901 100644 --- a/tests/NonstrictMode.test.cpp +++ b/tests/NonstrictMode.test.cpp @@ -1,5 +1,4 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details -#include "Luau/Parser.h" #include "Luau/Scope.h" #include "Luau/TypeInfer.h" #include "Luau/TypeVar.h" diff --git a/tests/Parser.test.cpp b/tests/Parser.test.cpp index c1a8887b..0d4c088d 100644 --- a/tests/Parser.test.cpp +++ b/tests/Parser.test.cpp @@ -1,6 +1,5 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #include "Luau/Parser.h" -#include "Luau/TypeInfer.h" #include "Fixture.h" #include "ScopedFlags.h" @@ -300,8 +299,9 @@ TEST_CASE_FIXTURE(Fixture, "functions_can_have_return_annotations") AstStatFunction* statFunction = block->body.data[0]->as(); REQUIRE(statFunction != nullptr); - CHECK_EQ(statFunction->func->returnAnnotation.types.size, 1); - CHECK(statFunction->func->returnAnnotation.tailType == nullptr); + REQUIRE(statFunction->func->returnAnnotation.has_value()); + CHECK_EQ(statFunction->func->returnAnnotation->types.size, 1); + CHECK(statFunction->func->returnAnnotation->tailType == nullptr); } TEST_CASE_FIXTURE(Fixture, "functions_can_have_a_function_type_annotation") @@ -316,9 +316,9 @@ TEST_CASE_FIXTURE(Fixture, "functions_can_have_a_function_type_annotation") AstStatFunction* statFunc = block->body.data[0]->as(); REQUIRE(statFunc != nullptr); - AstArray& retTypes = statFunc->func->returnAnnotation.types; - REQUIRE(statFunc->func->hasReturnAnnotation); - CHECK(statFunc->func->returnAnnotation.tailType == nullptr); + REQUIRE(statFunc->func->returnAnnotation.has_value()); + CHECK(statFunc->func->returnAnnotation->tailType == nullptr); + AstArray& retTypes = statFunc->func->returnAnnotation->types; REQUIRE(retTypes.size == 1); AstTypeFunction* funTy = retTypes.data[0]->as(); @@ -337,9 +337,9 @@ TEST_CASE_FIXTURE(Fixture, "function_return_type_should_disambiguate_from_functi AstStatFunction* statFunc = block->body.data[0]->as(); REQUIRE(statFunc != nullptr); - AstArray& retTypes = statFunc->func->returnAnnotation.types; - REQUIRE(statFunc->func->hasReturnAnnotation); - CHECK(statFunc->func->returnAnnotation.tailType == nullptr); + REQUIRE(statFunc->func->returnAnnotation.has_value()); + CHECK(statFunc->func->returnAnnotation->tailType == nullptr); + AstArray& retTypes = statFunc->func->returnAnnotation->types; REQUIRE(retTypes.size == 2); AstTypeReference* ty0 = retTypes.data[0]->as(); @@ -363,9 +363,9 @@ TEST_CASE_FIXTURE(Fixture, "function_return_type_should_parse_as_function_type_a AstStatFunction* statFunc = block->body.data[0]->as(); REQUIRE(statFunc != nullptr); - AstArray& retTypes = statFunc->func->returnAnnotation.types; - REQUIRE(statFunc->func->hasReturnAnnotation); - CHECK(statFunc->func->returnAnnotation.tailType == nullptr); + REQUIRE(statFunc->func->returnAnnotation.has_value()); + CHECK(statFunc->func->returnAnnotation->tailType == nullptr); + AstArray& retTypes = statFunc->func->returnAnnotation->types; REQUIRE(retTypes.size == 1); AstTypeFunction* funTy = retTypes.data[0]->as(); @@ -707,12 +707,25 @@ TEST_CASE_FIXTURE(Fixture, "mode_is_unset_if_no_hot_comment") TEST_CASE_FIXTURE(Fixture, "sense_hot_comment_on_first_line") { - ParseResult result = parseEx(" --!strict "); + ParseOptions options; + options.captureComments = true; + + ParseResult result = parseEx(" --!strict ", options); std::optional mode = parseMode(result.hotcomments); REQUIRE(bool(mode)); CHECK_EQ(int(*mode), int(Mode::Strict)); } +TEST_CASE_FIXTURE(Fixture, "non_header_hot_comments") +{ + ParseOptions options; + options.captureComments = true; + + ParseResult result = parseEx("do end --!strict", options); + std::optional mode = parseMode(result.hotcomments); + REQUIRE(!mode); +} + TEST_CASE_FIXTURE(Fixture, "stop_if_line_ends_with_hyphen") { CHECK_THROWS_AS(parse(" -"), std::exception); @@ -720,7 +733,10 @@ TEST_CASE_FIXTURE(Fixture, "stop_if_line_ends_with_hyphen") TEST_CASE_FIXTURE(Fixture, "nonstrict_mode") { - ParseResult result = parseEx("--!nonstrict"); + ParseOptions options; + options.captureComments = true; + + ParseResult result = parseEx("--!nonstrict", options); CHECK(result.errors.empty()); std::optional mode = parseMode(result.hotcomments); REQUIRE(bool(mode)); @@ -729,7 +745,10 @@ TEST_CASE_FIXTURE(Fixture, "nonstrict_mode") TEST_CASE_FIXTURE(Fixture, "nocheck_mode") { - ParseResult result = parseEx("--!nocheck"); + ParseOptions options; + options.captureComments = true; + + ParseResult result = parseEx("--!nocheck", options); CHECK(result.errors.empty()); std::optional mode = parseMode(result.hotcomments); REQUIRE(bool(mode)); @@ -1498,8 +1517,6 @@ return TEST_CASE_FIXTURE(Fixture, "parse_error_broken_comment") { - ScopedFastFlag luauStartingBrokenComment{"LuauStartingBrokenComment", true}; - const char* expected = "Expected identifier when parsing expression, got unfinished comment"; matchParseError("--[[unfinished work", expected); diff --git a/tests/Repl.test.cpp b/tests/Repl.test.cpp index 1f9c9739..87a1e1e2 100644 --- a/tests/Repl.test.cpp +++ b/tests/Repl.test.cpp @@ -73,7 +73,7 @@ public: private: std::unique_ptr luaState; - // This is a simplicitic and incomplete pretty printer. + // This is a simplistic and incomplete pretty printer. // It is included here to test that the pretty printer hook is being called. // More elaborate tests to ensure correct output can be added if we introduce // a more feature rich pretty printer. @@ -158,12 +158,25 @@ TEST_CASE_FIXTURE(ReplFixture, "CompleteGlobalVariables") myvariable1 = 5 myvariable2 = 5 )"); - CompletionSet completions = getCompletionSet("myvar"); + { + // Try to complete globals that are added by the user's script + CompletionSet completions = getCompletionSet("myvar"); - std::string prefix = ""; - CHECK(completions.size() == 2); - CHECK(checkCompletion(completions, prefix, "myvariable1")); - CHECK(checkCompletion(completions, prefix, "myvariable2")); + std::string prefix = ""; + CHECK(completions.size() == 2); + CHECK(checkCompletion(completions, prefix, "myvariable1")); + CHECK(checkCompletion(completions, prefix, "myvariable2")); + } + { + // Try completing some builtin functions + CompletionSet completions = getCompletionSet("math.m"); + + std::string prefix = "math."; + CHECK(completions.size() == 3); + CHECK(checkCompletion(completions, prefix, "max(")); + CHECK(checkCompletion(completions, prefix, "min(")); + CHECK(checkCompletion(completions, prefix, "modf(")); + } } TEST_CASE_FIXTURE(ReplFixture, "CompleteTableKeys") @@ -206,4 +219,188 @@ TEST_CASE_FIXTURE(ReplFixture, "StringMethods") } } +TEST_CASE_FIXTURE(ReplFixture, "TableWithMetatableIndexTable") +{ + runCode(L, R"( + -- Create 't' which is a table with a metatable with an __index table + mt = {} + mt.__index = mt + + t = {} + setmetatable(t, mt) + + mt.mtkey1 = {x="x value", y="y value", 1, 2} + mt.mtkey2 = 2 + + t.tkey1 = {data1 = 2, data2 = "str", 3, 4} + t.tkey2 = 4 +)"); + { + CompletionSet completions = getCompletionSet("t.t"); + + std::string prefix = "t."; + CHECK(completions.size() == 2); + CHECK(checkCompletion(completions, prefix, "tkey1")); + CHECK(checkCompletion(completions, prefix, "tkey2")); + } + { + CompletionSet completions = getCompletionSet("t.tkey1.data2:re"); + + std::string prefix = "t.tkey1.data2:"; + CHECK(completions.size() == 2); + CHECK(checkCompletion(completions, prefix, "rep(")); + CHECK(checkCompletion(completions, prefix, "reverse(")); + } + { + CompletionSet completions = getCompletionSet("t.mtk"); + + std::string prefix = "t."; + CHECK(completions.size() == 2); + CHECK(checkCompletion(completions, prefix, "mtkey1")); + CHECK(checkCompletion(completions, prefix, "mtkey2")); + } + { + CompletionSet completions = getCompletionSet("t.mtkey1."); + + std::string prefix = "t.mtkey1."; + CHECK(completions.size() == 2); + CHECK(checkCompletion(completions, prefix, "x")); + CHECK(checkCompletion(completions, prefix, "y")); + } +} + +TEST_CASE_FIXTURE(ReplFixture, "TableWithMetatableIndexFunction") +{ + runCode(L, R"( + -- Create 't' which is a table with a metatable with an __index function + mt = {} + mt.__index = function(table, key) + print("mt.__index called") + if key == "foo" then + return "FOO" + elseif key == "bar" then + return "BAR" + else + return nil + end + end + + t = {} + setmetatable(t, mt) + t.tkey = 0 +)"); + { + CompletionSet completions = getCompletionSet("t.t"); + + std::string prefix = "t."; + CHECK(completions.size() == 1); + CHECK(checkCompletion(completions, prefix, "tkey")); + } + { + // t.foo is a valid key, but should not be completed because it requires calling an __index function + CompletionSet completions = getCompletionSet("t.foo"); + + CHECK(completions.size() == 0); + } + { + // t.foo is a valid key, but should not be found because it requires calling an __index function + CompletionSet completions = getCompletionSet("t.foo:"); + + CHECK(completions.size() == 0); + } +} + +TEST_CASE_FIXTURE(ReplFixture, "TableWithMultipleMetatableIndexTables") +{ + runCode(L, R"( + -- Create a table with a chain of metatables + mt2 = {} + mt2.__index = mt2 + + mt = {} + mt.__index = mt + setmetatable(mt, mt2) + + t = {} + setmetatable(t, mt) + + mt2.mt2key = {x=1, y=2} + mt.mtkey = 2 + t.tkey = 3 +)"); + { + CompletionSet completions = getCompletionSet("t."); + + std::string prefix = "t."; + CHECK(completions.size() == 4); + CHECK(checkCompletion(completions, prefix, "__index")); + CHECK(checkCompletion(completions, prefix, "tkey")); + CHECK(checkCompletion(completions, prefix, "mtkey")); + CHECK(checkCompletion(completions, prefix, "mt2key")); + } + { + CompletionSet completions = getCompletionSet("t.__index."); + + std::string prefix = "t.__index."; + CHECK(completions.size() == 3); + CHECK(checkCompletion(completions, prefix, "__index")); + CHECK(checkCompletion(completions, prefix, "mtkey")); + CHECK(checkCompletion(completions, prefix, "mt2key")); + } + { + CompletionSet completions = getCompletionSet("t.mt2key."); + + std::string prefix = "t.mt2key."; + CHECK(completions.size() == 2); + CHECK(checkCompletion(completions, prefix, "x")); + CHECK(checkCompletion(completions, prefix, "y")); + } +} + +TEST_CASE_FIXTURE(ReplFixture, "TableWithDeepMetatableIndexTables") +{ + runCode(L, R"( +-- Creates a table with a chain of metatables of length `count` +function makeChainedTable(count) + local result = {} + result.__index = result + result[string.format("entry%d", count)] = { count = count } + if count == 0 then + return result + else + return setmetatable(result, makeChainedTable(count - 1)) + end +end + +t30 = makeChainedTable(30) +t60 = makeChainedTable(60) +)"); + { + // Check if entry0 exists + CompletionSet completions = getCompletionSet("t30.entry0"); + + std::string prefix = "t30."; + CHECK(checkCompletion(completions, prefix, "entry0")); + } + { + // Check if entry0.count exists + CompletionSet completions = getCompletionSet("t30.entry0.co"); + + std::string prefix = "t30.entry0."; + CHECK(checkCompletion(completions, prefix, "count")); + } + { + // Check if entry0 exists. With the max traversal limit of 50 in the repl, this should fail. + CompletionSet completions = getCompletionSet("t60.entry0"); + + CHECK(completions.size() == 0); + } + { + // Check if entry0.count exists. With the max traversal limit of 50 in the repl, this should fail. + CompletionSet completions = getCompletionSet("t60.entry0.co"); + + CHECK(completions.size() == 0); + } +} + TEST_SUITE_END(); diff --git a/tests/RequireTracer.test.cpp b/tests/RequireTracer.test.cpp index b9fd04d6..ba03f363 100644 --- a/tests/RequireTracer.test.cpp +++ b/tests/RequireTracer.test.cpp @@ -1,6 +1,6 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details -#include "Luau/Parser.h" #include "Luau/RequireTracer.h" +#include "Luau/Parser.h" #include "Fixture.h" diff --git a/tests/TypeInfer.aliases.test.cpp b/tests/TypeInfer.aliases.test.cpp index a8729268..31d7ef10 100644 --- a/tests/TypeInfer.aliases.test.cpp +++ b/tests/TypeInfer.aliases.test.cpp @@ -598,15 +598,13 @@ TEST_CASE_FIXTURE(Fixture, "generic_typevars_are_not_considered_to_escape_their_ /* * The two-pass alias definition system starts by ascribing a free TypeVar to each alias. It then * circles back to fill in the actual type later on. - * + * * If this free type is unified with something degenerate like `any`, we need to take extra care * to ensure that the alias actually binds to the type that the user expected. */ TEST_CASE_FIXTURE(Fixture, "forward_declared_alias_is_not_clobbered_by_prior_unification_with_any") { - ScopedFastFlag sff[] = { - {"LuauTwoPassAliasDefinitionFix", true} - }; + ScopedFastFlag sff[] = {{"LuauTwoPassAliasDefinitionFix", true}}; CheckResult result = check(R"( local function x() diff --git a/tests/TypeInfer.annotations.test.cpp b/tests/TypeInfer.annotations.test.cpp index 572b882d..2ad11d01 100644 --- a/tests/TypeInfer.annotations.test.cpp +++ b/tests/TypeInfer.annotations.test.cpp @@ -1,6 +1,5 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #include "Luau/BuiltinDefinitions.h" -#include "Luau/Parser.h" #include "Luau/TypeInfer.h" #include "Luau/TypeVar.h" diff --git a/tests/TypeInfer.builtins.test.cpp b/tests/TypeInfer.builtins.test.cpp index df06884d..f3dfb214 100644 --- a/tests/TypeInfer.builtins.test.cpp +++ b/tests/TypeInfer.builtins.test.cpp @@ -1,5 +1,4 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details -#include "Luau/Parser.h" #include "Luau/TypeInfer.h" #include "Luau/BuiltinDefinitions.h" diff --git a/tests/TypeInfer.classes.test.cpp b/tests/TypeInfer.classes.test.cpp index 0283ae19..98fa66eb 100644 --- a/tests/TypeInfer.classes.test.cpp +++ b/tests/TypeInfer.classes.test.cpp @@ -1,6 +1,5 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #include "Luau/BuiltinDefinitions.h" -#include "Luau/Parser.h" #include "Luau/TypeInfer.h" #include "Luau/TypeVar.h" diff --git a/tests/TypeInfer.definitions.test.cpp b/tests/TypeInfer.definitions.test.cpp index 2652486b..c6d55793 100644 --- a/tests/TypeInfer.definitions.test.cpp +++ b/tests/TypeInfer.definitions.test.cpp @@ -1,6 +1,5 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #include "Luau/BuiltinDefinitions.h" -#include "Luau/Parser.h" #include "Luau/TypeInfer.h" #include "Luau/TypeVar.h" diff --git a/tests/TypeInfer.generics.test.cpp b/tests/TypeInfer.generics.test.cpp index 8a2c6f27..f8fccf6b 100644 --- a/tests/TypeInfer.generics.test.cpp +++ b/tests/TypeInfer.generics.test.cpp @@ -1,5 +1,4 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details -#include "Luau/Parser.h" #include "Luau/TypeInfer.h" #include "Luau/TypeVar.h" diff --git a/tests/TypeInfer.intersectionTypes.test.cpp b/tests/TypeInfer.intersectionTypes.test.cpp index 93c0baf6..d677e28d 100644 --- a/tests/TypeInfer.intersectionTypes.test.cpp +++ b/tests/TypeInfer.intersectionTypes.test.cpp @@ -1,5 +1,4 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details -#include "Luau/Parser.h" #include "Luau/TypeInfer.h" #include "Luau/TypeVar.h" diff --git a/tests/TypeInfer.provisional.test.cpp b/tests/TypeInfer.provisional.test.cpp index 2bcd840c..eee0e0f1 100644 --- a/tests/TypeInfer.provisional.test.cpp +++ b/tests/TypeInfer.provisional.test.cpp @@ -1,5 +1,4 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details -#include "Luau/Parser.h" #include "Luau/TypeInfer.h" #include "Fixture.h" diff --git a/tests/TypeInfer.singletons.test.cpp b/tests/TypeInfer.singletons.test.cpp index df365fda..9021700d 100644 --- a/tests/TypeInfer.singletons.test.cpp +++ b/tests/TypeInfer.singletons.test.cpp @@ -426,8 +426,6 @@ TEST_CASE_FIXTURE(Fixture, "if_then_else_expression_singleton_options") {"LuauSingletonTypes", true}, {"LuauParseSingletonTypes", true}, {"LuauExpectedTypesOfProperties", true}, - {"LuauIfElseExpectedType2", true}, - {"LuauIfElseBranchTypeUnion", true}, }; CheckResult result = check(R"( diff --git a/tests/TypeInfer.tables.test.cpp b/tests/TypeInfer.tables.test.cpp index f19cb618..6bcd4b99 100644 --- a/tests/TypeInfer.tables.test.cpp +++ b/tests/TypeInfer.tables.test.cpp @@ -1,6 +1,5 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #include "Luau/BuiltinDefinitions.h" -#include "Luau/Parser.h" #include "Luau/TypeInfer.h" #include "Luau/TypeVar.h" @@ -2168,8 +2167,6 @@ b() TEST_CASE_FIXTURE(Fixture, "length_operator_union") { - ScopedFastFlag luauLengthOnCompositeType{"LuauLengthOnCompositeType", true}; - CheckResult result = check(R"( local x: {number} | {string} local y = #x @@ -2180,8 +2177,6 @@ local y = #x TEST_CASE_FIXTURE(Fixture, "length_operator_intersection") { - ScopedFastFlag luauLengthOnCompositeType{"LuauLengthOnCompositeType", true}; - CheckResult result = check(R"( local x: {number} & {z:string} -- mixed tables are evil local y = #x @@ -2192,8 +2187,6 @@ local y = #x TEST_CASE_FIXTURE(Fixture, "length_operator_non_table_union") { - ScopedFastFlag luauLengthOnCompositeType{"LuauLengthOnCompositeType", true}; - CheckResult result = check(R"( local x: {number} | any | string local y = #x @@ -2204,8 +2197,6 @@ local y = #x TEST_CASE_FIXTURE(Fixture, "length_operator_union_errors") { - ScopedFastFlag luauLengthOnCompositeType{"LuauLengthOnCompositeType", true}; - CheckResult result = check(R"( local x: {number} | number | string local y = #x @@ -2214,4 +2205,38 @@ local y = #x LUAU_REQUIRE_ERROR_COUNT(1, result); } +TEST_CASE_FIXTURE(Fixture, "dont_hang_when_trying_to_look_up_in_cyclic_metatable_index") +{ + ScopedFastFlag sff{"LuauTerminateCyclicMetatableIndexLookup", true}; + + // t :: t1 where t1 = {metatable {__index: t1, __tostring: (t1) -> string}} + CheckResult result = check(R"( + local mt = {} + local t = setmetatable({}, mt) + mt.__index = t + + function mt:__tostring() + return t.p + end + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ("Type 't' does not have key 'p'", toString(result.errors[0])); +} + +TEST_CASE_FIXTURE(Fixture, "give_up_after_one_metatable_index_look_up") +{ + CheckResult result = check(R"( + local data = { x = 5 } + local t1 = setmetatable({}, { __index = data }) + local t2 = setmetatable({}, t1) -- note: must be t1, not a new table + + local x1 = t1.x -- ok + local x2 = t2.x -- nope + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ("Type 't2' does not have key 'x'", toString(result.errors[0])); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.test.cpp b/tests/TypeInfer.test.cpp index 531a382f..32358571 100644 --- a/tests/TypeInfer.test.cpp +++ b/tests/TypeInfer.test.cpp @@ -2,7 +2,6 @@ #include "Luau/AstQuery.h" #include "Luau/BuiltinDefinitions.h" -#include "Luau/Parser.h" #include "Luau/Scope.h" #include "Luau/TypeInfer.h" #include "Luau/TypeVar.h" @@ -4654,8 +4653,6 @@ a = setmetatable(a, { __call = function(x) end }) TEST_CASE_FIXTURE(Fixture, "infer_through_group_expr") { - ScopedFastFlag luauGroupExpectedType{"LuauGroupExpectedType", true}; - CheckResult result = check(R"( local function f(a: (number, number) -> number) return a(1, 3) end f(((function(a, b) return a + b end))) @@ -4735,21 +4732,14 @@ local a = if false then "a" elseif false then "b" else "c" TEST_CASE_FIXTURE(Fixture, "tc_if_else_expressions_type_union") { - ScopedFastFlag sff3{"LuauIfElseBranchTypeUnion", true}; + CheckResult result = check(R"(local a: number? = if true then 42 else nil)"); - { - CheckResult result = check(R"(local a: number? = if true then 42 else nil)"); - - LUAU_REQUIRE_NO_ERRORS(result); - CHECK_EQ(toString(requireType("a"), {true}), "number?"); - } + LUAU_REQUIRE_NO_ERRORS(result); + CHECK_EQ(toString(requireType("a"), {true}), "number?"); } TEST_CASE_FIXTURE(Fixture, "tc_if_else_expressions_expected_type_1") { - ScopedFastFlag luauIfElseExpectedType2{"LuauIfElseExpectedType2", true}; - ScopedFastFlag luauIfElseBranchTypeUnion{"LuauIfElseBranchTypeUnion", true}; - CheckResult result = check(R"( type X = {number | string} local a: X = if true then {"1", 2, 3} else {4, 5, 6} @@ -4761,9 +4751,6 @@ local a: X = if true then {"1", 2, 3} else {4, 5, 6} TEST_CASE_FIXTURE(Fixture, "tc_if_else_expressions_expected_type_2") { - ScopedFastFlag luauIfElseExpectedType2{"LuauIfElseExpectedType2", true}; - ScopedFastFlag luauIfElseBranchTypeUnion{"LuauIfElseBranchTypeUnion", true}; - CheckResult result = check(R"( local a: number? = if true then 1 else nil )"); @@ -4773,8 +4760,6 @@ local a: number? = if true then 1 else nil TEST_CASE_FIXTURE(Fixture, "tc_if_else_expressions_expected_type_3") { - ScopedFastFlag luauIfElseExpectedType2{"LuauIfElseExpectedType2", true}; - CheckResult result = check(R"( local function times(n: any, f: () -> T) local result: {T} = {} @@ -5058,8 +5043,6 @@ end TEST_CASE_FIXTURE(Fixture, "recursive_metatable_crash") { - ScopedFastFlag luauMetatableAreEqualRecursion{"LuauMetatableAreEqualRecursion", true}; - CheckResult result = check(R"( local function getIt() local y @@ -5076,8 +5059,6 @@ local c = a or b TEST_CASE_FIXTURE(Fixture, "bound_typepack_promote") { - ScopedFastFlag luauCommittingTxnLogFreeTpPromote{"LuauCommittingTxnLogFreeTpPromote", true}; - // No assertions should trigger check(R"( local function p() @@ -5251,7 +5232,6 @@ TEST_CASE_FIXTURE(Fixture, "taking_the_length_of_string_singleton") {"LuauDiscriminableUnions2", true}, {"LuauRefactorTypeVarQuestions", true}, {"LuauSingletonTypes", true}, - {"LuauLengthOnCompositeType", true}, }; CheckResult result = check(R"( @@ -5272,7 +5252,6 @@ TEST_CASE_FIXTURE(Fixture, "taking_the_length_of_union_of_string_singleton") {"LuauDiscriminableUnions2", true}, {"LuauRefactorTypeVarQuestions", true}, {"LuauSingletonTypes", true}, - {"LuauLengthOnCompositeType", true}, }; CheckResult result = check(R"( diff --git a/tests/TypeInfer.tryUnify.test.cpp b/tests/TypeInfer.tryUnify.test.cpp index 8c7fb79a..4669ea8e 100644 --- a/tests/TypeInfer.tryUnify.test.cpp +++ b/tests/TypeInfer.tryUnify.test.cpp @@ -1,5 +1,4 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details -#include "Luau/Parser.h" #include "Luau/Scope.h" #include "Luau/TypeInfer.h" #include "Luau/TypeVar.h" diff --git a/tests/TypeInfer.typePacks.cpp b/tests/TypeInfer.typePacks.cpp index 079870f5..cbe2e48f 100644 --- a/tests/TypeInfer.typePacks.cpp +++ b/tests/TypeInfer.typePacks.cpp @@ -1,6 +1,5 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #include "Luau/BuiltinDefinitions.h" -#include "Luau/Parser.h" #include "Luau/TypeInfer.h" #include "Luau/TypeVar.h" @@ -930,8 +929,6 @@ type R = { m: F } TEST_CASE_FIXTURE(Fixture, "pack_tail_unification_check") { - ScopedFastFlag luauUnifyPackTails{"LuauUnifyPackTails", true}; - CheckResult result = check(R"( local a: () -> (number, ...string) local b: () -> (number, ...boolean) diff --git a/tests/TypeInfer.unionTypes.test.cpp b/tests/TypeInfer.unionTypes.test.cpp index 759794e6..3b53ddfe 100644 --- a/tests/TypeInfer.unionTypes.test.cpp +++ b/tests/TypeInfer.unionTypes.test.cpp @@ -1,5 +1,4 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details -#include "Luau/Parser.h" #include "Luau/TypeInfer.h" #include "Luau/TypeVar.h" diff --git a/tests/TypePack.test.cpp b/tests/TypePack.test.cpp index 8b056544..c4931578 100644 --- a/tests/TypePack.test.cpp +++ b/tests/TypePack.test.cpp @@ -1,5 +1,4 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details -#include "Luau/Parser.h" #include "Luau/TypeInfer.h" #include "Luau/TypeVar.h" diff --git a/tests/TypeVar.test.cpp b/tests/TypeVar.test.cpp index 329e7b1f..e43161fa 100644 --- a/tests/TypeVar.test.cpp +++ b/tests/TypeVar.test.cpp @@ -1,5 +1,4 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details -#include "Luau/Parser.h" #include "Luau/Scope.h" #include "Luau/TypeInfer.h" #include "Luau/TypeVar.h" diff --git a/tests/conformance/coroutine.lua b/tests/conformance/coroutine.lua index f2ecc96b..b4f81bba 100644 --- a/tests/conformance/coroutine.lua +++ b/tests/conformance/coroutine.lua @@ -371,6 +371,15 @@ do st, msg = coroutine.close(co) assert(st and msg == nil) assert(f() == 42) + + -- closing a coroutine with a large stack + co = coroutine.create(function() + local function f(depth) return if depth > 0 then f(depth - 1) + depth else 0 end + coroutine.yield(f(100)) + end) + assert(coroutine.resume(co)) + st, msg = coroutine.close(co) + assert(st and msg == nil) end return 'OK' diff --git a/tests/conformance/coverage.lua b/tests/conformance/coverage.lua index f899603f..14d843a4 100644 --- a/tests/conformance/coverage.lua +++ b/tests/conformance/coverage.lua @@ -49,16 +49,24 @@ foo() c = getcoverage(foo) assert(#c == 1) assert(c[1].name == "foo") +assert(c[1].linedefined == 4) +assert(c[1].depth == 0) assert(validate(c[1], {5, 6, 7}, {})) bar() c = getcoverage(bar) assert(#c == 3) assert(c[1].name == "bar") +assert(c[1].linedefined == 10) +assert(c[1].depth == 0) assert(validate(c[1], {11, 15, 19}, {})) assert(c[2].name == "one") +assert(c[2].linedefined == 11) +assert(c[2].depth == 1) assert(validate(c[2], {12}, {})) assert(c[3].name == nil) +assert(c[3].linedefined == 15) +assert(c[3].depth == 1) assert(validate(c[3], {}, {16})) return 'OK' diff --git a/tests/conformance/debug.lua b/tests/conformance/debug.lua index 0e410000..0c8cc2d8 100644 --- a/tests/conformance/debug.lua +++ b/tests/conformance/debug.lua @@ -76,6 +76,9 @@ assert(baz(co, 2, "n") == nil) assert(baz(math.sqrt, "n") == "sqrt") assert(baz(math.sqrt, "f") == math.sqrt) -- yes this is pointless +local t = { foo = function() return 1 end } +assert(baz(t.foo, "n") == "foo") + -- info multi-arg returns function quux(...) return {debug.info(...)} diff --git a/tests/main.cpp b/tests/main.cpp index cd24e100..2af9f702 100644 --- a/tests/main.cpp +++ b/tests/main.cpp @@ -10,6 +10,9 @@ #ifndef WIN32_LEAN_AND_MEAN #define WIN32_LEAN_AND_MEAN #endif +#ifndef NOMINMAX +#define NOMINMAX +#endif #include // IsDebuggerPresent #endif @@ -52,7 +55,7 @@ static bool debuggerPresent() #endif } -static int assertionHandler(const char* expr, const char* file, int line, const char* function) +static int testAssertionHandler(const char* expr, const char* file, int line, const char* function) { if (debuggerPresent()) LUAU_DEBUGBREAK(); @@ -218,7 +221,7 @@ static void setFastFlags(const std::vector& flags) int main(int argc, char** argv) { - Luau::assertHandler() = assertionHandler; + Luau::assertHandler() = testAssertionHandler; doctest::registerReporter("boost", 0, true); diff --git a/tools/natvis/Analysis.natvis b/tools/natvis/Analysis.natvis new file mode 100644 index 00000000..5de0140e --- /dev/null +++ b/tools/natvis/Analysis.natvis @@ -0,0 +1,78 @@ + + + + + AnyTypeVar + + + + {{ index=0, value={*($T1*)storage} }} + {{ index=1, value={*($T2*)storage} }} + {{ index=2, value={*($T3*)storage} }} + {{ index=3, value={*($T4*)storage} }} + {{ index=4, value={*($T5*)storage} }} + {{ index=5, value={*($T6*)storage} }} + {{ index=6, value={*($T7*)storage} }} + {{ index=7, value={*($T8*)storage} }} + {{ index=8, value={*($T9*)storage} }} + {{ index=9, value={*($T10*)storage} }} + {{ index=10, value={*($T11*)storage} }} + {{ index=11, value={*($T12*)storage} }} + {{ index=12, value={*($T13*)storage} }} + {{ index=13, value={*($T14*)storage} }} + {{ index=14, value={*($T15*)storage} }} + {{ index=15, value={*($T16*)storage} }} + {{ index=16, value={*($T17*)storage} }} + {{ index=17, value={*($T18*)storage} }} + {{ index=18, value={*($T19*)storage} }} + {{ index=19, value={*($T20*)storage} }} + {{ index=20, value={*($T21*)storage} }} + {{ index=21, value={*($T22*)storage} }} + {{ index=22, value={*($T23*)storage} }} + {{ index=23, value={*($T24*)storage} }} + {{ index=24, value={*($T25*)storage} }} + {{ index=25, value={*($T26*)storage} }} + {{ index=26, value={*($T27*)storage} }} + {{ index=27, value={*($T28*)storage} }} + {{ index=28, value={*($T29*)storage} }} + {{ index=29, value={*($T30*)storage} }} + {{ index=30, value={*($T31*)storage} }} + {{ index=31, value={*($T32*)storage} }} + + typeId + *($T1*)storage + *($T2*)storage + *($T3*)storage + *($T4*)storage + *($T5*)storage + *($T6*)storage + *($T7*)storage + *($T8*)storage + *($T9*)storage + *($T10*)storage + *($T11*)storage + *($T12*)storage + *($T13*)storage + *($T14*)storage + *($T15*)storage + *($T16*)storage + *($T17*)storage + *($T18*)storage + *($T19*)storage + *($T20*)storage + *($T21*)storage + *($T22*)storage + *($T23*)storage + *($T24*)storage + *($T25*)storage + *($T26*)storage + *($T27*)storage + *($T28*)storage + *($T29*)storage + *($T30*)storage + *($T31*)storage + *($T32*)storage + + + + diff --git a/tools/natvis/Ast.natvis b/tools/natvis/Ast.natvis new file mode 100644 index 00000000..322eb8f6 --- /dev/null +++ b/tools/natvis/Ast.natvis @@ -0,0 +1,25 @@ + + + + + AstArray size={size} + + size + + size + data + + + + + + + size_ + + size_ + storage._Mypair._Myval2._Myfirst + offset + + + + + diff --git a/tools/natvis/VM.natvis b/tools/natvis/VM.natvis new file mode 100644 index 00000000..9924e194 --- /dev/null +++ b/tools/natvis/VM.natvis @@ -0,0 +1,269 @@ + + + + + nil + {(bool)value.b} + lightuserdata {value.p} + number = {value.n} + vector = {value.v[0]}, {value.v[1]}, {*(float*)&extra} + {value.gc->ts} + {value.gc->h} + function {value.gc->cl,view(short)} + userdata {value.gc->u} + thread {value.gc->th} + proto {value.gc->p} + upvalue {value.gc->uv} + deadkey + empty + + value.p + value.gc->ts + value.gc->h + value.gc->cl + value.gc->cl + value.gc->u + value.gc->th + value.gc->p + value.gc->uv + + fixed ({(int)value.gc->gch.marked}) + black ({(int)value.gc->gch.marked}) + white ({(int)value.gc->gch.marked}) + white ({(int)value.gc->gch.marked}) + gray ({(int)value.gc->gch.marked}) + + + + + + nil + {(bool)value.b} + lightuserdata {value.p} + number = {value.n} + vector = {value.v[0]}, {value.v[1]}, {*(float*)&extra} + {value.gc->ts} + {value.gc->h} + function {value.gc->cl,view(short)} + userdata {value.gc->u} + thread {value.gc->th} + proto {value.gc->p} + upvalue {value.gc->uv} + deadkey + empty + + (void**)value.p + value.gc->ts + value.gc->h + value.gc->cl + value.gc->cl + value.gc->u + value.gc->th + value.gc->p + value.gc->uv + + next + + + + + {key,na} = {val} + --- + + + + table + + metatable + + + [size] {1<<lsizenode} + + + 1<<lsizenode + node[$i] + + + + + [size] {sizearray} + + + sizearray + array[$i] + + + + + + + + + + + + + 1 + + + + + metatable->node[i].val + + + + i = i + 1 + + + "unknown",sb + + + tag + len + metatable + data + + + + + {c.f,na} + {l.p,na} + {c} + {l} + invalid + + + + {data,s} + + + + + {ci->func->value.gc->cl.c.f,na} + + + {ci->func->value.gc->cl.l.p->source->data,sb}:{ci->func->value.gc->cl.l.p->linedefined,d} {ci->func->value.gc->cl.l.p->debugname->data,sb} + + + {ci->func->value.gc->cl.l.p->source->data,sb}:{ci->func->value.gc->cl.l.p->linedefined,d} + + thread + + + {ci-base_ci} frames + + + ci-base_ci + + + base_ci[ci-base_ci - $i].func->value.gc->cl,view(short) + + + + + + {top-base} values + + + top-base + base + + + + + {top-stack} values + + + top-stack + stack + + + + + + + openupval + u.l.next + this + + + + l_gt + env + userdata + + + + + {source->data,sb}:{linedefined} function {debugname->data,sb} [{(int)numparams} arg, {(int)nups} upval] + {source->data,sb}:{linedefined} [{(int)numparams} arg, {(int)nups} upval] + + debugname + + constants + + + sizek + k[$i] + + + + + locals + + + sizelocvars + locvars[$i] + + + + + bytecode + + + sizecode + code[$i] + + + + + functions + + + sizep + p[$i] + + + + + upvals + + + sizeupvalues + upvalues[$i] + + + + + source + + + + + + + {(lua_Type)tt} + + + fixed ({(int)marked}) + black ({(int)marked}) + white ({(int)marked}) + white ({(int)marked}) + gray ({(int)marked}) + unknown + + memcat + + + + From 6aaaafcc8d3e515514ff1a72621da500f0c83404 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Thu, 17 Feb 2022 16:58:43 -0800 Subject: [PATCH 168/399] Add documentation for upcoming CommentDirective lint (#361) --- docs/_pages/lint.md | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/docs/_pages/lint.md b/docs/_pages/lint.md index de4d247e..340d35a2 100644 --- a/docs/_pages/lint.md +++ b/docs/_pages/lint.md @@ -317,3 +317,13 @@ The code above can be rewritten as follows to avoid the warning and the associat ```lua local x = if flag then false else true ``` + +## CommentDirective (26) + +Luau uses comments that start from `!` to control certain aspects of analysis, for example setting type checking mode via `--!strict` or disabling individual lints with `--!nolint`. Unknown directives are ignored, for example `--!nostrict` doesn't have any effect on the type checking process as the correct spelling is `--!nonstrict`. This warning flags comment directives that are ignored during processing: + +```lua +--!nostrict +-- Unknown comment directive 'nostrict'; did you mean 'nonstrict'?" +``` +``` From 1ac64af48416a6ae801673c315cd7bc999958308 Mon Sep 17 00:00:00 2001 From: Lily Brown Date: Thu, 17 Feb 2022 17:15:33 -0800 Subject: [PATCH 169/399] Prototyping: Revise CI (#371) Introduces a test runner with test cases. Also significantly overhauls the GHA configuration. --- .github/workflows/prototyping.yml | 40 ++-- prototyping/.gitignore | 6 + prototyping/README.md | 18 ++ .../Tests/Interpreter/return_nil/in.lua | 5 + .../Tests/Interpreter/return_nil/out.txt | 1 + .../PrettyPrinter/smoke_test/in.lua} | 12 +- .../PrettyPrinter/smoke_test/out.txt} | 0 prototyping/tests | 197 ++++++++++++++++++ 8 files changed, 260 insertions(+), 19 deletions(-) create mode 100644 prototyping/Tests/Interpreter/return_nil/in.lua create mode 100644 prototyping/Tests/Interpreter/return_nil/out.txt rename prototyping/{Examples/SmokeTestOutput.lua => Tests/PrettyPrinter/smoke_test/in.lua} (72%) rename prototyping/{Examples/SmokeTest.lua => Tests/PrettyPrinter/smoke_test/out.txt} (100%) create mode 100755 prototyping/tests diff --git a/.github/workflows/prototyping.yml b/.github/workflows/prototyping.yml index 4a1b8e08..637ed840 100644 --- a/.github/workflows/prototyping.yml +++ b/.github/workflows/prototyping.yml @@ -34,6 +34,11 @@ jobs: with: path: ~/.cabal/store key: prototyping-${{ runner.os }}-${{ matrix.agda }} + - uses: actions/cache@v2 + id: luau-ast-cache + with: + path: ./build + key: prototyping-${{ runner.os }}-${{ hashFiles('Ast/**', 'Analysis/**', 'CLI/Ast.cpp', 'CLI/FileUtils.*')}} - name: install cabal run: sudo apt-get install -y cabal-install - name: cabal update @@ -43,26 +48,35 @@ jobs: working-directory: prototyping run: | cabal install Agda-${{ matrix.agda }} - cabal install --lib scientific --package-env . - cabal install --lib vector --package-env . - cabal install --lib aeson --package-env . - - name: check examples + cabal install --lib scientific vector aeson --package-env . + - name: check targets working-directory: prototyping - run: ~/.cabal/bin/agda Examples.agda + run: | + ~/.cabal/bin/agda Examples.agda + ~/.cabal/bin/agda Properties.agda - name: build executables working-directory: prototyping run: | ~/.cabal/bin/agda --compile PrettyPrinter.agda ~/.cabal/bin/agda --compile Interpreter.agda - name: cmake configure - run: cmake . + if: steps.luau-ast-cache.outputs.cache-hit != 'true' + run: | + mkdir -p build + cd build + cmake build ../ - name: cmake build luau-ast - run: cmake --build . --target Luau.Ast.CLI - - name: run smoketest + if: steps.luau-ast-cache.outputs.cache-hit != 'true' + run: | + cmake --build ./build --target Luau.Ast.CLI + - name: run tests working-directory: prototyping run: | - ../luau-ast Examples/SmokeTest.lua | ./PrettyPrinter > Examples/SmokeTestOutput.lua - ../luau-ast Examples/SmokeTest.lua | ./Interpreter - - name: diff smoketest - working-directory: prototyping - run: diff Examples/SmokeTest.lua Examples/SmokeTestOutput.lua + mkdir test-failures + python tests -l ../build/luau-ast --write-diff-failures --diff-failure-location test-failures/ + - uses: actions/upload-artifact@v2 + if: failure() + with: + name: test failures + path: prototyping/test-failures + retention-days: 5 diff --git a/prototyping/.gitignore b/prototyping/.gitignore index cca4f464..10edac52 100644 --- a/prototyping/.gitignore +++ b/prototyping/.gitignore @@ -2,5 +2,11 @@ *.agdai Main MAlonzo +Examples PrettyPrinter +Interpreter +Properties +!Tests/Interpreter +!Tests/PrettyPrinter .ghc.* +test-failures/ diff --git a/prototyping/README.md b/prototyping/README.md index f1761d1c..965c9c49 100644 --- a/prototyping/README.md +++ b/prototyping/README.md @@ -25,3 +25,21 @@ and run! ``` luau-ast Examples/SmokeTest.lua | ./PrettyPrinter ``` + +## Testing + +We have a series of snapshot tests in the `Tests/` directory. You interact with the tests using the `tests` Python script in the `prototyping` directory. To simply run the tests, run: + +```sh +tests --luau-cli ../build/luau-ast --build +``` + +This will build the test targets and run them. Run `tests --help` for information about all the command-line options. + +### Adding a new test + +To add a new test, add it to `Tests/{SUITE_NAME}/{CASE_NAME}`. You'll need an `in.lua` file and an `out.txt` file. The `in.lua` file is the input Luau source code, while the `out.txt` file is the expected output after running `luau-ast in.lua | test_executable`. + +### Updating a test + +If you make a change to the prototype that results in an expected change in behavior, you might want to update the test cases automatically. To do this, run `tests` with the `--accept-new-output` (`-a` for short) flag. Rather than diffing the output, this will overwrite the `out.txt` files for each test case with the actual result. Commit the resulting changes with your PR. diff --git a/prototyping/Tests/Interpreter/return_nil/in.lua b/prototyping/Tests/Interpreter/return_nil/in.lua new file mode 100644 index 00000000..4e6ce2db --- /dev/null +++ b/prototyping/Tests/Interpreter/return_nil/in.lua @@ -0,0 +1,5 @@ +local function foo(x) + return nil +end + +return foo(nil) diff --git a/prototyping/Tests/Interpreter/return_nil/out.txt b/prototyping/Tests/Interpreter/return_nil/out.txt new file mode 100644 index 00000000..607602cf --- /dev/null +++ b/prototyping/Tests/Interpreter/return_nil/out.txt @@ -0,0 +1 @@ +nil diff --git a/prototyping/Examples/SmokeTestOutput.lua b/prototyping/Tests/PrettyPrinter/smoke_test/in.lua similarity index 72% rename from prototyping/Examples/SmokeTestOutput.lua rename to prototyping/Tests/PrettyPrinter/smoke_test/in.lua index e663534a..0d23bbb2 100644 --- a/prototyping/Examples/SmokeTestOutput.lua +++ b/prototyping/Tests/PrettyPrinter/smoke_test/in.lua @@ -1,12 +1,12 @@ local function id(x) - return x + return x end local function comp(f) - return function(g) - return function(x) - return f(g(x)) - end - end + return function(g) + return function(x) + return f(g(x)) + end + end end local id2 = comp(id)(id) local nil2 = id2(nil) diff --git a/prototyping/Examples/SmokeTest.lua b/prototyping/Tests/PrettyPrinter/smoke_test/out.txt similarity index 100% rename from prototyping/Examples/SmokeTest.lua rename to prototyping/Tests/PrettyPrinter/smoke_test/out.txt diff --git a/prototyping/tests b/prototyping/tests new file mode 100755 index 00000000..89e105d0 --- /dev/null +++ b/prototyping/tests @@ -0,0 +1,197 @@ +#!/bin/python + +import argparse +import difflib +import enum +import os +import os.path +import subprocess +import sys + +SUITES = ["interpreter", "prettyprinter"] +IN_FILE_NAME = "in.lua" +OUT_FILE_NAME = "out.txt" +SUITE_EXE_NAMES = { + "interpreter": "Interpreter", + "prettyprinter": "PrettyPrinter", +} + +SUITE_ENTRY_POINTS = { + "interpreter": "Interpreter.agda", + "prettyprinter": "PrettyPrinter.agda", +} + +SUITE_ROOTS = { + "interpreter": "Tests/Interpreter", + "prettyprinter": "Tests/PrettyPrinter", +} + +class TestResultStatus(enum.Enum): + CLI_ERROR = 0 + EXE_ERROR = 1 + DIFF_ERROR = 2 + SUCCESS = 3 + WROTE_NEW = 4 + +class DiffFailure: + def __init__(self, expected, actual): + self.expected = expected + self.actual = actual + + def diff_text(self): + diff_generator = difflib.context_diff(self.expected.splitlines(), self.actual.splitlines(), fromfile="expected", tofile="actual", n=3) + return "".join(diff_generator) + + def diff_html(self): + differ = difflib.HtmlDiff(tabsize=4) + return differ.make_file(self.expected.splitlines(), self.actual.splitlines(), fromdesc="Expected", todesc="Actual", context=True, numlines=5) + +class TestCaseResult: + def __init__(self, suite, case, status, details): + self.suite = suite + self.case = case + self.status = status + self.details = details + + def did_pass(self): + return self.status == TestResultStatus.SUCCESS or self.status == TestResultStatus.WROTE_NEW + + def to_string(self): + prefix = f"[{self.suite}/{self.case}]: " + if self.status == TestResultStatus.CLI_ERROR: + return f"{prefix}CLI ERROR: {self.details}" + elif self.status == TestResultStatus.EXE_ERROR: + return f"{prefix}EXE ERROR: {self.details}" + elif self.status == TestResultStatus.DIFF_ERROR: + text_diff = self.details.diff_text() + return f"{prefix}FAILED:\n{text_diff}" + elif self.status == TestResultStatus.SUCCESS: + return f"{prefix}SUCCEEDED" + elif self.status == TestResultStatus.WROTE_NEW: + return f"{prefix}WROTE NEW RESULT" + + def write_artifact(self, artifact_root): + if self.status != TestResultStatus.DIFF_ERROR: + return + + filename = f"{self.suite}-{self.case}.out.html" + path = os.path.join(artifact_root, filename) + html = self.details.diff_html() + with open(path, "w") as file: + file.write(html) + +parser = argparse.ArgumentParser(description="Runs prototype test cases") +parser.add_argument("--luau-cli", "-l", dest="cli_location", required=True, help="The location of luau-cli") +parser.add_argument("--root", "-r", dest="prototype_root", required=False, default=os.getcwd(), help="The root of the prototype") +parser.add_argument("--build", "-b", dest="build", action="store_true", default=True, help="Whether to automatically build required test binaries") +parser.add_argument("--suite", "-s", dest="suites", action="append", default=[], choices=SUITES, help="Which test suites to run") +parser.add_argument("--case", "-c", dest="cases", action="append", default=[], help="Which test cases to run") +parser.add_argument("--accept-new-output", "-a", dest="snapshot", action="store_true", default=False, help="Whether to write the new output to files, instead of diffing against it") +parser.add_argument("--write-diff-failures", dest="write_diffs", action="store_true", default=False, help="Whether to write test failure diffs to files") +parser.add_argument("--diff-failure-location", dest="diff_location", default=None, help="Where to write diff failure files to") + +def build_suite(root, suite): + entry_point = SUITE_ENTRY_POINTS.get(suite) + if entry_point is None: + return (False, "Invalid suite") + + result = subprocess.run(["~/.cabal/bin/agda", "--compile", entry_point], shell=True, cwd=root, stdout=subprocess.PIPE, stderr=subprocess.STDOUT) + if result.returncode == 0: + return (True, None) + else: + return (False, result.stdout) + +def run_test(in_path, out_path, cli_path, exe_path, snapshot): + cli_result = subprocess.run([cli_path, in_path], capture_output=True) + if cli_result.returncode != 0: + return (TestResultStatus.CLI_ERROR, f"CLI error: {cli_result.stderr}") + + exe_result = subprocess.run(exe_path, input=cli_result.stdout, capture_output=True) + if exe_result.returncode != 0: + return (TestResultStatus.EXE_ERROR, f"Executable error; stdout:{exe_result.stdout}\n\nstderr: {exe_result.stderr}") + actual_result = exe_result.stdout.decode("utf-8") + + if snapshot: + with open(out_path, "w") as out_file: + out_file.write(actual_result) + return (TestResultStatus.WROTE_NEW, None) + else: + with open(out_path, "r") as out_file: + expected_result = out_file.read() + + if expected_result != actual_result: + return (TestResultStatus.DIFF_ERROR, DiffFailure(expected_result, actual_result)) + + return (TestResultStatus.SUCCESS, None) + +def should_run_case(case_name, filters): + if len(filters) == 0: + return True + + return any([f in case_name for f in filters]) + +def run_test_suite(args, suite, suite_root, suite_exe): + results = [] + + for entry in os.listdir(suite_root): + if not should_run_case(entry, args.cases): + continue + + case_path = os.path.join(suite_root, entry) + if os.path.isdir(case_path): + in_path = os.path.join(case_path, IN_FILE_NAME) + out_path = os.path.join(case_path, OUT_FILE_NAME) + + if not os.path.exists(in_path) or not os.path.exists(out_path): + continue + + status, details = run_test(in_path, out_path, args.cli_location, suite_exe, args.snapshot) + result = TestCaseResult(suite, entry, status, details) + results.append(result) + + return results + +def main(): + args = parser.parse_args() + + suites = args.suites if len(args.suites) > 0 else SUITES + root = os.path.abspath(args.prototype_root) + + if args.build: + for suite in suites: + success, reason = build_suite(root, suite) + + if not success: + print(f"Error building executable for test suite {suite}:\n{reason}") + sys.exit(1) + else: + print(f"Built executable for test suite {suite} successfully.") + + failed = False + for suite in suites: + suite_root = os.path.join(root, SUITE_ROOTS.get(suite)) + suite_exe = os.path.join(root, SUITE_EXE_NAMES.get(suite)) + print(f"Running test suite {suite}...") + results = run_test_suite(args, suite, suite_root, suite_exe) + + passed = 0 + total = len(results) + + for result in results: + if result.did_pass(): + passed += 1 + else: + failed = True + + print(f"Suite {suite} [{passed} / {total} passed]:") + for result in results: + print(result.to_string()) + + if args.write_diffs: + result.write_artifact(args.diff_location) + + if failed: + sys.exit(1) + +if __name__ == "__main__": + main() From 5b784650594ffbef70c731fb6c9cef136226121c Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Thu, 17 Feb 2022 17:18:01 -0800 Subject: [PATCH 170/399] Sync to upstream/release/514 (#372) --- Analysis/include/Luau/Documentation.h | 3 + Analysis/include/Luau/Frontend.h | 3 +- Analysis/include/Luau/Linter.h | 7 +- Analysis/include/Luau/Module.h | 4 +- Analysis/include/Luau/Quantify.h | 5 +- Analysis/include/Luau/Substitution.h | 20 +- Analysis/include/Luau/TypeInfer.h | 45 +++- Analysis/src/Autocomplete.cpp | 45 ++-- Analysis/src/Config.cpp | 2 +- Analysis/src/EmbeddedBuiltinDefinitions.cpp | 2 +- Analysis/src/Frontend.cpp | 35 ++- Analysis/src/JsonEncoder.cpp | 21 +- Analysis/src/Linter.cpp | 126 ++++++++- Analysis/src/Quantify.cpp | 10 +- Analysis/src/Substitution.cpp | 23 -- Analysis/src/TopoSortStatements.cpp | 3 +- Analysis/src/Transpiler.cpp | 8 +- Analysis/src/TypeAttach.cpp | 4 +- Analysis/src/TypeInfer.cpp | 142 +++-------- Analysis/src/TypeUtils.cpp | 12 + Analysis/src/TypeVar.cpp | 6 +- Analysis/src/TypedAllocator.cpp | 3 + Analysis/src/Unifier.cpp | 16 +- Ast/include/Luau/Ast.h | 17 +- Ast/include/Luau/Lexer.h | 5 + Ast/include/Luau/ParseResult.h | 69 +++++ Ast/include/Luau/Parser.h | 56 +--- Ast/src/Ast.cpp | 29 +-- Ast/src/Lexer.cpp | 12 +- Ast/src/Parser.cpp | 167 +++++++----- Ast/src/TimeTrace.cpp | 6 + CLI/FileUtils.cpp | 7 +- CLI/Repl.cpp | 171 +++++++++---- CMakeLists.txt | 14 + Sources.cmake | 1 + VM/src/lapi.cpp | 25 +- VM/src/ldebug.cpp | 2 +- VM/src/ldo.cpp | 13 +- VM/src/ldo.h | 2 +- VM/src/lgc.cpp | 4 +- VM/src/lgc.h | 2 +- VM/src/lgcdebug.cpp | 10 +- VM/src/lperf.cpp | 6 + VM/src/lstate.cpp | 35 ++- VM/src/lstate.h | 8 +- VM/src/lvmexecute.cpp | 2 +- VM/src/lvmload.cpp | 6 +- VM/src/lvmutils.cpp | 4 +- fuzz/linter.cpp | 2 +- fuzz/proto.cpp | 2 +- tests/Autocomplete.test.cpp | 22 +- tests/Compiler.test.cpp | 14 +- tests/Conformance.test.cpp | 8 + tests/Fixture.cpp | 4 +- tests/Fixture.h | 1 - tests/Frontend.test.cpp | 3 - tests/Linter.test.cpp | 48 +++- tests/NonstrictMode.test.cpp | 1 - tests/Parser.test.cpp | 51 ++-- tests/Repl.test.cpp | 209 ++++++++++++++- tests/RequireTracer.test.cpp | 2 +- tests/TypeInfer.aliases.test.cpp | 6 +- tests/TypeInfer.annotations.test.cpp | 1 - tests/TypeInfer.builtins.test.cpp | 1 - tests/TypeInfer.classes.test.cpp | 1 - tests/TypeInfer.definitions.test.cpp | 1 - tests/TypeInfer.generics.test.cpp | 1 - tests/TypeInfer.intersectionTypes.test.cpp | 1 - tests/TypeInfer.provisional.test.cpp | 1 - tests/TypeInfer.singletons.test.cpp | 2 - tests/TypeInfer.tables.test.cpp | 43 +++- tests/TypeInfer.test.cpp | 27 +- tests/TypeInfer.tryUnify.test.cpp | 1 - tests/TypeInfer.typePacks.cpp | 3 - tests/TypeInfer.unionTypes.test.cpp | 1 - tests/TypePack.test.cpp | 1 - tests/TypeVar.test.cpp | 1 - tests/conformance/coroutine.lua | 9 + tests/conformance/coverage.lua | 8 + tests/conformance/debug.lua | 3 + tests/main.cpp | 7 +- tools/natvis/Analysis.natvis | 78 ++++++ tools/natvis/Ast.natvis | 25 ++ tools/natvis/VM.natvis | 269 ++++++++++++++++++++ 84 files changed, 1497 insertions(+), 579 deletions(-) create mode 100644 Ast/include/Luau/ParseResult.h create mode 100644 tools/natvis/Analysis.natvis create mode 100644 tools/natvis/Ast.natvis create mode 100644 tools/natvis/VM.natvis diff --git a/Analysis/include/Luau/Documentation.h b/Analysis/include/Luau/Documentation.h index 68ff3a7c..7a2b56ff 100644 --- a/Analysis/include/Luau/Documentation.h +++ b/Analysis/include/Luau/Documentation.h @@ -21,6 +21,7 @@ struct BasicDocumentation { std::string documentation; std::string learnMoreLink; + std::string codeSample; }; struct FunctionParameterDocumentation @@ -37,6 +38,7 @@ struct FunctionDocumentation std::vector parameters; std::vector returns; std::string learnMoreLink; + std::string codeSample; }; struct OverloadedFunctionDocumentation @@ -52,6 +54,7 @@ struct TableDocumentation std::string documentation; Luau::DenseHashMap keys; std::string learnMoreLink; + std::string codeSample; }; using DocumentationDatabase = Luau::DenseHashMap; diff --git a/Analysis/include/Luau/Frontend.h b/Analysis/include/Luau/Frontend.h index 1f64db30..0bf8f362 100644 --- a/Analysis/include/Luau/Frontend.h +++ b/Analysis/include/Luau/Frontend.h @@ -24,6 +24,7 @@ struct TypeChecker; struct FileResolver; struct ModuleResolver; struct ParseResult; +struct HotComment; struct LoadDefinitionFileResult { @@ -35,7 +36,7 @@ struct LoadDefinitionFileResult LoadDefinitionFileResult loadDefinitionFile( TypeChecker& typeChecker, ScopePtr targetScope, std::string_view definition, const std::string& packageName); -std::optional parseMode(const std::vector& hotcomments); +std::optional parseMode(const std::vector& hotcomments); std::vector parsePathExpr(const AstExpr& pathExpr); diff --git a/Analysis/include/Luau/Linter.h b/Analysis/include/Luau/Linter.h index ec3c124d..6c7ce47f 100644 --- a/Analysis/include/Luau/Linter.h +++ b/Analysis/include/Luau/Linter.h @@ -14,6 +14,7 @@ class AstStat; class AstNameTable; struct TypeChecker; struct Module; +struct HotComment; using ScopePtr = std::shared_ptr; @@ -50,6 +51,7 @@ struct LintWarning Code_TableOperations = 23, Code_DuplicateCondition = 24, Code_MisleadingAndOr = 25, + Code_CommentDirective = 26, Code__Count }; @@ -60,7 +62,7 @@ struct LintWarning static const char* getName(Code code); static Code parseName(const char* name); - static uint64_t parseMask(const std::vector& hotcomments); + static uint64_t parseMask(const std::vector& hotcomments); }; struct LintResult @@ -90,7 +92,8 @@ struct LintOptions void setDefaults(); }; -std::vector lint(AstStat* root, const AstNameTable& names, const ScopePtr& env, const Module* module, const LintOptions& options); +std::vector lint(AstStat* root, const AstNameTable& names, const ScopePtr& env, const Module* module, + const std::vector& hotcomments, const LintOptions& options); std::vector getDeprecatedGlobals(const AstNameTable& names); diff --git a/Analysis/include/Luau/Module.h b/Analysis/include/Luau/Module.h index 1bf0473c..61200771 100644 --- a/Analysis/include/Luau/Module.h +++ b/Analysis/include/Luau/Module.h @@ -6,7 +6,7 @@ #include "Luau/TypedAllocator.h" #include "Luau/ParseOptions.h" #include "Luau/Error.h" -#include "Luau/Parser.h" +#include "Luau/ParseResult.h" #include #include @@ -37,8 +37,8 @@ struct SourceModule AstStatBlock* root = nullptr; std::optional mode; - uint64_t ignoreLints = 0; + std::vector hotcomments; std::vector commentLocations; SourceModule() diff --git a/Analysis/include/Luau/Quantify.h b/Analysis/include/Luau/Quantify.h index f46df146..e48cad40 100644 --- a/Analysis/include/Luau/Quantify.h +++ b/Analysis/include/Luau/Quantify.h @@ -6,9 +6,6 @@ namespace Luau { -struct Module; -using ModulePtr = std::shared_ptr; - -void quantify(ModulePtr module, TypeId ty, TypeLevel level); +void quantify(TypeId ty, TypeLevel level); } // namespace Luau diff --git a/Analysis/include/Luau/Substitution.h b/Analysis/include/Luau/Substitution.h index f85b4269..9662d5b3 100644 --- a/Analysis/include/Luau/Substitution.h +++ b/Analysis/include/Luau/Substitution.h @@ -101,9 +101,6 @@ struct Tarjan // This is hot code, so we optimize recursion to a stack. TarjanResult loop(); - // Clear the state - void clear(); - // Find or create the index for a vertex. // Return a boolean which is `true` if it's a freshly created index. std::pair indexify(TypeId ty); @@ -166,7 +163,17 @@ struct FindDirty : Tarjan // and replaces them with clean ones. struct Substitution : FindDirty { - ModulePtr currentModule; +protected: + Substitution(const TxnLog* log_, TypeArena* arena) + : arena(arena) + { + log = log_; + LUAU_ASSERT(log); + LUAU_ASSERT(arena); + } + +public: + TypeArena* arena; DenseHashMap newTypes{nullptr}; DenseHashMap newPacks{nullptr}; @@ -192,12 +199,13 @@ struct Substitution : FindDirty template TypeId addType(const T& tv) { - return currentModule->internalTypes.addType(tv); + return arena->addType(tv); } + template TypePackId addTypePack(const T& tp) { - return currentModule->internalTypes.addTypePack(TypePackVar{tp}); + return arena->addTypePack(TypePackVar{tp}); } }; diff --git a/Analysis/include/Luau/TypeInfer.h b/Analysis/include/Luau/TypeInfer.h index 5592fa1f..3c5ded3c 100644 --- a/Analysis/include/Luau/TypeInfer.h +++ b/Analysis/include/Luau/TypeInfer.h @@ -5,7 +5,6 @@ #include "Luau/Error.h" #include "Luau/Module.h" #include "Luau/Symbol.h" -#include "Luau/Parser.h" #include "Luau/Substitution.h" #include "Luau/TxnLog.h" #include "Luau/TypePack.h" @@ -37,6 +36,15 @@ struct Unifier; // A substitution which replaces generic types in a given set by free types. struct ReplaceGenerics : Substitution { + ReplaceGenerics( + const TxnLog* log, TypeArena* arena, TypeLevel level, const std::vector& generics, const std::vector& genericPacks) + : Substitution(log, arena) + , level(level) + , generics(generics) + , genericPacks(genericPacks) + { + } + TypeLevel level; std::vector generics; std::vector genericPacks; @@ -50,8 +58,13 @@ struct ReplaceGenerics : Substitution // A substitution which replaces generic functions by monomorphic functions struct Instantiation : Substitution { + Instantiation(const TxnLog* log, TypeArena* arena, TypeLevel level) + : Substitution(log, arena) + , level(level) + { + } + TypeLevel level; - ReplaceGenerics replaceGenerics; bool ignoreChildren(TypeId ty) override; bool isDirty(TypeId ty) override; bool isDirty(TypePackId tp) override; @@ -62,6 +75,12 @@ struct Instantiation : Substitution // A substitution which replaces free types by generic types. struct Quantification : Substitution { + Quantification(TypeArena* arena, TypeLevel level) + : Substitution(TxnLog::empty(), arena) + , level(level) + { + } + TypeLevel level; std::vector generics; std::vector genericPacks; @@ -74,6 +93,13 @@ struct Quantification : Substitution // A substitution which replaces free types by any struct Anyification : Substitution { + Anyification(TypeArena* arena, TypeId anyType, TypePackId anyTypePack) + : Substitution(TxnLog::empty(), arena) + , anyType(anyType) + , anyTypePack(anyTypePack) + { + } + TypeId anyType; TypePackId anyTypePack; bool isDirty(TypeId ty) override; @@ -85,6 +111,13 @@ struct Anyification : Substitution // A substitution which replaces the type parameters of a type function by arguments struct ApplyTypeFunction : Substitution { + ApplyTypeFunction(TypeArena* arena, TypeLevel level) + : Substitution(TxnLog::empty(), arena) + , level(level) + , encounteredForwardedType(false) + { + } + TypeLevel level; bool encounteredForwardedType; std::unordered_map typeArguments; @@ -351,8 +384,7 @@ private: // Note: `scope` must be a fresh scope. GenericTypeDefinitions createGenericTypes(const ScopePtr& scope, std::optional levelOpt, const AstNode& node, - const AstArray& genericNames, const AstArray& genericPackNames, - bool useCache = false); + const AstArray& genericNames, const AstArray& genericPackNames, bool useCache = false); public: ErrorVec resolve(const PredicateVec& predicates, const ScopePtr& scope, bool sense); @@ -392,11 +424,6 @@ public: ModulePtr currentModule; ModuleName currentModuleName; - Instantiation instantiation; - Quantification quantification; - Anyification anyification; - ApplyTypeFunction applyTypeFunction; - std::function prepareModuleScope; InternalErrorReporter* iceHandler; diff --git a/Analysis/src/Autocomplete.cpp b/Analysis/src/Autocomplete.cpp index 85099e12..5a1ae397 100644 --- a/Analysis/src/Autocomplete.cpp +++ b/Analysis/src/Autocomplete.cpp @@ -7,6 +7,7 @@ #include "Luau/ToString.h" #include "Luau/TypeInfer.h" #include "Luau/TypePack.h" +#include "Luau/Parser.h" // TODO: only needed for autocompleteSource which is deprecated #include #include @@ -14,9 +15,9 @@ LUAU_FASTFLAG(LuauUseCommittingTxnLog) LUAU_FASTFLAGVARIABLE(LuauAutocompleteAvoidMutation, false); -LUAU_FASTFLAGVARIABLE(LuauCompleteBrokenStringParams, false); LUAU_FASTFLAGVARIABLE(LuauMissingFollowACMetatables, false); LUAU_FASTFLAGVARIABLE(PreferToCallFunctionsForIntersects, false); +LUAU_FASTFLAGVARIABLE(LuauIfElseExprFixCompletionIssue, false); static const std::unordered_set kStatementStartingKeywords = { "while", "if", "local", "repeat", "function", "do", "for", "return", "break", "continue", "type", "export"}; @@ -380,7 +381,7 @@ static void autocompleteProps(const Module& module, TypeArena* typeArena, TypeId { // We are walking up the class hierarchy, so if we encounter a property that we have // already populated, it takes precedence over the property we found just now. - if (result.count(name) == 0 && name != Parser::errorName) + if (result.count(name) == 0 && name != kParseNameError) { Luau::TypeId type = Luau::follow(prop.type); TypeCorrectKind typeCorrect = indexType == PropIndexType::Key ? TypeCorrectKind::Correct @@ -948,9 +949,12 @@ AutocompleteEntryMap autocompleteTypeNames(const Module& module, Position positi } } - for (size_t i = 0; i < node->returnAnnotation.types.size; i++) + if (!node->returnAnnotation) + return result; + + for (size_t i = 0; i < node->returnAnnotation->types.size; i++) { - AstType* ret = node->returnAnnotation.types.data[i]; + AstType* ret = node->returnAnnotation->types.data[i]; if (ret->location.containsClosed(position)) { @@ -965,7 +969,7 @@ AutocompleteEntryMap autocompleteTypeNames(const Module& module, Position positi } } - if (AstTypePack* retTp = node->returnAnnotation.tailType) + if (AstTypePack* retTp = node->returnAnnotation->tailType) { if (auto variadic = retTp->as()) { @@ -1136,7 +1140,7 @@ static AutocompleteEntryMap autocompleteStatement( AstNode* parent = ancestry.rbegin()[1]; if (AstStatIf* statIf = parent->as()) { - if (!statIf->elsebody || (statIf->hasElse && statIf->elseLocation.containsClosed(position))) + if (!statIf->elsebody || (statIf->elseLocation && statIf->elseLocation->containsClosed(position))) { result.emplace("else", AutocompleteEntry{AutocompleteEntryKind::Keyword}); result.emplace("elseif", AutocompleteEntry{AutocompleteEntryKind::Keyword}); @@ -1164,8 +1168,7 @@ static AutocompleteEntryMap autocompleteStatement( return result; } -// Returns true if completions were generated (completions will be inserted into 'outResult') -// Returns false if no completions were generated +// Returns true iff `node` was handled by this function (completions, if any, are returned in `outResult`) static bool autocompleteIfElseExpression( const AstNode* node, const std::vector& ancestry, const Position& position, AutocompleteEntryMap& outResult) { @@ -1173,6 +1176,13 @@ static bool autocompleteIfElseExpression( if (!parent) return false; + if (FFlag::LuauIfElseExprFixCompletionIssue && node->is()) + { + // Don't try to complete when the current node is an if-else expression (i.e. only try to complete when the node is a child of an if-else + // expression. + return true; + } + AstExprIfElse* ifElseExpr = parent->as(); if (!ifElseExpr || ifElseExpr->condition->location.containsClosed(position)) { @@ -1310,7 +1320,7 @@ static std::optional autocompleteStringParams(const Source return std::nullopt; } - if (!nodes.back()->is() && (!FFlag::LuauCompleteBrokenStringParams || !nodes.back()->is())) + if (!nodes.back()->is() && !nodes.back()->is()) { return std::nullopt; } @@ -1408,8 +1418,8 @@ static AutocompleteResult autocomplete(const SourceModule& sourceModule, const M } else if (auto typeReference = node->as()) { - if (typeReference->hasPrefix) - return {autocompleteModuleTypes(*module, position, typeReference->prefix.value), finder.ancestry}; + if (typeReference->prefix) + return {autocompleteModuleTypes(*module, position, typeReference->prefix->value), finder.ancestry}; else return {autocompleteTypeNames(*module, position, finder.ancestry), finder.ancestry}; } @@ -1419,9 +1429,9 @@ static AutocompleteResult autocomplete(const SourceModule& sourceModule, const M } else if (AstStatLocal* statLocal = node->as()) { - if (statLocal->vars.size == 1 && (!statLocal->hasEqualsSign || position < statLocal->equalsSignLocation.begin)) + if (statLocal->vars.size == 1 && (!statLocal->equalsSignLocation || position < statLocal->equalsSignLocation->begin)) return {{{"function", AutocompleteEntry{AutocompleteEntryKind::Keyword}}}, finder.ancestry}; - else if (statLocal->hasEqualsSign && position >= statLocal->equalsSignLocation.end) + else if (statLocal->equalsSignLocation && position >= statLocal->equalsSignLocation->end) return {autocompleteExpression(sourceModule, *module, typeChecker, typeArena, finder.ancestry, position), finder.ancestry}; else return {}; @@ -1449,7 +1459,7 @@ static AutocompleteResult autocomplete(const SourceModule& sourceModule, const M if (!statForIn->hasIn || position <= statForIn->inLocation.begin) { AstLocal* lastName = statForIn->vars.data[statForIn->vars.size - 1]; - if (lastName->name == Parser::errorName || lastName->location.containsClosed(position)) + if (lastName->name == kParseNameError || lastName->location.containsClosed(position)) { // Here we are either working with a missing binding (as would be the case in a bare "for" keyword) or // the cursor is still touching a binding name. The user is still typing a new name, so we should not offer @@ -1499,7 +1509,7 @@ static AutocompleteResult autocomplete(const SourceModule& sourceModule, const M else if (AstStatWhile* statWhile = extractStat(finder.ancestry); statWhile && !statWhile->hasDo) return {{{"do", AutocompleteEntry{AutocompleteEntryKind::Keyword}}}, finder.ancestry}; - else if (AstStatIf* statIf = node->as(); statIf && !statIf->hasElse) + else if (AstStatIf* statIf = node->as(); statIf && !statIf->elseLocation.has_value()) { return {{{"else", AutocompleteEntry{AutocompleteEntryKind::Keyword}}, {"elseif", AutocompleteEntry{AutocompleteEntryKind::Keyword}}}, finder.ancestry}; @@ -1508,11 +1518,11 @@ static AutocompleteResult autocomplete(const SourceModule& sourceModule, const M { if (statIf->condition->is()) return {autocompleteExpression(sourceModule, *module, typeChecker, typeArena, finder.ancestry, position), finder.ancestry}; - else if (!statIf->hasThen || statIf->thenLocation.containsClosed(position)) + else if (!statIf->thenLocation || statIf->thenLocation->containsClosed(position)) return {{{"then", AutocompleteEntry{AutocompleteEntryKind::Keyword}}}, finder.ancestry}; } else if (AstStatIf* statIf = extractStat(finder.ancestry); - statIf && (!statIf->hasThen || statIf->thenLocation.containsClosed(position))) + statIf && (!statIf->thenLocation || statIf->thenLocation->containsClosed(position))) return {{{"then", AutocompleteEntry{AutocompleteEntryKind::Keyword}}}, finder.ancestry}; else if (AstStatRepeat* statRepeat = node->as(); statRepeat && statRepeat->condition->is()) return {autocompleteExpression(sourceModule, *module, typeChecker, typeArena, finder.ancestry, position), finder.ancestry}; @@ -1612,6 +1622,7 @@ AutocompleteResult autocomplete(Frontend& frontend, const ModuleName& moduleName OwningAutocompleteResult autocompleteSource(Frontend& frontend, std::string_view source, Position position, StringCompletionCallback callback) { + // TODO: Remove #include "Luau/Parser.h" with this function auto sourceModule = std::make_unique(); ParseOptions parseOptions; parseOptions.captureComments = true; diff --git a/Analysis/src/Config.cpp b/Analysis/src/Config.cpp index d9fc44f8..35a2259d 100644 --- a/Analysis/src/Config.cpp +++ b/Analysis/src/Config.cpp @@ -1,7 +1,7 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #include "Luau/Config.h" -#include "Luau/Parser.h" +#include "Luau/Lexer.h" #include "Luau/StringUtils.h" namespace diff --git a/Analysis/src/EmbeddedBuiltinDefinitions.cpp b/Analysis/src/EmbeddedBuiltinDefinitions.cpp index f3ef88fc..bf6e1193 100644 --- a/Analysis/src/EmbeddedBuiltinDefinitions.cpp +++ b/Analysis/src/EmbeddedBuiltinDefinitions.cpp @@ -167,7 +167,7 @@ declare function gcinfo(): number foreach: ({[K]: V}, (K, V) -> ()) -> (), foreachi: ({V}, (number, V) -> ()) -> (), - move: ({V}, number, number, number, {V}?) -> (), + move: ({V}, number, number, number, {V}?) -> {V}, clear: ({[K]: V}) -> (), freeze: ({[K]: V}) -> {[K]: V}, diff --git a/Analysis/src/Frontend.cpp b/Analysis/src/Frontend.cpp index 9001b19d..d8906f6e 100644 --- a/Analysis/src/Frontend.cpp +++ b/Analysis/src/Frontend.cpp @@ -4,6 +4,7 @@ #include "Luau/Common.h" #include "Luau/Config.h" #include "Luau/FileResolver.h" +#include "Luau/Parser.h" #include "Luau/Scope.h" #include "Luau/StringUtils.h" #include "Luau/TimeTrace.h" @@ -16,23 +17,25 @@ #include LUAU_FASTFLAG(LuauInferInNoCheckMode) -LUAU_FASTFLAGVARIABLE(LuauTypeCheckTwice, false) LUAU_FASTFLAGVARIABLE(LuauKnowsTheDataModel3, false) namespace Luau { -std::optional parseMode(const std::vector& hotcomments) +std::optional parseMode(const std::vector& hotcomments) { - for (const std::string& hc : hotcomments) + for (const HotComment& hc : hotcomments) { - if (hc == "nocheck") + if (!hc.header) + continue; + + if (hc.content == "nocheck") return Mode::NoCheck; - if (hc == "nonstrict") + if (hc.content == "nonstrict") return Mode::Nonstrict; - if (hc == "strict") + if (hc.content == "strict") return Mode::Strict; } @@ -607,13 +610,15 @@ std::pair Frontend::lintFragment(std::string_view sour SourceModule sourceModule = parse(ModuleName{}, source, config.parseOptions); + uint64_t ignoreLints = LintWarning::parseMask(sourceModule.hotcomments); + Luau::LintOptions lintOptions = enabledLintWarnings.value_or(config.enabledLint); - lintOptions.warningMask &= sourceModule.ignoreLints; + lintOptions.warningMask &= ~ignoreLints; double timestamp = getTimestamp(); - std::vector warnings = - Luau::lint(sourceModule.root, *sourceModule.names.get(), typeChecker.globalScope, nullptr, enabledLintWarnings.value_or(config.enabledLint)); + std::vector warnings = Luau::lint(sourceModule.root, *sourceModule.names.get(), typeChecker.globalScope, nullptr, + sourceModule.hotcomments, enabledLintWarnings.value_or(config.enabledLint)); stats.timeLint += getTimestamp() - timestamp; @@ -651,8 +656,10 @@ LintResult Frontend::lint(const SourceModule& module, std::optionalgetConfig(module.name); + uint64_t ignoreLints = LintWarning::parseMask(module.hotcomments); + LintOptions options = enabledLintWarnings.value_or(config.enabledLint); - options.warningMask &= ~module.ignoreLints; + options.warningMask &= ~ignoreLints; Mode mode = module.mode.value_or(config.mode); if (mode != Mode::NoCheck) @@ -671,7 +678,7 @@ LintResult Frontend::lint(const SourceModule& module, std::optional warnings = Luau::lint(module.root, *module.names, environmentScope, modulePtr.get(), options); + std::vector warnings = Luau::lint(module.root, *module.names, environmentScope, modulePtr.get(), module.hotcomments, options); stats.timeLint += getTimestamp() - timestamp; @@ -839,7 +846,6 @@ SourceModule Frontend::parse(const ModuleName& name, std::string_view src, const { sourceModule.root = parseResult.root; sourceModule.mode = parseMode(parseResult.hotcomments); - sourceModule.ignoreLints = LintWarning::parseMask(parseResult.hotcomments); } else { @@ -848,8 +854,13 @@ SourceModule Frontend::parse(const ModuleName& name, std::string_view src, const } sourceModule.name = name; + if (parseOptions.captureComments) + { sourceModule.commentLocations = std::move(parseResult.commentLocations); + sourceModule.hotcomments = std::move(parseResult.hotcomments); + } + return sourceModule; } diff --git a/Analysis/src/JsonEncoder.cpp b/Analysis/src/JsonEncoder.cpp index 0d48f2ca..ec399158 100644 --- a/Analysis/src/JsonEncoder.cpp +++ b/Analysis/src/JsonEncoder.cpp @@ -158,6 +158,13 @@ struct AstJsonEncoder : public AstVisitor { writeString(str); } + void write(std::optional name) + { + if (name) + write(*name); + else + writeRaw("null"); + } void write(AstName name) { writeString(name.value ? name.value : ""); @@ -327,7 +334,7 @@ struct AstJsonEncoder : public AstVisitor if (node->self) PROP(self); PROP(args); - if (node->hasReturnAnnotation) + if (node->returnAnnotation) PROP(returnAnnotation); PROP(vararg); PROP(varargLocation); @@ -341,6 +348,14 @@ struct AstJsonEncoder : public AstVisitor }); } + void write(const std::optional& typeList) + { + if (typeList) + write(*typeList); + else + writeRaw("null"); + } + void write(const AstTypeList& typeList) { writeRaw("{"); @@ -544,7 +559,7 @@ struct AstJsonEncoder : public AstVisitor PROP(thenbody); if (node->elsebody) PROP(elsebody); - PROP(hasThen); + write("hasThen", node->thenLocation.has_value()); PROP(hasEnd); }); } @@ -728,7 +743,7 @@ struct AstJsonEncoder : public AstVisitor void write(class AstTypeReference* node) { writeNode(node, "AstTypeReference", [&]() { - if (node->hasPrefix) + if (node->prefix) PROP(prefix); PROP(name); PROP(parameters); diff --git a/Analysis/src/Linter.cpp b/Analysis/src/Linter.cpp index 2ba6a0fc..8d7d2d97 100644 --- a/Analysis/src/Linter.cpp +++ b/Analysis/src/Linter.cpp @@ -12,6 +12,8 @@ #include #include +LUAU_FASTINTVARIABLE(LuauSuggestionDistance, 4) + namespace Luau { @@ -44,6 +46,7 @@ static const char* kWarningNames[] = { "TableOperations", "DuplicateCondition", "MisleadingAndOr", + "CommentDirective", }; // clang-format on @@ -732,13 +735,13 @@ private: bool visit(AstTypeReference* node) override { - if (!node->hasPrefix) + if (!node->prefix) return true; - if (!imports.contains(node->prefix)) + if (!imports.contains(*node->prefix)) return true; - AstLocal* astLocal = imports[node->prefix]; + AstLocal* astLocal = imports[*node->prefix]; Local& local = locals[astLocal]; LUAU_ASSERT(local.import); local.used = true; @@ -2527,13 +2530,108 @@ static void fillBuiltinGlobals(LintContext& context, const AstNameTable& names, } } +static const char* fuzzyMatch(std::string_view str, const char** array, size_t size) +{ + if (FInt::LuauSuggestionDistance == 0) + return nullptr; + + size_t bestDistance = FInt::LuauSuggestionDistance; + size_t bestMatch = size; + + for (size_t i = 0; i < size; ++i) + { + size_t ed = editDistance(str, array[i]); + + if (ed <= bestDistance) + { + bestDistance = ed; + bestMatch = i; + } + } + + return bestMatch < size ? array[bestMatch] : nullptr; +} + +static void lintComments(LintContext& context, const std::vector& hotcomments) +{ + bool seenMode = false; + + for (const HotComment& hc : hotcomments) + { + // We reserve --! for various informational (non-directive) comments + if (hc.content.empty() || hc.content[0] == ' ' || hc.content[0] == '\t') + continue; + + if (!hc.header) + { + emitWarning(context, LintWarning::Code_CommentDirective, hc.location, + "Comment directive is ignored because it is placed after the first non-comment token"); + } + else + { + std::string::size_type space = hc.content.find_first_of(" \t"); + std::string_view first = std::string_view(hc.content).substr(0, space); + + if (first == "nolint") + { + std::string::size_type notspace = hc.content.find_first_not_of(" \t", space); + + if (space == std::string::npos || notspace == std::string::npos) + { + // disables all lints + } + else if (LintWarning::parseName(hc.content.c_str() + notspace) == LintWarning::Code_Unknown) + { + const char* rule = hc.content.c_str() + notspace; + + // skip Unknown + if (const char* suggestion = fuzzyMatch(rule, kWarningNames + 1, LintWarning::Code__Count - 1)) + emitWarning(context, LintWarning::Code_CommentDirective, hc.location, + "nolint directive refers to unknown lint rule '%s'; did you mean '%s'?", rule, suggestion); + else + emitWarning( + context, LintWarning::Code_CommentDirective, hc.location, "nolint directive refers to unknown lint rule '%s'", rule); + } + } + else if (first == "nocheck" || first == "nonstrict" || first == "strict") + { + if (space != std::string::npos) + emitWarning(context, LintWarning::Code_CommentDirective, hc.location, + "Comment directive with the type checking mode has extra symbols at the end of the line"); + else if (seenMode) + emitWarning(context, LintWarning::Code_CommentDirective, hc.location, + "Comment directive with the type checking mode has already been used"); + else + seenMode = true; + } + else + { + static const char* kHotComments[] = { + "nolint", + "nocheck", + "nonstrict", + "strict", + }; + + if (const char* suggestion = fuzzyMatch(first, kHotComments, std::size(kHotComments))) + emitWarning(context, LintWarning::Code_CommentDirective, hc.location, "Unknown comment directive '%.*s'; did you mean '%s'?", + int(first.size()), first.data(), suggestion); + else + emitWarning(context, LintWarning::Code_CommentDirective, hc.location, "Unknown comment directive '%.*s'", int(first.size()), + first.data()); + } + } + } +} + void LintOptions::setDefaults() { // By default, we enable all warnings warningMask = ~0ull; } -std::vector lint(AstStat* root, const AstNameTable& names, const ScopePtr& env, const Module* module, const LintOptions& options) +std::vector lint(AstStat* root, const AstNameTable& names, const ScopePtr& env, const Module* module, + const std::vector& hotcomments, const LintOptions& options) { LintContext context; @@ -2609,6 +2707,9 @@ std::vector lint(AstStat* root, const AstNameTable& names, const Sc if (context.warningEnabled(LintWarning::Code_MisleadingAndOr)) LintMisleadingAndOr::process(context); + if (context.warningEnabled(LintWarning::Code_CommentDirective)) + lintComments(context, hotcomments); + std::sort(context.result.begin(), context.result.end(), WarningComparator()); return context.result; @@ -2630,23 +2731,30 @@ LintWarning::Code LintWarning::parseName(const char* name) return Code_Unknown; } -uint64_t LintWarning::parseMask(const std::vector& hotcomments) +uint64_t LintWarning::parseMask(const std::vector& hotcomments) { uint64_t result = 0; - for (const std::string& hc : hotcomments) + for (const HotComment& hc : hotcomments) { - if (hc.compare(0, 6, "nolint") != 0) + if (!hc.header) continue; - std::string::size_type name = hc.find_first_not_of(" \t", 6); + if (hc.content.compare(0, 6, "nolint") != 0) + continue; + + std::string::size_type name = hc.content.find_first_not_of(" \t", 6); // --!nolint disables everything if (name == std::string::npos) return ~0ull; + // --!nolint needs to be followed by a whitespace character + if (name == 6) + continue; + // --!nolint name disables the specific lint - LintWarning::Code code = LintWarning::parseName(hc.c_str() + name); + LintWarning::Code code = LintWarning::parseName(hc.content.c_str() + name); if (code != LintWarning::Code_Unknown) result |= 1ull << int(code); diff --git a/Analysis/src/Quantify.cpp b/Analysis/src/Quantify.cpp index 04ebffc1..94e169f1 100644 --- a/Analysis/src/Quantify.cpp +++ b/Analysis/src/Quantify.cpp @@ -9,14 +9,12 @@ namespace Luau struct Quantifier { - ModulePtr module; TypeLevel level; std::vector generics; std::vector genericPacks; - Quantifier(ModulePtr module, TypeLevel level) - : module(module) - , level(level) + Quantifier(TypeLevel level) + : level(level) { } @@ -76,9 +74,9 @@ struct Quantifier } }; -void quantify(ModulePtr module, TypeId ty, TypeLevel level) +void quantify(TypeId ty, TypeLevel level) { - Quantifier q{std::move(module), level}; + Quantifier q{level}; DenseHashSet seen{nullptr}; visitTypeVarOnce(ty, q, seen); diff --git a/Analysis/src/Substitution.cpp b/Analysis/src/Substitution.cpp index bacbca76..770c7a47 100644 --- a/Analysis/src/Substitution.cpp +++ b/Analysis/src/Substitution.cpp @@ -226,27 +226,11 @@ TarjanResult Tarjan::loop() return TarjanResult::Ok; } -void Tarjan::clear() -{ - typeToIndex.clear(); - indexToType.clear(); - packToIndex.clear(); - indexToPack.clear(); - lowlink.clear(); - stack.clear(); - onStack.clear(); - - edgesTy.clear(); - edgesTp.clear(); - worklist.clear(); -} - TarjanResult Tarjan::visitRoot(TypeId ty) { childCount = 0; ty = log->follow(ty); - clear(); auto [index, fresh] = indexify(ty); worklist.push_back({index, -1, -1}); return loop(); @@ -257,7 +241,6 @@ TarjanResult Tarjan::visitRoot(TypePackId tp) childCount = 0; tp = log->follow(tp); - clear(); auto [index, fresh] = indexify(tp); worklist.push_back({index, -1, -1}); return loop(); @@ -314,21 +297,17 @@ void FindDirty::visitSCC(int index) TarjanResult FindDirty::findDirty(TypeId ty) { - dirty.clear(); return visitRoot(ty); } TarjanResult FindDirty::findDirty(TypePackId tp) { - dirty.clear(); return visitRoot(tp); } std::optional Substitution::substitute(TypeId ty) { ty = log->follow(ty); - newTypes.clear(); - newPacks.clear(); auto result = findDirty(ty); if (result != TarjanResult::Ok) @@ -347,8 +326,6 @@ std::optional Substitution::substitute(TypeId ty) std::optional Substitution::substitute(TypePackId tp) { tp = log->follow(tp); - newTypes.clear(); - newPacks.clear(); auto result = findDirty(tp); if (result != TarjanResult::Ok) diff --git a/Analysis/src/TopoSortStatements.cpp b/Analysis/src/TopoSortStatements.cpp index dba694be..678001bf 100644 --- a/Analysis/src/TopoSortStatements.cpp +++ b/Analysis/src/TopoSortStatements.cpp @@ -26,9 +26,10 @@ * 3. Cyclic dependencies can be resolved by picking an arbitrary statement to check first. */ -#include "Luau/Parser.h" +#include "Luau/Ast.h" #include "Luau/DenseHash.h" #include "Luau/Common.h" +#include "Luau/StringUtils.h" #include #include diff --git a/Analysis/src/Transpiler.cpp b/Analysis/src/Transpiler.cpp index f5908683..54bd0d5e 100644 --- a/Analysis/src/Transpiler.cpp +++ b/Analysis/src/Transpiler.cpp @@ -933,12 +933,12 @@ struct Printer writer.symbol(")"); - if (writeTypes && func.hasReturnAnnotation) + if (writeTypes && func.returnAnnotation) { writer.symbol(":"); writer.space(); - visualizeTypeList(func.returnAnnotation, false); + visualizeTypeList(*func.returnAnnotation, false); } visualizeBlock(*func.body); @@ -989,9 +989,9 @@ struct Printer advance(typeAnnotation.location.begin); if (const auto& a = typeAnnotation.as()) { - if (a->hasPrefix) + if (a->prefix) { - writer.write(a->prefix.value); + writer.write(a->prefix->value); writer.symbol("."); } diff --git a/Analysis/src/TypeAttach.cpp b/Analysis/src/TypeAttach.cpp index 2208213f..d575e023 100644 --- a/Analysis/src/TypeAttach.cpp +++ b/Analysis/src/TypeAttach.cpp @@ -3,7 +3,6 @@ #include "Luau/Error.h" #include "Luau/Module.h" -#include "Luau/Parser.h" #include "Luau/RecursionCounter.h" #include "Luau/Scope.h" #include "Luau/ToString.h" @@ -476,12 +475,11 @@ public: visitLocal(arg); } - if (!fn->hasReturnAnnotation) + if (!fn->returnAnnotation) { if (auto result = getScope(fn->body->location)) { TypePackId ret = result->returnType; - fn->hasReturnAnnotation = true; AstTypePack* variadicAnnotation = nullptr; const auto& [v, tail] = flatten(ret); diff --git a/Analysis/src/TypeInfer.cpp b/Analysis/src/TypeInfer.cpp index f1c314cd..c29699b7 100644 --- a/Analysis/src/TypeInfer.cpp +++ b/Analysis/src/TypeInfer.cpp @@ -3,7 +3,6 @@ #include "Luau/Common.h" #include "Luau/ModuleResolver.h" -#include "Luau/Parser.h" #include "Luau/Quantify.h" #include "Luau/RecursionCounter.h" #include "Luau/Scope.h" @@ -24,16 +23,12 @@ LUAU_FASTINTVARIABLE(LuauTypeInferTypePackLoopLimit, 5000) LUAU_FASTINTVARIABLE(LuauCheckRecursionLimit, 500) LUAU_FASTFLAG(LuauKnowsTheDataModel3) LUAU_FASTFLAGVARIABLE(LuauEqConstraint, false) -LUAU_FASTFLAGVARIABLE(LuauGroupExpectedType, false) LUAU_FASTFLAGVARIABLE(LuauWeakEqConstraint, false) // Eventually removed as false. LUAU_FASTFLAG(LuauUseCommittingTxnLog) LUAU_FASTFLAGVARIABLE(DebugLuauFreezeDuringUnification, false) LUAU_FASTFLAGVARIABLE(LuauRecursiveTypeParameterRestriction, false) LUAU_FASTFLAGVARIABLE(LuauGenericFunctionsDontCacheTypeParams, false) -LUAU_FASTFLAGVARIABLE(LuauIfElseBranchTypeUnion, false) -LUAU_FASTFLAGVARIABLE(LuauIfElseExpectedType2, false) LUAU_FASTFLAGVARIABLE(LuauImmutableTypes, false) -LUAU_FASTFLAGVARIABLE(LuauLengthOnCompositeType, false) LUAU_FASTFLAGVARIABLE(LuauNoSealedTypeMod, false) LUAU_FASTFLAGVARIABLE(LuauQuantifyInPlace2, false) LUAU_FASTFLAGVARIABLE(LuauSealExports, false) @@ -43,7 +38,6 @@ LUAU_FASTFLAGVARIABLE(LuauTypeAliasDefaults, false) LUAU_FASTFLAGVARIABLE(LuauExpectedTypesOfProperties, false) LUAU_FASTFLAGVARIABLE(LuauErrorRecoveryType, false) LUAU_FASTFLAGVARIABLE(LuauPropertiesGetExpectedType, false) -LUAU_FASTFLAGVARIABLE(LuauPerModuleUnificationCache, false) LUAU_FASTFLAGVARIABLE(LuauProperTypeLevels, false) LUAU_FASTFLAGVARIABLE(LuauAscribeCorrectLevelToInferredProperitesOfFreeTables, false) LUAU_FASTFLAG(LuauUnionTagMatchFix) @@ -293,13 +287,10 @@ ModulePtr TypeChecker::check(const SourceModule& module, Mode mode, std::optiona GenericError{"Free types leaked into this module's public interface. This is an internal Luau error; please report it."}}); } - if (FFlag::LuauPerModuleUnificationCache) - { - // Clear unifier cache since it's keyed off internal types that get deallocated - // This avoids fake cross-module cache hits and keeps cache size at bay when typechecking large module graphs. - unifierState.cachedUnify.clear(); - unifierState.skipCacheForType.clear(); - } + // Clear unifier cache since it's keyed off internal types that get deallocated + // This avoids fake cross-module cache hits and keeps cache size at bay when typechecking large module graphs. + unifierState.cachedUnify.clear(); + unifierState.skipCacheForType.clear(); if (FFlag::LuauTwoPassAliasDefinitionFix) duplicateTypeAliases.clear(); @@ -509,7 +500,7 @@ LUAU_NOINLINE void TypeChecker::checkBlockTypeAliases(const ScopePtr& scope, std { if (const auto& typealias = stat->as()) { - if (FFlag::LuauTwoPassAliasDefinitionFix && typealias->name == Parser::errorName) + if (FFlag::LuauTwoPassAliasDefinitionFix && typealias->name == kParseNameError) continue; auto& bindings = typealias->exported ? scope->exportedTypeBindings : scope->privateTypeBindings; @@ -1193,7 +1184,7 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatTypeAlias& typealias Name name = typealias.name.value; // If the alias is missing a name, we can't do anything with it. Ignore it. - if (FFlag::LuauTwoPassAliasDefinitionFix && name == Parser::errorName) + if (FFlag::LuauTwoPassAliasDefinitionFix && name == kParseNameError) return; std::optional binding; @@ -1222,7 +1213,8 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatTypeAlias& typealias if (FFlag::LuauProperTypeLevels) aliasScope->level.subLevel = subLevel; - auto [generics, genericPacks] = createGenericTypes(aliasScope, scope->level, typealias, typealias.generics, typealias.genericPacks, /* useCache = */ true); + auto [generics, genericPacks] = + createGenericTypes(aliasScope, scope->level, typealias, typealias.generics, typealias.genericPacks, /* useCache = */ true); TypeId ty = freshType(aliasScope); FreeTypeVar* ftv = getMutable(ty); @@ -1464,7 +1456,7 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExpr& ExprResult result; if (auto a = expr.as()) - result = checkExpr(scope, *a->expr, FFlag::LuauGroupExpectedType ? expectedType : std::nullopt); + result = checkExpr(scope, *a->expr, expectedType); else if (expr.is()) result = {nilType}; else if (const AstExprConstantBool* bexpr = expr.as()) @@ -1508,7 +1500,7 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExpr& else if (auto a = expr.as()) result = checkExpr(scope, *a); else if (auto a = expr.as()) - result = checkExpr(scope, *a, FFlag::LuauIfElseExpectedType2 ? expectedType : std::nullopt); + result = checkExpr(scope, *a, expectedType); else ice("Unhandled AstExpr?"); @@ -2093,6 +2085,7 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprUn return {numberType}; } case AstExprUnary::Len: + { tablify(operandType); operandType = stripFromNilAndReport(operandType, expr.location); @@ -2100,30 +2093,13 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprUn if (get(operandType)) return {errorRecoveryType(scope)}; - if (FFlag::LuauLengthOnCompositeType) - { - DenseHashSet seen{nullptr}; + DenseHashSet seen{nullptr}; - if (!hasLength(operandType, seen, &recursionCount)) - reportError(TypeError{expr.location, NotATable{operandType}}); - } - else - { - if (get(operandType)) - return {numberType}; // Not strictly correct: metatables permit overriding this - - if (auto p = get(operandType)) - { - if (p->type == PrimitiveTypeVar::String) - return {numberType}; - } - - if (!getTableType(operandType)) - reportError(TypeError{expr.location, NotATable{operandType}}); - } + if (!hasLength(operandType, seen, &recursionCount)) + reportError(TypeError{expr.location, NotATable{operandType}}); return {numberType}; - + } default: ice("Unknown AstExprUnary " + std::to_string(int(expr.op))); } @@ -2618,22 +2594,11 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprIf resolve(result.predicates, falseScope, false); ExprResult falseType = checkExpr(falseScope, *expr.falseExpr, expectedType); - if (FFlag::LuauIfElseBranchTypeUnion) - { - if (falseType.type == trueType.type) - return {trueType.type}; - - std::vector types = reduceUnion({trueType.type, falseType.type}); - return {types.size() == 1 ? types[0] : addType(UnionTypeVar{std::move(types)})}; - } - else - { - unify(falseType.type, trueType.type, expr.location); - - // TODO: normalize(UnionTypeVar{{trueType, falseType}}) - // For now both trueType and falseType must be the same type. + if (falseType.type == trueType.type) return {trueType.type}; - } + + std::vector types = reduceUnion({trueType.type, falseType.type}); + return {types.size() == 1 ? types[0] : addType(UnionTypeVar{std::move(types)})}; } TypeId TypeChecker::checkLValue(const ScopePtr& scope, const AstExpr& expr) @@ -2986,8 +2951,8 @@ std::pair TypeChecker::checkFunctionSignature( auto [generics, genericPacks] = createGenericTypes(funScope, std::nullopt, expr, expr.generics, expr.genericPacks); TypePackId retPack; - if (expr.hasReturnAnnotation) - retPack = resolveTypePack(funScope, expr.returnAnnotation); + if (expr.returnAnnotation) + retPack = resolveTypePack(funScope, *expr.returnAnnotation); else if (isNonstrictMode()) retPack = anyTypePack; else if (expectedFunctionType) @@ -3181,7 +3146,7 @@ void TypeChecker::checkFunctionBody(const ScopePtr& scope, TypeId ty, const AstE // If we're in nonstrict mode we want to only report this missing return // statement if there are type annotations on the function. In strict mode // we report it regardless. - if (!isNonstrictMode() || function.hasReturnAnnotation) + if (!isNonstrictMode() || function.returnAnnotation) { reportError(getEndLocation(function), FunctionExitsWithoutReturning{funTy->retType}); } @@ -4403,11 +4368,7 @@ TypeId Instantiation::clean(TypeId ty) // Annoyingly, we have to do this even if there are no generics, // to replace any generic tables. - replaceGenerics.log = log; - replaceGenerics.level = level; - replaceGenerics.currentModule = currentModule; - replaceGenerics.generics.assign(ftv->generics.begin(), ftv->generics.end()); - replaceGenerics.genericPacks.assign(ftv->genericPacks.begin(), ftv->genericPacks.end()); + ReplaceGenerics replaceGenerics{log, arena, level, ftv->generics, ftv->genericPacks}; // TODO: What to do if this returns nullopt? // We don't have access to the error-reporting machinery @@ -4573,16 +4534,11 @@ TypeId TypeChecker::quantify(const ScopePtr& scope, TypeId ty, Location location if (FFlag::LuauQuantifyInPlace2) { - Luau::quantify(currentModule, ty, scope->level); + Luau::quantify(ty, scope->level); return ty; } - quantification.log = TxnLog::empty(); - quantification.level = scope->level; - quantification.generics.clear(); - quantification.genericPacks.clear(); - quantification.currentModule = currentModule; - + Quantification quantification{¤tModule->internalTypes, scope->level}; std::optional qty = quantification.substitute(ty); if (!qty.has_value()) @@ -4596,18 +4552,14 @@ TypeId TypeChecker::quantify(const ScopePtr& scope, TypeId ty, Location location FunctionTypeVar* qftv = getMutable(*qty); LUAU_ASSERT(qftv); - qftv->generics = quantification.generics; - qftv->genericPacks = quantification.genericPacks; + qftv->generics = std::move(quantification.generics); + qftv->genericPacks = std::move(quantification.genericPacks); return *qty; } TypeId TypeChecker::instantiate(const ScopePtr& scope, TypeId ty, Location location, const TxnLog* log) { - LUAU_ASSERT(log != nullptr); - - instantiation.log = FFlag::LuauUseCommittingTxnLog ? log : TxnLog::empty(); - instantiation.level = scope->level; - instantiation.currentModule = currentModule; + Instantiation instantiation{FFlag::LuauUseCommittingTxnLog ? log : TxnLog::empty(), ¤tModule->internalTypes, scope->level}; std::optional instantiated = instantiation.substitute(ty); if (instantiated.has_value()) return *instantiated; @@ -4620,10 +4572,7 @@ TypeId TypeChecker::instantiate(const ScopePtr& scope, TypeId ty, Location locat TypeId TypeChecker::anyify(const ScopePtr& scope, TypeId ty, Location location) { - anyification.log = TxnLog::empty(); - anyification.anyType = anyType; - anyification.anyTypePack = anyTypePack; - anyification.currentModule = currentModule; + Anyification anyification{¤tModule->internalTypes, anyType, anyTypePack}; std::optional any = anyification.substitute(ty); if (any.has_value()) return *any; @@ -4636,10 +4585,7 @@ TypeId TypeChecker::anyify(const ScopePtr& scope, TypeId ty, Location location) TypePackId TypeChecker::anyify(const ScopePtr& scope, TypePackId ty, Location location) { - anyification.log = TxnLog::empty(); - anyification.anyType = anyType; - anyification.anyTypePack = anyTypePack; - anyification.currentModule = currentModule; + Anyification anyification{¤tModule->internalTypes, anyType, anyTypePack}; std::optional any = anyification.substitute(ty); if (any.has_value()) return *any; @@ -4823,7 +4769,8 @@ TypePackId TypeChecker::errorRecoveryTypePack(TypePackId guess) return getSingletonTypes().errorRecoveryTypePack(guess); } -TypeIdPredicate TypeChecker::mkTruthyPredicate(bool sense) { +TypeIdPredicate TypeChecker::mkTruthyPredicate(bool sense) +{ return [this, sense](TypeId ty) -> std::optional { // any/error/free gets a special pass unconditionally because they can't be decided. if (get(ty) || get(ty) || get(ty)) @@ -4904,8 +4851,8 @@ TypeId TypeChecker::resolveType(const ScopePtr& scope, const AstType& annotation if (const auto& lit = annotation.as()) { std::optional tf; - if (lit->hasPrefix) - tf = scope->lookupImportedType(lit->prefix.value, lit->name.value); + if (lit->prefix) + tf = scope->lookupImportedType(lit->prefix->value, lit->name.value); else if (FFlag::DebugLuauMagicTypes && lit->name == "_luau_ice") ice("_luau_ice encountered", lit->location); @@ -4932,12 +4879,12 @@ TypeId TypeChecker::resolveType(const ScopePtr& scope, const AstType& annotation if (!tf) { - if (lit->name == Parser::errorName) + if (lit->name == kParseNameError) return errorRecoveryType(scope); std::string typeName; - if (lit->hasPrefix) - typeName = std::string(lit->prefix.value) + "."; + if (lit->prefix) + typeName = std::string(lit->prefix->value) + "."; typeName += lit->name.value; if (scope->lookupPack(typeName)) @@ -5038,12 +4985,7 @@ TypeId TypeChecker::resolveType(const ScopePtr& scope, const AstType& annotation if (notEnoughParameters && hasDefaultParameters) { // 'applyTypeFunction' is used to substitute default types that reference previous generic types - applyTypeFunction.log = TxnLog::empty(); - applyTypeFunction.typeArguments.clear(); - applyTypeFunction.typePackArguments.clear(); - applyTypeFunction.currentModule = currentModule; - applyTypeFunction.level = scope->level; - applyTypeFunction.encounteredForwardedType = false; + ApplyTypeFunction applyTypeFunction{¤tModule->internalTypes, scope->level}; for (size_t i = 0; i < typesProvided; ++i) applyTypeFunction.typeArguments[tf->typeParams[i].ty] = typeParams[i]; @@ -5362,18 +5304,14 @@ TypeId TypeChecker::instantiateTypeFun(const ScopePtr& scope, const TypeFun& tf, if (tf.typeParams.empty() && tf.typePackParams.empty()) return tf.type; - applyTypeFunction.typeArguments.clear(); + ApplyTypeFunction applyTypeFunction{¤tModule->internalTypes, scope->level}; + for (size_t i = 0; i < tf.typeParams.size(); ++i) applyTypeFunction.typeArguments[tf.typeParams[i].ty] = typeParams[i]; - applyTypeFunction.typePackArguments.clear(); for (size_t i = 0; i < tf.typePackParams.size(); ++i) applyTypeFunction.typePackArguments[tf.typePackParams[i].tp] = typePackParams[i]; - applyTypeFunction.log = TxnLog::empty(); - applyTypeFunction.currentModule = currentModule; - applyTypeFunction.level = scope->level; - applyTypeFunction.encounteredForwardedType = false; std::optional maybeInstantiated = applyTypeFunction.substitute(tf.type); if (!maybeInstantiated.has_value()) { diff --git a/Analysis/src/TypeUtils.cpp b/Analysis/src/TypeUtils.cpp index 8c6d5e49..593b54c8 100644 --- a/Analysis/src/TypeUtils.cpp +++ b/Analysis/src/TypeUtils.cpp @@ -5,6 +5,8 @@ #include "Luau/ToString.h" #include "Luau/TypeInfer.h" +LUAU_FASTFLAGVARIABLE(LuauTerminateCyclicMetatableIndexLookup, false) + namespace Luau { @@ -48,9 +50,19 @@ std::optional findTablePropertyRespectingMeta(ErrorVec& errors, const Sc } std::optional mtIndex = findMetatableEntry(errors, globalScope, ty, "__index", location); + int count = 0; while (mtIndex) { TypeId index = follow(*mtIndex); + + if (FFlag::LuauTerminateCyclicMetatableIndexLookup) + { + if (count >= 100) + return std::nullopt; + + ++count; + } + if (const auto& itt = getTableType(index)) { const auto& fit = itt->props.find(name); diff --git a/Analysis/src/TypeVar.cpp b/Analysis/src/TypeVar.cpp index 7e438e31..b2358c27 100644 --- a/Analysis/src/TypeVar.cpp +++ b/Analysis/src/TypeVar.cpp @@ -23,8 +23,6 @@ LUAU_FASTFLAG(DebugLuauFreezeArena) LUAU_FASTINTVARIABLE(LuauTypeMaximumStringifierLength, 500) LUAU_FASTINTVARIABLE(LuauTableTypeMaximumStringifierLength, 0) LUAU_FASTINT(LuauTypeInferRecursionLimit) -LUAU_FASTFLAG(LuauLengthOnCompositeType) -LUAU_FASTFLAGVARIABLE(LuauMetatableAreEqualRecursion, false) LUAU_FASTFLAGVARIABLE(LuauRefactorTypeVarQuestions, false) LUAU_FASTFLAG(LuauErrorRecoveryType) LUAU_FASTFLAG(LuauUnionTagMatchFix) @@ -385,8 +383,6 @@ bool maybeSingleton(TypeId ty) bool hasLength(TypeId ty, DenseHashSet& seen, int* recursionCount) { - LUAU_ASSERT(FFlag::LuauLengthOnCompositeType); - RecursionLimiter _rl(recursionCount, FInt::LuauTypeInferRecursionLimit); ty = follow(ty); @@ -555,7 +551,7 @@ bool areEqual(SeenSet& seen, const TableTypeVar& lhs, const TableTypeVar& rhs) static bool areEqual(SeenSet& seen, const MetatableTypeVar& lhs, const MetatableTypeVar& rhs) { - if (FFlag::LuauMetatableAreEqualRecursion && areSeen(seen, &lhs, &rhs)) + if (areSeen(seen, &lhs, &rhs)) return true; return areEqual(seen, *lhs.table, *rhs.table) && areEqual(seen, *lhs.metatable, *rhs.metatable); diff --git a/Analysis/src/TypedAllocator.cpp b/Analysis/src/TypedAllocator.cpp index f037351e..c7f31822 100644 --- a/Analysis/src/TypedAllocator.cpp +++ b/Analysis/src/TypedAllocator.cpp @@ -7,6 +7,9 @@ #ifndef WIN32_LEAN_AND_MEAN #define WIN32_LEAN_AND_MEAN #endif +#ifndef NOMINMAX +#define NOMINMAX +#endif #include const size_t kPageSize = 4096; diff --git a/Analysis/src/Unifier.cpp b/Analysis/src/Unifier.cpp index a8ad5159..322f6ebf 100644 --- a/Analysis/src/Unifier.cpp +++ b/Analysis/src/Unifier.cpp @@ -14,7 +14,6 @@ LUAU_FASTINT(LuauTypeInferRecursionLimit); LUAU_FASTINT(LuauTypeInferTypePackLoopLimit); -LUAU_FASTFLAGVARIABLE(LuauCommittingTxnLogFreeTpPromote, false) LUAU_FASTFLAG(LuauImmutableTypes) LUAU_FASTFLAG(LuauUseCommittingTxnLog) LUAU_FASTINTVARIABLE(LuauTypeInferIterationLimit, 2000); @@ -23,7 +22,6 @@ LUAU_FASTFLAGVARIABLE(LuauTableUnificationEarlyTest, false) LUAU_FASTFLAG(LuauSingletonTypes) LUAU_FASTFLAG(LuauErrorRecoveryType); LUAU_FASTFLAG(LuauProperTypeLevels); -LUAU_FASTFLAGVARIABLE(LuauUnifyPackTails, false) LUAU_FASTFLAGVARIABLE(LuauUnionTagMatchFix, false) LUAU_FASTFLAGVARIABLE(LuauFollowWithCommittingTxnLogInAnyUnification, false) @@ -116,7 +114,7 @@ struct PromoteTypeLevels { // Surprise, it's actually a BoundTypePack that hasn't been committed yet. // Calling getMutable on this will trigger an assertion. - if (FFlag::LuauCommittingTxnLogFreeTpPromote && FFlag::LuauUseCommittingTxnLog && !log.is(tp)) + if (FFlag::LuauUseCommittingTxnLog && !log.is(tp)) return true; promote(tp, FFlag::LuauUseCommittingTxnLog ? log.getMutable(tp) : getMutable(tp)); @@ -1242,7 +1240,7 @@ void Unifier::tryUnify_(TypePackId subTp, TypePackId superTp, bool isFunctionCal // If both are at the end, we're done if (!superIter.good() && !subIter.good()) { - if (FFlag::LuauUnifyPackTails && subTpv->tail && superTpv->tail) + if (subTpv->tail && superTpv->tail) { tryUnify_(*subTpv->tail, *superTpv->tail); break; @@ -1250,9 +1248,7 @@ void Unifier::tryUnify_(TypePackId subTp, TypePackId superTp, bool isFunctionCal const bool lFreeTail = superTpv->tail && log.getMutable(log.follow(*superTpv->tail)) != nullptr; const bool rFreeTail = subTpv->tail && log.getMutable(log.follow(*subTpv->tail)) != nullptr; - if (!FFlag::LuauUnifyPackTails && lFreeTail && rFreeTail) - tryUnify_(*subTpv->tail, *superTpv->tail); - else if (lFreeTail) + if (lFreeTail) tryUnify_(emptyTp, *superTpv->tail); else if (rFreeTail) tryUnify_(emptyTp, *subTpv->tail); @@ -1448,7 +1444,7 @@ void Unifier::tryUnify_(TypePackId subTp, TypePackId superTp, bool isFunctionCal // If both are at the end, we're done if (!superIter.good() && !subIter.good()) { - if (FFlag::LuauUnifyPackTails && subTpv->tail && superTpv->tail) + if (subTpv->tail && superTpv->tail) { tryUnify_(*subTpv->tail, *superTpv->tail); break; @@ -1456,9 +1452,7 @@ void Unifier::tryUnify_(TypePackId subTp, TypePackId superTp, bool isFunctionCal const bool lFreeTail = superTpv->tail && get(follow(*superTpv->tail)) != nullptr; const bool rFreeTail = subTpv->tail && get(follow(*subTpv->tail)) != nullptr; - if (!FFlag::LuauUnifyPackTails && lFreeTail && rFreeTail) - tryUnify_(*subTpv->tail, *superTpv->tail); - else if (lFreeTail) + if (lFreeTail) tryUnify_(emptyTp, *superTpv->tail); else if (rFreeTail) tryUnify_(emptyTp, *subTpv->tail); diff --git a/Ast/include/Luau/Ast.h b/Ast/include/Luau/Ast.h index ac5950c0..31cd01cc 100644 --- a/Ast/include/Luau/Ast.h +++ b/Ast/include/Luau/Ast.h @@ -594,8 +594,7 @@ public: AstArray genericPacks; AstLocal* self; AstArray args; - bool hasReturnAnnotation; - AstTypeList returnAnnotation; + std::optional returnAnnotation; bool vararg = false; Location varargLocation; AstTypePack* varargAnnotation; @@ -740,7 +739,7 @@ class AstStatIf : public AstStat public: LUAU_RTTI(AstStatIf) - AstStatIf(const Location& location, AstExpr* condition, AstStatBlock* thenbody, AstStat* elsebody, bool hasThen, const Location& thenLocation, + AstStatIf(const Location& location, AstExpr* condition, AstStatBlock* thenbody, AstStat* elsebody, const std::optional& thenLocation, const std::optional& elseLocation, bool hasEnd); void visit(AstVisitor* visitor) override; @@ -749,12 +748,10 @@ public: AstStatBlock* thenbody; AstStat* elsebody; - bool hasThen = false; - Location thenLocation; + std::optional thenLocation; // Active for 'elseif' as well - bool hasElse = false; - Location elseLocation; + std::optional elseLocation; bool hasEnd = false; }; @@ -849,8 +846,7 @@ public: AstArray vars; AstArray values; - bool hasEqualsSign = false; - Location equalsSignLocation; + std::optional equalsSignLocation; }; class AstStatFor : public AstStat @@ -1053,9 +1049,8 @@ public: void visit(AstVisitor* visitor) override; - bool hasPrefix; bool hasParameterList; - AstName prefix; + std::optional prefix; AstName name; AstArray parameters; }; diff --git a/Ast/include/Luau/Lexer.h b/Ast/include/Luau/Lexer.h index 460ef056..d7d867f4 100644 --- a/Ast/include/Luau/Lexer.h +++ b/Ast/include/Luau/Lexer.h @@ -233,4 +233,9 @@ private: bool readNames; }; +inline bool isSpace(char ch) +{ + return ch == ' ' || ch == '\t' || ch == '\r' || ch == '\n' || ch == '\v' || ch == '\f'; +} + } // namespace Luau diff --git a/Ast/include/Luau/ParseResult.h b/Ast/include/Luau/ParseResult.h new file mode 100644 index 00000000..17ce2e3b --- /dev/null +++ b/Ast/include/Luau/ParseResult.h @@ -0,0 +1,69 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +#pragma once + +#include "Luau/Common.h" +#include "Luau/Location.h" +#include "Luau/Lexer.h" +#include "Luau/StringUtils.h" + +namespace Luau +{ + +class AstStatBlock; + +class ParseError : public std::exception +{ +public: + ParseError(const Location& location, const std::string& message); + + virtual const char* what() const throw(); + + const Location& getLocation() const; + const std::string& getMessage() const; + + static LUAU_NORETURN void raise(const Location& location, const char* format, ...) LUAU_PRINTF_ATTR(2, 3); + +private: + Location location; + std::string message; +}; + +class ParseErrors : public std::exception +{ +public: + ParseErrors(std::vector errors); + + virtual const char* what() const throw(); + + const std::vector& getErrors() const; + +private: + std::vector errors; + std::string message; +}; + +struct HotComment +{ + bool header; + Location location; + std::string content; +}; + +struct Comment +{ + Lexeme::Type type; // Comment, BlockComment, or BrokenComment + Location location; +}; + +struct ParseResult +{ + AstStatBlock* root; + std::vector hotcomments; + std::vector errors; + + std::vector commentLocations; +}; + +static constexpr const char* kParseNameError = "%error-id%"; + +} // namespace Luau diff --git a/Ast/include/Luau/Parser.h b/Ast/include/Luau/Parser.h index 40ecdcdd..4b5ae315 100644 --- a/Ast/include/Luau/Parser.h +++ b/Ast/include/Luau/Parser.h @@ -4,6 +4,7 @@ #include "Luau/Ast.h" #include "Luau/Lexer.h" #include "Luau/ParseOptions.h" +#include "Luau/ParseResult.h" #include "Luau/StringUtils.h" #include "Luau/DenseHash.h" #include "Luau/Common.h" @@ -14,37 +15,6 @@ namespace Luau { -class ParseError : public std::exception -{ -public: - ParseError(const Location& location, const std::string& message); - - virtual const char* what() const throw(); - - const Location& getLocation() const; - const std::string& getMessage() const; - - static LUAU_NORETURN void raise(const Location& location, const char* format, ...) LUAU_PRINTF_ATTR(2, 3); - -private: - Location location; - std::string message; -}; - -class ParseErrors : public std::exception -{ -public: - ParseErrors(std::vector errors); - - virtual const char* what() const throw(); - - const std::vector& getErrors() const; - -private: - std::vector errors; - std::string message; -}; - template class TempVector { @@ -80,34 +50,17 @@ private: size_t size_; }; -struct Comment -{ - Lexeme::Type type; // Comment, BlockComment, or BrokenComment - Location location; -}; - -struct ParseResult -{ - AstStatBlock* root; - std::vector hotcomments; - std::vector errors; - - std::vector commentLocations; -}; - class Parser { public: static ParseResult parse( const char* buffer, std::size_t bufferSize, AstNameTable& names, Allocator& allocator, ParseOptions options = ParseOptions()); - static constexpr const char* errorName = "%error-id%"; - private: struct Name; struct Binding; - Parser(const char* buffer, std::size_t bufferSize, AstNameTable& names, Allocator& allocator); + Parser(const char* buffer, std::size_t bufferSize, AstNameTable& names, Allocator& allocator, const ParseOptions& options); bool blockFollow(const Lexeme& l); @@ -330,7 +283,7 @@ private: AstTypeError* reportTypeAnnotationError(const Location& location, const AstArray& types, bool isMissing, const char* format, ...) LUAU_PRINTF_ATTR(5, 6); - const Lexeme& nextLexeme(); + void nextLexeme(); struct Function { @@ -386,6 +339,9 @@ private: Allocator& allocator; std::vector commentLocations; + std::vector hotcomments; + + bool hotcommentHeader = true; unsigned int recursionCounter; diff --git a/Ast/src/Ast.cpp b/Ast/src/Ast.cpp index 9b5bc0c7..24a280da 100644 --- a/Ast/src/Ast.cpp +++ b/Ast/src/Ast.cpp @@ -167,8 +167,7 @@ AstExprFunction::AstExprFunction(const Location& location, const AstArrayreturnAnnotation = *returnAnnotation; } void AstExprFunction::visit(AstVisitor* visitor) @@ -195,8 +192,8 @@ void AstExprFunction::visit(AstVisitor* visitor) if (varargAnnotation) varargAnnotation->visit(visitor); - if (hasReturnAnnotation) - visitTypeList(visitor, returnAnnotation); + if (returnAnnotation) + visitTypeList(visitor, *returnAnnotation); body->visit(visitor); } @@ -375,21 +372,16 @@ void AstStatBlock::visit(AstVisitor* visitor) } } -AstStatIf::AstStatIf(const Location& location, AstExpr* condition, AstStatBlock* thenbody, AstStat* elsebody, bool hasThen, - const Location& thenLocation, const std::optional& elseLocation, bool hasEnd) +AstStatIf::AstStatIf(const Location& location, AstExpr* condition, AstStatBlock* thenbody, AstStat* elsebody, + const std::optional& thenLocation, const std::optional& elseLocation, bool hasEnd) : AstStat(ClassIndex(), location) , condition(condition) , thenbody(thenbody) , elsebody(elsebody) - , hasThen(hasThen) , thenLocation(thenLocation) + , elseLocation(elseLocation) , hasEnd(hasEnd) { - if (bool(elseLocation)) - { - hasElse = true; - this->elseLocation = *elseLocation; - } } void AstStatIf::visit(AstVisitor* visitor) @@ -492,12 +484,8 @@ AstStatLocal::AstStatLocal( : AstStat(ClassIndex(), location) , vars(vars) , values(values) + , equalsSignLocation(equalsSignLocation) { - if (bool(equalsSignLocation)) - { - hasEqualsSign = true; - this->equalsSignLocation = *equalsSignLocation; - } } void AstStatLocal::visit(AstVisitor* visitor) @@ -750,9 +738,8 @@ void AstStatError::visit(AstVisitor* visitor) AstTypeReference::AstTypeReference( const Location& location, std::optional prefix, AstName name, bool hasParameterList, const AstArray& parameters) : AstType(ClassIndex(), location) - , hasPrefix(bool(prefix)) , hasParameterList(hasParameterList) - , prefix(prefix ? *prefix : AstName()) + , prefix(prefix) , name(name) , parameters(parameters) { diff --git a/Ast/src/Lexer.cpp b/Ast/src/Lexer.cpp index a7aa24ca..d56c8860 100644 --- a/Ast/src/Lexer.cpp +++ b/Ast/src/Lexer.cpp @@ -101,11 +101,6 @@ Lexeme::Lexeme(const Location& location, Type type, const char* name) LUAU_ASSERT(type == Name || (type >= Reserved_BEGIN && type < Lexeme::Reserved_END)); } -static bool isComment(const Lexeme& lexeme) -{ - return lexeme.type == Lexeme::Comment || lexeme.type == Lexeme::BlockComment; -} - static const char* kReserved[] = {"and", "break", "do", "else", "elseif", "end", "false", "for", "function", "if", "in", "local", "nil", "not", "or", "repeat", "return", "then", "true", "until", "while"}; @@ -282,11 +277,6 @@ AstName AstNameTable::get(const char* name) const return getWithType(name, strlen(name)).first; } -inline bool isSpace(char ch) -{ - return ch == ' ' || ch == '\t' || ch == '\r' || ch == '\n' || ch == '\v' || ch == '\f'; -} - inline bool isAlpha(char ch) { // use or trick to convert to lower case and unsigned comparison to do range check @@ -372,7 +362,7 @@ const Lexeme& Lexer::next(bool skipComments) prevLocation = lexeme.location; lexeme = readNext(); - } while (skipComments && isComment(lexeme)); + } while (skipComments && (lexeme.type == Lexeme::Comment || lexeme.type == Lexeme::BlockComment)); return lexeme; } diff --git a/Ast/src/Parser.cpp b/Ast/src/Parser.cpp index 30b32f91..235d6349 100644 --- a/Ast/src/Parser.cpp +++ b/Ast/src/Parser.cpp @@ -13,18 +13,15 @@ LUAU_FASTINTVARIABLE(LuauParseErrorLimit, 100) LUAU_FASTFLAGVARIABLE(LuauParseSingletonTypes, false) LUAU_FASTFLAGVARIABLE(LuauParseTypeAliasDefaults, false) LUAU_FASTFLAGVARIABLE(LuauParseRecoverTypePackEllipsis, false) -LUAU_FASTFLAGVARIABLE(LuauStartingBrokenComment, false) +LUAU_FASTFLAGVARIABLE(LuauParseAllHotComments, false) +LUAU_FASTFLAGVARIABLE(LuauTableFieldFunctionDebugname, false) namespace Luau { -inline bool isSpace(char ch) -{ - return ch == ' ' || ch == '\t' || ch == '\r' || ch == '\n' || ch == '\v' || ch == '\f'; -} - static bool isComment(const Lexeme& lexeme) { + LUAU_ASSERT(!FFlag::LuauParseAllHotComments); return lexeme.type == Lexeme::Comment || lexeme.type == Lexeme::BlockComment; } @@ -151,31 +148,37 @@ ParseResult Parser::parse(const char* buffer, size_t bufferSize, AstNameTable& n { LUAU_TIMETRACE_SCOPE("Parser::parse", "Parser"); - Parser p(buffer, bufferSize, names, allocator); + Parser p(buffer, bufferSize, names, allocator, FFlag::LuauParseAllHotComments ? options : ParseOptions()); try { - std::vector hotcomments; - - while (isComment(p.lexer.current()) || p.lexer.current().type == Lexeme::BrokenComment) + if (FFlag::LuauParseAllHotComments) { - const char* text = p.lexer.current().data; - unsigned int length = p.lexer.current().length; + AstStatBlock* root = p.parseChunk(); - if (length && text[0] == '!') + return ParseResult{root, std::move(p.hotcomments), std::move(p.parseErrors), std::move(p.commentLocations)}; + } + else + { + std::vector hotcomments; + + while (isComment(p.lexer.current()) || p.lexer.current().type == Lexeme::BrokenComment) { - unsigned int end = length; - while (end > 0 && isSpace(text[end - 1])) - --end; + const char* text = p.lexer.current().data; + unsigned int length = p.lexer.current().length; - hotcomments.push_back(std::string(text + 1, text + end)); - } + if (length && text[0] == '!') + { + unsigned int end = length; + while (end > 0 && isSpace(text[end - 1])) + --end; - const Lexeme::Type type = p.lexer.current().type; - const Location loc = p.lexer.current().location; + hotcomments.push_back({true, p.lexer.current().location, std::string(text + 1, text + end)}); + } + + const Lexeme::Type type = p.lexer.current().type; + const Location loc = p.lexer.current().location; - if (FFlag::LuauStartingBrokenComment) - { if (options.captureComments) p.commentLocations.push_back(Comment{type, loc}); @@ -184,22 +187,15 @@ ParseResult Parser::parse(const char* buffer, size_t bufferSize, AstNameTable& n p.lexer.next(); } - else - { - p.lexer.next(); - if (options.captureComments) - p.commentLocations.push_back(Comment{type, loc}); - } + p.lexer.setSkipComments(true); + + p.options = options; + + AstStatBlock* root = p.parseChunk(); + + return ParseResult{root, hotcomments, p.parseErrors, std::move(p.commentLocations)}; } - - p.lexer.setSkipComments(true); - - p.options = options; - - AstStatBlock* root = p.parseChunk(); - - return ParseResult{root, hotcomments, p.parseErrors, std::move(p.commentLocations)}; } catch (ParseError& err) { @@ -210,8 +206,9 @@ ParseResult Parser::parse(const char* buffer, size_t bufferSize, AstNameTable& n } } -Parser::Parser(const char* buffer, size_t bufferSize, AstNameTable& names, Allocator& allocator) - : lexer(buffer, bufferSize, names) +Parser::Parser(const char* buffer, size_t bufferSize, AstNameTable& names, Allocator& allocator, const ParseOptions& options) + : options(options) + , lexer(buffer, bufferSize, names) , allocator(allocator) , recursionCounter(0) , endMismatchSuspect(Location(), Lexeme::Eof) @@ -224,14 +221,20 @@ Parser::Parser(const char* buffer, size_t bufferSize, AstNameTable& names, Alloc nameSelf = names.addStatic("self"); nameNumber = names.addStatic("number"); - nameError = names.addStatic(errorName); + nameError = names.addStatic(kParseNameError); nameNil = names.getOrAdd("nil"); // nil is a reserved keyword matchRecoveryStopOnToken.assign(Lexeme::Type::Reserved_END, 0); matchRecoveryStopOnToken[Lexeme::Type::Eof] = 1; + if (FFlag::LuauParseAllHotComments) + lexer.setSkipComments(true); + // read first lexeme nextLexeme(); + + // all hot comments parsed after the first non-comment lexeme are special in that they don't affect type checking / linting mode + hotcommentHeader = false; } bool Parser::blockFollow(const Lexeme& l) @@ -396,7 +399,9 @@ AstStat* Parser::parseIf() AstExpr* cond = parseExpr(); Lexeme matchThen = lexer.current(); - bool hasThen = expectAndConsume(Lexeme::ReservedThen, "if statement"); + std::optional thenLocation; + if (expectAndConsume(Lexeme::ReservedThen, "if statement")) + thenLocation = matchThen.location; AstStatBlock* thenbody = parseBlock(); @@ -434,7 +439,7 @@ AstStat* Parser::parseIf() hasEnd = expectMatchEndAndConsume(Lexeme::ReservedEnd, matchThenElse); } - return allocator.alloc(Location(start, end), cond, thenbody, elsebody, hasThen, matchThen.location, elseLocation, hasEnd); + return allocator.alloc(Location(start, end), cond, thenbody, elsebody, thenLocation, elseLocation, hasEnd); } // while exp do block end @@ -769,7 +774,7 @@ AstStat* Parser::parseTypeAlias(const Location& start, bool exported) { // note: `type` token is already parsed for us, so we just need to parse the rest - auto name = parseNameOpt("type name"); + std::optional name = parseNameOpt("type name"); // Use error name if the name is missing if (!name) @@ -925,7 +930,7 @@ AstStat* Parser::parseDeclaration(const Location& start) return allocator.alloc(Location(classStart, classEnd), className.name, superName, copy(props)); } - else if (auto globalName = parseNameOpt("global variable name")) + else if (std::optional globalName = parseNameOpt("global variable name")) { expectAndConsume(':', "global variable declaration"); @@ -1066,7 +1071,7 @@ void Parser::parseExprList(TempVector& result) Parser::Binding Parser::parseBinding() { - auto name = parseNameOpt("variable name"); + std::optional name = parseNameOpt("variable name"); // Use placeholder if the name is missing if (!name) @@ -1325,7 +1330,7 @@ AstType* Parser::parseTableTypeAnnotation() } else { - auto name = parseNameOpt("table field"); + std::optional name = parseNameOpt("table field"); if (!name) break; @@ -1422,7 +1427,7 @@ AstType* Parser::parseFunctionTypeAnnotationTail(const Lexeme& begin, AstArray(Location(begin.location, endLocation), generics, genericPacks, paramTypes, paramNames, returnTypeList); @@ -1869,7 +1874,7 @@ AstExpr* Parser::parseExpr(unsigned int limit) // NAME AstExpr* Parser::parseNameExpr(const char* context) { - auto name = parseNameOpt(context); + std::optional name = parseNameOpt(context); if (!name) return allocator.alloc(lexer.current().location, copy({}), unsigned(parseErrors.size() - 1)); @@ -2233,6 +2238,12 @@ AstExpr* Parser::parseTableConstructor() AstExpr* key = allocator.alloc(name.location, nameString); AstExpr* value = parseExpr(); + if (FFlag::LuauTableFieldFunctionDebugname) + { + if (AstExprFunction* func = value->as()) + func->debugname = name.name; + } + items.push_back({AstExprTable::Item::Record, key, value}); } else @@ -2313,7 +2324,7 @@ std::optional Parser::parseNameOpt(const char* context) Parser::Name Parser::parseName(const char* context) { - if (auto name = parseNameOpt(context)) + if (std::optional name = parseNameOpt(context)) return *name; Location location = lexer.current().location; @@ -2324,7 +2335,7 @@ Parser::Name Parser::parseName(const char* context) Parser::Name Parser::parseIndexName(const char* context, const Position& previous) { - if (auto name = parseNameOpt(context)) + if (std::optional name = parseNameOpt(context)) return *name; // If we have a reserved keyword next at the same line, assume it's an incomplete name @@ -2379,7 +2390,7 @@ std::pair, AstArray> Parser::parseG if (shouldParseTypePackAnnotation(lexer)) { - auto typePack = parseTypePackAnnotation(); + AstTypePack* typePack = parseTypePackAnnotation(); namePacks.push_back({name, nameLocation, typePack}); } @@ -2451,7 +2462,7 @@ AstArray Parser::parseTypeParams() { if (shouldParseTypePackAnnotation(lexer)) { - auto typePack = parseTypePackAnnotation(); + AstTypePack* typePack = parseTypePackAnnotation(); parameters.push_back({{}, typePack}); } @@ -2821,25 +2832,57 @@ AstTypeError* Parser::reportTypeAnnotationError(const Location& location, const return allocator.alloc(location, types, isMissing, unsigned(parseErrors.size() - 1)); } -const Lexeme& Parser::nextLexeme() +void Parser::nextLexeme() { if (options.captureComments) { - while (true) + if (FFlag::LuauParseAllHotComments) { - const Lexeme& lexeme = lexer.next(/*skipComments*/ false); - // Subtlety: Broken comments are weird because we record them as comments AND pass them to the parser as a lexeme. - // The parser will turn this into a proper syntax error. - if (lexeme.type == Lexeme::BrokenComment) + Lexeme::Type type = lexer.next(/* skipComments= */ false).type; + + while (type == Lexeme::BrokenComment || type == Lexeme::Comment || type == Lexeme::BlockComment) + { + const Lexeme& lexeme = lexer.current(); commentLocations.push_back(Comment{lexeme.type, lexeme.location}); - if (isComment(lexeme)) - commentLocations.push_back(Comment{lexeme.type, lexeme.location}); - else - return lexeme; + + // Subtlety: Broken comments are weird because we record them as comments AND pass them to the parser as a lexeme. + // The parser will turn this into a proper syntax error. + if (lexeme.type == Lexeme::BrokenComment) + return; + + // Comments starting with ! are called "hot comments" and contain directives for type checking / linting + if (lexeme.type == Lexeme::Comment && lexeme.length && lexeme.data[0] == '!') + { + const char* text = lexeme.data; + + unsigned int end = lexeme.length; + while (end > 0 && isSpace(text[end - 1])) + --end; + + hotcomments.push_back({hotcommentHeader, lexeme.location, std::string(text + 1, text + end)}); + } + + type = lexer.next(/* skipComments= */ false).type; + } + } + else + { + while (true) + { + const Lexeme& lexeme = lexer.next(/*skipComments*/ false); + // Subtlety: Broken comments are weird because we record them as comments AND pass them to the parser as a lexeme. + // The parser will turn this into a proper syntax error. + if (lexeme.type == Lexeme::BrokenComment) + commentLocations.push_back(Comment{lexeme.type, lexeme.location}); + if (isComment(lexeme)) + commentLocations.push_back(Comment{lexeme.type, lexeme.location}); + else + return; + } } } else - return lexer.next(); + lexer.next(); } } // namespace Luau diff --git a/Ast/src/TimeTrace.cpp b/Ast/src/TimeTrace.cpp index ded50e53..8079830b 100644 --- a/Ast/src/TimeTrace.cpp +++ b/Ast/src/TimeTrace.cpp @@ -9,6 +9,12 @@ #include #ifdef _WIN32 +#ifndef WIN32_LEAN_AND_MEAN +#define WIN32_LEAN_AND_MEAN +#endif +#ifndef NOMINMAX +#define NOMINMAX +#endif #include #endif diff --git a/CLI/FileUtils.cpp b/CLI/FileUtils.cpp index fe005aec..fb6ac373 100644 --- a/CLI/FileUtils.cpp +++ b/CLI/FileUtils.cpp @@ -4,8 +4,13 @@ #include "Luau/Common.h" #ifdef _WIN32 +#ifndef WIN32_LEAN_AND_MEAN #define WIN32_LEAN_AND_MEAN -#include +#endif +#ifndef NOMINMAX +#define NOMINMAX +#endif +#include #else #include #include diff --git a/CLI/Repl.cpp b/CLI/Repl.cpp index 9a6e25c2..13304d57 100644 --- a/CLI/Repl.cpp +++ b/CLI/Repl.cpp @@ -37,6 +37,8 @@ enum class CompileFormat Binary }; +constexpr int MaxTraversalLimit = 50; + struct GlobalOptions { int optimizationLevel = 1; @@ -243,10 +245,115 @@ std::string runCode(lua_State* L, const std::string& source) return std::string(); } +// Replaces the top of the lua stack with the metatable __index for the value +// if it exists. Returns true iff __index exists. +static bool tryReplaceTopWithIndex(lua_State* L) +{ + if (luaL_getmetafield(L, -1, "__index")) + { + // Remove the table leaving __index on the top of stack + lua_remove(L, -2); + return true; + } + return false; +} + + +// This function is similar to lua_gettable, but it avoids calling any +// lua callback functions (e.g. __index) which might modify the Lua VM state. +static void safeGetTable(lua_State* L, int tableIndex) +{ + lua_pushvalue(L, tableIndex); // Duplicate the table + + // The loop invariant is that the table to search is at -1 + // and the key is at -2. + for (int loopCount = 0;; loopCount++) + { + lua_pushvalue(L, -2); // Duplicate the key + lua_rawget(L, -2); // Try to find the key + if (!lua_isnil(L, -1) || loopCount >= MaxTraversalLimit) + { + // Either the key has been found, and/or we have reached the max traversal limit + break; + } + else + { + lua_pop(L, 1); // Pop the nil result + if (!luaL_getmetafield(L, -1, "__index")) + { + lua_pushnil(L); + break; + } + else if (lua_istable(L, -1)) + { + // Replace the current table being searched with __index table + lua_replace(L, -2); + } + else + { + lua_pop(L, 1); // Pop the value + lua_pushnil(L); + break; + } + } + } + + lua_remove(L, -2); // Remove the table + lua_remove(L, -2); // Remove the original key +} + +// completePartialMatches finds keys that match the specified 'prefix' +// Note: the table/object to be searched must be on the top of the Lua stack +static void completePartialMatches(lua_State* L, bool completeOnlyFunctions, const std::string& editBuffer, std::string_view prefix, + const AddCompletionCallback& addCompletionCallback) +{ + for (int i = 0; i < MaxTraversalLimit && lua_istable(L, -1); i++) + { + // table, key + lua_pushnil(L); + + // Loop over all the keys in the current table + while (lua_next(L, -2) != 0) + { + if (lua_type(L, -2) == LUA_TSTRING) + { + // table, key, value + std::string_view key = lua_tostring(L, -2); + int valueType = lua_type(L, -1); + + // If the last separator was a ':' (i.e. a method call) then only functions should be completed. + bool requiredValueType = (!completeOnlyFunctions || valueType == LUA_TFUNCTION); + + if (!key.empty() && requiredValueType && Luau::startsWith(key, prefix)) + { + std::string completedComponent(key.substr(prefix.size())); + std::string completion(editBuffer + completedComponent); + if (valueType == LUA_TFUNCTION) + { + // Add an opening paren for function calls by default. + completion += "("; + } + addCompletionCallback(completion, std::string(key)); + } + } + lua_pop(L, 1); + } + + // Replace the current table being searched with an __index table if one exists + if (!tryReplaceTopWithIndex(L)) + { + break; + } + } +} + static void completeIndexer(lua_State* L, const std::string& editBuffer, const AddCompletionCallback& addCompletionCallback) { std::string_view lookup = editBuffer; - char lastSep = 0; + bool completeOnlyFunctions = false; + + // Push the global variable table to begin the search + lua_pushvalue(L, LUA_GLOBALSINDEX); for (;;) { @@ -255,60 +362,26 @@ static void completeIndexer(lua_State* L, const std::string& editBuffer, const A if (sep == std::string_view::npos) { - // table, key - lua_pushnil(L); - - while (lua_next(L, -2) != 0) - { - if (lua_type(L, -2) == LUA_TSTRING) - { - // table, key, value - std::string_view key = lua_tostring(L, -2); - int valueType = lua_type(L, -1); - - // If the last separator was a ':' (i.e. a method call) then only functions should be completed. - bool requiredValueType = (lastSep != ':' || valueType == LUA_TFUNCTION); - - if (!key.empty() && requiredValueType && Luau::startsWith(key, prefix)) - { - std::string completedComponent(key.substr(prefix.size())); - std::string completion(editBuffer + completedComponent); - if (valueType == LUA_TFUNCTION) - { - // Add an opening paren for function calls by default. - completion += "("; - } - addCompletionCallback(completion, std::string(key)); - } - } - lua_pop(L, 1); - } - + completePartialMatches(L, completeOnlyFunctions, editBuffer, prefix, addCompletionCallback); break; } else { // find the key in the table lua_pushlstring(L, prefix.data(), prefix.size()); - lua_rawget(L, -2); + safeGetTable(L, -2); lua_remove(L, -2); - if (lua_type(L, -1) == LUA_TSTRING) + if (lua_istable(L, -1) || tryReplaceTopWithIndex(L)) { - // Replace the string object with the string class to perform further lookups of string functions - // Note: We retrieve the string class from _G to prevent issues if the user assigns to `string`. - lua_pop(L, 1); // Pop the string instance - lua_getglobal(L, "_G"); - lua_pushlstring(L, "string", 6); - lua_rawget(L, -2); - lua_remove(L, -2); // Remove the global table - LUAU_ASSERT(lua_istable(L, -1)); + completeOnlyFunctions = lookup[sep] == ':'; + lookup.remove_prefix(sep + 1); } - else if (!lua_istable(L, -1)) + else + { + // Unable to search for keys, so stop searching break; - - lastSep = lookup[sep]; - lookup.remove_prefix(sep + 1); + } } } @@ -317,12 +390,6 @@ static void completeIndexer(lua_State* L, const std::string& editBuffer, const A void getCompletions(lua_State* L, const std::string& editBuffer, const AddCompletionCallback& addCompletionCallback) { - // look the value up in current global table first - lua_pushvalue(L, LUA_GLOBALSINDEX); - completeIndexer(L, editBuffer, addCompletionCallback); - - // and in actual global table after that - lua_getglobal(L, "_G"); completeIndexer(L, editBuffer, addCompletionCallback); } @@ -365,9 +432,7 @@ struct LinenoiseScopedHistory ic_set_history(historyFilepath.c_str(), -1 /* default entries (= 200) */); } - ~LinenoiseScopedHistory() - { - } + ~LinenoiseScopedHistory() {} std::string historyFilepath; }; diff --git a/CMakeLists.txt b/CMakeLists.txt index c19d2b40..c6ccebc5 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -104,6 +104,20 @@ if (MSVC AND MSVC_VERSION GREATER_EQUAL 1924) set_source_files_properties(VM/src/lvmexecute.cpp PROPERTIES COMPILE_FLAGS /d2ssa-pre-) endif() +# embed .natvis inside the library debug information +if(MSVC) + target_link_options(Luau.Ast INTERFACE /NATVIS:${CMAKE_CURRENT_SOURCE_DIR}/tools/natvis/Ast.natvis) + target_link_options(Luau.Analysis INTERFACE /NATVIS:${CMAKE_CURRENT_SOURCE_DIR}/tools/natvis/Analysis.natvis) + target_link_options(Luau.VM INTERFACE /NATVIS:${CMAKE_CURRENT_SOURCE_DIR}/tools/natvis/VM.natvis) +endif() + +# make .natvis visible inside the solution +if(MSVC_IDE) + target_sources(Luau.Ast PRIVATE tools/natvis/Ast.natvis) + target_sources(Luau.Analysis PRIVATE tools/natvis/Analysis.natvis) + target_sources(Luau.VM PRIVATE tools/natvis/VM.natvis) +endif() + if(LUAU_BUILD_CLI) target_compile_options(Luau.Repl.CLI PRIVATE ${LUAU_OPTIONS}) target_compile_options(Luau.Analyze.CLI PRIVATE ${LUAU_OPTIONS}) diff --git a/Sources.cmake b/Sources.cmake index 773f6f35..615641eb 100644 --- a/Sources.cmake +++ b/Sources.cmake @@ -8,6 +8,7 @@ target_sources(Luau.Ast PRIVATE Ast/include/Luau/Location.h Ast/include/Luau/ParseOptions.h Ast/include/Luau/Parser.h + Ast/include/Luau/ParseResult.h Ast/include/Luau/StringUtils.h Ast/include/Luau/TimeTrace.h diff --git a/VM/src/lapi.cpp b/VM/src/lapi.cpp index 5cffba63..29d5f397 100644 --- a/VM/src/lapi.cpp +++ b/VM/src/lapi.cpp @@ -36,12 +36,9 @@ const char* luau_ident = "$Luau: Copyright (C) 2019-2022 Roblox Corporation $\n" static Table* getcurrenv(lua_State* L) { if (L->ci == L->base_ci) /* no enclosing function? */ - return hvalue(gt(L)); /* use global table as environment */ + return L->gt; /* use global table as environment */ else - { - Closure* func = curr_func(L); - return func->env; - } + return curr_func(L)->env; } static LUAU_NOINLINE TValue* pseudo2addr(lua_State* L, int idx) @@ -53,11 +50,14 @@ static LUAU_NOINLINE TValue* pseudo2addr(lua_State* L, int idx) return registry(L); case LUA_ENVIRONINDEX: { - sethvalue(L, &L->env, getcurrenv(L)); - return &L->env; + sethvalue(L, &L->global->pseudotemp, getcurrenv(L)); + return &L->global->pseudotemp; } case LUA_GLOBALSINDEX: - return gt(L); + { + sethvalue(L, &L->global->pseudotemp, L->gt); + return &L->global->pseudotemp; + } default: { Closure* func = curr_func(L); @@ -237,6 +237,11 @@ void lua_replace(lua_State* L, int idx) func->env = hvalue(L->top - 1); luaC_barrier(L, func, L->top - 1); } + else if (idx == LUA_GLOBALSINDEX) + { + api_check(L, ttistable(L->top - 1)); + L->gt = hvalue(L->top - 1); + } else { setobj(L, o, L->top - 1); @@ -783,7 +788,7 @@ void lua_getfenv(lua_State* L, int idx) sethvalue(L, L->top, clvalue(o)->env); break; case LUA_TTHREAD: - setobj2s(L, L->top, gt(thvalue(o))); + sethvalue(L, L->top, thvalue(o)->gt); break; default: setnilvalue(L->top); @@ -914,7 +919,7 @@ int lua_setfenv(lua_State* L, int idx) clvalue(o)->env = hvalue(L->top - 1); break; case LUA_TTHREAD: - sethvalue(L, gt(thvalue(o)), hvalue(L->top - 1)); + thvalue(o)->gt = hvalue(L->top - 1); break; default: res = 0; diff --git a/VM/src/ldebug.cpp b/VM/src/ldebug.cpp index e9930f7a..a4f93c62 100644 --- a/VM/src/ldebug.cpp +++ b/VM/src/ldebug.cpp @@ -419,7 +419,7 @@ static void getcoverage(Proto* p, int depth, int* buffer, size_t size, void* con } const char* debugname = p->debugname ? getstr(p->debugname) : NULL; - int linedefined = luaG_getline(p, 0); + int linedefined = getlinedefined(p); callback(context, debugname, linedefined, depth, buffer, size); diff --git a/VM/src/ldo.cpp b/VM/src/ldo.cpp index a3982bc6..d87f0661 100644 --- a/VM/src/ldo.cpp +++ b/VM/src/ldo.cpp @@ -17,6 +17,8 @@ #include +LUAU_FASTFLAG(LuauReduceStackReallocs) + /* ** {====================================================== ** Error-recovery functions @@ -164,13 +166,14 @@ static void correctstack(lua_State* L, TValue* oldstack) void luaD_reallocstack(lua_State* L, int newsize) { TValue* oldstack = L->stack; - int realsize = newsize + 1 + EXTRA_STACK; - LUAU_ASSERT(L->stack_last - L->stack == L->stacksize - EXTRA_STACK - 1); + int realsize = newsize + (FFlag::LuauReduceStackReallocs ? EXTRA_STACK : 1 + EXTRA_STACK); + LUAU_ASSERT(L->stack_last - L->stack == L->stacksize - (FFlag::LuauReduceStackReallocs ? EXTRA_STACK : 1 + EXTRA_STACK)); luaM_reallocarray(L, L->stack, L->stacksize, realsize, TValue, L->memcat); + TValue* newstack = L->stack; for (int i = L->stacksize; i < realsize; i++) - setnilvalue(L->stack + i); /* erase new segment */ + setnilvalue(newstack + i); /* erase new segment */ L->stacksize = realsize; - L->stack_last = L->stack + newsize; + L->stack_last = newstack + newsize; correctstack(L, oldstack); } @@ -512,7 +515,7 @@ static void callerrfunc(lua_State* L, void* ud) static void restore_stack_limit(lua_State* L) { - LUAU_ASSERT(L->stack_last - L->stack == L->stacksize - EXTRA_STACK - 1); + LUAU_ASSERT(L->stack_last - L->stack == L->stacksize - (FFlag::LuauReduceStackReallocs ? EXTRA_STACK : 1 + EXTRA_STACK)); if (L->size_ci > LUAI_MAXCALLS) { /* there was an overflow? */ int inuse = cast_int(L->ci - L->base_ci); diff --git a/VM/src/ldo.h b/VM/src/ldo.h index 72807f0f..1c1480d6 100644 --- a/VM/src/ldo.h +++ b/VM/src/ldo.h @@ -11,7 +11,7 @@ if ((char*)L->stack_last - (char*)L->top <= (n) * (int)sizeof(TValue)) \ luaD_growstack(L, n); \ else \ - condhardstacktests(luaD_reallocstack(L, L->stacksize - EXTRA_STACK - 1)); + condhardstacktests(luaD_reallocstack(L, L->stacksize - (FFlag::LuauReduceStackReallocs ? EXTRA_STACK : 1 + EXTRA_STACK))); #define incr_top(L) \ { \ diff --git a/VM/src/lgc.cpp b/VM/src/lgc.cpp index 835572fa..724b24b2 100644 --- a/VM/src/lgc.cpp +++ b/VM/src/lgc.cpp @@ -268,7 +268,7 @@ static void traverseclosure(global_State* g, Closure* cl) static void traversestack(global_State* g, lua_State* l, bool clearstack) { - markvalue(g, gt(l)); + markobject(g, l->gt); if (l->namecall) stringmark(l->namecall); for (StkId o = l->stack; o < l->top; o++) @@ -643,7 +643,7 @@ static void markroot(lua_State* L) g->weak = NULL; markobject(g, g->mainthread); /* make global table be traversed before main stack */ - markvalue(g, gt(g->mainthread)); + markobject(g, g->mainthread->gt); markvalue(g, registry(L)); markmt(g); g->gcstate = GCSpropagate; diff --git a/VM/src/lgc.h b/VM/src/lgc.h index 528d0944..2acb5d8a 100644 --- a/VM/src/lgc.h +++ b/VM/src/lgc.h @@ -77,7 +77,7 @@ #define luaC_checkGC(L) \ { \ - condhardstacktests(luaD_reallocstack(L, L->stacksize - EXTRA_STACK - 1)); \ + condhardstacktests(luaD_reallocstack(L, L->stacksize - (FFlag::LuauReduceStackReallocs ? EXTRA_STACK : 1 + EXTRA_STACK))); \ if (L->global->totalbytes >= L->global->GCthreshold) \ { \ condhardmemtests(luaC_validate(L), 1); \ diff --git a/VM/src/lgcdebug.cpp b/VM/src/lgcdebug.cpp index ce196520..30242e52 100644 --- a/VM/src/lgcdebug.cpp +++ b/VM/src/lgcdebug.cpp @@ -88,7 +88,7 @@ static void validateclosure(global_State* g, Closure* cl) static void validatestack(global_State* g, lua_State* l) { - validateref(g, obj2gco(l), gt(l)); + validateobjref(g, obj2gco(l), obj2gco(l->gt)); for (CallInfo* ci = l->base_ci; ci <= l->ci; ++ci) { @@ -370,6 +370,7 @@ static void dumpclosure(FILE* f, Closure* cl) fprintf(f, ",\"env\":"); dumpref(f, obj2gco(cl->env)); + if (cl->isC) { if (cl->nupvalues) @@ -411,11 +412,8 @@ static void dumpthread(FILE* f, lua_State* th) fprintf(f, "{\"type\":\"thread\",\"cat\":%d,\"size\":%d", th->memcat, int(size)); - if (iscollectable(&th->l_gt)) - { - fprintf(f, ",\"env\":"); - dumpref(f, gcvalue(&th->l_gt)); - } + fprintf(f, ",\"env\":"); + dumpref(f, obj2gco(th->gt)); Closure* tcl = 0; for (CallInfo* ci = th->base_ci; ci <= th->ci; ++ci) diff --git a/VM/src/lperf.cpp b/VM/src/lperf.cpp index 2f6c7297..da68e376 100644 --- a/VM/src/lperf.cpp +++ b/VM/src/lperf.cpp @@ -3,6 +3,12 @@ #include "lua.h" #ifdef _WIN32 +#ifndef WIN32_LEAN_AND_MEAN +#define WIN32_LEAN_AND_MEAN +#endif +#ifndef NOMINMAX +#define NOMINMAX +#endif #include #endif diff --git a/VM/src/lstate.cpp b/VM/src/lstate.cpp index 6762c638..d6d127c0 100644 --- a/VM/src/lstate.cpp +++ b/VM/src/lstate.cpp @@ -11,6 +11,7 @@ #include "ldebug.h" LUAU_FASTFLAG(LuauGcPagedSweep) +LUAU_FASTFLAGVARIABLE(LuauReduceStackReallocs, false) /* ** Main thread combines a thread state and the global state @@ -31,10 +32,11 @@ static void stack_init(lua_State* L1, lua_State* L) /* initialize stack array */ L1->stack = luaM_newarray(L, BASIC_STACK_SIZE + EXTRA_STACK, TValue, L1->memcat); L1->stacksize = BASIC_STACK_SIZE + EXTRA_STACK; + TValue* stack = L1->stack; for (int i = 0; i < BASIC_STACK_SIZE + EXTRA_STACK; i++) - setnilvalue(L1->stack + i); /* erase new stack */ - L1->top = L1->stack; - L1->stack_last = L1->stack + (L1->stacksize - EXTRA_STACK) - 1; + setnilvalue(stack + i); /* erase new stack */ + L1->top = stack; + L1->stack_last = stack + (L1->stacksize - (FFlag::LuauReduceStackReallocs ? EXTRA_STACK : 1 + EXTRA_STACK)); /* initialize first ci */ L1->ci->func = L1->top; setnilvalue(L1->top++); /* `function' entry for this `ci' */ @@ -55,7 +57,7 @@ static void f_luaopen(lua_State* L, void* ud) { global_State* g = L->global; stack_init(L, L); /* init stack */ - sethvalue(L, gt(L), luaH_new(L, 0, 2)); /* table of globals */ + L->gt = luaH_new(L, 0, 2); /* table of globals */ sethvalue(L, registry(L), luaH_new(L, 0, 2)); /* registry */ luaS_resize(L, LUA_MINSTRTABSIZE); /* initial size of string table */ luaT_init(L); @@ -69,6 +71,7 @@ static void preinit_state(lua_State* L, global_State* g) L->global = g; L->stack = NULL; L->stacksize = 0; + L->gt = NULL; L->openupval = NULL; L->size_ci = 0; L->nCcalls = L->baseCcalls = 0; @@ -80,7 +83,6 @@ static void preinit_state(lua_State* L, global_State* g) L->stackstate = 0; L->activememcat = 0; L->userdata = NULL; - setnilvalue(gt(L)); } static void close_state(lua_State* L) @@ -116,7 +118,7 @@ lua_State* luaE_newthread(lua_State* L) preinit_state(L1, L->global); L1->activememcat = L->activememcat; // inherit the active memory category stack_init(L1, L); /* init stack */ - setobj2n(L, gt(L1), gt(L)); /* share table of globals */ + L1->gt = L->gt; /* share table of globals */ L1->singlestep = L->singlestep; LUAU_ASSERT(iswhite(obj2gco(L1))); return L1; @@ -144,14 +146,30 @@ void lua_resetthread(lua_State* L) ci->top = ci->base + LUA_MINSTACK; setnilvalue(ci->func); L->ci = ci; - luaD_reallocCI(L, BASIC_CI_SIZE); + if (FFlag::LuauReduceStackReallocs) + { + if (L->size_ci != BASIC_CI_SIZE) + luaD_reallocCI(L, BASIC_CI_SIZE); + } + else + { + luaD_reallocCI(L, BASIC_CI_SIZE); + } /* clear thread state */ L->status = LUA_OK; L->base = L->ci->base; L->top = L->ci->base; L->nCcalls = L->baseCcalls = 0; /* clear thread stack */ - luaD_reallocstack(L, BASIC_STACK_SIZE); + if (FFlag::LuauReduceStackReallocs) + { + if (L->stacksize != BASIC_STACK_SIZE + EXTRA_STACK) + luaD_reallocstack(L, BASIC_STACK_SIZE); + } + else + { + luaD_reallocstack(L, BASIC_STACK_SIZE); + } for (int i = 0; i < L->stacksize; i++) setnilvalue(L->stack + i); } @@ -193,6 +211,7 @@ lua_State* lua_newstate(lua_Alloc f, void* ud) g->strt.size = 0; g->strt.nuse = 0; g->strt.hash = NULL; + setnilvalue(&g->pseudotemp); setnilvalue(registry(L)); g->gcstate = GCSpause; if (!FFlag::LuauGcPagedSweep) diff --git a/VM/src/lstate.h b/VM/src/lstate.h index 0708b71f..6dd89138 100644 --- a/VM/src/lstate.h +++ b/VM/src/lstate.h @@ -5,9 +5,6 @@ #include "lobject.h" #include "ltm.h" -/* table of globals */ -#define gt(L) (&L->l_gt) - /* registry */ #define registry(L) (&L->global->registry) @@ -177,6 +174,8 @@ typedef struct global_State TString* ttname[LUA_T_COUNT]; /* names for basic types */ TString* tmname[TM_N]; /* array with tag-method names */ + TValue pseudotemp; /* storage for temporary values used in pseudo2addr */ + TValue registry; /* registry table, used by lua_ref and LUA_REGISTRYINDEX */ int registryfree; /* next free slot in registry */ @@ -231,8 +230,7 @@ struct lua_State int cachedslot; /* when table operations or INDEX/NEWINDEX is invoked from Luau, what is the expected slot for lookup? */ - TValue l_gt; /* table of globals */ - TValue env; /* temporary place for environments */ + Table* gt; /* table of globals */ UpVal* openupval; /* list of open upvalues in this stack */ GCObject* gclist; diff --git a/VM/src/lvmexecute.cpp b/VM/src/lvmexecute.cpp index c3b662a2..6c31d36f 100644 --- a/VM/src/lvmexecute.cpp +++ b/VM/src/lvmexecute.cpp @@ -39,7 +39,7 @@ // When calling luau_callTM, we usually push the arguments to the top of the stack. // This is safe to do for complicated reasons: -// - stack guarantees 1 + EXTRA_STACK room beyond stack_last (see luaD_reallocstack) +// - stack guarantees EXTRA_STACK room beyond stack_last (see luaD_reallocstack) // - stack reallocation copies values past stack_last // All external function calls that can cause stack realloc or Lua calls have to be wrapped in VM_PROTECT diff --git a/VM/src/lvmload.cpp b/VM/src/lvmload.cpp index 2472cd90..4e5435b7 100644 --- a/VM/src/lvmload.cpp +++ b/VM/src/lvmload.cpp @@ -116,12 +116,12 @@ static void resolveImportSafe(lua_State* L, Table* env, TValue* k, uint32_t id) // note: we call getimport with nil propagation which means that accesses to table chains like A.B.C will resolve in nil // this is technically not necessary but it reduces the number of exceptions when loading scripts that rely on getfenv/setfenv for global // injection - luaV_getimport(L, hvalue(gt(L)), self->k, self->id, /* propagatenil= */ true); + luaV_getimport(L, L->gt, self->k, self->id, /* propagatenil= */ true); } }; ResolveImport ri = {k, id}; - if (hvalue(gt(L))->safeenv) + if (L->gt->safeenv) { // luaD_pcall will make sure that if any C/Lua calls during import resolution fail, the thread state is restored back int oldTop = lua_gettop(L); @@ -171,7 +171,7 @@ int luau_load(lua_State* L, const char* chunkname, const char* data, size_t size L->global->GCthreshold = SIZE_MAX; // env is 0 for current environment and a stack index otherwise - Table* envt = (env == 0) ? hvalue(gt(L)) : hvalue(luaA_toobject(L, env)); + Table* envt = (env == 0) ? L->gt : hvalue(luaA_toobject(L, env)); TString* source = luaS_new(L, chunkname); diff --git a/VM/src/lvmutils.cpp b/VM/src/lvmutils.cpp index 31dd59c8..8a18a4d4 100644 --- a/VM/src/lvmutils.cpp +++ b/VM/src/lvmutils.cpp @@ -55,7 +55,7 @@ static void callTMres(lua_State* L, StkId res, const TValue* f, const TValue* p1 { ptrdiff_t result = savestack(L, res); // using stack room beyond top is technically safe here, but for very complicated reasons: - // * The stack guarantees 1 + EXTRA_STACK room beyond stack_last (see luaD_reallocstack) will be allocated + // * The stack guarantees EXTRA_STACK room beyond stack_last (see luaD_reallocstack) will be allocated // * we cannot move luaD_checkstack above because the arguments are *sometimes* pointers to the lua // stack and checkstack may invalidate those pointers // * we cannot use savestack/restorestack because the arguments are sometimes on the C++ stack @@ -76,7 +76,7 @@ static void callTMres(lua_State* L, StkId res, const TValue* f, const TValue* p1 static void callTM(lua_State* L, const TValue* f, const TValue* p1, const TValue* p2, const TValue* p3) { // using stack room beyond top is technically safe here, but for very complicated reasons: - // * The stack guarantees 1 + EXTRA_STACK room beyond stack_last (see luaD_reallocstack) will be allocated + // * The stack guarantees EXTRA_STACK room beyond stack_last (see luaD_reallocstack) will be allocated // * we cannot move luaD_checkstack above because the arguments are *sometimes* pointers to the lua // stack and checkstack may invalidate those pointers // * we cannot use savestack/restorestack because the arguments are sometimes on the C++ stack diff --git a/fuzz/linter.cpp b/fuzz/linter.cpp index 55e0888b..04638d23 100644 --- a/fuzz/linter.cpp +++ b/fuzz/linter.cpp @@ -32,7 +32,7 @@ extern "C" int LLVMFuzzerTestOneInput(const uint8_t* Data, size_t Size) Luau::LintOptions lintOptions; lintOptions.warningMask = ~0ull; - Luau::lint(parseResult.root, names, typeck.globalScope, nullptr, lintOptions); + Luau::lint(parseResult.root, names, typeck.globalScope, nullptr, {}, lintOptions); } return 0; diff --git a/fuzz/proto.cpp b/fuzz/proto.cpp index 27e53492..f407248a 100644 --- a/fuzz/proto.cpp +++ b/fuzz/proto.cpp @@ -227,7 +227,7 @@ DEFINE_PROTO_FUZZER(const luau::StatBlock& message) if (kFuzzLinter) { Luau::LintOptions lintOptions = {~0u}; - Luau::lint(parseResult.root, names, sharedEnv.globalScope, module.get(), lintOptions); + Luau::lint(parseResult.root, names, sharedEnv.globalScope, module.get(), {}, lintOptions); } } diff --git a/tests/Autocomplete.test.cpp b/tests/Autocomplete.test.cpp index 59e12574..1978a0d3 100644 --- a/tests/Autocomplete.test.cpp +++ b/tests/Autocomplete.test.cpp @@ -1,7 +1,6 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #include "Luau/Autocomplete.h" #include "Luau/BuiltinDefinitions.h" -#include "Luau/Parser.h" #include "Luau/TypeInfer.h" #include "Luau/TypeVar.h" #include "Luau/VisitTypeVar.h" @@ -2610,6 +2609,27 @@ a = if temp then even elseif true then temp else e@9 CHECK(ac.entryMap.count("elseif") == 0); } +TEST_CASE_FIXTURE(ACFixture, "autocomplete_if_else_regression") +{ + ScopedFastFlag FFlagLuauIfElseExprFixCompletionIssue("LuauIfElseExprFixCompletionIssue", true); + check(R"( +local abcdef = 0; +local temp = false +local even = true; +local a +a = if temp then even else@1 +a = if temp then even else @2 +a = if temp then even else abc@3 + )"); + + auto ac = autocomplete('1'); + CHECK(ac.entryMap.count("else") == 0); + ac = autocomplete('2'); + CHECK(ac.entryMap.count("else") == 0); + ac = autocomplete('3'); + CHECK(ac.entryMap.count("abcdef")); +} + TEST_CASE_FIXTURE(ACFixture, "autocomplete_explicit_type_pack") { check(R"( diff --git a/tests/Compiler.test.cpp b/tests/Compiler.test.cpp index cd7a21d8..f982c86f 100644 --- a/tests/Compiler.test.cpp +++ b/tests/Compiler.test.cpp @@ -10,6 +10,11 @@ #include #include +namespace Luau +{ +std::string rep(const std::string& s, size_t n); +} + using namespace Luau; static std::string compileFunction(const char* source, uint32_t id) @@ -1960,15 +1965,6 @@ RETURN R8 -1 )"); } -static std::string rep(const std::string& s, size_t n) -{ - std::string r; - r.reserve(s.length() * n); - for (size_t i = 0; i < n; ++i) - r += s; - return r; -} - TEST_CASE("RecursionParse") { // The test forcibly pushes the stack limit during compilation; in NoOpt, the stack consumption is much larger so we need to reduce the limit to diff --git a/tests/Conformance.test.cpp b/tests/Conformance.test.cpp index 8b58d2ce..b09c1efb 100644 --- a/tests/Conformance.test.cpp +++ b/tests/Conformance.test.cpp @@ -492,6 +492,8 @@ TEST_CASE("DateTime") TEST_CASE("Debug") { + ScopedFastFlag luauTableFieldFunctionDebugname{"LuauTableFieldFunctionDebugname", true}; + runConformance("debug.lua"); } @@ -890,6 +892,12 @@ TEST_CASE("Coverage") lua_pushstring(L, function); lua_setfield(L, -2, "name"); + lua_pushinteger(L, linedefined); + lua_setfield(L, -2, "linedefined"); + + lua_pushinteger(L, depth); + lua_setfield(L, -2, "depth"); + for (size_t i = 0; i < size; ++i) if (hits[i] != -1) { diff --git a/tests/Fixture.cpp b/tests/Fixture.cpp index c74bfa27..dbdd06a4 100644 --- a/tests/Fixture.cpp +++ b/tests/Fixture.cpp @@ -2,6 +2,7 @@ #include "Fixture.h" #include "Luau/AstQuery.h" +#include "Luau/Parser.h" #include "Luau/TypeVar.h" #include "Luau/TypeAttach.h" #include "Luau/Transpiler.h" @@ -112,7 +113,7 @@ AstStatBlock* Fixture::parse(const std::string& source, const ParseOptions& pars sourceModule->name = fromString(mainModuleName); sourceModule->root = result.root; sourceModule->mode = parseMode(result.hotcomments); - sourceModule->ignoreLints = LintWarning::parseMask(result.hotcomments); + sourceModule->hotcomments = std::move(result.hotcomments); if (!result.errors.empty()) { @@ -157,6 +158,7 @@ CheckResult Fixture::check(const std::string& source) LintResult Fixture::lint(const std::string& source, const std::optional& lintOptions) { ParseOptions parseOptions; + parseOptions.captureComments = true; configResolver.defaultConfig.mode = Mode::Nonstrict; parse(source, parseOptions); diff --git a/tests/Fixture.h b/tests/Fixture.h index ab852ef6..4e45a952 100644 --- a/tests/Fixture.h +++ b/tests/Fixture.h @@ -8,7 +8,6 @@ #include "Luau/Linter.h" #include "Luau/Location.h" #include "Luau/ModuleResolver.h" -#include "Luau/Parser.h" #include "Luau/ToString.h" #include "Luau/TypeInfer.h" #include "Luau/TypeVar.h" diff --git a/tests/Frontend.test.cpp b/tests/Frontend.test.cpp index ea1a08fe..8a59acd1 100644 --- a/tests/Frontend.test.cpp +++ b/tests/Frontend.test.cpp @@ -2,7 +2,6 @@ #include "Luau/AstQuery.h" #include "Luau/BuiltinDefinitions.h" #include "Luau/Frontend.h" -#include "Luau/Parser.h" #include "Luau/RequireTracer.h" #include "Fixture.h" @@ -897,8 +896,6 @@ TEST_CASE_FIXTURE(FrontendFixture, "clearStats") TEST_CASE_FIXTURE(FrontendFixture, "typecheck_twice_for_ast_types") { - ScopedFastFlag sffs("LuauTypeCheckTwice", true); - fileResolver.source["Module/A"] = R"( local a = 1 )"; diff --git a/tests/Linter.test.cpp b/tests/Linter.test.cpp index 577415fc..d4b97360 100644 --- a/tests/Linter.test.cpp +++ b/tests/Linter.test.cpp @@ -1395,12 +1395,10 @@ TEST_CASE_FIXTURE(Fixture, "DeprecatedApi") TypeId colorType = typeChecker.globalTypes.addType(TableTypeVar{{}, std::nullopt, typeChecker.globalScope->level, Luau::TableState::Sealed}); - getMutable(colorType)->props = { - {"toHSV", {typeChecker.anyType, /* deprecated= */ true, "Color3:ToHSV"} } - }; + getMutable(colorType)->props = {{"toHSV", {typeChecker.anyType, /* deprecated= */ true, "Color3:ToHSV"}}}; addGlobalBinding(typeChecker, "Color3", Binding{colorType, {}}); - + freeze(typeChecker.globalTypes); LintResult result = lintTyped(R"( @@ -1554,8 +1552,46 @@ _ = (math.random() < 0.5 and false) or 42 -- currently ignored )"); REQUIRE_EQ(result.warnings.size(), 2); - CHECK_EQ(result.warnings[0].text, "The and-or expression always evaluates to the second alternative because the first alternative is false; consider using if-then-else expression instead"); - CHECK_EQ(result.warnings[1].text, "The and-or expression always evaluates to the second alternative because the first alternative is nil; consider using if-then-else expression instead"); + CHECK_EQ(result.warnings[0].text, "The and-or expression always evaluates to the second alternative because the first alternative is false; " + "consider using if-then-else expression instead"); + CHECK_EQ(result.warnings[1].text, "The and-or expression always evaluates to the second alternative because the first alternative is nil; " + "consider using if-then-else expression instead"); +} + +TEST_CASE_FIXTURE(Fixture, "WrongComment") +{ + ScopedFastFlag sff("LuauParseAllHotComments", true); + + LintResult result = lint(R"( +--!strict +--!struct +--!nolintGlobal +--!nolint Global +--!nolint KnownGlobal +--!nolint UnknownGlobal +--! no more lint +--!strict here +do end +--!nolint +)"); + + REQUIRE_EQ(result.warnings.size(), 6); + CHECK_EQ(result.warnings[0].text, "Unknown comment directive 'struct'; did you mean 'strict'?"); + CHECK_EQ(result.warnings[1].text, "Unknown comment directive 'nolintGlobal'"); + CHECK_EQ(result.warnings[2].text, "nolint directive refers to unknown lint rule 'Global'"); + CHECK_EQ(result.warnings[3].text, "nolint directive refers to unknown lint rule 'KnownGlobal'; did you mean 'UnknownGlobal'?"); + CHECK_EQ(result.warnings[4].text, "Comment directive with the type checking mode has extra symbols at the end of the line"); + CHECK_EQ(result.warnings[5].text, "Comment directive is ignored because it is placed after the first non-comment token"); +} + +TEST_CASE_FIXTURE(Fixture, "WrongCommentMuteSelf") +{ + LintResult result = lint(R"( +--!nolint +--!struct +)"); + + REQUIRE_EQ(result.warnings.size(), 0); // --!nolint disables WrongComment lint :) } TEST_SUITE_END(); diff --git a/tests/NonstrictMode.test.cpp b/tests/NonstrictMode.test.cpp index 931a8403..5bad9901 100644 --- a/tests/NonstrictMode.test.cpp +++ b/tests/NonstrictMode.test.cpp @@ -1,5 +1,4 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details -#include "Luau/Parser.h" #include "Luau/Scope.h" #include "Luau/TypeInfer.h" #include "Luau/TypeVar.h" diff --git a/tests/Parser.test.cpp b/tests/Parser.test.cpp index c1a8887b..0d4c088d 100644 --- a/tests/Parser.test.cpp +++ b/tests/Parser.test.cpp @@ -1,6 +1,5 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #include "Luau/Parser.h" -#include "Luau/TypeInfer.h" #include "Fixture.h" #include "ScopedFlags.h" @@ -300,8 +299,9 @@ TEST_CASE_FIXTURE(Fixture, "functions_can_have_return_annotations") AstStatFunction* statFunction = block->body.data[0]->as(); REQUIRE(statFunction != nullptr); - CHECK_EQ(statFunction->func->returnAnnotation.types.size, 1); - CHECK(statFunction->func->returnAnnotation.tailType == nullptr); + REQUIRE(statFunction->func->returnAnnotation.has_value()); + CHECK_EQ(statFunction->func->returnAnnotation->types.size, 1); + CHECK(statFunction->func->returnAnnotation->tailType == nullptr); } TEST_CASE_FIXTURE(Fixture, "functions_can_have_a_function_type_annotation") @@ -316,9 +316,9 @@ TEST_CASE_FIXTURE(Fixture, "functions_can_have_a_function_type_annotation") AstStatFunction* statFunc = block->body.data[0]->as(); REQUIRE(statFunc != nullptr); - AstArray& retTypes = statFunc->func->returnAnnotation.types; - REQUIRE(statFunc->func->hasReturnAnnotation); - CHECK(statFunc->func->returnAnnotation.tailType == nullptr); + REQUIRE(statFunc->func->returnAnnotation.has_value()); + CHECK(statFunc->func->returnAnnotation->tailType == nullptr); + AstArray& retTypes = statFunc->func->returnAnnotation->types; REQUIRE(retTypes.size == 1); AstTypeFunction* funTy = retTypes.data[0]->as(); @@ -337,9 +337,9 @@ TEST_CASE_FIXTURE(Fixture, "function_return_type_should_disambiguate_from_functi AstStatFunction* statFunc = block->body.data[0]->as(); REQUIRE(statFunc != nullptr); - AstArray& retTypes = statFunc->func->returnAnnotation.types; - REQUIRE(statFunc->func->hasReturnAnnotation); - CHECK(statFunc->func->returnAnnotation.tailType == nullptr); + REQUIRE(statFunc->func->returnAnnotation.has_value()); + CHECK(statFunc->func->returnAnnotation->tailType == nullptr); + AstArray& retTypes = statFunc->func->returnAnnotation->types; REQUIRE(retTypes.size == 2); AstTypeReference* ty0 = retTypes.data[0]->as(); @@ -363,9 +363,9 @@ TEST_CASE_FIXTURE(Fixture, "function_return_type_should_parse_as_function_type_a AstStatFunction* statFunc = block->body.data[0]->as(); REQUIRE(statFunc != nullptr); - AstArray& retTypes = statFunc->func->returnAnnotation.types; - REQUIRE(statFunc->func->hasReturnAnnotation); - CHECK(statFunc->func->returnAnnotation.tailType == nullptr); + REQUIRE(statFunc->func->returnAnnotation.has_value()); + CHECK(statFunc->func->returnAnnotation->tailType == nullptr); + AstArray& retTypes = statFunc->func->returnAnnotation->types; REQUIRE(retTypes.size == 1); AstTypeFunction* funTy = retTypes.data[0]->as(); @@ -707,12 +707,25 @@ TEST_CASE_FIXTURE(Fixture, "mode_is_unset_if_no_hot_comment") TEST_CASE_FIXTURE(Fixture, "sense_hot_comment_on_first_line") { - ParseResult result = parseEx(" --!strict "); + ParseOptions options; + options.captureComments = true; + + ParseResult result = parseEx(" --!strict ", options); std::optional mode = parseMode(result.hotcomments); REQUIRE(bool(mode)); CHECK_EQ(int(*mode), int(Mode::Strict)); } +TEST_CASE_FIXTURE(Fixture, "non_header_hot_comments") +{ + ParseOptions options; + options.captureComments = true; + + ParseResult result = parseEx("do end --!strict", options); + std::optional mode = parseMode(result.hotcomments); + REQUIRE(!mode); +} + TEST_CASE_FIXTURE(Fixture, "stop_if_line_ends_with_hyphen") { CHECK_THROWS_AS(parse(" -"), std::exception); @@ -720,7 +733,10 @@ TEST_CASE_FIXTURE(Fixture, "stop_if_line_ends_with_hyphen") TEST_CASE_FIXTURE(Fixture, "nonstrict_mode") { - ParseResult result = parseEx("--!nonstrict"); + ParseOptions options; + options.captureComments = true; + + ParseResult result = parseEx("--!nonstrict", options); CHECK(result.errors.empty()); std::optional mode = parseMode(result.hotcomments); REQUIRE(bool(mode)); @@ -729,7 +745,10 @@ TEST_CASE_FIXTURE(Fixture, "nonstrict_mode") TEST_CASE_FIXTURE(Fixture, "nocheck_mode") { - ParseResult result = parseEx("--!nocheck"); + ParseOptions options; + options.captureComments = true; + + ParseResult result = parseEx("--!nocheck", options); CHECK(result.errors.empty()); std::optional mode = parseMode(result.hotcomments); REQUIRE(bool(mode)); @@ -1498,8 +1517,6 @@ return TEST_CASE_FIXTURE(Fixture, "parse_error_broken_comment") { - ScopedFastFlag luauStartingBrokenComment{"LuauStartingBrokenComment", true}; - const char* expected = "Expected identifier when parsing expression, got unfinished comment"; matchParseError("--[[unfinished work", expected); diff --git a/tests/Repl.test.cpp b/tests/Repl.test.cpp index 1f9c9739..87a1e1e2 100644 --- a/tests/Repl.test.cpp +++ b/tests/Repl.test.cpp @@ -73,7 +73,7 @@ public: private: std::unique_ptr luaState; - // This is a simplicitic and incomplete pretty printer. + // This is a simplistic and incomplete pretty printer. // It is included here to test that the pretty printer hook is being called. // More elaborate tests to ensure correct output can be added if we introduce // a more feature rich pretty printer. @@ -158,12 +158,25 @@ TEST_CASE_FIXTURE(ReplFixture, "CompleteGlobalVariables") myvariable1 = 5 myvariable2 = 5 )"); - CompletionSet completions = getCompletionSet("myvar"); + { + // Try to complete globals that are added by the user's script + CompletionSet completions = getCompletionSet("myvar"); - std::string prefix = ""; - CHECK(completions.size() == 2); - CHECK(checkCompletion(completions, prefix, "myvariable1")); - CHECK(checkCompletion(completions, prefix, "myvariable2")); + std::string prefix = ""; + CHECK(completions.size() == 2); + CHECK(checkCompletion(completions, prefix, "myvariable1")); + CHECK(checkCompletion(completions, prefix, "myvariable2")); + } + { + // Try completing some builtin functions + CompletionSet completions = getCompletionSet("math.m"); + + std::string prefix = "math."; + CHECK(completions.size() == 3); + CHECK(checkCompletion(completions, prefix, "max(")); + CHECK(checkCompletion(completions, prefix, "min(")); + CHECK(checkCompletion(completions, prefix, "modf(")); + } } TEST_CASE_FIXTURE(ReplFixture, "CompleteTableKeys") @@ -206,4 +219,188 @@ TEST_CASE_FIXTURE(ReplFixture, "StringMethods") } } +TEST_CASE_FIXTURE(ReplFixture, "TableWithMetatableIndexTable") +{ + runCode(L, R"( + -- Create 't' which is a table with a metatable with an __index table + mt = {} + mt.__index = mt + + t = {} + setmetatable(t, mt) + + mt.mtkey1 = {x="x value", y="y value", 1, 2} + mt.mtkey2 = 2 + + t.tkey1 = {data1 = 2, data2 = "str", 3, 4} + t.tkey2 = 4 +)"); + { + CompletionSet completions = getCompletionSet("t.t"); + + std::string prefix = "t."; + CHECK(completions.size() == 2); + CHECK(checkCompletion(completions, prefix, "tkey1")); + CHECK(checkCompletion(completions, prefix, "tkey2")); + } + { + CompletionSet completions = getCompletionSet("t.tkey1.data2:re"); + + std::string prefix = "t.tkey1.data2:"; + CHECK(completions.size() == 2); + CHECK(checkCompletion(completions, prefix, "rep(")); + CHECK(checkCompletion(completions, prefix, "reverse(")); + } + { + CompletionSet completions = getCompletionSet("t.mtk"); + + std::string prefix = "t."; + CHECK(completions.size() == 2); + CHECK(checkCompletion(completions, prefix, "mtkey1")); + CHECK(checkCompletion(completions, prefix, "mtkey2")); + } + { + CompletionSet completions = getCompletionSet("t.mtkey1."); + + std::string prefix = "t.mtkey1."; + CHECK(completions.size() == 2); + CHECK(checkCompletion(completions, prefix, "x")); + CHECK(checkCompletion(completions, prefix, "y")); + } +} + +TEST_CASE_FIXTURE(ReplFixture, "TableWithMetatableIndexFunction") +{ + runCode(L, R"( + -- Create 't' which is a table with a metatable with an __index function + mt = {} + mt.__index = function(table, key) + print("mt.__index called") + if key == "foo" then + return "FOO" + elseif key == "bar" then + return "BAR" + else + return nil + end + end + + t = {} + setmetatable(t, mt) + t.tkey = 0 +)"); + { + CompletionSet completions = getCompletionSet("t.t"); + + std::string prefix = "t."; + CHECK(completions.size() == 1); + CHECK(checkCompletion(completions, prefix, "tkey")); + } + { + // t.foo is a valid key, but should not be completed because it requires calling an __index function + CompletionSet completions = getCompletionSet("t.foo"); + + CHECK(completions.size() == 0); + } + { + // t.foo is a valid key, but should not be found because it requires calling an __index function + CompletionSet completions = getCompletionSet("t.foo:"); + + CHECK(completions.size() == 0); + } +} + +TEST_CASE_FIXTURE(ReplFixture, "TableWithMultipleMetatableIndexTables") +{ + runCode(L, R"( + -- Create a table with a chain of metatables + mt2 = {} + mt2.__index = mt2 + + mt = {} + mt.__index = mt + setmetatable(mt, mt2) + + t = {} + setmetatable(t, mt) + + mt2.mt2key = {x=1, y=2} + mt.mtkey = 2 + t.tkey = 3 +)"); + { + CompletionSet completions = getCompletionSet("t."); + + std::string prefix = "t."; + CHECK(completions.size() == 4); + CHECK(checkCompletion(completions, prefix, "__index")); + CHECK(checkCompletion(completions, prefix, "tkey")); + CHECK(checkCompletion(completions, prefix, "mtkey")); + CHECK(checkCompletion(completions, prefix, "mt2key")); + } + { + CompletionSet completions = getCompletionSet("t.__index."); + + std::string prefix = "t.__index."; + CHECK(completions.size() == 3); + CHECK(checkCompletion(completions, prefix, "__index")); + CHECK(checkCompletion(completions, prefix, "mtkey")); + CHECK(checkCompletion(completions, prefix, "mt2key")); + } + { + CompletionSet completions = getCompletionSet("t.mt2key."); + + std::string prefix = "t.mt2key."; + CHECK(completions.size() == 2); + CHECK(checkCompletion(completions, prefix, "x")); + CHECK(checkCompletion(completions, prefix, "y")); + } +} + +TEST_CASE_FIXTURE(ReplFixture, "TableWithDeepMetatableIndexTables") +{ + runCode(L, R"( +-- Creates a table with a chain of metatables of length `count` +function makeChainedTable(count) + local result = {} + result.__index = result + result[string.format("entry%d", count)] = { count = count } + if count == 0 then + return result + else + return setmetatable(result, makeChainedTable(count - 1)) + end +end + +t30 = makeChainedTable(30) +t60 = makeChainedTable(60) +)"); + { + // Check if entry0 exists + CompletionSet completions = getCompletionSet("t30.entry0"); + + std::string prefix = "t30."; + CHECK(checkCompletion(completions, prefix, "entry0")); + } + { + // Check if entry0.count exists + CompletionSet completions = getCompletionSet("t30.entry0.co"); + + std::string prefix = "t30.entry0."; + CHECK(checkCompletion(completions, prefix, "count")); + } + { + // Check if entry0 exists. With the max traversal limit of 50 in the repl, this should fail. + CompletionSet completions = getCompletionSet("t60.entry0"); + + CHECK(completions.size() == 0); + } + { + // Check if entry0.count exists. With the max traversal limit of 50 in the repl, this should fail. + CompletionSet completions = getCompletionSet("t60.entry0.co"); + + CHECK(completions.size() == 0); + } +} + TEST_SUITE_END(); diff --git a/tests/RequireTracer.test.cpp b/tests/RequireTracer.test.cpp index b9fd04d6..ba03f363 100644 --- a/tests/RequireTracer.test.cpp +++ b/tests/RequireTracer.test.cpp @@ -1,6 +1,6 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details -#include "Luau/Parser.h" #include "Luau/RequireTracer.h" +#include "Luau/Parser.h" #include "Fixture.h" diff --git a/tests/TypeInfer.aliases.test.cpp b/tests/TypeInfer.aliases.test.cpp index a8729268..31d7ef10 100644 --- a/tests/TypeInfer.aliases.test.cpp +++ b/tests/TypeInfer.aliases.test.cpp @@ -598,15 +598,13 @@ TEST_CASE_FIXTURE(Fixture, "generic_typevars_are_not_considered_to_escape_their_ /* * The two-pass alias definition system starts by ascribing a free TypeVar to each alias. It then * circles back to fill in the actual type later on. - * + * * If this free type is unified with something degenerate like `any`, we need to take extra care * to ensure that the alias actually binds to the type that the user expected. */ TEST_CASE_FIXTURE(Fixture, "forward_declared_alias_is_not_clobbered_by_prior_unification_with_any") { - ScopedFastFlag sff[] = { - {"LuauTwoPassAliasDefinitionFix", true} - }; + ScopedFastFlag sff[] = {{"LuauTwoPassAliasDefinitionFix", true}}; CheckResult result = check(R"( local function x() diff --git a/tests/TypeInfer.annotations.test.cpp b/tests/TypeInfer.annotations.test.cpp index 572b882d..2ad11d01 100644 --- a/tests/TypeInfer.annotations.test.cpp +++ b/tests/TypeInfer.annotations.test.cpp @@ -1,6 +1,5 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #include "Luau/BuiltinDefinitions.h" -#include "Luau/Parser.h" #include "Luau/TypeInfer.h" #include "Luau/TypeVar.h" diff --git a/tests/TypeInfer.builtins.test.cpp b/tests/TypeInfer.builtins.test.cpp index df06884d..f3dfb214 100644 --- a/tests/TypeInfer.builtins.test.cpp +++ b/tests/TypeInfer.builtins.test.cpp @@ -1,5 +1,4 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details -#include "Luau/Parser.h" #include "Luau/TypeInfer.h" #include "Luau/BuiltinDefinitions.h" diff --git a/tests/TypeInfer.classes.test.cpp b/tests/TypeInfer.classes.test.cpp index 0283ae19..98fa66eb 100644 --- a/tests/TypeInfer.classes.test.cpp +++ b/tests/TypeInfer.classes.test.cpp @@ -1,6 +1,5 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #include "Luau/BuiltinDefinitions.h" -#include "Luau/Parser.h" #include "Luau/TypeInfer.h" #include "Luau/TypeVar.h" diff --git a/tests/TypeInfer.definitions.test.cpp b/tests/TypeInfer.definitions.test.cpp index 2652486b..c6d55793 100644 --- a/tests/TypeInfer.definitions.test.cpp +++ b/tests/TypeInfer.definitions.test.cpp @@ -1,6 +1,5 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #include "Luau/BuiltinDefinitions.h" -#include "Luau/Parser.h" #include "Luau/TypeInfer.h" #include "Luau/TypeVar.h" diff --git a/tests/TypeInfer.generics.test.cpp b/tests/TypeInfer.generics.test.cpp index 8a2c6f27..f8fccf6b 100644 --- a/tests/TypeInfer.generics.test.cpp +++ b/tests/TypeInfer.generics.test.cpp @@ -1,5 +1,4 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details -#include "Luau/Parser.h" #include "Luau/TypeInfer.h" #include "Luau/TypeVar.h" diff --git a/tests/TypeInfer.intersectionTypes.test.cpp b/tests/TypeInfer.intersectionTypes.test.cpp index 93c0baf6..d677e28d 100644 --- a/tests/TypeInfer.intersectionTypes.test.cpp +++ b/tests/TypeInfer.intersectionTypes.test.cpp @@ -1,5 +1,4 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details -#include "Luau/Parser.h" #include "Luau/TypeInfer.h" #include "Luau/TypeVar.h" diff --git a/tests/TypeInfer.provisional.test.cpp b/tests/TypeInfer.provisional.test.cpp index 2bcd840c..eee0e0f1 100644 --- a/tests/TypeInfer.provisional.test.cpp +++ b/tests/TypeInfer.provisional.test.cpp @@ -1,5 +1,4 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details -#include "Luau/Parser.h" #include "Luau/TypeInfer.h" #include "Fixture.h" diff --git a/tests/TypeInfer.singletons.test.cpp b/tests/TypeInfer.singletons.test.cpp index df365fda..9021700d 100644 --- a/tests/TypeInfer.singletons.test.cpp +++ b/tests/TypeInfer.singletons.test.cpp @@ -426,8 +426,6 @@ TEST_CASE_FIXTURE(Fixture, "if_then_else_expression_singleton_options") {"LuauSingletonTypes", true}, {"LuauParseSingletonTypes", true}, {"LuauExpectedTypesOfProperties", true}, - {"LuauIfElseExpectedType2", true}, - {"LuauIfElseBranchTypeUnion", true}, }; CheckResult result = check(R"( diff --git a/tests/TypeInfer.tables.test.cpp b/tests/TypeInfer.tables.test.cpp index f19cb618..6bcd4b99 100644 --- a/tests/TypeInfer.tables.test.cpp +++ b/tests/TypeInfer.tables.test.cpp @@ -1,6 +1,5 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #include "Luau/BuiltinDefinitions.h" -#include "Luau/Parser.h" #include "Luau/TypeInfer.h" #include "Luau/TypeVar.h" @@ -2168,8 +2167,6 @@ b() TEST_CASE_FIXTURE(Fixture, "length_operator_union") { - ScopedFastFlag luauLengthOnCompositeType{"LuauLengthOnCompositeType", true}; - CheckResult result = check(R"( local x: {number} | {string} local y = #x @@ -2180,8 +2177,6 @@ local y = #x TEST_CASE_FIXTURE(Fixture, "length_operator_intersection") { - ScopedFastFlag luauLengthOnCompositeType{"LuauLengthOnCompositeType", true}; - CheckResult result = check(R"( local x: {number} & {z:string} -- mixed tables are evil local y = #x @@ -2192,8 +2187,6 @@ local y = #x TEST_CASE_FIXTURE(Fixture, "length_operator_non_table_union") { - ScopedFastFlag luauLengthOnCompositeType{"LuauLengthOnCompositeType", true}; - CheckResult result = check(R"( local x: {number} | any | string local y = #x @@ -2204,8 +2197,6 @@ local y = #x TEST_CASE_FIXTURE(Fixture, "length_operator_union_errors") { - ScopedFastFlag luauLengthOnCompositeType{"LuauLengthOnCompositeType", true}; - CheckResult result = check(R"( local x: {number} | number | string local y = #x @@ -2214,4 +2205,38 @@ local y = #x LUAU_REQUIRE_ERROR_COUNT(1, result); } +TEST_CASE_FIXTURE(Fixture, "dont_hang_when_trying_to_look_up_in_cyclic_metatable_index") +{ + ScopedFastFlag sff{"LuauTerminateCyclicMetatableIndexLookup", true}; + + // t :: t1 where t1 = {metatable {__index: t1, __tostring: (t1) -> string}} + CheckResult result = check(R"( + local mt = {} + local t = setmetatable({}, mt) + mt.__index = t + + function mt:__tostring() + return t.p + end + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ("Type 't' does not have key 'p'", toString(result.errors[0])); +} + +TEST_CASE_FIXTURE(Fixture, "give_up_after_one_metatable_index_look_up") +{ + CheckResult result = check(R"( + local data = { x = 5 } + local t1 = setmetatable({}, { __index = data }) + local t2 = setmetatable({}, t1) -- note: must be t1, not a new table + + local x1 = t1.x -- ok + local x2 = t2.x -- nope + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ("Type 't2' does not have key 'x'", toString(result.errors[0])); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.test.cpp b/tests/TypeInfer.test.cpp index 531a382f..32358571 100644 --- a/tests/TypeInfer.test.cpp +++ b/tests/TypeInfer.test.cpp @@ -2,7 +2,6 @@ #include "Luau/AstQuery.h" #include "Luau/BuiltinDefinitions.h" -#include "Luau/Parser.h" #include "Luau/Scope.h" #include "Luau/TypeInfer.h" #include "Luau/TypeVar.h" @@ -4654,8 +4653,6 @@ a = setmetatable(a, { __call = function(x) end }) TEST_CASE_FIXTURE(Fixture, "infer_through_group_expr") { - ScopedFastFlag luauGroupExpectedType{"LuauGroupExpectedType", true}; - CheckResult result = check(R"( local function f(a: (number, number) -> number) return a(1, 3) end f(((function(a, b) return a + b end))) @@ -4735,21 +4732,14 @@ local a = if false then "a" elseif false then "b" else "c" TEST_CASE_FIXTURE(Fixture, "tc_if_else_expressions_type_union") { - ScopedFastFlag sff3{"LuauIfElseBranchTypeUnion", true}; + CheckResult result = check(R"(local a: number? = if true then 42 else nil)"); - { - CheckResult result = check(R"(local a: number? = if true then 42 else nil)"); - - LUAU_REQUIRE_NO_ERRORS(result); - CHECK_EQ(toString(requireType("a"), {true}), "number?"); - } + LUAU_REQUIRE_NO_ERRORS(result); + CHECK_EQ(toString(requireType("a"), {true}), "number?"); } TEST_CASE_FIXTURE(Fixture, "tc_if_else_expressions_expected_type_1") { - ScopedFastFlag luauIfElseExpectedType2{"LuauIfElseExpectedType2", true}; - ScopedFastFlag luauIfElseBranchTypeUnion{"LuauIfElseBranchTypeUnion", true}; - CheckResult result = check(R"( type X = {number | string} local a: X = if true then {"1", 2, 3} else {4, 5, 6} @@ -4761,9 +4751,6 @@ local a: X = if true then {"1", 2, 3} else {4, 5, 6} TEST_CASE_FIXTURE(Fixture, "tc_if_else_expressions_expected_type_2") { - ScopedFastFlag luauIfElseExpectedType2{"LuauIfElseExpectedType2", true}; - ScopedFastFlag luauIfElseBranchTypeUnion{"LuauIfElseBranchTypeUnion", true}; - CheckResult result = check(R"( local a: number? = if true then 1 else nil )"); @@ -4773,8 +4760,6 @@ local a: number? = if true then 1 else nil TEST_CASE_FIXTURE(Fixture, "tc_if_else_expressions_expected_type_3") { - ScopedFastFlag luauIfElseExpectedType2{"LuauIfElseExpectedType2", true}; - CheckResult result = check(R"( local function times(n: any, f: () -> T) local result: {T} = {} @@ -5058,8 +5043,6 @@ end TEST_CASE_FIXTURE(Fixture, "recursive_metatable_crash") { - ScopedFastFlag luauMetatableAreEqualRecursion{"LuauMetatableAreEqualRecursion", true}; - CheckResult result = check(R"( local function getIt() local y @@ -5076,8 +5059,6 @@ local c = a or b TEST_CASE_FIXTURE(Fixture, "bound_typepack_promote") { - ScopedFastFlag luauCommittingTxnLogFreeTpPromote{"LuauCommittingTxnLogFreeTpPromote", true}; - // No assertions should trigger check(R"( local function p() @@ -5251,7 +5232,6 @@ TEST_CASE_FIXTURE(Fixture, "taking_the_length_of_string_singleton") {"LuauDiscriminableUnions2", true}, {"LuauRefactorTypeVarQuestions", true}, {"LuauSingletonTypes", true}, - {"LuauLengthOnCompositeType", true}, }; CheckResult result = check(R"( @@ -5272,7 +5252,6 @@ TEST_CASE_FIXTURE(Fixture, "taking_the_length_of_union_of_string_singleton") {"LuauDiscriminableUnions2", true}, {"LuauRefactorTypeVarQuestions", true}, {"LuauSingletonTypes", true}, - {"LuauLengthOnCompositeType", true}, }; CheckResult result = check(R"( diff --git a/tests/TypeInfer.tryUnify.test.cpp b/tests/TypeInfer.tryUnify.test.cpp index 8c7fb79a..4669ea8e 100644 --- a/tests/TypeInfer.tryUnify.test.cpp +++ b/tests/TypeInfer.tryUnify.test.cpp @@ -1,5 +1,4 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details -#include "Luau/Parser.h" #include "Luau/Scope.h" #include "Luau/TypeInfer.h" #include "Luau/TypeVar.h" diff --git a/tests/TypeInfer.typePacks.cpp b/tests/TypeInfer.typePacks.cpp index 079870f5..cbe2e48f 100644 --- a/tests/TypeInfer.typePacks.cpp +++ b/tests/TypeInfer.typePacks.cpp @@ -1,6 +1,5 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #include "Luau/BuiltinDefinitions.h" -#include "Luau/Parser.h" #include "Luau/TypeInfer.h" #include "Luau/TypeVar.h" @@ -930,8 +929,6 @@ type R = { m: F } TEST_CASE_FIXTURE(Fixture, "pack_tail_unification_check") { - ScopedFastFlag luauUnifyPackTails{"LuauUnifyPackTails", true}; - CheckResult result = check(R"( local a: () -> (number, ...string) local b: () -> (number, ...boolean) diff --git a/tests/TypeInfer.unionTypes.test.cpp b/tests/TypeInfer.unionTypes.test.cpp index 759794e6..3b53ddfe 100644 --- a/tests/TypeInfer.unionTypes.test.cpp +++ b/tests/TypeInfer.unionTypes.test.cpp @@ -1,5 +1,4 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details -#include "Luau/Parser.h" #include "Luau/TypeInfer.h" #include "Luau/TypeVar.h" diff --git a/tests/TypePack.test.cpp b/tests/TypePack.test.cpp index 8b056544..c4931578 100644 --- a/tests/TypePack.test.cpp +++ b/tests/TypePack.test.cpp @@ -1,5 +1,4 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details -#include "Luau/Parser.h" #include "Luau/TypeInfer.h" #include "Luau/TypeVar.h" diff --git a/tests/TypeVar.test.cpp b/tests/TypeVar.test.cpp index 329e7b1f..e43161fa 100644 --- a/tests/TypeVar.test.cpp +++ b/tests/TypeVar.test.cpp @@ -1,5 +1,4 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details -#include "Luau/Parser.h" #include "Luau/Scope.h" #include "Luau/TypeInfer.h" #include "Luau/TypeVar.h" diff --git a/tests/conformance/coroutine.lua b/tests/conformance/coroutine.lua index f2ecc96b..b4f81bba 100644 --- a/tests/conformance/coroutine.lua +++ b/tests/conformance/coroutine.lua @@ -371,6 +371,15 @@ do st, msg = coroutine.close(co) assert(st and msg == nil) assert(f() == 42) + + -- closing a coroutine with a large stack + co = coroutine.create(function() + local function f(depth) return if depth > 0 then f(depth - 1) + depth else 0 end + coroutine.yield(f(100)) + end) + assert(coroutine.resume(co)) + st, msg = coroutine.close(co) + assert(st and msg == nil) end return 'OK' diff --git a/tests/conformance/coverage.lua b/tests/conformance/coverage.lua index f899603f..14d843a4 100644 --- a/tests/conformance/coverage.lua +++ b/tests/conformance/coverage.lua @@ -49,16 +49,24 @@ foo() c = getcoverage(foo) assert(#c == 1) assert(c[1].name == "foo") +assert(c[1].linedefined == 4) +assert(c[1].depth == 0) assert(validate(c[1], {5, 6, 7}, {})) bar() c = getcoverage(bar) assert(#c == 3) assert(c[1].name == "bar") +assert(c[1].linedefined == 10) +assert(c[1].depth == 0) assert(validate(c[1], {11, 15, 19}, {})) assert(c[2].name == "one") +assert(c[2].linedefined == 11) +assert(c[2].depth == 1) assert(validate(c[2], {12}, {})) assert(c[3].name == nil) +assert(c[3].linedefined == 15) +assert(c[3].depth == 1) assert(validate(c[3], {}, {16})) return 'OK' diff --git a/tests/conformance/debug.lua b/tests/conformance/debug.lua index 0e410000..0c8cc2d8 100644 --- a/tests/conformance/debug.lua +++ b/tests/conformance/debug.lua @@ -76,6 +76,9 @@ assert(baz(co, 2, "n") == nil) assert(baz(math.sqrt, "n") == "sqrt") assert(baz(math.sqrt, "f") == math.sqrt) -- yes this is pointless +local t = { foo = function() return 1 end } +assert(baz(t.foo, "n") == "foo") + -- info multi-arg returns function quux(...) return {debug.info(...)} diff --git a/tests/main.cpp b/tests/main.cpp index cd24e100..2af9f702 100644 --- a/tests/main.cpp +++ b/tests/main.cpp @@ -10,6 +10,9 @@ #ifndef WIN32_LEAN_AND_MEAN #define WIN32_LEAN_AND_MEAN #endif +#ifndef NOMINMAX +#define NOMINMAX +#endif #include // IsDebuggerPresent #endif @@ -52,7 +55,7 @@ static bool debuggerPresent() #endif } -static int assertionHandler(const char* expr, const char* file, int line, const char* function) +static int testAssertionHandler(const char* expr, const char* file, int line, const char* function) { if (debuggerPresent()) LUAU_DEBUGBREAK(); @@ -218,7 +221,7 @@ static void setFastFlags(const std::vector& flags) int main(int argc, char** argv) { - Luau::assertHandler() = assertionHandler; + Luau::assertHandler() = testAssertionHandler; doctest::registerReporter("boost", 0, true); diff --git a/tools/natvis/Analysis.natvis b/tools/natvis/Analysis.natvis new file mode 100644 index 00000000..5de0140e --- /dev/null +++ b/tools/natvis/Analysis.natvis @@ -0,0 +1,78 @@ + + + + + AnyTypeVar + + + + {{ index=0, value={*($T1*)storage} }} + {{ index=1, value={*($T2*)storage} }} + {{ index=2, value={*($T3*)storage} }} + {{ index=3, value={*($T4*)storage} }} + {{ index=4, value={*($T5*)storage} }} + {{ index=5, value={*($T6*)storage} }} + {{ index=6, value={*($T7*)storage} }} + {{ index=7, value={*($T8*)storage} }} + {{ index=8, value={*($T9*)storage} }} + {{ index=9, value={*($T10*)storage} }} + {{ index=10, value={*($T11*)storage} }} + {{ index=11, value={*($T12*)storage} }} + {{ index=12, value={*($T13*)storage} }} + {{ index=13, value={*($T14*)storage} }} + {{ index=14, value={*($T15*)storage} }} + {{ index=15, value={*($T16*)storage} }} + {{ index=16, value={*($T17*)storage} }} + {{ index=17, value={*($T18*)storage} }} + {{ index=18, value={*($T19*)storage} }} + {{ index=19, value={*($T20*)storage} }} + {{ index=20, value={*($T21*)storage} }} + {{ index=21, value={*($T22*)storage} }} + {{ index=22, value={*($T23*)storage} }} + {{ index=23, value={*($T24*)storage} }} + {{ index=24, value={*($T25*)storage} }} + {{ index=25, value={*($T26*)storage} }} + {{ index=26, value={*($T27*)storage} }} + {{ index=27, value={*($T28*)storage} }} + {{ index=28, value={*($T29*)storage} }} + {{ index=29, value={*($T30*)storage} }} + {{ index=30, value={*($T31*)storage} }} + {{ index=31, value={*($T32*)storage} }} + + typeId + *($T1*)storage + *($T2*)storage + *($T3*)storage + *($T4*)storage + *($T5*)storage + *($T6*)storage + *($T7*)storage + *($T8*)storage + *($T9*)storage + *($T10*)storage + *($T11*)storage + *($T12*)storage + *($T13*)storage + *($T14*)storage + *($T15*)storage + *($T16*)storage + *($T17*)storage + *($T18*)storage + *($T19*)storage + *($T20*)storage + *($T21*)storage + *($T22*)storage + *($T23*)storage + *($T24*)storage + *($T25*)storage + *($T26*)storage + *($T27*)storage + *($T28*)storage + *($T29*)storage + *($T30*)storage + *($T31*)storage + *($T32*)storage + + + + diff --git a/tools/natvis/Ast.natvis b/tools/natvis/Ast.natvis new file mode 100644 index 00000000..322eb8f6 --- /dev/null +++ b/tools/natvis/Ast.natvis @@ -0,0 +1,25 @@ + + + + + AstArray size={size} + + size + + size + data + + + + + + + size_ + + size_ + storage._Mypair._Myval2._Myfirst + offset + + + + + diff --git a/tools/natvis/VM.natvis b/tools/natvis/VM.natvis new file mode 100644 index 00000000..9924e194 --- /dev/null +++ b/tools/natvis/VM.natvis @@ -0,0 +1,269 @@ + + + + + nil + {(bool)value.b} + lightuserdata {value.p} + number = {value.n} + vector = {value.v[0]}, {value.v[1]}, {*(float*)&extra} + {value.gc->ts} + {value.gc->h} + function {value.gc->cl,view(short)} + userdata {value.gc->u} + thread {value.gc->th} + proto {value.gc->p} + upvalue {value.gc->uv} + deadkey + empty + + value.p + value.gc->ts + value.gc->h + value.gc->cl + value.gc->cl + value.gc->u + value.gc->th + value.gc->p + value.gc->uv + + fixed ({(int)value.gc->gch.marked}) + black ({(int)value.gc->gch.marked}) + white ({(int)value.gc->gch.marked}) + white ({(int)value.gc->gch.marked}) + gray ({(int)value.gc->gch.marked}) + + + + + + nil + {(bool)value.b} + lightuserdata {value.p} + number = {value.n} + vector = {value.v[0]}, {value.v[1]}, {*(float*)&extra} + {value.gc->ts} + {value.gc->h} + function {value.gc->cl,view(short)} + userdata {value.gc->u} + thread {value.gc->th} + proto {value.gc->p} + upvalue {value.gc->uv} + deadkey + empty + + (void**)value.p + value.gc->ts + value.gc->h + value.gc->cl + value.gc->cl + value.gc->u + value.gc->th + value.gc->p + value.gc->uv + + next + + + + + {key,na} = {val} + --- + + + + table + + metatable + + + [size] {1<<lsizenode} + + + 1<<lsizenode + node[$i] + + + + + [size] {sizearray} + + + sizearray + array[$i] + + + + + + + + + + + + + 1 + + + + + metatable->node[i].val + + + + i = i + 1 + + + "unknown",sb + + + tag + len + metatable + data + + + + + {c.f,na} + {l.p,na} + {c} + {l} + invalid + + + + {data,s} + + + + + {ci->func->value.gc->cl.c.f,na} + + + {ci->func->value.gc->cl.l.p->source->data,sb}:{ci->func->value.gc->cl.l.p->linedefined,d} {ci->func->value.gc->cl.l.p->debugname->data,sb} + + + {ci->func->value.gc->cl.l.p->source->data,sb}:{ci->func->value.gc->cl.l.p->linedefined,d} + + thread + + + {ci-base_ci} frames + + + ci-base_ci + + + base_ci[ci-base_ci - $i].func->value.gc->cl,view(short) + + + + + + {top-base} values + + + top-base + base + + + + + {top-stack} values + + + top-stack + stack + + + + + + + openupval + u.l.next + this + + + + l_gt + env + userdata + + + + + {source->data,sb}:{linedefined} function {debugname->data,sb} [{(int)numparams} arg, {(int)nups} upval] + {source->data,sb}:{linedefined} [{(int)numparams} arg, {(int)nups} upval] + + debugname + + constants + + + sizek + k[$i] + + + + + locals + + + sizelocvars + locvars[$i] + + + + + bytecode + + + sizecode + code[$i] + + + + + functions + + + sizep + p[$i] + + + + + upvals + + + sizeupvalues + upvalues[$i] + + + + + source + + + + + + + {(lua_Type)tt} + + + fixed ({(int)marked}) + black ({(int)marked}) + white ({(int)marked}) + white ({(int)marked}) + gray ({(int)marked}) + unknown + + memcat + + + + From a9bdce6cc06577cb412c38db757e44ea783f7c05 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Fri, 18 Feb 2022 10:04:38 -0800 Subject: [PATCH 171/399] Rename tests to tests.py (#374) --- .github/workflows/prototyping.yml | 2 +- prototyping/{tests => tests.py} | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) rename prototyping/{tests => tests.py} (99%) diff --git a/.github/workflows/prototyping.yml b/.github/workflows/prototyping.yml index 637ed840..fe458673 100644 --- a/.github/workflows/prototyping.yml +++ b/.github/workflows/prototyping.yml @@ -73,7 +73,7 @@ jobs: working-directory: prototyping run: | mkdir test-failures - python tests -l ../build/luau-ast --write-diff-failures --diff-failure-location test-failures/ + python tests.py -l ../build/luau-ast --write-diff-failures --diff-failure-location test-failures/ - uses: actions/upload-artifact@v2 if: failure() with: diff --git a/prototyping/tests b/prototyping/tests.py similarity index 99% rename from prototyping/tests rename to prototyping/tests.py index 89e105d0..20070ae0 100755 --- a/prototyping/tests +++ b/prototyping/tests.py @@ -1,4 +1,4 @@ -#!/bin/python +#!/usr/bin/python import argparse import difflib From 7f867ac166c39a3b291a5a66e70500335e3211a3 Mon Sep 17 00:00:00 2001 From: Lily Brown Date: Fri, 18 Feb 2022 11:09:00 -0800 Subject: [PATCH 172/399] Prototyping: numbers (#368) Adds number support to the prototype. Binary operators are next. --- prototyping/Examples/OpSem.agda | 1 - prototyping/Examples/Run.agda | 7 +++++-- prototyping/FFI/Data/Scientific.agda | 15 +++++++++++++++ prototyping/Interpreter.agda | 1 - prototyping/Luau/RuntimeError.agda | 5 +++-- prototyping/Luau/RuntimeError/ToString.agda | 8 ++++++-- prototyping/Luau/Substitution.agda | 4 ++-- prototyping/Luau/Syntax.agda | 3 ++- prototyping/Luau/Syntax/FromJSON.agda | 8 +++++++- prototyping/Luau/Syntax/ToString.agda | 4 +++- prototyping/Luau/Type.agda | 4 +++- prototyping/Luau/Type/FromJSON.agda | 3 ++- prototyping/Luau/Type/ToString.agda | 3 ++- prototyping/Luau/Value.agda | 7 ++++--- prototyping/Luau/Value/ToString.agda | 5 +++-- prototyping/PrettyPrinter.agda | 1 - prototyping/Properties/Step.agda | 8 +++++--- prototyping/Tests/PrettyPrinter/smoke_test/in.lua | 1 + .../Tests/PrettyPrinter/smoke_test/out.txt | 1 + 19 files changed, 64 insertions(+), 25 deletions(-) diff --git a/prototyping/Examples/OpSem.agda b/prototyping/Examples/OpSem.agda index ec8bce7b..c4cfc5e5 100644 --- a/prototyping/Examples/OpSem.agda +++ b/prototyping/Examples/OpSem.agda @@ -6,4 +6,3 @@ open import Luau.Heap using (∅) ex1 : ∅ ⊢ (local (var "x") ← nil ∙ return (var "x") ∙ done) ⟶ᴮ (return nil ∙ done) ⊣ ∅ ex1 = subst - diff --git a/prototyping/Examples/Run.agda b/prototyping/Examples/Run.agda index 88ca39b6..3cac564d 100644 --- a/prototyping/Examples/Run.agda +++ b/prototyping/Examples/Run.agda @@ -3,8 +3,8 @@ module Examples.Run where open import Agda.Builtin.Equality using (_≡_; refl) -open import Luau.Syntax using (nil; var; _$_; function_is_end; return; _∙_; done; _⟨_⟩) -open import Luau.Value using (nil) +open import Luau.Syntax using (nil; var; _$_; function_is_end; return; _∙_; done; _⟨_⟩; number) +open import Luau.Value using (nil; number) open import Luau.Run using (run; return) open import Luau.Heap using (lookup-next; next-emp; lookup-next-emp) @@ -13,3 +13,6 @@ import Agda.Builtin.Equality.Rewrite ex1 : (run (function "id" ⟨ var "x" ⟩ is return (var "x") ∙ done end ∙ return (var "id" $ nil) ∙ done) ≡ return nil _) ex1 = refl + +ex2 : (run (function "fn" ⟨ var "x" ⟩ is return (number 123.0) ∙ done end ∙ return (var "fn" $ nil) ∙ done) ≡ return (number 123.0) _) +ex2 = refl diff --git a/prototyping/FFI/Data/Scientific.agda b/prototyping/FFI/Data/Scientific.agda index 8a5be39e..772d3367 100644 --- a/prototyping/FFI/Data/Scientific.agda +++ b/prototyping/FFI/Data/Scientific.agda @@ -1,6 +1,21 @@ module FFI.Data.Scientific where +open import Agda.Builtin.Float using (Float) +open import FFI.Data.String using (String) +open import FFI.Data.HaskellString using (HaskellString; pack; unpack) + {-# FOREIGN GHC import qualified Data.Scientific #-} +{-# FOREIGN GHC import qualified Text.Show #-} postulate Scientific : Set {-# COMPILE GHC Scientific = type Data.Scientific.Scientific #-} + +postulate + showHaskell : Scientific → HaskellString + toFloat : Scientific → Float + +{-# COMPILE GHC showHaskell = \x -> Text.Show.show x #-} +{-# COMPILE GHC toFloat = \x -> Data.Scientific.toRealFloat x #-} + +show : Scientific → String +show x = pack (showHaskell x) diff --git a/prototyping/Interpreter.agda b/prototyping/Interpreter.agda index 3ec5b8d1..fe311e57 100644 --- a/prototyping/Interpreter.agda +++ b/prototyping/Interpreter.agda @@ -36,4 +36,3 @@ runString txt | (Right value) = runJSON value main : IO ⊤ main = getContents >>= runString - diff --git a/prototyping/Luau/RuntimeError.agda b/prototyping/Luau/RuntimeError.agda index e514dc9d..f9e684b3 100644 --- a/prototyping/Luau/RuntimeError.agda +++ b/prototyping/Luau/RuntimeError.agda @@ -3,13 +3,15 @@ module Luau.RuntimeError where open import Agda.Builtin.Equality using (_≡_) open import Luau.Heap using (Heap; _[_]) open import FFI.Data.Maybe using (just; nothing) -open import Luau.Syntax using (Block; Expr; nil; var; addr; block_is_end; _$_; local_←_; return; done; _∙_) +open import FFI.Data.String using (String) +open import Luau.Syntax using (Block; Expr; nil; var; addr; block_is_end; _$_; local_←_; return; done; _∙_; number) data RuntimeErrorᴮ {a} (H : Heap a) : Block a → Set data RuntimeErrorᴱ {a} (H : Heap a) : Expr a → Set data RuntimeErrorᴱ H where NilIsNotAFunction : ∀ {M} → RuntimeErrorᴱ H (nil $ M) + NumberIsNotAFunction : ∀ n {M} → RuntimeErrorᴱ H ((number n) $ M) UnboundVariable : ∀ x → RuntimeErrorᴱ H (var x) SEGV : ∀ a → (H [ a ] ≡ nothing) → RuntimeErrorᴱ H (addr a) app : ∀ {M N} → RuntimeErrorᴱ H M → RuntimeErrorᴱ H (M $ N) @@ -18,4 +20,3 @@ data RuntimeErrorᴱ H where data RuntimeErrorᴮ H where local : ∀ x {M B} → RuntimeErrorᴱ H M → RuntimeErrorᴮ H (local x ← M ∙ B) return : ∀ {M B} → RuntimeErrorᴱ H M → RuntimeErrorᴮ H (return M ∙ B) - diff --git a/prototyping/Luau/RuntimeError/ToString.agda b/prototyping/Luau/RuntimeError/ToString.agda index e8287157..ac760fab 100644 --- a/prototyping/Luau/RuntimeError/ToString.agda +++ b/prototyping/Luau/RuntimeError/ToString.agda @@ -1,15 +1,18 @@ module Luau.RuntimeError.ToString where +open import Agda.Builtin.Float using (primShowFloat) open import FFI.Data.String using (String; _++_) -open import Luau.RuntimeError using (RuntimeErrorᴮ; RuntimeErrorᴱ; local; return; NilIsNotAFunction; UnboundVariable; SEGV; app; block) +open import Luau.RuntimeError using (RuntimeErrorᴮ; RuntimeErrorᴱ; local; return; NilIsNotAFunction; NumberIsNotAFunction; UnboundVariable; SEGV; app; block) open import Luau.Addr.ToString using (addrToString) +open import Luau.Syntax.ToString using (exprToString) open import Luau.Var.ToString using (varToString) -open import Luau.Syntax using (name) +open import Luau.Syntax using (name; _$_) errToStringᴱ : ∀ {a H B} → RuntimeErrorᴱ {a} H B → String errToStringᴮ : ∀ {a H B} → RuntimeErrorᴮ {a} H B → String errToStringᴱ NilIsNotAFunction = "nil is not a function" +errToStringᴱ (NumberIsNotAFunction n) = "number " ++ primShowFloat n ++ " is not a function" errToStringᴱ (UnboundVariable x) = "variable " ++ varToString x ++ " is unbound" errToStringᴱ (SEGV a x) = "address " ++ addrToString a ++ " is unallocated" errToStringᴱ (app E) = errToStringᴱ E @@ -17,3 +20,4 @@ errToStringᴱ (block b E) = errToStringᴮ E ++ "\n in call of function " ++ v errToStringᴮ (local x E) = errToStringᴱ E ++ "\n in definition of " ++ varToString (name x) errToStringᴮ (return E) = errToStringᴱ E ++ "\n in return statement" + \ No newline at end of file diff --git a/prototyping/Luau/Substitution.agda b/prototyping/Luau/Substitution.agda index b956aeae..e364a000 100644 --- a/prototyping/Luau/Substitution.agda +++ b/prototyping/Luau/Substitution.agda @@ -1,6 +1,6 @@ module Luau.Substitution where -open import Luau.Syntax using (Expr; Stat; Block; nil; addr; var; function_is_end; _$_; block_is_end; local_←_; _∙_; done; return; _⟨_⟩ ; name; fun; arg) +open import Luau.Syntax using (Expr; Stat; Block; nil; addr; var; function_is_end; _$_; block_is_end; local_←_; _∙_; done; return; _⟨_⟩ ; name; fun; arg; number) open import Luau.Value using (Value; val) open import Luau.Var using (Var; _≡ⱽ_) open import Properties.Dec using (Dec; yes; no) @@ -13,6 +13,7 @@ _[_/_]ᴮunless_ : ∀ {a P} → Block a → Value → Var → (Dec P) → Block nil [ v / x ]ᴱ = nil var y [ v / x ]ᴱ = var y [ v / x ]ᴱwhenever (x ≡ⱽ y) addr a [ v / x ]ᴱ = addr a +(number y) [ v / x ]ᴱ = number y (M $ N) [ v / x ]ᴱ = (M [ v / x ]ᴱ) $ (N [ v / x ]ᴱ) function F is C end [ v / x ]ᴱ = function F is C [ v / x ]ᴮunless (x ≡ⱽ name(arg F)) end block b is C end [ v / x ]ᴱ = block b is C [ v / x ]ᴮ end @@ -27,4 +28,3 @@ var y [ v / x ]ᴱwhenever no p = var y B [ v / x ]ᴮunless yes p = B B [ v / x ]ᴮunless no p = B [ v / x ]ᴮ - diff --git a/prototyping/Luau/Syntax.agda b/prototyping/Luau/Syntax.agda index 04970b62..1313456b 100644 --- a/prototyping/Luau/Syntax.agda +++ b/prototyping/Luau/Syntax.agda @@ -1,6 +1,7 @@ module Luau.Syntax where open import Agda.Builtin.Equality using (_≡_) +open import Agda.Builtin.Float using (Float) open import Properties.Dec using (⊥) open import Luau.Var using (Var) open import Luau.Addr using (Addr) @@ -52,4 +53,4 @@ data Expr a where _$_ : Expr a → Expr a → Expr a function_is_end : FunDec a → Block a → Expr a block_is_end : Var → Block a → Expr a - + number : Float → Expr a diff --git a/prototyping/Luau/Syntax/FromJSON.agda b/prototyping/Luau/Syntax/FromJSON.agda index 8ae620bb..8191e9e7 100644 --- a/prototyping/Luau/Syntax/FromJSON.agda +++ b/prototyping/Luau/Syntax/FromJSON.agda @@ -1,6 +1,6 @@ module Luau.Syntax.FromJSON where -open import Luau.Syntax using (Block; Stat ; Expr; nil; _$_; var; var_∈_; function_is_end; _⟨_⟩; local_←_; return; done; _∙_; maybe; VarDec) +open import Luau.Syntax using (Block; Stat ; Expr; nil; _$_; var; var_∈_; function_is_end; _⟨_⟩; local_←_; return; done; _∙_; maybe; VarDec; number) open import Luau.Type.FromJSON using (typeFromJSON) open import Agda.Builtin.List using (List; _∷_; []) @@ -9,6 +9,7 @@ open import FFI.Data.Aeson using (Value; Array; Object; object; array; string; f open import FFI.Data.Bool using (true; false) open import FFI.Data.Either using (Either; Left; Right) open import FFI.Data.Maybe using (Maybe; nothing; just) +open import FFI.Data.Scientific using (toFloat) open import FFI.Data.String using (String; _++_) open import FFI.Data.Vector using (head; tail; null; empty) @@ -19,6 +20,7 @@ lokal = fromString "local" list = fromString "list" name = fromString "name" type = fromString "type" +value = fromString "value" values = fromString "values" vars = fromString "vars" @@ -83,6 +85,10 @@ exprFromObject obj | just (string "AstExprLocal") | just x with varDecFromJSON x exprFromObject obj | just (string "AstExprLocal") | just x | Right x′ = Right (var (Luau.Syntax.name x′)) exprFromObject obj | just (string "AstExprLocal") | just x | Left err = Left err exprFromObject obj | just (string "AstExprLocal") | nothing = Left "AstExprLocal missing local" +exprFromObject obj | just (string "AstExprConstantNumber") with lookup value obj +exprFromObject obj | just (string "AstExprConstantNumber") | just (FFI.Data.Aeson.Value.number x) = Right (number (toFloat x)) +exprFromObject obj | just (string "AstExprConstantNumber") | just _ = Left "AstExprConstantNumber value is not a number" +exprFromObject obj | just (string "AstExprConstantNumber") | nothing = Left "AstExprConstantNumber missing value" exprFromObject obj | just (string ty) = Left ("TODO: Unsupported AstExpr " ++ ty) exprFromObject obj | just _ = Left "AstExpr type not a string" exprFromObject obj | nothing = Left "AstExpr missing type" diff --git a/prototyping/Luau/Syntax/ToString.agda b/prototyping/Luau/Syntax/ToString.agda index 18d36175..a20b360b 100644 --- a/prototyping/Luau/Syntax/ToString.agda +++ b/prototyping/Luau/Syntax/ToString.agda @@ -1,6 +1,7 @@ module Luau.Syntax.ToString where -open import Luau.Syntax using (Block; Stat; Expr; VarDec; FunDec; nil; var; var_∈_; addr; _$_; function_is_end; return; local_←_; _∙_; done; block_is_end; _⟨_⟩; _⟨_⟩∈_) +open import Agda.Builtin.Float using (primShowFloat) +open import Luau.Syntax using (Block; Stat; Expr; VarDec; FunDec; nil; var; var_∈_; addr; _$_; function_is_end; return; local_←_; _∙_; done; block_is_end; _⟨_⟩; _⟨_⟩∈_; number) open import FFI.Data.String using (String; _++_) open import Luau.Addr.ToString using (addrToString) open import Luau.Type.ToString using (typeToString) @@ -36,6 +37,7 @@ exprToString′ lb (block b is B end) = "(" ++ b ++ "()" ++ lb ++ " " ++ (blockToString′ (lb ++ " ") B) ++ lb ++ "end)()" +exprToString′ lb (number x) = primShowFloat x statToString′ lb (function F is B end) = "local " ++ funDecToString F ++ lb ++ diff --git a/prototyping/Luau/Type.agda b/prototyping/Luau/Type.agda index 6e384c3b..af5f857c 100644 --- a/prototyping/Luau/Type.agda +++ b/prototyping/Luau/Type.agda @@ -7,6 +7,7 @@ data Type : Set where _⇒_ : Type → Type → Type none : Type any : Type + number : Type _∪_ : Type → Type → Type _∩_ : Type → Type → Type @@ -15,6 +16,7 @@ src nil = none src (S ⇒ T) = S src none = none src any = any +src number = none src (S ∪ T) = (src S) ∪ (src T) src (S ∩ T) = (src S) ∩ (src T) @@ -23,6 +25,7 @@ tgt nil = none tgt (S ⇒ T) = T tgt none = none tgt any = any +tgt number = none tgt (S ∪ T) = (tgt S) ∪ (tgt T) tgt (S ∩ T) = (tgt S) ∩ (tgt T) @@ -40,4 +43,3 @@ normalizeOptional (S ∪ T) | S′ | nil = optional S′ normalizeOptional (S ∪ T) | nil | T′ = optional T′ normalizeOptional (S ∪ T) | S′ | T′ = S′ ∪ T′ normalizeOptional T = T - diff --git a/prototyping/Luau/Type/FromJSON.agda b/prototyping/Luau/Type/FromJSON.agda index 45bda5f3..c585c3f3 100644 --- a/prototyping/Luau/Type/FromJSON.agda +++ b/prototyping/Luau/Type/FromJSON.agda @@ -1,6 +1,6 @@ module Luau.Type.FromJSON where -open import Luau.Type using (Type; nil; _⇒_; _∪_; _∩_; any) +open import Luau.Type using (Type; nil; _⇒_; _∪_; _∩_; any; number) open import Agda.Builtin.List using (List; _∷_; []) @@ -41,6 +41,7 @@ typeFromJSON (object o) | just (string "AstTypeFunction") | nothing | nothing = typeFromJSON (object o) | just (string "AstTypeReference") with lookup name o typeFromJSON (object o) | just (string "AstTypeReference") | just (string "nil") = Right nil typeFromJSON (object o) | just (string "AstTypeReference") | just (string "any") = Right any +typeFromJSON (object o) | just (string "AstTypeReference") | just (string "number") = Right number typeFromJSON (object o) | just (string "AstTypeReference") | _ = Left "Unknown referenced type" typeFromJSON (object o) | just (string "AstTypeUnion") with lookup types o diff --git a/prototyping/Luau/Type/ToString.agda b/prototyping/Luau/Type/ToString.agda index 698d6e8e..e7a7c1c0 100644 --- a/prototyping/Luau/Type/ToString.agda +++ b/prototyping/Luau/Type/ToString.agda @@ -1,7 +1,7 @@ module Luau.Type.ToString where open import FFI.Data.String using (String; _++_) -open import Luau.Type using (Type; nil; _⇒_; none; any; _∪_; _∩_; normalizeOptional) +open import Luau.Type using (Type; nil; _⇒_; none; any; number; _∪_; _∩_; normalizeOptional) {-# TERMINATING #-} typeToString : Type → String @@ -12,6 +12,7 @@ typeToString nil = "nil" typeToString (S ⇒ T) = "(" ++ (typeToString S) ++ ") -> " ++ (typeToString T) typeToString none = "none" typeToString any = "any" +typeToString number = "number" typeToString (S ∪ T) with normalizeOptional(S ∪ T) typeToString (S ∪ T) | ((S′ ⇒ T′) ∪ nil) = "(" ++ typeToString (S′ ⇒ T′) ++ ")?" typeToString (S ∪ T) | (S′ ∪ nil) = typeToString S′ ++ "?" diff --git a/prototyping/Luau/Value.agda b/prototyping/Luau/Value.agda index 4768f859..1086d39c 100644 --- a/prototyping/Luau/Value.agda +++ b/prototyping/Luau/Value.agda @@ -1,15 +1,16 @@ module Luau.Value where +open import Agda.Builtin.Float using (Float) open import Luau.Addr using (Addr) -open import Luau.Syntax using (Block; Expr; nil; addr) +open import Luau.Syntax using (Block; Expr; nil; addr; number) open import Luau.Var using (Var) data Value : Set where nil : Value addr : Addr → Value + number : Float → Value val : ∀ {a} → Value → Expr a val nil = nil val (addr a) = addr a - - +val (number x) = number x diff --git a/prototyping/Luau/Value/ToString.agda b/prototyping/Luau/Value/ToString.agda index 3cac3ee7..51c3fa78 100644 --- a/prototyping/Luau/Value/ToString.agda +++ b/prototyping/Luau/Value/ToString.agda @@ -1,10 +1,11 @@ module Luau.Value.ToString where open import Agda.Builtin.String using (String) -open import Luau.Value using (Value; nil; addr) +open import Agda.Builtin.Float using (primShowFloat) +open import Luau.Value using (Value; nil; addr; number) open import Luau.Addr.ToString using (addrToString) valueToString : Value → String valueToString nil = "nil" valueToString (addr a) = addrToString a - +valueToString (number x) = primShowFloat x diff --git a/prototyping/PrettyPrinter.agda b/prototyping/PrettyPrinter.agda index 4a6c7dd6..fdad32d6 100644 --- a/prototyping/PrettyPrinter.agda +++ b/prototyping/PrettyPrinter.agda @@ -30,4 +30,3 @@ runString txt | (Right value) = runJSON value main : IO ⊤ main = getContents >>= runString - diff --git a/prototyping/Properties/Step.agda b/prototyping/Properties/Step.agda index eeda4ef2..620dc618 100644 --- a/prototyping/Properties/Step.agda +++ b/prototyping/Properties/Step.agda @@ -3,11 +3,11 @@ module Properties.Step where open import Agda.Builtin.Equality using (_≡_; refl) open import FFI.Data.Maybe using (just; nothing) open import Luau.Heap using (Heap; _[_]; alloc; ok; function_is_end) -open import Luau.Syntax using (Block; Expr; nil; var; addr; function_is_end; block_is_end; _$_; local_←_; return; done; _∙_; name; fun; arg) +open import Luau.Syntax using (Block; Expr; nil; var; addr; function_is_end; block_is_end; _$_; local_←_; return; done; _∙_; name; fun; arg; number) open import Luau.OpSem using (_⊢_⟶ᴱ_⊣_; _⊢_⟶ᴮ_⊣_; app ; beta; function; block; return; done; local; subst) -open import Luau.RuntimeError using (RuntimeErrorᴱ; RuntimeErrorᴮ; NilIsNotAFunction; UnboundVariable; SEGV; app; block; local; return) +open import Luau.RuntimeError using (RuntimeErrorᴱ; RuntimeErrorᴮ; NilIsNotAFunction; NumberIsNotAFunction; UnboundVariable; SEGV; app; block; local; return) open import Luau.Substitution using (_[_/_]ᴮ) -open import Luau.Value using (nil; addr; val) +open import Luau.Value using (nil; addr; val; number) open import Properties.Remember using (remember; _,_) data StepResultᴮ {a} (H : Heap a) (B : Block a) : Set @@ -30,9 +30,11 @@ stepᴮ : ∀ {a} H B → StepResultᴮ {a} H B stepᴱ H nil = value nil refl stepᴱ H (var x) = error (UnboundVariable x) stepᴱ H (addr a) = value (addr a) refl +stepᴱ H (number x) = value (number x) refl stepᴱ H (M $ N) with stepᴱ H M stepᴱ H (M $ N) | step H′ M′ D = step H′ (M′ $ N) (app D) stepᴱ H (nil $ N) | value nil refl = error NilIsNotAFunction +stepᴱ H ((number _) $ N) | value (number x) refl = error (NumberIsNotAFunction x) stepᴱ H (addr a $ N) | value (addr a) refl with remember (H [ a ]) stepᴱ H (addr a $ N) | value (addr a) refl | (nothing , p) = error (app (SEGV a p)) stepᴱ H (addr a $ N) | value (addr a) refl | (just(function F is B end) , p) = step H (block fun F is (local arg F ← N) ∙ B end) (beta p) diff --git a/prototyping/Tests/PrettyPrinter/smoke_test/in.lua b/prototyping/Tests/PrettyPrinter/smoke_test/in.lua index 0d23bbb2..d26e3a0d 100644 --- a/prototyping/Tests/PrettyPrinter/smoke_test/in.lua +++ b/prototyping/Tests/PrettyPrinter/smoke_test/in.lua @@ -15,4 +15,5 @@ local b : nil = nil local c : (nil) -> nil = nil local d : (any & nil) = nil local e : any? = nil +local f : number = 123 return id2(nil2) diff --git a/prototyping/Tests/PrettyPrinter/smoke_test/out.txt b/prototyping/Tests/PrettyPrinter/smoke_test/out.txt index e663534a..34e0c4fe 100644 --- a/prototyping/Tests/PrettyPrinter/smoke_test/out.txt +++ b/prototyping/Tests/PrettyPrinter/smoke_test/out.txt @@ -15,4 +15,5 @@ local b : nil = nil local c : (nil) -> nil = nil local d : (any & nil) = nil local e : any? = nil +local f : number = 123.0 return id2(nil2) From 0b783d89327eb5de1d20223559aa6ac9111075b7 Mon Sep 17 00:00:00 2001 From: Alan Jeffrey <403333+asajeffrey@users.noreply.github.com> Date: Fri, 18 Feb 2022 16:47:04 -0600 Subject: [PATCH 173/399] Add Properties.Equality to prototyping (#376) --- prototyping/Properties.agda | 2 ++ prototyping/Properties/Contradiction.agda | 9 +++++++++ prototyping/Properties/Equality.agda | 23 +++++++++++++++++++++++ 3 files changed, 34 insertions(+) create mode 100644 prototyping/Properties/Contradiction.agda create mode 100644 prototyping/Properties/Equality.agda diff --git a/prototyping/Properties.agda b/prototyping/Properties.agda index b08a3a81..1a6f92fa 100644 --- a/prototyping/Properties.agda +++ b/prototyping/Properties.agda @@ -1,5 +1,7 @@ module Properties where +import Properties.Contradiction import Properties.Dec +import Properties.Equality import Properties.Step import Properties.Remember diff --git a/prototyping/Properties/Contradiction.agda b/prototyping/Properties/Contradiction.agda new file mode 100644 index 00000000..e9a92ad5 --- /dev/null +++ b/prototyping/Properties/Contradiction.agda @@ -0,0 +1,9 @@ +module Properties.Contradiction where + +data ⊥ : Set where + +¬ : Set → Set +¬ A = A → ⊥ + +CONTRADICTION : ∀ {A : Set} → ⊥ → A +CONTRADICTION () diff --git a/prototyping/Properties/Equality.agda b/prototyping/Properties/Equality.agda new file mode 100644 index 00000000..c027bee3 --- /dev/null +++ b/prototyping/Properties/Equality.agda @@ -0,0 +1,23 @@ +module Properties.Equality where + +open import Agda.Builtin.Equality using (_≡_; refl) +open import Properties.Contradiction using (¬) + +sym : ∀ {A : Set} {a b : A} → (a ≡ b) → (b ≡ a) +sym refl = refl + +trans : ∀ {A : Set} {a b c : A} → (a ≡ b) → (b ≡ c) → (a ≡ c) +trans refl refl = refl + +cong : ∀ {A B : Set} {a b : A} (f : A → B) → (a ≡ b) → (f a ≡ f b) +cong f refl = refl + +subst₁ : ∀ {A : Set} {a b : A} (F : A → Set) → (a ≡ b) → (F a) → (F b) +subst₁ F refl x = x + +subst₂ : ∀ {A B : Set} {a b : A} {c d : B} (F : A → B → Set) → (a ≡ b) → (c ≡ d) → (F a c) → (F b d) +subst₂ F refl refl x = x + +_≢_ : ∀ {A : Set} → A → A → Set +(a ≢ b) = ¬(a ≡ b) + From fc33b0c7026eea4f51af73fcf811ea5b3f3b497b Mon Sep 17 00:00:00 2001 From: Alan Jeffrey <403333+asajeffrey@users.noreply.github.com> Date: Fri, 18 Feb 2022 16:47:23 -0600 Subject: [PATCH 174/399] Fix evaluation rule for function application (#375) --- prototyping/Luau/OpSem.agda | 14 ++++++++++---- prototyping/Luau/RuntimeError.agda | 8 +++++--- prototyping/Luau/RuntimeError/ToString.agda | 7 ++++--- prototyping/Properties/Step.agda | 21 ++++++++++++--------- 4 files changed, 31 insertions(+), 19 deletions(-) diff --git a/prototyping/Luau/OpSem.agda b/prototyping/Luau/OpSem.agda index 878b8bd0..89564769 100644 --- a/prototyping/Luau/OpSem.agda +++ b/prototyping/Luau/OpSem.agda @@ -23,17 +23,23 @@ data _⊢_⟶ᴱ_⊣_ where ------------------------------------------- H ⊢ (function F is B end) ⟶ᴱ (addr a) ⊣ H′ - app : ∀ {H H′ M M′ N} → + app₁ : ∀ {H H′ M M′ N} → H ⊢ M ⟶ᴱ M′ ⊣ H′ → ----------------------------- H ⊢ (M $ N) ⟶ᴱ (M′ $ N) ⊣ H′ - beta : ∀ {H M a F B} → + app₂ : ∀ {H H′ V N N′} → + + H ⊢ N ⟶ᴱ N′ ⊣ H′ → + ----------------------------- + H ⊢ (val V $ N) ⟶ᴱ (val V $ N′) ⊣ H′ + + beta : ∀ {H a F B V} → H [ a ] ≡ just(function F is B end) → - ----------------------------------------------------- - H ⊢ (addr a $ M) ⟶ᴱ (block (fun F) is local (arg F) ← M ∙ B end) ⊣ H + ----------------------------------------------------------------------------- + H ⊢ (addr a $ val V) ⟶ᴱ (block (fun F) is (B [ V / name(arg F) ]ᴮ) end) ⊣ H block : ∀ {H H′ B B′ b} → diff --git a/prototyping/Luau/RuntimeError.agda b/prototyping/Luau/RuntimeError.agda index f9e684b3..a54d48fe 100644 --- a/prototyping/Luau/RuntimeError.agda +++ b/prototyping/Luau/RuntimeError.agda @@ -5,16 +5,18 @@ open import Luau.Heap using (Heap; _[_]) open import FFI.Data.Maybe using (just; nothing) open import FFI.Data.String using (String) open import Luau.Syntax using (Block; Expr; nil; var; addr; block_is_end; _$_; local_←_; return; done; _∙_; number) +open import Luau.Value using (val) data RuntimeErrorᴮ {a} (H : Heap a) : Block a → Set data RuntimeErrorᴱ {a} (H : Heap a) : Expr a → Set data RuntimeErrorᴱ H where - NilIsNotAFunction : ∀ {M} → RuntimeErrorᴱ H (nil $ M) - NumberIsNotAFunction : ∀ n {M} → RuntimeErrorᴱ H ((number n) $ M) + NilIsNotAFunction : ∀ {V} → RuntimeErrorᴱ H (nil $ val V) + NumberIsNotAFunction : ∀ n {V} → RuntimeErrorᴱ H (number n $ val V) UnboundVariable : ∀ x → RuntimeErrorᴱ H (var x) SEGV : ∀ a → (H [ a ] ≡ nothing) → RuntimeErrorᴱ H (addr a) - app : ∀ {M N} → RuntimeErrorᴱ H M → RuntimeErrorᴱ H (M $ N) + app₁ : ∀ {M N} → RuntimeErrorᴱ H M → RuntimeErrorᴱ H (M $ N) + app₂ : ∀ {M N} → RuntimeErrorᴱ H N → RuntimeErrorᴱ H (M $ N) block : ∀ b {B} → RuntimeErrorᴮ H B → RuntimeErrorᴱ H (block b is B end) data RuntimeErrorᴮ H where diff --git a/prototyping/Luau/RuntimeError/ToString.agda b/prototyping/Luau/RuntimeError/ToString.agda index ac760fab..10c0fe08 100644 --- a/prototyping/Luau/RuntimeError/ToString.agda +++ b/prototyping/Luau/RuntimeError/ToString.agda @@ -2,7 +2,7 @@ module Luau.RuntimeError.ToString where open import Agda.Builtin.Float using (primShowFloat) open import FFI.Data.String using (String; _++_) -open import Luau.RuntimeError using (RuntimeErrorᴮ; RuntimeErrorᴱ; local; return; NilIsNotAFunction; NumberIsNotAFunction; UnboundVariable; SEGV; app; block) +open import Luau.RuntimeError using (RuntimeErrorᴮ; RuntimeErrorᴱ; local; return; NilIsNotAFunction; NumberIsNotAFunction; UnboundVariable; SEGV; app₁; app₂; block) open import Luau.Addr.ToString using (addrToString) open import Luau.Syntax.ToString using (exprToString) open import Luau.Var.ToString using (varToString) @@ -15,9 +15,10 @@ errToStringᴱ NilIsNotAFunction = "nil is not a function" errToStringᴱ (NumberIsNotAFunction n) = "number " ++ primShowFloat n ++ " is not a function" errToStringᴱ (UnboundVariable x) = "variable " ++ varToString x ++ " is unbound" errToStringᴱ (SEGV a x) = "address " ++ addrToString a ++ " is unallocated" -errToStringᴱ (app E) = errToStringᴱ E +errToStringᴱ (app₁ E) = errToStringᴱ E +errToStringᴱ (app₂ E) = errToStringᴱ E errToStringᴱ (block b E) = errToStringᴮ E ++ "\n in call of function " ++ varToString b errToStringᴮ (local x E) = errToStringᴱ E ++ "\n in definition of " ++ varToString (name x) errToStringᴮ (return E) = errToStringᴱ E ++ "\n in return statement" - \ No newline at end of file + diff --git a/prototyping/Properties/Step.agda b/prototyping/Properties/Step.agda index 620dc618..914c3468 100644 --- a/prototyping/Properties/Step.agda +++ b/prototyping/Properties/Step.agda @@ -4,8 +4,8 @@ open import Agda.Builtin.Equality using (_≡_; refl) open import FFI.Data.Maybe using (just; nothing) open import Luau.Heap using (Heap; _[_]; alloc; ok; function_is_end) open import Luau.Syntax using (Block; Expr; nil; var; addr; function_is_end; block_is_end; _$_; local_←_; return; done; _∙_; name; fun; arg; number) -open import Luau.OpSem using (_⊢_⟶ᴱ_⊣_; _⊢_⟶ᴮ_⊣_; app ; beta; function; block; return; done; local; subst) -open import Luau.RuntimeError using (RuntimeErrorᴱ; RuntimeErrorᴮ; NilIsNotAFunction; NumberIsNotAFunction; UnboundVariable; SEGV; app; block; local; return) +open import Luau.OpSem using (_⊢_⟶ᴱ_⊣_; _⊢_⟶ᴮ_⊣_; app₁ ; app₂ ; beta; function; block; return; done; local; subst) +open import Luau.RuntimeError using (RuntimeErrorᴱ; RuntimeErrorᴮ; NilIsNotAFunction; NumberIsNotAFunction; UnboundVariable; SEGV; app₁; app₂; block; local; return) open import Luau.Substitution using (_[_/_]ᴮ) open import Luau.Value using (nil; addr; val; number) open import Properties.Remember using (remember; _,_) @@ -32,13 +32,16 @@ stepᴱ H (var x) = error (UnboundVariable x) stepᴱ H (addr a) = value (addr a) refl stepᴱ H (number x) = value (number x) refl stepᴱ H (M $ N) with stepᴱ H M -stepᴱ H (M $ N) | step H′ M′ D = step H′ (M′ $ N) (app D) -stepᴱ H (nil $ N) | value nil refl = error NilIsNotAFunction -stepᴱ H ((number _) $ N) | value (number x) refl = error (NumberIsNotAFunction x) -stepᴱ H (addr a $ N) | value (addr a) refl with remember (H [ a ]) -stepᴱ H (addr a $ N) | value (addr a) refl | (nothing , p) = error (app (SEGV a p)) -stepᴱ H (addr a $ N) | value (addr a) refl | (just(function F is B end) , p) = step H (block fun F is (local arg F ← N) ∙ B end) (beta p) -stepᴱ H (M $ N) | error E = error (app E) +stepᴱ H (M $ N) | step H′ M′ D = step H′ (M′ $ N) (app₁ D) +stepᴱ H (_ $ N) | value V refl with stepᴱ H N +stepᴱ H (_ $ N) | value V refl | step H′ N′ s = step H′ (val V $ N′) (app₂ s) +stepᴱ H (_ $ _) | value nil refl | value W refl = error NilIsNotAFunction +stepᴱ H (_ $ _) | value (number n) refl | value W refl = error (NumberIsNotAFunction n) +stepᴱ H (_ $ _) | value (addr a) refl | value W refl with remember (H [ a ]) +stepᴱ H (_ $ _) | value (addr a) refl | value W refl | (nothing , p) = error (app₁ (SEGV a p)) +stepᴱ H (_ $ _) | value (addr a) refl | value W refl | (just(function F is B end) , p) = step H (block fun F is B [ W / name (arg F) ]ᴮ end) (beta p) +stepᴱ H (M $ N) | value V p | error E = error (app₂ E) +stepᴱ H (M $ N) | error E = error (app₁ E) stepᴱ H (block b is B end) with stepᴮ H B stepᴱ H (block b is B end) | step H′ B′ D = step H′ (block b is B′ end) (block D) stepᴱ H (block b is (return _ ∙ B′) end) | return V refl = step H (val V) return From 3c42b3a01349b78c9c1abc1e15c67b390754f2e2 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Sat, 19 Feb 2022 10:57:29 -0800 Subject: [PATCH 175/399] Revert "Mark singleton types RFC as implemented (#370)" (#378) This reverts commit 731e197757ead8908e558cfc63f9d518f53940d5. --- rfcs/STATUS.md | 6 ++++++ rfcs/syntax-singleton-types.md | 2 -- 2 files changed, 6 insertions(+), 2 deletions(-) diff --git a/rfcs/STATUS.md b/rfcs/STATUS.md index 742cb501..72db2003 100644 --- a/rfcs/STATUS.md +++ b/rfcs/STATUS.md @@ -23,6 +23,12 @@ This document tracks unimplemented RFCs. **Status**: Implemented but depends on new transaction log implementation that is not fully live yet. +## Singleton types + +[RFC: Singleton types](https://github.com/Roblox/luau/blob/master/rfcs/syntax-singleton-types.md) + +**Status**: Implemented but not fully rolled out yet. + ## Nil-forgiving operator [RFC: nil-forgiving postfix operator !](https://github.com/Roblox/luau/blob/master/rfcs/syntax-nil-forgiving-operator.md) diff --git a/rfcs/syntax-singleton-types.md b/rfcs/syntax-singleton-types.md index 2c1f5442..26ea3028 100644 --- a/rfcs/syntax-singleton-types.md +++ b/rfcs/syntax-singleton-types.md @@ -2,8 +2,6 @@ > Note: this RFC was adapted from an internal proposal that predates RFC process -**Status**: Implemented - ## Summary Introduce a new kind of type variable, called singleton types. They are just like normal types but has the capability to represent a constant runtime value as a type. From 1334db600f9b1923b627f58a47cab76c7c370aff Mon Sep 17 00:00:00 2001 From: James Napora <85808999+TheGreatSageEqualToHeaven@users.noreply.github.com> Date: Tue, 22 Feb 2022 20:24:15 +0100 Subject: [PATCH 176/399] Update grammar.md (#379) --- docs/_pages/grammar.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/_pages/grammar.md b/docs/_pages/grammar.md index 585bac73..ffb1dbf6 100644 --- a/docs/_pages/grammar.md +++ b/docs/_pages/grammar.md @@ -68,8 +68,8 @@ SingletonType = STRING | 'true' | 'false' Type = SimpleType ['?'] | - SimpleType ['|' Type] | - SimpleType ['&' Type] + Type ['|' Type] | + Type ['&' Type] GenericTypePackParameter = NAME '...' ['=' (TypePack | VariadicTypePack | GenericTypePack)] GenericTypeParameterList = NAME ['=' Type] [',' GenericTypeParameterList] | GenericTypePackParameter {',' GenericTypePackParameter} From cd18adc20ecb805b8eeb770a9e5ef8e0cd123734 Mon Sep 17 00:00:00 2001 From: Lily Brown Date: Tue, 22 Feb 2022 15:52:56 -0800 Subject: [PATCH 177/399] Prototyping: binary operations (#377) Adds support for binary operations on numbers. --- prototyping/Examples/Run.agda | 5 ++- prototyping/Luau/OpSem.agda | 31 ++++++++++++++++--- prototyping/Luau/RuntimeError.agda | 9 ++++-- prototyping/Luau/RuntimeError/ToString.agda | 9 ++++-- prototyping/Luau/RuntimeType.agda | 13 ++++++++ prototyping/Luau/RuntimeType/ToString.agda | 9 ++++++ prototyping/Luau/Substitution.agda | 3 +- prototyping/Luau/Syntax.agda | 7 +++++ prototyping/Luau/Syntax/FromJSON.agda | 26 +++++++++++++++- prototyping/Luau/Syntax/ToString.agda | 9 +++++- prototyping/Properties/Step.agda | 23 +++++++++++--- .../Tests/Interpreter/binary_exps/in.lua | 1 + .../Tests/Interpreter/binary_exps/out.txt | 1 + 13 files changed, 127 insertions(+), 19 deletions(-) create mode 100644 prototyping/Luau/RuntimeType.agda create mode 100644 prototyping/Luau/RuntimeType/ToString.agda create mode 100644 prototyping/Tests/Interpreter/binary_exps/in.lua create mode 100644 prototyping/Tests/Interpreter/binary_exps/out.txt diff --git a/prototyping/Examples/Run.agda b/prototyping/Examples/Run.agda index 3cac564d..534bb122 100644 --- a/prototyping/Examples/Run.agda +++ b/prototyping/Examples/Run.agda @@ -3,7 +3,7 @@ module Examples.Run where open import Agda.Builtin.Equality using (_≡_; refl) -open import Luau.Syntax using (nil; var; _$_; function_is_end; return; _∙_; done; _⟨_⟩; number) +open import Luau.Syntax using (nil; var; _$_; function_is_end; return; _∙_; done; _⟨_⟩; number; binexp; +) open import Luau.Value using (nil; number) open import Luau.Run using (run; return) open import Luau.Heap using (lookup-next; next-emp; lookup-next-emp) @@ -16,3 +16,6 @@ ex1 = refl ex2 : (run (function "fn" ⟨ var "x" ⟩ is return (number 123.0) ∙ done end ∙ return (var "fn" $ nil) ∙ done) ≡ return (number 123.0) _) ex2 = refl + +ex3 : (run (function "fn" ⟨ var "x" ⟩ is return (binexp (number 1.0) + (number 2.0)) ∙ done end ∙ return (var "fn" $ nil) ∙ done) ≡ return (number 3.0) _) +ex3 = refl diff --git a/prototyping/Luau/OpSem.agda b/prototyping/Luau/OpSem.agda index 89564769..bfc63a4a 100644 --- a/prototyping/Luau/OpSem.agda +++ b/prototyping/Luau/OpSem.agda @@ -1,11 +1,18 @@ module Luau.OpSem where open import Agda.Builtin.Equality using (_≡_) +open import Agda.Builtin.Float using (Float; primFloatPlus; primFloatMinus; primFloatTimes; primFloatDiv) open import FFI.Data.Maybe using (just) open import Luau.Heap using (Heap; _≡_⊕_↦_; _[_]; function_is_end) open import Luau.Substitution using (_[_/_]ᴮ) -open import Luau.Syntax using (Expr; Stat; Block; nil; addr; var; function_is_end; _$_; block_is_end; local_←_; _∙_; done; return; name; fun; arg) -open import Luau.Value using (addr; val) +open import Luau.Syntax using (Expr; Stat; Block; nil; addr; var; function_is_end; _$_; block_is_end; local_←_; _∙_; done; return; name; fun; arg; binexp; BinaryOperator; +; -; *; /; number) +open import Luau.Value using (addr; val; number) + +evalBinOp : Float → BinaryOperator → Float → Float +evalBinOp x + y = primFloatPlus x y +evalBinOp x - y = primFloatMinus x y +evalBinOp x * y = primFloatTimes x y +evalBinOp x / y = primFloatDiv x y data _⊢_⟶ᴮ_⊣_ {a} : Heap a → Block a → Block a → Heap a → Set data _⊢_⟶ᴱ_⊣_ {a} : Heap a → Expr a → Expr a → Heap a → Set @@ -57,6 +64,24 @@ data _⊢_⟶ᴱ_⊣_ where --------------------------------- H ⊢ (block b is done end) ⟶ᴱ nil ⊣ H + binOpEval : + ∀ {H x op y} → + -------------------------------------------------------------------------- + H ⊢ (binexp (number x) op (number y)) ⟶ᴱ (number (evalBinOp x op y)) ⊣ H + + binOp₁ : + ∀ {H H′ x x′ op y} → + H ⊢ x ⟶ᴱ x′ ⊣ H′ → + --------------------------------------------- + H ⊢ (binexp x op y) ⟶ᴱ (binexp x′ op y) ⊣ H′ + + binOp₂ : + ∀ {H H′ x op y y′} → + H ⊢ y ⟶ᴱ y′ ⊣ H′ → + --------------------------------------------- + H ⊢ (binexp x op y) ⟶ᴱ (binexp x op y′) ⊣ H′ + + data _⊢_⟶ᴮ_⊣_ where local : ∀ {H H′ x M M′ B} → @@ -94,5 +119,3 @@ data _⊢_⟶*_⊣_ {a} : Heap a → Block a → Block a → Heap a → Set wher H′ ⊢ B′ ⟶* B″ ⊣ H″ → ------------------ H ⊢ B ⟶* B″ ⊣ H″ - - diff --git a/prototyping/Luau/RuntimeError.agda b/prototyping/Luau/RuntimeError.agda index a54d48fe..b479a72a 100644 --- a/prototyping/Luau/RuntimeError.agda +++ b/prototyping/Luau/RuntimeError.agda @@ -4,20 +4,23 @@ open import Agda.Builtin.Equality using (_≡_) open import Luau.Heap using (Heap; _[_]) open import FFI.Data.Maybe using (just; nothing) open import FFI.Data.String using (String) -open import Luau.Syntax using (Block; Expr; nil; var; addr; block_is_end; _$_; local_←_; return; done; _∙_; number) +open import Luau.Syntax using (Block; Expr; nil; var; addr; block_is_end; _$_; local_←_; return; done; _∙_; number; binexp) +open import Luau.RuntimeType using (RuntimeType; valueType) open import Luau.Value using (val) +open import Properties.Equality using (_≢_) data RuntimeErrorᴮ {a} (H : Heap a) : Block a → Set data RuntimeErrorᴱ {a} (H : Heap a) : Expr a → Set data RuntimeErrorᴱ H where - NilIsNotAFunction : ∀ {V} → RuntimeErrorᴱ H (nil $ val V) - NumberIsNotAFunction : ∀ n {V} → RuntimeErrorᴱ H (number n $ val V) + TypeMismatch : ∀ t v → (t ≢ valueType v) → RuntimeErrorᴱ H (val v) UnboundVariable : ∀ x → RuntimeErrorᴱ H (var x) SEGV : ∀ a → (H [ a ] ≡ nothing) → RuntimeErrorᴱ H (addr a) app₁ : ∀ {M N} → RuntimeErrorᴱ H M → RuntimeErrorᴱ H (M $ N) app₂ : ∀ {M N} → RuntimeErrorᴱ H N → RuntimeErrorᴱ H (M $ N) block : ∀ b {B} → RuntimeErrorᴮ H B → RuntimeErrorᴱ H (block b is B end) + bin₁ : ∀ {M N op} → RuntimeErrorᴱ H M → RuntimeErrorᴱ H (binexp M op N) + bin₂ : ∀ {M N op} → RuntimeErrorᴱ H N → RuntimeErrorᴱ H (binexp M op N) data RuntimeErrorᴮ H where local : ∀ x {M B} → RuntimeErrorᴱ H M → RuntimeErrorᴮ H (local x ← M ∙ B) diff --git a/prototyping/Luau/RuntimeError/ToString.agda b/prototyping/Luau/RuntimeError/ToString.agda index 10c0fe08..d5528c8b 100644 --- a/prototyping/Luau/RuntimeError/ToString.agda +++ b/prototyping/Luau/RuntimeError/ToString.agda @@ -2,22 +2,25 @@ module Luau.RuntimeError.ToString where open import Agda.Builtin.Float using (primShowFloat) open import FFI.Data.String using (String; _++_) -open import Luau.RuntimeError using (RuntimeErrorᴮ; RuntimeErrorᴱ; local; return; NilIsNotAFunction; NumberIsNotAFunction; UnboundVariable; SEGV; app₁; app₂; block) +open import Luau.RuntimeError using (RuntimeErrorᴮ; RuntimeErrorᴱ; local; return; TypeMismatch; UnboundVariable; SEGV; app₁; app₂; block; bin₁; bin₂) +open import Luau.RuntimeType.ToString using (runtimeTypeToString) open import Luau.Addr.ToString using (addrToString) open import Luau.Syntax.ToString using (exprToString) open import Luau.Var.ToString using (varToString) +open import Luau.Value.ToString using (valueToString) open import Luau.Syntax using (name; _$_) errToStringᴱ : ∀ {a H B} → RuntimeErrorᴱ {a} H B → String errToStringᴮ : ∀ {a H B} → RuntimeErrorᴮ {a} H B → String -errToStringᴱ NilIsNotAFunction = "nil is not a function" -errToStringᴱ (NumberIsNotAFunction n) = "number " ++ primShowFloat n ++ " is not a function" errToStringᴱ (UnboundVariable x) = "variable " ++ varToString x ++ " is unbound" errToStringᴱ (SEGV a x) = "address " ++ addrToString a ++ " is unallocated" errToStringᴱ (app₁ E) = errToStringᴱ E errToStringᴱ (app₂ E) = errToStringᴱ E +errToStringᴱ (bin₁ E) = errToStringᴱ E +errToStringᴱ (bin₂ E) = errToStringᴱ E errToStringᴱ (block b E) = errToStringᴮ E ++ "\n in call of function " ++ varToString b +errToStringᴱ (TypeMismatch t v _) = "value " ++ valueToString v ++ " is not a " ++ runtimeTypeToString t errToStringᴮ (local x E) = errToStringᴱ E ++ "\n in definition of " ++ varToString (name x) errToStringᴮ (return E) = errToStringᴱ E ++ "\n in return statement" diff --git a/prototyping/Luau/RuntimeType.agda b/prototyping/Luau/RuntimeType.agda new file mode 100644 index 00000000..25c0283a --- /dev/null +++ b/prototyping/Luau/RuntimeType.agda @@ -0,0 +1,13 @@ +module Luau.RuntimeType where + +open import Luau.Value using (Value; nil; addr; number) + +data RuntimeType : Set where + function : RuntimeType + number : RuntimeType + nil : RuntimeType + +valueType : Value → RuntimeType +valueType nil = nil +valueType (addr x) = function +valueType (number x) = number diff --git a/prototyping/Luau/RuntimeType/ToString.agda b/prototyping/Luau/RuntimeType/ToString.agda new file mode 100644 index 00000000..be67ee0c --- /dev/null +++ b/prototyping/Luau/RuntimeType/ToString.agda @@ -0,0 +1,9 @@ +module Luau.RuntimeType.ToString where + +open import FFI.Data.String using (String) +open import Luau.RuntimeType using (RuntimeType; function; number; nil) + +runtimeTypeToString : RuntimeType → String +runtimeTypeToString function = "function" +runtimeTypeToString number = "number" +runtimeTypeToString nil = "nil" diff --git a/prototyping/Luau/Substitution.agda b/prototyping/Luau/Substitution.agda index e364a000..61287d36 100644 --- a/prototyping/Luau/Substitution.agda +++ b/prototyping/Luau/Substitution.agda @@ -1,6 +1,6 @@ module Luau.Substitution where -open import Luau.Syntax using (Expr; Stat; Block; nil; addr; var; function_is_end; _$_; block_is_end; local_←_; _∙_; done; return; _⟨_⟩ ; name; fun; arg; number) +open import Luau.Syntax using (Expr; Stat; Block; nil; addr; var; function_is_end; _$_; block_is_end; local_←_; _∙_; done; return; _⟨_⟩ ; name; fun; arg; number; binexp) open import Luau.Value using (Value; val) open import Luau.Var using (Var; _≡ⱽ_) open import Properties.Dec using (Dec; yes; no) @@ -17,6 +17,7 @@ addr a [ v / x ]ᴱ = addr a (M $ N) [ v / x ]ᴱ = (M [ v / x ]ᴱ) $ (N [ v / x ]ᴱ) function F is C end [ v / x ]ᴱ = function F is C [ v / x ]ᴮunless (x ≡ⱽ name(arg F)) end block b is C end [ v / x ]ᴱ = block b is C [ v / x ]ᴮ end +(binexp e₁ op e₂) [ v / x ]ᴱ = binexp (e₁ [ v / x ]ᴱ) op (e₂ [ v / x ]ᴱ) (function F is C end ∙ B) [ v / x ]ᴮ = function F is (C [ v / x ]ᴮunless (x ≡ⱽ name(arg F))) end ∙ (B [ v / x ]ᴮunless (x ≡ⱽ fun F)) (local y ← M ∙ B) [ v / x ]ᴮ = local y ← (M [ v / x ]ᴱ) ∙ (B [ v / x ]ᴮunless (x ≡ⱽ name y)) diff --git a/prototyping/Luau/Syntax.agda b/prototyping/Luau/Syntax.agda index 1313456b..c1c31e42 100644 --- a/prototyping/Luau/Syntax.agda +++ b/prototyping/Luau/Syntax.agda @@ -33,6 +33,12 @@ arg : ∀ {a} → FunDec a → VarDec a arg (f ⟨ x ⟩∈ T) = x arg (f ⟨ x ⟩) = x +data BinaryOperator : Set where + + : BinaryOperator + - : BinaryOperator + * : BinaryOperator + / : BinaryOperator + data Block (a : Annotated) : Set data Stat (a : Annotated) : Set data Expr (a : Annotated) : Set @@ -54,3 +60,4 @@ data Expr a where function_is_end : FunDec a → Block a → Expr a block_is_end : Var → Block a → Expr a number : Float → Expr a + binexp : Expr a → BinaryOperator → Expr a → Expr a diff --git a/prototyping/Luau/Syntax/FromJSON.agda b/prototyping/Luau/Syntax/FromJSON.agda index 8191e9e7..a2c9a42f 100644 --- a/prototyping/Luau/Syntax/FromJSON.agda +++ b/prototyping/Luau/Syntax/FromJSON.agda @@ -1,6 +1,6 @@ module Luau.Syntax.FromJSON where -open import Luau.Syntax using (Block; Stat ; Expr; nil; _$_; var; var_∈_; function_is_end; _⟨_⟩; local_←_; return; done; _∙_; maybe; VarDec; number) +open import Luau.Syntax using (Block; Stat ; Expr; nil; _$_; var; var_∈_; function_is_end; _⟨_⟩; local_←_; return; done; _∙_; maybe; VarDec; number; binexp; BinaryOperator; +; -; *; /) open import Luau.Type.FromJSON using (typeFromJSON) open import Agda.Builtin.List using (List; _∷_; []) @@ -23,6 +23,9 @@ type = fromString "type" value = fromString "value" values = fromString "values" vars = fromString "vars" +op = fromString "op" +left = fromString "left" +right = fromString "right" data Lookup : Set where _,_ : String → Value → Lookup @@ -34,6 +37,8 @@ lookupIn (key ∷ keys) obj with lookup (fromString key) obj lookupIn (key ∷ keys) obj | nothing = lookupIn keys obj lookupIn (key ∷ keys) obj | just value = (key , value) +binOpFromJSON : Value → Either String BinaryOperator +binOpFromString : String → Either String BinaryOperator varDecFromJSON : Value → Either String (VarDec maybe) varDecFromObject : Object → Either String (VarDec maybe) exprFromJSON : Value → Either String (Expr maybe) @@ -43,6 +48,15 @@ statFromObject : Object → Either String (Stat maybe) blockFromJSON : Value → Either String (Block maybe) blockFromArray : Array → Either String (Block maybe) +binOpFromJSON (string s) = binOpFromString s +binOpFromJSON val = Left "Binary operator not a string" + +binOpFromString "Add" = Right + +binOpFromString "Sub" = Right - +binOpFromString "Mul" = Right * +binOpFromString "Div" = Right / +binOpFromString s = Left ("'" ++ s ++ "' is not a valid operator") + varDecFromJSON (object arg) = varDecFromObject arg varDecFromJSON val = Left "VarDec not an object" @@ -89,6 +103,15 @@ exprFromObject obj | just (string "AstExprConstantNumber") with lookup value obj exprFromObject obj | just (string "AstExprConstantNumber") | just (FFI.Data.Aeson.Value.number x) = Right (number (toFloat x)) exprFromObject obj | just (string "AstExprConstantNumber") | just _ = Left "AstExprConstantNumber value is not a number" exprFromObject obj | just (string "AstExprConstantNumber") | nothing = Left "AstExprConstantNumber missing value" +exprFromObject obj | just (string "AstExprBinary") with lookup op obj | lookup left obj | lookup right obj +exprFromObject obj | just (string "AstExprBinary") | just o | just l | just r with binOpFromJSON o | exprFromJSON l | exprFromJSON r +exprFromObject obj | just (string "AstExprBinary") | just o | just l | just r | Right o′ | Right l′ | Right r′ = Right (binexp l′ o′ r′) +exprFromObject obj | just (string "AstExprBinary") | just o | just l | just r | Left err | _ | _ = Left err +exprFromObject obj | just (string "AstExprBinary") | just o | just l | just r | _ | Left err | _ = Left err +exprFromObject obj | just (string "AstExprBinary") | just o | just l | just r | _ | _ | Left err = Left err +exprFromObject obj | just (string "AstExprBinary") | nothing | _ | _ = Left "Missing 'op' in AstExprBinary" +exprFromObject obj | just (string "AstExprBinary") | _ | nothing | _ = Left "Missing 'left' in AstExprBinary" +exprFromObject obj | just (string "AstExprBinary") | _ | _ | nothing = Left "Missing 'right' in AstExprBinary" exprFromObject obj | just (string ty) = Left ("TODO: Unsupported AstExpr " ++ ty) exprFromObject obj | just _ = Left "AstExpr type not a string" exprFromObject obj | nothing = Left "AstExpr missing type" @@ -146,3 +169,4 @@ blockFromArray arr | just value | Left err = Left err blockFromArray arr | just value | Right S with blockFromArray(tail arr) blockFromArray arr | just value | Right S | Left err = Left (err) blockFromArray arr | just value | Right S | Right B = Right (S ∙ B) + \ No newline at end of file diff --git a/prototyping/Luau/Syntax/ToString.agda b/prototyping/Luau/Syntax/ToString.agda index a20b360b..4a3061ba 100644 --- a/prototyping/Luau/Syntax/ToString.agda +++ b/prototyping/Luau/Syntax/ToString.agda @@ -1,7 +1,7 @@ module Luau.Syntax.ToString where open import Agda.Builtin.Float using (primShowFloat) -open import Luau.Syntax using (Block; Stat; Expr; VarDec; FunDec; nil; var; var_∈_; addr; _$_; function_is_end; return; local_←_; _∙_; done; block_is_end; _⟨_⟩; _⟨_⟩∈_; number) +open import Luau.Syntax using (Block; Stat; Expr; VarDec; FunDec; nil; var; var_∈_; addr; _$_; function_is_end; return; local_←_; _∙_; done; block_is_end; _⟨_⟩; _⟨_⟩∈_; number; BinaryOperator; +; -; *; /; binexp) open import FFI.Data.String using (String; _++_) open import Luau.Addr.ToString using (addrToString) open import Luau.Type.ToString using (typeToString) @@ -17,6 +17,12 @@ funDecToString ("" ⟨ x ⟩) = "function(" ++ varDecToString x ++ ")" funDecToString (f ⟨ x ⟩∈ T) = "function " ++ varToString f ++ "(" ++ varDecToString x ++ "): " ++ typeToString T funDecToString (f ⟨ x ⟩) = "function " ++ varToString f ++ "(" ++ varDecToString x ++ ")" +binOpToString : BinaryOperator → String +binOpToString + = "+" +binOpToString - = "-" +binOpToString * = "*" +binOpToString / = "/" + exprToString′ : ∀ {a} → String → Expr a → String statToString′ : ∀ {a} → String → Stat a → String blockToString′ : ∀ {a} → String → Block a → String @@ -38,6 +44,7 @@ exprToString′ lb (block b is B end) = " " ++ (blockToString′ (lb ++ " ") B) ++ lb ++ "end)()" exprToString′ lb (number x) = primShowFloat x +exprToString′ lb (binexp x op y) = exprToString′ lb x ++ " " ++ binOpToString op ++ " " ++ exprToString′ lb y statToString′ lb (function F is B end) = "local " ++ funDecToString F ++ lb ++ diff --git a/prototyping/Properties/Step.agda b/prototyping/Properties/Step.agda index 914c3468..2a0978ff 100644 --- a/prototyping/Properties/Step.agda +++ b/prototyping/Properties/Step.agda @@ -1,11 +1,13 @@ module Properties.Step where open import Agda.Builtin.Equality using (_≡_; refl) +open import Agda.Builtin.Float using (primFloatPlus; primFloatMinus; primFloatTimes; primFloatDiv) open import FFI.Data.Maybe using (just; nothing) open import Luau.Heap using (Heap; _[_]; alloc; ok; function_is_end) -open import Luau.Syntax using (Block; Expr; nil; var; addr; function_is_end; block_is_end; _$_; local_←_; return; done; _∙_; name; fun; arg; number) -open import Luau.OpSem using (_⊢_⟶ᴱ_⊣_; _⊢_⟶ᴮ_⊣_; app₁ ; app₂ ; beta; function; block; return; done; local; subst) -open import Luau.RuntimeError using (RuntimeErrorᴱ; RuntimeErrorᴮ; NilIsNotAFunction; NumberIsNotAFunction; UnboundVariable; SEGV; app₁; app₂; block; local; return) +open import Luau.Syntax using (Block; Expr; nil; var; addr; function_is_end; block_is_end; _$_; local_←_; return; done; _∙_; name; fun; arg; number; binexp; +) +open import Luau.OpSem using (_⊢_⟶ᴱ_⊣_; _⊢_⟶ᴮ_⊣_; app₁ ; app₂ ; beta; function; block; return; done; local; subst; binOpEval; evalBinOp; binOp₁; binOp₂) +open import Luau.RuntimeError using (RuntimeErrorᴱ; RuntimeErrorᴮ; TypeMismatch; UnboundVariable; SEGV; app₁; app₂; block; local; return; bin₁; bin₂) +open import Luau.RuntimeType using (function; number) open import Luau.Substitution using (_[_/_]ᴮ) open import Luau.Value using (nil; addr; val; number) open import Properties.Remember using (remember; _,_) @@ -35,8 +37,8 @@ stepᴱ H (M $ N) with stepᴱ H M stepᴱ H (M $ N) | step H′ M′ D = step H′ (M′ $ N) (app₁ D) stepᴱ H (_ $ N) | value V refl with stepᴱ H N stepᴱ H (_ $ N) | value V refl | step H′ N′ s = step H′ (val V $ N′) (app₂ s) -stepᴱ H (_ $ _) | value nil refl | value W refl = error NilIsNotAFunction -stepᴱ H (_ $ _) | value (number n) refl | value W refl = error (NumberIsNotAFunction n) +stepᴱ H (_ $ _) | value nil refl | value W refl = error (app₁ (TypeMismatch function nil λ())) +stepᴱ H (_ $ _) | value (number n) refl | value W refl = error (app₁ (TypeMismatch function (number n) λ())) stepᴱ H (_ $ _) | value (addr a) refl | value W refl with remember (H [ a ]) stepᴱ H (_ $ _) | value (addr a) refl | value W refl | (nothing , p) = error (app₁ (SEGV a p)) stepᴱ H (_ $ _) | value (addr a) refl | value W refl | (just(function F is B end) , p) = step H (block fun F is B [ W / name (arg F) ]ᴮ end) (beta p) @@ -49,6 +51,17 @@ stepᴱ H (block b is done end) | done refl = step H nil done stepᴱ H (block b is B end) | error E = error (block b E) stepᴱ H (function F is C end) with alloc H (function F is C end) stepᴱ H function F is C end | ok a H′ p = step H′ (addr a) (function p) +stepᴱ H (binexp x op y) with stepᴱ H x +stepᴱ H (binexp x op y) | value x′ refl with stepᴱ H y +stepᴱ H (binexp x op y) | value (number x′) refl | value (number y′) refl = step H (number (evalBinOp x′ op y′)) binOpEval +stepᴱ H (binexp x op y) | value (number x′) refl | step H′ y′ s = step H′ (binexp (number x′) op y′) (binOp₂ s) +stepᴱ H (binexp x op y) | value (number x′) refl | error E = error (bin₂ E) +stepᴱ H (binexp x op y) | value nil refl | _ = error (bin₁ (TypeMismatch number nil λ())) +stepᴱ H (binexp x op y) | _ | value nil refl = error (bin₂ (TypeMismatch number nil λ())) +stepᴱ H (binexp x op y) | value (addr a) refl | _ = error (bin₁ (TypeMismatch number (addr a) λ())) +stepᴱ H (binexp x op y) | _ | value (addr a) refl = error (bin₂ (TypeMismatch number (addr a) λ())) +stepᴱ H (binexp x op y) | step H′ x′ s = step H′ (binexp x′ op y) (binOp₁ s) +stepᴱ H (binexp x op y) | error E = error (bin₁ E) stepᴮ H (function F is C end ∙ B) with alloc H (function F is C end) stepᴮ H (function F is C end ∙ B) | ok a H′ p = step H′ (B [ addr a / fun F ]ᴮ) (function p) diff --git a/prototyping/Tests/Interpreter/binary_exps/in.lua b/prototyping/Tests/Interpreter/binary_exps/in.lua new file mode 100644 index 00000000..0750f9e6 --- /dev/null +++ b/prototyping/Tests/Interpreter/binary_exps/in.lua @@ -0,0 +1 @@ +return 1 + 2 - 2 * 2 / 2 diff --git a/prototyping/Tests/Interpreter/binary_exps/out.txt b/prototyping/Tests/Interpreter/binary_exps/out.txt new file mode 100644 index 00000000..d3827e75 --- /dev/null +++ b/prototyping/Tests/Interpreter/binary_exps/out.txt @@ -0,0 +1 @@ +1.0 From 0bc7c51afc3e4ce008176f291dc00e207a27571d Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Petri=20H=C3=A4kkinen?= Date: Wed, 23 Feb 2022 20:03:58 +0200 Subject: [PATCH 178/399] Lua API: add return types to table getters (#389) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: Petri Häkkinen --- VM/include/lua.h | 10 +++++----- VM/src/lapi.cpp | 20 +++++++++---------- tests/Conformance.test.cpp | 41 ++++++++++++++++++++++++++++++++++++++ 3 files changed, 56 insertions(+), 15 deletions(-) diff --git a/VM/include/lua.h b/VM/include/lua.h index c5dcef25..5f39cb3d 100644 --- a/VM/include/lua.h +++ b/VM/include/lua.h @@ -178,11 +178,11 @@ LUA_API int lua_pushthread(lua_State* L); /* ** get functions (Lua -> stack) */ -LUA_API void lua_gettable(lua_State* L, int idx); -LUA_API void lua_getfield(lua_State* L, int idx, const char* k); -LUA_API void lua_rawgetfield(lua_State* L, int idx, const char* k); -LUA_API void lua_rawget(lua_State* L, int idx); -LUA_API void lua_rawgeti(lua_State* L, int idx, int n); +LUA_API int lua_gettable(lua_State* L, int idx); +LUA_API int lua_getfield(lua_State* L, int idx, const char* k); +LUA_API int lua_rawgetfield(lua_State* L, int idx, const char* k); +LUA_API int lua_rawget(lua_State* L, int idx); +LUA_API int lua_rawgeti(lua_State* L, int idx, int n); LUA_API void lua_createtable(lua_State* L, int narr, int nrec); LUA_API void lua_setreadonly(lua_State* L, int idx, int enabled); diff --git a/VM/src/lapi.cpp b/VM/src/lapi.cpp index 29d5f397..39c76e08 100644 --- a/VM/src/lapi.cpp +++ b/VM/src/lapi.cpp @@ -659,16 +659,16 @@ int lua_pushthread(lua_State* L) ** get functions (Lua -> stack) */ -void lua_gettable(lua_State* L, int idx) +int lua_gettable(lua_State* L, int idx) { luaC_checkthreadsleep(L); StkId t = index2addr(L, idx); api_checkvalidindex(L, t); luaV_gettable(L, t, L->top - 1, L->top - 1); - return; + return ttype(L->top - 1); } -void lua_getfield(lua_State* L, int idx, const char* k) +int lua_getfield(lua_State* L, int idx, const char* k) { luaC_checkthreadsleep(L); StkId t = index2addr(L, idx); @@ -677,10 +677,10 @@ void lua_getfield(lua_State* L, int idx, const char* k) setsvalue(L, &key, luaS_new(L, k)); luaV_gettable(L, t, &key, L->top); api_incr_top(L); - return; + return ttype(L->top - 1); } -void lua_rawgetfield(lua_State* L, int idx, const char* k) +int lua_rawgetfield(lua_State* L, int idx, const char* k) { luaC_checkthreadsleep(L); StkId t = index2addr(L, idx); @@ -689,26 +689,26 @@ void lua_rawgetfield(lua_State* L, int idx, const char* k) setsvalue(L, &key, luaS_new(L, k)); setobj2s(L, L->top, luaH_getstr(hvalue(t), tsvalue(&key))); api_incr_top(L); - return; + return ttype(L->top - 1); } -void lua_rawget(lua_State* L, int idx) +int lua_rawget(lua_State* L, int idx) { luaC_checkthreadsleep(L); StkId t = index2addr(L, idx); api_check(L, ttistable(t)); setobj2s(L, L->top - 1, luaH_get(hvalue(t), L->top - 1)); - return; + return ttype(L->top - 1); } -void lua_rawgeti(lua_State* L, int idx, int n) +int lua_rawgeti(lua_State* L, int idx, int n) { luaC_checkthreadsleep(L); StkId t = index2addr(L, idx); api_check(L, ttistable(t)); setobj2s(L, L->top, luaH_getnum(hvalue(t), n)); api_incr_top(L); - return; + return ttype(L->top - 1); } void lua_createtable(lua_State* L, int narray, int nrec) diff --git a/tests/Conformance.test.cpp b/tests/Conformance.test.cpp index b09c1efb..23e90f4d 100644 --- a/tests/Conformance.test.cpp +++ b/tests/Conformance.test.cpp @@ -712,6 +712,47 @@ TEST_CASE("Reference") CHECK(dtorhits == 2); } +TEST_CASE("ApiTables") +{ + StateRef globalState(luaL_newstate(), lua_close); + lua_State* L = globalState.get(); + + lua_newtable(L); + lua_pushnumber(L, 123.0); + lua_setfield(L, -2, "key"); + lua_pushstring(L, "test"); + lua_rawseti(L, -2, 5); + + // lua_gettable + lua_pushstring(L, "key"); + CHECK(lua_gettable(L, -2) == LUA_TNUMBER); + CHECK(lua_tonumber(L, -1) == 123.0); + lua_pop(L, 1); + + // lua_getfield + CHECK(lua_getfield(L, -1, "key") == LUA_TNUMBER); + CHECK(lua_tonumber(L, -1) == 123.0); + lua_pop(L, 1); + + // lua_rawgetfield + CHECK(lua_rawgetfield(L, -1, "key") == LUA_TNUMBER); + CHECK(lua_tonumber(L, -1) == 123.0); + lua_pop(L, 1); + + // lua_rawget + lua_pushstring(L, "key"); + CHECK(lua_rawget(L, -2) == LUA_TNUMBER); + CHECK(lua_tonumber(L, -1) == 123.0); + lua_pop(L, 1); + + // lua_rawgeti + CHECK(lua_rawgeti(L, -1, 5) == LUA_TSTRING); + CHECK(strcmp(lua_tostring(L, -1), "test") == 0); + lua_pop(L, 1); + + lua_pop(L, 1); +} + TEST_CASE("ApiFunctionCalls") { StateRef globalState = runConformance("apicalls.lua"); From 0bd21762aef0b0523f8cfcffb0d4784d039e1c76 Mon Sep 17 00:00:00 2001 From: Lily Brown Date: Thu, 24 Feb 2022 11:17:46 -0800 Subject: [PATCH 179/399] Prototype bools and relational operators (#387) Prototypes booleans and relational operators. As part of this I removed `FFI/Data/Bool.agda`, because it was getting in the way - we already use `Agda.Builtin.Bool` instead for other cases. --- prototyping/Examples/Run.agda | 8 ++- prototyping/FFI/Data/Bool.agda | 8 --- prototyping/FFI/Data/Vector.agda | 2 +- prototyping/Luau/OpSem.agda | 64 +++++++++++++++---- prototyping/Luau/RuntimeType.agda | 4 +- prototyping/Luau/RuntimeType/ToString.agda | 3 +- prototyping/Luau/Substitution.agda | 4 +- prototyping/Luau/Syntax.agda | 8 +++ prototyping/Luau/Syntax/FromJSON.agda | 15 ++++- prototyping/Luau/Syntax/ToString.agda | 10 ++- prototyping/Luau/Type/FromJSON.agda | 2 +- prototyping/Luau/Value.agda | 6 +- prototyping/Luau/Value/ToString.agda | 5 +- prototyping/Properties/Step.agda | 17 +++-- .../Interpreter/binary_equality_bools/in.lua | 1 + .../Interpreter/binary_equality_bools/out.txt | 1 + .../binary_equality_numbers/in.lua | 1 + .../binary_equality_numbers/out.txt | 1 + prototyping/Utility/Bool.agda | 16 +++++ 19 files changed, 141 insertions(+), 35 deletions(-) delete mode 100644 prototyping/FFI/Data/Bool.agda create mode 100644 prototyping/Tests/Interpreter/binary_equality_bools/in.lua create mode 100644 prototyping/Tests/Interpreter/binary_equality_bools/out.txt create mode 100644 prototyping/Tests/Interpreter/binary_equality_numbers/in.lua create mode 100644 prototyping/Tests/Interpreter/binary_equality_numbers/out.txt create mode 100644 prototyping/Utility/Bool.agda diff --git a/prototyping/Examples/Run.agda b/prototyping/Examples/Run.agda index 534bb122..ddec8e8b 100644 --- a/prototyping/Examples/Run.agda +++ b/prototyping/Examples/Run.agda @@ -3,8 +3,9 @@ module Examples.Run where open import Agda.Builtin.Equality using (_≡_; refl) -open import Luau.Syntax using (nil; var; _$_; function_is_end; return; _∙_; done; _⟨_⟩; number; binexp; +) -open import Luau.Value using (nil; number) +open import Agda.Builtin.Bool using (true; false) +open import Luau.Syntax using (nil; var; _$_; function_is_end; return; _∙_; done; _⟨_⟩; number; binexp; +; <; true; false) +open import Luau.Value using (nil; number; bool) open import Luau.Run using (run; return) open import Luau.Heap using (lookup-next; next-emp; lookup-next-emp) @@ -19,3 +20,6 @@ ex2 = refl ex3 : (run (function "fn" ⟨ var "x" ⟩ is return (binexp (number 1.0) + (number 2.0)) ∙ done end ∙ return (var "fn" $ nil) ∙ done) ≡ return (number 3.0) _) ex3 = refl + +ex4 : (run (function "fn" ⟨ var "x" ⟩ is return (binexp (number 1.0) < (number 2.0)) ∙ done end ∙ return (var "fn" $ nil) ∙ done) ≡ return (bool true) _) +ex4 = refl diff --git a/prototyping/FFI/Data/Bool.agda b/prototyping/FFI/Data/Bool.agda deleted file mode 100644 index 4b04b033..00000000 --- a/prototyping/FFI/Data/Bool.agda +++ /dev/null @@ -1,8 +0,0 @@ -module FFI.Data.Bool where - -{-# FOREIGN GHC import qualified Data.Bool #-} - -data Bool : Set where - false : Bool - true : Bool -{-# COMPILE GHC Bool = data Data.Bool.Bool (Data.Bool.False|Data.Bool.True) #-} diff --git a/prototyping/FFI/Data/Vector.agda b/prototyping/FFI/Data/Vector.agda index 2c5d1925..7d2f5012 100644 --- a/prototyping/FFI/Data/Vector.agda +++ b/prototyping/FFI/Data/Vector.agda @@ -3,7 +3,7 @@ module FFI.Data.Vector where open import Agda.Builtin.Equality using (_≡_) open import Agda.Builtin.Int using (Int; pos; negsuc) open import Agda.Builtin.Nat using (Nat) -open import FFI.Data.Bool using (Bool; false; true) +open import Agda.Builtin.Bool using (Bool; false; true) open import FFI.Data.HaskellInt using (HaskellInt; haskellIntToInt; intToHaskellInt) open import FFI.Data.Maybe using (Maybe; just; nothing) diff --git a/prototyping/Luau/OpSem.agda b/prototyping/Luau/OpSem.agda index bfc63a4a..bd75d9d6 100644 --- a/prototyping/Luau/OpSem.agda +++ b/prototyping/Luau/OpSem.agda @@ -1,18 +1,50 @@ module Luau.OpSem where open import Agda.Builtin.Equality using (_≡_) -open import Agda.Builtin.Float using (Float; primFloatPlus; primFloatMinus; primFloatTimes; primFloatDiv) +open import Agda.Builtin.Float using (Float; primFloatPlus; primFloatMinus; primFloatTimes; primFloatDiv; primFloatEquality; primFloatLess; primFloatInequality) +open import Agda.Builtin.Bool using (Bool; true; false) +open import Utility.Bool using (not; _or_; _and_) +open import Agda.Builtin.Nat using (_==_) open import FFI.Data.Maybe using (just) open import Luau.Heap using (Heap; _≡_⊕_↦_; _[_]; function_is_end) open import Luau.Substitution using (_[_/_]ᴮ) -open import Luau.Syntax using (Expr; Stat; Block; nil; addr; var; function_is_end; _$_; block_is_end; local_←_; _∙_; done; return; name; fun; arg; binexp; BinaryOperator; +; -; *; /; number) -open import Luau.Value using (addr; val; number) +open import Luau.Syntax using (Expr; Stat; Block; nil; addr; var; function_is_end; _$_; block_is_end; local_←_; _∙_; done; return; name; fun; arg; binexp; BinaryOperator; +; -; *; /; <; >; ≡; ≅; ≤; ≥; number) +open import Luau.Value using (addr; val; number; Value; bool) +open import Luau.RuntimeType using (RuntimeType; valueType) -evalBinOp : Float → BinaryOperator → Float → Float -evalBinOp x + y = primFloatPlus x y -evalBinOp x - y = primFloatMinus x y -evalBinOp x * y = primFloatTimes x y -evalBinOp x / y = primFloatDiv x y +evalNumOp : Float → BinaryOperator → Float → Value +evalNumOp x + y = number (primFloatPlus x y) +evalNumOp x - y = number (primFloatMinus x y) +evalNumOp x * y = number (primFloatTimes x y) +evalNumOp x / y = number (primFloatDiv x y) +evalNumOp x < y = bool (primFloatLess x y) +evalNumOp x > y = bool (primFloatLess y x) +evalNumOp x ≡ y = bool (primFloatEquality x y) +evalNumOp x ≅ y = bool (primFloatInequality x y) +evalNumOp x ≤ y = bool ((primFloatLess x y) or (primFloatEquality x y)) +evalNumOp x ≥ y = bool ((primFloatLess y x) or (primFloatEquality x y)) + +evalEqOp : Value → Value → Value +evalEqOp Value.nil Value.nil = bool true +evalEqOp (addr x) (addr y) = bool (x == y) +evalEqOp (number x) (number y) = bool (primFloatEquality x y) +evalEqOp (bool true) (bool y) = bool y +evalEqOp (bool false) (bool y) = bool (not y) +evalEqOp _ _ = bool false + +evalNeqOp : Value → Value → Value +evalNeqOp Value.nil Value.nil = bool false +evalNeqOp (addr x) (addr y) = bool (not (x == y)) +evalNeqOp (number x) (number y) = bool (primFloatInequality x y) +evalNeqOp (bool true) (bool y) = bool (not y) +evalNeqOp (bool false) (bool y) = bool y +evalNeqOp _ _ = bool true + +coerceToBool : Value → Bool +coerceToBool Value.nil = false +coerceToBool (addr x) = true +coerceToBool (number x) = true +coerceToBool (bool x) = x data _⊢_⟶ᴮ_⊣_ {a} : Heap a → Block a → Block a → Heap a → Set data _⊢_⟶ᴱ_⊣_ {a} : Heap a → Expr a → Expr a → Heap a → Set @@ -64,10 +96,20 @@ data _⊢_⟶ᴱ_⊣_ where --------------------------------- H ⊢ (block b is done end) ⟶ᴱ nil ⊣ H - binOpEval : + binOpEquality : + ∀ {H x y} → + --------------------------------------------------------------------------- + H ⊢ (binexp (val x) BinaryOperator.≡ (val y)) ⟶ᴱ (val (evalEqOp x y)) ⊣ H + + binOpInequality : + ∀ {H x y} → + ---------------------------------------------------------------------------- + H ⊢ (binexp (val x) BinaryOperator.≅ (val y)) ⟶ᴱ (val (evalNeqOp x y)) ⊣ H + + binOpNumbers : ∀ {H x op y} → - -------------------------------------------------------------------------- - H ⊢ (binexp (number x) op (number y)) ⟶ᴱ (number (evalBinOp x op y)) ⊣ H + ----------------------------------------------------------------------- + H ⊢ (binexp (number x) op (number y)) ⟶ᴱ (val (evalNumOp x op y)) ⊣ H binOp₁ : ∀ {H H′ x x′ op y} → diff --git a/prototyping/Luau/RuntimeType.agda b/prototyping/Luau/RuntimeType.agda index 25c0283a..375fc540 100644 --- a/prototyping/Luau/RuntimeType.agda +++ b/prototyping/Luau/RuntimeType.agda @@ -1,13 +1,15 @@ module Luau.RuntimeType where -open import Luau.Value using (Value; nil; addr; number) +open import Luau.Value using (Value; nil; addr; number; bool) data RuntimeType : Set where function : RuntimeType number : RuntimeType nil : RuntimeType + boolean : RuntimeType valueType : Value → RuntimeType valueType nil = nil valueType (addr x) = function valueType (number x) = number +valueType (bool _) = boolean diff --git a/prototyping/Luau/RuntimeType/ToString.agda b/prototyping/Luau/RuntimeType/ToString.agda index be67ee0c..dfb7c955 100644 --- a/prototyping/Luau/RuntimeType/ToString.agda +++ b/prototyping/Luau/RuntimeType/ToString.agda @@ -1,9 +1,10 @@ module Luau.RuntimeType.ToString where open import FFI.Data.String using (String) -open import Luau.RuntimeType using (RuntimeType; function; number; nil) +open import Luau.RuntimeType using (RuntimeType; function; number; nil; boolean) runtimeTypeToString : RuntimeType → String runtimeTypeToString function = "function" runtimeTypeToString number = "number" runtimeTypeToString nil = "nil" +runtimeTypeToString boolean = "boolean" diff --git a/prototyping/Luau/Substitution.agda b/prototyping/Luau/Substitution.agda index 61287d36..d909c350 100644 --- a/prototyping/Luau/Substitution.agda +++ b/prototyping/Luau/Substitution.agda @@ -1,6 +1,6 @@ module Luau.Substitution where -open import Luau.Syntax using (Expr; Stat; Block; nil; addr; var; function_is_end; _$_; block_is_end; local_←_; _∙_; done; return; _⟨_⟩ ; name; fun; arg; number; binexp) +open import Luau.Syntax using (Expr; Stat; Block; nil; true; false; addr; var; function_is_end; _$_; block_is_end; local_←_; _∙_; done; return; _⟨_⟩ ; name; fun; arg; number; binexp) open import Luau.Value using (Value; val) open import Luau.Var using (Var; _≡ⱽ_) open import Properties.Dec using (Dec; yes; no) @@ -11,6 +11,8 @@ var_[_/_]ᴱwhenever_ : ∀ {a P} → Var → Value → Var → (Dec P) → Expr _[_/_]ᴮunless_ : ∀ {a P} → Block a → Value → Var → (Dec P) → Block a nil [ v / x ]ᴱ = nil +true [ v / x ]ᴱ = true +false [ v / x ]ᴱ = false var y [ v / x ]ᴱ = var y [ v / x ]ᴱwhenever (x ≡ⱽ y) addr a [ v / x ]ᴱ = addr a (number y) [ v / x ]ᴱ = number y diff --git a/prototyping/Luau/Syntax.agda b/prototyping/Luau/Syntax.agda index c1c31e42..57e59a7e 100644 --- a/prototyping/Luau/Syntax.agda +++ b/prototyping/Luau/Syntax.agda @@ -38,6 +38,12 @@ data BinaryOperator : Set where - : BinaryOperator * : BinaryOperator / : BinaryOperator + < : BinaryOperator + > : BinaryOperator + ≡ : BinaryOperator + ≅ : BinaryOperator + ≤ : BinaryOperator + ≥ : BinaryOperator data Block (a : Annotated) : Set data Stat (a : Annotated) : Set @@ -54,6 +60,8 @@ data Stat a where data Expr a where nil : Expr a + true : Expr a + false : Expr a var : Var → Expr a addr : Addr → Expr a _$_ : Expr a → Expr a → Expr a diff --git a/prototyping/Luau/Syntax/FromJSON.agda b/prototyping/Luau/Syntax/FromJSON.agda index a2c9a42f..330739b9 100644 --- a/prototyping/Luau/Syntax/FromJSON.agda +++ b/prototyping/Luau/Syntax/FromJSON.agda @@ -1,12 +1,12 @@ module Luau.Syntax.FromJSON where -open import Luau.Syntax using (Block; Stat ; Expr; nil; _$_; var; var_∈_; function_is_end; _⟨_⟩; local_←_; return; done; _∙_; maybe; VarDec; number; binexp; BinaryOperator; +; -; *; /) +open import Luau.Syntax using (Block; Stat ; Expr; nil; _$_; var; var_∈_; function_is_end; _⟨_⟩; local_←_; return; done; _∙_; maybe; VarDec; number; binexp; BinaryOperator; +; -; *; /; ≡; ≅; <; >; ≤; ≥) open import Luau.Type.FromJSON using (typeFromJSON) open import Agda.Builtin.List using (List; _∷_; []) +open import Agda.Builtin.Bool using (true; false) open import FFI.Data.Aeson using (Value; Array; Object; object; array; string; fromString; lookup) -open import FFI.Data.Bool using (true; false) open import FFI.Data.Either using (Either; Left; Right) open import FFI.Data.Maybe using (Maybe; nothing; just) open import FFI.Data.Scientific using (toFloat) @@ -55,6 +55,12 @@ binOpFromString "Add" = Right + binOpFromString "Sub" = Right - binOpFromString "Mul" = Right * binOpFromString "Div" = Right / +binOpFromString "CompareEq" = Right ≡ +binOpFromString "CompareNe" = Right ≅ +binOpFromString "CompareLt" = Right < +binOpFromString "CompareLe" = Right ≤ +binOpFromString "CompareGt" = Right > +binOpFromString "CompareGe" = Right ≥ binOpFromString s = Left ("'" ++ s ++ "' is not a valid operator") varDecFromJSON (object arg) = varDecFromObject arg @@ -103,6 +109,11 @@ exprFromObject obj | just (string "AstExprConstantNumber") with lookup value obj exprFromObject obj | just (string "AstExprConstantNumber") | just (FFI.Data.Aeson.Value.number x) = Right (number (toFloat x)) exprFromObject obj | just (string "AstExprConstantNumber") | just _ = Left "AstExprConstantNumber value is not a number" exprFromObject obj | just (string "AstExprConstantNumber") | nothing = Left "AstExprConstantNumber missing value" +exprFromObject obj | just (string "AstExprConstantBool") with lookup value obj +exprFromObject obj | just (string "AstExprConstantBool") | just (FFI.Data.Aeson.Value.bool true) = Right Expr.true +exprFromObject obj | just (string "AstExprConstantBool") | just (FFI.Data.Aeson.Value.bool false) = Right Expr.false +exprFromObject obj | just (string "AstExprConstantBool") | just _ = Left "AstExprConstantBool value is not a bool" +exprFromObject obj | just (string "AstExprConstantBool") | nothing = Left "AstExprConstantBool missing value" exprFromObject obj | just (string "AstExprBinary") with lookup op obj | lookup left obj | lookup right obj exprFromObject obj | just (string "AstExprBinary") | just o | just l | just r with binOpFromJSON o | exprFromJSON l | exprFromJSON r exprFromObject obj | just (string "AstExprBinary") | just o | just l | just r | Right o′ | Right l′ | Right r′ = Right (binexp l′ o′ r′) diff --git a/prototyping/Luau/Syntax/ToString.agda b/prototyping/Luau/Syntax/ToString.agda index 4a3061ba..58a784bf 100644 --- a/prototyping/Luau/Syntax/ToString.agda +++ b/prototyping/Luau/Syntax/ToString.agda @@ -1,7 +1,7 @@ module Luau.Syntax.ToString where open import Agda.Builtin.Float using (primShowFloat) -open import Luau.Syntax using (Block; Stat; Expr; VarDec; FunDec; nil; var; var_∈_; addr; _$_; function_is_end; return; local_←_; _∙_; done; block_is_end; _⟨_⟩; _⟨_⟩∈_; number; BinaryOperator; +; -; *; /; binexp) +open import Luau.Syntax using (Block; Stat; Expr; VarDec; FunDec; nil; var; var_∈_; addr; _$_; function_is_end; return; local_←_; _∙_; done; block_is_end; _⟨_⟩; _⟨_⟩∈_; number; BinaryOperator; +; -; *; /; <; >; ≡; ≅; ≤; ≥; binexp; true; false) open import FFI.Data.String using (String; _++_) open import Luau.Addr.ToString using (addrToString) open import Luau.Type.ToString using (typeToString) @@ -22,6 +22,12 @@ binOpToString + = "+" binOpToString - = "-" binOpToString * = "*" binOpToString / = "/" +binOpToString < = "<" +binOpToString > = ">" +binOpToString ≡ = "==" +binOpToString ≅ = "~=" +binOpToString ≤ = "<=" +binOpToString ≥ = ">=" exprToString′ : ∀ {a} → String → Expr a → String statToString′ : ∀ {a} → String → Stat a → String @@ -45,6 +51,8 @@ exprToString′ lb (block b is B end) = "end)()" exprToString′ lb (number x) = primShowFloat x exprToString′ lb (binexp x op y) = exprToString′ lb x ++ " " ++ binOpToString op ++ " " ++ exprToString′ lb y +exprToString′ lb true = "true" +exprToString′ lb false = "false" statToString′ lb (function F is B end) = "local " ++ funDecToString F ++ lb ++ diff --git a/prototyping/Luau/Type/FromJSON.agda b/prototyping/Luau/Type/FromJSON.agda index c585c3f3..49f74626 100644 --- a/prototyping/Luau/Type/FromJSON.agda +++ b/prototyping/Luau/Type/FromJSON.agda @@ -3,9 +3,9 @@ module Luau.Type.FromJSON where open import Luau.Type using (Type; nil; _⇒_; _∪_; _∩_; any; number) open import Agda.Builtin.List using (List; _∷_; []) +open import Agda.Builtin.Bool using (true; false) open import FFI.Data.Aeson using (Value; Array; Object; object; array; string; fromString; lookup) -open import FFI.Data.Bool using (true; false) open import FFI.Data.Either using (Either; Left; Right) open import FFI.Data.Maybe using (Maybe; nothing; just) open import FFI.Data.String using (String; _++_) diff --git a/prototyping/Luau/Value.agda b/prototyping/Luau/Value.agda index 1086d39c..ed51abd7 100644 --- a/prototyping/Luau/Value.agda +++ b/prototyping/Luau/Value.agda @@ -1,16 +1,20 @@ module Luau.Value where +open import Agda.Builtin.Bool using (Bool; true; false) open import Agda.Builtin.Float using (Float) open import Luau.Addr using (Addr) -open import Luau.Syntax using (Block; Expr; nil; addr; number) +open import Luau.Syntax using (Block; Expr; nil; addr; number; true; false) open import Luau.Var using (Var) data Value : Set where nil : Value addr : Addr → Value number : Float → Value + bool : Bool → Value val : ∀ {a} → Value → Expr a val nil = nil val (addr a) = addr a val (number x) = number x +val (bool false) = false +val (bool true) = true diff --git a/prototyping/Luau/Value/ToString.agda b/prototyping/Luau/Value/ToString.agda index 51c3fa78..3421d4cd 100644 --- a/prototyping/Luau/Value/ToString.agda +++ b/prototyping/Luau/Value/ToString.agda @@ -2,10 +2,13 @@ module Luau.Value.ToString where open import Agda.Builtin.String using (String) open import Agda.Builtin.Float using (primShowFloat) -open import Luau.Value using (Value; nil; addr; number) +open import Agda.Builtin.Bool using (true; false) +open import Luau.Value using (Value; nil; addr; number; bool) open import Luau.Addr.ToString using (addrToString) valueToString : Value → String valueToString nil = "nil" valueToString (addr a) = addrToString a valueToString (number x) = primShowFloat x +valueToString (bool false) = "false" +valueToString (bool true) = "true" diff --git a/prototyping/Properties/Step.agda b/prototyping/Properties/Step.agda index 2a0978ff..69dbac84 100644 --- a/prototyping/Properties/Step.agda +++ b/prototyping/Properties/Step.agda @@ -2,14 +2,15 @@ module Properties.Step where open import Agda.Builtin.Equality using (_≡_; refl) open import Agda.Builtin.Float using (primFloatPlus; primFloatMinus; primFloatTimes; primFloatDiv) +open import Agda.Builtin.Bool using (true; false) open import FFI.Data.Maybe using (just; nothing) open import Luau.Heap using (Heap; _[_]; alloc; ok; function_is_end) -open import Luau.Syntax using (Block; Expr; nil; var; addr; function_is_end; block_is_end; _$_; local_←_; return; done; _∙_; name; fun; arg; number; binexp; +) -open import Luau.OpSem using (_⊢_⟶ᴱ_⊣_; _⊢_⟶ᴮ_⊣_; app₁ ; app₂ ; beta; function; block; return; done; local; subst; binOpEval; evalBinOp; binOp₁; binOp₂) +open import Luau.Syntax using (Block; Expr; nil; var; addr; true; false; function_is_end; block_is_end; _$_; local_←_; return; done; _∙_; name; fun; arg; number; binexp; +; ≅) +open import Luau.OpSem using (_⊢_⟶ᴱ_⊣_; _⊢_⟶ᴮ_⊣_; app₁ ; app₂ ; beta; function; block; return; done; local; subst; binOpNumbers; evalNumOp; binOp₁; binOp₂; evalEqOp; evalNeqOp; binOpEquality; binOpInequality) open import Luau.RuntimeError using (RuntimeErrorᴱ; RuntimeErrorᴮ; TypeMismatch; UnboundVariable; SEGV; app₁; app₂; block; local; return; bin₁; bin₂) open import Luau.RuntimeType using (function; number) open import Luau.Substitution using (_[_/_]ᴮ) -open import Luau.Value using (nil; addr; val; number) +open import Luau.Value using (nil; addr; val; number; bool) open import Properties.Remember using (remember; _,_) data StepResultᴮ {a} (H : Heap a) (B : Block a) : Set @@ -33,12 +34,15 @@ stepᴱ H nil = value nil refl stepᴱ H (var x) = error (UnboundVariable x) stepᴱ H (addr a) = value (addr a) refl stepᴱ H (number x) = value (number x) refl +stepᴱ H (true) = value (bool true) refl +stepᴱ H (false) = value (bool false) refl stepᴱ H (M $ N) with stepᴱ H M stepᴱ H (M $ N) | step H′ M′ D = step H′ (M′ $ N) (app₁ D) stepᴱ H (_ $ N) | value V refl with stepᴱ H N stepᴱ H (_ $ N) | value V refl | step H′ N′ s = step H′ (val V $ N′) (app₂ s) stepᴱ H (_ $ _) | value nil refl | value W refl = error (app₁ (TypeMismatch function nil λ())) stepᴱ H (_ $ _) | value (number n) refl | value W refl = error (app₁ (TypeMismatch function (number n) λ())) +stepᴱ H (_ $ _) | value (bool x) refl | value W refl = error (app₁ (TypeMismatch function (bool x) λ())) stepᴱ H (_ $ _) | value (addr a) refl | value W refl with remember (H [ a ]) stepᴱ H (_ $ _) | value (addr a) refl | value W refl | (nothing , p) = error (app₁ (SEGV a p)) stepᴱ H (_ $ _) | value (addr a) refl | value W refl | (just(function F is B end) , p) = step H (block fun F is B [ W / name (arg F) ]ᴮ end) (beta p) @@ -53,13 +57,18 @@ stepᴱ H (function F is C end) with alloc H (function F is C end) stepᴱ H function F is C end | ok a H′ p = step H′ (addr a) (function p) stepᴱ H (binexp x op y) with stepᴱ H x stepᴱ H (binexp x op y) | value x′ refl with stepᴱ H y -stepᴱ H (binexp x op y) | value (number x′) refl | value (number y′) refl = step H (number (evalBinOp x′ op y′)) binOpEval +-- Have to use explicit form for ≡ here because it's a heavily overloaded symbol +stepᴱ H (binexp x Luau.Syntax.≡ y) | value x′ refl | value y′ refl = step H (val (evalEqOp x′ y′)) binOpEquality +stepᴱ H (binexp x ≅ y) | value x′ refl | value y′ refl = step H (val (evalNeqOp x′ y′)) binOpInequality +stepᴱ H (binexp x op y) | value (number x′) refl | value (number y′) refl = step H (val (evalNumOp x′ op y′)) binOpNumbers stepᴱ H (binexp x op y) | value (number x′) refl | step H′ y′ s = step H′ (binexp (number x′) op y′) (binOp₂ s) stepᴱ H (binexp x op y) | value (number x′) refl | error E = error (bin₂ E) stepᴱ H (binexp x op y) | value nil refl | _ = error (bin₁ (TypeMismatch number nil λ())) stepᴱ H (binexp x op y) | _ | value nil refl = error (bin₂ (TypeMismatch number nil λ())) stepᴱ H (binexp x op y) | value (addr a) refl | _ = error (bin₁ (TypeMismatch number (addr a) λ())) stepᴱ H (binexp x op y) | _ | value (addr a) refl = error (bin₂ (TypeMismatch number (addr a) λ())) +stepᴱ H (binexp x op y) | value (bool x′) refl | _ = error (bin₁ (TypeMismatch number (bool x′) λ())) +stepᴱ H (binexp x op y) | _ | value (bool y′) refl = error (bin₂ (TypeMismatch number (bool y′) λ())) stepᴱ H (binexp x op y) | step H′ x′ s = step H′ (binexp x′ op y) (binOp₁ s) stepᴱ H (binexp x op y) | error E = error (bin₁ E) diff --git a/prototyping/Tests/Interpreter/binary_equality_bools/in.lua b/prototyping/Tests/Interpreter/binary_equality_bools/in.lua new file mode 100644 index 00000000..c9c72dc6 --- /dev/null +++ b/prototyping/Tests/Interpreter/binary_equality_bools/in.lua @@ -0,0 +1 @@ +return true == false diff --git a/prototyping/Tests/Interpreter/binary_equality_bools/out.txt b/prototyping/Tests/Interpreter/binary_equality_bools/out.txt new file mode 100644 index 00000000..c508d536 --- /dev/null +++ b/prototyping/Tests/Interpreter/binary_equality_bools/out.txt @@ -0,0 +1 @@ +false diff --git a/prototyping/Tests/Interpreter/binary_equality_numbers/in.lua b/prototyping/Tests/Interpreter/binary_equality_numbers/in.lua new file mode 100644 index 00000000..4efc68a7 --- /dev/null +++ b/prototyping/Tests/Interpreter/binary_equality_numbers/in.lua @@ -0,0 +1 @@ +return 1 == 1 diff --git a/prototyping/Tests/Interpreter/binary_equality_numbers/out.txt b/prototyping/Tests/Interpreter/binary_equality_numbers/out.txt new file mode 100644 index 00000000..27ba77dd --- /dev/null +++ b/prototyping/Tests/Interpreter/binary_equality_numbers/out.txt @@ -0,0 +1 @@ +true diff --git a/prototyping/Utility/Bool.agda b/prototyping/Utility/Bool.agda new file mode 100644 index 00000000..1afffb0d --- /dev/null +++ b/prototyping/Utility/Bool.agda @@ -0,0 +1,16 @@ +module Utility.Bool where + +open import Agda.Builtin.Bool using (Bool; true; false) + +not : Bool → Bool +not false = true +not true = false + +_or_ : Bool → Bool → Bool +true or _ = true +_ or true = true +_ or _ = false + +_and_ : Bool → Bool → Bool +true and true = true +_ and _ = false From a8eabedd570e9b3aba7e02ff2b0f4d8bdbf9efbb Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Thu, 24 Feb 2022 15:15:41 -0800 Subject: [PATCH 180/399] Sync to upstream/release/516 --- Analysis/include/Luau/Module.h | 9 +- Analysis/include/Luau/TypeInfer.h | 2 + Analysis/include/Luau/TypeUtils.h | 4 +- Analysis/include/Luau/Unifier.h | 29 ++- Analysis/src/Autocomplete.cpp | 4 +- Analysis/src/Linter.cpp | 1 + Analysis/src/Module.cpp | 27 +-- Analysis/src/Transpiler.cpp | 8 + Analysis/src/TypeInfer.cpp | 67 +++--- Analysis/src/TypeUtils.cpp | 8 +- Analysis/src/TypeVar.cpp | 110 ++-------- Analysis/src/Unifier.cpp | 126 ++++++----- Ast/src/Parser.cpp | 8 +- VM/include/lua.h | 12 +- VM/src/lapi.cpp | 20 +- VM/src/ldo.cpp | 3 +- VM/src/lfunc.cpp | 62 ++---- VM/src/lgc.cpp | 218 +++---------------- VM/src/lgc.h | 10 +- VM/src/lgcdebug.cpp | 72 +------ VM/src/linit.cpp | 2 +- VM/src/lmem.cpp | 127 ++--------- VM/src/lmem.h | 4 - VM/src/lobject.h | 5 +- VM/src/lstate.cpp | 32 +-- VM/src/lstate.h | 5 - VM/src/lstring.cpp | 70 ++---- VM/src/ltable.cpp | 4 +- VM/src/ludata.cpp | 2 +- fuzz/linter.cpp | 8 +- fuzz/luau.proto | 55 +++-- fuzz/proto.cpp | 237 ++++++++++++++------- fuzz/protoprint.cpp | 129 +++++++++-- fuzz/prototest.cpp | 12 +- fuzz/typeck.cpp | 6 +- tests/Autocomplete.test.cpp | 1 - tests/Conformance.test.cpp | 92 +++++++- tests/Linter.test.cpp | 13 ++ tests/Parser.test.cpp | 1 - tests/Transpiler.test.cpp | 15 ++ tests/TypeInfer.intersectionTypes.test.cpp | 6 +- tests/TypeInfer.refinements.test.cpp | 16 ++ tests/TypeInfer.singletons.test.cpp | 124 +++++++++++ tests/TypeInfer.tables.test.cpp | 18 ++ tests/TypeInfer.test.cpp | 6 - tests/TypeInfer.tryUnify.test.cpp | 4 +- tests/TypeVar.test.cpp | 12 -- tools/natvis/VM.natvis | 5 +- 48 files changed, 919 insertions(+), 892 deletions(-) diff --git a/Analysis/include/Luau/Module.h b/Analysis/include/Luau/Module.h index 61200771..6c689b7c 100644 --- a/Analysis/include/Luau/Module.h +++ b/Analysis/include/Luau/Module.h @@ -13,8 +13,6 @@ #include #include -LUAU_FASTFLAG(LuauPrepopulateUnionOptionsBeforeAllocation) - namespace Luau { @@ -60,11 +58,8 @@ struct TypeArena template TypeId addType(T tv) { - if (FFlag::LuauPrepopulateUnionOptionsBeforeAllocation) - { - if constexpr (std::is_same_v) - LUAU_ASSERT(tv.options.size() >= 2); - } + if constexpr (std::is_same_v) + LUAU_ASSERT(tv.options.size() >= 2); return addTV(TypeVar(std::move(tv))); } diff --git a/Analysis/include/Luau/TypeInfer.h b/Analysis/include/Luau/TypeInfer.h index 3c5ded3c..2440c810 100644 --- a/Analysis/include/Luau/TypeInfer.h +++ b/Analysis/include/Luau/TypeInfer.h @@ -31,6 +31,7 @@ bool doesCallError(const AstExprCall* call); bool hasBreak(AstStat* node); const AstStat* getFallthrough(const AstStat* node); +struct UnifierOptions; struct Unifier; // A substitution which replaces generic types in a given set by free types. @@ -245,6 +246,7 @@ struct TypeChecker * Treat any failures as type errors in the final typecheck report. */ bool unify(TypeId subTy, TypeId superTy, const Location& location); + bool unify(TypeId subTy, TypeId superTy, const Location& location, const UnifierOptions& options); bool unify(TypePackId subTy, TypePackId superTy, const Location& location, CountMismatch::Context ctx = CountMismatch::Context::Arg); /** Attempt to unify the types. diff --git a/Analysis/include/Luau/TypeUtils.h b/Analysis/include/Luau/TypeUtils.h index ffddfe4b..42c1bc0b 100644 --- a/Analysis/include/Luau/TypeUtils.h +++ b/Analysis/include/Luau/TypeUtils.h @@ -13,7 +13,7 @@ namespace Luau using ScopePtr = std::shared_ptr; -std::optional findMetatableEntry(ErrorVec& errors, const ScopePtr& globalScope, TypeId type, std::string entry, Location location); -std::optional findTablePropertyRespectingMeta(ErrorVec& errors, const ScopePtr& globalScope, TypeId ty, Name name, Location location); +std::optional findMetatableEntry(ErrorVec& errors, TypeId type, std::string entry, Location location); +std::optional findTablePropertyRespectingMeta(ErrorVec& errors, TypeId ty, Name name, Location location); } // namespace Luau diff --git a/Analysis/include/Luau/Unifier.h b/Analysis/include/Luau/Unifier.h index 9db4e22b..fe822b01 100644 --- a/Analysis/include/Luau/Unifier.h +++ b/Analysis/include/Luau/Unifier.h @@ -19,11 +19,31 @@ enum Variance Invariant }; +// A substitution which replaces singleton types by their wider types +struct Widen : Substitution +{ + Widen(TypeArena* arena) + : Substitution(TxnLog::empty(), arena) + { + } + + bool isDirty(TypeId ty) override; + bool isDirty(TypePackId ty) override; + TypeId clean(TypeId ty) override; + TypePackId clean(TypePackId ty) override; + bool ignoreChildren(TypeId ty) override; +}; + +// TODO: Use this more widely. +struct UnifierOptions +{ + bool isFunctionCall = false; +}; + struct Unifier { TypeArena* const types; Mode mode; - ScopePtr globalScope; // sigh. Needed solely to get at string's metatable. DEPRECATED_TxnLog DEPRECATED_log; TxnLog log; @@ -34,9 +54,9 @@ struct Unifier UnifierSharedState& sharedState; - Unifier(TypeArena* types, Mode mode, ScopePtr globalScope, const Location& location, Variance variance, UnifierSharedState& sharedState, + Unifier(TypeArena* types, Mode mode, const Location& location, Variance variance, UnifierSharedState& sharedState, TxnLog* parentLog = nullptr); - Unifier(TypeArena* types, Mode mode, ScopePtr globalScope, std::vector>* sharedSeen, const Location& location, + Unifier(TypeArena* types, Mode mode, std::vector>* sharedSeen, const Location& location, Variance variance, UnifierSharedState& sharedState, TxnLog* parentLog = nullptr); // Test whether the two type vars unify. Never commits the result. @@ -65,7 +85,10 @@ private: void tryUnifyWithMetatable(TypeId subTy, TypeId superTy, bool reversed); void tryUnifyWithClass(TypeId subTy, TypeId superTy, bool reversed); void tryUnifyIndexer(const TableIndexer& subIndexer, const TableIndexer& superIndexer); + + TypeId widen(TypeId ty); TypeId deeplyOptional(TypeId ty, std::unordered_map seen = {}); + void cacheResult(TypeId subTy, TypeId superTy); public: diff --git a/Analysis/src/Autocomplete.cpp b/Analysis/src/Autocomplete.cpp index 5a1ae397..29a2c6b5 100644 --- a/Analysis/src/Autocomplete.cpp +++ b/Analysis/src/Autocomplete.cpp @@ -236,10 +236,10 @@ static TypeCorrectKind checkTypeCorrectKind(const Module& module, TypeArena* typ { ty = follow(ty); - auto canUnify = [&typeArena, &module](TypeId subTy, TypeId superTy) { + auto canUnify = [&typeArena](TypeId subTy, TypeId superTy) { InternalErrorReporter iceReporter; UnifierSharedState unifierState(&iceReporter); - Unifier unifier(typeArena, Mode::Strict, module.getModuleScope(), Location(), Variance::Covariant, unifierState); + Unifier unifier(typeArena, Mode::Strict, Location(), Variance::Covariant, unifierState); if (FFlag::LuauAutocompleteAvoidMutation && !FFlag::LuauUseCommittingTxnLog) { diff --git a/Analysis/src/Linter.cpp b/Analysis/src/Linter.cpp index 8d7d2d97..7635dc0f 100644 --- a/Analysis/src/Linter.cpp +++ b/Analysis/src/Linter.cpp @@ -201,6 +201,7 @@ static bool similar(AstExpr* lhs, AstExpr* rhs) return true; } + CASE(AstExprIfElse) return similar(le->condition, re->condition) && similar(le->trueExpr, re->trueExpr) && similar(le->falseExpr, re->falseExpr); else { LUAU_ASSERT(!"Unknown expression type"); diff --git a/Analysis/src/Module.cpp b/Analysis/src/Module.cpp index 817a33e9..412b78bb 100644 --- a/Analysis/src/Module.cpp +++ b/Analysis/src/Module.cpp @@ -16,7 +16,6 @@ LUAU_FASTFLAGVARIABLE(DebugLuauTrackOwningArena, false) // Remove with FFlagLuau LUAU_FASTINTVARIABLE(LuauTypeCloneRecursionLimit, 300) LUAU_FASTFLAG(LuauTypeAliasDefaults) LUAU_FASTFLAG(LuauImmutableTypes) -LUAU_FASTFLAGVARIABLE(LuauPrepopulateUnionOptionsBeforeAllocation, false) namespace Luau { @@ -379,28 +378,14 @@ void TypeCloner::operator()(const AnyTypeVar& t) void TypeCloner::operator()(const UnionTypeVar& t) { - if (FFlag::LuauPrepopulateUnionOptionsBeforeAllocation) - { - std::vector options; - options.reserve(t.options.size()); + std::vector options; + options.reserve(t.options.size()); - for (TypeId ty : t.options) - options.push_back(clone(ty, dest, seenTypes, seenTypePacks, cloneState)); + for (TypeId ty : t.options) + options.push_back(clone(ty, dest, seenTypes, seenTypePacks, cloneState)); - TypeId result = dest.addType(UnionTypeVar{std::move(options)}); - seenTypes[typeId] = result; - } - else - { - TypeId result = dest.addType(UnionTypeVar{}); - seenTypes[typeId] = result; - - UnionTypeVar* option = getMutable(result); - LUAU_ASSERT(option != nullptr); - - for (TypeId ty : t.options) - option->options.push_back(clone(ty, dest, seenTypes, seenTypePacks, cloneState)); - } + TypeId result = dest.addType(UnionTypeVar{std::move(options)}); + seenTypes[typeId] = result; } void TypeCloner::operator()(const IntersectionTypeVar& t) diff --git a/Analysis/src/Transpiler.cpp b/Analysis/src/Transpiler.cpp index 54bd0d5e..a02d396b 100644 --- a/Analysis/src/Transpiler.cpp +++ b/Analysis/src/Transpiler.cpp @@ -1153,6 +1153,14 @@ struct Printer writer.symbol(")"); } } + else if (const auto& a = typeAnnotation.as()) + { + writer.keyword(a->value ? "true" : "false"); + } + else if (const auto& a = typeAnnotation.as()) + { + writer.string(std::string_view(a->value.data, a->value.size)); + } else if (typeAnnotation.is()) { writer.symbol("%error-type%"); diff --git a/Analysis/src/TypeInfer.cpp b/Analysis/src/TypeInfer.cpp index c29699b7..faf60eb3 100644 --- a/Analysis/src/TypeInfer.cpp +++ b/Analysis/src/TypeInfer.cpp @@ -29,7 +29,6 @@ LUAU_FASTFLAGVARIABLE(DebugLuauFreezeDuringUnification, false) LUAU_FASTFLAGVARIABLE(LuauRecursiveTypeParameterRestriction, false) LUAU_FASTFLAGVARIABLE(LuauGenericFunctionsDontCacheTypeParams, false) LUAU_FASTFLAGVARIABLE(LuauImmutableTypes, false) -LUAU_FASTFLAGVARIABLE(LuauNoSealedTypeMod, false) LUAU_FASTFLAGVARIABLE(LuauQuantifyInPlace2, false) LUAU_FASTFLAGVARIABLE(LuauSealExports, false) LUAU_FASTFLAGVARIABLE(LuauSingletonTypes, false) @@ -38,14 +37,14 @@ LUAU_FASTFLAGVARIABLE(LuauTypeAliasDefaults, false) LUAU_FASTFLAGVARIABLE(LuauExpectedTypesOfProperties, false) LUAU_FASTFLAGVARIABLE(LuauErrorRecoveryType, false) LUAU_FASTFLAGVARIABLE(LuauPropertiesGetExpectedType, false) -LUAU_FASTFLAGVARIABLE(LuauProperTypeLevels, false) LUAU_FASTFLAGVARIABLE(LuauAscribeCorrectLevelToInferredProperitesOfFreeTables, false) -LUAU_FASTFLAG(LuauUnionTagMatchFix) LUAU_FASTFLAGVARIABLE(LuauUnsealedTableLiteral, false) LUAU_FASTFLAGVARIABLE(LuauTwoPassAliasDefinitionFix, false) LUAU_FASTFLAGVARIABLE(LuauAssertStripsFalsyTypes, false) LUAU_FASTFLAGVARIABLE(LuauReturnAnyInsteadOfICE, false) // Eventually removed as false. LUAU_FASTFLAGVARIABLE(LuauAnotherTypeLevelFix, false) +LUAU_FASTFLAG(LuauWidenIfSupertypeIsFree) +LUAU_FASTFLAGVARIABLE(LuauDoNotTryToReduce, false) namespace Luau { @@ -1125,7 +1124,7 @@ void TypeChecker::check(const ScopePtr& scope, TypeId ty, const ScopePtr& funSco ty = follow(ty); - if (tableSelf && (FFlag::LuauNoSealedTypeMod ? tableSelf->state != TableState::Sealed : !selfTy->persistent)) + if (tableSelf && tableSelf->state != TableState::Sealed) tableSelf->props[indexName->index.value] = {ty, /* deprecated */ false, {}, indexName->indexLocation}; const FunctionTypeVar* funTy = get(ty); @@ -1138,7 +1137,7 @@ void TypeChecker::check(const ScopePtr& scope, TypeId ty, const ScopePtr& funSco checkFunctionBody(funScope, ty, *function.func); - if (tableSelf && (FFlag::LuauNoSealedTypeMod ? tableSelf->state != TableState::Sealed : !selfTy->persistent)) + if (tableSelf && tableSelf->state != TableState::Sealed) tableSelf->props[indexName->index.value] = { follow(quantify(funScope, ty, indexName->indexLocation)), /* deprecated */ false, {}, indexName->indexLocation}; } @@ -1210,8 +1209,7 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatTypeAlias& typealias { ScopePtr aliasScope = childScope(scope, typealias.location); aliasScope->level = scope->level.incr(); - if (FFlag::LuauProperTypeLevels) - aliasScope->level.subLevel = subLevel; + aliasScope->level.subLevel = subLevel; auto [generics, genericPacks] = createGenericTypes(aliasScope, scope->level, typealias, typealias.generics, typealias.genericPacks, /* useCache = */ true); @@ -1624,7 +1622,7 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprIn std::optional TypeChecker::findTablePropertyRespectingMeta(TypeId lhsType, Name name, const Location& location) { ErrorVec errors; - auto result = Luau::findTablePropertyRespectingMeta(errors, globalScope, lhsType, name, location); + auto result = Luau::findTablePropertyRespectingMeta(errors, lhsType, name, location); reportErrors(errors); return result; } @@ -1632,7 +1630,7 @@ std::optional TypeChecker::findTablePropertyRespectingMeta(TypeId lhsTyp std::optional TypeChecker::findMetatableEntry(TypeId type, std::string entry, const Location& location) { ErrorVec errors; - auto result = Luau::findMetatableEntry(errors, globalScope, type, entry, location); + auto result = Luau::findMetatableEntry(errors, type, entry, location); reportErrors(errors); return result; } @@ -1751,13 +1749,23 @@ std::optional TypeChecker::getIndexTypeFromType( return std::nullopt; } - // TODO(amccord): Write some logic to correctly handle intersections. CLI-34659 - std::vector result = reduceUnion(parts); + if (FFlag::LuauDoNotTryToReduce) + { + if (parts.size() == 1) + return parts[0]; - if (result.size() == 1) - return result[0]; + return addType(IntersectionTypeVar{std::move(parts)}); // Not at all correct. + } + else + { + // TODO(amccord): Write some logic to correctly handle intersections. CLI-34659 + std::vector result = reduceUnion(parts); - return addType(IntersectionTypeVar{result}); + if (result.size() == 1) + return result[0]; + + return addType(IntersectionTypeVar{result}); + } } if (addErrors) @@ -2823,10 +2831,7 @@ TypeId TypeChecker::checkLValueBinding(const ScopePtr& scope, const AstExprIndex TypeId TypeChecker::checkFunctionName(const ScopePtr& scope, AstExpr& funName, TypeLevel level) { auto freshTy = [&]() { - if (FFlag::LuauProperTypeLevels) - return freshType(level); - else - return freshType(scope); + return freshType(level); }; if (auto globalName = funName.as()) @@ -3790,7 +3795,14 @@ std::optional> TypeChecker::checkCallOverload(const Scope // has been instantiated, so is a monotype. We can therefore // unify it with a monomorphic function. TypeId r = addType(FunctionTypeVar(scope->level, argPack, retPack)); - unify(fn, r, expr.location); + if (FFlag::LuauWidenIfSupertypeIsFree) + { + UnifierOptions options; + options.isFunctionCall = true; + unify(r, fn, expr.location, options); + } + else + unify(fn, r, expr.location); return {{retPack}}; } @@ -4243,9 +4255,15 @@ TypeId TypeChecker::anyIfNonstrict(TypeId ty) const } bool TypeChecker::unify(TypeId subTy, TypeId superTy, const Location& location) +{ + UnifierOptions options; + return unify(subTy, superTy, location, options); +} + +bool TypeChecker::unify(TypeId subTy, TypeId superTy, const Location& location, const UnifierOptions& options) { Unifier state = mkUnifier(location); - state.tryUnify(subTy, superTy); + state.tryUnify(subTy, superTy, options.isFunctionCall); if (FFlag::LuauUseCommittingTxnLog) state.log.commit(); @@ -4654,7 +4672,7 @@ void TypeChecker::diagnoseMissingTableKey(UnknownProperty* utk, TypeErrorData& d } }; - if (auto ttv = getTableType(FFlag::LuauUnionTagMatchFix ? utk->table : follow(utk->table))) + if (auto ttv = getTableType(utk->table)) accumulate(ttv->props); else if (auto ctv = get(follow(utk->table))) { @@ -4691,8 +4709,7 @@ ScopePtr TypeChecker::childFunctionScope(const ScopePtr& parent, const Location& ScopePtr TypeChecker::childScope(const ScopePtr& parent, const Location& location) { ScopePtr scope = std::make_shared(parent); - if (FFlag::LuauProperTypeLevels) - scope->level = parent->level; + scope->level = parent->level; scope->varargPack = parent->varargPack; currentModule->scopes.push_back(std::make_pair(location, scope)); @@ -4724,7 +4741,7 @@ void TypeChecker::merge(RefinementMap& l, const RefinementMap& r) Unifier TypeChecker::mkUnifier(const Location& location) { - return Unifier{¤tModule->internalTypes, currentModule->mode, globalScope, location, Variance::Covariant, unifierState}; + return Unifier{¤tModule->internalTypes, currentModule->mode, location, Variance::Covariant, unifierState}; } TypeId TypeChecker::freshType(const ScopePtr& scope) @@ -5444,7 +5461,7 @@ GenericTypeDefinitions TypeChecker::createGenericTypes(const ScopePtr& scope, st void TypeChecker::refineLValue(const LValue& lvalue, RefinementMap& refis, const ScopePtr& scope, TypeIdPredicate predicate) { - LUAU_ASSERT(FFlag::LuauDiscriminableUnions2); + LUAU_ASSERT(FFlag::LuauDiscriminableUnions2 || FFlag::LuauAssertStripsFalsyTypes); const LValue* target = &lvalue; std::optional key; // If set, we know we took the base of the lvalue path and should be walking down each option of the base's type. diff --git a/Analysis/src/TypeUtils.cpp b/Analysis/src/TypeUtils.cpp index 593b54c8..c2435890 100644 --- a/Analysis/src/TypeUtils.cpp +++ b/Analysis/src/TypeUtils.cpp @@ -10,7 +10,7 @@ LUAU_FASTFLAGVARIABLE(LuauTerminateCyclicMetatableIndexLookup, false) namespace Luau { -std::optional findMetatableEntry(ErrorVec& errors, const ScopePtr& globalScope, TypeId type, std::string entry, Location location) +std::optional findMetatableEntry(ErrorVec& errors, TypeId type, std::string entry, Location location) { type = follow(type); @@ -37,7 +37,7 @@ std::optional findMetatableEntry(ErrorVec& errors, const ScopePtr& globa return std::nullopt; } -std::optional findTablePropertyRespectingMeta(ErrorVec& errors, const ScopePtr& globalScope, TypeId ty, Name name, Location location) +std::optional findTablePropertyRespectingMeta(ErrorVec& errors, TypeId ty, Name name, Location location) { if (get(ty)) return ty; @@ -49,7 +49,7 @@ std::optional findTablePropertyRespectingMeta(ErrorVec& errors, const Sc return it->second.type; } - std::optional mtIndex = findMetatableEntry(errors, globalScope, ty, "__index", location); + std::optional mtIndex = findMetatableEntry(errors, ty, "__index", location); int count = 0; while (mtIndex) { @@ -82,7 +82,7 @@ std::optional findTablePropertyRespectingMeta(ErrorVec& errors, const Sc else errors.push_back(TypeError{location, GenericError{"__index should either be a function or table. Got " + toString(index)}}); - mtIndex = findMetatableEntry(errors, globalScope, *mtIndex, "__index", location); + mtIndex = findMetatableEntry(errors, *mtIndex, "__index", location); } return std::nullopt; diff --git a/Analysis/src/TypeVar.cpp b/Analysis/src/TypeVar.cpp index b2358c27..a1dcfdbe 100644 --- a/Analysis/src/TypeVar.cpp +++ b/Analysis/src/TypeVar.cpp @@ -23,9 +23,7 @@ LUAU_FASTFLAG(DebugLuauFreezeArena) LUAU_FASTINTVARIABLE(LuauTypeMaximumStringifierLength, 500) LUAU_FASTINTVARIABLE(LuauTableTypeMaximumStringifierLength, 0) LUAU_FASTINT(LuauTypeInferRecursionLimit) -LUAU_FASTFLAGVARIABLE(LuauRefactorTypeVarQuestions, false) LUAU_FASTFLAG(LuauErrorRecoveryType) -LUAU_FASTFLAG(LuauUnionTagMatchFix) LUAU_FASTFLAG(LuauDiscriminableUnions2) namespace Luau @@ -145,20 +143,13 @@ bool isNil(TypeId ty) bool isBoolean(TypeId ty) { - if (FFlag::LuauRefactorTypeVarQuestions) - { - if (isPrim(ty, PrimitiveTypeVar::Boolean) || get(get(follow(ty)))) - return true; + if (isPrim(ty, PrimitiveTypeVar::Boolean) || get(get(follow(ty)))) + return true; - if (auto utv = get(follow(ty))) - return std::all_of(begin(utv), end(utv), isBoolean); + if (auto utv = get(follow(ty))) + return std::all_of(begin(utv), end(utv), isBoolean); - return false; - } - else - { - return isPrim(ty, PrimitiveTypeVar::Boolean); - } + return false; } bool isNumber(TypeId ty) @@ -168,20 +159,13 @@ bool isNumber(TypeId ty) bool isString(TypeId ty) { - if (FFlag::LuauRefactorTypeVarQuestions) - { - if (isPrim(ty, PrimitiveTypeVar::String) || get(get(follow(ty)))) - return true; + if (isPrim(ty, PrimitiveTypeVar::String) || get(get(follow(ty)))) + return true; - if (auto utv = get(follow(ty))) - return std::all_of(begin(utv), end(utv), isString); + if (auto utv = get(follow(ty))) + return std::all_of(begin(utv), end(utv), isString); - return false; - } - else - { - return isPrim(ty, PrimitiveTypeVar::String); - } + return false; } bool isThread(TypeId ty) @@ -194,45 +178,11 @@ bool isOptional(TypeId ty) if (isNil(ty)) return true; - if (FFlag::LuauRefactorTypeVarQuestions) - { - auto utv = get(follow(ty)); - if (!utv) - return false; - - return std::any_of(begin(utv), end(utv), isNil); - } - else - { - std::unordered_set seen; - std::deque queue{ty}; - while (!queue.empty()) - { - TypeId current = follow(queue.front()); - queue.pop_front(); - - if (seen.count(current)) - continue; - - seen.insert(current); - - if (isNil(current)) - return true; - - if (auto u = get(current)) - { - for (TypeId option : u->options) - { - if (isNil(option)) - return true; - - queue.push_back(option); - } - } - } - + auto utv = get(follow(ty)); + if (!utv) return false; - } + + return std::any_of(begin(utv), end(utv), isNil); } bool isTableIntersection(TypeId ty) @@ -263,38 +213,24 @@ std::optional getMetatable(TypeId type) return mtType->metatable; else if (const ClassTypeVar* classType = get(type)) return classType->metatable; - else if (FFlag::LuauRefactorTypeVarQuestions) + else if (isString(type)) { - if (isString(type)) - { - auto ptv = get(getSingletonTypes().stringType); - LUAU_ASSERT(ptv && ptv->metatable); - return ptv->metatable; - } - else - return std::nullopt; - } - else - { - if (const PrimitiveTypeVar* primitiveType = get(type); primitiveType && primitiveType->metatable) - { - LUAU_ASSERT(primitiveType->type == PrimitiveTypeVar::String); - return primitiveType->metatable; - } - else - return std::nullopt; + auto ptv = get(getSingletonTypes().stringType); + LUAU_ASSERT(ptv && ptv->metatable); + return ptv->metatable; } + + return std::nullopt; } const TableTypeVar* getTableType(TypeId type) { - if (FFlag::LuauUnionTagMatchFix) - type = follow(type); + type = follow(type); if (const TableTypeVar* ttv = get(type)) return ttv; else if (const MetatableTypeVar* mtv = get(type)) - return get(FFlag::LuauUnionTagMatchFix ? follow(mtv->table) : mtv->table); + return get(follow(mtv->table)); else return nullptr; } @@ -311,7 +247,7 @@ const std::string* getName(TypeId type) { if (mtv->syntheticName) return &*mtv->syntheticName; - type = FFlag::LuauUnionTagMatchFix ? follow(mtv->table) : mtv->table; + type = follow(mtv->table); } if (auto ttv = get(type)) diff --git a/Analysis/src/Unifier.cpp b/Analysis/src/Unifier.cpp index 322f6ebf..d0eba013 100644 --- a/Analysis/src/Unifier.cpp +++ b/Analysis/src/Unifier.cpp @@ -21,9 +21,8 @@ LUAU_FASTFLAGVARIABLE(LuauTableSubtypingVariance2, false); LUAU_FASTFLAGVARIABLE(LuauTableUnificationEarlyTest, false) LUAU_FASTFLAG(LuauSingletonTypes) LUAU_FASTFLAG(LuauErrorRecoveryType); -LUAU_FASTFLAG(LuauProperTypeLevels); -LUAU_FASTFLAGVARIABLE(LuauUnionTagMatchFix, false) LUAU_FASTFLAGVARIABLE(LuauFollowWithCommittingTxnLogInAnyUnification, false) +LUAU_FASTFLAGVARIABLE(LuauWidenIfSupertypeIsFree, false) namespace Luau { @@ -122,7 +121,7 @@ struct PromoteTypeLevels } }; -void promoteTypeLevels(DEPRECATED_TxnLog& DEPRECATED_log, TxnLog& log, const TypeArena* typeArena, TypeLevel minLevel, TypeId ty) +static void promoteTypeLevels(DEPRECATED_TxnLog& DEPRECATED_log, TxnLog& log, const TypeArena* typeArena, TypeLevel minLevel, TypeId ty) { // Type levels of types from other modules are already global, so we don't need to promote anything inside if (FFlag::LuauImmutableTypes && ty->owningArena != typeArena) @@ -133,6 +132,7 @@ void promoteTypeLevels(DEPRECATED_TxnLog& DEPRECATED_log, TxnLog& log, const Typ visitTypeVarOnce(ty, ptl, seen); } +// TODO: use this and make it static. void promoteTypeLevels(DEPRECATED_TxnLog& DEPRECATED_log, TxnLog& log, const TypeArena* typeArena, TypeLevel minLevel, TypePackId tp) { // Type levels of types from other modules are already global, so we don't need to promote anything inside @@ -247,6 +247,48 @@ struct SkipCacheForType bool result = false; }; +bool Widen::isDirty(TypeId ty) +{ + return FFlag::LuauUseCommittingTxnLog ? log->is(ty) : bool(get(ty)); +} + +bool Widen::isDirty(TypePackId) +{ + return false; +} + +TypeId Widen::clean(TypeId ty) +{ + LUAU_ASSERT(isDirty(ty)); + auto stv = FFlag::LuauUseCommittingTxnLog ? log->getMutable(ty) : getMutable(ty); + LUAU_ASSERT(stv); + + if (get(stv)) + return getSingletonTypes().stringType; + else + { + // If this assert trips, it's likely we now have number singletons. + LUAU_ASSERT(get(stv)); + return getSingletonTypes().booleanType; + } +} + +TypePackId Widen::clean(TypePackId) +{ + throw std::runtime_error("Widen attempted to clean a dirty type pack?"); +} + +bool Widen::ignoreChildren(TypeId ty) +{ + // Sometimes we unify ("hi") -> free1 with (free2) -> free3, so don't ignore functions. + // TODO: should we be doing this? we would need to rework how checkCallOverload does the unification. + if (FFlag::LuauUseCommittingTxnLog ? log->is(ty) : bool(get(ty))) + return false; + + // We only care about unions. + return !(FFlag::LuauUseCommittingTxnLog ? log->is(ty) : bool(get(ty))); +} + static std::optional hasUnificationTooComplex(const ErrorVec& errors) { auto isUnificationTooComplex = [](const TypeError& te) { @@ -263,43 +305,22 @@ static std::optional hasUnificationTooComplex(const ErrorVec& errors) // Used for tagged union matching heuristic, returns first singleton type field static std::optional> getTableMatchTag(TypeId type) { - if (FFlag::LuauUnionTagMatchFix) + if (auto ttv = getTableType(type)) { - if (auto ttv = getTableType(type)) + for (auto&& [name, prop] : ttv->props) { - for (auto&& [name, prop] : ttv->props) - { - if (auto sing = get(follow(prop.type))) - return {{name, sing}}; - } - } - } - else - { - type = follow(type); - - if (auto ttv = get(type)) - { - for (auto&& [name, prop] : ttv->props) - { - if (auto sing = get(follow(prop.type))) - return {{name, sing}}; - } - } - else if (auto mttv = get(type)) - { - return getTableMatchTag(mttv->table); + if (auto sing = get(follow(prop.type))) + return {{name, sing}}; } } return std::nullopt; } -Unifier::Unifier(TypeArena* types, Mode mode, ScopePtr globalScope, const Location& location, Variance variance, UnifierSharedState& sharedState, +Unifier::Unifier(TypeArena* types, Mode mode, const Location& location, Variance variance, UnifierSharedState& sharedState, TxnLog* parentLog) : types(types) , mode(mode) - , globalScope(std::move(globalScope)) , log(parentLog) , location(location) , variance(variance) @@ -308,11 +329,10 @@ Unifier::Unifier(TypeArena* types, Mode mode, ScopePtr globalScope, const Locati LUAU_ASSERT(sharedState.iceHandler); } -Unifier::Unifier(TypeArena* types, Mode mode, ScopePtr globalScope, std::vector>* sharedSeen, const Location& location, +Unifier::Unifier(TypeArena* types, Mode mode, std::vector>* sharedSeen, const Location& location, Variance variance, UnifierSharedState& sharedState, TxnLog* parentLog) : types(types) , mode(mode) - , globalScope(std::move(globalScope)) , DEPRECATED_log(sharedSeen) , log(parentLog, sharedSeen) , location(location) @@ -435,6 +455,8 @@ void Unifier::tryUnify_(TypeId subTy, TypeId superTy, bool isFunctionCall, bool } else if (superFree) { + TypeLevel superLevel = superFree->level; + occursCheck(superTy, subTy); bool occursFailed = false; if (FFlag::LuauUseCommittingTxnLog) @@ -442,8 +464,6 @@ void Unifier::tryUnify_(TypeId subTy, TypeId superTy, bool isFunctionCall, bool else occursFailed = bool(get(superTy)); - TypeLevel superLevel = superFree->level; - // Unification can't change the level of a generic. auto subGeneric = FFlag::LuauUseCommittingTxnLog ? log.getMutable(subTy) : get(subTy); if (subGeneric && !subGeneric->level.subsumes(superLevel)) @@ -459,20 +479,14 @@ void Unifier::tryUnify_(TypeId subTy, TypeId superTy, bool isFunctionCall, bool if (FFlag::LuauUseCommittingTxnLog) { promoteTypeLevels(DEPRECATED_log, log, types, superLevel, subTy); - log.replace(superTy, BoundTypeVar(subTy)); + log.replace(superTy, BoundTypeVar(widen(subTy))); } else { - if (FFlag::LuauProperTypeLevels) - promoteTypeLevels(DEPRECATED_log, log, types, superLevel, subTy); - else if (auto subLevel = getMutableLevel(subTy)) - { - if (!subLevel->subsumes(superFree->level)) - *subLevel = superFree->level; - } + promoteTypeLevels(DEPRECATED_log, log, types, superLevel, subTy); DEPRECATED_log(superTy); - *asMutable(superTy) = BoundTypeVar(subTy); + *asMutable(superTy) = BoundTypeVar(widen(subTy)); } } @@ -507,16 +521,7 @@ void Unifier::tryUnify_(TypeId subTy, TypeId superTy, bool isFunctionCall, bool } else { - if (FFlag::LuauProperTypeLevels) - promoteTypeLevels(DEPRECATED_log, log, types, subLevel, superTy); - else if (auto superLevel = getMutableLevel(superTy)) - { - if (!superLevel->subsumes(subFree->level)) - { - DEPRECATED_log(superTy); - *superLevel = subFree->level; - } - } + promoteTypeLevels(DEPRECATED_log, log, types, subLevel, superTy); DEPRECATED_log(subTy); *asMutable(subTy) = BoundTypeVar(superTy); @@ -2064,6 +2069,17 @@ void Unifier::tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) } } +TypeId Unifier::widen(TypeId ty) +{ + if (!FFlag::LuauWidenIfSupertypeIsFree) + return ty; + + Widen widen{types}; + std::optional result = widen.substitute(ty); + // TODO: what does it mean for substitution to fail to widen? + return result.value_or(ty); +} + TypeId Unifier::deeplyOptional(TypeId ty, std::unordered_map seen) { ty = follow(ty); @@ -2915,7 +2931,7 @@ void Unifier::tryUnifyWithAny(TypePackId subTy, TypePackId anyTp) std::optional Unifier::findTablePropertyRespectingMeta(TypeId lhsType, Name name) { - return Luau::findTablePropertyRespectingMeta(errors, globalScope, lhsType, name, location); + return Luau::findTablePropertyRespectingMeta(errors, lhsType, name, location); } void Unifier::occursCheck(TypeId needle, TypeId haystack) @@ -3096,9 +3112,9 @@ void Unifier::occursCheck(DenseHashSet& seen, TypePackId needle, Typ Unifier Unifier::makeChildUnifier() { if (FFlag::LuauUseCommittingTxnLog) - return Unifier{types, mode, globalScope, log.sharedSeen, location, variance, sharedState, &log}; + return Unifier{types, mode, log.sharedSeen, location, variance, sharedState, &log}; else - return Unifier{types, mode, globalScope, DEPRECATED_log.sharedSeen, location, variance, sharedState, &log}; + return Unifier{types, mode, DEPRECATED_log.sharedSeen, location, variance, sharedState, &log}; } bool Unifier::isNonstrictMode() const diff --git a/Ast/src/Parser.cpp b/Ast/src/Parser.cpp index 235d6349..8767daa0 100644 --- a/Ast/src/Parser.cpp +++ b/Ast/src/Parser.cpp @@ -12,7 +12,6 @@ LUAU_FASTINTVARIABLE(LuauRecursionLimit, 1000) LUAU_FASTINTVARIABLE(LuauParseErrorLimit, 100) LUAU_FASTFLAGVARIABLE(LuauParseSingletonTypes, false) LUAU_FASTFLAGVARIABLE(LuauParseTypeAliasDefaults, false) -LUAU_FASTFLAGVARIABLE(LuauParseRecoverTypePackEllipsis, false) LUAU_FASTFLAGVARIABLE(LuauParseAllHotComments, false) LUAU_FASTFLAGVARIABLE(LuauTableFieldFunctionDebugname, false) @@ -2372,11 +2371,11 @@ std::pair, AstArray> Parser::parseG { Location nameLocation = lexer.current().location; AstName name = parseName().name; - if (lexer.current().type == Lexeme::Dot3 || (FFlag::LuauParseRecoverTypePackEllipsis && seenPack)) + if (lexer.current().type == Lexeme::Dot3 || seenPack) { seenPack = true; - if (FFlag::LuauParseRecoverTypePackEllipsis && lexer.current().type != Lexeme::Dot3) + if (lexer.current().type != Lexeme::Dot3) report(lexer.current().location, "Generic types come before generic type packs"); else nextLexeme(); @@ -2414,9 +2413,6 @@ std::pair, AstArray> Parser::parseG } else { - if (!FFlag::LuauParseRecoverTypePackEllipsis && seenPack) - report(lexer.current().location, "Generic types come before generic type packs"); - if (withDefaultValues && lexer.current().type == '=') { seenDefault = true; diff --git a/VM/include/lua.h b/VM/include/lua.h index c5dcef25..af0e2835 100644 --- a/VM/include/lua.h +++ b/VM/include/lua.h @@ -44,7 +44,7 @@ typedef int (*lua_Continuation)(lua_State* L, int status); ** prototype for memory-allocation functions */ -typedef void* (*lua_Alloc)(lua_State* L, void* ud, void* ptr, size_t osize, size_t nsize); +typedef void* (*lua_Alloc)(void* ud, void* ptr, size_t osize, size_t nsize); /* non-return type */ #define l_noret void LUA_NORETURN @@ -178,11 +178,11 @@ LUA_API int lua_pushthread(lua_State* L); /* ** get functions (Lua -> stack) */ -LUA_API void lua_gettable(lua_State* L, int idx); -LUA_API void lua_getfield(lua_State* L, int idx, const char* k); -LUA_API void lua_rawgetfield(lua_State* L, int idx, const char* k); -LUA_API void lua_rawget(lua_State* L, int idx); -LUA_API void lua_rawgeti(lua_State* L, int idx, int n); +LUA_API int lua_gettable(lua_State* L, int idx); +LUA_API int lua_getfield(lua_State* L, int idx, const char* k); +LUA_API int lua_rawgetfield(lua_State* L, int idx, const char* k); +LUA_API int lua_rawget(lua_State* L, int idx); +LUA_API int lua_rawgeti(lua_State* L, int idx, int n); LUA_API void lua_createtable(lua_State* L, int narr, int nrec); LUA_API void lua_setreadonly(lua_State* L, int idx, int enabled); diff --git a/VM/src/lapi.cpp b/VM/src/lapi.cpp index 29d5f397..39c76e08 100644 --- a/VM/src/lapi.cpp +++ b/VM/src/lapi.cpp @@ -659,16 +659,16 @@ int lua_pushthread(lua_State* L) ** get functions (Lua -> stack) */ -void lua_gettable(lua_State* L, int idx) +int lua_gettable(lua_State* L, int idx) { luaC_checkthreadsleep(L); StkId t = index2addr(L, idx); api_checkvalidindex(L, t); luaV_gettable(L, t, L->top - 1, L->top - 1); - return; + return ttype(L->top - 1); } -void lua_getfield(lua_State* L, int idx, const char* k) +int lua_getfield(lua_State* L, int idx, const char* k) { luaC_checkthreadsleep(L); StkId t = index2addr(L, idx); @@ -677,10 +677,10 @@ void lua_getfield(lua_State* L, int idx, const char* k) setsvalue(L, &key, luaS_new(L, k)); luaV_gettable(L, t, &key, L->top); api_incr_top(L); - return; + return ttype(L->top - 1); } -void lua_rawgetfield(lua_State* L, int idx, const char* k) +int lua_rawgetfield(lua_State* L, int idx, const char* k) { luaC_checkthreadsleep(L); StkId t = index2addr(L, idx); @@ -689,26 +689,26 @@ void lua_rawgetfield(lua_State* L, int idx, const char* k) setsvalue(L, &key, luaS_new(L, k)); setobj2s(L, L->top, luaH_getstr(hvalue(t), tsvalue(&key))); api_incr_top(L); - return; + return ttype(L->top - 1); } -void lua_rawget(lua_State* L, int idx) +int lua_rawget(lua_State* L, int idx) { luaC_checkthreadsleep(L); StkId t = index2addr(L, idx); api_check(L, ttistable(t)); setobj2s(L, L->top - 1, luaH_get(hvalue(t), L->top - 1)); - return; + return ttype(L->top - 1); } -void lua_rawgeti(lua_State* L, int idx, int n) +int lua_rawgeti(lua_State* L, int idx, int n) { luaC_checkthreadsleep(L); StkId t = index2addr(L, idx); api_check(L, ttistable(t)); setobj2s(L, L->top, luaH_getnum(hvalue(t), n)); api_incr_top(L); - return; + return ttype(L->top - 1); } void lua_createtable(lua_State* L, int narray, int nrec) diff --git a/VM/src/ldo.cpp b/VM/src/ldo.cpp index d87f0661..b5ae496b 100644 --- a/VM/src/ldo.cpp +++ b/VM/src/ldo.cpp @@ -151,8 +151,7 @@ l_noret luaD_throw(lua_State* L, int errcode) static void correctstack(lua_State* L, TValue* oldstack) { L->top = (L->top - oldstack) + L->stack; - // TODO (FFlagLuauGcPagedSweep): 'next' type will change after removal of the flag and the cast will not be required - for (UpVal* up = L->openupval; up != NULL; up = (UpVal*)up->next) + for (UpVal* up = L->openupval; up != NULL; up = up->u.l.threadnext) up->v = (up->v - oldstack) + L->stack; for (CallInfo* ci = L->base_ci; ci <= L->ci; ci++) { diff --git a/VM/src/lfunc.cpp b/VM/src/lfunc.cpp index 582d4627..66447a95 100644 --- a/VM/src/lfunc.cpp +++ b/VM/src/lfunc.cpp @@ -6,13 +6,10 @@ #include "lmem.h" #include "lgc.h" -LUAU_FASTFLAGVARIABLE(LuauNoDirectUpvalRemoval, false) -LUAU_FASTFLAG(LuauGcPagedSweep) - Proto* luaF_newproto(lua_State* L) { Proto* f = luaM_newgco(L, Proto, sizeof(Proto), L->activememcat); - luaC_link(L, f, LUA_TPROTO); + luaC_init(L, f, LUA_TPROTO); f->k = NULL; f->sizek = 0; f->p = NULL; @@ -40,7 +37,7 @@ Proto* luaF_newproto(lua_State* L) Closure* luaF_newLclosure(lua_State* L, int nelems, Table* e, Proto* p) { Closure* c = luaM_newgco(L, Closure, sizeLclosure(nelems), L->activememcat); - luaC_link(L, c, LUA_TFUNCTION); + luaC_init(L, c, LUA_TFUNCTION); c->isC = 0; c->env = e; c->nupvalues = cast_byte(nelems); @@ -55,7 +52,7 @@ Closure* luaF_newLclosure(lua_State* L, int nelems, Table* e, Proto* p) Closure* luaF_newCclosure(lua_State* L, int nelems, Table* e) { Closure* c = luaM_newgco(L, Closure, sizeCclosure(nelems), L->activememcat); - luaC_link(L, c, LUA_TFUNCTION); + luaC_init(L, c, LUA_TFUNCTION); c->isC = 1; c->env = e; c->nupvalues = cast_byte(nelems); @@ -82,8 +79,7 @@ UpVal* luaF_findupval(lua_State* L, StkId level) return p; } - // TODO (FFlagLuauGcPagedSweep): 'next' type will change after removal of the flag and the cast will not be required - pp = (UpVal**)&p->next; + pp = &p->u.l.threadnext; } UpVal* uv = luaM_newgco(L, UpVal, sizeof(UpVal), L->activememcat); /* not found: create a new one */ @@ -94,19 +90,10 @@ UpVal* luaF_findupval(lua_State* L, StkId level) // chain the upvalue in the threads open upvalue list at the proper position UpVal* next = *pp; - - // TODO (FFlagLuauGcPagedSweep): 'next' type will change after removal of the flag and the cast will not be required - uv->next = (GCObject*)next; - - if (FFlag::LuauGcPagedSweep) - { - uv->u.l.threadprev = pp; - if (next) - { - // TODO (FFlagLuauGcPagedSweep): 'next' type will change after removal of the flag and the cast will not be required - next->u.l.threadprev = (UpVal**)&uv->next; - } - } + uv->u.l.threadnext = next; + uv->u.l.threadprev = pp; + if (next) + next->u.l.threadprev = &uv->u.l.threadnext; *pp = uv; @@ -125,15 +112,11 @@ void luaF_unlinkupval(UpVal* uv) uv->u.l.next->u.l.prev = uv->u.l.prev; uv->u.l.prev->u.l.next = uv->u.l.next; - if (FFlag::LuauGcPagedSweep) - { - // unlink upvalue from the thread open upvalue list - // TODO (FFlagLuauGcPagedSweep): 'next' type will change after removal of the flag and this and the following cast will not be required - *uv->u.l.threadprev = (UpVal*)uv->next; + // unlink upvalue from the thread open upvalue list + *uv->u.l.threadprev = uv->u.l.threadnext; - if (UpVal* next = (UpVal*)uv->next) - next->u.l.threadprev = uv->u.l.threadprev; - } + if (UpVal* next = uv->u.l.threadnext) + next->u.l.threadprev = uv->u.l.threadprev; } void luaF_freeupval(lua_State* L, UpVal* uv, lua_Page* page) @@ -145,34 +128,27 @@ void luaF_freeupval(lua_State* L, UpVal* uv, lua_Page* page) void luaF_close(lua_State* L, StkId level) { - global_State* g = L->global; // TODO: remove with FFlagLuauNoDirectUpvalRemoval + global_State* g = L->global; UpVal* uv; while (L->openupval != NULL && (uv = L->openupval)->v >= level) { GCObject* o = obj2gco(uv); LUAU_ASSERT(!isblack(o) && uv->v != &uv->u.value); - if (!FFlag::LuauGcPagedSweep) - L->openupval = (UpVal*)uv->next; /* remove from `open' list */ + // by removing the upvalue from global/thread open upvalue lists, L->openupval will be pointing to the next upvalue + luaF_unlinkupval(uv); - if (FFlag::LuauGcPagedSweep && isdead(g, o)) + if (isdead(g, o)) { - // by removing the upvalue from global/thread open upvalue lists, L->openupval will be pointing to the next upvalue - luaF_unlinkupval(uv); // close the upvalue without copying the dead data so that luaF_freeupval will not unlink again uv->v = &uv->u.value; } - else if (!FFlag::LuauNoDirectUpvalRemoval && isdead(g, o)) - { - luaF_freeupval(L, uv, NULL); /* free upvalue */ - } else { - // by removing the upvalue from global/thread open upvalue lists, L->openupval will be pointing to the next upvalue - luaF_unlinkupval(uv); setobj(L, &uv->u.value, uv->v); - uv->v = &uv->u.value; /* now current value lives here */ - luaC_linkupval(L, uv); /* link upvalue into `gcroot' list */ + uv->v = &uv->u.value; + // GC state of a new closed upvalue has to be initialized + luaC_initupval(L, uv); } } } diff --git a/VM/src/lgc.cpp b/VM/src/lgc.cpp index 724b24b2..8c3a2029 100644 --- a/VM/src/lgc.cpp +++ b/VM/src/lgc.cpp @@ -13,8 +13,6 @@ #include -LUAU_FASTFLAGVARIABLE(LuauGcPagedSweep, false) - #define GC_SWEEPMAX 40 #define GC_SWEEPCOST 10 #define GC_SWEEPPAGESTEPCOST 4 @@ -64,7 +62,6 @@ static void recordGcStateTime(global_State* g, int startgcstate, double seconds, case GCSatomic: g->gcstats.currcycle.atomictime += seconds; break; - case GCSsweepstring: case GCSsweep: g->gcstats.currcycle.sweeptime += seconds; break; @@ -490,65 +487,6 @@ static void freeobj(lua_State* L, GCObject* o, lua_Page* page) } } -#define sweepwholelist(L, p) sweeplist(L, p, SIZE_MAX) - -static GCObject** sweeplist(lua_State* L, GCObject** p, size_t count) -{ - LUAU_ASSERT(!FFlag::LuauGcPagedSweep); - - GCObject* curr; - global_State* g = L->global; - int deadmask = otherwhite(g); - LUAU_ASSERT(testbit(deadmask, FIXEDBIT)); /* make sure we never sweep fixed objects */ - while ((curr = *p) != NULL && count-- > 0) - { - int alive = (curr->gch.marked ^ WHITEBITS) & deadmask; - if (curr->gch.tt == LUA_TTHREAD) - { - sweepwholelist(L, (GCObject**)&gco2th(curr)->openupval); /* sweep open upvalues */ - - lua_State* th = gco2th(curr); - - if (alive) - { - resetbit(th->stackstate, THREAD_SLEEPINGBIT); - shrinkstack(th); - } - } - if (alive) - { /* not dead? */ - LUAU_ASSERT(!isdead(g, curr)); - makewhite(g, curr); /* make it white (for next cycle) */ - p = &curr->gch.next; - } - else - { /* must erase `curr' */ - LUAU_ASSERT(isdead(g, curr)); - *p = curr->gch.next; - if (curr == g->rootgc) /* is the first element of the list? */ - g->rootgc = curr->gch.next; /* adjust first */ - freeobj(L, curr, NULL); - } - } - - return p; -} - -static void deletelist(lua_State* L, GCObject** p, GCObject* limit) -{ - LUAU_ASSERT(!FFlag::LuauGcPagedSweep); - - GCObject* curr; - while ((curr = *p) != limit) - { - if (curr->gch.tt == LUA_TTHREAD) /* delete open upvalues of each thread */ - deletelist(L, (GCObject**)&gco2th(curr)->openupval, NULL); - - *p = curr->gch.next; - freeobj(L, curr, NULL); - } -} - static void shrinkbuffers(lua_State* L) { global_State* g = L->global; @@ -570,8 +508,6 @@ static void shrinkbuffersfull(lua_State* L) static bool deletegco(void* context, lua_Page* page, GCObject* gco) { - LUAU_ASSERT(FFlag::LuauGcPagedSweep); - // we are in the process of deleting everything // threads with open upvalues will attempt to close them all on removal // but those upvalues might point to stack values that were already deleted @@ -598,32 +534,13 @@ void luaC_freeall(lua_State* L) LUAU_ASSERT(L == g->mainthread); - if (FFlag::LuauGcPagedSweep) - { - luaM_visitgco(L, L, deletegco); + luaM_visitgco(L, L, deletegco); - for (int i = 0; i < g->strt.size; i++) /* free all string lists */ - LUAU_ASSERT(g->strt.hash[i] == NULL); + for (int i = 0; i < g->strt.size; i++) /* free all string lists */ + LUAU_ASSERT(g->strt.hash[i] == NULL); - LUAU_ASSERT(L->global->strt.nuse == 0); - LUAU_ASSERT(g->strbufgc == NULL); - } - else - { - LUAU_ASSERT(L->next == NULL); /* mainthread is at the end of rootgc list */ - - deletelist(L, &g->rootgc, obj2gco(L)); - - for (int i = 0; i < g->strt.size; i++) /* free all string lists */ - deletelist(L, (GCObject**)&g->strt.hash[i], NULL); - - LUAU_ASSERT(L->global->strt.nuse == 0); - deletelist(L, (GCObject**)&g->strbufgc, NULL); - - // unfortunately, when string objects are freed, the string table use count is decremented - // even when the string is a buffer that wasn't placed into the table - L->global->strt.nuse = 0; - } + LUAU_ASSERT(L->global->strt.nuse == 0); + LUAU_ASSERT(g->strbufgc == NULL); } static void markmt(global_State* g) @@ -687,26 +604,13 @@ static size_t atomic(lua_State* L) g->weak = NULL; /* flip current white */ g->currentwhite = cast_byte(otherwhite(g)); - g->sweepstrgc = 0; - - if (FFlag::LuauGcPagedSweep) - { - g->sweepgcopage = g->allgcopages; - g->gcstate = GCSsweep; - } - else - { - g->sweepgc = &g->rootgc; - g->gcstate = GCSsweepstring; - } - + g->sweepgcopage = g->allgcopages; + g->gcstate = GCSsweep; return work; } static bool sweepgco(lua_State* L, lua_Page* page, GCObject* gco) { - LUAU_ASSERT(FFlag::LuauGcPagedSweep); - global_State* g = L->global; int deadmask = otherwhite(g); @@ -740,8 +644,6 @@ static bool sweepgco(lua_State* L, lua_Page* page, GCObject* gco) // a version of generic luaM_visitpage specialized for the main sweep stage static int sweepgcopage(lua_State* L, lua_Page* page) { - LUAU_ASSERT(FFlag::LuauGcPagedSweep); - char* start; char* end; int busyBlocks; @@ -819,75 +721,29 @@ static size_t gcstep(lua_State* L, size_t limit) cost = atomic(L); /* finish mark phase */ - if (FFlag::LuauGcPagedSweep) - LUAU_ASSERT(g->gcstate == GCSsweep); - else - LUAU_ASSERT(g->gcstate == GCSsweepstring); - break; - } - case GCSsweepstring: - { - LUAU_ASSERT(!FFlag::LuauGcPagedSweep); - - while (g->sweepstrgc < g->strt.size && cost < limit) - { - sweepwholelist(L, (GCObject**)&g->strt.hash[g->sweepstrgc++]); - - cost += GC_SWEEPCOST; - } - - // nothing more to sweep? - if (g->sweepstrgc >= g->strt.size) - { - // sweep string buffer list and preserve used string count - uint32_t nuse = L->global->strt.nuse; - - sweepwholelist(L, (GCObject**)&g->strbufgc); - - L->global->strt.nuse = nuse; - - g->gcstate = GCSsweep; // end sweep-string phase - } + LUAU_ASSERT(g->gcstate == GCSsweep); break; } case GCSsweep: { - if (FFlag::LuauGcPagedSweep) + while (g->sweepgcopage && cost < limit) { - while (g->sweepgcopage && cost < limit) - { - lua_Page* next = luaM_getnextgcopage(g->sweepgcopage); // page sweep might destroy the page + lua_Page* next = luaM_getnextgcopage(g->sweepgcopage); // page sweep might destroy the page - int steps = sweepgcopage(L, g->sweepgcopage); + int steps = sweepgcopage(L, g->sweepgcopage); - g->sweepgcopage = next; - cost += steps * GC_SWEEPPAGESTEPCOST; - } - - // nothing more to sweep? - if (g->sweepgcopage == NULL) - { - // don't forget to visit main thread - sweepgco(L, NULL, obj2gco(g->mainthread)); - - shrinkbuffers(L); - g->gcstate = GCSpause; /* end collection */ - } + g->sweepgcopage = next; + cost += steps * GC_SWEEPPAGESTEPCOST; } - else + + // nothing more to sweep? + if (g->sweepgcopage == NULL) { - while (*g->sweepgc && cost < limit) - { - g->sweepgc = sweeplist(L, g->sweepgc, GC_SWEEPMAX); + // don't forget to visit main thread + sweepgco(L, NULL, obj2gco(g->mainthread)); - cost += GC_SWEEPMAX * GC_SWEEPCOST; - } - - if (*g->sweepgc == NULL) - { /* nothing more to sweep? */ - shrinkbuffers(L); - g->gcstate = GCSpause; /* end collection */ - } + shrinkbuffers(L); + g->gcstate = GCSpause; /* end collection */ } break; } @@ -1013,26 +869,18 @@ void luaC_fullgc(lua_State* L) if (g->gcstate <= GCSatomic) { /* reset sweep marks to sweep all elements (returning them to white) */ - g->sweepstrgc = 0; - if (FFlag::LuauGcPagedSweep) - g->sweepgcopage = g->allgcopages; - else - g->sweepgc = &g->rootgc; + g->sweepgcopage = g->allgcopages; /* reset other collector lists */ g->gray = NULL; g->grayagain = NULL; g->weak = NULL; - - if (FFlag::LuauGcPagedSweep) - g->gcstate = GCSsweep; - else - g->gcstate = GCSsweepstring; + g->gcstate = GCSsweep; } - LUAU_ASSERT(g->gcstate == GCSsweepstring || g->gcstate == GCSsweep); + LUAU_ASSERT(g->gcstate == GCSsweep); /* finish any pending sweep phase */ while (g->gcstate != GCSpause) { - LUAU_ASSERT(g->gcstate == GCSsweepstring || g->gcstate == GCSsweep); + LUAU_ASSERT(g->gcstate == GCSsweep); gcstep(L, SIZE_MAX); } @@ -1120,30 +968,19 @@ void luaC_barrierback(lua_State* L, Table* t) g->grayagain = o; } -void luaC_linkobj(lua_State* L, GCObject* o, uint8_t tt) +void luaC_initobj(lua_State* L, GCObject* o, uint8_t tt) { global_State* g = L->global; - if (!FFlag::LuauGcPagedSweep) - { - o->gch.next = g->rootgc; - g->rootgc = o; - } o->gch.marked = luaC_white(g); o->gch.tt = tt; o->gch.memcat = L->activememcat; } -void luaC_linkupval(lua_State* L, UpVal* uv) +void luaC_initupval(lua_State* L, UpVal* uv) { global_State* g = L->global; GCObject* o = obj2gco(uv); - if (!FFlag::LuauGcPagedSweep) - { - o->gch.next = g->rootgc; /* link upvalue into `rootgc' list */ - g->rootgc = o; - } - if (isgray(o)) { if (keepinvariant(g)) @@ -1221,9 +1058,6 @@ const char* luaC_statename(int state) case GCSatomic: return "atomic"; - case GCSsweepstring: - return "sweepstring"; - case GCSsweep: return "sweep"; diff --git a/VM/src/lgc.h b/VM/src/lgc.h index 2acb5d8a..253e269f 100644 --- a/VM/src/lgc.h +++ b/VM/src/lgc.h @@ -13,9 +13,7 @@ #define GCSpropagate 1 #define GCSpropagateagain 2 #define GCSatomic 3 -// TODO: remove with FFlagLuauGcPagedSweep -#define GCSsweepstring 4 -#define GCSsweep 5 +#define GCSsweep 4 /* ** macro to tell when main invariant (white objects cannot point to black @@ -132,13 +130,13 @@ luaC_wakethread(L); \ } -#define luaC_link(L, o, tt) luaC_linkobj(L, cast_to(GCObject*, (o)), tt) +#define luaC_init(L, o, tt) luaC_initobj(L, cast_to(GCObject*, (o)), tt) LUAI_FUNC void luaC_freeall(lua_State* L); LUAI_FUNC void luaC_step(lua_State* L, bool assist); LUAI_FUNC void luaC_fullgc(lua_State* L); -LUAI_FUNC void luaC_linkobj(lua_State* L, GCObject* o, uint8_t tt); -LUAI_FUNC void luaC_linkupval(lua_State* L, UpVal* uv); +LUAI_FUNC void luaC_initobj(lua_State* L, GCObject* o, uint8_t tt); +LUAI_FUNC void luaC_initupval(lua_State* L, UpVal* uv); LUAI_FUNC void luaC_barrierupval(lua_State* L, GCObject* v); LUAI_FUNC void luaC_barrierf(lua_State* L, GCObject* o, GCObject* v); LUAI_FUNC void luaC_barriertable(lua_State* L, Table* t, GCObject* v); diff --git a/VM/src/lgcdebug.cpp b/VM/src/lgcdebug.cpp index 30242e52..2b38619b 100644 --- a/VM/src/lgcdebug.cpp +++ b/VM/src/lgcdebug.cpp @@ -13,8 +13,6 @@ #include #include -LUAU_FASTFLAG(LuauGcPagedSweep) - static void validateobjref(global_State* g, GCObject* f, GCObject* t) { LUAU_ASSERT(!isdead(g, t)); @@ -104,8 +102,7 @@ static void validatestack(global_State* g, lua_State* l) if (l->namecall) validateobjref(g, obj2gco(l), obj2gco(l->namecall)); - // TODO (FFlagLuauGcPagedSweep): 'next' type will change after removal of the flag and the cast will not be required - for (UpVal* uv = l->openupval; uv; uv = (UpVal*)uv->next) + for (UpVal* uv = l->openupval; uv; uv = uv->u.l.threadnext) { LUAU_ASSERT(uv->tt == LUA_TUPVAL); LUAU_ASSERT(uv->v != &uv->u.value); @@ -141,7 +138,7 @@ static void validateobj(global_State* g, GCObject* o) /* dead objects can only occur during sweep */ if (isdead(g, o)) { - LUAU_ASSERT(g->gcstate == GCSsweepstring || g->gcstate == GCSsweep); + LUAU_ASSERT(g->gcstate == GCSsweep); return; } @@ -180,18 +177,6 @@ static void validateobj(global_State* g, GCObject* o) } } -static void validatelist(global_State* g, GCObject* o) -{ - LUAU_ASSERT(!FFlag::LuauGcPagedSweep); - - while (o) - { - validateobj(g, o); - - o = o->gch.next; - } -} - static void validategraylist(global_State* g, GCObject* o) { if (!keepinvariant(g)) @@ -224,8 +209,6 @@ static void validategraylist(global_State* g, GCObject* o) static bool validategco(void* context, lua_Page* page, GCObject* gco) { - LUAU_ASSERT(FFlag::LuauGcPagedSweep); - lua_State* L = (lua_State*)context; global_State* g = L->global; @@ -248,20 +231,9 @@ void luaC_validate(lua_State* L) validategraylist(g, g->gray); validategraylist(g, g->grayagain); - if (FFlag::LuauGcPagedSweep) - { - validategco(L, NULL, obj2gco(g->mainthread)); + validategco(L, NULL, obj2gco(g->mainthread)); - luaM_visitgco(L, L, validategco); - } - else - { - for (int i = 0; i < g->strt.size; ++i) - validatelist(g, (GCObject*)(g->strt.hash[i])); - - validatelist(g, g->rootgc); - validatelist(g, (GCObject*)(g->strbufgc)); - } + luaM_visitgco(L, L, validategco); for (UpVal* uv = g->uvhead.u.l.next; uv != &g->uvhead; uv = uv->u.l.next) { @@ -521,30 +493,8 @@ static void dumpobj(FILE* f, GCObject* o) } } -static void dumplist(FILE* f, GCObject* o) -{ - LUAU_ASSERT(!FFlag::LuauGcPagedSweep); - - while (o) - { - dumpref(f, o); - fputc(':', f); - dumpobj(f, o); - fputc(',', f); - fputc('\n', f); - - // thread has additional list containing collectable objects that are not present in rootgc - if (o->gch.tt == LUA_TTHREAD) - dumplist(f, (GCObject*)gco2th(o)->openupval); - - o = o->gch.next; - } -} - static bool dumpgco(void* context, lua_Page* page, GCObject* gco) { - LUAU_ASSERT(FFlag::LuauGcPagedSweep); - FILE* f = (FILE*)context; dumpref(f, gco); @@ -563,19 +513,9 @@ void luaC_dump(lua_State* L, void* file, const char* (*categoryName)(lua_State* fprintf(f, "{\"objects\":{\n"); - if (FFlag::LuauGcPagedSweep) - { - dumpgco(f, NULL, obj2gco(g->mainthread)); + dumpgco(f, NULL, obj2gco(g->mainthread)); - luaM_visitgco(L, f, dumpgco); - } - else - { - dumplist(f, g->rootgc); - dumplist(f, (GCObject*)(g->strbufgc)); - for (int i = 0; i < g->strt.size; ++i) - dumplist(f, (GCObject*)(g->strt.hash[i])); - } + luaM_visitgco(L, f, dumpgco); fprintf(f, "\"0\":{\"type\":\"userdata\",\"cat\":0,\"size\":0}\n"); // to avoid issues with trailing , fprintf(f, "},\"roots\":{\n"); diff --git a/VM/src/linit.cpp b/VM/src/linit.cpp index c93f431f..fd95f596 100644 --- a/VM/src/linit.cpp +++ b/VM/src/linit.cpp @@ -68,7 +68,7 @@ void luaL_sandboxthread(lua_State* L) lua_setsafeenv(L, LUA_GLOBALSINDEX, true); } -static void* l_alloc(lua_State* L, void* ud, void* ptr, size_t osize, size_t nsize) +static void* l_alloc(void* ud, void* ptr, size_t osize, size_t nsize) { (void)ud; (void)osize; diff --git a/VM/src/lmem.cpp b/VM/src/lmem.cpp index 19617b8c..899cb0c0 100644 --- a/VM/src/lmem.cpp +++ b/VM/src/lmem.cpp @@ -78,8 +78,6 @@ * allocated pages. */ -LUAU_FASTFLAG(LuauGcPagedSweep) - #ifndef __has_feature #define __has_feature(x) 0 #endif @@ -98,8 +96,10 @@ LUAU_FASTFLAG(LuauGcPagedSweep) * To prevent some of them accidentally growing and us losing memory without realizing it, we're going to lock * the sizes of all critical structures down. */ -#if defined(__APPLE__) && !defined(__MACH__) +#if defined(__APPLE__) #define ABISWITCH(x64, ms32, gcc32) (sizeof(void*) == 8 ? x64 : gcc32) +#elif defined(__i386__) +#define ABISWITCH(x64, ms32, gcc32) (gcc32) #else // Android somehow uses a similar ABI to MSVC, *not* to iOS... #define ABISWITCH(x64, ms32, gcc32) (sizeof(void*) == 8 ? x64 : ms32) @@ -114,14 +114,8 @@ static_assert(sizeof(LuaNode) == ABISWITCH(32, 32, 32), "size mismatch for table #endif static_assert(offsetof(TString, data) == ABISWITCH(24, 20, 20), "size mismatch for string header"); -// TODO (FFlagLuauGcPagedSweep): this will become ABISWITCH(16, 16, 16) -static_assert(offsetof(Udata, data) == ABISWITCH(24, 16, 16), "size mismatch for userdata header"); -// TODO (FFlagLuauGcPagedSweep): this will become ABISWITCH(48, 32, 32) -static_assert(sizeof(Table) == ABISWITCH(56, 36, 36), "size mismatch for table header"); - -// TODO (FFlagLuauGcPagedSweep): new code with old 'next' pointer requires that GCObject start at the same point as TString/UpVal -static_assert(offsetof(GCObject, uv) == 0, "UpVal data must be located at the start of the GCObject"); -static_assert(offsetof(GCObject, ts) == 0, "TString data must be located at the start of the GCObject"); +static_assert(offsetof(Udata, data) == ABISWITCH(16, 16, 12), "size mismatch for userdata header"); +static_assert(sizeof(Table) == ABISWITCH(48, 32, 32), "size mismatch for table header"); const size_t kSizeClasses = LUA_SIZECLASSES; const size_t kMaxSmallSize = 512; @@ -208,53 +202,13 @@ l_noret luaM_toobig(lua_State* L) luaG_runerror(L, "memory allocation error: block too big"); } -static lua_Page* newpageold(lua_State* L, uint8_t sizeClass) -{ - LUAU_ASSERT(!FFlag::LuauGcPagedSweep); - - global_State* g = L->global; - lua_Page* page = (lua_Page*)(*g->frealloc)(L, g->ud, NULL, 0, kPageSize); - if (!page) - luaD_throw(L, LUA_ERRMEM); - - int blockSize = kSizeClassConfig.sizeOfClass[sizeClass] + kBlockHeader; - int blockCount = (kPageSize - offsetof(lua_Page, data)) / blockSize; - - ASAN_POISON_MEMORY_REGION(page->data, blockSize * blockCount); - - // setup page header - page->prev = NULL; - page->next = NULL; - - page->gcolistprev = NULL; - page->gcolistnext = NULL; - - page->pageSize = kPageSize; - page->blockSize = blockSize; - - // note: we start with the last block in the page and move downward - // either order would work, but that way we don't need to store the block count in the page - // additionally, GC stores objects in singly linked lists, and this way the GC lists end up in increasing pointer order - page->freeList = NULL; - page->freeNext = (blockCount - 1) * blockSize; - page->busyBlocks = 0; - - // prepend a page to page freelist (which is empty because we only ever allocate a new page when it is!) - LUAU_ASSERT(!g->freepages[sizeClass]); - g->freepages[sizeClass] = page; - - return page; -} - static lua_Page* newpage(lua_State* L, lua_Page** gcopageset, int pageSize, int blockSize, int blockCount) { - LUAU_ASSERT(FFlag::LuauGcPagedSweep); - global_State* g = L->global; LUAU_ASSERT(pageSize - int(offsetof(lua_Page, data)) >= blockSize * blockCount); - lua_Page* page = (lua_Page*)(*g->frealloc)(L, g->ud, NULL, 0, pageSize); + lua_Page* page = (lua_Page*)(*g->frealloc)(g->ud, NULL, 0, pageSize); if (!page) luaD_throw(L, LUA_ERRMEM); @@ -290,8 +244,6 @@ static lua_Page* newpage(lua_State* L, lua_Page** gcopageset, int pageSize, int static lua_Page* newclasspage(lua_State* L, lua_Page** freepageset, lua_Page** gcopageset, uint8_t sizeClass, bool storeMetadata) { - LUAU_ASSERT(FFlag::LuauGcPagedSweep); - int blockSize = kSizeClassConfig.sizeOfClass[sizeClass] + (storeMetadata ? kBlockHeader : 0); int blockCount = (kPageSize - offsetof(lua_Page, data)) / blockSize; @@ -304,29 +256,8 @@ static lua_Page* newclasspage(lua_State* L, lua_Page** freepageset, lua_Page** g return page; } -static void freepageold(lua_State* L, lua_Page* page, uint8_t sizeClass) -{ - LUAU_ASSERT(!FFlag::LuauGcPagedSweep); - - global_State* g = L->global; - - // remove page from freelist - if (page->next) - page->next->prev = page->prev; - - if (page->prev) - page->prev->next = page->next; - else if (g->freepages[sizeClass] == page) - g->freepages[sizeClass] = page->next; - - // so long - (*g->frealloc)(L, g->ud, page, kPageSize, 0); -} - static void freepage(lua_State* L, lua_Page** gcopageset, lua_Page* page) { - LUAU_ASSERT(FFlag::LuauGcPagedSweep); - global_State* g = L->global; if (gcopageset) @@ -342,13 +273,11 @@ static void freepage(lua_State* L, lua_Page** gcopageset, lua_Page* page) } // so long - (*g->frealloc)(L, g->ud, page, page->pageSize, 0); + (*g->frealloc)(g->ud, page, page->pageSize, 0); } static void freeclasspage(lua_State* L, lua_Page** freepageset, lua_Page** gcopageset, lua_Page* page, uint8_t sizeClass) { - LUAU_ASSERT(FFlag::LuauGcPagedSweep); - // remove page from freelist if (page->next) page->next->prev = page->prev; @@ -368,12 +297,7 @@ static void* newblock(lua_State* L, int sizeClass) // slow path: no page in the freelist, allocate a new one if (!page) - { - if (FFlag::LuauGcPagedSweep) - page = newclasspage(L, g->freepages, NULL, sizeClass, true); - else - page = newpageold(L, sizeClass); - } + page = newclasspage(L, g->freepages, NULL, sizeClass, true); LUAU_ASSERT(!page->prev); LUAU_ASSERT(page->freeList || page->freeNext >= 0); @@ -416,8 +340,6 @@ static void* newblock(lua_State* L, int sizeClass) static void* newgcoblock(lua_State* L, int sizeClass) { - LUAU_ASSERT(FFlag::LuauGcPagedSweep); - global_State* g = L->global; lua_Page* page = g->freegcopages[sizeClass]; @@ -496,17 +418,11 @@ static void freeblock(lua_State* L, int sizeClass, void* block) // if it's the last block in the page, we don't need the page if (page->busyBlocks == 0) - { - if (FFlag::LuauGcPagedSweep) - freeclasspage(L, g->freepages, NULL, page, sizeClass); - else - freepageold(L, page, sizeClass); - } + freeclasspage(L, g->freepages, NULL, page, sizeClass); } static void freegcoblock(lua_State* L, int sizeClass, void* block, lua_Page* page) { - LUAU_ASSERT(FFlag::LuauGcPagedSweep); LUAU_ASSERT(page && page->busyBlocks > 0); LUAU_ASSERT(page->blockSize == kSizeClassConfig.sizeOfClass[sizeClass]); LUAU_ASSERT(block >= page->data && block < (char*)page + page->pageSize); @@ -544,7 +460,7 @@ void* luaM_new_(lua_State* L, size_t nsize, uint8_t memcat) int nclass = sizeclass(nsize); - void* block = nclass >= 0 ? newblock(L, nclass) : (*g->frealloc)(L, g->ud, NULL, 0, nsize); + void* block = nclass >= 0 ? newblock(L, nclass) : (*g->frealloc)(g->ud, NULL, 0, nsize); if (block == NULL && nsize > 0) luaD_throw(L, LUA_ERRMEM); @@ -556,9 +472,6 @@ void* luaM_new_(lua_State* L, size_t nsize, uint8_t memcat) GCObject* luaM_newgco_(lua_State* L, size_t nsize, uint8_t memcat) { - if (!FFlag::LuauGcPagedSweep) - return (GCObject*)luaM_new_(L, nsize, memcat); - // we need to accommodate space for link for free blocks (freegcolink) LUAU_ASSERT(nsize >= kGCOLinkOffset + sizeof(void*)); @@ -602,7 +515,7 @@ void luaM_free_(lua_State* L, void* block, size_t osize, uint8_t memcat) if (oclass >= 0) freeblock(L, oclass, block); else - (*g->frealloc)(L, g->ud, block, osize, 0); + (*g->frealloc)(g->ud, block, osize, 0); g->totalbytes -= osize; g->memcatbytes[memcat] -= osize; @@ -610,12 +523,6 @@ void luaM_free_(lua_State* L, void* block, size_t osize, uint8_t memcat) void luaM_freegco_(lua_State* L, GCObject* block, size_t osize, uint8_t memcat, lua_Page* page) { - if (!FFlag::LuauGcPagedSweep) - { - luaM_free_(L, block, osize, memcat); - return; - } - global_State* g = L->global; LUAU_ASSERT((osize == 0) == (block == NULL)); @@ -652,7 +559,7 @@ void* luaM_realloc_(lua_State* L, void* block, size_t osize, size_t nsize, uint8 // if either block needs to be allocated using a block allocator, we can't use realloc directly if (nclass >= 0 || oclass >= 0) { - result = nclass >= 0 ? newblock(L, nclass) : (*g->frealloc)(L, g->ud, NULL, 0, nsize); + result = nclass >= 0 ? newblock(L, nclass) : (*g->frealloc)(g->ud, NULL, 0, nsize); if (result == NULL && nsize > 0) luaD_throw(L, LUA_ERRMEM); @@ -662,11 +569,11 @@ void* luaM_realloc_(lua_State* L, void* block, size_t osize, size_t nsize, uint8 if (oclass >= 0) freeblock(L, oclass, block); else - (*g->frealloc)(L, g->ud, block, osize, 0); + (*g->frealloc)(g->ud, block, osize, 0); } else { - result = (*g->frealloc)(L, g->ud, block, osize, nsize); + result = (*g->frealloc)(g->ud, block, osize, nsize); if (result == NULL && nsize > 0) luaD_throw(L, LUA_ERRMEM); } @@ -679,8 +586,6 @@ void* luaM_realloc_(lua_State* L, void* block, size_t osize, size_t nsize, uint8 void luaM_getpagewalkinfo(lua_Page* page, char** start, char** end, int* busyBlocks, int* blockSize) { - LUAU_ASSERT(FFlag::LuauGcPagedSweep); - int blockCount = (page->pageSize - offsetof(lua_Page, data)) / page->blockSize; LUAU_ASSERT(page->freeNext >= -page->blockSize && page->freeNext <= (blockCount - 1) * page->blockSize); @@ -700,8 +605,6 @@ lua_Page* luaM_getnextgcopage(lua_Page* page) void luaM_visitpage(lua_Page* page, void* context, bool (*visitor)(void* context, lua_Page* page, GCObject* gco)) { - LUAU_ASSERT(FFlag::LuauGcPagedSweep); - char* start; char* end; int busyBlocks; @@ -730,8 +633,6 @@ void luaM_visitpage(lua_Page* page, void* context, bool (*visitor)(void* context void luaM_visitgco(lua_State* L, void* context, bool (*visitor)(void* context, lua_Page* page, GCObject* gco)) { - LUAU_ASSERT(FFlag::LuauGcPagedSweep); - global_State* g = L->global; for (lua_Page* curr = g->allgcopages; curr;) diff --git a/VM/src/lmem.h b/VM/src/lmem.h index 1bfe48fa..00788452 100644 --- a/VM/src/lmem.h +++ b/VM/src/lmem.h @@ -7,11 +7,7 @@ struct lua_Page; union GCObject; -// TODO: remove with FFlagLuauGcPagedSweep and rename luaM_newgco to luaM_new -#define luaM_new(L, t, size, memcat) cast_to(t*, luaM_new_(L, size, memcat)) #define luaM_newgco(L, t, size, memcat) cast_to(t*, luaM_newgco_(L, size, memcat)) -// TODO: remove with FFlagLuauGcPagedSweep and rename luaM_freegco to luaM_free -#define luaM_free(L, p, size, memcat) luaM_free_(L, (p), size, memcat) #define luaM_freegco(L, p, size, memcat, page) luaM_freegco_(L, obj2gco(p), size, memcat, page) #define luaM_arraysize_(n, e) ((cast_to(size_t, (n)) <= SIZE_MAX / (e)) ? (n) * (e) : (luaM_toobig(L), SIZE_MAX)) diff --git a/VM/src/lobject.h b/VM/src/lobject.h index 57ffd82a..5e02c2ea 100644 --- a/VM/src/lobject.h +++ b/VM/src/lobject.h @@ -15,7 +15,6 @@ typedef union GCObject GCObject; */ // clang-format off #define CommonHeader \ - GCObject* next; /* TODO: remove with FFlagLuauGcPagedSweep */ \ uint8_t tt; uint8_t marked; uint8_t memcat // clang-format on @@ -233,6 +232,8 @@ typedef struct TString int16_t atom; // 2 byte padding + TString* next; // next string in the hash table bucket or the string buffer linked list + unsigned int hash; unsigned int len; @@ -327,7 +328,7 @@ typedef struct UpVal struct UpVal* next; /* thread double linked list (when open) */ - // TODO: when FFlagLuauGcPagedSweep is removed, old outer 'next' value will be placed here + struct UpVal* threadnext; /* note: this is the location of a pointer to this upvalue in the previous element that can be either an UpVal or a lua_State */ struct UpVal** threadprev; } l; diff --git a/VM/src/lstate.cpp b/VM/src/lstate.cpp index d6d127c0..d4f3f0a1 100644 --- a/VM/src/lstate.cpp +++ b/VM/src/lstate.cpp @@ -10,7 +10,6 @@ #include "ldo.h" #include "ldebug.h" -LUAU_FASTFLAG(LuauGcPagedSweep) LUAU_FASTFLAGVARIABLE(LuauReduceStackReallocs, false) /* @@ -90,8 +89,6 @@ static void close_state(lua_State* L) global_State* g = L->global; luaF_close(L, L->stack); /* close all upvalues for this thread */ luaC_freeall(L); /* collect all objects */ - if (!FFlag::LuauGcPagedSweep) - LUAU_ASSERT(g->rootgc == obj2gco(L)); LUAU_ASSERT(g->strbufgc == NULL); LUAU_ASSERT(g->strt.nuse == 0); luaM_freearray(L, L->global->strt.hash, L->global->strt.size, TString*, 0); @@ -99,22 +96,20 @@ static void close_state(lua_State* L) for (int i = 0; i < LUA_SIZECLASSES; i++) { LUAU_ASSERT(g->freepages[i] == NULL); - if (FFlag::LuauGcPagedSweep) - LUAU_ASSERT(g->freegcopages[i] == NULL); + LUAU_ASSERT(g->freegcopages[i] == NULL); } - if (FFlag::LuauGcPagedSweep) - LUAU_ASSERT(g->allgcopages == NULL); + LUAU_ASSERT(g->allgcopages == NULL); LUAU_ASSERT(g->totalbytes == sizeof(LG)); LUAU_ASSERT(g->memcatbytes[0] == sizeof(LG)); for (int i = 1; i < LUA_MEMORY_CATEGORIES; i++) LUAU_ASSERT(g->memcatbytes[i] == 0); - (*g->frealloc)(L, g->ud, L, sizeof(LG), 0); + (*g->frealloc)(g->ud, L, sizeof(LG), 0); } lua_State* luaE_newthread(lua_State* L) { lua_State* L1 = luaM_newgco(L, lua_State, sizeof(lua_State), L->activememcat); - luaC_link(L, L1, LUA_TTHREAD); + luaC_init(L, L1, LUA_TTHREAD); preinit_state(L1, L->global); L1->activememcat = L->activememcat; // inherit the active memory category stack_init(L1, L); /* init stack */ @@ -184,13 +179,11 @@ lua_State* lua_newstate(lua_Alloc f, void* ud) int i; lua_State* L; global_State* g; - void* l = (*f)(NULL, ud, NULL, 0, sizeof(LG)); + void* l = (*f)(ud, NULL, 0, sizeof(LG)); if (l == NULL) return NULL; L = (lua_State*)l; g = &((LG*)L)->g; - if (!FFlag::LuauGcPagedSweep) - L->next = NULL; L->tt = LUA_TTHREAD; L->marked = g->currentwhite = bit2mask(WHITE0BIT, FIXEDBIT); L->memcat = 0; @@ -214,11 +207,6 @@ lua_State* lua_newstate(lua_Alloc f, void* ud) setnilvalue(&g->pseudotemp); setnilvalue(registry(L)); g->gcstate = GCSpause; - if (!FFlag::LuauGcPagedSweep) - g->rootgc = obj2gco(L); - g->sweepstrgc = 0; - if (!FFlag::LuauGcPagedSweep) - g->sweepgc = &g->rootgc; g->gray = NULL; g->grayagain = NULL; g->weak = NULL; @@ -230,14 +218,10 @@ lua_State* lua_newstate(lua_Alloc f, void* ud) for (i = 0; i < LUA_SIZECLASSES; i++) { g->freepages[i] = NULL; - if (FFlag::LuauGcPagedSweep) - g->freegcopages[i] = NULL; - } - if (FFlag::LuauGcPagedSweep) - { - g->allgcopages = NULL; - g->sweepgcopage = NULL; + g->freegcopages[i] = NULL; } + g->allgcopages = NULL; + g->sweepgcopage = NULL; for (i = 0; i < LUA_T_COUNT; i++) g->mt[i] = NULL; for (i = 0; i < LUA_UTAG_LIMIT; i++) diff --git a/VM/src/lstate.h b/VM/src/lstate.h index 6dd89138..3ee96718 100644 --- a/VM/src/lstate.h +++ b/VM/src/lstate.h @@ -142,11 +142,6 @@ typedef struct global_State uint8_t gcstate; /* state of garbage collector */ - int sweepstrgc; /* position of sweep in `strt' */ - // TODO: remove with FFlagLuauGcPagedSweep - GCObject* rootgc; /* list of all collectable objects */ - // TODO: remove with FFlagLuauGcPagedSweep - GCObject** sweepgc; /* position of sweep in `rootgc' */ GCObject* gray; /* list of gray objects */ GCObject* grayagain; /* list of objects to be traversed atomically */ GCObject* weak; /* list of weak tables (to be cleared) */ diff --git a/VM/src/lstring.cpp b/VM/src/lstring.cpp index 9bbc43de..87250146 100644 --- a/VM/src/lstring.cpp +++ b/VM/src/lstring.cpp @@ -7,8 +7,6 @@ #include -LUAU_FASTFLAG(LuauGcPagedSweep) - unsigned int luaS_hash(const char* str, size_t len) { // Note that this hashing algorithm is replicated in BytecodeBuilder.cpp, BytecodeBuilder::getStringHash @@ -46,8 +44,6 @@ unsigned int luaS_hash(const char* str, size_t len) void luaS_resize(lua_State* L, int newsize) { - if (L->global->gcstate == GCSsweepstring) - return; /* cannot resize during GC traverse */ TString** newhash = luaM_newarray(L, newsize, TString*, 0); stringtable* tb = &L->global->strt; for (int i = 0; i < newsize; i++) @@ -58,13 +54,11 @@ void luaS_resize(lua_State* L, int newsize) TString* p = tb->hash[i]; while (p) { /* for each node in the list */ - // TODO (FFlagLuauGcPagedSweep): 'next' type will change after removal of the flag and the cast will not be required - TString* next = (TString*)p->next; /* save next */ + TString* next = p->next; /* save next */ unsigned int h = p->hash; int h1 = lmod(h, newsize); /* new position */ LUAU_ASSERT(cast_int(h % newsize) == lmod(h, newsize)); - // TODO (FFlagLuauGcPagedSweep): 'next' type will change after removal of the flag and the cast will not be required - p->next = (GCObject*)newhash[h1]; /* chain it */ + p->next = newhash[h1]; /* chain it */ newhash[h1] = p; p = next; } @@ -91,8 +85,7 @@ static TString* newlstr(lua_State* L, const char* str, size_t l, unsigned int h) ts->atom = L->global->cb.useratom ? L->global->cb.useratom(ts->data, l) : -1; tb = &L->global->strt; h = lmod(h, tb->size); - // TODO (FFlagLuauGcPagedSweep): 'next' type will change after removal of the flag and the case will not be required - ts->next = (GCObject*)tb->hash[h]; /* chain new entry */ + ts->next = tb->hash[h]; /* chain new entry */ tb->hash[h] = ts; tb->nuse++; if (tb->nuse > cast_to(uint32_t, tb->size) && tb->size <= INT_MAX / 2) @@ -104,20 +97,9 @@ static void linkstrbuf(lua_State* L, TString* ts) { global_State* g = L->global; - if (FFlag::LuauGcPagedSweep) - { - // TODO (FFlagLuauGcPagedSweep): 'next' type will change after removal of the flag and the cast will not be required - ts->next = (GCObject*)g->strbufgc; - g->strbufgc = ts; - ts->marked = luaC_white(g); - } - else - { - GCObject* o = obj2gco(ts); - o->gch.next = (GCObject*)g->strbufgc; - g->strbufgc = gco2ts(o); - o->gch.marked = luaC_white(g); - } + ts->next = g->strbufgc; + g->strbufgc = ts; + ts->marked = luaC_white(g); } static void unlinkstrbuf(lua_State* L, TString* ts) @@ -130,14 +112,12 @@ static void unlinkstrbuf(lua_State* L, TString* ts) { if (curr == ts) { - // TODO (FFlagLuauGcPagedSweep): 'next' type will change after removal of the flag and the cast will not be required - *p = (TString*)curr->next; + *p = curr->next; return; } else { - // TODO (FFlagLuauGcPagedSweep): 'next' type will change after removal of the flag and the cast will not be required - p = (TString**)&curr->next; + p = &curr->next; } } @@ -167,8 +147,7 @@ TString* luaS_buffinish(lua_State* L, TString* ts) int bucket = lmod(h, tb->size); // search if we already have this string in the hash table - // TODO (FFlagLuauGcPagedSweep): 'next' type will change after removal of the flag and the cast will not be required - for (TString* el = tb->hash[bucket]; el != NULL; el = (TString*)el->next) + for (TString* el = tb->hash[bucket]; el != NULL; el = el->next) { if (el->len == ts->len && memcmp(el->data, ts->data, ts->len) == 0) { @@ -187,8 +166,7 @@ TString* luaS_buffinish(lua_State* L, TString* ts) // Complete string object ts->atom = L->global->cb.useratom ? L->global->cb.useratom(ts->data, ts->len) : -1; - // TODO (FFlagLuauGcPagedSweep): 'next' type will change after removal of the flag and the cast will not be required - ts->next = (GCObject*)tb->hash[bucket]; // chain new entry + ts->next = tb->hash[bucket]; // chain new entry tb->hash[bucket] = ts; tb->nuse++; @@ -201,8 +179,7 @@ TString* luaS_buffinish(lua_State* L, TString* ts) TString* luaS_newlstr(lua_State* L, const char* str, size_t l) { unsigned int h = luaS_hash(str, l); - // TODO (FFlagLuauGcPagedSweep): 'next' type will change after removal of the flag and the cast will not be required - for (TString* el = L->global->strt.hash[lmod(h, L->global->strt.size)]; el != NULL; el = (TString*)el->next) + for (TString* el = L->global->strt.hash[lmod(h, L->global->strt.size)]; el != NULL; el = el->next) { if (el->len == l && (memcmp(str, getstr(el), l) == 0)) { @@ -217,8 +194,6 @@ TString* luaS_newlstr(lua_State* L, const char* str, size_t l) static bool unlinkstr(lua_State* L, TString* ts) { - LUAU_ASSERT(FFlag::LuauGcPagedSweep); - global_State* g = L->global; TString** p = &g->strt.hash[lmod(ts->hash, g->strt.size)]; @@ -227,14 +202,12 @@ static bool unlinkstr(lua_State* L, TString* ts) { if (curr == ts) { - // TODO (FFlagLuauGcPagedSweep): 'next' type will change after removal of the flag and the cast will not be required - *p = (TString*)curr->next; + *p = curr->next; return true; } else { - // TODO (FFlagLuauGcPagedSweep): 'next' type will change after removal of the flag and the cast will not be required - p = (TString**)&curr->next; + p = &curr->next; } } @@ -243,20 +216,11 @@ static bool unlinkstr(lua_State* L, TString* ts) void luaS_free(lua_State* L, TString* ts, lua_Page* page) { - if (FFlag::LuauGcPagedSweep) - { - // Unchain from the string table - if (!unlinkstr(L, ts)) - unlinkstrbuf(L, ts); // An unlikely scenario when we have a string buffer on our hands - else - L->global->strt.nuse--; - - luaM_freegco(L, ts, sizestring(ts->len), ts->memcat, page); - } + // Unchain from the string table + if (!unlinkstr(L, ts)) + unlinkstrbuf(L, ts); // An unlikely scenario when we have a string buffer on our hands else - { L->global->strt.nuse--; - luaM_free(L, ts, sizestring(ts->len), ts->memcat); - } + luaM_freegco(L, ts, sizestring(ts->len), ts->memcat, page); } diff --git a/VM/src/ltable.cpp b/VM/src/ltable.cpp index c57374e0..0412ea76 100644 --- a/VM/src/ltable.cpp +++ b/VM/src/ltable.cpp @@ -425,7 +425,7 @@ static void rehash(lua_State* L, Table* t, const TValue* ek) Table* luaH_new(lua_State* L, int narray, int nhash) { Table* t = luaM_newgco(L, Table, sizeof(Table), L->activememcat); - luaC_link(L, t, LUA_TTABLE); + luaC_init(L, t, LUA_TTABLE); t->metatable = NULL; t->flags = cast_byte(~0); t->array = NULL; @@ -742,7 +742,7 @@ int luaH_getn(Table* t) Table* luaH_clone(lua_State* L, Table* tt) { Table* t = luaM_newgco(L, Table, sizeof(Table), L->activememcat); - luaC_link(L, t, LUA_TTABLE); + luaC_init(L, t, LUA_TTABLE); t->metatable = tt->metatable; t->flags = tt->flags; t->array = NULL; diff --git a/VM/src/ludata.cpp b/VM/src/ludata.cpp index 758a9bdb..0dfac508 100644 --- a/VM/src/ludata.cpp +++ b/VM/src/ludata.cpp @@ -12,7 +12,7 @@ Udata* luaU_newudata(lua_State* L, size_t s, int tag) if (s > INT_MAX - sizeof(Udata)) luaM_toobig(L); Udata* u = luaM_newgco(L, Udata, sizeudata(s), L->activememcat); - luaC_link(L, u, LUA_TUSERDATA); + luaC_init(L, u, LUA_TUSERDATA); u->len = int(s); u->metatable = NULL; LUAU_ASSERT(tag >= 0 && tag <= 255); diff --git a/fuzz/linter.cpp b/fuzz/linter.cpp index 04638d23..0bdd49f5 100644 --- a/fuzz/linter.cpp +++ b/fuzz/linter.cpp @@ -1,10 +1,12 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #include -#include "Luau/TypeInfer.h" -#include "Luau/Linter.h" + #include "Luau/BuiltinDefinitions.h" -#include "Luau/ModuleResolver.h" #include "Luau/Common.h" +#include "Luau/Linter.h" +#include "Luau/ModuleResolver.h" +#include "Luau/Parser.h" +#include "Luau/TypeInfer.h" extern "C" int LLVMFuzzerTestOneInput(const uint8_t* Data, size_t Size) { diff --git a/fuzz/luau.proto b/fuzz/luau.proto index c78fcf31..190b8c5b 100644 --- a/fuzz/luau.proto +++ b/fuzz/luau.proto @@ -96,11 +96,13 @@ message ExprIndexExpr { } message ExprFunction { - repeated Local args = 1; - required bool vararg = 2; - required StatBlock body = 3; - repeated Type types = 4; - repeated Type rettypes = 5; + repeated Typename generics = 1; + repeated Typename genericpacks = 2; + repeated Local args = 3; + required bool vararg = 4; + required StatBlock body = 5; + repeated Type types = 6; + repeated Type rettypes = 7; } message TableItem { @@ -153,7 +155,10 @@ message ExprBinary { message ExprIfElse { required Expr cond = 1; required Expr then = 2; - required Expr else = 3; + oneof else_oneof { + Expr else = 3; + ExprIfElse elseif = 4; + } } message LValue { @@ -183,6 +188,7 @@ message Stat { StatFunction function = 14; StatLocalFunction local_function = 15; StatTypeAlias type_alias = 16; + StatRequireIntoLocalHelper require_into_local = 17; } } @@ -276,9 +282,16 @@ message StatLocalFunction { } message StatTypeAlias { - required Typename name = 1; - required Type type = 2; - repeated Typename generics = 3; + required bool export = 1; + required Typename name = 2; + required Type type = 3; + repeated Typename generics = 4; + repeated Typename genericpacks = 5; +} + +message StatRequireIntoLocalHelper { + required Local var = 1; + required int32 modulenum = 2; } message Type { @@ -292,6 +305,8 @@ message Type { TypeIntersection intersection = 7; TypeClass class = 8; TypeRef ref = 9; + TypeBoolean boolean = 10; + TypeString string = 11; } } @@ -301,7 +316,8 @@ message TypePrimitive { message TypeLiteral { required Typename name = 1; - repeated Typename generics = 2; + repeated Type generics = 2; + repeated Typename genericpacks = 3; } message TypeTableItem { @@ -320,8 +336,10 @@ message TypeTable { } message TypeFunction { - repeated Type args = 1; - repeated Type rets = 2; + repeated Typename generics = 1; + repeated Typename genericpacks = 2; + repeated Type args = 3; + repeated Type rets = 4; // TODO: vararg? } @@ -347,3 +365,16 @@ message TypeRef { required Local prefix = 1; required Typename index = 2; } + +message TypeBoolean { + required bool val = 1; +} + +message TypeString { + required string val = 1; +} + +message ModuleSet { + optional StatBlock module = 1; + required StatBlock program = 2; +} diff --git a/fuzz/proto.cpp b/fuzz/proto.cpp index f407248a..912fef23 100644 --- a/fuzz/proto.cpp +++ b/fuzz/proto.cpp @@ -2,16 +2,17 @@ #include "src/libfuzzer/libfuzzer_macro.h" #include "luau.pb.h" -#include "Luau/TypeInfer.h" #include "Luau/BuiltinDefinitions.h" -#include "Luau/ModuleResolver.h" -#include "Luau/ModuleResolver.h" -#include "Luau/Compiler.h" -#include "Luau/Linter.h" #include "Luau/BytecodeBuilder.h" #include "Luau/Common.h" +#include "Luau/Compiler.h" +#include "Luau/Frontend.h" +#include "Luau/Linter.h" +#include "Luau/ModuleResolver.h" +#include "Luau/Parser.h" #include "Luau/ToString.h" #include "Luau/Transpiler.h" +#include "Luau/TypeInfer.h" #include "lua.h" #include "lualib.h" @@ -30,7 +31,7 @@ const bool kFuzzTypes = true; static_assert(!(kFuzzVM && !kFuzzCompiler), "VM requires the compiler!"); -std::string protoprint(const luau::StatBlock& stat, bool types); +std::vector protoprint(const luau::ModuleSet& stat, bool types); LUAU_FASTINT(LuauTypeInferRecursionLimit) LUAU_FASTINT(LuauTypeInferTypePackLoopLimit) @@ -38,6 +39,7 @@ LUAU_FASTINT(LuauCheckRecursionLimit) LUAU_FASTINT(LuauTableTypeMaximumStringifierLength) LUAU_FASTINT(LuauTypeInferIterationLimit) LUAU_FASTINT(LuauTarjanChildLimit) +LUAU_FASTFLAG(DebugLuauFreezeArena) std::chrono::milliseconds kInterruptTimeout(10); std::chrono::time_point interruptDeadline; @@ -135,10 +137,58 @@ int registerTypes(Luau::TypeChecker& env) return 0; } +struct FuzzFileResolver : Luau::FileResolver +{ + std::optional readSource(const Luau::ModuleName& name) override + { + auto it = source.find(name); + if (it == source.end()) + return std::nullopt; -static std::string debugsource; + return Luau::SourceCode{it->second, Luau::SourceCode::Module}; + } -DEFINE_PROTO_FUZZER(const luau::StatBlock& message) + std::optional resolveModule(const Luau::ModuleInfo* context, Luau::AstExpr* expr) override + { + if (Luau::AstExprGlobal* g = expr->as()) + return Luau::ModuleInfo{g->name.value}; + + return std::nullopt; + } + + std::string getHumanReadableModuleName(const Luau::ModuleName& name) const override + { + return name; + } + + std::optional getEnvironmentForModule(const Luau::ModuleName& name) const override + { + return std::nullopt; + } + + std::unordered_map source; +}; + +struct FuzzConfigResolver : Luau::ConfigResolver +{ + FuzzConfigResolver() + { + defaultConfig.mode = Luau::Mode::Nonstrict; // typecheckTwice option will cover Strict mode + defaultConfig.enabledLint.warningMask = ~0ull; + defaultConfig.parseOptions.captureComments = true; + } + + virtual const Luau::Config& getConfig(const Luau::ModuleName& name) const override + { + return defaultConfig; + } + + Luau::Config defaultConfig; +}; + +static std::vector debugsources; + +DEFINE_PROTO_FUZZER(const luau::ModuleSet& message) { FInt::LuauTypeInferRecursionLimit.value = 100; FInt::LuauTypeInferTypePackLoopLimit.value = 100; @@ -151,91 +201,90 @@ DEFINE_PROTO_FUZZER(const luau::StatBlock& message) if (strncmp(flag->name, "Luau", 4) == 0) flag->value = true; - Luau::Allocator allocator; - Luau::AstNameTable names(allocator); + FFlag::DebugLuauFreezeArena.value = true; - std::string source = protoprint(message, kFuzzTypes); + std::vector sources = protoprint(message, kFuzzTypes); // stash source in a global for easier crash dump debugging - debugsource = source; - - Luau::ParseResult parseResult = Luau::Parser::parse(source.c_str(), source.size(), names, allocator); - - // "static" here is to accelerate fuzzing process by only creating and populating the type environment once - static Luau::NullModuleResolver moduleResolver; - static Luau::InternalErrorReporter iceHandler; - static Luau::TypeChecker sharedEnv(&moduleResolver, &iceHandler); - static int once = registerTypes(sharedEnv); - (void)once; - static int once2 = (Luau::freeze(sharedEnv.globalTypes), 0); - (void)once2; - - iceHandler.onInternalError = [](const char* error) { - printf("ICE: %s\n", error); - LUAU_ASSERT(!"ICE"); - }; + debugsources = sources; static bool debug = getenv("LUAU_DEBUG") != 0; if (debug) { - fprintf(stdout, "--\n%s\n", source.c_str()); + for (std::string& source : sources) + fprintf(stdout, "--\n%s\n", source.c_str()); fflush(stdout); } - std::string bytecode; + // parse all sources + std::vector> parseAllocators; + std::vector> parseNameTables; - // compile - if (kFuzzCompiler && parseResult.errors.empty()) + Luau::ParseOptions parseOptions; + parseOptions.captureComments = true; + + std::vector parseResults; + + for (std::string& source : sources) { - Luau::CompileOptions compileOptions; + parseAllocators.push_back(std::make_unique()); + parseNameTables.push_back(std::make_unique(*parseAllocators.back())); - try - { - Luau::BytecodeBuilder bcb; - Luau::compileOrThrow(bcb, parseResult.root, names, compileOptions); - bytecode = bcb.getBytecode(); - } - catch (const Luau::CompileError&) - { - // not all valid ASTs can be compiled due to limits on number of registers - } + parseResults.push_back(Luau::Parser::parse(source.c_str(), source.size(), *parseNameTables.back(), *parseAllocators.back(), parseOptions)); } - // typecheck - if (kFuzzTypeck && parseResult.root) - { - Luau::SourceModule sourceModule; - sourceModule.root = parseResult.root; - sourceModule.mode = Luau::Mode::Nonstrict; - - Luau::TypeChecker typeck(&moduleResolver, &iceHandler); - typeck.globalScope = sharedEnv.globalScope; - - Luau::ModulePtr module = nullptr; - - try - { - module = typeck.check(sourceModule, Luau::Mode::Nonstrict); - } - catch (std::exception&) - { - // This catches internal errors that the type checker currently (unfortunately) throws in some cases - } - - // lint (note that we need access to types so we need to do this with typeck in scope) - if (kFuzzLinter) - { - Luau::LintOptions lintOptions = {~0u}; - Luau::lint(parseResult.root, names, sharedEnv.globalScope, module.get(), {}, lintOptions); - } - } - - // validate sharedEnv post-typecheck; valuable for debugging some typeck crashes but slows fuzzing down - // note: it's important for typeck to be destroyed at this point! + // typecheck all sources if (kFuzzTypeck) { - for (auto& p : sharedEnv.globalScope->bindings) + static FuzzFileResolver fileResolver; + static Luau::NullConfigResolver configResolver; + static Luau::FrontendOptions options{true, true}; + static Luau::Frontend frontend(&fileResolver, &configResolver, options); + + static int once = registerTypes(frontend.typeChecker); + (void)once; + static int once2 = (Luau::freeze(frontend.typeChecker.globalTypes), 0); + (void)once2; + + frontend.iceHandler.onInternalError = [](const char* error) { + printf("ICE: %s\n", error); + LUAU_ASSERT(!"ICE"); + }; + + // restart + frontend.clear(); + fileResolver.source.clear(); + + // load sources + for (size_t i = 0; i < sources.size(); i++) + { + std::string name = "module" + std::to_string(i); + fileResolver.source[name] = sources[i]; + } + + // check sources + for (size_t i = 0; i < sources.size(); i++) + { + std::string name = "module" + std::to_string(i); + + try + { + Luau::CheckResult result = frontend.check(name, std::nullopt); + + // lint (note that we need access to types so we need to do this with typeck in scope) + if (kFuzzLinter && result.errors.empty()) + frontend.lint(name, std::nullopt); + } + catch (std::exception&) + { + // This catches internal errors that the type checker currently (unfortunately) throws in some cases + } + } + + // validate sharedEnv post-typecheck; valuable for debugging some typeck crashes but slows fuzzing down + // note: it's important for typeck to be destroyed at this point! + for (auto& p : frontend.typeChecker.globalScope->bindings) { Luau::ToStringOptions opts; opts.exhaustive = true; @@ -246,12 +295,44 @@ DEFINE_PROTO_FUZZER(const luau::StatBlock& message) } } - if (kFuzzTranspile && parseResult.root) + if (kFuzzTranspile) { - transpileWithTypes(*parseResult.root); + for (Luau::ParseResult& parseResult : parseResults) + { + if (parseResult.root) + transpileWithTypes(*parseResult.root); + } } - // run resulting bytecode + std::string bytecode; + + // compile + if (kFuzzCompiler) + { + for (size_t i = 0; i < parseResults.size(); i++) + { + Luau::ParseResult& parseResult = parseResults[i]; + Luau::AstNameTable& parseNameTable = *parseNameTables[i]; + + if (parseResult.errors.empty()) + { + Luau::CompileOptions compileOptions; + + try + { + Luau::BytecodeBuilder bcb; + Luau::compileOrThrow(bcb, parseResult.root, parseNameTable, compileOptions); + bytecode = bcb.getBytecode(); + } + catch (const Luau::CompileError&) + { + // not all valid ASTs can be compiled due to limits on number of registers + } + } + } + } + + // run resulting bytecode (from last successfully compiler module) if (kFuzzVM && bytecode.size()) { static lua_State* globalState = createGlobalState(); diff --git a/fuzz/protoprint.cpp b/fuzz/protoprint.cpp index e61b6936..66a89f24 100644 --- a/fuzz/protoprint.cpp +++ b/fuzz/protoprint.cpp @@ -208,6 +208,35 @@ struct ProtoToLuau source += std::to_string(name.index() & 0xff); } + template + void genericidents(const T& node) + { + if (node.generics_size() || node.genericpacks_size()) + { + source += '<'; + bool first = true; + + for (size_t i = 0; i < node.generics_size(); ++i) + { + if (!first) + source += ','; + first = false; + ident(node.generics(i)); + } + + for (size_t i = 0; i < node.genericpacks_size(); ++i) + { + if (!first) + source += ','; + first = false; + ident(node.genericpacks(i)); + source += "..."; + } + + source += '>'; + } + } + void print(const luau::Expr& expr) { if (expr.has_group()) @@ -240,6 +269,8 @@ struct ProtoToLuau print(expr.unary()); else if (expr.has_binary()) print(expr.binary()); + else if (expr.has_ifelse()) + print(expr.ifelse()); else source += "_"; } @@ -350,6 +381,7 @@ struct ProtoToLuau void function(const luau::ExprFunction& expr) { + genericidents(expr); source += "("; for (int i = 0; i < expr.args_size(); ++i) { @@ -478,12 +510,21 @@ struct ProtoToLuau void print(const luau::ExprIfElse& expr) { - source += " if "; + source += "if "; print(expr.cond()); source += " then "; print(expr.then()); - source += " else "; - print(expr.else_()); + + if (expr.has_else_()) + { + source += " else "; + print(expr.else_()); + } + else if (expr.has_elseif()) + { + source += " else"; + print(expr.elseif()); + } } void print(const luau::LValue& expr) @@ -534,6 +575,8 @@ struct ProtoToLuau print(stat.local_function()); else if (stat.has_type_alias()) print(stat.type_alias()); + else if (stat.has_require_into_local()) + print(stat.require_into_local()); else source += "do end\n"; } @@ -804,26 +847,24 @@ struct ProtoToLuau void print(const luau::StatTypeAlias& stat) { + if (stat.export_()) + source += "export "; + source += "type "; ident(stat.name()); - - if (stat.generics_size()) - { - source += '<'; - for (size_t i = 0; i < stat.generics_size(); ++i) - { - if (i != 0) - source += ','; - ident(stat.generics(i)); - } - source += '>'; - } - + genericidents(stat); source += " = "; print(stat.type()); source += '\n'; } + void print(const luau::StatRequireIntoLocalHelper& stat) + { + source += "local "; + print(stat.var()); + source += " = require(module" + std::to_string(stat.modulenum() % 2) + ")\n"; + } + void print(const luau::Type& type) { if (type.has_primitive()) @@ -844,6 +885,10 @@ struct ProtoToLuau print(type.class_()); else if (type.has_ref()) print(type.ref()); + else if (type.has_boolean()) + print(type.boolean()); + else if (type.has_string()) + print(type.string()); else source += "any"; } @@ -858,15 +903,28 @@ struct ProtoToLuau { ident(type.name()); - if (type.generics_size()) + if (type.generics_size() || type.genericpacks_size()) { source += '<'; + bool first = true; + for (size_t i = 0; i < type.generics_size(); ++i) { - if (i != 0) + if (!first) source += ','; - ident(type.generics(i)); + first = false; + print(type.generics(i)); } + + for (size_t i = 0; i < type.genericpacks_size(); ++i) + { + if (!first) + source += ','; + first = false; + ident(type.genericpacks(i)); + source += "..."; + } + source += '>'; } } @@ -893,6 +951,7 @@ struct ProtoToLuau void print(const luau::TypeFunction& type) { + genericidents(type); source += '('; for (size_t i = 0; i < type.args_size(); ++i) { @@ -950,12 +1009,38 @@ struct ProtoToLuau source += '.'; ident(type.index()); } + + void print(const luau::TypeBoolean& type) + { + source += type.val() ? "true" : "false"; + } + + void print(const luau::TypeString& type) + { + source += '"'; + for (char ch : type.val()) + if (isgraph(ch)) + source += ch; + source += '"'; + } }; -std::string protoprint(const luau::StatBlock& stat, bool types) +std::vector protoprint(const luau::ModuleSet& stat, bool types) { + std::vector result; + + if (stat.has_module()) + { + ProtoToLuau printer; + printer.types = types; + printer.print(stat.module()); + result.push_back(printer.source); + } + ProtoToLuau printer; printer.types = types; - printer.print(stat); - return printer.source; + printer.print(stat.program()); + result.push_back(printer.source); + + return result; } diff --git a/fuzz/prototest.cpp b/fuzz/prototest.cpp index 804e708a..ccaa1971 100644 --- a/fuzz/prototest.cpp +++ b/fuzz/prototest.cpp @@ -2,11 +2,15 @@ #include "src/libfuzzer/libfuzzer_macro.h" #include "luau.pb.h" -std::string protoprint(const luau::StatBlock& stat, bool types); +std::vector protoprint(const luau::ModuleSet& stat, bool types); -DEFINE_PROTO_FUZZER(const luau::StatBlock& message) +DEFINE_PROTO_FUZZER(const luau::ModuleSet& message) { - std::string source = protoprint(message, true); + std::vector sources = protoprint(message, true); - printf("%s\n", source.c_str()); + for (size_t i = 0; i < sources.size(); i++) + { + printf("Module 'l%d':\n", int(i)); + printf("%s\n", sources[i].c_str()); + } } diff --git a/fuzz/typeck.cpp b/fuzz/typeck.cpp index 5020c771..3905cc19 100644 --- a/fuzz/typeck.cpp +++ b/fuzz/typeck.cpp @@ -1,9 +1,11 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #include -#include "Luau/TypeInfer.h" + #include "Luau/BuiltinDefinitions.h" -#include "Luau/ModuleResolver.h" #include "Luau/Common.h" +#include "Luau/ModuleResolver.h" +#include "Luau/Parser.h" +#include "Luau/TypeInfer.h" LUAU_FASTINT(LuauTypeInferRecursionLimit) LUAU_FASTINT(LuauTypeInferTypePackLoopLimit) diff --git a/tests/Autocomplete.test.cpp b/tests/Autocomplete.test.cpp index 1978a0d3..6aadef32 100644 --- a/tests/Autocomplete.test.cpp +++ b/tests/Autocomplete.test.cpp @@ -2752,7 +2752,6 @@ TEST_CASE_FIXTURE(ACFixture, "autocomplete_on_string_singletons") ScopedFastFlag sffs[] = { {"LuauParseSingletonTypes", true}, {"LuauSingletonTypes", true}, - {"LuauRefactorTypeVarQuestions", true}, }; check(R"( diff --git a/tests/Conformance.test.cpp b/tests/Conformance.test.cpp index b09c1efb..eb6ca749 100644 --- a/tests/Conformance.test.cpp +++ b/tests/Conformance.test.cpp @@ -712,6 +712,47 @@ TEST_CASE("Reference") CHECK(dtorhits == 2); } +TEST_CASE("ApiTables") +{ + StateRef globalState(luaL_newstate(), lua_close); + lua_State* L = globalState.get(); + + lua_newtable(L); + lua_pushnumber(L, 123.0); + lua_setfield(L, -2, "key"); + lua_pushstring(L, "test"); + lua_rawseti(L, -2, 5); + + // lua_gettable + lua_pushstring(L, "key"); + CHECK(lua_gettable(L, -2) == LUA_TNUMBER); + CHECK(lua_tonumber(L, -1) == 123.0); + lua_pop(L, 1); + + // lua_getfield + CHECK(lua_getfield(L, -1, "key") == LUA_TNUMBER); + CHECK(lua_tonumber(L, -1) == 123.0); + lua_pop(L, 1); + + // lua_rawgetfield + CHECK(lua_rawgetfield(L, -1, "key") == LUA_TNUMBER); + CHECK(lua_tonumber(L, -1) == 123.0); + lua_pop(L, 1); + + // lua_rawget + lua_pushstring(L, "key"); + CHECK(lua_rawget(L, -2) == LUA_TNUMBER); + CHECK(lua_tonumber(L, -1) == 123.0); + lua_pop(L, 1); + + // lua_rawgeti + CHECK(lua_rawgeti(L, -1, 5) == LUA_TSTRING); + CHECK(strcmp(lua_tostring(L, -1), "test") == 0); + lua_pop(L, 1); + + lua_pop(L, 1); +} + TEST_CASE("ApiFunctionCalls") { StateRef globalState = runConformance("apicalls.lua"); @@ -796,7 +837,7 @@ TEST_CASE("ExceptionObject") return ExceptionResult{false, ""}; }; - auto reallocFunc = [](lua_State* L, void* /*ud*/, void* ptr, size_t /*osize*/, size_t nsize) -> void* { + auto reallocFunc = [](void* /*ud*/, void* ptr, size_t /*osize*/, size_t nsize) -> void* { if (nsize == 0) { free(ptr); @@ -923,4 +964,53 @@ TEST_CASE("StringConversion") runConformance("strconv.lua"); } +TEST_CASE("GCDump") +{ + // internal function, declared in lgc.h - not exposed via lua.h + extern void luaC_dump(lua_State* L, void* file, const char* (*categoryName)(lua_State* L, uint8_t memcat)); + + StateRef globalState(luaL_newstate(), lua_close); + lua_State* L = globalState.get(); + + // push various objects on stack to cover different paths + lua_createtable(L, 1, 2); + lua_pushstring(L, "value"); + lua_setfield(L, -2, "key"); + + lua_pushinteger(L, 42); + lua_rawseti(L, -2, 1000); + + lua_pushinteger(L, 42); + lua_rawseti(L, -2, 1); + + lua_pushvalue(L, -1); + lua_setmetatable(L, -2); + + lua_newuserdata(L, 42); + lua_pushvalue(L, -2); + lua_setmetatable(L, -2); + + lua_pushinteger(L, 1); + lua_pushcclosure(L, lua_silence, "test", 1); + + lua_State* CL = lua_newthread(L); + + lua_pushstring(CL, "local x x = {} local function f() x[1] = math.abs(42) end function foo() coroutine.yield() end foo() return f"); + lua_loadstring(CL); + lua_resume(CL, nullptr, 0); + +#ifdef _WIN32 + const char* path = "NUL"; +#else + const char* path = "/dev/null"; +#endif + + FILE* f = fopen(path, "w"); + REQUIRE(f); + + luaC_dump(L, f, nullptr); + + fclose(f); +} + TEST_SUITE_END(); diff --git a/tests/Linter.test.cpp b/tests/Linter.test.cpp index d4b97360..ab19cea3 100644 --- a/tests/Linter.test.cpp +++ b/tests/Linter.test.cpp @@ -1594,4 +1594,17 @@ TEST_CASE_FIXTURE(Fixture, "WrongCommentMuteSelf") REQUIRE_EQ(result.warnings.size(), 0); // --!nolint disables WrongComment lint :) } +TEST_CASE_FIXTURE(Fixture, "DuplicateConditionsIfStatAndExpr") +{ + LintResult result = lint(R"( +if if 1 then 2 else 3 then +elseif if 1 then 2 else 3 then +elseif if 0 then 5 else 4 then +end +)"); + + REQUIRE_EQ(result.warnings.size(), 1); + CHECK_EQ(result.warnings[0].text, "Condition has already been checked on line 2"); +} + TEST_SUITE_END(); diff --git a/tests/Parser.test.cpp b/tests/Parser.test.cpp index 0d4c088d..77e49ce3 100644 --- a/tests/Parser.test.cpp +++ b/tests/Parser.test.cpp @@ -2575,7 +2575,6 @@ do end TEST_CASE_FIXTURE(Fixture, "recover_expected_type_pack") { ScopedFastFlag luauParseTypeAliasDefaults{"LuauParseTypeAliasDefaults", true}; - ScopedFastFlag luauParseRecoverTypePackEllipsis{"LuauParseRecoverTypePackEllipsis", true}; ParseResult result = tryParse(R"( type Y = (T...) -> U... diff --git a/tests/Transpiler.test.cpp b/tests/Transpiler.test.cpp index ac5be859..332aba9e 100644 --- a/tests/Transpiler.test.cpp +++ b/tests/Transpiler.test.cpp @@ -651,4 +651,19 @@ local a: Packed CHECK_EQ(code, transpile(code, {}, true).code); } + +TEST_CASE_FIXTURE(Fixture, "transpile_singleton_types") +{ + ScopedFastFlag luauParseSingletonTypes{"LuauParseSingletonTypes", true}; + + std::string code = R"( +type t1 = 'hello' +type t2 = true +type t3 = '' +type t4 = false + )"; + + CHECK_EQ(code, transpile(code, {}, true).code); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.intersectionTypes.test.cpp b/tests/TypeInfer.intersectionTypes.test.cpp index d677e28d..26881b5c 100644 --- a/tests/TypeInfer.intersectionTypes.test.cpp +++ b/tests/TypeInfer.intersectionTypes.test.cpp @@ -175,6 +175,8 @@ TEST_CASE_FIXTURE(Fixture, "index_on_an_intersection_type_with_property_guarante TEST_CASE_FIXTURE(Fixture, "index_on_an_intersection_type_works_at_arbitrary_depth") { + ScopedFastFlag sff{"LuauDoNotTryToReduce", true}; + CheckResult result = check(R"( type A = {x: {y: {z: {thing: string}}}} type B = {x: {y: {z: {thing: string}}}} @@ -184,7 +186,7 @@ TEST_CASE_FIXTURE(Fixture, "index_on_an_intersection_type_works_at_arbitrary_dep )"); LUAU_REQUIRE_NO_ERRORS(result); - CHECK_EQ(*typeChecker.stringType, *requireType("r")); + CHECK_EQ("string & string", toString(requireType("r"))); } TEST_CASE_FIXTURE(Fixture, "index_on_an_intersection_type_with_mixed_types") @@ -218,7 +220,7 @@ TEST_CASE_FIXTURE(Fixture, "index_on_an_intersection_type_with_one_part_missing_ TEST_CASE_FIXTURE(Fixture, "index_on_an_intersection_type_with_one_property_of_type_any") { CheckResult result = check(R"( - type A = {x: number} + type A = {y: number} type B = {x: any} local t: A & B diff --git a/tests/TypeInfer.refinements.test.cpp b/tests/TypeInfer.refinements.test.cpp index 48e6be6a..bff8926c 100644 --- a/tests/TypeInfer.refinements.test.cpp +++ b/tests/TypeInfer.refinements.test.cpp @@ -1115,6 +1115,22 @@ TEST_CASE_FIXTURE(Fixture, "discriminate_on_properties_of_disjoint_tables_where_ LUAU_REQUIRE_NO_ERRORS(result); } +TEST_CASE_FIXTURE(Fixture, "refine_a_property_not_to_be_nil_through_an_intersection_table") +{ + ScopedFastFlag sff{"LuauDoNotTryToReduce", true}; + + CheckResult result = check(R"( + type T = {} & {f: ((string) -> string)?} + local function f(t: T, x) + if t.f then + t.f(x) + end + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + TEST_CASE_FIXTURE(RefinementClassFixture, "discriminate_from_isa_of_x") { ScopedFastFlag sff[] = { diff --git a/tests/TypeInfer.singletons.test.cpp b/tests/TypeInfer.singletons.test.cpp index 9021700d..856549bd 100644 --- a/tests/TypeInfer.singletons.test.cpp +++ b/tests/TypeInfer.singletons.test.cpp @@ -439,4 +439,128 @@ local a: Animal = if true then { tag = 'cat', catfood = 'something' } else { tag LUAU_REQUIRE_NO_ERRORS(result); } +TEST_CASE_FIXTURE(Fixture, "widen_the_supertype_if_it_is_free_and_subtype_has_singleton") +{ + ScopedFastFlag sff[]{ + {"LuauSingletonTypes", true}, + {"LuauEqConstraint", true}, + {"LuauDiscriminableUnions2", true}, + {"LuauWidenIfSupertypeIsFree", true}, + {"LuauWeakEqConstraint", false}, + }; + + CheckResult result = check(R"( + local function foo(f, x) + if x == "hi" then + f(x) + f("foo") + end + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ(R"("hi")", toString(requireTypeAtPosition({3, 18}))); + // should be ((string) -> a..., string) -> () but needs lower bounds calculation + CHECK_EQ("((string) -> (b...), a) -> ()", toString(requireType("foo"))); +} + +// TEST_CASE_FIXTURE(Fixture, "return_type_of_f_is_not_widened") +// { +// ScopedFastFlag sff[]{ +// {"LuauParseSingletonTypes", true}, +// {"LuauSingletonTypes", true}, +// {"LuauDiscriminableUnions2", true}, +// {"LuauEqConstraint", true}, +// {"LuauWidenIfSupertypeIsFree", true}, +// {"LuauWeakEqConstraint", false}, +// }; + +// CheckResult result = check(R"( +// local function foo(f, x): "hello"? -- anyone there? +// return if x == "hi" +// then f(x) +// else nil +// end +// )"); + +// LUAU_REQUIRE_NO_ERRORS(result); + +// CHECK_EQ(R"("hi")", toString(requireTypeAtPosition({3, 23}))); +// CHECK_EQ(R"(((string) -> ("hello"?, b...), a) -> "hello"?)", toString(requireType("foo"))); +// } + +TEST_CASE_FIXTURE(Fixture, "widening_happens_almost_everywhere") +{ + ScopedFastFlag sff[]{ + {"LuauParseSingletonTypes", true}, + {"LuauSingletonTypes", true}, + {"LuauWidenIfSupertypeIsFree", true}, + }; + + CheckResult result = check(R"( + local foo: "foo" = "foo" + local copy = foo + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + CHECK_EQ("string", toString(requireType("copy"))); +} + +TEST_CASE_FIXTURE(Fixture, "widening_happens_almost_everywhere_except_for_tables") +{ + ScopedFastFlag sff[]{ + {"LuauParseSingletonTypes", true}, + {"LuauSingletonTypes", true}, + {"LuauDiscriminableUnions2", true}, + {"LuauWidenIfSupertypeIsFree", true}, + }; + + CheckResult result = check(R"( + type Cat = {tag: "Cat", meows: boolean} + type Dog = {tag: "Dog", barks: boolean} + type Animal = Cat | Dog + + local function f(tag: "Cat" | "Dog"): Animal? + if tag == "Cat" then + local result = {tag = tag, meows = true} + return result + elseif tag == "Dog" then + local result = {tag = tag, barks = true} + return result + else + return nil + end + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "table_insert_with_a_singleton_argument") +{ + ScopedFastFlag sff[]{ + {"LuauParseSingletonTypes", true}, + {"LuauSingletonTypes", true}, + {"LuauWidenIfSupertypeIsFree", true}, + }; + + CheckResult result = check(R"( + local function foo(t, x) + if x == "hi" or x == "bye" then + table.insert(t, x) + end + + return t + end + + local t = foo({}, "hi") + table.insert(t, "totally_unrelated_type" :: "totally_unrelated_type") + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ("{string}", toString(requireType("t"))); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.tables.test.cpp b/tests/TypeInfer.tables.test.cpp index 6bcd4b99..aa949789 100644 --- a/tests/TypeInfer.tables.test.cpp +++ b/tests/TypeInfer.tables.test.cpp @@ -2239,4 +2239,22 @@ TEST_CASE_FIXTURE(Fixture, "give_up_after_one_metatable_index_look_up") CHECK_EQ("Type 't2' does not have key 'x'", toString(result.errors[0])); } +TEST_CASE_FIXTURE(Fixture, "confusing_indexing") +{ + ScopedFastFlag sff{"LuauDoNotTryToReduce", true}; + + CheckResult result = check(R"( + type T = {} & {p: number | string} + local function f(t: T) + return t.p + end + + local foo = f({p = "string"}) + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ("number | string", toString(requireType("foo"))); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.test.cpp b/tests/TypeInfer.test.cpp index 32358571..f44d9fd8 100644 --- a/tests/TypeInfer.test.cpp +++ b/tests/TypeInfer.test.cpp @@ -5127,8 +5127,6 @@ TEST_CASE_FIXTURE(Fixture, "cli_50041_committing_txnlog_in_apollo_client_error") TEST_CASE_FIXTURE(Fixture, "do_not_modify_imported_types") { - ScopedFastFlag noSealedTypeMod{"LuauNoSealedTypeMod", true}; - fileResolver.source["game/A"] = R"( export type Type = { unrelated: boolean } return {} @@ -5190,7 +5188,6 @@ TEST_CASE_FIXTURE(Fixture, "indexing_on_string_singletons") { ScopedFastFlag sff[]{ {"LuauDiscriminableUnions2", true}, - {"LuauRefactorTypeVarQuestions", true}, {"LuauSingletonTypes", true}, }; @@ -5210,7 +5207,6 @@ TEST_CASE_FIXTURE(Fixture, "indexing_on_union_of_string_singletons") { ScopedFastFlag sff[]{ {"LuauDiscriminableUnions2", true}, - {"LuauRefactorTypeVarQuestions", true}, {"LuauSingletonTypes", true}, }; @@ -5230,7 +5226,6 @@ TEST_CASE_FIXTURE(Fixture, "taking_the_length_of_string_singleton") { ScopedFastFlag sff[]{ {"LuauDiscriminableUnions2", true}, - {"LuauRefactorTypeVarQuestions", true}, {"LuauSingletonTypes", true}, }; @@ -5250,7 +5245,6 @@ TEST_CASE_FIXTURE(Fixture, "taking_the_length_of_union_of_string_singleton") { ScopedFastFlag sff[]{ {"LuauDiscriminableUnions2", true}, - {"LuauRefactorTypeVarQuestions", true}, {"LuauSingletonTypes", true}, }; diff --git a/tests/TypeInfer.tryUnify.test.cpp b/tests/TypeInfer.tryUnify.test.cpp index 4669ea8e..c9bf5103 100644 --- a/tests/TypeInfer.tryUnify.test.cpp +++ b/tests/TypeInfer.tryUnify.test.cpp @@ -19,7 +19,7 @@ struct TryUnifyFixture : Fixture ScopePtr globalScope{new Scope{arena.addTypePack({TypeId{}})}}; InternalErrorReporter iceHandler; UnifierSharedState unifierState{&iceHandler}; - Unifier state{&arena, Mode::Strict, globalScope, Location{}, Variance::Covariant, unifierState}; + Unifier state{&arena, Mode::Strict, Location{}, Variance::Covariant, unifierState}; }; TEST_SUITE_BEGIN("TryUnifyTests"); @@ -261,8 +261,6 @@ TEST_CASE_FIXTURE(TryUnifyFixture, "free_tail_is_grown_properly") TEST_CASE_FIXTURE(TryUnifyFixture, "recursive_metatable_getmatchtag") { - ScopedFastFlag luauUnionTagMatchFix{"LuauUnionTagMatchFix", true}; - TypeVar redirect{FreeTypeVar{TypeLevel{}}}; TypeVar table{TableTypeVar{}}; TypeVar metatable{MetatableTypeVar{&redirect, &table}}; diff --git a/tests/TypeVar.test.cpp b/tests/TypeVar.test.cpp index e43161fa..fd5f4dbc 100644 --- a/tests/TypeVar.test.cpp +++ b/tests/TypeVar.test.cpp @@ -361,16 +361,12 @@ local b: (T, T, T) -> T TEST_CASE("isString_on_string_singletons") { - ScopedFastFlag sff{"LuauRefactorTypeVarQuestions", true}; - TypeVar helloString{SingletonTypeVar{StringSingleton{"hello"}}}; CHECK(isString(&helloString)); } TEST_CASE("isString_on_unions_of_various_string_singletons") { - ScopedFastFlag sff{"LuauRefactorTypeVarQuestions", true}; - TypeVar helloString{SingletonTypeVar{StringSingleton{"hello"}}}; TypeVar byeString{SingletonTypeVar{StringSingleton{"bye"}}}; TypeVar union_{UnionTypeVar{{&helloString, &byeString}}}; @@ -380,8 +376,6 @@ TEST_CASE("isString_on_unions_of_various_string_singletons") TEST_CASE("proof_that_isString_uses_all_of") { - ScopedFastFlag sff{"LuauRefactorTypeVarQuestions", true}; - TypeVar helloString{SingletonTypeVar{StringSingleton{"hello"}}}; TypeVar byeString{SingletonTypeVar{StringSingleton{"bye"}}}; TypeVar booleanType{PrimitiveTypeVar{PrimitiveTypeVar::Boolean}}; @@ -392,16 +386,12 @@ TEST_CASE("proof_that_isString_uses_all_of") TEST_CASE("isBoolean_on_boolean_singletons") { - ScopedFastFlag sff{"LuauRefactorTypeVarQuestions", true}; - TypeVar trueBool{SingletonTypeVar{BooleanSingleton{true}}}; CHECK(isBoolean(&trueBool)); } TEST_CASE("isBoolean_on_unions_of_true_or_false_singletons") { - ScopedFastFlag sff{"LuauRefactorTypeVarQuestions", true}; - TypeVar trueBool{SingletonTypeVar{BooleanSingleton{true}}}; TypeVar falseBool{SingletonTypeVar{BooleanSingleton{false}}}; TypeVar union_{UnionTypeVar{{&trueBool, &falseBool}}}; @@ -411,8 +401,6 @@ TEST_CASE("isBoolean_on_unions_of_true_or_false_singletons") TEST_CASE("proof_that_isBoolean_uses_all_of") { - ScopedFastFlag sff{"LuauRefactorTypeVarQuestions", true}; - TypeVar trueBool{SingletonTypeVar{BooleanSingleton{true}}}; TypeVar falseBool{SingletonTypeVar{BooleanSingleton{false}}}; TypeVar stringType{PrimitiveTypeVar{PrimitiveTypeVar::String}}; diff --git a/tools/natvis/VM.natvis b/tools/natvis/VM.natvis index 9924e194..ccc7e390 100644 --- a/tools/natvis/VM.natvis +++ b/tools/natvis/VM.natvis @@ -183,13 +183,12 @@ openupval - u.l.next + u.l.threadnext this - l_gt - env + gt userdata From c7eca27909d787edc8136fc019dd1c13006c42cb Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Thu, 24 Feb 2022 15:53:37 -0800 Subject: [PATCH 181/399] Sync to upstream/release/516 (#397) --- Analysis/include/Luau/Module.h | 9 +- Analysis/include/Luau/TypeInfer.h | 2 + Analysis/include/Luau/TypeUtils.h | 4 +- Analysis/include/Luau/Unifier.h | 29 ++- Analysis/src/Autocomplete.cpp | 4 +- Analysis/src/Linter.cpp | 1 + Analysis/src/Module.cpp | 27 +-- Analysis/src/Transpiler.cpp | 8 + Analysis/src/TypeInfer.cpp | 67 +++--- Analysis/src/TypeUtils.cpp | 8 +- Analysis/src/TypeVar.cpp | 110 ++-------- Analysis/src/Unifier.cpp | 126 ++++++----- Ast/src/Parser.cpp | 8 +- VM/include/lua.h | 2 +- VM/src/ldo.cpp | 3 +- VM/src/lfunc.cpp | 62 ++---- VM/src/lgc.cpp | 218 +++---------------- VM/src/lgc.h | 10 +- VM/src/lgcdebug.cpp | 72 +------ VM/src/linit.cpp | 2 +- VM/src/lmem.cpp | 127 ++--------- VM/src/lmem.h | 4 - VM/src/lobject.h | 5 +- VM/src/lstate.cpp | 32 +-- VM/src/lstate.h | 5 - VM/src/lstring.cpp | 70 ++---- VM/src/ltable.cpp | 4 +- VM/src/ludata.cpp | 2 +- fuzz/linter.cpp | 8 +- fuzz/luau.proto | 55 +++-- fuzz/proto.cpp | 237 ++++++++++++++------- fuzz/protoprint.cpp | 129 +++++++++-- fuzz/prototest.cpp | 12 +- fuzz/typeck.cpp | 6 +- tests/Autocomplete.test.cpp | 1 - tests/Conformance.test.cpp | 51 ++++- tests/Linter.test.cpp | 13 ++ tests/Parser.test.cpp | 1 - tests/Transpiler.test.cpp | 15 ++ tests/TypeInfer.intersectionTypes.test.cpp | 6 +- tests/TypeInfer.refinements.test.cpp | 16 ++ tests/TypeInfer.singletons.test.cpp | 124 +++++++++++ tests/TypeInfer.tables.test.cpp | 18 ++ tests/TypeInfer.test.cpp | 6 - tests/TypeInfer.tryUnify.test.cpp | 4 +- tests/TypeVar.test.cpp | 12 -- tools/natvis/VM.natvis | 5 +- 47 files changed, 863 insertions(+), 877 deletions(-) diff --git a/Analysis/include/Luau/Module.h b/Analysis/include/Luau/Module.h index 61200771..6c689b7c 100644 --- a/Analysis/include/Luau/Module.h +++ b/Analysis/include/Luau/Module.h @@ -13,8 +13,6 @@ #include #include -LUAU_FASTFLAG(LuauPrepopulateUnionOptionsBeforeAllocation) - namespace Luau { @@ -60,11 +58,8 @@ struct TypeArena template TypeId addType(T tv) { - if (FFlag::LuauPrepopulateUnionOptionsBeforeAllocation) - { - if constexpr (std::is_same_v) - LUAU_ASSERT(tv.options.size() >= 2); - } + if constexpr (std::is_same_v) + LUAU_ASSERT(tv.options.size() >= 2); return addTV(TypeVar(std::move(tv))); } diff --git a/Analysis/include/Luau/TypeInfer.h b/Analysis/include/Luau/TypeInfer.h index 3c5ded3c..2440c810 100644 --- a/Analysis/include/Luau/TypeInfer.h +++ b/Analysis/include/Luau/TypeInfer.h @@ -31,6 +31,7 @@ bool doesCallError(const AstExprCall* call); bool hasBreak(AstStat* node); const AstStat* getFallthrough(const AstStat* node); +struct UnifierOptions; struct Unifier; // A substitution which replaces generic types in a given set by free types. @@ -245,6 +246,7 @@ struct TypeChecker * Treat any failures as type errors in the final typecheck report. */ bool unify(TypeId subTy, TypeId superTy, const Location& location); + bool unify(TypeId subTy, TypeId superTy, const Location& location, const UnifierOptions& options); bool unify(TypePackId subTy, TypePackId superTy, const Location& location, CountMismatch::Context ctx = CountMismatch::Context::Arg); /** Attempt to unify the types. diff --git a/Analysis/include/Luau/TypeUtils.h b/Analysis/include/Luau/TypeUtils.h index ffddfe4b..42c1bc0b 100644 --- a/Analysis/include/Luau/TypeUtils.h +++ b/Analysis/include/Luau/TypeUtils.h @@ -13,7 +13,7 @@ namespace Luau using ScopePtr = std::shared_ptr; -std::optional findMetatableEntry(ErrorVec& errors, const ScopePtr& globalScope, TypeId type, std::string entry, Location location); -std::optional findTablePropertyRespectingMeta(ErrorVec& errors, const ScopePtr& globalScope, TypeId ty, Name name, Location location); +std::optional findMetatableEntry(ErrorVec& errors, TypeId type, std::string entry, Location location); +std::optional findTablePropertyRespectingMeta(ErrorVec& errors, TypeId ty, Name name, Location location); } // namespace Luau diff --git a/Analysis/include/Luau/Unifier.h b/Analysis/include/Luau/Unifier.h index 9db4e22b..fe822b01 100644 --- a/Analysis/include/Luau/Unifier.h +++ b/Analysis/include/Luau/Unifier.h @@ -19,11 +19,31 @@ enum Variance Invariant }; +// A substitution which replaces singleton types by their wider types +struct Widen : Substitution +{ + Widen(TypeArena* arena) + : Substitution(TxnLog::empty(), arena) + { + } + + bool isDirty(TypeId ty) override; + bool isDirty(TypePackId ty) override; + TypeId clean(TypeId ty) override; + TypePackId clean(TypePackId ty) override; + bool ignoreChildren(TypeId ty) override; +}; + +// TODO: Use this more widely. +struct UnifierOptions +{ + bool isFunctionCall = false; +}; + struct Unifier { TypeArena* const types; Mode mode; - ScopePtr globalScope; // sigh. Needed solely to get at string's metatable. DEPRECATED_TxnLog DEPRECATED_log; TxnLog log; @@ -34,9 +54,9 @@ struct Unifier UnifierSharedState& sharedState; - Unifier(TypeArena* types, Mode mode, ScopePtr globalScope, const Location& location, Variance variance, UnifierSharedState& sharedState, + Unifier(TypeArena* types, Mode mode, const Location& location, Variance variance, UnifierSharedState& sharedState, TxnLog* parentLog = nullptr); - Unifier(TypeArena* types, Mode mode, ScopePtr globalScope, std::vector>* sharedSeen, const Location& location, + Unifier(TypeArena* types, Mode mode, std::vector>* sharedSeen, const Location& location, Variance variance, UnifierSharedState& sharedState, TxnLog* parentLog = nullptr); // Test whether the two type vars unify. Never commits the result. @@ -65,7 +85,10 @@ private: void tryUnifyWithMetatable(TypeId subTy, TypeId superTy, bool reversed); void tryUnifyWithClass(TypeId subTy, TypeId superTy, bool reversed); void tryUnifyIndexer(const TableIndexer& subIndexer, const TableIndexer& superIndexer); + + TypeId widen(TypeId ty); TypeId deeplyOptional(TypeId ty, std::unordered_map seen = {}); + void cacheResult(TypeId subTy, TypeId superTy); public: diff --git a/Analysis/src/Autocomplete.cpp b/Analysis/src/Autocomplete.cpp index 5a1ae397..29a2c6b5 100644 --- a/Analysis/src/Autocomplete.cpp +++ b/Analysis/src/Autocomplete.cpp @@ -236,10 +236,10 @@ static TypeCorrectKind checkTypeCorrectKind(const Module& module, TypeArena* typ { ty = follow(ty); - auto canUnify = [&typeArena, &module](TypeId subTy, TypeId superTy) { + auto canUnify = [&typeArena](TypeId subTy, TypeId superTy) { InternalErrorReporter iceReporter; UnifierSharedState unifierState(&iceReporter); - Unifier unifier(typeArena, Mode::Strict, module.getModuleScope(), Location(), Variance::Covariant, unifierState); + Unifier unifier(typeArena, Mode::Strict, Location(), Variance::Covariant, unifierState); if (FFlag::LuauAutocompleteAvoidMutation && !FFlag::LuauUseCommittingTxnLog) { diff --git a/Analysis/src/Linter.cpp b/Analysis/src/Linter.cpp index 8d7d2d97..7635dc0f 100644 --- a/Analysis/src/Linter.cpp +++ b/Analysis/src/Linter.cpp @@ -201,6 +201,7 @@ static bool similar(AstExpr* lhs, AstExpr* rhs) return true; } + CASE(AstExprIfElse) return similar(le->condition, re->condition) && similar(le->trueExpr, re->trueExpr) && similar(le->falseExpr, re->falseExpr); else { LUAU_ASSERT(!"Unknown expression type"); diff --git a/Analysis/src/Module.cpp b/Analysis/src/Module.cpp index 817a33e9..412b78bb 100644 --- a/Analysis/src/Module.cpp +++ b/Analysis/src/Module.cpp @@ -16,7 +16,6 @@ LUAU_FASTFLAGVARIABLE(DebugLuauTrackOwningArena, false) // Remove with FFlagLuau LUAU_FASTINTVARIABLE(LuauTypeCloneRecursionLimit, 300) LUAU_FASTFLAG(LuauTypeAliasDefaults) LUAU_FASTFLAG(LuauImmutableTypes) -LUAU_FASTFLAGVARIABLE(LuauPrepopulateUnionOptionsBeforeAllocation, false) namespace Luau { @@ -379,28 +378,14 @@ void TypeCloner::operator()(const AnyTypeVar& t) void TypeCloner::operator()(const UnionTypeVar& t) { - if (FFlag::LuauPrepopulateUnionOptionsBeforeAllocation) - { - std::vector options; - options.reserve(t.options.size()); + std::vector options; + options.reserve(t.options.size()); - for (TypeId ty : t.options) - options.push_back(clone(ty, dest, seenTypes, seenTypePacks, cloneState)); + for (TypeId ty : t.options) + options.push_back(clone(ty, dest, seenTypes, seenTypePacks, cloneState)); - TypeId result = dest.addType(UnionTypeVar{std::move(options)}); - seenTypes[typeId] = result; - } - else - { - TypeId result = dest.addType(UnionTypeVar{}); - seenTypes[typeId] = result; - - UnionTypeVar* option = getMutable(result); - LUAU_ASSERT(option != nullptr); - - for (TypeId ty : t.options) - option->options.push_back(clone(ty, dest, seenTypes, seenTypePacks, cloneState)); - } + TypeId result = dest.addType(UnionTypeVar{std::move(options)}); + seenTypes[typeId] = result; } void TypeCloner::operator()(const IntersectionTypeVar& t) diff --git a/Analysis/src/Transpiler.cpp b/Analysis/src/Transpiler.cpp index 54bd0d5e..a02d396b 100644 --- a/Analysis/src/Transpiler.cpp +++ b/Analysis/src/Transpiler.cpp @@ -1153,6 +1153,14 @@ struct Printer writer.symbol(")"); } } + else if (const auto& a = typeAnnotation.as()) + { + writer.keyword(a->value ? "true" : "false"); + } + else if (const auto& a = typeAnnotation.as()) + { + writer.string(std::string_view(a->value.data, a->value.size)); + } else if (typeAnnotation.is()) { writer.symbol("%error-type%"); diff --git a/Analysis/src/TypeInfer.cpp b/Analysis/src/TypeInfer.cpp index c29699b7..faf60eb3 100644 --- a/Analysis/src/TypeInfer.cpp +++ b/Analysis/src/TypeInfer.cpp @@ -29,7 +29,6 @@ LUAU_FASTFLAGVARIABLE(DebugLuauFreezeDuringUnification, false) LUAU_FASTFLAGVARIABLE(LuauRecursiveTypeParameterRestriction, false) LUAU_FASTFLAGVARIABLE(LuauGenericFunctionsDontCacheTypeParams, false) LUAU_FASTFLAGVARIABLE(LuauImmutableTypes, false) -LUAU_FASTFLAGVARIABLE(LuauNoSealedTypeMod, false) LUAU_FASTFLAGVARIABLE(LuauQuantifyInPlace2, false) LUAU_FASTFLAGVARIABLE(LuauSealExports, false) LUAU_FASTFLAGVARIABLE(LuauSingletonTypes, false) @@ -38,14 +37,14 @@ LUAU_FASTFLAGVARIABLE(LuauTypeAliasDefaults, false) LUAU_FASTFLAGVARIABLE(LuauExpectedTypesOfProperties, false) LUAU_FASTFLAGVARIABLE(LuauErrorRecoveryType, false) LUAU_FASTFLAGVARIABLE(LuauPropertiesGetExpectedType, false) -LUAU_FASTFLAGVARIABLE(LuauProperTypeLevels, false) LUAU_FASTFLAGVARIABLE(LuauAscribeCorrectLevelToInferredProperitesOfFreeTables, false) -LUAU_FASTFLAG(LuauUnionTagMatchFix) LUAU_FASTFLAGVARIABLE(LuauUnsealedTableLiteral, false) LUAU_FASTFLAGVARIABLE(LuauTwoPassAliasDefinitionFix, false) LUAU_FASTFLAGVARIABLE(LuauAssertStripsFalsyTypes, false) LUAU_FASTFLAGVARIABLE(LuauReturnAnyInsteadOfICE, false) // Eventually removed as false. LUAU_FASTFLAGVARIABLE(LuauAnotherTypeLevelFix, false) +LUAU_FASTFLAG(LuauWidenIfSupertypeIsFree) +LUAU_FASTFLAGVARIABLE(LuauDoNotTryToReduce, false) namespace Luau { @@ -1125,7 +1124,7 @@ void TypeChecker::check(const ScopePtr& scope, TypeId ty, const ScopePtr& funSco ty = follow(ty); - if (tableSelf && (FFlag::LuauNoSealedTypeMod ? tableSelf->state != TableState::Sealed : !selfTy->persistent)) + if (tableSelf && tableSelf->state != TableState::Sealed) tableSelf->props[indexName->index.value] = {ty, /* deprecated */ false, {}, indexName->indexLocation}; const FunctionTypeVar* funTy = get(ty); @@ -1138,7 +1137,7 @@ void TypeChecker::check(const ScopePtr& scope, TypeId ty, const ScopePtr& funSco checkFunctionBody(funScope, ty, *function.func); - if (tableSelf && (FFlag::LuauNoSealedTypeMod ? tableSelf->state != TableState::Sealed : !selfTy->persistent)) + if (tableSelf && tableSelf->state != TableState::Sealed) tableSelf->props[indexName->index.value] = { follow(quantify(funScope, ty, indexName->indexLocation)), /* deprecated */ false, {}, indexName->indexLocation}; } @@ -1210,8 +1209,7 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatTypeAlias& typealias { ScopePtr aliasScope = childScope(scope, typealias.location); aliasScope->level = scope->level.incr(); - if (FFlag::LuauProperTypeLevels) - aliasScope->level.subLevel = subLevel; + aliasScope->level.subLevel = subLevel; auto [generics, genericPacks] = createGenericTypes(aliasScope, scope->level, typealias, typealias.generics, typealias.genericPacks, /* useCache = */ true); @@ -1624,7 +1622,7 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprIn std::optional TypeChecker::findTablePropertyRespectingMeta(TypeId lhsType, Name name, const Location& location) { ErrorVec errors; - auto result = Luau::findTablePropertyRespectingMeta(errors, globalScope, lhsType, name, location); + auto result = Luau::findTablePropertyRespectingMeta(errors, lhsType, name, location); reportErrors(errors); return result; } @@ -1632,7 +1630,7 @@ std::optional TypeChecker::findTablePropertyRespectingMeta(TypeId lhsTyp std::optional TypeChecker::findMetatableEntry(TypeId type, std::string entry, const Location& location) { ErrorVec errors; - auto result = Luau::findMetatableEntry(errors, globalScope, type, entry, location); + auto result = Luau::findMetatableEntry(errors, type, entry, location); reportErrors(errors); return result; } @@ -1751,13 +1749,23 @@ std::optional TypeChecker::getIndexTypeFromType( return std::nullopt; } - // TODO(amccord): Write some logic to correctly handle intersections. CLI-34659 - std::vector result = reduceUnion(parts); + if (FFlag::LuauDoNotTryToReduce) + { + if (parts.size() == 1) + return parts[0]; - if (result.size() == 1) - return result[0]; + return addType(IntersectionTypeVar{std::move(parts)}); // Not at all correct. + } + else + { + // TODO(amccord): Write some logic to correctly handle intersections. CLI-34659 + std::vector result = reduceUnion(parts); - return addType(IntersectionTypeVar{result}); + if (result.size() == 1) + return result[0]; + + return addType(IntersectionTypeVar{result}); + } } if (addErrors) @@ -2823,10 +2831,7 @@ TypeId TypeChecker::checkLValueBinding(const ScopePtr& scope, const AstExprIndex TypeId TypeChecker::checkFunctionName(const ScopePtr& scope, AstExpr& funName, TypeLevel level) { auto freshTy = [&]() { - if (FFlag::LuauProperTypeLevels) - return freshType(level); - else - return freshType(scope); + return freshType(level); }; if (auto globalName = funName.as()) @@ -3790,7 +3795,14 @@ std::optional> TypeChecker::checkCallOverload(const Scope // has been instantiated, so is a monotype. We can therefore // unify it with a monomorphic function. TypeId r = addType(FunctionTypeVar(scope->level, argPack, retPack)); - unify(fn, r, expr.location); + if (FFlag::LuauWidenIfSupertypeIsFree) + { + UnifierOptions options; + options.isFunctionCall = true; + unify(r, fn, expr.location, options); + } + else + unify(fn, r, expr.location); return {{retPack}}; } @@ -4243,9 +4255,15 @@ TypeId TypeChecker::anyIfNonstrict(TypeId ty) const } bool TypeChecker::unify(TypeId subTy, TypeId superTy, const Location& location) +{ + UnifierOptions options; + return unify(subTy, superTy, location, options); +} + +bool TypeChecker::unify(TypeId subTy, TypeId superTy, const Location& location, const UnifierOptions& options) { Unifier state = mkUnifier(location); - state.tryUnify(subTy, superTy); + state.tryUnify(subTy, superTy, options.isFunctionCall); if (FFlag::LuauUseCommittingTxnLog) state.log.commit(); @@ -4654,7 +4672,7 @@ void TypeChecker::diagnoseMissingTableKey(UnknownProperty* utk, TypeErrorData& d } }; - if (auto ttv = getTableType(FFlag::LuauUnionTagMatchFix ? utk->table : follow(utk->table))) + if (auto ttv = getTableType(utk->table)) accumulate(ttv->props); else if (auto ctv = get(follow(utk->table))) { @@ -4691,8 +4709,7 @@ ScopePtr TypeChecker::childFunctionScope(const ScopePtr& parent, const Location& ScopePtr TypeChecker::childScope(const ScopePtr& parent, const Location& location) { ScopePtr scope = std::make_shared(parent); - if (FFlag::LuauProperTypeLevels) - scope->level = parent->level; + scope->level = parent->level; scope->varargPack = parent->varargPack; currentModule->scopes.push_back(std::make_pair(location, scope)); @@ -4724,7 +4741,7 @@ void TypeChecker::merge(RefinementMap& l, const RefinementMap& r) Unifier TypeChecker::mkUnifier(const Location& location) { - return Unifier{¤tModule->internalTypes, currentModule->mode, globalScope, location, Variance::Covariant, unifierState}; + return Unifier{¤tModule->internalTypes, currentModule->mode, location, Variance::Covariant, unifierState}; } TypeId TypeChecker::freshType(const ScopePtr& scope) @@ -5444,7 +5461,7 @@ GenericTypeDefinitions TypeChecker::createGenericTypes(const ScopePtr& scope, st void TypeChecker::refineLValue(const LValue& lvalue, RefinementMap& refis, const ScopePtr& scope, TypeIdPredicate predicate) { - LUAU_ASSERT(FFlag::LuauDiscriminableUnions2); + LUAU_ASSERT(FFlag::LuauDiscriminableUnions2 || FFlag::LuauAssertStripsFalsyTypes); const LValue* target = &lvalue; std::optional key; // If set, we know we took the base of the lvalue path and should be walking down each option of the base's type. diff --git a/Analysis/src/TypeUtils.cpp b/Analysis/src/TypeUtils.cpp index 593b54c8..c2435890 100644 --- a/Analysis/src/TypeUtils.cpp +++ b/Analysis/src/TypeUtils.cpp @@ -10,7 +10,7 @@ LUAU_FASTFLAGVARIABLE(LuauTerminateCyclicMetatableIndexLookup, false) namespace Luau { -std::optional findMetatableEntry(ErrorVec& errors, const ScopePtr& globalScope, TypeId type, std::string entry, Location location) +std::optional findMetatableEntry(ErrorVec& errors, TypeId type, std::string entry, Location location) { type = follow(type); @@ -37,7 +37,7 @@ std::optional findMetatableEntry(ErrorVec& errors, const ScopePtr& globa return std::nullopt; } -std::optional findTablePropertyRespectingMeta(ErrorVec& errors, const ScopePtr& globalScope, TypeId ty, Name name, Location location) +std::optional findTablePropertyRespectingMeta(ErrorVec& errors, TypeId ty, Name name, Location location) { if (get(ty)) return ty; @@ -49,7 +49,7 @@ std::optional findTablePropertyRespectingMeta(ErrorVec& errors, const Sc return it->second.type; } - std::optional mtIndex = findMetatableEntry(errors, globalScope, ty, "__index", location); + std::optional mtIndex = findMetatableEntry(errors, ty, "__index", location); int count = 0; while (mtIndex) { @@ -82,7 +82,7 @@ std::optional findTablePropertyRespectingMeta(ErrorVec& errors, const Sc else errors.push_back(TypeError{location, GenericError{"__index should either be a function or table. Got " + toString(index)}}); - mtIndex = findMetatableEntry(errors, globalScope, *mtIndex, "__index", location); + mtIndex = findMetatableEntry(errors, *mtIndex, "__index", location); } return std::nullopt; diff --git a/Analysis/src/TypeVar.cpp b/Analysis/src/TypeVar.cpp index b2358c27..a1dcfdbe 100644 --- a/Analysis/src/TypeVar.cpp +++ b/Analysis/src/TypeVar.cpp @@ -23,9 +23,7 @@ LUAU_FASTFLAG(DebugLuauFreezeArena) LUAU_FASTINTVARIABLE(LuauTypeMaximumStringifierLength, 500) LUAU_FASTINTVARIABLE(LuauTableTypeMaximumStringifierLength, 0) LUAU_FASTINT(LuauTypeInferRecursionLimit) -LUAU_FASTFLAGVARIABLE(LuauRefactorTypeVarQuestions, false) LUAU_FASTFLAG(LuauErrorRecoveryType) -LUAU_FASTFLAG(LuauUnionTagMatchFix) LUAU_FASTFLAG(LuauDiscriminableUnions2) namespace Luau @@ -145,20 +143,13 @@ bool isNil(TypeId ty) bool isBoolean(TypeId ty) { - if (FFlag::LuauRefactorTypeVarQuestions) - { - if (isPrim(ty, PrimitiveTypeVar::Boolean) || get(get(follow(ty)))) - return true; + if (isPrim(ty, PrimitiveTypeVar::Boolean) || get(get(follow(ty)))) + return true; - if (auto utv = get(follow(ty))) - return std::all_of(begin(utv), end(utv), isBoolean); + if (auto utv = get(follow(ty))) + return std::all_of(begin(utv), end(utv), isBoolean); - return false; - } - else - { - return isPrim(ty, PrimitiveTypeVar::Boolean); - } + return false; } bool isNumber(TypeId ty) @@ -168,20 +159,13 @@ bool isNumber(TypeId ty) bool isString(TypeId ty) { - if (FFlag::LuauRefactorTypeVarQuestions) - { - if (isPrim(ty, PrimitiveTypeVar::String) || get(get(follow(ty)))) - return true; + if (isPrim(ty, PrimitiveTypeVar::String) || get(get(follow(ty)))) + return true; - if (auto utv = get(follow(ty))) - return std::all_of(begin(utv), end(utv), isString); + if (auto utv = get(follow(ty))) + return std::all_of(begin(utv), end(utv), isString); - return false; - } - else - { - return isPrim(ty, PrimitiveTypeVar::String); - } + return false; } bool isThread(TypeId ty) @@ -194,45 +178,11 @@ bool isOptional(TypeId ty) if (isNil(ty)) return true; - if (FFlag::LuauRefactorTypeVarQuestions) - { - auto utv = get(follow(ty)); - if (!utv) - return false; - - return std::any_of(begin(utv), end(utv), isNil); - } - else - { - std::unordered_set seen; - std::deque queue{ty}; - while (!queue.empty()) - { - TypeId current = follow(queue.front()); - queue.pop_front(); - - if (seen.count(current)) - continue; - - seen.insert(current); - - if (isNil(current)) - return true; - - if (auto u = get(current)) - { - for (TypeId option : u->options) - { - if (isNil(option)) - return true; - - queue.push_back(option); - } - } - } - + auto utv = get(follow(ty)); + if (!utv) return false; - } + + return std::any_of(begin(utv), end(utv), isNil); } bool isTableIntersection(TypeId ty) @@ -263,38 +213,24 @@ std::optional getMetatable(TypeId type) return mtType->metatable; else if (const ClassTypeVar* classType = get(type)) return classType->metatable; - else if (FFlag::LuauRefactorTypeVarQuestions) + else if (isString(type)) { - if (isString(type)) - { - auto ptv = get(getSingletonTypes().stringType); - LUAU_ASSERT(ptv && ptv->metatable); - return ptv->metatable; - } - else - return std::nullopt; - } - else - { - if (const PrimitiveTypeVar* primitiveType = get(type); primitiveType && primitiveType->metatable) - { - LUAU_ASSERT(primitiveType->type == PrimitiveTypeVar::String); - return primitiveType->metatable; - } - else - return std::nullopt; + auto ptv = get(getSingletonTypes().stringType); + LUAU_ASSERT(ptv && ptv->metatable); + return ptv->metatable; } + + return std::nullopt; } const TableTypeVar* getTableType(TypeId type) { - if (FFlag::LuauUnionTagMatchFix) - type = follow(type); + type = follow(type); if (const TableTypeVar* ttv = get(type)) return ttv; else if (const MetatableTypeVar* mtv = get(type)) - return get(FFlag::LuauUnionTagMatchFix ? follow(mtv->table) : mtv->table); + return get(follow(mtv->table)); else return nullptr; } @@ -311,7 +247,7 @@ const std::string* getName(TypeId type) { if (mtv->syntheticName) return &*mtv->syntheticName; - type = FFlag::LuauUnionTagMatchFix ? follow(mtv->table) : mtv->table; + type = follow(mtv->table); } if (auto ttv = get(type)) diff --git a/Analysis/src/Unifier.cpp b/Analysis/src/Unifier.cpp index 322f6ebf..d0eba013 100644 --- a/Analysis/src/Unifier.cpp +++ b/Analysis/src/Unifier.cpp @@ -21,9 +21,8 @@ LUAU_FASTFLAGVARIABLE(LuauTableSubtypingVariance2, false); LUAU_FASTFLAGVARIABLE(LuauTableUnificationEarlyTest, false) LUAU_FASTFLAG(LuauSingletonTypes) LUAU_FASTFLAG(LuauErrorRecoveryType); -LUAU_FASTFLAG(LuauProperTypeLevels); -LUAU_FASTFLAGVARIABLE(LuauUnionTagMatchFix, false) LUAU_FASTFLAGVARIABLE(LuauFollowWithCommittingTxnLogInAnyUnification, false) +LUAU_FASTFLAGVARIABLE(LuauWidenIfSupertypeIsFree, false) namespace Luau { @@ -122,7 +121,7 @@ struct PromoteTypeLevels } }; -void promoteTypeLevels(DEPRECATED_TxnLog& DEPRECATED_log, TxnLog& log, const TypeArena* typeArena, TypeLevel minLevel, TypeId ty) +static void promoteTypeLevels(DEPRECATED_TxnLog& DEPRECATED_log, TxnLog& log, const TypeArena* typeArena, TypeLevel minLevel, TypeId ty) { // Type levels of types from other modules are already global, so we don't need to promote anything inside if (FFlag::LuauImmutableTypes && ty->owningArena != typeArena) @@ -133,6 +132,7 @@ void promoteTypeLevels(DEPRECATED_TxnLog& DEPRECATED_log, TxnLog& log, const Typ visitTypeVarOnce(ty, ptl, seen); } +// TODO: use this and make it static. void promoteTypeLevels(DEPRECATED_TxnLog& DEPRECATED_log, TxnLog& log, const TypeArena* typeArena, TypeLevel minLevel, TypePackId tp) { // Type levels of types from other modules are already global, so we don't need to promote anything inside @@ -247,6 +247,48 @@ struct SkipCacheForType bool result = false; }; +bool Widen::isDirty(TypeId ty) +{ + return FFlag::LuauUseCommittingTxnLog ? log->is(ty) : bool(get(ty)); +} + +bool Widen::isDirty(TypePackId) +{ + return false; +} + +TypeId Widen::clean(TypeId ty) +{ + LUAU_ASSERT(isDirty(ty)); + auto stv = FFlag::LuauUseCommittingTxnLog ? log->getMutable(ty) : getMutable(ty); + LUAU_ASSERT(stv); + + if (get(stv)) + return getSingletonTypes().stringType; + else + { + // If this assert trips, it's likely we now have number singletons. + LUAU_ASSERT(get(stv)); + return getSingletonTypes().booleanType; + } +} + +TypePackId Widen::clean(TypePackId) +{ + throw std::runtime_error("Widen attempted to clean a dirty type pack?"); +} + +bool Widen::ignoreChildren(TypeId ty) +{ + // Sometimes we unify ("hi") -> free1 with (free2) -> free3, so don't ignore functions. + // TODO: should we be doing this? we would need to rework how checkCallOverload does the unification. + if (FFlag::LuauUseCommittingTxnLog ? log->is(ty) : bool(get(ty))) + return false; + + // We only care about unions. + return !(FFlag::LuauUseCommittingTxnLog ? log->is(ty) : bool(get(ty))); +} + static std::optional hasUnificationTooComplex(const ErrorVec& errors) { auto isUnificationTooComplex = [](const TypeError& te) { @@ -263,43 +305,22 @@ static std::optional hasUnificationTooComplex(const ErrorVec& errors) // Used for tagged union matching heuristic, returns first singleton type field static std::optional> getTableMatchTag(TypeId type) { - if (FFlag::LuauUnionTagMatchFix) + if (auto ttv = getTableType(type)) { - if (auto ttv = getTableType(type)) + for (auto&& [name, prop] : ttv->props) { - for (auto&& [name, prop] : ttv->props) - { - if (auto sing = get(follow(prop.type))) - return {{name, sing}}; - } - } - } - else - { - type = follow(type); - - if (auto ttv = get(type)) - { - for (auto&& [name, prop] : ttv->props) - { - if (auto sing = get(follow(prop.type))) - return {{name, sing}}; - } - } - else if (auto mttv = get(type)) - { - return getTableMatchTag(mttv->table); + if (auto sing = get(follow(prop.type))) + return {{name, sing}}; } } return std::nullopt; } -Unifier::Unifier(TypeArena* types, Mode mode, ScopePtr globalScope, const Location& location, Variance variance, UnifierSharedState& sharedState, +Unifier::Unifier(TypeArena* types, Mode mode, const Location& location, Variance variance, UnifierSharedState& sharedState, TxnLog* parentLog) : types(types) , mode(mode) - , globalScope(std::move(globalScope)) , log(parentLog) , location(location) , variance(variance) @@ -308,11 +329,10 @@ Unifier::Unifier(TypeArena* types, Mode mode, ScopePtr globalScope, const Locati LUAU_ASSERT(sharedState.iceHandler); } -Unifier::Unifier(TypeArena* types, Mode mode, ScopePtr globalScope, std::vector>* sharedSeen, const Location& location, +Unifier::Unifier(TypeArena* types, Mode mode, std::vector>* sharedSeen, const Location& location, Variance variance, UnifierSharedState& sharedState, TxnLog* parentLog) : types(types) , mode(mode) - , globalScope(std::move(globalScope)) , DEPRECATED_log(sharedSeen) , log(parentLog, sharedSeen) , location(location) @@ -435,6 +455,8 @@ void Unifier::tryUnify_(TypeId subTy, TypeId superTy, bool isFunctionCall, bool } else if (superFree) { + TypeLevel superLevel = superFree->level; + occursCheck(superTy, subTy); bool occursFailed = false; if (FFlag::LuauUseCommittingTxnLog) @@ -442,8 +464,6 @@ void Unifier::tryUnify_(TypeId subTy, TypeId superTy, bool isFunctionCall, bool else occursFailed = bool(get(superTy)); - TypeLevel superLevel = superFree->level; - // Unification can't change the level of a generic. auto subGeneric = FFlag::LuauUseCommittingTxnLog ? log.getMutable(subTy) : get(subTy); if (subGeneric && !subGeneric->level.subsumes(superLevel)) @@ -459,20 +479,14 @@ void Unifier::tryUnify_(TypeId subTy, TypeId superTy, bool isFunctionCall, bool if (FFlag::LuauUseCommittingTxnLog) { promoteTypeLevels(DEPRECATED_log, log, types, superLevel, subTy); - log.replace(superTy, BoundTypeVar(subTy)); + log.replace(superTy, BoundTypeVar(widen(subTy))); } else { - if (FFlag::LuauProperTypeLevels) - promoteTypeLevels(DEPRECATED_log, log, types, superLevel, subTy); - else if (auto subLevel = getMutableLevel(subTy)) - { - if (!subLevel->subsumes(superFree->level)) - *subLevel = superFree->level; - } + promoteTypeLevels(DEPRECATED_log, log, types, superLevel, subTy); DEPRECATED_log(superTy); - *asMutable(superTy) = BoundTypeVar(subTy); + *asMutable(superTy) = BoundTypeVar(widen(subTy)); } } @@ -507,16 +521,7 @@ void Unifier::tryUnify_(TypeId subTy, TypeId superTy, bool isFunctionCall, bool } else { - if (FFlag::LuauProperTypeLevels) - promoteTypeLevels(DEPRECATED_log, log, types, subLevel, superTy); - else if (auto superLevel = getMutableLevel(superTy)) - { - if (!superLevel->subsumes(subFree->level)) - { - DEPRECATED_log(superTy); - *superLevel = subFree->level; - } - } + promoteTypeLevels(DEPRECATED_log, log, types, subLevel, superTy); DEPRECATED_log(subTy); *asMutable(subTy) = BoundTypeVar(superTy); @@ -2064,6 +2069,17 @@ void Unifier::tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) } } +TypeId Unifier::widen(TypeId ty) +{ + if (!FFlag::LuauWidenIfSupertypeIsFree) + return ty; + + Widen widen{types}; + std::optional result = widen.substitute(ty); + // TODO: what does it mean for substitution to fail to widen? + return result.value_or(ty); +} + TypeId Unifier::deeplyOptional(TypeId ty, std::unordered_map seen) { ty = follow(ty); @@ -2915,7 +2931,7 @@ void Unifier::tryUnifyWithAny(TypePackId subTy, TypePackId anyTp) std::optional Unifier::findTablePropertyRespectingMeta(TypeId lhsType, Name name) { - return Luau::findTablePropertyRespectingMeta(errors, globalScope, lhsType, name, location); + return Luau::findTablePropertyRespectingMeta(errors, lhsType, name, location); } void Unifier::occursCheck(TypeId needle, TypeId haystack) @@ -3096,9 +3112,9 @@ void Unifier::occursCheck(DenseHashSet& seen, TypePackId needle, Typ Unifier Unifier::makeChildUnifier() { if (FFlag::LuauUseCommittingTxnLog) - return Unifier{types, mode, globalScope, log.sharedSeen, location, variance, sharedState, &log}; + return Unifier{types, mode, log.sharedSeen, location, variance, sharedState, &log}; else - return Unifier{types, mode, globalScope, DEPRECATED_log.sharedSeen, location, variance, sharedState, &log}; + return Unifier{types, mode, DEPRECATED_log.sharedSeen, location, variance, sharedState, &log}; } bool Unifier::isNonstrictMode() const diff --git a/Ast/src/Parser.cpp b/Ast/src/Parser.cpp index 235d6349..8767daa0 100644 --- a/Ast/src/Parser.cpp +++ b/Ast/src/Parser.cpp @@ -12,7 +12,6 @@ LUAU_FASTINTVARIABLE(LuauRecursionLimit, 1000) LUAU_FASTINTVARIABLE(LuauParseErrorLimit, 100) LUAU_FASTFLAGVARIABLE(LuauParseSingletonTypes, false) LUAU_FASTFLAGVARIABLE(LuauParseTypeAliasDefaults, false) -LUAU_FASTFLAGVARIABLE(LuauParseRecoverTypePackEllipsis, false) LUAU_FASTFLAGVARIABLE(LuauParseAllHotComments, false) LUAU_FASTFLAGVARIABLE(LuauTableFieldFunctionDebugname, false) @@ -2372,11 +2371,11 @@ std::pair, AstArray> Parser::parseG { Location nameLocation = lexer.current().location; AstName name = parseName().name; - if (lexer.current().type == Lexeme::Dot3 || (FFlag::LuauParseRecoverTypePackEllipsis && seenPack)) + if (lexer.current().type == Lexeme::Dot3 || seenPack) { seenPack = true; - if (FFlag::LuauParseRecoverTypePackEllipsis && lexer.current().type != Lexeme::Dot3) + if (lexer.current().type != Lexeme::Dot3) report(lexer.current().location, "Generic types come before generic type packs"); else nextLexeme(); @@ -2414,9 +2413,6 @@ std::pair, AstArray> Parser::parseG } else { - if (!FFlag::LuauParseRecoverTypePackEllipsis && seenPack) - report(lexer.current().location, "Generic types come before generic type packs"); - if (withDefaultValues && lexer.current().type == '=') { seenDefault = true; diff --git a/VM/include/lua.h b/VM/include/lua.h index 5f39cb3d..af0e2835 100644 --- a/VM/include/lua.h +++ b/VM/include/lua.h @@ -44,7 +44,7 @@ typedef int (*lua_Continuation)(lua_State* L, int status); ** prototype for memory-allocation functions */ -typedef void* (*lua_Alloc)(lua_State* L, void* ud, void* ptr, size_t osize, size_t nsize); +typedef void* (*lua_Alloc)(void* ud, void* ptr, size_t osize, size_t nsize); /* non-return type */ #define l_noret void LUA_NORETURN diff --git a/VM/src/ldo.cpp b/VM/src/ldo.cpp index d87f0661..b5ae496b 100644 --- a/VM/src/ldo.cpp +++ b/VM/src/ldo.cpp @@ -151,8 +151,7 @@ l_noret luaD_throw(lua_State* L, int errcode) static void correctstack(lua_State* L, TValue* oldstack) { L->top = (L->top - oldstack) + L->stack; - // TODO (FFlagLuauGcPagedSweep): 'next' type will change after removal of the flag and the cast will not be required - for (UpVal* up = L->openupval; up != NULL; up = (UpVal*)up->next) + for (UpVal* up = L->openupval; up != NULL; up = up->u.l.threadnext) up->v = (up->v - oldstack) + L->stack; for (CallInfo* ci = L->base_ci; ci <= L->ci; ci++) { diff --git a/VM/src/lfunc.cpp b/VM/src/lfunc.cpp index 582d4627..66447a95 100644 --- a/VM/src/lfunc.cpp +++ b/VM/src/lfunc.cpp @@ -6,13 +6,10 @@ #include "lmem.h" #include "lgc.h" -LUAU_FASTFLAGVARIABLE(LuauNoDirectUpvalRemoval, false) -LUAU_FASTFLAG(LuauGcPagedSweep) - Proto* luaF_newproto(lua_State* L) { Proto* f = luaM_newgco(L, Proto, sizeof(Proto), L->activememcat); - luaC_link(L, f, LUA_TPROTO); + luaC_init(L, f, LUA_TPROTO); f->k = NULL; f->sizek = 0; f->p = NULL; @@ -40,7 +37,7 @@ Proto* luaF_newproto(lua_State* L) Closure* luaF_newLclosure(lua_State* L, int nelems, Table* e, Proto* p) { Closure* c = luaM_newgco(L, Closure, sizeLclosure(nelems), L->activememcat); - luaC_link(L, c, LUA_TFUNCTION); + luaC_init(L, c, LUA_TFUNCTION); c->isC = 0; c->env = e; c->nupvalues = cast_byte(nelems); @@ -55,7 +52,7 @@ Closure* luaF_newLclosure(lua_State* L, int nelems, Table* e, Proto* p) Closure* luaF_newCclosure(lua_State* L, int nelems, Table* e) { Closure* c = luaM_newgco(L, Closure, sizeCclosure(nelems), L->activememcat); - luaC_link(L, c, LUA_TFUNCTION); + luaC_init(L, c, LUA_TFUNCTION); c->isC = 1; c->env = e; c->nupvalues = cast_byte(nelems); @@ -82,8 +79,7 @@ UpVal* luaF_findupval(lua_State* L, StkId level) return p; } - // TODO (FFlagLuauGcPagedSweep): 'next' type will change after removal of the flag and the cast will not be required - pp = (UpVal**)&p->next; + pp = &p->u.l.threadnext; } UpVal* uv = luaM_newgco(L, UpVal, sizeof(UpVal), L->activememcat); /* not found: create a new one */ @@ -94,19 +90,10 @@ UpVal* luaF_findupval(lua_State* L, StkId level) // chain the upvalue in the threads open upvalue list at the proper position UpVal* next = *pp; - - // TODO (FFlagLuauGcPagedSweep): 'next' type will change after removal of the flag and the cast will not be required - uv->next = (GCObject*)next; - - if (FFlag::LuauGcPagedSweep) - { - uv->u.l.threadprev = pp; - if (next) - { - // TODO (FFlagLuauGcPagedSweep): 'next' type will change after removal of the flag and the cast will not be required - next->u.l.threadprev = (UpVal**)&uv->next; - } - } + uv->u.l.threadnext = next; + uv->u.l.threadprev = pp; + if (next) + next->u.l.threadprev = &uv->u.l.threadnext; *pp = uv; @@ -125,15 +112,11 @@ void luaF_unlinkupval(UpVal* uv) uv->u.l.next->u.l.prev = uv->u.l.prev; uv->u.l.prev->u.l.next = uv->u.l.next; - if (FFlag::LuauGcPagedSweep) - { - // unlink upvalue from the thread open upvalue list - // TODO (FFlagLuauGcPagedSweep): 'next' type will change after removal of the flag and this and the following cast will not be required - *uv->u.l.threadprev = (UpVal*)uv->next; + // unlink upvalue from the thread open upvalue list + *uv->u.l.threadprev = uv->u.l.threadnext; - if (UpVal* next = (UpVal*)uv->next) - next->u.l.threadprev = uv->u.l.threadprev; - } + if (UpVal* next = uv->u.l.threadnext) + next->u.l.threadprev = uv->u.l.threadprev; } void luaF_freeupval(lua_State* L, UpVal* uv, lua_Page* page) @@ -145,34 +128,27 @@ void luaF_freeupval(lua_State* L, UpVal* uv, lua_Page* page) void luaF_close(lua_State* L, StkId level) { - global_State* g = L->global; // TODO: remove with FFlagLuauNoDirectUpvalRemoval + global_State* g = L->global; UpVal* uv; while (L->openupval != NULL && (uv = L->openupval)->v >= level) { GCObject* o = obj2gco(uv); LUAU_ASSERT(!isblack(o) && uv->v != &uv->u.value); - if (!FFlag::LuauGcPagedSweep) - L->openupval = (UpVal*)uv->next; /* remove from `open' list */ + // by removing the upvalue from global/thread open upvalue lists, L->openupval will be pointing to the next upvalue + luaF_unlinkupval(uv); - if (FFlag::LuauGcPagedSweep && isdead(g, o)) + if (isdead(g, o)) { - // by removing the upvalue from global/thread open upvalue lists, L->openupval will be pointing to the next upvalue - luaF_unlinkupval(uv); // close the upvalue without copying the dead data so that luaF_freeupval will not unlink again uv->v = &uv->u.value; } - else if (!FFlag::LuauNoDirectUpvalRemoval && isdead(g, o)) - { - luaF_freeupval(L, uv, NULL); /* free upvalue */ - } else { - // by removing the upvalue from global/thread open upvalue lists, L->openupval will be pointing to the next upvalue - luaF_unlinkupval(uv); setobj(L, &uv->u.value, uv->v); - uv->v = &uv->u.value; /* now current value lives here */ - luaC_linkupval(L, uv); /* link upvalue into `gcroot' list */ + uv->v = &uv->u.value; + // GC state of a new closed upvalue has to be initialized + luaC_initupval(L, uv); } } } diff --git a/VM/src/lgc.cpp b/VM/src/lgc.cpp index 724b24b2..8c3a2029 100644 --- a/VM/src/lgc.cpp +++ b/VM/src/lgc.cpp @@ -13,8 +13,6 @@ #include -LUAU_FASTFLAGVARIABLE(LuauGcPagedSweep, false) - #define GC_SWEEPMAX 40 #define GC_SWEEPCOST 10 #define GC_SWEEPPAGESTEPCOST 4 @@ -64,7 +62,6 @@ static void recordGcStateTime(global_State* g, int startgcstate, double seconds, case GCSatomic: g->gcstats.currcycle.atomictime += seconds; break; - case GCSsweepstring: case GCSsweep: g->gcstats.currcycle.sweeptime += seconds; break; @@ -490,65 +487,6 @@ static void freeobj(lua_State* L, GCObject* o, lua_Page* page) } } -#define sweepwholelist(L, p) sweeplist(L, p, SIZE_MAX) - -static GCObject** sweeplist(lua_State* L, GCObject** p, size_t count) -{ - LUAU_ASSERT(!FFlag::LuauGcPagedSweep); - - GCObject* curr; - global_State* g = L->global; - int deadmask = otherwhite(g); - LUAU_ASSERT(testbit(deadmask, FIXEDBIT)); /* make sure we never sweep fixed objects */ - while ((curr = *p) != NULL && count-- > 0) - { - int alive = (curr->gch.marked ^ WHITEBITS) & deadmask; - if (curr->gch.tt == LUA_TTHREAD) - { - sweepwholelist(L, (GCObject**)&gco2th(curr)->openupval); /* sweep open upvalues */ - - lua_State* th = gco2th(curr); - - if (alive) - { - resetbit(th->stackstate, THREAD_SLEEPINGBIT); - shrinkstack(th); - } - } - if (alive) - { /* not dead? */ - LUAU_ASSERT(!isdead(g, curr)); - makewhite(g, curr); /* make it white (for next cycle) */ - p = &curr->gch.next; - } - else - { /* must erase `curr' */ - LUAU_ASSERT(isdead(g, curr)); - *p = curr->gch.next; - if (curr == g->rootgc) /* is the first element of the list? */ - g->rootgc = curr->gch.next; /* adjust first */ - freeobj(L, curr, NULL); - } - } - - return p; -} - -static void deletelist(lua_State* L, GCObject** p, GCObject* limit) -{ - LUAU_ASSERT(!FFlag::LuauGcPagedSweep); - - GCObject* curr; - while ((curr = *p) != limit) - { - if (curr->gch.tt == LUA_TTHREAD) /* delete open upvalues of each thread */ - deletelist(L, (GCObject**)&gco2th(curr)->openupval, NULL); - - *p = curr->gch.next; - freeobj(L, curr, NULL); - } -} - static void shrinkbuffers(lua_State* L) { global_State* g = L->global; @@ -570,8 +508,6 @@ static void shrinkbuffersfull(lua_State* L) static bool deletegco(void* context, lua_Page* page, GCObject* gco) { - LUAU_ASSERT(FFlag::LuauGcPagedSweep); - // we are in the process of deleting everything // threads with open upvalues will attempt to close them all on removal // but those upvalues might point to stack values that were already deleted @@ -598,32 +534,13 @@ void luaC_freeall(lua_State* L) LUAU_ASSERT(L == g->mainthread); - if (FFlag::LuauGcPagedSweep) - { - luaM_visitgco(L, L, deletegco); + luaM_visitgco(L, L, deletegco); - for (int i = 0; i < g->strt.size; i++) /* free all string lists */ - LUAU_ASSERT(g->strt.hash[i] == NULL); + for (int i = 0; i < g->strt.size; i++) /* free all string lists */ + LUAU_ASSERT(g->strt.hash[i] == NULL); - LUAU_ASSERT(L->global->strt.nuse == 0); - LUAU_ASSERT(g->strbufgc == NULL); - } - else - { - LUAU_ASSERT(L->next == NULL); /* mainthread is at the end of rootgc list */ - - deletelist(L, &g->rootgc, obj2gco(L)); - - for (int i = 0; i < g->strt.size; i++) /* free all string lists */ - deletelist(L, (GCObject**)&g->strt.hash[i], NULL); - - LUAU_ASSERT(L->global->strt.nuse == 0); - deletelist(L, (GCObject**)&g->strbufgc, NULL); - - // unfortunately, when string objects are freed, the string table use count is decremented - // even when the string is a buffer that wasn't placed into the table - L->global->strt.nuse = 0; - } + LUAU_ASSERT(L->global->strt.nuse == 0); + LUAU_ASSERT(g->strbufgc == NULL); } static void markmt(global_State* g) @@ -687,26 +604,13 @@ static size_t atomic(lua_State* L) g->weak = NULL; /* flip current white */ g->currentwhite = cast_byte(otherwhite(g)); - g->sweepstrgc = 0; - - if (FFlag::LuauGcPagedSweep) - { - g->sweepgcopage = g->allgcopages; - g->gcstate = GCSsweep; - } - else - { - g->sweepgc = &g->rootgc; - g->gcstate = GCSsweepstring; - } - + g->sweepgcopage = g->allgcopages; + g->gcstate = GCSsweep; return work; } static bool sweepgco(lua_State* L, lua_Page* page, GCObject* gco) { - LUAU_ASSERT(FFlag::LuauGcPagedSweep); - global_State* g = L->global; int deadmask = otherwhite(g); @@ -740,8 +644,6 @@ static bool sweepgco(lua_State* L, lua_Page* page, GCObject* gco) // a version of generic luaM_visitpage specialized for the main sweep stage static int sweepgcopage(lua_State* L, lua_Page* page) { - LUAU_ASSERT(FFlag::LuauGcPagedSweep); - char* start; char* end; int busyBlocks; @@ -819,75 +721,29 @@ static size_t gcstep(lua_State* L, size_t limit) cost = atomic(L); /* finish mark phase */ - if (FFlag::LuauGcPagedSweep) - LUAU_ASSERT(g->gcstate == GCSsweep); - else - LUAU_ASSERT(g->gcstate == GCSsweepstring); - break; - } - case GCSsweepstring: - { - LUAU_ASSERT(!FFlag::LuauGcPagedSweep); - - while (g->sweepstrgc < g->strt.size && cost < limit) - { - sweepwholelist(L, (GCObject**)&g->strt.hash[g->sweepstrgc++]); - - cost += GC_SWEEPCOST; - } - - // nothing more to sweep? - if (g->sweepstrgc >= g->strt.size) - { - // sweep string buffer list and preserve used string count - uint32_t nuse = L->global->strt.nuse; - - sweepwholelist(L, (GCObject**)&g->strbufgc); - - L->global->strt.nuse = nuse; - - g->gcstate = GCSsweep; // end sweep-string phase - } + LUAU_ASSERT(g->gcstate == GCSsweep); break; } case GCSsweep: { - if (FFlag::LuauGcPagedSweep) + while (g->sweepgcopage && cost < limit) { - while (g->sweepgcopage && cost < limit) - { - lua_Page* next = luaM_getnextgcopage(g->sweepgcopage); // page sweep might destroy the page + lua_Page* next = luaM_getnextgcopage(g->sweepgcopage); // page sweep might destroy the page - int steps = sweepgcopage(L, g->sweepgcopage); + int steps = sweepgcopage(L, g->sweepgcopage); - g->sweepgcopage = next; - cost += steps * GC_SWEEPPAGESTEPCOST; - } - - // nothing more to sweep? - if (g->sweepgcopage == NULL) - { - // don't forget to visit main thread - sweepgco(L, NULL, obj2gco(g->mainthread)); - - shrinkbuffers(L); - g->gcstate = GCSpause; /* end collection */ - } + g->sweepgcopage = next; + cost += steps * GC_SWEEPPAGESTEPCOST; } - else + + // nothing more to sweep? + if (g->sweepgcopage == NULL) { - while (*g->sweepgc && cost < limit) - { - g->sweepgc = sweeplist(L, g->sweepgc, GC_SWEEPMAX); + // don't forget to visit main thread + sweepgco(L, NULL, obj2gco(g->mainthread)); - cost += GC_SWEEPMAX * GC_SWEEPCOST; - } - - if (*g->sweepgc == NULL) - { /* nothing more to sweep? */ - shrinkbuffers(L); - g->gcstate = GCSpause; /* end collection */ - } + shrinkbuffers(L); + g->gcstate = GCSpause; /* end collection */ } break; } @@ -1013,26 +869,18 @@ void luaC_fullgc(lua_State* L) if (g->gcstate <= GCSatomic) { /* reset sweep marks to sweep all elements (returning them to white) */ - g->sweepstrgc = 0; - if (FFlag::LuauGcPagedSweep) - g->sweepgcopage = g->allgcopages; - else - g->sweepgc = &g->rootgc; + g->sweepgcopage = g->allgcopages; /* reset other collector lists */ g->gray = NULL; g->grayagain = NULL; g->weak = NULL; - - if (FFlag::LuauGcPagedSweep) - g->gcstate = GCSsweep; - else - g->gcstate = GCSsweepstring; + g->gcstate = GCSsweep; } - LUAU_ASSERT(g->gcstate == GCSsweepstring || g->gcstate == GCSsweep); + LUAU_ASSERT(g->gcstate == GCSsweep); /* finish any pending sweep phase */ while (g->gcstate != GCSpause) { - LUAU_ASSERT(g->gcstate == GCSsweepstring || g->gcstate == GCSsweep); + LUAU_ASSERT(g->gcstate == GCSsweep); gcstep(L, SIZE_MAX); } @@ -1120,30 +968,19 @@ void luaC_barrierback(lua_State* L, Table* t) g->grayagain = o; } -void luaC_linkobj(lua_State* L, GCObject* o, uint8_t tt) +void luaC_initobj(lua_State* L, GCObject* o, uint8_t tt) { global_State* g = L->global; - if (!FFlag::LuauGcPagedSweep) - { - o->gch.next = g->rootgc; - g->rootgc = o; - } o->gch.marked = luaC_white(g); o->gch.tt = tt; o->gch.memcat = L->activememcat; } -void luaC_linkupval(lua_State* L, UpVal* uv) +void luaC_initupval(lua_State* L, UpVal* uv) { global_State* g = L->global; GCObject* o = obj2gco(uv); - if (!FFlag::LuauGcPagedSweep) - { - o->gch.next = g->rootgc; /* link upvalue into `rootgc' list */ - g->rootgc = o; - } - if (isgray(o)) { if (keepinvariant(g)) @@ -1221,9 +1058,6 @@ const char* luaC_statename(int state) case GCSatomic: return "atomic"; - case GCSsweepstring: - return "sweepstring"; - case GCSsweep: return "sweep"; diff --git a/VM/src/lgc.h b/VM/src/lgc.h index 2acb5d8a..253e269f 100644 --- a/VM/src/lgc.h +++ b/VM/src/lgc.h @@ -13,9 +13,7 @@ #define GCSpropagate 1 #define GCSpropagateagain 2 #define GCSatomic 3 -// TODO: remove with FFlagLuauGcPagedSweep -#define GCSsweepstring 4 -#define GCSsweep 5 +#define GCSsweep 4 /* ** macro to tell when main invariant (white objects cannot point to black @@ -132,13 +130,13 @@ luaC_wakethread(L); \ } -#define luaC_link(L, o, tt) luaC_linkobj(L, cast_to(GCObject*, (o)), tt) +#define luaC_init(L, o, tt) luaC_initobj(L, cast_to(GCObject*, (o)), tt) LUAI_FUNC void luaC_freeall(lua_State* L); LUAI_FUNC void luaC_step(lua_State* L, bool assist); LUAI_FUNC void luaC_fullgc(lua_State* L); -LUAI_FUNC void luaC_linkobj(lua_State* L, GCObject* o, uint8_t tt); -LUAI_FUNC void luaC_linkupval(lua_State* L, UpVal* uv); +LUAI_FUNC void luaC_initobj(lua_State* L, GCObject* o, uint8_t tt); +LUAI_FUNC void luaC_initupval(lua_State* L, UpVal* uv); LUAI_FUNC void luaC_barrierupval(lua_State* L, GCObject* v); LUAI_FUNC void luaC_barrierf(lua_State* L, GCObject* o, GCObject* v); LUAI_FUNC void luaC_barriertable(lua_State* L, Table* t, GCObject* v); diff --git a/VM/src/lgcdebug.cpp b/VM/src/lgcdebug.cpp index 30242e52..2b38619b 100644 --- a/VM/src/lgcdebug.cpp +++ b/VM/src/lgcdebug.cpp @@ -13,8 +13,6 @@ #include #include -LUAU_FASTFLAG(LuauGcPagedSweep) - static void validateobjref(global_State* g, GCObject* f, GCObject* t) { LUAU_ASSERT(!isdead(g, t)); @@ -104,8 +102,7 @@ static void validatestack(global_State* g, lua_State* l) if (l->namecall) validateobjref(g, obj2gco(l), obj2gco(l->namecall)); - // TODO (FFlagLuauGcPagedSweep): 'next' type will change after removal of the flag and the cast will not be required - for (UpVal* uv = l->openupval; uv; uv = (UpVal*)uv->next) + for (UpVal* uv = l->openupval; uv; uv = uv->u.l.threadnext) { LUAU_ASSERT(uv->tt == LUA_TUPVAL); LUAU_ASSERT(uv->v != &uv->u.value); @@ -141,7 +138,7 @@ static void validateobj(global_State* g, GCObject* o) /* dead objects can only occur during sweep */ if (isdead(g, o)) { - LUAU_ASSERT(g->gcstate == GCSsweepstring || g->gcstate == GCSsweep); + LUAU_ASSERT(g->gcstate == GCSsweep); return; } @@ -180,18 +177,6 @@ static void validateobj(global_State* g, GCObject* o) } } -static void validatelist(global_State* g, GCObject* o) -{ - LUAU_ASSERT(!FFlag::LuauGcPagedSweep); - - while (o) - { - validateobj(g, o); - - o = o->gch.next; - } -} - static void validategraylist(global_State* g, GCObject* o) { if (!keepinvariant(g)) @@ -224,8 +209,6 @@ static void validategraylist(global_State* g, GCObject* o) static bool validategco(void* context, lua_Page* page, GCObject* gco) { - LUAU_ASSERT(FFlag::LuauGcPagedSweep); - lua_State* L = (lua_State*)context; global_State* g = L->global; @@ -248,20 +231,9 @@ void luaC_validate(lua_State* L) validategraylist(g, g->gray); validategraylist(g, g->grayagain); - if (FFlag::LuauGcPagedSweep) - { - validategco(L, NULL, obj2gco(g->mainthread)); + validategco(L, NULL, obj2gco(g->mainthread)); - luaM_visitgco(L, L, validategco); - } - else - { - for (int i = 0; i < g->strt.size; ++i) - validatelist(g, (GCObject*)(g->strt.hash[i])); - - validatelist(g, g->rootgc); - validatelist(g, (GCObject*)(g->strbufgc)); - } + luaM_visitgco(L, L, validategco); for (UpVal* uv = g->uvhead.u.l.next; uv != &g->uvhead; uv = uv->u.l.next) { @@ -521,30 +493,8 @@ static void dumpobj(FILE* f, GCObject* o) } } -static void dumplist(FILE* f, GCObject* o) -{ - LUAU_ASSERT(!FFlag::LuauGcPagedSweep); - - while (o) - { - dumpref(f, o); - fputc(':', f); - dumpobj(f, o); - fputc(',', f); - fputc('\n', f); - - // thread has additional list containing collectable objects that are not present in rootgc - if (o->gch.tt == LUA_TTHREAD) - dumplist(f, (GCObject*)gco2th(o)->openupval); - - o = o->gch.next; - } -} - static bool dumpgco(void* context, lua_Page* page, GCObject* gco) { - LUAU_ASSERT(FFlag::LuauGcPagedSweep); - FILE* f = (FILE*)context; dumpref(f, gco); @@ -563,19 +513,9 @@ void luaC_dump(lua_State* L, void* file, const char* (*categoryName)(lua_State* fprintf(f, "{\"objects\":{\n"); - if (FFlag::LuauGcPagedSweep) - { - dumpgco(f, NULL, obj2gco(g->mainthread)); + dumpgco(f, NULL, obj2gco(g->mainthread)); - luaM_visitgco(L, f, dumpgco); - } - else - { - dumplist(f, g->rootgc); - dumplist(f, (GCObject*)(g->strbufgc)); - for (int i = 0; i < g->strt.size; ++i) - dumplist(f, (GCObject*)(g->strt.hash[i])); - } + luaM_visitgco(L, f, dumpgco); fprintf(f, "\"0\":{\"type\":\"userdata\",\"cat\":0,\"size\":0}\n"); // to avoid issues with trailing , fprintf(f, "},\"roots\":{\n"); diff --git a/VM/src/linit.cpp b/VM/src/linit.cpp index c93f431f..fd95f596 100644 --- a/VM/src/linit.cpp +++ b/VM/src/linit.cpp @@ -68,7 +68,7 @@ void luaL_sandboxthread(lua_State* L) lua_setsafeenv(L, LUA_GLOBALSINDEX, true); } -static void* l_alloc(lua_State* L, void* ud, void* ptr, size_t osize, size_t nsize) +static void* l_alloc(void* ud, void* ptr, size_t osize, size_t nsize) { (void)ud; (void)osize; diff --git a/VM/src/lmem.cpp b/VM/src/lmem.cpp index 19617b8c..899cb0c0 100644 --- a/VM/src/lmem.cpp +++ b/VM/src/lmem.cpp @@ -78,8 +78,6 @@ * allocated pages. */ -LUAU_FASTFLAG(LuauGcPagedSweep) - #ifndef __has_feature #define __has_feature(x) 0 #endif @@ -98,8 +96,10 @@ LUAU_FASTFLAG(LuauGcPagedSweep) * To prevent some of them accidentally growing and us losing memory without realizing it, we're going to lock * the sizes of all critical structures down. */ -#if defined(__APPLE__) && !defined(__MACH__) +#if defined(__APPLE__) #define ABISWITCH(x64, ms32, gcc32) (sizeof(void*) == 8 ? x64 : gcc32) +#elif defined(__i386__) +#define ABISWITCH(x64, ms32, gcc32) (gcc32) #else // Android somehow uses a similar ABI to MSVC, *not* to iOS... #define ABISWITCH(x64, ms32, gcc32) (sizeof(void*) == 8 ? x64 : ms32) @@ -114,14 +114,8 @@ static_assert(sizeof(LuaNode) == ABISWITCH(32, 32, 32), "size mismatch for table #endif static_assert(offsetof(TString, data) == ABISWITCH(24, 20, 20), "size mismatch for string header"); -// TODO (FFlagLuauGcPagedSweep): this will become ABISWITCH(16, 16, 16) -static_assert(offsetof(Udata, data) == ABISWITCH(24, 16, 16), "size mismatch for userdata header"); -// TODO (FFlagLuauGcPagedSweep): this will become ABISWITCH(48, 32, 32) -static_assert(sizeof(Table) == ABISWITCH(56, 36, 36), "size mismatch for table header"); - -// TODO (FFlagLuauGcPagedSweep): new code with old 'next' pointer requires that GCObject start at the same point as TString/UpVal -static_assert(offsetof(GCObject, uv) == 0, "UpVal data must be located at the start of the GCObject"); -static_assert(offsetof(GCObject, ts) == 0, "TString data must be located at the start of the GCObject"); +static_assert(offsetof(Udata, data) == ABISWITCH(16, 16, 12), "size mismatch for userdata header"); +static_assert(sizeof(Table) == ABISWITCH(48, 32, 32), "size mismatch for table header"); const size_t kSizeClasses = LUA_SIZECLASSES; const size_t kMaxSmallSize = 512; @@ -208,53 +202,13 @@ l_noret luaM_toobig(lua_State* L) luaG_runerror(L, "memory allocation error: block too big"); } -static lua_Page* newpageold(lua_State* L, uint8_t sizeClass) -{ - LUAU_ASSERT(!FFlag::LuauGcPagedSweep); - - global_State* g = L->global; - lua_Page* page = (lua_Page*)(*g->frealloc)(L, g->ud, NULL, 0, kPageSize); - if (!page) - luaD_throw(L, LUA_ERRMEM); - - int blockSize = kSizeClassConfig.sizeOfClass[sizeClass] + kBlockHeader; - int blockCount = (kPageSize - offsetof(lua_Page, data)) / blockSize; - - ASAN_POISON_MEMORY_REGION(page->data, blockSize * blockCount); - - // setup page header - page->prev = NULL; - page->next = NULL; - - page->gcolistprev = NULL; - page->gcolistnext = NULL; - - page->pageSize = kPageSize; - page->blockSize = blockSize; - - // note: we start with the last block in the page and move downward - // either order would work, but that way we don't need to store the block count in the page - // additionally, GC stores objects in singly linked lists, and this way the GC lists end up in increasing pointer order - page->freeList = NULL; - page->freeNext = (blockCount - 1) * blockSize; - page->busyBlocks = 0; - - // prepend a page to page freelist (which is empty because we only ever allocate a new page when it is!) - LUAU_ASSERT(!g->freepages[sizeClass]); - g->freepages[sizeClass] = page; - - return page; -} - static lua_Page* newpage(lua_State* L, lua_Page** gcopageset, int pageSize, int blockSize, int blockCount) { - LUAU_ASSERT(FFlag::LuauGcPagedSweep); - global_State* g = L->global; LUAU_ASSERT(pageSize - int(offsetof(lua_Page, data)) >= blockSize * blockCount); - lua_Page* page = (lua_Page*)(*g->frealloc)(L, g->ud, NULL, 0, pageSize); + lua_Page* page = (lua_Page*)(*g->frealloc)(g->ud, NULL, 0, pageSize); if (!page) luaD_throw(L, LUA_ERRMEM); @@ -290,8 +244,6 @@ static lua_Page* newpage(lua_State* L, lua_Page** gcopageset, int pageSize, int static lua_Page* newclasspage(lua_State* L, lua_Page** freepageset, lua_Page** gcopageset, uint8_t sizeClass, bool storeMetadata) { - LUAU_ASSERT(FFlag::LuauGcPagedSweep); - int blockSize = kSizeClassConfig.sizeOfClass[sizeClass] + (storeMetadata ? kBlockHeader : 0); int blockCount = (kPageSize - offsetof(lua_Page, data)) / blockSize; @@ -304,29 +256,8 @@ static lua_Page* newclasspage(lua_State* L, lua_Page** freepageset, lua_Page** g return page; } -static void freepageold(lua_State* L, lua_Page* page, uint8_t sizeClass) -{ - LUAU_ASSERT(!FFlag::LuauGcPagedSweep); - - global_State* g = L->global; - - // remove page from freelist - if (page->next) - page->next->prev = page->prev; - - if (page->prev) - page->prev->next = page->next; - else if (g->freepages[sizeClass] == page) - g->freepages[sizeClass] = page->next; - - // so long - (*g->frealloc)(L, g->ud, page, kPageSize, 0); -} - static void freepage(lua_State* L, lua_Page** gcopageset, lua_Page* page) { - LUAU_ASSERT(FFlag::LuauGcPagedSweep); - global_State* g = L->global; if (gcopageset) @@ -342,13 +273,11 @@ static void freepage(lua_State* L, lua_Page** gcopageset, lua_Page* page) } // so long - (*g->frealloc)(L, g->ud, page, page->pageSize, 0); + (*g->frealloc)(g->ud, page, page->pageSize, 0); } static void freeclasspage(lua_State* L, lua_Page** freepageset, lua_Page** gcopageset, lua_Page* page, uint8_t sizeClass) { - LUAU_ASSERT(FFlag::LuauGcPagedSweep); - // remove page from freelist if (page->next) page->next->prev = page->prev; @@ -368,12 +297,7 @@ static void* newblock(lua_State* L, int sizeClass) // slow path: no page in the freelist, allocate a new one if (!page) - { - if (FFlag::LuauGcPagedSweep) - page = newclasspage(L, g->freepages, NULL, sizeClass, true); - else - page = newpageold(L, sizeClass); - } + page = newclasspage(L, g->freepages, NULL, sizeClass, true); LUAU_ASSERT(!page->prev); LUAU_ASSERT(page->freeList || page->freeNext >= 0); @@ -416,8 +340,6 @@ static void* newblock(lua_State* L, int sizeClass) static void* newgcoblock(lua_State* L, int sizeClass) { - LUAU_ASSERT(FFlag::LuauGcPagedSweep); - global_State* g = L->global; lua_Page* page = g->freegcopages[sizeClass]; @@ -496,17 +418,11 @@ static void freeblock(lua_State* L, int sizeClass, void* block) // if it's the last block in the page, we don't need the page if (page->busyBlocks == 0) - { - if (FFlag::LuauGcPagedSweep) - freeclasspage(L, g->freepages, NULL, page, sizeClass); - else - freepageold(L, page, sizeClass); - } + freeclasspage(L, g->freepages, NULL, page, sizeClass); } static void freegcoblock(lua_State* L, int sizeClass, void* block, lua_Page* page) { - LUAU_ASSERT(FFlag::LuauGcPagedSweep); LUAU_ASSERT(page && page->busyBlocks > 0); LUAU_ASSERT(page->blockSize == kSizeClassConfig.sizeOfClass[sizeClass]); LUAU_ASSERT(block >= page->data && block < (char*)page + page->pageSize); @@ -544,7 +460,7 @@ void* luaM_new_(lua_State* L, size_t nsize, uint8_t memcat) int nclass = sizeclass(nsize); - void* block = nclass >= 0 ? newblock(L, nclass) : (*g->frealloc)(L, g->ud, NULL, 0, nsize); + void* block = nclass >= 0 ? newblock(L, nclass) : (*g->frealloc)(g->ud, NULL, 0, nsize); if (block == NULL && nsize > 0) luaD_throw(L, LUA_ERRMEM); @@ -556,9 +472,6 @@ void* luaM_new_(lua_State* L, size_t nsize, uint8_t memcat) GCObject* luaM_newgco_(lua_State* L, size_t nsize, uint8_t memcat) { - if (!FFlag::LuauGcPagedSweep) - return (GCObject*)luaM_new_(L, nsize, memcat); - // we need to accommodate space for link for free blocks (freegcolink) LUAU_ASSERT(nsize >= kGCOLinkOffset + sizeof(void*)); @@ -602,7 +515,7 @@ void luaM_free_(lua_State* L, void* block, size_t osize, uint8_t memcat) if (oclass >= 0) freeblock(L, oclass, block); else - (*g->frealloc)(L, g->ud, block, osize, 0); + (*g->frealloc)(g->ud, block, osize, 0); g->totalbytes -= osize; g->memcatbytes[memcat] -= osize; @@ -610,12 +523,6 @@ void luaM_free_(lua_State* L, void* block, size_t osize, uint8_t memcat) void luaM_freegco_(lua_State* L, GCObject* block, size_t osize, uint8_t memcat, lua_Page* page) { - if (!FFlag::LuauGcPagedSweep) - { - luaM_free_(L, block, osize, memcat); - return; - } - global_State* g = L->global; LUAU_ASSERT((osize == 0) == (block == NULL)); @@ -652,7 +559,7 @@ void* luaM_realloc_(lua_State* L, void* block, size_t osize, size_t nsize, uint8 // if either block needs to be allocated using a block allocator, we can't use realloc directly if (nclass >= 0 || oclass >= 0) { - result = nclass >= 0 ? newblock(L, nclass) : (*g->frealloc)(L, g->ud, NULL, 0, nsize); + result = nclass >= 0 ? newblock(L, nclass) : (*g->frealloc)(g->ud, NULL, 0, nsize); if (result == NULL && nsize > 0) luaD_throw(L, LUA_ERRMEM); @@ -662,11 +569,11 @@ void* luaM_realloc_(lua_State* L, void* block, size_t osize, size_t nsize, uint8 if (oclass >= 0) freeblock(L, oclass, block); else - (*g->frealloc)(L, g->ud, block, osize, 0); + (*g->frealloc)(g->ud, block, osize, 0); } else { - result = (*g->frealloc)(L, g->ud, block, osize, nsize); + result = (*g->frealloc)(g->ud, block, osize, nsize); if (result == NULL && nsize > 0) luaD_throw(L, LUA_ERRMEM); } @@ -679,8 +586,6 @@ void* luaM_realloc_(lua_State* L, void* block, size_t osize, size_t nsize, uint8 void luaM_getpagewalkinfo(lua_Page* page, char** start, char** end, int* busyBlocks, int* blockSize) { - LUAU_ASSERT(FFlag::LuauGcPagedSweep); - int blockCount = (page->pageSize - offsetof(lua_Page, data)) / page->blockSize; LUAU_ASSERT(page->freeNext >= -page->blockSize && page->freeNext <= (blockCount - 1) * page->blockSize); @@ -700,8 +605,6 @@ lua_Page* luaM_getnextgcopage(lua_Page* page) void luaM_visitpage(lua_Page* page, void* context, bool (*visitor)(void* context, lua_Page* page, GCObject* gco)) { - LUAU_ASSERT(FFlag::LuauGcPagedSweep); - char* start; char* end; int busyBlocks; @@ -730,8 +633,6 @@ void luaM_visitpage(lua_Page* page, void* context, bool (*visitor)(void* context void luaM_visitgco(lua_State* L, void* context, bool (*visitor)(void* context, lua_Page* page, GCObject* gco)) { - LUAU_ASSERT(FFlag::LuauGcPagedSweep); - global_State* g = L->global; for (lua_Page* curr = g->allgcopages; curr;) diff --git a/VM/src/lmem.h b/VM/src/lmem.h index 1bfe48fa..00788452 100644 --- a/VM/src/lmem.h +++ b/VM/src/lmem.h @@ -7,11 +7,7 @@ struct lua_Page; union GCObject; -// TODO: remove with FFlagLuauGcPagedSweep and rename luaM_newgco to luaM_new -#define luaM_new(L, t, size, memcat) cast_to(t*, luaM_new_(L, size, memcat)) #define luaM_newgco(L, t, size, memcat) cast_to(t*, luaM_newgco_(L, size, memcat)) -// TODO: remove with FFlagLuauGcPagedSweep and rename luaM_freegco to luaM_free -#define luaM_free(L, p, size, memcat) luaM_free_(L, (p), size, memcat) #define luaM_freegco(L, p, size, memcat, page) luaM_freegco_(L, obj2gco(p), size, memcat, page) #define luaM_arraysize_(n, e) ((cast_to(size_t, (n)) <= SIZE_MAX / (e)) ? (n) * (e) : (luaM_toobig(L), SIZE_MAX)) diff --git a/VM/src/lobject.h b/VM/src/lobject.h index 57ffd82a..5e02c2ea 100644 --- a/VM/src/lobject.h +++ b/VM/src/lobject.h @@ -15,7 +15,6 @@ typedef union GCObject GCObject; */ // clang-format off #define CommonHeader \ - GCObject* next; /* TODO: remove with FFlagLuauGcPagedSweep */ \ uint8_t tt; uint8_t marked; uint8_t memcat // clang-format on @@ -233,6 +232,8 @@ typedef struct TString int16_t atom; // 2 byte padding + TString* next; // next string in the hash table bucket or the string buffer linked list + unsigned int hash; unsigned int len; @@ -327,7 +328,7 @@ typedef struct UpVal struct UpVal* next; /* thread double linked list (when open) */ - // TODO: when FFlagLuauGcPagedSweep is removed, old outer 'next' value will be placed here + struct UpVal* threadnext; /* note: this is the location of a pointer to this upvalue in the previous element that can be either an UpVal or a lua_State */ struct UpVal** threadprev; } l; diff --git a/VM/src/lstate.cpp b/VM/src/lstate.cpp index d6d127c0..d4f3f0a1 100644 --- a/VM/src/lstate.cpp +++ b/VM/src/lstate.cpp @@ -10,7 +10,6 @@ #include "ldo.h" #include "ldebug.h" -LUAU_FASTFLAG(LuauGcPagedSweep) LUAU_FASTFLAGVARIABLE(LuauReduceStackReallocs, false) /* @@ -90,8 +89,6 @@ static void close_state(lua_State* L) global_State* g = L->global; luaF_close(L, L->stack); /* close all upvalues for this thread */ luaC_freeall(L); /* collect all objects */ - if (!FFlag::LuauGcPagedSweep) - LUAU_ASSERT(g->rootgc == obj2gco(L)); LUAU_ASSERT(g->strbufgc == NULL); LUAU_ASSERT(g->strt.nuse == 0); luaM_freearray(L, L->global->strt.hash, L->global->strt.size, TString*, 0); @@ -99,22 +96,20 @@ static void close_state(lua_State* L) for (int i = 0; i < LUA_SIZECLASSES; i++) { LUAU_ASSERT(g->freepages[i] == NULL); - if (FFlag::LuauGcPagedSweep) - LUAU_ASSERT(g->freegcopages[i] == NULL); + LUAU_ASSERT(g->freegcopages[i] == NULL); } - if (FFlag::LuauGcPagedSweep) - LUAU_ASSERT(g->allgcopages == NULL); + LUAU_ASSERT(g->allgcopages == NULL); LUAU_ASSERT(g->totalbytes == sizeof(LG)); LUAU_ASSERT(g->memcatbytes[0] == sizeof(LG)); for (int i = 1; i < LUA_MEMORY_CATEGORIES; i++) LUAU_ASSERT(g->memcatbytes[i] == 0); - (*g->frealloc)(L, g->ud, L, sizeof(LG), 0); + (*g->frealloc)(g->ud, L, sizeof(LG), 0); } lua_State* luaE_newthread(lua_State* L) { lua_State* L1 = luaM_newgco(L, lua_State, sizeof(lua_State), L->activememcat); - luaC_link(L, L1, LUA_TTHREAD); + luaC_init(L, L1, LUA_TTHREAD); preinit_state(L1, L->global); L1->activememcat = L->activememcat; // inherit the active memory category stack_init(L1, L); /* init stack */ @@ -184,13 +179,11 @@ lua_State* lua_newstate(lua_Alloc f, void* ud) int i; lua_State* L; global_State* g; - void* l = (*f)(NULL, ud, NULL, 0, sizeof(LG)); + void* l = (*f)(ud, NULL, 0, sizeof(LG)); if (l == NULL) return NULL; L = (lua_State*)l; g = &((LG*)L)->g; - if (!FFlag::LuauGcPagedSweep) - L->next = NULL; L->tt = LUA_TTHREAD; L->marked = g->currentwhite = bit2mask(WHITE0BIT, FIXEDBIT); L->memcat = 0; @@ -214,11 +207,6 @@ lua_State* lua_newstate(lua_Alloc f, void* ud) setnilvalue(&g->pseudotemp); setnilvalue(registry(L)); g->gcstate = GCSpause; - if (!FFlag::LuauGcPagedSweep) - g->rootgc = obj2gco(L); - g->sweepstrgc = 0; - if (!FFlag::LuauGcPagedSweep) - g->sweepgc = &g->rootgc; g->gray = NULL; g->grayagain = NULL; g->weak = NULL; @@ -230,14 +218,10 @@ lua_State* lua_newstate(lua_Alloc f, void* ud) for (i = 0; i < LUA_SIZECLASSES; i++) { g->freepages[i] = NULL; - if (FFlag::LuauGcPagedSweep) - g->freegcopages[i] = NULL; - } - if (FFlag::LuauGcPagedSweep) - { - g->allgcopages = NULL; - g->sweepgcopage = NULL; + g->freegcopages[i] = NULL; } + g->allgcopages = NULL; + g->sweepgcopage = NULL; for (i = 0; i < LUA_T_COUNT; i++) g->mt[i] = NULL; for (i = 0; i < LUA_UTAG_LIMIT; i++) diff --git a/VM/src/lstate.h b/VM/src/lstate.h index 6dd89138..3ee96718 100644 --- a/VM/src/lstate.h +++ b/VM/src/lstate.h @@ -142,11 +142,6 @@ typedef struct global_State uint8_t gcstate; /* state of garbage collector */ - int sweepstrgc; /* position of sweep in `strt' */ - // TODO: remove with FFlagLuauGcPagedSweep - GCObject* rootgc; /* list of all collectable objects */ - // TODO: remove with FFlagLuauGcPagedSweep - GCObject** sweepgc; /* position of sweep in `rootgc' */ GCObject* gray; /* list of gray objects */ GCObject* grayagain; /* list of objects to be traversed atomically */ GCObject* weak; /* list of weak tables (to be cleared) */ diff --git a/VM/src/lstring.cpp b/VM/src/lstring.cpp index 9bbc43de..87250146 100644 --- a/VM/src/lstring.cpp +++ b/VM/src/lstring.cpp @@ -7,8 +7,6 @@ #include -LUAU_FASTFLAG(LuauGcPagedSweep) - unsigned int luaS_hash(const char* str, size_t len) { // Note that this hashing algorithm is replicated in BytecodeBuilder.cpp, BytecodeBuilder::getStringHash @@ -46,8 +44,6 @@ unsigned int luaS_hash(const char* str, size_t len) void luaS_resize(lua_State* L, int newsize) { - if (L->global->gcstate == GCSsweepstring) - return; /* cannot resize during GC traverse */ TString** newhash = luaM_newarray(L, newsize, TString*, 0); stringtable* tb = &L->global->strt; for (int i = 0; i < newsize; i++) @@ -58,13 +54,11 @@ void luaS_resize(lua_State* L, int newsize) TString* p = tb->hash[i]; while (p) { /* for each node in the list */ - // TODO (FFlagLuauGcPagedSweep): 'next' type will change after removal of the flag and the cast will not be required - TString* next = (TString*)p->next; /* save next */ + TString* next = p->next; /* save next */ unsigned int h = p->hash; int h1 = lmod(h, newsize); /* new position */ LUAU_ASSERT(cast_int(h % newsize) == lmod(h, newsize)); - // TODO (FFlagLuauGcPagedSweep): 'next' type will change after removal of the flag and the cast will not be required - p->next = (GCObject*)newhash[h1]; /* chain it */ + p->next = newhash[h1]; /* chain it */ newhash[h1] = p; p = next; } @@ -91,8 +85,7 @@ static TString* newlstr(lua_State* L, const char* str, size_t l, unsigned int h) ts->atom = L->global->cb.useratom ? L->global->cb.useratom(ts->data, l) : -1; tb = &L->global->strt; h = lmod(h, tb->size); - // TODO (FFlagLuauGcPagedSweep): 'next' type will change after removal of the flag and the case will not be required - ts->next = (GCObject*)tb->hash[h]; /* chain new entry */ + ts->next = tb->hash[h]; /* chain new entry */ tb->hash[h] = ts; tb->nuse++; if (tb->nuse > cast_to(uint32_t, tb->size) && tb->size <= INT_MAX / 2) @@ -104,20 +97,9 @@ static void linkstrbuf(lua_State* L, TString* ts) { global_State* g = L->global; - if (FFlag::LuauGcPagedSweep) - { - // TODO (FFlagLuauGcPagedSweep): 'next' type will change after removal of the flag and the cast will not be required - ts->next = (GCObject*)g->strbufgc; - g->strbufgc = ts; - ts->marked = luaC_white(g); - } - else - { - GCObject* o = obj2gco(ts); - o->gch.next = (GCObject*)g->strbufgc; - g->strbufgc = gco2ts(o); - o->gch.marked = luaC_white(g); - } + ts->next = g->strbufgc; + g->strbufgc = ts; + ts->marked = luaC_white(g); } static void unlinkstrbuf(lua_State* L, TString* ts) @@ -130,14 +112,12 @@ static void unlinkstrbuf(lua_State* L, TString* ts) { if (curr == ts) { - // TODO (FFlagLuauGcPagedSweep): 'next' type will change after removal of the flag and the cast will not be required - *p = (TString*)curr->next; + *p = curr->next; return; } else { - // TODO (FFlagLuauGcPagedSweep): 'next' type will change after removal of the flag and the cast will not be required - p = (TString**)&curr->next; + p = &curr->next; } } @@ -167,8 +147,7 @@ TString* luaS_buffinish(lua_State* L, TString* ts) int bucket = lmod(h, tb->size); // search if we already have this string in the hash table - // TODO (FFlagLuauGcPagedSweep): 'next' type will change after removal of the flag and the cast will not be required - for (TString* el = tb->hash[bucket]; el != NULL; el = (TString*)el->next) + for (TString* el = tb->hash[bucket]; el != NULL; el = el->next) { if (el->len == ts->len && memcmp(el->data, ts->data, ts->len) == 0) { @@ -187,8 +166,7 @@ TString* luaS_buffinish(lua_State* L, TString* ts) // Complete string object ts->atom = L->global->cb.useratom ? L->global->cb.useratom(ts->data, ts->len) : -1; - // TODO (FFlagLuauGcPagedSweep): 'next' type will change after removal of the flag and the cast will not be required - ts->next = (GCObject*)tb->hash[bucket]; // chain new entry + ts->next = tb->hash[bucket]; // chain new entry tb->hash[bucket] = ts; tb->nuse++; @@ -201,8 +179,7 @@ TString* luaS_buffinish(lua_State* L, TString* ts) TString* luaS_newlstr(lua_State* L, const char* str, size_t l) { unsigned int h = luaS_hash(str, l); - // TODO (FFlagLuauGcPagedSweep): 'next' type will change after removal of the flag and the cast will not be required - for (TString* el = L->global->strt.hash[lmod(h, L->global->strt.size)]; el != NULL; el = (TString*)el->next) + for (TString* el = L->global->strt.hash[lmod(h, L->global->strt.size)]; el != NULL; el = el->next) { if (el->len == l && (memcmp(str, getstr(el), l) == 0)) { @@ -217,8 +194,6 @@ TString* luaS_newlstr(lua_State* L, const char* str, size_t l) static bool unlinkstr(lua_State* L, TString* ts) { - LUAU_ASSERT(FFlag::LuauGcPagedSweep); - global_State* g = L->global; TString** p = &g->strt.hash[lmod(ts->hash, g->strt.size)]; @@ -227,14 +202,12 @@ static bool unlinkstr(lua_State* L, TString* ts) { if (curr == ts) { - // TODO (FFlagLuauGcPagedSweep): 'next' type will change after removal of the flag and the cast will not be required - *p = (TString*)curr->next; + *p = curr->next; return true; } else { - // TODO (FFlagLuauGcPagedSweep): 'next' type will change after removal of the flag and the cast will not be required - p = (TString**)&curr->next; + p = &curr->next; } } @@ -243,20 +216,11 @@ static bool unlinkstr(lua_State* L, TString* ts) void luaS_free(lua_State* L, TString* ts, lua_Page* page) { - if (FFlag::LuauGcPagedSweep) - { - // Unchain from the string table - if (!unlinkstr(L, ts)) - unlinkstrbuf(L, ts); // An unlikely scenario when we have a string buffer on our hands - else - L->global->strt.nuse--; - - luaM_freegco(L, ts, sizestring(ts->len), ts->memcat, page); - } + // Unchain from the string table + if (!unlinkstr(L, ts)) + unlinkstrbuf(L, ts); // An unlikely scenario when we have a string buffer on our hands else - { L->global->strt.nuse--; - luaM_free(L, ts, sizestring(ts->len), ts->memcat); - } + luaM_freegco(L, ts, sizestring(ts->len), ts->memcat, page); } diff --git a/VM/src/ltable.cpp b/VM/src/ltable.cpp index c57374e0..0412ea76 100644 --- a/VM/src/ltable.cpp +++ b/VM/src/ltable.cpp @@ -425,7 +425,7 @@ static void rehash(lua_State* L, Table* t, const TValue* ek) Table* luaH_new(lua_State* L, int narray, int nhash) { Table* t = luaM_newgco(L, Table, sizeof(Table), L->activememcat); - luaC_link(L, t, LUA_TTABLE); + luaC_init(L, t, LUA_TTABLE); t->metatable = NULL; t->flags = cast_byte(~0); t->array = NULL; @@ -742,7 +742,7 @@ int luaH_getn(Table* t) Table* luaH_clone(lua_State* L, Table* tt) { Table* t = luaM_newgco(L, Table, sizeof(Table), L->activememcat); - luaC_link(L, t, LUA_TTABLE); + luaC_init(L, t, LUA_TTABLE); t->metatable = tt->metatable; t->flags = tt->flags; t->array = NULL; diff --git a/VM/src/ludata.cpp b/VM/src/ludata.cpp index 758a9bdb..0dfac508 100644 --- a/VM/src/ludata.cpp +++ b/VM/src/ludata.cpp @@ -12,7 +12,7 @@ Udata* luaU_newudata(lua_State* L, size_t s, int tag) if (s > INT_MAX - sizeof(Udata)) luaM_toobig(L); Udata* u = luaM_newgco(L, Udata, sizeudata(s), L->activememcat); - luaC_link(L, u, LUA_TUSERDATA); + luaC_init(L, u, LUA_TUSERDATA); u->len = int(s); u->metatable = NULL; LUAU_ASSERT(tag >= 0 && tag <= 255); diff --git a/fuzz/linter.cpp b/fuzz/linter.cpp index 04638d23..0bdd49f5 100644 --- a/fuzz/linter.cpp +++ b/fuzz/linter.cpp @@ -1,10 +1,12 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #include -#include "Luau/TypeInfer.h" -#include "Luau/Linter.h" + #include "Luau/BuiltinDefinitions.h" -#include "Luau/ModuleResolver.h" #include "Luau/Common.h" +#include "Luau/Linter.h" +#include "Luau/ModuleResolver.h" +#include "Luau/Parser.h" +#include "Luau/TypeInfer.h" extern "C" int LLVMFuzzerTestOneInput(const uint8_t* Data, size_t Size) { diff --git a/fuzz/luau.proto b/fuzz/luau.proto index c78fcf31..190b8c5b 100644 --- a/fuzz/luau.proto +++ b/fuzz/luau.proto @@ -96,11 +96,13 @@ message ExprIndexExpr { } message ExprFunction { - repeated Local args = 1; - required bool vararg = 2; - required StatBlock body = 3; - repeated Type types = 4; - repeated Type rettypes = 5; + repeated Typename generics = 1; + repeated Typename genericpacks = 2; + repeated Local args = 3; + required bool vararg = 4; + required StatBlock body = 5; + repeated Type types = 6; + repeated Type rettypes = 7; } message TableItem { @@ -153,7 +155,10 @@ message ExprBinary { message ExprIfElse { required Expr cond = 1; required Expr then = 2; - required Expr else = 3; + oneof else_oneof { + Expr else = 3; + ExprIfElse elseif = 4; + } } message LValue { @@ -183,6 +188,7 @@ message Stat { StatFunction function = 14; StatLocalFunction local_function = 15; StatTypeAlias type_alias = 16; + StatRequireIntoLocalHelper require_into_local = 17; } } @@ -276,9 +282,16 @@ message StatLocalFunction { } message StatTypeAlias { - required Typename name = 1; - required Type type = 2; - repeated Typename generics = 3; + required bool export = 1; + required Typename name = 2; + required Type type = 3; + repeated Typename generics = 4; + repeated Typename genericpacks = 5; +} + +message StatRequireIntoLocalHelper { + required Local var = 1; + required int32 modulenum = 2; } message Type { @@ -292,6 +305,8 @@ message Type { TypeIntersection intersection = 7; TypeClass class = 8; TypeRef ref = 9; + TypeBoolean boolean = 10; + TypeString string = 11; } } @@ -301,7 +316,8 @@ message TypePrimitive { message TypeLiteral { required Typename name = 1; - repeated Typename generics = 2; + repeated Type generics = 2; + repeated Typename genericpacks = 3; } message TypeTableItem { @@ -320,8 +336,10 @@ message TypeTable { } message TypeFunction { - repeated Type args = 1; - repeated Type rets = 2; + repeated Typename generics = 1; + repeated Typename genericpacks = 2; + repeated Type args = 3; + repeated Type rets = 4; // TODO: vararg? } @@ -347,3 +365,16 @@ message TypeRef { required Local prefix = 1; required Typename index = 2; } + +message TypeBoolean { + required bool val = 1; +} + +message TypeString { + required string val = 1; +} + +message ModuleSet { + optional StatBlock module = 1; + required StatBlock program = 2; +} diff --git a/fuzz/proto.cpp b/fuzz/proto.cpp index f407248a..912fef23 100644 --- a/fuzz/proto.cpp +++ b/fuzz/proto.cpp @@ -2,16 +2,17 @@ #include "src/libfuzzer/libfuzzer_macro.h" #include "luau.pb.h" -#include "Luau/TypeInfer.h" #include "Luau/BuiltinDefinitions.h" -#include "Luau/ModuleResolver.h" -#include "Luau/ModuleResolver.h" -#include "Luau/Compiler.h" -#include "Luau/Linter.h" #include "Luau/BytecodeBuilder.h" #include "Luau/Common.h" +#include "Luau/Compiler.h" +#include "Luau/Frontend.h" +#include "Luau/Linter.h" +#include "Luau/ModuleResolver.h" +#include "Luau/Parser.h" #include "Luau/ToString.h" #include "Luau/Transpiler.h" +#include "Luau/TypeInfer.h" #include "lua.h" #include "lualib.h" @@ -30,7 +31,7 @@ const bool kFuzzTypes = true; static_assert(!(kFuzzVM && !kFuzzCompiler), "VM requires the compiler!"); -std::string protoprint(const luau::StatBlock& stat, bool types); +std::vector protoprint(const luau::ModuleSet& stat, bool types); LUAU_FASTINT(LuauTypeInferRecursionLimit) LUAU_FASTINT(LuauTypeInferTypePackLoopLimit) @@ -38,6 +39,7 @@ LUAU_FASTINT(LuauCheckRecursionLimit) LUAU_FASTINT(LuauTableTypeMaximumStringifierLength) LUAU_FASTINT(LuauTypeInferIterationLimit) LUAU_FASTINT(LuauTarjanChildLimit) +LUAU_FASTFLAG(DebugLuauFreezeArena) std::chrono::milliseconds kInterruptTimeout(10); std::chrono::time_point interruptDeadline; @@ -135,10 +137,58 @@ int registerTypes(Luau::TypeChecker& env) return 0; } +struct FuzzFileResolver : Luau::FileResolver +{ + std::optional readSource(const Luau::ModuleName& name) override + { + auto it = source.find(name); + if (it == source.end()) + return std::nullopt; -static std::string debugsource; + return Luau::SourceCode{it->second, Luau::SourceCode::Module}; + } -DEFINE_PROTO_FUZZER(const luau::StatBlock& message) + std::optional resolveModule(const Luau::ModuleInfo* context, Luau::AstExpr* expr) override + { + if (Luau::AstExprGlobal* g = expr->as()) + return Luau::ModuleInfo{g->name.value}; + + return std::nullopt; + } + + std::string getHumanReadableModuleName(const Luau::ModuleName& name) const override + { + return name; + } + + std::optional getEnvironmentForModule(const Luau::ModuleName& name) const override + { + return std::nullopt; + } + + std::unordered_map source; +}; + +struct FuzzConfigResolver : Luau::ConfigResolver +{ + FuzzConfigResolver() + { + defaultConfig.mode = Luau::Mode::Nonstrict; // typecheckTwice option will cover Strict mode + defaultConfig.enabledLint.warningMask = ~0ull; + defaultConfig.parseOptions.captureComments = true; + } + + virtual const Luau::Config& getConfig(const Luau::ModuleName& name) const override + { + return defaultConfig; + } + + Luau::Config defaultConfig; +}; + +static std::vector debugsources; + +DEFINE_PROTO_FUZZER(const luau::ModuleSet& message) { FInt::LuauTypeInferRecursionLimit.value = 100; FInt::LuauTypeInferTypePackLoopLimit.value = 100; @@ -151,91 +201,90 @@ DEFINE_PROTO_FUZZER(const luau::StatBlock& message) if (strncmp(flag->name, "Luau", 4) == 0) flag->value = true; - Luau::Allocator allocator; - Luau::AstNameTable names(allocator); + FFlag::DebugLuauFreezeArena.value = true; - std::string source = protoprint(message, kFuzzTypes); + std::vector sources = protoprint(message, kFuzzTypes); // stash source in a global for easier crash dump debugging - debugsource = source; - - Luau::ParseResult parseResult = Luau::Parser::parse(source.c_str(), source.size(), names, allocator); - - // "static" here is to accelerate fuzzing process by only creating and populating the type environment once - static Luau::NullModuleResolver moduleResolver; - static Luau::InternalErrorReporter iceHandler; - static Luau::TypeChecker sharedEnv(&moduleResolver, &iceHandler); - static int once = registerTypes(sharedEnv); - (void)once; - static int once2 = (Luau::freeze(sharedEnv.globalTypes), 0); - (void)once2; - - iceHandler.onInternalError = [](const char* error) { - printf("ICE: %s\n", error); - LUAU_ASSERT(!"ICE"); - }; + debugsources = sources; static bool debug = getenv("LUAU_DEBUG") != 0; if (debug) { - fprintf(stdout, "--\n%s\n", source.c_str()); + for (std::string& source : sources) + fprintf(stdout, "--\n%s\n", source.c_str()); fflush(stdout); } - std::string bytecode; + // parse all sources + std::vector> parseAllocators; + std::vector> parseNameTables; - // compile - if (kFuzzCompiler && parseResult.errors.empty()) + Luau::ParseOptions parseOptions; + parseOptions.captureComments = true; + + std::vector parseResults; + + for (std::string& source : sources) { - Luau::CompileOptions compileOptions; + parseAllocators.push_back(std::make_unique()); + parseNameTables.push_back(std::make_unique(*parseAllocators.back())); - try - { - Luau::BytecodeBuilder bcb; - Luau::compileOrThrow(bcb, parseResult.root, names, compileOptions); - bytecode = bcb.getBytecode(); - } - catch (const Luau::CompileError&) - { - // not all valid ASTs can be compiled due to limits on number of registers - } + parseResults.push_back(Luau::Parser::parse(source.c_str(), source.size(), *parseNameTables.back(), *parseAllocators.back(), parseOptions)); } - // typecheck - if (kFuzzTypeck && parseResult.root) - { - Luau::SourceModule sourceModule; - sourceModule.root = parseResult.root; - sourceModule.mode = Luau::Mode::Nonstrict; - - Luau::TypeChecker typeck(&moduleResolver, &iceHandler); - typeck.globalScope = sharedEnv.globalScope; - - Luau::ModulePtr module = nullptr; - - try - { - module = typeck.check(sourceModule, Luau::Mode::Nonstrict); - } - catch (std::exception&) - { - // This catches internal errors that the type checker currently (unfortunately) throws in some cases - } - - // lint (note that we need access to types so we need to do this with typeck in scope) - if (kFuzzLinter) - { - Luau::LintOptions lintOptions = {~0u}; - Luau::lint(parseResult.root, names, sharedEnv.globalScope, module.get(), {}, lintOptions); - } - } - - // validate sharedEnv post-typecheck; valuable for debugging some typeck crashes but slows fuzzing down - // note: it's important for typeck to be destroyed at this point! + // typecheck all sources if (kFuzzTypeck) { - for (auto& p : sharedEnv.globalScope->bindings) + static FuzzFileResolver fileResolver; + static Luau::NullConfigResolver configResolver; + static Luau::FrontendOptions options{true, true}; + static Luau::Frontend frontend(&fileResolver, &configResolver, options); + + static int once = registerTypes(frontend.typeChecker); + (void)once; + static int once2 = (Luau::freeze(frontend.typeChecker.globalTypes), 0); + (void)once2; + + frontend.iceHandler.onInternalError = [](const char* error) { + printf("ICE: %s\n", error); + LUAU_ASSERT(!"ICE"); + }; + + // restart + frontend.clear(); + fileResolver.source.clear(); + + // load sources + for (size_t i = 0; i < sources.size(); i++) + { + std::string name = "module" + std::to_string(i); + fileResolver.source[name] = sources[i]; + } + + // check sources + for (size_t i = 0; i < sources.size(); i++) + { + std::string name = "module" + std::to_string(i); + + try + { + Luau::CheckResult result = frontend.check(name, std::nullopt); + + // lint (note that we need access to types so we need to do this with typeck in scope) + if (kFuzzLinter && result.errors.empty()) + frontend.lint(name, std::nullopt); + } + catch (std::exception&) + { + // This catches internal errors that the type checker currently (unfortunately) throws in some cases + } + } + + // validate sharedEnv post-typecheck; valuable for debugging some typeck crashes but slows fuzzing down + // note: it's important for typeck to be destroyed at this point! + for (auto& p : frontend.typeChecker.globalScope->bindings) { Luau::ToStringOptions opts; opts.exhaustive = true; @@ -246,12 +295,44 @@ DEFINE_PROTO_FUZZER(const luau::StatBlock& message) } } - if (kFuzzTranspile && parseResult.root) + if (kFuzzTranspile) { - transpileWithTypes(*parseResult.root); + for (Luau::ParseResult& parseResult : parseResults) + { + if (parseResult.root) + transpileWithTypes(*parseResult.root); + } } - // run resulting bytecode + std::string bytecode; + + // compile + if (kFuzzCompiler) + { + for (size_t i = 0; i < parseResults.size(); i++) + { + Luau::ParseResult& parseResult = parseResults[i]; + Luau::AstNameTable& parseNameTable = *parseNameTables[i]; + + if (parseResult.errors.empty()) + { + Luau::CompileOptions compileOptions; + + try + { + Luau::BytecodeBuilder bcb; + Luau::compileOrThrow(bcb, parseResult.root, parseNameTable, compileOptions); + bytecode = bcb.getBytecode(); + } + catch (const Luau::CompileError&) + { + // not all valid ASTs can be compiled due to limits on number of registers + } + } + } + } + + // run resulting bytecode (from last successfully compiler module) if (kFuzzVM && bytecode.size()) { static lua_State* globalState = createGlobalState(); diff --git a/fuzz/protoprint.cpp b/fuzz/protoprint.cpp index e61b6936..66a89f24 100644 --- a/fuzz/protoprint.cpp +++ b/fuzz/protoprint.cpp @@ -208,6 +208,35 @@ struct ProtoToLuau source += std::to_string(name.index() & 0xff); } + template + void genericidents(const T& node) + { + if (node.generics_size() || node.genericpacks_size()) + { + source += '<'; + bool first = true; + + for (size_t i = 0; i < node.generics_size(); ++i) + { + if (!first) + source += ','; + first = false; + ident(node.generics(i)); + } + + for (size_t i = 0; i < node.genericpacks_size(); ++i) + { + if (!first) + source += ','; + first = false; + ident(node.genericpacks(i)); + source += "..."; + } + + source += '>'; + } + } + void print(const luau::Expr& expr) { if (expr.has_group()) @@ -240,6 +269,8 @@ struct ProtoToLuau print(expr.unary()); else if (expr.has_binary()) print(expr.binary()); + else if (expr.has_ifelse()) + print(expr.ifelse()); else source += "_"; } @@ -350,6 +381,7 @@ struct ProtoToLuau void function(const luau::ExprFunction& expr) { + genericidents(expr); source += "("; for (int i = 0; i < expr.args_size(); ++i) { @@ -478,12 +510,21 @@ struct ProtoToLuau void print(const luau::ExprIfElse& expr) { - source += " if "; + source += "if "; print(expr.cond()); source += " then "; print(expr.then()); - source += " else "; - print(expr.else_()); + + if (expr.has_else_()) + { + source += " else "; + print(expr.else_()); + } + else if (expr.has_elseif()) + { + source += " else"; + print(expr.elseif()); + } } void print(const luau::LValue& expr) @@ -534,6 +575,8 @@ struct ProtoToLuau print(stat.local_function()); else if (stat.has_type_alias()) print(stat.type_alias()); + else if (stat.has_require_into_local()) + print(stat.require_into_local()); else source += "do end\n"; } @@ -804,26 +847,24 @@ struct ProtoToLuau void print(const luau::StatTypeAlias& stat) { + if (stat.export_()) + source += "export "; + source += "type "; ident(stat.name()); - - if (stat.generics_size()) - { - source += '<'; - for (size_t i = 0; i < stat.generics_size(); ++i) - { - if (i != 0) - source += ','; - ident(stat.generics(i)); - } - source += '>'; - } - + genericidents(stat); source += " = "; print(stat.type()); source += '\n'; } + void print(const luau::StatRequireIntoLocalHelper& stat) + { + source += "local "; + print(stat.var()); + source += " = require(module" + std::to_string(stat.modulenum() % 2) + ")\n"; + } + void print(const luau::Type& type) { if (type.has_primitive()) @@ -844,6 +885,10 @@ struct ProtoToLuau print(type.class_()); else if (type.has_ref()) print(type.ref()); + else if (type.has_boolean()) + print(type.boolean()); + else if (type.has_string()) + print(type.string()); else source += "any"; } @@ -858,15 +903,28 @@ struct ProtoToLuau { ident(type.name()); - if (type.generics_size()) + if (type.generics_size() || type.genericpacks_size()) { source += '<'; + bool first = true; + for (size_t i = 0; i < type.generics_size(); ++i) { - if (i != 0) + if (!first) source += ','; - ident(type.generics(i)); + first = false; + print(type.generics(i)); } + + for (size_t i = 0; i < type.genericpacks_size(); ++i) + { + if (!first) + source += ','; + first = false; + ident(type.genericpacks(i)); + source += "..."; + } + source += '>'; } } @@ -893,6 +951,7 @@ struct ProtoToLuau void print(const luau::TypeFunction& type) { + genericidents(type); source += '('; for (size_t i = 0; i < type.args_size(); ++i) { @@ -950,12 +1009,38 @@ struct ProtoToLuau source += '.'; ident(type.index()); } + + void print(const luau::TypeBoolean& type) + { + source += type.val() ? "true" : "false"; + } + + void print(const luau::TypeString& type) + { + source += '"'; + for (char ch : type.val()) + if (isgraph(ch)) + source += ch; + source += '"'; + } }; -std::string protoprint(const luau::StatBlock& stat, bool types) +std::vector protoprint(const luau::ModuleSet& stat, bool types) { + std::vector result; + + if (stat.has_module()) + { + ProtoToLuau printer; + printer.types = types; + printer.print(stat.module()); + result.push_back(printer.source); + } + ProtoToLuau printer; printer.types = types; - printer.print(stat); - return printer.source; + printer.print(stat.program()); + result.push_back(printer.source); + + return result; } diff --git a/fuzz/prototest.cpp b/fuzz/prototest.cpp index 804e708a..ccaa1971 100644 --- a/fuzz/prototest.cpp +++ b/fuzz/prototest.cpp @@ -2,11 +2,15 @@ #include "src/libfuzzer/libfuzzer_macro.h" #include "luau.pb.h" -std::string protoprint(const luau::StatBlock& stat, bool types); +std::vector protoprint(const luau::ModuleSet& stat, bool types); -DEFINE_PROTO_FUZZER(const luau::StatBlock& message) +DEFINE_PROTO_FUZZER(const luau::ModuleSet& message) { - std::string source = protoprint(message, true); + std::vector sources = protoprint(message, true); - printf("%s\n", source.c_str()); + for (size_t i = 0; i < sources.size(); i++) + { + printf("Module 'l%d':\n", int(i)); + printf("%s\n", sources[i].c_str()); + } } diff --git a/fuzz/typeck.cpp b/fuzz/typeck.cpp index 5020c771..3905cc19 100644 --- a/fuzz/typeck.cpp +++ b/fuzz/typeck.cpp @@ -1,9 +1,11 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #include -#include "Luau/TypeInfer.h" + #include "Luau/BuiltinDefinitions.h" -#include "Luau/ModuleResolver.h" #include "Luau/Common.h" +#include "Luau/ModuleResolver.h" +#include "Luau/Parser.h" +#include "Luau/TypeInfer.h" LUAU_FASTINT(LuauTypeInferRecursionLimit) LUAU_FASTINT(LuauTypeInferTypePackLoopLimit) diff --git a/tests/Autocomplete.test.cpp b/tests/Autocomplete.test.cpp index 1978a0d3..6aadef32 100644 --- a/tests/Autocomplete.test.cpp +++ b/tests/Autocomplete.test.cpp @@ -2752,7 +2752,6 @@ TEST_CASE_FIXTURE(ACFixture, "autocomplete_on_string_singletons") ScopedFastFlag sffs[] = { {"LuauParseSingletonTypes", true}, {"LuauSingletonTypes", true}, - {"LuauRefactorTypeVarQuestions", true}, }; check(R"( diff --git a/tests/Conformance.test.cpp b/tests/Conformance.test.cpp index 23e90f4d..eb6ca749 100644 --- a/tests/Conformance.test.cpp +++ b/tests/Conformance.test.cpp @@ -837,7 +837,7 @@ TEST_CASE("ExceptionObject") return ExceptionResult{false, ""}; }; - auto reallocFunc = [](lua_State* L, void* /*ud*/, void* ptr, size_t /*osize*/, size_t nsize) -> void* { + auto reallocFunc = [](void* /*ud*/, void* ptr, size_t /*osize*/, size_t nsize) -> void* { if (nsize == 0) { free(ptr); @@ -964,4 +964,53 @@ TEST_CASE("StringConversion") runConformance("strconv.lua"); } +TEST_CASE("GCDump") +{ + // internal function, declared in lgc.h - not exposed via lua.h + extern void luaC_dump(lua_State* L, void* file, const char* (*categoryName)(lua_State* L, uint8_t memcat)); + + StateRef globalState(luaL_newstate(), lua_close); + lua_State* L = globalState.get(); + + // push various objects on stack to cover different paths + lua_createtable(L, 1, 2); + lua_pushstring(L, "value"); + lua_setfield(L, -2, "key"); + + lua_pushinteger(L, 42); + lua_rawseti(L, -2, 1000); + + lua_pushinteger(L, 42); + lua_rawseti(L, -2, 1); + + lua_pushvalue(L, -1); + lua_setmetatable(L, -2); + + lua_newuserdata(L, 42); + lua_pushvalue(L, -2); + lua_setmetatable(L, -2); + + lua_pushinteger(L, 1); + lua_pushcclosure(L, lua_silence, "test", 1); + + lua_State* CL = lua_newthread(L); + + lua_pushstring(CL, "local x x = {} local function f() x[1] = math.abs(42) end function foo() coroutine.yield() end foo() return f"); + lua_loadstring(CL); + lua_resume(CL, nullptr, 0); + +#ifdef _WIN32 + const char* path = "NUL"; +#else + const char* path = "/dev/null"; +#endif + + FILE* f = fopen(path, "w"); + REQUIRE(f); + + luaC_dump(L, f, nullptr); + + fclose(f); +} + TEST_SUITE_END(); diff --git a/tests/Linter.test.cpp b/tests/Linter.test.cpp index d4b97360..ab19cea3 100644 --- a/tests/Linter.test.cpp +++ b/tests/Linter.test.cpp @@ -1594,4 +1594,17 @@ TEST_CASE_FIXTURE(Fixture, "WrongCommentMuteSelf") REQUIRE_EQ(result.warnings.size(), 0); // --!nolint disables WrongComment lint :) } +TEST_CASE_FIXTURE(Fixture, "DuplicateConditionsIfStatAndExpr") +{ + LintResult result = lint(R"( +if if 1 then 2 else 3 then +elseif if 1 then 2 else 3 then +elseif if 0 then 5 else 4 then +end +)"); + + REQUIRE_EQ(result.warnings.size(), 1); + CHECK_EQ(result.warnings[0].text, "Condition has already been checked on line 2"); +} + TEST_SUITE_END(); diff --git a/tests/Parser.test.cpp b/tests/Parser.test.cpp index 0d4c088d..77e49ce3 100644 --- a/tests/Parser.test.cpp +++ b/tests/Parser.test.cpp @@ -2575,7 +2575,6 @@ do end TEST_CASE_FIXTURE(Fixture, "recover_expected_type_pack") { ScopedFastFlag luauParseTypeAliasDefaults{"LuauParseTypeAliasDefaults", true}; - ScopedFastFlag luauParseRecoverTypePackEllipsis{"LuauParseRecoverTypePackEllipsis", true}; ParseResult result = tryParse(R"( type Y = (T...) -> U... diff --git a/tests/Transpiler.test.cpp b/tests/Transpiler.test.cpp index ac5be859..332aba9e 100644 --- a/tests/Transpiler.test.cpp +++ b/tests/Transpiler.test.cpp @@ -651,4 +651,19 @@ local a: Packed CHECK_EQ(code, transpile(code, {}, true).code); } + +TEST_CASE_FIXTURE(Fixture, "transpile_singleton_types") +{ + ScopedFastFlag luauParseSingletonTypes{"LuauParseSingletonTypes", true}; + + std::string code = R"( +type t1 = 'hello' +type t2 = true +type t3 = '' +type t4 = false + )"; + + CHECK_EQ(code, transpile(code, {}, true).code); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.intersectionTypes.test.cpp b/tests/TypeInfer.intersectionTypes.test.cpp index d677e28d..26881b5c 100644 --- a/tests/TypeInfer.intersectionTypes.test.cpp +++ b/tests/TypeInfer.intersectionTypes.test.cpp @@ -175,6 +175,8 @@ TEST_CASE_FIXTURE(Fixture, "index_on_an_intersection_type_with_property_guarante TEST_CASE_FIXTURE(Fixture, "index_on_an_intersection_type_works_at_arbitrary_depth") { + ScopedFastFlag sff{"LuauDoNotTryToReduce", true}; + CheckResult result = check(R"( type A = {x: {y: {z: {thing: string}}}} type B = {x: {y: {z: {thing: string}}}} @@ -184,7 +186,7 @@ TEST_CASE_FIXTURE(Fixture, "index_on_an_intersection_type_works_at_arbitrary_dep )"); LUAU_REQUIRE_NO_ERRORS(result); - CHECK_EQ(*typeChecker.stringType, *requireType("r")); + CHECK_EQ("string & string", toString(requireType("r"))); } TEST_CASE_FIXTURE(Fixture, "index_on_an_intersection_type_with_mixed_types") @@ -218,7 +220,7 @@ TEST_CASE_FIXTURE(Fixture, "index_on_an_intersection_type_with_one_part_missing_ TEST_CASE_FIXTURE(Fixture, "index_on_an_intersection_type_with_one_property_of_type_any") { CheckResult result = check(R"( - type A = {x: number} + type A = {y: number} type B = {x: any} local t: A & B diff --git a/tests/TypeInfer.refinements.test.cpp b/tests/TypeInfer.refinements.test.cpp index 48e6be6a..bff8926c 100644 --- a/tests/TypeInfer.refinements.test.cpp +++ b/tests/TypeInfer.refinements.test.cpp @@ -1115,6 +1115,22 @@ TEST_CASE_FIXTURE(Fixture, "discriminate_on_properties_of_disjoint_tables_where_ LUAU_REQUIRE_NO_ERRORS(result); } +TEST_CASE_FIXTURE(Fixture, "refine_a_property_not_to_be_nil_through_an_intersection_table") +{ + ScopedFastFlag sff{"LuauDoNotTryToReduce", true}; + + CheckResult result = check(R"( + type T = {} & {f: ((string) -> string)?} + local function f(t: T, x) + if t.f then + t.f(x) + end + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + TEST_CASE_FIXTURE(RefinementClassFixture, "discriminate_from_isa_of_x") { ScopedFastFlag sff[] = { diff --git a/tests/TypeInfer.singletons.test.cpp b/tests/TypeInfer.singletons.test.cpp index 9021700d..856549bd 100644 --- a/tests/TypeInfer.singletons.test.cpp +++ b/tests/TypeInfer.singletons.test.cpp @@ -439,4 +439,128 @@ local a: Animal = if true then { tag = 'cat', catfood = 'something' } else { tag LUAU_REQUIRE_NO_ERRORS(result); } +TEST_CASE_FIXTURE(Fixture, "widen_the_supertype_if_it_is_free_and_subtype_has_singleton") +{ + ScopedFastFlag sff[]{ + {"LuauSingletonTypes", true}, + {"LuauEqConstraint", true}, + {"LuauDiscriminableUnions2", true}, + {"LuauWidenIfSupertypeIsFree", true}, + {"LuauWeakEqConstraint", false}, + }; + + CheckResult result = check(R"( + local function foo(f, x) + if x == "hi" then + f(x) + f("foo") + end + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ(R"("hi")", toString(requireTypeAtPosition({3, 18}))); + // should be ((string) -> a..., string) -> () but needs lower bounds calculation + CHECK_EQ("((string) -> (b...), a) -> ()", toString(requireType("foo"))); +} + +// TEST_CASE_FIXTURE(Fixture, "return_type_of_f_is_not_widened") +// { +// ScopedFastFlag sff[]{ +// {"LuauParseSingletonTypes", true}, +// {"LuauSingletonTypes", true}, +// {"LuauDiscriminableUnions2", true}, +// {"LuauEqConstraint", true}, +// {"LuauWidenIfSupertypeIsFree", true}, +// {"LuauWeakEqConstraint", false}, +// }; + +// CheckResult result = check(R"( +// local function foo(f, x): "hello"? -- anyone there? +// return if x == "hi" +// then f(x) +// else nil +// end +// )"); + +// LUAU_REQUIRE_NO_ERRORS(result); + +// CHECK_EQ(R"("hi")", toString(requireTypeAtPosition({3, 23}))); +// CHECK_EQ(R"(((string) -> ("hello"?, b...), a) -> "hello"?)", toString(requireType("foo"))); +// } + +TEST_CASE_FIXTURE(Fixture, "widening_happens_almost_everywhere") +{ + ScopedFastFlag sff[]{ + {"LuauParseSingletonTypes", true}, + {"LuauSingletonTypes", true}, + {"LuauWidenIfSupertypeIsFree", true}, + }; + + CheckResult result = check(R"( + local foo: "foo" = "foo" + local copy = foo + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + CHECK_EQ("string", toString(requireType("copy"))); +} + +TEST_CASE_FIXTURE(Fixture, "widening_happens_almost_everywhere_except_for_tables") +{ + ScopedFastFlag sff[]{ + {"LuauParseSingletonTypes", true}, + {"LuauSingletonTypes", true}, + {"LuauDiscriminableUnions2", true}, + {"LuauWidenIfSupertypeIsFree", true}, + }; + + CheckResult result = check(R"( + type Cat = {tag: "Cat", meows: boolean} + type Dog = {tag: "Dog", barks: boolean} + type Animal = Cat | Dog + + local function f(tag: "Cat" | "Dog"): Animal? + if tag == "Cat" then + local result = {tag = tag, meows = true} + return result + elseif tag == "Dog" then + local result = {tag = tag, barks = true} + return result + else + return nil + end + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "table_insert_with_a_singleton_argument") +{ + ScopedFastFlag sff[]{ + {"LuauParseSingletonTypes", true}, + {"LuauSingletonTypes", true}, + {"LuauWidenIfSupertypeIsFree", true}, + }; + + CheckResult result = check(R"( + local function foo(t, x) + if x == "hi" or x == "bye" then + table.insert(t, x) + end + + return t + end + + local t = foo({}, "hi") + table.insert(t, "totally_unrelated_type" :: "totally_unrelated_type") + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ("{string}", toString(requireType("t"))); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.tables.test.cpp b/tests/TypeInfer.tables.test.cpp index 6bcd4b99..aa949789 100644 --- a/tests/TypeInfer.tables.test.cpp +++ b/tests/TypeInfer.tables.test.cpp @@ -2239,4 +2239,22 @@ TEST_CASE_FIXTURE(Fixture, "give_up_after_one_metatable_index_look_up") CHECK_EQ("Type 't2' does not have key 'x'", toString(result.errors[0])); } +TEST_CASE_FIXTURE(Fixture, "confusing_indexing") +{ + ScopedFastFlag sff{"LuauDoNotTryToReduce", true}; + + CheckResult result = check(R"( + type T = {} & {p: number | string} + local function f(t: T) + return t.p + end + + local foo = f({p = "string"}) + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ("number | string", toString(requireType("foo"))); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.test.cpp b/tests/TypeInfer.test.cpp index 32358571..f44d9fd8 100644 --- a/tests/TypeInfer.test.cpp +++ b/tests/TypeInfer.test.cpp @@ -5127,8 +5127,6 @@ TEST_CASE_FIXTURE(Fixture, "cli_50041_committing_txnlog_in_apollo_client_error") TEST_CASE_FIXTURE(Fixture, "do_not_modify_imported_types") { - ScopedFastFlag noSealedTypeMod{"LuauNoSealedTypeMod", true}; - fileResolver.source["game/A"] = R"( export type Type = { unrelated: boolean } return {} @@ -5190,7 +5188,6 @@ TEST_CASE_FIXTURE(Fixture, "indexing_on_string_singletons") { ScopedFastFlag sff[]{ {"LuauDiscriminableUnions2", true}, - {"LuauRefactorTypeVarQuestions", true}, {"LuauSingletonTypes", true}, }; @@ -5210,7 +5207,6 @@ TEST_CASE_FIXTURE(Fixture, "indexing_on_union_of_string_singletons") { ScopedFastFlag sff[]{ {"LuauDiscriminableUnions2", true}, - {"LuauRefactorTypeVarQuestions", true}, {"LuauSingletonTypes", true}, }; @@ -5230,7 +5226,6 @@ TEST_CASE_FIXTURE(Fixture, "taking_the_length_of_string_singleton") { ScopedFastFlag sff[]{ {"LuauDiscriminableUnions2", true}, - {"LuauRefactorTypeVarQuestions", true}, {"LuauSingletonTypes", true}, }; @@ -5250,7 +5245,6 @@ TEST_CASE_FIXTURE(Fixture, "taking_the_length_of_union_of_string_singleton") { ScopedFastFlag sff[]{ {"LuauDiscriminableUnions2", true}, - {"LuauRefactorTypeVarQuestions", true}, {"LuauSingletonTypes", true}, }; diff --git a/tests/TypeInfer.tryUnify.test.cpp b/tests/TypeInfer.tryUnify.test.cpp index 4669ea8e..c9bf5103 100644 --- a/tests/TypeInfer.tryUnify.test.cpp +++ b/tests/TypeInfer.tryUnify.test.cpp @@ -19,7 +19,7 @@ struct TryUnifyFixture : Fixture ScopePtr globalScope{new Scope{arena.addTypePack({TypeId{}})}}; InternalErrorReporter iceHandler; UnifierSharedState unifierState{&iceHandler}; - Unifier state{&arena, Mode::Strict, globalScope, Location{}, Variance::Covariant, unifierState}; + Unifier state{&arena, Mode::Strict, Location{}, Variance::Covariant, unifierState}; }; TEST_SUITE_BEGIN("TryUnifyTests"); @@ -261,8 +261,6 @@ TEST_CASE_FIXTURE(TryUnifyFixture, "free_tail_is_grown_properly") TEST_CASE_FIXTURE(TryUnifyFixture, "recursive_metatable_getmatchtag") { - ScopedFastFlag luauUnionTagMatchFix{"LuauUnionTagMatchFix", true}; - TypeVar redirect{FreeTypeVar{TypeLevel{}}}; TypeVar table{TableTypeVar{}}; TypeVar metatable{MetatableTypeVar{&redirect, &table}}; diff --git a/tests/TypeVar.test.cpp b/tests/TypeVar.test.cpp index e43161fa..fd5f4dbc 100644 --- a/tests/TypeVar.test.cpp +++ b/tests/TypeVar.test.cpp @@ -361,16 +361,12 @@ local b: (T, T, T) -> T TEST_CASE("isString_on_string_singletons") { - ScopedFastFlag sff{"LuauRefactorTypeVarQuestions", true}; - TypeVar helloString{SingletonTypeVar{StringSingleton{"hello"}}}; CHECK(isString(&helloString)); } TEST_CASE("isString_on_unions_of_various_string_singletons") { - ScopedFastFlag sff{"LuauRefactorTypeVarQuestions", true}; - TypeVar helloString{SingletonTypeVar{StringSingleton{"hello"}}}; TypeVar byeString{SingletonTypeVar{StringSingleton{"bye"}}}; TypeVar union_{UnionTypeVar{{&helloString, &byeString}}}; @@ -380,8 +376,6 @@ TEST_CASE("isString_on_unions_of_various_string_singletons") TEST_CASE("proof_that_isString_uses_all_of") { - ScopedFastFlag sff{"LuauRefactorTypeVarQuestions", true}; - TypeVar helloString{SingletonTypeVar{StringSingleton{"hello"}}}; TypeVar byeString{SingletonTypeVar{StringSingleton{"bye"}}}; TypeVar booleanType{PrimitiveTypeVar{PrimitiveTypeVar::Boolean}}; @@ -392,16 +386,12 @@ TEST_CASE("proof_that_isString_uses_all_of") TEST_CASE("isBoolean_on_boolean_singletons") { - ScopedFastFlag sff{"LuauRefactorTypeVarQuestions", true}; - TypeVar trueBool{SingletonTypeVar{BooleanSingleton{true}}}; CHECK(isBoolean(&trueBool)); } TEST_CASE("isBoolean_on_unions_of_true_or_false_singletons") { - ScopedFastFlag sff{"LuauRefactorTypeVarQuestions", true}; - TypeVar trueBool{SingletonTypeVar{BooleanSingleton{true}}}; TypeVar falseBool{SingletonTypeVar{BooleanSingleton{false}}}; TypeVar union_{UnionTypeVar{{&trueBool, &falseBool}}}; @@ -411,8 +401,6 @@ TEST_CASE("isBoolean_on_unions_of_true_or_false_singletons") TEST_CASE("proof_that_isBoolean_uses_all_of") { - ScopedFastFlag sff{"LuauRefactorTypeVarQuestions", true}; - TypeVar trueBool{SingletonTypeVar{BooleanSingleton{true}}}; TypeVar falseBool{SingletonTypeVar{BooleanSingleton{false}}}; TypeVar stringType{PrimitiveTypeVar{PrimitiveTypeVar::String}}; diff --git a/tools/natvis/VM.natvis b/tools/natvis/VM.natvis index 9924e194..ccc7e390 100644 --- a/tools/natvis/VM.natvis +++ b/tools/natvis/VM.natvis @@ -183,13 +183,12 @@ openupval - u.l.next + u.l.threadnext this - l_gt - env + gt userdata From db3a8a2f0ff795f4c0a5398cb44ae07143e65af0 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Thu, 24 Feb 2022 17:08:54 -0800 Subject: [PATCH 182/399] Update prototyping.yml (#398) This limits the scope of prototyping action to PRs to prototyping branch to minimize the GHA cost / latency, as cabal install sometimes takes forever --- .github/workflows/prototyping.yml | 17 +---------------- 1 file changed, 1 insertion(+), 16 deletions(-) diff --git a/.github/workflows/prototyping.yml b/.github/workflows/prototyping.yml index fe458673..c92e5803 100644 --- a/.github/workflows/prototyping.yml +++ b/.github/workflows/prototyping.yml @@ -1,25 +1,10 @@ name: prototyping on: - push: - branches: - - 'master' - - 'prototyping-*' - paths: - - '.github/workflows/**' - - 'prototyping/**' - - 'Analysis/**' - - 'Ast/**' - - 'CLI/Ast.cpp' - - 'CLI/FileUtils.*' pull_request: paths: - - '.github/workflows/**' + - '.github/workflows/prototyping.yml' - 'prototyping/**' - - 'Analysis/**' - - 'Ast/**' - - 'CLI/Ast.cpp' - - 'CLI/FileUtils.*' jobs: linux: From fb11ca00f4df0e3a18b9274d47965a95db10d81c Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Mon, 28 Feb 2022 10:50:23 -0800 Subject: [PATCH 183/399] Update README.md linenoise -> isocline --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index d79a0bd6..2ed7348d 100644 --- a/README.md +++ b/README.md @@ -64,7 +64,7 @@ Makefile builds combine both into a single target and can be ran via `make test` Luau uses C++ as its implementation language. The runtime requires C++11, whereas the compiler and analysis components require C++17. It should build without issues using Microsoft Visual Studio 2017 or later, or gcc-7 or clang-7 or later. -Other than the STL/CRT, Luau library components don't have external dependencies. The test suite depends on [doctest](https://github.com/onqtam/doctest) testing framework, and the REPL command-line depends on [cpp-linenoise](https://github.com/yhirose/cpp-linenoise). +Other than the STL/CRT, Luau library components don't have external dependencies. The test suite depends on [doctest](https://github.com/onqtam/doctest) testing framework, and the REPL command-line depends on [isocline](https://github.com/daanx/isocline). # License From 5e8242aeda28ca60765926abfc905b3fb9ed85ff Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Mon, 28 Feb 2022 14:15:33 -0800 Subject: [PATCH 184/399] RFC: table.clone (#362) --- rfcs/function-table-clone.md | 64 ++++++++++++++++++++++++++++++++++++ 1 file changed, 64 insertions(+) create mode 100644 rfcs/function-table-clone.md diff --git a/rfcs/function-table-clone.md b/rfcs/function-table-clone.md new file mode 100644 index 00000000..78d126ef --- /dev/null +++ b/rfcs/function-table-clone.md @@ -0,0 +1,64 @@ +# table.clone + +## Summary + +Add `table.clone` function that, given a table, produces a copy of that table with the same keys/values/metatable. + +## Motivation + +There are multiple cases today when cloning tables is a useful operation. + +- When working with tables as data containers, some algorithms may require modifying the table that can't be done in place for some reason. +- When working with tables as objects, it can be useful to obtain an identical copy of the object for further modification, preserving the metatable. +- When working with immutable data structures, any modification needs to clone some parts of the data structure to produce a new version of the object. + +While it's possible to implement this function in user code today, it's impossible to implement it with maximum efficiency; furthermore, cloning is a reasonably fundamental +operation so from the ergonomics perspective it can be expected to be provided by the standard library. + +## Design + +`table.clone(t)` takes a table, `t`, and returns a new table that: + +- has the same metatable +- has the same keys and values +- is not frozen, even if `t` was + +The copy is shallow: implementing a deep recursive copy automatically is challenging (for similar reasons why we decided to avoid this in `table.freeze`), and often only certain keys need to be cloned recursively which can be done after the initial clone. + +The table can be modified after cloning; as such, functions that compute a slightly modified copy of the table can be easily built on top of `table.clone`. + +`table.clone(t)` is functionally equivalent to the following code, but it's more ergonomic (on the account of being built-in) and significantly faster: + +```lua +assert(type(t) == "table") +local nt = {} +for k,v in pairs(t) do + nt[k] = v +end +if type(getmetatable(t)) == "table" then + setmetatable(nt, getmetatable(t)) +end +``` + +The reason why `table.clone` can be dramatically more efficient is that it can directly copy the internal structure, preserving capacity and exact key order, and is thus +limited purely by memory bandwidth. In comparison, the code above can't predict the table size ahead of time, has to recreate the internal table structure one key at a time, +and bears the interpreter overhead (which can be avoided for numeric keys with `table.move` but that doesn't work for the general case of dictionaries). + +Out of the abundance of caution, `table.clone` will fail to clone the table if it has a protected metatable. This is motivated by the fact that you can't do this today, so +there are no new potential vectors to escape various sandboxes. Superficially it seems like it's probably reasonable to allow cloning tables with protected metatables, but +there may be cases where code manufactures tables with unique protected metatables expecting 1-1 relationship and cloning would break that, so for now this RFC proposes a more +conservative route. We are likely to relax this restriction in the future. + +## Drawbacks + +Adding a new function to `table` library theoretically increases complexity. In practice though, we already effectively implement `table.clone` internally for some VM optimizations, so exposing this to the users bears no cost. + +Assigning a type to this function is a little difficult if we want to enforce the "argument must be a table" constraint. It's likely that we'll need to type this as `table.clone(T): T` for the time being, which is less precise. + +## Alternatives + +We can implement something similar to `Object.assign` from JavaScript instead, that simultaneously assigns extra keys. However, this won't be fundamentally more efficient than +assigning the keys afterwards, and can be implemented in user space. Additionally, we can later extend `clone` with an extra argument if we so choose, so this proposal is the +minimal viable one. + +We can immediately remove the rule wrt protected metatables, as it's not clear that it's actually problematic to be able to clone tables with protected metatables. From 898c0501d17c502a6594d64a0f2851013074450e Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Mon, 28 Feb 2022 14:16:35 -0800 Subject: [PATCH 185/399] Update STATUS.md Add table.clone --- rfcs/STATUS.md | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/rfcs/STATUS.md b/rfcs/STATUS.md index 72db2003..5d2d835c 100644 --- a/rfcs/STATUS.md +++ b/rfcs/STATUS.md @@ -56,3 +56,9 @@ This document tracks unimplemented RFCs. [RFC: Generalized iteration](https://github.com/Roblox/luau/blob/master/rfcs/generalized-iteration.md) **Status**: Needs implementation + +## table.clone + +[RFC: table.clone](https://github.com/Roblox/luau/blob/master/rfcs/function-table-clone.md) + +**Status**: Needs implementation From d277cc2c3bfa80fa280f3c2a69164eaff779ca94 Mon Sep 17 00:00:00 2001 From: vegorov-rbx <75688451+vegorov-rbx@users.noreply.github.com> Date: Wed, 2 Mar 2022 06:27:35 -0800 Subject: [PATCH 186/399] Luau Recap: February 2022 (#403) * Luau Recap: February 2022 * Rebuild pages * Fixed Markdown linter warnings, converted tabs to spaces and added a note on 'select' optimization * Apply suggestions from code review Co-authored-by: Alexander McCord <11488393+alexmccord@users.noreply.github.com> * Added information about new lint * Update docs/_posts/2022-02-28-luau-recap-february-2022.md Co-authored-by: Alexander McCord <11488393+alexmccord@users.noreply.github.com> Co-authored-by: Alexander McCord <11488393+alexmccord@users.noreply.github.com> --- .../2022-02-28-luau-recap-february-2022.md | 164 ++++++++++++++++++ 1 file changed, 164 insertions(+) create mode 100644 docs/_posts/2022-02-28-luau-recap-february-2022.md diff --git a/docs/_posts/2022-02-28-luau-recap-february-2022.md b/docs/_posts/2022-02-28-luau-recap-february-2022.md new file mode 100644 index 00000000..412217df --- /dev/null +++ b/docs/_posts/2022-02-28-luau-recap-february-2022.md @@ -0,0 +1,164 @@ +--- +layout: single +title: "Luau Recap: February 2022" +--- + +Luau is our new language that you can read more about at [https://luau-lang.org](https://luau-lang.org). + +[Cross-posted to the [Roblox Developer Forum](https://devforum.roblox.com/t/luau-recap-february-2022/).] + +## Default type alias type parameters + +We have introduced a syntax to provide default type arguments inside the type alias type parameter list. + +It is now possible to have type functions where the instantiation can omit some type arguments. + +You can provide concrete types: + +```lua +--!strict +type FieldResolver = (T, Data) -> number + +local a: FieldResolver = ... +local b: FieldResolver = ... +``` + +Or reference parameters defined earlier in the list: + +```lua +--!strict +type EqComp = (l: T, r: U) -> boolean + +local a: EqComp = ... -- (l: number, r: number) -> boolean +local b: EqComp = ... -- (l: number, r: string) -> boolean +``` + +Type pack parameters can also have a default type pack: + +```lua +--!strict +type Process = (T) -> U... + +local a: Process = ... -- (number) -> ...string +local b: Process = ... -- (number) -> (boolean, string) +``` + +If all type parameters have a default type, it is now possible to reference that without providing any type arguments: + +```lua +--!strict +type All = (T) -> U + +local a: All -- ok +local b: All<> -- ok as well +``` + +For more details, you can read the original [RFC proposal](https://github.com/Roblox/luau/blob/master/rfcs/syntax-default-type-alias-type-parameters.md). + +## Typechecking improvements + +This month we had many fixes to improve our type inference and reduce false positive errors. + +if-then-else expression can now have different types in each branch: + +```lua +--!strict +local a = if x then 5 else nil -- 'a' will have type 'number?' +local b = if x then 1 else '2' -- 'b' will have type 'number | string' +``` + +And if the expected result type is known, you will not get an error in cases like these: + +```lua +--!strict +type T = {number | string} +-- different array element types don't give an error if that is expected +local c: T = if x then {1, "x", 2, "y"} else {0} +``` + +--- + +`assert` result is now known to not be 'falsy' (`false` or `nil`): + +```lua +--!strict +local function f(x: number?): number + return assert(x) -- no longer an error +end +``` + +--- + +We fixed cases where length operator `#` reported an error when used on a compatible type: + +```lua +--!strict +local union: {number} | {string} +local a = #union -- no longer an error +``` + +--- + +Functions with different variadic argument/return types are no longer compatible: + +```lua +--!strict +local function f(): (number, ...string) + return 2, "a", "b" +end + +local g: () -> (number, ...boolean) = f -- error +``` + +--- + +We have also fixed: + +* false positive errors caused by incorrect reuse of generic types across different function declarations +* issues with forward-declared intersection types +* wrong return type annotation for table.move +* various crashes reported by developers + +## Linter improvements + +A new static analysis warning was introduced to mark incorrect use of a '`a and b or c`' pattern. When 'b' is 'falsy' (`false` or `nil`), result will always be 'c', even if the expression 'a' was true: + +```lua +local function f(x: number) + -- The and-or expression always evaluates to the second alternative because the first alternative is false; consider using if-then-else expression instead + return x < 0.5 and false or 42 +end +``` + +Like we say in the warning, new if-then-else expression doesn't have this pitfall: + +```lua +local function g(x: number) + return if x < 0.5 then false else 42 +end +``` + +--- + +We have also introduced a check for misspelled comment directives: + +```lua +--!non-strict +-- ^ Unknown comment directive 'non-strict'; did you mean 'nonstrict'? +``` + +## Performance improvements + +For performance, we have changed how our Garbage Collector collects unreachable memory. +This rework makes it possible to free memory 2.5x faster and also comes with a small change to how we store Luau objects in memory. For example, each table now uses 16 fewer bytes on 64-bit platforms. + +Another optimization was made for `select(_, ...)` call. +It is now using a special fast path that has constant-time complexity in number of arguments (~3x faster with 10 arguments). + +## Thanks + +A special thanks to all the fine folks who contributed PRs this month! + +* [mikejsavage](https://github.com/mikejsavage) +* [TheGreatSageEqualToHeaven](https://github.com/TheGreatSageEqualToHeaven) +* [petrihakkinen](https://github.com/petrihakkinen) From c5477d522dcb31368a8ee864fb9c7b9c34d7730c Mon Sep 17 00:00:00 2001 From: Alan Jeffrey <403333+asajeffrey@users.noreply.github.com> Date: Wed, 2 Mar 2022 16:02:51 -0600 Subject: [PATCH 187/399] Prototyping strict mode (#399) * First cut of strict mode Co-authored-by: Lily Brown --- prototyping/.gitignore | 2 - prototyping/Everything.agda | 8 + prototyping/Examples/OpSem.agda | 8 +- prototyping/Examples/Run.agda | 15 +- prototyping/Examples/Syntax.agda | 4 +- prototyping/Examples/Type.agda | 1 + prototyping/FFI/Data/Aeson.agda | 29 +- prototyping/FFI/Data/Maybe.agda | 6 + prototyping/FFI/Data/Vector.agda | 9 + prototyping/Interpreter.agda | 32 +- prototyping/Luau/Addr.agda | 5 +- prototyping/Luau/Heap.agda | 42 +- prototyping/Luau/OpSem.agda | 132 ++--- prototyping/Luau/Run.agda | 7 +- prototyping/Luau/RuntimeError.agda | 29 +- prototyping/Luau/RuntimeError/ToString.agda | 36 +- prototyping/Luau/RuntimeType.agda | 8 +- prototyping/Luau/StrictMode.agda | 205 ++++++++ prototyping/Luau/StrictMode/ToString.agda | 39 ++ prototyping/Luau/Substitution.agda | 11 +- prototyping/Luau/Syntax.agda | 64 ++- prototyping/Luau/Syntax/FromJSON.agda | 59 ++- prototyping/Luau/Syntax/ToString.agda | 29 +- prototyping/Luau/Type.agda | 135 ++++- prototyping/Luau/Type/FromJSON.agda | 2 + prototyping/Luau/Type/ToString.agda | 3 +- prototyping/Luau/TypeCheck.agda | 143 ++++++ prototyping/Luau/Value.agda | 20 - prototyping/Luau/Value/ToString.agda | 14 - prototyping/Luau/Var.agda | 6 +- prototyping/Luau/VarCtxt.agda | 43 ++ prototyping/PrettyPrinter.agda | 2 + prototyping/Properties.agda | 6 +- prototyping/Properties/Dec.agda | 5 +- prototyping/Properties/Product.agda | 14 + prototyping/Properties/Step.agda | 145 ++++-- prototyping/Properties/StrictMode.agda | 470 ++++++++++++++++++ prototyping/Properties/TypeCheck.agda | 103 ++++ .../Interpreter/binary_equality_bools/out.txt | 5 +- .../binary_equality_numbers/out.txt | 5 +- .../Tests/Interpreter/binary_exps/out.txt | 5 +- .../Tests/Interpreter/return_nil/out.txt | 8 +- 42 files changed, 1598 insertions(+), 316 deletions(-) create mode 100644 prototyping/Everything.agda create mode 100644 prototyping/Luau/StrictMode.agda create mode 100644 prototyping/Luau/StrictMode/ToString.agda create mode 100644 prototyping/Luau/TypeCheck.agda delete mode 100644 prototyping/Luau/Value.agda delete mode 100644 prototyping/Luau/Value/ToString.agda create mode 100644 prototyping/Luau/VarCtxt.agda create mode 100644 prototyping/Properties/Product.agda create mode 100644 prototyping/Properties/StrictMode.agda create mode 100644 prototyping/Properties/TypeCheck.agda diff --git a/prototyping/.gitignore b/prototyping/.gitignore index 10edac52..31e8c073 100644 --- a/prototyping/.gitignore +++ b/prototyping/.gitignore @@ -2,10 +2,8 @@ *.agdai Main MAlonzo -Examples PrettyPrinter Interpreter -Properties !Tests/Interpreter !Tests/PrettyPrinter .ghc.* diff --git a/prototyping/Everything.agda b/prototyping/Everything.agda new file mode 100644 index 00000000..d8a1fd5a --- /dev/null +++ b/prototyping/Everything.agda @@ -0,0 +1,8 @@ +{-# OPTIONS --rewriting #-} + +module Everything where + +import Examples +import Properties +import PrettyPrinter +import Interpreter diff --git a/prototyping/Examples/OpSem.agda b/prototyping/Examples/OpSem.agda index c4cfc5e5..c3f74ec1 100644 --- a/prototyping/Examples/OpSem.agda +++ b/prototyping/Examples/OpSem.agda @@ -1,8 +1,10 @@ +{-# OPTIONS --rewriting #-} + module Examples.OpSem where open import Luau.OpSem using (_⊢_⟶ᴱ_⊣_; _⊢_⟶ᴮ_⊣_; subst) -open import Luau.Syntax using (Block; var; nil; local_←_; _∙_; done; return; block_is_end) +open import Luau.Syntax using (Block; var; val; nil; local_←_; _∙_; done; return; block_is_end) open import Luau.Heap using (∅) -ex1 : ∅ ⊢ (local (var "x") ← nil ∙ return (var "x") ∙ done) ⟶ᴮ (return nil ∙ done) ⊣ ∅ -ex1 = subst +ex1 : ∅ ⊢ (local (var "x") ← val nil ∙ return (var "x") ∙ done) ⟶ᴮ (return (val nil) ∙ done) ⊣ ∅ +ex1 = subst nil diff --git a/prototyping/Examples/Run.agda b/prototyping/Examples/Run.agda index ddec8e8b..545ecf51 100644 --- a/prototyping/Examples/Run.agda +++ b/prototyping/Examples/Run.agda @@ -4,22 +4,17 @@ module Examples.Run where open import Agda.Builtin.Equality using (_≡_; refl) open import Agda.Builtin.Bool using (true; false) -open import Luau.Syntax using (nil; var; _$_; function_is_end; return; _∙_; done; _⟨_⟩; number; binexp; +; <; true; false) -open import Luau.Value using (nil; number; bool) +open import Luau.Syntax using (nil; var; _$_; function_is_end; return; _∙_; done; _⟨_⟩; number; binexp; +; <; val; bool) open import Luau.Run using (run; return) -open import Luau.Heap using (lookup-next; next-emp; lookup-next-emp) -import Agda.Builtin.Equality.Rewrite -{-# REWRITE lookup-next next-emp lookup-next-emp #-} - -ex1 : (run (function "id" ⟨ var "x" ⟩ is return (var "x") ∙ done end ∙ return (var "id" $ nil) ∙ done) ≡ return nil _) +ex1 : (run (function "id" ⟨ var "x" ⟩ is return (var "x") ∙ done end ∙ return (var "id" $ val nil) ∙ done) ≡ return nil _) ex1 = refl -ex2 : (run (function "fn" ⟨ var "x" ⟩ is return (number 123.0) ∙ done end ∙ return (var "fn" $ nil) ∙ done) ≡ return (number 123.0) _) +ex2 : (run (function "fn" ⟨ var "x" ⟩ is return (val (number 123.0)) ∙ done end ∙ return (var "fn" $ val nil) ∙ done) ≡ return (number 123.0) _) ex2 = refl -ex3 : (run (function "fn" ⟨ var "x" ⟩ is return (binexp (number 1.0) + (number 2.0)) ∙ done end ∙ return (var "fn" $ nil) ∙ done) ≡ return (number 3.0) _) +ex3 : (run (function "fn" ⟨ var "x" ⟩ is return (binexp (val (number 1.0)) + (val (number 2.0))) ∙ done end ∙ return (var "fn" $ val nil) ∙ done) ≡ return (number 3.0) _) ex3 = refl -ex4 : (run (function "fn" ⟨ var "x" ⟩ is return (binexp (number 1.0) < (number 2.0)) ∙ done end ∙ return (var "fn" $ nil) ∙ done) ≡ return (bool true) _) +ex4 : (run (function "fn" ⟨ var "x" ⟩ is return (binexp (val (number 1.0)) < (val (number 2.0))) ∙ done end ∙ return (var "fn" $ val nil) ∙ done) ≡ return (bool true) _) ex4 = refl diff --git a/prototyping/Examples/Syntax.agda b/prototyping/Examples/Syntax.agda index 4ffcf4c5..e2793023 100644 --- a/prototyping/Examples/Syntax.agda +++ b/prototyping/Examples/Syntax.agda @@ -2,7 +2,7 @@ module Examples.Syntax where open import Agda.Builtin.Equality using (_≡_; refl) open import FFI.Data.String using (_++_) -open import Luau.Syntax using (var; _$_; return; nil; function_is_end; local_←_; done; _∙_; _⟨_⟩) +open import Luau.Syntax using (var; _$_; return; val; nil; function_is_end; local_←_; done; _∙_; _⟨_⟩) open import Luau.Syntax.ToString using (exprToString; blockToString) ex1 : exprToString(function "" ⟨ var "x" ⟩ is return (var "f" $ var "x") ∙ done end) ≡ @@ -11,7 +11,7 @@ ex1 : exprToString(function "" ⟨ var "x" ⟩ is return (var "f" $ var "x") ∙ "end" ex1 = refl -ex2 : blockToString(local var "x" ← nil ∙ return (var "x") ∙ done) ≡ +ex2 : blockToString(local var "x" ← (val nil) ∙ return (var "x") ∙ done) ≡ "local x = nil\n" ++ "return x" ex2 = refl diff --git a/prototyping/Examples/Type.agda b/prototyping/Examples/Type.agda index 859227ee..3fdd37d5 100644 --- a/prototyping/Examples/Type.agda +++ b/prototyping/Examples/Type.agda @@ -25,3 +25,4 @@ ex6 = refl ex7 : typeToString((nil ⇒ nil) ∪ ((nil ⇒ (nil ⇒ nil)) ∪ nil)) ≡ "((nil) -> nil | (nil) -> (nil) -> nil)?" ex7 = refl + diff --git a/prototyping/FFI/Data/Aeson.agda b/prototyping/FFI/Data/Aeson.agda index a9082d31..0cd27232 100644 --- a/prototyping/FFI/Data/Aeson.agda +++ b/prototyping/FFI/Data/Aeson.agda @@ -1,15 +1,21 @@ +{-# OPTIONS --rewriting #-} + module FFI.Data.Aeson where +open import Agda.Builtin.Equality using (_≡_) +open import Agda.Builtin.Equality.Rewrite using () open import Agda.Builtin.Bool using (Bool) open import Agda.Builtin.String using (String) open import FFI.Data.ByteString using (ByteString) open import FFI.Data.HaskellString using (HaskellString; pack) -open import FFI.Data.Maybe using (Maybe) +open import FFI.Data.Maybe using (Maybe; just; nothing) open import FFI.Data.Either using (Either; mapLeft) open import FFI.Data.Scientific using (Scientific) open import FFI.Data.Vector using (Vector) +open import Properties.Equality using (_≢_) + {-# FOREIGN GHC import qualified Data.Aeson #-} {-# FOREIGN GHC import qualified Data.Aeson.Key #-} {-# FOREIGN GHC import qualified Data.Aeson.KeyMap #-} @@ -19,14 +25,35 @@ postulate Key : Set fromString : String → Key toString : Key → String + empty : ∀ {A} → KeyMap A + singleton : ∀ {A} → Key → A → (KeyMap A) + insert : ∀ {A} → Key → A → (KeyMap A) → (KeyMap A) + delete : ∀ {A} → Key → (KeyMap A) → (KeyMap A) + unionWith : ∀ {A} → (A → A → A) → (KeyMap A) → (KeyMap A) → (KeyMap A) lookup : ∀ {A} → Key -> KeyMap A -> Maybe A {-# POLARITY KeyMap ++ #-} {-# COMPILE GHC KeyMap = type Data.Aeson.KeyMap.KeyMap #-} {-# COMPILE GHC Key = type Data.Aeson.Key.Key #-} {-# COMPILE GHC fromString = Data.Aeson.Key.fromText #-} {-# COMPILE GHC toString = Data.Aeson.Key.toText #-} +{-# COMPILE GHC empty = \_ -> Data.Aeson.KeyMap.empty #-} +{-# COMPILE GHC singleton = \_ -> Data.Aeson.KeyMap.singleton #-} +{-# COMPILE GHC insert = \_ -> Data.Aeson.KeyMap.insert #-} +{-# COMPILE GHC delete = \_ -> Data.Aeson.KeyMap.delete #-} +{-# COMPILE GHC unionWith = \_ -> Data.Aeson.KeyMap.unionWith #-} {-# COMPILE GHC lookup = \_ -> Data.Aeson.KeyMap.lookup #-} +postulate lookup-insert : ∀ {A} k v (m : KeyMap A) → (lookup k (insert k v m) ≡ just v) +postulate lookup-empty : ∀ {A} k → (lookup {A} k empty ≡ nothing) +postulate lookup-insert-not : ∀ {A} j k v (m : KeyMap A) → (j ≢ k) → (lookup k m ≡ lookup k (insert j v m)) +postulate singleton-insert-empty : ∀ {A} k (v : A) → (singleton k v ≡ insert k v empty) +postulate insert-swap : ∀ {A} j k (v w : A) m → (j ≢ k) → insert j v (insert k w m) ≡ insert k w (insert j v m) +postulate insert-over : ∀ {A} j k (v w : A) m → (j ≡ k) → insert j v (insert k w m) ≡ insert j v m +postulate to-from : ∀ k → toString(fromString k) ≡ k +postulate from-to : ∀ k → fromString(toString k) ≡ k + +{-# REWRITE lookup-insert lookup-empty singleton-insert-empty #-} + data Value : Set where object : KeyMap Value → Value array : Vector Value → Value diff --git a/prototyping/FFI/Data/Maybe.agda b/prototyping/FFI/Data/Maybe.agda index b16fa810..58fd1486 100644 --- a/prototyping/FFI/Data/Maybe.agda +++ b/prototyping/FFI/Data/Maybe.agda @@ -1,8 +1,14 @@ module FFI.Data.Maybe where +open import Agda.Builtin.Equality using (_≡_; refl) + {-# FOREIGN GHC import qualified Data.Maybe #-} data Maybe (A : Set) : Set where nothing : Maybe A just : A → Maybe A {-# COMPILE GHC Maybe = data Data.Maybe.Maybe (Data.Maybe.Nothing|Data.Maybe.Just) #-} + +just-inv : ∀ {A} {x y : A} → (just x ≡ just y) → (x ≡ y) +just-inv refl = refl + diff --git a/prototyping/FFI/Data/Vector.agda b/prototyping/FFI/Data/Vector.agda index 7d2f5012..08761edf 100644 --- a/prototyping/FFI/Data/Vector.agda +++ b/prototyping/FFI/Data/Vector.agda @@ -1,11 +1,15 @@ +{-# OPTIONS --rewriting #-} + module FFI.Data.Vector where open import Agda.Builtin.Equality using (_≡_) +open import Agda.Builtin.Equality.Rewrite using () open import Agda.Builtin.Int using (Int; pos; negsuc) open import Agda.Builtin.Nat using (Nat) open import Agda.Builtin.Bool using (Bool; false; true) open import FFI.Data.HaskellInt using (HaskellInt; haskellIntToInt; intToHaskellInt) open import FFI.Data.Maybe using (Maybe; just; nothing) +open import Properties.Equality using (_≢_) {-# FOREIGN GHC import qualified Data.Vector #-} @@ -30,8 +34,13 @@ postulate {-# COMPILE GHC snoc = \_ -> Data.Vector.snoc #-} postulate length-empty : ∀ {A} → (length (empty {A}) ≡ 0) +postulate lookup-empty : ∀ {A} n → (lookup (empty {A}) n ≡ nothing) postulate lookup-snoc : ∀ {A} (x : A) (v : Vector A) → (lookup (snoc v x) (length v) ≡ just x) +postulate lookup-length : ∀ {A} (v : Vector A) → (lookup v (length v) ≡ nothing) postulate lookup-snoc-empty : ∀ {A} (x : A) → (lookup (snoc empty x) 0 ≡ just x) +postulate lookup-snoc-not : ∀ {A n} (x : A) (v : Vector A) → (n ≢ length v) → (lookup v n ≡ lookup (snoc v x) n) + +{-# REWRITE length-empty lookup-snoc lookup-length lookup-snoc-empty lookup-empty #-} head : ∀ {A} → (Vector A) → (Maybe A) head vec with null vec diff --git a/prototyping/Interpreter.agda b/prototyping/Interpreter.agda index fe311e57..d8036740 100644 --- a/prototyping/Interpreter.agda +++ b/prototyping/Interpreter.agda @@ -1,3 +1,5 @@ +{-# OPTIONS --rewriting #-} + module Interpreter where open import Agda.Builtin.IO using (IO) @@ -7,31 +9,41 @@ open import Agda.Builtin.Unit using (⊤) open import FFI.IO using (getContents; putStrLn; _>>=_; _>>_) open import FFI.Data.Aeson using (Value; eitherDecode) open import FFI.Data.Either using (Left; Right) +open import FFI.Data.Maybe using (just; nothing) open import FFI.Data.String using (String; _++_) open import FFI.Data.Text.Encoding using (encodeUtf8) open import FFI.System.Exit using (exitWith; ExitFailure) -open import Luau.Syntax using (Block) +open import Luau.StrictMode.ToString using (warningToStringᴮ) +open import Luau.Syntax using (Block; yes; maybe; isAnnotatedᴮ) open import Luau.Syntax.FromJSON using (blockFromJSON) -open import Luau.Syntax.ToString using (blockToString) +open import Luau.Syntax.ToString using (blockToString; valueToString) open import Luau.Run using (run; return; done; error) open import Luau.RuntimeError.ToString using (errToStringᴮ) -open import Luau.Value.ToString using (valueToString) -runBlock : ∀ {a} → Block a → IO ⊤ -runBlock block with run block -runBlock block | return V D = putStrLn (valueToString V) -runBlock block | done D = putStrLn "nil" -runBlock block | error E D = putStrLn (errToStringᴮ E) +open import Properties.StrictMode using (wellTypedProgramsDontGoWrong) + +runBlock′ : ∀ a → Block a → IO ⊤ +runBlock′ a block with run block +runBlock′ a block | return V D = putStrLn ("\nRAN WITH RESULT: " ++ valueToString V) +runBlock′ a block | done D = putStrLn ("\nRAN") +runBlock′ maybe block | error E D = putStrLn ("\nRUNTIME ERROR:\n" ++ errToStringᴮ _ E) +runBlock′ yes block | error E D with wellTypedProgramsDontGoWrong _ block _ D E +runBlock′ yes block | error E D | W = putStrLn ("\nRUNTIME ERROR:\n" ++ errToStringᴮ _ E ++ "\n\nTYPE ERROR:\n" ++ warningToStringᴮ _ W) + +runBlock : Block maybe → IO ⊤ +runBlock B with isAnnotatedᴮ B +runBlock B | nothing = putStrLn ("UNANNOTATED PROGRAM:\n" ++ blockToString B) >> runBlock′ maybe B +runBlock B | just B′ = putStrLn ("ANNOTATED PROGRAM:\n" ++ blockToString B) >> runBlock′ yes B′ runJSON : Value → IO ⊤ runJSON value with blockFromJSON(value) -runJSON value | (Left err) = putStrLn ("Luau error: " ++ err) >> exitWith (ExitFailure (pos 1)) +runJSON value | (Left err) = putStrLn ("LUAU ERROR: " ++ err) >> exitWith (ExitFailure (pos 1)) runJSON value | (Right block) = runBlock block runString : String → IO ⊤ runString txt with eitherDecode (encodeUtf8 txt) -runString txt | (Left err) = putStrLn ("JSON error: " ++ err) >> exitWith (ExitFailure (pos 1)) +runString txt | (Left err) = putStrLn ("JSON ERROR: " ++ err) >> exitWith (ExitFailure (pos 1)) runString txt | (Right value) = runJSON value main : IO ⊤ diff --git a/prototyping/Luau/Addr.agda b/prototyping/Luau/Addr.agda index c1978047..b6f989f5 100644 --- a/prototyping/Luau/Addr.agda +++ b/prototyping/Luau/Addr.agda @@ -5,13 +5,14 @@ open import Agda.Builtin.Equality using (_≡_) open import Agda.Builtin.Nat using (Nat; _==_) open import Agda.Builtin.String using (String) open import Agda.Builtin.TrustMe using (primTrustMe) -open import Properties.Dec using (Dec; yes; no; ⊥) +open import Properties.Dec using (Dec; yes; no) +open import Properties.Equality using (_≢_) Addr : Set Addr = Nat _≡ᴬ_ : (a b : Addr) → Dec (a ≡ b) a ≡ᴬ b with a == b -a ≡ᴬ b | false = no p where postulate p : (a ≡ b) → ⊥ +a ≡ᴬ b | false = no p where postulate p : (a ≢ b) a ≡ᴬ b | true = yes primTrustMe diff --git a/prototyping/Luau/Heap.agda b/prototyping/Luau/Heap.agda index 12f8dab0..713b0a1a 100644 --- a/prototyping/Luau/Heap.agda +++ b/prototyping/Luau/Heap.agda @@ -1,32 +1,39 @@ +{-# OPTIONS --rewriting #-} + module Luau.Heap where -open import Agda.Builtin.Equality using (_≡_) -open import FFI.Data.Maybe using (Maybe; just) -open import FFI.Data.Vector using (Vector; length; snoc; empty) -open import Luau.Addr using (Addr) +open import Agda.Builtin.Equality using (_≡_; refl) +open import FFI.Data.Maybe using (Maybe; just; nothing) +open import FFI.Data.Vector using (Vector; length; snoc; empty; lookup-snoc-not) +open import Luau.Addr using (Addr; _≡ᴬ_) open import Luau.Var using (Var) -open import Luau.Syntax using (Block; Expr; Annotated; FunDec; nil; addr; function_is_end) +open import Luau.Syntax using (Block; Expr; Annotated; FunDec; nil; function_is_end) +open import Properties.Equality using (_≢_; trans) +open import Properties.Remember using (remember; _,_) +open import Properties.Dec using (yes; no) -data HeapValue (a : Annotated) : Set where - function_is_end : FunDec a → Block a → HeapValue a +-- Heap-allocated objects +data Object (a : Annotated) : Set where + + function_is_end : FunDec a → Block a → Object a Heap : Annotated → Set -Heap a = Vector (HeapValue a) +Heap a = Vector (Object a) -data _≡_⊕_↦_ {a} : Heap a → Heap a → Addr → HeapValue a → Set where +data _≡_⊕_↦_ {a} : Heap a → Heap a → Addr → Object a → Set where defn : ∀ {H val} → ----------------------------------- (snoc H val) ≡ H ⊕ (length H) ↦ val -_[_] : ∀ {a} → Heap a → Addr → Maybe (HeapValue a) +_[_] : ∀ {a} → Heap a → Addr → Maybe (Object a) _[_] = FFI.Data.Vector.lookup ∅ : ∀ {a} → Heap a ∅ = empty -data AllocResult a (H : Heap a) (V : HeapValue a) : Set where +data AllocResult a (H : Heap a) (V : Object a) : Set where ok : ∀ b H′ → (H′ ≡ H ⊕ b ↦ V) → AllocResult a H V alloc : ∀ {a} H V → AllocResult a H V @@ -35,15 +42,8 @@ alloc H V = ok (length H) (snoc H V) defn next : ∀ {a} → Heap a → Addr next = length -allocated : ∀ {a} → Heap a → HeapValue a → Heap a +allocated : ∀ {a} → Heap a → Object a → Heap a allocated = snoc --- next-emp : (length ∅ ≡ 0) -next-emp = FFI.Data.Vector.length-empty - --- lookup-next : ∀ V H → (lookup (allocated H V) (next H) ≡ just V) -lookup-next = FFI.Data.Vector.lookup-snoc - --- lookup-next-emp : ∀ V → (lookup (allocated emp V) 0 ≡ just V) -lookup-next-emp = FFI.Data.Vector.lookup-snoc-empty - +lookup-not-allocated : ∀ {a} {H H′ : Heap a} {b c O} → (H′ ≡ H ⊕ b ↦ O) → (c ≢ b) → (H [ c ] ≡ H′ [ c ]) +lookup-not-allocated {H = H} {O = O} defn p = lookup-snoc-not O H p diff --git a/prototyping/Luau/OpSem.agda b/prototyping/Luau/OpSem.agda index bd75d9d6..cbb55978 100644 --- a/prototyping/Luau/OpSem.agda +++ b/prototyping/Luau/OpSem.agda @@ -1,66 +1,53 @@ +{-# OPTIONS --rewriting #-} + module Luau.OpSem where open import Agda.Builtin.Equality using (_≡_) open import Agda.Builtin.Float using (Float; primFloatPlus; primFloatMinus; primFloatTimes; primFloatDiv; primFloatEquality; primFloatLess; primFloatInequality) open import Agda.Builtin.Bool using (Bool; true; false) open import Utility.Bool using (not; _or_; _and_) -open import Agda.Builtin.Nat using (_==_) -open import FFI.Data.Maybe using (just) +open import Agda.Builtin.Nat using () renaming (_==_ to _==ᴬ_) +open import FFI.Data.Maybe using (Maybe; just; nothing) open import Luau.Heap using (Heap; _≡_⊕_↦_; _[_]; function_is_end) open import Luau.Substitution using (_[_/_]ᴮ) -open import Luau.Syntax using (Expr; Stat; Block; nil; addr; var; function_is_end; _$_; block_is_end; local_←_; _∙_; done; return; name; fun; arg; binexp; BinaryOperator; +; -; *; /; <; >; ≡; ≅; ≤; ≥; number) -open import Luau.Value using (addr; val; number; Value; bool) +open import Luau.Syntax using (Value; Expr; Stat; Block; nil; addr; val; var; function_is_end; _$_; block_is_end; local_←_; _∙_; done; return; name; fun; arg; binexp; BinaryOperator; +; -; *; /; <; >; ==; ~=; <=; >=; number; bool) open import Luau.RuntimeType using (RuntimeType; valueType) +open import Properties.Product using (_×_; _,_) -evalNumOp : Float → BinaryOperator → Float → Value -evalNumOp x + y = number (primFloatPlus x y) -evalNumOp x - y = number (primFloatMinus x y) -evalNumOp x * y = number (primFloatTimes x y) -evalNumOp x / y = number (primFloatDiv x y) -evalNumOp x < y = bool (primFloatLess x y) -evalNumOp x > y = bool (primFloatLess y x) -evalNumOp x ≡ y = bool (primFloatEquality x y) -evalNumOp x ≅ y = bool (primFloatInequality x y) -evalNumOp x ≤ y = bool ((primFloatLess x y) or (primFloatEquality x y)) -evalNumOp x ≥ y = bool ((primFloatLess y x) or (primFloatEquality x y)) +evalEqOp : Value → Value → Bool +evalEqOp Value.nil Value.nil = true +evalEqOp (addr x) (addr y) = (x ==ᴬ y) +evalEqOp (number x) (number y) = primFloatEquality x y +evalEqOp (bool true) (bool y) = y +evalEqOp (bool false) (bool y) = not y +evalEqOp _ _ = false -evalEqOp : Value → Value → Value -evalEqOp Value.nil Value.nil = bool true -evalEqOp (addr x) (addr y) = bool (x == y) -evalEqOp (number x) (number y) = bool (primFloatEquality x y) -evalEqOp (bool true) (bool y) = bool y -evalEqOp (bool false) (bool y) = bool (not y) -evalEqOp _ _ = bool false - -evalNeqOp : Value → Value → Value -evalNeqOp Value.nil Value.nil = bool false -evalNeqOp (addr x) (addr y) = bool (not (x == y)) -evalNeqOp (number x) (number y) = bool (primFloatInequality x y) -evalNeqOp (bool true) (bool y) = bool (not y) -evalNeqOp (bool false) (bool y) = bool y -evalNeqOp _ _ = bool true - -coerceToBool : Value → Bool -coerceToBool Value.nil = false -coerceToBool (addr x) = true -coerceToBool (number x) = true -coerceToBool (bool x) = x +evalNeqOp : Value → Value → Bool +evalNeqOp (number x) (number y) = primFloatInequality x y +evalNeqOp x y = not (evalEqOp x y) +data _⟦_⟧_⟶_ : Value → BinaryOperator → Value → Value → Set where + + : ∀ m n → (number m) ⟦ + ⟧ (number n) ⟶ number (primFloatPlus m n) + - : ∀ m n → (number m) ⟦ - ⟧ (number n) ⟶ number (primFloatMinus m n) + / : ∀ m n → (number m) ⟦ / ⟧ (number n) ⟶ number (primFloatTimes m n) + * : ∀ m n → (number m) ⟦ * ⟧ (number n) ⟶ number (primFloatDiv m n) + < : ∀ m n → (number m) ⟦ < ⟧ (number n) ⟶ bool (primFloatLess m n) + > : ∀ m n → (number m) ⟦ > ⟧ (number n) ⟶ bool (primFloatLess n m) + <= : ∀ m n → (number m) ⟦ <= ⟧ (number n) ⟶ bool ((primFloatLess m n) or (primFloatEquality m n)) + >= : ∀ m n → (number m) ⟦ >= ⟧ (number n) ⟶ bool ((primFloatLess n m) or (primFloatEquality m n)) + == : ∀ v w → v ⟦ == ⟧ w ⟶ bool (evalEqOp v w) + ~= : ∀ v w → v ⟦ ~= ⟧ w ⟶ bool (evalNeqOp v w) + data _⊢_⟶ᴮ_⊣_ {a} : Heap a → Block a → Block a → Heap a → Set data _⊢_⟶ᴱ_⊣_ {a} : Heap a → Expr a → Expr a → Heap a → Set data _⊢_⟶ᴱ_⊣_ where - nil : ∀ {H} → - - ------------------- - H ⊢ nil ⟶ᴱ nil ⊣ H - - function : ∀ {H H′ a F B} → + function : ∀ a {H H′ F B} → H′ ≡ H ⊕ a ↦ (function F is B end) → ------------------------------------------- - H ⊢ (function F is B end) ⟶ᴱ (addr a) ⊣ H′ + H ⊢ (function F is B end) ⟶ᴱ val(addr a) ⊣ H′ app₁ : ∀ {H H′ M M′ N} → @@ -68,17 +55,18 @@ data _⊢_⟶ᴱ_⊣_ where ----------------------------- H ⊢ (M $ N) ⟶ᴱ (M′ $ N) ⊣ H′ - app₂ : ∀ {H H′ V N N′} → + app₂ : ∀ v {H H′ N N′} → H ⊢ N ⟶ᴱ N′ ⊣ H′ → ----------------------------- - H ⊢ (val V $ N) ⟶ᴱ (val V $ N′) ⊣ H′ + H ⊢ (val v $ N) ⟶ᴱ (val v $ N′) ⊣ H′ - beta : ∀ {H a F B V} → - - H [ a ] ≡ just(function F is B end) → + beta : ∀ O v {H a F B} → + + (O ≡ function F is B end) → + H [ a ] ≡ just(O) → ----------------------------------------------------------------------------- - H ⊢ (addr a $ val V) ⟶ᴱ (block (fun F) is (B [ V / name(arg F) ]ᴮ) end) ⊣ H + H ⊢ (val (addr a) $ val v) ⟶ᴱ (block (fun F) is (B [ v / name(arg F) ]ᴮ) end) ⊣ H block : ∀ {H H′ B B′ b} → @@ -86,44 +74,34 @@ data _⊢_⟶ᴱ_⊣_ where ---------------------------------------------------- H ⊢ (block b is B end) ⟶ᴱ (block b is B′ end) ⊣ H′ - return : ∀ {H V B b} → - + return : ∀ v {H B b} → + -------------------------------------------------------- - H ⊢ (block b is return (val V) ∙ B end) ⟶ᴱ (val V) ⊣ H + H ⊢ (block b is return (val v) ∙ B end) ⟶ᴱ val v ⊣ H done : ∀ {H b} → - --------------------------------- - H ⊢ (block b is done end) ⟶ᴱ nil ⊣ H + -------------------------------------------- + H ⊢ (block b is done end) ⟶ᴱ (val nil) ⊣ H - binOpEquality : - ∀ {H x y} → - --------------------------------------------------------------------------- - H ⊢ (binexp (val x) BinaryOperator.≡ (val y)) ⟶ᴱ (val (evalEqOp x y)) ⊣ H - - binOpInequality : - ∀ {H x y} → - ---------------------------------------------------------------------------- - H ⊢ (binexp (val x) BinaryOperator.≅ (val y)) ⟶ᴱ (val (evalNeqOp x y)) ⊣ H - - binOpNumbers : - ∀ {H x op y} → - ----------------------------------------------------------------------- - H ⊢ (binexp (number x) op (number y)) ⟶ᴱ (val (evalNumOp x op y)) ⊣ H - - binOp₁ : - ∀ {H H′ x x′ op y} → + binOp₀ : ∀ {H op v₁ v₂ w} → + + v₁ ⟦ op ⟧ v₂ ⟶ w → + -------------------------------------------------- + H ⊢ (binexp (val v₁) op (val v₂)) ⟶ᴱ (val w) ⊣ H + + binOp₁ : ∀ {H H′ x x′ op y} → + H ⊢ x ⟶ᴱ x′ ⊣ H′ → --------------------------------------------- H ⊢ (binexp x op y) ⟶ᴱ (binexp x′ op y) ⊣ H′ - binOp₂ : - ∀ {H H′ x op y y′} → + binOp₂ : ∀ {H H′ x op y y′} → + H ⊢ y ⟶ᴱ y′ ⊣ H′ → --------------------------------------------- H ⊢ (binexp x op y) ⟶ᴱ (binexp x op y′) ⊣ H′ - data _⊢_⟶ᴮ_⊣_ where local : ∀ {H H′ x M M′ B} → @@ -132,16 +110,16 @@ data _⊢_⟶ᴮ_⊣_ where ------------------------------------------------- H ⊢ (local x ← M ∙ B) ⟶ᴮ (local x ← M′ ∙ B) ⊣ H′ - subst : ∀ {H x v B} → + subst : ∀ v {H x B} → ------------------------------------------------------ H ⊢ (local x ← val v ∙ B) ⟶ᴮ (B [ v / name x ]ᴮ) ⊣ H - function : ∀ {H H′ a F B C} → + function : ∀ a {H H′ F B C} → H′ ≡ H ⊕ a ↦ (function F is C end) → -------------------------------------------------------------- - H ⊢ (function F is C end ∙ B) ⟶ᴮ (B [ addr a / fun F ]ᴮ) ⊣ H′ + H ⊢ (function F is C end ∙ B) ⟶ᴮ (B [ addr a / name(fun F) ]ᴮ) ⊣ H′ return : ∀ {H H′ M M′ B} → diff --git a/prototyping/Luau/Run.agda b/prototyping/Luau/Run.agda index d84f75de..bcd75578 100644 --- a/prototyping/Luau/Run.agda +++ b/prototyping/Luau/Run.agda @@ -1,15 +1,16 @@ +{-# OPTIONS --rewriting #-} + module Luau.Run where open import Agda.Builtin.Equality using (_≡_; refl) open import Luau.Heap using (Heap; ∅) -open import Luau.Syntax using (Block; return; _∙_; done) +open import Luau.Syntax using (Block; val; return; _∙_; done) open import Luau.OpSem using (_⊢_⟶*_⊣_; refl; step) -open import Luau.Value using (val) open import Properties.Step using (stepᴮ; step; return; done; error) open import Luau.RuntimeError using (RuntimeErrorᴮ) data RunResult {a} (H : Heap a) (B : Block a) : Set where - return : ∀ V {B′ H′} → (H ⊢ B ⟶* (return (val V) ∙ B′) ⊣ H′) → RunResult H B + return : ∀ v {B′ H′} → (H ⊢ B ⟶* (return (val v) ∙ B′) ⊣ H′) → RunResult H B done : ∀ {H′} → (H ⊢ B ⟶* done ⊣ H′) → RunResult H B error : ∀ {B′ H′} → (RuntimeErrorᴮ H′ B′) → (H ⊢ B ⟶* B′ ⊣ H′) → RunResult H B diff --git a/prototyping/Luau/RuntimeError.agda b/prototyping/Luau/RuntimeError.agda index b479a72a..4440d2b7 100644 --- a/prototyping/Luau/RuntimeError.agda +++ b/prototyping/Luau/RuntimeError.agda @@ -1,27 +1,40 @@ +{-# OPTIONS --rewriting #-} + module Luau.RuntimeError where open import Agda.Builtin.Equality using (_≡_) open import Luau.Heap using (Heap; _[_]) open import FFI.Data.Maybe using (just; nothing) open import FFI.Data.String using (String) -open import Luau.Syntax using (Block; Expr; nil; var; addr; block_is_end; _$_; local_←_; return; done; _∙_; number; binexp) -open import Luau.RuntimeType using (RuntimeType; valueType) -open import Luau.Value using (val) +open import Luau.Syntax using (BinaryOperator; Block; Expr; nil; var; val; addr; block_is_end; _$_; local_←_; return; done; _∙_; number; binexp; +; -; *; /; <; >; <=; >=) +open import Luau.RuntimeType using (RuntimeType; valueType; function; number) open import Properties.Equality using (_≢_) +data BinOpError : BinaryOperator → RuntimeType → Set where + + : ∀ {t} → (t ≢ number) → BinOpError + t + - : ∀ {t} → (t ≢ number) → BinOpError - t + * : ∀ {t} → (t ≢ number) → BinOpError * t + / : ∀ {t} → (t ≢ number) → BinOpError / t + < : ∀ {t} → (t ≢ number) → BinOpError < t + > : ∀ {t} → (t ≢ number) → BinOpError > t + <= : ∀ {t} → (t ≢ number) → BinOpError <= t + >= : ∀ {t} → (t ≢ number) → BinOpError >= t + data RuntimeErrorᴮ {a} (H : Heap a) : Block a → Set data RuntimeErrorᴱ {a} (H : Heap a) : Expr a → Set data RuntimeErrorᴱ H where - TypeMismatch : ∀ t v → (t ≢ valueType v) → RuntimeErrorᴱ H (val v) - UnboundVariable : ∀ x → RuntimeErrorᴱ H (var x) - SEGV : ∀ a → (H [ a ] ≡ nothing) → RuntimeErrorᴱ H (addr a) + FunctionMismatch : ∀ v w → (function ≢ valueType v) → RuntimeErrorᴱ H (val v $ val w) + BinOpMismatch₁ : ∀ v w {op} → (BinOpError op (valueType v)) → RuntimeErrorᴱ H (binexp (val v) op (val w)) + BinOpMismatch₂ : ∀ v w {op} → (BinOpError op (valueType w)) → RuntimeErrorᴱ H (binexp (val v) op (val w)) + UnboundVariable : ∀ {x} → RuntimeErrorᴱ H (var x) + SEGV : ∀ {a} → (H [ a ] ≡ nothing) → RuntimeErrorᴱ H (val (addr a)) app₁ : ∀ {M N} → RuntimeErrorᴱ H M → RuntimeErrorᴱ H (M $ N) app₂ : ∀ {M N} → RuntimeErrorᴱ H N → RuntimeErrorᴱ H (M $ N) - block : ∀ b {B} → RuntimeErrorᴮ H B → RuntimeErrorᴱ H (block b is B end) + block : ∀ {b B} → RuntimeErrorᴮ H B → RuntimeErrorᴱ H (block b is B end) bin₁ : ∀ {M N op} → RuntimeErrorᴱ H M → RuntimeErrorᴱ H (binexp M op N) bin₂ : ∀ {M N op} → RuntimeErrorᴱ H N → RuntimeErrorᴱ H (binexp M op N) data RuntimeErrorᴮ H where - local : ∀ x {M B} → RuntimeErrorᴱ H M → RuntimeErrorᴮ H (local x ← M ∙ B) + local : ∀ {x M B} → RuntimeErrorᴱ H M → RuntimeErrorᴮ H (local x ← M ∙ B) return : ∀ {M B} → RuntimeErrorᴱ H M → RuntimeErrorᴮ H (return M ∙ B) diff --git a/prototyping/Luau/RuntimeError/ToString.agda b/prototyping/Luau/RuntimeError/ToString.agda index d5528c8b..6a6ed3e2 100644 --- a/prototyping/Luau/RuntimeError/ToString.agda +++ b/prototyping/Luau/RuntimeError/ToString.agda @@ -1,27 +1,29 @@ +{-# OPTIONS --rewriting #-} + module Luau.RuntimeError.ToString where open import Agda.Builtin.Float using (primShowFloat) open import FFI.Data.String using (String; _++_) -open import Luau.RuntimeError using (RuntimeErrorᴮ; RuntimeErrorᴱ; local; return; TypeMismatch; UnboundVariable; SEGV; app₁; app₂; block; bin₁; bin₂) +open import Luau.RuntimeError using (RuntimeErrorᴮ; RuntimeErrorᴱ; local; return; FunctionMismatch; BinOpMismatch₁; BinOpMismatch₂; UnboundVariable; SEGV; app₁; app₂; block; bin₁; bin₂) open import Luau.RuntimeType.ToString using (runtimeTypeToString) open import Luau.Addr.ToString using (addrToString) -open import Luau.Syntax.ToString using (exprToString) +open import Luau.Syntax.ToString using (valueToString; exprToString) open import Luau.Var.ToString using (varToString) -open import Luau.Value.ToString using (valueToString) -open import Luau.Syntax using (name; _$_) +open import Luau.Syntax using (var; val; addr; binexp; block_is_end; local_←_; return; _∙_; name; _$_) -errToStringᴱ : ∀ {a H B} → RuntimeErrorᴱ {a} H B → String -errToStringᴮ : ∀ {a H B} → RuntimeErrorᴮ {a} H B → String +errToStringᴱ : ∀ {a H} M → RuntimeErrorᴱ {a} H M → String +errToStringᴮ : ∀ {a H} B → RuntimeErrorᴮ {a} H B → String -errToStringᴱ (UnboundVariable x) = "variable " ++ varToString x ++ " is unbound" -errToStringᴱ (SEGV a x) = "address " ++ addrToString a ++ " is unallocated" -errToStringᴱ (app₁ E) = errToStringᴱ E -errToStringᴱ (app₂ E) = errToStringᴱ E -errToStringᴱ (bin₁ E) = errToStringᴱ E -errToStringᴱ (bin₂ E) = errToStringᴱ E -errToStringᴱ (block b E) = errToStringᴮ E ++ "\n in call of function " ++ varToString b -errToStringᴱ (TypeMismatch t v _) = "value " ++ valueToString v ++ " is not a " ++ runtimeTypeToString t +errToStringᴱ (var x) (UnboundVariable) = "variable " ++ varToString x ++ " is unbound" +errToStringᴱ (val (addr a)) (SEGV p) = "address " ++ addrToString a ++ " is unallocated" +errToStringᴱ (M $ N) (FunctionMismatch v w p) = "value " ++ (valueToString v) ++ " is not a function" +errToStringᴱ (M $ N) (app₁ E) = errToStringᴱ M E +errToStringᴱ (M $ N) (app₂ E) = errToStringᴱ N E +errToStringᴱ (binexp M op N) (BinOpMismatch₁ v w p) = "value " ++ (valueToString v) ++ " is not a number" +errToStringᴱ (binexp M op N) (BinOpMismatch₂ v w p) = "value " ++ (valueToString w) ++ " is not a number" +errToStringᴱ (binexp M op N) (bin₁ E) = errToStringᴱ M E +errToStringᴱ (binexp M op N) (bin₂ E) = errToStringᴱ N E +errToStringᴱ (block b is B end) (block E) = errToStringᴮ B E ++ "\n in call of function " ++ varToString (name b) -errToStringᴮ (local x E) = errToStringᴱ E ++ "\n in definition of " ++ varToString (name x) -errToStringᴮ (return E) = errToStringᴱ E ++ "\n in return statement" - +errToStringᴮ (local x ← M ∙ B) (local E) = errToStringᴱ M E ++ "\n in definition of " ++ varToString (name x) +errToStringᴮ (return M ∙ B) (return E) = errToStringᴱ M E ++ "\n in return statement" diff --git a/prototyping/Luau/RuntimeType.agda b/prototyping/Luau/RuntimeType.agda index 375fc540..f6b8b4a4 100644 --- a/prototyping/Luau/RuntimeType.agda +++ b/prototyping/Luau/RuntimeType.agda @@ -1,6 +1,6 @@ module Luau.RuntimeType where -open import Luau.Value using (Value; nil; addr; number; bool) +open import Luau.Syntax using (Value; nil; addr; number; bool) data RuntimeType : Set where function : RuntimeType @@ -10,6 +10,6 @@ data RuntimeType : Set where valueType : Value → RuntimeType valueType nil = nil -valueType (addr x) = function -valueType (number x) = number -valueType (bool _) = boolean +valueType (addr a) = function +valueType (number n) = number +valueType (bool b) = boolean diff --git a/prototyping/Luau/StrictMode.agda b/prototyping/Luau/StrictMode.agda new file mode 100644 index 00000000..d7b2129f --- /dev/null +++ b/prototyping/Luau/StrictMode.agda @@ -0,0 +1,205 @@ +{-# OPTIONS --rewriting #-} + +module Luau.StrictMode where + +open import Agda.Builtin.Equality using (_≡_) +open import FFI.Data.Maybe using (just; nothing) +open import Luau.Syntax using (Expr; Stat; Block; BinaryOperator; yes; nil; addr; var; binexp; var_∈_; _⟨_⟩∈_; function_is_end; _$_; block_is_end; local_←_; _∙_; done; return; name; +; -; *; /; <; >; <=; >=) +open import Luau.Type using (Type; strict; nil; number; _⇒_; tgt) +open import Luau.Heap using (Heap; function_is_end) renaming (_[_] to _[_]ᴴ) +open import Luau.VarCtxt using (VarCtxt; ∅; _⋒_; _↦_; _⊕_↦_; _⊝_) renaming (_[_] to _[_]ⱽ) +open import Luau.TypeCheck(strict) using (_⊢ᴮ_∈_; _⊢ᴱ_∈_; ⊢ᴴ_; ⊢ᴼ_; _⊢ᴴᴱ_▷_∈_; _⊢ᴴᴮ_▷_∈_; var; addr; app; binexp; block; return; local; function) +open import Properties.Equality using (_≢_) +open import Properties.TypeCheck(strict) using (typeCheckᴮ) +open import Properties.Product using (_,_) + +src : Type → Type +src = Luau.Type.src strict + +data BinOpWarning : BinaryOperator → Type → Set where + + : ∀ {T} → (T ≢ number) → BinOpWarning + T + - : ∀ {T} → (T ≢ number) → BinOpWarning - T + * : ∀ {T} → (T ≢ number) → BinOpWarning * T + / : ∀ {T} → (T ≢ number) → BinOpWarning / T + < : ∀ {T} → (T ≢ number) → BinOpWarning < T + > : ∀ {T} → (T ≢ number) → BinOpWarning > T + <= : ∀ {T} → (T ≢ number) → BinOpWarning <= T + >= : ∀ {T} → (T ≢ number) → BinOpWarning >= T + +data Warningᴱ (H : Heap yes) {Γ} : ∀ {M T} → (Γ ⊢ᴱ M ∈ T) → Set +data Warningᴮ (H : Heap yes) {Γ} : ∀ {B T} → (Γ ⊢ᴮ B ∈ T) → Set + +data Warningᴱ H {Γ} where + + UnallocatedAddress : ∀ {a T} → + + (H [ a ]ᴴ ≡ nothing) → + --------------------- + Warningᴱ H (addr {a} T) + + UnboundVariable : ∀ {x T p} → + + (Γ [ x ]ⱽ ≡ nothing) → + ------------------------ + Warningᴱ H (var {x} {T} p) + + FunctionCallMismatch : ∀ {M N T U} {D₁ : Γ ⊢ᴱ M ∈ T} {D₂ : Γ ⊢ᴱ N ∈ U} → + + (src T ≢ U) → + ----------------- + Warningᴱ H (app D₁ D₂) + + app₁ : ∀ {M N T U} {D₁ : Γ ⊢ᴱ M ∈ T} {D₂ : Γ ⊢ᴱ N ∈ U} → + + Warningᴱ H D₁ → + ----------------- + Warningᴱ H (app D₁ D₂) + + app₂ : ∀ {M N T U} {D₁ : Γ ⊢ᴱ M ∈ T} {D₂ : Γ ⊢ᴱ N ∈ U} → + + Warningᴱ H D₂ → + ----------------- + Warningᴱ H (app D₁ D₂) + + BinOpMismatch₁ : ∀ {op M N T U} {D₁ : Γ ⊢ᴱ M ∈ T} {D₂ : Γ ⊢ᴱ N ∈ U} → + + BinOpWarning op T → + ------------------------------ + Warningᴱ H (binexp {op} D₁ D₂) + + BinOpMismatch₂ : ∀ {op M N T U} {D₁ : Γ ⊢ᴱ M ∈ T} {D₂ : Γ ⊢ᴱ N ∈ U} → + + BinOpWarning op U → + ------------------------------ + Warningᴱ H (binexp {op} D₁ D₂) + + bin₁ : ∀ {op M N T U} {D₁ : Γ ⊢ᴱ M ∈ T} {D₂ : Γ ⊢ᴱ N ∈ U} → + + Warningᴱ H D₁ → + ------------------------------ + Warningᴱ H (binexp {op} D₁ D₂) + + bin₂ : ∀ {op M N T U} {D₁ : Γ ⊢ᴱ M ∈ T} {D₂ : Γ ⊢ᴱ N ∈ U} → + + Warningᴱ H D₂ → + ------------------------------ + Warningᴱ H (binexp {op} D₁ D₂) + + FunctionDefnMismatch : ∀ {f x B T U V} {D : (Γ ⊕ x ↦ T) ⊢ᴮ B ∈ V} → + + (U ≢ V) → + ------------------------- + Warningᴱ H (function {f} {U = U} D) + + function₁ : ∀ {f x B T U V} {D : (Γ ⊕ x ↦ T) ⊢ᴮ B ∈ V} → + + Warningᴮ H D → + ------------------------- + Warningᴱ H (function {f} {U = U} D) + + BlockMismatch : ∀ {b B T U} {D : Γ ⊢ᴮ B ∈ U} → + + (T ≢ U) → + ------------------------------ + Warningᴱ H (block {b} {T = T} D) + + block₁ : ∀ {b B T U} {D : Γ ⊢ᴮ B ∈ U} → + + Warningᴮ H D → + ------------------------------ + Warningᴱ H (block {b} {T = T} D) + +data Warningᴮ H {Γ} where + + return : ∀ {M B T U} {D₁ : Γ ⊢ᴱ M ∈ T} {D₂ : Γ ⊢ᴮ B ∈ U} → + + Warningᴱ H D₁ → + ------------------ + Warningᴮ H (return D₁ D₂) + + LocalVarMismatch : ∀ {x M B T U V} {D₁ : Γ ⊢ᴱ M ∈ U} {D₂ : (Γ ⊕ x ↦ T) ⊢ᴮ B ∈ V} → + + (T ≢ U) → + -------------------- + Warningᴮ H (local D₁ D₂) + + local₁ : ∀ {x M B T U V} {D₁ : Γ ⊢ᴱ M ∈ U} {D₂ : (Γ ⊕ x ↦ T) ⊢ᴮ B ∈ V} → + + Warningᴱ H D₁ → + -------------------- + Warningᴮ H (local D₁ D₂) + + local₂ : ∀ {x M B T U V} {D₁ : Γ ⊢ᴱ M ∈ U} {D₂ : (Γ ⊕ x ↦ T) ⊢ᴮ B ∈ V} → + + Warningᴮ H D₂ → + -------------------- + Warningᴮ H (local D₁ D₂) + + FunctionDefnMismatch : ∀ {f x B C T U V W} {D₁ : (Γ ⊕ x ↦ T) ⊢ᴮ C ∈ V} {D₂ : (Γ ⊕ f ↦ (T ⇒ U)) ⊢ᴮ B ∈ W} → + + (U ≢ V) → + ------------------------------------- + Warningᴮ H (function D₁ D₂) + + function₁ : ∀ {f x B C T U V W} {D₁ : (Γ ⊕ x ↦ T) ⊢ᴮ C ∈ V} {D₂ : (Γ ⊕ f ↦ (T ⇒ U)) ⊢ᴮ B ∈ W} → + + Warningᴮ H D₁ → + -------------------- + Warningᴮ H (function D₁ D₂) + + function₂ : ∀ {f x B C T U V W} {D₁ : (Γ ⊕ x ↦ T) ⊢ᴮ C ∈ V} {D₂ : (Γ ⊕ f ↦ (T ⇒ U)) ⊢ᴮ B ∈ W} → + + Warningᴮ H D₂ → + -------------------- + Warningᴮ H (function D₁ D₂) + +data Warningᴼ (H : Heap yes) : ∀ {V} → (⊢ᴼ V) → Set where + + FunctionDefnMismatch : ∀ {f x B T U V} {D : (x ↦ T) ⊢ᴮ B ∈ V} → + + (U ≢ V) → + --------------------------------- + Warningᴼ H (function {f} {U = U} D) + + function₁ : ∀ {f x B T U V} {D : (x ↦ T) ⊢ᴮ B ∈ V} → + + Warningᴮ H D → + --------------------------------- + Warningᴼ H (function {f} {U = U} D) + +data Warningᴴ H (D : ⊢ᴴ H) : Set where + + addr : ∀ a {O} → + + (p : H [ a ]ᴴ ≡ O) → + Warningᴼ H (D a p) → + --------------- + Warningᴴ H D + +data Warningᴴᴱ H {Γ M T} : (Γ ⊢ᴴᴱ H ▷ M ∈ T) → Set where + + heap : ∀ {D₁ : ⊢ᴴ H} {D₂ : Γ ⊢ᴱ M ∈ T} → + + Warningᴴ H D₁ → + ----------------- + Warningᴴᴱ H (D₁ , D₂) + + expr : ∀ {D₁ : ⊢ᴴ H} {D₂ : Γ ⊢ᴱ M ∈ T} → + + Warningᴱ H D₂ → + --------------------- + Warningᴴᴱ H (D₁ , D₂) + +data Warningᴴᴮ H {Γ B T} : (Γ ⊢ᴴᴮ H ▷ B ∈ T) → Set where + + heap : ∀ {D₁ : ⊢ᴴ H} {D₂ : Γ ⊢ᴮ B ∈ T} → + + Warningᴴ H D₁ → + ----------------- + Warningᴴᴮ H (D₁ , D₂) + + block : ∀ {D₁ : ⊢ᴴ H} {D₂ : Γ ⊢ᴮ B ∈ T} → + + Warningᴮ H D₂ → + --------------------- + Warningᴴᴮ H (D₁ , D₂) diff --git a/prototyping/Luau/StrictMode/ToString.agda b/prototyping/Luau/StrictMode/ToString.agda new file mode 100644 index 00000000..ca55888c --- /dev/null +++ b/prototyping/Luau/StrictMode/ToString.agda @@ -0,0 +1,39 @@ +{-# OPTIONS --rewriting #-} + +module Luau.StrictMode.ToString where + +open import FFI.Data.String using (String; _++_) +open import Luau.StrictMode using (Warningᴱ; Warningᴮ; UnallocatedAddress; UnboundVariable; FunctionCallMismatch; FunctionDefnMismatch; BlockMismatch; app₁; app₂; BinOpMismatch₁; BinOpMismatch₂; bin₁; bin₂; block₁; return; LocalVarMismatch; local₁; local₂; function₁; function₂; heap; expr; block; addr) +open import Luau.Syntax using (Expr; val; yes; var; var_∈_; _⟨_⟩∈_; _$_; addr; number; binexp; nil; function_is_end; block_is_end; done; return; local_←_; _∙_; fun; arg; name) +open import Luau.Type using (strict) +open import Luau.TypeCheck(strict) using (_⊢ᴮ_∈_; _⊢ᴱ_∈_) +open import Luau.Addr.ToString using (addrToString) +open import Luau.Var.ToString using (varToString) +open import Luau.Type.ToString using (typeToString) +open import Luau.Syntax.ToString using (binOpToString) + +warningToStringᴱ : ∀ {H Γ T} M → {D : Γ ⊢ᴱ M ∈ T} → Warningᴱ H D → String +warningToStringᴮ : ∀ {H Γ T} B → {D : Γ ⊢ᴮ B ∈ T} → Warningᴮ H D → String + +warningToStringᴱ (var x) (UnboundVariable p) = "Unbound variable " ++ varToString x +warningToStringᴱ (val (addr a)) (UnallocatedAddress p) = "Unallocated adress " ++ addrToString a +warningToStringᴱ (M $ N) (FunctionCallMismatch {T = T} {U = U} p) = "Function has type " ++ typeToString T ++ " but argument has type " ++ typeToString U +warningToStringᴱ (M $ N) (app₁ W) = warningToStringᴱ M W +warningToStringᴱ (M $ N) (app₂ W) = warningToStringᴱ N W +warningToStringᴱ (function f ⟨ var x ∈ T ⟩∈ U is B end) (FunctionDefnMismatch {V = V} p) = "Function expresion " ++ varToString f ++ " has return type " ++ typeToString U ++ " but body returns " ++ typeToString V +warningToStringᴱ (function f ⟨ var x ∈ T ⟩∈ U is B end) (function₁ W) = warningToStringᴮ B W ++ "\n in function expression " ++ varToString f +warningToStringᴱ block var b ∈ T is B end (BlockMismatch {U = U} p) = "Block " ++ varToString b ++ " has type " ++ typeToString T ++ " but body returns " ++ typeToString U +warningToStringᴱ block var b ∈ T is B end (block₁ W) = warningToStringᴮ B W ++ "\n in block " ++ varToString b +warningToStringᴱ (binexp M op N) (BinOpMismatch₁ {T = T} p) = "Binary operator " ++ binOpToString op ++ " lhs has type " ++ typeToString T +warningToStringᴱ (binexp M op N) (BinOpMismatch₂ {U = U} p) = "Binary operator " ++ binOpToString op ++ " rhs has type " ++ typeToString U +warningToStringᴱ (binexp M op N) (bin₁ W) = warningToStringᴱ M W +warningToStringᴱ (binexp M op N) (bin₂ W) = warningToStringᴱ N W + +warningToStringᴮ (function f ⟨ var x ∈ T ⟩∈ U is C end ∙ B) (FunctionDefnMismatch {V = V} p) = "Function declaration " ++ varToString f ++ " has return type " ++ typeToString U ++ " but body returns " ++ typeToString V +warningToStringᴮ (function f ⟨ var x ∈ T ⟩∈ U is C end ∙ B) (function₁ W) = warningToStringᴮ C W ++ "\n in function declaration " ++ varToString f +warningToStringᴮ (function f ⟨ var x ∈ T ⟩∈ U is C end ∙ B) (function₂ W) = warningToStringᴮ B W +warningToStringᴮ (local var x ∈ T ← M ∙ B) (LocalVarMismatch {U = U} p) = "Local variable " ++ varToString x ++ " has type " ++ typeToString T ++ " but expression has type " ++ typeToString U +warningToStringᴮ (local var x ∈ T ← M ∙ B) (local₁ W) = warningToStringᴱ M W ++ "\n in local variable declaration " ++ varToString x +warningToStringᴮ (local var x ∈ T ← M ∙ B) (local₂ W) = warningToStringᴮ B W +warningToStringᴮ (return M ∙ B) (return W) = warningToStringᴱ M W ++ "\n in return statement" + diff --git a/prototyping/Luau/Substitution.agda b/prototyping/Luau/Substitution.agda index d909c350..883cc638 100644 --- a/prototyping/Luau/Substitution.agda +++ b/prototyping/Luau/Substitution.agda @@ -1,7 +1,6 @@ module Luau.Substitution where -open import Luau.Syntax using (Expr; Stat; Block; nil; true; false; addr; var; function_is_end; _$_; block_is_end; local_←_; _∙_; done; return; _⟨_⟩ ; name; fun; arg; number; binexp) -open import Luau.Value using (Value; val) +open import Luau.Syntax using (Value; Expr; Stat; Block; val; nil; bool; addr; var; function_is_end; _$_; block_is_end; local_←_; _∙_; done; return; _⟨_⟩ ; name; fun; arg; number; binexp) open import Luau.Var using (Var; _≡ⱽ_) open import Properties.Dec using (Dec; yes; no) @@ -10,18 +9,14 @@ _[_/_]ᴮ : ∀ {a} → Block a → Value → Var → Block a var_[_/_]ᴱwhenever_ : ∀ {a P} → Var → Value → Var → (Dec P) → Expr a _[_/_]ᴮunless_ : ∀ {a P} → Block a → Value → Var → (Dec P) → Block a -nil [ v / x ]ᴱ = nil -true [ v / x ]ᴱ = true -false [ v / x ]ᴱ = false +val w [ v / x ]ᴱ = val w var y [ v / x ]ᴱ = var y [ v / x ]ᴱwhenever (x ≡ⱽ y) -addr a [ v / x ]ᴱ = addr a -(number y) [ v / x ]ᴱ = number y (M $ N) [ v / x ]ᴱ = (M [ v / x ]ᴱ) $ (N [ v / x ]ᴱ) function F is C end [ v / x ]ᴱ = function F is C [ v / x ]ᴮunless (x ≡ⱽ name(arg F)) end block b is C end [ v / x ]ᴱ = block b is C [ v / x ]ᴮ end (binexp e₁ op e₂) [ v / x ]ᴱ = binexp (e₁ [ v / x ]ᴱ) op (e₂ [ v / x ]ᴱ) -(function F is C end ∙ B) [ v / x ]ᴮ = function F is (C [ v / x ]ᴮunless (x ≡ⱽ name(arg F))) end ∙ (B [ v / x ]ᴮunless (x ≡ⱽ fun F)) +(function F is C end ∙ B) [ v / x ]ᴮ = function F is (C [ v / x ]ᴮunless (x ≡ⱽ name(arg F))) end ∙ (B [ v / x ]ᴮunless (x ≡ⱽ name(fun F))) (local y ← M ∙ B) [ v / x ]ᴮ = local y ← (M [ v / x ]ᴱ) ∙ (B [ v / x ]ᴮunless (x ≡ⱽ name y)) (return M ∙ B) [ v / x ]ᴮ = return (M [ v / x ]ᴱ) ∙ (B [ v / x ]ᴮ) done [ v / x ]ᴮ = done diff --git a/prototyping/Luau/Syntax.agda b/prototyping/Luau/Syntax.agda index 57e59a7e..8be0830f 100644 --- a/prototyping/Luau/Syntax.agda +++ b/prototyping/Luau/Syntax.agda @@ -1,11 +1,12 @@ module Luau.Syntax where open import Agda.Builtin.Equality using (_≡_) +open import Agda.Builtin.Bool using (Bool; true; false) open import Agda.Builtin.Float using (Float) -open import Properties.Dec using (⊥) open import Luau.Var using (Var) open import Luau.Addr using (Addr) open import Luau.Type using (Type) +open import FFI.Data.Maybe using (Maybe; just; nothing) infixr 5 _∙_ @@ -25,9 +26,9 @@ data FunDec : Annotated → Set where _⟨_⟩∈_ : ∀ {a} → Var → VarDec a → Type → FunDec a _⟨_⟩ : Var → VarDec maybe → FunDec maybe -fun : ∀ {a} → FunDec a → Var -fun (f ⟨ x ⟩∈ T) = f -fun (f ⟨ x ⟩) = f +fun : ∀ {a} → FunDec a → VarDec a +fun (f ⟨ x ⟩∈ T) = (var f ∈ T) +fun (f ⟨ x ⟩) = (var f) arg : ∀ {a} → FunDec a → VarDec a arg (f ⟨ x ⟩∈ T) = x @@ -40,10 +41,16 @@ data BinaryOperator : Set where / : BinaryOperator < : BinaryOperator > : BinaryOperator - ≡ : BinaryOperator - ≅ : BinaryOperator - ≤ : BinaryOperator - ≥ : BinaryOperator + == : BinaryOperator + ~= : BinaryOperator + <= : BinaryOperator + >= : BinaryOperator + +data Value : Set where + nil : Value + addr : Addr → Value + number : Float → Value + bool : Bool → Value data Block (a : Annotated) : Set data Stat (a : Annotated) : Set @@ -59,13 +66,42 @@ data Stat a where return : Expr a → Stat a data Expr a where - nil : Expr a - true : Expr a - false : Expr a var : Var → Expr a - addr : Addr → Expr a + val : Value → Expr a _$_ : Expr a → Expr a → Expr a function_is_end : FunDec a → Block a → Expr a - block_is_end : Var → Block a → Expr a - number : Float → Expr a + block_is_end : VarDec a → Block a → Expr a binexp : Expr a → BinaryOperator → Expr a → Expr a + +isAnnotatedᴱ : ∀ {a} → Expr a → Maybe (Expr yes) +isAnnotatedᴮ : ∀ {a} → Block a → Maybe (Block yes) + +isAnnotatedᴱ (var x) = just (var x) +isAnnotatedᴱ (val v) = just (val v) +isAnnotatedᴱ (M $ N) with isAnnotatedᴱ M | isAnnotatedᴱ N +isAnnotatedᴱ (M $ N) | just M′ | just N′ = just (M′ $ N′) +isAnnotatedᴱ (M $ N) | _ | _ = nothing +isAnnotatedᴱ (function f ⟨ var x ∈ T ⟩∈ U is B end) with isAnnotatedᴮ B +isAnnotatedᴱ (function f ⟨ var x ∈ T ⟩∈ U is B end) | just B′ = just (function f ⟨ var x ∈ T ⟩∈ U is B′ end) +isAnnotatedᴱ (function f ⟨ var x ∈ T ⟩∈ U is B end) | _ = nothing +isAnnotatedᴱ (function _ is B end) = nothing +isAnnotatedᴱ (block var b ∈ T is B end) with isAnnotatedᴮ B +isAnnotatedᴱ (block var b ∈ T is B end) | just B′ = just (block var b ∈ T is B′ end) +isAnnotatedᴱ (block var b ∈ T is B end) | _ = nothing +isAnnotatedᴱ (block _ is B end) = nothing +isAnnotatedᴱ (binexp M op N) with isAnnotatedᴱ M | isAnnotatedᴱ N +isAnnotatedᴱ (binexp M op N) | just M′ | just N′ = just (binexp M′ op N′) +isAnnotatedᴱ (binexp M op N) | _ | _ = nothing + +isAnnotatedᴮ (function f ⟨ var x ∈ T ⟩∈ U is C end ∙ B) with isAnnotatedᴮ B | isAnnotatedᴮ C +isAnnotatedᴮ (function f ⟨ var x ∈ T ⟩∈ U is C end ∙ B) | just B′ | just C′ = just (function f ⟨ var x ∈ T ⟩∈ U is C′ end ∙ B′) +isAnnotatedᴮ (function f ⟨ var x ∈ T ⟩∈ U is C end ∙ B) | _ | _ = nothing +isAnnotatedᴮ (function _ is C end ∙ B) = nothing +isAnnotatedᴮ (local var x ∈ T ← M ∙ B) with isAnnotatedᴱ M | isAnnotatedᴮ B +isAnnotatedᴮ (local var x ∈ T ← M ∙ B) | just M′ | just B′ = just (local var x ∈ T ← M′ ∙ B′) +isAnnotatedᴮ (local var x ∈ T ← M ∙ B) | _ | _ = nothing +isAnnotatedᴮ (local _ ← M ∙ B) = nothing +isAnnotatedᴮ (return M ∙ B) with isAnnotatedᴱ M | isAnnotatedᴮ B +isAnnotatedᴮ (return M ∙ B) | just M′ | just B′ = just (return M′ ∙ B′) +isAnnotatedᴮ (return M ∙ B) | _ | _ = nothing +isAnnotatedᴮ done = just done diff --git a/prototyping/Luau/Syntax/FromJSON.agda b/prototyping/Luau/Syntax/FromJSON.agda index 330739b9..a727a5b0 100644 --- a/prototyping/Luau/Syntax/FromJSON.agda +++ b/prototyping/Luau/Syntax/FromJSON.agda @@ -1,6 +1,8 @@ +{-# OPTIONS --rewriting #-} + module Luau.Syntax.FromJSON where -open import Luau.Syntax using (Block; Stat ; Expr; nil; _$_; var; var_∈_; function_is_end; _⟨_⟩; local_←_; return; done; _∙_; maybe; VarDec; number; binexp; BinaryOperator; +; -; *; /; ≡; ≅; <; >; ≤; ≥) +open import Luau.Syntax using (Block; Stat ; Expr; _$_; val; nil; bool; number; var; var_∈_; function_is_end; _⟨_⟩; _⟨_⟩∈_; local_←_; return; done; _∙_; maybe; VarDec; binexp; BinaryOperator; +; -; *; /; ==; ~=; <; >; <=; >=) open import Luau.Type.FromJSON using (typeFromJSON) open import Agda.Builtin.List using (List; _∷_; []) @@ -26,6 +28,8 @@ vars = fromString "vars" op = fromString "op" left = fromString "left" right = fromString "right" +returnAnnotation = fromString "returnAnnotation" +types = fromString "types" data Lookup : Set where _,_ : String → Value → Lookup @@ -49,22 +53,22 @@ blockFromJSON : Value → Either String (Block maybe) blockFromArray : Array → Either String (Block maybe) binOpFromJSON (string s) = binOpFromString s -binOpFromJSON val = Left "Binary operator not a string" +binOpFromJSON _ = Left "Binary operator not a string" binOpFromString "Add" = Right + binOpFromString "Sub" = Right - binOpFromString "Mul" = Right * binOpFromString "Div" = Right / -binOpFromString "CompareEq" = Right ≡ -binOpFromString "CompareNe" = Right ≅ +binOpFromString "CompareEq" = Right == +binOpFromString "CompareNe" = Right ~= binOpFromString "CompareLt" = Right < -binOpFromString "CompareLe" = Right ≤ +binOpFromString "CompareLe" = Right <= binOpFromString "CompareGt" = Right > -binOpFromString "CompareGe" = Right ≥ +binOpFromString "CompareGe" = Right >= binOpFromString s = Left ("'" ++ s ++ "' is not a valid operator") varDecFromJSON (object arg) = varDecFromObject arg -varDecFromJSON val = Left "VarDec not an object" +varDecFromJSON _ = Left "VarDec not an object" varDecFromObject obj with lookup name obj | lookup type obj varDecFromObject obj | just (string name) | nothing = Right (var name) @@ -76,7 +80,7 @@ varDecFromObject obj | just _ | _ = Left "AstLocal name is not a string" varDecFromObject obj | nothing | _ = Left "AstLocal missing name" exprFromJSON (object obj) = exprFromObject obj -exprFromJSON val = Left "AstExpr not an object" +exprFromJSON _ = Left "AstExpr not an object" exprFromObject obj with lookup type obj exprFromObject obj | just (string "AstExprCall") with lookup func obj | lookup args obj @@ -89,29 +93,37 @@ exprFromObject obj | just (string "AstExprCall") | just value | just (array arr) exprFromObject obj | just (string "AstExprCall") | just value | just _ = Left ("AstExprCall args not an array") exprFromObject obj | just (string "AstExprCall") | nothing | _ = Left ("AstExprCall missing func") exprFromObject obj | just (string "AstExprCall") | _ | nothing = Left ("AstExprCall missing args") -exprFromObject obj | just (string "AstExprConstantNil") = Right nil -exprFromObject obj | just (string "AstExprFunction") with lookup args obj | lookup body obj -exprFromObject obj | just (string "AstExprFunction") | just (array arr) | just blockValue with head arr | blockFromJSON blockValue -exprFromObject obj | just (string "AstExprFunction") | just (array arr) | just blockValue | just argValue | Right B with varDecFromJSON argValue -exprFromObject obj | just (string "AstExprFunction") | just (array arr) | just blockValue | just argValue | Right B | Right arg = Right (function "" ⟨ arg ⟩ is B end) -exprFromObject obj | just (string "AstExprFunction") | just (array arr) | just blockValue | just argValue | Right B | Left err = Left err -exprFromObject obj | just (string "AstExprFunction") | just (array arr) | just blockValue | nothing | Right B = Left "Unsupported AstExprFunction empty args" -exprFromObject obj | just (string "AstExprFunction") | just (array arr) | just blockValue | _ | Left err = Left err -exprFromObject obj | just (string "AstExprFunction") | just _ | just _ = Left "AstExprFunction args not an array" -exprFromObject obj | just (string "AstExprFunction") | nothing | _ = Left "AstExprFunction missing args" -exprFromObject obj | just (string "AstExprFunction") | _ | nothing = Left "AstExprFunction missing body" +exprFromObject obj | just (string "AstExprConstantNil") = Right (val nil) +exprFromObject obj | just (string "AstExprFunction") with lookup args obj | lookup body obj | lookup returnAnnotation obj +exprFromObject obj | just (string "AstExprFunction") | just (array arr) | just blockValue | rtn with head arr | blockFromJSON blockValue +exprFromObject obj | just (string "AstExprFunction") | just (array arr) | just blockValue | rtn | just argValue | Right B with varDecFromJSON argValue +exprFromObject obj | just (string "AstExprFunction") | just (array arr) | just blockValue | just (object rtnObj) | just argValue | Right B | Right arg with lookup types rtnObj +exprFromObject obj | just (string "AstExprFunction") | just (array arr) | just blockValue | just (object rtnObj) | just argValue | Right B | Right arg | just (array rtnTypes) with head rtnTypes +exprFromObject obj | just (string "AstExprFunction") | just (array arr) | just blockValue | just (object rtnObj) | just argValue | Right B | Right arg | just (array rtnTypes) | just rtnType with typeFromJSON rtnType +exprFromObject obj | just (string "AstExprFunction") | just (array arr) | just blockValue | just (object rtnObj) | just argValue | Right B | Right arg | just (array rtnTypes) | just rtnType | Left err = Left err +exprFromObject obj | just (string "AstExprFunction") | just (array arr) | just blockValue | just (object rtnObj) | just argValue | Right B | Right arg | just (array rtnTypes) | just rtnType | Right T = Right (function "" ⟨ arg ⟩∈ T is B end) +exprFromObject obj | just (string "AstExprFunction") | just (array arr) | just blockValue | just (object rtnObj) | just argValue | Right B | Right arg | just (array rtnTypes) | nothing = Right (function "" ⟨ arg ⟩ is B end) +exprFromObject obj | just (string "AstExprFunction") | just (array arr) | just blockValue | just (object rtnObj) | just argValue | Right B | Right arg | just _ = Left "returnAnnotation types not an array" +exprFromObject obj | just (string "AstExprFunction") | just (array arr) | just blockValue | just (object rtnObj) | just argValue | Right B | Right arg | nothing = Left "returnAnnotation missing types" +exprFromObject obj | just (string "AstExprFunction") | just (array arr) | just blockValue | just _ | just argValue | Right B | Right arg = Left "returnAnnotation not an object" +exprFromObject obj | just (string "AstExprFunction") | just (array arr) | just blockValue | nothing | just argValue | Right B | Right arg = Right (function "" ⟨ arg ⟩ is B end) +exprFromObject obj | just (string "AstExprFunction") | just (array arr) | just blockValue | rtn | just argValue | Right B | Left err = Left err +exprFromObject obj | just (string "AstExprFunction") | just (array arr) | just blockValue | rtn | nothing | Right B = Left "Unsupported AstExprFunction empty args" +exprFromObject obj | just (string "AstExprFunction") | just (array arr) | just blockValue | rtn | _ | Left err = Left err +exprFromObject obj | just (string "AstExprFunction") | just _ | just _ | rtn = Left "AstExprFunction args not an array" +exprFromObject obj | just (string "AstExprFunction") | nothing | _ | rtn = Left "AstExprFunction missing args" +exprFromObject obj | just (string "AstExprFunction") | _ | nothing | rtn = Left "AstExprFunction missing body" exprFromObject obj | just (string "AstExprLocal") with lookup lokal obj exprFromObject obj | just (string "AstExprLocal") | just x with varDecFromJSON x exprFromObject obj | just (string "AstExprLocal") | just x | Right x′ = Right (var (Luau.Syntax.name x′)) exprFromObject obj | just (string "AstExprLocal") | just x | Left err = Left err exprFromObject obj | just (string "AstExprLocal") | nothing = Left "AstExprLocal missing local" exprFromObject obj | just (string "AstExprConstantNumber") with lookup value obj -exprFromObject obj | just (string "AstExprConstantNumber") | just (FFI.Data.Aeson.Value.number x) = Right (number (toFloat x)) +exprFromObject obj | just (string "AstExprConstantNumber") | just (FFI.Data.Aeson.Value.number x) = Right (val (number (toFloat x))) exprFromObject obj | just (string "AstExprConstantNumber") | just _ = Left "AstExprConstantNumber value is not a number" exprFromObject obj | just (string "AstExprConstantNumber") | nothing = Left "AstExprConstantNumber missing value" exprFromObject obj | just (string "AstExprConstantBool") with lookup value obj -exprFromObject obj | just (string "AstExprConstantBool") | just (FFI.Data.Aeson.Value.bool true) = Right Expr.true -exprFromObject obj | just (string "AstExprConstantBool") | just (FFI.Data.Aeson.Value.bool false) = Right Expr.false +exprFromObject obj | just (string "AstExprConstantBool") | just (FFI.Data.Aeson.Value.bool b) = Right (val (bool b)) exprFromObject obj | just (string "AstExprConstantBool") | just _ = Left "AstExprConstantBool value is not a bool" exprFromObject obj | just (string "AstExprConstantBool") | nothing = Left "AstExprConstantBool missing value" exprFromObject obj | just (string "AstExprBinary") with lookup op obj | lookup left obj | lookup right obj @@ -147,6 +159,7 @@ statFromObject obj | just(string "AstStatLocal") | nothing | _ = Left "AstStatLo statFromObject obj | just(string "AstStatLocalFunction") with lookup name obj | lookup func obj statFromObject obj | just(string "AstStatLocalFunction") | just fnName | just value with varDecFromJSON fnName | exprFromJSON value statFromObject obj | just(string "AstStatLocalFunction") | just fnName | just value | Right fnVar | Right (function "" ⟨ x ⟩ is B end) = Right (function (Luau.Syntax.name fnVar) ⟨ x ⟩ is B end) +statFromObject obj | just(string "AstStatLocalFunction") | just fnName | just value | Right fnVar | Right (function "" ⟨ x ⟩∈ T is B end) = Right (function (Luau.Syntax.name fnVar) ⟨ x ⟩∈ T is B end) statFromObject obj | just(string "AstStatLocalFunction") | just fnName | just value | Left err | _ = Left err statFromObject obj | just(string "AstStatLocalFunction") | just fnName | just value | _ | Left err = Left err statFromObject obj | just(string "AstStatLocalFunction") | just _ | just _ | Right _ | Right _ = Left "AstStatLocalFunction func is not an AstExprFunction" @@ -180,4 +193,4 @@ blockFromArray arr | just value | Left err = Left err blockFromArray arr | just value | Right S with blockFromArray(tail arr) blockFromArray arr | just value | Right S | Left err = Left (err) blockFromArray arr | just value | Right S | Right B = Right (S ∙ B) - \ No newline at end of file + diff --git a/prototyping/Luau/Syntax/ToString.agda b/prototyping/Luau/Syntax/ToString.agda index 58a784bf..60ec471d 100644 --- a/prototyping/Luau/Syntax/ToString.agda +++ b/prototyping/Luau/Syntax/ToString.agda @@ -1,7 +1,8 @@ module Luau.Syntax.ToString where +open import Agda.Builtin.Bool using (true; false) open import Agda.Builtin.Float using (primShowFloat) -open import Luau.Syntax using (Block; Stat; Expr; VarDec; FunDec; nil; var; var_∈_; addr; _$_; function_is_end; return; local_←_; _∙_; done; block_is_end; _⟨_⟩; _⟨_⟩∈_; number; BinaryOperator; +; -; *; /; <; >; ≡; ≅; ≤; ≥; binexp; true; false) +open import Luau.Syntax using (Value; Block; Stat; Expr; VarDec; FunDec; nil; bool; val; var; var_∈_; addr; _$_; function_is_end; return; local_←_; _∙_; done; block_is_end; _⟨_⟩; _⟨_⟩∈_; number; BinaryOperator; +; -; *; /; <; >; ==; ~=; <=; >=; binexp) open import FFI.Data.String using (String; _++_) open import Luau.Addr.ToString using (addrToString) open import Luau.Type.ToString using (typeToString) @@ -24,19 +25,24 @@ binOpToString * = "*" binOpToString / = "/" binOpToString < = "<" binOpToString > = ">" -binOpToString ≡ = "==" -binOpToString ≅ = "~=" -binOpToString ≤ = "<=" -binOpToString ≥ = ">=" +binOpToString == = "==" +binOpToString ~= = "~=" +binOpToString <= = "<=" +binOpToString >= = ">=" + +valueToString : Value → String +valueToString nil = "nil" +valueToString (addr a) = addrToString a +valueToString (number x) = primShowFloat x +valueToString (bool false) = "false" +valueToString (bool true) = "true" exprToString′ : ∀ {a} → String → Expr a → String statToString′ : ∀ {a} → String → Stat a → String blockToString′ : ∀ {a} → String → Block a → String -exprToString′ lb nil = - "nil" -exprToString′ lb (addr a) = - addrToString(a) +exprToString′ lb (val v) = + valueToString(v) exprToString′ lb (var x) = varToString(x) exprToString′ lb (M $ N) = @@ -46,13 +52,10 @@ exprToString′ lb (function F is B end) = " " ++ (blockToString′ (lb ++ " ") B) ++ lb ++ "end" exprToString′ lb (block b is B end) = - "(" ++ b ++ "()" ++ lb ++ + "(" ++ varDecToString b ++ "()" ++ lb ++ " " ++ (blockToString′ (lb ++ " ") B) ++ lb ++ "end)()" -exprToString′ lb (number x) = primShowFloat x exprToString′ lb (binexp x op y) = exprToString′ lb x ++ " " ++ binOpToString op ++ " " ++ exprToString′ lb y -exprToString′ lb true = "true" -exprToString′ lb false = "false" statToString′ lb (function F is B end) = "local " ++ funDecToString F ++ lb ++ diff --git a/prototyping/Luau/Type.agda b/prototyping/Luau/Type.agda index af5f857c..10f39c44 100644 --- a/prototyping/Luau/Type.agda +++ b/prototyping/Luau/Type.agda @@ -1,5 +1,9 @@ module Luau.Type where +open import FFI.Data.Maybe using (Maybe; just; nothing; just-inv) +open import Agda.Builtin.Equality using (_≡_; refl) +open import Properties.Dec using (Dec; yes; no) +open import Properties.Equality using (cong) open import FFI.Data.Maybe using (Maybe; just; nothing) data Type : Set where @@ -7,18 +11,132 @@ data Type : Set where _⇒_ : Type → Type → Type none : Type any : Type + boolean : Type number : Type _∪_ : Type → Type → Type _∩_ : Type → Type → Type -src : Type → Type -src nil = none -src (S ⇒ T) = S -src none = none -src any = any -src number = none -src (S ∪ T) = (src S) ∪ (src T) -src (S ∩ T) = (src S) ∩ (src T) +lhs : Type → Type +lhs (T ⇒ _) = T +lhs (T ∪ _) = T +lhs (T ∩ _) = T +lhs nil = nil +lhs none = none +lhs any = any +lhs number = number +lhs boolean = boolean + +rhs : Type → Type +rhs (_ ⇒ T) = T +rhs (_ ∪ T) = T +rhs (_ ∩ T) = T +rhs nil = nil +rhs none = none +rhs any = any +rhs number = number +rhs boolean = boolean + +_≡ᵀ_ : ∀ (T U : Type) → Dec(T ≡ U) +nil ≡ᵀ nil = yes refl +nil ≡ᵀ (S ⇒ T) = no (λ ()) +nil ≡ᵀ none = no (λ ()) +nil ≡ᵀ any = no (λ ()) +nil ≡ᵀ number = no (λ ()) +nil ≡ᵀ boolean = no (λ ()) +nil ≡ᵀ (S ∪ T) = no (λ ()) +nil ≡ᵀ (S ∩ T) = no (λ ()) +(S ⇒ T) ≡ᵀ nil = no (λ ()) +(S ⇒ T) ≡ᵀ (U ⇒ V) with (S ≡ᵀ U) | (T ≡ᵀ V) +(S ⇒ T) ≡ᵀ (S ⇒ T) | yes refl | yes refl = yes refl +(S ⇒ T) ≡ᵀ (U ⇒ V) | _ | no p = no (λ q → p (cong rhs q)) +(S ⇒ T) ≡ᵀ (U ⇒ V) | no p | _ = no (λ q → p (cong lhs q)) +(S ⇒ T) ≡ᵀ none = no (λ ()) +(S ⇒ T) ≡ᵀ any = no (λ ()) +(S ⇒ T) ≡ᵀ number = no (λ ()) +(S ⇒ T) ≡ᵀ boolean = no (λ ()) +(S ⇒ T) ≡ᵀ (U ∪ V) = no (λ ()) +(S ⇒ T) ≡ᵀ (U ∩ V) = no (λ ()) +none ≡ᵀ nil = no (λ ()) +none ≡ᵀ (U ⇒ V) = no (λ ()) +none ≡ᵀ none = yes refl +none ≡ᵀ any = no (λ ()) +none ≡ᵀ number = no (λ ()) +none ≡ᵀ boolean = no (λ ()) +none ≡ᵀ (U ∪ V) = no (λ ()) +none ≡ᵀ (U ∩ V) = no (λ ()) +any ≡ᵀ nil = no (λ ()) +any ≡ᵀ (U ⇒ V) = no (λ ()) +any ≡ᵀ none = no (λ ()) +any ≡ᵀ any = yes refl +any ≡ᵀ number = no (λ ()) +any ≡ᵀ boolean = no (λ ()) +any ≡ᵀ (U ∪ V) = no (λ ()) +any ≡ᵀ (U ∩ V) = no (λ ()) +number ≡ᵀ nil = no (λ ()) +number ≡ᵀ (T ⇒ U) = no (λ ()) +number ≡ᵀ none = no (λ ()) +number ≡ᵀ any = no (λ ()) +number ≡ᵀ number = yes refl +number ≡ᵀ boolean = no (λ ()) +number ≡ᵀ (T ∪ U) = no (λ ()) +number ≡ᵀ (T ∩ U) = no (λ ()) +boolean ≡ᵀ nil = no (λ ()) +boolean ≡ᵀ (T ⇒ U) = no (λ ()) +boolean ≡ᵀ none = no (λ ()) +boolean ≡ᵀ any = no (λ ()) +boolean ≡ᵀ boolean = yes refl +boolean ≡ᵀ number = no (λ ()) +boolean ≡ᵀ (T ∪ U) = no (λ ()) +boolean ≡ᵀ (T ∩ U) = no (λ ()) +(S ∪ T) ≡ᵀ nil = no (λ ()) +(S ∪ T) ≡ᵀ (U ⇒ V) = no (λ ()) +(S ∪ T) ≡ᵀ none = no (λ ()) +(S ∪ T) ≡ᵀ any = no (λ ()) +(S ∪ T) ≡ᵀ number = no (λ ()) +(S ∪ T) ≡ᵀ boolean = no (λ ()) +(S ∪ T) ≡ᵀ (U ∪ V) with (S ≡ᵀ U) | (T ≡ᵀ V) +(S ∪ T) ≡ᵀ (S ∪ T) | yes refl | yes refl = yes refl +(S ∪ T) ≡ᵀ (U ∪ V) | _ | no p = no (λ q → p (cong rhs q)) +(S ∪ T) ≡ᵀ (U ∪ V) | no p | _ = no (λ q → p (cong lhs q)) +(S ∪ T) ≡ᵀ (U ∩ V) = no (λ ()) +(S ∩ T) ≡ᵀ nil = no (λ ()) +(S ∩ T) ≡ᵀ (U ⇒ V) = no (λ ()) +(S ∩ T) ≡ᵀ none = no (λ ()) +(S ∩ T) ≡ᵀ any = no (λ ()) +(S ∩ T) ≡ᵀ number = no (λ ()) +(S ∩ T) ≡ᵀ boolean = no (λ ()) +(S ∩ T) ≡ᵀ (U ∪ V) = no (λ ()) +(S ∩ T) ≡ᵀ (U ∩ V) with (S ≡ᵀ U) | (T ≡ᵀ V) +(S ∩ T) ≡ᵀ (U ∩ V) | yes refl | yes refl = yes refl +(S ∩ T) ≡ᵀ (U ∩ V) | _ | no p = no (λ q → p (cong rhs q)) +(S ∩ T) ≡ᵀ (U ∩ V) | no p | _ = no (λ q → p (cong lhs q)) + +_≡ᴹᵀ_ : ∀ (T U : Maybe Type) → Dec(T ≡ U) +nothing ≡ᴹᵀ nothing = yes refl +nothing ≡ᴹᵀ just U = no (λ ()) +just T ≡ᴹᵀ nothing = no (λ ()) +just T ≡ᴹᵀ just U with T ≡ᵀ U +(just T ≡ᴹᵀ just T) | yes refl = yes refl +(just T ≡ᴹᵀ just U) | no p = no (λ q → p (just-inv q)) + +data Mode : Set where + strict : Mode + nonstrict : Mode + +src : Mode → Type → Type +src m nil = none +src m number = none +src m boolean = none +src m (S ⇒ T) = S +-- In nonstrict mode, functions are covaraiant, in strict mode they're contravariant +src strict (S ∪ T) = (src strict S) ∩ (src strict T) +src nonstrict (S ∪ T) = (src nonstrict S) ∪ (src nonstrict T) +src strict (S ∩ T) = (src strict S) ∪ (src strict T) +src nonstrict (S ∩ T) = (src nonstrict S) ∩ (src nonstrict T) +src strict none = any +src nonstrict none = none +src strict any = none +src nonstrict any = any tgt : Type → Type tgt nil = none @@ -26,6 +144,7 @@ tgt (S ⇒ T) = T tgt none = none tgt any = any tgt number = none +tgt boolean = none tgt (S ∪ T) = (tgt S) ∪ (tgt T) tgt (S ∩ T) = (tgt S) ∩ (tgt T) diff --git a/prototyping/Luau/Type/FromJSON.agda b/prototyping/Luau/Type/FromJSON.agda index 49f74626..9057fef4 100644 --- a/prototyping/Luau/Type/FromJSON.agda +++ b/prototyping/Luau/Type/FromJSON.agda @@ -1,3 +1,5 @@ +{-# OPTIONS --rewriting #-} + module Luau.Type.FromJSON where open import Luau.Type using (Type; nil; _⇒_; _∪_; _∩_; any; number) diff --git a/prototyping/Luau/Type/ToString.agda b/prototyping/Luau/Type/ToString.agda index e7a7c1c0..dec7c14c 100644 --- a/prototyping/Luau/Type/ToString.agda +++ b/prototyping/Luau/Type/ToString.agda @@ -1,7 +1,7 @@ module Luau.Type.ToString where open import FFI.Data.String using (String; _++_) -open import Luau.Type using (Type; nil; _⇒_; none; any; number; _∪_; _∩_; normalizeOptional) +open import Luau.Type using (Type; nil; _⇒_; none; any; number; boolean; _∪_; _∩_; normalizeOptional) {-# TERMINATING #-} typeToString : Type → String @@ -13,6 +13,7 @@ typeToString (S ⇒ T) = "(" ++ (typeToString S) ++ ") -> " ++ (typeToString T) typeToString none = "none" typeToString any = "any" typeToString number = "number" +typeToString boolean = "boolean" typeToString (S ∪ T) with normalizeOptional(S ∪ T) typeToString (S ∪ T) | ((S′ ⇒ T′) ∪ nil) = "(" ++ typeToString (S′ ⇒ T′) ++ ")?" typeToString (S ∪ T) | (S′ ∪ nil) = typeToString S′ ++ "?" diff --git a/prototyping/Luau/TypeCheck.agda b/prototyping/Luau/TypeCheck.agda new file mode 100644 index 00000000..f240e17e --- /dev/null +++ b/prototyping/Luau/TypeCheck.agda @@ -0,0 +1,143 @@ +{-# OPTIONS --rewriting #-} + +open import Luau.Type using (Mode) + +module Luau.TypeCheck (m : Mode) where + +open import Agda.Builtin.Equality using (_≡_) +open import FFI.Data.Maybe using (Maybe; just) +open import Luau.Syntax using (Expr; Stat; Block; BinaryOperator; yes; nil; addr; number; bool; val; var; var_∈_; _⟨_⟩∈_; function_is_end; _$_; block_is_end; binexp; local_←_; _∙_; done; return; name; +; -; *; /; <; >; ==; ~=; <=; >=) +open import Luau.Var using (Var) +open import Luau.Addr using (Addr) +open import Luau.Heap using (Heap; Object; function_is_end) renaming (_[_] to _[_]ᴴ) +open import Luau.Type using (Type; Mode; nil; none; number; boolean; _⇒_; tgt) +open import Luau.VarCtxt using (VarCtxt; ∅; _⋒_; _↦_; _⊕_↦_; _⊝_) renaming (_[_] to _[_]ⱽ) +open import FFI.Data.Vector using (Vector) +open import FFI.Data.Maybe using (Maybe; just; nothing) +open import Properties.Product using (_×_; _,_) + +src : Type → Type +src = Luau.Type.src m + +orNone : Maybe Type → Type +orNone nothing = none +orNone (just T) = T + +tgtBinOp : BinaryOperator → Type +tgtBinOp + = number +tgtBinOp - = number +tgtBinOp * = number +tgtBinOp / = number +tgtBinOp < = boolean +tgtBinOp > = boolean +tgtBinOp == = boolean +tgtBinOp ~= = boolean +tgtBinOp <= = boolean +tgtBinOp >= = boolean + +data _⊢ᴮ_∈_ : VarCtxt → Block yes → Type → Set +data _⊢ᴱ_∈_ : VarCtxt → Expr yes → Type → Set + +data _⊢ᴮ_∈_ where + + done : ∀ {Γ} → + + --------------- + Γ ⊢ᴮ done ∈ nil + + return : ∀ {M B T U Γ} → + + Γ ⊢ᴱ M ∈ T → + Γ ⊢ᴮ B ∈ U → + --------------------- + Γ ⊢ᴮ return M ∙ B ∈ T + + local : ∀ {x M B T U V Γ} → + + Γ ⊢ᴱ M ∈ U → + (Γ ⊕ x ↦ T) ⊢ᴮ B ∈ V → + -------------------------------- + Γ ⊢ᴮ local var x ∈ T ← M ∙ B ∈ V + + function : ∀ {f x B C T U V W Γ} → + + (Γ ⊕ x ↦ T) ⊢ᴮ C ∈ V → + (Γ ⊕ f ↦ (T ⇒ U)) ⊢ᴮ B ∈ W → + ------------------------------------------------- + Γ ⊢ᴮ function f ⟨ var x ∈ T ⟩∈ U is C end ∙ B ∈ W + +data _⊢ᴱ_∈_ where + + nil : ∀ {Γ} → + + -------------------- + Γ ⊢ᴱ (val nil) ∈ nil + + var : ∀ {x T Γ} → + + T ≡ orNone(Γ [ x ]ⱽ) → + ---------------- + Γ ⊢ᴱ (var x) ∈ T + + addr : ∀ {a Γ} T → + + ----------------- + Γ ⊢ᴱ val(addr a) ∈ T + + number : ∀ {n Γ} → + + --------------------------- + Γ ⊢ᴱ val(number n) ∈ number + + bool : ∀ {b Γ} → + + -------------------------- + Γ ⊢ᴱ val(bool b) ∈ boolean + + app : ∀ {M N T U Γ} → + + Γ ⊢ᴱ M ∈ T → + Γ ⊢ᴱ N ∈ U → + ---------------------- + Γ ⊢ᴱ (M $ N) ∈ (tgt T) + + function : ∀ {f x B T U V Γ} → + + (Γ ⊕ x ↦ T) ⊢ᴮ B ∈ V → + ----------------------------------------------------- + Γ ⊢ᴱ (function f ⟨ var x ∈ T ⟩∈ U is B end) ∈ (T ⇒ U) + + block : ∀ {b B T U Γ} → + + Γ ⊢ᴮ B ∈ U → + ------------------------------------ + Γ ⊢ᴱ (block var b ∈ T is B end) ∈ T + + binexp : ∀ {op Γ M N T U} → + + Γ ⊢ᴱ M ∈ T → + Γ ⊢ᴱ N ∈ U → + ---------------------------------- + Γ ⊢ᴱ (binexp M op N) ∈ tgtBinOp op + +data ⊢ᴼ_ : Maybe(Object yes) → Set where + + nothing : + + --------- + ⊢ᴼ nothing + + function : ∀ {f x T U V B} → + + (x ↦ T) ⊢ᴮ B ∈ V → + ---------------------------------------------- + ⊢ᴼ (just function f ⟨ var x ∈ T ⟩∈ U is B end) + +⊢ᴴ_ : Heap yes → Set +⊢ᴴ H = ∀ a {O} → (H [ a ]ᴴ ≡ O) → (⊢ᴼ O) + +_⊢ᴴᴱ_▷_∈_ : VarCtxt → Heap yes → Expr yes → Type → Set +(Γ ⊢ᴴᴱ H ▷ M ∈ T) = (⊢ᴴ H) × (Γ ⊢ᴱ M ∈ T) + +_⊢ᴴᴮ_▷_∈_ : VarCtxt → Heap yes → Block yes → Type → Set +(Γ ⊢ᴴᴮ H ▷ B ∈ T) = (⊢ᴴ H) × (Γ ⊢ᴮ B ∈ T) diff --git a/prototyping/Luau/Value.agda b/prototyping/Luau/Value.agda deleted file mode 100644 index ed51abd7..00000000 --- a/prototyping/Luau/Value.agda +++ /dev/null @@ -1,20 +0,0 @@ -module Luau.Value where - -open import Agda.Builtin.Bool using (Bool; true; false) -open import Agda.Builtin.Float using (Float) -open import Luau.Addr using (Addr) -open import Luau.Syntax using (Block; Expr; nil; addr; number; true; false) -open import Luau.Var using (Var) - -data Value : Set where - nil : Value - addr : Addr → Value - number : Float → Value - bool : Bool → Value - -val : ∀ {a} → Value → Expr a -val nil = nil -val (addr a) = addr a -val (number x) = number x -val (bool false) = false -val (bool true) = true diff --git a/prototyping/Luau/Value/ToString.agda b/prototyping/Luau/Value/ToString.agda deleted file mode 100644 index 3421d4cd..00000000 --- a/prototyping/Luau/Value/ToString.agda +++ /dev/null @@ -1,14 +0,0 @@ -module Luau.Value.ToString where - -open import Agda.Builtin.String using (String) -open import Agda.Builtin.Float using (primShowFloat) -open import Agda.Builtin.Bool using (true; false) -open import Luau.Value using (Value; nil; addr; number; bool) -open import Luau.Addr.ToString using (addrToString) - -valueToString : Value → String -valueToString nil = "nil" -valueToString (addr a) = addrToString a -valueToString (number x) = primShowFloat x -valueToString (bool false) = "false" -valueToString (bool true) = "true" diff --git a/prototyping/Luau/Var.agda b/prototyping/Luau/Var.agda index 091ce125..23d8b56e 100644 --- a/prototyping/Luau/Var.agda +++ b/prototyping/Luau/Var.agda @@ -4,13 +4,13 @@ open import Agda.Builtin.Bool using (true; false) open import Agda.Builtin.Equality using (_≡_) open import Agda.Builtin.String using (String; primStringEquality) open import Agda.Builtin.TrustMe using (primTrustMe) -open import Properties.Dec using (Dec; yes; no; ⊥) +open import Properties.Dec using (Dec; yes; no) +open import Properties.Equality using (_≢_) Var : Set Var = String _≡ⱽ_ : (a b : Var) → Dec (a ≡ b) a ≡ⱽ b with primStringEquality a b -a ≡ⱽ b | false = no p where postulate p : (a ≡ b) → ⊥ +a ≡ⱽ b | false = no p where postulate p : (a ≢ b) a ≡ⱽ b | true = yes primTrustMe - diff --git a/prototyping/Luau/VarCtxt.agda b/prototyping/Luau/VarCtxt.agda new file mode 100644 index 00000000..76f64dfe --- /dev/null +++ b/prototyping/Luau/VarCtxt.agda @@ -0,0 +1,43 @@ +{-# OPTIONS --rewriting #-} + +module Luau.VarCtxt where + +open import Agda.Builtin.Equality using (_≡_) +open import Luau.Type using (Type; _∪_; _∩_) +open import Luau.Var using (Var) +open import FFI.Data.Aeson using (KeyMap; Key; empty; unionWith; singleton; insert; delete; lookup; toString; fromString; lookup-insert; lookup-insert-not; lookup-empty; to-from; insert-swap; insert-over) +open import FFI.Data.Maybe using (Maybe; just; nothing) +open import Properties.Equality using (_≢_; cong; sym; trans) + +VarCtxt : Set +VarCtxt = KeyMap Type + +∅ : VarCtxt +∅ = empty + +_⋒_ : VarCtxt → VarCtxt → VarCtxt +_⋒_ = unionWith _∩_ + +_⋓_ : VarCtxt → VarCtxt → VarCtxt +_⋓_ = unionWith _∪_ + +_[_] : VarCtxt → Var → Maybe Type +Γ [ x ] = lookup (fromString x) Γ + +_⊝_ : VarCtxt → Var → VarCtxt +Γ ⊝ x = delete (fromString x) Γ + +_↦_ : Var → Type → VarCtxt +x ↦ T = singleton (fromString x) T + +_⊕_↦_ : VarCtxt → Var → Type → VarCtxt +Γ ⊕ x ↦ T = insert (fromString x) T Γ + +⊕-over : ∀ {Γ x y T U} → (x ≡ y) → ((Γ ⊕ x ↦ T) ⊕ y ↦ U) ≡ (Γ ⊕ y ↦ U) +⊕-over p = insert-over _ _ _ _ _ (cong fromString (sym p)) + +⊕-swap : ∀ {Γ x y T U} → (x ≢ y) → ((Γ ⊕ x ↦ T) ⊕ y ↦ U) ≡ ((Γ ⊕ y ↦ U) ⊕ x ↦ T) +⊕-swap p = insert-swap _ _ _ _ _ (λ q → p (trans (sym (to-from _)) (trans (cong toString (sym q) ) (to-from _))) ) + +⊕-lookup-miss : ∀ x y T Γ → (x ≢ y) → (Γ [ y ] ≡ (Γ ⊕ x ↦ T) [ y ]) +⊕-lookup-miss x y T Γ p = lookup-insert-not (fromString x) (fromString y) T Γ λ q → p (trans (sym (to-from x)) (trans (cong toString q) (to-from y))) diff --git a/prototyping/PrettyPrinter.agda b/prototyping/PrettyPrinter.agda index fdad32d6..3ce0905b 100644 --- a/prototyping/PrettyPrinter.agda +++ b/prototyping/PrettyPrinter.agda @@ -1,3 +1,5 @@ +{-# OPTIONS --rewriting #-} + module PrettyPrinter where open import Agda.Builtin.IO using (IO) diff --git a/prototyping/Properties.agda b/prototyping/Properties.agda index 1a6f92fa..8b30ce12 100644 --- a/prototyping/Properties.agda +++ b/prototyping/Properties.agda @@ -1,7 +1,11 @@ +{-# OPTIONS --rewriting #-} + module Properties where import Properties.Contradiction import Properties.Dec import Properties.Equality -import Properties.Step import Properties.Remember +import Properties.Step +import Properties.StrictMode +import Properties.TypeCheck diff --git a/prototyping/Properties/Dec.agda b/prototyping/Properties/Dec.agda index d89fd754..c61f2365 100644 --- a/prototyping/Properties/Dec.agda +++ b/prototyping/Properties/Dec.agda @@ -1,8 +1,7 @@ module Properties.Dec where -data ⊥ : Set where +open import Properties.Contradiction using (¬) data Dec(A : Set) : Set where yes : A → Dec A - no : (A → ⊥) → Dec A - + no : ¬ A → Dec A diff --git a/prototyping/Properties/Product.agda b/prototyping/Properties/Product.agda new file mode 100644 index 00000000..0d41c817 --- /dev/null +++ b/prototyping/Properties/Product.agda @@ -0,0 +1,14 @@ +module Properties.Product where + +infixr 5 _×_ _,_ + +record Σ {A : Set} (B : A → Set) : Set where + + constructor _,_ + field fst : A + field snd : B fst + +open Σ public + +_×_ : Set → Set → Set +A × B = Σ (λ (a : A) → B) diff --git a/prototyping/Properties/Step.agda b/prototyping/Properties/Step.agda index 69dbac84..0b5fd062 100644 --- a/prototyping/Properties/Step.agda +++ b/prototyping/Properties/Step.agda @@ -1,24 +1,91 @@ +{-# OPTIONS --rewriting #-} + module Properties.Step where open import Agda.Builtin.Equality using (_≡_; refl) -open import Agda.Builtin.Float using (primFloatPlus; primFloatMinus; primFloatTimes; primFloatDiv) +open import Agda.Builtin.Float using (primFloatPlus; primFloatMinus; primFloatTimes; primFloatDiv; primFloatEquality; primFloatLess) open import Agda.Builtin.Bool using (true; false) open import FFI.Data.Maybe using (just; nothing) open import Luau.Heap using (Heap; _[_]; alloc; ok; function_is_end) -open import Luau.Syntax using (Block; Expr; nil; var; addr; true; false; function_is_end; block_is_end; _$_; local_←_; return; done; _∙_; name; fun; arg; number; binexp; +; ≅) -open import Luau.OpSem using (_⊢_⟶ᴱ_⊣_; _⊢_⟶ᴮ_⊣_; app₁ ; app₂ ; beta; function; block; return; done; local; subst; binOpNumbers; evalNumOp; binOp₁; binOp₂; evalEqOp; evalNeqOp; binOpEquality; binOpInequality) -open import Luau.RuntimeError using (RuntimeErrorᴱ; RuntimeErrorᴮ; TypeMismatch; UnboundVariable; SEGV; app₁; app₂; block; local; return; bin₁; bin₂) -open import Luau.RuntimeType using (function; number) +open import Luau.Syntax using (Block; Expr; nil; var; val; addr; bool; function_is_end; block_is_end; _$_; local_←_; return; done; _∙_; name; fun; arg; number; binexp; +; -; *; /; <; >; <=; >=; ==; ~=) +open import Luau.OpSem using (_⟦_⟧_⟶_; _⊢_⟶ᴱ_⊣_; _⊢_⟶ᴮ_⊣_; app₁ ; app₂ ; beta; function; block; return; done; local; subst; binOp₀; binOp₁; binOp₂; +; -; *; /; <; >; <=; >=; ==; ~=; evalEqOp; evalNeqOp) +open import Luau.RuntimeError using (BinOpError; RuntimeErrorᴱ; RuntimeErrorᴮ; FunctionMismatch; BinOpMismatch₁; BinOpMismatch₂; UnboundVariable; SEGV; app₁; app₂; block; local; return; bin₁; bin₂; +; -; *; /; <; >; <=; >=) +open import Luau.RuntimeType using (valueType; function; number) open import Luau.Substitution using (_[_/_]ᴮ) -open import Luau.Value using (nil; addr; val; number; bool) open import Properties.Remember using (remember; _,_) +open import Utility.Bool using (not; _or_) + +data BinOpStepResult v op w : Set where + step : ∀ x → (v ⟦ op ⟧ w ⟶ x) → BinOpStepResult v op w + error₁ : BinOpError op (valueType(v)) → BinOpStepResult v op w + error₂ : BinOpError op (valueType(w)) → BinOpStepResult v op w + +binOpStep : ∀ v op w → BinOpStepResult v op w +binOpStep nil + w = error₁ (+ (λ ())) +binOpStep (addr a) + w = error₁ (+ (λ ())) +binOpStep (number m) + nil = error₂ (+ (λ ())) +binOpStep (number m) + (addr a) = error₂ (+ (λ ())) +binOpStep (number m) + (number n) = step (number (primFloatPlus m n)) (+ m n) +binOpStep (number m) + (bool b) = error₂ (+ (λ ())) +binOpStep (bool b) + w = error₁ (+ (λ ())) +binOpStep nil - w = error₁ (- (λ ())) +binOpStep (addr a) - w = error₁ (- (λ ())) +binOpStep (number x) - nil = error₂ (- (λ ())) +binOpStep (number x) - (addr a) = error₂ (- (λ ())) +binOpStep (number x) - (number n) = step (number (primFloatMinus x n)) (- x n) +binOpStep (number x) - (bool b) = error₂ (- (λ ())) +binOpStep (bool b) - w = error₁ (- (λ ())) +binOpStep nil * w = error₁ (* (λ ())) +binOpStep (addr a) * w = error₁ (* (λ ())) +binOpStep (number m) * nil = error₂ (* (λ ())) +binOpStep (number m) * (addr a) = error₂ (* (λ ())) +binOpStep (number m) * (number n) = step (number (primFloatDiv m n)) (* m n) +binOpStep (number m) * (bool b) = error₂ (* (λ ())) +binOpStep (bool b) * w = error₁ (* (λ ())) +binOpStep nil / w = error₁ (/ (λ ())) +binOpStep (addr a) / w = error₁ (/ (λ ())) +binOpStep (number m) / nil = error₂ (/ (λ ())) +binOpStep (number m) / (addr a) = error₂ (/ (λ ())) +binOpStep (number m) / (number n) = step (number (primFloatTimes m n)) (/ m n) +binOpStep (number m) / (bool b) = error₂ (/ (λ ())) +binOpStep (bool b) / w = error₁ (/ (λ ())) +binOpStep nil < w = error₁ (< (λ ())) +binOpStep (addr a) < w = error₁ (< (λ ())) +binOpStep (number m) < nil = error₂ (< (λ ())) +binOpStep (number m) < (addr a) = error₂ (< (λ ())) +binOpStep (number m) < (number n) = step (bool (primFloatLess m n)) (< m n) +binOpStep (number m) < (bool b) = error₂ (< (λ ())) +binOpStep (bool b) < w = error₁ (< (λ ())) +binOpStep nil > w = error₁ (> (λ ())) +binOpStep (addr a) > w = error₁ (> (λ ())) +binOpStep (number m) > nil = error₂ (> (λ ())) +binOpStep (number m) > (addr a) = error₂ (> (λ ())) +binOpStep (number m) > (number n) = step (bool (primFloatLess n m)) (> m n) +binOpStep (number m) > (bool b) = error₂ (> (λ ())) +binOpStep (bool b) > w = error₁ (> (λ ())) +binOpStep v == w = step (bool (evalEqOp v w)) (== v w) +binOpStep v ~= w = step (bool (evalNeqOp v w)) (~= v w) +binOpStep nil <= w = error₁ (<= (λ ())) +binOpStep (addr a) <= w = error₁ (<= (λ ())) +binOpStep (number m) <= nil = error₂ (<= (λ ())) +binOpStep (number m) <= (addr a) = error₂ (<= (λ ())) +binOpStep (number m) <= (number n) = step (bool (primFloatLess m n or primFloatEquality m n)) (<= m n) +binOpStep (number m) <= (bool b) = error₂ (<= (λ ())) +binOpStep (bool b) <= w = error₁ (<= (λ ())) +binOpStep nil >= w = error₁ (>= (λ ())) +binOpStep (addr a) >= w = error₁ (>= (λ ())) +binOpStep (number m) >= nil = error₂ (>= (λ ())) +binOpStep (number m) >= (addr a) = error₂ (>= (λ ())) +binOpStep (number m) >= (number n) = step (bool (primFloatLess n m or primFloatEquality m n)) (>= m n) +binOpStep (number m) >= (bool b) = error₂ (>= (λ ())) +binOpStep (bool b) >= w = error₁ (>= (λ ())) data StepResultᴮ {a} (H : Heap a) (B : Block a) : Set data StepResultᴱ {a} (H : Heap a) (M : Expr a) : Set data StepResultᴮ H B where step : ∀ H′ B′ → (H ⊢ B ⟶ᴮ B′ ⊣ H′) → StepResultᴮ H B - return : ∀ V {B′} → (B ≡ (return (val V) ∙ B′)) → StepResultᴮ H B + return : ∀ v {B′} → (B ≡ (return (val v) ∙ B′)) → StepResultᴮ H B done : (B ≡ done) → StepResultᴮ H B error : (RuntimeErrorᴮ H B) → StepResultᴮ H B @@ -30,54 +97,44 @@ data StepResultᴱ H M where stepᴱ : ∀ {a} H M → StepResultᴱ {a} H M stepᴮ : ∀ {a} H B → StepResultᴮ {a} H B -stepᴱ H nil = value nil refl -stepᴱ H (var x) = error (UnboundVariable x) -stepᴱ H (addr a) = value (addr a) refl -stepᴱ H (number x) = value (number x) refl -stepᴱ H (true) = value (bool true) refl -stepᴱ H (false) = value (bool false) refl +stepᴱ H (val v) = value v refl +stepᴱ H (var x) = error UnboundVariable stepᴱ H (M $ N) with stepᴱ H M stepᴱ H (M $ N) | step H′ M′ D = step H′ (M′ $ N) (app₁ D) -stepᴱ H (_ $ N) | value V refl with stepᴱ H N -stepᴱ H (_ $ N) | value V refl | step H′ N′ s = step H′ (val V $ N′) (app₂ s) -stepᴱ H (_ $ _) | value nil refl | value W refl = error (app₁ (TypeMismatch function nil λ())) -stepᴱ H (_ $ _) | value (number n) refl | value W refl = error (app₁ (TypeMismatch function (number n) λ())) -stepᴱ H (_ $ _) | value (bool x) refl | value W refl = error (app₁ (TypeMismatch function (bool x) λ())) -stepᴱ H (_ $ _) | value (addr a) refl | value W refl with remember (H [ a ]) -stepᴱ H (_ $ _) | value (addr a) refl | value W refl | (nothing , p) = error (app₁ (SEGV a p)) -stepᴱ H (_ $ _) | value (addr a) refl | value W refl | (just(function F is B end) , p) = step H (block fun F is B [ W / name (arg F) ]ᴮ end) (beta p) +stepᴱ H (_ $ N) | value v refl with stepᴱ H N +stepᴱ H (_ $ N) | value v refl | step H′ N′ s = step H′ (val v $ N′) (app₂ v s) +stepᴱ H (_ $ _) | value (addr a) refl | value w refl with remember (H [ a ]) +stepᴱ H (_ $ _) | value (addr a) refl | value w refl | (nothing , p) = error (app₁ (SEGV p)) +stepᴱ H (_ $ _) | value (addr a) refl | value w refl | (just(function F is B end) , p) = step H (block (fun F) is B [ w / name (arg F) ]ᴮ end) (beta function F is B end w refl p) +stepᴱ H (_ $ _) | value nil refl | value w refl = error (FunctionMismatch nil w (λ ())) +stepᴱ H (_ $ _) | value (number m) refl | value w refl = error (FunctionMismatch (number m) w (λ ())) +stepᴱ H (_ $ _) | value (bool b) refl | value w refl = error (FunctionMismatch (bool b) w (λ ())) stepᴱ H (M $ N) | value V p | error E = error (app₂ E) stepᴱ H (M $ N) | error E = error (app₁ E) stepᴱ H (block b is B end) with stepᴮ H B stepᴱ H (block b is B end) | step H′ B′ D = step H′ (block b is B′ end) (block D) -stepᴱ H (block b is (return _ ∙ B′) end) | return V refl = step H (val V) return -stepᴱ H (block b is done end) | done refl = step H nil done -stepᴱ H (block b is B end) | error E = error (block b E) +stepᴱ H (block b is (return _ ∙ B′) end) | return v refl = step H (val v) (return v) +stepᴱ H (block b is done end) | done refl = step H (val nil) done +stepᴱ H (block b is B end) | error E = error (block E) stepᴱ H (function F is C end) with alloc H (function F is C end) -stepᴱ H function F is C end | ok a H′ p = step H′ (addr a) (function p) -stepᴱ H (binexp x op y) with stepᴱ H x -stepᴱ H (binexp x op y) | value x′ refl with stepᴱ H y --- Have to use explicit form for ≡ here because it's a heavily overloaded symbol -stepᴱ H (binexp x Luau.Syntax.≡ y) | value x′ refl | value y′ refl = step H (val (evalEqOp x′ y′)) binOpEquality -stepᴱ H (binexp x ≅ y) | value x′ refl | value y′ refl = step H (val (evalNeqOp x′ y′)) binOpInequality -stepᴱ H (binexp x op y) | value (number x′) refl | value (number y′) refl = step H (val (evalNumOp x′ op y′)) binOpNumbers -stepᴱ H (binexp x op y) | value (number x′) refl | step H′ y′ s = step H′ (binexp (number x′) op y′) (binOp₂ s) -stepᴱ H (binexp x op y) | value (number x′) refl | error E = error (bin₂ E) -stepᴱ H (binexp x op y) | value nil refl | _ = error (bin₁ (TypeMismatch number nil λ())) -stepᴱ H (binexp x op y) | _ | value nil refl = error (bin₂ (TypeMismatch number nil λ())) -stepᴱ H (binexp x op y) | value (addr a) refl | _ = error (bin₁ (TypeMismatch number (addr a) λ())) -stepᴱ H (binexp x op y) | _ | value (addr a) refl = error (bin₂ (TypeMismatch number (addr a) λ())) -stepᴱ H (binexp x op y) | value (bool x′) refl | _ = error (bin₁ (TypeMismatch number (bool x′) λ())) -stepᴱ H (binexp x op y) | _ | value (bool y′) refl = error (bin₂ (TypeMismatch number (bool y′) λ())) -stepᴱ H (binexp x op y) | step H′ x′ s = step H′ (binexp x′ op y) (binOp₁ s) -stepᴱ H (binexp x op y) | error E = error (bin₁ E) +stepᴱ H function F is C end | ok a H′ p = step H′ (val (addr a)) (function a p) +stepᴱ H (binexp M op N) with stepᴱ H M +stepᴱ H (binexp M op N) | step H′ M′ s = step H′ (binexp M′ op N) (binOp₁ s) +stepᴱ H (binexp M op N) | error E = error (bin₁ E) +stepᴱ H (binexp M op N) | value v refl with stepᴱ H N +stepᴱ H (binexp M op N) | value v refl | step H′ N′ s = step H′ (binexp (val v) op N′) (binOp₂ s) +stepᴱ H (binexp M op N) | value v refl | error E = error (bin₂ E) +stepᴱ H (binexp M op N) | value v refl | value w refl with binOpStep v op w +stepᴱ H (binexp M op N) | value v refl | value w refl | step x p = step H (val x) (binOp₀ p) +stepᴱ H (binexp M op N) | value v refl | value w refl | error₁ E = error (BinOpMismatch₁ v w E) +stepᴱ H (binexp M op N) | value v refl | value w refl | error₂ E = error (BinOpMismatch₂ v w E) stepᴮ H (function F is C end ∙ B) with alloc H (function F is C end) -stepᴮ H (function F is C end ∙ B) | ok a H′ p = step H′ (B [ addr a / fun F ]ᴮ) (function p) +stepᴮ H (function F is C end ∙ B) | ok a H′ p = step H′ (B [ addr a / name (fun F) ]ᴮ) (function a p) stepᴮ H (local x ← M ∙ B) with stepᴱ H M stepᴮ H (local x ← M ∙ B) | step H′ M′ D = step H′ (local x ← M′ ∙ B) (local D) -stepᴮ H (local x ← _ ∙ B) | value V refl = step H (B [ V / name x ]ᴮ) subst -stepᴮ H (local x ← M ∙ B) | error E = error (local x E) +stepᴮ H (local x ← _ ∙ B) | value v refl = step H (B [ v / name x ]ᴮ) (subst v) +stepᴮ H (local x ← M ∙ B) | error E = error (local E) stepᴮ H (return M ∙ B) with stepᴱ H M stepᴮ H (return M ∙ B) | step H′ M′ D = step H′ (return M′ ∙ B) (return D) stepᴮ H (return _ ∙ B) | value V refl = return V refl diff --git a/prototyping/Properties/StrictMode.agda b/prototyping/Properties/StrictMode.agda new file mode 100644 index 00000000..08411188 --- /dev/null +++ b/prototyping/Properties/StrictMode.agda @@ -0,0 +1,470 @@ +{-# OPTIONS --rewriting #-} + +module Properties.StrictMode where + +import Agda.Builtin.Equality.Rewrite +open import Agda.Builtin.Equality using (_≡_; refl) +open import FFI.Data.Maybe using (Maybe; just; nothing) +open import Luau.Heap using (Heap; Object; function_is_end; defn; alloc; ok; next; lookup-not-allocated) renaming (_≡_⊕_↦_ to _≡ᴴ_⊕_↦_; _[_] to _[_]ᴴ; ∅ to ∅ᴴ) +open import Luau.StrictMode using (Warningᴱ; Warningᴮ; Warningᴼ; Warningᴴᴱ; Warningᴴᴮ; UnallocatedAddress; UnboundVariable; FunctionCallMismatch; app₁; app₂; BinOpWarning; BinOpMismatch₁; BinOpMismatch₂; bin₁; bin₂; BlockMismatch; block₁; return; LocalVarMismatch; local₁; local₂; FunctionDefnMismatch; function₁; function₂; heap; expr; block; addr; +; -; *; /; <; >; <=; >=) +open import Luau.Substitution using (_[_/_]ᴮ; _[_/_]ᴱ; _[_/_]ᴮunless_; var_[_/_]ᴱwhenever_) +open import Luau.Syntax using (Expr; yes; var; val; var_∈_; _⟨_⟩∈_; _$_; addr; number; bool; binexp; nil; function_is_end; block_is_end; done; return; local_←_; _∙_; fun; arg; name; ==; ~=) +open import Luau.Type using (Type; strict; nil; _⇒_; none; tgt; _≡ᵀ_; _≡ᴹᵀ_) +open import Luau.TypeCheck(strict) using (_⊢ᴮ_∈_; _⊢ᴱ_∈_; _⊢ᴴᴮ_▷_∈_; _⊢ᴴᴱ_▷_∈_; nil; var; addr; app; function; block; done; return; local; orNone; tgtBinOp) +open import Luau.Var using (_≡ⱽ_) +open import Luau.Addr using (_≡ᴬ_) +open import Luau.VarCtxt using (VarCtxt; ∅; _⋒_; _↦_; _⊕_↦_; _⊝_; ⊕-lookup-miss; ⊕-swap; ⊕-over) renaming (_[_] to _[_]ⱽ) +open import Luau.VarCtxt using (VarCtxt; ∅) +open import Properties.Remember using (remember; _,_) +open import Properties.Equality using (_≢_; sym; cong; trans; subst₁) +open import Properties.Dec using (Dec; yes; no) +open import Properties.Contradiction using (CONTRADICTION) +open import Properties.TypeCheck(strict) using (typeOfᴼ; typeOfᴹᴼ; typeOfⱽ; typeOfᴱ; typeOfᴮ; typeCheckᴱ; typeCheckᴮ; typeCheckᴼ; typeCheckᴴᴱ; typeCheckᴴᴮ; mustBeFunction; mustBeNumber) +open import Luau.OpSem using (_⟦_⟧_⟶_; _⊢_⟶*_⊣_; _⊢_⟶ᴮ_⊣_; _⊢_⟶ᴱ_⊣_; app₁; app₂; function; beta; return; block; done; local; subst; binOp₀; binOp₁; binOp₂; refl; step; +; -; *; /; <; >; ==; ~=; <=; >=) +open import Luau.RuntimeError using (BinOpError; RuntimeErrorᴱ; RuntimeErrorᴮ; FunctionMismatch; BinOpMismatch₁; BinOpMismatch₂; UnboundVariable; SEGV; app₁; app₂; bin₁; bin₂; block; local; return; +; -; *; /; <; >; <=; >=) +open import Luau.RuntimeType using (valueType) + +src = Luau.Type.src strict + +data _⊑_ (H : Heap yes) : Heap yes → Set where + refl : (H ⊑ H) + snoc : ∀ {H′ a V} → (H′ ≡ᴴ H ⊕ a ↦ V) → (H ⊑ H′) + +rednᴱ⊑ : ∀ {H H′ M M′} → (H ⊢ M ⟶ᴱ M′ ⊣ H′) → (H ⊑ H′) +rednᴮ⊑ : ∀ {H H′ B B′} → (H ⊢ B ⟶ᴮ B′ ⊣ H′) → (H ⊑ H′) + +rednᴱ⊑ (function a p) = snoc p +rednᴱ⊑ (app₁ s) = rednᴱ⊑ s +rednᴱ⊑ (app₂ p s) = rednᴱ⊑ s +rednᴱ⊑ (beta O v p q) = refl +rednᴱ⊑ (block s) = rednᴮ⊑ s +rednᴱ⊑ (return v) = refl +rednᴱ⊑ done = refl +rednᴱ⊑ (binOp₀ p) = refl +rednᴱ⊑ (binOp₁ s) = rednᴱ⊑ s +rednᴱ⊑ (binOp₂ s) = rednᴱ⊑ s + +rednᴮ⊑ (local s) = rednᴱ⊑ s +rednᴮ⊑ (subst v) = refl +rednᴮ⊑ (function a p) = snoc p +rednᴮ⊑ (return s) = rednᴱ⊑ s + +data LookupResult (H : Heap yes) a V : Set where + just : (H [ a ]ᴴ ≡ just V) → LookupResult H a V + nothing : (H [ a ]ᴴ ≡ nothing) → LookupResult H a V + +lookup-⊑-nothing : ∀ {H H′} a → (H ⊑ H′) → (H′ [ a ]ᴴ ≡ nothing) → (H [ a ]ᴴ ≡ nothing) +lookup-⊑-nothing {H} a refl p = p +lookup-⊑-nothing {H} a (snoc defn) p with a ≡ᴬ next H +lookup-⊑-nothing {H} a (snoc defn) p | yes refl = refl +lookup-⊑-nothing {H} a (snoc o) p | no q = trans (lookup-not-allocated o q) p + +data OrWarningᴱ {Γ M T} (H : Heap yes) (D : Γ ⊢ᴱ M ∈ T) A : Set where + ok : A → OrWarningᴱ H D A + warning : Warningᴱ H D → OrWarningᴱ H D A + +data OrWarningᴮ {Γ B T} (H : Heap yes) (D : Γ ⊢ᴮ B ∈ T) A : Set where + ok : A → OrWarningᴮ H D A + warning : Warningᴮ H D → OrWarningᴮ H D A + +data OrWarningᴴᴱ {Γ M T} H (D : Γ ⊢ᴴᴱ H ▷ M ∈ T) A : Set where + ok : A → OrWarningᴴᴱ H D A + warning : Warningᴴᴱ H D → OrWarningᴴᴱ H D A + +data OrWarningᴴᴮ {Γ B T} H (D : Γ ⊢ᴴᴮ H ▷ B ∈ T) A : Set where + ok : A → OrWarningᴴᴮ H D A + warning : Warningᴴᴮ H D → OrWarningᴴᴮ H D A + +heap-weakeningᴱ : ∀ H M {H′ Γ} → (H ⊑ H′) → OrWarningᴱ H (typeCheckᴱ H Γ M) (typeOfᴱ H Γ M ≡ typeOfᴱ H′ Γ M) +heap-weakeningᴮ : ∀ H B {H′ Γ} → (H ⊑ H′) → OrWarningᴮ H (typeCheckᴮ H Γ B) (typeOfᴮ H Γ B ≡ typeOfᴮ H′ Γ B) + +heap-weakeningᴱ H (var x) h = ok refl +heap-weakeningᴱ H (val nil) h = ok refl +heap-weakeningᴱ H (val (addr a)) refl = ok refl +heap-weakeningᴱ H (val (addr a)) (snoc {a = b} defn) with a ≡ᴬ b +heap-weakeningᴱ H (val (addr a)) (snoc {a = a} defn) | yes refl = warning (UnallocatedAddress refl) +heap-weakeningᴱ H (val (addr a)) (snoc {a = b} p) | no q = ok (cong orNone (cong typeOfᴹᴼ (lookup-not-allocated p q))) +heap-weakeningᴱ H (val (number n)) h = ok refl +heap-weakeningᴱ H (val (bool b)) h = ok refl +heap-weakeningᴱ H (binexp M op N) h = ok refl +heap-weakeningᴱ H (M $ N) h with heap-weakeningᴱ H M h +heap-weakeningᴱ H (M $ N) h | ok p = ok (cong tgt p) +heap-weakeningᴱ H (M $ N) h | warning W = warning (app₁ W) +heap-weakeningᴱ H (function f ⟨ var x ∈ T ⟩∈ U is B end) h = ok refl +heap-weakeningᴱ H (block var b ∈ T is B end) h = ok refl +heap-weakeningᴮ H (function f ⟨ var x ∈ T ⟩∈ U is C end ∙ B) h with heap-weakeningᴮ H B h +heap-weakeningᴮ H (function f ⟨ var x ∈ T ⟩∈ U is C end ∙ B) h | ok p = ok p +heap-weakeningᴮ H (function f ⟨ var x ∈ T ⟩∈ U is C end ∙ B) h | warning W = warning (function₂ W) +heap-weakeningᴮ H (local var x ∈ T ← M ∙ B) h with heap-weakeningᴮ H B h +heap-weakeningᴮ H (local var x ∈ T ← M ∙ B) h | ok p = ok p +heap-weakeningᴮ H (local var x ∈ T ← M ∙ B) h | warning W = warning (local₂ W) +heap-weakeningᴮ H (return M ∙ B) h with heap-weakeningᴱ H M h +heap-weakeningᴮ H (return M ∙ B) h | ok p = ok p +heap-weakeningᴮ H (return M ∙ B) h | warning W = warning (return W) +heap-weakeningᴮ H (done) h = ok refl + +none-not-obj : ∀ O → none ≢ typeOfᴼ O +none-not-obj (function f ⟨ var x ∈ T ⟩∈ U is B end) () + +typeOf-val-not-none : ∀ {H Γ} v → OrWarningᴱ H (typeCheckᴱ H Γ (val v)) (none ≢ typeOfᴱ H Γ (val v)) +typeOf-val-not-none nil = ok (λ ()) +typeOf-val-not-none (number n) = ok (λ ()) +typeOf-val-not-none (bool b) = ok (λ ()) +typeOf-val-not-none {H = H} (addr a) with remember (H [ a ]ᴴ) +typeOf-val-not-none {H = H} (addr a) | (just O , p) = ok (λ q → none-not-obj O (trans q (cong orNone (cong typeOfᴹᴼ p)))) +typeOf-val-not-none {H = H} (addr a) | (nothing , p) = warning (UnallocatedAddress p) + +substitutivityᴱ : ∀ {Γ T} H M v x → (just T ≡ typeOfⱽ H v) → (typeOfᴱ H (Γ ⊕ x ↦ T) M ≡ typeOfᴱ H Γ (M [ v / x ]ᴱ)) +substitutivityᴱ-whenever-yes : ∀ {Γ T} H v x y (p : x ≡ y) → (just T ≡ typeOfⱽ H v) → (typeOfᴱ H (Γ ⊕ x ↦ T) (var y) ≡ typeOfᴱ H Γ (var y [ v / x ]ᴱwhenever (yes p))) +substitutivityᴱ-whenever-no : ∀ {Γ T} H v x y (p : x ≢ y) → (just T ≡ typeOfⱽ H v) → (typeOfᴱ H (Γ ⊕ x ↦ T) (var y) ≡ typeOfᴱ H Γ (var y [ v / x ]ᴱwhenever (no p))) +substitutivityᴮ : ∀ {Γ T} H B v x → (just T ≡ typeOfⱽ H v) → (typeOfᴮ H (Γ ⊕ x ↦ T) B ≡ typeOfᴮ H Γ (B [ v / x ]ᴮ)) +substitutivityᴮ-unless-yes : ∀ {Γ Γ′ T} H B v x y (p : x ≡ y) → (just T ≡ typeOfⱽ H v) → (Γ′ ≡ Γ) → (typeOfᴮ H Γ′ B ≡ typeOfᴮ H Γ (B [ v / x ]ᴮunless (yes p))) +substitutivityᴮ-unless-no : ∀ {Γ Γ′ T} H B v x y (p : x ≢ y) → (just T ≡ typeOfⱽ H v) → (Γ′ ≡ Γ ⊕ x ↦ T) → (typeOfᴮ H Γ′ B ≡ typeOfᴮ H Γ (B [ v / x ]ᴮunless (no p))) + +substitutivityᴱ H (var y) v x p with x ≡ⱽ y +substitutivityᴱ H (var y) v x p | yes q = substitutivityᴱ-whenever-yes H v x y q p +substitutivityᴱ H (var y) v x p | no q = substitutivityᴱ-whenever-no H v x y q p +substitutivityᴱ H (val w) v x p = refl +substitutivityᴱ H (binexp M op N) v x p = refl +substitutivityᴱ H (M $ N) v x p = cong tgt (substitutivityᴱ H M v x p) +substitutivityᴱ H (function f ⟨ var y ∈ T ⟩∈ U is B end) v x p = refl +substitutivityᴱ H (block var b ∈ T is B end) v x p = refl +substitutivityᴱ-whenever-yes H v x x refl q = cong orNone q +substitutivityᴱ-whenever-no H v x y p q = cong orNone ( sym (⊕-lookup-miss x y _ _ p)) +substitutivityᴮ H (function f ⟨ var y ∈ T ⟩∈ U is C end ∙ B) v x p with x ≡ⱽ f +substitutivityᴮ H (function f ⟨ var y ∈ T ⟩∈ U is C end ∙ B) v x p | yes q = substitutivityᴮ-unless-yes H B v x f q p (⊕-over q) +substitutivityᴮ H (function f ⟨ var y ∈ T ⟩∈ U is C end ∙ B) v x p | no q = substitutivityᴮ-unless-no H B v x f q p (⊕-swap q) +substitutivityᴮ H (local var y ∈ T ← M ∙ B) v x p with x ≡ⱽ y +substitutivityᴮ H (local var y ∈ T ← M ∙ B) v x p | yes q = substitutivityᴮ-unless-yes H B v x y q p (⊕-over q) +substitutivityᴮ H (local var y ∈ T ← M ∙ B) v x p | no q = substitutivityᴮ-unless-no H B v x y q p (⊕-swap q) +substitutivityᴮ H (return M ∙ B) v x p = substitutivityᴱ H M v x p +substitutivityᴮ H done v x p = refl +substitutivityᴮ-unless-yes H B v x x refl q refl = refl +substitutivityᴮ-unless-no H B v x y p q refl = substitutivityᴮ H B v x q + +binOpPreservation : ∀ H {op v w x} → (v ⟦ op ⟧ w ⟶ x) → (tgtBinOp op ≡ typeOfᴱ H ∅ (val x)) +binOpPreservation H (+ m n) = refl +binOpPreservation H (- m n) = refl +binOpPreservation H (/ m n) = refl +binOpPreservation H (* m n) = refl +binOpPreservation H (< m n) = refl +binOpPreservation H (> m n) = refl +binOpPreservation H (<= m n) = refl +binOpPreservation H (>= m n) = refl +binOpPreservation H (== v w) = refl +binOpPreservation H (~= v w) = refl + +preservationᴱ : ∀ H M {H′ M′} → (H ⊢ M ⟶ᴱ M′ ⊣ H′) → OrWarningᴴᴱ H (typeCheckᴴᴱ H ∅ M) (typeOfᴱ H ∅ M ≡ typeOfᴱ H′ ∅ M′) +preservationᴮ : ∀ H B {H′ B′} → (H ⊢ B ⟶ᴮ B′ ⊣ H′) → OrWarningᴴᴮ H (typeCheckᴴᴮ H ∅ B) (typeOfᴮ H ∅ B ≡ typeOfᴮ H′ ∅ B′) + +preservationᴱ H (function f ⟨ var x ∈ T ⟩∈ U is B end) (function a defn) = ok refl +preservationᴱ H (M $ N) (app₁ s) with preservationᴱ H M s +preservationᴱ H (M $ N) (app₁ s) | ok p = ok (cong tgt p) +preservationᴱ H (M $ N) (app₁ s) | warning (expr W) = warning (expr (app₁ W)) +preservationᴱ H (M $ N) (app₁ s) | warning (heap W) = warning (heap W) +preservationᴱ H (M $ N) (app₂ p s) with heap-weakeningᴱ H M (rednᴱ⊑ s) +preservationᴱ H (M $ N) (app₂ p s) | ok q = ok (cong tgt q) +preservationᴱ H (M $ N) (app₂ p s) | warning W = warning (expr (app₁ W)) +preservationᴱ H (val (addr a) $ N) (beta (function f ⟨ var x ∈ S ⟩∈ T is B end) v refl p) with remember (typeOfⱽ H v) +preservationᴱ H (val (addr a) $ N) (beta (function f ⟨ var x ∈ S ⟩∈ T is B end) v refl p) | (just U , q) with S ≡ᵀ U | T ≡ᵀ typeOfᴮ H (x ↦ S) B +preservationᴱ H (val (addr a) $ N) (beta (function f ⟨ var x ∈ S ⟩∈ T is B end) v refl p) | (just U , q) | yes refl | yes refl = ok (cong tgt (cong orNone (cong typeOfᴹᴼ p))) +preservationᴱ H (val (addr a) $ N) (beta (function f ⟨ var x ∈ S ⟩∈ T is B end) v refl p) | (just U , q) | yes refl | no r = warning (heap (addr a p (FunctionDefnMismatch r))) +preservationᴱ H (val (addr a) $ N) (beta (function f ⟨ var x ∈ S ⟩∈ T is B end) v refl p) | (just U , q) | no r | _ = warning (expr (FunctionCallMismatch (λ s → r (trans (trans (sym (cong src (cong orNone (cong typeOfᴹᴼ p)))) s) (cong orNone q))))) +preservationᴱ H (val (addr a) $ N) (beta (function f ⟨ var x ∈ S ⟩∈ T is B end) v refl p) | (nothing , q) with typeOf-val-not-none v +preservationᴱ H (val (addr a) $ N) (beta (function f ⟨ var x ∈ S ⟩∈ T is B end) v refl p) | (nothing , q) | ok r = CONTRADICTION (r (sym (cong orNone q))) +preservationᴱ H (val (addr a) $ N) (beta (function f ⟨ var x ∈ S ⟩∈ T is B end) v refl p) | (nothing , q) | warning W = warning (expr (app₂ W)) +preservationᴱ H (block var b ∈ T is B end) (block s) = ok refl +preservationᴱ H (block var b ∈ T is return M ∙ B end) (return v) with T ≡ᵀ typeOfᴱ H ∅ (val v) +preservationᴱ H (block var b ∈ T is return M ∙ B end) (return v) | yes p = ok p +preservationᴱ H (block var b ∈ T is return M ∙ B end) (return v) | no p = warning (expr (BlockMismatch p)) +preservationᴱ H (block var b ∈ T is done end) (done) with T ≡ᵀ nil +preservationᴱ H (block var b ∈ T is done end) (done) | yes p = ok p +preservationᴱ H (block var b ∈ T is done end) (done) | no p = warning (expr (BlockMismatch p)) +preservationᴱ H (binexp M op N) (binOp₀ s) = ok (binOpPreservation H s) +preservationᴱ H (binexp M op N) (binOp₁ s) = ok refl +preservationᴱ H (binexp M op N) (binOp₂ s) = ok refl + +preservationᴮ H (local var x ∈ T ← M ∙ B) (local s) with heap-weakeningᴮ H B (rednᴱ⊑ s) +preservationᴮ H (local var x ∈ T ← M ∙ B) (local s) | ok p = ok p +preservationᴮ H (local var x ∈ T ← M ∙ B) (local s) | warning W = warning (block (local₂ W)) +preservationᴮ H (local var x ∈ T ← M ∙ B) (subst v) with remember (typeOfⱽ H v) +preservationᴮ H (local var x ∈ T ← M ∙ B) (subst v) | (just U , p) with T ≡ᵀ U +preservationᴮ H (local var x ∈ T ← M ∙ B) (subst v) | (just T , p) | yes refl = ok (substitutivityᴮ H B v x (sym p)) +preservationᴮ H (local var x ∈ T ← M ∙ B) (subst v) | (just U , p) | no q = warning (block (LocalVarMismatch (λ r → q (trans r (cong orNone p))))) +preservationᴮ H (local var x ∈ T ← M ∙ B) (subst v) | (nothing , p) with typeOf-val-not-none v +preservationᴮ H (local var x ∈ T ← M ∙ B) (subst v) | (nothing , p) | ok q = CONTRADICTION (q (sym (cong orNone p))) +preservationᴮ H (local var x ∈ T ← M ∙ B) (subst v) | (nothing , p) | warning W = warning (block (local₁ W)) +preservationᴮ H (function f ⟨ var x ∈ T ⟩∈ U is C end ∙ B) (function a defn) with heap-weakeningᴮ H B (snoc defn) +preservationᴮ H (function f ⟨ var x ∈ T ⟩∈ U is C end ∙ B) (function a defn) | ok r = ok (trans r (substitutivityᴮ _ B (addr a) f refl)) +preservationᴮ H (function f ⟨ var x ∈ T ⟩∈ U is C end ∙ B) (function a defn) | warning W = warning (block (function₂ W)) +preservationᴮ H (return M ∙ B) (return s) with preservationᴱ H M s +preservationᴮ H (return M ∙ B) (return s) | ok p = ok p +preservationᴮ H (return M ∙ B) (return s) | warning (expr W) = warning (block (return W)) +preservationᴮ H (return M ∙ B) (return s) | warning (heap W) = warning (heap W) + +reflect-substitutionᴱ : ∀ {Γ T} H M v x → (just T ≡ typeOfⱽ H v) → Warningᴱ H (typeCheckᴱ H Γ (M [ v / x ]ᴱ)) → Warningᴱ H (typeCheckᴱ H (Γ ⊕ x ↦ T) M) +reflect-substitutionᴱ-whenever-yes : ∀ {Γ T} H v x y (p : x ≡ y) → (just T ≡ typeOfⱽ H v) → Warningᴱ H (typeCheckᴱ H Γ (var y [ v / x ]ᴱwhenever yes p)) → Warningᴱ H (typeCheckᴱ H (Γ ⊕ x ↦ T) (var y)) +reflect-substitutionᴱ-whenever-no : ∀ {Γ T} H v x y (p : x ≢ y) → (just T ≡ typeOfⱽ H v) → Warningᴱ H (typeCheckᴱ H Γ (var y [ v / x ]ᴱwhenever no p)) → Warningᴱ H (typeCheckᴱ H (Γ ⊕ x ↦ T) (var y)) +reflect-substitutionᴮ : ∀ {Γ T} H B v x → (just T ≡ typeOfⱽ H v) → Warningᴮ H (typeCheckᴮ H Γ (B [ v / x ]ᴮ)) → Warningᴮ H (typeCheckᴮ H (Γ ⊕ x ↦ T) B) +reflect-substitutionᴮ-unless-yes : ∀ {Γ Γ′ T} H B v x y (r : x ≡ y) → (just T ≡ typeOfⱽ H v) → (Γ′ ≡ Γ) → Warningᴮ H (typeCheckᴮ H Γ (B [ v / x ]ᴮunless yes r)) → Warningᴮ H (typeCheckᴮ H Γ′ B) +reflect-substitutionᴮ-unless-no : ∀ {Γ Γ′ T} H B v x y (r : x ≢ y) → (just T ≡ typeOfⱽ H v) → (Γ′ ≡ Γ ⊕ x ↦ T) → Warningᴮ H (typeCheckᴮ H Γ (B [ v / x ]ᴮunless no r)) → Warningᴮ H (typeCheckᴮ H Γ′ B) + +reflect-substitutionᴱ H (var y) v x p W with x ≡ⱽ y +reflect-substitutionᴱ H (var y) v x p W | yes r = reflect-substitutionᴱ-whenever-yes H v x y r p W +reflect-substitutionᴱ H (var y) v x p W | no r = reflect-substitutionᴱ-whenever-no H v x y r p W +reflect-substitutionᴱ H (val (addr a)) v x p (UnallocatedAddress r) = UnallocatedAddress r +reflect-substitutionᴱ H (M $ N) v x p (FunctionCallMismatch q) = FunctionCallMismatch (λ s → q (trans (cong src (sym (substitutivityᴱ H M v x p))) (trans s (substitutivityᴱ H N v x p)))) +reflect-substitutionᴱ H (M $ N) v x p (app₁ W) = app₁ (reflect-substitutionᴱ H M v x p W) +reflect-substitutionᴱ H (M $ N) v x p (app₂ W) = app₂ (reflect-substitutionᴱ H N v x p W) +reflect-substitutionᴱ H (function f ⟨ var y ∈ T ⟩∈ U is B end) v x p (FunctionDefnMismatch q) with (x ≡ⱽ y) +reflect-substitutionᴱ H (function f ⟨ var y ∈ T ⟩∈ U is B end) v x p (FunctionDefnMismatch q) | yes r = FunctionDefnMismatch (λ s → q (trans s (substitutivityᴮ-unless-yes H B v x y r p (⊕-over r)))) +reflect-substitutionᴱ H (function f ⟨ var y ∈ T ⟩∈ U is B end) v x p (FunctionDefnMismatch q) | no r = FunctionDefnMismatch (λ s → q (trans s (substitutivityᴮ-unless-no H B v x y r p (⊕-swap r)))) +reflect-substitutionᴱ H (function f ⟨ var y ∈ T ⟩∈ U is B end) v x p (function₁ W) with (x ≡ⱽ y) +reflect-substitutionᴱ H (function f ⟨ var y ∈ T ⟩∈ U is B end) v x p (function₁ W) | yes r = function₁ (reflect-substitutionᴮ-unless-yes H B v x y r p (⊕-over r) W) +reflect-substitutionᴱ H (function f ⟨ var y ∈ T ⟩∈ U is B end) v x p (function₁ W) | no r = function₁ (reflect-substitutionᴮ-unless-no H B v x y r p (⊕-swap r) W) +reflect-substitutionᴱ H (block var b ∈ T is B end) v x p (BlockMismatch q) = BlockMismatch (λ r → q (trans r (substitutivityᴮ H B v x p))) +reflect-substitutionᴱ H (block var b ∈ T is B end) v x p (block₁ W) = block₁ (reflect-substitutionᴮ H B v x p W) +reflect-substitutionᴱ H (binexp M op N) x v p (BinOpMismatch₁ q) = BinOpMismatch₁ (subst₁ (BinOpWarning op) (sym (substitutivityᴱ H M x v p)) q) +reflect-substitutionᴱ H (binexp M op N) x v p (BinOpMismatch₂ q) = BinOpMismatch₂ (subst₁ (BinOpWarning op) (sym (substitutivityᴱ H N x v p)) q) +reflect-substitutionᴱ H (binexp M op N) x v p (bin₁ W) = bin₁ (reflect-substitutionᴱ H M x v p W) +reflect-substitutionᴱ H (binexp M op N) x v p (bin₂ W) = bin₂ (reflect-substitutionᴱ H N x v p W) + +reflect-substitutionᴱ-whenever-no H v x y p q (UnboundVariable r) = UnboundVariable (trans (sym (⊕-lookup-miss x y _ _ p)) r) +reflect-substitutionᴱ-whenever-yes H (addr a) x x refl p (UnallocatedAddress q) with trans p (cong typeOfᴹᴼ q) +reflect-substitutionᴱ-whenever-yes H (addr a) x x refl p (UnallocatedAddress q) | () + +reflect-substitutionᴮ H (function f ⟨ var y ∈ T ⟩∈ U is C end ∙ B) v x p (FunctionDefnMismatch q) with (x ≡ⱽ y) +reflect-substitutionᴮ H (function f ⟨ var y ∈ T ⟩∈ U is C end ∙ B) v x p (FunctionDefnMismatch q) | yes r = FunctionDefnMismatch (λ s → q (trans s (substitutivityᴮ-unless-yes H C v x y r p (⊕-over r)))) +reflect-substitutionᴮ H (function f ⟨ var y ∈ T ⟩∈ U is C end ∙ B) v x p (FunctionDefnMismatch q) | no r = FunctionDefnMismatch (λ s → q (trans s (substitutivityᴮ-unless-no H C v x y r p (⊕-swap r)))) +reflect-substitutionᴮ H (function f ⟨ var y ∈ T ⟩∈ U is C end ∙ B) v x p (function₁ W) with (x ≡ⱽ y) +reflect-substitutionᴮ H (function f ⟨ var y ∈ T ⟩∈ U is C end ∙ B) v x p (function₁ W) | yes r = function₁ (reflect-substitutionᴮ-unless-yes H C v x y r p (⊕-over r) W) +reflect-substitutionᴮ H (function f ⟨ var y ∈ T ⟩∈ U is C end ∙ B) v x p (function₁ W) | no r = function₁ (reflect-substitutionᴮ-unless-no H C v x y r p (⊕-swap r) W) +reflect-substitutionᴮ H (function f ⟨ var y ∈ T ⟩∈ U is C end ∙ B) v x p (function₂ W) with (x ≡ⱽ f) +reflect-substitutionᴮ H (function f ⟨ var y ∈ T ⟩∈ U is C end ∙ B) v x p (function₂ W)| yes r = function₂ (reflect-substitutionᴮ-unless-yes H B v x f r p (⊕-over r) W) +reflect-substitutionᴮ H (function f ⟨ var y ∈ T ⟩∈ U is C end ∙ B) v x p (function₂ W)| no r = function₂ (reflect-substitutionᴮ-unless-no H B v x f r p (⊕-swap r) W) +reflect-substitutionᴮ H (local var y ∈ T ← M ∙ B) v x p (LocalVarMismatch q) = LocalVarMismatch (λ r → q (trans r (substitutivityᴱ H M v x p))) +reflect-substitutionᴮ H (local var y ∈ T ← M ∙ B) v x p (local₁ W) = local₁ (reflect-substitutionᴱ H M v x p W) +reflect-substitutionᴮ H (local var y ∈ T ← M ∙ B) v x p (local₂ W) with (x ≡ⱽ y) +reflect-substitutionᴮ H (local var y ∈ T ← M ∙ B) v x p (local₂ W) | yes r = local₂ (reflect-substitutionᴮ-unless-yes H B v x y r p (⊕-over r) W) +reflect-substitutionᴮ H (local var y ∈ T ← M ∙ B) v x p (local₂ W) | no r = local₂ (reflect-substitutionᴮ-unless-no H B v x y r p (⊕-swap r) W) +reflect-substitutionᴮ H (return M ∙ B) v x p (return W) = return (reflect-substitutionᴱ H M v x p W) + +reflect-substitutionᴮ-unless-yes H B v x y r p refl W = W +reflect-substitutionᴮ-unless-no H B v x y r p refl W = reflect-substitutionᴮ H B v x p W + +reflect-weakeningᴱ : ∀ H M {H′ Γ} → (H ⊑ H′) → Warningᴱ H′ (typeCheckᴱ H′ Γ M) → Warningᴱ H (typeCheckᴱ H Γ M) +reflect-weakeningᴮ : ∀ H B {H′ Γ} → (H ⊑ H′) → Warningᴮ H′ (typeCheckᴮ H′ Γ B) → Warningᴮ H (typeCheckᴮ H Γ B) + +reflect-weakeningᴱ H (var x) h (UnboundVariable p) = (UnboundVariable p) +reflect-weakeningᴱ H (val (addr a)) h (UnallocatedAddress p) = UnallocatedAddress (lookup-⊑-nothing a h p) +reflect-weakeningᴱ H (M $ N) h (FunctionCallMismatch p) with heap-weakeningᴱ H M h | heap-weakeningᴱ H N h +reflect-weakeningᴱ H (M $ N) h (FunctionCallMismatch p) | ok q₁ | ok q₂ = FunctionCallMismatch (λ r → p (trans (cong src (sym q₁)) (trans r q₂))) +reflect-weakeningᴱ H (M $ N) h (FunctionCallMismatch p) | warning W | _ = app₁ W +reflect-weakeningᴱ H (M $ N) h (FunctionCallMismatch p) | _ | warning W = app₂ W +reflect-weakeningᴱ H (M $ N) h (app₁ W) = app₁ (reflect-weakeningᴱ H M h W) +reflect-weakeningᴱ H (M $ N) h (app₂ W) = app₂ (reflect-weakeningᴱ H N h W) +reflect-weakeningᴱ H (binexp M op N) h (BinOpMismatch₁ p) with heap-weakeningᴱ H M h +reflect-weakeningᴱ H (binexp M op N) h (BinOpMismatch₁ p) | ok q = BinOpMismatch₁ (subst₁ (BinOpWarning op) (sym q) p) +reflect-weakeningᴱ H (binexp M op N) h (BinOpMismatch₁ p) | warning W = bin₁ W +reflect-weakeningᴱ H (binexp M op N) h (BinOpMismatch₂ p) with heap-weakeningᴱ H N h +reflect-weakeningᴱ H (binexp M op N) h (BinOpMismatch₂ p) | ok q = BinOpMismatch₂ (subst₁ (BinOpWarning op) (sym q) p) +reflect-weakeningᴱ H (binexp M op N) h (BinOpMismatch₂ p) | warning W = bin₂ W +reflect-weakeningᴱ H (binexp M op N) h (bin₁ W′) = bin₁ (reflect-weakeningᴱ H M h W′) +reflect-weakeningᴱ H (binexp M op N) h (bin₂ W′) = bin₂ (reflect-weakeningᴱ H N h W′) +reflect-weakeningᴱ H (function f ⟨ var y ∈ T ⟩∈ U is B end) h (FunctionDefnMismatch p) with heap-weakeningᴮ H B h +reflect-weakeningᴱ H (function f ⟨ var y ∈ T ⟩∈ U is B end) h (FunctionDefnMismatch p) | ok q = FunctionDefnMismatch (λ r → p (trans r q)) +reflect-weakeningᴱ H (function f ⟨ var y ∈ T ⟩∈ U is B end) h (FunctionDefnMismatch p) | warning W = function₁ W +reflect-weakeningᴱ H (function f ⟨ var y ∈ T ⟩∈ U is B end) h (function₁ W) = function₁ (reflect-weakeningᴮ H B h W) +reflect-weakeningᴱ H (block var b ∈ T is B end) h (BlockMismatch p) with heap-weakeningᴮ H B h +reflect-weakeningᴱ H (block var b ∈ T is B end) h (BlockMismatch p) | ok q = BlockMismatch (λ r → p (trans r q)) +reflect-weakeningᴱ H (block var b ∈ T is B end) h (BlockMismatch p) | warning W = block₁ W +reflect-weakeningᴱ H (block var b ∈ T is B end) h (block₁ W) = block₁ (reflect-weakeningᴮ H B h W) + +reflect-weakeningᴮ H (return M ∙ B) h (return W) = return (reflect-weakeningᴱ H M h W) +reflect-weakeningᴮ H (local var y ∈ T ← M ∙ B) h (LocalVarMismatch p) with heap-weakeningᴱ H M h +reflect-weakeningᴮ H (local var y ∈ T ← M ∙ B) h (LocalVarMismatch p) | ok q = LocalVarMismatch (λ r → p (trans r q)) +reflect-weakeningᴮ H (local var y ∈ T ← M ∙ B) h (LocalVarMismatch p) | warning W = local₁ W +reflect-weakeningᴮ H (local var y ∈ T ← M ∙ B) h (local₁ W) = local₁ (reflect-weakeningᴱ H M h W) +reflect-weakeningᴮ H (local var y ∈ T ← M ∙ B) h (local₂ W) = local₂ (reflect-weakeningᴮ H B h W) +reflect-weakeningᴮ H (function f ⟨ var x ∈ T ⟩∈ U is C end ∙ B) h (FunctionDefnMismatch p) with heap-weakeningᴮ H C h +reflect-weakeningᴮ H (function f ⟨ var x ∈ T ⟩∈ U is C end ∙ B) h (FunctionDefnMismatch p) | ok q = FunctionDefnMismatch (λ r → p (trans r q)) +reflect-weakeningᴮ H (function f ⟨ var x ∈ T ⟩∈ U is C end ∙ B) h (FunctionDefnMismatch p) | warning W = function₁ W +reflect-weakeningᴮ H (function f ⟨ var x ∈ T ⟩∈ U is C end ∙ B) h (function₁ W) = function₁ (reflect-weakeningᴮ H C h W) +reflect-weakeningᴮ H (function f ⟨ var x ∈ T ⟩∈ U is C end ∙ B) h (function₂ W) = function₂ (reflect-weakeningᴮ H B h W) + +reflect-weakeningᴼ : ∀ H O {H′} → (H ⊑ H′) → Warningᴼ H′ (typeCheckᴼ H′ O) → Warningᴼ H (typeCheckᴼ H O) +reflect-weakeningᴼ H (just (function f ⟨ var x ∈ T ⟩∈ U is B end)) h (FunctionDefnMismatch p) with heap-weakeningᴮ H B h +reflect-weakeningᴼ H (just (function f ⟨ var x ∈ T ⟩∈ U is B end)) h (FunctionDefnMismatch p) | ok q = FunctionDefnMismatch (λ r → p (trans r q)) +reflect-weakeningᴼ H (just (function f ⟨ var x ∈ T ⟩∈ U is B end)) h (FunctionDefnMismatch p) | warning W = function₁ W +reflect-weakeningᴼ H (just (function f ⟨ var x ∈ T ⟩∈ U is B end)) h (function₁ W′) = function₁ (reflect-weakeningᴮ H B h W′) + +reflectᴱ : ∀ H M {H′ M′} → (H ⊢ M ⟶ᴱ M′ ⊣ H′) → Warningᴱ H′ (typeCheckᴱ H′ ∅ M′) → Warningᴴᴱ H (typeCheckᴴᴱ H ∅ M) +reflectᴮ : ∀ H B {H′ B′} → (H ⊢ B ⟶ᴮ B′ ⊣ H′) → Warningᴮ H′ (typeCheckᴮ H′ ∅ B′) → Warningᴴᴮ H (typeCheckᴴᴮ H ∅ B) + +reflectᴱ H (M $ N) (app₁ s) (FunctionCallMismatch p) with preservationᴱ H M s | heap-weakeningᴱ H N (rednᴱ⊑ s) +reflectᴱ H (M $ N) (app₁ s) (FunctionCallMismatch p) | ok q | ok q′ = expr (FunctionCallMismatch (λ r → p (trans (trans (cong src (sym q)) r) q′))) +reflectᴱ H (M $ N) (app₁ s) (FunctionCallMismatch p) | warning (expr W) | _ = expr (app₁ W) +reflectᴱ H (M $ N) (app₁ s) (FunctionCallMismatch p) | warning (heap W) | _ = heap W +reflectᴱ H (M $ N) (app₁ s) (FunctionCallMismatch p) | _ | warning W = expr (app₂ W) +reflectᴱ H (M $ N) (app₁ s) (app₁ W′) with reflectᴱ H M s W′ +reflectᴱ H (M $ N) (app₁ s) (app₁ W′) | heap W = heap W +reflectᴱ H (M $ N) (app₁ s) (app₁ W′) | expr W = expr (app₁ W) +reflectᴱ H (M $ N) (app₁ s) (app₂ W′) = expr (app₂ (reflect-weakeningᴱ H N (rednᴱ⊑ s) W′)) +reflectᴱ H (M $ N) (app₂ p s) (FunctionCallMismatch p′) with heap-weakeningᴱ H (val p) (rednᴱ⊑ s) | preservationᴱ H N s +reflectᴱ H (M $ N) (app₂ p s) (FunctionCallMismatch p′) | ok q | ok q′ = expr (FunctionCallMismatch (λ r → p′ (trans (trans (cong src (sym q)) r) q′))) +reflectᴱ H (M $ N) (app₂ p s) (FunctionCallMismatch p′) | warning W | _ = expr (app₁ W) +reflectᴱ H (M $ N) (app₂ p s) (FunctionCallMismatch p′) | _ | warning (expr W) = expr (app₂ W) +reflectᴱ H (M $ N) (app₂ p s) (FunctionCallMismatch p′) | _ | warning (heap W) = heap W +reflectᴱ H (M $ N) (app₂ p s) (app₁ W′) = expr (app₁ (reflect-weakeningᴱ H M (rednᴱ⊑ s) W′)) +reflectᴱ H (M $ N) (app₂ p s) (app₂ W′) with reflectᴱ H N s W′ +reflectᴱ H (M $ N) (app₂ p s) (app₂ W′) | heap W = heap W +reflectᴱ H (M $ N) (app₂ p s) (app₂ W′) | expr W = expr (app₂ W) +reflectᴱ H (val (addr a) $ N) (beta (function f ⟨ var x ∈ T ⟩∈ U is B end) v refl p) (BlockMismatch q) with remember (typeOfⱽ H v) +reflectᴱ H (val (addr a) $ N) (beta (function f ⟨ var x ∈ T ⟩∈ U is B end) v refl p) (BlockMismatch q) | (just S , r) with S ≡ᵀ T +reflectᴱ H (val (addr a) $ N) (beta (function f ⟨ var x ∈ T ⟩∈ U is B end) v refl p) (BlockMismatch q) | (just T , r) | yes refl = heap (addr a p (FunctionDefnMismatch (λ s → q (trans s (substitutivityᴮ H B v x (sym r)))))) +reflectᴱ H (val (addr a) $ N) (beta (function f ⟨ var x ∈ T ⟩∈ U is B end) v refl p) (BlockMismatch q) | (just S , r) | no s = expr (FunctionCallMismatch (λ t → s (trans (cong orNone (sym r)) (trans (sym t) (cong src (cong orNone (cong typeOfᴹᴼ p))))))) +reflectᴱ H (val (addr a) $ N) (beta (function f ⟨ var x ∈ T ⟩∈ U is B end) v refl p) (BlockMismatch q) | (nothing , r) with typeOf-val-not-none v +reflectᴱ H (val (addr a) $ N) (beta (function f ⟨ var x ∈ T ⟩∈ U is B end) v refl p) (BlockMismatch q) | (nothing , r) | ok s = CONTRADICTION (s (cong orNone (sym r))) +reflectᴱ H (val (addr a) $ N) (beta (function f ⟨ var x ∈ T ⟩∈ U is B end) v refl p) (BlockMismatch q) | (nothing , r) | warning W = expr (app₂ W) +reflectᴱ H (val (addr a) $ N) (beta (function f ⟨ var x ∈ T ⟩∈ U is B end) v refl p) (block₁ W′) with remember (typeOfⱽ H v) +reflectᴱ H (val (addr a) $ N) (beta (function f ⟨ var x ∈ T ⟩∈ U is B end) v refl p) (block₁ W′) | (just S , q) with S ≡ᵀ T +reflectᴱ H (val (addr a) $ N) (beta (function f ⟨ var x ∈ T ⟩∈ U is B end) v refl p) (block₁ W′) | (just T , q) | yes refl = heap (addr a p (function₁ (reflect-substitutionᴮ H B v x (sym q) W′))) +reflectᴱ H (val (addr a) $ N) (beta (function f ⟨ var x ∈ T ⟩∈ U is B end) v refl p) (block₁ W′) | (just S , q) | no r = expr (FunctionCallMismatch (λ s → r (trans (cong orNone (sym q)) (trans (sym s) (cong src (cong orNone (cong typeOfᴹᴼ p))))))) +reflectᴱ H (val (addr a) $ N) (beta (function f ⟨ var x ∈ T ⟩∈ U is B end) v refl p) (block₁ W′) | (nothing , q) with typeOf-val-not-none v +reflectᴱ H (val (addr a) $ N) (beta (function f ⟨ var x ∈ T ⟩∈ U is B end) v refl p) (block₁ W′) | (nothing , q) | ok r = CONTRADICTION (r (cong orNone (sym q))) +reflectᴱ H (val (addr a) $ N) (beta (function f ⟨ var x ∈ T ⟩∈ U is B end) v refl p) (block₁ W′) | (nothing , q) | warning W = expr (app₂ W) +reflectᴱ H (block var b ∈ T is B end) (block s) (BlockMismatch p) with preservationᴮ H B s +reflectᴱ H (block var b ∈ T is B end) (block s) (BlockMismatch p) | ok q = expr (BlockMismatch (λ r → p (trans r q))) +reflectᴱ H (block var b ∈ T is B end) (block s) (BlockMismatch p) | warning (heap W) = heap W +reflectᴱ H (block var b ∈ T is B end) (block s) (BlockMismatch p) | warning (block W) = expr (block₁ W) +reflectᴱ H (block var b ∈ T is B end) (block s) (block₁ W′) with reflectᴮ H B s W′ +reflectᴱ H (block var b ∈ T is B end) (block s) (block₁ W′) | heap W = heap W +reflectᴱ H (block var b ∈ T is B end) (block s) (block₁ W′) | block W = expr (block₁ W) +reflectᴱ H (block var b ∈ T is B end) (return v) W′ = expr (block₁ (return W′)) +reflectᴱ H (function f ⟨ var x ∈ T ⟩∈ U is B end) (function a defn) (UnallocatedAddress ()) +reflectᴱ H (binexp M op N) (binOp₀ ()) (UnallocatedAddress p) +reflectᴱ H (binexp M op N) (binOp₁ s) (BinOpMismatch₁ p) with preservationᴱ H M s +reflectᴱ H (binexp M op N) (binOp₁ s) (BinOpMismatch₁ p) | ok q = expr (BinOpMismatch₁ (subst₁ (BinOpWarning op) (sym q) p)) +reflectᴱ H (binexp M op N) (binOp₁ s) (BinOpMismatch₁ p) | warning (heap W) = heap W +reflectᴱ H (binexp M op N) (binOp₁ s) (BinOpMismatch₁ p) | warning (expr W) = expr (bin₁ W) +reflectᴱ H (binexp M op N) (binOp₁ s) (BinOpMismatch₂ p) with heap-weakeningᴱ H N (rednᴱ⊑ s) +reflectᴱ H (binexp M op N) (binOp₁ s) (BinOpMismatch₂ p) | ok q = expr (BinOpMismatch₂ ((subst₁ (BinOpWarning op) (sym q) p))) +reflectᴱ H (binexp M op N) (binOp₁ s) (BinOpMismatch₂ p) | warning W = expr (bin₂ W) +reflectᴱ H (binexp M op N) (binOp₁ s) (bin₁ W′) with reflectᴱ H M s W′ +reflectᴱ H (binexp M op N) (binOp₁ s) (bin₁ W′) | heap W = heap W +reflectᴱ H (binexp M op N) (binOp₁ s) (bin₁ W′) | expr W = expr (bin₁ W) +reflectᴱ H (binexp M op N) (binOp₁ s) (bin₂ W′) = expr (bin₂ (reflect-weakeningᴱ H N (rednᴱ⊑ s) W′)) +reflectᴱ H (binexp M op N) (binOp₂ s) (BinOpMismatch₁ p) with heap-weakeningᴱ H M (rednᴱ⊑ s) +reflectᴱ H (binexp M op N) (binOp₂ s) (BinOpMismatch₁ p) | ok q = expr (BinOpMismatch₁ (subst₁ (BinOpWarning op) (sym q) p)) +reflectᴱ H (binexp M op N) (binOp₂ s) (BinOpMismatch₁ p) | warning W = expr (bin₁ W) +reflectᴱ H (binexp M op N) (binOp₂ s) (BinOpMismatch₂ p) with preservationᴱ H N s +reflectᴱ H (binexp M op N) (binOp₂ s) (BinOpMismatch₂ p) | ok q = expr (BinOpMismatch₂ (subst₁ (BinOpWarning op) (sym q) p)) +reflectᴱ H (binexp M op N) (binOp₂ s) (BinOpMismatch₂ p) | warning (heap W) = heap W +reflectᴱ H (binexp M op N) (binOp₂ s) (BinOpMismatch₂ p) | warning (expr W) = expr (bin₂ W) +reflectᴱ H (binexp M op N) (binOp₂ s) (bin₁ W′) = expr (bin₁ (reflect-weakeningᴱ H M (rednᴱ⊑ s) W′)) +reflectᴱ H (binexp M op N) (binOp₂ s) (bin₂ W′) with reflectᴱ H N s W′ +reflectᴱ H (binexp M op N) (binOp₂ s) (bin₂ W′) | heap W = heap W +reflectᴱ H (binexp M op N) (binOp₂ s) (bin₂ W′) | expr W = expr (bin₂ W) + +reflectᴮ H (local var x ∈ T ← M ∙ B) (local s) (LocalVarMismatch p) with preservationᴱ H M s +reflectᴮ H (local var x ∈ T ← M ∙ B) (local s) (LocalVarMismatch p) | ok q = block (LocalVarMismatch (λ r → p (trans r q))) +reflectᴮ H (local var x ∈ T ← M ∙ B) (local s) (LocalVarMismatch p) | warning (expr W) = block (local₁ W) +reflectᴮ H (local var x ∈ T ← M ∙ B) (local s) (LocalVarMismatch p) | warning (heap W) = heap W +reflectᴮ H (local var x ∈ T ← M ∙ B) (local s) (local₁ W′) with reflectᴱ H M s W′ +reflectᴮ H (local var x ∈ T ← M ∙ B) (local s) (local₁ W′) | heap W = heap W +reflectᴮ H (local var x ∈ T ← M ∙ B) (local s) (local₁ W′) | expr W = block (local₁ W) +reflectᴮ H (local var x ∈ T ← M ∙ B) (local s) (local₂ W′) = block (local₂ (reflect-weakeningᴮ H B (rednᴱ⊑ s) W′)) +reflectᴮ H (local var x ∈ T ← M ∙ B) (subst v) W′ with remember (typeOfⱽ H v) +reflectᴮ H (local var x ∈ T ← M ∙ B) (subst v) W′ | (just S , p) with S ≡ᵀ T +reflectᴮ H (local var x ∈ T ← M ∙ B) (subst v) W′ | (just T , p) | yes refl = block (local₂ (reflect-substitutionᴮ H B v x (sym p) W′)) +reflectᴮ H (local var x ∈ T ← M ∙ B) (subst v) W′ | (just S , p) | no q = block (LocalVarMismatch (λ r → q (trans (cong orNone (sym p)) (sym r)))) +reflectᴮ H (local var x ∈ T ← M ∙ B) (subst v) W′ | (nothing , p) with typeOf-val-not-none v +reflectᴮ H (local var x ∈ T ← M ∙ B) (subst v) W′ | (nothing , p) | ok r = CONTRADICTION (r (cong orNone (sym p))) +reflectᴮ H (local var x ∈ T ← M ∙ B) (subst v) W′ | (nothing , p) | warning W = block (local₁ W) +reflectᴮ H (function f ⟨ var y ∈ T ⟩∈ U is C end ∙ B) (function a defn) W′ = block (function₂ (reflect-weakeningᴮ H B (snoc defn) (reflect-substitutionᴮ _ B (addr a) f refl W′))) +reflectᴮ H (return M ∙ B) (return s) (return W′) with reflectᴱ H M s W′ +reflectᴮ H (return M ∙ B) (return s) (return W′) | heap W = heap W +reflectᴮ H (return M ∙ B) (return s) (return W′) | expr W = block (return W) + +reflectᴴᴱ : ∀ H M {H′ M′} → (H ⊢ M ⟶ᴱ M′ ⊣ H′) → Warningᴴᴱ H′ (typeCheckᴴᴱ H′ ∅ M′) → Warningᴴᴱ H (typeCheckᴴᴱ H ∅ M) +reflectᴴᴮ : ∀ H B {H′ B′} → (H ⊢ B ⟶ᴮ B′ ⊣ H′) → Warningᴴᴮ H′ (typeCheckᴴᴮ H′ ∅ B′) → Warningᴴᴮ H (typeCheckᴴᴮ H ∅ B) + +reflectᴴᴱ H M s (expr W′) = reflectᴱ H M s W′ +reflectᴴᴱ H (function f ⟨ var x ∈ T ⟩∈ U is B end) (function a p) (heap (addr b refl W′)) with b ≡ᴬ a +reflectᴴᴱ H (function f ⟨ var x ∈ T ⟩∈ U is B end) (function a defn) (heap (addr a refl (FunctionDefnMismatch p))) | yes refl with heap-weakeningᴮ H B (snoc defn) +reflectᴴᴱ H (function f ⟨ var x ∈ T ⟩∈ U is B end) (function a defn) (heap (addr a refl (FunctionDefnMismatch p))) | yes refl | ok r = expr (FunctionDefnMismatch λ q → p (trans q r)) +reflectᴴᴱ H (function f ⟨ var x ∈ T ⟩∈ U is B end) (function a defn) (heap (addr a refl (FunctionDefnMismatch p))) | yes refl | warning W = expr (function₁ W) +reflectᴴᴱ H (function f ⟨ var x ∈ T ⟩∈ U is B end) (function a defn) (heap (addr a refl (function₁ W′))) | yes refl = expr (function₁ (reflect-weakeningᴮ H B (snoc defn) W′)) +reflectᴴᴱ H (function f ⟨ var x ∈ T ⟩∈ U is B end) (function a p) (heap (addr b refl W′)) | no r = heap (addr b (lookup-not-allocated p r) (reflect-weakeningᴼ H _ (snoc p) W′)) +reflectᴴᴱ H (M $ N) (app₁ s) (heap W′) with reflectᴴᴱ H M s (heap W′) +reflectᴴᴱ H (M $ N) (app₁ s) (heap W′) | heap W = heap W +reflectᴴᴱ H (M $ N) (app₁ s) (heap W′) | expr W = expr (app₁ W) +reflectᴴᴱ H (M $ N) (app₂ p s) (heap W′) with reflectᴴᴱ H N s (heap W′) +reflectᴴᴱ H (M $ N) (app₂ p s) (heap W′) | heap W = heap W +reflectᴴᴱ H (M $ N) (app₂ p s) (heap W′) | expr W = expr (app₂ W) +reflectᴴᴱ H (M $ N) (beta O v p q) (heap W′) = heap W′ +reflectᴴᴱ H (block var b ∈ T is B end) (block s) (heap W′) with reflectᴴᴮ H B s (heap W′) +reflectᴴᴱ H (block var b ∈ T is B end) (block s) (heap W′) | heap W = heap W +reflectᴴᴱ H (block var b ∈ T is B end) (block s) (heap W′) | block W = expr (block₁ W) +reflectᴴᴱ H (block var b ∈ T is return N ∙ B end) (return v) (heap W′) = heap W′ +reflectᴴᴱ H (block var b ∈ T is done end) done (heap W′) = heap W′ +reflectᴴᴱ H (binexp M op N) (binOp₀ s) (heap W′) = heap W′ +reflectᴴᴱ H (binexp M op N) (binOp₁ s) (heap W′) with reflectᴴᴱ H M s (heap W′) +reflectᴴᴱ H (binexp M op N) (binOp₁ s) (heap W′) | heap W = heap W +reflectᴴᴱ H (binexp M op N) (binOp₁ s) (heap W′) | expr W = expr (bin₁ W) +reflectᴴᴱ H (binexp M op N) (binOp₂ s) (heap W′) with reflectᴴᴱ H N s (heap W′) +reflectᴴᴱ H (binexp M op N) (binOp₂ s) (heap W′) | heap W = heap W +reflectᴴᴱ H (binexp M op N) (binOp₂ s) (heap W′) | expr W = expr (bin₂ W) + +reflectᴴᴮ H B s (block W′) = reflectᴮ H B s W′ +reflectᴴᴮ H (local var x ∈ T ← M ∙ B) (local s) (heap W′) with reflectᴴᴱ H M s (heap W′) +reflectᴴᴮ H (local var x ∈ T ← M ∙ B) (local s) (heap W′) | heap W = heap W +reflectᴴᴮ H (local var x ∈ T ← M ∙ B) (local s) (heap W′) | expr W = block (local₁ W) +reflectᴴᴮ H (local var x ∈ T ← M ∙ B) (subst v) (heap W′) = heap W′ +reflectᴴᴮ H (function f ⟨ var y ∈ T ⟩∈ U is C end ∙ B) (function a p) (heap (addr b refl W′)) with b ≡ᴬ a +reflectᴴᴮ H (function f ⟨ var y ∈ T ⟩∈ U is C end ∙ B) (function a defn) (heap (addr a refl (FunctionDefnMismatch p))) | yes refl with heap-weakeningᴮ H C (snoc defn) +reflectᴴᴮ H (function f ⟨ var y ∈ T ⟩∈ U is C end ∙ B) (function a defn) (heap (addr a refl (FunctionDefnMismatch p))) | yes refl | ok r = block (FunctionDefnMismatch (λ q → p (trans q r))) +reflectᴴᴮ H (function f ⟨ var y ∈ T ⟩∈ U is C end ∙ B) (function a defn) (heap (addr a refl (FunctionDefnMismatch p))) | yes refl | warning W = block (function₁ W) +reflectᴴᴮ H (function f ⟨ var y ∈ T ⟩∈ U is C end ∙ B) (function a defn) (heap (addr a refl (function₁ W′))) | yes refl = block (function₁ (reflect-weakeningᴮ H C (snoc defn) W′)) +reflectᴴᴮ H (function f ⟨ var y ∈ T ⟩∈ U is C end ∙ B) (function a p) (heap (addr b refl W′)) | no r = heap (addr b (lookup-not-allocated p r) (reflect-weakeningᴼ H _ (snoc p) W′)) +reflectᴴᴮ H (return M ∙ B) (return s) (heap W′) with reflectᴴᴱ H M s (heap W′) +reflectᴴᴮ H (return M ∙ B) (return s) (heap W′) | heap W = heap W +reflectᴴᴮ H (return M ∙ B) (return s) (heap W′) | expr W = block (return W) + +reflect* : ∀ H B {H′ B′} → (H ⊢ B ⟶* B′ ⊣ H′) → Warningᴴᴮ H′ (typeCheckᴴᴮ H′ ∅ B′) → Warningᴴᴮ H (typeCheckᴴᴮ H ∅ B) +reflect* H B refl W = W +reflect* H B (step s t) W = reflectᴴᴮ H B s (reflect* _ _ t W) + +runtimeBinOpWarning : ∀ H {op} v → BinOpError op (valueType v) → BinOpWarning op (orNone (typeOfⱽ H v)) +runtimeBinOpWarning H v (+ p) = + (λ q → p (mustBeNumber H ∅ v q)) +runtimeBinOpWarning H v (- p) = - (λ q → p (mustBeNumber H ∅ v q)) +runtimeBinOpWarning H v (* p) = * (λ q → p (mustBeNumber H ∅ v q)) +runtimeBinOpWarning H v (/ p) = / (λ q → p (mustBeNumber H ∅ v q)) +runtimeBinOpWarning H v (< p) = < (λ q → p (mustBeNumber H ∅ v q)) +runtimeBinOpWarning H v (> p) = > (λ q → p (mustBeNumber H ∅ v q)) +runtimeBinOpWarning H v (<= p) = <= (λ q → p (mustBeNumber H ∅ v q)) +runtimeBinOpWarning H v (>= p) = >= (λ q → p (mustBeNumber H ∅ v q)) + +runtimeWarningᴱ : ∀ H M → RuntimeErrorᴱ H M → Warningᴱ H (typeCheckᴱ H ∅ M) +runtimeWarningᴮ : ∀ H B → RuntimeErrorᴮ H B → Warningᴮ H (typeCheckᴮ H ∅ B) + +runtimeWarningᴱ H (var x) UnboundVariable = UnboundVariable refl +runtimeWarningᴱ H (val (addr a)) (SEGV p) = UnallocatedAddress p +runtimeWarningᴱ H (M $ N) (FunctionMismatch v w p) with typeOf-val-not-none w +runtimeWarningᴱ H (M $ N) (FunctionMismatch v w p) | ok q = FunctionCallMismatch (λ r → p (mustBeFunction H ∅ v (λ r′ → q (trans r′ r)))) +runtimeWarningᴱ H (M $ N) (FunctionMismatch v w p) | warning W = app₂ W +runtimeWarningᴱ H (M $ N) (app₁ err) = app₁ (runtimeWarningᴱ H M err) +runtimeWarningᴱ H (M $ N) (app₂ err) = app₂ (runtimeWarningᴱ H N err) +runtimeWarningᴱ H (block var b ∈ T is B end) (block err) = block₁ (runtimeWarningᴮ H B err) +runtimeWarningᴱ H (binexp M op N) (BinOpMismatch₁ v w p) = BinOpMismatch₁ (runtimeBinOpWarning H v p) +runtimeWarningᴱ H (binexp M op N) (BinOpMismatch₂ v w p) = BinOpMismatch₂ (runtimeBinOpWarning H w p) +runtimeWarningᴱ H (binexp M op N) (bin₁ err) = bin₁ (runtimeWarningᴱ H M err) +runtimeWarningᴱ H (binexp M op N) (bin₂ err) = bin₂ (runtimeWarningᴱ H N err) + +runtimeWarningᴮ H (local var x ∈ T ← M ∙ B) (local err) = local₁ (runtimeWarningᴱ H M err) +runtimeWarningᴮ H (return M ∙ B) (return err) = return (runtimeWarningᴱ H M err) + +wellTypedProgramsDontGoWrong : ∀ H′ B B′ → (∅ᴴ ⊢ B ⟶* B′ ⊣ H′) → (RuntimeErrorᴮ H′ B′) → Warningᴮ ∅ᴴ (typeCheckᴮ ∅ᴴ ∅ B) +wellTypedProgramsDontGoWrong H′ B B′ t err with reflect* ∅ᴴ B t (block (runtimeWarningᴮ H′ B′ err)) +wellTypedProgramsDontGoWrong H′ B B′ t err | heap (addr a refl ()) +wellTypedProgramsDontGoWrong H′ B B′ t err | block W = W diff --git a/prototyping/Properties/TypeCheck.agda b/prototyping/Properties/TypeCheck.agda new file mode 100644 index 00000000..c144bc12 --- /dev/null +++ b/prototyping/Properties/TypeCheck.agda @@ -0,0 +1,103 @@ +{-# OPTIONS --rewriting #-} + +open import Luau.Type using (Mode) + +module Properties.TypeCheck (m : Mode) where + +open import Agda.Builtin.Equality using (_≡_; refl) +open import Agda.Builtin.Bool using (Bool; true; false) +open import FFI.Data.Maybe using (Maybe; just; nothing) +open import FFI.Data.Either using (Either) +open import Luau.TypeCheck(m) using (_⊢ᴱ_∈_; _⊢ᴮ_∈_; ⊢ᴼ_; ⊢ᴴ_; _⊢ᴴᴱ_▷_∈_; _⊢ᴴᴮ_▷_∈_; nil; var; addr; number; bool; app; function; block; binexp; done; return; local; nothing; orNone; tgtBinOp) +open import Luau.Syntax using (Block; Expr; Value; BinaryOperator; yes; nil; addr; number; bool; val; var; binexp; _$_; function_is_end; block_is_end; _∙_; return; done; local_←_; _⟨_⟩; _⟨_⟩∈_; var_∈_; name; fun; arg; +; -; *; /; <; >; ==; ~=; <=; >=) +open import Luau.Type using (Type; nil; any; none; number; boolean; _⇒_; tgt) +open import Luau.RuntimeType using (RuntimeType; nil; number; function; valueType) +open import Luau.VarCtxt using (VarCtxt; ∅; _↦_; _⊕_↦_; _⋒_; _⊝_) renaming (_[_] to _[_]ⱽ) +open import Luau.Addr using (Addr) +open import Luau.Var using (Var; _≡ⱽ_) +open import Luau.Heap using (Heap; Object; function_is_end) renaming (_[_] to _[_]ᴴ) +open import Properties.Contradiction using (CONTRADICTION) +open import Properties.Dec using (yes; no) +open import Properties.Equality using (_≢_; sym; trans; cong) +open import Properties.Product using (_×_; _,_) +open import Properties.Remember using (Remember; remember; _,_) + +src : Type → Type +src = Luau.Type.src m + +typeOfᴼ : Object yes → Type +typeOfᴼ (function f ⟨ var x ∈ S ⟩∈ T is B end) = (S ⇒ T) + +typeOfᴹᴼ : Maybe(Object yes) → Maybe Type +typeOfᴹᴼ nothing = nothing +typeOfᴹᴼ (just O) = just (typeOfᴼ O) + +typeOfⱽ : Heap yes → Value → Maybe Type +typeOfⱽ H nil = just nil +typeOfⱽ H (bool b) = just boolean +typeOfⱽ H (addr a) = typeOfᴹᴼ (H [ a ]ᴴ) +typeOfⱽ H (number n) = just number + +typeOfᴱ : Heap yes → VarCtxt → (Expr yes) → Type +typeOfᴮ : Heap yes → VarCtxt → (Block yes) → Type + +typeOfᴱ H Γ (var x) = orNone(Γ [ x ]ⱽ) +typeOfᴱ H Γ (val v) = orNone(typeOfⱽ H v) +typeOfᴱ H Γ (M $ N) = tgt(typeOfᴱ H Γ M) +typeOfᴱ H Γ (function f ⟨ var x ∈ S ⟩∈ T is B end) = S ⇒ T +typeOfᴱ H Γ (block var b ∈ T is B end) = T +typeOfᴱ H Γ (binexp M op N) = tgtBinOp op + +typeOfᴮ H Γ (function f ⟨ var x ∈ S ⟩∈ T is C end ∙ B) = typeOfᴮ H (Γ ⊕ f ↦ (S ⇒ T)) B +typeOfᴮ H Γ (local var x ∈ T ← M ∙ B) = typeOfᴮ H (Γ ⊕ x ↦ T) B +typeOfᴮ H Γ (return M ∙ B) = typeOfᴱ H Γ M +typeOfᴮ H Γ done = nil + +mustBeFunction : ∀ H Γ v → (none ≢ src (typeOfᴱ H Γ (val v))) → (function ≡ valueType(v)) +mustBeFunction H Γ nil p = CONTRADICTION (p refl) +mustBeFunction H Γ (addr a) p = refl +mustBeFunction H Γ (number n) p = CONTRADICTION (p refl) +mustBeFunction H Γ (bool true) p = CONTRADICTION (p refl) +mustBeFunction H Γ (bool false) p = CONTRADICTION (p refl) + +mustBeNumber : ∀ H Γ v → (typeOfᴱ H Γ (val v) ≡ number) → (valueType(v) ≡ number) +mustBeNumber H Γ nil () +mustBeNumber H Γ (addr a) p with remember (H [ a ]ᴴ) +mustBeNumber H Γ (addr a) p | (just O , q) with trans (cong orNone (cong typeOfᴹᴼ (sym q))) p +mustBeNumber H Γ (addr a) p | (just function f ⟨ var x ∈ T ⟩∈ U is B end , q) | () +mustBeNumber H Γ (addr a) p | (nothing , q) with trans (cong orNone (cong typeOfᴹᴼ (sym q))) p +mustBeNumber H Γ (addr a) p | nothing , q | () +mustBeNumber H Γ (number n) p = refl +mustBeNumber H Γ (bool true) () +mustBeNumber H Γ (bool false) () + +typeCheckᴱ : ∀ H Γ M → (Γ ⊢ᴱ M ∈ (typeOfᴱ H Γ M)) +typeCheckᴮ : ∀ H Γ B → (Γ ⊢ᴮ B ∈ (typeOfᴮ H Γ B)) + +typeCheckᴱ H Γ (var x) = var refl +typeCheckᴱ H Γ (val nil) = nil +typeCheckᴱ H Γ (val (addr a)) = addr (orNone (typeOfᴹᴼ (H [ a ]ᴴ))) +typeCheckᴱ H Γ (val (number n)) = number +typeCheckᴱ H Γ (val (bool b)) = bool +typeCheckᴱ H Γ (M $ N) = app (typeCheckᴱ H Γ M) (typeCheckᴱ H Γ N) +typeCheckᴱ H Γ (function f ⟨ var x ∈ T ⟩∈ U is B end) = function (typeCheckᴮ H (Γ ⊕ x ↦ T) B) +typeCheckᴱ H Γ (block var b ∈ T is B end) = block (typeCheckᴮ H Γ B) +typeCheckᴱ H Γ (binexp M op N) = binexp (typeCheckᴱ H Γ M) (typeCheckᴱ H Γ N) + +typeCheckᴮ H Γ (function f ⟨ var x ∈ T ⟩∈ U is C end ∙ B) = function (typeCheckᴮ H (Γ ⊕ x ↦ T) C) (typeCheckᴮ H (Γ ⊕ f ↦ (T ⇒ U)) B) +typeCheckᴮ H Γ (local var x ∈ T ← M ∙ B) = local (typeCheckᴱ H Γ M) (typeCheckᴮ H (Γ ⊕ x ↦ T) B) +typeCheckᴮ H Γ (return M ∙ B) = return (typeCheckᴱ H Γ M) (typeCheckᴮ H Γ B) +typeCheckᴮ H Γ done = done + +typeCheckᴼ : ∀ H O → (⊢ᴼ O) +typeCheckᴼ H nothing = nothing +typeCheckᴼ H (just function f ⟨ var x ∈ T ⟩∈ U is B end) = function (typeCheckᴮ H (x ↦ T) B) + +typeCheckᴴ : ∀ H → (⊢ᴴ H) +typeCheckᴴ H a {O} p = typeCheckᴼ H (O) + +typeCheckᴴᴱ : ∀ H Γ M → (Γ ⊢ᴴᴱ H ▷ M ∈ typeOfᴱ H Γ M) +typeCheckᴴᴱ H Γ M = (typeCheckᴴ H , typeCheckᴱ H Γ M) + +typeCheckᴴᴮ : ∀ H Γ M → (Γ ⊢ᴴᴮ H ▷ M ∈ typeOfᴮ H Γ M) +typeCheckᴴᴮ H Γ M = (typeCheckᴴ H , typeCheckᴮ H Γ M) diff --git a/prototyping/Tests/Interpreter/binary_equality_bools/out.txt b/prototyping/Tests/Interpreter/binary_equality_bools/out.txt index c508d536..ee6f2a25 100644 --- a/prototyping/Tests/Interpreter/binary_equality_bools/out.txt +++ b/prototyping/Tests/Interpreter/binary_equality_bools/out.txt @@ -1 +1,4 @@ -false +ANNOTATED PROGRAM: +return true == false + +RAN WITH RESULT: false diff --git a/prototyping/Tests/Interpreter/binary_equality_numbers/out.txt b/prototyping/Tests/Interpreter/binary_equality_numbers/out.txt index 27ba77dd..86a499a8 100644 --- a/prototyping/Tests/Interpreter/binary_equality_numbers/out.txt +++ b/prototyping/Tests/Interpreter/binary_equality_numbers/out.txt @@ -1 +1,4 @@ -true +ANNOTATED PROGRAM: +return 1.0 == 1.0 + +RAN WITH RESULT: true diff --git a/prototyping/Tests/Interpreter/binary_exps/out.txt b/prototyping/Tests/Interpreter/binary_exps/out.txt index d3827e75..5b4d2644 100644 --- a/prototyping/Tests/Interpreter/binary_exps/out.txt +++ b/prototyping/Tests/Interpreter/binary_exps/out.txt @@ -1 +1,4 @@ -1.0 +ANNOTATED PROGRAM: +return 1.0 + 2.0 - 2.0 * 2.0 / 2.0 + +RAN WITH RESULT: 1.0 diff --git a/prototyping/Tests/Interpreter/return_nil/out.txt b/prototyping/Tests/Interpreter/return_nil/out.txt index 607602cf..1d5f45b6 100644 --- a/prototyping/Tests/Interpreter/return_nil/out.txt +++ b/prototyping/Tests/Interpreter/return_nil/out.txt @@ -1 +1,7 @@ -nil +UNANNOTATED PROGRAM: +local function foo(x) + return nil +end +return foo(nil) + +RAN WITH RESULT: nil From 6c923b88021df9dc902ed788933996cee556dc30 Mon Sep 17 00:00:00 2001 From: Lily Brown Date: Wed, 2 Mar 2022 15:26:58 -0800 Subject: [PATCH 188/399] Prototyping: strings (#390) --- .github/workflows/prototyping.yml | 3 +- prototyping/Examples/Run.agda | 5 ++- prototyping/Luau/OpSem.agda | 6 ++-- prototyping/Luau/RuntimeError.agda | 5 +-- prototyping/Luau/RuntimeError/ToString.agda | 4 ++- prototyping/Luau/RuntimeType.agda | 4 ++- prototyping/Luau/RuntimeType/ToString.agda | 3 +- prototyping/Luau/StrictMode.agda | 5 +-- prototyping/Luau/Syntax.agda | 3 ++ prototyping/Luau/Syntax/FromJSON.agda | 7 +++- prototyping/Luau/Syntax/ToString.agda | 5 ++- prototyping/Luau/Type.agda | 22 ++++++++++++ prototyping/Luau/Type/FromJSON.agda | 3 +- prototyping/Luau/Type/ToString.agda | 3 +- prototyping/Luau/TypeCheck.agda | 10 ++++-- prototyping/Properties/Step.agda | 36 +++++++++++++++++-- prototyping/Properties/StrictMode.agda | 14 +++++--- prototyping/Properties/TypeCheck.agda | 23 ++++++++---- .../concat_number_and_string/in.lua | 3 ++ .../concat_number_and_string/out.txt | 11 ++++++ .../Interpreter/concat_two_strings/in.lua | 3 ++ .../Interpreter/concat_two_strings/out.txt | 6 ++++ .../Tests/Interpreter/return_string/in.lua | 1 + .../Tests/Interpreter/return_string/out.txt | 4 +++ 24 files changed, 156 insertions(+), 33 deletions(-) create mode 100644 prototyping/Tests/Interpreter/concat_number_and_string/in.lua create mode 100644 prototyping/Tests/Interpreter/concat_number_and_string/out.txt create mode 100644 prototyping/Tests/Interpreter/concat_two_strings/in.lua create mode 100644 prototyping/Tests/Interpreter/concat_two_strings/out.txt create mode 100644 prototyping/Tests/Interpreter/return_string/in.lua create mode 100644 prototyping/Tests/Interpreter/return_string/out.txt diff --git a/.github/workflows/prototyping.yml b/.github/workflows/prototyping.yml index c92e5803..6bc8a81b 100644 --- a/.github/workflows/prototyping.yml +++ b/.github/workflows/prototyping.yml @@ -37,8 +37,7 @@ jobs: - name: check targets working-directory: prototyping run: | - ~/.cabal/bin/agda Examples.agda - ~/.cabal/bin/agda Properties.agda + ~/.cabal/bin/agda Everything.agda - name: build executables working-directory: prototyping run: | diff --git a/prototyping/Examples/Run.agda b/prototyping/Examples/Run.agda index 545ecf51..84ebf84e 100644 --- a/prototyping/Examples/Run.agda +++ b/prototyping/Examples/Run.agda @@ -4,7 +4,7 @@ module Examples.Run where open import Agda.Builtin.Equality using (_≡_; refl) open import Agda.Builtin.Bool using (true; false) -open import Luau.Syntax using (nil; var; _$_; function_is_end; return; _∙_; done; _⟨_⟩; number; binexp; +; <; val; bool) +open import Luau.Syntax using (nil; var; _$_; function_is_end; return; _∙_; done; _⟨_⟩; number; binexp; +; <; val; bool; ~=; string) open import Luau.Run using (run; return) ex1 : (run (function "id" ⟨ var "x" ⟩ is return (var "x") ∙ done end ∙ return (var "id" $ val nil) ∙ done) ≡ return nil _) @@ -18,3 +18,6 @@ ex3 = refl ex4 : (run (function "fn" ⟨ var "x" ⟩ is return (binexp (val (number 1.0)) < (val (number 2.0))) ∙ done end ∙ return (var "fn" $ val nil) ∙ done) ≡ return (bool true) _) ex4 = refl + +ex5 : (run (function "fn" ⟨ var "x" ⟩ is return (binexp (val (string "foo")) ~= (val (string "bar"))) ∙ done end ∙ return (var "fn" $ val nil) ∙ done) ≡ return (bool true) _) +ex5 = refl diff --git a/prototyping/Luau/OpSem.agda b/prototyping/Luau/OpSem.agda index cbb55978..1f616c7e 100644 --- a/prototyping/Luau/OpSem.agda +++ b/prototyping/Luau/OpSem.agda @@ -5,12 +5,13 @@ module Luau.OpSem where open import Agda.Builtin.Equality using (_≡_) open import Agda.Builtin.Float using (Float; primFloatPlus; primFloatMinus; primFloatTimes; primFloatDiv; primFloatEquality; primFloatLess; primFloatInequality) open import Agda.Builtin.Bool using (Bool; true; false) +open import Agda.Builtin.String using (primStringEquality; primStringAppend) open import Utility.Bool using (not; _or_; _and_) open import Agda.Builtin.Nat using () renaming (_==_ to _==ᴬ_) open import FFI.Data.Maybe using (Maybe; just; nothing) open import Luau.Heap using (Heap; _≡_⊕_↦_; _[_]; function_is_end) open import Luau.Substitution using (_[_/_]ᴮ) -open import Luau.Syntax using (Value; Expr; Stat; Block; nil; addr; val; var; function_is_end; _$_; block_is_end; local_←_; _∙_; done; return; name; fun; arg; binexp; BinaryOperator; +; -; *; /; <; >; ==; ~=; <=; >=; number; bool) +open import Luau.Syntax using (Value; Expr; Stat; Block; nil; addr; val; var; function_is_end; _$_; block_is_end; local_←_; _∙_; done; return; name; fun; arg; binexp; BinaryOperator; +; -; *; /; <; >; ==; ~=; <=; >=; ··; number; bool; string) open import Luau.RuntimeType using (RuntimeType; valueType) open import Properties.Product using (_×_; _,_) @@ -37,7 +38,8 @@ data _⟦_⟧_⟶_ : Value → BinaryOperator → Value → Value → Set where >= : ∀ m n → (number m) ⟦ >= ⟧ (number n) ⟶ bool ((primFloatLess n m) or (primFloatEquality m n)) == : ∀ v w → v ⟦ == ⟧ w ⟶ bool (evalEqOp v w) ~= : ∀ v w → v ⟦ ~= ⟧ w ⟶ bool (evalNeqOp v w) - + ·· : ∀ x y → (string x) ⟦ ·· ⟧ (string y) ⟶ string (primStringAppend x y) + data _⊢_⟶ᴮ_⊣_ {a} : Heap a → Block a → Block a → Heap a → Set data _⊢_⟶ᴱ_⊣_ {a} : Heap a → Expr a → Expr a → Heap a → Set diff --git a/prototyping/Luau/RuntimeError.agda b/prototyping/Luau/RuntimeError.agda index 4440d2b7..0c8a594b 100644 --- a/prototyping/Luau/RuntimeError.agda +++ b/prototyping/Luau/RuntimeError.agda @@ -6,8 +6,8 @@ open import Agda.Builtin.Equality using (_≡_) open import Luau.Heap using (Heap; _[_]) open import FFI.Data.Maybe using (just; nothing) open import FFI.Data.String using (String) -open import Luau.Syntax using (BinaryOperator; Block; Expr; nil; var; val; addr; block_is_end; _$_; local_←_; return; done; _∙_; number; binexp; +; -; *; /; <; >; <=; >=) -open import Luau.RuntimeType using (RuntimeType; valueType; function; number) +open import Luau.Syntax using (BinaryOperator; Block; Expr; nil; var; val; addr; block_is_end; _$_; local_←_; return; done; _∙_; number; string; binexp; +; -; *; /; <; >; <=; >=; ··) +open import Luau.RuntimeType using (RuntimeType; valueType; function; number; string) open import Properties.Equality using (_≢_) data BinOpError : BinaryOperator → RuntimeType → Set where @@ -19,6 +19,7 @@ data BinOpError : BinaryOperator → RuntimeType → Set where > : ∀ {t} → (t ≢ number) → BinOpError > t <= : ∀ {t} → (t ≢ number) → BinOpError <= t >= : ∀ {t} → (t ≢ number) → BinOpError >= t + ·· : ∀ {t} → (t ≢ string) → BinOpError ·· t data RuntimeErrorᴮ {a} (H : Heap a) : Block a → Set data RuntimeErrorᴱ {a} (H : Heap a) : Expr a → Set diff --git a/prototyping/Luau/RuntimeError/ToString.agda b/prototyping/Luau/RuntimeError/ToString.agda index 6a6ed3e2..cd5f18f2 100644 --- a/prototyping/Luau/RuntimeError/ToString.agda +++ b/prototyping/Luau/RuntimeError/ToString.agda @@ -9,7 +9,7 @@ open import Luau.RuntimeType.ToString using (runtimeTypeToString) open import Luau.Addr.ToString using (addrToString) open import Luau.Syntax.ToString using (valueToString; exprToString) open import Luau.Var.ToString using (varToString) -open import Luau.Syntax using (var; val; addr; binexp; block_is_end; local_←_; return; _∙_; name; _$_) +open import Luau.Syntax using (var; val; addr; binexp; block_is_end; local_←_; return; _∙_; name; _$_; ··) errToStringᴱ : ∀ {a H} M → RuntimeErrorᴱ {a} H M → String errToStringᴮ : ∀ {a H} B → RuntimeErrorᴮ {a} H B → String @@ -19,6 +19,8 @@ errToStringᴱ (val (addr a)) (SEGV p) = "address " ++ addrToString a ++ " is un errToStringᴱ (M $ N) (FunctionMismatch v w p) = "value " ++ (valueToString v) ++ " is not a function" errToStringᴱ (M $ N) (app₁ E) = errToStringᴱ M E errToStringᴱ (M $ N) (app₂ E) = errToStringᴱ N E +errToStringᴱ (binexp M ·· N) (BinOpMismatch₁ v w p) = "value " ++ (valueToString v) ++ " is not a string" +errToStringᴱ (binexp M ·· N) (BinOpMismatch₂ v w p) = "value " ++ (valueToString w) ++ " is not a string" errToStringᴱ (binexp M op N) (BinOpMismatch₁ v w p) = "value " ++ (valueToString v) ++ " is not a number" errToStringᴱ (binexp M op N) (BinOpMismatch₂ v w p) = "value " ++ (valueToString w) ++ " is not a number" errToStringᴱ (binexp M op N) (bin₁ E) = errToStringᴱ M E diff --git a/prototyping/Luau/RuntimeType.agda b/prototyping/Luau/RuntimeType.agda index f6b8b4a4..d585b51b 100644 --- a/prototyping/Luau/RuntimeType.agda +++ b/prototyping/Luau/RuntimeType.agda @@ -1,15 +1,17 @@ module Luau.RuntimeType where -open import Luau.Syntax using (Value; nil; addr; number; bool) +open import Luau.Syntax using (Value; nil; addr; number; bool; string) data RuntimeType : Set where function : RuntimeType number : RuntimeType nil : RuntimeType boolean : RuntimeType + string : RuntimeType valueType : Value → RuntimeType valueType nil = nil valueType (addr a) = function valueType (number n) = number valueType (bool b) = boolean +valueType (string x) = string diff --git a/prototyping/Luau/RuntimeType/ToString.agda b/prototyping/Luau/RuntimeType/ToString.agda index dfb7c955..f3dd1255 100644 --- a/prototyping/Luau/RuntimeType/ToString.agda +++ b/prototyping/Luau/RuntimeType/ToString.agda @@ -1,10 +1,11 @@ module Luau.RuntimeType.ToString where open import FFI.Data.String using (String) -open import Luau.RuntimeType using (RuntimeType; function; number; nil; boolean) +open import Luau.RuntimeType using (RuntimeType; function; number; nil; boolean; string) runtimeTypeToString : RuntimeType → String runtimeTypeToString function = "function" runtimeTypeToString number = "number" runtimeTypeToString nil = "nil" runtimeTypeToString boolean = "boolean" +runtimeTypeToString string = "string" diff --git a/prototyping/Luau/StrictMode.agda b/prototyping/Luau/StrictMode.agda index d7b2129f..08e0f50b 100644 --- a/prototyping/Luau/StrictMode.agda +++ b/prototyping/Luau/StrictMode.agda @@ -4,8 +4,8 @@ module Luau.StrictMode where open import Agda.Builtin.Equality using (_≡_) open import FFI.Data.Maybe using (just; nothing) -open import Luau.Syntax using (Expr; Stat; Block; BinaryOperator; yes; nil; addr; var; binexp; var_∈_; _⟨_⟩∈_; function_is_end; _$_; block_is_end; local_←_; _∙_; done; return; name; +; -; *; /; <; >; <=; >=) -open import Luau.Type using (Type; strict; nil; number; _⇒_; tgt) +open import Luau.Syntax using (Expr; Stat; Block; BinaryOperator; yes; nil; addr; var; binexp; var_∈_; _⟨_⟩∈_; function_is_end; _$_; block_is_end; local_←_; _∙_; done; return; name; +; -; *; /; <; >; <=; >=; ··) +open import Luau.Type using (Type; strict; nil; number; string; _⇒_; tgt) open import Luau.Heap using (Heap; function_is_end) renaming (_[_] to _[_]ᴴ) open import Luau.VarCtxt using (VarCtxt; ∅; _⋒_; _↦_; _⊕_↦_; _⊝_) renaming (_[_] to _[_]ⱽ) open import Luau.TypeCheck(strict) using (_⊢ᴮ_∈_; _⊢ᴱ_∈_; ⊢ᴴ_; ⊢ᴼ_; _⊢ᴴᴱ_▷_∈_; _⊢ᴴᴮ_▷_∈_; var; addr; app; binexp; block; return; local; function) @@ -25,6 +25,7 @@ data BinOpWarning : BinaryOperator → Type → Set where > : ∀ {T} → (T ≢ number) → BinOpWarning > T <= : ∀ {T} → (T ≢ number) → BinOpWarning <= T >= : ∀ {T} → (T ≢ number) → BinOpWarning >= T + ·· : ∀ {T} → (T ≢ string) → BinOpWarning ·· T data Warningᴱ (H : Heap yes) {Γ} : ∀ {M T} → (Γ ⊢ᴱ M ∈ T) → Set data Warningᴮ (H : Heap yes) {Γ} : ∀ {B T} → (Γ ⊢ᴮ B ∈ T) → Set diff --git a/prototyping/Luau/Syntax.agda b/prototyping/Luau/Syntax.agda index 8be0830f..d9175067 100644 --- a/prototyping/Luau/Syntax.agda +++ b/prototyping/Luau/Syntax.agda @@ -3,6 +3,7 @@ module Luau.Syntax where open import Agda.Builtin.Equality using (_≡_) open import Agda.Builtin.Bool using (Bool; true; false) open import Agda.Builtin.Float using (Float) +open import Agda.Builtin.String using (String) open import Luau.Var using (Var) open import Luau.Addr using (Addr) open import Luau.Type using (Type) @@ -45,12 +46,14 @@ data BinaryOperator : Set where ~= : BinaryOperator <= : BinaryOperator >= : BinaryOperator + ·· : BinaryOperator data Value : Set where nil : Value addr : Addr → Value number : Float → Value bool : Bool → Value + string : String → Value data Block (a : Annotated) : Set data Stat (a : Annotated) : Set diff --git a/prototyping/Luau/Syntax/FromJSON.agda b/prototyping/Luau/Syntax/FromJSON.agda index a727a5b0..2263615d 100644 --- a/prototyping/Luau/Syntax/FromJSON.agda +++ b/prototyping/Luau/Syntax/FromJSON.agda @@ -2,7 +2,7 @@ module Luau.Syntax.FromJSON where -open import Luau.Syntax using (Block; Stat ; Expr; _$_; val; nil; bool; number; var; var_∈_; function_is_end; _⟨_⟩; _⟨_⟩∈_; local_←_; return; done; _∙_; maybe; VarDec; binexp; BinaryOperator; +; -; *; /; ==; ~=; <; >; <=; >=) +open import Luau.Syntax using (Block; Stat ; Expr; _$_; val; nil; bool; number; var; var_∈_; function_is_end; _⟨_⟩; _⟨_⟩∈_; local_←_; return; done; _∙_; maybe; VarDec; binexp; BinaryOperator; +; -; *; /; ==; ~=; <; >; <=; >=; ··; string) open import Luau.Type.FromJSON using (typeFromJSON) open import Agda.Builtin.List using (List; _∷_; []) @@ -65,6 +65,7 @@ binOpFromString "CompareLt" = Right < binOpFromString "CompareLe" = Right <= binOpFromString "CompareGt" = Right > binOpFromString "CompareGe" = Right >= +binOpFromString "Concat" = Right ·· binOpFromString s = Left ("'" ++ s ++ "' is not a valid operator") varDecFromJSON (object arg) = varDecFromObject arg @@ -122,6 +123,10 @@ exprFromObject obj | just (string "AstExprConstantNumber") with lookup value obj exprFromObject obj | just (string "AstExprConstantNumber") | just (FFI.Data.Aeson.Value.number x) = Right (val (number (toFloat x))) exprFromObject obj | just (string "AstExprConstantNumber") | just _ = Left "AstExprConstantNumber value is not a number" exprFromObject obj | just (string "AstExprConstantNumber") | nothing = Left "AstExprConstantNumber missing value" +exprFromObject obj | just (string "AstExprConstantString") with lookup value obj +exprFromObject obj | just (string "AstExprConstantString") | just (string x) = Right (val (string x)) +exprFromObject obj | just (string "AstExprConstantString") | just _ = Left "AstExprConstantString value is not a string" +exprFromObject obj | just (string "AstExprConstantString") | nothing = Left "AstExprConstantString missing value" exprFromObject obj | just (string "AstExprConstantBool") with lookup value obj exprFromObject obj | just (string "AstExprConstantBool") | just (FFI.Data.Aeson.Value.bool b) = Right (val (bool b)) exprFromObject obj | just (string "AstExprConstantBool") | just _ = Left "AstExprConstantBool value is not a bool" diff --git a/prototyping/Luau/Syntax/ToString.agda b/prototyping/Luau/Syntax/ToString.agda index 60ec471d..1743071a 100644 --- a/prototyping/Luau/Syntax/ToString.agda +++ b/prototyping/Luau/Syntax/ToString.agda @@ -2,7 +2,8 @@ module Luau.Syntax.ToString where open import Agda.Builtin.Bool using (true; false) open import Agda.Builtin.Float using (primShowFloat) -open import Luau.Syntax using (Value; Block; Stat; Expr; VarDec; FunDec; nil; bool; val; var; var_∈_; addr; _$_; function_is_end; return; local_←_; _∙_; done; block_is_end; _⟨_⟩; _⟨_⟩∈_; number; BinaryOperator; +; -; *; /; <; >; ==; ~=; <=; >=; binexp) +open import Agda.Builtin.String using (primShowString) +open import Luau.Syntax using (Value; Block; Stat; Expr; VarDec; FunDec; nil; bool; val; var; var_∈_; addr; _$_; function_is_end; return; local_←_; _∙_; done; block_is_end; _⟨_⟩; _⟨_⟩∈_; number; BinaryOperator; +; -; *; /; <; >; ==; ~=; <=; >=; ··; binexp; string) open import FFI.Data.String using (String; _++_) open import Luau.Addr.ToString using (addrToString) open import Luau.Type.ToString using (typeToString) @@ -29,6 +30,7 @@ binOpToString == = "==" binOpToString ~= = "~=" binOpToString <= = "<=" binOpToString >= = ">=" +binOpToString ·· = ".." valueToString : Value → String valueToString nil = "nil" @@ -36,6 +38,7 @@ valueToString (addr a) = addrToString a valueToString (number x) = primShowFloat x valueToString (bool false) = "false" valueToString (bool true) = "true" +valueToString (string x) = primShowString x exprToString′ : ∀ {a} → String → Expr a → String statToString′ : ∀ {a} → String → Stat a → String diff --git a/prototyping/Luau/Type.agda b/prototyping/Luau/Type.agda index 10f39c44..c5b76142 100644 --- a/prototyping/Luau/Type.agda +++ b/prototyping/Luau/Type.agda @@ -13,6 +13,7 @@ data Type : Set where any : Type boolean : Type number : Type + string : Type _∪_ : Type → Type → Type _∩_ : Type → Type → Type @@ -25,6 +26,7 @@ lhs none = none lhs any = any lhs number = number lhs boolean = boolean +lhs string = string rhs : Type → Type rhs (_ ⇒ T) = T @@ -35,6 +37,7 @@ rhs none = none rhs any = any rhs number = number rhs boolean = boolean +rhs string = string _≡ᵀ_ : ∀ (T U : Type) → Dec(T ≡ U) nil ≡ᵀ nil = yes refl @@ -45,6 +48,14 @@ nil ≡ᵀ number = no (λ ()) nil ≡ᵀ boolean = no (λ ()) nil ≡ᵀ (S ∪ T) = no (λ ()) nil ≡ᵀ (S ∩ T) = no (λ ()) +nil ≡ᵀ string = no (λ ()) +(S ⇒ T) ≡ᵀ string = no (λ ()) +none ≡ᵀ string = no (λ ()) +any ≡ᵀ string = no (λ ()) +boolean ≡ᵀ string = no (λ ()) +number ≡ᵀ string = no (λ ()) +(S ∪ T) ≡ᵀ string = no (λ ()) +(S ∩ T) ≡ᵀ string = no (λ ()) (S ⇒ T) ≡ᵀ nil = no (λ ()) (S ⇒ T) ≡ᵀ (U ⇒ V) with (S ≡ᵀ U) | (T ≡ᵀ V) (S ⇒ T) ≡ᵀ (S ⇒ T) | yes refl | yes refl = yes refl @@ -88,6 +99,15 @@ boolean ≡ᵀ boolean = yes refl boolean ≡ᵀ number = no (λ ()) boolean ≡ᵀ (T ∪ U) = no (λ ()) boolean ≡ᵀ (T ∩ U) = no (λ ()) +string ≡ᵀ nil = no (λ ()) +string ≡ᵀ (x ⇒ x₁) = no (λ ()) +string ≡ᵀ none = no (λ ()) +string ≡ᵀ any = no (λ ()) +string ≡ᵀ boolean = no (λ ()) +string ≡ᵀ number = no (λ ()) +string ≡ᵀ string = yes refl +string ≡ᵀ (U ∪ V) = no (λ ()) +string ≡ᵀ (U ∩ V) = no (λ ()) (S ∪ T) ≡ᵀ nil = no (λ ()) (S ∪ T) ≡ᵀ (U ⇒ V) = no (λ ()) (S ∪ T) ≡ᵀ none = no (λ ()) @@ -127,6 +147,7 @@ src : Mode → Type → Type src m nil = none src m number = none src m boolean = none +src m string = none src m (S ⇒ T) = S -- In nonstrict mode, functions are covaraiant, in strict mode they're contravariant src strict (S ∪ T) = (src strict S) ∩ (src strict T) @@ -145,6 +166,7 @@ tgt none = none tgt any = any tgt number = none tgt boolean = none +tgt string = none tgt (S ∪ T) = (tgt S) ∪ (tgt T) tgt (S ∩ T) = (tgt S) ∩ (tgt T) diff --git a/prototyping/Luau/Type/FromJSON.agda b/prototyping/Luau/Type/FromJSON.agda index 9057fef4..2d6ba689 100644 --- a/prototyping/Luau/Type/FromJSON.agda +++ b/prototyping/Luau/Type/FromJSON.agda @@ -2,7 +2,7 @@ module Luau.Type.FromJSON where -open import Luau.Type using (Type; nil; _⇒_; _∪_; _∩_; any; number) +open import Luau.Type using (Type; nil; _⇒_; _∪_; _∩_; any; number; string) open import Agda.Builtin.List using (List; _∷_; []) open import Agda.Builtin.Bool using (true; false) @@ -44,6 +44,7 @@ typeFromJSON (object o) | just (string "AstTypeReference") with lookup name o typeFromJSON (object o) | just (string "AstTypeReference") | just (string "nil") = Right nil typeFromJSON (object o) | just (string "AstTypeReference") | just (string "any") = Right any typeFromJSON (object o) | just (string "AstTypeReference") | just (string "number") = Right number +typeFromJSON (object o) | just (string "AstTypeReference") | just (string "string") = Right string typeFromJSON (object o) | just (string "AstTypeReference") | _ = Left "Unknown referenced type" typeFromJSON (object o) | just (string "AstTypeUnion") with lookup types o diff --git a/prototyping/Luau/Type/ToString.agda b/prototyping/Luau/Type/ToString.agda index dec7c14c..2efe6632 100644 --- a/prototyping/Luau/Type/ToString.agda +++ b/prototyping/Luau/Type/ToString.agda @@ -1,7 +1,7 @@ module Luau.Type.ToString where open import FFI.Data.String using (String; _++_) -open import Luau.Type using (Type; nil; _⇒_; none; any; number; boolean; _∪_; _∩_; normalizeOptional) +open import Luau.Type using (Type; nil; _⇒_; none; any; number; boolean; string; _∪_; _∩_; normalizeOptional) {-# TERMINATING #-} typeToString : Type → String @@ -14,6 +14,7 @@ typeToString none = "none" typeToString any = "any" typeToString number = "number" typeToString boolean = "boolean" +typeToString string = "string" typeToString (S ∪ T) with normalizeOptional(S ∪ T) typeToString (S ∪ T) | ((S′ ⇒ T′) ∪ nil) = "(" ++ typeToString (S′ ⇒ T′) ++ ")?" typeToString (S ∪ T) | (S′ ∪ nil) = typeToString S′ ++ "?" diff --git a/prototyping/Luau/TypeCheck.agda b/prototyping/Luau/TypeCheck.agda index f240e17e..65ed1831 100644 --- a/prototyping/Luau/TypeCheck.agda +++ b/prototyping/Luau/TypeCheck.agda @@ -6,11 +6,11 @@ module Luau.TypeCheck (m : Mode) where open import Agda.Builtin.Equality using (_≡_) open import FFI.Data.Maybe using (Maybe; just) -open import Luau.Syntax using (Expr; Stat; Block; BinaryOperator; yes; nil; addr; number; bool; val; var; var_∈_; _⟨_⟩∈_; function_is_end; _$_; block_is_end; binexp; local_←_; _∙_; done; return; name; +; -; *; /; <; >; ==; ~=; <=; >=) +open import Luau.Syntax using (Expr; Stat; Block; BinaryOperator; yes; nil; addr; number; bool; string; val; var; var_∈_; _⟨_⟩∈_; function_is_end; _$_; block_is_end; binexp; local_←_; _∙_; done; return; name; +; -; *; /; <; >; ==; ~=; <=; >=; ··) open import Luau.Var using (Var) open import Luau.Addr using (Addr) open import Luau.Heap using (Heap; Object; function_is_end) renaming (_[_] to _[_]ᴴ) -open import Luau.Type using (Type; Mode; nil; none; number; boolean; _⇒_; tgt) +open import Luau.Type using (Type; Mode; nil; none; number; boolean; string; _⇒_; tgt) open import Luau.VarCtxt using (VarCtxt; ∅; _⋒_; _↦_; _⊕_↦_; _⊝_) renaming (_[_] to _[_]ⱽ) open import FFI.Data.Vector using (Vector) open import FFI.Data.Maybe using (Maybe; just; nothing) @@ -34,6 +34,7 @@ tgtBinOp == = boolean tgtBinOp ~= = boolean tgtBinOp <= = boolean tgtBinOp >= = boolean +tgtBinOp ·· = string data _⊢ᴮ_∈_ : VarCtxt → Block yes → Type → Set data _⊢ᴱ_∈_ : VarCtxt → Expr yes → Type → Set @@ -93,6 +94,11 @@ data _⊢ᴱ_∈_ where -------------------------- Γ ⊢ᴱ val(bool b) ∈ boolean + + string : ∀ {x Γ} → + + --------------------------- + Γ ⊢ᴱ val(string x) ∈ string app : ∀ {M N T U Γ} → diff --git a/prototyping/Properties/Step.agda b/prototyping/Properties/Step.agda index 0b5fd062..cadd1530 100644 --- a/prototyping/Properties/Step.agda +++ b/prototyping/Properties/Step.agda @@ -5,11 +5,12 @@ module Properties.Step where open import Agda.Builtin.Equality using (_≡_; refl) open import Agda.Builtin.Float using (primFloatPlus; primFloatMinus; primFloatTimes; primFloatDiv; primFloatEquality; primFloatLess) open import Agda.Builtin.Bool using (true; false) +open import Agda.Builtin.String using (primStringAppend) open import FFI.Data.Maybe using (just; nothing) open import Luau.Heap using (Heap; _[_]; alloc; ok; function_is_end) -open import Luau.Syntax using (Block; Expr; nil; var; val; addr; bool; function_is_end; block_is_end; _$_; local_←_; return; done; _∙_; name; fun; arg; number; binexp; +; -; *; /; <; >; <=; >=; ==; ~=) -open import Luau.OpSem using (_⟦_⟧_⟶_; _⊢_⟶ᴱ_⊣_; _⊢_⟶ᴮ_⊣_; app₁ ; app₂ ; beta; function; block; return; done; local; subst; binOp₀; binOp₁; binOp₂; +; -; *; /; <; >; <=; >=; ==; ~=; evalEqOp; evalNeqOp) -open import Luau.RuntimeError using (BinOpError; RuntimeErrorᴱ; RuntimeErrorᴮ; FunctionMismatch; BinOpMismatch₁; BinOpMismatch₂; UnboundVariable; SEGV; app₁; app₂; block; local; return; bin₁; bin₂; +; -; *; /; <; >; <=; >=) +open import Luau.Syntax using (Block; Expr; nil; var; val; addr; bool; function_is_end; block_is_end; _$_; local_←_; return; done; _∙_; name; fun; arg; number; binexp; +; -; *; /; <; >; <=; >=; ==; ~=; ··; string) +open import Luau.OpSem using (_⟦_⟧_⟶_; _⊢_⟶ᴱ_⊣_; _⊢_⟶ᴮ_⊣_; app₁ ; app₂ ; beta; function; block; return; done; local; subst; binOp₀; binOp₁; binOp₂; +; -; *; /; <; >; <=; >=; ==; ~=; ··; evalEqOp; evalNeqOp) +open import Luau.RuntimeError using (BinOpError; RuntimeErrorᴱ; RuntimeErrorᴮ; FunctionMismatch; BinOpMismatch₁; BinOpMismatch₂; UnboundVariable; SEGV; app₁; app₂; block; local; return; bin₁; bin₂; +; -; *; /; <; >; <=; >=; ··) open import Luau.RuntimeType using (valueType; function; number) open import Luau.Substitution using (_[_/_]ᴮ) open import Properties.Remember using (remember; _,_) @@ -27,6 +28,16 @@ binOpStep (number m) + nil = error₂ (+ (λ ())) binOpStep (number m) + (addr a) = error₂ (+ (λ ())) binOpStep (number m) + (number n) = step (number (primFloatPlus m n)) (+ m n) binOpStep (number m) + (bool b) = error₂ (+ (λ ())) +binOpStep (number m) + (string x) = error₂ (+ (λ ())) +binOpStep (number m) - (string x) = error₂ (- (λ ())) +binOpStep (number m) * (string x) = error₂ (* (λ ())) +binOpStep (number m) / (string x) = error₂ (/ (λ ())) +binOpStep (number m) < (string x) = error₂ (< (λ ())) +binOpStep (number m) > (string x) = error₂ (> (λ ())) +binOpStep (number m) == (string x) = step (bool false) (== (number m) (string x)) +binOpStep (number m) ~= (string x) = step (bool true) (~= (number m) (string x)) +binOpStep (number m) <= (string x) = error₂ (<= (λ ())) +binOpStep (number m) >= (string x) = error₂ (>= (λ ())) binOpStep (bool b) + w = error₁ (+ (λ ())) binOpStep nil - w = error₁ (- (λ ())) binOpStep (addr a) - w = error₁ (- (λ ())) @@ -79,6 +90,23 @@ binOpStep (number m) >= (addr a) = error₂ (>= (λ ())) binOpStep (number m) >= (number n) = step (bool (primFloatLess n m or primFloatEquality m n)) (>= m n) binOpStep (number m) >= (bool b) = error₂ (>= (λ ())) binOpStep (bool b) >= w = error₁ (>= (λ ())) +binOpStep (string x) + w = error₁ (+ (λ ())) +binOpStep (string x) - w = error₁ (- (λ ())) +binOpStep (string x) * w = error₁ (* (λ ())) +binOpStep (string x) / w = error₁ (/ (λ ())) +binOpStep (string x) < w = error₁ (< (λ ())) +binOpStep (string x) > w = error₁ (> (λ ())) +binOpStep (string x) <= w = error₁ (<= (λ ())) +binOpStep (string x) >= w = error₁ (>= (λ ())) +binOpStep nil ·· y = error₁ (·· (λ ())) +binOpStep (addr x) ·· y = error₁ (BinOpError.·· (λ ())) +binOpStep (number x) ·· y = error₁ (BinOpError.·· (λ ())) +binOpStep (bool x) ·· y = error₁ (BinOpError.·· (λ ())) +binOpStep (string x) ·· nil = error₂ (·· (λ ())) +binOpStep (string x) ·· (addr y) = error₂ (·· (λ ())) +binOpStep (string x) ·· (number y) = error₂ (·· (λ ())) +binOpStep (string x) ·· (bool y) = error₂ (·· (λ ())) +binOpStep (string x) ·· (string y) = step (string (primStringAppend x y)) (·· x y) data StepResultᴮ {a} (H : Heap a) (B : Block a) : Set data StepResultᴱ {a} (H : Heap a) (M : Expr a) : Set @@ -109,6 +137,7 @@ stepᴱ H (_ $ _) | value (addr a) refl | value w refl | (just(function F is B stepᴱ H (_ $ _) | value nil refl | value w refl = error (FunctionMismatch nil w (λ ())) stepᴱ H (_ $ _) | value (number m) refl | value w refl = error (FunctionMismatch (number m) w (λ ())) stepᴱ H (_ $ _) | value (bool b) refl | value w refl = error (FunctionMismatch (bool b) w (λ ())) +stepᴱ H (_ $ _) | value (string x) refl | value w refl = error (FunctionMismatch (string x) w (λ ())) stepᴱ H (M $ N) | value V p | error E = error (app₂ E) stepᴱ H (M $ N) | error E = error (app₁ E) stepᴱ H (block b is B end) with stepᴮ H B @@ -140,3 +169,4 @@ stepᴮ H (return M ∙ B) | step H′ M′ D = step H′ (return M′ ∙ B) (r stepᴮ H (return _ ∙ B) | value V refl = return V refl stepᴮ H (return M ∙ B) | error E = error (return E) stepᴮ H done = done refl + \ No newline at end of file diff --git a/prototyping/Properties/StrictMode.agda b/prototyping/Properties/StrictMode.agda index 08411188..77852f1a 100644 --- a/prototyping/Properties/StrictMode.agda +++ b/prototyping/Properties/StrictMode.agda @@ -6,9 +6,9 @@ import Agda.Builtin.Equality.Rewrite open import Agda.Builtin.Equality using (_≡_; refl) open import FFI.Data.Maybe using (Maybe; just; nothing) open import Luau.Heap using (Heap; Object; function_is_end; defn; alloc; ok; next; lookup-not-allocated) renaming (_≡_⊕_↦_ to _≡ᴴ_⊕_↦_; _[_] to _[_]ᴴ; ∅ to ∅ᴴ) -open import Luau.StrictMode using (Warningᴱ; Warningᴮ; Warningᴼ; Warningᴴᴱ; Warningᴴᴮ; UnallocatedAddress; UnboundVariable; FunctionCallMismatch; app₁; app₂; BinOpWarning; BinOpMismatch₁; BinOpMismatch₂; bin₁; bin₂; BlockMismatch; block₁; return; LocalVarMismatch; local₁; local₂; FunctionDefnMismatch; function₁; function₂; heap; expr; block; addr; +; -; *; /; <; >; <=; >=) +open import Luau.StrictMode using (Warningᴱ; Warningᴮ; Warningᴼ; Warningᴴᴱ; Warningᴴᴮ; UnallocatedAddress; UnboundVariable; FunctionCallMismatch; app₁; app₂; BinOpWarning; BinOpMismatch₁; BinOpMismatch₂; bin₁; bin₂; BlockMismatch; block₁; return; LocalVarMismatch; local₁; local₂; FunctionDefnMismatch; function₁; function₂; heap; expr; block; addr; +; -; *; /; <; >; <=; >=; ··) open import Luau.Substitution using (_[_/_]ᴮ; _[_/_]ᴱ; _[_/_]ᴮunless_; var_[_/_]ᴱwhenever_) -open import Luau.Syntax using (Expr; yes; var; val; var_∈_; _⟨_⟩∈_; _$_; addr; number; bool; binexp; nil; function_is_end; block_is_end; done; return; local_←_; _∙_; fun; arg; name; ==; ~=) +open import Luau.Syntax using (Expr; yes; var; val; var_∈_; _⟨_⟩∈_; _$_; addr; number; bool; string; binexp; nil; function_is_end; block_is_end; done; return; local_←_; _∙_; fun; arg; name; ==; ~=) open import Luau.Type using (Type; strict; nil; _⇒_; none; tgt; _≡ᵀ_; _≡ᴹᵀ_) open import Luau.TypeCheck(strict) using (_⊢ᴮ_∈_; _⊢ᴱ_∈_; _⊢ᴴᴮ_▷_∈_; _⊢ᴴᴱ_▷_∈_; nil; var; addr; app; function; block; done; return; local; orNone; tgtBinOp) open import Luau.Var using (_≡ⱽ_) @@ -19,9 +19,9 @@ open import Properties.Remember using (remember; _,_) open import Properties.Equality using (_≢_; sym; cong; trans; subst₁) open import Properties.Dec using (Dec; yes; no) open import Properties.Contradiction using (CONTRADICTION) -open import Properties.TypeCheck(strict) using (typeOfᴼ; typeOfᴹᴼ; typeOfⱽ; typeOfᴱ; typeOfᴮ; typeCheckᴱ; typeCheckᴮ; typeCheckᴼ; typeCheckᴴᴱ; typeCheckᴴᴮ; mustBeFunction; mustBeNumber) -open import Luau.OpSem using (_⟦_⟧_⟶_; _⊢_⟶*_⊣_; _⊢_⟶ᴮ_⊣_; _⊢_⟶ᴱ_⊣_; app₁; app₂; function; beta; return; block; done; local; subst; binOp₀; binOp₁; binOp₂; refl; step; +; -; *; /; <; >; ==; ~=; <=; >=) -open import Luau.RuntimeError using (BinOpError; RuntimeErrorᴱ; RuntimeErrorᴮ; FunctionMismatch; BinOpMismatch₁; BinOpMismatch₂; UnboundVariable; SEGV; app₁; app₂; bin₁; bin₂; block; local; return; +; -; *; /; <; >; <=; >=) +open import Properties.TypeCheck(strict) using (typeOfᴼ; typeOfᴹᴼ; typeOfⱽ; typeOfᴱ; typeOfᴮ; typeCheckᴱ; typeCheckᴮ; typeCheckᴼ; typeCheckᴴᴱ; typeCheckᴴᴮ; mustBeFunction; mustBeNumber; mustBeString) +open import Luau.OpSem using (_⟦_⟧_⟶_; _⊢_⟶*_⊣_; _⊢_⟶ᴮ_⊣_; _⊢_⟶ᴱ_⊣_; app₁; app₂; function; beta; return; block; done; local; subst; binOp₀; binOp₁; binOp₂; refl; step; +; -; *; /; <; >; ==; ~=; <=; >=; ··) +open import Luau.RuntimeError using (BinOpError; RuntimeErrorᴱ; RuntimeErrorᴮ; FunctionMismatch; BinOpMismatch₁; BinOpMismatch₂; UnboundVariable; SEGV; app₁; app₂; bin₁; bin₂; block; local; return; +; -; *; /; <; >; <=; >=; ··) open import Luau.RuntimeType using (valueType) src = Luau.Type.src strict @@ -86,6 +86,7 @@ heap-weakeningᴱ H (val (addr a)) (snoc {a = a} defn) | yes refl = warning (Una heap-weakeningᴱ H (val (addr a)) (snoc {a = b} p) | no q = ok (cong orNone (cong typeOfᴹᴼ (lookup-not-allocated p q))) heap-weakeningᴱ H (val (number n)) h = ok refl heap-weakeningᴱ H (val (bool b)) h = ok refl +heap-weakeningᴱ H (val (string x)) h = ok refl heap-weakeningᴱ H (binexp M op N) h = ok refl heap-weakeningᴱ H (M $ N) h with heap-weakeningᴱ H M h heap-weakeningᴱ H (M $ N) h | ok p = ok (cong tgt p) @@ -110,6 +111,7 @@ typeOf-val-not-none : ∀ {H Γ} v → OrWarningᴱ H (typeCheckᴱ H Γ (val v) typeOf-val-not-none nil = ok (λ ()) typeOf-val-not-none (number n) = ok (λ ()) typeOf-val-not-none (bool b) = ok (λ ()) +typeOf-val-not-none (string x) = ok (λ ()) typeOf-val-not-none {H = H} (addr a) with remember (H [ a ]ᴴ) typeOf-val-not-none {H = H} (addr a) | (just O , p) = ok (λ q → none-not-obj O (trans q (cong orNone (cong typeOfᴹᴼ p)))) typeOf-val-not-none {H = H} (addr a) | (nothing , p) = warning (UnallocatedAddress p) @@ -153,6 +155,7 @@ binOpPreservation H (<= m n) = refl binOpPreservation H (>= m n) = refl binOpPreservation H (== v w) = refl binOpPreservation H (~= v w) = refl +binOpPreservation H (·· v w) = refl preservationᴱ : ∀ H M {H′ M′} → (H ⊢ M ⟶ᴱ M′ ⊣ H′) → OrWarningᴴᴱ H (typeCheckᴴᴱ H ∅ M) (typeOfᴱ H ∅ M ≡ typeOfᴱ H′ ∅ M′) preservationᴮ : ∀ H B {H′ B′} → (H ⊢ B ⟶ᴮ B′ ⊣ H′) → OrWarningᴴᴮ H (typeCheckᴴᴮ H ∅ B) (typeOfᴮ H ∅ B ≡ typeOfᴮ H′ ∅ B′) @@ -444,6 +447,7 @@ runtimeBinOpWarning H v (< p) = < (λ q → p (mustBeNumber H ∅ v q)) runtimeBinOpWarning H v (> p) = > (λ q → p (mustBeNumber H ∅ v q)) runtimeBinOpWarning H v (<= p) = <= (λ q → p (mustBeNumber H ∅ v q)) runtimeBinOpWarning H v (>= p) = >= (λ q → p (mustBeNumber H ∅ v q)) +runtimeBinOpWarning H v (·· p) = ·· (λ q → p (mustBeString H ∅ v q)) runtimeWarningᴱ : ∀ H M → RuntimeErrorᴱ H M → Warningᴱ H (typeCheckᴱ H ∅ M) runtimeWarningᴮ : ∀ H B → RuntimeErrorᴮ H B → Warningᴮ H (typeCheckᴮ H ∅ B) diff --git a/prototyping/Properties/TypeCheck.agda b/prototyping/Properties/TypeCheck.agda index c144bc12..bec68841 100644 --- a/prototyping/Properties/TypeCheck.agda +++ b/prototyping/Properties/TypeCheck.agda @@ -8,10 +8,10 @@ open import Agda.Builtin.Equality using (_≡_; refl) open import Agda.Builtin.Bool using (Bool; true; false) open import FFI.Data.Maybe using (Maybe; just; nothing) open import FFI.Data.Either using (Either) -open import Luau.TypeCheck(m) using (_⊢ᴱ_∈_; _⊢ᴮ_∈_; ⊢ᴼ_; ⊢ᴴ_; _⊢ᴴᴱ_▷_∈_; _⊢ᴴᴮ_▷_∈_; nil; var; addr; number; bool; app; function; block; binexp; done; return; local; nothing; orNone; tgtBinOp) -open import Luau.Syntax using (Block; Expr; Value; BinaryOperator; yes; nil; addr; number; bool; val; var; binexp; _$_; function_is_end; block_is_end; _∙_; return; done; local_←_; _⟨_⟩; _⟨_⟩∈_; var_∈_; name; fun; arg; +; -; *; /; <; >; ==; ~=; <=; >=) -open import Luau.Type using (Type; nil; any; none; number; boolean; _⇒_; tgt) -open import Luau.RuntimeType using (RuntimeType; nil; number; function; valueType) +open import Luau.TypeCheck(m) using (_⊢ᴱ_∈_; _⊢ᴮ_∈_; ⊢ᴼ_; ⊢ᴴ_; _⊢ᴴᴱ_▷_∈_; _⊢ᴴᴮ_▷_∈_; nil; var; addr; number; bool; string; app; function; block; binexp; done; return; local; nothing; orNone; tgtBinOp) +open import Luau.Syntax using (Block; Expr; Value; BinaryOperator; yes; nil; addr; number; bool; string; val; var; binexp; _$_; function_is_end; block_is_end; _∙_; return; done; local_←_; _⟨_⟩; _⟨_⟩∈_; var_∈_; name; fun; arg; +; -; *; /; <; >; ==; ~=; <=; >=) +open import Luau.Type using (Type; nil; any; none; number; boolean; string; _⇒_; tgt) +open import Luau.RuntimeType using (RuntimeType; nil; number; function; string; valueType) open import Luau.VarCtxt using (VarCtxt; ∅; _↦_; _⊕_↦_; _⋒_; _⊝_) renaming (_[_] to _[_]ⱽ) open import Luau.Addr using (Addr) open import Luau.Var using (Var; _≡ⱽ_) @@ -37,6 +37,7 @@ typeOfⱽ H nil = just nil typeOfⱽ H (bool b) = just boolean typeOfⱽ H (addr a) = typeOfᴹᴼ (H [ a ]ᴴ) typeOfⱽ H (number n) = just number +typeOfⱽ H (string x) = just string typeOfᴱ : Heap yes → VarCtxt → (Expr yes) → Type typeOfᴮ : Heap yes → VarCtxt → (Block yes) → Type @@ -59,17 +60,23 @@ mustBeFunction H Γ (addr a) p = refl mustBeFunction H Γ (number n) p = CONTRADICTION (p refl) mustBeFunction H Γ (bool true) p = CONTRADICTION (p refl) mustBeFunction H Γ (bool false) p = CONTRADICTION (p refl) +mustBeFunction H Γ (string x) p = CONTRADICTION (p refl) mustBeNumber : ∀ H Γ v → (typeOfᴱ H Γ (val v) ≡ number) → (valueType(v) ≡ number) -mustBeNumber H Γ nil () mustBeNumber H Γ (addr a) p with remember (H [ a ]ᴴ) mustBeNumber H Γ (addr a) p | (just O , q) with trans (cong orNone (cong typeOfᴹᴼ (sym q))) p mustBeNumber H Γ (addr a) p | (just function f ⟨ var x ∈ T ⟩∈ U is B end , q) | () mustBeNumber H Γ (addr a) p | (nothing , q) with trans (cong orNone (cong typeOfᴹᴼ (sym q))) p mustBeNumber H Γ (addr a) p | nothing , q | () mustBeNumber H Γ (number n) p = refl -mustBeNumber H Γ (bool true) () -mustBeNumber H Γ (bool false) () + +mustBeString : ∀ H Γ v → (typeOfᴱ H Γ (val v) ≡ string) → (valueType(v) ≡ string) +mustBeString H Γ (addr a) p with remember (H [ a ]ᴴ) +mustBeString H Γ (addr a) p | (just O , q) with trans (cong orNone (cong typeOfᴹᴼ (sym q))) p +mustBeString H Γ (addr a) p | (just function f ⟨ var x ∈ T ⟩∈ U is B end , q) | () +mustBeString H Γ (addr a) p | (nothing , q) with trans (cong orNone (cong typeOfᴹᴼ (sym q))) p +mustBeString H Γ (addr a) p | (nothing , q) | () +mustBeString H Γ (string x) p = refl typeCheckᴱ : ∀ H Γ M → (Γ ⊢ᴱ M ∈ (typeOfᴱ H Γ M)) typeCheckᴮ : ∀ H Γ B → (Γ ⊢ᴮ B ∈ (typeOfᴮ H Γ B)) @@ -79,6 +86,7 @@ typeCheckᴱ H Γ (val nil) = nil typeCheckᴱ H Γ (val (addr a)) = addr (orNone (typeOfᴹᴼ (H [ a ]ᴴ))) typeCheckᴱ H Γ (val (number n)) = number typeCheckᴱ H Γ (val (bool b)) = bool +typeCheckᴱ H Γ (val (string x)) = string typeCheckᴱ H Γ (M $ N) = app (typeCheckᴱ H Γ M) (typeCheckᴱ H Γ N) typeCheckᴱ H Γ (function f ⟨ var x ∈ T ⟩∈ U is B end) = function (typeCheckᴮ H (Γ ⊕ x ↦ T) B) typeCheckᴱ H Γ (block var b ∈ T is B end) = block (typeCheckᴮ H Γ B) @@ -101,3 +109,4 @@ typeCheckᴴᴱ H Γ M = (typeCheckᴴ H , typeCheckᴱ H Γ M) typeCheckᴴᴮ : ∀ H Γ M → (Γ ⊢ᴴᴮ H ▷ M ∈ typeOfᴮ H Γ M) typeCheckᴴᴮ H Γ M = (typeCheckᴴ H , typeCheckᴮ H Γ M) + \ No newline at end of file diff --git a/prototyping/Tests/Interpreter/concat_number_and_string/in.lua b/prototyping/Tests/Interpreter/concat_number_and_string/in.lua new file mode 100644 index 00000000..ba5acb32 --- /dev/null +++ b/prototyping/Tests/Interpreter/concat_number_and_string/in.lua @@ -0,0 +1,3 @@ +local x: string = "hello" +local y: string = 37 +return x .. y diff --git a/prototyping/Tests/Interpreter/concat_number_and_string/out.txt b/prototyping/Tests/Interpreter/concat_number_and_string/out.txt new file mode 100644 index 00000000..a437438d --- /dev/null +++ b/prototyping/Tests/Interpreter/concat_number_and_string/out.txt @@ -0,0 +1,11 @@ +ANNOTATED PROGRAM: +local x : string = "hello" +local y : string = 37.0 +return x .. y + +RUNTIME ERROR: +value 37.0 is not a string + in return statement + +TYPE ERROR: +Local variable y has type string but expression has type number diff --git a/prototyping/Tests/Interpreter/concat_two_strings/in.lua b/prototyping/Tests/Interpreter/concat_two_strings/in.lua new file mode 100644 index 00000000..c6ccdce6 --- /dev/null +++ b/prototyping/Tests/Interpreter/concat_two_strings/in.lua @@ -0,0 +1,3 @@ +local x: string = "hello" +local y: string = "world" +return x .. y diff --git a/prototyping/Tests/Interpreter/concat_two_strings/out.txt b/prototyping/Tests/Interpreter/concat_two_strings/out.txt new file mode 100644 index 00000000..d45f589f --- /dev/null +++ b/prototyping/Tests/Interpreter/concat_two_strings/out.txt @@ -0,0 +1,6 @@ +ANNOTATED PROGRAM: +local x : string = "hello" +local y : string = "world" +return x .. y + +RAN WITH RESULT: "helloworld" diff --git a/prototyping/Tests/Interpreter/return_string/in.lua b/prototyping/Tests/Interpreter/return_string/in.lua new file mode 100644 index 00000000..67febb75 --- /dev/null +++ b/prototyping/Tests/Interpreter/return_string/in.lua @@ -0,0 +1 @@ +return "foo bar" diff --git a/prototyping/Tests/Interpreter/return_string/out.txt b/prototyping/Tests/Interpreter/return_string/out.txt new file mode 100644 index 00000000..81915c77 --- /dev/null +++ b/prototyping/Tests/Interpreter/return_string/out.txt @@ -0,0 +1,4 @@ +ANNOTATED PROGRAM: +return "foo bar" + +RAN WITH RESULT: "foo bar" From 9bfecab5baf796c0b358a88fbc8ca8d70af2da12 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Fri, 4 Mar 2022 08:19:20 -0800 Subject: [PATCH 189/399] Sync to upstream/release/517 --- Analysis/include/Luau/TxnLog.h | 32 ++- Analysis/include/Luau/TypeInfer.h | 18 -- Analysis/include/Luau/TypeVar.h | 5 +- Analysis/include/Luau/Unifier.h | 2 +- Analysis/src/Autocomplete.cpp | 66 ++--- Analysis/src/BuiltinDefinitions.cpp | 13 +- Analysis/src/EmbeddedBuiltinDefinitions.cpp | 1 - Analysis/src/JsonEncoder.cpp | 44 +-- Analysis/src/Linter.cpp | 157 +++++++++- Analysis/src/Module.cpp | 5 +- Analysis/src/ToString.cpp | 119 +------- Analysis/src/Transpiler.cpp | 59 ++-- Analysis/src/TxnLog.cpp | 84 +++++- Analysis/src/TypeInfer.cpp | 301 ++++++++------------ Analysis/src/TypeVar.cpp | 27 +- Analysis/src/Unifier.cpp | 137 +++++++-- Ast/src/Parser.cpp | 3 +- CLI/Repl.cpp | 35 +-- VM/include/lua.h | 3 + VM/src/lapi.cpp | 52 +++- VM/src/laux.cpp | 46 +-- VM/src/ldebug.cpp | 5 + VM/src/lgc.cpp | 74 ++++- VM/src/lgc.h | 7 - VM/src/lnumprint.cpp | 9 - VM/src/lnumutils.h | 1 - VM/src/lstate.h | 23 +- VM/src/ltable.cpp | 3 +- VM/src/ltablib.cpp | 21 ++ bench/gc/test_GC_Boehm_Trees.lua | 3 + bench/gc/test_GC_Tree_Pruning_Eager.lua | 2 +- bench/gc/test_GC_Tree_Pruning_Gen.lua | 2 +- bench/gc/test_GC_Tree_Pruning_Lazy.lua | 2 +- extern/isocline/include/isocline.h | 2 +- fuzz/number.cpp | 4 - fuzz/proto.cpp | 2 +- tests/Autocomplete.test.cpp | 10 +- tests/Conformance.test.cpp | 6 +- tests/Linter.test.cpp | 107 +++++++ tests/Parser.test.cpp | 6 - tests/ToDot.test.cpp | 2 - tests/ToString.test.cpp | 2 - tests/Transpiler.test.cpp | 3 - tests/TypeInfer.aliases.test.cpp | 3 +- tests/TypeInfer.builtins.test.cpp | 27 ++ tests/TypeInfer.generics.test.cpp | 89 ++++++ tests/TypeInfer.provisional.test.cpp | 95 +----- tests/TypeInfer.refinements.test.cpp | 11 +- tests/TypeInfer.singletons.test.cpp | 44 +-- tests/TypeInfer.tables.test.cpp | 81 +++++- tests/TypeInfer.test.cpp | 71 +++++ tests/TypeInfer.tryUnify.test.cpp | 7 +- tests/TypeInfer.typePacks.cpp | 39 --- tests/TypeInfer.unionTypes.test.cpp | 31 +- tests/conformance/nextvar.lua | 38 +++ 55 files changed, 1267 insertions(+), 774 deletions(-) diff --git a/Analysis/include/Luau/TxnLog.h b/Analysis/include/Luau/TxnLog.h index f238e258..f8105383 100644 --- a/Analysis/include/Luau/TxnLog.h +++ b/Analysis/include/Luau/TxnLog.h @@ -12,6 +12,8 @@ LUAU_FASTFLAG(LuauShareTxnSeen); namespace Luau { +using TypeOrPackId = const void*; + // Log of where what TypeIds we are rebinding and what they used to be // Remove with LuauUseCommitTxnLog struct DEPRECATED_TxnLog @@ -23,7 +25,7 @@ struct DEPRECATED_TxnLog { } - explicit DEPRECATED_TxnLog(std::vector>* sharedSeen) + explicit DEPRECATED_TxnLog(std::vector>* sharedSeen) : originalSeenSize(sharedSeen->size()) , ownedSeen() , sharedSeen(sharedSeen) @@ -48,15 +50,23 @@ struct DEPRECATED_TxnLog void pushSeen(TypeId lhs, TypeId rhs); void popSeen(TypeId lhs, TypeId rhs); + bool haveSeen(TypePackId lhs, TypePackId rhs); + void pushSeen(TypePackId lhs, TypePackId rhs); + void popSeen(TypePackId lhs, TypePackId rhs); + private: std::vector> typeVarChanges; std::vector> typePackChanges; std::vector>> tableChanges; size_t originalSeenSize; + bool haveSeen(TypeOrPackId lhs, TypeOrPackId rhs); + void pushSeen(TypeOrPackId lhs, TypeOrPackId rhs); + void popSeen(TypeOrPackId lhs, TypeOrPackId rhs); + public: - std::vector> ownedSeen; // used to avoid infinite recursion when types are cyclic - std::vector>* sharedSeen; // shared with all the descendent logs + std::vector> ownedSeen; // used to avoid infinite recursion when types are cyclic + std::vector>* sharedSeen; // shared with all the descendent logs }; // Pending state for a TypeVar. Generated by a TxnLog and committed via @@ -127,12 +137,12 @@ struct TxnLog } } - explicit TxnLog(std::vector>* sharedSeen) + explicit TxnLog(std::vector>* sharedSeen) : sharedSeen(sharedSeen) { } - TxnLog(TxnLog* parent, std::vector>* sharedSeen) + TxnLog(TxnLog* parent, std::vector>* sharedSeen) : parent(parent) , sharedSeen(sharedSeen) { @@ -173,6 +183,10 @@ struct TxnLog void pushSeen(TypeId lhs, TypeId rhs); void popSeen(TypeId lhs, TypeId rhs); + bool haveSeen(TypePackId lhs, TypePackId rhs) const; + void pushSeen(TypePackId lhs, TypePackId rhs); + void popSeen(TypePackId lhs, TypePackId rhs); + // Queues a type for modification. The original type will not change until commit // is called. Use pending to get the pending state. // @@ -316,12 +330,16 @@ private: // TxnLogs; use sharedSeen instead. This field exists because in the tree // of TxnLogs, the root must own its seen set. In all descendant TxnLogs, // this is an empty vector. - std::vector> ownedSeen; + std::vector> ownedSeen; + + bool haveSeen(TypeOrPackId lhs, TypeOrPackId rhs) const; + void pushSeen(TypeOrPackId lhs, TypeOrPackId rhs); + void popSeen(TypeOrPackId lhs, TypeOrPackId rhs); public: // Used to avoid infinite recursion when types are cyclic. // Shared with all the descendent TxnLogs. - std::vector>* sharedSeen; + std::vector>* sharedSeen; }; } // namespace Luau diff --git a/Analysis/include/Luau/TypeInfer.h b/Analysis/include/Luau/TypeInfer.h index 2440c810..839043cc 100644 --- a/Analysis/include/Luau/TypeInfer.h +++ b/Analysis/include/Luau/TypeInfer.h @@ -73,24 +73,6 @@ struct Instantiation : Substitution TypePackId clean(TypePackId tp) override; }; -// A substitution which replaces free types by generic types. -struct Quantification : Substitution -{ - Quantification(TypeArena* arena, TypeLevel level) - : Substitution(TxnLog::empty(), arena) - , level(level) - { - } - - TypeLevel level; - std::vector generics; - std::vector genericPacks; - bool isDirty(TypeId ty) override; - bool isDirty(TypePackId tp) override; - TypeId clean(TypeId ty) override; - TypePackId clean(TypePackId tp) override; -}; - // A substitution which replaces free types by any struct Anyification : Substitution { diff --git a/Analysis/include/Luau/TypeVar.h b/Analysis/include/Luau/TypeVar.h index 8d1a9fa6..29578dcd 100644 --- a/Analysis/include/Luau/TypeVar.h +++ b/Analysis/include/Luau/TypeVar.h @@ -298,7 +298,7 @@ struct TableTypeVar TableTypeVar() = default; explicit TableTypeVar(TableState state, TypeLevel level); - TableTypeVar(const Props& props, const std::optional& indexer, TypeLevel level, TableState state = TableState::Unsealed); + TableTypeVar(const Props& props, const std::optional& indexer, TypeLevel level, TableState state); Props props; std::optional indexer; @@ -477,6 +477,9 @@ bool isOptional(TypeId ty); bool isTableIntersection(TypeId ty); bool isOverloadedFunction(TypeId ty); +// True when string is a subtype of ty +bool maybeString(TypeId ty); + std::optional getMetatable(TypeId type); TableTypeVar* getMutableTableType(TypeId type); const TableTypeVar* getTableType(TypeId type); diff --git a/Analysis/include/Luau/Unifier.h b/Analysis/include/Luau/Unifier.h index fe822b01..4c0462fe 100644 --- a/Analysis/include/Luau/Unifier.h +++ b/Analysis/include/Luau/Unifier.h @@ -56,7 +56,7 @@ struct Unifier Unifier(TypeArena* types, Mode mode, const Location& location, Variance variance, UnifierSharedState& sharedState, TxnLog* parentLog = nullptr); - Unifier(TypeArena* types, Mode mode, std::vector>* sharedSeen, const Location& location, + Unifier(TypeArena* types, Mode mode, std::vector>* sharedSeen, const Location& location, Variance variance, UnifierSharedState& sharedState, TxnLog* parentLog = nullptr); // Test whether the two type vars unify. Never commits the result. diff --git a/Analysis/src/Autocomplete.cpp b/Analysis/src/Autocomplete.cpp index 29a2c6b5..c3de8d0e 100644 --- a/Analysis/src/Autocomplete.cpp +++ b/Analysis/src/Autocomplete.cpp @@ -16,7 +16,6 @@ LUAU_FASTFLAG(LuauUseCommittingTxnLog) LUAU_FASTFLAGVARIABLE(LuauAutocompleteAvoidMutation, false); LUAU_FASTFLAGVARIABLE(LuauMissingFollowACMetatables, false); -LUAU_FASTFLAGVARIABLE(PreferToCallFunctionsForIntersects, false); LUAU_FASTFLAGVARIABLE(LuauIfElseExprFixCompletionIssue, false); static const std::unordered_set kStatementStartingKeywords = { @@ -272,55 +271,34 @@ static TypeCorrectKind checkTypeCorrectKind(const Module& module, TypeArena* typ TypeId expectedType = follow(*typeAtPosition); - if (FFlag::PreferToCallFunctionsForIntersects) - { - auto checkFunctionType = [&canUnify, &expectedType](const FunctionTypeVar* ftv) { - auto [retHead, retTail] = flatten(ftv->retType); + auto checkFunctionType = [&canUnify, &expectedType](const FunctionTypeVar* ftv) { + auto [retHead, retTail] = flatten(ftv->retType); - if (!retHead.empty() && canUnify(retHead.front(), expectedType)) + if (!retHead.empty() && canUnify(retHead.front(), expectedType)) + return true; + + // We might only have a variadic tail pack, check if the element is compatible + if (retTail) + { + if (const VariadicTypePack* vtp = get(follow(*retTail)); vtp && canUnify(vtp->ty, expectedType)) return true; - - // We might only have a variadic tail pack, check if the element is compatible - if (retTail) - { - if (const VariadicTypePack* vtp = get(follow(*retTail)); vtp && canUnify(vtp->ty, expectedType)) - return true; - } - - return false; - }; - - // We also want to suggest functions that return compatible result - if (const FunctionTypeVar* ftv = get(ty); ftv && checkFunctionType(ftv)) - { - return TypeCorrectKind::CorrectFunctionResult; } - else if (const IntersectionTypeVar* itv = get(ty)) - { - for (TypeId id : itv->parts) - { - if (const FunctionTypeVar* ftv = get(id); ftv && checkFunctionType(ftv)) - { - return TypeCorrectKind::CorrectFunctionResult; - } - } - } - } - else + + return false; + }; + + // We also want to suggest functions that return compatible result + if (const FunctionTypeVar* ftv = get(ty); ftv && checkFunctionType(ftv)) { - // We also want to suggest functions that return compatible result - if (const FunctionTypeVar* ftv = get(ty)) + return TypeCorrectKind::CorrectFunctionResult; + } + else if (const IntersectionTypeVar* itv = get(ty)) + { + for (TypeId id : itv->parts) { - auto [retHead, retTail] = flatten(ftv->retType); - - if (!retHead.empty() && canUnify(retHead.front(), expectedType)) - return TypeCorrectKind::CorrectFunctionResult; - - // We might only have a variadic tail pack, check if the element is compatible - if (retTail) + if (const FunctionTypeVar* ftv = get(id); ftv && checkFunctionType(ftv)) { - if (const VariadicTypePack* vtp = get(follow(*retTail)); vtp && canUnify(vtp->ty, expectedType)) - return TypeCorrectKind::CorrectFunctionResult; + return TypeCorrectKind::CorrectFunctionResult; } } } diff --git a/Analysis/src/BuiltinDefinitions.cpp b/Analysis/src/BuiltinDefinitions.cpp index d72422a5..e4e5dab8 100644 --- a/Analysis/src/BuiltinDefinitions.cpp +++ b/Analysis/src/BuiltinDefinitions.cpp @@ -9,6 +9,7 @@ #include LUAU_FASTFLAG(LuauAssertStripsFalsyTypes) +LUAU_FASTFLAGVARIABLE(LuauTableCloneType, false) /** FIXME: Many of these type definitions are not quite completely accurate. * @@ -283,8 +284,16 @@ void registerBuiltinTypes(TypeChecker& typeChecker) attachMagicFunction(getGlobalBinding(typeChecker, "setmetatable"), magicFunctionSetMetaTable); attachMagicFunction(getGlobalBinding(typeChecker, "select"), magicFunctionSelect); - auto tableLib = getMutable(getGlobalBinding(typeChecker, "table")); - attachMagicFunction(tableLib->props["pack"].type, magicFunctionPack); + if (TableTypeVar* ttv = getMutable(getGlobalBinding(typeChecker, "table"))) + { + // tabTy is a generic table type which we can't express via declaration syntax yet + ttv->props["freeze"] = makeProperty(makeFunction(arena, std::nullopt, {tabTy}, {tabTy}), "@luau/global/table.freeze"); + + if (FFlag::LuauTableCloneType) + ttv->props["clone"] = makeProperty(makeFunction(arena, std::nullopt, {tabTy}, {tabTy}), "@luau/global/table.clone"); + + attachMagicFunction(ttv->props["pack"].type, magicFunctionPack); + } attachMagicFunction(getGlobalBinding(typeChecker, "require"), magicFunctionRequire); } diff --git a/Analysis/src/EmbeddedBuiltinDefinitions.cpp b/Analysis/src/EmbeddedBuiltinDefinitions.cpp index bf6e1193..471b61ad 100644 --- a/Analysis/src/EmbeddedBuiltinDefinitions.cpp +++ b/Analysis/src/EmbeddedBuiltinDefinitions.cpp @@ -170,7 +170,6 @@ declare function gcinfo(): number move: ({V}, number, number, number, {V}?) -> {V}, clear: ({[K]: V}) -> (), - freeze: ({[K]: V}) -> {[K]: V}, isfrozen: ({[K]: V}) -> boolean, } diff --git a/Analysis/src/JsonEncoder.cpp b/Analysis/src/JsonEncoder.cpp index ec399158..811e7c24 100644 --- a/Analysis/src/JsonEncoder.cpp +++ b/Analysis/src/JsonEncoder.cpp @@ -5,8 +5,6 @@ #include "Luau/StringUtils.h" #include "Luau/Common.h" -LUAU_FASTFLAG(LuauTypeAliasDefaults) - namespace Luau { @@ -369,38 +367,24 @@ struct AstJsonEncoder : public AstVisitor void write(const AstGenericType& genericType) { - if (FFlag::LuauTypeAliasDefaults) - { - writeRaw("{"); - bool c = pushComma(); - write("name", genericType.name); - if (genericType.defaultValue) - write("type", genericType.defaultValue); - popComma(c); - writeRaw("}"); - } - else - { - write(genericType.name); - } + writeRaw("{"); + bool c = pushComma(); + write("name", genericType.name); + if (genericType.defaultValue) + write("type", genericType.defaultValue); + popComma(c); + writeRaw("}"); } void write(const AstGenericTypePack& genericTypePack) { - if (FFlag::LuauTypeAliasDefaults) - { - writeRaw("{"); - bool c = pushComma(); - write("name", genericTypePack.name); - if (genericTypePack.defaultValue) - write("type", genericTypePack.defaultValue); - popComma(c); - writeRaw("}"); - } - else - { - write(genericTypePack.name); - } + writeRaw("{"); + bool c = pushComma(); + write("name", genericTypePack.name); + if (genericTypePack.defaultValue) + write("type", genericTypePack.defaultValue); + popComma(c); + writeRaw("}"); } void write(AstExprTable::Item::Kind kind) diff --git a/Analysis/src/Linter.cpp b/Analysis/src/Linter.cpp index 7635dc0f..56c4e3e8 100644 --- a/Analysis/src/Linter.cpp +++ b/Analysis/src/Linter.cpp @@ -13,6 +13,7 @@ #include LUAU_FASTINTVARIABLE(LuauSuggestionDistance, 4) +LUAU_FASTFLAGVARIABLE(LuauLintGlobalNeverReadBeforeWritten, false) namespace Luau { @@ -233,6 +234,20 @@ public: } private: + struct FunctionInfo + { + explicit FunctionInfo(AstExprFunction* ast) + : ast(ast) + , dominatedGlobals({}) + , conditionalExecution(false) + { + } + + AstExprFunction* ast; + DenseHashSet dominatedGlobals; + bool conditionalExecution; + }; + struct Global { AstExprGlobal* firstRef = nullptr; @@ -241,6 +256,9 @@ private: bool assigned = false; bool builtin = false; + bool definedInModuleScope = false; + bool definedAsFunction = false; + bool readBeforeWritten = false; std::optional deprecated; }; @@ -248,7 +266,8 @@ private: DenseHashMap globals; std::vector globalRefs; - std::vector functionStack; + std::vector functionStack; + LintGlobalLocal() : globals(AstName()) @@ -291,12 +310,18 @@ private: "Global '%s' is only used in the enclosing function defined at line %d; consider changing it to local", g.firstRef->name.value, top->location.begin.line + 1); } + else if (FFlag::LuauLintGlobalNeverReadBeforeWritten && g.assigned && !g.readBeforeWritten && !g.definedInModuleScope && + g.firstRef->name != context->placeholder) + { + emitWarning(*context, LintWarning::Code_GlobalUsedAsLocal, g.firstRef->location, + "Global '%s' is never read before being written. Consider changing it to local", g.firstRef->name.value); + } } } bool visit(AstExprFunction* node) override { - functionStack.push_back(node); + functionStack.emplace_back(node); node->body->visit(this); @@ -307,6 +332,11 @@ private: bool visit(AstExprGlobal* node) override { + if (FFlag::LuauLintGlobalNeverReadBeforeWritten && !functionStack.empty() && !functionStack.back().dominatedGlobals.contains(node->name)) + { + Global& g = globals[node->name]; + g.readBeforeWritten = true; + } trackGlobalRef(node); if (node->name == context->placeholder) @@ -335,6 +365,21 @@ private: { Global& g = globals[gv->name]; + if (FFlag::LuauLintGlobalNeverReadBeforeWritten) + { + if (functionStack.empty()) + { + g.definedInModuleScope = true; + } + else + { + if (!functionStack.back().conditionalExecution) + { + functionStack.back().dominatedGlobals.insert(gv->name); + } + } + } + if (g.builtin) emitWarning(*context, LintWarning::Code_BuiltinGlobalWrite, gv->location, "Built-in global '%s' is overwritten here; consider using a local or changing the name", gv->name.value); @@ -369,7 +414,14 @@ private: emitWarning(*context, LintWarning::Code_BuiltinGlobalWrite, gv->location, "Built-in global '%s' is overwritten here; consider using a local or changing the name", gv->name.value); else + { g.assigned = true; + if (FFlag::LuauLintGlobalNeverReadBeforeWritten) + { + g.definedAsFunction = true; + g.definedInModuleScope = functionStack.empty(); + } + } trackGlobalRef(gv); } @@ -377,6 +429,98 @@ private: return true; } + class HoldConditionalExecution + { + public: + HoldConditionalExecution(LintGlobalLocal& p) + : p(p) + { + if (!p.functionStack.empty() && !p.functionStack.back().conditionalExecution) + { + resetToFalse = true; + p.functionStack.back().conditionalExecution = true; + } + } + ~HoldConditionalExecution() + { + if (resetToFalse) + p.functionStack.back().conditionalExecution = false; + } + + private: + bool resetToFalse = false; + LintGlobalLocal& p; + }; + + bool visit(AstStatIf* node) override + { + if (!FFlag::LuauLintGlobalNeverReadBeforeWritten) + return true; + + HoldConditionalExecution ce(*this); + node->condition->visit(this); + node->thenbody->visit(this); + if (node->elsebody) + node->elsebody->visit(this); + + return false; + } + + bool visit(AstStatWhile* node) override + { + if (!FFlag::LuauLintGlobalNeverReadBeforeWritten) + return true; + + HoldConditionalExecution ce(*this); + node->condition->visit(this); + node->body->visit(this); + + return false; + } + + bool visit(AstStatRepeat* node) override + { + if (!FFlag::LuauLintGlobalNeverReadBeforeWritten) + return true; + + HoldConditionalExecution ce(*this); + node->condition->visit(this); + node->body->visit(this); + + return false; + } + + bool visit(AstStatFor* node) override + { + if (!FFlag::LuauLintGlobalNeverReadBeforeWritten) + return true; + + HoldConditionalExecution ce(*this); + node->from->visit(this); + node->to->visit(this); + + if (node->step) + node->step->visit(this); + + node->body->visit(this); + + return false; + } + + bool visit(AstStatForIn* node) override + { + if (!FFlag::LuauLintGlobalNeverReadBeforeWritten) + return true; + + HoldConditionalExecution ce(*this); + for (AstExpr* expr : node->values) + expr->visit(this); + + node->body->visit(this); + + return false; + } + void trackGlobalRef(AstExprGlobal* node) { Global& g = globals[node->name]; @@ -390,7 +534,12 @@ private: // to reduce the cost of tracking we only track this for user globals if (!g.builtin) { - g.functionRef = functionStack; + g.functionRef.clear(); + g.functionRef.reserve(functionStack.size()); + for (const FunctionInfo& entry : functionStack) + { + g.functionRef.push_back(entry.ast); + } } } else @@ -401,7 +550,7 @@ private: // we need to find a common prefix between all uses of a global size_t prefix = 0; - while (prefix < g.functionRef.size() && prefix < functionStack.size() && g.functionRef[prefix] == functionStack[prefix]) + while (prefix < g.functionRef.size() && prefix < functionStack.size() && g.functionRef[prefix] == functionStack[prefix].ast) prefix++; g.functionRef.resize(prefix); diff --git a/Analysis/src/Module.cpp b/Analysis/src/Module.cpp index 412b78bb..76dc72d2 100644 --- a/Analysis/src/Module.cpp +++ b/Analysis/src/Module.cpp @@ -14,7 +14,6 @@ LUAU_FASTFLAGVARIABLE(DebugLuauFreezeArena, false) LUAU_FASTFLAGVARIABLE(DebugLuauTrackOwningArena, false) // Remove with FFlagLuauImmutableTypes LUAU_FASTINTVARIABLE(LuauTypeCloneRecursionLimit, 300) -LUAU_FASTFLAG(LuauTypeAliasDefaults) LUAU_FASTFLAG(LuauImmutableTypes) namespace Luau @@ -463,7 +462,7 @@ TypeFun clone(const TypeFun& typeFun, TypeArena& dest, SeenTypes& seenTypes, See TypeId ty = clone(param.ty, dest, seenTypes, seenTypePacks, cloneState); std::optional defaultValue; - if (FFlag::LuauTypeAliasDefaults && param.defaultValue) + if (param.defaultValue) defaultValue = clone(*param.defaultValue, dest, seenTypes, seenTypePacks, cloneState); result.typeParams.push_back({ty, defaultValue}); @@ -474,7 +473,7 @@ TypeFun clone(const TypeFun& typeFun, TypeArena& dest, SeenTypes& seenTypes, See TypePackId tp = clone(param.tp, dest, seenTypes, seenTypePacks, cloneState); std::optional defaultValue; - if (FFlag::LuauTypeAliasDefaults && param.defaultValue) + if (param.defaultValue) defaultValue = clone(*param.defaultValue, dest, seenTypes, seenTypePacks, cloneState); result.typePackParams.push_back({tp, defaultValue}); diff --git a/Analysis/src/ToString.cpp b/Analysis/src/ToString.cpp index 5e79b841..010ca361 100644 --- a/Analysis/src/ToString.cpp +++ b/Analysis/src/ToString.cpp @@ -10,8 +10,6 @@ #include #include -LUAU_FASTFLAG(LuauTypeAliasDefaults) - /* * Prefix generic typenames with gen- * Additionally, free types will be prefixed with free- and suffixed with their level. eg free-a-4 @@ -288,28 +286,15 @@ struct TypeVarStringifier else first = false; - if (FFlag::LuauTypeAliasDefaults) - { - bool wrap = !singleTp && get(follow(tp)); + bool wrap = !singleTp && get(follow(tp)); - if (wrap) - state.emit("("); + if (wrap) + state.emit("("); - stringify(tp); + stringify(tp); - if (wrap) - state.emit(")"); - } - else - { - if (!singleTp) - state.emit("("); - - stringify(tp); - - if (!singleTp) - state.emit(")"); - } + if (wrap) + state.emit(")"); } if (types.size() || typePacks.size()) @@ -1105,100 +1090,8 @@ std::string toString(const TypePackVar& tp, const ToStringOptions& opts) return toString(const_cast(&tp), std::move(opts)); } -std::string toStringNamedFunction_DEPRECATED(const std::string& prefix, const FunctionTypeVar& ftv, ToStringOptions opts) -{ - std::string s = prefix; - - auto toString_ = [&opts](TypeId ty) -> std::string { - ToStringResult res = toStringDetailed(ty, opts); - opts.nameMap = std::move(res.nameMap); - return res.name; - }; - - auto toStringPack_ = [&opts](TypePackId ty) -> std::string { - ToStringResult res = toStringDetailed(ty, opts); - opts.nameMap = std::move(res.nameMap); - return res.name; - }; - - if (!opts.hideNamedFunctionTypeParameters && (!ftv.generics.empty() || !ftv.genericPacks.empty())) - { - s += "<"; - - bool first = true; - for (TypeId g : ftv.generics) - { - if (!first) - s += ", "; - first = false; - s += toString_(g); - } - - for (TypePackId gp : ftv.genericPacks) - { - if (!first) - s += ", "; - first = false; - s += toStringPack_(gp); - } - - s += ">"; - } - - s += "("; - - auto argPackIter = begin(ftv.argTypes); - auto argNameIter = ftv.argNames.begin(); - - bool first = true; - while (argPackIter != end(ftv.argTypes)) - { - if (!first) - s += ", "; - first = false; - - // We don't currently respect opts.functionTypeArguments. I don't think this function should. - if (argNameIter != ftv.argNames.end()) - { - s += (*argNameIter ? (*argNameIter)->name : "_") + ": "; - ++argNameIter; - } - else - { - s += "_: "; - } - - s += toString_(*argPackIter); - ++argPackIter; - } - - if (argPackIter.tail()) - { - if (auto vtp = get(*argPackIter.tail())) - s += ", ...: " + toString_(vtp->ty); - else - s += ", ...: " + toStringPack_(*argPackIter.tail()); - } - - s += "): "; - - size_t retSize = size(ftv.retType); - bool hasTail = !finite(ftv.retType); - if (retSize == 0 && !hasTail) - s += "()"; - else if ((retSize == 0 && hasTail) || (retSize == 1 && !hasTail)) - s += toStringPack_(ftv.retType); - else - s += "(" + toStringPack_(ftv.retType) + ")"; - - return s; -} - std::string toStringNamedFunction(const std::string& prefix, const FunctionTypeVar& ftv, ToStringOptions opts) { - if (!FFlag::LuauTypeAliasDefaults) - return toStringNamedFunction_DEPRECATED(prefix, ftv, opts); - ToStringResult result; StringifierState state(opts, result, opts.nameMap); TypeVarStringifier tvs{state}; diff --git a/Analysis/src/Transpiler.cpp b/Analysis/src/Transpiler.cpp index a02d396b..92ed241e 100644 --- a/Analysis/src/Transpiler.cpp +++ b/Analysis/src/Transpiler.cpp @@ -10,8 +10,6 @@ #include #include -LUAU_FASTFLAG(LuauTypeAliasDefaults) - namespace { bool isIdentifierStartChar(char c) @@ -796,21 +794,14 @@ struct Printer { comma(); - if (FFlag::LuauTypeAliasDefaults) - { - writer.advance(o.location.begin); - writer.identifier(o.name.value); + writer.advance(o.location.begin); + writer.identifier(o.name.value); - if (o.defaultValue) - { - writer.maybeSpace(o.defaultValue->location.begin, 2); - writer.symbol("="); - visualizeTypeAnnotation(*o.defaultValue); - } - } - else + if (o.defaultValue) { - writer.identifier(o.name.value); + writer.maybeSpace(o.defaultValue->location.begin, 2); + writer.symbol("="); + visualizeTypeAnnotation(*o.defaultValue); } } @@ -818,23 +809,15 @@ struct Printer { comma(); - if (FFlag::LuauTypeAliasDefaults) - { - writer.advance(o.location.begin); - writer.identifier(o.name.value); - writer.symbol("..."); + writer.advance(o.location.begin); + writer.identifier(o.name.value); + writer.symbol("..."); - if (o.defaultValue) - { - writer.maybeSpace(o.defaultValue->location.begin, 2); - writer.symbol("="); - visualizeTypePackAnnotation(*o.defaultValue, false); - } - } - else + if (o.defaultValue) { - writer.identifier(o.name.value); - writer.symbol("..."); + writer.maybeSpace(o.defaultValue->location.begin, 2); + writer.symbol("="); + visualizeTypePackAnnotation(*o.defaultValue, false); } } @@ -882,18 +865,14 @@ struct Printer { comma(); - if (FFlag::LuauTypeAliasDefaults) - writer.advance(o.location.begin); - + writer.advance(o.location.begin); writer.identifier(o.name.value); } for (const auto& o : func.genericPacks) { comma(); - if (FFlag::LuauTypeAliasDefaults) - writer.advance(o.location.begin); - + writer.advance(o.location.begin); writer.identifier(o.name.value); writer.symbol("..."); } @@ -1023,18 +1002,14 @@ struct Printer { comma(); - if (FFlag::LuauTypeAliasDefaults) - writer.advance(o.location.begin); - + writer.advance(o.location.begin); writer.identifier(o.name.value); } for (const auto& o : a->genericPacks) { comma(); - if (FFlag::LuauTypeAliasDefaults) - writer.advance(o.location.begin); - + writer.advance(o.location.begin); writer.identifier(o.name.value); writer.symbol("..."); } diff --git a/Analysis/src/TxnLog.cpp b/Analysis/src/TxnLog.cpp index 00067bdd..c7bf1e62 100644 --- a/Analysis/src/TxnLog.cpp +++ b/Analysis/src/TxnLog.cpp @@ -61,22 +61,52 @@ void DEPRECATED_TxnLog::concat(DEPRECATED_TxnLog rhs) bool DEPRECATED_TxnLog::haveSeen(TypeId lhs, TypeId rhs) { - LUAU_ASSERT(!FFlag::LuauUseCommittingTxnLog); - const std::pair sortedPair = (lhs > rhs) ? std::make_pair(lhs, rhs) : std::make_pair(rhs, lhs); - return (sharedSeen->end() != std::find(sharedSeen->begin(), sharedSeen->end(), sortedPair)); + return haveSeen((TypeOrPackId)lhs, (TypeOrPackId)rhs); } void DEPRECATED_TxnLog::pushSeen(TypeId lhs, TypeId rhs) { - LUAU_ASSERT(!FFlag::LuauUseCommittingTxnLog); - const std::pair sortedPair = (lhs > rhs) ? std::make_pair(lhs, rhs) : std::make_pair(rhs, lhs); - sharedSeen->push_back(sortedPair); + pushSeen((TypeOrPackId)lhs, (TypeOrPackId)rhs); } void DEPRECATED_TxnLog::popSeen(TypeId lhs, TypeId rhs) +{ + popSeen((TypeOrPackId)lhs, (TypeOrPackId)rhs); +} + +bool DEPRECATED_TxnLog::haveSeen(TypePackId lhs, TypePackId rhs) +{ + return haveSeen((TypeOrPackId)lhs, (TypeOrPackId)rhs); +} + +void DEPRECATED_TxnLog::pushSeen(TypePackId lhs, TypePackId rhs) +{ + pushSeen((TypeOrPackId)lhs, (TypeOrPackId)rhs); +} + +void DEPRECATED_TxnLog::popSeen(TypePackId lhs, TypePackId rhs) +{ + popSeen((TypeOrPackId)lhs, (TypeOrPackId)rhs); +} + +bool DEPRECATED_TxnLog::haveSeen(TypeOrPackId lhs, TypeOrPackId rhs) { LUAU_ASSERT(!FFlag::LuauUseCommittingTxnLog); - const std::pair sortedPair = (lhs > rhs) ? std::make_pair(lhs, rhs) : std::make_pair(rhs, lhs); + const std::pair sortedPair = (lhs > rhs) ? std::make_pair(lhs, rhs) : std::make_pair(rhs, lhs); + return (sharedSeen->end() != std::find(sharedSeen->begin(), sharedSeen->end(), sortedPair)); +} + +void DEPRECATED_TxnLog::pushSeen(TypeOrPackId lhs, TypeOrPackId rhs) +{ + LUAU_ASSERT(!FFlag::LuauUseCommittingTxnLog); + const std::pair sortedPair = (lhs > rhs) ? std::make_pair(lhs, rhs) : std::make_pair(rhs, lhs); + sharedSeen->push_back(sortedPair); +} + +void DEPRECATED_TxnLog::popSeen(TypeOrPackId lhs, TypeOrPackId rhs) +{ + LUAU_ASSERT(!FFlag::LuauUseCommittingTxnLog); + const std::pair sortedPair = (lhs > rhs) ? std::make_pair(lhs, rhs) : std::make_pair(rhs, lhs); LUAU_ASSERT(sortedPair == sharedSeen->back()); sharedSeen->pop_back(); } @@ -186,10 +216,40 @@ TxnLog TxnLog::inverse() } bool TxnLog::haveSeen(TypeId lhs, TypeId rhs) const +{ + return haveSeen((TypeOrPackId)lhs, (TypeOrPackId)rhs); +} + +void TxnLog::pushSeen(TypeId lhs, TypeId rhs) +{ + pushSeen((TypeOrPackId)lhs, (TypeOrPackId)rhs); +} + +void TxnLog::popSeen(TypeId lhs, TypeId rhs) +{ + popSeen((TypeOrPackId)lhs, (TypeOrPackId)rhs); +} + +bool TxnLog::haveSeen(TypePackId lhs, TypePackId rhs) const +{ + return haveSeen((TypeOrPackId)lhs, (TypeOrPackId)rhs); +} + +void TxnLog::pushSeen(TypePackId lhs, TypePackId rhs) +{ + pushSeen((TypeOrPackId)lhs, (TypeOrPackId)rhs); +} + +void TxnLog::popSeen(TypePackId lhs, TypePackId rhs) +{ + popSeen((TypeOrPackId)lhs, (TypeOrPackId)rhs); +} + +bool TxnLog::haveSeen(TypeOrPackId lhs, TypeOrPackId rhs) const { LUAU_ASSERT(FFlag::LuauUseCommittingTxnLog); - const std::pair sortedPair = (lhs > rhs) ? std::make_pair(lhs, rhs) : std::make_pair(rhs, lhs); + const std::pair sortedPair = (lhs > rhs) ? std::make_pair(lhs, rhs) : std::make_pair(rhs, lhs); if (sharedSeen->end() != std::find(sharedSeen->begin(), sharedSeen->end(), sortedPair)) { return true; @@ -203,19 +263,19 @@ bool TxnLog::haveSeen(TypeId lhs, TypeId rhs) const return false; } -void TxnLog::pushSeen(TypeId lhs, TypeId rhs) +void TxnLog::pushSeen(TypeOrPackId lhs, TypeOrPackId rhs) { LUAU_ASSERT(FFlag::LuauUseCommittingTxnLog); - const std::pair sortedPair = (lhs > rhs) ? std::make_pair(lhs, rhs) : std::make_pair(rhs, lhs); + const std::pair sortedPair = (lhs > rhs) ? std::make_pair(lhs, rhs) : std::make_pair(rhs, lhs); sharedSeen->push_back(sortedPair); } -void TxnLog::popSeen(TypeId lhs, TypeId rhs) +void TxnLog::popSeen(TypeOrPackId lhs, TypeOrPackId rhs) { LUAU_ASSERT(FFlag::LuauUseCommittingTxnLog); - const std::pair sortedPair = (lhs > rhs) ? std::make_pair(lhs, rhs) : std::make_pair(rhs, lhs); + const std::pair sortedPair = (lhs > rhs) ? std::make_pair(lhs, rhs) : std::make_pair(rhs, lhs); LUAU_ASSERT(sortedPair == sharedSeen->back()); sharedSeen->pop_back(); } diff --git a/Analysis/src/TypeInfer.cpp b/Analysis/src/TypeInfer.cpp index faf60eb3..8e6b3b52 100644 --- a/Analysis/src/TypeInfer.cpp +++ b/Analysis/src/TypeInfer.cpp @@ -29,22 +29,20 @@ LUAU_FASTFLAGVARIABLE(DebugLuauFreezeDuringUnification, false) LUAU_FASTFLAGVARIABLE(LuauRecursiveTypeParameterRestriction, false) LUAU_FASTFLAGVARIABLE(LuauGenericFunctionsDontCacheTypeParams, false) LUAU_FASTFLAGVARIABLE(LuauImmutableTypes, false) -LUAU_FASTFLAGVARIABLE(LuauQuantifyInPlace2, false) LUAU_FASTFLAGVARIABLE(LuauSealExports, false) LUAU_FASTFLAGVARIABLE(LuauSingletonTypes, false) LUAU_FASTFLAGVARIABLE(LuauDiscriminableUnions2, false) -LUAU_FASTFLAGVARIABLE(LuauTypeAliasDefaults, false) LUAU_FASTFLAGVARIABLE(LuauExpectedTypesOfProperties, false) LUAU_FASTFLAGVARIABLE(LuauErrorRecoveryType, false) +LUAU_FASTFLAGVARIABLE(LuauOnlyMutateInstantiatedTables, false) LUAU_FASTFLAGVARIABLE(LuauPropertiesGetExpectedType, false) -LUAU_FASTFLAGVARIABLE(LuauAscribeCorrectLevelToInferredProperitesOfFreeTables, false) LUAU_FASTFLAGVARIABLE(LuauUnsealedTableLiteral, false) LUAU_FASTFLAGVARIABLE(LuauTwoPassAliasDefinitionFix, false) LUAU_FASTFLAGVARIABLE(LuauAssertStripsFalsyTypes, false) LUAU_FASTFLAGVARIABLE(LuauReturnAnyInsteadOfICE, false) // Eventually removed as false. -LUAU_FASTFLAGVARIABLE(LuauAnotherTypeLevelFix, false) LUAU_FASTFLAG(LuauWidenIfSupertypeIsFree) LUAU_FASTFLAGVARIABLE(LuauDoNotTryToReduce, false) +LUAU_FASTFLAGVARIABLE(LuauDoNotAccidentallyDependOnPointerOrdering, false) namespace Luau { @@ -445,7 +443,7 @@ void TypeChecker::checkBlock(const ScopePtr& scope, const AstStatBlock& block) // function f(x:a):a local x: number = g(37) return x end // function g(x:number):number return f(x) end // ``` - if (FFlag::LuauQuantifyInPlace2 ? containsFunctionCallOrReturn(**protoIter) : containsFunctionCall(**protoIter)) + if (containsFunctionCallOrReturn(**protoIter)) { while (checkIter != protoIter) { @@ -1676,7 +1674,7 @@ std::optional TypeChecker::getIndexTypeFromType( } else if (tableType->state == TableState::Free) { - TypeId result = FFlag::LuauAscribeCorrectLevelToInferredProperitesOfFreeTables ? freshType(tableType->level) : freshType(scope); + TypeId result = freshType(tableType->level); tableType->props[name] = {result}; return result; } @@ -1776,31 +1774,62 @@ std::optional TypeChecker::getIndexTypeFromType( std::vector TypeChecker::reduceUnion(const std::vector& types) { - std::set s; - - for (TypeId t : types) + if (FFlag::LuauDoNotAccidentallyDependOnPointerOrdering) { - if (const UnionTypeVar* utv = get(follow(t))) + std::vector result; + for (TypeId t : types) { - std::vector r = reduceUnion(utv->options); - for (TypeId ty : r) - s.insert(ty); + t = follow(t); + if (get(t) || get(t)) + return {t}; + + if (const UnionTypeVar* utv = get(t)) + { + std::vector r = reduceUnion(utv->options); + for (TypeId ty : r) + { + ty = follow(ty); + if (get(ty) || get(ty)) + return {ty}; + + if (std::find(result.begin(), result.end(), ty) == result.end()) + result.push_back(ty); + } + } + else if (std::find(result.begin(), result.end(), t) == result.end()) + result.push_back(t); } - else - s.insert(t); - } - // If any of them are ErrorTypeVars/AnyTypeVars, decay into them. - for (TypeId t : s) + return result; + } + else { - t = follow(t); - if (get(t) || get(t)) - return {t}; - } + std::set s; - std::vector r(s.begin(), s.end()); - std::sort(r.begin(), r.end()); - return r; + for (TypeId t : types) + { + if (const UnionTypeVar* utv = get(follow(t))) + { + std::vector r = reduceUnion(utv->options); + for (TypeId ty : r) + s.insert(ty); + } + else + s.insert(t); + } + + // If any of them are ErrorTypeVars/AnyTypeVars, decay into them. + for (TypeId t : s) + { + t = follow(t); + if (get(t) || get(t)) + return {t}; + } + + std::vector r(s.begin(), s.end()); + std::sort(r.begin(), r.end()); + return r; + } } std::optional TypeChecker::tryStripUnionFromNil(TypeId ty) @@ -2811,7 +2840,7 @@ TypeId TypeChecker::checkLValueBinding(const ScopePtr& scope, const AstExprIndex } else if (exprTable->state == TableState::Unsealed || exprTable->state == TableState::Free) { - TypeId resultType = freshType(FFlag::LuauAnotherTypeLevelFix ? exprTable->level : scope->level); + TypeId resultType = freshType(exprTable->level); exprTable->indexer = TableIndexer{anyIfNonstrict(indexType), anyIfNonstrict(resultType)}; return resultType; } @@ -4453,51 +4482,6 @@ TypePackId ReplaceGenerics::clean(TypePackId tp) return addTypePack(TypePackVar(FreeTypePack{level})); } -bool Quantification::isDirty(TypeId ty) -{ - if (const TableTypeVar* ttv = log->getMutable(ty)) - return level.subsumes(ttv->level) && ((ttv->state == TableState::Free) || (ttv->state == TableState::Unsealed)); - else if (const FreeTypeVar* ftv = log->getMutable(ty)) - return level.subsumes(ftv->level); - else - return false; -} - -bool Quantification::isDirty(TypePackId tp) -{ - if (const FreeTypePack* ftv = log->getMutable(tp)) - return level.subsumes(ftv->level); - else - return false; -} - -TypeId Quantification::clean(TypeId ty) -{ - LUAU_ASSERT(isDirty(ty)); - if (const TableTypeVar* ttv = log->getMutable(ty)) - { - TableState state = (ttv->state == TableState::Unsealed ? TableState::Sealed : TableState::Generic); - TableTypeVar clone = TableTypeVar{ttv->props, ttv->indexer, level, state}; - clone.methodDefinitionLocations = ttv->methodDefinitionLocations; - clone.definitionModuleName = ttv->definitionModuleName; - return addType(std::move(clone)); - } - else - { - TypeId generic = addType(GenericTypeVar{level}); - generics.push_back(generic); - return generic; - } -} - -TypePackId Quantification::clean(TypePackId tp) -{ - LUAU_ASSERT(isDirty(tp)); - TypePackId genericPack = addTypePack(TypePackVar(GenericTypePack{level})); - genericPacks.push_back(genericPack); - return genericPack; -} - bool Anyification::isDirty(TypeId ty) { if (const TableTypeVar* ttv = log->getMutable(ty)) @@ -4550,29 +4534,8 @@ TypeId TypeChecker::quantify(const ScopePtr& scope, TypeId ty, Location location if (!ftv || !ftv->generics.empty() || !ftv->genericPacks.empty()) return ty; - if (FFlag::LuauQuantifyInPlace2) - { - Luau::quantify(ty, scope->level); - return ty; - } - - Quantification quantification{¤tModule->internalTypes, scope->level}; - std::optional qty = quantification.substitute(ty); - - if (!qty.has_value()) - { - reportError(location, UnificationTooComplex{}); - return errorRecoveryType(scope); - } - - if (ty == *qty) - return ty; - - FunctionTypeVar* qftv = getMutable(*qty); - LUAU_ASSERT(qftv); - qftv->generics = std::move(quantification.generics); - qftv->genericPacks = std::move(quantification.genericPacks); - return *qty; + Luau::quantify(ty, scope->level); + return ty; } TypeId TypeChecker::instantiate(const ScopePtr& scope, TypeId ty, Location location, const TxnLog* log) @@ -4915,35 +4878,20 @@ TypeId TypeChecker::resolveType(const ScopePtr& scope, const AstType& annotation if (lit->parameters.size == 0 && tf->typeParams.empty() && tf->typePackParams.empty()) return tf->type; - bool hasDefaultTypes = false; - bool hasDefaultPacks = false; bool parameterCountErrorReported = false; + bool hasDefaultTypes = std::any_of(tf->typeParams.begin(), tf->typeParams.end(), [](auto&& el) { + return el.defaultValue.has_value(); + }); + bool hasDefaultPacks = std::any_of(tf->typePackParams.begin(), tf->typePackParams.end(), [](auto&& el) { + return el.defaultValue.has_value(); + }); - if (FFlag::LuauTypeAliasDefaults) + if (!lit->hasParameterList) { - hasDefaultTypes = std::any_of(tf->typeParams.begin(), tf->typeParams.end(), [](auto&& el) { - return el.defaultValue.has_value(); - }); - hasDefaultPacks = std::any_of(tf->typePackParams.begin(), tf->typePackParams.end(), [](auto&& el) { - return el.defaultValue.has_value(); - }); - - if (!lit->hasParameterList) - { - if ((!tf->typeParams.empty() && !hasDefaultTypes) || (!tf->typePackParams.empty() && !hasDefaultPacks)) - { - reportError(TypeError{annotation.location, GenericError{"Type parameter list is required"}}); - parameterCountErrorReported = true; - if (!FFlag::LuauErrorRecoveryType) - return errorRecoveryType(scope); - } - } - } - else - { - if (!lit->hasParameterList && !tf->typePackParams.empty()) + if ((!tf->typeParams.empty() && !hasDefaultTypes) || (!tf->typePackParams.empty() && !hasDefaultPacks)) { reportError(TypeError{annotation.location, GenericError{"Type parameter list is required"}}); + parameterCountErrorReported = true; if (!FFlag::LuauErrorRecoveryType) return errorRecoveryType(scope); } @@ -4986,72 +4934,69 @@ TypeId TypeChecker::resolveType(const ScopePtr& scope, const AstType& annotation if (typePackParams.empty() && !extraTypes.empty()) typePackParams.push_back(addTypePack(extraTypes)); - if (FFlag::LuauTypeAliasDefaults) + size_t typesProvided = typeParams.size(); + size_t typesRequired = tf->typeParams.size(); + + size_t packsProvided = typePackParams.size(); + size_t packsRequired = tf->typePackParams.size(); + + bool notEnoughParameters = + (typesProvided < typesRequired && packsProvided == 0) || (typesProvided == typesRequired && packsProvided < packsRequired); + bool hasDefaultParameters = hasDefaultTypes || hasDefaultPacks; + + // Add default type and type pack parameters if that's required and it's possible + if (notEnoughParameters && hasDefaultParameters) { - size_t typesProvided = typeParams.size(); - size_t typesRequired = tf->typeParams.size(); + // 'applyTypeFunction' is used to substitute default types that reference previous generic types + ApplyTypeFunction applyTypeFunction{¤tModule->internalTypes, scope->level}; - size_t packsProvided = typePackParams.size(); - size_t packsRequired = tf->typePackParams.size(); + for (size_t i = 0; i < typesProvided; ++i) + applyTypeFunction.typeArguments[tf->typeParams[i].ty] = typeParams[i]; - bool notEnoughParameters = - (typesProvided < typesRequired && packsProvided == 0) || (typesProvided == typesRequired && packsProvided < packsRequired); - bool hasDefaultParameters = hasDefaultTypes || hasDefaultPacks; - - // Add default type and type pack parameters if that's required and it's possible - if (notEnoughParameters && hasDefaultParameters) + if (typesProvided < typesRequired) { - // 'applyTypeFunction' is used to substitute default types that reference previous generic types - ApplyTypeFunction applyTypeFunction{¤tModule->internalTypes, scope->level}; - - for (size_t i = 0; i < typesProvided; ++i) - applyTypeFunction.typeArguments[tf->typeParams[i].ty] = typeParams[i]; - - if (typesProvided < typesRequired) + for (size_t i = typesProvided; i < typesRequired; ++i) { - for (size_t i = typesProvided; i < typesRequired; ++i) + TypeId defaultTy = tf->typeParams[i].defaultValue.value_or(nullptr); + + if (!defaultTy) + break; + + std::optional maybeInstantiated = applyTypeFunction.substitute(defaultTy); + + if (!maybeInstantiated.has_value()) { - TypeId defaultTy = tf->typeParams[i].defaultValue.value_or(nullptr); - - if (!defaultTy) - break; - - std::optional maybeInstantiated = applyTypeFunction.substitute(defaultTy); - - if (!maybeInstantiated.has_value()) - { - reportError(annotation.location, UnificationTooComplex{}); - maybeInstantiated = errorRecoveryType(scope); - } - - applyTypeFunction.typeArguments[tf->typeParams[i].ty] = *maybeInstantiated; - typeParams.push_back(*maybeInstantiated); + reportError(annotation.location, UnificationTooComplex{}); + maybeInstantiated = errorRecoveryType(scope); } + + applyTypeFunction.typeArguments[tf->typeParams[i].ty] = *maybeInstantiated; + typeParams.push_back(*maybeInstantiated); } + } - for (size_t i = 0; i < packsProvided; ++i) - applyTypeFunction.typePackArguments[tf->typePackParams[i].tp] = typePackParams[i]; + for (size_t i = 0; i < packsProvided; ++i) + applyTypeFunction.typePackArguments[tf->typePackParams[i].tp] = typePackParams[i]; - if (packsProvided < packsRequired) + if (packsProvided < packsRequired) + { + for (size_t i = packsProvided; i < packsRequired; ++i) { - for (size_t i = packsProvided; i < packsRequired; ++i) + TypePackId defaultTp = tf->typePackParams[i].defaultValue.value_or(nullptr); + + if (!defaultTp) + break; + + std::optional maybeInstantiated = applyTypeFunction.substitute(defaultTp); + + if (!maybeInstantiated.has_value()) { - TypePackId defaultTp = tf->typePackParams[i].defaultValue.value_or(nullptr); - - if (!defaultTp) - break; - - std::optional maybeInstantiated = applyTypeFunction.substitute(defaultTp); - - if (!maybeInstantiated.has_value()) - { - reportError(annotation.location, UnificationTooComplex{}); - maybeInstantiated = errorRecoveryTypePack(scope); - } - - applyTypeFunction.typePackArguments[tf->typePackParams[i].tp] = *maybeInstantiated; - typePackParams.push_back(*maybeInstantiated); + reportError(annotation.location, UnificationTooComplex{}); + maybeInstantiated = errorRecoveryTypePack(scope); } + + applyTypeFunction.typePackArguments[tf->typePackParams[i].tp] = *maybeInstantiated; + typePackParams.push_back(*maybeInstantiated); } } } @@ -5343,12 +5288,12 @@ TypeId TypeChecker::instantiateTypeFun(const ScopePtr& scope, const TypeFun& tf, TypeId instantiated = *maybeInstantiated; - // TODO: CLI-46926 it's not a good idea to rename the type here TypeId target = follow(instantiated); bool needsClone = follow(tf.type) == target; + bool shouldMutate = (!FFlag::LuauOnlyMutateInstantiatedTables || getTableType(tf.type)); TableTypeVar* ttv = getMutableTableType(target); - - if (ttv && needsClone) + + if (shouldMutate && ttv && needsClone) { // Substitution::clone is a shallow clone. If this is a metatable type, we // want to mutate its table, so we need to explicitly clone that table as @@ -5368,7 +5313,7 @@ TypeId TypeChecker::instantiateTypeFun(const ScopePtr& scope, const TypeFun& tf, } } - if (ttv) + if (shouldMutate && ttv) { ttv->instantiatedTypeParams = typeParams; ttv->instantiatedTypePackParams = typePackParams; @@ -5382,7 +5327,7 @@ GenericTypeDefinitions TypeChecker::createGenericTypes(const ScopePtr& scope, st { LUAU_ASSERT(scope->parent); - const TypeLevel level = (FFlag::LuauQuantifyInPlace2 && levelOpt) ? *levelOpt : scope->level; + const TypeLevel level = levelOpt.value_or(scope->level); std::vector generics; @@ -5390,7 +5335,7 @@ GenericTypeDefinitions TypeChecker::createGenericTypes(const ScopePtr& scope, st { std::optional defaultValue; - if (FFlag::LuauTypeAliasDefaults && generic.defaultValue) + if (generic.defaultValue) defaultValue = resolveType(scope, *generic.defaultValue); Name n = generic.name.value; @@ -5426,7 +5371,7 @@ GenericTypeDefinitions TypeChecker::createGenericTypes(const ScopePtr& scope, st { std::optional defaultValue; - if (FFlag::LuauTypeAliasDefaults && genericPack.defaultValue) + if (genericPack.defaultValue) defaultValue = resolveTypePack(scope, *genericPack.defaultValue); Name n = genericPack.name.value; diff --git a/Analysis/src/TypeVar.cpp b/Analysis/src/TypeVar.cpp index a1dcfdbe..5af2c8a6 100644 --- a/Analysis/src/TypeVar.cpp +++ b/Analysis/src/TypeVar.cpp @@ -24,6 +24,7 @@ LUAU_FASTINTVARIABLE(LuauTypeMaximumStringifierLength, 500) LUAU_FASTINTVARIABLE(LuauTableTypeMaximumStringifierLength, 0) LUAU_FASTINT(LuauTypeInferRecursionLimit) LUAU_FASTFLAG(LuauErrorRecoveryType) +LUAU_FASTFLAG(LuauSubtypingAddOptPropsToUnsealedTables) LUAU_FASTFLAG(LuauDiscriminableUnions2) namespace Luau @@ -157,6 +158,7 @@ bool isNumber(TypeId ty) return isPrim(ty, PrimitiveTypeVar::Number); } +// Returns true when ty is a subtype of string bool isString(TypeId ty) { if (isPrim(ty, PrimitiveTypeVar::String) || get(get(follow(ty)))) @@ -168,6 +170,27 @@ bool isString(TypeId ty) return false; } +// Returns true when ty is a supertype of string +bool maybeString(TypeId ty) +{ + if (FFlag::LuauSubtypingAddOptPropsToUnsealedTables) + { + ty = follow(ty); + + if (isPrim(ty, PrimitiveTypeVar::String) || get(ty)) + return true; + + if (auto utv = get(ty)) + return std::any_of(begin(utv), end(utv), maybeString); + + return false; + } + else + { + return isString(ty); + } +} + bool isThread(TypeId ty) { return isPrim(ty, PrimitiveTypeVar::Thread); @@ -684,7 +707,7 @@ TypeId SingletonTypes::makeStringMetatable() {"sub", {makeFunction(*arena, stringType, {}, {}, {numberType, optionalNumber}, {}, {stringType})}}, {"upper", {stringToStringType}}, {"split", {makeFunction(*arena, stringType, {}, {}, {optionalString}, {}, - {arena->addType(TableTypeVar{{}, TableIndexer{numberType, stringType}, TypeLevel{}})})}}, + {arena->addType(TableTypeVar{{}, TableIndexer{numberType, stringType}, TypeLevel{}, TableState::Sealed})})}}, {"pack", {arena->addType(FunctionTypeVar{ arena->addTypePack(TypePack{{stringType}, anyTypePack}), oneStringPack, @@ -761,6 +784,8 @@ void persist(TypeId ty) } else if (auto ttv = get(t)) { + LUAU_ASSERT(ttv->state != TableState::Free && ttv->state != TableState::Unsealed); + for (const auto& [_name, prop] : ttv->props) queue.push_back(prop.type); diff --git a/Analysis/src/Unifier.cpp b/Analysis/src/Unifier.cpp index d0eba013..6c29486a 100644 --- a/Analysis/src/Unifier.cpp +++ b/Analysis/src/Unifier.cpp @@ -18,11 +18,13 @@ LUAU_FASTFLAG(LuauImmutableTypes) LUAU_FASTFLAG(LuauUseCommittingTxnLog) LUAU_FASTINTVARIABLE(LuauTypeInferIterationLimit, 2000); LUAU_FASTFLAGVARIABLE(LuauTableSubtypingVariance2, false); -LUAU_FASTFLAGVARIABLE(LuauTableUnificationEarlyTest, false) LUAU_FASTFLAG(LuauSingletonTypes) LUAU_FASTFLAG(LuauErrorRecoveryType); +LUAU_FASTFLAGVARIABLE(LuauSubtypingAddOptPropsToUnsealedTables, false) LUAU_FASTFLAGVARIABLE(LuauFollowWithCommittingTxnLogInAnyUnification, false) LUAU_FASTFLAGVARIABLE(LuauWidenIfSupertypeIsFree, false) +LUAU_FASTFLAGVARIABLE(LuauDifferentOrderOfUnificationDoesntMatter, false) +LUAU_FASTFLAGVARIABLE(LuauTxnLogSeesTypePacks2, true) namespace Luau { @@ -329,7 +331,7 @@ Unifier::Unifier(TypeArena* types, Mode mode, const Location& location, Variance LUAU_ASSERT(sharedState.iceHandler); } -Unifier::Unifier(TypeArena* types, Mode mode, std::vector>* sharedSeen, const Location& location, +Unifier::Unifier(TypeArena* types, Mode mode, std::vector>* sharedSeen, const Location& location, Variance variance, UnifierSharedState& sharedState, TxnLog* parentLog) : types(types) , mode(mode) @@ -656,26 +658,85 @@ void Unifier::tryUnifyUnionWithType(TypeId subTy, const UnionTypeVar* uv, TypeId failed = true; } - if (FFlag::LuauUseCommittingTxnLog) + if (FFlag::LuauDifferentOrderOfUnificationDoesntMatter) { - if (i == count - 1) - { - log.concat(std::move(innerState.log)); - } + if (!FFlag::LuauUseCommittingTxnLog) + innerState.DEPRECATED_log.rollback(); } else { - if (i != count - 1) + if (FFlag::LuauUseCommittingTxnLog) { - innerState.DEPRECATED_log.rollback(); + if (i == count - 1) + { + log.concat(std::move(innerState.log)); + } } else { - DEPRECATED_log.concat(std::move(innerState.DEPRECATED_log)); + if (i != count - 1) + { + innerState.DEPRECATED_log.rollback(); + } + else + { + DEPRECATED_log.concat(std::move(innerState.DEPRECATED_log)); + } } - } - ++i; + ++i; + } + } + + // even if A | B <: T fails, we want to bind some options of T with A | B iff A | B was a subtype of that option. + if (FFlag::LuauDifferentOrderOfUnificationDoesntMatter) + { + auto tryBind = [this, subTy](TypeId superOption) { + superOption = FFlag::LuauUseCommittingTxnLog ? log.follow(superOption) : follow(superOption); + + // just skip if the superOption is not free-ish. + auto ttv = log.getMutable(superOption); + if (!log.is(superOption) && (!ttv || ttv->state != TableState::Free)) + return; + + // Since we have already checked if S <: T, checking it again will not queue up the type for replacement. + // So we'll have to do it ourselves. We assume they unified cleanly if they are still in the seen set. + if (FFlag::LuauUseCommittingTxnLog) + { + if (log.haveSeen(subTy, superOption)) + { + // TODO: would it be nice for TxnLog::replace to do this? + if (log.is(superOption)) + log.bindTable(superOption, subTy); + else + log.replace(superOption, *subTy); + } + } + else + { + if (DEPRECATED_log.haveSeen(subTy, superOption)) + { + if (auto ttv = getMutable(superOption)) + { + DEPRECATED_log(ttv); + ttv->boundTo = subTy; + } + else + { + DEPRECATED_log(superOption); + *asMutable(superOption) = BoundTypeVar(subTy); + } + } + } + }; + + if (auto utv = (FFlag::LuauUseCommittingTxnLog ? log.getMutable(superTy) : get(superTy))) + { + for (TypeId ty : utv) + tryBind(ty); + } + else + tryBind(superTy); } if (unificationTooComplex) @@ -1163,6 +1224,9 @@ void Unifier::tryUnify_(TypePackId subTp, TypePackId superTp, bool isFunctionCal if (superTp == subTp) return; + if (FFlag::LuauTxnLogSeesTypePacks2 && log.haveSeen(superTp, subTp)) + return; + if (log.getMutable(superTp)) { occursCheck(superTp, subTp); @@ -1365,6 +1429,9 @@ void Unifier::tryUnify_(TypePackId subTp, TypePackId superTp, bool isFunctionCal if (superTp == subTp) return; + if (FFlag::LuauTxnLogSeesTypePacks2 && DEPRECATED_log.haveSeen(superTp, subTp)) + return; + if (get(superTp)) { occursCheck(superTp, subTp); @@ -1619,6 +1686,17 @@ void Unifier::tryUnifyFunctions(TypeId subTy, TypeId superTy, bool isFunctionCal DEPRECATED_log.pushSeen(superFunction->generics[i], subFunction->generics[i]); } + if (FFlag::LuauTxnLogSeesTypePacks2) + { + for (size_t i = 0; i < numGenericPacks; i++) + { + if (FFlag::LuauUseCommittingTxnLog) + log.pushSeen(superFunction->genericPacks[i], subFunction->genericPacks[i]); + else + DEPRECATED_log.pushSeen(superFunction->genericPacks[i], subFunction->genericPacks[i]); + } + } + CountMismatch::Context context = ctx; if (!isFunctionCall) @@ -1708,6 +1786,17 @@ void Unifier::tryUnifyFunctions(TypeId subTy, TypeId superTy, bool isFunctionCal ctx = context; + if (FFlag::LuauTxnLogSeesTypePacks2) + { + for (int i = int(numGenericPacks) - 1; 0 <= i; i--) + { + if (FFlag::LuauUseCommittingTxnLog) + log.popSeen(superFunction->genericPacks[i], subFunction->genericPacks[i]); + else + DEPRECATED_log.popSeen(superFunction->genericPacks[i], subFunction->genericPacks[i]); + } + } + for (int i = int(numGenerics) - 1; 0 <= i; i--) { if (FFlag::LuauUseCommittingTxnLog) @@ -1760,7 +1849,7 @@ void Unifier::tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) std::vector extraProperties; // Optimization: First test that the property sets are compatible without doing any recursive unification - if (FFlag::LuauTableUnificationEarlyTest && !subTable->indexer && subTable->state != TableState::Free) + if (!subTable->indexer && subTable->state != TableState::Free) { for (const auto& [propName, superProp] : superTable->props) { @@ -1769,7 +1858,7 @@ void Unifier::tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) bool isAny = FFlag::LuauUseCommittingTxnLog ? log.getMutable(log.follow(superProp.type)) : get(follow(superProp.type)); - if (subIter == subTable->props.end() && !isOptional(superProp.type) && !isAny) + if (subIter == subTable->props.end() && (!FFlag::LuauSubtypingAddOptPropsToUnsealedTables || subTable->state == TableState::Unsealed) && !isOptional(superProp.type) && !isAny) missingProperties.push_back(propName); } @@ -1781,7 +1870,7 @@ void Unifier::tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) } // And vice versa if we're invariant - if (FFlag::LuauTableUnificationEarlyTest && variance == Invariant && !superTable->indexer && superTable->state != TableState::Unsealed && + if (variance == Invariant && !superTable->indexer && superTable->state != TableState::Unsealed && superTable->state != TableState::Free) { for (const auto& [propName, subProp] : subTable->props) @@ -1790,7 +1879,7 @@ void Unifier::tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) bool isAny = FFlag::LuauUseCommittingTxnLog ? log.getMutable(log.follow(subProp.type)) : get(follow(subProp.type)); - if (superIter == superTable->props.end() && !isOptional(subProp.type) && !isAny) + if (superIter == superTable->props.end() && (FFlag::LuauSubtypingAddOptPropsToUnsealedTables || (!isOptional(subProp.type) && !isAny))) extraProperties.push_back(propName); } @@ -1830,7 +1919,7 @@ void Unifier::tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) innerState.DEPRECATED_log.rollback(); } } - else if (subTable->indexer && isString(subTable->indexer->indexType)) + else if (subTable->indexer && maybeString(subTable->indexer->indexType)) { // TODO: read-only indexers don't need invariance // TODO: really we should only allow this if prop.type is optional. @@ -1855,9 +1944,11 @@ void Unifier::tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) innerState.DEPRECATED_log.rollback(); } } - else if (isOptional(prop.type) || get(follow(prop.type))) - // TODO: this case is unsound, but without it our test suite fails. CLI-46031 + else if ((!FFlag::LuauSubtypingAddOptPropsToUnsealedTables || subTable->state == TableState::Unsealed) && (isOptional(prop.type) || get(follow(prop.type)))) + // This is sound because unsealed table types are precise, so `{ p : T } <: { p : T, q : U? }` + // since if `t : { p : T }` then we are guaranteed that `t.q` is `nil`. // TODO: should isOptional(anyType) be true? + // TODO: if the supertype is written to, the subtype may no longer be precise (alias analysis?) { } else if (subTable->state == TableState::Free) @@ -1887,7 +1978,7 @@ void Unifier::tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) // If both lt and rt contain the property, then // we're done since we already unified them above } - else if (superTable->indexer && isString(superTable->indexer->indexType)) + else if (superTable->indexer && maybeString(superTable->indexer->indexType)) { // TODO: read-only indexers don't need invariance // TODO: really we should only allow this if prop.type is optional. @@ -1936,9 +2027,7 @@ void Unifier::tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) else if (variance == Covariant) { } - else if (isOptional(prop.type) || get(follow(prop.type))) - // TODO: this case is unsound, but without it our test suite fails. CLI-46031 - // TODO: should isOptional(anyType) be true? + else if (!FFlag::LuauSubtypingAddOptPropsToUnsealedTables && (isOptional(prop.type) || get(follow(prop.type)))) { } else if (superTable->state == TableState::Free) @@ -2333,7 +2422,7 @@ void Unifier::tryUnifySealedTables(TypeId subTy, TypeId superTy, bool isIntersec bool errorReported = false; // Optimization: First test that the property sets are compatible without doing any recursive unification - if (FFlag::LuauTableUnificationEarlyTest && !subTable->indexer) + if (!subTable->indexer) { for (const auto& [propName, superProp] : superTable->props) { diff --git a/Ast/src/Parser.cpp b/Ast/src/Parser.cpp index 8767daa0..1cb8f134 100644 --- a/Ast/src/Parser.cpp +++ b/Ast/src/Parser.cpp @@ -11,7 +11,6 @@ LUAU_FASTINTVARIABLE(LuauRecursionLimit, 1000) LUAU_FASTINTVARIABLE(LuauParseErrorLimit, 100) LUAU_FASTFLAGVARIABLE(LuauParseSingletonTypes, false) -LUAU_FASTFLAGVARIABLE(LuauParseTypeAliasDefaults, false) LUAU_FASTFLAGVARIABLE(LuauParseAllHotComments, false) LUAU_FASTFLAGVARIABLE(LuauTableFieldFunctionDebugname, false) @@ -779,7 +778,7 @@ AstStat* Parser::parseTypeAlias(const Location& start, bool exported) if (!name) name = Name(nameError, lexer.current().location); - auto [generics, genericPacks] = parseGenericTypeList(/* withDefaultValues= */ FFlag::LuauParseTypeAliasDefaults); + auto [generics, genericPacks] = parseGenericTypeList(/* withDefaultValues= */ true); expectAndConsume('=', "type alias"); diff --git a/CLI/Repl.cpp b/CLI/Repl.cpp index 13304d57..5fd6d341 100644 --- a/CLI/Repl.cpp +++ b/CLI/Repl.cpp @@ -413,29 +413,22 @@ static void completeRepl(ic_completion_env_t* cenv, const char* editBuffer) ic_complete_word(cenv, editBuffer, icGetCompletions, isMethodOrFunctionChar); } -struct LinenoiseScopedHistory +static void loadHistory(const char* name) { - LinenoiseScopedHistory() + std::string path; + + if (const char* home = getenv("HOME")) { - const std::string name(".luau_history"); - - if (const char* home = getenv("HOME")) - { - historyFilepath = joinPaths(home, name); - } - else if (const char* userProfile = getenv("USERPROFILE")) - { - historyFilepath = joinPaths(userProfile, name); - } - - if (!historyFilepath.empty()) - ic_set_history(historyFilepath.c_str(), -1 /* default entries (= 200) */); + path = joinPaths(home, name); + } + else if (const char* userProfile = getenv("USERPROFILE")) + { + path = joinPaths(userProfile, name); } - ~LinenoiseScopedHistory() {} - - std::string historyFilepath; -}; + if (!path.empty()) + ic_set_history(path.c_str(), -1 /* default entries (= 200) */); +} static void runReplImpl(lua_State* L) { @@ -447,8 +440,10 @@ static void runReplImpl(lua_State* L) // Prevent auto insertion of braces ic_enable_brace_insertion(false); + // Loads history from the given file; isocline automatically saves the history on process exit + loadHistory(".luau_history"); + std::string buffer; - LinenoiseScopedHistory scopedHistory; for (;;) { diff --git a/VM/include/lua.h b/VM/include/lua.h index af0e2835..0a561f27 100644 --- a/VM/include/lua.h +++ b/VM/include/lua.h @@ -265,6 +265,8 @@ LUA_API double lua_clock(); LUA_API void lua_setuserdatadtor(lua_State* L, int tag, void (*dtor)(void*)); +LUA_API void lua_clonefunction(lua_State* L, int idx); + /* ** reference system, can be used to pin objects */ @@ -324,6 +326,7 @@ typedef struct lua_Debug lua_Debug; /* activation record */ /* Functions to be called by the debugger in specific events */ typedef void (*lua_Hook)(lua_State* L, lua_Debug* ar); +LUA_API int lua_stackdepth(lua_State* L); LUA_API int lua_getinfo(lua_State* L, int level, const char* what, lua_Debug* ar); LUA_API int lua_getargument(lua_State* L, int level, int n); LUA_API const char* lua_getlocal(lua_State* L, int level, int n); diff --git a/VM/src/lapi.cpp b/VM/src/lapi.cpp index 39c76e08..f7f15442 100644 --- a/VM/src/lapi.cpp +++ b/VM/src/lapi.cpp @@ -14,7 +14,7 @@ #include -LUAU_FASTFLAGVARIABLE(LuauGcForwardMetatableBarrier, false) +LUAU_FASTFLAG(LuauGcAdditionalStats) const char* lua_ident = "$Lua: Lua 5.1.4 Copyright (C) 1994-2008 Lua.org, PUC-Rio $\n" "$Authors: R. Ierusalimschy, L. H. de Figueiredo & W. Celes $\n" @@ -876,16 +876,7 @@ int lua_setmetatable(lua_State* L, int objindex) luaG_runerror(L, "Attempt to modify a readonly table"); hvalue(obj)->metatable = mt; if (mt) - { - if (FFlag::LuauGcForwardMetatableBarrier) - { - luaC_objbarrier(L, hvalue(obj), mt); - } - else - { - luaC_objbarriert(L, hvalue(obj), mt); - } - } + luaC_objbarrier(L, hvalue(obj), mt); break; } case LUA_TUSERDATA: @@ -1069,6 +1060,8 @@ int lua_gc(lua_State* L, int what, int data) g->GCthreshold = 0; bool waspaused = g->gcstate == GCSpause; + double startmarktime = g->gcstats.currcycle.marktime; + double startsweeptime = g->gcstats.currcycle.sweeptime; // track how much work the loop will actually perform size_t actualwork = 0; @@ -1086,6 +1079,31 @@ int lua_gc(lua_State* L, int what, int data) } } + if (FFlag::LuauGcAdditionalStats) + { + // record explicit step statistics + GCCycleStats* cyclestats = g->gcstate == GCSpause ? &g->gcstats.lastcycle : &g->gcstats.currcycle; + + double totalmarktime = cyclestats->marktime - startmarktime; + double totalsweeptime = cyclestats->sweeptime - startsweeptime; + + if (totalmarktime > 0.0) + { + cyclestats->markexplicitsteps++; + + if (totalmarktime > cyclestats->markmaxexplicittime) + cyclestats->markmaxexplicittime = totalmarktime; + } + + if (totalsweeptime > 0.0) + { + cyclestats->sweepexplicitsteps++; + + if (totalsweeptime > cyclestats->sweepmaxexplicittime) + cyclestats->sweepmaxexplicittime = totalsweeptime; + } + } + // if cycle hasn't finished, advance threshold forward for the amount of extra work performed if (g->gcstate != GCSpause) { @@ -1299,6 +1317,18 @@ void lua_setuserdatadtor(lua_State* L, int tag, void (*dtor)(void*)) L->global->udatagc[tag] = dtor; } +LUA_API void lua_clonefunction(lua_State* L, int idx) +{ + StkId p = index2addr(L, idx); + api_check(L, isLfunction(p)); + + luaC_checkthreadsleep(L); + + Closure* cl = clvalue(p); + Closure* newcl = luaF_newLclosure(L, 0, L->gt, cl->l.p); + setclvalue(L, L->top - 1, newcl); +} + lua_Callbacks* lua_callbacks(lua_State* L) { return &L->global->cb; diff --git a/VM/src/laux.cpp b/VM/src/laux.cpp index 71975a52..9a6f7793 100644 --- a/VM/src/laux.cpp +++ b/VM/src/laux.cpp @@ -11,8 +11,6 @@ #include -LUAU_FASTFLAG(LuauSchubfach) - /* convert a stack index to positive */ #define abs_index(L, i) ((i) > 0 || (i) <= LUA_REGISTRYINDEX ? (i) : lua_gettop(L) + (i) + 1) @@ -480,18 +478,13 @@ const char* luaL_tolstring(lua_State* L, int idx, size_t* len) switch (lua_type(L, idx)) { case LUA_TNUMBER: - if (FFlag::LuauSchubfach) - { - double n = lua_tonumber(L, idx); - char s[LUAI_MAXNUM2STR]; - char* e = luai_num2str(s, n); - lua_pushlstring(L, s, e - s); - } - else - { - lua_pushstring(L, lua_tostring(L, idx)); - } + { + double n = lua_tonumber(L, idx); + char s[LUAI_MAXNUM2STR]; + char* e = luai_num2str(s, n); + lua_pushlstring(L, s, e - s); break; + } case LUA_TSTRING: lua_pushvalue(L, idx); break; @@ -505,29 +498,18 @@ const char* luaL_tolstring(lua_State* L, int idx, size_t* len) { const float* v = lua_tovector(L, idx); - if (FFlag::LuauSchubfach) + char s[LUAI_MAXNUM2STR * LUA_VECTOR_SIZE]; + char* e = s; + for (int i = 0; i < LUA_VECTOR_SIZE; ++i) { - char s[LUAI_MAXNUM2STR * LUA_VECTOR_SIZE]; - char* e = s; - for (int i = 0; i < LUA_VECTOR_SIZE; ++i) + if (i != 0) { - if (i != 0) - { - *e++ = ','; - *e++ = ' '; - } - e = luai_num2str(e, v[i]); + *e++ = ','; + *e++ = ' '; } - lua_pushlstring(L, s, e - s); - } - else - { -#if LUA_VECTOR_SIZE == 4 - lua_pushfstring(L, LUA_NUMBER_FMT ", " LUA_NUMBER_FMT ", " LUA_NUMBER_FMT ", " LUA_NUMBER_FMT, v[0], v[1], v[2], v[3]); -#else - lua_pushfstring(L, LUA_NUMBER_FMT ", " LUA_NUMBER_FMT ", " LUA_NUMBER_FMT, v[0], v[1], v[2]); -#endif + e = luai_num2str(e, v[i]); } + lua_pushlstring(L, s, e - s); break; } default: diff --git a/VM/src/ldebug.cpp b/VM/src/ldebug.cpp index a4f93c62..7a9947b7 100644 --- a/VM/src/ldebug.cpp +++ b/VM/src/ldebug.cpp @@ -168,6 +168,11 @@ static int auxgetinfo(lua_State* L, const char* what, lua_Debug* ar, Closure* f, return status; } +int lua_stackdepth(lua_State* L) +{ + return int(L->ci - L->base_ci); +} + int lua_getinfo(lua_State* L, int level, const char* what, lua_Debug* ar) { int status = 0; diff --git a/VM/src/lgc.cpp b/VM/src/lgc.cpp index 8c3a2029..a656854e 100644 --- a/VM/src/lgc.cpp +++ b/VM/src/lgc.cpp @@ -11,6 +11,8 @@ #include "lmem.h" #include "ludata.h" +LUAU_FASTFLAGVARIABLE(LuauGcAdditionalStats, false) + #include #define GC_SWEEPMAX 40 @@ -53,17 +55,28 @@ static void recordGcStateTime(global_State* g, int startgcstate, double seconds, case GCSpause: // record root mark time if we have switched to next state if (g->gcstate == GCSpropagate) + { g->gcstats.currcycle.marktime += seconds; + + if (FFlag::LuauGcAdditionalStats && assist) + g->gcstats.currcycle.markassisttime += seconds; + } break; case GCSpropagate: case GCSpropagateagain: g->gcstats.currcycle.marktime += seconds; + + if (FFlag::LuauGcAdditionalStats && assist) + g->gcstats.currcycle.markassisttime += seconds; break; case GCSatomic: g->gcstats.currcycle.atomictime += seconds; break; case GCSsweep: g->gcstats.currcycle.sweeptime += seconds; + + if (FFlag::LuauGcAdditionalStats && assist) + g->gcstats.currcycle.sweepassisttime += seconds; break; default: LUAU_ASSERT(!"Unexpected GC state"); @@ -78,7 +91,7 @@ static void recordGcStateTime(global_State* g, int startgcstate, double seconds, static void startGcCycleStats(global_State* g) { g->gcstats.currcycle.starttimestamp = lua_clock(); - g->gcstats.currcycle.waittime = g->gcstats.currcycle.starttimestamp - g->gcstats.lastcycle.endtimestamp; + g->gcstats.currcycle.pausetime = g->gcstats.currcycle.starttimestamp - g->gcstats.lastcycle.endtimestamp; } static void finishGcCycleStats(global_State* g) @@ -585,10 +598,21 @@ static size_t atomic(lua_State* L) LUAU_ASSERT(g->gcstate == GCSatomic); size_t work = 0; + double currts = lua_clock(); + double prevts = currts; + /* remark occasional upvalues of (maybe) dead threads */ work += remarkupvals(g); /* traverse objects caught by write barrier and by 'remarkupvals' */ work += propagateall(g); + + if (FFlag::LuauGcAdditionalStats) + { + currts = lua_clock(); + g->gcstats.currcycle.atomictimeupval += currts - prevts; + prevts = currts; + } + /* remark weak tables */ g->gray = g->weak; g->weak = NULL; @@ -596,16 +620,41 @@ static size_t atomic(lua_State* L) markobject(g, L); /* mark running thread */ markmt(g); /* mark basic metatables (again) */ work += propagateall(g); + + if (FFlag::LuauGcAdditionalStats) + { + currts = lua_clock(); + g->gcstats.currcycle.atomictimeweak += currts - prevts; + prevts = currts; + } + /* remark gray again */ g->gray = g->grayagain; g->grayagain = NULL; work += propagateall(g); - work += cleartable(L, g->weak); /* remove collected objects from weak tables */ + + if (FFlag::LuauGcAdditionalStats) + { + currts = lua_clock(); + g->gcstats.currcycle.atomictimegray += currts - prevts; + prevts = currts; + } + + /* remove collected objects from weak tables */ + work += cleartable(L, g->weak); g->weak = NULL; + + if (FFlag::LuauGcAdditionalStats) + { + currts = lua_clock(); + g->gcstats.currcycle.atomictimeclear += currts - prevts; + } + /* flip current white */ g->currentwhite = cast_byte(otherwhite(g)); g->sweepgcopage = g->allgcopages; g->gcstate = GCSsweep; + return work; } @@ -693,6 +742,9 @@ static size_t gcstep(lua_State* L, size_t limit) if (!g->gray) { + if (FFlag::LuauGcAdditionalStats) + g->gcstats.currcycle.propagatework = g->gcstats.currcycle.explicitwork + g->gcstats.currcycle.assistwork; + // perform one iteration over 'gray again' list g->gray = g->grayagain; g->grayagain = NULL; @@ -710,6 +762,10 @@ static size_t gcstep(lua_State* L, size_t limit) if (!g->gray) /* no more `gray' objects */ { + if (FFlag::LuauGcAdditionalStats) + g->gcstats.currcycle.propagateagainwork = + g->gcstats.currcycle.explicitwork + g->gcstats.currcycle.assistwork - g->gcstats.currcycle.propagatework; + g->gcstate = GCSatomic; } break; @@ -811,6 +867,12 @@ static size_t getheaptrigger(global_State* g, size_t heapgoal) void luaC_step(lua_State* L, bool assist) { global_State* g = L->global; + + if (assist) + g->gcstats.currcycle.assistrequests += g->gcstepsize; + else + g->gcstats.currcycle.explicitrequests += g->gcstepsize; + int lim = (g->gcstepsize / 100) * g->gcstepmul; /* how much to work */ LUAU_ASSERT(g->totalbytes >= g->GCthreshold); size_t debt = g->totalbytes - g->GCthreshold; @@ -833,6 +895,11 @@ void luaC_step(lua_State* L, bool assist) recordGcStateTime(g, lastgcstate, lua_clock() - lasttimestamp, assist); + if (lastgcstate == GCSpropagate) + g->gcstats.currcycle.markrequests += g->gcstepsize; + else if (lastgcstate == GCSsweep) + g->gcstats.currcycle.sweeprequests += g->gcstepsize; + // at the end of the last cycle if (g->gcstate == GCSpause) { @@ -844,6 +911,9 @@ void luaC_step(lua_State* L, bool assist) finishGcCycleStats(g); + if (FFlag::LuauGcAdditionalStats) + g->gcstats.currcycle.starttotalsizebytes = g->totalbytes; + g->gcstats.currcycle.heapgoalsizebytes = heapgoal; g->gcstats.currcycle.heaptriggersizebytes = heaptrigger; } diff --git a/VM/src/lgc.h b/VM/src/lgc.h index 253e269f..cbeeebd4 100644 --- a/VM/src/lgc.h +++ b/VM/src/lgc.h @@ -111,13 +111,6 @@ luaC_barrierf(L, obj2gco(p), obj2gco(o)); \ } -// TODO: remove with FFlagLuauGcForwardMetatableBarrier -#define luaC_objbarriert(L, t, o) \ - { \ - if (isblack(obj2gco(t)) && iswhite(obj2gco(o))) \ - luaC_barriertable(L, t, obj2gco(o)); \ - } - #define luaC_upvalbarrier(L, uv, tv) \ { \ if (iscollectable(tv) && iswhite(gcvalue(tv)) && (!(uv) || ((UpVal*)uv)->v != &((UpVal*)uv)->u.value)) \ diff --git a/VM/src/lnumprint.cpp b/VM/src/lnumprint.cpp index 2fd0f1bb..d64e3ca4 100644 --- a/VM/src/lnumprint.cpp +++ b/VM/src/lnumprint.cpp @@ -6,7 +6,6 @@ #include "lcommon.h" #include -#include // TODO: Remove with LuauSchubfach #ifdef _MSC_VER #include @@ -18,8 +17,6 @@ // The code uses the notation from the paper for local variables where appropriate, and refers to paper sections/figures/results. -LUAU_FASTFLAGVARIABLE(LuauSchubfach, false) - // 9.8.2. Precomputed table for 128-bit overestimates of powers of 10 (see figure 3 for table bounds) // To avoid storing 616 128-bit numbers directly we use a technique inspired by Dragonbox implementation and store 16 consecutive // powers using a 128-bit baseline and a bitvector with 1-bit scale and 3-bit offset for the delta between each entry and base*5^k @@ -275,12 +272,6 @@ inline char* trimzero(char* end) char* luai_num2str(char* buf, double n) { - if (!FFlag::LuauSchubfach) - { - snprintf(buf, LUAI_MAXNUM2STR, LUA_NUMBER_FMT, n); - return buf + strlen(buf); - } - // IEEE-754 union { diff --git a/VM/src/lnumutils.h b/VM/src/lnumutils.h index fba07bc3..549b4630 100644 --- a/VM/src/lnumutils.h +++ b/VM/src/lnumutils.h @@ -55,7 +55,6 @@ LUAU_FASTMATH_END #define luai_num2unsigned(i, n) ((i) = (unsigned)(long long)(n)) #endif -#define LUA_NUMBER_FMT "%.14g" /* TODO: Remove with LuauSchubfach */ #define LUAI_MAXNUM2STR 48 LUAI_FUNC char* luai_num2str(char* buf, double n); diff --git a/VM/src/lstate.h b/VM/src/lstate.h index 3ee96718..b2bedb48 100644 --- a/VM/src/lstate.h +++ b/VM/src/lstate.h @@ -77,25 +77,46 @@ typedef struct CallInfo struct GCCycleStats { + size_t starttotalsizebytes = 0; size_t heapgoalsizebytes = 0; size_t heaptriggersizebytes = 0; - double waittime = 0.0; // time from end of the last cycle to the start of a new one + double pausetime = 0.0; // time from end of the last cycle to the start of a new one double starttimestamp = 0.0; double endtimestamp = 0.0; double marktime = 0.0; + double markassisttime = 0.0; + double markmaxexplicittime = 0.0; + size_t markexplicitsteps = 0; + size_t markrequests = 0; double atomicstarttimestamp = 0.0; size_t atomicstarttotalsizebytes = 0; double atomictime = 0.0; + // specific atomic stage parts + double atomictimeupval = 0.0; + double atomictimeweak = 0.0; + double atomictimegray = 0.0; + double atomictimeclear = 0.0; + double sweeptime = 0.0; + double sweepassisttime = 0.0; + double sweepmaxexplicittime = 0.0; + size_t sweepexplicitsteps = 0; + size_t sweeprequests = 0; + + size_t assistrequests = 0; + size_t explicitrequests = 0; size_t assistwork = 0; size_t explicitwork = 0; + size_t propagatework = 0; + size_t propagateagainwork = 0; + size_t endtotalsizebytes = 0; }; diff --git a/VM/src/ltable.cpp b/VM/src/ltable.cpp index 0412ea76..ef0b4b93 100644 --- a/VM/src/ltable.cpp +++ b/VM/src/ltable.cpp @@ -447,7 +447,8 @@ void luaH_free(lua_State* L, Table* t, lua_Page* page) { if (t->node != dummynode) luaM_freearray(L, t->node, sizenode(t), LuaNode, t->memcat); - luaM_freearray(L, t->array, t->sizearray, TValue, t->memcat); + if (t->array) + luaM_freearray(L, t->array, t->sizearray, TValue, t->memcat); luaM_freegco(L, t, sizeof(Table), t->memcat, page); } diff --git a/VM/src/ltablib.cpp b/VM/src/ltablib.cpp index 0d3374ef..00753742 100644 --- a/VM/src/ltablib.cpp +++ b/VM/src/ltablib.cpp @@ -2,6 +2,7 @@ // This code is based on Lua 5.x implementation licensed under MIT License; see lua_LICENSE.txt for details #include "lualib.h" +#include "lapi.h" #include "lstate.h" #include "ltable.h" #include "lstring.h" @@ -9,6 +10,8 @@ #include "ldebug.h" #include "lvm.h" +LUAU_FASTFLAGVARIABLE(LuauTableClone, false) + static int foreachi(lua_State* L) { luaL_checktype(L, 1, LUA_TTABLE); @@ -507,6 +510,23 @@ static int tisfrozen(lua_State* L) return 1; } +static int tclone(lua_State* L) +{ + if (!FFlag::LuauTableClone) + luaG_runerror(L, "table.clone is not available"); + + luaL_checktype(L, 1, LUA_TTABLE); + luaL_argcheck(L, !luaL_getmetafield(L, 1, "__metatable"), 1, "table has a protected metatable"); + + Table* tt = luaH_clone(L, hvalue(L->base)); + + TValue v; + sethvalue(L, &v, tt); + luaA_pushobject(L, &v); + + return 1; +} + static const luaL_Reg tab_funcs[] = { {"concat", tconcat}, {"foreach", foreach}, @@ -524,6 +544,7 @@ static const luaL_Reg tab_funcs[] = { {"clear", tclear}, {"freeze", tfreeze}, {"isfrozen", tisfrozen}, + {"clone", tclone}, {NULL, NULL}, }; diff --git a/bench/gc/test_GC_Boehm_Trees.lua b/bench/gc/test_GC_Boehm_Trees.lua index 1451769f..5abad1d8 100644 --- a/bench/gc/test_GC_Boehm_Trees.lua +++ b/bench/gc/test_GC_Boehm_Trees.lua @@ -74,4 +74,7 @@ function test() end end +bench.runs = 6 +bench.extraRuns = 2 + bench.runCode(test, "GC: Boehm tree") diff --git a/bench/gc/test_GC_Tree_Pruning_Eager.lua b/bench/gc/test_GC_Tree_Pruning_Eager.lua index 514766ae..2111d9ff 100644 --- a/bench/gc/test_GC_Tree_Pruning_Eager.lua +++ b/bench/gc/test_GC_Tree_Pruning_Eager.lua @@ -40,7 +40,7 @@ function test() local tree = { id = 0 } - for i = 1,1000 do + for i = 1,100 do fill_tree(tree, 10) prune_tree(tree, 0) diff --git a/bench/gc/test_GC_Tree_Pruning_Gen.lua b/bench/gc/test_GC_Tree_Pruning_Gen.lua index a8d0f40a..f88bd7f4 100644 --- a/bench/gc/test_GC_Tree_Pruning_Gen.lua +++ b/bench/gc/test_GC_Tree_Pruning_Gen.lua @@ -42,7 +42,7 @@ function test() local tree = { id = 0 } fill_tree(tree, 16) - for i = 1,1000 do + for i = 1,100 do local small_tree = { id = 0 } fill_tree(small_tree, 8) diff --git a/bench/gc/test_GC_Tree_Pruning_Lazy.lua b/bench/gc/test_GC_Tree_Pruning_Lazy.lua index 8cb69192..3ea6bbef 100644 --- a/bench/gc/test_GC_Tree_Pruning_Lazy.lua +++ b/bench/gc/test_GC_Tree_Pruning_Lazy.lua @@ -46,7 +46,7 @@ function test() local tree = { id = 0 } - for i = 1,1000 do + for i = 1,100 do fill_tree(tree, 10) prune_tree(tree, 0) diff --git a/extern/isocline/include/isocline.h b/extern/isocline/include/isocline.h index 0d46cf3f..a7e03ed2 100644 --- a/extern/isocline/include/isocline.h +++ b/extern/isocline/include/isocline.h @@ -259,7 +259,7 @@ void ic_complete_qword( ic_completion_env_t* cenv, const char* prefix, ic_comple /// The `escape_char` is the escaping character, usually `\` but use 0 to not have escape characters. /// The `quote_chars` define the quotes, use NULL for the default `"\'\""` quotes. /// @see ic_complete_word() which uses the default values for `non_word_chars`, `quote_chars` and `\` for escape characters. -void ic_complete_qword_ex( ic_completion_env_t* cenv, const char* prefix, ic_completer_fun_t fun, +void ic_complete_qword_ex( ic_completion_env_t* cenv, const char* prefix, ic_completer_fun_t* fun, ic_is_char_class_fun_t* is_word_char, char escape_char, const char* quote_chars ); /// \} diff --git a/fuzz/number.cpp b/fuzz/number.cpp index 70447409..31c953e3 100644 --- a/fuzz/number.cpp +++ b/fuzz/number.cpp @@ -6,8 +6,6 @@ #include #include -LUAU_FASTFLAG(LuauSchubfach); - #define LUAI_MAXNUM2STR 48 char* luai_num2str(char* buf, double n); @@ -17,8 +15,6 @@ extern "C" int LLVMFuzzerTestOneInput(const uint8_t* Data, size_t Size) if (Size < 8) return 0; - FFlag::LuauSchubfach.value = true; - double num; memcpy(&num, Data, 8); diff --git a/fuzz/proto.cpp b/fuzz/proto.cpp index 912fef23..1022831b 100644 --- a/fuzz/proto.cpp +++ b/fuzz/proto.cpp @@ -59,7 +59,7 @@ void interrupt(lua_State* L, int gc) } } -void* allocate(lua_State* L, void* ud, void* ptr, size_t osize, size_t nsize) +void* allocate(void* ud, void* ptr, size_t osize, size_t nsize) { if (nsize == 0) { diff --git a/tests/Autocomplete.test.cpp b/tests/Autocomplete.test.cpp index 6aadef32..ce890ba8 100644 --- a/tests/Autocomplete.test.cpp +++ b/tests/Autocomplete.test.cpp @@ -15,6 +15,7 @@ LUAU_FASTFLAG(LuauTraceTypesInNonstrictMode2) LUAU_FASTFLAG(LuauSetMetatableDoesNotTimeTravel) LUAU_FASTFLAG(LuauUseCommittingTxnLog) +LUAU_FASTFLAG(LuauTableCloneType) using namespace Luau; @@ -262,7 +263,7 @@ TEST_CASE_FIXTURE(ACFixture, "get_member_completions") auto ac = autocomplete('1'); - CHECK_EQ(16, ac.entryMap.size()); + CHECK_EQ(FFlag::LuauTableCloneType ? 17 : 16, ac.entryMap.size()); CHECK(ac.entryMap.count("find")); CHECK(ac.entryMap.count("pack")); CHECK(!ac.entryMap.count("math")); @@ -2235,7 +2236,7 @@ TEST_CASE_FIXTURE(ACFixture, "autocompleteSource") auto ac = autocompleteSource(frontend, source, Position{1, 24}, nullCallback).result; - CHECK_EQ(16, ac.entryMap.size()); + CHECK_EQ(FFlag::LuauTableCloneType ? 17 : 16, ac.entryMap.size()); CHECK(ac.entryMap.count("find")); CHECK(ac.entryMap.count("pack")); CHECK(!ac.entryMap.count("math")); @@ -2695,8 +2696,6 @@ local r4 = t:bar1(@4) TEST_CASE_FIXTURE(ACFixture, "autocomplete_default_type_parameters") { - ScopedFastFlag luauParseTypeAliasDefaults{"LuauParseTypeAliasDefaults", true}; - check(R"( type A = () -> T )"); @@ -2709,8 +2708,6 @@ type A = () -> T TEST_CASE_FIXTURE(ACFixture, "autocomplete_default_type_pack_parameters") { - ScopedFastFlag luauParseTypeAliasDefaults{"LuauParseTypeAliasDefaults", true}; - check(R"( type A = () -> T )"); @@ -2768,7 +2765,6 @@ TEST_CASE_FIXTURE(ACFixture, "autocomplete_on_string_singletons") TEST_CASE_FIXTURE(ACFixture, "function_in_assignment_has_parentheses_2") { ScopedFastFlag luauAutocompleteAvoidMutation("LuauAutocompleteAvoidMutation", true); - ScopedFastFlag preferToCallFunctionsForIntersects("PreferToCallFunctionsForIntersects", true); check(R"( local bar: ((number) -> number) & (number, number) -> number) diff --git a/tests/Conformance.test.cpp b/tests/Conformance.test.cpp index eb6ca749..63fbb363 100644 --- a/tests/Conformance.test.cpp +++ b/tests/Conformance.test.cpp @@ -241,6 +241,8 @@ TEST_CASE("Math") TEST_CASE("Table") { + ScopedFastFlag sff("LuauTableClone", true); + runConformance("nextvar.lua"); } @@ -465,6 +467,8 @@ static void populateRTTI(lua_State* L, Luau::TypeId type) TEST_CASE("Types") { + ScopedFastFlag sff("LuauTableCloneType", true); + runConformance("types.lua", [](lua_State* L) { Luau::NullModuleResolver moduleResolver; Luau::InternalErrorReporter iceHandler; @@ -959,8 +963,6 @@ TEST_CASE("Coverage") TEST_CASE("StringConversion") { - ScopedFastFlag sff{"LuauSchubfach", true}; - runConformance("strconv.lua"); } diff --git a/tests/Linter.test.cpp b/tests/Linter.test.cpp index ab19cea3..4d6c207c 100644 --- a/tests/Linter.test.cpp +++ b/tests/Linter.test.cpp @@ -157,6 +157,113 @@ return bar() CHECK_EQ(result.warnings[0].text, "Global 'foo' is only used in the enclosing function 'bar'; consider changing it to local"); } +TEST_CASE_FIXTURE(Fixture, "GlobalAsLocalMultiFx") +{ + ScopedFastFlag sff{"LuauLintGlobalNeverReadBeforeWritten", true}; + LintResult result = lint(R"( +function bar() + foo = 6 + return foo +end + +function baz() + foo = 6 + return foo +end + +return bar() + baz() +)"); + + REQUIRE_EQ(result.warnings.size(), 1); + CHECK_EQ(result.warnings[0].text, "Global 'foo' is never read before being written. Consider changing it to local"); +} + +TEST_CASE_FIXTURE(Fixture, "GlobalAsLocalMultiFxWithRead") +{ + ScopedFastFlag sff{"LuauLintGlobalNeverReadBeforeWritten", true}; + LintResult result = lint(R"( +function bar() + foo = 6 + return foo +end + +function baz() + foo = 6 + return foo +end + +function read() + print(foo) +end + +return bar() + baz() + read() +)"); + + CHECK_EQ(result.warnings.size(), 0); +} + +TEST_CASE_FIXTURE(Fixture, "GlobalAsLocalWithConditional") +{ + ScopedFastFlag sff{"LuauLintGlobalNeverReadBeforeWritten", true}; + LintResult result = lint(R"( +function bar() + if true then foo = 6 end + return foo +end + +function baz() + foo = 6 + return foo +end + +return bar() + baz() +)"); + + CHECK_EQ(result.warnings.size(), 0); +} + +TEST_CASE_FIXTURE(Fixture, "GlobalAsLocal3WithConditionalRead") +{ + ScopedFastFlag sff{"LuauLintGlobalNeverReadBeforeWritten", true}; + LintResult result = lint(R"( +function bar() + foo = 6 + return foo +end + +function baz() + foo = 6 + return foo +end + +function read() + if false then print(foo) end +end + +return bar() + baz() + read() +)"); + + CHECK_EQ(result.warnings.size(), 0); +} + +TEST_CASE_FIXTURE(Fixture, "GlobalAsLocalInnerRead") +{ + ScopedFastFlag sff{"LuauLintGlobalNeverReadBeforeWritten", true}; + LintResult result = lint(R"( +function foo() + local f = function() return bar end + f() + bar = 42 +end + +function baz() bar = 0 end + +return foo() + baz() +)"); + + CHECK_EQ(result.warnings.size(), 0); +} + TEST_CASE_FIXTURE(Fixture, "GlobalAsLocalMulti") { LintResult result = lint(R"( diff --git a/tests/Parser.test.cpp b/tests/Parser.test.cpp index 77e49ce3..7f6a6c0d 100644 --- a/tests/Parser.test.cpp +++ b/tests/Parser.test.cpp @@ -1988,8 +1988,6 @@ TEST_CASE_FIXTURE(Fixture, "function_type_matching_parenthesis") TEST_CASE_FIXTURE(Fixture, "parse_type_alias_default_type") { - ScopedFastFlag luauParseTypeAliasDefaults{"LuauParseTypeAliasDefaults", true}; - AstStat* stat = parse(R"( type A = {} type B = {} @@ -2005,8 +2003,6 @@ type G = (U...) -> T... TEST_CASE_FIXTURE(Fixture, "parse_type_alias_default_type_errors") { - ScopedFastFlag luauParseTypeAliasDefaults{"LuauParseTypeAliasDefaults", true}; - matchParseError("type Y = {}", "Expected default type after type name", Location{{0, 20}, {0, 21}}); matchParseError("type Y = {}", "Expected default type pack after type pack name", Location{{0, 29}, {0, 30}}); matchParseError("type Y number> = {}", "Expected type pack after '=', got type", Location{{0, 14}, {0, 32}}); @@ -2574,8 +2570,6 @@ do end TEST_CASE_FIXTURE(Fixture, "recover_expected_type_pack") { - ScopedFastFlag luauParseTypeAliasDefaults{"LuauParseTypeAliasDefaults", true}; - ParseResult result = tryParse(R"( type Y = (T...) -> U... )"); diff --git a/tests/ToDot.test.cpp b/tests/ToDot.test.cpp index 0ca9c994..29bdd866 100644 --- a/tests/ToDot.test.cpp +++ b/tests/ToDot.test.cpp @@ -96,8 +96,6 @@ n2 [label="number"]; TEST_CASE_FIXTURE(Fixture, "function") { - ScopedFastFlag luauQuantifyInPlace2{"LuauQuantifyInPlace2", true}; - CheckResult result = check(R"( local function f(a, ...: string) return a end )"); diff --git a/tests/ToString.test.cpp b/tests/ToString.test.cpp index bbb26291..6713a589 100644 --- a/tests/ToString.test.cpp +++ b/tests/ToString.test.cpp @@ -500,8 +500,6 @@ TEST_CASE_FIXTURE(Fixture, "toStringNamedFunction_map") TEST_CASE_FIXTURE(Fixture, "toStringNamedFunction_generic_pack") { - ScopedFastFlag luauTypeAliasDefaults{"LuauTypeAliasDefaults", true}; - CheckResult result = check(R"( local function f(a: number, b: string) end local function test(...: T...): U... diff --git a/tests/Transpiler.test.cpp b/tests/Transpiler.test.cpp index 332aba9e..5f0295b0 100644 --- a/tests/Transpiler.test.cpp +++ b/tests/Transpiler.test.cpp @@ -641,9 +641,6 @@ TEST_CASE_FIXTURE(Fixture, "transpile_to_string") TEST_CASE_FIXTURE(Fixture, "transpile_type_alias_default_type_parameters") { - ScopedFastFlag luauParseTypeAliasDefaults{"LuauParseTypeAliasDefaults", true}; - ScopedFastFlag luauTypeAliasDefaults{"LuauTypeAliasDefaults", true}; - std::string code = R"( type Packed = (T, U, V...)->(W...) local a: Packed diff --git a/tests/TypeInfer.aliases.test.cpp b/tests/TypeInfer.aliases.test.cpp index 31d7ef10..d584eb2d 100644 --- a/tests/TypeInfer.aliases.test.cpp +++ b/tests/TypeInfer.aliases.test.cpp @@ -625,9 +625,8 @@ TEST_CASE_FIXTURE(Fixture, "forward_declared_alias_is_not_clobbered_by_prior_uni ScopedFastFlag sff[] = { {"LuauTwoPassAliasDefinitionFix", true}, - // We also force these two flags because this surfaced an unfortunate interaction. + // We also force this flag because it surfaced an unfortunate interaction. {"LuauErrorRecoveryType", true}, - {"LuauQuantifyInPlace2", true}, }; CheckResult result = check(R"( diff --git a/tests/TypeInfer.builtins.test.cpp b/tests/TypeInfer.builtins.test.cpp index f3dfb214..bf990770 100644 --- a/tests/TypeInfer.builtins.test.cpp +++ b/tests/TypeInfer.builtins.test.cpp @@ -934,4 +934,31 @@ TEST_CASE_FIXTURE(Fixture, "assert_returns_false_and_string_iff_it_knows_the_fir CHECK_EQ("(nil) -> nil", toString(requireType("f"))); } +TEST_CASE_FIXTURE(Fixture, "table_freeze_is_generic") +{ + CheckResult result = check(R"( + local t1: {a: number} = {a = 42} + local t2: {b: string} = {b = "hello"} + local t3: {boolean} = {false, true} + + local tf1 = table.freeze(t1) + local tf2 = table.freeze(t2) + local tf3 = table.freeze(t3) + + local a = tf1.a + local b = tf2.b + local c = tf3[2] + + local d = tf1.b + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ("Key 'b' not found in table '{| a: number |}'", toString(result.errors[0])); + + CHECK_EQ("number", toString(requireType("a"))); + CHECK_EQ("string", toString(requireType("b"))); + CHECK_EQ("boolean", toString(requireType("c"))); + CHECK_EQ("*unknown*", toString(requireType("d"))); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.generics.test.cpp b/tests/TypeInfer.generics.test.cpp index f8fccf6b..c482847b 100644 --- a/tests/TypeInfer.generics.test.cpp +++ b/tests/TypeInfer.generics.test.cpp @@ -697,4 +697,93 @@ end LUAU_REQUIRE_NO_ERRORS(result); } +TEST_CASE_FIXTURE(Fixture, "generic_functions_should_be_memory_safe") +{ + ScopedFastFlag sffs[] = { + { "LuauTableSubtypingVariance2", true }, + { "LuauUnsealedTableLiteral", true }, + { "LuauPropertiesGetExpectedType", true }, + { "LuauRecursiveTypeParameterRestriction", true }, + }; + + CheckResult result = check(R"( +--!strict +-- At one point this produced a UAF +type T = { a: U, b: a } +type U = { c: T?, d : a } +local x: T = { a = { c = nil, d = 5 }, b = 37 } +x.a.c = x +local y: T = { a = { c = nil, d = 5 }, b = 37 } +y.a.c = y + )"); + + LUAU_REQUIRE_ERRORS(result); + CHECK_EQ(toString(result.errors[0]), + R"(Type 'y' could not be converted into 'T' +caused by: + Property 'a' is not compatible. Type '{ c: T?, d: number }' could not be converted into 'U' +caused by: + Property 'd' is not compatible. Type 'number' could not be converted into 'string')"); +} + +TEST_CASE_FIXTURE(Fixture, "generic_type_pack_unification1") +{ + ScopedFastFlag sff{"LuauTxnLogSeesTypePacks2", true}; + + CheckResult result = check(R"( +--!strict +type Dispatcher = { + useMemo: (create: () -> T...) -> T... +} + +local TheDispatcher: Dispatcher = { + useMemo = function(create: () -> U...): U... + return create() + end +} + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "generic_type_pack_unification2") +{ + ScopedFastFlag sff{"LuauTxnLogSeesTypePacks2", true}; + + CheckResult result = check(R"( +--!strict +type Dispatcher = { + useMemo: (create: () -> T...) -> T... +} + +local TheDispatcher: Dispatcher = { + useMemo = function(create) + return create() + end +} + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "generic_type_pack_unification3") +{ + ScopedFastFlag sff{"LuauTxnLogSeesTypePacks2", true}; + + CheckResult result = check(R"( +--!strict +type Dispatcher = { + useMemo: (arg: S, create: (S) -> T...) -> T... +} + +local TheDispatcher: Dispatcher = { + useMemo = function(arg: T, create: (T) -> U...): U... + return create(arg) + end +} + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.provisional.test.cpp b/tests/TypeInfer.provisional.test.cpp index eee0e0f1..2e16b21e 100644 --- a/tests/TypeInfer.provisional.test.cpp +++ b/tests/TypeInfer.provisional.test.cpp @@ -8,7 +8,6 @@ #include LUAU_FASTFLAG(LuauEqConstraint) -LUAU_FASTFLAG(LuauQuantifyInPlace2) using namespace Luau; @@ -40,16 +39,6 @@ TEST_CASE_FIXTURE(Fixture, "typeguard_inference_incomplete") end )"; - const std::string old_expected = R"( - function f(a:{fn:()->(free,free...)}): () - if type(a) == 'boolean'then - local a1:boolean=a - elseif a.fn()then - local a2:{fn:()->(free,free...)}=a - end - end - )"; - const std::string expected = R"( function f(a:{fn:()->(a,b...)}): () if type(a) == 'boolean'then @@ -60,10 +49,7 @@ TEST_CASE_FIXTURE(Fixture, "typeguard_inference_incomplete") end )"; - if (FFlag::LuauQuantifyInPlace2) - CHECK_EQ(expected, decorateWithTypes(code)); - else - CHECK_EQ(old_expected, decorateWithTypes(code)); + CHECK_EQ(expected, decorateWithTypes(code)); } TEST_CASE_FIXTURE(Fixture, "xpcall_returns_what_f_returns") @@ -135,46 +121,6 @@ TEST_CASE_FIXTURE(Fixture, "setmetatable_constrains_free_type_into_free_table") CHECK_EQ("number", toString(tm->givenType)); } -TEST_CASE_FIXTURE(Fixture, "pass_a_union_of_tables_to_a_function_that_requires_a_table") -{ - CheckResult result = check(R"( - local a: {x: number, y: number, [any]: any} | {y: number} - - function f(t) - t.y = 1 - return t - end - - local b = f(a) - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - - // :( - // Should be the same as the type of a - REQUIRE_EQ("{| y: number |}", toString(requireType("b"))); -} - -TEST_CASE_FIXTURE(Fixture, "pass_a_union_of_tables_to_a_function_that_requires_a_table_2") -{ - CheckResult result = check(R"( - local a: {y: number} | {x: number, y: number, [any]: any} - - function f(t) - t.y = 1 - return t - end - - local b = f(a) - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - - // :( - // Should be the same as the type of a - REQUIRE_EQ("{| [any]: any, x: number, y: number |}", toString(requireType("b"))); -} - // Luau currently doesn't yet know how to allow assignments when the binding was refined. TEST_CASE_FIXTURE(Fixture, "while_body_are_also_refined") { @@ -557,25 +503,6 @@ TEST_CASE_FIXTURE(Fixture, "bail_early_on_typescript_port_of_Result_type" * doct } } -TEST_CASE_FIXTURE(Fixture, "table_subtyping_shouldn't_add_optional_properties_to_sealed_tables") -{ - CheckResult result = check(R"( - --!strict - local function setNumber(t: { p: number? }, x:number) t.p = x end - local function getString(t: { p: string? }):string return t.p or "" end - -- This shouldn't type-check! - local function oh(x:number): string - local t: {} = {} - setNumber(t, x) - return getString(t) - end - local s: string = oh(37) - )"); - - // Really this should return an error, but it doesn't - LUAU_REQUIRE_NO_ERRORS(result); -} - // Should be in TypeInfer.tables.test.cpp // It's unsound to instantiate tables containing generic methods, // since mutating properties means table properties should be invariant. @@ -600,25 +527,9 @@ TEST_CASE_FIXTURE(Fixture, "invariant_table_properties_means_instantiating_table LUAU_REQUIRE_NO_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "self_recursive_instantiated_param") -{ - // Mutability in type function application right now can create strange recursive types - // TODO: instantiation right now is problematic, in this example should either leave the Table type alone - // or it should rename the type to 'Self' so that the result will be 'Self
' - CheckResult result = check(R"( -type Table = { a: number } -type Self = T -local a: Self
- )"); - - LUAU_REQUIRE_NO_ERRORS(result); - CHECK_EQ(toString(requireType("a")), "Table
"); -} - TEST_CASE_FIXTURE(Fixture, "do_not_ice_when_trying_to_pick_first_of_generic_type_pack") { ScopedFastFlag sff[]{ - {"LuauQuantifyInPlace2", true}, {"LuauReturnAnyInsteadOfICE", true}, }; @@ -664,8 +575,6 @@ TEST_CASE_FIXTURE(Fixture, "specialization_binds_with_prototypes_too_early") TEST_CASE_FIXTURE(Fixture, "weird_fail_to_unify_type_pack") { - ScopedFastFlag sff{"LuauQuantifyInPlace2", true}; - CheckResult result = check(R"( local function f() return end local g = function() return f() end @@ -676,8 +585,6 @@ TEST_CASE_FIXTURE(Fixture, "weird_fail_to_unify_type_pack") TEST_CASE_FIXTURE(Fixture, "weird_fail_to_unify_variadic_pack") { - ScopedFastFlag sff{"LuauQuantifyInPlace2", true}; - CheckResult result = check(R"( --!strict local function f(...) return ... end diff --git a/tests/TypeInfer.refinements.test.cpp b/tests/TypeInfer.refinements.test.cpp index bff8926c..a5147d56 100644 --- a/tests/TypeInfer.refinements.test.cpp +++ b/tests/TypeInfer.refinements.test.cpp @@ -8,7 +8,6 @@ LUAU_FASTFLAG(LuauDiscriminableUnions2) LUAU_FASTFLAG(LuauWeakEqConstraint) -LUAU_FASTFLAG(LuauQuantifyInPlace2) using namespace Luau; @@ -1179,20 +1178,14 @@ TEST_CASE_FIXTURE(RefinementClassFixture, "typeguard_cast_free_table_to_vector") { LUAU_REQUIRE_ERROR_COUNT(1, result); - if (FFlag::LuauQuantifyInPlace2) - CHECK_EQ("Type '{+ X: a, Y: b, Z: c +}' could not be converted into 'Instance'", toString(result.errors[0])); - else - CHECK_EQ("Type '{- X: a, Y: b, Z: c -}' could not be converted into 'Instance'", toString(result.errors[0])); + CHECK_EQ("Type '{+ X: a, Y: b, Z: c +}' could not be converted into 'Instance'", toString(result.errors[0])); } CHECK_EQ("Vector3", toString(requireTypeAtPosition({5, 28}))); // type(vec) == "vector" CHECK_EQ("*unknown*", toString(requireTypeAtPosition({7, 28}))); // typeof(vec) == "Instance" - if (FFlag::LuauQuantifyInPlace2) - CHECK_EQ("{+ X: a, Y: b, Z: c +}", toString(requireTypeAtPosition({9, 28}))); // type(vec) ~= "vector" and typeof(vec) ~= "Instance" - else - CHECK_EQ("{- X: a, Y: b, Z: c -}", toString(requireTypeAtPosition({9, 28}))); // type(vec) ~= "vector" and typeof(vec) ~= "Instance" + CHECK_EQ("{+ X: a, Y: b, Z: c +}", toString(requireTypeAtPosition({9, 28}))); // type(vec) ~= "vector" and typeof(vec) ~= "Instance" } TEST_CASE_FIXTURE(RefinementClassFixture, "typeguard_cast_instance_or_vector3_to_vector") diff --git a/tests/TypeInfer.singletons.test.cpp b/tests/TypeInfer.singletons.test.cpp index 856549bd..3ed536ea 100644 --- a/tests/TypeInfer.singletons.test.cpp +++ b/tests/TypeInfer.singletons.test.cpp @@ -465,30 +465,32 @@ TEST_CASE_FIXTURE(Fixture, "widen_the_supertype_if_it_is_free_and_subtype_has_si CHECK_EQ("((string) -> (b...), a) -> ()", toString(requireType("foo"))); } -// TEST_CASE_FIXTURE(Fixture, "return_type_of_f_is_not_widened") -// { -// ScopedFastFlag sff[]{ -// {"LuauParseSingletonTypes", true}, -// {"LuauSingletonTypes", true}, -// {"LuauDiscriminableUnions2", true}, -// {"LuauEqConstraint", true}, -// {"LuauWidenIfSupertypeIsFree", true}, -// {"LuauWeakEqConstraint", false}, -// }; +TEST_CASE_FIXTURE(Fixture, "return_type_of_f_is_not_widened") +{ + ScopedFastFlag sff[]{ + {"LuauParseSingletonTypes", true}, + {"LuauSingletonTypes", true}, + {"LuauDiscriminableUnions2", true}, + {"LuauEqConstraint", true}, + {"LuauWidenIfSupertypeIsFree", true}, + {"LuauWeakEqConstraint", false}, + {"LuauDoNotAccidentallyDependOnPointerOrdering", true} + }; -// CheckResult result = check(R"( -// local function foo(f, x): "hello"? -- anyone there? -// return if x == "hi" -// then f(x) -// else nil -// end -// )"); + CheckResult result = check(R"( + local function foo(f, x): "hello"? -- anyone there? + return if x == "hi" + then f(x) + else nil + end + )"); -// LUAU_REQUIRE_NO_ERRORS(result); + LUAU_REQUIRE_NO_ERRORS(result); -// CHECK_EQ(R"("hi")", toString(requireTypeAtPosition({3, 23}))); -// CHECK_EQ(R"(((string) -> ("hello"?, b...), a) -> "hello"?)", toString(requireType("foo"))); -// } + CHECK_EQ(R"("hi")", toString(requireTypeAtPosition({3, 23}))); + CHECK_EQ(R"(((string) -> (a, c...), b) -> "hello"?)", toString(requireType("foo"))); + // CHECK_EQ(R"(((string) -> ("hello"?, b...), a) -> "hello"?)", toString(requireType("foo"))); +} TEST_CASE_FIXTURE(Fixture, "widening_happens_almost_everywhere") { diff --git a/tests/TypeInfer.tables.test.cpp b/tests/TypeInfer.tables.test.cpp index aa949789..da035ba1 100644 --- a/tests/TypeInfer.tables.test.cpp +++ b/tests/TypeInfer.tables.test.cpp @@ -1219,13 +1219,12 @@ TEST_CASE_FIXTURE(Fixture, "passing_compatible_unions_to_a_generic_table_without { CheckResult result = check(R"( type A = {x: number, y: number, [any]: any} | {y: number} - local a: A function f(t) t.y = 1 end - f(a) + f({y = 5} :: A) )"); LUAU_REQUIRE_NO_ERRORS(result); @@ -2165,6 +2164,44 @@ b() CHECK_EQ(toString(result.errors[0]), R"(Cannot call non-function t1 where t1 = { @metatable { __call: t1 }, { } })"); } +TEST_CASE_FIXTURE(Fixture, "table_subtyping_shouldn't_add_optional_properties_to_sealed_tables") +{ + ScopedFastFlag sffs[] = { + {"LuauTableSubtypingVariance2", true}, + {"LuauSubtypingAddOptPropsToUnsealedTables", true}, + }; + + CheckResult result = check(R"( + --!strict + local function setNumber(t: { p: number? }, x:number) t.p = x end + local function getString(t: { p: string? }):string return t.p or "" end + -- This shouldn't type-check! + local function oh(x:number): string + local t: {} = {} + setNumber(t, x) + return getString(t) + end + local s: string = oh(37) + )"); + + LUAU_REQUIRE_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "top_table_type") +{ + CheckResult result = check(R"( + --!strict + type Table = { [any] : any } + type HasTable = { p: Table? } + type HasHasTable = { p: HasTable? } + local t : Table = { p = 5 } + local u : HasTable = { p = { p = 5 } } + local v : HasHasTable = { p = { p = { p = 5 } } } + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + TEST_CASE_FIXTURE(Fixture, "length_operator_union") { CheckResult result = check(R"( @@ -2257,4 +2294,44 @@ TEST_CASE_FIXTURE(Fixture, "confusing_indexing") CHECK_EQ("number | string", toString(requireType("foo"))); } +TEST_CASE_FIXTURE(Fixture, "pass_a_union_of_tables_to_a_function_that_requires_a_table") +{ + ScopedFastFlag sff{"LuauDifferentOrderOfUnificationDoesntMatter", true}; + + CheckResult result = check(R"( + local a: {x: number, y: number, [any]: any} | {y: number} + + function f(t) + t.y = 1 + return t + end + + local b = f(a) + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + REQUIRE_EQ("{| [any]: any, x: number, y: number |} | {| y: number |}", toString(requireType("b"))); +} + +TEST_CASE_FIXTURE(Fixture, "pass_a_union_of_tables_to_a_function_that_requires_a_table_2") +{ + ScopedFastFlag sff{"LuauDifferentOrderOfUnificationDoesntMatter", true}; + + CheckResult result = check(R"( + local a: {y: number} | {x: number, y: number, [any]: any} + + function f(t) + t.y = 1 + return t + end + + local b = f(a) + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + REQUIRE_EQ("{| [any]: any, x: number, y: number |} | {| y: number |}", toString(requireType("b"))); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.test.cpp b/tests/TypeInfer.test.cpp index f44d9fd8..f63579b5 100644 --- a/tests/TypeInfer.test.cpp +++ b/tests/TypeInfer.test.cpp @@ -4044,6 +4044,49 @@ type t0 = any CHECK(ttv->instantiatedTypeParams.empty()); } +TEST_CASE_FIXTURE(Fixture, "instantiate_table_cloning_2") +{ + ScopedFastFlag sff{"LuauOnlyMutateInstantiatedTables", true}; + + CheckResult result = check(R"( +type X = T +type K = X +)"); + + LUAU_REQUIRE_NO_ERRORS(result); + + std::optional ty = requireType("math"); + REQUIRE(ty); + + const TableTypeVar* ttv = get(*ty); + REQUIRE(ttv); + CHECK(ttv->instantiatedTypeParams.empty()); +} + +TEST_CASE_FIXTURE(Fixture, "instantiate_table_cloning_3") +{ + ScopedFastFlag sff{"LuauOnlyMutateInstantiatedTables", true}; + + CheckResult result = check(R"( +type X = T +local a = {} +a.x = 4 +local b: X +a.y = 5 +local c: X +c = b +)"); + + LUAU_REQUIRE_NO_ERRORS(result); + + std::optional ty = requireType("a"); + REQUIRE(ty); + + const TableTypeVar* ttv = get(*ty); + REQUIRE(ttv); + CHECK(ttv->instantiatedTypeParams.empty()); +} + TEST_CASE_FIXTURE(Fixture, "bound_free_table_export_is_ok") { CheckResult result = check(R"( @@ -4065,6 +4108,21 @@ return m LUAU_REQUIRE_NO_ERRORS(result); } +TEST_CASE_FIXTURE(Fixture, "self_recursive_instantiated_param") +{ + ScopedFastFlag sff{"LuauOnlyMutateInstantiatedTables", true}; + + // Mutability in type function application right now can create strange recursive types + CheckResult result = check(R"( +type Table = { a: number } +type Self = T +local a: Self
+ )"); + + LUAU_REQUIRE_NO_ERRORS(result); + CHECK_EQ(toString(requireType("a")), "Table"); +} + TEST_CASE_FIXTURE(Fixture, "no_persistent_typelevel_change") { TypeId mathTy = requireType(typeChecker.globalScope, "math"); @@ -5284,4 +5342,17 @@ TEST_CASE_FIXTURE(Fixture, "inferred_properties_of_a_table_should_start_with_the LUAU_REQUIRE_NO_ERRORS(result); } +TEST_CASE_FIXTURE(Fixture, "global_singleton_types_are_sealed") +{ + CheckResult result = check(R"( +local function f(x: string) + local p = x:split('a') + p = table.pack(table.unpack(p, 1, #p - 1)) + return p +end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.tryUnify.test.cpp b/tests/TypeInfer.tryUnify.test.cpp index c9bf5103..f6ee3ccc 100644 --- a/tests/TypeInfer.tryUnify.test.cpp +++ b/tests/TypeInfer.tryUnify.test.cpp @@ -7,8 +7,6 @@ #include "doctest.h" -LUAU_FASTFLAG(LuauQuantifyInPlace2); - using namespace Luau; LUAU_FASTFLAG(LuauUseCommittingTxnLog) @@ -167,10 +165,7 @@ TEST_CASE_FIXTURE(TryUnifyFixture, "typepack_unification_should_trim_free_tails" )"); LUAU_REQUIRE_ERROR_COUNT(1, result); - if (FFlag::LuauQuantifyInPlace2) - CHECK_EQ("(number) -> boolean", toString(requireType("f"))); - else - CHECK_EQ("(number) -> (boolean)", toString(requireType("f"))); + CHECK_EQ("(number) -> boolean", toString(requireType("f"))); } TEST_CASE_FIXTURE(TryUnifyFixture, "variadic_type_pack_unification") diff --git a/tests/TypeInfer.typePacks.cpp b/tests/TypeInfer.typePacks.cpp index cbe2e48f..6b96f449 100644 --- a/tests/TypeInfer.typePacks.cpp +++ b/tests/TypeInfer.typePacks.cpp @@ -622,9 +622,6 @@ type Other = Packed TEST_CASE_FIXTURE(Fixture, "type_alias_default_type_explicit") { - ScopedFastFlag luauParseTypeAliasDefaults{"LuauParseTypeAliasDefaults", true}; - ScopedFastFlag luauTypeAliasDefaults{"LuauTypeAliasDefaults", true}; - CheckResult result = check(R"( type Y = { a: T, b: U } @@ -654,9 +651,6 @@ local c: Y = { a = "s" } TEST_CASE_FIXTURE(Fixture, "type_alias_default_type_self") { - ScopedFastFlag luauParseTypeAliasDefaults{"LuauParseTypeAliasDefaults", true}; - ScopedFastFlag luauTypeAliasDefaults{"LuauTypeAliasDefaults", true}; - CheckResult result = check(R"( type Y = { a: T, b: U } @@ -682,9 +676,6 @@ local a: Y TEST_CASE_FIXTURE(Fixture, "type_alias_default_type_chained") { - ScopedFastFlag luauParseTypeAliasDefaults{"LuauParseTypeAliasDefaults", true}; - ScopedFastFlag luauTypeAliasDefaults{"LuauTypeAliasDefaults", true}; - CheckResult result = check(R"( type Y = { a: T, b: U, c: V } @@ -700,9 +691,6 @@ local b: Y TEST_CASE_FIXTURE(Fixture, "type_alias_default_type_pack_explicit") { - ScopedFastFlag luauParseTypeAliasDefaults{"LuauParseTypeAliasDefaults", true}; - ScopedFastFlag luauTypeAliasDefaults{"LuauTypeAliasDefaults", true}; - CheckResult result = check(R"( type Y = { a: (T...) -> () } local a: Y<> @@ -715,9 +703,6 @@ local a: Y<> TEST_CASE_FIXTURE(Fixture, "type_alias_default_type_pack_self_ty") { - ScopedFastFlag luauParseTypeAliasDefaults{"LuauParseTypeAliasDefaults", true}; - ScopedFastFlag luauTypeAliasDefaults{"LuauTypeAliasDefaults", true}; - CheckResult result = check(R"( type Y = { a: T, b: (U...) -> T } @@ -731,9 +716,6 @@ local a: Y TEST_CASE_FIXTURE(Fixture, "type_alias_default_type_pack_self_tp") { - ScopedFastFlag luauParseTypeAliasDefaults{"LuauParseTypeAliasDefaults", true}; - ScopedFastFlag luauTypeAliasDefaults{"LuauTypeAliasDefaults", true}; - CheckResult result = check(R"( type Y = { a: (T...) -> U... } local a: Y @@ -746,9 +728,6 @@ local a: Y TEST_CASE_FIXTURE(Fixture, "type_alias_default_type_pack_self_chained_tp") { - ScopedFastFlag luauParseTypeAliasDefaults{"LuauParseTypeAliasDefaults", true}; - ScopedFastFlag luauTypeAliasDefaults{"LuauTypeAliasDefaults", true}; - CheckResult result = check(R"( type Y = { a: (T...) -> U..., b: (T...) -> V... } local a: Y @@ -761,9 +740,6 @@ local a: Y TEST_CASE_FIXTURE(Fixture, "type_alias_default_mixed_self") { - ScopedFastFlag luauParseTypeAliasDefaults{"LuauParseTypeAliasDefaults", true}; - ScopedFastFlag luauTypeAliasDefaults{"LuauTypeAliasDefaults", true}; - CheckResult result = check(R"( type Y = { a: (T, U, V...) -> W... } local a: Y @@ -782,9 +758,6 @@ local d: Y ()> TEST_CASE_FIXTURE(Fixture, "type_alias_default_type_errors") { - ScopedFastFlag luauParseTypeAliasDefaults{"LuauParseTypeAliasDefaults", true}; - ScopedFastFlag luauTypeAliasDefaults{"LuauTypeAliasDefaults", true}; - CheckResult result = check(R"( type Y = { a: T } local a: Y = { a = 2 } @@ -834,9 +807,6 @@ local a: Y<...number> TEST_CASE_FIXTURE(Fixture, "type_alias_default_export") { - ScopedFastFlag luauParseTypeAliasDefaults{"LuauParseTypeAliasDefaults", true}; - ScopedFastFlag luauTypeAliasDefaults{"LuauTypeAliasDefaults", true}; - fileResolver.source["Module/Types"] = R"( export type A = { a: T, b: U } export type B = { a: T, b: U } @@ -882,9 +852,6 @@ local h: Types.H<> TEST_CASE_FIXTURE(Fixture, "type_alias_default_type_skip_brackets") { - ScopedFastFlag luauParseTypeAliasDefaults{"LuauParseTypeAliasDefaults", true}; - ScopedFastFlag luauTypeAliasDefaults{"LuauTypeAliasDefaults", true}; - CheckResult result = check(R"( type Y = (T...) -> number local a: Y @@ -897,9 +864,6 @@ local a: Y TEST_CASE_FIXTURE(Fixture, "type_alias_defaults_confusing_types") { - ScopedFastFlag luauParseTypeAliasDefaults{"LuauParseTypeAliasDefaults", true}; - ScopedFastFlag luauTypeAliasDefaults{"LuauTypeAliasDefaults", true}; - CheckResult result = check(R"( type A = (T, V...) -> (U, W...) type B = A @@ -914,9 +878,6 @@ type C = A TEST_CASE_FIXTURE(Fixture, "type_alias_defaults_recursive_type") { - ScopedFastFlag luauParseTypeAliasDefaults{"LuauParseTypeAliasDefaults", true}; - ScopedFastFlag luauTypeAliasDefaults{"LuauTypeAliasDefaults", true}; - CheckResult result = check(R"( type F ()> = (K) -> V type R = { m: F } diff --git a/tests/TypeInfer.unionTypes.test.cpp b/tests/TypeInfer.unionTypes.test.cpp index 3b53ddfe..0e0b6ebb 100644 --- a/tests/TypeInfer.unionTypes.test.cpp +++ b/tests/TypeInfer.unionTypes.test.cpp @@ -400,10 +400,10 @@ local e = a.z CHECK_EQ("Type 'A | B | C | D' does not have key 'z'", toString(result.errors[3])); } -TEST_CASE_FIXTURE(Fixture, "unify_sealed_table_union_check") +TEST_CASE_FIXTURE(Fixture, "unify_unsealed_table_union_check") { CheckResult result = check(R"( -local x: { x: number } = { x = 3 } +local x = { x = 3 } type A = number? type B = string? local y: { x: number, y: A | B } @@ -413,7 +413,7 @@ y = x LUAU_REQUIRE_NO_ERRORS(result); result = check(R"( -local x: { x: number } = { x = 3 } +local x = { x = 3 } local a: number? = 2 local y = {} @@ -426,6 +426,31 @@ y = x LUAU_REQUIRE_NO_ERRORS(result); } +TEST_CASE_FIXTURE(Fixture, "unify_sealed_table_union_check") +{ + ScopedFastFlag sffs[] = { + {"LuauTableSubtypingVariance2", true}, + {"LuauUnsealedTableLiteral", true}, + {"LuauSubtypingAddOptPropsToUnsealedTables", true}, + }; + + CheckResult result = check(R"( + -- the difference between this and unify_unsealed_table_union_check is the type annotation on x +local t = { x = 3, y = true } +local x: { x: number } = t +type A = number? +type B = string? +local y: { x: number, y: A | B } +-- Shouldn't typecheck! +y = x +-- If it does, we can convert any type to any other type +y.y = 5 +local oh : boolean = t.y + )"); + + LUAU_REQUIRE_ERRORS(result); +} + TEST_CASE_FIXTURE(Fixture, "error_detailed_union_part") { CheckResult result = check(R"( diff --git a/tests/conformance/nextvar.lua b/tests/conformance/nextvar.lua index 94ba5ccf..e85fcbe8 100644 --- a/tests/conformance/nextvar.lua +++ b/tests/conformance/nextvar.lua @@ -512,4 +512,42 @@ do assert(#t == 7) end +-- test clone +do + local t = {a = 1, b = 2, 3, 4, 5} + local tt = table.clone(t) + + assert(#tt == 3) + assert(tt.a == 1 and tt.b == 2) + + t.c = 3 + assert(tt.c == nil) + + t = table.freeze({"test"}) + tt = table.clone(t) + assert(table.isfrozen(t) and not table.isfrozen(tt)) + + t = setmetatable({}, {}) + tt = table.clone(t) + assert(getmetatable(t) == getmetatable(tt)) + + t = setmetatable({}, {__metatable = "protected"}) + assert(not pcall(table.clone, t)) + + function order(t) + local r = '' + for k,v in pairs(t) do + r ..= tostring(v) + end + return v + end + + t = {a = 1, b = 2, c = 3, d = 4, e = 5, f = 6} + tt = table.clone(t) + assert(order(t) == order(tt)) + + assert(not pcall(table.clone)) + assert(not pcall(table.clone, 42)) +end + return"OK" From dbdf91f3cad8ea9e881b8cf99931f654faf65c2e Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Fri, 4 Mar 2022 08:36:33 -0800 Subject: [PATCH 190/399] Sync to upstream/release/517 (#408) --- Analysis/include/Luau/TxnLog.h | 32 ++- Analysis/include/Luau/TypeInfer.h | 18 -- Analysis/include/Luau/TypeVar.h | 5 +- Analysis/include/Luau/Unifier.h | 2 +- Analysis/src/Autocomplete.cpp | 66 ++--- Analysis/src/BuiltinDefinitions.cpp | 13 +- Analysis/src/EmbeddedBuiltinDefinitions.cpp | 1 - Analysis/src/JsonEncoder.cpp | 44 +-- Analysis/src/Linter.cpp | 157 +++++++++- Analysis/src/Module.cpp | 5 +- Analysis/src/ToString.cpp | 119 +------- Analysis/src/Transpiler.cpp | 59 ++-- Analysis/src/TxnLog.cpp | 84 +++++- Analysis/src/TypeInfer.cpp | 301 ++++++++------------ Analysis/src/TypeVar.cpp | 27 +- Analysis/src/Unifier.cpp | 137 +++++++-- Ast/src/Parser.cpp | 3 +- CLI/Repl.cpp | 35 +-- VM/include/lua.h | 3 + VM/src/lapi.cpp | 52 +++- VM/src/laux.cpp | 46 +-- VM/src/ldebug.cpp | 5 + VM/src/lgc.cpp | 74 ++++- VM/src/lgc.h | 7 - VM/src/lnumprint.cpp | 9 - VM/src/lnumutils.h | 1 - VM/src/lstate.h | 23 +- VM/src/ltable.cpp | 3 +- VM/src/ltablib.cpp | 21 ++ bench/gc/test_GC_Boehm_Trees.lua | 3 + bench/gc/test_GC_Tree_Pruning_Eager.lua | 2 +- bench/gc/test_GC_Tree_Pruning_Gen.lua | 2 +- bench/gc/test_GC_Tree_Pruning_Lazy.lua | 2 +- extern/isocline/include/isocline.h | 2 +- fuzz/number.cpp | 4 - fuzz/proto.cpp | 2 +- tests/Autocomplete.test.cpp | 10 +- tests/Conformance.test.cpp | 6 +- tests/Linter.test.cpp | 107 +++++++ tests/Parser.test.cpp | 6 - tests/ToDot.test.cpp | 2 - tests/ToString.test.cpp | 2 - tests/Transpiler.test.cpp | 3 - tests/TypeInfer.aliases.test.cpp | 3 +- tests/TypeInfer.builtins.test.cpp | 27 ++ tests/TypeInfer.generics.test.cpp | 89 ++++++ tests/TypeInfer.provisional.test.cpp | 95 +----- tests/TypeInfer.refinements.test.cpp | 11 +- tests/TypeInfer.singletons.test.cpp | 44 +-- tests/TypeInfer.tables.test.cpp | 81 +++++- tests/TypeInfer.test.cpp | 71 +++++ tests/TypeInfer.tryUnify.test.cpp | 7 +- tests/TypeInfer.typePacks.cpp | 39 --- tests/TypeInfer.unionTypes.test.cpp | 31 +- tests/conformance/nextvar.lua | 38 +++ 55 files changed, 1267 insertions(+), 774 deletions(-) diff --git a/Analysis/include/Luau/TxnLog.h b/Analysis/include/Luau/TxnLog.h index f238e258..f8105383 100644 --- a/Analysis/include/Luau/TxnLog.h +++ b/Analysis/include/Luau/TxnLog.h @@ -12,6 +12,8 @@ LUAU_FASTFLAG(LuauShareTxnSeen); namespace Luau { +using TypeOrPackId = const void*; + // Log of where what TypeIds we are rebinding and what they used to be // Remove with LuauUseCommitTxnLog struct DEPRECATED_TxnLog @@ -23,7 +25,7 @@ struct DEPRECATED_TxnLog { } - explicit DEPRECATED_TxnLog(std::vector>* sharedSeen) + explicit DEPRECATED_TxnLog(std::vector>* sharedSeen) : originalSeenSize(sharedSeen->size()) , ownedSeen() , sharedSeen(sharedSeen) @@ -48,15 +50,23 @@ struct DEPRECATED_TxnLog void pushSeen(TypeId lhs, TypeId rhs); void popSeen(TypeId lhs, TypeId rhs); + bool haveSeen(TypePackId lhs, TypePackId rhs); + void pushSeen(TypePackId lhs, TypePackId rhs); + void popSeen(TypePackId lhs, TypePackId rhs); + private: std::vector> typeVarChanges; std::vector> typePackChanges; std::vector>> tableChanges; size_t originalSeenSize; + bool haveSeen(TypeOrPackId lhs, TypeOrPackId rhs); + void pushSeen(TypeOrPackId lhs, TypeOrPackId rhs); + void popSeen(TypeOrPackId lhs, TypeOrPackId rhs); + public: - std::vector> ownedSeen; // used to avoid infinite recursion when types are cyclic - std::vector>* sharedSeen; // shared with all the descendent logs + std::vector> ownedSeen; // used to avoid infinite recursion when types are cyclic + std::vector>* sharedSeen; // shared with all the descendent logs }; // Pending state for a TypeVar. Generated by a TxnLog and committed via @@ -127,12 +137,12 @@ struct TxnLog } } - explicit TxnLog(std::vector>* sharedSeen) + explicit TxnLog(std::vector>* sharedSeen) : sharedSeen(sharedSeen) { } - TxnLog(TxnLog* parent, std::vector>* sharedSeen) + TxnLog(TxnLog* parent, std::vector>* sharedSeen) : parent(parent) , sharedSeen(sharedSeen) { @@ -173,6 +183,10 @@ struct TxnLog void pushSeen(TypeId lhs, TypeId rhs); void popSeen(TypeId lhs, TypeId rhs); + bool haveSeen(TypePackId lhs, TypePackId rhs) const; + void pushSeen(TypePackId lhs, TypePackId rhs); + void popSeen(TypePackId lhs, TypePackId rhs); + // Queues a type for modification. The original type will not change until commit // is called. Use pending to get the pending state. // @@ -316,12 +330,16 @@ private: // TxnLogs; use sharedSeen instead. This field exists because in the tree // of TxnLogs, the root must own its seen set. In all descendant TxnLogs, // this is an empty vector. - std::vector> ownedSeen; + std::vector> ownedSeen; + + bool haveSeen(TypeOrPackId lhs, TypeOrPackId rhs) const; + void pushSeen(TypeOrPackId lhs, TypeOrPackId rhs); + void popSeen(TypeOrPackId lhs, TypeOrPackId rhs); public: // Used to avoid infinite recursion when types are cyclic. // Shared with all the descendent TxnLogs. - std::vector>* sharedSeen; + std::vector>* sharedSeen; }; } // namespace Luau diff --git a/Analysis/include/Luau/TypeInfer.h b/Analysis/include/Luau/TypeInfer.h index 2440c810..839043cc 100644 --- a/Analysis/include/Luau/TypeInfer.h +++ b/Analysis/include/Luau/TypeInfer.h @@ -73,24 +73,6 @@ struct Instantiation : Substitution TypePackId clean(TypePackId tp) override; }; -// A substitution which replaces free types by generic types. -struct Quantification : Substitution -{ - Quantification(TypeArena* arena, TypeLevel level) - : Substitution(TxnLog::empty(), arena) - , level(level) - { - } - - TypeLevel level; - std::vector generics; - std::vector genericPacks; - bool isDirty(TypeId ty) override; - bool isDirty(TypePackId tp) override; - TypeId clean(TypeId ty) override; - TypePackId clean(TypePackId tp) override; -}; - // A substitution which replaces free types by any struct Anyification : Substitution { diff --git a/Analysis/include/Luau/TypeVar.h b/Analysis/include/Luau/TypeVar.h index 8d1a9fa6..29578dcd 100644 --- a/Analysis/include/Luau/TypeVar.h +++ b/Analysis/include/Luau/TypeVar.h @@ -298,7 +298,7 @@ struct TableTypeVar TableTypeVar() = default; explicit TableTypeVar(TableState state, TypeLevel level); - TableTypeVar(const Props& props, const std::optional& indexer, TypeLevel level, TableState state = TableState::Unsealed); + TableTypeVar(const Props& props, const std::optional& indexer, TypeLevel level, TableState state); Props props; std::optional indexer; @@ -477,6 +477,9 @@ bool isOptional(TypeId ty); bool isTableIntersection(TypeId ty); bool isOverloadedFunction(TypeId ty); +// True when string is a subtype of ty +bool maybeString(TypeId ty); + std::optional getMetatable(TypeId type); TableTypeVar* getMutableTableType(TypeId type); const TableTypeVar* getTableType(TypeId type); diff --git a/Analysis/include/Luau/Unifier.h b/Analysis/include/Luau/Unifier.h index fe822b01..4c0462fe 100644 --- a/Analysis/include/Luau/Unifier.h +++ b/Analysis/include/Luau/Unifier.h @@ -56,7 +56,7 @@ struct Unifier Unifier(TypeArena* types, Mode mode, const Location& location, Variance variance, UnifierSharedState& sharedState, TxnLog* parentLog = nullptr); - Unifier(TypeArena* types, Mode mode, std::vector>* sharedSeen, const Location& location, + Unifier(TypeArena* types, Mode mode, std::vector>* sharedSeen, const Location& location, Variance variance, UnifierSharedState& sharedState, TxnLog* parentLog = nullptr); // Test whether the two type vars unify. Never commits the result. diff --git a/Analysis/src/Autocomplete.cpp b/Analysis/src/Autocomplete.cpp index 29a2c6b5..c3de8d0e 100644 --- a/Analysis/src/Autocomplete.cpp +++ b/Analysis/src/Autocomplete.cpp @@ -16,7 +16,6 @@ LUAU_FASTFLAG(LuauUseCommittingTxnLog) LUAU_FASTFLAGVARIABLE(LuauAutocompleteAvoidMutation, false); LUAU_FASTFLAGVARIABLE(LuauMissingFollowACMetatables, false); -LUAU_FASTFLAGVARIABLE(PreferToCallFunctionsForIntersects, false); LUAU_FASTFLAGVARIABLE(LuauIfElseExprFixCompletionIssue, false); static const std::unordered_set kStatementStartingKeywords = { @@ -272,55 +271,34 @@ static TypeCorrectKind checkTypeCorrectKind(const Module& module, TypeArena* typ TypeId expectedType = follow(*typeAtPosition); - if (FFlag::PreferToCallFunctionsForIntersects) - { - auto checkFunctionType = [&canUnify, &expectedType](const FunctionTypeVar* ftv) { - auto [retHead, retTail] = flatten(ftv->retType); + auto checkFunctionType = [&canUnify, &expectedType](const FunctionTypeVar* ftv) { + auto [retHead, retTail] = flatten(ftv->retType); - if (!retHead.empty() && canUnify(retHead.front(), expectedType)) + if (!retHead.empty() && canUnify(retHead.front(), expectedType)) + return true; + + // We might only have a variadic tail pack, check if the element is compatible + if (retTail) + { + if (const VariadicTypePack* vtp = get(follow(*retTail)); vtp && canUnify(vtp->ty, expectedType)) return true; - - // We might only have a variadic tail pack, check if the element is compatible - if (retTail) - { - if (const VariadicTypePack* vtp = get(follow(*retTail)); vtp && canUnify(vtp->ty, expectedType)) - return true; - } - - return false; - }; - - // We also want to suggest functions that return compatible result - if (const FunctionTypeVar* ftv = get(ty); ftv && checkFunctionType(ftv)) - { - return TypeCorrectKind::CorrectFunctionResult; } - else if (const IntersectionTypeVar* itv = get(ty)) - { - for (TypeId id : itv->parts) - { - if (const FunctionTypeVar* ftv = get(id); ftv && checkFunctionType(ftv)) - { - return TypeCorrectKind::CorrectFunctionResult; - } - } - } - } - else + + return false; + }; + + // We also want to suggest functions that return compatible result + if (const FunctionTypeVar* ftv = get(ty); ftv && checkFunctionType(ftv)) { - // We also want to suggest functions that return compatible result - if (const FunctionTypeVar* ftv = get(ty)) + return TypeCorrectKind::CorrectFunctionResult; + } + else if (const IntersectionTypeVar* itv = get(ty)) + { + for (TypeId id : itv->parts) { - auto [retHead, retTail] = flatten(ftv->retType); - - if (!retHead.empty() && canUnify(retHead.front(), expectedType)) - return TypeCorrectKind::CorrectFunctionResult; - - // We might only have a variadic tail pack, check if the element is compatible - if (retTail) + if (const FunctionTypeVar* ftv = get(id); ftv && checkFunctionType(ftv)) { - if (const VariadicTypePack* vtp = get(follow(*retTail)); vtp && canUnify(vtp->ty, expectedType)) - return TypeCorrectKind::CorrectFunctionResult; + return TypeCorrectKind::CorrectFunctionResult; } } } diff --git a/Analysis/src/BuiltinDefinitions.cpp b/Analysis/src/BuiltinDefinitions.cpp index d72422a5..e4e5dab8 100644 --- a/Analysis/src/BuiltinDefinitions.cpp +++ b/Analysis/src/BuiltinDefinitions.cpp @@ -9,6 +9,7 @@ #include LUAU_FASTFLAG(LuauAssertStripsFalsyTypes) +LUAU_FASTFLAGVARIABLE(LuauTableCloneType, false) /** FIXME: Many of these type definitions are not quite completely accurate. * @@ -283,8 +284,16 @@ void registerBuiltinTypes(TypeChecker& typeChecker) attachMagicFunction(getGlobalBinding(typeChecker, "setmetatable"), magicFunctionSetMetaTable); attachMagicFunction(getGlobalBinding(typeChecker, "select"), magicFunctionSelect); - auto tableLib = getMutable(getGlobalBinding(typeChecker, "table")); - attachMagicFunction(tableLib->props["pack"].type, magicFunctionPack); + if (TableTypeVar* ttv = getMutable(getGlobalBinding(typeChecker, "table"))) + { + // tabTy is a generic table type which we can't express via declaration syntax yet + ttv->props["freeze"] = makeProperty(makeFunction(arena, std::nullopt, {tabTy}, {tabTy}), "@luau/global/table.freeze"); + + if (FFlag::LuauTableCloneType) + ttv->props["clone"] = makeProperty(makeFunction(arena, std::nullopt, {tabTy}, {tabTy}), "@luau/global/table.clone"); + + attachMagicFunction(ttv->props["pack"].type, magicFunctionPack); + } attachMagicFunction(getGlobalBinding(typeChecker, "require"), magicFunctionRequire); } diff --git a/Analysis/src/EmbeddedBuiltinDefinitions.cpp b/Analysis/src/EmbeddedBuiltinDefinitions.cpp index bf6e1193..471b61ad 100644 --- a/Analysis/src/EmbeddedBuiltinDefinitions.cpp +++ b/Analysis/src/EmbeddedBuiltinDefinitions.cpp @@ -170,7 +170,6 @@ declare function gcinfo(): number move: ({V}, number, number, number, {V}?) -> {V}, clear: ({[K]: V}) -> (), - freeze: ({[K]: V}) -> {[K]: V}, isfrozen: ({[K]: V}) -> boolean, } diff --git a/Analysis/src/JsonEncoder.cpp b/Analysis/src/JsonEncoder.cpp index ec399158..811e7c24 100644 --- a/Analysis/src/JsonEncoder.cpp +++ b/Analysis/src/JsonEncoder.cpp @@ -5,8 +5,6 @@ #include "Luau/StringUtils.h" #include "Luau/Common.h" -LUAU_FASTFLAG(LuauTypeAliasDefaults) - namespace Luau { @@ -369,38 +367,24 @@ struct AstJsonEncoder : public AstVisitor void write(const AstGenericType& genericType) { - if (FFlag::LuauTypeAliasDefaults) - { - writeRaw("{"); - bool c = pushComma(); - write("name", genericType.name); - if (genericType.defaultValue) - write("type", genericType.defaultValue); - popComma(c); - writeRaw("}"); - } - else - { - write(genericType.name); - } + writeRaw("{"); + bool c = pushComma(); + write("name", genericType.name); + if (genericType.defaultValue) + write("type", genericType.defaultValue); + popComma(c); + writeRaw("}"); } void write(const AstGenericTypePack& genericTypePack) { - if (FFlag::LuauTypeAliasDefaults) - { - writeRaw("{"); - bool c = pushComma(); - write("name", genericTypePack.name); - if (genericTypePack.defaultValue) - write("type", genericTypePack.defaultValue); - popComma(c); - writeRaw("}"); - } - else - { - write(genericTypePack.name); - } + writeRaw("{"); + bool c = pushComma(); + write("name", genericTypePack.name); + if (genericTypePack.defaultValue) + write("type", genericTypePack.defaultValue); + popComma(c); + writeRaw("}"); } void write(AstExprTable::Item::Kind kind) diff --git a/Analysis/src/Linter.cpp b/Analysis/src/Linter.cpp index 7635dc0f..56c4e3e8 100644 --- a/Analysis/src/Linter.cpp +++ b/Analysis/src/Linter.cpp @@ -13,6 +13,7 @@ #include LUAU_FASTINTVARIABLE(LuauSuggestionDistance, 4) +LUAU_FASTFLAGVARIABLE(LuauLintGlobalNeverReadBeforeWritten, false) namespace Luau { @@ -233,6 +234,20 @@ public: } private: + struct FunctionInfo + { + explicit FunctionInfo(AstExprFunction* ast) + : ast(ast) + , dominatedGlobals({}) + , conditionalExecution(false) + { + } + + AstExprFunction* ast; + DenseHashSet dominatedGlobals; + bool conditionalExecution; + }; + struct Global { AstExprGlobal* firstRef = nullptr; @@ -241,6 +256,9 @@ private: bool assigned = false; bool builtin = false; + bool definedInModuleScope = false; + bool definedAsFunction = false; + bool readBeforeWritten = false; std::optional deprecated; }; @@ -248,7 +266,8 @@ private: DenseHashMap globals; std::vector globalRefs; - std::vector functionStack; + std::vector functionStack; + LintGlobalLocal() : globals(AstName()) @@ -291,12 +310,18 @@ private: "Global '%s' is only used in the enclosing function defined at line %d; consider changing it to local", g.firstRef->name.value, top->location.begin.line + 1); } + else if (FFlag::LuauLintGlobalNeverReadBeforeWritten && g.assigned && !g.readBeforeWritten && !g.definedInModuleScope && + g.firstRef->name != context->placeholder) + { + emitWarning(*context, LintWarning::Code_GlobalUsedAsLocal, g.firstRef->location, + "Global '%s' is never read before being written. Consider changing it to local", g.firstRef->name.value); + } } } bool visit(AstExprFunction* node) override { - functionStack.push_back(node); + functionStack.emplace_back(node); node->body->visit(this); @@ -307,6 +332,11 @@ private: bool visit(AstExprGlobal* node) override { + if (FFlag::LuauLintGlobalNeverReadBeforeWritten && !functionStack.empty() && !functionStack.back().dominatedGlobals.contains(node->name)) + { + Global& g = globals[node->name]; + g.readBeforeWritten = true; + } trackGlobalRef(node); if (node->name == context->placeholder) @@ -335,6 +365,21 @@ private: { Global& g = globals[gv->name]; + if (FFlag::LuauLintGlobalNeverReadBeforeWritten) + { + if (functionStack.empty()) + { + g.definedInModuleScope = true; + } + else + { + if (!functionStack.back().conditionalExecution) + { + functionStack.back().dominatedGlobals.insert(gv->name); + } + } + } + if (g.builtin) emitWarning(*context, LintWarning::Code_BuiltinGlobalWrite, gv->location, "Built-in global '%s' is overwritten here; consider using a local or changing the name", gv->name.value); @@ -369,7 +414,14 @@ private: emitWarning(*context, LintWarning::Code_BuiltinGlobalWrite, gv->location, "Built-in global '%s' is overwritten here; consider using a local or changing the name", gv->name.value); else + { g.assigned = true; + if (FFlag::LuauLintGlobalNeverReadBeforeWritten) + { + g.definedAsFunction = true; + g.definedInModuleScope = functionStack.empty(); + } + } trackGlobalRef(gv); } @@ -377,6 +429,98 @@ private: return true; } + class HoldConditionalExecution + { + public: + HoldConditionalExecution(LintGlobalLocal& p) + : p(p) + { + if (!p.functionStack.empty() && !p.functionStack.back().conditionalExecution) + { + resetToFalse = true; + p.functionStack.back().conditionalExecution = true; + } + } + ~HoldConditionalExecution() + { + if (resetToFalse) + p.functionStack.back().conditionalExecution = false; + } + + private: + bool resetToFalse = false; + LintGlobalLocal& p; + }; + + bool visit(AstStatIf* node) override + { + if (!FFlag::LuauLintGlobalNeverReadBeforeWritten) + return true; + + HoldConditionalExecution ce(*this); + node->condition->visit(this); + node->thenbody->visit(this); + if (node->elsebody) + node->elsebody->visit(this); + + return false; + } + + bool visit(AstStatWhile* node) override + { + if (!FFlag::LuauLintGlobalNeverReadBeforeWritten) + return true; + + HoldConditionalExecution ce(*this); + node->condition->visit(this); + node->body->visit(this); + + return false; + } + + bool visit(AstStatRepeat* node) override + { + if (!FFlag::LuauLintGlobalNeverReadBeforeWritten) + return true; + + HoldConditionalExecution ce(*this); + node->condition->visit(this); + node->body->visit(this); + + return false; + } + + bool visit(AstStatFor* node) override + { + if (!FFlag::LuauLintGlobalNeverReadBeforeWritten) + return true; + + HoldConditionalExecution ce(*this); + node->from->visit(this); + node->to->visit(this); + + if (node->step) + node->step->visit(this); + + node->body->visit(this); + + return false; + } + + bool visit(AstStatForIn* node) override + { + if (!FFlag::LuauLintGlobalNeverReadBeforeWritten) + return true; + + HoldConditionalExecution ce(*this); + for (AstExpr* expr : node->values) + expr->visit(this); + + node->body->visit(this); + + return false; + } + void trackGlobalRef(AstExprGlobal* node) { Global& g = globals[node->name]; @@ -390,7 +534,12 @@ private: // to reduce the cost of tracking we only track this for user globals if (!g.builtin) { - g.functionRef = functionStack; + g.functionRef.clear(); + g.functionRef.reserve(functionStack.size()); + for (const FunctionInfo& entry : functionStack) + { + g.functionRef.push_back(entry.ast); + } } } else @@ -401,7 +550,7 @@ private: // we need to find a common prefix between all uses of a global size_t prefix = 0; - while (prefix < g.functionRef.size() && prefix < functionStack.size() && g.functionRef[prefix] == functionStack[prefix]) + while (prefix < g.functionRef.size() && prefix < functionStack.size() && g.functionRef[prefix] == functionStack[prefix].ast) prefix++; g.functionRef.resize(prefix); diff --git a/Analysis/src/Module.cpp b/Analysis/src/Module.cpp index 412b78bb..76dc72d2 100644 --- a/Analysis/src/Module.cpp +++ b/Analysis/src/Module.cpp @@ -14,7 +14,6 @@ LUAU_FASTFLAGVARIABLE(DebugLuauFreezeArena, false) LUAU_FASTFLAGVARIABLE(DebugLuauTrackOwningArena, false) // Remove with FFlagLuauImmutableTypes LUAU_FASTINTVARIABLE(LuauTypeCloneRecursionLimit, 300) -LUAU_FASTFLAG(LuauTypeAliasDefaults) LUAU_FASTFLAG(LuauImmutableTypes) namespace Luau @@ -463,7 +462,7 @@ TypeFun clone(const TypeFun& typeFun, TypeArena& dest, SeenTypes& seenTypes, See TypeId ty = clone(param.ty, dest, seenTypes, seenTypePacks, cloneState); std::optional defaultValue; - if (FFlag::LuauTypeAliasDefaults && param.defaultValue) + if (param.defaultValue) defaultValue = clone(*param.defaultValue, dest, seenTypes, seenTypePacks, cloneState); result.typeParams.push_back({ty, defaultValue}); @@ -474,7 +473,7 @@ TypeFun clone(const TypeFun& typeFun, TypeArena& dest, SeenTypes& seenTypes, See TypePackId tp = clone(param.tp, dest, seenTypes, seenTypePacks, cloneState); std::optional defaultValue; - if (FFlag::LuauTypeAliasDefaults && param.defaultValue) + if (param.defaultValue) defaultValue = clone(*param.defaultValue, dest, seenTypes, seenTypePacks, cloneState); result.typePackParams.push_back({tp, defaultValue}); diff --git a/Analysis/src/ToString.cpp b/Analysis/src/ToString.cpp index 5e79b841..010ca361 100644 --- a/Analysis/src/ToString.cpp +++ b/Analysis/src/ToString.cpp @@ -10,8 +10,6 @@ #include #include -LUAU_FASTFLAG(LuauTypeAliasDefaults) - /* * Prefix generic typenames with gen- * Additionally, free types will be prefixed with free- and suffixed with their level. eg free-a-4 @@ -288,28 +286,15 @@ struct TypeVarStringifier else first = false; - if (FFlag::LuauTypeAliasDefaults) - { - bool wrap = !singleTp && get(follow(tp)); + bool wrap = !singleTp && get(follow(tp)); - if (wrap) - state.emit("("); + if (wrap) + state.emit("("); - stringify(tp); + stringify(tp); - if (wrap) - state.emit(")"); - } - else - { - if (!singleTp) - state.emit("("); - - stringify(tp); - - if (!singleTp) - state.emit(")"); - } + if (wrap) + state.emit(")"); } if (types.size() || typePacks.size()) @@ -1105,100 +1090,8 @@ std::string toString(const TypePackVar& tp, const ToStringOptions& opts) return toString(const_cast(&tp), std::move(opts)); } -std::string toStringNamedFunction_DEPRECATED(const std::string& prefix, const FunctionTypeVar& ftv, ToStringOptions opts) -{ - std::string s = prefix; - - auto toString_ = [&opts](TypeId ty) -> std::string { - ToStringResult res = toStringDetailed(ty, opts); - opts.nameMap = std::move(res.nameMap); - return res.name; - }; - - auto toStringPack_ = [&opts](TypePackId ty) -> std::string { - ToStringResult res = toStringDetailed(ty, opts); - opts.nameMap = std::move(res.nameMap); - return res.name; - }; - - if (!opts.hideNamedFunctionTypeParameters && (!ftv.generics.empty() || !ftv.genericPacks.empty())) - { - s += "<"; - - bool first = true; - for (TypeId g : ftv.generics) - { - if (!first) - s += ", "; - first = false; - s += toString_(g); - } - - for (TypePackId gp : ftv.genericPacks) - { - if (!first) - s += ", "; - first = false; - s += toStringPack_(gp); - } - - s += ">"; - } - - s += "("; - - auto argPackIter = begin(ftv.argTypes); - auto argNameIter = ftv.argNames.begin(); - - bool first = true; - while (argPackIter != end(ftv.argTypes)) - { - if (!first) - s += ", "; - first = false; - - // We don't currently respect opts.functionTypeArguments. I don't think this function should. - if (argNameIter != ftv.argNames.end()) - { - s += (*argNameIter ? (*argNameIter)->name : "_") + ": "; - ++argNameIter; - } - else - { - s += "_: "; - } - - s += toString_(*argPackIter); - ++argPackIter; - } - - if (argPackIter.tail()) - { - if (auto vtp = get(*argPackIter.tail())) - s += ", ...: " + toString_(vtp->ty); - else - s += ", ...: " + toStringPack_(*argPackIter.tail()); - } - - s += "): "; - - size_t retSize = size(ftv.retType); - bool hasTail = !finite(ftv.retType); - if (retSize == 0 && !hasTail) - s += "()"; - else if ((retSize == 0 && hasTail) || (retSize == 1 && !hasTail)) - s += toStringPack_(ftv.retType); - else - s += "(" + toStringPack_(ftv.retType) + ")"; - - return s; -} - std::string toStringNamedFunction(const std::string& prefix, const FunctionTypeVar& ftv, ToStringOptions opts) { - if (!FFlag::LuauTypeAliasDefaults) - return toStringNamedFunction_DEPRECATED(prefix, ftv, opts); - ToStringResult result; StringifierState state(opts, result, opts.nameMap); TypeVarStringifier tvs{state}; diff --git a/Analysis/src/Transpiler.cpp b/Analysis/src/Transpiler.cpp index a02d396b..92ed241e 100644 --- a/Analysis/src/Transpiler.cpp +++ b/Analysis/src/Transpiler.cpp @@ -10,8 +10,6 @@ #include #include -LUAU_FASTFLAG(LuauTypeAliasDefaults) - namespace { bool isIdentifierStartChar(char c) @@ -796,21 +794,14 @@ struct Printer { comma(); - if (FFlag::LuauTypeAliasDefaults) - { - writer.advance(o.location.begin); - writer.identifier(o.name.value); + writer.advance(o.location.begin); + writer.identifier(o.name.value); - if (o.defaultValue) - { - writer.maybeSpace(o.defaultValue->location.begin, 2); - writer.symbol("="); - visualizeTypeAnnotation(*o.defaultValue); - } - } - else + if (o.defaultValue) { - writer.identifier(o.name.value); + writer.maybeSpace(o.defaultValue->location.begin, 2); + writer.symbol("="); + visualizeTypeAnnotation(*o.defaultValue); } } @@ -818,23 +809,15 @@ struct Printer { comma(); - if (FFlag::LuauTypeAliasDefaults) - { - writer.advance(o.location.begin); - writer.identifier(o.name.value); - writer.symbol("..."); + writer.advance(o.location.begin); + writer.identifier(o.name.value); + writer.symbol("..."); - if (o.defaultValue) - { - writer.maybeSpace(o.defaultValue->location.begin, 2); - writer.symbol("="); - visualizeTypePackAnnotation(*o.defaultValue, false); - } - } - else + if (o.defaultValue) { - writer.identifier(o.name.value); - writer.symbol("..."); + writer.maybeSpace(o.defaultValue->location.begin, 2); + writer.symbol("="); + visualizeTypePackAnnotation(*o.defaultValue, false); } } @@ -882,18 +865,14 @@ struct Printer { comma(); - if (FFlag::LuauTypeAliasDefaults) - writer.advance(o.location.begin); - + writer.advance(o.location.begin); writer.identifier(o.name.value); } for (const auto& o : func.genericPacks) { comma(); - if (FFlag::LuauTypeAliasDefaults) - writer.advance(o.location.begin); - + writer.advance(o.location.begin); writer.identifier(o.name.value); writer.symbol("..."); } @@ -1023,18 +1002,14 @@ struct Printer { comma(); - if (FFlag::LuauTypeAliasDefaults) - writer.advance(o.location.begin); - + writer.advance(o.location.begin); writer.identifier(o.name.value); } for (const auto& o : a->genericPacks) { comma(); - if (FFlag::LuauTypeAliasDefaults) - writer.advance(o.location.begin); - + writer.advance(o.location.begin); writer.identifier(o.name.value); writer.symbol("..."); } diff --git a/Analysis/src/TxnLog.cpp b/Analysis/src/TxnLog.cpp index 00067bdd..c7bf1e62 100644 --- a/Analysis/src/TxnLog.cpp +++ b/Analysis/src/TxnLog.cpp @@ -61,22 +61,52 @@ void DEPRECATED_TxnLog::concat(DEPRECATED_TxnLog rhs) bool DEPRECATED_TxnLog::haveSeen(TypeId lhs, TypeId rhs) { - LUAU_ASSERT(!FFlag::LuauUseCommittingTxnLog); - const std::pair sortedPair = (lhs > rhs) ? std::make_pair(lhs, rhs) : std::make_pair(rhs, lhs); - return (sharedSeen->end() != std::find(sharedSeen->begin(), sharedSeen->end(), sortedPair)); + return haveSeen((TypeOrPackId)lhs, (TypeOrPackId)rhs); } void DEPRECATED_TxnLog::pushSeen(TypeId lhs, TypeId rhs) { - LUAU_ASSERT(!FFlag::LuauUseCommittingTxnLog); - const std::pair sortedPair = (lhs > rhs) ? std::make_pair(lhs, rhs) : std::make_pair(rhs, lhs); - sharedSeen->push_back(sortedPair); + pushSeen((TypeOrPackId)lhs, (TypeOrPackId)rhs); } void DEPRECATED_TxnLog::popSeen(TypeId lhs, TypeId rhs) +{ + popSeen((TypeOrPackId)lhs, (TypeOrPackId)rhs); +} + +bool DEPRECATED_TxnLog::haveSeen(TypePackId lhs, TypePackId rhs) +{ + return haveSeen((TypeOrPackId)lhs, (TypeOrPackId)rhs); +} + +void DEPRECATED_TxnLog::pushSeen(TypePackId lhs, TypePackId rhs) +{ + pushSeen((TypeOrPackId)lhs, (TypeOrPackId)rhs); +} + +void DEPRECATED_TxnLog::popSeen(TypePackId lhs, TypePackId rhs) +{ + popSeen((TypeOrPackId)lhs, (TypeOrPackId)rhs); +} + +bool DEPRECATED_TxnLog::haveSeen(TypeOrPackId lhs, TypeOrPackId rhs) { LUAU_ASSERT(!FFlag::LuauUseCommittingTxnLog); - const std::pair sortedPair = (lhs > rhs) ? std::make_pair(lhs, rhs) : std::make_pair(rhs, lhs); + const std::pair sortedPair = (lhs > rhs) ? std::make_pair(lhs, rhs) : std::make_pair(rhs, lhs); + return (sharedSeen->end() != std::find(sharedSeen->begin(), sharedSeen->end(), sortedPair)); +} + +void DEPRECATED_TxnLog::pushSeen(TypeOrPackId lhs, TypeOrPackId rhs) +{ + LUAU_ASSERT(!FFlag::LuauUseCommittingTxnLog); + const std::pair sortedPair = (lhs > rhs) ? std::make_pair(lhs, rhs) : std::make_pair(rhs, lhs); + sharedSeen->push_back(sortedPair); +} + +void DEPRECATED_TxnLog::popSeen(TypeOrPackId lhs, TypeOrPackId rhs) +{ + LUAU_ASSERT(!FFlag::LuauUseCommittingTxnLog); + const std::pair sortedPair = (lhs > rhs) ? std::make_pair(lhs, rhs) : std::make_pair(rhs, lhs); LUAU_ASSERT(sortedPair == sharedSeen->back()); sharedSeen->pop_back(); } @@ -186,10 +216,40 @@ TxnLog TxnLog::inverse() } bool TxnLog::haveSeen(TypeId lhs, TypeId rhs) const +{ + return haveSeen((TypeOrPackId)lhs, (TypeOrPackId)rhs); +} + +void TxnLog::pushSeen(TypeId lhs, TypeId rhs) +{ + pushSeen((TypeOrPackId)lhs, (TypeOrPackId)rhs); +} + +void TxnLog::popSeen(TypeId lhs, TypeId rhs) +{ + popSeen((TypeOrPackId)lhs, (TypeOrPackId)rhs); +} + +bool TxnLog::haveSeen(TypePackId lhs, TypePackId rhs) const +{ + return haveSeen((TypeOrPackId)lhs, (TypeOrPackId)rhs); +} + +void TxnLog::pushSeen(TypePackId lhs, TypePackId rhs) +{ + pushSeen((TypeOrPackId)lhs, (TypeOrPackId)rhs); +} + +void TxnLog::popSeen(TypePackId lhs, TypePackId rhs) +{ + popSeen((TypeOrPackId)lhs, (TypeOrPackId)rhs); +} + +bool TxnLog::haveSeen(TypeOrPackId lhs, TypeOrPackId rhs) const { LUAU_ASSERT(FFlag::LuauUseCommittingTxnLog); - const std::pair sortedPair = (lhs > rhs) ? std::make_pair(lhs, rhs) : std::make_pair(rhs, lhs); + const std::pair sortedPair = (lhs > rhs) ? std::make_pair(lhs, rhs) : std::make_pair(rhs, lhs); if (sharedSeen->end() != std::find(sharedSeen->begin(), sharedSeen->end(), sortedPair)) { return true; @@ -203,19 +263,19 @@ bool TxnLog::haveSeen(TypeId lhs, TypeId rhs) const return false; } -void TxnLog::pushSeen(TypeId lhs, TypeId rhs) +void TxnLog::pushSeen(TypeOrPackId lhs, TypeOrPackId rhs) { LUAU_ASSERT(FFlag::LuauUseCommittingTxnLog); - const std::pair sortedPair = (lhs > rhs) ? std::make_pair(lhs, rhs) : std::make_pair(rhs, lhs); + const std::pair sortedPair = (lhs > rhs) ? std::make_pair(lhs, rhs) : std::make_pair(rhs, lhs); sharedSeen->push_back(sortedPair); } -void TxnLog::popSeen(TypeId lhs, TypeId rhs) +void TxnLog::popSeen(TypeOrPackId lhs, TypeOrPackId rhs) { LUAU_ASSERT(FFlag::LuauUseCommittingTxnLog); - const std::pair sortedPair = (lhs > rhs) ? std::make_pair(lhs, rhs) : std::make_pair(rhs, lhs); + const std::pair sortedPair = (lhs > rhs) ? std::make_pair(lhs, rhs) : std::make_pair(rhs, lhs); LUAU_ASSERT(sortedPair == sharedSeen->back()); sharedSeen->pop_back(); } diff --git a/Analysis/src/TypeInfer.cpp b/Analysis/src/TypeInfer.cpp index faf60eb3..8e6b3b52 100644 --- a/Analysis/src/TypeInfer.cpp +++ b/Analysis/src/TypeInfer.cpp @@ -29,22 +29,20 @@ LUAU_FASTFLAGVARIABLE(DebugLuauFreezeDuringUnification, false) LUAU_FASTFLAGVARIABLE(LuauRecursiveTypeParameterRestriction, false) LUAU_FASTFLAGVARIABLE(LuauGenericFunctionsDontCacheTypeParams, false) LUAU_FASTFLAGVARIABLE(LuauImmutableTypes, false) -LUAU_FASTFLAGVARIABLE(LuauQuantifyInPlace2, false) LUAU_FASTFLAGVARIABLE(LuauSealExports, false) LUAU_FASTFLAGVARIABLE(LuauSingletonTypes, false) LUAU_FASTFLAGVARIABLE(LuauDiscriminableUnions2, false) -LUAU_FASTFLAGVARIABLE(LuauTypeAliasDefaults, false) LUAU_FASTFLAGVARIABLE(LuauExpectedTypesOfProperties, false) LUAU_FASTFLAGVARIABLE(LuauErrorRecoveryType, false) +LUAU_FASTFLAGVARIABLE(LuauOnlyMutateInstantiatedTables, false) LUAU_FASTFLAGVARIABLE(LuauPropertiesGetExpectedType, false) -LUAU_FASTFLAGVARIABLE(LuauAscribeCorrectLevelToInferredProperitesOfFreeTables, false) LUAU_FASTFLAGVARIABLE(LuauUnsealedTableLiteral, false) LUAU_FASTFLAGVARIABLE(LuauTwoPassAliasDefinitionFix, false) LUAU_FASTFLAGVARIABLE(LuauAssertStripsFalsyTypes, false) LUAU_FASTFLAGVARIABLE(LuauReturnAnyInsteadOfICE, false) // Eventually removed as false. -LUAU_FASTFLAGVARIABLE(LuauAnotherTypeLevelFix, false) LUAU_FASTFLAG(LuauWidenIfSupertypeIsFree) LUAU_FASTFLAGVARIABLE(LuauDoNotTryToReduce, false) +LUAU_FASTFLAGVARIABLE(LuauDoNotAccidentallyDependOnPointerOrdering, false) namespace Luau { @@ -445,7 +443,7 @@ void TypeChecker::checkBlock(const ScopePtr& scope, const AstStatBlock& block) // function f(x:a):a local x: number = g(37) return x end // function g(x:number):number return f(x) end // ``` - if (FFlag::LuauQuantifyInPlace2 ? containsFunctionCallOrReturn(**protoIter) : containsFunctionCall(**protoIter)) + if (containsFunctionCallOrReturn(**protoIter)) { while (checkIter != protoIter) { @@ -1676,7 +1674,7 @@ std::optional TypeChecker::getIndexTypeFromType( } else if (tableType->state == TableState::Free) { - TypeId result = FFlag::LuauAscribeCorrectLevelToInferredProperitesOfFreeTables ? freshType(tableType->level) : freshType(scope); + TypeId result = freshType(tableType->level); tableType->props[name] = {result}; return result; } @@ -1776,31 +1774,62 @@ std::optional TypeChecker::getIndexTypeFromType( std::vector TypeChecker::reduceUnion(const std::vector& types) { - std::set s; - - for (TypeId t : types) + if (FFlag::LuauDoNotAccidentallyDependOnPointerOrdering) { - if (const UnionTypeVar* utv = get(follow(t))) + std::vector result; + for (TypeId t : types) { - std::vector r = reduceUnion(utv->options); - for (TypeId ty : r) - s.insert(ty); + t = follow(t); + if (get(t) || get(t)) + return {t}; + + if (const UnionTypeVar* utv = get(t)) + { + std::vector r = reduceUnion(utv->options); + for (TypeId ty : r) + { + ty = follow(ty); + if (get(ty) || get(ty)) + return {ty}; + + if (std::find(result.begin(), result.end(), ty) == result.end()) + result.push_back(ty); + } + } + else if (std::find(result.begin(), result.end(), t) == result.end()) + result.push_back(t); } - else - s.insert(t); - } - // If any of them are ErrorTypeVars/AnyTypeVars, decay into them. - for (TypeId t : s) + return result; + } + else { - t = follow(t); - if (get(t) || get(t)) - return {t}; - } + std::set s; - std::vector r(s.begin(), s.end()); - std::sort(r.begin(), r.end()); - return r; + for (TypeId t : types) + { + if (const UnionTypeVar* utv = get(follow(t))) + { + std::vector r = reduceUnion(utv->options); + for (TypeId ty : r) + s.insert(ty); + } + else + s.insert(t); + } + + // If any of them are ErrorTypeVars/AnyTypeVars, decay into them. + for (TypeId t : s) + { + t = follow(t); + if (get(t) || get(t)) + return {t}; + } + + std::vector r(s.begin(), s.end()); + std::sort(r.begin(), r.end()); + return r; + } } std::optional TypeChecker::tryStripUnionFromNil(TypeId ty) @@ -2811,7 +2840,7 @@ TypeId TypeChecker::checkLValueBinding(const ScopePtr& scope, const AstExprIndex } else if (exprTable->state == TableState::Unsealed || exprTable->state == TableState::Free) { - TypeId resultType = freshType(FFlag::LuauAnotherTypeLevelFix ? exprTable->level : scope->level); + TypeId resultType = freshType(exprTable->level); exprTable->indexer = TableIndexer{anyIfNonstrict(indexType), anyIfNonstrict(resultType)}; return resultType; } @@ -4453,51 +4482,6 @@ TypePackId ReplaceGenerics::clean(TypePackId tp) return addTypePack(TypePackVar(FreeTypePack{level})); } -bool Quantification::isDirty(TypeId ty) -{ - if (const TableTypeVar* ttv = log->getMutable(ty)) - return level.subsumes(ttv->level) && ((ttv->state == TableState::Free) || (ttv->state == TableState::Unsealed)); - else if (const FreeTypeVar* ftv = log->getMutable(ty)) - return level.subsumes(ftv->level); - else - return false; -} - -bool Quantification::isDirty(TypePackId tp) -{ - if (const FreeTypePack* ftv = log->getMutable(tp)) - return level.subsumes(ftv->level); - else - return false; -} - -TypeId Quantification::clean(TypeId ty) -{ - LUAU_ASSERT(isDirty(ty)); - if (const TableTypeVar* ttv = log->getMutable(ty)) - { - TableState state = (ttv->state == TableState::Unsealed ? TableState::Sealed : TableState::Generic); - TableTypeVar clone = TableTypeVar{ttv->props, ttv->indexer, level, state}; - clone.methodDefinitionLocations = ttv->methodDefinitionLocations; - clone.definitionModuleName = ttv->definitionModuleName; - return addType(std::move(clone)); - } - else - { - TypeId generic = addType(GenericTypeVar{level}); - generics.push_back(generic); - return generic; - } -} - -TypePackId Quantification::clean(TypePackId tp) -{ - LUAU_ASSERT(isDirty(tp)); - TypePackId genericPack = addTypePack(TypePackVar(GenericTypePack{level})); - genericPacks.push_back(genericPack); - return genericPack; -} - bool Anyification::isDirty(TypeId ty) { if (const TableTypeVar* ttv = log->getMutable(ty)) @@ -4550,29 +4534,8 @@ TypeId TypeChecker::quantify(const ScopePtr& scope, TypeId ty, Location location if (!ftv || !ftv->generics.empty() || !ftv->genericPacks.empty()) return ty; - if (FFlag::LuauQuantifyInPlace2) - { - Luau::quantify(ty, scope->level); - return ty; - } - - Quantification quantification{¤tModule->internalTypes, scope->level}; - std::optional qty = quantification.substitute(ty); - - if (!qty.has_value()) - { - reportError(location, UnificationTooComplex{}); - return errorRecoveryType(scope); - } - - if (ty == *qty) - return ty; - - FunctionTypeVar* qftv = getMutable(*qty); - LUAU_ASSERT(qftv); - qftv->generics = std::move(quantification.generics); - qftv->genericPacks = std::move(quantification.genericPacks); - return *qty; + Luau::quantify(ty, scope->level); + return ty; } TypeId TypeChecker::instantiate(const ScopePtr& scope, TypeId ty, Location location, const TxnLog* log) @@ -4915,35 +4878,20 @@ TypeId TypeChecker::resolveType(const ScopePtr& scope, const AstType& annotation if (lit->parameters.size == 0 && tf->typeParams.empty() && tf->typePackParams.empty()) return tf->type; - bool hasDefaultTypes = false; - bool hasDefaultPacks = false; bool parameterCountErrorReported = false; + bool hasDefaultTypes = std::any_of(tf->typeParams.begin(), tf->typeParams.end(), [](auto&& el) { + return el.defaultValue.has_value(); + }); + bool hasDefaultPacks = std::any_of(tf->typePackParams.begin(), tf->typePackParams.end(), [](auto&& el) { + return el.defaultValue.has_value(); + }); - if (FFlag::LuauTypeAliasDefaults) + if (!lit->hasParameterList) { - hasDefaultTypes = std::any_of(tf->typeParams.begin(), tf->typeParams.end(), [](auto&& el) { - return el.defaultValue.has_value(); - }); - hasDefaultPacks = std::any_of(tf->typePackParams.begin(), tf->typePackParams.end(), [](auto&& el) { - return el.defaultValue.has_value(); - }); - - if (!lit->hasParameterList) - { - if ((!tf->typeParams.empty() && !hasDefaultTypes) || (!tf->typePackParams.empty() && !hasDefaultPacks)) - { - reportError(TypeError{annotation.location, GenericError{"Type parameter list is required"}}); - parameterCountErrorReported = true; - if (!FFlag::LuauErrorRecoveryType) - return errorRecoveryType(scope); - } - } - } - else - { - if (!lit->hasParameterList && !tf->typePackParams.empty()) + if ((!tf->typeParams.empty() && !hasDefaultTypes) || (!tf->typePackParams.empty() && !hasDefaultPacks)) { reportError(TypeError{annotation.location, GenericError{"Type parameter list is required"}}); + parameterCountErrorReported = true; if (!FFlag::LuauErrorRecoveryType) return errorRecoveryType(scope); } @@ -4986,72 +4934,69 @@ TypeId TypeChecker::resolveType(const ScopePtr& scope, const AstType& annotation if (typePackParams.empty() && !extraTypes.empty()) typePackParams.push_back(addTypePack(extraTypes)); - if (FFlag::LuauTypeAliasDefaults) + size_t typesProvided = typeParams.size(); + size_t typesRequired = tf->typeParams.size(); + + size_t packsProvided = typePackParams.size(); + size_t packsRequired = tf->typePackParams.size(); + + bool notEnoughParameters = + (typesProvided < typesRequired && packsProvided == 0) || (typesProvided == typesRequired && packsProvided < packsRequired); + bool hasDefaultParameters = hasDefaultTypes || hasDefaultPacks; + + // Add default type and type pack parameters if that's required and it's possible + if (notEnoughParameters && hasDefaultParameters) { - size_t typesProvided = typeParams.size(); - size_t typesRequired = tf->typeParams.size(); + // 'applyTypeFunction' is used to substitute default types that reference previous generic types + ApplyTypeFunction applyTypeFunction{¤tModule->internalTypes, scope->level}; - size_t packsProvided = typePackParams.size(); - size_t packsRequired = tf->typePackParams.size(); + for (size_t i = 0; i < typesProvided; ++i) + applyTypeFunction.typeArguments[tf->typeParams[i].ty] = typeParams[i]; - bool notEnoughParameters = - (typesProvided < typesRequired && packsProvided == 0) || (typesProvided == typesRequired && packsProvided < packsRequired); - bool hasDefaultParameters = hasDefaultTypes || hasDefaultPacks; - - // Add default type and type pack parameters if that's required and it's possible - if (notEnoughParameters && hasDefaultParameters) + if (typesProvided < typesRequired) { - // 'applyTypeFunction' is used to substitute default types that reference previous generic types - ApplyTypeFunction applyTypeFunction{¤tModule->internalTypes, scope->level}; - - for (size_t i = 0; i < typesProvided; ++i) - applyTypeFunction.typeArguments[tf->typeParams[i].ty] = typeParams[i]; - - if (typesProvided < typesRequired) + for (size_t i = typesProvided; i < typesRequired; ++i) { - for (size_t i = typesProvided; i < typesRequired; ++i) + TypeId defaultTy = tf->typeParams[i].defaultValue.value_or(nullptr); + + if (!defaultTy) + break; + + std::optional maybeInstantiated = applyTypeFunction.substitute(defaultTy); + + if (!maybeInstantiated.has_value()) { - TypeId defaultTy = tf->typeParams[i].defaultValue.value_or(nullptr); - - if (!defaultTy) - break; - - std::optional maybeInstantiated = applyTypeFunction.substitute(defaultTy); - - if (!maybeInstantiated.has_value()) - { - reportError(annotation.location, UnificationTooComplex{}); - maybeInstantiated = errorRecoveryType(scope); - } - - applyTypeFunction.typeArguments[tf->typeParams[i].ty] = *maybeInstantiated; - typeParams.push_back(*maybeInstantiated); + reportError(annotation.location, UnificationTooComplex{}); + maybeInstantiated = errorRecoveryType(scope); } + + applyTypeFunction.typeArguments[tf->typeParams[i].ty] = *maybeInstantiated; + typeParams.push_back(*maybeInstantiated); } + } - for (size_t i = 0; i < packsProvided; ++i) - applyTypeFunction.typePackArguments[tf->typePackParams[i].tp] = typePackParams[i]; + for (size_t i = 0; i < packsProvided; ++i) + applyTypeFunction.typePackArguments[tf->typePackParams[i].tp] = typePackParams[i]; - if (packsProvided < packsRequired) + if (packsProvided < packsRequired) + { + for (size_t i = packsProvided; i < packsRequired; ++i) { - for (size_t i = packsProvided; i < packsRequired; ++i) + TypePackId defaultTp = tf->typePackParams[i].defaultValue.value_or(nullptr); + + if (!defaultTp) + break; + + std::optional maybeInstantiated = applyTypeFunction.substitute(defaultTp); + + if (!maybeInstantiated.has_value()) { - TypePackId defaultTp = tf->typePackParams[i].defaultValue.value_or(nullptr); - - if (!defaultTp) - break; - - std::optional maybeInstantiated = applyTypeFunction.substitute(defaultTp); - - if (!maybeInstantiated.has_value()) - { - reportError(annotation.location, UnificationTooComplex{}); - maybeInstantiated = errorRecoveryTypePack(scope); - } - - applyTypeFunction.typePackArguments[tf->typePackParams[i].tp] = *maybeInstantiated; - typePackParams.push_back(*maybeInstantiated); + reportError(annotation.location, UnificationTooComplex{}); + maybeInstantiated = errorRecoveryTypePack(scope); } + + applyTypeFunction.typePackArguments[tf->typePackParams[i].tp] = *maybeInstantiated; + typePackParams.push_back(*maybeInstantiated); } } } @@ -5343,12 +5288,12 @@ TypeId TypeChecker::instantiateTypeFun(const ScopePtr& scope, const TypeFun& tf, TypeId instantiated = *maybeInstantiated; - // TODO: CLI-46926 it's not a good idea to rename the type here TypeId target = follow(instantiated); bool needsClone = follow(tf.type) == target; + bool shouldMutate = (!FFlag::LuauOnlyMutateInstantiatedTables || getTableType(tf.type)); TableTypeVar* ttv = getMutableTableType(target); - - if (ttv && needsClone) + + if (shouldMutate && ttv && needsClone) { // Substitution::clone is a shallow clone. If this is a metatable type, we // want to mutate its table, so we need to explicitly clone that table as @@ -5368,7 +5313,7 @@ TypeId TypeChecker::instantiateTypeFun(const ScopePtr& scope, const TypeFun& tf, } } - if (ttv) + if (shouldMutate && ttv) { ttv->instantiatedTypeParams = typeParams; ttv->instantiatedTypePackParams = typePackParams; @@ -5382,7 +5327,7 @@ GenericTypeDefinitions TypeChecker::createGenericTypes(const ScopePtr& scope, st { LUAU_ASSERT(scope->parent); - const TypeLevel level = (FFlag::LuauQuantifyInPlace2 && levelOpt) ? *levelOpt : scope->level; + const TypeLevel level = levelOpt.value_or(scope->level); std::vector generics; @@ -5390,7 +5335,7 @@ GenericTypeDefinitions TypeChecker::createGenericTypes(const ScopePtr& scope, st { std::optional defaultValue; - if (FFlag::LuauTypeAliasDefaults && generic.defaultValue) + if (generic.defaultValue) defaultValue = resolveType(scope, *generic.defaultValue); Name n = generic.name.value; @@ -5426,7 +5371,7 @@ GenericTypeDefinitions TypeChecker::createGenericTypes(const ScopePtr& scope, st { std::optional defaultValue; - if (FFlag::LuauTypeAliasDefaults && genericPack.defaultValue) + if (genericPack.defaultValue) defaultValue = resolveTypePack(scope, *genericPack.defaultValue); Name n = genericPack.name.value; diff --git a/Analysis/src/TypeVar.cpp b/Analysis/src/TypeVar.cpp index a1dcfdbe..5af2c8a6 100644 --- a/Analysis/src/TypeVar.cpp +++ b/Analysis/src/TypeVar.cpp @@ -24,6 +24,7 @@ LUAU_FASTINTVARIABLE(LuauTypeMaximumStringifierLength, 500) LUAU_FASTINTVARIABLE(LuauTableTypeMaximumStringifierLength, 0) LUAU_FASTINT(LuauTypeInferRecursionLimit) LUAU_FASTFLAG(LuauErrorRecoveryType) +LUAU_FASTFLAG(LuauSubtypingAddOptPropsToUnsealedTables) LUAU_FASTFLAG(LuauDiscriminableUnions2) namespace Luau @@ -157,6 +158,7 @@ bool isNumber(TypeId ty) return isPrim(ty, PrimitiveTypeVar::Number); } +// Returns true when ty is a subtype of string bool isString(TypeId ty) { if (isPrim(ty, PrimitiveTypeVar::String) || get(get(follow(ty)))) @@ -168,6 +170,27 @@ bool isString(TypeId ty) return false; } +// Returns true when ty is a supertype of string +bool maybeString(TypeId ty) +{ + if (FFlag::LuauSubtypingAddOptPropsToUnsealedTables) + { + ty = follow(ty); + + if (isPrim(ty, PrimitiveTypeVar::String) || get(ty)) + return true; + + if (auto utv = get(ty)) + return std::any_of(begin(utv), end(utv), maybeString); + + return false; + } + else + { + return isString(ty); + } +} + bool isThread(TypeId ty) { return isPrim(ty, PrimitiveTypeVar::Thread); @@ -684,7 +707,7 @@ TypeId SingletonTypes::makeStringMetatable() {"sub", {makeFunction(*arena, stringType, {}, {}, {numberType, optionalNumber}, {}, {stringType})}}, {"upper", {stringToStringType}}, {"split", {makeFunction(*arena, stringType, {}, {}, {optionalString}, {}, - {arena->addType(TableTypeVar{{}, TableIndexer{numberType, stringType}, TypeLevel{}})})}}, + {arena->addType(TableTypeVar{{}, TableIndexer{numberType, stringType}, TypeLevel{}, TableState::Sealed})})}}, {"pack", {arena->addType(FunctionTypeVar{ arena->addTypePack(TypePack{{stringType}, anyTypePack}), oneStringPack, @@ -761,6 +784,8 @@ void persist(TypeId ty) } else if (auto ttv = get(t)) { + LUAU_ASSERT(ttv->state != TableState::Free && ttv->state != TableState::Unsealed); + for (const auto& [_name, prop] : ttv->props) queue.push_back(prop.type); diff --git a/Analysis/src/Unifier.cpp b/Analysis/src/Unifier.cpp index d0eba013..6c29486a 100644 --- a/Analysis/src/Unifier.cpp +++ b/Analysis/src/Unifier.cpp @@ -18,11 +18,13 @@ LUAU_FASTFLAG(LuauImmutableTypes) LUAU_FASTFLAG(LuauUseCommittingTxnLog) LUAU_FASTINTVARIABLE(LuauTypeInferIterationLimit, 2000); LUAU_FASTFLAGVARIABLE(LuauTableSubtypingVariance2, false); -LUAU_FASTFLAGVARIABLE(LuauTableUnificationEarlyTest, false) LUAU_FASTFLAG(LuauSingletonTypes) LUAU_FASTFLAG(LuauErrorRecoveryType); +LUAU_FASTFLAGVARIABLE(LuauSubtypingAddOptPropsToUnsealedTables, false) LUAU_FASTFLAGVARIABLE(LuauFollowWithCommittingTxnLogInAnyUnification, false) LUAU_FASTFLAGVARIABLE(LuauWidenIfSupertypeIsFree, false) +LUAU_FASTFLAGVARIABLE(LuauDifferentOrderOfUnificationDoesntMatter, false) +LUAU_FASTFLAGVARIABLE(LuauTxnLogSeesTypePacks2, true) namespace Luau { @@ -329,7 +331,7 @@ Unifier::Unifier(TypeArena* types, Mode mode, const Location& location, Variance LUAU_ASSERT(sharedState.iceHandler); } -Unifier::Unifier(TypeArena* types, Mode mode, std::vector>* sharedSeen, const Location& location, +Unifier::Unifier(TypeArena* types, Mode mode, std::vector>* sharedSeen, const Location& location, Variance variance, UnifierSharedState& sharedState, TxnLog* parentLog) : types(types) , mode(mode) @@ -656,26 +658,85 @@ void Unifier::tryUnifyUnionWithType(TypeId subTy, const UnionTypeVar* uv, TypeId failed = true; } - if (FFlag::LuauUseCommittingTxnLog) + if (FFlag::LuauDifferentOrderOfUnificationDoesntMatter) { - if (i == count - 1) - { - log.concat(std::move(innerState.log)); - } + if (!FFlag::LuauUseCommittingTxnLog) + innerState.DEPRECATED_log.rollback(); } else { - if (i != count - 1) + if (FFlag::LuauUseCommittingTxnLog) { - innerState.DEPRECATED_log.rollback(); + if (i == count - 1) + { + log.concat(std::move(innerState.log)); + } } else { - DEPRECATED_log.concat(std::move(innerState.DEPRECATED_log)); + if (i != count - 1) + { + innerState.DEPRECATED_log.rollback(); + } + else + { + DEPRECATED_log.concat(std::move(innerState.DEPRECATED_log)); + } } - } - ++i; + ++i; + } + } + + // even if A | B <: T fails, we want to bind some options of T with A | B iff A | B was a subtype of that option. + if (FFlag::LuauDifferentOrderOfUnificationDoesntMatter) + { + auto tryBind = [this, subTy](TypeId superOption) { + superOption = FFlag::LuauUseCommittingTxnLog ? log.follow(superOption) : follow(superOption); + + // just skip if the superOption is not free-ish. + auto ttv = log.getMutable(superOption); + if (!log.is(superOption) && (!ttv || ttv->state != TableState::Free)) + return; + + // Since we have already checked if S <: T, checking it again will not queue up the type for replacement. + // So we'll have to do it ourselves. We assume they unified cleanly if they are still in the seen set. + if (FFlag::LuauUseCommittingTxnLog) + { + if (log.haveSeen(subTy, superOption)) + { + // TODO: would it be nice for TxnLog::replace to do this? + if (log.is(superOption)) + log.bindTable(superOption, subTy); + else + log.replace(superOption, *subTy); + } + } + else + { + if (DEPRECATED_log.haveSeen(subTy, superOption)) + { + if (auto ttv = getMutable(superOption)) + { + DEPRECATED_log(ttv); + ttv->boundTo = subTy; + } + else + { + DEPRECATED_log(superOption); + *asMutable(superOption) = BoundTypeVar(subTy); + } + } + } + }; + + if (auto utv = (FFlag::LuauUseCommittingTxnLog ? log.getMutable(superTy) : get(superTy))) + { + for (TypeId ty : utv) + tryBind(ty); + } + else + tryBind(superTy); } if (unificationTooComplex) @@ -1163,6 +1224,9 @@ void Unifier::tryUnify_(TypePackId subTp, TypePackId superTp, bool isFunctionCal if (superTp == subTp) return; + if (FFlag::LuauTxnLogSeesTypePacks2 && log.haveSeen(superTp, subTp)) + return; + if (log.getMutable(superTp)) { occursCheck(superTp, subTp); @@ -1365,6 +1429,9 @@ void Unifier::tryUnify_(TypePackId subTp, TypePackId superTp, bool isFunctionCal if (superTp == subTp) return; + if (FFlag::LuauTxnLogSeesTypePacks2 && DEPRECATED_log.haveSeen(superTp, subTp)) + return; + if (get(superTp)) { occursCheck(superTp, subTp); @@ -1619,6 +1686,17 @@ void Unifier::tryUnifyFunctions(TypeId subTy, TypeId superTy, bool isFunctionCal DEPRECATED_log.pushSeen(superFunction->generics[i], subFunction->generics[i]); } + if (FFlag::LuauTxnLogSeesTypePacks2) + { + for (size_t i = 0; i < numGenericPacks; i++) + { + if (FFlag::LuauUseCommittingTxnLog) + log.pushSeen(superFunction->genericPacks[i], subFunction->genericPacks[i]); + else + DEPRECATED_log.pushSeen(superFunction->genericPacks[i], subFunction->genericPacks[i]); + } + } + CountMismatch::Context context = ctx; if (!isFunctionCall) @@ -1708,6 +1786,17 @@ void Unifier::tryUnifyFunctions(TypeId subTy, TypeId superTy, bool isFunctionCal ctx = context; + if (FFlag::LuauTxnLogSeesTypePacks2) + { + for (int i = int(numGenericPacks) - 1; 0 <= i; i--) + { + if (FFlag::LuauUseCommittingTxnLog) + log.popSeen(superFunction->genericPacks[i], subFunction->genericPacks[i]); + else + DEPRECATED_log.popSeen(superFunction->genericPacks[i], subFunction->genericPacks[i]); + } + } + for (int i = int(numGenerics) - 1; 0 <= i; i--) { if (FFlag::LuauUseCommittingTxnLog) @@ -1760,7 +1849,7 @@ void Unifier::tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) std::vector extraProperties; // Optimization: First test that the property sets are compatible without doing any recursive unification - if (FFlag::LuauTableUnificationEarlyTest && !subTable->indexer && subTable->state != TableState::Free) + if (!subTable->indexer && subTable->state != TableState::Free) { for (const auto& [propName, superProp] : superTable->props) { @@ -1769,7 +1858,7 @@ void Unifier::tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) bool isAny = FFlag::LuauUseCommittingTxnLog ? log.getMutable(log.follow(superProp.type)) : get(follow(superProp.type)); - if (subIter == subTable->props.end() && !isOptional(superProp.type) && !isAny) + if (subIter == subTable->props.end() && (!FFlag::LuauSubtypingAddOptPropsToUnsealedTables || subTable->state == TableState::Unsealed) && !isOptional(superProp.type) && !isAny) missingProperties.push_back(propName); } @@ -1781,7 +1870,7 @@ void Unifier::tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) } // And vice versa if we're invariant - if (FFlag::LuauTableUnificationEarlyTest && variance == Invariant && !superTable->indexer && superTable->state != TableState::Unsealed && + if (variance == Invariant && !superTable->indexer && superTable->state != TableState::Unsealed && superTable->state != TableState::Free) { for (const auto& [propName, subProp] : subTable->props) @@ -1790,7 +1879,7 @@ void Unifier::tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) bool isAny = FFlag::LuauUseCommittingTxnLog ? log.getMutable(log.follow(subProp.type)) : get(follow(subProp.type)); - if (superIter == superTable->props.end() && !isOptional(subProp.type) && !isAny) + if (superIter == superTable->props.end() && (FFlag::LuauSubtypingAddOptPropsToUnsealedTables || (!isOptional(subProp.type) && !isAny))) extraProperties.push_back(propName); } @@ -1830,7 +1919,7 @@ void Unifier::tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) innerState.DEPRECATED_log.rollback(); } } - else if (subTable->indexer && isString(subTable->indexer->indexType)) + else if (subTable->indexer && maybeString(subTable->indexer->indexType)) { // TODO: read-only indexers don't need invariance // TODO: really we should only allow this if prop.type is optional. @@ -1855,9 +1944,11 @@ void Unifier::tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) innerState.DEPRECATED_log.rollback(); } } - else if (isOptional(prop.type) || get(follow(prop.type))) - // TODO: this case is unsound, but without it our test suite fails. CLI-46031 + else if ((!FFlag::LuauSubtypingAddOptPropsToUnsealedTables || subTable->state == TableState::Unsealed) && (isOptional(prop.type) || get(follow(prop.type)))) + // This is sound because unsealed table types are precise, so `{ p : T } <: { p : T, q : U? }` + // since if `t : { p : T }` then we are guaranteed that `t.q` is `nil`. // TODO: should isOptional(anyType) be true? + // TODO: if the supertype is written to, the subtype may no longer be precise (alias analysis?) { } else if (subTable->state == TableState::Free) @@ -1887,7 +1978,7 @@ void Unifier::tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) // If both lt and rt contain the property, then // we're done since we already unified them above } - else if (superTable->indexer && isString(superTable->indexer->indexType)) + else if (superTable->indexer && maybeString(superTable->indexer->indexType)) { // TODO: read-only indexers don't need invariance // TODO: really we should only allow this if prop.type is optional. @@ -1936,9 +2027,7 @@ void Unifier::tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) else if (variance == Covariant) { } - else if (isOptional(prop.type) || get(follow(prop.type))) - // TODO: this case is unsound, but without it our test suite fails. CLI-46031 - // TODO: should isOptional(anyType) be true? + else if (!FFlag::LuauSubtypingAddOptPropsToUnsealedTables && (isOptional(prop.type) || get(follow(prop.type)))) { } else if (superTable->state == TableState::Free) @@ -2333,7 +2422,7 @@ void Unifier::tryUnifySealedTables(TypeId subTy, TypeId superTy, bool isIntersec bool errorReported = false; // Optimization: First test that the property sets are compatible without doing any recursive unification - if (FFlag::LuauTableUnificationEarlyTest && !subTable->indexer) + if (!subTable->indexer) { for (const auto& [propName, superProp] : superTable->props) { diff --git a/Ast/src/Parser.cpp b/Ast/src/Parser.cpp index 8767daa0..1cb8f134 100644 --- a/Ast/src/Parser.cpp +++ b/Ast/src/Parser.cpp @@ -11,7 +11,6 @@ LUAU_FASTINTVARIABLE(LuauRecursionLimit, 1000) LUAU_FASTINTVARIABLE(LuauParseErrorLimit, 100) LUAU_FASTFLAGVARIABLE(LuauParseSingletonTypes, false) -LUAU_FASTFLAGVARIABLE(LuauParseTypeAliasDefaults, false) LUAU_FASTFLAGVARIABLE(LuauParseAllHotComments, false) LUAU_FASTFLAGVARIABLE(LuauTableFieldFunctionDebugname, false) @@ -779,7 +778,7 @@ AstStat* Parser::parseTypeAlias(const Location& start, bool exported) if (!name) name = Name(nameError, lexer.current().location); - auto [generics, genericPacks] = parseGenericTypeList(/* withDefaultValues= */ FFlag::LuauParseTypeAliasDefaults); + auto [generics, genericPacks] = parseGenericTypeList(/* withDefaultValues= */ true); expectAndConsume('=', "type alias"); diff --git a/CLI/Repl.cpp b/CLI/Repl.cpp index 13304d57..5fd6d341 100644 --- a/CLI/Repl.cpp +++ b/CLI/Repl.cpp @@ -413,29 +413,22 @@ static void completeRepl(ic_completion_env_t* cenv, const char* editBuffer) ic_complete_word(cenv, editBuffer, icGetCompletions, isMethodOrFunctionChar); } -struct LinenoiseScopedHistory +static void loadHistory(const char* name) { - LinenoiseScopedHistory() + std::string path; + + if (const char* home = getenv("HOME")) { - const std::string name(".luau_history"); - - if (const char* home = getenv("HOME")) - { - historyFilepath = joinPaths(home, name); - } - else if (const char* userProfile = getenv("USERPROFILE")) - { - historyFilepath = joinPaths(userProfile, name); - } - - if (!historyFilepath.empty()) - ic_set_history(historyFilepath.c_str(), -1 /* default entries (= 200) */); + path = joinPaths(home, name); + } + else if (const char* userProfile = getenv("USERPROFILE")) + { + path = joinPaths(userProfile, name); } - ~LinenoiseScopedHistory() {} - - std::string historyFilepath; -}; + if (!path.empty()) + ic_set_history(path.c_str(), -1 /* default entries (= 200) */); +} static void runReplImpl(lua_State* L) { @@ -447,8 +440,10 @@ static void runReplImpl(lua_State* L) // Prevent auto insertion of braces ic_enable_brace_insertion(false); + // Loads history from the given file; isocline automatically saves the history on process exit + loadHistory(".luau_history"); + std::string buffer; - LinenoiseScopedHistory scopedHistory; for (;;) { diff --git a/VM/include/lua.h b/VM/include/lua.h index af0e2835..0a561f27 100644 --- a/VM/include/lua.h +++ b/VM/include/lua.h @@ -265,6 +265,8 @@ LUA_API double lua_clock(); LUA_API void lua_setuserdatadtor(lua_State* L, int tag, void (*dtor)(void*)); +LUA_API void lua_clonefunction(lua_State* L, int idx); + /* ** reference system, can be used to pin objects */ @@ -324,6 +326,7 @@ typedef struct lua_Debug lua_Debug; /* activation record */ /* Functions to be called by the debugger in specific events */ typedef void (*lua_Hook)(lua_State* L, lua_Debug* ar); +LUA_API int lua_stackdepth(lua_State* L); LUA_API int lua_getinfo(lua_State* L, int level, const char* what, lua_Debug* ar); LUA_API int lua_getargument(lua_State* L, int level, int n); LUA_API const char* lua_getlocal(lua_State* L, int level, int n); diff --git a/VM/src/lapi.cpp b/VM/src/lapi.cpp index 39c76e08..f7f15442 100644 --- a/VM/src/lapi.cpp +++ b/VM/src/lapi.cpp @@ -14,7 +14,7 @@ #include -LUAU_FASTFLAGVARIABLE(LuauGcForwardMetatableBarrier, false) +LUAU_FASTFLAG(LuauGcAdditionalStats) const char* lua_ident = "$Lua: Lua 5.1.4 Copyright (C) 1994-2008 Lua.org, PUC-Rio $\n" "$Authors: R. Ierusalimschy, L. H. de Figueiredo & W. Celes $\n" @@ -876,16 +876,7 @@ int lua_setmetatable(lua_State* L, int objindex) luaG_runerror(L, "Attempt to modify a readonly table"); hvalue(obj)->metatable = mt; if (mt) - { - if (FFlag::LuauGcForwardMetatableBarrier) - { - luaC_objbarrier(L, hvalue(obj), mt); - } - else - { - luaC_objbarriert(L, hvalue(obj), mt); - } - } + luaC_objbarrier(L, hvalue(obj), mt); break; } case LUA_TUSERDATA: @@ -1069,6 +1060,8 @@ int lua_gc(lua_State* L, int what, int data) g->GCthreshold = 0; bool waspaused = g->gcstate == GCSpause; + double startmarktime = g->gcstats.currcycle.marktime; + double startsweeptime = g->gcstats.currcycle.sweeptime; // track how much work the loop will actually perform size_t actualwork = 0; @@ -1086,6 +1079,31 @@ int lua_gc(lua_State* L, int what, int data) } } + if (FFlag::LuauGcAdditionalStats) + { + // record explicit step statistics + GCCycleStats* cyclestats = g->gcstate == GCSpause ? &g->gcstats.lastcycle : &g->gcstats.currcycle; + + double totalmarktime = cyclestats->marktime - startmarktime; + double totalsweeptime = cyclestats->sweeptime - startsweeptime; + + if (totalmarktime > 0.0) + { + cyclestats->markexplicitsteps++; + + if (totalmarktime > cyclestats->markmaxexplicittime) + cyclestats->markmaxexplicittime = totalmarktime; + } + + if (totalsweeptime > 0.0) + { + cyclestats->sweepexplicitsteps++; + + if (totalsweeptime > cyclestats->sweepmaxexplicittime) + cyclestats->sweepmaxexplicittime = totalsweeptime; + } + } + // if cycle hasn't finished, advance threshold forward for the amount of extra work performed if (g->gcstate != GCSpause) { @@ -1299,6 +1317,18 @@ void lua_setuserdatadtor(lua_State* L, int tag, void (*dtor)(void*)) L->global->udatagc[tag] = dtor; } +LUA_API void lua_clonefunction(lua_State* L, int idx) +{ + StkId p = index2addr(L, idx); + api_check(L, isLfunction(p)); + + luaC_checkthreadsleep(L); + + Closure* cl = clvalue(p); + Closure* newcl = luaF_newLclosure(L, 0, L->gt, cl->l.p); + setclvalue(L, L->top - 1, newcl); +} + lua_Callbacks* lua_callbacks(lua_State* L) { return &L->global->cb; diff --git a/VM/src/laux.cpp b/VM/src/laux.cpp index 71975a52..9a6f7793 100644 --- a/VM/src/laux.cpp +++ b/VM/src/laux.cpp @@ -11,8 +11,6 @@ #include -LUAU_FASTFLAG(LuauSchubfach) - /* convert a stack index to positive */ #define abs_index(L, i) ((i) > 0 || (i) <= LUA_REGISTRYINDEX ? (i) : lua_gettop(L) + (i) + 1) @@ -480,18 +478,13 @@ const char* luaL_tolstring(lua_State* L, int idx, size_t* len) switch (lua_type(L, idx)) { case LUA_TNUMBER: - if (FFlag::LuauSchubfach) - { - double n = lua_tonumber(L, idx); - char s[LUAI_MAXNUM2STR]; - char* e = luai_num2str(s, n); - lua_pushlstring(L, s, e - s); - } - else - { - lua_pushstring(L, lua_tostring(L, idx)); - } + { + double n = lua_tonumber(L, idx); + char s[LUAI_MAXNUM2STR]; + char* e = luai_num2str(s, n); + lua_pushlstring(L, s, e - s); break; + } case LUA_TSTRING: lua_pushvalue(L, idx); break; @@ -505,29 +498,18 @@ const char* luaL_tolstring(lua_State* L, int idx, size_t* len) { const float* v = lua_tovector(L, idx); - if (FFlag::LuauSchubfach) + char s[LUAI_MAXNUM2STR * LUA_VECTOR_SIZE]; + char* e = s; + for (int i = 0; i < LUA_VECTOR_SIZE; ++i) { - char s[LUAI_MAXNUM2STR * LUA_VECTOR_SIZE]; - char* e = s; - for (int i = 0; i < LUA_VECTOR_SIZE; ++i) + if (i != 0) { - if (i != 0) - { - *e++ = ','; - *e++ = ' '; - } - e = luai_num2str(e, v[i]); + *e++ = ','; + *e++ = ' '; } - lua_pushlstring(L, s, e - s); - } - else - { -#if LUA_VECTOR_SIZE == 4 - lua_pushfstring(L, LUA_NUMBER_FMT ", " LUA_NUMBER_FMT ", " LUA_NUMBER_FMT ", " LUA_NUMBER_FMT, v[0], v[1], v[2], v[3]); -#else - lua_pushfstring(L, LUA_NUMBER_FMT ", " LUA_NUMBER_FMT ", " LUA_NUMBER_FMT, v[0], v[1], v[2]); -#endif + e = luai_num2str(e, v[i]); } + lua_pushlstring(L, s, e - s); break; } default: diff --git a/VM/src/ldebug.cpp b/VM/src/ldebug.cpp index a4f93c62..7a9947b7 100644 --- a/VM/src/ldebug.cpp +++ b/VM/src/ldebug.cpp @@ -168,6 +168,11 @@ static int auxgetinfo(lua_State* L, const char* what, lua_Debug* ar, Closure* f, return status; } +int lua_stackdepth(lua_State* L) +{ + return int(L->ci - L->base_ci); +} + int lua_getinfo(lua_State* L, int level, const char* what, lua_Debug* ar) { int status = 0; diff --git a/VM/src/lgc.cpp b/VM/src/lgc.cpp index 8c3a2029..a656854e 100644 --- a/VM/src/lgc.cpp +++ b/VM/src/lgc.cpp @@ -11,6 +11,8 @@ #include "lmem.h" #include "ludata.h" +LUAU_FASTFLAGVARIABLE(LuauGcAdditionalStats, false) + #include #define GC_SWEEPMAX 40 @@ -53,17 +55,28 @@ static void recordGcStateTime(global_State* g, int startgcstate, double seconds, case GCSpause: // record root mark time if we have switched to next state if (g->gcstate == GCSpropagate) + { g->gcstats.currcycle.marktime += seconds; + + if (FFlag::LuauGcAdditionalStats && assist) + g->gcstats.currcycle.markassisttime += seconds; + } break; case GCSpropagate: case GCSpropagateagain: g->gcstats.currcycle.marktime += seconds; + + if (FFlag::LuauGcAdditionalStats && assist) + g->gcstats.currcycle.markassisttime += seconds; break; case GCSatomic: g->gcstats.currcycle.atomictime += seconds; break; case GCSsweep: g->gcstats.currcycle.sweeptime += seconds; + + if (FFlag::LuauGcAdditionalStats && assist) + g->gcstats.currcycle.sweepassisttime += seconds; break; default: LUAU_ASSERT(!"Unexpected GC state"); @@ -78,7 +91,7 @@ static void recordGcStateTime(global_State* g, int startgcstate, double seconds, static void startGcCycleStats(global_State* g) { g->gcstats.currcycle.starttimestamp = lua_clock(); - g->gcstats.currcycle.waittime = g->gcstats.currcycle.starttimestamp - g->gcstats.lastcycle.endtimestamp; + g->gcstats.currcycle.pausetime = g->gcstats.currcycle.starttimestamp - g->gcstats.lastcycle.endtimestamp; } static void finishGcCycleStats(global_State* g) @@ -585,10 +598,21 @@ static size_t atomic(lua_State* L) LUAU_ASSERT(g->gcstate == GCSatomic); size_t work = 0; + double currts = lua_clock(); + double prevts = currts; + /* remark occasional upvalues of (maybe) dead threads */ work += remarkupvals(g); /* traverse objects caught by write barrier and by 'remarkupvals' */ work += propagateall(g); + + if (FFlag::LuauGcAdditionalStats) + { + currts = lua_clock(); + g->gcstats.currcycle.atomictimeupval += currts - prevts; + prevts = currts; + } + /* remark weak tables */ g->gray = g->weak; g->weak = NULL; @@ -596,16 +620,41 @@ static size_t atomic(lua_State* L) markobject(g, L); /* mark running thread */ markmt(g); /* mark basic metatables (again) */ work += propagateall(g); + + if (FFlag::LuauGcAdditionalStats) + { + currts = lua_clock(); + g->gcstats.currcycle.atomictimeweak += currts - prevts; + prevts = currts; + } + /* remark gray again */ g->gray = g->grayagain; g->grayagain = NULL; work += propagateall(g); - work += cleartable(L, g->weak); /* remove collected objects from weak tables */ + + if (FFlag::LuauGcAdditionalStats) + { + currts = lua_clock(); + g->gcstats.currcycle.atomictimegray += currts - prevts; + prevts = currts; + } + + /* remove collected objects from weak tables */ + work += cleartable(L, g->weak); g->weak = NULL; + + if (FFlag::LuauGcAdditionalStats) + { + currts = lua_clock(); + g->gcstats.currcycle.atomictimeclear += currts - prevts; + } + /* flip current white */ g->currentwhite = cast_byte(otherwhite(g)); g->sweepgcopage = g->allgcopages; g->gcstate = GCSsweep; + return work; } @@ -693,6 +742,9 @@ static size_t gcstep(lua_State* L, size_t limit) if (!g->gray) { + if (FFlag::LuauGcAdditionalStats) + g->gcstats.currcycle.propagatework = g->gcstats.currcycle.explicitwork + g->gcstats.currcycle.assistwork; + // perform one iteration over 'gray again' list g->gray = g->grayagain; g->grayagain = NULL; @@ -710,6 +762,10 @@ static size_t gcstep(lua_State* L, size_t limit) if (!g->gray) /* no more `gray' objects */ { + if (FFlag::LuauGcAdditionalStats) + g->gcstats.currcycle.propagateagainwork = + g->gcstats.currcycle.explicitwork + g->gcstats.currcycle.assistwork - g->gcstats.currcycle.propagatework; + g->gcstate = GCSatomic; } break; @@ -811,6 +867,12 @@ static size_t getheaptrigger(global_State* g, size_t heapgoal) void luaC_step(lua_State* L, bool assist) { global_State* g = L->global; + + if (assist) + g->gcstats.currcycle.assistrequests += g->gcstepsize; + else + g->gcstats.currcycle.explicitrequests += g->gcstepsize; + int lim = (g->gcstepsize / 100) * g->gcstepmul; /* how much to work */ LUAU_ASSERT(g->totalbytes >= g->GCthreshold); size_t debt = g->totalbytes - g->GCthreshold; @@ -833,6 +895,11 @@ void luaC_step(lua_State* L, bool assist) recordGcStateTime(g, lastgcstate, lua_clock() - lasttimestamp, assist); + if (lastgcstate == GCSpropagate) + g->gcstats.currcycle.markrequests += g->gcstepsize; + else if (lastgcstate == GCSsweep) + g->gcstats.currcycle.sweeprequests += g->gcstepsize; + // at the end of the last cycle if (g->gcstate == GCSpause) { @@ -844,6 +911,9 @@ void luaC_step(lua_State* L, bool assist) finishGcCycleStats(g); + if (FFlag::LuauGcAdditionalStats) + g->gcstats.currcycle.starttotalsizebytes = g->totalbytes; + g->gcstats.currcycle.heapgoalsizebytes = heapgoal; g->gcstats.currcycle.heaptriggersizebytes = heaptrigger; } diff --git a/VM/src/lgc.h b/VM/src/lgc.h index 253e269f..cbeeebd4 100644 --- a/VM/src/lgc.h +++ b/VM/src/lgc.h @@ -111,13 +111,6 @@ luaC_barrierf(L, obj2gco(p), obj2gco(o)); \ } -// TODO: remove with FFlagLuauGcForwardMetatableBarrier -#define luaC_objbarriert(L, t, o) \ - { \ - if (isblack(obj2gco(t)) && iswhite(obj2gco(o))) \ - luaC_barriertable(L, t, obj2gco(o)); \ - } - #define luaC_upvalbarrier(L, uv, tv) \ { \ if (iscollectable(tv) && iswhite(gcvalue(tv)) && (!(uv) || ((UpVal*)uv)->v != &((UpVal*)uv)->u.value)) \ diff --git a/VM/src/lnumprint.cpp b/VM/src/lnumprint.cpp index 2fd0f1bb..d64e3ca4 100644 --- a/VM/src/lnumprint.cpp +++ b/VM/src/lnumprint.cpp @@ -6,7 +6,6 @@ #include "lcommon.h" #include -#include // TODO: Remove with LuauSchubfach #ifdef _MSC_VER #include @@ -18,8 +17,6 @@ // The code uses the notation from the paper for local variables where appropriate, and refers to paper sections/figures/results. -LUAU_FASTFLAGVARIABLE(LuauSchubfach, false) - // 9.8.2. Precomputed table for 128-bit overestimates of powers of 10 (see figure 3 for table bounds) // To avoid storing 616 128-bit numbers directly we use a technique inspired by Dragonbox implementation and store 16 consecutive // powers using a 128-bit baseline and a bitvector with 1-bit scale and 3-bit offset for the delta between each entry and base*5^k @@ -275,12 +272,6 @@ inline char* trimzero(char* end) char* luai_num2str(char* buf, double n) { - if (!FFlag::LuauSchubfach) - { - snprintf(buf, LUAI_MAXNUM2STR, LUA_NUMBER_FMT, n); - return buf + strlen(buf); - } - // IEEE-754 union { diff --git a/VM/src/lnumutils.h b/VM/src/lnumutils.h index fba07bc3..549b4630 100644 --- a/VM/src/lnumutils.h +++ b/VM/src/lnumutils.h @@ -55,7 +55,6 @@ LUAU_FASTMATH_END #define luai_num2unsigned(i, n) ((i) = (unsigned)(long long)(n)) #endif -#define LUA_NUMBER_FMT "%.14g" /* TODO: Remove with LuauSchubfach */ #define LUAI_MAXNUM2STR 48 LUAI_FUNC char* luai_num2str(char* buf, double n); diff --git a/VM/src/lstate.h b/VM/src/lstate.h index 3ee96718..b2bedb48 100644 --- a/VM/src/lstate.h +++ b/VM/src/lstate.h @@ -77,25 +77,46 @@ typedef struct CallInfo struct GCCycleStats { + size_t starttotalsizebytes = 0; size_t heapgoalsizebytes = 0; size_t heaptriggersizebytes = 0; - double waittime = 0.0; // time from end of the last cycle to the start of a new one + double pausetime = 0.0; // time from end of the last cycle to the start of a new one double starttimestamp = 0.0; double endtimestamp = 0.0; double marktime = 0.0; + double markassisttime = 0.0; + double markmaxexplicittime = 0.0; + size_t markexplicitsteps = 0; + size_t markrequests = 0; double atomicstarttimestamp = 0.0; size_t atomicstarttotalsizebytes = 0; double atomictime = 0.0; + // specific atomic stage parts + double atomictimeupval = 0.0; + double atomictimeweak = 0.0; + double atomictimegray = 0.0; + double atomictimeclear = 0.0; + double sweeptime = 0.0; + double sweepassisttime = 0.0; + double sweepmaxexplicittime = 0.0; + size_t sweepexplicitsteps = 0; + size_t sweeprequests = 0; + + size_t assistrequests = 0; + size_t explicitrequests = 0; size_t assistwork = 0; size_t explicitwork = 0; + size_t propagatework = 0; + size_t propagateagainwork = 0; + size_t endtotalsizebytes = 0; }; diff --git a/VM/src/ltable.cpp b/VM/src/ltable.cpp index 0412ea76..ef0b4b93 100644 --- a/VM/src/ltable.cpp +++ b/VM/src/ltable.cpp @@ -447,7 +447,8 @@ void luaH_free(lua_State* L, Table* t, lua_Page* page) { if (t->node != dummynode) luaM_freearray(L, t->node, sizenode(t), LuaNode, t->memcat); - luaM_freearray(L, t->array, t->sizearray, TValue, t->memcat); + if (t->array) + luaM_freearray(L, t->array, t->sizearray, TValue, t->memcat); luaM_freegco(L, t, sizeof(Table), t->memcat, page); } diff --git a/VM/src/ltablib.cpp b/VM/src/ltablib.cpp index 0d3374ef..00753742 100644 --- a/VM/src/ltablib.cpp +++ b/VM/src/ltablib.cpp @@ -2,6 +2,7 @@ // This code is based on Lua 5.x implementation licensed under MIT License; see lua_LICENSE.txt for details #include "lualib.h" +#include "lapi.h" #include "lstate.h" #include "ltable.h" #include "lstring.h" @@ -9,6 +10,8 @@ #include "ldebug.h" #include "lvm.h" +LUAU_FASTFLAGVARIABLE(LuauTableClone, false) + static int foreachi(lua_State* L) { luaL_checktype(L, 1, LUA_TTABLE); @@ -507,6 +510,23 @@ static int tisfrozen(lua_State* L) return 1; } +static int tclone(lua_State* L) +{ + if (!FFlag::LuauTableClone) + luaG_runerror(L, "table.clone is not available"); + + luaL_checktype(L, 1, LUA_TTABLE); + luaL_argcheck(L, !luaL_getmetafield(L, 1, "__metatable"), 1, "table has a protected metatable"); + + Table* tt = luaH_clone(L, hvalue(L->base)); + + TValue v; + sethvalue(L, &v, tt); + luaA_pushobject(L, &v); + + return 1; +} + static const luaL_Reg tab_funcs[] = { {"concat", tconcat}, {"foreach", foreach}, @@ -524,6 +544,7 @@ static const luaL_Reg tab_funcs[] = { {"clear", tclear}, {"freeze", tfreeze}, {"isfrozen", tisfrozen}, + {"clone", tclone}, {NULL, NULL}, }; diff --git a/bench/gc/test_GC_Boehm_Trees.lua b/bench/gc/test_GC_Boehm_Trees.lua index 1451769f..5abad1d8 100644 --- a/bench/gc/test_GC_Boehm_Trees.lua +++ b/bench/gc/test_GC_Boehm_Trees.lua @@ -74,4 +74,7 @@ function test() end end +bench.runs = 6 +bench.extraRuns = 2 + bench.runCode(test, "GC: Boehm tree") diff --git a/bench/gc/test_GC_Tree_Pruning_Eager.lua b/bench/gc/test_GC_Tree_Pruning_Eager.lua index 514766ae..2111d9ff 100644 --- a/bench/gc/test_GC_Tree_Pruning_Eager.lua +++ b/bench/gc/test_GC_Tree_Pruning_Eager.lua @@ -40,7 +40,7 @@ function test() local tree = { id = 0 } - for i = 1,1000 do + for i = 1,100 do fill_tree(tree, 10) prune_tree(tree, 0) diff --git a/bench/gc/test_GC_Tree_Pruning_Gen.lua b/bench/gc/test_GC_Tree_Pruning_Gen.lua index a8d0f40a..f88bd7f4 100644 --- a/bench/gc/test_GC_Tree_Pruning_Gen.lua +++ b/bench/gc/test_GC_Tree_Pruning_Gen.lua @@ -42,7 +42,7 @@ function test() local tree = { id = 0 } fill_tree(tree, 16) - for i = 1,1000 do + for i = 1,100 do local small_tree = { id = 0 } fill_tree(small_tree, 8) diff --git a/bench/gc/test_GC_Tree_Pruning_Lazy.lua b/bench/gc/test_GC_Tree_Pruning_Lazy.lua index 8cb69192..3ea6bbef 100644 --- a/bench/gc/test_GC_Tree_Pruning_Lazy.lua +++ b/bench/gc/test_GC_Tree_Pruning_Lazy.lua @@ -46,7 +46,7 @@ function test() local tree = { id = 0 } - for i = 1,1000 do + for i = 1,100 do fill_tree(tree, 10) prune_tree(tree, 0) diff --git a/extern/isocline/include/isocline.h b/extern/isocline/include/isocline.h index 0d46cf3f..a7e03ed2 100644 --- a/extern/isocline/include/isocline.h +++ b/extern/isocline/include/isocline.h @@ -259,7 +259,7 @@ void ic_complete_qword( ic_completion_env_t* cenv, const char* prefix, ic_comple /// The `escape_char` is the escaping character, usually `\` but use 0 to not have escape characters. /// The `quote_chars` define the quotes, use NULL for the default `"\'\""` quotes. /// @see ic_complete_word() which uses the default values for `non_word_chars`, `quote_chars` and `\` for escape characters. -void ic_complete_qword_ex( ic_completion_env_t* cenv, const char* prefix, ic_completer_fun_t fun, +void ic_complete_qword_ex( ic_completion_env_t* cenv, const char* prefix, ic_completer_fun_t* fun, ic_is_char_class_fun_t* is_word_char, char escape_char, const char* quote_chars ); /// \} diff --git a/fuzz/number.cpp b/fuzz/number.cpp index 70447409..31c953e3 100644 --- a/fuzz/number.cpp +++ b/fuzz/number.cpp @@ -6,8 +6,6 @@ #include #include -LUAU_FASTFLAG(LuauSchubfach); - #define LUAI_MAXNUM2STR 48 char* luai_num2str(char* buf, double n); @@ -17,8 +15,6 @@ extern "C" int LLVMFuzzerTestOneInput(const uint8_t* Data, size_t Size) if (Size < 8) return 0; - FFlag::LuauSchubfach.value = true; - double num; memcpy(&num, Data, 8); diff --git a/fuzz/proto.cpp b/fuzz/proto.cpp index 912fef23..1022831b 100644 --- a/fuzz/proto.cpp +++ b/fuzz/proto.cpp @@ -59,7 +59,7 @@ void interrupt(lua_State* L, int gc) } } -void* allocate(lua_State* L, void* ud, void* ptr, size_t osize, size_t nsize) +void* allocate(void* ud, void* ptr, size_t osize, size_t nsize) { if (nsize == 0) { diff --git a/tests/Autocomplete.test.cpp b/tests/Autocomplete.test.cpp index 6aadef32..ce890ba8 100644 --- a/tests/Autocomplete.test.cpp +++ b/tests/Autocomplete.test.cpp @@ -15,6 +15,7 @@ LUAU_FASTFLAG(LuauTraceTypesInNonstrictMode2) LUAU_FASTFLAG(LuauSetMetatableDoesNotTimeTravel) LUAU_FASTFLAG(LuauUseCommittingTxnLog) +LUAU_FASTFLAG(LuauTableCloneType) using namespace Luau; @@ -262,7 +263,7 @@ TEST_CASE_FIXTURE(ACFixture, "get_member_completions") auto ac = autocomplete('1'); - CHECK_EQ(16, ac.entryMap.size()); + CHECK_EQ(FFlag::LuauTableCloneType ? 17 : 16, ac.entryMap.size()); CHECK(ac.entryMap.count("find")); CHECK(ac.entryMap.count("pack")); CHECK(!ac.entryMap.count("math")); @@ -2235,7 +2236,7 @@ TEST_CASE_FIXTURE(ACFixture, "autocompleteSource") auto ac = autocompleteSource(frontend, source, Position{1, 24}, nullCallback).result; - CHECK_EQ(16, ac.entryMap.size()); + CHECK_EQ(FFlag::LuauTableCloneType ? 17 : 16, ac.entryMap.size()); CHECK(ac.entryMap.count("find")); CHECK(ac.entryMap.count("pack")); CHECK(!ac.entryMap.count("math")); @@ -2695,8 +2696,6 @@ local r4 = t:bar1(@4) TEST_CASE_FIXTURE(ACFixture, "autocomplete_default_type_parameters") { - ScopedFastFlag luauParseTypeAliasDefaults{"LuauParseTypeAliasDefaults", true}; - check(R"( type A = () -> T )"); @@ -2709,8 +2708,6 @@ type A = () -> T TEST_CASE_FIXTURE(ACFixture, "autocomplete_default_type_pack_parameters") { - ScopedFastFlag luauParseTypeAliasDefaults{"LuauParseTypeAliasDefaults", true}; - check(R"( type A = () -> T )"); @@ -2768,7 +2765,6 @@ TEST_CASE_FIXTURE(ACFixture, "autocomplete_on_string_singletons") TEST_CASE_FIXTURE(ACFixture, "function_in_assignment_has_parentheses_2") { ScopedFastFlag luauAutocompleteAvoidMutation("LuauAutocompleteAvoidMutation", true); - ScopedFastFlag preferToCallFunctionsForIntersects("PreferToCallFunctionsForIntersects", true); check(R"( local bar: ((number) -> number) & (number, number) -> number) diff --git a/tests/Conformance.test.cpp b/tests/Conformance.test.cpp index eb6ca749..63fbb363 100644 --- a/tests/Conformance.test.cpp +++ b/tests/Conformance.test.cpp @@ -241,6 +241,8 @@ TEST_CASE("Math") TEST_CASE("Table") { + ScopedFastFlag sff("LuauTableClone", true); + runConformance("nextvar.lua"); } @@ -465,6 +467,8 @@ static void populateRTTI(lua_State* L, Luau::TypeId type) TEST_CASE("Types") { + ScopedFastFlag sff("LuauTableCloneType", true); + runConformance("types.lua", [](lua_State* L) { Luau::NullModuleResolver moduleResolver; Luau::InternalErrorReporter iceHandler; @@ -959,8 +963,6 @@ TEST_CASE("Coverage") TEST_CASE("StringConversion") { - ScopedFastFlag sff{"LuauSchubfach", true}; - runConformance("strconv.lua"); } diff --git a/tests/Linter.test.cpp b/tests/Linter.test.cpp index ab19cea3..4d6c207c 100644 --- a/tests/Linter.test.cpp +++ b/tests/Linter.test.cpp @@ -157,6 +157,113 @@ return bar() CHECK_EQ(result.warnings[0].text, "Global 'foo' is only used in the enclosing function 'bar'; consider changing it to local"); } +TEST_CASE_FIXTURE(Fixture, "GlobalAsLocalMultiFx") +{ + ScopedFastFlag sff{"LuauLintGlobalNeverReadBeforeWritten", true}; + LintResult result = lint(R"( +function bar() + foo = 6 + return foo +end + +function baz() + foo = 6 + return foo +end + +return bar() + baz() +)"); + + REQUIRE_EQ(result.warnings.size(), 1); + CHECK_EQ(result.warnings[0].text, "Global 'foo' is never read before being written. Consider changing it to local"); +} + +TEST_CASE_FIXTURE(Fixture, "GlobalAsLocalMultiFxWithRead") +{ + ScopedFastFlag sff{"LuauLintGlobalNeverReadBeforeWritten", true}; + LintResult result = lint(R"( +function bar() + foo = 6 + return foo +end + +function baz() + foo = 6 + return foo +end + +function read() + print(foo) +end + +return bar() + baz() + read() +)"); + + CHECK_EQ(result.warnings.size(), 0); +} + +TEST_CASE_FIXTURE(Fixture, "GlobalAsLocalWithConditional") +{ + ScopedFastFlag sff{"LuauLintGlobalNeverReadBeforeWritten", true}; + LintResult result = lint(R"( +function bar() + if true then foo = 6 end + return foo +end + +function baz() + foo = 6 + return foo +end + +return bar() + baz() +)"); + + CHECK_EQ(result.warnings.size(), 0); +} + +TEST_CASE_FIXTURE(Fixture, "GlobalAsLocal3WithConditionalRead") +{ + ScopedFastFlag sff{"LuauLintGlobalNeverReadBeforeWritten", true}; + LintResult result = lint(R"( +function bar() + foo = 6 + return foo +end + +function baz() + foo = 6 + return foo +end + +function read() + if false then print(foo) end +end + +return bar() + baz() + read() +)"); + + CHECK_EQ(result.warnings.size(), 0); +} + +TEST_CASE_FIXTURE(Fixture, "GlobalAsLocalInnerRead") +{ + ScopedFastFlag sff{"LuauLintGlobalNeverReadBeforeWritten", true}; + LintResult result = lint(R"( +function foo() + local f = function() return bar end + f() + bar = 42 +end + +function baz() bar = 0 end + +return foo() + baz() +)"); + + CHECK_EQ(result.warnings.size(), 0); +} + TEST_CASE_FIXTURE(Fixture, "GlobalAsLocalMulti") { LintResult result = lint(R"( diff --git a/tests/Parser.test.cpp b/tests/Parser.test.cpp index 77e49ce3..7f6a6c0d 100644 --- a/tests/Parser.test.cpp +++ b/tests/Parser.test.cpp @@ -1988,8 +1988,6 @@ TEST_CASE_FIXTURE(Fixture, "function_type_matching_parenthesis") TEST_CASE_FIXTURE(Fixture, "parse_type_alias_default_type") { - ScopedFastFlag luauParseTypeAliasDefaults{"LuauParseTypeAliasDefaults", true}; - AstStat* stat = parse(R"( type A = {} type B = {} @@ -2005,8 +2003,6 @@ type G = (U...) -> T... TEST_CASE_FIXTURE(Fixture, "parse_type_alias_default_type_errors") { - ScopedFastFlag luauParseTypeAliasDefaults{"LuauParseTypeAliasDefaults", true}; - matchParseError("type Y = {}", "Expected default type after type name", Location{{0, 20}, {0, 21}}); matchParseError("type Y = {}", "Expected default type pack after type pack name", Location{{0, 29}, {0, 30}}); matchParseError("type Y number> = {}", "Expected type pack after '=', got type", Location{{0, 14}, {0, 32}}); @@ -2574,8 +2570,6 @@ do end TEST_CASE_FIXTURE(Fixture, "recover_expected_type_pack") { - ScopedFastFlag luauParseTypeAliasDefaults{"LuauParseTypeAliasDefaults", true}; - ParseResult result = tryParse(R"( type Y = (T...) -> U... )"); diff --git a/tests/ToDot.test.cpp b/tests/ToDot.test.cpp index 0ca9c994..29bdd866 100644 --- a/tests/ToDot.test.cpp +++ b/tests/ToDot.test.cpp @@ -96,8 +96,6 @@ n2 [label="number"]; TEST_CASE_FIXTURE(Fixture, "function") { - ScopedFastFlag luauQuantifyInPlace2{"LuauQuantifyInPlace2", true}; - CheckResult result = check(R"( local function f(a, ...: string) return a end )"); diff --git a/tests/ToString.test.cpp b/tests/ToString.test.cpp index bbb26291..6713a589 100644 --- a/tests/ToString.test.cpp +++ b/tests/ToString.test.cpp @@ -500,8 +500,6 @@ TEST_CASE_FIXTURE(Fixture, "toStringNamedFunction_map") TEST_CASE_FIXTURE(Fixture, "toStringNamedFunction_generic_pack") { - ScopedFastFlag luauTypeAliasDefaults{"LuauTypeAliasDefaults", true}; - CheckResult result = check(R"( local function f(a: number, b: string) end local function test(...: T...): U... diff --git a/tests/Transpiler.test.cpp b/tests/Transpiler.test.cpp index 332aba9e..5f0295b0 100644 --- a/tests/Transpiler.test.cpp +++ b/tests/Transpiler.test.cpp @@ -641,9 +641,6 @@ TEST_CASE_FIXTURE(Fixture, "transpile_to_string") TEST_CASE_FIXTURE(Fixture, "transpile_type_alias_default_type_parameters") { - ScopedFastFlag luauParseTypeAliasDefaults{"LuauParseTypeAliasDefaults", true}; - ScopedFastFlag luauTypeAliasDefaults{"LuauTypeAliasDefaults", true}; - std::string code = R"( type Packed = (T, U, V...)->(W...) local a: Packed diff --git a/tests/TypeInfer.aliases.test.cpp b/tests/TypeInfer.aliases.test.cpp index 31d7ef10..d584eb2d 100644 --- a/tests/TypeInfer.aliases.test.cpp +++ b/tests/TypeInfer.aliases.test.cpp @@ -625,9 +625,8 @@ TEST_CASE_FIXTURE(Fixture, "forward_declared_alias_is_not_clobbered_by_prior_uni ScopedFastFlag sff[] = { {"LuauTwoPassAliasDefinitionFix", true}, - // We also force these two flags because this surfaced an unfortunate interaction. + // We also force this flag because it surfaced an unfortunate interaction. {"LuauErrorRecoveryType", true}, - {"LuauQuantifyInPlace2", true}, }; CheckResult result = check(R"( diff --git a/tests/TypeInfer.builtins.test.cpp b/tests/TypeInfer.builtins.test.cpp index f3dfb214..bf990770 100644 --- a/tests/TypeInfer.builtins.test.cpp +++ b/tests/TypeInfer.builtins.test.cpp @@ -934,4 +934,31 @@ TEST_CASE_FIXTURE(Fixture, "assert_returns_false_and_string_iff_it_knows_the_fir CHECK_EQ("(nil) -> nil", toString(requireType("f"))); } +TEST_CASE_FIXTURE(Fixture, "table_freeze_is_generic") +{ + CheckResult result = check(R"( + local t1: {a: number} = {a = 42} + local t2: {b: string} = {b = "hello"} + local t3: {boolean} = {false, true} + + local tf1 = table.freeze(t1) + local tf2 = table.freeze(t2) + local tf3 = table.freeze(t3) + + local a = tf1.a + local b = tf2.b + local c = tf3[2] + + local d = tf1.b + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ("Key 'b' not found in table '{| a: number |}'", toString(result.errors[0])); + + CHECK_EQ("number", toString(requireType("a"))); + CHECK_EQ("string", toString(requireType("b"))); + CHECK_EQ("boolean", toString(requireType("c"))); + CHECK_EQ("*unknown*", toString(requireType("d"))); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.generics.test.cpp b/tests/TypeInfer.generics.test.cpp index f8fccf6b..c482847b 100644 --- a/tests/TypeInfer.generics.test.cpp +++ b/tests/TypeInfer.generics.test.cpp @@ -697,4 +697,93 @@ end LUAU_REQUIRE_NO_ERRORS(result); } +TEST_CASE_FIXTURE(Fixture, "generic_functions_should_be_memory_safe") +{ + ScopedFastFlag sffs[] = { + { "LuauTableSubtypingVariance2", true }, + { "LuauUnsealedTableLiteral", true }, + { "LuauPropertiesGetExpectedType", true }, + { "LuauRecursiveTypeParameterRestriction", true }, + }; + + CheckResult result = check(R"( +--!strict +-- At one point this produced a UAF +type T = { a: U, b: a } +type U = { c: T?, d : a } +local x: T = { a = { c = nil, d = 5 }, b = 37 } +x.a.c = x +local y: T = { a = { c = nil, d = 5 }, b = 37 } +y.a.c = y + )"); + + LUAU_REQUIRE_ERRORS(result); + CHECK_EQ(toString(result.errors[0]), + R"(Type 'y' could not be converted into 'T' +caused by: + Property 'a' is not compatible. Type '{ c: T?, d: number }' could not be converted into 'U' +caused by: + Property 'd' is not compatible. Type 'number' could not be converted into 'string')"); +} + +TEST_CASE_FIXTURE(Fixture, "generic_type_pack_unification1") +{ + ScopedFastFlag sff{"LuauTxnLogSeesTypePacks2", true}; + + CheckResult result = check(R"( +--!strict +type Dispatcher = { + useMemo: (create: () -> T...) -> T... +} + +local TheDispatcher: Dispatcher = { + useMemo = function(create: () -> U...): U... + return create() + end +} + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "generic_type_pack_unification2") +{ + ScopedFastFlag sff{"LuauTxnLogSeesTypePacks2", true}; + + CheckResult result = check(R"( +--!strict +type Dispatcher = { + useMemo: (create: () -> T...) -> T... +} + +local TheDispatcher: Dispatcher = { + useMemo = function(create) + return create() + end +} + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "generic_type_pack_unification3") +{ + ScopedFastFlag sff{"LuauTxnLogSeesTypePacks2", true}; + + CheckResult result = check(R"( +--!strict +type Dispatcher = { + useMemo: (arg: S, create: (S) -> T...) -> T... +} + +local TheDispatcher: Dispatcher = { + useMemo = function(arg: T, create: (T) -> U...): U... + return create(arg) + end +} + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.provisional.test.cpp b/tests/TypeInfer.provisional.test.cpp index eee0e0f1..2e16b21e 100644 --- a/tests/TypeInfer.provisional.test.cpp +++ b/tests/TypeInfer.provisional.test.cpp @@ -8,7 +8,6 @@ #include LUAU_FASTFLAG(LuauEqConstraint) -LUAU_FASTFLAG(LuauQuantifyInPlace2) using namespace Luau; @@ -40,16 +39,6 @@ TEST_CASE_FIXTURE(Fixture, "typeguard_inference_incomplete") end )"; - const std::string old_expected = R"( - function f(a:{fn:()->(free,free...)}): () - if type(a) == 'boolean'then - local a1:boolean=a - elseif a.fn()then - local a2:{fn:()->(free,free...)}=a - end - end - )"; - const std::string expected = R"( function f(a:{fn:()->(a,b...)}): () if type(a) == 'boolean'then @@ -60,10 +49,7 @@ TEST_CASE_FIXTURE(Fixture, "typeguard_inference_incomplete") end )"; - if (FFlag::LuauQuantifyInPlace2) - CHECK_EQ(expected, decorateWithTypes(code)); - else - CHECK_EQ(old_expected, decorateWithTypes(code)); + CHECK_EQ(expected, decorateWithTypes(code)); } TEST_CASE_FIXTURE(Fixture, "xpcall_returns_what_f_returns") @@ -135,46 +121,6 @@ TEST_CASE_FIXTURE(Fixture, "setmetatable_constrains_free_type_into_free_table") CHECK_EQ("number", toString(tm->givenType)); } -TEST_CASE_FIXTURE(Fixture, "pass_a_union_of_tables_to_a_function_that_requires_a_table") -{ - CheckResult result = check(R"( - local a: {x: number, y: number, [any]: any} | {y: number} - - function f(t) - t.y = 1 - return t - end - - local b = f(a) - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - - // :( - // Should be the same as the type of a - REQUIRE_EQ("{| y: number |}", toString(requireType("b"))); -} - -TEST_CASE_FIXTURE(Fixture, "pass_a_union_of_tables_to_a_function_that_requires_a_table_2") -{ - CheckResult result = check(R"( - local a: {y: number} | {x: number, y: number, [any]: any} - - function f(t) - t.y = 1 - return t - end - - local b = f(a) - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - - // :( - // Should be the same as the type of a - REQUIRE_EQ("{| [any]: any, x: number, y: number |}", toString(requireType("b"))); -} - // Luau currently doesn't yet know how to allow assignments when the binding was refined. TEST_CASE_FIXTURE(Fixture, "while_body_are_also_refined") { @@ -557,25 +503,6 @@ TEST_CASE_FIXTURE(Fixture, "bail_early_on_typescript_port_of_Result_type" * doct } } -TEST_CASE_FIXTURE(Fixture, "table_subtyping_shouldn't_add_optional_properties_to_sealed_tables") -{ - CheckResult result = check(R"( - --!strict - local function setNumber(t: { p: number? }, x:number) t.p = x end - local function getString(t: { p: string? }):string return t.p or "" end - -- This shouldn't type-check! - local function oh(x:number): string - local t: {} = {} - setNumber(t, x) - return getString(t) - end - local s: string = oh(37) - )"); - - // Really this should return an error, but it doesn't - LUAU_REQUIRE_NO_ERRORS(result); -} - // Should be in TypeInfer.tables.test.cpp // It's unsound to instantiate tables containing generic methods, // since mutating properties means table properties should be invariant. @@ -600,25 +527,9 @@ TEST_CASE_FIXTURE(Fixture, "invariant_table_properties_means_instantiating_table LUAU_REQUIRE_NO_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "self_recursive_instantiated_param") -{ - // Mutability in type function application right now can create strange recursive types - // TODO: instantiation right now is problematic, in this example should either leave the Table type alone - // or it should rename the type to 'Self' so that the result will be 'Self
' - CheckResult result = check(R"( -type Table = { a: number } -type Self = T -local a: Self
- )"); - - LUAU_REQUIRE_NO_ERRORS(result); - CHECK_EQ(toString(requireType("a")), "Table
"); -} - TEST_CASE_FIXTURE(Fixture, "do_not_ice_when_trying_to_pick_first_of_generic_type_pack") { ScopedFastFlag sff[]{ - {"LuauQuantifyInPlace2", true}, {"LuauReturnAnyInsteadOfICE", true}, }; @@ -664,8 +575,6 @@ TEST_CASE_FIXTURE(Fixture, "specialization_binds_with_prototypes_too_early") TEST_CASE_FIXTURE(Fixture, "weird_fail_to_unify_type_pack") { - ScopedFastFlag sff{"LuauQuantifyInPlace2", true}; - CheckResult result = check(R"( local function f() return end local g = function() return f() end @@ -676,8 +585,6 @@ TEST_CASE_FIXTURE(Fixture, "weird_fail_to_unify_type_pack") TEST_CASE_FIXTURE(Fixture, "weird_fail_to_unify_variadic_pack") { - ScopedFastFlag sff{"LuauQuantifyInPlace2", true}; - CheckResult result = check(R"( --!strict local function f(...) return ... end diff --git a/tests/TypeInfer.refinements.test.cpp b/tests/TypeInfer.refinements.test.cpp index bff8926c..a5147d56 100644 --- a/tests/TypeInfer.refinements.test.cpp +++ b/tests/TypeInfer.refinements.test.cpp @@ -8,7 +8,6 @@ LUAU_FASTFLAG(LuauDiscriminableUnions2) LUAU_FASTFLAG(LuauWeakEqConstraint) -LUAU_FASTFLAG(LuauQuantifyInPlace2) using namespace Luau; @@ -1179,20 +1178,14 @@ TEST_CASE_FIXTURE(RefinementClassFixture, "typeguard_cast_free_table_to_vector") { LUAU_REQUIRE_ERROR_COUNT(1, result); - if (FFlag::LuauQuantifyInPlace2) - CHECK_EQ("Type '{+ X: a, Y: b, Z: c +}' could not be converted into 'Instance'", toString(result.errors[0])); - else - CHECK_EQ("Type '{- X: a, Y: b, Z: c -}' could not be converted into 'Instance'", toString(result.errors[0])); + CHECK_EQ("Type '{+ X: a, Y: b, Z: c +}' could not be converted into 'Instance'", toString(result.errors[0])); } CHECK_EQ("Vector3", toString(requireTypeAtPosition({5, 28}))); // type(vec) == "vector" CHECK_EQ("*unknown*", toString(requireTypeAtPosition({7, 28}))); // typeof(vec) == "Instance" - if (FFlag::LuauQuantifyInPlace2) - CHECK_EQ("{+ X: a, Y: b, Z: c +}", toString(requireTypeAtPosition({9, 28}))); // type(vec) ~= "vector" and typeof(vec) ~= "Instance" - else - CHECK_EQ("{- X: a, Y: b, Z: c -}", toString(requireTypeAtPosition({9, 28}))); // type(vec) ~= "vector" and typeof(vec) ~= "Instance" + CHECK_EQ("{+ X: a, Y: b, Z: c +}", toString(requireTypeAtPosition({9, 28}))); // type(vec) ~= "vector" and typeof(vec) ~= "Instance" } TEST_CASE_FIXTURE(RefinementClassFixture, "typeguard_cast_instance_or_vector3_to_vector") diff --git a/tests/TypeInfer.singletons.test.cpp b/tests/TypeInfer.singletons.test.cpp index 856549bd..3ed536ea 100644 --- a/tests/TypeInfer.singletons.test.cpp +++ b/tests/TypeInfer.singletons.test.cpp @@ -465,30 +465,32 @@ TEST_CASE_FIXTURE(Fixture, "widen_the_supertype_if_it_is_free_and_subtype_has_si CHECK_EQ("((string) -> (b...), a) -> ()", toString(requireType("foo"))); } -// TEST_CASE_FIXTURE(Fixture, "return_type_of_f_is_not_widened") -// { -// ScopedFastFlag sff[]{ -// {"LuauParseSingletonTypes", true}, -// {"LuauSingletonTypes", true}, -// {"LuauDiscriminableUnions2", true}, -// {"LuauEqConstraint", true}, -// {"LuauWidenIfSupertypeIsFree", true}, -// {"LuauWeakEqConstraint", false}, -// }; +TEST_CASE_FIXTURE(Fixture, "return_type_of_f_is_not_widened") +{ + ScopedFastFlag sff[]{ + {"LuauParseSingletonTypes", true}, + {"LuauSingletonTypes", true}, + {"LuauDiscriminableUnions2", true}, + {"LuauEqConstraint", true}, + {"LuauWidenIfSupertypeIsFree", true}, + {"LuauWeakEqConstraint", false}, + {"LuauDoNotAccidentallyDependOnPointerOrdering", true} + }; -// CheckResult result = check(R"( -// local function foo(f, x): "hello"? -- anyone there? -// return if x == "hi" -// then f(x) -// else nil -// end -// )"); + CheckResult result = check(R"( + local function foo(f, x): "hello"? -- anyone there? + return if x == "hi" + then f(x) + else nil + end + )"); -// LUAU_REQUIRE_NO_ERRORS(result); + LUAU_REQUIRE_NO_ERRORS(result); -// CHECK_EQ(R"("hi")", toString(requireTypeAtPosition({3, 23}))); -// CHECK_EQ(R"(((string) -> ("hello"?, b...), a) -> "hello"?)", toString(requireType("foo"))); -// } + CHECK_EQ(R"("hi")", toString(requireTypeAtPosition({3, 23}))); + CHECK_EQ(R"(((string) -> (a, c...), b) -> "hello"?)", toString(requireType("foo"))); + // CHECK_EQ(R"(((string) -> ("hello"?, b...), a) -> "hello"?)", toString(requireType("foo"))); +} TEST_CASE_FIXTURE(Fixture, "widening_happens_almost_everywhere") { diff --git a/tests/TypeInfer.tables.test.cpp b/tests/TypeInfer.tables.test.cpp index aa949789..da035ba1 100644 --- a/tests/TypeInfer.tables.test.cpp +++ b/tests/TypeInfer.tables.test.cpp @@ -1219,13 +1219,12 @@ TEST_CASE_FIXTURE(Fixture, "passing_compatible_unions_to_a_generic_table_without { CheckResult result = check(R"( type A = {x: number, y: number, [any]: any} | {y: number} - local a: A function f(t) t.y = 1 end - f(a) + f({y = 5} :: A) )"); LUAU_REQUIRE_NO_ERRORS(result); @@ -2165,6 +2164,44 @@ b() CHECK_EQ(toString(result.errors[0]), R"(Cannot call non-function t1 where t1 = { @metatable { __call: t1 }, { } })"); } +TEST_CASE_FIXTURE(Fixture, "table_subtyping_shouldn't_add_optional_properties_to_sealed_tables") +{ + ScopedFastFlag sffs[] = { + {"LuauTableSubtypingVariance2", true}, + {"LuauSubtypingAddOptPropsToUnsealedTables", true}, + }; + + CheckResult result = check(R"( + --!strict + local function setNumber(t: { p: number? }, x:number) t.p = x end + local function getString(t: { p: string? }):string return t.p or "" end + -- This shouldn't type-check! + local function oh(x:number): string + local t: {} = {} + setNumber(t, x) + return getString(t) + end + local s: string = oh(37) + )"); + + LUAU_REQUIRE_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "top_table_type") +{ + CheckResult result = check(R"( + --!strict + type Table = { [any] : any } + type HasTable = { p: Table? } + type HasHasTable = { p: HasTable? } + local t : Table = { p = 5 } + local u : HasTable = { p = { p = 5 } } + local v : HasHasTable = { p = { p = { p = 5 } } } + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + TEST_CASE_FIXTURE(Fixture, "length_operator_union") { CheckResult result = check(R"( @@ -2257,4 +2294,44 @@ TEST_CASE_FIXTURE(Fixture, "confusing_indexing") CHECK_EQ("number | string", toString(requireType("foo"))); } +TEST_CASE_FIXTURE(Fixture, "pass_a_union_of_tables_to_a_function_that_requires_a_table") +{ + ScopedFastFlag sff{"LuauDifferentOrderOfUnificationDoesntMatter", true}; + + CheckResult result = check(R"( + local a: {x: number, y: number, [any]: any} | {y: number} + + function f(t) + t.y = 1 + return t + end + + local b = f(a) + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + REQUIRE_EQ("{| [any]: any, x: number, y: number |} | {| y: number |}", toString(requireType("b"))); +} + +TEST_CASE_FIXTURE(Fixture, "pass_a_union_of_tables_to_a_function_that_requires_a_table_2") +{ + ScopedFastFlag sff{"LuauDifferentOrderOfUnificationDoesntMatter", true}; + + CheckResult result = check(R"( + local a: {y: number} | {x: number, y: number, [any]: any} + + function f(t) + t.y = 1 + return t + end + + local b = f(a) + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + REQUIRE_EQ("{| [any]: any, x: number, y: number |} | {| y: number |}", toString(requireType("b"))); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.test.cpp b/tests/TypeInfer.test.cpp index f44d9fd8..f63579b5 100644 --- a/tests/TypeInfer.test.cpp +++ b/tests/TypeInfer.test.cpp @@ -4044,6 +4044,49 @@ type t0 = any CHECK(ttv->instantiatedTypeParams.empty()); } +TEST_CASE_FIXTURE(Fixture, "instantiate_table_cloning_2") +{ + ScopedFastFlag sff{"LuauOnlyMutateInstantiatedTables", true}; + + CheckResult result = check(R"( +type X = T +type K = X +)"); + + LUAU_REQUIRE_NO_ERRORS(result); + + std::optional ty = requireType("math"); + REQUIRE(ty); + + const TableTypeVar* ttv = get(*ty); + REQUIRE(ttv); + CHECK(ttv->instantiatedTypeParams.empty()); +} + +TEST_CASE_FIXTURE(Fixture, "instantiate_table_cloning_3") +{ + ScopedFastFlag sff{"LuauOnlyMutateInstantiatedTables", true}; + + CheckResult result = check(R"( +type X = T +local a = {} +a.x = 4 +local b: X +a.y = 5 +local c: X +c = b +)"); + + LUAU_REQUIRE_NO_ERRORS(result); + + std::optional ty = requireType("a"); + REQUIRE(ty); + + const TableTypeVar* ttv = get(*ty); + REQUIRE(ttv); + CHECK(ttv->instantiatedTypeParams.empty()); +} + TEST_CASE_FIXTURE(Fixture, "bound_free_table_export_is_ok") { CheckResult result = check(R"( @@ -4065,6 +4108,21 @@ return m LUAU_REQUIRE_NO_ERRORS(result); } +TEST_CASE_FIXTURE(Fixture, "self_recursive_instantiated_param") +{ + ScopedFastFlag sff{"LuauOnlyMutateInstantiatedTables", true}; + + // Mutability in type function application right now can create strange recursive types + CheckResult result = check(R"( +type Table = { a: number } +type Self = T +local a: Self
+ )"); + + LUAU_REQUIRE_NO_ERRORS(result); + CHECK_EQ(toString(requireType("a")), "Table"); +} + TEST_CASE_FIXTURE(Fixture, "no_persistent_typelevel_change") { TypeId mathTy = requireType(typeChecker.globalScope, "math"); @@ -5284,4 +5342,17 @@ TEST_CASE_FIXTURE(Fixture, "inferred_properties_of_a_table_should_start_with_the LUAU_REQUIRE_NO_ERRORS(result); } +TEST_CASE_FIXTURE(Fixture, "global_singleton_types_are_sealed") +{ + CheckResult result = check(R"( +local function f(x: string) + local p = x:split('a') + p = table.pack(table.unpack(p, 1, #p - 1)) + return p +end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.tryUnify.test.cpp b/tests/TypeInfer.tryUnify.test.cpp index c9bf5103..f6ee3ccc 100644 --- a/tests/TypeInfer.tryUnify.test.cpp +++ b/tests/TypeInfer.tryUnify.test.cpp @@ -7,8 +7,6 @@ #include "doctest.h" -LUAU_FASTFLAG(LuauQuantifyInPlace2); - using namespace Luau; LUAU_FASTFLAG(LuauUseCommittingTxnLog) @@ -167,10 +165,7 @@ TEST_CASE_FIXTURE(TryUnifyFixture, "typepack_unification_should_trim_free_tails" )"); LUAU_REQUIRE_ERROR_COUNT(1, result); - if (FFlag::LuauQuantifyInPlace2) - CHECK_EQ("(number) -> boolean", toString(requireType("f"))); - else - CHECK_EQ("(number) -> (boolean)", toString(requireType("f"))); + CHECK_EQ("(number) -> boolean", toString(requireType("f"))); } TEST_CASE_FIXTURE(TryUnifyFixture, "variadic_type_pack_unification") diff --git a/tests/TypeInfer.typePacks.cpp b/tests/TypeInfer.typePacks.cpp index cbe2e48f..6b96f449 100644 --- a/tests/TypeInfer.typePacks.cpp +++ b/tests/TypeInfer.typePacks.cpp @@ -622,9 +622,6 @@ type Other = Packed TEST_CASE_FIXTURE(Fixture, "type_alias_default_type_explicit") { - ScopedFastFlag luauParseTypeAliasDefaults{"LuauParseTypeAliasDefaults", true}; - ScopedFastFlag luauTypeAliasDefaults{"LuauTypeAliasDefaults", true}; - CheckResult result = check(R"( type Y = { a: T, b: U } @@ -654,9 +651,6 @@ local c: Y = { a = "s" } TEST_CASE_FIXTURE(Fixture, "type_alias_default_type_self") { - ScopedFastFlag luauParseTypeAliasDefaults{"LuauParseTypeAliasDefaults", true}; - ScopedFastFlag luauTypeAliasDefaults{"LuauTypeAliasDefaults", true}; - CheckResult result = check(R"( type Y = { a: T, b: U } @@ -682,9 +676,6 @@ local a: Y TEST_CASE_FIXTURE(Fixture, "type_alias_default_type_chained") { - ScopedFastFlag luauParseTypeAliasDefaults{"LuauParseTypeAliasDefaults", true}; - ScopedFastFlag luauTypeAliasDefaults{"LuauTypeAliasDefaults", true}; - CheckResult result = check(R"( type Y = { a: T, b: U, c: V } @@ -700,9 +691,6 @@ local b: Y TEST_CASE_FIXTURE(Fixture, "type_alias_default_type_pack_explicit") { - ScopedFastFlag luauParseTypeAliasDefaults{"LuauParseTypeAliasDefaults", true}; - ScopedFastFlag luauTypeAliasDefaults{"LuauTypeAliasDefaults", true}; - CheckResult result = check(R"( type Y = { a: (T...) -> () } local a: Y<> @@ -715,9 +703,6 @@ local a: Y<> TEST_CASE_FIXTURE(Fixture, "type_alias_default_type_pack_self_ty") { - ScopedFastFlag luauParseTypeAliasDefaults{"LuauParseTypeAliasDefaults", true}; - ScopedFastFlag luauTypeAliasDefaults{"LuauTypeAliasDefaults", true}; - CheckResult result = check(R"( type Y = { a: T, b: (U...) -> T } @@ -731,9 +716,6 @@ local a: Y TEST_CASE_FIXTURE(Fixture, "type_alias_default_type_pack_self_tp") { - ScopedFastFlag luauParseTypeAliasDefaults{"LuauParseTypeAliasDefaults", true}; - ScopedFastFlag luauTypeAliasDefaults{"LuauTypeAliasDefaults", true}; - CheckResult result = check(R"( type Y = { a: (T...) -> U... } local a: Y @@ -746,9 +728,6 @@ local a: Y TEST_CASE_FIXTURE(Fixture, "type_alias_default_type_pack_self_chained_tp") { - ScopedFastFlag luauParseTypeAliasDefaults{"LuauParseTypeAliasDefaults", true}; - ScopedFastFlag luauTypeAliasDefaults{"LuauTypeAliasDefaults", true}; - CheckResult result = check(R"( type Y = { a: (T...) -> U..., b: (T...) -> V... } local a: Y @@ -761,9 +740,6 @@ local a: Y TEST_CASE_FIXTURE(Fixture, "type_alias_default_mixed_self") { - ScopedFastFlag luauParseTypeAliasDefaults{"LuauParseTypeAliasDefaults", true}; - ScopedFastFlag luauTypeAliasDefaults{"LuauTypeAliasDefaults", true}; - CheckResult result = check(R"( type Y = { a: (T, U, V...) -> W... } local a: Y @@ -782,9 +758,6 @@ local d: Y ()> TEST_CASE_FIXTURE(Fixture, "type_alias_default_type_errors") { - ScopedFastFlag luauParseTypeAliasDefaults{"LuauParseTypeAliasDefaults", true}; - ScopedFastFlag luauTypeAliasDefaults{"LuauTypeAliasDefaults", true}; - CheckResult result = check(R"( type Y = { a: T } local a: Y = { a = 2 } @@ -834,9 +807,6 @@ local a: Y<...number> TEST_CASE_FIXTURE(Fixture, "type_alias_default_export") { - ScopedFastFlag luauParseTypeAliasDefaults{"LuauParseTypeAliasDefaults", true}; - ScopedFastFlag luauTypeAliasDefaults{"LuauTypeAliasDefaults", true}; - fileResolver.source["Module/Types"] = R"( export type A = { a: T, b: U } export type B = { a: T, b: U } @@ -882,9 +852,6 @@ local h: Types.H<> TEST_CASE_FIXTURE(Fixture, "type_alias_default_type_skip_brackets") { - ScopedFastFlag luauParseTypeAliasDefaults{"LuauParseTypeAliasDefaults", true}; - ScopedFastFlag luauTypeAliasDefaults{"LuauTypeAliasDefaults", true}; - CheckResult result = check(R"( type Y = (T...) -> number local a: Y @@ -897,9 +864,6 @@ local a: Y TEST_CASE_FIXTURE(Fixture, "type_alias_defaults_confusing_types") { - ScopedFastFlag luauParseTypeAliasDefaults{"LuauParseTypeAliasDefaults", true}; - ScopedFastFlag luauTypeAliasDefaults{"LuauTypeAliasDefaults", true}; - CheckResult result = check(R"( type A = (T, V...) -> (U, W...) type B = A @@ -914,9 +878,6 @@ type C = A TEST_CASE_FIXTURE(Fixture, "type_alias_defaults_recursive_type") { - ScopedFastFlag luauParseTypeAliasDefaults{"LuauParseTypeAliasDefaults", true}; - ScopedFastFlag luauTypeAliasDefaults{"LuauTypeAliasDefaults", true}; - CheckResult result = check(R"( type F ()> = (K) -> V type R = { m: F } diff --git a/tests/TypeInfer.unionTypes.test.cpp b/tests/TypeInfer.unionTypes.test.cpp index 3b53ddfe..0e0b6ebb 100644 --- a/tests/TypeInfer.unionTypes.test.cpp +++ b/tests/TypeInfer.unionTypes.test.cpp @@ -400,10 +400,10 @@ local e = a.z CHECK_EQ("Type 'A | B | C | D' does not have key 'z'", toString(result.errors[3])); } -TEST_CASE_FIXTURE(Fixture, "unify_sealed_table_union_check") +TEST_CASE_FIXTURE(Fixture, "unify_unsealed_table_union_check") { CheckResult result = check(R"( -local x: { x: number } = { x = 3 } +local x = { x = 3 } type A = number? type B = string? local y: { x: number, y: A | B } @@ -413,7 +413,7 @@ y = x LUAU_REQUIRE_NO_ERRORS(result); result = check(R"( -local x: { x: number } = { x = 3 } +local x = { x = 3 } local a: number? = 2 local y = {} @@ -426,6 +426,31 @@ y = x LUAU_REQUIRE_NO_ERRORS(result); } +TEST_CASE_FIXTURE(Fixture, "unify_sealed_table_union_check") +{ + ScopedFastFlag sffs[] = { + {"LuauTableSubtypingVariance2", true}, + {"LuauUnsealedTableLiteral", true}, + {"LuauSubtypingAddOptPropsToUnsealedTables", true}, + }; + + CheckResult result = check(R"( + -- the difference between this and unify_unsealed_table_union_check is the type annotation on x +local t = { x = 3, y = true } +local x: { x: number } = t +type A = number? +type B = string? +local y: { x: number, y: A | B } +-- Shouldn't typecheck! +y = x +-- If it does, we can convert any type to any other type +y.y = 5 +local oh : boolean = t.y + )"); + + LUAU_REQUIRE_ERRORS(result); +} + TEST_CASE_FIXTURE(Fixture, "error_detailed_union_part") { CheckResult result = check(R"( diff --git a/tests/conformance/nextvar.lua b/tests/conformance/nextvar.lua index 94ba5ccf..e85fcbe8 100644 --- a/tests/conformance/nextvar.lua +++ b/tests/conformance/nextvar.lua @@ -512,4 +512,42 @@ do assert(#t == 7) end +-- test clone +do + local t = {a = 1, b = 2, 3, 4, 5} + local tt = table.clone(t) + + assert(#tt == 3) + assert(tt.a == 1 and tt.b == 2) + + t.c = 3 + assert(tt.c == nil) + + t = table.freeze({"test"}) + tt = table.clone(t) + assert(table.isfrozen(t) and not table.isfrozen(tt)) + + t = setmetatable({}, {}) + tt = table.clone(t) + assert(getmetatable(t) == getmetatable(tt)) + + t = setmetatable({}, {__metatable = "protected"}) + assert(not pcall(table.clone, t)) + + function order(t) + local r = '' + for k,v in pairs(t) do + r ..= tostring(v) + end + return v + end + + t = {a = 1, b = 2, c = 3, d = 4, e = 5, f = 6} + tt = table.clone(t) + assert(order(t) == order(tt)) + + assert(not pcall(table.clone)) + assert(not pcall(table.clone, 42)) +end + return"OK" From feea507be3f6991a76907e92220f50db05d7a98e Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Fri, 11 Mar 2022 08:31:18 -0800 Subject: [PATCH 191/399] Sync to upstream/release/518 --- Analysis/include/Luau/TxnLog.h | 55 - Analysis/include/Luau/TypePack.h | 1 - Analysis/include/Luau/Unifier.h | 1 - Analysis/src/Autocomplete.cpp | 56 +- Analysis/src/BuiltinDefinitions.cpp | 17 +- Analysis/src/TxnLog.cpp | 117 -- Analysis/src/TypeInfer.cpp | 672 ++++------ Analysis/src/TypePack.cpp | 26 +- Analysis/src/Unifier.cpp | 1826 +++++++-------------------- VM/include/lua.h | 35 +- VM/include/luaconf.h | 27 - VM/include/lualib.h | 4 +- VM/src/laux.cpp | 15 + VM/src/lbaselib.cpp | 11 +- VM/src/lgc.h | 7 + tests/Autocomplete.test.cpp | 52 +- tests/TypeInfer.aliases.test.cpp | 23 + tests/TypeInfer.builtins.test.cpp | 51 +- tests/TypeInfer.generics.test.cpp | 45 + tests/TypeInfer.tables.test.cpp | 50 + tests/TypeInfer.test.cpp | 38 +- tests/TypeInfer.tryUnify.test.cpp | 44 +- tests/TypeInfer.typePacks.cpp | 13 +- tests/TypeInfer.unionTypes.test.cpp | 10 +- tools/LuauVisualize.py | 107 ++ tools/lldb-formatters.lldb | 2 + 26 files changed, 1137 insertions(+), 2168 deletions(-) create mode 100644 tools/LuauVisualize.py create mode 100644 tools/lldb-formatters.lldb diff --git a/Analysis/include/Luau/TxnLog.h b/Analysis/include/Luau/TxnLog.h index f8105383..c8ebaaeb 100644 --- a/Analysis/include/Luau/TxnLog.h +++ b/Analysis/include/Luau/TxnLog.h @@ -14,61 +14,6 @@ namespace Luau using TypeOrPackId = const void*; -// Log of where what TypeIds we are rebinding and what they used to be -// Remove with LuauUseCommitTxnLog -struct DEPRECATED_TxnLog -{ - DEPRECATED_TxnLog() - : originalSeenSize(0) - , ownedSeen() - , sharedSeen(&ownedSeen) - { - } - - explicit DEPRECATED_TxnLog(std::vector>* sharedSeen) - : originalSeenSize(sharedSeen->size()) - , ownedSeen() - , sharedSeen(sharedSeen) - { - } - - DEPRECATED_TxnLog(const DEPRECATED_TxnLog&) = delete; - DEPRECATED_TxnLog& operator=(const DEPRECATED_TxnLog&) = delete; - - DEPRECATED_TxnLog(DEPRECATED_TxnLog&&) = default; - DEPRECATED_TxnLog& operator=(DEPRECATED_TxnLog&&) = default; - - void operator()(TypeId a); - void operator()(TypePackId a); - void operator()(TableTypeVar* a); - - void rollback(); - - void concat(DEPRECATED_TxnLog rhs); - - bool haveSeen(TypeId lhs, TypeId rhs); - void pushSeen(TypeId lhs, TypeId rhs); - void popSeen(TypeId lhs, TypeId rhs); - - bool haveSeen(TypePackId lhs, TypePackId rhs); - void pushSeen(TypePackId lhs, TypePackId rhs); - void popSeen(TypePackId lhs, TypePackId rhs); - -private: - std::vector> typeVarChanges; - std::vector> typePackChanges; - std::vector>> tableChanges; - size_t originalSeenSize; - - bool haveSeen(TypeOrPackId lhs, TypeOrPackId rhs); - void pushSeen(TypeOrPackId lhs, TypeOrPackId rhs); - void popSeen(TypeOrPackId lhs, TypeOrPackId rhs); - -public: - std::vector> ownedSeen; // used to avoid infinite recursion when types are cyclic - std::vector>* sharedSeen; // shared with all the descendent logs -}; - // Pending state for a TypeVar. Generated by a TxnLog and committed via // TxnLog::commit. struct PendingType diff --git a/Analysis/include/Luau/TypePack.h b/Analysis/include/Luau/TypePack.h index c74bad11..946be356 100644 --- a/Analysis/include/Luau/TypePack.h +++ b/Analysis/include/Luau/TypePack.h @@ -105,7 +105,6 @@ private: const TypePack* tp = nullptr; size_t currentIndex = 0; - // Only used if LuauUseCommittingTxnLog is true. const TxnLog* log; }; diff --git a/Analysis/include/Luau/Unifier.h b/Analysis/include/Luau/Unifier.h index 4c0462fe..71958f4a 100644 --- a/Analysis/include/Luau/Unifier.h +++ b/Analysis/include/Luau/Unifier.h @@ -45,7 +45,6 @@ struct Unifier TypeArena* const types; Mode mode; - DEPRECATED_TxnLog DEPRECATED_log; TxnLog log; ErrorVec errors; Location location; diff --git a/Analysis/src/Autocomplete.cpp b/Analysis/src/Autocomplete.cpp index c3de8d0e..e94c432f 100644 --- a/Analysis/src/Autocomplete.cpp +++ b/Analysis/src/Autocomplete.cpp @@ -13,9 +13,6 @@ #include #include -LUAU_FASTFLAG(LuauUseCommittingTxnLog) -LUAU_FASTFLAGVARIABLE(LuauAutocompleteAvoidMutation, false); -LUAU_FASTFLAGVARIABLE(LuauMissingFollowACMetatables, false); LUAU_FASTFLAGVARIABLE(LuauIfElseExprFixCompletionIssue, false); static const std::unordered_set kStatementStartingKeywords = { @@ -240,28 +237,9 @@ static TypeCorrectKind checkTypeCorrectKind(const Module& module, TypeArena* typ UnifierSharedState unifierState(&iceReporter); Unifier unifier(typeArena, Mode::Strict, Location(), Variance::Covariant, unifierState); - if (FFlag::LuauAutocompleteAvoidMutation && !FFlag::LuauUseCommittingTxnLog) - { - SeenTypes seenTypes; - SeenTypePacks seenTypePacks; - CloneState cloneState; - superTy = clone(superTy, *typeArena, seenTypes, seenTypePacks, cloneState); - subTy = clone(subTy, *typeArena, seenTypes, seenTypePacks, cloneState); - - auto errors = unifier.canUnify(subTy, superTy); - return errors.empty(); - } - else - { - unifier.tryUnify(subTy, superTy); - - bool ok = unifier.errors.empty(); - - if (!FFlag::LuauUseCommittingTxnLog) - unifier.DEPRECATED_log.rollback(); - - return ok; - } + unifier.tryUnify(subTy, superTy); + bool ok = unifier.errors.empty(); + return ok; }; auto typeAtPosition = findExpectedTypeAt(module, node, position); @@ -403,28 +381,14 @@ static void autocompleteProps(const Module& module, TypeArena* typeArena, TypeId auto indexIt = mtable->props.find("__index"); if (indexIt != mtable->props.end()) { - if (FFlag::LuauMissingFollowACMetatables) + TypeId followed = follow(indexIt->second.type); + if (get(followed) || get(followed)) + autocompleteProps(module, typeArena, followed, indexType, nodes, result, seen); + else if (auto indexFunction = get(followed)) { - TypeId followed = follow(indexIt->second.type); - if (get(followed) || get(followed)) - autocompleteProps(module, typeArena, followed, indexType, nodes, result, seen); - else if (auto indexFunction = get(followed)) - { - std::optional indexFunctionResult = first(indexFunction->retType); - if (indexFunctionResult) - autocompleteProps(module, typeArena, *indexFunctionResult, indexType, nodes, result, seen); - } - } - else - { - if (get(indexIt->second.type) || get(indexIt->second.type)) - autocompleteProps(module, typeArena, indexIt->second.type, indexType, nodes, result, seen); - else if (auto indexFunction = get(indexIt->second.type)) - { - std::optional indexFunctionResult = first(indexFunction->retType); - if (indexFunctionResult) - autocompleteProps(module, typeArena, *indexFunctionResult, indexType, nodes, result, seen); - } + std::optional indexFunctionResult = first(indexFunction->retType); + if (indexFunctionResult) + autocompleteProps(module, typeArena, *indexFunctionResult, indexType, nodes, result, seen); } } } diff --git a/Analysis/src/BuiltinDefinitions.cpp b/Analysis/src/BuiltinDefinitions.cpp index e4e5dab8..bf9ef303 100644 --- a/Analysis/src/BuiltinDefinitions.cpp +++ b/Analysis/src/BuiltinDefinitions.cpp @@ -10,6 +10,7 @@ LUAU_FASTFLAG(LuauAssertStripsFalsyTypes) LUAU_FASTFLAGVARIABLE(LuauTableCloneType, false) +LUAU_FASTFLAGVARIABLE(LuauSetMetaTableArgsCheck, false) /** FIXME: Many of these type definitions are not quite completely accurate. * @@ -376,11 +377,19 @@ static std::optional> magicFunctionSetMetaTable( TypeId mtTy = arena.addType(mtv); - AstExpr* targetExpr = expr.args.data[0]; - if (AstExprLocal* targetLocal = targetExpr->as()) + if (FFlag::LuauSetMetaTableArgsCheck && expr.args.size < 1) { - const Name targetName(targetLocal->local->name.value); - scope->bindings[targetLocal->local] = Binding{mtTy, expr.location}; + return ExprResult{}; + } + + if (!FFlag::LuauSetMetaTableArgsCheck || !expr.self) + { + AstExpr* targetExpr = expr.args.data[0]; + if (AstExprLocal* targetLocal = targetExpr->as()) + { + const Name targetName(targetLocal->local->name.value); + scope->bindings[targetLocal->local] = Binding{mtTy, expr.location}; + } } return ExprResult{arena.addTypePack({mtTy})}; diff --git a/Analysis/src/TxnLog.cpp b/Analysis/src/TxnLog.cpp index c7bf1e62..876f5f05 100644 --- a/Analysis/src/TxnLog.cpp +++ b/Analysis/src/TxnLog.cpp @@ -7,110 +7,9 @@ #include #include -LUAU_FASTFLAGVARIABLE(LuauUseCommittingTxnLog, false) - namespace Luau { -void DEPRECATED_TxnLog::operator()(TypeId a) -{ - LUAU_ASSERT(!FFlag::LuauUseCommittingTxnLog); - typeVarChanges.emplace_back(a, *a); -} - -void DEPRECATED_TxnLog::operator()(TypePackId a) -{ - LUAU_ASSERT(!FFlag::LuauUseCommittingTxnLog); - typePackChanges.emplace_back(a, *a); -} - -void DEPRECATED_TxnLog::operator()(TableTypeVar* a) -{ - LUAU_ASSERT(!FFlag::LuauUseCommittingTxnLog); - tableChanges.emplace_back(a, a->boundTo); -} - -void DEPRECATED_TxnLog::rollback() -{ - LUAU_ASSERT(!FFlag::LuauUseCommittingTxnLog); - for (auto it = typeVarChanges.rbegin(); it != typeVarChanges.rend(); ++it) - std::swap(*asMutable(it->first), it->second); - - for (auto it = typePackChanges.rbegin(); it != typePackChanges.rend(); ++it) - std::swap(*asMutable(it->first), it->second); - - for (auto it = tableChanges.rbegin(); it != tableChanges.rend(); ++it) - std::swap(it->first->boundTo, it->second); - - LUAU_ASSERT(originalSeenSize <= sharedSeen->size()); - sharedSeen->resize(originalSeenSize); -} - -void DEPRECATED_TxnLog::concat(DEPRECATED_TxnLog rhs) -{ - LUAU_ASSERT(!FFlag::LuauUseCommittingTxnLog); - typeVarChanges.insert(typeVarChanges.end(), rhs.typeVarChanges.begin(), rhs.typeVarChanges.end()); - rhs.typeVarChanges.clear(); - - typePackChanges.insert(typePackChanges.end(), rhs.typePackChanges.begin(), rhs.typePackChanges.end()); - rhs.typePackChanges.clear(); - - tableChanges.insert(tableChanges.end(), rhs.tableChanges.begin(), rhs.tableChanges.end()); - rhs.tableChanges.clear(); -} - -bool DEPRECATED_TxnLog::haveSeen(TypeId lhs, TypeId rhs) -{ - return haveSeen((TypeOrPackId)lhs, (TypeOrPackId)rhs); -} - -void DEPRECATED_TxnLog::pushSeen(TypeId lhs, TypeId rhs) -{ - pushSeen((TypeOrPackId)lhs, (TypeOrPackId)rhs); -} - -void DEPRECATED_TxnLog::popSeen(TypeId lhs, TypeId rhs) -{ - popSeen((TypeOrPackId)lhs, (TypeOrPackId)rhs); -} - -bool DEPRECATED_TxnLog::haveSeen(TypePackId lhs, TypePackId rhs) -{ - return haveSeen((TypeOrPackId)lhs, (TypeOrPackId)rhs); -} - -void DEPRECATED_TxnLog::pushSeen(TypePackId lhs, TypePackId rhs) -{ - pushSeen((TypeOrPackId)lhs, (TypeOrPackId)rhs); -} - -void DEPRECATED_TxnLog::popSeen(TypePackId lhs, TypePackId rhs) -{ - popSeen((TypeOrPackId)lhs, (TypeOrPackId)rhs); -} - -bool DEPRECATED_TxnLog::haveSeen(TypeOrPackId lhs, TypeOrPackId rhs) -{ - LUAU_ASSERT(!FFlag::LuauUseCommittingTxnLog); - const std::pair sortedPair = (lhs > rhs) ? std::make_pair(lhs, rhs) : std::make_pair(rhs, lhs); - return (sharedSeen->end() != std::find(sharedSeen->begin(), sharedSeen->end(), sortedPair)); -} - -void DEPRECATED_TxnLog::pushSeen(TypeOrPackId lhs, TypeOrPackId rhs) -{ - LUAU_ASSERT(!FFlag::LuauUseCommittingTxnLog); - const std::pair sortedPair = (lhs > rhs) ? std::make_pair(lhs, rhs) : std::make_pair(rhs, lhs); - sharedSeen->push_back(sortedPair); -} - -void DEPRECATED_TxnLog::popSeen(TypeOrPackId lhs, TypeOrPackId rhs) -{ - LUAU_ASSERT(!FFlag::LuauUseCommittingTxnLog); - const std::pair sortedPair = (lhs > rhs) ? std::make_pair(lhs, rhs) : std::make_pair(rhs, lhs); - LUAU_ASSERT(sortedPair == sharedSeen->back()); - sharedSeen->pop_back(); -} - const std::string nullPendingResult = ""; std::string toString(PendingType* pending) @@ -170,8 +69,6 @@ const TxnLog* TxnLog::empty() void TxnLog::concat(TxnLog rhs) { - LUAU_ASSERT(FFlag::LuauUseCommittingTxnLog); - for (auto& [ty, rep] : rhs.typeVarChanges) typeVarChanges[ty] = std::move(rep); @@ -181,8 +78,6 @@ void TxnLog::concat(TxnLog rhs) void TxnLog::commit() { - LUAU_ASSERT(FFlag::LuauUseCommittingTxnLog); - for (auto& [ty, rep] : typeVarChanges) *asMutable(ty) = rep.get()->pending; @@ -194,16 +89,12 @@ void TxnLog::commit() void TxnLog::clear() { - LUAU_ASSERT(FFlag::LuauUseCommittingTxnLog); - typeVarChanges.clear(); typePackChanges.clear(); } TxnLog TxnLog::inverse() { - LUAU_ASSERT(FFlag::LuauUseCommittingTxnLog); - TxnLog inversed(sharedSeen); for (auto& [ty, _rep] : typeVarChanges) @@ -247,8 +138,6 @@ void TxnLog::popSeen(TypePackId lhs, TypePackId rhs) bool TxnLog::haveSeen(TypeOrPackId lhs, TypeOrPackId rhs) const { - LUAU_ASSERT(FFlag::LuauUseCommittingTxnLog); - const std::pair sortedPair = (lhs > rhs) ? std::make_pair(lhs, rhs) : std::make_pair(rhs, lhs); if (sharedSeen->end() != std::find(sharedSeen->begin(), sharedSeen->end(), sortedPair)) { @@ -265,16 +154,12 @@ bool TxnLog::haveSeen(TypeOrPackId lhs, TypeOrPackId rhs) const void TxnLog::pushSeen(TypeOrPackId lhs, TypeOrPackId rhs) { - LUAU_ASSERT(FFlag::LuauUseCommittingTxnLog); - const std::pair sortedPair = (lhs > rhs) ? std::make_pair(lhs, rhs) : std::make_pair(rhs, lhs); sharedSeen->push_back(sortedPair); } void TxnLog::popSeen(TypeOrPackId lhs, TypeOrPackId rhs) { - LUAU_ASSERT(FFlag::LuauUseCommittingTxnLog); - const std::pair sortedPair = (lhs > rhs) ? std::make_pair(lhs, rhs) : std::make_pair(rhs, lhs); LUAU_ASSERT(sortedPair == sharedSeen->back()); sharedSeen->pop_back(); @@ -282,7 +167,6 @@ void TxnLog::popSeen(TypeOrPackId lhs, TypeOrPackId rhs) PendingType* TxnLog::queue(TypeId ty) { - LUAU_ASSERT(FFlag::LuauUseCommittingTxnLog); LUAU_ASSERT(!ty->persistent); // Explicitly don't look in ancestors. If we have discovered something new @@ -296,7 +180,6 @@ PendingType* TxnLog::queue(TypeId ty) PendingTypePack* TxnLog::queue(TypePackId tp) { - LUAU_ASSERT(FFlag::LuauUseCommittingTxnLog); LUAU_ASSERT(!tp->persistent); // Explicitly don't look in ancestors. If we have discovered something new diff --git a/Analysis/src/TypeInfer.cpp b/Analysis/src/TypeInfer.cpp index 8e6b3b52..3fe4c90e 100644 --- a/Analysis/src/TypeInfer.cpp +++ b/Analysis/src/TypeInfer.cpp @@ -24,7 +24,6 @@ LUAU_FASTINTVARIABLE(LuauCheckRecursionLimit, 500) LUAU_FASTFLAG(LuauKnowsTheDataModel3) LUAU_FASTFLAGVARIABLE(LuauEqConstraint, false) LUAU_FASTFLAGVARIABLE(LuauWeakEqConstraint, false) // Eventually removed as false. -LUAU_FASTFLAG(LuauUseCommittingTxnLog) LUAU_FASTFLAGVARIABLE(DebugLuauFreezeDuringUnification, false) LUAU_FASTFLAGVARIABLE(LuauRecursiveTypeParameterRestriction, false) LUAU_FASTFLAGVARIABLE(LuauGenericFunctionsDontCacheTypeParams, false) @@ -36,6 +35,7 @@ LUAU_FASTFLAGVARIABLE(LuauExpectedTypesOfProperties, false) LUAU_FASTFLAGVARIABLE(LuauErrorRecoveryType, false) LUAU_FASTFLAGVARIABLE(LuauOnlyMutateInstantiatedTables, false) LUAU_FASTFLAGVARIABLE(LuauPropertiesGetExpectedType, false) +LUAU_FASTFLAGVARIABLE(LuauStatFunctionSimplify, false) LUAU_FASTFLAGVARIABLE(LuauUnsealedTableLiteral, false) LUAU_FASTFLAGVARIABLE(LuauTwoPassAliasDefinitionFix, false) LUAU_FASTFLAGVARIABLE(LuauAssertStripsFalsyTypes, false) @@ -43,6 +43,8 @@ LUAU_FASTFLAGVARIABLE(LuauReturnAnyInsteadOfICE, false) // Eventually removed as LUAU_FASTFLAG(LuauWidenIfSupertypeIsFree) LUAU_FASTFLAGVARIABLE(LuauDoNotTryToReduce, false) LUAU_FASTFLAGVARIABLE(LuauDoNotAccidentallyDependOnPointerOrdering, false) +LUAU_FASTFLAGVARIABLE(LuauFixArgumentCountMismatchAmountWithGenericTypes, false) +LUAU_FASTFLAGVARIABLE(LuauFixIncorrectLineNumberDuplicateType, false) namespace Luau { @@ -652,18 +654,15 @@ ErrorVec TypeChecker::tryUnify_(Id subTy, Id superTy, const Location& location) { Unifier state = mkUnifier(location); - if (FFlag::LuauUseCommittingTxnLog && FFlag::DebugLuauFreezeDuringUnification) + if (FFlag::DebugLuauFreezeDuringUnification) freeze(currentModule->internalTypes); state.tryUnify(subTy, superTy); - if (FFlag::LuauUseCommittingTxnLog && FFlag::DebugLuauFreezeDuringUnification) + if (FFlag::DebugLuauFreezeDuringUnification) unfreeze(currentModule->internalTypes); - if (!state.errors.empty() && !FFlag::LuauUseCommittingTxnLog) - state.DEPRECATED_log.rollback(); - - if (state.errors.empty() && FFlag::LuauUseCommittingTxnLog) + if (state.errors.empty()) state.log.commit(); return state.errors; @@ -847,8 +846,7 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatLocal& local) state.tryUnify(valuePack, variablePack); reportErrors(state.errors); - if (FFlag::LuauUseCommittingTxnLog) - state.log.commit(); + state.log.commit(); // In the code 'local T = {}', we wish to ascribe the name 'T' to the type of the table for error-reporting purposes. // We also want to do this for 'local T = setmetatable(...)'. @@ -1040,8 +1038,7 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatForIn& forin) Unifier state = mkUnifier(firstValue->location); checkArgumentList(loopScope, state, argPack, iterFunc->argTypes, /*argLocations*/ {}); - if (FFlag::LuauUseCommittingTxnLog) - state.log.commit(); + state.log.commit(); reportErrors(state.errors); } @@ -1102,8 +1099,53 @@ void TypeChecker::check(const ScopePtr& scope, TypeId ty, const ScopePtr& funSco scope->bindings[name->local] = {anyIfNonstrict(quantify(funScope, ty, name->local->location)), name->local->location}; return; } + else if (auto name = function.name->as(); name && FFlag::LuauStatFunctionSimplify) + { + TypeId exprTy = checkExpr(scope, *name->expr).type; + TableTypeVar* ttv = getMutableTableType(exprTy); + if (!ttv) + { + if (isTableIntersection(exprTy)) + reportError(TypeError{function.location, CannotExtendTable{exprTy, CannotExtendTable::Property, name->index.value}}); + else if (!get(exprTy) && !get(exprTy)) + reportError(TypeError{function.location, OnlyTablesCanHaveMethods{exprTy}}); + } + else if (ttv->state == TableState::Sealed) + reportError(TypeError{function.location, CannotExtendTable{exprTy, CannotExtendTable::Property, name->index.value}}); + + ty = follow(ty); + + if (ttv && ttv->state != TableState::Sealed) + ttv->props[name->index.value] = {ty, /* deprecated */ false, {}, name->indexLocation}; + + if (function.func->self) + { + const FunctionTypeVar* funTy = get(ty); + if (!funTy) + ice("Methods should be functions"); + + std::optional arg0 = first(funTy->argTypes); + if (!arg0) + ice("Methods should always have at least 1 argument (self)"); + } + + checkFunctionBody(funScope, ty, *function.func); + + if (ttv && ttv->state != TableState::Sealed) + ttv->props[name->index.value] = {follow(quantify(funScope, ty, name->indexLocation)), /* deprecated */ false, {}, name->indexLocation}; + } + else if (FFlag::LuauStatFunctionSimplify) + { + LUAU_ASSERT(function.name->is()); + + ty = follow(ty); + + checkFunctionBody(funScope, ty, *function.func); + } else if (function.func->self) { + LUAU_ASSERT(!FFlag::LuauStatFunctionSimplify); + AstExprIndexName* indexName = function.name->as(); if (!indexName) ice("member function declaration has malformed name expression"); @@ -1141,6 +1183,8 @@ void TypeChecker::check(const ScopePtr& scope, TypeId ty, const ScopePtr& funSco } else { + LUAU_ASSERT(!FFlag::LuauStatFunctionSimplify); + TypeId leftType = checkLValueBinding(scope, *function.name); checkFunctionBody(funScope, ty, *function.func); @@ -1217,6 +1261,9 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatTypeAlias& typealias LUAU_ASSERT(ftv); ftv->forwardedTypeAlias = true; bindingsMap[name] = {std::move(generics), std::move(genericPacks), ty}; + + if (FFlag::LuauFixIncorrectLineNumberDuplicateType) + scope->typeAliasLocations[name] = typealias.location; } } else @@ -2102,9 +2149,7 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprUn Unifier state = mkUnifier(expr.location); state.tryUnify(actualFunctionType, expectedFunctionType, /*isFunctionCall*/ true); - - if (FFlag::LuauUseCommittingTxnLog) - state.log.commit(); + state.log.commit(); TypeId retType = first(retTypePack).value_or(nilType); if (!state.errors.empty()) @@ -2283,9 +2328,7 @@ TypeId TypeChecker::checkRelationalOperation( if (!isEquality) { state.tryUnify(rhsType, lhsType); - - if (FFlag::LuauUseCommittingTxnLog) - state.log.commit(); + state.log.commit(); } bool needsMetamethod = !isEquality; @@ -2336,8 +2379,7 @@ TypeId TypeChecker::checkRelationalOperation( return errorRecoveryType(booleanType); } - if (FFlag::LuauUseCommittingTxnLog) - state.log.commit(); + state.log.commit(); } } @@ -2347,8 +2389,7 @@ TypeId TypeChecker::checkRelationalOperation( state.tryUnify( instantiate(scope, actualFunctionType, expr.location), instantiate(scope, *metamethod, expr.location), /*isFunctionCall*/ true); - if (FFlag::LuauUseCommittingTxnLog) - state.log.commit(); + state.log.commit(); reportErrors(state.errors); return booleanType; @@ -2464,25 +2505,15 @@ TypeId TypeChecker::checkBinaryOperation( TypePackId fallbackArguments = freshTypePack(scope); TypeId fallbackFunctionType = addType(FunctionTypeVar(scope->level, fallbackArguments, retTypePack)); state.errors.clear(); - - if (FFlag::LuauUseCommittingTxnLog) - { - state.log.clear(); - } - else - { - state.DEPRECATED_log.rollback(); - } + state.log.clear(); state.tryUnify(actualFunctionType, fallbackFunctionType, /*isFunctionCall*/ true); - if (FFlag::LuauUseCommittingTxnLog && state.errors.empty()) + if (state.errors.empty()) state.log.commit(); - else if (!state.errors.empty() && !FFlag::LuauUseCommittingTxnLog) - state.DEPRECATED_log.rollback(); } - if (FFlag::LuauUseCommittingTxnLog && !hasErrors) + if (!hasErrors) { state.log.commit(); } @@ -2729,13 +2760,11 @@ TypeId TypeChecker::checkLValueBinding(const ScopePtr& scope, const AstExprIndex TypeId retType = indexer->indexResultType; if (!state.errors.empty()) { - if (!FFlag::LuauUseCommittingTxnLog) - state.DEPRECATED_log.rollback(); reportError(expr.location, UnknownProperty{lhs, name}); retType = errorRecoveryType(retType); } - else if (FFlag::LuauUseCommittingTxnLog) + else state.log.commit(); return retType; @@ -3209,7 +3238,7 @@ ExprResult TypeChecker::checkExprPack(const ScopePtr& scope, const A } // Returns the minimum number of arguments the argument list can accept. -static size_t getMinParameterCount(TypePackId tp) +static size_t getMinParameterCount_DEPRECATED(TypePackId tp) { size_t minCount = 0; size_t optionalCount = 0; @@ -3235,6 +3264,32 @@ static size_t getMinParameterCount(TypePackId tp) return minCount; } +static size_t getMinParameterCount(TxnLog* log, TypePackId tp) +{ + size_t minCount = 0; + size_t optionalCount = 0; + + auto it = begin(tp, log); + auto endIter = end(tp); + + while (it != endIter) + { + TypeId ty = *it; + if (isOptional(ty)) + ++optionalCount; + else + { + minCount += optionalCount; + optionalCount = 0; + minCount++; + } + + ++it; + } + + return minCount; +} + void TypeChecker::checkArgumentList( const ScopePtr& scope, Unifier& state, TypePackId argPack, TypePackId paramPack, const std::vector& argLocations) { @@ -3248,396 +3303,199 @@ void TypeChecker::checkArgumentList( size_t paramIndex = 0; - size_t minParams = getMinParameterCount(paramPack); + size_t minParams = FFlag::LuauFixIncorrectLineNumberDuplicateType ? 0 : getMinParameterCount_DEPRECATED(paramPack); - if (FFlag::LuauUseCommittingTxnLog) + while (true) { - while (true) + state.location = paramIndex < argLocations.size() ? argLocations[paramIndex] : state.location; + + if (argIter == endIter && paramIter == endIter) { - state.location = paramIndex < argLocations.size() ? argLocations[paramIndex] : state.location; + std::optional argTail = argIter.tail(); + std::optional paramTail = paramIter.tail(); - if (argIter == endIter && paramIter == endIter) + // If we hit the end of both type packs simultaneously, then there are definitely no further type + // errors to report. All we need to do is tie up any free tails. + // + // If one side has a free tail and the other has none at all, we create an empty pack and bind the + // free tail to that. + + if (argTail) { - std::optional argTail = argIter.tail(); - std::optional paramTail = paramIter.tail(); - - // If we hit the end of both type packs simultaneously, then there are definitely no further type - // errors to report. All we need to do is tie up any free tails. - // - // If one side has a free tail and the other has none at all, we create an empty pack and bind the - // free tail to that. - - if (argTail) + if (state.log.getMutable(state.log.follow(*argTail))) { - if (state.log.getMutable(state.log.follow(*argTail))) - { - if (paramTail) - state.tryUnify(*paramTail, *argTail); - else - state.log.replace(*argTail, TypePackVar(TypePack{{}})); - } - } - else if (paramTail) - { - // argTail is definitely empty - if (state.log.getMutable(state.log.follow(*paramTail))) - state.log.replace(*paramTail, TypePackVar(TypePack{{}})); - } - - return; - } - else if (argIter == endIter) - { - // Not enough arguments. - - // Might be ok if we are forwarding a vararg along. This is a common thing to occur in nonstrict mode. - if (argIter.tail()) - { - TypePackId tail = *argIter.tail(); - if (state.log.getMutable(tail)) - { - // Unify remaining parameters so we don't leave any free-types hanging around. - while (paramIter != endIter) - { - state.tryUnify(errorRecoveryType(anyType), *paramIter); - ++paramIter; - } - return; - } - else if (auto vtp = state.log.getMutable(tail)) - { - while (paramIter != endIter) - { - state.tryUnify(vtp->ty, *paramIter); - ++paramIter; - } - - return; - } - else if (state.log.getMutable(tail)) - { - std::vector rest; - rest.reserve(std::distance(paramIter, endIter)); - while (paramIter != endIter) - { - rest.push_back(*paramIter); - ++paramIter; - } - - TypePackId varPack = addTypePack(TypePackVar{TypePack{rest, paramIter.tail()}}); - state.tryUnify(varPack, tail); - return; - } - } - - // If any remaining unfulfilled parameters are nonoptional, this is a problem. - while (paramIter != endIter) - { - TypeId t = state.log.follow(*paramIter); - if (isOptional(t)) - { - } // ok - else if (state.log.getMutable(t)) - { - } // ok - else if (isNonstrictMode() && state.log.getMutable(t)) - { - } // ok + if (paramTail) + state.tryUnify(*paramTail, *argTail); else - { - state.reportError(TypeError{state.location, CountMismatch{minParams, paramIndex}}); - return; - } - ++paramIter; + state.log.replace(*argTail, TypePackVar(TypePack{{}})); } } - else if (paramIter == endIter) + else if (paramTail) { - // too many parameters passed - if (!paramIter.tail()) - { - while (argIter != endIter) - { - // The use of unify here is deliberate. We don't want this unification - // to be undoable. - unify(errorRecoveryType(scope), *argIter, state.location); - ++argIter; - } - // For this case, we want the error span to cover every errant extra parameter - Location location = state.location; - if (!argLocations.empty()) - location = {state.location.begin, argLocations.back().end}; - state.reportError(TypeError{location, CountMismatch{minParams, std::distance(begin(argPack), end(argPack))}}); - return; - } - TypePackId tail = state.log.follow(*paramIter.tail()); + // argTail is definitely empty + if (state.log.getMutable(state.log.follow(*paramTail))) + state.log.replace(*paramTail, TypePackVar(TypePack{{}})); + } + return; + } + else if (argIter == endIter) + { + // Not enough arguments. + + // Might be ok if we are forwarding a vararg along. This is a common thing to occur in nonstrict mode. + if (argIter.tail()) + { + TypePackId tail = *argIter.tail(); if (state.log.getMutable(tail)) { - // Function is variadic. Ok. + // Unify remaining parameters so we don't leave any free-types hanging around. + while (paramIter != endIter) + { + state.tryUnify(errorRecoveryType(anyType), *paramIter); + ++paramIter; + } return; } else if (auto vtp = state.log.getMutable(tail)) { - // Function is variadic and requires that all subsequent parameters - // be compatible with a type. - size_t argIndex = paramIndex; - while (argIter != endIter) + while (paramIter != endIter) { - Location location = state.location; - - if (argIndex < argLocations.size()) - location = argLocations[argIndex]; - - unify(*argIter, vtp->ty, location); - ++argIter; - ++argIndex; + state.tryUnify(vtp->ty, *paramIter); + ++paramIter; } return; } else if (state.log.getMutable(tail)) { - // Create a type pack out of the remaining argument types - // and unify it with the tail. std::vector rest; - rest.reserve(std::distance(argIter, endIter)); - while (argIter != endIter) + rest.reserve(std::distance(paramIter, endIter)); + while (paramIter != endIter) { - rest.push_back(*argIter); - ++argIter; + rest.push_back(*paramIter); + ++paramIter; } - TypePackId varPack = addTypePack(TypePackVar{TypePack{rest, argIter.tail()}}); + TypePackId varPack = addTypePack(TypePackVar{TypePack{rest, paramIter.tail()}}); state.tryUnify(varPack, tail); return; } - else if (state.log.getMutable(tail)) - { - state.log.replace(tail, TypePackVar(TypePack{{}})); - return; - } - else if (state.log.getMutable(tail)) - { - // For this case, we want the error span to cover every errant extra parameter - Location location = state.location; - if (!argLocations.empty()) - location = {state.location.begin, argLocations.back().end}; - // TODO: Better error message? - state.reportError(TypeError{location, CountMismatch{minParams, std::distance(begin(argPack), end(argPack))}}); - return; - } } - else + + // If any remaining unfulfilled parameters are nonoptional, this is a problem. + while (paramIter != endIter) { - unifyWithInstantiationIfNeeded(scope, *argIter, *paramIter, state); - ++argIter; + TypeId t = state.log.follow(*paramIter); + if (isOptional(t)) + { + } // ok + else if (state.log.getMutable(t)) + { + } // ok + else if (isNonstrictMode() && state.log.getMutable(t)) + { + } // ok + else + { + if (FFlag::LuauFixArgumentCountMismatchAmountWithGenericTypes) + minParams = getMinParameterCount(&state.log, paramPack); + state.reportError(TypeError{state.location, CountMismatch{minParams, paramIndex}}); + return; + } ++paramIter; } - - ++paramIndex; } - } - else - { - while (true) + else if (paramIter == endIter) { - state.location = paramIndex < argLocations.size() ? argLocations[paramIndex] : state.location; - - if (argIter == endIter && paramIter == endIter) + // too many parameters passed + if (!paramIter.tail()) { - std::optional argTail = argIter.tail(); - std::optional paramTail = paramIter.tail(); - - // If we hit the end of both type packs simultaneously, then there are definitely no further type - // errors to report. All we need to do is tie up any free tails. - // - // If one side has a free tail and the other has none at all, we create an empty pack and bind the - // free tail to that. - - if (argTail) + while (argIter != endIter) { - if (get(*argTail)) - { - if (paramTail) - state.tryUnify(*paramTail, *argTail); - else - { - state.DEPRECATED_log(*argTail); - *asMutable(*argTail) = TypePack{{}}; - } - } + // The use of unify here is deliberate. We don't want this unification + // to be undoable. + unify(errorRecoveryType(scope), *argIter, state.location); + ++argIter; } - else if (paramTail) + // For this case, we want the error span to cover every errant extra parameter + Location location = state.location; + if (!argLocations.empty()) + location = {state.location.begin, argLocations.back().end}; + + if (FFlag::LuauFixArgumentCountMismatchAmountWithGenericTypes) + minParams = getMinParameterCount(&state.log, paramPack); + state.reportError(TypeError{location, CountMismatch{minParams, std::distance(begin(argPack), end(argPack))}}); + return; + } + TypePackId tail = state.log.follow(*paramIter.tail()); + + if (state.log.getMutable(tail)) + { + // Function is variadic. Ok. + return; + } + else if (auto vtp = state.log.getMutable(tail)) + { + // Function is variadic and requires that all subsequent parameters + // be compatible with a type. + size_t argIndex = paramIndex; + while (argIter != endIter) { - // argTail is definitely empty - if (get(*paramTail)) - { - state.DEPRECATED_log(*paramTail); - *asMutable(*paramTail) = TypePack{{}}; - } + Location location = state.location; + + if (argIndex < argLocations.size()) + location = argLocations[argIndex]; + + unify(*argIter, vtp->ty, location); + ++argIter; + ++argIndex; } return; } - else if (argIter == endIter) + else if (state.log.getMutable(tail)) { - // Not enough arguments. - - // Might be ok if we are forwarding a vararg along. This is a common thing to occur in nonstrict mode. - if (argIter.tail()) + // Create a type pack out of the remaining argument types + // and unify it with the tail. + std::vector rest; + rest.reserve(std::distance(argIter, endIter)); + while (argIter != endIter) { - TypePackId tail = *argIter.tail(); - if (get(tail)) - { - // Unify remaining parameters so we don't leave any free-types hanging around. - while (paramIter != endIter) - { - state.tryUnify(*paramIter, errorRecoveryType(anyType)); - ++paramIter; - } - return; - } - else if (auto vtp = get(tail)) - { - while (paramIter != endIter) - { - state.tryUnify(*paramIter, vtp->ty); - ++paramIter; - } - - return; - } - else if (get(tail)) - { - std::vector rest; - rest.reserve(std::distance(paramIter, endIter)); - while (paramIter != endIter) - { - rest.push_back(*paramIter); - ++paramIter; - } - - TypePackId varPack = addTypePack(TypePackVar{TypePack{rest, paramIter.tail()}}); - state.tryUnify(varPack, tail); - return; - } + rest.push_back(*argIter); + ++argIter; } - // If any remaining unfulfilled parameters are nonoptional, this is a problem. - while (paramIter != endIter) - { - TypeId t = follow(*paramIter); - if (isOptional(t)) - { - } // ok - else if (get(t)) - { - } // ok - else if (isNonstrictMode() && get(t)) - { - } // ok - else - { - state.reportError(TypeError{state.location, CountMismatch{minParams, paramIndex}}); - return; - } - ++paramIter; - } + TypePackId varPack = addTypePack(TypePackVar{TypePack{rest, argIter.tail()}}); + state.tryUnify(varPack, tail); + return; } - else if (paramIter == endIter) + else if (state.log.getMutable(tail)) { - // too many parameters passed - if (!paramIter.tail()) - { - while (argIter != endIter) - { - unify(*argIter, errorRecoveryType(scope), state.location); - ++argIter; - } - // For this case, we want the error span to cover every errant extra parameter - Location location = state.location; - if (!argLocations.empty()) - location = {state.location.begin, argLocations.back().end}; - state.reportError(TypeError{location, CountMismatch{minParams, std::distance(begin(argPack), end(argPack))}}); - return; - } - TypePackId tail = *paramIter.tail(); - - if (get(tail)) - { - // Function is variadic. Ok. - return; - } - else if (auto vtp = get(tail)) - { - // Function is variadic and requires that all subsequent parameters - // be compatible with a type. - size_t argIndex = paramIndex; - while (argIter != endIter) - { - Location location = state.location; - - if (argIndex < argLocations.size()) - location = argLocations[argIndex]; - - unify(*argIter, vtp->ty, location); - ++argIter; - ++argIndex; - } - - return; - } - else if (get(tail)) - { - // Create a type pack out of the remaining argument types - // and unify it with the tail. - std::vector rest; - rest.reserve(std::distance(argIter, endIter)); - while (argIter != endIter) - { - rest.push_back(*argIter); - ++argIter; - } - - TypePackId varPack = addTypePack(TypePackVar{TypePack{rest, argIter.tail()}}); - state.tryUnify(tail, varPack); - return; - } - else if (get(tail)) - { - if (FFlag::LuauUseCommittingTxnLog) - { - state.log.replace(tail, TypePackVar(TypePack{{}})); - } - else - { - state.DEPRECATED_log(tail); - *asMutable(tail) = TypePack{}; - } - - return; - } - else if (get(tail)) - { - // For this case, we want the error span to cover every errant extra parameter - Location location = state.location; - if (!argLocations.empty()) - location = {state.location.begin, argLocations.back().end}; - // TODO: Better error message? - state.reportError(TypeError{location, CountMismatch{minParams, std::distance(begin(argPack), end(argPack))}}); - return; - } + state.log.replace(tail, TypePackVar(TypePack{{}})); + return; } - else + else if (state.log.getMutable(tail)) { - unifyWithInstantiationIfNeeded(scope, *argIter, *paramIter, state); - ++argIter; - ++paramIter; + // For this case, we want the error span to cover every errant extra parameter + Location location = state.location; + if (!argLocations.empty()) + location = {state.location.begin, argLocations.back().end}; + // TODO: Better error message? + if (FFlag::LuauFixArgumentCountMismatchAmountWithGenericTypes) + minParams = getMinParameterCount(&state.log, paramPack); + state.reportError(TypeError{location, CountMismatch{minParams, std::distance(begin(argPack), end(argPack))}}); + return; } - - ++paramIndex; } + else + { + unifyWithInstantiationIfNeeded(scope, *argIter, *paramIter, state); + ++argIter; + ++paramIter; + } + + ++paramIndex; } } @@ -3882,9 +3740,6 @@ std::optional> TypeChecker::checkCallOverload(const Scope checkArgumentList(scope, state, retPack, ftv->retType, /*argLocations*/ {}); if (!state.errors.empty()) { - if (!FFlag::LuauUseCommittingTxnLog) - state.DEPRECATED_log.rollback(); - return {}; } @@ -3912,14 +3767,10 @@ std::optional> TypeChecker::checkCallOverload(const Scope overloadsThatDont.push_back(fn); errors.emplace_back(std::move(state.errors), args->head, ftv); - - if (!FFlag::LuauUseCommittingTxnLog) - state.DEPRECATED_log.rollback(); } else { - if (FFlag::LuauUseCommittingTxnLog) - state.log.commit(); + state.log.commit(); if (isNonstrictMode() && !expr.self && expr.func->is() && ftv->hasSelf) { @@ -3976,8 +3827,7 @@ bool TypeChecker::handleSelfCallMismatch(const ScopePtr& scope, const AstExprCal if (editedState.errors.empty()) { - if (FFlag::LuauUseCommittingTxnLog) - editedState.log.commit(); + editedState.log.commit(); reportError(TypeError{expr.location, FunctionDoesNotTakeSelf{}}); // This is a little bit suspect: If this overload would work with a . replaced by a : @@ -3987,8 +3837,6 @@ bool TypeChecker::handleSelfCallMismatch(const ScopePtr& scope, const AstExprCal // checkArgumentList(scope, editedState, retPack, ftv->retType, retLocations, CountMismatch::Return); return true; } - else if (!FFlag::LuauUseCommittingTxnLog) - editedState.DEPRECATED_log.rollback(); } else if (ftv->hasSelf) { @@ -4010,8 +3858,7 @@ bool TypeChecker::handleSelfCallMismatch(const ScopePtr& scope, const AstExprCal if (editedState.errors.empty()) { - if (FFlag::LuauUseCommittingTxnLog) - editedState.log.commit(); + editedState.log.commit(); reportError(TypeError{expr.location, FunctionRequiresSelf{}}); // This is a little bit suspect: If this overload would work with a : replaced by a . @@ -4021,8 +3868,6 @@ bool TypeChecker::handleSelfCallMismatch(const ScopePtr& scope, const AstExprCal // checkArgumentList(scope, editedState, retPack, ftv->retType, retLocations, CountMismatch::Return); return true; } - else if (!FFlag::LuauUseCommittingTxnLog) - editedState.DEPRECATED_log.rollback(); } } } @@ -4082,7 +3927,7 @@ void TypeChecker::reportOverloadResolutionError(const ScopePtr& scope, const Ast checkArgumentList(scope, state, argPack, ftv->argTypes, argLocations); } - if (FFlag::LuauUseCommittingTxnLog && state.errors.empty()) + if (state.errors.empty()) state.log.commit(); if (i > 0) @@ -4092,9 +3937,6 @@ void TypeChecker::reportOverloadResolutionError(const ScopePtr& scope, const Ast s += "and "; s += toString(overload); - - if (!FFlag::LuauUseCommittingTxnLog) - state.DEPRECATED_log.rollback(); } if (overloadsThatMatchArgCount.size() == 0) @@ -4168,24 +4010,16 @@ ExprResult TypeChecker::checkExprList(const ScopePtr& scope, const L // just performed. There's not a great way to pass that into checkExpr. Instead, we store // the inverse of the current log, and commit it. When we're done, we'll commit all the // inverses. This isn't optimal, and a better solution is welcome here. - if (FFlag::LuauUseCommittingTxnLog) - { - inverseLogs.push_back(state.log.inverse()); - state.log.commit(); - } + inverseLogs.push_back(state.log.inverse()); + state.log.commit(); } tp->head.push_back(actualType); } } - if (FFlag::LuauUseCommittingTxnLog) - { - for (TxnLog& log : inverseLogs) - log.commit(); - } - else - state.DEPRECATED_log.rollback(); + for (TxnLog& log : inverseLogs) + log.commit(); return {pack, predicates}; } @@ -4294,8 +4128,7 @@ bool TypeChecker::unify(TypeId subTy, TypeId superTy, const Location& location, Unifier state = mkUnifier(location); state.tryUnify(subTy, superTy, options.isFunctionCall); - if (FFlag::LuauUseCommittingTxnLog) - state.log.commit(); + state.log.commit(); reportErrors(state.errors); @@ -4308,8 +4141,7 @@ bool TypeChecker::unify(TypePackId subTy, TypePackId superTy, const Location& lo state.ctx = ctx; state.tryUnify(subTy, superTy); - if (FFlag::LuauUseCommittingTxnLog) - state.log.commit(); + state.log.commit(); reportErrors(state.errors); @@ -4321,8 +4153,7 @@ bool TypeChecker::unifyWithInstantiationIfNeeded(const ScopePtr& scope, TypeId s Unifier state = mkUnifier(location); unifyWithInstantiationIfNeeded(scope, subTy, superTy, state); - if (FFlag::LuauUseCommittingTxnLog) - state.log.commit(); + state.log.commit(); reportErrors(state.errors); @@ -4352,31 +4183,18 @@ void TypeChecker::unifyWithInstantiationIfNeeded(const ScopePtr& scope, TypeId s if (subTy == instantiated) { // Instantiating the argument made no difference, so just report any child errors - if (FFlag::LuauUseCommittingTxnLog) - state.log.concat(std::move(child.log)); - else - state.DEPRECATED_log.concat(std::move(child.DEPRECATED_log)); + state.log.concat(std::move(child.log)); state.errors.insert(state.errors.end(), child.errors.begin(), child.errors.end()); } else { - if (!FFlag::LuauUseCommittingTxnLog) - child.DEPRECATED_log.rollback(); - state.tryUnify(instantiated, superTy, /*isFunctionCall*/ false); } } else { - if (FFlag::LuauUseCommittingTxnLog) - { - state.log.concat(std::move(child.log)); - } - else - { - state.DEPRECATED_log.concat(std::move(child.DEPRECATED_log)); - } + state.log.concat(std::move(child.log)); } } } @@ -4540,7 +4358,7 @@ TypeId TypeChecker::quantify(const ScopePtr& scope, TypeId ty, Location location TypeId TypeChecker::instantiate(const ScopePtr& scope, TypeId ty, Location location, const TxnLog* log) { - Instantiation instantiation{FFlag::LuauUseCommittingTxnLog ? log : TxnLog::empty(), ¤tModule->internalTypes, scope->level}; + Instantiation instantiation{log, ¤tModule->internalTypes, scope->level}; std::optional instantiated = instantiation.substitute(ty); if (instantiated.has_value()) return *instantiated; diff --git a/Analysis/src/TypePack.cpp b/Analysis/src/TypePack.cpp index b15548a8..91123f46 100644 --- a/Analysis/src/TypePack.cpp +++ b/Analysis/src/TypePack.cpp @@ -5,8 +5,6 @@ #include -LUAU_FASTFLAG(LuauUseCommittingTxnLog) - namespace Luau { @@ -51,16 +49,8 @@ TypePackIterator::TypePackIterator(TypePackId typePack, const TxnLog* log) { while (tp && tp->head.empty()) { - if (FFlag::LuauUseCommittingTxnLog) - { - currentTypePack = tp->tail ? log->follow(*tp->tail) : nullptr; - tp = currentTypePack ? log->getMutable(currentTypePack) : nullptr; - } - else - { - currentTypePack = tp->tail ? follow(*tp->tail) : nullptr; - tp = currentTypePack ? get(currentTypePack) : nullptr; - } + currentTypePack = tp->tail ? log->follow(*tp->tail) : nullptr; + tp = currentTypePack ? log->getMutable(currentTypePack) : nullptr; } } @@ -71,16 +61,8 @@ TypePackIterator& TypePackIterator::operator++() ++currentIndex; while (tp && currentIndex >= tp->head.size()) { - if (FFlag::LuauUseCommittingTxnLog) - { - currentTypePack = tp->tail ? log->follow(*tp->tail) : nullptr; - tp = currentTypePack ? log->getMutable(currentTypePack) : nullptr; - } - else - { - currentTypePack = tp->tail ? follow(*tp->tail) : nullptr; - tp = currentTypePack ? get(currentTypePack) : nullptr; - } + currentTypePack = tp->tail ? log->follow(*tp->tail) : nullptr; + tp = currentTypePack ? log->getMutable(currentTypePack) : nullptr; currentIndex = 0; } diff --git a/Analysis/src/Unifier.cpp b/Analysis/src/Unifier.cpp index 6c29486a..7b781f26 100644 --- a/Analysis/src/Unifier.cpp +++ b/Analysis/src/Unifier.cpp @@ -15,30 +15,28 @@ LUAU_FASTINT(LuauTypeInferRecursionLimit); LUAU_FASTINT(LuauTypeInferTypePackLoopLimit); LUAU_FASTFLAG(LuauImmutableTypes) -LUAU_FASTFLAG(LuauUseCommittingTxnLog) LUAU_FASTINTVARIABLE(LuauTypeInferIterationLimit, 2000); LUAU_FASTFLAGVARIABLE(LuauTableSubtypingVariance2, false); LUAU_FASTFLAG(LuauSingletonTypes) LUAU_FASTFLAG(LuauErrorRecoveryType); LUAU_FASTFLAGVARIABLE(LuauSubtypingAddOptPropsToUnsealedTables, false) -LUAU_FASTFLAGVARIABLE(LuauFollowWithCommittingTxnLogInAnyUnification, false) LUAU_FASTFLAGVARIABLE(LuauWidenIfSupertypeIsFree, false) LUAU_FASTFLAGVARIABLE(LuauDifferentOrderOfUnificationDoesntMatter, false) -LUAU_FASTFLAGVARIABLE(LuauTxnLogSeesTypePacks2, true) +LUAU_FASTFLAGVARIABLE(LuauTxnLogSeesTypePacks2, false) +LUAU_FASTFLAGVARIABLE(LuauTxnLogCheckForInvalidation, false) +LUAU_FASTFLAGVARIABLE(LuauTxnLogDontRetryForIndexers, false) namespace Luau { struct PromoteTypeLevels { - DEPRECATED_TxnLog& DEPRECATED_log; TxnLog& log; const TypeArena* typeArena = nullptr; TypeLevel minLevel; - explicit PromoteTypeLevels(DEPRECATED_TxnLog& DEPRECATED_log, TxnLog& log, const TypeArena* typeArena, TypeLevel minLevel) - : DEPRECATED_log(DEPRECATED_log) - , log(log) + PromoteTypeLevels(TxnLog& log, const TypeArena* typeArena, TypeLevel minLevel) + : log(log) , typeArena(typeArena) , minLevel(minLevel) { @@ -50,15 +48,7 @@ struct PromoteTypeLevels LUAU_ASSERT(t); if (minLevel.subsumesStrict(t->level)) { - if (FFlag::LuauUseCommittingTxnLog) - { - log.changeLevel(ty, minLevel); - } - else - { - DEPRECATED_log(ty); - t->level = minLevel; - } + log.changeLevel(ty, minLevel); } } @@ -81,10 +71,10 @@ struct PromoteTypeLevels { // Surprise, it's actually a BoundTypeVar that hasn't been committed yet. // Calling getMutable on this will trigger an assertion. - if (FFlag::LuauUseCommittingTxnLog && !log.is(ty)) + if (!log.is(ty)) return true; - promote(ty, FFlag::LuauUseCommittingTxnLog ? log.getMutable(ty) : getMutable(ty)); + promote(ty, log.getMutable(ty)); return true; } @@ -94,7 +84,7 @@ struct PromoteTypeLevels if (FFlag::LuauImmutableTypes && ty->owningArena != typeArena) return false; - promote(ty, FFlag::LuauUseCommittingTxnLog ? log.getMutable(ty) : getMutable(ty)); + promote(ty, log.getMutable(ty)); return true; } @@ -107,7 +97,7 @@ struct PromoteTypeLevels if (ttv.state != TableState::Free && ttv.state != TableState::Generic) return true; - promote(ty, FFlag::LuauUseCommittingTxnLog ? log.getMutable(ty) : getMutable(ty)); + promote(ty, log.getMutable(ty)); return true; } @@ -115,33 +105,33 @@ struct PromoteTypeLevels { // Surprise, it's actually a BoundTypePack that hasn't been committed yet. // Calling getMutable on this will trigger an assertion. - if (FFlag::LuauUseCommittingTxnLog && !log.is(tp)) + if (!log.is(tp)) return true; - promote(tp, FFlag::LuauUseCommittingTxnLog ? log.getMutable(tp) : getMutable(tp)); + promote(tp, log.getMutable(tp)); return true; } }; -static void promoteTypeLevels(DEPRECATED_TxnLog& DEPRECATED_log, TxnLog& log, const TypeArena* typeArena, TypeLevel minLevel, TypeId ty) +static void promoteTypeLevels(TxnLog& log, const TypeArena* typeArena, TypeLevel minLevel, TypeId ty) { // Type levels of types from other modules are already global, so we don't need to promote anything inside if (FFlag::LuauImmutableTypes && ty->owningArena != typeArena) return; - PromoteTypeLevels ptl{DEPRECATED_log, log, typeArena, minLevel}; + PromoteTypeLevels ptl{log, typeArena, minLevel}; DenseHashSet seen{nullptr}; visitTypeVarOnce(ty, ptl, seen); } // TODO: use this and make it static. -void promoteTypeLevels(DEPRECATED_TxnLog& DEPRECATED_log, TxnLog& log, const TypeArena* typeArena, TypeLevel minLevel, TypePackId tp) +void promoteTypeLevels(TxnLog& log, const TypeArena* typeArena, TypeLevel minLevel, TypePackId tp) { // Type levels of types from other modules are already global, so we don't need to promote anything inside if (FFlag::LuauImmutableTypes && tp->owningArena != typeArena) return; - PromoteTypeLevels ptl{DEPRECATED_log, log, typeArena, minLevel}; + PromoteTypeLevels ptl{log, typeArena, minLevel}; DenseHashSet seen{nullptr}; visitTypeVarOnce(tp, ptl, seen); } @@ -251,7 +241,7 @@ struct SkipCacheForType bool Widen::isDirty(TypeId ty) { - return FFlag::LuauUseCommittingTxnLog ? log->is(ty) : bool(get(ty)); + return log->is(ty); } bool Widen::isDirty(TypePackId) @@ -262,7 +252,7 @@ bool Widen::isDirty(TypePackId) TypeId Widen::clean(TypeId ty) { LUAU_ASSERT(isDirty(ty)); - auto stv = FFlag::LuauUseCommittingTxnLog ? log->getMutable(ty) : getMutable(ty); + auto stv = log->getMutable(ty); LUAU_ASSERT(stv); if (get(stv)) @@ -284,11 +274,11 @@ bool Widen::ignoreChildren(TypeId ty) { // Sometimes we unify ("hi") -> free1 with (free2) -> free3, so don't ignore functions. // TODO: should we be doing this? we would need to rework how checkCallOverload does the unification. - if (FFlag::LuauUseCommittingTxnLog ? log->is(ty) : bool(get(ty))) + if (log->is(ty)) return false; // We only care about unions. - return !(FFlag::LuauUseCommittingTxnLog ? log->is(ty) : bool(get(ty))); + return !log->is(ty); } static std::optional hasUnificationTooComplex(const ErrorVec& errors) @@ -335,7 +325,6 @@ Unifier::Unifier(TypeArena* types, Mode mode, std::vector(superTy); - auto subFree = getMutable(subTy); - - if (FFlag::LuauUseCommittingTxnLog) - { - superFree = log.getMutable(superTy); - subFree = log.getMutable(subTy); - } + auto superFree = log.getMutable(superTy); + auto subFree = log.getMutable(subTy); if (superFree && subFree && superFree->level.subsumes(subFree->level)) { occursCheck(subTy, superTy); // The occurrence check might have caused superTy no longer to be a free type - bool occursFailed = false; - if (FFlag::LuauUseCommittingTxnLog) - occursFailed = bool(log.getMutable(subTy)); - else - occursFailed = bool(get(subTy)); + bool occursFailed = bool(log.getMutable(subTy)); if (!occursFailed) { - if (FFlag::LuauUseCommittingTxnLog) - { - log.replace(subTy, BoundTypeVar(superTy)); - } - else - { - DEPRECATED_log(subTy); - *asMutable(subTy) = BoundTypeVar(superTy); - } + log.replace(subTy, BoundTypeVar(superTy)); } return; } else if (superFree && subFree) { - if (!FFlag::LuauErrorRecoveryType && !FFlag::LuauUseCommittingTxnLog) - { - DEPRECATED_log(superTy); - subFree->level = min(subFree->level, superFree->level); - } - occursCheck(superTy, subTy); - bool occursFailed = false; - if (FFlag::LuauUseCommittingTxnLog) - occursFailed = bool(log.getMutable(superTy)); - else - occursFailed = bool(get(superTy)); - - if (!FFlag::LuauErrorRecoveryType && !FFlag::LuauUseCommittingTxnLog) - { - *asMutable(superTy) = BoundTypeVar(subTy); - return; - } + bool occursFailed = bool(log.getMutable(superTy)); if (!occursFailed) { - if (FFlag::LuauUseCommittingTxnLog) + if (superFree->level.subsumes(subFree->level)) { - if (superFree->level.subsumes(subFree->level)) - { - log.changeLevel(subTy, superFree->level); - } + log.changeLevel(subTy, superFree->level); + } - log.replace(superTy, BoundTypeVar(subTy)); - } - else - { - DEPRECATED_log(superTy); - *asMutable(superTy) = BoundTypeVar(subTy); - subFree->level = min(subFree->level, superFree->level); - } + log.replace(superTy, BoundTypeVar(subTy)); } return; @@ -460,14 +398,10 @@ void Unifier::tryUnify_(TypeId subTy, TypeId superTy, bool isFunctionCall, bool TypeLevel superLevel = superFree->level; occursCheck(superTy, subTy); - bool occursFailed = false; - if (FFlag::LuauUseCommittingTxnLog) - occursFailed = bool(log.getMutable(superTy)); - else - occursFailed = bool(get(superTy)); + bool occursFailed = bool(log.getMutable(superTy)); // Unification can't change the level of a generic. - auto subGeneric = FFlag::LuauUseCommittingTxnLog ? log.getMutable(subTy) : get(subTy); + auto subGeneric = log.getMutable(subTy); if (subGeneric && !subGeneric->level.subsumes(superLevel)) { // TODO: a more informative error message? CLI-39912 @@ -478,18 +412,8 @@ void Unifier::tryUnify_(TypeId subTy, TypeId superTy, bool isFunctionCall, bool // The occurrence check might have caused superTy no longer to be a free type if (!occursFailed) { - if (FFlag::LuauUseCommittingTxnLog) - { - promoteTypeLevels(DEPRECATED_log, log, types, superLevel, subTy); - log.replace(superTy, BoundTypeVar(widen(subTy))); - } - else - { - promoteTypeLevels(DEPRECATED_log, log, types, superLevel, subTy); - - DEPRECATED_log(superTy); - *asMutable(superTy) = BoundTypeVar(widen(subTy)); - } + promoteTypeLevels(log, types, superLevel, subTy); + log.replace(superTy, BoundTypeVar(widen(subTy))); } return; @@ -499,14 +423,10 @@ void Unifier::tryUnify_(TypeId subTy, TypeId superTy, bool isFunctionCall, bool TypeLevel subLevel = subFree->level; occursCheck(subTy, superTy); - bool occursFailed = false; - if (FFlag::LuauUseCommittingTxnLog) - occursFailed = bool(log.getMutable(subTy)); - else - occursFailed = bool(get(subTy)); + bool occursFailed = bool(log.getMutable(subTy)); // Unification can't change the level of a generic. - auto superGeneric = FFlag::LuauUseCommittingTxnLog ? log.getMutable(superTy) : get(superTy); + auto superGeneric = log.getMutable(superTy); if (superGeneric && !superGeneric->level.subsumes(subFree->level)) { // TODO: a more informative error message? CLI-39912 @@ -516,18 +436,8 @@ void Unifier::tryUnify_(TypeId subTy, TypeId superTy, bool isFunctionCall, bool if (!occursFailed) { - if (FFlag::LuauUseCommittingTxnLog) - { - promoteTypeLevels(DEPRECATED_log, log, types, subLevel, superTy); - log.replace(subTy, BoundTypeVar(superTy)); - } - else - { - promoteTypeLevels(DEPRECATED_log, log, types, subLevel, superTy); - - DEPRECATED_log(subTy); - *asMutable(subTy) = BoundTypeVar(superTy); - } + promoteTypeLevels(log, types, subLevel, superTy); + log.replace(subTy, BoundTypeVar(superTy)); } return; @@ -550,55 +460,38 @@ void Unifier::tryUnify_(TypeId subTy, TypeId superTy, bool isFunctionCall, bool // Here, we assume that the types unify. If they do not, we will find out as we roll back // the stack. - if (FFlag::LuauUseCommittingTxnLog) - { - if (log.haveSeen(superTy, subTy)) - return; + if (log.haveSeen(superTy, subTy)) + return; - log.pushSeen(superTy, subTy); - } - else - { - if (DEPRECATED_log.haveSeen(superTy, subTy)) - return; + log.pushSeen(superTy, subTy); - DEPRECATED_log.pushSeen(superTy, subTy); - } - - if (const UnionTypeVar* uv = FFlag::LuauUseCommittingTxnLog ? log.getMutable(subTy) : get(subTy)) + if (const UnionTypeVar* uv = log.getMutable(subTy)) { tryUnifyUnionWithType(subTy, uv, superTy); } - else if (const UnionTypeVar* uv = FFlag::LuauUseCommittingTxnLog ? log.getMutable(superTy) : get(superTy)) + else if (const UnionTypeVar* uv = log.getMutable(superTy)) { tryUnifyTypeWithUnion(subTy, superTy, uv, cacheEnabled, isFunctionCall); } - else if (const IntersectionTypeVar* uv = - FFlag::LuauUseCommittingTxnLog ? log.getMutable(superTy) : get(superTy)) + else if (const IntersectionTypeVar* uv = log.getMutable(superTy)) { tryUnifyTypeWithIntersection(subTy, superTy, uv); } - else if (const IntersectionTypeVar* uv = - FFlag::LuauUseCommittingTxnLog ? log.getMutable(subTy) : get(subTy)) + else if (const IntersectionTypeVar* uv = log.getMutable(subTy)) { tryUnifyIntersectionWithType(subTy, uv, superTy, cacheEnabled, isFunctionCall); } - else if ((FFlag::LuauUseCommittingTxnLog && log.getMutable(superTy) && log.getMutable(subTy)) || - (!FFlag::LuauUseCommittingTxnLog && get(superTy) && get(subTy))) + else if (log.getMutable(superTy) && log.getMutable(subTy)) tryUnifyPrimitives(subTy, superTy); - else if (FFlag::LuauSingletonTypes && - ((FFlag::LuauUseCommittingTxnLog ? log.getMutable(superTy) : get(superTy)) || - (FFlag::LuauUseCommittingTxnLog ? log.getMutable(superTy) : get(superTy))) && - (FFlag::LuauUseCommittingTxnLog ? log.getMutable(subTy) : get(subTy))) + else if (FFlag::LuauSingletonTypes && (log.getMutable(superTy) || log.getMutable(superTy)) && + log.getMutable(subTy)) tryUnifySingletons(subTy, superTy); - else if ((FFlag::LuauUseCommittingTxnLog && log.getMutable(superTy) && log.getMutable(subTy)) || - (!FFlag::LuauUseCommittingTxnLog && get(superTy) && get(subTy))) + else if (log.getMutable(superTy) && log.getMutable(subTy)) tryUnifyFunctions(subTy, superTy, isFunctionCall); - else if ((FFlag::LuauUseCommittingTxnLog && log.getMutable(superTy) && log.getMutable(subTy)) || - (!FFlag::LuauUseCommittingTxnLog && get(superTy) && get(subTy))) + else if (log.getMutable(superTy) && log.getMutable(subTy)) { tryUnifyTables(subTy, superTy, isIntersection); @@ -607,29 +500,23 @@ void Unifier::tryUnify_(TypeId subTy, TypeId superTy, bool isFunctionCall, bool } // tryUnifyWithMetatable assumes its first argument is a MetatableTypeVar. The check is otherwise symmetrical. - else if ((FFlag::LuauUseCommittingTxnLog && log.getMutable(superTy)) || - (!FFlag::LuauUseCommittingTxnLog && get(superTy))) + else if (log.getMutable(superTy)) tryUnifyWithMetatable(subTy, superTy, /*reversed*/ false); - else if ((FFlag::LuauUseCommittingTxnLog && log.getMutable(subTy)) || - (!FFlag::LuauUseCommittingTxnLog && get(subTy))) + else if (log.getMutable(subTy)) tryUnifyWithMetatable(superTy, subTy, /*reversed*/ true); - else if ((FFlag::LuauUseCommittingTxnLog && log.getMutable(superTy)) || - (!FFlag::LuauUseCommittingTxnLog && get(superTy))) + else if (log.getMutable(superTy)) tryUnifyWithClass(subTy, superTy, /*reversed*/ false); // Unification of nonclasses with classes is almost, but not quite symmetrical. // The order in which we perform this test is significant in the case that both types are classes. - else if ((FFlag::LuauUseCommittingTxnLog && log.getMutable(subTy)) || (!FFlag::LuauUseCommittingTxnLog && get(subTy))) + else if (log.getMutable(subTy)) tryUnifyWithClass(subTy, superTy, /*reversed*/ true); else reportError(TypeError{location, TypeMismatch{superTy, subTy}}); - if (FFlag::LuauUseCommittingTxnLog) - log.popSeen(superTy, subTy); - else - DEPRECATED_log.popSeen(superTy, subTy); + log.popSeen(superTy, subTy); } void Unifier::tryUnifyUnionWithType(TypeId subTy, const UnionTypeVar* uv, TypeId superTy) @@ -660,28 +547,12 @@ void Unifier::tryUnifyUnionWithType(TypeId subTy, const UnionTypeVar* uv, TypeId if (FFlag::LuauDifferentOrderOfUnificationDoesntMatter) { - if (!FFlag::LuauUseCommittingTxnLog) - innerState.DEPRECATED_log.rollback(); } else { - if (FFlag::LuauUseCommittingTxnLog) + if (i == count - 1) { - if (i == count - 1) - { - log.concat(std::move(innerState.log)); - } - } - else - { - if (i != count - 1) - { - innerState.DEPRECATED_log.rollback(); - } - else - { - DEPRECATED_log.concat(std::move(innerState.DEPRECATED_log)); - } + log.concat(std::move(innerState.log)); } ++i; @@ -692,7 +563,7 @@ void Unifier::tryUnifyUnionWithType(TypeId subTy, const UnionTypeVar* uv, TypeId if (FFlag::LuauDifferentOrderOfUnificationDoesntMatter) { auto tryBind = [this, subTy](TypeId superOption) { - superOption = FFlag::LuauUseCommittingTxnLog ? log.follow(superOption) : follow(superOption); + superOption = log.follow(superOption); // just skip if the superOption is not free-ish. auto ttv = log.getMutable(superOption); @@ -701,36 +572,17 @@ void Unifier::tryUnifyUnionWithType(TypeId subTy, const UnionTypeVar* uv, TypeId // Since we have already checked if S <: T, checking it again will not queue up the type for replacement. // So we'll have to do it ourselves. We assume they unified cleanly if they are still in the seen set. - if (FFlag::LuauUseCommittingTxnLog) + if (log.haveSeen(subTy, superOption)) { - if (log.haveSeen(subTy, superOption)) - { - // TODO: would it be nice for TxnLog::replace to do this? - if (log.is(superOption)) - log.bindTable(superOption, subTy); - else - log.replace(superOption, *subTy); - } - } - else - { - if (DEPRECATED_log.haveSeen(subTy, superOption)) - { - if (auto ttv = getMutable(superOption)) - { - DEPRECATED_log(ttv); - ttv->boundTo = subTy; - } - else - { - DEPRECATED_log(superOption); - *asMutable(superOption) = BoundTypeVar(subTy); - } - } + // TODO: would it be nice for TxnLog::replace to do this? + if (log.is(superOption)) + log.bindTable(superOption, subTy); + else + log.replace(superOption, *subTy); } }; - if (auto utv = (FFlag::LuauUseCommittingTxnLog ? log.getMutable(superTy) : get(superTy))) + if (auto utv = log.getMutable(superTy)) { for (TypeId ty : utv) tryBind(ty); @@ -815,10 +667,7 @@ void Unifier::tryUnifyTypeWithUnion(TypeId subTy, TypeId superTy, const UnionTyp if (innerState.errors.empty()) { found = true; - if (FFlag::LuauUseCommittingTxnLog) - log.concat(std::move(innerState.log)); - else - DEPRECATED_log.concat(std::move(innerState.DEPRECATED_log)); + log.concat(std::move(innerState.log)); break; } @@ -833,9 +682,6 @@ void Unifier::tryUnifyTypeWithUnion(TypeId subTy, TypeId superTy, const UnionTyp if (!failedOption) failedOption = {innerState.errors.front()}; } - - if (!FFlag::LuauUseCommittingTxnLog) - innerState.DEPRECATED_log.rollback(); } if (unificationTooComplex) @@ -870,10 +716,7 @@ void Unifier::tryUnifyTypeWithIntersection(TypeId subTy, TypeId superTy, const I firstFailedOption = {innerState.errors.front()}; } - if (FFlag::LuauUseCommittingTxnLog) - log.concat(std::move(innerState.log)); - else - DEPRECATED_log.concat(std::move(innerState.DEPRECATED_log)); + log.concat(std::move(innerState.log)); } if (unificationTooComplex) @@ -915,19 +758,13 @@ void Unifier::tryUnifyIntersectionWithType(TypeId subTy, const IntersectionTypeV if (innerState.errors.empty()) { found = true; - if (FFlag::LuauUseCommittingTxnLog) - log.concat(std::move(innerState.log)); - else - DEPRECATED_log.concat(std::move(innerState.DEPRECATED_log)); + log.concat(std::move(innerState.log)); break; } else if (auto e = hasUnificationTooComplex(innerState.errors)) { unificationTooComplex = e; } - - if (!FFlag::LuauUseCommittingTxnLog) - innerState.DEPRECATED_log.rollback(); } if (unificationTooComplex) @@ -971,78 +808,6 @@ void Unifier::cacheResult(TypeId subTy, TypeId superTy) sharedState.cachedUnify.insert({subTy, superTy}); } -struct DEPRECATED_WeirdIter -{ - TypePackId packId; - const TypePack* pack; - size_t index; - bool growing; - TypeLevel level; - - DEPRECATED_WeirdIter(TypePackId packId) - : packId(packId) - , pack(get(packId)) - , index(0) - , growing(false) - { - while (pack && pack->head.empty() && pack->tail) - { - packId = *pack->tail; - pack = get(packId); - } - } - - DEPRECATED_WeirdIter(const DEPRECATED_WeirdIter&) = default; - - const TypeId& operator*() - { - LUAU_ASSERT(good()); - return pack->head[index]; - } - - bool good() const - { - return pack != nullptr && index < pack->head.size(); - } - - bool advance() - { - if (!pack) - return good(); - - if (index < pack->head.size()) - ++index; - - if (growing || index < pack->head.size()) - return good(); - - if (pack->tail) - { - packId = follow(*pack->tail); - pack = get(packId); - index = 0; - } - - return good(); - } - - bool canGrow() const - { - return nullptr != get(packId); - } - - void grow(TypePackId newTail) - { - LUAU_ASSERT(canGrow()); - level = get(packId)->level; - *asMutable(packId) = Unifiable::Bound(newTail); - packId = newTail; - pack = get(newTail); - index = 0; - growing = true; - } -}; - struct WeirdIter { TypePackId packId; @@ -1141,9 +906,6 @@ ErrorVec Unifier::canUnify(TypeId subTy, TypeId superTy) Unifier s = makeChildUnifier(); s.tryUnify_(subTy, superTy); - if (!FFlag::LuauUseCommittingTxnLog) - s.DEPRECATED_log.rollback(); - return s.errors; } @@ -1152,9 +914,6 @@ ErrorVec Unifier::canUnify(TypePackId subTy, TypePackId superTy, bool isFunction Unifier s = makeChildUnifier(); s.tryUnify_(subTy, superTy, isFunctionCall); - if (!FFlag::LuauUseCommittingTxnLog) - s.DEPRECATED_log.rollback(); - return s.errors; } @@ -1200,419 +959,207 @@ void Unifier::tryUnify_(TypePackId subTp, TypePackId superTp, bool isFunctionCal return; } - if (FFlag::LuauUseCommittingTxnLog) + superTp = log.follow(superTp); + subTp = log.follow(subTp); + + while (auto tp = log.getMutable(subTp)) { - superTp = log.follow(superTp); - subTp = log.follow(subTp); + if (tp->head.empty() && tp->tail) + subTp = log.follow(*tp->tail); + else + break; + } - while (auto tp = log.getMutable(subTp)) + while (auto tp = log.getMutable(superTp)) + { + if (tp->head.empty() && tp->tail) + superTp = log.follow(*tp->tail); + else + break; + } + + if (superTp == subTp) + return; + + if (FFlag::LuauTxnLogSeesTypePacks2 && log.haveSeen(superTp, subTp)) + return; + + if (log.getMutable(superTp)) + { + occursCheck(superTp, subTp); + + if (!log.getMutable(superTp)) { - if (tp->head.empty() && tp->tail) - subTp = log.follow(*tp->tail); - else - break; + log.replace(superTp, Unifiable::Bound(subTp)); } + } + else if (log.getMutable(subTp)) + { + occursCheck(subTp, superTp); - while (auto tp = log.getMutable(superTp)) + if (!log.getMutable(subTp)) { - if (tp->head.empty() && tp->tail) - superTp = log.follow(*tp->tail); - else - break; + log.replace(subTp, Unifiable::Bound(superTp)); } + } + else if (log.getMutable(superTp)) + tryUnifyWithAny(subTp, superTp); + else if (log.getMutable(subTp)) + tryUnifyWithAny(superTp, subTp); + else if (log.getMutable(superTp)) + tryUnifyVariadics(subTp, superTp, false); + else if (log.getMutable(subTp)) + tryUnifyVariadics(superTp, subTp, true); + else if (log.getMutable(superTp) && log.getMutable(subTp)) + { + auto superTpv = log.getMutable(superTp); + auto subTpv = log.getMutable(subTp); - if (superTp == subTp) - return; + // If the size of two heads does not match, but both packs have free tail + // We set the sentinel variable to say so to avoid growing it forever. + auto [superTypes, superTail] = logAwareFlatten(superTp, log); + auto [subTypes, subTail] = logAwareFlatten(subTp, log); - if (FFlag::LuauTxnLogSeesTypePacks2 && log.haveSeen(superTp, subTp)) - return; + bool noInfiniteGrowth = (superTypes.size() != subTypes.size()) && (superTail && log.getMutable(*superTail)) && + (subTail && log.getMutable(*subTail)); - if (log.getMutable(superTp)) + auto superIter = WeirdIter(superTp, log); + auto subIter = WeirdIter(subTp, log); + + auto mkFreshType = [this](TypeLevel level) { + return types->freshType(level); + }; + + const TypePackId emptyTp = types->addTypePack(TypePack{{}, std::nullopt}); + + int loopCount = 0; + + do { - occursCheck(superTp, subTp); + if (FInt::LuauTypeInferTypePackLoopLimit > 0 && loopCount >= FInt::LuauTypeInferTypePackLoopLimit) + ice("Detected possibly infinite TypePack growth"); - if (!log.getMutable(superTp)) + ++loopCount; + + if (superIter.good() && subIter.growing) { - log.replace(superTp, Unifiable::Bound(subTp)); + subIter.pushType(mkFreshType(subIter.level)); } - } - else if (log.getMutable(subTp)) - { - occursCheck(subTp, superTp); - if (!log.getMutable(subTp)) + if (subIter.good() && superIter.growing) { - log.replace(subTp, Unifiable::Bound(superTp)); + superIter.pushType(mkFreshType(superIter.level)); } - } - else if (log.getMutable(superTp)) - tryUnifyWithAny(subTp, superTp); - else if (log.getMutable(subTp)) - tryUnifyWithAny(superTp, subTp); - else if (log.getMutable(superTp)) - tryUnifyVariadics(subTp, superTp, false); - else if (log.getMutable(subTp)) - tryUnifyVariadics(superTp, subTp, true); - else if (log.getMutable(superTp) && log.getMutable(subTp)) - { - auto superTpv = log.getMutable(superTp); - auto subTpv = log.getMutable(subTp); - // If the size of two heads does not match, but both packs have free tail - // We set the sentinel variable to say so to avoid growing it forever. - auto [superTypes, superTail] = logAwareFlatten(superTp, log); - auto [subTypes, subTail] = logAwareFlatten(subTp, log); - - bool noInfiniteGrowth = (superTypes.size() != subTypes.size()) && (superTail && log.getMutable(*superTail)) && - (subTail && log.getMutable(*subTail)); - - auto superIter = WeirdIter(superTp, log); - auto subIter = WeirdIter(subTp, log); - - auto mkFreshType = [this](TypeLevel level) { - return types->freshType(level); - }; - - const TypePackId emptyTp = types->addTypePack(TypePack{{}, std::nullopt}); - - int loopCount = 0; - - do + if (superIter.good() && subIter.good()) { - if (FInt::LuauTypeInferTypePackLoopLimit > 0 && loopCount >= FInt::LuauTypeInferTypePackLoopLimit) - ice("Detected possibly infinite TypePack growth"); + tryUnify_(*subIter, *superIter); - ++loopCount; + if (!errors.empty() && !firstPackErrorPos) + firstPackErrorPos = loopCount; - if (superIter.good() && subIter.growing) + superIter.advance(); + subIter.advance(); + continue; + } + + // If both are at the end, we're done + if (!superIter.good() && !subIter.good()) + { + if (subTpv->tail && superTpv->tail) { - subIter.pushType(mkFreshType(subIter.level)); + tryUnify_(*subTpv->tail, *superTpv->tail); + break; } - if (subIter.good() && superIter.growing) + const bool lFreeTail = superTpv->tail && log.getMutable(log.follow(*superTpv->tail)) != nullptr; + const bool rFreeTail = subTpv->tail && log.getMutable(log.follow(*subTpv->tail)) != nullptr; + if (lFreeTail) + tryUnify_(emptyTp, *superTpv->tail); + else if (rFreeTail) + tryUnify_(emptyTp, *subTpv->tail); + + break; + } + + // If both tails are free, bind one to the other and call it a day + if (superIter.canGrow() && subIter.canGrow()) + return tryUnify_(*subIter.pack->tail, *superIter.pack->tail); + + // If just one side is free on its tail, grow it to fit the other side. + // FIXME: The tail-most tail of the growing pack should be the same as the tail-most tail of the non-growing pack. + if (superIter.canGrow()) + superIter.grow(types->addTypePack(TypePackVar(TypePack{}))); + else if (subIter.canGrow()) + subIter.grow(types->addTypePack(TypePackVar(TypePack{}))); + else + { + // A union type including nil marks an optional argument + if (superIter.good() && isOptional(*superIter)) { - superIter.pushType(mkFreshType(superIter.level)); - } - - if (superIter.good() && subIter.good()) - { - tryUnify_(*subIter, *superIter); - - if (!errors.empty() && !firstPackErrorPos) - firstPackErrorPos = loopCount; - superIter.advance(); + continue; + } + else if (subIter.good() && isOptional(*subIter)) + { subIter.advance(); continue; } - // If both are at the end, we're done - if (!superIter.good() && !subIter.good()) + // In nonstrict mode, any also marks an optional argument. + else if (superIter.good() && isNonstrictMode() && log.getMutable(log.follow(*superIter))) { - if (subTpv->tail && superTpv->tail) - { - tryUnify_(*subTpv->tail, *superTpv->tail); - break; - } - - const bool lFreeTail = superTpv->tail && log.getMutable(log.follow(*superTpv->tail)) != nullptr; - const bool rFreeTail = subTpv->tail && log.getMutable(log.follow(*subTpv->tail)) != nullptr; - if (lFreeTail) - tryUnify_(emptyTp, *superTpv->tail); - else if (rFreeTail) - tryUnify_(emptyTp, *subTpv->tail); - - break; + superIter.advance(); + continue; } - // If both tails are free, bind one to the other and call it a day - if (superIter.canGrow() && subIter.canGrow()) - return tryUnify_(*subIter.pack->tail, *superIter.pack->tail); - - // If just one side is free on its tail, grow it to fit the other side. - // FIXME: The tail-most tail of the growing pack should be the same as the tail-most tail of the non-growing pack. - if (superIter.canGrow()) - superIter.grow(types->addTypePack(TypePackVar(TypePack{}))); - else if (subIter.canGrow()) - subIter.grow(types->addTypePack(TypePackVar(TypePack{}))); - else + if (log.getMutable(superIter.packId)) { - // A union type including nil marks an optional argument - if (superIter.good() && isOptional(*superIter)) - { - superIter.advance(); - continue; - } - else if (subIter.good() && isOptional(*subIter)) - { - subIter.advance(); - continue; - } - - // In nonstrict mode, any also marks an optional argument. - else if (superIter.good() && isNonstrictMode() && log.getMutable(log.follow(*superIter))) - { - superIter.advance(); - continue; - } - - if (log.getMutable(superIter.packId)) - { - tryUnifyVariadics(subIter.packId, superIter.packId, false, int(subIter.index)); - return; - } - - if (log.getMutable(subIter.packId)) - { - tryUnifyVariadics(superIter.packId, subIter.packId, true, int(superIter.index)); - return; - } - - if (!isFunctionCall && subIter.good()) - { - // Sometimes it is ok to pass too many arguments - return; - } - - // This is a bit weird because we don't actually know expected vs actual. We just know - // subtype vs supertype. If we are checking the values returned by a function, we swap - // these to produce the expected error message. - size_t expectedSize = size(superTp); - size_t actualSize = size(subTp); - if (ctx == CountMismatch::Result) - std::swap(expectedSize, actualSize); - reportError(TypeError{location, CountMismatch{expectedSize, actualSize, ctx}}); - - while (superIter.good()) - { - tryUnify_(*superIter, getSingletonTypes().errorRecoveryType()); - superIter.advance(); - } - - while (subIter.good()) - { - tryUnify_(*subIter, getSingletonTypes().errorRecoveryType()); - subIter.advance(); - } - + tryUnifyVariadics(subIter.packId, superIter.packId, false, int(subIter.index)); return; } - } while (!noInfiniteGrowth); - } - else - { - reportError(TypeError{location, GenericError{"Failed to unify type packs"}}); - } + if (log.getMutable(subIter.packId)) + { + tryUnifyVariadics(superIter.packId, subIter.packId, true, int(superIter.index)); + return; + } + + if (!isFunctionCall && subIter.good()) + { + // Sometimes it is ok to pass too many arguments + return; + } + + // This is a bit weird because we don't actually know expected vs actual. We just know + // subtype vs supertype. If we are checking the values returned by a function, we swap + // these to produce the expected error message. + size_t expectedSize = size(superTp); + size_t actualSize = size(subTp); + if (ctx == CountMismatch::Result) + std::swap(expectedSize, actualSize); + reportError(TypeError{location, CountMismatch{expectedSize, actualSize, ctx}}); + + while (superIter.good()) + { + tryUnify_(*superIter, getSingletonTypes().errorRecoveryType()); + superIter.advance(); + } + + while (subIter.good()) + { + tryUnify_(*subIter, getSingletonTypes().errorRecoveryType()); + subIter.advance(); + } + + return; + } + + } while (!noInfiniteGrowth); } else { - superTp = follow(superTp); - subTp = follow(subTp); - - while (auto tp = get(subTp)) - { - if (tp->head.empty() && tp->tail) - subTp = follow(*tp->tail); - else - break; - } - - while (auto tp = get(superTp)) - { - if (tp->head.empty() && tp->tail) - superTp = follow(*tp->tail); - else - break; - } - - if (superTp == subTp) - return; - - if (FFlag::LuauTxnLogSeesTypePacks2 && DEPRECATED_log.haveSeen(superTp, subTp)) - return; - - if (get(superTp)) - { - occursCheck(superTp, subTp); - - if (!get(superTp)) - { - DEPRECATED_log(superTp); - *asMutable(superTp) = Unifiable::Bound(subTp); - } - } - else if (get(subTp)) - { - occursCheck(subTp, superTp); - - if (!get(subTp)) - { - DEPRECATED_log(subTp); - *asMutable(subTp) = Unifiable::Bound(superTp); - } - } - - else if (get(superTp)) - tryUnifyWithAny(subTp, superTp); - - else if (get(subTp)) - tryUnifyWithAny(superTp, subTp); - - else if (get(superTp)) - tryUnifyVariadics(subTp, superTp, false); - else if (get(subTp)) - tryUnifyVariadics(superTp, subTp, true); - - else if (get(superTp) && get(subTp)) - { - auto superTpv = get(superTp); - auto subTpv = get(subTp); - - // If the size of two heads does not match, but both packs have free tail - // We set the sentinel variable to say so to avoid growing it forever. - auto [superTypes, superTail] = flatten(superTp); - auto [subTypes, subTail] = flatten(subTp); - - bool noInfiniteGrowth = - (superTypes.size() != subTypes.size()) && (superTail && get(*superTail)) && (subTail && get(*subTail)); - - auto superIter = DEPRECATED_WeirdIter{superTp}; - auto subIter = DEPRECATED_WeirdIter{subTp}; - - auto mkFreshType = [this](TypeLevel level) { - return types->freshType(level); - }; - - const TypePackId emptyTp = types->addTypePack(TypePack{{}, std::nullopt}); - - int loopCount = 0; - - do - { - if (FInt::LuauTypeInferTypePackLoopLimit > 0 && loopCount >= FInt::LuauTypeInferTypePackLoopLimit) - ice("Detected possibly infinite TypePack growth"); - - ++loopCount; - - if (superIter.good() && subIter.growing) - asMutable(subIter.pack)->head.push_back(mkFreshType(subIter.level)); - - if (subIter.good() && superIter.growing) - asMutable(superIter.pack)->head.push_back(mkFreshType(superIter.level)); - - if (superIter.good() && subIter.good()) - { - tryUnify_(*subIter, *superIter); - - if (!errors.empty() && !firstPackErrorPos) - firstPackErrorPos = loopCount; - - superIter.advance(); - subIter.advance(); - continue; - } - - // If both are at the end, we're done - if (!superIter.good() && !subIter.good()) - { - if (subTpv->tail && superTpv->tail) - { - tryUnify_(*subTpv->tail, *superTpv->tail); - break; - } - - const bool lFreeTail = superTpv->tail && get(follow(*superTpv->tail)) != nullptr; - const bool rFreeTail = subTpv->tail && get(follow(*subTpv->tail)) != nullptr; - if (lFreeTail) - tryUnify_(emptyTp, *superTpv->tail); - else if (rFreeTail) - tryUnify_(emptyTp, *subTpv->tail); - - break; - } - - // If both tails are free, bind one to the other and call it a day - if (superIter.canGrow() && subIter.canGrow()) - return tryUnify_(*subIter.pack->tail, *superIter.pack->tail); - - // If just one side is free on its tail, grow it to fit the other side. - // FIXME: The tail-most tail of the growing pack should be the same as the tail-most tail of the non-growing pack. - if (superIter.canGrow()) - superIter.grow(types->addTypePack(TypePackVar(TypePack{}))); - - else if (subIter.canGrow()) - subIter.grow(types->addTypePack(TypePackVar(TypePack{}))); - - else - { - // A union type including nil marks an optional argument - if (superIter.good() && isOptional(*superIter)) - { - superIter.advance(); - continue; - } - else if (subIter.good() && isOptional(*subIter)) - { - subIter.advance(); - continue; - } - - // In nonstrict mode, any also marks an optional argument. - else if (superIter.good() && isNonstrictMode() && get(follow(*superIter))) - { - superIter.advance(); - continue; - } - - if (get(superIter.packId)) - { - tryUnifyVariadics(subIter.packId, superIter.packId, false, int(subIter.index)); - return; - } - - if (get(subIter.packId)) - { - tryUnifyVariadics(superIter.packId, subIter.packId, true, int(superIter.index)); - return; - } - - if (!isFunctionCall && subIter.good()) - { - // Sometimes it is ok to pass too many arguments - return; - } - - // This is a bit weird because we don't actually know expected vs actual. We just know - // subtype vs supertype. If we are checking the values returned by a function, we swap - // these to produce the expected error message. - size_t expectedSize = size(superTp); - size_t actualSize = size(subTp); - if (ctx == CountMismatch::Result) - std::swap(expectedSize, actualSize); - reportError(TypeError{location, CountMismatch{expectedSize, actualSize, ctx}}); - - while (superIter.good()) - { - tryUnify_(*superIter, getSingletonTypes().errorRecoveryType()); - superIter.advance(); - } - - while (subIter.good()) - { - tryUnify_(*subIter, getSingletonTypes().errorRecoveryType()); - subIter.advance(); - } - - return; - } - - } while (!noInfiniteGrowth); - } - else - { - reportError(TypeError{location, GenericError{"Failed to unify type packs"}}); - } + reportError(TypeError{location, GenericError{"Failed to unify type packs"}}); } } @@ -1650,14 +1197,8 @@ void Unifier::tryUnifySingletons(TypeId subTy, TypeId superTy) void Unifier::tryUnifyFunctions(TypeId subTy, TypeId superTy, bool isFunctionCall) { - FunctionTypeVar* superFunction = getMutable(superTy); - FunctionTypeVar* subFunction = getMutable(subTy); - - if (FFlag::LuauUseCommittingTxnLog) - { - superFunction = log.getMutable(superTy); - subFunction = log.getMutable(subTy); - } + FunctionTypeVar* superFunction = log.getMutable(superTy); + FunctionTypeVar* subFunction = log.getMutable(subTy); if (!superFunction || !subFunction) ice("passed non-function types to unifyFunction"); @@ -1680,20 +1221,14 @@ void Unifier::tryUnifyFunctions(TypeId subTy, TypeId superTy, bool isFunctionCal for (size_t i = 0; i < numGenerics; i++) { - if (FFlag::LuauUseCommittingTxnLog) - log.pushSeen(superFunction->generics[i], subFunction->generics[i]); - else - DEPRECATED_log.pushSeen(superFunction->generics[i], subFunction->generics[i]); + log.pushSeen(superFunction->generics[i], subFunction->generics[i]); } if (FFlag::LuauTxnLogSeesTypePacks2) { for (size_t i = 0; i < numGenericPacks; i++) { - if (FFlag::LuauUseCommittingTxnLog) - log.pushSeen(superFunction->genericPacks[i], subFunction->genericPacks[i]); - else - DEPRECATED_log.pushSeen(superFunction->genericPacks[i], subFunction->genericPacks[i]); + log.pushSeen(superFunction->genericPacks[i], subFunction->genericPacks[i]); } } @@ -1734,14 +1269,7 @@ void Unifier::tryUnifyFunctions(TypeId subTy, TypeId superTy, bool isFunctionCal reportError(TypeError{location, TypeMismatch{superTy, subTy, "", innerState.errors.front()}}); } - if (FFlag::LuauUseCommittingTxnLog) - { - log.concat(std::move(innerState.log)); - } - else - { - DEPRECATED_log.concat(std::move(innerState.DEPRECATED_log)); - } + log.concat(std::move(innerState.log)); } else { @@ -1754,33 +1282,19 @@ void Unifier::tryUnifyFunctions(TypeId subTy, TypeId superTy, bool isFunctionCal if (!FFlag::LuauImmutableTypes) { - if (FFlag::LuauUseCommittingTxnLog) + if (superFunction->definition && !subFunction->definition && !subTy->persistent) { - if (superFunction->definition && !subFunction->definition && !subTy->persistent) - { - PendingType* newSubTy = log.queue(subTy); - FunctionTypeVar* newSubFtv = getMutable(newSubTy); - LUAU_ASSERT(newSubFtv); - newSubFtv->definition = superFunction->definition; - } - else if (!superFunction->definition && subFunction->definition && !superTy->persistent) - { - PendingType* newSuperTy = log.queue(superTy); - FunctionTypeVar* newSuperFtv = getMutable(newSuperTy); - LUAU_ASSERT(newSuperFtv); - newSuperFtv->definition = subFunction->definition; - } + PendingType* newSubTy = log.queue(subTy); + FunctionTypeVar* newSubFtv = getMutable(newSubTy); + LUAU_ASSERT(newSubFtv); + newSubFtv->definition = superFunction->definition; } - else + else if (!superFunction->definition && subFunction->definition && !superTy->persistent) { - if (superFunction->definition && !subFunction->definition && !subTy->persistent) - { - subFunction->definition = superFunction->definition; - } - else if (!superFunction->definition && subFunction->definition && !superTy->persistent) - { - superFunction->definition = subFunction->definition; - } + PendingType* newSuperTy = log.queue(superTy); + FunctionTypeVar* newSuperFtv = getMutable(newSuperTy); + LUAU_ASSERT(newSuperFtv); + newSuperFtv->definition = subFunction->definition; } } @@ -1790,19 +1304,13 @@ void Unifier::tryUnifyFunctions(TypeId subTy, TypeId superTy, bool isFunctionCal { for (int i = int(numGenericPacks) - 1; 0 <= i; i--) { - if (FFlag::LuauUseCommittingTxnLog) - log.popSeen(superFunction->genericPacks[i], subFunction->genericPacks[i]); - else - DEPRECATED_log.popSeen(superFunction->genericPacks[i], subFunction->genericPacks[i]); + log.popSeen(superFunction->genericPacks[i], subFunction->genericPacks[i]); } } for (int i = int(numGenerics) - 1; 0 <= i; i--) { - if (FFlag::LuauUseCommittingTxnLog) - log.popSeen(superFunction->generics[i], subFunction->generics[i]); - else - DEPRECATED_log.popSeen(superFunction->generics[i], subFunction->generics[i]); + log.popSeen(superFunction->generics[i], subFunction->generics[i]); } } @@ -1833,14 +1341,8 @@ void Unifier::tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) if (!FFlag::LuauTableSubtypingVariance2) return DEPRECATED_tryUnifyTables(subTy, superTy, isIntersection); - TableTypeVar* superTable = getMutable(superTy); - TableTypeVar* subTable = getMutable(subTy); - - if (FFlag::LuauUseCommittingTxnLog) - { - superTable = log.getMutable(superTy); - subTable = log.getMutable(subTy); - } + TableTypeVar* superTable = log.getMutable(superTy); + TableTypeVar* subTable = log.getMutable(subTy); if (!superTable || !subTable) ice("passed non-table types to unifyTables"); @@ -1855,8 +1357,7 @@ void Unifier::tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) { auto subIter = subTable->props.find(propName); - bool isAny = - FFlag::LuauUseCommittingTxnLog ? log.getMutable(log.follow(superProp.type)) : get(follow(superProp.type)); + bool isAny = log.getMutable(log.follow(superProp.type)); if (subIter == subTable->props.end() && (!FFlag::LuauSubtypingAddOptPropsToUnsealedTables || subTable->state == TableState::Unsealed) && !isOptional(superProp.type) && !isAny) missingProperties.push_back(propName); @@ -1877,8 +1378,7 @@ void Unifier::tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) { auto superIter = superTable->props.find(propName); - bool isAny = - FFlag::LuauUseCommittingTxnLog ? log.getMutable(log.follow(subProp.type)) : get(follow(subProp.type)); + bool isAny = log.is(log.follow(subProp.type)); if (superIter == superTable->props.end() && (FFlag::LuauSubtypingAddOptPropsToUnsealedTables || (!isOptional(subProp.type) && !isAny))) extraProperties.push_back(propName); } @@ -1906,18 +1406,8 @@ void Unifier::tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) checkChildUnifierTypeMismatch(innerState.errors, name, superTy, subTy); - if (FFlag::LuauUseCommittingTxnLog) - { - if (innerState.errors.empty()) - log.concat(std::move(innerState.log)); - } - else - { - if (innerState.errors.empty()) - DEPRECATED_log.concat(std::move(innerState.DEPRECATED_log)); - else - innerState.DEPRECATED_log.rollback(); - } + if (innerState.errors.empty()) + log.concat(std::move(innerState.log)); } else if (subTable->indexer && maybeString(subTable->indexer->indexType)) { @@ -1931,18 +1421,8 @@ void Unifier::tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) checkChildUnifierTypeMismatch(innerState.errors, name, superTy, subTy); - if (FFlag::LuauUseCommittingTxnLog) - { - if (innerState.errors.empty()) - log.concat(std::move(innerState.log)); - } - else - { - if (innerState.errors.empty()) - DEPRECATED_log.concat(std::move(innerState.DEPRECATED_log)); - else - innerState.DEPRECATED_log.rollback(); - } + if (innerState.errors.empty()) + log.concat(std::move(innerState.log)); } else if ((!FFlag::LuauSubtypingAddOptPropsToUnsealedTables || subTable->state == TableState::Unsealed) && (isOptional(prop.type) || get(follow(prop.type)))) // This is sound because unsealed table types are precise, so `{ p : T } <: { p : T, q : U? }` @@ -1953,22 +1433,30 @@ void Unifier::tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) } else if (subTable->state == TableState::Free) { - if (FFlag::LuauUseCommittingTxnLog) - { - PendingType* pendingSub = log.queue(subTy); - TableTypeVar* ttv = getMutable(pendingSub); - LUAU_ASSERT(ttv); - ttv->props[name] = prop; - subTable = ttv; - } - else - { - DEPRECATED_log(subTy); - subTable->props[name] = prop; - } + PendingType* pendingSub = log.queue(subTy); + TableTypeVar* ttv = getMutable(pendingSub); + LUAU_ASSERT(ttv); + ttv->props[name] = prop; + subTable = ttv; } else missingProperties.push_back(name); + + if (FFlag::LuauTxnLogCheckForInvalidation) + { + // Recursive unification can change the txn log, and invalidate the old + // table. If we detect that this has happened, we start over, with the updated + // txn log. + TableTypeVar* newSuperTable = log.getMutable(superTy); + TableTypeVar* newSubTable = log.getMutable(subTy); + if (superTable != newSuperTable || subTable != newSubTable) + { + if (errors.empty()) + return tryUnifyTables(subTy, superTy, isIntersection); + else + return; + } + } } for (const auto& [name, prop] : subTable->props) @@ -1990,18 +1478,8 @@ void Unifier::tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) checkChildUnifierTypeMismatch(innerState.errors, name, superTy, subTy); - if (FFlag::LuauUseCommittingTxnLog) - { - if (innerState.errors.empty()) - log.concat(std::move(innerState.log)); - } - else - { - if (innerState.errors.empty()) - DEPRECATED_log.concat(std::move(innerState.DEPRECATED_log)); - else - innerState.DEPRECATED_log.rollback(); - } + if (innerState.errors.empty()) + log.concat(std::move(innerState.log)); } else if (superTable->state == TableState::Unsealed) { @@ -2011,18 +1489,10 @@ void Unifier::tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) Property clone = prop; clone.type = deeplyOptional(clone.type); - if (FFlag::LuauUseCommittingTxnLog) - { - PendingType* pendingSuper = log.queue(superTy); - TableTypeVar* pendingSuperTtv = getMutable(pendingSuper); - pendingSuperTtv->props[name] = clone; - superTable = pendingSuperTtv; - } - else - { - DEPRECATED_log(superTy); - superTable->props[name] = clone; - } + PendingType* pendingSuper = log.queue(superTy); + TableTypeVar* pendingSuperTtv = getMutable(pendingSuper); + pendingSuperTtv->props[name] = clone; + superTable = pendingSuperTtv; } else if (variance == Covariant) { @@ -2032,21 +1502,29 @@ void Unifier::tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) } else if (superTable->state == TableState::Free) { - if (FFlag::LuauUseCommittingTxnLog) - { - PendingType* pendingSuper = log.queue(superTy); - TableTypeVar* pendingSuperTtv = getMutable(pendingSuper); - pendingSuperTtv->props[name] = prop; - superTable = pendingSuperTtv; - } - else - { - DEPRECATED_log(superTy); - superTable->props[name] = prop; - } + PendingType* pendingSuper = log.queue(superTy); + TableTypeVar* pendingSuperTtv = getMutable(pendingSuper); + pendingSuperTtv->props[name] = prop; + superTable = pendingSuperTtv; } else extraProperties.push_back(name); + + if (FFlag::LuauTxnLogCheckForInvalidation) + { + // Recursive unification can change the txn log, and invalidate the old + // table. If we detect that this has happened, we start over, with the updated + // txn log. + TableTypeVar* newSuperTable = log.getMutable(superTy); + TableTypeVar* newSubTable = log.getMutable(subTy); + if (superTable != newSuperTable || subTable != newSubTable) + { + if (errors.empty()) + return tryUnifyTables(subTy, superTy, isIntersection); + else + return; + } + } } // Unify indexers @@ -2060,18 +1538,8 @@ void Unifier::tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) innerState.tryUnifyIndexer(*subTable->indexer, *superTable->indexer); checkChildUnifierTypeMismatch(innerState.errors, superTy, subTy); - if (FFlag::LuauUseCommittingTxnLog) - { - if (innerState.errors.empty()) - log.concat(std::move(innerState.log)); - } - else - { - if (innerState.errors.empty()) - DEPRECATED_log.concat(std::move(innerState.DEPRECATED_log)); - else - innerState.DEPRECATED_log.rollback(); - } + if (innerState.errors.empty()) + log.concat(std::move(innerState.log)); } else if (superTable->indexer) { @@ -2081,15 +1549,7 @@ void Unifier::tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) // e.g. table.insert(t, 1) where t is a non-sealed table and doesn't have an indexer. // TODO: we only need to do this if the supertype's indexer is read/write // since that can add indexed elements. - if (FFlag::LuauUseCommittingTxnLog) - { - log.changeIndexer(subTy, superTable->indexer); - } - else - { - DEPRECATED_log(subTy); - subTable->indexer = superTable->indexer; - } + log.changeIndexer(subTy, superTable->indexer); } } else if (subTable->indexer && variance == Invariant) @@ -2097,15 +1557,29 @@ void Unifier::tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) // Symmetric if we are invariant if (superTable->state == TableState::Unsealed || superTable->state == TableState::Free) { - if (FFlag::LuauUseCommittingTxnLog) - { - log.changeIndexer(superTy, subTable->indexer); - } + log.changeIndexer(superTy, subTable->indexer); + } + } + + if (FFlag::LuauTxnLogDontRetryForIndexers) + { + // Changing the indexer can invalidate the table pointers. + superTable = log.getMutable(superTy); + subTable = log.getMutable(subTy); + } + else if (FFlag::LuauTxnLogCheckForInvalidation) + { + // Recursive unification can change the txn log, and invalidate the old + // table. If we detect that this has happened, we start over, with the updated + // txn log. + TableTypeVar* newSuperTable = log.getMutable(superTy); + TableTypeVar* newSubTable = log.getMutable(subTy); + if (superTable != newSuperTable || subTable != newSubTable) + { + if (errors.empty()) + return tryUnifyTables(subTy, superTy, isIntersection); else - { - DEPRECATED_log(superTy); - superTable->indexer = subTable->indexer; - } + return; } } @@ -2134,27 +1608,11 @@ void Unifier::tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) if (superTable->state == TableState::Free) { - if (FFlag::LuauUseCommittingTxnLog) - { - log.bindTable(superTy, subTy); - } - else - { - DEPRECATED_log(superTable); - superTable->boundTo = subTy; - } + log.bindTable(superTy, subTy); } else if (subTable->state == TableState::Free) { - if (FFlag::LuauUseCommittingTxnLog) - { - log.bindTable(subTy, superTy); - } - else - { - DEPRECATED_log(subTable); - subTable->boundTo = superTy; - } + log.bindTable(subTy, superTy); } } @@ -2197,14 +1655,8 @@ void Unifier::DEPRECATED_tryUnifyTables(TypeId subTy, TypeId superTy, bool isInt Resetter resetter{&variance}; variance = Invariant; - TableTypeVar* superTable = getMutable(superTy); - TableTypeVar* subTable = getMutable(subTy); - - if (FFlag::LuauUseCommittingTxnLog) - { - superTable = log.getMutable(superTy); - subTable = log.getMutable(subTy); - } + TableTypeVar* superTable = log.getMutable(superTy); + TableTypeVar* subTable = log.getMutable(subTy); if (!superTable || !subTable) ice("passed non-table types to unifyTables"); @@ -2231,15 +1683,7 @@ void Unifier::DEPRECATED_tryUnifyTables(TypeId subTy, TypeId superTy, bool isInt // avoid creating a cycle when the types are already pointing at each other if (follow(superTy) != follow(subTy)) { - if (FFlag::LuauUseCommittingTxnLog) - { - log.bindTable(superTy, subTy); - } - else - { - DEPRECATED_log(superTable); - superTable->boundTo = subTy; - } + log.bindTable(superTy, subTy); } return; } @@ -2268,14 +1712,7 @@ void Unifier::DEPRECATED_tryUnifyTables(TypeId subTy, TypeId superTy, bool isInt // e.g. table.insert(t, 1) where t is a non-sealed table and doesn't have an indexer. if (subTable->state == TableState::Unsealed) { - if (FFlag::LuauUseCommittingTxnLog) - { - log.changeIndexer(subTy, superTable->indexer); - } - else - { - subTable->indexer = superTable->indexer; - } + log.changeIndexer(subTy, superTable->indexer); } else reportError(TypeError{location, CannotExtendTable{subTy, CannotExtendTable::Indexer}}); @@ -2295,14 +1732,8 @@ void Unifier::DEPRECATED_tryUnifyTables(TypeId subTy, TypeId superTy, bool isInt void Unifier::tryUnifyFreeTable(TypeId subTy, TypeId superTy) { - TableTypeVar* freeTable = getMutable(superTy); - TableTypeVar* subTable = getMutable(subTy); - - if (FFlag::LuauUseCommittingTxnLog) - { - freeTable = log.getMutable(superTy); - subTable = log.getMutable(subTy); - } + TableTypeVar* freeTable = log.getMutable(superTy); + TableTypeVar* subTable = log.getMutable(subTy); if (!freeTable || !subTable) ice("passed non-table types to tryUnifyFreeTable"); @@ -2323,22 +1754,11 @@ void Unifier::tryUnifyFreeTable(TypeId subTy, TypeId superTy) * I believe this is guaranteed to terminate eventually because this will * only happen when a free table is bound to another table. */ - if (FFlag::LuauUseCommittingTxnLog) - { - if (!log.getMutable(superTy) || !log.getMutable(subTy)) - return tryUnify_(subTy, superTy); + if (!log.getMutable(superTy) || !log.getMutable(subTy)) + return tryUnify_(subTy, superTy); - if (TableTypeVar* pendingFreeTtv = log.getMutable(superTy); pendingFreeTtv && pendingFreeTtv->boundTo) - return tryUnify_(subTy, superTy); - } - else - { - if (!get(superTy) || !get(subTy)) - return tryUnify_(subTy, superTy); - - if (freeTable->boundTo) - return tryUnify_(subTy, superTy); - } + if (TableTypeVar* pendingFreeTtv = log.getMutable(superTy); pendingFreeTtv && pendingFreeTtv->boundTo) + return tryUnify_(subTy, superTy); } else { @@ -2346,17 +1766,10 @@ void Unifier::tryUnifyFreeTable(TypeId subTy, TypeId superTy) // properties than we previously thought. Else, it is an error. if (subTable->state == TableState::Free) { - if (FFlag::LuauUseCommittingTxnLog) - { - PendingType* pendingSub = log.queue(subTy); - TableTypeVar* pendingSubTtv = getMutable(pendingSub); - LUAU_ASSERT(pendingSubTtv); - pendingSubTtv->props.insert({freeName, freeProp}); - } - else - { - subTable->props.insert({freeName, freeProp}); - } + PendingType* pendingSub = log.queue(subTy); + TableTypeVar* pendingSubTtv = getMutable(pendingSub); + LUAU_ASSERT(pendingSubTtv); + pendingSubTtv->props.insert({freeName, freeProp}); } else reportError(TypeError{location, UnknownProperty{subTy, freeName}}); @@ -2370,47 +1783,23 @@ void Unifier::tryUnifyFreeTable(TypeId subTy, TypeId superTy) checkChildUnifierTypeMismatch(innerState.errors, superTy, subTy); - if (FFlag::LuauUseCommittingTxnLog) - log.concat(std::move(innerState.log)); - else - DEPRECATED_log.concat(std::move(innerState.DEPRECATED_log)); + log.concat(std::move(innerState.log)); } else if (subTable->state == TableState::Free && freeTable->indexer) { - if (FFlag::LuauUseCommittingTxnLog) - { - log.changeIndexer(superTy, subTable->indexer); - } - else - { - freeTable->indexer = subTable->indexer; - } + log.changeIndexer(superTy, subTable->indexer); } if (!freeTable->boundTo && subTable->state != TableState::Free) { - if (FFlag::LuauUseCommittingTxnLog) - { - log.bindTable(superTy, subTy); - } - else - { - DEPRECATED_log(freeTable); - freeTable->boundTo = subTy; - } + log.bindTable(superTy, subTy); } } void Unifier::tryUnifySealedTables(TypeId subTy, TypeId superTy, bool isIntersection) { - TableTypeVar* superTable = getMutable(superTy); - TableTypeVar* subTable = getMutable(subTy); - - if (FFlag::LuauUseCommittingTxnLog) - { - superTable = log.getMutable(superTy); - subTable = log.getMutable(subTy); - } + TableTypeVar* superTable = log.getMutable(superTy); + TableTypeVar* subTable = log.getMutable(subTy); if (!superTable || !subTable) ice("passed non-table types to unifySealedTables"); @@ -2476,77 +1865,39 @@ void Unifier::tryUnifySealedTables(TypeId subTy, TypeId superTy, bool isIntersec if (superTable->indexer || subTable->indexer) { - if (FFlag::LuauUseCommittingTxnLog) + if (superTable->indexer && subTable->indexer) + innerState.tryUnifyIndexer(*subTable->indexer, *superTable->indexer); + else if (subTable->state == TableState::Unsealed) { - if (superTable->indexer && subTable->indexer) - innerState.tryUnifyIndexer(*subTable->indexer, *superTable->indexer); - else if (subTable->state == TableState::Unsealed) + if (superTable->indexer && !subTable->indexer) { - if (superTable->indexer && !subTable->indexer) - { - log.changeIndexer(subTy, superTable->indexer); - } + log.changeIndexer(subTy, superTable->indexer); } - else if (superTable->state == TableState::Unsealed) + } + else if (superTable->state == TableState::Unsealed) + { + if (subTable->indexer && !superTable->indexer) { - if (subTable->indexer && !superTable->indexer) - { - log.changeIndexer(superTy, subTable->indexer); - } + log.changeIndexer(superTy, subTable->indexer); } - else if (superTable->indexer) + } + else if (superTable->indexer) + { + innerState.tryUnify_(getSingletonTypes().stringType, superTable->indexer->indexType); + for (const auto& [name, type] : subTable->props) { - innerState.tryUnify_(getSingletonTypes().stringType, superTable->indexer->indexType); - for (const auto& [name, type] : subTable->props) - { - const auto& it = superTable->props.find(name); - if (it == superTable->props.end()) - innerState.tryUnify_(type.type, superTable->indexer->indexResultType); - } + const auto& it = superTable->props.find(name); + if (it == superTable->props.end()) + innerState.tryUnify_(type.type, superTable->indexer->indexResultType); } - else - innerState.reportError(TypeError{location, TypeMismatch{superTy, subTy}}); } else - { - if (superTable->indexer && subTable->indexer) - innerState.tryUnifyIndexer(*subTable->indexer, *superTable->indexer); - else if (subTable->state == TableState::Unsealed) - { - if (superTable->indexer && !subTable->indexer) - subTable->indexer = superTable->indexer; - } - else if (superTable->state == TableState::Unsealed) - { - if (subTable->indexer && !superTable->indexer) - superTable->indexer = subTable->indexer; - } - else if (superTable->indexer) - { - innerState.tryUnify_(getSingletonTypes().stringType, superTable->indexer->indexType); - // We already try to unify properties in both tables. - // Skip those and just look for the ones remaining and see if they fit into the indexer. - for (const auto& [name, type] : subTable->props) - { - const auto& it = superTable->props.find(name); - if (it == superTable->props.end()) - innerState.tryUnify_(type.type, superTable->indexer->indexResultType); - } - } - else - innerState.reportError(TypeError{location, TypeMismatch{superTy, subTy}}); - } + innerState.reportError(TypeError{location, TypeMismatch{superTy, subTy}}); } - if (FFlag::LuauUseCommittingTxnLog) - { - if (!errorReported) - log.concat(std::move(innerState.log)); - } + if (!errorReported) + log.concat(std::move(innerState.log)); else - DEPRECATED_log.concat(std::move(innerState.DEPRECATED_log)); - - if (errorReported) return; if (!missingPropertiesInSuper.empty()) @@ -2594,8 +1945,7 @@ void Unifier::tryUnifyWithMetatable(TypeId subTy, TypeId superTy, bool reversed) TypeError mismatchError = TypeError{location, TypeMismatch{reversed ? subTy : superTy, reversed ? superTy : subTy}}; - if (const MetatableTypeVar* subMetatable = - FFlag::LuauUseCommittingTxnLog ? log.getMutable(subTy) : get(subTy)) + if (const MetatableTypeVar* subMetatable = log.getMutable(subTy)) { Unifier innerState = makeChildUnifier(); innerState.tryUnify_(subMetatable->table, superMetatable->table); @@ -2606,27 +1956,16 @@ void Unifier::tryUnifyWithMetatable(TypeId subTy, TypeId superTy, bool reversed) else if (!innerState.errors.empty()) reportError(TypeError{location, TypeMismatch{reversed ? subTy : superTy, reversed ? superTy : subTy, "", innerState.errors.front()}}); - if (FFlag::LuauUseCommittingTxnLog) - log.concat(std::move(innerState.log)); - else - DEPRECATED_log.concat(std::move(innerState.DEPRECATED_log)); + log.concat(std::move(innerState.log)); } - else if (TableTypeVar* subTable = FFlag::LuauUseCommittingTxnLog ? log.getMutable(subTy) : getMutable(subTy)) + else if (TableTypeVar* subTable = log.getMutable(subTy)) { switch (subTable->state) { case TableState::Free: { tryUnify_(subTy, superMetatable->table); - - if (FFlag::LuauUseCommittingTxnLog) - { - log.bindTable(subTy, superTy); - } - else - { - subTable->boundTo = superTy; - } + log.bindTable(subTy, superTy); break; } @@ -2637,8 +1976,7 @@ void Unifier::tryUnifyWithMetatable(TypeId subTy, TypeId superTy, bool reversed) reportError(mismatchError); } } - else if (FFlag::LuauUseCommittingTxnLog ? (log.getMutable(subTy) || log.getMutable(subTy)) - : (get(subTy) || get(subTy))) + else if (log.getMutable(subTy) || log.getMutable(subTy)) { } else @@ -2711,28 +2049,13 @@ void Unifier::tryUnifyWithClass(TypeId subTy, TypeId superTy, bool reversed) checkChildUnifierTypeMismatch(innerState.errors, propName, reversed ? subTy : superTy, reversed ? superTy : subTy); - if (FFlag::LuauUseCommittingTxnLog) + if (innerState.errors.empty()) { - if (innerState.errors.empty()) - { - log.concat(std::move(innerState.log)); - } - else - { - ok = false; - } + log.concat(std::move(innerState.log)); } else { - if (innerState.errors.empty()) - { - DEPRECATED_log.concat(std::move(innerState.DEPRECATED_log)); - } - else - { - ok = false; - innerState.DEPRECATED_log.rollback(); - } + ok = false; } } } @@ -2747,15 +2070,7 @@ void Unifier::tryUnifyWithClass(TypeId subTy, TypeId superTy, bool reversed) if (!ok) return; - if (FFlag::LuauUseCommittingTxnLog) - { - log.bindTable(subTy, superTy); - } - else - { - DEPRECATED_log(subTable); - subTable->boundTo = superTy; - } + log.bindTable(subTy, superTy); } else return fail(); @@ -2771,54 +2086,30 @@ static void queueTypePack(std::vector& queue, DenseHashSet& { while (true) { - a = FFlag::LuauFollowWithCommittingTxnLogInAnyUnification ? state.log.follow(a) : follow(a); + a = state.log.follow(a); if (seenTypePacks.find(a)) break; seenTypePacks.insert(a); - if (FFlag::LuauUseCommittingTxnLog) + if (state.log.getMutable(a)) { - if (state.log.getMutable(a)) - { - state.log.replace(a, Unifiable::Bound{anyTypePack}); - } - else if (auto tp = state.log.getMutable(a)) - { - queue.insert(queue.end(), tp->head.begin(), tp->head.end()); - if (tp->tail) - a = *tp->tail; - else - break; - } + state.log.replace(a, Unifiable::Bound{anyTypePack}); } - else + else if (auto tp = state.log.getMutable(a)) { - if (get(a)) - { - state.DEPRECATED_log(a); - *asMutable(a) = Unifiable::Bound{anyTypePack}; - } - else if (auto tp = get(a)) - { - queue.insert(queue.end(), tp->head.begin(), tp->head.end()); - if (tp->tail) - a = *tp->tail; - else - break; - } + queue.insert(queue.end(), tp->head.begin(), tp->head.end()); + if (tp->tail) + a = *tp->tail; + else + break; } } } void Unifier::tryUnifyVariadics(TypePackId subTp, TypePackId superTp, bool reversed, int subOffset) { - const VariadicTypePack* superVariadic = get(superTp); - - if (FFlag::LuauUseCommittingTxnLog) - { - superVariadic = log.getMutable(superTp); - } + const VariadicTypePack* superVariadic = log.getMutable(superTp); if (!superVariadic) ice("passed non-variadic pack to tryUnifyVariadics"); @@ -2843,15 +2134,7 @@ void Unifier::tryUnifyVariadics(TypePackId subTp, TypePackId superTp, bool rever TypePackId tail = follow(*maybeTail); if (get(tail)) { - if (FFlag::LuauUseCommittingTxnLog) - { - log.replace(tail, BoundTypePack(superTp)); - } - else - { - DEPRECATED_log(tail); - *asMutable(tail) = BoundTypePack{superTp}; - } + log.replace(tail, BoundTypePack(superTp)); } else if (const VariadicTypePack* vtp = get(tail)) { @@ -2882,103 +2165,54 @@ static void tryUnifyWithAny(std::vector& queue, Unifier& state, DenseHas { while (!queue.empty()) { - if (FFlag::LuauUseCommittingTxnLog) + TypeId ty = state.log.follow(queue.back()); + queue.pop_back(); + + // Types from other modules don't have free types + if (FFlag::LuauImmutableTypes && ty->owningArena != typeArena) + continue; + + if (seen.find(ty)) + continue; + + seen.insert(ty); + + if (state.log.getMutable(ty)) { - TypeId ty = state.log.follow(queue.back()); - queue.pop_back(); - - // Types from other modules don't have free types - if (FFlag::LuauImmutableTypes && ty->owningArena != typeArena) - continue; - - if (seen.find(ty)) - continue; - - seen.insert(ty); - - if (state.log.getMutable(ty)) - { - state.log.replace(ty, BoundTypeVar{anyType}); - } - else if (auto fun = state.log.getMutable(ty)) - { - queueTypePack(queue, seenTypePacks, state, fun->argTypes, anyTypePack); - queueTypePack(queue, seenTypePacks, state, fun->retType, anyTypePack); - } - else if (auto table = state.log.getMutable(ty)) - { - for (const auto& [_name, prop] : table->props) - queue.push_back(prop.type); - - if (table->indexer) - { - queue.push_back(table->indexer->indexType); - queue.push_back(table->indexer->indexResultType); - } - } - else if (auto mt = state.log.getMutable(ty)) - { - queue.push_back(mt->table); - queue.push_back(mt->metatable); - } - else if (state.log.getMutable(ty)) - { - // ClassTypeVars never contain free typevars. - } - else if (auto union_ = state.log.getMutable(ty)) - queue.insert(queue.end(), union_->options.begin(), union_->options.end()); - else if (auto intersection = state.log.getMutable(ty)) - queue.insert(queue.end(), intersection->parts.begin(), intersection->parts.end()); - else - { - } // Primitives, any, errors, and generics are left untouched. + state.log.replace(ty, BoundTypeVar{anyType}); } + else if (auto fun = state.log.getMutable(ty)) + { + queueTypePack(queue, seenTypePacks, state, fun->argTypes, anyTypePack); + queueTypePack(queue, seenTypePacks, state, fun->retType, anyTypePack); + } + else if (auto table = state.log.getMutable(ty)) + { + for (const auto& [_name, prop] : table->props) + queue.push_back(prop.type); + + if (table->indexer) + { + queue.push_back(table->indexer->indexType); + queue.push_back(table->indexer->indexResultType); + } + } + else if (auto mt = state.log.getMutable(ty)) + { + queue.push_back(mt->table); + queue.push_back(mt->metatable); + } + else if (state.log.getMutable(ty)) + { + // ClassTypeVars never contain free typevars. + } + else if (auto union_ = state.log.getMutable(ty)) + queue.insert(queue.end(), union_->options.begin(), union_->options.end()); + else if (auto intersection = state.log.getMutable(ty)) + queue.insert(queue.end(), intersection->parts.begin(), intersection->parts.end()); else { - TypeId ty = follow(queue.back()); - queue.pop_back(); - if (seen.find(ty)) - continue; - seen.insert(ty); - - if (get(ty)) - { - state.DEPRECATED_log(ty); - *asMutable(ty) = BoundTypeVar{anyType}; - } - else if (auto fun = get(ty)) - { - queueTypePack(queue, seenTypePacks, state, fun->argTypes, anyTypePack); - queueTypePack(queue, seenTypePacks, state, fun->retType, anyTypePack); - } - else if (auto table = get(ty)) - { - for (const auto& [_name, prop] : table->props) - queue.push_back(prop.type); - - if (table->indexer) - { - queue.push_back(table->indexer->indexType); - queue.push_back(table->indexer->indexResultType); - } - } - else if (auto mt = get(ty)) - { - queue.push_back(mt->table); - queue.push_back(mt->metatable); - } - else if (get(ty)) - { - // ClassTypeVars never contain free typevars. - } - else if (auto union_ = get(ty)) - queue.insert(queue.end(), union_->options.begin(), union_->options.end()); - else if (auto intersection = get(ty)) - queue.insert(queue.end(), intersection->parts.begin(), intersection->parts.end()); - else - { - } // Primitives, any, errors, and generics are left untouched. - } + } // Primitives, any, errors, and generics are left untouched. } } @@ -3038,79 +2272,39 @@ void Unifier::occursCheck(DenseHashSet& seen, TypeId needle, TypeId hays occursCheck(seen, needle, tv); }; - if (FFlag::LuauUseCommittingTxnLog) + needle = log.follow(needle); + haystack = log.follow(haystack); + + if (seen.find(haystack)) + return; + + seen.insert(haystack); + + if (log.getMutable(needle)) + return; + + if (!log.getMutable(needle)) + ice("Expected needle to be free"); + + if (needle == haystack) { - needle = log.follow(needle); - haystack = log.follow(haystack); + reportError(TypeError{location, OccursCheckFailed{}}); + log.replace(needle, *getSingletonTypes().errorRecoveryType()); - if (seen.find(haystack)) - return; - - seen.insert(haystack); - - if (log.getMutable(needle)) - return; - - if (!log.getMutable(needle)) - ice("Expected needle to be free"); - - if (needle == haystack) - { - reportError(TypeError{location, OccursCheckFailed{}}); - log.replace(needle, *getSingletonTypes().errorRecoveryType()); - - return; - } - - if (log.getMutable(haystack)) - return; - else if (auto a = log.getMutable(haystack)) - { - for (TypeId ty : a->options) - check(ty); - } - else if (auto a = log.getMutable(haystack)) - { - for (TypeId ty : a->parts) - check(ty); - } + return; } - else + + if (log.getMutable(haystack)) + return; + else if (auto a = log.getMutable(haystack)) { - needle = follow(needle); - haystack = follow(haystack); - - if (seen.find(haystack)) - return; - - seen.insert(haystack); - - if (get(needle)) - return; - - if (!get(needle)) - ice("Expected needle to be free"); - - if (needle == haystack) - { - reportError(TypeError{location, OccursCheckFailed{}}); - DEPRECATED_log(needle); - *asMutable(needle) = *getSingletonTypes().errorRecoveryType(); - return; - } - - if (get(haystack)) - return; - else if (auto a = get(haystack)) - { - for (TypeId ty : a->options) - check(ty); - } - else if (auto a = get(haystack)) - { - for (TypeId ty : a->parts) - check(ty); - } + for (TypeId ty : a->options) + check(ty); + } + else if (auto a = log.getMutable(haystack)) + { + for (TypeId ty : a->parts) + check(ty); } } @@ -3123,87 +2317,45 @@ void Unifier::occursCheck(TypePackId needle, TypePackId haystack) void Unifier::occursCheck(DenseHashSet& seen, TypePackId needle, TypePackId haystack) { - if (FFlag::LuauUseCommittingTxnLog) + needle = log.follow(needle); + haystack = log.follow(haystack); + + if (seen.find(haystack)) + return; + + seen.insert(haystack); + + if (log.getMutable(needle)) + return; + + if (!log.getMutable(needle)) + ice("Expected needle pack to be free"); + + RecursionLimiter _ra(&sharedState.counters.recursionCount, FInt::LuauTypeInferRecursionLimit); + + while (!log.getMutable(haystack)) { - needle = log.follow(needle); - haystack = log.follow(haystack); - - if (seen.find(haystack)) - return; - - seen.insert(haystack); - - if (log.getMutable(needle)) - return; - - if (!log.getMutable(needle)) - ice("Expected needle pack to be free"); - - RecursionLimiter _ra(&sharedState.counters.recursionCount, FInt::LuauTypeInferRecursionLimit); - - while (!log.getMutable(haystack)) + if (needle == haystack) { - if (needle == haystack) - { - reportError(TypeError{location, OccursCheckFailed{}}); - log.replace(needle, *getSingletonTypes().errorRecoveryTypePack()); + reportError(TypeError{location, OccursCheckFailed{}}); + log.replace(needle, *getSingletonTypes().errorRecoveryTypePack()); - return; - } - - if (auto a = get(haystack); a && a->tail) - { - haystack = log.follow(*a->tail); - continue; - } - - break; + return; } - } - else - { - needle = follow(needle); - haystack = follow(haystack); - if (seen.find(haystack)) - return; - - seen.insert(haystack); - - if (get(needle)) - return; - - if (!get(needle)) - ice("Expected needle pack to be free"); - - RecursionLimiter _ra(&sharedState.counters.recursionCount, FInt::LuauTypeInferRecursionLimit); - - while (!get(haystack)) + if (auto a = get(haystack); a && a->tail) { - if (needle == haystack) - { - reportError(TypeError{location, OccursCheckFailed{}}); - DEPRECATED_log(needle); - *asMutable(needle) = *getSingletonTypes().errorRecoveryTypePack(); - } - - if (auto a = get(haystack); a && a->tail) - { - haystack = follow(*a->tail); - continue; - } - - break; + haystack = log.follow(*a->tail); + continue; } + + break; } } Unifier Unifier::makeChildUnifier() { - if (FFlag::LuauUseCommittingTxnLog) - return Unifier{types, mode, log.sharedSeen, location, variance, sharedState, &log}; - else - return Unifier{types, mode, DEPRECATED_log.sharedSeen, location, variance, sharedState, &log}; + return Unifier{types, mode, log.sharedSeen, location, variance, sharedState, &log}; } bool Unifier::isNonstrictMode() const diff --git a/VM/include/lua.h b/VM/include/lua.h index 0a561f27..274c4ed9 100644 --- a/VM/include/lua.h +++ b/VM/include/lua.h @@ -229,19 +229,46 @@ LUA_API void lua_setthreaddata(lua_State* L, void* data); enum lua_GCOp { + /* stop and resume incremental garbage collection */ LUA_GCSTOP, LUA_GCRESTART, + + /* run a full GC cycle; not recommended for latency sensitive applications */ LUA_GCCOLLECT, + + /* return the heap size in KB and the remainder in bytes */ LUA_GCCOUNT, LUA_GCCOUNTB, + + /* return 1 if GC is active (not stopped); note that GC may not be actively collecting even if it's running */ LUA_GCISRUNNING, - // garbage collection is handled by 'assists' that perform some amount of GC work matching pace of allocation - // explicit GC steps allow to perform some amount of work at custom points to offset the need for GC assists - // note that GC might also be paused for some duration (until bytes allocated meet the threshold) - // if an explicit step is performed during this pause, it will trigger the start of the next collection cycle + /* + ** perform an explicit GC step, with the step size specified in KB + ** + ** garbage collection is handled by 'assists' that perform some amount of GC work matching pace of allocation + ** explicit GC steps allow to perform some amount of work at custom points to offset the need for GC assists + ** note that GC might also be paused for some duration (until bytes allocated meet the threshold) + ** if an explicit step is performed during this pause, it will trigger the start of the next collection cycle + */ LUA_GCSTEP, + /* + ** tune GC parameters G (goal), S (step multiplier) and step size (usually best left ignored) + ** + ** garbage collection is incremental and tries to maintain the heap size to balance memory and performance overhead + ** this overhead is determined by G (goal) which is the ratio between total heap size and the amount of live data in it + ** G is specified in percentages; by default G=200% which means that the heap is allowed to grow to ~2x the size of live data. + ** + ** collector tries to collect S% of allocated bytes by interrupting the application after step size bytes were allocated. + ** when S is too small, collector may not be able to catch up and the effective goal that can be reached will be larger. + ** S is specified in percentages; by default S=200% which means that collector will run at ~2x the pace of allocations. + ** + ** it is recommended to set S in the interval [100 / (G - 100), 100 + 100 / (G - 100))] with a minimum value of 150%; for example: + ** - for G=200%, S should be in the interval [150%, 200%] + ** - for G=150%, S should be in the interval [200%, 300%] + ** - for G=125%, S should be in the interval [400%, 500%] + */ LUA_GCSETGOAL, LUA_GCSETSTEPMUL, LUA_GCSETSTEPSIZE, diff --git a/VM/include/luaconf.h b/VM/include/luaconf.h index c5bf1c18..b93cbf7c 100644 --- a/VM/include/luaconf.h +++ b/VM/include/luaconf.h @@ -59,33 +59,6 @@ #define LUA_IDSIZE 256 #endif -/* -@@ LUAI_GCGOAL defines the desired top heap size in relation to the live heap -@* size at the end of the GC cycle -** CHANGE it if you want the GC to run faster or slower (higher values -** mean larger GC pauses which mean slower collection.) You can also change -** this value dynamically. -*/ -#ifndef LUAI_GCGOAL -#define LUAI_GCGOAL 200 /* 200% (allow heap to double compared to live heap size) */ -#endif - -/* -@@ LUAI_GCSTEPMUL / LUAI_GCSTEPSIZE define the default speed of garbage collection -@* relative to memory allocation. -** Every LUAI_GCSTEPSIZE KB allocated, incremental collector collects LUAI_GCSTEPSIZE -** times LUAI_GCSTEPMUL% bytes. -** CHANGE it if you want to change the granularity of the garbage -** collection. -*/ -#ifndef LUAI_GCSTEPMUL -#define LUAI_GCSTEPMUL 200 /* GC runs 'twice the speed' of memory allocation */ -#endif - -#ifndef LUAI_GCSTEPSIZE -#define LUAI_GCSTEPSIZE 1 /* GC runs every KB of memory allocation */ -#endif - /* LUA_MINSTACK is the guaranteed number of Lua stack slots available to a C function */ #ifndef LUA_MINSTACK #define LUA_MINSTACK 20 diff --git a/VM/include/lualib.h b/VM/include/lualib.h index baf27b47..bebd0a0f 100644 --- a/VM/include/lualib.h +++ b/VM/include/lualib.h @@ -54,6 +54,8 @@ LUALIB_API lua_State* luaL_newstate(void); LUALIB_API const char* luaL_findtable(lua_State* L, int idx, const char* fname, int szhint); +LUALIB_API const char* luaL_typename(lua_State* L, int idx); + /* ** =============================================================== ** some useful macros @@ -66,8 +68,6 @@ LUALIB_API const char* luaL_findtable(lua_State* L, int idx, const char* fname, #define luaL_checkstring(L, n) (luaL_checklstring(L, (n), NULL)) #define luaL_optstring(L, n, d) (luaL_optlstring(L, (n), (d), NULL)) -#define luaL_typename(L, i) lua_typename(L, lua_type(L, (i))) - #define luaL_getmetatable(L, n) (lua_getfield(L, LUA_REGISTRYINDEX, (n))) #define luaL_opt(L, f, n, d) (lua_isnoneornil(L, (n)) ? (d) : f(L, (n))) diff --git a/VM/src/laux.cpp b/VM/src/laux.cpp index 9a6f7793..9fe2ebb6 100644 --- a/VM/src/laux.cpp +++ b/VM/src/laux.cpp @@ -11,6 +11,8 @@ #include +LUAU_DYNAMIC_FASTFLAGVARIABLE(LuauMorePreciseLuaLTypeName, false) + /* convert a stack index to positive */ #define abs_index(L, i) ((i) > 0 || (i) <= LUA_REGISTRYINDEX ? (i) : lua_gettop(L) + (i) + 1) @@ -333,6 +335,19 @@ const char* luaL_findtable(lua_State* L, int idx, const char* fname, int szhint) return NULL; } +const char* luaL_typename(lua_State* L, int idx) +{ + if (DFFlag::LuauMorePreciseLuaLTypeName) + { + const TValue* obj = luaA_toobject(L, idx); + return luaT_objtypename(L, obj); + } + else + { + return lua_typename(L, lua_type(L, idx)); + } +} + /* ** {====================================================== ** Generic Buffer manipulation diff --git a/VM/src/lbaselib.cpp b/VM/src/lbaselib.cpp index 988fd315..2307598e 100644 --- a/VM/src/lbaselib.cpp +++ b/VM/src/lbaselib.cpp @@ -10,6 +10,8 @@ #include #include +LUAU_DYNAMIC_FASTFLAG(LuauMorePreciseLuaLTypeName) + static void writestring(const char* s, size_t l) { fwrite(s, 1, l, stdout); @@ -186,7 +188,14 @@ static int luaB_gcinfo(lua_State* L) static int luaB_type(lua_State* L) { luaL_checkany(L, 1); - lua_pushstring(L, luaL_typename(L, 1)); + if (DFFlag::LuauMorePreciseLuaLTypeName) + { + lua_pushstring(L, lua_typename(L, lua_type(L, 1))); + } + else + { + lua_pushstring(L, luaL_typename(L, 1)); + } return 1; } diff --git a/VM/src/lgc.h b/VM/src/lgc.h index cbeeebd4..ad8ee78a 100644 --- a/VM/src/lgc.h +++ b/VM/src/lgc.h @@ -6,6 +6,13 @@ #include "lobject.h" #include "lstate.h" +/* +** Default settings for GC tunables (settable via lua_gc) +*/ +#define LUAI_GCGOAL 200 /* 200% (allow heap to double compared to live heap size) */ +#define LUAI_GCSTEPMUL 200 /* GC runs 'twice the speed' of memory allocation */ +#define LUAI_GCSTEPSIZE 1 /* GC runs every KB of memory allocation */ + /* ** Possible states of the Garbage Collector */ diff --git a/tests/Autocomplete.test.cpp b/tests/Autocomplete.test.cpp index ce890ba8..55b0618a 100644 --- a/tests/Autocomplete.test.cpp +++ b/tests/Autocomplete.test.cpp @@ -14,7 +14,6 @@ LUAU_FASTFLAG(LuauTraceTypesInNonstrictMode2) LUAU_FASTFLAG(LuauSetMetatableDoesNotTimeTravel) -LUAU_FASTFLAG(LuauUseCommittingTxnLog) LUAU_FASTFLAG(LuauTableCloneType) using namespace Luau; @@ -1912,14 +1911,9 @@ local bar: @1= foo CHECK(!ac.entryMap.count("foo")); } -// Switch back to TEST_CASE_FIXTURE with regular ACFixture when removing the -// LuauUseCommittingTxnLog flag. -TEST_CASE("type_correct_function_no_parenthesis") +TEST_CASE_FIXTURE(ACFixture, "type_correct_function_no_parenthesis") { - ScopedFastFlag sff_LuauUseCommittingTxnLog = ScopedFastFlag("LuauUseCommittingTxnLog", true); - ACFixture fix; - - fix.check(R"( + check(R"( local function target(a: (number) -> number) return a(4) end local function bar1(a: number) return -a end local function bar2(a: string) return a .. 'x' end @@ -1927,7 +1921,7 @@ local function bar2(a: string) return a .. 'x' end return target(b@1 )"); - auto ac = fix.autocomplete('1'); + auto ac = autocomplete('1'); CHECK(ac.entryMap.count("bar1")); CHECK(ac.entryMap["bar1"].typeCorrect == TypeCorrectKind::Correct); @@ -1937,8 +1931,6 @@ return target(b@1 TEST_CASE_FIXTURE(ACFixture, "function_in_assignment_has_parentheses") { - ScopedFastFlag luauAutocompleteAvoidMutation("LuauAutocompleteAvoidMutation", true); - check(R"( local function bar(a: number) return -a end local abc = b@1 @@ -1952,8 +1944,6 @@ local abc = b@1 TEST_CASE_FIXTURE(ACFixture, "function_result_passed_to_function_has_parentheses") { - ScopedFastFlag luauAutocompleteAvoidMutation("LuauAutocompleteAvoidMutation", true); - check(R"( local function foo() return 1 end local function bar(a: number) return -a end @@ -1978,14 +1968,9 @@ local fp: @1= f CHECK(ac.entryMap.count("({ x: number, y: number }) -> number")); } -// Switch back to TEST_CASE_FIXTURE with regular ACFixture when removing the -// LuauUseCommittingTxnLog flag. -TEST_CASE("type_correct_keywords") +TEST_CASE_FIXTURE(ACFixture, "type_correct_keywords") { - ScopedFastFlag sff_LuauUseCommittingTxnLog = ScopedFastFlag("LuauUseCommittingTxnLog", true); - ACFixture fix; - - fix.check(R"( + check(R"( local function a(x: boolean) end local function b(x: number?) end local function c(x: (number) -> string) end @@ -2002,26 +1987,26 @@ local dc = d(f@4) local ec = e(f@5) )"); - auto ac = fix.autocomplete('1'); + auto ac = autocomplete('1'); CHECK(ac.entryMap.count("tru")); CHECK(ac.entryMap["tru"].typeCorrect == TypeCorrectKind::None); CHECK(ac.entryMap["true"].typeCorrect == TypeCorrectKind::Correct); CHECK(ac.entryMap["false"].typeCorrect == TypeCorrectKind::Correct); - ac = fix.autocomplete('2'); + ac = autocomplete('2'); CHECK(ac.entryMap.count("ni")); CHECK(ac.entryMap["ni"].typeCorrect == TypeCorrectKind::None); CHECK(ac.entryMap["nil"].typeCorrect == TypeCorrectKind::Correct); - ac = fix.autocomplete('3'); + ac = autocomplete('3'); CHECK(ac.entryMap.count("false")); CHECK(ac.entryMap["false"].typeCorrect == TypeCorrectKind::None); CHECK(ac.entryMap["function"].typeCorrect == TypeCorrectKind::Correct); - ac = fix.autocomplete('4'); + ac = autocomplete('4'); CHECK(ac.entryMap["function"].typeCorrect == TypeCorrectKind::Correct); - ac = fix.autocomplete('5'); + ac = autocomplete('5'); CHECK(ac.entryMap["function"].typeCorrect == TypeCorrectKind::Correct); } @@ -2512,23 +2497,21 @@ local t = { CHECK(ac.entryMap.count("second")); } -TEST_CASE("autocomplete_documentation_symbols") +TEST_CASE_FIXTURE(Fixture, "autocomplete_documentation_symbols") { - Fixture fix(FFlag::LuauUseCommittingTxnLog); - - fix.loadDefinition(R"( + loadDefinition(R"( declare y: { x: number, } )"); - fix.fileResolver.source["Module/A"] = R"( + fileResolver.source["Module/A"] = R"( local a = y. )"; - fix.frontend.check("Module/A"); + frontend.check("Module/A"); - auto ac = autocomplete(fix.frontend, "Module/A", Position{1, 21}, nullCallback); + auto ac = autocomplete(frontend, "Module/A", Position{1, 21}, nullCallback); REQUIRE(ac.entryMap.count("x")); CHECK_EQ(ac.entryMap["x"].documentationSymbol, "@test/global/y.x"); @@ -2646,8 +2629,6 @@ local a: A<(number, s@1> TEST_CASE_FIXTURE(ACFixture, "autocomplete_first_function_arg_expected_type") { - ScopedFastFlag luauAutocompleteAvoidMutation("LuauAutocompleteAvoidMutation", true); - check(R"( local function foo1() return 1 end local function foo2() return "1" end @@ -2720,7 +2701,6 @@ type A = () -> T TEST_CASE_FIXTURE(ACFixture, "autocomplete_oop_implicit_self") { - ScopedFastFlag flag("LuauMissingFollowACMetatables", true); check(R"( --!strict local Class = {} @@ -2764,8 +2744,6 @@ TEST_CASE_FIXTURE(ACFixture, "autocomplete_on_string_singletons") TEST_CASE_FIXTURE(ACFixture, "function_in_assignment_has_parentheses_2") { - ScopedFastFlag luauAutocompleteAvoidMutation("LuauAutocompleteAvoidMutation", true); - check(R"( local bar: ((number) -> number) & (number, number) -> number) local abc = b@1 diff --git a/tests/TypeInfer.aliases.test.cpp b/tests/TypeInfer.aliases.test.cpp index d584eb2d..711c0aa1 100644 --- a/tests/TypeInfer.aliases.test.cpp +++ b/tests/TypeInfer.aliases.test.cpp @@ -7,6 +7,8 @@ using namespace Luau; +LUAU_FASTFLAG(LuauFixIncorrectLineNumberDuplicateType) + TEST_SUITE_BEGIN("TypeAliases"); TEST_CASE_FIXTURE(Fixture, "cyclic_function_type_in_type_alias") @@ -241,6 +243,27 @@ TEST_CASE_FIXTURE(Fixture, "export_type_and_type_alias_are_duplicates") CHECK_EQ(dtd->name, "Foo"); } +TEST_CASE_FIXTURE(Fixture, "reported_location_is_correct_when_type_alias_are_duplicates") +{ + CheckResult result = check(R"( + type A = string + type B = number + type C = string + type B = number + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + auto dtd = get(result.errors[0]); + REQUIRE(dtd); + CHECK_EQ(dtd->name, "B"); + + if (FFlag::LuauFixIncorrectLineNumberDuplicateType) + CHECK_EQ(dtd->previousLocation.begin.line + 1, 3); + else + CHECK_EQ(dtd->previousLocation.begin.line + 1, 1); +} + TEST_CASE_FIXTURE(Fixture, "stringify_optional_parameterized_alias") { CheckResult result = check(R"( diff --git a/tests/TypeInfer.builtins.test.cpp b/tests/TypeInfer.builtins.test.cpp index bf990770..8da655b3 100644 --- a/tests/TypeInfer.builtins.test.cpp +++ b/tests/TypeInfer.builtins.test.cpp @@ -8,8 +8,6 @@ using namespace Luau; -LUAU_FASTFLAG(LuauUseCommittingTxnLog) - TEST_SUITE_BEGIN("BuiltinTests"); TEST_CASE_FIXTURE(Fixture, "math_things_are_defined") @@ -443,28 +441,19 @@ TEST_CASE_FIXTURE(Fixture, "os_time_takes_optional_date_table") CHECK_EQ(*typeChecker.numberType, *requireType("n3")); } -// Switch back to TEST_CASE_FIXTURE with regular Fixture when removing the -// LuauUseCommittingTxnLog flag. -TEST_CASE("thread_is_a_type") +TEST_CASE_FIXTURE(Fixture, "thread_is_a_type") { - Fixture fix(FFlag::LuauUseCommittingTxnLog); - - CheckResult result = fix.check(R"( + CheckResult result = check(R"( local co = coroutine.create(function() end) )"); - // Replace with LUAU_REQUIRE_NO_ERRORS(result) when using TEST_CASE_FIXTURE. - CHECK(result.errors.size() == 0); - CHECK_EQ(*fix.typeChecker.threadType, *fix.requireType("co")); + LUAU_REQUIRE_NO_ERRORS(result); + CHECK_EQ(*typeChecker.threadType, *requireType("co")); } -// Switch back to TEST_CASE_FIXTURE with regular Fixture when removing the -// LuauUseCommittingTxnLog flag. -TEST_CASE("coroutine_resume_anything_goes") +TEST_CASE_FIXTURE(Fixture, "coroutine_resume_anything_goes") { - Fixture fix(FFlag::LuauUseCommittingTxnLog); - - CheckResult result = fix.check(R"( + CheckResult result = check(R"( local function nifty(x, y) print(x, y) local z = coroutine.yield(1, 2) @@ -477,17 +466,12 @@ TEST_CASE("coroutine_resume_anything_goes") local answer = coroutine.resume(co, 3) )"); - // Replace with LUAU_REQUIRE_NO_ERRORS(result) when using TEST_CASE_FIXTURE. - CHECK(result.errors.size() == 0); + LUAU_REQUIRE_NO_ERRORS(result); } -// Switch back to TEST_CASE_FIXTURE with regular Fixture when removing the -// LuauUseCommittingTxnLog flag. -TEST_CASE("coroutine_wrap_anything_goes") +TEST_CASE_FIXTURE(Fixture, "coroutine_wrap_anything_goes") { - Fixture fix(FFlag::LuauUseCommittingTxnLog); - - CheckResult result = fix.check(R"( + CheckResult result = check(R"( --!nonstrict local function nifty(x, y) print(x, y) @@ -501,8 +485,7 @@ TEST_CASE("coroutine_wrap_anything_goes") local answer = f(3) )"); - // Replace with LUAU_REQUIRE_NO_ERRORS(result) when using TEST_CASE_FIXTURE. - CHECK(result.errors.size() == 0); + LUAU_REQUIRE_NO_ERRORS(result); } TEST_CASE_FIXTURE(Fixture, "setmetatable_should_not_mutate_persisted_types") @@ -961,4 +944,18 @@ TEST_CASE_FIXTURE(Fixture, "table_freeze_is_generic") CHECK_EQ("*unknown*", toString(requireType("d"))); } +TEST_CASE_FIXTURE(Fixture, "set_metatable_needs_arguments") +{ + ScopedFastFlag sff{"LuauSetMetaTableArgsCheck", true}; + CheckResult result = check(R"( +local a = {b=setmetatable} +a.b() +a:b() +a:b({}) + )"); + LUAU_REQUIRE_ERROR_COUNT(2, result); + CHECK_EQ(result.errors[0], (TypeError{Location{{2, 0}, {2, 5}}, CountMismatch{2, 0}})); + CHECK_EQ(result.errors[1], (TypeError{Location{{3, 0}, {3, 5}}, CountMismatch{2, 1}})); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.generics.test.cpp b/tests/TypeInfer.generics.test.cpp index c482847b..547fbab1 100644 --- a/tests/TypeInfer.generics.test.cpp +++ b/tests/TypeInfer.generics.test.cpp @@ -8,6 +8,8 @@ using namespace Luau; +LUAU_FASTFLAG(LuauFixArgumentCountMismatchAmountWithGenericTypes) + TEST_SUITE_BEGIN("GenericsTests"); TEST_CASE_FIXTURE(Fixture, "check_generic_function") @@ -786,4 +788,47 @@ local TheDispatcher: Dispatcher = { LUAU_REQUIRE_NO_ERRORS(result); } +TEST_CASE_FIXTURE(Fixture, "generic_argument_count_too_few") +{ + CheckResult result = check(R"( +function test(a: number) + return 1 +end + +function wrapper(f: (A...) -> number, ...: A...) +end + +wrapper(test) + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + if (FFlag::LuauFixArgumentCountMismatchAmountWithGenericTypes) + CHECK_EQ(toString(result.errors[0]), R"(Argument count mismatch. Function expects 2 arguments, but only 1 is specified)"); + else + CHECK_EQ(toString(result.errors[0]), R"(Argument count mismatch. Function expects 1 argument, but 1 is specified)"); +} + +TEST_CASE_FIXTURE(Fixture, "generic_argument_count_too_many") +{ + CheckResult result = check(R"( +function test2(a: number, b: string) + return 1 +end + +function wrapper(f: (A...) -> number, ...: A...) +end + +wrapper(test2, 1, "", 3) + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + if (FFlag::LuauFixArgumentCountMismatchAmountWithGenericTypes) + CHECK_EQ(toString(result.errors[0]), R"(Argument count mismatch. Function expects 3 arguments, but 4 are specified)"); + else + CHECK_EQ(toString(result.errors[0]), R"(Argument count mismatch. Function expects 1 argument, but 4 are specified)"); +} + + TEST_SUITE_END(); diff --git a/tests/TypeInfer.tables.test.cpp b/tests/TypeInfer.tables.test.cpp index da035ba1..a5eba5df 100644 --- a/tests/TypeInfer.tables.test.cpp +++ b/tests/TypeInfer.tables.test.cpp @@ -2334,4 +2334,54 @@ TEST_CASE_FIXTURE(Fixture, "pass_a_union_of_tables_to_a_function_that_requires_a REQUIRE_EQ("{| [any]: any, x: number, y: number |} | {| y: number |}", toString(requireType("b"))); } +TEST_CASE_FIXTURE(Fixture, "unifying_tables_shouldnt_uaf1") +{ + ScopedFastFlag sff{"LuauTxnLogCheckForInvalidation", true}; + + CheckResult result = check(R"( +-- This example produced a UAF at one point, caused by pointers to table types becoming +-- invalidated by child unifiers. (Calling log.concat can cause pointers to become invalid.) +type _Entry = { + a: number, + + middle: (self: _Entry) -> (), + + z: number +} + +export type AnyEntry = _Entry + +local Entry = {} +Entry.__index = Entry + +function Entry:dispose() + self:middle() + forgetChildren(self) -- unify free with sealed AnyEntry +end + +function forgetChildren(parent: AnyEntry) +end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "unifying_tables_shouldnt_uaf2") +{ + ScopedFastFlag sff{"LuauTxnLogCheckForInvalidation", true}; + + CheckResult result = check(R"( +-- Another example that UAFd, this time found by fuzzing. +local _ +do +_._ *= (_[{n0=_[{[{[_]=_,}]=_,}],}])[_] +_ = (_.n0) +end +_._ *= (_[false])[_] +_ = (_.cos) + )"); + + LUAU_REQUIRE_ERRORS(result); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.test.cpp b/tests/TypeInfer.test.cpp index f63579b5..d7bbad20 100644 --- a/tests/TypeInfer.test.cpp +++ b/tests/TypeInfer.test.cpp @@ -5144,7 +5144,6 @@ end TEST_CASE_FIXTURE(Fixture, "cli_50041_committing_txnlog_in_apollo_client_error") { - ScopedFastFlag committingTxnLog{"LuauUseCommittingTxnLog", true}; ScopedFastFlag subtypingVariance{"LuauTableSubtypingVariance2", true}; CheckResult result = check(R"( @@ -5355,4 +5354,41 @@ end LUAU_REQUIRE_NO_ERRORS(result); } +TEST_CASE_FIXTURE(Fixture, "function_decl_quantify_right_type") +{ + ScopedFastFlag statFunctionSimplify{"LuauStatFunctionSimplify", true}; + + fileResolver.source["game/isAMagicMock"] = R"( +--!nonstrict +return function(value) + return false +end + )"; + + CheckResult result = check(R"( +--!nonstrict +local MagicMock = {} +MagicMock.is = require(game.isAMagicMock) + +function MagicMock.is(value) + return false +end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "function_decl_non_self_sealed_overwrite") +{ + ScopedFastFlag statFunctionSimplify{"LuauStatFunctionSimplify", true}; + + CheckResult result = check(R"( +function string.len(): number + return 1 +end + )"); + + LUAU_REQUIRE_ERRORS(result); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.tryUnify.test.cpp b/tests/TypeInfer.tryUnify.test.cpp index f6ee3ccc..d8de2594 100644 --- a/tests/TypeInfer.tryUnify.test.cpp +++ b/tests/TypeInfer.tryUnify.test.cpp @@ -9,8 +9,6 @@ using namespace Luau; -LUAU_FASTFLAG(LuauUseCommittingTxnLog) - struct TryUnifyFixture : Fixture { TypeArena arena; @@ -43,8 +41,7 @@ TEST_CASE_FIXTURE(TryUnifyFixture, "compatible_functions_are_unified") state.tryUnify(&functionTwo, &functionOne); CHECK(state.errors.empty()); - if (FFlag::LuauUseCommittingTxnLog) - state.log.commit(); + state.log.commit(); CHECK_EQ(functionOne, functionTwo); } @@ -86,8 +83,7 @@ TEST_CASE_FIXTURE(TryUnifyFixture, "tables_can_be_unified") CHECK(state.errors.empty()); - if (FFlag::LuauUseCommittingTxnLog) - state.log.commit(); + state.log.commit(); CHECK_EQ(*getMutable(&tableOne)->props["foo"].type, *getMutable(&tableTwo)->props["foo"].type); } @@ -110,9 +106,6 @@ TEST_CASE_FIXTURE(TryUnifyFixture, "incompatible_tables_are_preserved") CHECK_EQ(1, state.errors.size()); - if (!FFlag::LuauUseCommittingTxnLog) - state.DEPRECATED_log.rollback(); - CHECK_NE(*getMutable(&tableOne)->props["foo"].type, *getMutable(&tableTwo)->props["foo"].type); } @@ -217,34 +210,6 @@ TEST_CASE_FIXTURE(TryUnifyFixture, "cli_41095_concat_log_in_sealed_table_unifica CHECK_EQ(toString(result.errors[1]), "Available overloads: ({a}, a) -> (); and ({a}, number, a) -> ()"); } -TEST_CASE("undo_new_prop_on_unsealed_table") -{ - ScopedFastFlag flags[] = { - {"LuauTableSubtypingVariance2", true}, - // This test makes no sense with a committing TxnLog. - {"LuauUseCommittingTxnLog", false}, - }; - // I am not sure how to make this happen in Luau code. - - TryUnifyFixture fix; - - TypeId unsealedTable = fix.arena.addType(TableTypeVar{TableState::Unsealed, TypeLevel{}}); - TypeId sealedTable = - fix.arena.addType(TableTypeVar{{{"prop", Property{getSingletonTypes().numberType}}}, std::nullopt, TypeLevel{}, TableState::Sealed}); - - const TableTypeVar* ttv = get(unsealedTable); - REQUIRE(ttv); - - fix.state.tryUnify(sealedTable, unsealedTable); - - // To be honest, it's really quite spooky here that we're amending an unsealed table in this case. - CHECK(!ttv->props.empty()); - - fix.state.DEPRECATED_log.rollback(); - - CHECK(ttv->props.empty()); -} - TEST_CASE_FIXTURE(TryUnifyFixture, "free_tail_is_grown_properly") { TypePackId threeNumbers = arena.addTypePack(TypePack{{typeChecker.numberType, typeChecker.numberType, typeChecker.numberType}, std::nullopt}); @@ -267,11 +232,6 @@ TEST_CASE_FIXTURE(TryUnifyFixture, "recursive_metatable_getmatchtag") TEST_CASE_FIXTURE(TryUnifyFixture, "cli_50320_follow_in_any_unification") { - ScopedFastFlag sffs[] = { - {"LuauUseCommittingTxnLog", true}, - {"LuauFollowWithCommittingTxnLogInAnyUnification", true}, - }; - TypePackVar free{FreeTypePack{TypeLevel{}}}; TypePackVar target{TypePack{}}; diff --git a/tests/TypeInfer.typePacks.cpp b/tests/TypeInfer.typePacks.cpp index 6b96f449..fcc21c18 100644 --- a/tests/TypeInfer.typePacks.cpp +++ b/tests/TypeInfer.typePacks.cpp @@ -9,8 +9,6 @@ using namespace Luau; -LUAU_FASTFLAG(LuauUseCommittingTxnLog) - TEST_SUITE_BEGIN("TypePackTests"); TEST_CASE_FIXTURE(Fixture, "infer_multi_return") @@ -264,13 +262,9 @@ TEST_CASE_FIXTURE(Fixture, "variadic_pack_syntax") CHECK_EQ(toString(requireType("foo")), "(...number) -> ()"); } -// Switch back to TEST_CASE_FIXTURE with regular Fixture when removing the -// LuauUseCommittingTxnLog flag. -TEST_CASE("type_pack_hidden_free_tail_infinite_growth") +TEST_CASE_FIXTURE(Fixture, "type_pack_hidden_free_tail_infinite_growth") { - Fixture fix(FFlag::LuauUseCommittingTxnLog); - - CheckResult result = fix.check(R"( + CheckResult result = check(R"( --!nonstrict if _ then _[function(l0)end],l0 = _ @@ -282,8 +276,7 @@ elseif _ then end )"); - // Switch back to LUAU_REQUIRE_ERRORS(result) when using TEST_CASE_FIXTURE. - CHECK(result.errors.size() > 0); + LUAU_REQUIRE_ERRORS(result); } TEST_CASE_FIXTURE(Fixture, "variadic_argument_tail") diff --git a/tests/TypeInfer.unionTypes.test.cpp b/tests/TypeInfer.unionTypes.test.cpp index 0e0b6ebb..ad4cecd8 100644 --- a/tests/TypeInfer.unionTypes.test.cpp +++ b/tests/TypeInfer.unionTypes.test.cpp @@ -7,7 +7,6 @@ #include "doctest.h" LUAU_FASTFLAG(LuauEqConstraint) -LUAU_FASTFLAG(LuauUseCommittingTxnLog) using namespace Luau; @@ -282,19 +281,16 @@ local c = b:foo(1, 2) CHECK_EQ("Value of type 'A?' could be nil", toString(result.errors[0])); } -TEST_CASE("optional_union_follow") +TEST_CASE_FIXTURE(Fixture, "optional_union_follow") { - Fixture fix(FFlag::LuauUseCommittingTxnLog); - - CheckResult result = fix.check(R"( + CheckResult result = check(R"( local y: number? = 2 local x = y local function f(a: number, b: typeof(x), c: typeof(x)) return -a end return f() )"); - REQUIRE_EQ(result.errors.size(), 1); - // LUAU_REQUIRE_ERROR_COUNT(1, result); + LUAU_REQUIRE_ERROR_COUNT(1, result); auto acm = get(result.errors[0]); REQUIRE(acm); diff --git a/tools/LuauVisualize.py b/tools/LuauVisualize.py new file mode 100644 index 00000000..40f8d6be --- /dev/null +++ b/tools/LuauVisualize.py @@ -0,0 +1,107 @@ +# This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details + +# HACK: LLDB's python API doesn't afford anything helpful for getting at variadic template parameters. +# We're forced to resort to parsing names as strings. +def templateParams(s): + depth = 0 + start = s.find('<') + 1 + result = [] + for i, c in enumerate(s[start:], start): + if c == '<': + depth += 1 + elif c == '>': + if depth == 0: + result.append(s[start: i].strip()) + break + depth -= 1 + elif c == ',' and depth == 0: + result.append(s[start: i].strip()) + start = i + 1 + return result + +def getType(target, typeName): + stars = 0 + + typeName = typeName.strip() + while typeName.endswith('*'): + stars += 1 + typeName = typeName[:-1] + + if typeName.startswith('const '): + typeName = typeName[6:] + + ty = target.FindFirstType(typeName.strip()) + for _ in range(stars): + ty = ty.GetPointerType() + + return ty + +def luau_variant_summary(valobj, internal_dict, options): + type_id = valobj.GetChildMemberWithName("typeid").GetValueAsUnsigned() + storage = valobj.GetChildMemberWithName("storage") + params = templateParams(valobj.GetType().GetCanonicalType().GetName()) + stored_type = params[type_id] + value = storage.Cast(stored_type.GetPointerType()).Dereference() + return stored_type.GetDisplayTypeName() + " [" + value.GetValue() + "]" + +class LuauVariantSyntheticChildrenProvider: + node_names = ["type", "value"] + + def __init__(self, valobj, internal_dict): + self.valobj = valobj + self.type_index = None + self.current_type = None + self.type_params = [] + self.stored_value = None + + def num_children(self): + return len(self.node_names) + + def has_children(self): + return True + + def get_child_index(self, name): + try: + return self.node_names.index(name) + except ValueError: + return -1 + + def get_child_at_index(self, index): + try: + node = self.node_names[index] + except IndexError: + return None + + if node == "type": + if self.current_type: + return self.valobj.CreateValueFromExpression(node, f"(const char*)\"{self.current_type.GetDisplayTypeName()}\"") + else: + return self.valobj.CreateValueFromExpression(node, "(const char*)\"\"") + elif node == "value": + if self.stored_value is not None: + if self.current_type is not None: + return self.valobj.CreateValueFromData(node, self.stored_value.GetData(), self.current_type) + else: + return self.valobj.CreateValueExpression(node, "(const char*)\"\"") + else: + return self.valobj.CreateValueFromExpression(node, "(const char*)\"\"") + else: + return None + + def update(self): + self.type_index = self.valobj.GetChildMemberWithName("typeid").GetValueAsSigned() + self.type_params = templateParams(self.valobj.GetType().GetCanonicalType().GetName()) + + if len(self.type_params) > self.type_index: + self.current_type = getType(self.valobj.GetTarget(), self.type_params[self.type_index]) + + if self.current_type: + storage = self.valobj.GetChildMemberWithName("storage") + self.stored_value = storage.Cast(self.current_type.GetPointerType()).Dereference() + else: + self.stored_value = None + else: + self.current_type = None + self.stored_value = None + + return False diff --git a/tools/lldb-formatters.lldb b/tools/lldb-formatters.lldb new file mode 100644 index 00000000..3868ac20 --- /dev/null +++ b/tools/lldb-formatters.lldb @@ -0,0 +1,2 @@ +type synthetic add -x "^Luau::Variant<.+>$" -l LuauVisualize.LuauVariantSyntheticChildrenProvider +type summary add -x "^Luau::Variant<.+>$" -l LuauVisualize.luau_variant_summary From a44b7906b62273739d77ff8fe4179c08cec4acc5 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Fri, 11 Mar 2022 08:55:02 -0800 Subject: [PATCH 192/399] Sync to upstream/release/518 (#419) --- Analysis/include/Luau/TxnLog.h | 55 - Analysis/include/Luau/TypePack.h | 1 - Analysis/include/Luau/Unifier.h | 1 - Analysis/src/Autocomplete.cpp | 56 +- Analysis/src/BuiltinDefinitions.cpp | 17 +- Analysis/src/TxnLog.cpp | 117 -- Analysis/src/TypeInfer.cpp | 672 ++++------ Analysis/src/TypePack.cpp | 26 +- Analysis/src/Unifier.cpp | 1826 +++++++-------------------- VM/include/lua.h | 35 +- VM/include/luaconf.h | 27 - VM/include/lualib.h | 4 +- VM/src/laux.cpp | 15 + VM/src/lbaselib.cpp | 11 +- VM/src/lgc.h | 7 + tests/Autocomplete.test.cpp | 52 +- tests/TypeInfer.aliases.test.cpp | 23 + tests/TypeInfer.builtins.test.cpp | 51 +- tests/TypeInfer.generics.test.cpp | 45 + tests/TypeInfer.tables.test.cpp | 50 + tests/TypeInfer.test.cpp | 38 +- tests/TypeInfer.tryUnify.test.cpp | 44 +- tests/TypeInfer.typePacks.cpp | 13 +- tests/TypeInfer.unionTypes.test.cpp | 10 +- tools/LuauVisualize.py | 107 ++ tools/lldb-formatters.lldb | 2 + 26 files changed, 1137 insertions(+), 2168 deletions(-) create mode 100644 tools/LuauVisualize.py create mode 100644 tools/lldb-formatters.lldb diff --git a/Analysis/include/Luau/TxnLog.h b/Analysis/include/Luau/TxnLog.h index f8105383..c8ebaaeb 100644 --- a/Analysis/include/Luau/TxnLog.h +++ b/Analysis/include/Luau/TxnLog.h @@ -14,61 +14,6 @@ namespace Luau using TypeOrPackId = const void*; -// Log of where what TypeIds we are rebinding and what they used to be -// Remove with LuauUseCommitTxnLog -struct DEPRECATED_TxnLog -{ - DEPRECATED_TxnLog() - : originalSeenSize(0) - , ownedSeen() - , sharedSeen(&ownedSeen) - { - } - - explicit DEPRECATED_TxnLog(std::vector>* sharedSeen) - : originalSeenSize(sharedSeen->size()) - , ownedSeen() - , sharedSeen(sharedSeen) - { - } - - DEPRECATED_TxnLog(const DEPRECATED_TxnLog&) = delete; - DEPRECATED_TxnLog& operator=(const DEPRECATED_TxnLog&) = delete; - - DEPRECATED_TxnLog(DEPRECATED_TxnLog&&) = default; - DEPRECATED_TxnLog& operator=(DEPRECATED_TxnLog&&) = default; - - void operator()(TypeId a); - void operator()(TypePackId a); - void operator()(TableTypeVar* a); - - void rollback(); - - void concat(DEPRECATED_TxnLog rhs); - - bool haveSeen(TypeId lhs, TypeId rhs); - void pushSeen(TypeId lhs, TypeId rhs); - void popSeen(TypeId lhs, TypeId rhs); - - bool haveSeen(TypePackId lhs, TypePackId rhs); - void pushSeen(TypePackId lhs, TypePackId rhs); - void popSeen(TypePackId lhs, TypePackId rhs); - -private: - std::vector> typeVarChanges; - std::vector> typePackChanges; - std::vector>> tableChanges; - size_t originalSeenSize; - - bool haveSeen(TypeOrPackId lhs, TypeOrPackId rhs); - void pushSeen(TypeOrPackId lhs, TypeOrPackId rhs); - void popSeen(TypeOrPackId lhs, TypeOrPackId rhs); - -public: - std::vector> ownedSeen; // used to avoid infinite recursion when types are cyclic - std::vector>* sharedSeen; // shared with all the descendent logs -}; - // Pending state for a TypeVar. Generated by a TxnLog and committed via // TxnLog::commit. struct PendingType diff --git a/Analysis/include/Luau/TypePack.h b/Analysis/include/Luau/TypePack.h index c74bad11..946be356 100644 --- a/Analysis/include/Luau/TypePack.h +++ b/Analysis/include/Luau/TypePack.h @@ -105,7 +105,6 @@ private: const TypePack* tp = nullptr; size_t currentIndex = 0; - // Only used if LuauUseCommittingTxnLog is true. const TxnLog* log; }; diff --git a/Analysis/include/Luau/Unifier.h b/Analysis/include/Luau/Unifier.h index 4c0462fe..71958f4a 100644 --- a/Analysis/include/Luau/Unifier.h +++ b/Analysis/include/Luau/Unifier.h @@ -45,7 +45,6 @@ struct Unifier TypeArena* const types; Mode mode; - DEPRECATED_TxnLog DEPRECATED_log; TxnLog log; ErrorVec errors; Location location; diff --git a/Analysis/src/Autocomplete.cpp b/Analysis/src/Autocomplete.cpp index c3de8d0e..e94c432f 100644 --- a/Analysis/src/Autocomplete.cpp +++ b/Analysis/src/Autocomplete.cpp @@ -13,9 +13,6 @@ #include #include -LUAU_FASTFLAG(LuauUseCommittingTxnLog) -LUAU_FASTFLAGVARIABLE(LuauAutocompleteAvoidMutation, false); -LUAU_FASTFLAGVARIABLE(LuauMissingFollowACMetatables, false); LUAU_FASTFLAGVARIABLE(LuauIfElseExprFixCompletionIssue, false); static const std::unordered_set kStatementStartingKeywords = { @@ -240,28 +237,9 @@ static TypeCorrectKind checkTypeCorrectKind(const Module& module, TypeArena* typ UnifierSharedState unifierState(&iceReporter); Unifier unifier(typeArena, Mode::Strict, Location(), Variance::Covariant, unifierState); - if (FFlag::LuauAutocompleteAvoidMutation && !FFlag::LuauUseCommittingTxnLog) - { - SeenTypes seenTypes; - SeenTypePacks seenTypePacks; - CloneState cloneState; - superTy = clone(superTy, *typeArena, seenTypes, seenTypePacks, cloneState); - subTy = clone(subTy, *typeArena, seenTypes, seenTypePacks, cloneState); - - auto errors = unifier.canUnify(subTy, superTy); - return errors.empty(); - } - else - { - unifier.tryUnify(subTy, superTy); - - bool ok = unifier.errors.empty(); - - if (!FFlag::LuauUseCommittingTxnLog) - unifier.DEPRECATED_log.rollback(); - - return ok; - } + unifier.tryUnify(subTy, superTy); + bool ok = unifier.errors.empty(); + return ok; }; auto typeAtPosition = findExpectedTypeAt(module, node, position); @@ -403,28 +381,14 @@ static void autocompleteProps(const Module& module, TypeArena* typeArena, TypeId auto indexIt = mtable->props.find("__index"); if (indexIt != mtable->props.end()) { - if (FFlag::LuauMissingFollowACMetatables) + TypeId followed = follow(indexIt->second.type); + if (get(followed) || get(followed)) + autocompleteProps(module, typeArena, followed, indexType, nodes, result, seen); + else if (auto indexFunction = get(followed)) { - TypeId followed = follow(indexIt->second.type); - if (get(followed) || get(followed)) - autocompleteProps(module, typeArena, followed, indexType, nodes, result, seen); - else if (auto indexFunction = get(followed)) - { - std::optional indexFunctionResult = first(indexFunction->retType); - if (indexFunctionResult) - autocompleteProps(module, typeArena, *indexFunctionResult, indexType, nodes, result, seen); - } - } - else - { - if (get(indexIt->second.type) || get(indexIt->second.type)) - autocompleteProps(module, typeArena, indexIt->second.type, indexType, nodes, result, seen); - else if (auto indexFunction = get(indexIt->second.type)) - { - std::optional indexFunctionResult = first(indexFunction->retType); - if (indexFunctionResult) - autocompleteProps(module, typeArena, *indexFunctionResult, indexType, nodes, result, seen); - } + std::optional indexFunctionResult = first(indexFunction->retType); + if (indexFunctionResult) + autocompleteProps(module, typeArena, *indexFunctionResult, indexType, nodes, result, seen); } } } diff --git a/Analysis/src/BuiltinDefinitions.cpp b/Analysis/src/BuiltinDefinitions.cpp index e4e5dab8..bf9ef303 100644 --- a/Analysis/src/BuiltinDefinitions.cpp +++ b/Analysis/src/BuiltinDefinitions.cpp @@ -10,6 +10,7 @@ LUAU_FASTFLAG(LuauAssertStripsFalsyTypes) LUAU_FASTFLAGVARIABLE(LuauTableCloneType, false) +LUAU_FASTFLAGVARIABLE(LuauSetMetaTableArgsCheck, false) /** FIXME: Many of these type definitions are not quite completely accurate. * @@ -376,11 +377,19 @@ static std::optional> magicFunctionSetMetaTable( TypeId mtTy = arena.addType(mtv); - AstExpr* targetExpr = expr.args.data[0]; - if (AstExprLocal* targetLocal = targetExpr->as()) + if (FFlag::LuauSetMetaTableArgsCheck && expr.args.size < 1) { - const Name targetName(targetLocal->local->name.value); - scope->bindings[targetLocal->local] = Binding{mtTy, expr.location}; + return ExprResult{}; + } + + if (!FFlag::LuauSetMetaTableArgsCheck || !expr.self) + { + AstExpr* targetExpr = expr.args.data[0]; + if (AstExprLocal* targetLocal = targetExpr->as()) + { + const Name targetName(targetLocal->local->name.value); + scope->bindings[targetLocal->local] = Binding{mtTy, expr.location}; + } } return ExprResult{arena.addTypePack({mtTy})}; diff --git a/Analysis/src/TxnLog.cpp b/Analysis/src/TxnLog.cpp index c7bf1e62..876f5f05 100644 --- a/Analysis/src/TxnLog.cpp +++ b/Analysis/src/TxnLog.cpp @@ -7,110 +7,9 @@ #include #include -LUAU_FASTFLAGVARIABLE(LuauUseCommittingTxnLog, false) - namespace Luau { -void DEPRECATED_TxnLog::operator()(TypeId a) -{ - LUAU_ASSERT(!FFlag::LuauUseCommittingTxnLog); - typeVarChanges.emplace_back(a, *a); -} - -void DEPRECATED_TxnLog::operator()(TypePackId a) -{ - LUAU_ASSERT(!FFlag::LuauUseCommittingTxnLog); - typePackChanges.emplace_back(a, *a); -} - -void DEPRECATED_TxnLog::operator()(TableTypeVar* a) -{ - LUAU_ASSERT(!FFlag::LuauUseCommittingTxnLog); - tableChanges.emplace_back(a, a->boundTo); -} - -void DEPRECATED_TxnLog::rollback() -{ - LUAU_ASSERT(!FFlag::LuauUseCommittingTxnLog); - for (auto it = typeVarChanges.rbegin(); it != typeVarChanges.rend(); ++it) - std::swap(*asMutable(it->first), it->second); - - for (auto it = typePackChanges.rbegin(); it != typePackChanges.rend(); ++it) - std::swap(*asMutable(it->first), it->second); - - for (auto it = tableChanges.rbegin(); it != tableChanges.rend(); ++it) - std::swap(it->first->boundTo, it->second); - - LUAU_ASSERT(originalSeenSize <= sharedSeen->size()); - sharedSeen->resize(originalSeenSize); -} - -void DEPRECATED_TxnLog::concat(DEPRECATED_TxnLog rhs) -{ - LUAU_ASSERT(!FFlag::LuauUseCommittingTxnLog); - typeVarChanges.insert(typeVarChanges.end(), rhs.typeVarChanges.begin(), rhs.typeVarChanges.end()); - rhs.typeVarChanges.clear(); - - typePackChanges.insert(typePackChanges.end(), rhs.typePackChanges.begin(), rhs.typePackChanges.end()); - rhs.typePackChanges.clear(); - - tableChanges.insert(tableChanges.end(), rhs.tableChanges.begin(), rhs.tableChanges.end()); - rhs.tableChanges.clear(); -} - -bool DEPRECATED_TxnLog::haveSeen(TypeId lhs, TypeId rhs) -{ - return haveSeen((TypeOrPackId)lhs, (TypeOrPackId)rhs); -} - -void DEPRECATED_TxnLog::pushSeen(TypeId lhs, TypeId rhs) -{ - pushSeen((TypeOrPackId)lhs, (TypeOrPackId)rhs); -} - -void DEPRECATED_TxnLog::popSeen(TypeId lhs, TypeId rhs) -{ - popSeen((TypeOrPackId)lhs, (TypeOrPackId)rhs); -} - -bool DEPRECATED_TxnLog::haveSeen(TypePackId lhs, TypePackId rhs) -{ - return haveSeen((TypeOrPackId)lhs, (TypeOrPackId)rhs); -} - -void DEPRECATED_TxnLog::pushSeen(TypePackId lhs, TypePackId rhs) -{ - pushSeen((TypeOrPackId)lhs, (TypeOrPackId)rhs); -} - -void DEPRECATED_TxnLog::popSeen(TypePackId lhs, TypePackId rhs) -{ - popSeen((TypeOrPackId)lhs, (TypeOrPackId)rhs); -} - -bool DEPRECATED_TxnLog::haveSeen(TypeOrPackId lhs, TypeOrPackId rhs) -{ - LUAU_ASSERT(!FFlag::LuauUseCommittingTxnLog); - const std::pair sortedPair = (lhs > rhs) ? std::make_pair(lhs, rhs) : std::make_pair(rhs, lhs); - return (sharedSeen->end() != std::find(sharedSeen->begin(), sharedSeen->end(), sortedPair)); -} - -void DEPRECATED_TxnLog::pushSeen(TypeOrPackId lhs, TypeOrPackId rhs) -{ - LUAU_ASSERT(!FFlag::LuauUseCommittingTxnLog); - const std::pair sortedPair = (lhs > rhs) ? std::make_pair(lhs, rhs) : std::make_pair(rhs, lhs); - sharedSeen->push_back(sortedPair); -} - -void DEPRECATED_TxnLog::popSeen(TypeOrPackId lhs, TypeOrPackId rhs) -{ - LUAU_ASSERT(!FFlag::LuauUseCommittingTxnLog); - const std::pair sortedPair = (lhs > rhs) ? std::make_pair(lhs, rhs) : std::make_pair(rhs, lhs); - LUAU_ASSERT(sortedPair == sharedSeen->back()); - sharedSeen->pop_back(); -} - const std::string nullPendingResult = ""; std::string toString(PendingType* pending) @@ -170,8 +69,6 @@ const TxnLog* TxnLog::empty() void TxnLog::concat(TxnLog rhs) { - LUAU_ASSERT(FFlag::LuauUseCommittingTxnLog); - for (auto& [ty, rep] : rhs.typeVarChanges) typeVarChanges[ty] = std::move(rep); @@ -181,8 +78,6 @@ void TxnLog::concat(TxnLog rhs) void TxnLog::commit() { - LUAU_ASSERT(FFlag::LuauUseCommittingTxnLog); - for (auto& [ty, rep] : typeVarChanges) *asMutable(ty) = rep.get()->pending; @@ -194,16 +89,12 @@ void TxnLog::commit() void TxnLog::clear() { - LUAU_ASSERT(FFlag::LuauUseCommittingTxnLog); - typeVarChanges.clear(); typePackChanges.clear(); } TxnLog TxnLog::inverse() { - LUAU_ASSERT(FFlag::LuauUseCommittingTxnLog); - TxnLog inversed(sharedSeen); for (auto& [ty, _rep] : typeVarChanges) @@ -247,8 +138,6 @@ void TxnLog::popSeen(TypePackId lhs, TypePackId rhs) bool TxnLog::haveSeen(TypeOrPackId lhs, TypeOrPackId rhs) const { - LUAU_ASSERT(FFlag::LuauUseCommittingTxnLog); - const std::pair sortedPair = (lhs > rhs) ? std::make_pair(lhs, rhs) : std::make_pair(rhs, lhs); if (sharedSeen->end() != std::find(sharedSeen->begin(), sharedSeen->end(), sortedPair)) { @@ -265,16 +154,12 @@ bool TxnLog::haveSeen(TypeOrPackId lhs, TypeOrPackId rhs) const void TxnLog::pushSeen(TypeOrPackId lhs, TypeOrPackId rhs) { - LUAU_ASSERT(FFlag::LuauUseCommittingTxnLog); - const std::pair sortedPair = (lhs > rhs) ? std::make_pair(lhs, rhs) : std::make_pair(rhs, lhs); sharedSeen->push_back(sortedPair); } void TxnLog::popSeen(TypeOrPackId lhs, TypeOrPackId rhs) { - LUAU_ASSERT(FFlag::LuauUseCommittingTxnLog); - const std::pair sortedPair = (lhs > rhs) ? std::make_pair(lhs, rhs) : std::make_pair(rhs, lhs); LUAU_ASSERT(sortedPair == sharedSeen->back()); sharedSeen->pop_back(); @@ -282,7 +167,6 @@ void TxnLog::popSeen(TypeOrPackId lhs, TypeOrPackId rhs) PendingType* TxnLog::queue(TypeId ty) { - LUAU_ASSERT(FFlag::LuauUseCommittingTxnLog); LUAU_ASSERT(!ty->persistent); // Explicitly don't look in ancestors. If we have discovered something new @@ -296,7 +180,6 @@ PendingType* TxnLog::queue(TypeId ty) PendingTypePack* TxnLog::queue(TypePackId tp) { - LUAU_ASSERT(FFlag::LuauUseCommittingTxnLog); LUAU_ASSERT(!tp->persistent); // Explicitly don't look in ancestors. If we have discovered something new diff --git a/Analysis/src/TypeInfer.cpp b/Analysis/src/TypeInfer.cpp index 8e6b3b52..3fe4c90e 100644 --- a/Analysis/src/TypeInfer.cpp +++ b/Analysis/src/TypeInfer.cpp @@ -24,7 +24,6 @@ LUAU_FASTINTVARIABLE(LuauCheckRecursionLimit, 500) LUAU_FASTFLAG(LuauKnowsTheDataModel3) LUAU_FASTFLAGVARIABLE(LuauEqConstraint, false) LUAU_FASTFLAGVARIABLE(LuauWeakEqConstraint, false) // Eventually removed as false. -LUAU_FASTFLAG(LuauUseCommittingTxnLog) LUAU_FASTFLAGVARIABLE(DebugLuauFreezeDuringUnification, false) LUAU_FASTFLAGVARIABLE(LuauRecursiveTypeParameterRestriction, false) LUAU_FASTFLAGVARIABLE(LuauGenericFunctionsDontCacheTypeParams, false) @@ -36,6 +35,7 @@ LUAU_FASTFLAGVARIABLE(LuauExpectedTypesOfProperties, false) LUAU_FASTFLAGVARIABLE(LuauErrorRecoveryType, false) LUAU_FASTFLAGVARIABLE(LuauOnlyMutateInstantiatedTables, false) LUAU_FASTFLAGVARIABLE(LuauPropertiesGetExpectedType, false) +LUAU_FASTFLAGVARIABLE(LuauStatFunctionSimplify, false) LUAU_FASTFLAGVARIABLE(LuauUnsealedTableLiteral, false) LUAU_FASTFLAGVARIABLE(LuauTwoPassAliasDefinitionFix, false) LUAU_FASTFLAGVARIABLE(LuauAssertStripsFalsyTypes, false) @@ -43,6 +43,8 @@ LUAU_FASTFLAGVARIABLE(LuauReturnAnyInsteadOfICE, false) // Eventually removed as LUAU_FASTFLAG(LuauWidenIfSupertypeIsFree) LUAU_FASTFLAGVARIABLE(LuauDoNotTryToReduce, false) LUAU_FASTFLAGVARIABLE(LuauDoNotAccidentallyDependOnPointerOrdering, false) +LUAU_FASTFLAGVARIABLE(LuauFixArgumentCountMismatchAmountWithGenericTypes, false) +LUAU_FASTFLAGVARIABLE(LuauFixIncorrectLineNumberDuplicateType, false) namespace Luau { @@ -652,18 +654,15 @@ ErrorVec TypeChecker::tryUnify_(Id subTy, Id superTy, const Location& location) { Unifier state = mkUnifier(location); - if (FFlag::LuauUseCommittingTxnLog && FFlag::DebugLuauFreezeDuringUnification) + if (FFlag::DebugLuauFreezeDuringUnification) freeze(currentModule->internalTypes); state.tryUnify(subTy, superTy); - if (FFlag::LuauUseCommittingTxnLog && FFlag::DebugLuauFreezeDuringUnification) + if (FFlag::DebugLuauFreezeDuringUnification) unfreeze(currentModule->internalTypes); - if (!state.errors.empty() && !FFlag::LuauUseCommittingTxnLog) - state.DEPRECATED_log.rollback(); - - if (state.errors.empty() && FFlag::LuauUseCommittingTxnLog) + if (state.errors.empty()) state.log.commit(); return state.errors; @@ -847,8 +846,7 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatLocal& local) state.tryUnify(valuePack, variablePack); reportErrors(state.errors); - if (FFlag::LuauUseCommittingTxnLog) - state.log.commit(); + state.log.commit(); // In the code 'local T = {}', we wish to ascribe the name 'T' to the type of the table for error-reporting purposes. // We also want to do this for 'local T = setmetatable(...)'. @@ -1040,8 +1038,7 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatForIn& forin) Unifier state = mkUnifier(firstValue->location); checkArgumentList(loopScope, state, argPack, iterFunc->argTypes, /*argLocations*/ {}); - if (FFlag::LuauUseCommittingTxnLog) - state.log.commit(); + state.log.commit(); reportErrors(state.errors); } @@ -1102,8 +1099,53 @@ void TypeChecker::check(const ScopePtr& scope, TypeId ty, const ScopePtr& funSco scope->bindings[name->local] = {anyIfNonstrict(quantify(funScope, ty, name->local->location)), name->local->location}; return; } + else if (auto name = function.name->as(); name && FFlag::LuauStatFunctionSimplify) + { + TypeId exprTy = checkExpr(scope, *name->expr).type; + TableTypeVar* ttv = getMutableTableType(exprTy); + if (!ttv) + { + if (isTableIntersection(exprTy)) + reportError(TypeError{function.location, CannotExtendTable{exprTy, CannotExtendTable::Property, name->index.value}}); + else if (!get(exprTy) && !get(exprTy)) + reportError(TypeError{function.location, OnlyTablesCanHaveMethods{exprTy}}); + } + else if (ttv->state == TableState::Sealed) + reportError(TypeError{function.location, CannotExtendTable{exprTy, CannotExtendTable::Property, name->index.value}}); + + ty = follow(ty); + + if (ttv && ttv->state != TableState::Sealed) + ttv->props[name->index.value] = {ty, /* deprecated */ false, {}, name->indexLocation}; + + if (function.func->self) + { + const FunctionTypeVar* funTy = get(ty); + if (!funTy) + ice("Methods should be functions"); + + std::optional arg0 = first(funTy->argTypes); + if (!arg0) + ice("Methods should always have at least 1 argument (self)"); + } + + checkFunctionBody(funScope, ty, *function.func); + + if (ttv && ttv->state != TableState::Sealed) + ttv->props[name->index.value] = {follow(quantify(funScope, ty, name->indexLocation)), /* deprecated */ false, {}, name->indexLocation}; + } + else if (FFlag::LuauStatFunctionSimplify) + { + LUAU_ASSERT(function.name->is()); + + ty = follow(ty); + + checkFunctionBody(funScope, ty, *function.func); + } else if (function.func->self) { + LUAU_ASSERT(!FFlag::LuauStatFunctionSimplify); + AstExprIndexName* indexName = function.name->as(); if (!indexName) ice("member function declaration has malformed name expression"); @@ -1141,6 +1183,8 @@ void TypeChecker::check(const ScopePtr& scope, TypeId ty, const ScopePtr& funSco } else { + LUAU_ASSERT(!FFlag::LuauStatFunctionSimplify); + TypeId leftType = checkLValueBinding(scope, *function.name); checkFunctionBody(funScope, ty, *function.func); @@ -1217,6 +1261,9 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatTypeAlias& typealias LUAU_ASSERT(ftv); ftv->forwardedTypeAlias = true; bindingsMap[name] = {std::move(generics), std::move(genericPacks), ty}; + + if (FFlag::LuauFixIncorrectLineNumberDuplicateType) + scope->typeAliasLocations[name] = typealias.location; } } else @@ -2102,9 +2149,7 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprUn Unifier state = mkUnifier(expr.location); state.tryUnify(actualFunctionType, expectedFunctionType, /*isFunctionCall*/ true); - - if (FFlag::LuauUseCommittingTxnLog) - state.log.commit(); + state.log.commit(); TypeId retType = first(retTypePack).value_or(nilType); if (!state.errors.empty()) @@ -2283,9 +2328,7 @@ TypeId TypeChecker::checkRelationalOperation( if (!isEquality) { state.tryUnify(rhsType, lhsType); - - if (FFlag::LuauUseCommittingTxnLog) - state.log.commit(); + state.log.commit(); } bool needsMetamethod = !isEquality; @@ -2336,8 +2379,7 @@ TypeId TypeChecker::checkRelationalOperation( return errorRecoveryType(booleanType); } - if (FFlag::LuauUseCommittingTxnLog) - state.log.commit(); + state.log.commit(); } } @@ -2347,8 +2389,7 @@ TypeId TypeChecker::checkRelationalOperation( state.tryUnify( instantiate(scope, actualFunctionType, expr.location), instantiate(scope, *metamethod, expr.location), /*isFunctionCall*/ true); - if (FFlag::LuauUseCommittingTxnLog) - state.log.commit(); + state.log.commit(); reportErrors(state.errors); return booleanType; @@ -2464,25 +2505,15 @@ TypeId TypeChecker::checkBinaryOperation( TypePackId fallbackArguments = freshTypePack(scope); TypeId fallbackFunctionType = addType(FunctionTypeVar(scope->level, fallbackArguments, retTypePack)); state.errors.clear(); - - if (FFlag::LuauUseCommittingTxnLog) - { - state.log.clear(); - } - else - { - state.DEPRECATED_log.rollback(); - } + state.log.clear(); state.tryUnify(actualFunctionType, fallbackFunctionType, /*isFunctionCall*/ true); - if (FFlag::LuauUseCommittingTxnLog && state.errors.empty()) + if (state.errors.empty()) state.log.commit(); - else if (!state.errors.empty() && !FFlag::LuauUseCommittingTxnLog) - state.DEPRECATED_log.rollback(); } - if (FFlag::LuauUseCommittingTxnLog && !hasErrors) + if (!hasErrors) { state.log.commit(); } @@ -2729,13 +2760,11 @@ TypeId TypeChecker::checkLValueBinding(const ScopePtr& scope, const AstExprIndex TypeId retType = indexer->indexResultType; if (!state.errors.empty()) { - if (!FFlag::LuauUseCommittingTxnLog) - state.DEPRECATED_log.rollback(); reportError(expr.location, UnknownProperty{lhs, name}); retType = errorRecoveryType(retType); } - else if (FFlag::LuauUseCommittingTxnLog) + else state.log.commit(); return retType; @@ -3209,7 +3238,7 @@ ExprResult TypeChecker::checkExprPack(const ScopePtr& scope, const A } // Returns the minimum number of arguments the argument list can accept. -static size_t getMinParameterCount(TypePackId tp) +static size_t getMinParameterCount_DEPRECATED(TypePackId tp) { size_t minCount = 0; size_t optionalCount = 0; @@ -3235,6 +3264,32 @@ static size_t getMinParameterCount(TypePackId tp) return minCount; } +static size_t getMinParameterCount(TxnLog* log, TypePackId tp) +{ + size_t minCount = 0; + size_t optionalCount = 0; + + auto it = begin(tp, log); + auto endIter = end(tp); + + while (it != endIter) + { + TypeId ty = *it; + if (isOptional(ty)) + ++optionalCount; + else + { + minCount += optionalCount; + optionalCount = 0; + minCount++; + } + + ++it; + } + + return minCount; +} + void TypeChecker::checkArgumentList( const ScopePtr& scope, Unifier& state, TypePackId argPack, TypePackId paramPack, const std::vector& argLocations) { @@ -3248,396 +3303,199 @@ void TypeChecker::checkArgumentList( size_t paramIndex = 0; - size_t minParams = getMinParameterCount(paramPack); + size_t minParams = FFlag::LuauFixIncorrectLineNumberDuplicateType ? 0 : getMinParameterCount_DEPRECATED(paramPack); - if (FFlag::LuauUseCommittingTxnLog) + while (true) { - while (true) + state.location = paramIndex < argLocations.size() ? argLocations[paramIndex] : state.location; + + if (argIter == endIter && paramIter == endIter) { - state.location = paramIndex < argLocations.size() ? argLocations[paramIndex] : state.location; + std::optional argTail = argIter.tail(); + std::optional paramTail = paramIter.tail(); - if (argIter == endIter && paramIter == endIter) + // If we hit the end of both type packs simultaneously, then there are definitely no further type + // errors to report. All we need to do is tie up any free tails. + // + // If one side has a free tail and the other has none at all, we create an empty pack and bind the + // free tail to that. + + if (argTail) { - std::optional argTail = argIter.tail(); - std::optional paramTail = paramIter.tail(); - - // If we hit the end of both type packs simultaneously, then there are definitely no further type - // errors to report. All we need to do is tie up any free tails. - // - // If one side has a free tail and the other has none at all, we create an empty pack and bind the - // free tail to that. - - if (argTail) + if (state.log.getMutable(state.log.follow(*argTail))) { - if (state.log.getMutable(state.log.follow(*argTail))) - { - if (paramTail) - state.tryUnify(*paramTail, *argTail); - else - state.log.replace(*argTail, TypePackVar(TypePack{{}})); - } - } - else if (paramTail) - { - // argTail is definitely empty - if (state.log.getMutable(state.log.follow(*paramTail))) - state.log.replace(*paramTail, TypePackVar(TypePack{{}})); - } - - return; - } - else if (argIter == endIter) - { - // Not enough arguments. - - // Might be ok if we are forwarding a vararg along. This is a common thing to occur in nonstrict mode. - if (argIter.tail()) - { - TypePackId tail = *argIter.tail(); - if (state.log.getMutable(tail)) - { - // Unify remaining parameters so we don't leave any free-types hanging around. - while (paramIter != endIter) - { - state.tryUnify(errorRecoveryType(anyType), *paramIter); - ++paramIter; - } - return; - } - else if (auto vtp = state.log.getMutable(tail)) - { - while (paramIter != endIter) - { - state.tryUnify(vtp->ty, *paramIter); - ++paramIter; - } - - return; - } - else if (state.log.getMutable(tail)) - { - std::vector rest; - rest.reserve(std::distance(paramIter, endIter)); - while (paramIter != endIter) - { - rest.push_back(*paramIter); - ++paramIter; - } - - TypePackId varPack = addTypePack(TypePackVar{TypePack{rest, paramIter.tail()}}); - state.tryUnify(varPack, tail); - return; - } - } - - // If any remaining unfulfilled parameters are nonoptional, this is a problem. - while (paramIter != endIter) - { - TypeId t = state.log.follow(*paramIter); - if (isOptional(t)) - { - } // ok - else if (state.log.getMutable(t)) - { - } // ok - else if (isNonstrictMode() && state.log.getMutable(t)) - { - } // ok + if (paramTail) + state.tryUnify(*paramTail, *argTail); else - { - state.reportError(TypeError{state.location, CountMismatch{minParams, paramIndex}}); - return; - } - ++paramIter; + state.log.replace(*argTail, TypePackVar(TypePack{{}})); } } - else if (paramIter == endIter) + else if (paramTail) { - // too many parameters passed - if (!paramIter.tail()) - { - while (argIter != endIter) - { - // The use of unify here is deliberate. We don't want this unification - // to be undoable. - unify(errorRecoveryType(scope), *argIter, state.location); - ++argIter; - } - // For this case, we want the error span to cover every errant extra parameter - Location location = state.location; - if (!argLocations.empty()) - location = {state.location.begin, argLocations.back().end}; - state.reportError(TypeError{location, CountMismatch{minParams, std::distance(begin(argPack), end(argPack))}}); - return; - } - TypePackId tail = state.log.follow(*paramIter.tail()); + // argTail is definitely empty + if (state.log.getMutable(state.log.follow(*paramTail))) + state.log.replace(*paramTail, TypePackVar(TypePack{{}})); + } + return; + } + else if (argIter == endIter) + { + // Not enough arguments. + + // Might be ok if we are forwarding a vararg along. This is a common thing to occur in nonstrict mode. + if (argIter.tail()) + { + TypePackId tail = *argIter.tail(); if (state.log.getMutable(tail)) { - // Function is variadic. Ok. + // Unify remaining parameters so we don't leave any free-types hanging around. + while (paramIter != endIter) + { + state.tryUnify(errorRecoveryType(anyType), *paramIter); + ++paramIter; + } return; } else if (auto vtp = state.log.getMutable(tail)) { - // Function is variadic and requires that all subsequent parameters - // be compatible with a type. - size_t argIndex = paramIndex; - while (argIter != endIter) + while (paramIter != endIter) { - Location location = state.location; - - if (argIndex < argLocations.size()) - location = argLocations[argIndex]; - - unify(*argIter, vtp->ty, location); - ++argIter; - ++argIndex; + state.tryUnify(vtp->ty, *paramIter); + ++paramIter; } return; } else if (state.log.getMutable(tail)) { - // Create a type pack out of the remaining argument types - // and unify it with the tail. std::vector rest; - rest.reserve(std::distance(argIter, endIter)); - while (argIter != endIter) + rest.reserve(std::distance(paramIter, endIter)); + while (paramIter != endIter) { - rest.push_back(*argIter); - ++argIter; + rest.push_back(*paramIter); + ++paramIter; } - TypePackId varPack = addTypePack(TypePackVar{TypePack{rest, argIter.tail()}}); + TypePackId varPack = addTypePack(TypePackVar{TypePack{rest, paramIter.tail()}}); state.tryUnify(varPack, tail); return; } - else if (state.log.getMutable(tail)) - { - state.log.replace(tail, TypePackVar(TypePack{{}})); - return; - } - else if (state.log.getMutable(tail)) - { - // For this case, we want the error span to cover every errant extra parameter - Location location = state.location; - if (!argLocations.empty()) - location = {state.location.begin, argLocations.back().end}; - // TODO: Better error message? - state.reportError(TypeError{location, CountMismatch{minParams, std::distance(begin(argPack), end(argPack))}}); - return; - } } - else + + // If any remaining unfulfilled parameters are nonoptional, this is a problem. + while (paramIter != endIter) { - unifyWithInstantiationIfNeeded(scope, *argIter, *paramIter, state); - ++argIter; + TypeId t = state.log.follow(*paramIter); + if (isOptional(t)) + { + } // ok + else if (state.log.getMutable(t)) + { + } // ok + else if (isNonstrictMode() && state.log.getMutable(t)) + { + } // ok + else + { + if (FFlag::LuauFixArgumentCountMismatchAmountWithGenericTypes) + minParams = getMinParameterCount(&state.log, paramPack); + state.reportError(TypeError{state.location, CountMismatch{minParams, paramIndex}}); + return; + } ++paramIter; } - - ++paramIndex; } - } - else - { - while (true) + else if (paramIter == endIter) { - state.location = paramIndex < argLocations.size() ? argLocations[paramIndex] : state.location; - - if (argIter == endIter && paramIter == endIter) + // too many parameters passed + if (!paramIter.tail()) { - std::optional argTail = argIter.tail(); - std::optional paramTail = paramIter.tail(); - - // If we hit the end of both type packs simultaneously, then there are definitely no further type - // errors to report. All we need to do is tie up any free tails. - // - // If one side has a free tail and the other has none at all, we create an empty pack and bind the - // free tail to that. - - if (argTail) + while (argIter != endIter) { - if (get(*argTail)) - { - if (paramTail) - state.tryUnify(*paramTail, *argTail); - else - { - state.DEPRECATED_log(*argTail); - *asMutable(*argTail) = TypePack{{}}; - } - } + // The use of unify here is deliberate. We don't want this unification + // to be undoable. + unify(errorRecoveryType(scope), *argIter, state.location); + ++argIter; } - else if (paramTail) + // For this case, we want the error span to cover every errant extra parameter + Location location = state.location; + if (!argLocations.empty()) + location = {state.location.begin, argLocations.back().end}; + + if (FFlag::LuauFixArgumentCountMismatchAmountWithGenericTypes) + minParams = getMinParameterCount(&state.log, paramPack); + state.reportError(TypeError{location, CountMismatch{minParams, std::distance(begin(argPack), end(argPack))}}); + return; + } + TypePackId tail = state.log.follow(*paramIter.tail()); + + if (state.log.getMutable(tail)) + { + // Function is variadic. Ok. + return; + } + else if (auto vtp = state.log.getMutable(tail)) + { + // Function is variadic and requires that all subsequent parameters + // be compatible with a type. + size_t argIndex = paramIndex; + while (argIter != endIter) { - // argTail is definitely empty - if (get(*paramTail)) - { - state.DEPRECATED_log(*paramTail); - *asMutable(*paramTail) = TypePack{{}}; - } + Location location = state.location; + + if (argIndex < argLocations.size()) + location = argLocations[argIndex]; + + unify(*argIter, vtp->ty, location); + ++argIter; + ++argIndex; } return; } - else if (argIter == endIter) + else if (state.log.getMutable(tail)) { - // Not enough arguments. - - // Might be ok if we are forwarding a vararg along. This is a common thing to occur in nonstrict mode. - if (argIter.tail()) + // Create a type pack out of the remaining argument types + // and unify it with the tail. + std::vector rest; + rest.reserve(std::distance(argIter, endIter)); + while (argIter != endIter) { - TypePackId tail = *argIter.tail(); - if (get(tail)) - { - // Unify remaining parameters so we don't leave any free-types hanging around. - while (paramIter != endIter) - { - state.tryUnify(*paramIter, errorRecoveryType(anyType)); - ++paramIter; - } - return; - } - else if (auto vtp = get(tail)) - { - while (paramIter != endIter) - { - state.tryUnify(*paramIter, vtp->ty); - ++paramIter; - } - - return; - } - else if (get(tail)) - { - std::vector rest; - rest.reserve(std::distance(paramIter, endIter)); - while (paramIter != endIter) - { - rest.push_back(*paramIter); - ++paramIter; - } - - TypePackId varPack = addTypePack(TypePackVar{TypePack{rest, paramIter.tail()}}); - state.tryUnify(varPack, tail); - return; - } + rest.push_back(*argIter); + ++argIter; } - // If any remaining unfulfilled parameters are nonoptional, this is a problem. - while (paramIter != endIter) - { - TypeId t = follow(*paramIter); - if (isOptional(t)) - { - } // ok - else if (get(t)) - { - } // ok - else if (isNonstrictMode() && get(t)) - { - } // ok - else - { - state.reportError(TypeError{state.location, CountMismatch{minParams, paramIndex}}); - return; - } - ++paramIter; - } + TypePackId varPack = addTypePack(TypePackVar{TypePack{rest, argIter.tail()}}); + state.tryUnify(varPack, tail); + return; } - else if (paramIter == endIter) + else if (state.log.getMutable(tail)) { - // too many parameters passed - if (!paramIter.tail()) - { - while (argIter != endIter) - { - unify(*argIter, errorRecoveryType(scope), state.location); - ++argIter; - } - // For this case, we want the error span to cover every errant extra parameter - Location location = state.location; - if (!argLocations.empty()) - location = {state.location.begin, argLocations.back().end}; - state.reportError(TypeError{location, CountMismatch{minParams, std::distance(begin(argPack), end(argPack))}}); - return; - } - TypePackId tail = *paramIter.tail(); - - if (get(tail)) - { - // Function is variadic. Ok. - return; - } - else if (auto vtp = get(tail)) - { - // Function is variadic and requires that all subsequent parameters - // be compatible with a type. - size_t argIndex = paramIndex; - while (argIter != endIter) - { - Location location = state.location; - - if (argIndex < argLocations.size()) - location = argLocations[argIndex]; - - unify(*argIter, vtp->ty, location); - ++argIter; - ++argIndex; - } - - return; - } - else if (get(tail)) - { - // Create a type pack out of the remaining argument types - // and unify it with the tail. - std::vector rest; - rest.reserve(std::distance(argIter, endIter)); - while (argIter != endIter) - { - rest.push_back(*argIter); - ++argIter; - } - - TypePackId varPack = addTypePack(TypePackVar{TypePack{rest, argIter.tail()}}); - state.tryUnify(tail, varPack); - return; - } - else if (get(tail)) - { - if (FFlag::LuauUseCommittingTxnLog) - { - state.log.replace(tail, TypePackVar(TypePack{{}})); - } - else - { - state.DEPRECATED_log(tail); - *asMutable(tail) = TypePack{}; - } - - return; - } - else if (get(tail)) - { - // For this case, we want the error span to cover every errant extra parameter - Location location = state.location; - if (!argLocations.empty()) - location = {state.location.begin, argLocations.back().end}; - // TODO: Better error message? - state.reportError(TypeError{location, CountMismatch{minParams, std::distance(begin(argPack), end(argPack))}}); - return; - } + state.log.replace(tail, TypePackVar(TypePack{{}})); + return; } - else + else if (state.log.getMutable(tail)) { - unifyWithInstantiationIfNeeded(scope, *argIter, *paramIter, state); - ++argIter; - ++paramIter; + // For this case, we want the error span to cover every errant extra parameter + Location location = state.location; + if (!argLocations.empty()) + location = {state.location.begin, argLocations.back().end}; + // TODO: Better error message? + if (FFlag::LuauFixArgumentCountMismatchAmountWithGenericTypes) + minParams = getMinParameterCount(&state.log, paramPack); + state.reportError(TypeError{location, CountMismatch{minParams, std::distance(begin(argPack), end(argPack))}}); + return; } - - ++paramIndex; } + else + { + unifyWithInstantiationIfNeeded(scope, *argIter, *paramIter, state); + ++argIter; + ++paramIter; + } + + ++paramIndex; } } @@ -3882,9 +3740,6 @@ std::optional> TypeChecker::checkCallOverload(const Scope checkArgumentList(scope, state, retPack, ftv->retType, /*argLocations*/ {}); if (!state.errors.empty()) { - if (!FFlag::LuauUseCommittingTxnLog) - state.DEPRECATED_log.rollback(); - return {}; } @@ -3912,14 +3767,10 @@ std::optional> TypeChecker::checkCallOverload(const Scope overloadsThatDont.push_back(fn); errors.emplace_back(std::move(state.errors), args->head, ftv); - - if (!FFlag::LuauUseCommittingTxnLog) - state.DEPRECATED_log.rollback(); } else { - if (FFlag::LuauUseCommittingTxnLog) - state.log.commit(); + state.log.commit(); if (isNonstrictMode() && !expr.self && expr.func->is() && ftv->hasSelf) { @@ -3976,8 +3827,7 @@ bool TypeChecker::handleSelfCallMismatch(const ScopePtr& scope, const AstExprCal if (editedState.errors.empty()) { - if (FFlag::LuauUseCommittingTxnLog) - editedState.log.commit(); + editedState.log.commit(); reportError(TypeError{expr.location, FunctionDoesNotTakeSelf{}}); // This is a little bit suspect: If this overload would work with a . replaced by a : @@ -3987,8 +3837,6 @@ bool TypeChecker::handleSelfCallMismatch(const ScopePtr& scope, const AstExprCal // checkArgumentList(scope, editedState, retPack, ftv->retType, retLocations, CountMismatch::Return); return true; } - else if (!FFlag::LuauUseCommittingTxnLog) - editedState.DEPRECATED_log.rollback(); } else if (ftv->hasSelf) { @@ -4010,8 +3858,7 @@ bool TypeChecker::handleSelfCallMismatch(const ScopePtr& scope, const AstExprCal if (editedState.errors.empty()) { - if (FFlag::LuauUseCommittingTxnLog) - editedState.log.commit(); + editedState.log.commit(); reportError(TypeError{expr.location, FunctionRequiresSelf{}}); // This is a little bit suspect: If this overload would work with a : replaced by a . @@ -4021,8 +3868,6 @@ bool TypeChecker::handleSelfCallMismatch(const ScopePtr& scope, const AstExprCal // checkArgumentList(scope, editedState, retPack, ftv->retType, retLocations, CountMismatch::Return); return true; } - else if (!FFlag::LuauUseCommittingTxnLog) - editedState.DEPRECATED_log.rollback(); } } } @@ -4082,7 +3927,7 @@ void TypeChecker::reportOverloadResolutionError(const ScopePtr& scope, const Ast checkArgumentList(scope, state, argPack, ftv->argTypes, argLocations); } - if (FFlag::LuauUseCommittingTxnLog && state.errors.empty()) + if (state.errors.empty()) state.log.commit(); if (i > 0) @@ -4092,9 +3937,6 @@ void TypeChecker::reportOverloadResolutionError(const ScopePtr& scope, const Ast s += "and "; s += toString(overload); - - if (!FFlag::LuauUseCommittingTxnLog) - state.DEPRECATED_log.rollback(); } if (overloadsThatMatchArgCount.size() == 0) @@ -4168,24 +4010,16 @@ ExprResult TypeChecker::checkExprList(const ScopePtr& scope, const L // just performed. There's not a great way to pass that into checkExpr. Instead, we store // the inverse of the current log, and commit it. When we're done, we'll commit all the // inverses. This isn't optimal, and a better solution is welcome here. - if (FFlag::LuauUseCommittingTxnLog) - { - inverseLogs.push_back(state.log.inverse()); - state.log.commit(); - } + inverseLogs.push_back(state.log.inverse()); + state.log.commit(); } tp->head.push_back(actualType); } } - if (FFlag::LuauUseCommittingTxnLog) - { - for (TxnLog& log : inverseLogs) - log.commit(); - } - else - state.DEPRECATED_log.rollback(); + for (TxnLog& log : inverseLogs) + log.commit(); return {pack, predicates}; } @@ -4294,8 +4128,7 @@ bool TypeChecker::unify(TypeId subTy, TypeId superTy, const Location& location, Unifier state = mkUnifier(location); state.tryUnify(subTy, superTy, options.isFunctionCall); - if (FFlag::LuauUseCommittingTxnLog) - state.log.commit(); + state.log.commit(); reportErrors(state.errors); @@ -4308,8 +4141,7 @@ bool TypeChecker::unify(TypePackId subTy, TypePackId superTy, const Location& lo state.ctx = ctx; state.tryUnify(subTy, superTy); - if (FFlag::LuauUseCommittingTxnLog) - state.log.commit(); + state.log.commit(); reportErrors(state.errors); @@ -4321,8 +4153,7 @@ bool TypeChecker::unifyWithInstantiationIfNeeded(const ScopePtr& scope, TypeId s Unifier state = mkUnifier(location); unifyWithInstantiationIfNeeded(scope, subTy, superTy, state); - if (FFlag::LuauUseCommittingTxnLog) - state.log.commit(); + state.log.commit(); reportErrors(state.errors); @@ -4352,31 +4183,18 @@ void TypeChecker::unifyWithInstantiationIfNeeded(const ScopePtr& scope, TypeId s if (subTy == instantiated) { // Instantiating the argument made no difference, so just report any child errors - if (FFlag::LuauUseCommittingTxnLog) - state.log.concat(std::move(child.log)); - else - state.DEPRECATED_log.concat(std::move(child.DEPRECATED_log)); + state.log.concat(std::move(child.log)); state.errors.insert(state.errors.end(), child.errors.begin(), child.errors.end()); } else { - if (!FFlag::LuauUseCommittingTxnLog) - child.DEPRECATED_log.rollback(); - state.tryUnify(instantiated, superTy, /*isFunctionCall*/ false); } } else { - if (FFlag::LuauUseCommittingTxnLog) - { - state.log.concat(std::move(child.log)); - } - else - { - state.DEPRECATED_log.concat(std::move(child.DEPRECATED_log)); - } + state.log.concat(std::move(child.log)); } } } @@ -4540,7 +4358,7 @@ TypeId TypeChecker::quantify(const ScopePtr& scope, TypeId ty, Location location TypeId TypeChecker::instantiate(const ScopePtr& scope, TypeId ty, Location location, const TxnLog* log) { - Instantiation instantiation{FFlag::LuauUseCommittingTxnLog ? log : TxnLog::empty(), ¤tModule->internalTypes, scope->level}; + Instantiation instantiation{log, ¤tModule->internalTypes, scope->level}; std::optional instantiated = instantiation.substitute(ty); if (instantiated.has_value()) return *instantiated; diff --git a/Analysis/src/TypePack.cpp b/Analysis/src/TypePack.cpp index b15548a8..91123f46 100644 --- a/Analysis/src/TypePack.cpp +++ b/Analysis/src/TypePack.cpp @@ -5,8 +5,6 @@ #include -LUAU_FASTFLAG(LuauUseCommittingTxnLog) - namespace Luau { @@ -51,16 +49,8 @@ TypePackIterator::TypePackIterator(TypePackId typePack, const TxnLog* log) { while (tp && tp->head.empty()) { - if (FFlag::LuauUseCommittingTxnLog) - { - currentTypePack = tp->tail ? log->follow(*tp->tail) : nullptr; - tp = currentTypePack ? log->getMutable(currentTypePack) : nullptr; - } - else - { - currentTypePack = tp->tail ? follow(*tp->tail) : nullptr; - tp = currentTypePack ? get(currentTypePack) : nullptr; - } + currentTypePack = tp->tail ? log->follow(*tp->tail) : nullptr; + tp = currentTypePack ? log->getMutable(currentTypePack) : nullptr; } } @@ -71,16 +61,8 @@ TypePackIterator& TypePackIterator::operator++() ++currentIndex; while (tp && currentIndex >= tp->head.size()) { - if (FFlag::LuauUseCommittingTxnLog) - { - currentTypePack = tp->tail ? log->follow(*tp->tail) : nullptr; - tp = currentTypePack ? log->getMutable(currentTypePack) : nullptr; - } - else - { - currentTypePack = tp->tail ? follow(*tp->tail) : nullptr; - tp = currentTypePack ? get(currentTypePack) : nullptr; - } + currentTypePack = tp->tail ? log->follow(*tp->tail) : nullptr; + tp = currentTypePack ? log->getMutable(currentTypePack) : nullptr; currentIndex = 0; } diff --git a/Analysis/src/Unifier.cpp b/Analysis/src/Unifier.cpp index 6c29486a..7b781f26 100644 --- a/Analysis/src/Unifier.cpp +++ b/Analysis/src/Unifier.cpp @@ -15,30 +15,28 @@ LUAU_FASTINT(LuauTypeInferRecursionLimit); LUAU_FASTINT(LuauTypeInferTypePackLoopLimit); LUAU_FASTFLAG(LuauImmutableTypes) -LUAU_FASTFLAG(LuauUseCommittingTxnLog) LUAU_FASTINTVARIABLE(LuauTypeInferIterationLimit, 2000); LUAU_FASTFLAGVARIABLE(LuauTableSubtypingVariance2, false); LUAU_FASTFLAG(LuauSingletonTypes) LUAU_FASTFLAG(LuauErrorRecoveryType); LUAU_FASTFLAGVARIABLE(LuauSubtypingAddOptPropsToUnsealedTables, false) -LUAU_FASTFLAGVARIABLE(LuauFollowWithCommittingTxnLogInAnyUnification, false) LUAU_FASTFLAGVARIABLE(LuauWidenIfSupertypeIsFree, false) LUAU_FASTFLAGVARIABLE(LuauDifferentOrderOfUnificationDoesntMatter, false) -LUAU_FASTFLAGVARIABLE(LuauTxnLogSeesTypePacks2, true) +LUAU_FASTFLAGVARIABLE(LuauTxnLogSeesTypePacks2, false) +LUAU_FASTFLAGVARIABLE(LuauTxnLogCheckForInvalidation, false) +LUAU_FASTFLAGVARIABLE(LuauTxnLogDontRetryForIndexers, false) namespace Luau { struct PromoteTypeLevels { - DEPRECATED_TxnLog& DEPRECATED_log; TxnLog& log; const TypeArena* typeArena = nullptr; TypeLevel minLevel; - explicit PromoteTypeLevels(DEPRECATED_TxnLog& DEPRECATED_log, TxnLog& log, const TypeArena* typeArena, TypeLevel minLevel) - : DEPRECATED_log(DEPRECATED_log) - , log(log) + PromoteTypeLevels(TxnLog& log, const TypeArena* typeArena, TypeLevel minLevel) + : log(log) , typeArena(typeArena) , minLevel(minLevel) { @@ -50,15 +48,7 @@ struct PromoteTypeLevels LUAU_ASSERT(t); if (minLevel.subsumesStrict(t->level)) { - if (FFlag::LuauUseCommittingTxnLog) - { - log.changeLevel(ty, minLevel); - } - else - { - DEPRECATED_log(ty); - t->level = minLevel; - } + log.changeLevel(ty, minLevel); } } @@ -81,10 +71,10 @@ struct PromoteTypeLevels { // Surprise, it's actually a BoundTypeVar that hasn't been committed yet. // Calling getMutable on this will trigger an assertion. - if (FFlag::LuauUseCommittingTxnLog && !log.is(ty)) + if (!log.is(ty)) return true; - promote(ty, FFlag::LuauUseCommittingTxnLog ? log.getMutable(ty) : getMutable(ty)); + promote(ty, log.getMutable(ty)); return true; } @@ -94,7 +84,7 @@ struct PromoteTypeLevels if (FFlag::LuauImmutableTypes && ty->owningArena != typeArena) return false; - promote(ty, FFlag::LuauUseCommittingTxnLog ? log.getMutable(ty) : getMutable(ty)); + promote(ty, log.getMutable(ty)); return true; } @@ -107,7 +97,7 @@ struct PromoteTypeLevels if (ttv.state != TableState::Free && ttv.state != TableState::Generic) return true; - promote(ty, FFlag::LuauUseCommittingTxnLog ? log.getMutable(ty) : getMutable(ty)); + promote(ty, log.getMutable(ty)); return true; } @@ -115,33 +105,33 @@ struct PromoteTypeLevels { // Surprise, it's actually a BoundTypePack that hasn't been committed yet. // Calling getMutable on this will trigger an assertion. - if (FFlag::LuauUseCommittingTxnLog && !log.is(tp)) + if (!log.is(tp)) return true; - promote(tp, FFlag::LuauUseCommittingTxnLog ? log.getMutable(tp) : getMutable(tp)); + promote(tp, log.getMutable(tp)); return true; } }; -static void promoteTypeLevels(DEPRECATED_TxnLog& DEPRECATED_log, TxnLog& log, const TypeArena* typeArena, TypeLevel minLevel, TypeId ty) +static void promoteTypeLevels(TxnLog& log, const TypeArena* typeArena, TypeLevel minLevel, TypeId ty) { // Type levels of types from other modules are already global, so we don't need to promote anything inside if (FFlag::LuauImmutableTypes && ty->owningArena != typeArena) return; - PromoteTypeLevels ptl{DEPRECATED_log, log, typeArena, minLevel}; + PromoteTypeLevels ptl{log, typeArena, minLevel}; DenseHashSet seen{nullptr}; visitTypeVarOnce(ty, ptl, seen); } // TODO: use this and make it static. -void promoteTypeLevels(DEPRECATED_TxnLog& DEPRECATED_log, TxnLog& log, const TypeArena* typeArena, TypeLevel minLevel, TypePackId tp) +void promoteTypeLevels(TxnLog& log, const TypeArena* typeArena, TypeLevel minLevel, TypePackId tp) { // Type levels of types from other modules are already global, so we don't need to promote anything inside if (FFlag::LuauImmutableTypes && tp->owningArena != typeArena) return; - PromoteTypeLevels ptl{DEPRECATED_log, log, typeArena, minLevel}; + PromoteTypeLevels ptl{log, typeArena, minLevel}; DenseHashSet seen{nullptr}; visitTypeVarOnce(tp, ptl, seen); } @@ -251,7 +241,7 @@ struct SkipCacheForType bool Widen::isDirty(TypeId ty) { - return FFlag::LuauUseCommittingTxnLog ? log->is(ty) : bool(get(ty)); + return log->is(ty); } bool Widen::isDirty(TypePackId) @@ -262,7 +252,7 @@ bool Widen::isDirty(TypePackId) TypeId Widen::clean(TypeId ty) { LUAU_ASSERT(isDirty(ty)); - auto stv = FFlag::LuauUseCommittingTxnLog ? log->getMutable(ty) : getMutable(ty); + auto stv = log->getMutable(ty); LUAU_ASSERT(stv); if (get(stv)) @@ -284,11 +274,11 @@ bool Widen::ignoreChildren(TypeId ty) { // Sometimes we unify ("hi") -> free1 with (free2) -> free3, so don't ignore functions. // TODO: should we be doing this? we would need to rework how checkCallOverload does the unification. - if (FFlag::LuauUseCommittingTxnLog ? log->is(ty) : bool(get(ty))) + if (log->is(ty)) return false; // We only care about unions. - return !(FFlag::LuauUseCommittingTxnLog ? log->is(ty) : bool(get(ty))); + return !log->is(ty); } static std::optional hasUnificationTooComplex(const ErrorVec& errors) @@ -335,7 +325,6 @@ Unifier::Unifier(TypeArena* types, Mode mode, std::vector(superTy); - auto subFree = getMutable(subTy); - - if (FFlag::LuauUseCommittingTxnLog) - { - superFree = log.getMutable(superTy); - subFree = log.getMutable(subTy); - } + auto superFree = log.getMutable(superTy); + auto subFree = log.getMutable(subTy); if (superFree && subFree && superFree->level.subsumes(subFree->level)) { occursCheck(subTy, superTy); // The occurrence check might have caused superTy no longer to be a free type - bool occursFailed = false; - if (FFlag::LuauUseCommittingTxnLog) - occursFailed = bool(log.getMutable(subTy)); - else - occursFailed = bool(get(subTy)); + bool occursFailed = bool(log.getMutable(subTy)); if (!occursFailed) { - if (FFlag::LuauUseCommittingTxnLog) - { - log.replace(subTy, BoundTypeVar(superTy)); - } - else - { - DEPRECATED_log(subTy); - *asMutable(subTy) = BoundTypeVar(superTy); - } + log.replace(subTy, BoundTypeVar(superTy)); } return; } else if (superFree && subFree) { - if (!FFlag::LuauErrorRecoveryType && !FFlag::LuauUseCommittingTxnLog) - { - DEPRECATED_log(superTy); - subFree->level = min(subFree->level, superFree->level); - } - occursCheck(superTy, subTy); - bool occursFailed = false; - if (FFlag::LuauUseCommittingTxnLog) - occursFailed = bool(log.getMutable(superTy)); - else - occursFailed = bool(get(superTy)); - - if (!FFlag::LuauErrorRecoveryType && !FFlag::LuauUseCommittingTxnLog) - { - *asMutable(superTy) = BoundTypeVar(subTy); - return; - } + bool occursFailed = bool(log.getMutable(superTy)); if (!occursFailed) { - if (FFlag::LuauUseCommittingTxnLog) + if (superFree->level.subsumes(subFree->level)) { - if (superFree->level.subsumes(subFree->level)) - { - log.changeLevel(subTy, superFree->level); - } + log.changeLevel(subTy, superFree->level); + } - log.replace(superTy, BoundTypeVar(subTy)); - } - else - { - DEPRECATED_log(superTy); - *asMutable(superTy) = BoundTypeVar(subTy); - subFree->level = min(subFree->level, superFree->level); - } + log.replace(superTy, BoundTypeVar(subTy)); } return; @@ -460,14 +398,10 @@ void Unifier::tryUnify_(TypeId subTy, TypeId superTy, bool isFunctionCall, bool TypeLevel superLevel = superFree->level; occursCheck(superTy, subTy); - bool occursFailed = false; - if (FFlag::LuauUseCommittingTxnLog) - occursFailed = bool(log.getMutable(superTy)); - else - occursFailed = bool(get(superTy)); + bool occursFailed = bool(log.getMutable(superTy)); // Unification can't change the level of a generic. - auto subGeneric = FFlag::LuauUseCommittingTxnLog ? log.getMutable(subTy) : get(subTy); + auto subGeneric = log.getMutable(subTy); if (subGeneric && !subGeneric->level.subsumes(superLevel)) { // TODO: a more informative error message? CLI-39912 @@ -478,18 +412,8 @@ void Unifier::tryUnify_(TypeId subTy, TypeId superTy, bool isFunctionCall, bool // The occurrence check might have caused superTy no longer to be a free type if (!occursFailed) { - if (FFlag::LuauUseCommittingTxnLog) - { - promoteTypeLevels(DEPRECATED_log, log, types, superLevel, subTy); - log.replace(superTy, BoundTypeVar(widen(subTy))); - } - else - { - promoteTypeLevels(DEPRECATED_log, log, types, superLevel, subTy); - - DEPRECATED_log(superTy); - *asMutable(superTy) = BoundTypeVar(widen(subTy)); - } + promoteTypeLevels(log, types, superLevel, subTy); + log.replace(superTy, BoundTypeVar(widen(subTy))); } return; @@ -499,14 +423,10 @@ void Unifier::tryUnify_(TypeId subTy, TypeId superTy, bool isFunctionCall, bool TypeLevel subLevel = subFree->level; occursCheck(subTy, superTy); - bool occursFailed = false; - if (FFlag::LuauUseCommittingTxnLog) - occursFailed = bool(log.getMutable(subTy)); - else - occursFailed = bool(get(subTy)); + bool occursFailed = bool(log.getMutable(subTy)); // Unification can't change the level of a generic. - auto superGeneric = FFlag::LuauUseCommittingTxnLog ? log.getMutable(superTy) : get(superTy); + auto superGeneric = log.getMutable(superTy); if (superGeneric && !superGeneric->level.subsumes(subFree->level)) { // TODO: a more informative error message? CLI-39912 @@ -516,18 +436,8 @@ void Unifier::tryUnify_(TypeId subTy, TypeId superTy, bool isFunctionCall, bool if (!occursFailed) { - if (FFlag::LuauUseCommittingTxnLog) - { - promoteTypeLevels(DEPRECATED_log, log, types, subLevel, superTy); - log.replace(subTy, BoundTypeVar(superTy)); - } - else - { - promoteTypeLevels(DEPRECATED_log, log, types, subLevel, superTy); - - DEPRECATED_log(subTy); - *asMutable(subTy) = BoundTypeVar(superTy); - } + promoteTypeLevels(log, types, subLevel, superTy); + log.replace(subTy, BoundTypeVar(superTy)); } return; @@ -550,55 +460,38 @@ void Unifier::tryUnify_(TypeId subTy, TypeId superTy, bool isFunctionCall, bool // Here, we assume that the types unify. If they do not, we will find out as we roll back // the stack. - if (FFlag::LuauUseCommittingTxnLog) - { - if (log.haveSeen(superTy, subTy)) - return; + if (log.haveSeen(superTy, subTy)) + return; - log.pushSeen(superTy, subTy); - } - else - { - if (DEPRECATED_log.haveSeen(superTy, subTy)) - return; + log.pushSeen(superTy, subTy); - DEPRECATED_log.pushSeen(superTy, subTy); - } - - if (const UnionTypeVar* uv = FFlag::LuauUseCommittingTxnLog ? log.getMutable(subTy) : get(subTy)) + if (const UnionTypeVar* uv = log.getMutable(subTy)) { tryUnifyUnionWithType(subTy, uv, superTy); } - else if (const UnionTypeVar* uv = FFlag::LuauUseCommittingTxnLog ? log.getMutable(superTy) : get(superTy)) + else if (const UnionTypeVar* uv = log.getMutable(superTy)) { tryUnifyTypeWithUnion(subTy, superTy, uv, cacheEnabled, isFunctionCall); } - else if (const IntersectionTypeVar* uv = - FFlag::LuauUseCommittingTxnLog ? log.getMutable(superTy) : get(superTy)) + else if (const IntersectionTypeVar* uv = log.getMutable(superTy)) { tryUnifyTypeWithIntersection(subTy, superTy, uv); } - else if (const IntersectionTypeVar* uv = - FFlag::LuauUseCommittingTxnLog ? log.getMutable(subTy) : get(subTy)) + else if (const IntersectionTypeVar* uv = log.getMutable(subTy)) { tryUnifyIntersectionWithType(subTy, uv, superTy, cacheEnabled, isFunctionCall); } - else if ((FFlag::LuauUseCommittingTxnLog && log.getMutable(superTy) && log.getMutable(subTy)) || - (!FFlag::LuauUseCommittingTxnLog && get(superTy) && get(subTy))) + else if (log.getMutable(superTy) && log.getMutable(subTy)) tryUnifyPrimitives(subTy, superTy); - else if (FFlag::LuauSingletonTypes && - ((FFlag::LuauUseCommittingTxnLog ? log.getMutable(superTy) : get(superTy)) || - (FFlag::LuauUseCommittingTxnLog ? log.getMutable(superTy) : get(superTy))) && - (FFlag::LuauUseCommittingTxnLog ? log.getMutable(subTy) : get(subTy))) + else if (FFlag::LuauSingletonTypes && (log.getMutable(superTy) || log.getMutable(superTy)) && + log.getMutable(subTy)) tryUnifySingletons(subTy, superTy); - else if ((FFlag::LuauUseCommittingTxnLog && log.getMutable(superTy) && log.getMutable(subTy)) || - (!FFlag::LuauUseCommittingTxnLog && get(superTy) && get(subTy))) + else if (log.getMutable(superTy) && log.getMutable(subTy)) tryUnifyFunctions(subTy, superTy, isFunctionCall); - else if ((FFlag::LuauUseCommittingTxnLog && log.getMutable(superTy) && log.getMutable(subTy)) || - (!FFlag::LuauUseCommittingTxnLog && get(superTy) && get(subTy))) + else if (log.getMutable(superTy) && log.getMutable(subTy)) { tryUnifyTables(subTy, superTy, isIntersection); @@ -607,29 +500,23 @@ void Unifier::tryUnify_(TypeId subTy, TypeId superTy, bool isFunctionCall, bool } // tryUnifyWithMetatable assumes its first argument is a MetatableTypeVar. The check is otherwise symmetrical. - else if ((FFlag::LuauUseCommittingTxnLog && log.getMutable(superTy)) || - (!FFlag::LuauUseCommittingTxnLog && get(superTy))) + else if (log.getMutable(superTy)) tryUnifyWithMetatable(subTy, superTy, /*reversed*/ false); - else if ((FFlag::LuauUseCommittingTxnLog && log.getMutable(subTy)) || - (!FFlag::LuauUseCommittingTxnLog && get(subTy))) + else if (log.getMutable(subTy)) tryUnifyWithMetatable(superTy, subTy, /*reversed*/ true); - else if ((FFlag::LuauUseCommittingTxnLog && log.getMutable(superTy)) || - (!FFlag::LuauUseCommittingTxnLog && get(superTy))) + else if (log.getMutable(superTy)) tryUnifyWithClass(subTy, superTy, /*reversed*/ false); // Unification of nonclasses with classes is almost, but not quite symmetrical. // The order in which we perform this test is significant in the case that both types are classes. - else if ((FFlag::LuauUseCommittingTxnLog && log.getMutable(subTy)) || (!FFlag::LuauUseCommittingTxnLog && get(subTy))) + else if (log.getMutable(subTy)) tryUnifyWithClass(subTy, superTy, /*reversed*/ true); else reportError(TypeError{location, TypeMismatch{superTy, subTy}}); - if (FFlag::LuauUseCommittingTxnLog) - log.popSeen(superTy, subTy); - else - DEPRECATED_log.popSeen(superTy, subTy); + log.popSeen(superTy, subTy); } void Unifier::tryUnifyUnionWithType(TypeId subTy, const UnionTypeVar* uv, TypeId superTy) @@ -660,28 +547,12 @@ void Unifier::tryUnifyUnionWithType(TypeId subTy, const UnionTypeVar* uv, TypeId if (FFlag::LuauDifferentOrderOfUnificationDoesntMatter) { - if (!FFlag::LuauUseCommittingTxnLog) - innerState.DEPRECATED_log.rollback(); } else { - if (FFlag::LuauUseCommittingTxnLog) + if (i == count - 1) { - if (i == count - 1) - { - log.concat(std::move(innerState.log)); - } - } - else - { - if (i != count - 1) - { - innerState.DEPRECATED_log.rollback(); - } - else - { - DEPRECATED_log.concat(std::move(innerState.DEPRECATED_log)); - } + log.concat(std::move(innerState.log)); } ++i; @@ -692,7 +563,7 @@ void Unifier::tryUnifyUnionWithType(TypeId subTy, const UnionTypeVar* uv, TypeId if (FFlag::LuauDifferentOrderOfUnificationDoesntMatter) { auto tryBind = [this, subTy](TypeId superOption) { - superOption = FFlag::LuauUseCommittingTxnLog ? log.follow(superOption) : follow(superOption); + superOption = log.follow(superOption); // just skip if the superOption is not free-ish. auto ttv = log.getMutable(superOption); @@ -701,36 +572,17 @@ void Unifier::tryUnifyUnionWithType(TypeId subTy, const UnionTypeVar* uv, TypeId // Since we have already checked if S <: T, checking it again will not queue up the type for replacement. // So we'll have to do it ourselves. We assume they unified cleanly if they are still in the seen set. - if (FFlag::LuauUseCommittingTxnLog) + if (log.haveSeen(subTy, superOption)) { - if (log.haveSeen(subTy, superOption)) - { - // TODO: would it be nice for TxnLog::replace to do this? - if (log.is(superOption)) - log.bindTable(superOption, subTy); - else - log.replace(superOption, *subTy); - } - } - else - { - if (DEPRECATED_log.haveSeen(subTy, superOption)) - { - if (auto ttv = getMutable(superOption)) - { - DEPRECATED_log(ttv); - ttv->boundTo = subTy; - } - else - { - DEPRECATED_log(superOption); - *asMutable(superOption) = BoundTypeVar(subTy); - } - } + // TODO: would it be nice for TxnLog::replace to do this? + if (log.is(superOption)) + log.bindTable(superOption, subTy); + else + log.replace(superOption, *subTy); } }; - if (auto utv = (FFlag::LuauUseCommittingTxnLog ? log.getMutable(superTy) : get(superTy))) + if (auto utv = log.getMutable(superTy)) { for (TypeId ty : utv) tryBind(ty); @@ -815,10 +667,7 @@ void Unifier::tryUnifyTypeWithUnion(TypeId subTy, TypeId superTy, const UnionTyp if (innerState.errors.empty()) { found = true; - if (FFlag::LuauUseCommittingTxnLog) - log.concat(std::move(innerState.log)); - else - DEPRECATED_log.concat(std::move(innerState.DEPRECATED_log)); + log.concat(std::move(innerState.log)); break; } @@ -833,9 +682,6 @@ void Unifier::tryUnifyTypeWithUnion(TypeId subTy, TypeId superTy, const UnionTyp if (!failedOption) failedOption = {innerState.errors.front()}; } - - if (!FFlag::LuauUseCommittingTxnLog) - innerState.DEPRECATED_log.rollback(); } if (unificationTooComplex) @@ -870,10 +716,7 @@ void Unifier::tryUnifyTypeWithIntersection(TypeId subTy, TypeId superTy, const I firstFailedOption = {innerState.errors.front()}; } - if (FFlag::LuauUseCommittingTxnLog) - log.concat(std::move(innerState.log)); - else - DEPRECATED_log.concat(std::move(innerState.DEPRECATED_log)); + log.concat(std::move(innerState.log)); } if (unificationTooComplex) @@ -915,19 +758,13 @@ void Unifier::tryUnifyIntersectionWithType(TypeId subTy, const IntersectionTypeV if (innerState.errors.empty()) { found = true; - if (FFlag::LuauUseCommittingTxnLog) - log.concat(std::move(innerState.log)); - else - DEPRECATED_log.concat(std::move(innerState.DEPRECATED_log)); + log.concat(std::move(innerState.log)); break; } else if (auto e = hasUnificationTooComplex(innerState.errors)) { unificationTooComplex = e; } - - if (!FFlag::LuauUseCommittingTxnLog) - innerState.DEPRECATED_log.rollback(); } if (unificationTooComplex) @@ -971,78 +808,6 @@ void Unifier::cacheResult(TypeId subTy, TypeId superTy) sharedState.cachedUnify.insert({subTy, superTy}); } -struct DEPRECATED_WeirdIter -{ - TypePackId packId; - const TypePack* pack; - size_t index; - bool growing; - TypeLevel level; - - DEPRECATED_WeirdIter(TypePackId packId) - : packId(packId) - , pack(get(packId)) - , index(0) - , growing(false) - { - while (pack && pack->head.empty() && pack->tail) - { - packId = *pack->tail; - pack = get(packId); - } - } - - DEPRECATED_WeirdIter(const DEPRECATED_WeirdIter&) = default; - - const TypeId& operator*() - { - LUAU_ASSERT(good()); - return pack->head[index]; - } - - bool good() const - { - return pack != nullptr && index < pack->head.size(); - } - - bool advance() - { - if (!pack) - return good(); - - if (index < pack->head.size()) - ++index; - - if (growing || index < pack->head.size()) - return good(); - - if (pack->tail) - { - packId = follow(*pack->tail); - pack = get(packId); - index = 0; - } - - return good(); - } - - bool canGrow() const - { - return nullptr != get(packId); - } - - void grow(TypePackId newTail) - { - LUAU_ASSERT(canGrow()); - level = get(packId)->level; - *asMutable(packId) = Unifiable::Bound(newTail); - packId = newTail; - pack = get(newTail); - index = 0; - growing = true; - } -}; - struct WeirdIter { TypePackId packId; @@ -1141,9 +906,6 @@ ErrorVec Unifier::canUnify(TypeId subTy, TypeId superTy) Unifier s = makeChildUnifier(); s.tryUnify_(subTy, superTy); - if (!FFlag::LuauUseCommittingTxnLog) - s.DEPRECATED_log.rollback(); - return s.errors; } @@ -1152,9 +914,6 @@ ErrorVec Unifier::canUnify(TypePackId subTy, TypePackId superTy, bool isFunction Unifier s = makeChildUnifier(); s.tryUnify_(subTy, superTy, isFunctionCall); - if (!FFlag::LuauUseCommittingTxnLog) - s.DEPRECATED_log.rollback(); - return s.errors; } @@ -1200,419 +959,207 @@ void Unifier::tryUnify_(TypePackId subTp, TypePackId superTp, bool isFunctionCal return; } - if (FFlag::LuauUseCommittingTxnLog) + superTp = log.follow(superTp); + subTp = log.follow(subTp); + + while (auto tp = log.getMutable(subTp)) { - superTp = log.follow(superTp); - subTp = log.follow(subTp); + if (tp->head.empty() && tp->tail) + subTp = log.follow(*tp->tail); + else + break; + } - while (auto tp = log.getMutable(subTp)) + while (auto tp = log.getMutable(superTp)) + { + if (tp->head.empty() && tp->tail) + superTp = log.follow(*tp->tail); + else + break; + } + + if (superTp == subTp) + return; + + if (FFlag::LuauTxnLogSeesTypePacks2 && log.haveSeen(superTp, subTp)) + return; + + if (log.getMutable(superTp)) + { + occursCheck(superTp, subTp); + + if (!log.getMutable(superTp)) { - if (tp->head.empty() && tp->tail) - subTp = log.follow(*tp->tail); - else - break; + log.replace(superTp, Unifiable::Bound(subTp)); } + } + else if (log.getMutable(subTp)) + { + occursCheck(subTp, superTp); - while (auto tp = log.getMutable(superTp)) + if (!log.getMutable(subTp)) { - if (tp->head.empty() && tp->tail) - superTp = log.follow(*tp->tail); - else - break; + log.replace(subTp, Unifiable::Bound(superTp)); } + } + else if (log.getMutable(superTp)) + tryUnifyWithAny(subTp, superTp); + else if (log.getMutable(subTp)) + tryUnifyWithAny(superTp, subTp); + else if (log.getMutable(superTp)) + tryUnifyVariadics(subTp, superTp, false); + else if (log.getMutable(subTp)) + tryUnifyVariadics(superTp, subTp, true); + else if (log.getMutable(superTp) && log.getMutable(subTp)) + { + auto superTpv = log.getMutable(superTp); + auto subTpv = log.getMutable(subTp); - if (superTp == subTp) - return; + // If the size of two heads does not match, but both packs have free tail + // We set the sentinel variable to say so to avoid growing it forever. + auto [superTypes, superTail] = logAwareFlatten(superTp, log); + auto [subTypes, subTail] = logAwareFlatten(subTp, log); - if (FFlag::LuauTxnLogSeesTypePacks2 && log.haveSeen(superTp, subTp)) - return; + bool noInfiniteGrowth = (superTypes.size() != subTypes.size()) && (superTail && log.getMutable(*superTail)) && + (subTail && log.getMutable(*subTail)); - if (log.getMutable(superTp)) + auto superIter = WeirdIter(superTp, log); + auto subIter = WeirdIter(subTp, log); + + auto mkFreshType = [this](TypeLevel level) { + return types->freshType(level); + }; + + const TypePackId emptyTp = types->addTypePack(TypePack{{}, std::nullopt}); + + int loopCount = 0; + + do { - occursCheck(superTp, subTp); + if (FInt::LuauTypeInferTypePackLoopLimit > 0 && loopCount >= FInt::LuauTypeInferTypePackLoopLimit) + ice("Detected possibly infinite TypePack growth"); - if (!log.getMutable(superTp)) + ++loopCount; + + if (superIter.good() && subIter.growing) { - log.replace(superTp, Unifiable::Bound(subTp)); + subIter.pushType(mkFreshType(subIter.level)); } - } - else if (log.getMutable(subTp)) - { - occursCheck(subTp, superTp); - if (!log.getMutable(subTp)) + if (subIter.good() && superIter.growing) { - log.replace(subTp, Unifiable::Bound(superTp)); + superIter.pushType(mkFreshType(superIter.level)); } - } - else if (log.getMutable(superTp)) - tryUnifyWithAny(subTp, superTp); - else if (log.getMutable(subTp)) - tryUnifyWithAny(superTp, subTp); - else if (log.getMutable(superTp)) - tryUnifyVariadics(subTp, superTp, false); - else if (log.getMutable(subTp)) - tryUnifyVariadics(superTp, subTp, true); - else if (log.getMutable(superTp) && log.getMutable(subTp)) - { - auto superTpv = log.getMutable(superTp); - auto subTpv = log.getMutable(subTp); - // If the size of two heads does not match, but both packs have free tail - // We set the sentinel variable to say so to avoid growing it forever. - auto [superTypes, superTail] = logAwareFlatten(superTp, log); - auto [subTypes, subTail] = logAwareFlatten(subTp, log); - - bool noInfiniteGrowth = (superTypes.size() != subTypes.size()) && (superTail && log.getMutable(*superTail)) && - (subTail && log.getMutable(*subTail)); - - auto superIter = WeirdIter(superTp, log); - auto subIter = WeirdIter(subTp, log); - - auto mkFreshType = [this](TypeLevel level) { - return types->freshType(level); - }; - - const TypePackId emptyTp = types->addTypePack(TypePack{{}, std::nullopt}); - - int loopCount = 0; - - do + if (superIter.good() && subIter.good()) { - if (FInt::LuauTypeInferTypePackLoopLimit > 0 && loopCount >= FInt::LuauTypeInferTypePackLoopLimit) - ice("Detected possibly infinite TypePack growth"); + tryUnify_(*subIter, *superIter); - ++loopCount; + if (!errors.empty() && !firstPackErrorPos) + firstPackErrorPos = loopCount; - if (superIter.good() && subIter.growing) + superIter.advance(); + subIter.advance(); + continue; + } + + // If both are at the end, we're done + if (!superIter.good() && !subIter.good()) + { + if (subTpv->tail && superTpv->tail) { - subIter.pushType(mkFreshType(subIter.level)); + tryUnify_(*subTpv->tail, *superTpv->tail); + break; } - if (subIter.good() && superIter.growing) + const bool lFreeTail = superTpv->tail && log.getMutable(log.follow(*superTpv->tail)) != nullptr; + const bool rFreeTail = subTpv->tail && log.getMutable(log.follow(*subTpv->tail)) != nullptr; + if (lFreeTail) + tryUnify_(emptyTp, *superTpv->tail); + else if (rFreeTail) + tryUnify_(emptyTp, *subTpv->tail); + + break; + } + + // If both tails are free, bind one to the other and call it a day + if (superIter.canGrow() && subIter.canGrow()) + return tryUnify_(*subIter.pack->tail, *superIter.pack->tail); + + // If just one side is free on its tail, grow it to fit the other side. + // FIXME: The tail-most tail of the growing pack should be the same as the tail-most tail of the non-growing pack. + if (superIter.canGrow()) + superIter.grow(types->addTypePack(TypePackVar(TypePack{}))); + else if (subIter.canGrow()) + subIter.grow(types->addTypePack(TypePackVar(TypePack{}))); + else + { + // A union type including nil marks an optional argument + if (superIter.good() && isOptional(*superIter)) { - superIter.pushType(mkFreshType(superIter.level)); - } - - if (superIter.good() && subIter.good()) - { - tryUnify_(*subIter, *superIter); - - if (!errors.empty() && !firstPackErrorPos) - firstPackErrorPos = loopCount; - superIter.advance(); + continue; + } + else if (subIter.good() && isOptional(*subIter)) + { subIter.advance(); continue; } - // If both are at the end, we're done - if (!superIter.good() && !subIter.good()) + // In nonstrict mode, any also marks an optional argument. + else if (superIter.good() && isNonstrictMode() && log.getMutable(log.follow(*superIter))) { - if (subTpv->tail && superTpv->tail) - { - tryUnify_(*subTpv->tail, *superTpv->tail); - break; - } - - const bool lFreeTail = superTpv->tail && log.getMutable(log.follow(*superTpv->tail)) != nullptr; - const bool rFreeTail = subTpv->tail && log.getMutable(log.follow(*subTpv->tail)) != nullptr; - if (lFreeTail) - tryUnify_(emptyTp, *superTpv->tail); - else if (rFreeTail) - tryUnify_(emptyTp, *subTpv->tail); - - break; + superIter.advance(); + continue; } - // If both tails are free, bind one to the other and call it a day - if (superIter.canGrow() && subIter.canGrow()) - return tryUnify_(*subIter.pack->tail, *superIter.pack->tail); - - // If just one side is free on its tail, grow it to fit the other side. - // FIXME: The tail-most tail of the growing pack should be the same as the tail-most tail of the non-growing pack. - if (superIter.canGrow()) - superIter.grow(types->addTypePack(TypePackVar(TypePack{}))); - else if (subIter.canGrow()) - subIter.grow(types->addTypePack(TypePackVar(TypePack{}))); - else + if (log.getMutable(superIter.packId)) { - // A union type including nil marks an optional argument - if (superIter.good() && isOptional(*superIter)) - { - superIter.advance(); - continue; - } - else if (subIter.good() && isOptional(*subIter)) - { - subIter.advance(); - continue; - } - - // In nonstrict mode, any also marks an optional argument. - else if (superIter.good() && isNonstrictMode() && log.getMutable(log.follow(*superIter))) - { - superIter.advance(); - continue; - } - - if (log.getMutable(superIter.packId)) - { - tryUnifyVariadics(subIter.packId, superIter.packId, false, int(subIter.index)); - return; - } - - if (log.getMutable(subIter.packId)) - { - tryUnifyVariadics(superIter.packId, subIter.packId, true, int(superIter.index)); - return; - } - - if (!isFunctionCall && subIter.good()) - { - // Sometimes it is ok to pass too many arguments - return; - } - - // This is a bit weird because we don't actually know expected vs actual. We just know - // subtype vs supertype. If we are checking the values returned by a function, we swap - // these to produce the expected error message. - size_t expectedSize = size(superTp); - size_t actualSize = size(subTp); - if (ctx == CountMismatch::Result) - std::swap(expectedSize, actualSize); - reportError(TypeError{location, CountMismatch{expectedSize, actualSize, ctx}}); - - while (superIter.good()) - { - tryUnify_(*superIter, getSingletonTypes().errorRecoveryType()); - superIter.advance(); - } - - while (subIter.good()) - { - tryUnify_(*subIter, getSingletonTypes().errorRecoveryType()); - subIter.advance(); - } - + tryUnifyVariadics(subIter.packId, superIter.packId, false, int(subIter.index)); return; } - } while (!noInfiniteGrowth); - } - else - { - reportError(TypeError{location, GenericError{"Failed to unify type packs"}}); - } + if (log.getMutable(subIter.packId)) + { + tryUnifyVariadics(superIter.packId, subIter.packId, true, int(superIter.index)); + return; + } + + if (!isFunctionCall && subIter.good()) + { + // Sometimes it is ok to pass too many arguments + return; + } + + // This is a bit weird because we don't actually know expected vs actual. We just know + // subtype vs supertype. If we are checking the values returned by a function, we swap + // these to produce the expected error message. + size_t expectedSize = size(superTp); + size_t actualSize = size(subTp); + if (ctx == CountMismatch::Result) + std::swap(expectedSize, actualSize); + reportError(TypeError{location, CountMismatch{expectedSize, actualSize, ctx}}); + + while (superIter.good()) + { + tryUnify_(*superIter, getSingletonTypes().errorRecoveryType()); + superIter.advance(); + } + + while (subIter.good()) + { + tryUnify_(*subIter, getSingletonTypes().errorRecoveryType()); + subIter.advance(); + } + + return; + } + + } while (!noInfiniteGrowth); } else { - superTp = follow(superTp); - subTp = follow(subTp); - - while (auto tp = get(subTp)) - { - if (tp->head.empty() && tp->tail) - subTp = follow(*tp->tail); - else - break; - } - - while (auto tp = get(superTp)) - { - if (tp->head.empty() && tp->tail) - superTp = follow(*tp->tail); - else - break; - } - - if (superTp == subTp) - return; - - if (FFlag::LuauTxnLogSeesTypePacks2 && DEPRECATED_log.haveSeen(superTp, subTp)) - return; - - if (get(superTp)) - { - occursCheck(superTp, subTp); - - if (!get(superTp)) - { - DEPRECATED_log(superTp); - *asMutable(superTp) = Unifiable::Bound(subTp); - } - } - else if (get(subTp)) - { - occursCheck(subTp, superTp); - - if (!get(subTp)) - { - DEPRECATED_log(subTp); - *asMutable(subTp) = Unifiable::Bound(superTp); - } - } - - else if (get(superTp)) - tryUnifyWithAny(subTp, superTp); - - else if (get(subTp)) - tryUnifyWithAny(superTp, subTp); - - else if (get(superTp)) - tryUnifyVariadics(subTp, superTp, false); - else if (get(subTp)) - tryUnifyVariadics(superTp, subTp, true); - - else if (get(superTp) && get(subTp)) - { - auto superTpv = get(superTp); - auto subTpv = get(subTp); - - // If the size of two heads does not match, but both packs have free tail - // We set the sentinel variable to say so to avoid growing it forever. - auto [superTypes, superTail] = flatten(superTp); - auto [subTypes, subTail] = flatten(subTp); - - bool noInfiniteGrowth = - (superTypes.size() != subTypes.size()) && (superTail && get(*superTail)) && (subTail && get(*subTail)); - - auto superIter = DEPRECATED_WeirdIter{superTp}; - auto subIter = DEPRECATED_WeirdIter{subTp}; - - auto mkFreshType = [this](TypeLevel level) { - return types->freshType(level); - }; - - const TypePackId emptyTp = types->addTypePack(TypePack{{}, std::nullopt}); - - int loopCount = 0; - - do - { - if (FInt::LuauTypeInferTypePackLoopLimit > 0 && loopCount >= FInt::LuauTypeInferTypePackLoopLimit) - ice("Detected possibly infinite TypePack growth"); - - ++loopCount; - - if (superIter.good() && subIter.growing) - asMutable(subIter.pack)->head.push_back(mkFreshType(subIter.level)); - - if (subIter.good() && superIter.growing) - asMutable(superIter.pack)->head.push_back(mkFreshType(superIter.level)); - - if (superIter.good() && subIter.good()) - { - tryUnify_(*subIter, *superIter); - - if (!errors.empty() && !firstPackErrorPos) - firstPackErrorPos = loopCount; - - superIter.advance(); - subIter.advance(); - continue; - } - - // If both are at the end, we're done - if (!superIter.good() && !subIter.good()) - { - if (subTpv->tail && superTpv->tail) - { - tryUnify_(*subTpv->tail, *superTpv->tail); - break; - } - - const bool lFreeTail = superTpv->tail && get(follow(*superTpv->tail)) != nullptr; - const bool rFreeTail = subTpv->tail && get(follow(*subTpv->tail)) != nullptr; - if (lFreeTail) - tryUnify_(emptyTp, *superTpv->tail); - else if (rFreeTail) - tryUnify_(emptyTp, *subTpv->tail); - - break; - } - - // If both tails are free, bind one to the other and call it a day - if (superIter.canGrow() && subIter.canGrow()) - return tryUnify_(*subIter.pack->tail, *superIter.pack->tail); - - // If just one side is free on its tail, grow it to fit the other side. - // FIXME: The tail-most tail of the growing pack should be the same as the tail-most tail of the non-growing pack. - if (superIter.canGrow()) - superIter.grow(types->addTypePack(TypePackVar(TypePack{}))); - - else if (subIter.canGrow()) - subIter.grow(types->addTypePack(TypePackVar(TypePack{}))); - - else - { - // A union type including nil marks an optional argument - if (superIter.good() && isOptional(*superIter)) - { - superIter.advance(); - continue; - } - else if (subIter.good() && isOptional(*subIter)) - { - subIter.advance(); - continue; - } - - // In nonstrict mode, any also marks an optional argument. - else if (superIter.good() && isNonstrictMode() && get(follow(*superIter))) - { - superIter.advance(); - continue; - } - - if (get(superIter.packId)) - { - tryUnifyVariadics(subIter.packId, superIter.packId, false, int(subIter.index)); - return; - } - - if (get(subIter.packId)) - { - tryUnifyVariadics(superIter.packId, subIter.packId, true, int(superIter.index)); - return; - } - - if (!isFunctionCall && subIter.good()) - { - // Sometimes it is ok to pass too many arguments - return; - } - - // This is a bit weird because we don't actually know expected vs actual. We just know - // subtype vs supertype. If we are checking the values returned by a function, we swap - // these to produce the expected error message. - size_t expectedSize = size(superTp); - size_t actualSize = size(subTp); - if (ctx == CountMismatch::Result) - std::swap(expectedSize, actualSize); - reportError(TypeError{location, CountMismatch{expectedSize, actualSize, ctx}}); - - while (superIter.good()) - { - tryUnify_(*superIter, getSingletonTypes().errorRecoveryType()); - superIter.advance(); - } - - while (subIter.good()) - { - tryUnify_(*subIter, getSingletonTypes().errorRecoveryType()); - subIter.advance(); - } - - return; - } - - } while (!noInfiniteGrowth); - } - else - { - reportError(TypeError{location, GenericError{"Failed to unify type packs"}}); - } + reportError(TypeError{location, GenericError{"Failed to unify type packs"}}); } } @@ -1650,14 +1197,8 @@ void Unifier::tryUnifySingletons(TypeId subTy, TypeId superTy) void Unifier::tryUnifyFunctions(TypeId subTy, TypeId superTy, bool isFunctionCall) { - FunctionTypeVar* superFunction = getMutable(superTy); - FunctionTypeVar* subFunction = getMutable(subTy); - - if (FFlag::LuauUseCommittingTxnLog) - { - superFunction = log.getMutable(superTy); - subFunction = log.getMutable(subTy); - } + FunctionTypeVar* superFunction = log.getMutable(superTy); + FunctionTypeVar* subFunction = log.getMutable(subTy); if (!superFunction || !subFunction) ice("passed non-function types to unifyFunction"); @@ -1680,20 +1221,14 @@ void Unifier::tryUnifyFunctions(TypeId subTy, TypeId superTy, bool isFunctionCal for (size_t i = 0; i < numGenerics; i++) { - if (FFlag::LuauUseCommittingTxnLog) - log.pushSeen(superFunction->generics[i], subFunction->generics[i]); - else - DEPRECATED_log.pushSeen(superFunction->generics[i], subFunction->generics[i]); + log.pushSeen(superFunction->generics[i], subFunction->generics[i]); } if (FFlag::LuauTxnLogSeesTypePacks2) { for (size_t i = 0; i < numGenericPacks; i++) { - if (FFlag::LuauUseCommittingTxnLog) - log.pushSeen(superFunction->genericPacks[i], subFunction->genericPacks[i]); - else - DEPRECATED_log.pushSeen(superFunction->genericPacks[i], subFunction->genericPacks[i]); + log.pushSeen(superFunction->genericPacks[i], subFunction->genericPacks[i]); } } @@ -1734,14 +1269,7 @@ void Unifier::tryUnifyFunctions(TypeId subTy, TypeId superTy, bool isFunctionCal reportError(TypeError{location, TypeMismatch{superTy, subTy, "", innerState.errors.front()}}); } - if (FFlag::LuauUseCommittingTxnLog) - { - log.concat(std::move(innerState.log)); - } - else - { - DEPRECATED_log.concat(std::move(innerState.DEPRECATED_log)); - } + log.concat(std::move(innerState.log)); } else { @@ -1754,33 +1282,19 @@ void Unifier::tryUnifyFunctions(TypeId subTy, TypeId superTy, bool isFunctionCal if (!FFlag::LuauImmutableTypes) { - if (FFlag::LuauUseCommittingTxnLog) + if (superFunction->definition && !subFunction->definition && !subTy->persistent) { - if (superFunction->definition && !subFunction->definition && !subTy->persistent) - { - PendingType* newSubTy = log.queue(subTy); - FunctionTypeVar* newSubFtv = getMutable(newSubTy); - LUAU_ASSERT(newSubFtv); - newSubFtv->definition = superFunction->definition; - } - else if (!superFunction->definition && subFunction->definition && !superTy->persistent) - { - PendingType* newSuperTy = log.queue(superTy); - FunctionTypeVar* newSuperFtv = getMutable(newSuperTy); - LUAU_ASSERT(newSuperFtv); - newSuperFtv->definition = subFunction->definition; - } + PendingType* newSubTy = log.queue(subTy); + FunctionTypeVar* newSubFtv = getMutable(newSubTy); + LUAU_ASSERT(newSubFtv); + newSubFtv->definition = superFunction->definition; } - else + else if (!superFunction->definition && subFunction->definition && !superTy->persistent) { - if (superFunction->definition && !subFunction->definition && !subTy->persistent) - { - subFunction->definition = superFunction->definition; - } - else if (!superFunction->definition && subFunction->definition && !superTy->persistent) - { - superFunction->definition = subFunction->definition; - } + PendingType* newSuperTy = log.queue(superTy); + FunctionTypeVar* newSuperFtv = getMutable(newSuperTy); + LUAU_ASSERT(newSuperFtv); + newSuperFtv->definition = subFunction->definition; } } @@ -1790,19 +1304,13 @@ void Unifier::tryUnifyFunctions(TypeId subTy, TypeId superTy, bool isFunctionCal { for (int i = int(numGenericPacks) - 1; 0 <= i; i--) { - if (FFlag::LuauUseCommittingTxnLog) - log.popSeen(superFunction->genericPacks[i], subFunction->genericPacks[i]); - else - DEPRECATED_log.popSeen(superFunction->genericPacks[i], subFunction->genericPacks[i]); + log.popSeen(superFunction->genericPacks[i], subFunction->genericPacks[i]); } } for (int i = int(numGenerics) - 1; 0 <= i; i--) { - if (FFlag::LuauUseCommittingTxnLog) - log.popSeen(superFunction->generics[i], subFunction->generics[i]); - else - DEPRECATED_log.popSeen(superFunction->generics[i], subFunction->generics[i]); + log.popSeen(superFunction->generics[i], subFunction->generics[i]); } } @@ -1833,14 +1341,8 @@ void Unifier::tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) if (!FFlag::LuauTableSubtypingVariance2) return DEPRECATED_tryUnifyTables(subTy, superTy, isIntersection); - TableTypeVar* superTable = getMutable(superTy); - TableTypeVar* subTable = getMutable(subTy); - - if (FFlag::LuauUseCommittingTxnLog) - { - superTable = log.getMutable(superTy); - subTable = log.getMutable(subTy); - } + TableTypeVar* superTable = log.getMutable(superTy); + TableTypeVar* subTable = log.getMutable(subTy); if (!superTable || !subTable) ice("passed non-table types to unifyTables"); @@ -1855,8 +1357,7 @@ void Unifier::tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) { auto subIter = subTable->props.find(propName); - bool isAny = - FFlag::LuauUseCommittingTxnLog ? log.getMutable(log.follow(superProp.type)) : get(follow(superProp.type)); + bool isAny = log.getMutable(log.follow(superProp.type)); if (subIter == subTable->props.end() && (!FFlag::LuauSubtypingAddOptPropsToUnsealedTables || subTable->state == TableState::Unsealed) && !isOptional(superProp.type) && !isAny) missingProperties.push_back(propName); @@ -1877,8 +1378,7 @@ void Unifier::tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) { auto superIter = superTable->props.find(propName); - bool isAny = - FFlag::LuauUseCommittingTxnLog ? log.getMutable(log.follow(subProp.type)) : get(follow(subProp.type)); + bool isAny = log.is(log.follow(subProp.type)); if (superIter == superTable->props.end() && (FFlag::LuauSubtypingAddOptPropsToUnsealedTables || (!isOptional(subProp.type) && !isAny))) extraProperties.push_back(propName); } @@ -1906,18 +1406,8 @@ void Unifier::tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) checkChildUnifierTypeMismatch(innerState.errors, name, superTy, subTy); - if (FFlag::LuauUseCommittingTxnLog) - { - if (innerState.errors.empty()) - log.concat(std::move(innerState.log)); - } - else - { - if (innerState.errors.empty()) - DEPRECATED_log.concat(std::move(innerState.DEPRECATED_log)); - else - innerState.DEPRECATED_log.rollback(); - } + if (innerState.errors.empty()) + log.concat(std::move(innerState.log)); } else if (subTable->indexer && maybeString(subTable->indexer->indexType)) { @@ -1931,18 +1421,8 @@ void Unifier::tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) checkChildUnifierTypeMismatch(innerState.errors, name, superTy, subTy); - if (FFlag::LuauUseCommittingTxnLog) - { - if (innerState.errors.empty()) - log.concat(std::move(innerState.log)); - } - else - { - if (innerState.errors.empty()) - DEPRECATED_log.concat(std::move(innerState.DEPRECATED_log)); - else - innerState.DEPRECATED_log.rollback(); - } + if (innerState.errors.empty()) + log.concat(std::move(innerState.log)); } else if ((!FFlag::LuauSubtypingAddOptPropsToUnsealedTables || subTable->state == TableState::Unsealed) && (isOptional(prop.type) || get(follow(prop.type)))) // This is sound because unsealed table types are precise, so `{ p : T } <: { p : T, q : U? }` @@ -1953,22 +1433,30 @@ void Unifier::tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) } else if (subTable->state == TableState::Free) { - if (FFlag::LuauUseCommittingTxnLog) - { - PendingType* pendingSub = log.queue(subTy); - TableTypeVar* ttv = getMutable(pendingSub); - LUAU_ASSERT(ttv); - ttv->props[name] = prop; - subTable = ttv; - } - else - { - DEPRECATED_log(subTy); - subTable->props[name] = prop; - } + PendingType* pendingSub = log.queue(subTy); + TableTypeVar* ttv = getMutable(pendingSub); + LUAU_ASSERT(ttv); + ttv->props[name] = prop; + subTable = ttv; } else missingProperties.push_back(name); + + if (FFlag::LuauTxnLogCheckForInvalidation) + { + // Recursive unification can change the txn log, and invalidate the old + // table. If we detect that this has happened, we start over, with the updated + // txn log. + TableTypeVar* newSuperTable = log.getMutable(superTy); + TableTypeVar* newSubTable = log.getMutable(subTy); + if (superTable != newSuperTable || subTable != newSubTable) + { + if (errors.empty()) + return tryUnifyTables(subTy, superTy, isIntersection); + else + return; + } + } } for (const auto& [name, prop] : subTable->props) @@ -1990,18 +1478,8 @@ void Unifier::tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) checkChildUnifierTypeMismatch(innerState.errors, name, superTy, subTy); - if (FFlag::LuauUseCommittingTxnLog) - { - if (innerState.errors.empty()) - log.concat(std::move(innerState.log)); - } - else - { - if (innerState.errors.empty()) - DEPRECATED_log.concat(std::move(innerState.DEPRECATED_log)); - else - innerState.DEPRECATED_log.rollback(); - } + if (innerState.errors.empty()) + log.concat(std::move(innerState.log)); } else if (superTable->state == TableState::Unsealed) { @@ -2011,18 +1489,10 @@ void Unifier::tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) Property clone = prop; clone.type = deeplyOptional(clone.type); - if (FFlag::LuauUseCommittingTxnLog) - { - PendingType* pendingSuper = log.queue(superTy); - TableTypeVar* pendingSuperTtv = getMutable(pendingSuper); - pendingSuperTtv->props[name] = clone; - superTable = pendingSuperTtv; - } - else - { - DEPRECATED_log(superTy); - superTable->props[name] = clone; - } + PendingType* pendingSuper = log.queue(superTy); + TableTypeVar* pendingSuperTtv = getMutable(pendingSuper); + pendingSuperTtv->props[name] = clone; + superTable = pendingSuperTtv; } else if (variance == Covariant) { @@ -2032,21 +1502,29 @@ void Unifier::tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) } else if (superTable->state == TableState::Free) { - if (FFlag::LuauUseCommittingTxnLog) - { - PendingType* pendingSuper = log.queue(superTy); - TableTypeVar* pendingSuperTtv = getMutable(pendingSuper); - pendingSuperTtv->props[name] = prop; - superTable = pendingSuperTtv; - } - else - { - DEPRECATED_log(superTy); - superTable->props[name] = prop; - } + PendingType* pendingSuper = log.queue(superTy); + TableTypeVar* pendingSuperTtv = getMutable(pendingSuper); + pendingSuperTtv->props[name] = prop; + superTable = pendingSuperTtv; } else extraProperties.push_back(name); + + if (FFlag::LuauTxnLogCheckForInvalidation) + { + // Recursive unification can change the txn log, and invalidate the old + // table. If we detect that this has happened, we start over, with the updated + // txn log. + TableTypeVar* newSuperTable = log.getMutable(superTy); + TableTypeVar* newSubTable = log.getMutable(subTy); + if (superTable != newSuperTable || subTable != newSubTable) + { + if (errors.empty()) + return tryUnifyTables(subTy, superTy, isIntersection); + else + return; + } + } } // Unify indexers @@ -2060,18 +1538,8 @@ void Unifier::tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) innerState.tryUnifyIndexer(*subTable->indexer, *superTable->indexer); checkChildUnifierTypeMismatch(innerState.errors, superTy, subTy); - if (FFlag::LuauUseCommittingTxnLog) - { - if (innerState.errors.empty()) - log.concat(std::move(innerState.log)); - } - else - { - if (innerState.errors.empty()) - DEPRECATED_log.concat(std::move(innerState.DEPRECATED_log)); - else - innerState.DEPRECATED_log.rollback(); - } + if (innerState.errors.empty()) + log.concat(std::move(innerState.log)); } else if (superTable->indexer) { @@ -2081,15 +1549,7 @@ void Unifier::tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) // e.g. table.insert(t, 1) where t is a non-sealed table and doesn't have an indexer. // TODO: we only need to do this if the supertype's indexer is read/write // since that can add indexed elements. - if (FFlag::LuauUseCommittingTxnLog) - { - log.changeIndexer(subTy, superTable->indexer); - } - else - { - DEPRECATED_log(subTy); - subTable->indexer = superTable->indexer; - } + log.changeIndexer(subTy, superTable->indexer); } } else if (subTable->indexer && variance == Invariant) @@ -2097,15 +1557,29 @@ void Unifier::tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) // Symmetric if we are invariant if (superTable->state == TableState::Unsealed || superTable->state == TableState::Free) { - if (FFlag::LuauUseCommittingTxnLog) - { - log.changeIndexer(superTy, subTable->indexer); - } + log.changeIndexer(superTy, subTable->indexer); + } + } + + if (FFlag::LuauTxnLogDontRetryForIndexers) + { + // Changing the indexer can invalidate the table pointers. + superTable = log.getMutable(superTy); + subTable = log.getMutable(subTy); + } + else if (FFlag::LuauTxnLogCheckForInvalidation) + { + // Recursive unification can change the txn log, and invalidate the old + // table. If we detect that this has happened, we start over, with the updated + // txn log. + TableTypeVar* newSuperTable = log.getMutable(superTy); + TableTypeVar* newSubTable = log.getMutable(subTy); + if (superTable != newSuperTable || subTable != newSubTable) + { + if (errors.empty()) + return tryUnifyTables(subTy, superTy, isIntersection); else - { - DEPRECATED_log(superTy); - superTable->indexer = subTable->indexer; - } + return; } } @@ -2134,27 +1608,11 @@ void Unifier::tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) if (superTable->state == TableState::Free) { - if (FFlag::LuauUseCommittingTxnLog) - { - log.bindTable(superTy, subTy); - } - else - { - DEPRECATED_log(superTable); - superTable->boundTo = subTy; - } + log.bindTable(superTy, subTy); } else if (subTable->state == TableState::Free) { - if (FFlag::LuauUseCommittingTxnLog) - { - log.bindTable(subTy, superTy); - } - else - { - DEPRECATED_log(subTable); - subTable->boundTo = superTy; - } + log.bindTable(subTy, superTy); } } @@ -2197,14 +1655,8 @@ void Unifier::DEPRECATED_tryUnifyTables(TypeId subTy, TypeId superTy, bool isInt Resetter resetter{&variance}; variance = Invariant; - TableTypeVar* superTable = getMutable(superTy); - TableTypeVar* subTable = getMutable(subTy); - - if (FFlag::LuauUseCommittingTxnLog) - { - superTable = log.getMutable(superTy); - subTable = log.getMutable(subTy); - } + TableTypeVar* superTable = log.getMutable(superTy); + TableTypeVar* subTable = log.getMutable(subTy); if (!superTable || !subTable) ice("passed non-table types to unifyTables"); @@ -2231,15 +1683,7 @@ void Unifier::DEPRECATED_tryUnifyTables(TypeId subTy, TypeId superTy, bool isInt // avoid creating a cycle when the types are already pointing at each other if (follow(superTy) != follow(subTy)) { - if (FFlag::LuauUseCommittingTxnLog) - { - log.bindTable(superTy, subTy); - } - else - { - DEPRECATED_log(superTable); - superTable->boundTo = subTy; - } + log.bindTable(superTy, subTy); } return; } @@ -2268,14 +1712,7 @@ void Unifier::DEPRECATED_tryUnifyTables(TypeId subTy, TypeId superTy, bool isInt // e.g. table.insert(t, 1) where t is a non-sealed table and doesn't have an indexer. if (subTable->state == TableState::Unsealed) { - if (FFlag::LuauUseCommittingTxnLog) - { - log.changeIndexer(subTy, superTable->indexer); - } - else - { - subTable->indexer = superTable->indexer; - } + log.changeIndexer(subTy, superTable->indexer); } else reportError(TypeError{location, CannotExtendTable{subTy, CannotExtendTable::Indexer}}); @@ -2295,14 +1732,8 @@ void Unifier::DEPRECATED_tryUnifyTables(TypeId subTy, TypeId superTy, bool isInt void Unifier::tryUnifyFreeTable(TypeId subTy, TypeId superTy) { - TableTypeVar* freeTable = getMutable(superTy); - TableTypeVar* subTable = getMutable(subTy); - - if (FFlag::LuauUseCommittingTxnLog) - { - freeTable = log.getMutable(superTy); - subTable = log.getMutable(subTy); - } + TableTypeVar* freeTable = log.getMutable(superTy); + TableTypeVar* subTable = log.getMutable(subTy); if (!freeTable || !subTable) ice("passed non-table types to tryUnifyFreeTable"); @@ -2323,22 +1754,11 @@ void Unifier::tryUnifyFreeTable(TypeId subTy, TypeId superTy) * I believe this is guaranteed to terminate eventually because this will * only happen when a free table is bound to another table. */ - if (FFlag::LuauUseCommittingTxnLog) - { - if (!log.getMutable(superTy) || !log.getMutable(subTy)) - return tryUnify_(subTy, superTy); + if (!log.getMutable(superTy) || !log.getMutable(subTy)) + return tryUnify_(subTy, superTy); - if (TableTypeVar* pendingFreeTtv = log.getMutable(superTy); pendingFreeTtv && pendingFreeTtv->boundTo) - return tryUnify_(subTy, superTy); - } - else - { - if (!get(superTy) || !get(subTy)) - return tryUnify_(subTy, superTy); - - if (freeTable->boundTo) - return tryUnify_(subTy, superTy); - } + if (TableTypeVar* pendingFreeTtv = log.getMutable(superTy); pendingFreeTtv && pendingFreeTtv->boundTo) + return tryUnify_(subTy, superTy); } else { @@ -2346,17 +1766,10 @@ void Unifier::tryUnifyFreeTable(TypeId subTy, TypeId superTy) // properties than we previously thought. Else, it is an error. if (subTable->state == TableState::Free) { - if (FFlag::LuauUseCommittingTxnLog) - { - PendingType* pendingSub = log.queue(subTy); - TableTypeVar* pendingSubTtv = getMutable(pendingSub); - LUAU_ASSERT(pendingSubTtv); - pendingSubTtv->props.insert({freeName, freeProp}); - } - else - { - subTable->props.insert({freeName, freeProp}); - } + PendingType* pendingSub = log.queue(subTy); + TableTypeVar* pendingSubTtv = getMutable(pendingSub); + LUAU_ASSERT(pendingSubTtv); + pendingSubTtv->props.insert({freeName, freeProp}); } else reportError(TypeError{location, UnknownProperty{subTy, freeName}}); @@ -2370,47 +1783,23 @@ void Unifier::tryUnifyFreeTable(TypeId subTy, TypeId superTy) checkChildUnifierTypeMismatch(innerState.errors, superTy, subTy); - if (FFlag::LuauUseCommittingTxnLog) - log.concat(std::move(innerState.log)); - else - DEPRECATED_log.concat(std::move(innerState.DEPRECATED_log)); + log.concat(std::move(innerState.log)); } else if (subTable->state == TableState::Free && freeTable->indexer) { - if (FFlag::LuauUseCommittingTxnLog) - { - log.changeIndexer(superTy, subTable->indexer); - } - else - { - freeTable->indexer = subTable->indexer; - } + log.changeIndexer(superTy, subTable->indexer); } if (!freeTable->boundTo && subTable->state != TableState::Free) { - if (FFlag::LuauUseCommittingTxnLog) - { - log.bindTable(superTy, subTy); - } - else - { - DEPRECATED_log(freeTable); - freeTable->boundTo = subTy; - } + log.bindTable(superTy, subTy); } } void Unifier::tryUnifySealedTables(TypeId subTy, TypeId superTy, bool isIntersection) { - TableTypeVar* superTable = getMutable(superTy); - TableTypeVar* subTable = getMutable(subTy); - - if (FFlag::LuauUseCommittingTxnLog) - { - superTable = log.getMutable(superTy); - subTable = log.getMutable(subTy); - } + TableTypeVar* superTable = log.getMutable(superTy); + TableTypeVar* subTable = log.getMutable(subTy); if (!superTable || !subTable) ice("passed non-table types to unifySealedTables"); @@ -2476,77 +1865,39 @@ void Unifier::tryUnifySealedTables(TypeId subTy, TypeId superTy, bool isIntersec if (superTable->indexer || subTable->indexer) { - if (FFlag::LuauUseCommittingTxnLog) + if (superTable->indexer && subTable->indexer) + innerState.tryUnifyIndexer(*subTable->indexer, *superTable->indexer); + else if (subTable->state == TableState::Unsealed) { - if (superTable->indexer && subTable->indexer) - innerState.tryUnifyIndexer(*subTable->indexer, *superTable->indexer); - else if (subTable->state == TableState::Unsealed) + if (superTable->indexer && !subTable->indexer) { - if (superTable->indexer && !subTable->indexer) - { - log.changeIndexer(subTy, superTable->indexer); - } + log.changeIndexer(subTy, superTable->indexer); } - else if (superTable->state == TableState::Unsealed) + } + else if (superTable->state == TableState::Unsealed) + { + if (subTable->indexer && !superTable->indexer) { - if (subTable->indexer && !superTable->indexer) - { - log.changeIndexer(superTy, subTable->indexer); - } + log.changeIndexer(superTy, subTable->indexer); } - else if (superTable->indexer) + } + else if (superTable->indexer) + { + innerState.tryUnify_(getSingletonTypes().stringType, superTable->indexer->indexType); + for (const auto& [name, type] : subTable->props) { - innerState.tryUnify_(getSingletonTypes().stringType, superTable->indexer->indexType); - for (const auto& [name, type] : subTable->props) - { - const auto& it = superTable->props.find(name); - if (it == superTable->props.end()) - innerState.tryUnify_(type.type, superTable->indexer->indexResultType); - } + const auto& it = superTable->props.find(name); + if (it == superTable->props.end()) + innerState.tryUnify_(type.type, superTable->indexer->indexResultType); } - else - innerState.reportError(TypeError{location, TypeMismatch{superTy, subTy}}); } else - { - if (superTable->indexer && subTable->indexer) - innerState.tryUnifyIndexer(*subTable->indexer, *superTable->indexer); - else if (subTable->state == TableState::Unsealed) - { - if (superTable->indexer && !subTable->indexer) - subTable->indexer = superTable->indexer; - } - else if (superTable->state == TableState::Unsealed) - { - if (subTable->indexer && !superTable->indexer) - superTable->indexer = subTable->indexer; - } - else if (superTable->indexer) - { - innerState.tryUnify_(getSingletonTypes().stringType, superTable->indexer->indexType); - // We already try to unify properties in both tables. - // Skip those and just look for the ones remaining and see if they fit into the indexer. - for (const auto& [name, type] : subTable->props) - { - const auto& it = superTable->props.find(name); - if (it == superTable->props.end()) - innerState.tryUnify_(type.type, superTable->indexer->indexResultType); - } - } - else - innerState.reportError(TypeError{location, TypeMismatch{superTy, subTy}}); - } + innerState.reportError(TypeError{location, TypeMismatch{superTy, subTy}}); } - if (FFlag::LuauUseCommittingTxnLog) - { - if (!errorReported) - log.concat(std::move(innerState.log)); - } + if (!errorReported) + log.concat(std::move(innerState.log)); else - DEPRECATED_log.concat(std::move(innerState.DEPRECATED_log)); - - if (errorReported) return; if (!missingPropertiesInSuper.empty()) @@ -2594,8 +1945,7 @@ void Unifier::tryUnifyWithMetatable(TypeId subTy, TypeId superTy, bool reversed) TypeError mismatchError = TypeError{location, TypeMismatch{reversed ? subTy : superTy, reversed ? superTy : subTy}}; - if (const MetatableTypeVar* subMetatable = - FFlag::LuauUseCommittingTxnLog ? log.getMutable(subTy) : get(subTy)) + if (const MetatableTypeVar* subMetatable = log.getMutable(subTy)) { Unifier innerState = makeChildUnifier(); innerState.tryUnify_(subMetatable->table, superMetatable->table); @@ -2606,27 +1956,16 @@ void Unifier::tryUnifyWithMetatable(TypeId subTy, TypeId superTy, bool reversed) else if (!innerState.errors.empty()) reportError(TypeError{location, TypeMismatch{reversed ? subTy : superTy, reversed ? superTy : subTy, "", innerState.errors.front()}}); - if (FFlag::LuauUseCommittingTxnLog) - log.concat(std::move(innerState.log)); - else - DEPRECATED_log.concat(std::move(innerState.DEPRECATED_log)); + log.concat(std::move(innerState.log)); } - else if (TableTypeVar* subTable = FFlag::LuauUseCommittingTxnLog ? log.getMutable(subTy) : getMutable(subTy)) + else if (TableTypeVar* subTable = log.getMutable(subTy)) { switch (subTable->state) { case TableState::Free: { tryUnify_(subTy, superMetatable->table); - - if (FFlag::LuauUseCommittingTxnLog) - { - log.bindTable(subTy, superTy); - } - else - { - subTable->boundTo = superTy; - } + log.bindTable(subTy, superTy); break; } @@ -2637,8 +1976,7 @@ void Unifier::tryUnifyWithMetatable(TypeId subTy, TypeId superTy, bool reversed) reportError(mismatchError); } } - else if (FFlag::LuauUseCommittingTxnLog ? (log.getMutable(subTy) || log.getMutable(subTy)) - : (get(subTy) || get(subTy))) + else if (log.getMutable(subTy) || log.getMutable(subTy)) { } else @@ -2711,28 +2049,13 @@ void Unifier::tryUnifyWithClass(TypeId subTy, TypeId superTy, bool reversed) checkChildUnifierTypeMismatch(innerState.errors, propName, reversed ? subTy : superTy, reversed ? superTy : subTy); - if (FFlag::LuauUseCommittingTxnLog) + if (innerState.errors.empty()) { - if (innerState.errors.empty()) - { - log.concat(std::move(innerState.log)); - } - else - { - ok = false; - } + log.concat(std::move(innerState.log)); } else { - if (innerState.errors.empty()) - { - DEPRECATED_log.concat(std::move(innerState.DEPRECATED_log)); - } - else - { - ok = false; - innerState.DEPRECATED_log.rollback(); - } + ok = false; } } } @@ -2747,15 +2070,7 @@ void Unifier::tryUnifyWithClass(TypeId subTy, TypeId superTy, bool reversed) if (!ok) return; - if (FFlag::LuauUseCommittingTxnLog) - { - log.bindTable(subTy, superTy); - } - else - { - DEPRECATED_log(subTable); - subTable->boundTo = superTy; - } + log.bindTable(subTy, superTy); } else return fail(); @@ -2771,54 +2086,30 @@ static void queueTypePack(std::vector& queue, DenseHashSet& { while (true) { - a = FFlag::LuauFollowWithCommittingTxnLogInAnyUnification ? state.log.follow(a) : follow(a); + a = state.log.follow(a); if (seenTypePacks.find(a)) break; seenTypePacks.insert(a); - if (FFlag::LuauUseCommittingTxnLog) + if (state.log.getMutable(a)) { - if (state.log.getMutable(a)) - { - state.log.replace(a, Unifiable::Bound{anyTypePack}); - } - else if (auto tp = state.log.getMutable(a)) - { - queue.insert(queue.end(), tp->head.begin(), tp->head.end()); - if (tp->tail) - a = *tp->tail; - else - break; - } + state.log.replace(a, Unifiable::Bound{anyTypePack}); } - else + else if (auto tp = state.log.getMutable(a)) { - if (get(a)) - { - state.DEPRECATED_log(a); - *asMutable(a) = Unifiable::Bound{anyTypePack}; - } - else if (auto tp = get(a)) - { - queue.insert(queue.end(), tp->head.begin(), tp->head.end()); - if (tp->tail) - a = *tp->tail; - else - break; - } + queue.insert(queue.end(), tp->head.begin(), tp->head.end()); + if (tp->tail) + a = *tp->tail; + else + break; } } } void Unifier::tryUnifyVariadics(TypePackId subTp, TypePackId superTp, bool reversed, int subOffset) { - const VariadicTypePack* superVariadic = get(superTp); - - if (FFlag::LuauUseCommittingTxnLog) - { - superVariadic = log.getMutable(superTp); - } + const VariadicTypePack* superVariadic = log.getMutable(superTp); if (!superVariadic) ice("passed non-variadic pack to tryUnifyVariadics"); @@ -2843,15 +2134,7 @@ void Unifier::tryUnifyVariadics(TypePackId subTp, TypePackId superTp, bool rever TypePackId tail = follow(*maybeTail); if (get(tail)) { - if (FFlag::LuauUseCommittingTxnLog) - { - log.replace(tail, BoundTypePack(superTp)); - } - else - { - DEPRECATED_log(tail); - *asMutable(tail) = BoundTypePack{superTp}; - } + log.replace(tail, BoundTypePack(superTp)); } else if (const VariadicTypePack* vtp = get(tail)) { @@ -2882,103 +2165,54 @@ static void tryUnifyWithAny(std::vector& queue, Unifier& state, DenseHas { while (!queue.empty()) { - if (FFlag::LuauUseCommittingTxnLog) + TypeId ty = state.log.follow(queue.back()); + queue.pop_back(); + + // Types from other modules don't have free types + if (FFlag::LuauImmutableTypes && ty->owningArena != typeArena) + continue; + + if (seen.find(ty)) + continue; + + seen.insert(ty); + + if (state.log.getMutable(ty)) { - TypeId ty = state.log.follow(queue.back()); - queue.pop_back(); - - // Types from other modules don't have free types - if (FFlag::LuauImmutableTypes && ty->owningArena != typeArena) - continue; - - if (seen.find(ty)) - continue; - - seen.insert(ty); - - if (state.log.getMutable(ty)) - { - state.log.replace(ty, BoundTypeVar{anyType}); - } - else if (auto fun = state.log.getMutable(ty)) - { - queueTypePack(queue, seenTypePacks, state, fun->argTypes, anyTypePack); - queueTypePack(queue, seenTypePacks, state, fun->retType, anyTypePack); - } - else if (auto table = state.log.getMutable(ty)) - { - for (const auto& [_name, prop] : table->props) - queue.push_back(prop.type); - - if (table->indexer) - { - queue.push_back(table->indexer->indexType); - queue.push_back(table->indexer->indexResultType); - } - } - else if (auto mt = state.log.getMutable(ty)) - { - queue.push_back(mt->table); - queue.push_back(mt->metatable); - } - else if (state.log.getMutable(ty)) - { - // ClassTypeVars never contain free typevars. - } - else if (auto union_ = state.log.getMutable(ty)) - queue.insert(queue.end(), union_->options.begin(), union_->options.end()); - else if (auto intersection = state.log.getMutable(ty)) - queue.insert(queue.end(), intersection->parts.begin(), intersection->parts.end()); - else - { - } // Primitives, any, errors, and generics are left untouched. + state.log.replace(ty, BoundTypeVar{anyType}); } + else if (auto fun = state.log.getMutable(ty)) + { + queueTypePack(queue, seenTypePacks, state, fun->argTypes, anyTypePack); + queueTypePack(queue, seenTypePacks, state, fun->retType, anyTypePack); + } + else if (auto table = state.log.getMutable(ty)) + { + for (const auto& [_name, prop] : table->props) + queue.push_back(prop.type); + + if (table->indexer) + { + queue.push_back(table->indexer->indexType); + queue.push_back(table->indexer->indexResultType); + } + } + else if (auto mt = state.log.getMutable(ty)) + { + queue.push_back(mt->table); + queue.push_back(mt->metatable); + } + else if (state.log.getMutable(ty)) + { + // ClassTypeVars never contain free typevars. + } + else if (auto union_ = state.log.getMutable(ty)) + queue.insert(queue.end(), union_->options.begin(), union_->options.end()); + else if (auto intersection = state.log.getMutable(ty)) + queue.insert(queue.end(), intersection->parts.begin(), intersection->parts.end()); else { - TypeId ty = follow(queue.back()); - queue.pop_back(); - if (seen.find(ty)) - continue; - seen.insert(ty); - - if (get(ty)) - { - state.DEPRECATED_log(ty); - *asMutable(ty) = BoundTypeVar{anyType}; - } - else if (auto fun = get(ty)) - { - queueTypePack(queue, seenTypePacks, state, fun->argTypes, anyTypePack); - queueTypePack(queue, seenTypePacks, state, fun->retType, anyTypePack); - } - else if (auto table = get(ty)) - { - for (const auto& [_name, prop] : table->props) - queue.push_back(prop.type); - - if (table->indexer) - { - queue.push_back(table->indexer->indexType); - queue.push_back(table->indexer->indexResultType); - } - } - else if (auto mt = get(ty)) - { - queue.push_back(mt->table); - queue.push_back(mt->metatable); - } - else if (get(ty)) - { - // ClassTypeVars never contain free typevars. - } - else if (auto union_ = get(ty)) - queue.insert(queue.end(), union_->options.begin(), union_->options.end()); - else if (auto intersection = get(ty)) - queue.insert(queue.end(), intersection->parts.begin(), intersection->parts.end()); - else - { - } // Primitives, any, errors, and generics are left untouched. - } + } // Primitives, any, errors, and generics are left untouched. } } @@ -3038,79 +2272,39 @@ void Unifier::occursCheck(DenseHashSet& seen, TypeId needle, TypeId hays occursCheck(seen, needle, tv); }; - if (FFlag::LuauUseCommittingTxnLog) + needle = log.follow(needle); + haystack = log.follow(haystack); + + if (seen.find(haystack)) + return; + + seen.insert(haystack); + + if (log.getMutable(needle)) + return; + + if (!log.getMutable(needle)) + ice("Expected needle to be free"); + + if (needle == haystack) { - needle = log.follow(needle); - haystack = log.follow(haystack); + reportError(TypeError{location, OccursCheckFailed{}}); + log.replace(needle, *getSingletonTypes().errorRecoveryType()); - if (seen.find(haystack)) - return; - - seen.insert(haystack); - - if (log.getMutable(needle)) - return; - - if (!log.getMutable(needle)) - ice("Expected needle to be free"); - - if (needle == haystack) - { - reportError(TypeError{location, OccursCheckFailed{}}); - log.replace(needle, *getSingletonTypes().errorRecoveryType()); - - return; - } - - if (log.getMutable(haystack)) - return; - else if (auto a = log.getMutable(haystack)) - { - for (TypeId ty : a->options) - check(ty); - } - else if (auto a = log.getMutable(haystack)) - { - for (TypeId ty : a->parts) - check(ty); - } + return; } - else + + if (log.getMutable(haystack)) + return; + else if (auto a = log.getMutable(haystack)) { - needle = follow(needle); - haystack = follow(haystack); - - if (seen.find(haystack)) - return; - - seen.insert(haystack); - - if (get(needle)) - return; - - if (!get(needle)) - ice("Expected needle to be free"); - - if (needle == haystack) - { - reportError(TypeError{location, OccursCheckFailed{}}); - DEPRECATED_log(needle); - *asMutable(needle) = *getSingletonTypes().errorRecoveryType(); - return; - } - - if (get(haystack)) - return; - else if (auto a = get(haystack)) - { - for (TypeId ty : a->options) - check(ty); - } - else if (auto a = get(haystack)) - { - for (TypeId ty : a->parts) - check(ty); - } + for (TypeId ty : a->options) + check(ty); + } + else if (auto a = log.getMutable(haystack)) + { + for (TypeId ty : a->parts) + check(ty); } } @@ -3123,87 +2317,45 @@ void Unifier::occursCheck(TypePackId needle, TypePackId haystack) void Unifier::occursCheck(DenseHashSet& seen, TypePackId needle, TypePackId haystack) { - if (FFlag::LuauUseCommittingTxnLog) + needle = log.follow(needle); + haystack = log.follow(haystack); + + if (seen.find(haystack)) + return; + + seen.insert(haystack); + + if (log.getMutable(needle)) + return; + + if (!log.getMutable(needle)) + ice("Expected needle pack to be free"); + + RecursionLimiter _ra(&sharedState.counters.recursionCount, FInt::LuauTypeInferRecursionLimit); + + while (!log.getMutable(haystack)) { - needle = log.follow(needle); - haystack = log.follow(haystack); - - if (seen.find(haystack)) - return; - - seen.insert(haystack); - - if (log.getMutable(needle)) - return; - - if (!log.getMutable(needle)) - ice("Expected needle pack to be free"); - - RecursionLimiter _ra(&sharedState.counters.recursionCount, FInt::LuauTypeInferRecursionLimit); - - while (!log.getMutable(haystack)) + if (needle == haystack) { - if (needle == haystack) - { - reportError(TypeError{location, OccursCheckFailed{}}); - log.replace(needle, *getSingletonTypes().errorRecoveryTypePack()); + reportError(TypeError{location, OccursCheckFailed{}}); + log.replace(needle, *getSingletonTypes().errorRecoveryTypePack()); - return; - } - - if (auto a = get(haystack); a && a->tail) - { - haystack = log.follow(*a->tail); - continue; - } - - break; + return; } - } - else - { - needle = follow(needle); - haystack = follow(haystack); - if (seen.find(haystack)) - return; - - seen.insert(haystack); - - if (get(needle)) - return; - - if (!get(needle)) - ice("Expected needle pack to be free"); - - RecursionLimiter _ra(&sharedState.counters.recursionCount, FInt::LuauTypeInferRecursionLimit); - - while (!get(haystack)) + if (auto a = get(haystack); a && a->tail) { - if (needle == haystack) - { - reportError(TypeError{location, OccursCheckFailed{}}); - DEPRECATED_log(needle); - *asMutable(needle) = *getSingletonTypes().errorRecoveryTypePack(); - } - - if (auto a = get(haystack); a && a->tail) - { - haystack = follow(*a->tail); - continue; - } - - break; + haystack = log.follow(*a->tail); + continue; } + + break; } } Unifier Unifier::makeChildUnifier() { - if (FFlag::LuauUseCommittingTxnLog) - return Unifier{types, mode, log.sharedSeen, location, variance, sharedState, &log}; - else - return Unifier{types, mode, DEPRECATED_log.sharedSeen, location, variance, sharedState, &log}; + return Unifier{types, mode, log.sharedSeen, location, variance, sharedState, &log}; } bool Unifier::isNonstrictMode() const diff --git a/VM/include/lua.h b/VM/include/lua.h index 0a561f27..274c4ed9 100644 --- a/VM/include/lua.h +++ b/VM/include/lua.h @@ -229,19 +229,46 @@ LUA_API void lua_setthreaddata(lua_State* L, void* data); enum lua_GCOp { + /* stop and resume incremental garbage collection */ LUA_GCSTOP, LUA_GCRESTART, + + /* run a full GC cycle; not recommended for latency sensitive applications */ LUA_GCCOLLECT, + + /* return the heap size in KB and the remainder in bytes */ LUA_GCCOUNT, LUA_GCCOUNTB, + + /* return 1 if GC is active (not stopped); note that GC may not be actively collecting even if it's running */ LUA_GCISRUNNING, - // garbage collection is handled by 'assists' that perform some amount of GC work matching pace of allocation - // explicit GC steps allow to perform some amount of work at custom points to offset the need for GC assists - // note that GC might also be paused for some duration (until bytes allocated meet the threshold) - // if an explicit step is performed during this pause, it will trigger the start of the next collection cycle + /* + ** perform an explicit GC step, with the step size specified in KB + ** + ** garbage collection is handled by 'assists' that perform some amount of GC work matching pace of allocation + ** explicit GC steps allow to perform some amount of work at custom points to offset the need for GC assists + ** note that GC might also be paused for some duration (until bytes allocated meet the threshold) + ** if an explicit step is performed during this pause, it will trigger the start of the next collection cycle + */ LUA_GCSTEP, + /* + ** tune GC parameters G (goal), S (step multiplier) and step size (usually best left ignored) + ** + ** garbage collection is incremental and tries to maintain the heap size to balance memory and performance overhead + ** this overhead is determined by G (goal) which is the ratio between total heap size and the amount of live data in it + ** G is specified in percentages; by default G=200% which means that the heap is allowed to grow to ~2x the size of live data. + ** + ** collector tries to collect S% of allocated bytes by interrupting the application after step size bytes were allocated. + ** when S is too small, collector may not be able to catch up and the effective goal that can be reached will be larger. + ** S is specified in percentages; by default S=200% which means that collector will run at ~2x the pace of allocations. + ** + ** it is recommended to set S in the interval [100 / (G - 100), 100 + 100 / (G - 100))] with a minimum value of 150%; for example: + ** - for G=200%, S should be in the interval [150%, 200%] + ** - for G=150%, S should be in the interval [200%, 300%] + ** - for G=125%, S should be in the interval [400%, 500%] + */ LUA_GCSETGOAL, LUA_GCSETSTEPMUL, LUA_GCSETSTEPSIZE, diff --git a/VM/include/luaconf.h b/VM/include/luaconf.h index c5bf1c18..b93cbf7c 100644 --- a/VM/include/luaconf.h +++ b/VM/include/luaconf.h @@ -59,33 +59,6 @@ #define LUA_IDSIZE 256 #endif -/* -@@ LUAI_GCGOAL defines the desired top heap size in relation to the live heap -@* size at the end of the GC cycle -** CHANGE it if you want the GC to run faster or slower (higher values -** mean larger GC pauses which mean slower collection.) You can also change -** this value dynamically. -*/ -#ifndef LUAI_GCGOAL -#define LUAI_GCGOAL 200 /* 200% (allow heap to double compared to live heap size) */ -#endif - -/* -@@ LUAI_GCSTEPMUL / LUAI_GCSTEPSIZE define the default speed of garbage collection -@* relative to memory allocation. -** Every LUAI_GCSTEPSIZE KB allocated, incremental collector collects LUAI_GCSTEPSIZE -** times LUAI_GCSTEPMUL% bytes. -** CHANGE it if you want to change the granularity of the garbage -** collection. -*/ -#ifndef LUAI_GCSTEPMUL -#define LUAI_GCSTEPMUL 200 /* GC runs 'twice the speed' of memory allocation */ -#endif - -#ifndef LUAI_GCSTEPSIZE -#define LUAI_GCSTEPSIZE 1 /* GC runs every KB of memory allocation */ -#endif - /* LUA_MINSTACK is the guaranteed number of Lua stack slots available to a C function */ #ifndef LUA_MINSTACK #define LUA_MINSTACK 20 diff --git a/VM/include/lualib.h b/VM/include/lualib.h index baf27b47..bebd0a0f 100644 --- a/VM/include/lualib.h +++ b/VM/include/lualib.h @@ -54,6 +54,8 @@ LUALIB_API lua_State* luaL_newstate(void); LUALIB_API const char* luaL_findtable(lua_State* L, int idx, const char* fname, int szhint); +LUALIB_API const char* luaL_typename(lua_State* L, int idx); + /* ** =============================================================== ** some useful macros @@ -66,8 +68,6 @@ LUALIB_API const char* luaL_findtable(lua_State* L, int idx, const char* fname, #define luaL_checkstring(L, n) (luaL_checklstring(L, (n), NULL)) #define luaL_optstring(L, n, d) (luaL_optlstring(L, (n), (d), NULL)) -#define luaL_typename(L, i) lua_typename(L, lua_type(L, (i))) - #define luaL_getmetatable(L, n) (lua_getfield(L, LUA_REGISTRYINDEX, (n))) #define luaL_opt(L, f, n, d) (lua_isnoneornil(L, (n)) ? (d) : f(L, (n))) diff --git a/VM/src/laux.cpp b/VM/src/laux.cpp index 9a6f7793..9fe2ebb6 100644 --- a/VM/src/laux.cpp +++ b/VM/src/laux.cpp @@ -11,6 +11,8 @@ #include +LUAU_DYNAMIC_FASTFLAGVARIABLE(LuauMorePreciseLuaLTypeName, false) + /* convert a stack index to positive */ #define abs_index(L, i) ((i) > 0 || (i) <= LUA_REGISTRYINDEX ? (i) : lua_gettop(L) + (i) + 1) @@ -333,6 +335,19 @@ const char* luaL_findtable(lua_State* L, int idx, const char* fname, int szhint) return NULL; } +const char* luaL_typename(lua_State* L, int idx) +{ + if (DFFlag::LuauMorePreciseLuaLTypeName) + { + const TValue* obj = luaA_toobject(L, idx); + return luaT_objtypename(L, obj); + } + else + { + return lua_typename(L, lua_type(L, idx)); + } +} + /* ** {====================================================== ** Generic Buffer manipulation diff --git a/VM/src/lbaselib.cpp b/VM/src/lbaselib.cpp index 988fd315..2307598e 100644 --- a/VM/src/lbaselib.cpp +++ b/VM/src/lbaselib.cpp @@ -10,6 +10,8 @@ #include #include +LUAU_DYNAMIC_FASTFLAG(LuauMorePreciseLuaLTypeName) + static void writestring(const char* s, size_t l) { fwrite(s, 1, l, stdout); @@ -186,7 +188,14 @@ static int luaB_gcinfo(lua_State* L) static int luaB_type(lua_State* L) { luaL_checkany(L, 1); - lua_pushstring(L, luaL_typename(L, 1)); + if (DFFlag::LuauMorePreciseLuaLTypeName) + { + lua_pushstring(L, lua_typename(L, lua_type(L, 1))); + } + else + { + lua_pushstring(L, luaL_typename(L, 1)); + } return 1; } diff --git a/VM/src/lgc.h b/VM/src/lgc.h index cbeeebd4..ad8ee78a 100644 --- a/VM/src/lgc.h +++ b/VM/src/lgc.h @@ -6,6 +6,13 @@ #include "lobject.h" #include "lstate.h" +/* +** Default settings for GC tunables (settable via lua_gc) +*/ +#define LUAI_GCGOAL 200 /* 200% (allow heap to double compared to live heap size) */ +#define LUAI_GCSTEPMUL 200 /* GC runs 'twice the speed' of memory allocation */ +#define LUAI_GCSTEPSIZE 1 /* GC runs every KB of memory allocation */ + /* ** Possible states of the Garbage Collector */ diff --git a/tests/Autocomplete.test.cpp b/tests/Autocomplete.test.cpp index ce890ba8..55b0618a 100644 --- a/tests/Autocomplete.test.cpp +++ b/tests/Autocomplete.test.cpp @@ -14,7 +14,6 @@ LUAU_FASTFLAG(LuauTraceTypesInNonstrictMode2) LUAU_FASTFLAG(LuauSetMetatableDoesNotTimeTravel) -LUAU_FASTFLAG(LuauUseCommittingTxnLog) LUAU_FASTFLAG(LuauTableCloneType) using namespace Luau; @@ -1912,14 +1911,9 @@ local bar: @1= foo CHECK(!ac.entryMap.count("foo")); } -// Switch back to TEST_CASE_FIXTURE with regular ACFixture when removing the -// LuauUseCommittingTxnLog flag. -TEST_CASE("type_correct_function_no_parenthesis") +TEST_CASE_FIXTURE(ACFixture, "type_correct_function_no_parenthesis") { - ScopedFastFlag sff_LuauUseCommittingTxnLog = ScopedFastFlag("LuauUseCommittingTxnLog", true); - ACFixture fix; - - fix.check(R"( + check(R"( local function target(a: (number) -> number) return a(4) end local function bar1(a: number) return -a end local function bar2(a: string) return a .. 'x' end @@ -1927,7 +1921,7 @@ local function bar2(a: string) return a .. 'x' end return target(b@1 )"); - auto ac = fix.autocomplete('1'); + auto ac = autocomplete('1'); CHECK(ac.entryMap.count("bar1")); CHECK(ac.entryMap["bar1"].typeCorrect == TypeCorrectKind::Correct); @@ -1937,8 +1931,6 @@ return target(b@1 TEST_CASE_FIXTURE(ACFixture, "function_in_assignment_has_parentheses") { - ScopedFastFlag luauAutocompleteAvoidMutation("LuauAutocompleteAvoidMutation", true); - check(R"( local function bar(a: number) return -a end local abc = b@1 @@ -1952,8 +1944,6 @@ local abc = b@1 TEST_CASE_FIXTURE(ACFixture, "function_result_passed_to_function_has_parentheses") { - ScopedFastFlag luauAutocompleteAvoidMutation("LuauAutocompleteAvoidMutation", true); - check(R"( local function foo() return 1 end local function bar(a: number) return -a end @@ -1978,14 +1968,9 @@ local fp: @1= f CHECK(ac.entryMap.count("({ x: number, y: number }) -> number")); } -// Switch back to TEST_CASE_FIXTURE with regular ACFixture when removing the -// LuauUseCommittingTxnLog flag. -TEST_CASE("type_correct_keywords") +TEST_CASE_FIXTURE(ACFixture, "type_correct_keywords") { - ScopedFastFlag sff_LuauUseCommittingTxnLog = ScopedFastFlag("LuauUseCommittingTxnLog", true); - ACFixture fix; - - fix.check(R"( + check(R"( local function a(x: boolean) end local function b(x: number?) end local function c(x: (number) -> string) end @@ -2002,26 +1987,26 @@ local dc = d(f@4) local ec = e(f@5) )"); - auto ac = fix.autocomplete('1'); + auto ac = autocomplete('1'); CHECK(ac.entryMap.count("tru")); CHECK(ac.entryMap["tru"].typeCorrect == TypeCorrectKind::None); CHECK(ac.entryMap["true"].typeCorrect == TypeCorrectKind::Correct); CHECK(ac.entryMap["false"].typeCorrect == TypeCorrectKind::Correct); - ac = fix.autocomplete('2'); + ac = autocomplete('2'); CHECK(ac.entryMap.count("ni")); CHECK(ac.entryMap["ni"].typeCorrect == TypeCorrectKind::None); CHECK(ac.entryMap["nil"].typeCorrect == TypeCorrectKind::Correct); - ac = fix.autocomplete('3'); + ac = autocomplete('3'); CHECK(ac.entryMap.count("false")); CHECK(ac.entryMap["false"].typeCorrect == TypeCorrectKind::None); CHECK(ac.entryMap["function"].typeCorrect == TypeCorrectKind::Correct); - ac = fix.autocomplete('4'); + ac = autocomplete('4'); CHECK(ac.entryMap["function"].typeCorrect == TypeCorrectKind::Correct); - ac = fix.autocomplete('5'); + ac = autocomplete('5'); CHECK(ac.entryMap["function"].typeCorrect == TypeCorrectKind::Correct); } @@ -2512,23 +2497,21 @@ local t = { CHECK(ac.entryMap.count("second")); } -TEST_CASE("autocomplete_documentation_symbols") +TEST_CASE_FIXTURE(Fixture, "autocomplete_documentation_symbols") { - Fixture fix(FFlag::LuauUseCommittingTxnLog); - - fix.loadDefinition(R"( + loadDefinition(R"( declare y: { x: number, } )"); - fix.fileResolver.source["Module/A"] = R"( + fileResolver.source["Module/A"] = R"( local a = y. )"; - fix.frontend.check("Module/A"); + frontend.check("Module/A"); - auto ac = autocomplete(fix.frontend, "Module/A", Position{1, 21}, nullCallback); + auto ac = autocomplete(frontend, "Module/A", Position{1, 21}, nullCallback); REQUIRE(ac.entryMap.count("x")); CHECK_EQ(ac.entryMap["x"].documentationSymbol, "@test/global/y.x"); @@ -2646,8 +2629,6 @@ local a: A<(number, s@1> TEST_CASE_FIXTURE(ACFixture, "autocomplete_first_function_arg_expected_type") { - ScopedFastFlag luauAutocompleteAvoidMutation("LuauAutocompleteAvoidMutation", true); - check(R"( local function foo1() return 1 end local function foo2() return "1" end @@ -2720,7 +2701,6 @@ type A = () -> T TEST_CASE_FIXTURE(ACFixture, "autocomplete_oop_implicit_self") { - ScopedFastFlag flag("LuauMissingFollowACMetatables", true); check(R"( --!strict local Class = {} @@ -2764,8 +2744,6 @@ TEST_CASE_FIXTURE(ACFixture, "autocomplete_on_string_singletons") TEST_CASE_FIXTURE(ACFixture, "function_in_assignment_has_parentheses_2") { - ScopedFastFlag luauAutocompleteAvoidMutation("LuauAutocompleteAvoidMutation", true); - check(R"( local bar: ((number) -> number) & (number, number) -> number) local abc = b@1 diff --git a/tests/TypeInfer.aliases.test.cpp b/tests/TypeInfer.aliases.test.cpp index d584eb2d..711c0aa1 100644 --- a/tests/TypeInfer.aliases.test.cpp +++ b/tests/TypeInfer.aliases.test.cpp @@ -7,6 +7,8 @@ using namespace Luau; +LUAU_FASTFLAG(LuauFixIncorrectLineNumberDuplicateType) + TEST_SUITE_BEGIN("TypeAliases"); TEST_CASE_FIXTURE(Fixture, "cyclic_function_type_in_type_alias") @@ -241,6 +243,27 @@ TEST_CASE_FIXTURE(Fixture, "export_type_and_type_alias_are_duplicates") CHECK_EQ(dtd->name, "Foo"); } +TEST_CASE_FIXTURE(Fixture, "reported_location_is_correct_when_type_alias_are_duplicates") +{ + CheckResult result = check(R"( + type A = string + type B = number + type C = string + type B = number + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + auto dtd = get(result.errors[0]); + REQUIRE(dtd); + CHECK_EQ(dtd->name, "B"); + + if (FFlag::LuauFixIncorrectLineNumberDuplicateType) + CHECK_EQ(dtd->previousLocation.begin.line + 1, 3); + else + CHECK_EQ(dtd->previousLocation.begin.line + 1, 1); +} + TEST_CASE_FIXTURE(Fixture, "stringify_optional_parameterized_alias") { CheckResult result = check(R"( diff --git a/tests/TypeInfer.builtins.test.cpp b/tests/TypeInfer.builtins.test.cpp index bf990770..8da655b3 100644 --- a/tests/TypeInfer.builtins.test.cpp +++ b/tests/TypeInfer.builtins.test.cpp @@ -8,8 +8,6 @@ using namespace Luau; -LUAU_FASTFLAG(LuauUseCommittingTxnLog) - TEST_SUITE_BEGIN("BuiltinTests"); TEST_CASE_FIXTURE(Fixture, "math_things_are_defined") @@ -443,28 +441,19 @@ TEST_CASE_FIXTURE(Fixture, "os_time_takes_optional_date_table") CHECK_EQ(*typeChecker.numberType, *requireType("n3")); } -// Switch back to TEST_CASE_FIXTURE with regular Fixture when removing the -// LuauUseCommittingTxnLog flag. -TEST_CASE("thread_is_a_type") +TEST_CASE_FIXTURE(Fixture, "thread_is_a_type") { - Fixture fix(FFlag::LuauUseCommittingTxnLog); - - CheckResult result = fix.check(R"( + CheckResult result = check(R"( local co = coroutine.create(function() end) )"); - // Replace with LUAU_REQUIRE_NO_ERRORS(result) when using TEST_CASE_FIXTURE. - CHECK(result.errors.size() == 0); - CHECK_EQ(*fix.typeChecker.threadType, *fix.requireType("co")); + LUAU_REQUIRE_NO_ERRORS(result); + CHECK_EQ(*typeChecker.threadType, *requireType("co")); } -// Switch back to TEST_CASE_FIXTURE with regular Fixture when removing the -// LuauUseCommittingTxnLog flag. -TEST_CASE("coroutine_resume_anything_goes") +TEST_CASE_FIXTURE(Fixture, "coroutine_resume_anything_goes") { - Fixture fix(FFlag::LuauUseCommittingTxnLog); - - CheckResult result = fix.check(R"( + CheckResult result = check(R"( local function nifty(x, y) print(x, y) local z = coroutine.yield(1, 2) @@ -477,17 +466,12 @@ TEST_CASE("coroutine_resume_anything_goes") local answer = coroutine.resume(co, 3) )"); - // Replace with LUAU_REQUIRE_NO_ERRORS(result) when using TEST_CASE_FIXTURE. - CHECK(result.errors.size() == 0); + LUAU_REQUIRE_NO_ERRORS(result); } -// Switch back to TEST_CASE_FIXTURE with regular Fixture when removing the -// LuauUseCommittingTxnLog flag. -TEST_CASE("coroutine_wrap_anything_goes") +TEST_CASE_FIXTURE(Fixture, "coroutine_wrap_anything_goes") { - Fixture fix(FFlag::LuauUseCommittingTxnLog); - - CheckResult result = fix.check(R"( + CheckResult result = check(R"( --!nonstrict local function nifty(x, y) print(x, y) @@ -501,8 +485,7 @@ TEST_CASE("coroutine_wrap_anything_goes") local answer = f(3) )"); - // Replace with LUAU_REQUIRE_NO_ERRORS(result) when using TEST_CASE_FIXTURE. - CHECK(result.errors.size() == 0); + LUAU_REQUIRE_NO_ERRORS(result); } TEST_CASE_FIXTURE(Fixture, "setmetatable_should_not_mutate_persisted_types") @@ -961,4 +944,18 @@ TEST_CASE_FIXTURE(Fixture, "table_freeze_is_generic") CHECK_EQ("*unknown*", toString(requireType("d"))); } +TEST_CASE_FIXTURE(Fixture, "set_metatable_needs_arguments") +{ + ScopedFastFlag sff{"LuauSetMetaTableArgsCheck", true}; + CheckResult result = check(R"( +local a = {b=setmetatable} +a.b() +a:b() +a:b({}) + )"); + LUAU_REQUIRE_ERROR_COUNT(2, result); + CHECK_EQ(result.errors[0], (TypeError{Location{{2, 0}, {2, 5}}, CountMismatch{2, 0}})); + CHECK_EQ(result.errors[1], (TypeError{Location{{3, 0}, {3, 5}}, CountMismatch{2, 1}})); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.generics.test.cpp b/tests/TypeInfer.generics.test.cpp index c482847b..547fbab1 100644 --- a/tests/TypeInfer.generics.test.cpp +++ b/tests/TypeInfer.generics.test.cpp @@ -8,6 +8,8 @@ using namespace Luau; +LUAU_FASTFLAG(LuauFixArgumentCountMismatchAmountWithGenericTypes) + TEST_SUITE_BEGIN("GenericsTests"); TEST_CASE_FIXTURE(Fixture, "check_generic_function") @@ -786,4 +788,47 @@ local TheDispatcher: Dispatcher = { LUAU_REQUIRE_NO_ERRORS(result); } +TEST_CASE_FIXTURE(Fixture, "generic_argument_count_too_few") +{ + CheckResult result = check(R"( +function test(a: number) + return 1 +end + +function wrapper(f: (A...) -> number, ...: A...) +end + +wrapper(test) + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + if (FFlag::LuauFixArgumentCountMismatchAmountWithGenericTypes) + CHECK_EQ(toString(result.errors[0]), R"(Argument count mismatch. Function expects 2 arguments, but only 1 is specified)"); + else + CHECK_EQ(toString(result.errors[0]), R"(Argument count mismatch. Function expects 1 argument, but 1 is specified)"); +} + +TEST_CASE_FIXTURE(Fixture, "generic_argument_count_too_many") +{ + CheckResult result = check(R"( +function test2(a: number, b: string) + return 1 +end + +function wrapper(f: (A...) -> number, ...: A...) +end + +wrapper(test2, 1, "", 3) + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + if (FFlag::LuauFixArgumentCountMismatchAmountWithGenericTypes) + CHECK_EQ(toString(result.errors[0]), R"(Argument count mismatch. Function expects 3 arguments, but 4 are specified)"); + else + CHECK_EQ(toString(result.errors[0]), R"(Argument count mismatch. Function expects 1 argument, but 4 are specified)"); +} + + TEST_SUITE_END(); diff --git a/tests/TypeInfer.tables.test.cpp b/tests/TypeInfer.tables.test.cpp index da035ba1..a5eba5df 100644 --- a/tests/TypeInfer.tables.test.cpp +++ b/tests/TypeInfer.tables.test.cpp @@ -2334,4 +2334,54 @@ TEST_CASE_FIXTURE(Fixture, "pass_a_union_of_tables_to_a_function_that_requires_a REQUIRE_EQ("{| [any]: any, x: number, y: number |} | {| y: number |}", toString(requireType("b"))); } +TEST_CASE_FIXTURE(Fixture, "unifying_tables_shouldnt_uaf1") +{ + ScopedFastFlag sff{"LuauTxnLogCheckForInvalidation", true}; + + CheckResult result = check(R"( +-- This example produced a UAF at one point, caused by pointers to table types becoming +-- invalidated by child unifiers. (Calling log.concat can cause pointers to become invalid.) +type _Entry = { + a: number, + + middle: (self: _Entry) -> (), + + z: number +} + +export type AnyEntry = _Entry + +local Entry = {} +Entry.__index = Entry + +function Entry:dispose() + self:middle() + forgetChildren(self) -- unify free with sealed AnyEntry +end + +function forgetChildren(parent: AnyEntry) +end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "unifying_tables_shouldnt_uaf2") +{ + ScopedFastFlag sff{"LuauTxnLogCheckForInvalidation", true}; + + CheckResult result = check(R"( +-- Another example that UAFd, this time found by fuzzing. +local _ +do +_._ *= (_[{n0=_[{[{[_]=_,}]=_,}],}])[_] +_ = (_.n0) +end +_._ *= (_[false])[_] +_ = (_.cos) + )"); + + LUAU_REQUIRE_ERRORS(result); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.test.cpp b/tests/TypeInfer.test.cpp index f63579b5..d7bbad20 100644 --- a/tests/TypeInfer.test.cpp +++ b/tests/TypeInfer.test.cpp @@ -5144,7 +5144,6 @@ end TEST_CASE_FIXTURE(Fixture, "cli_50041_committing_txnlog_in_apollo_client_error") { - ScopedFastFlag committingTxnLog{"LuauUseCommittingTxnLog", true}; ScopedFastFlag subtypingVariance{"LuauTableSubtypingVariance2", true}; CheckResult result = check(R"( @@ -5355,4 +5354,41 @@ end LUAU_REQUIRE_NO_ERRORS(result); } +TEST_CASE_FIXTURE(Fixture, "function_decl_quantify_right_type") +{ + ScopedFastFlag statFunctionSimplify{"LuauStatFunctionSimplify", true}; + + fileResolver.source["game/isAMagicMock"] = R"( +--!nonstrict +return function(value) + return false +end + )"; + + CheckResult result = check(R"( +--!nonstrict +local MagicMock = {} +MagicMock.is = require(game.isAMagicMock) + +function MagicMock.is(value) + return false +end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "function_decl_non_self_sealed_overwrite") +{ + ScopedFastFlag statFunctionSimplify{"LuauStatFunctionSimplify", true}; + + CheckResult result = check(R"( +function string.len(): number + return 1 +end + )"); + + LUAU_REQUIRE_ERRORS(result); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.tryUnify.test.cpp b/tests/TypeInfer.tryUnify.test.cpp index f6ee3ccc..d8de2594 100644 --- a/tests/TypeInfer.tryUnify.test.cpp +++ b/tests/TypeInfer.tryUnify.test.cpp @@ -9,8 +9,6 @@ using namespace Luau; -LUAU_FASTFLAG(LuauUseCommittingTxnLog) - struct TryUnifyFixture : Fixture { TypeArena arena; @@ -43,8 +41,7 @@ TEST_CASE_FIXTURE(TryUnifyFixture, "compatible_functions_are_unified") state.tryUnify(&functionTwo, &functionOne); CHECK(state.errors.empty()); - if (FFlag::LuauUseCommittingTxnLog) - state.log.commit(); + state.log.commit(); CHECK_EQ(functionOne, functionTwo); } @@ -86,8 +83,7 @@ TEST_CASE_FIXTURE(TryUnifyFixture, "tables_can_be_unified") CHECK(state.errors.empty()); - if (FFlag::LuauUseCommittingTxnLog) - state.log.commit(); + state.log.commit(); CHECK_EQ(*getMutable(&tableOne)->props["foo"].type, *getMutable(&tableTwo)->props["foo"].type); } @@ -110,9 +106,6 @@ TEST_CASE_FIXTURE(TryUnifyFixture, "incompatible_tables_are_preserved") CHECK_EQ(1, state.errors.size()); - if (!FFlag::LuauUseCommittingTxnLog) - state.DEPRECATED_log.rollback(); - CHECK_NE(*getMutable(&tableOne)->props["foo"].type, *getMutable(&tableTwo)->props["foo"].type); } @@ -217,34 +210,6 @@ TEST_CASE_FIXTURE(TryUnifyFixture, "cli_41095_concat_log_in_sealed_table_unifica CHECK_EQ(toString(result.errors[1]), "Available overloads: ({a}, a) -> (); and ({a}, number, a) -> ()"); } -TEST_CASE("undo_new_prop_on_unsealed_table") -{ - ScopedFastFlag flags[] = { - {"LuauTableSubtypingVariance2", true}, - // This test makes no sense with a committing TxnLog. - {"LuauUseCommittingTxnLog", false}, - }; - // I am not sure how to make this happen in Luau code. - - TryUnifyFixture fix; - - TypeId unsealedTable = fix.arena.addType(TableTypeVar{TableState::Unsealed, TypeLevel{}}); - TypeId sealedTable = - fix.arena.addType(TableTypeVar{{{"prop", Property{getSingletonTypes().numberType}}}, std::nullopt, TypeLevel{}, TableState::Sealed}); - - const TableTypeVar* ttv = get(unsealedTable); - REQUIRE(ttv); - - fix.state.tryUnify(sealedTable, unsealedTable); - - // To be honest, it's really quite spooky here that we're amending an unsealed table in this case. - CHECK(!ttv->props.empty()); - - fix.state.DEPRECATED_log.rollback(); - - CHECK(ttv->props.empty()); -} - TEST_CASE_FIXTURE(TryUnifyFixture, "free_tail_is_grown_properly") { TypePackId threeNumbers = arena.addTypePack(TypePack{{typeChecker.numberType, typeChecker.numberType, typeChecker.numberType}, std::nullopt}); @@ -267,11 +232,6 @@ TEST_CASE_FIXTURE(TryUnifyFixture, "recursive_metatable_getmatchtag") TEST_CASE_FIXTURE(TryUnifyFixture, "cli_50320_follow_in_any_unification") { - ScopedFastFlag sffs[] = { - {"LuauUseCommittingTxnLog", true}, - {"LuauFollowWithCommittingTxnLogInAnyUnification", true}, - }; - TypePackVar free{FreeTypePack{TypeLevel{}}}; TypePackVar target{TypePack{}}; diff --git a/tests/TypeInfer.typePacks.cpp b/tests/TypeInfer.typePacks.cpp index 6b96f449..fcc21c18 100644 --- a/tests/TypeInfer.typePacks.cpp +++ b/tests/TypeInfer.typePacks.cpp @@ -9,8 +9,6 @@ using namespace Luau; -LUAU_FASTFLAG(LuauUseCommittingTxnLog) - TEST_SUITE_BEGIN("TypePackTests"); TEST_CASE_FIXTURE(Fixture, "infer_multi_return") @@ -264,13 +262,9 @@ TEST_CASE_FIXTURE(Fixture, "variadic_pack_syntax") CHECK_EQ(toString(requireType("foo")), "(...number) -> ()"); } -// Switch back to TEST_CASE_FIXTURE with regular Fixture when removing the -// LuauUseCommittingTxnLog flag. -TEST_CASE("type_pack_hidden_free_tail_infinite_growth") +TEST_CASE_FIXTURE(Fixture, "type_pack_hidden_free_tail_infinite_growth") { - Fixture fix(FFlag::LuauUseCommittingTxnLog); - - CheckResult result = fix.check(R"( + CheckResult result = check(R"( --!nonstrict if _ then _[function(l0)end],l0 = _ @@ -282,8 +276,7 @@ elseif _ then end )"); - // Switch back to LUAU_REQUIRE_ERRORS(result) when using TEST_CASE_FIXTURE. - CHECK(result.errors.size() > 0); + LUAU_REQUIRE_ERRORS(result); } TEST_CASE_FIXTURE(Fixture, "variadic_argument_tail") diff --git a/tests/TypeInfer.unionTypes.test.cpp b/tests/TypeInfer.unionTypes.test.cpp index 0e0b6ebb..ad4cecd8 100644 --- a/tests/TypeInfer.unionTypes.test.cpp +++ b/tests/TypeInfer.unionTypes.test.cpp @@ -7,7 +7,6 @@ #include "doctest.h" LUAU_FASTFLAG(LuauEqConstraint) -LUAU_FASTFLAG(LuauUseCommittingTxnLog) using namespace Luau; @@ -282,19 +281,16 @@ local c = b:foo(1, 2) CHECK_EQ("Value of type 'A?' could be nil", toString(result.errors[0])); } -TEST_CASE("optional_union_follow") +TEST_CASE_FIXTURE(Fixture, "optional_union_follow") { - Fixture fix(FFlag::LuauUseCommittingTxnLog); - - CheckResult result = fix.check(R"( + CheckResult result = check(R"( local y: number? = 2 local x = y local function f(a: number, b: typeof(x), c: typeof(x)) return -a end return f() )"); - REQUIRE_EQ(result.errors.size(), 1); - // LUAU_REQUIRE_ERROR_COUNT(1, result); + LUAU_REQUIRE_ERROR_COUNT(1, result); auto acm = get(result.errors[0]); REQUIRE(acm); diff --git a/tools/LuauVisualize.py b/tools/LuauVisualize.py new file mode 100644 index 00000000..40f8d6be --- /dev/null +++ b/tools/LuauVisualize.py @@ -0,0 +1,107 @@ +# This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details + +# HACK: LLDB's python API doesn't afford anything helpful for getting at variadic template parameters. +# We're forced to resort to parsing names as strings. +def templateParams(s): + depth = 0 + start = s.find('<') + 1 + result = [] + for i, c in enumerate(s[start:], start): + if c == '<': + depth += 1 + elif c == '>': + if depth == 0: + result.append(s[start: i].strip()) + break + depth -= 1 + elif c == ',' and depth == 0: + result.append(s[start: i].strip()) + start = i + 1 + return result + +def getType(target, typeName): + stars = 0 + + typeName = typeName.strip() + while typeName.endswith('*'): + stars += 1 + typeName = typeName[:-1] + + if typeName.startswith('const '): + typeName = typeName[6:] + + ty = target.FindFirstType(typeName.strip()) + for _ in range(stars): + ty = ty.GetPointerType() + + return ty + +def luau_variant_summary(valobj, internal_dict, options): + type_id = valobj.GetChildMemberWithName("typeid").GetValueAsUnsigned() + storage = valobj.GetChildMemberWithName("storage") + params = templateParams(valobj.GetType().GetCanonicalType().GetName()) + stored_type = params[type_id] + value = storage.Cast(stored_type.GetPointerType()).Dereference() + return stored_type.GetDisplayTypeName() + " [" + value.GetValue() + "]" + +class LuauVariantSyntheticChildrenProvider: + node_names = ["type", "value"] + + def __init__(self, valobj, internal_dict): + self.valobj = valobj + self.type_index = None + self.current_type = None + self.type_params = [] + self.stored_value = None + + def num_children(self): + return len(self.node_names) + + def has_children(self): + return True + + def get_child_index(self, name): + try: + return self.node_names.index(name) + except ValueError: + return -1 + + def get_child_at_index(self, index): + try: + node = self.node_names[index] + except IndexError: + return None + + if node == "type": + if self.current_type: + return self.valobj.CreateValueFromExpression(node, f"(const char*)\"{self.current_type.GetDisplayTypeName()}\"") + else: + return self.valobj.CreateValueFromExpression(node, "(const char*)\"\"") + elif node == "value": + if self.stored_value is not None: + if self.current_type is not None: + return self.valobj.CreateValueFromData(node, self.stored_value.GetData(), self.current_type) + else: + return self.valobj.CreateValueExpression(node, "(const char*)\"\"") + else: + return self.valobj.CreateValueFromExpression(node, "(const char*)\"\"") + else: + return None + + def update(self): + self.type_index = self.valobj.GetChildMemberWithName("typeid").GetValueAsSigned() + self.type_params = templateParams(self.valobj.GetType().GetCanonicalType().GetName()) + + if len(self.type_params) > self.type_index: + self.current_type = getType(self.valobj.GetTarget(), self.type_params[self.type_index]) + + if self.current_type: + storage = self.valobj.GetChildMemberWithName("storage") + self.stored_value = storage.Cast(self.current_type.GetPointerType()).Dereference() + else: + self.stored_value = None + else: + self.current_type = None + self.stored_value = None + + return False diff --git a/tools/lldb-formatters.lldb b/tools/lldb-formatters.lldb new file mode 100644 index 00000000..3868ac20 --- /dev/null +++ b/tools/lldb-formatters.lldb @@ -0,0 +1,2 @@ +type synthetic add -x "^Luau::Variant<.+>$" -l LuauVisualize.LuauVariantSyntheticChildrenProvider +type summary add -x "^Luau::Variant<.+>$" -l LuauVisualize.luau_variant_summary From fe71aff7af4d09bc398bd14d128277c31c12e824 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Mon, 14 Mar 2022 10:30:49 -0700 Subject: [PATCH 193/399] Update library.md --- docs/_pages/library.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/_pages/library.md b/docs/_pages/library.md index 7695762d..28e7035a 100644 --- a/docs/_pages/library.md +++ b/docs/_pages/library.md @@ -128,7 +128,7 @@ Returns the type of the object, which is one of `"nil"`, `"boolean"`, `"number"` function typeof(obj: any): string ``` -Returns the type of the object; for userdata objects that have a metatable with the `__type` field *and* are defined by the host with an internal tag, returns the value for that key. +Returns the type of the object; for userdata objects that have a metatable with the `__type` field *and* are defined by the host (not `newproxy`), returns the value for that key. For custom userdata objects, such as ones returned by `newproxy`, this function returns `"userdata"` to make sure host-defined types can not be spoofed. ``` From 4fdead5e3ca0e1fd3dfa31ab295c44f7556773c6 Mon Sep 17 00:00:00 2001 From: Luca Salmin <40690017+morgoth990@users.noreply.github.com> Date: Mon, 14 Mar 2022 19:43:40 +0100 Subject: [PATCH 194/399] yield from interrupt #287 (#413) Co-authored-by: luca salmin --- VM/src/lvmexecute.cpp | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/VM/src/lvmexecute.cpp b/VM/src/lvmexecute.cpp index 6c31d36f..5e2ef688 100644 --- a/VM/src/lvmexecute.cpp +++ b/VM/src/lvmexecute.cpp @@ -77,6 +77,11 @@ if (LUAU_UNLIKELY(!!interrupt)) \ { /* the interrupt hook is called right before we advance pc */ \ VM_PROTECT(L->ci->savedpc++; interrupt(L, -1)); \ + if (L->status != 0) \ + { \ + L->ci->savedpc--; \ + goto exit; \ + } \ } \ } #endif From adecd840675c642f1f553d39930aa8014e82faa7 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Thu, 17 Mar 2022 17:06:25 -0700 Subject: [PATCH 195/399] Sync to upstream/release/519 --- Analysis/include/Luau/Error.h | 1 + Analysis/include/Luau/Unifier.h | 2 + Analysis/src/Autocomplete.cpp | 198 +- Analysis/src/EmbeddedBuiltinDefinitions.cpp | 144 +- Analysis/src/Error.cpp | 11 +- Analysis/src/Module.cpp | 7 + Analysis/src/TypeInfer.cpp | 127 +- Analysis/src/TypeVar.cpp | 10 +- Analysis/src/Unifier.cpp | 78 +- Ast/src/Parser.cpp | 117 +- Compiler/src/Builtins.cpp | 4 +- Compiler/src/Compiler.cpp | 4 - Sources.cmake | 7 + VM/include/lua.h | 15 +- VM/src/lapi.cpp | 20 +- VM/src/lbaselib.cpp | 16 +- VM/src/lgc.h | 4 +- VM/src/lmem.cpp | 2 +- VM/src/lstring.cpp | 2 +- VM/src/ltable.cpp | 71 +- VM/src/ltm.cpp | 3 +- VM/src/ludata.cpp | 8 +- VM/src/ludata.h | 3 + VM/src/lvmexecute.cpp | 5 + tests/Autocomplete.test.cpp | 162 + tests/Compiler.test.cpp | 2 - tests/Conformance.test.cpp | 142 +- tests/Linter.test.cpp | 2 - tests/Module.test.cpp | 10 +- tests/NonstrictMode.test.cpp | 22 +- tests/TypeInfer.aliases.test.cpp | 21 + tests/TypeInfer.anyerror.test.cpp | 335 ++ tests/TypeInfer.builtins.test.cpp | 60 + tests/TypeInfer.definitions.test.cpp | 18 + tests/TypeInfer.functions.test.cpp | 1338 +++++ tests/TypeInfer.generics.test.cpp | 301 ++ tests/TypeInfer.intersectionTypes.test.cpp | 28 + tests/TypeInfer.loops.test.cpp | 473 ++ tests/TypeInfer.modules.test.cpp | 310 ++ tests/TypeInfer.oop.test.cpp | 275 + tests/TypeInfer.operators.test.cpp | 759 +++ tests/TypeInfer.primitives.test.cpp | 100 + tests/TypeInfer.refinements.test.cpp | 18 + tests/TypeInfer.singletons.test.cpp | 116 +- tests/TypeInfer.tables.test.cpp | 500 ++ tests/TypeInfer.test.cpp | 4413 ----------------- tests/TypeInfer.typePacks.cpp | 83 + tests/TypeInfer.unionTypes.test.cpp | 16 + tests/conformance/basic.lua | 19 + tests/conformance/debugger.lua | 4 +- tests/conformance/errors.lua | 2 + tests/conformance/interrupt.lua | 11 + tools/{gdb-printers.py => gdb_printers.py} | 0 tools/lldb-formatters.lldb | 2 - tools/lldb_formatters.lldb | 2 + .../{LuauVisualize.py => lldb_formatters.py} | 0 56 files changed, 5643 insertions(+), 4760 deletions(-) create mode 100644 tests/TypeInfer.anyerror.test.cpp create mode 100644 tests/TypeInfer.functions.test.cpp create mode 100644 tests/TypeInfer.loops.test.cpp create mode 100644 tests/TypeInfer.modules.test.cpp create mode 100644 tests/TypeInfer.oop.test.cpp create mode 100644 tests/TypeInfer.operators.test.cpp create mode 100644 tests/TypeInfer.primitives.test.cpp create mode 100644 tests/conformance/interrupt.lua rename tools/{gdb-printers.py => gdb_printers.py} (100%) delete mode 100644 tools/lldb-formatters.lldb create mode 100644 tools/lldb_formatters.lldb rename tools/{LuauVisualize.py => lldb_formatters.py} (100%) diff --git a/Analysis/include/Luau/Error.h b/Analysis/include/Luau/Error.h index a71e0224..72350255 100644 --- a/Analysis/include/Luau/Error.h +++ b/Analysis/include/Luau/Error.h @@ -107,6 +107,7 @@ struct FunctionDoesNotTakeSelf struct FunctionRequiresSelf { + // TODO: Delete with LuauAnyInIsOptionalIsOptional int requiredExtraNils = 0; bool operator==(const FunctionRequiresSelf& rhs) const; diff --git a/Analysis/include/Luau/Unifier.h b/Analysis/include/Luau/Unifier.h index 71958f4a..f1ffbcc0 100644 --- a/Analysis/include/Luau/Unifier.h +++ b/Analysis/include/Luau/Unifier.h @@ -86,6 +86,8 @@ private: void tryUnifyIndexer(const TableIndexer& subIndexer, const TableIndexer& superIndexer); TypeId widen(TypeId ty); + TypePackId widen(TypePackId tp); + TypeId deeplyOptional(TypeId ty, std::unordered_map seen = {}); void cacheResult(TypeId subTy, TypeId superTy); diff --git a/Analysis/src/Autocomplete.cpp b/Analysis/src/Autocomplete.cpp index e94c432f..492edf25 100644 --- a/Analysis/src/Autocomplete.cpp +++ b/Analysis/src/Autocomplete.cpp @@ -14,6 +14,7 @@ #include LUAU_FASTFLAGVARIABLE(LuauIfElseExprFixCompletionIssue, false); +LUAU_FASTFLAG(LuauSelfCallAutocompleteFix) static const std::unordered_set kStatementStartingKeywords = { "while", "if", "local", "repeat", "function", "do", "for", "return", "break", "continue", "type", "export"}; @@ -228,11 +229,22 @@ static std::optional findExpectedTypeAt(const Module& module, AstNode* n return *it; } +static bool checkTypeMatch(TypeArena* typeArena, TypeId subTy, TypeId superTy) +{ + InternalErrorReporter iceReporter; + UnifierSharedState unifierState(&iceReporter); + Unifier unifier(typeArena, Mode::Strict, Location(), Variance::Covariant, unifierState); + + return unifier.canUnify(subTy, superTy).empty(); +} + static TypeCorrectKind checkTypeCorrectKind(const Module& module, TypeArena* typeArena, AstNode* node, Position position, TypeId ty) { ty = follow(ty); auto canUnify = [&typeArena](TypeId subTy, TypeId superTy) { + LUAU_ASSERT(!FFlag::LuauSelfCallAutocompleteFix); + InternalErrorReporter iceReporter; UnifierSharedState unifierState(&iceReporter); Unifier unifier(typeArena, Mode::Strict, Location(), Variance::Covariant, unifierState); @@ -249,20 +261,30 @@ static TypeCorrectKind checkTypeCorrectKind(const Module& module, TypeArena* typ TypeId expectedType = follow(*typeAtPosition); - auto checkFunctionType = [&canUnify, &expectedType](const FunctionTypeVar* ftv) { - auto [retHead, retTail] = flatten(ftv->retType); - - if (!retHead.empty() && canUnify(retHead.front(), expectedType)) - return true; - - // We might only have a variadic tail pack, check if the element is compatible - if (retTail) + auto checkFunctionType = [typeArena, &canUnify, &expectedType](const FunctionTypeVar* ftv) { + if (FFlag::LuauSelfCallAutocompleteFix) { - if (const VariadicTypePack* vtp = get(follow(*retTail)); vtp && canUnify(vtp->ty, expectedType)) - return true; - } + if (std::optional firstRetTy = first(ftv->retType)) + return checkTypeMatch(typeArena, *firstRetTy, expectedType); - return false; + return false; + } + else + { + auto [retHead, retTail] = flatten(ftv->retType); + + if (!retHead.empty() && canUnify(retHead.front(), expectedType)) + return true; + + // We might only have a variadic tail pack, check if the element is compatible + if (retTail) + { + if (const VariadicTypePack* vtp = get(follow(*retTail)); vtp && canUnify(vtp->ty, expectedType)) + return true; + } + + return false; + } }; // We also want to suggest functions that return compatible result @@ -281,7 +303,10 @@ static TypeCorrectKind checkTypeCorrectKind(const Module& module, TypeArena* typ } } - return canUnify(ty, expectedType) ? TypeCorrectKind::Correct : TypeCorrectKind::None; + if (FFlag::LuauSelfCallAutocompleteFix) + return checkTypeMatch(typeArena, ty, expectedType) ? TypeCorrectKind::Correct : TypeCorrectKind::None; + else + return canUnify(ty, expectedType) ? TypeCorrectKind::Correct : TypeCorrectKind::None; } enum class PropIndexType @@ -291,16 +316,22 @@ enum class PropIndexType Key, }; -static void autocompleteProps(const Module& module, TypeArena* typeArena, TypeId ty, PropIndexType indexType, const std::vector& nodes, - AutocompleteEntryMap& result, std::unordered_set& seen, std::optional containingClass = std::nullopt) +static void autocompleteProps(const Module& module, TypeArena* typeArena, TypeId rootTy, TypeId ty, PropIndexType indexType, + const std::vector& nodes, AutocompleteEntryMap& result, std::unordered_set& seen, + std::optional containingClass = std::nullopt) { + if (FFlag::LuauSelfCallAutocompleteFix) + rootTy = follow(rootTy); + ty = follow(ty); if (seen.count(ty)) return; seen.insert(ty); - auto isWrongIndexer = [indexType, useStrictFunctionIndexers = !!get(ty)](Luau::TypeId type) { + auto isWrongIndexer_DEPRECATED = [indexType, useStrictFunctionIndexers = !!get(ty)](Luau::TypeId type) { + LUAU_ASSERT(!FFlag::LuauSelfCallAutocompleteFix); + if (indexType == PropIndexType::Key) return false; @@ -331,6 +362,48 @@ static void autocompleteProps(const Module& module, TypeArena* typeArena, TypeId return colonIndex; } }; + auto isWrongIndexer = [typeArena, rootTy, indexType](Luau::TypeId type) { + LUAU_ASSERT(FFlag::LuauSelfCallAutocompleteFix); + + if (indexType == PropIndexType::Key) + return false; + + bool calledWithSelf = indexType == PropIndexType::Colon; + + auto isCompatibleCall = [typeArena, rootTy, calledWithSelf](const FunctionTypeVar* ftv) { + if (get(rootTy)) + { + // Calls on classes require strict match between how function is declared and how it's called + return calledWithSelf == ftv->hasSelf; + } + + if (std::optional firstArgTy = first(ftv->argTypes)) + { + if (checkTypeMatch(typeArena, rootTy, *firstArgTy)) + return calledWithSelf; + } + + return !calledWithSelf; + }; + + if (const FunctionTypeVar* ftv = get(type)) + return !isCompatibleCall(ftv); + + // For intersections, any part that is successful makes the whole call successful + if (const IntersectionTypeVar* itv = get(type)) + { + for (auto subType : itv->parts) + { + if (const FunctionTypeVar* ftv = get(Luau::follow(subType))) + { + if (isCompatibleCall(ftv)) + return false; + } + } + } + + return calledWithSelf; + }; auto fillProps = [&](const ClassTypeVar::Props& props) { for (const auto& [name, prop] : props) @@ -349,7 +422,7 @@ static void autocompleteProps(const Module& module, TypeArena* typeArena, TypeId AutocompleteEntryKind::Property, type, prop.deprecated, - isWrongIndexer(type), + FFlag::LuauSelfCallAutocompleteFix ? isWrongIndexer(type) : isWrongIndexer_DEPRECATED(type), typeCorrect, containingClass, &prop, @@ -361,34 +434,60 @@ static void autocompleteProps(const Module& module, TypeArena* typeArena, TypeId } }; - if (auto cls = get(ty)) - { - containingClass = containingClass.value_or(cls); - fillProps(cls->props); - if (cls->parent) - autocompleteProps(module, typeArena, *cls->parent, indexType, nodes, result, seen, cls); - } - else if (auto tbl = get(ty)) - fillProps(tbl->props); - else if (auto mt = get(ty)) - { - autocompleteProps(module, typeArena, mt->table, indexType, nodes, result, seen); - - auto mtable = get(mt->metatable); - if (!mtable) - return; - + auto fillMetatableProps = [&](const TableTypeVar* mtable) { auto indexIt = mtable->props.find("__index"); if (indexIt != mtable->props.end()) { TypeId followed = follow(indexIt->second.type); if (get(followed) || get(followed)) - autocompleteProps(module, typeArena, followed, indexType, nodes, result, seen); + { + autocompleteProps(module, typeArena, rootTy, followed, indexType, nodes, result, seen); + } else if (auto indexFunction = get(followed)) { std::optional indexFunctionResult = first(indexFunction->retType); if (indexFunctionResult) - autocompleteProps(module, typeArena, *indexFunctionResult, indexType, nodes, result, seen); + autocompleteProps(module, typeArena, rootTy, *indexFunctionResult, indexType, nodes, result, seen); + } + } + }; + + if (auto cls = get(ty)) + { + containingClass = containingClass.value_or(cls); + fillProps(cls->props); + if (cls->parent) + autocompleteProps(module, typeArena, rootTy, *cls->parent, indexType, nodes, result, seen, cls); + } + else if (auto tbl = get(ty)) + fillProps(tbl->props); + else if (auto mt = get(ty)) + { + autocompleteProps(module, typeArena, rootTy, mt->table, indexType, nodes, result, seen); + + if (FFlag::LuauSelfCallAutocompleteFix) + { + if (auto mtable = get(mt->metatable)) + fillMetatableProps(mtable); + } + else + { + auto mtable = get(mt->metatable); + if (!mtable) + return; + + auto indexIt = mtable->props.find("__index"); + if (indexIt != mtable->props.end()) + { + TypeId followed = follow(indexIt->second.type); + if (get(followed) || get(followed)) + autocompleteProps(module, typeArena, rootTy, followed, indexType, nodes, result, seen); + else if (auto indexFunction = get(followed)) + { + std::optional indexFunctionResult = first(indexFunction->retType); + if (indexFunctionResult) + autocompleteProps(module, typeArena, rootTy, *indexFunctionResult, indexType, nodes, result, seen); + } } } } @@ -400,7 +499,7 @@ static void autocompleteProps(const Module& module, TypeArena* typeArena, TypeId AutocompleteEntryMap inner; std::unordered_set innerSeen = seen; - autocompleteProps(module, typeArena, ty, indexType, nodes, inner, innerSeen); + autocompleteProps(module, typeArena, rootTy, ty, indexType, nodes, inner, innerSeen); for (auto& pair : inner) result.insert(pair); @@ -423,14 +522,17 @@ static void autocompleteProps(const Module& module, TypeArena* typeArena, TypeId if (iter == endIter) return; - autocompleteProps(module, typeArena, *iter, indexType, nodes, result, seen); + autocompleteProps(module, typeArena, rootTy, *iter, indexType, nodes, result, seen); ++iter; while (iter != endIter) { AutocompleteEntryMap inner; - std::unordered_set innerSeen = seen; + std::unordered_set innerSeen; + + if (!FFlag::LuauSelfCallAutocompleteFix) + innerSeen = seen; if (isNil(*iter)) { @@ -438,7 +540,7 @@ static void autocompleteProps(const Module& module, TypeArena* typeArena, TypeId continue; } - autocompleteProps(module, typeArena, *iter, indexType, nodes, inner, innerSeen); + autocompleteProps(module, typeArena, rootTy, *iter, indexType, nodes, inner, innerSeen); std::unordered_set toRemove; @@ -455,6 +557,18 @@ static void autocompleteProps(const Module& module, TypeArena* typeArena, TypeId ++iter; } } + else if (auto pt = get(ty); pt && FFlag::LuauSelfCallAutocompleteFix) + { + if (pt->metatable) + { + if (auto mtable = get(*pt->metatable)) + fillMetatableProps(mtable); + } + } + else if (FFlag::LuauSelfCallAutocompleteFix && get(get(ty))) + { + autocompleteProps(module, typeArena, rootTy, getSingletonTypes().stringType, indexType, nodes, result, seen); + } } static void autocompleteKeywords( @@ -482,7 +596,7 @@ static void autocompleteProps( const Module& module, TypeArena* typeArena, TypeId ty, PropIndexType indexType, const std::vector& nodes, AutocompleteEntryMap& result) { std::unordered_set seen; - autocompleteProps(module, typeArena, ty, indexType, nodes, result, seen); + autocompleteProps(module, typeArena, ty, ty, indexType, nodes, result, seen); } AutocompleteEntryMap autocompleteProps( @@ -1352,7 +1466,7 @@ static AutocompleteResult autocomplete(const SourceModule& sourceModule, const M TypeId ty = follow(*it); PropIndexType indexType = indexName->op == ':' ? PropIndexType::Colon : PropIndexType::Point; - if (isString(ty)) + if (!FFlag::LuauSelfCallAutocompleteFix && isString(ty)) return {autocompleteProps(*module, typeArena, typeChecker.globalScope->bindings[AstName{"string"}].typeId, indexType, finder.ancestry), finder.ancestry}; else diff --git a/Analysis/src/EmbeddedBuiltinDefinitions.cpp b/Analysis/src/EmbeddedBuiltinDefinitions.cpp index 471b61ad..be3fcd7d 100644 --- a/Analysis/src/EmbeddedBuiltinDefinitions.cpp +++ b/Analysis/src/EmbeddedBuiltinDefinitions.cpp @@ -95,104 +95,104 @@ declare os: { declare function require(target: any): any -declare function getfenv(target: any?): { [string]: any } +declare function getfenv(target: any): { [string]: any } declare _G: any declare _VERSION: string declare function gcinfo(): number - declare function print(...: T...) +declare function print(...: T...) - declare function type(value: T): string - declare function typeof(value: T): string - - -- `assert` has a magic function attached that will give more detailed type information - declare function assert(value: T, errorMessage: string?): T +declare function type(value: T): string +declare function typeof(value: T): string - declare function error(message: T, level: number?) +-- `assert` has a magic function attached that will give more detailed type information +declare function assert(value: T, errorMessage: string?): T - declare function tostring(value: T): string - declare function tonumber(value: T, radix: number?): number? +declare function error(message: T, level: number?) - declare function rawequal(a: T1, b: T2): boolean - declare function rawget(tab: {[K]: V}, k: K): V - declare function rawset(tab: {[K]: V}, k: K, v: V): {[K]: V} +declare function tostring(value: T): string +declare function tonumber(value: T, radix: number?): number? - declare function setfenv(target: number | (T...) -> R..., env: {[string]: any}): ((T...) -> R...)? +declare function rawequal(a: T1, b: T2): boolean +declare function rawget(tab: {[K]: V}, k: K): V +declare function rawset(tab: {[K]: V}, k: K, v: V): {[K]: V} - declare function ipairs(tab: {V}): (({V}, number) -> (number, V), {V}, number) +declare function setfenv(target: number | (T...) -> R..., env: {[string]: any}): ((T...) -> R...)? - declare function pcall(f: (A...) -> R..., ...: A...): (boolean, R...) +declare function ipairs(tab: {V}): (({V}, number) -> (number, V), {V}, number) - -- FIXME: The actual type of `xpcall` is: - -- (f: (A...) -> R1..., err: (E) -> R2..., A...) -> (true, R1...) | (false, R2...) - -- Since we can't represent the return value, we use (boolean, R1...). - declare function xpcall(f: (A...) -> R1..., err: (E) -> R2..., ...: A...): (boolean, R1...) +declare function pcall(f: (A...) -> R..., ...: A...): (boolean, R...) - -- `select` has a magic function attached to provide more detailed type information - declare function select(i: string | number, ...: A...): ...any +-- FIXME: The actual type of `xpcall` is: +-- (f: (A...) -> R1..., err: (E) -> R2..., A...) -> (true, R1...) | (false, R2...) +-- Since we can't represent the return value, we use (boolean, R1...). +declare function xpcall(f: (A...) -> R1..., err: (E) -> R2..., ...: A...): (boolean, R1...) - -- FIXME: This type is not entirely correct - `loadstring` returns a function or - -- (nil, string). - declare function loadstring(src: string, chunkname: string?): (((A...) -> any)?, string?) +-- `select` has a magic function attached to provide more detailed type information +declare function select(i: string | number, ...: A...): ...any - declare function newproxy(mt: boolean?): any +-- FIXME: This type is not entirely correct - `loadstring` returns a function or +-- (nil, string). +declare function loadstring(src: string, chunkname: string?): (((A...) -> any)?, string?) - declare coroutine: { - create: ((A...) -> R...) -> thread, - resume: (thread, A...) -> (boolean, R...), - running: () -> thread, - status: (thread) -> string, - -- FIXME: This technically returns a function, but we can't represent this yet. - wrap: ((A...) -> R...) -> any, - yield: (A...) -> R..., - isyieldable: () -> boolean, - close: (thread) -> (boolean, any?) - } +declare function newproxy(mt: boolean?): any - declare table: { - concat: ({V}, string?, number?, number?) -> string, - insert: (({V}, V) -> ()) & (({V}, number, V) -> ()), - maxn: ({V}) -> number, - remove: ({V}, number?) -> V?, - sort: ({V}, ((V, V) -> boolean)?) -> (), - create: (number, V?) -> {V}, - find: ({V}, V, number?) -> number?, +declare coroutine: { + create: ((A...) -> R...) -> thread, + resume: (thread, A...) -> (boolean, R...), + running: () -> thread, + status: (thread) -> string, + -- FIXME: This technically returns a function, but we can't represent this yet. + wrap: ((A...) -> R...) -> any, + yield: (A...) -> R..., + isyieldable: () -> boolean, + close: (thread) -> (boolean, any) +} - unpack: ({V}, number?, number?) -> ...V, - pack: (...V) -> { n: number, [number]: V }, +declare table: { + concat: ({V}, string?, number?, number?) -> string, + insert: (({V}, V) -> ()) & (({V}, number, V) -> ()), + maxn: ({V}) -> number, + remove: ({V}, number?) -> V?, + sort: ({V}, ((V, V) -> boolean)?) -> (), + create: (number, V?) -> {V}, + find: ({V}, V, number?) -> number?, - getn: ({V}) -> number, - foreach: ({[K]: V}, (K, V) -> ()) -> (), - foreachi: ({V}, (number, V) -> ()) -> (), + unpack: ({V}, number?, number?) -> ...V, + pack: (...V) -> { n: number, [number]: V }, - move: ({V}, number, number, number, {V}?) -> {V}, - clear: ({[K]: V}) -> (), + getn: ({V}) -> number, + foreach: ({[K]: V}, (K, V) -> ()) -> (), + foreachi: ({V}, (number, V) -> ()) -> (), - isfrozen: ({[K]: V}) -> boolean, - } + move: ({V}, number, number, number, {V}?) -> {V}, + clear: ({[K]: V}) -> (), - declare debug: { - info: ((thread, number, string) -> R...) & ((number, string) -> R...) & (((A...) -> R1..., string) -> R2...), - traceback: ((string?, number?) -> string) & ((thread, string?, number?) -> string), - } + isfrozen: ({[K]: V}) -> boolean, +} - declare utf8: { - char: (number, ...number) -> string, - charpattern: string, - codes: (string) -> ((string, number) -> (number, number), string, number), - -- FIXME - codepoint: (string, number?, number?) -> (number, ...number), - len: (string, number?, number?) -> (number?, number?), - offset: (string, number?, number?) -> number, - nfdnormalize: (string) -> string, - nfcnormalize: (string) -> string, - graphemes: (string, number?, number?) -> (() -> (number, number)), - } +declare debug: { + info: ((thread, number, string) -> R...) & ((number, string) -> R...) & (((A...) -> R1..., string) -> R2...), + traceback: ((string?, number?) -> string) & ((thread, string?, number?) -> string), +} - -- Cannot use `typeof` here because it will produce a polytype when we expect a monotype. - declare function unpack(tab: {V}, i: number?, j: number?): ...V +declare utf8: { + char: (number, ...number) -> string, + charpattern: string, + codes: (string) -> ((string, number) -> (number, number), string, number), + -- FIXME + codepoint: (string, number?, number?) -> (number, ...number), + len: (string, number?, number?) -> (number?, number?), + offset: (string, number?, number?) -> number, + nfdnormalize: (string) -> string, + nfcnormalize: (string) -> string, + graphemes: (string, number?, number?) -> (() -> (number, number)), +} + +-- Cannot use `typeof` here because it will produce a polytype when we expect a monotype. +declare function unpack(tab: {V}, i: number?, j: number?): ...V )BUILTIN_SRC"; diff --git a/Analysis/src/Error.cpp b/Analysis/src/Error.cpp index 88069f1f..26d3b76d 100644 --- a/Analysis/src/Error.cpp +++ b/Analysis/src/Error.cpp @@ -7,6 +7,8 @@ #include +LUAU_FASTFLAGVARIABLE(BetterDiagnosticCodesInStudio, false); + static std::string wrongNumberOfArgsString(size_t expectedCount, size_t actualCount, const char* argPrefix = nullptr, bool isVariadic = false) { std::string s = "expects "; @@ -223,7 +225,14 @@ struct ErrorConverter std::string operator()(const Luau::SyntaxError& e) const { - return "Syntax error: " + e.message; + if (FFlag::BetterDiagnosticCodesInStudio) + { + return e.message; + } + else + { + return "Syntax error: " + e.message; + } } std::string operator()(const Luau::CodeTooComplex&) const diff --git a/Analysis/src/Module.cpp b/Analysis/src/Module.cpp index 76dc72d2..a330a98d 100644 --- a/Analysis/src/Module.cpp +++ b/Analysis/src/Module.cpp @@ -14,6 +14,7 @@ LUAU_FASTFLAGVARIABLE(DebugLuauFreezeArena, false) LUAU_FASTFLAGVARIABLE(DebugLuauTrackOwningArena, false) // Remove with FFlagLuauImmutableTypes LUAU_FASTINTVARIABLE(LuauTypeCloneRecursionLimit, 300) +LUAU_FASTFLAGVARIABLE(LuauCloneDeclaredGlobals, false) LUAU_FASTFLAG(LuauImmutableTypes) namespace Luau @@ -536,6 +537,12 @@ bool Module::clonePublicInterface() if (get(follow(ty))) *asMutable(ty) = AnyTypeVar{}; + if (FFlag::LuauCloneDeclaredGlobals) + { + for (auto& [name, ty] : declaredGlobals) + ty = clone(ty, interfaceTypes, seenTypes, seenTypePacks, cloneState); + } + freeze(internalTypes); freeze(interfaceTypes); diff --git a/Analysis/src/TypeInfer.cpp b/Analysis/src/TypeInfer.cpp index 3fe4c90e..41e8ce55 100644 --- a/Analysis/src/TypeInfer.cpp +++ b/Analysis/src/TypeInfer.cpp @@ -29,22 +29,24 @@ LUAU_FASTFLAGVARIABLE(LuauRecursiveTypeParameterRestriction, false) LUAU_FASTFLAGVARIABLE(LuauGenericFunctionsDontCacheTypeParams, false) LUAU_FASTFLAGVARIABLE(LuauImmutableTypes, false) LUAU_FASTFLAGVARIABLE(LuauSealExports, false) +LUAU_FASTFLAGVARIABLE(LuauSelfCallAutocompleteFix, false) LUAU_FASTFLAGVARIABLE(LuauSingletonTypes, false) LUAU_FASTFLAGVARIABLE(LuauDiscriminableUnions2, false) LUAU_FASTFLAGVARIABLE(LuauExpectedTypesOfProperties, false) LUAU_FASTFLAGVARIABLE(LuauErrorRecoveryType, false) LUAU_FASTFLAGVARIABLE(LuauOnlyMutateInstantiatedTables, false) LUAU_FASTFLAGVARIABLE(LuauPropertiesGetExpectedType, false) -LUAU_FASTFLAGVARIABLE(LuauStatFunctionSimplify, false) +LUAU_FASTFLAGVARIABLE(LuauStatFunctionSimplify2, false) LUAU_FASTFLAGVARIABLE(LuauUnsealedTableLiteral, false) LUAU_FASTFLAGVARIABLE(LuauTwoPassAliasDefinitionFix, false) LUAU_FASTFLAGVARIABLE(LuauAssertStripsFalsyTypes, false) LUAU_FASTFLAGVARIABLE(LuauReturnAnyInsteadOfICE, false) // Eventually removed as false. -LUAU_FASTFLAG(LuauWidenIfSupertypeIsFree) +LUAU_FASTFLAG(LuauWidenIfSupertypeIsFree2) LUAU_FASTFLAGVARIABLE(LuauDoNotTryToReduce, false) LUAU_FASTFLAGVARIABLE(LuauDoNotAccidentallyDependOnPointerOrdering, false) LUAU_FASTFLAGVARIABLE(LuauFixArgumentCountMismatchAmountWithGenericTypes, false) LUAU_FASTFLAGVARIABLE(LuauFixIncorrectLineNumberDuplicateType, false) +LUAU_FASTFLAG(LuauAnyInIsOptionalIsOptional) namespace Luau { @@ -1099,7 +1101,7 @@ void TypeChecker::check(const ScopePtr& scope, TypeId ty, const ScopePtr& funSco scope->bindings[name->local] = {anyIfNonstrict(quantify(funScope, ty, name->local->location)), name->local->location}; return; } - else if (auto name = function.name->as(); name && FFlag::LuauStatFunctionSimplify) + else if (auto name = function.name->as(); name && FFlag::LuauStatFunctionSimplify2) { TypeId exprTy = checkExpr(scope, *name->expr).type; TableTypeVar* ttv = getMutableTableType(exprTy); @@ -1111,7 +1113,10 @@ void TypeChecker::check(const ScopePtr& scope, TypeId ty, const ScopePtr& funSco reportError(TypeError{function.location, OnlyTablesCanHaveMethods{exprTy}}); } else if (ttv->state == TableState::Sealed) - reportError(TypeError{function.location, CannotExtendTable{exprTy, CannotExtendTable::Property, name->index.value}}); + { + if (!ttv->indexer || !isPrim(ttv->indexer->indexType, PrimitiveTypeVar::String)) + reportError(TypeError{function.location, CannotExtendTable{exprTy, CannotExtendTable::Property, name->index.value}}); + } ty = follow(ty); @@ -1134,7 +1139,7 @@ void TypeChecker::check(const ScopePtr& scope, TypeId ty, const ScopePtr& funSco if (ttv && ttv->state != TableState::Sealed) ttv->props[name->index.value] = {follow(quantify(funScope, ty, name->indexLocation)), /* deprecated */ false, {}, name->indexLocation}; } - else if (FFlag::LuauStatFunctionSimplify) + else if (FFlag::LuauStatFunctionSimplify2) { LUAU_ASSERT(function.name->is()); @@ -1144,7 +1149,7 @@ void TypeChecker::check(const ScopePtr& scope, TypeId ty, const ScopePtr& funSco } else if (function.func->self) { - LUAU_ASSERT(!FFlag::LuauStatFunctionSimplify); + LUAU_ASSERT(!FFlag::LuauStatFunctionSimplify2); AstExprIndexName* indexName = function.name->as(); if (!indexName) @@ -1183,7 +1188,7 @@ void TypeChecker::check(const ScopePtr& scope, TypeId ty, const ScopePtr& funSco } else { - LUAU_ASSERT(!FFlag::LuauStatFunctionSimplify); + LUAU_ASSERT(!FFlag::LuauStatFunctionSimplify2); TypeId leftType = checkLValueBinding(scope, *function.name); @@ -1410,6 +1415,9 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatDeclareClass& declar { ftv->argNames.insert(ftv->argNames.begin(), FunctionArgument{"self", {}}); ftv->argTypes = addTypePack(TypePack{{classTy}, ftv->argTypes}); + + if (FFlag::LuauSelfCallAutocompleteFix) + ftv->hasSelf = true; } } @@ -1883,19 +1891,27 @@ std::optional TypeChecker::tryStripUnionFromNil(TypeId ty) { if (const UnionTypeVar* utv = get(ty)) { - bool hasNil = false; - - for (TypeId option : utv) + if (FFlag::LuauAnyInIsOptionalIsOptional) { - if (isNil(option)) - { - hasNil = true; - break; - } + if (!std::any_of(begin(utv), end(utv), isNil)) + return ty; } + else + { + bool hasNil = false; - if (!hasNil) - return ty; + for (TypeId option : utv) + { + if (isNil(option)) + { + hasNil = true; + break; + } + } + + if (!hasNil) + return ty; + } std::vector result; @@ -1916,14 +1932,34 @@ std::optional TypeChecker::tryStripUnionFromNil(TypeId ty) TypeId TypeChecker::stripFromNilAndReport(TypeId ty, const Location& location) { - if (isOptional(ty)) + if (FFlag::LuauAnyInIsOptionalIsOptional) { - if (std::optional strippedUnion = tryStripUnionFromNil(follow(ty))) + ty = follow(ty); + + if (auto utv = get(ty)) + { + if (!std::any_of(begin(utv), end(utv), isNil)) + return ty; + + } + + if (std::optional strippedUnion = tryStripUnionFromNil(ty)) { reportError(location, OptionalValueAccess{ty}); return follow(*strippedUnion); } } + else + { + if (isOptional(ty)) + { + if (std::optional strippedUnion = tryStripUnionFromNil(follow(ty))) + { + reportError(location, OptionalValueAccess{ty}); + return follow(*strippedUnion); + } + } + } return ty; } @@ -2935,9 +2971,25 @@ TypeId TypeChecker::checkFunctionName(const ScopePtr& scope, AstExpr& funName, T return errorRecoveryType(scope); } - // Cannot extend sealed table, but we dont report an error here because it will be reported during AstStatFunction check - if (lhsType->persistent || ttv->state == TableState::Sealed) - return errorRecoveryType(scope); + if (FFlag::LuauStatFunctionSimplify2) + { + if (lhsType->persistent) + return errorRecoveryType(scope); + + // Cannot extend sealed table, but we dont report an error here because it will be reported during AstStatFunction check + if (ttv->state == TableState::Sealed) + { + if (ttv->indexer && isPrim(ttv->indexer->indexType, PrimitiveTypeVar::String)) + return ttv->indexer->indexResultType; + else + return errorRecoveryType(scope); + } + } + else + { + if (lhsType->persistent || ttv->state == TableState::Sealed) + return errorRecoveryType(scope); + } Name name = indexName->index.value; @@ -3393,7 +3445,7 @@ void TypeChecker::checkArgumentList( else if (state.log.getMutable(t)) { } // ok - else if (isNonstrictMode() && state.log.getMutable(t)) + else if (!FFlag::LuauAnyInIsOptionalIsOptional && isNonstrictMode() && state.log.getMutable(t)) { } // ok else @@ -3467,7 +3519,11 @@ void TypeChecker::checkArgumentList( } TypePackId varPack = addTypePack(TypePackVar{TypePack{rest, argIter.tail()}}); - state.tryUnify(varPack, tail); + if (FFlag::LuauWidenIfSupertypeIsFree2) + state.tryUnify(varPack, tail); + else + state.tryUnify(tail, varPack); + return; } else if (state.log.getMutable(tail)) @@ -3542,6 +3598,23 @@ ExprResult TypeChecker::checkExprPack(const ScopePtr& scope, const A actualFunctionType = follow(actualFunctionType); + TypePackId retPack; + if (!FFlag::LuauWidenIfSupertypeIsFree2) + { + retPack = freshTypePack(scope->level); + } + else + { + if (auto free = get(actualFunctionType)) + { + retPack = freshTypePack(free->level); + TypePackId freshArgPack = freshTypePack(free->level); + *asMutable(actualFunctionType) = FunctionTypeVar(free->level, freshArgPack, retPack); + } + else + retPack = freshTypePack(scope->level); + } + // checkExpr will log the pre-instantiated type of the function. // That's not nearly as interesting as the instantiated type, which will include details about how // generic functions are being instantiated for this particular callsite. @@ -3550,8 +3623,6 @@ ExprResult TypeChecker::checkExprPack(const ScopePtr& scope, const A std::vector overloads = flattenIntersection(actualFunctionType); - TypePackId retPack = freshTypePack(scope->level); - std::vector> expectedTypes = getExpectedTypesForCall(overloads, expr.args.size, expr.self); ExprResult argListResult = checkExprList(scope, expr.location, expr.args, false, {}, expectedTypes); @@ -3682,7 +3753,7 @@ std::optional> TypeChecker::checkCallOverload(const Scope // has been instantiated, so is a monotype. We can therefore // unify it with a monomorphic function. TypeId r = addType(FunctionTypeVar(scope->level, argPack, retPack)); - if (FFlag::LuauWidenIfSupertypeIsFree) + if (FFlag::LuauWidenIfSupertypeIsFree2) { UnifierOptions options; options.isFunctionCall = true; @@ -3772,7 +3843,7 @@ std::optional> TypeChecker::checkCallOverload(const Scope { state.log.commit(); - if (isNonstrictMode() && !expr.self && expr.func->is() && ftv->hasSelf) + if (!FFlag::LuauAnyInIsOptionalIsOptional && isNonstrictMode() && !expr.self && expr.func->is() && ftv->hasSelf) { // If we are running in nonstrict mode, passing fewer arguments than the function is declared to take AND // the function is declared with colon notation AND we use dot notation, warn. diff --git a/Analysis/src/TypeVar.cpp b/Analysis/src/TypeVar.cpp index 5af2c8a6..89549535 100644 --- a/Analysis/src/TypeVar.cpp +++ b/Analysis/src/TypeVar.cpp @@ -26,6 +26,7 @@ LUAU_FASTINT(LuauTypeInferRecursionLimit) LUAU_FASTFLAG(LuauErrorRecoveryType) LUAU_FASTFLAG(LuauSubtypingAddOptPropsToUnsealedTables) LUAU_FASTFLAG(LuauDiscriminableUnions2) +LUAU_FASTFLAGVARIABLE(LuauAnyInIsOptionalIsOptional, false) namespace Luau { @@ -201,11 +202,16 @@ bool isOptional(TypeId ty) if (isNil(ty)) return true; - auto utv = get(follow(ty)); + ty = follow(ty); + + if (FFlag::LuauAnyInIsOptionalIsOptional && get(ty)) + return true; + + auto utv = get(ty); if (!utv) return false; - return std::any_of(begin(utv), end(utv), isNil); + return std::any_of(begin(utv), end(utv), FFlag::LuauAnyInIsOptionalIsOptional ? isOptional : isNil); } bool isTableIntersection(TypeId ty) diff --git a/Analysis/src/Unifier.cpp b/Analysis/src/Unifier.cpp index 7b781f26..60a9c9a5 100644 --- a/Analysis/src/Unifier.cpp +++ b/Analysis/src/Unifier.cpp @@ -20,11 +20,13 @@ LUAU_FASTFLAGVARIABLE(LuauTableSubtypingVariance2, false); LUAU_FASTFLAG(LuauSingletonTypes) LUAU_FASTFLAG(LuauErrorRecoveryType); LUAU_FASTFLAGVARIABLE(LuauSubtypingAddOptPropsToUnsealedTables, false) -LUAU_FASTFLAGVARIABLE(LuauWidenIfSupertypeIsFree, false) +LUAU_FASTFLAGVARIABLE(LuauWidenIfSupertypeIsFree2, false) LUAU_FASTFLAGVARIABLE(LuauDifferentOrderOfUnificationDoesntMatter, false) LUAU_FASTFLAGVARIABLE(LuauTxnLogSeesTypePacks2, false) LUAU_FASTFLAGVARIABLE(LuauTxnLogCheckForInvalidation, false) +LUAU_FASTFLAGVARIABLE(LuauTxnLogRefreshFunctionPointers, false) LUAU_FASTFLAGVARIABLE(LuauTxnLogDontRetryForIndexers, false) +LUAU_FASTFLAG(LuauAnyInIsOptionalIsOptional) namespace Luau { @@ -272,12 +274,6 @@ TypePackId Widen::clean(TypePackId) bool Widen::ignoreChildren(TypeId ty) { - // Sometimes we unify ("hi") -> free1 with (free2) -> free3, so don't ignore functions. - // TODO: should we be doing this? we would need to rework how checkCallOverload does the unification. - if (log->is(ty)) - return false; - - // We only care about unions. return !log->is(ty); } @@ -990,7 +986,7 @@ void Unifier::tryUnify_(TypePackId subTp, TypePackId superTp, bool isFunctionCal if (!log.getMutable(superTp)) { - log.replace(superTp, Unifiable::Bound(subTp)); + log.replace(superTp, Unifiable::Bound(widen(subTp))); } } else if (log.getMutable(subTp)) @@ -1107,7 +1103,7 @@ void Unifier::tryUnify_(TypePackId subTp, TypePackId superTp, bool isFunctionCal } // In nonstrict mode, any also marks an optional argument. - else if (superIter.good() && isNonstrictMode() && log.getMutable(log.follow(*superIter))) + else if (!FFlag::LuauAnyInIsOptionalIsOptional && superIter.good() && isNonstrictMode() && log.getMutable(log.follow(*superIter))) { superIter.advance(); continue; @@ -1280,6 +1276,13 @@ void Unifier::tryUnifyFunctions(TypeId subTy, TypeId superTy, bool isFunctionCal tryUnify_(subFunction->retType, superFunction->retType); } + if (FFlag::LuauTxnLogRefreshFunctionPointers) + { + // Updating the log may have invalidated the function pointers + superFunction = log.getMutable(superTy); + subFunction = log.getMutable(subTy); + } + if (!FFlag::LuauImmutableTypes) { if (superFunction->definition && !subFunction->definition && !subTy->persistent) @@ -1357,10 +1360,18 @@ void Unifier::tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) { auto subIter = subTable->props.find(propName); - bool isAny = log.getMutable(log.follow(superProp.type)); + if (FFlag::LuauAnyInIsOptionalIsOptional) + { + if (subIter == subTable->props.end() && (!FFlag::LuauSubtypingAddOptPropsToUnsealedTables || subTable->state == TableState::Unsealed) && !isOptional(superProp.type)) + missingProperties.push_back(propName); + } + else + { + bool isAny = log.getMutable(log.follow(superProp.type)); - if (subIter == subTable->props.end() && (!FFlag::LuauSubtypingAddOptPropsToUnsealedTables || subTable->state == TableState::Unsealed) && !isOptional(superProp.type) && !isAny) - missingProperties.push_back(propName); + if (subIter == subTable->props.end() && (!FFlag::LuauSubtypingAddOptPropsToUnsealedTables || subTable->state == TableState::Unsealed) && !isOptional(superProp.type) && !isAny) + missingProperties.push_back(propName); + } } if (!missingProperties.empty()) @@ -1378,9 +1389,17 @@ void Unifier::tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) { auto superIter = superTable->props.find(propName); - bool isAny = log.is(log.follow(subProp.type)); - if (superIter == superTable->props.end() && (FFlag::LuauSubtypingAddOptPropsToUnsealedTables || (!isOptional(subProp.type) && !isAny))) - extraProperties.push_back(propName); + if (FFlag::LuauAnyInIsOptionalIsOptional) + { + if (superIter == superTable->props.end() && (FFlag::LuauSubtypingAddOptPropsToUnsealedTables || !isOptional(subProp.type))) + extraProperties.push_back(propName); + } + else + { + bool isAny = log.is(log.follow(subProp.type)); + if (superIter == superTable->props.end() && (FFlag::LuauSubtypingAddOptPropsToUnsealedTables || (!isOptional(subProp.type) && !isAny))) + extraProperties.push_back(propName); + } } if (!extraProperties.empty()) @@ -1424,6 +1443,12 @@ void Unifier::tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) if (innerState.errors.empty()) log.concat(std::move(innerState.log)); } + else if (FFlag::LuauAnyInIsOptionalIsOptional && (!FFlag::LuauSubtypingAddOptPropsToUnsealedTables || subTable->state == TableState::Unsealed) && isOptional(prop.type)) + // This is sound because unsealed table types are precise, so `{ p : T } <: { p : T, q : U? }` + // since if `t : { p : T }` then we are guaranteed that `t.q` is `nil`. + // TODO: if the supertype is written to, the subtype may no longer be precise (alias analysis?) + { + } else if ((!FFlag::LuauSubtypingAddOptPropsToUnsealedTables || subTable->state == TableState::Unsealed) && (isOptional(prop.type) || get(follow(prop.type)))) // This is sound because unsealed table types are precise, so `{ p : T } <: { p : T, q : U? }` // since if `t : { p : T }` then we are guaranteed that `t.q` is `nil`. @@ -1497,6 +1522,9 @@ void Unifier::tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) else if (variance == Covariant) { } + else if (FFlag::LuauAnyInIsOptionalIsOptional && !FFlag::LuauSubtypingAddOptPropsToUnsealedTables && isOptional(prop.type)) + { + } else if (!FFlag::LuauSubtypingAddOptPropsToUnsealedTables && (isOptional(prop.type) || get(follow(prop.type)))) { } @@ -1618,7 +1646,7 @@ void Unifier::tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) TypeId Unifier::widen(TypeId ty) { - if (!FFlag::LuauWidenIfSupertypeIsFree) + if (!FFlag::LuauWidenIfSupertypeIsFree2) return ty; Widen widen{types}; @@ -1627,10 +1655,21 @@ TypeId Unifier::widen(TypeId ty) return result.value_or(ty); } +TypePackId Unifier::widen(TypePackId tp) +{ + if (!FFlag::LuauWidenIfSupertypeIsFree2) + return tp; + + Widen widen{types}; + std::optional result = widen.substitute(tp); + // TODO: what does it mean for substitution to fail to widen? + return result.value_or(tp); +} + TypeId Unifier::deeplyOptional(TypeId ty, std::unordered_map seen) { ty = follow(ty); - if (get(ty)) + if (!FFlag::LuauAnyInIsOptionalIsOptional && get(ty)) return ty; else if (isOptional(ty)) return ty; @@ -1744,7 +1783,10 @@ void Unifier::tryUnifyFreeTable(TypeId subTy, TypeId superTy) { if (auto subProp = findTablePropertyRespectingMeta(subTy, freeName)) { - tryUnify_(freeProp.type, *subProp); + if (FFlag::LuauWidenIfSupertypeIsFree2) + tryUnify_(*subProp, freeProp.type); + else + tryUnify_(freeProp.type, *subProp); /* * TypeVars are commonly cyclic, so it is entirely possible diff --git a/Ast/src/Parser.cpp b/Ast/src/Parser.cpp index 1cb8f134..941a3ea4 100644 --- a/Ast/src/Parser.cpp +++ b/Ast/src/Parser.cpp @@ -11,18 +11,11 @@ LUAU_FASTINTVARIABLE(LuauRecursionLimit, 1000) LUAU_FASTINTVARIABLE(LuauParseErrorLimit, 100) LUAU_FASTFLAGVARIABLE(LuauParseSingletonTypes, false) -LUAU_FASTFLAGVARIABLE(LuauParseAllHotComments, false) LUAU_FASTFLAGVARIABLE(LuauTableFieldFunctionDebugname, false) namespace Luau { -static bool isComment(const Lexeme& lexeme) -{ - LUAU_ASSERT(!FFlag::LuauParseAllHotComments); - return lexeme.type == Lexeme::Comment || lexeme.type == Lexeme::BlockComment; -} - ParseError::ParseError(const Location& location, const std::string& message) : location(location) , message(message) @@ -146,54 +139,13 @@ ParseResult Parser::parse(const char* buffer, size_t bufferSize, AstNameTable& n { LUAU_TIMETRACE_SCOPE("Parser::parse", "Parser"); - Parser p(buffer, bufferSize, names, allocator, FFlag::LuauParseAllHotComments ? options : ParseOptions()); + Parser p(buffer, bufferSize, names, allocator, options); try { - if (FFlag::LuauParseAllHotComments) - { - AstStatBlock* root = p.parseChunk(); + AstStatBlock* root = p.parseChunk(); - return ParseResult{root, std::move(p.hotcomments), std::move(p.parseErrors), std::move(p.commentLocations)}; - } - else - { - std::vector hotcomments; - - while (isComment(p.lexer.current()) || p.lexer.current().type == Lexeme::BrokenComment) - { - const char* text = p.lexer.current().data; - unsigned int length = p.lexer.current().length; - - if (length && text[0] == '!') - { - unsigned int end = length; - while (end > 0 && isSpace(text[end - 1])) - --end; - - hotcomments.push_back({true, p.lexer.current().location, std::string(text + 1, text + end)}); - } - - const Lexeme::Type type = p.lexer.current().type; - const Location loc = p.lexer.current().location; - - if (options.captureComments) - p.commentLocations.push_back(Comment{type, loc}); - - if (type == Lexeme::BrokenComment) - break; - - p.lexer.next(); - } - - p.lexer.setSkipComments(true); - - p.options = options; - - AstStatBlock* root = p.parseChunk(); - - return ParseResult{root, hotcomments, p.parseErrors, std::move(p.commentLocations)}; - } + return ParseResult{root, std::move(p.hotcomments), std::move(p.parseErrors), std::move(p.commentLocations)}; } catch (ParseError& err) { @@ -225,10 +177,11 @@ Parser::Parser(const char* buffer, size_t bufferSize, AstNameTable& names, Alloc matchRecoveryStopOnToken.assign(Lexeme::Type::Reserved_END, 0); matchRecoveryStopOnToken[Lexeme::Type::Eof] = 1; - if (FFlag::LuauParseAllHotComments) - lexer.setSkipComments(true); + // required for lookahead() to work across a comment boundary and for nextLexeme() to work when captureComments is false + lexer.setSkipComments(true); - // read first lexeme + // read first lexeme (any hot comments get .header = true) + LUAU_ASSERT(hotcommentHeader); nextLexeme(); // all hot comments parsed after the first non-comment lexeme are special in that they don't affect type checking / linting mode @@ -2831,49 +2784,31 @@ void Parser::nextLexeme() { if (options.captureComments) { - if (FFlag::LuauParseAllHotComments) + Lexeme::Type type = lexer.next(/* skipComments= */ false).type; + + while (type == Lexeme::BrokenComment || type == Lexeme::Comment || type == Lexeme::BlockComment) { - Lexeme::Type type = lexer.next(/* skipComments= */ false).type; + const Lexeme& lexeme = lexer.current(); + commentLocations.push_back(Comment{lexeme.type, lexeme.location}); - while (type == Lexeme::BrokenComment || type == Lexeme::Comment || type == Lexeme::BlockComment) + // Subtlety: Broken comments are weird because we record them as comments AND pass them to the parser as a lexeme. + // The parser will turn this into a proper syntax error. + if (lexeme.type == Lexeme::BrokenComment) + return; + + // Comments starting with ! are called "hot comments" and contain directives for type checking / linting + if (lexeme.type == Lexeme::Comment && lexeme.length && lexeme.data[0] == '!') { - const Lexeme& lexeme = lexer.current(); - commentLocations.push_back(Comment{lexeme.type, lexeme.location}); + const char* text = lexeme.data; - // Subtlety: Broken comments are weird because we record them as comments AND pass them to the parser as a lexeme. - // The parser will turn this into a proper syntax error. - if (lexeme.type == Lexeme::BrokenComment) - return; + unsigned int end = lexeme.length; + while (end > 0 && isSpace(text[end - 1])) + --end; - // Comments starting with ! are called "hot comments" and contain directives for type checking / linting - if (lexeme.type == Lexeme::Comment && lexeme.length && lexeme.data[0] == '!') - { - const char* text = lexeme.data; - - unsigned int end = lexeme.length; - while (end > 0 && isSpace(text[end - 1])) - --end; - - hotcomments.push_back({hotcommentHeader, lexeme.location, std::string(text + 1, text + end)}); - } - - type = lexer.next(/* skipComments= */ false).type; - } - } - else - { - while (true) - { - const Lexeme& lexeme = lexer.next(/*skipComments*/ false); - // Subtlety: Broken comments are weird because we record them as comments AND pass them to the parser as a lexeme. - // The parser will turn this into a proper syntax error. - if (lexeme.type == Lexeme::BrokenComment) - commentLocations.push_back(Comment{lexeme.type, lexeme.location}); - if (isComment(lexeme)) - commentLocations.push_back(Comment{lexeme.type, lexeme.location}); - else - return; + hotcomments.push_back({hotcommentHeader, lexeme.location, std::string(text + 1, text + end)}); } + + type = lexer.next(/* skipComments= */ false).type; } } else diff --git a/Compiler/src/Builtins.cpp b/Compiler/src/Builtins.cpp index 26360c49..ff753112 100644 --- a/Compiler/src/Builtins.cpp +++ b/Compiler/src/Builtins.cpp @@ -4,8 +4,6 @@ #include "Luau/Bytecode.h" #include "Luau/Compiler.h" -LUAU_FASTFLAGVARIABLE(LuauCompileSelectBuiltin2, false) - namespace Luau { namespace Compile @@ -64,7 +62,7 @@ int getBuiltinFunctionId(const Builtin& builtin, const CompileOptions& options) if (builtin.isGlobal("unpack")) return LBF_TABLE_UNPACK; - if (FFlag::LuauCompileSelectBuiltin2 && builtin.isGlobal("select")) + if (builtin.isGlobal("select")) return LBF_SELECT_VARARG; if (builtin.object == "math") diff --git a/Compiler/src/Compiler.cpp b/Compiler/src/Compiler.cpp index 656a9926..6330bf1f 100644 --- a/Compiler/src/Compiler.cpp +++ b/Compiler/src/Compiler.cpp @@ -15,8 +15,6 @@ #include #include -LUAU_FASTFLAG(LuauCompileSelectBuiltin2) - namespace Luau { @@ -265,7 +263,6 @@ struct Compiler void compileExprSelectVararg(AstExprCall* expr, uint8_t target, uint8_t targetCount, bool targetTop, bool multRet, uint8_t regs) { - LUAU_ASSERT(FFlag::LuauCompileSelectBuiltin2); LUAU_ASSERT(targetCount == 1); LUAU_ASSERT(!expr->self); LUAU_ASSERT(expr->args.size == 2 && expr->args.data[1]->is()); @@ -407,7 +404,6 @@ struct Compiler if (bfid == LBF_SELECT_VARARG) { - LUAU_ASSERT(FFlag::LuauCompileSelectBuiltin2); // Optimization: compile select(_, ...) as FASTCALL1; the builtin will read variadic arguments directly // note: for now we restrict this to single-return expressions since our runtime code doesn't deal with general cases if (multRet == false && targetCount == 1 && expr->args.size == 2 && expr->args.data[1]->is()) diff --git a/Sources.cmake b/Sources.cmake index 615641eb..59b38497 100644 --- a/Sources.cmake +++ b/Sources.cmake @@ -232,11 +232,18 @@ if(TARGET Luau.UnitTest) tests/Transpiler.test.cpp tests/TypeInfer.aliases.test.cpp tests/TypeInfer.annotations.test.cpp + tests/TypeInfer.anyerror.test.cpp tests/TypeInfer.builtins.test.cpp tests/TypeInfer.classes.test.cpp tests/TypeInfer.definitions.test.cpp + tests/TypeInfer.functions.test.cpp tests/TypeInfer.generics.test.cpp tests/TypeInfer.intersectionTypes.test.cpp + tests/TypeInfer.loops.test.cpp + tests/TypeInfer.modules.test.cpp + tests/TypeInfer.oop.test.cpp + tests/TypeInfer.operators.test.cpp + tests/TypeInfer.primitives.test.cpp tests/TypeInfer.provisional.test.cpp tests/TypeInfer.refinements.test.cpp tests/TypeInfer.singletons.test.cpp diff --git a/VM/include/lua.h b/VM/include/lua.h index 274c4ed9..d08b73ea 100644 --- a/VM/include/lua.h +++ b/VM/include/lua.h @@ -172,9 +172,12 @@ LUA_API const char* lua_pushvfstring(lua_State* L, const char* fmt, va_list argp LUA_API LUA_PRINTF_ATTR(2, 3) const char* lua_pushfstringL(lua_State* L, const char* fmt, ...); LUA_API void lua_pushcclosurek(lua_State* L, lua_CFunction fn, const char* debugname, int nup, lua_Continuation cont); LUA_API void lua_pushboolean(lua_State* L, int b); -LUA_API void lua_pushlightuserdata(lua_State* L, void* p); LUA_API int lua_pushthread(lua_State* L); +LUA_API void lua_pushlightuserdata(lua_State* L, void* p); +LUA_API void* lua_newuserdatatagged(lua_State* L, size_t sz, int tag); +LUA_API void* lua_newuserdatadtor(lua_State* L, size_t sz, void (*dtor)(void*)); + /* ** get functions (Lua -> stack) */ @@ -189,8 +192,6 @@ LUA_API void lua_setreadonly(lua_State* L, int idx, int enabled); LUA_API int lua_getreadonly(lua_State* L, int idx); LUA_API void lua_setsafeenv(lua_State* L, int idx, int enabled); -LUA_API void* lua_newuserdatatagged(lua_State* L, size_t sz, int tag); -LUA_API void* lua_newuserdatadtor(lua_State* L, size_t sz, void (*dtor)(void*)); LUA_API int lua_getmetatable(lua_State* L, int objindex); LUA_API void lua_getfenv(lua_State* L, int idx); @@ -276,6 +277,14 @@ enum lua_GCOp LUA_API int lua_gc(lua_State* L, int what, int data); +/* +** memory statistics +** all allocated bytes are attributed to the memory category of the running thread (0..LUA_MEMORY_CATEGORIES-1) +*/ + +LUA_API void lua_setmemcat(lua_State* L, int category); +LUA_API size_t lua_totalbytes(lua_State* L, int category); + /* ** miscellaneous functions */ diff --git a/VM/src/lapi.cpp b/VM/src/lapi.cpp index f7f15442..3c087314 100644 --- a/VM/src/lapi.cpp +++ b/VM/src/lapi.cpp @@ -35,8 +35,8 @@ const char* luau_ident = "$Luau: Copyright (C) 2019-2022 Roblox Corporation $\n" static Table* getcurrenv(lua_State* L) { - if (L->ci == L->base_ci) /* no enclosing function? */ - return L->gt; /* use global table as environment */ + if (L->ci == L->base_ci) /* no enclosing function? */ + return L->gt; /* use global table as environment */ else return curr_func(L)->env; } @@ -1188,7 +1188,7 @@ void lua_concat(lua_State* L, int n) void* lua_newuserdatatagged(lua_State* L, size_t sz, int tag) { - api_check(L, unsigned(tag) < LUA_UTAG_LIMIT); + api_check(L, unsigned(tag) < LUA_UTAG_LIMIT || tag == UTAG_PROXY); luaC_checkGC(L); luaC_checkthreadsleep(L); Udata* u = luaU_newudata(L, sz, tag); @@ -1317,7 +1317,7 @@ void lua_setuserdatadtor(lua_State* L, int tag, void (*dtor)(void*)) L->global->udatagc[tag] = dtor; } -LUA_API void lua_clonefunction(lua_State* L, int idx) +void lua_clonefunction(lua_State* L, int idx) { StkId p = index2addr(L, idx); api_check(L, isLfunction(p)); @@ -1333,3 +1333,15 @@ lua_Callbacks* lua_callbacks(lua_State* L) { return &L->global->cb; } + +void lua_setmemcat(lua_State* L, int category) +{ + api_check(L, unsigned(category) < LUA_MEMORY_CATEGORIES); + L->activememcat = uint8_t(category); +} + +size_t lua_totalbytes(lua_State* L, int category) +{ + api_check(L, category < LUA_MEMORY_CATEGORIES); + return category < 0 ? L->global->totalbytes : L->global->memcatbytes[category]; +} diff --git a/VM/src/lbaselib.cpp b/VM/src/lbaselib.cpp index 2307598e..96ad493b 100644 --- a/VM/src/lbaselib.cpp +++ b/VM/src/lbaselib.cpp @@ -5,6 +5,7 @@ #include "lstate.h" #include "lapi.h" #include "ldo.h" +#include "ludata.h" #include #include @@ -190,6 +191,7 @@ static int luaB_type(lua_State* L) luaL_checkany(L, 1); if (DFFlag::LuauMorePreciseLuaLTypeName) { + /* resulting name doesn't differentiate between userdata types */ lua_pushstring(L, lua_typename(L, lua_type(L, 1))); } else @@ -202,8 +204,16 @@ static int luaB_type(lua_State* L) static int luaB_typeof(lua_State* L) { luaL_checkany(L, 1); - const TValue* obj = luaA_toobject(L, 1); - lua_pushstring(L, luaT_objtypename(L, obj)); + if (DFFlag::LuauMorePreciseLuaLTypeName) + { + /* resulting name returns __type if specified unless the input is a newproxy-created userdata */ + lua_pushstring(L, luaL_typename(L, 1)); + } + else + { + const TValue* obj = luaA_toobject(L, 1); + lua_pushstring(L, luaT_objtypename(L, obj)); + } return 1; } @@ -412,7 +422,7 @@ static int luaB_newproxy(lua_State* L) bool needsmt = lua_toboolean(L, 1); - lua_newuserdata(L, 0); + lua_newuserdatatagged(L, 0, UTAG_PROXY); if (needsmt) { diff --git a/VM/src/lgc.h b/VM/src/lgc.h index ad8ee78a..ebf999b5 100644 --- a/VM/src/lgc.h +++ b/VM/src/lgc.h @@ -9,9 +9,9 @@ /* ** Default settings for GC tunables (settable via lua_gc) */ -#define LUAI_GCGOAL 200 /* 200% (allow heap to double compared to live heap size) */ +#define LUAI_GCGOAL 200 /* 200% (allow heap to double compared to live heap size) */ #define LUAI_GCSTEPMUL 200 /* GC runs 'twice the speed' of memory allocation */ -#define LUAI_GCSTEPSIZE 1 /* GC runs every KB of memory allocation */ +#define LUAI_GCSTEPSIZE 1 /* GC runs every KB of memory allocation */ /* ** Possible states of the Garbage Collector diff --git a/VM/src/lmem.cpp b/VM/src/lmem.cpp index 899cb0c0..3cbdafff 100644 --- a/VM/src/lmem.cpp +++ b/VM/src/lmem.cpp @@ -98,7 +98,7 @@ */ #if defined(__APPLE__) #define ABISWITCH(x64, ms32, gcc32) (sizeof(void*) == 8 ? x64 : gcc32) -#elif defined(__i386__) +#elif defined(__i386__) && !defined(_MSC_VER) #define ABISWITCH(x64, ms32, gcc32) (gcc32) #else // Android somehow uses a similar ABI to MSVC, *not* to iOS... diff --git a/VM/src/lstring.cpp b/VM/src/lstring.cpp index 87250146..c0cd3e26 100644 --- a/VM/src/lstring.cpp +++ b/VM/src/lstring.cpp @@ -53,7 +53,7 @@ void luaS_resize(lua_State* L, int newsize) { TString* p = tb->hash[i]; while (p) - { /* for each node in the list */ + { /* for each node in the list */ TString* next = p->next; /* save next */ unsigned int h = p->hash; int h1 = lmod(h, newsize); /* new position */ diff --git a/VM/src/ltable.cpp b/VM/src/ltable.cpp index ef0b4b93..2deec2b9 100644 --- a/VM/src/ltable.cpp +++ b/VM/src/ltable.cpp @@ -24,6 +24,8 @@ #include +LUAU_FASTFLAGVARIABLE(LuauTableRehashRework, false) + // max size of both array and hash part is 2^MAXBITS #define MAXBITS 26 #define MAXSIZE (1 << MAXBITS) @@ -351,6 +353,22 @@ static void setnodevector(lua_State* L, Table* t, int size) t->lastfree = size; /* all positions are free */ } +static TValue* newkey(lua_State* L, Table* t, const TValue* key); + +static TValue* arrayornewkey(lua_State* L, Table* t, const TValue* key) +{ + if (ttisnumber(key)) + { + int k; + double n = nvalue(key); + luai_num2int(k, n); + if (luai_numeq(cast_num(k), n) && cast_to(unsigned int, k - 1) < cast_to(unsigned int, t->sizearray)) + return &t->array[k - 1]; + } + + return newkey(L, t, key); +} + static void resize(lua_State* L, Table* t, int nasize, int nhsize) { if (nasize > MAXSIZE || nhsize > MAXSIZE) @@ -369,22 +387,50 @@ static void resize(lua_State* L, Table* t, int nasize, int nhsize) for (int i = nasize; i < oldasize; i++) { if (!ttisnil(&t->array[i])) - setobjt2t(L, luaH_setnum(L, t, i + 1), &t->array[i]); + { + if (FFlag::LuauTableRehashRework) + { + TValue ok; + setnvalue(&ok, cast_num(i + 1)); + setobjt2t(L, newkey(L, t, &ok), &t->array[i]); + } + else + { + setobjt2t(L, luaH_setnum(L, t, i + 1), &t->array[i]); + } + } } /* shrink array */ luaM_reallocarray(L, t->array, oldasize, nasize, TValue, t->memcat); } /* re-insert elements from hash part */ - for (int i = twoto(oldhsize) - 1; i >= 0; i--) + if (FFlag::LuauTableRehashRework) { - LuaNode* old = nold + i; - if (!ttisnil(gval(old))) + for (int i = twoto(oldhsize) - 1; i >= 0; i--) { - TValue ok; - getnodekey(L, &ok, old); - setobjt2t(L, luaH_set(L, t, &ok), gval(old)); + LuaNode* old = nold + i; + if (!ttisnil(gval(old))) + { + TValue ok; + getnodekey(L, &ok, old); + setobjt2t(L, arrayornewkey(L, t, &ok), gval(old)); + } } } + else + { + for (int i = twoto(oldhsize) - 1; i >= 0; i--) + { + LuaNode* old = nold + i; + if (!ttisnil(gval(old))) + { + TValue ok; + getnodekey(L, &ok, old); + setobjt2t(L, luaH_set(L, t, &ok), gval(old)); + } + } + } + if (nold != dummynode) luaM_freearray(L, nold, twoto(oldhsize), LuaNode, t->memcat); /* free old array */ } @@ -482,7 +528,16 @@ static TValue* newkey(lua_State* L, Table* t, const TValue* key) if (n == NULL) { /* cannot find a free place? */ rehash(L, t, key); /* grow table */ - return luaH_set(L, t, key); /* re-insert key into grown table */ + + if (!FFlag::LuauTableRehashRework) + { + return luaH_set(L, t, key); /* re-insert key into grown table */ + } + else + { + // after rehash, numeric keys might be located in the new array part, but won't be found in the node part + return arrayornewkey(L, t, key); + } } LUAU_ASSERT(n != dummynode); TValue mk; diff --git a/VM/src/ltm.cpp b/VM/src/ltm.cpp index a77a7c72..106efb2b 100644 --- a/VM/src/ltm.cpp +++ b/VM/src/ltm.cpp @@ -4,6 +4,7 @@ #include "lstate.h" #include "lstring.h" +#include "ludata.h" #include "ltable.h" #include "lgc.h" @@ -116,7 +117,7 @@ const TValue* luaT_gettmbyobj(lua_State* L, const TValue* o, TMS event) const TString* luaT_objtypenamestr(lua_State* L, const TValue* o) { - if (ttisuserdata(o) && uvalue(o)->tag && uvalue(o)->metatable) + if (ttisuserdata(o) && uvalue(o)->tag != UTAG_PROXY && uvalue(o)->metatable) { const TValue* type = luaH_getstr(uvalue(o)->metatable, L->global->tmname[TM_TYPE]); diff --git a/VM/src/ludata.cpp b/VM/src/ludata.cpp index 0dfac508..819d1863 100644 --- a/VM/src/ludata.cpp +++ b/VM/src/ludata.cpp @@ -22,13 +22,11 @@ Udata* luaU_newudata(lua_State* L, size_t s, int tag) void luaU_freeudata(lua_State* L, Udata* u, lua_Page* page) { - LUAU_ASSERT(u->tag < LUA_UTAG_LIMIT || u->tag == UTAG_IDTOR); - void (*dtor)(void*) = nullptr; - if (u->tag == UTAG_IDTOR) - memcpy(&dtor, &u->data + u->len - sizeof(dtor), sizeof(dtor)); - else if (u->tag) + if (u->tag < LUA_UTAG_LIMIT) dtor = L->global->udatagc[u->tag]; + else if (u->tag == UTAG_IDTOR) + memcpy(&dtor, &u->data + u->len - sizeof(dtor), sizeof(dtor)); if (dtor) dtor(u->data); diff --git a/VM/src/ludata.h b/VM/src/ludata.h index ec374c28..f24e4a32 100644 --- a/VM/src/ludata.h +++ b/VM/src/ludata.h @@ -7,6 +7,9 @@ /* special tag value is used for user data with inline dtors */ #define UTAG_IDTOR LUA_UTAG_LIMIT +/* special tag value is used for newproxy-created user data (all other user data objects are host-exposed) */ +#define UTAG_PROXY (LUA_UTAG_LIMIT + 1) + #define sizeudata(len) (offsetof(Udata, data) + len) LUAI_FUNC Udata* luaU_newudata(lua_State* L, size_t s, int tag); diff --git a/VM/src/lvmexecute.cpp b/VM/src/lvmexecute.cpp index 6c31d36f..96a87b7e 100644 --- a/VM/src/lvmexecute.cpp +++ b/VM/src/lvmexecute.cpp @@ -77,6 +77,11 @@ if (LUAU_UNLIKELY(!!interrupt)) \ { /* the interrupt hook is called right before we advance pc */ \ VM_PROTECT(L->ci->savedpc++; interrupt(L, -1)); \ + if (L->status != 0) \ + { \ + L->ci->savedpc--; \ + goto exit; \ + } \ } \ } #endif diff --git a/tests/Autocomplete.test.cpp b/tests/Autocomplete.test.cpp index 55b0618a..17fd6b13 100644 --- a/tests/Autocomplete.test.cpp +++ b/tests/Autocomplete.test.cpp @@ -2755,4 +2755,166 @@ local abc = b@1 CHECK(ac.entryMap["bar"].parens == ParenthesesRecommendation::CursorInside); } +TEST_CASE_FIXTURE(ACFixture, "no_incompatible_self_calls_on_class") +{ + ScopedFastFlag selfCallAutocompleteFix{"LuauSelfCallAutocompleteFix", true}; + + loadDefinition(R"( +declare class Foo + function one(self): number + two: () -> number +end + )"); + + { + check(R"( +local t: Foo +t:@1 + )"); + + auto ac = autocomplete('1'); + + REQUIRE(ac.entryMap.count("one")); + REQUIRE(ac.entryMap.count("two")); + CHECK(!ac.entryMap["one"].wrongIndexType); + CHECK(ac.entryMap["two"].wrongIndexType); + } + + { + check(R"( +local t: Foo +t.@1 + )"); + + auto ac = autocomplete('1'); + + REQUIRE(ac.entryMap.count("one")); + REQUIRE(ac.entryMap.count("two")); + CHECK(ac.entryMap["one"].wrongIndexType); + CHECK(!ac.entryMap["two"].wrongIndexType); + } +} + +TEST_CASE_FIXTURE(ACFixture, "no_incompatible_self_calls") +{ + ScopedFastFlag selfCallAutocompleteFix{"LuauSelfCallAutocompleteFix", true}; + + check(R"( +local t = {} +function t.m() end +t:@1 + )"); + + auto ac = autocomplete('1'); + + REQUIRE(ac.entryMap.count("m")); + CHECK(ac.entryMap["m"].wrongIndexType); +} + +TEST_CASE_FIXTURE(ACFixture, "no_incompatible_self_calls_2") +{ + ScopedFastFlag selfCallAutocompleteFix{"LuauSelfCallAutocompleteFix", true}; + + check(R"( +local f: (() -> number) & ((number) -> number) = function(x: number?) return 2 end +local t = {} +t.f = f +t:@1 + )"); + + auto ac = autocomplete('1'); + + REQUIRE(ac.entryMap.count("f")); + CHECK(ac.entryMap["f"].wrongIndexType); +} + +TEST_CASE_FIXTURE(ACFixture, "no_incompatible_self_calls_provisional") +{ + check(R"( +local t = {} +function t.m(x: typeof(t)) end +t:@1 + )"); + + auto ac = autocomplete('1'); + + REQUIRE(ac.entryMap.count("m")); + // We can make changes to mark this as a wrong way to call even though it's compatible + CHECK(!ac.entryMap["m"].wrongIndexType); +} + +TEST_CASE_FIXTURE(ACFixture, "string_prim_self_calls_are_fine") +{ + ScopedFastFlag selfCallAutocompleteFix{"LuauSelfCallAutocompleteFix", true}; + + check(R"( +local s = "hello" +s:@1 + )"); + + auto ac = autocomplete('1'); + + REQUIRE(ac.entryMap.count("byte")); + CHECK(ac.entryMap["byte"].wrongIndexType == false); + REQUIRE(ac.entryMap.count("char")); + CHECK(ac.entryMap["char"].wrongIndexType == true); + REQUIRE(ac.entryMap.count("sub")); + CHECK(ac.entryMap["sub"].wrongIndexType == false); +} + +TEST_CASE_FIXTURE(ACFixture, "string_prim_non_self_calls_are_avoided") +{ + ScopedFastFlag selfCallAutocompleteFix{"LuauSelfCallAutocompleteFix", true}; + + check(R"( +local s = "hello" +s.@1 + )"); + + auto ac = autocomplete('1'); + + REQUIRE(ac.entryMap.count("byte")); + CHECK(ac.entryMap["byte"].wrongIndexType == true); + REQUIRE(ac.entryMap.count("char")); + CHECK(ac.entryMap["char"].wrongIndexType == false); + REQUIRE(ac.entryMap.count("sub")); + CHECK(ac.entryMap["sub"].wrongIndexType == true); +} + +TEST_CASE_FIXTURE(ACFixture, "string_library_non_self_calls_are_fine") +{ + ScopedFastFlag selfCallAutocompleteFix{"LuauSelfCallAutocompleteFix", true}; + + check(R"( +string.@1 + )"); + + auto ac = autocomplete('1'); + + REQUIRE(ac.entryMap.count("byte")); + CHECK(ac.entryMap["byte"].wrongIndexType == false); + REQUIRE(ac.entryMap.count("char")); + CHECK(ac.entryMap["char"].wrongIndexType == false); + REQUIRE(ac.entryMap.count("sub")); + CHECK(ac.entryMap["sub"].wrongIndexType == false); +} + +TEST_CASE_FIXTURE(ACFixture, "string_library_self_calls_are_invalid") +{ + ScopedFastFlag selfCallAutocompleteFix{"LuauSelfCallAutocompleteFix", true}; + + check(R"( +string:@1 + )"); + + auto ac = autocomplete('1'); + + REQUIRE(ac.entryMap.count("byte")); + CHECK(ac.entryMap["byte"].wrongIndexType == true); + REQUIRE(ac.entryMap.count("char")); + CHECK(ac.entryMap["char"].wrongIndexType == true); + REQUIRE(ac.entryMap.count("sub")); + CHECK(ac.entryMap["sub"].wrongIndexType == true); +} + TEST_SUITE_END(); diff --git a/tests/Compiler.test.cpp b/tests/Compiler.test.cpp index f982c86f..3dc57da0 100644 --- a/tests/Compiler.test.cpp +++ b/tests/Compiler.test.cpp @@ -2819,8 +2819,6 @@ RETURN R1 -1 TEST_CASE("FastcallSelect") { - ScopedFastFlag sff("LuauCompileSelectBuiltin2", true); - // select(_, ...) compiles to a builtin call CHECK_EQ("\n" + compileFunction0("return (select('#', ...))"), R"( LOADK R1 K0 diff --git a/tests/Conformance.test.cpp b/tests/Conformance.test.cpp index 63fbb363..9e4cb4a5 100644 --- a/tests/Conformance.test.cpp +++ b/tests/Conformance.test.cpp @@ -569,6 +569,11 @@ TEST_CASE("Debugger") CHECK(lua_tointeger(L, -1) == 50); lua_pop(L, 1); + int v = lua_getargument(L, 0, 2); + REQUIRE(v); + CHECK(lua_tointeger(L, -1) == 42); + lua_pop(L, 1); + // test lua_getlocal const char* l = lua_getlocal(L, 0, 1); REQUIRE(l); @@ -652,31 +657,6 @@ TEST_CASE("SameHash") CHECK(luaS_hash(buf + 1, 120) == luaS_hash(buf + 2, 120)); } -TEST_CASE("InlineDtor") -{ - static int dtorhits = 0; - - dtorhits = 0; - - { - StateRef globalState(luaL_newstate(), lua_close); - lua_State* L = globalState.get(); - - void* u1 = lua_newuserdatadtor(L, 4, [](void* data) { - dtorhits += *(int*)data; - }); - - void* u2 = lua_newuserdatadtor(L, 1, [](void* data) { - dtorhits += *(char*)data; - }); - - *(int*)u1 = 39; - *(char*)u2 = 3; - } - - CHECK(dtorhits == 42); -} - TEST_CASE("Reference") { static int dtorhits = 0; @@ -969,7 +949,7 @@ TEST_CASE("StringConversion") TEST_CASE("GCDump") { // internal function, declared in lgc.h - not exposed via lua.h - extern void luaC_dump(lua_State* L, void* file, const char* (*categoryName)(lua_State* L, uint8_t memcat)); + extern void luaC_dump(lua_State * L, void* file, const char* (*categoryName)(lua_State * L, uint8_t memcat)); StateRef globalState(luaL_newstate(), lua_close); lua_State* L = globalState.get(); @@ -1015,4 +995,114 @@ TEST_CASE("GCDump") fclose(f); } +TEST_CASE("Interrupt") +{ + static const int expectedhits[] = { + 2, + 9, + 5, + 5, + 5, + 5, + 5, + 5, + 5, + 5, + 5, + 5, + 5, + 6, + 11, + }; + static int index; + + index = 0; + + runConformance( + "interrupt.lua", + [](lua_State* L) { + auto* cb = lua_callbacks(L); + + // note: for simplicity here we setup the interrupt callback once + // however, this carries a noticeable performance cost. in a real application, + // it's advised to set interrupt callback on a timer from a different thread, + // and set it back to nullptr once the interrupt triggered. + cb->interrupt = [](lua_State* L, int gc) { + if (gc >= 0) + return; + + CHECK(index < int(std::size(expectedhits))); + + lua_Debug ar = {}; + lua_getinfo(L, 0, "l", &ar); + + CHECK(ar.currentline == expectedhits[index]); + + index++; + + // check that we can yield inside an interrupt + if (index == 5) + lua_yield(L, 0); + }; + }, + [](lua_State* L) { + CHECK(index == 5); // a single yield point + }); + + CHECK(index == int(std::size(expectedhits))); +} + +TEST_CASE("UserdataApi") +{ + static int dtorhits = 0; + + dtorhits = 0; + + StateRef globalState(luaL_newstate(), lua_close); + lua_State* L = globalState.get(); + + // setup dtor for tag 42 (created later) + lua_setuserdatadtor(L, 42, [](void* data) { + dtorhits += *(int*)data; + }); + + // light user data + int lud; + lua_pushlightuserdata(L, &lud); + + CHECK(lua_touserdata(L, -1) == &lud); + CHECK(lua_topointer(L, -1) == &lud); + + // regular user data + int* ud1 = (int*)lua_newuserdata(L, 4); + *ud1 = 42; + + CHECK(lua_touserdata(L, -1) == ud1); + CHECK(lua_topointer(L, -1) == ud1); + + // tagged user data + int* ud2 = (int*)lua_newuserdatatagged(L, 4, 42); + *ud2 = -4; + + CHECK(lua_touserdatatagged(L, -1, 42) == ud2); + CHECK(lua_touserdatatagged(L, -1, 41) == nullptr); + CHECK(lua_userdatatag(L, -1) == 42); + + // user data with inline dtor + void* ud3 = lua_newuserdatadtor(L, 4, [](void* data) { + dtorhits += *(int*)data; + }); + + void* ud4 = lua_newuserdatadtor(L, 1, [](void* data) { + dtorhits += *(char*)data; + }); + + *(int*)ud3 = 43; + *(char*)ud4 = 3; + + globalState.reset(); + + CHECK(dtorhits == 42); +} + TEST_SUITE_END(); diff --git a/tests/Linter.test.cpp b/tests/Linter.test.cpp index 4d6c207c..91b23197 100644 --- a/tests/Linter.test.cpp +++ b/tests/Linter.test.cpp @@ -1667,8 +1667,6 @@ _ = (math.random() < 0.5 and false) or 42 -- currently ignored TEST_CASE_FIXTURE(Fixture, "WrongComment") { - ScopedFastFlag sff("LuauParseAllHotComments", true); - LintResult result = lint(R"( --!strict --!struct diff --git a/tests/Module.test.cpp b/tests/Module.test.cpp index e3993cc5..82b7a350 100644 --- a/tests/Module.test.cpp +++ b/tests/Module.test.cpp @@ -248,10 +248,12 @@ TEST_CASE_FIXTURE(Fixture, "clone_seal_free_tables") TEST_CASE_FIXTURE(Fixture, "clone_self_property") { + ScopedFastFlag sff{"LuauAnyInIsOptionalIsOptional", true}; + fileResolver.source["Module/A"] = R"( --!nonstrict local a = {} - function a:foo(x) + function a:foo(x: number) return -x; end return a; @@ -267,10 +269,10 @@ TEST_CASE_FIXTURE(Fixture, "clone_self_property") )"; result = frontend.check("Module/B"); - LUAU_REQUIRE_ERRORS(result); - CHECK_EQ(toString(result.errors[0]), "This function was declared to accept self, but you did not pass enough arguments. Use a colon instead of a " - "dot or pass 1 extra nil to suppress this warning"); + LUAU_REQUIRE_ERROR_COUNT(1, result); + + CHECK_EQ("This function must be called with self. Did you mean to use a colon instead of a dot?", toString(result.errors[0])); } TEST_CASE_FIXTURE(Fixture, "clone_recursion_limit") diff --git a/tests/NonstrictMode.test.cpp b/tests/NonstrictMode.test.cpp index 5bad9901..d3faea2a 100644 --- a/tests/NonstrictMode.test.cpp +++ b/tests/NonstrictMode.test.cpp @@ -126,6 +126,8 @@ TEST_CASE_FIXTURE(Fixture, "parameters_having_type_any_are_optional") TEST_CASE_FIXTURE(Fixture, "local_tables_are_not_any") { + ScopedFastFlag sff{"LuauAnyInIsOptionalIsOptional", true}; + CheckResult result = check(R"( --!nonstrict local T = {} @@ -136,31 +138,25 @@ TEST_CASE_FIXTURE(Fixture, "local_tables_are_not_any") T:staticmethod() )"); - LUAU_REQUIRE_ERROR_COUNT(2, result); + LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK(std::any_of(result.errors.begin(), result.errors.end(), [](const TypeError& e) { - return get(e); - })); - CHECK(std::any_of(result.errors.begin(), result.errors.end(), [](const TypeError& e) { - return get(e); - })); + CHECK_EQ("This function does not take self. Did you mean to use a dot instead of a colon?", toString(result.errors[0])); } TEST_CASE_FIXTURE(Fixture, "offer_a_hint_if_you_use_a_dot_instead_of_a_colon") { + ScopedFastFlag sff{"LuauAnyInIsOptionalIsOptional", true}; + CheckResult result = check(R"( --!nonstrict local T = {} - function T:method() end - T.method() + function T:method(x: number) end + T.method(5) )"); LUAU_REQUIRE_ERROR_COUNT(1, result); - auto e = get(result.errors[0]); - REQUIRE(e != nullptr); - - REQUIRE_EQ(1, e->requiredExtraNils); + CHECK_EQ("This function must be called with self. Did you mean to use a colon instead of a dot?", toString(result.errors[0])); } TEST_CASE_FIXTURE(Fixture, "table_props_are_any") diff --git a/tests/TypeInfer.aliases.test.cpp b/tests/TypeInfer.aliases.test.cpp index 711c0aa1..b2e76052 100644 --- a/tests/TypeInfer.aliases.test.cpp +++ b/tests/TypeInfer.aliases.test.cpp @@ -676,4 +676,25 @@ TEST_CASE_FIXTURE(Fixture, "forward_declared_alias_is_not_clobbered_by_prior_uni LUAU_REQUIRE_NO_ERRORS(result); } +TEST_CASE_FIXTURE(Fixture, "recursive_types_restriction_ok") +{ + CheckResult result = check(R"( + type Tree = { data: T, children: {Tree} } + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "recursive_types_restriction_not_ok") +{ + ScopedFastFlag sff{"LuauRecursiveTypeParameterRestriction", true}; + + CheckResult result = check(R"( + -- this would be an infinite type if we allowed it + type Tree = { data: T, children: {Tree<{T}>} } + )"); + + LUAU_REQUIRE_ERRORS(result); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.anyerror.test.cpp b/tests/TypeInfer.anyerror.test.cpp new file mode 100644 index 00000000..5224b5d8 --- /dev/null +++ b/tests/TypeInfer.anyerror.test.cpp @@ -0,0 +1,335 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details + +#include "Luau/AstQuery.h" +#include "Luau/BuiltinDefinitions.h" +#include "Luau/Scope.h" +#include "Luau/TypeInfer.h" +#include "Luau/TypeVar.h" +#include "Luau/VisitTypeVar.h" + +#include "Fixture.h" + +#include "doctest.h" + +using namespace Luau; + +TEST_SUITE_BEGIN("TypeInferAnyError"); + +TEST_CASE_FIXTURE(Fixture, "for_in_loop_iterator_returns_any") +{ + CheckResult result = check(R"( + function bar(): any + return true + end + + local a + for b in bar do + a = b + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ(typeChecker.anyType, requireType("a")); +} + +TEST_CASE_FIXTURE(Fixture, "for_in_loop_iterator_returns_any2") +{ + CheckResult result = check(R"( + function bar(): any + return true + end + + local a + for b in bar() do + a = b + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ("any", toString(requireType("a"))); +} + +TEST_CASE_FIXTURE(Fixture, "for_in_loop_iterator_is_any") +{ + CheckResult result = check(R"( + local bar: any + + local a + for b in bar do + a = b + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ("any", toString(requireType("a"))); +} + +TEST_CASE_FIXTURE(Fixture, "for_in_loop_iterator_is_any2") +{ + CheckResult result = check(R"( + local bar: any + + local a + for b in bar() do + a = b + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ("any", toString(requireType("a"))); +} + +TEST_CASE_FIXTURE(Fixture, "for_in_loop_iterator_is_error") +{ + CheckResult result = check(R"( + local a + for b in bar do + a = b + end + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + CHECK_EQ("*unknown*", toString(requireType("a"))); +} + +TEST_CASE_FIXTURE(Fixture, "for_in_loop_iterator_is_error2") +{ + CheckResult result = check(R"( + function bar(c) return c end + + local a + for b in bar() do + a = b + end + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + CHECK_EQ("*unknown*", toString(requireType("a"))); +} + +TEST_CASE_FIXTURE(Fixture, "length_of_error_type_does_not_produce_an_error") +{ + CheckResult result = check(R"( + local l = #this_is_not_defined + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); +} + +TEST_CASE_FIXTURE(Fixture, "indexing_error_type_does_not_produce_an_error") +{ + CheckResult result = check(R"( + local originalReward = unknown.Parent.Reward:GetChildren()[1] + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); +} + +TEST_CASE_FIXTURE(Fixture, "dot_on_error_type_does_not_produce_an_error") +{ + CheckResult result = check(R"( + local foo = (true).x + foo.x = foo.y + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); +} + +TEST_CASE_FIXTURE(Fixture, "any_type_propagates") +{ + CheckResult result = check(R"( + local foo: any + local bar = foo:method("argument") + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ("any", toString(requireType("bar"))); +} + +TEST_CASE_FIXTURE(Fixture, "can_subscript_any") +{ + CheckResult result = check(R"( + local foo: any + local bar = foo[5] + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ("any", toString(requireType("bar"))); +} + +// Not strictly correct: metatables permit overriding this +TEST_CASE_FIXTURE(Fixture, "can_get_length_of_any") +{ + CheckResult result = check(R"( + local foo: any = {} + local bar = #foo + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ(PrimitiveTypeVar::Number, getPrimitiveType(requireType("bar"))); +} + +TEST_CASE_FIXTURE(Fixture, "assign_prop_to_table_by_calling_any_yields_any") +{ + CheckResult result = check(R"( + local f: any + local T = {} + + T.prop = f() + + return T + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + TableTypeVar* ttv = getMutable(requireType("T")); + REQUIRE(ttv); + REQUIRE(ttv->props.count("prop")); + + REQUIRE_EQ("any", toString(ttv->props["prop"].type)); +} + +TEST_CASE_FIXTURE(Fixture, "quantify_any_does_not_bind_to_itself") +{ + CheckResult result = check(R"( + local A : any + function A.B() end + A:C() + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + TypeId aType = requireType("A"); + CHECK_EQ(aType, typeChecker.anyType); +} + +TEST_CASE_FIXTURE(Fixture, "calling_error_type_yields_error") +{ + CheckResult result = check(R"( + local a = unknown.Parent.Reward.GetChildren() + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + UnknownSymbol* err = get(result.errors[0]); + REQUIRE(err != nullptr); + + CHECK_EQ("unknown", err->name); + + CHECK_EQ("*unknown*", toString(requireType("a"))); +} + +TEST_CASE_FIXTURE(Fixture, "chain_calling_error_type_yields_error") +{ + CheckResult result = check(R"( + local a = Utility.Create "Foo" {} + )"); + + CHECK_EQ("*unknown*", toString(requireType("a"))); +} + +TEST_CASE_FIXTURE(Fixture, "replace_every_free_type_when_unifying_a_complex_function_with_any") +{ + CheckResult result = check(R"( + local a: any + local b + for _, i in pairs(a) do + b = i + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + CHECK_EQ("any", toString(requireType("b"))); +} + +TEST_CASE_FIXTURE(Fixture, "call_to_any_yields_any") +{ + CheckResult result = check(R"( + local a: any + local b = a() + )"); + + REQUIRE_EQ("any", toString(requireType("b"))); +} + +TEST_CASE_FIXTURE(Fixture, "CheckMethodsOfAny") +{ + CheckResult result = check(R"( +local x: any = {} +function x:y(z: number) + local s: string = z +end +)"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); +} + +TEST_CASE_FIXTURE(Fixture, "CheckMethodsOfError") +{ + CheckResult result = check(R"( +local x = (true).foo +function x:y(z: number) + local s: string = z +end +)"); + + LUAU_REQUIRE_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "metatable_of_any_can_be_a_table") +{ + CheckResult result = check(R"( +--!strict +local T: any +T = {} +T.__index = T +function T.new(...) + local self = {} + setmetatable(self, T) + self:construct(...) + return self +end +function T:construct(index) +end +)"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "type_error_addition") +{ + CheckResult result = check(R"( +--!strict +local foo = makesandwich() +local bar = foo.nutrition + 100 + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + // We should definitely get this error + CHECK_EQ("Unknown global 'makesandwich'", toString(result.errors[0])); + // We get this error if makesandwich() returns a free type + // CHECK_EQ("Unknown type used in + operation; consider adding a type annotation to 'foo'", toString(result.errors[1])); +} + +TEST_CASE_FIXTURE(Fixture, "prop_access_on_any_with_other_options") +{ + CheckResult result = check(R"( + local function f(thing: any | string) + local foo = thing.SomeRandomKey + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_SUITE_END(); diff --git a/tests/TypeInfer.builtins.test.cpp b/tests/TypeInfer.builtins.test.cpp index 8da655b3..ec20a2c7 100644 --- a/tests/TypeInfer.builtins.test.cpp +++ b/tests/TypeInfer.builtins.test.cpp @@ -871,6 +871,7 @@ TEST_CASE_FIXTURE(Fixture, "assert_removes_falsy_types") ScopedFastFlag sff[]{ {"LuauAssertStripsFalsyTypes", true}, {"LuauDiscriminableUnions2", true}, + {"LuauWidenIfSupertypeIsFree2", true}, }; CheckResult result = check(R"( @@ -879,6 +880,26 @@ TEST_CASE_FIXTURE(Fixture, "assert_removes_falsy_types") end )"); + LUAU_REQUIRE_NO_ERRORS(result); + CHECK_EQ("((boolean | number)?) -> boolean | number", toString(requireType("f"))); +} + +TEST_CASE_FIXTURE(Fixture, "assert_removes_falsy_types2") +{ + ScopedFastFlag sff[]{ + {"LuauParseSingletonTypes", true}, + {"LuauSingletonTypes", true}, + {"LuauAssertStripsFalsyTypes", true}, + {"LuauDiscriminableUnions2", true}, + {"LuauWidenIfSupertypeIsFree2", true}, + }; + + CheckResult result = check(R"( + local function f(x: (number | boolean)?): number | true + return assert(x) + end + )"); + LUAU_REQUIRE_NO_ERRORS(result); CHECK_EQ("((boolean | number)?) -> number | true", toString(requireType("f"))); } @@ -958,4 +979,43 @@ a:b({}) CHECK_EQ(result.errors[1], (TypeError{Location{{3, 0}, {3, 5}}, CountMismatch{2, 1}})); } +TEST_CASE_FIXTURE(Fixture, "typeof_unresolved_function") +{ + CheckResult result = check(R"( +local function f(a: typeof(f)) end +)"); + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ("Unknown global 'f'", toString(result.errors[0])); +} + +TEST_CASE_FIXTURE(Fixture, "no_persistent_typelevel_change") +{ + TypeId mathTy = requireType(typeChecker.globalScope, "math"); + REQUIRE(mathTy); + TableTypeVar* ttv = getMutable(mathTy); + REQUIRE(ttv); + const FunctionTypeVar* ftv = get(ttv->props["frexp"].type); + REQUIRE(ftv); + auto original = ftv->level; + + CheckResult result = check("local a = math.frexp"); + + LUAU_REQUIRE_NO_ERRORS(result); + CHECK(ftv->level.level == original.level); + CHECK(ftv->level.subLevel == original.subLevel); +} + +TEST_CASE_FIXTURE(Fixture, "global_singleton_types_are_sealed") +{ + CheckResult result = check(R"( +local function f(x: string) + local p = x:split('a') + p = table.pack(table.unpack(p, 1, #p - 1)) + return p +end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.definitions.test.cpp b/tests/TypeInfer.definitions.test.cpp index c6d55793..898d8902 100644 --- a/tests/TypeInfer.definitions.test.cpp +++ b/tests/TypeInfer.definitions.test.cpp @@ -293,4 +293,22 @@ TEST_CASE_FIXTURE(Fixture, "documentation_symbols_dont_attach_to_persistent_type CHECK_EQ(ty->type->documentationSymbol, std::nullopt); } +TEST_CASE_FIXTURE(Fixture, "single_class_type_identity_in_global_types") +{ + ScopedFastFlag luauCloneDeclaredGlobals{"LuauCloneDeclaredGlobals", true}; + + loadDefinition(R"( +declare class Cls +end + +declare GetCls: () -> (Cls) + )"); + + CheckResult result = check(R"( +local s : Cls = GetCls() + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.functions.test.cpp b/tests/TypeInfer.functions.test.cpp new file mode 100644 index 00000000..4288098a --- /dev/null +++ b/tests/TypeInfer.functions.test.cpp @@ -0,0 +1,1338 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details + +#include "Luau/AstQuery.h" +#include "Luau/BuiltinDefinitions.h" +#include "Luau/Scope.h" +#include "Luau/TypeInfer.h" +#include "Luau/TypeVar.h" +#include "Luau/VisitTypeVar.h" + +#include "Fixture.h" + +#include "doctest.h" + +using namespace Luau; + +TEST_SUITE_BEGIN("TypeInferFunctions"); + +TEST_CASE_FIXTURE(Fixture, "tc_function") +{ + CheckResult result = check("function five() return 5 end"); + LUAU_REQUIRE_NO_ERRORS(result); + + const FunctionTypeVar* fiveType = get(requireType("five")); + REQUIRE(fiveType != nullptr); +} + +TEST_CASE_FIXTURE(Fixture, "check_function_bodies") +{ + CheckResult result = check("function myFunction() local a = 0 a = true end"); + LUAU_REQUIRE_ERROR_COUNT(1, result); + + CHECK_EQ(result.errors[0], (TypeError{Location{Position{0, 44}, Position{0, 48}}, TypeMismatch{ + typeChecker.numberType, + typeChecker.booleanType, + }})); +} + +TEST_CASE_FIXTURE(Fixture, "infer_return_type") +{ + CheckResult result = check("function take_five() return 5 end"); + LUAU_REQUIRE_NO_ERRORS(result); + + const FunctionTypeVar* takeFiveType = get(requireType("take_five")); + REQUIRE(takeFiveType != nullptr); + + std::vector retVec = flatten(takeFiveType->retType).first; + REQUIRE(!retVec.empty()); + + REQUIRE_EQ(*follow(retVec[0]), *typeChecker.numberType); +} + +TEST_CASE_FIXTURE(Fixture, "infer_from_function_return_type") +{ + CheckResult result = check("function take_five() return 5 end local five = take_five()"); + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ(*typeChecker.numberType, *follow(requireType("five"))); +} + +TEST_CASE_FIXTURE(Fixture, "infer_that_function_does_not_return_a_table") +{ + CheckResult result = check(R"( + function take_five() + return 5 + end + + take_five().prop = 888 + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ(result.errors[0], (TypeError{Location{Position{5, 8}, Position{5, 24}}, NotATable{typeChecker.numberType}})); +} + +TEST_CASE_FIXTURE(Fixture, "vararg_functions_should_allow_calls_of_any_types_and_size") +{ + CheckResult result = check(R"( + function f(...) end + + f(1) + f("foo", 2) + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "vararg_function_is_quantified") +{ + CheckResult result = check(R"( + local T = {} + function T.f(...) + local result = {} + + for i = 1, select("#", ...) do + local dictionary = select(i, ...) + for key, value in pairs(dictionary) do + result[key] = value + end + end + + return result + end + + return T + )"); + + auto r = first(getMainModule()->getModuleScope()->returnType); + REQUIRE(r); + + TableTypeVar* ttv = getMutable(*r); + REQUIRE(ttv); + + TypeId k = ttv->props["f"].type; + REQUIRE(k); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "list_only_alternative_overloads_that_match_argument_count") +{ + CheckResult result = check(R"( + local multiply: ((number)->number) & ((number)->string) & ((number, number)->number) + multiply("") + )"); + + LUAU_REQUIRE_ERROR_COUNT(2, result); + + TypeMismatch* tm = get(result.errors[0]); + REQUIRE(tm); + CHECK_EQ(typeChecker.numberType, tm->wantedType); + CHECK_EQ(typeChecker.stringType, tm->givenType); + + ExtraInformation* ei = get(result.errors[1]); + REQUIRE(ei); + CHECK_EQ("Other overloads are also not viable: (number) -> string", ei->message); +} + +TEST_CASE_FIXTURE(Fixture, "list_all_overloads_if_no_overload_takes_given_argument_count") +{ + CheckResult result = check(R"( + local multiply: ((number)->number) & ((number)->string) & ((number, number)->number) + multiply() + )"); + + LUAU_REQUIRE_ERROR_COUNT(2, result); + + GenericError* ge = get(result.errors[0]); + REQUIRE(ge); + CHECK_EQ("No overload for function accepts 0 arguments.", ge->message); + + ExtraInformation* ei = get(result.errors[1]); + REQUIRE(ei); + CHECK_EQ("Available overloads: (number) -> number; (number) -> string; and (number, number) -> number", ei->message); +} + +TEST_CASE_FIXTURE(Fixture, "dont_give_other_overloads_message_if_only_one_argument_matching_overload_exists") +{ + CheckResult result = check(R"( + local multiply: ((number)->number) & ((number)->string) & ((number, number)->number) + multiply(1, "") + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + TypeMismatch* tm = get(result.errors[0]); + REQUIRE(tm); + CHECK_EQ(typeChecker.numberType, tm->wantedType); + CHECK_EQ(typeChecker.stringType, tm->givenType); +} + +TEST_CASE_FIXTURE(Fixture, "infer_return_type_from_selected_overload") +{ + CheckResult result = check(R"( + type T = {method: ((T, number) -> number) & ((number) -> string)} + local T: T + + local a = T.method(T, 4) + local b = T.method(5) + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + CHECK_EQ("number", toString(requireType("a"))); + CHECK_EQ("string", toString(requireType("b"))); +} + +TEST_CASE_FIXTURE(Fixture, "too_many_arguments") +{ + CheckResult result = check(R"( + --!nonstrict + + function g(a: number) end + + g() + + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + auto err = result.errors[0]; + auto acm = get(err); + REQUIRE(acm); + + CHECK_EQ(1, acm->expected); + CHECK_EQ(0, acm->actual); +} + +TEST_CASE_FIXTURE(Fixture, "recursive_function") +{ + CheckResult result = check(R"( + function count(n: number) + if n == 0 then + return 0 + else + return count(n - 1) + end + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "lambda_form_of_local_function_cannot_be_recursive") +{ + CheckResult result = check(R"( + local f = function() return f() end + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); +} + +TEST_CASE_FIXTURE(Fixture, "recursive_local_function") +{ + CheckResult result = check(R"( + local function count(n: number) + if n == 0 then + return 0 + else + return count(n - 1) + end + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +// FIXME: This and the above case get handled very differently. It's pretty dumb. +// We really should unify the two code paths, probably by deleting AstStatFunction. +TEST_CASE_FIXTURE(Fixture, "another_recursive_local_function") +{ + CheckResult result = check(R"( + local count + function count(n: number) + if n == 0 then + return 0 + else + return count(n - 1) + end + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "cyclic_function_type_in_rets") +{ + CheckResult result = check(R"( + function f() + return f + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + CHECK_EQ("t1 where t1 = () -> t1", toString(requireType("f"))); +} + +TEST_CASE_FIXTURE(Fixture, "cyclic_function_type_in_args") +{ + CheckResult result = check(R"( + function f(g) + return f(f) + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + CHECK_EQ("t1 where t1 = (t1) -> ()", toString(requireType("f"))); +} + +TEST_CASE_FIXTURE(Fixture, "another_higher_order_function") +{ + CheckResult result = check(R"( + local Get_des + function Get_des(func) + Get_des(func) + end + + local function f(d) + d:IsA("BasePart") + d.Parent:FindFirstChild("Humanoid") + d:IsA("Decal") + end + Get_des(f) + + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "another_other_higher_order_function") +{ + CheckResult result = check(R"( + local d + d:foo() + d:foo() + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "local_function") +{ + CheckResult result = check(R"( + function f() + return 8 + end + + function g() + local function f() + return 'hello' + end + return f + end + + local h = g() + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + TypeId h = follow(requireType("h")); + + const FunctionTypeVar* ftv = get(h); + REQUIRE(ftv != nullptr); + + std::optional rt = first(ftv->retType); + REQUIRE(bool(rt)); + + TypeId retType = follow(*rt); + CHECK_EQ(PrimitiveTypeVar::String, getPrimitiveType(retType)); +} + +TEST_CASE_FIXTURE(Fixture, "func_expr_doesnt_leak_free") +{ + CheckResult result = check(R"( + local p = function(x) return x end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + const Luau::FunctionTypeVar* fn = get(requireType("p")); + REQUIRE(fn); + auto ret = first(fn->retType); + REQUIRE(ret); + REQUIRE(get(follow(*ret))); +} + +TEST_CASE_FIXTURE(Fixture, "first_argument_can_be_optional") +{ + CheckResult result = check(R"( + local T = {} + function T.new(a: number?, b: number?, c: number?) return 5 end + local m = T.new() + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + dumpErrors(result); +} + +TEST_CASE_FIXTURE(Fixture, "it_is_ok_not_to_supply_enough_retvals") +{ + CheckResult result = check(R"( + function get_two() return 5, 6 end + + local a = get_two() + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + dumpErrors(result); +} + +TEST_CASE_FIXTURE(Fixture, "duplicate_functions2") +{ + CheckResult result = check(R"( + function foo() end + + function bar() + local function foo() end + end + )"); + + LUAU_REQUIRE_ERROR_COUNT(0, result); +} + +TEST_CASE_FIXTURE(Fixture, "duplicate_functions_allowed_in_nonstrict") +{ + CheckResult result = check(R"( + --!nonstrict + function foo() end + + function foo() end + + function bar() + local function foo() end + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "duplicate_functions_with_different_signatures_not_allowed_in_nonstrict") +{ + CheckResult result = check(R"( + --!nonstrict + function foo(): number + return 1 + end + foo() + + function foo(n: number): number + return 2 + end + foo() + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + TypeMismatch* tm = get(result.errors[0]); + REQUIRE(tm); + CHECK_EQ("() -> number", toString(tm->wantedType)); + CHECK_EQ("(number) -> number", toString(tm->givenType)); +} + +TEST_CASE_FIXTURE(Fixture, "complicated_return_types_require_an_explicit_annotation") +{ + CheckResult result = check(R"( + local i = 0 + function most_of_the_natural_numbers(): number? + if i < 10 then + i = i + 1 + return i + else + return nil + end + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + const FunctionTypeVar* functionType = get(requireType("most_of_the_natural_numbers")); + + std::optional retType = first(functionType->retType); + REQUIRE(retType); + CHECK(get(*retType)); +} + +TEST_CASE_FIXTURE(Fixture, "infer_higher_order_function") +{ + CheckResult result = check(R"( + function apply(f, x) + return f(x) + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + const FunctionTypeVar* ftv = get(requireType("apply")); + REQUIRE(ftv != nullptr); + + std::vector argVec = flatten(ftv->argTypes).first; + + REQUIRE_EQ(2, argVec.size()); + + const FunctionTypeVar* fType = get(follow(argVec[0])); + REQUIRE(fType != nullptr); + + std::vector fArgs = flatten(fType->argTypes).first; + + TypeId xType = argVec[1]; + + CHECK_EQ(1, fArgs.size()); + CHECK_EQ(xType, fArgs[0]); +} + +TEST_CASE_FIXTURE(Fixture, "higher_order_function_2") +{ + CheckResult result = check(R"( + function bottomupmerge(comp, a, b, left, mid, right) + local i, j = left, mid + for k = left, right do + if i < mid and (j > right or not comp(a[j], a[i])) then + b[k] = a[i] + i = i + 1 + else + b[k] = a[j] + j = j + 1 + end + end + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + const FunctionTypeVar* ftv = get(requireType("bottomupmerge")); + REQUIRE(ftv != nullptr); + + std::vector argVec = flatten(ftv->argTypes).first; + + REQUIRE_EQ(6, argVec.size()); + + const FunctionTypeVar* fType = get(follow(argVec[0])); + REQUIRE(fType != nullptr); +} + +TEST_CASE_FIXTURE(Fixture, "higher_order_function_3") +{ + CheckResult result = check(R"( + function swap(p) + local t = p[0] + p[0] = p[1] + p[1] = t + return nil + end + + function swapTwice(p) + swap(p) + swap(p) + return p + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + const FunctionTypeVar* ftv = get(requireType("swapTwice")); + REQUIRE(ftv != nullptr); + + std::vector argVec = flatten(ftv->argTypes).first; + + REQUIRE_EQ(1, argVec.size()); + + const TableTypeVar* argType = get(follow(argVec[0])); + REQUIRE(argType != nullptr); + + CHECK(bool(argType->indexer)); +} + +TEST_CASE_FIXTURE(Fixture, "higher_order_function_4") +{ + CheckResult result = check(R"( + function bottomupmerge(comp, a, b, left, mid, right) + local i, j = left, mid + for k = left, right do + if i < mid and (j > right or not comp(a[j], a[i])) then + b[k] = a[i] + i = i + 1 + else + b[k] = a[j] + j = j + 1 + end + end + end + + function mergesort(arr, comp) + local work = {} + for i = 1, #arr do + work[i] = arr[i] + end + local width = 1 + while width < #arr do + for i = 1, #arr, 2*width do + bottomupmerge(comp, arr, work, i, math.min(i+width, #arr), math.min(i+2*width-1, #arr)) + end + local temp = work + work = arr + arr = temp + width = width * 2 + end + return arr + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + dumpErrors(result); + + /* + * mergesort takes two arguments: an array of some type T and a function that takes two Ts. + * We must assert that these two types are in fact the same type. + * In other words, comp(arr[x], arr[y]) is well-typed. + */ + + const FunctionTypeVar* ftv = get(requireType("mergesort")); + REQUIRE(ftv != nullptr); + + std::vector argVec = flatten(ftv->argTypes).first; + + REQUIRE_EQ(2, argVec.size()); + + const TableTypeVar* arg0 = get(follow(argVec[0])); + REQUIRE(arg0 != nullptr); + REQUIRE(bool(arg0->indexer)); + + const FunctionTypeVar* arg1 = get(follow(argVec[1])); + REQUIRE(arg1 != nullptr); + REQUIRE_EQ(2, size(arg1->argTypes)); + + std::vector arg1Args = flatten(arg1->argTypes).first; + + CHECK_EQ(*arg0->indexer->indexResultType, *arg1Args[0]); + CHECK_EQ(*arg0->indexer->indexResultType, *arg1Args[1]); +} + +TEST_CASE_FIXTURE(Fixture, "mutual_recursion") +{ + CheckResult result = check(R"( + --!strict + + function newPlayerCharacter() + startGui() -- Unknown symbol 'startGui' + end + + local characterAddedConnection: any + function startGui() + characterAddedConnection = game:GetService("Players").LocalPlayer.CharacterAdded:connect(newPlayerCharacter) + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + dumpErrors(result); +} + +TEST_CASE_FIXTURE(Fixture, "toposort_doesnt_break_mutual_recursion") +{ + CheckResult result = check(R"( + --!strict + local x = nil + function f() g() end + -- make sure print(x) doesn't get toposorted here, breaking the mutual block + function g() x = f end + print(x) + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + dumpErrors(result); +} + +TEST_CASE_FIXTURE(Fixture, "check_function_before_lambda_that_uses_it") +{ + CheckResult result = check(R"( + --!nonstrict + + function f() + return 114 + end + + return function() + return f():andThen() + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "it_is_ok_to_oversaturate_a_higher_order_function_argument") +{ + CheckResult result = check(R"( + function onerror() end + function foo() end + xpcall(foo, onerror) + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "another_indirect_function_case_where_it_is_ok_to_provide_too_many_arguments") +{ + CheckResult result = check(R"( + local mycb: (number, number) -> () + + function f() end + + mycb = f + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "report_exiting_without_return_nonstrict") +{ + CheckResult result = check(R"( + --!nonstrict + + local function f1(v): number? + if v then + return 1 + end + end + + local function f2(v) + if v then + return 1 + end + end + + local function f3(v): () + if v then + return + end + end + + local function f4(v) + if v then + return + end + end + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + FunctionExitsWithoutReturning* err = get(result.errors[0]); + CHECK(err); +} + +TEST_CASE_FIXTURE(Fixture, "report_exiting_without_return_strict") +{ + CheckResult result = check(R"( + --!strict + + local function f1(v): number? + if v then + return 1 + end + end + + local function f2(v) + if v then + return 1 + end + end + + local function f3(v): () + if v then + return + end + end + + local function f4(v) + if v then + return + end + end + )"); + + LUAU_REQUIRE_ERROR_COUNT(2, result); + FunctionExitsWithoutReturning* annotatedErr = get(result.errors[0]); + CHECK(annotatedErr); + + FunctionExitsWithoutReturning* inferredErr = get(result.errors[1]); + CHECK(inferredErr); +} + +TEST_CASE_FIXTURE(Fixture, "calling_function_with_incorrect_argument_type_yields_errors_spanning_argument") +{ + CheckResult result = check(R"( + function foo(a: number, b: string) end + + foo("Test", 123) + )"); + + LUAU_REQUIRE_ERROR_COUNT(2, result); + + CHECK_EQ(result.errors[0], (TypeError{Location{Position{3, 12}, Position{3, 18}}, TypeMismatch{ + typeChecker.numberType, + typeChecker.stringType, + }})); + + CHECK_EQ(result.errors[1], (TypeError{Location{Position{3, 20}, Position{3, 23}}, TypeMismatch{ + typeChecker.stringType, + typeChecker.numberType, + }})); +} + +TEST_CASE_FIXTURE(Fixture, "calling_function_with_anytypepack_doesnt_leak_free_types") +{ + CheckResult result = check(R"( + --!nonstrict + + function Test(a) + return 1, "" + end + + + local tab = {} + table.insert(tab, Test(1)); + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + ToStringOptions opts; + opts.exhaustive = true; + opts.maxTableLength = 0; + + CHECK_EQ("{any}", toString(requireType("tab"), opts)); +} + +TEST_CASE_FIXTURE(Fixture, "too_many_return_values") +{ + CheckResult result = check(R"( + --!strict + + function f() + return 55 + end + + local a, b = f() + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + CountMismatch* acm = get(result.errors[0]); + REQUIRE(acm); + CHECK_EQ(acm->context, CountMismatch::Result); + CHECK_EQ(acm->expected, 1); + CHECK_EQ(acm->actual, 2); +} + +TEST_CASE_FIXTURE(Fixture, "ignored_return_values") +{ + CheckResult result = check(R"( + --!strict + + function f() + return 55, "" + end + + local a = f() + )"); + + LUAU_REQUIRE_ERROR_COUNT(0, result); +} + +TEST_CASE_FIXTURE(Fixture, "function_does_not_return_enough_values") +{ + CheckResult result = check(R"( + --!strict + + function f(): (number, string) + return 55 + end + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + CountMismatch* acm = get(result.errors[0]); + REQUIRE(acm); + CHECK_EQ(acm->context, CountMismatch::Return); + CHECK_EQ(acm->expected, 2); + CHECK_EQ(acm->actual, 1); +} + +TEST_CASE_FIXTURE(Fixture, "function_cast_error_uses_correct_language") +{ + CheckResult result = check(R"( + function foo(a, b): number + return 0 + end + + local a: (string)->number = foo + local b: (number, number)->(number, number) = foo + + local c: (string, number)->number = foo -- no error + )"); + + LUAU_REQUIRE_ERROR_COUNT(2, result); + + auto tm1 = get(result.errors[0]); + REQUIRE(tm1); + + CHECK_EQ("(string) -> number", toString(tm1->wantedType)); + CHECK_EQ("(string, *unknown*) -> number", toString(tm1->givenType)); + + auto tm2 = get(result.errors[1]); + REQUIRE(tm2); + + CHECK_EQ("(number, number) -> (number, number)", toString(tm2->wantedType)); + CHECK_EQ("(string, *unknown*) -> number", toString(tm2->givenType)); +} + +TEST_CASE_FIXTURE(Fixture, "no_lossy_function_type") +{ + CheckResult result = check(R"( + --!strict + local tbl = {} + function tbl:abc(a: number, b: number) + return a + end + tbl:abc(1, 2) -- Line 6 + -- | Column 14 + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + TypeId type = requireTypeAtPosition(Position(6, 14)); + CHECK_EQ("(tbl, number, number) -> number", toString(type)); + auto ftv = get(type); + REQUIRE(ftv); + CHECK(ftv->hasSelf); +} + +TEST_CASE_FIXTURE(Fixture, "record_matching_overload") +{ + CheckResult result = check(R"( + type Overload = ((string) -> string) & ((number) -> number) + local abc: Overload + abc(1) + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + // AstExprCall is the node that has the overload stored on it. + // findTypeAtPosition will look at the AstExprLocal, but this is not what + // we want to look at. + std::vector ancestry = findAstAncestryOfPosition(*getMainSourceModule(), Position(3, 10)); + REQUIRE_GE(ancestry.size(), 2); + AstExpr* parentExpr = ancestry[ancestry.size() - 2]->asExpr(); + REQUIRE(bool(parentExpr)); + REQUIRE(parentExpr->is()); + + ModulePtr module = getMainModule(); + auto it = module->astOverloadResolvedTypes.find(parentExpr); + REQUIRE(it); + CHECK_EQ(toString(*it), "(number) -> number"); +} + +TEST_CASE_FIXTURE(Fixture, "return_type_by_overload") +{ + ScopedFastFlag sff{"LuauErrorRecoveryType", true}; + + CheckResult result = check(R"( + type Overload = ((string) -> string) & ((number, number) -> number) + local abc: Overload + local x = abc(true) + local y = abc(true,true) + local z = abc(true,true,true) + )"); + + LUAU_REQUIRE_ERRORS(result); + CHECK_EQ("string", toString(requireType("x"))); + CHECK_EQ("number", toString(requireType("y"))); + // Should this be string|number? + CHECK_EQ("string", toString(requireType("z"))); +} + +TEST_CASE_FIXTURE(Fixture, "infer_anonymous_function_arguments") +{ + // Simple direct arg to arg propagation + CheckResult result = check(R"( +type Table = { x: number, y: number } +local function f(a: (Table) -> number) return a({x = 1, y = 2}) end +f(function(a) return a.x + a.y end) + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + // An optional function is accepted, but since we already provide a function, nil can be ignored + result = check(R"( +type Table = { x: number, y: number } +local function f(a: ((Table) -> number)?) if a then return a({x = 1, y = 2}) else return 0 end end +f(function(a) return a.x + a.y end) + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + // Make sure self calls match correct index + result = check(R"( +type Table = { x: number, y: number } +local x = {} +x.b = {x = 1, y = 2} +function x:f(a: (Table) -> number) return a(self.b) end +x:f(function(a) return a.x + a.y end) + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + // Mix inferred and explicit argument types + result = check(R"( +function f(a: (a: number, b: number, c: boolean) -> number) return a(1, 2, true) end +f(function(a: number, b, c) return c and a + b or b - a end) + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + // Anonymous function has a variadic pack + result = check(R"( +type Table = { x: number, y: number } +local function f(a: (Table) -> number) return a({x = 1, y = 2}) end +f(function(...) return select(1, ...).z end) + )"); + + LUAU_REQUIRE_ERRORS(result); + CHECK_EQ("Key 'z' not found in table 'Table'", toString(result.errors[0])); + + // Can't accept more arguments than provided + result = check(R"( +function f(a: (a: number, b: number) -> number) return a(1, 2) end +f(function(a, b, c, ...) return a + b end) + )"); + + LUAU_REQUIRE_ERRORS(result); + + CHECK_EQ(R"(Type '(number, number, a) -> number' could not be converted into '(number, number) -> number' +caused by: + Argument count mismatch. Function expects 3 arguments, but only 2 are specified)", + toString(result.errors[0])); + + // Infer from variadic packs into elements + result = check(R"( +function f(a: (...number) -> number) return a(1, 2) end +f(function(a, b) return a + b end) + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + // Infer from variadic packs into variadic packs + result = check(R"( +type Table = { x: number, y: number } +function f(a: (...Table) -> number) return a({x = 1, y = 2}, {x = 3, y = 4}) end +f(function(a, ...) local b = ... return b.z end) + )"); + + LUAU_REQUIRE_ERRORS(result); + CHECK_EQ("Key 'z' not found in table 'Table'", toString(result.errors[0])); + + // Return type inference + result = check(R"( +type Table = { x: number, y: number } +function f(a: (number) -> Table) return a(4) end +f(function(x) return x * 2 end) + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ("Type 'number' could not be converted into 'Table'", toString(result.errors[0])); + + // Return type doesn't inference 'nil' + result = check(R"( +function f(a: (number) -> nil) return a(4) end +f(function(x) print(x) end) + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "infer_anonymous_function_arguments") +{ + // Simple direct arg to arg propagation + CheckResult result = check(R"( +type Table = { x: number, y: number } +local function f(a: (Table) -> number) return a({x = 1, y = 2}) end +f(function(a) return a.x + a.y end) + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + // An optional function is accepted, but since we already provide a function, nil can be ignored + result = check(R"( +type Table = { x: number, y: number } +local function f(a: ((Table) -> number)?) if a then return a({x = 1, y = 2}) else return 0 end end +f(function(a) return a.x + a.y end) + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + // Make sure self calls match correct index + result = check(R"( +type Table = { x: number, y: number } +local x = {} +x.b = {x = 1, y = 2} +function x:f(a: (Table) -> number) return a(self.b) end +x:f(function(a) return a.x + a.y end) + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + // Mix inferred and explicit argument types + result = check(R"( +function f(a: (a: number, b: number, c: boolean) -> number) return a(1, 2, true) end +f(function(a: number, b, c) return c and a + b or b - a end) + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + // Anonymous function has a variadic pack + result = check(R"( +type Table = { x: number, y: number } +local function f(a: (Table) -> number) return a({x = 1, y = 2}) end +f(function(...) return select(1, ...).z end) + )"); + + LUAU_REQUIRE_ERRORS(result); + CHECK_EQ("Key 'z' not found in table 'Table'", toString(result.errors[0])); + + // Can't accept more arguments than provided + result = check(R"( +function f(a: (a: number, b: number) -> number) return a(1, 2) end +f(function(a, b, c, ...) return a + b end) + )"); + + LUAU_REQUIRE_ERRORS(result); + + CHECK_EQ(R"(Type '(number, number, a) -> number' could not be converted into '(number, number) -> number' +caused by: + Argument count mismatch. Function expects 3 arguments, but only 2 are specified)", + toString(result.errors[0])); + + // Infer from variadic packs into elements + result = check(R"( +function f(a: (...number) -> number) return a(1, 2) end +f(function(a, b) return a + b end) + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + // Infer from variadic packs into variadic packs + result = check(R"( +type Table = { x: number, y: number } +function f(a: (...Table) -> number) return a({x = 1, y = 2}, {x = 3, y = 4}) end +f(function(a, ...) local b = ... return b.z end) + )"); + + LUAU_REQUIRE_ERRORS(result); + CHECK_EQ("Key 'z' not found in table 'Table'", toString(result.errors[0])); + + // Return type inference + result = check(R"( +type Table = { x: number, y: number } +function f(a: (number) -> Table) return a(4) end +f(function(x) return x * 2 end) + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ("Type 'number' could not be converted into 'Table'", toString(result.errors[0])); + + // Return type doesn't inference 'nil' + result = check(R"( +function f(a: (number) -> nil) return a(4) end +f(function(x) print(x) end) + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "infer_anonymous_function_arguments_outside_call") +{ + CheckResult result = check(R"( +type Table = { x: number, y: number } +local f: (Table) -> number = function(t) return t.x + t.y end + +type TableWithFunc = { x: number, y: number, f: (number, number) -> number } +local a: TableWithFunc = { x = 3, y = 4, f = function(a, b) return a + b end } + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "infer_return_value_type") +{ + CheckResult result = check(R"( +local function f(): {string|number} + return {1, "b", 3} +end + +local function g(): (number, {string|number}) + return 4, {1, "b", 3} +end + +local function h(): ...{string|number} + return {4}, {1, "b", 3}, {"s"} +end + +local function i(): ...{string|number} + return {1, "b", 3}, h() +end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "error_detailed_function_mismatch_arg_count") +{ + CheckResult result = check(R"( +type A = (number, number) -> string +type B = (number) -> string + +local a: A +local b: B = a + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ(toString(result.errors[0]), R"(Type '(number, number) -> string' could not be converted into '(number) -> string' +caused by: + Argument count mismatch. Function expects 2 arguments, but only 1 is specified)"); +} + +TEST_CASE_FIXTURE(Fixture, "error_detailed_function_mismatch_arg") +{ + CheckResult result = check(R"( +type A = (number, number) -> string +type B = (number, string) -> string + +local a: A +local b: B = a + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ(toString(result.errors[0]), R"(Type '(number, number) -> string' could not be converted into '(number, string) -> string' +caused by: + Argument #2 type is not compatible. Type 'string' could not be converted into 'number')"); +} + +TEST_CASE_FIXTURE(Fixture, "error_detailed_function_mismatch_ret_count") +{ + CheckResult result = check(R"( +type A = (number, number) -> (number) +type B = (number, number) -> (number, boolean) + +local a: A +local b: B = a + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ(toString(result.errors[0]), R"(Type '(number, number) -> number' could not be converted into '(number, number) -> (number, boolean)' +caused by: + Function only returns 1 value. 2 are required here)"); +} + +TEST_CASE_FIXTURE(Fixture, "error_detailed_function_mismatch_ret") +{ + CheckResult result = check(R"( +type A = (number, number) -> string +type B = (number, number) -> number + +local a: A +local b: B = a + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ(toString(result.errors[0]), R"(Type '(number, number) -> string' could not be converted into '(number, number) -> number' +caused by: + Return type is not compatible. Type 'string' could not be converted into 'number')"); +} + +TEST_CASE_FIXTURE(Fixture, "error_detailed_function_mismatch_ret_mult") +{ + CheckResult result = check(R"( +type A = (number, number) -> (number, string) +type B = (number, number) -> (number, boolean) + +local a: A +local b: B = a + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ(toString(result.errors[0]), + R"(Type '(number, number) -> (number, string)' could not be converted into '(number, number) -> (number, boolean)' +caused by: + Return #2 type is not compatible. Type 'string' could not be converted into 'boolean')"); +} + +TEST_CASE_FIXTURE(Fixture, "function_decl_quantify_right_type") +{ + ScopedFastFlag statFunctionSimplify{"LuauStatFunctionSimplify2", true}; + + fileResolver.source["game/isAMagicMock"] = R"( +--!nonstrict +return function(value) + return false +end + )"; + + CheckResult result = check(R"( +--!nonstrict +local MagicMock = {} +MagicMock.is = require(game.isAMagicMock) + +function MagicMock.is(value) + return false +end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "function_decl_non_self_sealed_overwrite") +{ + ScopedFastFlag statFunctionSimplify{"LuauStatFunctionSimplify2", true}; + + CheckResult result = check(R"( +function string.len(): number + return 1 +end + )"); + + LUAU_REQUIRE_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "strict_mode_ok_with_missing_arguments") +{ + ScopedFastFlag sff{"LuauAnyInIsOptionalIsOptional", true}; + + CheckResult result = check(R"( + local function f(x: any) end + f() + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "function_statement_sealed_table_assignment_through_indexer") +{ + ScopedFastFlag statFunctionSimplify{"LuauStatFunctionSimplify2", true}; + + CheckResult result = check(R"( +local t: {[string]: () -> number} = {} + +function t.a() return 1 end -- OK +function t:b() return 2 end -- not OK + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ(R"(Type '(*unknown*) -> number' could not be converted into '() -> number' +caused by: + Argument count mismatch. Function expects 1 argument, but none are specified)", + toString(result.errors[0])); +} + +TEST_SUITE_END(); diff --git a/tests/TypeInfer.generics.test.cpp b/tests/TypeInfer.generics.test.cpp index 547fbab1..f360a77c 100644 --- a/tests/TypeInfer.generics.test.cpp +++ b/tests/TypeInfer.generics.test.cpp @@ -1,6 +1,9 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #include "Luau/TypeInfer.h" #include "Luau/TypeVar.h" +#include "Luau/Scope.h" + +#include #include "Fixture.h" @@ -830,5 +833,303 @@ wrapper(test2, 1, "", 3) CHECK_EQ(toString(result.errors[0]), R"(Argument count mismatch. Function expects 1 argument, but 4 are specified)"); } +TEST_CASE_FIXTURE(Fixture, "generic_function") +{ + CheckResult result = check(R"( + function id(x) return x end + local a = id(55) + local b = id(nil) + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ(*typeChecker.numberType, *requireType("a")); + CHECK_EQ(*typeChecker.nilType, *requireType("b")); +} + +TEST_CASE_FIXTURE(Fixture, "generic_table_method") +{ + CheckResult result = check(R"( + local T = {} + + function T:bar(i) + return i + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + TypeId tType = requireType("T"); + TableTypeVar* tTable = getMutable(tType); + REQUIRE(tTable != nullptr); + + TypeId barType = tTable->props["bar"].type; + REQUIRE(barType != nullptr); + + const FunctionTypeVar* ftv = get(follow(barType)); + REQUIRE_MESSAGE(ftv != nullptr, "Should be a function: " << *barType); + + std::vector args = flatten(ftv->argTypes).first; + TypeId argType = args.at(1); + + CHECK_MESSAGE(get(argType), "Should be generic: " << *barType); +} + +TEST_CASE_FIXTURE(Fixture, "correctly_instantiate_polymorphic_member_functions") +{ + CheckResult result = check(R"( + local T = {} + + function T:foo() + return T:bar(5) + end + + function T:bar(i) + return i + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + dumpErrors(result); + + const TableTypeVar* t = get(requireType("T")); + REQUIRE(t != nullptr); + + std::optional fooProp = get(t->props, "foo"); + REQUIRE(bool(fooProp)); + + const FunctionTypeVar* foo = get(follow(fooProp->type)); + REQUIRE(bool(foo)); + + std::optional ret_ = first(foo->retType); + REQUIRE(bool(ret_)); + TypeId ret = follow(*ret_); + + REQUIRE_EQ(getPrimitiveType(ret), PrimitiveTypeVar::Number); +} + +/* + * We had a bug in instantiation where the argument types of 'f' and 'g' would be inferred as + * f {+ method: function(): (t2, T3...) +} + * g {+ method: function({+ method: function(): (t2, T3...) +}): (t5, T6...) +} + * + * The type of 'g' is totally wrong as t2 and t5 should be unified, as should T3 with T6. + * + * The correct unification of the argument to 'g' is + * + * {+ method: function(): (t5, T6...) +} + */ +TEST_CASE_FIXTURE(Fixture, "instantiate_cyclic_generic_function") +{ + auto result = check(R"( + function f(o) + o:method() + end + + function g(o) + f(o) + end + )"); + + TypeId g = requireType("g"); + const FunctionTypeVar* gFun = get(g); + REQUIRE(gFun != nullptr); + + auto optionArg = first(gFun->argTypes); + REQUIRE(bool(optionArg)); + + TypeId arg = follow(*optionArg); + const TableTypeVar* argTable = get(arg); + REQUIRE(argTable != nullptr); + + std::optional methodProp = get(argTable->props, "method"); + REQUIRE(bool(methodProp)); + + const FunctionTypeVar* methodFunction = get(methodProp->type); + REQUIRE(methodFunction != nullptr); + + std::optional methodArg = first(methodFunction->argTypes); + REQUIRE(bool(methodArg)); + + REQUIRE_EQ(follow(*methodArg), follow(arg)); +} + +TEST_CASE_FIXTURE(Fixture, "instantiate_generic_function_in_assignments") +{ + CheckResult result = check(R"( + function foo(a, b) + return a(b) + end + + function bar() + local c: ((number)->number, number)->number = foo -- no error + c = foo -- no error + local d: ((number)->number, string)->number = foo -- error from arg 2 (string) not being convertable to number from the call a(b) + end + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + TypeMismatch* tm = get(result.errors[0]); + REQUIRE(tm); + CHECK_EQ("((number) -> number, string) -> number", toString(tm->wantedType)); + CHECK_EQ("((number) -> number, number) -> number", toString(tm->givenType)); +} + +TEST_CASE_FIXTURE(Fixture, "instantiate_generic_function_in_assignments2") +{ + CheckResult result = check(R"( + function foo(a, b) + return a(b) + end + + function bar() + local _: (string, string)->number = foo -- string cannot be converted to (string)->number + end + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + TypeMismatch* tm = get(result.errors[0]); + REQUIRE(tm); + CHECK_EQ("(string, string) -> number", toString(tm->wantedType)); + CHECK_EQ("((string) -> number, string) -> number", toString(*tm->givenType)); +} + +TEST_CASE_FIXTURE(Fixture, "self_recursive_instantiated_param") +{ + ScopedFastFlag sff{"LuauOnlyMutateInstantiatedTables", true}; + + // Mutability in type function application right now can create strange recursive types + CheckResult result = check(R"( +type Table = { a: number } +type Self = T +local a: Self
+ )"); + + LUAU_REQUIRE_NO_ERRORS(result); + CHECK_EQ(toString(requireType("a")), "Table"); +} + +TEST_CASE_FIXTURE(Fixture, "no_stack_overflow_from_quantifying") +{ + CheckResult result = check(R"( + function _(l0:t0): (any, ()->()) + end + + type t0 = t0 | {} + )"); + + CHECK_LE(0, result.errors.size()); + + std::optional t0 = getMainModule()->getModuleScope()->lookupType("t0"); + REQUIRE(t0); + CHECK_EQ("*unknown*", toString(t0->type)); + + auto it = std::find_if(result.errors.begin(), result.errors.end(), [](TypeError& err) { + return get(err); + }); + CHECK(it != result.errors.end()); +} + +TEST_CASE_FIXTURE(Fixture, "infer_generic_function_function_argument") +{ + ScopedFastFlag sff{"LuauUnsealedTableLiteral", true}; + + CheckResult result = check(R"( +local function sum(x: a, y: a, f: (a, a) -> a) return f(x, y) end +return sum(2, 3, function(a, b) return a + b end) + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + result = check(R"( +local function map(arr: {a}, f: (a) -> b) local r = {} for i,v in ipairs(arr) do table.insert(r, f(v)) end return r end +local a = {1, 2, 3} +local r = map(a, function(a) return a + a > 100 end) + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + REQUIRE_EQ("{boolean}", toString(requireType("r"))); + + check(R"( +local function foldl(arr: {a}, init: b, f: (b, a) -> b) local r = init for i,v in ipairs(arr) do r = f(r, v) end return r end +local a = {1, 2, 3} +local r = foldl(a, {s=0,c=0}, function(a, b) return {s = a.s + b, c = a.c + 1} end) + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + REQUIRE_EQ("{ c: number, s: number }", toString(requireType("r"))); +} + +TEST_CASE_FIXTURE(Fixture, "infer_generic_function_function_argument_overloaded") +{ + CheckResult result = check(R"( +local function g1(a: T, f: (T) -> T) return f(a) end +local function g2(a: T, b: T, f: (T, T) -> T) return f(a, b) end + +local g12: typeof(g1) & typeof(g2) + +g12(1, function(x) return x + x end) +g12(1, 2, function(x, y) return x + y end) + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + result = check(R"( +local function g1(a: T, f: (T) -> T) return f(a) end +local function g2(a: T, b: T, f: (T, T) -> T) return f(a, b) end + +local g12: typeof(g1) & typeof(g2) + +g12({x=1}, function(x) return {x=-x.x} end) +g12({x=1}, {x=2}, function(x, y) return {x=x.x + y.x} end) + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "infer_generic_lib_function_function_argument") +{ + CheckResult result = check(R"( +local a = {{x=4}, {x=7}, {x=1}} +table.sort(a, function(x, y) return x.x < y.x end) + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "do_not_infer_generic_functions") +{ + CheckResult result = check(R"( +local function sum(x: a, y: a, f: (a, a) -> a) return f(x, y) end + +local function sumrec(f: typeof(sum)) + return sum(2, 3, function(a, b) return a + b end) +end + +local b = sumrec(sum) -- ok +local c = sumrec(function(x, y, f) return f(x, y) end) -- type binders are not inferred + )"); + + LUAU_REQUIRE_ERRORS(result); + CHECK_EQ("Type '(a, b, (a, b) -> (c...)) -> (c...)' could not be converted into '(a, a, (a, a) -> a) -> a'; different number of generic type " + "parameters", + toString(result.errors[0])); +} + +TEST_CASE_FIXTURE(Fixture, "substitution_with_bound_table") +{ + CheckResult result = check(R"( +type A = { x: number } +local a: A = { x = 1 } +local b = a +type B = typeof(b) +type X = T +local c: X + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} TEST_SUITE_END(); diff --git a/tests/TypeInfer.intersectionTypes.test.cpp b/tests/TypeInfer.intersectionTypes.test.cpp index 26881b5c..d146f4e8 100644 --- a/tests/TypeInfer.intersectionTypes.test.cpp +++ b/tests/TypeInfer.intersectionTypes.test.cpp @@ -377,4 +377,32 @@ local b: number = a CHECK_EQ(toString(result.errors[0]), R"(Type 'X & Y & Z' could not be converted into 'number'; none of the intersection parts are compatible)"); } +TEST_CASE_FIXTURE(Fixture, "overload_is_not_a_function") +{ + check(R"( +--!nonstrict +function _(...):((typeof(not _))&(typeof(not _)))&((typeof(not _))&(typeof(not _))) +_(...)(setfenv,_,not _,"")[_] = nil +end +do end +_(...)(...,setfenv,_):_G() +)"); +} + +TEST_CASE_FIXTURE(Fixture, "no_stack_overflow_from_flattenintersection") +{ + CheckResult result = check(R"( + local l0,l0 + repeat + type t0 = ((any)|((any)&((any)|((any)&((any)|(any))))))&(t0) + function _(l0):(t0)&(t0) + while nil do + end + end + until _(_)(_)._ + )"); + + CHECK_LE(0, result.errors.size()); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.loops.test.cpp b/tests/TypeInfer.loops.test.cpp new file mode 100644 index 00000000..30df717b --- /dev/null +++ b/tests/TypeInfer.loops.test.cpp @@ -0,0 +1,473 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details + +#include "Luau/AstQuery.h" +#include "Luau/BuiltinDefinitions.h" +#include "Luau/Scope.h" +#include "Luau/TypeInfer.h" +#include "Luau/TypeVar.h" +#include "Luau/VisitTypeVar.h" + +#include "Fixture.h" + +#include "doctest.h" + +using namespace Luau; + +TEST_SUITE_BEGIN("TypeInferLoops"); + +TEST_CASE_FIXTURE(Fixture, "for_loop") +{ + CheckResult result = check(R"( + local q + for i=0, 50, 2 do + q = i + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ(*typeChecker.numberType, *requireType("q")); +} + +TEST_CASE_FIXTURE(Fixture, "for_in_loop") +{ + CheckResult result = check(R"( + local n + local s + for i, v in pairs({ "foo" }) do + n = i + s = v + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ(*typeChecker.numberType, *requireType("n")); + CHECK_EQ(*typeChecker.stringType, *requireType("s")); +} + +TEST_CASE_FIXTURE(Fixture, "for_in_loop_with_next") +{ + CheckResult result = check(R"( + local n + local s + for i, v in next, { "foo", "bar" } do + n = i + s = v + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ(*typeChecker.numberType, *requireType("n")); + CHECK_EQ(*typeChecker.stringType, *requireType("s")); +} + +TEST_CASE_FIXTURE(Fixture, "for_in_with_an_iterator_of_type_any") +{ + CheckResult result = check(R"( + local it: any + local a, b + for i, v in it do + a, b = i, v + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "for_in_loop_should_fail_with_non_function_iterator") +{ + CheckResult result = check(R"( + local foo = "bar" + for i, v in foo do + end + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); +} + +TEST_CASE_FIXTURE(Fixture, "for_in_with_just_one_iterator_is_ok") +{ + CheckResult result = check(R"( + local function keys(dictionary) + local new = {} + local index = 1 + + for key in pairs(dictionary) do + new[index] = key + index = index + 1 + end + + return new + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "for_in_with_a_custom_iterator_should_type_check") +{ + CheckResult result = check(R"( + local function range(l, h): () -> number + return function() + return l + end + end + + for n: string in range(1, 10) do + print(n) + end + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); +} + +TEST_CASE_FIXTURE(Fixture, "for_in_loop_on_error") +{ + CheckResult result = check(R"( + function f(x) + gobble.prop = x.otherprop + end + + local p + for _, part in i_am_not_defined do + p = part + f(part) + part.thirdprop = false + end + )"); + + CHECK_EQ(2, result.errors.size()); + + TypeId p = requireType("p"); + CHECK_EQ("*unknown*", toString(p)); +} + +TEST_CASE_FIXTURE(Fixture, "for_in_loop_on_non_function") +{ + CheckResult result = check(R"( + local bad_iter = 5 + + for a in bad_iter() do + end + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + REQUIRE(get(result.errors[0])); +} + +TEST_CASE_FIXTURE(Fixture, "for_in_loop_error_on_factory_not_returning_the_right_amount_of_values") +{ + CheckResult result = check(R"( + local function hasDivisors(value: number, table) + return false + end + + function prime_iter(state, index) + while hasDivisors(index, state) do + index += 1 + end + + state[index] = true + return index + end + + function primes1() + return prime_iter, {} + end + + function primes2() + return prime_iter, {}, "" + end + + function primes3() + return prime_iter, {}, 2 + end + + for p in primes1() do print(p) end -- mismatch in argument count + + for p in primes2() do print(p) end -- mismatch in argument types, prime_iter takes {}, number, we are given {}, string + + for p in primes3() do print(p) end -- no error + )"); + + LUAU_REQUIRE_ERROR_COUNT(2, result); + + CountMismatch* acm = get(result.errors[0]); + REQUIRE(acm); + CHECK_EQ(acm->context, CountMismatch::Arg); + CHECK_EQ(2, acm->expected); + CHECK_EQ(1, acm->actual); + + TypeMismatch* tm = get(result.errors[1]); + REQUIRE(tm); + CHECK_EQ(typeChecker.numberType, tm->wantedType); + CHECK_EQ(typeChecker.stringType, tm->givenType); +} + +TEST_CASE_FIXTURE(Fixture, "for_in_loop_error_on_iterator_requiring_args_but_none_given") +{ + CheckResult result = check(R"( + function prime_iter(state, index) + return 1 + end + + for p in prime_iter do print(p) end + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + CountMismatch* acm = get(result.errors[0]); + REQUIRE(acm); + CHECK_EQ(acm->context, CountMismatch::Arg); + CHECK_EQ(2, acm->expected); + CHECK_EQ(0, acm->actual); +} + +TEST_CASE_FIXTURE(Fixture, "for_in_loop_with_custom_iterator") +{ + CheckResult result = check(R"( + function primes() + return function (state: number) end, 2 + end + + for p, q in primes do + q = "" + end + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + TypeMismatch* tm = get(result.errors[0]); + REQUIRE(tm); + CHECK_EQ(typeChecker.numberType, tm->wantedType); + CHECK_EQ(typeChecker.stringType, tm->givenType); +} + +TEST_CASE_FIXTURE(Fixture, "while_loop") +{ + CheckResult result = check(R"( + local i + while true do + i = 8 + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ(*typeChecker.numberType, *requireType("i")); +} + +TEST_CASE_FIXTURE(Fixture, "repeat_loop") +{ + CheckResult result = check(R"( + local i + repeat + i = 'hi' + until true + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ(*typeChecker.stringType, *requireType("i")); +} + +TEST_CASE_FIXTURE(Fixture, "repeat_loop_condition_binds_to_its_block") +{ + CheckResult result = check(R"( + repeat + local x = true + until x + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "symbols_in_repeat_block_should_not_be_visible_beyond_until_condition") +{ + CheckResult result = check(R"( + repeat + local x = true + until x + + print(x) + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); +} + +TEST_CASE_FIXTURE(Fixture, "varlist_declared_by_for_in_loop_should_be_free") +{ + CheckResult result = check(R"( + local T = {} + + function T.f(p) + for i, v in pairs(p) do + T.f(v) + end + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "properly_infer_iteratee_is_a_free_table") +{ + // In this case, we cannot know the element type of the table {}. It could be anything. + // We therefore must initially ascribe a free typevar to iter. + CheckResult result = check(R"( + for iter in pairs({}) do + iter:g().p = true + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "correctly_scope_locals_while") +{ + CheckResult result = check(R"( + while true do + local a = 1 + end + + print(a) -- oops! + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + UnknownSymbol* us = get(result.errors[0]); + REQUIRE(us); + CHECK_EQ(us->name, "a"); +} + +TEST_CASE_FIXTURE(Fixture, "ipairs_produces_integral_indices") +{ + CheckResult result = check(R"( + local key + for i, e in ipairs({}) do key = i end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + REQUIRE_EQ("number", toString(requireType("key"))); +} + +TEST_CASE_FIXTURE(Fixture, "for_in_loop_where_iteratee_is_free") +{ + // This code doesn't pass typechecking. We just care that it doesn't crash. + (void)check(R"( + --!nonstrict + function _:_(...) + end + + repeat + if _ then + else + _ = ... + end + until _ + + for _ in _() do + end + )"); +} + +TEST_CASE_FIXTURE(Fixture, "unreachable_code_after_infinite_loop") +{ + { + CheckResult result = check(R"( + function unreachablecodepath(a): number + while true do + if a then return 10 end + end + -- unreachable + end + unreachablecodepath(4) + )"); + + LUAU_REQUIRE_ERROR_COUNT(0, result); + } + + { + CheckResult result = check(R"( + function reachablecodepath(a): number + while true do + if a then break end + return 10 + end + + print("x") -- correct error + end + reachablecodepath(4) + )"); + + LUAU_REQUIRE_ERRORS(result); + CHECK(get(result.errors[0])); + } + + { + CheckResult result = check(R"( + function unreachablecodepath(a): number + repeat + if a then return 10 end + until false + + -- unreachable + end + unreachablecodepath(4) + )"); + + LUAU_REQUIRE_ERROR_COUNT(0, result); + } + + { + CheckResult result = check(R"( + function reachablecodepath(a, b): number + repeat + if a then break end + + if b then return 10 end + until false + + print("x") -- correct error + end + reachablecodepath(4) + )"); + + LUAU_REQUIRE_ERRORS(result); + CHECK(get(result.errors[0])); + } + + { + CheckResult result = check(R"( + function unreachablecodepath(a: number?): number + repeat + return 10 + until a ~= nil + + -- unreachable + end + unreachablecodepath(4) + )"); + + LUAU_REQUIRE_ERROR_COUNT(0, result); + } +} + +TEST_CASE_FIXTURE(Fixture, "loop_typecheck_crash_on_empty_optional") +{ + CheckResult result = check(R"( + local t = {} + for _ in t do + for _ in assert(missing()) do + end + end + )"); + + LUAU_REQUIRE_ERROR_COUNT(2, result); +} + +TEST_SUITE_END(); diff --git a/tests/TypeInfer.modules.test.cpp b/tests/TypeInfer.modules.test.cpp new file mode 100644 index 00000000..63643610 --- /dev/null +++ b/tests/TypeInfer.modules.test.cpp @@ -0,0 +1,310 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details + +#include "Luau/AstQuery.h" +#include "Luau/BuiltinDefinitions.h" +#include "Luau/Scope.h" +#include "Luau/TypeInfer.h" +#include "Luau/TypeVar.h" +#include "Luau/VisitTypeVar.h" + +#include "Fixture.h" + +#include "doctest.h" + +using namespace Luau; + +TEST_SUITE_BEGIN("TypeInferModules"); + +TEST_CASE_FIXTURE(Fixture, "require") +{ + fileResolver.source["game/A"] = R"( + local function hooty(x: number): string + return "Hi there!" + end + + return {hooty=hooty} + )"; + + fileResolver.source["game/B"] = R"( + local Hooty = require(game.A) + + local h -- free! + local i = Hooty.hooty(h) + )"; + + CheckResult aResult = frontend.check("game/A"); + dumpErrors(aResult); + LUAU_REQUIRE_NO_ERRORS(aResult); + + CheckResult bResult = frontend.check("game/B"); + dumpErrors(bResult); + LUAU_REQUIRE_NO_ERRORS(bResult); + + ModulePtr b = frontend.moduleResolver.modules["game/B"]; + + REQUIRE(b != nullptr); + + dumpErrors(bResult); + + std::optional iType = requireType(b, "i"); + REQUIRE_EQ("string", toString(*iType)); + + std::optional hType = requireType(b, "h"); + REQUIRE_EQ("number", toString(*hType)); +} + +TEST_CASE_FIXTURE(Fixture, "require_types") +{ + fileResolver.source["workspace/A"] = R"( + export type Point = {x: number, y: number} + + return {} + )"; + + fileResolver.source["workspace/B"] = R"( + local Hooty = require(workspace.A) + + local h: Hooty.Point + )"; + + CheckResult bResult = frontend.check("workspace/B"); + dumpErrors(bResult); + + ModulePtr b = frontend.moduleResolver.modules["workspace/B"]; + REQUIRE(b != nullptr); + + TypeId hType = requireType(b, "h"); + REQUIRE_MESSAGE(bool(get(hType)), "Expected table but got " << toString(hType)); +} + +TEST_CASE_FIXTURE(Fixture, "require_a_variadic_function") +{ + fileResolver.source["game/A"] = R"( + local T = {} + function T.f(...) end + return T + )"; + + fileResolver.source["game/B"] = R"( + local A = require(game.A) + local f = A.f + )"; + + CheckResult result = frontend.check("game/B"); + + ModulePtr bModule = frontend.moduleResolver.getModule("game/B"); + REQUIRE(bModule != nullptr); + + TypeId f = follow(requireType(bModule, "f")); + + const FunctionTypeVar* ftv = get(f); + REQUIRE(ftv); + + auto iter = begin(ftv->argTypes); + auto endIter = end(ftv->argTypes); + + REQUIRE(iter == endIter); + REQUIRE(iter.tail()); + + CHECK(get(*iter.tail())); +} + +TEST_CASE_FIXTURE(Fixture, "type_error_of_unknown_qualified_type") +{ + CheckResult result = check(R"( + local p: SomeModule.DoesNotExist + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + REQUIRE_EQ(result.errors[0], (TypeError{Location{{1, 17}, {1, 40}}, UnknownSymbol{"SomeModule.DoesNotExist"}})); +} + +TEST_CASE_FIXTURE(Fixture, "require_module_that_does_not_export") +{ + const std::string sourceA = R"( + )"; + + const std::string sourceB = R"( + local Hooty = require(script.Parent.A) + )"; + + fileResolver.source["game/Workspace/A"] = sourceA; + fileResolver.source["game/Workspace/B"] = sourceB; + + frontend.check("game/Workspace/A"); + frontend.check("game/Workspace/B"); + + ModulePtr aModule = frontend.moduleResolver.modules["game/Workspace/A"]; + ModulePtr bModule = frontend.moduleResolver.modules["game/Workspace/B"]; + + CHECK(aModule->errors.empty()); + REQUIRE_EQ(1, bModule->errors.size()); + CHECK_MESSAGE(get(bModule->errors[0]), "Should be IllegalRequire: " << toString(bModule->errors[0])); + + auto hootyType = requireType(bModule, "Hooty"); + + CHECK_EQ("*unknown*", toString(hootyType)); +} + +TEST_CASE_FIXTURE(Fixture, "warn_if_you_try_to_require_a_non_modulescript") +{ + fileResolver.source["Modules/A"] = ""; + fileResolver.sourceTypes["Modules/A"] = SourceCode::Local; + + fileResolver.source["Modules/B"] = R"( + local M = require(script.Parent.A) + )"; + + CheckResult result = frontend.check("Modules/B"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + CHECK(get(result.errors[0])); +} + +TEST_CASE_FIXTURE(Fixture, "general_require_call_expression") +{ + fileResolver.source["game/A"] = R"( +--!strict +return { def = 4 } + )"; + + fileResolver.source["game/B"] = R"( +--!strict +local tbl = { abc = require(game.A) } +local a : string = "" +a = tbl.abc.def + )"; + + CheckResult result = frontend.check("game/B"); + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ("Type 'number' could not be converted into 'string'", toString(result.errors[0])); +} + +TEST_CASE_FIXTURE(Fixture, "general_require_type_mismatch") +{ + fileResolver.source["game/A"] = R"( +return { def = 4 } + )"; + + fileResolver.source["game/B"] = R"( +local tbl: string = require(game.A) + )"; + + CheckResult result = frontend.check("game/B"); + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ("Type '{| def: number |}' could not be converted into 'string'", toString(result.errors[0])); +} + +TEST_CASE_FIXTURE(Fixture, "bound_free_table_export_is_ok") +{ + CheckResult result = check(R"( +local n = {} +function n:Clone() end + +local m = {} + +function m.a(x) + x:Clone() +end + +function m.b() + m.a(n) +end + +return m +)"); + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "custom_require_global") +{ + CheckResult result = check(R"( +--!nonstrict +require = function(a) end + +local crash = require(game.A) + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "require_failed_module") +{ + fileResolver.source["game/A"] = R"( +return unfortunately() + )"; + + CheckResult aResult = frontend.check("game/A"); + LUAU_REQUIRE_ERRORS(aResult); + + CheckResult result = check(R"( +local ModuleA = require(game.A) + )"); + LUAU_REQUIRE_NO_ERRORS(result); + + std::optional oty = requireType("ModuleA"); + CHECK_EQ("*unknown*", toString(*oty)); +} + +TEST_CASE_FIXTURE(Fixture, "do_not_modify_imported_types") +{ + fileResolver.source["game/A"] = R"( +export type Type = { unrelated: boolean } +return {} + )"; + + fileResolver.source["game/B"] = R"( +local types = require(game.A) +type Type = types.Type +local x: Type = {} +function x:Destroy(): () end + )"; + + CheckResult result = frontend.check("game/B"); + LUAU_REQUIRE_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "do_not_modify_imported_types_2") +{ + ScopedFastFlag immutableTypes{"LuauImmutableTypes", true}; + + fileResolver.source["game/A"] = R"( +export type Type = { x: { a: number } } +return {} + )"; + + fileResolver.source["game/B"] = R"( +local types = require(game.A) +type Type = types.Type +local x: Type = { x = { a = 2 } } +type Rename = typeof(x.x) + )"; + + CheckResult result = frontend.check("game/B"); + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "do_not_modify_imported_types_3") +{ + ScopedFastFlag immutableTypes{"LuauImmutableTypes", true}; + + fileResolver.source["game/A"] = R"( +local y = setmetatable({}, {}) +export type Type = { x: typeof(y) } +return { x = y } + )"; + + fileResolver.source["game/B"] = R"( +local types = require(game.A) +type Type = types.Type +local x: Type = types +type Rename = typeof(x.x) + )"; + + CheckResult result = frontend.check("game/B"); + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_SUITE_END(); diff --git a/tests/TypeInfer.oop.test.cpp b/tests/TypeInfer.oop.test.cpp new file mode 100644 index 00000000..40831bf6 --- /dev/null +++ b/tests/TypeInfer.oop.test.cpp @@ -0,0 +1,275 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details + +#include "Luau/AstQuery.h" +#include "Luau/BuiltinDefinitions.h" +#include "Luau/Scope.h" +#include "Luau/TypeInfer.h" +#include "Luau/TypeVar.h" +#include "Luau/VisitTypeVar.h" + +#include "Fixture.h" + +#include "doctest.h" + +using namespace Luau; + +TEST_SUITE_BEGIN("TypeInferOOP"); + +TEST_CASE_FIXTURE(Fixture, "dont_suggest_using_colon_rather_than_dot_if_not_defined_with_colon") +{ + CheckResult result = check(R"( + local someTable = {} + + someTable.Function1 = function(Arg1) + end + + someTable.Function1() -- Argument count mismatch + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + REQUIRE(get(result.errors[0])); +} + +TEST_CASE_FIXTURE(Fixture, "dont_suggest_using_colon_rather_than_dot_if_it_wont_help_2") +{ + CheckResult result = check(R"( + local someTable = {} + + someTable.Function2 = function(Arg1, Arg2) + end + + someTable.Function2() -- Argument count mismatch + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + REQUIRE(get(result.errors[0])); +} + +TEST_CASE_FIXTURE(Fixture, "dont_suggest_using_colon_rather_than_dot_if_another_overload_works") +{ + CheckResult result = check(R"( + type T = {method: ((T, number) -> number) & ((number) -> number)} + local T: T + + T.method(4) + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "method_depends_on_table") +{ + CheckResult result = check(R"( + -- This catches a bug where x:m didn't count as a use of x + -- so toposort would happily reorder a definition of + -- function x:m before the definition of x. + function g() f() end + local x = {} + function x:m() end + function f() x:m() end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "methods_are_topologically_sorted") +{ + CheckResult result = check(R"( + local T = {} + + function T:foo() + return T:bar(999), T:bar("hi") + end + + function T:bar(i) + return i + end + + local a, b = T:foo() + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + dumpErrors(result); + + CHECK_EQ(PrimitiveTypeVar::Number, getPrimitiveType(requireType("a"))); + CHECK_EQ(PrimitiveTypeVar::String, getPrimitiveType(requireType("b"))); +} + +TEST_CASE_FIXTURE(Fixture, "quantify_methods_defined_using_dot_syntax_and_explicit_self_parameter") +{ + check(R"( + local T = {} + + function T.method(self) + self:method() + end + + function T.method2(self) + self:method() + end + + T:method2() + )"); +} + +TEST_CASE_FIXTURE(Fixture, "inferring_hundreds_of_self_calls_should_not_suffocate_memory") +{ + CheckResult result = check(R"( + ("foo") + :lower() + :lower() + :lower() + :lower() + :lower() + :lower() + :lower() + :lower() + :lower() + :lower() + :lower() + :lower() + :lower() + :lower() + :lower() + :lower() + :lower() + :lower() + )"); + + ModulePtr module = getMainModule(); + CHECK_GE(50, module->internalTypes.typeVars.size()); +} + +TEST_CASE_FIXTURE(Fixture, "object_constructor_can_refer_to_method_of_self") +{ + // CLI-30902 + CheckResult result = check(R"( + --!strict + + type Foo = { + fooConn: () -> () | nil + } + + local Foo = {} + Foo.__index = Foo + + function Foo.new() + local self: Foo = { + fooConn = nil, + } + setmetatable(self, Foo) + + self.fooConn = function() + self:method() -- Key 'method' not found in table self + end + + return self + end + + function Foo:method() + print("foo") + end + + local foo = Foo.new() + + -- TODO This is the best our current refinement support can offer :( + local bar = foo.fooConn + if bar then bar() end + + -- foo.fooConn() + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "CheckMethodsOfSealed") +{ + CheckResult result = check(R"( +local x: {prop: number} = {prop=9999} +function x:y(z: number) + local s: string = z +end +)"); + + LUAU_REQUIRE_ERROR_COUNT(2, result); +} + +TEST_CASE_FIXTURE(Fixture, "nonstrict_self_mismatch_tail") +{ + CheckResult result = check(R"( +--!nonstrict +local f = {} +function f:foo(a: number, b: number) end + +function bar(...) + f.foo(f, 1, ...) +end + +bar(2) +)"); + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "inferred_methods_of_free_tables_have_the_same_level_as_the_enclosing_table") +{ + check(R"( + function Base64FileReader(data) + local reader = {} + local index: number + + function reader:PeekByte() + return data:byte(index) + end + + function reader:Byte() + return data:byte(index - 1) + end + + return reader + end + + Base64FileReader() + + function ReadMidiEvents(data) + + local reader = Base64FileReader(data) + + while reader:HasMore() do + (reader:Byte() % 128) + end + end + )"); +} + +TEST_CASE_FIXTURE(Fixture, "table_oop") +{ + CheckResult result = check(R"( + --!strict +local Class = {} +Class.__index = Class + +type Class = typeof(setmetatable({} :: { x: number }, Class)) + +function Class.new(x: number): Class + return setmetatable({x = x}, Class) +end + +function Class.getx(self: Class) + return self.x +end + +function test() + local c = Class.new(42) + local n = c:getx() + local nn = c.x + + print(string.format("%d %d", n, nn)) +end +)"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_SUITE_END(); diff --git a/tests/TypeInfer.operators.test.cpp b/tests/TypeInfer.operators.test.cpp new file mode 100644 index 00000000..baa25978 --- /dev/null +++ b/tests/TypeInfer.operators.test.cpp @@ -0,0 +1,759 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details + +#include "Luau/AstQuery.h" +#include "Luau/BuiltinDefinitions.h" +#include "Luau/Scope.h" +#include "Luau/TypeInfer.h" +#include "Luau/TypeVar.h" +#include "Luau/VisitTypeVar.h" + +#include "Fixture.h" + +#include "doctest.h" + +using namespace Luau; + +TEST_SUITE_BEGIN("TypeInferOperators"); + +TEST_CASE_FIXTURE(Fixture, "or_joins_types") +{ + CheckResult result = check(R"( + local s = "a" or 10 + local x:string|number = s + )"); + LUAU_REQUIRE_NO_ERRORS(result); + CHECK_EQ(toString(*requireType("s")), "number | string"); + CHECK_EQ(toString(*requireType("x")), "number | string"); +} + +TEST_CASE_FIXTURE(Fixture, "or_joins_types_with_no_extras") +{ + CheckResult result = check(R"( + local s = "a" or 10 + local x:number|string = s + local y = x or "s" + )"); + CHECK_EQ(0, result.errors.size()); + CHECK_EQ(toString(*requireType("s")), "number | string"); + CHECK_EQ(toString(*requireType("y")), "number | string"); +} + +TEST_CASE_FIXTURE(Fixture, "or_joins_types_with_no_superfluous_union") +{ + CheckResult result = check(R"( + local s = "a" or "b" + local x:string = s + )"); + CHECK_EQ(0, result.errors.size()); + CHECK_EQ(*requireType("s"), *typeChecker.stringType); +} + +TEST_CASE_FIXTURE(Fixture, "and_adds_boolean") +{ + CheckResult result = check(R"( + local s = "a" and 10 + local x:boolean|number = s + )"); + CHECK_EQ(0, result.errors.size()); + CHECK_EQ(toString(*requireType("s")), "boolean | number"); +} + +TEST_CASE_FIXTURE(Fixture, "and_adds_boolean_no_superfluous_union") +{ + CheckResult result = check(R"( + local s = "a" and true + local x:boolean = s + )"); + CHECK_EQ(0, result.errors.size()); + CHECK_EQ(*requireType("x"), *typeChecker.booleanType); +} + +TEST_CASE_FIXTURE(Fixture, "and_or_ternary") +{ + CheckResult result = check(R"( + local s = (1/2) > 0.5 and "a" or 10 + )"); + CHECK_EQ(0, result.errors.size()); + CHECK_EQ(toString(*requireType("s")), "number | string"); +} + +TEST_CASE_FIXTURE(Fixture, "primitive_arith_no_metatable") +{ + CheckResult result = check(R"( + function add(a: number, b: string) + return a + (tonumber(b) :: number), a .. b + end + local n, s = add(2,"3") + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + const FunctionTypeVar* functionType = get(requireType("add")); + + std::optional retType = first(functionType->retType); + CHECK_EQ(std::optional(typeChecker.numberType), retType); + CHECK_EQ(requireType("n"), typeChecker.numberType); + CHECK_EQ(requireType("s"), typeChecker.stringType); +} + +TEST_CASE_FIXTURE(Fixture, "primitive_arith_no_metatable_with_follows") +{ + CheckResult result = check(R"( + local PI=3.1415926535897931 + local SOLAR_MASS=4*PI * PI + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + CHECK_EQ(requireType("SOLAR_MASS"), typeChecker.numberType); +} + +TEST_CASE_FIXTURE(Fixture, "primitive_arith_possible_metatable") +{ + CheckResult result = check(R"( + function add(a: number, b: any) + return a + b + end + local t = add(1,2) + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + CHECK_EQ("any", toString(requireType("t"))); +} + +TEST_CASE_FIXTURE(Fixture, "some_primitive_binary_ops") +{ + CheckResult result = check(R"( + local a = 4 + 8 + local b = a + 9 + local s = 'hotdogs' + local t = s .. s + local c = b - a + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ("number", toString(requireType("a"))); + CHECK_EQ("number", toString(requireType("b"))); + CHECK_EQ("string", toString(requireType("s"))); + CHECK_EQ("string", toString(requireType("t"))); + CHECK_EQ("number", toString(requireType("c"))); +} + +TEST_CASE_FIXTURE(Fixture, "typecheck_overloaded_multiply_that_is_an_intersection") +{ + ScopedFastFlag sff{"LuauErrorRecoveryType", true}; + + CheckResult result = check(R"( + --!strict + local Vec3 = {} + Vec3.__index = Vec3 + function Vec3.new() + return setmetatable({x=0, y=0, z=0}, Vec3) + end + + export type Vec3 = typeof(Vec3.new()) + + local thefun: any = function(self, o) return self end + + local multiply: ((Vec3, Vec3) -> Vec3) & ((Vec3, number) -> Vec3) = thefun + + Vec3.__mul = multiply + + local a = Vec3.new() + local b = Vec3.new() + local c = a * b + local d = a * 2 + local e = a * 'cabbage' + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + CHECK_EQ("Vec3", toString(requireType("a"))); + CHECK_EQ("Vec3", toString(requireType("b"))); + CHECK_EQ("Vec3", toString(requireType("c"))); + CHECK_EQ("Vec3", toString(requireType("d"))); + CHECK_EQ("Vec3", toString(requireType("e"))); +} + +TEST_CASE_FIXTURE(Fixture, "typecheck_overloaded_multiply_that_is_an_intersection_on_rhs") +{ + ScopedFastFlag sff{"LuauErrorRecoveryType", true}; + + CheckResult result = check(R"( + --!strict + local Vec3 = {} + Vec3.__index = Vec3 + function Vec3.new() + return setmetatable({x=0, y=0, z=0}, Vec3) + end + + export type Vec3 = typeof(Vec3.new()) + + local thefun: any = function(self, o) return self end + + local multiply: ((Vec3, Vec3) -> Vec3) & ((Vec3, number) -> Vec3) = thefun + + Vec3.__mul = multiply + + local a = Vec3.new() + local b = Vec3.new() + local c = b * a + local d = 2 * a + local e = 'cabbage' * a + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + CHECK_EQ("Vec3", toString(requireType("a"))); + CHECK_EQ("Vec3", toString(requireType("b"))); + CHECK_EQ("Vec3", toString(requireType("c"))); + CHECK_EQ("Vec3", toString(requireType("d"))); + CHECK_EQ("Vec3", toString(requireType("e"))); +} + +TEST_CASE_FIXTURE(Fixture, "compare_numbers") +{ + CheckResult result = check(R"( + local a = 441 + local b = 0 + local c = a < b + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "compare_strings") +{ + CheckResult result = check(R"( + local a = '441' + local b = '0' + local c = a < b + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "cannot_indirectly_compare_types_that_do_not_have_a_metatable") +{ + CheckResult result = check(R"( + local a = {} + local b = {} + local c = a < b + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + GenericError* gen = get(result.errors[0]); + + REQUIRE_EQ(gen->message, "Type a cannot be compared with < because it has no metatable"); +} + +TEST_CASE_FIXTURE(Fixture, "cannot_indirectly_compare_types_that_do_not_offer_overloaded_ordering_operators") +{ + CheckResult result = check(R"( + local M = {} + function M.new() + return setmetatable({}, M) + end + type M = typeof(M.new()) + + local a = M.new() + local b = M.new() + local c = a < b + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + GenericError* gen = get(result.errors[0]); + REQUIRE(gen != nullptr); + REQUIRE_EQ(gen->message, "Table M does not offer metamethod __lt"); +} + +TEST_CASE_FIXTURE(Fixture, "cannot_compare_tables_that_do_not_have_the_same_metatable") +{ + CheckResult result = check(R"( + --!strict + local M = {} + function M.new() + return setmetatable({}, M) + end + function M.__lt(left, right) return true end + + local a = M.new() + local b = {} + local c = a < b -- line 10 + local d = b < a -- line 11 + )"); + + LUAU_REQUIRE_ERROR_COUNT(2, result); + + REQUIRE_EQ((Location{{10, 18}, {10, 23}}), result.errors[0].location); + + REQUIRE_EQ((Location{{11, 18}, {11, 23}}), result.errors[1].location); +} + +TEST_CASE_FIXTURE(Fixture, "produce_the_correct_error_message_when_comparing_a_table_with_a_metatable_with_one_that_does_not") +{ + CheckResult result = check(R"( + --!strict + local M = {} + function M.new() + return setmetatable({}, M) + end + function M.__lt(left, right) return true end + type M = typeof(M.new()) + + local a = M.new() + local b = {} + local c = a < b -- line 10 + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + auto err = get(result.errors[0]); + REQUIRE(err != nullptr); + + // Frail. :| + REQUIRE_EQ("Types M and b cannot be compared with < because they do not have the same metatable", err->message); +} + +TEST_CASE_FIXTURE(Fixture, "in_nonstrict_mode_strip_nil_from_intersections_when_considering_relational_operators") +{ + CheckResult result = check(R"( + --!nonstrict + + function maybe_a_number(): number? + return 50 + end + + local a = maybe_a_number() < maybe_a_number() + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "compound_assign_basic") +{ + CheckResult result = check(R"( + local s = 10 + s += 20 + )"); + CHECK_EQ(0, result.errors.size()); + CHECK_EQ(toString(*requireType("s")), "number"); +} + +TEST_CASE_FIXTURE(Fixture, "compound_assign_mismatch_op") +{ + CheckResult result = check(R"( + local s = 10 + s += true + )"); + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ(result.errors[0], (TypeError{Location{{2, 13}, {2, 17}}, TypeMismatch{typeChecker.numberType, typeChecker.booleanType}})); +} + +TEST_CASE_FIXTURE(Fixture, "compound_assign_mismatch_result") +{ + CheckResult result = check(R"( + local s = 'hello' + s += 10 + )"); + + LUAU_REQUIRE_ERROR_COUNT(2, result); + CHECK_EQ(result.errors[0], (TypeError{Location{{2, 8}, {2, 9}}, TypeMismatch{typeChecker.numberType, typeChecker.stringType}})); + CHECK_EQ(result.errors[1], (TypeError{Location{{2, 8}, {2, 15}}, TypeMismatch{typeChecker.stringType, typeChecker.numberType}})); +} + +TEST_CASE_FIXTURE(Fixture, "compound_assign_metatable") +{ + CheckResult result = check(R"( + --!strict + type V2B = { x: number, y: number } + local v2b: V2B = { x = 0, y = 0 } + local VMT = {} + type V2 = typeof(setmetatable(v2b, VMT)) + + function VMT.__add(a: V2, b: V2): V2 + return setmetatable({ x = a.x + b.x, y = a.y + b.y }, VMT) + end + + local v1: V2 = setmetatable({ x = 1, y = 2 }, VMT) + local v2: V2 = setmetatable({ x = 3, y = 4 }, VMT) + v1 += v2 + )"); + CHECK_EQ(0, result.errors.size()); +} + +TEST_CASE_FIXTURE(Fixture, "compound_assign_mismatch_metatable") +{ + CheckResult result = check(R"( + --!strict + type V2B = { x: number, y: number } + local v2b: V2B = { x = 0, y = 0 } + local VMT = {} + type V2 = typeof(setmetatable(v2b, VMT)) + + function VMT.__mod(a: V2, b: V2): number + return a.x * b.x + a.y * b.y + end + + local v1: V2 = setmetatable({ x = 1, y = 2 }, VMT) + local v2: V2 = setmetatable({ x = 3, y = 4 }, VMT) + v1 %= v2 + )"); + LUAU_REQUIRE_ERROR_COUNT(1, result); + + TypeMismatch* tm = get(result.errors[0]); + CHECK_EQ(*tm->wantedType, *requireType("v2")); + CHECK_EQ(*tm->givenType, *typeChecker.numberType); +} + +TEST_CASE_FIXTURE(Fixture, "CallOrOfFunctions") +{ + CheckResult result = check(R"( +function f() return 1; end +function g() return 2; end +(f or g)() +)"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "CallAndOrOfFunctions") +{ + CheckResult result = check(R"( +function f() return 1; end +function g() return 2; end +local x = false +(x and f or g)() +)"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "typecheck_unary_minus") +{ + CheckResult result = check(R"( + --!strict + local foo = { + value = 10 + } + local mt = {} + setmetatable(foo, mt) + + mt.__unm = function(val: typeof(foo)): string + return val.value .. "test" + end + + local a = -foo + + local b = 1+-1 + + local bar = { + value = 10 + } + local c = -bar -- disallowed + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + CHECK_EQ("string", toString(requireType("a"))); + CHECK_EQ("number", toString(requireType("b"))); + + GenericError* gen = get(result.errors[0]); + REQUIRE_EQ(gen->message, "Unary operator '-' not supported by type 'bar'"); +} + +TEST_CASE_FIXTURE(Fixture, "unary_not_is_boolean") +{ + CheckResult result = check(R"( + local b = not "string" + local c = not (math.random() > 0.5 and "string" or 7) + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + REQUIRE_EQ("boolean", toString(requireType("b"))); + REQUIRE_EQ("boolean", toString(requireType("c"))); +} + +TEST_CASE_FIXTURE(Fixture, "disallow_string_and_types_without_metatables_from_arithmetic_binary_ops") +{ + CheckResult result = check(R"( + --!strict + local a = "1.24" + 123 -- not allowed + + local foo = { + value = 10 + } + + local b = foo + 1 -- not allowed + + local bar = { + value = 1 + } + + local mt = {} + + setmetatable(bar, mt) + + mt.__add = function(a: typeof(bar), b: number): number + return a.value + b + end + + local c = bar + 1 -- allowed + + local d = bar + foo -- not allowed + )"); + + LUAU_REQUIRE_ERROR_COUNT(3, result); + + TypeMismatch* tm = get(result.errors[0]); + REQUIRE_EQ(*tm->wantedType, *typeChecker.numberType); + REQUIRE_EQ(*tm->givenType, *typeChecker.stringType); + + TypeMismatch* tm2 = get(result.errors[2]); + CHECK_EQ(*tm2->wantedType, *typeChecker.numberType); + CHECK_EQ(*tm2->givenType, *requireType("foo")); + + GenericError* gen2 = get(result.errors[1]); + REQUIRE_EQ(gen2->message, "Binary operator '+' not supported by types 'foo' and 'number'"); +} + +// CLI-29033 +TEST_CASE_FIXTURE(Fixture, "unknown_type_in_comparison") +{ + CheckResult result = check(R"( + function merge(lower, greater) + if lower.y == greater.y then + end + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "concat_op_on_free_lhs_and_string_rhs") +{ + CheckResult result = check(R"( + local function f(x) + return x .. "y" + end + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + REQUIRE(get(result.errors[0])); +} + +TEST_CASE_FIXTURE(Fixture, "concat_op_on_string_lhs_and_free_rhs") +{ + CheckResult result = check(R"( + local function f(x) + return "foo" .. x + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ("(string) -> string", toString(requireType("f"))); +} + +TEST_CASE_FIXTURE(Fixture, "strict_binary_op_where_lhs_unknown") +{ + std::vector ops = {"+", "-", "*", "/", "%", "^", ".."}; + + std::string src = R"( + function foo(a, b) + )"; + + for (const auto& op : ops) + src += "local _ = a " + op + "b\n"; + + src += "end"; + + CheckResult result = check(src); + LUAU_REQUIRE_ERROR_COUNT(ops.size(), result); + + CHECK_EQ("Unknown type used in + operation; consider adding a type annotation to 'a'", toString(result.errors[0])); +} + +TEST_CASE_FIXTURE(Fixture, "and_binexps_dont_unify") +{ + CheckResult result = check(R"( + --!strict + local t = {} + while true and t[1] do + print(t[1].test) + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "error_on_invalid_operand_types_to_relational_operators") +{ + CheckResult result = check(R"( + local a: boolean = true + local b: boolean = false + local foo = a < b + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + GenericError* ge = get(result.errors[0]); + REQUIRE(ge); + CHECK_EQ("Type 'boolean' cannot be compared with relational operator <", ge->message); +} + +TEST_CASE_FIXTURE(Fixture, "error_on_invalid_operand_types_to_relational_operators2") +{ + CheckResult result = check(R"( + local a: number | string = "" + local b: number | string = 1 + local foo = a < b + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + GenericError* ge = get(result.errors[0]); + REQUIRE(ge); + CHECK_EQ("Type 'number | string' cannot be compared with relational operator <", ge->message); +} + +TEST_CASE_FIXTURE(Fixture, "cli_38355_recursive_union") +{ + CheckResult result = check(R"( + --!strict + local _ + _ += _ and _ or _ and _ or _ and _ + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ("Type contains a self-recursive construct that cannot be resolved", toString(result.errors[0])); +} + +TEST_CASE_FIXTURE(Fixture, "UnknownGlobalCompoundAssign") +{ + // In non-strict mode, global definition is still allowed + { + CheckResult result = check(R"( + --!nonstrict + a = a + 1 + print(a) + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ(toString(result.errors[0]), "Unknown global 'a'"); + } + + // In strict mode we no longer generate two errors from lhs + { + CheckResult result = check(R"( + --!strict + a += 1 + print(a) + )"); + + LUAU_REQUIRE_ERRORS(result); + CHECK_EQ(toString(result.errors[0]), "Unknown global 'a'"); + } + + // In non-strict mode, compound assignment is not a definition, it's a modification + { + CheckResult result = check(R"( + --!nonstrict + a += 1 + print(a) + )"); + + LUAU_REQUIRE_ERROR_COUNT(2, result); + CHECK_EQ(toString(result.errors[0]), "Unknown global 'a'"); + } +} + +TEST_CASE_FIXTURE(Fixture, "strip_nil_from_lhs_or_operator") +{ + CheckResult result = check(R"( +--!strict +local a: number? = nil +local b: number = a or 1 + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "strip_nil_from_lhs_or_operator2") +{ + CheckResult result = check(R"( +--!nonstrict +local a: number? = nil +local b: number = a or 1 + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "dont_strip_nil_from_rhs_or_operator") +{ + CheckResult result = check(R"( +--!strict +local a: number? = nil +local b: number = 1 or a + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + TypeMismatch* tm = get(result.errors[0]); + REQUIRE(tm); + CHECK_EQ(typeChecker.numberType, tm->wantedType); + CHECK_EQ("number?", toString(tm->givenType)); +} + +TEST_CASE_FIXTURE(Fixture, "operator_eq_verifies_types_do_intersect") +{ + CheckResult result = check(R"( + type Array = { [number]: T } + type Fiber = { id: number } + type null = {} + + local fiberStack: Array = {} + local index = 0 + + local function f(fiber: Fiber) + local a = fiber ~= fiberStack[index] + local b = fiberStack[index] ~= fiber + end + + return f + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "operator_eq_operands_are_not_subtypes_of_each_other_but_has_overlap") +{ + ScopedFastFlag sff1{"LuauEqConstraint", true}; + + CheckResult result = check(R"( + local function f(a: string | number, b: boolean | number) + return a == b + end + )"); + + // This doesn't produce any errors but for the wrong reasons. + // This unit test serves as a reminder to not try and unify the operands on `==`/`~=`. + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "refine_and_or") +{ + CheckResult result = check(R"( + local t: {x: number?}? = {x = nil} + local u = t and t.x or 5 + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ("number", toString(requireType("u"))); +} + +TEST_SUITE_END(); diff --git a/tests/TypeInfer.primitives.test.cpp b/tests/TypeInfer.primitives.test.cpp new file mode 100644 index 00000000..44b7b0d0 --- /dev/null +++ b/tests/TypeInfer.primitives.test.cpp @@ -0,0 +1,100 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details + +#include "Luau/AstQuery.h" +#include "Luau/BuiltinDefinitions.h" +#include "Luau/Scope.h" +#include "Luau/TypeInfer.h" +#include "Luau/TypeVar.h" +#include "Luau/VisitTypeVar.h" + +#include "Fixture.h" + +#include "doctest.h" + +using namespace Luau; + +TEST_SUITE_BEGIN("TypeInferPrimitives"); + +TEST_CASE_FIXTURE(Fixture, "cannot_call_primitives") +{ + CheckResult result = check("local foo = 5 foo()"); + LUAU_REQUIRE_ERROR_COUNT(1, result); + + REQUIRE(get(result.errors[0]) != nullptr); +} + +TEST_CASE_FIXTURE(Fixture, "string_length") +{ + CheckResult result = check(R"( + local s = "Hello, World!" + local t = #s + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + CHECK_EQ("number", toString(requireType("t"))); +} + +TEST_CASE_FIXTURE(Fixture, "string_index") +{ + CheckResult result = check(R"( + local s = "Hello, World!" + local t = s[4] + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + NotATable* nat = get(result.errors[0]); + REQUIRE(nat); + CHECK_EQ("string", toString(nat->ty)); + + CHECK_EQ("*unknown*", toString(requireType("t"))); +} + +TEST_CASE_FIXTURE(Fixture, "string_method") +{ + CheckResult result = check(R"( + local p = ("tacos"):len() + )"); + CHECK_EQ(0, result.errors.size()); + + CHECK_EQ(*requireType("p"), *typeChecker.numberType); +} + +TEST_CASE_FIXTURE(Fixture, "string_function_indirect") +{ + CheckResult result = check(R"( + local s:string + local l = s.lower + local p = l(s) + )"); + CHECK_EQ(0, result.errors.size()); + + CHECK_EQ(*requireType("p"), *typeChecker.stringType); +} + +TEST_CASE_FIXTURE(Fixture, "string_function_other") +{ + CheckResult result = check(R"( + local s:string + local p = s:match("foo") + )"); + CHECK_EQ(0, result.errors.size()); + + CHECK_EQ(toString(requireType("p")), "string?"); +} + +TEST_CASE_FIXTURE(Fixture, "CheckMethodsOfNumber") +{ + ScopedFastFlag sff{"LuauErrorRecoveryType", true}; + + CheckResult result = check(R"( +local x: number = 9999 +function x:y(z: number) + local s: string = z +end +)"); + + LUAU_REQUIRE_ERROR_COUNT(2, result); +} + +TEST_SUITE_END(); diff --git a/tests/TypeInfer.refinements.test.cpp b/tests/TypeInfer.refinements.test.cpp index a5147d56..9b347921 100644 --- a/tests/TypeInfer.refinements.test.cpp +++ b/tests/TypeInfer.refinements.test.cpp @@ -1298,4 +1298,22 @@ TEST_CASE_FIXTURE(RefinementClassFixture, "x_is_not_instance_or_else_not_part") CHECK_EQ("Part", toString(requireTypeAtPosition({5, 28}))); } +TEST_CASE_FIXTURE(Fixture, "typeguard_doesnt_leak_to_elseif") +{ + const std::string code = R"( + function f(a) + if type(a) == "boolean" then + local a1 = a + elseif a.fn() then + local a2 = a + else + local a3 = a + end + end + )"; + CheckResult result = check(code); + LUAU_REQUIRE_NO_ERRORS(result); + dumpErrors(result); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.singletons.test.cpp b/tests/TypeInfer.singletons.test.cpp index 3ed536ea..7f8d8fec 100644 --- a/tests/TypeInfer.singletons.test.cpp +++ b/tests/TypeInfer.singletons.test.cpp @@ -5,6 +5,8 @@ #include "doctest.h" #include "Luau/BuiltinDefinitions.h" +LUAU_FASTFLAG(BetterDiagnosticCodesInStudio) + using namespace Luau; TEST_SUITE_BEGIN("TypeSingletons"); @@ -353,7 +355,14 @@ TEST_CASE_FIXTURE(Fixture, "table_properties_alias_or_parens_is_indexer") )"); LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK_EQ("Syntax error: Cannot have more than one table indexer", toString(result.errors[0])); + if (FFlag::BetterDiagnosticCodesInStudio) + { + CHECK_EQ("Cannot have more than one table indexer", toString(result.errors[0])); + } + else + { + CHECK_EQ("Syntax error: Cannot have more than one table indexer", toString(result.errors[0])); + } } TEST_CASE_FIXTURE(Fixture, "table_properties_type_error_escapes") @@ -445,7 +454,7 @@ TEST_CASE_FIXTURE(Fixture, "widen_the_supertype_if_it_is_free_and_subtype_has_si {"LuauSingletonTypes", true}, {"LuauEqConstraint", true}, {"LuauDiscriminableUnions2", true}, - {"LuauWidenIfSupertypeIsFree", true}, + {"LuauWidenIfSupertypeIsFree2", true}, {"LuauWeakEqConstraint", false}, }; @@ -472,9 +481,9 @@ TEST_CASE_FIXTURE(Fixture, "return_type_of_f_is_not_widened") {"LuauSingletonTypes", true}, {"LuauDiscriminableUnions2", true}, {"LuauEqConstraint", true}, - {"LuauWidenIfSupertypeIsFree", true}, + {"LuauWidenIfSupertypeIsFree2", true}, {"LuauWeakEqConstraint", false}, - {"LuauDoNotAccidentallyDependOnPointerOrdering", true} + {"LuauDoNotAccidentallyDependOnPointerOrdering", true}, }; CheckResult result = check(R"( @@ -497,7 +506,7 @@ TEST_CASE_FIXTURE(Fixture, "widening_happens_almost_everywhere") ScopedFastFlag sff[]{ {"LuauParseSingletonTypes", true}, {"LuauSingletonTypes", true}, - {"LuauWidenIfSupertypeIsFree", true}, + {"LuauWidenIfSupertypeIsFree2", true}, }; CheckResult result = check(R"( @@ -515,7 +524,7 @@ TEST_CASE_FIXTURE(Fixture, "widening_happens_almost_everywhere_except_for_tables {"LuauParseSingletonTypes", true}, {"LuauSingletonTypes", true}, {"LuauDiscriminableUnions2", true}, - {"LuauWidenIfSupertypeIsFree", true}, + {"LuauWidenIfSupertypeIsFree2", true}, }; CheckResult result = check(R"( @@ -544,7 +553,7 @@ TEST_CASE_FIXTURE(Fixture, "table_insert_with_a_singleton_argument") ScopedFastFlag sff[]{ {"LuauParseSingletonTypes", true}, {"LuauSingletonTypes", true}, - {"LuauWidenIfSupertypeIsFree", true}, + {"LuauWidenIfSupertypeIsFree2", true}, }; CheckResult result = check(R"( @@ -565,4 +574,97 @@ TEST_CASE_FIXTURE(Fixture, "table_insert_with_a_singleton_argument") CHECK_EQ("{string}", toString(requireType("t"))); } +TEST_CASE_FIXTURE(Fixture, "functions_are_not_to_be_widened") +{ + ScopedFastFlag sff[]{ + {"LuauParseSingletonTypes", true}, + {"LuauSingletonTypes", true}, + {"LuauWidenIfSupertypeIsFree2", true}, + }; + + CheckResult result = check(R"( + local function foo(my_enum: "A" | "B") end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ(R"(("A" | "B") -> ())", toString(requireType("foo"))); +} + +TEST_CASE_FIXTURE(Fixture, "indexing_on_string_singletons") +{ + ScopedFastFlag sff[]{ + {"LuauDiscriminableUnions2", true}, + {"LuauSingletonTypes", true}, + }; + + CheckResult result = check(R"( + local a: string = "hi" + if a == "hi" then + local x = a:byte() + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ(R"("hi")", toString(requireTypeAtPosition({3, 22}))); +} + +TEST_CASE_FIXTURE(Fixture, "indexing_on_union_of_string_singletons") +{ + ScopedFastFlag sff[]{ + {"LuauDiscriminableUnions2", true}, + {"LuauSingletonTypes", true}, + }; + + CheckResult result = check(R"( + local a: string = "hi" + if a == "hi" or a == "bye" then + local x = a:byte() + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ(R"("bye" | "hi")", toString(requireTypeAtPosition({3, 22}))); +} + +TEST_CASE_FIXTURE(Fixture, "taking_the_length_of_string_singleton") +{ + ScopedFastFlag sff[]{ + {"LuauDiscriminableUnions2", true}, + {"LuauSingletonTypes", true}, + }; + + CheckResult result = check(R"( + local a: string = "hi" + if a == "hi" then + local x = #a + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ(R"("hi")", toString(requireTypeAtPosition({3, 23}))); +} + +TEST_CASE_FIXTURE(Fixture, "taking_the_length_of_union_of_string_singleton") +{ + ScopedFastFlag sff[]{ + {"LuauDiscriminableUnions2", true}, + {"LuauSingletonTypes", true}, + }; + + CheckResult result = check(R"( + local a: string = "hi" + if a == "hi" or a == "bye" then + local x = #a + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ(R"("bye" | "hi")", toString(requireTypeAtPosition({3, 23}))); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.tables.test.cpp b/tests/TypeInfer.tables.test.cpp index a5eba5df..91140aaa 100644 --- a/tests/TypeInfer.tables.test.cpp +++ b/tests/TypeInfer.tables.test.cpp @@ -2384,4 +2384,504 @@ _ = (_.cos) LUAU_REQUIRE_ERRORS(result); } +TEST_CASE_FIXTURE(Fixture, "cannot_call_tables") +{ + CheckResult result = check("local foo = {} foo()"); + LUAU_REQUIRE_ERROR_COUNT(1, result); + + CHECK(get(result.errors[0]) != nullptr); +} + +TEST_CASE_FIXTURE(Fixture, "table_length") +{ + CheckResult result = check(R"( + local t = {} + local s = #t + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK(nullptr != get(requireType("t"))); + CHECK_EQ(*typeChecker.numberType, *requireType("s")); +} + +TEST_CASE_FIXTURE(Fixture, "nil_assign_doesnt_hit_indexer") +{ + CheckResult result = check("local a = {} a[0] = 7 a[0] = nil"); + LUAU_REQUIRE_ERROR_COUNT(0, result); +} + +TEST_CASE_FIXTURE(Fixture, "wrong_assign_does_hit_indexer") +{ + CheckResult result = check("local a = {} a[0] = 7 a[0] = 't'"); + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ(result.errors[0], (TypeError{Location{Position{0, 30}, Position{0, 33}}, TypeMismatch{ + typeChecker.numberType, + typeChecker.stringType, + }})); +} + +TEST_CASE_FIXTURE(Fixture, "nil_assign_doesnt_hit_no_indexer") +{ + CheckResult result = check("local a = {a=1, b=2} a['a'] = nil"); + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ(result.errors[0], (TypeError{Location{Position{0, 30}, Position{0, 33}}, TypeMismatch{ + typeChecker.numberType, + typeChecker.nilType, + }})); +} + +TEST_CASE_FIXTURE(Fixture, "free_rhs_table_can_also_be_bound") +{ + check(R"( + local o + local v = o:i() + + function g(u) + v = u + end + + o:f(g) + o:h() + o:h() + )"); +} + +TEST_CASE_FIXTURE(Fixture, "table_unifies_into_map") +{ + CheckResult result = check(R"( + local Instance: any + local UDim2: any + + function Create(instanceType) + return function(data) + local obj = Instance.new(instanceType) + for k, v in pairs(data) do + if type(k) == 'number' then + --v.Parent = obj + else + obj[k] = v + end + end + return obj + end + end + + local topbarShadow = Create'ImageLabel'{ + Name = "TopBarShadow"; + Size = UDim2.new(1, 0, 0, 3); + Position = UDim2.new(0, 0, 1, 0); + Image = "rbxasset://textures/ui/TopBar/dropshadow.png"; + BackgroundTransparency = 1; + Active = false; + Visible = false; + }; + + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "tables_get_names_from_their_locals") +{ + CheckResult result = check(R"( + local T = {} + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ("T", toString(requireType("T"))); +} + +TEST_CASE_FIXTURE(Fixture, "generalize_table_argument") +{ + CheckResult result = check(R"( + function foo(arr) + local work = {} + for i = 1, #arr do + work[i] = arr[i] + end + + return arr + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + dumpErrors(result); + + const FunctionTypeVar* fooType = get(requireType("foo")); + REQUIRE(fooType); + + std::optional fooArg1 = first(fooType->argTypes); + REQUIRE(fooArg1); + + const TableTypeVar* fooArg1Table = get(*fooArg1); + REQUIRE(fooArg1Table); + + CHECK_EQ(fooArg1Table->state, TableState::Generic); +} + +/* + * This test case exposed an oversight in the treatment of free tables. + * Free tables, like free TypeVars, need to record the scope depth where they were created so that + * we do not erroneously let-generalize them when they are used in a nested lambda. + * + * For more information about let-generalization, see + * + * The important idea here is that the return type of Counter.new is a table with some metatable. + * That metatable *must* be the same TypeVar as the type of Counter. If it is a copy (produced by + * the generalization process), then it loses the knowledge that its metatable will have an :incr() + * method. + */ +TEST_CASE_FIXTURE(Fixture, "dont_quantify_table_that_belongs_to_outer_scope") +{ + CheckResult result = check(R"( + local Counter = {} + Counter.__index = Counter + + function Counter.new() + local self = setmetatable({count=0}, Counter) + return self + end + + function Counter:incr() + self.count = 1 + return self.count + end + + local self = Counter.new() + print(self:incr()) + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + TableTypeVar* counterType = getMutable(requireType("Counter")); + REQUIRE(counterType); + + const FunctionTypeVar* newType = get(follow(counterType->props["new"].type)); + REQUIRE(newType); + + std::optional newRetType = *first(newType->retType); + REQUIRE(newRetType); + + const MetatableTypeVar* newRet = get(follow(*newRetType)); + REQUIRE(newRet); + + const TableTypeVar* newRetMeta = get(newRet->metatable); + REQUIRE(newRetMeta); + + CHECK(newRetMeta->props.count("incr")); + CHECK_EQ(follow(newRet->metatable), follow(requireType("Counter"))); +} + +// TODO: CLI-39624 +TEST_CASE_FIXTURE(Fixture, "instantiate_tables_at_scope_level") +{ + CheckResult result = check(R"( + --!strict + local Option = {} + Option.__index = Option + function Option.Is(obj) + return (type(obj) == "table" and getmetatable(obj) == Option) + end + return Option + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "inferring_crazy_table_should_also_be_quick") +{ + CheckResult result = check(R"( + --!strict + function f(U) + U(w:s(an):c()():c():U(s):c():c():U(s):c():U(s):cU()):c():U(s):c():U(s):c():c():U(s):c():U(s):cU() + end + )"); + + ModulePtr module = getMainModule(); + CHECK_GE(100, module->internalTypes.typeVars.size()); +} + +TEST_CASE_FIXTURE(Fixture, "MixedPropertiesAndIndexers") +{ + CheckResult result = check(R"( +local x = {} +x.a = "a" +x[0] = true +x.b = 37 +)"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "setmetatable_cant_be_used_to_mutate_global_types") +{ + { + Fixture fix; + + // inherit env from parent fixture checker + fix.typeChecker.globalScope = typeChecker.globalScope; + + fix.check(R"( +--!nonstrict +type MT = typeof(setmetatable) +function wtf(arg: {MT}): typeof(table) + arg = wtf(arg) +end +)"); + } + + // validate sharedEnv post-typecheck; valuable for debugging some typeck crashes but slows fuzzing down + // note: it's important for typeck to be destroyed at this point! + { + for (auto& p : typeChecker.globalScope->bindings) + { + toString(p.second.typeId); // toString walks the entire type, making sure ASAN catches access to destroyed type arenas + } + } +} + +TEST_CASE_FIXTURE(Fixture, "evil_table_unification") +{ + // this code re-infers the type of _ while processing fields of _, which can cause use-after-free + check(R"( +--!nonstrict +_ = ... +_:table(_,string)[_:gsub(_,...,n0)],_,_:gsub(_,string)[""],_:split(_,...,table)._,n0 = nil +do end +)"); +} + +TEST_CASE_FIXTURE(Fixture, "dont_crash_when_setmetatable_does_not_produce_a_metatabletypevar") +{ + CheckResult result = check("local x = setmetatable({})"); + LUAU_REQUIRE_ERROR_COUNT(1, result); +} + +TEST_CASE_FIXTURE(Fixture, "instantiate_table_cloning") +{ + CheckResult result = check(R"( +--!nonstrict +local l0:any,l61:t0 = _,math +while _ do +_() +end +function _():t0 +end +type t0 = any +)"); + + std::optional ty = requireType("math"); + REQUIRE(ty); + + const TableTypeVar* ttv = get(*ty); + REQUIRE(ttv); + CHECK(ttv->instantiatedTypeParams.empty()); +} + +TEST_CASE_FIXTURE(Fixture, "instantiate_table_cloning_2") +{ + ScopedFastFlag sff{"LuauOnlyMutateInstantiatedTables", true}; + + CheckResult result = check(R"( +type X = T +type K = X +)"); + + LUAU_REQUIRE_NO_ERRORS(result); + + std::optional ty = requireType("math"); + REQUIRE(ty); + + const TableTypeVar* ttv = get(*ty); + REQUIRE(ttv); + CHECK(ttv->instantiatedTypeParams.empty()); +} + +TEST_CASE_FIXTURE(Fixture, "instantiate_table_cloning_3") +{ + ScopedFastFlag sff{"LuauOnlyMutateInstantiatedTables", true}; + + CheckResult result = check(R"( +type X = T +local a = {} +a.x = 4 +local b: X +a.y = 5 +local c: X +c = b +)"); + + LUAU_REQUIRE_NO_ERRORS(result); + + std::optional ty = requireType("a"); + REQUIRE(ty); + + const TableTypeVar* ttv = get(*ty); + REQUIRE(ttv); + CHECK(ttv->instantiatedTypeParams.empty()); +} + +TEST_CASE_FIXTURE(Fixture, "table_indexing_error_location") +{ + CheckResult result = check(R"( +local foo = {42} +local bar: number? +local baz = foo[bar] + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + CHECK_EQ(result.errors[0].location, Location{Position{3, 16}, Position{3, 19}}); +} + +TEST_CASE_FIXTURE(Fixture, "table_simple_call") +{ + CheckResult result = check(R"( +local a = setmetatable({ x = 2 }, { + __call = function(self) + return (self.x :: number) * 2 -- should work without annotation in the future + end +}) +local b = a() +local c = a(2) -- too many arguments + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ("Argument count mismatch. Function expects 1 argument, but 2 are specified", toString(result.errors[0])); +} + +TEST_CASE_FIXTURE(Fixture, "access_index_metamethod_that_returns_variadic") +{ + CheckResult result = check(R"( + type Foo = {x: string} + local t = {} + setmetatable(t, { + __index = function(x: string): ...Foo + return {x = x} + end + }) + + local foo = t.bar + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + ToStringOptions o; + o.exhaustive = true; + CHECK_EQ("{| x: string |}", toString(requireType("foo"), o)); +} + +TEST_CASE_FIXTURE(Fixture, "dont_invalidate_the_properties_iterator_of_free_table_when_rolled_back") +{ + fileResolver.source["Module/Backend/Types"] = R"( + export type Fiber = { + return_: Fiber? + } + return {} + )"; + + fileResolver.source["Module/Backend"] = R"( + local Types = require(script.Types) + type Fiber = Types.Fiber + type ReactRenderer = { findFiberByHostInstance: () -> Fiber? } + + local function attach(renderer): () + local function getPrimaryFiber(fiber) + local alternate = fiber.alternate + return fiber + end + + local function getFiberIDForNative() + local fiber = renderer.findFiberByHostInstance() + fiber = fiber.return_ + return getPrimaryFiber(fiber) + end + end + + function culprit(renderer: ReactRenderer): () + attach(renderer) + end + + return culprit + )"; + + CheckResult result = frontend.check("Module/Backend"); +} + +TEST_CASE_FIXTURE(Fixture, "checked_prop_too_early") +{ + CheckResult result = check(R"( + local t: {x: number?}? = {x = nil} + local u = t.x and t or 5 + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ("Value of type '{| x: number? |}?' could be nil", toString(result.errors[0])); + CHECK_EQ("number | {| x: number? |}", toString(requireType("u"))); +} + +TEST_CASE_FIXTURE(Fixture, "accidentally_checked_prop_in_opposite_branch") +{ + CheckResult result = check(R"( + local t: {x: number?}? = {x = nil} + local u = t and t.x == 5 or t.x == 31337 + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ("Value of type '{| x: number? |}?' could be nil", toString(result.errors[0])); + CHECK_EQ("boolean", toString(requireType("u"))); +} + +/* + * We had an issue where part of the type of pairs() was an unsealed table. + * This test depends on FFlagDebugLuauFreezeArena to trigger it. + */ +TEST_CASE_FIXTURE(Fixture, "pairs_parameters_are_not_unsealed_tables") +{ + check(R"( + function _(l0:{n0:any}) + _ = pairs + end + )"); +} + +TEST_CASE_FIXTURE(Fixture, "table_function_check_use_after_free") +{ + CheckResult result = check(R"( +local t = {} + +function t.x(value) + for k,v in pairs(t) do end +end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +/* + * When we add new properties to an unsealed table, we should do a level check and promote the property type to be at + * the level of the table. + */ +TEST_CASE_FIXTURE(Fixture, "inferred_properties_of_a_table_should_start_with_the_same_TypeLevel_of_that_table") +{ + CheckResult result = check(R"( + --!strict + local T = {} + + local function f(prop) + T[1] = { + prop = prop, + } + end + + local function g() + local l = T[1].prop + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.test.cpp b/tests/TypeInfer.test.cpp index d7bbad20..571d0f8d 100644 --- a/tests/TypeInfer.test.cpp +++ b/tests/TypeInfer.test.cpp @@ -60,15 +60,6 @@ TEST_CASE_FIXTURE(Fixture, "tc_error_2") }})); } -TEST_CASE_FIXTURE(Fixture, "tc_function") -{ - CheckResult result = check("function five() return 5 end"); - LUAU_REQUIRE_NO_ERRORS(result); - - const FunctionTypeVar* fiveType = get(requireType("five")); - REQUIRE(fiveType != nullptr); -} - TEST_CASE_FIXTURE(Fixture, "infer_locals_with_nil_value") { CheckResult result = check("local f = nil; f = 'hello world'"); @@ -108,462 +99,12 @@ TEST_CASE_FIXTURE(Fixture, "infer_in_nocheck_mode") LUAU_REQUIRE_NO_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "check_function_bodies") -{ - CheckResult result = check("function myFunction() local a = 0 a = true end"); - LUAU_REQUIRE_ERROR_COUNT(1, result); - - CHECK_EQ(result.errors[0], (TypeError{Location{Position{0, 44}, Position{0, 48}}, TypeMismatch{ - typeChecker.numberType, - typeChecker.booleanType, - }})); -} - -TEST_CASE_FIXTURE(Fixture, "infer_return_type") -{ - CheckResult result = check("function take_five() return 5 end"); - LUAU_REQUIRE_NO_ERRORS(result); - - const FunctionTypeVar* takeFiveType = get(requireType("take_five")); - REQUIRE(takeFiveType != nullptr); - - std::vector retVec = flatten(takeFiveType->retType).first; - REQUIRE(!retVec.empty()); - - REQUIRE_EQ(*follow(retVec[0]), *typeChecker.numberType); -} - -TEST_CASE_FIXTURE(Fixture, "infer_from_function_return_type") -{ - CheckResult result = check("function take_five() return 5 end local five = take_five()"); - LUAU_REQUIRE_NO_ERRORS(result); - - CHECK_EQ(*typeChecker.numberType, *follow(requireType("five"))); -} - -TEST_CASE_FIXTURE(Fixture, "cannot_call_primitives") -{ - CheckResult result = check("local foo = 5 foo()"); - LUAU_REQUIRE_ERROR_COUNT(1, result); - - REQUIRE(get(result.errors[0]) != nullptr); -} - -TEST_CASE_FIXTURE(Fixture, "cannot_call_tables") -{ - CheckResult result = check("local foo = {} foo()"); - LUAU_REQUIRE_ERROR_COUNT(1, result); - - CHECK(get(result.errors[0]) != nullptr); -} - -TEST_CASE_FIXTURE(Fixture, "infer_that_function_does_not_return_a_table") -{ - CheckResult result = check(R"( - function take_five() - return 5 - end - - take_five().prop = 888 - )"); - - LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK_EQ(result.errors[0], (TypeError{Location{Position{5, 8}, Position{5, 24}}, NotATable{typeChecker.numberType}})); -} - TEST_CASE_FIXTURE(Fixture, "expr_statement") { CheckResult result = check("local foo = 5 foo()"); LUAU_REQUIRE_ERROR_COUNT(1, result); } -TEST_CASE_FIXTURE(Fixture, "generic_function") -{ - CheckResult result = check(R"( - function id(x) return x end - local a = id(55) - local b = id(nil) - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - - CHECK_EQ(*typeChecker.numberType, *requireType("a")); - CHECK_EQ(*typeChecker.nilType, *requireType("b")); -} - -TEST_CASE_FIXTURE(Fixture, "vararg_functions_should_allow_calls_of_any_types_and_size") -{ - CheckResult result = check(R"( - function f(...) end - - f(1) - f("foo", 2) - )"); - - LUAU_REQUIRE_NO_ERRORS(result); -} - -TEST_CASE_FIXTURE(Fixture, "vararg_function_is_quantified") -{ - CheckResult result = check(R"( - local T = {} - function T.f(...) - local result = {} - - for i = 1, select("#", ...) do - local dictionary = select(i, ...) - for key, value in pairs(dictionary) do - result[key] = value - end - end - - return result - end - - return T - )"); - - auto r = first(getMainModule()->getModuleScope()->returnType); - REQUIRE(r); - - TableTypeVar* ttv = getMutable(*r); - REQUIRE(ttv); - - TypeId k = ttv->props["f"].type; - REQUIRE(k); - - LUAU_REQUIRE_NO_ERRORS(result); -} - -TEST_CASE_FIXTURE(Fixture, "for_loop") -{ - CheckResult result = check(R"( - local q - for i=0, 50, 2 do - q = i - end - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - - CHECK_EQ(*typeChecker.numberType, *requireType("q")); -} - -TEST_CASE_FIXTURE(Fixture, "for_in_loop") -{ - CheckResult result = check(R"( - local n - local s - for i, v in pairs({ "foo" }) do - n = i - s = v - end - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - - CHECK_EQ(*typeChecker.numberType, *requireType("n")); - CHECK_EQ(*typeChecker.stringType, *requireType("s")); -} - -TEST_CASE_FIXTURE(Fixture, "for_in_loop_with_next") -{ - CheckResult result = check(R"( - local n - local s - for i, v in next, { "foo", "bar" } do - n = i - s = v - end - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - - CHECK_EQ(*typeChecker.numberType, *requireType("n")); - CHECK_EQ(*typeChecker.stringType, *requireType("s")); -} - -TEST_CASE_FIXTURE(Fixture, "for_in_with_an_iterator_of_type_any") -{ - CheckResult result = check(R"( - local it: any - local a, b - for i, v in it do - a, b = i, v - end - )"); - - LUAU_REQUIRE_NO_ERRORS(result); -} - -TEST_CASE_FIXTURE(Fixture, "for_in_loop_should_fail_with_non_function_iterator") -{ - CheckResult result = check(R"( - local foo = "bar" - for i, v in foo do - end - )"); - - LUAU_REQUIRE_ERROR_COUNT(1, result); -} - -TEST_CASE_FIXTURE(Fixture, "for_in_with_just_one_iterator_is_ok") -{ - CheckResult result = check(R"( - local function keys(dictionary) - local new = {} - local index = 1 - - for key in pairs(dictionary) do - new[index] = key - index = index + 1 - end - - return new - end - )"); - - LUAU_REQUIRE_NO_ERRORS(result); -} - -TEST_CASE_FIXTURE(Fixture, "for_in_with_a_custom_iterator_should_type_check") -{ - CheckResult result = check(R"( - local function range(l, h): () -> number - return function() - return l - end - end - - for n: string in range(1, 10) do - print(n) - end - )"); - - LUAU_REQUIRE_ERROR_COUNT(1, result); -} - -TEST_CASE_FIXTURE(Fixture, "for_in_loop_on_error") -{ - CheckResult result = check(R"( - function f(x) - gobble.prop = x.otherprop - end - - local p - for _, part in i_am_not_defined do - p = part - f(part) - part.thirdprop = false - end - )"); - - CHECK_EQ(2, result.errors.size()); - - TypeId p = requireType("p"); - CHECK_EQ("*unknown*", toString(p)); -} - -TEST_CASE_FIXTURE(Fixture, "for_in_loop_on_non_function") -{ - CheckResult result = check(R"( - local bad_iter = 5 - - for a in bad_iter() do - end - )"); - - LUAU_REQUIRE_ERROR_COUNT(1, result); - - REQUIRE(get(result.errors[0])); -} - -TEST_CASE_FIXTURE(Fixture, "for_in_loop_error_on_factory_not_returning_the_right_amount_of_values") -{ - CheckResult result = check(R"( - local function hasDivisors(value: number, table) - return false - end - - function prime_iter(state, index) - while hasDivisors(index, state) do - index += 1 - end - - state[index] = true - return index - end - - function primes1() - return prime_iter, {} - end - - function primes2() - return prime_iter, {}, "" - end - - function primes3() - return prime_iter, {}, 2 - end - - for p in primes1() do print(p) end -- mismatch in argument count - - for p in primes2() do print(p) end -- mismatch in argument types, prime_iter takes {}, number, we are given {}, string - - for p in primes3() do print(p) end -- no error - )"); - - LUAU_REQUIRE_ERROR_COUNT(2, result); - - CountMismatch* acm = get(result.errors[0]); - REQUIRE(acm); - CHECK_EQ(acm->context, CountMismatch::Arg); - CHECK_EQ(2, acm->expected); - CHECK_EQ(1, acm->actual); - - TypeMismatch* tm = get(result.errors[1]); - REQUIRE(tm); - CHECK_EQ(typeChecker.numberType, tm->wantedType); - CHECK_EQ(typeChecker.stringType, tm->givenType); -} - -TEST_CASE_FIXTURE(Fixture, "for_in_loop_error_on_iterator_requiring_args_but_none_given") -{ - CheckResult result = check(R"( - function prime_iter(state, index) - return 1 - end - - for p in prime_iter do print(p) end - )"); - - LUAU_REQUIRE_ERROR_COUNT(1, result); - - CountMismatch* acm = get(result.errors[0]); - REQUIRE(acm); - CHECK_EQ(acm->context, CountMismatch::Arg); - CHECK_EQ(2, acm->expected); - CHECK_EQ(0, acm->actual); -} - -TEST_CASE_FIXTURE(Fixture, "for_in_loop_iterator_returns_any") -{ - CheckResult result = check(R"( - function bar(): any - return true - end - - local a - for b in bar do - a = b - end - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - - CHECK_EQ(typeChecker.anyType, requireType("a")); -} - -TEST_CASE_FIXTURE(Fixture, "for_in_loop_iterator_returns_any2") -{ - CheckResult result = check(R"( - function bar(): any - return true - end - - local a - for b in bar() do - a = b - end - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - - CHECK_EQ("any", toString(requireType("a"))); -} - -TEST_CASE_FIXTURE(Fixture, "for_in_loop_iterator_is_any") -{ - CheckResult result = check(R"( - local bar: any - - local a - for b in bar do - a = b - end - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - - CHECK_EQ("any", toString(requireType("a"))); -} - -TEST_CASE_FIXTURE(Fixture, "for_in_loop_iterator_is_any2") -{ - CheckResult result = check(R"( - local bar: any - - local a - for b in bar() do - a = b - end - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - - CHECK_EQ("any", toString(requireType("a"))); -} - -TEST_CASE_FIXTURE(Fixture, "for_in_loop_iterator_is_error") -{ - CheckResult result = check(R"( - local a - for b in bar do - a = b - end - )"); - - LUAU_REQUIRE_ERROR_COUNT(1, result); - - CHECK_EQ("*unknown*", toString(requireType("a"))); -} - -TEST_CASE_FIXTURE(Fixture, "for_in_loop_iterator_is_error2") -{ - CheckResult result = check(R"( - function bar(c) return c end - - local a - for b in bar() do - a = b - end - )"); - - LUAU_REQUIRE_ERROR_COUNT(1, result); - - CHECK_EQ("*unknown*", toString(requireType("a"))); -} - -TEST_CASE_FIXTURE(Fixture, "for_in_loop_with_custom_iterator") -{ - CheckResult result = check(R"( - function primes() - return function (state: number) end, 2 - end - - for p, q in primes do - q = "" - end - )"); - - LUAU_REQUIRE_ERROR_COUNT(1, result); - - TypeMismatch* tm = get(result.errors[0]); - REQUIRE(tm); - CHECK_EQ(typeChecker.numberType, tm->wantedType); - CHECK_EQ(typeChecker.stringType, tm->givenType); -} - TEST_CASE_FIXTURE(Fixture, "if_statement") { CheckResult result = check(R"( @@ -583,474 +124,6 @@ TEST_CASE_FIXTURE(Fixture, "if_statement") CHECK_EQ(*typeChecker.numberType, *requireType("b")); } -TEST_CASE_FIXTURE(Fixture, "while_loop") -{ - CheckResult result = check(R"( - local i - while true do - i = 8 - end - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - - CHECK_EQ(*typeChecker.numberType, *requireType("i")); -} - -TEST_CASE_FIXTURE(Fixture, "repeat_loop") -{ - CheckResult result = check(R"( - local i - repeat - i = 'hi' - until true - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - - CHECK_EQ(*typeChecker.stringType, *requireType("i")); -} - -TEST_CASE_FIXTURE(Fixture, "repeat_loop_condition_binds_to_its_block") -{ - CheckResult result = check(R"( - repeat - local x = true - until x - )"); - - LUAU_REQUIRE_NO_ERRORS(result); -} - -TEST_CASE_FIXTURE(Fixture, "symbols_in_repeat_block_should_not_be_visible_beyond_until_condition") -{ - CheckResult result = check(R"( - repeat - local x = true - until x - - print(x) - )"); - - LUAU_REQUIRE_ERROR_COUNT(1, result); -} - -TEST_CASE_FIXTURE(Fixture, "table_length") -{ - CheckResult result = check(R"( - local t = {} - local s = #t - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - - CHECK(nullptr != get(requireType("t"))); - CHECK_EQ(*typeChecker.numberType, *requireType("s")); -} - -TEST_CASE_FIXTURE(Fixture, "string_length") -{ - CheckResult result = check(R"( - local s = "Hello, World!" - local t = #s - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - CHECK_EQ("number", toString(requireType("t"))); -} - -TEST_CASE_FIXTURE(Fixture, "string_index") -{ - CheckResult result = check(R"( - local s = "Hello, World!" - local t = s[4] - )"); - - LUAU_REQUIRE_ERROR_COUNT(1, result); - - NotATable* nat = get(result.errors[0]); - REQUIRE(nat); - CHECK_EQ("string", toString(nat->ty)); - - CHECK_EQ("*unknown*", toString(requireType("t"))); -} - -TEST_CASE_FIXTURE(Fixture, "length_of_error_type_does_not_produce_an_error") -{ - CheckResult result = check(R"( - local l = #this_is_not_defined - )"); - - LUAU_REQUIRE_ERROR_COUNT(1, result); -} - -TEST_CASE_FIXTURE(Fixture, "indexing_error_type_does_not_produce_an_error") -{ - CheckResult result = check(R"( - local originalReward = unknown.Parent.Reward:GetChildren()[1] - )"); - - LUAU_REQUIRE_ERROR_COUNT(1, result); -} - -TEST_CASE_FIXTURE(Fixture, "nil_assign_doesnt_hit_indexer") -{ - CheckResult result = check("local a = {} a[0] = 7 a[0] = nil"); - LUAU_REQUIRE_ERROR_COUNT(0, result); -} - -TEST_CASE_FIXTURE(Fixture, "wrong_assign_does_hit_indexer") -{ - CheckResult result = check("local a = {} a[0] = 7 a[0] = 't'"); - LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK_EQ(result.errors[0], (TypeError{Location{Position{0, 30}, Position{0, 33}}, TypeMismatch{ - typeChecker.numberType, - typeChecker.stringType, - }})); -} - -TEST_CASE_FIXTURE(Fixture, "nil_assign_doesnt_hit_no_indexer") -{ - CheckResult result = check("local a = {a=1, b=2} a['a'] = nil"); - LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK_EQ(result.errors[0], (TypeError{Location{Position{0, 30}, Position{0, 33}}, TypeMismatch{ - typeChecker.numberType, - typeChecker.nilType, - }})); -} - -TEST_CASE_FIXTURE(Fixture, "dot_on_error_type_does_not_produce_an_error") -{ - CheckResult result = check(R"( - local foo = (true).x - foo.x = foo.y - )"); - - LUAU_REQUIRE_ERROR_COUNT(1, result); -} - -TEST_CASE_FIXTURE(Fixture, "dont_suggest_using_colon_rather_than_dot_if_not_defined_with_colon") -{ - CheckResult result = check(R"( - local someTable = {} - - someTable.Function1 = function(Arg1) - end - - someTable.Function1() -- Argument count mismatch - )"); - - LUAU_REQUIRE_ERROR_COUNT(1, result); - - REQUIRE(get(result.errors[0])); -} - -TEST_CASE_FIXTURE(Fixture, "dont_suggest_using_colon_rather_than_dot_if_it_wont_help_2") -{ - CheckResult result = check(R"( - local someTable = {} - - someTable.Function2 = function(Arg1, Arg2) - end - - someTable.Function2() -- Argument count mismatch - )"); - - LUAU_REQUIRE_ERROR_COUNT(1, result); - - REQUIRE(get(result.errors[0])); -} - -TEST_CASE_FIXTURE(Fixture, "dont_suggest_using_colon_rather_than_dot_if_another_overload_works") -{ - CheckResult result = check(R"( - type T = {method: ((T, number) -> number) & ((number) -> number)} - local T: T - - T.method(4) - )"); - - LUAU_REQUIRE_NO_ERRORS(result); -} - -TEST_CASE_FIXTURE(Fixture, "list_only_alternative_overloads_that_match_argument_count") -{ - CheckResult result = check(R"( - local multiply: ((number)->number) & ((number)->string) & ((number, number)->number) - multiply("") - )"); - - LUAU_REQUIRE_ERROR_COUNT(2, result); - - TypeMismatch* tm = get(result.errors[0]); - REQUIRE(tm); - CHECK_EQ(typeChecker.numberType, tm->wantedType); - CHECK_EQ(typeChecker.stringType, tm->givenType); - - ExtraInformation* ei = get(result.errors[1]); - REQUIRE(ei); - CHECK_EQ("Other overloads are also not viable: (number) -> string", ei->message); -} - -TEST_CASE_FIXTURE(Fixture, "list_all_overloads_if_no_overload_takes_given_argument_count") -{ - CheckResult result = check(R"( - local multiply: ((number)->number) & ((number)->string) & ((number, number)->number) - multiply() - )"); - - LUAU_REQUIRE_ERROR_COUNT(2, result); - - GenericError* ge = get(result.errors[0]); - REQUIRE(ge); - CHECK_EQ("No overload for function accepts 0 arguments.", ge->message); - - ExtraInformation* ei = get(result.errors[1]); - REQUIRE(ei); - CHECK_EQ("Available overloads: (number) -> number; (number) -> string; and (number, number) -> number", ei->message); -} - -TEST_CASE_FIXTURE(Fixture, "dont_give_other_overloads_message_if_only_one_argument_matching_overload_exists") -{ - CheckResult result = check(R"( - local multiply: ((number)->number) & ((number)->string) & ((number, number)->number) - multiply(1, "") - )"); - - LUAU_REQUIRE_ERROR_COUNT(1, result); - - TypeMismatch* tm = get(result.errors[0]); - REQUIRE(tm); - CHECK_EQ(typeChecker.numberType, tm->wantedType); - CHECK_EQ(typeChecker.stringType, tm->givenType); -} - -TEST_CASE_FIXTURE(Fixture, "infer_return_type_from_selected_overload") -{ - CheckResult result = check(R"( - type T = {method: ((T, number) -> number) & ((number) -> string)} - local T: T - - local a = T.method(T, 4) - local b = T.method(5) - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - CHECK_EQ("number", toString(requireType("a"))); - CHECK_EQ("string", toString(requireType("b"))); -} - -TEST_CASE_FIXTURE(Fixture, "too_many_arguments") -{ - CheckResult result = check(R"( - --!nonstrict - - function g(a: number) end - - g() - - )"); - - LUAU_REQUIRE_ERROR_COUNT(1, result); - - auto err = result.errors[0]; - auto acm = get(err); - REQUIRE(acm); - - CHECK_EQ(1, acm->expected); - CHECK_EQ(0, acm->actual); -} - -TEST_CASE_FIXTURE(Fixture, "any_type_propagates") -{ - CheckResult result = check(R"( - local foo: any - local bar = foo:method("argument") - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - - CHECK_EQ("any", toString(requireType("bar"))); -} - -TEST_CASE_FIXTURE(Fixture, "can_subscript_any") -{ - CheckResult result = check(R"( - local foo: any - local bar = foo[5] - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - - CHECK_EQ("any", toString(requireType("bar"))); -} - -// Not strictly correct: metatables permit overriding this -TEST_CASE_FIXTURE(Fixture, "can_get_length_of_any") -{ - CheckResult result = check(R"( - local foo: any = {} - local bar = #foo - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - - CHECK_EQ(PrimitiveTypeVar::Number, getPrimitiveType(requireType("bar"))); -} - -TEST_CASE_FIXTURE(Fixture, "recursive_function") -{ - CheckResult result = check(R"( - function count(n: number) - if n == 0 then - return 0 - else - return count(n - 1) - end - end - )"); - - LUAU_REQUIRE_NO_ERRORS(result); -} - -TEST_CASE_FIXTURE(Fixture, "lambda_form_of_local_function_cannot_be_recursive") -{ - CheckResult result = check(R"( - local f = function() return f() end - )"); - - LUAU_REQUIRE_ERROR_COUNT(1, result); -} - -TEST_CASE_FIXTURE(Fixture, "recursive_local_function") -{ - CheckResult result = check(R"( - local function count(n: number) - if n == 0 then - return 0 - else - return count(n - 1) - end - end - )"); - - LUAU_REQUIRE_NO_ERRORS(result); -} - -// FIXME: This and the above case get handled very differently. It's pretty dumb. -// We really should unify the two code paths, probably by deleting AstStatFunction. -TEST_CASE_FIXTURE(Fixture, "another_recursive_local_function") -{ - CheckResult result = check(R"( - local count - function count(n: number) - if n == 0 then - return 0 - else - return count(n - 1) - end - end - )"); - - LUAU_REQUIRE_NO_ERRORS(result); -} - -TEST_CASE_FIXTURE(Fixture, "cyclic_function_type_in_rets") -{ - CheckResult result = check(R"( - function f() - return f - end - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - CHECK_EQ("t1 where t1 = () -> t1", toString(requireType("f"))); -} - -TEST_CASE_FIXTURE(Fixture, "cyclic_function_type_in_args") -{ - CheckResult result = check(R"( - function f(g) - return f(f) - end - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - CHECK_EQ("t1 where t1 = (t1) -> ()", toString(requireType("f"))); -} - -// TODO: File a Jira about this -/* -TEST_CASE_FIXTURE(Fixture, "unifying_vararg_pack_with_fixed_length_pack_produces_fixed_length_pack") -{ - CheckResult result = check(R"( - function a(x) return 1 end - a(...) - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - - REQUIRE(bool(getMainModule()->getModuleScope()->varargPack)); - - TypePackId varargPack = *getMainModule()->getModuleScope()->varargPack; - - auto iter = begin(varargPack); - auto endIter = end(varargPack); - - CHECK(iter != endIter); - ++iter; - CHECK(iter == endIter); - - CHECK(!iter.tail()); -} -*/ - -TEST_CASE_FIXTURE(Fixture, "method_depends_on_table") -{ - CheckResult result = check(R"( - -- This catches a bug where x:m didn't count as a use of x - -- so toposort would happily reorder a definition of - -- function x:m before the definition of x. - function g() f() end - local x = {} - function x:m() end - function f() x:m() end - )"); - - LUAU_REQUIRE_NO_ERRORS(result); -} - -TEST_CASE_FIXTURE(Fixture, "another_higher_order_function") -{ - CheckResult result = check(R"( - local Get_des - function Get_des(func) - Get_des(func) - end - - local function f(d) - d:IsA("BasePart") - d.Parent:FindFirstChild("Humanoid") - d:IsA("Decal") - end - Get_des(f) - - )"); - - LUAU_REQUIRE_NO_ERRORS(result); -} - -TEST_CASE_FIXTURE(Fixture, "another_other_higher_order_function") -{ - CheckResult result = check(R"( - local d - d:foo() - d:foo() - )"); - - LUAU_REQUIRE_NO_ERRORS(result); -} - TEST_CASE_FIXTURE(Fixture, "statements_are_topologically_sorted") { CheckResult result = check(R"( @@ -1067,121 +140,6 @@ TEST_CASE_FIXTURE(Fixture, "statements_are_topologically_sorted") dumpErrors(result); } -TEST_CASE_FIXTURE(Fixture, "generic_table_method") -{ - CheckResult result = check(R"( - local T = {} - - function T:bar(i) - return i - end - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - - TypeId tType = requireType("T"); - TableTypeVar* tTable = getMutable(tType); - REQUIRE(tTable != nullptr); - - TypeId barType = tTable->props["bar"].type; - REQUIRE(barType != nullptr); - - const FunctionTypeVar* ftv = get(follow(barType)); - REQUIRE_MESSAGE(ftv != nullptr, "Should be a function: " << *barType); - - std::vector args = flatten(ftv->argTypes).first; - TypeId argType = args.at(1); - - CHECK_MESSAGE(get(argType), "Should be generic: " << *barType); -} - -TEST_CASE_FIXTURE(Fixture, "correctly_instantiate_polymorphic_member_functions") -{ - CheckResult result = check(R"( - local T = {} - - function T:foo() - return T:bar(5) - end - - function T:bar(i) - return i - end - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - dumpErrors(result); - - const TableTypeVar* t = get(requireType("T")); - REQUIRE(t != nullptr); - - std::optional fooProp = get(t->props, "foo"); - REQUIRE(bool(fooProp)); - - const FunctionTypeVar* foo = get(follow(fooProp->type)); - REQUIRE(bool(foo)); - - std::optional ret_ = first(foo->retType); - REQUIRE(bool(ret_)); - TypeId ret = follow(*ret_); - - REQUIRE_EQ(getPrimitiveType(ret), PrimitiveTypeVar::Number); -} - -TEST_CASE_FIXTURE(Fixture, "methods_are_topologically_sorted") -{ - CheckResult result = check(R"( - local T = {} - - function T:foo() - return T:bar(999), T:bar("hi") - end - - function T:bar(i) - return i - end - - local a, b = T:foo() - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - dumpErrors(result); - - CHECK_EQ(PrimitiveTypeVar::Number, getPrimitiveType(requireType("a"))); - CHECK_EQ(PrimitiveTypeVar::String, getPrimitiveType(requireType("b"))); -} - -TEST_CASE_FIXTURE(Fixture, "local_function") -{ - CheckResult result = check(R"( - function f() - return 8 - end - - function g() - local function f() - return 'hello' - end - return f - end - - local h = g() - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - - TypeId h = follow(requireType("h")); - - const FunctionTypeVar* ftv = get(h); - REQUIRE(ftv != nullptr); - - std::optional rt = first(ftv->retType); - REQUIRE(bool(rt)); - - TypeId retType = follow(*rt); - CHECK_EQ(PrimitiveTypeVar::String, getPrimitiveType(retType)); -} - TEST_CASE_FIXTURE(Fixture, "unify_nearly_identical_recursive_types") { CheckResult result = check(R"( @@ -1193,267 +151,10 @@ TEST_CASE_FIXTURE(Fixture, "unify_nearly_identical_recursive_types") o = p )"); -} - -/* - * We had a bug in instantiation where the argument types of 'f' and 'g' would be inferred as - * f {+ method: function(): (t2, T3...) +} - * g {+ method: function({+ method: function(): (t2, T3...) +}): (t5, T6...) +} - * - * The type of 'g' is totally wrong as t2 and t5 should be unified, as should T3 with T6. - * - * The correct unification of the argument to 'g' is - * - * {+ method: function(): (t5, T6...) +} - */ -TEST_CASE_FIXTURE(Fixture, "instantiate_cyclic_generic_function") -{ - auto result = check(R"( - function f(o) - o:method() - end - - function g(o) - f(o) - end - )"); - - TypeId g = requireType("g"); - const FunctionTypeVar* gFun = get(g); - REQUIRE(gFun != nullptr); - - auto optionArg = first(gFun->argTypes); - REQUIRE(bool(optionArg)); - - TypeId arg = follow(*optionArg); - const TableTypeVar* argTable = get(arg); - REQUIRE(argTable != nullptr); - - std::optional methodProp = get(argTable->props, "method"); - REQUIRE(bool(methodProp)); - - const FunctionTypeVar* methodFunction = get(methodProp->type); - REQUIRE(methodFunction != nullptr); - - std::optional methodArg = first(methodFunction->argTypes); - REQUIRE(bool(methodArg)); - - REQUIRE_EQ(follow(*methodArg), follow(arg)); -} - -TEST_CASE_FIXTURE(Fixture, "varlist_declared_by_for_in_loop_should_be_free") -{ - CheckResult result = check(R"( - local T = {} - - function T.f(p) - for i, v in pairs(p) do - T.f(v) - end - end - )"); LUAU_REQUIRE_NO_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "properly_infer_iteratee_is_a_free_table") -{ - // In this case, we cannot know the element type of the table {}. It could be anything. - // We therefore must initially ascribe a free typevar to iter. - CheckResult result = check(R"( - for iter in pairs({}) do - iter:g().p = true - end - )"); - - LUAU_REQUIRE_NO_ERRORS(result); -} - -TEST_CASE_FIXTURE(Fixture, "quantify_methods_defined_using_dot_syntax_and_explicit_self_parameter") -{ - check(R"( - local T = {} - - function T.method(self) - self:method() - end - - function T.method2(self) - self:method() - end - - T:method2() - )"); -} - -TEST_CASE_FIXTURE(Fixture, "free_rhs_table_can_also_be_bound") -{ - check(R"( - local o - local v = o:i() - - function g(u) - v = u - end - - o:f(g) - o:h() - o:h() - )"); -} - -TEST_CASE_FIXTURE(Fixture, "require") -{ - fileResolver.source["game/A"] = R"( - local function hooty(x: number): string - return "Hi there!" - end - - return {hooty=hooty} - )"; - - fileResolver.source["game/B"] = R"( - local Hooty = require(game.A) - - local h -- free! - local i = Hooty.hooty(h) - )"; - - CheckResult aResult = frontend.check("game/A"); - dumpErrors(aResult); - LUAU_REQUIRE_NO_ERRORS(aResult); - - CheckResult bResult = frontend.check("game/B"); - dumpErrors(bResult); - LUAU_REQUIRE_NO_ERRORS(bResult); - - ModulePtr b = frontend.moduleResolver.modules["game/B"]; - - REQUIRE(b != nullptr); - - dumpErrors(bResult); - - std::optional iType = requireType(b, "i"); - REQUIRE_EQ("string", toString(*iType)); - - std::optional hType = requireType(b, "h"); - REQUIRE_EQ("number", toString(*hType)); -} - -TEST_CASE_FIXTURE(Fixture, "require_types") -{ - fileResolver.source["workspace/A"] = R"( - export type Point = {x: number, y: number} - - return {} - )"; - - fileResolver.source["workspace/B"] = R"( - local Hooty = require(workspace.A) - - local h: Hooty.Point - )"; - - CheckResult bResult = frontend.check("workspace/B"); - dumpErrors(bResult); - - ModulePtr b = frontend.moduleResolver.modules["workspace/B"]; - REQUIRE(b != nullptr); - - TypeId hType = requireType(b, "h"); - REQUIRE_MESSAGE(bool(get(hType)), "Expected table but got " << toString(hType)); -} - -TEST_CASE_FIXTURE(Fixture, "require_a_variadic_function") -{ - fileResolver.source["game/A"] = R"( - local T = {} - function T.f(...) end - return T - )"; - - fileResolver.source["game/B"] = R"( - local A = require(game.A) - local f = A.f - )"; - - CheckResult result = frontend.check("game/B"); - - ModulePtr bModule = frontend.moduleResolver.getModule("game/B"); - REQUIRE(bModule != nullptr); - - TypeId f = follow(requireType(bModule, "f")); - - const FunctionTypeVar* ftv = get(f); - REQUIRE(ftv); - - auto iter = begin(ftv->argTypes); - auto endIter = end(ftv->argTypes); - - REQUIRE(iter == endIter); - REQUIRE(iter.tail()); - - CHECK(get(*iter.tail())); -} - -TEST_CASE_FIXTURE(Fixture, "assign_prop_to_table_by_calling_any_yields_any") -{ - CheckResult result = check(R"( - local f: any - local T = {} - - T.prop = f() - - return T - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - - TableTypeVar* ttv = getMutable(requireType("T")); - REQUIRE(ttv); - REQUIRE(ttv->props.count("prop")); - - REQUIRE_EQ("any", toString(ttv->props["prop"].type)); -} - -TEST_CASE_FIXTURE(Fixture, "type_error_of_unknown_qualified_type") -{ - CheckResult result = check(R"( - local p: SomeModule.DoesNotExist - )"); - - LUAU_REQUIRE_ERROR_COUNT(1, result); - - REQUIRE_EQ(result.errors[0], (TypeError{Location{{1, 17}, {1, 40}}, UnknownSymbol{"SomeModule.DoesNotExist"}})); -} - -TEST_CASE_FIXTURE(Fixture, "require_module_that_does_not_export") -{ - const std::string sourceA = R"( - )"; - - const std::string sourceB = R"( - local Hooty = require(script.Parent.A) - )"; - - fileResolver.source["game/Workspace/A"] = sourceA; - fileResolver.source["game/Workspace/B"] = sourceB; - - frontend.check("game/Workspace/A"); - frontend.check("game/Workspace/B"); - - ModulePtr aModule = frontend.moduleResolver.modules["game/Workspace/A"]; - ModulePtr bModule = frontend.moduleResolver.modules["game/Workspace/B"]; - - CHECK(aModule->errors.empty()); - REQUIRE_EQ(1, bModule->errors.size()); - CHECK_MESSAGE(get(bModule->errors[0]), "Should be IllegalRequire: " << toString(bModule->errors[0])); - - auto hootyType = requireType(bModule, "Hooty"); - - CHECK_EQ("*unknown*", toString(hootyType)); -} - TEST_CASE_FIXTURE(Fixture, "warn_on_lowercase_parent_property") { CheckResult result = check(R"( @@ -1468,144 +169,6 @@ TEST_CASE_FIXTURE(Fixture, "warn_on_lowercase_parent_property") REQUIRE_EQ("parent", ed->symbol); } -TEST_CASE_FIXTURE(Fixture, "quantify_any_does_not_bind_to_itself") -{ - CheckResult result = check(R"( - local A : any - function A.B() end - A:C() - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - - TypeId aType = requireType("A"); - CHECK_EQ(aType, typeChecker.anyType); -} - -TEST_CASE_FIXTURE(Fixture, "table_unifies_into_map") -{ - CheckResult result = check(R"( - local Instance: any - local UDim2: any - - function Create(instanceType) - return function(data) - local obj = Instance.new(instanceType) - for k, v in pairs(data) do - if type(k) == 'number' then - --v.Parent = obj - else - obj[k] = v - end - end - return obj - end - end - - local topbarShadow = Create'ImageLabel'{ - Name = "TopBarShadow"; - Size = UDim2.new(1, 0, 0, 3); - Position = UDim2.new(0, 0, 1, 0); - Image = "rbxasset://textures/ui/TopBar/dropshadow.png"; - BackgroundTransparency = 1; - Active = false; - Visible = false; - }; - - )"); - - LUAU_REQUIRE_NO_ERRORS(result); -} - -TEST_CASE_FIXTURE(Fixture, "func_expr_doesnt_leak_free") -{ - CheckResult result = check(R"( - local p = function(x) return x end - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - const Luau::FunctionTypeVar* fn = get(requireType("p")); - REQUIRE(fn); - auto ret = first(fn->retType); - REQUIRE(ret); - REQUIRE(get(follow(*ret))); -} - -TEST_CASE_FIXTURE(Fixture, "instantiate_generic_function_in_assignments") -{ - CheckResult result = check(R"( - function foo(a, b) - return a(b) - end - - function bar() - local c: ((number)->number, number)->number = foo -- no error - c = foo -- no error - local d: ((number)->number, string)->number = foo -- error from arg 2 (string) not being convertable to number from the call a(b) - end - )"); - - LUAU_REQUIRE_ERROR_COUNT(1, result); - - TypeMismatch* tm = get(result.errors[0]); - REQUIRE(tm); - CHECK_EQ("((number) -> number, string) -> number", toString(tm->wantedType)); - CHECK_EQ("((number) -> number, number) -> number", toString(tm->givenType)); -} - -TEST_CASE_FIXTURE(Fixture, "instantiate_generic_function_in_assignments2") -{ - CheckResult result = check(R"( - function foo(a, b) - return a(b) - end - - function bar() - local _: (string, string)->number = foo -- string cannot be converted to (string)->number - end - )"); - - LUAU_REQUIRE_ERROR_COUNT(1, result); - - TypeMismatch* tm = get(result.errors[0]); - REQUIRE(tm); - CHECK_EQ("(string, string) -> number", toString(tm->wantedType)); - CHECK_EQ("((string) -> number, string) -> number", toString(*tm->givenType)); -} - -TEST_CASE_FIXTURE(Fixture, "string_method") -{ - CheckResult result = check(R"( - local p = ("tacos"):len() - )"); - CHECK_EQ(0, result.errors.size()); - - CHECK_EQ(*requireType("p"), *typeChecker.numberType); -} - -TEST_CASE_FIXTURE(Fixture, "string_function_indirect") -{ - CheckResult result = check(R"( - local s:string - local l = s.lower - local p = l(s) - )"); - CHECK_EQ(0, result.errors.size()); - - CHECK_EQ(*requireType("p"), *typeChecker.stringType); -} - -TEST_CASE_FIXTURE(Fixture, "string_function_other") -{ - CheckResult result = check(R"( - local s:string - local p = s:match("foo") - )"); - CHECK_EQ(0, result.errors.size()); - - CHECK_EQ(toString(requireType("p")), "string?"); -} - TEST_CASE_FIXTURE(Fixture, "weird_case") { CheckResult result = check(R"( @@ -1617,80 +180,6 @@ TEST_CASE_FIXTURE(Fixture, "weird_case") dumpErrors(result); } -TEST_CASE_FIXTURE(Fixture, "or_joins_types") -{ - CheckResult result = check(R"( - local s = "a" or 10 - local x:string|number = s - )"); - LUAU_REQUIRE_NO_ERRORS(result); - CHECK_EQ(toString(*requireType("s")), "number | string"); - CHECK_EQ(toString(*requireType("x")), "number | string"); -} - -TEST_CASE_FIXTURE(Fixture, "or_joins_types_with_no_extras") -{ - CheckResult result = check(R"( - local s = "a" or 10 - local x:number|string = s - local y = x or "s" - )"); - CHECK_EQ(0, result.errors.size()); - CHECK_EQ(toString(*requireType("s")), "number | string"); - CHECK_EQ(toString(*requireType("y")), "number | string"); -} - -TEST_CASE_FIXTURE(Fixture, "or_joins_types_with_no_superfluous_union") -{ - CheckResult result = check(R"( - local s = "a" or "b" - local x:string = s - )"); - CHECK_EQ(0, result.errors.size()); - CHECK_EQ(*requireType("s"), *typeChecker.stringType); -} - -TEST_CASE_FIXTURE(Fixture, "and_adds_boolean") -{ - CheckResult result = check(R"( - local s = "a" and 10 - local x:boolean|number = s - )"); - CHECK_EQ(0, result.errors.size()); - CHECK_EQ(toString(*requireType("s")), "boolean | number"); -} - -TEST_CASE_FIXTURE(Fixture, "and_adds_boolean_no_superfluous_union") -{ - CheckResult result = check(R"( - local s = "a" and true - local x:boolean = s - )"); - CHECK_EQ(0, result.errors.size()); - CHECK_EQ(*requireType("x"), *typeChecker.booleanType); -} - -TEST_CASE_FIXTURE(Fixture, "and_or_ternary") -{ - CheckResult result = check(R"( - local s = (1/2) > 0.5 and "a" or 10 - )"); - CHECK_EQ(0, result.errors.size()); - CHECK_EQ(toString(*requireType("s")), "number | string"); -} - -TEST_CASE_FIXTURE(Fixture, "first_argument_can_be_optional") -{ - CheckResult result = check(R"( - local T = {} - function T.new(a: number?, b: number?, c: number?) return 5 end - local m = T.new() - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - dumpErrors(result); -} - TEST_CASE_FIXTURE(Fixture, "dont_ice_when_failing_the_occurs_check") { CheckResult result = check(R"( @@ -1726,303 +215,6 @@ TEST_CASE_FIXTURE(Fixture, "crazy_complexity") } #endif -// We had a bug where a cyclic union caused a stack overflow. -// ex type U = number | U -TEST_CASE_FIXTURE(Fixture, "dont_allow_cyclic_unions_to_be_inferred") -{ - CheckResult result = check(R"( - --!strict - - function f(a, b) - a:g(b or {}) - a:g(b) - end - )"); - - LUAU_REQUIRE_NO_ERRORS(result); -} - -TEST_CASE_FIXTURE(Fixture, "it_is_ok_not_to_supply_enough_retvals") -{ - CheckResult result = check(R"( - function get_two() return 5, 6 end - - local a = get_two() - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - dumpErrors(result); -} - -TEST_CASE_FIXTURE(Fixture, "duplicate_functions2") -{ - CheckResult result = check(R"( - function foo() end - - function bar() - local function foo() end - end - )"); - - LUAU_REQUIRE_ERROR_COUNT(0, result); -} - -TEST_CASE_FIXTURE(Fixture, "duplicate_functions_allowed_in_nonstrict") -{ - CheckResult result = check(R"( - --!nonstrict - function foo() end - - function foo() end - - function bar() - local function foo() end - end - )"); - - LUAU_REQUIRE_NO_ERRORS(result); -} - -TEST_CASE_FIXTURE(Fixture, "duplicate_functions_with_different_signatures_not_allowed_in_nonstrict") -{ - CheckResult result = check(R"( - --!nonstrict - function foo(): number - return 1 - end - foo() - - function foo(n: number): number - return 2 - end - foo() - )"); - - LUAU_REQUIRE_ERROR_COUNT(1, result); - - TypeMismatch* tm = get(result.errors[0]); - REQUIRE(tm); - CHECK_EQ("() -> number", toString(tm->wantedType)); - CHECK_EQ("(number) -> number", toString(tm->givenType)); -} - -TEST_CASE_FIXTURE(Fixture, "tables_get_names_from_their_locals") -{ - CheckResult result = check(R"( - local T = {} - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - - CHECK_EQ("T", toString(requireType("T"))); -} - -TEST_CASE_FIXTURE(Fixture, "generalize_table_argument") -{ - CheckResult result = check(R"( - function foo(arr) - local work = {} - for i = 1, #arr do - work[i] = arr[i] - end - - return arr - end - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - dumpErrors(result); - - const FunctionTypeVar* fooType = get(requireType("foo")); - REQUIRE(fooType); - - std::optional fooArg1 = first(fooType->argTypes); - REQUIRE(fooArg1); - - const TableTypeVar* fooArg1Table = get(*fooArg1); - REQUIRE(fooArg1Table); - - CHECK_EQ(fooArg1Table->state, TableState::Generic); -} - -TEST_CASE_FIXTURE(Fixture, "complicated_return_types_require_an_explicit_annotation") -{ - CheckResult result = check(R"( - local i = 0 - function most_of_the_natural_numbers(): number? - if i < 10 then - i = i + 1 - return i - else - return nil - end - end - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - - const FunctionTypeVar* functionType = get(requireType("most_of_the_natural_numbers")); - - std::optional retType = first(functionType->retType); - REQUIRE(retType); - CHECK(get(*retType)); -} - -TEST_CASE_FIXTURE(Fixture, "infer_higher_order_function") -{ - CheckResult result = check(R"( - function apply(f, x) - return f(x) - end - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - - const FunctionTypeVar* ftv = get(requireType("apply")); - REQUIRE(ftv != nullptr); - - std::vector argVec = flatten(ftv->argTypes).first; - - REQUIRE_EQ(2, argVec.size()); - - const FunctionTypeVar* fType = get(follow(argVec[0])); - REQUIRE(fType != nullptr); - - std::vector fArgs = flatten(fType->argTypes).first; - - TypeId xType = argVec[1]; - - CHECK_EQ(1, fArgs.size()); - CHECK_EQ(xType, fArgs[0]); -} - -TEST_CASE_FIXTURE(Fixture, "higher_order_function_2") -{ - CheckResult result = check(R"( - function bottomupmerge(comp, a, b, left, mid, right) - local i, j = left, mid - for k = left, right do - if i < mid and (j > right or not comp(a[j], a[i])) then - b[k] = a[i] - i = i + 1 - else - b[k] = a[j] - j = j + 1 - end - end - end - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - - const FunctionTypeVar* ftv = get(requireType("bottomupmerge")); - REQUIRE(ftv != nullptr); - - std::vector argVec = flatten(ftv->argTypes).first; - - REQUIRE_EQ(6, argVec.size()); - - const FunctionTypeVar* fType = get(follow(argVec[0])); - REQUIRE(fType != nullptr); -} - -TEST_CASE_FIXTURE(Fixture, "higher_order_function_3") -{ - CheckResult result = check(R"( - function swap(p) - local t = p[0] - p[0] = p[1] - p[1] = t - return nil - end - - function swapTwice(p) - swap(p) - swap(p) - return p - end - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - - const FunctionTypeVar* ftv = get(requireType("swapTwice")); - REQUIRE(ftv != nullptr); - - std::vector argVec = flatten(ftv->argTypes).first; - - REQUIRE_EQ(1, argVec.size()); - - const TableTypeVar* argType = get(follow(argVec[0])); - REQUIRE(argType != nullptr); - - CHECK(bool(argType->indexer)); -} - -TEST_CASE_FIXTURE(Fixture, "higher_order_function_4") -{ - CheckResult result = check(R"( - function bottomupmerge(comp, a, b, left, mid, right) - local i, j = left, mid - for k = left, right do - if i < mid and (j > right or not comp(a[j], a[i])) then - b[k] = a[i] - i = i + 1 - else - b[k] = a[j] - j = j + 1 - end - end - end - - function mergesort(arr, comp) - local work = {} - for i = 1, #arr do - work[i] = arr[i] - end - local width = 1 - while width < #arr do - for i = 1, #arr, 2*width do - bottomupmerge(comp, arr, work, i, math.min(i+width, #arr), math.min(i+2*width-1, #arr)) - end - local temp = work - work = arr - arr = temp - width = width * 2 - end - return arr - end - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - dumpErrors(result); - - /* - * mergesort takes two arguments: an array of some type T and a function that takes two Ts. - * We must assert that these two types are in fact the same type. - * In other words, comp(arr[x], arr[y]) is well-typed. - */ - - const FunctionTypeVar* ftv = get(requireType("mergesort")); - REQUIRE(ftv != nullptr); - - std::vector argVec = flatten(ftv->argTypes).first; - - REQUIRE_EQ(2, argVec.size()); - - const TableTypeVar* arg0 = get(follow(argVec[0])); - REQUIRE(arg0 != nullptr); - REQUIRE(bool(arg0->indexer)); - - const FunctionTypeVar* arg1 = get(follow(argVec[1])); - REQUIRE(arg1 != nullptr); - REQUIRE_EQ(2, size(arg1->argTypes)); - - std::vector arg1Args = flatten(arg1->argTypes).first; - - CHECK_EQ(*arg0->indexer->indexResultType, *arg1Args[0]); - CHECK_EQ(*arg0->indexer->indexResultType, *arg1Args[1]); -} - TEST_CASE_FIXTURE(Fixture, "type_errors_infer_types") { CheckResult result = check(R"( @@ -2046,373 +238,6 @@ TEST_CASE_FIXTURE(Fixture, "type_errors_infer_types") CHECK_EQ("*unknown*", toString(requireType("f"))); } -TEST_CASE_FIXTURE(Fixture, "calling_error_type_yields_error") -{ - CheckResult result = check(R"( - local a = unknown.Parent.Reward.GetChildren() - )"); - - LUAU_REQUIRE_ERROR_COUNT(1, result); - - UnknownSymbol* err = get(result.errors[0]); - REQUIRE(err != nullptr); - - CHECK_EQ("unknown", err->name); - - CHECK_EQ("*unknown*", toString(requireType("a"))); -} - -TEST_CASE_FIXTURE(Fixture, "chain_calling_error_type_yields_error") -{ - CheckResult result = check(R"( - local a = Utility.Create "Foo" {} - )"); - - CHECK_EQ("*unknown*", toString(requireType("a"))); -} - -TEST_CASE_FIXTURE(Fixture, "primitive_arith_no_metatable") -{ - CheckResult result = check(R"( - function add(a: number, b: string) - return a + (tonumber(b) :: number), a .. b - end - local n, s = add(2,"3") - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - - const FunctionTypeVar* functionType = get(requireType("add")); - - std::optional retType = first(functionType->retType); - CHECK_EQ(std::optional(typeChecker.numberType), retType); - CHECK_EQ(requireType("n"), typeChecker.numberType); - CHECK_EQ(requireType("s"), typeChecker.stringType); -} - -TEST_CASE_FIXTURE(Fixture, "primitive_arith_no_metatable_with_follows") -{ - CheckResult result = check(R"( - local PI=3.1415926535897931 - local SOLAR_MASS=4*PI * PI - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - CHECK_EQ(requireType("SOLAR_MASS"), typeChecker.numberType); -} - -TEST_CASE_FIXTURE(Fixture, "primitive_arith_possible_metatable") -{ - CheckResult result = check(R"( - function add(a: number, b: any) - return a + b - end - local t = add(1,2) - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - CHECK_EQ("any", toString(requireType("t"))); -} - -TEST_CASE_FIXTURE(Fixture, "some_primitive_binary_ops") -{ - CheckResult result = check(R"( - local a = 4 + 8 - local b = a + 9 - local s = 'hotdogs' - local t = s .. s - local c = b - a - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - - CHECK_EQ("number", toString(requireType("a"))); - CHECK_EQ("number", toString(requireType("b"))); - CHECK_EQ("string", toString(requireType("s"))); - CHECK_EQ("string", toString(requireType("t"))); - CHECK_EQ("number", toString(requireType("c"))); -} - -TEST_CASE_FIXTURE(Fixture, "typecheck_overloaded_multiply_that_is_an_intersection") -{ - ScopedFastFlag sff{"LuauErrorRecoveryType", true}; - - CheckResult result = check(R"( - --!strict - local Vec3 = {} - Vec3.__index = Vec3 - function Vec3.new() - return setmetatable({x=0, y=0, z=0}, Vec3) - end - - export type Vec3 = typeof(Vec3.new()) - - local thefun: any = function(self, o) return self end - - local multiply: ((Vec3, Vec3) -> Vec3) & ((Vec3, number) -> Vec3) = thefun - - Vec3.__mul = multiply - - local a = Vec3.new() - local b = Vec3.new() - local c = a * b - local d = a * 2 - local e = a * 'cabbage' - )"); - - LUAU_REQUIRE_ERROR_COUNT(1, result); - - CHECK_EQ("Vec3", toString(requireType("a"))); - CHECK_EQ("Vec3", toString(requireType("b"))); - CHECK_EQ("Vec3", toString(requireType("c"))); - CHECK_EQ("Vec3", toString(requireType("d"))); - CHECK_EQ("Vec3", toString(requireType("e"))); -} - -TEST_CASE_FIXTURE(Fixture, "typecheck_overloaded_multiply_that_is_an_intersection_on_rhs") -{ - ScopedFastFlag sff{"LuauErrorRecoveryType", true}; - - CheckResult result = check(R"( - --!strict - local Vec3 = {} - Vec3.__index = Vec3 - function Vec3.new() - return setmetatable({x=0, y=0, z=0}, Vec3) - end - - export type Vec3 = typeof(Vec3.new()) - - local thefun: any = function(self, o) return self end - - local multiply: ((Vec3, Vec3) -> Vec3) & ((Vec3, number) -> Vec3) = thefun - - Vec3.__mul = multiply - - local a = Vec3.new() - local b = Vec3.new() - local c = b * a - local d = 2 * a - local e = 'cabbage' * a - )"); - - LUAU_REQUIRE_ERROR_COUNT(1, result); - - CHECK_EQ("Vec3", toString(requireType("a"))); - CHECK_EQ("Vec3", toString(requireType("b"))); - CHECK_EQ("Vec3", toString(requireType("c"))); - CHECK_EQ("Vec3", toString(requireType("d"))); - CHECK_EQ("Vec3", toString(requireType("e"))); -} - -TEST_CASE_FIXTURE(Fixture, "compare_numbers") -{ - CheckResult result = check(R"( - local a = 441 - local b = 0 - local c = a < b - )"); - - LUAU_REQUIRE_NO_ERRORS(result); -} - -TEST_CASE_FIXTURE(Fixture, "compare_strings") -{ - CheckResult result = check(R"( - local a = '441' - local b = '0' - local c = a < b - )"); - - LUAU_REQUIRE_NO_ERRORS(result); -} - -TEST_CASE_FIXTURE(Fixture, "cannot_indirectly_compare_types_that_do_not_have_a_metatable") -{ - CheckResult result = check(R"( - local a = {} - local b = {} - local c = a < b - )"); - - LUAU_REQUIRE_ERROR_COUNT(1, result); - - GenericError* gen = get(result.errors[0]); - - REQUIRE_EQ(gen->message, "Type a cannot be compared with < because it has no metatable"); -} - -TEST_CASE_FIXTURE(Fixture, "cannot_indirectly_compare_types_that_do_not_offer_overloaded_ordering_operators") -{ - CheckResult result = check(R"( - local M = {} - function M.new() - return setmetatable({}, M) - end - type M = typeof(M.new()) - - local a = M.new() - local b = M.new() - local c = a < b - )"); - - LUAU_REQUIRE_ERROR_COUNT(1, result); - - GenericError* gen = get(result.errors[0]); - REQUIRE(gen != nullptr); - REQUIRE_EQ(gen->message, "Table M does not offer metamethod __lt"); -} - -TEST_CASE_FIXTURE(Fixture, "cannot_compare_tables_that_do_not_have_the_same_metatable") -{ - CheckResult result = check(R"( - --!strict - local M = {} - function M.new() - return setmetatable({}, M) - end - function M.__lt(left, right) return true end - - local a = M.new() - local b = {} - local c = a < b -- line 10 - local d = b < a -- line 11 - )"); - - LUAU_REQUIRE_ERROR_COUNT(2, result); - - REQUIRE_EQ((Location{{10, 18}, {10, 23}}), result.errors[0].location); - - REQUIRE_EQ((Location{{11, 18}, {11, 23}}), result.errors[1].location); -} - -TEST_CASE_FIXTURE(Fixture, "produce_the_correct_error_message_when_comparing_a_table_with_a_metatable_with_one_that_does_not") -{ - CheckResult result = check(R"( - --!strict - local M = {} - function M.new() - return setmetatable({}, M) - end - function M.__lt(left, right) return true end - type M = typeof(M.new()) - - local a = M.new() - local b = {} - local c = a < b -- line 10 - )"); - - LUAU_REQUIRE_ERROR_COUNT(1, result); - - auto err = get(result.errors[0]); - REQUIRE(err != nullptr); - - // Frail. :| - REQUIRE_EQ("Types M and b cannot be compared with < because they do not have the same metatable", err->message); -} - -TEST_CASE_FIXTURE(Fixture, "in_nonstrict_mode_strip_nil_from_intersections_when_considering_relational_operators") -{ - CheckResult result = check(R"( - --!nonstrict - - function maybe_a_number(): number? - return 50 - end - - local a = maybe_a_number() < maybe_a_number() - )"); - - LUAU_REQUIRE_NO_ERRORS(result); -} - -/* - * This test case exposed an oversight in the treatment of free tables. - * Free tables, like free TypeVars, need to record the scope depth where they were created so that - * we do not erroneously let-generalize them when they are used in a nested lambda. - * - * For more information about let-generalization, see - * - * The important idea here is that the return type of Counter.new is a table with some metatable. - * That metatable *must* be the same TypeVar as the type of Counter. If it is a copy (produced by - * the generalization process), then it loses the knowledge that its metatable will have an :incr() - * method. - */ -TEST_CASE_FIXTURE(Fixture, "dont_quantify_table_that_belongs_to_outer_scope") -{ - CheckResult result = check(R"( - local Counter = {} - Counter.__index = Counter - - function Counter.new() - local self = setmetatable({count=0}, Counter) - return self - end - - function Counter:incr() - self.count = 1 - return self.count - end - - local self = Counter.new() - print(self:incr()) - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - - TableTypeVar* counterType = getMutable(requireType("Counter")); - REQUIRE(counterType); - - const FunctionTypeVar* newType = get(follow(counterType->props["new"].type)); - REQUIRE(newType); - - std::optional newRetType = *first(newType->retType); - REQUIRE(newRetType); - - const MetatableTypeVar* newRet = get(follow(*newRetType)); - REQUIRE(newRet); - - const TableTypeVar* newRetMeta = get(newRet->metatable); - REQUIRE(newRetMeta); - - CHECK(newRetMeta->props.count("incr")); - CHECK_EQ(follow(newRet->metatable), follow(requireType("Counter"))); -} - -// TODO: CLI-39624 -TEST_CASE_FIXTURE(Fixture, "instantiate_tables_at_scope_level") -{ - CheckResult result = check(R"( - --!strict - local Option = {} - Option.__index = Option - function Option.Is(obj) - return (type(obj) == "table" and getmetatable(obj) == Option) - end - return Option - )"); - - LUAU_REQUIRE_NO_ERRORS(result); -} - -TEST_CASE_FIXTURE(Fixture, "typeguard_doesnt_leak_to_elseif") -{ - const std::string code = R"( - function f(a) - if type(a) == "boolean" then - local a1 = a - elseif a.fn() then - local a2 = a - else - local a3 = a - end - end - )"; - CheckResult result = check(code); - LUAU_REQUIRE_NO_ERRORS(result); - dumpErrors(result); -} - TEST_CASE_FIXTURE(Fixture, "should_be_able_to_infer_this_without_stack_overflowing") { CheckResult result = check(R"( @@ -2428,47 +253,6 @@ TEST_CASE_FIXTURE(Fixture, "should_be_able_to_infer_this_without_stack_overflowi LUAU_REQUIRE_NO_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "inferring_hundreds_of_self_calls_should_not_suffocate_memory") -{ - CheckResult result = check(R"( - ("foo") - :lower() - :lower() - :lower() - :lower() - :lower() - :lower() - :lower() - :lower() - :lower() - :lower() - :lower() - :lower() - :lower() - :lower() - :lower() - :lower() - :lower() - :lower() - )"); - - ModulePtr module = getMainModule(); - CHECK_GE(50, module->internalTypes.typeVars.size()); -} - -TEST_CASE_FIXTURE(Fixture, "inferring_crazy_table_should_also_be_quick") -{ - CheckResult result = check(R"( - --!strict - function f(U) - U(w:s(an):c()():c():U(s):c():c():U(s):c():U(s):cU()):c():U(s):c():U(s):c():c():U(s):c():U(s):cU() - end - )"); - - ModulePtr module = getMainModule(); - CHECK_GE(100, module->internalTypes.typeVars.size()); -} - TEST_CASE_FIXTURE(Fixture, "exponential_blowup_from_copying_types") { CheckResult result = check(R"( @@ -2507,96 +291,6 @@ TEST_CASE_FIXTURE(Fixture, "exponential_blowup_from_copying_types") CHECK_GE(5, module->interfaceTypes.typeVars.size()); } -TEST_CASE_FIXTURE(Fixture, "mutual_recursion") -{ - CheckResult result = check(R"( - --!strict - - function newPlayerCharacter() - startGui() -- Unknown symbol 'startGui' - end - - local characterAddedConnection: any - function startGui() - characterAddedConnection = game:GetService("Players").LocalPlayer.CharacterAdded:connect(newPlayerCharacter) - end - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - dumpErrors(result); -} - -TEST_CASE_FIXTURE(Fixture, "toposort_doesnt_break_mutual_recursion") -{ - CheckResult result = check(R"( - --!strict - local x = nil - function f() g() end - -- make sure print(x) doesn't get toposorted here, breaking the mutual block - function g() x = f end - print(x) - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - dumpErrors(result); -} - -TEST_CASE_FIXTURE(Fixture, "object_constructor_can_refer_to_method_of_self") -{ - // CLI-30902 - CheckResult result = check(R"( - --!strict - - type Foo = { - fooConn: () -> () | nil - } - - local Foo = {} - Foo.__index = Foo - - function Foo.new() - local self: Foo = { - fooConn = nil, - } - setmetatable(self, Foo) - - self.fooConn = function() - self:method() -- Key 'method' not found in table self - end - - return self - end - - function Foo:method() - print("foo") - end - - local foo = Foo.new() - - -- TODO This is the best our current refinement support can offer :( - local bar = foo.fooConn - if bar then bar() end - - -- foo.fooConn() - )"); - - LUAU_REQUIRE_NO_ERRORS(result); -} - -TEST_CASE_FIXTURE(Fixture, "replace_every_free_type_when_unifying_a_complex_function_with_any") -{ - CheckResult result = check(R"( - local a: any - local b - for _, i in pairs(a) do - b = i - end - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - CHECK_EQ("any", toString(requireType("b"))); -} - // In these tests, a successful parse is required, so we need the parser to return the AST and then we can test the recursion depth limit in type // checker. We also want it to somewhat match up with production values, so we push up the parser recursion limit a little bit instead. TEST_CASE_FIXTURE(Fixture, "check_type_infer_recursion_limit") @@ -2653,146 +347,6 @@ TEST_CASE_FIXTURE(Fixture, "check_expr_recursion_limit") CHECK(nullptr != get(result.errors[0])); } -TEST_CASE_FIXTURE(Fixture, "compound_assign_basic") -{ - CheckResult result = check(R"( - local s = 10 - s += 20 - )"); - CHECK_EQ(0, result.errors.size()); - CHECK_EQ(toString(*requireType("s")), "number"); -} - -TEST_CASE_FIXTURE(Fixture, "compound_assign_mismatch_op") -{ - CheckResult result = check(R"( - local s = 10 - s += true - )"); - LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK_EQ(result.errors[0], (TypeError{Location{{2, 13}, {2, 17}}, TypeMismatch{typeChecker.numberType, typeChecker.booleanType}})); -} - -TEST_CASE_FIXTURE(Fixture, "compound_assign_mismatch_result") -{ - CheckResult result = check(R"( - local s = 'hello' - s += 10 - )"); - - LUAU_REQUIRE_ERROR_COUNT(2, result); - CHECK_EQ(result.errors[0], (TypeError{Location{{2, 8}, {2, 9}}, TypeMismatch{typeChecker.numberType, typeChecker.stringType}})); - CHECK_EQ(result.errors[1], (TypeError{Location{{2, 8}, {2, 15}}, TypeMismatch{typeChecker.stringType, typeChecker.numberType}})); -} - -TEST_CASE_FIXTURE(Fixture, "compound_assign_metatable") -{ - CheckResult result = check(R"( - --!strict - type V2B = { x: number, y: number } - local v2b: V2B = { x = 0, y = 0 } - local VMT = {} - type V2 = typeof(setmetatable(v2b, VMT)) - - function VMT.__add(a: V2, b: V2): V2 - return setmetatable({ x = a.x + b.x, y = a.y + b.y }, VMT) - end - - local v1: V2 = setmetatable({ x = 1, y = 2 }, VMT) - local v2: V2 = setmetatable({ x = 3, y = 4 }, VMT) - v1 += v2 - )"); - CHECK_EQ(0, result.errors.size()); -} - -TEST_CASE_FIXTURE(Fixture, "compound_assign_mismatch_metatable") -{ - CheckResult result = check(R"( - --!strict - type V2B = { x: number, y: number } - local v2b: V2B = { x = 0, y = 0 } - local VMT = {} - type V2 = typeof(setmetatable(v2b, VMT)) - - function VMT.__mod(a: V2, b: V2): number - return a.x * b.x + a.y * b.y - end - - local v1: V2 = setmetatable({ x = 1, y = 2 }, VMT) - local v2: V2 = setmetatable({ x = 3, y = 4 }, VMT) - v1 %= v2 - )"); - LUAU_REQUIRE_ERROR_COUNT(1, result); - - TypeMismatch* tm = get(result.errors[0]); - CHECK_EQ(*tm->wantedType, *requireType("v2")); - CHECK_EQ(*tm->givenType, *typeChecker.numberType); -} - -TEST_CASE_FIXTURE(Fixture, "dont_ice_if_a_TypePack_is_an_error") -{ - CheckResult result = check(R"( - --!strict - function f(s) - print(s) - return f - end - - f("foo")("bar") - )"); -} - -TEST_CASE_FIXTURE(Fixture, "check_function_before_lambda_that_uses_it") -{ - CheckResult result = check(R"( - --!nonstrict - - function f() - return 114 - end - - return function() - return f():andThen() - end - )"); - - LUAU_REQUIRE_NO_ERRORS(result); -} - -TEST_CASE_FIXTURE(Fixture, "it_is_ok_to_oversaturate_a_higher_order_function_argument") -{ - CheckResult result = check(R"( - function onerror() end - function foo() end - xpcall(foo, onerror) - )"); - - LUAU_REQUIRE_NO_ERRORS(result); -} - -TEST_CASE_FIXTURE(Fixture, "another_indirect_function_case_where_it_is_ok_to_provide_too_many_arguments") -{ - CheckResult result = check(R"( - local mycb: (number, number) -> () - - function f() end - - mycb = f - )"); - - LUAU_REQUIRE_NO_ERRORS(result); -} - -TEST_CASE_FIXTURE(Fixture, "call_to_any_yields_any") -{ - CheckResult result = check(R"( - local a: any - local b = a() - )"); - - REQUIRE_EQ("any", toString(requireType("b"))); -} - TEST_CASE_FIXTURE(Fixture, "globals") { CheckResult result = check(R"( @@ -2853,91 +407,6 @@ TEST_CASE_FIXTURE(Fixture, "globals_everywhere") CHECK_EQ("any", toString(requireType("bar"))); } -TEST_CASE_FIXTURE(Fixture, "CheckMethodsOfAny") -{ - CheckResult result = check(R"( -local x: any = {} -function x:y(z: number) - local s: string = z -end -)"); - - LUAU_REQUIRE_ERROR_COUNT(1, result); -} - -TEST_CASE_FIXTURE(Fixture, "CheckMethodsOfSealed") -{ - CheckResult result = check(R"( -local x: {prop: number} = {prop=9999} -function x:y(z: number) - local s: string = z -end -)"); - - LUAU_REQUIRE_ERROR_COUNT(2, result); -} - -TEST_CASE_FIXTURE(Fixture, "CheckMethodsOfNumber") -{ - ScopedFastFlag sff{"LuauErrorRecoveryType", true}; - - CheckResult result = check(R"( -local x: number = 9999 -function x:y(z: number) - local s: string = z -end -)"); - - LUAU_REQUIRE_ERROR_COUNT(2, result); -} - -TEST_CASE_FIXTURE(Fixture, "CheckMethodsOfError") -{ - CheckResult result = check(R"( -local x = (true).foo -function x:y(z: number) - local s: string = z -end -)"); - - LUAU_REQUIRE_ERRORS(result); -} - -TEST_CASE_FIXTURE(Fixture, "CallOrOfFunctions") -{ - CheckResult result = check(R"( -function f() return 1; end -function g() return 2; end -(f or g)() -)"); - - LUAU_REQUIRE_NO_ERRORS(result); -} - -TEST_CASE_FIXTURE(Fixture, "CallAndOrOfFunctions") -{ - CheckResult result = check(R"( -function f() return 1; end -function g() return 2; end -local x = false -(x and f or g)() -)"); - - LUAU_REQUIRE_NO_ERRORS(result); -} - -TEST_CASE_FIXTURE(Fixture, "MixedPropertiesAndIndexers") -{ - CheckResult result = check(R"( -local x = {} -x.a = "a" -x[0] = true -x.b = 37 -)"); - - LUAU_REQUIRE_NO_ERRORS(result); -} - TEST_CASE_FIXTURE(Fixture, "correctly_scope_locals_do") { CheckResult result = check(R"( @@ -2955,35 +424,6 @@ TEST_CASE_FIXTURE(Fixture, "correctly_scope_locals_do") CHECK_EQ(us->name, "a"); } -TEST_CASE_FIXTURE(Fixture, "correctly_scope_locals_while") -{ - CheckResult result = check(R"( - while true do - local a = 1 - end - - print(a) -- oops! - )"); - - LUAU_REQUIRE_ERROR_COUNT(1, result); - - UnknownSymbol* us = get(result.errors[0]); - REQUIRE(us); - CHECK_EQ(us->name, "a"); -} - -TEST_CASE_FIXTURE(Fixture, "ipairs_produces_integral_indices") -{ - CheckResult result = check(R"( - local key - for i, e in ipairs({}) do key = i end - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - - REQUIRE_EQ("number", toString(requireType("key"))); -} - TEST_CASE_FIXTURE(Fixture, "checking_should_not_ice") { CHECK_NOTHROW(check(R"( @@ -2997,79 +437,6 @@ TEST_CASE_FIXTURE(Fixture, "checking_should_not_ice") CHECK_EQ("any", toString(requireType("value"))); } -TEST_CASE_FIXTURE(Fixture, "report_exiting_without_return_nonstrict") -{ - CheckResult result = check(R"( - --!nonstrict - - local function f1(v): number? - if v then - return 1 - end - end - - local function f2(v) - if v then - return 1 - end - end - - local function f3(v): () - if v then - return - end - end - - local function f4(v) - if v then - return - end - end - )"); - - LUAU_REQUIRE_ERROR_COUNT(1, result); - FunctionExitsWithoutReturning* err = get(result.errors[0]); - CHECK(err); -} - -TEST_CASE_FIXTURE(Fixture, "report_exiting_without_return_strict") -{ - CheckResult result = check(R"( - --!strict - - local function f1(v): number? - if v then - return 1 - end - end - - local function f2(v) - if v then - return 1 - end - end - - local function f3(v): () - if v then - return - end - end - - local function f4(v) - if v then - return - end - end - )"); - - LUAU_REQUIRE_ERROR_COUNT(2, result); - FunctionExitsWithoutReturning* annotatedErr = get(result.errors[0]); - CHECK(annotatedErr); - - FunctionExitsWithoutReturning* inferredErr = get(result.errors[1]); - CHECK(inferredErr); -} - // TEST_CASE_FIXTURE(Fixture, "infer_method_signature_of_argument") // { // CheckResult result = check(R"( @@ -3085,363 +452,6 @@ TEST_CASE_FIXTURE(Fixture, "report_exiting_without_return_strict") // CHECK_EQ("A", toString(requireType("f"))); // } -TEST_CASE_FIXTURE(Fixture, "warn_if_you_try_to_require_a_non_modulescript") -{ - fileResolver.source["Modules/A"] = ""; - fileResolver.sourceTypes["Modules/A"] = SourceCode::Local; - - fileResolver.source["Modules/B"] = R"( - local M = require(script.Parent.A) - )"; - - CheckResult result = frontend.check("Modules/B"); - - LUAU_REQUIRE_ERROR_COUNT(1, result); - - CHECK(get(result.errors[0])); -} - -TEST_CASE_FIXTURE(Fixture, "calling_function_with_incorrect_argument_type_yields_errors_spanning_argument") -{ - CheckResult result = check(R"( - function foo(a: number, b: string) end - - foo("Test", 123) - )"); - - LUAU_REQUIRE_ERROR_COUNT(2, result); - - CHECK_EQ(result.errors[0], (TypeError{Location{Position{3, 12}, Position{3, 18}}, TypeMismatch{ - typeChecker.numberType, - typeChecker.stringType, - }})); - - CHECK_EQ(result.errors[1], (TypeError{Location{Position{3, 20}, Position{3, 23}}, TypeMismatch{ - typeChecker.stringType, - typeChecker.numberType, - }})); -} - -TEST_CASE_FIXTURE(Fixture, "calling_function_with_anytypepack_doesnt_leak_free_types") -{ - CheckResult result = check(R"( - --!nonstrict - - function Test(a) - return 1, "" - end - - - local tab = {} - table.insert(tab, Test(1)); - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - - ToStringOptions opts; - opts.exhaustive = true; - opts.maxTableLength = 0; - - CHECK_EQ("{any}", toString(requireType("tab"), opts)); -} - -TEST_CASE_FIXTURE(Fixture, "too_many_return_values") -{ - CheckResult result = check(R"( - --!strict - - function f() - return 55 - end - - local a, b = f() - )"); - - LUAU_REQUIRE_ERROR_COUNT(1, result); - - CountMismatch* acm = get(result.errors[0]); - REQUIRE(acm); - CHECK_EQ(acm->context, CountMismatch::Result); - CHECK_EQ(acm->expected, 1); - CHECK_EQ(acm->actual, 2); -} - -TEST_CASE_FIXTURE(Fixture, "ignored_return_values") -{ - CheckResult result = check(R"( - --!strict - - function f() - return 55, "" - end - - local a = f() - )"); - - LUAU_REQUIRE_ERROR_COUNT(0, result); -} - -TEST_CASE_FIXTURE(Fixture, "function_does_not_return_enough_values") -{ - CheckResult result = check(R"( - --!strict - - function f(): (number, string) - return 55 - end - )"); - - LUAU_REQUIRE_ERROR_COUNT(1, result); - - CountMismatch* acm = get(result.errors[0]); - REQUIRE(acm); - CHECK_EQ(acm->context, CountMismatch::Return); - CHECK_EQ(acm->expected, 2); - CHECK_EQ(acm->actual, 1); -} - -TEST_CASE_FIXTURE(Fixture, "typecheck_unary_minus") -{ - CheckResult result = check(R"( - --!strict - local foo = { - value = 10 - } - local mt = {} - setmetatable(foo, mt) - - mt.__unm = function(val: typeof(foo)): string - return val.value .. "test" - end - - local a = -foo - - local b = 1+-1 - - local bar = { - value = 10 - } - local c = -bar -- disallowed - )"); - - LUAU_REQUIRE_ERROR_COUNT(1, result); - - CHECK_EQ("string", toString(requireType("a"))); - CHECK_EQ("number", toString(requireType("b"))); - - GenericError* gen = get(result.errors[0]); - REQUIRE_EQ(gen->message, "Unary operator '-' not supported by type 'bar'"); -} - -TEST_CASE_FIXTURE(Fixture, "unary_not_is_boolean") -{ - CheckResult result = check(R"( - local b = not "string" - local c = not (math.random() > 0.5 and "string" or 7) - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - REQUIRE_EQ("boolean", toString(requireType("b"))); - REQUIRE_EQ("boolean", toString(requireType("c"))); -} - -TEST_CASE_FIXTURE(Fixture, "disallow_string_and_types_without_metatables_from_arithmetic_binary_ops") -{ - CheckResult result = check(R"( - --!strict - local a = "1.24" + 123 -- not allowed - - local foo = { - value = 10 - } - - local b = foo + 1 -- not allowed - - local bar = { - value = 1 - } - - local mt = {} - - setmetatable(bar, mt) - - mt.__add = function(a: typeof(bar), b: number): number - return a.value + b - end - - local c = bar + 1 -- allowed - - local d = bar + foo -- not allowed - )"); - - LUAU_REQUIRE_ERROR_COUNT(3, result); - - TypeMismatch* tm = get(result.errors[0]); - REQUIRE_EQ(*tm->wantedType, *typeChecker.numberType); - REQUIRE_EQ(*tm->givenType, *typeChecker.stringType); - - TypeMismatch* tm2 = get(result.errors[2]); - CHECK_EQ(*tm2->wantedType, *typeChecker.numberType); - CHECK_EQ(*tm2->givenType, *requireType("foo")); - - GenericError* gen2 = get(result.errors[1]); - REQUIRE_EQ(gen2->message, "Binary operator '+' not supported by types 'foo' and 'number'"); -} - -// CLI-29033 -TEST_CASE_FIXTURE(Fixture, "unknown_type_in_comparison") -{ - CheckResult result = check(R"( - function merge(lower, greater) - if lower.y == greater.y then - end - end - )"); - - LUAU_REQUIRE_NO_ERRORS(result); -} - -TEST_CASE_FIXTURE(Fixture, "concat_op_on_free_lhs_and_string_rhs") -{ - CheckResult result = check(R"( - local function f(x) - return x .. "y" - end - )"); - - LUAU_REQUIRE_ERROR_COUNT(1, result); - REQUIRE(get(result.errors[0])); -} - -TEST_CASE_FIXTURE(Fixture, "concat_op_on_string_lhs_and_free_rhs") -{ - CheckResult result = check(R"( - local function f(x) - return "foo" .. x - end - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - - CHECK_EQ("(string) -> string", toString(requireType("f"))); -} - -TEST_CASE_FIXTURE(Fixture, "strict_binary_op_where_lhs_unknown") -{ - std::vector ops = {"+", "-", "*", "/", "%", "^", ".."}; - - std::string src = R"( - function foo(a, b) - )"; - - for (const auto& op : ops) - src += "local _ = a " + op + "b\n"; - - src += "end"; - - CheckResult result = check(src); - LUAU_REQUIRE_ERROR_COUNT(ops.size(), result); - - CHECK_EQ("Unknown type used in + operation; consider adding a type annotation to 'a'", toString(result.errors[0])); -} - -TEST_CASE_FIXTURE(Fixture, "function_cast_error_uses_correct_language") -{ - CheckResult result = check(R"( - function foo(a, b): number - return 0 - end - - local a: (string)->number = foo - local b: (number, number)->(number, number) = foo - - local c: (string, number)->number = foo -- no error - )"); - - LUAU_REQUIRE_ERROR_COUNT(2, result); - - auto tm1 = get(result.errors[0]); - REQUIRE(tm1); - - CHECK_EQ("(string) -> number", toString(tm1->wantedType)); - CHECK_EQ("(string, *unknown*) -> number", toString(tm1->givenType)); - - auto tm2 = get(result.errors[1]); - REQUIRE(tm2); - - CHECK_EQ("(number, number) -> (number, number)", toString(tm2->wantedType)); - CHECK_EQ("(string, *unknown*) -> number", toString(tm2->givenType)); -} - -TEST_CASE_FIXTURE(Fixture, "setmetatable_cant_be_used_to_mutate_global_types") -{ - { - Fixture fix; - - // inherit env from parent fixture checker - fix.typeChecker.globalScope = typeChecker.globalScope; - - fix.check(R"( ---!nonstrict -type MT = typeof(setmetatable) -function wtf(arg: {MT}): typeof(table) - arg = wtf(arg) -end -)"); - } - - // validate sharedEnv post-typecheck; valuable for debugging some typeck crashes but slows fuzzing down - // note: it's important for typeck to be destroyed at this point! - { - for (auto& p : typeChecker.globalScope->bindings) - { - toString(p.second.typeId); // toString walks the entire type, making sure ASAN catches access to destroyed type arenas - } - } -} - -TEST_CASE_FIXTURE(Fixture, "evil_table_unification") -{ - // this code re-infers the type of _ while processing fields of _, which can cause use-after-free - check(R"( ---!nonstrict -_ = ... -_:table(_,string)[_:gsub(_,...,n0)],_,_:gsub(_,string)[""],_:split(_,...,table)._,n0 = nil -do end -)"); -} - -TEST_CASE_FIXTURE(Fixture, "overload_is_not_a_function") -{ - check(R"( ---!nonstrict -function _(...):((typeof(not _))&(typeof(not _)))&((typeof(not _))&(typeof(not _))) -_(...)(setfenv,_,not _,"")[_] = nil -end -do end -_(...)(...,setfenv,_):_G() -)"); -} - -TEST_CASE_FIXTURE(Fixture, "cyclic_type_packs") -{ - // this has a risk of creating cyclic type packs, causing infinite loops / OOMs - check(R"( ---!nonstrict -_ += _(_,...) -repeat -_ += _(...) -until ... + _ -)"); - - check(R"( ---!nonstrict -_ += _(_(...,...),_(...)) -repeat -until _ -)"); -} - TEST_CASE_FIXTURE(Fixture, "cyclic_follow") { check(R"( @@ -3470,19 +480,6 @@ end )"); } -TEST_CASE_FIXTURE(Fixture, "and_binexps_dont_unify") -{ - CheckResult result = check(R"( - --!strict - local t = {} - while true and t[1] do - print(t[1].test) - end - )"); - - LUAU_REQUIRE_NO_ERRORS(result); -} - struct FindFreeTypeVars { bool foundOne = false; @@ -3506,32 +503,6 @@ struct FindFreeTypeVars } }; -TEST_CASE_FIXTURE(Fixture, "dont_crash_when_setmetatable_does_not_produce_a_metatabletypevar") -{ - CheckResult result = check("local x = setmetatable({})"); - LUAU_REQUIRE_ERROR_COUNT(1, result); -} - -TEST_CASE_FIXTURE(Fixture, "for_in_loop_where_iteratee_is_free") -{ - // This code doesn't pass typechecking. We just care that it doesn't crash. - (void)check(R"( - --!nonstrict - function _:_(...) - end - - repeat - if _ then - else - _ = ... - end - until _ - - for _ in _() do - end - )"); -} - TEST_CASE_FIXTURE(Fixture, "tc_after_error_recovery") { CheckResult result = check(R"( @@ -3604,36 +575,6 @@ TEST_CASE_FIXTURE(Fixture, "tc_after_error_recovery_no_replacement_name_in_error } } -TEST_CASE_FIXTURE(Fixture, "error_on_invalid_operand_types_to_relational_operators") -{ - CheckResult result = check(R"( - local a: boolean = true - local b: boolean = false - local foo = a < b - )"); - - LUAU_REQUIRE_ERROR_COUNT(1, result); - - GenericError* ge = get(result.errors[0]); - REQUIRE(ge); - CHECK_EQ("Type 'boolean' cannot be compared with relational operator <", ge->message); -} - -TEST_CASE_FIXTURE(Fixture, "error_on_invalid_operand_types_to_relational_operators2") -{ - CheckResult result = check(R"( - local a: number | string = "" - local b: number | string = 1 - local foo = a < b - )"); - - LUAU_REQUIRE_ERROR_COUNT(1, result); - - GenericError* ge = get(result.errors[0]); - REQUIRE(ge); - CHECK_EQ("Type 'number | string' cannot be compared with relational operator <", ge->message); -} - TEST_CASE_FIXTURE(Fixture, "index_expr_should_be_checked") { CheckResult result = check(R"( @@ -3650,100 +591,6 @@ TEST_CASE_FIXTURE(Fixture, "index_expr_should_be_checked") CHECK_EQ("x", up->key); } -TEST_CASE_FIXTURE(Fixture, "unreachable_code_after_infinite_loop") -{ - { - CheckResult result = check(R"( - function unreachablecodepath(a): number - while true do - if a then return 10 end - end - -- unreachable - end - unreachablecodepath(4) - )"); - - LUAU_REQUIRE_ERROR_COUNT(0, result); - } - - { - CheckResult result = check(R"( - function reachablecodepath(a): number - while true do - if a then break end - return 10 - end - - print("x") -- correct error - end - reachablecodepath(4) - )"); - - LUAU_REQUIRE_ERRORS(result); - CHECK(get(result.errors[0])); - } - - { - CheckResult result = check(R"( - function unreachablecodepath(a): number - repeat - if a then return 10 end - until false - - -- unreachable - end - unreachablecodepath(4) - )"); - - LUAU_REQUIRE_ERROR_COUNT(0, result); - } - - { - CheckResult result = check(R"( - function reachablecodepath(a, b): number - repeat - if a then break end - - if b then return 10 end - until false - - print("x") -- correct error - end - reachablecodepath(4) - )"); - - LUAU_REQUIRE_ERRORS(result); - CHECK(get(result.errors[0])); - } - - { - CheckResult result = check(R"( - function unreachablecodepath(a: number?): number - repeat - return 10 - until a ~= nil - - -- unreachable - end - unreachablecodepath(4) - )"); - - LUAU_REQUIRE_ERROR_COUNT(0, result); - } -} - -TEST_CASE_FIXTURE(Fixture, "cli_38355_recursive_union") -{ - CheckResult result = check(R"( - --!strict - local _ - _ += _ and _ or _ and _ or _ and _ - )"); - - LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK_EQ("Type contains a self-recursive construct that cannot be resolved", toString(result.errors[0])); -} - TEST_CASE_FIXTURE(Fixture, "stringify_nested_unions_with_optionals") { CheckResult result = check(R"( @@ -3759,58 +606,6 @@ TEST_CASE_FIXTURE(Fixture, "stringify_nested_unions_with_optionals") CHECK_EQ("(boolean | number | string)?", toString(tm->givenType)); } -TEST_CASE_FIXTURE(Fixture, "UnknownGlobalCompoundAssign") -{ - // In non-strict mode, global definition is still allowed - { - CheckResult result = check(R"( - --!nonstrict - a = a + 1 - print(a) - )"); - - LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK_EQ(toString(result.errors[0]), "Unknown global 'a'"); - } - - // In strict mode we no longer generate two errors from lhs - { - CheckResult result = check(R"( - --!strict - a += 1 - print(a) - )"); - - LUAU_REQUIRE_ERRORS(result); - CHECK_EQ(toString(result.errors[0]), "Unknown global 'a'"); - } - - // In non-strict mode, compound assignment is not a definition, it's a modification - { - CheckResult result = check(R"( - --!nonstrict - a += 1 - print(a) - )"); - - LUAU_REQUIRE_ERROR_COUNT(2, result); - CHECK_EQ(toString(result.errors[0]), "Unknown global 'a'"); - } -} - -TEST_CASE_FIXTURE(Fixture, "loop_typecheck_crash_on_empty_optional") -{ - CheckResult result = check(R"( - local t = {} - for _ in t do - for _ in assert(missing()) do - end - end - )"); - - LUAU_REQUIRE_ERROR_COUNT(2, result); -} - TEST_CASE_FIXTURE(Fixture, "cli_39932_use_unifier_in_ensure_methods") { CheckResult result = check(R"( @@ -3821,26 +616,6 @@ TEST_CASE_FIXTURE(Fixture, "cli_39932_use_unifier_in_ensure_methods") LUAU_REQUIRE_NO_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "metatable_of_any_can_be_a_table") -{ - CheckResult result = check(R"( ---!strict -local T: any -T = {} -T.__index = T -function T.new(...) - local self = {} - setmetatable(self, T) - self:construct(...) - return self -end -function T:construct(index) -end -)"); - - LUAU_REQUIRE_NO_ERRORS(result); -} - TEST_CASE_FIXTURE(Fixture, "dont_report_type_errors_within_an_AstStatError") { CheckResult result = check(R"( @@ -3868,64 +643,6 @@ TEST_CASE_FIXTURE(Fixture, "dont_ice_on_astexprerror") LUAU_REQUIRE_ERROR_COUNT(1, result); } -TEST_CASE_FIXTURE(Fixture, "strip_nil_from_lhs_or_operator") -{ - CheckResult result = check(R"( ---!strict -local a: number? = nil -local b: number = a or 1 - )"); - - LUAU_REQUIRE_NO_ERRORS(result); -} - -TEST_CASE_FIXTURE(Fixture, "strip_nil_from_lhs_or_operator2") -{ - CheckResult result = check(R"( ---!nonstrict -local a: number? = nil -local b: number = a or 1 - )"); - - LUAU_REQUIRE_NO_ERRORS(result); -} - -TEST_CASE_FIXTURE(Fixture, "dont_strip_nil_from_rhs_or_operator") -{ - CheckResult result = check(R"( ---!strict -local a: number? = nil -local b: number = 1 or a - )"); - - LUAU_REQUIRE_ERROR_COUNT(1, result); - - TypeMismatch* tm = get(result.errors[0]); - REQUIRE(tm); - CHECK_EQ(typeChecker.numberType, tm->wantedType); - CHECK_EQ("number?", toString(tm->givenType)); -} - -TEST_CASE_FIXTURE(Fixture, "no_lossy_function_type") -{ - CheckResult result = check(R"( - --!strict - local tbl = {} - function tbl:abc(a: number, b: number) - return a - end - tbl:abc(1, 2) -- Line 6 - -- | Column 14 - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - TypeId type = requireTypeAtPosition(Position(6, 14)); - CHECK_EQ("(tbl, number, number) -> number", toString(type)); - auto ftv = get(type); - REQUIRE(ftv); - CHECK(ftv->hasSelf); -} - TEST_CASE_FIXTURE(Fixture, "luau_resolves_symbols_the_same_way_lua_does") { CheckResult result = check(R"( @@ -3943,326 +660,6 @@ TEST_CASE_FIXTURE(Fixture, "luau_resolves_symbols_the_same_way_lua_does") REQUIRE_MESSAGE(get(e) != nullptr, "Expected UnknownSymbol, but got " << e); } -TEST_CASE_FIXTURE(Fixture, "operator_eq_verifies_types_do_intersect") -{ - CheckResult result = check(R"( - type Array = { [number]: T } - type Fiber = { id: number } - type null = {} - - local fiberStack: Array = {} - local index = 0 - - local function f(fiber: Fiber) - local a = fiber ~= fiberStack[index] - local b = fiberStack[index] ~= fiber - end - - return f - )"); - - LUAU_REQUIRE_NO_ERRORS(result); -} - -TEST_CASE_FIXTURE(Fixture, "general_require_call_expression") -{ - fileResolver.source["game/A"] = R"( ---!strict -return { def = 4 } - )"; - - fileResolver.source["game/B"] = R"( ---!strict -local tbl = { abc = require(game.A) } -local a : string = "" -a = tbl.abc.def - )"; - - CheckResult result = frontend.check("game/B"); - LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK_EQ("Type 'number' could not be converted into 'string'", toString(result.errors[0])); -} - -TEST_CASE_FIXTURE(Fixture, "general_require_type_mismatch") -{ - fileResolver.source["game/A"] = R"( -return { def = 4 } - )"; - - fileResolver.source["game/B"] = R"( -local tbl: string = require(game.A) - )"; - - CheckResult result = frontend.check("game/B"); - LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK_EQ("Type '{| def: number |}' could not be converted into 'string'", toString(result.errors[0])); -} - -TEST_CASE_FIXTURE(Fixture, "nonstrict_self_mismatch_tail") -{ - CheckResult result = check(R"( ---!nonstrict -local f = {} -function f:foo(a: number, b: number) end - -function bar(...) - f.foo(f, 1, ...) -end - -bar(2) -)"); - LUAU_REQUIRE_NO_ERRORS(result); -} - -TEST_CASE_FIXTURE(Fixture, "typeof_unresolved_function") -{ - CheckResult result = check(R"( -local function f(a: typeof(f)) end -)"); - LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK_EQ("Unknown global 'f'", toString(result.errors[0])); -} - -TEST_CASE_FIXTURE(Fixture, "instantiate_table_cloning") -{ - CheckResult result = check(R"( ---!nonstrict -local l0:any,l61:t0 = _,math -while _ do -_() -end -function _():t0 -end -type t0 = any -)"); - - std::optional ty = requireType("math"); - REQUIRE(ty); - - const TableTypeVar* ttv = get(*ty); - REQUIRE(ttv); - CHECK(ttv->instantiatedTypeParams.empty()); -} - -TEST_CASE_FIXTURE(Fixture, "instantiate_table_cloning_2") -{ - ScopedFastFlag sff{"LuauOnlyMutateInstantiatedTables", true}; - - CheckResult result = check(R"( -type X = T -type K = X -)"); - - LUAU_REQUIRE_NO_ERRORS(result); - - std::optional ty = requireType("math"); - REQUIRE(ty); - - const TableTypeVar* ttv = get(*ty); - REQUIRE(ttv); - CHECK(ttv->instantiatedTypeParams.empty()); -} - -TEST_CASE_FIXTURE(Fixture, "instantiate_table_cloning_3") -{ - ScopedFastFlag sff{"LuauOnlyMutateInstantiatedTables", true}; - - CheckResult result = check(R"( -type X = T -local a = {} -a.x = 4 -local b: X -a.y = 5 -local c: X -c = b -)"); - - LUAU_REQUIRE_NO_ERRORS(result); - - std::optional ty = requireType("a"); - REQUIRE(ty); - - const TableTypeVar* ttv = get(*ty); - REQUIRE(ttv); - CHECK(ttv->instantiatedTypeParams.empty()); -} - -TEST_CASE_FIXTURE(Fixture, "bound_free_table_export_is_ok") -{ - CheckResult result = check(R"( -local n = {} -function n:Clone() end - -local m = {} - -function m.a(x) - x:Clone() -end - -function m.b() - m.a(n) -end - -return m -)"); - LUAU_REQUIRE_NO_ERRORS(result); -} - -TEST_CASE_FIXTURE(Fixture, "self_recursive_instantiated_param") -{ - ScopedFastFlag sff{"LuauOnlyMutateInstantiatedTables", true}; - - // Mutability in type function application right now can create strange recursive types - CheckResult result = check(R"( -type Table = { a: number } -type Self = T -local a: Self
- )"); - - LUAU_REQUIRE_NO_ERRORS(result); - CHECK_EQ(toString(requireType("a")), "Table"); -} - -TEST_CASE_FIXTURE(Fixture, "no_persistent_typelevel_change") -{ - TypeId mathTy = requireType(typeChecker.globalScope, "math"); - REQUIRE(mathTy); - TableTypeVar* ttv = getMutable(mathTy); - REQUIRE(ttv); - const FunctionTypeVar* ftv = get(ttv->props["frexp"].type); - REQUIRE(ftv); - auto original = ftv->level; - - CheckResult result = check("local a = math.frexp"); - - LUAU_REQUIRE_NO_ERRORS(result); - CHECK(ftv->level.level == original.level); - CHECK(ftv->level.subLevel == original.subLevel); -} - -TEST_CASE_FIXTURE(Fixture, "table_indexing_error_location") -{ - CheckResult result = check(R"( -local foo = {42} -local bar: number? -local baz = foo[bar] - )"); - - LUAU_REQUIRE_ERROR_COUNT(1, result); - - CHECK_EQ(result.errors[0].location, Location{Position{3, 16}, Position{3, 19}}); -} - -TEST_CASE_FIXTURE(Fixture, "table_simple_call") -{ - CheckResult result = check(R"( -local a = setmetatable({ x = 2 }, { - __call = function(self) - return (self.x :: number) * 2 -- should work without annotation in the future - end -}) -local b = a() -local c = a(2) -- too many arguments - )"); - - LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK_EQ("Argument count mismatch. Function expects 1 argument, but 2 are specified", toString(result.errors[0])); -} - -TEST_CASE_FIXTURE(Fixture, "custom_require_global") -{ - CheckResult result = check(R"( ---!nonstrict -require = function(a) end - -local crash = require(game.A) - )"); - - LUAU_REQUIRE_NO_ERRORS(result); -} - -TEST_CASE_FIXTURE(Fixture, "operator_eq_operands_are_not_subtypes_of_each_other_but_has_overlap") -{ - ScopedFastFlag sff1{"LuauEqConstraint", true}; - - CheckResult result = check(R"( - local function f(a: string | number, b: boolean | number) - return a == b - end - )"); - - // This doesn't produce any errors but for the wrong reasons. - // This unit test serves as a reminder to not try and unify the operands on `==`/`~=`. - LUAU_REQUIRE_NO_ERRORS(result); -} - -TEST_CASE_FIXTURE(Fixture, "access_index_metamethod_that_returns_variadic") -{ - CheckResult result = check(R"( - type Foo = {x: string} - local t = {} - setmetatable(t, { - __index = function(x: string): ...Foo - return {x = x} - end - }) - - local foo = t.bar - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - - ToStringOptions o; - o.exhaustive = true; - CHECK_EQ("{| x: string |}", toString(requireType("foo"), o)); -} - -TEST_CASE_FIXTURE(Fixture, "detect_cyclic_typepacks") -{ - CheckResult result = check(R"( - type ( ... ) ( ) ; - ( ... ) ( - - ... ) ( - ... ) - type = ( ... ) ; - ( ... ) ( ) ( ... ) ; - ( ... ) "" - )"); - - CHECK_LE(0, result.errors.size()); -} - -TEST_CASE_FIXTURE(Fixture, "detect_cyclic_typepacks2") -{ - CheckResult result = check(R"( - function _(l0:((typeof((pcall)))|((((t0)->())|(typeof(-67108864)))|(any)))|(any),...):(((typeof(0))|(any))|(any),typeof(-67108864),any) - xpcall(_,_,_) - _(_,_,_) - end - )"); - - CHECK_LE(0, result.errors.size()); -} - -TEST_CASE_FIXTURE(Fixture, "no_stack_overflow_from_quantifying") -{ - CheckResult result = check(R"( - function _(l0:t0): (any, ()->()) - end - - type t0 = t0 | {} - )"); - - CHECK_LE(0, result.errors.size()); - - std::optional t0 = getMainModule()->getModuleScope()->lookupType("t0"); - REQUIRE(t0); - CHECK_EQ("*unknown*", toString(t0->type)); - - auto it = std::find_if(result.errors.begin(), result.errors.end(), [](TypeError& err) { - return get(err); - }); - CHECK(it != result.errors.end()); -} - TEST_CASE_FIXTURE(Fixture, "no_stack_overflow_from_isoptional") { CheckResult result = check(R"( @@ -4316,22 +713,6 @@ TEST_CASE_FIXTURE(Fixture, "no_infinite_loop_when_trying_to_unify_uh_this") CHECK_LE(0, result.errors.size()); } -TEST_CASE_FIXTURE(Fixture, "no_stack_overflow_from_flattenintersection") -{ - CheckResult result = check(R"( - local l0,l0 - repeat - type t0 = ((any)|((any)&((any)|((any)&((any)|(any))))))&(t0) - function _(l0):(t0)&(t0) - while nil do - end - end - until _(_)(_)._ - )"); - - CHECK_LE(0, result.errors.size()); -} - TEST_CASE_FIXTURE(Fixture, "no_heap_use_after_free_error") { CheckResult result = check(R"( @@ -4349,329 +730,6 @@ TEST_CASE_FIXTURE(Fixture, "no_heap_use_after_free_error") CHECK_LE(0, result.errors.size()); } -TEST_CASE_FIXTURE(Fixture, "dont_invalidate_the_properties_iterator_of_free_table_when_rolled_back") -{ - fileResolver.source["Module/Backend/Types"] = R"( - export type Fiber = { - return_: Fiber? - } - return {} - )"; - - fileResolver.source["Module/Backend"] = R"( - local Types = require(script.Types) - type Fiber = Types.Fiber - type ReactRenderer = { findFiberByHostInstance: () -> Fiber? } - - local function attach(renderer): () - local function getPrimaryFiber(fiber) - local alternate = fiber.alternate - return fiber - end - - local function getFiberIDForNative() - local fiber = renderer.findFiberByHostInstance() - fiber = fiber.return_ - return getPrimaryFiber(fiber) - end - end - - function culprit(renderer: ReactRenderer): () - attach(renderer) - end - - return culprit - )"; - - CheckResult result = frontend.check("Module/Backend"); -} - -TEST_CASE_FIXTURE(Fixture, "recursive_types_restriction_ok") -{ - CheckResult result = check(R"( - type Tree = { data: T, children: {Tree} } - )"); - - LUAU_REQUIRE_NO_ERRORS(result); -} - -TEST_CASE_FIXTURE(Fixture, "recursive_types_restriction_not_ok") -{ - ScopedFastFlag sff{"LuauRecursiveTypeParameterRestriction", true}; - - CheckResult result = check(R"( - -- this would be an infinite type if we allowed it - type Tree = { data: T, children: {Tree<{T}>} } - )"); - - LUAU_REQUIRE_ERRORS(result); -} - -TEST_CASE_FIXTURE(Fixture, "record_matching_overload") -{ - CheckResult result = check(R"( - type Overload = ((string) -> string) & ((number) -> number) - local abc: Overload - abc(1) - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - - // AstExprCall is the node that has the overload stored on it. - // findTypeAtPosition will look at the AstExprLocal, but this is not what - // we want to look at. - std::vector ancestry = findAstAncestryOfPosition(*getMainSourceModule(), Position(3, 10)); - REQUIRE_GE(ancestry.size(), 2); - AstExpr* parentExpr = ancestry[ancestry.size() - 2]->asExpr(); - REQUIRE(bool(parentExpr)); - REQUIRE(parentExpr->is()); - - ModulePtr module = getMainModule(); - auto it = module->astOverloadResolvedTypes.find(parentExpr); - REQUIRE(it); - CHECK_EQ(toString(*it), "(number) -> number"); -} - -TEST_CASE_FIXTURE(Fixture, "return_type_by_overload") -{ - ScopedFastFlag sff{"LuauErrorRecoveryType", true}; - - CheckResult result = check(R"( - type Overload = ((string) -> string) & ((number, number) -> number) - local abc: Overload - local x = abc(true) - local y = abc(true,true) - local z = abc(true,true,true) - )"); - - LUAU_REQUIRE_ERRORS(result); - CHECK_EQ("string", toString(requireType("x"))); - CHECK_EQ("number", toString(requireType("y"))); - // Should this be string|number? - CHECK_EQ("string", toString(requireType("z"))); -} - -TEST_CASE_FIXTURE(Fixture, "infer_anonymous_function_arguments") -{ - // Simple direct arg to arg propagation - CheckResult result = check(R"( -type Table = { x: number, y: number } -local function f(a: (Table) -> number) return a({x = 1, y = 2}) end -f(function(a) return a.x + a.y end) - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - - // An optional function is accepted, but since we already provide a function, nil can be ignored - result = check(R"( -type Table = { x: number, y: number } -local function f(a: ((Table) -> number)?) if a then return a({x = 1, y = 2}) else return 0 end end -f(function(a) return a.x + a.y end) - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - - // Make sure self calls match correct index - result = check(R"( -type Table = { x: number, y: number } -local x = {} -x.b = {x = 1, y = 2} -function x:f(a: (Table) -> number) return a(self.b) end -x:f(function(a) return a.x + a.y end) - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - - // Mix inferred and explicit argument types - result = check(R"( -function f(a: (a: number, b: number, c: boolean) -> number) return a(1, 2, true) end -f(function(a: number, b, c) return c and a + b or b - a end) - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - - // Anonymous function has a variadic pack - result = check(R"( -type Table = { x: number, y: number } -local function f(a: (Table) -> number) return a({x = 1, y = 2}) end -f(function(...) return select(1, ...).z end) - )"); - - LUAU_REQUIRE_ERRORS(result); - CHECK_EQ("Key 'z' not found in table 'Table'", toString(result.errors[0])); - - // Can't accept more arguments than provided - result = check(R"( -function f(a: (a: number, b: number) -> number) return a(1, 2) end -f(function(a, b, c, ...) return a + b end) - )"); - - LUAU_REQUIRE_ERRORS(result); - - CHECK_EQ(R"(Type '(number, number, a) -> number' could not be converted into '(number, number) -> number' -caused by: - Argument count mismatch. Function expects 3 arguments, but only 2 are specified)", - toString(result.errors[0])); - - // Infer from variadic packs into elements - result = check(R"( -function f(a: (...number) -> number) return a(1, 2) end -f(function(a, b) return a + b end) - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - - // Infer from variadic packs into variadic packs - result = check(R"( -type Table = { x: number, y: number } -function f(a: (...Table) -> number) return a({x = 1, y = 2}, {x = 3, y = 4}) end -f(function(a, ...) local b = ... return b.z end) - )"); - - LUAU_REQUIRE_ERRORS(result); - CHECK_EQ("Key 'z' not found in table 'Table'", toString(result.errors[0])); - - // Return type inference - result = check(R"( -type Table = { x: number, y: number } -function f(a: (number) -> Table) return a(4) end -f(function(x) return x * 2 end) - )"); - - LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK_EQ("Type 'number' could not be converted into 'Table'", toString(result.errors[0])); - - // Return type doesn't inference 'nil' - result = check(R"( -function f(a: (number) -> nil) return a(4) end -f(function(x) print(x) end) - )"); - - LUAU_REQUIRE_NO_ERRORS(result); -} - -TEST_CASE_FIXTURE(Fixture, "infer_generic_function_function_argument") -{ - ScopedFastFlag sff{"LuauUnsealedTableLiteral", true}; - - CheckResult result = check(R"( -local function sum(x: a, y: a, f: (a, a) -> a) return f(x, y) end -return sum(2, 3, function(a, b) return a + b end) - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - - result = check(R"( -local function map(arr: {a}, f: (a) -> b) local r = {} for i,v in ipairs(arr) do table.insert(r, f(v)) end return r end -local a = {1, 2, 3} -local r = map(a, function(a) return a + a > 100 end) - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - REQUIRE_EQ("{boolean}", toString(requireType("r"))); - - check(R"( -local function foldl(arr: {a}, init: b, f: (b, a) -> b) local r = init for i,v in ipairs(arr) do r = f(r, v) end return r end -local a = {1, 2, 3} -local r = foldl(a, {s=0,c=0}, function(a, b) return {s = a.s + b, c = a.c + 1} end) - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - REQUIRE_EQ("{ c: number, s: number }", toString(requireType("r"))); -} - -TEST_CASE_FIXTURE(Fixture, "infer_generic_function_function_argument_overloaded") -{ - CheckResult result = check(R"( -local function g1(a: T, f: (T) -> T) return f(a) end -local function g2(a: T, b: T, f: (T, T) -> T) return f(a, b) end - -local g12: typeof(g1) & typeof(g2) - -g12(1, function(x) return x + x end) -g12(1, 2, function(x, y) return x + y end) - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - - result = check(R"( -local function g1(a: T, f: (T) -> T) return f(a) end -local function g2(a: T, b: T, f: (T, T) -> T) return f(a, b) end - -local g12: typeof(g1) & typeof(g2) - -g12({x=1}, function(x) return {x=-x.x} end) -g12({x=1}, {x=2}, function(x, y) return {x=x.x + y.x} end) - )"); - - LUAU_REQUIRE_NO_ERRORS(result); -} - -TEST_CASE_FIXTURE(Fixture, "infer_generic_lib_function_function_argument") -{ - CheckResult result = check(R"( -local a = {{x=4}, {x=7}, {x=1}} -table.sort(a, function(x, y) return x.x < y.x end) - )"); - - LUAU_REQUIRE_NO_ERRORS(result); -} - -TEST_CASE_FIXTURE(Fixture, "infer_anonymous_function_arguments_outside_call") -{ - CheckResult result = check(R"( -type Table = { x: number, y: number } -local f: (Table) -> number = function(t) return t.x + t.y end - -type TableWithFunc = { x: number, y: number, f: (number, number) -> number } -local a: TableWithFunc = { x = 3, y = 4, f = function(a, b) return a + b end } - )"); - - LUAU_REQUIRE_NO_ERRORS(result); -} - -TEST_CASE_FIXTURE(Fixture, "do_not_infer_generic_functions") -{ - CheckResult result = check(R"( -local function sum(x: a, y: a, f: (a, a) -> a) return f(x, y) end - -local function sumrec(f: typeof(sum)) - return sum(2, 3, function(a, b) return a + b end) -end - -local b = sumrec(sum) -- ok -local c = sumrec(function(x, y, f) return f(x, y) end) -- type binders are not inferred - )"); - - LUAU_REQUIRE_ERRORS(result); - CHECK_EQ("Type '(a, b, (a, b) -> (c...)) -> (c...)' could not be converted into '(a, a, (a, a) -> a) -> a'; different number of generic type " - "parameters", - toString(result.errors[0])); -} - -TEST_CASE_FIXTURE(Fixture, "infer_return_value_type") -{ - CheckResult result = check(R"( -local function f(): {string|number} - return {1, "b", 3} -end - -local function g(): (number, {string|number}) - return 4, {1, "b", 3} -end - -local function h(): ...{string|number} - return {4}, {1, "b", 3}, {"s"} -end - -local function i(): ...{string|number} - return {1, "b", 3}, h() -end - )"); - - LUAU_REQUIRE_NO_ERRORS(result); -} - TEST_CASE_FIXTURE(Fixture, "infer_type_assertion_value_type") { CheckResult result = check(R"( @@ -4719,56 +777,6 @@ f(((function(a, b) return a + b end))) LUAU_REQUIRE_NO_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "refine_and_or") -{ - CheckResult result = check(R"( - local t: {x: number?}? = {x = nil} - local u = t and t.x or 5 - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - - CHECK_EQ("number", toString(requireType("u"))); -} - -TEST_CASE_FIXTURE(Fixture, "checked_prop_too_early") -{ - CheckResult result = check(R"( - local t: {x: number?}? = {x = nil} - local u = t.x and t or 5 - )"); - - LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK_EQ("Value of type '{| x: number? |}?' could be nil", toString(result.errors[0])); - CHECK_EQ("number | {| x: number? |}", toString(requireType("u"))); -} - -TEST_CASE_FIXTURE(Fixture, "accidentally_checked_prop_in_opposite_branch") -{ - CheckResult result = check(R"( - local t: {x: number?}? = {x = nil} - local u = t and t.x == 5 or t.x == 31337 - )"); - - LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK_EQ("Value of type '{| x: number? |}?' could be nil", toString(result.errors[0])); - CHECK_EQ("boolean", toString(requireType("u"))); -} - -TEST_CASE_FIXTURE(Fixture, "substitution_with_bound_table") -{ - CheckResult result = check(R"( -type A = { x: number } -local a: A = { x = 1 } -local b = a -type B = typeof(b) -type X = T -local c: X - )"); - - LUAU_REQUIRE_NO_ERRORS(result); -} - TEST_CASE_FIXTURE(Fixture, "tc_if_else_expressions1") { CheckResult result = check(R"(local a = if true then "true" else "false")"); @@ -4830,40 +838,6 @@ end LUAU_REQUIRE_NO_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "type_error_addition") -{ - CheckResult result = check(R"( ---!strict -local foo = makesandwich() -local bar = foo.nutrition + 100 - )"); - - LUAU_REQUIRE_ERROR_COUNT(1, result); - - // We should definitely get this error - CHECK_EQ("Unknown global 'makesandwich'", toString(result.errors[0])); - // We get this error if makesandwich() returns a free type - // CHECK_EQ("Unknown type used in + operation; consider adding a type annotation to 'foo'", toString(result.errors[1])); -} - -TEST_CASE_FIXTURE(Fixture, "require_failed_module") -{ - fileResolver.source["game/A"] = R"( -return unfortunately() - )"; - - CheckResult aResult = frontend.check("game/A"); - LUAU_REQUIRE_ERRORS(aResult); - - CheckResult result = check(R"( -local ModuleA = require(game.A) - )"); - LUAU_REQUIRE_NO_ERRORS(result); - - std::optional oty = requireType("ModuleA"); - CHECK_EQ("*unknown*", toString(*oty)); -} - /* * If it wasn't instantly obvious, we have the fuzzer to thank for this gem of a test. * @@ -4921,184 +895,6 @@ TEST_CASE_FIXTURE(Fixture, "fuzzer_found_this") )"); } -/* - * We had an issue where part of the type of pairs() was an unsealed table. - * This test depends on FFlagDebugLuauFreezeArena to trigger it. - */ -TEST_CASE_FIXTURE(Fixture, "pairs_parameters_are_not_unsealed_tables") -{ - check(R"( - function _(l0:{n0:any}) - _ = pairs - end - )"); -} - -TEST_CASE_FIXTURE(Fixture, "inferred_methods_of_free_tables_have_the_same_level_as_the_enclosing_table") -{ - check(R"( - function Base64FileReader(data) - local reader = {} - local index: number - - function reader:PeekByte() - return data:byte(index) - end - - function reader:Byte() - return data:byte(index - 1) - end - - return reader - end - - Base64FileReader() - - function ReadMidiEvents(data) - - local reader = Base64FileReader(data) - - while reader:HasMore() do - (reader:Byte() % 128) - end - end - )"); -} - -TEST_CASE_FIXTURE(Fixture, "error_detailed_function_mismatch_arg_count") -{ - CheckResult result = check(R"( -type A = (number, number) -> string -type B = (number) -> string - -local a: A -local b: B = a - )"); - - LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK_EQ(toString(result.errors[0]), R"(Type '(number, number) -> string' could not be converted into '(number) -> string' -caused by: - Argument count mismatch. Function expects 2 arguments, but only 1 is specified)"); -} - -TEST_CASE_FIXTURE(Fixture, "error_detailed_function_mismatch_arg") -{ - CheckResult result = check(R"( -type A = (number, number) -> string -type B = (number, string) -> string - -local a: A -local b: B = a - )"); - - LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK_EQ(toString(result.errors[0]), R"(Type '(number, number) -> string' could not be converted into '(number, string) -> string' -caused by: - Argument #2 type is not compatible. Type 'string' could not be converted into 'number')"); -} - -TEST_CASE_FIXTURE(Fixture, "error_detailed_function_mismatch_ret_count") -{ - CheckResult result = check(R"( -type A = (number, number) -> (number) -type B = (number, number) -> (number, boolean) - -local a: A -local b: B = a - )"); - - LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK_EQ(toString(result.errors[0]), R"(Type '(number, number) -> number' could not be converted into '(number, number) -> (number, boolean)' -caused by: - Function only returns 1 value. 2 are required here)"); -} - -TEST_CASE_FIXTURE(Fixture, "error_detailed_function_mismatch_ret") -{ - CheckResult result = check(R"( -type A = (number, number) -> string -type B = (number, number) -> number - -local a: A -local b: B = a - )"); - - LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK_EQ(toString(result.errors[0]), R"(Type '(number, number) -> string' could not be converted into '(number, number) -> number' -caused by: - Return type is not compatible. Type 'string' could not be converted into 'number')"); -} - -TEST_CASE_FIXTURE(Fixture, "error_detailed_function_mismatch_ret_mult") -{ - CheckResult result = check(R"( -type A = (number, number) -> (number, string) -type B = (number, number) -> (number, boolean) - -local a: A -local b: B = a - )"); - - LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK_EQ(toString(result.errors[0]), - R"(Type '(number, number) -> (number, string)' could not be converted into '(number, number) -> (number, boolean)' -caused by: - Return #2 type is not compatible. Type 'string' could not be converted into 'boolean')"); -} - -TEST_CASE_FIXTURE(Fixture, "prop_access_on_any_with_other_options") -{ - CheckResult result = check(R"( - local function f(thing: any | string) - local foo = thing.SomeRandomKey - end - )"); - - LUAU_REQUIRE_NO_ERRORS(result); -} - -TEST_CASE_FIXTURE(Fixture, "table_function_check_use_after_free") -{ - CheckResult result = check(R"( -local t = {} - -function t.x(value) - for k,v in pairs(t) do end -end - )"); - - LUAU_REQUIRE_NO_ERRORS(result); -} - -TEST_CASE_FIXTURE(Fixture, "table_oop") -{ - CheckResult result = check(R"( - --!strict -local Class = {} -Class.__index = Class - -type Class = typeof(setmetatable({} :: { x: number }, Class)) - -function Class.new(x: number): Class - return setmetatable({x = x}, Class) -end - -function Class.getx(self: Class) - return self.x -end - -function test() - local c = Class.new(42) - local n = c:getx() - local nn = c.x - - print(string.format("%d %d", n, nn)) -end -)"); - - LUAU_REQUIRE_NO_ERRORS(result); -} - TEST_CASE_FIXTURE(Fixture, "recursive_metatable_crash") { CheckResult result = check(R"( @@ -5182,213 +978,4 @@ TEST_CASE_FIXTURE(Fixture, "cli_50041_committing_txnlog_in_apollo_client_error") LUAU_REQUIRE_NO_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "do_not_modify_imported_types") -{ - fileResolver.source["game/A"] = R"( -export type Type = { unrelated: boolean } -return {} - )"; - - fileResolver.source["game/B"] = R"( -local types = require(game.A) -type Type = types.Type -local x: Type = {} -function x:Destroy(): () end - )"; - - CheckResult result = frontend.check("game/B"); - LUAU_REQUIRE_ERRORS(result); -} - -TEST_CASE_FIXTURE(Fixture, "do_not_modify_imported_types_2") -{ - ScopedFastFlag immutableTypes{"LuauImmutableTypes", true}; - - fileResolver.source["game/A"] = R"( -export type Type = { x: { a: number } } -return {} - )"; - - fileResolver.source["game/B"] = R"( -local types = require(game.A) -type Type = types.Type -local x: Type = { x = { a = 2 } } -type Rename = typeof(x.x) - )"; - - CheckResult result = frontend.check("game/B"); - LUAU_REQUIRE_NO_ERRORS(result); -} - -TEST_CASE_FIXTURE(Fixture, "do_not_modify_imported_types_3") -{ - ScopedFastFlag immutableTypes{"LuauImmutableTypes", true}; - - fileResolver.source["game/A"] = R"( -local y = setmetatable({}, {}) -export type Type = { x: typeof(y) } -return { x = y } - )"; - - fileResolver.source["game/B"] = R"( -local types = require(game.A) -type Type = types.Type -local x: Type = types -type Rename = typeof(x.x) - )"; - - CheckResult result = frontend.check("game/B"); - LUAU_REQUIRE_NO_ERRORS(result); -} - -TEST_CASE_FIXTURE(Fixture, "indexing_on_string_singletons") -{ - ScopedFastFlag sff[]{ - {"LuauDiscriminableUnions2", true}, - {"LuauSingletonTypes", true}, - }; - - CheckResult result = check(R"( - local a: string = "hi" - if a == "hi" then - local x = a:byte() - end - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - - CHECK_EQ(R"("hi")", toString(requireTypeAtPosition({3, 22}))); -} - -TEST_CASE_FIXTURE(Fixture, "indexing_on_union_of_string_singletons") -{ - ScopedFastFlag sff[]{ - {"LuauDiscriminableUnions2", true}, - {"LuauSingletonTypes", true}, - }; - - CheckResult result = check(R"( - local a: string = "hi" - if a == "hi" or a == "bye" then - local x = a:byte() - end - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - - CHECK_EQ(R"("bye" | "hi")", toString(requireTypeAtPosition({3, 22}))); -} - -TEST_CASE_FIXTURE(Fixture, "taking_the_length_of_string_singleton") -{ - ScopedFastFlag sff[]{ - {"LuauDiscriminableUnions2", true}, - {"LuauSingletonTypes", true}, - }; - - CheckResult result = check(R"( - local a: string = "hi" - if a == "hi" then - local x = #a - end - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - - CHECK_EQ(R"("hi")", toString(requireTypeAtPosition({3, 23}))); -} - -TEST_CASE_FIXTURE(Fixture, "taking_the_length_of_union_of_string_singleton") -{ - ScopedFastFlag sff[]{ - {"LuauDiscriminableUnions2", true}, - {"LuauSingletonTypes", true}, - }; - - CheckResult result = check(R"( - local a: string = "hi" - if a == "hi" or a == "bye" then - local x = #a - end - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - - CHECK_EQ(R"("bye" | "hi")", toString(requireTypeAtPosition({3, 23}))); -} - -/* - * When we add new properties to an unsealed table, we should do a level check and promote the property type to be at - * the level of the table. - */ -TEST_CASE_FIXTURE(Fixture, "inferred_properties_of_a_table_should_start_with_the_same_TypeLevel_of_that_table") -{ - CheckResult result = check(R"( - --!strict - local T = {} - - local function f(prop) - T[1] = { - prop = prop, - } - end - - local function g() - local l = T[1].prop - end - )"); - - LUAU_REQUIRE_NO_ERRORS(result); -} - -TEST_CASE_FIXTURE(Fixture, "global_singleton_types_are_sealed") -{ - CheckResult result = check(R"( -local function f(x: string) - local p = x:split('a') - p = table.pack(table.unpack(p, 1, #p - 1)) - return p -end - )"); - - LUAU_REQUIRE_NO_ERRORS(result); -} - -TEST_CASE_FIXTURE(Fixture, "function_decl_quantify_right_type") -{ - ScopedFastFlag statFunctionSimplify{"LuauStatFunctionSimplify", true}; - - fileResolver.source["game/isAMagicMock"] = R"( ---!nonstrict -return function(value) - return false -end - )"; - - CheckResult result = check(R"( ---!nonstrict -local MagicMock = {} -MagicMock.is = require(game.isAMagicMock) - -function MagicMock.is(value) - return false -end - )"); - - LUAU_REQUIRE_NO_ERRORS(result); -} - -TEST_CASE_FIXTURE(Fixture, "function_decl_non_self_sealed_overwrite") -{ - ScopedFastFlag statFunctionSimplify{"LuauStatFunctionSimplify", true}; - - CheckResult result = check(R"( -function string.len(): number - return 1 -end - )"); - - LUAU_REQUIRE_ERRORS(result); -} - TEST_SUITE_END(); diff --git a/tests/TypeInfer.typePacks.cpp b/tests/TypeInfer.typePacks.cpp index fcc21c18..130f33d7 100644 --- a/tests/TypeInfer.typePacks.cpp +++ b/tests/TypeInfer.typePacks.cpp @@ -895,4 +895,87 @@ caused by: Type 'boolean' could not be converted into 'string')"); } +// TODO: File a Jira about this +/* +TEST_CASE_FIXTURE(Fixture, "unifying_vararg_pack_with_fixed_length_pack_produces_fixed_length_pack") +{ + CheckResult result = check(R"( + function a(x) return 1 end + a(...) + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + REQUIRE(bool(getMainModule()->getModuleScope()->varargPack)); + + TypePackId varargPack = *getMainModule()->getModuleScope()->varargPack; + + auto iter = begin(varargPack); + auto endIter = end(varargPack); + + CHECK(iter != endIter); + ++iter; + CHECK(iter == endIter); + + CHECK(!iter.tail()); +} +*/ + +TEST_CASE_FIXTURE(Fixture, "dont_ice_if_a_TypePack_is_an_error") +{ + CheckResult result = check(R"( + --!strict + function f(s) + print(s) + return f + end + + f("foo")("bar") + )"); +} + +TEST_CASE_FIXTURE(Fixture, "cyclic_type_packs") +{ + // this has a risk of creating cyclic type packs, causing infinite loops / OOMs + check(R"( +--!nonstrict +_ += _(_,...) +repeat +_ += _(...) +until ... + _ +)"); + + check(R"( +--!nonstrict +_ += _(_(...,...),_(...)) +repeat +until _ +)"); +} + +TEST_CASE_FIXTURE(Fixture, "detect_cyclic_typepacks") +{ + CheckResult result = check(R"( + type ( ... ) ( ) ; + ( ... ) ( - - ... ) ( - ... ) + type = ( ... ) ; + ( ... ) ( ) ( ... ) ; + ( ... ) "" + )"); + + CHECK_LE(0, result.errors.size()); +} + +TEST_CASE_FIXTURE(Fixture, "detect_cyclic_typepacks2") +{ + CheckResult result = check(R"( + function _(l0:((typeof((pcall)))|((((t0)->())|(typeof(-67108864)))|(any)))|(any),...):(((typeof(0))|(any))|(any),typeof(-67108864),any) + xpcall(_,_,_) + _(_,_,_) + end + )"); + + CHECK_LE(0, result.errors.size()); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.unionTypes.test.cpp b/tests/TypeInfer.unionTypes.test.cpp index ad4cecd8..ad1e31e5 100644 --- a/tests/TypeInfer.unionTypes.test.cpp +++ b/tests/TypeInfer.unionTypes.test.cpp @@ -496,4 +496,20 @@ caused by: None of the union options are compatible. For example: Table type 'a' not compatible with type 'X' because the former is missing field 'x')"); } +// We had a bug where a cyclic union caused a stack overflow. +// ex type U = number | U +TEST_CASE_FIXTURE(Fixture, "dont_allow_cyclic_unions_to_be_inferred") +{ + CheckResult result = check(R"( + --!strict + + function f(a, b) + a:g(b or {}) + a:g(b) + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + TEST_SUITE_END(); diff --git a/tests/conformance/basic.lua b/tests/conformance/basic.lua index 78d90077..f803c319 100644 --- a/tests/conformance/basic.lua +++ b/tests/conformance/basic.lua @@ -833,6 +833,17 @@ assert((function() return sum end)() == 105) +-- shrinking array part +assert((function() + local t = table.create(100, 42) + for i=1,90 do t[i] = nil end + t[101] = 42 + local sum = 0 + for _,v in ipairs(t) do sum += v end + for _,v in pairs(t) do sum += v end + return sum +end)() == 462) + -- upvalues: recursive capture assert((function() local function fact(n) return n < 1 and 1 or n * fact(n-1) end return fact(5) end)() == 120) @@ -881,6 +892,14 @@ end)() == "6,8,10") -- typeof == type in absence of custom userdata assert(concat(typeof(5), typeof(nil), typeof({}), typeof(newproxy())) == "number,nil,table,userdata") +-- type/typeof/newproxy interaction with metatables: __type doesn't work intentionally to avoid spoofing +assert((function() + local ud = newproxy(true) + getmetatable(ud).__type = "number" + + return concat(type(ud),typeof(ud)) +end)() == "userdata,userdata") + testgetfenv() -- DONT MOVE THIS LINE return 'OK' diff --git a/tests/conformance/debugger.lua b/tests/conformance/debugger.lua index 6ba99fb9..ec0b412e 100644 --- a/tests/conformance/debugger.lua +++ b/tests/conformance/debugger.lua @@ -3,14 +3,14 @@ print "testing debugger" -- note, this file can't run in isolation from C tests local a = 5 -function foo(b) +function foo(b, ...) print("in foo", b) a = 6 end breakpoint(8) -foo(50) +foo(50, 42) breakpoint(16) -- next line print("here") diff --git a/tests/conformance/errors.lua b/tests/conformance/errors.lua index 751188be..297cf011 100644 --- a/tests/conformance/errors.lua +++ b/tests/conformance/errors.lua @@ -305,4 +305,6 @@ assert(ecall(function() return "a" + "b" end) == "attempt to perform arithmetic assert(ecall(function() return 1 > nil end) == "attempt to compare nil < number") -- note reversed order (by design) assert(ecall(function() return "a" <= 5 end) == "attempt to compare string <= number") +assert(ecall(function() local t = {} setmetatable(t, { __newindex = function(t,i,v) end }) t[nil] = 2 end) == "table index is nil") + return('OK') diff --git a/tests/conformance/interrupt.lua b/tests/conformance/interrupt.lua new file mode 100644 index 00000000..2b127099 --- /dev/null +++ b/tests/conformance/interrupt.lua @@ -0,0 +1,11 @@ +-- This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +print("testing interrupts") + +function foo() + for i=1,10 do end + return +end + +foo() + +return "OK" diff --git a/tools/gdb-printers.py b/tools/gdb_printers.py similarity index 100% rename from tools/gdb-printers.py rename to tools/gdb_printers.py diff --git a/tools/lldb-formatters.lldb b/tools/lldb-formatters.lldb deleted file mode 100644 index 3868ac20..00000000 --- a/tools/lldb-formatters.lldb +++ /dev/null @@ -1,2 +0,0 @@ -type synthetic add -x "^Luau::Variant<.+>$" -l LuauVisualize.LuauVariantSyntheticChildrenProvider -type summary add -x "^Luau::Variant<.+>$" -l LuauVisualize.luau_variant_summary diff --git a/tools/lldb_formatters.lldb b/tools/lldb_formatters.lldb new file mode 100644 index 00000000..f6fa6cf5 --- /dev/null +++ b/tools/lldb_formatters.lldb @@ -0,0 +1,2 @@ +type synthetic add -x "^Luau::Variant<.+>$" -l lldb_formatters.LuauVariantSyntheticChildrenProvider +type summary add -x "^Luau::Variant<.+>$" -l lldb_formatters.luau_variant_summary diff --git a/tools/LuauVisualize.py b/tools/lldb_formatters.py similarity index 100% rename from tools/LuauVisualize.py rename to tools/lldb_formatters.py From 57faf7aaf2d6dd20eb69bc4088e61549cfa6552f Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Thu, 17 Mar 2022 17:32:02 -0700 Subject: [PATCH 196/399] Lower the stack limit to make tests pass in debug --- tests/TypeInfer.test.cpp | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/TypeInfer.test.cpp b/tests/TypeInfer.test.cpp index 571d0f8d..660ddcfc 100644 --- a/tests/TypeInfer.test.cpp +++ b/tests/TypeInfer.test.cpp @@ -334,7 +334,7 @@ TEST_CASE_FIXTURE(Fixture, "check_expr_recursion_limit") #if defined(LUAU_ENABLE_ASAN) int limit = 250; #elif defined(_DEBUG) || defined(_NOOPT) - int limit = 350; + int limit = 300; #else int limit = 600; #endif From 362428f8b4b6f5c9d43f4daf55bcf7873f536c3f Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Thu, 17 Mar 2022 17:46:04 -0700 Subject: [PATCH 197/399] Sync to upstream/release/519 (#422) --- Analysis/include/Luau/Error.h | 1 + Analysis/include/Luau/Unifier.h | 2 + Analysis/src/Autocomplete.cpp | 198 +- Analysis/src/EmbeddedBuiltinDefinitions.cpp | 144 +- Analysis/src/Error.cpp | 11 +- Analysis/src/Module.cpp | 7 + Analysis/src/TypeInfer.cpp | 127 +- Analysis/src/TypeVar.cpp | 10 +- Analysis/src/Unifier.cpp | 78 +- Ast/src/Parser.cpp | 117 +- Compiler/src/Builtins.cpp | 4 +- Compiler/src/Compiler.cpp | 4 - Sources.cmake | 7 + VM/include/lua.h | 15 +- VM/src/lapi.cpp | 20 +- VM/src/lbaselib.cpp | 16 +- VM/src/lgc.h | 4 +- VM/src/lmem.cpp | 2 +- VM/src/lstring.cpp | 2 +- VM/src/ltable.cpp | 71 +- VM/src/ltm.cpp | 3 +- VM/src/ludata.cpp | 8 +- VM/src/ludata.h | 3 + VM/src/lvmexecute.cpp | 2 +- tests/Autocomplete.test.cpp | 162 + tests/Compiler.test.cpp | 2 - tests/Conformance.test.cpp | 142 +- tests/Linter.test.cpp | 2 - tests/Module.test.cpp | 10 +- tests/NonstrictMode.test.cpp | 22 +- tests/TypeInfer.aliases.test.cpp | 21 + tests/TypeInfer.anyerror.test.cpp | 335 ++ tests/TypeInfer.builtins.test.cpp | 60 + tests/TypeInfer.definitions.test.cpp | 18 + tests/TypeInfer.functions.test.cpp | 1338 +++++ tests/TypeInfer.generics.test.cpp | 301 ++ tests/TypeInfer.intersectionTypes.test.cpp | 28 + tests/TypeInfer.loops.test.cpp | 473 ++ tests/TypeInfer.modules.test.cpp | 310 ++ tests/TypeInfer.oop.test.cpp | 275 + tests/TypeInfer.operators.test.cpp | 759 +++ tests/TypeInfer.primitives.test.cpp | 100 + tests/TypeInfer.refinements.test.cpp | 18 + tests/TypeInfer.singletons.test.cpp | 116 +- tests/TypeInfer.tables.test.cpp | 500 ++ tests/TypeInfer.test.cpp | 4415 +---------------- tests/TypeInfer.typePacks.cpp | 83 + tests/TypeInfer.unionTypes.test.cpp | 16 + tests/conformance/basic.lua | 19 + tests/conformance/debugger.lua | 4 +- tests/conformance/errors.lua | 2 + tests/conformance/interrupt.lua | 11 + tools/{gdb-printers.py => gdb_printers.py} | 0 tools/lldb-formatters.lldb | 2 - tools/lldb_formatters.lldb | 2 + .../{LuauVisualize.py => lldb_formatters.py} | 0 56 files changed, 5640 insertions(+), 4762 deletions(-) create mode 100644 tests/TypeInfer.anyerror.test.cpp create mode 100644 tests/TypeInfer.functions.test.cpp create mode 100644 tests/TypeInfer.loops.test.cpp create mode 100644 tests/TypeInfer.modules.test.cpp create mode 100644 tests/TypeInfer.oop.test.cpp create mode 100644 tests/TypeInfer.operators.test.cpp create mode 100644 tests/TypeInfer.primitives.test.cpp create mode 100644 tests/conformance/interrupt.lua rename tools/{gdb-printers.py => gdb_printers.py} (100%) delete mode 100644 tools/lldb-formatters.lldb create mode 100644 tools/lldb_formatters.lldb rename tools/{LuauVisualize.py => lldb_formatters.py} (100%) diff --git a/Analysis/include/Luau/Error.h b/Analysis/include/Luau/Error.h index a71e0224..72350255 100644 --- a/Analysis/include/Luau/Error.h +++ b/Analysis/include/Luau/Error.h @@ -107,6 +107,7 @@ struct FunctionDoesNotTakeSelf struct FunctionRequiresSelf { + // TODO: Delete with LuauAnyInIsOptionalIsOptional int requiredExtraNils = 0; bool operator==(const FunctionRequiresSelf& rhs) const; diff --git a/Analysis/include/Luau/Unifier.h b/Analysis/include/Luau/Unifier.h index 71958f4a..f1ffbcc0 100644 --- a/Analysis/include/Luau/Unifier.h +++ b/Analysis/include/Luau/Unifier.h @@ -86,6 +86,8 @@ private: void tryUnifyIndexer(const TableIndexer& subIndexer, const TableIndexer& superIndexer); TypeId widen(TypeId ty); + TypePackId widen(TypePackId tp); + TypeId deeplyOptional(TypeId ty, std::unordered_map seen = {}); void cacheResult(TypeId subTy, TypeId superTy); diff --git a/Analysis/src/Autocomplete.cpp b/Analysis/src/Autocomplete.cpp index e94c432f..492edf25 100644 --- a/Analysis/src/Autocomplete.cpp +++ b/Analysis/src/Autocomplete.cpp @@ -14,6 +14,7 @@ #include LUAU_FASTFLAGVARIABLE(LuauIfElseExprFixCompletionIssue, false); +LUAU_FASTFLAG(LuauSelfCallAutocompleteFix) static const std::unordered_set kStatementStartingKeywords = { "while", "if", "local", "repeat", "function", "do", "for", "return", "break", "continue", "type", "export"}; @@ -228,11 +229,22 @@ static std::optional findExpectedTypeAt(const Module& module, AstNode* n return *it; } +static bool checkTypeMatch(TypeArena* typeArena, TypeId subTy, TypeId superTy) +{ + InternalErrorReporter iceReporter; + UnifierSharedState unifierState(&iceReporter); + Unifier unifier(typeArena, Mode::Strict, Location(), Variance::Covariant, unifierState); + + return unifier.canUnify(subTy, superTy).empty(); +} + static TypeCorrectKind checkTypeCorrectKind(const Module& module, TypeArena* typeArena, AstNode* node, Position position, TypeId ty) { ty = follow(ty); auto canUnify = [&typeArena](TypeId subTy, TypeId superTy) { + LUAU_ASSERT(!FFlag::LuauSelfCallAutocompleteFix); + InternalErrorReporter iceReporter; UnifierSharedState unifierState(&iceReporter); Unifier unifier(typeArena, Mode::Strict, Location(), Variance::Covariant, unifierState); @@ -249,20 +261,30 @@ static TypeCorrectKind checkTypeCorrectKind(const Module& module, TypeArena* typ TypeId expectedType = follow(*typeAtPosition); - auto checkFunctionType = [&canUnify, &expectedType](const FunctionTypeVar* ftv) { - auto [retHead, retTail] = flatten(ftv->retType); - - if (!retHead.empty() && canUnify(retHead.front(), expectedType)) - return true; - - // We might only have a variadic tail pack, check if the element is compatible - if (retTail) + auto checkFunctionType = [typeArena, &canUnify, &expectedType](const FunctionTypeVar* ftv) { + if (FFlag::LuauSelfCallAutocompleteFix) { - if (const VariadicTypePack* vtp = get(follow(*retTail)); vtp && canUnify(vtp->ty, expectedType)) - return true; - } + if (std::optional firstRetTy = first(ftv->retType)) + return checkTypeMatch(typeArena, *firstRetTy, expectedType); - return false; + return false; + } + else + { + auto [retHead, retTail] = flatten(ftv->retType); + + if (!retHead.empty() && canUnify(retHead.front(), expectedType)) + return true; + + // We might only have a variadic tail pack, check if the element is compatible + if (retTail) + { + if (const VariadicTypePack* vtp = get(follow(*retTail)); vtp && canUnify(vtp->ty, expectedType)) + return true; + } + + return false; + } }; // We also want to suggest functions that return compatible result @@ -281,7 +303,10 @@ static TypeCorrectKind checkTypeCorrectKind(const Module& module, TypeArena* typ } } - return canUnify(ty, expectedType) ? TypeCorrectKind::Correct : TypeCorrectKind::None; + if (FFlag::LuauSelfCallAutocompleteFix) + return checkTypeMatch(typeArena, ty, expectedType) ? TypeCorrectKind::Correct : TypeCorrectKind::None; + else + return canUnify(ty, expectedType) ? TypeCorrectKind::Correct : TypeCorrectKind::None; } enum class PropIndexType @@ -291,16 +316,22 @@ enum class PropIndexType Key, }; -static void autocompleteProps(const Module& module, TypeArena* typeArena, TypeId ty, PropIndexType indexType, const std::vector& nodes, - AutocompleteEntryMap& result, std::unordered_set& seen, std::optional containingClass = std::nullopt) +static void autocompleteProps(const Module& module, TypeArena* typeArena, TypeId rootTy, TypeId ty, PropIndexType indexType, + const std::vector& nodes, AutocompleteEntryMap& result, std::unordered_set& seen, + std::optional containingClass = std::nullopt) { + if (FFlag::LuauSelfCallAutocompleteFix) + rootTy = follow(rootTy); + ty = follow(ty); if (seen.count(ty)) return; seen.insert(ty); - auto isWrongIndexer = [indexType, useStrictFunctionIndexers = !!get(ty)](Luau::TypeId type) { + auto isWrongIndexer_DEPRECATED = [indexType, useStrictFunctionIndexers = !!get(ty)](Luau::TypeId type) { + LUAU_ASSERT(!FFlag::LuauSelfCallAutocompleteFix); + if (indexType == PropIndexType::Key) return false; @@ -331,6 +362,48 @@ static void autocompleteProps(const Module& module, TypeArena* typeArena, TypeId return colonIndex; } }; + auto isWrongIndexer = [typeArena, rootTy, indexType](Luau::TypeId type) { + LUAU_ASSERT(FFlag::LuauSelfCallAutocompleteFix); + + if (indexType == PropIndexType::Key) + return false; + + bool calledWithSelf = indexType == PropIndexType::Colon; + + auto isCompatibleCall = [typeArena, rootTy, calledWithSelf](const FunctionTypeVar* ftv) { + if (get(rootTy)) + { + // Calls on classes require strict match between how function is declared and how it's called + return calledWithSelf == ftv->hasSelf; + } + + if (std::optional firstArgTy = first(ftv->argTypes)) + { + if (checkTypeMatch(typeArena, rootTy, *firstArgTy)) + return calledWithSelf; + } + + return !calledWithSelf; + }; + + if (const FunctionTypeVar* ftv = get(type)) + return !isCompatibleCall(ftv); + + // For intersections, any part that is successful makes the whole call successful + if (const IntersectionTypeVar* itv = get(type)) + { + for (auto subType : itv->parts) + { + if (const FunctionTypeVar* ftv = get(Luau::follow(subType))) + { + if (isCompatibleCall(ftv)) + return false; + } + } + } + + return calledWithSelf; + }; auto fillProps = [&](const ClassTypeVar::Props& props) { for (const auto& [name, prop] : props) @@ -349,7 +422,7 @@ static void autocompleteProps(const Module& module, TypeArena* typeArena, TypeId AutocompleteEntryKind::Property, type, prop.deprecated, - isWrongIndexer(type), + FFlag::LuauSelfCallAutocompleteFix ? isWrongIndexer(type) : isWrongIndexer_DEPRECATED(type), typeCorrect, containingClass, &prop, @@ -361,34 +434,60 @@ static void autocompleteProps(const Module& module, TypeArena* typeArena, TypeId } }; - if (auto cls = get(ty)) - { - containingClass = containingClass.value_or(cls); - fillProps(cls->props); - if (cls->parent) - autocompleteProps(module, typeArena, *cls->parent, indexType, nodes, result, seen, cls); - } - else if (auto tbl = get(ty)) - fillProps(tbl->props); - else if (auto mt = get(ty)) - { - autocompleteProps(module, typeArena, mt->table, indexType, nodes, result, seen); - - auto mtable = get(mt->metatable); - if (!mtable) - return; - + auto fillMetatableProps = [&](const TableTypeVar* mtable) { auto indexIt = mtable->props.find("__index"); if (indexIt != mtable->props.end()) { TypeId followed = follow(indexIt->second.type); if (get(followed) || get(followed)) - autocompleteProps(module, typeArena, followed, indexType, nodes, result, seen); + { + autocompleteProps(module, typeArena, rootTy, followed, indexType, nodes, result, seen); + } else if (auto indexFunction = get(followed)) { std::optional indexFunctionResult = first(indexFunction->retType); if (indexFunctionResult) - autocompleteProps(module, typeArena, *indexFunctionResult, indexType, nodes, result, seen); + autocompleteProps(module, typeArena, rootTy, *indexFunctionResult, indexType, nodes, result, seen); + } + } + }; + + if (auto cls = get(ty)) + { + containingClass = containingClass.value_or(cls); + fillProps(cls->props); + if (cls->parent) + autocompleteProps(module, typeArena, rootTy, *cls->parent, indexType, nodes, result, seen, cls); + } + else if (auto tbl = get(ty)) + fillProps(tbl->props); + else if (auto mt = get(ty)) + { + autocompleteProps(module, typeArena, rootTy, mt->table, indexType, nodes, result, seen); + + if (FFlag::LuauSelfCallAutocompleteFix) + { + if (auto mtable = get(mt->metatable)) + fillMetatableProps(mtable); + } + else + { + auto mtable = get(mt->metatable); + if (!mtable) + return; + + auto indexIt = mtable->props.find("__index"); + if (indexIt != mtable->props.end()) + { + TypeId followed = follow(indexIt->second.type); + if (get(followed) || get(followed)) + autocompleteProps(module, typeArena, rootTy, followed, indexType, nodes, result, seen); + else if (auto indexFunction = get(followed)) + { + std::optional indexFunctionResult = first(indexFunction->retType); + if (indexFunctionResult) + autocompleteProps(module, typeArena, rootTy, *indexFunctionResult, indexType, nodes, result, seen); + } } } } @@ -400,7 +499,7 @@ static void autocompleteProps(const Module& module, TypeArena* typeArena, TypeId AutocompleteEntryMap inner; std::unordered_set innerSeen = seen; - autocompleteProps(module, typeArena, ty, indexType, nodes, inner, innerSeen); + autocompleteProps(module, typeArena, rootTy, ty, indexType, nodes, inner, innerSeen); for (auto& pair : inner) result.insert(pair); @@ -423,14 +522,17 @@ static void autocompleteProps(const Module& module, TypeArena* typeArena, TypeId if (iter == endIter) return; - autocompleteProps(module, typeArena, *iter, indexType, nodes, result, seen); + autocompleteProps(module, typeArena, rootTy, *iter, indexType, nodes, result, seen); ++iter; while (iter != endIter) { AutocompleteEntryMap inner; - std::unordered_set innerSeen = seen; + std::unordered_set innerSeen; + + if (!FFlag::LuauSelfCallAutocompleteFix) + innerSeen = seen; if (isNil(*iter)) { @@ -438,7 +540,7 @@ static void autocompleteProps(const Module& module, TypeArena* typeArena, TypeId continue; } - autocompleteProps(module, typeArena, *iter, indexType, nodes, inner, innerSeen); + autocompleteProps(module, typeArena, rootTy, *iter, indexType, nodes, inner, innerSeen); std::unordered_set toRemove; @@ -455,6 +557,18 @@ static void autocompleteProps(const Module& module, TypeArena* typeArena, TypeId ++iter; } } + else if (auto pt = get(ty); pt && FFlag::LuauSelfCallAutocompleteFix) + { + if (pt->metatable) + { + if (auto mtable = get(*pt->metatable)) + fillMetatableProps(mtable); + } + } + else if (FFlag::LuauSelfCallAutocompleteFix && get(get(ty))) + { + autocompleteProps(module, typeArena, rootTy, getSingletonTypes().stringType, indexType, nodes, result, seen); + } } static void autocompleteKeywords( @@ -482,7 +596,7 @@ static void autocompleteProps( const Module& module, TypeArena* typeArena, TypeId ty, PropIndexType indexType, const std::vector& nodes, AutocompleteEntryMap& result) { std::unordered_set seen; - autocompleteProps(module, typeArena, ty, indexType, nodes, result, seen); + autocompleteProps(module, typeArena, ty, ty, indexType, nodes, result, seen); } AutocompleteEntryMap autocompleteProps( @@ -1352,7 +1466,7 @@ static AutocompleteResult autocomplete(const SourceModule& sourceModule, const M TypeId ty = follow(*it); PropIndexType indexType = indexName->op == ':' ? PropIndexType::Colon : PropIndexType::Point; - if (isString(ty)) + if (!FFlag::LuauSelfCallAutocompleteFix && isString(ty)) return {autocompleteProps(*module, typeArena, typeChecker.globalScope->bindings[AstName{"string"}].typeId, indexType, finder.ancestry), finder.ancestry}; else diff --git a/Analysis/src/EmbeddedBuiltinDefinitions.cpp b/Analysis/src/EmbeddedBuiltinDefinitions.cpp index 471b61ad..be3fcd7d 100644 --- a/Analysis/src/EmbeddedBuiltinDefinitions.cpp +++ b/Analysis/src/EmbeddedBuiltinDefinitions.cpp @@ -95,104 +95,104 @@ declare os: { declare function require(target: any): any -declare function getfenv(target: any?): { [string]: any } +declare function getfenv(target: any): { [string]: any } declare _G: any declare _VERSION: string declare function gcinfo(): number - declare function print(...: T...) +declare function print(...: T...) - declare function type(value: T): string - declare function typeof(value: T): string - - -- `assert` has a magic function attached that will give more detailed type information - declare function assert(value: T, errorMessage: string?): T +declare function type(value: T): string +declare function typeof(value: T): string - declare function error(message: T, level: number?) +-- `assert` has a magic function attached that will give more detailed type information +declare function assert(value: T, errorMessage: string?): T - declare function tostring(value: T): string - declare function tonumber(value: T, radix: number?): number? +declare function error(message: T, level: number?) - declare function rawequal(a: T1, b: T2): boolean - declare function rawget(tab: {[K]: V}, k: K): V - declare function rawset(tab: {[K]: V}, k: K, v: V): {[K]: V} +declare function tostring(value: T): string +declare function tonumber(value: T, radix: number?): number? - declare function setfenv(target: number | (T...) -> R..., env: {[string]: any}): ((T...) -> R...)? +declare function rawequal(a: T1, b: T2): boolean +declare function rawget(tab: {[K]: V}, k: K): V +declare function rawset(tab: {[K]: V}, k: K, v: V): {[K]: V} - declare function ipairs(tab: {V}): (({V}, number) -> (number, V), {V}, number) +declare function setfenv(target: number | (T...) -> R..., env: {[string]: any}): ((T...) -> R...)? - declare function pcall(f: (A...) -> R..., ...: A...): (boolean, R...) +declare function ipairs(tab: {V}): (({V}, number) -> (number, V), {V}, number) - -- FIXME: The actual type of `xpcall` is: - -- (f: (A...) -> R1..., err: (E) -> R2..., A...) -> (true, R1...) | (false, R2...) - -- Since we can't represent the return value, we use (boolean, R1...). - declare function xpcall(f: (A...) -> R1..., err: (E) -> R2..., ...: A...): (boolean, R1...) +declare function pcall(f: (A...) -> R..., ...: A...): (boolean, R...) - -- `select` has a magic function attached to provide more detailed type information - declare function select(i: string | number, ...: A...): ...any +-- FIXME: The actual type of `xpcall` is: +-- (f: (A...) -> R1..., err: (E) -> R2..., A...) -> (true, R1...) | (false, R2...) +-- Since we can't represent the return value, we use (boolean, R1...). +declare function xpcall(f: (A...) -> R1..., err: (E) -> R2..., ...: A...): (boolean, R1...) - -- FIXME: This type is not entirely correct - `loadstring` returns a function or - -- (nil, string). - declare function loadstring(src: string, chunkname: string?): (((A...) -> any)?, string?) +-- `select` has a magic function attached to provide more detailed type information +declare function select(i: string | number, ...: A...): ...any - declare function newproxy(mt: boolean?): any +-- FIXME: This type is not entirely correct - `loadstring` returns a function or +-- (nil, string). +declare function loadstring(src: string, chunkname: string?): (((A...) -> any)?, string?) - declare coroutine: { - create: ((A...) -> R...) -> thread, - resume: (thread, A...) -> (boolean, R...), - running: () -> thread, - status: (thread) -> string, - -- FIXME: This technically returns a function, but we can't represent this yet. - wrap: ((A...) -> R...) -> any, - yield: (A...) -> R..., - isyieldable: () -> boolean, - close: (thread) -> (boolean, any?) - } +declare function newproxy(mt: boolean?): any - declare table: { - concat: ({V}, string?, number?, number?) -> string, - insert: (({V}, V) -> ()) & (({V}, number, V) -> ()), - maxn: ({V}) -> number, - remove: ({V}, number?) -> V?, - sort: ({V}, ((V, V) -> boolean)?) -> (), - create: (number, V?) -> {V}, - find: ({V}, V, number?) -> number?, +declare coroutine: { + create: ((A...) -> R...) -> thread, + resume: (thread, A...) -> (boolean, R...), + running: () -> thread, + status: (thread) -> string, + -- FIXME: This technically returns a function, but we can't represent this yet. + wrap: ((A...) -> R...) -> any, + yield: (A...) -> R..., + isyieldable: () -> boolean, + close: (thread) -> (boolean, any) +} - unpack: ({V}, number?, number?) -> ...V, - pack: (...V) -> { n: number, [number]: V }, +declare table: { + concat: ({V}, string?, number?, number?) -> string, + insert: (({V}, V) -> ()) & (({V}, number, V) -> ()), + maxn: ({V}) -> number, + remove: ({V}, number?) -> V?, + sort: ({V}, ((V, V) -> boolean)?) -> (), + create: (number, V?) -> {V}, + find: ({V}, V, number?) -> number?, - getn: ({V}) -> number, - foreach: ({[K]: V}, (K, V) -> ()) -> (), - foreachi: ({V}, (number, V) -> ()) -> (), + unpack: ({V}, number?, number?) -> ...V, + pack: (...V) -> { n: number, [number]: V }, - move: ({V}, number, number, number, {V}?) -> {V}, - clear: ({[K]: V}) -> (), + getn: ({V}) -> number, + foreach: ({[K]: V}, (K, V) -> ()) -> (), + foreachi: ({V}, (number, V) -> ()) -> (), - isfrozen: ({[K]: V}) -> boolean, - } + move: ({V}, number, number, number, {V}?) -> {V}, + clear: ({[K]: V}) -> (), - declare debug: { - info: ((thread, number, string) -> R...) & ((number, string) -> R...) & (((A...) -> R1..., string) -> R2...), - traceback: ((string?, number?) -> string) & ((thread, string?, number?) -> string), - } + isfrozen: ({[K]: V}) -> boolean, +} - declare utf8: { - char: (number, ...number) -> string, - charpattern: string, - codes: (string) -> ((string, number) -> (number, number), string, number), - -- FIXME - codepoint: (string, number?, number?) -> (number, ...number), - len: (string, number?, number?) -> (number?, number?), - offset: (string, number?, number?) -> number, - nfdnormalize: (string) -> string, - nfcnormalize: (string) -> string, - graphemes: (string, number?, number?) -> (() -> (number, number)), - } +declare debug: { + info: ((thread, number, string) -> R...) & ((number, string) -> R...) & (((A...) -> R1..., string) -> R2...), + traceback: ((string?, number?) -> string) & ((thread, string?, number?) -> string), +} - -- Cannot use `typeof` here because it will produce a polytype when we expect a monotype. - declare function unpack(tab: {V}, i: number?, j: number?): ...V +declare utf8: { + char: (number, ...number) -> string, + charpattern: string, + codes: (string) -> ((string, number) -> (number, number), string, number), + -- FIXME + codepoint: (string, number?, number?) -> (number, ...number), + len: (string, number?, number?) -> (number?, number?), + offset: (string, number?, number?) -> number, + nfdnormalize: (string) -> string, + nfcnormalize: (string) -> string, + graphemes: (string, number?, number?) -> (() -> (number, number)), +} + +-- Cannot use `typeof` here because it will produce a polytype when we expect a monotype. +declare function unpack(tab: {V}, i: number?, j: number?): ...V )BUILTIN_SRC"; diff --git a/Analysis/src/Error.cpp b/Analysis/src/Error.cpp index 88069f1f..26d3b76d 100644 --- a/Analysis/src/Error.cpp +++ b/Analysis/src/Error.cpp @@ -7,6 +7,8 @@ #include +LUAU_FASTFLAGVARIABLE(BetterDiagnosticCodesInStudio, false); + static std::string wrongNumberOfArgsString(size_t expectedCount, size_t actualCount, const char* argPrefix = nullptr, bool isVariadic = false) { std::string s = "expects "; @@ -223,7 +225,14 @@ struct ErrorConverter std::string operator()(const Luau::SyntaxError& e) const { - return "Syntax error: " + e.message; + if (FFlag::BetterDiagnosticCodesInStudio) + { + return e.message; + } + else + { + return "Syntax error: " + e.message; + } } std::string operator()(const Luau::CodeTooComplex&) const diff --git a/Analysis/src/Module.cpp b/Analysis/src/Module.cpp index 76dc72d2..a330a98d 100644 --- a/Analysis/src/Module.cpp +++ b/Analysis/src/Module.cpp @@ -14,6 +14,7 @@ LUAU_FASTFLAGVARIABLE(DebugLuauFreezeArena, false) LUAU_FASTFLAGVARIABLE(DebugLuauTrackOwningArena, false) // Remove with FFlagLuauImmutableTypes LUAU_FASTINTVARIABLE(LuauTypeCloneRecursionLimit, 300) +LUAU_FASTFLAGVARIABLE(LuauCloneDeclaredGlobals, false) LUAU_FASTFLAG(LuauImmutableTypes) namespace Luau @@ -536,6 +537,12 @@ bool Module::clonePublicInterface() if (get(follow(ty))) *asMutable(ty) = AnyTypeVar{}; + if (FFlag::LuauCloneDeclaredGlobals) + { + for (auto& [name, ty] : declaredGlobals) + ty = clone(ty, interfaceTypes, seenTypes, seenTypePacks, cloneState); + } + freeze(internalTypes); freeze(interfaceTypes); diff --git a/Analysis/src/TypeInfer.cpp b/Analysis/src/TypeInfer.cpp index 3fe4c90e..41e8ce55 100644 --- a/Analysis/src/TypeInfer.cpp +++ b/Analysis/src/TypeInfer.cpp @@ -29,22 +29,24 @@ LUAU_FASTFLAGVARIABLE(LuauRecursiveTypeParameterRestriction, false) LUAU_FASTFLAGVARIABLE(LuauGenericFunctionsDontCacheTypeParams, false) LUAU_FASTFLAGVARIABLE(LuauImmutableTypes, false) LUAU_FASTFLAGVARIABLE(LuauSealExports, false) +LUAU_FASTFLAGVARIABLE(LuauSelfCallAutocompleteFix, false) LUAU_FASTFLAGVARIABLE(LuauSingletonTypes, false) LUAU_FASTFLAGVARIABLE(LuauDiscriminableUnions2, false) LUAU_FASTFLAGVARIABLE(LuauExpectedTypesOfProperties, false) LUAU_FASTFLAGVARIABLE(LuauErrorRecoveryType, false) LUAU_FASTFLAGVARIABLE(LuauOnlyMutateInstantiatedTables, false) LUAU_FASTFLAGVARIABLE(LuauPropertiesGetExpectedType, false) -LUAU_FASTFLAGVARIABLE(LuauStatFunctionSimplify, false) +LUAU_FASTFLAGVARIABLE(LuauStatFunctionSimplify2, false) LUAU_FASTFLAGVARIABLE(LuauUnsealedTableLiteral, false) LUAU_FASTFLAGVARIABLE(LuauTwoPassAliasDefinitionFix, false) LUAU_FASTFLAGVARIABLE(LuauAssertStripsFalsyTypes, false) LUAU_FASTFLAGVARIABLE(LuauReturnAnyInsteadOfICE, false) // Eventually removed as false. -LUAU_FASTFLAG(LuauWidenIfSupertypeIsFree) +LUAU_FASTFLAG(LuauWidenIfSupertypeIsFree2) LUAU_FASTFLAGVARIABLE(LuauDoNotTryToReduce, false) LUAU_FASTFLAGVARIABLE(LuauDoNotAccidentallyDependOnPointerOrdering, false) LUAU_FASTFLAGVARIABLE(LuauFixArgumentCountMismatchAmountWithGenericTypes, false) LUAU_FASTFLAGVARIABLE(LuauFixIncorrectLineNumberDuplicateType, false) +LUAU_FASTFLAG(LuauAnyInIsOptionalIsOptional) namespace Luau { @@ -1099,7 +1101,7 @@ void TypeChecker::check(const ScopePtr& scope, TypeId ty, const ScopePtr& funSco scope->bindings[name->local] = {anyIfNonstrict(quantify(funScope, ty, name->local->location)), name->local->location}; return; } - else if (auto name = function.name->as(); name && FFlag::LuauStatFunctionSimplify) + else if (auto name = function.name->as(); name && FFlag::LuauStatFunctionSimplify2) { TypeId exprTy = checkExpr(scope, *name->expr).type; TableTypeVar* ttv = getMutableTableType(exprTy); @@ -1111,7 +1113,10 @@ void TypeChecker::check(const ScopePtr& scope, TypeId ty, const ScopePtr& funSco reportError(TypeError{function.location, OnlyTablesCanHaveMethods{exprTy}}); } else if (ttv->state == TableState::Sealed) - reportError(TypeError{function.location, CannotExtendTable{exprTy, CannotExtendTable::Property, name->index.value}}); + { + if (!ttv->indexer || !isPrim(ttv->indexer->indexType, PrimitiveTypeVar::String)) + reportError(TypeError{function.location, CannotExtendTable{exprTy, CannotExtendTable::Property, name->index.value}}); + } ty = follow(ty); @@ -1134,7 +1139,7 @@ void TypeChecker::check(const ScopePtr& scope, TypeId ty, const ScopePtr& funSco if (ttv && ttv->state != TableState::Sealed) ttv->props[name->index.value] = {follow(quantify(funScope, ty, name->indexLocation)), /* deprecated */ false, {}, name->indexLocation}; } - else if (FFlag::LuauStatFunctionSimplify) + else if (FFlag::LuauStatFunctionSimplify2) { LUAU_ASSERT(function.name->is()); @@ -1144,7 +1149,7 @@ void TypeChecker::check(const ScopePtr& scope, TypeId ty, const ScopePtr& funSco } else if (function.func->self) { - LUAU_ASSERT(!FFlag::LuauStatFunctionSimplify); + LUAU_ASSERT(!FFlag::LuauStatFunctionSimplify2); AstExprIndexName* indexName = function.name->as(); if (!indexName) @@ -1183,7 +1188,7 @@ void TypeChecker::check(const ScopePtr& scope, TypeId ty, const ScopePtr& funSco } else { - LUAU_ASSERT(!FFlag::LuauStatFunctionSimplify); + LUAU_ASSERT(!FFlag::LuauStatFunctionSimplify2); TypeId leftType = checkLValueBinding(scope, *function.name); @@ -1410,6 +1415,9 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatDeclareClass& declar { ftv->argNames.insert(ftv->argNames.begin(), FunctionArgument{"self", {}}); ftv->argTypes = addTypePack(TypePack{{classTy}, ftv->argTypes}); + + if (FFlag::LuauSelfCallAutocompleteFix) + ftv->hasSelf = true; } } @@ -1883,19 +1891,27 @@ std::optional TypeChecker::tryStripUnionFromNil(TypeId ty) { if (const UnionTypeVar* utv = get(ty)) { - bool hasNil = false; - - for (TypeId option : utv) + if (FFlag::LuauAnyInIsOptionalIsOptional) { - if (isNil(option)) - { - hasNil = true; - break; - } + if (!std::any_of(begin(utv), end(utv), isNil)) + return ty; } + else + { + bool hasNil = false; - if (!hasNil) - return ty; + for (TypeId option : utv) + { + if (isNil(option)) + { + hasNil = true; + break; + } + } + + if (!hasNil) + return ty; + } std::vector result; @@ -1916,14 +1932,34 @@ std::optional TypeChecker::tryStripUnionFromNil(TypeId ty) TypeId TypeChecker::stripFromNilAndReport(TypeId ty, const Location& location) { - if (isOptional(ty)) + if (FFlag::LuauAnyInIsOptionalIsOptional) { - if (std::optional strippedUnion = tryStripUnionFromNil(follow(ty))) + ty = follow(ty); + + if (auto utv = get(ty)) + { + if (!std::any_of(begin(utv), end(utv), isNil)) + return ty; + + } + + if (std::optional strippedUnion = tryStripUnionFromNil(ty)) { reportError(location, OptionalValueAccess{ty}); return follow(*strippedUnion); } } + else + { + if (isOptional(ty)) + { + if (std::optional strippedUnion = tryStripUnionFromNil(follow(ty))) + { + reportError(location, OptionalValueAccess{ty}); + return follow(*strippedUnion); + } + } + } return ty; } @@ -2935,9 +2971,25 @@ TypeId TypeChecker::checkFunctionName(const ScopePtr& scope, AstExpr& funName, T return errorRecoveryType(scope); } - // Cannot extend sealed table, but we dont report an error here because it will be reported during AstStatFunction check - if (lhsType->persistent || ttv->state == TableState::Sealed) - return errorRecoveryType(scope); + if (FFlag::LuauStatFunctionSimplify2) + { + if (lhsType->persistent) + return errorRecoveryType(scope); + + // Cannot extend sealed table, but we dont report an error here because it will be reported during AstStatFunction check + if (ttv->state == TableState::Sealed) + { + if (ttv->indexer && isPrim(ttv->indexer->indexType, PrimitiveTypeVar::String)) + return ttv->indexer->indexResultType; + else + return errorRecoveryType(scope); + } + } + else + { + if (lhsType->persistent || ttv->state == TableState::Sealed) + return errorRecoveryType(scope); + } Name name = indexName->index.value; @@ -3393,7 +3445,7 @@ void TypeChecker::checkArgumentList( else if (state.log.getMutable(t)) { } // ok - else if (isNonstrictMode() && state.log.getMutable(t)) + else if (!FFlag::LuauAnyInIsOptionalIsOptional && isNonstrictMode() && state.log.getMutable(t)) { } // ok else @@ -3467,7 +3519,11 @@ void TypeChecker::checkArgumentList( } TypePackId varPack = addTypePack(TypePackVar{TypePack{rest, argIter.tail()}}); - state.tryUnify(varPack, tail); + if (FFlag::LuauWidenIfSupertypeIsFree2) + state.tryUnify(varPack, tail); + else + state.tryUnify(tail, varPack); + return; } else if (state.log.getMutable(tail)) @@ -3542,6 +3598,23 @@ ExprResult TypeChecker::checkExprPack(const ScopePtr& scope, const A actualFunctionType = follow(actualFunctionType); + TypePackId retPack; + if (!FFlag::LuauWidenIfSupertypeIsFree2) + { + retPack = freshTypePack(scope->level); + } + else + { + if (auto free = get(actualFunctionType)) + { + retPack = freshTypePack(free->level); + TypePackId freshArgPack = freshTypePack(free->level); + *asMutable(actualFunctionType) = FunctionTypeVar(free->level, freshArgPack, retPack); + } + else + retPack = freshTypePack(scope->level); + } + // checkExpr will log the pre-instantiated type of the function. // That's not nearly as interesting as the instantiated type, which will include details about how // generic functions are being instantiated for this particular callsite. @@ -3550,8 +3623,6 @@ ExprResult TypeChecker::checkExprPack(const ScopePtr& scope, const A std::vector overloads = flattenIntersection(actualFunctionType); - TypePackId retPack = freshTypePack(scope->level); - std::vector> expectedTypes = getExpectedTypesForCall(overloads, expr.args.size, expr.self); ExprResult argListResult = checkExprList(scope, expr.location, expr.args, false, {}, expectedTypes); @@ -3682,7 +3753,7 @@ std::optional> TypeChecker::checkCallOverload(const Scope // has been instantiated, so is a monotype. We can therefore // unify it with a monomorphic function. TypeId r = addType(FunctionTypeVar(scope->level, argPack, retPack)); - if (FFlag::LuauWidenIfSupertypeIsFree) + if (FFlag::LuauWidenIfSupertypeIsFree2) { UnifierOptions options; options.isFunctionCall = true; @@ -3772,7 +3843,7 @@ std::optional> TypeChecker::checkCallOverload(const Scope { state.log.commit(); - if (isNonstrictMode() && !expr.self && expr.func->is() && ftv->hasSelf) + if (!FFlag::LuauAnyInIsOptionalIsOptional && isNonstrictMode() && !expr.self && expr.func->is() && ftv->hasSelf) { // If we are running in nonstrict mode, passing fewer arguments than the function is declared to take AND // the function is declared with colon notation AND we use dot notation, warn. diff --git a/Analysis/src/TypeVar.cpp b/Analysis/src/TypeVar.cpp index 5af2c8a6..89549535 100644 --- a/Analysis/src/TypeVar.cpp +++ b/Analysis/src/TypeVar.cpp @@ -26,6 +26,7 @@ LUAU_FASTINT(LuauTypeInferRecursionLimit) LUAU_FASTFLAG(LuauErrorRecoveryType) LUAU_FASTFLAG(LuauSubtypingAddOptPropsToUnsealedTables) LUAU_FASTFLAG(LuauDiscriminableUnions2) +LUAU_FASTFLAGVARIABLE(LuauAnyInIsOptionalIsOptional, false) namespace Luau { @@ -201,11 +202,16 @@ bool isOptional(TypeId ty) if (isNil(ty)) return true; - auto utv = get(follow(ty)); + ty = follow(ty); + + if (FFlag::LuauAnyInIsOptionalIsOptional && get(ty)) + return true; + + auto utv = get(ty); if (!utv) return false; - return std::any_of(begin(utv), end(utv), isNil); + return std::any_of(begin(utv), end(utv), FFlag::LuauAnyInIsOptionalIsOptional ? isOptional : isNil); } bool isTableIntersection(TypeId ty) diff --git a/Analysis/src/Unifier.cpp b/Analysis/src/Unifier.cpp index 7b781f26..60a9c9a5 100644 --- a/Analysis/src/Unifier.cpp +++ b/Analysis/src/Unifier.cpp @@ -20,11 +20,13 @@ LUAU_FASTFLAGVARIABLE(LuauTableSubtypingVariance2, false); LUAU_FASTFLAG(LuauSingletonTypes) LUAU_FASTFLAG(LuauErrorRecoveryType); LUAU_FASTFLAGVARIABLE(LuauSubtypingAddOptPropsToUnsealedTables, false) -LUAU_FASTFLAGVARIABLE(LuauWidenIfSupertypeIsFree, false) +LUAU_FASTFLAGVARIABLE(LuauWidenIfSupertypeIsFree2, false) LUAU_FASTFLAGVARIABLE(LuauDifferentOrderOfUnificationDoesntMatter, false) LUAU_FASTFLAGVARIABLE(LuauTxnLogSeesTypePacks2, false) LUAU_FASTFLAGVARIABLE(LuauTxnLogCheckForInvalidation, false) +LUAU_FASTFLAGVARIABLE(LuauTxnLogRefreshFunctionPointers, false) LUAU_FASTFLAGVARIABLE(LuauTxnLogDontRetryForIndexers, false) +LUAU_FASTFLAG(LuauAnyInIsOptionalIsOptional) namespace Luau { @@ -272,12 +274,6 @@ TypePackId Widen::clean(TypePackId) bool Widen::ignoreChildren(TypeId ty) { - // Sometimes we unify ("hi") -> free1 with (free2) -> free3, so don't ignore functions. - // TODO: should we be doing this? we would need to rework how checkCallOverload does the unification. - if (log->is(ty)) - return false; - - // We only care about unions. return !log->is(ty); } @@ -990,7 +986,7 @@ void Unifier::tryUnify_(TypePackId subTp, TypePackId superTp, bool isFunctionCal if (!log.getMutable(superTp)) { - log.replace(superTp, Unifiable::Bound(subTp)); + log.replace(superTp, Unifiable::Bound(widen(subTp))); } } else if (log.getMutable(subTp)) @@ -1107,7 +1103,7 @@ void Unifier::tryUnify_(TypePackId subTp, TypePackId superTp, bool isFunctionCal } // In nonstrict mode, any also marks an optional argument. - else if (superIter.good() && isNonstrictMode() && log.getMutable(log.follow(*superIter))) + else if (!FFlag::LuauAnyInIsOptionalIsOptional && superIter.good() && isNonstrictMode() && log.getMutable(log.follow(*superIter))) { superIter.advance(); continue; @@ -1280,6 +1276,13 @@ void Unifier::tryUnifyFunctions(TypeId subTy, TypeId superTy, bool isFunctionCal tryUnify_(subFunction->retType, superFunction->retType); } + if (FFlag::LuauTxnLogRefreshFunctionPointers) + { + // Updating the log may have invalidated the function pointers + superFunction = log.getMutable(superTy); + subFunction = log.getMutable(subTy); + } + if (!FFlag::LuauImmutableTypes) { if (superFunction->definition && !subFunction->definition && !subTy->persistent) @@ -1357,10 +1360,18 @@ void Unifier::tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) { auto subIter = subTable->props.find(propName); - bool isAny = log.getMutable(log.follow(superProp.type)); + if (FFlag::LuauAnyInIsOptionalIsOptional) + { + if (subIter == subTable->props.end() && (!FFlag::LuauSubtypingAddOptPropsToUnsealedTables || subTable->state == TableState::Unsealed) && !isOptional(superProp.type)) + missingProperties.push_back(propName); + } + else + { + bool isAny = log.getMutable(log.follow(superProp.type)); - if (subIter == subTable->props.end() && (!FFlag::LuauSubtypingAddOptPropsToUnsealedTables || subTable->state == TableState::Unsealed) && !isOptional(superProp.type) && !isAny) - missingProperties.push_back(propName); + if (subIter == subTable->props.end() && (!FFlag::LuauSubtypingAddOptPropsToUnsealedTables || subTable->state == TableState::Unsealed) && !isOptional(superProp.type) && !isAny) + missingProperties.push_back(propName); + } } if (!missingProperties.empty()) @@ -1378,9 +1389,17 @@ void Unifier::tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) { auto superIter = superTable->props.find(propName); - bool isAny = log.is(log.follow(subProp.type)); - if (superIter == superTable->props.end() && (FFlag::LuauSubtypingAddOptPropsToUnsealedTables || (!isOptional(subProp.type) && !isAny))) - extraProperties.push_back(propName); + if (FFlag::LuauAnyInIsOptionalIsOptional) + { + if (superIter == superTable->props.end() && (FFlag::LuauSubtypingAddOptPropsToUnsealedTables || !isOptional(subProp.type))) + extraProperties.push_back(propName); + } + else + { + bool isAny = log.is(log.follow(subProp.type)); + if (superIter == superTable->props.end() && (FFlag::LuauSubtypingAddOptPropsToUnsealedTables || (!isOptional(subProp.type) && !isAny))) + extraProperties.push_back(propName); + } } if (!extraProperties.empty()) @@ -1424,6 +1443,12 @@ void Unifier::tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) if (innerState.errors.empty()) log.concat(std::move(innerState.log)); } + else if (FFlag::LuauAnyInIsOptionalIsOptional && (!FFlag::LuauSubtypingAddOptPropsToUnsealedTables || subTable->state == TableState::Unsealed) && isOptional(prop.type)) + // This is sound because unsealed table types are precise, so `{ p : T } <: { p : T, q : U? }` + // since if `t : { p : T }` then we are guaranteed that `t.q` is `nil`. + // TODO: if the supertype is written to, the subtype may no longer be precise (alias analysis?) + { + } else if ((!FFlag::LuauSubtypingAddOptPropsToUnsealedTables || subTable->state == TableState::Unsealed) && (isOptional(prop.type) || get(follow(prop.type)))) // This is sound because unsealed table types are precise, so `{ p : T } <: { p : T, q : U? }` // since if `t : { p : T }` then we are guaranteed that `t.q` is `nil`. @@ -1497,6 +1522,9 @@ void Unifier::tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) else if (variance == Covariant) { } + else if (FFlag::LuauAnyInIsOptionalIsOptional && !FFlag::LuauSubtypingAddOptPropsToUnsealedTables && isOptional(prop.type)) + { + } else if (!FFlag::LuauSubtypingAddOptPropsToUnsealedTables && (isOptional(prop.type) || get(follow(prop.type)))) { } @@ -1618,7 +1646,7 @@ void Unifier::tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) TypeId Unifier::widen(TypeId ty) { - if (!FFlag::LuauWidenIfSupertypeIsFree) + if (!FFlag::LuauWidenIfSupertypeIsFree2) return ty; Widen widen{types}; @@ -1627,10 +1655,21 @@ TypeId Unifier::widen(TypeId ty) return result.value_or(ty); } +TypePackId Unifier::widen(TypePackId tp) +{ + if (!FFlag::LuauWidenIfSupertypeIsFree2) + return tp; + + Widen widen{types}; + std::optional result = widen.substitute(tp); + // TODO: what does it mean for substitution to fail to widen? + return result.value_or(tp); +} + TypeId Unifier::deeplyOptional(TypeId ty, std::unordered_map seen) { ty = follow(ty); - if (get(ty)) + if (!FFlag::LuauAnyInIsOptionalIsOptional && get(ty)) return ty; else if (isOptional(ty)) return ty; @@ -1744,7 +1783,10 @@ void Unifier::tryUnifyFreeTable(TypeId subTy, TypeId superTy) { if (auto subProp = findTablePropertyRespectingMeta(subTy, freeName)) { - tryUnify_(freeProp.type, *subProp); + if (FFlag::LuauWidenIfSupertypeIsFree2) + tryUnify_(*subProp, freeProp.type); + else + tryUnify_(freeProp.type, *subProp); /* * TypeVars are commonly cyclic, so it is entirely possible diff --git a/Ast/src/Parser.cpp b/Ast/src/Parser.cpp index 1cb8f134..941a3ea4 100644 --- a/Ast/src/Parser.cpp +++ b/Ast/src/Parser.cpp @@ -11,18 +11,11 @@ LUAU_FASTINTVARIABLE(LuauRecursionLimit, 1000) LUAU_FASTINTVARIABLE(LuauParseErrorLimit, 100) LUAU_FASTFLAGVARIABLE(LuauParseSingletonTypes, false) -LUAU_FASTFLAGVARIABLE(LuauParseAllHotComments, false) LUAU_FASTFLAGVARIABLE(LuauTableFieldFunctionDebugname, false) namespace Luau { -static bool isComment(const Lexeme& lexeme) -{ - LUAU_ASSERT(!FFlag::LuauParseAllHotComments); - return lexeme.type == Lexeme::Comment || lexeme.type == Lexeme::BlockComment; -} - ParseError::ParseError(const Location& location, const std::string& message) : location(location) , message(message) @@ -146,54 +139,13 @@ ParseResult Parser::parse(const char* buffer, size_t bufferSize, AstNameTable& n { LUAU_TIMETRACE_SCOPE("Parser::parse", "Parser"); - Parser p(buffer, bufferSize, names, allocator, FFlag::LuauParseAllHotComments ? options : ParseOptions()); + Parser p(buffer, bufferSize, names, allocator, options); try { - if (FFlag::LuauParseAllHotComments) - { - AstStatBlock* root = p.parseChunk(); + AstStatBlock* root = p.parseChunk(); - return ParseResult{root, std::move(p.hotcomments), std::move(p.parseErrors), std::move(p.commentLocations)}; - } - else - { - std::vector hotcomments; - - while (isComment(p.lexer.current()) || p.lexer.current().type == Lexeme::BrokenComment) - { - const char* text = p.lexer.current().data; - unsigned int length = p.lexer.current().length; - - if (length && text[0] == '!') - { - unsigned int end = length; - while (end > 0 && isSpace(text[end - 1])) - --end; - - hotcomments.push_back({true, p.lexer.current().location, std::string(text + 1, text + end)}); - } - - const Lexeme::Type type = p.lexer.current().type; - const Location loc = p.lexer.current().location; - - if (options.captureComments) - p.commentLocations.push_back(Comment{type, loc}); - - if (type == Lexeme::BrokenComment) - break; - - p.lexer.next(); - } - - p.lexer.setSkipComments(true); - - p.options = options; - - AstStatBlock* root = p.parseChunk(); - - return ParseResult{root, hotcomments, p.parseErrors, std::move(p.commentLocations)}; - } + return ParseResult{root, std::move(p.hotcomments), std::move(p.parseErrors), std::move(p.commentLocations)}; } catch (ParseError& err) { @@ -225,10 +177,11 @@ Parser::Parser(const char* buffer, size_t bufferSize, AstNameTable& names, Alloc matchRecoveryStopOnToken.assign(Lexeme::Type::Reserved_END, 0); matchRecoveryStopOnToken[Lexeme::Type::Eof] = 1; - if (FFlag::LuauParseAllHotComments) - lexer.setSkipComments(true); + // required for lookahead() to work across a comment boundary and for nextLexeme() to work when captureComments is false + lexer.setSkipComments(true); - // read first lexeme + // read first lexeme (any hot comments get .header = true) + LUAU_ASSERT(hotcommentHeader); nextLexeme(); // all hot comments parsed after the first non-comment lexeme are special in that they don't affect type checking / linting mode @@ -2831,49 +2784,31 @@ void Parser::nextLexeme() { if (options.captureComments) { - if (FFlag::LuauParseAllHotComments) + Lexeme::Type type = lexer.next(/* skipComments= */ false).type; + + while (type == Lexeme::BrokenComment || type == Lexeme::Comment || type == Lexeme::BlockComment) { - Lexeme::Type type = lexer.next(/* skipComments= */ false).type; + const Lexeme& lexeme = lexer.current(); + commentLocations.push_back(Comment{lexeme.type, lexeme.location}); - while (type == Lexeme::BrokenComment || type == Lexeme::Comment || type == Lexeme::BlockComment) + // Subtlety: Broken comments are weird because we record them as comments AND pass them to the parser as a lexeme. + // The parser will turn this into a proper syntax error. + if (lexeme.type == Lexeme::BrokenComment) + return; + + // Comments starting with ! are called "hot comments" and contain directives for type checking / linting + if (lexeme.type == Lexeme::Comment && lexeme.length && lexeme.data[0] == '!') { - const Lexeme& lexeme = lexer.current(); - commentLocations.push_back(Comment{lexeme.type, lexeme.location}); + const char* text = lexeme.data; - // Subtlety: Broken comments are weird because we record them as comments AND pass them to the parser as a lexeme. - // The parser will turn this into a proper syntax error. - if (lexeme.type == Lexeme::BrokenComment) - return; + unsigned int end = lexeme.length; + while (end > 0 && isSpace(text[end - 1])) + --end; - // Comments starting with ! are called "hot comments" and contain directives for type checking / linting - if (lexeme.type == Lexeme::Comment && lexeme.length && lexeme.data[0] == '!') - { - const char* text = lexeme.data; - - unsigned int end = lexeme.length; - while (end > 0 && isSpace(text[end - 1])) - --end; - - hotcomments.push_back({hotcommentHeader, lexeme.location, std::string(text + 1, text + end)}); - } - - type = lexer.next(/* skipComments= */ false).type; - } - } - else - { - while (true) - { - const Lexeme& lexeme = lexer.next(/*skipComments*/ false); - // Subtlety: Broken comments are weird because we record them as comments AND pass them to the parser as a lexeme. - // The parser will turn this into a proper syntax error. - if (lexeme.type == Lexeme::BrokenComment) - commentLocations.push_back(Comment{lexeme.type, lexeme.location}); - if (isComment(lexeme)) - commentLocations.push_back(Comment{lexeme.type, lexeme.location}); - else - return; + hotcomments.push_back({hotcommentHeader, lexeme.location, std::string(text + 1, text + end)}); } + + type = lexer.next(/* skipComments= */ false).type; } } else diff --git a/Compiler/src/Builtins.cpp b/Compiler/src/Builtins.cpp index 26360c49..ff753112 100644 --- a/Compiler/src/Builtins.cpp +++ b/Compiler/src/Builtins.cpp @@ -4,8 +4,6 @@ #include "Luau/Bytecode.h" #include "Luau/Compiler.h" -LUAU_FASTFLAGVARIABLE(LuauCompileSelectBuiltin2, false) - namespace Luau { namespace Compile @@ -64,7 +62,7 @@ int getBuiltinFunctionId(const Builtin& builtin, const CompileOptions& options) if (builtin.isGlobal("unpack")) return LBF_TABLE_UNPACK; - if (FFlag::LuauCompileSelectBuiltin2 && builtin.isGlobal("select")) + if (builtin.isGlobal("select")) return LBF_SELECT_VARARG; if (builtin.object == "math") diff --git a/Compiler/src/Compiler.cpp b/Compiler/src/Compiler.cpp index 656a9926..6330bf1f 100644 --- a/Compiler/src/Compiler.cpp +++ b/Compiler/src/Compiler.cpp @@ -15,8 +15,6 @@ #include #include -LUAU_FASTFLAG(LuauCompileSelectBuiltin2) - namespace Luau { @@ -265,7 +263,6 @@ struct Compiler void compileExprSelectVararg(AstExprCall* expr, uint8_t target, uint8_t targetCount, bool targetTop, bool multRet, uint8_t regs) { - LUAU_ASSERT(FFlag::LuauCompileSelectBuiltin2); LUAU_ASSERT(targetCount == 1); LUAU_ASSERT(!expr->self); LUAU_ASSERT(expr->args.size == 2 && expr->args.data[1]->is()); @@ -407,7 +404,6 @@ struct Compiler if (bfid == LBF_SELECT_VARARG) { - LUAU_ASSERT(FFlag::LuauCompileSelectBuiltin2); // Optimization: compile select(_, ...) as FASTCALL1; the builtin will read variadic arguments directly // note: for now we restrict this to single-return expressions since our runtime code doesn't deal with general cases if (multRet == false && targetCount == 1 && expr->args.size == 2 && expr->args.data[1]->is()) diff --git a/Sources.cmake b/Sources.cmake index 615641eb..59b38497 100644 --- a/Sources.cmake +++ b/Sources.cmake @@ -232,11 +232,18 @@ if(TARGET Luau.UnitTest) tests/Transpiler.test.cpp tests/TypeInfer.aliases.test.cpp tests/TypeInfer.annotations.test.cpp + tests/TypeInfer.anyerror.test.cpp tests/TypeInfer.builtins.test.cpp tests/TypeInfer.classes.test.cpp tests/TypeInfer.definitions.test.cpp + tests/TypeInfer.functions.test.cpp tests/TypeInfer.generics.test.cpp tests/TypeInfer.intersectionTypes.test.cpp + tests/TypeInfer.loops.test.cpp + tests/TypeInfer.modules.test.cpp + tests/TypeInfer.oop.test.cpp + tests/TypeInfer.operators.test.cpp + tests/TypeInfer.primitives.test.cpp tests/TypeInfer.provisional.test.cpp tests/TypeInfer.refinements.test.cpp tests/TypeInfer.singletons.test.cpp diff --git a/VM/include/lua.h b/VM/include/lua.h index 274c4ed9..d08b73ea 100644 --- a/VM/include/lua.h +++ b/VM/include/lua.h @@ -172,9 +172,12 @@ LUA_API const char* lua_pushvfstring(lua_State* L, const char* fmt, va_list argp LUA_API LUA_PRINTF_ATTR(2, 3) const char* lua_pushfstringL(lua_State* L, const char* fmt, ...); LUA_API void lua_pushcclosurek(lua_State* L, lua_CFunction fn, const char* debugname, int nup, lua_Continuation cont); LUA_API void lua_pushboolean(lua_State* L, int b); -LUA_API void lua_pushlightuserdata(lua_State* L, void* p); LUA_API int lua_pushthread(lua_State* L); +LUA_API void lua_pushlightuserdata(lua_State* L, void* p); +LUA_API void* lua_newuserdatatagged(lua_State* L, size_t sz, int tag); +LUA_API void* lua_newuserdatadtor(lua_State* L, size_t sz, void (*dtor)(void*)); + /* ** get functions (Lua -> stack) */ @@ -189,8 +192,6 @@ LUA_API void lua_setreadonly(lua_State* L, int idx, int enabled); LUA_API int lua_getreadonly(lua_State* L, int idx); LUA_API void lua_setsafeenv(lua_State* L, int idx, int enabled); -LUA_API void* lua_newuserdatatagged(lua_State* L, size_t sz, int tag); -LUA_API void* lua_newuserdatadtor(lua_State* L, size_t sz, void (*dtor)(void*)); LUA_API int lua_getmetatable(lua_State* L, int objindex); LUA_API void lua_getfenv(lua_State* L, int idx); @@ -276,6 +277,14 @@ enum lua_GCOp LUA_API int lua_gc(lua_State* L, int what, int data); +/* +** memory statistics +** all allocated bytes are attributed to the memory category of the running thread (0..LUA_MEMORY_CATEGORIES-1) +*/ + +LUA_API void lua_setmemcat(lua_State* L, int category); +LUA_API size_t lua_totalbytes(lua_State* L, int category); + /* ** miscellaneous functions */ diff --git a/VM/src/lapi.cpp b/VM/src/lapi.cpp index f7f15442..3c087314 100644 --- a/VM/src/lapi.cpp +++ b/VM/src/lapi.cpp @@ -35,8 +35,8 @@ const char* luau_ident = "$Luau: Copyright (C) 2019-2022 Roblox Corporation $\n" static Table* getcurrenv(lua_State* L) { - if (L->ci == L->base_ci) /* no enclosing function? */ - return L->gt; /* use global table as environment */ + if (L->ci == L->base_ci) /* no enclosing function? */ + return L->gt; /* use global table as environment */ else return curr_func(L)->env; } @@ -1188,7 +1188,7 @@ void lua_concat(lua_State* L, int n) void* lua_newuserdatatagged(lua_State* L, size_t sz, int tag) { - api_check(L, unsigned(tag) < LUA_UTAG_LIMIT); + api_check(L, unsigned(tag) < LUA_UTAG_LIMIT || tag == UTAG_PROXY); luaC_checkGC(L); luaC_checkthreadsleep(L); Udata* u = luaU_newudata(L, sz, tag); @@ -1317,7 +1317,7 @@ void lua_setuserdatadtor(lua_State* L, int tag, void (*dtor)(void*)) L->global->udatagc[tag] = dtor; } -LUA_API void lua_clonefunction(lua_State* L, int idx) +void lua_clonefunction(lua_State* L, int idx) { StkId p = index2addr(L, idx); api_check(L, isLfunction(p)); @@ -1333,3 +1333,15 @@ lua_Callbacks* lua_callbacks(lua_State* L) { return &L->global->cb; } + +void lua_setmemcat(lua_State* L, int category) +{ + api_check(L, unsigned(category) < LUA_MEMORY_CATEGORIES); + L->activememcat = uint8_t(category); +} + +size_t lua_totalbytes(lua_State* L, int category) +{ + api_check(L, category < LUA_MEMORY_CATEGORIES); + return category < 0 ? L->global->totalbytes : L->global->memcatbytes[category]; +} diff --git a/VM/src/lbaselib.cpp b/VM/src/lbaselib.cpp index 2307598e..96ad493b 100644 --- a/VM/src/lbaselib.cpp +++ b/VM/src/lbaselib.cpp @@ -5,6 +5,7 @@ #include "lstate.h" #include "lapi.h" #include "ldo.h" +#include "ludata.h" #include #include @@ -190,6 +191,7 @@ static int luaB_type(lua_State* L) luaL_checkany(L, 1); if (DFFlag::LuauMorePreciseLuaLTypeName) { + /* resulting name doesn't differentiate between userdata types */ lua_pushstring(L, lua_typename(L, lua_type(L, 1))); } else @@ -202,8 +204,16 @@ static int luaB_type(lua_State* L) static int luaB_typeof(lua_State* L) { luaL_checkany(L, 1); - const TValue* obj = luaA_toobject(L, 1); - lua_pushstring(L, luaT_objtypename(L, obj)); + if (DFFlag::LuauMorePreciseLuaLTypeName) + { + /* resulting name returns __type if specified unless the input is a newproxy-created userdata */ + lua_pushstring(L, luaL_typename(L, 1)); + } + else + { + const TValue* obj = luaA_toobject(L, 1); + lua_pushstring(L, luaT_objtypename(L, obj)); + } return 1; } @@ -412,7 +422,7 @@ static int luaB_newproxy(lua_State* L) bool needsmt = lua_toboolean(L, 1); - lua_newuserdata(L, 0); + lua_newuserdatatagged(L, 0, UTAG_PROXY); if (needsmt) { diff --git a/VM/src/lgc.h b/VM/src/lgc.h index ad8ee78a..ebf999b5 100644 --- a/VM/src/lgc.h +++ b/VM/src/lgc.h @@ -9,9 +9,9 @@ /* ** Default settings for GC tunables (settable via lua_gc) */ -#define LUAI_GCGOAL 200 /* 200% (allow heap to double compared to live heap size) */ +#define LUAI_GCGOAL 200 /* 200% (allow heap to double compared to live heap size) */ #define LUAI_GCSTEPMUL 200 /* GC runs 'twice the speed' of memory allocation */ -#define LUAI_GCSTEPSIZE 1 /* GC runs every KB of memory allocation */ +#define LUAI_GCSTEPSIZE 1 /* GC runs every KB of memory allocation */ /* ** Possible states of the Garbage Collector diff --git a/VM/src/lmem.cpp b/VM/src/lmem.cpp index 899cb0c0..3cbdafff 100644 --- a/VM/src/lmem.cpp +++ b/VM/src/lmem.cpp @@ -98,7 +98,7 @@ */ #if defined(__APPLE__) #define ABISWITCH(x64, ms32, gcc32) (sizeof(void*) == 8 ? x64 : gcc32) -#elif defined(__i386__) +#elif defined(__i386__) && !defined(_MSC_VER) #define ABISWITCH(x64, ms32, gcc32) (gcc32) #else // Android somehow uses a similar ABI to MSVC, *not* to iOS... diff --git a/VM/src/lstring.cpp b/VM/src/lstring.cpp index 87250146..c0cd3e26 100644 --- a/VM/src/lstring.cpp +++ b/VM/src/lstring.cpp @@ -53,7 +53,7 @@ void luaS_resize(lua_State* L, int newsize) { TString* p = tb->hash[i]; while (p) - { /* for each node in the list */ + { /* for each node in the list */ TString* next = p->next; /* save next */ unsigned int h = p->hash; int h1 = lmod(h, newsize); /* new position */ diff --git a/VM/src/ltable.cpp b/VM/src/ltable.cpp index ef0b4b93..2deec2b9 100644 --- a/VM/src/ltable.cpp +++ b/VM/src/ltable.cpp @@ -24,6 +24,8 @@ #include +LUAU_FASTFLAGVARIABLE(LuauTableRehashRework, false) + // max size of both array and hash part is 2^MAXBITS #define MAXBITS 26 #define MAXSIZE (1 << MAXBITS) @@ -351,6 +353,22 @@ static void setnodevector(lua_State* L, Table* t, int size) t->lastfree = size; /* all positions are free */ } +static TValue* newkey(lua_State* L, Table* t, const TValue* key); + +static TValue* arrayornewkey(lua_State* L, Table* t, const TValue* key) +{ + if (ttisnumber(key)) + { + int k; + double n = nvalue(key); + luai_num2int(k, n); + if (luai_numeq(cast_num(k), n) && cast_to(unsigned int, k - 1) < cast_to(unsigned int, t->sizearray)) + return &t->array[k - 1]; + } + + return newkey(L, t, key); +} + static void resize(lua_State* L, Table* t, int nasize, int nhsize) { if (nasize > MAXSIZE || nhsize > MAXSIZE) @@ -369,22 +387,50 @@ static void resize(lua_State* L, Table* t, int nasize, int nhsize) for (int i = nasize; i < oldasize; i++) { if (!ttisnil(&t->array[i])) - setobjt2t(L, luaH_setnum(L, t, i + 1), &t->array[i]); + { + if (FFlag::LuauTableRehashRework) + { + TValue ok; + setnvalue(&ok, cast_num(i + 1)); + setobjt2t(L, newkey(L, t, &ok), &t->array[i]); + } + else + { + setobjt2t(L, luaH_setnum(L, t, i + 1), &t->array[i]); + } + } } /* shrink array */ luaM_reallocarray(L, t->array, oldasize, nasize, TValue, t->memcat); } /* re-insert elements from hash part */ - for (int i = twoto(oldhsize) - 1; i >= 0; i--) + if (FFlag::LuauTableRehashRework) { - LuaNode* old = nold + i; - if (!ttisnil(gval(old))) + for (int i = twoto(oldhsize) - 1; i >= 0; i--) { - TValue ok; - getnodekey(L, &ok, old); - setobjt2t(L, luaH_set(L, t, &ok), gval(old)); + LuaNode* old = nold + i; + if (!ttisnil(gval(old))) + { + TValue ok; + getnodekey(L, &ok, old); + setobjt2t(L, arrayornewkey(L, t, &ok), gval(old)); + } } } + else + { + for (int i = twoto(oldhsize) - 1; i >= 0; i--) + { + LuaNode* old = nold + i; + if (!ttisnil(gval(old))) + { + TValue ok; + getnodekey(L, &ok, old); + setobjt2t(L, luaH_set(L, t, &ok), gval(old)); + } + } + } + if (nold != dummynode) luaM_freearray(L, nold, twoto(oldhsize), LuaNode, t->memcat); /* free old array */ } @@ -482,7 +528,16 @@ static TValue* newkey(lua_State* L, Table* t, const TValue* key) if (n == NULL) { /* cannot find a free place? */ rehash(L, t, key); /* grow table */ - return luaH_set(L, t, key); /* re-insert key into grown table */ + + if (!FFlag::LuauTableRehashRework) + { + return luaH_set(L, t, key); /* re-insert key into grown table */ + } + else + { + // after rehash, numeric keys might be located in the new array part, but won't be found in the node part + return arrayornewkey(L, t, key); + } } LUAU_ASSERT(n != dummynode); TValue mk; diff --git a/VM/src/ltm.cpp b/VM/src/ltm.cpp index a77a7c72..106efb2b 100644 --- a/VM/src/ltm.cpp +++ b/VM/src/ltm.cpp @@ -4,6 +4,7 @@ #include "lstate.h" #include "lstring.h" +#include "ludata.h" #include "ltable.h" #include "lgc.h" @@ -116,7 +117,7 @@ const TValue* luaT_gettmbyobj(lua_State* L, const TValue* o, TMS event) const TString* luaT_objtypenamestr(lua_State* L, const TValue* o) { - if (ttisuserdata(o) && uvalue(o)->tag && uvalue(o)->metatable) + if (ttisuserdata(o) && uvalue(o)->tag != UTAG_PROXY && uvalue(o)->metatable) { const TValue* type = luaH_getstr(uvalue(o)->metatable, L->global->tmname[TM_TYPE]); diff --git a/VM/src/ludata.cpp b/VM/src/ludata.cpp index 0dfac508..819d1863 100644 --- a/VM/src/ludata.cpp +++ b/VM/src/ludata.cpp @@ -22,13 +22,11 @@ Udata* luaU_newudata(lua_State* L, size_t s, int tag) void luaU_freeudata(lua_State* L, Udata* u, lua_Page* page) { - LUAU_ASSERT(u->tag < LUA_UTAG_LIMIT || u->tag == UTAG_IDTOR); - void (*dtor)(void*) = nullptr; - if (u->tag == UTAG_IDTOR) - memcpy(&dtor, &u->data + u->len - sizeof(dtor), sizeof(dtor)); - else if (u->tag) + if (u->tag < LUA_UTAG_LIMIT) dtor = L->global->udatagc[u->tag]; + else if (u->tag == UTAG_IDTOR) + memcpy(&dtor, &u->data + u->len - sizeof(dtor), sizeof(dtor)); if (dtor) dtor(u->data); diff --git a/VM/src/ludata.h b/VM/src/ludata.h index ec374c28..f24e4a32 100644 --- a/VM/src/ludata.h +++ b/VM/src/ludata.h @@ -7,6 +7,9 @@ /* special tag value is used for user data with inline dtors */ #define UTAG_IDTOR LUA_UTAG_LIMIT +/* special tag value is used for newproxy-created user data (all other user data objects are host-exposed) */ +#define UTAG_PROXY (LUA_UTAG_LIMIT + 1) + #define sizeudata(len) (offsetof(Udata, data) + len) LUAI_FUNC Udata* luaU_newudata(lua_State* L, size_t s, int tag); diff --git a/VM/src/lvmexecute.cpp b/VM/src/lvmexecute.cpp index 5e2ef688..96a87b7e 100644 --- a/VM/src/lvmexecute.cpp +++ b/VM/src/lvmexecute.cpp @@ -77,7 +77,7 @@ if (LUAU_UNLIKELY(!!interrupt)) \ { /* the interrupt hook is called right before we advance pc */ \ VM_PROTECT(L->ci->savedpc++; interrupt(L, -1)); \ - if (L->status != 0) \ + if (L->status != 0) \ { \ L->ci->savedpc--; \ goto exit; \ diff --git a/tests/Autocomplete.test.cpp b/tests/Autocomplete.test.cpp index 55b0618a..17fd6b13 100644 --- a/tests/Autocomplete.test.cpp +++ b/tests/Autocomplete.test.cpp @@ -2755,4 +2755,166 @@ local abc = b@1 CHECK(ac.entryMap["bar"].parens == ParenthesesRecommendation::CursorInside); } +TEST_CASE_FIXTURE(ACFixture, "no_incompatible_self_calls_on_class") +{ + ScopedFastFlag selfCallAutocompleteFix{"LuauSelfCallAutocompleteFix", true}; + + loadDefinition(R"( +declare class Foo + function one(self): number + two: () -> number +end + )"); + + { + check(R"( +local t: Foo +t:@1 + )"); + + auto ac = autocomplete('1'); + + REQUIRE(ac.entryMap.count("one")); + REQUIRE(ac.entryMap.count("two")); + CHECK(!ac.entryMap["one"].wrongIndexType); + CHECK(ac.entryMap["two"].wrongIndexType); + } + + { + check(R"( +local t: Foo +t.@1 + )"); + + auto ac = autocomplete('1'); + + REQUIRE(ac.entryMap.count("one")); + REQUIRE(ac.entryMap.count("two")); + CHECK(ac.entryMap["one"].wrongIndexType); + CHECK(!ac.entryMap["two"].wrongIndexType); + } +} + +TEST_CASE_FIXTURE(ACFixture, "no_incompatible_self_calls") +{ + ScopedFastFlag selfCallAutocompleteFix{"LuauSelfCallAutocompleteFix", true}; + + check(R"( +local t = {} +function t.m() end +t:@1 + )"); + + auto ac = autocomplete('1'); + + REQUIRE(ac.entryMap.count("m")); + CHECK(ac.entryMap["m"].wrongIndexType); +} + +TEST_CASE_FIXTURE(ACFixture, "no_incompatible_self_calls_2") +{ + ScopedFastFlag selfCallAutocompleteFix{"LuauSelfCallAutocompleteFix", true}; + + check(R"( +local f: (() -> number) & ((number) -> number) = function(x: number?) return 2 end +local t = {} +t.f = f +t:@1 + )"); + + auto ac = autocomplete('1'); + + REQUIRE(ac.entryMap.count("f")); + CHECK(ac.entryMap["f"].wrongIndexType); +} + +TEST_CASE_FIXTURE(ACFixture, "no_incompatible_self_calls_provisional") +{ + check(R"( +local t = {} +function t.m(x: typeof(t)) end +t:@1 + )"); + + auto ac = autocomplete('1'); + + REQUIRE(ac.entryMap.count("m")); + // We can make changes to mark this as a wrong way to call even though it's compatible + CHECK(!ac.entryMap["m"].wrongIndexType); +} + +TEST_CASE_FIXTURE(ACFixture, "string_prim_self_calls_are_fine") +{ + ScopedFastFlag selfCallAutocompleteFix{"LuauSelfCallAutocompleteFix", true}; + + check(R"( +local s = "hello" +s:@1 + )"); + + auto ac = autocomplete('1'); + + REQUIRE(ac.entryMap.count("byte")); + CHECK(ac.entryMap["byte"].wrongIndexType == false); + REQUIRE(ac.entryMap.count("char")); + CHECK(ac.entryMap["char"].wrongIndexType == true); + REQUIRE(ac.entryMap.count("sub")); + CHECK(ac.entryMap["sub"].wrongIndexType == false); +} + +TEST_CASE_FIXTURE(ACFixture, "string_prim_non_self_calls_are_avoided") +{ + ScopedFastFlag selfCallAutocompleteFix{"LuauSelfCallAutocompleteFix", true}; + + check(R"( +local s = "hello" +s.@1 + )"); + + auto ac = autocomplete('1'); + + REQUIRE(ac.entryMap.count("byte")); + CHECK(ac.entryMap["byte"].wrongIndexType == true); + REQUIRE(ac.entryMap.count("char")); + CHECK(ac.entryMap["char"].wrongIndexType == false); + REQUIRE(ac.entryMap.count("sub")); + CHECK(ac.entryMap["sub"].wrongIndexType == true); +} + +TEST_CASE_FIXTURE(ACFixture, "string_library_non_self_calls_are_fine") +{ + ScopedFastFlag selfCallAutocompleteFix{"LuauSelfCallAutocompleteFix", true}; + + check(R"( +string.@1 + )"); + + auto ac = autocomplete('1'); + + REQUIRE(ac.entryMap.count("byte")); + CHECK(ac.entryMap["byte"].wrongIndexType == false); + REQUIRE(ac.entryMap.count("char")); + CHECK(ac.entryMap["char"].wrongIndexType == false); + REQUIRE(ac.entryMap.count("sub")); + CHECK(ac.entryMap["sub"].wrongIndexType == false); +} + +TEST_CASE_FIXTURE(ACFixture, "string_library_self_calls_are_invalid") +{ + ScopedFastFlag selfCallAutocompleteFix{"LuauSelfCallAutocompleteFix", true}; + + check(R"( +string:@1 + )"); + + auto ac = autocomplete('1'); + + REQUIRE(ac.entryMap.count("byte")); + CHECK(ac.entryMap["byte"].wrongIndexType == true); + REQUIRE(ac.entryMap.count("char")); + CHECK(ac.entryMap["char"].wrongIndexType == true); + REQUIRE(ac.entryMap.count("sub")); + CHECK(ac.entryMap["sub"].wrongIndexType == true); +} + TEST_SUITE_END(); diff --git a/tests/Compiler.test.cpp b/tests/Compiler.test.cpp index f982c86f..3dc57da0 100644 --- a/tests/Compiler.test.cpp +++ b/tests/Compiler.test.cpp @@ -2819,8 +2819,6 @@ RETURN R1 -1 TEST_CASE("FastcallSelect") { - ScopedFastFlag sff("LuauCompileSelectBuiltin2", true); - // select(_, ...) compiles to a builtin call CHECK_EQ("\n" + compileFunction0("return (select('#', ...))"), R"( LOADK R1 K0 diff --git a/tests/Conformance.test.cpp b/tests/Conformance.test.cpp index 63fbb363..9e4cb4a5 100644 --- a/tests/Conformance.test.cpp +++ b/tests/Conformance.test.cpp @@ -569,6 +569,11 @@ TEST_CASE("Debugger") CHECK(lua_tointeger(L, -1) == 50); lua_pop(L, 1); + int v = lua_getargument(L, 0, 2); + REQUIRE(v); + CHECK(lua_tointeger(L, -1) == 42); + lua_pop(L, 1); + // test lua_getlocal const char* l = lua_getlocal(L, 0, 1); REQUIRE(l); @@ -652,31 +657,6 @@ TEST_CASE("SameHash") CHECK(luaS_hash(buf + 1, 120) == luaS_hash(buf + 2, 120)); } -TEST_CASE("InlineDtor") -{ - static int dtorhits = 0; - - dtorhits = 0; - - { - StateRef globalState(luaL_newstate(), lua_close); - lua_State* L = globalState.get(); - - void* u1 = lua_newuserdatadtor(L, 4, [](void* data) { - dtorhits += *(int*)data; - }); - - void* u2 = lua_newuserdatadtor(L, 1, [](void* data) { - dtorhits += *(char*)data; - }); - - *(int*)u1 = 39; - *(char*)u2 = 3; - } - - CHECK(dtorhits == 42); -} - TEST_CASE("Reference") { static int dtorhits = 0; @@ -969,7 +949,7 @@ TEST_CASE("StringConversion") TEST_CASE("GCDump") { // internal function, declared in lgc.h - not exposed via lua.h - extern void luaC_dump(lua_State* L, void* file, const char* (*categoryName)(lua_State* L, uint8_t memcat)); + extern void luaC_dump(lua_State * L, void* file, const char* (*categoryName)(lua_State * L, uint8_t memcat)); StateRef globalState(luaL_newstate(), lua_close); lua_State* L = globalState.get(); @@ -1015,4 +995,114 @@ TEST_CASE("GCDump") fclose(f); } +TEST_CASE("Interrupt") +{ + static const int expectedhits[] = { + 2, + 9, + 5, + 5, + 5, + 5, + 5, + 5, + 5, + 5, + 5, + 5, + 5, + 6, + 11, + }; + static int index; + + index = 0; + + runConformance( + "interrupt.lua", + [](lua_State* L) { + auto* cb = lua_callbacks(L); + + // note: for simplicity here we setup the interrupt callback once + // however, this carries a noticeable performance cost. in a real application, + // it's advised to set interrupt callback on a timer from a different thread, + // and set it back to nullptr once the interrupt triggered. + cb->interrupt = [](lua_State* L, int gc) { + if (gc >= 0) + return; + + CHECK(index < int(std::size(expectedhits))); + + lua_Debug ar = {}; + lua_getinfo(L, 0, "l", &ar); + + CHECK(ar.currentline == expectedhits[index]); + + index++; + + // check that we can yield inside an interrupt + if (index == 5) + lua_yield(L, 0); + }; + }, + [](lua_State* L) { + CHECK(index == 5); // a single yield point + }); + + CHECK(index == int(std::size(expectedhits))); +} + +TEST_CASE("UserdataApi") +{ + static int dtorhits = 0; + + dtorhits = 0; + + StateRef globalState(luaL_newstate(), lua_close); + lua_State* L = globalState.get(); + + // setup dtor for tag 42 (created later) + lua_setuserdatadtor(L, 42, [](void* data) { + dtorhits += *(int*)data; + }); + + // light user data + int lud; + lua_pushlightuserdata(L, &lud); + + CHECK(lua_touserdata(L, -1) == &lud); + CHECK(lua_topointer(L, -1) == &lud); + + // regular user data + int* ud1 = (int*)lua_newuserdata(L, 4); + *ud1 = 42; + + CHECK(lua_touserdata(L, -1) == ud1); + CHECK(lua_topointer(L, -1) == ud1); + + // tagged user data + int* ud2 = (int*)lua_newuserdatatagged(L, 4, 42); + *ud2 = -4; + + CHECK(lua_touserdatatagged(L, -1, 42) == ud2); + CHECK(lua_touserdatatagged(L, -1, 41) == nullptr); + CHECK(lua_userdatatag(L, -1) == 42); + + // user data with inline dtor + void* ud3 = lua_newuserdatadtor(L, 4, [](void* data) { + dtorhits += *(int*)data; + }); + + void* ud4 = lua_newuserdatadtor(L, 1, [](void* data) { + dtorhits += *(char*)data; + }); + + *(int*)ud3 = 43; + *(char*)ud4 = 3; + + globalState.reset(); + + CHECK(dtorhits == 42); +} + TEST_SUITE_END(); diff --git a/tests/Linter.test.cpp b/tests/Linter.test.cpp index 4d6c207c..91b23197 100644 --- a/tests/Linter.test.cpp +++ b/tests/Linter.test.cpp @@ -1667,8 +1667,6 @@ _ = (math.random() < 0.5 and false) or 42 -- currently ignored TEST_CASE_FIXTURE(Fixture, "WrongComment") { - ScopedFastFlag sff("LuauParseAllHotComments", true); - LintResult result = lint(R"( --!strict --!struct diff --git a/tests/Module.test.cpp b/tests/Module.test.cpp index e3993cc5..82b7a350 100644 --- a/tests/Module.test.cpp +++ b/tests/Module.test.cpp @@ -248,10 +248,12 @@ TEST_CASE_FIXTURE(Fixture, "clone_seal_free_tables") TEST_CASE_FIXTURE(Fixture, "clone_self_property") { + ScopedFastFlag sff{"LuauAnyInIsOptionalIsOptional", true}; + fileResolver.source["Module/A"] = R"( --!nonstrict local a = {} - function a:foo(x) + function a:foo(x: number) return -x; end return a; @@ -267,10 +269,10 @@ TEST_CASE_FIXTURE(Fixture, "clone_self_property") )"; result = frontend.check("Module/B"); - LUAU_REQUIRE_ERRORS(result); - CHECK_EQ(toString(result.errors[0]), "This function was declared to accept self, but you did not pass enough arguments. Use a colon instead of a " - "dot or pass 1 extra nil to suppress this warning"); + LUAU_REQUIRE_ERROR_COUNT(1, result); + + CHECK_EQ("This function must be called with self. Did you mean to use a colon instead of a dot?", toString(result.errors[0])); } TEST_CASE_FIXTURE(Fixture, "clone_recursion_limit") diff --git a/tests/NonstrictMode.test.cpp b/tests/NonstrictMode.test.cpp index 5bad9901..d3faea2a 100644 --- a/tests/NonstrictMode.test.cpp +++ b/tests/NonstrictMode.test.cpp @@ -126,6 +126,8 @@ TEST_CASE_FIXTURE(Fixture, "parameters_having_type_any_are_optional") TEST_CASE_FIXTURE(Fixture, "local_tables_are_not_any") { + ScopedFastFlag sff{"LuauAnyInIsOptionalIsOptional", true}; + CheckResult result = check(R"( --!nonstrict local T = {} @@ -136,31 +138,25 @@ TEST_CASE_FIXTURE(Fixture, "local_tables_are_not_any") T:staticmethod() )"); - LUAU_REQUIRE_ERROR_COUNT(2, result); + LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK(std::any_of(result.errors.begin(), result.errors.end(), [](const TypeError& e) { - return get(e); - })); - CHECK(std::any_of(result.errors.begin(), result.errors.end(), [](const TypeError& e) { - return get(e); - })); + CHECK_EQ("This function does not take self. Did you mean to use a dot instead of a colon?", toString(result.errors[0])); } TEST_CASE_FIXTURE(Fixture, "offer_a_hint_if_you_use_a_dot_instead_of_a_colon") { + ScopedFastFlag sff{"LuauAnyInIsOptionalIsOptional", true}; + CheckResult result = check(R"( --!nonstrict local T = {} - function T:method() end - T.method() + function T:method(x: number) end + T.method(5) )"); LUAU_REQUIRE_ERROR_COUNT(1, result); - auto e = get(result.errors[0]); - REQUIRE(e != nullptr); - - REQUIRE_EQ(1, e->requiredExtraNils); + CHECK_EQ("This function must be called with self. Did you mean to use a colon instead of a dot?", toString(result.errors[0])); } TEST_CASE_FIXTURE(Fixture, "table_props_are_any") diff --git a/tests/TypeInfer.aliases.test.cpp b/tests/TypeInfer.aliases.test.cpp index 711c0aa1..b2e76052 100644 --- a/tests/TypeInfer.aliases.test.cpp +++ b/tests/TypeInfer.aliases.test.cpp @@ -676,4 +676,25 @@ TEST_CASE_FIXTURE(Fixture, "forward_declared_alias_is_not_clobbered_by_prior_uni LUAU_REQUIRE_NO_ERRORS(result); } +TEST_CASE_FIXTURE(Fixture, "recursive_types_restriction_ok") +{ + CheckResult result = check(R"( + type Tree = { data: T, children: {Tree} } + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "recursive_types_restriction_not_ok") +{ + ScopedFastFlag sff{"LuauRecursiveTypeParameterRestriction", true}; + + CheckResult result = check(R"( + -- this would be an infinite type if we allowed it + type Tree = { data: T, children: {Tree<{T}>} } + )"); + + LUAU_REQUIRE_ERRORS(result); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.anyerror.test.cpp b/tests/TypeInfer.anyerror.test.cpp new file mode 100644 index 00000000..5224b5d8 --- /dev/null +++ b/tests/TypeInfer.anyerror.test.cpp @@ -0,0 +1,335 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details + +#include "Luau/AstQuery.h" +#include "Luau/BuiltinDefinitions.h" +#include "Luau/Scope.h" +#include "Luau/TypeInfer.h" +#include "Luau/TypeVar.h" +#include "Luau/VisitTypeVar.h" + +#include "Fixture.h" + +#include "doctest.h" + +using namespace Luau; + +TEST_SUITE_BEGIN("TypeInferAnyError"); + +TEST_CASE_FIXTURE(Fixture, "for_in_loop_iterator_returns_any") +{ + CheckResult result = check(R"( + function bar(): any + return true + end + + local a + for b in bar do + a = b + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ(typeChecker.anyType, requireType("a")); +} + +TEST_CASE_FIXTURE(Fixture, "for_in_loop_iterator_returns_any2") +{ + CheckResult result = check(R"( + function bar(): any + return true + end + + local a + for b in bar() do + a = b + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ("any", toString(requireType("a"))); +} + +TEST_CASE_FIXTURE(Fixture, "for_in_loop_iterator_is_any") +{ + CheckResult result = check(R"( + local bar: any + + local a + for b in bar do + a = b + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ("any", toString(requireType("a"))); +} + +TEST_CASE_FIXTURE(Fixture, "for_in_loop_iterator_is_any2") +{ + CheckResult result = check(R"( + local bar: any + + local a + for b in bar() do + a = b + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ("any", toString(requireType("a"))); +} + +TEST_CASE_FIXTURE(Fixture, "for_in_loop_iterator_is_error") +{ + CheckResult result = check(R"( + local a + for b in bar do + a = b + end + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + CHECK_EQ("*unknown*", toString(requireType("a"))); +} + +TEST_CASE_FIXTURE(Fixture, "for_in_loop_iterator_is_error2") +{ + CheckResult result = check(R"( + function bar(c) return c end + + local a + for b in bar() do + a = b + end + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + CHECK_EQ("*unknown*", toString(requireType("a"))); +} + +TEST_CASE_FIXTURE(Fixture, "length_of_error_type_does_not_produce_an_error") +{ + CheckResult result = check(R"( + local l = #this_is_not_defined + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); +} + +TEST_CASE_FIXTURE(Fixture, "indexing_error_type_does_not_produce_an_error") +{ + CheckResult result = check(R"( + local originalReward = unknown.Parent.Reward:GetChildren()[1] + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); +} + +TEST_CASE_FIXTURE(Fixture, "dot_on_error_type_does_not_produce_an_error") +{ + CheckResult result = check(R"( + local foo = (true).x + foo.x = foo.y + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); +} + +TEST_CASE_FIXTURE(Fixture, "any_type_propagates") +{ + CheckResult result = check(R"( + local foo: any + local bar = foo:method("argument") + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ("any", toString(requireType("bar"))); +} + +TEST_CASE_FIXTURE(Fixture, "can_subscript_any") +{ + CheckResult result = check(R"( + local foo: any + local bar = foo[5] + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ("any", toString(requireType("bar"))); +} + +// Not strictly correct: metatables permit overriding this +TEST_CASE_FIXTURE(Fixture, "can_get_length_of_any") +{ + CheckResult result = check(R"( + local foo: any = {} + local bar = #foo + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ(PrimitiveTypeVar::Number, getPrimitiveType(requireType("bar"))); +} + +TEST_CASE_FIXTURE(Fixture, "assign_prop_to_table_by_calling_any_yields_any") +{ + CheckResult result = check(R"( + local f: any + local T = {} + + T.prop = f() + + return T + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + TableTypeVar* ttv = getMutable(requireType("T")); + REQUIRE(ttv); + REQUIRE(ttv->props.count("prop")); + + REQUIRE_EQ("any", toString(ttv->props["prop"].type)); +} + +TEST_CASE_FIXTURE(Fixture, "quantify_any_does_not_bind_to_itself") +{ + CheckResult result = check(R"( + local A : any + function A.B() end + A:C() + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + TypeId aType = requireType("A"); + CHECK_EQ(aType, typeChecker.anyType); +} + +TEST_CASE_FIXTURE(Fixture, "calling_error_type_yields_error") +{ + CheckResult result = check(R"( + local a = unknown.Parent.Reward.GetChildren() + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + UnknownSymbol* err = get(result.errors[0]); + REQUIRE(err != nullptr); + + CHECK_EQ("unknown", err->name); + + CHECK_EQ("*unknown*", toString(requireType("a"))); +} + +TEST_CASE_FIXTURE(Fixture, "chain_calling_error_type_yields_error") +{ + CheckResult result = check(R"( + local a = Utility.Create "Foo" {} + )"); + + CHECK_EQ("*unknown*", toString(requireType("a"))); +} + +TEST_CASE_FIXTURE(Fixture, "replace_every_free_type_when_unifying_a_complex_function_with_any") +{ + CheckResult result = check(R"( + local a: any + local b + for _, i in pairs(a) do + b = i + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + CHECK_EQ("any", toString(requireType("b"))); +} + +TEST_CASE_FIXTURE(Fixture, "call_to_any_yields_any") +{ + CheckResult result = check(R"( + local a: any + local b = a() + )"); + + REQUIRE_EQ("any", toString(requireType("b"))); +} + +TEST_CASE_FIXTURE(Fixture, "CheckMethodsOfAny") +{ + CheckResult result = check(R"( +local x: any = {} +function x:y(z: number) + local s: string = z +end +)"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); +} + +TEST_CASE_FIXTURE(Fixture, "CheckMethodsOfError") +{ + CheckResult result = check(R"( +local x = (true).foo +function x:y(z: number) + local s: string = z +end +)"); + + LUAU_REQUIRE_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "metatable_of_any_can_be_a_table") +{ + CheckResult result = check(R"( +--!strict +local T: any +T = {} +T.__index = T +function T.new(...) + local self = {} + setmetatable(self, T) + self:construct(...) + return self +end +function T:construct(index) +end +)"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "type_error_addition") +{ + CheckResult result = check(R"( +--!strict +local foo = makesandwich() +local bar = foo.nutrition + 100 + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + // We should definitely get this error + CHECK_EQ("Unknown global 'makesandwich'", toString(result.errors[0])); + // We get this error if makesandwich() returns a free type + // CHECK_EQ("Unknown type used in + operation; consider adding a type annotation to 'foo'", toString(result.errors[1])); +} + +TEST_CASE_FIXTURE(Fixture, "prop_access_on_any_with_other_options") +{ + CheckResult result = check(R"( + local function f(thing: any | string) + local foo = thing.SomeRandomKey + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_SUITE_END(); diff --git a/tests/TypeInfer.builtins.test.cpp b/tests/TypeInfer.builtins.test.cpp index 8da655b3..ec20a2c7 100644 --- a/tests/TypeInfer.builtins.test.cpp +++ b/tests/TypeInfer.builtins.test.cpp @@ -871,6 +871,7 @@ TEST_CASE_FIXTURE(Fixture, "assert_removes_falsy_types") ScopedFastFlag sff[]{ {"LuauAssertStripsFalsyTypes", true}, {"LuauDiscriminableUnions2", true}, + {"LuauWidenIfSupertypeIsFree2", true}, }; CheckResult result = check(R"( @@ -879,6 +880,26 @@ TEST_CASE_FIXTURE(Fixture, "assert_removes_falsy_types") end )"); + LUAU_REQUIRE_NO_ERRORS(result); + CHECK_EQ("((boolean | number)?) -> boolean | number", toString(requireType("f"))); +} + +TEST_CASE_FIXTURE(Fixture, "assert_removes_falsy_types2") +{ + ScopedFastFlag sff[]{ + {"LuauParseSingletonTypes", true}, + {"LuauSingletonTypes", true}, + {"LuauAssertStripsFalsyTypes", true}, + {"LuauDiscriminableUnions2", true}, + {"LuauWidenIfSupertypeIsFree2", true}, + }; + + CheckResult result = check(R"( + local function f(x: (number | boolean)?): number | true + return assert(x) + end + )"); + LUAU_REQUIRE_NO_ERRORS(result); CHECK_EQ("((boolean | number)?) -> number | true", toString(requireType("f"))); } @@ -958,4 +979,43 @@ a:b({}) CHECK_EQ(result.errors[1], (TypeError{Location{{3, 0}, {3, 5}}, CountMismatch{2, 1}})); } +TEST_CASE_FIXTURE(Fixture, "typeof_unresolved_function") +{ + CheckResult result = check(R"( +local function f(a: typeof(f)) end +)"); + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ("Unknown global 'f'", toString(result.errors[0])); +} + +TEST_CASE_FIXTURE(Fixture, "no_persistent_typelevel_change") +{ + TypeId mathTy = requireType(typeChecker.globalScope, "math"); + REQUIRE(mathTy); + TableTypeVar* ttv = getMutable(mathTy); + REQUIRE(ttv); + const FunctionTypeVar* ftv = get(ttv->props["frexp"].type); + REQUIRE(ftv); + auto original = ftv->level; + + CheckResult result = check("local a = math.frexp"); + + LUAU_REQUIRE_NO_ERRORS(result); + CHECK(ftv->level.level == original.level); + CHECK(ftv->level.subLevel == original.subLevel); +} + +TEST_CASE_FIXTURE(Fixture, "global_singleton_types_are_sealed") +{ + CheckResult result = check(R"( +local function f(x: string) + local p = x:split('a') + p = table.pack(table.unpack(p, 1, #p - 1)) + return p +end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.definitions.test.cpp b/tests/TypeInfer.definitions.test.cpp index c6d55793..898d8902 100644 --- a/tests/TypeInfer.definitions.test.cpp +++ b/tests/TypeInfer.definitions.test.cpp @@ -293,4 +293,22 @@ TEST_CASE_FIXTURE(Fixture, "documentation_symbols_dont_attach_to_persistent_type CHECK_EQ(ty->type->documentationSymbol, std::nullopt); } +TEST_CASE_FIXTURE(Fixture, "single_class_type_identity_in_global_types") +{ + ScopedFastFlag luauCloneDeclaredGlobals{"LuauCloneDeclaredGlobals", true}; + + loadDefinition(R"( +declare class Cls +end + +declare GetCls: () -> (Cls) + )"); + + CheckResult result = check(R"( +local s : Cls = GetCls() + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.functions.test.cpp b/tests/TypeInfer.functions.test.cpp new file mode 100644 index 00000000..4288098a --- /dev/null +++ b/tests/TypeInfer.functions.test.cpp @@ -0,0 +1,1338 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details + +#include "Luau/AstQuery.h" +#include "Luau/BuiltinDefinitions.h" +#include "Luau/Scope.h" +#include "Luau/TypeInfer.h" +#include "Luau/TypeVar.h" +#include "Luau/VisitTypeVar.h" + +#include "Fixture.h" + +#include "doctest.h" + +using namespace Luau; + +TEST_SUITE_BEGIN("TypeInferFunctions"); + +TEST_CASE_FIXTURE(Fixture, "tc_function") +{ + CheckResult result = check("function five() return 5 end"); + LUAU_REQUIRE_NO_ERRORS(result); + + const FunctionTypeVar* fiveType = get(requireType("five")); + REQUIRE(fiveType != nullptr); +} + +TEST_CASE_FIXTURE(Fixture, "check_function_bodies") +{ + CheckResult result = check("function myFunction() local a = 0 a = true end"); + LUAU_REQUIRE_ERROR_COUNT(1, result); + + CHECK_EQ(result.errors[0], (TypeError{Location{Position{0, 44}, Position{0, 48}}, TypeMismatch{ + typeChecker.numberType, + typeChecker.booleanType, + }})); +} + +TEST_CASE_FIXTURE(Fixture, "infer_return_type") +{ + CheckResult result = check("function take_five() return 5 end"); + LUAU_REQUIRE_NO_ERRORS(result); + + const FunctionTypeVar* takeFiveType = get(requireType("take_five")); + REQUIRE(takeFiveType != nullptr); + + std::vector retVec = flatten(takeFiveType->retType).first; + REQUIRE(!retVec.empty()); + + REQUIRE_EQ(*follow(retVec[0]), *typeChecker.numberType); +} + +TEST_CASE_FIXTURE(Fixture, "infer_from_function_return_type") +{ + CheckResult result = check("function take_five() return 5 end local five = take_five()"); + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ(*typeChecker.numberType, *follow(requireType("five"))); +} + +TEST_CASE_FIXTURE(Fixture, "infer_that_function_does_not_return_a_table") +{ + CheckResult result = check(R"( + function take_five() + return 5 + end + + take_five().prop = 888 + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ(result.errors[0], (TypeError{Location{Position{5, 8}, Position{5, 24}}, NotATable{typeChecker.numberType}})); +} + +TEST_CASE_FIXTURE(Fixture, "vararg_functions_should_allow_calls_of_any_types_and_size") +{ + CheckResult result = check(R"( + function f(...) end + + f(1) + f("foo", 2) + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "vararg_function_is_quantified") +{ + CheckResult result = check(R"( + local T = {} + function T.f(...) + local result = {} + + for i = 1, select("#", ...) do + local dictionary = select(i, ...) + for key, value in pairs(dictionary) do + result[key] = value + end + end + + return result + end + + return T + )"); + + auto r = first(getMainModule()->getModuleScope()->returnType); + REQUIRE(r); + + TableTypeVar* ttv = getMutable(*r); + REQUIRE(ttv); + + TypeId k = ttv->props["f"].type; + REQUIRE(k); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "list_only_alternative_overloads_that_match_argument_count") +{ + CheckResult result = check(R"( + local multiply: ((number)->number) & ((number)->string) & ((number, number)->number) + multiply("") + )"); + + LUAU_REQUIRE_ERROR_COUNT(2, result); + + TypeMismatch* tm = get(result.errors[0]); + REQUIRE(tm); + CHECK_EQ(typeChecker.numberType, tm->wantedType); + CHECK_EQ(typeChecker.stringType, tm->givenType); + + ExtraInformation* ei = get(result.errors[1]); + REQUIRE(ei); + CHECK_EQ("Other overloads are also not viable: (number) -> string", ei->message); +} + +TEST_CASE_FIXTURE(Fixture, "list_all_overloads_if_no_overload_takes_given_argument_count") +{ + CheckResult result = check(R"( + local multiply: ((number)->number) & ((number)->string) & ((number, number)->number) + multiply() + )"); + + LUAU_REQUIRE_ERROR_COUNT(2, result); + + GenericError* ge = get(result.errors[0]); + REQUIRE(ge); + CHECK_EQ("No overload for function accepts 0 arguments.", ge->message); + + ExtraInformation* ei = get(result.errors[1]); + REQUIRE(ei); + CHECK_EQ("Available overloads: (number) -> number; (number) -> string; and (number, number) -> number", ei->message); +} + +TEST_CASE_FIXTURE(Fixture, "dont_give_other_overloads_message_if_only_one_argument_matching_overload_exists") +{ + CheckResult result = check(R"( + local multiply: ((number)->number) & ((number)->string) & ((number, number)->number) + multiply(1, "") + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + TypeMismatch* tm = get(result.errors[0]); + REQUIRE(tm); + CHECK_EQ(typeChecker.numberType, tm->wantedType); + CHECK_EQ(typeChecker.stringType, tm->givenType); +} + +TEST_CASE_FIXTURE(Fixture, "infer_return_type_from_selected_overload") +{ + CheckResult result = check(R"( + type T = {method: ((T, number) -> number) & ((number) -> string)} + local T: T + + local a = T.method(T, 4) + local b = T.method(5) + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + CHECK_EQ("number", toString(requireType("a"))); + CHECK_EQ("string", toString(requireType("b"))); +} + +TEST_CASE_FIXTURE(Fixture, "too_many_arguments") +{ + CheckResult result = check(R"( + --!nonstrict + + function g(a: number) end + + g() + + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + auto err = result.errors[0]; + auto acm = get(err); + REQUIRE(acm); + + CHECK_EQ(1, acm->expected); + CHECK_EQ(0, acm->actual); +} + +TEST_CASE_FIXTURE(Fixture, "recursive_function") +{ + CheckResult result = check(R"( + function count(n: number) + if n == 0 then + return 0 + else + return count(n - 1) + end + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "lambda_form_of_local_function_cannot_be_recursive") +{ + CheckResult result = check(R"( + local f = function() return f() end + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); +} + +TEST_CASE_FIXTURE(Fixture, "recursive_local_function") +{ + CheckResult result = check(R"( + local function count(n: number) + if n == 0 then + return 0 + else + return count(n - 1) + end + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +// FIXME: This and the above case get handled very differently. It's pretty dumb. +// We really should unify the two code paths, probably by deleting AstStatFunction. +TEST_CASE_FIXTURE(Fixture, "another_recursive_local_function") +{ + CheckResult result = check(R"( + local count + function count(n: number) + if n == 0 then + return 0 + else + return count(n - 1) + end + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "cyclic_function_type_in_rets") +{ + CheckResult result = check(R"( + function f() + return f + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + CHECK_EQ("t1 where t1 = () -> t1", toString(requireType("f"))); +} + +TEST_CASE_FIXTURE(Fixture, "cyclic_function_type_in_args") +{ + CheckResult result = check(R"( + function f(g) + return f(f) + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + CHECK_EQ("t1 where t1 = (t1) -> ()", toString(requireType("f"))); +} + +TEST_CASE_FIXTURE(Fixture, "another_higher_order_function") +{ + CheckResult result = check(R"( + local Get_des + function Get_des(func) + Get_des(func) + end + + local function f(d) + d:IsA("BasePart") + d.Parent:FindFirstChild("Humanoid") + d:IsA("Decal") + end + Get_des(f) + + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "another_other_higher_order_function") +{ + CheckResult result = check(R"( + local d + d:foo() + d:foo() + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "local_function") +{ + CheckResult result = check(R"( + function f() + return 8 + end + + function g() + local function f() + return 'hello' + end + return f + end + + local h = g() + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + TypeId h = follow(requireType("h")); + + const FunctionTypeVar* ftv = get(h); + REQUIRE(ftv != nullptr); + + std::optional rt = first(ftv->retType); + REQUIRE(bool(rt)); + + TypeId retType = follow(*rt); + CHECK_EQ(PrimitiveTypeVar::String, getPrimitiveType(retType)); +} + +TEST_CASE_FIXTURE(Fixture, "func_expr_doesnt_leak_free") +{ + CheckResult result = check(R"( + local p = function(x) return x end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + const Luau::FunctionTypeVar* fn = get(requireType("p")); + REQUIRE(fn); + auto ret = first(fn->retType); + REQUIRE(ret); + REQUIRE(get(follow(*ret))); +} + +TEST_CASE_FIXTURE(Fixture, "first_argument_can_be_optional") +{ + CheckResult result = check(R"( + local T = {} + function T.new(a: number?, b: number?, c: number?) return 5 end + local m = T.new() + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + dumpErrors(result); +} + +TEST_CASE_FIXTURE(Fixture, "it_is_ok_not_to_supply_enough_retvals") +{ + CheckResult result = check(R"( + function get_two() return 5, 6 end + + local a = get_two() + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + dumpErrors(result); +} + +TEST_CASE_FIXTURE(Fixture, "duplicate_functions2") +{ + CheckResult result = check(R"( + function foo() end + + function bar() + local function foo() end + end + )"); + + LUAU_REQUIRE_ERROR_COUNT(0, result); +} + +TEST_CASE_FIXTURE(Fixture, "duplicate_functions_allowed_in_nonstrict") +{ + CheckResult result = check(R"( + --!nonstrict + function foo() end + + function foo() end + + function bar() + local function foo() end + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "duplicate_functions_with_different_signatures_not_allowed_in_nonstrict") +{ + CheckResult result = check(R"( + --!nonstrict + function foo(): number + return 1 + end + foo() + + function foo(n: number): number + return 2 + end + foo() + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + TypeMismatch* tm = get(result.errors[0]); + REQUIRE(tm); + CHECK_EQ("() -> number", toString(tm->wantedType)); + CHECK_EQ("(number) -> number", toString(tm->givenType)); +} + +TEST_CASE_FIXTURE(Fixture, "complicated_return_types_require_an_explicit_annotation") +{ + CheckResult result = check(R"( + local i = 0 + function most_of_the_natural_numbers(): number? + if i < 10 then + i = i + 1 + return i + else + return nil + end + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + const FunctionTypeVar* functionType = get(requireType("most_of_the_natural_numbers")); + + std::optional retType = first(functionType->retType); + REQUIRE(retType); + CHECK(get(*retType)); +} + +TEST_CASE_FIXTURE(Fixture, "infer_higher_order_function") +{ + CheckResult result = check(R"( + function apply(f, x) + return f(x) + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + const FunctionTypeVar* ftv = get(requireType("apply")); + REQUIRE(ftv != nullptr); + + std::vector argVec = flatten(ftv->argTypes).first; + + REQUIRE_EQ(2, argVec.size()); + + const FunctionTypeVar* fType = get(follow(argVec[0])); + REQUIRE(fType != nullptr); + + std::vector fArgs = flatten(fType->argTypes).first; + + TypeId xType = argVec[1]; + + CHECK_EQ(1, fArgs.size()); + CHECK_EQ(xType, fArgs[0]); +} + +TEST_CASE_FIXTURE(Fixture, "higher_order_function_2") +{ + CheckResult result = check(R"( + function bottomupmerge(comp, a, b, left, mid, right) + local i, j = left, mid + for k = left, right do + if i < mid and (j > right or not comp(a[j], a[i])) then + b[k] = a[i] + i = i + 1 + else + b[k] = a[j] + j = j + 1 + end + end + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + const FunctionTypeVar* ftv = get(requireType("bottomupmerge")); + REQUIRE(ftv != nullptr); + + std::vector argVec = flatten(ftv->argTypes).first; + + REQUIRE_EQ(6, argVec.size()); + + const FunctionTypeVar* fType = get(follow(argVec[0])); + REQUIRE(fType != nullptr); +} + +TEST_CASE_FIXTURE(Fixture, "higher_order_function_3") +{ + CheckResult result = check(R"( + function swap(p) + local t = p[0] + p[0] = p[1] + p[1] = t + return nil + end + + function swapTwice(p) + swap(p) + swap(p) + return p + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + const FunctionTypeVar* ftv = get(requireType("swapTwice")); + REQUIRE(ftv != nullptr); + + std::vector argVec = flatten(ftv->argTypes).first; + + REQUIRE_EQ(1, argVec.size()); + + const TableTypeVar* argType = get(follow(argVec[0])); + REQUIRE(argType != nullptr); + + CHECK(bool(argType->indexer)); +} + +TEST_CASE_FIXTURE(Fixture, "higher_order_function_4") +{ + CheckResult result = check(R"( + function bottomupmerge(comp, a, b, left, mid, right) + local i, j = left, mid + for k = left, right do + if i < mid and (j > right or not comp(a[j], a[i])) then + b[k] = a[i] + i = i + 1 + else + b[k] = a[j] + j = j + 1 + end + end + end + + function mergesort(arr, comp) + local work = {} + for i = 1, #arr do + work[i] = arr[i] + end + local width = 1 + while width < #arr do + for i = 1, #arr, 2*width do + bottomupmerge(comp, arr, work, i, math.min(i+width, #arr), math.min(i+2*width-1, #arr)) + end + local temp = work + work = arr + arr = temp + width = width * 2 + end + return arr + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + dumpErrors(result); + + /* + * mergesort takes two arguments: an array of some type T and a function that takes two Ts. + * We must assert that these two types are in fact the same type. + * In other words, comp(arr[x], arr[y]) is well-typed. + */ + + const FunctionTypeVar* ftv = get(requireType("mergesort")); + REQUIRE(ftv != nullptr); + + std::vector argVec = flatten(ftv->argTypes).first; + + REQUIRE_EQ(2, argVec.size()); + + const TableTypeVar* arg0 = get(follow(argVec[0])); + REQUIRE(arg0 != nullptr); + REQUIRE(bool(arg0->indexer)); + + const FunctionTypeVar* arg1 = get(follow(argVec[1])); + REQUIRE(arg1 != nullptr); + REQUIRE_EQ(2, size(arg1->argTypes)); + + std::vector arg1Args = flatten(arg1->argTypes).first; + + CHECK_EQ(*arg0->indexer->indexResultType, *arg1Args[0]); + CHECK_EQ(*arg0->indexer->indexResultType, *arg1Args[1]); +} + +TEST_CASE_FIXTURE(Fixture, "mutual_recursion") +{ + CheckResult result = check(R"( + --!strict + + function newPlayerCharacter() + startGui() -- Unknown symbol 'startGui' + end + + local characterAddedConnection: any + function startGui() + characterAddedConnection = game:GetService("Players").LocalPlayer.CharacterAdded:connect(newPlayerCharacter) + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + dumpErrors(result); +} + +TEST_CASE_FIXTURE(Fixture, "toposort_doesnt_break_mutual_recursion") +{ + CheckResult result = check(R"( + --!strict + local x = nil + function f() g() end + -- make sure print(x) doesn't get toposorted here, breaking the mutual block + function g() x = f end + print(x) + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + dumpErrors(result); +} + +TEST_CASE_FIXTURE(Fixture, "check_function_before_lambda_that_uses_it") +{ + CheckResult result = check(R"( + --!nonstrict + + function f() + return 114 + end + + return function() + return f():andThen() + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "it_is_ok_to_oversaturate_a_higher_order_function_argument") +{ + CheckResult result = check(R"( + function onerror() end + function foo() end + xpcall(foo, onerror) + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "another_indirect_function_case_where_it_is_ok_to_provide_too_many_arguments") +{ + CheckResult result = check(R"( + local mycb: (number, number) -> () + + function f() end + + mycb = f + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "report_exiting_without_return_nonstrict") +{ + CheckResult result = check(R"( + --!nonstrict + + local function f1(v): number? + if v then + return 1 + end + end + + local function f2(v) + if v then + return 1 + end + end + + local function f3(v): () + if v then + return + end + end + + local function f4(v) + if v then + return + end + end + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + FunctionExitsWithoutReturning* err = get(result.errors[0]); + CHECK(err); +} + +TEST_CASE_FIXTURE(Fixture, "report_exiting_without_return_strict") +{ + CheckResult result = check(R"( + --!strict + + local function f1(v): number? + if v then + return 1 + end + end + + local function f2(v) + if v then + return 1 + end + end + + local function f3(v): () + if v then + return + end + end + + local function f4(v) + if v then + return + end + end + )"); + + LUAU_REQUIRE_ERROR_COUNT(2, result); + FunctionExitsWithoutReturning* annotatedErr = get(result.errors[0]); + CHECK(annotatedErr); + + FunctionExitsWithoutReturning* inferredErr = get(result.errors[1]); + CHECK(inferredErr); +} + +TEST_CASE_FIXTURE(Fixture, "calling_function_with_incorrect_argument_type_yields_errors_spanning_argument") +{ + CheckResult result = check(R"( + function foo(a: number, b: string) end + + foo("Test", 123) + )"); + + LUAU_REQUIRE_ERROR_COUNT(2, result); + + CHECK_EQ(result.errors[0], (TypeError{Location{Position{3, 12}, Position{3, 18}}, TypeMismatch{ + typeChecker.numberType, + typeChecker.stringType, + }})); + + CHECK_EQ(result.errors[1], (TypeError{Location{Position{3, 20}, Position{3, 23}}, TypeMismatch{ + typeChecker.stringType, + typeChecker.numberType, + }})); +} + +TEST_CASE_FIXTURE(Fixture, "calling_function_with_anytypepack_doesnt_leak_free_types") +{ + CheckResult result = check(R"( + --!nonstrict + + function Test(a) + return 1, "" + end + + + local tab = {} + table.insert(tab, Test(1)); + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + ToStringOptions opts; + opts.exhaustive = true; + opts.maxTableLength = 0; + + CHECK_EQ("{any}", toString(requireType("tab"), opts)); +} + +TEST_CASE_FIXTURE(Fixture, "too_many_return_values") +{ + CheckResult result = check(R"( + --!strict + + function f() + return 55 + end + + local a, b = f() + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + CountMismatch* acm = get(result.errors[0]); + REQUIRE(acm); + CHECK_EQ(acm->context, CountMismatch::Result); + CHECK_EQ(acm->expected, 1); + CHECK_EQ(acm->actual, 2); +} + +TEST_CASE_FIXTURE(Fixture, "ignored_return_values") +{ + CheckResult result = check(R"( + --!strict + + function f() + return 55, "" + end + + local a = f() + )"); + + LUAU_REQUIRE_ERROR_COUNT(0, result); +} + +TEST_CASE_FIXTURE(Fixture, "function_does_not_return_enough_values") +{ + CheckResult result = check(R"( + --!strict + + function f(): (number, string) + return 55 + end + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + CountMismatch* acm = get(result.errors[0]); + REQUIRE(acm); + CHECK_EQ(acm->context, CountMismatch::Return); + CHECK_EQ(acm->expected, 2); + CHECK_EQ(acm->actual, 1); +} + +TEST_CASE_FIXTURE(Fixture, "function_cast_error_uses_correct_language") +{ + CheckResult result = check(R"( + function foo(a, b): number + return 0 + end + + local a: (string)->number = foo + local b: (number, number)->(number, number) = foo + + local c: (string, number)->number = foo -- no error + )"); + + LUAU_REQUIRE_ERROR_COUNT(2, result); + + auto tm1 = get(result.errors[0]); + REQUIRE(tm1); + + CHECK_EQ("(string) -> number", toString(tm1->wantedType)); + CHECK_EQ("(string, *unknown*) -> number", toString(tm1->givenType)); + + auto tm2 = get(result.errors[1]); + REQUIRE(tm2); + + CHECK_EQ("(number, number) -> (number, number)", toString(tm2->wantedType)); + CHECK_EQ("(string, *unknown*) -> number", toString(tm2->givenType)); +} + +TEST_CASE_FIXTURE(Fixture, "no_lossy_function_type") +{ + CheckResult result = check(R"( + --!strict + local tbl = {} + function tbl:abc(a: number, b: number) + return a + end + tbl:abc(1, 2) -- Line 6 + -- | Column 14 + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + TypeId type = requireTypeAtPosition(Position(6, 14)); + CHECK_EQ("(tbl, number, number) -> number", toString(type)); + auto ftv = get(type); + REQUIRE(ftv); + CHECK(ftv->hasSelf); +} + +TEST_CASE_FIXTURE(Fixture, "record_matching_overload") +{ + CheckResult result = check(R"( + type Overload = ((string) -> string) & ((number) -> number) + local abc: Overload + abc(1) + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + // AstExprCall is the node that has the overload stored on it. + // findTypeAtPosition will look at the AstExprLocal, but this is not what + // we want to look at. + std::vector ancestry = findAstAncestryOfPosition(*getMainSourceModule(), Position(3, 10)); + REQUIRE_GE(ancestry.size(), 2); + AstExpr* parentExpr = ancestry[ancestry.size() - 2]->asExpr(); + REQUIRE(bool(parentExpr)); + REQUIRE(parentExpr->is()); + + ModulePtr module = getMainModule(); + auto it = module->astOverloadResolvedTypes.find(parentExpr); + REQUIRE(it); + CHECK_EQ(toString(*it), "(number) -> number"); +} + +TEST_CASE_FIXTURE(Fixture, "return_type_by_overload") +{ + ScopedFastFlag sff{"LuauErrorRecoveryType", true}; + + CheckResult result = check(R"( + type Overload = ((string) -> string) & ((number, number) -> number) + local abc: Overload + local x = abc(true) + local y = abc(true,true) + local z = abc(true,true,true) + )"); + + LUAU_REQUIRE_ERRORS(result); + CHECK_EQ("string", toString(requireType("x"))); + CHECK_EQ("number", toString(requireType("y"))); + // Should this be string|number? + CHECK_EQ("string", toString(requireType("z"))); +} + +TEST_CASE_FIXTURE(Fixture, "infer_anonymous_function_arguments") +{ + // Simple direct arg to arg propagation + CheckResult result = check(R"( +type Table = { x: number, y: number } +local function f(a: (Table) -> number) return a({x = 1, y = 2}) end +f(function(a) return a.x + a.y end) + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + // An optional function is accepted, but since we already provide a function, nil can be ignored + result = check(R"( +type Table = { x: number, y: number } +local function f(a: ((Table) -> number)?) if a then return a({x = 1, y = 2}) else return 0 end end +f(function(a) return a.x + a.y end) + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + // Make sure self calls match correct index + result = check(R"( +type Table = { x: number, y: number } +local x = {} +x.b = {x = 1, y = 2} +function x:f(a: (Table) -> number) return a(self.b) end +x:f(function(a) return a.x + a.y end) + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + // Mix inferred and explicit argument types + result = check(R"( +function f(a: (a: number, b: number, c: boolean) -> number) return a(1, 2, true) end +f(function(a: number, b, c) return c and a + b or b - a end) + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + // Anonymous function has a variadic pack + result = check(R"( +type Table = { x: number, y: number } +local function f(a: (Table) -> number) return a({x = 1, y = 2}) end +f(function(...) return select(1, ...).z end) + )"); + + LUAU_REQUIRE_ERRORS(result); + CHECK_EQ("Key 'z' not found in table 'Table'", toString(result.errors[0])); + + // Can't accept more arguments than provided + result = check(R"( +function f(a: (a: number, b: number) -> number) return a(1, 2) end +f(function(a, b, c, ...) return a + b end) + )"); + + LUAU_REQUIRE_ERRORS(result); + + CHECK_EQ(R"(Type '(number, number, a) -> number' could not be converted into '(number, number) -> number' +caused by: + Argument count mismatch. Function expects 3 arguments, but only 2 are specified)", + toString(result.errors[0])); + + // Infer from variadic packs into elements + result = check(R"( +function f(a: (...number) -> number) return a(1, 2) end +f(function(a, b) return a + b end) + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + // Infer from variadic packs into variadic packs + result = check(R"( +type Table = { x: number, y: number } +function f(a: (...Table) -> number) return a({x = 1, y = 2}, {x = 3, y = 4}) end +f(function(a, ...) local b = ... return b.z end) + )"); + + LUAU_REQUIRE_ERRORS(result); + CHECK_EQ("Key 'z' not found in table 'Table'", toString(result.errors[0])); + + // Return type inference + result = check(R"( +type Table = { x: number, y: number } +function f(a: (number) -> Table) return a(4) end +f(function(x) return x * 2 end) + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ("Type 'number' could not be converted into 'Table'", toString(result.errors[0])); + + // Return type doesn't inference 'nil' + result = check(R"( +function f(a: (number) -> nil) return a(4) end +f(function(x) print(x) end) + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "infer_anonymous_function_arguments") +{ + // Simple direct arg to arg propagation + CheckResult result = check(R"( +type Table = { x: number, y: number } +local function f(a: (Table) -> number) return a({x = 1, y = 2}) end +f(function(a) return a.x + a.y end) + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + // An optional function is accepted, but since we already provide a function, nil can be ignored + result = check(R"( +type Table = { x: number, y: number } +local function f(a: ((Table) -> number)?) if a then return a({x = 1, y = 2}) else return 0 end end +f(function(a) return a.x + a.y end) + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + // Make sure self calls match correct index + result = check(R"( +type Table = { x: number, y: number } +local x = {} +x.b = {x = 1, y = 2} +function x:f(a: (Table) -> number) return a(self.b) end +x:f(function(a) return a.x + a.y end) + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + // Mix inferred and explicit argument types + result = check(R"( +function f(a: (a: number, b: number, c: boolean) -> number) return a(1, 2, true) end +f(function(a: number, b, c) return c and a + b or b - a end) + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + // Anonymous function has a variadic pack + result = check(R"( +type Table = { x: number, y: number } +local function f(a: (Table) -> number) return a({x = 1, y = 2}) end +f(function(...) return select(1, ...).z end) + )"); + + LUAU_REQUIRE_ERRORS(result); + CHECK_EQ("Key 'z' not found in table 'Table'", toString(result.errors[0])); + + // Can't accept more arguments than provided + result = check(R"( +function f(a: (a: number, b: number) -> number) return a(1, 2) end +f(function(a, b, c, ...) return a + b end) + )"); + + LUAU_REQUIRE_ERRORS(result); + + CHECK_EQ(R"(Type '(number, number, a) -> number' could not be converted into '(number, number) -> number' +caused by: + Argument count mismatch. Function expects 3 arguments, but only 2 are specified)", + toString(result.errors[0])); + + // Infer from variadic packs into elements + result = check(R"( +function f(a: (...number) -> number) return a(1, 2) end +f(function(a, b) return a + b end) + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + // Infer from variadic packs into variadic packs + result = check(R"( +type Table = { x: number, y: number } +function f(a: (...Table) -> number) return a({x = 1, y = 2}, {x = 3, y = 4}) end +f(function(a, ...) local b = ... return b.z end) + )"); + + LUAU_REQUIRE_ERRORS(result); + CHECK_EQ("Key 'z' not found in table 'Table'", toString(result.errors[0])); + + // Return type inference + result = check(R"( +type Table = { x: number, y: number } +function f(a: (number) -> Table) return a(4) end +f(function(x) return x * 2 end) + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ("Type 'number' could not be converted into 'Table'", toString(result.errors[0])); + + // Return type doesn't inference 'nil' + result = check(R"( +function f(a: (number) -> nil) return a(4) end +f(function(x) print(x) end) + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "infer_anonymous_function_arguments_outside_call") +{ + CheckResult result = check(R"( +type Table = { x: number, y: number } +local f: (Table) -> number = function(t) return t.x + t.y end + +type TableWithFunc = { x: number, y: number, f: (number, number) -> number } +local a: TableWithFunc = { x = 3, y = 4, f = function(a, b) return a + b end } + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "infer_return_value_type") +{ + CheckResult result = check(R"( +local function f(): {string|number} + return {1, "b", 3} +end + +local function g(): (number, {string|number}) + return 4, {1, "b", 3} +end + +local function h(): ...{string|number} + return {4}, {1, "b", 3}, {"s"} +end + +local function i(): ...{string|number} + return {1, "b", 3}, h() +end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "error_detailed_function_mismatch_arg_count") +{ + CheckResult result = check(R"( +type A = (number, number) -> string +type B = (number) -> string + +local a: A +local b: B = a + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ(toString(result.errors[0]), R"(Type '(number, number) -> string' could not be converted into '(number) -> string' +caused by: + Argument count mismatch. Function expects 2 arguments, but only 1 is specified)"); +} + +TEST_CASE_FIXTURE(Fixture, "error_detailed_function_mismatch_arg") +{ + CheckResult result = check(R"( +type A = (number, number) -> string +type B = (number, string) -> string + +local a: A +local b: B = a + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ(toString(result.errors[0]), R"(Type '(number, number) -> string' could not be converted into '(number, string) -> string' +caused by: + Argument #2 type is not compatible. Type 'string' could not be converted into 'number')"); +} + +TEST_CASE_FIXTURE(Fixture, "error_detailed_function_mismatch_ret_count") +{ + CheckResult result = check(R"( +type A = (number, number) -> (number) +type B = (number, number) -> (number, boolean) + +local a: A +local b: B = a + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ(toString(result.errors[0]), R"(Type '(number, number) -> number' could not be converted into '(number, number) -> (number, boolean)' +caused by: + Function only returns 1 value. 2 are required here)"); +} + +TEST_CASE_FIXTURE(Fixture, "error_detailed_function_mismatch_ret") +{ + CheckResult result = check(R"( +type A = (number, number) -> string +type B = (number, number) -> number + +local a: A +local b: B = a + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ(toString(result.errors[0]), R"(Type '(number, number) -> string' could not be converted into '(number, number) -> number' +caused by: + Return type is not compatible. Type 'string' could not be converted into 'number')"); +} + +TEST_CASE_FIXTURE(Fixture, "error_detailed_function_mismatch_ret_mult") +{ + CheckResult result = check(R"( +type A = (number, number) -> (number, string) +type B = (number, number) -> (number, boolean) + +local a: A +local b: B = a + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ(toString(result.errors[0]), + R"(Type '(number, number) -> (number, string)' could not be converted into '(number, number) -> (number, boolean)' +caused by: + Return #2 type is not compatible. Type 'string' could not be converted into 'boolean')"); +} + +TEST_CASE_FIXTURE(Fixture, "function_decl_quantify_right_type") +{ + ScopedFastFlag statFunctionSimplify{"LuauStatFunctionSimplify2", true}; + + fileResolver.source["game/isAMagicMock"] = R"( +--!nonstrict +return function(value) + return false +end + )"; + + CheckResult result = check(R"( +--!nonstrict +local MagicMock = {} +MagicMock.is = require(game.isAMagicMock) + +function MagicMock.is(value) + return false +end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "function_decl_non_self_sealed_overwrite") +{ + ScopedFastFlag statFunctionSimplify{"LuauStatFunctionSimplify2", true}; + + CheckResult result = check(R"( +function string.len(): number + return 1 +end + )"); + + LUAU_REQUIRE_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "strict_mode_ok_with_missing_arguments") +{ + ScopedFastFlag sff{"LuauAnyInIsOptionalIsOptional", true}; + + CheckResult result = check(R"( + local function f(x: any) end + f() + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "function_statement_sealed_table_assignment_through_indexer") +{ + ScopedFastFlag statFunctionSimplify{"LuauStatFunctionSimplify2", true}; + + CheckResult result = check(R"( +local t: {[string]: () -> number} = {} + +function t.a() return 1 end -- OK +function t:b() return 2 end -- not OK + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ(R"(Type '(*unknown*) -> number' could not be converted into '() -> number' +caused by: + Argument count mismatch. Function expects 1 argument, but none are specified)", + toString(result.errors[0])); +} + +TEST_SUITE_END(); diff --git a/tests/TypeInfer.generics.test.cpp b/tests/TypeInfer.generics.test.cpp index 547fbab1..f360a77c 100644 --- a/tests/TypeInfer.generics.test.cpp +++ b/tests/TypeInfer.generics.test.cpp @@ -1,6 +1,9 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #include "Luau/TypeInfer.h" #include "Luau/TypeVar.h" +#include "Luau/Scope.h" + +#include #include "Fixture.h" @@ -830,5 +833,303 @@ wrapper(test2, 1, "", 3) CHECK_EQ(toString(result.errors[0]), R"(Argument count mismatch. Function expects 1 argument, but 4 are specified)"); } +TEST_CASE_FIXTURE(Fixture, "generic_function") +{ + CheckResult result = check(R"( + function id(x) return x end + local a = id(55) + local b = id(nil) + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ(*typeChecker.numberType, *requireType("a")); + CHECK_EQ(*typeChecker.nilType, *requireType("b")); +} + +TEST_CASE_FIXTURE(Fixture, "generic_table_method") +{ + CheckResult result = check(R"( + local T = {} + + function T:bar(i) + return i + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + TypeId tType = requireType("T"); + TableTypeVar* tTable = getMutable(tType); + REQUIRE(tTable != nullptr); + + TypeId barType = tTable->props["bar"].type; + REQUIRE(barType != nullptr); + + const FunctionTypeVar* ftv = get(follow(barType)); + REQUIRE_MESSAGE(ftv != nullptr, "Should be a function: " << *barType); + + std::vector args = flatten(ftv->argTypes).first; + TypeId argType = args.at(1); + + CHECK_MESSAGE(get(argType), "Should be generic: " << *barType); +} + +TEST_CASE_FIXTURE(Fixture, "correctly_instantiate_polymorphic_member_functions") +{ + CheckResult result = check(R"( + local T = {} + + function T:foo() + return T:bar(5) + end + + function T:bar(i) + return i + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + dumpErrors(result); + + const TableTypeVar* t = get(requireType("T")); + REQUIRE(t != nullptr); + + std::optional fooProp = get(t->props, "foo"); + REQUIRE(bool(fooProp)); + + const FunctionTypeVar* foo = get(follow(fooProp->type)); + REQUIRE(bool(foo)); + + std::optional ret_ = first(foo->retType); + REQUIRE(bool(ret_)); + TypeId ret = follow(*ret_); + + REQUIRE_EQ(getPrimitiveType(ret), PrimitiveTypeVar::Number); +} + +/* + * We had a bug in instantiation where the argument types of 'f' and 'g' would be inferred as + * f {+ method: function(): (t2, T3...) +} + * g {+ method: function({+ method: function(): (t2, T3...) +}): (t5, T6...) +} + * + * The type of 'g' is totally wrong as t2 and t5 should be unified, as should T3 with T6. + * + * The correct unification of the argument to 'g' is + * + * {+ method: function(): (t5, T6...) +} + */ +TEST_CASE_FIXTURE(Fixture, "instantiate_cyclic_generic_function") +{ + auto result = check(R"( + function f(o) + o:method() + end + + function g(o) + f(o) + end + )"); + + TypeId g = requireType("g"); + const FunctionTypeVar* gFun = get(g); + REQUIRE(gFun != nullptr); + + auto optionArg = first(gFun->argTypes); + REQUIRE(bool(optionArg)); + + TypeId arg = follow(*optionArg); + const TableTypeVar* argTable = get(arg); + REQUIRE(argTable != nullptr); + + std::optional methodProp = get(argTable->props, "method"); + REQUIRE(bool(methodProp)); + + const FunctionTypeVar* methodFunction = get(methodProp->type); + REQUIRE(methodFunction != nullptr); + + std::optional methodArg = first(methodFunction->argTypes); + REQUIRE(bool(methodArg)); + + REQUIRE_EQ(follow(*methodArg), follow(arg)); +} + +TEST_CASE_FIXTURE(Fixture, "instantiate_generic_function_in_assignments") +{ + CheckResult result = check(R"( + function foo(a, b) + return a(b) + end + + function bar() + local c: ((number)->number, number)->number = foo -- no error + c = foo -- no error + local d: ((number)->number, string)->number = foo -- error from arg 2 (string) not being convertable to number from the call a(b) + end + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + TypeMismatch* tm = get(result.errors[0]); + REQUIRE(tm); + CHECK_EQ("((number) -> number, string) -> number", toString(tm->wantedType)); + CHECK_EQ("((number) -> number, number) -> number", toString(tm->givenType)); +} + +TEST_CASE_FIXTURE(Fixture, "instantiate_generic_function_in_assignments2") +{ + CheckResult result = check(R"( + function foo(a, b) + return a(b) + end + + function bar() + local _: (string, string)->number = foo -- string cannot be converted to (string)->number + end + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + TypeMismatch* tm = get(result.errors[0]); + REQUIRE(tm); + CHECK_EQ("(string, string) -> number", toString(tm->wantedType)); + CHECK_EQ("((string) -> number, string) -> number", toString(*tm->givenType)); +} + +TEST_CASE_FIXTURE(Fixture, "self_recursive_instantiated_param") +{ + ScopedFastFlag sff{"LuauOnlyMutateInstantiatedTables", true}; + + // Mutability in type function application right now can create strange recursive types + CheckResult result = check(R"( +type Table = { a: number } +type Self = T +local a: Self
+ )"); + + LUAU_REQUIRE_NO_ERRORS(result); + CHECK_EQ(toString(requireType("a")), "Table"); +} + +TEST_CASE_FIXTURE(Fixture, "no_stack_overflow_from_quantifying") +{ + CheckResult result = check(R"( + function _(l0:t0): (any, ()->()) + end + + type t0 = t0 | {} + )"); + + CHECK_LE(0, result.errors.size()); + + std::optional t0 = getMainModule()->getModuleScope()->lookupType("t0"); + REQUIRE(t0); + CHECK_EQ("*unknown*", toString(t0->type)); + + auto it = std::find_if(result.errors.begin(), result.errors.end(), [](TypeError& err) { + return get(err); + }); + CHECK(it != result.errors.end()); +} + +TEST_CASE_FIXTURE(Fixture, "infer_generic_function_function_argument") +{ + ScopedFastFlag sff{"LuauUnsealedTableLiteral", true}; + + CheckResult result = check(R"( +local function sum(x: a, y: a, f: (a, a) -> a) return f(x, y) end +return sum(2, 3, function(a, b) return a + b end) + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + result = check(R"( +local function map(arr: {a}, f: (a) -> b) local r = {} for i,v in ipairs(arr) do table.insert(r, f(v)) end return r end +local a = {1, 2, 3} +local r = map(a, function(a) return a + a > 100 end) + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + REQUIRE_EQ("{boolean}", toString(requireType("r"))); + + check(R"( +local function foldl(arr: {a}, init: b, f: (b, a) -> b) local r = init for i,v in ipairs(arr) do r = f(r, v) end return r end +local a = {1, 2, 3} +local r = foldl(a, {s=0,c=0}, function(a, b) return {s = a.s + b, c = a.c + 1} end) + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + REQUIRE_EQ("{ c: number, s: number }", toString(requireType("r"))); +} + +TEST_CASE_FIXTURE(Fixture, "infer_generic_function_function_argument_overloaded") +{ + CheckResult result = check(R"( +local function g1(a: T, f: (T) -> T) return f(a) end +local function g2(a: T, b: T, f: (T, T) -> T) return f(a, b) end + +local g12: typeof(g1) & typeof(g2) + +g12(1, function(x) return x + x end) +g12(1, 2, function(x, y) return x + y end) + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + result = check(R"( +local function g1(a: T, f: (T) -> T) return f(a) end +local function g2(a: T, b: T, f: (T, T) -> T) return f(a, b) end + +local g12: typeof(g1) & typeof(g2) + +g12({x=1}, function(x) return {x=-x.x} end) +g12({x=1}, {x=2}, function(x, y) return {x=x.x + y.x} end) + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "infer_generic_lib_function_function_argument") +{ + CheckResult result = check(R"( +local a = {{x=4}, {x=7}, {x=1}} +table.sort(a, function(x, y) return x.x < y.x end) + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "do_not_infer_generic_functions") +{ + CheckResult result = check(R"( +local function sum(x: a, y: a, f: (a, a) -> a) return f(x, y) end + +local function sumrec(f: typeof(sum)) + return sum(2, 3, function(a, b) return a + b end) +end + +local b = sumrec(sum) -- ok +local c = sumrec(function(x, y, f) return f(x, y) end) -- type binders are not inferred + )"); + + LUAU_REQUIRE_ERRORS(result); + CHECK_EQ("Type '(a, b, (a, b) -> (c...)) -> (c...)' could not be converted into '(a, a, (a, a) -> a) -> a'; different number of generic type " + "parameters", + toString(result.errors[0])); +} + +TEST_CASE_FIXTURE(Fixture, "substitution_with_bound_table") +{ + CheckResult result = check(R"( +type A = { x: number } +local a: A = { x = 1 } +local b = a +type B = typeof(b) +type X = T +local c: X + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} TEST_SUITE_END(); diff --git a/tests/TypeInfer.intersectionTypes.test.cpp b/tests/TypeInfer.intersectionTypes.test.cpp index 26881b5c..d146f4e8 100644 --- a/tests/TypeInfer.intersectionTypes.test.cpp +++ b/tests/TypeInfer.intersectionTypes.test.cpp @@ -377,4 +377,32 @@ local b: number = a CHECK_EQ(toString(result.errors[0]), R"(Type 'X & Y & Z' could not be converted into 'number'; none of the intersection parts are compatible)"); } +TEST_CASE_FIXTURE(Fixture, "overload_is_not_a_function") +{ + check(R"( +--!nonstrict +function _(...):((typeof(not _))&(typeof(not _)))&((typeof(not _))&(typeof(not _))) +_(...)(setfenv,_,not _,"")[_] = nil +end +do end +_(...)(...,setfenv,_):_G() +)"); +} + +TEST_CASE_FIXTURE(Fixture, "no_stack_overflow_from_flattenintersection") +{ + CheckResult result = check(R"( + local l0,l0 + repeat + type t0 = ((any)|((any)&((any)|((any)&((any)|(any))))))&(t0) + function _(l0):(t0)&(t0) + while nil do + end + end + until _(_)(_)._ + )"); + + CHECK_LE(0, result.errors.size()); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.loops.test.cpp b/tests/TypeInfer.loops.test.cpp new file mode 100644 index 00000000..30df717b --- /dev/null +++ b/tests/TypeInfer.loops.test.cpp @@ -0,0 +1,473 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details + +#include "Luau/AstQuery.h" +#include "Luau/BuiltinDefinitions.h" +#include "Luau/Scope.h" +#include "Luau/TypeInfer.h" +#include "Luau/TypeVar.h" +#include "Luau/VisitTypeVar.h" + +#include "Fixture.h" + +#include "doctest.h" + +using namespace Luau; + +TEST_SUITE_BEGIN("TypeInferLoops"); + +TEST_CASE_FIXTURE(Fixture, "for_loop") +{ + CheckResult result = check(R"( + local q + for i=0, 50, 2 do + q = i + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ(*typeChecker.numberType, *requireType("q")); +} + +TEST_CASE_FIXTURE(Fixture, "for_in_loop") +{ + CheckResult result = check(R"( + local n + local s + for i, v in pairs({ "foo" }) do + n = i + s = v + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ(*typeChecker.numberType, *requireType("n")); + CHECK_EQ(*typeChecker.stringType, *requireType("s")); +} + +TEST_CASE_FIXTURE(Fixture, "for_in_loop_with_next") +{ + CheckResult result = check(R"( + local n + local s + for i, v in next, { "foo", "bar" } do + n = i + s = v + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ(*typeChecker.numberType, *requireType("n")); + CHECK_EQ(*typeChecker.stringType, *requireType("s")); +} + +TEST_CASE_FIXTURE(Fixture, "for_in_with_an_iterator_of_type_any") +{ + CheckResult result = check(R"( + local it: any + local a, b + for i, v in it do + a, b = i, v + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "for_in_loop_should_fail_with_non_function_iterator") +{ + CheckResult result = check(R"( + local foo = "bar" + for i, v in foo do + end + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); +} + +TEST_CASE_FIXTURE(Fixture, "for_in_with_just_one_iterator_is_ok") +{ + CheckResult result = check(R"( + local function keys(dictionary) + local new = {} + local index = 1 + + for key in pairs(dictionary) do + new[index] = key + index = index + 1 + end + + return new + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "for_in_with_a_custom_iterator_should_type_check") +{ + CheckResult result = check(R"( + local function range(l, h): () -> number + return function() + return l + end + end + + for n: string in range(1, 10) do + print(n) + end + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); +} + +TEST_CASE_FIXTURE(Fixture, "for_in_loop_on_error") +{ + CheckResult result = check(R"( + function f(x) + gobble.prop = x.otherprop + end + + local p + for _, part in i_am_not_defined do + p = part + f(part) + part.thirdprop = false + end + )"); + + CHECK_EQ(2, result.errors.size()); + + TypeId p = requireType("p"); + CHECK_EQ("*unknown*", toString(p)); +} + +TEST_CASE_FIXTURE(Fixture, "for_in_loop_on_non_function") +{ + CheckResult result = check(R"( + local bad_iter = 5 + + for a in bad_iter() do + end + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + REQUIRE(get(result.errors[0])); +} + +TEST_CASE_FIXTURE(Fixture, "for_in_loop_error_on_factory_not_returning_the_right_amount_of_values") +{ + CheckResult result = check(R"( + local function hasDivisors(value: number, table) + return false + end + + function prime_iter(state, index) + while hasDivisors(index, state) do + index += 1 + end + + state[index] = true + return index + end + + function primes1() + return prime_iter, {} + end + + function primes2() + return prime_iter, {}, "" + end + + function primes3() + return prime_iter, {}, 2 + end + + for p in primes1() do print(p) end -- mismatch in argument count + + for p in primes2() do print(p) end -- mismatch in argument types, prime_iter takes {}, number, we are given {}, string + + for p in primes3() do print(p) end -- no error + )"); + + LUAU_REQUIRE_ERROR_COUNT(2, result); + + CountMismatch* acm = get(result.errors[0]); + REQUIRE(acm); + CHECK_EQ(acm->context, CountMismatch::Arg); + CHECK_EQ(2, acm->expected); + CHECK_EQ(1, acm->actual); + + TypeMismatch* tm = get(result.errors[1]); + REQUIRE(tm); + CHECK_EQ(typeChecker.numberType, tm->wantedType); + CHECK_EQ(typeChecker.stringType, tm->givenType); +} + +TEST_CASE_FIXTURE(Fixture, "for_in_loop_error_on_iterator_requiring_args_but_none_given") +{ + CheckResult result = check(R"( + function prime_iter(state, index) + return 1 + end + + for p in prime_iter do print(p) end + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + CountMismatch* acm = get(result.errors[0]); + REQUIRE(acm); + CHECK_EQ(acm->context, CountMismatch::Arg); + CHECK_EQ(2, acm->expected); + CHECK_EQ(0, acm->actual); +} + +TEST_CASE_FIXTURE(Fixture, "for_in_loop_with_custom_iterator") +{ + CheckResult result = check(R"( + function primes() + return function (state: number) end, 2 + end + + for p, q in primes do + q = "" + end + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + TypeMismatch* tm = get(result.errors[0]); + REQUIRE(tm); + CHECK_EQ(typeChecker.numberType, tm->wantedType); + CHECK_EQ(typeChecker.stringType, tm->givenType); +} + +TEST_CASE_FIXTURE(Fixture, "while_loop") +{ + CheckResult result = check(R"( + local i + while true do + i = 8 + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ(*typeChecker.numberType, *requireType("i")); +} + +TEST_CASE_FIXTURE(Fixture, "repeat_loop") +{ + CheckResult result = check(R"( + local i + repeat + i = 'hi' + until true + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ(*typeChecker.stringType, *requireType("i")); +} + +TEST_CASE_FIXTURE(Fixture, "repeat_loop_condition_binds_to_its_block") +{ + CheckResult result = check(R"( + repeat + local x = true + until x + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "symbols_in_repeat_block_should_not_be_visible_beyond_until_condition") +{ + CheckResult result = check(R"( + repeat + local x = true + until x + + print(x) + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); +} + +TEST_CASE_FIXTURE(Fixture, "varlist_declared_by_for_in_loop_should_be_free") +{ + CheckResult result = check(R"( + local T = {} + + function T.f(p) + for i, v in pairs(p) do + T.f(v) + end + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "properly_infer_iteratee_is_a_free_table") +{ + // In this case, we cannot know the element type of the table {}. It could be anything. + // We therefore must initially ascribe a free typevar to iter. + CheckResult result = check(R"( + for iter in pairs({}) do + iter:g().p = true + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "correctly_scope_locals_while") +{ + CheckResult result = check(R"( + while true do + local a = 1 + end + + print(a) -- oops! + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + UnknownSymbol* us = get(result.errors[0]); + REQUIRE(us); + CHECK_EQ(us->name, "a"); +} + +TEST_CASE_FIXTURE(Fixture, "ipairs_produces_integral_indices") +{ + CheckResult result = check(R"( + local key + for i, e in ipairs({}) do key = i end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + REQUIRE_EQ("number", toString(requireType("key"))); +} + +TEST_CASE_FIXTURE(Fixture, "for_in_loop_where_iteratee_is_free") +{ + // This code doesn't pass typechecking. We just care that it doesn't crash. + (void)check(R"( + --!nonstrict + function _:_(...) + end + + repeat + if _ then + else + _ = ... + end + until _ + + for _ in _() do + end + )"); +} + +TEST_CASE_FIXTURE(Fixture, "unreachable_code_after_infinite_loop") +{ + { + CheckResult result = check(R"( + function unreachablecodepath(a): number + while true do + if a then return 10 end + end + -- unreachable + end + unreachablecodepath(4) + )"); + + LUAU_REQUIRE_ERROR_COUNT(0, result); + } + + { + CheckResult result = check(R"( + function reachablecodepath(a): number + while true do + if a then break end + return 10 + end + + print("x") -- correct error + end + reachablecodepath(4) + )"); + + LUAU_REQUIRE_ERRORS(result); + CHECK(get(result.errors[0])); + } + + { + CheckResult result = check(R"( + function unreachablecodepath(a): number + repeat + if a then return 10 end + until false + + -- unreachable + end + unreachablecodepath(4) + )"); + + LUAU_REQUIRE_ERROR_COUNT(0, result); + } + + { + CheckResult result = check(R"( + function reachablecodepath(a, b): number + repeat + if a then break end + + if b then return 10 end + until false + + print("x") -- correct error + end + reachablecodepath(4) + )"); + + LUAU_REQUIRE_ERRORS(result); + CHECK(get(result.errors[0])); + } + + { + CheckResult result = check(R"( + function unreachablecodepath(a: number?): number + repeat + return 10 + until a ~= nil + + -- unreachable + end + unreachablecodepath(4) + )"); + + LUAU_REQUIRE_ERROR_COUNT(0, result); + } +} + +TEST_CASE_FIXTURE(Fixture, "loop_typecheck_crash_on_empty_optional") +{ + CheckResult result = check(R"( + local t = {} + for _ in t do + for _ in assert(missing()) do + end + end + )"); + + LUAU_REQUIRE_ERROR_COUNT(2, result); +} + +TEST_SUITE_END(); diff --git a/tests/TypeInfer.modules.test.cpp b/tests/TypeInfer.modules.test.cpp new file mode 100644 index 00000000..63643610 --- /dev/null +++ b/tests/TypeInfer.modules.test.cpp @@ -0,0 +1,310 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details + +#include "Luau/AstQuery.h" +#include "Luau/BuiltinDefinitions.h" +#include "Luau/Scope.h" +#include "Luau/TypeInfer.h" +#include "Luau/TypeVar.h" +#include "Luau/VisitTypeVar.h" + +#include "Fixture.h" + +#include "doctest.h" + +using namespace Luau; + +TEST_SUITE_BEGIN("TypeInferModules"); + +TEST_CASE_FIXTURE(Fixture, "require") +{ + fileResolver.source["game/A"] = R"( + local function hooty(x: number): string + return "Hi there!" + end + + return {hooty=hooty} + )"; + + fileResolver.source["game/B"] = R"( + local Hooty = require(game.A) + + local h -- free! + local i = Hooty.hooty(h) + )"; + + CheckResult aResult = frontend.check("game/A"); + dumpErrors(aResult); + LUAU_REQUIRE_NO_ERRORS(aResult); + + CheckResult bResult = frontend.check("game/B"); + dumpErrors(bResult); + LUAU_REQUIRE_NO_ERRORS(bResult); + + ModulePtr b = frontend.moduleResolver.modules["game/B"]; + + REQUIRE(b != nullptr); + + dumpErrors(bResult); + + std::optional iType = requireType(b, "i"); + REQUIRE_EQ("string", toString(*iType)); + + std::optional hType = requireType(b, "h"); + REQUIRE_EQ("number", toString(*hType)); +} + +TEST_CASE_FIXTURE(Fixture, "require_types") +{ + fileResolver.source["workspace/A"] = R"( + export type Point = {x: number, y: number} + + return {} + )"; + + fileResolver.source["workspace/B"] = R"( + local Hooty = require(workspace.A) + + local h: Hooty.Point + )"; + + CheckResult bResult = frontend.check("workspace/B"); + dumpErrors(bResult); + + ModulePtr b = frontend.moduleResolver.modules["workspace/B"]; + REQUIRE(b != nullptr); + + TypeId hType = requireType(b, "h"); + REQUIRE_MESSAGE(bool(get(hType)), "Expected table but got " << toString(hType)); +} + +TEST_CASE_FIXTURE(Fixture, "require_a_variadic_function") +{ + fileResolver.source["game/A"] = R"( + local T = {} + function T.f(...) end + return T + )"; + + fileResolver.source["game/B"] = R"( + local A = require(game.A) + local f = A.f + )"; + + CheckResult result = frontend.check("game/B"); + + ModulePtr bModule = frontend.moduleResolver.getModule("game/B"); + REQUIRE(bModule != nullptr); + + TypeId f = follow(requireType(bModule, "f")); + + const FunctionTypeVar* ftv = get(f); + REQUIRE(ftv); + + auto iter = begin(ftv->argTypes); + auto endIter = end(ftv->argTypes); + + REQUIRE(iter == endIter); + REQUIRE(iter.tail()); + + CHECK(get(*iter.tail())); +} + +TEST_CASE_FIXTURE(Fixture, "type_error_of_unknown_qualified_type") +{ + CheckResult result = check(R"( + local p: SomeModule.DoesNotExist + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + REQUIRE_EQ(result.errors[0], (TypeError{Location{{1, 17}, {1, 40}}, UnknownSymbol{"SomeModule.DoesNotExist"}})); +} + +TEST_CASE_FIXTURE(Fixture, "require_module_that_does_not_export") +{ + const std::string sourceA = R"( + )"; + + const std::string sourceB = R"( + local Hooty = require(script.Parent.A) + )"; + + fileResolver.source["game/Workspace/A"] = sourceA; + fileResolver.source["game/Workspace/B"] = sourceB; + + frontend.check("game/Workspace/A"); + frontend.check("game/Workspace/B"); + + ModulePtr aModule = frontend.moduleResolver.modules["game/Workspace/A"]; + ModulePtr bModule = frontend.moduleResolver.modules["game/Workspace/B"]; + + CHECK(aModule->errors.empty()); + REQUIRE_EQ(1, bModule->errors.size()); + CHECK_MESSAGE(get(bModule->errors[0]), "Should be IllegalRequire: " << toString(bModule->errors[0])); + + auto hootyType = requireType(bModule, "Hooty"); + + CHECK_EQ("*unknown*", toString(hootyType)); +} + +TEST_CASE_FIXTURE(Fixture, "warn_if_you_try_to_require_a_non_modulescript") +{ + fileResolver.source["Modules/A"] = ""; + fileResolver.sourceTypes["Modules/A"] = SourceCode::Local; + + fileResolver.source["Modules/B"] = R"( + local M = require(script.Parent.A) + )"; + + CheckResult result = frontend.check("Modules/B"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + CHECK(get(result.errors[0])); +} + +TEST_CASE_FIXTURE(Fixture, "general_require_call_expression") +{ + fileResolver.source["game/A"] = R"( +--!strict +return { def = 4 } + )"; + + fileResolver.source["game/B"] = R"( +--!strict +local tbl = { abc = require(game.A) } +local a : string = "" +a = tbl.abc.def + )"; + + CheckResult result = frontend.check("game/B"); + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ("Type 'number' could not be converted into 'string'", toString(result.errors[0])); +} + +TEST_CASE_FIXTURE(Fixture, "general_require_type_mismatch") +{ + fileResolver.source["game/A"] = R"( +return { def = 4 } + )"; + + fileResolver.source["game/B"] = R"( +local tbl: string = require(game.A) + )"; + + CheckResult result = frontend.check("game/B"); + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ("Type '{| def: number |}' could not be converted into 'string'", toString(result.errors[0])); +} + +TEST_CASE_FIXTURE(Fixture, "bound_free_table_export_is_ok") +{ + CheckResult result = check(R"( +local n = {} +function n:Clone() end + +local m = {} + +function m.a(x) + x:Clone() +end + +function m.b() + m.a(n) +end + +return m +)"); + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "custom_require_global") +{ + CheckResult result = check(R"( +--!nonstrict +require = function(a) end + +local crash = require(game.A) + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "require_failed_module") +{ + fileResolver.source["game/A"] = R"( +return unfortunately() + )"; + + CheckResult aResult = frontend.check("game/A"); + LUAU_REQUIRE_ERRORS(aResult); + + CheckResult result = check(R"( +local ModuleA = require(game.A) + )"); + LUAU_REQUIRE_NO_ERRORS(result); + + std::optional oty = requireType("ModuleA"); + CHECK_EQ("*unknown*", toString(*oty)); +} + +TEST_CASE_FIXTURE(Fixture, "do_not_modify_imported_types") +{ + fileResolver.source["game/A"] = R"( +export type Type = { unrelated: boolean } +return {} + )"; + + fileResolver.source["game/B"] = R"( +local types = require(game.A) +type Type = types.Type +local x: Type = {} +function x:Destroy(): () end + )"; + + CheckResult result = frontend.check("game/B"); + LUAU_REQUIRE_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "do_not_modify_imported_types_2") +{ + ScopedFastFlag immutableTypes{"LuauImmutableTypes", true}; + + fileResolver.source["game/A"] = R"( +export type Type = { x: { a: number } } +return {} + )"; + + fileResolver.source["game/B"] = R"( +local types = require(game.A) +type Type = types.Type +local x: Type = { x = { a = 2 } } +type Rename = typeof(x.x) + )"; + + CheckResult result = frontend.check("game/B"); + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "do_not_modify_imported_types_3") +{ + ScopedFastFlag immutableTypes{"LuauImmutableTypes", true}; + + fileResolver.source["game/A"] = R"( +local y = setmetatable({}, {}) +export type Type = { x: typeof(y) } +return { x = y } + )"; + + fileResolver.source["game/B"] = R"( +local types = require(game.A) +type Type = types.Type +local x: Type = types +type Rename = typeof(x.x) + )"; + + CheckResult result = frontend.check("game/B"); + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_SUITE_END(); diff --git a/tests/TypeInfer.oop.test.cpp b/tests/TypeInfer.oop.test.cpp new file mode 100644 index 00000000..40831bf6 --- /dev/null +++ b/tests/TypeInfer.oop.test.cpp @@ -0,0 +1,275 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details + +#include "Luau/AstQuery.h" +#include "Luau/BuiltinDefinitions.h" +#include "Luau/Scope.h" +#include "Luau/TypeInfer.h" +#include "Luau/TypeVar.h" +#include "Luau/VisitTypeVar.h" + +#include "Fixture.h" + +#include "doctest.h" + +using namespace Luau; + +TEST_SUITE_BEGIN("TypeInferOOP"); + +TEST_CASE_FIXTURE(Fixture, "dont_suggest_using_colon_rather_than_dot_if_not_defined_with_colon") +{ + CheckResult result = check(R"( + local someTable = {} + + someTable.Function1 = function(Arg1) + end + + someTable.Function1() -- Argument count mismatch + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + REQUIRE(get(result.errors[0])); +} + +TEST_CASE_FIXTURE(Fixture, "dont_suggest_using_colon_rather_than_dot_if_it_wont_help_2") +{ + CheckResult result = check(R"( + local someTable = {} + + someTable.Function2 = function(Arg1, Arg2) + end + + someTable.Function2() -- Argument count mismatch + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + REQUIRE(get(result.errors[0])); +} + +TEST_CASE_FIXTURE(Fixture, "dont_suggest_using_colon_rather_than_dot_if_another_overload_works") +{ + CheckResult result = check(R"( + type T = {method: ((T, number) -> number) & ((number) -> number)} + local T: T + + T.method(4) + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "method_depends_on_table") +{ + CheckResult result = check(R"( + -- This catches a bug where x:m didn't count as a use of x + -- so toposort would happily reorder a definition of + -- function x:m before the definition of x. + function g() f() end + local x = {} + function x:m() end + function f() x:m() end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "methods_are_topologically_sorted") +{ + CheckResult result = check(R"( + local T = {} + + function T:foo() + return T:bar(999), T:bar("hi") + end + + function T:bar(i) + return i + end + + local a, b = T:foo() + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + dumpErrors(result); + + CHECK_EQ(PrimitiveTypeVar::Number, getPrimitiveType(requireType("a"))); + CHECK_EQ(PrimitiveTypeVar::String, getPrimitiveType(requireType("b"))); +} + +TEST_CASE_FIXTURE(Fixture, "quantify_methods_defined_using_dot_syntax_and_explicit_self_parameter") +{ + check(R"( + local T = {} + + function T.method(self) + self:method() + end + + function T.method2(self) + self:method() + end + + T:method2() + )"); +} + +TEST_CASE_FIXTURE(Fixture, "inferring_hundreds_of_self_calls_should_not_suffocate_memory") +{ + CheckResult result = check(R"( + ("foo") + :lower() + :lower() + :lower() + :lower() + :lower() + :lower() + :lower() + :lower() + :lower() + :lower() + :lower() + :lower() + :lower() + :lower() + :lower() + :lower() + :lower() + :lower() + )"); + + ModulePtr module = getMainModule(); + CHECK_GE(50, module->internalTypes.typeVars.size()); +} + +TEST_CASE_FIXTURE(Fixture, "object_constructor_can_refer_to_method_of_self") +{ + // CLI-30902 + CheckResult result = check(R"( + --!strict + + type Foo = { + fooConn: () -> () | nil + } + + local Foo = {} + Foo.__index = Foo + + function Foo.new() + local self: Foo = { + fooConn = nil, + } + setmetatable(self, Foo) + + self.fooConn = function() + self:method() -- Key 'method' not found in table self + end + + return self + end + + function Foo:method() + print("foo") + end + + local foo = Foo.new() + + -- TODO This is the best our current refinement support can offer :( + local bar = foo.fooConn + if bar then bar() end + + -- foo.fooConn() + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "CheckMethodsOfSealed") +{ + CheckResult result = check(R"( +local x: {prop: number} = {prop=9999} +function x:y(z: number) + local s: string = z +end +)"); + + LUAU_REQUIRE_ERROR_COUNT(2, result); +} + +TEST_CASE_FIXTURE(Fixture, "nonstrict_self_mismatch_tail") +{ + CheckResult result = check(R"( +--!nonstrict +local f = {} +function f:foo(a: number, b: number) end + +function bar(...) + f.foo(f, 1, ...) +end + +bar(2) +)"); + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "inferred_methods_of_free_tables_have_the_same_level_as_the_enclosing_table") +{ + check(R"( + function Base64FileReader(data) + local reader = {} + local index: number + + function reader:PeekByte() + return data:byte(index) + end + + function reader:Byte() + return data:byte(index - 1) + end + + return reader + end + + Base64FileReader() + + function ReadMidiEvents(data) + + local reader = Base64FileReader(data) + + while reader:HasMore() do + (reader:Byte() % 128) + end + end + )"); +} + +TEST_CASE_FIXTURE(Fixture, "table_oop") +{ + CheckResult result = check(R"( + --!strict +local Class = {} +Class.__index = Class + +type Class = typeof(setmetatable({} :: { x: number }, Class)) + +function Class.new(x: number): Class + return setmetatable({x = x}, Class) +end + +function Class.getx(self: Class) + return self.x +end + +function test() + local c = Class.new(42) + local n = c:getx() + local nn = c.x + + print(string.format("%d %d", n, nn)) +end +)"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_SUITE_END(); diff --git a/tests/TypeInfer.operators.test.cpp b/tests/TypeInfer.operators.test.cpp new file mode 100644 index 00000000..baa25978 --- /dev/null +++ b/tests/TypeInfer.operators.test.cpp @@ -0,0 +1,759 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details + +#include "Luau/AstQuery.h" +#include "Luau/BuiltinDefinitions.h" +#include "Luau/Scope.h" +#include "Luau/TypeInfer.h" +#include "Luau/TypeVar.h" +#include "Luau/VisitTypeVar.h" + +#include "Fixture.h" + +#include "doctest.h" + +using namespace Luau; + +TEST_SUITE_BEGIN("TypeInferOperators"); + +TEST_CASE_FIXTURE(Fixture, "or_joins_types") +{ + CheckResult result = check(R"( + local s = "a" or 10 + local x:string|number = s + )"); + LUAU_REQUIRE_NO_ERRORS(result); + CHECK_EQ(toString(*requireType("s")), "number | string"); + CHECK_EQ(toString(*requireType("x")), "number | string"); +} + +TEST_CASE_FIXTURE(Fixture, "or_joins_types_with_no_extras") +{ + CheckResult result = check(R"( + local s = "a" or 10 + local x:number|string = s + local y = x or "s" + )"); + CHECK_EQ(0, result.errors.size()); + CHECK_EQ(toString(*requireType("s")), "number | string"); + CHECK_EQ(toString(*requireType("y")), "number | string"); +} + +TEST_CASE_FIXTURE(Fixture, "or_joins_types_with_no_superfluous_union") +{ + CheckResult result = check(R"( + local s = "a" or "b" + local x:string = s + )"); + CHECK_EQ(0, result.errors.size()); + CHECK_EQ(*requireType("s"), *typeChecker.stringType); +} + +TEST_CASE_FIXTURE(Fixture, "and_adds_boolean") +{ + CheckResult result = check(R"( + local s = "a" and 10 + local x:boolean|number = s + )"); + CHECK_EQ(0, result.errors.size()); + CHECK_EQ(toString(*requireType("s")), "boolean | number"); +} + +TEST_CASE_FIXTURE(Fixture, "and_adds_boolean_no_superfluous_union") +{ + CheckResult result = check(R"( + local s = "a" and true + local x:boolean = s + )"); + CHECK_EQ(0, result.errors.size()); + CHECK_EQ(*requireType("x"), *typeChecker.booleanType); +} + +TEST_CASE_FIXTURE(Fixture, "and_or_ternary") +{ + CheckResult result = check(R"( + local s = (1/2) > 0.5 and "a" or 10 + )"); + CHECK_EQ(0, result.errors.size()); + CHECK_EQ(toString(*requireType("s")), "number | string"); +} + +TEST_CASE_FIXTURE(Fixture, "primitive_arith_no_metatable") +{ + CheckResult result = check(R"( + function add(a: number, b: string) + return a + (tonumber(b) :: number), a .. b + end + local n, s = add(2,"3") + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + const FunctionTypeVar* functionType = get(requireType("add")); + + std::optional retType = first(functionType->retType); + CHECK_EQ(std::optional(typeChecker.numberType), retType); + CHECK_EQ(requireType("n"), typeChecker.numberType); + CHECK_EQ(requireType("s"), typeChecker.stringType); +} + +TEST_CASE_FIXTURE(Fixture, "primitive_arith_no_metatable_with_follows") +{ + CheckResult result = check(R"( + local PI=3.1415926535897931 + local SOLAR_MASS=4*PI * PI + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + CHECK_EQ(requireType("SOLAR_MASS"), typeChecker.numberType); +} + +TEST_CASE_FIXTURE(Fixture, "primitive_arith_possible_metatable") +{ + CheckResult result = check(R"( + function add(a: number, b: any) + return a + b + end + local t = add(1,2) + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + CHECK_EQ("any", toString(requireType("t"))); +} + +TEST_CASE_FIXTURE(Fixture, "some_primitive_binary_ops") +{ + CheckResult result = check(R"( + local a = 4 + 8 + local b = a + 9 + local s = 'hotdogs' + local t = s .. s + local c = b - a + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ("number", toString(requireType("a"))); + CHECK_EQ("number", toString(requireType("b"))); + CHECK_EQ("string", toString(requireType("s"))); + CHECK_EQ("string", toString(requireType("t"))); + CHECK_EQ("number", toString(requireType("c"))); +} + +TEST_CASE_FIXTURE(Fixture, "typecheck_overloaded_multiply_that_is_an_intersection") +{ + ScopedFastFlag sff{"LuauErrorRecoveryType", true}; + + CheckResult result = check(R"( + --!strict + local Vec3 = {} + Vec3.__index = Vec3 + function Vec3.new() + return setmetatable({x=0, y=0, z=0}, Vec3) + end + + export type Vec3 = typeof(Vec3.new()) + + local thefun: any = function(self, o) return self end + + local multiply: ((Vec3, Vec3) -> Vec3) & ((Vec3, number) -> Vec3) = thefun + + Vec3.__mul = multiply + + local a = Vec3.new() + local b = Vec3.new() + local c = a * b + local d = a * 2 + local e = a * 'cabbage' + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + CHECK_EQ("Vec3", toString(requireType("a"))); + CHECK_EQ("Vec3", toString(requireType("b"))); + CHECK_EQ("Vec3", toString(requireType("c"))); + CHECK_EQ("Vec3", toString(requireType("d"))); + CHECK_EQ("Vec3", toString(requireType("e"))); +} + +TEST_CASE_FIXTURE(Fixture, "typecheck_overloaded_multiply_that_is_an_intersection_on_rhs") +{ + ScopedFastFlag sff{"LuauErrorRecoveryType", true}; + + CheckResult result = check(R"( + --!strict + local Vec3 = {} + Vec3.__index = Vec3 + function Vec3.new() + return setmetatable({x=0, y=0, z=0}, Vec3) + end + + export type Vec3 = typeof(Vec3.new()) + + local thefun: any = function(self, o) return self end + + local multiply: ((Vec3, Vec3) -> Vec3) & ((Vec3, number) -> Vec3) = thefun + + Vec3.__mul = multiply + + local a = Vec3.new() + local b = Vec3.new() + local c = b * a + local d = 2 * a + local e = 'cabbage' * a + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + CHECK_EQ("Vec3", toString(requireType("a"))); + CHECK_EQ("Vec3", toString(requireType("b"))); + CHECK_EQ("Vec3", toString(requireType("c"))); + CHECK_EQ("Vec3", toString(requireType("d"))); + CHECK_EQ("Vec3", toString(requireType("e"))); +} + +TEST_CASE_FIXTURE(Fixture, "compare_numbers") +{ + CheckResult result = check(R"( + local a = 441 + local b = 0 + local c = a < b + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "compare_strings") +{ + CheckResult result = check(R"( + local a = '441' + local b = '0' + local c = a < b + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "cannot_indirectly_compare_types_that_do_not_have_a_metatable") +{ + CheckResult result = check(R"( + local a = {} + local b = {} + local c = a < b + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + GenericError* gen = get(result.errors[0]); + + REQUIRE_EQ(gen->message, "Type a cannot be compared with < because it has no metatable"); +} + +TEST_CASE_FIXTURE(Fixture, "cannot_indirectly_compare_types_that_do_not_offer_overloaded_ordering_operators") +{ + CheckResult result = check(R"( + local M = {} + function M.new() + return setmetatable({}, M) + end + type M = typeof(M.new()) + + local a = M.new() + local b = M.new() + local c = a < b + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + GenericError* gen = get(result.errors[0]); + REQUIRE(gen != nullptr); + REQUIRE_EQ(gen->message, "Table M does not offer metamethod __lt"); +} + +TEST_CASE_FIXTURE(Fixture, "cannot_compare_tables_that_do_not_have_the_same_metatable") +{ + CheckResult result = check(R"( + --!strict + local M = {} + function M.new() + return setmetatable({}, M) + end + function M.__lt(left, right) return true end + + local a = M.new() + local b = {} + local c = a < b -- line 10 + local d = b < a -- line 11 + )"); + + LUAU_REQUIRE_ERROR_COUNT(2, result); + + REQUIRE_EQ((Location{{10, 18}, {10, 23}}), result.errors[0].location); + + REQUIRE_EQ((Location{{11, 18}, {11, 23}}), result.errors[1].location); +} + +TEST_CASE_FIXTURE(Fixture, "produce_the_correct_error_message_when_comparing_a_table_with_a_metatable_with_one_that_does_not") +{ + CheckResult result = check(R"( + --!strict + local M = {} + function M.new() + return setmetatable({}, M) + end + function M.__lt(left, right) return true end + type M = typeof(M.new()) + + local a = M.new() + local b = {} + local c = a < b -- line 10 + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + auto err = get(result.errors[0]); + REQUIRE(err != nullptr); + + // Frail. :| + REQUIRE_EQ("Types M and b cannot be compared with < because they do not have the same metatable", err->message); +} + +TEST_CASE_FIXTURE(Fixture, "in_nonstrict_mode_strip_nil_from_intersections_when_considering_relational_operators") +{ + CheckResult result = check(R"( + --!nonstrict + + function maybe_a_number(): number? + return 50 + end + + local a = maybe_a_number() < maybe_a_number() + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "compound_assign_basic") +{ + CheckResult result = check(R"( + local s = 10 + s += 20 + )"); + CHECK_EQ(0, result.errors.size()); + CHECK_EQ(toString(*requireType("s")), "number"); +} + +TEST_CASE_FIXTURE(Fixture, "compound_assign_mismatch_op") +{ + CheckResult result = check(R"( + local s = 10 + s += true + )"); + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ(result.errors[0], (TypeError{Location{{2, 13}, {2, 17}}, TypeMismatch{typeChecker.numberType, typeChecker.booleanType}})); +} + +TEST_CASE_FIXTURE(Fixture, "compound_assign_mismatch_result") +{ + CheckResult result = check(R"( + local s = 'hello' + s += 10 + )"); + + LUAU_REQUIRE_ERROR_COUNT(2, result); + CHECK_EQ(result.errors[0], (TypeError{Location{{2, 8}, {2, 9}}, TypeMismatch{typeChecker.numberType, typeChecker.stringType}})); + CHECK_EQ(result.errors[1], (TypeError{Location{{2, 8}, {2, 15}}, TypeMismatch{typeChecker.stringType, typeChecker.numberType}})); +} + +TEST_CASE_FIXTURE(Fixture, "compound_assign_metatable") +{ + CheckResult result = check(R"( + --!strict + type V2B = { x: number, y: number } + local v2b: V2B = { x = 0, y = 0 } + local VMT = {} + type V2 = typeof(setmetatable(v2b, VMT)) + + function VMT.__add(a: V2, b: V2): V2 + return setmetatable({ x = a.x + b.x, y = a.y + b.y }, VMT) + end + + local v1: V2 = setmetatable({ x = 1, y = 2 }, VMT) + local v2: V2 = setmetatable({ x = 3, y = 4 }, VMT) + v1 += v2 + )"); + CHECK_EQ(0, result.errors.size()); +} + +TEST_CASE_FIXTURE(Fixture, "compound_assign_mismatch_metatable") +{ + CheckResult result = check(R"( + --!strict + type V2B = { x: number, y: number } + local v2b: V2B = { x = 0, y = 0 } + local VMT = {} + type V2 = typeof(setmetatable(v2b, VMT)) + + function VMT.__mod(a: V2, b: V2): number + return a.x * b.x + a.y * b.y + end + + local v1: V2 = setmetatable({ x = 1, y = 2 }, VMT) + local v2: V2 = setmetatable({ x = 3, y = 4 }, VMT) + v1 %= v2 + )"); + LUAU_REQUIRE_ERROR_COUNT(1, result); + + TypeMismatch* tm = get(result.errors[0]); + CHECK_EQ(*tm->wantedType, *requireType("v2")); + CHECK_EQ(*tm->givenType, *typeChecker.numberType); +} + +TEST_CASE_FIXTURE(Fixture, "CallOrOfFunctions") +{ + CheckResult result = check(R"( +function f() return 1; end +function g() return 2; end +(f or g)() +)"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "CallAndOrOfFunctions") +{ + CheckResult result = check(R"( +function f() return 1; end +function g() return 2; end +local x = false +(x and f or g)() +)"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "typecheck_unary_minus") +{ + CheckResult result = check(R"( + --!strict + local foo = { + value = 10 + } + local mt = {} + setmetatable(foo, mt) + + mt.__unm = function(val: typeof(foo)): string + return val.value .. "test" + end + + local a = -foo + + local b = 1+-1 + + local bar = { + value = 10 + } + local c = -bar -- disallowed + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + CHECK_EQ("string", toString(requireType("a"))); + CHECK_EQ("number", toString(requireType("b"))); + + GenericError* gen = get(result.errors[0]); + REQUIRE_EQ(gen->message, "Unary operator '-' not supported by type 'bar'"); +} + +TEST_CASE_FIXTURE(Fixture, "unary_not_is_boolean") +{ + CheckResult result = check(R"( + local b = not "string" + local c = not (math.random() > 0.5 and "string" or 7) + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + REQUIRE_EQ("boolean", toString(requireType("b"))); + REQUIRE_EQ("boolean", toString(requireType("c"))); +} + +TEST_CASE_FIXTURE(Fixture, "disallow_string_and_types_without_metatables_from_arithmetic_binary_ops") +{ + CheckResult result = check(R"( + --!strict + local a = "1.24" + 123 -- not allowed + + local foo = { + value = 10 + } + + local b = foo + 1 -- not allowed + + local bar = { + value = 1 + } + + local mt = {} + + setmetatable(bar, mt) + + mt.__add = function(a: typeof(bar), b: number): number + return a.value + b + end + + local c = bar + 1 -- allowed + + local d = bar + foo -- not allowed + )"); + + LUAU_REQUIRE_ERROR_COUNT(3, result); + + TypeMismatch* tm = get(result.errors[0]); + REQUIRE_EQ(*tm->wantedType, *typeChecker.numberType); + REQUIRE_EQ(*tm->givenType, *typeChecker.stringType); + + TypeMismatch* tm2 = get(result.errors[2]); + CHECK_EQ(*tm2->wantedType, *typeChecker.numberType); + CHECK_EQ(*tm2->givenType, *requireType("foo")); + + GenericError* gen2 = get(result.errors[1]); + REQUIRE_EQ(gen2->message, "Binary operator '+' not supported by types 'foo' and 'number'"); +} + +// CLI-29033 +TEST_CASE_FIXTURE(Fixture, "unknown_type_in_comparison") +{ + CheckResult result = check(R"( + function merge(lower, greater) + if lower.y == greater.y then + end + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "concat_op_on_free_lhs_and_string_rhs") +{ + CheckResult result = check(R"( + local function f(x) + return x .. "y" + end + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + REQUIRE(get(result.errors[0])); +} + +TEST_CASE_FIXTURE(Fixture, "concat_op_on_string_lhs_and_free_rhs") +{ + CheckResult result = check(R"( + local function f(x) + return "foo" .. x + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ("(string) -> string", toString(requireType("f"))); +} + +TEST_CASE_FIXTURE(Fixture, "strict_binary_op_where_lhs_unknown") +{ + std::vector ops = {"+", "-", "*", "/", "%", "^", ".."}; + + std::string src = R"( + function foo(a, b) + )"; + + for (const auto& op : ops) + src += "local _ = a " + op + "b\n"; + + src += "end"; + + CheckResult result = check(src); + LUAU_REQUIRE_ERROR_COUNT(ops.size(), result); + + CHECK_EQ("Unknown type used in + operation; consider adding a type annotation to 'a'", toString(result.errors[0])); +} + +TEST_CASE_FIXTURE(Fixture, "and_binexps_dont_unify") +{ + CheckResult result = check(R"( + --!strict + local t = {} + while true and t[1] do + print(t[1].test) + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "error_on_invalid_operand_types_to_relational_operators") +{ + CheckResult result = check(R"( + local a: boolean = true + local b: boolean = false + local foo = a < b + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + GenericError* ge = get(result.errors[0]); + REQUIRE(ge); + CHECK_EQ("Type 'boolean' cannot be compared with relational operator <", ge->message); +} + +TEST_CASE_FIXTURE(Fixture, "error_on_invalid_operand_types_to_relational_operators2") +{ + CheckResult result = check(R"( + local a: number | string = "" + local b: number | string = 1 + local foo = a < b + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + GenericError* ge = get(result.errors[0]); + REQUIRE(ge); + CHECK_EQ("Type 'number | string' cannot be compared with relational operator <", ge->message); +} + +TEST_CASE_FIXTURE(Fixture, "cli_38355_recursive_union") +{ + CheckResult result = check(R"( + --!strict + local _ + _ += _ and _ or _ and _ or _ and _ + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ("Type contains a self-recursive construct that cannot be resolved", toString(result.errors[0])); +} + +TEST_CASE_FIXTURE(Fixture, "UnknownGlobalCompoundAssign") +{ + // In non-strict mode, global definition is still allowed + { + CheckResult result = check(R"( + --!nonstrict + a = a + 1 + print(a) + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ(toString(result.errors[0]), "Unknown global 'a'"); + } + + // In strict mode we no longer generate two errors from lhs + { + CheckResult result = check(R"( + --!strict + a += 1 + print(a) + )"); + + LUAU_REQUIRE_ERRORS(result); + CHECK_EQ(toString(result.errors[0]), "Unknown global 'a'"); + } + + // In non-strict mode, compound assignment is not a definition, it's a modification + { + CheckResult result = check(R"( + --!nonstrict + a += 1 + print(a) + )"); + + LUAU_REQUIRE_ERROR_COUNT(2, result); + CHECK_EQ(toString(result.errors[0]), "Unknown global 'a'"); + } +} + +TEST_CASE_FIXTURE(Fixture, "strip_nil_from_lhs_or_operator") +{ + CheckResult result = check(R"( +--!strict +local a: number? = nil +local b: number = a or 1 + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "strip_nil_from_lhs_or_operator2") +{ + CheckResult result = check(R"( +--!nonstrict +local a: number? = nil +local b: number = a or 1 + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "dont_strip_nil_from_rhs_or_operator") +{ + CheckResult result = check(R"( +--!strict +local a: number? = nil +local b: number = 1 or a + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + TypeMismatch* tm = get(result.errors[0]); + REQUIRE(tm); + CHECK_EQ(typeChecker.numberType, tm->wantedType); + CHECK_EQ("number?", toString(tm->givenType)); +} + +TEST_CASE_FIXTURE(Fixture, "operator_eq_verifies_types_do_intersect") +{ + CheckResult result = check(R"( + type Array = { [number]: T } + type Fiber = { id: number } + type null = {} + + local fiberStack: Array = {} + local index = 0 + + local function f(fiber: Fiber) + local a = fiber ~= fiberStack[index] + local b = fiberStack[index] ~= fiber + end + + return f + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "operator_eq_operands_are_not_subtypes_of_each_other_but_has_overlap") +{ + ScopedFastFlag sff1{"LuauEqConstraint", true}; + + CheckResult result = check(R"( + local function f(a: string | number, b: boolean | number) + return a == b + end + )"); + + // This doesn't produce any errors but for the wrong reasons. + // This unit test serves as a reminder to not try and unify the operands on `==`/`~=`. + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "refine_and_or") +{ + CheckResult result = check(R"( + local t: {x: number?}? = {x = nil} + local u = t and t.x or 5 + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ("number", toString(requireType("u"))); +} + +TEST_SUITE_END(); diff --git a/tests/TypeInfer.primitives.test.cpp b/tests/TypeInfer.primitives.test.cpp new file mode 100644 index 00000000..44b7b0d0 --- /dev/null +++ b/tests/TypeInfer.primitives.test.cpp @@ -0,0 +1,100 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details + +#include "Luau/AstQuery.h" +#include "Luau/BuiltinDefinitions.h" +#include "Luau/Scope.h" +#include "Luau/TypeInfer.h" +#include "Luau/TypeVar.h" +#include "Luau/VisitTypeVar.h" + +#include "Fixture.h" + +#include "doctest.h" + +using namespace Luau; + +TEST_SUITE_BEGIN("TypeInferPrimitives"); + +TEST_CASE_FIXTURE(Fixture, "cannot_call_primitives") +{ + CheckResult result = check("local foo = 5 foo()"); + LUAU_REQUIRE_ERROR_COUNT(1, result); + + REQUIRE(get(result.errors[0]) != nullptr); +} + +TEST_CASE_FIXTURE(Fixture, "string_length") +{ + CheckResult result = check(R"( + local s = "Hello, World!" + local t = #s + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + CHECK_EQ("number", toString(requireType("t"))); +} + +TEST_CASE_FIXTURE(Fixture, "string_index") +{ + CheckResult result = check(R"( + local s = "Hello, World!" + local t = s[4] + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + NotATable* nat = get(result.errors[0]); + REQUIRE(nat); + CHECK_EQ("string", toString(nat->ty)); + + CHECK_EQ("*unknown*", toString(requireType("t"))); +} + +TEST_CASE_FIXTURE(Fixture, "string_method") +{ + CheckResult result = check(R"( + local p = ("tacos"):len() + )"); + CHECK_EQ(0, result.errors.size()); + + CHECK_EQ(*requireType("p"), *typeChecker.numberType); +} + +TEST_CASE_FIXTURE(Fixture, "string_function_indirect") +{ + CheckResult result = check(R"( + local s:string + local l = s.lower + local p = l(s) + )"); + CHECK_EQ(0, result.errors.size()); + + CHECK_EQ(*requireType("p"), *typeChecker.stringType); +} + +TEST_CASE_FIXTURE(Fixture, "string_function_other") +{ + CheckResult result = check(R"( + local s:string + local p = s:match("foo") + )"); + CHECK_EQ(0, result.errors.size()); + + CHECK_EQ(toString(requireType("p")), "string?"); +} + +TEST_CASE_FIXTURE(Fixture, "CheckMethodsOfNumber") +{ + ScopedFastFlag sff{"LuauErrorRecoveryType", true}; + + CheckResult result = check(R"( +local x: number = 9999 +function x:y(z: number) + local s: string = z +end +)"); + + LUAU_REQUIRE_ERROR_COUNT(2, result); +} + +TEST_SUITE_END(); diff --git a/tests/TypeInfer.refinements.test.cpp b/tests/TypeInfer.refinements.test.cpp index a5147d56..9b347921 100644 --- a/tests/TypeInfer.refinements.test.cpp +++ b/tests/TypeInfer.refinements.test.cpp @@ -1298,4 +1298,22 @@ TEST_CASE_FIXTURE(RefinementClassFixture, "x_is_not_instance_or_else_not_part") CHECK_EQ("Part", toString(requireTypeAtPosition({5, 28}))); } +TEST_CASE_FIXTURE(Fixture, "typeguard_doesnt_leak_to_elseif") +{ + const std::string code = R"( + function f(a) + if type(a) == "boolean" then + local a1 = a + elseif a.fn() then + local a2 = a + else + local a3 = a + end + end + )"; + CheckResult result = check(code); + LUAU_REQUIRE_NO_ERRORS(result); + dumpErrors(result); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.singletons.test.cpp b/tests/TypeInfer.singletons.test.cpp index 3ed536ea..7f8d8fec 100644 --- a/tests/TypeInfer.singletons.test.cpp +++ b/tests/TypeInfer.singletons.test.cpp @@ -5,6 +5,8 @@ #include "doctest.h" #include "Luau/BuiltinDefinitions.h" +LUAU_FASTFLAG(BetterDiagnosticCodesInStudio) + using namespace Luau; TEST_SUITE_BEGIN("TypeSingletons"); @@ -353,7 +355,14 @@ TEST_CASE_FIXTURE(Fixture, "table_properties_alias_or_parens_is_indexer") )"); LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK_EQ("Syntax error: Cannot have more than one table indexer", toString(result.errors[0])); + if (FFlag::BetterDiagnosticCodesInStudio) + { + CHECK_EQ("Cannot have more than one table indexer", toString(result.errors[0])); + } + else + { + CHECK_EQ("Syntax error: Cannot have more than one table indexer", toString(result.errors[0])); + } } TEST_CASE_FIXTURE(Fixture, "table_properties_type_error_escapes") @@ -445,7 +454,7 @@ TEST_CASE_FIXTURE(Fixture, "widen_the_supertype_if_it_is_free_and_subtype_has_si {"LuauSingletonTypes", true}, {"LuauEqConstraint", true}, {"LuauDiscriminableUnions2", true}, - {"LuauWidenIfSupertypeIsFree", true}, + {"LuauWidenIfSupertypeIsFree2", true}, {"LuauWeakEqConstraint", false}, }; @@ -472,9 +481,9 @@ TEST_CASE_FIXTURE(Fixture, "return_type_of_f_is_not_widened") {"LuauSingletonTypes", true}, {"LuauDiscriminableUnions2", true}, {"LuauEqConstraint", true}, - {"LuauWidenIfSupertypeIsFree", true}, + {"LuauWidenIfSupertypeIsFree2", true}, {"LuauWeakEqConstraint", false}, - {"LuauDoNotAccidentallyDependOnPointerOrdering", true} + {"LuauDoNotAccidentallyDependOnPointerOrdering", true}, }; CheckResult result = check(R"( @@ -497,7 +506,7 @@ TEST_CASE_FIXTURE(Fixture, "widening_happens_almost_everywhere") ScopedFastFlag sff[]{ {"LuauParseSingletonTypes", true}, {"LuauSingletonTypes", true}, - {"LuauWidenIfSupertypeIsFree", true}, + {"LuauWidenIfSupertypeIsFree2", true}, }; CheckResult result = check(R"( @@ -515,7 +524,7 @@ TEST_CASE_FIXTURE(Fixture, "widening_happens_almost_everywhere_except_for_tables {"LuauParseSingletonTypes", true}, {"LuauSingletonTypes", true}, {"LuauDiscriminableUnions2", true}, - {"LuauWidenIfSupertypeIsFree", true}, + {"LuauWidenIfSupertypeIsFree2", true}, }; CheckResult result = check(R"( @@ -544,7 +553,7 @@ TEST_CASE_FIXTURE(Fixture, "table_insert_with_a_singleton_argument") ScopedFastFlag sff[]{ {"LuauParseSingletonTypes", true}, {"LuauSingletonTypes", true}, - {"LuauWidenIfSupertypeIsFree", true}, + {"LuauWidenIfSupertypeIsFree2", true}, }; CheckResult result = check(R"( @@ -565,4 +574,97 @@ TEST_CASE_FIXTURE(Fixture, "table_insert_with_a_singleton_argument") CHECK_EQ("{string}", toString(requireType("t"))); } +TEST_CASE_FIXTURE(Fixture, "functions_are_not_to_be_widened") +{ + ScopedFastFlag sff[]{ + {"LuauParseSingletonTypes", true}, + {"LuauSingletonTypes", true}, + {"LuauWidenIfSupertypeIsFree2", true}, + }; + + CheckResult result = check(R"( + local function foo(my_enum: "A" | "B") end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ(R"(("A" | "B") -> ())", toString(requireType("foo"))); +} + +TEST_CASE_FIXTURE(Fixture, "indexing_on_string_singletons") +{ + ScopedFastFlag sff[]{ + {"LuauDiscriminableUnions2", true}, + {"LuauSingletonTypes", true}, + }; + + CheckResult result = check(R"( + local a: string = "hi" + if a == "hi" then + local x = a:byte() + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ(R"("hi")", toString(requireTypeAtPosition({3, 22}))); +} + +TEST_CASE_FIXTURE(Fixture, "indexing_on_union_of_string_singletons") +{ + ScopedFastFlag sff[]{ + {"LuauDiscriminableUnions2", true}, + {"LuauSingletonTypes", true}, + }; + + CheckResult result = check(R"( + local a: string = "hi" + if a == "hi" or a == "bye" then + local x = a:byte() + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ(R"("bye" | "hi")", toString(requireTypeAtPosition({3, 22}))); +} + +TEST_CASE_FIXTURE(Fixture, "taking_the_length_of_string_singleton") +{ + ScopedFastFlag sff[]{ + {"LuauDiscriminableUnions2", true}, + {"LuauSingletonTypes", true}, + }; + + CheckResult result = check(R"( + local a: string = "hi" + if a == "hi" then + local x = #a + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ(R"("hi")", toString(requireTypeAtPosition({3, 23}))); +} + +TEST_CASE_FIXTURE(Fixture, "taking_the_length_of_union_of_string_singleton") +{ + ScopedFastFlag sff[]{ + {"LuauDiscriminableUnions2", true}, + {"LuauSingletonTypes", true}, + }; + + CheckResult result = check(R"( + local a: string = "hi" + if a == "hi" or a == "bye" then + local x = #a + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ(R"("bye" | "hi")", toString(requireTypeAtPosition({3, 23}))); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.tables.test.cpp b/tests/TypeInfer.tables.test.cpp index a5eba5df..91140aaa 100644 --- a/tests/TypeInfer.tables.test.cpp +++ b/tests/TypeInfer.tables.test.cpp @@ -2384,4 +2384,504 @@ _ = (_.cos) LUAU_REQUIRE_ERRORS(result); } +TEST_CASE_FIXTURE(Fixture, "cannot_call_tables") +{ + CheckResult result = check("local foo = {} foo()"); + LUAU_REQUIRE_ERROR_COUNT(1, result); + + CHECK(get(result.errors[0]) != nullptr); +} + +TEST_CASE_FIXTURE(Fixture, "table_length") +{ + CheckResult result = check(R"( + local t = {} + local s = #t + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK(nullptr != get(requireType("t"))); + CHECK_EQ(*typeChecker.numberType, *requireType("s")); +} + +TEST_CASE_FIXTURE(Fixture, "nil_assign_doesnt_hit_indexer") +{ + CheckResult result = check("local a = {} a[0] = 7 a[0] = nil"); + LUAU_REQUIRE_ERROR_COUNT(0, result); +} + +TEST_CASE_FIXTURE(Fixture, "wrong_assign_does_hit_indexer") +{ + CheckResult result = check("local a = {} a[0] = 7 a[0] = 't'"); + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ(result.errors[0], (TypeError{Location{Position{0, 30}, Position{0, 33}}, TypeMismatch{ + typeChecker.numberType, + typeChecker.stringType, + }})); +} + +TEST_CASE_FIXTURE(Fixture, "nil_assign_doesnt_hit_no_indexer") +{ + CheckResult result = check("local a = {a=1, b=2} a['a'] = nil"); + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ(result.errors[0], (TypeError{Location{Position{0, 30}, Position{0, 33}}, TypeMismatch{ + typeChecker.numberType, + typeChecker.nilType, + }})); +} + +TEST_CASE_FIXTURE(Fixture, "free_rhs_table_can_also_be_bound") +{ + check(R"( + local o + local v = o:i() + + function g(u) + v = u + end + + o:f(g) + o:h() + o:h() + )"); +} + +TEST_CASE_FIXTURE(Fixture, "table_unifies_into_map") +{ + CheckResult result = check(R"( + local Instance: any + local UDim2: any + + function Create(instanceType) + return function(data) + local obj = Instance.new(instanceType) + for k, v in pairs(data) do + if type(k) == 'number' then + --v.Parent = obj + else + obj[k] = v + end + end + return obj + end + end + + local topbarShadow = Create'ImageLabel'{ + Name = "TopBarShadow"; + Size = UDim2.new(1, 0, 0, 3); + Position = UDim2.new(0, 0, 1, 0); + Image = "rbxasset://textures/ui/TopBar/dropshadow.png"; + BackgroundTransparency = 1; + Active = false; + Visible = false; + }; + + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "tables_get_names_from_their_locals") +{ + CheckResult result = check(R"( + local T = {} + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ("T", toString(requireType("T"))); +} + +TEST_CASE_FIXTURE(Fixture, "generalize_table_argument") +{ + CheckResult result = check(R"( + function foo(arr) + local work = {} + for i = 1, #arr do + work[i] = arr[i] + end + + return arr + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + dumpErrors(result); + + const FunctionTypeVar* fooType = get(requireType("foo")); + REQUIRE(fooType); + + std::optional fooArg1 = first(fooType->argTypes); + REQUIRE(fooArg1); + + const TableTypeVar* fooArg1Table = get(*fooArg1); + REQUIRE(fooArg1Table); + + CHECK_EQ(fooArg1Table->state, TableState::Generic); +} + +/* + * This test case exposed an oversight in the treatment of free tables. + * Free tables, like free TypeVars, need to record the scope depth where they were created so that + * we do not erroneously let-generalize them when they are used in a nested lambda. + * + * For more information about let-generalization, see + * + * The important idea here is that the return type of Counter.new is a table with some metatable. + * That metatable *must* be the same TypeVar as the type of Counter. If it is a copy (produced by + * the generalization process), then it loses the knowledge that its metatable will have an :incr() + * method. + */ +TEST_CASE_FIXTURE(Fixture, "dont_quantify_table_that_belongs_to_outer_scope") +{ + CheckResult result = check(R"( + local Counter = {} + Counter.__index = Counter + + function Counter.new() + local self = setmetatable({count=0}, Counter) + return self + end + + function Counter:incr() + self.count = 1 + return self.count + end + + local self = Counter.new() + print(self:incr()) + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + TableTypeVar* counterType = getMutable(requireType("Counter")); + REQUIRE(counterType); + + const FunctionTypeVar* newType = get(follow(counterType->props["new"].type)); + REQUIRE(newType); + + std::optional newRetType = *first(newType->retType); + REQUIRE(newRetType); + + const MetatableTypeVar* newRet = get(follow(*newRetType)); + REQUIRE(newRet); + + const TableTypeVar* newRetMeta = get(newRet->metatable); + REQUIRE(newRetMeta); + + CHECK(newRetMeta->props.count("incr")); + CHECK_EQ(follow(newRet->metatable), follow(requireType("Counter"))); +} + +// TODO: CLI-39624 +TEST_CASE_FIXTURE(Fixture, "instantiate_tables_at_scope_level") +{ + CheckResult result = check(R"( + --!strict + local Option = {} + Option.__index = Option + function Option.Is(obj) + return (type(obj) == "table" and getmetatable(obj) == Option) + end + return Option + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "inferring_crazy_table_should_also_be_quick") +{ + CheckResult result = check(R"( + --!strict + function f(U) + U(w:s(an):c()():c():U(s):c():c():U(s):c():U(s):cU()):c():U(s):c():U(s):c():c():U(s):c():U(s):cU() + end + )"); + + ModulePtr module = getMainModule(); + CHECK_GE(100, module->internalTypes.typeVars.size()); +} + +TEST_CASE_FIXTURE(Fixture, "MixedPropertiesAndIndexers") +{ + CheckResult result = check(R"( +local x = {} +x.a = "a" +x[0] = true +x.b = 37 +)"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "setmetatable_cant_be_used_to_mutate_global_types") +{ + { + Fixture fix; + + // inherit env from parent fixture checker + fix.typeChecker.globalScope = typeChecker.globalScope; + + fix.check(R"( +--!nonstrict +type MT = typeof(setmetatable) +function wtf(arg: {MT}): typeof(table) + arg = wtf(arg) +end +)"); + } + + // validate sharedEnv post-typecheck; valuable for debugging some typeck crashes but slows fuzzing down + // note: it's important for typeck to be destroyed at this point! + { + for (auto& p : typeChecker.globalScope->bindings) + { + toString(p.second.typeId); // toString walks the entire type, making sure ASAN catches access to destroyed type arenas + } + } +} + +TEST_CASE_FIXTURE(Fixture, "evil_table_unification") +{ + // this code re-infers the type of _ while processing fields of _, which can cause use-after-free + check(R"( +--!nonstrict +_ = ... +_:table(_,string)[_:gsub(_,...,n0)],_,_:gsub(_,string)[""],_:split(_,...,table)._,n0 = nil +do end +)"); +} + +TEST_CASE_FIXTURE(Fixture, "dont_crash_when_setmetatable_does_not_produce_a_metatabletypevar") +{ + CheckResult result = check("local x = setmetatable({})"); + LUAU_REQUIRE_ERROR_COUNT(1, result); +} + +TEST_CASE_FIXTURE(Fixture, "instantiate_table_cloning") +{ + CheckResult result = check(R"( +--!nonstrict +local l0:any,l61:t0 = _,math +while _ do +_() +end +function _():t0 +end +type t0 = any +)"); + + std::optional ty = requireType("math"); + REQUIRE(ty); + + const TableTypeVar* ttv = get(*ty); + REQUIRE(ttv); + CHECK(ttv->instantiatedTypeParams.empty()); +} + +TEST_CASE_FIXTURE(Fixture, "instantiate_table_cloning_2") +{ + ScopedFastFlag sff{"LuauOnlyMutateInstantiatedTables", true}; + + CheckResult result = check(R"( +type X = T +type K = X +)"); + + LUAU_REQUIRE_NO_ERRORS(result); + + std::optional ty = requireType("math"); + REQUIRE(ty); + + const TableTypeVar* ttv = get(*ty); + REQUIRE(ttv); + CHECK(ttv->instantiatedTypeParams.empty()); +} + +TEST_CASE_FIXTURE(Fixture, "instantiate_table_cloning_3") +{ + ScopedFastFlag sff{"LuauOnlyMutateInstantiatedTables", true}; + + CheckResult result = check(R"( +type X = T +local a = {} +a.x = 4 +local b: X +a.y = 5 +local c: X +c = b +)"); + + LUAU_REQUIRE_NO_ERRORS(result); + + std::optional ty = requireType("a"); + REQUIRE(ty); + + const TableTypeVar* ttv = get(*ty); + REQUIRE(ttv); + CHECK(ttv->instantiatedTypeParams.empty()); +} + +TEST_CASE_FIXTURE(Fixture, "table_indexing_error_location") +{ + CheckResult result = check(R"( +local foo = {42} +local bar: number? +local baz = foo[bar] + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + CHECK_EQ(result.errors[0].location, Location{Position{3, 16}, Position{3, 19}}); +} + +TEST_CASE_FIXTURE(Fixture, "table_simple_call") +{ + CheckResult result = check(R"( +local a = setmetatable({ x = 2 }, { + __call = function(self) + return (self.x :: number) * 2 -- should work without annotation in the future + end +}) +local b = a() +local c = a(2) -- too many arguments + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ("Argument count mismatch. Function expects 1 argument, but 2 are specified", toString(result.errors[0])); +} + +TEST_CASE_FIXTURE(Fixture, "access_index_metamethod_that_returns_variadic") +{ + CheckResult result = check(R"( + type Foo = {x: string} + local t = {} + setmetatable(t, { + __index = function(x: string): ...Foo + return {x = x} + end + }) + + local foo = t.bar + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + ToStringOptions o; + o.exhaustive = true; + CHECK_EQ("{| x: string |}", toString(requireType("foo"), o)); +} + +TEST_CASE_FIXTURE(Fixture, "dont_invalidate_the_properties_iterator_of_free_table_when_rolled_back") +{ + fileResolver.source["Module/Backend/Types"] = R"( + export type Fiber = { + return_: Fiber? + } + return {} + )"; + + fileResolver.source["Module/Backend"] = R"( + local Types = require(script.Types) + type Fiber = Types.Fiber + type ReactRenderer = { findFiberByHostInstance: () -> Fiber? } + + local function attach(renderer): () + local function getPrimaryFiber(fiber) + local alternate = fiber.alternate + return fiber + end + + local function getFiberIDForNative() + local fiber = renderer.findFiberByHostInstance() + fiber = fiber.return_ + return getPrimaryFiber(fiber) + end + end + + function culprit(renderer: ReactRenderer): () + attach(renderer) + end + + return culprit + )"; + + CheckResult result = frontend.check("Module/Backend"); +} + +TEST_CASE_FIXTURE(Fixture, "checked_prop_too_early") +{ + CheckResult result = check(R"( + local t: {x: number?}? = {x = nil} + local u = t.x and t or 5 + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ("Value of type '{| x: number? |}?' could be nil", toString(result.errors[0])); + CHECK_EQ("number | {| x: number? |}", toString(requireType("u"))); +} + +TEST_CASE_FIXTURE(Fixture, "accidentally_checked_prop_in_opposite_branch") +{ + CheckResult result = check(R"( + local t: {x: number?}? = {x = nil} + local u = t and t.x == 5 or t.x == 31337 + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ("Value of type '{| x: number? |}?' could be nil", toString(result.errors[0])); + CHECK_EQ("boolean", toString(requireType("u"))); +} + +/* + * We had an issue where part of the type of pairs() was an unsealed table. + * This test depends on FFlagDebugLuauFreezeArena to trigger it. + */ +TEST_CASE_FIXTURE(Fixture, "pairs_parameters_are_not_unsealed_tables") +{ + check(R"( + function _(l0:{n0:any}) + _ = pairs + end + )"); +} + +TEST_CASE_FIXTURE(Fixture, "table_function_check_use_after_free") +{ + CheckResult result = check(R"( +local t = {} + +function t.x(value) + for k,v in pairs(t) do end +end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +/* + * When we add new properties to an unsealed table, we should do a level check and promote the property type to be at + * the level of the table. + */ +TEST_CASE_FIXTURE(Fixture, "inferred_properties_of_a_table_should_start_with_the_same_TypeLevel_of_that_table") +{ + CheckResult result = check(R"( + --!strict + local T = {} + + local function f(prop) + T[1] = { + prop = prop, + } + end + + local function g() + local l = T[1].prop + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.test.cpp b/tests/TypeInfer.test.cpp index d7bbad20..660ddcfc 100644 --- a/tests/TypeInfer.test.cpp +++ b/tests/TypeInfer.test.cpp @@ -60,15 +60,6 @@ TEST_CASE_FIXTURE(Fixture, "tc_error_2") }})); } -TEST_CASE_FIXTURE(Fixture, "tc_function") -{ - CheckResult result = check("function five() return 5 end"); - LUAU_REQUIRE_NO_ERRORS(result); - - const FunctionTypeVar* fiveType = get(requireType("five")); - REQUIRE(fiveType != nullptr); -} - TEST_CASE_FIXTURE(Fixture, "infer_locals_with_nil_value") { CheckResult result = check("local f = nil; f = 'hello world'"); @@ -108,462 +99,12 @@ TEST_CASE_FIXTURE(Fixture, "infer_in_nocheck_mode") LUAU_REQUIRE_NO_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "check_function_bodies") -{ - CheckResult result = check("function myFunction() local a = 0 a = true end"); - LUAU_REQUIRE_ERROR_COUNT(1, result); - - CHECK_EQ(result.errors[0], (TypeError{Location{Position{0, 44}, Position{0, 48}}, TypeMismatch{ - typeChecker.numberType, - typeChecker.booleanType, - }})); -} - -TEST_CASE_FIXTURE(Fixture, "infer_return_type") -{ - CheckResult result = check("function take_five() return 5 end"); - LUAU_REQUIRE_NO_ERRORS(result); - - const FunctionTypeVar* takeFiveType = get(requireType("take_five")); - REQUIRE(takeFiveType != nullptr); - - std::vector retVec = flatten(takeFiveType->retType).first; - REQUIRE(!retVec.empty()); - - REQUIRE_EQ(*follow(retVec[0]), *typeChecker.numberType); -} - -TEST_CASE_FIXTURE(Fixture, "infer_from_function_return_type") -{ - CheckResult result = check("function take_five() return 5 end local five = take_five()"); - LUAU_REQUIRE_NO_ERRORS(result); - - CHECK_EQ(*typeChecker.numberType, *follow(requireType("five"))); -} - -TEST_CASE_FIXTURE(Fixture, "cannot_call_primitives") -{ - CheckResult result = check("local foo = 5 foo()"); - LUAU_REQUIRE_ERROR_COUNT(1, result); - - REQUIRE(get(result.errors[0]) != nullptr); -} - -TEST_CASE_FIXTURE(Fixture, "cannot_call_tables") -{ - CheckResult result = check("local foo = {} foo()"); - LUAU_REQUIRE_ERROR_COUNT(1, result); - - CHECK(get(result.errors[0]) != nullptr); -} - -TEST_CASE_FIXTURE(Fixture, "infer_that_function_does_not_return_a_table") -{ - CheckResult result = check(R"( - function take_five() - return 5 - end - - take_five().prop = 888 - )"); - - LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK_EQ(result.errors[0], (TypeError{Location{Position{5, 8}, Position{5, 24}}, NotATable{typeChecker.numberType}})); -} - TEST_CASE_FIXTURE(Fixture, "expr_statement") { CheckResult result = check("local foo = 5 foo()"); LUAU_REQUIRE_ERROR_COUNT(1, result); } -TEST_CASE_FIXTURE(Fixture, "generic_function") -{ - CheckResult result = check(R"( - function id(x) return x end - local a = id(55) - local b = id(nil) - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - - CHECK_EQ(*typeChecker.numberType, *requireType("a")); - CHECK_EQ(*typeChecker.nilType, *requireType("b")); -} - -TEST_CASE_FIXTURE(Fixture, "vararg_functions_should_allow_calls_of_any_types_and_size") -{ - CheckResult result = check(R"( - function f(...) end - - f(1) - f("foo", 2) - )"); - - LUAU_REQUIRE_NO_ERRORS(result); -} - -TEST_CASE_FIXTURE(Fixture, "vararg_function_is_quantified") -{ - CheckResult result = check(R"( - local T = {} - function T.f(...) - local result = {} - - for i = 1, select("#", ...) do - local dictionary = select(i, ...) - for key, value in pairs(dictionary) do - result[key] = value - end - end - - return result - end - - return T - )"); - - auto r = first(getMainModule()->getModuleScope()->returnType); - REQUIRE(r); - - TableTypeVar* ttv = getMutable(*r); - REQUIRE(ttv); - - TypeId k = ttv->props["f"].type; - REQUIRE(k); - - LUAU_REQUIRE_NO_ERRORS(result); -} - -TEST_CASE_FIXTURE(Fixture, "for_loop") -{ - CheckResult result = check(R"( - local q - for i=0, 50, 2 do - q = i - end - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - - CHECK_EQ(*typeChecker.numberType, *requireType("q")); -} - -TEST_CASE_FIXTURE(Fixture, "for_in_loop") -{ - CheckResult result = check(R"( - local n - local s - for i, v in pairs({ "foo" }) do - n = i - s = v - end - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - - CHECK_EQ(*typeChecker.numberType, *requireType("n")); - CHECK_EQ(*typeChecker.stringType, *requireType("s")); -} - -TEST_CASE_FIXTURE(Fixture, "for_in_loop_with_next") -{ - CheckResult result = check(R"( - local n - local s - for i, v in next, { "foo", "bar" } do - n = i - s = v - end - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - - CHECK_EQ(*typeChecker.numberType, *requireType("n")); - CHECK_EQ(*typeChecker.stringType, *requireType("s")); -} - -TEST_CASE_FIXTURE(Fixture, "for_in_with_an_iterator_of_type_any") -{ - CheckResult result = check(R"( - local it: any - local a, b - for i, v in it do - a, b = i, v - end - )"); - - LUAU_REQUIRE_NO_ERRORS(result); -} - -TEST_CASE_FIXTURE(Fixture, "for_in_loop_should_fail_with_non_function_iterator") -{ - CheckResult result = check(R"( - local foo = "bar" - for i, v in foo do - end - )"); - - LUAU_REQUIRE_ERROR_COUNT(1, result); -} - -TEST_CASE_FIXTURE(Fixture, "for_in_with_just_one_iterator_is_ok") -{ - CheckResult result = check(R"( - local function keys(dictionary) - local new = {} - local index = 1 - - for key in pairs(dictionary) do - new[index] = key - index = index + 1 - end - - return new - end - )"); - - LUAU_REQUIRE_NO_ERRORS(result); -} - -TEST_CASE_FIXTURE(Fixture, "for_in_with_a_custom_iterator_should_type_check") -{ - CheckResult result = check(R"( - local function range(l, h): () -> number - return function() - return l - end - end - - for n: string in range(1, 10) do - print(n) - end - )"); - - LUAU_REQUIRE_ERROR_COUNT(1, result); -} - -TEST_CASE_FIXTURE(Fixture, "for_in_loop_on_error") -{ - CheckResult result = check(R"( - function f(x) - gobble.prop = x.otherprop - end - - local p - for _, part in i_am_not_defined do - p = part - f(part) - part.thirdprop = false - end - )"); - - CHECK_EQ(2, result.errors.size()); - - TypeId p = requireType("p"); - CHECK_EQ("*unknown*", toString(p)); -} - -TEST_CASE_FIXTURE(Fixture, "for_in_loop_on_non_function") -{ - CheckResult result = check(R"( - local bad_iter = 5 - - for a in bad_iter() do - end - )"); - - LUAU_REQUIRE_ERROR_COUNT(1, result); - - REQUIRE(get(result.errors[0])); -} - -TEST_CASE_FIXTURE(Fixture, "for_in_loop_error_on_factory_not_returning_the_right_amount_of_values") -{ - CheckResult result = check(R"( - local function hasDivisors(value: number, table) - return false - end - - function prime_iter(state, index) - while hasDivisors(index, state) do - index += 1 - end - - state[index] = true - return index - end - - function primes1() - return prime_iter, {} - end - - function primes2() - return prime_iter, {}, "" - end - - function primes3() - return prime_iter, {}, 2 - end - - for p in primes1() do print(p) end -- mismatch in argument count - - for p in primes2() do print(p) end -- mismatch in argument types, prime_iter takes {}, number, we are given {}, string - - for p in primes3() do print(p) end -- no error - )"); - - LUAU_REQUIRE_ERROR_COUNT(2, result); - - CountMismatch* acm = get(result.errors[0]); - REQUIRE(acm); - CHECK_EQ(acm->context, CountMismatch::Arg); - CHECK_EQ(2, acm->expected); - CHECK_EQ(1, acm->actual); - - TypeMismatch* tm = get(result.errors[1]); - REQUIRE(tm); - CHECK_EQ(typeChecker.numberType, tm->wantedType); - CHECK_EQ(typeChecker.stringType, tm->givenType); -} - -TEST_CASE_FIXTURE(Fixture, "for_in_loop_error_on_iterator_requiring_args_but_none_given") -{ - CheckResult result = check(R"( - function prime_iter(state, index) - return 1 - end - - for p in prime_iter do print(p) end - )"); - - LUAU_REQUIRE_ERROR_COUNT(1, result); - - CountMismatch* acm = get(result.errors[0]); - REQUIRE(acm); - CHECK_EQ(acm->context, CountMismatch::Arg); - CHECK_EQ(2, acm->expected); - CHECK_EQ(0, acm->actual); -} - -TEST_CASE_FIXTURE(Fixture, "for_in_loop_iterator_returns_any") -{ - CheckResult result = check(R"( - function bar(): any - return true - end - - local a - for b in bar do - a = b - end - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - - CHECK_EQ(typeChecker.anyType, requireType("a")); -} - -TEST_CASE_FIXTURE(Fixture, "for_in_loop_iterator_returns_any2") -{ - CheckResult result = check(R"( - function bar(): any - return true - end - - local a - for b in bar() do - a = b - end - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - - CHECK_EQ("any", toString(requireType("a"))); -} - -TEST_CASE_FIXTURE(Fixture, "for_in_loop_iterator_is_any") -{ - CheckResult result = check(R"( - local bar: any - - local a - for b in bar do - a = b - end - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - - CHECK_EQ("any", toString(requireType("a"))); -} - -TEST_CASE_FIXTURE(Fixture, "for_in_loop_iterator_is_any2") -{ - CheckResult result = check(R"( - local bar: any - - local a - for b in bar() do - a = b - end - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - - CHECK_EQ("any", toString(requireType("a"))); -} - -TEST_CASE_FIXTURE(Fixture, "for_in_loop_iterator_is_error") -{ - CheckResult result = check(R"( - local a - for b in bar do - a = b - end - )"); - - LUAU_REQUIRE_ERROR_COUNT(1, result); - - CHECK_EQ("*unknown*", toString(requireType("a"))); -} - -TEST_CASE_FIXTURE(Fixture, "for_in_loop_iterator_is_error2") -{ - CheckResult result = check(R"( - function bar(c) return c end - - local a - for b in bar() do - a = b - end - )"); - - LUAU_REQUIRE_ERROR_COUNT(1, result); - - CHECK_EQ("*unknown*", toString(requireType("a"))); -} - -TEST_CASE_FIXTURE(Fixture, "for_in_loop_with_custom_iterator") -{ - CheckResult result = check(R"( - function primes() - return function (state: number) end, 2 - end - - for p, q in primes do - q = "" - end - )"); - - LUAU_REQUIRE_ERROR_COUNT(1, result); - - TypeMismatch* tm = get(result.errors[0]); - REQUIRE(tm); - CHECK_EQ(typeChecker.numberType, tm->wantedType); - CHECK_EQ(typeChecker.stringType, tm->givenType); -} - TEST_CASE_FIXTURE(Fixture, "if_statement") { CheckResult result = check(R"( @@ -583,474 +124,6 @@ TEST_CASE_FIXTURE(Fixture, "if_statement") CHECK_EQ(*typeChecker.numberType, *requireType("b")); } -TEST_CASE_FIXTURE(Fixture, "while_loop") -{ - CheckResult result = check(R"( - local i - while true do - i = 8 - end - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - - CHECK_EQ(*typeChecker.numberType, *requireType("i")); -} - -TEST_CASE_FIXTURE(Fixture, "repeat_loop") -{ - CheckResult result = check(R"( - local i - repeat - i = 'hi' - until true - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - - CHECK_EQ(*typeChecker.stringType, *requireType("i")); -} - -TEST_CASE_FIXTURE(Fixture, "repeat_loop_condition_binds_to_its_block") -{ - CheckResult result = check(R"( - repeat - local x = true - until x - )"); - - LUAU_REQUIRE_NO_ERRORS(result); -} - -TEST_CASE_FIXTURE(Fixture, "symbols_in_repeat_block_should_not_be_visible_beyond_until_condition") -{ - CheckResult result = check(R"( - repeat - local x = true - until x - - print(x) - )"); - - LUAU_REQUIRE_ERROR_COUNT(1, result); -} - -TEST_CASE_FIXTURE(Fixture, "table_length") -{ - CheckResult result = check(R"( - local t = {} - local s = #t - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - - CHECK(nullptr != get(requireType("t"))); - CHECK_EQ(*typeChecker.numberType, *requireType("s")); -} - -TEST_CASE_FIXTURE(Fixture, "string_length") -{ - CheckResult result = check(R"( - local s = "Hello, World!" - local t = #s - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - CHECK_EQ("number", toString(requireType("t"))); -} - -TEST_CASE_FIXTURE(Fixture, "string_index") -{ - CheckResult result = check(R"( - local s = "Hello, World!" - local t = s[4] - )"); - - LUAU_REQUIRE_ERROR_COUNT(1, result); - - NotATable* nat = get(result.errors[0]); - REQUIRE(nat); - CHECK_EQ("string", toString(nat->ty)); - - CHECK_EQ("*unknown*", toString(requireType("t"))); -} - -TEST_CASE_FIXTURE(Fixture, "length_of_error_type_does_not_produce_an_error") -{ - CheckResult result = check(R"( - local l = #this_is_not_defined - )"); - - LUAU_REQUIRE_ERROR_COUNT(1, result); -} - -TEST_CASE_FIXTURE(Fixture, "indexing_error_type_does_not_produce_an_error") -{ - CheckResult result = check(R"( - local originalReward = unknown.Parent.Reward:GetChildren()[1] - )"); - - LUAU_REQUIRE_ERROR_COUNT(1, result); -} - -TEST_CASE_FIXTURE(Fixture, "nil_assign_doesnt_hit_indexer") -{ - CheckResult result = check("local a = {} a[0] = 7 a[0] = nil"); - LUAU_REQUIRE_ERROR_COUNT(0, result); -} - -TEST_CASE_FIXTURE(Fixture, "wrong_assign_does_hit_indexer") -{ - CheckResult result = check("local a = {} a[0] = 7 a[0] = 't'"); - LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK_EQ(result.errors[0], (TypeError{Location{Position{0, 30}, Position{0, 33}}, TypeMismatch{ - typeChecker.numberType, - typeChecker.stringType, - }})); -} - -TEST_CASE_FIXTURE(Fixture, "nil_assign_doesnt_hit_no_indexer") -{ - CheckResult result = check("local a = {a=1, b=2} a['a'] = nil"); - LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK_EQ(result.errors[0], (TypeError{Location{Position{0, 30}, Position{0, 33}}, TypeMismatch{ - typeChecker.numberType, - typeChecker.nilType, - }})); -} - -TEST_CASE_FIXTURE(Fixture, "dot_on_error_type_does_not_produce_an_error") -{ - CheckResult result = check(R"( - local foo = (true).x - foo.x = foo.y - )"); - - LUAU_REQUIRE_ERROR_COUNT(1, result); -} - -TEST_CASE_FIXTURE(Fixture, "dont_suggest_using_colon_rather_than_dot_if_not_defined_with_colon") -{ - CheckResult result = check(R"( - local someTable = {} - - someTable.Function1 = function(Arg1) - end - - someTable.Function1() -- Argument count mismatch - )"); - - LUAU_REQUIRE_ERROR_COUNT(1, result); - - REQUIRE(get(result.errors[0])); -} - -TEST_CASE_FIXTURE(Fixture, "dont_suggest_using_colon_rather_than_dot_if_it_wont_help_2") -{ - CheckResult result = check(R"( - local someTable = {} - - someTable.Function2 = function(Arg1, Arg2) - end - - someTable.Function2() -- Argument count mismatch - )"); - - LUAU_REQUIRE_ERROR_COUNT(1, result); - - REQUIRE(get(result.errors[0])); -} - -TEST_CASE_FIXTURE(Fixture, "dont_suggest_using_colon_rather_than_dot_if_another_overload_works") -{ - CheckResult result = check(R"( - type T = {method: ((T, number) -> number) & ((number) -> number)} - local T: T - - T.method(4) - )"); - - LUAU_REQUIRE_NO_ERRORS(result); -} - -TEST_CASE_FIXTURE(Fixture, "list_only_alternative_overloads_that_match_argument_count") -{ - CheckResult result = check(R"( - local multiply: ((number)->number) & ((number)->string) & ((number, number)->number) - multiply("") - )"); - - LUAU_REQUIRE_ERROR_COUNT(2, result); - - TypeMismatch* tm = get(result.errors[0]); - REQUIRE(tm); - CHECK_EQ(typeChecker.numberType, tm->wantedType); - CHECK_EQ(typeChecker.stringType, tm->givenType); - - ExtraInformation* ei = get(result.errors[1]); - REQUIRE(ei); - CHECK_EQ("Other overloads are also not viable: (number) -> string", ei->message); -} - -TEST_CASE_FIXTURE(Fixture, "list_all_overloads_if_no_overload_takes_given_argument_count") -{ - CheckResult result = check(R"( - local multiply: ((number)->number) & ((number)->string) & ((number, number)->number) - multiply() - )"); - - LUAU_REQUIRE_ERROR_COUNT(2, result); - - GenericError* ge = get(result.errors[0]); - REQUIRE(ge); - CHECK_EQ("No overload for function accepts 0 arguments.", ge->message); - - ExtraInformation* ei = get(result.errors[1]); - REQUIRE(ei); - CHECK_EQ("Available overloads: (number) -> number; (number) -> string; and (number, number) -> number", ei->message); -} - -TEST_CASE_FIXTURE(Fixture, "dont_give_other_overloads_message_if_only_one_argument_matching_overload_exists") -{ - CheckResult result = check(R"( - local multiply: ((number)->number) & ((number)->string) & ((number, number)->number) - multiply(1, "") - )"); - - LUAU_REQUIRE_ERROR_COUNT(1, result); - - TypeMismatch* tm = get(result.errors[0]); - REQUIRE(tm); - CHECK_EQ(typeChecker.numberType, tm->wantedType); - CHECK_EQ(typeChecker.stringType, tm->givenType); -} - -TEST_CASE_FIXTURE(Fixture, "infer_return_type_from_selected_overload") -{ - CheckResult result = check(R"( - type T = {method: ((T, number) -> number) & ((number) -> string)} - local T: T - - local a = T.method(T, 4) - local b = T.method(5) - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - CHECK_EQ("number", toString(requireType("a"))); - CHECK_EQ("string", toString(requireType("b"))); -} - -TEST_CASE_FIXTURE(Fixture, "too_many_arguments") -{ - CheckResult result = check(R"( - --!nonstrict - - function g(a: number) end - - g() - - )"); - - LUAU_REQUIRE_ERROR_COUNT(1, result); - - auto err = result.errors[0]; - auto acm = get(err); - REQUIRE(acm); - - CHECK_EQ(1, acm->expected); - CHECK_EQ(0, acm->actual); -} - -TEST_CASE_FIXTURE(Fixture, "any_type_propagates") -{ - CheckResult result = check(R"( - local foo: any - local bar = foo:method("argument") - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - - CHECK_EQ("any", toString(requireType("bar"))); -} - -TEST_CASE_FIXTURE(Fixture, "can_subscript_any") -{ - CheckResult result = check(R"( - local foo: any - local bar = foo[5] - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - - CHECK_EQ("any", toString(requireType("bar"))); -} - -// Not strictly correct: metatables permit overriding this -TEST_CASE_FIXTURE(Fixture, "can_get_length_of_any") -{ - CheckResult result = check(R"( - local foo: any = {} - local bar = #foo - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - - CHECK_EQ(PrimitiveTypeVar::Number, getPrimitiveType(requireType("bar"))); -} - -TEST_CASE_FIXTURE(Fixture, "recursive_function") -{ - CheckResult result = check(R"( - function count(n: number) - if n == 0 then - return 0 - else - return count(n - 1) - end - end - )"); - - LUAU_REQUIRE_NO_ERRORS(result); -} - -TEST_CASE_FIXTURE(Fixture, "lambda_form_of_local_function_cannot_be_recursive") -{ - CheckResult result = check(R"( - local f = function() return f() end - )"); - - LUAU_REQUIRE_ERROR_COUNT(1, result); -} - -TEST_CASE_FIXTURE(Fixture, "recursive_local_function") -{ - CheckResult result = check(R"( - local function count(n: number) - if n == 0 then - return 0 - else - return count(n - 1) - end - end - )"); - - LUAU_REQUIRE_NO_ERRORS(result); -} - -// FIXME: This and the above case get handled very differently. It's pretty dumb. -// We really should unify the two code paths, probably by deleting AstStatFunction. -TEST_CASE_FIXTURE(Fixture, "another_recursive_local_function") -{ - CheckResult result = check(R"( - local count - function count(n: number) - if n == 0 then - return 0 - else - return count(n - 1) - end - end - )"); - - LUAU_REQUIRE_NO_ERRORS(result); -} - -TEST_CASE_FIXTURE(Fixture, "cyclic_function_type_in_rets") -{ - CheckResult result = check(R"( - function f() - return f - end - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - CHECK_EQ("t1 where t1 = () -> t1", toString(requireType("f"))); -} - -TEST_CASE_FIXTURE(Fixture, "cyclic_function_type_in_args") -{ - CheckResult result = check(R"( - function f(g) - return f(f) - end - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - CHECK_EQ("t1 where t1 = (t1) -> ()", toString(requireType("f"))); -} - -// TODO: File a Jira about this -/* -TEST_CASE_FIXTURE(Fixture, "unifying_vararg_pack_with_fixed_length_pack_produces_fixed_length_pack") -{ - CheckResult result = check(R"( - function a(x) return 1 end - a(...) - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - - REQUIRE(bool(getMainModule()->getModuleScope()->varargPack)); - - TypePackId varargPack = *getMainModule()->getModuleScope()->varargPack; - - auto iter = begin(varargPack); - auto endIter = end(varargPack); - - CHECK(iter != endIter); - ++iter; - CHECK(iter == endIter); - - CHECK(!iter.tail()); -} -*/ - -TEST_CASE_FIXTURE(Fixture, "method_depends_on_table") -{ - CheckResult result = check(R"( - -- This catches a bug where x:m didn't count as a use of x - -- so toposort would happily reorder a definition of - -- function x:m before the definition of x. - function g() f() end - local x = {} - function x:m() end - function f() x:m() end - )"); - - LUAU_REQUIRE_NO_ERRORS(result); -} - -TEST_CASE_FIXTURE(Fixture, "another_higher_order_function") -{ - CheckResult result = check(R"( - local Get_des - function Get_des(func) - Get_des(func) - end - - local function f(d) - d:IsA("BasePart") - d.Parent:FindFirstChild("Humanoid") - d:IsA("Decal") - end - Get_des(f) - - )"); - - LUAU_REQUIRE_NO_ERRORS(result); -} - -TEST_CASE_FIXTURE(Fixture, "another_other_higher_order_function") -{ - CheckResult result = check(R"( - local d - d:foo() - d:foo() - )"); - - LUAU_REQUIRE_NO_ERRORS(result); -} - TEST_CASE_FIXTURE(Fixture, "statements_are_topologically_sorted") { CheckResult result = check(R"( @@ -1067,121 +140,6 @@ TEST_CASE_FIXTURE(Fixture, "statements_are_topologically_sorted") dumpErrors(result); } -TEST_CASE_FIXTURE(Fixture, "generic_table_method") -{ - CheckResult result = check(R"( - local T = {} - - function T:bar(i) - return i - end - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - - TypeId tType = requireType("T"); - TableTypeVar* tTable = getMutable(tType); - REQUIRE(tTable != nullptr); - - TypeId barType = tTable->props["bar"].type; - REQUIRE(barType != nullptr); - - const FunctionTypeVar* ftv = get(follow(barType)); - REQUIRE_MESSAGE(ftv != nullptr, "Should be a function: " << *barType); - - std::vector args = flatten(ftv->argTypes).first; - TypeId argType = args.at(1); - - CHECK_MESSAGE(get(argType), "Should be generic: " << *barType); -} - -TEST_CASE_FIXTURE(Fixture, "correctly_instantiate_polymorphic_member_functions") -{ - CheckResult result = check(R"( - local T = {} - - function T:foo() - return T:bar(5) - end - - function T:bar(i) - return i - end - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - dumpErrors(result); - - const TableTypeVar* t = get(requireType("T")); - REQUIRE(t != nullptr); - - std::optional fooProp = get(t->props, "foo"); - REQUIRE(bool(fooProp)); - - const FunctionTypeVar* foo = get(follow(fooProp->type)); - REQUIRE(bool(foo)); - - std::optional ret_ = first(foo->retType); - REQUIRE(bool(ret_)); - TypeId ret = follow(*ret_); - - REQUIRE_EQ(getPrimitiveType(ret), PrimitiveTypeVar::Number); -} - -TEST_CASE_FIXTURE(Fixture, "methods_are_topologically_sorted") -{ - CheckResult result = check(R"( - local T = {} - - function T:foo() - return T:bar(999), T:bar("hi") - end - - function T:bar(i) - return i - end - - local a, b = T:foo() - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - dumpErrors(result); - - CHECK_EQ(PrimitiveTypeVar::Number, getPrimitiveType(requireType("a"))); - CHECK_EQ(PrimitiveTypeVar::String, getPrimitiveType(requireType("b"))); -} - -TEST_CASE_FIXTURE(Fixture, "local_function") -{ - CheckResult result = check(R"( - function f() - return 8 - end - - function g() - local function f() - return 'hello' - end - return f - end - - local h = g() - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - - TypeId h = follow(requireType("h")); - - const FunctionTypeVar* ftv = get(h); - REQUIRE(ftv != nullptr); - - std::optional rt = first(ftv->retType); - REQUIRE(bool(rt)); - - TypeId retType = follow(*rt); - CHECK_EQ(PrimitiveTypeVar::String, getPrimitiveType(retType)); -} - TEST_CASE_FIXTURE(Fixture, "unify_nearly_identical_recursive_types") { CheckResult result = check(R"( @@ -1193,267 +151,10 @@ TEST_CASE_FIXTURE(Fixture, "unify_nearly_identical_recursive_types") o = p )"); -} - -/* - * We had a bug in instantiation where the argument types of 'f' and 'g' would be inferred as - * f {+ method: function(): (t2, T3...) +} - * g {+ method: function({+ method: function(): (t2, T3...) +}): (t5, T6...) +} - * - * The type of 'g' is totally wrong as t2 and t5 should be unified, as should T3 with T6. - * - * The correct unification of the argument to 'g' is - * - * {+ method: function(): (t5, T6...) +} - */ -TEST_CASE_FIXTURE(Fixture, "instantiate_cyclic_generic_function") -{ - auto result = check(R"( - function f(o) - o:method() - end - - function g(o) - f(o) - end - )"); - - TypeId g = requireType("g"); - const FunctionTypeVar* gFun = get(g); - REQUIRE(gFun != nullptr); - - auto optionArg = first(gFun->argTypes); - REQUIRE(bool(optionArg)); - - TypeId arg = follow(*optionArg); - const TableTypeVar* argTable = get(arg); - REQUIRE(argTable != nullptr); - - std::optional methodProp = get(argTable->props, "method"); - REQUIRE(bool(methodProp)); - - const FunctionTypeVar* methodFunction = get(methodProp->type); - REQUIRE(methodFunction != nullptr); - - std::optional methodArg = first(methodFunction->argTypes); - REQUIRE(bool(methodArg)); - - REQUIRE_EQ(follow(*methodArg), follow(arg)); -} - -TEST_CASE_FIXTURE(Fixture, "varlist_declared_by_for_in_loop_should_be_free") -{ - CheckResult result = check(R"( - local T = {} - - function T.f(p) - for i, v in pairs(p) do - T.f(v) - end - end - )"); LUAU_REQUIRE_NO_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "properly_infer_iteratee_is_a_free_table") -{ - // In this case, we cannot know the element type of the table {}. It could be anything. - // We therefore must initially ascribe a free typevar to iter. - CheckResult result = check(R"( - for iter in pairs({}) do - iter:g().p = true - end - )"); - - LUAU_REQUIRE_NO_ERRORS(result); -} - -TEST_CASE_FIXTURE(Fixture, "quantify_methods_defined_using_dot_syntax_and_explicit_self_parameter") -{ - check(R"( - local T = {} - - function T.method(self) - self:method() - end - - function T.method2(self) - self:method() - end - - T:method2() - )"); -} - -TEST_CASE_FIXTURE(Fixture, "free_rhs_table_can_also_be_bound") -{ - check(R"( - local o - local v = o:i() - - function g(u) - v = u - end - - o:f(g) - o:h() - o:h() - )"); -} - -TEST_CASE_FIXTURE(Fixture, "require") -{ - fileResolver.source["game/A"] = R"( - local function hooty(x: number): string - return "Hi there!" - end - - return {hooty=hooty} - )"; - - fileResolver.source["game/B"] = R"( - local Hooty = require(game.A) - - local h -- free! - local i = Hooty.hooty(h) - )"; - - CheckResult aResult = frontend.check("game/A"); - dumpErrors(aResult); - LUAU_REQUIRE_NO_ERRORS(aResult); - - CheckResult bResult = frontend.check("game/B"); - dumpErrors(bResult); - LUAU_REQUIRE_NO_ERRORS(bResult); - - ModulePtr b = frontend.moduleResolver.modules["game/B"]; - - REQUIRE(b != nullptr); - - dumpErrors(bResult); - - std::optional iType = requireType(b, "i"); - REQUIRE_EQ("string", toString(*iType)); - - std::optional hType = requireType(b, "h"); - REQUIRE_EQ("number", toString(*hType)); -} - -TEST_CASE_FIXTURE(Fixture, "require_types") -{ - fileResolver.source["workspace/A"] = R"( - export type Point = {x: number, y: number} - - return {} - )"; - - fileResolver.source["workspace/B"] = R"( - local Hooty = require(workspace.A) - - local h: Hooty.Point - )"; - - CheckResult bResult = frontend.check("workspace/B"); - dumpErrors(bResult); - - ModulePtr b = frontend.moduleResolver.modules["workspace/B"]; - REQUIRE(b != nullptr); - - TypeId hType = requireType(b, "h"); - REQUIRE_MESSAGE(bool(get(hType)), "Expected table but got " << toString(hType)); -} - -TEST_CASE_FIXTURE(Fixture, "require_a_variadic_function") -{ - fileResolver.source["game/A"] = R"( - local T = {} - function T.f(...) end - return T - )"; - - fileResolver.source["game/B"] = R"( - local A = require(game.A) - local f = A.f - )"; - - CheckResult result = frontend.check("game/B"); - - ModulePtr bModule = frontend.moduleResolver.getModule("game/B"); - REQUIRE(bModule != nullptr); - - TypeId f = follow(requireType(bModule, "f")); - - const FunctionTypeVar* ftv = get(f); - REQUIRE(ftv); - - auto iter = begin(ftv->argTypes); - auto endIter = end(ftv->argTypes); - - REQUIRE(iter == endIter); - REQUIRE(iter.tail()); - - CHECK(get(*iter.tail())); -} - -TEST_CASE_FIXTURE(Fixture, "assign_prop_to_table_by_calling_any_yields_any") -{ - CheckResult result = check(R"( - local f: any - local T = {} - - T.prop = f() - - return T - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - - TableTypeVar* ttv = getMutable(requireType("T")); - REQUIRE(ttv); - REQUIRE(ttv->props.count("prop")); - - REQUIRE_EQ("any", toString(ttv->props["prop"].type)); -} - -TEST_CASE_FIXTURE(Fixture, "type_error_of_unknown_qualified_type") -{ - CheckResult result = check(R"( - local p: SomeModule.DoesNotExist - )"); - - LUAU_REQUIRE_ERROR_COUNT(1, result); - - REQUIRE_EQ(result.errors[0], (TypeError{Location{{1, 17}, {1, 40}}, UnknownSymbol{"SomeModule.DoesNotExist"}})); -} - -TEST_CASE_FIXTURE(Fixture, "require_module_that_does_not_export") -{ - const std::string sourceA = R"( - )"; - - const std::string sourceB = R"( - local Hooty = require(script.Parent.A) - )"; - - fileResolver.source["game/Workspace/A"] = sourceA; - fileResolver.source["game/Workspace/B"] = sourceB; - - frontend.check("game/Workspace/A"); - frontend.check("game/Workspace/B"); - - ModulePtr aModule = frontend.moduleResolver.modules["game/Workspace/A"]; - ModulePtr bModule = frontend.moduleResolver.modules["game/Workspace/B"]; - - CHECK(aModule->errors.empty()); - REQUIRE_EQ(1, bModule->errors.size()); - CHECK_MESSAGE(get(bModule->errors[0]), "Should be IllegalRequire: " << toString(bModule->errors[0])); - - auto hootyType = requireType(bModule, "Hooty"); - - CHECK_EQ("*unknown*", toString(hootyType)); -} - TEST_CASE_FIXTURE(Fixture, "warn_on_lowercase_parent_property") { CheckResult result = check(R"( @@ -1468,144 +169,6 @@ TEST_CASE_FIXTURE(Fixture, "warn_on_lowercase_parent_property") REQUIRE_EQ("parent", ed->symbol); } -TEST_CASE_FIXTURE(Fixture, "quantify_any_does_not_bind_to_itself") -{ - CheckResult result = check(R"( - local A : any - function A.B() end - A:C() - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - - TypeId aType = requireType("A"); - CHECK_EQ(aType, typeChecker.anyType); -} - -TEST_CASE_FIXTURE(Fixture, "table_unifies_into_map") -{ - CheckResult result = check(R"( - local Instance: any - local UDim2: any - - function Create(instanceType) - return function(data) - local obj = Instance.new(instanceType) - for k, v in pairs(data) do - if type(k) == 'number' then - --v.Parent = obj - else - obj[k] = v - end - end - return obj - end - end - - local topbarShadow = Create'ImageLabel'{ - Name = "TopBarShadow"; - Size = UDim2.new(1, 0, 0, 3); - Position = UDim2.new(0, 0, 1, 0); - Image = "rbxasset://textures/ui/TopBar/dropshadow.png"; - BackgroundTransparency = 1; - Active = false; - Visible = false; - }; - - )"); - - LUAU_REQUIRE_NO_ERRORS(result); -} - -TEST_CASE_FIXTURE(Fixture, "func_expr_doesnt_leak_free") -{ - CheckResult result = check(R"( - local p = function(x) return x end - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - const Luau::FunctionTypeVar* fn = get(requireType("p")); - REQUIRE(fn); - auto ret = first(fn->retType); - REQUIRE(ret); - REQUIRE(get(follow(*ret))); -} - -TEST_CASE_FIXTURE(Fixture, "instantiate_generic_function_in_assignments") -{ - CheckResult result = check(R"( - function foo(a, b) - return a(b) - end - - function bar() - local c: ((number)->number, number)->number = foo -- no error - c = foo -- no error - local d: ((number)->number, string)->number = foo -- error from arg 2 (string) not being convertable to number from the call a(b) - end - )"); - - LUAU_REQUIRE_ERROR_COUNT(1, result); - - TypeMismatch* tm = get(result.errors[0]); - REQUIRE(tm); - CHECK_EQ("((number) -> number, string) -> number", toString(tm->wantedType)); - CHECK_EQ("((number) -> number, number) -> number", toString(tm->givenType)); -} - -TEST_CASE_FIXTURE(Fixture, "instantiate_generic_function_in_assignments2") -{ - CheckResult result = check(R"( - function foo(a, b) - return a(b) - end - - function bar() - local _: (string, string)->number = foo -- string cannot be converted to (string)->number - end - )"); - - LUAU_REQUIRE_ERROR_COUNT(1, result); - - TypeMismatch* tm = get(result.errors[0]); - REQUIRE(tm); - CHECK_EQ("(string, string) -> number", toString(tm->wantedType)); - CHECK_EQ("((string) -> number, string) -> number", toString(*tm->givenType)); -} - -TEST_CASE_FIXTURE(Fixture, "string_method") -{ - CheckResult result = check(R"( - local p = ("tacos"):len() - )"); - CHECK_EQ(0, result.errors.size()); - - CHECK_EQ(*requireType("p"), *typeChecker.numberType); -} - -TEST_CASE_FIXTURE(Fixture, "string_function_indirect") -{ - CheckResult result = check(R"( - local s:string - local l = s.lower - local p = l(s) - )"); - CHECK_EQ(0, result.errors.size()); - - CHECK_EQ(*requireType("p"), *typeChecker.stringType); -} - -TEST_CASE_FIXTURE(Fixture, "string_function_other") -{ - CheckResult result = check(R"( - local s:string - local p = s:match("foo") - )"); - CHECK_EQ(0, result.errors.size()); - - CHECK_EQ(toString(requireType("p")), "string?"); -} - TEST_CASE_FIXTURE(Fixture, "weird_case") { CheckResult result = check(R"( @@ -1617,80 +180,6 @@ TEST_CASE_FIXTURE(Fixture, "weird_case") dumpErrors(result); } -TEST_CASE_FIXTURE(Fixture, "or_joins_types") -{ - CheckResult result = check(R"( - local s = "a" or 10 - local x:string|number = s - )"); - LUAU_REQUIRE_NO_ERRORS(result); - CHECK_EQ(toString(*requireType("s")), "number | string"); - CHECK_EQ(toString(*requireType("x")), "number | string"); -} - -TEST_CASE_FIXTURE(Fixture, "or_joins_types_with_no_extras") -{ - CheckResult result = check(R"( - local s = "a" or 10 - local x:number|string = s - local y = x or "s" - )"); - CHECK_EQ(0, result.errors.size()); - CHECK_EQ(toString(*requireType("s")), "number | string"); - CHECK_EQ(toString(*requireType("y")), "number | string"); -} - -TEST_CASE_FIXTURE(Fixture, "or_joins_types_with_no_superfluous_union") -{ - CheckResult result = check(R"( - local s = "a" or "b" - local x:string = s - )"); - CHECK_EQ(0, result.errors.size()); - CHECK_EQ(*requireType("s"), *typeChecker.stringType); -} - -TEST_CASE_FIXTURE(Fixture, "and_adds_boolean") -{ - CheckResult result = check(R"( - local s = "a" and 10 - local x:boolean|number = s - )"); - CHECK_EQ(0, result.errors.size()); - CHECK_EQ(toString(*requireType("s")), "boolean | number"); -} - -TEST_CASE_FIXTURE(Fixture, "and_adds_boolean_no_superfluous_union") -{ - CheckResult result = check(R"( - local s = "a" and true - local x:boolean = s - )"); - CHECK_EQ(0, result.errors.size()); - CHECK_EQ(*requireType("x"), *typeChecker.booleanType); -} - -TEST_CASE_FIXTURE(Fixture, "and_or_ternary") -{ - CheckResult result = check(R"( - local s = (1/2) > 0.5 and "a" or 10 - )"); - CHECK_EQ(0, result.errors.size()); - CHECK_EQ(toString(*requireType("s")), "number | string"); -} - -TEST_CASE_FIXTURE(Fixture, "first_argument_can_be_optional") -{ - CheckResult result = check(R"( - local T = {} - function T.new(a: number?, b: number?, c: number?) return 5 end - local m = T.new() - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - dumpErrors(result); -} - TEST_CASE_FIXTURE(Fixture, "dont_ice_when_failing_the_occurs_check") { CheckResult result = check(R"( @@ -1726,303 +215,6 @@ TEST_CASE_FIXTURE(Fixture, "crazy_complexity") } #endif -// We had a bug where a cyclic union caused a stack overflow. -// ex type U = number | U -TEST_CASE_FIXTURE(Fixture, "dont_allow_cyclic_unions_to_be_inferred") -{ - CheckResult result = check(R"( - --!strict - - function f(a, b) - a:g(b or {}) - a:g(b) - end - )"); - - LUAU_REQUIRE_NO_ERRORS(result); -} - -TEST_CASE_FIXTURE(Fixture, "it_is_ok_not_to_supply_enough_retvals") -{ - CheckResult result = check(R"( - function get_two() return 5, 6 end - - local a = get_two() - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - dumpErrors(result); -} - -TEST_CASE_FIXTURE(Fixture, "duplicate_functions2") -{ - CheckResult result = check(R"( - function foo() end - - function bar() - local function foo() end - end - )"); - - LUAU_REQUIRE_ERROR_COUNT(0, result); -} - -TEST_CASE_FIXTURE(Fixture, "duplicate_functions_allowed_in_nonstrict") -{ - CheckResult result = check(R"( - --!nonstrict - function foo() end - - function foo() end - - function bar() - local function foo() end - end - )"); - - LUAU_REQUIRE_NO_ERRORS(result); -} - -TEST_CASE_FIXTURE(Fixture, "duplicate_functions_with_different_signatures_not_allowed_in_nonstrict") -{ - CheckResult result = check(R"( - --!nonstrict - function foo(): number - return 1 - end - foo() - - function foo(n: number): number - return 2 - end - foo() - )"); - - LUAU_REQUIRE_ERROR_COUNT(1, result); - - TypeMismatch* tm = get(result.errors[0]); - REQUIRE(tm); - CHECK_EQ("() -> number", toString(tm->wantedType)); - CHECK_EQ("(number) -> number", toString(tm->givenType)); -} - -TEST_CASE_FIXTURE(Fixture, "tables_get_names_from_their_locals") -{ - CheckResult result = check(R"( - local T = {} - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - - CHECK_EQ("T", toString(requireType("T"))); -} - -TEST_CASE_FIXTURE(Fixture, "generalize_table_argument") -{ - CheckResult result = check(R"( - function foo(arr) - local work = {} - for i = 1, #arr do - work[i] = arr[i] - end - - return arr - end - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - dumpErrors(result); - - const FunctionTypeVar* fooType = get(requireType("foo")); - REQUIRE(fooType); - - std::optional fooArg1 = first(fooType->argTypes); - REQUIRE(fooArg1); - - const TableTypeVar* fooArg1Table = get(*fooArg1); - REQUIRE(fooArg1Table); - - CHECK_EQ(fooArg1Table->state, TableState::Generic); -} - -TEST_CASE_FIXTURE(Fixture, "complicated_return_types_require_an_explicit_annotation") -{ - CheckResult result = check(R"( - local i = 0 - function most_of_the_natural_numbers(): number? - if i < 10 then - i = i + 1 - return i - else - return nil - end - end - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - - const FunctionTypeVar* functionType = get(requireType("most_of_the_natural_numbers")); - - std::optional retType = first(functionType->retType); - REQUIRE(retType); - CHECK(get(*retType)); -} - -TEST_CASE_FIXTURE(Fixture, "infer_higher_order_function") -{ - CheckResult result = check(R"( - function apply(f, x) - return f(x) - end - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - - const FunctionTypeVar* ftv = get(requireType("apply")); - REQUIRE(ftv != nullptr); - - std::vector argVec = flatten(ftv->argTypes).first; - - REQUIRE_EQ(2, argVec.size()); - - const FunctionTypeVar* fType = get(follow(argVec[0])); - REQUIRE(fType != nullptr); - - std::vector fArgs = flatten(fType->argTypes).first; - - TypeId xType = argVec[1]; - - CHECK_EQ(1, fArgs.size()); - CHECK_EQ(xType, fArgs[0]); -} - -TEST_CASE_FIXTURE(Fixture, "higher_order_function_2") -{ - CheckResult result = check(R"( - function bottomupmerge(comp, a, b, left, mid, right) - local i, j = left, mid - for k = left, right do - if i < mid and (j > right or not comp(a[j], a[i])) then - b[k] = a[i] - i = i + 1 - else - b[k] = a[j] - j = j + 1 - end - end - end - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - - const FunctionTypeVar* ftv = get(requireType("bottomupmerge")); - REQUIRE(ftv != nullptr); - - std::vector argVec = flatten(ftv->argTypes).first; - - REQUIRE_EQ(6, argVec.size()); - - const FunctionTypeVar* fType = get(follow(argVec[0])); - REQUIRE(fType != nullptr); -} - -TEST_CASE_FIXTURE(Fixture, "higher_order_function_3") -{ - CheckResult result = check(R"( - function swap(p) - local t = p[0] - p[0] = p[1] - p[1] = t - return nil - end - - function swapTwice(p) - swap(p) - swap(p) - return p - end - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - - const FunctionTypeVar* ftv = get(requireType("swapTwice")); - REQUIRE(ftv != nullptr); - - std::vector argVec = flatten(ftv->argTypes).first; - - REQUIRE_EQ(1, argVec.size()); - - const TableTypeVar* argType = get(follow(argVec[0])); - REQUIRE(argType != nullptr); - - CHECK(bool(argType->indexer)); -} - -TEST_CASE_FIXTURE(Fixture, "higher_order_function_4") -{ - CheckResult result = check(R"( - function bottomupmerge(comp, a, b, left, mid, right) - local i, j = left, mid - for k = left, right do - if i < mid and (j > right or not comp(a[j], a[i])) then - b[k] = a[i] - i = i + 1 - else - b[k] = a[j] - j = j + 1 - end - end - end - - function mergesort(arr, comp) - local work = {} - for i = 1, #arr do - work[i] = arr[i] - end - local width = 1 - while width < #arr do - for i = 1, #arr, 2*width do - bottomupmerge(comp, arr, work, i, math.min(i+width, #arr), math.min(i+2*width-1, #arr)) - end - local temp = work - work = arr - arr = temp - width = width * 2 - end - return arr - end - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - dumpErrors(result); - - /* - * mergesort takes two arguments: an array of some type T and a function that takes two Ts. - * We must assert that these two types are in fact the same type. - * In other words, comp(arr[x], arr[y]) is well-typed. - */ - - const FunctionTypeVar* ftv = get(requireType("mergesort")); - REQUIRE(ftv != nullptr); - - std::vector argVec = flatten(ftv->argTypes).first; - - REQUIRE_EQ(2, argVec.size()); - - const TableTypeVar* arg0 = get(follow(argVec[0])); - REQUIRE(arg0 != nullptr); - REQUIRE(bool(arg0->indexer)); - - const FunctionTypeVar* arg1 = get(follow(argVec[1])); - REQUIRE(arg1 != nullptr); - REQUIRE_EQ(2, size(arg1->argTypes)); - - std::vector arg1Args = flatten(arg1->argTypes).first; - - CHECK_EQ(*arg0->indexer->indexResultType, *arg1Args[0]); - CHECK_EQ(*arg0->indexer->indexResultType, *arg1Args[1]); -} - TEST_CASE_FIXTURE(Fixture, "type_errors_infer_types") { CheckResult result = check(R"( @@ -2046,373 +238,6 @@ TEST_CASE_FIXTURE(Fixture, "type_errors_infer_types") CHECK_EQ("*unknown*", toString(requireType("f"))); } -TEST_CASE_FIXTURE(Fixture, "calling_error_type_yields_error") -{ - CheckResult result = check(R"( - local a = unknown.Parent.Reward.GetChildren() - )"); - - LUAU_REQUIRE_ERROR_COUNT(1, result); - - UnknownSymbol* err = get(result.errors[0]); - REQUIRE(err != nullptr); - - CHECK_EQ("unknown", err->name); - - CHECK_EQ("*unknown*", toString(requireType("a"))); -} - -TEST_CASE_FIXTURE(Fixture, "chain_calling_error_type_yields_error") -{ - CheckResult result = check(R"( - local a = Utility.Create "Foo" {} - )"); - - CHECK_EQ("*unknown*", toString(requireType("a"))); -} - -TEST_CASE_FIXTURE(Fixture, "primitive_arith_no_metatable") -{ - CheckResult result = check(R"( - function add(a: number, b: string) - return a + (tonumber(b) :: number), a .. b - end - local n, s = add(2,"3") - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - - const FunctionTypeVar* functionType = get(requireType("add")); - - std::optional retType = first(functionType->retType); - CHECK_EQ(std::optional(typeChecker.numberType), retType); - CHECK_EQ(requireType("n"), typeChecker.numberType); - CHECK_EQ(requireType("s"), typeChecker.stringType); -} - -TEST_CASE_FIXTURE(Fixture, "primitive_arith_no_metatable_with_follows") -{ - CheckResult result = check(R"( - local PI=3.1415926535897931 - local SOLAR_MASS=4*PI * PI - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - CHECK_EQ(requireType("SOLAR_MASS"), typeChecker.numberType); -} - -TEST_CASE_FIXTURE(Fixture, "primitive_arith_possible_metatable") -{ - CheckResult result = check(R"( - function add(a: number, b: any) - return a + b - end - local t = add(1,2) - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - CHECK_EQ("any", toString(requireType("t"))); -} - -TEST_CASE_FIXTURE(Fixture, "some_primitive_binary_ops") -{ - CheckResult result = check(R"( - local a = 4 + 8 - local b = a + 9 - local s = 'hotdogs' - local t = s .. s - local c = b - a - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - - CHECK_EQ("number", toString(requireType("a"))); - CHECK_EQ("number", toString(requireType("b"))); - CHECK_EQ("string", toString(requireType("s"))); - CHECK_EQ("string", toString(requireType("t"))); - CHECK_EQ("number", toString(requireType("c"))); -} - -TEST_CASE_FIXTURE(Fixture, "typecheck_overloaded_multiply_that_is_an_intersection") -{ - ScopedFastFlag sff{"LuauErrorRecoveryType", true}; - - CheckResult result = check(R"( - --!strict - local Vec3 = {} - Vec3.__index = Vec3 - function Vec3.new() - return setmetatable({x=0, y=0, z=0}, Vec3) - end - - export type Vec3 = typeof(Vec3.new()) - - local thefun: any = function(self, o) return self end - - local multiply: ((Vec3, Vec3) -> Vec3) & ((Vec3, number) -> Vec3) = thefun - - Vec3.__mul = multiply - - local a = Vec3.new() - local b = Vec3.new() - local c = a * b - local d = a * 2 - local e = a * 'cabbage' - )"); - - LUAU_REQUIRE_ERROR_COUNT(1, result); - - CHECK_EQ("Vec3", toString(requireType("a"))); - CHECK_EQ("Vec3", toString(requireType("b"))); - CHECK_EQ("Vec3", toString(requireType("c"))); - CHECK_EQ("Vec3", toString(requireType("d"))); - CHECK_EQ("Vec3", toString(requireType("e"))); -} - -TEST_CASE_FIXTURE(Fixture, "typecheck_overloaded_multiply_that_is_an_intersection_on_rhs") -{ - ScopedFastFlag sff{"LuauErrorRecoveryType", true}; - - CheckResult result = check(R"( - --!strict - local Vec3 = {} - Vec3.__index = Vec3 - function Vec3.new() - return setmetatable({x=0, y=0, z=0}, Vec3) - end - - export type Vec3 = typeof(Vec3.new()) - - local thefun: any = function(self, o) return self end - - local multiply: ((Vec3, Vec3) -> Vec3) & ((Vec3, number) -> Vec3) = thefun - - Vec3.__mul = multiply - - local a = Vec3.new() - local b = Vec3.new() - local c = b * a - local d = 2 * a - local e = 'cabbage' * a - )"); - - LUAU_REQUIRE_ERROR_COUNT(1, result); - - CHECK_EQ("Vec3", toString(requireType("a"))); - CHECK_EQ("Vec3", toString(requireType("b"))); - CHECK_EQ("Vec3", toString(requireType("c"))); - CHECK_EQ("Vec3", toString(requireType("d"))); - CHECK_EQ("Vec3", toString(requireType("e"))); -} - -TEST_CASE_FIXTURE(Fixture, "compare_numbers") -{ - CheckResult result = check(R"( - local a = 441 - local b = 0 - local c = a < b - )"); - - LUAU_REQUIRE_NO_ERRORS(result); -} - -TEST_CASE_FIXTURE(Fixture, "compare_strings") -{ - CheckResult result = check(R"( - local a = '441' - local b = '0' - local c = a < b - )"); - - LUAU_REQUIRE_NO_ERRORS(result); -} - -TEST_CASE_FIXTURE(Fixture, "cannot_indirectly_compare_types_that_do_not_have_a_metatable") -{ - CheckResult result = check(R"( - local a = {} - local b = {} - local c = a < b - )"); - - LUAU_REQUIRE_ERROR_COUNT(1, result); - - GenericError* gen = get(result.errors[0]); - - REQUIRE_EQ(gen->message, "Type a cannot be compared with < because it has no metatable"); -} - -TEST_CASE_FIXTURE(Fixture, "cannot_indirectly_compare_types_that_do_not_offer_overloaded_ordering_operators") -{ - CheckResult result = check(R"( - local M = {} - function M.new() - return setmetatable({}, M) - end - type M = typeof(M.new()) - - local a = M.new() - local b = M.new() - local c = a < b - )"); - - LUAU_REQUIRE_ERROR_COUNT(1, result); - - GenericError* gen = get(result.errors[0]); - REQUIRE(gen != nullptr); - REQUIRE_EQ(gen->message, "Table M does not offer metamethod __lt"); -} - -TEST_CASE_FIXTURE(Fixture, "cannot_compare_tables_that_do_not_have_the_same_metatable") -{ - CheckResult result = check(R"( - --!strict - local M = {} - function M.new() - return setmetatable({}, M) - end - function M.__lt(left, right) return true end - - local a = M.new() - local b = {} - local c = a < b -- line 10 - local d = b < a -- line 11 - )"); - - LUAU_REQUIRE_ERROR_COUNT(2, result); - - REQUIRE_EQ((Location{{10, 18}, {10, 23}}), result.errors[0].location); - - REQUIRE_EQ((Location{{11, 18}, {11, 23}}), result.errors[1].location); -} - -TEST_CASE_FIXTURE(Fixture, "produce_the_correct_error_message_when_comparing_a_table_with_a_metatable_with_one_that_does_not") -{ - CheckResult result = check(R"( - --!strict - local M = {} - function M.new() - return setmetatable({}, M) - end - function M.__lt(left, right) return true end - type M = typeof(M.new()) - - local a = M.new() - local b = {} - local c = a < b -- line 10 - )"); - - LUAU_REQUIRE_ERROR_COUNT(1, result); - - auto err = get(result.errors[0]); - REQUIRE(err != nullptr); - - // Frail. :| - REQUIRE_EQ("Types M and b cannot be compared with < because they do not have the same metatable", err->message); -} - -TEST_CASE_FIXTURE(Fixture, "in_nonstrict_mode_strip_nil_from_intersections_when_considering_relational_operators") -{ - CheckResult result = check(R"( - --!nonstrict - - function maybe_a_number(): number? - return 50 - end - - local a = maybe_a_number() < maybe_a_number() - )"); - - LUAU_REQUIRE_NO_ERRORS(result); -} - -/* - * This test case exposed an oversight in the treatment of free tables. - * Free tables, like free TypeVars, need to record the scope depth where they were created so that - * we do not erroneously let-generalize them when they are used in a nested lambda. - * - * For more information about let-generalization, see - * - * The important idea here is that the return type of Counter.new is a table with some metatable. - * That metatable *must* be the same TypeVar as the type of Counter. If it is a copy (produced by - * the generalization process), then it loses the knowledge that its metatable will have an :incr() - * method. - */ -TEST_CASE_FIXTURE(Fixture, "dont_quantify_table_that_belongs_to_outer_scope") -{ - CheckResult result = check(R"( - local Counter = {} - Counter.__index = Counter - - function Counter.new() - local self = setmetatable({count=0}, Counter) - return self - end - - function Counter:incr() - self.count = 1 - return self.count - end - - local self = Counter.new() - print(self:incr()) - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - - TableTypeVar* counterType = getMutable(requireType("Counter")); - REQUIRE(counterType); - - const FunctionTypeVar* newType = get(follow(counterType->props["new"].type)); - REQUIRE(newType); - - std::optional newRetType = *first(newType->retType); - REQUIRE(newRetType); - - const MetatableTypeVar* newRet = get(follow(*newRetType)); - REQUIRE(newRet); - - const TableTypeVar* newRetMeta = get(newRet->metatable); - REQUIRE(newRetMeta); - - CHECK(newRetMeta->props.count("incr")); - CHECK_EQ(follow(newRet->metatable), follow(requireType("Counter"))); -} - -// TODO: CLI-39624 -TEST_CASE_FIXTURE(Fixture, "instantiate_tables_at_scope_level") -{ - CheckResult result = check(R"( - --!strict - local Option = {} - Option.__index = Option - function Option.Is(obj) - return (type(obj) == "table" and getmetatable(obj) == Option) - end - return Option - )"); - - LUAU_REQUIRE_NO_ERRORS(result); -} - -TEST_CASE_FIXTURE(Fixture, "typeguard_doesnt_leak_to_elseif") -{ - const std::string code = R"( - function f(a) - if type(a) == "boolean" then - local a1 = a - elseif a.fn() then - local a2 = a - else - local a3 = a - end - end - )"; - CheckResult result = check(code); - LUAU_REQUIRE_NO_ERRORS(result); - dumpErrors(result); -} - TEST_CASE_FIXTURE(Fixture, "should_be_able_to_infer_this_without_stack_overflowing") { CheckResult result = check(R"( @@ -2428,47 +253,6 @@ TEST_CASE_FIXTURE(Fixture, "should_be_able_to_infer_this_without_stack_overflowi LUAU_REQUIRE_NO_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "inferring_hundreds_of_self_calls_should_not_suffocate_memory") -{ - CheckResult result = check(R"( - ("foo") - :lower() - :lower() - :lower() - :lower() - :lower() - :lower() - :lower() - :lower() - :lower() - :lower() - :lower() - :lower() - :lower() - :lower() - :lower() - :lower() - :lower() - :lower() - )"); - - ModulePtr module = getMainModule(); - CHECK_GE(50, module->internalTypes.typeVars.size()); -} - -TEST_CASE_FIXTURE(Fixture, "inferring_crazy_table_should_also_be_quick") -{ - CheckResult result = check(R"( - --!strict - function f(U) - U(w:s(an):c()():c():U(s):c():c():U(s):c():U(s):cU()):c():U(s):c():U(s):c():c():U(s):c():U(s):cU() - end - )"); - - ModulePtr module = getMainModule(); - CHECK_GE(100, module->internalTypes.typeVars.size()); -} - TEST_CASE_FIXTURE(Fixture, "exponential_blowup_from_copying_types") { CheckResult result = check(R"( @@ -2507,96 +291,6 @@ TEST_CASE_FIXTURE(Fixture, "exponential_blowup_from_copying_types") CHECK_GE(5, module->interfaceTypes.typeVars.size()); } -TEST_CASE_FIXTURE(Fixture, "mutual_recursion") -{ - CheckResult result = check(R"( - --!strict - - function newPlayerCharacter() - startGui() -- Unknown symbol 'startGui' - end - - local characterAddedConnection: any - function startGui() - characterAddedConnection = game:GetService("Players").LocalPlayer.CharacterAdded:connect(newPlayerCharacter) - end - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - dumpErrors(result); -} - -TEST_CASE_FIXTURE(Fixture, "toposort_doesnt_break_mutual_recursion") -{ - CheckResult result = check(R"( - --!strict - local x = nil - function f() g() end - -- make sure print(x) doesn't get toposorted here, breaking the mutual block - function g() x = f end - print(x) - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - dumpErrors(result); -} - -TEST_CASE_FIXTURE(Fixture, "object_constructor_can_refer_to_method_of_self") -{ - // CLI-30902 - CheckResult result = check(R"( - --!strict - - type Foo = { - fooConn: () -> () | nil - } - - local Foo = {} - Foo.__index = Foo - - function Foo.new() - local self: Foo = { - fooConn = nil, - } - setmetatable(self, Foo) - - self.fooConn = function() - self:method() -- Key 'method' not found in table self - end - - return self - end - - function Foo:method() - print("foo") - end - - local foo = Foo.new() - - -- TODO This is the best our current refinement support can offer :( - local bar = foo.fooConn - if bar then bar() end - - -- foo.fooConn() - )"); - - LUAU_REQUIRE_NO_ERRORS(result); -} - -TEST_CASE_FIXTURE(Fixture, "replace_every_free_type_when_unifying_a_complex_function_with_any") -{ - CheckResult result = check(R"( - local a: any - local b - for _, i in pairs(a) do - b = i - end - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - CHECK_EQ("any", toString(requireType("b"))); -} - // In these tests, a successful parse is required, so we need the parser to return the AST and then we can test the recursion depth limit in type // checker. We also want it to somewhat match up with production values, so we push up the parser recursion limit a little bit instead. TEST_CASE_FIXTURE(Fixture, "check_type_infer_recursion_limit") @@ -2640,7 +334,7 @@ TEST_CASE_FIXTURE(Fixture, "check_expr_recursion_limit") #if defined(LUAU_ENABLE_ASAN) int limit = 250; #elif defined(_DEBUG) || defined(_NOOPT) - int limit = 350; + int limit = 300; #else int limit = 600; #endif @@ -2653,146 +347,6 @@ TEST_CASE_FIXTURE(Fixture, "check_expr_recursion_limit") CHECK(nullptr != get(result.errors[0])); } -TEST_CASE_FIXTURE(Fixture, "compound_assign_basic") -{ - CheckResult result = check(R"( - local s = 10 - s += 20 - )"); - CHECK_EQ(0, result.errors.size()); - CHECK_EQ(toString(*requireType("s")), "number"); -} - -TEST_CASE_FIXTURE(Fixture, "compound_assign_mismatch_op") -{ - CheckResult result = check(R"( - local s = 10 - s += true - )"); - LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK_EQ(result.errors[0], (TypeError{Location{{2, 13}, {2, 17}}, TypeMismatch{typeChecker.numberType, typeChecker.booleanType}})); -} - -TEST_CASE_FIXTURE(Fixture, "compound_assign_mismatch_result") -{ - CheckResult result = check(R"( - local s = 'hello' - s += 10 - )"); - - LUAU_REQUIRE_ERROR_COUNT(2, result); - CHECK_EQ(result.errors[0], (TypeError{Location{{2, 8}, {2, 9}}, TypeMismatch{typeChecker.numberType, typeChecker.stringType}})); - CHECK_EQ(result.errors[1], (TypeError{Location{{2, 8}, {2, 15}}, TypeMismatch{typeChecker.stringType, typeChecker.numberType}})); -} - -TEST_CASE_FIXTURE(Fixture, "compound_assign_metatable") -{ - CheckResult result = check(R"( - --!strict - type V2B = { x: number, y: number } - local v2b: V2B = { x = 0, y = 0 } - local VMT = {} - type V2 = typeof(setmetatable(v2b, VMT)) - - function VMT.__add(a: V2, b: V2): V2 - return setmetatable({ x = a.x + b.x, y = a.y + b.y }, VMT) - end - - local v1: V2 = setmetatable({ x = 1, y = 2 }, VMT) - local v2: V2 = setmetatable({ x = 3, y = 4 }, VMT) - v1 += v2 - )"); - CHECK_EQ(0, result.errors.size()); -} - -TEST_CASE_FIXTURE(Fixture, "compound_assign_mismatch_metatable") -{ - CheckResult result = check(R"( - --!strict - type V2B = { x: number, y: number } - local v2b: V2B = { x = 0, y = 0 } - local VMT = {} - type V2 = typeof(setmetatable(v2b, VMT)) - - function VMT.__mod(a: V2, b: V2): number - return a.x * b.x + a.y * b.y - end - - local v1: V2 = setmetatable({ x = 1, y = 2 }, VMT) - local v2: V2 = setmetatable({ x = 3, y = 4 }, VMT) - v1 %= v2 - )"); - LUAU_REQUIRE_ERROR_COUNT(1, result); - - TypeMismatch* tm = get(result.errors[0]); - CHECK_EQ(*tm->wantedType, *requireType("v2")); - CHECK_EQ(*tm->givenType, *typeChecker.numberType); -} - -TEST_CASE_FIXTURE(Fixture, "dont_ice_if_a_TypePack_is_an_error") -{ - CheckResult result = check(R"( - --!strict - function f(s) - print(s) - return f - end - - f("foo")("bar") - )"); -} - -TEST_CASE_FIXTURE(Fixture, "check_function_before_lambda_that_uses_it") -{ - CheckResult result = check(R"( - --!nonstrict - - function f() - return 114 - end - - return function() - return f():andThen() - end - )"); - - LUAU_REQUIRE_NO_ERRORS(result); -} - -TEST_CASE_FIXTURE(Fixture, "it_is_ok_to_oversaturate_a_higher_order_function_argument") -{ - CheckResult result = check(R"( - function onerror() end - function foo() end - xpcall(foo, onerror) - )"); - - LUAU_REQUIRE_NO_ERRORS(result); -} - -TEST_CASE_FIXTURE(Fixture, "another_indirect_function_case_where_it_is_ok_to_provide_too_many_arguments") -{ - CheckResult result = check(R"( - local mycb: (number, number) -> () - - function f() end - - mycb = f - )"); - - LUAU_REQUIRE_NO_ERRORS(result); -} - -TEST_CASE_FIXTURE(Fixture, "call_to_any_yields_any") -{ - CheckResult result = check(R"( - local a: any - local b = a() - )"); - - REQUIRE_EQ("any", toString(requireType("b"))); -} - TEST_CASE_FIXTURE(Fixture, "globals") { CheckResult result = check(R"( @@ -2853,91 +407,6 @@ TEST_CASE_FIXTURE(Fixture, "globals_everywhere") CHECK_EQ("any", toString(requireType("bar"))); } -TEST_CASE_FIXTURE(Fixture, "CheckMethodsOfAny") -{ - CheckResult result = check(R"( -local x: any = {} -function x:y(z: number) - local s: string = z -end -)"); - - LUAU_REQUIRE_ERROR_COUNT(1, result); -} - -TEST_CASE_FIXTURE(Fixture, "CheckMethodsOfSealed") -{ - CheckResult result = check(R"( -local x: {prop: number} = {prop=9999} -function x:y(z: number) - local s: string = z -end -)"); - - LUAU_REQUIRE_ERROR_COUNT(2, result); -} - -TEST_CASE_FIXTURE(Fixture, "CheckMethodsOfNumber") -{ - ScopedFastFlag sff{"LuauErrorRecoveryType", true}; - - CheckResult result = check(R"( -local x: number = 9999 -function x:y(z: number) - local s: string = z -end -)"); - - LUAU_REQUIRE_ERROR_COUNT(2, result); -} - -TEST_CASE_FIXTURE(Fixture, "CheckMethodsOfError") -{ - CheckResult result = check(R"( -local x = (true).foo -function x:y(z: number) - local s: string = z -end -)"); - - LUAU_REQUIRE_ERRORS(result); -} - -TEST_CASE_FIXTURE(Fixture, "CallOrOfFunctions") -{ - CheckResult result = check(R"( -function f() return 1; end -function g() return 2; end -(f or g)() -)"); - - LUAU_REQUIRE_NO_ERRORS(result); -} - -TEST_CASE_FIXTURE(Fixture, "CallAndOrOfFunctions") -{ - CheckResult result = check(R"( -function f() return 1; end -function g() return 2; end -local x = false -(x and f or g)() -)"); - - LUAU_REQUIRE_NO_ERRORS(result); -} - -TEST_CASE_FIXTURE(Fixture, "MixedPropertiesAndIndexers") -{ - CheckResult result = check(R"( -local x = {} -x.a = "a" -x[0] = true -x.b = 37 -)"); - - LUAU_REQUIRE_NO_ERRORS(result); -} - TEST_CASE_FIXTURE(Fixture, "correctly_scope_locals_do") { CheckResult result = check(R"( @@ -2955,35 +424,6 @@ TEST_CASE_FIXTURE(Fixture, "correctly_scope_locals_do") CHECK_EQ(us->name, "a"); } -TEST_CASE_FIXTURE(Fixture, "correctly_scope_locals_while") -{ - CheckResult result = check(R"( - while true do - local a = 1 - end - - print(a) -- oops! - )"); - - LUAU_REQUIRE_ERROR_COUNT(1, result); - - UnknownSymbol* us = get(result.errors[0]); - REQUIRE(us); - CHECK_EQ(us->name, "a"); -} - -TEST_CASE_FIXTURE(Fixture, "ipairs_produces_integral_indices") -{ - CheckResult result = check(R"( - local key - for i, e in ipairs({}) do key = i end - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - - REQUIRE_EQ("number", toString(requireType("key"))); -} - TEST_CASE_FIXTURE(Fixture, "checking_should_not_ice") { CHECK_NOTHROW(check(R"( @@ -2997,79 +437,6 @@ TEST_CASE_FIXTURE(Fixture, "checking_should_not_ice") CHECK_EQ("any", toString(requireType("value"))); } -TEST_CASE_FIXTURE(Fixture, "report_exiting_without_return_nonstrict") -{ - CheckResult result = check(R"( - --!nonstrict - - local function f1(v): number? - if v then - return 1 - end - end - - local function f2(v) - if v then - return 1 - end - end - - local function f3(v): () - if v then - return - end - end - - local function f4(v) - if v then - return - end - end - )"); - - LUAU_REQUIRE_ERROR_COUNT(1, result); - FunctionExitsWithoutReturning* err = get(result.errors[0]); - CHECK(err); -} - -TEST_CASE_FIXTURE(Fixture, "report_exiting_without_return_strict") -{ - CheckResult result = check(R"( - --!strict - - local function f1(v): number? - if v then - return 1 - end - end - - local function f2(v) - if v then - return 1 - end - end - - local function f3(v): () - if v then - return - end - end - - local function f4(v) - if v then - return - end - end - )"); - - LUAU_REQUIRE_ERROR_COUNT(2, result); - FunctionExitsWithoutReturning* annotatedErr = get(result.errors[0]); - CHECK(annotatedErr); - - FunctionExitsWithoutReturning* inferredErr = get(result.errors[1]); - CHECK(inferredErr); -} - // TEST_CASE_FIXTURE(Fixture, "infer_method_signature_of_argument") // { // CheckResult result = check(R"( @@ -3085,363 +452,6 @@ TEST_CASE_FIXTURE(Fixture, "report_exiting_without_return_strict") // CHECK_EQ("A", toString(requireType("f"))); // } -TEST_CASE_FIXTURE(Fixture, "warn_if_you_try_to_require_a_non_modulescript") -{ - fileResolver.source["Modules/A"] = ""; - fileResolver.sourceTypes["Modules/A"] = SourceCode::Local; - - fileResolver.source["Modules/B"] = R"( - local M = require(script.Parent.A) - )"; - - CheckResult result = frontend.check("Modules/B"); - - LUAU_REQUIRE_ERROR_COUNT(1, result); - - CHECK(get(result.errors[0])); -} - -TEST_CASE_FIXTURE(Fixture, "calling_function_with_incorrect_argument_type_yields_errors_spanning_argument") -{ - CheckResult result = check(R"( - function foo(a: number, b: string) end - - foo("Test", 123) - )"); - - LUAU_REQUIRE_ERROR_COUNT(2, result); - - CHECK_EQ(result.errors[0], (TypeError{Location{Position{3, 12}, Position{3, 18}}, TypeMismatch{ - typeChecker.numberType, - typeChecker.stringType, - }})); - - CHECK_EQ(result.errors[1], (TypeError{Location{Position{3, 20}, Position{3, 23}}, TypeMismatch{ - typeChecker.stringType, - typeChecker.numberType, - }})); -} - -TEST_CASE_FIXTURE(Fixture, "calling_function_with_anytypepack_doesnt_leak_free_types") -{ - CheckResult result = check(R"( - --!nonstrict - - function Test(a) - return 1, "" - end - - - local tab = {} - table.insert(tab, Test(1)); - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - - ToStringOptions opts; - opts.exhaustive = true; - opts.maxTableLength = 0; - - CHECK_EQ("{any}", toString(requireType("tab"), opts)); -} - -TEST_CASE_FIXTURE(Fixture, "too_many_return_values") -{ - CheckResult result = check(R"( - --!strict - - function f() - return 55 - end - - local a, b = f() - )"); - - LUAU_REQUIRE_ERROR_COUNT(1, result); - - CountMismatch* acm = get(result.errors[0]); - REQUIRE(acm); - CHECK_EQ(acm->context, CountMismatch::Result); - CHECK_EQ(acm->expected, 1); - CHECK_EQ(acm->actual, 2); -} - -TEST_CASE_FIXTURE(Fixture, "ignored_return_values") -{ - CheckResult result = check(R"( - --!strict - - function f() - return 55, "" - end - - local a = f() - )"); - - LUAU_REQUIRE_ERROR_COUNT(0, result); -} - -TEST_CASE_FIXTURE(Fixture, "function_does_not_return_enough_values") -{ - CheckResult result = check(R"( - --!strict - - function f(): (number, string) - return 55 - end - )"); - - LUAU_REQUIRE_ERROR_COUNT(1, result); - - CountMismatch* acm = get(result.errors[0]); - REQUIRE(acm); - CHECK_EQ(acm->context, CountMismatch::Return); - CHECK_EQ(acm->expected, 2); - CHECK_EQ(acm->actual, 1); -} - -TEST_CASE_FIXTURE(Fixture, "typecheck_unary_minus") -{ - CheckResult result = check(R"( - --!strict - local foo = { - value = 10 - } - local mt = {} - setmetatable(foo, mt) - - mt.__unm = function(val: typeof(foo)): string - return val.value .. "test" - end - - local a = -foo - - local b = 1+-1 - - local bar = { - value = 10 - } - local c = -bar -- disallowed - )"); - - LUAU_REQUIRE_ERROR_COUNT(1, result); - - CHECK_EQ("string", toString(requireType("a"))); - CHECK_EQ("number", toString(requireType("b"))); - - GenericError* gen = get(result.errors[0]); - REQUIRE_EQ(gen->message, "Unary operator '-' not supported by type 'bar'"); -} - -TEST_CASE_FIXTURE(Fixture, "unary_not_is_boolean") -{ - CheckResult result = check(R"( - local b = not "string" - local c = not (math.random() > 0.5 and "string" or 7) - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - REQUIRE_EQ("boolean", toString(requireType("b"))); - REQUIRE_EQ("boolean", toString(requireType("c"))); -} - -TEST_CASE_FIXTURE(Fixture, "disallow_string_and_types_without_metatables_from_arithmetic_binary_ops") -{ - CheckResult result = check(R"( - --!strict - local a = "1.24" + 123 -- not allowed - - local foo = { - value = 10 - } - - local b = foo + 1 -- not allowed - - local bar = { - value = 1 - } - - local mt = {} - - setmetatable(bar, mt) - - mt.__add = function(a: typeof(bar), b: number): number - return a.value + b - end - - local c = bar + 1 -- allowed - - local d = bar + foo -- not allowed - )"); - - LUAU_REQUIRE_ERROR_COUNT(3, result); - - TypeMismatch* tm = get(result.errors[0]); - REQUIRE_EQ(*tm->wantedType, *typeChecker.numberType); - REQUIRE_EQ(*tm->givenType, *typeChecker.stringType); - - TypeMismatch* tm2 = get(result.errors[2]); - CHECK_EQ(*tm2->wantedType, *typeChecker.numberType); - CHECK_EQ(*tm2->givenType, *requireType("foo")); - - GenericError* gen2 = get(result.errors[1]); - REQUIRE_EQ(gen2->message, "Binary operator '+' not supported by types 'foo' and 'number'"); -} - -// CLI-29033 -TEST_CASE_FIXTURE(Fixture, "unknown_type_in_comparison") -{ - CheckResult result = check(R"( - function merge(lower, greater) - if lower.y == greater.y then - end - end - )"); - - LUAU_REQUIRE_NO_ERRORS(result); -} - -TEST_CASE_FIXTURE(Fixture, "concat_op_on_free_lhs_and_string_rhs") -{ - CheckResult result = check(R"( - local function f(x) - return x .. "y" - end - )"); - - LUAU_REQUIRE_ERROR_COUNT(1, result); - REQUIRE(get(result.errors[0])); -} - -TEST_CASE_FIXTURE(Fixture, "concat_op_on_string_lhs_and_free_rhs") -{ - CheckResult result = check(R"( - local function f(x) - return "foo" .. x - end - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - - CHECK_EQ("(string) -> string", toString(requireType("f"))); -} - -TEST_CASE_FIXTURE(Fixture, "strict_binary_op_where_lhs_unknown") -{ - std::vector ops = {"+", "-", "*", "/", "%", "^", ".."}; - - std::string src = R"( - function foo(a, b) - )"; - - for (const auto& op : ops) - src += "local _ = a " + op + "b\n"; - - src += "end"; - - CheckResult result = check(src); - LUAU_REQUIRE_ERROR_COUNT(ops.size(), result); - - CHECK_EQ("Unknown type used in + operation; consider adding a type annotation to 'a'", toString(result.errors[0])); -} - -TEST_CASE_FIXTURE(Fixture, "function_cast_error_uses_correct_language") -{ - CheckResult result = check(R"( - function foo(a, b): number - return 0 - end - - local a: (string)->number = foo - local b: (number, number)->(number, number) = foo - - local c: (string, number)->number = foo -- no error - )"); - - LUAU_REQUIRE_ERROR_COUNT(2, result); - - auto tm1 = get(result.errors[0]); - REQUIRE(tm1); - - CHECK_EQ("(string) -> number", toString(tm1->wantedType)); - CHECK_EQ("(string, *unknown*) -> number", toString(tm1->givenType)); - - auto tm2 = get(result.errors[1]); - REQUIRE(tm2); - - CHECK_EQ("(number, number) -> (number, number)", toString(tm2->wantedType)); - CHECK_EQ("(string, *unknown*) -> number", toString(tm2->givenType)); -} - -TEST_CASE_FIXTURE(Fixture, "setmetatable_cant_be_used_to_mutate_global_types") -{ - { - Fixture fix; - - // inherit env from parent fixture checker - fix.typeChecker.globalScope = typeChecker.globalScope; - - fix.check(R"( ---!nonstrict -type MT = typeof(setmetatable) -function wtf(arg: {MT}): typeof(table) - arg = wtf(arg) -end -)"); - } - - // validate sharedEnv post-typecheck; valuable for debugging some typeck crashes but slows fuzzing down - // note: it's important for typeck to be destroyed at this point! - { - for (auto& p : typeChecker.globalScope->bindings) - { - toString(p.second.typeId); // toString walks the entire type, making sure ASAN catches access to destroyed type arenas - } - } -} - -TEST_CASE_FIXTURE(Fixture, "evil_table_unification") -{ - // this code re-infers the type of _ while processing fields of _, which can cause use-after-free - check(R"( ---!nonstrict -_ = ... -_:table(_,string)[_:gsub(_,...,n0)],_,_:gsub(_,string)[""],_:split(_,...,table)._,n0 = nil -do end -)"); -} - -TEST_CASE_FIXTURE(Fixture, "overload_is_not_a_function") -{ - check(R"( ---!nonstrict -function _(...):((typeof(not _))&(typeof(not _)))&((typeof(not _))&(typeof(not _))) -_(...)(setfenv,_,not _,"")[_] = nil -end -do end -_(...)(...,setfenv,_):_G() -)"); -} - -TEST_CASE_FIXTURE(Fixture, "cyclic_type_packs") -{ - // this has a risk of creating cyclic type packs, causing infinite loops / OOMs - check(R"( ---!nonstrict -_ += _(_,...) -repeat -_ += _(...) -until ... + _ -)"); - - check(R"( ---!nonstrict -_ += _(_(...,...),_(...)) -repeat -until _ -)"); -} - TEST_CASE_FIXTURE(Fixture, "cyclic_follow") { check(R"( @@ -3470,19 +480,6 @@ end )"); } -TEST_CASE_FIXTURE(Fixture, "and_binexps_dont_unify") -{ - CheckResult result = check(R"( - --!strict - local t = {} - while true and t[1] do - print(t[1].test) - end - )"); - - LUAU_REQUIRE_NO_ERRORS(result); -} - struct FindFreeTypeVars { bool foundOne = false; @@ -3506,32 +503,6 @@ struct FindFreeTypeVars } }; -TEST_CASE_FIXTURE(Fixture, "dont_crash_when_setmetatable_does_not_produce_a_metatabletypevar") -{ - CheckResult result = check("local x = setmetatable({})"); - LUAU_REQUIRE_ERROR_COUNT(1, result); -} - -TEST_CASE_FIXTURE(Fixture, "for_in_loop_where_iteratee_is_free") -{ - // This code doesn't pass typechecking. We just care that it doesn't crash. - (void)check(R"( - --!nonstrict - function _:_(...) - end - - repeat - if _ then - else - _ = ... - end - until _ - - for _ in _() do - end - )"); -} - TEST_CASE_FIXTURE(Fixture, "tc_after_error_recovery") { CheckResult result = check(R"( @@ -3604,36 +575,6 @@ TEST_CASE_FIXTURE(Fixture, "tc_after_error_recovery_no_replacement_name_in_error } } -TEST_CASE_FIXTURE(Fixture, "error_on_invalid_operand_types_to_relational_operators") -{ - CheckResult result = check(R"( - local a: boolean = true - local b: boolean = false - local foo = a < b - )"); - - LUAU_REQUIRE_ERROR_COUNT(1, result); - - GenericError* ge = get(result.errors[0]); - REQUIRE(ge); - CHECK_EQ("Type 'boolean' cannot be compared with relational operator <", ge->message); -} - -TEST_CASE_FIXTURE(Fixture, "error_on_invalid_operand_types_to_relational_operators2") -{ - CheckResult result = check(R"( - local a: number | string = "" - local b: number | string = 1 - local foo = a < b - )"); - - LUAU_REQUIRE_ERROR_COUNT(1, result); - - GenericError* ge = get(result.errors[0]); - REQUIRE(ge); - CHECK_EQ("Type 'number | string' cannot be compared with relational operator <", ge->message); -} - TEST_CASE_FIXTURE(Fixture, "index_expr_should_be_checked") { CheckResult result = check(R"( @@ -3650,100 +591,6 @@ TEST_CASE_FIXTURE(Fixture, "index_expr_should_be_checked") CHECK_EQ("x", up->key); } -TEST_CASE_FIXTURE(Fixture, "unreachable_code_after_infinite_loop") -{ - { - CheckResult result = check(R"( - function unreachablecodepath(a): number - while true do - if a then return 10 end - end - -- unreachable - end - unreachablecodepath(4) - )"); - - LUAU_REQUIRE_ERROR_COUNT(0, result); - } - - { - CheckResult result = check(R"( - function reachablecodepath(a): number - while true do - if a then break end - return 10 - end - - print("x") -- correct error - end - reachablecodepath(4) - )"); - - LUAU_REQUIRE_ERRORS(result); - CHECK(get(result.errors[0])); - } - - { - CheckResult result = check(R"( - function unreachablecodepath(a): number - repeat - if a then return 10 end - until false - - -- unreachable - end - unreachablecodepath(4) - )"); - - LUAU_REQUIRE_ERROR_COUNT(0, result); - } - - { - CheckResult result = check(R"( - function reachablecodepath(a, b): number - repeat - if a then break end - - if b then return 10 end - until false - - print("x") -- correct error - end - reachablecodepath(4) - )"); - - LUAU_REQUIRE_ERRORS(result); - CHECK(get(result.errors[0])); - } - - { - CheckResult result = check(R"( - function unreachablecodepath(a: number?): number - repeat - return 10 - until a ~= nil - - -- unreachable - end - unreachablecodepath(4) - )"); - - LUAU_REQUIRE_ERROR_COUNT(0, result); - } -} - -TEST_CASE_FIXTURE(Fixture, "cli_38355_recursive_union") -{ - CheckResult result = check(R"( - --!strict - local _ - _ += _ and _ or _ and _ or _ and _ - )"); - - LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK_EQ("Type contains a self-recursive construct that cannot be resolved", toString(result.errors[0])); -} - TEST_CASE_FIXTURE(Fixture, "stringify_nested_unions_with_optionals") { CheckResult result = check(R"( @@ -3759,58 +606,6 @@ TEST_CASE_FIXTURE(Fixture, "stringify_nested_unions_with_optionals") CHECK_EQ("(boolean | number | string)?", toString(tm->givenType)); } -TEST_CASE_FIXTURE(Fixture, "UnknownGlobalCompoundAssign") -{ - // In non-strict mode, global definition is still allowed - { - CheckResult result = check(R"( - --!nonstrict - a = a + 1 - print(a) - )"); - - LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK_EQ(toString(result.errors[0]), "Unknown global 'a'"); - } - - // In strict mode we no longer generate two errors from lhs - { - CheckResult result = check(R"( - --!strict - a += 1 - print(a) - )"); - - LUAU_REQUIRE_ERRORS(result); - CHECK_EQ(toString(result.errors[0]), "Unknown global 'a'"); - } - - // In non-strict mode, compound assignment is not a definition, it's a modification - { - CheckResult result = check(R"( - --!nonstrict - a += 1 - print(a) - )"); - - LUAU_REQUIRE_ERROR_COUNT(2, result); - CHECK_EQ(toString(result.errors[0]), "Unknown global 'a'"); - } -} - -TEST_CASE_FIXTURE(Fixture, "loop_typecheck_crash_on_empty_optional") -{ - CheckResult result = check(R"( - local t = {} - for _ in t do - for _ in assert(missing()) do - end - end - )"); - - LUAU_REQUIRE_ERROR_COUNT(2, result); -} - TEST_CASE_FIXTURE(Fixture, "cli_39932_use_unifier_in_ensure_methods") { CheckResult result = check(R"( @@ -3821,26 +616,6 @@ TEST_CASE_FIXTURE(Fixture, "cli_39932_use_unifier_in_ensure_methods") LUAU_REQUIRE_NO_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "metatable_of_any_can_be_a_table") -{ - CheckResult result = check(R"( ---!strict -local T: any -T = {} -T.__index = T -function T.new(...) - local self = {} - setmetatable(self, T) - self:construct(...) - return self -end -function T:construct(index) -end -)"); - - LUAU_REQUIRE_NO_ERRORS(result); -} - TEST_CASE_FIXTURE(Fixture, "dont_report_type_errors_within_an_AstStatError") { CheckResult result = check(R"( @@ -3868,64 +643,6 @@ TEST_CASE_FIXTURE(Fixture, "dont_ice_on_astexprerror") LUAU_REQUIRE_ERROR_COUNT(1, result); } -TEST_CASE_FIXTURE(Fixture, "strip_nil_from_lhs_or_operator") -{ - CheckResult result = check(R"( ---!strict -local a: number? = nil -local b: number = a or 1 - )"); - - LUAU_REQUIRE_NO_ERRORS(result); -} - -TEST_CASE_FIXTURE(Fixture, "strip_nil_from_lhs_or_operator2") -{ - CheckResult result = check(R"( ---!nonstrict -local a: number? = nil -local b: number = a or 1 - )"); - - LUAU_REQUIRE_NO_ERRORS(result); -} - -TEST_CASE_FIXTURE(Fixture, "dont_strip_nil_from_rhs_or_operator") -{ - CheckResult result = check(R"( ---!strict -local a: number? = nil -local b: number = 1 or a - )"); - - LUAU_REQUIRE_ERROR_COUNT(1, result); - - TypeMismatch* tm = get(result.errors[0]); - REQUIRE(tm); - CHECK_EQ(typeChecker.numberType, tm->wantedType); - CHECK_EQ("number?", toString(tm->givenType)); -} - -TEST_CASE_FIXTURE(Fixture, "no_lossy_function_type") -{ - CheckResult result = check(R"( - --!strict - local tbl = {} - function tbl:abc(a: number, b: number) - return a - end - tbl:abc(1, 2) -- Line 6 - -- | Column 14 - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - TypeId type = requireTypeAtPosition(Position(6, 14)); - CHECK_EQ("(tbl, number, number) -> number", toString(type)); - auto ftv = get(type); - REQUIRE(ftv); - CHECK(ftv->hasSelf); -} - TEST_CASE_FIXTURE(Fixture, "luau_resolves_symbols_the_same_way_lua_does") { CheckResult result = check(R"( @@ -3943,326 +660,6 @@ TEST_CASE_FIXTURE(Fixture, "luau_resolves_symbols_the_same_way_lua_does") REQUIRE_MESSAGE(get(e) != nullptr, "Expected UnknownSymbol, but got " << e); } -TEST_CASE_FIXTURE(Fixture, "operator_eq_verifies_types_do_intersect") -{ - CheckResult result = check(R"( - type Array = { [number]: T } - type Fiber = { id: number } - type null = {} - - local fiberStack: Array = {} - local index = 0 - - local function f(fiber: Fiber) - local a = fiber ~= fiberStack[index] - local b = fiberStack[index] ~= fiber - end - - return f - )"); - - LUAU_REQUIRE_NO_ERRORS(result); -} - -TEST_CASE_FIXTURE(Fixture, "general_require_call_expression") -{ - fileResolver.source["game/A"] = R"( ---!strict -return { def = 4 } - )"; - - fileResolver.source["game/B"] = R"( ---!strict -local tbl = { abc = require(game.A) } -local a : string = "" -a = tbl.abc.def - )"; - - CheckResult result = frontend.check("game/B"); - LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK_EQ("Type 'number' could not be converted into 'string'", toString(result.errors[0])); -} - -TEST_CASE_FIXTURE(Fixture, "general_require_type_mismatch") -{ - fileResolver.source["game/A"] = R"( -return { def = 4 } - )"; - - fileResolver.source["game/B"] = R"( -local tbl: string = require(game.A) - )"; - - CheckResult result = frontend.check("game/B"); - LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK_EQ("Type '{| def: number |}' could not be converted into 'string'", toString(result.errors[0])); -} - -TEST_CASE_FIXTURE(Fixture, "nonstrict_self_mismatch_tail") -{ - CheckResult result = check(R"( ---!nonstrict -local f = {} -function f:foo(a: number, b: number) end - -function bar(...) - f.foo(f, 1, ...) -end - -bar(2) -)"); - LUAU_REQUIRE_NO_ERRORS(result); -} - -TEST_CASE_FIXTURE(Fixture, "typeof_unresolved_function") -{ - CheckResult result = check(R"( -local function f(a: typeof(f)) end -)"); - LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK_EQ("Unknown global 'f'", toString(result.errors[0])); -} - -TEST_CASE_FIXTURE(Fixture, "instantiate_table_cloning") -{ - CheckResult result = check(R"( ---!nonstrict -local l0:any,l61:t0 = _,math -while _ do -_() -end -function _():t0 -end -type t0 = any -)"); - - std::optional ty = requireType("math"); - REQUIRE(ty); - - const TableTypeVar* ttv = get(*ty); - REQUIRE(ttv); - CHECK(ttv->instantiatedTypeParams.empty()); -} - -TEST_CASE_FIXTURE(Fixture, "instantiate_table_cloning_2") -{ - ScopedFastFlag sff{"LuauOnlyMutateInstantiatedTables", true}; - - CheckResult result = check(R"( -type X = T -type K = X -)"); - - LUAU_REQUIRE_NO_ERRORS(result); - - std::optional ty = requireType("math"); - REQUIRE(ty); - - const TableTypeVar* ttv = get(*ty); - REQUIRE(ttv); - CHECK(ttv->instantiatedTypeParams.empty()); -} - -TEST_CASE_FIXTURE(Fixture, "instantiate_table_cloning_3") -{ - ScopedFastFlag sff{"LuauOnlyMutateInstantiatedTables", true}; - - CheckResult result = check(R"( -type X = T -local a = {} -a.x = 4 -local b: X -a.y = 5 -local c: X -c = b -)"); - - LUAU_REQUIRE_NO_ERRORS(result); - - std::optional ty = requireType("a"); - REQUIRE(ty); - - const TableTypeVar* ttv = get(*ty); - REQUIRE(ttv); - CHECK(ttv->instantiatedTypeParams.empty()); -} - -TEST_CASE_FIXTURE(Fixture, "bound_free_table_export_is_ok") -{ - CheckResult result = check(R"( -local n = {} -function n:Clone() end - -local m = {} - -function m.a(x) - x:Clone() -end - -function m.b() - m.a(n) -end - -return m -)"); - LUAU_REQUIRE_NO_ERRORS(result); -} - -TEST_CASE_FIXTURE(Fixture, "self_recursive_instantiated_param") -{ - ScopedFastFlag sff{"LuauOnlyMutateInstantiatedTables", true}; - - // Mutability in type function application right now can create strange recursive types - CheckResult result = check(R"( -type Table = { a: number } -type Self = T -local a: Self
- )"); - - LUAU_REQUIRE_NO_ERRORS(result); - CHECK_EQ(toString(requireType("a")), "Table"); -} - -TEST_CASE_FIXTURE(Fixture, "no_persistent_typelevel_change") -{ - TypeId mathTy = requireType(typeChecker.globalScope, "math"); - REQUIRE(mathTy); - TableTypeVar* ttv = getMutable(mathTy); - REQUIRE(ttv); - const FunctionTypeVar* ftv = get(ttv->props["frexp"].type); - REQUIRE(ftv); - auto original = ftv->level; - - CheckResult result = check("local a = math.frexp"); - - LUAU_REQUIRE_NO_ERRORS(result); - CHECK(ftv->level.level == original.level); - CHECK(ftv->level.subLevel == original.subLevel); -} - -TEST_CASE_FIXTURE(Fixture, "table_indexing_error_location") -{ - CheckResult result = check(R"( -local foo = {42} -local bar: number? -local baz = foo[bar] - )"); - - LUAU_REQUIRE_ERROR_COUNT(1, result); - - CHECK_EQ(result.errors[0].location, Location{Position{3, 16}, Position{3, 19}}); -} - -TEST_CASE_FIXTURE(Fixture, "table_simple_call") -{ - CheckResult result = check(R"( -local a = setmetatable({ x = 2 }, { - __call = function(self) - return (self.x :: number) * 2 -- should work without annotation in the future - end -}) -local b = a() -local c = a(2) -- too many arguments - )"); - - LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK_EQ("Argument count mismatch. Function expects 1 argument, but 2 are specified", toString(result.errors[0])); -} - -TEST_CASE_FIXTURE(Fixture, "custom_require_global") -{ - CheckResult result = check(R"( ---!nonstrict -require = function(a) end - -local crash = require(game.A) - )"); - - LUAU_REQUIRE_NO_ERRORS(result); -} - -TEST_CASE_FIXTURE(Fixture, "operator_eq_operands_are_not_subtypes_of_each_other_but_has_overlap") -{ - ScopedFastFlag sff1{"LuauEqConstraint", true}; - - CheckResult result = check(R"( - local function f(a: string | number, b: boolean | number) - return a == b - end - )"); - - // This doesn't produce any errors but for the wrong reasons. - // This unit test serves as a reminder to not try and unify the operands on `==`/`~=`. - LUAU_REQUIRE_NO_ERRORS(result); -} - -TEST_CASE_FIXTURE(Fixture, "access_index_metamethod_that_returns_variadic") -{ - CheckResult result = check(R"( - type Foo = {x: string} - local t = {} - setmetatable(t, { - __index = function(x: string): ...Foo - return {x = x} - end - }) - - local foo = t.bar - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - - ToStringOptions o; - o.exhaustive = true; - CHECK_EQ("{| x: string |}", toString(requireType("foo"), o)); -} - -TEST_CASE_FIXTURE(Fixture, "detect_cyclic_typepacks") -{ - CheckResult result = check(R"( - type ( ... ) ( ) ; - ( ... ) ( - - ... ) ( - ... ) - type = ( ... ) ; - ( ... ) ( ) ( ... ) ; - ( ... ) "" - )"); - - CHECK_LE(0, result.errors.size()); -} - -TEST_CASE_FIXTURE(Fixture, "detect_cyclic_typepacks2") -{ - CheckResult result = check(R"( - function _(l0:((typeof((pcall)))|((((t0)->())|(typeof(-67108864)))|(any)))|(any),...):(((typeof(0))|(any))|(any),typeof(-67108864),any) - xpcall(_,_,_) - _(_,_,_) - end - )"); - - CHECK_LE(0, result.errors.size()); -} - -TEST_CASE_FIXTURE(Fixture, "no_stack_overflow_from_quantifying") -{ - CheckResult result = check(R"( - function _(l0:t0): (any, ()->()) - end - - type t0 = t0 | {} - )"); - - CHECK_LE(0, result.errors.size()); - - std::optional t0 = getMainModule()->getModuleScope()->lookupType("t0"); - REQUIRE(t0); - CHECK_EQ("*unknown*", toString(t0->type)); - - auto it = std::find_if(result.errors.begin(), result.errors.end(), [](TypeError& err) { - return get(err); - }); - CHECK(it != result.errors.end()); -} - TEST_CASE_FIXTURE(Fixture, "no_stack_overflow_from_isoptional") { CheckResult result = check(R"( @@ -4316,22 +713,6 @@ TEST_CASE_FIXTURE(Fixture, "no_infinite_loop_when_trying_to_unify_uh_this") CHECK_LE(0, result.errors.size()); } -TEST_CASE_FIXTURE(Fixture, "no_stack_overflow_from_flattenintersection") -{ - CheckResult result = check(R"( - local l0,l0 - repeat - type t0 = ((any)|((any)&((any)|((any)&((any)|(any))))))&(t0) - function _(l0):(t0)&(t0) - while nil do - end - end - until _(_)(_)._ - )"); - - CHECK_LE(0, result.errors.size()); -} - TEST_CASE_FIXTURE(Fixture, "no_heap_use_after_free_error") { CheckResult result = check(R"( @@ -4349,329 +730,6 @@ TEST_CASE_FIXTURE(Fixture, "no_heap_use_after_free_error") CHECK_LE(0, result.errors.size()); } -TEST_CASE_FIXTURE(Fixture, "dont_invalidate_the_properties_iterator_of_free_table_when_rolled_back") -{ - fileResolver.source["Module/Backend/Types"] = R"( - export type Fiber = { - return_: Fiber? - } - return {} - )"; - - fileResolver.source["Module/Backend"] = R"( - local Types = require(script.Types) - type Fiber = Types.Fiber - type ReactRenderer = { findFiberByHostInstance: () -> Fiber? } - - local function attach(renderer): () - local function getPrimaryFiber(fiber) - local alternate = fiber.alternate - return fiber - end - - local function getFiberIDForNative() - local fiber = renderer.findFiberByHostInstance() - fiber = fiber.return_ - return getPrimaryFiber(fiber) - end - end - - function culprit(renderer: ReactRenderer): () - attach(renderer) - end - - return culprit - )"; - - CheckResult result = frontend.check("Module/Backend"); -} - -TEST_CASE_FIXTURE(Fixture, "recursive_types_restriction_ok") -{ - CheckResult result = check(R"( - type Tree = { data: T, children: {Tree} } - )"); - - LUAU_REQUIRE_NO_ERRORS(result); -} - -TEST_CASE_FIXTURE(Fixture, "recursive_types_restriction_not_ok") -{ - ScopedFastFlag sff{"LuauRecursiveTypeParameterRestriction", true}; - - CheckResult result = check(R"( - -- this would be an infinite type if we allowed it - type Tree = { data: T, children: {Tree<{T}>} } - )"); - - LUAU_REQUIRE_ERRORS(result); -} - -TEST_CASE_FIXTURE(Fixture, "record_matching_overload") -{ - CheckResult result = check(R"( - type Overload = ((string) -> string) & ((number) -> number) - local abc: Overload - abc(1) - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - - // AstExprCall is the node that has the overload stored on it. - // findTypeAtPosition will look at the AstExprLocal, but this is not what - // we want to look at. - std::vector ancestry = findAstAncestryOfPosition(*getMainSourceModule(), Position(3, 10)); - REQUIRE_GE(ancestry.size(), 2); - AstExpr* parentExpr = ancestry[ancestry.size() - 2]->asExpr(); - REQUIRE(bool(parentExpr)); - REQUIRE(parentExpr->is()); - - ModulePtr module = getMainModule(); - auto it = module->astOverloadResolvedTypes.find(parentExpr); - REQUIRE(it); - CHECK_EQ(toString(*it), "(number) -> number"); -} - -TEST_CASE_FIXTURE(Fixture, "return_type_by_overload") -{ - ScopedFastFlag sff{"LuauErrorRecoveryType", true}; - - CheckResult result = check(R"( - type Overload = ((string) -> string) & ((number, number) -> number) - local abc: Overload - local x = abc(true) - local y = abc(true,true) - local z = abc(true,true,true) - )"); - - LUAU_REQUIRE_ERRORS(result); - CHECK_EQ("string", toString(requireType("x"))); - CHECK_EQ("number", toString(requireType("y"))); - // Should this be string|number? - CHECK_EQ("string", toString(requireType("z"))); -} - -TEST_CASE_FIXTURE(Fixture, "infer_anonymous_function_arguments") -{ - // Simple direct arg to arg propagation - CheckResult result = check(R"( -type Table = { x: number, y: number } -local function f(a: (Table) -> number) return a({x = 1, y = 2}) end -f(function(a) return a.x + a.y end) - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - - // An optional function is accepted, but since we already provide a function, nil can be ignored - result = check(R"( -type Table = { x: number, y: number } -local function f(a: ((Table) -> number)?) if a then return a({x = 1, y = 2}) else return 0 end end -f(function(a) return a.x + a.y end) - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - - // Make sure self calls match correct index - result = check(R"( -type Table = { x: number, y: number } -local x = {} -x.b = {x = 1, y = 2} -function x:f(a: (Table) -> number) return a(self.b) end -x:f(function(a) return a.x + a.y end) - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - - // Mix inferred and explicit argument types - result = check(R"( -function f(a: (a: number, b: number, c: boolean) -> number) return a(1, 2, true) end -f(function(a: number, b, c) return c and a + b or b - a end) - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - - // Anonymous function has a variadic pack - result = check(R"( -type Table = { x: number, y: number } -local function f(a: (Table) -> number) return a({x = 1, y = 2}) end -f(function(...) return select(1, ...).z end) - )"); - - LUAU_REQUIRE_ERRORS(result); - CHECK_EQ("Key 'z' not found in table 'Table'", toString(result.errors[0])); - - // Can't accept more arguments than provided - result = check(R"( -function f(a: (a: number, b: number) -> number) return a(1, 2) end -f(function(a, b, c, ...) return a + b end) - )"); - - LUAU_REQUIRE_ERRORS(result); - - CHECK_EQ(R"(Type '(number, number, a) -> number' could not be converted into '(number, number) -> number' -caused by: - Argument count mismatch. Function expects 3 arguments, but only 2 are specified)", - toString(result.errors[0])); - - // Infer from variadic packs into elements - result = check(R"( -function f(a: (...number) -> number) return a(1, 2) end -f(function(a, b) return a + b end) - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - - // Infer from variadic packs into variadic packs - result = check(R"( -type Table = { x: number, y: number } -function f(a: (...Table) -> number) return a({x = 1, y = 2}, {x = 3, y = 4}) end -f(function(a, ...) local b = ... return b.z end) - )"); - - LUAU_REQUIRE_ERRORS(result); - CHECK_EQ("Key 'z' not found in table 'Table'", toString(result.errors[0])); - - // Return type inference - result = check(R"( -type Table = { x: number, y: number } -function f(a: (number) -> Table) return a(4) end -f(function(x) return x * 2 end) - )"); - - LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK_EQ("Type 'number' could not be converted into 'Table'", toString(result.errors[0])); - - // Return type doesn't inference 'nil' - result = check(R"( -function f(a: (number) -> nil) return a(4) end -f(function(x) print(x) end) - )"); - - LUAU_REQUIRE_NO_ERRORS(result); -} - -TEST_CASE_FIXTURE(Fixture, "infer_generic_function_function_argument") -{ - ScopedFastFlag sff{"LuauUnsealedTableLiteral", true}; - - CheckResult result = check(R"( -local function sum(x: a, y: a, f: (a, a) -> a) return f(x, y) end -return sum(2, 3, function(a, b) return a + b end) - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - - result = check(R"( -local function map(arr: {a}, f: (a) -> b) local r = {} for i,v in ipairs(arr) do table.insert(r, f(v)) end return r end -local a = {1, 2, 3} -local r = map(a, function(a) return a + a > 100 end) - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - REQUIRE_EQ("{boolean}", toString(requireType("r"))); - - check(R"( -local function foldl(arr: {a}, init: b, f: (b, a) -> b) local r = init for i,v in ipairs(arr) do r = f(r, v) end return r end -local a = {1, 2, 3} -local r = foldl(a, {s=0,c=0}, function(a, b) return {s = a.s + b, c = a.c + 1} end) - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - REQUIRE_EQ("{ c: number, s: number }", toString(requireType("r"))); -} - -TEST_CASE_FIXTURE(Fixture, "infer_generic_function_function_argument_overloaded") -{ - CheckResult result = check(R"( -local function g1(a: T, f: (T) -> T) return f(a) end -local function g2(a: T, b: T, f: (T, T) -> T) return f(a, b) end - -local g12: typeof(g1) & typeof(g2) - -g12(1, function(x) return x + x end) -g12(1, 2, function(x, y) return x + y end) - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - - result = check(R"( -local function g1(a: T, f: (T) -> T) return f(a) end -local function g2(a: T, b: T, f: (T, T) -> T) return f(a, b) end - -local g12: typeof(g1) & typeof(g2) - -g12({x=1}, function(x) return {x=-x.x} end) -g12({x=1}, {x=2}, function(x, y) return {x=x.x + y.x} end) - )"); - - LUAU_REQUIRE_NO_ERRORS(result); -} - -TEST_CASE_FIXTURE(Fixture, "infer_generic_lib_function_function_argument") -{ - CheckResult result = check(R"( -local a = {{x=4}, {x=7}, {x=1}} -table.sort(a, function(x, y) return x.x < y.x end) - )"); - - LUAU_REQUIRE_NO_ERRORS(result); -} - -TEST_CASE_FIXTURE(Fixture, "infer_anonymous_function_arguments_outside_call") -{ - CheckResult result = check(R"( -type Table = { x: number, y: number } -local f: (Table) -> number = function(t) return t.x + t.y end - -type TableWithFunc = { x: number, y: number, f: (number, number) -> number } -local a: TableWithFunc = { x = 3, y = 4, f = function(a, b) return a + b end } - )"); - - LUAU_REQUIRE_NO_ERRORS(result); -} - -TEST_CASE_FIXTURE(Fixture, "do_not_infer_generic_functions") -{ - CheckResult result = check(R"( -local function sum(x: a, y: a, f: (a, a) -> a) return f(x, y) end - -local function sumrec(f: typeof(sum)) - return sum(2, 3, function(a, b) return a + b end) -end - -local b = sumrec(sum) -- ok -local c = sumrec(function(x, y, f) return f(x, y) end) -- type binders are not inferred - )"); - - LUAU_REQUIRE_ERRORS(result); - CHECK_EQ("Type '(a, b, (a, b) -> (c...)) -> (c...)' could not be converted into '(a, a, (a, a) -> a) -> a'; different number of generic type " - "parameters", - toString(result.errors[0])); -} - -TEST_CASE_FIXTURE(Fixture, "infer_return_value_type") -{ - CheckResult result = check(R"( -local function f(): {string|number} - return {1, "b", 3} -end - -local function g(): (number, {string|number}) - return 4, {1, "b", 3} -end - -local function h(): ...{string|number} - return {4}, {1, "b", 3}, {"s"} -end - -local function i(): ...{string|number} - return {1, "b", 3}, h() -end - )"); - - LUAU_REQUIRE_NO_ERRORS(result); -} - TEST_CASE_FIXTURE(Fixture, "infer_type_assertion_value_type") { CheckResult result = check(R"( @@ -4719,56 +777,6 @@ f(((function(a, b) return a + b end))) LUAU_REQUIRE_NO_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "refine_and_or") -{ - CheckResult result = check(R"( - local t: {x: number?}? = {x = nil} - local u = t and t.x or 5 - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - - CHECK_EQ("number", toString(requireType("u"))); -} - -TEST_CASE_FIXTURE(Fixture, "checked_prop_too_early") -{ - CheckResult result = check(R"( - local t: {x: number?}? = {x = nil} - local u = t.x and t or 5 - )"); - - LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK_EQ("Value of type '{| x: number? |}?' could be nil", toString(result.errors[0])); - CHECK_EQ("number | {| x: number? |}", toString(requireType("u"))); -} - -TEST_CASE_FIXTURE(Fixture, "accidentally_checked_prop_in_opposite_branch") -{ - CheckResult result = check(R"( - local t: {x: number?}? = {x = nil} - local u = t and t.x == 5 or t.x == 31337 - )"); - - LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK_EQ("Value of type '{| x: number? |}?' could be nil", toString(result.errors[0])); - CHECK_EQ("boolean", toString(requireType("u"))); -} - -TEST_CASE_FIXTURE(Fixture, "substitution_with_bound_table") -{ - CheckResult result = check(R"( -type A = { x: number } -local a: A = { x = 1 } -local b = a -type B = typeof(b) -type X = T -local c: X - )"); - - LUAU_REQUIRE_NO_ERRORS(result); -} - TEST_CASE_FIXTURE(Fixture, "tc_if_else_expressions1") { CheckResult result = check(R"(local a = if true then "true" else "false")"); @@ -4830,40 +838,6 @@ end LUAU_REQUIRE_NO_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "type_error_addition") -{ - CheckResult result = check(R"( ---!strict -local foo = makesandwich() -local bar = foo.nutrition + 100 - )"); - - LUAU_REQUIRE_ERROR_COUNT(1, result); - - // We should definitely get this error - CHECK_EQ("Unknown global 'makesandwich'", toString(result.errors[0])); - // We get this error if makesandwich() returns a free type - // CHECK_EQ("Unknown type used in + operation; consider adding a type annotation to 'foo'", toString(result.errors[1])); -} - -TEST_CASE_FIXTURE(Fixture, "require_failed_module") -{ - fileResolver.source["game/A"] = R"( -return unfortunately() - )"; - - CheckResult aResult = frontend.check("game/A"); - LUAU_REQUIRE_ERRORS(aResult); - - CheckResult result = check(R"( -local ModuleA = require(game.A) - )"); - LUAU_REQUIRE_NO_ERRORS(result); - - std::optional oty = requireType("ModuleA"); - CHECK_EQ("*unknown*", toString(*oty)); -} - /* * If it wasn't instantly obvious, we have the fuzzer to thank for this gem of a test. * @@ -4921,184 +895,6 @@ TEST_CASE_FIXTURE(Fixture, "fuzzer_found_this") )"); } -/* - * We had an issue where part of the type of pairs() was an unsealed table. - * This test depends on FFlagDebugLuauFreezeArena to trigger it. - */ -TEST_CASE_FIXTURE(Fixture, "pairs_parameters_are_not_unsealed_tables") -{ - check(R"( - function _(l0:{n0:any}) - _ = pairs - end - )"); -} - -TEST_CASE_FIXTURE(Fixture, "inferred_methods_of_free_tables_have_the_same_level_as_the_enclosing_table") -{ - check(R"( - function Base64FileReader(data) - local reader = {} - local index: number - - function reader:PeekByte() - return data:byte(index) - end - - function reader:Byte() - return data:byte(index - 1) - end - - return reader - end - - Base64FileReader() - - function ReadMidiEvents(data) - - local reader = Base64FileReader(data) - - while reader:HasMore() do - (reader:Byte() % 128) - end - end - )"); -} - -TEST_CASE_FIXTURE(Fixture, "error_detailed_function_mismatch_arg_count") -{ - CheckResult result = check(R"( -type A = (number, number) -> string -type B = (number) -> string - -local a: A -local b: B = a - )"); - - LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK_EQ(toString(result.errors[0]), R"(Type '(number, number) -> string' could not be converted into '(number) -> string' -caused by: - Argument count mismatch. Function expects 2 arguments, but only 1 is specified)"); -} - -TEST_CASE_FIXTURE(Fixture, "error_detailed_function_mismatch_arg") -{ - CheckResult result = check(R"( -type A = (number, number) -> string -type B = (number, string) -> string - -local a: A -local b: B = a - )"); - - LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK_EQ(toString(result.errors[0]), R"(Type '(number, number) -> string' could not be converted into '(number, string) -> string' -caused by: - Argument #2 type is not compatible. Type 'string' could not be converted into 'number')"); -} - -TEST_CASE_FIXTURE(Fixture, "error_detailed_function_mismatch_ret_count") -{ - CheckResult result = check(R"( -type A = (number, number) -> (number) -type B = (number, number) -> (number, boolean) - -local a: A -local b: B = a - )"); - - LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK_EQ(toString(result.errors[0]), R"(Type '(number, number) -> number' could not be converted into '(number, number) -> (number, boolean)' -caused by: - Function only returns 1 value. 2 are required here)"); -} - -TEST_CASE_FIXTURE(Fixture, "error_detailed_function_mismatch_ret") -{ - CheckResult result = check(R"( -type A = (number, number) -> string -type B = (number, number) -> number - -local a: A -local b: B = a - )"); - - LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK_EQ(toString(result.errors[0]), R"(Type '(number, number) -> string' could not be converted into '(number, number) -> number' -caused by: - Return type is not compatible. Type 'string' could not be converted into 'number')"); -} - -TEST_CASE_FIXTURE(Fixture, "error_detailed_function_mismatch_ret_mult") -{ - CheckResult result = check(R"( -type A = (number, number) -> (number, string) -type B = (number, number) -> (number, boolean) - -local a: A -local b: B = a - )"); - - LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK_EQ(toString(result.errors[0]), - R"(Type '(number, number) -> (number, string)' could not be converted into '(number, number) -> (number, boolean)' -caused by: - Return #2 type is not compatible. Type 'string' could not be converted into 'boolean')"); -} - -TEST_CASE_FIXTURE(Fixture, "prop_access_on_any_with_other_options") -{ - CheckResult result = check(R"( - local function f(thing: any | string) - local foo = thing.SomeRandomKey - end - )"); - - LUAU_REQUIRE_NO_ERRORS(result); -} - -TEST_CASE_FIXTURE(Fixture, "table_function_check_use_after_free") -{ - CheckResult result = check(R"( -local t = {} - -function t.x(value) - for k,v in pairs(t) do end -end - )"); - - LUAU_REQUIRE_NO_ERRORS(result); -} - -TEST_CASE_FIXTURE(Fixture, "table_oop") -{ - CheckResult result = check(R"( - --!strict -local Class = {} -Class.__index = Class - -type Class = typeof(setmetatable({} :: { x: number }, Class)) - -function Class.new(x: number): Class - return setmetatable({x = x}, Class) -end - -function Class.getx(self: Class) - return self.x -end - -function test() - local c = Class.new(42) - local n = c:getx() - local nn = c.x - - print(string.format("%d %d", n, nn)) -end -)"); - - LUAU_REQUIRE_NO_ERRORS(result); -} - TEST_CASE_FIXTURE(Fixture, "recursive_metatable_crash") { CheckResult result = check(R"( @@ -5182,213 +978,4 @@ TEST_CASE_FIXTURE(Fixture, "cli_50041_committing_txnlog_in_apollo_client_error") LUAU_REQUIRE_NO_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "do_not_modify_imported_types") -{ - fileResolver.source["game/A"] = R"( -export type Type = { unrelated: boolean } -return {} - )"; - - fileResolver.source["game/B"] = R"( -local types = require(game.A) -type Type = types.Type -local x: Type = {} -function x:Destroy(): () end - )"; - - CheckResult result = frontend.check("game/B"); - LUAU_REQUIRE_ERRORS(result); -} - -TEST_CASE_FIXTURE(Fixture, "do_not_modify_imported_types_2") -{ - ScopedFastFlag immutableTypes{"LuauImmutableTypes", true}; - - fileResolver.source["game/A"] = R"( -export type Type = { x: { a: number } } -return {} - )"; - - fileResolver.source["game/B"] = R"( -local types = require(game.A) -type Type = types.Type -local x: Type = { x = { a = 2 } } -type Rename = typeof(x.x) - )"; - - CheckResult result = frontend.check("game/B"); - LUAU_REQUIRE_NO_ERRORS(result); -} - -TEST_CASE_FIXTURE(Fixture, "do_not_modify_imported_types_3") -{ - ScopedFastFlag immutableTypes{"LuauImmutableTypes", true}; - - fileResolver.source["game/A"] = R"( -local y = setmetatable({}, {}) -export type Type = { x: typeof(y) } -return { x = y } - )"; - - fileResolver.source["game/B"] = R"( -local types = require(game.A) -type Type = types.Type -local x: Type = types -type Rename = typeof(x.x) - )"; - - CheckResult result = frontend.check("game/B"); - LUAU_REQUIRE_NO_ERRORS(result); -} - -TEST_CASE_FIXTURE(Fixture, "indexing_on_string_singletons") -{ - ScopedFastFlag sff[]{ - {"LuauDiscriminableUnions2", true}, - {"LuauSingletonTypes", true}, - }; - - CheckResult result = check(R"( - local a: string = "hi" - if a == "hi" then - local x = a:byte() - end - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - - CHECK_EQ(R"("hi")", toString(requireTypeAtPosition({3, 22}))); -} - -TEST_CASE_FIXTURE(Fixture, "indexing_on_union_of_string_singletons") -{ - ScopedFastFlag sff[]{ - {"LuauDiscriminableUnions2", true}, - {"LuauSingletonTypes", true}, - }; - - CheckResult result = check(R"( - local a: string = "hi" - if a == "hi" or a == "bye" then - local x = a:byte() - end - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - - CHECK_EQ(R"("bye" | "hi")", toString(requireTypeAtPosition({3, 22}))); -} - -TEST_CASE_FIXTURE(Fixture, "taking_the_length_of_string_singleton") -{ - ScopedFastFlag sff[]{ - {"LuauDiscriminableUnions2", true}, - {"LuauSingletonTypes", true}, - }; - - CheckResult result = check(R"( - local a: string = "hi" - if a == "hi" then - local x = #a - end - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - - CHECK_EQ(R"("hi")", toString(requireTypeAtPosition({3, 23}))); -} - -TEST_CASE_FIXTURE(Fixture, "taking_the_length_of_union_of_string_singleton") -{ - ScopedFastFlag sff[]{ - {"LuauDiscriminableUnions2", true}, - {"LuauSingletonTypes", true}, - }; - - CheckResult result = check(R"( - local a: string = "hi" - if a == "hi" or a == "bye" then - local x = #a - end - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - - CHECK_EQ(R"("bye" | "hi")", toString(requireTypeAtPosition({3, 23}))); -} - -/* - * When we add new properties to an unsealed table, we should do a level check and promote the property type to be at - * the level of the table. - */ -TEST_CASE_FIXTURE(Fixture, "inferred_properties_of_a_table_should_start_with_the_same_TypeLevel_of_that_table") -{ - CheckResult result = check(R"( - --!strict - local T = {} - - local function f(prop) - T[1] = { - prop = prop, - } - end - - local function g() - local l = T[1].prop - end - )"); - - LUAU_REQUIRE_NO_ERRORS(result); -} - -TEST_CASE_FIXTURE(Fixture, "global_singleton_types_are_sealed") -{ - CheckResult result = check(R"( -local function f(x: string) - local p = x:split('a') - p = table.pack(table.unpack(p, 1, #p - 1)) - return p -end - )"); - - LUAU_REQUIRE_NO_ERRORS(result); -} - -TEST_CASE_FIXTURE(Fixture, "function_decl_quantify_right_type") -{ - ScopedFastFlag statFunctionSimplify{"LuauStatFunctionSimplify", true}; - - fileResolver.source["game/isAMagicMock"] = R"( ---!nonstrict -return function(value) - return false -end - )"; - - CheckResult result = check(R"( ---!nonstrict -local MagicMock = {} -MagicMock.is = require(game.isAMagicMock) - -function MagicMock.is(value) - return false -end - )"); - - LUAU_REQUIRE_NO_ERRORS(result); -} - -TEST_CASE_FIXTURE(Fixture, "function_decl_non_self_sealed_overwrite") -{ - ScopedFastFlag statFunctionSimplify{"LuauStatFunctionSimplify", true}; - - CheckResult result = check(R"( -function string.len(): number - return 1 -end - )"); - - LUAU_REQUIRE_ERRORS(result); -} - TEST_SUITE_END(); diff --git a/tests/TypeInfer.typePacks.cpp b/tests/TypeInfer.typePacks.cpp index fcc21c18..130f33d7 100644 --- a/tests/TypeInfer.typePacks.cpp +++ b/tests/TypeInfer.typePacks.cpp @@ -895,4 +895,87 @@ caused by: Type 'boolean' could not be converted into 'string')"); } +// TODO: File a Jira about this +/* +TEST_CASE_FIXTURE(Fixture, "unifying_vararg_pack_with_fixed_length_pack_produces_fixed_length_pack") +{ + CheckResult result = check(R"( + function a(x) return 1 end + a(...) + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + REQUIRE(bool(getMainModule()->getModuleScope()->varargPack)); + + TypePackId varargPack = *getMainModule()->getModuleScope()->varargPack; + + auto iter = begin(varargPack); + auto endIter = end(varargPack); + + CHECK(iter != endIter); + ++iter; + CHECK(iter == endIter); + + CHECK(!iter.tail()); +} +*/ + +TEST_CASE_FIXTURE(Fixture, "dont_ice_if_a_TypePack_is_an_error") +{ + CheckResult result = check(R"( + --!strict + function f(s) + print(s) + return f + end + + f("foo")("bar") + )"); +} + +TEST_CASE_FIXTURE(Fixture, "cyclic_type_packs") +{ + // this has a risk of creating cyclic type packs, causing infinite loops / OOMs + check(R"( +--!nonstrict +_ += _(_,...) +repeat +_ += _(...) +until ... + _ +)"); + + check(R"( +--!nonstrict +_ += _(_(...,...),_(...)) +repeat +until _ +)"); +} + +TEST_CASE_FIXTURE(Fixture, "detect_cyclic_typepacks") +{ + CheckResult result = check(R"( + type ( ... ) ( ) ; + ( ... ) ( - - ... ) ( - ... ) + type = ( ... ) ; + ( ... ) ( ) ( ... ) ; + ( ... ) "" + )"); + + CHECK_LE(0, result.errors.size()); +} + +TEST_CASE_FIXTURE(Fixture, "detect_cyclic_typepacks2") +{ + CheckResult result = check(R"( + function _(l0:((typeof((pcall)))|((((t0)->())|(typeof(-67108864)))|(any)))|(any),...):(((typeof(0))|(any))|(any),typeof(-67108864),any) + xpcall(_,_,_) + _(_,_,_) + end + )"); + + CHECK_LE(0, result.errors.size()); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.unionTypes.test.cpp b/tests/TypeInfer.unionTypes.test.cpp index ad4cecd8..ad1e31e5 100644 --- a/tests/TypeInfer.unionTypes.test.cpp +++ b/tests/TypeInfer.unionTypes.test.cpp @@ -496,4 +496,20 @@ caused by: None of the union options are compatible. For example: Table type 'a' not compatible with type 'X' because the former is missing field 'x')"); } +// We had a bug where a cyclic union caused a stack overflow. +// ex type U = number | U +TEST_CASE_FIXTURE(Fixture, "dont_allow_cyclic_unions_to_be_inferred") +{ + CheckResult result = check(R"( + --!strict + + function f(a, b) + a:g(b or {}) + a:g(b) + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + TEST_SUITE_END(); diff --git a/tests/conformance/basic.lua b/tests/conformance/basic.lua index 78d90077..f803c319 100644 --- a/tests/conformance/basic.lua +++ b/tests/conformance/basic.lua @@ -833,6 +833,17 @@ assert((function() return sum end)() == 105) +-- shrinking array part +assert((function() + local t = table.create(100, 42) + for i=1,90 do t[i] = nil end + t[101] = 42 + local sum = 0 + for _,v in ipairs(t) do sum += v end + for _,v in pairs(t) do sum += v end + return sum +end)() == 462) + -- upvalues: recursive capture assert((function() local function fact(n) return n < 1 and 1 or n * fact(n-1) end return fact(5) end)() == 120) @@ -881,6 +892,14 @@ end)() == "6,8,10") -- typeof == type in absence of custom userdata assert(concat(typeof(5), typeof(nil), typeof({}), typeof(newproxy())) == "number,nil,table,userdata") +-- type/typeof/newproxy interaction with metatables: __type doesn't work intentionally to avoid spoofing +assert((function() + local ud = newproxy(true) + getmetatable(ud).__type = "number" + + return concat(type(ud),typeof(ud)) +end)() == "userdata,userdata") + testgetfenv() -- DONT MOVE THIS LINE return 'OK' diff --git a/tests/conformance/debugger.lua b/tests/conformance/debugger.lua index 6ba99fb9..ec0b412e 100644 --- a/tests/conformance/debugger.lua +++ b/tests/conformance/debugger.lua @@ -3,14 +3,14 @@ print "testing debugger" -- note, this file can't run in isolation from C tests local a = 5 -function foo(b) +function foo(b, ...) print("in foo", b) a = 6 end breakpoint(8) -foo(50) +foo(50, 42) breakpoint(16) -- next line print("here") diff --git a/tests/conformance/errors.lua b/tests/conformance/errors.lua index 751188be..297cf011 100644 --- a/tests/conformance/errors.lua +++ b/tests/conformance/errors.lua @@ -305,4 +305,6 @@ assert(ecall(function() return "a" + "b" end) == "attempt to perform arithmetic assert(ecall(function() return 1 > nil end) == "attempt to compare nil < number") -- note reversed order (by design) assert(ecall(function() return "a" <= 5 end) == "attempt to compare string <= number") +assert(ecall(function() local t = {} setmetatable(t, { __newindex = function(t,i,v) end }) t[nil] = 2 end) == "table index is nil") + return('OK') diff --git a/tests/conformance/interrupt.lua b/tests/conformance/interrupt.lua new file mode 100644 index 00000000..2b127099 --- /dev/null +++ b/tests/conformance/interrupt.lua @@ -0,0 +1,11 @@ +-- This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +print("testing interrupts") + +function foo() + for i=1,10 do end + return +end + +foo() + +return "OK" diff --git a/tools/gdb-printers.py b/tools/gdb_printers.py similarity index 100% rename from tools/gdb-printers.py rename to tools/gdb_printers.py diff --git a/tools/lldb-formatters.lldb b/tools/lldb-formatters.lldb deleted file mode 100644 index 3868ac20..00000000 --- a/tools/lldb-formatters.lldb +++ /dev/null @@ -1,2 +0,0 @@ -type synthetic add -x "^Luau::Variant<.+>$" -l LuauVisualize.LuauVariantSyntheticChildrenProvider -type summary add -x "^Luau::Variant<.+>$" -l LuauVisualize.luau_variant_summary diff --git a/tools/lldb_formatters.lldb b/tools/lldb_formatters.lldb new file mode 100644 index 00000000..f6fa6cf5 --- /dev/null +++ b/tools/lldb_formatters.lldb @@ -0,0 +1,2 @@ +type synthetic add -x "^Luau::Variant<.+>$" -l lldb_formatters.LuauVariantSyntheticChildrenProvider +type summary add -x "^Luau::Variant<.+>$" -l lldb_formatters.luau_variant_summary diff --git a/tools/LuauVisualize.py b/tools/lldb_formatters.py similarity index 100% rename from tools/LuauVisualize.py rename to tools/lldb_formatters.py From 7721955ba54295696064a05a969b2a4fb29f43af Mon Sep 17 00:00:00 2001 From: Alan Jeffrey <403333+asajeffrey@users.noreply.github.com> Date: Wed, 23 Mar 2022 15:02:57 -0500 Subject: [PATCH 198/399] Prototyping: add semantic subtyping (#424) Adds subtyping to strict mode. --- prototyping/FFI/Data/Aeson.agda | 4 +- prototyping/FFI/Data/Either.agda | 24 +- prototyping/Luau/RuntimeError.agda | 2 +- prototyping/Luau/StrictMode.agda | 34 +- prototyping/Luau/StrictMode/ToString.agda | 39 +- prototyping/Luau/Subtyping.agda | 62 ++ prototyping/Luau/Type.agda | 7 + prototyping/Luau/TypeCheck.agda | 23 +- prototyping/Properties.agda | 2 + prototyping/Properties/Functions.agda | 6 + prototyping/Properties/StrictMode.agda | 594 +++++++----------- prototyping/Properties/Subtyping.agda | 221 +++++++ prototyping/Properties/TypeCheck.agda | 18 +- .../concat_number_and_string/out.txt | 1 + 14 files changed, 623 insertions(+), 414 deletions(-) create mode 100644 prototyping/Luau/Subtyping.agda create mode 100644 prototyping/Properties/Functions.agda create mode 100644 prototyping/Properties/Subtyping.agda diff --git a/prototyping/FFI/Data/Aeson.agda b/prototyping/FFI/Data/Aeson.agda index 0cd27232..43014719 100644 --- a/prototyping/FFI/Data/Aeson.agda +++ b/prototyping/FFI/Data/Aeson.agda @@ -10,7 +10,7 @@ open import Agda.Builtin.String using (String) open import FFI.Data.ByteString using (ByteString) open import FFI.Data.HaskellString using (HaskellString; pack) open import FFI.Data.Maybe using (Maybe; just; nothing) -open import FFI.Data.Either using (Either; mapLeft) +open import FFI.Data.Either using (Either; mapL) open import FFI.Data.Scientific using (Scientific) open import FFI.Data.Vector using (Vector) @@ -73,5 +73,5 @@ postulate {-# COMPILE GHC eitherHDecode = Data.Aeson.eitherDecodeStrict #-} eitherDecode : ByteString → Either String Value -eitherDecode bytes = mapLeft pack (eitherHDecode bytes) +eitherDecode bytes = mapL pack (eitherHDecode bytes) diff --git a/prototyping/FFI/Data/Either.agda b/prototyping/FFI/Data/Either.agda index d024c695..8a5d3e66 100644 --- a/prototyping/FFI/Data/Either.agda +++ b/prototyping/FFI/Data/Either.agda @@ -7,10 +7,22 @@ data Either (A B : Set) : Set where Right : B → Either A B {-# COMPILE GHC Either = data Data.Either.Either (Data.Either.Left|Data.Either.Right) #-} -mapLeft : ∀ {A B C} → (A → B) → (Either A C) → (Either B C) -mapLeft f (Left x) = Left (f x) -mapLeft f (Right x) = Right x +swapLR : ∀ {A B} → Either A B → Either B A +swapLR (Left x) = Right x +swapLR (Right x) = Left x -mapRight : ∀ {A B C} → (B → C) → (Either A B) → (Either A C) -mapRight f (Left x) = Left x -mapRight f (Right x) = Right (f x) +mapL : ∀ {A B C} → (A → B) → Either A C → Either B C +mapL f (Left x) = Left (f x) +mapL f (Right x) = Right x + +mapR : ∀ {A B C} → (B → C) → Either A B → Either A C +mapR f (Left x) = Left x +mapR f (Right x) = Right (f x) + +mapLR : ∀ {A B C D} → (A → B) → (C → D) → Either A C → Either B D +mapLR f g (Left x) = Left (f x) +mapLR f g (Right x) = Right (g x) + +cond : ∀ {A B C : Set} → (A → C) → (B → C) → (Either A B) → C +cond f g (Left x) = f x +cond f g (Right x) = g x diff --git a/prototyping/Luau/RuntimeError.agda b/prototyping/Luau/RuntimeError.agda index 0c8a594b..b9b305c9 100644 --- a/prototyping/Luau/RuntimeError.agda +++ b/prototyping/Luau/RuntimeError.agda @@ -25,7 +25,7 @@ data RuntimeErrorᴮ {a} (H : Heap a) : Block a → Set data RuntimeErrorᴱ {a} (H : Heap a) : Expr a → Set data RuntimeErrorᴱ H where - FunctionMismatch : ∀ v w → (function ≢ valueType v) → RuntimeErrorᴱ H (val v $ val w) + FunctionMismatch : ∀ v w → (valueType v ≢ function) → RuntimeErrorᴱ H (val v $ val w) BinOpMismatch₁ : ∀ v w {op} → (BinOpError op (valueType v)) → RuntimeErrorᴱ H (binexp (val v) op (val w)) BinOpMismatch₂ : ∀ v w {op} → (BinOpError op (valueType w)) → RuntimeErrorᴱ H (binexp (val v) op (val w)) UnboundVariable : ∀ {x} → RuntimeErrorᴱ H (var x) diff --git a/prototyping/Luau/StrictMode.agda b/prototyping/Luau/StrictMode.agda index 08e0f50b..0b5fe0da 100644 --- a/prototyping/Luau/StrictMode.agda +++ b/prototyping/Luau/StrictMode.agda @@ -5,28 +5,18 @@ module Luau.StrictMode where open import Agda.Builtin.Equality using (_≡_) open import FFI.Data.Maybe using (just; nothing) open import Luau.Syntax using (Expr; Stat; Block; BinaryOperator; yes; nil; addr; var; binexp; var_∈_; _⟨_⟩∈_; function_is_end; _$_; block_is_end; local_←_; _∙_; done; return; name; +; -; *; /; <; >; <=; >=; ··) -open import Luau.Type using (Type; strict; nil; number; string; _⇒_; tgt) +open import Luau.Type using (Type; strict; nil; number; string; boolean; none; any; _⇒_; _∪_; _∩_; tgt) +open import Luau.Subtyping using (_≮:_) open import Luau.Heap using (Heap; function_is_end) renaming (_[_] to _[_]ᴴ) open import Luau.VarCtxt using (VarCtxt; ∅; _⋒_; _↦_; _⊕_↦_; _⊝_) renaming (_[_] to _[_]ⱽ) -open import Luau.TypeCheck(strict) using (_⊢ᴮ_∈_; _⊢ᴱ_∈_; ⊢ᴴ_; ⊢ᴼ_; _⊢ᴴᴱ_▷_∈_; _⊢ᴴᴮ_▷_∈_; var; addr; app; binexp; block; return; local; function) -open import Properties.Equality using (_≢_) +open import Luau.TypeCheck(strict) using (_⊢ᴮ_∈_; _⊢ᴱ_∈_; ⊢ᴴ_; ⊢ᴼ_; _⊢ᴴᴱ_▷_∈_; _⊢ᴴᴮ_▷_∈_; var; addr; app; binexp; block; return; local; function; srcBinOp) +open import Properties.Contradiction using (¬) open import Properties.TypeCheck(strict) using (typeCheckᴮ) open import Properties.Product using (_,_) src : Type → Type src = Luau.Type.src strict -data BinOpWarning : BinaryOperator → Type → Set where - + : ∀ {T} → (T ≢ number) → BinOpWarning + T - - : ∀ {T} → (T ≢ number) → BinOpWarning - T - * : ∀ {T} → (T ≢ number) → BinOpWarning * T - / : ∀ {T} → (T ≢ number) → BinOpWarning / T - < : ∀ {T} → (T ≢ number) → BinOpWarning < T - > : ∀ {T} → (T ≢ number) → BinOpWarning > T - <= : ∀ {T} → (T ≢ number) → BinOpWarning <= T - >= : ∀ {T} → (T ≢ number) → BinOpWarning >= T - ·· : ∀ {T} → (T ≢ string) → BinOpWarning ·· T - data Warningᴱ (H : Heap yes) {Γ} : ∀ {M T} → (Γ ⊢ᴱ M ∈ T) → Set data Warningᴮ (H : Heap yes) {Γ} : ∀ {B T} → (Γ ⊢ᴮ B ∈ T) → Set @@ -46,7 +36,7 @@ data Warningᴱ H {Γ} where FunctionCallMismatch : ∀ {M N T U} {D₁ : Γ ⊢ᴱ M ∈ T} {D₂ : Γ ⊢ᴱ N ∈ U} → - (src T ≢ U) → + (U ≮: src T) → ----------------- Warningᴱ H (app D₁ D₂) @@ -64,13 +54,13 @@ data Warningᴱ H {Γ} where BinOpMismatch₁ : ∀ {op M N T U} {D₁ : Γ ⊢ᴱ M ∈ T} {D₂ : Γ ⊢ᴱ N ∈ U} → - BinOpWarning op T → + (T ≮: srcBinOp op) → ------------------------------ Warningᴱ H (binexp {op} D₁ D₂) BinOpMismatch₂ : ∀ {op M N T U} {D₁ : Γ ⊢ᴱ M ∈ T} {D₂ : Γ ⊢ᴱ N ∈ U} → - BinOpWarning op U → + (U ≮: srcBinOp op) → ------------------------------ Warningᴱ H (binexp {op} D₁ D₂) @@ -88,7 +78,7 @@ data Warningᴱ H {Γ} where FunctionDefnMismatch : ∀ {f x B T U V} {D : (Γ ⊕ x ↦ T) ⊢ᴮ B ∈ V} → - (U ≢ V) → + (V ≮: U) → ------------------------- Warningᴱ H (function {f} {U = U} D) @@ -100,7 +90,7 @@ data Warningᴱ H {Γ} where BlockMismatch : ∀ {b B T U} {D : Γ ⊢ᴮ B ∈ U} → - (T ≢ U) → + (U ≮: T) → ------------------------------ Warningᴱ H (block {b} {T = T} D) @@ -120,7 +110,7 @@ data Warningᴮ H {Γ} where LocalVarMismatch : ∀ {x M B T U V} {D₁ : Γ ⊢ᴱ M ∈ U} {D₂ : (Γ ⊕ x ↦ T) ⊢ᴮ B ∈ V} → - (T ≢ U) → + (U ≮: T) → -------------------- Warningᴮ H (local D₁ D₂) @@ -138,7 +128,7 @@ data Warningᴮ H {Γ} where FunctionDefnMismatch : ∀ {f x B C T U V W} {D₁ : (Γ ⊕ x ↦ T) ⊢ᴮ C ∈ V} {D₂ : (Γ ⊕ f ↦ (T ⇒ U)) ⊢ᴮ B ∈ W} → - (U ≢ V) → + (V ≮: U) → ------------------------------------- Warningᴮ H (function D₁ D₂) @@ -158,7 +148,7 @@ data Warningᴼ (H : Heap yes) : ∀ {V} → (⊢ᴼ V) → Set where FunctionDefnMismatch : ∀ {f x B T U V} {D : (x ↦ T) ⊢ᴮ B ∈ V} → - (U ≢ V) → + (V ≮: U) → --------------------------------- Warningᴼ H (function {f} {U = U} D) diff --git a/prototyping/Luau/StrictMode/ToString.agda b/prototyping/Luau/StrictMode/ToString.agda index ca55888c..08ee13b8 100644 --- a/prototyping/Luau/StrictMode/ToString.agda +++ b/prototyping/Luau/StrictMode/ToString.agda @@ -2,37 +2,58 @@ module Luau.StrictMode.ToString where +open import Agda.Builtin.Nat using (Nat; suc) open import FFI.Data.String using (String; _++_) +open import Luau.Subtyping using (_≮:_; Tree; witness; scalar; function; function-ok; function-err) open import Luau.StrictMode using (Warningᴱ; Warningᴮ; UnallocatedAddress; UnboundVariable; FunctionCallMismatch; FunctionDefnMismatch; BlockMismatch; app₁; app₂; BinOpMismatch₁; BinOpMismatch₂; bin₁; bin₂; block₁; return; LocalVarMismatch; local₁; local₂; function₁; function₂; heap; expr; block; addr) open import Luau.Syntax using (Expr; val; yes; var; var_∈_; _⟨_⟩∈_; _$_; addr; number; binexp; nil; function_is_end; block_is_end; done; return; local_←_; _∙_; fun; arg; name) -open import Luau.Type using (strict) +open import Luau.Type using (strict; number; boolean; string; nil) open import Luau.TypeCheck(strict) using (_⊢ᴮ_∈_; _⊢ᴱ_∈_) open import Luau.Addr.ToString using (addrToString) open import Luau.Var.ToString using (varToString) open import Luau.Type.ToString using (typeToString) open import Luau.Syntax.ToString using (binOpToString) +tmp : Nat → String +tmp 0 = "w" +tmp 1 = "x" +tmp 2 = "y" +tmp 3 = "z" +tmp (suc (suc (suc n))) = tmp n ++ "'" + +treeToString : Tree → Nat → String → String +treeToString (scalar number) n v = v ++ " is a number" +treeToString (scalar boolean) n v = v ++ " is a boolean" +treeToString (scalar string) n v = v ++ " is a string" +treeToString (scalar nil) n v = v ++ " is nil" +treeToString function n v = v ++ " is a function" +treeToString (function-ok t) n v = treeToString t n (v ++ "()") +treeToString (function-err t) n v = v ++ "(" ++ w ++ ") can error when\n " ++ treeToString t (suc n) w where w = tmp n + +subtypeWarningToString : ∀ {T U} → (T ≮: U) → String +subtypeWarningToString (witness t p q) = "\n because provided type contains v, where " ++ treeToString t 0 "v" + warningToStringᴱ : ∀ {H Γ T} M → {D : Γ ⊢ᴱ M ∈ T} → Warningᴱ H D → String warningToStringᴮ : ∀ {H Γ T} B → {D : Γ ⊢ᴮ B ∈ T} → Warningᴮ H D → String warningToStringᴱ (var x) (UnboundVariable p) = "Unbound variable " ++ varToString x -warningToStringᴱ (val (addr a)) (UnallocatedAddress p) = "Unallocated adress " ++ addrToString a -warningToStringᴱ (M $ N) (FunctionCallMismatch {T = T} {U = U} p) = "Function has type " ++ typeToString T ++ " but argument has type " ++ typeToString U +warningToStringᴱ (val (addr a)) (UnallocatedAddress p) = "Unallocated address " ++ addrToString a +warningToStringᴱ (M $ N) (FunctionCallMismatch {T = T} {U = U} p) = "Function has type " ++ typeToString T ++ " but argument has type " ++ typeToString U ++ subtypeWarningToString p warningToStringᴱ (M $ N) (app₁ W) = warningToStringᴱ M W warningToStringᴱ (M $ N) (app₂ W) = warningToStringᴱ N W -warningToStringᴱ (function f ⟨ var x ∈ T ⟩∈ U is B end) (FunctionDefnMismatch {V = V} p) = "Function expresion " ++ varToString f ++ " has return type " ++ typeToString U ++ " but body returns " ++ typeToString V +warningToStringᴱ (function f ⟨ var x ∈ T ⟩∈ U is B end) (FunctionDefnMismatch {V = V} p) = "Function expresion " ++ varToString f ++ " has return type " ++ typeToString U ++ " but body returns " ++ typeToString V ++ subtypeWarningToString p warningToStringᴱ (function f ⟨ var x ∈ T ⟩∈ U is B end) (function₁ W) = warningToStringᴮ B W ++ "\n in function expression " ++ varToString f -warningToStringᴱ block var b ∈ T is B end (BlockMismatch {U = U} p) = "Block " ++ varToString b ++ " has type " ++ typeToString T ++ " but body returns " ++ typeToString U +warningToStringᴱ block var b ∈ T is B end (BlockMismatch {U = U} p) = "Block " ++ varToString b ++ " has type " ++ typeToString T ++ " but body returns " ++ typeToString U ++ subtypeWarningToString p warningToStringᴱ block var b ∈ T is B end (block₁ W) = warningToStringᴮ B W ++ "\n in block " ++ varToString b -warningToStringᴱ (binexp M op N) (BinOpMismatch₁ {T = T} p) = "Binary operator " ++ binOpToString op ++ " lhs has type " ++ typeToString T -warningToStringᴱ (binexp M op N) (BinOpMismatch₂ {U = U} p) = "Binary operator " ++ binOpToString op ++ " rhs has type " ++ typeToString U +warningToStringᴱ (binexp M op N) (BinOpMismatch₁ {T = T} p) = "Binary operator " ++ binOpToString op ++ " lhs has type " ++ typeToString T ++ subtypeWarningToString p +warningToStringᴱ (binexp M op N) (BinOpMismatch₂ {U = U} p) = "Binary operator " ++ binOpToString op ++ " rhs has type " ++ typeToString U ++ subtypeWarningToString p warningToStringᴱ (binexp M op N) (bin₁ W) = warningToStringᴱ M W warningToStringᴱ (binexp M op N) (bin₂ W) = warningToStringᴱ N W -warningToStringᴮ (function f ⟨ var x ∈ T ⟩∈ U is C end ∙ B) (FunctionDefnMismatch {V = V} p) = "Function declaration " ++ varToString f ++ " has return type " ++ typeToString U ++ " but body returns " ++ typeToString V +warningToStringᴮ (function f ⟨ var x ∈ T ⟩∈ U is C end ∙ B) (FunctionDefnMismatch {V = V} p) = "Function declaration " ++ varToString f ++ " has return type " ++ typeToString U ++ " but body returns " ++ typeToString V ++ subtypeWarningToString p warningToStringᴮ (function f ⟨ var x ∈ T ⟩∈ U is C end ∙ B) (function₁ W) = warningToStringᴮ C W ++ "\n in function declaration " ++ varToString f warningToStringᴮ (function f ⟨ var x ∈ T ⟩∈ U is C end ∙ B) (function₂ W) = warningToStringᴮ B W -warningToStringᴮ (local var x ∈ T ← M ∙ B) (LocalVarMismatch {U = U} p) = "Local variable " ++ varToString x ++ " has type " ++ typeToString T ++ " but expression has type " ++ typeToString U +warningToStringᴮ (local var x ∈ T ← M ∙ B) (LocalVarMismatch {U = U} p) = "Local variable " ++ varToString x ++ " has type " ++ typeToString T ++ " but expression has type " ++ typeToString U ++ subtypeWarningToString p warningToStringᴮ (local var x ∈ T ← M ∙ B) (local₁ W) = warningToStringᴱ M W ++ "\n in local variable declaration " ++ varToString x warningToStringᴮ (local var x ∈ T ← M ∙ B) (local₂ W) = warningToStringᴮ B W warningToStringᴮ (return M ∙ B) (return W) = warningToStringᴱ M W ++ "\n in return statement" diff --git a/prototyping/Luau/Subtyping.agda b/prototyping/Luau/Subtyping.agda new file mode 100644 index 00000000..7d67eb43 --- /dev/null +++ b/prototyping/Luau/Subtyping.agda @@ -0,0 +1,62 @@ +{-# OPTIONS --rewriting #-} + +open import Luau.Type using (Type; Scalar; nil; number; string; boolean; none; any; _⇒_; _∪_; _∩_) +open import Properties.Equality using (_≢_) + +module Luau.Subtyping where + +-- An implementation of semantic subtyping + +-- We think of types as languages of trees + +data Tree : Set where + + scalar : ∀ {T} → Scalar T → Tree + function : Tree + function-ok : Tree → Tree + function-err : Tree → Tree + +data Language : Type → Tree → Set +data ¬Language : Type → Tree → Set + +data Language where + + scalar : ∀ {T} → (s : Scalar T) → Language T (scalar s) + function : ∀ {T U} → Language (T ⇒ U) function + function-ok : ∀ {T U u} → (Language U u) → Language (T ⇒ U) (function-ok u) + function-err : ∀ {T U t} → (¬Language T t) → Language (T ⇒ U) (function-err t) + scalar-function-err : ∀ {S t} → (Scalar S) → Language S (function-err t) + left : ∀ {T U t} → Language T t → Language (T ∪ U) t + right : ∀ {T U u} → Language U u → Language (T ∪ U) u + _,_ : ∀ {T U t} → Language T t → Language U t → Language (T ∩ U) t + any : ∀ {t} → Language any t + +data ¬Language where + + scalar-scalar : ∀ {S T} → (s : Scalar S) → (Scalar T) → (S ≢ T) → ¬Language T (scalar s) + scalar-function : ∀ {S} → (Scalar S) → ¬Language S function + scalar-function-ok : ∀ {S u} → (Scalar S) → ¬Language S (function-ok u) + function-scalar : ∀ {S T U} (s : Scalar S) → ¬Language (T ⇒ U) (scalar s) + function-ok : ∀ {T U u} → (¬Language U u) → ¬Language (T ⇒ U) (function-ok u) + function-err : ∀ {T U t} → (Language T t) → ¬Language (T ⇒ U) (function-err t) + _,_ : ∀ {T U t} → ¬Language T t → ¬Language U t → ¬Language (T ∪ U) t + left : ∀ {T U t} → ¬Language T t → ¬Language (T ∩ U) t + right : ∀ {T U u} → ¬Language U u → ¬Language (T ∩ U) u + none : ∀ {t} → ¬Language none t + +-- Subtyping as language inclusion + +_<:_ : Type → Type → Set +(T <: U) = ∀ t → (Language T t) → (Language U t) + +-- For warnings, we are interested in failures of subtyping, +-- which is whrn there is a tree in T's language that isn't in U's. + +data _≮:_ (T U : Type) : Set where + + witness : ∀ t → + + Language T t → + ¬Language U t → + ----------------- + T ≮: U diff --git a/prototyping/Luau/Type.agda b/prototyping/Luau/Type.agda index c5b76142..87815ddb 100644 --- a/prototyping/Luau/Type.agda +++ b/prototyping/Luau/Type.agda @@ -17,6 +17,13 @@ data Type : Set where _∪_ : Type → Type → Type _∩_ : Type → Type → Type +data Scalar : Type → Set where + + number : Scalar number + boolean : Scalar boolean + string : Scalar string + nil : Scalar nil + lhs : Type → Type lhs (T ⇒ _) = T lhs (T ∪ _) = T diff --git a/prototyping/Luau/TypeCheck.agda b/prototyping/Luau/TypeCheck.agda index 65ed1831..c22618bc 100644 --- a/prototyping/Luau/TypeCheck.agda +++ b/prototyping/Luau/TypeCheck.agda @@ -10,7 +10,7 @@ open import Luau.Syntax using (Expr; Stat; Block; BinaryOperator; yes; nil; addr open import Luau.Var using (Var) open import Luau.Addr using (Addr) open import Luau.Heap using (Heap; Object; function_is_end) renaming (_[_] to _[_]ᴴ) -open import Luau.Type using (Type; Mode; nil; none; number; boolean; string; _⇒_; tgt) +open import Luau.Type using (Type; Mode; nil; any; number; boolean; string; _⇒_; tgt) open import Luau.VarCtxt using (VarCtxt; ∅; _⋒_; _↦_; _⊕_↦_; _⊝_) renaming (_[_] to _[_]ⱽ) open import FFI.Data.Vector using (Vector) open import FFI.Data.Maybe using (Maybe; just; nothing) @@ -19,9 +19,22 @@ open import Properties.Product using (_×_; _,_) src : Type → Type src = Luau.Type.src m -orNone : Maybe Type → Type -orNone nothing = none -orNone (just T) = T +orAny : Maybe Type → Type +orAny nothing = any +orAny (just T) = T + +srcBinOp : BinaryOperator → Type +srcBinOp + = number +srcBinOp - = number +srcBinOp * = number +srcBinOp / = number +srcBinOp < = number +srcBinOp > = number +srcBinOp == = any +srcBinOp ~= = any +srcBinOp <= = number +srcBinOp >= = number +srcBinOp ·· = string tgtBinOp : BinaryOperator → Type tgtBinOp + = number @@ -76,7 +89,7 @@ data _⊢ᴱ_∈_ where var : ∀ {x T Γ} → - T ≡ orNone(Γ [ x ]ⱽ) → + T ≡ orAny(Γ [ x ]ⱽ) → ---------------- Γ ⊢ᴱ (var x) ∈ T diff --git a/prototyping/Properties.agda b/prototyping/Properties.agda index 8b30ce12..5594812e 100644 --- a/prototyping/Properties.agda +++ b/prototyping/Properties.agda @@ -5,7 +5,9 @@ module Properties where import Properties.Contradiction import Properties.Dec import Properties.Equality +import Properties.Functions import Properties.Remember import Properties.Step import Properties.StrictMode +import Properties.Subtyping import Properties.TypeCheck diff --git a/prototyping/Properties/Functions.agda b/prototyping/Properties/Functions.agda new file mode 100644 index 00000000..313b0ff2 --- /dev/null +++ b/prototyping/Properties/Functions.agda @@ -0,0 +1,6 @@ +module Properties.Functions where + +infixr 5 _∘_ + +_∘_ : ∀ {A B C : Set} → (B → C) → (A → B) → (A → C) +(f ∘ g) x = f (g x) diff --git a/prototyping/Properties/StrictMode.agda b/prototyping/Properties/StrictMode.agda index 77852f1a..2ff2b153 100644 --- a/prototyping/Properties/StrictMode.agda +++ b/prototyping/Properties/StrictMode.agda @@ -4,13 +4,15 @@ module Properties.StrictMode where import Agda.Builtin.Equality.Rewrite open import Agda.Builtin.Equality using (_≡_; refl) +open import FFI.Data.Either using (Either; Left; Right; mapL; mapR; mapLR; swapLR; cond) open import FFI.Data.Maybe using (Maybe; just; nothing) open import Luau.Heap using (Heap; Object; function_is_end; defn; alloc; ok; next; lookup-not-allocated) renaming (_≡_⊕_↦_ to _≡ᴴ_⊕_↦_; _[_] to _[_]ᴴ; ∅ to ∅ᴴ) -open import Luau.StrictMode using (Warningᴱ; Warningᴮ; Warningᴼ; Warningᴴᴱ; Warningᴴᴮ; UnallocatedAddress; UnboundVariable; FunctionCallMismatch; app₁; app₂; BinOpWarning; BinOpMismatch₁; BinOpMismatch₂; bin₁; bin₂; BlockMismatch; block₁; return; LocalVarMismatch; local₁; local₂; FunctionDefnMismatch; function₁; function₂; heap; expr; block; addr; +; -; *; /; <; >; <=; >=; ··) +open import Luau.StrictMode using (Warningᴱ; Warningᴮ; Warningᴼ; Warningᴴ; UnallocatedAddress; UnboundVariable; FunctionCallMismatch; app₁; app₂; BinOpMismatch₁; BinOpMismatch₂; bin₁; bin₂; BlockMismatch; block₁; return; LocalVarMismatch; local₁; local₂; FunctionDefnMismatch; function₁; function₂; heap; expr; block; addr) open import Luau.Substitution using (_[_/_]ᴮ; _[_/_]ᴱ; _[_/_]ᴮunless_; var_[_/_]ᴱwhenever_) +open import Luau.Subtyping using (_≮:_; witness; any; none; scalar; function; scalar-function; scalar-function-ok; scalar-function-err; scalar-scalar; function-scalar; function-ok; function-err; left; right; _,_; Tree; Language; ¬Language) open import Luau.Syntax using (Expr; yes; var; val; var_∈_; _⟨_⟩∈_; _$_; addr; number; bool; string; binexp; nil; function_is_end; block_is_end; done; return; local_←_; _∙_; fun; arg; name; ==; ~=) -open import Luau.Type using (Type; strict; nil; _⇒_; none; tgt; _≡ᵀ_; _≡ᴹᵀ_) -open import Luau.TypeCheck(strict) using (_⊢ᴮ_∈_; _⊢ᴱ_∈_; _⊢ᴴᴮ_▷_∈_; _⊢ᴴᴱ_▷_∈_; nil; var; addr; app; function; block; done; return; local; orNone; tgtBinOp) +open import Luau.Type using (Type; strict; nil; number; boolean; string; _⇒_; none; any; _∩_; _∪_; tgt; _≡ᵀ_; _≡ᴹᵀ_) +open import Luau.TypeCheck(strict) using (_⊢ᴮ_∈_; _⊢ᴱ_∈_; _⊢ᴴᴮ_▷_∈_; _⊢ᴴᴱ_▷_∈_; nil; var; addr; app; function; block; done; return; local; orAny; srcBinOp; tgtBinOp) open import Luau.Var using (_≡ⱽ_) open import Luau.Addr using (_≡ᴬ_) open import Luau.VarCtxt using (VarCtxt; ∅; _⋒_; _↦_; _⊕_↦_; _⊝_; ⊕-lookup-miss; ⊕-swap; ⊕-over) renaming (_[_] to _[_]ⱽ) @@ -18,17 +20,19 @@ open import Luau.VarCtxt using (VarCtxt; ∅) open import Properties.Remember using (remember; _,_) open import Properties.Equality using (_≢_; sym; cong; trans; subst₁) open import Properties.Dec using (Dec; yes; no) -open import Properties.Contradiction using (CONTRADICTION) -open import Properties.TypeCheck(strict) using (typeOfᴼ; typeOfᴹᴼ; typeOfⱽ; typeOfᴱ; typeOfᴮ; typeCheckᴱ; typeCheckᴮ; typeCheckᴼ; typeCheckᴴᴱ; typeCheckᴴᴮ; mustBeFunction; mustBeNumber; mustBeString) +open import Properties.Contradiction using (CONTRADICTION; ¬) +open import Properties.Functions using (_∘_) +open import Properties.Subtyping using (any-≮:; ≡-trans-≮:; ≮:-trans-≡; none-tgt-≮:; tgt-none-≮:; src-any-≮:; any-src-≮:; ≮:-trans; ≮:-refl; scalar-≢-impl-≮:; function-≮:-scalar; scalar-≮:-function; function-≮:-none; any-≮:-scalar; scalar-≮:-none; any-≮:-none) +open import Properties.TypeCheck(strict) using (typeOfᴼ; typeOfᴹᴼ; typeOfⱽ; typeOfᴱ; typeOfᴮ; typeCheckᴱ; typeCheckᴮ; typeCheckᴼ; typeCheckᴴ) open import Luau.OpSem using (_⟦_⟧_⟶_; _⊢_⟶*_⊣_; _⊢_⟶ᴮ_⊣_; _⊢_⟶ᴱ_⊣_; app₁; app₂; function; beta; return; block; done; local; subst; binOp₀; binOp₁; binOp₂; refl; step; +; -; *; /; <; >; ==; ~=; <=; >=; ··) open import Luau.RuntimeError using (BinOpError; RuntimeErrorᴱ; RuntimeErrorᴮ; FunctionMismatch; BinOpMismatch₁; BinOpMismatch₂; UnboundVariable; SEGV; app₁; app₂; bin₁; bin₂; block; local; return; +; -; *; /; <; >; <=; >=; ··) -open import Luau.RuntimeType using (valueType) +open import Luau.RuntimeType using (RuntimeType; valueType; number; string; boolean; nil; function) src = Luau.Type.src strict data _⊑_ (H : Heap yes) : Heap yes → Set where refl : (H ⊑ H) - snoc : ∀ {H′ a V} → (H′ ≡ᴴ H ⊕ a ↦ V) → (H ⊑ H′) + snoc : ∀ {H′ a O} → (H′ ≡ᴴ H ⊕ a ↦ O) → (H ⊑ H′) rednᴱ⊑ : ∀ {H H′ M M′} → (H ⊢ M ⟶ᴱ M′ ⊣ H′) → (H ⊑ H′) rednᴮ⊑ : ∀ {H H′ B B′} → (H ⊢ B ⟶ᴮ B′ ⊣ H′) → (H ⊑ H′) @@ -59,90 +63,51 @@ lookup-⊑-nothing {H} a (snoc defn) p with a ≡ᴬ next H lookup-⊑-nothing {H} a (snoc defn) p | yes refl = refl lookup-⊑-nothing {H} a (snoc o) p | no q = trans (lookup-not-allocated o q) p -data OrWarningᴱ {Γ M T} (H : Heap yes) (D : Γ ⊢ᴱ M ∈ T) A : Set where - ok : A → OrWarningᴱ H D A - warning : Warningᴱ H D → OrWarningᴱ H D A +heap-weakeningᴱ : ∀ Γ H M {H′ U} → (H ⊑ H′) → (typeOfᴱ H′ Γ M ≮: U) → (typeOfᴱ H Γ M ≮: U) +heap-weakeningᴱ Γ H (var x) h p = p +heap-weakeningᴱ Γ H (val nil) h p = p +heap-weakeningᴱ Γ H (val (addr a)) refl p = p +heap-weakeningᴱ Γ H (val (addr a)) (snoc {a = b} q) p with a ≡ᴬ b +heap-weakeningᴱ Γ H (val (addr a)) (snoc {a = a} defn) p | yes refl = any-≮: p +heap-weakeningᴱ Γ H (val (addr a)) (snoc {a = b} q) p | no r = ≡-trans-≮: (cong orAny (cong typeOfᴹᴼ (lookup-not-allocated q r))) p +heap-weakeningᴱ Γ H (val (number x)) h p = p +heap-weakeningᴱ Γ H (val (bool x)) h p = p +heap-weakeningᴱ Γ H (val (string x)) h p = p +heap-weakeningᴱ Γ H (M $ N) h p = none-tgt-≮: (heap-weakeningᴱ Γ H M h (tgt-none-≮: p)) +heap-weakeningᴱ Γ H (function f ⟨ var x ∈ T ⟩∈ U is B end) h p = p +heap-weakeningᴱ Γ H (block var b ∈ T is B end) h p = p +heap-weakeningᴱ Γ H (binexp M op N) h p = p -data OrWarningᴮ {Γ B T} (H : Heap yes) (D : Γ ⊢ᴮ B ∈ T) A : Set where - ok : A → OrWarningᴮ H D A - warning : Warningᴮ H D → OrWarningᴮ H D A +heap-weakeningᴮ : ∀ Γ H B {H′ U} → (H ⊑ H′) → (typeOfᴮ H′ Γ B ≮: U) → (typeOfᴮ H Γ B ≮: U) +heap-weakeningᴮ Γ H (function f ⟨ var x ∈ T ⟩∈ U is C end ∙ B) h p = heap-weakeningᴮ (Γ ⊕ f ↦ (T ⇒ U)) H B h p +heap-weakeningᴮ Γ H (local var x ∈ T ← M ∙ B) h p = heap-weakeningᴮ (Γ ⊕ x ↦ T) H B h p +heap-weakeningᴮ Γ H (return M ∙ B) h p = heap-weakeningᴱ Γ H M h p +heap-weakeningᴮ Γ H done h p = p -data OrWarningᴴᴱ {Γ M T} H (D : Γ ⊢ᴴᴱ H ▷ M ∈ T) A : Set where - ok : A → OrWarningᴴᴱ H D A - warning : Warningᴴᴱ H D → OrWarningᴴᴱ H D A +substitutivityᴱ : ∀ {Γ T U} H M v x → (typeOfᴱ H Γ (M [ v / x ]ᴱ) ≮: U) → Either (typeOfᴱ H (Γ ⊕ x ↦ T) M ≮: U) (typeOfᴱ H ∅ (val v) ≮: T) +substitutivityᴱ-whenever : ∀ {Γ T U} H v x y (r : Dec(x ≡ y)) → (typeOfᴱ H Γ (var y [ v / x ]ᴱwhenever r) ≮: U) → Either (typeOfᴱ H (Γ ⊕ x ↦ T) (var y) ≮: U) (typeOfᴱ H ∅ (val v) ≮: T) +substitutivityᴮ : ∀ {Γ T U} H B v x → (typeOfᴮ H Γ (B [ v / x ]ᴮ) ≮: U) → Either (typeOfᴮ H (Γ ⊕ x ↦ T) B ≮: U) (typeOfᴱ H ∅ (val v) ≮: T) +substitutivityᴮ-unless : ∀ {Γ T U V} H B v x y (r : Dec(x ≡ y)) → (typeOfᴮ H (Γ ⊕ y ↦ U) (B [ v / x ]ᴮunless r) ≮: V) → Either (typeOfᴮ H ((Γ ⊕ x ↦ T) ⊕ y ↦ U) B ≮: V) (typeOfᴱ H ∅ (val v) ≮: T) +substitutivityᴮ-unless-yes : ∀ {Γ Γ′ T V} H B v x y (r : x ≡ y) → (Γ′ ≡ Γ) → (typeOfᴮ H Γ (B [ v / x ]ᴮunless yes r) ≮: V) → Either (typeOfᴮ H Γ′ B ≮: V) (typeOfᴱ H ∅ (val v) ≮: T) +substitutivityᴮ-unless-no : ∀ {Γ Γ′ T V} H B v x y (r : x ≢ y) → (Γ′ ≡ Γ ⊕ x ↦ T) → (typeOfᴮ H Γ (B [ v / x ]ᴮunless no r) ≮: V) → Either (typeOfᴮ H Γ′ B ≮: V) (typeOfᴱ H ∅ (val v) ≮: T) -data OrWarningᴴᴮ {Γ B T} H (D : Γ ⊢ᴴᴮ H ▷ B ∈ T) A : Set where - ok : A → OrWarningᴴᴮ H D A - warning : Warningᴴᴮ H D → OrWarningᴴᴮ H D A +substitutivityᴱ H (var y) v x p = substitutivityᴱ-whenever H v x y (x ≡ⱽ y) p +substitutivityᴱ H (val w) v x p = Left p +substitutivityᴱ H (binexp M op N) v x p = Left p +substitutivityᴱ H (M $ N) v x p = mapL none-tgt-≮: (substitutivityᴱ H M v x (tgt-none-≮: p)) +substitutivityᴱ H (function f ⟨ var y ∈ T ⟩∈ U is B end) v x p = Left p +substitutivityᴱ H (block var b ∈ T is B end) v x p = Left p +substitutivityᴱ-whenever H v x x (yes refl) q = swapLR (≮:-trans q) +substitutivityᴱ-whenever H v x y (no p) q = Left (≡-trans-≮: (cong orAny (sym (⊕-lookup-miss x y _ _ p))) q) -heap-weakeningᴱ : ∀ H M {H′ Γ} → (H ⊑ H′) → OrWarningᴱ H (typeCheckᴱ H Γ M) (typeOfᴱ H Γ M ≡ typeOfᴱ H′ Γ M) -heap-weakeningᴮ : ∀ H B {H′ Γ} → (H ⊑ H′) → OrWarningᴮ H (typeCheckᴮ H Γ B) (typeOfᴮ H Γ B ≡ typeOfᴮ H′ Γ B) - -heap-weakeningᴱ H (var x) h = ok refl -heap-weakeningᴱ H (val nil) h = ok refl -heap-weakeningᴱ H (val (addr a)) refl = ok refl -heap-weakeningᴱ H (val (addr a)) (snoc {a = b} defn) with a ≡ᴬ b -heap-weakeningᴱ H (val (addr a)) (snoc {a = a} defn) | yes refl = warning (UnallocatedAddress refl) -heap-weakeningᴱ H (val (addr a)) (snoc {a = b} p) | no q = ok (cong orNone (cong typeOfᴹᴼ (lookup-not-allocated p q))) -heap-weakeningᴱ H (val (number n)) h = ok refl -heap-weakeningᴱ H (val (bool b)) h = ok refl -heap-weakeningᴱ H (val (string x)) h = ok refl -heap-weakeningᴱ H (binexp M op N) h = ok refl -heap-weakeningᴱ H (M $ N) h with heap-weakeningᴱ H M h -heap-weakeningᴱ H (M $ N) h | ok p = ok (cong tgt p) -heap-weakeningᴱ H (M $ N) h | warning W = warning (app₁ W) -heap-weakeningᴱ H (function f ⟨ var x ∈ T ⟩∈ U is B end) h = ok refl -heap-weakeningᴱ H (block var b ∈ T is B end) h = ok refl -heap-weakeningᴮ H (function f ⟨ var x ∈ T ⟩∈ U is C end ∙ B) h with heap-weakeningᴮ H B h -heap-weakeningᴮ H (function f ⟨ var x ∈ T ⟩∈ U is C end ∙ B) h | ok p = ok p -heap-weakeningᴮ H (function f ⟨ var x ∈ T ⟩∈ U is C end ∙ B) h | warning W = warning (function₂ W) -heap-weakeningᴮ H (local var x ∈ T ← M ∙ B) h with heap-weakeningᴮ H B h -heap-weakeningᴮ H (local var x ∈ T ← M ∙ B) h | ok p = ok p -heap-weakeningᴮ H (local var x ∈ T ← M ∙ B) h | warning W = warning (local₂ W) -heap-weakeningᴮ H (return M ∙ B) h with heap-weakeningᴱ H M h -heap-weakeningᴮ H (return M ∙ B) h | ok p = ok p -heap-weakeningᴮ H (return M ∙ B) h | warning W = warning (return W) -heap-weakeningᴮ H (done) h = ok refl - -none-not-obj : ∀ O → none ≢ typeOfᴼ O -none-not-obj (function f ⟨ var x ∈ T ⟩∈ U is B end) () - -typeOf-val-not-none : ∀ {H Γ} v → OrWarningᴱ H (typeCheckᴱ H Γ (val v)) (none ≢ typeOfᴱ H Γ (val v)) -typeOf-val-not-none nil = ok (λ ()) -typeOf-val-not-none (number n) = ok (λ ()) -typeOf-val-not-none (bool b) = ok (λ ()) -typeOf-val-not-none (string x) = ok (λ ()) -typeOf-val-not-none {H = H} (addr a) with remember (H [ a ]ᴴ) -typeOf-val-not-none {H = H} (addr a) | (just O , p) = ok (λ q → none-not-obj O (trans q (cong orNone (cong typeOfᴹᴼ p)))) -typeOf-val-not-none {H = H} (addr a) | (nothing , p) = warning (UnallocatedAddress p) - -substitutivityᴱ : ∀ {Γ T} H M v x → (just T ≡ typeOfⱽ H v) → (typeOfᴱ H (Γ ⊕ x ↦ T) M ≡ typeOfᴱ H Γ (M [ v / x ]ᴱ)) -substitutivityᴱ-whenever-yes : ∀ {Γ T} H v x y (p : x ≡ y) → (just T ≡ typeOfⱽ H v) → (typeOfᴱ H (Γ ⊕ x ↦ T) (var y) ≡ typeOfᴱ H Γ (var y [ v / x ]ᴱwhenever (yes p))) -substitutivityᴱ-whenever-no : ∀ {Γ T} H v x y (p : x ≢ y) → (just T ≡ typeOfⱽ H v) → (typeOfᴱ H (Γ ⊕ x ↦ T) (var y) ≡ typeOfᴱ H Γ (var y [ v / x ]ᴱwhenever (no p))) -substitutivityᴮ : ∀ {Γ T} H B v x → (just T ≡ typeOfⱽ H v) → (typeOfᴮ H (Γ ⊕ x ↦ T) B ≡ typeOfᴮ H Γ (B [ v / x ]ᴮ)) -substitutivityᴮ-unless-yes : ∀ {Γ Γ′ T} H B v x y (p : x ≡ y) → (just T ≡ typeOfⱽ H v) → (Γ′ ≡ Γ) → (typeOfᴮ H Γ′ B ≡ typeOfᴮ H Γ (B [ v / x ]ᴮunless (yes p))) -substitutivityᴮ-unless-no : ∀ {Γ Γ′ T} H B v x y (p : x ≢ y) → (just T ≡ typeOfⱽ H v) → (Γ′ ≡ Γ ⊕ x ↦ T) → (typeOfᴮ H Γ′ B ≡ typeOfᴮ H Γ (B [ v / x ]ᴮunless (no p))) - -substitutivityᴱ H (var y) v x p with x ≡ⱽ y -substitutivityᴱ H (var y) v x p | yes q = substitutivityᴱ-whenever-yes H v x y q p -substitutivityᴱ H (var y) v x p | no q = substitutivityᴱ-whenever-no H v x y q p -substitutivityᴱ H (val w) v x p = refl -substitutivityᴱ H (binexp M op N) v x p = refl -substitutivityᴱ H (M $ N) v x p = cong tgt (substitutivityᴱ H M v x p) -substitutivityᴱ H (function f ⟨ var y ∈ T ⟩∈ U is B end) v x p = refl -substitutivityᴱ H (block var b ∈ T is B end) v x p = refl -substitutivityᴱ-whenever-yes H v x x refl q = cong orNone q -substitutivityᴱ-whenever-no H v x y p q = cong orNone ( sym (⊕-lookup-miss x y _ _ p)) -substitutivityᴮ H (function f ⟨ var y ∈ T ⟩∈ U is C end ∙ B) v x p with x ≡ⱽ f -substitutivityᴮ H (function f ⟨ var y ∈ T ⟩∈ U is C end ∙ B) v x p | yes q = substitutivityᴮ-unless-yes H B v x f q p (⊕-over q) -substitutivityᴮ H (function f ⟨ var y ∈ T ⟩∈ U is C end ∙ B) v x p | no q = substitutivityᴮ-unless-no H B v x f q p (⊕-swap q) -substitutivityᴮ H (local var y ∈ T ← M ∙ B) v x p with x ≡ⱽ y -substitutivityᴮ H (local var y ∈ T ← M ∙ B) v x p | yes q = substitutivityᴮ-unless-yes H B v x y q p (⊕-over q) -substitutivityᴮ H (local var y ∈ T ← M ∙ B) v x p | no q = substitutivityᴮ-unless-no H B v x y q p (⊕-swap q) +substitutivityᴮ H (function f ⟨ var y ∈ T ⟩∈ U is C end ∙ B) v x p = substitutivityᴮ-unless H B v x f (x ≡ⱽ f) p +substitutivityᴮ H (local var y ∈ T ← M ∙ B) v x p = substitutivityᴮ-unless H B v x y (x ≡ⱽ y) p substitutivityᴮ H (return M ∙ B) v x p = substitutivityᴱ H M v x p -substitutivityᴮ H done v x p = refl -substitutivityᴮ-unless-yes H B v x x refl q refl = refl -substitutivityᴮ-unless-no H B v x y p q refl = substitutivityᴮ H B v x q +substitutivityᴮ H done v x p = Left p +substitutivityᴮ-unless H B v x y (yes p) q = substitutivityᴮ-unless-yes H B v x y p (⊕-over p) q +substitutivityᴮ-unless H B v x y (no p) q = substitutivityᴮ-unless-no H B v x y p (⊕-swap p) q +substitutivityᴮ-unless-yes H B v x y refl refl p = Left p +substitutivityᴮ-unless-no H B v x y p refl q = substitutivityᴮ H B v x q binOpPreservation : ∀ H {op v w x} → (v ⟦ op ⟧ w ⟶ x) → (tgtBinOp op ≡ typeOfᴱ H ∅ (val x)) binOpPreservation H (+ m n) = refl @@ -157,306 +122,215 @@ binOpPreservation H (== v w) = refl binOpPreservation H (~= v w) = refl binOpPreservation H (·· v w) = refl -preservationᴱ : ∀ H M {H′ M′} → (H ⊢ M ⟶ᴱ M′ ⊣ H′) → OrWarningᴴᴱ H (typeCheckᴴᴱ H ∅ M) (typeOfᴱ H ∅ M ≡ typeOfᴱ H′ ∅ M′) -preservationᴮ : ∀ H B {H′ B′} → (H ⊢ B ⟶ᴮ B′ ⊣ H′) → OrWarningᴴᴮ H (typeCheckᴴᴮ H ∅ B) (typeOfᴮ H ∅ B ≡ typeOfᴮ H′ ∅ B′) +reflect-subtypingᴱ : ∀ H M {H′ M′ T} → (H ⊢ M ⟶ᴱ M′ ⊣ H′) → (typeOfᴱ H′ ∅ M′ ≮: T) → Either (typeOfᴱ H ∅ M ≮: T) (Warningᴱ H (typeCheckᴱ H ∅ M)) +reflect-subtypingᴮ : ∀ H B {H′ B′ T} → (H ⊢ B ⟶ᴮ B′ ⊣ H′) → (typeOfᴮ H′ ∅ B′ ≮: T) → Either (typeOfᴮ H ∅ B ≮: T) (Warningᴮ H (typeCheckᴮ H ∅ B)) -preservationᴱ H (function f ⟨ var x ∈ T ⟩∈ U is B end) (function a defn) = ok refl -preservationᴱ H (M $ N) (app₁ s) with preservationᴱ H M s -preservationᴱ H (M $ N) (app₁ s) | ok p = ok (cong tgt p) -preservationᴱ H (M $ N) (app₁ s) | warning (expr W) = warning (expr (app₁ W)) -preservationᴱ H (M $ N) (app₁ s) | warning (heap W) = warning (heap W) -preservationᴱ H (M $ N) (app₂ p s) with heap-weakeningᴱ H M (rednᴱ⊑ s) -preservationᴱ H (M $ N) (app₂ p s) | ok q = ok (cong tgt q) -preservationᴱ H (M $ N) (app₂ p s) | warning W = warning (expr (app₁ W)) -preservationᴱ H (val (addr a) $ N) (beta (function f ⟨ var x ∈ S ⟩∈ T is B end) v refl p) with remember (typeOfⱽ H v) -preservationᴱ H (val (addr a) $ N) (beta (function f ⟨ var x ∈ S ⟩∈ T is B end) v refl p) | (just U , q) with S ≡ᵀ U | T ≡ᵀ typeOfᴮ H (x ↦ S) B -preservationᴱ H (val (addr a) $ N) (beta (function f ⟨ var x ∈ S ⟩∈ T is B end) v refl p) | (just U , q) | yes refl | yes refl = ok (cong tgt (cong orNone (cong typeOfᴹᴼ p))) -preservationᴱ H (val (addr a) $ N) (beta (function f ⟨ var x ∈ S ⟩∈ T is B end) v refl p) | (just U , q) | yes refl | no r = warning (heap (addr a p (FunctionDefnMismatch r))) -preservationᴱ H (val (addr a) $ N) (beta (function f ⟨ var x ∈ S ⟩∈ T is B end) v refl p) | (just U , q) | no r | _ = warning (expr (FunctionCallMismatch (λ s → r (trans (trans (sym (cong src (cong orNone (cong typeOfᴹᴼ p)))) s) (cong orNone q))))) -preservationᴱ H (val (addr a) $ N) (beta (function f ⟨ var x ∈ S ⟩∈ T is B end) v refl p) | (nothing , q) with typeOf-val-not-none v -preservationᴱ H (val (addr a) $ N) (beta (function f ⟨ var x ∈ S ⟩∈ T is B end) v refl p) | (nothing , q) | ok r = CONTRADICTION (r (sym (cong orNone q))) -preservationᴱ H (val (addr a) $ N) (beta (function f ⟨ var x ∈ S ⟩∈ T is B end) v refl p) | (nothing , q) | warning W = warning (expr (app₂ W)) -preservationᴱ H (block var b ∈ T is B end) (block s) = ok refl -preservationᴱ H (block var b ∈ T is return M ∙ B end) (return v) with T ≡ᵀ typeOfᴱ H ∅ (val v) -preservationᴱ H (block var b ∈ T is return M ∙ B end) (return v) | yes p = ok p -preservationᴱ H (block var b ∈ T is return M ∙ B end) (return v) | no p = warning (expr (BlockMismatch p)) -preservationᴱ H (block var b ∈ T is done end) (done) with T ≡ᵀ nil -preservationᴱ H (block var b ∈ T is done end) (done) | yes p = ok p -preservationᴱ H (block var b ∈ T is done end) (done) | no p = warning (expr (BlockMismatch p)) -preservationᴱ H (binexp M op N) (binOp₀ s) = ok (binOpPreservation H s) -preservationᴱ H (binexp M op N) (binOp₁ s) = ok refl -preservationᴱ H (binexp M op N) (binOp₂ s) = ok refl +reflect-subtypingᴱ H (M $ N) (app₁ s) p = mapLR none-tgt-≮: app₁ (reflect-subtypingᴱ H M s (tgt-none-≮: p)) +reflect-subtypingᴱ H (M $ N) (app₂ v s) p = Left (none-tgt-≮: (heap-weakeningᴱ ∅ H M (rednᴱ⊑ s) (tgt-none-≮: p))) +reflect-subtypingᴱ H (M $ N) (beta (function f ⟨ var y ∈ T ⟩∈ U is B end) v refl q) p = Left (≡-trans-≮: (cong tgt (cong orAny (cong typeOfᴹᴼ q))) p) +reflect-subtypingᴱ H (function f ⟨ var x ∈ T ⟩∈ U is B end) (function a defn) p = Left p +reflect-subtypingᴱ H (block var b ∈ T is B end) (block s) p = Left p +reflect-subtypingᴱ H (block var b ∈ T is return (val v) ∙ B end) (return v) p = mapR BlockMismatch (swapLR (≮:-trans p)) +reflect-subtypingᴱ H (block var b ∈ T is done end) done p = mapR BlockMismatch (swapLR (≮:-trans p)) +reflect-subtypingᴱ H (binexp M op N) (binOp₀ s) p = Left (≡-trans-≮: (binOpPreservation H s) p) +reflect-subtypingᴱ H (binexp M op N) (binOp₁ s) p = Left p +reflect-subtypingᴱ H (binexp M op N) (binOp₂ s) p = Left p -preservationᴮ H (local var x ∈ T ← M ∙ B) (local s) with heap-weakeningᴮ H B (rednᴱ⊑ s) -preservationᴮ H (local var x ∈ T ← M ∙ B) (local s) | ok p = ok p -preservationᴮ H (local var x ∈ T ← M ∙ B) (local s) | warning W = warning (block (local₂ W)) -preservationᴮ H (local var x ∈ T ← M ∙ B) (subst v) with remember (typeOfⱽ H v) -preservationᴮ H (local var x ∈ T ← M ∙ B) (subst v) | (just U , p) with T ≡ᵀ U -preservationᴮ H (local var x ∈ T ← M ∙ B) (subst v) | (just T , p) | yes refl = ok (substitutivityᴮ H B v x (sym p)) -preservationᴮ H (local var x ∈ T ← M ∙ B) (subst v) | (just U , p) | no q = warning (block (LocalVarMismatch (λ r → q (trans r (cong orNone p))))) -preservationᴮ H (local var x ∈ T ← M ∙ B) (subst v) | (nothing , p) with typeOf-val-not-none v -preservationᴮ H (local var x ∈ T ← M ∙ B) (subst v) | (nothing , p) | ok q = CONTRADICTION (q (sym (cong orNone p))) -preservationᴮ H (local var x ∈ T ← M ∙ B) (subst v) | (nothing , p) | warning W = warning (block (local₁ W)) -preservationᴮ H (function f ⟨ var x ∈ T ⟩∈ U is C end ∙ B) (function a defn) with heap-weakeningᴮ H B (snoc defn) -preservationᴮ H (function f ⟨ var x ∈ T ⟩∈ U is C end ∙ B) (function a defn) | ok r = ok (trans r (substitutivityᴮ _ B (addr a) f refl)) -preservationᴮ H (function f ⟨ var x ∈ T ⟩∈ U is C end ∙ B) (function a defn) | warning W = warning (block (function₂ W)) -preservationᴮ H (return M ∙ B) (return s) with preservationᴱ H M s -preservationᴮ H (return M ∙ B) (return s) | ok p = ok p -preservationᴮ H (return M ∙ B) (return s) | warning (expr W) = warning (block (return W)) -preservationᴮ H (return M ∙ B) (return s) | warning (heap W) = warning (heap W) +reflect-subtypingᴮ H (function f ⟨ var x ∈ T ⟩∈ U is C end ∙ B) (function a defn) p = mapLR (heap-weakeningᴮ _ _ B (snoc defn)) (CONTRADICTION ∘ ≮:-refl) (substitutivityᴮ _ B (addr a) f p) +reflect-subtypingᴮ H (local var x ∈ T ← M ∙ B) (local s) p = Left (heap-weakeningᴮ (x ↦ T) H B (rednᴱ⊑ s) p) +reflect-subtypingᴮ H (local var x ∈ T ← M ∙ B) (subst v) p = mapR LocalVarMismatch (substitutivityᴮ H B v x p) +reflect-subtypingᴮ H (return M ∙ B) (return s) p = mapR return (reflect-subtypingᴱ H M s p) -reflect-substitutionᴱ : ∀ {Γ T} H M v x → (just T ≡ typeOfⱽ H v) → Warningᴱ H (typeCheckᴱ H Γ (M [ v / x ]ᴱ)) → Warningᴱ H (typeCheckᴱ H (Γ ⊕ x ↦ T) M) -reflect-substitutionᴱ-whenever-yes : ∀ {Γ T} H v x y (p : x ≡ y) → (just T ≡ typeOfⱽ H v) → Warningᴱ H (typeCheckᴱ H Γ (var y [ v / x ]ᴱwhenever yes p)) → Warningᴱ H (typeCheckᴱ H (Γ ⊕ x ↦ T) (var y)) -reflect-substitutionᴱ-whenever-no : ∀ {Γ T} H v x y (p : x ≢ y) → (just T ≡ typeOfⱽ H v) → Warningᴱ H (typeCheckᴱ H Γ (var y [ v / x ]ᴱwhenever no p)) → Warningᴱ H (typeCheckᴱ H (Γ ⊕ x ↦ T) (var y)) -reflect-substitutionᴮ : ∀ {Γ T} H B v x → (just T ≡ typeOfⱽ H v) → Warningᴮ H (typeCheckᴮ H Γ (B [ v / x ]ᴮ)) → Warningᴮ H (typeCheckᴮ H (Γ ⊕ x ↦ T) B) -reflect-substitutionᴮ-unless-yes : ∀ {Γ Γ′ T} H B v x y (r : x ≡ y) → (just T ≡ typeOfⱽ H v) → (Γ′ ≡ Γ) → Warningᴮ H (typeCheckᴮ H Γ (B [ v / x ]ᴮunless yes r)) → Warningᴮ H (typeCheckᴮ H Γ′ B) -reflect-substitutionᴮ-unless-no : ∀ {Γ Γ′ T} H B v x y (r : x ≢ y) → (just T ≡ typeOfⱽ H v) → (Γ′ ≡ Γ ⊕ x ↦ T) → Warningᴮ H (typeCheckᴮ H Γ (B [ v / x ]ᴮunless no r)) → Warningᴮ H (typeCheckᴮ H Γ′ B) +reflect-substitutionᴱ : ∀ {Γ T} H M v x → Warningᴱ H (typeCheckᴱ H Γ (M [ v / x ]ᴱ)) → Either (Warningᴱ H (typeCheckᴱ H (Γ ⊕ x ↦ T) M)) (Either (Warningᴱ H (typeCheckᴱ H ∅ (val v))) (typeOfᴱ H ∅ (val v) ≮: T)) +reflect-substitutionᴱ-whenever : ∀ {Γ T} H v x y (p : Dec(x ≡ y)) → Warningᴱ H (typeCheckᴱ H Γ (var y [ v / x ]ᴱwhenever p)) → Either (Warningᴱ H (typeCheckᴱ H (Γ ⊕ x ↦ T) (var y))) (Either (Warningᴱ H (typeCheckᴱ H ∅ (val v))) (typeOfᴱ H ∅ (val v) ≮: T)) +reflect-substitutionᴮ : ∀ {Γ T} H B v x → Warningᴮ H (typeCheckᴮ H Γ (B [ v / x ]ᴮ)) → Either (Warningᴮ H (typeCheckᴮ H (Γ ⊕ x ↦ T) B)) (Either (Warningᴱ H (typeCheckᴱ H ∅ (val v))) (typeOfᴱ H ∅ (val v) ≮: T)) +reflect-substitutionᴮ-unless : ∀ {Γ T U} H B v x y (r : Dec(x ≡ y)) → Warningᴮ H (typeCheckᴮ H (Γ ⊕ y ↦ U) (B [ v / x ]ᴮunless r)) → Either (Warningᴮ H (typeCheckᴮ H ((Γ ⊕ x ↦ T) ⊕ y ↦ U) B)) (Either (Warningᴱ H (typeCheckᴱ H ∅ (val v))) (typeOfᴱ H ∅ (val v) ≮: T)) +reflect-substitutionᴮ-unless-yes : ∀ {Γ Γ′ T} H B v x y (r : x ≡ y) → (Γ′ ≡ Γ) → Warningᴮ H (typeCheckᴮ H Γ (B [ v / x ]ᴮunless yes r)) → Either (Warningᴮ H (typeCheckᴮ H Γ′ B)) (Either (Warningᴱ H (typeCheckᴱ H ∅ (val v))) (typeOfᴱ H ∅ (val v) ≮: T)) +reflect-substitutionᴮ-unless-no : ∀ {Γ Γ′ T} H B v x y (r : x ≢ y) → (Γ′ ≡ Γ ⊕ x ↦ T) → Warningᴮ H (typeCheckᴮ H Γ (B [ v / x ]ᴮunless no r)) → Either (Warningᴮ H (typeCheckᴮ H Γ′ B)) (Either (Warningᴱ H (typeCheckᴱ H ∅ (val v))) (typeOfᴱ H ∅ (val v) ≮: T)) -reflect-substitutionᴱ H (var y) v x p W with x ≡ⱽ y -reflect-substitutionᴱ H (var y) v x p W | yes r = reflect-substitutionᴱ-whenever-yes H v x y r p W -reflect-substitutionᴱ H (var y) v x p W | no r = reflect-substitutionᴱ-whenever-no H v x y r p W -reflect-substitutionᴱ H (val (addr a)) v x p (UnallocatedAddress r) = UnallocatedAddress r -reflect-substitutionᴱ H (M $ N) v x p (FunctionCallMismatch q) = FunctionCallMismatch (λ s → q (trans (cong src (sym (substitutivityᴱ H M v x p))) (trans s (substitutivityᴱ H N v x p)))) -reflect-substitutionᴱ H (M $ N) v x p (app₁ W) = app₁ (reflect-substitutionᴱ H M v x p W) -reflect-substitutionᴱ H (M $ N) v x p (app₂ W) = app₂ (reflect-substitutionᴱ H N v x p W) -reflect-substitutionᴱ H (function f ⟨ var y ∈ T ⟩∈ U is B end) v x p (FunctionDefnMismatch q) with (x ≡ⱽ y) -reflect-substitutionᴱ H (function f ⟨ var y ∈ T ⟩∈ U is B end) v x p (FunctionDefnMismatch q) | yes r = FunctionDefnMismatch (λ s → q (trans s (substitutivityᴮ-unless-yes H B v x y r p (⊕-over r)))) -reflect-substitutionᴱ H (function f ⟨ var y ∈ T ⟩∈ U is B end) v x p (FunctionDefnMismatch q) | no r = FunctionDefnMismatch (λ s → q (trans s (substitutivityᴮ-unless-no H B v x y r p (⊕-swap r)))) -reflect-substitutionᴱ H (function f ⟨ var y ∈ T ⟩∈ U is B end) v x p (function₁ W) with (x ≡ⱽ y) -reflect-substitutionᴱ H (function f ⟨ var y ∈ T ⟩∈ U is B end) v x p (function₁ W) | yes r = function₁ (reflect-substitutionᴮ-unless-yes H B v x y r p (⊕-over r) W) -reflect-substitutionᴱ H (function f ⟨ var y ∈ T ⟩∈ U is B end) v x p (function₁ W) | no r = function₁ (reflect-substitutionᴮ-unless-no H B v x y r p (⊕-swap r) W) -reflect-substitutionᴱ H (block var b ∈ T is B end) v x p (BlockMismatch q) = BlockMismatch (λ r → q (trans r (substitutivityᴮ H B v x p))) -reflect-substitutionᴱ H (block var b ∈ T is B end) v x p (block₁ W) = block₁ (reflect-substitutionᴮ H B v x p W) -reflect-substitutionᴱ H (binexp M op N) x v p (BinOpMismatch₁ q) = BinOpMismatch₁ (subst₁ (BinOpWarning op) (sym (substitutivityᴱ H M x v p)) q) -reflect-substitutionᴱ H (binexp M op N) x v p (BinOpMismatch₂ q) = BinOpMismatch₂ (subst₁ (BinOpWarning op) (sym (substitutivityᴱ H N x v p)) q) -reflect-substitutionᴱ H (binexp M op N) x v p (bin₁ W) = bin₁ (reflect-substitutionᴱ H M x v p W) -reflect-substitutionᴱ H (binexp M op N) x v p (bin₂ W) = bin₂ (reflect-substitutionᴱ H N x v p W) +reflect-substitutionᴱ H (var y) v x W = reflect-substitutionᴱ-whenever H v x y (x ≡ⱽ y) W +reflect-substitutionᴱ H (val (addr a)) v x (UnallocatedAddress r) = Left (UnallocatedAddress r) +reflect-substitutionᴱ H (M $ N) v x (FunctionCallMismatch p) with substitutivityᴱ H N v x p +reflect-substitutionᴱ H (M $ N) v x (FunctionCallMismatch p) | Right W = Right (Right W) +reflect-substitutionᴱ H (M $ N) v x (FunctionCallMismatch p) | Left q with substitutivityᴱ H M v x (src-any-≮: q) +reflect-substitutionᴱ H (M $ N) v x (FunctionCallMismatch p) | Left q | Left r = Left ((FunctionCallMismatch ∘ any-src-≮: q) r) +reflect-substitutionᴱ H (M $ N) v x (FunctionCallMismatch p) | Left q | Right W = Right (Right W) +reflect-substitutionᴱ H (M $ N) v x (app₁ W) = mapL app₁ (reflect-substitutionᴱ H M v x W) +reflect-substitutionᴱ H (M $ N) v x (app₂ W) = mapL app₂ (reflect-substitutionᴱ H N v x W) +reflect-substitutionᴱ H (function f ⟨ var y ∈ T ⟩∈ U is B end) v x (FunctionDefnMismatch q) = mapLR FunctionDefnMismatch Right (substitutivityᴮ-unless H B v x y (x ≡ⱽ y) q) +reflect-substitutionᴱ H (function f ⟨ var y ∈ T ⟩∈ U is B end) v x (function₁ W) = mapL function₁ (reflect-substitutionᴮ-unless H B v x y (x ≡ⱽ y) W) +reflect-substitutionᴱ H (block var b ∈ T is B end) v x (BlockMismatch q) = mapLR BlockMismatch Right (substitutivityᴮ H B v x q) +reflect-substitutionᴱ H (block var b ∈ T is B end) v x (block₁ W′) = mapL block₁ (reflect-substitutionᴮ H B v x W′) +reflect-substitutionᴱ H (binexp M op N) v x (BinOpMismatch₁ q) = mapLR BinOpMismatch₁ Right (substitutivityᴱ H M v x q) +reflect-substitutionᴱ H (binexp M op N) v x (BinOpMismatch₂ q) = mapLR BinOpMismatch₂ Right (substitutivityᴱ H N v x q) +reflect-substitutionᴱ H (binexp M op N) v x (bin₁ W) = mapL bin₁ (reflect-substitutionᴱ H M v x W) +reflect-substitutionᴱ H (binexp M op N) v x (bin₂ W) = mapL bin₂ (reflect-substitutionᴱ H N v x W) -reflect-substitutionᴱ-whenever-no H v x y p q (UnboundVariable r) = UnboundVariable (trans (sym (⊕-lookup-miss x y _ _ p)) r) -reflect-substitutionᴱ-whenever-yes H (addr a) x x refl p (UnallocatedAddress q) with trans p (cong typeOfᴹᴼ q) -reflect-substitutionᴱ-whenever-yes H (addr a) x x refl p (UnallocatedAddress q) | () +reflect-substitutionᴱ-whenever H a x x (yes refl) (UnallocatedAddress p) = Right (Left (UnallocatedAddress p)) +reflect-substitutionᴱ-whenever H v x y (no p) (UnboundVariable q) = Left (UnboundVariable (trans (sym (⊕-lookup-miss x y _ _ p)) q)) -reflect-substitutionᴮ H (function f ⟨ var y ∈ T ⟩∈ U is C end ∙ B) v x p (FunctionDefnMismatch q) with (x ≡ⱽ y) -reflect-substitutionᴮ H (function f ⟨ var y ∈ T ⟩∈ U is C end ∙ B) v x p (FunctionDefnMismatch q) | yes r = FunctionDefnMismatch (λ s → q (trans s (substitutivityᴮ-unless-yes H C v x y r p (⊕-over r)))) -reflect-substitutionᴮ H (function f ⟨ var y ∈ T ⟩∈ U is C end ∙ B) v x p (FunctionDefnMismatch q) | no r = FunctionDefnMismatch (λ s → q (trans s (substitutivityᴮ-unless-no H C v x y r p (⊕-swap r)))) -reflect-substitutionᴮ H (function f ⟨ var y ∈ T ⟩∈ U is C end ∙ B) v x p (function₁ W) with (x ≡ⱽ y) -reflect-substitutionᴮ H (function f ⟨ var y ∈ T ⟩∈ U is C end ∙ B) v x p (function₁ W) | yes r = function₁ (reflect-substitutionᴮ-unless-yes H C v x y r p (⊕-over r) W) -reflect-substitutionᴮ H (function f ⟨ var y ∈ T ⟩∈ U is C end ∙ B) v x p (function₁ W) | no r = function₁ (reflect-substitutionᴮ-unless-no H C v x y r p (⊕-swap r) W) -reflect-substitutionᴮ H (function f ⟨ var y ∈ T ⟩∈ U is C end ∙ B) v x p (function₂ W) with (x ≡ⱽ f) -reflect-substitutionᴮ H (function f ⟨ var y ∈ T ⟩∈ U is C end ∙ B) v x p (function₂ W)| yes r = function₂ (reflect-substitutionᴮ-unless-yes H B v x f r p (⊕-over r) W) -reflect-substitutionᴮ H (function f ⟨ var y ∈ T ⟩∈ U is C end ∙ B) v x p (function₂ W)| no r = function₂ (reflect-substitutionᴮ-unless-no H B v x f r p (⊕-swap r) W) -reflect-substitutionᴮ H (local var y ∈ T ← M ∙ B) v x p (LocalVarMismatch q) = LocalVarMismatch (λ r → q (trans r (substitutivityᴱ H M v x p))) -reflect-substitutionᴮ H (local var y ∈ T ← M ∙ B) v x p (local₁ W) = local₁ (reflect-substitutionᴱ H M v x p W) -reflect-substitutionᴮ H (local var y ∈ T ← M ∙ B) v x p (local₂ W) with (x ≡ⱽ y) -reflect-substitutionᴮ H (local var y ∈ T ← M ∙ B) v x p (local₂ W) | yes r = local₂ (reflect-substitutionᴮ-unless-yes H B v x y r p (⊕-over r) W) -reflect-substitutionᴮ H (local var y ∈ T ← M ∙ B) v x p (local₂ W) | no r = local₂ (reflect-substitutionᴮ-unless-no H B v x y r p (⊕-swap r) W) -reflect-substitutionᴮ H (return M ∙ B) v x p (return W) = return (reflect-substitutionᴱ H M v x p W) +reflect-substitutionᴮ H (function f ⟨ var y ∈ T ⟩∈ U is C end ∙ B) v x (FunctionDefnMismatch q) = mapLR FunctionDefnMismatch Right (substitutivityᴮ-unless H C v x y (x ≡ⱽ y) q) +reflect-substitutionᴮ H (function f ⟨ var y ∈ T ⟩∈ U is C end ∙ B) v x (function₁ W) = mapL function₁ (reflect-substitutionᴮ-unless H C v x y (x ≡ⱽ y) W) +reflect-substitutionᴮ H (function f ⟨ var y ∈ T ⟩∈ U is C end ∙ B) v x (function₂ W) = mapL function₂ (reflect-substitutionᴮ-unless H B v x f (x ≡ⱽ f) W) +reflect-substitutionᴮ H (local var y ∈ T ← M ∙ B) v x (LocalVarMismatch q) = mapLR LocalVarMismatch Right (substitutivityᴱ H M v x q) +reflect-substitutionᴮ H (local var y ∈ T ← M ∙ B) v x (local₁ W) = mapL local₁ (reflect-substitutionᴱ H M v x W) +reflect-substitutionᴮ H (local var y ∈ T ← M ∙ B) v x (local₂ W) = mapL local₂ (reflect-substitutionᴮ-unless H B v x y (x ≡ⱽ y) W) +reflect-substitutionᴮ H (return M ∙ B) v x (return W) = mapL return (reflect-substitutionᴱ H M v x W) -reflect-substitutionᴮ-unless-yes H B v x y r p refl W = W -reflect-substitutionᴮ-unless-no H B v x y r p refl W = reflect-substitutionᴮ H B v x p W +reflect-substitutionᴮ-unless H B v x y (yes p) W = reflect-substitutionᴮ-unless-yes H B v x y p (⊕-over p) W +reflect-substitutionᴮ-unless H B v x y (no p) W = reflect-substitutionᴮ-unless-no H B v x y p (⊕-swap p) W +reflect-substitutionᴮ-unless-yes H B v x x refl refl W = Left W +reflect-substitutionᴮ-unless-no H B v x y p refl W = reflect-substitutionᴮ H B v x W -reflect-weakeningᴱ : ∀ H M {H′ Γ} → (H ⊑ H′) → Warningᴱ H′ (typeCheckᴱ H′ Γ M) → Warningᴱ H (typeCheckᴱ H Γ M) -reflect-weakeningᴮ : ∀ H B {H′ Γ} → (H ⊑ H′) → Warningᴮ H′ (typeCheckᴮ H′ Γ B) → Warningᴮ H (typeCheckᴮ H Γ B) +reflect-weakeningᴱ : ∀ Γ H M {H′} → (H ⊑ H′) → Warningᴱ H′ (typeCheckᴱ H′ Γ M) → Warningᴱ H (typeCheckᴱ H Γ M) +reflect-weakeningᴮ : ∀ Γ H B {H′} → (H ⊑ H′) → Warningᴮ H′ (typeCheckᴮ H′ Γ B) → Warningᴮ H (typeCheckᴮ H Γ B) -reflect-weakeningᴱ H (var x) h (UnboundVariable p) = (UnboundVariable p) -reflect-weakeningᴱ H (val (addr a)) h (UnallocatedAddress p) = UnallocatedAddress (lookup-⊑-nothing a h p) -reflect-weakeningᴱ H (M $ N) h (FunctionCallMismatch p) with heap-weakeningᴱ H M h | heap-weakeningᴱ H N h -reflect-weakeningᴱ H (M $ N) h (FunctionCallMismatch p) | ok q₁ | ok q₂ = FunctionCallMismatch (λ r → p (trans (cong src (sym q₁)) (trans r q₂))) -reflect-weakeningᴱ H (M $ N) h (FunctionCallMismatch p) | warning W | _ = app₁ W -reflect-weakeningᴱ H (M $ N) h (FunctionCallMismatch p) | _ | warning W = app₂ W -reflect-weakeningᴱ H (M $ N) h (app₁ W) = app₁ (reflect-weakeningᴱ H M h W) -reflect-weakeningᴱ H (M $ N) h (app₂ W) = app₂ (reflect-weakeningᴱ H N h W) -reflect-weakeningᴱ H (binexp M op N) h (BinOpMismatch₁ p) with heap-weakeningᴱ H M h -reflect-weakeningᴱ H (binexp M op N) h (BinOpMismatch₁ p) | ok q = BinOpMismatch₁ (subst₁ (BinOpWarning op) (sym q) p) -reflect-weakeningᴱ H (binexp M op N) h (BinOpMismatch₁ p) | warning W = bin₁ W -reflect-weakeningᴱ H (binexp M op N) h (BinOpMismatch₂ p) with heap-weakeningᴱ H N h -reflect-weakeningᴱ H (binexp M op N) h (BinOpMismatch₂ p) | ok q = BinOpMismatch₂ (subst₁ (BinOpWarning op) (sym q) p) -reflect-weakeningᴱ H (binexp M op N) h (BinOpMismatch₂ p) | warning W = bin₂ W -reflect-weakeningᴱ H (binexp M op N) h (bin₁ W′) = bin₁ (reflect-weakeningᴱ H M h W′) -reflect-weakeningᴱ H (binexp M op N) h (bin₂ W′) = bin₂ (reflect-weakeningᴱ H N h W′) -reflect-weakeningᴱ H (function f ⟨ var y ∈ T ⟩∈ U is B end) h (FunctionDefnMismatch p) with heap-weakeningᴮ H B h -reflect-weakeningᴱ H (function f ⟨ var y ∈ T ⟩∈ U is B end) h (FunctionDefnMismatch p) | ok q = FunctionDefnMismatch (λ r → p (trans r q)) -reflect-weakeningᴱ H (function f ⟨ var y ∈ T ⟩∈ U is B end) h (FunctionDefnMismatch p) | warning W = function₁ W -reflect-weakeningᴱ H (function f ⟨ var y ∈ T ⟩∈ U is B end) h (function₁ W) = function₁ (reflect-weakeningᴮ H B h W) -reflect-weakeningᴱ H (block var b ∈ T is B end) h (BlockMismatch p) with heap-weakeningᴮ H B h -reflect-weakeningᴱ H (block var b ∈ T is B end) h (BlockMismatch p) | ok q = BlockMismatch (λ r → p (trans r q)) -reflect-weakeningᴱ H (block var b ∈ T is B end) h (BlockMismatch p) | warning W = block₁ W -reflect-weakeningᴱ H (block var b ∈ T is B end) h (block₁ W) = block₁ (reflect-weakeningᴮ H B h W) +reflect-weakeningᴱ Γ H (var x) h (UnboundVariable p) = (UnboundVariable p) +reflect-weakeningᴱ Γ H (val (addr a)) h (UnallocatedAddress p) = UnallocatedAddress (lookup-⊑-nothing a h p) +reflect-weakeningᴱ Γ H (M $ N) h (FunctionCallMismatch p) = FunctionCallMismatch (heap-weakeningᴱ Γ H N h (any-src-≮: p (heap-weakeningᴱ Γ H M h (src-any-≮: p)))) +reflect-weakeningᴱ Γ H (M $ N) h (app₁ W) = app₁ (reflect-weakeningᴱ Γ H M h W) +reflect-weakeningᴱ Γ H (M $ N) h (app₂ W) = app₂ (reflect-weakeningᴱ Γ H N h W) +reflect-weakeningᴱ Γ H (binexp M op N) h (BinOpMismatch₁ p) = BinOpMismatch₁ (heap-weakeningᴱ Γ H M h p) +reflect-weakeningᴱ Γ H (binexp M op N) h (BinOpMismatch₂ p) = BinOpMismatch₂ (heap-weakeningᴱ Γ H N h p) +reflect-weakeningᴱ Γ H (binexp M op N) h (bin₁ W′) = bin₁ (reflect-weakeningᴱ Γ H M h W′) +reflect-weakeningᴱ Γ H (binexp M op N) h (bin₂ W′) = bin₂ (reflect-weakeningᴱ Γ H N h W′) +reflect-weakeningᴱ Γ H (function f ⟨ var y ∈ T ⟩∈ U is B end) h (FunctionDefnMismatch p) = FunctionDefnMismatch (heap-weakeningᴮ (Γ ⊕ y ↦ T) H B h p) +reflect-weakeningᴱ Γ H (function f ⟨ var y ∈ T ⟩∈ U is B end) h (function₁ W) = function₁ (reflect-weakeningᴮ (Γ ⊕ y ↦ T) H B h W) +reflect-weakeningᴱ Γ H (block var b ∈ T is B end) h (BlockMismatch p) = BlockMismatch (heap-weakeningᴮ Γ H B h p) +reflect-weakeningᴱ Γ H (block var b ∈ T is B end) h (block₁ W) = block₁ (reflect-weakeningᴮ Γ H B h W) -reflect-weakeningᴮ H (return M ∙ B) h (return W) = return (reflect-weakeningᴱ H M h W) -reflect-weakeningᴮ H (local var y ∈ T ← M ∙ B) h (LocalVarMismatch p) with heap-weakeningᴱ H M h -reflect-weakeningᴮ H (local var y ∈ T ← M ∙ B) h (LocalVarMismatch p) | ok q = LocalVarMismatch (λ r → p (trans r q)) -reflect-weakeningᴮ H (local var y ∈ T ← M ∙ B) h (LocalVarMismatch p) | warning W = local₁ W -reflect-weakeningᴮ H (local var y ∈ T ← M ∙ B) h (local₁ W) = local₁ (reflect-weakeningᴱ H M h W) -reflect-weakeningᴮ H (local var y ∈ T ← M ∙ B) h (local₂ W) = local₂ (reflect-weakeningᴮ H B h W) -reflect-weakeningᴮ H (function f ⟨ var x ∈ T ⟩∈ U is C end ∙ B) h (FunctionDefnMismatch p) with heap-weakeningᴮ H C h -reflect-weakeningᴮ H (function f ⟨ var x ∈ T ⟩∈ U is C end ∙ B) h (FunctionDefnMismatch p) | ok q = FunctionDefnMismatch (λ r → p (trans r q)) -reflect-weakeningᴮ H (function f ⟨ var x ∈ T ⟩∈ U is C end ∙ B) h (FunctionDefnMismatch p) | warning W = function₁ W -reflect-weakeningᴮ H (function f ⟨ var x ∈ T ⟩∈ U is C end ∙ B) h (function₁ W) = function₁ (reflect-weakeningᴮ H C h W) -reflect-weakeningᴮ H (function f ⟨ var x ∈ T ⟩∈ U is C end ∙ B) h (function₂ W) = function₂ (reflect-weakeningᴮ H B h W) +reflect-weakeningᴮ Γ H (return M ∙ B) h (return W) = return (reflect-weakeningᴱ Γ H M h W) +reflect-weakeningᴮ Γ H (local var y ∈ T ← M ∙ B) h (LocalVarMismatch p) = LocalVarMismatch (heap-weakeningᴱ Γ H M h p) +reflect-weakeningᴮ Γ H (local var y ∈ T ← M ∙ B) h (local₁ W) = local₁ (reflect-weakeningᴱ Γ H M h W) +reflect-weakeningᴮ Γ H (local var y ∈ T ← M ∙ B) h (local₂ W) = local₂ (reflect-weakeningᴮ (Γ ⊕ y ↦ T) H B h W) +reflect-weakeningᴮ Γ H (function f ⟨ var x ∈ T ⟩∈ U is C end ∙ B) h (FunctionDefnMismatch p) = FunctionDefnMismatch (heap-weakeningᴮ (Γ ⊕ x ↦ T) H C h p) +reflect-weakeningᴮ Γ H (function f ⟨ var x ∈ T ⟩∈ U is C end ∙ B) h (function₁ W) = function₁ (reflect-weakeningᴮ (Γ ⊕ x ↦ T) H C h W) +reflect-weakeningᴮ Γ H (function f ⟨ var x ∈ T ⟩∈ U is C end ∙ B) h (function₂ W) = function₂ (reflect-weakeningᴮ (Γ ⊕ f ↦ (T ⇒ U)) H B h W) reflect-weakeningᴼ : ∀ H O {H′} → (H ⊑ H′) → Warningᴼ H′ (typeCheckᴼ H′ O) → Warningᴼ H (typeCheckᴼ H O) -reflect-weakeningᴼ H (just (function f ⟨ var x ∈ T ⟩∈ U is B end)) h (FunctionDefnMismatch p) with heap-weakeningᴮ H B h -reflect-weakeningᴼ H (just (function f ⟨ var x ∈ T ⟩∈ U is B end)) h (FunctionDefnMismatch p) | ok q = FunctionDefnMismatch (λ r → p (trans r q)) -reflect-weakeningᴼ H (just (function f ⟨ var x ∈ T ⟩∈ U is B end)) h (FunctionDefnMismatch p) | warning W = function₁ W -reflect-weakeningᴼ H (just (function f ⟨ var x ∈ T ⟩∈ U is B end)) h (function₁ W′) = function₁ (reflect-weakeningᴮ H B h W′) +reflect-weakeningᴼ H (just function f ⟨ var x ∈ T ⟩∈ U is B end) h (FunctionDefnMismatch p) = FunctionDefnMismatch (heap-weakeningᴮ (x ↦ T) H B h p) +reflect-weakeningᴼ H (just function f ⟨ var x ∈ T ⟩∈ U is B end) h (function₁ W) = function₁ (reflect-weakeningᴮ (x ↦ T) H B h W) -reflectᴱ : ∀ H M {H′ M′} → (H ⊢ M ⟶ᴱ M′ ⊣ H′) → Warningᴱ H′ (typeCheckᴱ H′ ∅ M′) → Warningᴴᴱ H (typeCheckᴴᴱ H ∅ M) -reflectᴮ : ∀ H B {H′ B′} → (H ⊢ B ⟶ᴮ B′ ⊣ H′) → Warningᴮ H′ (typeCheckᴮ H′ ∅ B′) → Warningᴴᴮ H (typeCheckᴴᴮ H ∅ B) +reflectᴱ : ∀ H M {H′ M′} → (H ⊢ M ⟶ᴱ M′ ⊣ H′) → Warningᴱ H′ (typeCheckᴱ H′ ∅ M′) → Either (Warningᴱ H (typeCheckᴱ H ∅ M)) (Warningᴴ H (typeCheckᴴ H)) +reflectᴮ : ∀ H B {H′ B′} → (H ⊢ B ⟶ᴮ B′ ⊣ H′) → Warningᴮ H′ (typeCheckᴮ H′ ∅ B′) → Either (Warningᴮ H (typeCheckᴮ H ∅ B)) (Warningᴴ H (typeCheckᴴ H)) -reflectᴱ H (M $ N) (app₁ s) (FunctionCallMismatch p) with preservationᴱ H M s | heap-weakeningᴱ H N (rednᴱ⊑ s) -reflectᴱ H (M $ N) (app₁ s) (FunctionCallMismatch p) | ok q | ok q′ = expr (FunctionCallMismatch (λ r → p (trans (trans (cong src (sym q)) r) q′))) -reflectᴱ H (M $ N) (app₁ s) (FunctionCallMismatch p) | warning (expr W) | _ = expr (app₁ W) -reflectᴱ H (M $ N) (app₁ s) (FunctionCallMismatch p) | warning (heap W) | _ = heap W -reflectᴱ H (M $ N) (app₁ s) (FunctionCallMismatch p) | _ | warning W = expr (app₂ W) -reflectᴱ H (M $ N) (app₁ s) (app₁ W′) with reflectᴱ H M s W′ -reflectᴱ H (M $ N) (app₁ s) (app₁ W′) | heap W = heap W -reflectᴱ H (M $ N) (app₁ s) (app₁ W′) | expr W = expr (app₁ W) -reflectᴱ H (M $ N) (app₁ s) (app₂ W′) = expr (app₂ (reflect-weakeningᴱ H N (rednᴱ⊑ s) W′)) -reflectᴱ H (M $ N) (app₂ p s) (FunctionCallMismatch p′) with heap-weakeningᴱ H (val p) (rednᴱ⊑ s) | preservationᴱ H N s -reflectᴱ H (M $ N) (app₂ p s) (FunctionCallMismatch p′) | ok q | ok q′ = expr (FunctionCallMismatch (λ r → p′ (trans (trans (cong src (sym q)) r) q′))) -reflectᴱ H (M $ N) (app₂ p s) (FunctionCallMismatch p′) | warning W | _ = expr (app₁ W) -reflectᴱ H (M $ N) (app₂ p s) (FunctionCallMismatch p′) | _ | warning (expr W) = expr (app₂ W) -reflectᴱ H (M $ N) (app₂ p s) (FunctionCallMismatch p′) | _ | warning (heap W) = heap W -reflectᴱ H (M $ N) (app₂ p s) (app₁ W′) = expr (app₁ (reflect-weakeningᴱ H M (rednᴱ⊑ s) W′)) -reflectᴱ H (M $ N) (app₂ p s) (app₂ W′) with reflectᴱ H N s W′ -reflectᴱ H (M $ N) (app₂ p s) (app₂ W′) | heap W = heap W -reflectᴱ H (M $ N) (app₂ p s) (app₂ W′) | expr W = expr (app₂ W) -reflectᴱ H (val (addr a) $ N) (beta (function f ⟨ var x ∈ T ⟩∈ U is B end) v refl p) (BlockMismatch q) with remember (typeOfⱽ H v) -reflectᴱ H (val (addr a) $ N) (beta (function f ⟨ var x ∈ T ⟩∈ U is B end) v refl p) (BlockMismatch q) | (just S , r) with S ≡ᵀ T -reflectᴱ H (val (addr a) $ N) (beta (function f ⟨ var x ∈ T ⟩∈ U is B end) v refl p) (BlockMismatch q) | (just T , r) | yes refl = heap (addr a p (FunctionDefnMismatch (λ s → q (trans s (substitutivityᴮ H B v x (sym r)))))) -reflectᴱ H (val (addr a) $ N) (beta (function f ⟨ var x ∈ T ⟩∈ U is B end) v refl p) (BlockMismatch q) | (just S , r) | no s = expr (FunctionCallMismatch (λ t → s (trans (cong orNone (sym r)) (trans (sym t) (cong src (cong orNone (cong typeOfᴹᴼ p))))))) -reflectᴱ H (val (addr a) $ N) (beta (function f ⟨ var x ∈ T ⟩∈ U is B end) v refl p) (BlockMismatch q) | (nothing , r) with typeOf-val-not-none v -reflectᴱ H (val (addr a) $ N) (beta (function f ⟨ var x ∈ T ⟩∈ U is B end) v refl p) (BlockMismatch q) | (nothing , r) | ok s = CONTRADICTION (s (cong orNone (sym r))) -reflectᴱ H (val (addr a) $ N) (beta (function f ⟨ var x ∈ T ⟩∈ U is B end) v refl p) (BlockMismatch q) | (nothing , r) | warning W = expr (app₂ W) -reflectᴱ H (val (addr a) $ N) (beta (function f ⟨ var x ∈ T ⟩∈ U is B end) v refl p) (block₁ W′) with remember (typeOfⱽ H v) -reflectᴱ H (val (addr a) $ N) (beta (function f ⟨ var x ∈ T ⟩∈ U is B end) v refl p) (block₁ W′) | (just S , q) with S ≡ᵀ T -reflectᴱ H (val (addr a) $ N) (beta (function f ⟨ var x ∈ T ⟩∈ U is B end) v refl p) (block₁ W′) | (just T , q) | yes refl = heap (addr a p (function₁ (reflect-substitutionᴮ H B v x (sym q) W′))) -reflectᴱ H (val (addr a) $ N) (beta (function f ⟨ var x ∈ T ⟩∈ U is B end) v refl p) (block₁ W′) | (just S , q) | no r = expr (FunctionCallMismatch (λ s → r (trans (cong orNone (sym q)) (trans (sym s) (cong src (cong orNone (cong typeOfᴹᴼ p))))))) -reflectᴱ H (val (addr a) $ N) (beta (function f ⟨ var x ∈ T ⟩∈ U is B end) v refl p) (block₁ W′) | (nothing , q) with typeOf-val-not-none v -reflectᴱ H (val (addr a) $ N) (beta (function f ⟨ var x ∈ T ⟩∈ U is B end) v refl p) (block₁ W′) | (nothing , q) | ok r = CONTRADICTION (r (cong orNone (sym q))) -reflectᴱ H (val (addr a) $ N) (beta (function f ⟨ var x ∈ T ⟩∈ U is B end) v refl p) (block₁ W′) | (nothing , q) | warning W = expr (app₂ W) -reflectᴱ H (block var b ∈ T is B end) (block s) (BlockMismatch p) with preservationᴮ H B s -reflectᴱ H (block var b ∈ T is B end) (block s) (BlockMismatch p) | ok q = expr (BlockMismatch (λ r → p (trans r q))) -reflectᴱ H (block var b ∈ T is B end) (block s) (BlockMismatch p) | warning (heap W) = heap W -reflectᴱ H (block var b ∈ T is B end) (block s) (BlockMismatch p) | warning (block W) = expr (block₁ W) -reflectᴱ H (block var b ∈ T is B end) (block s) (block₁ W′) with reflectᴮ H B s W′ -reflectᴱ H (block var b ∈ T is B end) (block s) (block₁ W′) | heap W = heap W -reflectᴱ H (block var b ∈ T is B end) (block s) (block₁ W′) | block W = expr (block₁ W) -reflectᴱ H (block var b ∈ T is B end) (return v) W′ = expr (block₁ (return W′)) +reflectᴱ H (M $ N) (app₁ s) (FunctionCallMismatch p) = cond (Left ∘ FunctionCallMismatch ∘ heap-weakeningᴱ ∅ H N (rednᴱ⊑ s) ∘ any-src-≮: p) (Left ∘ app₁) (reflect-subtypingᴱ H M s (src-any-≮: p)) +reflectᴱ H (M $ N) (app₁ s) (app₁ W′) = mapL app₁ (reflectᴱ H M s W′) +reflectᴱ H (M $ N) (app₁ s) (app₂ W′) = Left (app₂ (reflect-weakeningᴱ ∅ H N (rednᴱ⊑ s) W′)) +reflectᴱ H (M $ N) (app₂ p s) (FunctionCallMismatch q) = cond (λ r → Left (FunctionCallMismatch (any-src-≮: r (heap-weakeningᴱ ∅ H M (rednᴱ⊑ s) (src-any-≮: r))))) (Left ∘ app₂) (reflect-subtypingᴱ H N s q) +reflectᴱ H (M $ N) (app₂ p s) (app₁ W′) = Left (app₁ (reflect-weakeningᴱ ∅ H M (rednᴱ⊑ s) W′)) +reflectᴱ H (M $ N) (app₂ p s) (app₂ W′) = mapL app₂ (reflectᴱ H N s W′) +reflectᴱ H (val (addr a) $ N) (beta (function f ⟨ var x ∈ T ⟩∈ U is B end) v refl p) (BlockMismatch q) with substitutivityᴮ H B v x q +reflectᴱ H (val (addr a) $ N) (beta (function f ⟨ var x ∈ T ⟩∈ U is B end) v refl p) (BlockMismatch q) | Left r = Right (addr a p (FunctionDefnMismatch r)) +reflectᴱ H (val (addr a) $ N) (beta (function f ⟨ var x ∈ T ⟩∈ U is B end) v refl p) (BlockMismatch q) | Right r = Left (FunctionCallMismatch (≮:-trans-≡ r ((cong src (cong orAny (cong typeOfᴹᴼ (sym p))))))) +reflectᴱ H (val (addr a) $ N) (beta (function f ⟨ var x ∈ T ⟩∈ U is B end) v refl p) (block₁ W′) with reflect-substitutionᴮ _ B v x W′ +reflectᴱ H (val (addr a) $ N) (beta (function f ⟨ var x ∈ T ⟩∈ U is B end) v refl p) (block₁ W′) | Left W = Right (addr a p (function₁ W)) +reflectᴱ H (val (addr a) $ N) (beta (function f ⟨ var x ∈ T ⟩∈ U is B end) v refl p) (block₁ W′) | Right (Left W) = Left (app₂ W) +reflectᴱ H (val (addr a) $ N) (beta (function f ⟨ var x ∈ T ⟩∈ U is B end) v refl p) (block₁ W′) | Right (Right q) = Left (FunctionCallMismatch (≮:-trans-≡ q (cong src (cong orAny (cong typeOfᴹᴼ (sym p)))))) +reflectᴱ H (block var b ∈ T is B end) (block s) (BlockMismatch p) = Left (cond BlockMismatch block₁ (reflect-subtypingᴮ H B s p)) +reflectᴱ H (block var b ∈ T is B end) (block s) (block₁ W′) = mapL block₁ (reflectᴮ H B s W′) +reflectᴱ H (block var b ∈ T is B end) (return v) W′ = Left (block₁ (return W′)) reflectᴱ H (function f ⟨ var x ∈ T ⟩∈ U is B end) (function a defn) (UnallocatedAddress ()) reflectᴱ H (binexp M op N) (binOp₀ ()) (UnallocatedAddress p) -reflectᴱ H (binexp M op N) (binOp₁ s) (BinOpMismatch₁ p) with preservationᴱ H M s -reflectᴱ H (binexp M op N) (binOp₁ s) (BinOpMismatch₁ p) | ok q = expr (BinOpMismatch₁ (subst₁ (BinOpWarning op) (sym q) p)) -reflectᴱ H (binexp M op N) (binOp₁ s) (BinOpMismatch₁ p) | warning (heap W) = heap W -reflectᴱ H (binexp M op N) (binOp₁ s) (BinOpMismatch₁ p) | warning (expr W) = expr (bin₁ W) -reflectᴱ H (binexp M op N) (binOp₁ s) (BinOpMismatch₂ p) with heap-weakeningᴱ H N (rednᴱ⊑ s) -reflectᴱ H (binexp M op N) (binOp₁ s) (BinOpMismatch₂ p) | ok q = expr (BinOpMismatch₂ ((subst₁ (BinOpWarning op) (sym q) p))) -reflectᴱ H (binexp M op N) (binOp₁ s) (BinOpMismatch₂ p) | warning W = expr (bin₂ W) -reflectᴱ H (binexp M op N) (binOp₁ s) (bin₁ W′) with reflectᴱ H M s W′ -reflectᴱ H (binexp M op N) (binOp₁ s) (bin₁ W′) | heap W = heap W -reflectᴱ H (binexp M op N) (binOp₁ s) (bin₁ W′) | expr W = expr (bin₁ W) -reflectᴱ H (binexp M op N) (binOp₁ s) (bin₂ W′) = expr (bin₂ (reflect-weakeningᴱ H N (rednᴱ⊑ s) W′)) -reflectᴱ H (binexp M op N) (binOp₂ s) (BinOpMismatch₁ p) with heap-weakeningᴱ H M (rednᴱ⊑ s) -reflectᴱ H (binexp M op N) (binOp₂ s) (BinOpMismatch₁ p) | ok q = expr (BinOpMismatch₁ (subst₁ (BinOpWarning op) (sym q) p)) -reflectᴱ H (binexp M op N) (binOp₂ s) (BinOpMismatch₁ p) | warning W = expr (bin₁ W) -reflectᴱ H (binexp M op N) (binOp₂ s) (BinOpMismatch₂ p) with preservationᴱ H N s -reflectᴱ H (binexp M op N) (binOp₂ s) (BinOpMismatch₂ p) | ok q = expr (BinOpMismatch₂ (subst₁ (BinOpWarning op) (sym q) p)) -reflectᴱ H (binexp M op N) (binOp₂ s) (BinOpMismatch₂ p) | warning (heap W) = heap W -reflectᴱ H (binexp M op N) (binOp₂ s) (BinOpMismatch₂ p) | warning (expr W) = expr (bin₂ W) -reflectᴱ H (binexp M op N) (binOp₂ s) (bin₁ W′) = expr (bin₁ (reflect-weakeningᴱ H M (rednᴱ⊑ s) W′)) -reflectᴱ H (binexp M op N) (binOp₂ s) (bin₂ W′) with reflectᴱ H N s W′ -reflectᴱ H (binexp M op N) (binOp₂ s) (bin₂ W′) | heap W = heap W -reflectᴱ H (binexp M op N) (binOp₂ s) (bin₂ W′) | expr W = expr (bin₂ W) +reflectᴱ H (binexp M op N) (binOp₁ s) (BinOpMismatch₁ p) = Left (cond BinOpMismatch₁ bin₁ (reflect-subtypingᴱ H M s p)) +reflectᴱ H (binexp M op N) (binOp₁ s) (BinOpMismatch₂ p) = Left (BinOpMismatch₂ (heap-weakeningᴱ ∅ H N (rednᴱ⊑ s) p)) +reflectᴱ H (binexp M op N) (binOp₁ s) (bin₁ W′) = mapL bin₁ (reflectᴱ H M s W′) +reflectᴱ H (binexp M op N) (binOp₁ s) (bin₂ W′) = Left (bin₂ (reflect-weakeningᴱ ∅ H N (rednᴱ⊑ s) W′)) +reflectᴱ H (binexp M op N) (binOp₂ s) (BinOpMismatch₁ p) = Left (BinOpMismatch₁ (heap-weakeningᴱ ∅ H M (rednᴱ⊑ s) p)) +reflectᴱ H (binexp M op N) (binOp₂ s) (BinOpMismatch₂ p) = Left (cond BinOpMismatch₂ bin₂ (reflect-subtypingᴱ H N s p)) +reflectᴱ H (binexp M op N) (binOp₂ s) (bin₁ W′) = Left (bin₁ (reflect-weakeningᴱ ∅ H M (rednᴱ⊑ s) W′)) +reflectᴱ H (binexp M op N) (binOp₂ s) (bin₂ W′) = mapL bin₂ (reflectᴱ H N s W′) -reflectᴮ H (local var x ∈ T ← M ∙ B) (local s) (LocalVarMismatch p) with preservationᴱ H M s -reflectᴮ H (local var x ∈ T ← M ∙ B) (local s) (LocalVarMismatch p) | ok q = block (LocalVarMismatch (λ r → p (trans r q))) -reflectᴮ H (local var x ∈ T ← M ∙ B) (local s) (LocalVarMismatch p) | warning (expr W) = block (local₁ W) -reflectᴮ H (local var x ∈ T ← M ∙ B) (local s) (LocalVarMismatch p) | warning (heap W) = heap W -reflectᴮ H (local var x ∈ T ← M ∙ B) (local s) (local₁ W′) with reflectᴱ H M s W′ -reflectᴮ H (local var x ∈ T ← M ∙ B) (local s) (local₁ W′) | heap W = heap W -reflectᴮ H (local var x ∈ T ← M ∙ B) (local s) (local₁ W′) | expr W = block (local₁ W) -reflectᴮ H (local var x ∈ T ← M ∙ B) (local s) (local₂ W′) = block (local₂ (reflect-weakeningᴮ H B (rednᴱ⊑ s) W′)) -reflectᴮ H (local var x ∈ T ← M ∙ B) (subst v) W′ with remember (typeOfⱽ H v) -reflectᴮ H (local var x ∈ T ← M ∙ B) (subst v) W′ | (just S , p) with S ≡ᵀ T -reflectᴮ H (local var x ∈ T ← M ∙ B) (subst v) W′ | (just T , p) | yes refl = block (local₂ (reflect-substitutionᴮ H B v x (sym p) W′)) -reflectᴮ H (local var x ∈ T ← M ∙ B) (subst v) W′ | (just S , p) | no q = block (LocalVarMismatch (λ r → q (trans (cong orNone (sym p)) (sym r)))) -reflectᴮ H (local var x ∈ T ← M ∙ B) (subst v) W′ | (nothing , p) with typeOf-val-not-none v -reflectᴮ H (local var x ∈ T ← M ∙ B) (subst v) W′ | (nothing , p) | ok r = CONTRADICTION (r (cong orNone (sym p))) -reflectᴮ H (local var x ∈ T ← M ∙ B) (subst v) W′ | (nothing , p) | warning W = block (local₁ W) -reflectᴮ H (function f ⟨ var y ∈ T ⟩∈ U is C end ∙ B) (function a defn) W′ = block (function₂ (reflect-weakeningᴮ H B (snoc defn) (reflect-substitutionᴮ _ B (addr a) f refl W′))) -reflectᴮ H (return M ∙ B) (return s) (return W′) with reflectᴱ H M s W′ -reflectᴮ H (return M ∙ B) (return s) (return W′) | heap W = heap W -reflectᴮ H (return M ∙ B) (return s) (return W′) | expr W = block (return W) +reflectᴮ H (local var x ∈ T ← M ∙ B) (local s) (LocalVarMismatch p) = Left (cond LocalVarMismatch local₁ (reflect-subtypingᴱ H M s p)) +reflectᴮ H (local var x ∈ T ← M ∙ B) (local s) (local₁ W′) = mapL local₁ (reflectᴱ H M s W′) +reflectᴮ H (local var x ∈ T ← M ∙ B) (local s) (local₂ W′) = Left (local₂ (reflect-weakeningᴮ (x ↦ T) H B (rednᴱ⊑ s) W′)) +reflectᴮ H (local var x ∈ T ← M ∙ B) (subst v) W′ = Left (cond local₂ (cond local₁ LocalVarMismatch) (reflect-substitutionᴮ H B v x W′)) +reflectᴮ H (function f ⟨ var y ∈ T ⟩∈ U is C end ∙ B) (function a defn) W′ with reflect-substitutionᴮ _ B (addr a) f W′ +reflectᴮ H (function f ⟨ var y ∈ T ⟩∈ U is C end ∙ B) (function a defn) W′ | Left W = Left (function₂ (reflect-weakeningᴮ (f ↦ (T ⇒ U)) H B (snoc defn) W)) +reflectᴮ H (function f ⟨ var y ∈ T ⟩∈ U is C end ∙ B) (function a defn) W′ | Right (Left (UnallocatedAddress ())) +reflectᴮ H (function f ⟨ var y ∈ T ⟩∈ U is C end ∙ B) (function a defn) W′ | Right (Right p) = CONTRADICTION (≮:-refl p) +reflectᴮ H (return M ∙ B) (return s) (return W′) = mapL return (reflectᴱ H M s W′) -reflectᴴᴱ : ∀ H M {H′ M′} → (H ⊢ M ⟶ᴱ M′ ⊣ H′) → Warningᴴᴱ H′ (typeCheckᴴᴱ H′ ∅ M′) → Warningᴴᴱ H (typeCheckᴴᴱ H ∅ M) -reflectᴴᴮ : ∀ H B {H′ B′} → (H ⊢ B ⟶ᴮ B′ ⊣ H′) → Warningᴴᴮ H′ (typeCheckᴴᴮ H′ ∅ B′) → Warningᴴᴮ H (typeCheckᴴᴮ H ∅ B) +reflectᴴᴱ : ∀ H M {H′ M′} → (H ⊢ M ⟶ᴱ M′ ⊣ H′) → Warningᴴ H′ (typeCheckᴴ H′) → Either (Warningᴱ H (typeCheckᴱ H ∅ M)) (Warningᴴ H (typeCheckᴴ H)) +reflectᴴᴮ : ∀ H B {H′ B′} → (H ⊢ B ⟶ᴮ B′ ⊣ H′) → Warningᴴ H′ (typeCheckᴴ H′) → Either (Warningᴮ H (typeCheckᴮ H ∅ B)) (Warningᴴ H (typeCheckᴴ H)) -reflectᴴᴱ H M s (expr W′) = reflectᴱ H M s W′ -reflectᴴᴱ H (function f ⟨ var x ∈ T ⟩∈ U is B end) (function a p) (heap (addr b refl W′)) with b ≡ᴬ a -reflectᴴᴱ H (function f ⟨ var x ∈ T ⟩∈ U is B end) (function a defn) (heap (addr a refl (FunctionDefnMismatch p))) | yes refl with heap-weakeningᴮ H B (snoc defn) -reflectᴴᴱ H (function f ⟨ var x ∈ T ⟩∈ U is B end) (function a defn) (heap (addr a refl (FunctionDefnMismatch p))) | yes refl | ok r = expr (FunctionDefnMismatch λ q → p (trans q r)) -reflectᴴᴱ H (function f ⟨ var x ∈ T ⟩∈ U is B end) (function a defn) (heap (addr a refl (FunctionDefnMismatch p))) | yes refl | warning W = expr (function₁ W) -reflectᴴᴱ H (function f ⟨ var x ∈ T ⟩∈ U is B end) (function a defn) (heap (addr a refl (function₁ W′))) | yes refl = expr (function₁ (reflect-weakeningᴮ H B (snoc defn) W′)) -reflectᴴᴱ H (function f ⟨ var x ∈ T ⟩∈ U is B end) (function a p) (heap (addr b refl W′)) | no r = heap (addr b (lookup-not-allocated p r) (reflect-weakeningᴼ H _ (snoc p) W′)) -reflectᴴᴱ H (M $ N) (app₁ s) (heap W′) with reflectᴴᴱ H M s (heap W′) -reflectᴴᴱ H (M $ N) (app₁ s) (heap W′) | heap W = heap W -reflectᴴᴱ H (M $ N) (app₁ s) (heap W′) | expr W = expr (app₁ W) -reflectᴴᴱ H (M $ N) (app₂ p s) (heap W′) with reflectᴴᴱ H N s (heap W′) -reflectᴴᴱ H (M $ N) (app₂ p s) (heap W′) | heap W = heap W -reflectᴴᴱ H (M $ N) (app₂ p s) (heap W′) | expr W = expr (app₂ W) -reflectᴴᴱ H (M $ N) (beta O v p q) (heap W′) = heap W′ -reflectᴴᴱ H (block var b ∈ T is B end) (block s) (heap W′) with reflectᴴᴮ H B s (heap W′) -reflectᴴᴱ H (block var b ∈ T is B end) (block s) (heap W′) | heap W = heap W -reflectᴴᴱ H (block var b ∈ T is B end) (block s) (heap W′) | block W = expr (block₁ W) -reflectᴴᴱ H (block var b ∈ T is return N ∙ B end) (return v) (heap W′) = heap W′ -reflectᴴᴱ H (block var b ∈ T is done end) done (heap W′) = heap W′ -reflectᴴᴱ H (binexp M op N) (binOp₀ s) (heap W′) = heap W′ -reflectᴴᴱ H (binexp M op N) (binOp₁ s) (heap W′) with reflectᴴᴱ H M s (heap W′) -reflectᴴᴱ H (binexp M op N) (binOp₁ s) (heap W′) | heap W = heap W -reflectᴴᴱ H (binexp M op N) (binOp₁ s) (heap W′) | expr W = expr (bin₁ W) -reflectᴴᴱ H (binexp M op N) (binOp₂ s) (heap W′) with reflectᴴᴱ H N s (heap W′) -reflectᴴᴱ H (binexp M op N) (binOp₂ s) (heap W′) | heap W = heap W -reflectᴴᴱ H (binexp M op N) (binOp₂ s) (heap W′) | expr W = expr (bin₂ W) +reflectᴴᴱ H (M $ N) (app₁ s) W = mapL app₁ (reflectᴴᴱ H M s W) +reflectᴴᴱ H (M $ N) (app₂ v s) W = mapL app₂ (reflectᴴᴱ H N s W) +reflectᴴᴱ H (M $ N) (beta O v refl p) W = Right W +reflectᴴᴱ H (function f ⟨ var x ∈ T ⟩∈ U is B end) (function a p) (addr b refl W) with b ≡ᴬ a +reflectᴴᴱ H (function f ⟨ var x ∈ T ⟩∈ U is B end) (function a defn) (addr b refl (FunctionDefnMismatch p)) | yes refl = Left (FunctionDefnMismatch (heap-weakeningᴮ (x ↦ T) H B (snoc defn) p)) +reflectᴴᴱ H (function f ⟨ var x ∈ T ⟩∈ U is B end) (function a defn) (addr b refl (function₁ W)) | yes refl = Left (function₁ (reflect-weakeningᴮ (x ↦ T) H B (snoc defn) W)) +reflectᴴᴱ H (function f ⟨ var x ∈ T ⟩∈ U is B end) (function a p) (addr b refl W) | no q = Right (addr b (lookup-not-allocated p q) (reflect-weakeningᴼ H _ (snoc p) W)) +reflectᴴᴱ H (block var b ∈ T is B end) (block s) W = mapL block₁ (reflectᴴᴮ H B s W) +reflectᴴᴱ H (block var b ∈ T is return (val v) ∙ B end) (return v) W = Right W +reflectᴴᴱ H (block var b ∈ T is done end) done W = Right W +reflectᴴᴱ H (binexp M op N) (binOp₀ s) W = Right W +reflectᴴᴱ H (binexp M op N) (binOp₁ s) W = mapL bin₁ (reflectᴴᴱ H M s W) +reflectᴴᴱ H (binexp M op N) (binOp₂ s) W = mapL bin₂ (reflectᴴᴱ H N s W) -reflectᴴᴮ H B s (block W′) = reflectᴮ H B s W′ -reflectᴴᴮ H (local var x ∈ T ← M ∙ B) (local s) (heap W′) with reflectᴴᴱ H M s (heap W′) -reflectᴴᴮ H (local var x ∈ T ← M ∙ B) (local s) (heap W′) | heap W = heap W -reflectᴴᴮ H (local var x ∈ T ← M ∙ B) (local s) (heap W′) | expr W = block (local₁ W) -reflectᴴᴮ H (local var x ∈ T ← M ∙ B) (subst v) (heap W′) = heap W′ -reflectᴴᴮ H (function f ⟨ var y ∈ T ⟩∈ U is C end ∙ B) (function a p) (heap (addr b refl W′)) with b ≡ᴬ a -reflectᴴᴮ H (function f ⟨ var y ∈ T ⟩∈ U is C end ∙ B) (function a defn) (heap (addr a refl (FunctionDefnMismatch p))) | yes refl with heap-weakeningᴮ H C (snoc defn) -reflectᴴᴮ H (function f ⟨ var y ∈ T ⟩∈ U is C end ∙ B) (function a defn) (heap (addr a refl (FunctionDefnMismatch p))) | yes refl | ok r = block (FunctionDefnMismatch (λ q → p (trans q r))) -reflectᴴᴮ H (function f ⟨ var y ∈ T ⟩∈ U is C end ∙ B) (function a defn) (heap (addr a refl (FunctionDefnMismatch p))) | yes refl | warning W = block (function₁ W) -reflectᴴᴮ H (function f ⟨ var y ∈ T ⟩∈ U is C end ∙ B) (function a defn) (heap (addr a refl (function₁ W′))) | yes refl = block (function₁ (reflect-weakeningᴮ H C (snoc defn) W′)) -reflectᴴᴮ H (function f ⟨ var y ∈ T ⟩∈ U is C end ∙ B) (function a p) (heap (addr b refl W′)) | no r = heap (addr b (lookup-not-allocated p r) (reflect-weakeningᴼ H _ (snoc p) W′)) -reflectᴴᴮ H (return M ∙ B) (return s) (heap W′) with reflectᴴᴱ H M s (heap W′) -reflectᴴᴮ H (return M ∙ B) (return s) (heap W′) | heap W = heap W -reflectᴴᴮ H (return M ∙ B) (return s) (heap W′) | expr W = block (return W) +reflectᴴᴮ H (function f ⟨ var x ∈ T ⟩∈ U is C end ∙ B) (function a p) (addr b refl W) with b ≡ᴬ a +reflectᴴᴮ H (function f ⟨ var x ∈ T ⟩∈ U is C end ∙ B) (function a defn) (addr b refl (FunctionDefnMismatch p)) | yes refl = Left (FunctionDefnMismatch (heap-weakeningᴮ (x ↦ T) H C (snoc defn) p)) +reflectᴴᴮ H (function f ⟨ var x ∈ T ⟩∈ U is C end ∙ B) (function a defn) (addr b refl (function₁ W)) | yes refl = Left (function₁ (reflect-weakeningᴮ (x ↦ T) H C (snoc defn) W)) +reflectᴴᴮ H (function f ⟨ var x ∈ T ⟩∈ U is C end ∙ B) (function a p) (addr b refl W) | no q = Right (addr b (lookup-not-allocated p q) (reflect-weakeningᴼ H _ (snoc p) W)) +reflectᴴᴮ H (local var x ∈ T ← M ∙ B) (local s) W = mapL local₁ (reflectᴴᴱ H M s W) +reflectᴴᴮ H (local var x ∈ T ← M ∙ B) (subst v) W = Right W +reflectᴴᴮ H (return M ∙ B) (return s) W = mapL return (reflectᴴᴱ H M s W) -reflect* : ∀ H B {H′ B′} → (H ⊢ B ⟶* B′ ⊣ H′) → Warningᴴᴮ H′ (typeCheckᴴᴮ H′ ∅ B′) → Warningᴴᴮ H (typeCheckᴴᴮ H ∅ B) +reflect* : ∀ H B {H′ B′} → (H ⊢ B ⟶* B′ ⊣ H′) → Either (Warningᴮ H′ (typeCheckᴮ H′ ∅ B′)) (Warningᴴ H′ (typeCheckᴴ H′)) → Either (Warningᴮ H (typeCheckᴮ H ∅ B)) (Warningᴴ H (typeCheckᴴ H)) reflect* H B refl W = W -reflect* H B (step s t) W = reflectᴴᴮ H B s (reflect* _ _ t W) +reflect* H B (step s t) W = cond (reflectᴮ H B s) (reflectᴴᴮ H B s) (reflect* _ _ t W) -runtimeBinOpWarning : ∀ H {op} v → BinOpError op (valueType v) → BinOpWarning op (orNone (typeOfⱽ H v)) -runtimeBinOpWarning H v (+ p) = + (λ q → p (mustBeNumber H ∅ v q)) -runtimeBinOpWarning H v (- p) = - (λ q → p (mustBeNumber H ∅ v q)) -runtimeBinOpWarning H v (* p) = * (λ q → p (mustBeNumber H ∅ v q)) -runtimeBinOpWarning H v (/ p) = / (λ q → p (mustBeNumber H ∅ v q)) -runtimeBinOpWarning H v (< p) = < (λ q → p (mustBeNumber H ∅ v q)) -runtimeBinOpWarning H v (> p) = > (λ q → p (mustBeNumber H ∅ v q)) -runtimeBinOpWarning H v (<= p) = <= (λ q → p (mustBeNumber H ∅ v q)) -runtimeBinOpWarning H v (>= p) = >= (λ q → p (mustBeNumber H ∅ v q)) -runtimeBinOpWarning H v (·· p) = ·· (λ q → p (mustBeString H ∅ v q)) +isntNumber : ∀ H v → (valueType v ≢ number) → (typeOfᴱ H ∅ (val v) ≮: number) +isntNumber H nil p = scalar-≢-impl-≮: nil number (λ ()) +isntNumber H (addr a) p with remember (H [ a ]ᴴ) +isntNumber H (addr a) p | (just (function f ⟨ var x ∈ T ⟩∈ U is B end) , q) = ≡-trans-≮: (cong orAny (cong typeOfᴹᴼ q)) (function-≮:-scalar number) +isntNumber H (addr a) p | (nothing , q) = ≡-trans-≮: (cong orAny (cong typeOfᴹᴼ q)) (any-≮:-scalar number) +isntNumber H (number x) p = CONTRADICTION (p refl) +isntNumber H (bool x) p = scalar-≢-impl-≮: boolean number (λ ()) +isntNumber H (string x) p = scalar-≢-impl-≮: string number (λ ()) + +isntString : ∀ H v → (valueType v ≢ string) → (typeOfᴱ H ∅ (val v) ≮: string) +isntString H nil p = scalar-≢-impl-≮: nil string (λ ()) +isntString H (addr a) p with remember (H [ a ]ᴴ) +isntString H (addr a) p | (just (function f ⟨ var x ∈ T ⟩∈ U is B end) , q) = ≡-trans-≮: (cong orAny (cong typeOfᴹᴼ q)) (function-≮:-scalar string) +isntString H (addr a) p | (nothing , q) = ≡-trans-≮: (cong orAny (cong typeOfᴹᴼ q)) (any-≮:-scalar string) +isntString H (number x) p = scalar-≢-impl-≮: number string (λ ()) +isntString H (bool x) p = scalar-≢-impl-≮: boolean string (λ ()) +isntString H (string x) p = CONTRADICTION (p refl) + +isntFunction : ∀ H v {T U} → (valueType v ≢ function) → (typeOfᴱ H ∅ (val v) ≮: (T ⇒ U)) +isntFunction H nil p = scalar-≮:-function nil +isntFunction H (addr a) p = CONTRADICTION (p refl) +isntFunction H (number x) p = scalar-≮:-function number +isntFunction H (bool x) p = scalar-≮:-function boolean +isntFunction H (string x) p = scalar-≮:-function string + +isntEmpty : ∀ H v → (typeOfᴱ H ∅ (val v) ≮: none) +isntEmpty H nil = scalar-≮:-none nil +isntEmpty H (addr a) with remember (H [ a ]ᴴ) +isntEmpty H (addr a) | (just (function f ⟨ var x ∈ T ⟩∈ U is B end) , p) = ≡-trans-≮: (cong orAny (cong typeOfᴹᴼ p)) function-≮:-none +isntEmpty H (addr a) | (nothing , p) = ≡-trans-≮: (cong orAny (cong typeOfᴹᴼ p)) any-≮:-none +isntEmpty H (number x) = scalar-≮:-none number +isntEmpty H (bool x) = scalar-≮:-none boolean +isntEmpty H (string x) = scalar-≮:-none string + +runtimeBinOpWarning : ∀ H {op} v → BinOpError op (valueType v) → (typeOfᴱ H ∅ (val v) ≮: srcBinOp op) +runtimeBinOpWarning H v (+ p) = isntNumber H v p +runtimeBinOpWarning H v (- p) = isntNumber H v p +runtimeBinOpWarning H v (* p) = isntNumber H v p +runtimeBinOpWarning H v (/ p) = isntNumber H v p +runtimeBinOpWarning H v (< p) = isntNumber H v p +runtimeBinOpWarning H v (> p) = isntNumber H v p +runtimeBinOpWarning H v (<= p) = isntNumber H v p +runtimeBinOpWarning H v (>= p) = isntNumber H v p +runtimeBinOpWarning H v (·· p) = isntString H v p runtimeWarningᴱ : ∀ H M → RuntimeErrorᴱ H M → Warningᴱ H (typeCheckᴱ H ∅ M) runtimeWarningᴮ : ∀ H B → RuntimeErrorᴮ H B → Warningᴮ H (typeCheckᴮ H ∅ B) runtimeWarningᴱ H (var x) UnboundVariable = UnboundVariable refl runtimeWarningᴱ H (val (addr a)) (SEGV p) = UnallocatedAddress p -runtimeWarningᴱ H (M $ N) (FunctionMismatch v w p) with typeOf-val-not-none w -runtimeWarningᴱ H (M $ N) (FunctionMismatch v w p) | ok q = FunctionCallMismatch (λ r → p (mustBeFunction H ∅ v (λ r′ → q (trans r′ r)))) -runtimeWarningᴱ H (M $ N) (FunctionMismatch v w p) | warning W = app₂ W +runtimeWarningᴱ H (M $ N) (FunctionMismatch v w p) = FunctionCallMismatch (any-src-≮: (isntEmpty H w) (isntFunction H v p)) runtimeWarningᴱ H (M $ N) (app₁ err) = app₁ (runtimeWarningᴱ H M err) runtimeWarningᴱ H (M $ N) (app₂ err) = app₂ (runtimeWarningᴱ H N err) runtimeWarningᴱ H (block var b ∈ T is B end) (block err) = block₁ (runtimeWarningᴮ H B err) @@ -469,6 +343,6 @@ runtimeWarningᴮ H (local var x ∈ T ← M ∙ B) (local err) = local₁ (runt runtimeWarningᴮ H (return M ∙ B) (return err) = return (runtimeWarningᴱ H M err) wellTypedProgramsDontGoWrong : ∀ H′ B B′ → (∅ᴴ ⊢ B ⟶* B′ ⊣ H′) → (RuntimeErrorᴮ H′ B′) → Warningᴮ ∅ᴴ (typeCheckᴮ ∅ᴴ ∅ B) -wellTypedProgramsDontGoWrong H′ B B′ t err with reflect* ∅ᴴ B t (block (runtimeWarningᴮ H′ B′ err)) -wellTypedProgramsDontGoWrong H′ B B′ t err | heap (addr a refl ()) -wellTypedProgramsDontGoWrong H′ B B′ t err | block W = W +wellTypedProgramsDontGoWrong H′ B B′ t err with reflect* ∅ᴴ B t (Left (runtimeWarningᴮ H′ B′ err)) +wellTypedProgramsDontGoWrong H′ B B′ t err | Right (addr a refl ()) +wellTypedProgramsDontGoWrong H′ B B′ t err | Left W = W diff --git a/prototyping/Properties/Subtyping.agda b/prototyping/Properties/Subtyping.agda new file mode 100644 index 00000000..6a0b4203 --- /dev/null +++ b/prototyping/Properties/Subtyping.agda @@ -0,0 +1,221 @@ +{-# OPTIONS --rewriting #-} + +module Properties.Subtyping where + +open import Agda.Builtin.Equality using (_≡_; refl) +open import FFI.Data.Either using (Either; Left; Right; mapLR; swapLR; cond) +open import Luau.Subtyping using (_<:_; _≮:_; Tree; Language; ¬Language; witness; any; none; scalar; function; scalar-function; scalar-function-ok; scalar-function-err; scalar-scalar; function-scalar; function-ok; function-err; left; right; _,_) +open import Luau.Type using (Type; Scalar; strict; nil; number; string; boolean; none; any; _⇒_; _∪_; _∩_; tgt) +open import Properties.Contradiction using (CONTRADICTION; ¬) +open import Properties.Equality using (_≢_) +open import Properties.Functions using (_∘_) + +src = Luau.Type.src strict + +-- Language membership is decidable +dec-language : ∀ T t → Either (¬Language T t) (Language T t) +dec-language nil (scalar number) = Left (scalar-scalar number nil (λ ())) +dec-language nil (scalar boolean) = Left (scalar-scalar boolean nil (λ ())) +dec-language nil (scalar string) = Left (scalar-scalar string nil (λ ())) +dec-language nil (scalar nil) = Right (scalar nil) +dec-language nil function = Left (scalar-function nil) +dec-language nil (function-ok t) = Left (scalar-function-ok nil) +dec-language nil (function-err t) = Right (scalar-function-err nil) +dec-language boolean (scalar number) = Left (scalar-scalar number boolean (λ ())) +dec-language boolean (scalar boolean) = Right (scalar boolean) +dec-language boolean (scalar string) = Left (scalar-scalar string boolean (λ ())) +dec-language boolean (scalar nil) = Left (scalar-scalar nil boolean (λ ())) +dec-language boolean function = Left (scalar-function boolean) +dec-language boolean (function-ok t) = Left (scalar-function-ok boolean) +dec-language boolean (function-err t) = Right (scalar-function-err boolean) +dec-language number (scalar number) = Right (scalar number) +dec-language number (scalar boolean) = Left (scalar-scalar boolean number (λ ())) +dec-language number (scalar string) = Left (scalar-scalar string number (λ ())) +dec-language number (scalar nil) = Left (scalar-scalar nil number (λ ())) +dec-language number function = Left (scalar-function number) +dec-language number (function-ok t) = Left (scalar-function-ok number) +dec-language number (function-err t) = Right (scalar-function-err number) +dec-language string (scalar number) = Left (scalar-scalar number string (λ ())) +dec-language string (scalar boolean) = Left (scalar-scalar boolean string (λ ())) +dec-language string (scalar string) = Right (scalar string) +dec-language string (scalar nil) = Left (scalar-scalar nil string (λ ())) +dec-language string function = Left (scalar-function string) +dec-language string (function-ok t) = Left (scalar-function-ok string) +dec-language string (function-err t) = Right (scalar-function-err string) +dec-language (T₁ ⇒ T₂) (scalar s) = Left (function-scalar s) +dec-language (T₁ ⇒ T₂) function = Right function +dec-language (T₁ ⇒ T₂) (function-ok t) = mapLR function-ok function-ok (dec-language T₂ t) +dec-language (T₁ ⇒ T₂) (function-err t) = mapLR function-err function-err (swapLR (dec-language T₁ t)) +dec-language none t = Left none +dec-language any t = Right any +dec-language (T₁ ∪ T₂) t = cond (λ p → cond (Left ∘ _,_ p) (Right ∘ right) (dec-language T₂ t)) (Right ∘ left) (dec-language T₁ t) +dec-language (T₁ ∩ T₂) t = cond (Left ∘ left) (λ p → cond (Left ∘ right) (Right ∘ _,_ p) (dec-language T₂ t)) (dec-language T₁ t) + +-- ¬Language T is the complement of Language T +language-comp : ∀ {T} t → ¬Language T t → ¬(Language T t) +language-comp t (p₁ , p₂) (left q) = language-comp t p₁ q +language-comp t (p₁ , p₂) (right q) = language-comp t p₂ q +language-comp t (left p) (q₁ , q₂) = language-comp t p q₁ +language-comp t (right p) (q₁ , q₂) = language-comp t p q₂ +language-comp (scalar s) (scalar-scalar s p₁ p₂) (scalar s) = p₂ refl +language-comp (scalar s) (function-scalar s) (scalar s) = language-comp function (scalar-function s) function +language-comp (scalar s) none (scalar ()) +language-comp function (scalar-function ()) function +language-comp (function-ok t) (scalar-function-ok ()) (function-ok q) +language-comp (function-ok t) (function-ok p) (function-ok q) = language-comp t p q +language-comp (function-err t) (function-err p) (function-err q) = language-comp t q p + +-- ≮: is the complement of <: +¬≮:-impl-<: : ∀ {T U} → ¬(T ≮: U) → (T <: U) +¬≮:-impl-<: {T} {U} p t q with dec-language U t +¬≮:-impl-<: {T} {U} p t q | Left r = CONTRADICTION (p (witness t q r)) +¬≮:-impl-<: {T} {U} p t q | Right r = r + +<:-impl-¬≮: : ∀ {T U} → (T <: U) → ¬(T ≮: U) +<:-impl-¬≮: p (witness t q r) = language-comp t r (p t q) + +-- reflexivity +≮:-refl : ∀ {T} → ¬(T ≮: T) +≮:-refl (witness t p q) = language-comp t q p + +<:-refl : ∀ {T} → (T <: T) +<:-refl = ¬≮:-impl-<: ≮:-refl + +-- transititivity +≮:-trans-≡ : ∀ {S T U} → (S ≮: T) → (T ≡ U) → (S ≮: U) +≮:-trans-≡ p refl = p + +≡-trans-≮: : ∀ {S T U} → (S ≡ T) → (T ≮: U) → (S ≮: U) +≡-trans-≮: refl p = p + +≮:-trans : ∀ {S T U} → (S ≮: U) → Either (S ≮: T) (T ≮: U) +≮:-trans {T = T} (witness t p q) = mapLR (witness t p) (λ z → witness t z q) (dec-language T t) + +<:-trans : ∀ {S T U} → (S <: T) → (T <: U) → (S <: U) +<:-trans p q = ¬≮:-impl-<: (cond (<:-impl-¬≮: p) (<:-impl-¬≮: q) ∘ ≮:-trans) + +-- Properties of scalars +skalar = number ∪ (string ∪ (nil ∪ boolean)) + +function-≮:-scalar : ∀ {S T U} → (Scalar U) → ((S ⇒ T) ≮: U) +function-≮:-scalar s = witness function function (scalar-function s) + +scalar-≮:-function : ∀ {S T U} → (Scalar U) → (U ≮: (S ⇒ T)) +scalar-≮:-function s = witness (scalar s) (scalar s) (function-scalar s) + +any-≮:-scalar : ∀ {U} → (Scalar U) → (any ≮: U) +any-≮:-scalar s = witness (function-ok (scalar s)) any (scalar-function-ok s) + +scalar-≮:-none : ∀ {U} → (Scalar U) → (U ≮: none) +scalar-≮:-none s = witness (scalar s) (scalar s) none + +scalar-≢-impl-≮: : ∀ {T U} → (Scalar T) → (Scalar U) → (T ≢ U) → (T ≮: U) +scalar-≢-impl-≮: s₁ s₂ p = witness (scalar s₁) (scalar s₁) (scalar-scalar s₁ s₂ p) + +-- Properties of tgt +tgt-function-ok : ∀ {T t} → (Language (tgt T) t) → Language T (function-ok t) +tgt-function-ok {T = nil} (scalar ()) +tgt-function-ok {T = T₁ ⇒ T₂} p = function-ok p +tgt-function-ok {T = none} (scalar ()) +tgt-function-ok {T = any} p = any +tgt-function-ok {T = boolean} (scalar ()) +tgt-function-ok {T = number} (scalar ()) +tgt-function-ok {T = string} (scalar ()) +tgt-function-ok {T = T₁ ∪ T₂} (left p) = left (tgt-function-ok p) +tgt-function-ok {T = T₁ ∪ T₂} (right p) = right (tgt-function-ok p) +tgt-function-ok {T = T₁ ∩ T₂} (p₁ , p₂) = (tgt-function-ok p₁ , tgt-function-ok p₂) + +function-ok-tgt : ∀ {T t} → Language T (function-ok t) → (Language (tgt T) t) +function-ok-tgt (function-ok p) = p +function-ok-tgt (left p) = left (function-ok-tgt p) +function-ok-tgt (right p) = right (function-ok-tgt p) +function-ok-tgt (p₁ , p₂) = (function-ok-tgt p₁ , function-ok-tgt p₂) +function-ok-tgt any = any + +skalar-function-ok : ∀ {t} → (¬Language skalar (function-ok t)) +skalar-function-ok = (scalar-function-ok number , (scalar-function-ok string , (scalar-function-ok nil , scalar-function-ok boolean))) + +skalar-scalar : ∀ {T} (s : Scalar T) → (Language skalar (scalar s)) +skalar-scalar number = left (scalar number) +skalar-scalar boolean = right (right (right (scalar boolean))) +skalar-scalar string = right (left (scalar string)) +skalar-scalar nil = right (right (left (scalar nil))) + +tgt-none-≮: : ∀ {T U} → (tgt T ≮: U) → (T ≮: (skalar ∪ (none ⇒ U))) +tgt-none-≮: (witness t p q) = witness (function-ok t) (tgt-function-ok p) (skalar-function-ok , function-ok q) + +none-tgt-≮: : ∀ {T U} → (T ≮: (skalar ∪ (none ⇒ U))) → (tgt T ≮: U) +none-tgt-≮: (witness (scalar s) p (q₁ , q₂)) = CONTRADICTION (≮:-refl (witness (scalar s) (skalar-scalar s) q₁)) +none-tgt-≮: (witness function p (q₁ , scalar-function ())) +none-tgt-≮: (witness (function-ok t) p (q₁ , function-ok q₂)) = witness t (function-ok-tgt p) q₂ +none-tgt-≮: (witness (function-err (scalar s)) p (q₁ , function-err (scalar ()))) + +-- Properties of src +function-err-src : ∀ {T t} → (¬Language (src T) t) → Language T (function-err t) +function-err-src {T = nil} none = scalar-function-err nil +function-err-src {T = T₁ ⇒ T₂} p = function-err p +function-err-src {T = none} (scalar-scalar number () p) +function-err-src {T = none} (scalar-function-ok ()) +function-err-src {T = any} none = any +function-err-src {T = boolean} p = scalar-function-err boolean +function-err-src {T = number} p = scalar-function-err number +function-err-src {T = string} p = scalar-function-err string +function-err-src {T = T₁ ∪ T₂} (left p) = left (function-err-src p) +function-err-src {T = T₁ ∪ T₂} (right p) = right (function-err-src p) +function-err-src {T = T₁ ∩ T₂} (p₁ , p₂) = function-err-src p₁ , function-err-src p₂ + +¬function-err-src : ∀ {T t} → (Language (src T) t) → ¬Language T (function-err t) +¬function-err-src {T = nil} (scalar ()) +¬function-err-src {T = T₁ ⇒ T₂} p = function-err p +¬function-err-src {T = none} any = none +¬function-err-src {T = any} (scalar ()) +¬function-err-src {T = boolean} (scalar ()) +¬function-err-src {T = number} (scalar ()) +¬function-err-src {T = string} (scalar ()) +¬function-err-src {T = T₁ ∪ T₂} (p₁ , p₂) = (¬function-err-src p₁ , ¬function-err-src p₂) +¬function-err-src {T = T₁ ∩ T₂} (left p) = left (¬function-err-src p) +¬function-err-src {T = T₁ ∩ T₂} (right p) = right (¬function-err-src p) + +src-¬function-err : ∀ {T t} → Language T (function-err t) → (¬Language (src T) t) +src-¬function-err {T = nil} p = none +src-¬function-err {T = T₁ ⇒ T₂} (function-err p) = p +src-¬function-err {T = none} (scalar-function-err ()) +src-¬function-err {T = any} p = none +src-¬function-err {T = boolean} p = none +src-¬function-err {T = number} p = none +src-¬function-err {T = string} p = none +src-¬function-err {T = T₁ ∪ T₂} (left p) = left (src-¬function-err p) +src-¬function-err {T = T₁ ∪ T₂} (right p) = right (src-¬function-err p) +src-¬function-err {T = T₁ ∩ T₂} (p₁ , p₂) = (src-¬function-err p₁ , src-¬function-err p₂) + +src-¬scalar : ∀ {S T t} (s : Scalar S) → Language T (scalar s) → (¬Language (src T) t) +src-¬scalar number (scalar number) = none +src-¬scalar boolean (scalar boolean) = none +src-¬scalar string (scalar string) = none +src-¬scalar nil (scalar nil) = none +src-¬scalar s (left p) = left (src-¬scalar s p) +src-¬scalar s (right p) = right (src-¬scalar s p) +src-¬scalar s (p₁ , p₂) = (src-¬scalar s p₁ , src-¬scalar s p₂) +src-¬scalar s any = none + +src-any-≮: : ∀ {T U} → (T ≮: src U) → (U ≮: (T ⇒ any)) +src-any-≮: (witness t p q) = witness (function-err t) (function-err-src q) (¬function-err-src p) + +any-src-≮: : ∀ {S T U} → (U ≮: S) → (T ≮: (U ⇒ any)) → (U ≮: src T) +any-src-≮: (witness t x x₁) (witness (scalar s) p (function-scalar s)) = witness t x (src-¬scalar s p) +any-src-≮: r (witness (function-ok (scalar s)) p (function-ok (scalar-scalar s () q))) +any-src-≮: r (witness (function-ok (function-ok _)) p (function-ok (scalar-function-ok ()))) +any-src-≮: r (witness (function-err t) p (function-err q)) = witness t q (src-¬function-err p) + +-- Properties of any and none +any-≮: : ∀ {T U} → (T ≮: U) → (any ≮: U) +any-≮: (witness t p q) = witness t any q + +none-≮: : ∀ {T U} → (T ≮: U) → (T ≮: none) +none-≮: (witness t p q) = witness t p none + +any-≮:-none : (any ≮: none) +any-≮:-none = witness (scalar nil) any none + +function-≮:-none : ∀ {T U} → ((T ⇒ U) ≮: none) +function-≮:-none = witness function function none diff --git a/prototyping/Properties/TypeCheck.agda b/prototyping/Properties/TypeCheck.agda index bec68841..ead0c097 100644 --- a/prototyping/Properties/TypeCheck.agda +++ b/prototyping/Properties/TypeCheck.agda @@ -8,7 +8,7 @@ open import Agda.Builtin.Equality using (_≡_; refl) open import Agda.Builtin.Bool using (Bool; true; false) open import FFI.Data.Maybe using (Maybe; just; nothing) open import FFI.Data.Either using (Either) -open import Luau.TypeCheck(m) using (_⊢ᴱ_∈_; _⊢ᴮ_∈_; ⊢ᴼ_; ⊢ᴴ_; _⊢ᴴᴱ_▷_∈_; _⊢ᴴᴮ_▷_∈_; nil; var; addr; number; bool; string; app; function; block; binexp; done; return; local; nothing; orNone; tgtBinOp) +open import Luau.TypeCheck(m) using (_⊢ᴱ_∈_; _⊢ᴮ_∈_; ⊢ᴼ_; ⊢ᴴ_; _⊢ᴴᴱ_▷_∈_; _⊢ᴴᴮ_▷_∈_; nil; var; addr; number; bool; string; app; function; block; binexp; done; return; local; nothing; orAny; tgtBinOp) open import Luau.Syntax using (Block; Expr; Value; BinaryOperator; yes; nil; addr; number; bool; string; val; var; binexp; _$_; function_is_end; block_is_end; _∙_; return; done; local_←_; _⟨_⟩; _⟨_⟩∈_; var_∈_; name; fun; arg; +; -; *; /; <; >; ==; ~=; <=; >=) open import Luau.Type using (Type; nil; any; none; number; boolean; string; _⇒_; tgt) open import Luau.RuntimeType using (RuntimeType; nil; number; function; string; valueType) @@ -42,8 +42,8 @@ typeOfⱽ H (string x) = just string typeOfᴱ : Heap yes → VarCtxt → (Expr yes) → Type typeOfᴮ : Heap yes → VarCtxt → (Block yes) → Type -typeOfᴱ H Γ (var x) = orNone(Γ [ x ]ⱽ) -typeOfᴱ H Γ (val v) = orNone(typeOfⱽ H v) +typeOfᴱ H Γ (var x) = orAny(Γ [ x ]ⱽ) +typeOfᴱ H Γ (val v) = orAny(typeOfⱽ H v) typeOfᴱ H Γ (M $ N) = tgt(typeOfᴱ H Γ M) typeOfᴱ H Γ (function f ⟨ var x ∈ S ⟩∈ T is B end) = S ⇒ T typeOfᴱ H Γ (block var b ∈ T is B end) = T @@ -64,17 +64,17 @@ mustBeFunction H Γ (string x) p = CONTRADICTION (p refl) mustBeNumber : ∀ H Γ v → (typeOfᴱ H Γ (val v) ≡ number) → (valueType(v) ≡ number) mustBeNumber H Γ (addr a) p with remember (H [ a ]ᴴ) -mustBeNumber H Γ (addr a) p | (just O , q) with trans (cong orNone (cong typeOfᴹᴼ (sym q))) p +mustBeNumber H Γ (addr a) p | (just O , q) with trans (cong orAny (cong typeOfᴹᴼ (sym q))) p mustBeNumber H Γ (addr a) p | (just function f ⟨ var x ∈ T ⟩∈ U is B end , q) | () -mustBeNumber H Γ (addr a) p | (nothing , q) with trans (cong orNone (cong typeOfᴹᴼ (sym q))) p +mustBeNumber H Γ (addr a) p | (nothing , q) with trans (cong orAny (cong typeOfᴹᴼ (sym q))) p mustBeNumber H Γ (addr a) p | nothing , q | () mustBeNumber H Γ (number n) p = refl mustBeString : ∀ H Γ v → (typeOfᴱ H Γ (val v) ≡ string) → (valueType(v) ≡ string) mustBeString H Γ (addr a) p with remember (H [ a ]ᴴ) -mustBeString H Γ (addr a) p | (just O , q) with trans (cong orNone (cong typeOfᴹᴼ (sym q))) p +mustBeString H Γ (addr a) p | (just O , q) with trans (cong orAny (cong typeOfᴹᴼ (sym q))) p mustBeString H Γ (addr a) p | (just function f ⟨ var x ∈ T ⟩∈ U is B end , q) | () -mustBeString H Γ (addr a) p | (nothing , q) with trans (cong orNone (cong typeOfᴹᴼ (sym q))) p +mustBeString H Γ (addr a) p | (nothing , q) with trans (cong orAny (cong typeOfᴹᴼ (sym q))) p mustBeString H Γ (addr a) p | (nothing , q) | () mustBeString H Γ (string x) p = refl @@ -83,7 +83,7 @@ typeCheckᴮ : ∀ H Γ B → (Γ ⊢ᴮ B ∈ (typeOfᴮ H Γ B)) typeCheckᴱ H Γ (var x) = var refl typeCheckᴱ H Γ (val nil) = nil -typeCheckᴱ H Γ (val (addr a)) = addr (orNone (typeOfᴹᴼ (H [ a ]ᴴ))) +typeCheckᴱ H Γ (val (addr a)) = addr (orAny (typeOfᴹᴼ (H [ a ]ᴴ))) typeCheckᴱ H Γ (val (number n)) = number typeCheckᴱ H Γ (val (bool b)) = bool typeCheckᴱ H Γ (val (string x)) = string @@ -109,4 +109,4 @@ typeCheckᴴᴱ H Γ M = (typeCheckᴴ H , typeCheckᴱ H Γ M) typeCheckᴴᴮ : ∀ H Γ M → (Γ ⊢ᴴᴮ H ▷ M ∈ typeOfᴮ H Γ M) typeCheckᴴᴮ H Γ M = (typeCheckᴴ H , typeCheckᴮ H Γ M) - \ No newline at end of file + diff --git a/prototyping/Tests/Interpreter/concat_number_and_string/out.txt b/prototyping/Tests/Interpreter/concat_number_and_string/out.txt index a437438d..80c9d6ee 100644 --- a/prototyping/Tests/Interpreter/concat_number_and_string/out.txt +++ b/prototyping/Tests/Interpreter/concat_number_and_string/out.txt @@ -9,3 +9,4 @@ value 37.0 is not a string TYPE ERROR: Local variable y has type string but expression has type number + because provided type contains v, where v is a number From 2f150796422ba49ed4cd0a12e0e9be230e8a13d8 Mon Sep 17 00:00:00 2001 From: Alexander McCord <11488393+alexmccord@users.noreply.github.com> Date: Wed, 23 Mar 2022 16:53:52 -0700 Subject: [PATCH 199/399] Fold in rationale for making `else` branch mandatory in if-then-else expressions. (#426) This has been discussed but just didn't get incorporated into the RFC. --- rfcs/syntax-if-expression.md | 22 +++++++++++++++++++--- 1 file changed, 19 insertions(+), 3 deletions(-) diff --git a/rfcs/syntax-if-expression.md b/rfcs/syntax-if-expression.md index a435fd55..76f76cf1 100644 --- a/rfcs/syntax-if-expression.md +++ b/rfcs/syntax-if-expression.md @@ -20,7 +20,7 @@ Instead of `and/or`, `if/else` statement can be used but since that requires a s To solve these problems, we propose introducing a first-class ternary conditional. Instead of `? :` common in C-like languages, we propose an `if-then-else` expression form that is syntactically similar to `if-then-else` statement, but lacks terminating `end`. -Concretely, the `if-then-else` expression must match `if then else `; it can also contain an arbitrary number of `elseif` clauses, like `if then elseif then else `. Note that in either case, `else` is mandatory. +Concretely, the `if-then-else` expression must match `if then else `; it can also contain an arbitrary number of `elseif` clauses, like `if then elseif then else `. Unlike if statements, `else` is mandatory. The result of the expression is the then-expression when condition is truthy (not `nil` or `false`) and else-expression otherwise. Only one of the two possible resulting expressions is evaluated. @@ -30,11 +30,27 @@ Example: local x = if FFlagFoo then A else B MyComponent.validateProps = t.strictInterface({ - layoutOrder = t.optional(t.number), - newThing = if FFlagUseNewThing then t.whatever() else nil, + layoutOrder = t.optional(t.number), + newThing = if FFlagUseNewThing then t.whatever() else nil, }) ``` +Note that `else` is mandatory because it's always better to be explicit. If it weren't mandatory, it opens the possiblity that someone might be writing a chain of if-then-else and forgot to add in the final `else` that _doesn't_ return a `nil` value! Enforcing this syntactically ensures the program does not run. Also, with it being mandatory, it solves many cases where parsing the expression is ambiguous due to the infamous [dangling else](https://en.wikipedia.org/wiki/Dangling_else). + +This example will not do what it looks like it's supposed to do! The if expression will _successfully_ parse and be interpreted as to return `h()` if `g()` evaluates to some falsy value, when in actual fact the clear intention is to evaluate `h()` only if `f()` is falsy. + +```lua +if f() then + ... + local foo = if g() then x +else + h() + ... +end +``` + +The only way to solve this had we chose optional `else` branch would be to wrap the if expression in parentheses or to place a semi-colon. + ## Drawbacks Studio's script editor autocomplete currently adds an indented block followed by `end` whenever a line ends that includes a `then` token. This can make use of the if expression unpleasant as developers have to keep fixing the code by removing auto-inserted `end`. We can work around this on the editor side by (short-term) differentiating between whether `if` token is the first on its line, and (long-term) by refactoring completion engine to use infallible parser for the block completer. From 7ab76e582cea0e8977e5b4b16af1d8a418ab5dba Mon Sep 17 00:00:00 2001 From: Alexander McCord <11488393+alexmccord@users.noreply.github.com> Date: Thu, 24 Mar 2022 08:38:16 -0700 Subject: [PATCH 200/399] RFC: Do not implement non-nil postfix `!` operator. (#420) This is a meta-RFC. I'd like to propose to remove the postfix `!` operator from the language. I'd like to argue that this operator will eventually become useless and can only be used incorrectly at some point in the future, and that there are likely ways for them to be sound without having to reach for the `!` hammer. With that, I present these counterarguments: Shortcoming #1: Refinements does not apply to the block after `return`/`break`/`continue`. ```lua if not x or not y then return end -- both x and y are truthy ``` This will be solved by implementing control flow analysis where it'd apply the inverse of the condition leading up to the control transfer statement to the rest of the scope. Shortcoming #2: Type checker is not aware of the actual state of various locations. ```lua type Foo = { x: { y: number }? }? local foo: Foo = { x = { y = 5 } } print(foo.x.y) -- prints 5 at runtime, type checker warns on this ``` This will be solved by implementing type states where it would inspect the initialization sites as well as assignments to know their actual states. That is, rather than trusting the type annotation `Foo` as the state which gets us far enough, we'd start seeing these type annotations as a subtype constraint for the location `foo`. --- If there are other use cases not covered in this message, we should talk about that and see if there exists an alternative direction that can solve these use cases soundly. --- rfcs/syntax-nil-forgiving-operator.md | 90 --------------------------- 1 file changed, 90 deletions(-) delete mode 100644 rfcs/syntax-nil-forgiving-operator.md diff --git a/rfcs/syntax-nil-forgiving-operator.md b/rfcs/syntax-nil-forgiving-operator.md deleted file mode 100644 index 63d5cabb..00000000 --- a/rfcs/syntax-nil-forgiving-operator.md +++ /dev/null @@ -1,90 +0,0 @@ -# nil-forgiving postfix operator ! - -## Summary - -Introduce syntax to suppress typechecking errors for nilable types by ascribing them into a non-nil type. - -## Motivation - -Typechecking might not be able to figure out that a certain expression is a non-nil type, but the user might know additional context of the expression. - -Using `::` ascriptions is the current work-around for this issue, but it's much more verbose and requires providing the full type name when you only want to ascribe T? to T. - -The nil-forgiving operator will also allow chaining to be written in a very terse manner: -```lua -local p = a!.b!.c -``` -instead of -```lua -local ((p :: Part).b :: Folder).c -``` - -Note that nil-forgiving operator is **not** a part of member access operator, it can be used in standalone expressions, indexing and other places: -```lua -local p = f(a!)! -local q = b!['X'] -``` - -Nil-forgiving operator can be found in some programming languages such as C# (null-forgiving or null-suppression operator) and TypeScript (non-null assertion operator). - -## Design - -To implement this, we will change the syntax of the *primaryexp*. - -Before: -``` -primaryexp ::= prefixexp { `.' NAME | `[' exp `]' | `:' NAME funcargs | funcargs } -``` -After: -``` -postfixeexp ::= (`.' NAME | `[' exp `]' | `:' NAME funcargs | funcargs) [`!'] -primaryexp ::= prefixexp [`!'] { postfixeexp } -``` - -When we get the `!` token, we will wrap the expression that we have into a new AstExprNonNilAssertion node. - -An error is generated when the type of expression node this is one of the following: -* AstExprConstantBool -* AstExprConstantNumber -* AstExprConstantString -* AstExprFunction -* AstExprTable - -This operator doesn't have any impact on the run-time behavior of the program, it will only affect the type of the expression in the typechecker. - ---- -While parsing an assignment expression starts with a *primaryexp*, it performs a check that it has an l-value based on a fixed set of AstNode types. - -Since using `!` on an l-value has no effect, we don't extend this list with the new node and will generate a specialized parse error for code like: -```lua -p.a! = b -``` - ---- -When operator is used on expression of a union type with a `nil` option, it removes that option from the set. -If only one option remains, the union type is replaced with the type of a single option. - -If the type is `nil`, typechecker will generate a warning that the operator cannot be applied to the expression. - -For any other type, it has no effect and doesn't generate additional warnings. - -The reason for the last rule is to simplify movement of existing code where context in each location is slightly different. - -As an example from Roblox, instance path could dynamically change from being know to exist to be missing when script is changed in edit mode. - -## Drawbacks - -### Unnecessary operator use - -It might be useful to warn about unnecessary uses of this operator when the value cannot be `nil`, but we have no way of enabling this behavior. - -### Bad practice - -The operator might be placed by users to ignore/silence correct warnings and lower the strength of type checking that Luau provides. - -## Alternatives - -Aside from type assertion operator :: it should be possible to place `assert` function calls before the operation. -Type refinement/constraints should handle that statement and avoid warning in the following expressions. - -But `assert` call will introduce runtime overhead without adding extra safety to the case when the type is nil at run time, in both cases an error will be thrown. From 5a21c413634f628c89323ce06b77720f4b53cb82 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Thu, 24 Mar 2022 09:26:42 -0700 Subject: [PATCH 201/399] Update STATUS.md --- rfcs/STATUS.md | 14 -------------- 1 file changed, 14 deletions(-) diff --git a/rfcs/STATUS.md b/rfcs/STATUS.md index 5d2d835c..2b372622 100644 --- a/rfcs/STATUS.md +++ b/rfcs/STATUS.md @@ -29,14 +29,6 @@ This document tracks unimplemented RFCs. **Status**: Implemented but not fully rolled out yet. -## Nil-forgiving operator - -[RFC: nil-forgiving postfix operator !](https://github.com/Roblox/luau/blob/master/rfcs/syntax-nil-forgiving-operator.md) - -**Status**: Needs implementation. - -**Notes**: Do we need to reevaluate the necessity given `assert` and improved refinements? - ## Safe navigation operator [RFC: Safe navigation postfix operator (?)](https://github.com/Roblox/luau/blob/master/rfcs/syntax-safe-navigation-operator.md) @@ -56,9 +48,3 @@ This document tracks unimplemented RFCs. [RFC: Generalized iteration](https://github.com/Roblox/luau/blob/master/rfcs/generalized-iteration.md) **Status**: Needs implementation - -## table.clone - -[RFC: table.clone](https://github.com/Roblox/luau/blob/master/rfcs/function-table-clone.md) - -**Status**: Needs implementation From b8e025311b5ec8599d7ea77ab69225189c9f820b Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Thu, 24 Mar 2022 09:29:20 -0700 Subject: [PATCH 202/399] Mark table.clone as implemented --- rfcs/function-table-clone.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/rfcs/function-table-clone.md b/rfcs/function-table-clone.md index 78d126ef..8cb97984 100644 --- a/rfcs/function-table-clone.md +++ b/rfcs/function-table-clone.md @@ -1,5 +1,7 @@ # table.clone +**Status**: Implemented + ## Summary Add `table.clone` function that, given a table, produces a copy of that table with the same keys/values/metatable. From 2335b26ffc09fda428bc80f126e2f884815f7514 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Thu, 24 Mar 2022 09:31:18 -0700 Subject: [PATCH 203/399] Update library.md Add documentation for table.clone --- docs/_pages/library.md | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/docs/_pages/library.md b/docs/_pages/library.md index 28e7035a..eeada336 100644 --- a/docs/_pages/library.md +++ b/docs/_pages/library.md @@ -466,6 +466,13 @@ function table.isfrozen(t: table): boolean Returns `true` iff the input table is frozen. +``` +function table.clone(t: table): table +``` + +Returns a copy of the input table that has the same metatable, same keys and values, and is not frozen even if `t` was. +The copy is shallow: implementing a deep recursive copy automatically is challenging, and often only certain keys need to be cloned recursively which can be done after the initial clone by modifying the resulting table. + ## string library ``` From ab64a097cd72282d4e223cf914b02766f188c12c Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Thu, 24 Mar 2022 13:10:30 -0700 Subject: [PATCH 204/399] Mark sealed table subtyping RFC as implemented --- rfcs/sealed-table-subtyping.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/rfcs/sealed-table-subtyping.md b/rfcs/sealed-table-subtyping.md index 7310795a..73714909 100644 --- a/rfcs/sealed-table-subtyping.md +++ b/rfcs/sealed-table-subtyping.md @@ -1,5 +1,9 @@ # Sealed table subtyping +**Status**: Implemented + +## Summary + In Luau, tables have a state, which can, among others, be "sealed". A sealed table is one that we know the full shape of and cannot have new properties added to it. We would like to introduce subtyping for sealed tables, to allow users to express some subtyping relationships that they currently cannot. ## Motivation From 5e7e462104061f604b9dabcd6cb78573aaf37e8d Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Thu, 24 Mar 2022 13:10:56 -0700 Subject: [PATCH 205/399] Update STATUS.md --- rfcs/STATUS.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/rfcs/STATUS.md b/rfcs/STATUS.md index 2b372622..93a09ece 100644 --- a/rfcs/STATUS.md +++ b/rfcs/STATUS.md @@ -17,11 +17,10 @@ This document tracks unimplemented RFCs. ## Sealed/unsealed typing changes -[RFC: Sealed table subtyping](https://github.com/Roblox/luau/blob/master/rfcs/sealed-table-subtyping.md) | [RFC: Unsealed table literals](https://github.com/Roblox/luau/blob/master/rfcs/unsealed-table-literals.md) | [RFC: Only strip optional properties from unsealed tables during subtyping](https://github.com/Roblox/luau/blob/master/rfcs/unsealed-table-subtyping-strips-optional-properties.md) -**Status**: Implemented but depends on new transaction log implementation that is not fully live yet. +**Status**: Implemented but not fully rolled out yet. ## Singleton types From 373da161e915de2aa71ba83fe9baf23b269057f3 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Thu, 24 Mar 2022 14:49:08 -0700 Subject: [PATCH 206/399] Sync to upstream/release/520 --- Analysis/include/Luau/Error.h | 1 + Analysis/include/Luau/ToString.h | 3 +- Analysis/include/Luau/TypePack.h | 6 +- Analysis/include/Luau/TypeVar.h | 3 + Analysis/include/Luau/Unifier.h | 4 +- Analysis/include/Luau/UnifierSharedState.h | 2 + Analysis/src/Error.cpp | 30 ++- Analysis/src/Linter.cpp | 43 +++- Analysis/src/Module.cpp | 31 +-- Analysis/src/ToString.cpp | 72 +++++-- Analysis/src/TypeInfer.cpp | 69 ++++--- Analysis/src/TypePack.cpp | 19 +- Analysis/src/TypeVar.cpp | 18 ++ Analysis/src/Unifier.cpp | 173 +++++++++++----- Ast/src/Parser.cpp | 20 +- VM/src/lapi.cpp | 50 ++--- VM/src/ldo.cpp | 21 +- VM/src/ldo.h | 2 +- VM/src/lgc.cpp | 228 +++++++++++---------- VM/src/lgc.h | 2 +- VM/src/lstate.cpp | 26 +-- VM/src/lstate.h | 44 ++-- VM/src/ltable.cpp | 4 +- tests/Autocomplete.test.cpp | 5 - tests/Conformance.test.cpp | 2 - tests/Fixture.cpp | 19 +- tests/Linter.test.cpp | 85 ++------ tests/ToString.test.cpp | 25 +++ tests/Transpiler.test.cpp | 2 - tests/TypeInfer.builtins.test.cpp | 2 - tests/TypeInfer.functions.test.cpp | 76 +++++++ tests/TypeInfer.modules.test.cpp | 85 +++++++- tests/TypeInfer.operators.test.cpp | 26 +++ tests/TypeInfer.refinements.test.cpp | 11 - tests/TypeInfer.singletons.test.cpp | 118 +---------- tests/TypeInfer.tables.test.cpp | 38 ++++ tests/TypeInfer.test.cpp | 2 +- tests/TypeInfer.unionTypes.test.cpp | 1 + 38 files changed, 805 insertions(+), 563 deletions(-) diff --git a/Analysis/include/Luau/Error.h b/Analysis/include/Luau/Error.h index 72350255..53b946a0 100644 --- a/Analysis/include/Luau/Error.h +++ b/Analysis/include/Luau/Error.h @@ -96,6 +96,7 @@ struct CountMismatch size_t expected; size_t actual; Context context = Arg; + bool isVariadic = false; bool operator==(const CountMismatch& rhs) const; }; diff --git a/Analysis/include/Luau/ToString.h b/Analysis/include/Luau/ToString.h index a97bf6d6..49ee82fe 100644 --- a/Analysis/include/Luau/ToString.h +++ b/Analysis/include/Luau/ToString.h @@ -32,6 +32,7 @@ struct ToStringOptions size_t maxTypeLength = size_t(FInt::LuauTypeMaximumStringifierLength); std::optional nameMap; std::shared_ptr scope; // If present, module names will be added and types that are not available in scope will be marked as 'invalid' + std::vector namedFunctionOverrideArgNames; // If present, named function argument names will be overridden }; struct ToStringResult @@ -65,7 +66,7 @@ inline std::string toString(TypePackId ty) std::string toString(const TypeVar& tv, const ToStringOptions& opts = {}); std::string toString(const TypePackVar& tp, const ToStringOptions& opts = {}); -std::string toStringNamedFunction(const std::string& prefix, const FunctionTypeVar& ftv, ToStringOptions opts = {}); +std::string toStringNamedFunction(const std::string& funcName, const FunctionTypeVar& ftv, const ToStringOptions& opts = {}); // It could be useful to see the text representation of a type during a debugging session instead of exploring the content of the class // These functions will dump the type to stdout and can be evaluated in Watch/Immediate windows or as gdb/lldb expression diff --git a/Analysis/include/Luau/TypePack.h b/Analysis/include/Luau/TypePack.h index 946be356..85fa467f 100644 --- a/Analysis/include/Luau/TypePack.h +++ b/Analysis/include/Luau/TypePack.h @@ -119,9 +119,9 @@ bool areEqual(SeenSet& seen, const TypePackVar& lhs, const TypePackVar& rhs); TypePackId follow(TypePackId tp); TypePackId follow(TypePackId tp, std::function mapper); -size_t size(TypePackId tp); -bool finite(TypePackId tp); -size_t size(const TypePack& tp); +size_t size(TypePackId tp, TxnLog* log = nullptr); +bool finite(TypePackId tp, TxnLog* log = nullptr); +size_t size(const TypePack& tp, TxnLog* log = nullptr); std::optional first(TypePackId tp); TypePackVar* asMutable(TypePackId tp); diff --git a/Analysis/include/Luau/TypeVar.h b/Analysis/include/Luau/TypeVar.h index 29578dcd..b8c4b362 100644 --- a/Analysis/include/Luau/TypeVar.h +++ b/Analysis/include/Luau/TypeVar.h @@ -488,6 +488,9 @@ const TableTypeVar* getTableType(TypeId type); // Returns nullptr if the type has no name. const std::string* getName(TypeId type); +// Returns name of the module where type was defined if type has that information +std::optional getDefinitionModuleName(TypeId type); + // Checks whether a union contains all types of another union. bool isSubset(const UnionTypeVar& super, const UnionTypeVar& sub); diff --git a/Analysis/include/Luau/Unifier.h b/Analysis/include/Luau/Unifier.h index f1ffbcc0..474af50c 100644 --- a/Analysis/include/Luau/Unifier.h +++ b/Analysis/include/Luau/Unifier.h @@ -90,7 +90,9 @@ private: TypeId deeplyOptional(TypeId ty, std::unordered_map seen = {}); - void cacheResult(TypeId subTy, TypeId superTy); + bool canCacheResult(TypeId subTy, TypeId superTy); + void cacheResult(TypeId subTy, TypeId superTy, size_t prevErrorCount); + void cacheResult_DEPRECATED(TypeId subTy, TypeId superTy); public: void tryUnify(TypePackId subTy, TypePackId superTy, bool isFunctionCall = false); diff --git a/Analysis/include/Luau/UnifierSharedState.h b/Analysis/include/Luau/UnifierSharedState.h index 88997c41..9a3ba56d 100644 --- a/Analysis/include/Luau/UnifierSharedState.h +++ b/Analysis/include/Luau/UnifierSharedState.h @@ -2,6 +2,7 @@ #pragma once #include "Luau/DenseHash.h" +#include "Luau/Error.h" #include "Luau/TypeVar.h" #include "Luau/TypePack.h" @@ -42,6 +43,7 @@ struct UnifierSharedState DenseHashSet seenAny{nullptr}; DenseHashMap skipCacheForType{nullptr}; DenseHashSet, TypeIdPairHash> cachedUnify{{nullptr, nullptr}}; + DenseHashMap, TypeErrorData, TypeIdPairHash> cachedUnifyError{{nullptr, nullptr}}; DenseHashSet tempSeenTy{nullptr}; DenseHashSet tempSeenTp{nullptr}; diff --git a/Analysis/src/Error.cpp b/Analysis/src/Error.cpp index 26d3b76d..210c0191 100644 --- a/Analysis/src/Error.cpp +++ b/Analysis/src/Error.cpp @@ -8,6 +8,7 @@ #include LUAU_FASTFLAGVARIABLE(BetterDiagnosticCodesInStudio, false); +LUAU_FASTFLAGVARIABLE(LuauTypeMismatchModuleName, false); static std::string wrongNumberOfArgsString(size_t expectedCount, size_t actualCount, const char* argPrefix = nullptr, bool isVariadic = false) { @@ -53,7 +54,32 @@ struct ErrorConverter { std::string operator()(const Luau::TypeMismatch& tm) const { - std::string result = "Type '" + Luau::toString(tm.givenType) + "' could not be converted into '" + Luau::toString(tm.wantedType) + "'"; + std::string givenTypeName = Luau::toString(tm.givenType); + std::string wantedTypeName = Luau::toString(tm.wantedType); + + std::string result; + + if (FFlag::LuauTypeMismatchModuleName) + { + if (givenTypeName == wantedTypeName) + { + if (auto givenDefinitionModule = getDefinitionModuleName(tm.givenType)) + { + if (auto wantedDefinitionModule = getDefinitionModuleName(tm.wantedType)) + { + result = "Type '" + givenTypeName + "' from '" + *givenDefinitionModule + "' could not be converted into '" + wantedTypeName + + "' from '" + *wantedDefinitionModule + "'"; + } + } + } + + if (result.empty()) + result = "Type '" + givenTypeName + "' could not be converted into '" + wantedTypeName + "'"; + } + else + { + result = "Type '" + givenTypeName + "' could not be converted into '" + wantedTypeName + "'"; + } if (tm.error) { @@ -147,7 +173,7 @@ struct ErrorConverter return "Function only returns " + std::to_string(e.expected) + " value" + expectedS + ". " + std::to_string(e.actual) + " are required here"; case CountMismatch::Arg: - return "Argument count mismatch. Function " + wrongNumberOfArgsString(e.expected, e.actual); + return "Argument count mismatch. Function " + wrongNumberOfArgsString(e.expected, e.actual, /*argPrefix*/ nullptr, e.isVariadic); } LUAU_ASSERT(!"Unknown context"); diff --git a/Analysis/src/Linter.cpp b/Analysis/src/Linter.cpp index 56c4e3e8..b7480e34 100644 --- a/Analysis/src/Linter.cpp +++ b/Analysis/src/Linter.cpp @@ -14,6 +14,7 @@ LUAU_FASTINTVARIABLE(LuauSuggestionDistance, 4) LUAU_FASTFLAGVARIABLE(LuauLintGlobalNeverReadBeforeWritten, false) +LUAU_FASTFLAGVARIABLE(LuauLintNoRobloxBits, false) namespace Luau { @@ -1135,16 +1136,20 @@ private: enum TypeKind { - Kind_Invalid, + Kind_Unknown, Kind_Primitive, // primitive type supported by VM - boolean/userdata/etc. No differentiation between types of userdata. - Kind_Vector, // For 'vector' but only used when type is used - Kind_Userdata, // custom userdata type - Vector3/etc. + Kind_Vector, // 'vector' but only used when type is used + Kind_Userdata, // custom userdata type + + // TODO: remove these with LuauLintNoRobloxBits Kind_Class, // custom userdata type that reflects Roblox Instance-derived hierarchy - Part/etc. Kind_Enum, // custom userdata type referring to an enum item of enum classes, e.g. Enum.NormalId.Back/Enum.Axis.X/etc. }; bool containsPropName(TypeId ty, const std::string& propName) { + LUAU_ASSERT(!FFlag::LuauLintNoRobloxBits); + if (auto ctv = get(ty)) return lookupClassProp(ctv, propName) != nullptr; @@ -1163,13 +1168,23 @@ private: if (name == "vector") return Kind_Vector; - if (std::optional maybeTy = context->scope->lookupType(name)) - // Kind_Userdata is probably not 100% precise but is close enough - return containsPropName(maybeTy->type, "ClassName") ? Kind_Class : Kind_Userdata; - else if (std::optional maybeTy = context->scope->lookupImportedType("Enum", name)) - return Kind_Enum; + if (FFlag::LuauLintNoRobloxBits) + { + if (std::optional maybeTy = context->scope->lookupType(name)) + return Kind_Userdata; - return Kind_Invalid; + return Kind_Unknown; + } + else + { + if (std::optional maybeTy = context->scope->lookupType(name)) + // Kind_Userdata is probably not 100% precise but is close enough + return containsPropName(maybeTy->type, "ClassName") ? Kind_Class : Kind_Userdata; + else if (std::optional maybeTy = context->scope->lookupImportedType("Enum", name)) + return Kind_Enum; + + return Kind_Unknown; + } } void validateType(AstExprConstantString* expr, std::initializer_list expected, const char* expectedString) @@ -1177,7 +1192,7 @@ private: std::string name(expr->value.data, expr->value.size); TypeKind kind = getTypeKind(name); - if (kind == Kind_Invalid) + if (kind == Kind_Unknown) { emitWarning(*context, LintWarning::Code_UnknownType, expr->location, "Unknown type '%s'", name.c_str()); return; @@ -1189,7 +1204,7 @@ private: return; // as a special case, Instance and EnumItem are both a userdata type (as returned by typeof) and a class type - if (ek == Kind_Userdata && (name == "Instance" || name == "EnumItem")) + if (!FFlag::LuauLintNoRobloxBits && ek == Kind_Userdata && (name == "Instance" || name == "EnumItem")) return; } @@ -1198,12 +1213,18 @@ private: bool acceptsClassName(AstName method) { + LUAU_ASSERT(!FFlag::LuauLintNoRobloxBits); + return method.value[0] == 'F' && (method == "FindFirstChildOfClass" || method == "FindFirstChildWhichIsA" || method == "FindFirstAncestorOfClass" || method == "FindFirstAncestorWhichIsA"); } bool visit(AstExprCall* node) override { + // TODO: Simply remove the override + if (FFlag::LuauLintNoRobloxBits) + return true; + if (AstExprIndexName* index = node->func->as()) { AstExprConstantString* arg0 = node->args.size > 0 ? node->args.data[0]->as() : NULL; diff --git a/Analysis/src/Module.cpp b/Analysis/src/Module.cpp index a330a98d..0787d3a4 100644 --- a/Analysis/src/Module.cpp +++ b/Analysis/src/Module.cpp @@ -12,10 +12,8 @@ #include LUAU_FASTFLAGVARIABLE(DebugLuauFreezeArena, false) -LUAU_FASTFLAGVARIABLE(DebugLuauTrackOwningArena, false) // Remove with FFlagLuauImmutableTypes LUAU_FASTINTVARIABLE(LuauTypeCloneRecursionLimit, 300) LUAU_FASTFLAGVARIABLE(LuauCloneDeclaredGlobals, false) -LUAU_FASTFLAG(LuauImmutableTypes) namespace Luau { @@ -65,8 +63,7 @@ TypeId TypeArena::addTV(TypeVar&& tv) { TypeId allocated = typeVars.allocate(std::move(tv)); - if (FFlag::DebugLuauTrackOwningArena || FFlag::LuauImmutableTypes) - asMutable(allocated)->owningArena = this; + asMutable(allocated)->owningArena = this; return allocated; } @@ -75,8 +72,7 @@ TypeId TypeArena::freshType(TypeLevel level) { TypeId allocated = typeVars.allocate(FreeTypeVar{level}); - if (FFlag::DebugLuauTrackOwningArena || FFlag::LuauImmutableTypes) - asMutable(allocated)->owningArena = this; + asMutable(allocated)->owningArena = this; return allocated; } @@ -85,8 +81,7 @@ TypePackId TypeArena::addTypePack(std::initializer_list types) { TypePackId allocated = typePacks.allocate(TypePack{std::move(types)}); - if (FFlag::DebugLuauTrackOwningArena || FFlag::LuauImmutableTypes) - asMutable(allocated)->owningArena = this; + asMutable(allocated)->owningArena = this; return allocated; } @@ -95,8 +90,7 @@ TypePackId TypeArena::addTypePack(std::vector types) { TypePackId allocated = typePacks.allocate(TypePack{std::move(types)}); - if (FFlag::DebugLuauTrackOwningArena || FFlag::LuauImmutableTypes) - asMutable(allocated)->owningArena = this; + asMutable(allocated)->owningArena = this; return allocated; } @@ -105,8 +99,7 @@ TypePackId TypeArena::addTypePack(TypePack tp) { TypePackId allocated = typePacks.allocate(std::move(tp)); - if (FFlag::DebugLuauTrackOwningArena || FFlag::LuauImmutableTypes) - asMutable(allocated)->owningArena = this; + asMutable(allocated)->owningArena = this; return allocated; } @@ -115,8 +108,7 @@ TypePackId TypeArena::addTypePack(TypePackVar tp) { TypePackId allocated = typePacks.allocate(std::move(tp)); - if (FFlag::DebugLuauTrackOwningArena || FFlag::LuauImmutableTypes) - asMutable(allocated)->owningArena = this; + asMutable(allocated)->owningArena = this; return allocated; } @@ -439,16 +431,9 @@ TypeId clone(TypeId typeId, TypeArena& dest, SeenTypes& seenTypes, SeenTypePacks TypeCloner cloner{dest, typeId, seenTypes, seenTypePacks, cloneState}; Luau::visit(cloner, typeId->ty); // Mutates the storage that 'res' points into. - if (FFlag::LuauImmutableTypes) - { - // Persistent types are not being cloned and we get the original type back which might be read-only - if (!res->persistent) - asMutable(res)->documentationSymbol = typeId->documentationSymbol; - } - else - { + // Persistent types are not being cloned and we get the original type back which might be read-only + if (!res->persistent) asMutable(res)->documentationSymbol = typeId->documentationSymbol; - } } return res; diff --git a/Analysis/src/ToString.cpp b/Analysis/src/ToString.cpp index 010ca361..59ee6de2 100644 --- a/Analysis/src/ToString.cpp +++ b/Analysis/src/ToString.cpp @@ -16,6 +16,7 @@ * Fair warning: Setting this will break a lot of Luau unit tests. */ LUAU_FASTFLAGVARIABLE(DebugLuauVerboseTypeNames, false) +LUAU_FASTFLAGVARIABLE(LuauDocFuncParameters, false) namespace Luau { @@ -769,6 +770,7 @@ struct TypePackStringifier else state.emit(", "); + // Do not respect opts.namedFunctionOverrideArgNames here if (elemIndex < elemNames.size() && elemNames[elemIndex]) { state.emit(elemNames[elemIndex]->name); @@ -1090,13 +1092,13 @@ std::string toString(const TypePackVar& tp, const ToStringOptions& opts) return toString(const_cast(&tp), std::move(opts)); } -std::string toStringNamedFunction(const std::string& prefix, const FunctionTypeVar& ftv, ToStringOptions opts) +std::string toStringNamedFunction(const std::string& funcName, const FunctionTypeVar& ftv, const ToStringOptions& opts) { ToStringResult result; StringifierState state(opts, result, opts.nameMap); TypeVarStringifier tvs{state}; - state.emit(prefix); + state.emit(funcName); if (!opts.hideNamedFunctionTypeParameters) tvs.stringify(ftv.generics, ftv.genericPacks); @@ -1104,28 +1106,59 @@ std::string toStringNamedFunction(const std::string& prefix, const FunctionTypeV state.emit("("); auto argPackIter = begin(ftv.argTypes); - auto argNameIter = ftv.argNames.begin(); bool first = true; - while (argPackIter != end(ftv.argTypes)) + if (FFlag::LuauDocFuncParameters) { - if (!first) - state.emit(", "); - first = false; - - // We don't currently respect opts.functionTypeArguments. I don't think this function should. - if (argNameIter != ftv.argNames.end()) + size_t idx = 0; + while (argPackIter != end(ftv.argTypes)) { - state.emit((*argNameIter ? (*argNameIter)->name : "_") + ": "); - ++argNameIter; - } - else - { - state.emit("_: "); - } + if (!first) + state.emit(", "); + first = false; - tvs.stringify(*argPackIter); - ++argPackIter; + // We don't respect opts.functionTypeArguments + if (idx < opts.namedFunctionOverrideArgNames.size()) + { + state.emit(opts.namedFunctionOverrideArgNames[idx] + ": "); + } + else if (idx < ftv.argNames.size() && ftv.argNames[idx]) + { + state.emit(ftv.argNames[idx]->name + ": "); + } + else + { + state.emit("_: "); + } + tvs.stringify(*argPackIter); + + ++argPackIter; + ++idx; + } + } + else + { + auto argNameIter = ftv.argNames.begin(); + while (argPackIter != end(ftv.argTypes)) + { + if (!first) + state.emit(", "); + first = false; + + // We don't currently respect opts.functionTypeArguments. I don't think this function should. + if (argNameIter != ftv.argNames.end()) + { + state.emit((*argNameIter ? (*argNameIter)->name : "_") + ": "); + ++argNameIter; + } + else + { + state.emit("_: "); + } + + tvs.stringify(*argPackIter); + ++argPackIter; + } } if (argPackIter.tail()) @@ -1134,7 +1167,6 @@ std::string toStringNamedFunction(const std::string& prefix, const FunctionTypeV state.emit(", "); state.emit("...: "); - if (auto vtp = get(*argPackIter.tail())) tvs.stringify(vtp->ty); else diff --git a/Analysis/src/TypeInfer.cpp b/Analysis/src/TypeInfer.cpp index 41e8ce55..9965d5aa 100644 --- a/Analysis/src/TypeInfer.cpp +++ b/Analysis/src/TypeInfer.cpp @@ -27,10 +27,8 @@ LUAU_FASTFLAGVARIABLE(LuauWeakEqConstraint, false) // Eventually removed as fals LUAU_FASTFLAGVARIABLE(DebugLuauFreezeDuringUnification, false) LUAU_FASTFLAGVARIABLE(LuauRecursiveTypeParameterRestriction, false) LUAU_FASTFLAGVARIABLE(LuauGenericFunctionsDontCacheTypeParams, false) -LUAU_FASTFLAGVARIABLE(LuauImmutableTypes, false) LUAU_FASTFLAGVARIABLE(LuauSealExports, false) LUAU_FASTFLAGVARIABLE(LuauSelfCallAutocompleteFix, false) -LUAU_FASTFLAGVARIABLE(LuauSingletonTypes, false) LUAU_FASTFLAGVARIABLE(LuauDiscriminableUnions2, false) LUAU_FASTFLAGVARIABLE(LuauExpectedTypesOfProperties, false) LUAU_FASTFLAGVARIABLE(LuauErrorRecoveryType, false) @@ -38,6 +36,7 @@ LUAU_FASTFLAGVARIABLE(LuauOnlyMutateInstantiatedTables, false) LUAU_FASTFLAGVARIABLE(LuauPropertiesGetExpectedType, false) LUAU_FASTFLAGVARIABLE(LuauStatFunctionSimplify2, false) LUAU_FASTFLAGVARIABLE(LuauUnsealedTableLiteral, false) +LUAU_FASTFLAG(LuauTypeMismatchModuleName) LUAU_FASTFLAGVARIABLE(LuauTwoPassAliasDefinitionFix, false) LUAU_FASTFLAGVARIABLE(LuauAssertStripsFalsyTypes, false) LUAU_FASTFLAGVARIABLE(LuauReturnAnyInsteadOfICE, false) // Eventually removed as false. @@ -47,6 +46,8 @@ LUAU_FASTFLAGVARIABLE(LuauDoNotAccidentallyDependOnPointerOrdering, false) LUAU_FASTFLAGVARIABLE(LuauFixArgumentCountMismatchAmountWithGenericTypes, false) LUAU_FASTFLAGVARIABLE(LuauFixIncorrectLineNumberDuplicateType, false) LUAU_FASTFLAG(LuauAnyInIsOptionalIsOptional) +LUAU_FASTFLAGVARIABLE(LuauDecoupleOperatorInferenceFromUnifiedTypeInference, false) +LUAU_FASTFLAGVARIABLE(LuauArgCountMismatchSaysAtLeastWhenVariadic, false) namespace Luau { @@ -291,6 +292,7 @@ ModulePtr TypeChecker::check(const SourceModule& module, Mode mode, std::optiona // Clear unifier cache since it's keyed off internal types that get deallocated // This avoids fake cross-module cache hits and keeps cache size at bay when typechecking large module graphs. unifierState.cachedUnify.clear(); + unifierState.cachedUnifyError.clear(); unifierState.skipCacheForType.clear(); if (FFlag::LuauTwoPassAliasDefinitionFix) @@ -1303,7 +1305,7 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatTypeAlias& typealias { // If the table is already named and we want to rename the type function, we have to bind new alias to a copy // Additionally, we can't modify types that come from other modules - if (ttv->name || (FFlag::LuauImmutableTypes && follow(ty)->owningArena != ¤tModule->internalTypes)) + if (ttv->name || follow(ty)->owningArena != ¤tModule->internalTypes) { bool sameTys = std::equal(ttv->instantiatedTypeParams.begin(), ttv->instantiatedTypeParams.end(), binding->typeParams.begin(), binding->typeParams.end(), [](auto&& itp, auto&& tp) { @@ -1315,7 +1317,7 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatTypeAlias& typealias }); // Copy can be skipped if this is an identical alias - if ((FFlag::LuauImmutableTypes && !ttv->name) || ttv->name != name || !sameTys || !sameTps) + if (!ttv->name || ttv->name != name || !sameTys || !sameTps) { // This is a shallow clone, original recursive links to self are not updated TableTypeVar clone = TableTypeVar{ttv->props, ttv->indexer, ttv->level, ttv->state}; @@ -1349,7 +1351,7 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatTypeAlias& typealias else if (auto mtv = getMutable(follow(ty))) { // We can't modify types that come from other modules - if (!FFlag::LuauImmutableTypes || follow(ty)->owningArena == ¤tModule->internalTypes) + if (follow(ty)->owningArena == ¤tModule->internalTypes) mtv->syntheticName = name; } @@ -1512,14 +1514,14 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExpr& result = {nilType}; else if (const AstExprConstantBool* bexpr = expr.as()) { - if (FFlag::LuauSingletonTypes && (forceSingleton || (expectedType && maybeSingleton(*expectedType)))) + if (forceSingleton || (expectedType && maybeSingleton(*expectedType))) result = {singletonType(bexpr->value)}; else result = {booleanType}; } else if (const AstExprConstantString* sexpr = expr.as()) { - if (FFlag::LuauSingletonTypes && (forceSingleton || (expectedType && maybeSingleton(*expectedType)))) + if (forceSingleton || (expectedType && maybeSingleton(*expectedType))) result = {singletonType(std::string(sexpr->value.data, sexpr->value.size))}; else result = {stringType}; @@ -2490,12 +2492,24 @@ TypeId TypeChecker::checkBinaryOperation( lhsType = follow(lhsType); rhsType = follow(rhsType); - if (!isNonstrictMode() && get(lhsType)) + if (FFlag::LuauDecoupleOperatorInferenceFromUnifiedTypeInference) { - auto name = getIdentifierOfBaseVar(expr.left); - reportError(expr.location, CannotInferBinaryOperation{expr.op, name, CannotInferBinaryOperation::Operation}); - if (!FFlag::LuauErrorRecoveryType) - return errorRecoveryType(scope); + if (!isNonstrictMode() && get(lhsType)) + { + auto name = getIdentifierOfBaseVar(expr.left); + reportError(expr.location, CannotInferBinaryOperation{expr.op, name, CannotInferBinaryOperation::Operation}); + // We will fall-through to the `return anyType` check below. + } + } + else + { + if (!isNonstrictMode() && get(lhsType)) + { + auto name = getIdentifierOfBaseVar(expr.left); + reportError(expr.location, CannotInferBinaryOperation{expr.op, name, CannotInferBinaryOperation::Operation}); + if (!FFlag::LuauErrorRecoveryType) + return errorRecoveryType(scope); + } } // If we know nothing at all about the lhs type, we can usually say nothing about the result. @@ -3452,7 +3466,8 @@ void TypeChecker::checkArgumentList( { if (FFlag::LuauFixArgumentCountMismatchAmountWithGenericTypes) minParams = getMinParameterCount(&state.log, paramPack); - state.reportError(TypeError{state.location, CountMismatch{minParams, paramIndex}}); + bool isVariadic = FFlag::LuauArgCountMismatchSaysAtLeastWhenVariadic && !finite(paramPack, &state.log); + state.reportError(TypeError{state.location, CountMismatch{minParams, paramIndex, CountMismatch::Context::Arg, isVariadic}}); return; } ++paramIter; @@ -4163,13 +4178,7 @@ TypeId TypeChecker::checkRequire(const ScopePtr& scope, const ModuleInfo& module return errorRecoveryType(scope); } - if (FFlag::LuauImmutableTypes) - return *moduleType; - - SeenTypes seenTypes; - SeenTypePacks seenTypePacks; - CloneState cloneState; - return clone(*moduleType, currentModule->internalTypes, seenTypes, seenTypePacks, cloneState); + return *moduleType; } void TypeChecker::tablify(TypeId type) @@ -4941,10 +4950,19 @@ TypeId TypeChecker::resolveType(const ScopePtr& scope, const AstType& annotation if (const auto& indexer = table->indexer) tableIndexer = TableIndexer(resolveType(scope, *indexer->indexType), resolveType(scope, *indexer->resultType)); - return addType(TableTypeVar{ - props, tableIndexer, scope->level, - TableState::Sealed // FIXME: probably want a way to annotate other kinds of tables maybe - }); + if (FFlag::LuauTypeMismatchModuleName) + { + TableTypeVar ttv{props, tableIndexer, scope->level, TableState::Sealed}; + ttv.definitionModuleName = currentModuleName; + return addType(std::move(ttv)); + } + else + { + return addType(TableTypeVar{ + props, tableIndexer, scope->level, + TableState::Sealed // FIXME: probably want a way to annotate other kinds of tables maybe + }); + } } else if (const auto& func = annotation.as()) { @@ -5206,6 +5224,9 @@ TypeId TypeChecker::instantiateTypeFun(const ScopePtr& scope, const TypeFun& tf, { ttv->instantiatedTypeParams = typeParams; ttv->instantiatedTypePackParams = typePackParams; + + if (FFlag::LuauTypeMismatchModuleName) + ttv->definitionModuleName = currentModuleName; } return instantiated; diff --git a/Analysis/src/TypePack.cpp b/Analysis/src/TypePack.cpp index 91123f46..5bb05234 100644 --- a/Analysis/src/TypePack.cpp +++ b/Analysis/src/TypePack.cpp @@ -222,20 +222,21 @@ TypePackId follow(TypePackId tp, std::function mapper) } } -size_t size(TypePackId tp) +size_t size(TypePackId tp, TxnLog* log) { - if (auto pack = get(follow(tp))) - return size(*pack); + tp = log ? log->follow(tp) : follow(tp); + if (auto pack = get(tp)) + return size(*pack, log); else return 0; } -bool finite(TypePackId tp) +bool finite(TypePackId tp, TxnLog* log) { - tp = follow(tp); + tp = log ? log->follow(tp) : follow(tp); if (auto pack = get(tp)) - return pack->tail ? finite(*pack->tail) : true; + return pack->tail ? finite(*pack->tail, log) : true; if (get(tp)) return false; @@ -243,14 +244,14 @@ bool finite(TypePackId tp) return true; } -size_t size(const TypePack& tp) +size_t size(const TypePack& tp, TxnLog* log) { size_t result = tp.head.size(); if (tp.tail) { - const TypePack* tail = get(follow(*tp.tail)); + const TypePack* tail = get(log ? log->follow(*tp.tail) : follow(*tp.tail)); if (tail) - result += size(*tail); + result += size(*tail, log); } return result; } diff --git a/Analysis/src/TypeVar.cpp b/Analysis/src/TypeVar.cpp index 89549535..36545ad9 100644 --- a/Analysis/src/TypeVar.cpp +++ b/Analysis/src/TypeVar.cpp @@ -290,6 +290,24 @@ const std::string* getName(TypeId type) return nullptr; } +std::optional getDefinitionModuleName(TypeId type) +{ + type = follow(type); + + if (auto ttv = get(type)) + { + if (!ttv->definitionModuleName.empty()) + return ttv->definitionModuleName; + } + else if (auto ftv = get(type)) + { + if (ftv->definition) + return ftv->definition->definitionModuleName; + } + + return std::nullopt; +} + bool isSubset(const UnionTypeVar& super, const UnionTypeVar& sub) { std::unordered_set superTypes; diff --git a/Analysis/src/Unifier.cpp b/Analysis/src/Unifier.cpp index 60a9c9a5..398dc9e2 100644 --- a/Analysis/src/Unifier.cpp +++ b/Analysis/src/Unifier.cpp @@ -14,10 +14,9 @@ LUAU_FASTINT(LuauTypeInferRecursionLimit); LUAU_FASTINT(LuauTypeInferTypePackLoopLimit); -LUAU_FASTFLAG(LuauImmutableTypes) LUAU_FASTINTVARIABLE(LuauTypeInferIterationLimit, 2000); +LUAU_FASTFLAGVARIABLE(LuauExtendedIndexerError, false); LUAU_FASTFLAGVARIABLE(LuauTableSubtypingVariance2, false); -LUAU_FASTFLAG(LuauSingletonTypes) LUAU_FASTFLAG(LuauErrorRecoveryType); LUAU_FASTFLAGVARIABLE(LuauSubtypingAddOptPropsToUnsealedTables, false) LUAU_FASTFLAGVARIABLE(LuauWidenIfSupertypeIsFree2, false) @@ -26,6 +25,7 @@ LUAU_FASTFLAGVARIABLE(LuauTxnLogSeesTypePacks2, false) LUAU_FASTFLAGVARIABLE(LuauTxnLogCheckForInvalidation, false) LUAU_FASTFLAGVARIABLE(LuauTxnLogRefreshFunctionPointers, false) LUAU_FASTFLAGVARIABLE(LuauTxnLogDontRetryForIndexers, false) +LUAU_FASTFLAGVARIABLE(LuauUnifierCacheErrors, false) LUAU_FASTFLAG(LuauAnyInIsOptionalIsOptional) namespace Luau @@ -63,7 +63,7 @@ struct PromoteTypeLevels bool operator()(TID ty, const T&) { // Type levels of types from other modules are already global, so we don't need to promote anything inside - if (FFlag::LuauImmutableTypes && ty->owningArena != typeArena) + if (ty->owningArena != typeArena) return false; return true; @@ -83,7 +83,7 @@ struct PromoteTypeLevels bool operator()(TypeId ty, const FunctionTypeVar&) { // Type levels of types from other modules are already global, so we don't need to promote anything inside - if (FFlag::LuauImmutableTypes && ty->owningArena != typeArena) + if (ty->owningArena != typeArena) return false; promote(ty, log.getMutable(ty)); @@ -93,7 +93,7 @@ struct PromoteTypeLevels bool operator()(TypeId ty, const TableTypeVar& ttv) { // Type levels of types from other modules are already global, so we don't need to promote anything inside - if (FFlag::LuauImmutableTypes && ty->owningArena != typeArena) + if (ty->owningArena != typeArena) return false; if (ttv.state != TableState::Free && ttv.state != TableState::Generic) @@ -118,7 +118,7 @@ struct PromoteTypeLevels static void promoteTypeLevels(TxnLog& log, const TypeArena* typeArena, TypeLevel minLevel, TypeId ty) { // Type levels of types from other modules are already global, so we don't need to promote anything inside - if (FFlag::LuauImmutableTypes && ty->owningArena != typeArena) + if (ty->owningArena != typeArena) return; PromoteTypeLevels ptl{log, typeArena, minLevel}; @@ -130,7 +130,7 @@ static void promoteTypeLevels(TxnLog& log, const TypeArena* typeArena, TypeLevel void promoteTypeLevels(TxnLog& log, const TypeArena* typeArena, TypeLevel minLevel, TypePackId tp) { // Type levels of types from other modules are already global, so we don't need to promote anything inside - if (FFlag::LuauImmutableTypes && tp->owningArena != typeArena) + if (tp->owningArena != typeArena) return; PromoteTypeLevels ptl{log, typeArena, minLevel}; @@ -170,7 +170,7 @@ struct SkipCacheForType bool operator()(TypeId ty, const TableTypeVar&) { // Types from other modules don't contain mutable elements and are ok to cache - if (FFlag::LuauImmutableTypes && ty->owningArena != typeArena) + if (ty->owningArena != typeArena) return false; TableTypeVar& ttv = *getMutable(ty); @@ -194,7 +194,7 @@ struct SkipCacheForType bool operator()(TypeId ty, const T& t) { // Types from other modules don't contain mutable elements and are ok to cache - if (FFlag::LuauImmutableTypes && ty->owningArena != typeArena) + if (ty->owningArena != typeArena) return false; const bool* prev = skipCacheForType.find(ty); @@ -212,7 +212,7 @@ struct SkipCacheForType bool operator()(TypePackId tp, const T&) { // Types from other modules don't contain mutable elements and are ok to cache - if (FFlag::LuauImmutableTypes && tp->owningArena != typeArena) + if (tp->owningArena != typeArena) return false; return true; @@ -445,12 +445,33 @@ void Unifier::tryUnify_(TypeId subTy, TypeId superTy, bool isFunctionCall, bool if (get(subTy) || get(subTy)) return tryUnifyWithAny(superTy, subTy); - bool cacheEnabled = !isFunctionCall && !isIntersection; + bool cacheEnabled; auto& cache = sharedState.cachedUnify; // What if the types are immutable and we proved their relation before - if (cacheEnabled && cache.contains({superTy, subTy}) && (variance == Covariant || cache.contains({subTy, superTy}))) - return; + if (FFlag::LuauUnifierCacheErrors) + { + cacheEnabled = !isFunctionCall && !isIntersection && variance == Invariant; + + if (cacheEnabled) + { + if (cache.contains({subTy, superTy})) + return; + + if (auto error = sharedState.cachedUnifyError.find({subTy, superTy})) + { + reportError(TypeError{location, *error}); + return; + } + } + } + else + { + cacheEnabled = !isFunctionCall && !isIntersection; + + if (cacheEnabled && cache.contains({superTy, subTy}) && (variance == Covariant || cache.contains({subTy, superTy}))) + return; + } // If we have seen this pair of types before, we are currently recursing into cyclic types. // Here, we assume that the types unify. If they do not, we will find out as we roll back @@ -461,6 +482,8 @@ void Unifier::tryUnify_(TypeId subTy, TypeId superTy, bool isFunctionCall, bool log.pushSeen(superTy, subTy); + size_t errorCount = errors.size(); + if (const UnionTypeVar* uv = log.getMutable(subTy)) { tryUnifyUnionWithType(subTy, uv, superTy); @@ -480,8 +503,7 @@ void Unifier::tryUnify_(TypeId subTy, TypeId superTy, bool isFunctionCall, bool else if (log.getMutable(superTy) && log.getMutable(subTy)) tryUnifyPrimitives(subTy, superTy); - else if (FFlag::LuauSingletonTypes && (log.getMutable(superTy) || log.getMutable(superTy)) && - log.getMutable(subTy)) + else if ((log.getMutable(superTy) || log.getMutable(superTy)) && log.getMutable(subTy)) tryUnifySingletons(subTy, superTy); else if (log.getMutable(superTy) && log.getMutable(subTy)) @@ -491,8 +513,11 @@ void Unifier::tryUnify_(TypeId subTy, TypeId superTy, bool isFunctionCall, bool { tryUnifyTables(subTy, superTy, isIntersection); - if (cacheEnabled && errors.empty()) - cacheResult(subTy, superTy); + if (!FFlag::LuauUnifierCacheErrors) + { + if (cacheEnabled && errors.empty()) + cacheResult_DEPRECATED(subTy, superTy); + } } // tryUnifyWithMetatable assumes its first argument is a MetatableTypeVar. The check is otherwise symmetrical. @@ -512,6 +537,9 @@ void Unifier::tryUnify_(TypeId subTy, TypeId superTy, bool isFunctionCall, bool else reportError(TypeError{location, TypeMismatch{superTy, subTy}}); + if (FFlag::LuauUnifierCacheErrors && cacheEnabled) + cacheResult(subTy, superTy, errorCount); + log.popSeen(superTy, subTy); } @@ -646,10 +674,21 @@ void Unifier::tryUnifyTypeWithUnion(TypeId subTy, TypeId superTy, const UnionTyp { TypeId type = uv->options[i]; - if (cache.contains({type, subTy}) && (variance == Covariant || cache.contains({subTy, type}))) + if (FFlag::LuauUnifierCacheErrors) { - startIndex = i; - break; + if (cache.contains({subTy, type})) + { + startIndex = i; + break; + } + } + else + { + if (cache.contains({type, subTy}) && (variance == Covariant || cache.contains({subTy, type}))) + { + startIndex = i; + break; + } } } } @@ -737,10 +776,21 @@ void Unifier::tryUnifyIntersectionWithType(TypeId subTy, const IntersectionTypeV { TypeId type = uv->parts[i]; - if (cache.contains({superTy, type}) && (variance == Covariant || cache.contains({type, superTy}))) + if (FFlag::LuauUnifierCacheErrors) { - startIndex = i; - break; + if (cache.contains({type, superTy})) + { + startIndex = i; + break; + } + } + else + { + if (cache.contains({superTy, type}) && (variance == Covariant || cache.contains({type, superTy}))) + { + startIndex = i; + break; + } } } } @@ -771,17 +821,17 @@ void Unifier::tryUnifyIntersectionWithType(TypeId subTy, const IntersectionTypeV } } -void Unifier::cacheResult(TypeId subTy, TypeId superTy) +bool Unifier::canCacheResult(TypeId subTy, TypeId superTy) { bool* superTyInfo = sharedState.skipCacheForType.find(superTy); if (superTyInfo && *superTyInfo) - return; + return false; bool* subTyInfo = sharedState.skipCacheForType.find(subTy); if (subTyInfo && *subTyInfo) - return; + return false; auto skipCacheFor = [this](TypeId ty) { SkipCacheForType visitor{sharedState.skipCacheForType, types}; @@ -793,9 +843,33 @@ void Unifier::cacheResult(TypeId subTy, TypeId superTy) }; if (!superTyInfo && skipCacheFor(superTy)) - return; + return false; if (!subTyInfo && skipCacheFor(subTy)) + return false; + + return true; +} + +void Unifier::cacheResult(TypeId subTy, TypeId superTy, size_t prevErrorCount) +{ + if (errors.size() == prevErrorCount) + { + if (canCacheResult(subTy, superTy)) + sharedState.cachedUnify.insert({subTy, superTy}); + } + else if (errors.size() == prevErrorCount + 1) + { + if (canCacheResult(subTy, superTy)) + sharedState.cachedUnifyError[{subTy, superTy}] = errors.back().data; + } +} + +void Unifier::cacheResult_DEPRECATED(TypeId subTy, TypeId superTy) +{ + LUAU_ASSERT(!FFlag::LuauUnifierCacheErrors); + + if (!canCacheResult(subTy, superTy)) return; sharedState.cachedUnify.insert({superTy, subTy}); @@ -1283,24 +1357,6 @@ void Unifier::tryUnifyFunctions(TypeId subTy, TypeId superTy, bool isFunctionCal subFunction = log.getMutable(subTy); } - if (!FFlag::LuauImmutableTypes) - { - if (superFunction->definition && !subFunction->definition && !subTy->persistent) - { - PendingType* newSubTy = log.queue(subTy); - FunctionTypeVar* newSubFtv = getMutable(newSubTy); - LUAU_ASSERT(newSubFtv); - newSubFtv->definition = superFunction->definition; - } - else if (!superFunction->definition && subFunction->definition && !superTy->persistent) - { - PendingType* newSuperTy = log.queue(superTy); - FunctionTypeVar* newSuperFtv = getMutable(newSuperTy); - LUAU_ASSERT(newSuperFtv); - newSuperFtv->definition = subFunction->definition; - } - } - ctx = context; if (FFlag::LuauTxnLogSeesTypePacks2) @@ -1563,8 +1619,25 @@ void Unifier::tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) variance = Invariant; Unifier innerState = makeChildUnifier(); - innerState.tryUnifyIndexer(*subTable->indexer, *superTable->indexer); - checkChildUnifierTypeMismatch(innerState.errors, superTy, subTy); + + if (FFlag::LuauExtendedIndexerError) + { + innerState.tryUnify_(subTable->indexer->indexType, superTable->indexer->indexType); + + bool reported = !innerState.errors.empty(); + + checkChildUnifierTypeMismatch(innerState.errors, "[indexer key]", superTy, subTy); + + innerState.tryUnify_(subTable->indexer->indexResultType, superTable->indexer->indexResultType); + + if (!reported) + checkChildUnifierTypeMismatch(innerState.errors, "[indexer value]", superTy, subTy); + } + else + { + innerState.tryUnifyIndexer(*subTable->indexer, *superTable->indexer); + checkChildUnifierTypeMismatch(innerState.errors, superTy, subTy); + } if (innerState.errors.empty()) log.concat(std::move(innerState.log)); @@ -1771,6 +1844,7 @@ void Unifier::DEPRECATED_tryUnifyTables(TypeId subTy, TypeId superTy, bool isInt void Unifier::tryUnifyFreeTable(TypeId subTy, TypeId superTy) { + LUAU_ASSERT(!FFlag::LuauTableSubtypingVariance2); TableTypeVar* freeTable = log.getMutable(superTy); TableTypeVar* subTable = log.getMutable(subTy); @@ -1840,6 +1914,7 @@ void Unifier::tryUnifyFreeTable(TypeId subTy, TypeId superTy) void Unifier::tryUnifySealedTables(TypeId subTy, TypeId superTy, bool isIntersection) { + LUAU_ASSERT(!FFlag::LuauTableSubtypingVariance2); TableTypeVar* superTable = log.getMutable(superTy); TableTypeVar* subTable = log.getMutable(subTy); @@ -2120,6 +2195,8 @@ void Unifier::tryUnifyWithClass(TypeId subTy, TypeId superTy, bool reversed) void Unifier::tryUnifyIndexer(const TableIndexer& subIndexer, const TableIndexer& superIndexer) { + LUAU_ASSERT(!FFlag::LuauTableSubtypingVariance2 || !FFlag::LuauExtendedIndexerError); + tryUnify_(subIndexer.indexType, superIndexer.indexType); tryUnify_(subIndexer.indexResultType, superIndexer.indexResultType); } @@ -2211,7 +2288,7 @@ static void tryUnifyWithAny(std::vector& queue, Unifier& state, DenseHas queue.pop_back(); // Types from other modules don't have free types - if (FFlag::LuauImmutableTypes && ty->owningArena != typeArena) + if (ty->owningArena != typeArena) continue; if (seen.find(ty)) diff --git a/Ast/src/Parser.cpp b/Ast/src/Parser.cpp index 941a3ea4..f6dfd904 100644 --- a/Ast/src/Parser.cpp +++ b/Ast/src/Parser.cpp @@ -10,8 +10,6 @@ // See docs/SyntaxChanges.md for an explanation. LUAU_FASTINTVARIABLE(LuauRecursionLimit, 1000) LUAU_FASTINTVARIABLE(LuauParseErrorLimit, 100) -LUAU_FASTFLAGVARIABLE(LuauParseSingletonTypes, false) -LUAU_FASTFLAGVARIABLE(LuauTableFieldFunctionDebugname, false) namespace Luau { @@ -1233,8 +1231,7 @@ AstType* Parser::parseTableTypeAnnotation() while (lexer.current().type != '}') { - if (FFlag::LuauParseSingletonTypes && lexer.current().type == '[' && - (lexer.lookahead().type == Lexeme::RawString || lexer.lookahead().type == Lexeme::QuotedString)) + if (lexer.current().type == '[' && (lexer.lookahead().type == Lexeme::RawString || lexer.lookahead().type == Lexeme::QuotedString)) { const Lexeme begin = lexer.current(); nextLexeme(); // [ @@ -1500,17 +1497,17 @@ AstTypeOrPack Parser::parseSimpleTypeAnnotation(bool allowPack) nextLexeme(); return {allocator.alloc(begin, std::nullopt, nameNil), {}}; } - else if (FFlag::LuauParseSingletonTypes && lexer.current().type == Lexeme::ReservedTrue) + else if (lexer.current().type == Lexeme::ReservedTrue) { nextLexeme(); return {allocator.alloc(begin, true)}; } - else if (FFlag::LuauParseSingletonTypes && lexer.current().type == Lexeme::ReservedFalse) + else if (lexer.current().type == Lexeme::ReservedFalse) { nextLexeme(); return {allocator.alloc(begin, false)}; } - else if (FFlag::LuauParseSingletonTypes && (lexer.current().type == Lexeme::RawString || lexer.current().type == Lexeme::QuotedString)) + else if (lexer.current().type == Lexeme::RawString || lexer.current().type == Lexeme::QuotedString) { if (std::optional> value = parseCharArray()) { @@ -1520,7 +1517,7 @@ AstTypeOrPack Parser::parseSimpleTypeAnnotation(bool allowPack) else return {reportTypeAnnotationError(begin, {}, /*isMissing*/ false, "String literal contains malformed escape sequence")}; } - else if (FFlag::LuauParseSingletonTypes && lexer.current().type == Lexeme::BrokenString) + else if (lexer.current().type == Lexeme::BrokenString) { Location location = lexer.current().location; nextLexeme(); @@ -2189,11 +2186,8 @@ AstExpr* Parser::parseTableConstructor() AstExpr* key = allocator.alloc(name.location, nameString); AstExpr* value = parseExpr(); - if (FFlag::LuauTableFieldFunctionDebugname) - { - if (AstExprFunction* func = value->as()) - func->debugname = name.name; - } + if (AstExprFunction* func = value->as()) + func->debugname = name.name; items.push_back({AstExprTable::Item::Record, key, value}); } diff --git a/VM/src/lapi.cpp b/VM/src/lapi.cpp index 3c087314..46b10934 100644 --- a/VM/src/lapi.cpp +++ b/VM/src/lapi.cpp @@ -14,8 +14,6 @@ #include -LUAU_FASTFLAG(LuauGcAdditionalStats) - const char* lua_ident = "$Lua: Lua 5.1.4 Copyright (C) 1994-2008 Lua.org, PUC-Rio $\n" "$Authors: R. Ierusalimschy, L. H. de Figueiredo & W. Celes $\n" "$URL: www.lua.org $\n"; @@ -1060,8 +1058,11 @@ int lua_gc(lua_State* L, int what, int data) g->GCthreshold = 0; bool waspaused = g->gcstate == GCSpause; - double startmarktime = g->gcstats.currcycle.marktime; - double startsweeptime = g->gcstats.currcycle.sweeptime; + +#ifdef LUAI_GCMETRICS + double startmarktime = g->gcmetrics.currcycle.marktime; + double startsweeptime = g->gcmetrics.currcycle.sweeptime; +#endif // track how much work the loop will actually perform size_t actualwork = 0; @@ -1079,31 +1080,30 @@ int lua_gc(lua_State* L, int what, int data) } } - if (FFlag::LuauGcAdditionalStats) +#ifdef LUAI_GCMETRICS + // record explicit step statistics + GCCycleMetrics* cyclemetrics = g->gcstate == GCSpause ? &g->gcmetrics.lastcycle : &g->gcmetrics.currcycle; + + double totalmarktime = cyclemetrics->marktime - startmarktime; + double totalsweeptime = cyclemetrics->sweeptime - startsweeptime; + + if (totalmarktime > 0.0) { - // record explicit step statistics - GCCycleStats* cyclestats = g->gcstate == GCSpause ? &g->gcstats.lastcycle : &g->gcstats.currcycle; + cyclemetrics->markexplicitsteps++; - double totalmarktime = cyclestats->marktime - startmarktime; - double totalsweeptime = cyclestats->sweeptime - startsweeptime; - - if (totalmarktime > 0.0) - { - cyclestats->markexplicitsteps++; - - if (totalmarktime > cyclestats->markmaxexplicittime) - cyclestats->markmaxexplicittime = totalmarktime; - } - - if (totalsweeptime > 0.0) - { - cyclestats->sweepexplicitsteps++; - - if (totalsweeptime > cyclestats->sweepmaxexplicittime) - cyclestats->sweepmaxexplicittime = totalsweeptime; - } + if (totalmarktime > cyclemetrics->markmaxexplicittime) + cyclemetrics->markmaxexplicittime = totalmarktime; } + if (totalsweeptime > 0.0) + { + cyclemetrics->sweepexplicitsteps++; + + if (totalsweeptime > cyclemetrics->sweepmaxexplicittime) + cyclemetrics->sweepmaxexplicittime = totalsweeptime; + } +#endif + // if cycle hasn't finished, advance threshold forward for the amount of extra work performed if (g->gcstate != GCSpause) { diff --git a/VM/src/ldo.cpp b/VM/src/ldo.cpp index b5ae496b..c133a59e 100644 --- a/VM/src/ldo.cpp +++ b/VM/src/ldo.cpp @@ -17,8 +17,6 @@ #include -LUAU_FASTFLAG(LuauReduceStackReallocs) - /* ** {====================================================== ** Error-recovery functions @@ -33,6 +31,15 @@ struct lua_jmpbuf jmp_buf buf; }; +/* use POSIX versions of setjmp/longjmp if possible: they don't save/restore signal mask and are therefore faster */ +#if defined(__linux__) || defined(__APPLE__) +#define LUAU_SETJMP(buf) _setjmp(buf) +#define LUAU_LONGJMP(buf, code) _longjmp(buf, code) +#else +#define LUAU_SETJMP(buf) setjmp(buf) +#define LUAU_LONGJMP(buf, code) longjmp(buf, code) +#endif + int luaD_rawrunprotected(lua_State* L, Pfunc f, void* ud) { lua_jmpbuf jb; @@ -40,7 +47,7 @@ int luaD_rawrunprotected(lua_State* L, Pfunc f, void* ud) jb.status = 0; L->global->errorjmp = &jb; - if (setjmp(jb.buf) == 0) + if (LUAU_SETJMP(jb.buf) == 0) f(L, ud); L->global->errorjmp = jb.prev; @@ -52,7 +59,7 @@ l_noret luaD_throw(lua_State* L, int errcode) if (lua_jmpbuf* jb = L->global->errorjmp) { jb->status = errcode; - longjmp(jb->buf, 1); + LUAU_LONGJMP(jb->buf, 1); } if (L->global->cb.panic) @@ -165,8 +172,8 @@ static void correctstack(lua_State* L, TValue* oldstack) void luaD_reallocstack(lua_State* L, int newsize) { TValue* oldstack = L->stack; - int realsize = newsize + (FFlag::LuauReduceStackReallocs ? EXTRA_STACK : 1 + EXTRA_STACK); - LUAU_ASSERT(L->stack_last - L->stack == L->stacksize - (FFlag::LuauReduceStackReallocs ? EXTRA_STACK : 1 + EXTRA_STACK)); + int realsize = newsize + EXTRA_STACK; + LUAU_ASSERT(L->stack_last - L->stack == L->stacksize - EXTRA_STACK); luaM_reallocarray(L, L->stack, L->stacksize, realsize, TValue, L->memcat); TValue* newstack = L->stack; for (int i = L->stacksize; i < realsize; i++) @@ -514,7 +521,7 @@ static void callerrfunc(lua_State* L, void* ud) static void restore_stack_limit(lua_State* L) { - LUAU_ASSERT(L->stack_last - L->stack == L->stacksize - (FFlag::LuauReduceStackReallocs ? EXTRA_STACK : 1 + EXTRA_STACK)); + LUAU_ASSERT(L->stack_last - L->stack == L->stacksize - EXTRA_STACK); if (L->size_ci > LUAI_MAXCALLS) { /* there was an overflow? */ int inuse = cast_int(L->ci - L->base_ci); diff --git a/VM/src/ldo.h b/VM/src/ldo.h index 1c1480d6..6e16e6f1 100644 --- a/VM/src/ldo.h +++ b/VM/src/ldo.h @@ -11,7 +11,7 @@ if ((char*)L->stack_last - (char*)L->top <= (n) * (int)sizeof(TValue)) \ luaD_growstack(L, n); \ else \ - condhardstacktests(luaD_reallocstack(L, L->stacksize - (FFlag::LuauReduceStackReallocs ? EXTRA_STACK : 1 + EXTRA_STACK))); + condhardstacktests(luaD_reallocstack(L, L->stacksize - EXTRA_STACK)); #define incr_top(L) \ { \ diff --git a/VM/src/lgc.cpp b/VM/src/lgc.cpp index a656854e..8fc930d5 100644 --- a/VM/src/lgc.cpp +++ b/VM/src/lgc.cpp @@ -11,8 +11,6 @@ #include "lmem.h" #include "ludata.h" -LUAU_FASTFLAGVARIABLE(LuauGcAdditionalStats, false) - #include #define GC_SWEEPMAX 40 @@ -48,7 +46,8 @@ LUAU_FASTFLAGVARIABLE(LuauGcAdditionalStats, false) reallymarkobject(g, obj2gco(t)); \ } -static void recordGcStateTime(global_State* g, int startgcstate, double seconds, bool assist) +#ifdef LUAI_GCMETRICS +static void recordGcStateStep(global_State* g, int startgcstate, double seconds, bool assist, size_t work) { switch (startgcstate) { @@ -56,58 +55,76 @@ static void recordGcStateTime(global_State* g, int startgcstate, double seconds, // record root mark time if we have switched to next state if (g->gcstate == GCSpropagate) { - g->gcstats.currcycle.marktime += seconds; + g->gcmetrics.currcycle.marktime += seconds; - if (FFlag::LuauGcAdditionalStats && assist) - g->gcstats.currcycle.markassisttime += seconds; + if (assist) + g->gcmetrics.currcycle.markassisttime += seconds; } break; case GCSpropagate: case GCSpropagateagain: - g->gcstats.currcycle.marktime += seconds; + g->gcmetrics.currcycle.marktime += seconds; + g->gcmetrics.currcycle.markrequests += g->gcstepsize; - if (FFlag::LuauGcAdditionalStats && assist) - g->gcstats.currcycle.markassisttime += seconds; + if (assist) + g->gcmetrics.currcycle.markassisttime += seconds; break; case GCSatomic: - g->gcstats.currcycle.atomictime += seconds; + g->gcmetrics.currcycle.atomictime += seconds; break; case GCSsweep: - g->gcstats.currcycle.sweeptime += seconds; + g->gcmetrics.currcycle.sweeptime += seconds; + g->gcmetrics.currcycle.sweeprequests += g->gcstepsize; - if (FFlag::LuauGcAdditionalStats && assist) - g->gcstats.currcycle.sweepassisttime += seconds; + if (assist) + g->gcmetrics.currcycle.sweepassisttime += seconds; break; default: LUAU_ASSERT(!"Unexpected GC state"); } if (assist) - g->gcstats.stepassisttimeacc += seconds; + { + g->gcmetrics.stepassisttimeacc += seconds; + g->gcmetrics.currcycle.assistwork += work; + g->gcmetrics.currcycle.assistrequests += g->gcstepsize; + } else - g->gcstats.stepexplicittimeacc += seconds; + { + g->gcmetrics.stepexplicittimeacc += seconds; + g->gcmetrics.currcycle.explicitwork += work; + g->gcmetrics.currcycle.explicitrequests += g->gcstepsize; + } } -static void startGcCycleStats(global_State* g) +static double recordGcDeltaTime(double& timer) { - g->gcstats.currcycle.starttimestamp = lua_clock(); - g->gcstats.currcycle.pausetime = g->gcstats.currcycle.starttimestamp - g->gcstats.lastcycle.endtimestamp; + double now = lua_clock(); + double delta = now - timer; + timer = now; + return delta; } -static void finishGcCycleStats(global_State* g) +static void startGcCycleMetrics(global_State* g) { - g->gcstats.currcycle.endtimestamp = lua_clock(); - g->gcstats.currcycle.endtotalsizebytes = g->totalbytes; - - g->gcstats.completedcycles++; - g->gcstats.lastcycle = g->gcstats.currcycle; - g->gcstats.currcycle = GCCycleStats(); - - g->gcstats.cyclestatsacc.marktime += g->gcstats.lastcycle.marktime; - g->gcstats.cyclestatsacc.atomictime += g->gcstats.lastcycle.atomictime; - g->gcstats.cyclestatsacc.sweeptime += g->gcstats.lastcycle.sweeptime; + g->gcmetrics.currcycle.starttimestamp = lua_clock(); + g->gcmetrics.currcycle.pausetime = g->gcmetrics.currcycle.starttimestamp - g->gcmetrics.lastcycle.endtimestamp; } +static void finishGcCycleMetrics(global_State* g) +{ + g->gcmetrics.currcycle.endtimestamp = lua_clock(); + g->gcmetrics.currcycle.endtotalsizebytes = g->totalbytes; + + g->gcmetrics.completedcycles++; + g->gcmetrics.lastcycle = g->gcmetrics.currcycle; + g->gcmetrics.currcycle = GCCycleMetrics(); + + g->gcmetrics.currcycle.starttotalsizebytes = g->totalbytes; + g->gcmetrics.currcycle.heaptriggersizebytes = g->GCthreshold; +} +#endif + static void removeentry(LuaNode* n) { LUAU_ASSERT(ttisnil(gval(n))); @@ -598,20 +615,19 @@ static size_t atomic(lua_State* L) LUAU_ASSERT(g->gcstate == GCSatomic); size_t work = 0; + +#ifdef LUAI_GCMETRICS double currts = lua_clock(); - double prevts = currts; +#endif /* remark occasional upvalues of (maybe) dead threads */ work += remarkupvals(g); /* traverse objects caught by write barrier and by 'remarkupvals' */ work += propagateall(g); - if (FFlag::LuauGcAdditionalStats) - { - currts = lua_clock(); - g->gcstats.currcycle.atomictimeupval += currts - prevts; - prevts = currts; - } +#ifdef LUAI_GCMETRICS + g->gcmetrics.currcycle.atomictimeupval += recordGcDeltaTime(currts); +#endif /* remark weak tables */ g->gray = g->weak; @@ -621,34 +637,26 @@ static size_t atomic(lua_State* L) markmt(g); /* mark basic metatables (again) */ work += propagateall(g); - if (FFlag::LuauGcAdditionalStats) - { - currts = lua_clock(); - g->gcstats.currcycle.atomictimeweak += currts - prevts; - prevts = currts; - } +#ifdef LUAI_GCMETRICS + g->gcmetrics.currcycle.atomictimeweak += recordGcDeltaTime(currts); +#endif /* remark gray again */ g->gray = g->grayagain; g->grayagain = NULL; work += propagateall(g); - if (FFlag::LuauGcAdditionalStats) - { - currts = lua_clock(); - g->gcstats.currcycle.atomictimegray += currts - prevts; - prevts = currts; - } +#ifdef LUAI_GCMETRICS + g->gcmetrics.currcycle.atomictimegray += recordGcDeltaTime(currts); +#endif /* remove collected objects from weak tables */ work += cleartable(L, g->weak); g->weak = NULL; - if (FFlag::LuauGcAdditionalStats) - { - currts = lua_clock(); - g->gcstats.currcycle.atomictimeclear += currts - prevts; - } +#ifdef LUAI_GCMETRICS + g->gcmetrics.currcycle.atomictimeclear += recordGcDeltaTime(currts); +#endif /* flip current white */ g->currentwhite = cast_byte(otherwhite(g)); @@ -742,8 +750,9 @@ static size_t gcstep(lua_State* L, size_t limit) if (!g->gray) { - if (FFlag::LuauGcAdditionalStats) - g->gcstats.currcycle.propagatework = g->gcstats.currcycle.explicitwork + g->gcstats.currcycle.assistwork; +#ifdef LUAI_GCMETRICS + g->gcmetrics.currcycle.propagatework = g->gcmetrics.currcycle.explicitwork + g->gcmetrics.currcycle.assistwork; +#endif // perform one iteration over 'gray again' list g->gray = g->grayagain; @@ -762,9 +771,10 @@ static size_t gcstep(lua_State* L, size_t limit) if (!g->gray) /* no more `gray' objects */ { - if (FFlag::LuauGcAdditionalStats) - g->gcstats.currcycle.propagateagainwork = - g->gcstats.currcycle.explicitwork + g->gcstats.currcycle.assistwork - g->gcstats.currcycle.propagatework; +#ifdef LUAI_GCMETRICS + g->gcmetrics.currcycle.propagateagainwork = + g->gcmetrics.currcycle.explicitwork + g->gcmetrics.currcycle.assistwork - g->gcmetrics.currcycle.propagatework; +#endif g->gcstate = GCSatomic; } @@ -772,8 +782,13 @@ static size_t gcstep(lua_State* L, size_t limit) } case GCSatomic: { - g->gcstats.currcycle.atomicstarttimestamp = lua_clock(); - g->gcstats.currcycle.atomicstarttotalsizebytes = g->totalbytes; +#ifdef LUAI_GCMETRICS + g->gcmetrics.currcycle.atomicstarttimestamp = lua_clock(); + g->gcmetrics.currcycle.atomicstarttotalsizebytes = g->totalbytes; +#endif + + g->gcstats.atomicstarttimestamp = lua_clock(); + g->gcstats.atomicstarttotalsizebytes = g->totalbytes; cost = atomic(L); /* finish mark phase */ @@ -809,18 +824,20 @@ static size_t gcstep(lua_State* L, size_t limit) return cost; } -static int64_t getheaptriggererroroffset(GCHeapTriggerStats* triggerstats, GCCycleStats* cyclestats) +static int64_t getheaptriggererroroffset(global_State* g) { // adjust for error using Proportional-Integral controller // https://en.wikipedia.org/wiki/PID_controller - int32_t errorKb = int32_t((cyclestats->atomicstarttotalsizebytes - cyclestats->heapgoalsizebytes) / 1024); + int32_t errorKb = int32_t((g->gcstats.atomicstarttotalsizebytes - g->gcstats.heapgoalsizebytes) / 1024); // we use sliding window for the error integral to avoid error sum 'windup' when the desired target cannot be reached - int32_t* slot = &triggerstats->terms[triggerstats->termpos % triggerstats->termcount]; + const size_t triggertermcount = sizeof(g->gcstats.triggerterms) / sizeof(g->gcstats.triggerterms[0]); + + int32_t* slot = &g->gcstats.triggerterms[g->gcstats.triggertermpos % triggertermcount]; int32_t prev = *slot; *slot = errorKb; - triggerstats->integral += errorKb - prev; - triggerstats->termpos++; + g->gcstats.triggerintegral += errorKb - prev; + g->gcstats.triggertermpos++; // controller tuning // https://en.wikipedia.org/wiki/Ziegler%E2%80%93Nichols_method @@ -832,7 +849,7 @@ static int64_t getheaptriggererroroffset(GCHeapTriggerStats* triggerstats, GCCyc const double Ki = 0.54 * Ku / Ti; // integral gain double proportionalTerm = Kp * errorKb; - double integralTerm = Ki * triggerstats->integral; + double integralTerm = Ki * g->gcstats.triggerintegral; double totalTerm = proportionalTerm + integralTerm; @@ -841,23 +858,20 @@ static int64_t getheaptriggererroroffset(GCHeapTriggerStats* triggerstats, GCCyc static size_t getheaptrigger(global_State* g, size_t heapgoal) { - GCCycleStats* lastcycle = &g->gcstats.lastcycle; - GCCycleStats* currcycle = &g->gcstats.currcycle; - // adjust threshold based on a guess of how many bytes will be allocated between the cycle start and sweep phase // our goal is to begin the sweep when used memory has reached the heap goal const double durationthreshold = 1e-3; - double allocationduration = currcycle->atomicstarttimestamp - lastcycle->endtimestamp; + double allocationduration = g->gcstats.atomicstarttimestamp - g->gcstats.endtimestamp; // avoid measuring intervals smaller than 1ms if (allocationduration < durationthreshold) return heapgoal; - double allocationrate = (currcycle->atomicstarttotalsizebytes - lastcycle->endtotalsizebytes) / allocationduration; - double markduration = currcycle->atomicstarttimestamp - currcycle->starttimestamp; + double allocationrate = (g->gcstats.atomicstarttotalsizebytes - g->gcstats.endtotalsizebytes) / allocationduration; + double markduration = g->gcstats.atomicstarttimestamp - g->gcstats.starttimestamp; int64_t expectedgrowth = int64_t(markduration * allocationrate); - int64_t offset = getheaptriggererroroffset(&g->gcstats.triggerstats, currcycle); + int64_t offset = getheaptriggererroroffset(g); int64_t heaptrigger = heapgoal - (expectedgrowth + offset); // clamp the trigger between memory use at the end of the cycle and the heap goal @@ -868,11 +882,6 @@ void luaC_step(lua_State* L, bool assist) { global_State* g = L->global; - if (assist) - g->gcstats.currcycle.assistrequests += g->gcstepsize; - else - g->gcstats.currcycle.explicitrequests += g->gcstepsize; - int lim = (g->gcstepsize / 100) * g->gcstepmul; /* how much to work */ LUAU_ASSERT(g->totalbytes >= g->GCthreshold); size_t debt = g->totalbytes - g->GCthreshold; @@ -881,24 +890,23 @@ void luaC_step(lua_State* L, bool assist) // at the start of the new cycle if (g->gcstate == GCSpause) - startGcCycleStats(g); + g->gcstats.starttimestamp = lua_clock(); + +#ifdef LUAI_GCMETRICS + if (g->gcstate == GCSpause) + startGcCycleMetrics(g); + + double lasttimestamp = lua_clock(); +#endif int lastgcstate = g->gcstate; - double lasttimestamp = lua_clock(); size_t work = gcstep(L, lim); + (void)work; - if (assist) - g->gcstats.currcycle.assistwork += work; - else - g->gcstats.currcycle.explicitwork += work; - - recordGcStateTime(g, lastgcstate, lua_clock() - lasttimestamp, assist); - - if (lastgcstate == GCSpropagate) - g->gcstats.currcycle.markrequests += g->gcstepsize; - else if (lastgcstate == GCSsweep) - g->gcstats.currcycle.sweeprequests += g->gcstepsize; +#ifdef LUAI_GCMETRICS + recordGcStateStep(g, lastgcstate, lua_clock() - lasttimestamp, assist, work); +#endif // at the end of the last cycle if (g->gcstate == GCSpause) @@ -909,13 +917,13 @@ void luaC_step(lua_State* L, bool assist) g->GCthreshold = heaptrigger; - finishGcCycleStats(g); + g->gcstats.heapgoalsizebytes = heapgoal; + g->gcstats.endtimestamp = lua_clock(); + g->gcstats.endtotalsizebytes = g->totalbytes; - if (FFlag::LuauGcAdditionalStats) - g->gcstats.currcycle.starttotalsizebytes = g->totalbytes; - - g->gcstats.currcycle.heapgoalsizebytes = heapgoal; - g->gcstats.currcycle.heaptriggersizebytes = heaptrigger; +#ifdef LUAI_GCMETRICS + finishGcCycleMetrics(g); +#endif } else { @@ -933,8 +941,10 @@ void luaC_fullgc(lua_State* L) { global_State* g = L->global; +#ifdef LUAI_GCMETRICS if (g->gcstate == GCSpause) - startGcCycleStats(g); + startGcCycleMetrics(g); +#endif if (g->gcstate <= GCSatomic) { @@ -954,11 +964,12 @@ void luaC_fullgc(lua_State* L) gcstep(L, SIZE_MAX); } - finishGcCycleStats(g); +#ifdef LUAI_GCMETRICS + finishGcCycleMetrics(g); + startGcCycleMetrics(g); +#endif /* run a full collection cycle */ - startGcCycleStats(g); - markroot(L); while (g->gcstate != GCSpause) { @@ -980,10 +991,11 @@ void luaC_fullgc(lua_State* L) if (g->GCthreshold < g->totalbytes) g->GCthreshold = g->totalbytes; - finishGcCycleStats(g); + g->gcstats.heapgoalsizebytes = heapgoalsizebytes; - g->gcstats.currcycle.heapgoalsizebytes = heapgoalsizebytes; - g->gcstats.currcycle.heaptriggersizebytes = g->GCthreshold; +#ifdef LUAI_GCMETRICS + finishGcCycleMetrics(g); +#endif } void luaC_barrierupval(lua_State* L, GCObject* v) @@ -1075,21 +1087,21 @@ int64_t luaC_allocationrate(lua_State* L) if (g->gcstate <= GCSatomic) { - double duration = lua_clock() - g->gcstats.lastcycle.endtimestamp; + double duration = lua_clock() - g->gcstats.endtimestamp; if (duration < durationthreshold) return -1; - return int64_t((g->totalbytes - g->gcstats.lastcycle.endtotalsizebytes) / duration); + return int64_t((g->totalbytes - g->gcstats.endtotalsizebytes) / duration); } // totalbytes is unstable during the sweep, use the rate measured at the end of mark phase - double duration = g->gcstats.currcycle.atomicstarttimestamp - g->gcstats.lastcycle.endtimestamp; + double duration = g->gcstats.atomicstarttimestamp - g->gcstats.endtimestamp; if (duration < durationthreshold) return -1; - return int64_t((g->gcstats.currcycle.atomicstarttotalsizebytes - g->gcstats.lastcycle.endtotalsizebytes) / duration); + return int64_t((g->gcstats.atomicstarttotalsizebytes - g->gcstats.endtotalsizebytes) / duration); } void luaC_wakethread(lua_State* L) diff --git a/VM/src/lgc.h b/VM/src/lgc.h index ebf999b5..dcd070b7 100644 --- a/VM/src/lgc.h +++ b/VM/src/lgc.h @@ -82,7 +82,7 @@ #define luaC_checkGC(L) \ { \ - condhardstacktests(luaD_reallocstack(L, L->stacksize - (FFlag::LuauReduceStackReallocs ? EXTRA_STACK : 1 + EXTRA_STACK))); \ + condhardstacktests(luaD_reallocstack(L, L->stacksize - EXTRA_STACK)); \ if (L->global->totalbytes >= L->global->GCthreshold) \ { \ condhardmemtests(luaC_validate(L), 1); \ diff --git a/VM/src/lstate.cpp b/VM/src/lstate.cpp index d4f3f0a1..fbc6fb1e 100644 --- a/VM/src/lstate.cpp +++ b/VM/src/lstate.cpp @@ -10,8 +10,6 @@ #include "ldo.h" #include "ldebug.h" -LUAU_FASTFLAGVARIABLE(LuauReduceStackReallocs, false) - /* ** Main thread combines a thread state and the global state */ @@ -35,7 +33,7 @@ static void stack_init(lua_State* L1, lua_State* L) for (int i = 0; i < BASIC_STACK_SIZE + EXTRA_STACK; i++) setnilvalue(stack + i); /* erase new stack */ L1->top = stack; - L1->stack_last = stack + (L1->stacksize - (FFlag::LuauReduceStackReallocs ? EXTRA_STACK : 1 + EXTRA_STACK)); + L1->stack_last = stack + (L1->stacksize - EXTRA_STACK); /* initialize first ci */ L1->ci->func = L1->top; setnilvalue(L1->top++); /* `function' entry for this `ci' */ @@ -141,30 +139,16 @@ void lua_resetthread(lua_State* L) ci->top = ci->base + LUA_MINSTACK; setnilvalue(ci->func); L->ci = ci; - if (FFlag::LuauReduceStackReallocs) - { - if (L->size_ci != BASIC_CI_SIZE) - luaD_reallocCI(L, BASIC_CI_SIZE); - } - else - { + if (L->size_ci != BASIC_CI_SIZE) luaD_reallocCI(L, BASIC_CI_SIZE); - } /* clear thread state */ L->status = LUA_OK; L->base = L->ci->base; L->top = L->ci->base; L->nCcalls = L->baseCcalls = 0; /* clear thread stack */ - if (FFlag::LuauReduceStackReallocs) - { - if (L->stacksize != BASIC_STACK_SIZE + EXTRA_STACK) - luaD_reallocstack(L, BASIC_STACK_SIZE); - } - else - { + if (L->stacksize != BASIC_STACK_SIZE + EXTRA_STACK) luaD_reallocstack(L, BASIC_STACK_SIZE); - } for (int i = 0; i < L->stacksize; i++) setnilvalue(L->stack + i); } @@ -234,6 +218,10 @@ lua_State* lua_newstate(lua_Alloc f, void* ud) g->cb = lua_Callbacks(); g->gcstats = GCStats(); +#ifdef LUAI_GCMETRICS + g->gcmetrics = GCMetrics(); +#endif + if (luaD_rawrunprotected(L, f_luaopen, NULL) != 0) { /* memory allocation error: free partial state */ diff --git a/VM/src/lstate.h b/VM/src/lstate.h index b2bedb48..e7c37373 100644 --- a/VM/src/lstate.h +++ b/VM/src/lstate.h @@ -75,10 +75,26 @@ typedef struct CallInfo #define f_isLua(ci) (!ci_func(ci)->isC) #define isLua(ci) (ttisfunction((ci)->func) && f_isLua(ci)) -struct GCCycleStats +struct GCStats +{ + // data for proportional-integral controller of heap trigger value + int32_t triggerterms[32] = {0}; + uint32_t triggertermpos = 0; + int32_t triggerintegral = 0; + + size_t atomicstarttotalsizebytes = 0; + size_t endtotalsizebytes = 0; + size_t heapgoalsizebytes = 0; + + double starttimestamp = 0; + double atomicstarttimestamp = 0; + double endtimestamp = 0; +}; + +#ifdef LUAI_GCMETRICS +struct GCCycleMetrics { size_t starttotalsizebytes = 0; - size_t heapgoalsizebytes = 0; size_t heaptriggersizebytes = 0; double pausetime = 0.0; // time from end of the last cycle to the start of a new one @@ -120,16 +136,7 @@ struct GCCycleStats size_t endtotalsizebytes = 0; }; -// data for proportional-integral controller of heap trigger value -struct GCHeapTriggerStats -{ - static const unsigned termcount = 32; - int32_t terms[termcount] = {0}; - uint32_t termpos = 0; - int32_t integral = 0; -}; - -struct GCStats +struct GCMetrics { double stepexplicittimeacc = 0.0; double stepassisttimeacc = 0.0; @@ -137,14 +144,10 @@ struct GCStats // when cycle is completed, last cycle values are updated uint64_t completedcycles = 0; - GCCycleStats lastcycle; - GCCycleStats currcycle; - - // only step count and their time is accumulated - GCCycleStats cyclestatsacc; - - GCHeapTriggerStats triggerstats; + GCCycleMetrics lastcycle; + GCCycleMetrics currcycle; }; +#endif /* ** `global state', shared by all threads of this state @@ -206,6 +209,9 @@ typedef struct global_State GCStats gcstats; +#ifdef LUAI_GCMETRICS + GCMetrics gcmetrics; +#endif } global_State; // clang-format on diff --git a/VM/src/ltable.cpp b/VM/src/ltable.cpp index 2deec2b9..431501f3 100644 --- a/VM/src/ltable.cpp +++ b/VM/src/ltable.cpp @@ -526,8 +526,8 @@ static TValue* newkey(lua_State* L, Table* t, const TValue* key) LuaNode* othern; LuaNode* n = getfreepos(t); /* get a free place */ if (n == NULL) - { /* cannot find a free place? */ - rehash(L, t, key); /* grow table */ + { /* cannot find a free place? */ + rehash(L, t, key); /* grow table */ if (!FFlag::LuauTableRehashRework) { diff --git a/tests/Autocomplete.test.cpp b/tests/Autocomplete.test.cpp index 17fd6b13..1db782cc 100644 --- a/tests/Autocomplete.test.cpp +++ b/tests/Autocomplete.test.cpp @@ -2726,11 +2726,6 @@ end TEST_CASE_FIXTURE(ACFixture, "autocomplete_on_string_singletons") { - ScopedFastFlag sffs[] = { - {"LuauParseSingletonTypes", true}, - {"LuauSingletonTypes", true}, - }; - check(R"( --!strict local foo: "hello" | "bye" = "hello" diff --git a/tests/Conformance.test.cpp b/tests/Conformance.test.cpp index 9e4cb4a5..83d4518d 100644 --- a/tests/Conformance.test.cpp +++ b/tests/Conformance.test.cpp @@ -496,8 +496,6 @@ TEST_CASE("DateTime") TEST_CASE("Debug") { - ScopedFastFlag luauTableFieldFunctionDebugname{"LuauTableFieldFunctionDebugname", true}; - runConformance("debug.lua"); } diff --git a/tests/Fixture.cpp b/tests/Fixture.cpp index dbdd06a4..a7e7ea39 100644 --- a/tests/Fixture.cpp +++ b/tests/Fixture.cpp @@ -132,21 +132,9 @@ AstStatBlock* Fixture::parse(const std::string& source, const ParseOptions& pars } CheckResult Fixture::check(Mode mode, std::string source) -{ - configResolver.defaultConfig.mode = mode; - fileResolver.source[mainModuleName] = std::move(source); - - CheckResult result = frontend.check(fromString(mainModuleName)); - - configResolver.defaultConfig.mode = Mode::Strict; - - return result; -} - -CheckResult Fixture::check(const std::string& source) { ModuleName mm = fromString(mainModuleName); - configResolver.defaultConfig.mode = Mode::Strict; + configResolver.defaultConfig.mode = mode; fileResolver.source[mm] = std::move(source); frontend.markDirty(mm); @@ -155,6 +143,11 @@ CheckResult Fixture::check(const std::string& source) return result; } +CheckResult Fixture::check(const std::string& source) +{ + return check(Mode::Strict, source); +} + LintResult Fixture::lint(const std::string& source, const std::optional& lintOptions) { ParseOptions parseOptions; diff --git a/tests/Linter.test.cpp b/tests/Linter.test.cpp index 91b23197..9ce9a4c2 100644 --- a/tests/Linter.test.cpp +++ b/tests/Linter.test.cpp @@ -597,6 +597,8 @@ return foo1 TEST_CASE_FIXTURE(Fixture, "UnknownType") { + ScopedFastFlag sff("LuauLintNoRobloxBits", true); + unfreeze(typeChecker.globalTypes); TableTypeVar::Props instanceProps{ {"ClassName", {typeChecker.anyType}}, @@ -606,81 +608,26 @@ TEST_CASE_FIXTURE(Fixture, "UnknownType") TypeId instanceType = typeChecker.globalTypes.addType(instanceTable); TypeFun instanceTypeFun{{}, instanceType}; - ClassTypeVar::Props enumItemProps{ - {"EnumType", {typeChecker.anyType}}, - }; - - ClassTypeVar enumItemClass{"EnumItem", enumItemProps, std::nullopt, std::nullopt, {}, {}}; - TypeId enumItemType = typeChecker.globalTypes.addType(enumItemClass); - TypeFun enumItemTypeFun{{}, enumItemType}; - - ClassTypeVar normalIdClass{"NormalId", {}, enumItemType, std::nullopt, {}, {}}; - TypeId normalIdType = typeChecker.globalTypes.addType(normalIdClass); - TypeFun normalIdTypeFun{{}, normalIdType}; - - // Normally this would be defined externally, so hack it in for testing - addGlobalBinding(typeChecker, "game", typeChecker.anyType, "@test"); - addGlobalBinding(typeChecker, "typeof", typeChecker.anyType, "@test"); typeChecker.globalScope->exportedTypeBindings["Part"] = instanceTypeFun; - typeChecker.globalScope->exportedTypeBindings["Workspace"] = instanceTypeFun; - typeChecker.globalScope->exportedTypeBindings["RunService"] = instanceTypeFun; - typeChecker.globalScope->exportedTypeBindings["Instance"] = instanceTypeFun; - typeChecker.globalScope->exportedTypeBindings["ColorSequence"] = TypeFun{{}, typeChecker.anyType}; - typeChecker.globalScope->exportedTypeBindings["EnumItem"] = enumItemTypeFun; - typeChecker.globalScope->importedTypeBindings["Enum"] = {{"NormalId", normalIdTypeFun}}; - freeze(typeChecker.globalTypes); LintResult result = lint(R"( -local _e01 = game:GetService("Foo") -local _e02 = game:GetService("NormalId") -local _e03 = game:FindService("table") -local _e04 = type(game) == "Part" -local _e05 = type(game) == "NormalId" -local _e06 = typeof(game) == "Bar" -local _e07 = typeof(game) == "Part" -local _e08 = typeof(game) == "vector" -local _e09 = typeof(game) == "NormalId" -local _e10 = game:IsA("ColorSequence") -local _e11 = game:IsA("Enum.NormalId") -local _e12 = game:FindFirstChildWhichIsA("function") +local game = ... +local _e01 = type(game) == "Part" +local _e02 = typeof(game) == "Bar" +local _e03 = typeof(game) == "vector" -local _o01 = game:GetService("Workspace") -local _o02 = game:FindService("RunService") -local _o03 = type(game) == "number" -local _o04 = type(game) == "vector" -local _o05 = typeof(game) == "string" -local _o06 = typeof(game) == "Instance" -local _o07 = typeof(game) == "EnumItem" -local _o08 = game:IsA("Part") -local _o09 = game:IsA("NormalId") -local _o10 = game:FindFirstChildWhichIsA("Part") +local _o01 = type(game) == "number" +local _o02 = type(game) == "vector" +local _o03 = typeof(game) == "Part" )"); - REQUIRE_EQ(result.warnings.size(), 12); - CHECK_EQ(result.warnings[0].location.begin.line, 1); - CHECK_EQ(result.warnings[0].text, "Unknown type 'Foo'"); - CHECK_EQ(result.warnings[1].location.begin.line, 2); - CHECK_EQ(result.warnings[1].text, "Unknown type 'NormalId' (expected class type)"); - CHECK_EQ(result.warnings[2].location.begin.line, 3); - CHECK_EQ(result.warnings[2].text, "Unknown type 'table' (expected class type)"); - CHECK_EQ(result.warnings[3].location.begin.line, 4); - CHECK_EQ(result.warnings[3].text, "Unknown type 'Part' (expected primitive type)"); - CHECK_EQ(result.warnings[4].location.begin.line, 5); - CHECK_EQ(result.warnings[4].text, "Unknown type 'NormalId' (expected primitive type)"); - CHECK_EQ(result.warnings[5].location.begin.line, 6); - CHECK_EQ(result.warnings[5].text, "Unknown type 'Bar'"); - CHECK_EQ(result.warnings[6].location.begin.line, 7); - CHECK_EQ(result.warnings[6].text, "Unknown type 'Part' (expected primitive or userdata type)"); - CHECK_EQ(result.warnings[7].location.begin.line, 8); - CHECK_EQ(result.warnings[7].text, "Unknown type 'vector' (expected primitive or userdata type)"); - CHECK_EQ(result.warnings[8].location.begin.line, 9); - CHECK_EQ(result.warnings[8].text, "Unknown type 'NormalId' (expected primitive or userdata type)"); - CHECK_EQ(result.warnings[9].location.begin.line, 10); - CHECK_EQ(result.warnings[9].text, "Unknown type 'ColorSequence' (expected class or enum type)"); - CHECK_EQ(result.warnings[10].location.begin.line, 11); - CHECK_EQ(result.warnings[10].text, "Unknown type 'Enum.NormalId'"); - CHECK_EQ(result.warnings[11].location.begin.line, 12); - CHECK_EQ(result.warnings[11].text, "Unknown type 'function' (expected class type)"); + REQUIRE_EQ(result.warnings.size(), 3); + CHECK_EQ(result.warnings[0].location.begin.line, 2); + CHECK_EQ(result.warnings[0].text, "Unknown type 'Part' (expected primitive type)"); + CHECK_EQ(result.warnings[1].location.begin.line, 3); + CHECK_EQ(result.warnings[1].text, "Unknown type 'Bar'"); + CHECK_EQ(result.warnings[2].location.begin.line, 4); + CHECK_EQ(result.warnings[2].text, "Unknown type 'vector' (expected primitive or userdata type)"); } TEST_CASE_FIXTURE(Fixture, "ForRangeTable") diff --git a/tests/ToString.test.cpp b/tests/ToString.test.cpp index 6713a589..3051e209 100644 --- a/tests/ToString.test.cpp +++ b/tests/ToString.test.cpp @@ -470,6 +470,7 @@ TEST_CASE_FIXTURE(Fixture, "self_recursive_instantiated_param") TEST_CASE_FIXTURE(Fixture, "toStringNamedFunction_id") { + ScopedFastFlag flag{"LuauDocFuncParameters", true}; CheckResult result = check(R"( local function id(x) return x end )"); @@ -482,6 +483,7 @@ TEST_CASE_FIXTURE(Fixture, "toStringNamedFunction_id") TEST_CASE_FIXTURE(Fixture, "toStringNamedFunction_map") { + ScopedFastFlag flag{"LuauDocFuncParameters", true}; CheckResult result = check(R"( local function map(arr, fn) local t = {} @@ -500,6 +502,7 @@ TEST_CASE_FIXTURE(Fixture, "toStringNamedFunction_map") TEST_CASE_FIXTURE(Fixture, "toStringNamedFunction_generic_pack") { + ScopedFastFlag flag{"LuauDocFuncParameters", true}; CheckResult result = check(R"( local function f(a: number, b: string) end local function test(...: T...): U... @@ -516,6 +519,7 @@ TEST_CASE_FIXTURE(Fixture, "toStringNamedFunction_generic_pack") TEST_CASE("toStringNamedFunction_unit_f") { + ScopedFastFlag flag{"LuauDocFuncParameters", true}; TypePackVar empty{TypePack{}}; FunctionTypeVar ftv{&empty, &empty, {}, false}; CHECK_EQ("f(): ()", toStringNamedFunction("f", ftv)); @@ -523,6 +527,7 @@ TEST_CASE("toStringNamedFunction_unit_f") TEST_CASE_FIXTURE(Fixture, "toStringNamedFunction_variadics") { + ScopedFastFlag flag{"LuauDocFuncParameters", true}; CheckResult result = check(R"( local function f(x: a, ...): (a, a, b...) return x, x, ... @@ -537,6 +542,7 @@ TEST_CASE_FIXTURE(Fixture, "toStringNamedFunction_variadics") TEST_CASE_FIXTURE(Fixture, "toStringNamedFunction_variadics2") { + ScopedFastFlag flag{"LuauDocFuncParameters", true}; CheckResult result = check(R"( local function f(): ...number return 1, 2, 3 @@ -551,6 +557,7 @@ TEST_CASE_FIXTURE(Fixture, "toStringNamedFunction_variadics2") TEST_CASE_FIXTURE(Fixture, "toStringNamedFunction_variadics3") { + ScopedFastFlag flag{"LuauDocFuncParameters", true}; CheckResult result = check(R"( local function f(): (string, ...number) return 'a', 1, 2, 3 @@ -565,6 +572,7 @@ TEST_CASE_FIXTURE(Fixture, "toStringNamedFunction_variadics3") TEST_CASE_FIXTURE(Fixture, "toStringNamedFunction_type_annotation_has_partial_argnames") { + ScopedFastFlag flag{"LuauDocFuncParameters", true}; CheckResult result = check(R"( local f: (number, y: number) -> number )"); @@ -577,6 +585,7 @@ TEST_CASE_FIXTURE(Fixture, "toStringNamedFunction_type_annotation_has_partial_ar TEST_CASE_FIXTURE(Fixture, "toStringNamedFunction_hide_type_params") { + ScopedFastFlag flag{"LuauDocFuncParameters", true}; CheckResult result = check(R"( local function f(x: T, g: (T) -> U)): () end @@ -590,4 +599,20 @@ TEST_CASE_FIXTURE(Fixture, "toStringNamedFunction_hide_type_params") CHECK_EQ("f(x: T, g: (T) -> U): ()", toStringNamedFunction("f", *ftv, opts)); } +TEST_CASE_FIXTURE(Fixture, "toStringNamedFunction_overrides_param_names") +{ + ScopedFastFlag flag{"LuauDocFuncParameters", true}; + + CheckResult result = check(R"( + local function test(a, b : string, ... : number) return a end + )"); + + TypeId ty = requireType("test"); + const FunctionTypeVar* ftv = get(follow(ty)); + + ToStringOptions opts; + opts.namedFunctionOverrideArgNames = {"first", "second", "third"}; + CHECK_EQ("test(first: a, second: string, ...: number): a", toStringNamedFunction("test", *ftv, opts)); +} + TEST_SUITE_END(); diff --git a/tests/Transpiler.test.cpp b/tests/Transpiler.test.cpp index 5f0295b0..5ac45ff2 100644 --- a/tests/Transpiler.test.cpp +++ b/tests/Transpiler.test.cpp @@ -651,8 +651,6 @@ local a: Packed TEST_CASE_FIXTURE(Fixture, "transpile_singleton_types") { - ScopedFastFlag luauParseSingletonTypes{"LuauParseSingletonTypes", true}; - std::string code = R"( type t1 = 'hello' type t2 = true diff --git a/tests/TypeInfer.builtins.test.cpp b/tests/TypeInfer.builtins.test.cpp index ec20a2c7..c6fbebed 100644 --- a/tests/TypeInfer.builtins.test.cpp +++ b/tests/TypeInfer.builtins.test.cpp @@ -887,8 +887,6 @@ TEST_CASE_FIXTURE(Fixture, "assert_removes_falsy_types") TEST_CASE_FIXTURE(Fixture, "assert_removes_falsy_types2") { ScopedFastFlag sff[]{ - {"LuauParseSingletonTypes", true}, - {"LuauSingletonTypes", true}, {"LuauAssertStripsFalsyTypes", true}, {"LuauDiscriminableUnions2", true}, {"LuauWidenIfSupertypeIsFree2", true}, diff --git a/tests/TypeInfer.functions.test.cpp b/tests/TypeInfer.functions.test.cpp index 4288098a..da4ea074 100644 --- a/tests/TypeInfer.functions.test.cpp +++ b/tests/TypeInfer.functions.test.cpp @@ -1335,4 +1335,80 @@ caused by: toString(result.errors[0])); } +TEST_CASE_FIXTURE(Fixture, "too_few_arguments_variadic") +{ + ScopedFastFlag sff{"LuauArgCountMismatchSaysAtLeastWhenVariadic", true}; + CheckResult result = check(R"( + function test(a: number, b: string, ...) + end + + test(1) + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + auto err = result.errors[0]; + auto acm = get(err); + REQUIRE(acm); + + CHECK_EQ(2, acm->expected); + CHECK_EQ(1, acm->actual); + CHECK_EQ(CountMismatch::Context::Arg, acm->context); + CHECK(acm->isVariadic); +} + +TEST_CASE_FIXTURE(Fixture, "too_few_arguments_variadic_generic") +{ + ScopedFastFlag sff1{"LuauArgCountMismatchSaysAtLeastWhenVariadic", true}; + ScopedFastFlag sff2{"LuauFixArgumentCountMismatchAmountWithGenericTypes", true}; + CheckResult result = check(R"( +function test(a: number, b: string, ...) + return 1 +end + +function wrapper(f: (A...) -> number, ...: A...) +end + +wrapper(test) + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + auto err = result.errors[0]; + auto acm = get(err); + REQUIRE(acm); + + CHECK_EQ(3, acm->expected); + CHECK_EQ(1, acm->actual); + CHECK_EQ(CountMismatch::Context::Arg, acm->context); + CHECK(acm->isVariadic); +} + +TEST_CASE_FIXTURE(Fixture, "too_few_arguments_variadic_generic2") +{ + ScopedFastFlag sff1{"LuauArgCountMismatchSaysAtLeastWhenVariadic", true}; + ScopedFastFlag sff2{"LuauFixArgumentCountMismatchAmountWithGenericTypes", true}; + CheckResult result = check(R"( +function test(a: number, b: string, ...) + return 1 +end + +function wrapper(f: (A...) -> number, ...: A...) +end + +pcall(wrapper, test) + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + auto err = result.errors[0]; + auto acm = get(err); + REQUIRE(acm); + + CHECK_EQ(4, acm->expected); + CHECK_EQ(2, acm->actual); + CHECK_EQ(CountMismatch::Context::Arg, acm->context); + CHECK(acm->isVariadic); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.modules.test.cpp b/tests/TypeInfer.modules.test.cpp index 63643610..e5eeae31 100644 --- a/tests/TypeInfer.modules.test.cpp +++ b/tests/TypeInfer.modules.test.cpp @@ -13,6 +13,8 @@ using namespace Luau; +LUAU_FASTFLAG(LuauTableSubtypingVariance2) + TEST_SUITE_BEGIN("TypeInferModules"); TEST_CASE_FIXTURE(Fixture, "require") @@ -268,8 +270,6 @@ function x:Destroy(): () end TEST_CASE_FIXTURE(Fixture, "do_not_modify_imported_types_2") { - ScopedFastFlag immutableTypes{"LuauImmutableTypes", true}; - fileResolver.source["game/A"] = R"( export type Type = { x: { a: number } } return {} @@ -288,8 +288,6 @@ type Rename = typeof(x.x) TEST_CASE_FIXTURE(Fixture, "do_not_modify_imported_types_3") { - ScopedFastFlag immutableTypes{"LuauImmutableTypes", true}; - fileResolver.source["game/A"] = R"( local y = setmetatable({}, {}) export type Type = { x: typeof(y) } @@ -307,4 +305,83 @@ type Rename = typeof(x.x) LUAU_REQUIRE_NO_ERRORS(result); } +TEST_CASE_FIXTURE(Fixture, "module_type_conflict") +{ + ScopedFastFlag luauTypeMismatchModuleName{"LuauTypeMismatchModuleName", true}; + + fileResolver.source["game/A"] = R"( +export type T = { x: number } +return {} + )"; + + fileResolver.source["game/B"] = R"( +export type T = { x: string } +return {} + )"; + + fileResolver.source["game/C"] = R"( +local A = require(game.A) +local B = require(game.B) +local a: A.T = { x = 2 } +local b: B.T = a + )"; + + CheckResult result = frontend.check("game/C"); + LUAU_REQUIRE_ERROR_COUNT(1, result); + + if (FFlag::LuauTableSubtypingVariance2) + { + CHECK_EQ(toString(result.errors[0]), R"(Type 'T' from 'game/A' could not be converted into 'T' from 'game/B' +caused by: + Property 'x' is not compatible. Type 'number' could not be converted into 'string')"); + } + else + { + CHECK_EQ(toString(result.errors[0]), "Type 'T' from 'game/A' could not be converted into 'T' from 'game/B'"); + } +} + +TEST_CASE_FIXTURE(Fixture, "module_type_conflict_instantiated") +{ + ScopedFastFlag luauTypeMismatchModuleName{"LuauTypeMismatchModuleName", true}; + + fileResolver.source["game/A"] = R"( +export type Wrap = { x: T } +return {} + )"; + + fileResolver.source["game/B"] = R"( +local A = require(game.A) +export type T = A.Wrap +return {} + )"; + + fileResolver.source["game/C"] = R"( +local A = require(game.A) +export type T = A.Wrap +return {} + )"; + + fileResolver.source["game/D"] = R"( +local A = require(game.B) +local B = require(game.C) +local a: A.T = { x = 2 } +local b: B.T = a + )"; + + CheckResult result = frontend.check("game/D"); + LUAU_REQUIRE_ERROR_COUNT(1, result); + + if (FFlag::LuauTableSubtypingVariance2) + { + CHECK_EQ(toString(result.errors[0]), R"(Type 'T' from 'game/B' could not be converted into 'T' from 'game/C' +caused by: + Property 'x' is not compatible. Type 'number' could not be converted into 'string')"); + } + else + { + CHECK_EQ(toString(result.errors[0]), "Type 'T' from 'game/B' could not be converted into 'T' from 'game/C'"); + } +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.operators.test.cpp b/tests/TypeInfer.operators.test.cpp index baa25978..6a8a9d93 100644 --- a/tests/TypeInfer.operators.test.cpp +++ b/tests/TypeInfer.operators.test.cpp @@ -756,4 +756,30 @@ TEST_CASE_FIXTURE(Fixture, "refine_and_or") CHECK_EQ("number", toString(requireType("u"))); } +TEST_CASE_FIXTURE(Fixture, "infer_any_in_all_modes_when_lhs_is_unknown") +{ + ScopedFastFlag sff{"LuauDecoupleOperatorInferenceFromUnifiedTypeInference", true}; + + CheckResult result = check(Mode::Strict, R"( + local function f(x, y) + return x + y + end + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ(toString(result.errors[0]), "Unknown type used in + operation; consider adding a type annotation to 'x'"); + + result = check(Mode::Nonstrict, R"( + local function f(x, y) + return x + y + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + // When type inference is unified, we could add an assertion that + // the strict and nonstrict types are equivalent. This isn't actually + // the case right now, though. +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.refinements.test.cpp b/tests/TypeInfer.refinements.test.cpp index 9b347921..cddeab6e 100644 --- a/tests/TypeInfer.refinements.test.cpp +++ b/tests/TypeInfer.refinements.test.cpp @@ -435,7 +435,6 @@ TEST_CASE_FIXTURE(Fixture, "term_is_equal_to_an_lvalue") { ScopedFastFlag sff[] = { {"LuauDiscriminableUnions2", true}, - {"LuauSingletonTypes", true}, }; CheckResult result = check(R"( @@ -1002,8 +1001,6 @@ TEST_CASE_FIXTURE(Fixture, "discriminate_from_truthiness_of_x") { ScopedFastFlag sff[] = { {"LuauDiscriminableUnions2", true}, - {"LuauParseSingletonTypes", true}, - {"LuauSingletonTypes", true}, }; CheckResult result = check(R"( @@ -1028,8 +1025,6 @@ TEST_CASE_FIXTURE(Fixture, "discriminate_tag") { ScopedFastFlag sff[] = { {"LuauDiscriminableUnions2", true}, - {"LuauParseSingletonTypes", true}, - {"LuauSingletonTypes", true}, }; CheckResult result = check(R"( @@ -1066,8 +1061,6 @@ TEST_CASE_FIXTURE(Fixture, "and_or_peephole_refinement") TEST_CASE_FIXTURE(Fixture, "narrow_boolean_to_true_or_false") { ScopedFastFlag sff[]{ - {"LuauParseSingletonTypes", true}, - {"LuauSingletonTypes", true}, {"LuauDiscriminableUnions2", true}, {"LuauAssertStripsFalsyTypes", true}, }; @@ -1091,8 +1084,6 @@ TEST_CASE_FIXTURE(Fixture, "narrow_boolean_to_true_or_false") TEST_CASE_FIXTURE(Fixture, "discriminate_on_properties_of_disjoint_tables_where_that_property_is_true_or_false") { ScopedFastFlag sff[]{ - {"LuauParseSingletonTypes", true}, - {"LuauSingletonTypes", true}, {"LuauDiscriminableUnions2", true}, {"LuauAssertStripsFalsyTypes", true}, }; @@ -1134,8 +1125,6 @@ TEST_CASE_FIXTURE(RefinementClassFixture, "discriminate_from_isa_of_x") { ScopedFastFlag sff[] = { {"LuauDiscriminableUnions2", true}, - {"LuauParseSingletonTypes", true}, - {"LuauSingletonTypes", true}, }; CheckResult result = check(R"( diff --git a/tests/TypeInfer.singletons.test.cpp b/tests/TypeInfer.singletons.test.cpp index 7f8d8fec..d39341ea 100644 --- a/tests/TypeInfer.singletons.test.cpp +++ b/tests/TypeInfer.singletons.test.cpp @@ -13,11 +13,6 @@ TEST_SUITE_BEGIN("TypeSingletons"); TEST_CASE_FIXTURE(Fixture, "bool_singletons") { - ScopedFastFlag sffs[] = { - {"LuauSingletonTypes", true}, - {"LuauParseSingletonTypes", true}, - }; - CheckResult result = check(R"( local a: true = true local b: false = false @@ -28,11 +23,6 @@ TEST_CASE_FIXTURE(Fixture, "bool_singletons") TEST_CASE_FIXTURE(Fixture, "string_singletons") { - ScopedFastFlag sffs[] = { - {"LuauSingletonTypes", true}, - {"LuauParseSingletonTypes", true}, - }; - CheckResult result = check(R"( local a: "foo" = "foo" local b: "bar" = "bar" @@ -43,11 +33,6 @@ TEST_CASE_FIXTURE(Fixture, "string_singletons") TEST_CASE_FIXTURE(Fixture, "bool_singletons_mismatch") { - ScopedFastFlag sffs[] = { - {"LuauSingletonTypes", true}, - {"LuauParseSingletonTypes", true}, - }; - CheckResult result = check(R"( local a: true = false )"); @@ -58,11 +43,6 @@ TEST_CASE_FIXTURE(Fixture, "bool_singletons_mismatch") TEST_CASE_FIXTURE(Fixture, "string_singletons_mismatch") { - ScopedFastFlag sffs[] = { - {"LuauSingletonTypes", true}, - {"LuauParseSingletonTypes", true}, - }; - CheckResult result = check(R"( local a: "foo" = "bar" )"); @@ -73,11 +53,6 @@ TEST_CASE_FIXTURE(Fixture, "string_singletons_mismatch") TEST_CASE_FIXTURE(Fixture, "string_singletons_escape_chars") { - ScopedFastFlag sffs[] = { - {"LuauSingletonTypes", true}, - {"LuauParseSingletonTypes", true}, - }; - CheckResult result = check(R"( local a: "\n" = "\000\r" )"); @@ -88,11 +63,6 @@ TEST_CASE_FIXTURE(Fixture, "string_singletons_escape_chars") TEST_CASE_FIXTURE(Fixture, "bool_singleton_subtype") { - ScopedFastFlag sffs[] = { - {"LuauSingletonTypes", true}, - {"LuauParseSingletonTypes", true}, - }; - CheckResult result = check(R"( local a: true = true local b: boolean = a @@ -103,11 +73,6 @@ TEST_CASE_FIXTURE(Fixture, "bool_singleton_subtype") TEST_CASE_FIXTURE(Fixture, "string_singleton_subtype") { - ScopedFastFlag sffs[] = { - {"LuauSingletonTypes", true}, - {"LuauParseSingletonTypes", true}, - }; - CheckResult result = check(R"( local a: "foo" = "foo" local b: string = a @@ -118,11 +83,6 @@ TEST_CASE_FIXTURE(Fixture, "string_singleton_subtype") TEST_CASE_FIXTURE(Fixture, "function_call_with_singletons") { - ScopedFastFlag sffs[] = { - {"LuauSingletonTypes", true}, - {"LuauParseSingletonTypes", true}, - }; - CheckResult result = check(R"( function f(a: true, b: "foo") end f(true, "foo") @@ -133,11 +93,6 @@ TEST_CASE_FIXTURE(Fixture, "function_call_with_singletons") TEST_CASE_FIXTURE(Fixture, "function_call_with_singletons_mismatch") { - ScopedFastFlag sffs[] = { - {"LuauSingletonTypes", true}, - {"LuauParseSingletonTypes", true}, - }; - CheckResult result = check(R"( function f(a: true, b: "foo") end f(true, "bar") @@ -149,11 +104,6 @@ TEST_CASE_FIXTURE(Fixture, "function_call_with_singletons_mismatch") TEST_CASE_FIXTURE(Fixture, "overloaded_function_call_with_singletons") { - ScopedFastFlag sffs[] = { - {"LuauSingletonTypes", true}, - {"LuauParseSingletonTypes", true}, - }; - CheckResult result = check(R"( function f(a, b) end local g : ((true, string) -> ()) & ((false, number) -> ()) = (f::any) @@ -166,11 +116,6 @@ TEST_CASE_FIXTURE(Fixture, "overloaded_function_call_with_singletons") TEST_CASE_FIXTURE(Fixture, "overloaded_function_call_with_singletons_mismatch") { - ScopedFastFlag sffs[] = { - {"LuauSingletonTypes", true}, - {"LuauParseSingletonTypes", true}, - }; - CheckResult result = check(R"( function f(a, b) end local g : ((true, string) -> ()) & ((false, number) -> ()) = (f::any) @@ -184,11 +129,6 @@ TEST_CASE_FIXTURE(Fixture, "overloaded_function_call_with_singletons_mismatch") TEST_CASE_FIXTURE(Fixture, "enums_using_singletons") { - ScopedFastFlag sffs[] = { - {"LuauSingletonTypes", true}, - {"LuauParseSingletonTypes", true}, - }; - CheckResult result = check(R"( type MyEnum = "foo" | "bar" | "baz" local a : MyEnum = "foo" @@ -201,11 +141,6 @@ TEST_CASE_FIXTURE(Fixture, "enums_using_singletons") TEST_CASE_FIXTURE(Fixture, "enums_using_singletons_mismatch") { - ScopedFastFlag sffs[] = { - {"LuauSingletonTypes", true}, - {"LuauParseSingletonTypes", true}, - }; - CheckResult result = check(R"( type MyEnum = "foo" | "bar" | "baz" local a : MyEnum = "bang" @@ -218,11 +153,6 @@ TEST_CASE_FIXTURE(Fixture, "enums_using_singletons_mismatch") TEST_CASE_FIXTURE(Fixture, "enums_using_singletons_subtyping") { - ScopedFastFlag sffs[] = { - {"LuauSingletonTypes", true}, - {"LuauParseSingletonTypes", true}, - }; - CheckResult result = check(R"( type MyEnum1 = "foo" | "bar" type MyEnum2 = MyEnum1 | "baz" @@ -237,8 +167,6 @@ TEST_CASE_FIXTURE(Fixture, "enums_using_singletons_subtyping") TEST_CASE_FIXTURE(Fixture, "tagged_unions_using_singletons") { ScopedFastFlag sffs[] = { - {"LuauSingletonTypes", true}, - {"LuauParseSingletonTypes", true}, {"LuauExpectedTypesOfProperties", true}, }; @@ -257,11 +185,6 @@ TEST_CASE_FIXTURE(Fixture, "tagged_unions_using_singletons") TEST_CASE_FIXTURE(Fixture, "tagged_unions_using_singletons_mismatch") { - ScopedFastFlag sffs[] = { - {"LuauSingletonTypes", true}, - {"LuauParseSingletonTypes", true}, - }; - CheckResult result = check(R"( type Dog = { tag: "Dog", howls: boolean } type Cat = { tag: "Cat", meows: boolean } @@ -274,11 +197,6 @@ TEST_CASE_FIXTURE(Fixture, "tagged_unions_using_singletons_mismatch") TEST_CASE_FIXTURE(Fixture, "tagged_unions_immutable_tag") { - ScopedFastFlag sffs[] = { - {"LuauSingletonTypes", true}, - {"LuauParseSingletonTypes", true}, - }; - CheckResult result = check(R"( type Dog = { tag: "Dog", howls: boolean } type Cat = { tag: "Cat", meows: boolean } @@ -292,10 +210,6 @@ TEST_CASE_FIXTURE(Fixture, "tagged_unions_immutable_tag") TEST_CASE_FIXTURE(Fixture, "table_properties_singleton_strings") { - ScopedFastFlag sffs[] = { - {"LuauParseSingletonTypes", true}, - }; - CheckResult result = check(R"( --!strict type T = { @@ -320,10 +234,6 @@ TEST_CASE_FIXTURE(Fixture, "table_properties_singleton_strings") } TEST_CASE_FIXTURE(Fixture, "table_properties_singleton_strings_mismatch") { - ScopedFastFlag sffs[] = { - {"LuauParseSingletonTypes", true}, - }; - CheckResult result = check(R"( --!strict type T = { @@ -341,10 +251,6 @@ TEST_CASE_FIXTURE(Fixture, "table_properties_singleton_strings_mismatch") TEST_CASE_FIXTURE(Fixture, "table_properties_alias_or_parens_is_indexer") { - ScopedFastFlag sffs[] = { - {"LuauParseSingletonTypes", true}, - }; - CheckResult result = check(R"( --!strict type S = "bar" @@ -367,8 +273,7 @@ TEST_CASE_FIXTURE(Fixture, "table_properties_alias_or_parens_is_indexer") TEST_CASE_FIXTURE(Fixture, "table_properties_type_error_escapes") { - ScopedFastFlag sffs[] = { - {"LuauParseSingletonTypes", true}, + ScopedFastFlag sffs[]{ {"LuauUnsealedTableLiteral", true}, }; @@ -386,8 +291,6 @@ TEST_CASE_FIXTURE(Fixture, "table_properties_type_error_escapes") TEST_CASE_FIXTURE(Fixture, "error_detailed_tagged_union_mismatch_string") { ScopedFastFlag sffs[] = { - {"LuauSingletonTypes", true}, - {"LuauParseSingletonTypes", true}, {"LuauExpectedTypesOfProperties", true}, }; @@ -409,8 +312,6 @@ caused by: TEST_CASE_FIXTURE(Fixture, "error_detailed_tagged_union_mismatch_bool") { ScopedFastFlag sffs[] = { - {"LuauSingletonTypes", true}, - {"LuauParseSingletonTypes", true}, {"LuauExpectedTypesOfProperties", true}, }; @@ -432,8 +333,6 @@ caused by: TEST_CASE_FIXTURE(Fixture, "if_then_else_expression_singleton_options") { ScopedFastFlag sffs[] = { - {"LuauSingletonTypes", true}, - {"LuauParseSingletonTypes", true}, {"LuauExpectedTypesOfProperties", true}, }; @@ -451,7 +350,6 @@ local a: Animal = if true then { tag = 'cat', catfood = 'something' } else { tag TEST_CASE_FIXTURE(Fixture, "widen_the_supertype_if_it_is_free_and_subtype_has_singleton") { ScopedFastFlag sff[]{ - {"LuauSingletonTypes", true}, {"LuauEqConstraint", true}, {"LuauDiscriminableUnions2", true}, {"LuauWidenIfSupertypeIsFree2", true}, @@ -477,8 +375,6 @@ TEST_CASE_FIXTURE(Fixture, "widen_the_supertype_if_it_is_free_and_subtype_has_si TEST_CASE_FIXTURE(Fixture, "return_type_of_f_is_not_widened") { ScopedFastFlag sff[]{ - {"LuauParseSingletonTypes", true}, - {"LuauSingletonTypes", true}, {"LuauDiscriminableUnions2", true}, {"LuauEqConstraint", true}, {"LuauWidenIfSupertypeIsFree2", true}, @@ -504,8 +400,6 @@ TEST_CASE_FIXTURE(Fixture, "return_type_of_f_is_not_widened") TEST_CASE_FIXTURE(Fixture, "widening_happens_almost_everywhere") { ScopedFastFlag sff[]{ - {"LuauParseSingletonTypes", true}, - {"LuauSingletonTypes", true}, {"LuauWidenIfSupertypeIsFree2", true}, }; @@ -521,8 +415,6 @@ TEST_CASE_FIXTURE(Fixture, "widening_happens_almost_everywhere") TEST_CASE_FIXTURE(Fixture, "widening_happens_almost_everywhere_except_for_tables") { ScopedFastFlag sff[]{ - {"LuauParseSingletonTypes", true}, - {"LuauSingletonTypes", true}, {"LuauDiscriminableUnions2", true}, {"LuauWidenIfSupertypeIsFree2", true}, }; @@ -551,8 +443,6 @@ TEST_CASE_FIXTURE(Fixture, "widening_happens_almost_everywhere_except_for_tables TEST_CASE_FIXTURE(Fixture, "table_insert_with_a_singleton_argument") { ScopedFastFlag sff[]{ - {"LuauParseSingletonTypes", true}, - {"LuauSingletonTypes", true}, {"LuauWidenIfSupertypeIsFree2", true}, }; @@ -577,8 +467,6 @@ TEST_CASE_FIXTURE(Fixture, "table_insert_with_a_singleton_argument") TEST_CASE_FIXTURE(Fixture, "functions_are_not_to_be_widened") { ScopedFastFlag sff[]{ - {"LuauParseSingletonTypes", true}, - {"LuauSingletonTypes", true}, {"LuauWidenIfSupertypeIsFree2", true}, }; @@ -595,7 +483,6 @@ TEST_CASE_FIXTURE(Fixture, "indexing_on_string_singletons") { ScopedFastFlag sff[]{ {"LuauDiscriminableUnions2", true}, - {"LuauSingletonTypes", true}, }; CheckResult result = check(R"( @@ -614,7 +501,6 @@ TEST_CASE_FIXTURE(Fixture, "indexing_on_union_of_string_singletons") { ScopedFastFlag sff[]{ {"LuauDiscriminableUnions2", true}, - {"LuauSingletonTypes", true}, }; CheckResult result = check(R"( @@ -633,7 +519,6 @@ TEST_CASE_FIXTURE(Fixture, "taking_the_length_of_string_singleton") { ScopedFastFlag sff[]{ {"LuauDiscriminableUnions2", true}, - {"LuauSingletonTypes", true}, }; CheckResult result = check(R"( @@ -652,7 +537,6 @@ TEST_CASE_FIXTURE(Fixture, "taking_the_length_of_union_of_string_singleton") { ScopedFastFlag sff[]{ {"LuauDiscriminableUnions2", true}, - {"LuauSingletonTypes", true}, }; CheckResult result = check(R"( diff --git a/tests/TypeInfer.tables.test.cpp b/tests/TypeInfer.tables.test.cpp index 91140aaa..0cc12d19 100644 --- a/tests/TypeInfer.tables.test.cpp +++ b/tests/TypeInfer.tables.test.cpp @@ -2078,6 +2078,44 @@ caused by: Property '__call' is not compatible. Type '(a, b) -> ()' could not be converted into '(a) -> ()'; different number of generic type parameters)"); } +TEST_CASE_FIXTURE(Fixture, "error_detailed_indexer_key") +{ + ScopedFastFlag luauTableSubtypingVariance2{"LuauTableSubtypingVariance2", true}; // Only for new path + ScopedFastFlag luauExtendedIndexerError{"LuauExtendedIndexerError", true}; + + CheckResult result = check(R"( + type A = { [number]: string } + type B = { [string]: string } + + local a: A = { 'a', 'b' } + local b: B = a + )"); + + LUAU_REQUIRE_ERRORS(result); + CHECK_EQ(toString(result.errors[0]), R"(Type 'A' could not be converted into 'B' +caused by: + Property '[indexer key]' is not compatible. Type 'number' could not be converted into 'string')"); +} + +TEST_CASE_FIXTURE(Fixture, "error_detailed_indexer_value") +{ + ScopedFastFlag luauTableSubtypingVariance2{"LuauTableSubtypingVariance2", true}; // Only for new path + ScopedFastFlag luauExtendedIndexerError{"LuauExtendedIndexerError", true}; + + CheckResult result = check(R"( + type A = { [number]: number } + type B = { [number]: string } + + local a: A = { 1, 2, 3 } + local b: B = a + )"); + + LUAU_REQUIRE_ERRORS(result); + CHECK_EQ(toString(result.errors[0]), R"(Type 'A' could not be converted into 'B' +caused by: + Property '[indexer value]' is not compatible. Type 'number' could not be converted into 'string')"); +} + TEST_CASE_FIXTURE(Fixture, "explicitly_typed_table") { ScopedFastFlag sffs[]{ diff --git a/tests/TypeInfer.test.cpp b/tests/TypeInfer.test.cpp index 571d0f8d..660ddcfc 100644 --- a/tests/TypeInfer.test.cpp +++ b/tests/TypeInfer.test.cpp @@ -334,7 +334,7 @@ TEST_CASE_FIXTURE(Fixture, "check_expr_recursion_limit") #if defined(LUAU_ENABLE_ASAN) int limit = 250; #elif defined(_DEBUG) || defined(_NOOPT) - int limit = 350; + int limit = 300; #else int limit = 600; #endif diff --git a/tests/TypeInfer.unionTypes.test.cpp b/tests/TypeInfer.unionTypes.test.cpp index ad1e31e5..68b7c4fb 100644 --- a/tests/TypeInfer.unionTypes.test.cpp +++ b/tests/TypeInfer.unionTypes.test.cpp @@ -296,6 +296,7 @@ return f() REQUIRE(acm); CHECK_EQ(1, acm->expected); CHECK_EQ(0, acm->actual); + CHECK_FALSE(acm->isVariadic); } TEST_CASE_FIXTURE(Fixture, "optional_field_access_error") From 2c339d52c08e5e12b18333dce4fed13b14b6060a Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Thu, 24 Mar 2022 15:04:14 -0700 Subject: [PATCH 207/399] Sync to upstream/release/520 (#427) --- Analysis/include/Luau/Error.h | 1 + Analysis/include/Luau/ToString.h | 3 +- Analysis/include/Luau/TypePack.h | 6 +- Analysis/include/Luau/TypeVar.h | 3 + Analysis/include/Luau/Unifier.h | 4 +- Analysis/include/Luau/UnifierSharedState.h | 2 + Analysis/src/Error.cpp | 30 ++- Analysis/src/Linter.cpp | 43 +++- Analysis/src/Module.cpp | 31 +-- Analysis/src/ToString.cpp | 72 +++++-- Analysis/src/TypeInfer.cpp | 69 ++++--- Analysis/src/TypePack.cpp | 19 +- Analysis/src/TypeVar.cpp | 18 ++ Analysis/src/Unifier.cpp | 173 +++++++++++----- Ast/src/Parser.cpp | 20 +- VM/src/lapi.cpp | 50 ++--- VM/src/ldo.cpp | 21 +- VM/src/ldo.h | 2 +- VM/src/lgc.cpp | 228 +++++++++++---------- VM/src/lgc.h | 2 +- VM/src/lstate.cpp | 26 +-- VM/src/lstate.h | 44 ++-- VM/src/ltable.cpp | 4 +- tests/Autocomplete.test.cpp | 5 - tests/Conformance.test.cpp | 2 - tests/Fixture.cpp | 19 +- tests/Linter.test.cpp | 85 ++------ tests/ToString.test.cpp | 25 +++ tests/Transpiler.test.cpp | 2 - tests/TypeInfer.builtins.test.cpp | 2 - tests/TypeInfer.functions.test.cpp | 76 +++++++ tests/TypeInfer.modules.test.cpp | 85 +++++++- tests/TypeInfer.operators.test.cpp | 26 +++ tests/TypeInfer.refinements.test.cpp | 11 - tests/TypeInfer.singletons.test.cpp | 118 +---------- tests/TypeInfer.tables.test.cpp | 38 ++++ tests/TypeInfer.unionTypes.test.cpp | 1 + 37 files changed, 804 insertions(+), 562 deletions(-) diff --git a/Analysis/include/Luau/Error.h b/Analysis/include/Luau/Error.h index 72350255..53b946a0 100644 --- a/Analysis/include/Luau/Error.h +++ b/Analysis/include/Luau/Error.h @@ -96,6 +96,7 @@ struct CountMismatch size_t expected; size_t actual; Context context = Arg; + bool isVariadic = false; bool operator==(const CountMismatch& rhs) const; }; diff --git a/Analysis/include/Luau/ToString.h b/Analysis/include/Luau/ToString.h index a97bf6d6..49ee82fe 100644 --- a/Analysis/include/Luau/ToString.h +++ b/Analysis/include/Luau/ToString.h @@ -32,6 +32,7 @@ struct ToStringOptions size_t maxTypeLength = size_t(FInt::LuauTypeMaximumStringifierLength); std::optional nameMap; std::shared_ptr scope; // If present, module names will be added and types that are not available in scope will be marked as 'invalid' + std::vector namedFunctionOverrideArgNames; // If present, named function argument names will be overridden }; struct ToStringResult @@ -65,7 +66,7 @@ inline std::string toString(TypePackId ty) std::string toString(const TypeVar& tv, const ToStringOptions& opts = {}); std::string toString(const TypePackVar& tp, const ToStringOptions& opts = {}); -std::string toStringNamedFunction(const std::string& prefix, const FunctionTypeVar& ftv, ToStringOptions opts = {}); +std::string toStringNamedFunction(const std::string& funcName, const FunctionTypeVar& ftv, const ToStringOptions& opts = {}); // It could be useful to see the text representation of a type during a debugging session instead of exploring the content of the class // These functions will dump the type to stdout and can be evaluated in Watch/Immediate windows or as gdb/lldb expression diff --git a/Analysis/include/Luau/TypePack.h b/Analysis/include/Luau/TypePack.h index 946be356..85fa467f 100644 --- a/Analysis/include/Luau/TypePack.h +++ b/Analysis/include/Luau/TypePack.h @@ -119,9 +119,9 @@ bool areEqual(SeenSet& seen, const TypePackVar& lhs, const TypePackVar& rhs); TypePackId follow(TypePackId tp); TypePackId follow(TypePackId tp, std::function mapper); -size_t size(TypePackId tp); -bool finite(TypePackId tp); -size_t size(const TypePack& tp); +size_t size(TypePackId tp, TxnLog* log = nullptr); +bool finite(TypePackId tp, TxnLog* log = nullptr); +size_t size(const TypePack& tp, TxnLog* log = nullptr); std::optional first(TypePackId tp); TypePackVar* asMutable(TypePackId tp); diff --git a/Analysis/include/Luau/TypeVar.h b/Analysis/include/Luau/TypeVar.h index 29578dcd..b8c4b362 100644 --- a/Analysis/include/Luau/TypeVar.h +++ b/Analysis/include/Luau/TypeVar.h @@ -488,6 +488,9 @@ const TableTypeVar* getTableType(TypeId type); // Returns nullptr if the type has no name. const std::string* getName(TypeId type); +// Returns name of the module where type was defined if type has that information +std::optional getDefinitionModuleName(TypeId type); + // Checks whether a union contains all types of another union. bool isSubset(const UnionTypeVar& super, const UnionTypeVar& sub); diff --git a/Analysis/include/Luau/Unifier.h b/Analysis/include/Luau/Unifier.h index f1ffbcc0..474af50c 100644 --- a/Analysis/include/Luau/Unifier.h +++ b/Analysis/include/Luau/Unifier.h @@ -90,7 +90,9 @@ private: TypeId deeplyOptional(TypeId ty, std::unordered_map seen = {}); - void cacheResult(TypeId subTy, TypeId superTy); + bool canCacheResult(TypeId subTy, TypeId superTy); + void cacheResult(TypeId subTy, TypeId superTy, size_t prevErrorCount); + void cacheResult_DEPRECATED(TypeId subTy, TypeId superTy); public: void tryUnify(TypePackId subTy, TypePackId superTy, bool isFunctionCall = false); diff --git a/Analysis/include/Luau/UnifierSharedState.h b/Analysis/include/Luau/UnifierSharedState.h index 88997c41..9a3ba56d 100644 --- a/Analysis/include/Luau/UnifierSharedState.h +++ b/Analysis/include/Luau/UnifierSharedState.h @@ -2,6 +2,7 @@ #pragma once #include "Luau/DenseHash.h" +#include "Luau/Error.h" #include "Luau/TypeVar.h" #include "Luau/TypePack.h" @@ -42,6 +43,7 @@ struct UnifierSharedState DenseHashSet seenAny{nullptr}; DenseHashMap skipCacheForType{nullptr}; DenseHashSet, TypeIdPairHash> cachedUnify{{nullptr, nullptr}}; + DenseHashMap, TypeErrorData, TypeIdPairHash> cachedUnifyError{{nullptr, nullptr}}; DenseHashSet tempSeenTy{nullptr}; DenseHashSet tempSeenTp{nullptr}; diff --git a/Analysis/src/Error.cpp b/Analysis/src/Error.cpp index 26d3b76d..210c0191 100644 --- a/Analysis/src/Error.cpp +++ b/Analysis/src/Error.cpp @@ -8,6 +8,7 @@ #include LUAU_FASTFLAGVARIABLE(BetterDiagnosticCodesInStudio, false); +LUAU_FASTFLAGVARIABLE(LuauTypeMismatchModuleName, false); static std::string wrongNumberOfArgsString(size_t expectedCount, size_t actualCount, const char* argPrefix = nullptr, bool isVariadic = false) { @@ -53,7 +54,32 @@ struct ErrorConverter { std::string operator()(const Luau::TypeMismatch& tm) const { - std::string result = "Type '" + Luau::toString(tm.givenType) + "' could not be converted into '" + Luau::toString(tm.wantedType) + "'"; + std::string givenTypeName = Luau::toString(tm.givenType); + std::string wantedTypeName = Luau::toString(tm.wantedType); + + std::string result; + + if (FFlag::LuauTypeMismatchModuleName) + { + if (givenTypeName == wantedTypeName) + { + if (auto givenDefinitionModule = getDefinitionModuleName(tm.givenType)) + { + if (auto wantedDefinitionModule = getDefinitionModuleName(tm.wantedType)) + { + result = "Type '" + givenTypeName + "' from '" + *givenDefinitionModule + "' could not be converted into '" + wantedTypeName + + "' from '" + *wantedDefinitionModule + "'"; + } + } + } + + if (result.empty()) + result = "Type '" + givenTypeName + "' could not be converted into '" + wantedTypeName + "'"; + } + else + { + result = "Type '" + givenTypeName + "' could not be converted into '" + wantedTypeName + "'"; + } if (tm.error) { @@ -147,7 +173,7 @@ struct ErrorConverter return "Function only returns " + std::to_string(e.expected) + " value" + expectedS + ". " + std::to_string(e.actual) + " are required here"; case CountMismatch::Arg: - return "Argument count mismatch. Function " + wrongNumberOfArgsString(e.expected, e.actual); + return "Argument count mismatch. Function " + wrongNumberOfArgsString(e.expected, e.actual, /*argPrefix*/ nullptr, e.isVariadic); } LUAU_ASSERT(!"Unknown context"); diff --git a/Analysis/src/Linter.cpp b/Analysis/src/Linter.cpp index 56c4e3e8..b7480e34 100644 --- a/Analysis/src/Linter.cpp +++ b/Analysis/src/Linter.cpp @@ -14,6 +14,7 @@ LUAU_FASTINTVARIABLE(LuauSuggestionDistance, 4) LUAU_FASTFLAGVARIABLE(LuauLintGlobalNeverReadBeforeWritten, false) +LUAU_FASTFLAGVARIABLE(LuauLintNoRobloxBits, false) namespace Luau { @@ -1135,16 +1136,20 @@ private: enum TypeKind { - Kind_Invalid, + Kind_Unknown, Kind_Primitive, // primitive type supported by VM - boolean/userdata/etc. No differentiation between types of userdata. - Kind_Vector, // For 'vector' but only used when type is used - Kind_Userdata, // custom userdata type - Vector3/etc. + Kind_Vector, // 'vector' but only used when type is used + Kind_Userdata, // custom userdata type + + // TODO: remove these with LuauLintNoRobloxBits Kind_Class, // custom userdata type that reflects Roblox Instance-derived hierarchy - Part/etc. Kind_Enum, // custom userdata type referring to an enum item of enum classes, e.g. Enum.NormalId.Back/Enum.Axis.X/etc. }; bool containsPropName(TypeId ty, const std::string& propName) { + LUAU_ASSERT(!FFlag::LuauLintNoRobloxBits); + if (auto ctv = get(ty)) return lookupClassProp(ctv, propName) != nullptr; @@ -1163,13 +1168,23 @@ private: if (name == "vector") return Kind_Vector; - if (std::optional maybeTy = context->scope->lookupType(name)) - // Kind_Userdata is probably not 100% precise but is close enough - return containsPropName(maybeTy->type, "ClassName") ? Kind_Class : Kind_Userdata; - else if (std::optional maybeTy = context->scope->lookupImportedType("Enum", name)) - return Kind_Enum; + if (FFlag::LuauLintNoRobloxBits) + { + if (std::optional maybeTy = context->scope->lookupType(name)) + return Kind_Userdata; - return Kind_Invalid; + return Kind_Unknown; + } + else + { + if (std::optional maybeTy = context->scope->lookupType(name)) + // Kind_Userdata is probably not 100% precise but is close enough + return containsPropName(maybeTy->type, "ClassName") ? Kind_Class : Kind_Userdata; + else if (std::optional maybeTy = context->scope->lookupImportedType("Enum", name)) + return Kind_Enum; + + return Kind_Unknown; + } } void validateType(AstExprConstantString* expr, std::initializer_list expected, const char* expectedString) @@ -1177,7 +1192,7 @@ private: std::string name(expr->value.data, expr->value.size); TypeKind kind = getTypeKind(name); - if (kind == Kind_Invalid) + if (kind == Kind_Unknown) { emitWarning(*context, LintWarning::Code_UnknownType, expr->location, "Unknown type '%s'", name.c_str()); return; @@ -1189,7 +1204,7 @@ private: return; // as a special case, Instance and EnumItem are both a userdata type (as returned by typeof) and a class type - if (ek == Kind_Userdata && (name == "Instance" || name == "EnumItem")) + if (!FFlag::LuauLintNoRobloxBits && ek == Kind_Userdata && (name == "Instance" || name == "EnumItem")) return; } @@ -1198,12 +1213,18 @@ private: bool acceptsClassName(AstName method) { + LUAU_ASSERT(!FFlag::LuauLintNoRobloxBits); + return method.value[0] == 'F' && (method == "FindFirstChildOfClass" || method == "FindFirstChildWhichIsA" || method == "FindFirstAncestorOfClass" || method == "FindFirstAncestorWhichIsA"); } bool visit(AstExprCall* node) override { + // TODO: Simply remove the override + if (FFlag::LuauLintNoRobloxBits) + return true; + if (AstExprIndexName* index = node->func->as()) { AstExprConstantString* arg0 = node->args.size > 0 ? node->args.data[0]->as() : NULL; diff --git a/Analysis/src/Module.cpp b/Analysis/src/Module.cpp index a330a98d..0787d3a4 100644 --- a/Analysis/src/Module.cpp +++ b/Analysis/src/Module.cpp @@ -12,10 +12,8 @@ #include LUAU_FASTFLAGVARIABLE(DebugLuauFreezeArena, false) -LUAU_FASTFLAGVARIABLE(DebugLuauTrackOwningArena, false) // Remove with FFlagLuauImmutableTypes LUAU_FASTINTVARIABLE(LuauTypeCloneRecursionLimit, 300) LUAU_FASTFLAGVARIABLE(LuauCloneDeclaredGlobals, false) -LUAU_FASTFLAG(LuauImmutableTypes) namespace Luau { @@ -65,8 +63,7 @@ TypeId TypeArena::addTV(TypeVar&& tv) { TypeId allocated = typeVars.allocate(std::move(tv)); - if (FFlag::DebugLuauTrackOwningArena || FFlag::LuauImmutableTypes) - asMutable(allocated)->owningArena = this; + asMutable(allocated)->owningArena = this; return allocated; } @@ -75,8 +72,7 @@ TypeId TypeArena::freshType(TypeLevel level) { TypeId allocated = typeVars.allocate(FreeTypeVar{level}); - if (FFlag::DebugLuauTrackOwningArena || FFlag::LuauImmutableTypes) - asMutable(allocated)->owningArena = this; + asMutable(allocated)->owningArena = this; return allocated; } @@ -85,8 +81,7 @@ TypePackId TypeArena::addTypePack(std::initializer_list types) { TypePackId allocated = typePacks.allocate(TypePack{std::move(types)}); - if (FFlag::DebugLuauTrackOwningArena || FFlag::LuauImmutableTypes) - asMutable(allocated)->owningArena = this; + asMutable(allocated)->owningArena = this; return allocated; } @@ -95,8 +90,7 @@ TypePackId TypeArena::addTypePack(std::vector types) { TypePackId allocated = typePacks.allocate(TypePack{std::move(types)}); - if (FFlag::DebugLuauTrackOwningArena || FFlag::LuauImmutableTypes) - asMutable(allocated)->owningArena = this; + asMutable(allocated)->owningArena = this; return allocated; } @@ -105,8 +99,7 @@ TypePackId TypeArena::addTypePack(TypePack tp) { TypePackId allocated = typePacks.allocate(std::move(tp)); - if (FFlag::DebugLuauTrackOwningArena || FFlag::LuauImmutableTypes) - asMutable(allocated)->owningArena = this; + asMutable(allocated)->owningArena = this; return allocated; } @@ -115,8 +108,7 @@ TypePackId TypeArena::addTypePack(TypePackVar tp) { TypePackId allocated = typePacks.allocate(std::move(tp)); - if (FFlag::DebugLuauTrackOwningArena || FFlag::LuauImmutableTypes) - asMutable(allocated)->owningArena = this; + asMutable(allocated)->owningArena = this; return allocated; } @@ -439,16 +431,9 @@ TypeId clone(TypeId typeId, TypeArena& dest, SeenTypes& seenTypes, SeenTypePacks TypeCloner cloner{dest, typeId, seenTypes, seenTypePacks, cloneState}; Luau::visit(cloner, typeId->ty); // Mutates the storage that 'res' points into. - if (FFlag::LuauImmutableTypes) - { - // Persistent types are not being cloned and we get the original type back which might be read-only - if (!res->persistent) - asMutable(res)->documentationSymbol = typeId->documentationSymbol; - } - else - { + // Persistent types are not being cloned and we get the original type back which might be read-only + if (!res->persistent) asMutable(res)->documentationSymbol = typeId->documentationSymbol; - } } return res; diff --git a/Analysis/src/ToString.cpp b/Analysis/src/ToString.cpp index 010ca361..59ee6de2 100644 --- a/Analysis/src/ToString.cpp +++ b/Analysis/src/ToString.cpp @@ -16,6 +16,7 @@ * Fair warning: Setting this will break a lot of Luau unit tests. */ LUAU_FASTFLAGVARIABLE(DebugLuauVerboseTypeNames, false) +LUAU_FASTFLAGVARIABLE(LuauDocFuncParameters, false) namespace Luau { @@ -769,6 +770,7 @@ struct TypePackStringifier else state.emit(", "); + // Do not respect opts.namedFunctionOverrideArgNames here if (elemIndex < elemNames.size() && elemNames[elemIndex]) { state.emit(elemNames[elemIndex]->name); @@ -1090,13 +1092,13 @@ std::string toString(const TypePackVar& tp, const ToStringOptions& opts) return toString(const_cast(&tp), std::move(opts)); } -std::string toStringNamedFunction(const std::string& prefix, const FunctionTypeVar& ftv, ToStringOptions opts) +std::string toStringNamedFunction(const std::string& funcName, const FunctionTypeVar& ftv, const ToStringOptions& opts) { ToStringResult result; StringifierState state(opts, result, opts.nameMap); TypeVarStringifier tvs{state}; - state.emit(prefix); + state.emit(funcName); if (!opts.hideNamedFunctionTypeParameters) tvs.stringify(ftv.generics, ftv.genericPacks); @@ -1104,28 +1106,59 @@ std::string toStringNamedFunction(const std::string& prefix, const FunctionTypeV state.emit("("); auto argPackIter = begin(ftv.argTypes); - auto argNameIter = ftv.argNames.begin(); bool first = true; - while (argPackIter != end(ftv.argTypes)) + if (FFlag::LuauDocFuncParameters) { - if (!first) - state.emit(", "); - first = false; - - // We don't currently respect opts.functionTypeArguments. I don't think this function should. - if (argNameIter != ftv.argNames.end()) + size_t idx = 0; + while (argPackIter != end(ftv.argTypes)) { - state.emit((*argNameIter ? (*argNameIter)->name : "_") + ": "); - ++argNameIter; - } - else - { - state.emit("_: "); - } + if (!first) + state.emit(", "); + first = false; - tvs.stringify(*argPackIter); - ++argPackIter; + // We don't respect opts.functionTypeArguments + if (idx < opts.namedFunctionOverrideArgNames.size()) + { + state.emit(opts.namedFunctionOverrideArgNames[idx] + ": "); + } + else if (idx < ftv.argNames.size() && ftv.argNames[idx]) + { + state.emit(ftv.argNames[idx]->name + ": "); + } + else + { + state.emit("_: "); + } + tvs.stringify(*argPackIter); + + ++argPackIter; + ++idx; + } + } + else + { + auto argNameIter = ftv.argNames.begin(); + while (argPackIter != end(ftv.argTypes)) + { + if (!first) + state.emit(", "); + first = false; + + // We don't currently respect opts.functionTypeArguments. I don't think this function should. + if (argNameIter != ftv.argNames.end()) + { + state.emit((*argNameIter ? (*argNameIter)->name : "_") + ": "); + ++argNameIter; + } + else + { + state.emit("_: "); + } + + tvs.stringify(*argPackIter); + ++argPackIter; + } } if (argPackIter.tail()) @@ -1134,7 +1167,6 @@ std::string toStringNamedFunction(const std::string& prefix, const FunctionTypeV state.emit(", "); state.emit("...: "); - if (auto vtp = get(*argPackIter.tail())) tvs.stringify(vtp->ty); else diff --git a/Analysis/src/TypeInfer.cpp b/Analysis/src/TypeInfer.cpp index 41e8ce55..9965d5aa 100644 --- a/Analysis/src/TypeInfer.cpp +++ b/Analysis/src/TypeInfer.cpp @@ -27,10 +27,8 @@ LUAU_FASTFLAGVARIABLE(LuauWeakEqConstraint, false) // Eventually removed as fals LUAU_FASTFLAGVARIABLE(DebugLuauFreezeDuringUnification, false) LUAU_FASTFLAGVARIABLE(LuauRecursiveTypeParameterRestriction, false) LUAU_FASTFLAGVARIABLE(LuauGenericFunctionsDontCacheTypeParams, false) -LUAU_FASTFLAGVARIABLE(LuauImmutableTypes, false) LUAU_FASTFLAGVARIABLE(LuauSealExports, false) LUAU_FASTFLAGVARIABLE(LuauSelfCallAutocompleteFix, false) -LUAU_FASTFLAGVARIABLE(LuauSingletonTypes, false) LUAU_FASTFLAGVARIABLE(LuauDiscriminableUnions2, false) LUAU_FASTFLAGVARIABLE(LuauExpectedTypesOfProperties, false) LUAU_FASTFLAGVARIABLE(LuauErrorRecoveryType, false) @@ -38,6 +36,7 @@ LUAU_FASTFLAGVARIABLE(LuauOnlyMutateInstantiatedTables, false) LUAU_FASTFLAGVARIABLE(LuauPropertiesGetExpectedType, false) LUAU_FASTFLAGVARIABLE(LuauStatFunctionSimplify2, false) LUAU_FASTFLAGVARIABLE(LuauUnsealedTableLiteral, false) +LUAU_FASTFLAG(LuauTypeMismatchModuleName) LUAU_FASTFLAGVARIABLE(LuauTwoPassAliasDefinitionFix, false) LUAU_FASTFLAGVARIABLE(LuauAssertStripsFalsyTypes, false) LUAU_FASTFLAGVARIABLE(LuauReturnAnyInsteadOfICE, false) // Eventually removed as false. @@ -47,6 +46,8 @@ LUAU_FASTFLAGVARIABLE(LuauDoNotAccidentallyDependOnPointerOrdering, false) LUAU_FASTFLAGVARIABLE(LuauFixArgumentCountMismatchAmountWithGenericTypes, false) LUAU_FASTFLAGVARIABLE(LuauFixIncorrectLineNumberDuplicateType, false) LUAU_FASTFLAG(LuauAnyInIsOptionalIsOptional) +LUAU_FASTFLAGVARIABLE(LuauDecoupleOperatorInferenceFromUnifiedTypeInference, false) +LUAU_FASTFLAGVARIABLE(LuauArgCountMismatchSaysAtLeastWhenVariadic, false) namespace Luau { @@ -291,6 +292,7 @@ ModulePtr TypeChecker::check(const SourceModule& module, Mode mode, std::optiona // Clear unifier cache since it's keyed off internal types that get deallocated // This avoids fake cross-module cache hits and keeps cache size at bay when typechecking large module graphs. unifierState.cachedUnify.clear(); + unifierState.cachedUnifyError.clear(); unifierState.skipCacheForType.clear(); if (FFlag::LuauTwoPassAliasDefinitionFix) @@ -1303,7 +1305,7 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatTypeAlias& typealias { // If the table is already named and we want to rename the type function, we have to bind new alias to a copy // Additionally, we can't modify types that come from other modules - if (ttv->name || (FFlag::LuauImmutableTypes && follow(ty)->owningArena != ¤tModule->internalTypes)) + if (ttv->name || follow(ty)->owningArena != ¤tModule->internalTypes) { bool sameTys = std::equal(ttv->instantiatedTypeParams.begin(), ttv->instantiatedTypeParams.end(), binding->typeParams.begin(), binding->typeParams.end(), [](auto&& itp, auto&& tp) { @@ -1315,7 +1317,7 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatTypeAlias& typealias }); // Copy can be skipped if this is an identical alias - if ((FFlag::LuauImmutableTypes && !ttv->name) || ttv->name != name || !sameTys || !sameTps) + if (!ttv->name || ttv->name != name || !sameTys || !sameTps) { // This is a shallow clone, original recursive links to self are not updated TableTypeVar clone = TableTypeVar{ttv->props, ttv->indexer, ttv->level, ttv->state}; @@ -1349,7 +1351,7 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatTypeAlias& typealias else if (auto mtv = getMutable(follow(ty))) { // We can't modify types that come from other modules - if (!FFlag::LuauImmutableTypes || follow(ty)->owningArena == ¤tModule->internalTypes) + if (follow(ty)->owningArena == ¤tModule->internalTypes) mtv->syntheticName = name; } @@ -1512,14 +1514,14 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExpr& result = {nilType}; else if (const AstExprConstantBool* bexpr = expr.as()) { - if (FFlag::LuauSingletonTypes && (forceSingleton || (expectedType && maybeSingleton(*expectedType)))) + if (forceSingleton || (expectedType && maybeSingleton(*expectedType))) result = {singletonType(bexpr->value)}; else result = {booleanType}; } else if (const AstExprConstantString* sexpr = expr.as()) { - if (FFlag::LuauSingletonTypes && (forceSingleton || (expectedType && maybeSingleton(*expectedType)))) + if (forceSingleton || (expectedType && maybeSingleton(*expectedType))) result = {singletonType(std::string(sexpr->value.data, sexpr->value.size))}; else result = {stringType}; @@ -2490,12 +2492,24 @@ TypeId TypeChecker::checkBinaryOperation( lhsType = follow(lhsType); rhsType = follow(rhsType); - if (!isNonstrictMode() && get(lhsType)) + if (FFlag::LuauDecoupleOperatorInferenceFromUnifiedTypeInference) { - auto name = getIdentifierOfBaseVar(expr.left); - reportError(expr.location, CannotInferBinaryOperation{expr.op, name, CannotInferBinaryOperation::Operation}); - if (!FFlag::LuauErrorRecoveryType) - return errorRecoveryType(scope); + if (!isNonstrictMode() && get(lhsType)) + { + auto name = getIdentifierOfBaseVar(expr.left); + reportError(expr.location, CannotInferBinaryOperation{expr.op, name, CannotInferBinaryOperation::Operation}); + // We will fall-through to the `return anyType` check below. + } + } + else + { + if (!isNonstrictMode() && get(lhsType)) + { + auto name = getIdentifierOfBaseVar(expr.left); + reportError(expr.location, CannotInferBinaryOperation{expr.op, name, CannotInferBinaryOperation::Operation}); + if (!FFlag::LuauErrorRecoveryType) + return errorRecoveryType(scope); + } } // If we know nothing at all about the lhs type, we can usually say nothing about the result. @@ -3452,7 +3466,8 @@ void TypeChecker::checkArgumentList( { if (FFlag::LuauFixArgumentCountMismatchAmountWithGenericTypes) minParams = getMinParameterCount(&state.log, paramPack); - state.reportError(TypeError{state.location, CountMismatch{minParams, paramIndex}}); + bool isVariadic = FFlag::LuauArgCountMismatchSaysAtLeastWhenVariadic && !finite(paramPack, &state.log); + state.reportError(TypeError{state.location, CountMismatch{minParams, paramIndex, CountMismatch::Context::Arg, isVariadic}}); return; } ++paramIter; @@ -4163,13 +4178,7 @@ TypeId TypeChecker::checkRequire(const ScopePtr& scope, const ModuleInfo& module return errorRecoveryType(scope); } - if (FFlag::LuauImmutableTypes) - return *moduleType; - - SeenTypes seenTypes; - SeenTypePacks seenTypePacks; - CloneState cloneState; - return clone(*moduleType, currentModule->internalTypes, seenTypes, seenTypePacks, cloneState); + return *moduleType; } void TypeChecker::tablify(TypeId type) @@ -4941,10 +4950,19 @@ TypeId TypeChecker::resolveType(const ScopePtr& scope, const AstType& annotation if (const auto& indexer = table->indexer) tableIndexer = TableIndexer(resolveType(scope, *indexer->indexType), resolveType(scope, *indexer->resultType)); - return addType(TableTypeVar{ - props, tableIndexer, scope->level, - TableState::Sealed // FIXME: probably want a way to annotate other kinds of tables maybe - }); + if (FFlag::LuauTypeMismatchModuleName) + { + TableTypeVar ttv{props, tableIndexer, scope->level, TableState::Sealed}; + ttv.definitionModuleName = currentModuleName; + return addType(std::move(ttv)); + } + else + { + return addType(TableTypeVar{ + props, tableIndexer, scope->level, + TableState::Sealed // FIXME: probably want a way to annotate other kinds of tables maybe + }); + } } else if (const auto& func = annotation.as()) { @@ -5206,6 +5224,9 @@ TypeId TypeChecker::instantiateTypeFun(const ScopePtr& scope, const TypeFun& tf, { ttv->instantiatedTypeParams = typeParams; ttv->instantiatedTypePackParams = typePackParams; + + if (FFlag::LuauTypeMismatchModuleName) + ttv->definitionModuleName = currentModuleName; } return instantiated; diff --git a/Analysis/src/TypePack.cpp b/Analysis/src/TypePack.cpp index 91123f46..5bb05234 100644 --- a/Analysis/src/TypePack.cpp +++ b/Analysis/src/TypePack.cpp @@ -222,20 +222,21 @@ TypePackId follow(TypePackId tp, std::function mapper) } } -size_t size(TypePackId tp) +size_t size(TypePackId tp, TxnLog* log) { - if (auto pack = get(follow(tp))) - return size(*pack); + tp = log ? log->follow(tp) : follow(tp); + if (auto pack = get(tp)) + return size(*pack, log); else return 0; } -bool finite(TypePackId tp) +bool finite(TypePackId tp, TxnLog* log) { - tp = follow(tp); + tp = log ? log->follow(tp) : follow(tp); if (auto pack = get(tp)) - return pack->tail ? finite(*pack->tail) : true; + return pack->tail ? finite(*pack->tail, log) : true; if (get(tp)) return false; @@ -243,14 +244,14 @@ bool finite(TypePackId tp) return true; } -size_t size(const TypePack& tp) +size_t size(const TypePack& tp, TxnLog* log) { size_t result = tp.head.size(); if (tp.tail) { - const TypePack* tail = get(follow(*tp.tail)); + const TypePack* tail = get(log ? log->follow(*tp.tail) : follow(*tp.tail)); if (tail) - result += size(*tail); + result += size(*tail, log); } return result; } diff --git a/Analysis/src/TypeVar.cpp b/Analysis/src/TypeVar.cpp index 89549535..36545ad9 100644 --- a/Analysis/src/TypeVar.cpp +++ b/Analysis/src/TypeVar.cpp @@ -290,6 +290,24 @@ const std::string* getName(TypeId type) return nullptr; } +std::optional getDefinitionModuleName(TypeId type) +{ + type = follow(type); + + if (auto ttv = get(type)) + { + if (!ttv->definitionModuleName.empty()) + return ttv->definitionModuleName; + } + else if (auto ftv = get(type)) + { + if (ftv->definition) + return ftv->definition->definitionModuleName; + } + + return std::nullopt; +} + bool isSubset(const UnionTypeVar& super, const UnionTypeVar& sub) { std::unordered_set superTypes; diff --git a/Analysis/src/Unifier.cpp b/Analysis/src/Unifier.cpp index 60a9c9a5..398dc9e2 100644 --- a/Analysis/src/Unifier.cpp +++ b/Analysis/src/Unifier.cpp @@ -14,10 +14,9 @@ LUAU_FASTINT(LuauTypeInferRecursionLimit); LUAU_FASTINT(LuauTypeInferTypePackLoopLimit); -LUAU_FASTFLAG(LuauImmutableTypes) LUAU_FASTINTVARIABLE(LuauTypeInferIterationLimit, 2000); +LUAU_FASTFLAGVARIABLE(LuauExtendedIndexerError, false); LUAU_FASTFLAGVARIABLE(LuauTableSubtypingVariance2, false); -LUAU_FASTFLAG(LuauSingletonTypes) LUAU_FASTFLAG(LuauErrorRecoveryType); LUAU_FASTFLAGVARIABLE(LuauSubtypingAddOptPropsToUnsealedTables, false) LUAU_FASTFLAGVARIABLE(LuauWidenIfSupertypeIsFree2, false) @@ -26,6 +25,7 @@ LUAU_FASTFLAGVARIABLE(LuauTxnLogSeesTypePacks2, false) LUAU_FASTFLAGVARIABLE(LuauTxnLogCheckForInvalidation, false) LUAU_FASTFLAGVARIABLE(LuauTxnLogRefreshFunctionPointers, false) LUAU_FASTFLAGVARIABLE(LuauTxnLogDontRetryForIndexers, false) +LUAU_FASTFLAGVARIABLE(LuauUnifierCacheErrors, false) LUAU_FASTFLAG(LuauAnyInIsOptionalIsOptional) namespace Luau @@ -63,7 +63,7 @@ struct PromoteTypeLevels bool operator()(TID ty, const T&) { // Type levels of types from other modules are already global, so we don't need to promote anything inside - if (FFlag::LuauImmutableTypes && ty->owningArena != typeArena) + if (ty->owningArena != typeArena) return false; return true; @@ -83,7 +83,7 @@ struct PromoteTypeLevels bool operator()(TypeId ty, const FunctionTypeVar&) { // Type levels of types from other modules are already global, so we don't need to promote anything inside - if (FFlag::LuauImmutableTypes && ty->owningArena != typeArena) + if (ty->owningArena != typeArena) return false; promote(ty, log.getMutable(ty)); @@ -93,7 +93,7 @@ struct PromoteTypeLevels bool operator()(TypeId ty, const TableTypeVar& ttv) { // Type levels of types from other modules are already global, so we don't need to promote anything inside - if (FFlag::LuauImmutableTypes && ty->owningArena != typeArena) + if (ty->owningArena != typeArena) return false; if (ttv.state != TableState::Free && ttv.state != TableState::Generic) @@ -118,7 +118,7 @@ struct PromoteTypeLevels static void promoteTypeLevels(TxnLog& log, const TypeArena* typeArena, TypeLevel minLevel, TypeId ty) { // Type levels of types from other modules are already global, so we don't need to promote anything inside - if (FFlag::LuauImmutableTypes && ty->owningArena != typeArena) + if (ty->owningArena != typeArena) return; PromoteTypeLevels ptl{log, typeArena, minLevel}; @@ -130,7 +130,7 @@ static void promoteTypeLevels(TxnLog& log, const TypeArena* typeArena, TypeLevel void promoteTypeLevels(TxnLog& log, const TypeArena* typeArena, TypeLevel minLevel, TypePackId tp) { // Type levels of types from other modules are already global, so we don't need to promote anything inside - if (FFlag::LuauImmutableTypes && tp->owningArena != typeArena) + if (tp->owningArena != typeArena) return; PromoteTypeLevels ptl{log, typeArena, minLevel}; @@ -170,7 +170,7 @@ struct SkipCacheForType bool operator()(TypeId ty, const TableTypeVar&) { // Types from other modules don't contain mutable elements and are ok to cache - if (FFlag::LuauImmutableTypes && ty->owningArena != typeArena) + if (ty->owningArena != typeArena) return false; TableTypeVar& ttv = *getMutable(ty); @@ -194,7 +194,7 @@ struct SkipCacheForType bool operator()(TypeId ty, const T& t) { // Types from other modules don't contain mutable elements and are ok to cache - if (FFlag::LuauImmutableTypes && ty->owningArena != typeArena) + if (ty->owningArena != typeArena) return false; const bool* prev = skipCacheForType.find(ty); @@ -212,7 +212,7 @@ struct SkipCacheForType bool operator()(TypePackId tp, const T&) { // Types from other modules don't contain mutable elements and are ok to cache - if (FFlag::LuauImmutableTypes && tp->owningArena != typeArena) + if (tp->owningArena != typeArena) return false; return true; @@ -445,12 +445,33 @@ void Unifier::tryUnify_(TypeId subTy, TypeId superTy, bool isFunctionCall, bool if (get(subTy) || get(subTy)) return tryUnifyWithAny(superTy, subTy); - bool cacheEnabled = !isFunctionCall && !isIntersection; + bool cacheEnabled; auto& cache = sharedState.cachedUnify; // What if the types are immutable and we proved their relation before - if (cacheEnabled && cache.contains({superTy, subTy}) && (variance == Covariant || cache.contains({subTy, superTy}))) - return; + if (FFlag::LuauUnifierCacheErrors) + { + cacheEnabled = !isFunctionCall && !isIntersection && variance == Invariant; + + if (cacheEnabled) + { + if (cache.contains({subTy, superTy})) + return; + + if (auto error = sharedState.cachedUnifyError.find({subTy, superTy})) + { + reportError(TypeError{location, *error}); + return; + } + } + } + else + { + cacheEnabled = !isFunctionCall && !isIntersection; + + if (cacheEnabled && cache.contains({superTy, subTy}) && (variance == Covariant || cache.contains({subTy, superTy}))) + return; + } // If we have seen this pair of types before, we are currently recursing into cyclic types. // Here, we assume that the types unify. If they do not, we will find out as we roll back @@ -461,6 +482,8 @@ void Unifier::tryUnify_(TypeId subTy, TypeId superTy, bool isFunctionCall, bool log.pushSeen(superTy, subTy); + size_t errorCount = errors.size(); + if (const UnionTypeVar* uv = log.getMutable(subTy)) { tryUnifyUnionWithType(subTy, uv, superTy); @@ -480,8 +503,7 @@ void Unifier::tryUnify_(TypeId subTy, TypeId superTy, bool isFunctionCall, bool else if (log.getMutable(superTy) && log.getMutable(subTy)) tryUnifyPrimitives(subTy, superTy); - else if (FFlag::LuauSingletonTypes && (log.getMutable(superTy) || log.getMutable(superTy)) && - log.getMutable(subTy)) + else if ((log.getMutable(superTy) || log.getMutable(superTy)) && log.getMutable(subTy)) tryUnifySingletons(subTy, superTy); else if (log.getMutable(superTy) && log.getMutable(subTy)) @@ -491,8 +513,11 @@ void Unifier::tryUnify_(TypeId subTy, TypeId superTy, bool isFunctionCall, bool { tryUnifyTables(subTy, superTy, isIntersection); - if (cacheEnabled && errors.empty()) - cacheResult(subTy, superTy); + if (!FFlag::LuauUnifierCacheErrors) + { + if (cacheEnabled && errors.empty()) + cacheResult_DEPRECATED(subTy, superTy); + } } // tryUnifyWithMetatable assumes its first argument is a MetatableTypeVar. The check is otherwise symmetrical. @@ -512,6 +537,9 @@ void Unifier::tryUnify_(TypeId subTy, TypeId superTy, bool isFunctionCall, bool else reportError(TypeError{location, TypeMismatch{superTy, subTy}}); + if (FFlag::LuauUnifierCacheErrors && cacheEnabled) + cacheResult(subTy, superTy, errorCount); + log.popSeen(superTy, subTy); } @@ -646,10 +674,21 @@ void Unifier::tryUnifyTypeWithUnion(TypeId subTy, TypeId superTy, const UnionTyp { TypeId type = uv->options[i]; - if (cache.contains({type, subTy}) && (variance == Covariant || cache.contains({subTy, type}))) + if (FFlag::LuauUnifierCacheErrors) { - startIndex = i; - break; + if (cache.contains({subTy, type})) + { + startIndex = i; + break; + } + } + else + { + if (cache.contains({type, subTy}) && (variance == Covariant || cache.contains({subTy, type}))) + { + startIndex = i; + break; + } } } } @@ -737,10 +776,21 @@ void Unifier::tryUnifyIntersectionWithType(TypeId subTy, const IntersectionTypeV { TypeId type = uv->parts[i]; - if (cache.contains({superTy, type}) && (variance == Covariant || cache.contains({type, superTy}))) + if (FFlag::LuauUnifierCacheErrors) { - startIndex = i; - break; + if (cache.contains({type, superTy})) + { + startIndex = i; + break; + } + } + else + { + if (cache.contains({superTy, type}) && (variance == Covariant || cache.contains({type, superTy}))) + { + startIndex = i; + break; + } } } } @@ -771,17 +821,17 @@ void Unifier::tryUnifyIntersectionWithType(TypeId subTy, const IntersectionTypeV } } -void Unifier::cacheResult(TypeId subTy, TypeId superTy) +bool Unifier::canCacheResult(TypeId subTy, TypeId superTy) { bool* superTyInfo = sharedState.skipCacheForType.find(superTy); if (superTyInfo && *superTyInfo) - return; + return false; bool* subTyInfo = sharedState.skipCacheForType.find(subTy); if (subTyInfo && *subTyInfo) - return; + return false; auto skipCacheFor = [this](TypeId ty) { SkipCacheForType visitor{sharedState.skipCacheForType, types}; @@ -793,9 +843,33 @@ void Unifier::cacheResult(TypeId subTy, TypeId superTy) }; if (!superTyInfo && skipCacheFor(superTy)) - return; + return false; if (!subTyInfo && skipCacheFor(subTy)) + return false; + + return true; +} + +void Unifier::cacheResult(TypeId subTy, TypeId superTy, size_t prevErrorCount) +{ + if (errors.size() == prevErrorCount) + { + if (canCacheResult(subTy, superTy)) + sharedState.cachedUnify.insert({subTy, superTy}); + } + else if (errors.size() == prevErrorCount + 1) + { + if (canCacheResult(subTy, superTy)) + sharedState.cachedUnifyError[{subTy, superTy}] = errors.back().data; + } +} + +void Unifier::cacheResult_DEPRECATED(TypeId subTy, TypeId superTy) +{ + LUAU_ASSERT(!FFlag::LuauUnifierCacheErrors); + + if (!canCacheResult(subTy, superTy)) return; sharedState.cachedUnify.insert({superTy, subTy}); @@ -1283,24 +1357,6 @@ void Unifier::tryUnifyFunctions(TypeId subTy, TypeId superTy, bool isFunctionCal subFunction = log.getMutable(subTy); } - if (!FFlag::LuauImmutableTypes) - { - if (superFunction->definition && !subFunction->definition && !subTy->persistent) - { - PendingType* newSubTy = log.queue(subTy); - FunctionTypeVar* newSubFtv = getMutable(newSubTy); - LUAU_ASSERT(newSubFtv); - newSubFtv->definition = superFunction->definition; - } - else if (!superFunction->definition && subFunction->definition && !superTy->persistent) - { - PendingType* newSuperTy = log.queue(superTy); - FunctionTypeVar* newSuperFtv = getMutable(newSuperTy); - LUAU_ASSERT(newSuperFtv); - newSuperFtv->definition = subFunction->definition; - } - } - ctx = context; if (FFlag::LuauTxnLogSeesTypePacks2) @@ -1563,8 +1619,25 @@ void Unifier::tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) variance = Invariant; Unifier innerState = makeChildUnifier(); - innerState.tryUnifyIndexer(*subTable->indexer, *superTable->indexer); - checkChildUnifierTypeMismatch(innerState.errors, superTy, subTy); + + if (FFlag::LuauExtendedIndexerError) + { + innerState.tryUnify_(subTable->indexer->indexType, superTable->indexer->indexType); + + bool reported = !innerState.errors.empty(); + + checkChildUnifierTypeMismatch(innerState.errors, "[indexer key]", superTy, subTy); + + innerState.tryUnify_(subTable->indexer->indexResultType, superTable->indexer->indexResultType); + + if (!reported) + checkChildUnifierTypeMismatch(innerState.errors, "[indexer value]", superTy, subTy); + } + else + { + innerState.tryUnifyIndexer(*subTable->indexer, *superTable->indexer); + checkChildUnifierTypeMismatch(innerState.errors, superTy, subTy); + } if (innerState.errors.empty()) log.concat(std::move(innerState.log)); @@ -1771,6 +1844,7 @@ void Unifier::DEPRECATED_tryUnifyTables(TypeId subTy, TypeId superTy, bool isInt void Unifier::tryUnifyFreeTable(TypeId subTy, TypeId superTy) { + LUAU_ASSERT(!FFlag::LuauTableSubtypingVariance2); TableTypeVar* freeTable = log.getMutable(superTy); TableTypeVar* subTable = log.getMutable(subTy); @@ -1840,6 +1914,7 @@ void Unifier::tryUnifyFreeTable(TypeId subTy, TypeId superTy) void Unifier::tryUnifySealedTables(TypeId subTy, TypeId superTy, bool isIntersection) { + LUAU_ASSERT(!FFlag::LuauTableSubtypingVariance2); TableTypeVar* superTable = log.getMutable(superTy); TableTypeVar* subTable = log.getMutable(subTy); @@ -2120,6 +2195,8 @@ void Unifier::tryUnifyWithClass(TypeId subTy, TypeId superTy, bool reversed) void Unifier::tryUnifyIndexer(const TableIndexer& subIndexer, const TableIndexer& superIndexer) { + LUAU_ASSERT(!FFlag::LuauTableSubtypingVariance2 || !FFlag::LuauExtendedIndexerError); + tryUnify_(subIndexer.indexType, superIndexer.indexType); tryUnify_(subIndexer.indexResultType, superIndexer.indexResultType); } @@ -2211,7 +2288,7 @@ static void tryUnifyWithAny(std::vector& queue, Unifier& state, DenseHas queue.pop_back(); // Types from other modules don't have free types - if (FFlag::LuauImmutableTypes && ty->owningArena != typeArena) + if (ty->owningArena != typeArena) continue; if (seen.find(ty)) diff --git a/Ast/src/Parser.cpp b/Ast/src/Parser.cpp index 941a3ea4..f6dfd904 100644 --- a/Ast/src/Parser.cpp +++ b/Ast/src/Parser.cpp @@ -10,8 +10,6 @@ // See docs/SyntaxChanges.md for an explanation. LUAU_FASTINTVARIABLE(LuauRecursionLimit, 1000) LUAU_FASTINTVARIABLE(LuauParseErrorLimit, 100) -LUAU_FASTFLAGVARIABLE(LuauParseSingletonTypes, false) -LUAU_FASTFLAGVARIABLE(LuauTableFieldFunctionDebugname, false) namespace Luau { @@ -1233,8 +1231,7 @@ AstType* Parser::parseTableTypeAnnotation() while (lexer.current().type != '}') { - if (FFlag::LuauParseSingletonTypes && lexer.current().type == '[' && - (lexer.lookahead().type == Lexeme::RawString || lexer.lookahead().type == Lexeme::QuotedString)) + if (lexer.current().type == '[' && (lexer.lookahead().type == Lexeme::RawString || lexer.lookahead().type == Lexeme::QuotedString)) { const Lexeme begin = lexer.current(); nextLexeme(); // [ @@ -1500,17 +1497,17 @@ AstTypeOrPack Parser::parseSimpleTypeAnnotation(bool allowPack) nextLexeme(); return {allocator.alloc(begin, std::nullopt, nameNil), {}}; } - else if (FFlag::LuauParseSingletonTypes && lexer.current().type == Lexeme::ReservedTrue) + else if (lexer.current().type == Lexeme::ReservedTrue) { nextLexeme(); return {allocator.alloc(begin, true)}; } - else if (FFlag::LuauParseSingletonTypes && lexer.current().type == Lexeme::ReservedFalse) + else if (lexer.current().type == Lexeme::ReservedFalse) { nextLexeme(); return {allocator.alloc(begin, false)}; } - else if (FFlag::LuauParseSingletonTypes && (lexer.current().type == Lexeme::RawString || lexer.current().type == Lexeme::QuotedString)) + else if (lexer.current().type == Lexeme::RawString || lexer.current().type == Lexeme::QuotedString) { if (std::optional> value = parseCharArray()) { @@ -1520,7 +1517,7 @@ AstTypeOrPack Parser::parseSimpleTypeAnnotation(bool allowPack) else return {reportTypeAnnotationError(begin, {}, /*isMissing*/ false, "String literal contains malformed escape sequence")}; } - else if (FFlag::LuauParseSingletonTypes && lexer.current().type == Lexeme::BrokenString) + else if (lexer.current().type == Lexeme::BrokenString) { Location location = lexer.current().location; nextLexeme(); @@ -2189,11 +2186,8 @@ AstExpr* Parser::parseTableConstructor() AstExpr* key = allocator.alloc(name.location, nameString); AstExpr* value = parseExpr(); - if (FFlag::LuauTableFieldFunctionDebugname) - { - if (AstExprFunction* func = value->as()) - func->debugname = name.name; - } + if (AstExprFunction* func = value->as()) + func->debugname = name.name; items.push_back({AstExprTable::Item::Record, key, value}); } diff --git a/VM/src/lapi.cpp b/VM/src/lapi.cpp index 3c087314..46b10934 100644 --- a/VM/src/lapi.cpp +++ b/VM/src/lapi.cpp @@ -14,8 +14,6 @@ #include -LUAU_FASTFLAG(LuauGcAdditionalStats) - const char* lua_ident = "$Lua: Lua 5.1.4 Copyright (C) 1994-2008 Lua.org, PUC-Rio $\n" "$Authors: R. Ierusalimschy, L. H. de Figueiredo & W. Celes $\n" "$URL: www.lua.org $\n"; @@ -1060,8 +1058,11 @@ int lua_gc(lua_State* L, int what, int data) g->GCthreshold = 0; bool waspaused = g->gcstate == GCSpause; - double startmarktime = g->gcstats.currcycle.marktime; - double startsweeptime = g->gcstats.currcycle.sweeptime; + +#ifdef LUAI_GCMETRICS + double startmarktime = g->gcmetrics.currcycle.marktime; + double startsweeptime = g->gcmetrics.currcycle.sweeptime; +#endif // track how much work the loop will actually perform size_t actualwork = 0; @@ -1079,31 +1080,30 @@ int lua_gc(lua_State* L, int what, int data) } } - if (FFlag::LuauGcAdditionalStats) +#ifdef LUAI_GCMETRICS + // record explicit step statistics + GCCycleMetrics* cyclemetrics = g->gcstate == GCSpause ? &g->gcmetrics.lastcycle : &g->gcmetrics.currcycle; + + double totalmarktime = cyclemetrics->marktime - startmarktime; + double totalsweeptime = cyclemetrics->sweeptime - startsweeptime; + + if (totalmarktime > 0.0) { - // record explicit step statistics - GCCycleStats* cyclestats = g->gcstate == GCSpause ? &g->gcstats.lastcycle : &g->gcstats.currcycle; + cyclemetrics->markexplicitsteps++; - double totalmarktime = cyclestats->marktime - startmarktime; - double totalsweeptime = cyclestats->sweeptime - startsweeptime; - - if (totalmarktime > 0.0) - { - cyclestats->markexplicitsteps++; - - if (totalmarktime > cyclestats->markmaxexplicittime) - cyclestats->markmaxexplicittime = totalmarktime; - } - - if (totalsweeptime > 0.0) - { - cyclestats->sweepexplicitsteps++; - - if (totalsweeptime > cyclestats->sweepmaxexplicittime) - cyclestats->sweepmaxexplicittime = totalsweeptime; - } + if (totalmarktime > cyclemetrics->markmaxexplicittime) + cyclemetrics->markmaxexplicittime = totalmarktime; } + if (totalsweeptime > 0.0) + { + cyclemetrics->sweepexplicitsteps++; + + if (totalsweeptime > cyclemetrics->sweepmaxexplicittime) + cyclemetrics->sweepmaxexplicittime = totalsweeptime; + } +#endif + // if cycle hasn't finished, advance threshold forward for the amount of extra work performed if (g->gcstate != GCSpause) { diff --git a/VM/src/ldo.cpp b/VM/src/ldo.cpp index b5ae496b..c133a59e 100644 --- a/VM/src/ldo.cpp +++ b/VM/src/ldo.cpp @@ -17,8 +17,6 @@ #include -LUAU_FASTFLAG(LuauReduceStackReallocs) - /* ** {====================================================== ** Error-recovery functions @@ -33,6 +31,15 @@ struct lua_jmpbuf jmp_buf buf; }; +/* use POSIX versions of setjmp/longjmp if possible: they don't save/restore signal mask and are therefore faster */ +#if defined(__linux__) || defined(__APPLE__) +#define LUAU_SETJMP(buf) _setjmp(buf) +#define LUAU_LONGJMP(buf, code) _longjmp(buf, code) +#else +#define LUAU_SETJMP(buf) setjmp(buf) +#define LUAU_LONGJMP(buf, code) longjmp(buf, code) +#endif + int luaD_rawrunprotected(lua_State* L, Pfunc f, void* ud) { lua_jmpbuf jb; @@ -40,7 +47,7 @@ int luaD_rawrunprotected(lua_State* L, Pfunc f, void* ud) jb.status = 0; L->global->errorjmp = &jb; - if (setjmp(jb.buf) == 0) + if (LUAU_SETJMP(jb.buf) == 0) f(L, ud); L->global->errorjmp = jb.prev; @@ -52,7 +59,7 @@ l_noret luaD_throw(lua_State* L, int errcode) if (lua_jmpbuf* jb = L->global->errorjmp) { jb->status = errcode; - longjmp(jb->buf, 1); + LUAU_LONGJMP(jb->buf, 1); } if (L->global->cb.panic) @@ -165,8 +172,8 @@ static void correctstack(lua_State* L, TValue* oldstack) void luaD_reallocstack(lua_State* L, int newsize) { TValue* oldstack = L->stack; - int realsize = newsize + (FFlag::LuauReduceStackReallocs ? EXTRA_STACK : 1 + EXTRA_STACK); - LUAU_ASSERT(L->stack_last - L->stack == L->stacksize - (FFlag::LuauReduceStackReallocs ? EXTRA_STACK : 1 + EXTRA_STACK)); + int realsize = newsize + EXTRA_STACK; + LUAU_ASSERT(L->stack_last - L->stack == L->stacksize - EXTRA_STACK); luaM_reallocarray(L, L->stack, L->stacksize, realsize, TValue, L->memcat); TValue* newstack = L->stack; for (int i = L->stacksize; i < realsize; i++) @@ -514,7 +521,7 @@ static void callerrfunc(lua_State* L, void* ud) static void restore_stack_limit(lua_State* L) { - LUAU_ASSERT(L->stack_last - L->stack == L->stacksize - (FFlag::LuauReduceStackReallocs ? EXTRA_STACK : 1 + EXTRA_STACK)); + LUAU_ASSERT(L->stack_last - L->stack == L->stacksize - EXTRA_STACK); if (L->size_ci > LUAI_MAXCALLS) { /* there was an overflow? */ int inuse = cast_int(L->ci - L->base_ci); diff --git a/VM/src/ldo.h b/VM/src/ldo.h index 1c1480d6..6e16e6f1 100644 --- a/VM/src/ldo.h +++ b/VM/src/ldo.h @@ -11,7 +11,7 @@ if ((char*)L->stack_last - (char*)L->top <= (n) * (int)sizeof(TValue)) \ luaD_growstack(L, n); \ else \ - condhardstacktests(luaD_reallocstack(L, L->stacksize - (FFlag::LuauReduceStackReallocs ? EXTRA_STACK : 1 + EXTRA_STACK))); + condhardstacktests(luaD_reallocstack(L, L->stacksize - EXTRA_STACK)); #define incr_top(L) \ { \ diff --git a/VM/src/lgc.cpp b/VM/src/lgc.cpp index a656854e..8fc930d5 100644 --- a/VM/src/lgc.cpp +++ b/VM/src/lgc.cpp @@ -11,8 +11,6 @@ #include "lmem.h" #include "ludata.h" -LUAU_FASTFLAGVARIABLE(LuauGcAdditionalStats, false) - #include #define GC_SWEEPMAX 40 @@ -48,7 +46,8 @@ LUAU_FASTFLAGVARIABLE(LuauGcAdditionalStats, false) reallymarkobject(g, obj2gco(t)); \ } -static void recordGcStateTime(global_State* g, int startgcstate, double seconds, bool assist) +#ifdef LUAI_GCMETRICS +static void recordGcStateStep(global_State* g, int startgcstate, double seconds, bool assist, size_t work) { switch (startgcstate) { @@ -56,58 +55,76 @@ static void recordGcStateTime(global_State* g, int startgcstate, double seconds, // record root mark time if we have switched to next state if (g->gcstate == GCSpropagate) { - g->gcstats.currcycle.marktime += seconds; + g->gcmetrics.currcycle.marktime += seconds; - if (FFlag::LuauGcAdditionalStats && assist) - g->gcstats.currcycle.markassisttime += seconds; + if (assist) + g->gcmetrics.currcycle.markassisttime += seconds; } break; case GCSpropagate: case GCSpropagateagain: - g->gcstats.currcycle.marktime += seconds; + g->gcmetrics.currcycle.marktime += seconds; + g->gcmetrics.currcycle.markrequests += g->gcstepsize; - if (FFlag::LuauGcAdditionalStats && assist) - g->gcstats.currcycle.markassisttime += seconds; + if (assist) + g->gcmetrics.currcycle.markassisttime += seconds; break; case GCSatomic: - g->gcstats.currcycle.atomictime += seconds; + g->gcmetrics.currcycle.atomictime += seconds; break; case GCSsweep: - g->gcstats.currcycle.sweeptime += seconds; + g->gcmetrics.currcycle.sweeptime += seconds; + g->gcmetrics.currcycle.sweeprequests += g->gcstepsize; - if (FFlag::LuauGcAdditionalStats && assist) - g->gcstats.currcycle.sweepassisttime += seconds; + if (assist) + g->gcmetrics.currcycle.sweepassisttime += seconds; break; default: LUAU_ASSERT(!"Unexpected GC state"); } if (assist) - g->gcstats.stepassisttimeacc += seconds; + { + g->gcmetrics.stepassisttimeacc += seconds; + g->gcmetrics.currcycle.assistwork += work; + g->gcmetrics.currcycle.assistrequests += g->gcstepsize; + } else - g->gcstats.stepexplicittimeacc += seconds; + { + g->gcmetrics.stepexplicittimeacc += seconds; + g->gcmetrics.currcycle.explicitwork += work; + g->gcmetrics.currcycle.explicitrequests += g->gcstepsize; + } } -static void startGcCycleStats(global_State* g) +static double recordGcDeltaTime(double& timer) { - g->gcstats.currcycle.starttimestamp = lua_clock(); - g->gcstats.currcycle.pausetime = g->gcstats.currcycle.starttimestamp - g->gcstats.lastcycle.endtimestamp; + double now = lua_clock(); + double delta = now - timer; + timer = now; + return delta; } -static void finishGcCycleStats(global_State* g) +static void startGcCycleMetrics(global_State* g) { - g->gcstats.currcycle.endtimestamp = lua_clock(); - g->gcstats.currcycle.endtotalsizebytes = g->totalbytes; - - g->gcstats.completedcycles++; - g->gcstats.lastcycle = g->gcstats.currcycle; - g->gcstats.currcycle = GCCycleStats(); - - g->gcstats.cyclestatsacc.marktime += g->gcstats.lastcycle.marktime; - g->gcstats.cyclestatsacc.atomictime += g->gcstats.lastcycle.atomictime; - g->gcstats.cyclestatsacc.sweeptime += g->gcstats.lastcycle.sweeptime; + g->gcmetrics.currcycle.starttimestamp = lua_clock(); + g->gcmetrics.currcycle.pausetime = g->gcmetrics.currcycle.starttimestamp - g->gcmetrics.lastcycle.endtimestamp; } +static void finishGcCycleMetrics(global_State* g) +{ + g->gcmetrics.currcycle.endtimestamp = lua_clock(); + g->gcmetrics.currcycle.endtotalsizebytes = g->totalbytes; + + g->gcmetrics.completedcycles++; + g->gcmetrics.lastcycle = g->gcmetrics.currcycle; + g->gcmetrics.currcycle = GCCycleMetrics(); + + g->gcmetrics.currcycle.starttotalsizebytes = g->totalbytes; + g->gcmetrics.currcycle.heaptriggersizebytes = g->GCthreshold; +} +#endif + static void removeentry(LuaNode* n) { LUAU_ASSERT(ttisnil(gval(n))); @@ -598,20 +615,19 @@ static size_t atomic(lua_State* L) LUAU_ASSERT(g->gcstate == GCSatomic); size_t work = 0; + +#ifdef LUAI_GCMETRICS double currts = lua_clock(); - double prevts = currts; +#endif /* remark occasional upvalues of (maybe) dead threads */ work += remarkupvals(g); /* traverse objects caught by write barrier and by 'remarkupvals' */ work += propagateall(g); - if (FFlag::LuauGcAdditionalStats) - { - currts = lua_clock(); - g->gcstats.currcycle.atomictimeupval += currts - prevts; - prevts = currts; - } +#ifdef LUAI_GCMETRICS + g->gcmetrics.currcycle.atomictimeupval += recordGcDeltaTime(currts); +#endif /* remark weak tables */ g->gray = g->weak; @@ -621,34 +637,26 @@ static size_t atomic(lua_State* L) markmt(g); /* mark basic metatables (again) */ work += propagateall(g); - if (FFlag::LuauGcAdditionalStats) - { - currts = lua_clock(); - g->gcstats.currcycle.atomictimeweak += currts - prevts; - prevts = currts; - } +#ifdef LUAI_GCMETRICS + g->gcmetrics.currcycle.atomictimeweak += recordGcDeltaTime(currts); +#endif /* remark gray again */ g->gray = g->grayagain; g->grayagain = NULL; work += propagateall(g); - if (FFlag::LuauGcAdditionalStats) - { - currts = lua_clock(); - g->gcstats.currcycle.atomictimegray += currts - prevts; - prevts = currts; - } +#ifdef LUAI_GCMETRICS + g->gcmetrics.currcycle.atomictimegray += recordGcDeltaTime(currts); +#endif /* remove collected objects from weak tables */ work += cleartable(L, g->weak); g->weak = NULL; - if (FFlag::LuauGcAdditionalStats) - { - currts = lua_clock(); - g->gcstats.currcycle.atomictimeclear += currts - prevts; - } +#ifdef LUAI_GCMETRICS + g->gcmetrics.currcycle.atomictimeclear += recordGcDeltaTime(currts); +#endif /* flip current white */ g->currentwhite = cast_byte(otherwhite(g)); @@ -742,8 +750,9 @@ static size_t gcstep(lua_State* L, size_t limit) if (!g->gray) { - if (FFlag::LuauGcAdditionalStats) - g->gcstats.currcycle.propagatework = g->gcstats.currcycle.explicitwork + g->gcstats.currcycle.assistwork; +#ifdef LUAI_GCMETRICS + g->gcmetrics.currcycle.propagatework = g->gcmetrics.currcycle.explicitwork + g->gcmetrics.currcycle.assistwork; +#endif // perform one iteration over 'gray again' list g->gray = g->grayagain; @@ -762,9 +771,10 @@ static size_t gcstep(lua_State* L, size_t limit) if (!g->gray) /* no more `gray' objects */ { - if (FFlag::LuauGcAdditionalStats) - g->gcstats.currcycle.propagateagainwork = - g->gcstats.currcycle.explicitwork + g->gcstats.currcycle.assistwork - g->gcstats.currcycle.propagatework; +#ifdef LUAI_GCMETRICS + g->gcmetrics.currcycle.propagateagainwork = + g->gcmetrics.currcycle.explicitwork + g->gcmetrics.currcycle.assistwork - g->gcmetrics.currcycle.propagatework; +#endif g->gcstate = GCSatomic; } @@ -772,8 +782,13 @@ static size_t gcstep(lua_State* L, size_t limit) } case GCSatomic: { - g->gcstats.currcycle.atomicstarttimestamp = lua_clock(); - g->gcstats.currcycle.atomicstarttotalsizebytes = g->totalbytes; +#ifdef LUAI_GCMETRICS + g->gcmetrics.currcycle.atomicstarttimestamp = lua_clock(); + g->gcmetrics.currcycle.atomicstarttotalsizebytes = g->totalbytes; +#endif + + g->gcstats.atomicstarttimestamp = lua_clock(); + g->gcstats.atomicstarttotalsizebytes = g->totalbytes; cost = atomic(L); /* finish mark phase */ @@ -809,18 +824,20 @@ static size_t gcstep(lua_State* L, size_t limit) return cost; } -static int64_t getheaptriggererroroffset(GCHeapTriggerStats* triggerstats, GCCycleStats* cyclestats) +static int64_t getheaptriggererroroffset(global_State* g) { // adjust for error using Proportional-Integral controller // https://en.wikipedia.org/wiki/PID_controller - int32_t errorKb = int32_t((cyclestats->atomicstarttotalsizebytes - cyclestats->heapgoalsizebytes) / 1024); + int32_t errorKb = int32_t((g->gcstats.atomicstarttotalsizebytes - g->gcstats.heapgoalsizebytes) / 1024); // we use sliding window for the error integral to avoid error sum 'windup' when the desired target cannot be reached - int32_t* slot = &triggerstats->terms[triggerstats->termpos % triggerstats->termcount]; + const size_t triggertermcount = sizeof(g->gcstats.triggerterms) / sizeof(g->gcstats.triggerterms[0]); + + int32_t* slot = &g->gcstats.triggerterms[g->gcstats.triggertermpos % triggertermcount]; int32_t prev = *slot; *slot = errorKb; - triggerstats->integral += errorKb - prev; - triggerstats->termpos++; + g->gcstats.triggerintegral += errorKb - prev; + g->gcstats.triggertermpos++; // controller tuning // https://en.wikipedia.org/wiki/Ziegler%E2%80%93Nichols_method @@ -832,7 +849,7 @@ static int64_t getheaptriggererroroffset(GCHeapTriggerStats* triggerstats, GCCyc const double Ki = 0.54 * Ku / Ti; // integral gain double proportionalTerm = Kp * errorKb; - double integralTerm = Ki * triggerstats->integral; + double integralTerm = Ki * g->gcstats.triggerintegral; double totalTerm = proportionalTerm + integralTerm; @@ -841,23 +858,20 @@ static int64_t getheaptriggererroroffset(GCHeapTriggerStats* triggerstats, GCCyc static size_t getheaptrigger(global_State* g, size_t heapgoal) { - GCCycleStats* lastcycle = &g->gcstats.lastcycle; - GCCycleStats* currcycle = &g->gcstats.currcycle; - // adjust threshold based on a guess of how many bytes will be allocated between the cycle start and sweep phase // our goal is to begin the sweep when used memory has reached the heap goal const double durationthreshold = 1e-3; - double allocationduration = currcycle->atomicstarttimestamp - lastcycle->endtimestamp; + double allocationduration = g->gcstats.atomicstarttimestamp - g->gcstats.endtimestamp; // avoid measuring intervals smaller than 1ms if (allocationduration < durationthreshold) return heapgoal; - double allocationrate = (currcycle->atomicstarttotalsizebytes - lastcycle->endtotalsizebytes) / allocationduration; - double markduration = currcycle->atomicstarttimestamp - currcycle->starttimestamp; + double allocationrate = (g->gcstats.atomicstarttotalsizebytes - g->gcstats.endtotalsizebytes) / allocationduration; + double markduration = g->gcstats.atomicstarttimestamp - g->gcstats.starttimestamp; int64_t expectedgrowth = int64_t(markduration * allocationrate); - int64_t offset = getheaptriggererroroffset(&g->gcstats.triggerstats, currcycle); + int64_t offset = getheaptriggererroroffset(g); int64_t heaptrigger = heapgoal - (expectedgrowth + offset); // clamp the trigger between memory use at the end of the cycle and the heap goal @@ -868,11 +882,6 @@ void luaC_step(lua_State* L, bool assist) { global_State* g = L->global; - if (assist) - g->gcstats.currcycle.assistrequests += g->gcstepsize; - else - g->gcstats.currcycle.explicitrequests += g->gcstepsize; - int lim = (g->gcstepsize / 100) * g->gcstepmul; /* how much to work */ LUAU_ASSERT(g->totalbytes >= g->GCthreshold); size_t debt = g->totalbytes - g->GCthreshold; @@ -881,24 +890,23 @@ void luaC_step(lua_State* L, bool assist) // at the start of the new cycle if (g->gcstate == GCSpause) - startGcCycleStats(g); + g->gcstats.starttimestamp = lua_clock(); + +#ifdef LUAI_GCMETRICS + if (g->gcstate == GCSpause) + startGcCycleMetrics(g); + + double lasttimestamp = lua_clock(); +#endif int lastgcstate = g->gcstate; - double lasttimestamp = lua_clock(); size_t work = gcstep(L, lim); + (void)work; - if (assist) - g->gcstats.currcycle.assistwork += work; - else - g->gcstats.currcycle.explicitwork += work; - - recordGcStateTime(g, lastgcstate, lua_clock() - lasttimestamp, assist); - - if (lastgcstate == GCSpropagate) - g->gcstats.currcycle.markrequests += g->gcstepsize; - else if (lastgcstate == GCSsweep) - g->gcstats.currcycle.sweeprequests += g->gcstepsize; +#ifdef LUAI_GCMETRICS + recordGcStateStep(g, lastgcstate, lua_clock() - lasttimestamp, assist, work); +#endif // at the end of the last cycle if (g->gcstate == GCSpause) @@ -909,13 +917,13 @@ void luaC_step(lua_State* L, bool assist) g->GCthreshold = heaptrigger; - finishGcCycleStats(g); + g->gcstats.heapgoalsizebytes = heapgoal; + g->gcstats.endtimestamp = lua_clock(); + g->gcstats.endtotalsizebytes = g->totalbytes; - if (FFlag::LuauGcAdditionalStats) - g->gcstats.currcycle.starttotalsizebytes = g->totalbytes; - - g->gcstats.currcycle.heapgoalsizebytes = heapgoal; - g->gcstats.currcycle.heaptriggersizebytes = heaptrigger; +#ifdef LUAI_GCMETRICS + finishGcCycleMetrics(g); +#endif } else { @@ -933,8 +941,10 @@ void luaC_fullgc(lua_State* L) { global_State* g = L->global; +#ifdef LUAI_GCMETRICS if (g->gcstate == GCSpause) - startGcCycleStats(g); + startGcCycleMetrics(g); +#endif if (g->gcstate <= GCSatomic) { @@ -954,11 +964,12 @@ void luaC_fullgc(lua_State* L) gcstep(L, SIZE_MAX); } - finishGcCycleStats(g); +#ifdef LUAI_GCMETRICS + finishGcCycleMetrics(g); + startGcCycleMetrics(g); +#endif /* run a full collection cycle */ - startGcCycleStats(g); - markroot(L); while (g->gcstate != GCSpause) { @@ -980,10 +991,11 @@ void luaC_fullgc(lua_State* L) if (g->GCthreshold < g->totalbytes) g->GCthreshold = g->totalbytes; - finishGcCycleStats(g); + g->gcstats.heapgoalsizebytes = heapgoalsizebytes; - g->gcstats.currcycle.heapgoalsizebytes = heapgoalsizebytes; - g->gcstats.currcycle.heaptriggersizebytes = g->GCthreshold; +#ifdef LUAI_GCMETRICS + finishGcCycleMetrics(g); +#endif } void luaC_barrierupval(lua_State* L, GCObject* v) @@ -1075,21 +1087,21 @@ int64_t luaC_allocationrate(lua_State* L) if (g->gcstate <= GCSatomic) { - double duration = lua_clock() - g->gcstats.lastcycle.endtimestamp; + double duration = lua_clock() - g->gcstats.endtimestamp; if (duration < durationthreshold) return -1; - return int64_t((g->totalbytes - g->gcstats.lastcycle.endtotalsizebytes) / duration); + return int64_t((g->totalbytes - g->gcstats.endtotalsizebytes) / duration); } // totalbytes is unstable during the sweep, use the rate measured at the end of mark phase - double duration = g->gcstats.currcycle.atomicstarttimestamp - g->gcstats.lastcycle.endtimestamp; + double duration = g->gcstats.atomicstarttimestamp - g->gcstats.endtimestamp; if (duration < durationthreshold) return -1; - return int64_t((g->gcstats.currcycle.atomicstarttotalsizebytes - g->gcstats.lastcycle.endtotalsizebytes) / duration); + return int64_t((g->gcstats.atomicstarttotalsizebytes - g->gcstats.endtotalsizebytes) / duration); } void luaC_wakethread(lua_State* L) diff --git a/VM/src/lgc.h b/VM/src/lgc.h index ebf999b5..dcd070b7 100644 --- a/VM/src/lgc.h +++ b/VM/src/lgc.h @@ -82,7 +82,7 @@ #define luaC_checkGC(L) \ { \ - condhardstacktests(luaD_reallocstack(L, L->stacksize - (FFlag::LuauReduceStackReallocs ? EXTRA_STACK : 1 + EXTRA_STACK))); \ + condhardstacktests(luaD_reallocstack(L, L->stacksize - EXTRA_STACK)); \ if (L->global->totalbytes >= L->global->GCthreshold) \ { \ condhardmemtests(luaC_validate(L), 1); \ diff --git a/VM/src/lstate.cpp b/VM/src/lstate.cpp index d4f3f0a1..fbc6fb1e 100644 --- a/VM/src/lstate.cpp +++ b/VM/src/lstate.cpp @@ -10,8 +10,6 @@ #include "ldo.h" #include "ldebug.h" -LUAU_FASTFLAGVARIABLE(LuauReduceStackReallocs, false) - /* ** Main thread combines a thread state and the global state */ @@ -35,7 +33,7 @@ static void stack_init(lua_State* L1, lua_State* L) for (int i = 0; i < BASIC_STACK_SIZE + EXTRA_STACK; i++) setnilvalue(stack + i); /* erase new stack */ L1->top = stack; - L1->stack_last = stack + (L1->stacksize - (FFlag::LuauReduceStackReallocs ? EXTRA_STACK : 1 + EXTRA_STACK)); + L1->stack_last = stack + (L1->stacksize - EXTRA_STACK); /* initialize first ci */ L1->ci->func = L1->top; setnilvalue(L1->top++); /* `function' entry for this `ci' */ @@ -141,30 +139,16 @@ void lua_resetthread(lua_State* L) ci->top = ci->base + LUA_MINSTACK; setnilvalue(ci->func); L->ci = ci; - if (FFlag::LuauReduceStackReallocs) - { - if (L->size_ci != BASIC_CI_SIZE) - luaD_reallocCI(L, BASIC_CI_SIZE); - } - else - { + if (L->size_ci != BASIC_CI_SIZE) luaD_reallocCI(L, BASIC_CI_SIZE); - } /* clear thread state */ L->status = LUA_OK; L->base = L->ci->base; L->top = L->ci->base; L->nCcalls = L->baseCcalls = 0; /* clear thread stack */ - if (FFlag::LuauReduceStackReallocs) - { - if (L->stacksize != BASIC_STACK_SIZE + EXTRA_STACK) - luaD_reallocstack(L, BASIC_STACK_SIZE); - } - else - { + if (L->stacksize != BASIC_STACK_SIZE + EXTRA_STACK) luaD_reallocstack(L, BASIC_STACK_SIZE); - } for (int i = 0; i < L->stacksize; i++) setnilvalue(L->stack + i); } @@ -234,6 +218,10 @@ lua_State* lua_newstate(lua_Alloc f, void* ud) g->cb = lua_Callbacks(); g->gcstats = GCStats(); +#ifdef LUAI_GCMETRICS + g->gcmetrics = GCMetrics(); +#endif + if (luaD_rawrunprotected(L, f_luaopen, NULL) != 0) { /* memory allocation error: free partial state */ diff --git a/VM/src/lstate.h b/VM/src/lstate.h index b2bedb48..e7c37373 100644 --- a/VM/src/lstate.h +++ b/VM/src/lstate.h @@ -75,10 +75,26 @@ typedef struct CallInfo #define f_isLua(ci) (!ci_func(ci)->isC) #define isLua(ci) (ttisfunction((ci)->func) && f_isLua(ci)) -struct GCCycleStats +struct GCStats +{ + // data for proportional-integral controller of heap trigger value + int32_t triggerterms[32] = {0}; + uint32_t triggertermpos = 0; + int32_t triggerintegral = 0; + + size_t atomicstarttotalsizebytes = 0; + size_t endtotalsizebytes = 0; + size_t heapgoalsizebytes = 0; + + double starttimestamp = 0; + double atomicstarttimestamp = 0; + double endtimestamp = 0; +}; + +#ifdef LUAI_GCMETRICS +struct GCCycleMetrics { size_t starttotalsizebytes = 0; - size_t heapgoalsizebytes = 0; size_t heaptriggersizebytes = 0; double pausetime = 0.0; // time from end of the last cycle to the start of a new one @@ -120,16 +136,7 @@ struct GCCycleStats size_t endtotalsizebytes = 0; }; -// data for proportional-integral controller of heap trigger value -struct GCHeapTriggerStats -{ - static const unsigned termcount = 32; - int32_t terms[termcount] = {0}; - uint32_t termpos = 0; - int32_t integral = 0; -}; - -struct GCStats +struct GCMetrics { double stepexplicittimeacc = 0.0; double stepassisttimeacc = 0.0; @@ -137,14 +144,10 @@ struct GCStats // when cycle is completed, last cycle values are updated uint64_t completedcycles = 0; - GCCycleStats lastcycle; - GCCycleStats currcycle; - - // only step count and their time is accumulated - GCCycleStats cyclestatsacc; - - GCHeapTriggerStats triggerstats; + GCCycleMetrics lastcycle; + GCCycleMetrics currcycle; }; +#endif /* ** `global state', shared by all threads of this state @@ -206,6 +209,9 @@ typedef struct global_State GCStats gcstats; +#ifdef LUAI_GCMETRICS + GCMetrics gcmetrics; +#endif } global_State; // clang-format on diff --git a/VM/src/ltable.cpp b/VM/src/ltable.cpp index 2deec2b9..431501f3 100644 --- a/VM/src/ltable.cpp +++ b/VM/src/ltable.cpp @@ -526,8 +526,8 @@ static TValue* newkey(lua_State* L, Table* t, const TValue* key) LuaNode* othern; LuaNode* n = getfreepos(t); /* get a free place */ if (n == NULL) - { /* cannot find a free place? */ - rehash(L, t, key); /* grow table */ + { /* cannot find a free place? */ + rehash(L, t, key); /* grow table */ if (!FFlag::LuauTableRehashRework) { diff --git a/tests/Autocomplete.test.cpp b/tests/Autocomplete.test.cpp index 17fd6b13..1db782cc 100644 --- a/tests/Autocomplete.test.cpp +++ b/tests/Autocomplete.test.cpp @@ -2726,11 +2726,6 @@ end TEST_CASE_FIXTURE(ACFixture, "autocomplete_on_string_singletons") { - ScopedFastFlag sffs[] = { - {"LuauParseSingletonTypes", true}, - {"LuauSingletonTypes", true}, - }; - check(R"( --!strict local foo: "hello" | "bye" = "hello" diff --git a/tests/Conformance.test.cpp b/tests/Conformance.test.cpp index 9e4cb4a5..83d4518d 100644 --- a/tests/Conformance.test.cpp +++ b/tests/Conformance.test.cpp @@ -496,8 +496,6 @@ TEST_CASE("DateTime") TEST_CASE("Debug") { - ScopedFastFlag luauTableFieldFunctionDebugname{"LuauTableFieldFunctionDebugname", true}; - runConformance("debug.lua"); } diff --git a/tests/Fixture.cpp b/tests/Fixture.cpp index dbdd06a4..a7e7ea39 100644 --- a/tests/Fixture.cpp +++ b/tests/Fixture.cpp @@ -132,21 +132,9 @@ AstStatBlock* Fixture::parse(const std::string& source, const ParseOptions& pars } CheckResult Fixture::check(Mode mode, std::string source) -{ - configResolver.defaultConfig.mode = mode; - fileResolver.source[mainModuleName] = std::move(source); - - CheckResult result = frontend.check(fromString(mainModuleName)); - - configResolver.defaultConfig.mode = Mode::Strict; - - return result; -} - -CheckResult Fixture::check(const std::string& source) { ModuleName mm = fromString(mainModuleName); - configResolver.defaultConfig.mode = Mode::Strict; + configResolver.defaultConfig.mode = mode; fileResolver.source[mm] = std::move(source); frontend.markDirty(mm); @@ -155,6 +143,11 @@ CheckResult Fixture::check(const std::string& source) return result; } +CheckResult Fixture::check(const std::string& source) +{ + return check(Mode::Strict, source); +} + LintResult Fixture::lint(const std::string& source, const std::optional& lintOptions) { ParseOptions parseOptions; diff --git a/tests/Linter.test.cpp b/tests/Linter.test.cpp index 91b23197..9ce9a4c2 100644 --- a/tests/Linter.test.cpp +++ b/tests/Linter.test.cpp @@ -597,6 +597,8 @@ return foo1 TEST_CASE_FIXTURE(Fixture, "UnknownType") { + ScopedFastFlag sff("LuauLintNoRobloxBits", true); + unfreeze(typeChecker.globalTypes); TableTypeVar::Props instanceProps{ {"ClassName", {typeChecker.anyType}}, @@ -606,81 +608,26 @@ TEST_CASE_FIXTURE(Fixture, "UnknownType") TypeId instanceType = typeChecker.globalTypes.addType(instanceTable); TypeFun instanceTypeFun{{}, instanceType}; - ClassTypeVar::Props enumItemProps{ - {"EnumType", {typeChecker.anyType}}, - }; - - ClassTypeVar enumItemClass{"EnumItem", enumItemProps, std::nullopt, std::nullopt, {}, {}}; - TypeId enumItemType = typeChecker.globalTypes.addType(enumItemClass); - TypeFun enumItemTypeFun{{}, enumItemType}; - - ClassTypeVar normalIdClass{"NormalId", {}, enumItemType, std::nullopt, {}, {}}; - TypeId normalIdType = typeChecker.globalTypes.addType(normalIdClass); - TypeFun normalIdTypeFun{{}, normalIdType}; - - // Normally this would be defined externally, so hack it in for testing - addGlobalBinding(typeChecker, "game", typeChecker.anyType, "@test"); - addGlobalBinding(typeChecker, "typeof", typeChecker.anyType, "@test"); typeChecker.globalScope->exportedTypeBindings["Part"] = instanceTypeFun; - typeChecker.globalScope->exportedTypeBindings["Workspace"] = instanceTypeFun; - typeChecker.globalScope->exportedTypeBindings["RunService"] = instanceTypeFun; - typeChecker.globalScope->exportedTypeBindings["Instance"] = instanceTypeFun; - typeChecker.globalScope->exportedTypeBindings["ColorSequence"] = TypeFun{{}, typeChecker.anyType}; - typeChecker.globalScope->exportedTypeBindings["EnumItem"] = enumItemTypeFun; - typeChecker.globalScope->importedTypeBindings["Enum"] = {{"NormalId", normalIdTypeFun}}; - freeze(typeChecker.globalTypes); LintResult result = lint(R"( -local _e01 = game:GetService("Foo") -local _e02 = game:GetService("NormalId") -local _e03 = game:FindService("table") -local _e04 = type(game) == "Part" -local _e05 = type(game) == "NormalId" -local _e06 = typeof(game) == "Bar" -local _e07 = typeof(game) == "Part" -local _e08 = typeof(game) == "vector" -local _e09 = typeof(game) == "NormalId" -local _e10 = game:IsA("ColorSequence") -local _e11 = game:IsA("Enum.NormalId") -local _e12 = game:FindFirstChildWhichIsA("function") +local game = ... +local _e01 = type(game) == "Part" +local _e02 = typeof(game) == "Bar" +local _e03 = typeof(game) == "vector" -local _o01 = game:GetService("Workspace") -local _o02 = game:FindService("RunService") -local _o03 = type(game) == "number" -local _o04 = type(game) == "vector" -local _o05 = typeof(game) == "string" -local _o06 = typeof(game) == "Instance" -local _o07 = typeof(game) == "EnumItem" -local _o08 = game:IsA("Part") -local _o09 = game:IsA("NormalId") -local _o10 = game:FindFirstChildWhichIsA("Part") +local _o01 = type(game) == "number" +local _o02 = type(game) == "vector" +local _o03 = typeof(game) == "Part" )"); - REQUIRE_EQ(result.warnings.size(), 12); - CHECK_EQ(result.warnings[0].location.begin.line, 1); - CHECK_EQ(result.warnings[0].text, "Unknown type 'Foo'"); - CHECK_EQ(result.warnings[1].location.begin.line, 2); - CHECK_EQ(result.warnings[1].text, "Unknown type 'NormalId' (expected class type)"); - CHECK_EQ(result.warnings[2].location.begin.line, 3); - CHECK_EQ(result.warnings[2].text, "Unknown type 'table' (expected class type)"); - CHECK_EQ(result.warnings[3].location.begin.line, 4); - CHECK_EQ(result.warnings[3].text, "Unknown type 'Part' (expected primitive type)"); - CHECK_EQ(result.warnings[4].location.begin.line, 5); - CHECK_EQ(result.warnings[4].text, "Unknown type 'NormalId' (expected primitive type)"); - CHECK_EQ(result.warnings[5].location.begin.line, 6); - CHECK_EQ(result.warnings[5].text, "Unknown type 'Bar'"); - CHECK_EQ(result.warnings[6].location.begin.line, 7); - CHECK_EQ(result.warnings[6].text, "Unknown type 'Part' (expected primitive or userdata type)"); - CHECK_EQ(result.warnings[7].location.begin.line, 8); - CHECK_EQ(result.warnings[7].text, "Unknown type 'vector' (expected primitive or userdata type)"); - CHECK_EQ(result.warnings[8].location.begin.line, 9); - CHECK_EQ(result.warnings[8].text, "Unknown type 'NormalId' (expected primitive or userdata type)"); - CHECK_EQ(result.warnings[9].location.begin.line, 10); - CHECK_EQ(result.warnings[9].text, "Unknown type 'ColorSequence' (expected class or enum type)"); - CHECK_EQ(result.warnings[10].location.begin.line, 11); - CHECK_EQ(result.warnings[10].text, "Unknown type 'Enum.NormalId'"); - CHECK_EQ(result.warnings[11].location.begin.line, 12); - CHECK_EQ(result.warnings[11].text, "Unknown type 'function' (expected class type)"); + REQUIRE_EQ(result.warnings.size(), 3); + CHECK_EQ(result.warnings[0].location.begin.line, 2); + CHECK_EQ(result.warnings[0].text, "Unknown type 'Part' (expected primitive type)"); + CHECK_EQ(result.warnings[1].location.begin.line, 3); + CHECK_EQ(result.warnings[1].text, "Unknown type 'Bar'"); + CHECK_EQ(result.warnings[2].location.begin.line, 4); + CHECK_EQ(result.warnings[2].text, "Unknown type 'vector' (expected primitive or userdata type)"); } TEST_CASE_FIXTURE(Fixture, "ForRangeTable") diff --git a/tests/ToString.test.cpp b/tests/ToString.test.cpp index 6713a589..3051e209 100644 --- a/tests/ToString.test.cpp +++ b/tests/ToString.test.cpp @@ -470,6 +470,7 @@ TEST_CASE_FIXTURE(Fixture, "self_recursive_instantiated_param") TEST_CASE_FIXTURE(Fixture, "toStringNamedFunction_id") { + ScopedFastFlag flag{"LuauDocFuncParameters", true}; CheckResult result = check(R"( local function id(x) return x end )"); @@ -482,6 +483,7 @@ TEST_CASE_FIXTURE(Fixture, "toStringNamedFunction_id") TEST_CASE_FIXTURE(Fixture, "toStringNamedFunction_map") { + ScopedFastFlag flag{"LuauDocFuncParameters", true}; CheckResult result = check(R"( local function map(arr, fn) local t = {} @@ -500,6 +502,7 @@ TEST_CASE_FIXTURE(Fixture, "toStringNamedFunction_map") TEST_CASE_FIXTURE(Fixture, "toStringNamedFunction_generic_pack") { + ScopedFastFlag flag{"LuauDocFuncParameters", true}; CheckResult result = check(R"( local function f(a: number, b: string) end local function test(...: T...): U... @@ -516,6 +519,7 @@ TEST_CASE_FIXTURE(Fixture, "toStringNamedFunction_generic_pack") TEST_CASE("toStringNamedFunction_unit_f") { + ScopedFastFlag flag{"LuauDocFuncParameters", true}; TypePackVar empty{TypePack{}}; FunctionTypeVar ftv{&empty, &empty, {}, false}; CHECK_EQ("f(): ()", toStringNamedFunction("f", ftv)); @@ -523,6 +527,7 @@ TEST_CASE("toStringNamedFunction_unit_f") TEST_CASE_FIXTURE(Fixture, "toStringNamedFunction_variadics") { + ScopedFastFlag flag{"LuauDocFuncParameters", true}; CheckResult result = check(R"( local function f(x: a, ...): (a, a, b...) return x, x, ... @@ -537,6 +542,7 @@ TEST_CASE_FIXTURE(Fixture, "toStringNamedFunction_variadics") TEST_CASE_FIXTURE(Fixture, "toStringNamedFunction_variadics2") { + ScopedFastFlag flag{"LuauDocFuncParameters", true}; CheckResult result = check(R"( local function f(): ...number return 1, 2, 3 @@ -551,6 +557,7 @@ TEST_CASE_FIXTURE(Fixture, "toStringNamedFunction_variadics2") TEST_CASE_FIXTURE(Fixture, "toStringNamedFunction_variadics3") { + ScopedFastFlag flag{"LuauDocFuncParameters", true}; CheckResult result = check(R"( local function f(): (string, ...number) return 'a', 1, 2, 3 @@ -565,6 +572,7 @@ TEST_CASE_FIXTURE(Fixture, "toStringNamedFunction_variadics3") TEST_CASE_FIXTURE(Fixture, "toStringNamedFunction_type_annotation_has_partial_argnames") { + ScopedFastFlag flag{"LuauDocFuncParameters", true}; CheckResult result = check(R"( local f: (number, y: number) -> number )"); @@ -577,6 +585,7 @@ TEST_CASE_FIXTURE(Fixture, "toStringNamedFunction_type_annotation_has_partial_ar TEST_CASE_FIXTURE(Fixture, "toStringNamedFunction_hide_type_params") { + ScopedFastFlag flag{"LuauDocFuncParameters", true}; CheckResult result = check(R"( local function f(x: T, g: (T) -> U)): () end @@ -590,4 +599,20 @@ TEST_CASE_FIXTURE(Fixture, "toStringNamedFunction_hide_type_params") CHECK_EQ("f(x: T, g: (T) -> U): ()", toStringNamedFunction("f", *ftv, opts)); } +TEST_CASE_FIXTURE(Fixture, "toStringNamedFunction_overrides_param_names") +{ + ScopedFastFlag flag{"LuauDocFuncParameters", true}; + + CheckResult result = check(R"( + local function test(a, b : string, ... : number) return a end + )"); + + TypeId ty = requireType("test"); + const FunctionTypeVar* ftv = get(follow(ty)); + + ToStringOptions opts; + opts.namedFunctionOverrideArgNames = {"first", "second", "third"}; + CHECK_EQ("test(first: a, second: string, ...: number): a", toStringNamedFunction("test", *ftv, opts)); +} + TEST_SUITE_END(); diff --git a/tests/Transpiler.test.cpp b/tests/Transpiler.test.cpp index 5f0295b0..5ac45ff2 100644 --- a/tests/Transpiler.test.cpp +++ b/tests/Transpiler.test.cpp @@ -651,8 +651,6 @@ local a: Packed TEST_CASE_FIXTURE(Fixture, "transpile_singleton_types") { - ScopedFastFlag luauParseSingletonTypes{"LuauParseSingletonTypes", true}; - std::string code = R"( type t1 = 'hello' type t2 = true diff --git a/tests/TypeInfer.builtins.test.cpp b/tests/TypeInfer.builtins.test.cpp index ec20a2c7..c6fbebed 100644 --- a/tests/TypeInfer.builtins.test.cpp +++ b/tests/TypeInfer.builtins.test.cpp @@ -887,8 +887,6 @@ TEST_CASE_FIXTURE(Fixture, "assert_removes_falsy_types") TEST_CASE_FIXTURE(Fixture, "assert_removes_falsy_types2") { ScopedFastFlag sff[]{ - {"LuauParseSingletonTypes", true}, - {"LuauSingletonTypes", true}, {"LuauAssertStripsFalsyTypes", true}, {"LuauDiscriminableUnions2", true}, {"LuauWidenIfSupertypeIsFree2", true}, diff --git a/tests/TypeInfer.functions.test.cpp b/tests/TypeInfer.functions.test.cpp index 4288098a..da4ea074 100644 --- a/tests/TypeInfer.functions.test.cpp +++ b/tests/TypeInfer.functions.test.cpp @@ -1335,4 +1335,80 @@ caused by: toString(result.errors[0])); } +TEST_CASE_FIXTURE(Fixture, "too_few_arguments_variadic") +{ + ScopedFastFlag sff{"LuauArgCountMismatchSaysAtLeastWhenVariadic", true}; + CheckResult result = check(R"( + function test(a: number, b: string, ...) + end + + test(1) + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + auto err = result.errors[0]; + auto acm = get(err); + REQUIRE(acm); + + CHECK_EQ(2, acm->expected); + CHECK_EQ(1, acm->actual); + CHECK_EQ(CountMismatch::Context::Arg, acm->context); + CHECK(acm->isVariadic); +} + +TEST_CASE_FIXTURE(Fixture, "too_few_arguments_variadic_generic") +{ + ScopedFastFlag sff1{"LuauArgCountMismatchSaysAtLeastWhenVariadic", true}; + ScopedFastFlag sff2{"LuauFixArgumentCountMismatchAmountWithGenericTypes", true}; + CheckResult result = check(R"( +function test(a: number, b: string, ...) + return 1 +end + +function wrapper(f: (A...) -> number, ...: A...) +end + +wrapper(test) + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + auto err = result.errors[0]; + auto acm = get(err); + REQUIRE(acm); + + CHECK_EQ(3, acm->expected); + CHECK_EQ(1, acm->actual); + CHECK_EQ(CountMismatch::Context::Arg, acm->context); + CHECK(acm->isVariadic); +} + +TEST_CASE_FIXTURE(Fixture, "too_few_arguments_variadic_generic2") +{ + ScopedFastFlag sff1{"LuauArgCountMismatchSaysAtLeastWhenVariadic", true}; + ScopedFastFlag sff2{"LuauFixArgumentCountMismatchAmountWithGenericTypes", true}; + CheckResult result = check(R"( +function test(a: number, b: string, ...) + return 1 +end + +function wrapper(f: (A...) -> number, ...: A...) +end + +pcall(wrapper, test) + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + auto err = result.errors[0]; + auto acm = get(err); + REQUIRE(acm); + + CHECK_EQ(4, acm->expected); + CHECK_EQ(2, acm->actual); + CHECK_EQ(CountMismatch::Context::Arg, acm->context); + CHECK(acm->isVariadic); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.modules.test.cpp b/tests/TypeInfer.modules.test.cpp index 63643610..e5eeae31 100644 --- a/tests/TypeInfer.modules.test.cpp +++ b/tests/TypeInfer.modules.test.cpp @@ -13,6 +13,8 @@ using namespace Luau; +LUAU_FASTFLAG(LuauTableSubtypingVariance2) + TEST_SUITE_BEGIN("TypeInferModules"); TEST_CASE_FIXTURE(Fixture, "require") @@ -268,8 +270,6 @@ function x:Destroy(): () end TEST_CASE_FIXTURE(Fixture, "do_not_modify_imported_types_2") { - ScopedFastFlag immutableTypes{"LuauImmutableTypes", true}; - fileResolver.source["game/A"] = R"( export type Type = { x: { a: number } } return {} @@ -288,8 +288,6 @@ type Rename = typeof(x.x) TEST_CASE_FIXTURE(Fixture, "do_not_modify_imported_types_3") { - ScopedFastFlag immutableTypes{"LuauImmutableTypes", true}; - fileResolver.source["game/A"] = R"( local y = setmetatable({}, {}) export type Type = { x: typeof(y) } @@ -307,4 +305,83 @@ type Rename = typeof(x.x) LUAU_REQUIRE_NO_ERRORS(result); } +TEST_CASE_FIXTURE(Fixture, "module_type_conflict") +{ + ScopedFastFlag luauTypeMismatchModuleName{"LuauTypeMismatchModuleName", true}; + + fileResolver.source["game/A"] = R"( +export type T = { x: number } +return {} + )"; + + fileResolver.source["game/B"] = R"( +export type T = { x: string } +return {} + )"; + + fileResolver.source["game/C"] = R"( +local A = require(game.A) +local B = require(game.B) +local a: A.T = { x = 2 } +local b: B.T = a + )"; + + CheckResult result = frontend.check("game/C"); + LUAU_REQUIRE_ERROR_COUNT(1, result); + + if (FFlag::LuauTableSubtypingVariance2) + { + CHECK_EQ(toString(result.errors[0]), R"(Type 'T' from 'game/A' could not be converted into 'T' from 'game/B' +caused by: + Property 'x' is not compatible. Type 'number' could not be converted into 'string')"); + } + else + { + CHECK_EQ(toString(result.errors[0]), "Type 'T' from 'game/A' could not be converted into 'T' from 'game/B'"); + } +} + +TEST_CASE_FIXTURE(Fixture, "module_type_conflict_instantiated") +{ + ScopedFastFlag luauTypeMismatchModuleName{"LuauTypeMismatchModuleName", true}; + + fileResolver.source["game/A"] = R"( +export type Wrap = { x: T } +return {} + )"; + + fileResolver.source["game/B"] = R"( +local A = require(game.A) +export type T = A.Wrap +return {} + )"; + + fileResolver.source["game/C"] = R"( +local A = require(game.A) +export type T = A.Wrap +return {} + )"; + + fileResolver.source["game/D"] = R"( +local A = require(game.B) +local B = require(game.C) +local a: A.T = { x = 2 } +local b: B.T = a + )"; + + CheckResult result = frontend.check("game/D"); + LUAU_REQUIRE_ERROR_COUNT(1, result); + + if (FFlag::LuauTableSubtypingVariance2) + { + CHECK_EQ(toString(result.errors[0]), R"(Type 'T' from 'game/B' could not be converted into 'T' from 'game/C' +caused by: + Property 'x' is not compatible. Type 'number' could not be converted into 'string')"); + } + else + { + CHECK_EQ(toString(result.errors[0]), "Type 'T' from 'game/B' could not be converted into 'T' from 'game/C'"); + } +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.operators.test.cpp b/tests/TypeInfer.operators.test.cpp index baa25978..6a8a9d93 100644 --- a/tests/TypeInfer.operators.test.cpp +++ b/tests/TypeInfer.operators.test.cpp @@ -756,4 +756,30 @@ TEST_CASE_FIXTURE(Fixture, "refine_and_or") CHECK_EQ("number", toString(requireType("u"))); } +TEST_CASE_FIXTURE(Fixture, "infer_any_in_all_modes_when_lhs_is_unknown") +{ + ScopedFastFlag sff{"LuauDecoupleOperatorInferenceFromUnifiedTypeInference", true}; + + CheckResult result = check(Mode::Strict, R"( + local function f(x, y) + return x + y + end + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ(toString(result.errors[0]), "Unknown type used in + operation; consider adding a type annotation to 'x'"); + + result = check(Mode::Nonstrict, R"( + local function f(x, y) + return x + y + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + // When type inference is unified, we could add an assertion that + // the strict and nonstrict types are equivalent. This isn't actually + // the case right now, though. +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.refinements.test.cpp b/tests/TypeInfer.refinements.test.cpp index 9b347921..cddeab6e 100644 --- a/tests/TypeInfer.refinements.test.cpp +++ b/tests/TypeInfer.refinements.test.cpp @@ -435,7 +435,6 @@ TEST_CASE_FIXTURE(Fixture, "term_is_equal_to_an_lvalue") { ScopedFastFlag sff[] = { {"LuauDiscriminableUnions2", true}, - {"LuauSingletonTypes", true}, }; CheckResult result = check(R"( @@ -1002,8 +1001,6 @@ TEST_CASE_FIXTURE(Fixture, "discriminate_from_truthiness_of_x") { ScopedFastFlag sff[] = { {"LuauDiscriminableUnions2", true}, - {"LuauParseSingletonTypes", true}, - {"LuauSingletonTypes", true}, }; CheckResult result = check(R"( @@ -1028,8 +1025,6 @@ TEST_CASE_FIXTURE(Fixture, "discriminate_tag") { ScopedFastFlag sff[] = { {"LuauDiscriminableUnions2", true}, - {"LuauParseSingletonTypes", true}, - {"LuauSingletonTypes", true}, }; CheckResult result = check(R"( @@ -1066,8 +1061,6 @@ TEST_CASE_FIXTURE(Fixture, "and_or_peephole_refinement") TEST_CASE_FIXTURE(Fixture, "narrow_boolean_to_true_or_false") { ScopedFastFlag sff[]{ - {"LuauParseSingletonTypes", true}, - {"LuauSingletonTypes", true}, {"LuauDiscriminableUnions2", true}, {"LuauAssertStripsFalsyTypes", true}, }; @@ -1091,8 +1084,6 @@ TEST_CASE_FIXTURE(Fixture, "narrow_boolean_to_true_or_false") TEST_CASE_FIXTURE(Fixture, "discriminate_on_properties_of_disjoint_tables_where_that_property_is_true_or_false") { ScopedFastFlag sff[]{ - {"LuauParseSingletonTypes", true}, - {"LuauSingletonTypes", true}, {"LuauDiscriminableUnions2", true}, {"LuauAssertStripsFalsyTypes", true}, }; @@ -1134,8 +1125,6 @@ TEST_CASE_FIXTURE(RefinementClassFixture, "discriminate_from_isa_of_x") { ScopedFastFlag sff[] = { {"LuauDiscriminableUnions2", true}, - {"LuauParseSingletonTypes", true}, - {"LuauSingletonTypes", true}, }; CheckResult result = check(R"( diff --git a/tests/TypeInfer.singletons.test.cpp b/tests/TypeInfer.singletons.test.cpp index 7f8d8fec..d39341ea 100644 --- a/tests/TypeInfer.singletons.test.cpp +++ b/tests/TypeInfer.singletons.test.cpp @@ -13,11 +13,6 @@ TEST_SUITE_BEGIN("TypeSingletons"); TEST_CASE_FIXTURE(Fixture, "bool_singletons") { - ScopedFastFlag sffs[] = { - {"LuauSingletonTypes", true}, - {"LuauParseSingletonTypes", true}, - }; - CheckResult result = check(R"( local a: true = true local b: false = false @@ -28,11 +23,6 @@ TEST_CASE_FIXTURE(Fixture, "bool_singletons") TEST_CASE_FIXTURE(Fixture, "string_singletons") { - ScopedFastFlag sffs[] = { - {"LuauSingletonTypes", true}, - {"LuauParseSingletonTypes", true}, - }; - CheckResult result = check(R"( local a: "foo" = "foo" local b: "bar" = "bar" @@ -43,11 +33,6 @@ TEST_CASE_FIXTURE(Fixture, "string_singletons") TEST_CASE_FIXTURE(Fixture, "bool_singletons_mismatch") { - ScopedFastFlag sffs[] = { - {"LuauSingletonTypes", true}, - {"LuauParseSingletonTypes", true}, - }; - CheckResult result = check(R"( local a: true = false )"); @@ -58,11 +43,6 @@ TEST_CASE_FIXTURE(Fixture, "bool_singletons_mismatch") TEST_CASE_FIXTURE(Fixture, "string_singletons_mismatch") { - ScopedFastFlag sffs[] = { - {"LuauSingletonTypes", true}, - {"LuauParseSingletonTypes", true}, - }; - CheckResult result = check(R"( local a: "foo" = "bar" )"); @@ -73,11 +53,6 @@ TEST_CASE_FIXTURE(Fixture, "string_singletons_mismatch") TEST_CASE_FIXTURE(Fixture, "string_singletons_escape_chars") { - ScopedFastFlag sffs[] = { - {"LuauSingletonTypes", true}, - {"LuauParseSingletonTypes", true}, - }; - CheckResult result = check(R"( local a: "\n" = "\000\r" )"); @@ -88,11 +63,6 @@ TEST_CASE_FIXTURE(Fixture, "string_singletons_escape_chars") TEST_CASE_FIXTURE(Fixture, "bool_singleton_subtype") { - ScopedFastFlag sffs[] = { - {"LuauSingletonTypes", true}, - {"LuauParseSingletonTypes", true}, - }; - CheckResult result = check(R"( local a: true = true local b: boolean = a @@ -103,11 +73,6 @@ TEST_CASE_FIXTURE(Fixture, "bool_singleton_subtype") TEST_CASE_FIXTURE(Fixture, "string_singleton_subtype") { - ScopedFastFlag sffs[] = { - {"LuauSingletonTypes", true}, - {"LuauParseSingletonTypes", true}, - }; - CheckResult result = check(R"( local a: "foo" = "foo" local b: string = a @@ -118,11 +83,6 @@ TEST_CASE_FIXTURE(Fixture, "string_singleton_subtype") TEST_CASE_FIXTURE(Fixture, "function_call_with_singletons") { - ScopedFastFlag sffs[] = { - {"LuauSingletonTypes", true}, - {"LuauParseSingletonTypes", true}, - }; - CheckResult result = check(R"( function f(a: true, b: "foo") end f(true, "foo") @@ -133,11 +93,6 @@ TEST_CASE_FIXTURE(Fixture, "function_call_with_singletons") TEST_CASE_FIXTURE(Fixture, "function_call_with_singletons_mismatch") { - ScopedFastFlag sffs[] = { - {"LuauSingletonTypes", true}, - {"LuauParseSingletonTypes", true}, - }; - CheckResult result = check(R"( function f(a: true, b: "foo") end f(true, "bar") @@ -149,11 +104,6 @@ TEST_CASE_FIXTURE(Fixture, "function_call_with_singletons_mismatch") TEST_CASE_FIXTURE(Fixture, "overloaded_function_call_with_singletons") { - ScopedFastFlag sffs[] = { - {"LuauSingletonTypes", true}, - {"LuauParseSingletonTypes", true}, - }; - CheckResult result = check(R"( function f(a, b) end local g : ((true, string) -> ()) & ((false, number) -> ()) = (f::any) @@ -166,11 +116,6 @@ TEST_CASE_FIXTURE(Fixture, "overloaded_function_call_with_singletons") TEST_CASE_FIXTURE(Fixture, "overloaded_function_call_with_singletons_mismatch") { - ScopedFastFlag sffs[] = { - {"LuauSingletonTypes", true}, - {"LuauParseSingletonTypes", true}, - }; - CheckResult result = check(R"( function f(a, b) end local g : ((true, string) -> ()) & ((false, number) -> ()) = (f::any) @@ -184,11 +129,6 @@ TEST_CASE_FIXTURE(Fixture, "overloaded_function_call_with_singletons_mismatch") TEST_CASE_FIXTURE(Fixture, "enums_using_singletons") { - ScopedFastFlag sffs[] = { - {"LuauSingletonTypes", true}, - {"LuauParseSingletonTypes", true}, - }; - CheckResult result = check(R"( type MyEnum = "foo" | "bar" | "baz" local a : MyEnum = "foo" @@ -201,11 +141,6 @@ TEST_CASE_FIXTURE(Fixture, "enums_using_singletons") TEST_CASE_FIXTURE(Fixture, "enums_using_singletons_mismatch") { - ScopedFastFlag sffs[] = { - {"LuauSingletonTypes", true}, - {"LuauParseSingletonTypes", true}, - }; - CheckResult result = check(R"( type MyEnum = "foo" | "bar" | "baz" local a : MyEnum = "bang" @@ -218,11 +153,6 @@ TEST_CASE_FIXTURE(Fixture, "enums_using_singletons_mismatch") TEST_CASE_FIXTURE(Fixture, "enums_using_singletons_subtyping") { - ScopedFastFlag sffs[] = { - {"LuauSingletonTypes", true}, - {"LuauParseSingletonTypes", true}, - }; - CheckResult result = check(R"( type MyEnum1 = "foo" | "bar" type MyEnum2 = MyEnum1 | "baz" @@ -237,8 +167,6 @@ TEST_CASE_FIXTURE(Fixture, "enums_using_singletons_subtyping") TEST_CASE_FIXTURE(Fixture, "tagged_unions_using_singletons") { ScopedFastFlag sffs[] = { - {"LuauSingletonTypes", true}, - {"LuauParseSingletonTypes", true}, {"LuauExpectedTypesOfProperties", true}, }; @@ -257,11 +185,6 @@ TEST_CASE_FIXTURE(Fixture, "tagged_unions_using_singletons") TEST_CASE_FIXTURE(Fixture, "tagged_unions_using_singletons_mismatch") { - ScopedFastFlag sffs[] = { - {"LuauSingletonTypes", true}, - {"LuauParseSingletonTypes", true}, - }; - CheckResult result = check(R"( type Dog = { tag: "Dog", howls: boolean } type Cat = { tag: "Cat", meows: boolean } @@ -274,11 +197,6 @@ TEST_CASE_FIXTURE(Fixture, "tagged_unions_using_singletons_mismatch") TEST_CASE_FIXTURE(Fixture, "tagged_unions_immutable_tag") { - ScopedFastFlag sffs[] = { - {"LuauSingletonTypes", true}, - {"LuauParseSingletonTypes", true}, - }; - CheckResult result = check(R"( type Dog = { tag: "Dog", howls: boolean } type Cat = { tag: "Cat", meows: boolean } @@ -292,10 +210,6 @@ TEST_CASE_FIXTURE(Fixture, "tagged_unions_immutable_tag") TEST_CASE_FIXTURE(Fixture, "table_properties_singleton_strings") { - ScopedFastFlag sffs[] = { - {"LuauParseSingletonTypes", true}, - }; - CheckResult result = check(R"( --!strict type T = { @@ -320,10 +234,6 @@ TEST_CASE_FIXTURE(Fixture, "table_properties_singleton_strings") } TEST_CASE_FIXTURE(Fixture, "table_properties_singleton_strings_mismatch") { - ScopedFastFlag sffs[] = { - {"LuauParseSingletonTypes", true}, - }; - CheckResult result = check(R"( --!strict type T = { @@ -341,10 +251,6 @@ TEST_CASE_FIXTURE(Fixture, "table_properties_singleton_strings_mismatch") TEST_CASE_FIXTURE(Fixture, "table_properties_alias_or_parens_is_indexer") { - ScopedFastFlag sffs[] = { - {"LuauParseSingletonTypes", true}, - }; - CheckResult result = check(R"( --!strict type S = "bar" @@ -367,8 +273,7 @@ TEST_CASE_FIXTURE(Fixture, "table_properties_alias_or_parens_is_indexer") TEST_CASE_FIXTURE(Fixture, "table_properties_type_error_escapes") { - ScopedFastFlag sffs[] = { - {"LuauParseSingletonTypes", true}, + ScopedFastFlag sffs[]{ {"LuauUnsealedTableLiteral", true}, }; @@ -386,8 +291,6 @@ TEST_CASE_FIXTURE(Fixture, "table_properties_type_error_escapes") TEST_CASE_FIXTURE(Fixture, "error_detailed_tagged_union_mismatch_string") { ScopedFastFlag sffs[] = { - {"LuauSingletonTypes", true}, - {"LuauParseSingletonTypes", true}, {"LuauExpectedTypesOfProperties", true}, }; @@ -409,8 +312,6 @@ caused by: TEST_CASE_FIXTURE(Fixture, "error_detailed_tagged_union_mismatch_bool") { ScopedFastFlag sffs[] = { - {"LuauSingletonTypes", true}, - {"LuauParseSingletonTypes", true}, {"LuauExpectedTypesOfProperties", true}, }; @@ -432,8 +333,6 @@ caused by: TEST_CASE_FIXTURE(Fixture, "if_then_else_expression_singleton_options") { ScopedFastFlag sffs[] = { - {"LuauSingletonTypes", true}, - {"LuauParseSingletonTypes", true}, {"LuauExpectedTypesOfProperties", true}, }; @@ -451,7 +350,6 @@ local a: Animal = if true then { tag = 'cat', catfood = 'something' } else { tag TEST_CASE_FIXTURE(Fixture, "widen_the_supertype_if_it_is_free_and_subtype_has_singleton") { ScopedFastFlag sff[]{ - {"LuauSingletonTypes", true}, {"LuauEqConstraint", true}, {"LuauDiscriminableUnions2", true}, {"LuauWidenIfSupertypeIsFree2", true}, @@ -477,8 +375,6 @@ TEST_CASE_FIXTURE(Fixture, "widen_the_supertype_if_it_is_free_and_subtype_has_si TEST_CASE_FIXTURE(Fixture, "return_type_of_f_is_not_widened") { ScopedFastFlag sff[]{ - {"LuauParseSingletonTypes", true}, - {"LuauSingletonTypes", true}, {"LuauDiscriminableUnions2", true}, {"LuauEqConstraint", true}, {"LuauWidenIfSupertypeIsFree2", true}, @@ -504,8 +400,6 @@ TEST_CASE_FIXTURE(Fixture, "return_type_of_f_is_not_widened") TEST_CASE_FIXTURE(Fixture, "widening_happens_almost_everywhere") { ScopedFastFlag sff[]{ - {"LuauParseSingletonTypes", true}, - {"LuauSingletonTypes", true}, {"LuauWidenIfSupertypeIsFree2", true}, }; @@ -521,8 +415,6 @@ TEST_CASE_FIXTURE(Fixture, "widening_happens_almost_everywhere") TEST_CASE_FIXTURE(Fixture, "widening_happens_almost_everywhere_except_for_tables") { ScopedFastFlag sff[]{ - {"LuauParseSingletonTypes", true}, - {"LuauSingletonTypes", true}, {"LuauDiscriminableUnions2", true}, {"LuauWidenIfSupertypeIsFree2", true}, }; @@ -551,8 +443,6 @@ TEST_CASE_FIXTURE(Fixture, "widening_happens_almost_everywhere_except_for_tables TEST_CASE_FIXTURE(Fixture, "table_insert_with_a_singleton_argument") { ScopedFastFlag sff[]{ - {"LuauParseSingletonTypes", true}, - {"LuauSingletonTypes", true}, {"LuauWidenIfSupertypeIsFree2", true}, }; @@ -577,8 +467,6 @@ TEST_CASE_FIXTURE(Fixture, "table_insert_with_a_singleton_argument") TEST_CASE_FIXTURE(Fixture, "functions_are_not_to_be_widened") { ScopedFastFlag sff[]{ - {"LuauParseSingletonTypes", true}, - {"LuauSingletonTypes", true}, {"LuauWidenIfSupertypeIsFree2", true}, }; @@ -595,7 +483,6 @@ TEST_CASE_FIXTURE(Fixture, "indexing_on_string_singletons") { ScopedFastFlag sff[]{ {"LuauDiscriminableUnions2", true}, - {"LuauSingletonTypes", true}, }; CheckResult result = check(R"( @@ -614,7 +501,6 @@ TEST_CASE_FIXTURE(Fixture, "indexing_on_union_of_string_singletons") { ScopedFastFlag sff[]{ {"LuauDiscriminableUnions2", true}, - {"LuauSingletonTypes", true}, }; CheckResult result = check(R"( @@ -633,7 +519,6 @@ TEST_CASE_FIXTURE(Fixture, "taking_the_length_of_string_singleton") { ScopedFastFlag sff[]{ {"LuauDiscriminableUnions2", true}, - {"LuauSingletonTypes", true}, }; CheckResult result = check(R"( @@ -652,7 +537,6 @@ TEST_CASE_FIXTURE(Fixture, "taking_the_length_of_union_of_string_singleton") { ScopedFastFlag sff[]{ {"LuauDiscriminableUnions2", true}, - {"LuauSingletonTypes", true}, }; CheckResult result = check(R"( diff --git a/tests/TypeInfer.tables.test.cpp b/tests/TypeInfer.tables.test.cpp index 91140aaa..0cc12d19 100644 --- a/tests/TypeInfer.tables.test.cpp +++ b/tests/TypeInfer.tables.test.cpp @@ -2078,6 +2078,44 @@ caused by: Property '__call' is not compatible. Type '(a, b) -> ()' could not be converted into '(a) -> ()'; different number of generic type parameters)"); } +TEST_CASE_FIXTURE(Fixture, "error_detailed_indexer_key") +{ + ScopedFastFlag luauTableSubtypingVariance2{"LuauTableSubtypingVariance2", true}; // Only for new path + ScopedFastFlag luauExtendedIndexerError{"LuauExtendedIndexerError", true}; + + CheckResult result = check(R"( + type A = { [number]: string } + type B = { [string]: string } + + local a: A = { 'a', 'b' } + local b: B = a + )"); + + LUAU_REQUIRE_ERRORS(result); + CHECK_EQ(toString(result.errors[0]), R"(Type 'A' could not be converted into 'B' +caused by: + Property '[indexer key]' is not compatible. Type 'number' could not be converted into 'string')"); +} + +TEST_CASE_FIXTURE(Fixture, "error_detailed_indexer_value") +{ + ScopedFastFlag luauTableSubtypingVariance2{"LuauTableSubtypingVariance2", true}; // Only for new path + ScopedFastFlag luauExtendedIndexerError{"LuauExtendedIndexerError", true}; + + CheckResult result = check(R"( + type A = { [number]: number } + type B = { [number]: string } + + local a: A = { 1, 2, 3 } + local b: B = a + )"); + + LUAU_REQUIRE_ERRORS(result); + CHECK_EQ(toString(result.errors[0]), R"(Type 'A' could not be converted into 'B' +caused by: + Property '[indexer value]' is not compatible. Type 'number' could not be converted into 'string')"); +} + TEST_CASE_FIXTURE(Fixture, "explicitly_typed_table") { ScopedFastFlag sffs[]{ diff --git a/tests/TypeInfer.unionTypes.test.cpp b/tests/TypeInfer.unionTypes.test.cpp index ad1e31e5..68b7c4fb 100644 --- a/tests/TypeInfer.unionTypes.test.cpp +++ b/tests/TypeInfer.unionTypes.test.cpp @@ -296,6 +296,7 @@ return f() REQUIRE(acm); CHECK_EQ(1, acm->expected); CHECK_EQ(0, acm->actual); + CHECK_FALSE(acm->isVariadic); } TEST_CASE_FIXTURE(Fixture, "optional_field_access_error") From 75bccce3db6eb63404a9bd896f99d8ab2ab8aafe Mon Sep 17 00:00:00 2001 From: Andy Friesen Date: Tue, 29 Mar 2022 12:37:14 -0700 Subject: [PATCH 208/399] RFC: Lower Bounds Calculation (#388) Co-authored-by: vegorov-rbx <75688451+vegorov-rbx@users.noreply.github.com> --- rfcs/lower-bounds-calculation.md | 217 +++++++++++++++++++++++++++++++ 1 file changed, 217 insertions(+) create mode 100644 rfcs/lower-bounds-calculation.md diff --git a/rfcs/lower-bounds-calculation.md b/rfcs/lower-bounds-calculation.md new file mode 100644 index 00000000..a1793884 --- /dev/null +++ b/rfcs/lower-bounds-calculation.md @@ -0,0 +1,217 @@ +# Lower Bounds Calculation + +## Summary + +We propose adapting lower bounds calculation from Pierce's Local Type Inference paper into the Luau type inference algorithm. + +https://www.cis.upenn.edu/~bcpierce/papers/lti-toplas.pdf + +## Motivation + +There are a number of important scenarios that occur where Luau cannot infer a sensible type without annotations. + +Many of these revolve around type variables that occur in contravariant positions. + +### Function Return Types + +A very common thing to write in Luau is a function to try to find something in some data structure. These functions habitually return the relevant datum when it is successfully found, or `nil` in the case that it cannot. For instance: + +```lua +-- A.lua +function find_first_if(vec, f) + for i, e in ipairs(vec) do + if f(e) then + return i + end + end + + return nil +end +``` + +This function has two `return` statements: One returns `number` and the other `nil`. Today, Luau flags this as an error. We ask authors to add a return annotation to make this error go away. + +We would like to automatically infer `find_first_if : ({T}, (T) -> boolean) -> number?`. + +Higher order functions also present a similar problem. + +```lua +-- B.lua +function foo(f) + f(5) + f("string") +end +``` + +There is nothing wrong with the implementation of `foo` here, but Luau fails to typecheck it all the same because `f` is used in an inconsistent way. This too can be worked around by introducing a type annotation for `f`. + +The fact that the return type of `f` is never used confounds things a little, but for now it would be a big improvement if we inferred `f : ((number | string) -> T...) -> ()`. + +## Design + +We introduce a new kind of TypeVar, `ConstrainedTypeVar` to represent a TypeVar whose lower bounds are known. We will never expose syntax for a user to write these types: They only temporarily exist as type inference is being performed. + +When unifying some type with a `ConstrainedTypeVar` we _broaden_ the set of constraints that can be placed upon it. + +It may help to realize that what we have been doing up until now has been _upper bounds calculation_. + +When we `quantify` a function, we will _normalize_ each type and convert each `ConstrainedTypeVar` into a `UnionTypeVar`. + +### Normalization + +When computing lower bounds, we need to have some process by which we reduce types down to a minimal shape and canonicalize them, if only to have a clean way to flush out degenerate unions like `A | A`. Normalization is about reducing union and intersection types to a minimal, canonicalizable shape. + +A normalized union is one where there do not exist two branches on the union where one is a subtype of the other. It is quite straightforward to implement. + +A normalized intersection is a little bit more complicated: + +1. The tables of an intersection are always combined into a single table. Coincident properties are merged into intersections of their own. + * eg `normalize({x: number, y: string} & {y: number, z: number}) == {x: number, y: string & number, z: number}` + * This is recursive. eg `normalize({x: {y: number}} & {x: {y: string}}) == {x: {y: number & string}}` +1. If two functions in the intersection have a subtyping relationship, the normalization results only in the super-type-most function. (more on function subtyping later) + +### Function subtyping relationships + +If we are going to infer intersections of functions, then we need to be very careful about keeping combinatorics under control. We therefore need to be very deliberate about what subtyping rules we have for functions of differing arity. We have some important requirements: + +* We'd like some way to canonicalize intersections of functions, and yet +* optional function arguments are a great feature that we don't want to break + +A very important use case for us is the case where the user is providing a callback to some higher-order function, and that function will be invoked with extra arguments that the original customer doesn't actually care about. For example: + +```lua +-- C.lua +function map_array(arr, f) + local result = {} + for i, e in ipairs(arr) do + table.insert(result, f(e, i, arr)) + end + return result +end + +local example = {1, 2, 3, 4} +local example_result = map_array(example, function(i) return i * 2 end) +``` + +This function mirrors the actual `Array.map` function in JavaScript. It is very frequent for users of this function to provide a lambda that only accepts one argument. It would be annoying for callers to be forced to provide a lambda that accepts two unused arguments. This obviously becomes even worse if the function later changes to provide yet more optional information to the callback. + +This use case is very important for Roblox, as we have many APIs that accept callbacks. Implementors of those callbacks frequently omit arguments that they don't care about. + +Here is an example straight out of the Roblox developer documentation. ([full example here](https://developer.roblox.com/en-us/api-reference/event/BasePart/Touched)) + +```lua +-- D.lua +local part = script.Parent + +local function blink() + -- ... +end + +part.Touched:Connect(blink) +``` + +The `Touched` event actually passes a single argument: the part that touched the `Instance` in question. In this example, it is omitted from the callback handler. + +We therefore want _oversaturation_ of a function to be allowed, but this combines with optional function arguments to create a problem with soundness. Consider the following: + +```lua +-- E.lua +type Callback = (Instance) -> () + +local cb: Callback +function register_callback(c: Callback) + cb = c +end + +function invoke_callback(i: Instance) + cb(i) +end + +--- + +function bad_callback(x: number?) +end + +local obscured: () -> () = bad_callback + +register_callback(obscured) + +function good_callback() +end + +register_callback(good_callback) +``` + +The problem we run into is, if we allow the subtyping rule `(T?) -> () <: () -> ()` and also allow oversaturation of a function, it becomes easy to obscure an argument type and pass the wrong type of value to it. + +Next, consider the following type alias + +```lua +-- F.lua +type OldFunctionType = (any, any) -> any +type NewFunctionType = (any) -> any +type FunctionType = OldFunctionType & NewFunctionType +``` + +If we have a subtyping rule `(T0..TN) <: (T0..TN-1)` to permit the function subtyping relationship `(T0..TN-1) -> R <: (T0..TN) -> R`, then the above type alias normalizes to `(any) -> any`. In order to call the two-argument variation, we would need to permit oversaturation, which runs afoul of the soundness hole from the previous example. + +We need a solution here. + +To resolve this, let's reframe things in simpler terms: + +If there is never a subtyping relationship between packs of different length, then we don't have any soundness issues, but we find ourselves unable to register `good_callback`. + +To resolve _that_, consider that we are in truth being a bit hasty when we say `good_callback : () -> ()`. We can pass any number of arguments to this function safely. We could choose to type `good_callback : () -> () & (any) -> () & (any, any) -> () & ...`. Luau already has syntax for this particular sort of infinite intersection: `good_callback : (any...) -> ()`. + +So, we propose some different inference rules for functions: + +1. The AST fragment `function(arg0..argN) ... end` is typed `(T0..TN, any...) -> R` where `arg0..argN : T0..TN` and `R` is the inferred return type of the function body. Function statements are inferred the same way. +1. Type annotations are unchanged. `() -> ()` is still a nullary function. + +For reference, the subtyping rules for unions and functions are unchanged. We include them here for clarity. + +1. `A <: A | B` +1. `B <: A | B` +1. `A | B <: T` if `A <: T` or `B <: T` +1. `T -> R <: U -> S` if `U <: T` and `R <: S` + +We propose new subtyping rules for type packs: + +1. `(T0..TN) <: (U0..UN)` if, for each `T` and `U`, `T <: U` +1. `(U...)` is the same as `() | (U) | (U, U) | (U, U, U) | ...`, therefore +1. `(T0..TN) <: (U...)` if for each `T`, `T <: U`, therefore +1. `(U...) -> R <: (T0..TN) -> R` if for each `T`, `T <: U` + +The important difference is that we remove all subtyping rules that mention options. Functions of different arities are no longer considered subtypes of one another. Optional function arguments are still allowed, but function as a feature of function calls. + +Under these rules, functions of different arities can never be converted to one another, but actual functions are known to be safe to oversaturate with anything, and so gain a type that says so. + +Under these subtyping rules, snippets `C.lua` and `D.lua`, check the way we want: literal functions are implicitly safe to oversaturate, so it is fine to cast them as the necessary callback function type. + +`E.lua` also typechecks the way we need it to: `(Instance) -> () ()` and so `obscured` cannot receive the value `bad_callback`, which prevents it from being passed to `register_callback`. However, `good_callback : (any...) -> ()` and `(any...) -> () <: (Instance) -> ()` and so it is safe to register `good_callback`. + +Snippet `F.lua` is also fixed with this ruleset: There is no subtyping relationship between `(any) -> ()` and `(any, any) -> ()`, so the intersection is not combined under normalization. + +This works, but itself creates some small problems that we need to resolve: + +First, the `...` symbol still needs to be unavailable for functions that have been given this implicit `...any` type. This is actually taken care of in the Luau parser, so no code change is required. + +Secondly, we do not want to silently allow oversaturation of direct calls to a function if we know that the arguments will be ignored. We need to treat these variadic packs differently when unifying for function calls. + +Thirdly, we don't want to display this variadic in the signature if the author doesn't expect to see it. + +We solve these issues by adding a property `bool VariadicTypePack::hidden` to the implementation and switching on it in the above scenarios. The implementation is relatively straightforward for all 3 cases. + +## Drawbacks + +There is a potential cause for concern that we will be inferring unions of functions in cases where we previously did not. Unions are known to be potential sources of performance issues. One possibility is to allow Luau to be less intelligent and have it "give up" and produce less precise types. This would come at the cost of accuracy and soundness. + +If we allow functions to be oversaturated, we are going to miss out on opportunities to warn the user about legitimate problems with their program. I think we will have to work out some kind of special logic to detect when we are oversaturating a function whose exact definition is known and warn on that. + +Allowing indirect function calls to be oversaturated with `nil` values only should be safe, but a little bit unfortunate. As long as we statically know for certain that `nil` is actually a permissible value for that argument position, it should be safe. + +## Alternatives + +If we are willing to sacrifice soundness, we could adopt success typing and come up with an inference algorithm that produces less precise type information. + +We could also technically choose to do nothing, but this has some unpalatable consequences: Something I would like to do in the near future is to have the inference algorithm assume the same `self` type for all methods of a table. This will make inference of common OO patterns dramatically more intuitive and ergonomic, but inference of polymorphic methods requires some kind of lower bounds calculation to work correctly. From af64680a5ef527ae59adbf24fd8581e2c80575ad Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Tue, 29 Mar 2022 16:58:59 -0700 Subject: [PATCH 209/399] Mark singleton types and unsealed table literals RFCs as implemented (#438) --- rfcs/STATUS.md | 13 ++++++------- rfcs/function-bit32-countlz-countrz.md | 4 ++-- rfcs/function-coroutine-close.md | 4 ++-- rfcs/syntax-safe-navigation-operator.md | 2 ++ rfcs/syntax-singleton-types.md | 2 ++ rfcs/syntax-type-ascription-bidi.md | 4 ++-- rfcs/unsealed-table-assign-optional-property.md | 2 ++ rfcs/unsealed-table-literals.md | 4 +++- 8 files changed, 21 insertions(+), 14 deletions(-) diff --git a/rfcs/STATUS.md b/rfcs/STATUS.md index 93a09ece..e3e227a0 100644 --- a/rfcs/STATUS.md +++ b/rfcs/STATUS.md @@ -17,17 +17,10 @@ This document tracks unimplemented RFCs. ## Sealed/unsealed typing changes -[RFC: Unsealed table literals](https://github.com/Roblox/luau/blob/master/rfcs/unsealed-table-literals.md) | [RFC: Only strip optional properties from unsealed tables during subtyping](https://github.com/Roblox/luau/blob/master/rfcs/unsealed-table-subtyping-strips-optional-properties.md) **Status**: Implemented but not fully rolled out yet. -## Singleton types - -[RFC: Singleton types](https://github.com/Roblox/luau/blob/master/rfcs/syntax-singleton-types.md) - -**Status**: Implemented but not fully rolled out yet. - ## Safe navigation operator [RFC: Safe navigation postfix operator (?)](https://github.com/Roblox/luau/blob/master/rfcs/syntax-safe-navigation-operator.md) @@ -47,3 +40,9 @@ This document tracks unimplemented RFCs. [RFC: Generalized iteration](https://github.com/Roblox/luau/blob/master/rfcs/generalized-iteration.md) **Status**: Needs implementation + +## Lower Bounds Calculation + +[RFC: Lower bounds calculation](https://github.com/Roblox/luau/blob/master/rfcs/lower-bounds-calculation.md) + +**Status**: Needs implementation diff --git a/rfcs/function-bit32-countlz-countrz.md b/rfcs/function-bit32-countlz-countrz.md index d2439f72..b4ccb197 100644 --- a/rfcs/function-bit32-countlz-countrz.md +++ b/rfcs/function-bit32-countlz-countrz.md @@ -1,11 +1,11 @@ # bit32.countlz/countrz +**Status**: Implemented + ## Summary Add bit32.countlz (count left zeroes) and bit32.countrz (count right zeroes) to accelerate bit scanning -**Status**: Implemented - ## Motivation All CPUs have instructions to determine the position of first/last set bit in an integer. These instructions have a variety of uses, the popular ones being: diff --git a/rfcs/function-coroutine-close.md b/rfcs/function-coroutine-close.md index 6def1533..b9ffbf6f 100644 --- a/rfcs/function-coroutine-close.md +++ b/rfcs/function-coroutine-close.md @@ -1,11 +1,11 @@ # coroutine.close +**Status**: Implemented + ## Summary Add `coroutine.close` function from Lua 5.4 that takes a suspended coroutine and makes it "dead" (non-runnable). -**Status**: Implemented - ## Motivation When implementing various higher level objects on top of coroutines, such as promises, it can be useful to cancel the coroutine execution externally - when the caller is not diff --git a/rfcs/syntax-safe-navigation-operator.md b/rfcs/syntax-safe-navigation-operator.md index c98f3957..11c4b37f 100644 --- a/rfcs/syntax-safe-navigation-operator.md +++ b/rfcs/syntax-safe-navigation-operator.md @@ -1,5 +1,7 @@ # Safe navigation postfix operator (?) +**Note**: We have unresolved issues with interaction between this feature and Roblox instance hierarchy. This may affect the viability of this proposal. + ## Summary Introduce syntax to navigate through `nil` values, or short-circuit with `nil` if it was encountered. diff --git a/rfcs/syntax-singleton-types.md b/rfcs/syntax-singleton-types.md index 26ea3028..2c1f5442 100644 --- a/rfcs/syntax-singleton-types.md +++ b/rfcs/syntax-singleton-types.md @@ -2,6 +2,8 @@ > Note: this RFC was adapted from an internal proposal that predates RFC process +**Status**: Implemented + ## Summary Introduce a new kind of type variable, called singleton types. They are just like normal types but has the capability to represent a constant runtime value as a type. diff --git a/rfcs/syntax-type-ascription-bidi.md b/rfcs/syntax-type-ascription-bidi.md index bf37eca2..0831aba5 100644 --- a/rfcs/syntax-type-ascription-bidi.md +++ b/rfcs/syntax-type-ascription-bidi.md @@ -1,11 +1,11 @@ # Relaxing type assertions +**Status**: Implemented + ## Summary The way `::` works today is really strange. The best solution we can come up with is to allow `::` to convert between any two related types. -**Status**: Implemented - ## Motivation Due to an accident of the implementation, the Luau `::` operator can only be used for downcasts and casts to `any`. diff --git a/rfcs/unsealed-table-assign-optional-property.md b/rfcs/unsealed-table-assign-optional-property.md index ed037b14..477399c2 100644 --- a/rfcs/unsealed-table-assign-optional-property.md +++ b/rfcs/unsealed-table-assign-optional-property.md @@ -1,5 +1,7 @@ # Unsealed table assignment creates an optional property +**Status**: Implemented + ## Summary In Luau, tables have a state, which can, among others, be "unsealed". diff --git a/rfcs/unsealed-table-literals.md b/rfcs/unsealed-table-literals.md index 320bf7ca..669b67d4 100644 --- a/rfcs/unsealed-table-literals.md +++ b/rfcs/unsealed-table-literals.md @@ -1,5 +1,7 @@ # Unsealed table literals +**Status**: Implemented + ## Summary Currently the only way to create an unsealed table is as an empty table literal `{}`. @@ -73,4 +75,4 @@ We could introduce a new table state for unsealed-but-precise tables. The trade-off is that that would be more precise, at the cost of adding user-visible complexity to the type system. -We could continue to treat array-like tables as sealed. \ No newline at end of file +We could continue to treat array-like tables as sealed. From f3ea2f96f736f646e07f03bec6797a0a41021022 Mon Sep 17 00:00:00 2001 From: Alan Jeffrey <403333+asajeffrey@users.noreply.github.com> Date: Wed, 30 Mar 2022 18:38:55 -0500 Subject: [PATCH 210/399] Recap March 2022 (#439) * March Recap Co-authored-by: Arseny Kapoulkine --- .../2022-03-31-luau-recap-march-2022.md | 109 ++++++++++++++++++ 1 file changed, 109 insertions(+) create mode 100644 docs/_posts/2022-03-31-luau-recap-march-2022.md diff --git a/docs/_posts/2022-03-31-luau-recap-march-2022.md b/docs/_posts/2022-03-31-luau-recap-march-2022.md new file mode 100644 index 00000000..8ac88732 --- /dev/null +++ b/docs/_posts/2022-03-31-luau-recap-march-2022.md @@ -0,0 +1,109 @@ +--- +layout: single +title: "Luau Recap: March 2022" +--- + +Luau is our new language that you can read more about at [https://luau-lang.org](https://luau-lang.org). + +[Cross-posted to the [Roblox Developer Forum](https://devforum.roblox.com/t/luau-recap-march-2022/).] + +## Singleton types + +We added support for singleton types! These allow you to use string or +boolean literals in types. These types are only inhabited by the +literal, for example if a variable `x` has type `"foo"`, then `x == +"foo"` is guaranteed to be true. + +Singleton types are particularly useful when combined with union types, +for example: + +```lua +type Animals = "Dog" | "Cat" | "Bird" +``` + +or: + +```lua +type Falsey = false | nil +``` + +In particular, singleton types play well with unions of tables, +allowing tagged unions (also known as discriminated unions): + +```lua +type Ok = { type: "ok", value: T } +type Err = { type: "error", error: E } +type Result = Ok | Err + +local result: Result = ... +if result.type == "ok" then + -- result :: Ok + print(result.value) +else + -- result :: Err + error(result.error) +end +``` + +The RFC for singleton types is https://github.com/Roblox/luau/blob/master/rfcs/syntax-singleton-types.md + +## Width subtyping + +A common idiom for programming with tables is to provide a public interface type, but to keep some of the concrete implementation private, for example: + +```lua +type Interface = { + name: string, +} + +type Concrete = { + name: string, + id: number, +} +``` + +Within a module, a developer might use the concrete type, but export functions using the interface type: + +```lua +local x: Concrete = { + name = "foo", + id = 123, +} + +local function get(): Interface + return x +end +``` + +Previously examples like this did not typecheck but now they do! + +This language feature is called *width subtyping* (it allows tables to get *wider*, that is to have more properties). + +The RFC for width subtyping is https://github.com/Roblox/luau/blob/master/rfcs/sealed-table-subtyping.md + +## Typechecking improvements + + * Generic function type inference now works the same for generic types and generic type packs. + * We improved some error messages. + * There are now fewer crashes (hopefully none!) due to mutating types inside the Luau typechecker. + * We fixed a bug that could cause two incompatible copies of the same class to be created. + * Luau now copes better with cyclic metatable types (it gives a type error rather than hanging). + * Fixed a case where types are not properly bound to all of the subtype when the subtype is a union. + * We fixed a bug that confused union and intersection types of table properties. + * Functions declared as `function f(x : any)` can now be called as `f()` without a type error. + +## API improvements + + * Implement `table.clone` which takes a table and returns a new table that has the same keys/values/metatable. The cloning is shallow - if some keys refer to tables that need to be cloned, that can be done manually by modifying the resulting table. + +## Debugger improvements + + * Use the property name as the name of methods in the debugger. + +## Performance improvements + + * Optimize table rehashing (~15% faster dictionary table resize on average) + * Improve performance of freeing tables (~5% lift on some GC benchmarks) + * Improve gathering performance metrics for GC. + * Reduce stack memory reallocation. + From 06bbfd90b5d8496d3f63b082ce6dd26b7849e6e3 Mon Sep 17 00:00:00 2001 From: Alan Jeffrey <403333+asajeffrey@users.noreply.github.com> Date: Thu, 31 Mar 2022 09:31:06 -0500 Subject: [PATCH 211/399] Fix code sample in March 2022 Recap (#442) --- docs/_posts/2022-03-31-luau-recap-march-2022.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/_posts/2022-03-31-luau-recap-march-2022.md b/docs/_posts/2022-03-31-luau-recap-march-2022.md index 8ac88732..ff3a4d0f 100644 --- a/docs/_posts/2022-03-31-luau-recap-march-2022.md +++ b/docs/_posts/2022-03-31-luau-recap-march-2022.md @@ -39,7 +39,7 @@ local result: Result = ... if result.type == "ok" then -- result :: Ok print(result.value) -else +elseif result.type == "error" then -- result :: Err error(result.error) end From ba60730e0f67c3b06d209cb406fd1d632866c742 Mon Sep 17 00:00:00 2001 From: Alexander McCord <11488393+alexmccord@users.noreply.github.com> Date: Thu, 31 Mar 2022 11:54:06 -0700 Subject: [PATCH 212/399] Add documentation on singleton types and tagged unions to typecheck.md. (#440) Update the typecheck.md page to talk about singleton types and their uses, tagged unions. As a driveby, improve the documentation on type refinements. And delete the unknown symbols part, this is really dated. * Update docs/_pages/typecheck.md to fix a typo Co-authored-by: Arseny Kapoulkine --- docs/_pages/typecheck.md | 124 ++++++++++++++++++++++++++++++--------- 1 file changed, 95 insertions(+), 29 deletions(-) diff --git a/docs/_pages/typecheck.md b/docs/_pages/typecheck.md index 3580d66e..9443f112 100644 --- a/docs/_pages/typecheck.md +++ b/docs/_pages/typecheck.md @@ -31,20 +31,6 @@ foo = 1 However, given the second snippet in strict mode, the type checker would be able to infer `number` for `foo`. -## Unknown symbols - -Consider how often you're likely to assign a new value to a local variable. What if you accidentally misspelled it? Oops, it's now assigned globally and your local variable is still using the old value. - -```lua -local someLocal = 1 - -soeLocal = 2 -- the bug - -print(someLocal) -``` - -Because of this, Luau type checker currently emits an error in strict mode; use local variables instead. - ## Structural type system Luau's type system is structural by default, which is to say that we inspect the shape of two tables to see if they are similar enough. This was the obvious choice because Lua 5.1 is inherently structural. @@ -267,6 +253,23 @@ Note: it's impossible to create an intersection type of some primitive types, e. Note: Luau still does not support user-defined overloaded functions. Some of Roblox and Lua 5.1 functions have different function signature, so inherently requires overloaded functions. +## Singleton types (aka literal types) + +Luau's type system also supports singleton types, which means it's a type that represents one single value at runtime. At this time, both string and booleans are representable in types. + +> We do not currently support numbers as types. For now, this is intentional. + +```lua +local foo: "Foo" = "Foo" -- ok +local bar: "Bar" = foo -- not ok +local baz: string = foo -- ok + +local t: true = true -- ok +local f: false = false -- ok +``` + +This happens all the time, especially through [type refinements](#type-refinements) and is also incredibly useful when you want to enforce program invariants in the type system! See [tagged unions](#tagged-unions) for more information. + ## Variadic types Luau permits assigning a type to the `...` variadic symbol like any other parameter: @@ -375,22 +378,40 @@ local account: Account = Account.new("Alexander", 500) --^^^^^^^ not ok, 'Account' does not exist ``` +## Tagged unions + +Tagged unions are just union types! In particular, they're union types of tables where they have at least _some_ common properties but the structure of the tables are different enough. Here's one example: + +```lua +type Result = { type: "ok", value: T } | { type: "err", error: E } +``` + +This `Result` type can be discriminated by using type refinements on the property `type`, like so: + +```lua +if result.type == "ok" then + -- result is known to be { type: "ok", value: T } + -- and attempting to index for error here will fail + print(result.value) +elseif result.type == "err" then + -- result is known to be { type: "err", error: E } + -- and attempting to index for value here will fail + print(result.error) +end +``` + +Which works out because `value: T` exists only when `type` is in actual fact `"ok"`, and `error: E` exists only when `type` is in actual fact `"err"`. + ## Type refinements -When we check the type of a value, what we're doing is we're refining the type, hence "type refinement." Currently, the support for this is somewhat basic. +When we check the type of any lvalue (a global, a local, or a property), what we're doing is we're refining the type, hence "type refinement." The support for this is arbitrarily complex, so go crazy! -Using `type` comparison: -```lua -local stringOrNumber: string | number = "foo" +Here are all the ways you can refine: +1. Truthy test: `if x then` will refine `x` to be truthy. +2. Type guards: `if type(x) == "number" then` will refine `x` to be `number`. +3. Equality: `x == "hello"` will refine `x` to be a singleton type `"hello"`. -if type(x) == "string" then - local onlyString: string = stringOrNumber -- ok - local onlyNumber: number = stringOrNumber -- not ok -end - -local onlyString: string = stringOrNumber -- not ok -local onlyNumber: number = stringOrNumber -- not ok -``` +And they can be composed with many of `and`/`or`/`not`. `not`, just like `~=`, will flip the resulting refinements, that is `not x` will refine `x` to be falsy. Using truthy test: ```lua @@ -398,10 +419,55 @@ local maybeString: string? = nil if maybeString then local onlyString: string = maybeString -- ok + local onlyNil: nil = maybeString -- not ok +end + +if not maybeString then + local onlyString: string = maybeString -- not ok + local onlyNil: nil = maybeString -- ok end ``` -And using `assert` will work with the above type guards: +Using `type` test: +```lua +local stringOrNumber: string | number = "foo" + +if type(stringOrNumber) == "string" then + local onlyString: string = stringOrNumber -- ok + local onlyNumber: number = stringOrNumber -- not ok +end + +if type(stringOrNumber) ~= "string" then + local onlyString: string = stringOrNumber -- not ok + local onlyNumber: number = stringOrNumber -- ok +end +``` + +Using equality test: +```lua +local myString: string = f() + +if myString == "hello" then + local hello: "hello" = myString -- ok because it is absolutely "hello"! + local copy: string = myString -- ok +end +``` + +And as said earlier, we can compose as many of `and`/`or`/`not` as we wish with these refinements: +```lua +local function f(x: any, y: any) + if (x == "hello" or x == "bye") and type(y) == "string" then + -- x is of type "hello" | "bye" + -- y is of type string + end + + if not (x ~= "hi") then + -- x is of type "hi" + end +end +``` + +`assert` can also be used to refine in all the same ways: ```lua local stringOrNumber: string | number = "foo" @@ -411,7 +477,7 @@ local onlyString: string = stringOrNumber -- ok local onlyNumber: number = stringOrNumber -- not ok ``` -## Typecasts +## Type casts Expressions may be typecast using `::`. Typecasting is useful for specifying the type of an expression when the automatically inferred type is too generic. @@ -487,4 +553,4 @@ There are some caveats here though. For instance, the require path must be resol Cyclic module dependencies can cause problems for the type checker. In order to break a module dependency cycle a typecast of the module to `any` may be used: ```lua local myModule = require(MyModule) :: any -``` \ No newline at end of file +``` From 83c1c48e09105d5c83055c2827b288071fe4e055 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Thu, 31 Mar 2022 13:37:49 -0700 Subject: [PATCH 213/399] Sync to upstream/release/521 --- Analysis/src/BuiltinDefinitions.cpp | 5 +-- Analysis/src/TypeInfer.cpp | 28 +++++++++---- Ast/include/Luau/TimeTrace.h | 20 ++++----- Ast/src/Lexer.cpp | 8 +++- Ast/src/TimeTrace.cpp | 5 +-- Compiler/include/Luau/Bytecode.h | 3 +- Compiler/src/BytecodeBuilder.cpp | 2 +- VM/src/laux.cpp | 13 +----- VM/src/lbaselib.cpp | 25 ++--------- VM/src/ldebug.cpp | 18 ++------ VM/src/lmem.h | 6 +-- VM/src/ltable.cpp | 64 +++++++++++++++++++++++------ VM/src/ltablib.cpp | 30 ++++++++++++-- VM/src/lvmexecute.cpp | 6 ++- VM/src/lvmload.cpp | 13 ++---- tests/Autocomplete.test.cpp | 5 +-- tests/Conformance.test.cpp | 4 -- tests/Parser.test.cpp | 14 +++++++ tests/TypeInfer.functions.test.cpp | 41 ++++++++++++++++-- tests/conformance/nextvar.lua | 31 ++++++++++++++ 20 files changed, 222 insertions(+), 119 deletions(-) diff --git a/Analysis/src/BuiltinDefinitions.cpp b/Analysis/src/BuiltinDefinitions.cpp index bf9ef303..3895b01b 100644 --- a/Analysis/src/BuiltinDefinitions.cpp +++ b/Analysis/src/BuiltinDefinitions.cpp @@ -9,7 +9,6 @@ #include LUAU_FASTFLAG(LuauAssertStripsFalsyTypes) -LUAU_FASTFLAGVARIABLE(LuauTableCloneType, false) LUAU_FASTFLAGVARIABLE(LuauSetMetaTableArgsCheck, false) /** FIXME: Many of these type definitions are not quite completely accurate. @@ -289,9 +288,7 @@ void registerBuiltinTypes(TypeChecker& typeChecker) { // tabTy is a generic table type which we can't express via declaration syntax yet ttv->props["freeze"] = makeProperty(makeFunction(arena, std::nullopt, {tabTy}, {tabTy}), "@luau/global/table.freeze"); - - if (FFlag::LuauTableCloneType) - ttv->props["clone"] = makeProperty(makeFunction(arena, std::nullopt, {tabTy}, {tabTy}), "@luau/global/table.clone"); + ttv->props["clone"] = makeProperty(makeFunction(arena, std::nullopt, {tabTy}, {tabTy}), "@luau/global/table.clone"); attachMagicFunction(ttv->props["pack"].type, magicFunctionPack); } diff --git a/Analysis/src/TypeInfer.cpp b/Analysis/src/TypeInfer.cpp index 9965d5aa..6df6bff0 100644 --- a/Analysis/src/TypeInfer.cpp +++ b/Analysis/src/TypeInfer.cpp @@ -27,6 +27,7 @@ LUAU_FASTFLAGVARIABLE(LuauWeakEqConstraint, false) // Eventually removed as fals LUAU_FASTFLAGVARIABLE(DebugLuauFreezeDuringUnification, false) LUAU_FASTFLAGVARIABLE(LuauRecursiveTypeParameterRestriction, false) LUAU_FASTFLAGVARIABLE(LuauGenericFunctionsDontCacheTypeParams, false) +LUAU_FASTFLAGVARIABLE(LuauInferStatFunction, false) LUAU_FASTFLAGVARIABLE(LuauSealExports, false) LUAU_FASTFLAGVARIABLE(LuauSelfCallAutocompleteFix, false) LUAU_FASTFLAGVARIABLE(LuauDiscriminableUnions2, false) @@ -34,7 +35,7 @@ LUAU_FASTFLAGVARIABLE(LuauExpectedTypesOfProperties, false) LUAU_FASTFLAGVARIABLE(LuauErrorRecoveryType, false) LUAU_FASTFLAGVARIABLE(LuauOnlyMutateInstantiatedTables, false) LUAU_FASTFLAGVARIABLE(LuauPropertiesGetExpectedType, false) -LUAU_FASTFLAGVARIABLE(LuauStatFunctionSimplify2, false) +LUAU_FASTFLAGVARIABLE(LuauStatFunctionSimplify3, false) LUAU_FASTFLAGVARIABLE(LuauUnsealedTableLiteral, false) LUAU_FASTFLAG(LuauTypeMismatchModuleName) LUAU_FASTFLAGVARIABLE(LuauTwoPassAliasDefinitionFix, false) @@ -463,7 +464,18 @@ void TypeChecker::checkBlock(const ScopePtr& scope, const AstStatBlock& block) } else if (auto fun = (*protoIter)->as()) { - auto pair = checkFunctionSignature(scope, subLevel, *fun->func, fun->name->location, std::nullopt); + std::optional expectedType; + + if (FFlag::LuauInferStatFunction && !fun->func->self) + { + if (auto name = fun->name->as()) + { + TypeId exprTy = checkExpr(scope, *name->expr).type; + expectedType = getIndexTypeFromType(scope, exprTy, name->index.value, name->indexLocation, false); + } + } + + auto pair = checkFunctionSignature(scope, subLevel, *fun->func, fun->name->location, expectedType); auto [funTy, funScope] = pair; functionDecls[*protoIter] = pair; @@ -1103,7 +1115,7 @@ void TypeChecker::check(const ScopePtr& scope, TypeId ty, const ScopePtr& funSco scope->bindings[name->local] = {anyIfNonstrict(quantify(funScope, ty, name->local->location)), name->local->location}; return; } - else if (auto name = function.name->as(); name && FFlag::LuauStatFunctionSimplify2) + else if (auto name = function.name->as(); name && FFlag::LuauStatFunctionSimplify3) { TypeId exprTy = checkExpr(scope, *name->expr).type; TableTypeVar* ttv = getMutableTableType(exprTy); @@ -1116,7 +1128,7 @@ void TypeChecker::check(const ScopePtr& scope, TypeId ty, const ScopePtr& funSco } else if (ttv->state == TableState::Sealed) { - if (!ttv->indexer || !isPrim(ttv->indexer->indexType, PrimitiveTypeVar::String)) + if (!getIndexTypeFromType(scope, exprTy, name->index.value, name->indexLocation, false)) reportError(TypeError{function.location, CannotExtendTable{exprTy, CannotExtendTable::Property, name->index.value}}); } @@ -1141,7 +1153,7 @@ void TypeChecker::check(const ScopePtr& scope, TypeId ty, const ScopePtr& funSco if (ttv && ttv->state != TableState::Sealed) ttv->props[name->index.value] = {follow(quantify(funScope, ty, name->indexLocation)), /* deprecated */ false, {}, name->indexLocation}; } - else if (FFlag::LuauStatFunctionSimplify2) + else if (FFlag::LuauStatFunctionSimplify3) { LUAU_ASSERT(function.name->is()); @@ -1151,7 +1163,7 @@ void TypeChecker::check(const ScopePtr& scope, TypeId ty, const ScopePtr& funSco } else if (function.func->self) { - LUAU_ASSERT(!FFlag::LuauStatFunctionSimplify2); + LUAU_ASSERT(!FFlag::LuauStatFunctionSimplify3); AstExprIndexName* indexName = function.name->as(); if (!indexName) @@ -1190,7 +1202,7 @@ void TypeChecker::check(const ScopePtr& scope, TypeId ty, const ScopePtr& funSco } else { - LUAU_ASSERT(!FFlag::LuauStatFunctionSimplify2); + LUAU_ASSERT(!FFlag::LuauStatFunctionSimplify3); TypeId leftType = checkLValueBinding(scope, *function.name); @@ -2985,7 +2997,7 @@ TypeId TypeChecker::checkFunctionName(const ScopePtr& scope, AstExpr& funName, T return errorRecoveryType(scope); } - if (FFlag::LuauStatFunctionSimplify2) + if (FFlag::LuauStatFunctionSimplify3) { if (lhsType->persistent) return errorRecoveryType(scope); diff --git a/Ast/include/Luau/TimeTrace.h b/Ast/include/Luau/TimeTrace.h index 503eca61..5018456f 100644 --- a/Ast/include/Luau/TimeTrace.h +++ b/Ast/include/Luau/TimeTrace.h @@ -130,8 +130,8 @@ ThreadContext& getThreadContext(); struct Scope { - explicit Scope(ThreadContext& context, uint16_t token) - : context(context) + explicit Scope(uint16_t token) + : context(getThreadContext()) { if (!FFlag::DebugLuauTimeTracing) return; @@ -152,8 +152,8 @@ struct Scope struct OptionalTailScope { - explicit OptionalTailScope(ThreadContext& context, uint16_t token, uint32_t threshold) - : context(context) + explicit OptionalTailScope(uint16_t token, uint32_t threshold) + : context(getThreadContext()) , token(token) , threshold(threshold) { @@ -188,27 +188,27 @@ struct OptionalTailScope uint32_t pos; }; -LUAU_NOINLINE std::pair createScopeData(const char* name, const char* category); +LUAU_NOINLINE uint16_t createScopeData(const char* name, const char* category); } // namespace TimeTrace } // namespace Luau // Regular scope #define LUAU_TIMETRACE_SCOPE(name, category) \ - static auto lttScopeStatic = Luau::TimeTrace::createScopeData(name, category); \ - Luau::TimeTrace::Scope lttScope(lttScopeStatic.second, lttScopeStatic.first) + static uint16_t lttScopeStatic = Luau::TimeTrace::createScopeData(name, category); \ + Luau::TimeTrace::Scope lttScope(lttScopeStatic) // A scope without nested scopes that may be skipped if the time it took is less than the threshold #define LUAU_TIMETRACE_OPTIONAL_TAIL_SCOPE(name, category, microsec) \ - static auto lttScopeStaticOptTail = Luau::TimeTrace::createScopeData(name, category); \ - Luau::TimeTrace::OptionalTailScope lttScope(lttScopeStaticOptTail.second, lttScopeStaticOptTail.first, microsec) + static uint16_t lttScopeStaticOptTail = Luau::TimeTrace::createScopeData(name, category); \ + Luau::TimeTrace::OptionalTailScope lttScope(lttScopeStaticOptTail, microsec) // Extra key/value data can be added to regular scopes #define LUAU_TIMETRACE_ARGUMENT(name, value) \ do \ { \ if (FFlag::DebugLuauTimeTracing) \ - lttScopeStatic.second.eventArgument(name, value); \ + lttScope.context.eventArgument(name, value); \ } while (false) #else diff --git a/Ast/src/Lexer.cpp b/Ast/src/Lexer.cpp index d56c8860..70c6c78d 100644 --- a/Ast/src/Lexer.cpp +++ b/Ast/src/Lexer.cpp @@ -6,6 +6,8 @@ #include +LUAU_FASTFLAGVARIABLE(LuauParseLocationIgnoreCommentSkip, false) + namespace Luau { @@ -352,6 +354,8 @@ const Lexeme& Lexer::next() const Lexeme& Lexer::next(bool skipComments) { + bool first = true; + // in skipComments mode we reject valid comments do { @@ -359,9 +363,11 @@ const Lexeme& Lexer::next(bool skipComments) while (isSpace(peekch())) consume(); - prevLocation = lexeme.location; + if (!FFlag::LuauParseLocationIgnoreCommentSkip || first) + prevLocation = lexeme.location; lexeme = readNext(); + first = false; } while (skipComments && (lexeme.type == Lexeme::Comment || lexeme.type == Lexeme::BlockComment)); return lexeme; diff --git a/Ast/src/TimeTrace.cpp b/Ast/src/TimeTrace.cpp index 8079830b..19564f05 100644 --- a/Ast/src/TimeTrace.cpp +++ b/Ast/src/TimeTrace.cpp @@ -246,10 +246,9 @@ ThreadContext& getThreadContext() return context; } -std::pair createScopeData(const char* name, const char* category) +uint16_t createScopeData(const char* name, const char* category) { - uint16_t token = createToken(Luau::TimeTrace::getGlobalContext(), name, category); - return {token, Luau::TimeTrace::getThreadContext()}; + return createToken(Luau::TimeTrace::getGlobalContext(), name, category); } } // namespace TimeTrace } // namespace Luau diff --git a/Compiler/include/Luau/Bytecode.h b/Compiler/include/Luau/Bytecode.h index 679712f6..c6e5a03b 100644 --- a/Compiler/include/Luau/Bytecode.h +++ b/Compiler/include/Luau/Bytecode.h @@ -376,8 +376,7 @@ enum LuauOpcode enum LuauBytecodeTag { // Bytecode version - LBC_VERSION = 1, - LBC_VERSION_FUTURE = 2, // TODO: This will be removed in favor of LBC_VERSION with LuauBytecodeV2Force + LBC_VERSION = 2, // Types of constant table entries LBC_CONSTANT_NIL = 0, LBC_CONSTANT_BOOLEAN, diff --git a/Compiler/src/BytecodeBuilder.cpp b/Compiler/src/BytecodeBuilder.cpp index 09f06b68..6944de0f 100644 --- a/Compiler/src/BytecodeBuilder.cpp +++ b/Compiler/src/BytecodeBuilder.cpp @@ -508,7 +508,7 @@ uint32_t BytecodeBuilder::getDebugPC() const void BytecodeBuilder::finalize() { LUAU_ASSERT(bytecode.empty()); - bytecode = char(LBC_VERSION_FUTURE); + bytecode = char(LBC_VERSION); writeStringTable(bytecode); diff --git a/VM/src/laux.cpp b/VM/src/laux.cpp index 9fe2ebb6..72169a86 100644 --- a/VM/src/laux.cpp +++ b/VM/src/laux.cpp @@ -11,8 +11,6 @@ #include -LUAU_DYNAMIC_FASTFLAGVARIABLE(LuauMorePreciseLuaLTypeName, false) - /* convert a stack index to positive */ #define abs_index(L, i) ((i) > 0 || (i) <= LUA_REGISTRYINDEX ? (i) : lua_gettop(L) + (i) + 1) @@ -337,15 +335,8 @@ const char* luaL_findtable(lua_State* L, int idx, const char* fname, int szhint) const char* luaL_typename(lua_State* L, int idx) { - if (DFFlag::LuauMorePreciseLuaLTypeName) - { - const TValue* obj = luaA_toobject(L, idx); - return luaT_objtypename(L, obj); - } - else - { - return lua_typename(L, lua_type(L, idx)); - } + const TValue* obj = luaA_toobject(L, idx); + return luaT_objtypename(L, obj); } /* diff --git a/VM/src/lbaselib.cpp b/VM/src/lbaselib.cpp index 96ad493b..f7917611 100644 --- a/VM/src/lbaselib.cpp +++ b/VM/src/lbaselib.cpp @@ -11,8 +11,6 @@ #include #include -LUAU_DYNAMIC_FASTFLAG(LuauMorePreciseLuaLTypeName) - static void writestring(const char* s, size_t l) { fwrite(s, 1, l, stdout); @@ -189,31 +187,16 @@ static int luaB_gcinfo(lua_State* L) static int luaB_type(lua_State* L) { luaL_checkany(L, 1); - if (DFFlag::LuauMorePreciseLuaLTypeName) - { - /* resulting name doesn't differentiate between userdata types */ - lua_pushstring(L, lua_typename(L, lua_type(L, 1))); - } - else - { - lua_pushstring(L, luaL_typename(L, 1)); - } + /* resulting name doesn't differentiate between userdata types */ + lua_pushstring(L, lua_typename(L, lua_type(L, 1))); return 1; } static int luaB_typeof(lua_State* L) { luaL_checkany(L, 1); - if (DFFlag::LuauMorePreciseLuaLTypeName) - { - /* resulting name returns __type if specified unless the input is a newproxy-created userdata */ - lua_pushstring(L, luaL_typename(L, 1)); - } - else - { - const TValue* obj = luaA_toobject(L, 1); - lua_pushstring(L, luaT_objtypename(L, obj)); - } + /* resulting name returns __type if specified unless the input is a newproxy-created userdata */ + lua_pushstring(L, luaL_typename(L, 1)); return 1; } diff --git a/VM/src/ldebug.cpp b/VM/src/ldebug.cpp index 7a9947b7..e050050e 100644 --- a/VM/src/ldebug.cpp +++ b/VM/src/ldebug.cpp @@ -12,8 +12,6 @@ #include #include -LUAU_FASTFLAG(LuauBytecodeV2Force) - static const char* getfuncname(Closure* f); static int currentpc(lua_State* L, CallInfo* ci) @@ -91,16 +89,6 @@ const char* lua_setlocal(lua_State* L, int level, int n) return name; } -static int getlinedefined(Proto* p) -{ - if (FFlag::LuauBytecodeV2Force) - return p->linedefined; - else if (p->linedefined >= 0) - return p->linedefined; - else - return luaG_getline(p, 0); -} - static int auxgetinfo(lua_State* L, const char* what, lua_Debug* ar, Closure* f, CallInfo* ci) { int status = 1; @@ -120,7 +108,7 @@ static int auxgetinfo(lua_State* L, const char* what, lua_Debug* ar, Closure* f, { ar->source = getstr(f->l.p->source); ar->what = "Lua"; - ar->linedefined = getlinedefined(f->l.p); + ar->linedefined = f->l.p->linedefined; } luaO_chunkid(ar->short_src, ar->source, LUA_IDSIZE); break; @@ -133,7 +121,7 @@ static int auxgetinfo(lua_State* L, const char* what, lua_Debug* ar, Closure* f, } else { - ar->currentline = f->isC ? -1 : getlinedefined(f->l.p); + ar->currentline = f->isC ? -1 : f->l.p->linedefined; } break; @@ -424,7 +412,7 @@ static void getcoverage(Proto* p, int depth, int* buffer, size_t size, void* con } const char* debugname = p->debugname ? getstr(p->debugname) : NULL; - int linedefined = getlinedefined(p); + int linedefined = p->linedefined; callback(context, debugname, linedefined, depth, buffer, size); diff --git a/VM/src/lmem.h b/VM/src/lmem.h index 00788452..e552d739 100644 --- a/VM/src/lmem.h +++ b/VM/src/lmem.h @@ -10,12 +10,12 @@ union GCObject; #define luaM_newgco(L, t, size, memcat) cast_to(t*, luaM_newgco_(L, size, memcat)) #define luaM_freegco(L, p, size, memcat, page) luaM_freegco_(L, obj2gco(p), size, memcat, page) -#define luaM_arraysize_(n, e) ((cast_to(size_t, (n)) <= SIZE_MAX / (e)) ? (n) * (e) : (luaM_toobig(L), SIZE_MAX)) +#define luaM_arraysize_(L, n, e) ((cast_to(size_t, (n)) <= SIZE_MAX / (e)) ? (n) * (e) : (luaM_toobig(L), SIZE_MAX)) -#define luaM_newarray(L, n, t, memcat) cast_to(t*, luaM_new_(L, luaM_arraysize_(n, sizeof(t)), memcat)) +#define luaM_newarray(L, n, t, memcat) cast_to(t*, luaM_new_(L, luaM_arraysize_(L, n, sizeof(t)), memcat)) #define luaM_freearray(L, b, n, t, memcat) luaM_free_(L, (b), (n) * sizeof(t), memcat) #define luaM_reallocarray(L, v, oldn, n, t, memcat) \ - ((v) = cast_to(t*, luaM_realloc_(L, v, (oldn) * sizeof(t), luaM_arraysize_(n, sizeof(t)), memcat))) + ((v) = cast_to(t*, luaM_realloc_(L, v, (oldn) * sizeof(t), luaM_arraysize_(L, n, sizeof(t)), memcat))) LUAI_FUNC void* luaM_new_(lua_State* L, size_t nsize, uint8_t memcat); LUAI_FUNC GCObject* luaM_newgco_(lua_State* L, size_t nsize, uint8_t memcat); diff --git a/VM/src/ltable.cpp b/VM/src/ltable.cpp index 431501f3..1c75c0b0 100644 --- a/VM/src/ltable.cpp +++ b/VM/src/ltable.cpp @@ -2,17 +2,26 @@ // This code is based on Lua 5.x implementation licensed under MIT License; see lua_LICENSE.txt for details /* -** Implementation of tables (aka arrays, objects, or hash tables). -** Tables keep its elements in two parts: an array part and a hash part. -** Non-negative integer keys are all candidates to be kept in the array -** part. The actual size of the array is the largest `n' such that at -** least half the slots between 0 and n are in use. -** Hash uses a mix of chained scatter table with Brent's variation. -** A main invariant of these tables is that, if an element is not -** in its main position (i.e. the `original' position that its hash gives -** to it), then the colliding element is in its own main position. -** Hence even when the load factor reaches 100%, performance remains good. -*/ + * Implementation of tables (aka arrays, objects, or hash tables). + * + * Tables keep the elements in two parts: an array part and a hash part. + * Integer keys >=1 are all candidates to be kept in the array part. The actual size of the array is the + * largest n such that at least half the slots between 0 and n are in use. + * Hash uses a mix of chained scatter table with Brent's variation. + * + * A main invariant of these tables is that, if an element is not in its main position (i.e. the original + * position that its hash gives to it), then the colliding element is in its own main position. + * Hence even when the load factor reaches 100%, performance remains good. + * + * Table keys can be arbitrary values unless they contain NaN. Keys are hashed and compared using raw equality, + * so even if the key is a userdata with an overridden __eq, it's not used during hash lookups. + * + * Each table has a "boundary", defined as the index k where t[k] ~= nil and t[k+1] == nil. The boundary can be + * computed using a binary search and can be adjusted when the table is modified; crucially, Luau enforces an + * invariant where the boundary must be in the array part - this enforces a consistent iteration order through the + * prefix of the table when using pairs(), and allows to implement algorithms that access elements in 1..#t range + * more efficiently. + */ #include "ltable.h" @@ -25,6 +34,7 @@ #include LUAU_FASTFLAGVARIABLE(LuauTableRehashRework, false) +LUAU_FASTFLAGVARIABLE(LuauTableNewBoundary, false) // max size of both array and hash part is 2^MAXBITS #define MAXBITS 26 @@ -460,7 +470,20 @@ static void rehash(lua_State* L, Table* t, const TValue* ek) totaluse++; /* compute new size for array part */ int na = computesizes(nums, &nasize); + /* enforce the boundary invariant; for performance, only do hash lookups if we must */ + if (FFlag::LuauTableNewBoundary) + { + bool tbound = t->node != dummynode || nasize < t->sizearray; + int ekindex = ttisnumber(ek) ? arrayindex(nvalue(ek)) : -1; + /* move the array size up until the boundary is guaranteed to be inside the array part */ + while (nasize + 1 == ekindex || (tbound && !ttisnil(luaH_getnum(t, nasize + 1)))) + { + nasize++; + na++; + } + } /* resize the table to new computed sizes */ + LUAU_ASSERT(na <= totaluse); resize(L, t, nasize, totaluse - na); } @@ -520,10 +543,18 @@ static LuaNode* getfreepos(Table* t) */ static TValue* newkey(lua_State* L, Table* t, const TValue* key) { + /* enforce boundary invariant */ + if (FFlag::LuauTableNewBoundary && ttisnumber(key) && nvalue(key) == t->sizearray + 1) + { + rehash(L, t, key); /* grow table */ + + // after rehash, numeric keys might be located in the new array part, but won't be found in the node part + return arrayornewkey(L, t, key); + } + LuaNode* mp = mainposition(t, key); if (!ttisnil(gval(mp)) || mp == dummynode) { - LuaNode* othern; LuaNode* n = getfreepos(t); /* get a free place */ if (n == NULL) { /* cannot find a free place? */ @@ -542,7 +573,7 @@ static TValue* newkey(lua_State* L, Table* t, const TValue* key) LUAU_ASSERT(n != dummynode); TValue mk; getnodekey(L, &mk, mp); - othern = mainposition(t, &mk); + LuaNode* othern = mainposition(t, &mk); if (othern != mp) { /* is colliding node out of its main position? */ /* yes; move colliding node into free position */ @@ -704,6 +735,7 @@ TValue* luaH_setstr(lua_State* L, Table* t, TString* key) static LUAU_NOINLINE int unbound_search(Table* t, unsigned int j) { + LUAU_ASSERT(!FFlag::LuauTableNewBoundary); unsigned int i = j; /* i is zero or a present index */ j++; /* find `i' and `j' such that i is present and j is not */ @@ -788,6 +820,12 @@ int luaH_getn(Table* t) maybesetaboundary(t, boundary); return boundary; } + else if (FFlag::LuauTableNewBoundary) + { + /* validate boundary invariant */ + LUAU_ASSERT(t->node == dummynode || ttisnil(luaH_getnum(t, j + 1))); + return j; + } /* else must find a boundary in hash part */ else if (t->node == dummynode) /* hash part is empty? */ return j; /* that is easy... */ diff --git a/VM/src/ltablib.cpp b/VM/src/ltablib.cpp index 00753742..241a99e3 100644 --- a/VM/src/ltablib.cpp +++ b/VM/src/ltablib.cpp @@ -10,7 +10,9 @@ #include "ldebug.h" #include "lvm.h" -LUAU_FASTFLAGVARIABLE(LuauTableClone, false) +LUAU_DYNAMIC_FASTFLAGVARIABLE(LuauTableMoveTelemetry2, false) + +void (*lua_table_move_telemetry)(lua_State* L, int f, int e, int t, int nf, int nt); static int foreachi(lua_State* L) { @@ -197,6 +199,29 @@ static int tmove(lua_State* L) int tt = !lua_isnoneornil(L, 5) ? 5 : 1; /* destination table */ luaL_checktype(L, tt, LUA_TTABLE); + void (*telemetrycb)(lua_State* L, int f, int e, int t, int nf, int nt) = lua_table_move_telemetry; + + if (DFFlag::LuauTableMoveTelemetry2 && telemetrycb) + { + int nf = lua_objlen(L, 1); + int nt = lua_objlen(L, tt); + + bool report = false; + + // source index range must be in bounds in source table unless the table is empty (permits 1..#t moves) + if (!(f == 1 || (f >= 1 && f <= nf))) + report = true; + if (!(e == nf || (e >= 1 && e <= nf))) + report = true; + + // destination index must be in bounds in dest table or be exactly at the first empty element (permits concats) + if (!(t == nt + 1 || (t >= 1 && t <= nt))) + report = true; + + if (report) + telemetrycb(L, f, e, t, nf, nt); + } + if (e >= f) { /* otherwise, nothing to move */ luaL_argcheck(L, f > 0 || e < INT_MAX + f, 3, "too many elements to move"); @@ -512,9 +537,6 @@ static int tisfrozen(lua_State* L) static int tclone(lua_State* L) { - if (!FFlag::LuauTableClone) - luaG_runerror(L, "table.clone is not available"); - luaL_checktype(L, 1, LUA_TTABLE); luaL_argcheck(L, !luaL_getmetafield(L, 1, "__metatable"), 1, "table has a protected metatable"); diff --git a/VM/src/lvmexecute.cpp b/VM/src/lvmexecute.cpp index 96a87b7e..34949efb 100644 --- a/VM/src/lvmexecute.cpp +++ b/VM/src/lvmexecute.cpp @@ -16,6 +16,8 @@ #include +LUAU_FASTFLAG(LuauTableNewBoundary) + // Disable c99-designator to avoid the warning in CGOTO dispatch table #ifdef __clang__ #if __has_warning("-Wc99-designator") @@ -2266,9 +2268,9 @@ static void luau_execute(lua_State* L) VM_NEXT(); } } - else if (h->lsizenode == 0 && ttisnil(gval(h->node))) + else if (FFlag::LuauTableNewBoundary || (h->lsizenode == 0 && ttisnil(gval(h->node)))) { - // hash part is empty: fallthrough to exit + // fallthrough to exit VM_NEXT(); } else diff --git a/VM/src/lvmload.cpp b/VM/src/lvmload.cpp index 4e5435b7..8b742f1c 100644 --- a/VM/src/lvmload.cpp +++ b/VM/src/lvmload.cpp @@ -13,8 +13,6 @@ #include -LUAU_FASTFLAGVARIABLE(LuauBytecodeV2Force, false) - // TODO: RAII deallocation doesn't work for longjmp builds if a memory error happens template struct TempBuffer @@ -156,12 +154,11 @@ int luau_load(lua_State* L, const char* chunkname, const char* data, size_t size return 1; } - if (FFlag::LuauBytecodeV2Force ? (version != LBC_VERSION_FUTURE) : (version != LBC_VERSION && version != LBC_VERSION_FUTURE)) + if (version != LBC_VERSION) { char chunkid[LUA_IDSIZE]; luaO_chunkid(chunkid, chunkname, LUA_IDSIZE); - lua_pushfstring(L, "%s: bytecode version mismatch (expected %d, got %d)", chunkid, - FFlag::LuauBytecodeV2Force ? LBC_VERSION_FUTURE : LBC_VERSION, version); + lua_pushfstring(L, "%s: bytecode version mismatch (expected %d, got %d)", chunkid, LBC_VERSION, version); return 1; } @@ -292,11 +289,7 @@ int luau_load(lua_State* L, const char* chunkname, const char* data, size_t size p->p[j] = protos[fid]; } - if (FFlag::LuauBytecodeV2Force || version == LBC_VERSION_FUTURE) - p->linedefined = readVarInt(data, size, offset); - else - p->linedefined = -1; - + p->linedefined = readVarInt(data, size, offset); p->debugname = readString(strings, data, size, offset); uint8_t lineinfo = read(data, size, offset); diff --git a/tests/Autocomplete.test.cpp b/tests/Autocomplete.test.cpp index 1db782cc..4e8a1d55 100644 --- a/tests/Autocomplete.test.cpp +++ b/tests/Autocomplete.test.cpp @@ -14,7 +14,6 @@ LUAU_FASTFLAG(LuauTraceTypesInNonstrictMode2) LUAU_FASTFLAG(LuauSetMetatableDoesNotTimeTravel) -LUAU_FASTFLAG(LuauTableCloneType) using namespace Luau; @@ -262,7 +261,7 @@ TEST_CASE_FIXTURE(ACFixture, "get_member_completions") auto ac = autocomplete('1'); - CHECK_EQ(FFlag::LuauTableCloneType ? 17 : 16, ac.entryMap.size()); + CHECK_EQ(17, ac.entryMap.size()); CHECK(ac.entryMap.count("find")); CHECK(ac.entryMap.count("pack")); CHECK(!ac.entryMap.count("math")); @@ -2221,7 +2220,7 @@ TEST_CASE_FIXTURE(ACFixture, "autocompleteSource") auto ac = autocompleteSource(frontend, source, Position{1, 24}, nullCallback).result; - CHECK_EQ(FFlag::LuauTableCloneType ? 17 : 16, ac.entryMap.size()); + CHECK_EQ(17, ac.entryMap.size()); CHECK(ac.entryMap.count("find")); CHECK(ac.entryMap.count("pack")); CHECK(!ac.entryMap.count("math")); diff --git a/tests/Conformance.test.cpp b/tests/Conformance.test.cpp index 83d4518d..0ed7dc44 100644 --- a/tests/Conformance.test.cpp +++ b/tests/Conformance.test.cpp @@ -241,8 +241,6 @@ TEST_CASE("Math") TEST_CASE("Table") { - ScopedFastFlag sff("LuauTableClone", true); - runConformance("nextvar.lua"); } @@ -467,8 +465,6 @@ static void populateRTTI(lua_State* L, Luau::TypeId type) TEST_CASE("Types") { - ScopedFastFlag sff("LuauTableCloneType", true); - runConformance("types.lua", [](lua_State* L) { Luau::NullModuleResolver moduleResolver; Luau::InternalErrorReporter iceHandler; diff --git a/tests/Parser.test.cpp b/tests/Parser.test.cpp index 7f6a6c0d..7dacc669 100644 --- a/tests/Parser.test.cpp +++ b/tests/Parser.test.cpp @@ -1604,6 +1604,20 @@ TEST_CASE_FIXTURE(Fixture, "end_extent_of_functions_unions_and_intersections") CHECK_EQ((Position{3, 42}), block->body.data[2]->location.end); } +TEST_CASE_FIXTURE(Fixture, "end_extent_doesnt_consume_comments") +{ + ScopedFastFlag luauParseLocationIgnoreCommentSkip{"LuauParseLocationIgnoreCommentSkip", true}; + + AstStatBlock* block = parse(R"( + type F = number + --comment + print('hello') + )"); + + REQUIRE_EQ(2, block->body.size); + CHECK_EQ((Position{1, 23}), block->body.data[0]->location.end); +} + TEST_CASE_FIXTURE(Fixture, "parse_error_loop_control") { matchParseError("break", "break statement must be inside a loop"); diff --git a/tests/TypeInfer.functions.test.cpp b/tests/TypeInfer.functions.test.cpp index da4ea074..dbae7b54 100644 --- a/tests/TypeInfer.functions.test.cpp +++ b/tests/TypeInfer.functions.test.cpp @@ -1270,7 +1270,7 @@ caused by: TEST_CASE_FIXTURE(Fixture, "function_decl_quantify_right_type") { - ScopedFastFlag statFunctionSimplify{"LuauStatFunctionSimplify2", true}; + ScopedFastFlag statFunctionSimplify{"LuauStatFunctionSimplify3", true}; fileResolver.source["game/isAMagicMock"] = R"( --!nonstrict @@ -1294,7 +1294,7 @@ end TEST_CASE_FIXTURE(Fixture, "function_decl_non_self_sealed_overwrite") { - ScopedFastFlag statFunctionSimplify{"LuauStatFunctionSimplify2", true}; + ScopedFastFlag statFunctionSimplify{"LuauStatFunctionSimplify3", true}; CheckResult result = check(R"( function string.len(): number @@ -1302,7 +1302,40 @@ function string.len(): number end )"); - LUAU_REQUIRE_ERRORS(result); + LUAU_REQUIRE_NO_ERRORS(result); + + // if 'string' library property was replaced with an internal module type, it will be freed and the next check will crash + frontend.clear(); + + result = check(R"( +print(string.len('hello')) + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "function_decl_non_self_sealed_overwrite_2") +{ + ScopedFastFlag statFunctionSimplify{"LuauStatFunctionSimplify3", true}; + ScopedFastFlag inferStatFunction{"LuauInferStatFunction", true}; + + CheckResult result = check(R"( +local t: { f: ((x: number) -> number)? } = {} + +function t.f(x) + print(x + 5) + return x .. "asd" +end + +t.f = function(x) + print(x + 5) + return x .. "asd" +end + )"); + + LUAU_REQUIRE_ERROR_COUNT(2, result); + CHECK_EQ(toString(result.errors[0]), R"(Type 'string' could not be converted into 'number')"); + CHECK_EQ(toString(result.errors[1]), R"(Type 'string' could not be converted into 'number')"); } TEST_CASE_FIXTURE(Fixture, "strict_mode_ok_with_missing_arguments") @@ -1319,7 +1352,7 @@ TEST_CASE_FIXTURE(Fixture, "strict_mode_ok_with_missing_arguments") TEST_CASE_FIXTURE(Fixture, "function_statement_sealed_table_assignment_through_indexer") { - ScopedFastFlag statFunctionSimplify{"LuauStatFunctionSimplify2", true}; + ScopedFastFlag statFunctionSimplify{"LuauStatFunctionSimplify3", true}; CheckResult result = check(R"( local t: {[string]: () -> number} = {} diff --git a/tests/conformance/nextvar.lua b/tests/conformance/nextvar.lua index e85fcbe8..ab9be42c 100644 --- a/tests/conformance/nextvar.lua +++ b/tests/conformance/nextvar.lua @@ -550,4 +550,35 @@ do assert(not pcall(table.clone, 42)) end +-- test boundary invariant maintenance during rehash +do + local arr = table.create(5, 42) + + arr[1] = nil + arr.a = 'a' -- trigger rehash + + assert(#arr == 5) -- technically 0 is also valid, but it happens to be 5 because array capacity is 5 +end + +-- test boundary invariant maintenance when replacing hash keys +do + local arr = {} + arr.a = 'a' + arr.a = nil + arr[1] = 1 -- should rehash and resize array part, otherwise # won't find the boundary in array part + + assert(#arr == 1) +end + +-- test boundary invariant maintenance when table is filled from the end +do + local arr = {} + for i=5,2,-1 do + arr[i] = i + assert(#arr == 0) + end + arr[1] = 1 + assert(#arr == 5) +end + return"OK" From 4c1f208d7a2172202fc3b424cca8c75532e98af9 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Thu, 31 Mar 2022 14:01:51 -0700 Subject: [PATCH 214/399] Sync to upstream/release/521 (#443) --- Analysis/src/BuiltinDefinitions.cpp | 5 +-- Analysis/src/TypeInfer.cpp | 28 +++++++++---- Ast/include/Luau/TimeTrace.h | 20 ++++----- Ast/src/Lexer.cpp | 8 +++- Ast/src/TimeTrace.cpp | 5 +-- Compiler/include/Luau/Bytecode.h | 3 +- Compiler/src/BytecodeBuilder.cpp | 2 +- VM/src/laux.cpp | 13 +----- VM/src/lbaselib.cpp | 25 ++--------- VM/src/ldebug.cpp | 18 ++------ VM/src/lmem.h | 6 +-- VM/src/ltable.cpp | 64 +++++++++++++++++++++++------ VM/src/ltablib.cpp | 30 ++++++++++++-- VM/src/lvmexecute.cpp | 6 ++- VM/src/lvmload.cpp | 13 ++---- tests/Autocomplete.test.cpp | 5 +-- tests/Conformance.test.cpp | 4 -- tests/Parser.test.cpp | 14 +++++++ tests/TypeInfer.functions.test.cpp | 41 ++++++++++++++++-- tests/conformance/nextvar.lua | 31 ++++++++++++++ 20 files changed, 222 insertions(+), 119 deletions(-) diff --git a/Analysis/src/BuiltinDefinitions.cpp b/Analysis/src/BuiltinDefinitions.cpp index bf9ef303..3895b01b 100644 --- a/Analysis/src/BuiltinDefinitions.cpp +++ b/Analysis/src/BuiltinDefinitions.cpp @@ -9,7 +9,6 @@ #include LUAU_FASTFLAG(LuauAssertStripsFalsyTypes) -LUAU_FASTFLAGVARIABLE(LuauTableCloneType, false) LUAU_FASTFLAGVARIABLE(LuauSetMetaTableArgsCheck, false) /** FIXME: Many of these type definitions are not quite completely accurate. @@ -289,9 +288,7 @@ void registerBuiltinTypes(TypeChecker& typeChecker) { // tabTy is a generic table type which we can't express via declaration syntax yet ttv->props["freeze"] = makeProperty(makeFunction(arena, std::nullopt, {tabTy}, {tabTy}), "@luau/global/table.freeze"); - - if (FFlag::LuauTableCloneType) - ttv->props["clone"] = makeProperty(makeFunction(arena, std::nullopt, {tabTy}, {tabTy}), "@luau/global/table.clone"); + ttv->props["clone"] = makeProperty(makeFunction(arena, std::nullopt, {tabTy}, {tabTy}), "@luau/global/table.clone"); attachMagicFunction(ttv->props["pack"].type, magicFunctionPack); } diff --git a/Analysis/src/TypeInfer.cpp b/Analysis/src/TypeInfer.cpp index 9965d5aa..6df6bff0 100644 --- a/Analysis/src/TypeInfer.cpp +++ b/Analysis/src/TypeInfer.cpp @@ -27,6 +27,7 @@ LUAU_FASTFLAGVARIABLE(LuauWeakEqConstraint, false) // Eventually removed as fals LUAU_FASTFLAGVARIABLE(DebugLuauFreezeDuringUnification, false) LUAU_FASTFLAGVARIABLE(LuauRecursiveTypeParameterRestriction, false) LUAU_FASTFLAGVARIABLE(LuauGenericFunctionsDontCacheTypeParams, false) +LUAU_FASTFLAGVARIABLE(LuauInferStatFunction, false) LUAU_FASTFLAGVARIABLE(LuauSealExports, false) LUAU_FASTFLAGVARIABLE(LuauSelfCallAutocompleteFix, false) LUAU_FASTFLAGVARIABLE(LuauDiscriminableUnions2, false) @@ -34,7 +35,7 @@ LUAU_FASTFLAGVARIABLE(LuauExpectedTypesOfProperties, false) LUAU_FASTFLAGVARIABLE(LuauErrorRecoveryType, false) LUAU_FASTFLAGVARIABLE(LuauOnlyMutateInstantiatedTables, false) LUAU_FASTFLAGVARIABLE(LuauPropertiesGetExpectedType, false) -LUAU_FASTFLAGVARIABLE(LuauStatFunctionSimplify2, false) +LUAU_FASTFLAGVARIABLE(LuauStatFunctionSimplify3, false) LUAU_FASTFLAGVARIABLE(LuauUnsealedTableLiteral, false) LUAU_FASTFLAG(LuauTypeMismatchModuleName) LUAU_FASTFLAGVARIABLE(LuauTwoPassAliasDefinitionFix, false) @@ -463,7 +464,18 @@ void TypeChecker::checkBlock(const ScopePtr& scope, const AstStatBlock& block) } else if (auto fun = (*protoIter)->as()) { - auto pair = checkFunctionSignature(scope, subLevel, *fun->func, fun->name->location, std::nullopt); + std::optional expectedType; + + if (FFlag::LuauInferStatFunction && !fun->func->self) + { + if (auto name = fun->name->as()) + { + TypeId exprTy = checkExpr(scope, *name->expr).type; + expectedType = getIndexTypeFromType(scope, exprTy, name->index.value, name->indexLocation, false); + } + } + + auto pair = checkFunctionSignature(scope, subLevel, *fun->func, fun->name->location, expectedType); auto [funTy, funScope] = pair; functionDecls[*protoIter] = pair; @@ -1103,7 +1115,7 @@ void TypeChecker::check(const ScopePtr& scope, TypeId ty, const ScopePtr& funSco scope->bindings[name->local] = {anyIfNonstrict(quantify(funScope, ty, name->local->location)), name->local->location}; return; } - else if (auto name = function.name->as(); name && FFlag::LuauStatFunctionSimplify2) + else if (auto name = function.name->as(); name && FFlag::LuauStatFunctionSimplify3) { TypeId exprTy = checkExpr(scope, *name->expr).type; TableTypeVar* ttv = getMutableTableType(exprTy); @@ -1116,7 +1128,7 @@ void TypeChecker::check(const ScopePtr& scope, TypeId ty, const ScopePtr& funSco } else if (ttv->state == TableState::Sealed) { - if (!ttv->indexer || !isPrim(ttv->indexer->indexType, PrimitiveTypeVar::String)) + if (!getIndexTypeFromType(scope, exprTy, name->index.value, name->indexLocation, false)) reportError(TypeError{function.location, CannotExtendTable{exprTy, CannotExtendTable::Property, name->index.value}}); } @@ -1141,7 +1153,7 @@ void TypeChecker::check(const ScopePtr& scope, TypeId ty, const ScopePtr& funSco if (ttv && ttv->state != TableState::Sealed) ttv->props[name->index.value] = {follow(quantify(funScope, ty, name->indexLocation)), /* deprecated */ false, {}, name->indexLocation}; } - else if (FFlag::LuauStatFunctionSimplify2) + else if (FFlag::LuauStatFunctionSimplify3) { LUAU_ASSERT(function.name->is()); @@ -1151,7 +1163,7 @@ void TypeChecker::check(const ScopePtr& scope, TypeId ty, const ScopePtr& funSco } else if (function.func->self) { - LUAU_ASSERT(!FFlag::LuauStatFunctionSimplify2); + LUAU_ASSERT(!FFlag::LuauStatFunctionSimplify3); AstExprIndexName* indexName = function.name->as(); if (!indexName) @@ -1190,7 +1202,7 @@ void TypeChecker::check(const ScopePtr& scope, TypeId ty, const ScopePtr& funSco } else { - LUAU_ASSERT(!FFlag::LuauStatFunctionSimplify2); + LUAU_ASSERT(!FFlag::LuauStatFunctionSimplify3); TypeId leftType = checkLValueBinding(scope, *function.name); @@ -2985,7 +2997,7 @@ TypeId TypeChecker::checkFunctionName(const ScopePtr& scope, AstExpr& funName, T return errorRecoveryType(scope); } - if (FFlag::LuauStatFunctionSimplify2) + if (FFlag::LuauStatFunctionSimplify3) { if (lhsType->persistent) return errorRecoveryType(scope); diff --git a/Ast/include/Luau/TimeTrace.h b/Ast/include/Luau/TimeTrace.h index 503eca61..5018456f 100644 --- a/Ast/include/Luau/TimeTrace.h +++ b/Ast/include/Luau/TimeTrace.h @@ -130,8 +130,8 @@ ThreadContext& getThreadContext(); struct Scope { - explicit Scope(ThreadContext& context, uint16_t token) - : context(context) + explicit Scope(uint16_t token) + : context(getThreadContext()) { if (!FFlag::DebugLuauTimeTracing) return; @@ -152,8 +152,8 @@ struct Scope struct OptionalTailScope { - explicit OptionalTailScope(ThreadContext& context, uint16_t token, uint32_t threshold) - : context(context) + explicit OptionalTailScope(uint16_t token, uint32_t threshold) + : context(getThreadContext()) , token(token) , threshold(threshold) { @@ -188,27 +188,27 @@ struct OptionalTailScope uint32_t pos; }; -LUAU_NOINLINE std::pair createScopeData(const char* name, const char* category); +LUAU_NOINLINE uint16_t createScopeData(const char* name, const char* category); } // namespace TimeTrace } // namespace Luau // Regular scope #define LUAU_TIMETRACE_SCOPE(name, category) \ - static auto lttScopeStatic = Luau::TimeTrace::createScopeData(name, category); \ - Luau::TimeTrace::Scope lttScope(lttScopeStatic.second, lttScopeStatic.first) + static uint16_t lttScopeStatic = Luau::TimeTrace::createScopeData(name, category); \ + Luau::TimeTrace::Scope lttScope(lttScopeStatic) // A scope without nested scopes that may be skipped if the time it took is less than the threshold #define LUAU_TIMETRACE_OPTIONAL_TAIL_SCOPE(name, category, microsec) \ - static auto lttScopeStaticOptTail = Luau::TimeTrace::createScopeData(name, category); \ - Luau::TimeTrace::OptionalTailScope lttScope(lttScopeStaticOptTail.second, lttScopeStaticOptTail.first, microsec) + static uint16_t lttScopeStaticOptTail = Luau::TimeTrace::createScopeData(name, category); \ + Luau::TimeTrace::OptionalTailScope lttScope(lttScopeStaticOptTail, microsec) // Extra key/value data can be added to regular scopes #define LUAU_TIMETRACE_ARGUMENT(name, value) \ do \ { \ if (FFlag::DebugLuauTimeTracing) \ - lttScopeStatic.second.eventArgument(name, value); \ + lttScope.context.eventArgument(name, value); \ } while (false) #else diff --git a/Ast/src/Lexer.cpp b/Ast/src/Lexer.cpp index d56c8860..70c6c78d 100644 --- a/Ast/src/Lexer.cpp +++ b/Ast/src/Lexer.cpp @@ -6,6 +6,8 @@ #include +LUAU_FASTFLAGVARIABLE(LuauParseLocationIgnoreCommentSkip, false) + namespace Luau { @@ -352,6 +354,8 @@ const Lexeme& Lexer::next() const Lexeme& Lexer::next(bool skipComments) { + bool first = true; + // in skipComments mode we reject valid comments do { @@ -359,9 +363,11 @@ const Lexeme& Lexer::next(bool skipComments) while (isSpace(peekch())) consume(); - prevLocation = lexeme.location; + if (!FFlag::LuauParseLocationIgnoreCommentSkip || first) + prevLocation = lexeme.location; lexeme = readNext(); + first = false; } while (skipComments && (lexeme.type == Lexeme::Comment || lexeme.type == Lexeme::BlockComment)); return lexeme; diff --git a/Ast/src/TimeTrace.cpp b/Ast/src/TimeTrace.cpp index 8079830b..19564f05 100644 --- a/Ast/src/TimeTrace.cpp +++ b/Ast/src/TimeTrace.cpp @@ -246,10 +246,9 @@ ThreadContext& getThreadContext() return context; } -std::pair createScopeData(const char* name, const char* category) +uint16_t createScopeData(const char* name, const char* category) { - uint16_t token = createToken(Luau::TimeTrace::getGlobalContext(), name, category); - return {token, Luau::TimeTrace::getThreadContext()}; + return createToken(Luau::TimeTrace::getGlobalContext(), name, category); } } // namespace TimeTrace } // namespace Luau diff --git a/Compiler/include/Luau/Bytecode.h b/Compiler/include/Luau/Bytecode.h index 679712f6..c6e5a03b 100644 --- a/Compiler/include/Luau/Bytecode.h +++ b/Compiler/include/Luau/Bytecode.h @@ -376,8 +376,7 @@ enum LuauOpcode enum LuauBytecodeTag { // Bytecode version - LBC_VERSION = 1, - LBC_VERSION_FUTURE = 2, // TODO: This will be removed in favor of LBC_VERSION with LuauBytecodeV2Force + LBC_VERSION = 2, // Types of constant table entries LBC_CONSTANT_NIL = 0, LBC_CONSTANT_BOOLEAN, diff --git a/Compiler/src/BytecodeBuilder.cpp b/Compiler/src/BytecodeBuilder.cpp index 09f06b68..6944de0f 100644 --- a/Compiler/src/BytecodeBuilder.cpp +++ b/Compiler/src/BytecodeBuilder.cpp @@ -508,7 +508,7 @@ uint32_t BytecodeBuilder::getDebugPC() const void BytecodeBuilder::finalize() { LUAU_ASSERT(bytecode.empty()); - bytecode = char(LBC_VERSION_FUTURE); + bytecode = char(LBC_VERSION); writeStringTable(bytecode); diff --git a/VM/src/laux.cpp b/VM/src/laux.cpp index 9fe2ebb6..72169a86 100644 --- a/VM/src/laux.cpp +++ b/VM/src/laux.cpp @@ -11,8 +11,6 @@ #include -LUAU_DYNAMIC_FASTFLAGVARIABLE(LuauMorePreciseLuaLTypeName, false) - /* convert a stack index to positive */ #define abs_index(L, i) ((i) > 0 || (i) <= LUA_REGISTRYINDEX ? (i) : lua_gettop(L) + (i) + 1) @@ -337,15 +335,8 @@ const char* luaL_findtable(lua_State* L, int idx, const char* fname, int szhint) const char* luaL_typename(lua_State* L, int idx) { - if (DFFlag::LuauMorePreciseLuaLTypeName) - { - const TValue* obj = luaA_toobject(L, idx); - return luaT_objtypename(L, obj); - } - else - { - return lua_typename(L, lua_type(L, idx)); - } + const TValue* obj = luaA_toobject(L, idx); + return luaT_objtypename(L, obj); } /* diff --git a/VM/src/lbaselib.cpp b/VM/src/lbaselib.cpp index 96ad493b..f7917611 100644 --- a/VM/src/lbaselib.cpp +++ b/VM/src/lbaselib.cpp @@ -11,8 +11,6 @@ #include #include -LUAU_DYNAMIC_FASTFLAG(LuauMorePreciseLuaLTypeName) - static void writestring(const char* s, size_t l) { fwrite(s, 1, l, stdout); @@ -189,31 +187,16 @@ static int luaB_gcinfo(lua_State* L) static int luaB_type(lua_State* L) { luaL_checkany(L, 1); - if (DFFlag::LuauMorePreciseLuaLTypeName) - { - /* resulting name doesn't differentiate between userdata types */ - lua_pushstring(L, lua_typename(L, lua_type(L, 1))); - } - else - { - lua_pushstring(L, luaL_typename(L, 1)); - } + /* resulting name doesn't differentiate between userdata types */ + lua_pushstring(L, lua_typename(L, lua_type(L, 1))); return 1; } static int luaB_typeof(lua_State* L) { luaL_checkany(L, 1); - if (DFFlag::LuauMorePreciseLuaLTypeName) - { - /* resulting name returns __type if specified unless the input is a newproxy-created userdata */ - lua_pushstring(L, luaL_typename(L, 1)); - } - else - { - const TValue* obj = luaA_toobject(L, 1); - lua_pushstring(L, luaT_objtypename(L, obj)); - } + /* resulting name returns __type if specified unless the input is a newproxy-created userdata */ + lua_pushstring(L, luaL_typename(L, 1)); return 1; } diff --git a/VM/src/ldebug.cpp b/VM/src/ldebug.cpp index 7a9947b7..e050050e 100644 --- a/VM/src/ldebug.cpp +++ b/VM/src/ldebug.cpp @@ -12,8 +12,6 @@ #include #include -LUAU_FASTFLAG(LuauBytecodeV2Force) - static const char* getfuncname(Closure* f); static int currentpc(lua_State* L, CallInfo* ci) @@ -91,16 +89,6 @@ const char* lua_setlocal(lua_State* L, int level, int n) return name; } -static int getlinedefined(Proto* p) -{ - if (FFlag::LuauBytecodeV2Force) - return p->linedefined; - else if (p->linedefined >= 0) - return p->linedefined; - else - return luaG_getline(p, 0); -} - static int auxgetinfo(lua_State* L, const char* what, lua_Debug* ar, Closure* f, CallInfo* ci) { int status = 1; @@ -120,7 +108,7 @@ static int auxgetinfo(lua_State* L, const char* what, lua_Debug* ar, Closure* f, { ar->source = getstr(f->l.p->source); ar->what = "Lua"; - ar->linedefined = getlinedefined(f->l.p); + ar->linedefined = f->l.p->linedefined; } luaO_chunkid(ar->short_src, ar->source, LUA_IDSIZE); break; @@ -133,7 +121,7 @@ static int auxgetinfo(lua_State* L, const char* what, lua_Debug* ar, Closure* f, } else { - ar->currentline = f->isC ? -1 : getlinedefined(f->l.p); + ar->currentline = f->isC ? -1 : f->l.p->linedefined; } break; @@ -424,7 +412,7 @@ static void getcoverage(Proto* p, int depth, int* buffer, size_t size, void* con } const char* debugname = p->debugname ? getstr(p->debugname) : NULL; - int linedefined = getlinedefined(p); + int linedefined = p->linedefined; callback(context, debugname, linedefined, depth, buffer, size); diff --git a/VM/src/lmem.h b/VM/src/lmem.h index 00788452..e552d739 100644 --- a/VM/src/lmem.h +++ b/VM/src/lmem.h @@ -10,12 +10,12 @@ union GCObject; #define luaM_newgco(L, t, size, memcat) cast_to(t*, luaM_newgco_(L, size, memcat)) #define luaM_freegco(L, p, size, memcat, page) luaM_freegco_(L, obj2gco(p), size, memcat, page) -#define luaM_arraysize_(n, e) ((cast_to(size_t, (n)) <= SIZE_MAX / (e)) ? (n) * (e) : (luaM_toobig(L), SIZE_MAX)) +#define luaM_arraysize_(L, n, e) ((cast_to(size_t, (n)) <= SIZE_MAX / (e)) ? (n) * (e) : (luaM_toobig(L), SIZE_MAX)) -#define luaM_newarray(L, n, t, memcat) cast_to(t*, luaM_new_(L, luaM_arraysize_(n, sizeof(t)), memcat)) +#define luaM_newarray(L, n, t, memcat) cast_to(t*, luaM_new_(L, luaM_arraysize_(L, n, sizeof(t)), memcat)) #define luaM_freearray(L, b, n, t, memcat) luaM_free_(L, (b), (n) * sizeof(t), memcat) #define luaM_reallocarray(L, v, oldn, n, t, memcat) \ - ((v) = cast_to(t*, luaM_realloc_(L, v, (oldn) * sizeof(t), luaM_arraysize_(n, sizeof(t)), memcat))) + ((v) = cast_to(t*, luaM_realloc_(L, v, (oldn) * sizeof(t), luaM_arraysize_(L, n, sizeof(t)), memcat))) LUAI_FUNC void* luaM_new_(lua_State* L, size_t nsize, uint8_t memcat); LUAI_FUNC GCObject* luaM_newgco_(lua_State* L, size_t nsize, uint8_t memcat); diff --git a/VM/src/ltable.cpp b/VM/src/ltable.cpp index 431501f3..1c75c0b0 100644 --- a/VM/src/ltable.cpp +++ b/VM/src/ltable.cpp @@ -2,17 +2,26 @@ // This code is based on Lua 5.x implementation licensed under MIT License; see lua_LICENSE.txt for details /* -** Implementation of tables (aka arrays, objects, or hash tables). -** Tables keep its elements in two parts: an array part and a hash part. -** Non-negative integer keys are all candidates to be kept in the array -** part. The actual size of the array is the largest `n' such that at -** least half the slots between 0 and n are in use. -** Hash uses a mix of chained scatter table with Brent's variation. -** A main invariant of these tables is that, if an element is not -** in its main position (i.e. the `original' position that its hash gives -** to it), then the colliding element is in its own main position. -** Hence even when the load factor reaches 100%, performance remains good. -*/ + * Implementation of tables (aka arrays, objects, or hash tables). + * + * Tables keep the elements in two parts: an array part and a hash part. + * Integer keys >=1 are all candidates to be kept in the array part. The actual size of the array is the + * largest n such that at least half the slots between 0 and n are in use. + * Hash uses a mix of chained scatter table with Brent's variation. + * + * A main invariant of these tables is that, if an element is not in its main position (i.e. the original + * position that its hash gives to it), then the colliding element is in its own main position. + * Hence even when the load factor reaches 100%, performance remains good. + * + * Table keys can be arbitrary values unless they contain NaN. Keys are hashed and compared using raw equality, + * so even if the key is a userdata with an overridden __eq, it's not used during hash lookups. + * + * Each table has a "boundary", defined as the index k where t[k] ~= nil and t[k+1] == nil. The boundary can be + * computed using a binary search and can be adjusted when the table is modified; crucially, Luau enforces an + * invariant where the boundary must be in the array part - this enforces a consistent iteration order through the + * prefix of the table when using pairs(), and allows to implement algorithms that access elements in 1..#t range + * more efficiently. + */ #include "ltable.h" @@ -25,6 +34,7 @@ #include LUAU_FASTFLAGVARIABLE(LuauTableRehashRework, false) +LUAU_FASTFLAGVARIABLE(LuauTableNewBoundary, false) // max size of both array and hash part is 2^MAXBITS #define MAXBITS 26 @@ -460,7 +470,20 @@ static void rehash(lua_State* L, Table* t, const TValue* ek) totaluse++; /* compute new size for array part */ int na = computesizes(nums, &nasize); + /* enforce the boundary invariant; for performance, only do hash lookups if we must */ + if (FFlag::LuauTableNewBoundary) + { + bool tbound = t->node != dummynode || nasize < t->sizearray; + int ekindex = ttisnumber(ek) ? arrayindex(nvalue(ek)) : -1; + /* move the array size up until the boundary is guaranteed to be inside the array part */ + while (nasize + 1 == ekindex || (tbound && !ttisnil(luaH_getnum(t, nasize + 1)))) + { + nasize++; + na++; + } + } /* resize the table to new computed sizes */ + LUAU_ASSERT(na <= totaluse); resize(L, t, nasize, totaluse - na); } @@ -520,10 +543,18 @@ static LuaNode* getfreepos(Table* t) */ static TValue* newkey(lua_State* L, Table* t, const TValue* key) { + /* enforce boundary invariant */ + if (FFlag::LuauTableNewBoundary && ttisnumber(key) && nvalue(key) == t->sizearray + 1) + { + rehash(L, t, key); /* grow table */ + + // after rehash, numeric keys might be located in the new array part, but won't be found in the node part + return arrayornewkey(L, t, key); + } + LuaNode* mp = mainposition(t, key); if (!ttisnil(gval(mp)) || mp == dummynode) { - LuaNode* othern; LuaNode* n = getfreepos(t); /* get a free place */ if (n == NULL) { /* cannot find a free place? */ @@ -542,7 +573,7 @@ static TValue* newkey(lua_State* L, Table* t, const TValue* key) LUAU_ASSERT(n != dummynode); TValue mk; getnodekey(L, &mk, mp); - othern = mainposition(t, &mk); + LuaNode* othern = mainposition(t, &mk); if (othern != mp) { /* is colliding node out of its main position? */ /* yes; move colliding node into free position */ @@ -704,6 +735,7 @@ TValue* luaH_setstr(lua_State* L, Table* t, TString* key) static LUAU_NOINLINE int unbound_search(Table* t, unsigned int j) { + LUAU_ASSERT(!FFlag::LuauTableNewBoundary); unsigned int i = j; /* i is zero or a present index */ j++; /* find `i' and `j' such that i is present and j is not */ @@ -788,6 +820,12 @@ int luaH_getn(Table* t) maybesetaboundary(t, boundary); return boundary; } + else if (FFlag::LuauTableNewBoundary) + { + /* validate boundary invariant */ + LUAU_ASSERT(t->node == dummynode || ttisnil(luaH_getnum(t, j + 1))); + return j; + } /* else must find a boundary in hash part */ else if (t->node == dummynode) /* hash part is empty? */ return j; /* that is easy... */ diff --git a/VM/src/ltablib.cpp b/VM/src/ltablib.cpp index 00753742..241a99e3 100644 --- a/VM/src/ltablib.cpp +++ b/VM/src/ltablib.cpp @@ -10,7 +10,9 @@ #include "ldebug.h" #include "lvm.h" -LUAU_FASTFLAGVARIABLE(LuauTableClone, false) +LUAU_DYNAMIC_FASTFLAGVARIABLE(LuauTableMoveTelemetry2, false) + +void (*lua_table_move_telemetry)(lua_State* L, int f, int e, int t, int nf, int nt); static int foreachi(lua_State* L) { @@ -197,6 +199,29 @@ static int tmove(lua_State* L) int tt = !lua_isnoneornil(L, 5) ? 5 : 1; /* destination table */ luaL_checktype(L, tt, LUA_TTABLE); + void (*telemetrycb)(lua_State* L, int f, int e, int t, int nf, int nt) = lua_table_move_telemetry; + + if (DFFlag::LuauTableMoveTelemetry2 && telemetrycb) + { + int nf = lua_objlen(L, 1); + int nt = lua_objlen(L, tt); + + bool report = false; + + // source index range must be in bounds in source table unless the table is empty (permits 1..#t moves) + if (!(f == 1 || (f >= 1 && f <= nf))) + report = true; + if (!(e == nf || (e >= 1 && e <= nf))) + report = true; + + // destination index must be in bounds in dest table or be exactly at the first empty element (permits concats) + if (!(t == nt + 1 || (t >= 1 && t <= nt))) + report = true; + + if (report) + telemetrycb(L, f, e, t, nf, nt); + } + if (e >= f) { /* otherwise, nothing to move */ luaL_argcheck(L, f > 0 || e < INT_MAX + f, 3, "too many elements to move"); @@ -512,9 +537,6 @@ static int tisfrozen(lua_State* L) static int tclone(lua_State* L) { - if (!FFlag::LuauTableClone) - luaG_runerror(L, "table.clone is not available"); - luaL_checktype(L, 1, LUA_TTABLE); luaL_argcheck(L, !luaL_getmetafield(L, 1, "__metatable"), 1, "table has a protected metatable"); diff --git a/VM/src/lvmexecute.cpp b/VM/src/lvmexecute.cpp index 96a87b7e..34949efb 100644 --- a/VM/src/lvmexecute.cpp +++ b/VM/src/lvmexecute.cpp @@ -16,6 +16,8 @@ #include +LUAU_FASTFLAG(LuauTableNewBoundary) + // Disable c99-designator to avoid the warning in CGOTO dispatch table #ifdef __clang__ #if __has_warning("-Wc99-designator") @@ -2266,9 +2268,9 @@ static void luau_execute(lua_State* L) VM_NEXT(); } } - else if (h->lsizenode == 0 && ttisnil(gval(h->node))) + else if (FFlag::LuauTableNewBoundary || (h->lsizenode == 0 && ttisnil(gval(h->node)))) { - // hash part is empty: fallthrough to exit + // fallthrough to exit VM_NEXT(); } else diff --git a/VM/src/lvmload.cpp b/VM/src/lvmload.cpp index 4e5435b7..8b742f1c 100644 --- a/VM/src/lvmload.cpp +++ b/VM/src/lvmload.cpp @@ -13,8 +13,6 @@ #include -LUAU_FASTFLAGVARIABLE(LuauBytecodeV2Force, false) - // TODO: RAII deallocation doesn't work for longjmp builds if a memory error happens template struct TempBuffer @@ -156,12 +154,11 @@ int luau_load(lua_State* L, const char* chunkname, const char* data, size_t size return 1; } - if (FFlag::LuauBytecodeV2Force ? (version != LBC_VERSION_FUTURE) : (version != LBC_VERSION && version != LBC_VERSION_FUTURE)) + if (version != LBC_VERSION) { char chunkid[LUA_IDSIZE]; luaO_chunkid(chunkid, chunkname, LUA_IDSIZE); - lua_pushfstring(L, "%s: bytecode version mismatch (expected %d, got %d)", chunkid, - FFlag::LuauBytecodeV2Force ? LBC_VERSION_FUTURE : LBC_VERSION, version); + lua_pushfstring(L, "%s: bytecode version mismatch (expected %d, got %d)", chunkid, LBC_VERSION, version); return 1; } @@ -292,11 +289,7 @@ int luau_load(lua_State* L, const char* chunkname, const char* data, size_t size p->p[j] = protos[fid]; } - if (FFlag::LuauBytecodeV2Force || version == LBC_VERSION_FUTURE) - p->linedefined = readVarInt(data, size, offset); - else - p->linedefined = -1; - + p->linedefined = readVarInt(data, size, offset); p->debugname = readString(strings, data, size, offset); uint8_t lineinfo = read(data, size, offset); diff --git a/tests/Autocomplete.test.cpp b/tests/Autocomplete.test.cpp index 1db782cc..4e8a1d55 100644 --- a/tests/Autocomplete.test.cpp +++ b/tests/Autocomplete.test.cpp @@ -14,7 +14,6 @@ LUAU_FASTFLAG(LuauTraceTypesInNonstrictMode2) LUAU_FASTFLAG(LuauSetMetatableDoesNotTimeTravel) -LUAU_FASTFLAG(LuauTableCloneType) using namespace Luau; @@ -262,7 +261,7 @@ TEST_CASE_FIXTURE(ACFixture, "get_member_completions") auto ac = autocomplete('1'); - CHECK_EQ(FFlag::LuauTableCloneType ? 17 : 16, ac.entryMap.size()); + CHECK_EQ(17, ac.entryMap.size()); CHECK(ac.entryMap.count("find")); CHECK(ac.entryMap.count("pack")); CHECK(!ac.entryMap.count("math")); @@ -2221,7 +2220,7 @@ TEST_CASE_FIXTURE(ACFixture, "autocompleteSource") auto ac = autocompleteSource(frontend, source, Position{1, 24}, nullCallback).result; - CHECK_EQ(FFlag::LuauTableCloneType ? 17 : 16, ac.entryMap.size()); + CHECK_EQ(17, ac.entryMap.size()); CHECK(ac.entryMap.count("find")); CHECK(ac.entryMap.count("pack")); CHECK(!ac.entryMap.count("math")); diff --git a/tests/Conformance.test.cpp b/tests/Conformance.test.cpp index 83d4518d..0ed7dc44 100644 --- a/tests/Conformance.test.cpp +++ b/tests/Conformance.test.cpp @@ -241,8 +241,6 @@ TEST_CASE("Math") TEST_CASE("Table") { - ScopedFastFlag sff("LuauTableClone", true); - runConformance("nextvar.lua"); } @@ -467,8 +465,6 @@ static void populateRTTI(lua_State* L, Luau::TypeId type) TEST_CASE("Types") { - ScopedFastFlag sff("LuauTableCloneType", true); - runConformance("types.lua", [](lua_State* L) { Luau::NullModuleResolver moduleResolver; Luau::InternalErrorReporter iceHandler; diff --git a/tests/Parser.test.cpp b/tests/Parser.test.cpp index 7f6a6c0d..7dacc669 100644 --- a/tests/Parser.test.cpp +++ b/tests/Parser.test.cpp @@ -1604,6 +1604,20 @@ TEST_CASE_FIXTURE(Fixture, "end_extent_of_functions_unions_and_intersections") CHECK_EQ((Position{3, 42}), block->body.data[2]->location.end); } +TEST_CASE_FIXTURE(Fixture, "end_extent_doesnt_consume_comments") +{ + ScopedFastFlag luauParseLocationIgnoreCommentSkip{"LuauParseLocationIgnoreCommentSkip", true}; + + AstStatBlock* block = parse(R"( + type F = number + --comment + print('hello') + )"); + + REQUIRE_EQ(2, block->body.size); + CHECK_EQ((Position{1, 23}), block->body.data[0]->location.end); +} + TEST_CASE_FIXTURE(Fixture, "parse_error_loop_control") { matchParseError("break", "break statement must be inside a loop"); diff --git a/tests/TypeInfer.functions.test.cpp b/tests/TypeInfer.functions.test.cpp index da4ea074..dbae7b54 100644 --- a/tests/TypeInfer.functions.test.cpp +++ b/tests/TypeInfer.functions.test.cpp @@ -1270,7 +1270,7 @@ caused by: TEST_CASE_FIXTURE(Fixture, "function_decl_quantify_right_type") { - ScopedFastFlag statFunctionSimplify{"LuauStatFunctionSimplify2", true}; + ScopedFastFlag statFunctionSimplify{"LuauStatFunctionSimplify3", true}; fileResolver.source["game/isAMagicMock"] = R"( --!nonstrict @@ -1294,7 +1294,7 @@ end TEST_CASE_FIXTURE(Fixture, "function_decl_non_self_sealed_overwrite") { - ScopedFastFlag statFunctionSimplify{"LuauStatFunctionSimplify2", true}; + ScopedFastFlag statFunctionSimplify{"LuauStatFunctionSimplify3", true}; CheckResult result = check(R"( function string.len(): number @@ -1302,7 +1302,40 @@ function string.len(): number end )"); - LUAU_REQUIRE_ERRORS(result); + LUAU_REQUIRE_NO_ERRORS(result); + + // if 'string' library property was replaced with an internal module type, it will be freed and the next check will crash + frontend.clear(); + + result = check(R"( +print(string.len('hello')) + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "function_decl_non_self_sealed_overwrite_2") +{ + ScopedFastFlag statFunctionSimplify{"LuauStatFunctionSimplify3", true}; + ScopedFastFlag inferStatFunction{"LuauInferStatFunction", true}; + + CheckResult result = check(R"( +local t: { f: ((x: number) -> number)? } = {} + +function t.f(x) + print(x + 5) + return x .. "asd" +end + +t.f = function(x) + print(x + 5) + return x .. "asd" +end + )"); + + LUAU_REQUIRE_ERROR_COUNT(2, result); + CHECK_EQ(toString(result.errors[0]), R"(Type 'string' could not be converted into 'number')"); + CHECK_EQ(toString(result.errors[1]), R"(Type 'string' could not be converted into 'number')"); } TEST_CASE_FIXTURE(Fixture, "strict_mode_ok_with_missing_arguments") @@ -1319,7 +1352,7 @@ TEST_CASE_FIXTURE(Fixture, "strict_mode_ok_with_missing_arguments") TEST_CASE_FIXTURE(Fixture, "function_statement_sealed_table_assignment_through_indexer") { - ScopedFastFlag statFunctionSimplify{"LuauStatFunctionSimplify2", true}; + ScopedFastFlag statFunctionSimplify{"LuauStatFunctionSimplify3", true}; CheckResult result = check(R"( local t: {[string]: () -> number} = {} diff --git a/tests/conformance/nextvar.lua b/tests/conformance/nextvar.lua index e85fcbe8..ab9be42c 100644 --- a/tests/conformance/nextvar.lua +++ b/tests/conformance/nextvar.lua @@ -550,4 +550,35 @@ do assert(not pcall(table.clone, 42)) end +-- test boundary invariant maintenance during rehash +do + local arr = table.create(5, 42) + + arr[1] = nil + arr.a = 'a' -- trigger rehash + + assert(#arr == 5) -- technically 0 is also valid, but it happens to be 5 because array capacity is 5 +end + +-- test boundary invariant maintenance when replacing hash keys +do + local arr = {} + arr.a = 'a' + arr.a = nil + arr[1] = 1 -- should rehash and resize array part, otherwise # won't find the boundary in array part + + assert(#arr == 1) +end + +-- test boundary invariant maintenance when table is filled from the end +do + local arr = {} + for i=5,2,-1 do + arr[i] = i + assert(#arr == 0) + end + arr[1] = 1 + assert(#arr == 5) +end + return"OK" From 916c83fdc47c03bcf238a8b7dfbb3e9a4385f842 Mon Sep 17 00:00:00 2001 From: Alan Jeffrey <403333+asajeffrey@users.noreply.github.com> Date: Thu, 31 Mar 2022 18:29:42 -0500 Subject: [PATCH 215/399] Prototype: Added a discussion of set-theoretic models of subtyping (#431) * Added a discussion of set-theoretic models of subtyping to the prototype --- prototyping/Properties/Subtyping.agda | 90 +++++++++++++++++++++++++++ 1 file changed, 90 insertions(+) diff --git a/prototyping/Properties/Subtyping.agda b/prototyping/Properties/Subtyping.agda index 6a0b4203..0a20f244 100644 --- a/prototyping/Properties/Subtyping.agda +++ b/prototyping/Properties/Subtyping.agda @@ -9,6 +9,7 @@ open import Luau.Type using (Type; Scalar; strict; nil; number; string; boolean; open import Properties.Contradiction using (CONTRADICTION; ¬) open import Properties.Equality using (_≢_) open import Properties.Functions using (_∘_) +open import Properties.Product using (_×_; _,_) src = Luau.Type.src strict @@ -219,3 +220,92 @@ any-≮:-none = witness (scalar nil) any none function-≮:-none : ∀ {T U} → ((T ⇒ U) ≮: none) function-≮:-none = witness function function none + +-- A Gentle Introduction To Semantic Subtyping (https://www.cduce.org/papers/gentle.pdf) +-- defines a "set-theoretic" model (sec 2.5) +-- Unfortunately we don't quite have this property, due to uninhabited types, +-- for example (none -> T) is equivalent to (none -> U) +-- when types are interpreted as sets of syntactic values. + +_⊆_ : ∀ {A : Set} → (A → Set) → (A → Set) → Set +(P ⊆ Q) = ∀ a → (P a) → (Q a) + +_⊗_ : ∀ {A B : Set} → (A → Set) → (B → Set) → ((A × B) → Set) +(P ⊗ Q) (a , b) = (P a) × (Q b) + +Comp : ∀ {A : Set} → (A → Set) → (A → Set) +Comp P a = ¬(P a) + +set-theoretic-if : ∀ {S₁ T₁ S₂ T₂} → + + -- This is the "if" part of being a set-theoretic model + (Language (S₁ ⇒ T₁) ⊆ Language (S₂ ⇒ T₂)) → + (∀ Q → Q ⊆ Comp((Language S₁) ⊗ Comp(Language T₁)) → Q ⊆ Comp((Language S₂) ⊗ Comp(Language T₂))) + +set-theoretic-if {S₁} {T₁} {S₂} {T₂} p Q q (t , u) Qtu (S₂t , ¬T₂u) = q (t , u) Qtu (S₁t , ¬T₁u) where + + S₁t : Language S₁ t + S₁t with dec-language S₁ t + S₁t | Left ¬S₁t with p (function-err t) (function-err ¬S₁t) + S₁t | Left ¬S₁t | function-err ¬S₂t = CONTRADICTION (language-comp t ¬S₂t S₂t) + S₁t | Right r = r + + ¬T₁u : ¬(Language T₁ u) + ¬T₁u T₁u with p (function-ok u) (function-ok T₁u) + ¬T₁u T₁u | function-ok T₂u = ¬T₂u T₂u + +not-quite-set-theoretic-only-if : ∀ {S₁ T₁ S₂ T₂} → + + -- We don't quite have that this is a set-theoretic model + -- it's only true when Language T₁ and ¬Language T₂ t₂ are inhabited + -- in particular it's not true when T₁ is none, or T₂ is any. + ∀ s₂ t₂ → Language S₂ s₂ → ¬Language T₂ t₂ → + + -- This is the "only if" part of being a set-theoretic model + (∀ Q → Q ⊆ Comp((Language S₁) ⊗ Comp(Language T₁)) → Q ⊆ Comp((Language S₂) ⊗ Comp(Language T₂))) → + (Language (S₁ ⇒ T₁) ⊆ Language (S₂ ⇒ T₂)) + +not-quite-set-theoretic-only-if {S₁} {T₁} {S₂} {T₂} s₂ t₂ S₂s₂ ¬T₂t₂ p = r where + + Q : (Tree × Tree) → Set + Q (t , u) = Either (¬Language S₁ t) (Language T₁ u) + + q : Q ⊆ Comp((Language S₁) ⊗ Comp(Language T₁)) + q (t , u) (Left ¬S₁t) (S₁t , ¬T₁u) = language-comp t ¬S₁t S₁t + q (t , u) (Right T₂u) (S₁t , ¬T₁u) = ¬T₁u T₂u + + r : Language (S₁ ⇒ T₁) ⊆ Language (S₂ ⇒ T₂) + r function function = function + r (function-err t) (function-err ¬S₁t) with dec-language S₂ t + r (function-err t) (function-err ¬S₁t) | Left ¬S₂t = function-err ¬S₂t + r (function-err t) (function-err ¬S₁t) | Right S₂t = CONTRADICTION (p Q q (t , t₂) (Left ¬S₁t) (S₂t , language-comp t₂ ¬T₂t₂)) + r (function-ok t) (function-ok T₁t) with dec-language T₂ t + r (function-ok t) (function-ok T₁t) | Left ¬T₂t = CONTRADICTION (p Q q (s₂ , t) (Right T₁t) (S₂s₂ , language-comp t ¬T₂t)) + r (function-ok t) (function-ok T₁t) | Right T₂t = function-ok T₂t + +-- A counterexample when the argument type is empty. + +set-theoretic-counterexample-one : (∀ Q → Q ⊆ Comp((Language none) ⊗ Comp(Language number)) → Q ⊆ Comp((Language none) ⊗ Comp(Language string))) +set-theoretic-counterexample-one Q q ((scalar s) , u) Qtu (scalar () , p) +set-theoretic-counterexample-one Q q ((function-err t) , u) Qtu (scalar-function-err () , p) + +set-theoretic-counterexample-two : (none ⇒ number) ≮: (none ⇒ string) +set-theoretic-counterexample-two = witness + (function-ok (scalar number)) (function-ok (scalar number)) + (function-ok (scalar-scalar number string (λ ()))) + +-- At some point we may deal with overloaded function resolution, which should fix this problem... +-- The reason why this is connected to overloaded functions is that currently we have that the type of +-- f(x) is (tgt T) where f:T. Really we should have the type depend on the type of x, that is use (tgt T U), +-- where U is the type of x. In particular (tgt (S => T) (U & V)) should be the same as (tgt ((S&U) => T) V) +-- and tgt(none => T) should be any. For example +-- +-- tgt((number => string) & (string => bool))(number) +-- is tgt(number => string)(number) & tgt(string => bool)(number) +-- is tgt(number => string)(number) & tgt(string => bool)(number&any) +-- is tgt(number => string)(number) & tgt(string&number => bool)(any) +-- is tgt(number => string)(number) & tgt(none => bool)(any) +-- is string & any +-- is string +-- +-- there's some discussion of this in the Gentle Introduction paper. From dc32a3253ed87ac817ccd6652f536f6f18d347e8 Mon Sep 17 00:00:00 2001 From: Alan Jeffrey <403333+asajeffrey@users.noreply.github.com> Date: Thu, 31 Mar 2022 18:44:41 -0500 Subject: [PATCH 216/399] Add short example of width subtyping (#444) --- docs/_pages/typecheck.md | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/docs/_pages/typecheck.md b/docs/_pages/typecheck.md index 9443f112..20f73987 100644 --- a/docs/_pages/typecheck.md +++ b/docs/_pages/typecheck.md @@ -137,6 +137,15 @@ local t = {x = 1} -- {x: number} t.y = 2 -- not ok ``` +Sealed tables support *width subtyping*, which allows a table with more properties to be used as a table with fewer + +```lua +type Point1D = { x : number } +type Point2D = { x : number, y : number } +local p : Point2D = { x = 5, y = 37 } +local q : Point1D = p -- ok because Point2D has more properties than Point1D +``` + ### Generic tables This typically occurs when the symbol does not have any annotated types or were not inferred anything concrete. In this case, when you index on a parameter, you're requesting that there is a table with a matching interface. From ffff25a9e502df4555c3bb30feb5f3625fd7d135 Mon Sep 17 00:00:00 2001 From: Alexander McCord <11488393+alexmccord@users.noreply.github.com> Date: Thu, 7 Apr 2022 09:16:44 -0700 Subject: [PATCH 217/399] Improve the UX of reading tagged unions a smidge. (#449) The page is a little narrow, and having to scroll on this horizontally isn't too nice. This fixes the UX for this specific part. --- docs/_pages/typecheck.md | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/docs/_pages/typecheck.md b/docs/_pages/typecheck.md index 20f73987..63e4c8bb 100644 --- a/docs/_pages/typecheck.md +++ b/docs/_pages/typecheck.md @@ -392,18 +392,20 @@ local account: Account = Account.new("Alexander", 500) Tagged unions are just union types! In particular, they're union types of tables where they have at least _some_ common properties but the structure of the tables are different enough. Here's one example: ```lua -type Result = { type: "ok", value: T } | { type: "err", error: E } +type Ok = { type: "ok", value: T } +type Err = { type: "err", error: E } +type Result = Ok | Err ``` This `Result` type can be discriminated by using type refinements on the property `type`, like so: ```lua if result.type == "ok" then - -- result is known to be { type: "ok", value: T } + -- result is known to be Ok -- and attempting to index for error here will fail print(result.value) elseif result.type == "err" then - -- result is known to be { type: "err", error: E } + -- result is known to be Err -- and attempting to index for value here will fail print(result.error) end From d42a5dbe48bdc597d0a8a2c79e52f3a6d908169d Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Thu, 7 Apr 2022 13:53:47 -0700 Subject: [PATCH 218/399] Sync to upstream/release/522 --- Analysis/include/Luau/Clone.h | 25 ++ Analysis/include/Luau/Frontend.h | 23 +- Analysis/include/Luau/Module.h | 29 +- Analysis/include/Luau/TypeInfer.h | 10 + Analysis/include/Luau/TypeVar.h | 2 + Analysis/include/Luau/Variant.h | 36 +- Analysis/src/Autocomplete.cpp | 117 +++++- Analysis/src/Clone.cpp | 371 ++++++++++++++++++ Analysis/src/Error.cpp | 1 + Analysis/src/Frontend.cpp | 144 +++++-- Analysis/src/IostreamHelpers.cpp | 419 +++++++++------------ Analysis/src/Module.cpp | 359 +----------------- Analysis/src/TxnLog.cpp | 31 +- Analysis/src/TypeInfer.cpp | 156 +++++--- Analysis/src/TypeVar.cpp | 4 + Ast/include/Luau/TimeTrace.h | 11 +- Ast/src/Parser.cpp | 11 + Ast/src/TimeTrace.cpp | 19 +- Compiler/src/ConstantFolding.cpp | 3 +- Sources.cmake | 2 + VM/src/lfunc.h | 2 +- VM/src/ltablib.cpp | 2 +- tests/Autocomplete.test.cpp | 139 ++++++- tests/Compiler.test.cpp | 33 ++ tests/Fixture.cpp | 5 +- tests/Fixture.h | 2 +- tests/Frontend.test.cpp | 64 ++++ tests/Module.test.cpp | 1 + tests/Parser.test.cpp | 21 ++ tests/TypeInfer.functions.test.cpp | 39 +- tests/TypeInfer.intersectionTypes.test.cpp | 39 +- tests/TypeInfer.primitives.test.cpp | 2 + tests/TypeInfer.tables.test.cpp | 15 + tests/TypeInfer.tryUnify.test.cpp | 26 ++ tests/TypeInfer.unionTypes.test.cpp | 25 ++ tools/lldb_formatters.py | 4 +- 36 files changed, 1407 insertions(+), 785 deletions(-) create mode 100644 Analysis/include/Luau/Clone.h create mode 100644 Analysis/src/Clone.cpp diff --git a/Analysis/include/Luau/Clone.h b/Analysis/include/Luau/Clone.h new file mode 100644 index 00000000..917ef801 --- /dev/null +++ b/Analysis/include/Luau/Clone.h @@ -0,0 +1,25 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +#pragma once + +#include "Luau/TypeVar.h" + +#include + +namespace Luau +{ + +// Only exposed so they can be unit tested. +using SeenTypes = std::unordered_map; +using SeenTypePacks = std::unordered_map; + +struct CloneState +{ + int recursionCount = 0; + bool encounteredFreeType = false; +}; + +TypePackId clone(TypePackId tp, TypeArena& dest, SeenTypes& seenTypes, SeenTypePacks& seenTypePacks, CloneState& cloneState); +TypeId clone(TypeId tp, TypeArena& dest, SeenTypes& seenTypes, SeenTypePacks& seenTypePacks, CloneState& cloneState); +TypeFun clone(const TypeFun& typeFun, TypeArena& dest, SeenTypes& seenTypes, SeenTypePacks& seenTypePacks, CloneState& cloneState); + +} // namespace Luau diff --git a/Analysis/include/Luau/Frontend.h b/Analysis/include/Luau/Frontend.h index 0bf8f362..2266f548 100644 --- a/Analysis/include/Luau/Frontend.h +++ b/Analysis/include/Luau/Frontend.h @@ -12,6 +12,8 @@ #include #include +LUAU_FASTFLAG(LuauSeparateTypechecks) + namespace Luau { @@ -55,10 +57,19 @@ std::optional pathExprToModuleName(const ModuleName& currentModuleNa struct SourceNode { + bool isDirty(bool forAutocomplete) const + { + if (FFlag::LuauSeparateTypechecks) + return forAutocomplete ? dirtyAutocomplete : dirty; + else + return dirty; + } + ModuleName name; std::unordered_set requires; std::vector> requireLocations; bool dirty = true; + bool dirtyAutocomplete = true; }; struct FrontendOptions @@ -71,12 +82,16 @@ struct FrontendOptions // When true, we run typechecking twice, once in the regular mode, and once in strict mode // in order to get more precise type information (e.g. for autocomplete). - bool typecheckTwice = false; + bool typecheckTwice_DEPRECATED = false; + + // Run typechecking only in mode required for autocomplete (strict mode in order to get more precise type information) + bool forAutocomplete = false; }; struct CheckResult { std::vector errors; + std::vector timeoutHits; }; struct FrontendModuleResolver : ModuleResolver @@ -123,7 +138,7 @@ struct Frontend CheckResult check(const SourceModule& module); // OLD. TODO KILL LintResult lint(const SourceModule& module, std::optional enabledLintWarnings = {}); - bool isDirty(const ModuleName& name) const; + bool isDirty(const ModuleName& name, bool forAutocomplete = false) const; void markDirty(const ModuleName& name, std::vector* markedDirty = nullptr); /** Borrow a pointer into the SourceModule cache. @@ -147,10 +162,10 @@ struct Frontend void applyBuiltinDefinitionToEnvironment(const std::string& environmentName, const std::string& definitionName); private: - std::pair getSourceNode(CheckResult& checkResult, const ModuleName& name); + std::pair getSourceNode(CheckResult& checkResult, const ModuleName& name, bool forAutocomplete); SourceModule parse(const ModuleName& name, std::string_view src, const ParseOptions& parseOptions); - bool parseGraph(std::vector& buildQueue, CheckResult& checkResult, const ModuleName& root); + bool parseGraph(std::vector& buildQueue, CheckResult& checkResult, const ModuleName& root, bool forAutocomplete); static LintResult classifyLints(const std::vector& warnings, const Config& config); diff --git a/Analysis/include/Luau/Module.h b/Analysis/include/Luau/Module.h index 6c689b7c..9a32f614 100644 --- a/Analysis/include/Luau/Module.h +++ b/Analysis/include/Luau/Module.h @@ -29,8 +29,8 @@ struct SourceModule std::optional environmentName; bool cyclic = false; - std::unique_ptr allocator; - std::unique_ptr names; + std::shared_ptr allocator; + std::shared_ptr names; std::vector parseErrors; AstStatBlock* root = nullptr; @@ -48,6 +48,12 @@ struct SourceModule bool isWithinComment(const SourceModule& sourceModule, Position pos); +struct RequireCycle +{ + Location location; + std::vector path; // one of the paths for a require() to go all the way back to the originating module +}; + struct TypeArena { TypedAllocator typeVars; @@ -77,20 +83,6 @@ struct TypeArena void freeze(TypeArena& arena); void unfreeze(TypeArena& arena); -// Only exposed so they can be unit tested. -using SeenTypes = std::unordered_map; -using SeenTypePacks = std::unordered_map; - -struct CloneState -{ - int recursionCount = 0; - bool encounteredFreeType = false; -}; - -TypePackId clone(TypePackId tp, TypeArena& dest, SeenTypes& seenTypes, SeenTypePacks& seenTypePacks, CloneState& cloneState); -TypeId clone(TypeId tp, TypeArena& dest, SeenTypes& seenTypes, SeenTypePacks& seenTypePacks, CloneState& cloneState); -TypeFun clone(const TypeFun& typeFun, TypeArena& dest, SeenTypes& seenTypes, SeenTypePacks& seenTypePacks, CloneState& cloneState); - struct Module { ~Module(); @@ -98,6 +90,10 @@ struct Module TypeArena interfaceTypes; TypeArena internalTypes; + // Scopes and AST types refer to parse data, so we need to keep that alive + std::shared_ptr allocator; + std::shared_ptr names; + std::vector> scopes; // never empty DenseHashMap astTypes{nullptr}; @@ -109,6 +105,7 @@ struct Module ErrorVec errors; Mode mode; SourceCode::Type type; + bool timeout = false; ScopePtr getModuleScope() const; diff --git a/Analysis/include/Luau/TypeInfer.h b/Analysis/include/Luau/TypeInfer.h index 839043cc..215da67f 100644 --- a/Analysis/include/Luau/TypeInfer.h +++ b/Analysis/include/Luau/TypeInfer.h @@ -124,6 +124,12 @@ struct HashBoolNamePair size_t operator()(const std::pair& pair) const; }; +class TimeLimitError : public std::exception +{ +public: + virtual const char* what() const throw(); +}; + // All TypeVars are retained via Environment::typeVars. All TypeIds // within a program are borrowed pointers into this set. struct TypeChecker @@ -413,6 +419,10 @@ public: UnifierSharedState unifierState; + std::vector requireCycles; + + std::optional finishTime; + public: const TypeId nilType; const TypeId numberType; diff --git a/Analysis/include/Luau/TypeVar.h b/Analysis/include/Luau/TypeVar.h index b8c4b362..f61e4044 100644 --- a/Analysis/include/Luau/TypeVar.h +++ b/Analysis/include/Luau/TypeVar.h @@ -513,6 +513,8 @@ struct SingletonTypes const TypeId stringType; const TypeId booleanType; const TypeId threadType; + const TypeId trueType; + const TypeId falseType; const TypeId anyType; const TypeId optionalNumberType; diff --git a/Analysis/include/Luau/Variant.h b/Analysis/include/Luau/Variant.h index 63d5a65c..5efe89ed 100644 --- a/Analysis/include/Luau/Variant.h +++ b/Analysis/include/Luau/Variant.h @@ -2,45 +2,14 @@ #pragma once #include "Luau/Common.h" - -#ifndef LUAU_USE_STD_VARIANT -#define LUAU_USE_STD_VARIANT 0 -#endif - -#if LUAU_USE_STD_VARIANT -#include -#else #include #include #include #include -#endif namespace Luau { -#if LUAU_USE_STD_VARIANT -template -using Variant = std::variant; - -template -auto visit(Visitor&& vis, Variant&& var) -{ - // This change resolves the ABI issues with std::variant on libc++; std::visit normally throws bad_variant_access - // but it requires an update to libc++.dylib which ships with macOS 10.14. To work around this, we assert on valueless - // variants since we will never generate them and call into a libc++ function that doesn't throw. - LUAU_ASSERT(!var.valueless_by_exception()); - -#ifdef __APPLE__ - // See https://stackoverflow.com/a/53868971/503215 - return std::__variant_detail::__visitation::__variant::__visit_value(vis, var); -#else - return std::visit(vis, var); -#endif -} - -using std::get_if; -#else template class Variant { @@ -248,6 +217,8 @@ static void fnVisitV(Visitor& vis, std::conditional_t, const template auto visit(Visitor&& vis, const Variant& var) { + static_assert(std::conjunction_v...>, "visitor must accept every alternative as an argument"); + using Result = std::invoke_result_t::first_alternative>; static_assert(std::conjunction_v>...>, "visitor result type must be consistent between alternatives"); @@ -273,6 +244,8 @@ auto visit(Visitor&& vis, const Variant& var) template auto visit(Visitor&& vis, Variant& var) { + static_assert(std::conjunction_v...>, "visitor must accept every alternative as an argument"); + using Result = std::invoke_result_t::first_alternative&>; static_assert(std::conjunction_v>...>, "visitor result type must be consistent between alternatives"); @@ -294,7 +267,6 @@ auto visit(Visitor&& vis, Variant& var) return res; } } -#endif template inline constexpr bool always_false_v = false; diff --git a/Analysis/src/Autocomplete.cpp b/Analysis/src/Autocomplete.cpp index 492edf25..b7201ab3 100644 --- a/Analysis/src/Autocomplete.cpp +++ b/Analysis/src/Autocomplete.cpp @@ -14,6 +14,7 @@ #include LUAU_FASTFLAGVARIABLE(LuauIfElseExprFixCompletionIssue, false); +LUAU_FASTFLAGVARIABLE(LuauAutocompleteSingletonTypes, false); LUAU_FASTFLAG(LuauSelfCallAutocompleteFix) static const std::unordered_set kStatementStartingKeywords = { @@ -625,6 +626,31 @@ AutocompleteEntryMap autocompleteModuleTypes(const Module& module, Position posi return result; } +static void autocompleteStringSingleton(TypeId ty, bool addQuotes, AutocompleteEntryMap& result) +{ + auto formatKey = [addQuotes](const std::string& key) { + if (addQuotes) + return "\"" + escape(key) + "\""; + + return escape(key); + }; + + ty = follow(ty); + + if (auto ss = get(get(ty))) + { + result[formatKey(ss->value)] = AutocompleteEntry{AutocompleteEntryKind::String, ty, false, false, TypeCorrectKind::Correct}; + } + else if (auto uty = get(ty)) + { + for (auto el : uty) + { + if (auto ss = get(get(el))) + result[formatKey(ss->value)] = AutocompleteEntry{AutocompleteEntryKind::String, ty, false, false, TypeCorrectKind::Correct}; + } + } +}; + static bool canSuggestInferredType(ScopePtr scope, TypeId ty) { ty = follow(ty); @@ -1309,17 +1335,38 @@ static void autocompleteExpression(const SourceModule& sourceModule, const Modul scope = scope->parent; } - TypeCorrectKind correctForNil = checkTypeCorrectKind(module, typeArena, node, position, typeChecker.nilType); - TypeCorrectKind correctForBoolean = checkTypeCorrectKind(module, typeArena, node, position, typeChecker.booleanType); - TypeCorrectKind correctForFunction = - functionIsExpectedAt(module, node, position).value_or(false) ? TypeCorrectKind::Correct : TypeCorrectKind::None; + if (FFlag::LuauAutocompleteSingletonTypes) + { + TypeCorrectKind correctForNil = checkTypeCorrectKind(module, typeArena, node, position, typeChecker.nilType); + TypeCorrectKind correctForTrue = checkTypeCorrectKind(module, typeArena, node, position, getSingletonTypes().trueType); + TypeCorrectKind correctForFalse = checkTypeCorrectKind(module, typeArena, node, position, getSingletonTypes().falseType); + TypeCorrectKind correctForFunction = + functionIsExpectedAt(module, node, position).value_or(false) ? TypeCorrectKind::Correct : TypeCorrectKind::None; - result["if"] = {AutocompleteEntryKind::Keyword, std::nullopt, false, false}; - result["true"] = {AutocompleteEntryKind::Keyword, typeChecker.booleanType, false, false, correctForBoolean}; - result["false"] = {AutocompleteEntryKind::Keyword, typeChecker.booleanType, false, false, correctForBoolean}; - result["nil"] = {AutocompleteEntryKind::Keyword, typeChecker.nilType, false, false, correctForNil}; - result["not"] = {AutocompleteEntryKind::Keyword}; - result["function"] = {AutocompleteEntryKind::Keyword, std::nullopt, false, false, correctForFunction}; + result["if"] = {AutocompleteEntryKind::Keyword, std::nullopt, false, false}; + result["true"] = {AutocompleteEntryKind::Keyword, typeChecker.booleanType, false, false, correctForTrue}; + result["false"] = {AutocompleteEntryKind::Keyword, typeChecker.booleanType, false, false, correctForFalse}; + result["nil"] = {AutocompleteEntryKind::Keyword, typeChecker.nilType, false, false, correctForNil}; + result["not"] = {AutocompleteEntryKind::Keyword}; + result["function"] = {AutocompleteEntryKind::Keyword, std::nullopt, false, false, correctForFunction}; + + if (auto ty = findExpectedTypeAt(module, node, position)) + autocompleteStringSingleton(*ty, true, result); + } + else + { + TypeCorrectKind correctForNil = checkTypeCorrectKind(module, typeArena, node, position, typeChecker.nilType); + TypeCorrectKind correctForBoolean = checkTypeCorrectKind(module, typeArena, node, position, typeChecker.booleanType); + TypeCorrectKind correctForFunction = + functionIsExpectedAt(module, node, position).value_or(false) ? TypeCorrectKind::Correct : TypeCorrectKind::None; + + result["if"] = {AutocompleteEntryKind::Keyword, std::nullopt, false, false}; + result["true"] = {AutocompleteEntryKind::Keyword, typeChecker.booleanType, false, false, correctForBoolean}; + result["false"] = {AutocompleteEntryKind::Keyword, typeChecker.booleanType, false, false, correctForBoolean}; + result["nil"] = {AutocompleteEntryKind::Keyword, typeChecker.nilType, false, false, correctForNil}; + result["not"] = {AutocompleteEntryKind::Keyword}; + result["function"] = {AutocompleteEntryKind::Keyword, std::nullopt, false, false, correctForFunction}; + } } } @@ -1625,17 +1672,33 @@ static AutocompleteResult autocomplete(const SourceModule& sourceModule, const M } else if (node->is()) { + AutocompleteEntryMap result; + + if (FFlag::LuauAutocompleteSingletonTypes) + { + if (auto it = module->astExpectedTypes.find(node->asExpr())) + autocompleteStringSingleton(*it, false, result); + } + if (finder.ancestry.size() >= 2) { if (auto idxExpr = finder.ancestry.at(finder.ancestry.size() - 2)->as()) { if (auto it = module->astTypes.find(idxExpr->expr)) + autocompleteProps(*module, typeArena, follow(*it), PropIndexType::Point, finder.ancestry, result); + } + else if (auto binExpr = finder.ancestry.at(finder.ancestry.size() - 2)->as(); + binExpr && FFlag::LuauAutocompleteSingletonTypes) + { + if (binExpr->op == AstExprBinary::CompareEq || binExpr->op == AstExprBinary::CompareNe) { - return {autocompleteProps(*module, typeArena, follow(*it), PropIndexType::Point, finder.ancestry), finder.ancestry}; + if (auto it = module->astTypes.find(node == binExpr->left ? binExpr->right : binExpr->left)) + autocompleteStringSingleton(*it, false, result); } } } - return {}; + + return {result, finder.ancestry}; } if (node->is()) @@ -1653,18 +1716,31 @@ static AutocompleteResult autocomplete(const SourceModule& sourceModule, const M AutocompleteResult autocomplete(Frontend& frontend, const ModuleName& moduleName, Position position, StringCompletionCallback callback) { - // FIXME: We can improve performance here by parsing without checking. - // The old type graph is probably fine. (famous last words!) - // FIXME: We don't need to typecheck for script analysis here, just for autocomplete. - frontend.check(moduleName); + if (FFlag::LuauSeparateTypechecks) + { + // FIXME: We can improve performance here by parsing without checking. + // The old type graph is probably fine. (famous last words!) + FrontendOptions opts; + opts.forAutocomplete = true; + frontend.check(moduleName, opts); + } + else + { + // FIXME: We can improve performance here by parsing without checking. + // The old type graph is probably fine. (famous last words!) + // FIXME: We don't need to typecheck for script analysis here, just for autocomplete. + frontend.check(moduleName); + } const SourceModule* sourceModule = frontend.getSourceModule(moduleName); if (!sourceModule) return {}; - TypeChecker& typeChecker = (frontend.options.typecheckTwice ? frontend.typeCheckerForAutocomplete : frontend.typeChecker); - ModulePtr module = (frontend.options.typecheckTwice ? frontend.moduleResolverForAutocomplete.getModule(moduleName) - : frontend.moduleResolver.getModule(moduleName)); + TypeChecker& typeChecker = + (frontend.options.typecheckTwice_DEPRECATED || FFlag::LuauSeparateTypechecks ? frontend.typeCheckerForAutocomplete : frontend.typeChecker); + ModulePtr module = + (frontend.options.typecheckTwice_DEPRECATED || FFlag::LuauSeparateTypechecks ? frontend.moduleResolverForAutocomplete.getModule(moduleName) + : frontend.moduleResolver.getModule(moduleName)); if (!module) return {}; @@ -1692,7 +1768,8 @@ OwningAutocompleteResult autocompleteSource(Frontend& frontend, std::string_view sourceModule->mode = Mode::Strict; sourceModule->commentLocations = std::move(result.commentLocations); - TypeChecker& typeChecker = (frontend.options.typecheckTwice ? frontend.typeCheckerForAutocomplete : frontend.typeChecker); + TypeChecker& typeChecker = + (frontend.options.typecheckTwice_DEPRECATED || FFlag::LuauSeparateTypechecks ? frontend.typeCheckerForAutocomplete : frontend.typeChecker); ModulePtr module = typeChecker.check(*sourceModule, Mode::Strict); diff --git a/Analysis/src/Clone.cpp b/Analysis/src/Clone.cpp new file mode 100644 index 00000000..ac9705a7 --- /dev/null +++ b/Analysis/src/Clone.cpp @@ -0,0 +1,371 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details + +#include "Luau/Clone.h" +#include "Luau/Module.h" +#include "Luau/RecursionCounter.h" +#include "Luau/TypePack.h" +#include "Luau/Unifiable.h" + +LUAU_FASTINTVARIABLE(LuauTypeCloneRecursionLimit, 300) + +namespace Luau +{ + +namespace +{ + +struct TypePackCloner; + +/* + * Both TypeCloner and TypePackCloner work by depositing the requested type variable into the appropriate 'seen' set. + * They do not return anything because their sole consumer (the deepClone function) already has a pointer into this storage. + */ + +struct TypeCloner +{ + TypeCloner(TypeArena& dest, TypeId typeId, SeenTypes& seenTypes, SeenTypePacks& seenTypePacks, CloneState& cloneState) + : dest(dest) + , typeId(typeId) + , seenTypes(seenTypes) + , seenTypePacks(seenTypePacks) + , cloneState(cloneState) + { + } + + TypeArena& dest; + TypeId typeId; + SeenTypes& seenTypes; + SeenTypePacks& seenTypePacks; + CloneState& cloneState; + + template + void defaultClone(const T& t); + + void operator()(const Unifiable::Free& t); + void operator()(const Unifiable::Generic& t); + void operator()(const Unifiable::Bound& t); + void operator()(const Unifiable::Error& t); + void operator()(const PrimitiveTypeVar& t); + void operator()(const SingletonTypeVar& t); + void operator()(const FunctionTypeVar& t); + void operator()(const TableTypeVar& t); + void operator()(const MetatableTypeVar& t); + void operator()(const ClassTypeVar& t); + void operator()(const AnyTypeVar& t); + void operator()(const UnionTypeVar& t); + void operator()(const IntersectionTypeVar& t); + void operator()(const LazyTypeVar& t); +}; + +struct TypePackCloner +{ + TypeArena& dest; + TypePackId typePackId; + SeenTypes& seenTypes; + SeenTypePacks& seenTypePacks; + CloneState& cloneState; + + TypePackCloner(TypeArena& dest, TypePackId typePackId, SeenTypes& seenTypes, SeenTypePacks& seenTypePacks, CloneState& cloneState) + : dest(dest) + , typePackId(typePackId) + , seenTypes(seenTypes) + , seenTypePacks(seenTypePacks) + , cloneState(cloneState) + { + } + + template + void defaultClone(const T& t) + { + TypePackId cloned = dest.addTypePack(TypePackVar{t}); + seenTypePacks[typePackId] = cloned; + } + + void operator()(const Unifiable::Free& t) + { + cloneState.encounteredFreeType = true; + + TypePackId err = getSingletonTypes().errorRecoveryTypePack(getSingletonTypes().anyTypePack); + TypePackId cloned = dest.addTypePack(*err); + seenTypePacks[typePackId] = cloned; + } + + void operator()(const Unifiable::Generic& t) + { + defaultClone(t); + } + void operator()(const Unifiable::Error& t) + { + defaultClone(t); + } + + // While we are a-cloning, we can flatten out bound TypeVars and make things a bit tighter. + // We just need to be sure that we rewrite pointers both to the binder and the bindee to the same pointer. + void operator()(const Unifiable::Bound& t) + { + TypePackId cloned = clone(t.boundTo, dest, seenTypes, seenTypePacks, cloneState); + seenTypePacks[typePackId] = cloned; + } + + void operator()(const VariadicTypePack& t) + { + TypePackId cloned = dest.addTypePack(TypePackVar{VariadicTypePack{clone(t.ty, dest, seenTypes, seenTypePacks, cloneState)}}); + seenTypePacks[typePackId] = cloned; + } + + void operator()(const TypePack& t) + { + TypePackId cloned = dest.addTypePack(TypePack{}); + TypePack* destTp = getMutable(cloned); + LUAU_ASSERT(destTp != nullptr); + seenTypePacks[typePackId] = cloned; + + for (TypeId ty : t.head) + destTp->head.push_back(clone(ty, dest, seenTypes, seenTypePacks, cloneState)); + + if (t.tail) + destTp->tail = clone(*t.tail, dest, seenTypes, seenTypePacks, cloneState); + } +}; + +template +void TypeCloner::defaultClone(const T& t) +{ + TypeId cloned = dest.addType(t); + seenTypes[typeId] = cloned; +} + +void TypeCloner::operator()(const Unifiable::Free& t) +{ + cloneState.encounteredFreeType = true; + TypeId err = getSingletonTypes().errorRecoveryType(getSingletonTypes().anyType); + TypeId cloned = dest.addType(*err); + seenTypes[typeId] = cloned; +} + +void TypeCloner::operator()(const Unifiable::Generic& t) +{ + defaultClone(t); +} + +void TypeCloner::operator()(const Unifiable::Bound& t) +{ + TypeId boundTo = clone(t.boundTo, dest, seenTypes, seenTypePacks, cloneState); + seenTypes[typeId] = boundTo; +} + +void TypeCloner::operator()(const Unifiable::Error& t) +{ + defaultClone(t); +} + +void TypeCloner::operator()(const PrimitiveTypeVar& t) +{ + defaultClone(t); +} + +void TypeCloner::operator()(const SingletonTypeVar& t) +{ + defaultClone(t); +} + +void TypeCloner::operator()(const FunctionTypeVar& t) +{ + TypeId result = dest.addType(FunctionTypeVar{TypeLevel{0, 0}, {}, {}, nullptr, nullptr, t.definition, t.hasSelf}); + FunctionTypeVar* ftv = getMutable(result); + LUAU_ASSERT(ftv != nullptr); + + seenTypes[typeId] = result; + + for (TypeId generic : t.generics) + ftv->generics.push_back(clone(generic, dest, seenTypes, seenTypePacks, cloneState)); + + for (TypePackId genericPack : t.genericPacks) + ftv->genericPacks.push_back(clone(genericPack, dest, seenTypes, seenTypePacks, cloneState)); + + ftv->tags = t.tags; + ftv->argTypes = clone(t.argTypes, dest, seenTypes, seenTypePacks, cloneState); + ftv->argNames = t.argNames; + ftv->retType = clone(t.retType, dest, seenTypes, seenTypePacks, cloneState); +} + +void TypeCloner::operator()(const TableTypeVar& t) +{ + // If table is now bound to another one, we ignore the content of the original + if (t.boundTo) + { + TypeId boundTo = clone(*t.boundTo, dest, seenTypes, seenTypePacks, cloneState); + seenTypes[typeId] = boundTo; + return; + } + + TypeId result = dest.addType(TableTypeVar{}); + TableTypeVar* ttv = getMutable(result); + LUAU_ASSERT(ttv != nullptr); + + *ttv = t; + + seenTypes[typeId] = result; + + ttv->level = TypeLevel{0, 0}; + + for (const auto& [name, prop] : t.props) + ttv->props[name] = {clone(prop.type, dest, seenTypes, seenTypePacks, cloneState), prop.deprecated, {}, prop.location, prop.tags}; + + if (t.indexer) + ttv->indexer = TableIndexer{clone(t.indexer->indexType, dest, seenTypes, seenTypePacks, cloneState), + clone(t.indexer->indexResultType, dest, seenTypes, seenTypePacks, cloneState)}; + + for (TypeId& arg : ttv->instantiatedTypeParams) + arg = clone(arg, dest, seenTypes, seenTypePacks, cloneState); + + for (TypePackId& arg : ttv->instantiatedTypePackParams) + arg = clone(arg, dest, seenTypes, seenTypePacks, cloneState); + + if (ttv->state == TableState::Free) + { + cloneState.encounteredFreeType = true; + + ttv->state = TableState::Sealed; + } + + ttv->definitionModuleName = t.definitionModuleName; + ttv->methodDefinitionLocations = t.methodDefinitionLocations; + ttv->tags = t.tags; +} + +void TypeCloner::operator()(const MetatableTypeVar& t) +{ + TypeId result = dest.addType(MetatableTypeVar{}); + MetatableTypeVar* mtv = getMutable(result); + seenTypes[typeId] = result; + + mtv->table = clone(t.table, dest, seenTypes, seenTypePacks, cloneState); + mtv->metatable = clone(t.metatable, dest, seenTypes, seenTypePacks, cloneState); +} + +void TypeCloner::operator()(const ClassTypeVar& t) +{ + TypeId result = dest.addType(ClassTypeVar{t.name, {}, std::nullopt, std::nullopt, t.tags, t.userData}); + ClassTypeVar* ctv = getMutable(result); + + seenTypes[typeId] = result; + + for (const auto& [name, prop] : t.props) + ctv->props[name] = {clone(prop.type, dest, seenTypes, seenTypePacks, cloneState), prop.deprecated, {}, prop.location, prop.tags}; + + if (t.parent) + ctv->parent = clone(*t.parent, dest, seenTypes, seenTypePacks, cloneState); + + if (t.metatable) + ctv->metatable = clone(*t.metatable, dest, seenTypes, seenTypePacks, cloneState); +} + +void TypeCloner::operator()(const AnyTypeVar& t) +{ + defaultClone(t); +} + +void TypeCloner::operator()(const UnionTypeVar& t) +{ + std::vector options; + options.reserve(t.options.size()); + + for (TypeId ty : t.options) + options.push_back(clone(ty, dest, seenTypes, seenTypePacks, cloneState)); + + TypeId result = dest.addType(UnionTypeVar{std::move(options)}); + seenTypes[typeId] = result; +} + +void TypeCloner::operator()(const IntersectionTypeVar& t) +{ + TypeId result = dest.addType(IntersectionTypeVar{}); + seenTypes[typeId] = result; + + IntersectionTypeVar* option = getMutable(result); + LUAU_ASSERT(option != nullptr); + + for (TypeId ty : t.parts) + option->parts.push_back(clone(ty, dest, seenTypes, seenTypePacks, cloneState)); +} + +void TypeCloner::operator()(const LazyTypeVar& t) +{ + defaultClone(t); +} + +} // anonymous namespace + +TypePackId clone(TypePackId tp, TypeArena& dest, SeenTypes& seenTypes, SeenTypePacks& seenTypePacks, CloneState& cloneState) +{ + if (tp->persistent) + return tp; + + RecursionLimiter _ra(&cloneState.recursionCount, FInt::LuauTypeCloneRecursionLimit); + + TypePackId& res = seenTypePacks[tp]; + + if (res == nullptr) + { + TypePackCloner cloner{dest, tp, seenTypes, seenTypePacks, cloneState}; + Luau::visit(cloner, tp->ty); // Mutates the storage that 'res' points into. + } + + return res; +} + +TypeId clone(TypeId typeId, TypeArena& dest, SeenTypes& seenTypes, SeenTypePacks& seenTypePacks, CloneState& cloneState) +{ + if (typeId->persistent) + return typeId; + + RecursionLimiter _ra(&cloneState.recursionCount, FInt::LuauTypeCloneRecursionLimit); + + TypeId& res = seenTypes[typeId]; + + if (res == nullptr) + { + TypeCloner cloner{dest, typeId, seenTypes, seenTypePacks, cloneState}; + Luau::visit(cloner, typeId->ty); // Mutates the storage that 'res' points into. + + // Persistent types are not being cloned and we get the original type back which might be read-only + if (!res->persistent) + asMutable(res)->documentationSymbol = typeId->documentationSymbol; + } + + return res; +} + +TypeFun clone(const TypeFun& typeFun, TypeArena& dest, SeenTypes& seenTypes, SeenTypePacks& seenTypePacks, CloneState& cloneState) +{ + TypeFun result; + + for (auto param : typeFun.typeParams) + { + TypeId ty = clone(param.ty, dest, seenTypes, seenTypePacks, cloneState); + std::optional defaultValue; + + if (param.defaultValue) + defaultValue = clone(*param.defaultValue, dest, seenTypes, seenTypePacks, cloneState); + + result.typeParams.push_back({ty, defaultValue}); + } + + for (auto param : typeFun.typePackParams) + { + TypePackId tp = clone(param.tp, dest, seenTypes, seenTypePacks, cloneState); + std::optional defaultValue; + + if (param.defaultValue) + defaultValue = clone(*param.defaultValue, dest, seenTypes, seenTypePacks, cloneState); + + result.typePackParams.push_back({tp, defaultValue}); + } + + result.type = clone(typeFun.type, dest, seenTypes, seenTypePacks, cloneState); + + return result; +} + +} // namespace Luau diff --git a/Analysis/src/Error.cpp b/Analysis/src/Error.cpp index 210c0191..5eb2ea2a 100644 --- a/Analysis/src/Error.cpp +++ b/Analysis/src/Error.cpp @@ -1,6 +1,7 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #include "Luau/Error.h" +#include "Luau/Clone.h" #include "Luau/Module.h" #include "Luau/StringUtils.h" #include "Luau/ToString.h" diff --git a/Analysis/src/Frontend.cpp b/Analysis/src/Frontend.cpp index d8906f6e..000769fe 100644 --- a/Analysis/src/Frontend.cpp +++ b/Analysis/src/Frontend.cpp @@ -2,6 +2,7 @@ #include "Luau/Frontend.h" #include "Luau/Common.h" +#include "Luau/Clone.h" #include "Luau/Config.h" #include "Luau/FileResolver.h" #include "Luau/Parser.h" @@ -16,8 +17,11 @@ #include #include +LUAU_FASTFLAG(LuauCyclicModuleTypeSurface) LUAU_FASTFLAG(LuauInferInNoCheckMode) LUAU_FASTFLAGVARIABLE(LuauKnowsTheDataModel3, false) +LUAU_FASTFLAGVARIABLE(LuauSeparateTypechecks, false) +LUAU_FASTINTVARIABLE(LuauAutocompleteCheckTimeoutMs, 0) namespace Luau { @@ -234,12 +238,6 @@ ErrorVec accumulateErrors( return result; } -struct RequireCycle -{ - Location location; - std::vector path; // one of the paths for a require() to go all the way back to the originating module -}; - // Given a source node (start), find all requires that start a transitive dependency path that ends back at start // For each such path, record the full path and the location of the require in the starting module. // Note that this is O(V^2) for a fully connected graph and produces O(V) paths of length O(V) @@ -356,33 +354,55 @@ CheckResult Frontend::check(const ModuleName& name, std::optionalsecond.dirty) + if (it != sourceNodes.end() && !it->second.isDirty(frontendOptions.forAutocomplete)) { // No recheck required. - auto it2 = moduleResolver.modules.find(name); - if (it2 == moduleResolver.modules.end() || it2->second == nullptr) - throw std::runtime_error("Frontend::modules does not have data for " + name); + if (FFlag::LuauSeparateTypechecks) + { + if (frontendOptions.forAutocomplete) + { + auto it2 = moduleResolverForAutocomplete.modules.find(name); + if (it2 == moduleResolverForAutocomplete.modules.end() || it2->second == nullptr) + throw std::runtime_error("Frontend::modules does not have data for " + name); + } + else + { + auto it2 = moduleResolver.modules.find(name); + if (it2 == moduleResolver.modules.end() || it2->second == nullptr) + throw std::runtime_error("Frontend::modules does not have data for " + name); + } - return CheckResult{accumulateErrors(sourceNodes, moduleResolver.modules, name)}; + return CheckResult{accumulateErrors( + sourceNodes, frontendOptions.forAutocomplete ? moduleResolverForAutocomplete.modules : moduleResolver.modules, name)}; + } + else + { + auto it2 = moduleResolver.modules.find(name); + if (it2 == moduleResolver.modules.end() || it2->second == nullptr) + throw std::runtime_error("Frontend::modules does not have data for " + name); + + return CheckResult{accumulateErrors(sourceNodes, moduleResolver.modules, name)}; + } } std::vector buildQueue; - bool cycleDetected = parseGraph(buildQueue, checkResult, name); - - FrontendOptions frontendOptions = optionOverride.value_or(options); + bool cycleDetected = parseGraph(buildQueue, checkResult, name, frontendOptions.forAutocomplete); // Keep track of which AST nodes we've reported cycles in std::unordered_set reportedCycles; + double autocompleteTimeLimit = FInt::LuauAutocompleteCheckTimeoutMs / 1000.0; + for (const ModuleName& moduleName : buildQueue) { LUAU_ASSERT(sourceNodes.count(moduleName)); SourceNode& sourceNode = sourceNodes[moduleName]; - if (!sourceNode.dirty) + if (!sourceNode.isDirty(frontendOptions.forAutocomplete)) continue; LUAU_ASSERT(sourceModules.count(moduleName)); @@ -408,13 +428,44 @@ CheckResult Frontend::check(const ModuleName& name, std::optionaltimeout) + checkResult.timeoutHits.push_back(moduleName); + + stats.timeCheck += getTimestamp() - timestamp; + stats.filesStrict += 1; + + sourceNode.dirtyAutocomplete = false; + continue; + } + + if (FFlag::LuauCyclicModuleTypeSurface) + typeChecker.requireCycles = requireCycles; + ModulePtr module = typeChecker.check(sourceModule, mode, environmentScope); // If we're typechecking twice, we do so. // The second typecheck is always in strict mode with DM awareness // to provide better typen information for IDE features. - if (frontendOptions.typecheckTwice) + if (!FFlag::LuauSeparateTypechecks && frontendOptions.typecheckTwice_DEPRECATED) { + if (FFlag::LuauCyclicModuleTypeSurface) + typeCheckerForAutocomplete.requireCycles = requireCycles; + ModulePtr moduleForAutocomplete = typeCheckerForAutocomplete.check(sourceModule, Mode::Strict); moduleResolverForAutocomplete.modules[moduleName] = moduleForAutocomplete; } @@ -467,7 +518,7 @@ CheckResult Frontend::check(const ModuleName& name, std::optional& buildQueue, CheckResult& checkResult, const ModuleName& root) +bool Frontend::parseGraph(std::vector& buildQueue, CheckResult& checkResult, const ModuleName& root, bool forAutocomplete) { LUAU_TIMETRACE_SCOPE("Frontend::parseGraph", "Frontend"); LUAU_TIMETRACE_ARGUMENT("root", root.c_str()); @@ -486,7 +537,7 @@ bool Frontend::parseGraph(std::vector& buildQueue, CheckResult& chec bool cyclic = false; { - auto [sourceNode, _] = getSourceNode(checkResult, root); + auto [sourceNode, _] = getSourceNode(checkResult, root, forAutocomplete); if (sourceNode) stack.push_back(sourceNode); } @@ -538,7 +589,7 @@ bool Frontend::parseGraph(std::vector& buildQueue, CheckResult& chec // this relies on the fact that markDirty marks reverse-dependencies dirty as well // thus if a node is not dirty, all its transitive deps aren't dirty, which means that they won't ever need // to be built, *and* can't form a cycle with any nodes we did process. - if (!it->second.dirty) + if (!it->second.isDirty(forAutocomplete)) continue; // note: this check is technically redundant *except* that getSourceNode has somewhat broken memoization @@ -550,7 +601,7 @@ bool Frontend::parseGraph(std::vector& buildQueue, CheckResult& chec } } - auto [sourceNode, _] = getSourceNode(checkResult, dep); + auto [sourceNode, _] = getSourceNode(checkResult, dep, forAutocomplete); if (sourceNode) { stack.push_back(sourceNode); @@ -594,7 +645,7 @@ LintResult Frontend::lint(const ModuleName& name, std::optionalsecond.dirty; + return it == sourceNodes.end() || it->second.isDirty(forAutocomplete); } /* @@ -699,8 +750,16 @@ bool Frontend::isDirty(const ModuleName& name) const */ void Frontend::markDirty(const ModuleName& name, std::vector* markedDirty) { - if (!moduleResolver.modules.count(name)) - return; + if (FFlag::LuauSeparateTypechecks) + { + if (!moduleResolver.modules.count(name) && !moduleResolverForAutocomplete.modules.count(name)) + return; + } + else + { + if (!moduleResolver.modules.count(name)) + return; + } std::unordered_map> reverseDeps; for (const auto& module : sourceNodes) @@ -722,10 +781,21 @@ void Frontend::markDirty(const ModuleName& name, std::vector* marked if (markedDirty) markedDirty->push_back(next); - if (sourceNode.dirty) - continue; + if (FFlag::LuauSeparateTypechecks) + { + if (sourceNode.dirty && sourceNode.dirtyAutocomplete) + continue; - sourceNode.dirty = true; + sourceNode.dirty = true; + sourceNode.dirtyAutocomplete = true; + } + else + { + if (sourceNode.dirty) + continue; + + sourceNode.dirty = true; + } if (0 == reverseDeps.count(name)) continue; @@ -752,13 +822,13 @@ const SourceModule* Frontend::getSourceModule(const ModuleName& moduleName) cons } // Read AST into sourceModules if necessary. Trace require()s. Report parse errors. -std::pair Frontend::getSourceNode(CheckResult& checkResult, const ModuleName& name) +std::pair Frontend::getSourceNode(CheckResult& checkResult, const ModuleName& name, bool forAutocomplete) { LUAU_TIMETRACE_SCOPE("Frontend::getSourceNode", "Frontend"); LUAU_TIMETRACE_ARGUMENT("name", name.c_str()); auto it = sourceNodes.find(name); - if (it != sourceNodes.end() && !it->second.dirty) + if (it != sourceNodes.end() && !it->second.isDirty(forAutocomplete)) { auto moduleIt = sourceModules.find(name); if (moduleIt != sourceModules.end()) @@ -801,7 +871,19 @@ std::pair Frontend::getSourceNode(CheckResult& check sourceNode.name = name; sourceNode.requires.clear(); sourceNode.requireLocations.clear(); - sourceNode.dirty = true; + + if (FFlag::LuauSeparateTypechecks) + { + if (it == sourceNodes.end()) + { + sourceNode.dirty = true; + sourceNode.dirtyAutocomplete = true; + } + } + else + { + sourceNode.dirty = true; + } for (const auto& [moduleName, location] : requireTrace.requires) sourceNode.requires.insert(moduleName); diff --git a/Analysis/src/IostreamHelpers.cpp b/Analysis/src/IostreamHelpers.cpp index 19c2ddab..a8f67589 100644 --- a/Analysis/src/IostreamHelpers.cpp +++ b/Analysis/src/IostreamHelpers.cpp @@ -23,9 +23,178 @@ std::ostream& operator<<(std::ostream& stream, const AstName& name) return stream << ""; } -std::ostream& operator<<(std::ostream& stream, const TypeMismatch& tm) +template +static void errorToString(std::ostream& stream, const T& err) { - return stream << "TypeMismatch { " << toString(tm.wantedType) << ", " << toString(tm.givenType) << " }"; + if constexpr (false) + { + } + else if constexpr (std::is_same_v) + stream << "TypeMismatch { " << toString(err.wantedType) << ", " << toString(err.givenType) << " }"; + else if constexpr (std::is_same_v) + stream << "UnknownSymbol { " << err.name << " , context " << err.context << " }"; + else if constexpr (std::is_same_v) + stream << "UnknownProperty { " << toString(err.table) << ", key = " << err.key << " }"; + else if constexpr (std::is_same_v) + stream << "NotATable { " << toString(err.ty) << " }"; + else if constexpr (std::is_same_v) + stream << "CannotExtendTable { " << toString(err.tableType) << ", context " << err.context << ", prop \"" << err.prop << "\" }"; + else if constexpr (std::is_same_v) + stream << "OnlyTablesCanHaveMethods { " << toString(err.tableType) << " }"; + else if constexpr (std::is_same_v) + stream << "DuplicateTypeDefinition { " << err.name << " }"; + else if constexpr (std::is_same_v) + stream << "CountMismatch { expected " << err.expected << ", got " << err.actual << ", context " << err.context << " }"; + else if constexpr (std::is_same_v) + stream << "FunctionDoesNotTakeSelf { }"; + else if constexpr (std::is_same_v) + stream << "FunctionRequiresSelf { extraNils " << err.requiredExtraNils << " }"; + else if constexpr (std::is_same_v) + stream << "OccursCheckFailed { }"; + else if constexpr (std::is_same_v) + stream << "UnknownRequire { " << err.modulePath << " }"; + else if constexpr (std::is_same_v) + { + stream << "IncorrectGenericParameterCount { name = " << err.name; + + if (!err.typeFun.typeParams.empty() || !err.typeFun.typePackParams.empty()) + { + stream << "<"; + bool first = true; + for (auto param : err.typeFun.typeParams) + { + if (first) + first = false; + else + stream << ", "; + + stream << toString(param.ty); + } + + for (auto param : err.typeFun.typePackParams) + { + if (first) + first = false; + else + stream << ", "; + + stream << toString(param.tp); + } + + stream << ">"; + } + + stream << ", typeFun = " << toString(err.typeFun.type) << ", actualCount = " << err.actualParameters << " }"; + } + else if constexpr (std::is_same_v) + stream << "SyntaxError { " << err.message << " }"; + else if constexpr (std::is_same_v) + stream << "CodeTooComplex {}"; + else if constexpr (std::is_same_v) + stream << "UnificationTooComplex {}"; + else if constexpr (std::is_same_v) + { + stream << "UnknownPropButFoundLikeProp { key = '" << err.key << "', suggested = { "; + + bool first = true; + for (Name name : err.candidates) + { + if (first) + first = false; + else + stream << ", "; + + stream << "'" << name << "'"; + } + + stream << " }, table = " << toString(err.table) << " } "; + } + else if constexpr (std::is_same_v) + stream << "GenericError { " << err.message << " }"; + else if constexpr (std::is_same_v) + stream << "CannotCallNonFunction { " << toString(err.ty) << " }"; + else if constexpr (std::is_same_v) + stream << "ExtraInformation { " << err.message << " }"; + else if constexpr (std::is_same_v) + stream << "DeprecatedApiUsed { " << err.symbol << ", useInstead = " << err.useInstead << " }"; + else if constexpr (std::is_same_v) + { + stream << "ModuleHasCyclicDependency {"; + + bool first = true; + for (const ModuleName& name : err.cycle) + { + if (first) + first = false; + else + stream << ", "; + + stream << name; + } + + stream << "}"; + } + else if constexpr (std::is_same_v) + stream << "IllegalRequire { " << err.moduleName << ", reason = " << err.reason << " }"; + else if constexpr (std::is_same_v) + stream << "FunctionExitsWithoutReturning {" << toString(err.expectedReturnType) << "}"; + else if constexpr (std::is_same_v) + stream << "DuplicateGenericParameter { " + err.parameterName + " }"; + else if constexpr (std::is_same_v) + stream << "CannotInferBinaryOperation { op = " + toString(err.op) + ", suggested = '" + + (err.suggestedToAnnotate ? *err.suggestedToAnnotate : "") + "', kind " + << err.kind << "}"; + else if constexpr (std::is_same_v) + { + stream << "MissingProperties { superType = '" << toString(err.superType) << "', subType = '" << toString(err.subType) << "', properties = { "; + + bool first = true; + for (Name name : err.properties) + { + if (first) + first = false; + else + stream << ", "; + + stream << "'" << name << "'"; + } + + stream << " }, context " << err.context << " } "; + } + else if constexpr (std::is_same_v) + stream << "SwappedGenericTypeParameter { name = '" + err.name + "', kind = " + std::to_string(err.kind) + " }"; + else if constexpr (std::is_same_v) + stream << "OptionalValueAccess { optional = '" + toString(err.optional) + "' }"; + else if constexpr (std::is_same_v) + { + stream << "MissingUnionProperty { type = '" + toString(err.type) + "', missing = { "; + + bool first = true; + for (auto ty : err.missing) + { + if (first) + first = false; + else + stream << ", "; + + stream << "'" << toString(ty) << "'"; + } + + stream << " }, key = '" + err.key + "' }"; + } + else if constexpr (std::is_same_v) + stream << "TypesAreUnrelated { left = '" + toString(err.left) + "', right = '" + toString(err.right) + "' }"; + else + static_assert(always_false_v, "Non-exhaustive type switch"); +} + +std::ostream& operator<<(std::ostream& stream, const TypeErrorData& data) +{ + auto cb = [&](const auto& e) { + return errorToString(stream, e); + }; + visit(cb, data); + return stream; } std::ostream& operator<<(std::ostream& stream, const TypeError& error) @@ -33,241 +202,6 @@ std::ostream& operator<<(std::ostream& stream, const TypeError& error) return stream << "TypeError { \"" << error.moduleName << "\", " << error.location << ", " << error.data << " }"; } -std::ostream& operator<<(std::ostream& stream, const UnknownSymbol& error) -{ - return stream << "UnknownSymbol { " << error.name << " , context " << error.context << " }"; -} - -std::ostream& operator<<(std::ostream& stream, const UnknownProperty& error) -{ - return stream << "UnknownProperty { " << toString(error.table) << ", key = " << error.key << " }"; -} - -std::ostream& operator<<(std::ostream& stream, const NotATable& ge) -{ - return stream << "NotATable { " << toString(ge.ty) << " }"; -} - -std::ostream& operator<<(std::ostream& stream, const CannotExtendTable& error) -{ - return stream << "CannotExtendTable { " << toString(error.tableType) << ", context " << error.context << ", prop \"" << error.prop << "\" }"; -} - -std::ostream& operator<<(std::ostream& stream, const OnlyTablesCanHaveMethods& error) -{ - return stream << "OnlyTablesCanHaveMethods { " << toString(error.tableType) << " }"; -} - -std::ostream& operator<<(std::ostream& stream, const DuplicateTypeDefinition& error) -{ - return stream << "DuplicateTypeDefinition { " << error.name << " }"; -} - -std::ostream& operator<<(std::ostream& stream, const CountMismatch& error) -{ - return stream << "CountMismatch { expected " << error.expected << ", got " << error.actual << ", context " << error.context << " }"; -} - -std::ostream& operator<<(std::ostream& stream, const FunctionDoesNotTakeSelf&) -{ - return stream << "FunctionDoesNotTakeSelf { }"; -} - -std::ostream& operator<<(std::ostream& stream, const FunctionRequiresSelf& error) -{ - return stream << "FunctionRequiresSelf { extraNils " << error.requiredExtraNils << " }"; -} - -std::ostream& operator<<(std::ostream& stream, const OccursCheckFailed&) -{ - return stream << "OccursCheckFailed { }"; -} - -std::ostream& operator<<(std::ostream& stream, const UnknownRequire& error) -{ - return stream << "UnknownRequire { " << error.modulePath << " }"; -} - -std::ostream& operator<<(std::ostream& stream, const IncorrectGenericParameterCount& error) -{ - stream << "IncorrectGenericParameterCount { name = " << error.name; - - if (!error.typeFun.typeParams.empty() || !error.typeFun.typePackParams.empty()) - { - stream << "<"; - bool first = true; - for (auto param : error.typeFun.typeParams) - { - if (first) - first = false; - else - stream << ", "; - - stream << toString(param.ty); - } - - for (auto param : error.typeFun.typePackParams) - { - if (first) - first = false; - else - stream << ", "; - - stream << toString(param.tp); - } - - stream << ">"; - } - - stream << ", typeFun = " << toString(error.typeFun.type) << ", actualCount = " << error.actualParameters << " }"; - return stream; -} - -std::ostream& operator<<(std::ostream& stream, const SyntaxError& ge) -{ - return stream << "SyntaxError { " << ge.message << " }"; -} - -std::ostream& operator<<(std::ostream& stream, const CodeTooComplex&) -{ - return stream << "CodeTooComplex {}"; -} - -std::ostream& operator<<(std::ostream& stream, const UnificationTooComplex&) -{ - return stream << "UnificationTooComplex {}"; -} - -std::ostream& operator<<(std::ostream& stream, const UnknownPropButFoundLikeProp& e) -{ - stream << "UnknownPropButFoundLikeProp { key = '" << e.key << "', suggested = { "; - - bool first = true; - for (Name name : e.candidates) - { - if (first) - first = false; - else - stream << ", "; - - stream << "'" << name << "'"; - } - - return stream << " }, table = " << toString(e.table) << " } "; -} - -std::ostream& operator<<(std::ostream& stream, const GenericError& ge) -{ - return stream << "GenericError { " << ge.message << " }"; -} - -std::ostream& operator<<(std::ostream& stream, const CannotCallNonFunction& e) -{ - return stream << "CannotCallNonFunction { " << toString(e.ty) << " }"; -} - -std::ostream& operator<<(std::ostream& stream, const FunctionExitsWithoutReturning& error) -{ - return stream << "FunctionExitsWithoutReturning {" << toString(error.expectedReturnType) << "}"; -} - -std::ostream& operator<<(std::ostream& stream, const ExtraInformation& e) -{ - return stream << "ExtraInformation { " << e.message << " }"; -} - -std::ostream& operator<<(std::ostream& stream, const DeprecatedApiUsed& e) -{ - return stream << "DeprecatedApiUsed { " << e.symbol << ", useInstead = " << e.useInstead << " }"; -} - -std::ostream& operator<<(std::ostream& stream, const ModuleHasCyclicDependency& e) -{ - stream << "ModuleHasCyclicDependency {"; - - bool first = true; - for (const ModuleName& name : e.cycle) - { - if (first) - first = false; - else - stream << ", "; - - stream << name; - } - - return stream << "}"; -} - -std::ostream& operator<<(std::ostream& stream, const IllegalRequire& e) -{ - return stream << "IllegalRequire { " << e.moduleName << ", reason = " << e.reason << " }"; -} - -std::ostream& operator<<(std::ostream& stream, const MissingProperties& e) -{ - stream << "MissingProperties { superType = '" << toString(e.superType) << "', subType = '" << toString(e.subType) << "', properties = { "; - - bool first = true; - for (Name name : e.properties) - { - if (first) - first = false; - else - stream << ", "; - - stream << "'" << name << "'"; - } - - return stream << " }, context " << e.context << " } "; -} - -std::ostream& operator<<(std::ostream& stream, const DuplicateGenericParameter& error) -{ - return stream << "DuplicateGenericParameter { " + error.parameterName + " }"; -} - -std::ostream& operator<<(std::ostream& stream, const CannotInferBinaryOperation& error) -{ - return stream << "CannotInferBinaryOperation { op = " + toString(error.op) + ", suggested = '" + - (error.suggestedToAnnotate ? *error.suggestedToAnnotate : "") + "', kind " - << error.kind << "}"; -} - -std::ostream& operator<<(std::ostream& stream, const SwappedGenericTypeParameter& error) -{ - return stream << "SwappedGenericTypeParameter { name = '" + error.name + "', kind = " + std::to_string(error.kind) + " }"; -} - -std::ostream& operator<<(std::ostream& stream, const OptionalValueAccess& error) -{ - return stream << "OptionalValueAccess { optional = '" + toString(error.optional) + "' }"; -} - -std::ostream& operator<<(std::ostream& stream, const MissingUnionProperty& error) -{ - stream << "MissingUnionProperty { type = '" + toString(error.type) + "', missing = { "; - - bool first = true; - for (auto ty : error.missing) - { - if (first) - first = false; - else - stream << ", "; - - stream << "'" << toString(ty) << "'"; - } - - return stream << " }, key = '" + error.key + "' }"; -} - -std::ostream& operator<<(std::ostream& stream, const TypesAreUnrelated& error) -{ - stream << "TypesAreUnrelated { left = '" + toString(error.left) + "', right = '" + toString(error.right) + "' }"; - return stream; -} - std::ostream& operator<<(std::ostream& stream, const TableState& tv) { return stream << static_cast::type>(tv); @@ -283,15 +217,4 @@ std::ostream& operator<<(std::ostream& stream, const TypePackVar& tv) return stream << toString(tv); } -std::ostream& operator<<(std::ostream& lhs, const TypeErrorData& ted) -{ - Luau::visit( - [&](const auto& a) { - lhs << a; - }, - ted); - - return lhs; -} - } // namespace Luau diff --git a/Analysis/src/Module.cpp b/Analysis/src/Module.cpp index 0787d3a4..6bb45245 100644 --- a/Analysis/src/Module.cpp +++ b/Analysis/src/Module.cpp @@ -2,6 +2,7 @@ #include "Luau/Module.h" #include "Luau/Common.h" +#include "Luau/Clone.h" #include "Luau/RecursionCounter.h" #include "Luau/Scope.h" #include "Luau/TypeInfer.h" @@ -12,7 +13,6 @@ #include LUAU_FASTFLAGVARIABLE(DebugLuauFreezeArena, false) -LUAU_FASTINTVARIABLE(LuauTypeCloneRecursionLimit, 300) LUAU_FASTFLAGVARIABLE(LuauCloneDeclaredGlobals, false) namespace Luau @@ -113,363 +113,6 @@ TypePackId TypeArena::addTypePack(TypePackVar tp) return allocated; } -namespace -{ - -struct TypePackCloner; - -/* - * Both TypeCloner and TypePackCloner work by depositing the requested type variable into the appropriate 'seen' set. - * They do not return anything because their sole consumer (the deepClone function) already has a pointer into this storage. - */ - -struct TypeCloner -{ - TypeCloner(TypeArena& dest, TypeId typeId, SeenTypes& seenTypes, SeenTypePacks& seenTypePacks, CloneState& cloneState) - : dest(dest) - , typeId(typeId) - , seenTypes(seenTypes) - , seenTypePacks(seenTypePacks) - , cloneState(cloneState) - { - } - - TypeArena& dest; - TypeId typeId; - SeenTypes& seenTypes; - SeenTypePacks& seenTypePacks; - CloneState& cloneState; - - template - void defaultClone(const T& t); - - void operator()(const Unifiable::Free& t); - void operator()(const Unifiable::Generic& t); - void operator()(const Unifiable::Bound& t); - void operator()(const Unifiable::Error& t); - void operator()(const PrimitiveTypeVar& t); - void operator()(const SingletonTypeVar& t); - void operator()(const FunctionTypeVar& t); - void operator()(const TableTypeVar& t); - void operator()(const MetatableTypeVar& t); - void operator()(const ClassTypeVar& t); - void operator()(const AnyTypeVar& t); - void operator()(const UnionTypeVar& t); - void operator()(const IntersectionTypeVar& t); - void operator()(const LazyTypeVar& t); -}; - -struct TypePackCloner -{ - TypeArena& dest; - TypePackId typePackId; - SeenTypes& seenTypes; - SeenTypePacks& seenTypePacks; - CloneState& cloneState; - - TypePackCloner(TypeArena& dest, TypePackId typePackId, SeenTypes& seenTypes, SeenTypePacks& seenTypePacks, CloneState& cloneState) - : dest(dest) - , typePackId(typePackId) - , seenTypes(seenTypes) - , seenTypePacks(seenTypePacks) - , cloneState(cloneState) - { - } - - template - void defaultClone(const T& t) - { - TypePackId cloned = dest.addTypePack(TypePackVar{t}); - seenTypePacks[typePackId] = cloned; - } - - void operator()(const Unifiable::Free& t) - { - cloneState.encounteredFreeType = true; - - TypePackId err = getSingletonTypes().errorRecoveryTypePack(getSingletonTypes().anyTypePack); - TypePackId cloned = dest.addTypePack(*err); - seenTypePacks[typePackId] = cloned; - } - - void operator()(const Unifiable::Generic& t) - { - defaultClone(t); - } - void operator()(const Unifiable::Error& t) - { - defaultClone(t); - } - - // While we are a-cloning, we can flatten out bound TypeVars and make things a bit tighter. - // We just need to be sure that we rewrite pointers both to the binder and the bindee to the same pointer. - void operator()(const Unifiable::Bound& t) - { - TypePackId cloned = clone(t.boundTo, dest, seenTypes, seenTypePacks, cloneState); - seenTypePacks[typePackId] = cloned; - } - - void operator()(const VariadicTypePack& t) - { - TypePackId cloned = dest.addTypePack(TypePackVar{VariadicTypePack{clone(t.ty, dest, seenTypes, seenTypePacks, cloneState)}}); - seenTypePacks[typePackId] = cloned; - } - - void operator()(const TypePack& t) - { - TypePackId cloned = dest.addTypePack(TypePack{}); - TypePack* destTp = getMutable(cloned); - LUAU_ASSERT(destTp != nullptr); - seenTypePacks[typePackId] = cloned; - - for (TypeId ty : t.head) - destTp->head.push_back(clone(ty, dest, seenTypes, seenTypePacks, cloneState)); - - if (t.tail) - destTp->tail = clone(*t.tail, dest, seenTypes, seenTypePacks, cloneState); - } -}; - -template -void TypeCloner::defaultClone(const T& t) -{ - TypeId cloned = dest.addType(t); - seenTypes[typeId] = cloned; -} - -void TypeCloner::operator()(const Unifiable::Free& t) -{ - cloneState.encounteredFreeType = true; - TypeId err = getSingletonTypes().errorRecoveryType(getSingletonTypes().anyType); - TypeId cloned = dest.addType(*err); - seenTypes[typeId] = cloned; -} - -void TypeCloner::operator()(const Unifiable::Generic& t) -{ - defaultClone(t); -} - -void TypeCloner::operator()(const Unifiable::Bound& t) -{ - TypeId boundTo = clone(t.boundTo, dest, seenTypes, seenTypePacks, cloneState); - seenTypes[typeId] = boundTo; -} - -void TypeCloner::operator()(const Unifiable::Error& t) -{ - defaultClone(t); -} - -void TypeCloner::operator()(const PrimitiveTypeVar& t) -{ - defaultClone(t); -} - -void TypeCloner::operator()(const SingletonTypeVar& t) -{ - defaultClone(t); -} - -void TypeCloner::operator()(const FunctionTypeVar& t) -{ - TypeId result = dest.addType(FunctionTypeVar{TypeLevel{0, 0}, {}, {}, nullptr, nullptr, t.definition, t.hasSelf}); - FunctionTypeVar* ftv = getMutable(result); - LUAU_ASSERT(ftv != nullptr); - - seenTypes[typeId] = result; - - for (TypeId generic : t.generics) - ftv->generics.push_back(clone(generic, dest, seenTypes, seenTypePacks, cloneState)); - - for (TypePackId genericPack : t.genericPacks) - ftv->genericPacks.push_back(clone(genericPack, dest, seenTypes, seenTypePacks, cloneState)); - - ftv->tags = t.tags; - ftv->argTypes = clone(t.argTypes, dest, seenTypes, seenTypePacks, cloneState); - ftv->argNames = t.argNames; - ftv->retType = clone(t.retType, dest, seenTypes, seenTypePacks, cloneState); -} - -void TypeCloner::operator()(const TableTypeVar& t) -{ - // If table is now bound to another one, we ignore the content of the original - if (t.boundTo) - { - TypeId boundTo = clone(*t.boundTo, dest, seenTypes, seenTypePacks, cloneState); - seenTypes[typeId] = boundTo; - return; - } - - TypeId result = dest.addType(TableTypeVar{}); - TableTypeVar* ttv = getMutable(result); - LUAU_ASSERT(ttv != nullptr); - - *ttv = t; - - seenTypes[typeId] = result; - - ttv->level = TypeLevel{0, 0}; - - for (const auto& [name, prop] : t.props) - ttv->props[name] = {clone(prop.type, dest, seenTypes, seenTypePacks, cloneState), prop.deprecated, {}, prop.location, prop.tags}; - - if (t.indexer) - ttv->indexer = TableIndexer{clone(t.indexer->indexType, dest, seenTypes, seenTypePacks, cloneState), - clone(t.indexer->indexResultType, dest, seenTypes, seenTypePacks, cloneState)}; - - for (TypeId& arg : ttv->instantiatedTypeParams) - arg = clone(arg, dest, seenTypes, seenTypePacks, cloneState); - - for (TypePackId& arg : ttv->instantiatedTypePackParams) - arg = clone(arg, dest, seenTypes, seenTypePacks, cloneState); - - if (ttv->state == TableState::Free) - { - cloneState.encounteredFreeType = true; - - ttv->state = TableState::Sealed; - } - - ttv->definitionModuleName = t.definitionModuleName; - ttv->methodDefinitionLocations = t.methodDefinitionLocations; - ttv->tags = t.tags; -} - -void TypeCloner::operator()(const MetatableTypeVar& t) -{ - TypeId result = dest.addType(MetatableTypeVar{}); - MetatableTypeVar* mtv = getMutable(result); - seenTypes[typeId] = result; - - mtv->table = clone(t.table, dest, seenTypes, seenTypePacks, cloneState); - mtv->metatable = clone(t.metatable, dest, seenTypes, seenTypePacks, cloneState); -} - -void TypeCloner::operator()(const ClassTypeVar& t) -{ - TypeId result = dest.addType(ClassTypeVar{t.name, {}, std::nullopt, std::nullopt, t.tags, t.userData}); - ClassTypeVar* ctv = getMutable(result); - - seenTypes[typeId] = result; - - for (const auto& [name, prop] : t.props) - ctv->props[name] = {clone(prop.type, dest, seenTypes, seenTypePacks, cloneState), prop.deprecated, {}, prop.location, prop.tags}; - - if (t.parent) - ctv->parent = clone(*t.parent, dest, seenTypes, seenTypePacks, cloneState); - - if (t.metatable) - ctv->metatable = clone(*t.metatable, dest, seenTypes, seenTypePacks, cloneState); -} - -void TypeCloner::operator()(const AnyTypeVar& t) -{ - defaultClone(t); -} - -void TypeCloner::operator()(const UnionTypeVar& t) -{ - std::vector options; - options.reserve(t.options.size()); - - for (TypeId ty : t.options) - options.push_back(clone(ty, dest, seenTypes, seenTypePacks, cloneState)); - - TypeId result = dest.addType(UnionTypeVar{std::move(options)}); - seenTypes[typeId] = result; -} - -void TypeCloner::operator()(const IntersectionTypeVar& t) -{ - TypeId result = dest.addType(IntersectionTypeVar{}); - seenTypes[typeId] = result; - - IntersectionTypeVar* option = getMutable(result); - LUAU_ASSERT(option != nullptr); - - for (TypeId ty : t.parts) - option->parts.push_back(clone(ty, dest, seenTypes, seenTypePacks, cloneState)); -} - -void TypeCloner::operator()(const LazyTypeVar& t) -{ - defaultClone(t); -} - -} // anonymous namespace - -TypePackId clone(TypePackId tp, TypeArena& dest, SeenTypes& seenTypes, SeenTypePacks& seenTypePacks, CloneState& cloneState) -{ - if (tp->persistent) - return tp; - - RecursionLimiter _ra(&cloneState.recursionCount, FInt::LuauTypeCloneRecursionLimit); - - TypePackId& res = seenTypePacks[tp]; - - if (res == nullptr) - { - TypePackCloner cloner{dest, tp, seenTypes, seenTypePacks, cloneState}; - Luau::visit(cloner, tp->ty); // Mutates the storage that 'res' points into. - } - - return res; -} - -TypeId clone(TypeId typeId, TypeArena& dest, SeenTypes& seenTypes, SeenTypePacks& seenTypePacks, CloneState& cloneState) -{ - if (typeId->persistent) - return typeId; - - RecursionLimiter _ra(&cloneState.recursionCount, FInt::LuauTypeCloneRecursionLimit); - - TypeId& res = seenTypes[typeId]; - - if (res == nullptr) - { - TypeCloner cloner{dest, typeId, seenTypes, seenTypePacks, cloneState}; - Luau::visit(cloner, typeId->ty); // Mutates the storage that 'res' points into. - - // Persistent types are not being cloned and we get the original type back which might be read-only - if (!res->persistent) - asMutable(res)->documentationSymbol = typeId->documentationSymbol; - } - - return res; -} - -TypeFun clone(const TypeFun& typeFun, TypeArena& dest, SeenTypes& seenTypes, SeenTypePacks& seenTypePacks, CloneState& cloneState) -{ - TypeFun result; - - for (auto param : typeFun.typeParams) - { - TypeId ty = clone(param.ty, dest, seenTypes, seenTypePacks, cloneState); - std::optional defaultValue; - - if (param.defaultValue) - defaultValue = clone(*param.defaultValue, dest, seenTypes, seenTypePacks, cloneState); - - result.typeParams.push_back({ty, defaultValue}); - } - - for (auto param : typeFun.typePackParams) - { - TypePackId tp = clone(param.tp, dest, seenTypes, seenTypePacks, cloneState); - std::optional defaultValue; - - if (param.defaultValue) - defaultValue = clone(*param.defaultValue, dest, seenTypes, seenTypePacks, cloneState); - - result.typePackParams.push_back({tp, defaultValue}); - } - - result.type = clone(typeFun.type, dest, seenTypes, seenTypePacks, cloneState); - - return result; -} - ScopePtr Module::getModuleScope() const { LUAU_ASSERT(!scopes.empty()); diff --git a/Analysis/src/TxnLog.cpp b/Analysis/src/TxnLog.cpp index 876f5f05..5fbb596d 100644 --- a/Analysis/src/TxnLog.cpp +++ b/Analysis/src/TxnLog.cpp @@ -7,6 +7,8 @@ #include #include +LUAU_FASTFLAGVARIABLE(LuauTxnLogPreserveOwner, false) + namespace Luau { @@ -78,11 +80,32 @@ void TxnLog::concat(TxnLog rhs) void TxnLog::commit() { - for (auto& [ty, rep] : typeVarChanges) - *asMutable(ty) = rep.get()->pending; + if (FFlag::LuauTxnLogPreserveOwner) + { + for (auto& [ty, rep] : typeVarChanges) + { + TypeArena* owningArena = ty->owningArena; + TypeVar* mtv = asMutable(ty); + *mtv = rep.get()->pending; + mtv->owningArena = owningArena; + } - for (auto& [tp, rep] : typePackChanges) - *asMutable(tp) = rep.get()->pending; + for (auto& [tp, rep] : typePackChanges) + { + TypeArena* owningArena = tp->owningArena; + TypePackVar* mpv = asMutable(tp); + *mpv = rep.get()->pending; + mpv->owningArena = owningArena; + } + } + else + { + for (auto& [ty, rep] : typeVarChanges) + *asMutable(ty) = rep.get()->pending; + + for (auto& [tp, rep] : typePackChanges) + *asMutable(tp) = rep.get()->pending; + } clear(); } diff --git a/Analysis/src/TypeInfer.cpp b/Analysis/src/TypeInfer.cpp index 6df6bff0..10930248 100644 --- a/Analysis/src/TypeInfer.cpp +++ b/Analysis/src/TypeInfer.cpp @@ -22,6 +22,9 @@ LUAU_FASTINTVARIABLE(LuauTypeInferRecursionLimit, 500) LUAU_FASTINTVARIABLE(LuauTypeInferTypePackLoopLimit, 5000) LUAU_FASTINTVARIABLE(LuauCheckRecursionLimit, 500) LUAU_FASTFLAG(LuauKnowsTheDataModel3) +LUAU_FASTFLAG(LuauSeparateTypechecks) +LUAU_FASTFLAG(LuauAutocompleteSingletonTypes) +LUAU_FASTFLAGVARIABLE(LuauCyclicModuleTypeSurface, false) LUAU_FASTFLAGVARIABLE(LuauEqConstraint, false) LUAU_FASTFLAGVARIABLE(LuauWeakEqConstraint, false) // Eventually removed as false. LUAU_FASTFLAGVARIABLE(DebugLuauFreezeDuringUnification, false) @@ -35,7 +38,7 @@ LUAU_FASTFLAGVARIABLE(LuauExpectedTypesOfProperties, false) LUAU_FASTFLAGVARIABLE(LuauErrorRecoveryType, false) LUAU_FASTFLAGVARIABLE(LuauOnlyMutateInstantiatedTables, false) LUAU_FASTFLAGVARIABLE(LuauPropertiesGetExpectedType, false) -LUAU_FASTFLAGVARIABLE(LuauStatFunctionSimplify3, false) +LUAU_FASTFLAGVARIABLE(LuauStatFunctionSimplify4, false) LUAU_FASTFLAGVARIABLE(LuauUnsealedTableLiteral, false) LUAU_FASTFLAG(LuauTypeMismatchModuleName) LUAU_FASTFLAGVARIABLE(LuauTwoPassAliasDefinitionFix, false) @@ -46,6 +49,7 @@ LUAU_FASTFLAGVARIABLE(LuauDoNotTryToReduce, false) LUAU_FASTFLAGVARIABLE(LuauDoNotAccidentallyDependOnPointerOrdering, false) LUAU_FASTFLAGVARIABLE(LuauFixArgumentCountMismatchAmountWithGenericTypes, false) LUAU_FASTFLAGVARIABLE(LuauFixIncorrectLineNumberDuplicateType, false) +LUAU_FASTFLAGVARIABLE(LuauCheckImplicitNumbericKeys, false) LUAU_FASTFLAG(LuauAnyInIsOptionalIsOptional) LUAU_FASTFLAGVARIABLE(LuauDecoupleOperatorInferenceFromUnifiedTypeInference, false) LUAU_FASTFLAGVARIABLE(LuauArgCountMismatchSaysAtLeastWhenVariadic, false) @@ -53,6 +57,11 @@ LUAU_FASTFLAGVARIABLE(LuauArgCountMismatchSaysAtLeastWhenVariadic, false) namespace Luau { +const char* TimeLimitError::what() const throw() +{ + return "Typeinfer failed to complete in allotted time"; +} + static bool typeCouldHaveMetatable(TypeId ty) { return get(follow(ty)) || get(follow(ty)) || get(follow(ty)); @@ -251,6 +260,12 @@ ModulePtr TypeChecker::check(const SourceModule& module, Mode mode, std::optiona currentModule.reset(new Module()); currentModule->type = module.type; + if (FFlag::LuauSeparateTypechecks) + { + currentModule->allocator = module.allocator; + currentModule->names = module.names; + } + iceHandler->moduleName = module.name; ScopePtr parentScope = environmentScope.value_or(globalScope); @@ -271,7 +286,21 @@ ModulePtr TypeChecker::check(const SourceModule& module, Mode mode, std::optiona if (prepareModuleScope) prepareModuleScope(module.name, currentModule->getModuleScope()); - checkBlock(moduleScope, *module.root); + if (FFlag::LuauSeparateTypechecks) + { + try + { + checkBlock(moduleScope, *module.root); + } + catch (const TimeLimitError&) + { + currentModule->timeout = true; + } + } + else + { + checkBlock(moduleScope, *module.root); + } if (get(follow(moduleScope->returnType))) moduleScope->returnType = addTypePack(TypePack{{}, std::nullopt}); @@ -366,6 +395,9 @@ void TypeChecker::check(const ScopePtr& scope, const AstStat& program) } else ice("Unknown AstStat"); + + if (FFlag::LuauSeparateTypechecks && finishTime && TimeTrace::getClock() > *finishTime) + throw TimeLimitError(); } // This particular overload is for do...end. If you need to not increase the scope level, use checkBlock directly. @@ -1115,22 +1147,18 @@ void TypeChecker::check(const ScopePtr& scope, TypeId ty, const ScopePtr& funSco scope->bindings[name->local] = {anyIfNonstrict(quantify(funScope, ty, name->local->location)), name->local->location}; return; } - else if (auto name = function.name->as(); name && FFlag::LuauStatFunctionSimplify3) + else if (auto name = function.name->as(); name && FFlag::LuauStatFunctionSimplify4) { TypeId exprTy = checkExpr(scope, *name->expr).type; TableTypeVar* ttv = getMutableTableType(exprTy); - if (!ttv) + + if (!getIndexTypeFromType(scope, exprTy, name->index.value, name->indexLocation, false)) { - if (isTableIntersection(exprTy)) + if (ttv || isTableIntersection(exprTy)) reportError(TypeError{function.location, CannotExtendTable{exprTy, CannotExtendTable::Property, name->index.value}}); - else if (!get(exprTy) && !get(exprTy)) + else reportError(TypeError{function.location, OnlyTablesCanHaveMethods{exprTy}}); } - else if (ttv->state == TableState::Sealed) - { - if (!getIndexTypeFromType(scope, exprTy, name->index.value, name->indexLocation, false)) - reportError(TypeError{function.location, CannotExtendTable{exprTy, CannotExtendTable::Property, name->index.value}}); - } ty = follow(ty); @@ -1153,7 +1181,7 @@ void TypeChecker::check(const ScopePtr& scope, TypeId ty, const ScopePtr& funSco if (ttv && ttv->state != TableState::Sealed) ttv->props[name->index.value] = {follow(quantify(funScope, ty, name->indexLocation)), /* deprecated */ false, {}, name->indexLocation}; } - else if (FFlag::LuauStatFunctionSimplify3) + else if (FFlag::LuauStatFunctionSimplify4) { LUAU_ASSERT(function.name->is()); @@ -1163,7 +1191,7 @@ void TypeChecker::check(const ScopePtr& scope, TypeId ty, const ScopePtr& funSco } else if (function.func->self) { - LUAU_ASSERT(!FFlag::LuauStatFunctionSimplify3); + LUAU_ASSERT(!FFlag::LuauStatFunctionSimplify4); AstExprIndexName* indexName = function.name->as(); if (!indexName) @@ -1202,7 +1230,7 @@ void TypeChecker::check(const ScopePtr& scope, TypeId ty, const ScopePtr& funSco } else { - LUAU_ASSERT(!FFlag::LuauStatFunctionSimplify3); + LUAU_ASSERT(!FFlag::LuauStatFunctionSimplify4); TypeId leftType = checkLValueBinding(scope, *function.name); @@ -2030,7 +2058,11 @@ TypeId TypeChecker::checkExprTable( indexer = expectedTable->indexer; if (indexer) + { + if (FFlag::LuauCheckImplicitNumbericKeys) + unify(numberType, indexer->indexType, value->location); unify(valueType, indexer->indexResultType, value->location); + } else indexer = TableIndexer{numberType, anyIfNonstrict(valueType)}; } @@ -2984,35 +3016,33 @@ TypeId TypeChecker::checkFunctionName(const ScopePtr& scope, AstExpr& funName, T else if (auto indexName = funName.as()) { TypeId lhsType = checkExpr(scope, *indexName->expr).type; - if (get(lhsType) || get(lhsType)) + + if (!FFlag::LuauStatFunctionSimplify4 && (get(lhsType) || get(lhsType))) return lhsType; TableTypeVar* ttv = getMutableTableType(lhsType); - if (!ttv) + + if (FFlag::LuauStatFunctionSimplify4) { - if (!FFlag::LuauErrorRecoveryType && !isTableIntersection(lhsType)) - // This error now gets reported when we check the function body. - reportError(TypeError{funName.location, OnlyTablesCanHaveMethods{lhsType}}); - - return errorRecoveryType(scope); - } - - if (FFlag::LuauStatFunctionSimplify3) - { - if (lhsType->persistent) - return errorRecoveryType(scope); - - // Cannot extend sealed table, but we dont report an error here because it will be reported during AstStatFunction check - if (ttv->state == TableState::Sealed) + if (!ttv || ttv->state == TableState::Sealed) { - if (ttv->indexer && isPrim(ttv->indexer->indexType, PrimitiveTypeVar::String)) - return ttv->indexer->indexResultType; - else - return errorRecoveryType(scope); + if (auto ty = getIndexTypeFromType(scope, lhsType, indexName->index.value, indexName->indexLocation, false)) + return *ty; + + return errorRecoveryType(scope); } } else { + if (!ttv) + { + if (!FFlag::LuauErrorRecoveryType && !isTableIntersection(lhsType)) + // This error now gets reported when we check the function body. + reportError(TypeError{funName.location, OnlyTablesCanHaveMethods{lhsType}}); + + return errorRecoveryType(scope); + } + if (lhsType->persistent || ttv->state == TableState::Sealed) return errorRecoveryType(scope); } @@ -3020,7 +3050,12 @@ TypeId TypeChecker::checkFunctionName(const ScopePtr& scope, AstExpr& funName, T Name name = indexName->index.value; if (ttv->props.count(name)) - return errorRecoveryType(scope); + { + if (FFlag::LuauStatFunctionSimplify4) + return ttv->props[name].type; + else + return errorRecoveryType(scope); + } Property& property = ttv->props[name]; @@ -4155,6 +4190,20 @@ TypeId TypeChecker::checkRequire(const ScopePtr& scope, const ModuleInfo& module return anyType; } + // Types of requires that transitively refer to current module have to be replaced with 'any' + std::string humanReadableName; + + if (FFlag::LuauCyclicModuleTypeSurface) + { + humanReadableName = resolver->getHumanReadableModuleName(moduleInfo.name); + + for (const auto& [location, path] : requireCycles) + { + if (!path.empty() && path.front() == humanReadableName) + return anyType; + } + } + ModulePtr module = resolver->getModule(moduleInfo.name); if (!module) { @@ -4163,8 +4212,15 @@ TypeId TypeChecker::checkRequire(const ScopePtr& scope, const ModuleInfo& module // we will already have reported the error. if (!resolver->moduleExists(moduleInfo.name) && !moduleInfo.optional) { - std::string reportedModulePath = resolver->getHumanReadableModuleName(moduleInfo.name); - reportError(TypeError{location, UnknownRequire{reportedModulePath}}); + if (FFlag::LuauCyclicModuleTypeSurface) + { + reportError(TypeError{location, UnknownRequire{humanReadableName}}); + } + else + { + std::string reportedModulePath = resolver->getHumanReadableModuleName(moduleInfo.name); + reportError(TypeError{location, UnknownRequire{reportedModulePath}}); + } } return errorRecoveryType(scope); @@ -4172,8 +4228,15 @@ TypeId TypeChecker::checkRequire(const ScopePtr& scope, const ModuleInfo& module if (module->type != SourceCode::Module) { - std::string humanReadableName = resolver->getHumanReadableModuleName(moduleInfo.name); - reportError(location, IllegalRequire{humanReadableName, "Module is not a ModuleScript. It cannot be required."}); + if (FFlag::LuauCyclicModuleTypeSurface) + { + reportError(location, IllegalRequire{humanReadableName, "Module is not a ModuleScript. It cannot be required."}); + } + else + { + std::string humanReadableName = resolver->getHumanReadableModuleName(moduleInfo.name); + reportError(location, IllegalRequire{humanReadableName, "Module is not a ModuleScript. It cannot be required."}); + } return errorRecoveryType(scope); } @@ -4185,8 +4248,15 @@ TypeId TypeChecker::checkRequire(const ScopePtr& scope, const ModuleInfo& module std::optional moduleType = first(modulePack); if (!moduleType) { - std::string humanReadableName = resolver->getHumanReadableModuleName(moduleInfo.name); - reportError(location, IllegalRequire{humanReadableName, "Module does not return exactly 1 value. It cannot be required."}); + if (FFlag::LuauCyclicModuleTypeSurface) + { + reportError(location, IllegalRequire{humanReadableName, "Module does not return exactly 1 value. It cannot be required."}); + } + else + { + std::string humanReadableName = resolver->getHumanReadableModuleName(moduleInfo.name); + reportError(location, IllegalRequire{humanReadableName, "Module does not return exactly 1 value. It cannot be required."}); + } return errorRecoveryType(scope); } @@ -4629,7 +4699,9 @@ TypeId TypeChecker::freshType(TypeLevel level) TypeId TypeChecker::singletonType(bool value) { - // TODO: cache singleton types + if (FFlag::LuauAutocompleteSingletonTypes) + return value ? getSingletonTypes().trueType : getSingletonTypes().falseType; + return currentModule->internalTypes.addType(TypeVar(SingletonTypeVar(BooleanSingleton{value}))); } diff --git a/Analysis/src/TypeVar.cpp b/Analysis/src/TypeVar.cpp index 36545ad9..dbc412fc 100644 --- a/Analysis/src/TypeVar.cpp +++ b/Analysis/src/TypeVar.cpp @@ -652,6 +652,8 @@ static TypeVar numberType_{PrimitiveTypeVar{PrimitiveTypeVar::Number}, /*persist static TypeVar stringType_{PrimitiveTypeVar{PrimitiveTypeVar::String}, /*persistent*/ true}; static TypeVar booleanType_{PrimitiveTypeVar{PrimitiveTypeVar::Boolean}, /*persistent*/ true}; static TypeVar threadType_{PrimitiveTypeVar{PrimitiveTypeVar::Thread}, /*persistent*/ true}; +static TypeVar trueType_{SingletonTypeVar{BooleanSingleton{true}}, /*persistent*/ true}; +static TypeVar falseType_{SingletonTypeVar{BooleanSingleton{false}}, /*persistent*/ true}; static TypeVar anyType_{AnyTypeVar{}}; static TypeVar errorType_{ErrorTypeVar{}}; static TypeVar optionalNumberType_{UnionTypeVar{{&numberType_, &nilType_}}}; @@ -665,6 +667,8 @@ SingletonTypes::SingletonTypes() , stringType(&stringType_) , booleanType(&booleanType_) , threadType(&threadType_) + , trueType(&trueType_) + , falseType(&falseType_) , anyType(&anyType_) , optionalNumberType(&optionalNumberType_) , anyTypePack(&anyTypePack_) diff --git a/Ast/include/Luau/TimeTrace.h b/Ast/include/Luau/TimeTrace.h index 5018456f..9f7b2bdf 100644 --- a/Ast/include/Luau/TimeTrace.h +++ b/Ast/include/Luau/TimeTrace.h @@ -9,14 +9,21 @@ LUAU_FASTFLAG(DebugLuauTimeTracing) +namespace Luau +{ +namespace TimeTrace +{ +double getClock(); +uint32_t getClockMicroseconds(); +} // namespace TimeTrace +} // namespace Luau + #if defined(LUAU_ENABLE_TIME_TRACE) namespace Luau { namespace TimeTrace { -uint32_t getClockMicroseconds(); - struct Token { const char* name; diff --git a/Ast/src/Parser.cpp b/Ast/src/Parser.cpp index f6dfd904..f9d32178 100644 --- a/Ast/src/Parser.cpp +++ b/Ast/src/Parser.cpp @@ -10,6 +10,7 @@ // See docs/SyntaxChanges.md for an explanation. LUAU_FASTINTVARIABLE(LuauRecursionLimit, 1000) LUAU_FASTINTVARIABLE(LuauParseErrorLimit, 100) +LUAU_FASTFLAGVARIABLE(LuauParseRecoverUnexpectedPack, false) namespace Luau { @@ -1420,6 +1421,11 @@ AstType* Parser::parseTypeAnnotation(TempVector& parts, const Location parts.push_back(parseSimpleTypeAnnotation(/* allowPack= */ false).type); isIntersection = true; } + else if (FFlag::LuauParseRecoverUnexpectedPack && c == Lexeme::Dot3) + { + report(lexer.current().location, "Unexpected '...' after type annotation"); + nextLexeme(); + } else break; } @@ -1536,6 +1542,11 @@ AstTypeOrPack Parser::parseSimpleTypeAnnotation(bool allowPack) prefix = name.name; name = parseIndexName("field name", pointPosition); } + else if (FFlag::LuauParseRecoverUnexpectedPack && lexer.current().type == Lexeme::Dot3) + { + report(lexer.current().location, "Unexpected '...' after type name; type pack is not allowed in this context"); + nextLexeme(); + } else if (name.name == "typeof") { Lexeme typeofBegin = lexer.current(); diff --git a/Ast/src/TimeTrace.cpp b/Ast/src/TimeTrace.cpp index 19564f05..e3807683 100644 --- a/Ast/src/TimeTrace.cpp +++ b/Ast/src/TimeTrace.cpp @@ -26,9 +26,6 @@ #include LUAU_FASTFLAGVARIABLE(DebugLuauTimeTracing, false) - -#if defined(LUAU_ENABLE_TIME_TRACE) - namespace Luau { namespace TimeTrace @@ -67,6 +64,14 @@ static double getClockTimestamp() #endif } +double getClock() +{ + static double period = getClockPeriod(); + static double start = getClockTimestamp(); + + return (getClockTimestamp() - start) * period; +} + uint32_t getClockMicroseconds() { static double period = getClockPeriod() * 1e6; @@ -74,7 +79,15 @@ uint32_t getClockMicroseconds() return uint32_t((getClockTimestamp() - start) * period); } +} // namespace TimeTrace +} // namespace Luau +#if defined(LUAU_ENABLE_TIME_TRACE) + +namespace Luau +{ +namespace TimeTrace +{ struct GlobalContext { GlobalContext() = default; diff --git a/Compiler/src/ConstantFolding.cpp b/Compiler/src/ConstantFolding.cpp index 60a7c169..35ea0bf0 100644 --- a/Compiler/src/ConstantFolding.cpp +++ b/Compiler/src/ConstantFolding.cpp @@ -290,7 +290,8 @@ struct ConstantVisitor : AstVisitor Constant la = analyze(expr->left); Constant ra = analyze(expr->right); - if (la.type != Constant::Type_Unknown && ra.type != Constant::Type_Unknown) + // note: ra doesn't need to be constant to fold and/or + if (la.type != Constant::Type_Unknown) foldBinary(result, expr->op, la, ra); } else if (AstExprTypeAssertion* expr = node->as()) diff --git a/Sources.cmake b/Sources.cmake index 59b38497..6f110f1f 100644 --- a/Sources.cmake +++ b/Sources.cmake @@ -47,6 +47,7 @@ target_sources(Luau.Analysis PRIVATE Analysis/include/Luau/Autocomplete.h Analysis/include/Luau/BuiltinDefinitions.h Analysis/include/Luau/Config.h + Analysis/include/Luau/Clone.h Analysis/include/Luau/Documentation.h Analysis/include/Luau/Error.h Analysis/include/Luau/FileResolver.h @@ -85,6 +86,7 @@ target_sources(Luau.Analysis PRIVATE Analysis/src/Autocomplete.cpp Analysis/src/BuiltinDefinitions.cpp Analysis/src/Config.cpp + Analysis/src/Clone.cpp Analysis/src/Error.cpp Analysis/src/Frontend.cpp Analysis/src/IostreamHelpers.cpp diff --git a/VM/src/lfunc.h b/VM/src/lfunc.h index 8047cebe..a260d00a 100644 --- a/VM/src/lfunc.h +++ b/VM/src/lfunc.h @@ -14,6 +14,6 @@ LUAI_FUNC UpVal* luaF_findupval(lua_State* L, StkId level); LUAI_FUNC void luaF_close(lua_State* L, StkId level); LUAI_FUNC void luaF_freeproto(lua_State* L, Proto* f, struct lua_Page* page); LUAI_FUNC void luaF_freeclosure(lua_State* L, Closure* c, struct lua_Page* page); -void luaF_unlinkupval(UpVal* uv); +LUAI_FUNC void luaF_unlinkupval(UpVal* uv); LUAI_FUNC void luaF_freeupval(lua_State* L, UpVal* uv, struct lua_Page* page); LUAI_FUNC const LocVar* luaF_getlocal(const Proto* func, int local_number, int pc); diff --git a/VM/src/ltablib.cpp b/VM/src/ltablib.cpp index 241a99e3..41887f4b 100644 --- a/VM/src/ltablib.cpp +++ b/VM/src/ltablib.cpp @@ -201,7 +201,7 @@ static int tmove(lua_State* L) void (*telemetrycb)(lua_State* L, int f, int e, int t, int nf, int nt) = lua_table_move_telemetry; - if (DFFlag::LuauTableMoveTelemetry2 && telemetrycb) + if (DFFlag::LuauTableMoveTelemetry2 && telemetrycb && e >= f) { int nf = lua_objlen(L, 1); int nt = lua_objlen(L, tt); diff --git a/tests/Autocomplete.test.cpp b/tests/Autocomplete.test.cpp index 4e8a1d55..2e7902f5 100644 --- a/tests/Autocomplete.test.cpp +++ b/tests/Autocomplete.test.cpp @@ -14,6 +14,7 @@ LUAU_FASTFLAG(LuauTraceTypesInNonstrictMode2) LUAU_FASTFLAG(LuauSetMetatableDoesNotTimeTravel) +LUAU_FASTFLAG(LuauSeparateTypechecks) using namespace Luau; @@ -25,6 +26,11 @@ static std::optional nullCallback(std::string tag, std::op template struct ACFixtureImpl : BaseType { + ACFixtureImpl() + : Fixture(true, true) + { + } + AutocompleteResult autocomplete(unsigned row, unsigned column) { return Luau::autocomplete(this->frontend, "MainModule", Position{row, column}, nullCallback); @@ -72,7 +78,25 @@ struct ACFixtureImpl : BaseType } LUAU_ASSERT("Digit expected after @ symbol" && prevChar != '@'); - return Fixture::check(filteredSource); + return BaseType::check(filteredSource); + } + + LoadDefinitionFileResult loadDefinition(const std::string& source) + { + if (FFlag::LuauSeparateTypechecks) + { + TypeChecker& typeChecker = this->frontend.typeCheckerForAutocomplete; + unfreeze(typeChecker.globalTypes); + LoadDefinitionFileResult result = loadDefinitionFile(typeChecker, typeChecker.globalScope, source, "@test"); + freeze(typeChecker.globalTypes); + + REQUIRE_MESSAGE(result.success, "loadDefinition: unable to load definition file"); + return result; + } + else + { + return BaseType::loadDefinition(source); + } } const Position& getPosition(char marker) const @@ -2496,7 +2520,7 @@ local t = { CHECK(ac.entryMap.count("second")); } -TEST_CASE_FIXTURE(Fixture, "autocomplete_documentation_symbols") +TEST_CASE_FIXTURE(ACFixture, "autocomplete_documentation_symbols") { loadDefinition(R"( declare y: { @@ -2504,13 +2528,11 @@ TEST_CASE_FIXTURE(Fixture, "autocomplete_documentation_symbols") } )"); - fileResolver.source["Module/A"] = R"( - local a = y. - )"; + check(R"( + local a = y.@1 + )"); - frontend.check("Module/A"); - - auto ac = autocomplete(frontend, "Module/A", Position{1, 21}, nullCallback); + auto ac = autocomplete('1'); REQUIRE(ac.entryMap.count("x")); CHECK_EQ(ac.entryMap["x"].documentationSymbol, "@test/global/y.x"); @@ -2736,6 +2758,107 @@ TEST_CASE_FIXTURE(ACFixture, "autocomplete_on_string_singletons") CHECK(ac.entryMap.count("format")); } +TEST_CASE_FIXTURE(ACFixture, "autocomplete_string_singletons") +{ + ScopedFastFlag luauAutocompleteSingletonTypes{"LuauAutocompleteSingletonTypes", true}; + ScopedFastFlag luauExpectedTypesOfProperties{"LuauExpectedTypesOfProperties", true}; + + check(R"( + type tag = "cat" | "dog" + local function f(a: tag) end + f("@1") + f(@2) + local x: tag = "@3" + )"); + + auto ac = autocomplete('1'); + + CHECK(ac.entryMap.count("cat")); + CHECK(ac.entryMap.count("dog")); + + ac = autocomplete('2'); + + CHECK(ac.entryMap.count("\"cat\"")); + CHECK(ac.entryMap.count("\"dog\"")); + + ac = autocomplete('3'); + + CHECK(ac.entryMap.count("cat")); + CHECK(ac.entryMap.count("dog")); + + check(R"( + type tagged = {tag:"cat", fieldx:number} | {tag:"dog", fieldy:number} + local x: tagged = {tag="@4"} + )"); + + ac = autocomplete('4'); + + CHECK(ac.entryMap.count("cat")); + CHECK(ac.entryMap.count("dog")); +} + +TEST_CASE_FIXTURE(ACFixture, "autocomplete_string_singleton_equality") +{ + ScopedFastFlag luauAutocompleteSingletonTypes{"LuauAutocompleteSingletonTypes", true}; + + check(R"( + type tagged = {tag:"cat", fieldx:number} | {tag:"dog", fieldy:number} + local x: tagged = {tag="cat", fieldx=2} + if x.tag == "@1" or "@2" ~= x.tag then end + )"); + + auto ac = autocomplete('1'); + + CHECK(ac.entryMap.count("cat")); + CHECK(ac.entryMap.count("dog")); + + ac = autocomplete('2'); + + CHECK(ac.entryMap.count("cat")); + CHECK(ac.entryMap.count("dog")); + + // CLI-48823: assignment to x.tag should also autocomplete, but union l-values are not supported yet +} + +TEST_CASE_FIXTURE(ACFixture, "autocomplete_boolean_singleton") +{ + ScopedFastFlag luauAutocompleteSingletonTypes{"LuauAutocompleteSingletonTypes", true}; + + check(R"( +local function f(x: true) end +f(@1) + )"); + + auto ac = autocomplete('1'); + + REQUIRE(ac.entryMap.count("true")); + CHECK(ac.entryMap["true"].typeCorrect == TypeCorrectKind::Correct); + REQUIRE(ac.entryMap.count("false")); + CHECK(ac.entryMap["false"].typeCorrect == TypeCorrectKind::None); +} + +TEST_CASE_FIXTURE(ACFixture, "autocomplete_string_singleton_escape") +{ + ScopedFastFlag luauAutocompleteSingletonTypes{"LuauAutocompleteSingletonTypes", true}; + + check(R"( + type tag = "strange\t\"cat\"" | 'nice\t"dog"' + local function f(x: tag) end + f(@1) + f("@2") + )"); + + auto ac = autocomplete('1'); + + CHECK(ac.entryMap.count("\"strange\\t\\\"cat\\\"\"")); + CHECK(ac.entryMap.count("\"nice\\t\\\"dog\\\"\"")); + + ac = autocomplete('2'); + + CHECK(ac.entryMap.count("strange\\t\\\"cat\\\"")); + CHECK(ac.entryMap.count("nice\\t\\\"dog\\\"")); +} + TEST_CASE_FIXTURE(ACFixture, "function_in_assignment_has_parentheses_2") { check(R"( diff --git a/tests/Compiler.test.cpp b/tests/Compiler.test.cpp index 3dc57da0..83dad729 100644 --- a/tests/Compiler.test.cpp +++ b/tests/Compiler.test.cpp @@ -1074,6 +1074,39 @@ RETURN R1 1 )"); } +TEST_CASE("AndOrFoldLeft") +{ + // constant folding and/or expression is possible even if just the left hand is constant + CHECK_EQ("\n" + compileFunction0("local a = false if a and b then b() end"), R"( +RETURN R0 0 +)"); + + CHECK_EQ("\n" + compileFunction0("local a = true if a or b then b() end"), R"( +GETIMPORT R0 1 +CALL R0 0 0 +RETURN R0 0 +)"); + + // however, if right hand side is constant we can't constant fold the entire expression + // (note that we don't need to evaluate the right hand side, but we do need a branch) + CHECK_EQ("\n" + compileFunction0("local a = false if b and a then b() end"), R"( +GETIMPORT R0 1 +JUMPIFNOT R0 +4 +RETURN R0 0 +GETIMPORT R0 1 +CALL R0 0 0 +RETURN R0 0 +)"); + + CHECK_EQ("\n" + compileFunction0("local a = true if b or a then b() end"), R"( +GETIMPORT R0 1 +JUMPIF R0 +0 +GETIMPORT R0 1 +CALL R0 0 0 +RETURN R0 0 +)"); +} + TEST_CASE("AndOrChainCodegen") { const char* source = R"( diff --git a/tests/Fixture.cpp b/tests/Fixture.cpp index a7e7ea39..9dc9feee 100644 --- a/tests/Fixture.cpp +++ b/tests/Fixture.cpp @@ -83,7 +83,7 @@ std::optional TestFileResolver::getEnvironmentForModule(const Modul return std::nullopt; } -Fixture::Fixture(bool freeze) +Fixture::Fixture(bool freeze, bool prepareAutocomplete) : sff_DebugLuauFreezeArena("DebugLuauFreezeArena", freeze) , frontend(&fileResolver, &configResolver, {/* retainFullTypeGraphs= */ true}) , typeChecker(frontend.typeChecker) @@ -93,8 +93,11 @@ Fixture::Fixture(bool freeze) configResolver.defaultConfig.parseOptions.captureComments = true; registerBuiltinTypes(frontend.typeChecker); + if (prepareAutocomplete) + registerBuiltinTypes(frontend.typeCheckerForAutocomplete); registerTestTypes(); Luau::freeze(frontend.typeChecker.globalTypes); + Luau::freeze(frontend.typeCheckerForAutocomplete.globalTypes); Luau::setPrintLine([](auto s) {}); } diff --git a/tests/Fixture.h b/tests/Fixture.h index 4e45a952..0d1233bf 100644 --- a/tests/Fixture.h +++ b/tests/Fixture.h @@ -91,7 +91,7 @@ struct TestConfigResolver : ConfigResolver struct Fixture { - explicit Fixture(bool freeze = true); + explicit Fixture(bool freeze = true, bool prepareAutocomplete = false); ~Fixture(); // Throws Luau::ParseErrors if the parse fails. diff --git a/tests/Frontend.test.cpp b/tests/Frontend.test.cpp index 8a59acd1..9fc0a005 100644 --- a/tests/Frontend.test.cpp +++ b/tests/Frontend.test.cpp @@ -384,6 +384,70 @@ TEST_CASE_FIXTURE(FrontendFixture, "cycle_error_paths") CHECK_EQ(ce2->cycle[1], "game/Gui/Modules/A"); } +TEST_CASE_FIXTURE(FrontendFixture, "cycle_incremental_type_surface") +{ + ScopedFastFlag luauCyclicModuleTypeSurface{"LuauCyclicModuleTypeSurface", true}; + + fileResolver.source["game/A"] = R"( + return {hello = 2} + )"; + + CheckResult result = frontend.check("game/A"); + LUAU_REQUIRE_NO_ERRORS(result); + + fileResolver.source["game/A"] = R"( + local me = require(game.A) + return {hello = 2} + )"; + frontend.markDirty("game/A"); + + result = frontend.check("game/A"); + LUAU_REQUIRE_ERRORS(result); + + auto ty = requireType("game/A", "me"); + CHECK_EQ(toString(ty), "any"); +} + +TEST_CASE_FIXTURE(FrontendFixture, "cycle_incremental_type_surface_longer") +{ + ScopedFastFlag luauCyclicModuleTypeSurface{"LuauCyclicModuleTypeSurface", true}; + + fileResolver.source["game/A"] = R"( + return {mod_a = 2} + )"; + + CheckResult result = frontend.check("game/A"); + LUAU_REQUIRE_NO_ERRORS(result); + + fileResolver.source["game/B"] = R"( + local me = require(game.A) + return {mod_b = 4} + )"; + + result = frontend.check("game/B"); + LUAU_REQUIRE_NO_ERRORS(result); + + fileResolver.source["game/A"] = R"( + local me = require(game.B) + return {mod_a_prime = 3} + )"; + + frontend.markDirty("game/A"); + frontend.markDirty("game/B"); + + result = frontend.check("game/A"); + LUAU_REQUIRE_ERRORS(result); + + TypeId tyA = requireType("game/A", "me"); + CHECK_EQ(toString(tyA), "any"); + + result = frontend.check("game/B"); + LUAU_REQUIRE_ERRORS(result); + + TypeId tyB = requireType("game/B", "me"); + CHECK_EQ(toString(tyB), "any"); +} + TEST_CASE_FIXTURE(FrontendFixture, "dont_reparse_clean_file_when_linting") { fileResolver.source["Modules/A"] = R"( diff --git a/tests/Module.test.cpp b/tests/Module.test.cpp index 82b7a350..de063121 100644 --- a/tests/Module.test.cpp +++ b/tests/Module.test.cpp @@ -1,4 +1,5 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +#include "Luau/Clone.h" #include "Luau/Module.h" #include "Luau/Scope.h" diff --git a/tests/Parser.test.cpp b/tests/Parser.test.cpp index 7dacc669..79f9ecab 100644 --- a/tests/Parser.test.cpp +++ b/tests/Parser.test.cpp @@ -2022,6 +2022,15 @@ TEST_CASE_FIXTURE(Fixture, "parse_type_alias_default_type_errors") matchParseError("type Y number> = {}", "Expected type pack after '=', got type", Location{{0, 14}, {0, 32}}); } +TEST_CASE_FIXTURE(Fixture, "parse_type_pack_errors") +{ + ScopedFastFlag luauParseRecoverUnexpectedPack{"LuauParseRecoverUnexpectedPack", true}; + + matchParseError("type Y = {a: T..., b: number}", "Unexpected '...' after type name; type pack is not allowed in this context", + Location{{0, 20}, {0, 23}}); + matchParseError("type Y = {a: (number | string)...", "Unexpected '...' after type annotation", Location{{0, 36}, {0, 39}}); +} + TEST_CASE_FIXTURE(Fixture, "parse_if_else_expression") { { @@ -2590,4 +2599,16 @@ type Y = (T...) -> U... CHECK_EQ(1, result.errors.size()); } +TEST_CASE_FIXTURE(Fixture, "recover_unexpected_type_pack") +{ + ScopedFastFlag luauParseRecoverUnexpectedPack{"LuauParseRecoverUnexpectedPack", true}; + + ParseResult result = tryParse(R"( +type X = { a: T..., b: number } +type Y = { a: T..., b: number } +type Z = { a: string | T..., b: number } + )"); + REQUIRE_EQ(3, result.errors.size()); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.functions.test.cpp b/tests/TypeInfer.functions.test.cpp index dbae7b54..1713216a 100644 --- a/tests/TypeInfer.functions.test.cpp +++ b/tests/TypeInfer.functions.test.cpp @@ -1270,7 +1270,7 @@ caused by: TEST_CASE_FIXTURE(Fixture, "function_decl_quantify_right_type") { - ScopedFastFlag statFunctionSimplify{"LuauStatFunctionSimplify3", true}; + ScopedFastFlag statFunctionSimplify{"LuauStatFunctionSimplify4", true}; fileResolver.source["game/isAMagicMock"] = R"( --!nonstrict @@ -1294,7 +1294,7 @@ end TEST_CASE_FIXTURE(Fixture, "function_decl_non_self_sealed_overwrite") { - ScopedFastFlag statFunctionSimplify{"LuauStatFunctionSimplify3", true}; + ScopedFastFlag statFunctionSimplify{"LuauStatFunctionSimplify4", true}; CheckResult result = check(R"( function string.len(): number @@ -1316,7 +1316,7 @@ print(string.len('hello')) TEST_CASE_FIXTURE(Fixture, "function_decl_non_self_sealed_overwrite_2") { - ScopedFastFlag statFunctionSimplify{"LuauStatFunctionSimplify3", true}; + ScopedFastFlag statFunctionSimplify{"LuauStatFunctionSimplify4", true}; ScopedFastFlag inferStatFunction{"LuauInferStatFunction", true}; CheckResult result = check(R"( @@ -1324,12 +1324,12 @@ local t: { f: ((x: number) -> number)? } = {} function t.f(x) print(x + 5) - return x .. "asd" + return x .. "asd" -- 1st error: we know that return type is a number, not a string end t.f = function(x) print(x + 5) - return x .. "asd" + return x .. "asd" -- 2nd error: we know that return type is a number, not a string end )"); @@ -1338,6 +1338,33 @@ end CHECK_EQ(toString(result.errors[1]), R"(Type 'string' could not be converted into 'number')"); } +TEST_CASE_FIXTURE(Fixture, "function_decl_non_self_unsealed_overwrite") +{ + ScopedFastFlag statFunctionSimplify{"LuauStatFunctionSimplify4", true}; + ScopedFastFlag inferStatFunction{"LuauInferStatFunction", true}; + + CheckResult result = check(R"( +local t = { f = nil :: ((x: number) -> number)? } + +function t.f(x: string): string -- 1st error: new function value type is incompatible + return x .. "asd" +end + +t.f = function(x) + print(x + 5) + return x .. "asd" -- 2nd error: we know that return type is a number, not a string +end + )"); + + LUAU_REQUIRE_ERROR_COUNT(2, result); + CHECK_EQ(toString(result.errors[0]), R"(Type '(string) -> string' could not be converted into '((number) -> number)?' +caused by: + None of the union options are compatible. For example: Type '(string) -> string' could not be converted into '(number) -> number' +caused by: + Argument #1 type is not compatible. Type 'number' could not be converted into 'string')"); + CHECK_EQ(toString(result.errors[1]), R"(Type 'string' could not be converted into 'number')"); +} + TEST_CASE_FIXTURE(Fixture, "strict_mode_ok_with_missing_arguments") { ScopedFastFlag sff{"LuauAnyInIsOptionalIsOptional", true}; @@ -1352,7 +1379,7 @@ TEST_CASE_FIXTURE(Fixture, "strict_mode_ok_with_missing_arguments") TEST_CASE_FIXTURE(Fixture, "function_statement_sealed_table_assignment_through_indexer") { - ScopedFastFlag statFunctionSimplify{"LuauStatFunctionSimplify3", true}; + ScopedFastFlag statFunctionSimplify{"LuauStatFunctionSimplify4", true}; CheckResult result = check(R"( local t: {[string]: () -> number} = {} diff --git a/tests/TypeInfer.intersectionTypes.test.cpp b/tests/TypeInfer.intersectionTypes.test.cpp index d146f4e8..ac7a6532 100644 --- a/tests/TypeInfer.intersectionTypes.test.cpp +++ b/tests/TypeInfer.intersectionTypes.test.cpp @@ -311,6 +311,8 @@ TEST_CASE_FIXTURE(Fixture, "table_intersection_write_sealed") TEST_CASE_FIXTURE(Fixture, "table_intersection_write_sealed_indirect") { + ScopedFastFlag statFunctionSimplify{"LuauStatFunctionSimplify4", true}; + CheckResult result = check(R"( type X = { x: (number) -> number } type Y = { y: (string) -> string } @@ -326,10 +328,39 @@ TEST_CASE_FIXTURE(Fixture, "table_intersection_write_sealed_indirect") function xy:w(a:number) return a * 10 end )"); - LUAU_REQUIRE_ERROR_COUNT(3, result); - CHECK_EQ(toString(result.errors[0]), "Cannot add property 'z' to table 'X & Y'"); - CHECK_EQ(toString(result.errors[1]), "Cannot add property 'y' to table 'X & Y'"); - CHECK_EQ(toString(result.errors[2]), "Cannot add property 'w' to table 'X & Y'"); + LUAU_REQUIRE_ERROR_COUNT(4, result); + CHECK_EQ(toString(result.errors[0]), R"(Type '(string, number) -> string' could not be converted into '(string) -> string' +caused by: + Argument count mismatch. Function expects 2 arguments, but only 1 is specified)"); + CHECK_EQ(toString(result.errors[1]), "Cannot add property 'z' to table 'X & Y'"); + CHECK_EQ(toString(result.errors[2]), "Type 'number' could not be converted into 'string'"); + CHECK_EQ(toString(result.errors[3]), "Cannot add property 'w' to table 'X & Y'"); +} + +TEST_CASE_FIXTURE(Fixture, "table_write_sealed_indirect") +{ + ScopedFastFlag statFunctionSimplify{"LuauStatFunctionSimplify4", true}; + + // After normalization, previous 'table_intersection_write_sealed_indirect' is identical to this one + CheckResult result = check(R"( + type XY = { x: (number) -> number, y: (string) -> string } + + local xy : XY = { + x = function(a: number) return -a end, + y = function(a: string) return a .. "b" end + } + function xy.z(a:number) return a * 10 end + function xy:y(a:number) return a * 10 end + function xy:w(a:number) return a * 10 end + )"); + + LUAU_REQUIRE_ERROR_COUNT(4, result); + CHECK_EQ(toString(result.errors[0]), R"(Type '(string, number) -> string' could not be converted into '(string) -> string' +caused by: + Argument count mismatch. Function expects 2 arguments, but only 1 is specified)"); + CHECK_EQ(toString(result.errors[1]), "Cannot add property 'z' to table 'XY'"); + CHECK_EQ(toString(result.errors[2]), "Type 'number' could not be converted into 'string'"); + CHECK_EQ(toString(result.errors[3]), "Cannot add property 'w' to table 'XY'"); } TEST_CASE_FIXTURE(Fixture, "table_intersection_setmetatable") diff --git a/tests/TypeInfer.primitives.test.cpp b/tests/TypeInfer.primitives.test.cpp index 44b7b0d0..3ddf9813 100644 --- a/tests/TypeInfer.primitives.test.cpp +++ b/tests/TypeInfer.primitives.test.cpp @@ -95,6 +95,8 @@ end )"); LUAU_REQUIRE_ERROR_COUNT(2, result); + CHECK_EQ(toString(result.errors[0]), "Cannot add method to non-table type 'number'"); + CHECK_EQ(toString(result.errors[1]), "Type 'number' could not be converted into 'string'"); } TEST_SUITE_END(); diff --git a/tests/TypeInfer.tables.test.cpp b/tests/TypeInfer.tables.test.cpp index 0cc12d19..0484351d 100644 --- a/tests/TypeInfer.tables.test.cpp +++ b/tests/TypeInfer.tables.test.cpp @@ -2922,4 +2922,19 @@ TEST_CASE_FIXTURE(Fixture, "inferred_properties_of_a_table_should_start_with_the LUAU_REQUIRE_NO_ERRORS(result); } +TEST_CASE_FIXTURE(Fixture, "mixed_tables_with_implicit_numbered_keys") +{ + ScopedFastFlag sff{"LuauCheckImplicitNumbericKeys", true}; + + CheckResult result = check(R"( + local t: { [string]: number } = { 5, 6, 7 } + )"); + + LUAU_REQUIRE_ERROR_COUNT(3, result); + + CHECK_EQ("Type 'number' could not be converted into 'string'", toString(result.errors[0])); + CHECK_EQ("Type 'number' could not be converted into 'string'", toString(result.errors[1])); + CHECK_EQ("Type 'number' could not be converted into 'string'", toString(result.errors[2])); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.tryUnify.test.cpp b/tests/TypeInfer.tryUnify.test.cpp index d8de2594..c21e1625 100644 --- a/tests/TypeInfer.tryUnify.test.cpp +++ b/tests/TypeInfer.tryUnify.test.cpp @@ -242,4 +242,30 @@ TEST_CASE_FIXTURE(TryUnifyFixture, "cli_50320_follow_in_any_unification") state.tryUnify(&func, typeChecker.anyType); } +TEST_CASE_FIXTURE(TryUnifyFixture, "txnlog_preserves_type_owner") +{ + ScopedFastFlag luauTxnLogPreserveOwner{"LuauTxnLogPreserveOwner", true}; + + TypeId a = arena.addType(TypeVar{FreeTypeVar{TypeLevel{}}}); + TypeId b = typeChecker.numberType; + + state.tryUnify(a, b); + state.log.commit(); + + CHECK_EQ(a->owningArena, &arena); +} + +TEST_CASE_FIXTURE(TryUnifyFixture, "txnlog_preserves_pack_owner") +{ + ScopedFastFlag luauTxnLogPreserveOwner{"LuauTxnLogPreserveOwner", true}; + + TypePackId a = arena.addTypePack(TypePackVar{FreeTypePack{TypeLevel{}}}); + TypePackId b = typeChecker.anyTypePack; + + state.tryUnify(a, b); + state.log.commit(); + + CHECK_EQ(a->owningArena, &arena); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.unionTypes.test.cpp b/tests/TypeInfer.unionTypes.test.cpp index 68b7c4fb..ff207a18 100644 --- a/tests/TypeInfer.unionTypes.test.cpp +++ b/tests/TypeInfer.unionTypes.test.cpp @@ -513,4 +513,29 @@ TEST_CASE_FIXTURE(Fixture, "dont_allow_cyclic_unions_to_be_inferred") LUAU_REQUIRE_NO_ERRORS(result); } +TEST_CASE_FIXTURE(Fixture, "table_union_write_indirect") +{ + ScopedFastFlag statFunctionSimplify{"LuauStatFunctionSimplify4", true}; + + CheckResult result = check(R"( + type A = { x: number, y: (number) -> string } | { z: number, y: (number) -> string } + + local a:A = nil + + function a.y(x) + return tostring(x * 2) + end + + function a.y(x: string): number + return tonumber(x) or 0 + end + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + // NOTE: union normalization will improve this message + CHECK_EQ(toString(result.errors[0]), + R"(Type '(string) -> number' could not be converted into '((number) -> string) | ((number) -> string)'; none of the union options are compatible)"); +} + + TEST_SUITE_END(); diff --git a/tools/lldb_formatters.py b/tools/lldb_formatters.py index 40f8d6be..b3d2b4f5 100644 --- a/tools/lldb_formatters.py +++ b/tools/lldb_formatters.py @@ -37,7 +37,7 @@ def getType(target, typeName): return ty def luau_variant_summary(valobj, internal_dict, options): - type_id = valobj.GetChildMemberWithName("typeid").GetValueAsUnsigned() + type_id = valobj.GetChildMemberWithName("typeId").GetValueAsUnsigned() storage = valobj.GetChildMemberWithName("storage") params = templateParams(valobj.GetType().GetCanonicalType().GetName()) stored_type = params[type_id] @@ -89,7 +89,7 @@ class LuauVariantSyntheticChildrenProvider: return None def update(self): - self.type_index = self.valobj.GetChildMemberWithName("typeid").GetValueAsSigned() + self.type_index = self.valobj.GetChildMemberWithName("typeId").GetValueAsSigned() self.type_params = templateParams(self.valobj.GetType().GetCanonicalType().GetName()) if len(self.type_params) > self.type_index: From de1381e3f131f2c307f68033ea494d1c35f2c3ca Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Thu, 7 Apr 2022 14:29:01 -0700 Subject: [PATCH 219/399] Sync to upstream/release/522 (#450) --- Analysis/include/Luau/Clone.h | 25 ++ Analysis/include/Luau/Frontend.h | 23 +- Analysis/include/Luau/Module.h | 29 +- Analysis/include/Luau/TypeInfer.h | 10 + Analysis/include/Luau/TypeVar.h | 2 + Analysis/include/Luau/Variant.h | 36 +- Analysis/src/Autocomplete.cpp | 117 +++++- Analysis/src/Clone.cpp | 371 ++++++++++++++++++ Analysis/src/Error.cpp | 1 + Analysis/src/Frontend.cpp | 144 +++++-- Analysis/src/IostreamHelpers.cpp | 419 +++++++++------------ Analysis/src/Module.cpp | 359 +----------------- Analysis/src/TxnLog.cpp | 31 +- Analysis/src/TypeInfer.cpp | 156 +++++--- Analysis/src/TypeVar.cpp | 4 + Ast/include/Luau/TimeTrace.h | 11 +- Ast/src/Parser.cpp | 11 + Ast/src/TimeTrace.cpp | 19 +- Compiler/src/ConstantFolding.cpp | 3 +- Sources.cmake | 2 + VM/src/lfunc.h | 2 +- VM/src/ltablib.cpp | 2 +- tests/Autocomplete.test.cpp | 139 ++++++- tests/Compiler.test.cpp | 33 ++ tests/Fixture.cpp | 5 +- tests/Fixture.h | 2 +- tests/Frontend.test.cpp | 64 ++++ tests/Module.test.cpp | 1 + tests/Parser.test.cpp | 21 ++ tests/TypeInfer.functions.test.cpp | 39 +- tests/TypeInfer.intersectionTypes.test.cpp | 39 +- tests/TypeInfer.primitives.test.cpp | 2 + tests/TypeInfer.tables.test.cpp | 15 + tests/TypeInfer.tryUnify.test.cpp | 26 ++ tests/TypeInfer.unionTypes.test.cpp | 25 ++ tools/lldb_formatters.py | 4 +- 36 files changed, 1407 insertions(+), 785 deletions(-) create mode 100644 Analysis/include/Luau/Clone.h create mode 100644 Analysis/src/Clone.cpp diff --git a/Analysis/include/Luau/Clone.h b/Analysis/include/Luau/Clone.h new file mode 100644 index 00000000..917ef801 --- /dev/null +++ b/Analysis/include/Luau/Clone.h @@ -0,0 +1,25 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +#pragma once + +#include "Luau/TypeVar.h" + +#include + +namespace Luau +{ + +// Only exposed so they can be unit tested. +using SeenTypes = std::unordered_map; +using SeenTypePacks = std::unordered_map; + +struct CloneState +{ + int recursionCount = 0; + bool encounteredFreeType = false; +}; + +TypePackId clone(TypePackId tp, TypeArena& dest, SeenTypes& seenTypes, SeenTypePacks& seenTypePacks, CloneState& cloneState); +TypeId clone(TypeId tp, TypeArena& dest, SeenTypes& seenTypes, SeenTypePacks& seenTypePacks, CloneState& cloneState); +TypeFun clone(const TypeFun& typeFun, TypeArena& dest, SeenTypes& seenTypes, SeenTypePacks& seenTypePacks, CloneState& cloneState); + +} // namespace Luau diff --git a/Analysis/include/Luau/Frontend.h b/Analysis/include/Luau/Frontend.h index 0bf8f362..2266f548 100644 --- a/Analysis/include/Luau/Frontend.h +++ b/Analysis/include/Luau/Frontend.h @@ -12,6 +12,8 @@ #include #include +LUAU_FASTFLAG(LuauSeparateTypechecks) + namespace Luau { @@ -55,10 +57,19 @@ std::optional pathExprToModuleName(const ModuleName& currentModuleNa struct SourceNode { + bool isDirty(bool forAutocomplete) const + { + if (FFlag::LuauSeparateTypechecks) + return forAutocomplete ? dirtyAutocomplete : dirty; + else + return dirty; + } + ModuleName name; std::unordered_set requires; std::vector> requireLocations; bool dirty = true; + bool dirtyAutocomplete = true; }; struct FrontendOptions @@ -71,12 +82,16 @@ struct FrontendOptions // When true, we run typechecking twice, once in the regular mode, and once in strict mode // in order to get more precise type information (e.g. for autocomplete). - bool typecheckTwice = false; + bool typecheckTwice_DEPRECATED = false; + + // Run typechecking only in mode required for autocomplete (strict mode in order to get more precise type information) + bool forAutocomplete = false; }; struct CheckResult { std::vector errors; + std::vector timeoutHits; }; struct FrontendModuleResolver : ModuleResolver @@ -123,7 +138,7 @@ struct Frontend CheckResult check(const SourceModule& module); // OLD. TODO KILL LintResult lint(const SourceModule& module, std::optional enabledLintWarnings = {}); - bool isDirty(const ModuleName& name) const; + bool isDirty(const ModuleName& name, bool forAutocomplete = false) const; void markDirty(const ModuleName& name, std::vector* markedDirty = nullptr); /** Borrow a pointer into the SourceModule cache. @@ -147,10 +162,10 @@ struct Frontend void applyBuiltinDefinitionToEnvironment(const std::string& environmentName, const std::string& definitionName); private: - std::pair getSourceNode(CheckResult& checkResult, const ModuleName& name); + std::pair getSourceNode(CheckResult& checkResult, const ModuleName& name, bool forAutocomplete); SourceModule parse(const ModuleName& name, std::string_view src, const ParseOptions& parseOptions); - bool parseGraph(std::vector& buildQueue, CheckResult& checkResult, const ModuleName& root); + bool parseGraph(std::vector& buildQueue, CheckResult& checkResult, const ModuleName& root, bool forAutocomplete); static LintResult classifyLints(const std::vector& warnings, const Config& config); diff --git a/Analysis/include/Luau/Module.h b/Analysis/include/Luau/Module.h index 6c689b7c..9a32f614 100644 --- a/Analysis/include/Luau/Module.h +++ b/Analysis/include/Luau/Module.h @@ -29,8 +29,8 @@ struct SourceModule std::optional environmentName; bool cyclic = false; - std::unique_ptr allocator; - std::unique_ptr names; + std::shared_ptr allocator; + std::shared_ptr names; std::vector parseErrors; AstStatBlock* root = nullptr; @@ -48,6 +48,12 @@ struct SourceModule bool isWithinComment(const SourceModule& sourceModule, Position pos); +struct RequireCycle +{ + Location location; + std::vector path; // one of the paths for a require() to go all the way back to the originating module +}; + struct TypeArena { TypedAllocator typeVars; @@ -77,20 +83,6 @@ struct TypeArena void freeze(TypeArena& arena); void unfreeze(TypeArena& arena); -// Only exposed so they can be unit tested. -using SeenTypes = std::unordered_map; -using SeenTypePacks = std::unordered_map; - -struct CloneState -{ - int recursionCount = 0; - bool encounteredFreeType = false; -}; - -TypePackId clone(TypePackId tp, TypeArena& dest, SeenTypes& seenTypes, SeenTypePacks& seenTypePacks, CloneState& cloneState); -TypeId clone(TypeId tp, TypeArena& dest, SeenTypes& seenTypes, SeenTypePacks& seenTypePacks, CloneState& cloneState); -TypeFun clone(const TypeFun& typeFun, TypeArena& dest, SeenTypes& seenTypes, SeenTypePacks& seenTypePacks, CloneState& cloneState); - struct Module { ~Module(); @@ -98,6 +90,10 @@ struct Module TypeArena interfaceTypes; TypeArena internalTypes; + // Scopes and AST types refer to parse data, so we need to keep that alive + std::shared_ptr allocator; + std::shared_ptr names; + std::vector> scopes; // never empty DenseHashMap astTypes{nullptr}; @@ -109,6 +105,7 @@ struct Module ErrorVec errors; Mode mode; SourceCode::Type type; + bool timeout = false; ScopePtr getModuleScope() const; diff --git a/Analysis/include/Luau/TypeInfer.h b/Analysis/include/Luau/TypeInfer.h index 839043cc..215da67f 100644 --- a/Analysis/include/Luau/TypeInfer.h +++ b/Analysis/include/Luau/TypeInfer.h @@ -124,6 +124,12 @@ struct HashBoolNamePair size_t operator()(const std::pair& pair) const; }; +class TimeLimitError : public std::exception +{ +public: + virtual const char* what() const throw(); +}; + // All TypeVars are retained via Environment::typeVars. All TypeIds // within a program are borrowed pointers into this set. struct TypeChecker @@ -413,6 +419,10 @@ public: UnifierSharedState unifierState; + std::vector requireCycles; + + std::optional finishTime; + public: const TypeId nilType; const TypeId numberType; diff --git a/Analysis/include/Luau/TypeVar.h b/Analysis/include/Luau/TypeVar.h index b8c4b362..f61e4044 100644 --- a/Analysis/include/Luau/TypeVar.h +++ b/Analysis/include/Luau/TypeVar.h @@ -513,6 +513,8 @@ struct SingletonTypes const TypeId stringType; const TypeId booleanType; const TypeId threadType; + const TypeId trueType; + const TypeId falseType; const TypeId anyType; const TypeId optionalNumberType; diff --git a/Analysis/include/Luau/Variant.h b/Analysis/include/Luau/Variant.h index 63d5a65c..5efe89ed 100644 --- a/Analysis/include/Luau/Variant.h +++ b/Analysis/include/Luau/Variant.h @@ -2,45 +2,14 @@ #pragma once #include "Luau/Common.h" - -#ifndef LUAU_USE_STD_VARIANT -#define LUAU_USE_STD_VARIANT 0 -#endif - -#if LUAU_USE_STD_VARIANT -#include -#else #include #include #include #include -#endif namespace Luau { -#if LUAU_USE_STD_VARIANT -template -using Variant = std::variant; - -template -auto visit(Visitor&& vis, Variant&& var) -{ - // This change resolves the ABI issues with std::variant on libc++; std::visit normally throws bad_variant_access - // but it requires an update to libc++.dylib which ships with macOS 10.14. To work around this, we assert on valueless - // variants since we will never generate them and call into a libc++ function that doesn't throw. - LUAU_ASSERT(!var.valueless_by_exception()); - -#ifdef __APPLE__ - // See https://stackoverflow.com/a/53868971/503215 - return std::__variant_detail::__visitation::__variant::__visit_value(vis, var); -#else - return std::visit(vis, var); -#endif -} - -using std::get_if; -#else template class Variant { @@ -248,6 +217,8 @@ static void fnVisitV(Visitor& vis, std::conditional_t, const template auto visit(Visitor&& vis, const Variant& var) { + static_assert(std::conjunction_v...>, "visitor must accept every alternative as an argument"); + using Result = std::invoke_result_t::first_alternative>; static_assert(std::conjunction_v>...>, "visitor result type must be consistent between alternatives"); @@ -273,6 +244,8 @@ auto visit(Visitor&& vis, const Variant& var) template auto visit(Visitor&& vis, Variant& var) { + static_assert(std::conjunction_v...>, "visitor must accept every alternative as an argument"); + using Result = std::invoke_result_t::first_alternative&>; static_assert(std::conjunction_v>...>, "visitor result type must be consistent between alternatives"); @@ -294,7 +267,6 @@ auto visit(Visitor&& vis, Variant& var) return res; } } -#endif template inline constexpr bool always_false_v = false; diff --git a/Analysis/src/Autocomplete.cpp b/Analysis/src/Autocomplete.cpp index 492edf25..b7201ab3 100644 --- a/Analysis/src/Autocomplete.cpp +++ b/Analysis/src/Autocomplete.cpp @@ -14,6 +14,7 @@ #include LUAU_FASTFLAGVARIABLE(LuauIfElseExprFixCompletionIssue, false); +LUAU_FASTFLAGVARIABLE(LuauAutocompleteSingletonTypes, false); LUAU_FASTFLAG(LuauSelfCallAutocompleteFix) static const std::unordered_set kStatementStartingKeywords = { @@ -625,6 +626,31 @@ AutocompleteEntryMap autocompleteModuleTypes(const Module& module, Position posi return result; } +static void autocompleteStringSingleton(TypeId ty, bool addQuotes, AutocompleteEntryMap& result) +{ + auto formatKey = [addQuotes](const std::string& key) { + if (addQuotes) + return "\"" + escape(key) + "\""; + + return escape(key); + }; + + ty = follow(ty); + + if (auto ss = get(get(ty))) + { + result[formatKey(ss->value)] = AutocompleteEntry{AutocompleteEntryKind::String, ty, false, false, TypeCorrectKind::Correct}; + } + else if (auto uty = get(ty)) + { + for (auto el : uty) + { + if (auto ss = get(get(el))) + result[formatKey(ss->value)] = AutocompleteEntry{AutocompleteEntryKind::String, ty, false, false, TypeCorrectKind::Correct}; + } + } +}; + static bool canSuggestInferredType(ScopePtr scope, TypeId ty) { ty = follow(ty); @@ -1309,17 +1335,38 @@ static void autocompleteExpression(const SourceModule& sourceModule, const Modul scope = scope->parent; } - TypeCorrectKind correctForNil = checkTypeCorrectKind(module, typeArena, node, position, typeChecker.nilType); - TypeCorrectKind correctForBoolean = checkTypeCorrectKind(module, typeArena, node, position, typeChecker.booleanType); - TypeCorrectKind correctForFunction = - functionIsExpectedAt(module, node, position).value_or(false) ? TypeCorrectKind::Correct : TypeCorrectKind::None; + if (FFlag::LuauAutocompleteSingletonTypes) + { + TypeCorrectKind correctForNil = checkTypeCorrectKind(module, typeArena, node, position, typeChecker.nilType); + TypeCorrectKind correctForTrue = checkTypeCorrectKind(module, typeArena, node, position, getSingletonTypes().trueType); + TypeCorrectKind correctForFalse = checkTypeCorrectKind(module, typeArena, node, position, getSingletonTypes().falseType); + TypeCorrectKind correctForFunction = + functionIsExpectedAt(module, node, position).value_or(false) ? TypeCorrectKind::Correct : TypeCorrectKind::None; - result["if"] = {AutocompleteEntryKind::Keyword, std::nullopt, false, false}; - result["true"] = {AutocompleteEntryKind::Keyword, typeChecker.booleanType, false, false, correctForBoolean}; - result["false"] = {AutocompleteEntryKind::Keyword, typeChecker.booleanType, false, false, correctForBoolean}; - result["nil"] = {AutocompleteEntryKind::Keyword, typeChecker.nilType, false, false, correctForNil}; - result["not"] = {AutocompleteEntryKind::Keyword}; - result["function"] = {AutocompleteEntryKind::Keyword, std::nullopt, false, false, correctForFunction}; + result["if"] = {AutocompleteEntryKind::Keyword, std::nullopt, false, false}; + result["true"] = {AutocompleteEntryKind::Keyword, typeChecker.booleanType, false, false, correctForTrue}; + result["false"] = {AutocompleteEntryKind::Keyword, typeChecker.booleanType, false, false, correctForFalse}; + result["nil"] = {AutocompleteEntryKind::Keyword, typeChecker.nilType, false, false, correctForNil}; + result["not"] = {AutocompleteEntryKind::Keyword}; + result["function"] = {AutocompleteEntryKind::Keyword, std::nullopt, false, false, correctForFunction}; + + if (auto ty = findExpectedTypeAt(module, node, position)) + autocompleteStringSingleton(*ty, true, result); + } + else + { + TypeCorrectKind correctForNil = checkTypeCorrectKind(module, typeArena, node, position, typeChecker.nilType); + TypeCorrectKind correctForBoolean = checkTypeCorrectKind(module, typeArena, node, position, typeChecker.booleanType); + TypeCorrectKind correctForFunction = + functionIsExpectedAt(module, node, position).value_or(false) ? TypeCorrectKind::Correct : TypeCorrectKind::None; + + result["if"] = {AutocompleteEntryKind::Keyword, std::nullopt, false, false}; + result["true"] = {AutocompleteEntryKind::Keyword, typeChecker.booleanType, false, false, correctForBoolean}; + result["false"] = {AutocompleteEntryKind::Keyword, typeChecker.booleanType, false, false, correctForBoolean}; + result["nil"] = {AutocompleteEntryKind::Keyword, typeChecker.nilType, false, false, correctForNil}; + result["not"] = {AutocompleteEntryKind::Keyword}; + result["function"] = {AutocompleteEntryKind::Keyword, std::nullopt, false, false, correctForFunction}; + } } } @@ -1625,17 +1672,33 @@ static AutocompleteResult autocomplete(const SourceModule& sourceModule, const M } else if (node->is()) { + AutocompleteEntryMap result; + + if (FFlag::LuauAutocompleteSingletonTypes) + { + if (auto it = module->astExpectedTypes.find(node->asExpr())) + autocompleteStringSingleton(*it, false, result); + } + if (finder.ancestry.size() >= 2) { if (auto idxExpr = finder.ancestry.at(finder.ancestry.size() - 2)->as()) { if (auto it = module->astTypes.find(idxExpr->expr)) + autocompleteProps(*module, typeArena, follow(*it), PropIndexType::Point, finder.ancestry, result); + } + else if (auto binExpr = finder.ancestry.at(finder.ancestry.size() - 2)->as(); + binExpr && FFlag::LuauAutocompleteSingletonTypes) + { + if (binExpr->op == AstExprBinary::CompareEq || binExpr->op == AstExprBinary::CompareNe) { - return {autocompleteProps(*module, typeArena, follow(*it), PropIndexType::Point, finder.ancestry), finder.ancestry}; + if (auto it = module->astTypes.find(node == binExpr->left ? binExpr->right : binExpr->left)) + autocompleteStringSingleton(*it, false, result); } } } - return {}; + + return {result, finder.ancestry}; } if (node->is()) @@ -1653,18 +1716,31 @@ static AutocompleteResult autocomplete(const SourceModule& sourceModule, const M AutocompleteResult autocomplete(Frontend& frontend, const ModuleName& moduleName, Position position, StringCompletionCallback callback) { - // FIXME: We can improve performance here by parsing without checking. - // The old type graph is probably fine. (famous last words!) - // FIXME: We don't need to typecheck for script analysis here, just for autocomplete. - frontend.check(moduleName); + if (FFlag::LuauSeparateTypechecks) + { + // FIXME: We can improve performance here by parsing without checking. + // The old type graph is probably fine. (famous last words!) + FrontendOptions opts; + opts.forAutocomplete = true; + frontend.check(moduleName, opts); + } + else + { + // FIXME: We can improve performance here by parsing without checking. + // The old type graph is probably fine. (famous last words!) + // FIXME: We don't need to typecheck for script analysis here, just for autocomplete. + frontend.check(moduleName); + } const SourceModule* sourceModule = frontend.getSourceModule(moduleName); if (!sourceModule) return {}; - TypeChecker& typeChecker = (frontend.options.typecheckTwice ? frontend.typeCheckerForAutocomplete : frontend.typeChecker); - ModulePtr module = (frontend.options.typecheckTwice ? frontend.moduleResolverForAutocomplete.getModule(moduleName) - : frontend.moduleResolver.getModule(moduleName)); + TypeChecker& typeChecker = + (frontend.options.typecheckTwice_DEPRECATED || FFlag::LuauSeparateTypechecks ? frontend.typeCheckerForAutocomplete : frontend.typeChecker); + ModulePtr module = + (frontend.options.typecheckTwice_DEPRECATED || FFlag::LuauSeparateTypechecks ? frontend.moduleResolverForAutocomplete.getModule(moduleName) + : frontend.moduleResolver.getModule(moduleName)); if (!module) return {}; @@ -1692,7 +1768,8 @@ OwningAutocompleteResult autocompleteSource(Frontend& frontend, std::string_view sourceModule->mode = Mode::Strict; sourceModule->commentLocations = std::move(result.commentLocations); - TypeChecker& typeChecker = (frontend.options.typecheckTwice ? frontend.typeCheckerForAutocomplete : frontend.typeChecker); + TypeChecker& typeChecker = + (frontend.options.typecheckTwice_DEPRECATED || FFlag::LuauSeparateTypechecks ? frontend.typeCheckerForAutocomplete : frontend.typeChecker); ModulePtr module = typeChecker.check(*sourceModule, Mode::Strict); diff --git a/Analysis/src/Clone.cpp b/Analysis/src/Clone.cpp new file mode 100644 index 00000000..ac9705a7 --- /dev/null +++ b/Analysis/src/Clone.cpp @@ -0,0 +1,371 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details + +#include "Luau/Clone.h" +#include "Luau/Module.h" +#include "Luau/RecursionCounter.h" +#include "Luau/TypePack.h" +#include "Luau/Unifiable.h" + +LUAU_FASTINTVARIABLE(LuauTypeCloneRecursionLimit, 300) + +namespace Luau +{ + +namespace +{ + +struct TypePackCloner; + +/* + * Both TypeCloner and TypePackCloner work by depositing the requested type variable into the appropriate 'seen' set. + * They do not return anything because their sole consumer (the deepClone function) already has a pointer into this storage. + */ + +struct TypeCloner +{ + TypeCloner(TypeArena& dest, TypeId typeId, SeenTypes& seenTypes, SeenTypePacks& seenTypePacks, CloneState& cloneState) + : dest(dest) + , typeId(typeId) + , seenTypes(seenTypes) + , seenTypePacks(seenTypePacks) + , cloneState(cloneState) + { + } + + TypeArena& dest; + TypeId typeId; + SeenTypes& seenTypes; + SeenTypePacks& seenTypePacks; + CloneState& cloneState; + + template + void defaultClone(const T& t); + + void operator()(const Unifiable::Free& t); + void operator()(const Unifiable::Generic& t); + void operator()(const Unifiable::Bound& t); + void operator()(const Unifiable::Error& t); + void operator()(const PrimitiveTypeVar& t); + void operator()(const SingletonTypeVar& t); + void operator()(const FunctionTypeVar& t); + void operator()(const TableTypeVar& t); + void operator()(const MetatableTypeVar& t); + void operator()(const ClassTypeVar& t); + void operator()(const AnyTypeVar& t); + void operator()(const UnionTypeVar& t); + void operator()(const IntersectionTypeVar& t); + void operator()(const LazyTypeVar& t); +}; + +struct TypePackCloner +{ + TypeArena& dest; + TypePackId typePackId; + SeenTypes& seenTypes; + SeenTypePacks& seenTypePacks; + CloneState& cloneState; + + TypePackCloner(TypeArena& dest, TypePackId typePackId, SeenTypes& seenTypes, SeenTypePacks& seenTypePacks, CloneState& cloneState) + : dest(dest) + , typePackId(typePackId) + , seenTypes(seenTypes) + , seenTypePacks(seenTypePacks) + , cloneState(cloneState) + { + } + + template + void defaultClone(const T& t) + { + TypePackId cloned = dest.addTypePack(TypePackVar{t}); + seenTypePacks[typePackId] = cloned; + } + + void operator()(const Unifiable::Free& t) + { + cloneState.encounteredFreeType = true; + + TypePackId err = getSingletonTypes().errorRecoveryTypePack(getSingletonTypes().anyTypePack); + TypePackId cloned = dest.addTypePack(*err); + seenTypePacks[typePackId] = cloned; + } + + void operator()(const Unifiable::Generic& t) + { + defaultClone(t); + } + void operator()(const Unifiable::Error& t) + { + defaultClone(t); + } + + // While we are a-cloning, we can flatten out bound TypeVars and make things a bit tighter. + // We just need to be sure that we rewrite pointers both to the binder and the bindee to the same pointer. + void operator()(const Unifiable::Bound& t) + { + TypePackId cloned = clone(t.boundTo, dest, seenTypes, seenTypePacks, cloneState); + seenTypePacks[typePackId] = cloned; + } + + void operator()(const VariadicTypePack& t) + { + TypePackId cloned = dest.addTypePack(TypePackVar{VariadicTypePack{clone(t.ty, dest, seenTypes, seenTypePacks, cloneState)}}); + seenTypePacks[typePackId] = cloned; + } + + void operator()(const TypePack& t) + { + TypePackId cloned = dest.addTypePack(TypePack{}); + TypePack* destTp = getMutable(cloned); + LUAU_ASSERT(destTp != nullptr); + seenTypePacks[typePackId] = cloned; + + for (TypeId ty : t.head) + destTp->head.push_back(clone(ty, dest, seenTypes, seenTypePacks, cloneState)); + + if (t.tail) + destTp->tail = clone(*t.tail, dest, seenTypes, seenTypePacks, cloneState); + } +}; + +template +void TypeCloner::defaultClone(const T& t) +{ + TypeId cloned = dest.addType(t); + seenTypes[typeId] = cloned; +} + +void TypeCloner::operator()(const Unifiable::Free& t) +{ + cloneState.encounteredFreeType = true; + TypeId err = getSingletonTypes().errorRecoveryType(getSingletonTypes().anyType); + TypeId cloned = dest.addType(*err); + seenTypes[typeId] = cloned; +} + +void TypeCloner::operator()(const Unifiable::Generic& t) +{ + defaultClone(t); +} + +void TypeCloner::operator()(const Unifiable::Bound& t) +{ + TypeId boundTo = clone(t.boundTo, dest, seenTypes, seenTypePacks, cloneState); + seenTypes[typeId] = boundTo; +} + +void TypeCloner::operator()(const Unifiable::Error& t) +{ + defaultClone(t); +} + +void TypeCloner::operator()(const PrimitiveTypeVar& t) +{ + defaultClone(t); +} + +void TypeCloner::operator()(const SingletonTypeVar& t) +{ + defaultClone(t); +} + +void TypeCloner::operator()(const FunctionTypeVar& t) +{ + TypeId result = dest.addType(FunctionTypeVar{TypeLevel{0, 0}, {}, {}, nullptr, nullptr, t.definition, t.hasSelf}); + FunctionTypeVar* ftv = getMutable(result); + LUAU_ASSERT(ftv != nullptr); + + seenTypes[typeId] = result; + + for (TypeId generic : t.generics) + ftv->generics.push_back(clone(generic, dest, seenTypes, seenTypePacks, cloneState)); + + for (TypePackId genericPack : t.genericPacks) + ftv->genericPacks.push_back(clone(genericPack, dest, seenTypes, seenTypePacks, cloneState)); + + ftv->tags = t.tags; + ftv->argTypes = clone(t.argTypes, dest, seenTypes, seenTypePacks, cloneState); + ftv->argNames = t.argNames; + ftv->retType = clone(t.retType, dest, seenTypes, seenTypePacks, cloneState); +} + +void TypeCloner::operator()(const TableTypeVar& t) +{ + // If table is now bound to another one, we ignore the content of the original + if (t.boundTo) + { + TypeId boundTo = clone(*t.boundTo, dest, seenTypes, seenTypePacks, cloneState); + seenTypes[typeId] = boundTo; + return; + } + + TypeId result = dest.addType(TableTypeVar{}); + TableTypeVar* ttv = getMutable(result); + LUAU_ASSERT(ttv != nullptr); + + *ttv = t; + + seenTypes[typeId] = result; + + ttv->level = TypeLevel{0, 0}; + + for (const auto& [name, prop] : t.props) + ttv->props[name] = {clone(prop.type, dest, seenTypes, seenTypePacks, cloneState), prop.deprecated, {}, prop.location, prop.tags}; + + if (t.indexer) + ttv->indexer = TableIndexer{clone(t.indexer->indexType, dest, seenTypes, seenTypePacks, cloneState), + clone(t.indexer->indexResultType, dest, seenTypes, seenTypePacks, cloneState)}; + + for (TypeId& arg : ttv->instantiatedTypeParams) + arg = clone(arg, dest, seenTypes, seenTypePacks, cloneState); + + for (TypePackId& arg : ttv->instantiatedTypePackParams) + arg = clone(arg, dest, seenTypes, seenTypePacks, cloneState); + + if (ttv->state == TableState::Free) + { + cloneState.encounteredFreeType = true; + + ttv->state = TableState::Sealed; + } + + ttv->definitionModuleName = t.definitionModuleName; + ttv->methodDefinitionLocations = t.methodDefinitionLocations; + ttv->tags = t.tags; +} + +void TypeCloner::operator()(const MetatableTypeVar& t) +{ + TypeId result = dest.addType(MetatableTypeVar{}); + MetatableTypeVar* mtv = getMutable(result); + seenTypes[typeId] = result; + + mtv->table = clone(t.table, dest, seenTypes, seenTypePacks, cloneState); + mtv->metatable = clone(t.metatable, dest, seenTypes, seenTypePacks, cloneState); +} + +void TypeCloner::operator()(const ClassTypeVar& t) +{ + TypeId result = dest.addType(ClassTypeVar{t.name, {}, std::nullopt, std::nullopt, t.tags, t.userData}); + ClassTypeVar* ctv = getMutable(result); + + seenTypes[typeId] = result; + + for (const auto& [name, prop] : t.props) + ctv->props[name] = {clone(prop.type, dest, seenTypes, seenTypePacks, cloneState), prop.deprecated, {}, prop.location, prop.tags}; + + if (t.parent) + ctv->parent = clone(*t.parent, dest, seenTypes, seenTypePacks, cloneState); + + if (t.metatable) + ctv->metatable = clone(*t.metatable, dest, seenTypes, seenTypePacks, cloneState); +} + +void TypeCloner::operator()(const AnyTypeVar& t) +{ + defaultClone(t); +} + +void TypeCloner::operator()(const UnionTypeVar& t) +{ + std::vector options; + options.reserve(t.options.size()); + + for (TypeId ty : t.options) + options.push_back(clone(ty, dest, seenTypes, seenTypePacks, cloneState)); + + TypeId result = dest.addType(UnionTypeVar{std::move(options)}); + seenTypes[typeId] = result; +} + +void TypeCloner::operator()(const IntersectionTypeVar& t) +{ + TypeId result = dest.addType(IntersectionTypeVar{}); + seenTypes[typeId] = result; + + IntersectionTypeVar* option = getMutable(result); + LUAU_ASSERT(option != nullptr); + + for (TypeId ty : t.parts) + option->parts.push_back(clone(ty, dest, seenTypes, seenTypePacks, cloneState)); +} + +void TypeCloner::operator()(const LazyTypeVar& t) +{ + defaultClone(t); +} + +} // anonymous namespace + +TypePackId clone(TypePackId tp, TypeArena& dest, SeenTypes& seenTypes, SeenTypePacks& seenTypePacks, CloneState& cloneState) +{ + if (tp->persistent) + return tp; + + RecursionLimiter _ra(&cloneState.recursionCount, FInt::LuauTypeCloneRecursionLimit); + + TypePackId& res = seenTypePacks[tp]; + + if (res == nullptr) + { + TypePackCloner cloner{dest, tp, seenTypes, seenTypePacks, cloneState}; + Luau::visit(cloner, tp->ty); // Mutates the storage that 'res' points into. + } + + return res; +} + +TypeId clone(TypeId typeId, TypeArena& dest, SeenTypes& seenTypes, SeenTypePacks& seenTypePacks, CloneState& cloneState) +{ + if (typeId->persistent) + return typeId; + + RecursionLimiter _ra(&cloneState.recursionCount, FInt::LuauTypeCloneRecursionLimit); + + TypeId& res = seenTypes[typeId]; + + if (res == nullptr) + { + TypeCloner cloner{dest, typeId, seenTypes, seenTypePacks, cloneState}; + Luau::visit(cloner, typeId->ty); // Mutates the storage that 'res' points into. + + // Persistent types are not being cloned and we get the original type back which might be read-only + if (!res->persistent) + asMutable(res)->documentationSymbol = typeId->documentationSymbol; + } + + return res; +} + +TypeFun clone(const TypeFun& typeFun, TypeArena& dest, SeenTypes& seenTypes, SeenTypePacks& seenTypePacks, CloneState& cloneState) +{ + TypeFun result; + + for (auto param : typeFun.typeParams) + { + TypeId ty = clone(param.ty, dest, seenTypes, seenTypePacks, cloneState); + std::optional defaultValue; + + if (param.defaultValue) + defaultValue = clone(*param.defaultValue, dest, seenTypes, seenTypePacks, cloneState); + + result.typeParams.push_back({ty, defaultValue}); + } + + for (auto param : typeFun.typePackParams) + { + TypePackId tp = clone(param.tp, dest, seenTypes, seenTypePacks, cloneState); + std::optional defaultValue; + + if (param.defaultValue) + defaultValue = clone(*param.defaultValue, dest, seenTypes, seenTypePacks, cloneState); + + result.typePackParams.push_back({tp, defaultValue}); + } + + result.type = clone(typeFun.type, dest, seenTypes, seenTypePacks, cloneState); + + return result; +} + +} // namespace Luau diff --git a/Analysis/src/Error.cpp b/Analysis/src/Error.cpp index 210c0191..5eb2ea2a 100644 --- a/Analysis/src/Error.cpp +++ b/Analysis/src/Error.cpp @@ -1,6 +1,7 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #include "Luau/Error.h" +#include "Luau/Clone.h" #include "Luau/Module.h" #include "Luau/StringUtils.h" #include "Luau/ToString.h" diff --git a/Analysis/src/Frontend.cpp b/Analysis/src/Frontend.cpp index d8906f6e..000769fe 100644 --- a/Analysis/src/Frontend.cpp +++ b/Analysis/src/Frontend.cpp @@ -2,6 +2,7 @@ #include "Luau/Frontend.h" #include "Luau/Common.h" +#include "Luau/Clone.h" #include "Luau/Config.h" #include "Luau/FileResolver.h" #include "Luau/Parser.h" @@ -16,8 +17,11 @@ #include #include +LUAU_FASTFLAG(LuauCyclicModuleTypeSurface) LUAU_FASTFLAG(LuauInferInNoCheckMode) LUAU_FASTFLAGVARIABLE(LuauKnowsTheDataModel3, false) +LUAU_FASTFLAGVARIABLE(LuauSeparateTypechecks, false) +LUAU_FASTINTVARIABLE(LuauAutocompleteCheckTimeoutMs, 0) namespace Luau { @@ -234,12 +238,6 @@ ErrorVec accumulateErrors( return result; } -struct RequireCycle -{ - Location location; - std::vector path; // one of the paths for a require() to go all the way back to the originating module -}; - // Given a source node (start), find all requires that start a transitive dependency path that ends back at start // For each such path, record the full path and the location of the require in the starting module. // Note that this is O(V^2) for a fully connected graph and produces O(V) paths of length O(V) @@ -356,33 +354,55 @@ CheckResult Frontend::check(const ModuleName& name, std::optionalsecond.dirty) + if (it != sourceNodes.end() && !it->second.isDirty(frontendOptions.forAutocomplete)) { // No recheck required. - auto it2 = moduleResolver.modules.find(name); - if (it2 == moduleResolver.modules.end() || it2->second == nullptr) - throw std::runtime_error("Frontend::modules does not have data for " + name); + if (FFlag::LuauSeparateTypechecks) + { + if (frontendOptions.forAutocomplete) + { + auto it2 = moduleResolverForAutocomplete.modules.find(name); + if (it2 == moduleResolverForAutocomplete.modules.end() || it2->second == nullptr) + throw std::runtime_error("Frontend::modules does not have data for " + name); + } + else + { + auto it2 = moduleResolver.modules.find(name); + if (it2 == moduleResolver.modules.end() || it2->second == nullptr) + throw std::runtime_error("Frontend::modules does not have data for " + name); + } - return CheckResult{accumulateErrors(sourceNodes, moduleResolver.modules, name)}; + return CheckResult{accumulateErrors( + sourceNodes, frontendOptions.forAutocomplete ? moduleResolverForAutocomplete.modules : moduleResolver.modules, name)}; + } + else + { + auto it2 = moduleResolver.modules.find(name); + if (it2 == moduleResolver.modules.end() || it2->second == nullptr) + throw std::runtime_error("Frontend::modules does not have data for " + name); + + return CheckResult{accumulateErrors(sourceNodes, moduleResolver.modules, name)}; + } } std::vector buildQueue; - bool cycleDetected = parseGraph(buildQueue, checkResult, name); - - FrontendOptions frontendOptions = optionOverride.value_or(options); + bool cycleDetected = parseGraph(buildQueue, checkResult, name, frontendOptions.forAutocomplete); // Keep track of which AST nodes we've reported cycles in std::unordered_set reportedCycles; + double autocompleteTimeLimit = FInt::LuauAutocompleteCheckTimeoutMs / 1000.0; + for (const ModuleName& moduleName : buildQueue) { LUAU_ASSERT(sourceNodes.count(moduleName)); SourceNode& sourceNode = sourceNodes[moduleName]; - if (!sourceNode.dirty) + if (!sourceNode.isDirty(frontendOptions.forAutocomplete)) continue; LUAU_ASSERT(sourceModules.count(moduleName)); @@ -408,13 +428,44 @@ CheckResult Frontend::check(const ModuleName& name, std::optionaltimeout) + checkResult.timeoutHits.push_back(moduleName); + + stats.timeCheck += getTimestamp() - timestamp; + stats.filesStrict += 1; + + sourceNode.dirtyAutocomplete = false; + continue; + } + + if (FFlag::LuauCyclicModuleTypeSurface) + typeChecker.requireCycles = requireCycles; + ModulePtr module = typeChecker.check(sourceModule, mode, environmentScope); // If we're typechecking twice, we do so. // The second typecheck is always in strict mode with DM awareness // to provide better typen information for IDE features. - if (frontendOptions.typecheckTwice) + if (!FFlag::LuauSeparateTypechecks && frontendOptions.typecheckTwice_DEPRECATED) { + if (FFlag::LuauCyclicModuleTypeSurface) + typeCheckerForAutocomplete.requireCycles = requireCycles; + ModulePtr moduleForAutocomplete = typeCheckerForAutocomplete.check(sourceModule, Mode::Strict); moduleResolverForAutocomplete.modules[moduleName] = moduleForAutocomplete; } @@ -467,7 +518,7 @@ CheckResult Frontend::check(const ModuleName& name, std::optional& buildQueue, CheckResult& checkResult, const ModuleName& root) +bool Frontend::parseGraph(std::vector& buildQueue, CheckResult& checkResult, const ModuleName& root, bool forAutocomplete) { LUAU_TIMETRACE_SCOPE("Frontend::parseGraph", "Frontend"); LUAU_TIMETRACE_ARGUMENT("root", root.c_str()); @@ -486,7 +537,7 @@ bool Frontend::parseGraph(std::vector& buildQueue, CheckResult& chec bool cyclic = false; { - auto [sourceNode, _] = getSourceNode(checkResult, root); + auto [sourceNode, _] = getSourceNode(checkResult, root, forAutocomplete); if (sourceNode) stack.push_back(sourceNode); } @@ -538,7 +589,7 @@ bool Frontend::parseGraph(std::vector& buildQueue, CheckResult& chec // this relies on the fact that markDirty marks reverse-dependencies dirty as well // thus if a node is not dirty, all its transitive deps aren't dirty, which means that they won't ever need // to be built, *and* can't form a cycle with any nodes we did process. - if (!it->second.dirty) + if (!it->second.isDirty(forAutocomplete)) continue; // note: this check is technically redundant *except* that getSourceNode has somewhat broken memoization @@ -550,7 +601,7 @@ bool Frontend::parseGraph(std::vector& buildQueue, CheckResult& chec } } - auto [sourceNode, _] = getSourceNode(checkResult, dep); + auto [sourceNode, _] = getSourceNode(checkResult, dep, forAutocomplete); if (sourceNode) { stack.push_back(sourceNode); @@ -594,7 +645,7 @@ LintResult Frontend::lint(const ModuleName& name, std::optionalsecond.dirty; + return it == sourceNodes.end() || it->second.isDirty(forAutocomplete); } /* @@ -699,8 +750,16 @@ bool Frontend::isDirty(const ModuleName& name) const */ void Frontend::markDirty(const ModuleName& name, std::vector* markedDirty) { - if (!moduleResolver.modules.count(name)) - return; + if (FFlag::LuauSeparateTypechecks) + { + if (!moduleResolver.modules.count(name) && !moduleResolverForAutocomplete.modules.count(name)) + return; + } + else + { + if (!moduleResolver.modules.count(name)) + return; + } std::unordered_map> reverseDeps; for (const auto& module : sourceNodes) @@ -722,10 +781,21 @@ void Frontend::markDirty(const ModuleName& name, std::vector* marked if (markedDirty) markedDirty->push_back(next); - if (sourceNode.dirty) - continue; + if (FFlag::LuauSeparateTypechecks) + { + if (sourceNode.dirty && sourceNode.dirtyAutocomplete) + continue; - sourceNode.dirty = true; + sourceNode.dirty = true; + sourceNode.dirtyAutocomplete = true; + } + else + { + if (sourceNode.dirty) + continue; + + sourceNode.dirty = true; + } if (0 == reverseDeps.count(name)) continue; @@ -752,13 +822,13 @@ const SourceModule* Frontend::getSourceModule(const ModuleName& moduleName) cons } // Read AST into sourceModules if necessary. Trace require()s. Report parse errors. -std::pair Frontend::getSourceNode(CheckResult& checkResult, const ModuleName& name) +std::pair Frontend::getSourceNode(CheckResult& checkResult, const ModuleName& name, bool forAutocomplete) { LUAU_TIMETRACE_SCOPE("Frontend::getSourceNode", "Frontend"); LUAU_TIMETRACE_ARGUMENT("name", name.c_str()); auto it = sourceNodes.find(name); - if (it != sourceNodes.end() && !it->second.dirty) + if (it != sourceNodes.end() && !it->second.isDirty(forAutocomplete)) { auto moduleIt = sourceModules.find(name); if (moduleIt != sourceModules.end()) @@ -801,7 +871,19 @@ std::pair Frontend::getSourceNode(CheckResult& check sourceNode.name = name; sourceNode.requires.clear(); sourceNode.requireLocations.clear(); - sourceNode.dirty = true; + + if (FFlag::LuauSeparateTypechecks) + { + if (it == sourceNodes.end()) + { + sourceNode.dirty = true; + sourceNode.dirtyAutocomplete = true; + } + } + else + { + sourceNode.dirty = true; + } for (const auto& [moduleName, location] : requireTrace.requires) sourceNode.requires.insert(moduleName); diff --git a/Analysis/src/IostreamHelpers.cpp b/Analysis/src/IostreamHelpers.cpp index 19c2ddab..a8f67589 100644 --- a/Analysis/src/IostreamHelpers.cpp +++ b/Analysis/src/IostreamHelpers.cpp @@ -23,9 +23,178 @@ std::ostream& operator<<(std::ostream& stream, const AstName& name) return stream << ""; } -std::ostream& operator<<(std::ostream& stream, const TypeMismatch& tm) +template +static void errorToString(std::ostream& stream, const T& err) { - return stream << "TypeMismatch { " << toString(tm.wantedType) << ", " << toString(tm.givenType) << " }"; + if constexpr (false) + { + } + else if constexpr (std::is_same_v) + stream << "TypeMismatch { " << toString(err.wantedType) << ", " << toString(err.givenType) << " }"; + else if constexpr (std::is_same_v) + stream << "UnknownSymbol { " << err.name << " , context " << err.context << " }"; + else if constexpr (std::is_same_v) + stream << "UnknownProperty { " << toString(err.table) << ", key = " << err.key << " }"; + else if constexpr (std::is_same_v) + stream << "NotATable { " << toString(err.ty) << " }"; + else if constexpr (std::is_same_v) + stream << "CannotExtendTable { " << toString(err.tableType) << ", context " << err.context << ", prop \"" << err.prop << "\" }"; + else if constexpr (std::is_same_v) + stream << "OnlyTablesCanHaveMethods { " << toString(err.tableType) << " }"; + else if constexpr (std::is_same_v) + stream << "DuplicateTypeDefinition { " << err.name << " }"; + else if constexpr (std::is_same_v) + stream << "CountMismatch { expected " << err.expected << ", got " << err.actual << ", context " << err.context << " }"; + else if constexpr (std::is_same_v) + stream << "FunctionDoesNotTakeSelf { }"; + else if constexpr (std::is_same_v) + stream << "FunctionRequiresSelf { extraNils " << err.requiredExtraNils << " }"; + else if constexpr (std::is_same_v) + stream << "OccursCheckFailed { }"; + else if constexpr (std::is_same_v) + stream << "UnknownRequire { " << err.modulePath << " }"; + else if constexpr (std::is_same_v) + { + stream << "IncorrectGenericParameterCount { name = " << err.name; + + if (!err.typeFun.typeParams.empty() || !err.typeFun.typePackParams.empty()) + { + stream << "<"; + bool first = true; + for (auto param : err.typeFun.typeParams) + { + if (first) + first = false; + else + stream << ", "; + + stream << toString(param.ty); + } + + for (auto param : err.typeFun.typePackParams) + { + if (first) + first = false; + else + stream << ", "; + + stream << toString(param.tp); + } + + stream << ">"; + } + + stream << ", typeFun = " << toString(err.typeFun.type) << ", actualCount = " << err.actualParameters << " }"; + } + else if constexpr (std::is_same_v) + stream << "SyntaxError { " << err.message << " }"; + else if constexpr (std::is_same_v) + stream << "CodeTooComplex {}"; + else if constexpr (std::is_same_v) + stream << "UnificationTooComplex {}"; + else if constexpr (std::is_same_v) + { + stream << "UnknownPropButFoundLikeProp { key = '" << err.key << "', suggested = { "; + + bool first = true; + for (Name name : err.candidates) + { + if (first) + first = false; + else + stream << ", "; + + stream << "'" << name << "'"; + } + + stream << " }, table = " << toString(err.table) << " } "; + } + else if constexpr (std::is_same_v) + stream << "GenericError { " << err.message << " }"; + else if constexpr (std::is_same_v) + stream << "CannotCallNonFunction { " << toString(err.ty) << " }"; + else if constexpr (std::is_same_v) + stream << "ExtraInformation { " << err.message << " }"; + else if constexpr (std::is_same_v) + stream << "DeprecatedApiUsed { " << err.symbol << ", useInstead = " << err.useInstead << " }"; + else if constexpr (std::is_same_v) + { + stream << "ModuleHasCyclicDependency {"; + + bool first = true; + for (const ModuleName& name : err.cycle) + { + if (first) + first = false; + else + stream << ", "; + + stream << name; + } + + stream << "}"; + } + else if constexpr (std::is_same_v) + stream << "IllegalRequire { " << err.moduleName << ", reason = " << err.reason << " }"; + else if constexpr (std::is_same_v) + stream << "FunctionExitsWithoutReturning {" << toString(err.expectedReturnType) << "}"; + else if constexpr (std::is_same_v) + stream << "DuplicateGenericParameter { " + err.parameterName + " }"; + else if constexpr (std::is_same_v) + stream << "CannotInferBinaryOperation { op = " + toString(err.op) + ", suggested = '" + + (err.suggestedToAnnotate ? *err.suggestedToAnnotate : "") + "', kind " + << err.kind << "}"; + else if constexpr (std::is_same_v) + { + stream << "MissingProperties { superType = '" << toString(err.superType) << "', subType = '" << toString(err.subType) << "', properties = { "; + + bool first = true; + for (Name name : err.properties) + { + if (first) + first = false; + else + stream << ", "; + + stream << "'" << name << "'"; + } + + stream << " }, context " << err.context << " } "; + } + else if constexpr (std::is_same_v) + stream << "SwappedGenericTypeParameter { name = '" + err.name + "', kind = " + std::to_string(err.kind) + " }"; + else if constexpr (std::is_same_v) + stream << "OptionalValueAccess { optional = '" + toString(err.optional) + "' }"; + else if constexpr (std::is_same_v) + { + stream << "MissingUnionProperty { type = '" + toString(err.type) + "', missing = { "; + + bool first = true; + for (auto ty : err.missing) + { + if (first) + first = false; + else + stream << ", "; + + stream << "'" << toString(ty) << "'"; + } + + stream << " }, key = '" + err.key + "' }"; + } + else if constexpr (std::is_same_v) + stream << "TypesAreUnrelated { left = '" + toString(err.left) + "', right = '" + toString(err.right) + "' }"; + else + static_assert(always_false_v, "Non-exhaustive type switch"); +} + +std::ostream& operator<<(std::ostream& stream, const TypeErrorData& data) +{ + auto cb = [&](const auto& e) { + return errorToString(stream, e); + }; + visit(cb, data); + return stream; } std::ostream& operator<<(std::ostream& stream, const TypeError& error) @@ -33,241 +202,6 @@ std::ostream& operator<<(std::ostream& stream, const TypeError& error) return stream << "TypeError { \"" << error.moduleName << "\", " << error.location << ", " << error.data << " }"; } -std::ostream& operator<<(std::ostream& stream, const UnknownSymbol& error) -{ - return stream << "UnknownSymbol { " << error.name << " , context " << error.context << " }"; -} - -std::ostream& operator<<(std::ostream& stream, const UnknownProperty& error) -{ - return stream << "UnknownProperty { " << toString(error.table) << ", key = " << error.key << " }"; -} - -std::ostream& operator<<(std::ostream& stream, const NotATable& ge) -{ - return stream << "NotATable { " << toString(ge.ty) << " }"; -} - -std::ostream& operator<<(std::ostream& stream, const CannotExtendTable& error) -{ - return stream << "CannotExtendTable { " << toString(error.tableType) << ", context " << error.context << ", prop \"" << error.prop << "\" }"; -} - -std::ostream& operator<<(std::ostream& stream, const OnlyTablesCanHaveMethods& error) -{ - return stream << "OnlyTablesCanHaveMethods { " << toString(error.tableType) << " }"; -} - -std::ostream& operator<<(std::ostream& stream, const DuplicateTypeDefinition& error) -{ - return stream << "DuplicateTypeDefinition { " << error.name << " }"; -} - -std::ostream& operator<<(std::ostream& stream, const CountMismatch& error) -{ - return stream << "CountMismatch { expected " << error.expected << ", got " << error.actual << ", context " << error.context << " }"; -} - -std::ostream& operator<<(std::ostream& stream, const FunctionDoesNotTakeSelf&) -{ - return stream << "FunctionDoesNotTakeSelf { }"; -} - -std::ostream& operator<<(std::ostream& stream, const FunctionRequiresSelf& error) -{ - return stream << "FunctionRequiresSelf { extraNils " << error.requiredExtraNils << " }"; -} - -std::ostream& operator<<(std::ostream& stream, const OccursCheckFailed&) -{ - return stream << "OccursCheckFailed { }"; -} - -std::ostream& operator<<(std::ostream& stream, const UnknownRequire& error) -{ - return stream << "UnknownRequire { " << error.modulePath << " }"; -} - -std::ostream& operator<<(std::ostream& stream, const IncorrectGenericParameterCount& error) -{ - stream << "IncorrectGenericParameterCount { name = " << error.name; - - if (!error.typeFun.typeParams.empty() || !error.typeFun.typePackParams.empty()) - { - stream << "<"; - bool first = true; - for (auto param : error.typeFun.typeParams) - { - if (first) - first = false; - else - stream << ", "; - - stream << toString(param.ty); - } - - for (auto param : error.typeFun.typePackParams) - { - if (first) - first = false; - else - stream << ", "; - - stream << toString(param.tp); - } - - stream << ">"; - } - - stream << ", typeFun = " << toString(error.typeFun.type) << ", actualCount = " << error.actualParameters << " }"; - return stream; -} - -std::ostream& operator<<(std::ostream& stream, const SyntaxError& ge) -{ - return stream << "SyntaxError { " << ge.message << " }"; -} - -std::ostream& operator<<(std::ostream& stream, const CodeTooComplex&) -{ - return stream << "CodeTooComplex {}"; -} - -std::ostream& operator<<(std::ostream& stream, const UnificationTooComplex&) -{ - return stream << "UnificationTooComplex {}"; -} - -std::ostream& operator<<(std::ostream& stream, const UnknownPropButFoundLikeProp& e) -{ - stream << "UnknownPropButFoundLikeProp { key = '" << e.key << "', suggested = { "; - - bool first = true; - for (Name name : e.candidates) - { - if (first) - first = false; - else - stream << ", "; - - stream << "'" << name << "'"; - } - - return stream << " }, table = " << toString(e.table) << " } "; -} - -std::ostream& operator<<(std::ostream& stream, const GenericError& ge) -{ - return stream << "GenericError { " << ge.message << " }"; -} - -std::ostream& operator<<(std::ostream& stream, const CannotCallNonFunction& e) -{ - return stream << "CannotCallNonFunction { " << toString(e.ty) << " }"; -} - -std::ostream& operator<<(std::ostream& stream, const FunctionExitsWithoutReturning& error) -{ - return stream << "FunctionExitsWithoutReturning {" << toString(error.expectedReturnType) << "}"; -} - -std::ostream& operator<<(std::ostream& stream, const ExtraInformation& e) -{ - return stream << "ExtraInformation { " << e.message << " }"; -} - -std::ostream& operator<<(std::ostream& stream, const DeprecatedApiUsed& e) -{ - return stream << "DeprecatedApiUsed { " << e.symbol << ", useInstead = " << e.useInstead << " }"; -} - -std::ostream& operator<<(std::ostream& stream, const ModuleHasCyclicDependency& e) -{ - stream << "ModuleHasCyclicDependency {"; - - bool first = true; - for (const ModuleName& name : e.cycle) - { - if (first) - first = false; - else - stream << ", "; - - stream << name; - } - - return stream << "}"; -} - -std::ostream& operator<<(std::ostream& stream, const IllegalRequire& e) -{ - return stream << "IllegalRequire { " << e.moduleName << ", reason = " << e.reason << " }"; -} - -std::ostream& operator<<(std::ostream& stream, const MissingProperties& e) -{ - stream << "MissingProperties { superType = '" << toString(e.superType) << "', subType = '" << toString(e.subType) << "', properties = { "; - - bool first = true; - for (Name name : e.properties) - { - if (first) - first = false; - else - stream << ", "; - - stream << "'" << name << "'"; - } - - return stream << " }, context " << e.context << " } "; -} - -std::ostream& operator<<(std::ostream& stream, const DuplicateGenericParameter& error) -{ - return stream << "DuplicateGenericParameter { " + error.parameterName + " }"; -} - -std::ostream& operator<<(std::ostream& stream, const CannotInferBinaryOperation& error) -{ - return stream << "CannotInferBinaryOperation { op = " + toString(error.op) + ", suggested = '" + - (error.suggestedToAnnotate ? *error.suggestedToAnnotate : "") + "', kind " - << error.kind << "}"; -} - -std::ostream& operator<<(std::ostream& stream, const SwappedGenericTypeParameter& error) -{ - return stream << "SwappedGenericTypeParameter { name = '" + error.name + "', kind = " + std::to_string(error.kind) + " }"; -} - -std::ostream& operator<<(std::ostream& stream, const OptionalValueAccess& error) -{ - return stream << "OptionalValueAccess { optional = '" + toString(error.optional) + "' }"; -} - -std::ostream& operator<<(std::ostream& stream, const MissingUnionProperty& error) -{ - stream << "MissingUnionProperty { type = '" + toString(error.type) + "', missing = { "; - - bool first = true; - for (auto ty : error.missing) - { - if (first) - first = false; - else - stream << ", "; - - stream << "'" << toString(ty) << "'"; - } - - return stream << " }, key = '" + error.key + "' }"; -} - -std::ostream& operator<<(std::ostream& stream, const TypesAreUnrelated& error) -{ - stream << "TypesAreUnrelated { left = '" + toString(error.left) + "', right = '" + toString(error.right) + "' }"; - return stream; -} - std::ostream& operator<<(std::ostream& stream, const TableState& tv) { return stream << static_cast::type>(tv); @@ -283,15 +217,4 @@ std::ostream& operator<<(std::ostream& stream, const TypePackVar& tv) return stream << toString(tv); } -std::ostream& operator<<(std::ostream& lhs, const TypeErrorData& ted) -{ - Luau::visit( - [&](const auto& a) { - lhs << a; - }, - ted); - - return lhs; -} - } // namespace Luau diff --git a/Analysis/src/Module.cpp b/Analysis/src/Module.cpp index 0787d3a4..6bb45245 100644 --- a/Analysis/src/Module.cpp +++ b/Analysis/src/Module.cpp @@ -2,6 +2,7 @@ #include "Luau/Module.h" #include "Luau/Common.h" +#include "Luau/Clone.h" #include "Luau/RecursionCounter.h" #include "Luau/Scope.h" #include "Luau/TypeInfer.h" @@ -12,7 +13,6 @@ #include LUAU_FASTFLAGVARIABLE(DebugLuauFreezeArena, false) -LUAU_FASTINTVARIABLE(LuauTypeCloneRecursionLimit, 300) LUAU_FASTFLAGVARIABLE(LuauCloneDeclaredGlobals, false) namespace Luau @@ -113,363 +113,6 @@ TypePackId TypeArena::addTypePack(TypePackVar tp) return allocated; } -namespace -{ - -struct TypePackCloner; - -/* - * Both TypeCloner and TypePackCloner work by depositing the requested type variable into the appropriate 'seen' set. - * They do not return anything because their sole consumer (the deepClone function) already has a pointer into this storage. - */ - -struct TypeCloner -{ - TypeCloner(TypeArena& dest, TypeId typeId, SeenTypes& seenTypes, SeenTypePacks& seenTypePacks, CloneState& cloneState) - : dest(dest) - , typeId(typeId) - , seenTypes(seenTypes) - , seenTypePacks(seenTypePacks) - , cloneState(cloneState) - { - } - - TypeArena& dest; - TypeId typeId; - SeenTypes& seenTypes; - SeenTypePacks& seenTypePacks; - CloneState& cloneState; - - template - void defaultClone(const T& t); - - void operator()(const Unifiable::Free& t); - void operator()(const Unifiable::Generic& t); - void operator()(const Unifiable::Bound& t); - void operator()(const Unifiable::Error& t); - void operator()(const PrimitiveTypeVar& t); - void operator()(const SingletonTypeVar& t); - void operator()(const FunctionTypeVar& t); - void operator()(const TableTypeVar& t); - void operator()(const MetatableTypeVar& t); - void operator()(const ClassTypeVar& t); - void operator()(const AnyTypeVar& t); - void operator()(const UnionTypeVar& t); - void operator()(const IntersectionTypeVar& t); - void operator()(const LazyTypeVar& t); -}; - -struct TypePackCloner -{ - TypeArena& dest; - TypePackId typePackId; - SeenTypes& seenTypes; - SeenTypePacks& seenTypePacks; - CloneState& cloneState; - - TypePackCloner(TypeArena& dest, TypePackId typePackId, SeenTypes& seenTypes, SeenTypePacks& seenTypePacks, CloneState& cloneState) - : dest(dest) - , typePackId(typePackId) - , seenTypes(seenTypes) - , seenTypePacks(seenTypePacks) - , cloneState(cloneState) - { - } - - template - void defaultClone(const T& t) - { - TypePackId cloned = dest.addTypePack(TypePackVar{t}); - seenTypePacks[typePackId] = cloned; - } - - void operator()(const Unifiable::Free& t) - { - cloneState.encounteredFreeType = true; - - TypePackId err = getSingletonTypes().errorRecoveryTypePack(getSingletonTypes().anyTypePack); - TypePackId cloned = dest.addTypePack(*err); - seenTypePacks[typePackId] = cloned; - } - - void operator()(const Unifiable::Generic& t) - { - defaultClone(t); - } - void operator()(const Unifiable::Error& t) - { - defaultClone(t); - } - - // While we are a-cloning, we can flatten out bound TypeVars and make things a bit tighter. - // We just need to be sure that we rewrite pointers both to the binder and the bindee to the same pointer. - void operator()(const Unifiable::Bound& t) - { - TypePackId cloned = clone(t.boundTo, dest, seenTypes, seenTypePacks, cloneState); - seenTypePacks[typePackId] = cloned; - } - - void operator()(const VariadicTypePack& t) - { - TypePackId cloned = dest.addTypePack(TypePackVar{VariadicTypePack{clone(t.ty, dest, seenTypes, seenTypePacks, cloneState)}}); - seenTypePacks[typePackId] = cloned; - } - - void operator()(const TypePack& t) - { - TypePackId cloned = dest.addTypePack(TypePack{}); - TypePack* destTp = getMutable(cloned); - LUAU_ASSERT(destTp != nullptr); - seenTypePacks[typePackId] = cloned; - - for (TypeId ty : t.head) - destTp->head.push_back(clone(ty, dest, seenTypes, seenTypePacks, cloneState)); - - if (t.tail) - destTp->tail = clone(*t.tail, dest, seenTypes, seenTypePacks, cloneState); - } -}; - -template -void TypeCloner::defaultClone(const T& t) -{ - TypeId cloned = dest.addType(t); - seenTypes[typeId] = cloned; -} - -void TypeCloner::operator()(const Unifiable::Free& t) -{ - cloneState.encounteredFreeType = true; - TypeId err = getSingletonTypes().errorRecoveryType(getSingletonTypes().anyType); - TypeId cloned = dest.addType(*err); - seenTypes[typeId] = cloned; -} - -void TypeCloner::operator()(const Unifiable::Generic& t) -{ - defaultClone(t); -} - -void TypeCloner::operator()(const Unifiable::Bound& t) -{ - TypeId boundTo = clone(t.boundTo, dest, seenTypes, seenTypePacks, cloneState); - seenTypes[typeId] = boundTo; -} - -void TypeCloner::operator()(const Unifiable::Error& t) -{ - defaultClone(t); -} - -void TypeCloner::operator()(const PrimitiveTypeVar& t) -{ - defaultClone(t); -} - -void TypeCloner::operator()(const SingletonTypeVar& t) -{ - defaultClone(t); -} - -void TypeCloner::operator()(const FunctionTypeVar& t) -{ - TypeId result = dest.addType(FunctionTypeVar{TypeLevel{0, 0}, {}, {}, nullptr, nullptr, t.definition, t.hasSelf}); - FunctionTypeVar* ftv = getMutable(result); - LUAU_ASSERT(ftv != nullptr); - - seenTypes[typeId] = result; - - for (TypeId generic : t.generics) - ftv->generics.push_back(clone(generic, dest, seenTypes, seenTypePacks, cloneState)); - - for (TypePackId genericPack : t.genericPacks) - ftv->genericPacks.push_back(clone(genericPack, dest, seenTypes, seenTypePacks, cloneState)); - - ftv->tags = t.tags; - ftv->argTypes = clone(t.argTypes, dest, seenTypes, seenTypePacks, cloneState); - ftv->argNames = t.argNames; - ftv->retType = clone(t.retType, dest, seenTypes, seenTypePacks, cloneState); -} - -void TypeCloner::operator()(const TableTypeVar& t) -{ - // If table is now bound to another one, we ignore the content of the original - if (t.boundTo) - { - TypeId boundTo = clone(*t.boundTo, dest, seenTypes, seenTypePacks, cloneState); - seenTypes[typeId] = boundTo; - return; - } - - TypeId result = dest.addType(TableTypeVar{}); - TableTypeVar* ttv = getMutable(result); - LUAU_ASSERT(ttv != nullptr); - - *ttv = t; - - seenTypes[typeId] = result; - - ttv->level = TypeLevel{0, 0}; - - for (const auto& [name, prop] : t.props) - ttv->props[name] = {clone(prop.type, dest, seenTypes, seenTypePacks, cloneState), prop.deprecated, {}, prop.location, prop.tags}; - - if (t.indexer) - ttv->indexer = TableIndexer{clone(t.indexer->indexType, dest, seenTypes, seenTypePacks, cloneState), - clone(t.indexer->indexResultType, dest, seenTypes, seenTypePacks, cloneState)}; - - for (TypeId& arg : ttv->instantiatedTypeParams) - arg = clone(arg, dest, seenTypes, seenTypePacks, cloneState); - - for (TypePackId& arg : ttv->instantiatedTypePackParams) - arg = clone(arg, dest, seenTypes, seenTypePacks, cloneState); - - if (ttv->state == TableState::Free) - { - cloneState.encounteredFreeType = true; - - ttv->state = TableState::Sealed; - } - - ttv->definitionModuleName = t.definitionModuleName; - ttv->methodDefinitionLocations = t.methodDefinitionLocations; - ttv->tags = t.tags; -} - -void TypeCloner::operator()(const MetatableTypeVar& t) -{ - TypeId result = dest.addType(MetatableTypeVar{}); - MetatableTypeVar* mtv = getMutable(result); - seenTypes[typeId] = result; - - mtv->table = clone(t.table, dest, seenTypes, seenTypePacks, cloneState); - mtv->metatable = clone(t.metatable, dest, seenTypes, seenTypePacks, cloneState); -} - -void TypeCloner::operator()(const ClassTypeVar& t) -{ - TypeId result = dest.addType(ClassTypeVar{t.name, {}, std::nullopt, std::nullopt, t.tags, t.userData}); - ClassTypeVar* ctv = getMutable(result); - - seenTypes[typeId] = result; - - for (const auto& [name, prop] : t.props) - ctv->props[name] = {clone(prop.type, dest, seenTypes, seenTypePacks, cloneState), prop.deprecated, {}, prop.location, prop.tags}; - - if (t.parent) - ctv->parent = clone(*t.parent, dest, seenTypes, seenTypePacks, cloneState); - - if (t.metatable) - ctv->metatable = clone(*t.metatable, dest, seenTypes, seenTypePacks, cloneState); -} - -void TypeCloner::operator()(const AnyTypeVar& t) -{ - defaultClone(t); -} - -void TypeCloner::operator()(const UnionTypeVar& t) -{ - std::vector options; - options.reserve(t.options.size()); - - for (TypeId ty : t.options) - options.push_back(clone(ty, dest, seenTypes, seenTypePacks, cloneState)); - - TypeId result = dest.addType(UnionTypeVar{std::move(options)}); - seenTypes[typeId] = result; -} - -void TypeCloner::operator()(const IntersectionTypeVar& t) -{ - TypeId result = dest.addType(IntersectionTypeVar{}); - seenTypes[typeId] = result; - - IntersectionTypeVar* option = getMutable(result); - LUAU_ASSERT(option != nullptr); - - for (TypeId ty : t.parts) - option->parts.push_back(clone(ty, dest, seenTypes, seenTypePacks, cloneState)); -} - -void TypeCloner::operator()(const LazyTypeVar& t) -{ - defaultClone(t); -} - -} // anonymous namespace - -TypePackId clone(TypePackId tp, TypeArena& dest, SeenTypes& seenTypes, SeenTypePacks& seenTypePacks, CloneState& cloneState) -{ - if (tp->persistent) - return tp; - - RecursionLimiter _ra(&cloneState.recursionCount, FInt::LuauTypeCloneRecursionLimit); - - TypePackId& res = seenTypePacks[tp]; - - if (res == nullptr) - { - TypePackCloner cloner{dest, tp, seenTypes, seenTypePacks, cloneState}; - Luau::visit(cloner, tp->ty); // Mutates the storage that 'res' points into. - } - - return res; -} - -TypeId clone(TypeId typeId, TypeArena& dest, SeenTypes& seenTypes, SeenTypePacks& seenTypePacks, CloneState& cloneState) -{ - if (typeId->persistent) - return typeId; - - RecursionLimiter _ra(&cloneState.recursionCount, FInt::LuauTypeCloneRecursionLimit); - - TypeId& res = seenTypes[typeId]; - - if (res == nullptr) - { - TypeCloner cloner{dest, typeId, seenTypes, seenTypePacks, cloneState}; - Luau::visit(cloner, typeId->ty); // Mutates the storage that 'res' points into. - - // Persistent types are not being cloned and we get the original type back which might be read-only - if (!res->persistent) - asMutable(res)->documentationSymbol = typeId->documentationSymbol; - } - - return res; -} - -TypeFun clone(const TypeFun& typeFun, TypeArena& dest, SeenTypes& seenTypes, SeenTypePacks& seenTypePacks, CloneState& cloneState) -{ - TypeFun result; - - for (auto param : typeFun.typeParams) - { - TypeId ty = clone(param.ty, dest, seenTypes, seenTypePacks, cloneState); - std::optional defaultValue; - - if (param.defaultValue) - defaultValue = clone(*param.defaultValue, dest, seenTypes, seenTypePacks, cloneState); - - result.typeParams.push_back({ty, defaultValue}); - } - - for (auto param : typeFun.typePackParams) - { - TypePackId tp = clone(param.tp, dest, seenTypes, seenTypePacks, cloneState); - std::optional defaultValue; - - if (param.defaultValue) - defaultValue = clone(*param.defaultValue, dest, seenTypes, seenTypePacks, cloneState); - - result.typePackParams.push_back({tp, defaultValue}); - } - - result.type = clone(typeFun.type, dest, seenTypes, seenTypePacks, cloneState); - - return result; -} - ScopePtr Module::getModuleScope() const { LUAU_ASSERT(!scopes.empty()); diff --git a/Analysis/src/TxnLog.cpp b/Analysis/src/TxnLog.cpp index 876f5f05..5fbb596d 100644 --- a/Analysis/src/TxnLog.cpp +++ b/Analysis/src/TxnLog.cpp @@ -7,6 +7,8 @@ #include #include +LUAU_FASTFLAGVARIABLE(LuauTxnLogPreserveOwner, false) + namespace Luau { @@ -78,11 +80,32 @@ void TxnLog::concat(TxnLog rhs) void TxnLog::commit() { - for (auto& [ty, rep] : typeVarChanges) - *asMutable(ty) = rep.get()->pending; + if (FFlag::LuauTxnLogPreserveOwner) + { + for (auto& [ty, rep] : typeVarChanges) + { + TypeArena* owningArena = ty->owningArena; + TypeVar* mtv = asMutable(ty); + *mtv = rep.get()->pending; + mtv->owningArena = owningArena; + } - for (auto& [tp, rep] : typePackChanges) - *asMutable(tp) = rep.get()->pending; + for (auto& [tp, rep] : typePackChanges) + { + TypeArena* owningArena = tp->owningArena; + TypePackVar* mpv = asMutable(tp); + *mpv = rep.get()->pending; + mpv->owningArena = owningArena; + } + } + else + { + for (auto& [ty, rep] : typeVarChanges) + *asMutable(ty) = rep.get()->pending; + + for (auto& [tp, rep] : typePackChanges) + *asMutable(tp) = rep.get()->pending; + } clear(); } diff --git a/Analysis/src/TypeInfer.cpp b/Analysis/src/TypeInfer.cpp index 6df6bff0..10930248 100644 --- a/Analysis/src/TypeInfer.cpp +++ b/Analysis/src/TypeInfer.cpp @@ -22,6 +22,9 @@ LUAU_FASTINTVARIABLE(LuauTypeInferRecursionLimit, 500) LUAU_FASTINTVARIABLE(LuauTypeInferTypePackLoopLimit, 5000) LUAU_FASTINTVARIABLE(LuauCheckRecursionLimit, 500) LUAU_FASTFLAG(LuauKnowsTheDataModel3) +LUAU_FASTFLAG(LuauSeparateTypechecks) +LUAU_FASTFLAG(LuauAutocompleteSingletonTypes) +LUAU_FASTFLAGVARIABLE(LuauCyclicModuleTypeSurface, false) LUAU_FASTFLAGVARIABLE(LuauEqConstraint, false) LUAU_FASTFLAGVARIABLE(LuauWeakEqConstraint, false) // Eventually removed as false. LUAU_FASTFLAGVARIABLE(DebugLuauFreezeDuringUnification, false) @@ -35,7 +38,7 @@ LUAU_FASTFLAGVARIABLE(LuauExpectedTypesOfProperties, false) LUAU_FASTFLAGVARIABLE(LuauErrorRecoveryType, false) LUAU_FASTFLAGVARIABLE(LuauOnlyMutateInstantiatedTables, false) LUAU_FASTFLAGVARIABLE(LuauPropertiesGetExpectedType, false) -LUAU_FASTFLAGVARIABLE(LuauStatFunctionSimplify3, false) +LUAU_FASTFLAGVARIABLE(LuauStatFunctionSimplify4, false) LUAU_FASTFLAGVARIABLE(LuauUnsealedTableLiteral, false) LUAU_FASTFLAG(LuauTypeMismatchModuleName) LUAU_FASTFLAGVARIABLE(LuauTwoPassAliasDefinitionFix, false) @@ -46,6 +49,7 @@ LUAU_FASTFLAGVARIABLE(LuauDoNotTryToReduce, false) LUAU_FASTFLAGVARIABLE(LuauDoNotAccidentallyDependOnPointerOrdering, false) LUAU_FASTFLAGVARIABLE(LuauFixArgumentCountMismatchAmountWithGenericTypes, false) LUAU_FASTFLAGVARIABLE(LuauFixIncorrectLineNumberDuplicateType, false) +LUAU_FASTFLAGVARIABLE(LuauCheckImplicitNumbericKeys, false) LUAU_FASTFLAG(LuauAnyInIsOptionalIsOptional) LUAU_FASTFLAGVARIABLE(LuauDecoupleOperatorInferenceFromUnifiedTypeInference, false) LUAU_FASTFLAGVARIABLE(LuauArgCountMismatchSaysAtLeastWhenVariadic, false) @@ -53,6 +57,11 @@ LUAU_FASTFLAGVARIABLE(LuauArgCountMismatchSaysAtLeastWhenVariadic, false) namespace Luau { +const char* TimeLimitError::what() const throw() +{ + return "Typeinfer failed to complete in allotted time"; +} + static bool typeCouldHaveMetatable(TypeId ty) { return get(follow(ty)) || get(follow(ty)) || get(follow(ty)); @@ -251,6 +260,12 @@ ModulePtr TypeChecker::check(const SourceModule& module, Mode mode, std::optiona currentModule.reset(new Module()); currentModule->type = module.type; + if (FFlag::LuauSeparateTypechecks) + { + currentModule->allocator = module.allocator; + currentModule->names = module.names; + } + iceHandler->moduleName = module.name; ScopePtr parentScope = environmentScope.value_or(globalScope); @@ -271,7 +286,21 @@ ModulePtr TypeChecker::check(const SourceModule& module, Mode mode, std::optiona if (prepareModuleScope) prepareModuleScope(module.name, currentModule->getModuleScope()); - checkBlock(moduleScope, *module.root); + if (FFlag::LuauSeparateTypechecks) + { + try + { + checkBlock(moduleScope, *module.root); + } + catch (const TimeLimitError&) + { + currentModule->timeout = true; + } + } + else + { + checkBlock(moduleScope, *module.root); + } if (get(follow(moduleScope->returnType))) moduleScope->returnType = addTypePack(TypePack{{}, std::nullopt}); @@ -366,6 +395,9 @@ void TypeChecker::check(const ScopePtr& scope, const AstStat& program) } else ice("Unknown AstStat"); + + if (FFlag::LuauSeparateTypechecks && finishTime && TimeTrace::getClock() > *finishTime) + throw TimeLimitError(); } // This particular overload is for do...end. If you need to not increase the scope level, use checkBlock directly. @@ -1115,22 +1147,18 @@ void TypeChecker::check(const ScopePtr& scope, TypeId ty, const ScopePtr& funSco scope->bindings[name->local] = {anyIfNonstrict(quantify(funScope, ty, name->local->location)), name->local->location}; return; } - else if (auto name = function.name->as(); name && FFlag::LuauStatFunctionSimplify3) + else if (auto name = function.name->as(); name && FFlag::LuauStatFunctionSimplify4) { TypeId exprTy = checkExpr(scope, *name->expr).type; TableTypeVar* ttv = getMutableTableType(exprTy); - if (!ttv) + + if (!getIndexTypeFromType(scope, exprTy, name->index.value, name->indexLocation, false)) { - if (isTableIntersection(exprTy)) + if (ttv || isTableIntersection(exprTy)) reportError(TypeError{function.location, CannotExtendTable{exprTy, CannotExtendTable::Property, name->index.value}}); - else if (!get(exprTy) && !get(exprTy)) + else reportError(TypeError{function.location, OnlyTablesCanHaveMethods{exprTy}}); } - else if (ttv->state == TableState::Sealed) - { - if (!getIndexTypeFromType(scope, exprTy, name->index.value, name->indexLocation, false)) - reportError(TypeError{function.location, CannotExtendTable{exprTy, CannotExtendTable::Property, name->index.value}}); - } ty = follow(ty); @@ -1153,7 +1181,7 @@ void TypeChecker::check(const ScopePtr& scope, TypeId ty, const ScopePtr& funSco if (ttv && ttv->state != TableState::Sealed) ttv->props[name->index.value] = {follow(quantify(funScope, ty, name->indexLocation)), /* deprecated */ false, {}, name->indexLocation}; } - else if (FFlag::LuauStatFunctionSimplify3) + else if (FFlag::LuauStatFunctionSimplify4) { LUAU_ASSERT(function.name->is()); @@ -1163,7 +1191,7 @@ void TypeChecker::check(const ScopePtr& scope, TypeId ty, const ScopePtr& funSco } else if (function.func->self) { - LUAU_ASSERT(!FFlag::LuauStatFunctionSimplify3); + LUAU_ASSERT(!FFlag::LuauStatFunctionSimplify4); AstExprIndexName* indexName = function.name->as(); if (!indexName) @@ -1202,7 +1230,7 @@ void TypeChecker::check(const ScopePtr& scope, TypeId ty, const ScopePtr& funSco } else { - LUAU_ASSERT(!FFlag::LuauStatFunctionSimplify3); + LUAU_ASSERT(!FFlag::LuauStatFunctionSimplify4); TypeId leftType = checkLValueBinding(scope, *function.name); @@ -2030,7 +2058,11 @@ TypeId TypeChecker::checkExprTable( indexer = expectedTable->indexer; if (indexer) + { + if (FFlag::LuauCheckImplicitNumbericKeys) + unify(numberType, indexer->indexType, value->location); unify(valueType, indexer->indexResultType, value->location); + } else indexer = TableIndexer{numberType, anyIfNonstrict(valueType)}; } @@ -2984,35 +3016,33 @@ TypeId TypeChecker::checkFunctionName(const ScopePtr& scope, AstExpr& funName, T else if (auto indexName = funName.as()) { TypeId lhsType = checkExpr(scope, *indexName->expr).type; - if (get(lhsType) || get(lhsType)) + + if (!FFlag::LuauStatFunctionSimplify4 && (get(lhsType) || get(lhsType))) return lhsType; TableTypeVar* ttv = getMutableTableType(lhsType); - if (!ttv) + + if (FFlag::LuauStatFunctionSimplify4) { - if (!FFlag::LuauErrorRecoveryType && !isTableIntersection(lhsType)) - // This error now gets reported when we check the function body. - reportError(TypeError{funName.location, OnlyTablesCanHaveMethods{lhsType}}); - - return errorRecoveryType(scope); - } - - if (FFlag::LuauStatFunctionSimplify3) - { - if (lhsType->persistent) - return errorRecoveryType(scope); - - // Cannot extend sealed table, but we dont report an error here because it will be reported during AstStatFunction check - if (ttv->state == TableState::Sealed) + if (!ttv || ttv->state == TableState::Sealed) { - if (ttv->indexer && isPrim(ttv->indexer->indexType, PrimitiveTypeVar::String)) - return ttv->indexer->indexResultType; - else - return errorRecoveryType(scope); + if (auto ty = getIndexTypeFromType(scope, lhsType, indexName->index.value, indexName->indexLocation, false)) + return *ty; + + return errorRecoveryType(scope); } } else { + if (!ttv) + { + if (!FFlag::LuauErrorRecoveryType && !isTableIntersection(lhsType)) + // This error now gets reported when we check the function body. + reportError(TypeError{funName.location, OnlyTablesCanHaveMethods{lhsType}}); + + return errorRecoveryType(scope); + } + if (lhsType->persistent || ttv->state == TableState::Sealed) return errorRecoveryType(scope); } @@ -3020,7 +3050,12 @@ TypeId TypeChecker::checkFunctionName(const ScopePtr& scope, AstExpr& funName, T Name name = indexName->index.value; if (ttv->props.count(name)) - return errorRecoveryType(scope); + { + if (FFlag::LuauStatFunctionSimplify4) + return ttv->props[name].type; + else + return errorRecoveryType(scope); + } Property& property = ttv->props[name]; @@ -4155,6 +4190,20 @@ TypeId TypeChecker::checkRequire(const ScopePtr& scope, const ModuleInfo& module return anyType; } + // Types of requires that transitively refer to current module have to be replaced with 'any' + std::string humanReadableName; + + if (FFlag::LuauCyclicModuleTypeSurface) + { + humanReadableName = resolver->getHumanReadableModuleName(moduleInfo.name); + + for (const auto& [location, path] : requireCycles) + { + if (!path.empty() && path.front() == humanReadableName) + return anyType; + } + } + ModulePtr module = resolver->getModule(moduleInfo.name); if (!module) { @@ -4163,8 +4212,15 @@ TypeId TypeChecker::checkRequire(const ScopePtr& scope, const ModuleInfo& module // we will already have reported the error. if (!resolver->moduleExists(moduleInfo.name) && !moduleInfo.optional) { - std::string reportedModulePath = resolver->getHumanReadableModuleName(moduleInfo.name); - reportError(TypeError{location, UnknownRequire{reportedModulePath}}); + if (FFlag::LuauCyclicModuleTypeSurface) + { + reportError(TypeError{location, UnknownRequire{humanReadableName}}); + } + else + { + std::string reportedModulePath = resolver->getHumanReadableModuleName(moduleInfo.name); + reportError(TypeError{location, UnknownRequire{reportedModulePath}}); + } } return errorRecoveryType(scope); @@ -4172,8 +4228,15 @@ TypeId TypeChecker::checkRequire(const ScopePtr& scope, const ModuleInfo& module if (module->type != SourceCode::Module) { - std::string humanReadableName = resolver->getHumanReadableModuleName(moduleInfo.name); - reportError(location, IllegalRequire{humanReadableName, "Module is not a ModuleScript. It cannot be required."}); + if (FFlag::LuauCyclicModuleTypeSurface) + { + reportError(location, IllegalRequire{humanReadableName, "Module is not a ModuleScript. It cannot be required."}); + } + else + { + std::string humanReadableName = resolver->getHumanReadableModuleName(moduleInfo.name); + reportError(location, IllegalRequire{humanReadableName, "Module is not a ModuleScript. It cannot be required."}); + } return errorRecoveryType(scope); } @@ -4185,8 +4248,15 @@ TypeId TypeChecker::checkRequire(const ScopePtr& scope, const ModuleInfo& module std::optional moduleType = first(modulePack); if (!moduleType) { - std::string humanReadableName = resolver->getHumanReadableModuleName(moduleInfo.name); - reportError(location, IllegalRequire{humanReadableName, "Module does not return exactly 1 value. It cannot be required."}); + if (FFlag::LuauCyclicModuleTypeSurface) + { + reportError(location, IllegalRequire{humanReadableName, "Module does not return exactly 1 value. It cannot be required."}); + } + else + { + std::string humanReadableName = resolver->getHumanReadableModuleName(moduleInfo.name); + reportError(location, IllegalRequire{humanReadableName, "Module does not return exactly 1 value. It cannot be required."}); + } return errorRecoveryType(scope); } @@ -4629,7 +4699,9 @@ TypeId TypeChecker::freshType(TypeLevel level) TypeId TypeChecker::singletonType(bool value) { - // TODO: cache singleton types + if (FFlag::LuauAutocompleteSingletonTypes) + return value ? getSingletonTypes().trueType : getSingletonTypes().falseType; + return currentModule->internalTypes.addType(TypeVar(SingletonTypeVar(BooleanSingleton{value}))); } diff --git a/Analysis/src/TypeVar.cpp b/Analysis/src/TypeVar.cpp index 36545ad9..dbc412fc 100644 --- a/Analysis/src/TypeVar.cpp +++ b/Analysis/src/TypeVar.cpp @@ -652,6 +652,8 @@ static TypeVar numberType_{PrimitiveTypeVar{PrimitiveTypeVar::Number}, /*persist static TypeVar stringType_{PrimitiveTypeVar{PrimitiveTypeVar::String}, /*persistent*/ true}; static TypeVar booleanType_{PrimitiveTypeVar{PrimitiveTypeVar::Boolean}, /*persistent*/ true}; static TypeVar threadType_{PrimitiveTypeVar{PrimitiveTypeVar::Thread}, /*persistent*/ true}; +static TypeVar trueType_{SingletonTypeVar{BooleanSingleton{true}}, /*persistent*/ true}; +static TypeVar falseType_{SingletonTypeVar{BooleanSingleton{false}}, /*persistent*/ true}; static TypeVar anyType_{AnyTypeVar{}}; static TypeVar errorType_{ErrorTypeVar{}}; static TypeVar optionalNumberType_{UnionTypeVar{{&numberType_, &nilType_}}}; @@ -665,6 +667,8 @@ SingletonTypes::SingletonTypes() , stringType(&stringType_) , booleanType(&booleanType_) , threadType(&threadType_) + , trueType(&trueType_) + , falseType(&falseType_) , anyType(&anyType_) , optionalNumberType(&optionalNumberType_) , anyTypePack(&anyTypePack_) diff --git a/Ast/include/Luau/TimeTrace.h b/Ast/include/Luau/TimeTrace.h index 5018456f..9f7b2bdf 100644 --- a/Ast/include/Luau/TimeTrace.h +++ b/Ast/include/Luau/TimeTrace.h @@ -9,14 +9,21 @@ LUAU_FASTFLAG(DebugLuauTimeTracing) +namespace Luau +{ +namespace TimeTrace +{ +double getClock(); +uint32_t getClockMicroseconds(); +} // namespace TimeTrace +} // namespace Luau + #if defined(LUAU_ENABLE_TIME_TRACE) namespace Luau { namespace TimeTrace { -uint32_t getClockMicroseconds(); - struct Token { const char* name; diff --git a/Ast/src/Parser.cpp b/Ast/src/Parser.cpp index f6dfd904..f9d32178 100644 --- a/Ast/src/Parser.cpp +++ b/Ast/src/Parser.cpp @@ -10,6 +10,7 @@ // See docs/SyntaxChanges.md for an explanation. LUAU_FASTINTVARIABLE(LuauRecursionLimit, 1000) LUAU_FASTINTVARIABLE(LuauParseErrorLimit, 100) +LUAU_FASTFLAGVARIABLE(LuauParseRecoverUnexpectedPack, false) namespace Luau { @@ -1420,6 +1421,11 @@ AstType* Parser::parseTypeAnnotation(TempVector& parts, const Location parts.push_back(parseSimpleTypeAnnotation(/* allowPack= */ false).type); isIntersection = true; } + else if (FFlag::LuauParseRecoverUnexpectedPack && c == Lexeme::Dot3) + { + report(lexer.current().location, "Unexpected '...' after type annotation"); + nextLexeme(); + } else break; } @@ -1536,6 +1542,11 @@ AstTypeOrPack Parser::parseSimpleTypeAnnotation(bool allowPack) prefix = name.name; name = parseIndexName("field name", pointPosition); } + else if (FFlag::LuauParseRecoverUnexpectedPack && lexer.current().type == Lexeme::Dot3) + { + report(lexer.current().location, "Unexpected '...' after type name; type pack is not allowed in this context"); + nextLexeme(); + } else if (name.name == "typeof") { Lexeme typeofBegin = lexer.current(); diff --git a/Ast/src/TimeTrace.cpp b/Ast/src/TimeTrace.cpp index 19564f05..e3807683 100644 --- a/Ast/src/TimeTrace.cpp +++ b/Ast/src/TimeTrace.cpp @@ -26,9 +26,6 @@ #include LUAU_FASTFLAGVARIABLE(DebugLuauTimeTracing, false) - -#if defined(LUAU_ENABLE_TIME_TRACE) - namespace Luau { namespace TimeTrace @@ -67,6 +64,14 @@ static double getClockTimestamp() #endif } +double getClock() +{ + static double period = getClockPeriod(); + static double start = getClockTimestamp(); + + return (getClockTimestamp() - start) * period; +} + uint32_t getClockMicroseconds() { static double period = getClockPeriod() * 1e6; @@ -74,7 +79,15 @@ uint32_t getClockMicroseconds() return uint32_t((getClockTimestamp() - start) * period); } +} // namespace TimeTrace +} // namespace Luau +#if defined(LUAU_ENABLE_TIME_TRACE) + +namespace Luau +{ +namespace TimeTrace +{ struct GlobalContext { GlobalContext() = default; diff --git a/Compiler/src/ConstantFolding.cpp b/Compiler/src/ConstantFolding.cpp index 60a7c169..35ea0bf0 100644 --- a/Compiler/src/ConstantFolding.cpp +++ b/Compiler/src/ConstantFolding.cpp @@ -290,7 +290,8 @@ struct ConstantVisitor : AstVisitor Constant la = analyze(expr->left); Constant ra = analyze(expr->right); - if (la.type != Constant::Type_Unknown && ra.type != Constant::Type_Unknown) + // note: ra doesn't need to be constant to fold and/or + if (la.type != Constant::Type_Unknown) foldBinary(result, expr->op, la, ra); } else if (AstExprTypeAssertion* expr = node->as()) diff --git a/Sources.cmake b/Sources.cmake index 59b38497..6f110f1f 100644 --- a/Sources.cmake +++ b/Sources.cmake @@ -47,6 +47,7 @@ target_sources(Luau.Analysis PRIVATE Analysis/include/Luau/Autocomplete.h Analysis/include/Luau/BuiltinDefinitions.h Analysis/include/Luau/Config.h + Analysis/include/Luau/Clone.h Analysis/include/Luau/Documentation.h Analysis/include/Luau/Error.h Analysis/include/Luau/FileResolver.h @@ -85,6 +86,7 @@ target_sources(Luau.Analysis PRIVATE Analysis/src/Autocomplete.cpp Analysis/src/BuiltinDefinitions.cpp Analysis/src/Config.cpp + Analysis/src/Clone.cpp Analysis/src/Error.cpp Analysis/src/Frontend.cpp Analysis/src/IostreamHelpers.cpp diff --git a/VM/src/lfunc.h b/VM/src/lfunc.h index 8047cebe..a260d00a 100644 --- a/VM/src/lfunc.h +++ b/VM/src/lfunc.h @@ -14,6 +14,6 @@ LUAI_FUNC UpVal* luaF_findupval(lua_State* L, StkId level); LUAI_FUNC void luaF_close(lua_State* L, StkId level); LUAI_FUNC void luaF_freeproto(lua_State* L, Proto* f, struct lua_Page* page); LUAI_FUNC void luaF_freeclosure(lua_State* L, Closure* c, struct lua_Page* page); -void luaF_unlinkupval(UpVal* uv); +LUAI_FUNC void luaF_unlinkupval(UpVal* uv); LUAI_FUNC void luaF_freeupval(lua_State* L, UpVal* uv, struct lua_Page* page); LUAI_FUNC const LocVar* luaF_getlocal(const Proto* func, int local_number, int pc); diff --git a/VM/src/ltablib.cpp b/VM/src/ltablib.cpp index 241a99e3..41887f4b 100644 --- a/VM/src/ltablib.cpp +++ b/VM/src/ltablib.cpp @@ -201,7 +201,7 @@ static int tmove(lua_State* L) void (*telemetrycb)(lua_State* L, int f, int e, int t, int nf, int nt) = lua_table_move_telemetry; - if (DFFlag::LuauTableMoveTelemetry2 && telemetrycb) + if (DFFlag::LuauTableMoveTelemetry2 && telemetrycb && e >= f) { int nf = lua_objlen(L, 1); int nt = lua_objlen(L, tt); diff --git a/tests/Autocomplete.test.cpp b/tests/Autocomplete.test.cpp index 4e8a1d55..2e7902f5 100644 --- a/tests/Autocomplete.test.cpp +++ b/tests/Autocomplete.test.cpp @@ -14,6 +14,7 @@ LUAU_FASTFLAG(LuauTraceTypesInNonstrictMode2) LUAU_FASTFLAG(LuauSetMetatableDoesNotTimeTravel) +LUAU_FASTFLAG(LuauSeparateTypechecks) using namespace Luau; @@ -25,6 +26,11 @@ static std::optional nullCallback(std::string tag, std::op template struct ACFixtureImpl : BaseType { + ACFixtureImpl() + : Fixture(true, true) + { + } + AutocompleteResult autocomplete(unsigned row, unsigned column) { return Luau::autocomplete(this->frontend, "MainModule", Position{row, column}, nullCallback); @@ -72,7 +78,25 @@ struct ACFixtureImpl : BaseType } LUAU_ASSERT("Digit expected after @ symbol" && prevChar != '@'); - return Fixture::check(filteredSource); + return BaseType::check(filteredSource); + } + + LoadDefinitionFileResult loadDefinition(const std::string& source) + { + if (FFlag::LuauSeparateTypechecks) + { + TypeChecker& typeChecker = this->frontend.typeCheckerForAutocomplete; + unfreeze(typeChecker.globalTypes); + LoadDefinitionFileResult result = loadDefinitionFile(typeChecker, typeChecker.globalScope, source, "@test"); + freeze(typeChecker.globalTypes); + + REQUIRE_MESSAGE(result.success, "loadDefinition: unable to load definition file"); + return result; + } + else + { + return BaseType::loadDefinition(source); + } } const Position& getPosition(char marker) const @@ -2496,7 +2520,7 @@ local t = { CHECK(ac.entryMap.count("second")); } -TEST_CASE_FIXTURE(Fixture, "autocomplete_documentation_symbols") +TEST_CASE_FIXTURE(ACFixture, "autocomplete_documentation_symbols") { loadDefinition(R"( declare y: { @@ -2504,13 +2528,11 @@ TEST_CASE_FIXTURE(Fixture, "autocomplete_documentation_symbols") } )"); - fileResolver.source["Module/A"] = R"( - local a = y. - )"; + check(R"( + local a = y.@1 + )"); - frontend.check("Module/A"); - - auto ac = autocomplete(frontend, "Module/A", Position{1, 21}, nullCallback); + auto ac = autocomplete('1'); REQUIRE(ac.entryMap.count("x")); CHECK_EQ(ac.entryMap["x"].documentationSymbol, "@test/global/y.x"); @@ -2736,6 +2758,107 @@ TEST_CASE_FIXTURE(ACFixture, "autocomplete_on_string_singletons") CHECK(ac.entryMap.count("format")); } +TEST_CASE_FIXTURE(ACFixture, "autocomplete_string_singletons") +{ + ScopedFastFlag luauAutocompleteSingletonTypes{"LuauAutocompleteSingletonTypes", true}; + ScopedFastFlag luauExpectedTypesOfProperties{"LuauExpectedTypesOfProperties", true}; + + check(R"( + type tag = "cat" | "dog" + local function f(a: tag) end + f("@1") + f(@2) + local x: tag = "@3" + )"); + + auto ac = autocomplete('1'); + + CHECK(ac.entryMap.count("cat")); + CHECK(ac.entryMap.count("dog")); + + ac = autocomplete('2'); + + CHECK(ac.entryMap.count("\"cat\"")); + CHECK(ac.entryMap.count("\"dog\"")); + + ac = autocomplete('3'); + + CHECK(ac.entryMap.count("cat")); + CHECK(ac.entryMap.count("dog")); + + check(R"( + type tagged = {tag:"cat", fieldx:number} | {tag:"dog", fieldy:number} + local x: tagged = {tag="@4"} + )"); + + ac = autocomplete('4'); + + CHECK(ac.entryMap.count("cat")); + CHECK(ac.entryMap.count("dog")); +} + +TEST_CASE_FIXTURE(ACFixture, "autocomplete_string_singleton_equality") +{ + ScopedFastFlag luauAutocompleteSingletonTypes{"LuauAutocompleteSingletonTypes", true}; + + check(R"( + type tagged = {tag:"cat", fieldx:number} | {tag:"dog", fieldy:number} + local x: tagged = {tag="cat", fieldx=2} + if x.tag == "@1" or "@2" ~= x.tag then end + )"); + + auto ac = autocomplete('1'); + + CHECK(ac.entryMap.count("cat")); + CHECK(ac.entryMap.count("dog")); + + ac = autocomplete('2'); + + CHECK(ac.entryMap.count("cat")); + CHECK(ac.entryMap.count("dog")); + + // CLI-48823: assignment to x.tag should also autocomplete, but union l-values are not supported yet +} + +TEST_CASE_FIXTURE(ACFixture, "autocomplete_boolean_singleton") +{ + ScopedFastFlag luauAutocompleteSingletonTypes{"LuauAutocompleteSingletonTypes", true}; + + check(R"( +local function f(x: true) end +f(@1) + )"); + + auto ac = autocomplete('1'); + + REQUIRE(ac.entryMap.count("true")); + CHECK(ac.entryMap["true"].typeCorrect == TypeCorrectKind::Correct); + REQUIRE(ac.entryMap.count("false")); + CHECK(ac.entryMap["false"].typeCorrect == TypeCorrectKind::None); +} + +TEST_CASE_FIXTURE(ACFixture, "autocomplete_string_singleton_escape") +{ + ScopedFastFlag luauAutocompleteSingletonTypes{"LuauAutocompleteSingletonTypes", true}; + + check(R"( + type tag = "strange\t\"cat\"" | 'nice\t"dog"' + local function f(x: tag) end + f(@1) + f("@2") + )"); + + auto ac = autocomplete('1'); + + CHECK(ac.entryMap.count("\"strange\\t\\\"cat\\\"\"")); + CHECK(ac.entryMap.count("\"nice\\t\\\"dog\\\"\"")); + + ac = autocomplete('2'); + + CHECK(ac.entryMap.count("strange\\t\\\"cat\\\"")); + CHECK(ac.entryMap.count("nice\\t\\\"dog\\\"")); +} + TEST_CASE_FIXTURE(ACFixture, "function_in_assignment_has_parentheses_2") { check(R"( diff --git a/tests/Compiler.test.cpp b/tests/Compiler.test.cpp index 3dc57da0..83dad729 100644 --- a/tests/Compiler.test.cpp +++ b/tests/Compiler.test.cpp @@ -1074,6 +1074,39 @@ RETURN R1 1 )"); } +TEST_CASE("AndOrFoldLeft") +{ + // constant folding and/or expression is possible even if just the left hand is constant + CHECK_EQ("\n" + compileFunction0("local a = false if a and b then b() end"), R"( +RETURN R0 0 +)"); + + CHECK_EQ("\n" + compileFunction0("local a = true if a or b then b() end"), R"( +GETIMPORT R0 1 +CALL R0 0 0 +RETURN R0 0 +)"); + + // however, if right hand side is constant we can't constant fold the entire expression + // (note that we don't need to evaluate the right hand side, but we do need a branch) + CHECK_EQ("\n" + compileFunction0("local a = false if b and a then b() end"), R"( +GETIMPORT R0 1 +JUMPIFNOT R0 +4 +RETURN R0 0 +GETIMPORT R0 1 +CALL R0 0 0 +RETURN R0 0 +)"); + + CHECK_EQ("\n" + compileFunction0("local a = true if b or a then b() end"), R"( +GETIMPORT R0 1 +JUMPIF R0 +0 +GETIMPORT R0 1 +CALL R0 0 0 +RETURN R0 0 +)"); +} + TEST_CASE("AndOrChainCodegen") { const char* source = R"( diff --git a/tests/Fixture.cpp b/tests/Fixture.cpp index a7e7ea39..9dc9feee 100644 --- a/tests/Fixture.cpp +++ b/tests/Fixture.cpp @@ -83,7 +83,7 @@ std::optional TestFileResolver::getEnvironmentForModule(const Modul return std::nullopt; } -Fixture::Fixture(bool freeze) +Fixture::Fixture(bool freeze, bool prepareAutocomplete) : sff_DebugLuauFreezeArena("DebugLuauFreezeArena", freeze) , frontend(&fileResolver, &configResolver, {/* retainFullTypeGraphs= */ true}) , typeChecker(frontend.typeChecker) @@ -93,8 +93,11 @@ Fixture::Fixture(bool freeze) configResolver.defaultConfig.parseOptions.captureComments = true; registerBuiltinTypes(frontend.typeChecker); + if (prepareAutocomplete) + registerBuiltinTypes(frontend.typeCheckerForAutocomplete); registerTestTypes(); Luau::freeze(frontend.typeChecker.globalTypes); + Luau::freeze(frontend.typeCheckerForAutocomplete.globalTypes); Luau::setPrintLine([](auto s) {}); } diff --git a/tests/Fixture.h b/tests/Fixture.h index 4e45a952..0d1233bf 100644 --- a/tests/Fixture.h +++ b/tests/Fixture.h @@ -91,7 +91,7 @@ struct TestConfigResolver : ConfigResolver struct Fixture { - explicit Fixture(bool freeze = true); + explicit Fixture(bool freeze = true, bool prepareAutocomplete = false); ~Fixture(); // Throws Luau::ParseErrors if the parse fails. diff --git a/tests/Frontend.test.cpp b/tests/Frontend.test.cpp index 8a59acd1..9fc0a005 100644 --- a/tests/Frontend.test.cpp +++ b/tests/Frontend.test.cpp @@ -384,6 +384,70 @@ TEST_CASE_FIXTURE(FrontendFixture, "cycle_error_paths") CHECK_EQ(ce2->cycle[1], "game/Gui/Modules/A"); } +TEST_CASE_FIXTURE(FrontendFixture, "cycle_incremental_type_surface") +{ + ScopedFastFlag luauCyclicModuleTypeSurface{"LuauCyclicModuleTypeSurface", true}; + + fileResolver.source["game/A"] = R"( + return {hello = 2} + )"; + + CheckResult result = frontend.check("game/A"); + LUAU_REQUIRE_NO_ERRORS(result); + + fileResolver.source["game/A"] = R"( + local me = require(game.A) + return {hello = 2} + )"; + frontend.markDirty("game/A"); + + result = frontend.check("game/A"); + LUAU_REQUIRE_ERRORS(result); + + auto ty = requireType("game/A", "me"); + CHECK_EQ(toString(ty), "any"); +} + +TEST_CASE_FIXTURE(FrontendFixture, "cycle_incremental_type_surface_longer") +{ + ScopedFastFlag luauCyclicModuleTypeSurface{"LuauCyclicModuleTypeSurface", true}; + + fileResolver.source["game/A"] = R"( + return {mod_a = 2} + )"; + + CheckResult result = frontend.check("game/A"); + LUAU_REQUIRE_NO_ERRORS(result); + + fileResolver.source["game/B"] = R"( + local me = require(game.A) + return {mod_b = 4} + )"; + + result = frontend.check("game/B"); + LUAU_REQUIRE_NO_ERRORS(result); + + fileResolver.source["game/A"] = R"( + local me = require(game.B) + return {mod_a_prime = 3} + )"; + + frontend.markDirty("game/A"); + frontend.markDirty("game/B"); + + result = frontend.check("game/A"); + LUAU_REQUIRE_ERRORS(result); + + TypeId tyA = requireType("game/A", "me"); + CHECK_EQ(toString(tyA), "any"); + + result = frontend.check("game/B"); + LUAU_REQUIRE_ERRORS(result); + + TypeId tyB = requireType("game/B", "me"); + CHECK_EQ(toString(tyB), "any"); +} + TEST_CASE_FIXTURE(FrontendFixture, "dont_reparse_clean_file_when_linting") { fileResolver.source["Modules/A"] = R"( diff --git a/tests/Module.test.cpp b/tests/Module.test.cpp index 82b7a350..de063121 100644 --- a/tests/Module.test.cpp +++ b/tests/Module.test.cpp @@ -1,4 +1,5 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +#include "Luau/Clone.h" #include "Luau/Module.h" #include "Luau/Scope.h" diff --git a/tests/Parser.test.cpp b/tests/Parser.test.cpp index 7dacc669..79f9ecab 100644 --- a/tests/Parser.test.cpp +++ b/tests/Parser.test.cpp @@ -2022,6 +2022,15 @@ TEST_CASE_FIXTURE(Fixture, "parse_type_alias_default_type_errors") matchParseError("type Y number> = {}", "Expected type pack after '=', got type", Location{{0, 14}, {0, 32}}); } +TEST_CASE_FIXTURE(Fixture, "parse_type_pack_errors") +{ + ScopedFastFlag luauParseRecoverUnexpectedPack{"LuauParseRecoverUnexpectedPack", true}; + + matchParseError("type Y = {a: T..., b: number}", "Unexpected '...' after type name; type pack is not allowed in this context", + Location{{0, 20}, {0, 23}}); + matchParseError("type Y = {a: (number | string)...", "Unexpected '...' after type annotation", Location{{0, 36}, {0, 39}}); +} + TEST_CASE_FIXTURE(Fixture, "parse_if_else_expression") { { @@ -2590,4 +2599,16 @@ type Y = (T...) -> U... CHECK_EQ(1, result.errors.size()); } +TEST_CASE_FIXTURE(Fixture, "recover_unexpected_type_pack") +{ + ScopedFastFlag luauParseRecoverUnexpectedPack{"LuauParseRecoverUnexpectedPack", true}; + + ParseResult result = tryParse(R"( +type X = { a: T..., b: number } +type Y = { a: T..., b: number } +type Z = { a: string | T..., b: number } + )"); + REQUIRE_EQ(3, result.errors.size()); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.functions.test.cpp b/tests/TypeInfer.functions.test.cpp index dbae7b54..1713216a 100644 --- a/tests/TypeInfer.functions.test.cpp +++ b/tests/TypeInfer.functions.test.cpp @@ -1270,7 +1270,7 @@ caused by: TEST_CASE_FIXTURE(Fixture, "function_decl_quantify_right_type") { - ScopedFastFlag statFunctionSimplify{"LuauStatFunctionSimplify3", true}; + ScopedFastFlag statFunctionSimplify{"LuauStatFunctionSimplify4", true}; fileResolver.source["game/isAMagicMock"] = R"( --!nonstrict @@ -1294,7 +1294,7 @@ end TEST_CASE_FIXTURE(Fixture, "function_decl_non_self_sealed_overwrite") { - ScopedFastFlag statFunctionSimplify{"LuauStatFunctionSimplify3", true}; + ScopedFastFlag statFunctionSimplify{"LuauStatFunctionSimplify4", true}; CheckResult result = check(R"( function string.len(): number @@ -1316,7 +1316,7 @@ print(string.len('hello')) TEST_CASE_FIXTURE(Fixture, "function_decl_non_self_sealed_overwrite_2") { - ScopedFastFlag statFunctionSimplify{"LuauStatFunctionSimplify3", true}; + ScopedFastFlag statFunctionSimplify{"LuauStatFunctionSimplify4", true}; ScopedFastFlag inferStatFunction{"LuauInferStatFunction", true}; CheckResult result = check(R"( @@ -1324,12 +1324,12 @@ local t: { f: ((x: number) -> number)? } = {} function t.f(x) print(x + 5) - return x .. "asd" + return x .. "asd" -- 1st error: we know that return type is a number, not a string end t.f = function(x) print(x + 5) - return x .. "asd" + return x .. "asd" -- 2nd error: we know that return type is a number, not a string end )"); @@ -1338,6 +1338,33 @@ end CHECK_EQ(toString(result.errors[1]), R"(Type 'string' could not be converted into 'number')"); } +TEST_CASE_FIXTURE(Fixture, "function_decl_non_self_unsealed_overwrite") +{ + ScopedFastFlag statFunctionSimplify{"LuauStatFunctionSimplify4", true}; + ScopedFastFlag inferStatFunction{"LuauInferStatFunction", true}; + + CheckResult result = check(R"( +local t = { f = nil :: ((x: number) -> number)? } + +function t.f(x: string): string -- 1st error: new function value type is incompatible + return x .. "asd" +end + +t.f = function(x) + print(x + 5) + return x .. "asd" -- 2nd error: we know that return type is a number, not a string +end + )"); + + LUAU_REQUIRE_ERROR_COUNT(2, result); + CHECK_EQ(toString(result.errors[0]), R"(Type '(string) -> string' could not be converted into '((number) -> number)?' +caused by: + None of the union options are compatible. For example: Type '(string) -> string' could not be converted into '(number) -> number' +caused by: + Argument #1 type is not compatible. Type 'number' could not be converted into 'string')"); + CHECK_EQ(toString(result.errors[1]), R"(Type 'string' could not be converted into 'number')"); +} + TEST_CASE_FIXTURE(Fixture, "strict_mode_ok_with_missing_arguments") { ScopedFastFlag sff{"LuauAnyInIsOptionalIsOptional", true}; @@ -1352,7 +1379,7 @@ TEST_CASE_FIXTURE(Fixture, "strict_mode_ok_with_missing_arguments") TEST_CASE_FIXTURE(Fixture, "function_statement_sealed_table_assignment_through_indexer") { - ScopedFastFlag statFunctionSimplify{"LuauStatFunctionSimplify3", true}; + ScopedFastFlag statFunctionSimplify{"LuauStatFunctionSimplify4", true}; CheckResult result = check(R"( local t: {[string]: () -> number} = {} diff --git a/tests/TypeInfer.intersectionTypes.test.cpp b/tests/TypeInfer.intersectionTypes.test.cpp index d146f4e8..ac7a6532 100644 --- a/tests/TypeInfer.intersectionTypes.test.cpp +++ b/tests/TypeInfer.intersectionTypes.test.cpp @@ -311,6 +311,8 @@ TEST_CASE_FIXTURE(Fixture, "table_intersection_write_sealed") TEST_CASE_FIXTURE(Fixture, "table_intersection_write_sealed_indirect") { + ScopedFastFlag statFunctionSimplify{"LuauStatFunctionSimplify4", true}; + CheckResult result = check(R"( type X = { x: (number) -> number } type Y = { y: (string) -> string } @@ -326,10 +328,39 @@ TEST_CASE_FIXTURE(Fixture, "table_intersection_write_sealed_indirect") function xy:w(a:number) return a * 10 end )"); - LUAU_REQUIRE_ERROR_COUNT(3, result); - CHECK_EQ(toString(result.errors[0]), "Cannot add property 'z' to table 'X & Y'"); - CHECK_EQ(toString(result.errors[1]), "Cannot add property 'y' to table 'X & Y'"); - CHECK_EQ(toString(result.errors[2]), "Cannot add property 'w' to table 'X & Y'"); + LUAU_REQUIRE_ERROR_COUNT(4, result); + CHECK_EQ(toString(result.errors[0]), R"(Type '(string, number) -> string' could not be converted into '(string) -> string' +caused by: + Argument count mismatch. Function expects 2 arguments, but only 1 is specified)"); + CHECK_EQ(toString(result.errors[1]), "Cannot add property 'z' to table 'X & Y'"); + CHECK_EQ(toString(result.errors[2]), "Type 'number' could not be converted into 'string'"); + CHECK_EQ(toString(result.errors[3]), "Cannot add property 'w' to table 'X & Y'"); +} + +TEST_CASE_FIXTURE(Fixture, "table_write_sealed_indirect") +{ + ScopedFastFlag statFunctionSimplify{"LuauStatFunctionSimplify4", true}; + + // After normalization, previous 'table_intersection_write_sealed_indirect' is identical to this one + CheckResult result = check(R"( + type XY = { x: (number) -> number, y: (string) -> string } + + local xy : XY = { + x = function(a: number) return -a end, + y = function(a: string) return a .. "b" end + } + function xy.z(a:number) return a * 10 end + function xy:y(a:number) return a * 10 end + function xy:w(a:number) return a * 10 end + )"); + + LUAU_REQUIRE_ERROR_COUNT(4, result); + CHECK_EQ(toString(result.errors[0]), R"(Type '(string, number) -> string' could not be converted into '(string) -> string' +caused by: + Argument count mismatch. Function expects 2 arguments, but only 1 is specified)"); + CHECK_EQ(toString(result.errors[1]), "Cannot add property 'z' to table 'XY'"); + CHECK_EQ(toString(result.errors[2]), "Type 'number' could not be converted into 'string'"); + CHECK_EQ(toString(result.errors[3]), "Cannot add property 'w' to table 'XY'"); } TEST_CASE_FIXTURE(Fixture, "table_intersection_setmetatable") diff --git a/tests/TypeInfer.primitives.test.cpp b/tests/TypeInfer.primitives.test.cpp index 44b7b0d0..3ddf9813 100644 --- a/tests/TypeInfer.primitives.test.cpp +++ b/tests/TypeInfer.primitives.test.cpp @@ -95,6 +95,8 @@ end )"); LUAU_REQUIRE_ERROR_COUNT(2, result); + CHECK_EQ(toString(result.errors[0]), "Cannot add method to non-table type 'number'"); + CHECK_EQ(toString(result.errors[1]), "Type 'number' could not be converted into 'string'"); } TEST_SUITE_END(); diff --git a/tests/TypeInfer.tables.test.cpp b/tests/TypeInfer.tables.test.cpp index 0cc12d19..0484351d 100644 --- a/tests/TypeInfer.tables.test.cpp +++ b/tests/TypeInfer.tables.test.cpp @@ -2922,4 +2922,19 @@ TEST_CASE_FIXTURE(Fixture, "inferred_properties_of_a_table_should_start_with_the LUAU_REQUIRE_NO_ERRORS(result); } +TEST_CASE_FIXTURE(Fixture, "mixed_tables_with_implicit_numbered_keys") +{ + ScopedFastFlag sff{"LuauCheckImplicitNumbericKeys", true}; + + CheckResult result = check(R"( + local t: { [string]: number } = { 5, 6, 7 } + )"); + + LUAU_REQUIRE_ERROR_COUNT(3, result); + + CHECK_EQ("Type 'number' could not be converted into 'string'", toString(result.errors[0])); + CHECK_EQ("Type 'number' could not be converted into 'string'", toString(result.errors[1])); + CHECK_EQ("Type 'number' could not be converted into 'string'", toString(result.errors[2])); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.tryUnify.test.cpp b/tests/TypeInfer.tryUnify.test.cpp index d8de2594..c21e1625 100644 --- a/tests/TypeInfer.tryUnify.test.cpp +++ b/tests/TypeInfer.tryUnify.test.cpp @@ -242,4 +242,30 @@ TEST_CASE_FIXTURE(TryUnifyFixture, "cli_50320_follow_in_any_unification") state.tryUnify(&func, typeChecker.anyType); } +TEST_CASE_FIXTURE(TryUnifyFixture, "txnlog_preserves_type_owner") +{ + ScopedFastFlag luauTxnLogPreserveOwner{"LuauTxnLogPreserveOwner", true}; + + TypeId a = arena.addType(TypeVar{FreeTypeVar{TypeLevel{}}}); + TypeId b = typeChecker.numberType; + + state.tryUnify(a, b); + state.log.commit(); + + CHECK_EQ(a->owningArena, &arena); +} + +TEST_CASE_FIXTURE(TryUnifyFixture, "txnlog_preserves_pack_owner") +{ + ScopedFastFlag luauTxnLogPreserveOwner{"LuauTxnLogPreserveOwner", true}; + + TypePackId a = arena.addTypePack(TypePackVar{FreeTypePack{TypeLevel{}}}); + TypePackId b = typeChecker.anyTypePack; + + state.tryUnify(a, b); + state.log.commit(); + + CHECK_EQ(a->owningArena, &arena); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.unionTypes.test.cpp b/tests/TypeInfer.unionTypes.test.cpp index 68b7c4fb..ff207a18 100644 --- a/tests/TypeInfer.unionTypes.test.cpp +++ b/tests/TypeInfer.unionTypes.test.cpp @@ -513,4 +513,29 @@ TEST_CASE_FIXTURE(Fixture, "dont_allow_cyclic_unions_to_be_inferred") LUAU_REQUIRE_NO_ERRORS(result); } +TEST_CASE_FIXTURE(Fixture, "table_union_write_indirect") +{ + ScopedFastFlag statFunctionSimplify{"LuauStatFunctionSimplify4", true}; + + CheckResult result = check(R"( + type A = { x: number, y: (number) -> string } | { z: number, y: (number) -> string } + + local a:A = nil + + function a.y(x) + return tostring(x * 2) + end + + function a.y(x: string): number + return tonumber(x) or 0 + end + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + // NOTE: union normalization will improve this message + CHECK_EQ(toString(result.errors[0]), + R"(Type '(string) -> number' could not be converted into '((number) -> string) | ((number) -> string)'; none of the union options are compatible)"); +} + + TEST_SUITE_END(); diff --git a/tools/lldb_formatters.py b/tools/lldb_formatters.py index 40f8d6be..b3d2b4f5 100644 --- a/tools/lldb_formatters.py +++ b/tools/lldb_formatters.py @@ -37,7 +37,7 @@ def getType(target, typeName): return ty def luau_variant_summary(valobj, internal_dict, options): - type_id = valobj.GetChildMemberWithName("typeid").GetValueAsUnsigned() + type_id = valobj.GetChildMemberWithName("typeId").GetValueAsUnsigned() storage = valobj.GetChildMemberWithName("storage") params = templateParams(valobj.GetType().GetCanonicalType().GetName()) stored_type = params[type_id] @@ -89,7 +89,7 @@ class LuauVariantSyntheticChildrenProvider: return None def update(self): - self.type_index = self.valobj.GetChildMemberWithName("typeid").GetValueAsSigned() + self.type_index = self.valobj.GetChildMemberWithName("typeId").GetValueAsSigned() self.type_params = templateParams(self.valobj.GetType().GetCanonicalType().GetName()) if len(self.type_params) > self.type_index: From 510aed7d3ffcb509aaad264f6fd763c4b9c7f6b2 Mon Sep 17 00:00:00 2001 From: Lily Brown Date: Fri, 8 Apr 2022 11:26:47 -0700 Subject: [PATCH 220/399] Fix JsonEncoder for AstExprTable (#454) JsonEncoder wasn't producing valid JSON for `AstExprTable`s. This PR fixes it. The new output looks like ```json { "type": "AstStatBlock", "location": "0,0 - 6,4", "body": [ { "type": "AstStatLocal", "location": "1,8 - 5,9", "vars": [ { "name": "x", "location": "1,14 - 1,15" } ], "values": [ { "type": "AstExprTable", "location": "3,12 - 5,9", "items": [ { "kind": "record", "key": { "type": "AstExprConstantString", "location": "4,12 - 4,15", "value": "foo" }, "value": { "type": "AstExprConstantNumber", "location": "4,18 - 4,21", "value": 123 } } ] } ] } ] } ``` --- Analysis/src/JsonEncoder.cpp | 21 ++++++--------------- tests/JsonEncoder.test.cpp | 21 +++++++++++++++++++++ 2 files changed, 27 insertions(+), 15 deletions(-) diff --git a/Analysis/src/JsonEncoder.cpp b/Analysis/src/JsonEncoder.cpp index 811e7c24..829ffa02 100644 --- a/Analysis/src/JsonEncoder.cpp +++ b/Analysis/src/JsonEncoder.cpp @@ -403,35 +403,26 @@ struct AstJsonEncoder : public AstVisitor void write(const AstExprTable::Item& item) { writeRaw("{"); - bool comma = pushComma(); + bool c = pushComma(); write("kind", item.kind); switch (item.kind) { case AstExprTable::Item::List: - write(item.value); + write("value", item.value); break; default: - write(item.key); - writeRaw(","); - write(item.value); + write("key", item.key); + write("value", item.value); break; } - popComma(comma); + popComma(c); writeRaw("}"); } void write(class AstExprTable* node) { writeNode(node, "AstExprTable", [&]() { - bool comma = false; - for (const auto& prop : node->items) - { - if (comma) - writeRaw(","); - else - comma = true; - write(prop); - } + PROP(items); }); } diff --git a/tests/JsonEncoder.test.cpp b/tests/JsonEncoder.test.cpp index cb508072..6711d979 100644 --- a/tests/JsonEncoder.test.cpp +++ b/tests/JsonEncoder.test.cpp @@ -1,6 +1,7 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #include "Luau/Ast.h" #include "Luau/JsonEncoder.h" +#include "Luau/Parser.h" #include "doctest.h" @@ -50,4 +51,24 @@ TEST_CASE("encode_AstStatBlock") toJson(&block)); } +TEST_CASE("encode_tables") +{ + std::string src = R"( + local x: { + foo: number + } = { + foo = 123, + } + )"; + + Allocator allocator; + AstNameTable names(allocator); + ParseResult parseResult = Parser::parse(src.c_str(), src.length(), names, allocator); + + REQUIRE(parseResult.errors.size() == 0); + std::string json = toJson(parseResult.root); + + CHECK(json == R"({"type":"AstStatBlock","location":"0,0 - 6,4","body":[{"type":"AstStatLocal","location":"1,8 - 5,9","vars":[{"type":{"type":"AstTypeTable","location":"1,17 - 3,9","props":[{"name":"foo","location":"2,12 - 2,15","type":{"type":"AstTypeReference","location":"2,17 - 2,23","name":"number","parameters":[]}}],"indexer":false},"name":"x","location":"1,14 - 1,15"}],"values":[{"type":"AstExprTable","location":"3,12 - 5,9","items":[{"kind":"record","key":{"type":"AstExprConstantString","location":"4,12 - 4,15","value":"foo"},"value":{"type":"AstExprConstantNumber","location":"4,18 - 4,21","value":123}}]}]}]})"); +} + TEST_SUITE_END(); From d37d0c857ba543ea47f0b8fce5678f7aadf5239e Mon Sep 17 00:00:00 2001 From: Alan Jeffrey <403333+asajeffrey@users.noreply.github.com> Date: Sat, 9 Apr 2022 00:07:08 -0500 Subject: [PATCH 221/399] Prototype: Renamed any/none to unknown/never (#447) * Renamed any/none to unknown/never * Pin hackage version * Update Agda version --- .github/workflows/prototyping.yml | 10 +- prototyping/Luau/StrictMode.agda | 2 +- prototyping/Luau/Subtyping.agda | 6 +- prototyping/Luau/Type.agda | 104 +++++++-------- prototyping/Luau/Type/FromJSON.agda | 6 +- prototyping/Luau/Type/ToString.agda | 6 +- prototyping/Luau/TypeCheck.agda | 14 +- prototyping/Properties/StrictMode.agda | 62 ++++----- prototyping/Properties/Subtyping.agda | 122 +++++++++--------- prototyping/Properties/TypeCheck.agda | 20 +-- .../Tests/PrettyPrinter/smoke_test/out.txt | 6 +- 11 files changed, 181 insertions(+), 177 deletions(-) diff --git a/.github/workflows/prototyping.yml b/.github/workflows/prototyping.yml index 6bc8a81b..ff66881d 100644 --- a/.github/workflows/prototyping.yml +++ b/.github/workflows/prototyping.yml @@ -10,7 +10,9 @@ jobs: linux: strategy: matrix: - agda: [2.6.2.1] + agda: [2.6.2.2] + hackageDate: ["2022-04-07"] + hackageTime: ["23:06:28"] name: prototyping runs-on: ubuntu-latest steps: @@ -18,7 +20,7 @@ jobs: - uses: actions/cache@v2 with: path: ~/.cabal/store - key: prototyping-${{ runner.os }}-${{ matrix.agda }} + key: "prototyping-${{ runner.os }}-${{ matrix.agda }}-${{ matrix.hackageDate }}-${{ matrix.hackageTime }}" - uses: actions/cache@v2 id: luau-ast-cache with: @@ -28,12 +30,12 @@ jobs: run: sudo apt-get install -y cabal-install - name: cabal update working-directory: prototyping - run: cabal update + run: cabal v2-update "hackage.haskell.org,${{ matrix.hackageDate }}T${{ matrix.hackageTime }}Z" - name: cabal install working-directory: prototyping run: | - cabal install Agda-${{ matrix.agda }} cabal install --lib scientific vector aeson --package-env . + cabal install --allow-newer Agda-${{ matrix.agda }} - name: check targets working-directory: prototyping run: | diff --git a/prototyping/Luau/StrictMode.agda b/prototyping/Luau/StrictMode.agda index 0b5fe0da..b6769f01 100644 --- a/prototyping/Luau/StrictMode.agda +++ b/prototyping/Luau/StrictMode.agda @@ -5,7 +5,7 @@ module Luau.StrictMode where open import Agda.Builtin.Equality using (_≡_) open import FFI.Data.Maybe using (just; nothing) open import Luau.Syntax using (Expr; Stat; Block; BinaryOperator; yes; nil; addr; var; binexp; var_∈_; _⟨_⟩∈_; function_is_end; _$_; block_is_end; local_←_; _∙_; done; return; name; +; -; *; /; <; >; <=; >=; ··) -open import Luau.Type using (Type; strict; nil; number; string; boolean; none; any; _⇒_; _∪_; _∩_; tgt) +open import Luau.Type using (Type; strict; nil; number; string; boolean; _⇒_; _∪_; _∩_; tgt) open import Luau.Subtyping using (_≮:_) open import Luau.Heap using (Heap; function_is_end) renaming (_[_] to _[_]ᴴ) open import Luau.VarCtxt using (VarCtxt; ∅; _⋒_; _↦_; _⊕_↦_; _⊝_) renaming (_[_] to _[_]ⱽ) diff --git a/prototyping/Luau/Subtyping.agda b/prototyping/Luau/Subtyping.agda index 7d67eb43..943f459b 100644 --- a/prototyping/Luau/Subtyping.agda +++ b/prototyping/Luau/Subtyping.agda @@ -1,6 +1,6 @@ {-# OPTIONS --rewriting #-} -open import Luau.Type using (Type; Scalar; nil; number; string; boolean; none; any; _⇒_; _∪_; _∩_) +open import Luau.Type using (Type; Scalar; nil; number; string; boolean; never; unknown; _⇒_; _∪_; _∩_) open import Properties.Equality using (_≢_) module Luau.Subtyping where @@ -29,7 +29,7 @@ data Language where left : ∀ {T U t} → Language T t → Language (T ∪ U) t right : ∀ {T U u} → Language U u → Language (T ∪ U) u _,_ : ∀ {T U t} → Language T t → Language U t → Language (T ∩ U) t - any : ∀ {t} → Language any t + unknown : ∀ {t} → Language unknown t data ¬Language where @@ -42,7 +42,7 @@ data ¬Language where _,_ : ∀ {T U t} → ¬Language T t → ¬Language U t → ¬Language (T ∪ U) t left : ∀ {T U t} → ¬Language T t → ¬Language (T ∩ U) t right : ∀ {T U u} → ¬Language U u → ¬Language (T ∩ U) u - none : ∀ {t} → ¬Language none t + never : ∀ {t} → ¬Language never t -- Subtyping as language inclusion diff --git a/prototyping/Luau/Type.agda b/prototyping/Luau/Type.agda index 87815ddb..30c45388 100644 --- a/prototyping/Luau/Type.agda +++ b/prototyping/Luau/Type.agda @@ -9,8 +9,8 @@ open import FFI.Data.Maybe using (Maybe; just; nothing) data Type : Set where nil : Type _⇒_ : Type → Type → Type - none : Type - any : Type + never : Type + unknown : Type boolean : Type number : Type string : Type @@ -29,8 +29,8 @@ lhs (T ⇒ _) = T lhs (T ∪ _) = T lhs (T ∩ _) = T lhs nil = nil -lhs none = none -lhs any = any +lhs never = never +lhs unknown = unknown lhs number = number lhs boolean = boolean lhs string = string @@ -40,8 +40,8 @@ rhs (_ ⇒ T) = T rhs (_ ∪ T) = T rhs (_ ∩ T) = T rhs nil = nil -rhs none = none -rhs any = any +rhs never = never +rhs unknown = unknown rhs number = number rhs boolean = boolean rhs string = string @@ -49,16 +49,16 @@ rhs string = string _≡ᵀ_ : ∀ (T U : Type) → Dec(T ≡ U) nil ≡ᵀ nil = yes refl nil ≡ᵀ (S ⇒ T) = no (λ ()) -nil ≡ᵀ none = no (λ ()) -nil ≡ᵀ any = no (λ ()) +nil ≡ᵀ never = no (λ ()) +nil ≡ᵀ unknown = no (λ ()) nil ≡ᵀ number = no (λ ()) nil ≡ᵀ boolean = no (λ ()) nil ≡ᵀ (S ∪ T) = no (λ ()) nil ≡ᵀ (S ∩ T) = no (λ ()) nil ≡ᵀ string = no (λ ()) (S ⇒ T) ≡ᵀ string = no (λ ()) -none ≡ᵀ string = no (λ ()) -any ≡ᵀ string = no (λ ()) +never ≡ᵀ string = no (λ ()) +unknown ≡ᵀ string = no (λ ()) boolean ≡ᵀ string = no (λ ()) number ≡ᵀ string = no (λ ()) (S ∪ T) ≡ᵀ string = no (λ ()) @@ -68,48 +68,48 @@ number ≡ᵀ string = no (λ ()) (S ⇒ T) ≡ᵀ (S ⇒ T) | yes refl | yes refl = yes refl (S ⇒ T) ≡ᵀ (U ⇒ V) | _ | no p = no (λ q → p (cong rhs q)) (S ⇒ T) ≡ᵀ (U ⇒ V) | no p | _ = no (λ q → p (cong lhs q)) -(S ⇒ T) ≡ᵀ none = no (λ ()) -(S ⇒ T) ≡ᵀ any = no (λ ()) +(S ⇒ T) ≡ᵀ never = no (λ ()) +(S ⇒ T) ≡ᵀ unknown = no (λ ()) (S ⇒ T) ≡ᵀ number = no (λ ()) (S ⇒ T) ≡ᵀ boolean = no (λ ()) (S ⇒ T) ≡ᵀ (U ∪ V) = no (λ ()) (S ⇒ T) ≡ᵀ (U ∩ V) = no (λ ()) -none ≡ᵀ nil = no (λ ()) -none ≡ᵀ (U ⇒ V) = no (λ ()) -none ≡ᵀ none = yes refl -none ≡ᵀ any = no (λ ()) -none ≡ᵀ number = no (λ ()) -none ≡ᵀ boolean = no (λ ()) -none ≡ᵀ (U ∪ V) = no (λ ()) -none ≡ᵀ (U ∩ V) = no (λ ()) -any ≡ᵀ nil = no (λ ()) -any ≡ᵀ (U ⇒ V) = no (λ ()) -any ≡ᵀ none = no (λ ()) -any ≡ᵀ any = yes refl -any ≡ᵀ number = no (λ ()) -any ≡ᵀ boolean = no (λ ()) -any ≡ᵀ (U ∪ V) = no (λ ()) -any ≡ᵀ (U ∩ V) = no (λ ()) +never ≡ᵀ nil = no (λ ()) +never ≡ᵀ (U ⇒ V) = no (λ ()) +never ≡ᵀ never = yes refl +never ≡ᵀ unknown = no (λ ()) +never ≡ᵀ number = no (λ ()) +never ≡ᵀ boolean = no (λ ()) +never ≡ᵀ (U ∪ V) = no (λ ()) +never ≡ᵀ (U ∩ V) = no (λ ()) +unknown ≡ᵀ nil = no (λ ()) +unknown ≡ᵀ (U ⇒ V) = no (λ ()) +unknown ≡ᵀ never = no (λ ()) +unknown ≡ᵀ unknown = yes refl +unknown ≡ᵀ number = no (λ ()) +unknown ≡ᵀ boolean = no (λ ()) +unknown ≡ᵀ (U ∪ V) = no (λ ()) +unknown ≡ᵀ (U ∩ V) = no (λ ()) number ≡ᵀ nil = no (λ ()) number ≡ᵀ (T ⇒ U) = no (λ ()) -number ≡ᵀ none = no (λ ()) -number ≡ᵀ any = no (λ ()) +number ≡ᵀ never = no (λ ()) +number ≡ᵀ unknown = no (λ ()) number ≡ᵀ number = yes refl number ≡ᵀ boolean = no (λ ()) number ≡ᵀ (T ∪ U) = no (λ ()) number ≡ᵀ (T ∩ U) = no (λ ()) boolean ≡ᵀ nil = no (λ ()) boolean ≡ᵀ (T ⇒ U) = no (λ ()) -boolean ≡ᵀ none = no (λ ()) -boolean ≡ᵀ any = no (λ ()) +boolean ≡ᵀ never = no (λ ()) +boolean ≡ᵀ unknown = no (λ ()) boolean ≡ᵀ boolean = yes refl boolean ≡ᵀ number = no (λ ()) boolean ≡ᵀ (T ∪ U) = no (λ ()) boolean ≡ᵀ (T ∩ U) = no (λ ()) string ≡ᵀ nil = no (λ ()) string ≡ᵀ (x ⇒ x₁) = no (λ ()) -string ≡ᵀ none = no (λ ()) -string ≡ᵀ any = no (λ ()) +string ≡ᵀ never = no (λ ()) +string ≡ᵀ unknown = no (λ ()) string ≡ᵀ boolean = no (λ ()) string ≡ᵀ number = no (λ ()) string ≡ᵀ string = yes refl @@ -117,8 +117,8 @@ string ≡ᵀ (U ∪ V) = no (λ ()) string ≡ᵀ (U ∩ V) = no (λ ()) (S ∪ T) ≡ᵀ nil = no (λ ()) (S ∪ T) ≡ᵀ (U ⇒ V) = no (λ ()) -(S ∪ T) ≡ᵀ none = no (λ ()) -(S ∪ T) ≡ᵀ any = no (λ ()) +(S ∪ T) ≡ᵀ never = no (λ ()) +(S ∪ T) ≡ᵀ unknown = no (λ ()) (S ∪ T) ≡ᵀ number = no (λ ()) (S ∪ T) ≡ᵀ boolean = no (λ ()) (S ∪ T) ≡ᵀ (U ∪ V) with (S ≡ᵀ U) | (T ≡ᵀ V) @@ -128,8 +128,8 @@ string ≡ᵀ (U ∩ V) = no (λ ()) (S ∪ T) ≡ᵀ (U ∩ V) = no (λ ()) (S ∩ T) ≡ᵀ nil = no (λ ()) (S ∩ T) ≡ᵀ (U ⇒ V) = no (λ ()) -(S ∩ T) ≡ᵀ none = no (λ ()) -(S ∩ T) ≡ᵀ any = no (λ ()) +(S ∩ T) ≡ᵀ never = no (λ ()) +(S ∩ T) ≡ᵀ unknown = no (λ ()) (S ∩ T) ≡ᵀ number = no (λ ()) (S ∩ T) ≡ᵀ boolean = no (λ ()) (S ∩ T) ≡ᵀ (U ∪ V) = no (λ ()) @@ -151,29 +151,29 @@ data Mode : Set where nonstrict : Mode src : Mode → Type → Type -src m nil = none -src m number = none -src m boolean = none -src m string = none +src m nil = never +src m number = never +src m boolean = never +src m string = never src m (S ⇒ T) = S -- In nonstrict mode, functions are covaraiant, in strict mode they're contravariant src strict (S ∪ T) = (src strict S) ∩ (src strict T) src nonstrict (S ∪ T) = (src nonstrict S) ∪ (src nonstrict T) src strict (S ∩ T) = (src strict S) ∪ (src strict T) src nonstrict (S ∩ T) = (src nonstrict S) ∩ (src nonstrict T) -src strict none = any -src nonstrict none = none -src strict any = none -src nonstrict any = any +src strict never = unknown +src nonstrict never = never +src strict unknown = never +src nonstrict unknown = unknown tgt : Type → Type -tgt nil = none +tgt nil = never tgt (S ⇒ T) = T -tgt none = none -tgt any = any -tgt number = none -tgt boolean = none -tgt string = none +tgt never = never +tgt unknown = unknown +tgt number = never +tgt boolean = never +tgt string = never tgt (S ∪ T) = (tgt S) ∪ (tgt T) tgt (S ∩ T) = (tgt S) ∩ (tgt T) diff --git a/prototyping/Luau/Type/FromJSON.agda b/prototyping/Luau/Type/FromJSON.agda index 2d6ba689..e3d1e8e7 100644 --- a/prototyping/Luau/Type/FromJSON.agda +++ b/prototyping/Luau/Type/FromJSON.agda @@ -2,7 +2,7 @@ module Luau.Type.FromJSON where -open import Luau.Type using (Type; nil; _⇒_; _∪_; _∩_; any; number; string) +open import Luau.Type using (Type; nil; _⇒_; _∪_; _∩_; unknown; never; number; string) open import Agda.Builtin.List using (List; _∷_; []) open import Agda.Builtin.Bool using (true; false) @@ -42,7 +42,9 @@ typeFromJSON (object o) | just (string "AstTypeFunction") | nothing | nothing = typeFromJSON (object o) | just (string "AstTypeReference") with lookup name o typeFromJSON (object o) | just (string "AstTypeReference") | just (string "nil") = Right nil -typeFromJSON (object o) | just (string "AstTypeReference") | just (string "any") = Right any +typeFromJSON (object o) | just (string "AstTypeReference") | just (string "any") = Right unknown -- not quite right +typeFromJSON (object o) | just (string "AstTypeReference") | just (string "unknown") = Right unknown +typeFromJSON (object o) | just (string "AstTypeReference") | just (string "never") = Right never typeFromJSON (object o) | just (string "AstTypeReference") | just (string "number") = Right number typeFromJSON (object o) | just (string "AstTypeReference") | just (string "string") = Right string typeFromJSON (object o) | just (string "AstTypeReference") | _ = Left "Unknown referenced type" diff --git a/prototyping/Luau/Type/ToString.agda b/prototyping/Luau/Type/ToString.agda index 2efe6632..a41ecec2 100644 --- a/prototyping/Luau/Type/ToString.agda +++ b/prototyping/Luau/Type/ToString.agda @@ -1,7 +1,7 @@ module Luau.Type.ToString where open import FFI.Data.String using (String; _++_) -open import Luau.Type using (Type; nil; _⇒_; none; any; number; boolean; string; _∪_; _∩_; normalizeOptional) +open import Luau.Type using (Type; nil; _⇒_; never; unknown; number; boolean; string; _∪_; _∩_; normalizeOptional) {-# TERMINATING #-} typeToString : Type → String @@ -10,8 +10,8 @@ typeToStringᴵ : Type → String typeToString nil = "nil" typeToString (S ⇒ T) = "(" ++ (typeToString S) ++ ") -> " ++ (typeToString T) -typeToString none = "none" -typeToString any = "any" +typeToString never = "never" +typeToString unknown = "unknown" typeToString number = "number" typeToString boolean = "boolean" typeToString string = "string" diff --git a/prototyping/Luau/TypeCheck.agda b/prototyping/Luau/TypeCheck.agda index c22618bc..aea6507a 100644 --- a/prototyping/Luau/TypeCheck.agda +++ b/prototyping/Luau/TypeCheck.agda @@ -10,7 +10,7 @@ open import Luau.Syntax using (Expr; Stat; Block; BinaryOperator; yes; nil; addr open import Luau.Var using (Var) open import Luau.Addr using (Addr) open import Luau.Heap using (Heap; Object; function_is_end) renaming (_[_] to _[_]ᴴ) -open import Luau.Type using (Type; Mode; nil; any; number; boolean; string; _⇒_; tgt) +open import Luau.Type using (Type; Mode; nil; unknown; number; boolean; string; _⇒_; tgt) open import Luau.VarCtxt using (VarCtxt; ∅; _⋒_; _↦_; _⊕_↦_; _⊝_) renaming (_[_] to _[_]ⱽ) open import FFI.Data.Vector using (Vector) open import FFI.Data.Maybe using (Maybe; just; nothing) @@ -19,9 +19,9 @@ open import Properties.Product using (_×_; _,_) src : Type → Type src = Luau.Type.src m -orAny : Maybe Type → Type -orAny nothing = any -orAny (just T) = T +orUnknown : Maybe Type → Type +orUnknown nothing = unknown +orUnknown (just T) = T srcBinOp : BinaryOperator → Type srcBinOp + = number @@ -30,8 +30,8 @@ srcBinOp * = number srcBinOp / = number srcBinOp < = number srcBinOp > = number -srcBinOp == = any -srcBinOp ~= = any +srcBinOp == = unknown +srcBinOp ~= = unknown srcBinOp <= = number srcBinOp >= = number srcBinOp ·· = string @@ -89,7 +89,7 @@ data _⊢ᴱ_∈_ where var : ∀ {x T Γ} → - T ≡ orAny(Γ [ x ]ⱽ) → + T ≡ orUnknown(Γ [ x ]ⱽ) → ---------------- Γ ⊢ᴱ (var x) ∈ T diff --git a/prototyping/Properties/StrictMode.agda b/prototyping/Properties/StrictMode.agda index 2ff2b153..1165fdaa 100644 --- a/prototyping/Properties/StrictMode.agda +++ b/prototyping/Properties/StrictMode.agda @@ -9,10 +9,10 @@ open import FFI.Data.Maybe using (Maybe; just; nothing) open import Luau.Heap using (Heap; Object; function_is_end; defn; alloc; ok; next; lookup-not-allocated) renaming (_≡_⊕_↦_ to _≡ᴴ_⊕_↦_; _[_] to _[_]ᴴ; ∅ to ∅ᴴ) open import Luau.StrictMode using (Warningᴱ; Warningᴮ; Warningᴼ; Warningᴴ; UnallocatedAddress; UnboundVariable; FunctionCallMismatch; app₁; app₂; BinOpMismatch₁; BinOpMismatch₂; bin₁; bin₂; BlockMismatch; block₁; return; LocalVarMismatch; local₁; local₂; FunctionDefnMismatch; function₁; function₂; heap; expr; block; addr) open import Luau.Substitution using (_[_/_]ᴮ; _[_/_]ᴱ; _[_/_]ᴮunless_; var_[_/_]ᴱwhenever_) -open import Luau.Subtyping using (_≮:_; witness; any; none; scalar; function; scalar-function; scalar-function-ok; scalar-function-err; scalar-scalar; function-scalar; function-ok; function-err; left; right; _,_; Tree; Language; ¬Language) +open import Luau.Subtyping using (_≮:_; witness; unknown; never; scalar; function; scalar-function; scalar-function-ok; scalar-function-err; scalar-scalar; function-scalar; function-ok; function-err; left; right; _,_; Tree; Language; ¬Language) open import Luau.Syntax using (Expr; yes; var; val; var_∈_; _⟨_⟩∈_; _$_; addr; number; bool; string; binexp; nil; function_is_end; block_is_end; done; return; local_←_; _∙_; fun; arg; name; ==; ~=) -open import Luau.Type using (Type; strict; nil; number; boolean; string; _⇒_; none; any; _∩_; _∪_; tgt; _≡ᵀ_; _≡ᴹᵀ_) -open import Luau.TypeCheck(strict) using (_⊢ᴮ_∈_; _⊢ᴱ_∈_; _⊢ᴴᴮ_▷_∈_; _⊢ᴴᴱ_▷_∈_; nil; var; addr; app; function; block; done; return; local; orAny; srcBinOp; tgtBinOp) +open import Luau.Type using (Type; strict; nil; number; boolean; string; _⇒_; never; unknown; _∩_; _∪_; tgt; _≡ᵀ_; _≡ᴹᵀ_) +open import Luau.TypeCheck(strict) using (_⊢ᴮ_∈_; _⊢ᴱ_∈_; _⊢ᴴᴮ_▷_∈_; _⊢ᴴᴱ_▷_∈_; nil; var; addr; app; function; block; done; return; local; orUnknown; srcBinOp; tgtBinOp) open import Luau.Var using (_≡ⱽ_) open import Luau.Addr using (_≡ᴬ_) open import Luau.VarCtxt using (VarCtxt; ∅; _⋒_; _↦_; _⊕_↦_; _⊝_; ⊕-lookup-miss; ⊕-swap; ⊕-over) renaming (_[_] to _[_]ⱽ) @@ -22,7 +22,7 @@ open import Properties.Equality using (_≢_; sym; cong; trans; subst₁) open import Properties.Dec using (Dec; yes; no) open import Properties.Contradiction using (CONTRADICTION; ¬) open import Properties.Functions using (_∘_) -open import Properties.Subtyping using (any-≮:; ≡-trans-≮:; ≮:-trans-≡; none-tgt-≮:; tgt-none-≮:; src-any-≮:; any-src-≮:; ≮:-trans; ≮:-refl; scalar-≢-impl-≮:; function-≮:-scalar; scalar-≮:-function; function-≮:-none; any-≮:-scalar; scalar-≮:-none; any-≮:-none) +open import Properties.Subtyping using (unknown-≮:; ≡-trans-≮:; ≮:-trans-≡; never-tgt-≮:; tgt-never-≮:; src-unknown-≮:; unknown-src-≮:; ≮:-trans; ≮:-refl; scalar-≢-impl-≮:; function-≮:-scalar; scalar-≮:-function; function-≮:-never; unknown-≮:-scalar; scalar-≮:-never; unknown-≮:-never) open import Properties.TypeCheck(strict) using (typeOfᴼ; typeOfᴹᴼ; typeOfⱽ; typeOfᴱ; typeOfᴮ; typeCheckᴱ; typeCheckᴮ; typeCheckᴼ; typeCheckᴴ) open import Luau.OpSem using (_⟦_⟧_⟶_; _⊢_⟶*_⊣_; _⊢_⟶ᴮ_⊣_; _⊢_⟶ᴱ_⊣_; app₁; app₂; function; beta; return; block; done; local; subst; binOp₀; binOp₁; binOp₂; refl; step; +; -; *; /; <; >; ==; ~=; <=; >=; ··) open import Luau.RuntimeError using (BinOpError; RuntimeErrorᴱ; RuntimeErrorᴮ; FunctionMismatch; BinOpMismatch₁; BinOpMismatch₂; UnboundVariable; SEGV; app₁; app₂; bin₁; bin₂; block; local; return; +; -; *; /; <; >; <=; >=; ··) @@ -68,12 +68,12 @@ heap-weakeningᴱ Γ H (var x) h p = p heap-weakeningᴱ Γ H (val nil) h p = p heap-weakeningᴱ Γ H (val (addr a)) refl p = p heap-weakeningᴱ Γ H (val (addr a)) (snoc {a = b} q) p with a ≡ᴬ b -heap-weakeningᴱ Γ H (val (addr a)) (snoc {a = a} defn) p | yes refl = any-≮: p -heap-weakeningᴱ Γ H (val (addr a)) (snoc {a = b} q) p | no r = ≡-trans-≮: (cong orAny (cong typeOfᴹᴼ (lookup-not-allocated q r))) p +heap-weakeningᴱ Γ H (val (addr a)) (snoc {a = a} defn) p | yes refl = unknown-≮: p +heap-weakeningᴱ Γ H (val (addr a)) (snoc {a = b} q) p | no r = ≡-trans-≮: (cong orUnknown (cong typeOfᴹᴼ (lookup-not-allocated q r))) p heap-weakeningᴱ Γ H (val (number x)) h p = p heap-weakeningᴱ Γ H (val (bool x)) h p = p heap-weakeningᴱ Γ H (val (string x)) h p = p -heap-weakeningᴱ Γ H (M $ N) h p = none-tgt-≮: (heap-weakeningᴱ Γ H M h (tgt-none-≮: p)) +heap-weakeningᴱ Γ H (M $ N) h p = never-tgt-≮: (heap-weakeningᴱ Γ H M h (tgt-never-≮: p)) heap-weakeningᴱ Γ H (function f ⟨ var x ∈ T ⟩∈ U is B end) h p = p heap-weakeningᴱ Γ H (block var b ∈ T is B end) h p = p heap-weakeningᴱ Γ H (binexp M op N) h p = p @@ -94,11 +94,11 @@ substitutivityᴮ-unless-no : ∀ {Γ Γ′ T V} H B v x y (r : x ≢ y) → (Γ substitutivityᴱ H (var y) v x p = substitutivityᴱ-whenever H v x y (x ≡ⱽ y) p substitutivityᴱ H (val w) v x p = Left p substitutivityᴱ H (binexp M op N) v x p = Left p -substitutivityᴱ H (M $ N) v x p = mapL none-tgt-≮: (substitutivityᴱ H M v x (tgt-none-≮: p)) +substitutivityᴱ H (M $ N) v x p = mapL never-tgt-≮: (substitutivityᴱ H M v x (tgt-never-≮: p)) substitutivityᴱ H (function f ⟨ var y ∈ T ⟩∈ U is B end) v x p = Left p substitutivityᴱ H (block var b ∈ T is B end) v x p = Left p substitutivityᴱ-whenever H v x x (yes refl) q = swapLR (≮:-trans q) -substitutivityᴱ-whenever H v x y (no p) q = Left (≡-trans-≮: (cong orAny (sym (⊕-lookup-miss x y _ _ p))) q) +substitutivityᴱ-whenever H v x y (no p) q = Left (≡-trans-≮: (cong orUnknown (sym (⊕-lookup-miss x y _ _ p))) q) substitutivityᴮ H (function f ⟨ var y ∈ T ⟩∈ U is C end ∙ B) v x p = substitutivityᴮ-unless H B v x f (x ≡ⱽ f) p substitutivityᴮ H (local var y ∈ T ← M ∙ B) v x p = substitutivityᴮ-unless H B v x y (x ≡ⱽ y) p @@ -125,9 +125,9 @@ binOpPreservation H (·· v w) = refl reflect-subtypingᴱ : ∀ H M {H′ M′ T} → (H ⊢ M ⟶ᴱ M′ ⊣ H′) → (typeOfᴱ H′ ∅ M′ ≮: T) → Either (typeOfᴱ H ∅ M ≮: T) (Warningᴱ H (typeCheckᴱ H ∅ M)) reflect-subtypingᴮ : ∀ H B {H′ B′ T} → (H ⊢ B ⟶ᴮ B′ ⊣ H′) → (typeOfᴮ H′ ∅ B′ ≮: T) → Either (typeOfᴮ H ∅ B ≮: T) (Warningᴮ H (typeCheckᴮ H ∅ B)) -reflect-subtypingᴱ H (M $ N) (app₁ s) p = mapLR none-tgt-≮: app₁ (reflect-subtypingᴱ H M s (tgt-none-≮: p)) -reflect-subtypingᴱ H (M $ N) (app₂ v s) p = Left (none-tgt-≮: (heap-weakeningᴱ ∅ H M (rednᴱ⊑ s) (tgt-none-≮: p))) -reflect-subtypingᴱ H (M $ N) (beta (function f ⟨ var y ∈ T ⟩∈ U is B end) v refl q) p = Left (≡-trans-≮: (cong tgt (cong orAny (cong typeOfᴹᴼ q))) p) +reflect-subtypingᴱ H (M $ N) (app₁ s) p = mapLR never-tgt-≮: app₁ (reflect-subtypingᴱ H M s (tgt-never-≮: p)) +reflect-subtypingᴱ H (M $ N) (app₂ v s) p = Left (never-tgt-≮: (heap-weakeningᴱ ∅ H M (rednᴱ⊑ s) (tgt-never-≮: p))) +reflect-subtypingᴱ H (M $ N) (beta (function f ⟨ var y ∈ T ⟩∈ U is B end) v refl q) p = Left (≡-trans-≮: (cong tgt (cong orUnknown (cong typeOfᴹᴼ q))) p) reflect-subtypingᴱ H (function f ⟨ var x ∈ T ⟩∈ U is B end) (function a defn) p = Left p reflect-subtypingᴱ H (block var b ∈ T is B end) (block s) p = Left p reflect-subtypingᴱ H (block var b ∈ T is return (val v) ∙ B end) (return v) p = mapR BlockMismatch (swapLR (≮:-trans p)) @@ -152,8 +152,8 @@ reflect-substitutionᴱ H (var y) v x W = reflect-substitutionᴱ-whenever H v x reflect-substitutionᴱ H (val (addr a)) v x (UnallocatedAddress r) = Left (UnallocatedAddress r) reflect-substitutionᴱ H (M $ N) v x (FunctionCallMismatch p) with substitutivityᴱ H N v x p reflect-substitutionᴱ H (M $ N) v x (FunctionCallMismatch p) | Right W = Right (Right W) -reflect-substitutionᴱ H (M $ N) v x (FunctionCallMismatch p) | Left q with substitutivityᴱ H M v x (src-any-≮: q) -reflect-substitutionᴱ H (M $ N) v x (FunctionCallMismatch p) | Left q | Left r = Left ((FunctionCallMismatch ∘ any-src-≮: q) r) +reflect-substitutionᴱ H (M $ N) v x (FunctionCallMismatch p) | Left q with substitutivityᴱ H M v x (src-unknown-≮: q) +reflect-substitutionᴱ H (M $ N) v x (FunctionCallMismatch p) | Left q | Left r = Left ((FunctionCallMismatch ∘ unknown-src-≮: q) r) reflect-substitutionᴱ H (M $ N) v x (FunctionCallMismatch p) | Left q | Right W = Right (Right W) reflect-substitutionᴱ H (M $ N) v x (app₁ W) = mapL app₁ (reflect-substitutionᴱ H M v x W) reflect-substitutionᴱ H (M $ N) v x (app₂ W) = mapL app₂ (reflect-substitutionᴱ H N v x W) @@ -187,7 +187,7 @@ reflect-weakeningᴮ : ∀ Γ H B {H′} → (H ⊑ H′) → Warningᴮ H′ (t reflect-weakeningᴱ Γ H (var x) h (UnboundVariable p) = (UnboundVariable p) reflect-weakeningᴱ Γ H (val (addr a)) h (UnallocatedAddress p) = UnallocatedAddress (lookup-⊑-nothing a h p) -reflect-weakeningᴱ Γ H (M $ N) h (FunctionCallMismatch p) = FunctionCallMismatch (heap-weakeningᴱ Γ H N h (any-src-≮: p (heap-weakeningᴱ Γ H M h (src-any-≮: p)))) +reflect-weakeningᴱ Γ H (M $ N) h (FunctionCallMismatch p) = FunctionCallMismatch (heap-weakeningᴱ Γ H N h (unknown-src-≮: p (heap-weakeningᴱ Γ H M h (src-unknown-≮: p)))) reflect-weakeningᴱ Γ H (M $ N) h (app₁ W) = app₁ (reflect-weakeningᴱ Γ H M h W) reflect-weakeningᴱ Γ H (M $ N) h (app₂ W) = app₂ (reflect-weakeningᴱ Γ H N h W) reflect-weakeningᴱ Γ H (binexp M op N) h (BinOpMismatch₁ p) = BinOpMismatch₁ (heap-weakeningᴱ Γ H M h p) @@ -214,19 +214,19 @@ reflect-weakeningᴼ H (just function f ⟨ var x ∈ T ⟩∈ U is B end) h (fu reflectᴱ : ∀ H M {H′ M′} → (H ⊢ M ⟶ᴱ M′ ⊣ H′) → Warningᴱ H′ (typeCheckᴱ H′ ∅ M′) → Either (Warningᴱ H (typeCheckᴱ H ∅ M)) (Warningᴴ H (typeCheckᴴ H)) reflectᴮ : ∀ H B {H′ B′} → (H ⊢ B ⟶ᴮ B′ ⊣ H′) → Warningᴮ H′ (typeCheckᴮ H′ ∅ B′) → Either (Warningᴮ H (typeCheckᴮ H ∅ B)) (Warningᴴ H (typeCheckᴴ H)) -reflectᴱ H (M $ N) (app₁ s) (FunctionCallMismatch p) = cond (Left ∘ FunctionCallMismatch ∘ heap-weakeningᴱ ∅ H N (rednᴱ⊑ s) ∘ any-src-≮: p) (Left ∘ app₁) (reflect-subtypingᴱ H M s (src-any-≮: p)) +reflectᴱ H (M $ N) (app₁ s) (FunctionCallMismatch p) = cond (Left ∘ FunctionCallMismatch ∘ heap-weakeningᴱ ∅ H N (rednᴱ⊑ s) ∘ unknown-src-≮: p) (Left ∘ app₁) (reflect-subtypingᴱ H M s (src-unknown-≮: p)) reflectᴱ H (M $ N) (app₁ s) (app₁ W′) = mapL app₁ (reflectᴱ H M s W′) reflectᴱ H (M $ N) (app₁ s) (app₂ W′) = Left (app₂ (reflect-weakeningᴱ ∅ H N (rednᴱ⊑ s) W′)) -reflectᴱ H (M $ N) (app₂ p s) (FunctionCallMismatch q) = cond (λ r → Left (FunctionCallMismatch (any-src-≮: r (heap-weakeningᴱ ∅ H M (rednᴱ⊑ s) (src-any-≮: r))))) (Left ∘ app₂) (reflect-subtypingᴱ H N s q) +reflectᴱ H (M $ N) (app₂ p s) (FunctionCallMismatch q) = cond (λ r → Left (FunctionCallMismatch (unknown-src-≮: r (heap-weakeningᴱ ∅ H M (rednᴱ⊑ s) (src-unknown-≮: r))))) (Left ∘ app₂) (reflect-subtypingᴱ H N s q) reflectᴱ H (M $ N) (app₂ p s) (app₁ W′) = Left (app₁ (reflect-weakeningᴱ ∅ H M (rednᴱ⊑ s) W′)) reflectᴱ H (M $ N) (app₂ p s) (app₂ W′) = mapL app₂ (reflectᴱ H N s W′) reflectᴱ H (val (addr a) $ N) (beta (function f ⟨ var x ∈ T ⟩∈ U is B end) v refl p) (BlockMismatch q) with substitutivityᴮ H B v x q reflectᴱ H (val (addr a) $ N) (beta (function f ⟨ var x ∈ T ⟩∈ U is B end) v refl p) (BlockMismatch q) | Left r = Right (addr a p (FunctionDefnMismatch r)) -reflectᴱ H (val (addr a) $ N) (beta (function f ⟨ var x ∈ T ⟩∈ U is B end) v refl p) (BlockMismatch q) | Right r = Left (FunctionCallMismatch (≮:-trans-≡ r ((cong src (cong orAny (cong typeOfᴹᴼ (sym p))))))) +reflectᴱ H (val (addr a) $ N) (beta (function f ⟨ var x ∈ T ⟩∈ U is B end) v refl p) (BlockMismatch q) | Right r = Left (FunctionCallMismatch (≮:-trans-≡ r ((cong src (cong orUnknown (cong typeOfᴹᴼ (sym p))))))) reflectᴱ H (val (addr a) $ N) (beta (function f ⟨ var x ∈ T ⟩∈ U is B end) v refl p) (block₁ W′) with reflect-substitutionᴮ _ B v x W′ reflectᴱ H (val (addr a) $ N) (beta (function f ⟨ var x ∈ T ⟩∈ U is B end) v refl p) (block₁ W′) | Left W = Right (addr a p (function₁ W)) reflectᴱ H (val (addr a) $ N) (beta (function f ⟨ var x ∈ T ⟩∈ U is B end) v refl p) (block₁ W′) | Right (Left W) = Left (app₂ W) -reflectᴱ H (val (addr a) $ N) (beta (function f ⟨ var x ∈ T ⟩∈ U is B end) v refl p) (block₁ W′) | Right (Right q) = Left (FunctionCallMismatch (≮:-trans-≡ q (cong src (cong orAny (cong typeOfᴹᴼ (sym p)))))) +reflectᴱ H (val (addr a) $ N) (beta (function f ⟨ var x ∈ T ⟩∈ U is B end) v refl p) (block₁ W′) | Right (Right q) = Left (FunctionCallMismatch (≮:-trans-≡ q (cong src (cong orUnknown (cong typeOfᴹᴼ (sym p)))))) reflectᴱ H (block var b ∈ T is B end) (block s) (BlockMismatch p) = Left (cond BlockMismatch block₁ (reflect-subtypingᴮ H B s p)) reflectᴱ H (block var b ∈ T is B end) (block s) (block₁ W′) = mapL block₁ (reflectᴮ H B s W′) reflectᴱ H (block var b ∈ T is B end) (return v) W′ = Left (block₁ (return W′)) @@ -283,8 +283,8 @@ reflect* H B (step s t) W = cond (reflectᴮ H B s) (reflectᴴᴮ H B s) (refle isntNumber : ∀ H v → (valueType v ≢ number) → (typeOfᴱ H ∅ (val v) ≮: number) isntNumber H nil p = scalar-≢-impl-≮: nil number (λ ()) isntNumber H (addr a) p with remember (H [ a ]ᴴ) -isntNumber H (addr a) p | (just (function f ⟨ var x ∈ T ⟩∈ U is B end) , q) = ≡-trans-≮: (cong orAny (cong typeOfᴹᴼ q)) (function-≮:-scalar number) -isntNumber H (addr a) p | (nothing , q) = ≡-trans-≮: (cong orAny (cong typeOfᴹᴼ q)) (any-≮:-scalar number) +isntNumber H (addr a) p | (just (function f ⟨ var x ∈ T ⟩∈ U is B end) , q) = ≡-trans-≮: (cong orUnknown (cong typeOfᴹᴼ q)) (function-≮:-scalar number) +isntNumber H (addr a) p | (nothing , q) = ≡-trans-≮: (cong orUnknown (cong typeOfᴹᴼ q)) (unknown-≮:-scalar number) isntNumber H (number x) p = CONTRADICTION (p refl) isntNumber H (bool x) p = scalar-≢-impl-≮: boolean number (λ ()) isntNumber H (string x) p = scalar-≢-impl-≮: string number (λ ()) @@ -292,8 +292,8 @@ isntNumber H (string x) p = scalar-≢-impl-≮: string number (λ ()) isntString : ∀ H v → (valueType v ≢ string) → (typeOfᴱ H ∅ (val v) ≮: string) isntString H nil p = scalar-≢-impl-≮: nil string (λ ()) isntString H (addr a) p with remember (H [ a ]ᴴ) -isntString H (addr a) p | (just (function f ⟨ var x ∈ T ⟩∈ U is B end) , q) = ≡-trans-≮: (cong orAny (cong typeOfᴹᴼ q)) (function-≮:-scalar string) -isntString H (addr a) p | (nothing , q) = ≡-trans-≮: (cong orAny (cong typeOfᴹᴼ q)) (any-≮:-scalar string) +isntString H (addr a) p | (just (function f ⟨ var x ∈ T ⟩∈ U is B end) , q) = ≡-trans-≮: (cong orUnknown (cong typeOfᴹᴼ q)) (function-≮:-scalar string) +isntString H (addr a) p | (nothing , q) = ≡-trans-≮: (cong orUnknown (cong typeOfᴹᴼ q)) (unknown-≮:-scalar string) isntString H (number x) p = scalar-≢-impl-≮: number string (λ ()) isntString H (bool x) p = scalar-≢-impl-≮: boolean string (λ ()) isntString H (string x) p = CONTRADICTION (p refl) @@ -305,14 +305,14 @@ isntFunction H (number x) p = scalar-≮:-function number isntFunction H (bool x) p = scalar-≮:-function boolean isntFunction H (string x) p = scalar-≮:-function string -isntEmpty : ∀ H v → (typeOfᴱ H ∅ (val v) ≮: none) -isntEmpty H nil = scalar-≮:-none nil +isntEmpty : ∀ H v → (typeOfᴱ H ∅ (val v) ≮: never) +isntEmpty H nil = scalar-≮:-never nil isntEmpty H (addr a) with remember (H [ a ]ᴴ) -isntEmpty H (addr a) | (just (function f ⟨ var x ∈ T ⟩∈ U is B end) , p) = ≡-trans-≮: (cong orAny (cong typeOfᴹᴼ p)) function-≮:-none -isntEmpty H (addr a) | (nothing , p) = ≡-trans-≮: (cong orAny (cong typeOfᴹᴼ p)) any-≮:-none -isntEmpty H (number x) = scalar-≮:-none number -isntEmpty H (bool x) = scalar-≮:-none boolean -isntEmpty H (string x) = scalar-≮:-none string +isntEmpty H (addr a) | (just (function f ⟨ var x ∈ T ⟩∈ U is B end) , p) = ≡-trans-≮: (cong orUnknown (cong typeOfᴹᴼ p)) function-≮:-never +isntEmpty H (addr a) | (nothing , p) = ≡-trans-≮: (cong orUnknown (cong typeOfᴹᴼ p)) unknown-≮:-never +isntEmpty H (number x) = scalar-≮:-never number +isntEmpty H (bool x) = scalar-≮:-never boolean +isntEmpty H (string x) = scalar-≮:-never string runtimeBinOpWarning : ∀ H {op} v → BinOpError op (valueType v) → (typeOfᴱ H ∅ (val v) ≮: srcBinOp op) runtimeBinOpWarning H v (+ p) = isntNumber H v p @@ -330,7 +330,7 @@ runtimeWarningᴮ : ∀ H B → RuntimeErrorᴮ H B → Warningᴮ H (typeCheck runtimeWarningᴱ H (var x) UnboundVariable = UnboundVariable refl runtimeWarningᴱ H (val (addr a)) (SEGV p) = UnallocatedAddress p -runtimeWarningᴱ H (M $ N) (FunctionMismatch v w p) = FunctionCallMismatch (any-src-≮: (isntEmpty H w) (isntFunction H v p)) +runtimeWarningᴱ H (M $ N) (FunctionMismatch v w p) = FunctionCallMismatch (unknown-src-≮: (isntEmpty H w) (isntFunction H v p)) runtimeWarningᴱ H (M $ N) (app₁ err) = app₁ (runtimeWarningᴱ H M err) runtimeWarningᴱ H (M $ N) (app₂ err) = app₂ (runtimeWarningᴱ H N err) runtimeWarningᴱ H (block var b ∈ T is B end) (block err) = block₁ (runtimeWarningᴮ H B err) diff --git a/prototyping/Properties/Subtyping.agda b/prototyping/Properties/Subtyping.agda index 0a20f244..cc6bb5c1 100644 --- a/prototyping/Properties/Subtyping.agda +++ b/prototyping/Properties/Subtyping.agda @@ -4,8 +4,8 @@ module Properties.Subtyping where open import Agda.Builtin.Equality using (_≡_; refl) open import FFI.Data.Either using (Either; Left; Right; mapLR; swapLR; cond) -open import Luau.Subtyping using (_<:_; _≮:_; Tree; Language; ¬Language; witness; any; none; scalar; function; scalar-function; scalar-function-ok; scalar-function-err; scalar-scalar; function-scalar; function-ok; function-err; left; right; _,_) -open import Luau.Type using (Type; Scalar; strict; nil; number; string; boolean; none; any; _⇒_; _∪_; _∩_; tgt) +open import Luau.Subtyping using (_<:_; _≮:_; Tree; Language; ¬Language; witness; unknown; never; scalar; function; scalar-function; scalar-function-ok; scalar-function-err; scalar-scalar; function-scalar; function-ok; function-err; left; right; _,_) +open import Luau.Type using (Type; Scalar; strict; nil; number; string; boolean; never; unknown; _⇒_; _∪_; _∩_; tgt) open import Properties.Contradiction using (CONTRADICTION; ¬) open import Properties.Equality using (_≢_) open import Properties.Functions using (_∘_) @@ -47,8 +47,8 @@ dec-language (T₁ ⇒ T₂) (scalar s) = Left (function-scalar s) dec-language (T₁ ⇒ T₂) function = Right function dec-language (T₁ ⇒ T₂) (function-ok t) = mapLR function-ok function-ok (dec-language T₂ t) dec-language (T₁ ⇒ T₂) (function-err t) = mapLR function-err function-err (swapLR (dec-language T₁ t)) -dec-language none t = Left none -dec-language any t = Right any +dec-language never t = Left never +dec-language unknown t = Right unknown dec-language (T₁ ∪ T₂) t = cond (λ p → cond (Left ∘ _,_ p) (Right ∘ right) (dec-language T₂ t)) (Right ∘ left) (dec-language T₁ t) dec-language (T₁ ∩ T₂) t = cond (Left ∘ left) (λ p → cond (Left ∘ right) (Right ∘ _,_ p) (dec-language T₂ t)) (dec-language T₁ t) @@ -60,7 +60,7 @@ language-comp t (left p) (q₁ , q₂) = language-comp t p q₁ language-comp t (right p) (q₁ , q₂) = language-comp t p q₂ language-comp (scalar s) (scalar-scalar s p₁ p₂) (scalar s) = p₂ refl language-comp (scalar s) (function-scalar s) (scalar s) = language-comp function (scalar-function s) function -language-comp (scalar s) none (scalar ()) +language-comp (scalar s) never (scalar ()) language-comp function (scalar-function ()) function language-comp (function-ok t) (scalar-function-ok ()) (function-ok q) language-comp (function-ok t) (function-ok p) (function-ok q) = language-comp t p q @@ -104,11 +104,11 @@ function-≮:-scalar s = witness function function (scalar-function s) scalar-≮:-function : ∀ {S T U} → (Scalar U) → (U ≮: (S ⇒ T)) scalar-≮:-function s = witness (scalar s) (scalar s) (function-scalar s) -any-≮:-scalar : ∀ {U} → (Scalar U) → (any ≮: U) -any-≮:-scalar s = witness (function-ok (scalar s)) any (scalar-function-ok s) +unknown-≮:-scalar : ∀ {U} → (Scalar U) → (unknown ≮: U) +unknown-≮:-scalar s = witness (function-ok (scalar s)) unknown (scalar-function-ok s) -scalar-≮:-none : ∀ {U} → (Scalar U) → (U ≮: none) -scalar-≮:-none s = witness (scalar s) (scalar s) none +scalar-≮:-never : ∀ {U} → (Scalar U) → (U ≮: never) +scalar-≮:-never s = witness (scalar s) (scalar s) never scalar-≢-impl-≮: : ∀ {T U} → (Scalar T) → (Scalar U) → (T ≢ U) → (T ≮: U) scalar-≢-impl-≮: s₁ s₂ p = witness (scalar s₁) (scalar s₁) (scalar-scalar s₁ s₂ p) @@ -117,8 +117,8 @@ scalar-≢-impl-≮: s₁ s₂ p = witness (scalar s₁) (scalar s₁) (scalar-s tgt-function-ok : ∀ {T t} → (Language (tgt T) t) → Language T (function-ok t) tgt-function-ok {T = nil} (scalar ()) tgt-function-ok {T = T₁ ⇒ T₂} p = function-ok p -tgt-function-ok {T = none} (scalar ()) -tgt-function-ok {T = any} p = any +tgt-function-ok {T = never} (scalar ()) +tgt-function-ok {T = unknown} p = unknown tgt-function-ok {T = boolean} (scalar ()) tgt-function-ok {T = number} (scalar ()) tgt-function-ok {T = string} (scalar ()) @@ -131,7 +131,7 @@ function-ok-tgt (function-ok p) = p function-ok-tgt (left p) = left (function-ok-tgt p) function-ok-tgt (right p) = right (function-ok-tgt p) function-ok-tgt (p₁ , p₂) = (function-ok-tgt p₁ , function-ok-tgt p₂) -function-ok-tgt any = any +function-ok-tgt unknown = unknown skalar-function-ok : ∀ {t} → (¬Language skalar (function-ok t)) skalar-function-ok = (scalar-function-ok number , (scalar-function-ok string , (scalar-function-ok nil , scalar-function-ok boolean))) @@ -142,22 +142,22 @@ skalar-scalar boolean = right (right (right (scalar boolean))) skalar-scalar string = right (left (scalar string)) skalar-scalar nil = right (right (left (scalar nil))) -tgt-none-≮: : ∀ {T U} → (tgt T ≮: U) → (T ≮: (skalar ∪ (none ⇒ U))) -tgt-none-≮: (witness t p q) = witness (function-ok t) (tgt-function-ok p) (skalar-function-ok , function-ok q) +tgt-never-≮: : ∀ {T U} → (tgt T ≮: U) → (T ≮: (skalar ∪ (never ⇒ U))) +tgt-never-≮: (witness t p q) = witness (function-ok t) (tgt-function-ok p) (skalar-function-ok , function-ok q) -none-tgt-≮: : ∀ {T U} → (T ≮: (skalar ∪ (none ⇒ U))) → (tgt T ≮: U) -none-tgt-≮: (witness (scalar s) p (q₁ , q₂)) = CONTRADICTION (≮:-refl (witness (scalar s) (skalar-scalar s) q₁)) -none-tgt-≮: (witness function p (q₁ , scalar-function ())) -none-tgt-≮: (witness (function-ok t) p (q₁ , function-ok q₂)) = witness t (function-ok-tgt p) q₂ -none-tgt-≮: (witness (function-err (scalar s)) p (q₁ , function-err (scalar ()))) +never-tgt-≮: : ∀ {T U} → (T ≮: (skalar ∪ (never ⇒ U))) → (tgt T ≮: U) +never-tgt-≮: (witness (scalar s) p (q₁ , q₂)) = CONTRADICTION (≮:-refl (witness (scalar s) (skalar-scalar s) q₁)) +never-tgt-≮: (witness function p (q₁ , scalar-function ())) +never-tgt-≮: (witness (function-ok t) p (q₁ , function-ok q₂)) = witness t (function-ok-tgt p) q₂ +never-tgt-≮: (witness (function-err (scalar s)) p (q₁ , function-err (scalar ()))) -- Properties of src function-err-src : ∀ {T t} → (¬Language (src T) t) → Language T (function-err t) -function-err-src {T = nil} none = scalar-function-err nil +function-err-src {T = nil} never = scalar-function-err nil function-err-src {T = T₁ ⇒ T₂} p = function-err p -function-err-src {T = none} (scalar-scalar number () p) -function-err-src {T = none} (scalar-function-ok ()) -function-err-src {T = any} none = any +function-err-src {T = never} (scalar-scalar number () p) +function-err-src {T = never} (scalar-function-ok ()) +function-err-src {T = unknown} never = unknown function-err-src {T = boolean} p = scalar-function-err boolean function-err-src {T = number} p = scalar-function-err number function-err-src {T = string} p = scalar-function-err string @@ -168,8 +168,8 @@ function-err-src {T = T₁ ∩ T₂} (p₁ , p₂) = function-err-src p₁ , fun ¬function-err-src : ∀ {T t} → (Language (src T) t) → ¬Language T (function-err t) ¬function-err-src {T = nil} (scalar ()) ¬function-err-src {T = T₁ ⇒ T₂} p = function-err p -¬function-err-src {T = none} any = none -¬function-err-src {T = any} (scalar ()) +¬function-err-src {T = never} unknown = never +¬function-err-src {T = unknown} (scalar ()) ¬function-err-src {T = boolean} (scalar ()) ¬function-err-src {T = number} (scalar ()) ¬function-err-src {T = string} (scalar ()) @@ -178,53 +178,53 @@ function-err-src {T = T₁ ∩ T₂} (p₁ , p₂) = function-err-src p₁ , fun ¬function-err-src {T = T₁ ∩ T₂} (right p) = right (¬function-err-src p) src-¬function-err : ∀ {T t} → Language T (function-err t) → (¬Language (src T) t) -src-¬function-err {T = nil} p = none +src-¬function-err {T = nil} p = never src-¬function-err {T = T₁ ⇒ T₂} (function-err p) = p -src-¬function-err {T = none} (scalar-function-err ()) -src-¬function-err {T = any} p = none -src-¬function-err {T = boolean} p = none -src-¬function-err {T = number} p = none -src-¬function-err {T = string} p = none +src-¬function-err {T = never} (scalar-function-err ()) +src-¬function-err {T = unknown} p = never +src-¬function-err {T = boolean} p = never +src-¬function-err {T = number} p = never +src-¬function-err {T = string} p = never src-¬function-err {T = T₁ ∪ T₂} (left p) = left (src-¬function-err p) src-¬function-err {T = T₁ ∪ T₂} (right p) = right (src-¬function-err p) src-¬function-err {T = T₁ ∩ T₂} (p₁ , p₂) = (src-¬function-err p₁ , src-¬function-err p₂) src-¬scalar : ∀ {S T t} (s : Scalar S) → Language T (scalar s) → (¬Language (src T) t) -src-¬scalar number (scalar number) = none -src-¬scalar boolean (scalar boolean) = none -src-¬scalar string (scalar string) = none -src-¬scalar nil (scalar nil) = none +src-¬scalar number (scalar number) = never +src-¬scalar boolean (scalar boolean) = never +src-¬scalar string (scalar string) = never +src-¬scalar nil (scalar nil) = never src-¬scalar s (left p) = left (src-¬scalar s p) src-¬scalar s (right p) = right (src-¬scalar s p) src-¬scalar s (p₁ , p₂) = (src-¬scalar s p₁ , src-¬scalar s p₂) -src-¬scalar s any = none +src-¬scalar s unknown = never -src-any-≮: : ∀ {T U} → (T ≮: src U) → (U ≮: (T ⇒ any)) -src-any-≮: (witness t p q) = witness (function-err t) (function-err-src q) (¬function-err-src p) +src-unknown-≮: : ∀ {T U} → (T ≮: src U) → (U ≮: (T ⇒ unknown)) +src-unknown-≮: (witness t p q) = witness (function-err t) (function-err-src q) (¬function-err-src p) -any-src-≮: : ∀ {S T U} → (U ≮: S) → (T ≮: (U ⇒ any)) → (U ≮: src T) -any-src-≮: (witness t x x₁) (witness (scalar s) p (function-scalar s)) = witness t x (src-¬scalar s p) -any-src-≮: r (witness (function-ok (scalar s)) p (function-ok (scalar-scalar s () q))) -any-src-≮: r (witness (function-ok (function-ok _)) p (function-ok (scalar-function-ok ()))) -any-src-≮: r (witness (function-err t) p (function-err q)) = witness t q (src-¬function-err p) +unknown-src-≮: : ∀ {S T U} → (U ≮: S) → (T ≮: (U ⇒ unknown)) → (U ≮: src T) +unknown-src-≮: (witness t x x₁) (witness (scalar s) p (function-scalar s)) = witness t x (src-¬scalar s p) +unknown-src-≮: r (witness (function-ok (scalar s)) p (function-ok (scalar-scalar s () q))) +unknown-src-≮: r (witness (function-ok (function-ok _)) p (function-ok (scalar-function-ok ()))) +unknown-src-≮: r (witness (function-err t) p (function-err q)) = witness t q (src-¬function-err p) --- Properties of any and none -any-≮: : ∀ {T U} → (T ≮: U) → (any ≮: U) -any-≮: (witness t p q) = witness t any q +-- Properties of unknown and never +unknown-≮: : ∀ {T U} → (T ≮: U) → (unknown ≮: U) +unknown-≮: (witness t p q) = witness t unknown q -none-≮: : ∀ {T U} → (T ≮: U) → (T ≮: none) -none-≮: (witness t p q) = witness t p none +never-≮: : ∀ {T U} → (T ≮: U) → (T ≮: never) +never-≮: (witness t p q) = witness t p never -any-≮:-none : (any ≮: none) -any-≮:-none = witness (scalar nil) any none +unknown-≮:-never : (unknown ≮: never) +unknown-≮:-never = witness (scalar nil) unknown never -function-≮:-none : ∀ {T U} → ((T ⇒ U) ≮: none) -function-≮:-none = witness function function none +function-≮:-never : ∀ {T U} → ((T ⇒ U) ≮: never) +function-≮:-never = witness function function never -- A Gentle Introduction To Semantic Subtyping (https://www.cduce.org/papers/gentle.pdf) -- defines a "set-theoretic" model (sec 2.5) -- Unfortunately we don't quite have this property, due to uninhabited types, --- for example (none -> T) is equivalent to (none -> U) +-- for example (never -> T) is equivalent to (never -> U) -- when types are interpreted as sets of syntactic values. _⊆_ : ∀ {A : Set} → (A → Set) → (A → Set) → Set @@ -258,7 +258,7 @@ not-quite-set-theoretic-only-if : ∀ {S₁ T₁ S₂ T₂} → -- We don't quite have that this is a set-theoretic model -- it's only true when Language T₁ and ¬Language T₂ t₂ are inhabited - -- in particular it's not true when T₁ is none, or T₂ is any. + -- in particular it's not true when T₁ is never, or T₂ is unknown. ∀ s₂ t₂ → Language S₂ s₂ → ¬Language T₂ t₂ → -- This is the "only if" part of being a set-theoretic model @@ -285,11 +285,11 @@ not-quite-set-theoretic-only-if {S₁} {T₁} {S₂} {T₂} s₂ t₂ S₂s₂ -- A counterexample when the argument type is empty. -set-theoretic-counterexample-one : (∀ Q → Q ⊆ Comp((Language none) ⊗ Comp(Language number)) → Q ⊆ Comp((Language none) ⊗ Comp(Language string))) +set-theoretic-counterexample-one : (∀ Q → Q ⊆ Comp((Language never) ⊗ Comp(Language number)) → Q ⊆ Comp((Language never) ⊗ Comp(Language string))) set-theoretic-counterexample-one Q q ((scalar s) , u) Qtu (scalar () , p) set-theoretic-counterexample-one Q q ((function-err t) , u) Qtu (scalar-function-err () , p) -set-theoretic-counterexample-two : (none ⇒ number) ≮: (none ⇒ string) +set-theoretic-counterexample-two : (never ⇒ number) ≮: (never ⇒ string) set-theoretic-counterexample-two = witness (function-ok (scalar number)) (function-ok (scalar number)) (function-ok (scalar-scalar number string (λ ()))) @@ -298,14 +298,14 @@ set-theoretic-counterexample-two = witness -- The reason why this is connected to overloaded functions is that currently we have that the type of -- f(x) is (tgt T) where f:T. Really we should have the type depend on the type of x, that is use (tgt T U), -- where U is the type of x. In particular (tgt (S => T) (U & V)) should be the same as (tgt ((S&U) => T) V) --- and tgt(none => T) should be any. For example +-- and tgt(never => T) should be unknown. For example -- -- tgt((number => string) & (string => bool))(number) -- is tgt(number => string)(number) & tgt(string => bool)(number) --- is tgt(number => string)(number) & tgt(string => bool)(number&any) --- is tgt(number => string)(number) & tgt(string&number => bool)(any) --- is tgt(number => string)(number) & tgt(none => bool)(any) --- is string & any +-- is tgt(number => string)(number) & tgt(string => bool)(number&unknown) +-- is tgt(number => string)(number) & tgt(string&number => bool)(unknown) +-- is tgt(number => string)(number) & tgt(never => bool)(unknown) +-- is string & unknown -- is string -- -- there's some discussion of this in the Gentle Introduction paper. diff --git a/prototyping/Properties/TypeCheck.agda b/prototyping/Properties/TypeCheck.agda index ead0c097..a5916a13 100644 --- a/prototyping/Properties/TypeCheck.agda +++ b/prototyping/Properties/TypeCheck.agda @@ -8,9 +8,9 @@ open import Agda.Builtin.Equality using (_≡_; refl) open import Agda.Builtin.Bool using (Bool; true; false) open import FFI.Data.Maybe using (Maybe; just; nothing) open import FFI.Data.Either using (Either) -open import Luau.TypeCheck(m) using (_⊢ᴱ_∈_; _⊢ᴮ_∈_; ⊢ᴼ_; ⊢ᴴ_; _⊢ᴴᴱ_▷_∈_; _⊢ᴴᴮ_▷_∈_; nil; var; addr; number; bool; string; app; function; block; binexp; done; return; local; nothing; orAny; tgtBinOp) +open import Luau.TypeCheck(m) using (_⊢ᴱ_∈_; _⊢ᴮ_∈_; ⊢ᴼ_; ⊢ᴴ_; _⊢ᴴᴱ_▷_∈_; _⊢ᴴᴮ_▷_∈_; nil; var; addr; number; bool; string; app; function; block; binexp; done; return; local; nothing; orUnknown; tgtBinOp) open import Luau.Syntax using (Block; Expr; Value; BinaryOperator; yes; nil; addr; number; bool; string; val; var; binexp; _$_; function_is_end; block_is_end; _∙_; return; done; local_←_; _⟨_⟩; _⟨_⟩∈_; var_∈_; name; fun; arg; +; -; *; /; <; >; ==; ~=; <=; >=) -open import Luau.Type using (Type; nil; any; none; number; boolean; string; _⇒_; tgt) +open import Luau.Type using (Type; nil; unknown; never; number; boolean; string; _⇒_; tgt) open import Luau.RuntimeType using (RuntimeType; nil; number; function; string; valueType) open import Luau.VarCtxt using (VarCtxt; ∅; _↦_; _⊕_↦_; _⋒_; _⊝_) renaming (_[_] to _[_]ⱽ) open import Luau.Addr using (Addr) @@ -42,8 +42,8 @@ typeOfⱽ H (string x) = just string typeOfᴱ : Heap yes → VarCtxt → (Expr yes) → Type typeOfᴮ : Heap yes → VarCtxt → (Block yes) → Type -typeOfᴱ H Γ (var x) = orAny(Γ [ x ]ⱽ) -typeOfᴱ H Γ (val v) = orAny(typeOfⱽ H v) +typeOfᴱ H Γ (var x) = orUnknown(Γ [ x ]ⱽ) +typeOfᴱ H Γ (val v) = orUnknown(typeOfⱽ H v) typeOfᴱ H Γ (M $ N) = tgt(typeOfᴱ H Γ M) typeOfᴱ H Γ (function f ⟨ var x ∈ S ⟩∈ T is B end) = S ⇒ T typeOfᴱ H Γ (block var b ∈ T is B end) = T @@ -54,7 +54,7 @@ typeOfᴮ H Γ (local var x ∈ T ← M ∙ B) = typeOfᴮ H (Γ ⊕ x ↦ T) B typeOfᴮ H Γ (return M ∙ B) = typeOfᴱ H Γ M typeOfᴮ H Γ done = nil -mustBeFunction : ∀ H Γ v → (none ≢ src (typeOfᴱ H Γ (val v))) → (function ≡ valueType(v)) +mustBeFunction : ∀ H Γ v → (never ≢ src (typeOfᴱ H Γ (val v))) → (function ≡ valueType(v)) mustBeFunction H Γ nil p = CONTRADICTION (p refl) mustBeFunction H Γ (addr a) p = refl mustBeFunction H Γ (number n) p = CONTRADICTION (p refl) @@ -64,17 +64,17 @@ mustBeFunction H Γ (string x) p = CONTRADICTION (p refl) mustBeNumber : ∀ H Γ v → (typeOfᴱ H Γ (val v) ≡ number) → (valueType(v) ≡ number) mustBeNumber H Γ (addr a) p with remember (H [ a ]ᴴ) -mustBeNumber H Γ (addr a) p | (just O , q) with trans (cong orAny (cong typeOfᴹᴼ (sym q))) p +mustBeNumber H Γ (addr a) p | (just O , q) with trans (cong orUnknown (cong typeOfᴹᴼ (sym q))) p mustBeNumber H Γ (addr a) p | (just function f ⟨ var x ∈ T ⟩∈ U is B end , q) | () -mustBeNumber H Γ (addr a) p | (nothing , q) with trans (cong orAny (cong typeOfᴹᴼ (sym q))) p +mustBeNumber H Γ (addr a) p | (nothing , q) with trans (cong orUnknown (cong typeOfᴹᴼ (sym q))) p mustBeNumber H Γ (addr a) p | nothing , q | () mustBeNumber H Γ (number n) p = refl mustBeString : ∀ H Γ v → (typeOfᴱ H Γ (val v) ≡ string) → (valueType(v) ≡ string) mustBeString H Γ (addr a) p with remember (H [ a ]ᴴ) -mustBeString H Γ (addr a) p | (just O , q) with trans (cong orAny (cong typeOfᴹᴼ (sym q))) p +mustBeString H Γ (addr a) p | (just O , q) with trans (cong orUnknown (cong typeOfᴹᴼ (sym q))) p mustBeString H Γ (addr a) p | (just function f ⟨ var x ∈ T ⟩∈ U is B end , q) | () -mustBeString H Γ (addr a) p | (nothing , q) with trans (cong orAny (cong typeOfᴹᴼ (sym q))) p +mustBeString H Γ (addr a) p | (nothing , q) with trans (cong orUnknown (cong typeOfᴹᴼ (sym q))) p mustBeString H Γ (addr a) p | (nothing , q) | () mustBeString H Γ (string x) p = refl @@ -83,7 +83,7 @@ typeCheckᴮ : ∀ H Γ B → (Γ ⊢ᴮ B ∈ (typeOfᴮ H Γ B)) typeCheckᴱ H Γ (var x) = var refl typeCheckᴱ H Γ (val nil) = nil -typeCheckᴱ H Γ (val (addr a)) = addr (orAny (typeOfᴹᴼ (H [ a ]ᴴ))) +typeCheckᴱ H Γ (val (addr a)) = addr (orUnknown (typeOfᴹᴼ (H [ a ]ᴴ))) typeCheckᴱ H Γ (val (number n)) = number typeCheckᴱ H Γ (val (bool b)) = bool typeCheckᴱ H Γ (val (string x)) = string diff --git a/prototyping/Tests/PrettyPrinter/smoke_test/out.txt b/prototyping/Tests/PrettyPrinter/smoke_test/out.txt index 34e0c4fe..ca95cae9 100644 --- a/prototyping/Tests/PrettyPrinter/smoke_test/out.txt +++ b/prototyping/Tests/PrettyPrinter/smoke_test/out.txt @@ -10,10 +10,10 @@ local function comp(f) end local id2 = comp(id)(id) local nil2 = id2(nil) -local a : any = nil +local a : unknown = nil local b : nil = nil local c : (nil) -> nil = nil -local d : (any & nil) = nil -local e : any? = nil +local d : (unknown & nil) = nil +local e : unknown? = nil local f : number = 123.0 return id2(nil2) From 02ed5373ecea3bdf8a68fd40e43733d43a9fbed6 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Thu, 14 Apr 2022 14:57:15 -0700 Subject: [PATCH 222/399] Sync to upstream/release/523 --- Analysis/include/Luau/Clone.h | 9 +- Analysis/include/Luau/Error.h | 10 +- Analysis/include/Luau/Frontend.h | 1 + Analysis/include/Luau/LValue.h | 4 + Analysis/include/Luau/Module.h | 2 +- Analysis/include/Luau/Normalize.h | 19 + Analysis/include/Luau/RecursionCounter.h | 26 +- Analysis/include/Luau/Substitution.h | 1 + Analysis/include/Luau/ToString.h | 3 + Analysis/include/Luau/TxnLog.h | 32 +- Analysis/include/Luau/TypeInfer.h | 24 +- Analysis/include/Luau/TypePack.h | 14 +- Analysis/include/Luau/TypeVar.h | 31 +- Analysis/include/Luau/Unifiable.h | 17 +- Analysis/include/Luau/Unifier.h | 22 +- Analysis/include/Luau/UnifierSharedState.h | 2 + Analysis/include/Luau/VisitTypeVar.h | 9 + Analysis/src/Autocomplete.cpp | 8 +- Analysis/src/Clone.cpp | 116 ++- Analysis/src/Error.cpp | 28 +- Analysis/src/Frontend.cpp | 41 +- Analysis/src/IostreamHelpers.cpp | 2 + Analysis/src/JsonEncoder.cpp | 21 +- Analysis/src/LValue.cpp | 17 + Analysis/src/Linter.cpp | 93 +- Analysis/src/Module.cpp | 39 +- Analysis/src/Normalize.cpp | 814 +++++++++++++++++ Analysis/src/Quantify.cpp | 36 + Analysis/src/Substitution.cpp | 138 ++- Analysis/src/ToDot.cpp | 31 + Analysis/src/ToString.cpp | 165 +++- Analysis/src/TopoSortStatements.cpp | 1 + Analysis/src/TxnLog.cpp | 44 +- Analysis/src/TypeAttach.cpp | 13 + Analysis/src/TypeInfer.cpp | 488 +++++++++-- Analysis/src/TypePack.cpp | 46 +- Analysis/src/TypeVar.cpp | 28 +- Analysis/src/Unifier.cpp | 337 +++++-- Ast/include/Luau/DenseHash.h | 118 ++- Ast/include/Luau/Lexer.h | 2 +- Ast/src/Lexer.cpp | 10 +- Ast/src/Parser.cpp | 5 +- Compiler/src/Compiler.cpp | 4 +- Compiler/src/CostModel.cpp | 258 ++++++ Compiler/src/CostModel.h | 18 + Sources.cmake | 6 + VM/src/ltable.cpp | 47 +- VM/src/ltablib.cpp | 2 +- VM/src/lvmexecute.cpp | 4 +- tests/CostModel.test.cpp | 101 +++ tests/Fixture.cpp | 4 +- tests/JsonEncoder.test.cpp | 23 + tests/Linter.test.cpp | 3 +- tests/Module.test.cpp | 75 +- tests/NonstrictMode.test.cpp | 34 + tests/Normalize.test.cpp | 967 +++++++++++++++++++++ tests/Parser.test.cpp | 20 + tests/ToDot.test.cpp | 77 +- tests/ToString.test.cpp | 2 + tests/TopoSort.test.cpp | 32 +- tests/Transpiler.test.cpp | 2 +- tests/TypeInfer.annotations.test.cpp | 10 + tests/TypeInfer.builtins.test.cpp | 13 +- tests/TypeInfer.classes.test.cpp | 3 + tests/TypeInfer.functions.test.cpp | 177 +++- tests/TypeInfer.generics.test.cpp | 77 +- tests/TypeInfer.intersectionTypes.test.cpp | 46 +- tests/TypeInfer.oop.test.cpp | 16 +- tests/TypeInfer.operators.test.cpp | 3 +- tests/TypeInfer.provisional.test.cpp | 131 ++- tests/TypeInfer.refinements.test.cpp | 12 +- tests/TypeInfer.singletons.test.cpp | 11 +- tests/TypeInfer.tables.test.cpp | 61 +- tests/TypeInfer.test.cpp | 66 +- tests/TypeInfer.typePacks.cpp | 38 +- tests/TypeInfer.unionTypes.test.cpp | 25 +- tests/conformance/nextvar.lua | 15 + 77 files changed, 4597 insertions(+), 653 deletions(-) create mode 100644 Analysis/include/Luau/Normalize.h create mode 100644 Analysis/src/Normalize.cpp create mode 100644 Compiler/src/CostModel.cpp create mode 100644 Compiler/src/CostModel.h create mode 100644 tests/CostModel.test.cpp create mode 100644 tests/Normalize.test.cpp diff --git a/Analysis/include/Luau/Clone.h b/Analysis/include/Luau/Clone.h index 917ef801..78aa92c7 100644 --- a/Analysis/include/Luau/Clone.h +++ b/Analysis/include/Luau/Clone.h @@ -14,12 +14,15 @@ using SeenTypePacks = std::unordered_map; struct CloneState { + SeenTypes seenTypes; + SeenTypePacks seenTypePacks; + int recursionCount = 0; bool encounteredFreeType = false; }; -TypePackId clone(TypePackId tp, TypeArena& dest, SeenTypes& seenTypes, SeenTypePacks& seenTypePacks, CloneState& cloneState); -TypeId clone(TypeId tp, TypeArena& dest, SeenTypes& seenTypes, SeenTypePacks& seenTypePacks, CloneState& cloneState); -TypeFun clone(const TypeFun& typeFun, TypeArena& dest, SeenTypes& seenTypes, SeenTypePacks& seenTypePacks, CloneState& cloneState); +TypePackId clone(TypePackId tp, TypeArena& dest, CloneState& cloneState); +TypeId clone(TypeId tp, TypeArena& dest, CloneState& cloneState); +TypeFun clone(const TypeFun& typeFun, TypeArena& dest, CloneState& cloneState); } // namespace Luau diff --git a/Analysis/include/Luau/Error.h b/Analysis/include/Luau/Error.h index 53b946a0..70683141 100644 --- a/Analysis/include/Luau/Error.h +++ b/Analysis/include/Luau/Error.h @@ -287,12 +287,20 @@ struct TypesAreUnrelated bool operator==(const TypesAreUnrelated& rhs) const; }; +struct NormalizationTooComplex +{ + bool operator==(const NormalizationTooComplex&) const + { + return true; + } +}; + using TypeErrorData = Variant; + MissingProperties, SwappedGenericTypeParameter, OptionalValueAccess, MissingUnionProperty, TypesAreUnrelated, NormalizationTooComplex>; struct TypeError { diff --git a/Analysis/include/Luau/Frontend.h b/Analysis/include/Luau/Frontend.h index 2266f548..e24e433c 100644 --- a/Analysis/include/Luau/Frontend.h +++ b/Analysis/include/Luau/Frontend.h @@ -70,6 +70,7 @@ struct SourceNode std::vector> requireLocations; bool dirty = true; bool dirtyAutocomplete = true; + double autocompleteLimitsMult = 1.0; }; struct FrontendOptions diff --git a/Analysis/include/Luau/LValue.h b/Analysis/include/Luau/LValue.h index 3d510d5f..afb71415 100644 --- a/Analysis/include/Luau/LValue.h +++ b/Analysis/include/Luau/LValue.h @@ -35,8 +35,12 @@ const LValue* baseof(const LValue& lvalue); std::optional tryGetLValue(const class AstExpr& expr); // Utility function: breaks down an LValue to get at the Symbol, and reverses the vector of keys. +// TODO: remove with FFlagLuauTypecheckOptPass std::pair> getFullName(const LValue& lvalue); +// Utility function: breaks down an LValue to get at the Symbol +Symbol getBaseSymbol(const LValue& lvalue); + template const T* get(const LValue& lvalue) { diff --git a/Analysis/include/Luau/Module.h b/Analysis/include/Luau/Module.h index 9a32f614..0dd44188 100644 --- a/Analysis/include/Luau/Module.h +++ b/Analysis/include/Luau/Module.h @@ -113,7 +113,7 @@ struct Module // This helps us to force TypeVar ownership into a DAG rather than a DCG. // Returns true if there were any free types encountered in the public interface. This // indicates a bug in the type checker that we want to surface. - bool clonePublicInterface(); + bool clonePublicInterface(InternalErrorReporter& ice); }; } // namespace Luau diff --git a/Analysis/include/Luau/Normalize.h b/Analysis/include/Luau/Normalize.h new file mode 100644 index 00000000..262b54b2 --- /dev/null +++ b/Analysis/include/Luau/Normalize.h @@ -0,0 +1,19 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details + +#include "Luau/Substitution.h" +#include "Luau/TypeVar.h" +#include "Luau/Module.h" + +namespace Luau +{ + +struct InternalErrorReporter; + +bool isSubtype(TypeId superTy, TypeId subTy, InternalErrorReporter& ice); + +std::pair normalize(TypeId ty, TypeArena& arena, InternalErrorReporter& ice); +std::pair normalize(TypeId ty, const ModulePtr& module, InternalErrorReporter& ice); +std::pair normalize(TypePackId ty, TypeArena& arena, InternalErrorReporter& ice); +std::pair normalize(TypePackId ty, const ModulePtr& module, InternalErrorReporter& ice); + +} // namespace Luau diff --git a/Analysis/include/Luau/RecursionCounter.h b/Analysis/include/Luau/RecursionCounter.h index 89632cea..03ae2c83 100644 --- a/Analysis/include/Luau/RecursionCounter.h +++ b/Analysis/include/Luau/RecursionCounter.h @@ -4,10 +4,21 @@ #include "Luau/Common.h" #include +#include + +LUAU_FASTFLAG(LuauRecursionLimitException); namespace Luau { +struct RecursionLimitException : public std::exception +{ + const char* what() const noexcept + { + return "Internal recursion counter limit exceeded"; + } +}; + struct RecursionCounter { RecursionCounter(int* count) @@ -28,11 +39,22 @@ private: struct RecursionLimiter : RecursionCounter { - RecursionLimiter(int* count, int limit) + // TODO: remove ctx after LuauRecursionLimitException is removed + RecursionLimiter(int* count, int limit, const char* ctx) : RecursionCounter(count) { + LUAU_ASSERT(ctx); if (limit > 0 && *count > limit) - throw std::runtime_error("Internal recursion counter limit exceeded"); + { + if (FFlag::LuauRecursionLimitException) + throw RecursionLimitException(); + else + { + std::string m = "Internal recursion counter limit exceeded: "; + m += ctx; + throw std::runtime_error(m); + } + } } }; diff --git a/Analysis/include/Luau/Substitution.h b/Analysis/include/Luau/Substitution.h index 9662d5b3..6f5931e1 100644 --- a/Analysis/include/Luau/Substitution.h +++ b/Analysis/include/Luau/Substitution.h @@ -90,6 +90,7 @@ struct Tarjan std::vector lowlink; int childCount = 0; + int childLimit = 0; // This should never be null; ensure you initialize it before calling // substitution methods. diff --git a/Analysis/include/Luau/ToString.h b/Analysis/include/Luau/ToString.h index 49ee82fe..f4db5e35 100644 --- a/Analysis/include/Luau/ToString.h +++ b/Analysis/include/Luau/ToString.h @@ -28,6 +28,7 @@ struct ToStringOptions bool functionTypeArguments = false; // If true, output function type argument names when they are available bool hideTableKind = false; // If true, all tables will be surrounded with plain '{}' bool hideNamedFunctionTypeParameters = false; // If true, type parameters of functions will be hidden at top-level. + bool indent = false; size_t maxTableLength = size_t(FInt::LuauTableTypeMaximumStringifierLength); // Only applied to TableTypeVars size_t maxTypeLength = size_t(FInt::LuauTypeMaximumStringifierLength); std::optional nameMap; @@ -73,6 +74,8 @@ std::string toStringNamedFunction(const std::string& funcName, const FunctionTyp std::string dump(TypeId ty); std::string dump(TypePackId ty); +std::string dump(const std::shared_ptr& scope, const char* name); + std::string generateName(size_t n); } // namespace Luau diff --git a/Analysis/include/Luau/TxnLog.h b/Analysis/include/Luau/TxnLog.h index c8ebaaeb..995ed6c6 100644 --- a/Analysis/include/Luau/TxnLog.h +++ b/Analysis/include/Luau/TxnLog.h @@ -7,7 +7,7 @@ #include "Luau/TypeVar.h" #include "Luau/TypePack.h" -LUAU_FASTFLAG(LuauShareTxnSeen); +LUAU_FASTFLAG(LuauTypecheckOptPass) namespace Luau { @@ -64,13 +64,17 @@ T* getMutable(PendingTypePack* pending) struct TxnLog { TxnLog() - : ownedSeen() + : typeVarChanges(nullptr) + , typePackChanges(nullptr) + , ownedSeen() , sharedSeen(&ownedSeen) { } explicit TxnLog(TxnLog* parent) - : parent(parent) + : typeVarChanges(nullptr) + , typePackChanges(nullptr) + , parent(parent) { if (parent) { @@ -83,14 +87,19 @@ struct TxnLog } explicit TxnLog(std::vector>* sharedSeen) - : sharedSeen(sharedSeen) + : typeVarChanges(nullptr) + , typePackChanges(nullptr) + , sharedSeen(sharedSeen) { } TxnLog(TxnLog* parent, std::vector>* sharedSeen) - : parent(parent) + : typeVarChanges(nullptr) + , typePackChanges(nullptr) + , parent(parent) , sharedSeen(sharedSeen) { + LUAU_ASSERT(!FFlag::LuauTypecheckOptPass); } TxnLog(const TxnLog&) = delete; @@ -243,6 +252,12 @@ struct TxnLog return Luau::getMutable(ty); } + template + const T* get(TID ty) const + { + return this->getMutable(ty); + } + // Returns whether a given type or type pack is a given state, respecting the // log's pending state. // @@ -263,11 +278,8 @@ private: // unique_ptr is used to give us stable pointers across insertions into the // map. Otherwise, it would be really easy to accidentally invalidate the // pointers returned from queue/pending. - // - // We can't use a DenseHashMap here because we need a non-const iterator - // over the map when we concatenate. - std::unordered_map, DenseHashPointer> typeVarChanges; - std::unordered_map, DenseHashPointer> typePackChanges; + DenseHashMap> typeVarChanges; + DenseHashMap> typePackChanges; TxnLog* parent = nullptr; diff --git a/Analysis/include/Luau/TypeInfer.h b/Analysis/include/Luau/TypeInfer.h index 215da67f..ac880135 100644 --- a/Analysis/include/Luau/TypeInfer.h +++ b/Analysis/include/Luau/TypeInfer.h @@ -76,19 +76,32 @@ struct Instantiation : Substitution // A substitution which replaces free types by any struct Anyification : Substitution { - Anyification(TypeArena* arena, TypeId anyType, TypePackId anyTypePack) + Anyification(TypeArena* arena, InternalErrorReporter* iceHandler, TypeId anyType, TypePackId anyTypePack) : Substitution(TxnLog::empty(), arena) + , iceHandler(iceHandler) , anyType(anyType) , anyTypePack(anyTypePack) { } + InternalErrorReporter* iceHandler; + TypeId anyType; TypePackId anyTypePack; + bool normalizationTooComplex = false; bool isDirty(TypeId ty) override; bool isDirty(TypePackId tp) override; TypeId clean(TypeId ty) override; TypePackId clean(TypePackId tp) override; + + bool ignoreChildren(TypeId ty) override + { + return ty->persistent; + } + bool ignoreChildren(TypePackId ty) override + { + return ty->persistent; + } }; // A substitution which replaces the type parameters of a type function by arguments @@ -139,6 +152,7 @@ struct TypeChecker TypeChecker& operator=(const TypeChecker&) = delete; ModulePtr check(const SourceModule& module, Mode mode, std::optional environmentScope = std::nullopt); + ModulePtr checkWithoutRecursionCheck(const SourceModule& module, Mode mode, std::optional environmentScope = std::nullopt); std::vector> getScopes() const; @@ -160,6 +174,7 @@ struct TypeChecker void check(const ScopePtr& scope, const AstStatDeclareFunction& declaredFunction); void checkBlock(const ScopePtr& scope, const AstStatBlock& statement); + void checkBlockWithoutRecursionCheck(const ScopePtr& scope, const AstStatBlock& statement); void checkBlockTypeAliases(const ScopePtr& scope, std::vector& sorted); ExprResult checkExpr( @@ -172,6 +187,7 @@ struct TypeChecker ExprResult checkExpr(const ScopePtr& scope, const AstExprIndexExpr& expr); ExprResult checkExpr(const ScopePtr& scope, const AstExprFunction& expr, std::optional expectedType = std::nullopt); ExprResult checkExpr(const ScopePtr& scope, const AstExprTable& expr, std::optional expectedType = std::nullopt); + ExprResult checkExpr_(const ScopePtr& scope, const AstExprTable& expr, std::optional expectedType = std::nullopt); ExprResult checkExpr(const ScopePtr& scope, const AstExprUnary& expr); TypeId checkRelationalOperation( const ScopePtr& scope, const AstExprBinary& expr, TypeId lhsType, TypeId rhsType, const PredicateVec& predicates = {}); @@ -258,6 +274,8 @@ struct TypeChecker ErrorVec canUnify(TypeId subTy, TypeId superTy, const Location& location); ErrorVec canUnify(TypePackId subTy, TypePackId superTy, const Location& location); + void unifyLowerBound(TypePackId subTy, TypePackId superTy, const Location& location); + std::optional findMetatableEntry(TypeId type, std::string entry, const Location& location); std::optional findTablePropertyRespectingMeta(TypeId lhsType, Name name, const Location& location); @@ -395,6 +413,7 @@ private: void resolve(const EqPredicate& eqP, ErrorVec& errVec, RefinementMap& refis, const ScopePtr& scope, bool sense); bool isNonstrictMode() const; + bool useConstrainedIntersections() const; public: /** Extract the types in a type pack, given the assumption that the pack must have some exact length. @@ -421,7 +440,10 @@ public: std::vector requireCycles; + // Type inference limits std::optional finishTime; + std::optional instantiationChildLimit; + std::optional unifierIterationLimit; public: const TypeId nilType; diff --git a/Analysis/include/Luau/TypePack.h b/Analysis/include/Luau/TypePack.h index 85fa467f..bbc65f94 100644 --- a/Analysis/include/Luau/TypePack.h +++ b/Analysis/include/Luau/TypePack.h @@ -40,6 +40,7 @@ struct TypePack struct VariadicTypePack { TypeId ty; + bool hidden = false; // if true, we don't display this when toString()ing a pack with this variadic as its tail. }; struct TypePackVar @@ -109,10 +110,10 @@ private: }; TypePackIterator begin(TypePackId tp); -TypePackIterator begin(TypePackId tp, TxnLog* log); +TypePackIterator begin(TypePackId tp, const TxnLog* log); TypePackIterator end(TypePackId tp); -using SeenSet = std::set>; +using SeenSet = std::set>; bool areEqual(SeenSet& seen, const TypePackVar& lhs, const TypePackVar& rhs); @@ -122,7 +123,7 @@ TypePackId follow(TypePackId tp, std::function mapper); size_t size(TypePackId tp, TxnLog* log = nullptr); bool finite(TypePackId tp, TxnLog* log = nullptr); size_t size(const TypePack& tp, TxnLog* log = nullptr); -std::optional first(TypePackId tp); +std::optional first(TypePackId tp, bool ignoreHiddenVariadics = true); TypePackVar* asMutable(TypePackId tp); TypePack* asMutable(const TypePack* tp); @@ -154,5 +155,12 @@ bool isEmpty(TypePackId tp); /// Flattens out a type pack. Also returns a valid TypePackId tail if the type pack's full size is not known std::pair, std::optional> flatten(TypePackId tp); +std::pair, std::optional> flatten(TypePackId tp, const TxnLog& log); + +/// Returs true if the type pack arose from a function that is declared to be variadic. +/// Returns *false* for function argument packs that are inferred to be safe to oversaturate! +bool isVariadic(TypePackId tp); +bool isVariadic(TypePackId tp, const TxnLog& log); + } // namespace Luau diff --git a/Analysis/include/Luau/TypeVar.h b/Analysis/include/Luau/TypeVar.h index f61e4044..ae7d1377 100644 --- a/Analysis/include/Luau/TypeVar.h +++ b/Analysis/include/Luau/TypeVar.h @@ -109,6 +109,23 @@ struct PrimitiveTypeVar } }; +struct ConstrainedTypeVar +{ + explicit ConstrainedTypeVar(TypeLevel level) + : level(level) + { + } + + explicit ConstrainedTypeVar(TypeLevel level, const std::vector& parts) + : parts(parts) + , level(level) + { + } + + std::vector parts; + TypeLevel level; +}; + // Singleton types https://github.com/Roblox/luau/blob/master/rfcs/syntax-singleton-types.md // Types for true and false struct BooleanSingleton @@ -248,6 +265,7 @@ struct FunctionTypeVar MagicFunction magicFunction = nullptr; // Function pointer, can be nullptr. bool hasSelf; Tags tags; + bool hasNoGenerics = false; }; enum class TableState @@ -418,8 +436,8 @@ struct LazyTypeVar using ErrorTypeVar = Unifiable::Error; -using TypeVariant = Unifiable::Variant; +using TypeVariant = Unifiable::Variant; struct TypeVar final { @@ -436,6 +454,7 @@ struct TypeVar final TypeVar(const TypeVariant& ty, bool persistent) : ty(ty) , persistent(persistent) + , normal(persistent) // We assume that all persistent types are irreducable. { } @@ -446,6 +465,10 @@ struct TypeVar final // Persistent TypeVars do not get cloned. bool persistent = false; + // Normalization sets this for types that are fully normalized. + // This implies that they are transitively immutable. + bool normal = false; + std::optional documentationSymbol; // Pointer to the type arena that allocated this type. @@ -458,7 +481,7 @@ struct TypeVar final TypeVar& operator=(TypeVariant&& rhs); }; -using SeenSet = std::set>; +using SeenSet = std::set>; bool areEqual(SeenSet& seen, const TypeVar& lhs, const TypeVar& rhs); // Follow BoundTypeVars until we get to something real @@ -545,6 +568,8 @@ void persist(TypePackId tp); const TypeLevel* getLevel(TypeId ty); TypeLevel* getMutableLevel(TypeId ty); +std::optional getLevel(TypePackId tp); + const Property* lookupClassProp(const ClassTypeVar* cls, const Name& name); bool isSubclass(const ClassTypeVar* cls, const ClassTypeVar* parent); diff --git a/Analysis/include/Luau/Unifiable.h b/Analysis/include/Luau/Unifiable.h index e8eafe68..64fa131d 100644 --- a/Analysis/include/Luau/Unifiable.h +++ b/Analysis/include/Luau/Unifiable.h @@ -56,6 +56,14 @@ struct TypeLevel } }; +inline TypeLevel max(const TypeLevel& a, const TypeLevel& b) +{ + if (a.subsumes(b)) + return b; + else + return a; +} + inline TypeLevel min(const TypeLevel& a, const TypeLevel& b) { if (a.subsumes(b)) @@ -64,7 +72,9 @@ inline TypeLevel min(const TypeLevel& a, const TypeLevel& b) return b; } -namespace Unifiable +} // namespace Luau + +namespace Luau::Unifiable { using Name = std::string; @@ -125,7 +135,6 @@ private: }; template -using Variant = Variant, Generic, Error, Value...>; +using Variant = Luau::Variant, Generic, Error, Value...>; -} // namespace Unifiable -} // namespace Luau +} // namespace Luau::Unifiable diff --git a/Analysis/include/Luau/Unifier.h b/Analysis/include/Luau/Unifier.h index 474af50c..340feb7f 100644 --- a/Analysis/include/Luau/Unifier.h +++ b/Analysis/include/Luau/Unifier.h @@ -49,14 +49,14 @@ struct Unifier ErrorVec errors; Location location; Variance variance = Covariant; + bool anyIsTop = false; // If true, we consider any to be a top type. If false, it is a familiar but weird mix of top and bottom all at once. CountMismatch::Context ctx = CountMismatch::Arg; UnifierSharedState& sharedState; - Unifier(TypeArena* types, Mode mode, const Location& location, Variance variance, UnifierSharedState& sharedState, - TxnLog* parentLog = nullptr); - Unifier(TypeArena* types, Mode mode, std::vector>* sharedSeen, const Location& location, - Variance variance, UnifierSharedState& sharedState, TxnLog* parentLog = nullptr); + Unifier(TypeArena* types, Mode mode, const Location& location, Variance variance, UnifierSharedState& sharedState, TxnLog* parentLog = nullptr); + Unifier(TypeArena* types, Mode mode, std::vector>* sharedSeen, const Location& location, Variance variance, + UnifierSharedState& sharedState, TxnLog* parentLog = nullptr); // Test whether the two type vars unify. Never commits the result. ErrorVec canUnify(TypeId subTy, TypeId superTy); @@ -106,7 +106,12 @@ private: std::optional findTablePropertyRespectingMeta(TypeId lhsType, Name name); + void tryUnifyWithConstrainedSubTypeVar(TypeId subTy, TypeId superTy); + void tryUnifyWithConstrainedSuperTypeVar(TypeId subTy, TypeId superTy); + public: + void unifyLowerBound(TypePackId subTy, TypePackId superTy); + // Report an "infinite type error" if the type "needle" already occurs within "haystack" void occursCheck(TypeId needle, TypeId haystack); void occursCheck(DenseHashSet& seen, TypeId needle, TypeId haystack); @@ -115,12 +120,7 @@ public: Unifier makeChildUnifier(); - // A utility function that appends the given error to the unifier's error log. - // This allows setting a breakpoint wherever the unifier reports an error. - void reportError(TypeError error) - { - errors.push_back(error); - } + void reportError(TypeError err); private: bool isNonstrictMode() const; @@ -135,4 +135,6 @@ private: std::optional firstPackErrorPos; }; +void promoteTypeLevels(TxnLog& log, const TypeArena* arena, TypeLevel minLevel, TypePackId tp); + } // namespace Luau diff --git a/Analysis/include/Luau/UnifierSharedState.h b/Analysis/include/Luau/UnifierSharedState.h index 9a3ba56d..1a0b8b76 100644 --- a/Analysis/include/Luau/UnifierSharedState.h +++ b/Analysis/include/Luau/UnifierSharedState.h @@ -28,7 +28,9 @@ struct TypeIdPairHash struct UnifierCounters { int recursionCount = 0; + int recursionLimit = 0; int iterationCount = 0; + int iterationLimit = 0; }; struct UnifierSharedState diff --git a/Analysis/include/Luau/VisitTypeVar.h b/Analysis/include/Luau/VisitTypeVar.h index 740854b3..d11cbd0d 100644 --- a/Analysis/include/Luau/VisitTypeVar.h +++ b/Analysis/include/Luau/VisitTypeVar.h @@ -82,6 +82,15 @@ void visit(TypeId ty, F& f, Set& seen) else if (auto etv = get(ty)) apply(ty, *etv, seen, f); + else if (auto ctv = get(ty)) + { + if (apply(ty, *ctv, seen, f)) + { + for (TypeId part : ctv->parts) + visit(part, f, seen); + } + } + else if (auto ptv = get(ty)) apply(ty, *ptv, seen, f); diff --git a/Analysis/src/Autocomplete.cpp b/Analysis/src/Autocomplete.cpp index b7201ab3..e0e79cb4 100644 --- a/Analysis/src/Autocomplete.cpp +++ b/Analysis/src/Autocomplete.cpp @@ -151,8 +151,12 @@ static ParenthesesRecommendation getParenRecommendationForFunc(const FunctionTyp auto idxExpr = nodes.back()->as(); bool hasImplicitSelf = idxExpr && idxExpr->op == ':'; - auto args = Luau::flatten(func->argTypes); - bool noArgFunction = (args.first.empty() || (hasImplicitSelf && args.first.size() == 1)) && !args.second.has_value(); + auto [argTypes, argVariadicPack] = Luau::flatten(func->argTypes); + + if (argVariadicPack.has_value() && isVariadic(*argVariadicPack)) + return ParenthesesRecommendation::CursorInside; + + bool noArgFunction = argTypes.empty() || (hasImplicitSelf && argTypes.size() == 1); return noArgFunction ? ParenthesesRecommendation::CursorAfter : ParenthesesRecommendation::CursorInside; } diff --git a/Analysis/src/Clone.cpp b/Analysis/src/Clone.cpp index ac9705a7..8e7f7c07 100644 --- a/Analysis/src/Clone.cpp +++ b/Analysis/src/Clone.cpp @@ -6,7 +6,10 @@ #include "Luau/TypePack.h" #include "Luau/Unifiable.h" +LUAU_FASTFLAG(DebugLuauCopyBeforeNormalizing) + LUAU_FASTINTVARIABLE(LuauTypeCloneRecursionLimit, 300) +LUAU_FASTFLAG(LuauTypecheckOptPass) namespace Luau { @@ -23,11 +26,11 @@ struct TypePackCloner; struct TypeCloner { - TypeCloner(TypeArena& dest, TypeId typeId, SeenTypes& seenTypes, SeenTypePacks& seenTypePacks, CloneState& cloneState) + TypeCloner(TypeArena& dest, TypeId typeId, CloneState& cloneState) : dest(dest) , typeId(typeId) - , seenTypes(seenTypes) - , seenTypePacks(seenTypePacks) + , seenTypes(cloneState.seenTypes) + , seenTypePacks(cloneState.seenTypePacks) , cloneState(cloneState) { } @@ -46,6 +49,7 @@ struct TypeCloner void operator()(const Unifiable::Bound& t); void operator()(const Unifiable::Error& t); void operator()(const PrimitiveTypeVar& t); + void operator()(const ConstrainedTypeVar& t); void operator()(const SingletonTypeVar& t); void operator()(const FunctionTypeVar& t); void operator()(const TableTypeVar& t); @@ -65,11 +69,11 @@ struct TypePackCloner SeenTypePacks& seenTypePacks; CloneState& cloneState; - TypePackCloner(TypeArena& dest, TypePackId typePackId, SeenTypes& seenTypes, SeenTypePacks& seenTypePacks, CloneState& cloneState) + TypePackCloner(TypeArena& dest, TypePackId typePackId, CloneState& cloneState) : dest(dest) , typePackId(typePackId) - , seenTypes(seenTypes) - , seenTypePacks(seenTypePacks) + , seenTypes(cloneState.seenTypes) + , seenTypePacks(cloneState.seenTypePacks) , cloneState(cloneState) { } @@ -103,13 +107,15 @@ struct TypePackCloner // We just need to be sure that we rewrite pointers both to the binder and the bindee to the same pointer. void operator()(const Unifiable::Bound& t) { - TypePackId cloned = clone(t.boundTo, dest, seenTypes, seenTypePacks, cloneState); + TypePackId cloned = clone(t.boundTo, dest, cloneState); + if (FFlag::DebugLuauCopyBeforeNormalizing) + cloned = dest.addTypePack(TypePackVar{BoundTypePack{cloned}}); seenTypePacks[typePackId] = cloned; } void operator()(const VariadicTypePack& t) { - TypePackId cloned = dest.addTypePack(TypePackVar{VariadicTypePack{clone(t.ty, dest, seenTypes, seenTypePacks, cloneState)}}); + TypePackId cloned = dest.addTypePack(TypePackVar{VariadicTypePack{clone(t.ty, dest, cloneState), /*hidden*/ t.hidden}}); seenTypePacks[typePackId] = cloned; } @@ -121,10 +127,10 @@ struct TypePackCloner seenTypePacks[typePackId] = cloned; for (TypeId ty : t.head) - destTp->head.push_back(clone(ty, dest, seenTypes, seenTypePacks, cloneState)); + destTp->head.push_back(clone(ty, dest, cloneState)); if (t.tail) - destTp->tail = clone(*t.tail, dest, seenTypes, seenTypePacks, cloneState); + destTp->tail = clone(*t.tail, dest, cloneState); } }; @@ -150,7 +156,9 @@ void TypeCloner::operator()(const Unifiable::Generic& t) void TypeCloner::operator()(const Unifiable::Bound& t) { - TypeId boundTo = clone(t.boundTo, dest, seenTypes, seenTypePacks, cloneState); + TypeId boundTo = clone(t.boundTo, dest, cloneState); + if (FFlag::DebugLuauCopyBeforeNormalizing) + boundTo = dest.addType(BoundTypeVar{boundTo}); seenTypes[typeId] = boundTo; } @@ -164,6 +172,23 @@ void TypeCloner::operator()(const PrimitiveTypeVar& t) defaultClone(t); } +void TypeCloner::operator()(const ConstrainedTypeVar& t) +{ + cloneState.encounteredFreeType = true; + + TypeId res = dest.addType(ConstrainedTypeVar{t.level}); + ConstrainedTypeVar* ctv = getMutable(res); + LUAU_ASSERT(ctv); + + seenTypes[typeId] = res; + + std::vector parts; + for (TypeId part : t.parts) + parts.push_back(clone(part, dest, cloneState)); + + ctv->parts = std::move(parts); +} + void TypeCloner::operator()(const SingletonTypeVar& t) { defaultClone(t); @@ -178,23 +203,26 @@ void TypeCloner::operator()(const FunctionTypeVar& t) seenTypes[typeId] = result; for (TypeId generic : t.generics) - ftv->generics.push_back(clone(generic, dest, seenTypes, seenTypePacks, cloneState)); + ftv->generics.push_back(clone(generic, dest, cloneState)); for (TypePackId genericPack : t.genericPacks) - ftv->genericPacks.push_back(clone(genericPack, dest, seenTypes, seenTypePacks, cloneState)); + ftv->genericPacks.push_back(clone(genericPack, dest, cloneState)); ftv->tags = t.tags; - ftv->argTypes = clone(t.argTypes, dest, seenTypes, seenTypePacks, cloneState); + ftv->argTypes = clone(t.argTypes, dest, cloneState); ftv->argNames = t.argNames; - ftv->retType = clone(t.retType, dest, seenTypes, seenTypePacks, cloneState); + ftv->retType = clone(t.retType, dest, cloneState); + + if (FFlag::LuauTypecheckOptPass) + ftv->hasNoGenerics = t.hasNoGenerics; } void TypeCloner::operator()(const TableTypeVar& t) { // If table is now bound to another one, we ignore the content of the original - if (t.boundTo) + if (!FFlag::DebugLuauCopyBeforeNormalizing && t.boundTo) { - TypeId boundTo = clone(*t.boundTo, dest, seenTypes, seenTypePacks, cloneState); + TypeId boundTo = clone(*t.boundTo, dest, cloneState); seenTypes[typeId] = boundTo; return; } @@ -209,18 +237,20 @@ void TypeCloner::operator()(const TableTypeVar& t) ttv->level = TypeLevel{0, 0}; + if (FFlag::DebugLuauCopyBeforeNormalizing && t.boundTo) + ttv->boundTo = clone(*t.boundTo, dest, cloneState); + for (const auto& [name, prop] : t.props) - ttv->props[name] = {clone(prop.type, dest, seenTypes, seenTypePacks, cloneState), prop.deprecated, {}, prop.location, prop.tags}; + ttv->props[name] = {clone(prop.type, dest, cloneState), prop.deprecated, {}, prop.location, prop.tags}; if (t.indexer) - ttv->indexer = TableIndexer{clone(t.indexer->indexType, dest, seenTypes, seenTypePacks, cloneState), - clone(t.indexer->indexResultType, dest, seenTypes, seenTypePacks, cloneState)}; + ttv->indexer = TableIndexer{clone(t.indexer->indexType, dest, cloneState), clone(t.indexer->indexResultType, dest, cloneState)}; for (TypeId& arg : ttv->instantiatedTypeParams) - arg = clone(arg, dest, seenTypes, seenTypePacks, cloneState); + arg = clone(arg, dest, cloneState); for (TypePackId& arg : ttv->instantiatedTypePackParams) - arg = clone(arg, dest, seenTypes, seenTypePacks, cloneState); + arg = clone(arg, dest, cloneState); if (ttv->state == TableState::Free) { @@ -240,8 +270,8 @@ void TypeCloner::operator()(const MetatableTypeVar& t) MetatableTypeVar* mtv = getMutable(result); seenTypes[typeId] = result; - mtv->table = clone(t.table, dest, seenTypes, seenTypePacks, cloneState); - mtv->metatable = clone(t.metatable, dest, seenTypes, seenTypePacks, cloneState); + mtv->table = clone(t.table, dest, cloneState); + mtv->metatable = clone(t.metatable, dest, cloneState); } void TypeCloner::operator()(const ClassTypeVar& t) @@ -252,13 +282,13 @@ void TypeCloner::operator()(const ClassTypeVar& t) seenTypes[typeId] = result; for (const auto& [name, prop] : t.props) - ctv->props[name] = {clone(prop.type, dest, seenTypes, seenTypePacks, cloneState), prop.deprecated, {}, prop.location, prop.tags}; + ctv->props[name] = {clone(prop.type, dest, cloneState), prop.deprecated, {}, prop.location, prop.tags}; if (t.parent) - ctv->parent = clone(*t.parent, dest, seenTypes, seenTypePacks, cloneState); + ctv->parent = clone(*t.parent, dest, cloneState); if (t.metatable) - ctv->metatable = clone(*t.metatable, dest, seenTypes, seenTypePacks, cloneState); + ctv->metatable = clone(*t.metatable, dest, cloneState); } void TypeCloner::operator()(const AnyTypeVar& t) @@ -272,7 +302,7 @@ void TypeCloner::operator()(const UnionTypeVar& t) options.reserve(t.options.size()); for (TypeId ty : t.options) - options.push_back(clone(ty, dest, seenTypes, seenTypePacks, cloneState)); + options.push_back(clone(ty, dest, cloneState)); TypeId result = dest.addType(UnionTypeVar{std::move(options)}); seenTypes[typeId] = result; @@ -287,7 +317,7 @@ void TypeCloner::operator()(const IntersectionTypeVar& t) LUAU_ASSERT(option != nullptr); for (TypeId ty : t.parts) - option->parts.push_back(clone(ty, dest, seenTypes, seenTypePacks, cloneState)); + option->parts.push_back(clone(ty, dest, cloneState)); } void TypeCloner::operator()(const LazyTypeVar& t) @@ -297,36 +327,36 @@ void TypeCloner::operator()(const LazyTypeVar& t) } // anonymous namespace -TypePackId clone(TypePackId tp, TypeArena& dest, SeenTypes& seenTypes, SeenTypePacks& seenTypePacks, CloneState& cloneState) +TypePackId clone(TypePackId tp, TypeArena& dest, CloneState& cloneState) { if (tp->persistent) return tp; - RecursionLimiter _ra(&cloneState.recursionCount, FInt::LuauTypeCloneRecursionLimit); + RecursionLimiter _ra(&cloneState.recursionCount, FInt::LuauTypeCloneRecursionLimit, "cloning TypePackId"); - TypePackId& res = seenTypePacks[tp]; + TypePackId& res = cloneState.seenTypePacks[tp]; if (res == nullptr) { - TypePackCloner cloner{dest, tp, seenTypes, seenTypePacks, cloneState}; + TypePackCloner cloner{dest, tp, cloneState}; Luau::visit(cloner, tp->ty); // Mutates the storage that 'res' points into. } return res; } -TypeId clone(TypeId typeId, TypeArena& dest, SeenTypes& seenTypes, SeenTypePacks& seenTypePacks, CloneState& cloneState) +TypeId clone(TypeId typeId, TypeArena& dest, CloneState& cloneState) { if (typeId->persistent) return typeId; - RecursionLimiter _ra(&cloneState.recursionCount, FInt::LuauTypeCloneRecursionLimit); + RecursionLimiter _ra(&cloneState.recursionCount, FInt::LuauTypeCloneRecursionLimit, "cloning TypeId"); - TypeId& res = seenTypes[typeId]; + TypeId& res = cloneState.seenTypes[typeId]; if (res == nullptr) { - TypeCloner cloner{dest, typeId, seenTypes, seenTypePacks, cloneState}; + TypeCloner cloner{dest, typeId, cloneState}; Luau::visit(cloner, typeId->ty); // Mutates the storage that 'res' points into. // Persistent types are not being cloned and we get the original type back which might be read-only @@ -337,33 +367,33 @@ TypeId clone(TypeId typeId, TypeArena& dest, SeenTypes& seenTypes, SeenTypePacks return res; } -TypeFun clone(const TypeFun& typeFun, TypeArena& dest, SeenTypes& seenTypes, SeenTypePacks& seenTypePacks, CloneState& cloneState) +TypeFun clone(const TypeFun& typeFun, TypeArena& dest, CloneState& cloneState) { TypeFun result; for (auto param : typeFun.typeParams) { - TypeId ty = clone(param.ty, dest, seenTypes, seenTypePacks, cloneState); + TypeId ty = clone(param.ty, dest, cloneState); std::optional defaultValue; if (param.defaultValue) - defaultValue = clone(*param.defaultValue, dest, seenTypes, seenTypePacks, cloneState); + defaultValue = clone(*param.defaultValue, dest, cloneState); result.typeParams.push_back({ty, defaultValue}); } for (auto param : typeFun.typePackParams) { - TypePackId tp = clone(param.tp, dest, seenTypes, seenTypePacks, cloneState); + TypePackId tp = clone(param.tp, dest, cloneState); std::optional defaultValue; if (param.defaultValue) - defaultValue = clone(*param.defaultValue, dest, seenTypes, seenTypePacks, cloneState); + defaultValue = clone(*param.defaultValue, dest, cloneState); result.typePackParams.push_back({tp, defaultValue}); } - result.type = clone(typeFun.type, dest, seenTypes, seenTypePacks, cloneState); + result.type = clone(typeFun.type, dest, cloneState); return result; } diff --git a/Analysis/src/Error.cpp b/Analysis/src/Error.cpp index 5eb2ea2a..cbec0b15 100644 --- a/Analysis/src/Error.cpp +++ b/Analysis/src/Error.cpp @@ -8,7 +8,6 @@ #include -LUAU_FASTFLAGVARIABLE(BetterDiagnosticCodesInStudio, false); LUAU_FASTFLAGVARIABLE(LuauTypeMismatchModuleName, false); static std::string wrongNumberOfArgsString(size_t expectedCount, size_t actualCount, const char* argPrefix = nullptr, bool isVariadic = false) @@ -252,14 +251,7 @@ struct ErrorConverter std::string operator()(const Luau::SyntaxError& e) const { - if (FFlag::BetterDiagnosticCodesInStudio) - { - return e.message; - } - else - { - return "Syntax error: " + e.message; - } + return e.message; } std::string operator()(const Luau::CodeTooComplex&) const @@ -451,6 +443,11 @@ struct ErrorConverter { return "Cannot cast '" + toString(e.left) + "' into '" + toString(e.right) + "' because the types are unrelated"; } + + std::string operator()(const NormalizationTooComplex&) const + { + return "Code is too complex to typecheck! Consider simplifying the code around this area"; + } }; struct InvalidNameChecker @@ -716,14 +713,14 @@ bool containsParseErrorName(const TypeError& error) } template -void copyError(T& e, TypeArena& destArena, SeenTypes& seenTypes, SeenTypePacks& seenTypePacks, CloneState cloneState) +void copyError(T& e, TypeArena& destArena, CloneState cloneState) { auto clone = [&](auto&& ty) { - return ::Luau::clone(ty, destArena, seenTypes, seenTypePacks, cloneState); + return ::Luau::clone(ty, destArena, cloneState); }; auto visitErrorData = [&](auto&& e) { - copyError(e, destArena, seenTypes, seenTypePacks, cloneState); + copyError(e, destArena, cloneState); }; if constexpr (false) @@ -844,18 +841,19 @@ void copyError(T& e, TypeArena& destArena, SeenTypes& seenTypes, SeenTypePacks& e.left = clone(e.left); e.right = clone(e.right); } + else if constexpr (std::is_same_v) + { + } else static_assert(always_false_v, "Non-exhaustive type switch"); } void copyErrors(ErrorVec& errors, TypeArena& destArena) { - SeenTypes seenTypes; - SeenTypePacks seenTypePacks; CloneState cloneState; auto visitErrorData = [&](auto&& e) { - copyError(e, destArena, seenTypes, seenTypePacks, cloneState); + copyError(e, destArena, cloneState); }; LUAU_ASSERT(!destArena.typeVars.isFrozen()); diff --git a/Analysis/src/Frontend.cpp b/Analysis/src/Frontend.cpp index 000769fe..8b0b2210 100644 --- a/Analysis/src/Frontend.cpp +++ b/Analysis/src/Frontend.cpp @@ -11,16 +11,18 @@ #include "Luau/TimeTrace.h" #include "Luau/TypeInfer.h" #include "Luau/Variant.h" -#include "Luau/Common.h" #include #include #include +LUAU_FASTINT(LuauTypeInferIterationLimit) +LUAU_FASTINT(LuauTarjanChildLimit) LUAU_FASTFLAG(LuauCyclicModuleTypeSurface) LUAU_FASTFLAG(LuauInferInNoCheckMode) LUAU_FASTFLAGVARIABLE(LuauKnowsTheDataModel3, false) LUAU_FASTFLAGVARIABLE(LuauSeparateTypechecks, false) +LUAU_FASTFLAGVARIABLE(LuauAutocompleteDynamicLimits, false) LUAU_FASTINTVARIABLE(LuauAutocompleteCheckTimeoutMs, 0) namespace Luau @@ -97,13 +99,11 @@ LoadDefinitionFileResult loadDefinitionFile(TypeChecker& typeChecker, ScopePtr t if (checkedModule->errors.size() > 0) return LoadDefinitionFileResult{false, parseResult, checkedModule}; - SeenTypes seenTypes; - SeenTypePacks seenTypePacks; CloneState cloneState; for (const auto& [name, ty] : checkedModule->declaredGlobals) { - TypeId globalTy = clone(ty, typeChecker.globalTypes, seenTypes, seenTypePacks, cloneState); + TypeId globalTy = clone(ty, typeChecker.globalTypes, cloneState); std::string documentationSymbol = packageName + "/global/" + name; generateDocumentationSymbols(globalTy, documentationSymbol); targetScope->bindings[typeChecker.globalNames.names->getOrAdd(name.c_str())] = {globalTy, Location(), false, {}, documentationSymbol}; @@ -113,7 +113,7 @@ LoadDefinitionFileResult loadDefinitionFile(TypeChecker& typeChecker, ScopePtr t for (const auto& [name, ty] : checkedModule->getModuleScope()->exportedTypeBindings) { - TypeFun globalTy = clone(ty, typeChecker.globalTypes, seenTypes, seenTypePacks, cloneState); + TypeFun globalTy = clone(ty, typeChecker.globalTypes, cloneState); std::string documentationSymbol = packageName + "/globaltype/" + name; generateDocumentationSymbols(globalTy.type, documentationSymbol); targetScope->exportedTypeBindings[name] = globalTy; @@ -440,13 +440,42 @@ CheckResult Frontend::check(const ModuleName& name, std::optional 0) + typeCheckerForAutocomplete.instantiationChildLimit = + std::max(1, int(FInt::LuauTarjanChildLimit * sourceNode.autocompleteLimitsMult)); + else + typeCheckerForAutocomplete.instantiationChildLimit = std::nullopt; + + if (FInt::LuauTypeInferIterationLimit > 0) + typeCheckerForAutocomplete.unifierIterationLimit = + std::max(1, int(FInt::LuauTypeInferIterationLimit * sourceNode.autocompleteLimitsMult)); + else + typeCheckerForAutocomplete.unifierIterationLimit = std::nullopt; + } + ModulePtr moduleForAutocomplete = typeCheckerForAutocomplete.check(sourceModule, Mode::Strict); moduleResolverForAutocomplete.modules[moduleName] = moduleForAutocomplete; + double duration = getTimestamp() - timestamp; + if (moduleForAutocomplete->timeout) + { checkResult.timeoutHits.push_back(moduleName); - stats.timeCheck += getTimestamp() - timestamp; + if (FFlag::LuauAutocompleteDynamicLimits) + sourceNode.autocompleteLimitsMult = sourceNode.autocompleteLimitsMult / 2.0; + } + else if (FFlag::LuauAutocompleteDynamicLimits && duration < autocompleteTimeLimit / 2.0) + { + sourceNode.autocompleteLimitsMult = std::min(sourceNode.autocompleteLimitsMult * 2.0, 1.0); + } + + stats.timeCheck += duration; stats.filesStrict += 1; sourceNode.dirtyAutocomplete = false; diff --git a/Analysis/src/IostreamHelpers.cpp b/Analysis/src/IostreamHelpers.cpp index a8f67589..0eaa485e 100644 --- a/Analysis/src/IostreamHelpers.cpp +++ b/Analysis/src/IostreamHelpers.cpp @@ -184,6 +184,8 @@ static void errorToString(std::ostream& stream, const T& err) } else if constexpr (std::is_same_v) stream << "TypesAreUnrelated { left = '" + toString(err.left) + "', right = '" + toString(err.right) + "' }"; + else if constexpr (std::is_same_v) + stream << "NormalizationTooComplex { }"; else static_assert(always_false_v, "Non-exhaustive type switch"); } diff --git a/Analysis/src/JsonEncoder.cpp b/Analysis/src/JsonEncoder.cpp index 811e7c24..829ffa02 100644 --- a/Analysis/src/JsonEncoder.cpp +++ b/Analysis/src/JsonEncoder.cpp @@ -403,35 +403,26 @@ struct AstJsonEncoder : public AstVisitor void write(const AstExprTable::Item& item) { writeRaw("{"); - bool comma = pushComma(); + bool c = pushComma(); write("kind", item.kind); switch (item.kind) { case AstExprTable::Item::List: - write(item.value); + write("value", item.value); break; default: - write(item.key); - writeRaw(","); - write(item.value); + write("key", item.key); + write("value", item.value); break; } - popComma(comma); + popComma(c); writeRaw("}"); } void write(class AstExprTable* node) { writeNode(node, "AstExprTable", [&]() { - bool comma = false; - for (const auto& prop : node->items) - { - if (comma) - writeRaw(","); - else - comma = true; - write(prop); - } + PROP(items); }); } diff --git a/Analysis/src/LValue.cpp b/Analysis/src/LValue.cpp index c9466a40..72555ab4 100644 --- a/Analysis/src/LValue.cpp +++ b/Analysis/src/LValue.cpp @@ -5,6 +5,8 @@ #include +LUAU_FASTFLAG(LuauTypecheckOptPass) + namespace Luau { @@ -79,6 +81,8 @@ std::optional tryGetLValue(const AstExpr& node) std::pair> getFullName(const LValue& lvalue) { + LUAU_ASSERT(!FFlag::LuauTypecheckOptPass); + const LValue* current = &lvalue; std::vector keys; while (auto field = get(*current)) @@ -92,6 +96,19 @@ std::pair> getFullName(const LValue& lvalue) return {*symbol, std::vector(keys.rbegin(), keys.rend())}; } +Symbol getBaseSymbol(const LValue& lvalue) +{ + LUAU_ASSERT(FFlag::LuauTypecheckOptPass); + + const LValue* current = &lvalue; + while (auto field = get(*current)) + current = baseof(*current); + + const Symbol* symbol = get(*current); + LUAU_ASSERT(symbol); + return *symbol; +} + void merge(RefinementMap& l, const RefinementMap& r, std::function f) { for (const auto& [k, a] : r) diff --git a/Analysis/src/Linter.cpp b/Analysis/src/Linter.cpp index b7480e34..5608e4b3 100644 --- a/Analysis/src/Linter.cpp +++ b/Analysis/src/Linter.cpp @@ -14,7 +14,6 @@ LUAU_FASTINTVARIABLE(LuauSuggestionDistance, 4) LUAU_FASTFLAGVARIABLE(LuauLintGlobalNeverReadBeforeWritten, false) -LUAU_FASTFLAGVARIABLE(LuauLintNoRobloxBits, false) namespace Luau { @@ -1140,25 +1139,8 @@ private: Kind_Primitive, // primitive type supported by VM - boolean/userdata/etc. No differentiation between types of userdata. Kind_Vector, // 'vector' but only used when type is used Kind_Userdata, // custom userdata type - - // TODO: remove these with LuauLintNoRobloxBits - Kind_Class, // custom userdata type that reflects Roblox Instance-derived hierarchy - Part/etc. - Kind_Enum, // custom userdata type referring to an enum item of enum classes, e.g. Enum.NormalId.Back/Enum.Axis.X/etc. }; - bool containsPropName(TypeId ty, const std::string& propName) - { - LUAU_ASSERT(!FFlag::LuauLintNoRobloxBits); - - if (auto ctv = get(ty)) - return lookupClassProp(ctv, propName) != nullptr; - - if (auto ttv = get(ty)) - return ttv->props.find(propName) != ttv->props.end(); - - return false; - } - TypeKind getTypeKind(const std::string& name) { if (name == "nil" || name == "boolean" || name == "userdata" || name == "number" || name == "string" || name == "table" || @@ -1168,23 +1150,10 @@ private: if (name == "vector") return Kind_Vector; - if (FFlag::LuauLintNoRobloxBits) - { - if (std::optional maybeTy = context->scope->lookupType(name)) - return Kind_Userdata; + if (std::optional maybeTy = context->scope->lookupType(name)) + return Kind_Userdata; - return Kind_Unknown; - } - else - { - if (std::optional maybeTy = context->scope->lookupType(name)) - // Kind_Userdata is probably not 100% precise but is close enough - return containsPropName(maybeTy->type, "ClassName") ? Kind_Class : Kind_Userdata; - else if (std::optional maybeTy = context->scope->lookupImportedType("Enum", name)) - return Kind_Enum; - - return Kind_Unknown; - } + return Kind_Unknown; } void validateType(AstExprConstantString* expr, std::initializer_list expected, const char* expectedString) @@ -1202,67 +1171,11 @@ private: { if (kind == ek) return; - - // as a special case, Instance and EnumItem are both a userdata type (as returned by typeof) and a class type - if (!FFlag::LuauLintNoRobloxBits && ek == Kind_Userdata && (name == "Instance" || name == "EnumItem")) - return; } emitWarning(*context, LintWarning::Code_UnknownType, expr->location, "Unknown type '%s' (expected %s)", name.c_str(), expectedString); } - bool acceptsClassName(AstName method) - { - LUAU_ASSERT(!FFlag::LuauLintNoRobloxBits); - - return method.value[0] == 'F' && (method == "FindFirstChildOfClass" || method == "FindFirstChildWhichIsA" || - method == "FindFirstAncestorOfClass" || method == "FindFirstAncestorWhichIsA"); - } - - bool visit(AstExprCall* node) override - { - // TODO: Simply remove the override - if (FFlag::LuauLintNoRobloxBits) - return true; - - if (AstExprIndexName* index = node->func->as()) - { - AstExprConstantString* arg0 = node->args.size > 0 ? node->args.data[0]->as() : NULL; - - if (arg0) - { - if (node->self && index->index == "IsA" && node->args.size == 1) - { - validateType(arg0, {Kind_Class, Kind_Enum}, "class or enum type"); - } - else if (node->self && (index->index == "GetService" || index->index == "FindService") && node->args.size == 1) - { - AstExprGlobal* g = index->expr->as(); - - if (g && (g->name == "game" || g->name == "Game")) - { - validateType(arg0, {Kind_Class}, "class type"); - } - } - else if (node->self && acceptsClassName(index->index) && node->args.size == 1) - { - validateType(arg0, {Kind_Class}, "class type"); - } - else if (!node->self && index->index == "new" && node->args.size <= 2) - { - AstExprGlobal* g = index->expr->as(); - - if (g && g->name == "Instance") - { - validateType(arg0, {Kind_Class}, "class type"); - } - } - } - } - - return true; - } - bool visit(AstExprBinary* node) override { if (node->op == AstExprBinary::CompareNe || node->op == AstExprBinary::CompareEq) diff --git a/Analysis/src/Module.cpp b/Analysis/src/Module.cpp index 6bb45245..e2e3b436 100644 --- a/Analysis/src/Module.cpp +++ b/Analysis/src/Module.cpp @@ -1,8 +1,9 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #include "Luau/Module.h" -#include "Luau/Common.h" #include "Luau/Clone.h" +#include "Luau/Common.h" +#include "Luau/Normalize.h" #include "Luau/RecursionCounter.h" #include "Luau/Scope.h" #include "Luau/TypeInfer.h" @@ -14,6 +15,7 @@ LUAU_FASTFLAGVARIABLE(DebugLuauFreezeArena, false) LUAU_FASTFLAGVARIABLE(LuauCloneDeclaredGlobals, false) +LUAU_FASTFLAG(LuauLowerBoundsCalculation) namespace Luau { @@ -143,32 +145,51 @@ Module::~Module() unfreeze(internalTypes); } -bool Module::clonePublicInterface() +bool Module::clonePublicInterface(InternalErrorReporter& ice) { LUAU_ASSERT(interfaceTypes.typeVars.empty()); LUAU_ASSERT(interfaceTypes.typePacks.empty()); - SeenTypes seenTypes; - SeenTypePacks seenTypePacks; CloneState cloneState; ScopePtr moduleScope = getModuleScope(); - moduleScope->returnType = clone(moduleScope->returnType, interfaceTypes, seenTypes, seenTypePacks, cloneState); + moduleScope->returnType = clone(moduleScope->returnType, interfaceTypes, cloneState); if (moduleScope->varargPack) - moduleScope->varargPack = clone(*moduleScope->varargPack, interfaceTypes, seenTypes, seenTypePacks, cloneState); + moduleScope->varargPack = clone(*moduleScope->varargPack, interfaceTypes, cloneState); + + if (FFlag::LuauLowerBoundsCalculation) + { + normalize(moduleScope->returnType, interfaceTypes, ice); + if (moduleScope->varargPack) + normalize(*moduleScope->varargPack, interfaceTypes, ice); + } for (auto& [name, tf] : moduleScope->exportedTypeBindings) - tf = clone(tf, interfaceTypes, seenTypes, seenTypePacks, cloneState); + { + tf = clone(tf, interfaceTypes, cloneState); + if (FFlag::LuauLowerBoundsCalculation) + normalize(tf.type, interfaceTypes, ice); + } for (TypeId ty : moduleScope->returnType) + { if (get(follow(ty))) - *asMutable(ty) = AnyTypeVar{}; + { + auto t = asMutable(ty); + t->ty = AnyTypeVar{}; + t->normal = true; + } + } if (FFlag::LuauCloneDeclaredGlobals) { for (auto& [name, ty] : declaredGlobals) - ty = clone(ty, interfaceTypes, seenTypes, seenTypePacks, cloneState); + { + ty = clone(ty, interfaceTypes, cloneState); + if (FFlag::LuauLowerBoundsCalculation) + normalize(ty, interfaceTypes, ice); + } } freeze(internalTypes); diff --git a/Analysis/src/Normalize.cpp b/Analysis/src/Normalize.cpp new file mode 100644 index 00000000..40341ac1 --- /dev/null +++ b/Analysis/src/Normalize.cpp @@ -0,0 +1,814 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details + +#include "Luau/Normalize.h" + +#include + +#include "Luau/Clone.h" +#include "Luau/DenseHash.h" +#include "Luau/Substitution.h" +#include "Luau/Unifier.h" +#include "Luau/VisitTypeVar.h" + +LUAU_FASTFLAGVARIABLE(DebugLuauCopyBeforeNormalizing, false) + +// This could theoretically be 2000 on amd64, but x86 requires this. +LUAU_FASTINTVARIABLE(LuauNormalizeIterationLimit, 1200); +LUAU_FASTFLAGVARIABLE(LuauNormalizeCombineTableFix, false); +LUAU_FASTFLAGVARIABLE(LuauNormalizeCombineIntersectionFix, false); + +namespace Luau +{ + +namespace +{ + +struct Replacer : Substitution +{ + TypeId sourceType; + TypeId replacedType; + DenseHashMap replacedTypes{nullptr}; + DenseHashMap replacedPacks{nullptr}; + + Replacer(TypeArena* arena, TypeId sourceType, TypeId replacedType) + : Substitution(TxnLog::empty(), arena) + , sourceType(sourceType) + , replacedType(replacedType) + { + } + + bool isDirty(TypeId ty) override + { + if (!sourceType) + return false; + + auto vecHasSourceType = [sourceType = sourceType](const auto& vec) { + return end(vec) != std::find(begin(vec), end(vec), sourceType); + }; + + // Walk every kind of TypeVar and find pointers to sourceType + if (auto t = get(ty)) + return false; + else if (auto t = get(ty)) + return false; + else if (auto t = get(ty)) + return false; + else if (auto t = get(ty)) + return false; + else if (auto t = get(ty)) + return vecHasSourceType(t->parts); + else if (auto t = get(ty)) + return false; + else if (auto t = get(ty)) + { + if (vecHasSourceType(t->generics)) + return true; + + return false; + } + else if (auto t = get(ty)) + { + if (t->boundTo) + return *t->boundTo == sourceType; + + for (const auto& [_name, prop] : t->props) + { + if (prop.type == sourceType) + return true; + } + + if (auto indexer = t->indexer) + { + if (indexer->indexType == sourceType || indexer->indexResultType == sourceType) + return true; + } + + if (vecHasSourceType(t->instantiatedTypeParams)) + return true; + + return false; + } + else if (auto t = get(ty)) + return t->table == sourceType || t->metatable == sourceType; + else if (auto t = get(ty)) + return false; + else if (auto t = get(ty)) + return false; + else if (auto t = get(ty)) + return vecHasSourceType(t->options); + else if (auto t = get(ty)) + return vecHasSourceType(t->parts); + else if (auto t = get(ty)) + return false; + + LUAU_ASSERT(!"Luau::Replacer::isDirty internal error: Unknown TypeVar type"); + LUAU_UNREACHABLE(); + } + + bool isDirty(TypePackId tp) override + { + if (auto it = replacedPacks.find(tp)) + return false; + + if (auto pack = get(tp)) + { + for (TypeId ty : pack->head) + { + if (ty == sourceType) + return true; + } + return false; + } + else if (auto vtp = get(tp)) + return vtp->ty == sourceType; + else + return false; + } + + TypeId clean(TypeId ty) override + { + LUAU_ASSERT(sourceType && replacedType); + + // Walk every kind of TypeVar and create a copy with sourceType replaced by replacedType + // Before returning, memoize the result for later use. + + // Helpfully, Substitution::clone() only shallow-clones the kinds of types that we care to work with. This + // function returns the identity for things like primitives. + TypeId res = clone(ty); + + if (auto t = get(res)) + LUAU_ASSERT(!"Impossible"); + else if (auto t = get(res)) + LUAU_ASSERT(!"Impossible"); + else if (auto t = get(res)) + LUAU_ASSERT(!"Impossible"); + else if (auto t = get(res)) + LUAU_ASSERT(!"Impossible"); + else if (auto t = getMutable(res)) + { + for (TypeId& part : t->parts) + { + if (part == sourceType) + part = replacedType; + } + } + else if (auto t = get(res)) + LUAU_ASSERT(!"Impossible"); + else if (auto t = getMutable(res)) + { + // The constituent typepacks are cleaned separately. We just need to walk the generics array. + for (TypeId& g : t->generics) + { + if (g == sourceType) + g = replacedType; + } + } + else if (auto t = getMutable(res)) + { + for (auto& [_key, prop] : t->props) + { + if (prop.type == sourceType) + prop.type = replacedType; + } + } + else if (auto t = getMutable(res)) + { + if (t->table == sourceType) + t->table = replacedType; + if (t->metatable == sourceType) + t->table = replacedType; + } + else if (auto t = get(res)) + LUAU_ASSERT(!"Impossible"); + else if (auto t = get(res)) + LUAU_ASSERT(!"Impossible"); + else if (auto t = getMutable(res)) + { + for (TypeId& option : t->options) + { + if (option == sourceType) + option = replacedType; + } + } + else if (auto t = getMutable(res)) + { + for (TypeId& part : t->parts) + { + if (part == sourceType) + part = replacedType; + } + } + else if (auto t = get(res)) + LUAU_ASSERT(!"Impossible"); + else + LUAU_ASSERT(!"Luau::Replacer::clean internal error: Unknown TypeVar type"); + + replacedTypes[ty] = res; + return res; + } + + TypePackId clean(TypePackId tp) override + { + TypePackId res = clone(tp); + + if (auto pack = getMutable(res)) + { + for (TypeId& type : pack->head) + { + if (type == sourceType) + type = replacedType; + } + } + else if (auto vtp = getMutable(res)) + { + if (vtp->ty == sourceType) + vtp->ty = replacedType; + } + + replacedPacks[tp] = res; + return res; + } + + TypeId smartClone(TypeId t) + { + std::optional res = replace(t); + LUAU_ASSERT(res.has_value()); // TODO think about this + if (*res == t) + return clone(t); + return *res; + } +}; + +} // anonymous namespace + +bool isSubtype(TypeId subTy, TypeId superTy, InternalErrorReporter& ice) +{ + UnifierSharedState sharedState{&ice}; + TypeArena arena; + Unifier u{&arena, Mode::Strict, Location{}, Covariant, sharedState}; + u.anyIsTop = true; + + u.tryUnify(subTy, superTy); + const bool ok = u.errors.empty() && u.log.empty(); + return ok; +} + +template +static bool areNormal_(const T& t, const DenseHashSet& seen, InternalErrorReporter& ice) +{ + int count = 0; + auto isNormal = [&](TypeId ty) { + ++count; + if (count >= FInt::LuauNormalizeIterationLimit) + ice.ice("Luau::areNormal hit iteration limit"); + + return ty->normal || seen.find(asMutable(ty)); + }; + + return std::all_of(begin(t), end(t), isNormal); +} + +static bool areNormal(const std::vector& types, const DenseHashSet& seen, InternalErrorReporter& ice) +{ + return areNormal_(types, seen, ice); +} + +static bool areNormal(TypePackId tp, const DenseHashSet& seen, InternalErrorReporter& ice) +{ + tp = follow(tp); + if (get(tp)) + return false; + + auto [head, tail] = flatten(tp); + + if (!areNormal_(head, seen, ice)) + return false; + + if (!tail) + return true; + + if (auto vtp = get(*tail)) + return vtp->ty->normal || seen.find(asMutable(vtp->ty)); + + return true; +} + +#define CHECK_ITERATION_LIMIT(...) \ + do \ + { \ + if (iterationLimit > FInt::LuauNormalizeIterationLimit) \ + { \ + limitExceeded = true; \ + return __VA_ARGS__; \ + } \ + ++iterationLimit; \ + } while (false) + +struct Normalize +{ + TypeArena& arena; + InternalErrorReporter& ice; + + // Debug data. Types being normalized are invalidated but trying to see what's going on is painful. + // To actually see the original type, read it by using the pointer of the type being normalized. + // e.g. in lldb, `e dump(originalTys[ty])`. + SeenTypes originalTys; + SeenTypePacks originalTps; + + int iterationLimit = 0; + bool limitExceeded = false; + + template + bool operator()(TypePackId, const T&) + { + return true; + } + + template + void cycle(TID) + { + } + + bool operator()(TypeId ty, const FreeTypeVar&) + { + LUAU_ASSERT(!ty->normal); + return false; + } + + bool operator()(TypeId ty, const BoundTypeVar& btv) + { + // It should never be the case that this TypeVar is normal, but is bound to a non-normal type. + LUAU_ASSERT(!ty->normal || ty->normal == btv.boundTo->normal); + + asMutable(ty)->normal = btv.boundTo->normal; + return !ty->normal; + } + + bool operator()(TypeId ty, const PrimitiveTypeVar&) + { + LUAU_ASSERT(ty->normal); + return false; + } + + bool operator()(TypeId ty, const GenericTypeVar&) + { + if (!ty->normal) + asMutable(ty)->normal = true; + + return false; + } + + bool operator()(TypeId ty, const ErrorTypeVar&) + { + if (!ty->normal) + asMutable(ty)->normal = true; + return false; + } + + bool operator()(TypeId ty, const ConstrainedTypeVar& ctvRef, DenseHashSet& seen) + { + CHECK_ITERATION_LIMIT(false); + + ConstrainedTypeVar* ctv = const_cast(&ctvRef); + + std::vector parts = std::move(ctv->parts); + + // We might transmute, so it's not safe to rely on the builtin traversal logic of visitTypeVar + for (TypeId part : parts) + visit_detail::visit(part, *this, seen); + + std::vector newParts = normalizeUnion(parts); + + const bool normal = areNormal(newParts, seen, ice); + + if (newParts.size() == 1) + *asMutable(ty) = BoundTypeVar{newParts[0]}; + else + *asMutable(ty) = UnionTypeVar{std::move(newParts)}; + + asMutable(ty)->normal = normal; + + return false; + } + + bool operator()(TypeId ty, const FunctionTypeVar& ftv) = delete; + bool operator()(TypeId ty, const FunctionTypeVar& ftv, DenseHashSet& seen) + { + CHECK_ITERATION_LIMIT(false); + + if (ty->normal) + return false; + + visit_detail::visit(ftv.argTypes, *this, seen); + visit_detail::visit(ftv.retType, *this, seen); + + asMutable(ty)->normal = areNormal(ftv.argTypes, seen, ice) && areNormal(ftv.retType, seen, ice); + + return false; + } + + bool operator()(TypeId ty, const TableTypeVar& ttv, DenseHashSet& seen) + { + CHECK_ITERATION_LIMIT(false); + + if (ty->normal) + return false; + + bool normal = true; + + auto checkNormal = [&](TypeId t) { + // if t is on the stack, it is possible that this type is normal. + // If t is not normal and it is not on the stack, this type is definitely not normal. + if (!t->normal && !seen.find(asMutable(t))) + normal = false; + }; + + if (ttv.boundTo) + { + visit_detail::visit(*ttv.boundTo, *this, seen); + asMutable(ty)->normal = (*ttv.boundTo)->normal; + return false; + } + + for (const auto& [_name, prop] : ttv.props) + { + visit_detail::visit(prop.type, *this, seen); + checkNormal(prop.type); + } + + if (ttv.indexer) + { + visit_detail::visit(ttv.indexer->indexType, *this, seen); + checkNormal(ttv.indexer->indexType); + visit_detail::visit(ttv.indexer->indexResultType, *this, seen); + checkNormal(ttv.indexer->indexResultType); + } + + asMutable(ty)->normal = normal; + + return false; + } + + bool operator()(TypeId ty, const MetatableTypeVar& mtv, DenseHashSet& seen) + { + CHECK_ITERATION_LIMIT(false); + + if (ty->normal) + return false; + + visit_detail::visit(mtv.table, *this, seen); + visit_detail::visit(mtv.metatable, *this, seen); + + asMutable(ty)->normal = mtv.table->normal && mtv.metatable->normal; + + return false; + } + + bool operator()(TypeId ty, const ClassTypeVar& ctv) + { + if (!ty->normal) + asMutable(ty)->normal = true; + return false; + } + + bool operator()(TypeId ty, const AnyTypeVar&) + { + LUAU_ASSERT(ty->normal); + return false; + } + + bool operator()(TypeId ty, const UnionTypeVar& utvRef, DenseHashSet& seen) + { + CHECK_ITERATION_LIMIT(false); + + if (ty->normal) + return false; + + UnionTypeVar* utv = &const_cast(utvRef); + std::vector options = std::move(utv->options); + + // We might transmute, so it's not safe to rely on the builtin traversal logic of visitTypeVar + for (TypeId option : options) + visit_detail::visit(option, *this, seen); + + std::vector newOptions = normalizeUnion(options); + + const bool normal = areNormal(newOptions, seen, ice); + + LUAU_ASSERT(!newOptions.empty()); + + if (newOptions.size() == 1) + *asMutable(ty) = BoundTypeVar{newOptions[0]}; + else + utv->options = std::move(newOptions); + + asMutable(ty)->normal = normal; + + return false; + } + + bool operator()(TypeId ty, const IntersectionTypeVar& itvRef, DenseHashSet& seen) + { + CHECK_ITERATION_LIMIT(false); + + if (ty->normal) + return false; + + IntersectionTypeVar* itv = &const_cast(itvRef); + + std::vector oldParts = std::move(itv->parts); + + for (TypeId part : oldParts) + visit_detail::visit(part, *this, seen); + + std::vector tables; + for (TypeId part : oldParts) + { + part = follow(part); + if (get(part)) + tables.push_back(part); + else + { + Replacer replacer{&arena, nullptr, nullptr}; // FIXME this is super super WEIRD + combineIntoIntersection(replacer, itv, part); + } + } + + // Don't allocate a new table if there's just one in the intersection. + if (tables.size() == 1) + itv->parts.push_back(tables[0]); + else if (!tables.empty()) + { + const TableTypeVar* first = get(tables[0]); + LUAU_ASSERT(first); + + TypeId newTable = arena.addType(TableTypeVar{first->state, first->level}); + TableTypeVar* ttv = getMutable(newTable); + for (TypeId part : tables) + { + // Intuition: If combineIntoTable() needs to clone a table, any references to 'part' are cyclic and need + // to be rewritten to point at 'newTable' in the clone. + Replacer replacer{&arena, part, newTable}; + combineIntoTable(replacer, ttv, part); + } + + itv->parts.push_back(newTable); + } + + asMutable(ty)->normal = areNormal(itv->parts, seen, ice); + + if (itv->parts.size() == 1) + { + TypeId part = itv->parts[0]; + *asMutable(ty) = BoundTypeVar{part}; + } + + return false; + } + + bool operator()(TypeId ty, const LazyTypeVar&) + { + return false; + } + + std::vector normalizeUnion(const std::vector& options) + { + if (options.size() == 1) + return options; + + std::vector result; + + for (TypeId part : options) + combineIntoUnion(result, part); + + return result; + } + + void combineIntoUnion(std::vector& result, TypeId ty) + { + ty = follow(ty); + if (auto utv = get(ty)) + { + for (TypeId t : utv) + combineIntoUnion(result, t); + return; + } + + for (TypeId& part : result) + { + if (isSubtype(ty, part, ice)) + return; // no need to do anything + else if (isSubtype(part, ty, ice)) + { + part = ty; // replace the less general type by the more general one + return; + } + } + + result.push_back(ty); + } + + /** + * @param replacer knows how to clone a type such that any recursive references point at the new containing type. + * @param result is an intersection that is safe for us to mutate in-place. + */ + void combineIntoIntersection(Replacer& replacer, IntersectionTypeVar* result, TypeId ty) + { + // Note: this check guards against running out of stack space + // so if you increase the size of a stack frame, you'll need to decrease the limit. + CHECK_ITERATION_LIMIT(); + + ty = follow(ty); + if (auto itv = get(ty)) + { + for (TypeId part : itv->parts) + combineIntoIntersection(replacer, result, part); + return; + } + + // Let's say that the last part of our result intersection is always a table, if any table is part of this intersection + if (get(ty)) + { + if (result->parts.empty()) + result->parts.push_back(arena.addType(TableTypeVar{TableState::Sealed, TypeLevel{}})); + + TypeId theTable = result->parts.back(); + + if (!get(FFlag::LuauNormalizeCombineIntersectionFix ? follow(theTable) : theTable)) + { + result->parts.push_back(arena.addType(TableTypeVar{TableState::Sealed, TypeLevel{}})); + theTable = result->parts.back(); + } + + TypeId newTable = replacer.smartClone(theTable); + result->parts.back() = newTable; + + combineIntoTable(replacer, getMutable(newTable), ty); + } + else if (auto ftv = get(ty)) + { + bool merged = false; + for (TypeId& part : result->parts) + { + if (isSubtype(part, ty, ice)) + { + merged = true; + break; // no need to do anything + } + else if (isSubtype(ty, part, ice)) + { + merged = true; + part = ty; // replace the less general type by the more general one + break; + } + } + + if (!merged) + result->parts.push_back(ty); + } + else + result->parts.push_back(ty); + } + + TableState combineTableStates(TableState lhs, TableState rhs) + { + if (lhs == rhs) + return lhs; + + if (lhs == TableState::Free || rhs == TableState::Free) + return TableState::Free; + + if (lhs == TableState::Unsealed || rhs == TableState::Unsealed) + return TableState::Unsealed; + + return lhs; + } + + /** + * @param replacer gives us a way to clone a type such that recursive references are rewritten to the new + * "containing" type. + * @param table always points into a table that is safe for us to mutate. + */ + void combineIntoTable(Replacer& replacer, TableTypeVar* table, TypeId ty) + { + // Note: this check guards against running out of stack space + // so if you increase the size of a stack frame, you'll need to decrease the limit. + CHECK_ITERATION_LIMIT(); + + LUAU_ASSERT(table); + + ty = follow(ty); + + TableTypeVar* tyTable = getMutable(ty); + LUAU_ASSERT(tyTable); + + for (const auto& [propName, prop] : tyTable->props) + { + if (auto it = table->props.find(propName); it != table->props.end()) + { + /** + * If we are going to recursively merge intersections of tables, we need to ensure that we never mutate + * a table that comes from somewhere else in the type graph. + * + * smarClone() does some nice things for us: It will perform a clone that is as shallow as possible + * while still rewriting any cyclic references back to the new 'root' table. + * + * replacer also keeps a mapping of types that have previously been copied, so we have the added + * advantage here of knowing that, whether or not a new copy was actually made, the resulting TypeVar is + * safe for us to mutate in-place. + */ + TypeId clone = replacer.smartClone(it->second.type); + it->second.type = combine(replacer, clone, prop.type); + } + else + table->props.insert({propName, prop}); + } + + table->state = combineTableStates(table->state, tyTable->state); + table->level = max(table->level, tyTable->level); + } + + /** + * @param a is always cloned by the caller. It is safe to mutate in-place. + * @param b will never be mutated. + */ + TypeId combine(Replacer& replacer, TypeId a, TypeId b) + { + if (FFlag::LuauNormalizeCombineTableFix && a == b) + return a; + + if (!get(a) && !get(a)) + { + if (!FFlag::LuauNormalizeCombineTableFix && a == b) + return a; + else + return arena.addType(IntersectionTypeVar{{a, b}}); + } + + if (auto itv = getMutable(a)) + { + combineIntoIntersection(replacer, itv, b); + return a; + } + else if (auto ttv = getMutable(a)) + { + if (FFlag::LuauNormalizeCombineTableFix && !get(follow(b))) + return arena.addType(IntersectionTypeVar{{a, b}}); + combineIntoTable(replacer, ttv, b); + return a; + } + + LUAU_ASSERT(!"Impossible"); + LUAU_UNREACHABLE(); + } +}; + +#undef CHECK_ITERATION_LIMIT + +/** + * @returns A tuple of TypeId and a success indicator. (true indicates that the normalization completed successfully) + */ +std::pair normalize(TypeId ty, TypeArena& arena, InternalErrorReporter& ice) +{ + CloneState state; + if (FFlag::DebugLuauCopyBeforeNormalizing) + (void)clone(ty, arena, state); + + Normalize n{arena, ice, std::move(state.seenTypes), std::move(state.seenTypePacks)}; + DenseHashSet seen{nullptr}; + visitTypeVarOnce(ty, n, seen); + + return {ty, !n.limitExceeded}; +} + +// TODO: Think about using a temporary arena and cloning types out of it so that we +// reclaim memory used by wantonly allocated intermediate types here. +// The main wrinkle here is that we don't want clone() to copy a type if the source and dest +// arena are the same. +std::pair normalize(TypeId ty, const ModulePtr& module, InternalErrorReporter& ice) +{ + return normalize(ty, module->internalTypes, ice); +} + +/** + * @returns A tuple of TypeId and a success indicator. (true indicates that the normalization completed successfully) + */ +std::pair normalize(TypePackId tp, TypeArena& arena, InternalErrorReporter& ice) +{ + CloneState state; + if (FFlag::DebugLuauCopyBeforeNormalizing) + (void)clone(tp, arena, state); + + Normalize n{arena, ice, std::move(state.seenTypes), std::move(state.seenTypePacks)}; + DenseHashSet seen{nullptr}; + visitTypeVarOnce(tp, n, seen); + + return {tp, !n.limitExceeded}; +} + +std::pair normalize(TypePackId tp, const ModulePtr& module, InternalErrorReporter& ice) +{ + return normalize(tp, module->internalTypes, ice); +} + +} // namespace Luau diff --git a/Analysis/src/Quantify.cpp b/Analysis/src/Quantify.cpp index 94e169f1..305f83ce 100644 --- a/Analysis/src/Quantify.cpp +++ b/Analysis/src/Quantify.cpp @@ -4,6 +4,8 @@ #include "Luau/VisitTypeVar.h" +LUAU_FASTFLAG(LuauTypecheckOptPass) + namespace Luau { @@ -12,6 +14,8 @@ struct Quantifier TypeLevel level; std::vector generics; std::vector genericPacks; + bool seenGenericType = false; + bool seenMutableType = false; Quantifier(TypeLevel level) : level(level) @@ -23,6 +27,9 @@ struct Quantifier bool operator()(TypeId ty, const FreeTypeVar& ftv) { + if (FFlag::LuauTypecheckOptPass) + seenMutableType = true; + if (!level.subsumes(ftv.level)) return false; @@ -44,17 +51,40 @@ struct Quantifier return true; } + bool operator()(TypeId ty, const ConstrainedTypeVar&) + { + return true; + } + bool operator()(TypeId ty, const TableTypeVar&) { TableTypeVar& ttv = *getMutable(ty); + if (FFlag::LuauTypecheckOptPass) + { + if (ttv.state == TableState::Generic) + seenGenericType = true; + + if (ttv.state == TableState::Free) + seenMutableType = true; + } + if (ttv.state == TableState::Sealed || ttv.state == TableState::Generic) return false; if (!level.subsumes(ttv.level)) + { + if (FFlag::LuauTypecheckOptPass && ttv.state == TableState::Unsealed) + seenMutableType = true; return false; + } if (ttv.state == TableState::Free) + { ttv.state = TableState::Generic; + + if (FFlag::LuauTypecheckOptPass) + seenGenericType = true; + } else if (ttv.state == TableState::Unsealed) ttv.state = TableState::Sealed; @@ -65,6 +95,9 @@ struct Quantifier bool operator()(TypePackId tp, const FreeTypePack& ftp) { + if (FFlag::LuauTypecheckOptPass) + seenMutableType = true; + if (!level.subsumes(ftp.level)) return false; @@ -84,6 +117,9 @@ void quantify(TypeId ty, TypeLevel level) LUAU_ASSERT(ftv); ftv->generics = q.generics; ftv->genericPacks = q.genericPacks; + + if (FFlag::LuauTypecheckOptPass && ftv->generics.empty() && ftv->genericPacks.empty() && !q.seenMutableType && !q.seenGenericType) + ftv->hasNoGenerics = true; } } // namespace Luau diff --git a/Analysis/src/Substitution.cpp b/Analysis/src/Substitution.cpp index 770c7a47..8648b21e 100644 --- a/Analysis/src/Substitution.cpp +++ b/Analysis/src/Substitution.cpp @@ -7,24 +7,36 @@ #include #include +LUAU_FASTFLAG(LuauLowerBoundsCalculation) LUAU_FASTINTVARIABLE(LuauTarjanChildLimit, 1000) +LUAU_FASTFLAG(LuauTypecheckOptPass) +LUAU_FASTFLAGVARIABLE(LuauSubstituteFollowNewTypes, false) namespace Luau { void Tarjan::visitChildren(TypeId ty, int index) { - ty = log->follow(ty); + if (FFlag::LuauTypecheckOptPass) + LUAU_ASSERT(ty == log->follow(ty)); + else + ty = log->follow(ty); if (ignoreChildren(ty)) return; - if (const FunctionTypeVar* ftv = log->getMutable(ty)) + if (FFlag::LuauTypecheckOptPass) + { + if (auto pty = log->pending(ty)) + ty = &pty->pending; + } + + if (const FunctionTypeVar* ftv = FFlag::LuauTypecheckOptPass ? get(ty) : log->getMutable(ty)) { visitChild(ftv->argTypes); visitChild(ftv->retType); } - else if (const TableTypeVar* ttv = log->getMutable(ty)) + else if (const TableTypeVar* ttv = FFlag::LuauTypecheckOptPass ? get(ty) : log->getMutable(ty)) { LUAU_ASSERT(!ttv->boundTo); for (const auto& [name, prop] : ttv->props) @@ -41,38 +53,52 @@ void Tarjan::visitChildren(TypeId ty, int index) for (TypePackId itp : ttv->instantiatedTypePackParams) visitChild(itp); } - else if (const MetatableTypeVar* mtv = log->getMutable(ty)) + else if (const MetatableTypeVar* mtv = FFlag::LuauTypecheckOptPass ? get(ty) : log->getMutable(ty)) { visitChild(mtv->table); visitChild(mtv->metatable); } - else if (const UnionTypeVar* utv = log->getMutable(ty)) + else if (const UnionTypeVar* utv = FFlag::LuauTypecheckOptPass ? get(ty) : log->getMutable(ty)) { for (TypeId opt : utv->options) visitChild(opt); } - else if (const IntersectionTypeVar* itv = log->getMutable(ty)) + else if (const IntersectionTypeVar* itv = FFlag::LuauTypecheckOptPass ? get(ty) : log->getMutable(ty)) { for (TypeId part : itv->parts) visitChild(part); } + else if (const ConstrainedTypeVar* ctv = get(ty)) + { + for (TypeId part : ctv->parts) + visitChild(part); + } } void Tarjan::visitChildren(TypePackId tp, int index) { - tp = log->follow(tp); + if (FFlag::LuauTypecheckOptPass) + LUAU_ASSERT(tp == log->follow(tp)); + else + tp = log->follow(tp); if (ignoreChildren(tp)) return; - if (const TypePack* tpp = log->getMutable(tp)) + if (FFlag::LuauTypecheckOptPass) + { + if (auto ptp = log->pending(tp)) + tp = &ptp->pending; + } + + if (const TypePack* tpp = FFlag::LuauTypecheckOptPass ? get(tp) : log->getMutable(tp)) { for (TypeId tv : tpp->head) visitChild(tv); if (tpp->tail) visitChild(*tpp->tail); } - else if (const VariadicTypePack* vtp = log->getMutable(tp)) + else if (const VariadicTypePack* vtp = FFlag::LuauTypecheckOptPass ? get(tp) : log->getMutable(tp)) { visitChild(vtp->ty); } @@ -80,7 +106,10 @@ void Tarjan::visitChildren(TypePackId tp, int index) std::pair Tarjan::indexify(TypeId ty) { - ty = log->follow(ty); + if (FFlag::LuauTypecheckOptPass) + LUAU_ASSERT(ty == log->follow(ty)); + else + ty = log->follow(ty); bool fresh = !typeToIndex.contains(ty); int& index = typeToIndex[ty]; @@ -98,7 +127,10 @@ std::pair Tarjan::indexify(TypeId ty) std::pair Tarjan::indexify(TypePackId tp) { - tp = log->follow(tp); + if (FFlag::LuauTypecheckOptPass) + LUAU_ASSERT(tp == log->follow(tp)); + else + tp = log->follow(tp); bool fresh = !packToIndex.contains(tp); int& index = packToIndex[tp]; @@ -141,7 +173,7 @@ TarjanResult Tarjan::loop() if (currEdge == -1) { ++childCount; - if (FInt::LuauTarjanChildLimit > 0 && FInt::LuauTarjanChildLimit < childCount) + if (childLimit > 0 && childLimit < childCount) return TarjanResult::TooManyChildren; stack.push_back(index); @@ -229,6 +261,9 @@ TarjanResult Tarjan::loop() TarjanResult Tarjan::visitRoot(TypeId ty) { childCount = 0; + if (childLimit == 0) + childLimit = FInt::LuauTarjanChildLimit; + ty = log->follow(ty); auto [index, fresh] = indexify(ty); @@ -239,6 +274,9 @@ TarjanResult Tarjan::visitRoot(TypeId ty) TarjanResult Tarjan::visitRoot(TypePackId tp) { childCount = 0; + if (childLimit == 0) + childLimit = FInt::LuauTarjanChildLimit; + tp = log->follow(tp); auto [index, fresh] = indexify(tp); @@ -347,7 +385,13 @@ TypeId Substitution::clone(TypeId ty) TypeId result = ty; - if (const FunctionTypeVar* ftv = log->getMutable(ty)) + if (FFlag::LuauTypecheckOptPass) + { + if (auto pty = log->pending(ty)) + ty = &pty->pending; + } + + if (const FunctionTypeVar* ftv = FFlag::LuauTypecheckOptPass ? get(ty) : log->getMutable(ty)) { FunctionTypeVar clone = FunctionTypeVar{ftv->level, ftv->argTypes, ftv->retType, ftv->definition, ftv->hasSelf}; clone.generics = ftv->generics; @@ -357,7 +401,7 @@ TypeId Substitution::clone(TypeId ty) clone.argNames = ftv->argNames; result = addType(std::move(clone)); } - else if (const TableTypeVar* ttv = log->getMutable(ty)) + else if (const TableTypeVar* ttv = FFlag::LuauTypecheckOptPass ? get(ty) : log->getMutable(ty)) { LUAU_ASSERT(!ttv->boundTo); TableTypeVar clone = TableTypeVar{ttv->props, ttv->indexer, ttv->level, ttv->state}; @@ -370,24 +414,29 @@ TypeId Substitution::clone(TypeId ty) clone.tags = ttv->tags; result = addType(std::move(clone)); } - else if (const MetatableTypeVar* mtv = log->getMutable(ty)) + else if (const MetatableTypeVar* mtv = FFlag::LuauTypecheckOptPass ? get(ty) : log->getMutable(ty)) { MetatableTypeVar clone = MetatableTypeVar{mtv->table, mtv->metatable}; clone.syntheticName = mtv->syntheticName; result = addType(std::move(clone)); } - else if (const UnionTypeVar* utv = log->getMutable(ty)) + else if (const UnionTypeVar* utv = FFlag::LuauTypecheckOptPass ? get(ty) : log->getMutable(ty)) { UnionTypeVar clone; clone.options = utv->options; result = addType(std::move(clone)); } - else if (const IntersectionTypeVar* itv = log->getMutable(ty)) + else if (const IntersectionTypeVar* itv = FFlag::LuauTypecheckOptPass ? get(ty) : log->getMutable(ty)) { IntersectionTypeVar clone; clone.parts = itv->parts; result = addType(std::move(clone)); } + else if (const ConstrainedTypeVar* ctv = get(ty)) + { + ConstrainedTypeVar clone{ctv->level, ctv->parts}; + result = addType(std::move(clone)); + } asMutable(result)->documentationSymbol = ty->documentationSymbol; return result; @@ -396,14 +445,21 @@ TypeId Substitution::clone(TypeId ty) TypePackId Substitution::clone(TypePackId tp) { tp = log->follow(tp); - if (const TypePack* tpp = log->getMutable(tp)) + + if (FFlag::LuauTypecheckOptPass) + { + if (auto ptp = log->pending(tp)) + tp = &ptp->pending; + } + + if (const TypePack* tpp = FFlag::LuauTypecheckOptPass ? get(tp) : log->getMutable(tp)) { TypePack clone; clone.head = tpp->head; clone.tail = tpp->tail; return addTypePack(std::move(clone)); } - else if (const VariadicTypePack* vtp = log->getMutable(tp)) + else if (const VariadicTypePack* vtp = FFlag::LuauTypecheckOptPass ? get(tp) : log->getMutable(tp)) { VariadicTypePack clone; clone.ty = vtp->ty; @@ -415,25 +471,34 @@ TypePackId Substitution::clone(TypePackId tp) void Substitution::foundDirty(TypeId ty) { - ty = log->follow(ty); - if (isDirty(ty)) - newTypes[ty] = clean(ty); + if (FFlag::LuauTypecheckOptPass) + LUAU_ASSERT(ty == log->follow(ty)); else - newTypes[ty] = clone(ty); + ty = log->follow(ty); + + if (isDirty(ty)) + newTypes[ty] = FFlag::LuauSubstituteFollowNewTypes ? follow(clean(ty)) : clean(ty); + else + newTypes[ty] = FFlag::LuauSubstituteFollowNewTypes ? follow(clone(ty)) : clone(ty); } void Substitution::foundDirty(TypePackId tp) { - tp = log->follow(tp); - if (isDirty(tp)) - newPacks[tp] = clean(tp); + if (FFlag::LuauTypecheckOptPass) + LUAU_ASSERT(tp == log->follow(tp)); else - newPacks[tp] = clone(tp); + tp = log->follow(tp); + + if (isDirty(tp)) + newPacks[tp] = FFlag::LuauSubstituteFollowNewTypes ? follow(clean(tp)) : clean(tp); + else + newPacks[tp] = FFlag::LuauSubstituteFollowNewTypes ? follow(clone(tp)) : clone(tp); } TypeId Substitution::replace(TypeId ty) { ty = log->follow(ty); + if (TypeId* prevTy = newTypes.find(ty)) return *prevTy; else @@ -443,6 +508,7 @@ TypeId Substitution::replace(TypeId ty) TypePackId Substitution::replace(TypePackId tp) { tp = log->follow(tp); + if (TypePackId* prevTp = newPacks.find(tp)) return *prevTp; else @@ -451,7 +517,13 @@ TypePackId Substitution::replace(TypePackId tp) void Substitution::replaceChildren(TypeId ty) { - ty = log->follow(ty); + if (BoundTypeVar* btv = log->getMutable(ty); FFlag::LuauLowerBoundsCalculation && btv) + btv->boundTo = replace(btv->boundTo); + + if (FFlag::LuauTypecheckOptPass) + LUAU_ASSERT(ty == log->follow(ty)); + else + ty = log->follow(ty); if (ignoreChildren(ty)) return; @@ -493,11 +565,19 @@ void Substitution::replaceChildren(TypeId ty) for (TypeId& part : itv->parts) part = replace(part); } + else if (ConstrainedTypeVar* ctv = getMutable(ty)) + { + for (TypeId& part : ctv->parts) + part = replace(part); + } } void Substitution::replaceChildren(TypePackId tp) { - tp = log->follow(tp); + if (FFlag::LuauTypecheckOptPass) + LUAU_ASSERT(tp == log->follow(tp)); + else + tp = log->follow(tp); if (ignoreChildren(tp)) return; diff --git a/Analysis/src/ToDot.cpp b/Analysis/src/ToDot.cpp index df9d4188..cb54bfc1 100644 --- a/Analysis/src/ToDot.cpp +++ b/Analysis/src/ToDot.cpp @@ -237,6 +237,15 @@ void StateDot::visitChildren(TypeId ty, int index) finishNodeLabel(ty); finishNode(); } + else if (const ConstrainedTypeVar* ctv = get(ty)) + { + formatAppend(result, "ConstrainedTypeVar %d", index); + finishNodeLabel(ty); + finishNode(); + + for (TypeId part : ctv->parts) + visitChild(part, index); + } else if (get(ty)) { formatAppend(result, "ErrorTypeVar %d", index); @@ -258,6 +267,28 @@ void StateDot::visitChildren(TypeId ty, int index) if (ctv->metatable) visitChild(*ctv->metatable, index, "[metatable]"); } + else if (const SingletonTypeVar* stv = get(ty)) + { + std::string res; + + if (const StringSingleton* ss = get(stv)) + { + // Don't put in quotes anywhere. If it's outside of the call to escape, + // then it's invalid syntax. If it's inside, then escaping is super noisy. + res = "string: " + escape(ss->value); + } + else if (const BooleanSingleton* bs = get(stv)) + { + res = "boolean: "; + res += bs->value ? "true" : "false"; + } + else + LUAU_ASSERT(!"unknown singleton type"); + + formatAppend(result, "SingletonTypeVar %s", res.c_str()); + finishNodeLabel(ty); + finishNode(); + } else { LUAU_ASSERT(!"unknown type kind"); diff --git a/Analysis/src/ToString.cpp b/Analysis/src/ToString.cpp index 59ee6de2..610842da 100644 --- a/Analysis/src/ToString.cpp +++ b/Analysis/src/ToString.cpp @@ -10,6 +10,8 @@ #include #include +LUAU_FASTFLAG(LuauLowerBoundsCalculation) + /* * Prefix generic typenames with gen- * Additionally, free types will be prefixed with free- and suffixed with their level. eg free-a-4 @@ -33,8 +35,8 @@ struct FindCyclicTypes bool exhaustive = false; std::unordered_set visited; std::unordered_set visitedPacks; - std::unordered_set cycles; - std::unordered_set cycleTPs; + std::set cycles; + std::set cycleTPs; void cycle(TypeId ty) { @@ -86,7 +88,7 @@ struct FindCyclicTypes }; template -void findCyclicTypes(std::unordered_set& cycles, std::unordered_set& cycleTPs, TID ty, bool exhaustive) +void findCyclicTypes(std::set& cycles, std::set& cycleTPs, TID ty, bool exhaustive) { FindCyclicTypes fct; fct.exhaustive = exhaustive; @@ -124,6 +126,7 @@ struct StringifierState std::unordered_map cycleTpNames; std::unordered_set seen; std::unordered_set usedNames; + size_t indentation = 0; bool exhaustive; @@ -216,6 +219,34 @@ struct StringifierState result.name += s; } + + void indent() + { + indentation += 4; + } + + void dedent() + { + indentation -= 4; + } + + void newline() + { + if (!opts.useLineBreaks) + return emit(" "); + + emit("\n"); + emitIndentation(); + } + +private: + void emitIndentation() + { + if (!opts.indent) + return; + + emit(std::string(indentation, ' ')); + } }; struct TypeVarStringifier @@ -321,7 +352,7 @@ struct TypeVarStringifier stringify(btv.boundTo); } - void operator()(TypeId ty, const Unifiable::Generic& gtv) + void operator()(TypeId ty, const GenericTypeVar& gtv) { if (gtv.explicitName) { @@ -332,6 +363,26 @@ struct TypeVarStringifier state.emit(state.getName(ty)); } + void operator()(TypeId, const ConstrainedTypeVar& ctv) + { + state.result.invalid = true; + + state.emit("[["); + + bool first = true; + for (TypeId ty : ctv.parts) + { + if (first) + first = false; + else + state.emit("|"); + + stringify(ty); + } + + state.emit("]]"); + } + void operator()(TypeId, const PrimitiveTypeVar& ptv) { switch (ptv.type) @@ -415,10 +466,25 @@ struct TypeVarStringifier state.emit(") -> "); bool plural = true; - if (auto retPack = get(follow(ftv.retType))) + + if (FFlag::LuauLowerBoundsCalculation) { - if (retPack->head.size() == 1 && !retPack->tail) - plural = false; + auto retBegin = begin(ftv.retType); + auto retEnd = end(ftv.retType); + if (retBegin != retEnd) + { + ++retBegin; + if (retBegin == retEnd && !retBegin.tail()) + plural = false; + } + } + else + { + if (auto retPack = get(follow(ftv.retType))) + { + if (retPack->head.size() == 1 && !retPack->tail) + plural = false; + } } if (plural) @@ -511,6 +577,7 @@ struct TypeVarStringifier } state.emit(openbrace); + state.indent(); bool comma = false; if (ttv.indexer) @@ -527,7 +594,10 @@ struct TypeVarStringifier for (const auto& [name, prop] : ttv.props) { if (comma) - state.emit(state.opts.useLineBreaks ? ",\n" : ", "); + { + state.emit(","); + state.newline(); + } size_t length = state.result.name.length() - oldLength; @@ -553,6 +623,7 @@ struct TypeVarStringifier ++index; } + state.dedent(); state.emit(closedbrace); state.unsee(&ttv); @@ -563,7 +634,8 @@ struct TypeVarStringifier state.result.invalid = true; state.emit("{ @metatable "); stringify(mtv.metatable); - state.emit(state.opts.useLineBreaks ? ",\n" : ", "); + state.emit(","); + state.newline(); stringify(mtv.table); state.emit(" }"); } @@ -784,13 +856,16 @@ struct TypePackStringifier if (tp.tail && !isEmpty(*tp.tail)) { - const auto& tail = *tp.tail; - if (first) - first = false; - else - state.emit(", "); + TypePackId tail = follow(*tp.tail); + if (auto vtp = get(tail); !vtp || (!FFlag::DebugLuauVerboseTypeNames && !vtp->hidden)) + { + if (first) + first = false; + else + state.emit(", "); - stringify(tail); + stringify(tail); + } } state.unsee(&tp); @@ -805,6 +880,8 @@ struct TypePackStringifier void operator()(TypePackId, const VariadicTypePack& pack) { state.emit("..."); + if (FFlag::DebugLuauVerboseTypeNames && pack.hidden) + state.emit(""); stringify(pack.ty); } @@ -858,15 +935,12 @@ void TypeVarStringifier::stringify(TypePackId tpid, const std::vector& cycles, const std::unordered_set& cycleTPs, +static void assignCycleNames(const std::set& cycles, const std::set& cycleTPs, std::unordered_map& cycleNames, std::unordered_map& cycleTpNames, bool exhaustive) { int nextIndex = 1; - std::vector sortedCycles{cycles.begin(), cycles.end()}; - std::sort(sortedCycles.begin(), sortedCycles.end(), std::less{}); - - for (TypeId cycleTy : sortedCycles) + for (TypeId cycleTy : cycles) { std::string name; @@ -888,10 +962,7 @@ static void assignCycleNames(const std::unordered_set& cycles, const std cycleNames[cycleTy] = std::move(name); } - std::vector sortedCycleTps{cycleTPs.begin(), cycleTPs.end()}; - std::sort(sortedCycleTps.begin(), sortedCycleTps.end(), std::less()); - - for (TypePackId tp : sortedCycleTps) + for (TypePackId tp : cycleTPs) { std::string name = "tp" + std::to_string(nextIndex); ++nextIndex; @@ -913,8 +984,8 @@ ToStringResult toStringDetailed(TypeId ty, const ToStringOptions& opts) StringifierState state{opts, result, opts.nameMap}; - std::unordered_set cycles; - std::unordered_set cycleTPs; + std::set cycles; + std::set cycleTPs; findCyclicTypes(cycles, cycleTPs, ty, opts.exhaustive); @@ -1016,8 +1087,8 @@ ToStringResult toStringDetailed(TypePackId tp, const ToStringOptions& opts) ToStringResult result; StringifierState state{opts, result, opts.nameMap}; - std::unordered_set cycles; - std::unordered_set cycleTPs; + std::set cycles; + std::set cycleTPs; findCyclicTypes(cycles, cycleTPs, tp, opts.exhaustive); @@ -1058,7 +1129,7 @@ ToStringResult toStringDetailed(TypePackId tp, const ToStringOptions& opts) state.emit(name); state.emit(" = "); Luau::visit( - [&tvs, cycleTy = cycleTy](auto&& t) { + [&tvs, cycleTy = cycleTy](auto t) { return tvs(cycleTy, t); }, cycleTy->ty); @@ -1163,14 +1234,18 @@ std::string toStringNamedFunction(const std::string& funcName, const FunctionTyp if (argPackIter.tail()) { - if (!first) - state.emit(", "); + if (auto vtp = get(*argPackIter.tail()); !vtp || !vtp->hidden) + { + if (!first) + state.emit(", "); - state.emit("...: "); - if (auto vtp = get(*argPackIter.tail())) - tvs.stringify(vtp->ty); - else - tvs.stringify(*argPackIter.tail()); + state.emit("...: "); + + if (vtp) + tvs.stringify(vtp->ty); + else + tvs.stringify(*argPackIter.tail()); + } } state.emit("): "); @@ -1210,6 +1285,24 @@ std::string dump(TypePackId ty) return s; } +std::string dump(const ScopePtr& scope, const char* name) +{ + auto binding = scope->linearSearchForBinding(name); + if (!binding) + { + printf("No binding %s\n", name); + return {}; + } + + TypeId ty = binding->typeId; + ToStringOptions opts; + opts.exhaustive = true; + opts.functionTypeArguments = true; + std::string s = toString(ty, opts); + printf("%s\n", s.c_str()); + return s; +} + std::string generateName(size_t i) { std::string n; diff --git a/Analysis/src/TopoSortStatements.cpp b/Analysis/src/TopoSortStatements.cpp index 678001bf..1ea2e27d 100644 --- a/Analysis/src/TopoSortStatements.cpp +++ b/Analysis/src/TopoSortStatements.cpp @@ -215,6 +215,7 @@ struct ArcCollector : public AstVisitor } } + // Adds a dependency from the current node to the named node. void add(const Identifier& name) { Node** it = map.find(name); diff --git a/Analysis/src/TxnLog.cpp b/Analysis/src/TxnLog.cpp index 5fbb596d..a5f9d26c 100644 --- a/Analysis/src/TxnLog.cpp +++ b/Analysis/src/TxnLog.cpp @@ -8,6 +8,7 @@ #include LUAU_FASTFLAGVARIABLE(LuauTxnLogPreserveOwner, false) +LUAU_FASTFLAGVARIABLE(LuauJustOneCallFrameForHaveSeen, false) namespace Luau { @@ -161,18 +162,37 @@ void TxnLog::popSeen(TypePackId lhs, TypePackId rhs) bool TxnLog::haveSeen(TypeOrPackId lhs, TypeOrPackId rhs) const { - const std::pair sortedPair = (lhs > rhs) ? std::make_pair(lhs, rhs) : std::make_pair(rhs, lhs); - if (sharedSeen->end() != std::find(sharedSeen->begin(), sharedSeen->end(), sortedPair)) + if (FFlag::LuauJustOneCallFrameForHaveSeen && !FFlag::LuauTypecheckOptPass) { - return true; - } + // This function will technically work if `this` is nullptr, but this + // indicates a bug, so we explicitly assert. + LUAU_ASSERT(static_cast(this) != nullptr); - if (parent) + const std::pair sortedPair = (lhs > rhs) ? std::make_pair(lhs, rhs) : std::make_pair(rhs, lhs); + + for (const TxnLog* current = this; current; current = current->parent) + { + if (current->sharedSeen->end() != std::find(current->sharedSeen->begin(), current->sharedSeen->end(), sortedPair)) + return true; + } + + return false; + } + else { - return parent->haveSeen(lhs, rhs); - } + const std::pair sortedPair = (lhs > rhs) ? std::make_pair(lhs, rhs) : std::make_pair(rhs, lhs); + if (sharedSeen->end() != std::find(sharedSeen->begin(), sharedSeen->end(), sortedPair)) + { + return true; + } - return false; + if (!FFlag::LuauTypecheckOptPass && parent) + { + return parent->haveSeen(lhs, rhs); + } + + return false; + } } void TxnLog::pushSeen(TypeOrPackId lhs, TypeOrPackId rhs) @@ -222,8 +242,8 @@ PendingType* TxnLog::pending(TypeId ty) const for (const TxnLog* current = this; current; current = current->parent) { - if (auto it = current->typeVarChanges.find(ty); it != current->typeVarChanges.end()) - return it->second.get(); + if (auto it = current->typeVarChanges.find(ty)) + return it->get(); } return nullptr; @@ -237,8 +257,8 @@ PendingTypePack* TxnLog::pending(TypePackId tp) const for (const TxnLog* current = this; current; current = current->parent) { - if (auto it = current->typePackChanges.find(tp); it != current->typePackChanges.end()) - return it->second.get(); + if (auto it = current->typePackChanges.find(tp)) + return it->get(); } return nullptr; diff --git a/Analysis/src/TypeAttach.cpp b/Analysis/src/TypeAttach.cpp index d575e023..bc8d0d4e 100644 --- a/Analysis/src/TypeAttach.cpp +++ b/Analysis/src/TypeAttach.cpp @@ -94,6 +94,16 @@ public: } } + AstType* operator()(const ConstrainedTypeVar& ctv) + { + AstArray types; + types.size = ctv.parts.size(); + types.data = static_cast(allocator->allocate(sizeof(AstType*) * ctv.parts.size())); + for (size_t i = 0; i < ctv.parts.size(); ++i) + types.data[i] = Luau::visit(*this, ctv.parts[i]->ty); + return allocator->alloc(Location(), types); + } + AstType* operator()(const SingletonTypeVar& stv) { if (const BooleanSingleton* bs = get(&stv)) @@ -364,6 +374,9 @@ public: AstTypePack* operator()(const VariadicTypePack& vtp) const { + if (vtp.hidden) + return nullptr; + return allocator->alloc(Location(), Luau::visit(*typeVisitor, vtp.ty->ty)); } diff --git a/Analysis/src/TypeInfer.cpp b/Analysis/src/TypeInfer.cpp index 10930248..af42a4e6 100644 --- a/Analysis/src/TypeInfer.cpp +++ b/Analysis/src/TypeInfer.cpp @@ -3,12 +3,15 @@ #include "Luau/Common.h" #include "Luau/ModuleResolver.h" +#include "Luau/Normalize.h" +#include "Luau/Parser.h" #include "Luau/Quantify.h" #include "Luau/RecursionCounter.h" #include "Luau/Scope.h" #include "Luau/Substitution.h" #include "Luau/TopoSortStatements.h" #include "Luau/TypePack.h" +#include "Luau/ToString.h" #include "Luau/TypeUtils.h" #include "Luau/ToString.h" #include "Luau/TypeVar.h" @@ -19,14 +22,17 @@ LUAU_FASTFLAGVARIABLE(DebugLuauMagicTypes, false) LUAU_FASTINTVARIABLE(LuauTypeInferRecursionLimit, 500) +LUAU_FASTINTVARIABLE(LuauTypeInferIterationLimit, 2000) LUAU_FASTINTVARIABLE(LuauTypeInferTypePackLoopLimit, 5000) LUAU_FASTINTVARIABLE(LuauCheckRecursionLimit, 500) LUAU_FASTFLAG(LuauKnowsTheDataModel3) LUAU_FASTFLAG(LuauSeparateTypechecks) +LUAU_FASTFLAG(LuauAutocompleteDynamicLimits) LUAU_FASTFLAG(LuauAutocompleteSingletonTypes) LUAU_FASTFLAGVARIABLE(LuauCyclicModuleTypeSurface, false) LUAU_FASTFLAGVARIABLE(LuauEqConstraint, false) LUAU_FASTFLAGVARIABLE(LuauWeakEqConstraint, false) // Eventually removed as false. +LUAU_FASTFLAGVARIABLE(LuauLowerBoundsCalculation, false) LUAU_FASTFLAGVARIABLE(DebugLuauFreezeDuringUnification, false) LUAU_FASTFLAGVARIABLE(LuauRecursiveTypeParameterRestriction, false) LUAU_FASTFLAGVARIABLE(LuauGenericFunctionsDontCacheTypeParams, false) @@ -39,6 +45,7 @@ LUAU_FASTFLAGVARIABLE(LuauErrorRecoveryType, false) LUAU_FASTFLAGVARIABLE(LuauOnlyMutateInstantiatedTables, false) LUAU_FASTFLAGVARIABLE(LuauPropertiesGetExpectedType, false) LUAU_FASTFLAGVARIABLE(LuauStatFunctionSimplify4, false) +LUAU_FASTFLAGVARIABLE(LuauTypecheckOptPass, false) LUAU_FASTFLAGVARIABLE(LuauUnsealedTableLiteral, false) LUAU_FASTFLAG(LuauTypeMismatchModuleName) LUAU_FASTFLAGVARIABLE(LuauTwoPassAliasDefinitionFix, false) @@ -53,6 +60,8 @@ LUAU_FASTFLAGVARIABLE(LuauCheckImplicitNumbericKeys, false) LUAU_FASTFLAG(LuauAnyInIsOptionalIsOptional) LUAU_FASTFLAGVARIABLE(LuauDecoupleOperatorInferenceFromUnifiedTypeInference, false) LUAU_FASTFLAGVARIABLE(LuauArgCountMismatchSaysAtLeastWhenVariadic, false) +LUAU_FASTFLAGVARIABLE(LuauTableUseCounterInstead, false) +LUAU_FASTFLAGVARIABLE(LuauRecursionLimitException, false); namespace Luau { @@ -140,6 +149,34 @@ bool hasBreak(AstStat* node) } } +static bool hasReturn(const AstStat* node) +{ + struct Searcher : AstVisitor + { + bool result = false; + + bool visit(AstStat*) override + { + return !result; // if we've already found a return statement, don't bother to traverse inward anymore + } + + bool visit(AstStatReturn*) override + { + result = true; + return false; + } + + bool visit(AstExprFunction*) override + { + return false; // We don't care if the function uses a lambda that itself returns + } + }; + + Searcher searcher; + const_cast(node)->visit(&searcher); + return searcher.result; +} + // returns the last statement before the block exits, or nullptr if the block never exits const AstStat* getFallthrough(const AstStat* node) { @@ -253,6 +290,26 @@ TypeChecker::TypeChecker(ModuleResolver* resolver, InternalErrorReporter* iceHan } ModulePtr TypeChecker::check(const SourceModule& module, Mode mode, std::optional environmentScope) +{ + if (FFlag::LuauRecursionLimitException) + { + try + { + return checkWithoutRecursionCheck(module, mode, environmentScope); + } + catch (const RecursionLimitException&) + { + reportErrorCodeTooComplex(module.root->location); + return std::move(currentModule); + } + } + else + { + return checkWithoutRecursionCheck(module, mode, environmentScope); + } +} + +ModulePtr TypeChecker::checkWithoutRecursionCheck(const SourceModule& module, Mode mode, std::optional environmentScope) { LUAU_TIMETRACE_SCOPE("TypeChecker::check", "TypeChecker"); LUAU_TIMETRACE_ARGUMENT("module", module.name.c_str()); @@ -268,6 +325,12 @@ ModulePtr TypeChecker::check(const SourceModule& module, Mode mode, std::optiona iceHandler->moduleName = module.name; + if (FFlag::LuauAutocompleteDynamicLimits) + { + unifierState.counters.recursionLimit = FInt::LuauTypeInferRecursionLimit; + unifierState.counters.iterationLimit = unifierIterationLimit ? *unifierIterationLimit : FInt::LuauTypeInferIterationLimit; + } + ScopePtr parentScope = environmentScope.value_or(globalScope); ScopePtr moduleScope = std::make_shared(parentScope); @@ -312,7 +375,7 @@ ModulePtr TypeChecker::check(const SourceModule& module, Mode mode, std::optiona prepareErrorsForDisplay(currentModule->errors); - bool encounteredFreeType = currentModule->clonePublicInterface(); + bool encounteredFreeType = currentModule->clonePublicInterface(*iceHandler); if (encounteredFreeType) { reportError(TypeError{module.root->location, @@ -415,7 +478,26 @@ void TypeChecker::checkBlock(const ScopePtr& scope, const AstStatBlock& block) reportErrorCodeTooComplex(block.location); return; } + if (FFlag::LuauRecursionLimitException) + { + try + { + checkBlockWithoutRecursionCheck(scope, block); + } + catch (const RecursionLimitException&) + { + reportErrorCodeTooComplex(block.location); + return; + } + } + else + { + checkBlockWithoutRecursionCheck(scope, block); + } +} +void TypeChecker::checkBlockWithoutRecursionCheck(const ScopePtr& scope, const AstStatBlock& block) +{ int subLevel = 0; std::vector sorted(block.body.data, block.body.data + block.body.size); @@ -435,6 +517,16 @@ void TypeChecker::checkBlock(const ScopePtr& scope, const AstStatBlock& block) std::unordered_map> functionDecls; + auto isLocalLambda = [](AstStat* stat) -> AstStatLocal* { + AstStatLocal* local = stat->as(); + + if (FFlag::LuauLowerBoundsCalculation && local && local->vars.size == 1 && local->values.size == 1 && + local->values.data[0]->is()) + return local; + else + return nullptr; + }; + auto checkBody = [&](AstStat* stat) { if (auto fun = stat->as()) { @@ -482,7 +574,7 @@ void TypeChecker::checkBlock(const ScopePtr& scope, const AstStatBlock& block) // function f(x:a):a local x: number = g(37) return x end // function g(x:number):number return f(x) end // ``` - if (containsFunctionCallOrReturn(**protoIter)) + if (containsFunctionCallOrReturn(**protoIter) || (FFlag::LuauLowerBoundsCalculation && isLocalLambda(*protoIter))) { while (checkIter != protoIter) { @@ -513,7 +605,8 @@ void TypeChecker::checkBlock(const ScopePtr& scope, const AstStatBlock& block) functionDecls[*protoIter] = pair; ++subLevel; - TypeId leftType = checkFunctionName(scope, *fun->name, funScope->level); + TypeId leftType = follow(checkFunctionName(scope, *fun->name, funScope->level)); + unify(funTy, leftType, fun->location); } else if (auto fun = (*protoIter)->as()) @@ -658,6 +751,16 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatRepeat& statement) checkExpr(repScope, *statement.condition); } +void TypeChecker::unifyLowerBound(TypePackId subTy, TypePackId superTy, const Location& location) +{ + Unifier state = mkUnifier(location); + state.unifyLowerBound(subTy, superTy); + + state.log.commit(); + + reportErrors(state.errors); +} + void TypeChecker::check(const ScopePtr& scope, const AstStatReturn& return_) { std::vector> expectedTypes; @@ -682,6 +785,12 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatReturn& return_) TypePackId retPack = checkExprList(scope, return_.location, return_.list, false, {}, expectedTypes).type; + if (useConstrainedIntersections()) + { + unifyLowerBound(retPack, scope->returnType, return_.location); + return; + } + // HACK: Nonstrict mode gets a bit too smart and strict for us when we // start typechecking everything across module boundaries. if (isNonstrictMode() && follow(scope->returnType) == follow(currentModule->getModuleScope()->returnType)) @@ -1209,9 +1318,11 @@ void TypeChecker::check(const ScopePtr& scope, TypeId ty, const ScopePtr& funSco else if (tableSelf->state == TableState::Sealed) reportError(TypeError{function.location, CannotExtendTable{selfTy, CannotExtendTable::Property, indexName->index.value}}); + const bool tableIsExtendable = tableSelf && tableSelf->state != TableState::Sealed; + ty = follow(ty); - if (tableSelf && tableSelf->state != TableState::Sealed) + if (tableIsExtendable) tableSelf->props[indexName->index.value] = {ty, /* deprecated */ false, {}, indexName->indexLocation}; const FunctionTypeVar* funTy = get(ty); @@ -1224,7 +1335,7 @@ void TypeChecker::check(const ScopePtr& scope, TypeId ty, const ScopePtr& funSco checkFunctionBody(funScope, ty, *function.func); - if (tableSelf && tableSelf->state != TableState::Sealed) + if (tableIsExtendable) tableSelf->props[indexName->index.value] = { follow(quantify(funScope, ty, indexName->indexLocation)), /* deprecated */ false, {}, indexName->indexLocation}; } @@ -1372,7 +1483,11 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatTypeAlias& typealias for (auto param : binding->typePackParams) clone.instantiatedTypePackParams.push_back(param.tp); + bool isNormal = ty->normal; ty = addType(std::move(clone)); + + if (FFlag::LuauLowerBoundsCalculation) + asMutable(ty)->normal = isNormal; } } else @@ -1400,6 +1515,14 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatTypeAlias& typealias if (FFlag::LuauTwoPassAliasDefinitionFix && ok) bindingType = ty; + + if (FFlag::LuauLowerBoundsCalculation) + { + auto [t, ok] = normalize(bindingType, currentModule, *iceHandler); + bindingType = t; + if (!ok) + reportError(typealias.location, NormalizationTooComplex{}); + } } } @@ -1673,10 +1796,11 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprCa { return {pack->head.empty() ? nilType : pack->head[0], std::move(result.predicates)}; } - else if (get(retPack)) + else if (const FreeTypePack* ftp = get(retPack)) { - TypeId head = freshType(scope); - TypePackId pack = addTypePack(TypePackVar{TypePack{{head}, freshTypePack(scope)}}); + TypeLevel level = FFlag::LuauLowerBoundsCalculation ? ftp->level : scope->level; + TypeId head = freshType(level); + TypePackId pack = addTypePack(TypePackVar{TypePack{{head}, freshTypePack(level)}}); unify(pack, retPack, expr.location); return {head, std::move(result.predicates)}; } @@ -1793,7 +1917,7 @@ std::optional TypeChecker::getIndexTypeFromType( for (TypeId t : utv) { - RecursionLimiter _rl(&recursionCount, FInt::LuauTypeInferRecursionLimit); + RecursionLimiter _rl(&recursionCount, FInt::LuauTypeInferRecursionLimit, "getIndexTypeForType unions"); // Not needed when we normalize types. if (get(follow(t))) @@ -1817,12 +1941,25 @@ std::optional TypeChecker::getIndexTypeFromType( return std::nullopt; } - std::vector result = reduceUnion(goodOptions); + if (FFlag::LuauLowerBoundsCalculation) + { + auto [t, ok] = normalize(addType(UnionTypeVar{std::move(goodOptions)}), currentModule, + *iceHandler); // FIXME Inefficient. We craft a UnionTypeVar and immediately throw it away. - if (result.size() == 1) - return result[0]; + if (!ok) + reportError(location, NormalizationTooComplex{}); - return addType(UnionTypeVar{std::move(result)}); + return t; + } + else + { + std::vector result = reduceUnion(goodOptions); + + if (result.size() == 1) + return result[0]; + + return addType(UnionTypeVar{std::move(result)}); + } } else if (const IntersectionTypeVar* itv = get(type)) { @@ -1830,7 +1967,7 @@ std::optional TypeChecker::getIndexTypeFromType( for (TypeId t : itv->parts) { - RecursionLimiter _rl(&recursionCount, FInt::LuauTypeInferRecursionLimit); + RecursionLimiter _rl(&recursionCount, FInt::LuauTypeInferRecursionLimit, "getIndexTypeFromType intersections"); if (std::optional ty = getIndexTypeFromType(scope, t, name, location, false)) parts.push_back(*ty); @@ -1982,7 +2119,6 @@ TypeId TypeChecker::stripFromNilAndReport(TypeId ty, const Location& location) { if (!std::any_of(begin(utv), end(utv), isNil)) return ty; - } if (std::optional strippedUnion = tryStripUnionFromNil(ty)) @@ -2124,7 +2260,26 @@ TypeId TypeChecker::checkExprTable( ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprTable& expr, std::optional expectedType) { - RecursionLimiter _rl(&recursionCount, FInt::LuauTypeInferRecursionLimit); + if (FFlag::LuauTableUseCounterInstead) + { + RecursionCounter _rc(&checkRecursionCount); + if (FInt::LuauCheckRecursionLimit > 0 && checkRecursionCount >= FInt::LuauCheckRecursionLimit) + { + reportErrorCodeTooComplex(expr.location); + return {errorRecoveryType(scope)}; + } + + return checkExpr_(scope, expr, expectedType); + } + else + { + RecursionLimiter _rl(&recursionCount, FInt::LuauTypeInferRecursionLimit, "checkExpr for tables"); + return checkExpr_(scope, expr, expectedType); + } +} + +ExprResult TypeChecker::checkExpr_(const ScopePtr& scope, const AstExprTable& expr, std::optional expectedType) +{ std::vector> fieldTypes(expr.items.size); const TableTypeVar* expectedTable = nullptr; @@ -3176,6 +3331,10 @@ std::pair TypeChecker::checkFunctionSignature( funScope->varargPack = anyTypePack; } } + else if (FFlag::LuauLowerBoundsCalculation && !isNonstrictMode()) + { + funScope->varargPack = addTypePack(TypePackVar{VariadicTypePack{anyType, /*hidden*/ true}}); + } std::vector argTypes; @@ -3311,9 +3470,24 @@ void TypeChecker::checkFunctionBody(const ScopePtr& scope, TypeId ty, const AstE { check(scope, *function.body); - // We explicitly don't follow here to check if we have a 'true' free type instead of bound one - if (get_if(&funTy->retType->ty)) - *asMutable(funTy->retType) = TypePack{{}, std::nullopt}; + if (useConstrainedIntersections()) + { + TypePackId retPack = follow(funTy->retType); + // It is possible for a function to have no annotation and no return statement, and yet still have an ascribed return type + // if it is expected to conform to some other interface. (eg the function may be a lambda passed as a callback) + if (!hasReturn(function.body) && !function.returnAnnotation.has_value() && get(retPack)) + { + auto level = getLevel(retPack); + if (level && scope->level.subsumes(*level)) + *asMutable(retPack) = TypePack{{}, std::nullopt}; + } + } + else + { + // We explicitly don't follow here to check if we have a 'true' free type instead of bound one + if (get_if(&funTy->retType->ty)) + *asMutable(funTy->retType) = TypePack{{}, std::nullopt}; + } bool reachesImplicitReturn = getFallthrough(function.body) != nullptr; @@ -3418,6 +3592,19 @@ void TypeChecker::checkArgumentList( size_t minParams = FFlag::LuauFixIncorrectLineNumberDuplicateType ? 0 : getMinParameterCount_DEPRECATED(paramPack); + auto reportCountMismatchError = [&state, &argLocations, minParams, paramPack, argPack]() { + // For this case, we want the error span to cover every errant extra parameter + Location location = state.location; + if (!argLocations.empty()) + location = {state.location.begin, argLocations.back().end}; + + size_t mp = minParams; + if (FFlag::LuauFixArgumentCountMismatchAmountWithGenericTypes) + mp = getMinParameterCount(&state.log, paramPack); + + state.reportError(TypeError{location, CountMismatch{mp, std::distance(begin(argPack), end(argPack))}}); + }; + while (true) { state.location = paramIndex < argLocations.size() ? argLocations[paramIndex] : state.location; @@ -3472,6 +3659,8 @@ void TypeChecker::checkArgumentList( } else if (auto vtp = state.log.getMutable(tail)) { + // Function is variadic and requires that all subsequent parameters + // be compatible with a type. while (paramIter != endIter) { state.tryUnify(vtp->ty, *paramIter); @@ -3506,14 +3695,22 @@ void TypeChecker::checkArgumentList( else if (state.log.getMutable(t)) { } // ok - else if (!FFlag::LuauAnyInIsOptionalIsOptional && isNonstrictMode() && state.log.getMutable(t)) + else if (!FFlag::LuauAnyInIsOptionalIsOptional && isNonstrictMode() && state.log.get(t)) { } // ok else { if (FFlag::LuauFixArgumentCountMismatchAmountWithGenericTypes) minParams = getMinParameterCount(&state.log, paramPack); - bool isVariadic = FFlag::LuauArgCountMismatchSaysAtLeastWhenVariadic && !finite(paramPack, &state.log); + + bool isVariadic = false; + if (FFlag::LuauArgCountMismatchSaysAtLeastWhenVariadic) + { + std::optional tail = flatten(paramPack, state.log).second; + if (tail) + isVariadic = Luau::isVariadic(*tail); + } + state.reportError(TypeError{state.location, CountMismatch{minParams, paramIndex, CountMismatch::Context::Arg, isVariadic}}); return; } @@ -3532,14 +3729,7 @@ void TypeChecker::checkArgumentList( unify(errorRecoveryType(scope), *argIter, state.location); ++argIter; } - // For this case, we want the error span to cover every errant extra parameter - Location location = state.location; - if (!argLocations.empty()) - location = {state.location.begin, argLocations.back().end}; - - if (FFlag::LuauFixArgumentCountMismatchAmountWithGenericTypes) - minParams = getMinParameterCount(&state.log, paramPack); - state.reportError(TypeError{location, CountMismatch{minParams, std::distance(begin(argPack), end(argPack))}}); + reportCountMismatchError(); return; } TypePackId tail = state.log.follow(*paramIter.tail()); @@ -3551,6 +3741,21 @@ void TypeChecker::checkArgumentList( } else if (auto vtp = state.log.getMutable(tail)) { + if (FFlag::LuauLowerBoundsCalculation && vtp->hidden) + { + // We know that this function can technically be oversaturated, but we have its definition and we + // know that it's useless. + + TypeId e = errorRecoveryType(scope); + while (argIter != endIter) + { + unify(e, *argIter, state.location); + ++argIter; + } + + reportCountMismatchError(); + return; + } // Function is variadic and requires that all subsequent parameters // be compatible with a type. size_t argIndex = paramIndex; @@ -3595,14 +3800,7 @@ void TypeChecker::checkArgumentList( } else if (state.log.getMutable(tail)) { - // For this case, we want the error span to cover every errant extra parameter - Location location = state.location; - if (!argLocations.empty()) - location = {state.location.begin, argLocations.back().end}; - // TODO: Better error message? - if (FFlag::LuauFixArgumentCountMismatchAmountWithGenericTypes) - minParams = getMinParameterCount(&state.log, paramPack); - state.reportError(TypeError{location, CountMismatch{minParams, std::distance(begin(argPack), end(argPack))}}); + reportCountMismatchError(); return; } } @@ -3661,7 +3859,7 @@ ExprResult TypeChecker::checkExprPack(const ScopePtr& scope, const A actualFunctionType = follow(actualFunctionType); TypePackId retPack; - if (!FFlag::LuauWidenIfSupertypeIsFree2) + if (FFlag::LuauLowerBoundsCalculation || !FFlag::LuauWidenIfSupertypeIsFree2) { retPack = freshTypePack(scope->level); } @@ -3809,21 +4007,49 @@ std::optional> TypeChecker::checkCallOverload(const Scope return {{errorRecoveryTypePack(scope)}}; } - if (get(fn)) + if (auto ftv = get(fn)) { // fn is one of the overloads of actualFunctionType, which // has been instantiated, so is a monotype. We can therefore // unify it with a monomorphic function. - TypeId r = addType(FunctionTypeVar(scope->level, argPack, retPack)); - if (FFlag::LuauWidenIfSupertypeIsFree2) + if (useConstrainedIntersections()) { - UnifierOptions options; - options.isFunctionCall = true; - unify(r, fn, expr.location, options); + // This ternary is phrased deliberately. We need ties between sibling scopes to bias toward ftv->level. + const TypeLevel level = scope->level.subsumes(ftv->level) ? scope->level : ftv->level; + + std::vector adjustedArgTypes; + auto it = begin(argPack); + auto endIt = end(argPack); + Widen widen{¤tModule->internalTypes}; + for (; it != endIt; ++it) + { + TypeId t = *it; + TypeId widened = widen.substitute(t).value_or(t); // Surely widening is infallible + adjustedArgTypes.push_back(addType(ConstrainedTypeVar{level, {widened}})); + } + + TypePackId adjustedArgPack = addTypePack(TypePack{std::move(adjustedArgTypes), it.tail()}); + + TxnLog log; + promoteTypeLevels(log, ¤tModule->internalTypes, level, retPack); + log.commit(); + + *asMutable(fn) = FunctionTypeVar{level, adjustedArgPack, retPack}; + return {{retPack}}; } else - unify(fn, r, expr.location); - return {{retPack}}; + { + TypeId r = addType(FunctionTypeVar(scope->level, argPack, retPack)); + if (FFlag::LuauWidenIfSupertypeIsFree2) + { + UnifierOptions options; + options.isFunctionCall = true; + unify(r, fn, expr.location, options); + } + else + unify(fn, r, expr.location); + return {{retPack}}; + } } std::vector metaArgLocations; @@ -4363,10 +4589,17 @@ void TypeChecker::unifyWithInstantiationIfNeeded(const ScopePtr& scope, TypeId s bool Instantiation::isDirty(TypeId ty) { - if (log->getMutable(ty)) + if (const FunctionTypeVar* ftv = log->getMutable(ty)) + { + if (FFlag::LuauTypecheckOptPass && ftv->hasNoGenerics) + return false; + return true; + } else + { return false; + } } bool Instantiation::isDirty(TypePackId tp) @@ -4414,14 +4647,21 @@ TypePackId Instantiation::clean(TypePackId tp) bool ReplaceGenerics::ignoreChildren(TypeId ty) { if (const FunctionTypeVar* ftv = log->getMutable(ty)) + { + if (FFlag::LuauTypecheckOptPass && ftv->hasNoGenerics) + return true; + // We aren't recursing in the case of a generic function which // binds the same generics. This can happen if, for example, there's recursive types. // If T = (a,T)->T then instantiating T should produce T' = (X,T)->T not T' = (X,T')->T'. // It's OK to use vector equality here, since we always generate fresh generics // whenever we quantify, so the vectors overlap if and only if they are equal. return (!generics.empty() || !genericPacks.empty()) && (ftv->generics == generics) && (ftv->genericPacks == genericPacks); + } else + { return false; + } } bool ReplaceGenerics::isDirty(TypeId ty) @@ -4464,16 +4704,24 @@ TypePackId ReplaceGenerics::clean(TypePackId tp) bool Anyification::isDirty(TypeId ty) { + if (ty->persistent) + return false; + if (const TableTypeVar* ttv = log->getMutable(ty)) return (ttv->state == TableState::Free || (FFlag::LuauSealExports && ttv->state == TableState::Unsealed)); else if (log->getMutable(ty)) return true; + else if (get(ty)) + return true; else return false; } bool Anyification::isDirty(TypePackId tp) { + if (tp->persistent) + return false; + if (log->getMutable(tp)) return true; else @@ -4494,7 +4742,16 @@ TypeId Anyification::clean(TypeId ty) clone.syntheticName = ttv->syntheticName; clone.tags = ttv->tags; } - return addType(std::move(clone)); + TypeId res = addType(std::move(clone)); + asMutable(res)->normal = ty->normal; + return res; + } + else if (auto ctv = get(ty)) + { + auto [t, ok] = normalize(ty, *arena, *iceHandler); + if (!ok) + normalizationTooComplex = true; + return t; } else return anyType; @@ -4511,16 +4768,34 @@ TypeId TypeChecker::quantify(const ScopePtr& scope, TypeId ty, Location location ty = follow(ty); const FunctionTypeVar* ftv = get(ty); - if (!ftv || !ftv->generics.empty() || !ftv->genericPacks.empty()) - return ty; + if (ftv && ftv->generics.empty() && ftv->genericPacks.empty()) + Luau::quantify(ty, scope->level); + + if (FFlag::LuauLowerBoundsCalculation && ftv) + { + auto [t, ok] = Luau::normalize(ty, currentModule, *iceHandler); + if (!ok) + reportError(location, NormalizationTooComplex{}); + return t; + } - Luau::quantify(ty, scope->level); return ty; } TypeId TypeChecker::instantiate(const ScopePtr& scope, TypeId ty, Location location, const TxnLog* log) { + if (FFlag::LuauTypecheckOptPass) + { + const FunctionTypeVar* ftv = get(follow(ty)); + if (ftv && ftv->hasNoGenerics) + return ty; + } + Instantiation instantiation{log, ¤tModule->internalTypes, scope->level}; + + if (FFlag::LuauAutocompleteDynamicLimits && instantiationChildLimit) + instantiation.childLimit = *instantiationChildLimit; + std::optional instantiated = instantiation.substitute(ty); if (instantiated.has_value()) return *instantiated; @@ -4533,8 +4808,18 @@ TypeId TypeChecker::instantiate(const ScopePtr& scope, TypeId ty, Location locat TypeId TypeChecker::anyify(const ScopePtr& scope, TypeId ty, Location location) { - Anyification anyification{¤tModule->internalTypes, anyType, anyTypePack}; + if (FFlag::LuauLowerBoundsCalculation) + { + auto [t, ok] = normalize(ty, currentModule, *iceHandler); + if (!ok) + reportError(location, NormalizationTooComplex{}); + ty = t; + } + + Anyification anyification{¤tModule->internalTypes, iceHandler, anyType, anyTypePack}; std::optional any = anyification.substitute(ty); + if (anyification.normalizationTooComplex) + reportError(location, NormalizationTooComplex{}); if (any.has_value()) return *any; else @@ -4546,7 +4831,15 @@ TypeId TypeChecker::anyify(const ScopePtr& scope, TypeId ty, Location location) TypePackId TypeChecker::anyify(const ScopePtr& scope, TypePackId ty, Location location) { - Anyification anyification{¤tModule->internalTypes, anyType, anyTypePack}; + if (FFlag::LuauLowerBoundsCalculation) + { + auto [t, ok] = normalize(ty, currentModule, *iceHandler); + if (!ok) + reportError(location, NormalizationTooComplex{}); + ty = t; + } + + Anyification anyification{¤tModule->internalTypes, iceHandler, anyType, anyTypePack}; std::optional any = anyification.substitute(ty); if (any.has_value()) return *any; @@ -4830,6 +5123,7 @@ TypeId TypeChecker::resolveType(const ScopePtr& scope, const AstType& annotation ToStringOptions opts; opts.exhaustive = true; opts.maxTableLength = 0; + opts.useLineBreaks = true; TypeId param = resolveType(scope, *lit->parameters.data[0].type); luauPrintLine(format("_luau_print\t%s\t|\t%s", toString(param, opts).c_str(), toString(lit->location).c_str())); @@ -5283,7 +5577,7 @@ TypeId TypeChecker::instantiateTypeFun(const ScopePtr& scope, const TypeFun& tf, bool needsClone = follow(tf.type) == target; bool shouldMutate = (!FFlag::LuauOnlyMutateInstantiatedTables || getTableType(tf.type)); TableTypeVar* ttv = getMutableTableType(target); - + if (shouldMutate && ttv && needsClone) { // Substitution::clone is a shallow clone. If this is a metatable type, we @@ -5487,25 +5781,82 @@ std::optional TypeChecker::resolveLValue(const ScopePtr& scope, const LV // We need to search in the provided Scope. Find t.x.y first. // We fail to find t.x.y. Try t.x. We found it. Now we must return the type of the property y from the mapped-to type of t.x. // If we completely fail to find the Symbol t but the Scope has that entry, then we should walk that all the way through and terminate. - const auto& [symbol, keys] = getFullName(lvalue); + if (!FFlag::LuauTypecheckOptPass) + { + const auto& [symbol, keys] = getFullName(lvalue); + + ScopePtr currentScope = scope; + while (currentScope) + { + std::optional found; + + std::vector childKeys; + const LValue* currentLValue = &lvalue; + while (currentLValue) + { + if (auto it = currentScope->refinements.find(*currentLValue); it != currentScope->refinements.end()) + { + found = it->second; + break; + } + + childKeys.push_back(*currentLValue); + currentLValue = baseof(*currentLValue); + } + + if (!found) + { + // Should not be using scope->lookup. This is already recursive. + if (auto it = currentScope->bindings.find(symbol); it != currentScope->bindings.end()) + found = it->second.typeId; + else + { + // Nothing exists in this Scope. Just skip and try the parent one. + currentScope = currentScope->parent; + continue; + } + } + + for (auto it = childKeys.rbegin(); it != childKeys.rend(); ++it) + { + const LValue& key = *it; + + // Symbol can happen. Skip. + if (get(key)) + continue; + else if (auto field = get(key)) + { + found = getIndexTypeFromType(scope, *found, field->key, Location(), false); + if (!found) + return std::nullopt; // Turns out this type doesn't have the property at all. We're done. + } + else + LUAU_ASSERT(!"New LValue alternative not handled here."); + } + + return found; + } + + // No entry for it at all. Can happen when LValue root is a global. + return std::nullopt; + } + + const Symbol symbol = getBaseSymbol(lvalue); ScopePtr currentScope = scope; while (currentScope) { std::optional found; - std::vector childKeys; - const LValue* currentLValue = &lvalue; - while (currentLValue) + const LValue* topLValue = nullptr; + + for (topLValue = &lvalue; topLValue; topLValue = baseof(*topLValue)) { - if (auto it = currentScope->refinements.find(*currentLValue); it != currentScope->refinements.end()) + if (auto it = currentScope->refinements.find(*topLValue); it != currentScope->refinements.end()) { found = it->second; break; } - - childKeys.push_back(*currentLValue); - currentLValue = baseof(*currentLValue); } if (!found) @@ -5521,9 +5872,15 @@ std::optional TypeChecker::resolveLValue(const ScopePtr& scope, const LV } } + // We need to walk the l-value path in reverse, so we collect components into a vector + std::vector childKeys; + + for (const LValue* curr = &lvalue; curr != topLValue; curr = baseof(*curr)) + childKeys.push_back(curr); + for (auto it = childKeys.rbegin(); it != childKeys.rend(); ++it) { - const LValue& key = *it; + const LValue& key = **it; // Symbol can happen. Skip. if (get(key)) @@ -5938,6 +6295,11 @@ bool TypeChecker::isNonstrictMode() const return (currentModule->mode == Mode::Nonstrict) || (currentModule->mode == Mode::NoCheck); } +bool TypeChecker::useConstrainedIntersections() const +{ + return FFlag::LuauLowerBoundsCalculation && !isNonstrictMode(); +} + std::vector TypeChecker::unTypePack(const ScopePtr& scope, TypePackId tp, size_t expectedLength, const Location& location) { TypePackId expectedTypePack = addTypePack({}); diff --git a/Analysis/src/TypePack.cpp b/Analysis/src/TypePack.cpp index 5bb05234..30503233 100644 --- a/Analysis/src/TypePack.cpp +++ b/Analysis/src/TypePack.cpp @@ -104,7 +104,7 @@ TypePackIterator begin(TypePackId tp) return TypePackIterator{tp}; } -TypePackIterator begin(TypePackId tp, TxnLog* log) +TypePackIterator begin(TypePackId tp, const TxnLog* log) { return TypePackIterator{tp, log}; } @@ -256,7 +256,7 @@ size_t size(const TypePack& tp, TxnLog* log) return result; } -std::optional first(TypePackId tp) +std::optional first(TypePackId tp, bool ignoreHiddenVariadics) { auto it = begin(tp); auto endIter = end(tp); @@ -266,7 +266,7 @@ std::optional first(TypePackId tp) if (auto tail = it.tail()) { - if (auto vtp = get(*tail)) + if (auto vtp = get(*tail); vtp && (!vtp->hidden || !ignoreHiddenVariadics)) return vtp->ty; } @@ -299,6 +299,46 @@ std::pair, std::optional> flatten(TypePackId tp) return {res, iter.tail()}; } +std::pair, std::optional> flatten(TypePackId tp, const TxnLog& log) +{ + tp = log.follow(tp); + + std::vector flattened; + std::optional tail = std::nullopt; + + TypePackIterator it(tp, &log); + + for (; it != end(tp); ++it) + { + flattened.push_back(*it); + } + + tail = it.tail(); + + return {flattened, tail}; +} + +bool isVariadic(TypePackId tp) +{ + return isVariadic(tp, *TxnLog::empty()); +} + +bool isVariadic(TypePackId tp, const TxnLog& log) +{ + std::optional tail = flatten(tp, log).second; + + if (!tail) + return false; + + if (log.get(*tail)) + return true; + + if (auto vtp = log.get(*tail); vtp && !vtp->hidden) + return true; + + return false; +} + TypePackVar* asMutable(TypePackId tp) { return const_cast(tp); diff --git a/Analysis/src/TypeVar.cpp b/Analysis/src/TypeVar.cpp index dbc412fc..0fbfdbf0 100644 --- a/Analysis/src/TypeVar.cpp +++ b/Analysis/src/TypeVar.cpp @@ -177,7 +177,7 @@ bool maybeString(TypeId ty) if (FFlag::LuauSubtypingAddOptPropsToUnsealedTables) { ty = follow(ty); - + if (isPrim(ty, PrimitiveTypeVar::String) || get(ty)) return true; @@ -366,7 +366,7 @@ bool maybeSingleton(TypeId ty) bool hasLength(TypeId ty, DenseHashSet& seen, int* recursionCount) { - RecursionLimiter _rl(recursionCount, FInt::LuauTypeInferRecursionLimit); + RecursionLimiter _rl(recursionCount, FInt::LuauTypeInferRecursionLimit, "hasLength"); ty = follow(ty); @@ -654,9 +654,9 @@ static TypeVar booleanType_{PrimitiveTypeVar{PrimitiveTypeVar::Boolean}, /*persi static TypeVar threadType_{PrimitiveTypeVar{PrimitiveTypeVar::Thread}, /*persistent*/ true}; static TypeVar trueType_{SingletonTypeVar{BooleanSingleton{true}}, /*persistent*/ true}; static TypeVar falseType_{SingletonTypeVar{BooleanSingleton{false}}, /*persistent*/ true}; -static TypeVar anyType_{AnyTypeVar{}}; -static TypeVar errorType_{ErrorTypeVar{}}; -static TypeVar optionalNumberType_{UnionTypeVar{{&numberType_, &nilType_}}}; +static TypeVar anyType_{AnyTypeVar{}, /*persistent*/ true}; +static TypeVar errorType_{ErrorTypeVar{}, /*persistent*/ true}; +static TypeVar optionalNumberType_{UnionTypeVar{{&numberType_, &nilType_}}, /*persistent*/ true}; static TypePackVar anyTypePack_{VariadicTypePack{&anyType_}, true}; static TypePackVar errorTypePack_{Unifiable::Error{}}; @@ -698,7 +698,7 @@ TypeId SingletonTypes::makeStringMetatable() { const TypeId optionalNumber = arena->addType(UnionTypeVar{{nilType, numberType}}); const TypeId optionalString = arena->addType(UnionTypeVar{{nilType, stringType}}); - const TypeId optionalBoolean = arena->addType(UnionTypeVar{{nilType, &booleanType_}}); + const TypeId optionalBoolean = arena->addType(UnionTypeVar{{nilType, booleanType}}); const TypePackId oneStringPack = arena->addTypePack({stringType}); const TypePackId anyTypePack = arena->addTypePack(TypePackVar{VariadicTypePack{anyType}, true}); @@ -802,6 +802,7 @@ void persist(TypeId ty) continue; asMutable(t)->persistent = true; + asMutable(t)->normal = true; // all persistent types are assumed to be normal if (auto btv = get(t)) queue.push_back(btv->boundTo); @@ -838,6 +839,11 @@ void persist(TypeId ty) for (TypeId opt : itv->parts) queue.push_back(opt); } + else if (auto ctv = get(t)) + { + for (TypeId opt : ctv->parts) + queue.push_back(opt); + } else if (auto mtv = get(t)) { queue.push_back(mtv->table); @@ -899,6 +905,16 @@ TypeLevel* getMutableLevel(TypeId ty) return const_cast(getLevel(ty)); } +std::optional getLevel(TypePackId tp) +{ + tp = follow(tp); + + if (auto ftv = get(tp)) + return ftv->level; + else + return std::nullopt; +} + const Property* lookupClassProp(const ClassTypeVar* cls, const Name& name) { while (cls) diff --git a/Analysis/src/Unifier.cpp b/Analysis/src/Unifier.cpp index 398dc9e2..f9ea58cc 100644 --- a/Analysis/src/Unifier.cpp +++ b/Analysis/src/Unifier.cpp @@ -14,9 +14,12 @@ LUAU_FASTINT(LuauTypeInferRecursionLimit); LUAU_FASTINT(LuauTypeInferTypePackLoopLimit); -LUAU_FASTINTVARIABLE(LuauTypeInferIterationLimit, 2000); +LUAU_FASTINT(LuauTypeInferIterationLimit); +LUAU_FASTFLAG(LuauAutocompleteDynamicLimits) +LUAU_FASTINTVARIABLE(LuauTypeInferLowerBoundsIterationLimit, 2000); LUAU_FASTFLAGVARIABLE(LuauExtendedIndexerError, false); LUAU_FASTFLAGVARIABLE(LuauTableSubtypingVariance2, false); +LUAU_FASTFLAG(LuauLowerBoundsCalculation); LUAU_FASTFLAG(LuauErrorRecoveryType); LUAU_FASTFLAGVARIABLE(LuauSubtypingAddOptPropsToUnsealedTables, false) LUAU_FASTFLAGVARIABLE(LuauWidenIfSupertypeIsFree2, false) @@ -27,6 +30,7 @@ LUAU_FASTFLAGVARIABLE(LuauTxnLogRefreshFunctionPointers, false) LUAU_FASTFLAGVARIABLE(LuauTxnLogDontRetryForIndexers, false) LUAU_FASTFLAGVARIABLE(LuauUnifierCacheErrors, false) LUAU_FASTFLAG(LuauAnyInIsOptionalIsOptional) +LUAU_FASTFLAG(LuauTypecheckOptPass) namespace Luau { @@ -126,7 +130,6 @@ static void promoteTypeLevels(TxnLog& log, const TypeArena* typeArena, TypeLevel visitTypeVarOnce(ty, ptl, seen); } -// TODO: use this and make it static. void promoteTypeLevels(TxnLog& log, const TypeArena* typeArena, TypeLevel minLevel, TypePackId tp) { // Type levels of types from other modules are already global, so we don't need to promote anything inside @@ -305,8 +308,7 @@ static std::optional> getTableMat return std::nullopt; } -Unifier::Unifier(TypeArena* types, Mode mode, const Location& location, Variance variance, UnifierSharedState& sharedState, - TxnLog* parentLog) +Unifier::Unifier(TypeArena* types, Mode mode, const Location& location, Variance variance, UnifierSharedState& sharedState, TxnLog* parentLog) : types(types) , mode(mode) , log(parentLog) @@ -326,6 +328,7 @@ Unifier::Unifier(TypeArena* types, Mode mode, std::vector 0 && FInt::LuauTypeInferIterationLimit < sharedState.counters.iterationCount) + if (FFlag::LuauAutocompleteDynamicLimits) { - reportError(TypeError{location, UnificationTooComplex{}}); - return; + if (sharedState.counters.iterationLimit > 0 && sharedState.counters.iterationLimit < sharedState.counters.iterationCount) + { + reportError(TypeError{location, UnificationTooComplex{}}); + return; + } + } + else + { + if (FInt::LuauTypeInferIterationLimit > 0 && FInt::LuauTypeInferIterationLimit < sharedState.counters.iterationCount) + { + reportError(TypeError{location, UnificationTooComplex{}}); + return; + } } superTy = log.follow(superTy); @@ -354,6 +369,9 @@ void Unifier::tryUnify_(TypeId subTy, TypeId superTy, bool isFunctionCall, bool if (superTy == subTy) return; + if (log.get(superTy)) + return tryUnifyWithConstrainedSuperTypeVar(subTy, superTy); + auto superFree = log.getMutable(superTy); auto subFree = log.getMutable(subTy); @@ -442,7 +460,18 @@ void Unifier::tryUnify_(TypeId subTy, TypeId superTy, bool isFunctionCall, bool if (get(superTy) || get(superTy)) return tryUnifyWithAny(subTy, superTy); - if (get(subTy) || get(subTy)) + if (get(subTy)) + { + if (anyIsTop) + { + reportError(TypeError{location, TypeMismatch{superTy, subTy}}); + return; + } + else + return tryUnifyWithAny(superTy, subTy); + } + + if (get(subTy)) return tryUnifyWithAny(superTy, subTy); bool cacheEnabled; @@ -484,7 +513,9 @@ void Unifier::tryUnify_(TypeId subTy, TypeId superTy, bool isFunctionCall, bool size_t errorCount = errors.size(); - if (const UnionTypeVar* uv = log.getMutable(subTy)) + if (log.get(subTy)) + tryUnifyWithConstrainedSubTypeVar(subTy, superTy); + else if (const UnionTypeVar* uv = log.getMutable(subTy)) { tryUnifyUnionWithType(subTy, uv, superTy); } @@ -946,7 +977,7 @@ struct WeirdIter LUAU_ASSERT(log.getMutable(newTail)); level = log.getMutable(packId)->level; - log.replace(packId, Unifiable::Bound(newTail)); + log.replace(packId, BoundTypePack(newTail)); packId = newTail; pack = log.getMutable(newTail); index = 0; @@ -994,39 +1025,32 @@ void Unifier::tryUnify(TypePackId subTp, TypePackId superTp, bool isFunctionCall tryUnify_(subTp, superTp, isFunctionCall); } -static std::pair, std::optional> logAwareFlatten(TypePackId tp, const TxnLog& log) -{ - tp = log.follow(tp); - - std::vector flattened; - std::optional tail = std::nullopt; - - TypePackIterator it(tp, &log); - - for (; it != end(tp); ++it) - { - flattened.push_back(*it); - } - - tail = it.tail(); - - return {flattened, tail}; -} - /* * This is quite tricky: we are walking two rope-like structures and unifying corresponding elements. * If one is longer than the other, but the short end is free, we grow it to the required length. */ void Unifier::tryUnify_(TypePackId subTp, TypePackId superTp, bool isFunctionCall) { - RecursionLimiter _ra(&sharedState.counters.recursionCount, FInt::LuauTypeInferRecursionLimit); + RecursionLimiter _ra(&sharedState.counters.recursionCount, + FFlag::LuauAutocompleteDynamicLimits ? sharedState.counters.recursionLimit : FInt::LuauTypeInferRecursionLimit, "TypePackId tryUnify_"); ++sharedState.counters.iterationCount; - if (FInt::LuauTypeInferIterationLimit > 0 && FInt::LuauTypeInferIterationLimit < sharedState.counters.iterationCount) + if (FFlag::LuauAutocompleteDynamicLimits) { - reportError(TypeError{location, UnificationTooComplex{}}); - return; + if (sharedState.counters.iterationLimit > 0 && sharedState.counters.iterationLimit < sharedState.counters.iterationCount) + { + reportError(TypeError{location, UnificationTooComplex{}}); + return; + } + } + else + { + if (FInt::LuauTypeInferIterationLimit > 0 && FInt::LuauTypeInferIterationLimit < sharedState.counters.iterationCount) + { + reportError(TypeError{location, UnificationTooComplex{}}); + return; + } } superTp = log.follow(superTp); @@ -1087,8 +1111,8 @@ void Unifier::tryUnify_(TypePackId subTp, TypePackId superTp, bool isFunctionCal // If the size of two heads does not match, but both packs have free tail // We set the sentinel variable to say so to avoid growing it forever. - auto [superTypes, superTail] = logAwareFlatten(superTp, log); - auto [subTypes, subTail] = logAwareFlatten(subTp, log); + auto [superTypes, superTail] = flatten(superTp, log); + auto [subTypes, subTail] = flatten(subTp, log); bool noInfiniteGrowth = (superTypes.size() != subTypes.size()) && (superTail && log.getMutable(*superTail)) && (subTail && log.getMutable(*subTail)); @@ -1165,19 +1189,20 @@ void Unifier::tryUnify_(TypePackId subTp, TypePackId superTp, bool isFunctionCal else { // A union type including nil marks an optional argument - if (superIter.good() && isOptional(*superIter)) + if ((!FFlag::LuauLowerBoundsCalculation || isNonstrictMode()) && superIter.good() && isOptional(*superIter)) { superIter.advance(); continue; } - else if (subIter.good() && isOptional(*subIter)) + else if ((!FFlag::LuauLowerBoundsCalculation || isNonstrictMode()) && subIter.good() && isOptional(*subIter)) { subIter.advance(); continue; } // In nonstrict mode, any also marks an optional argument. - else if (!FFlag::LuauAnyInIsOptionalIsOptional && superIter.good() && isNonstrictMode() && log.getMutable(log.follow(*superIter))) + else if (!FFlag::LuauAnyInIsOptionalIsOptional && superIter.good() && isNonstrictMode() && + log.getMutable(log.follow(*superIter))) { superIter.advance(); continue; @@ -1195,7 +1220,7 @@ void Unifier::tryUnify_(TypePackId subTp, TypePackId superTp, bool isFunctionCal return; } - if (!isFunctionCall && subIter.good()) + if ((!FFlag::LuauLowerBoundsCalculation || isNonstrictMode()) && !isFunctionCall && subIter.good()) { // Sometimes it is ok to pass too many arguments return; @@ -1418,14 +1443,17 @@ void Unifier::tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) if (FFlag::LuauAnyInIsOptionalIsOptional) { - if (subIter == subTable->props.end() && (!FFlag::LuauSubtypingAddOptPropsToUnsealedTables || subTable->state == TableState::Unsealed) && !isOptional(superProp.type)) + if (subIter == subTable->props.end() && + (!FFlag::LuauSubtypingAddOptPropsToUnsealedTables || subTable->state == TableState::Unsealed) && !isOptional(superProp.type)) missingProperties.push_back(propName); } else { bool isAny = log.getMutable(log.follow(superProp.type)); - if (subIter == subTable->props.end() && (!FFlag::LuauSubtypingAddOptPropsToUnsealedTables || subTable->state == TableState::Unsealed) && !isOptional(superProp.type) && !isAny) + if (subIter == subTable->props.end() && + (!FFlag::LuauSubtypingAddOptPropsToUnsealedTables || subTable->state == TableState::Unsealed) && !isOptional(superProp.type) && + !isAny) missingProperties.push_back(propName); } } @@ -1438,8 +1466,7 @@ void Unifier::tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) } // And vice versa if we're invariant - if (variance == Invariant && !superTable->indexer && superTable->state != TableState::Unsealed && - superTable->state != TableState::Free) + if (variance == Invariant && !superTable->indexer && superTable->state != TableState::Unsealed && superTable->state != TableState::Free) { for (const auto& [propName, subProp] : subTable->props) { @@ -1453,7 +1480,8 @@ void Unifier::tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) else { bool isAny = log.is(log.follow(subProp.type)); - if (superIter == superTable->props.end() && (FFlag::LuauSubtypingAddOptPropsToUnsealedTables || (!isOptional(subProp.type) && !isAny))) + if (superIter == superTable->props.end() && + (FFlag::LuauSubtypingAddOptPropsToUnsealedTables || (!isOptional(subProp.type) && !isAny))) extraProperties.push_back(propName); } } @@ -1499,13 +1527,15 @@ void Unifier::tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) if (innerState.errors.empty()) log.concat(std::move(innerState.log)); } - else if (FFlag::LuauAnyInIsOptionalIsOptional && (!FFlag::LuauSubtypingAddOptPropsToUnsealedTables || subTable->state == TableState::Unsealed) && isOptional(prop.type)) + else if (FFlag::LuauAnyInIsOptionalIsOptional && + (!FFlag::LuauSubtypingAddOptPropsToUnsealedTables || subTable->state == TableState::Unsealed) && isOptional(prop.type)) // This is sound because unsealed table types are precise, so `{ p : T } <: { p : T, q : U? }` // since if `t : { p : T }` then we are guaranteed that `t.q` is `nil`. // TODO: if the supertype is written to, the subtype may no longer be precise (alias analysis?) { } - else if ((!FFlag::LuauSubtypingAddOptPropsToUnsealedTables || subTable->state == TableState::Unsealed) && (isOptional(prop.type) || get(follow(prop.type)))) + else if ((!FFlag::LuauSubtypingAddOptPropsToUnsealedTables || subTable->state == TableState::Unsealed) && + (isOptional(prop.type) || get(follow(prop.type)))) // This is sound because unsealed table types are precise, so `{ p : T } <: { p : T, q : U? }` // since if `t : { p : T }` then we are guaranteed that `t.q` is `nil`. // TODO: should isOptional(anyType) be true? @@ -1664,9 +1694,9 @@ void Unifier::tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) if (FFlag::LuauTxnLogDontRetryForIndexers) { - // Changing the indexer can invalidate the table pointers. - superTable = log.getMutable(superTy); - subTable = log.getMutable(subTy); + // Changing the indexer can invalidate the table pointers. + superTable = log.getMutable(superTy); + subTable = log.getMutable(subTy); } else if (FFlag::LuauTxnLogCheckForInvalidation) { @@ -1921,8 +1951,6 @@ void Unifier::tryUnifySealedTables(TypeId subTy, TypeId superTy, bool isIntersec if (!superTable || !subTable) ice("passed non-table types to unifySealedTables"); - Unifier innerState = makeChildUnifier(); - std::vector missingPropertiesInSuper; bool isUnnamedTable = subTable->name == std::nullopt && subTable->syntheticName == std::nullopt; bool errorReported = false; @@ -1944,6 +1972,8 @@ void Unifier::tryUnifySealedTables(TypeId subTy, TypeId superTy, bool isIntersec } } + Unifier innerState = makeChildUnifier(); + // Tables must have exactly the same props and their types must all unify for (const auto& it : superTable->props) { @@ -2376,6 +2406,180 @@ std::optional Unifier::findTablePropertyRespectingMeta(TypeId lhsType, N return Luau::findTablePropertyRespectingMeta(errors, lhsType, name, location); } +void Unifier::tryUnifyWithConstrainedSubTypeVar(TypeId subTy, TypeId superTy) +{ + const ConstrainedTypeVar* subConstrained = get(subTy); + if (!subConstrained) + ice("tryUnifyWithConstrainedSubTypeVar received non-ConstrainedTypeVar subTy!"); + + const std::vector& subTyParts = subConstrained->parts; + + // A | B <: T if A <: T and B <: T + bool failed = false; + std::optional unificationTooComplex; + + const size_t count = subTyParts.size(); + + for (size_t i = 0; i < count; ++i) + { + TypeId type = subTyParts[i]; + Unifier innerState = makeChildUnifier(); + innerState.tryUnify_(type, superTy); + + if (i == count - 1) + log.concat(std::move(innerState.log)); + + ++i; + + if (auto e = hasUnificationTooComplex(innerState.errors)) + unificationTooComplex = e; + + if (!innerState.errors.empty()) + { + failed = true; + break; + } + } + + if (unificationTooComplex) + reportError(*unificationTooComplex); + else if (failed) + reportError(TypeError{location, TypeMismatch{superTy, subTy}}); + else + log.replace(subTy, BoundTypeVar{superTy}); +} + +void Unifier::tryUnifyWithConstrainedSuperTypeVar(TypeId subTy, TypeId superTy) +{ + ConstrainedTypeVar* superC = log.getMutable(superTy); + if (!superC) + ice("tryUnifyWithConstrainedSuperTypeVar received non-ConstrainedTypeVar superTy!"); + + // subTy could be a + // table + // metatable + // class + // function + // primitive + // free + // generic + // intersection + // union + // Do we really just tack it on? I think we might! + // We can certainly do some deduplication. + // Is there any point to deducing Player|Instance when we could just reduce to Instance? + // Is it actually ok to have multiple free types in a single intersection? What if they are later unified into the same type? + // Maybe we do a simplification step during quantification. + + auto it = std::find(superC->parts.begin(), superC->parts.end(), subTy); + if (it != superC->parts.end()) + return; + + superC->parts.push_back(subTy); +} + +void Unifier::unifyLowerBound(TypePackId subTy, TypePackId superTy) +{ + // The duplication between this and regular typepack unification is tragic. + + auto superIter = begin(superTy, &log); + auto superEndIter = end(superTy); + + auto subIter = begin(subTy, &log); + auto subEndIter = end(subTy); + + int count = FInt::LuauTypeInferLowerBoundsIterationLimit; + + for (; subIter != subEndIter; ++subIter) + { + if (0 >= --count) + ice("Internal recursion counter limit exceeded in Unifier::unifyLowerBound"); + + if (superIter != superEndIter) + { + tryUnify_(*subIter, *superIter); + ++superIter; + continue; + } + + if (auto t = superIter.tail()) + { + TypePackId tailPack = follow(*t); + + if (log.get(tailPack)) + occursCheck(tailPack, subTy); + + FreeTypePack* freeTailPack = log.getMutable(tailPack); + if (!freeTailPack) + return; + + TypeLevel level = freeTailPack->level; + + TypePack* tp = getMutable(log.replace(tailPack, TypePack{})); + + for (; subIter != subEndIter; ++subIter) + { + tp->head.push_back(types->addType(ConstrainedTypeVar{level, {follow(*subIter)}})); + } + + tp->tail = subIter.tail(); + } + + return; + } + + if (superIter != superEndIter) + { + if (auto subTail = subIter.tail()) + { + TypePackId subTailPack = follow(*subTail); + if (get(subTailPack)) + { + TypePack* tp = getMutable(log.replace(subTailPack, TypePack{})); + + for (; superIter != superEndIter; ++superIter) + tp->head.push_back(*superIter); + } + } + else + { + while (superIter != superEndIter) + { + if (!isOptional(*superIter)) + { + errors.push_back(TypeError{location, CountMismatch{size(superTy), size(subTy), CountMismatch::Return}}); + return; + } + ++superIter; + } + } + + return; + } + + // Both iters are at their respective tails + auto subTail = subIter.tail(); + auto superTail = superIter.tail(); + if (subTail && superTail) + tryUnify(*subTail, *superTail); + else if (subTail) + { + const FreeTypePack* freeSubTail = log.getMutable(*subTail); + if (freeSubTail) + { + log.replace(*subTail, TypePack{}); + } + } + else if (superTail) + { + const FreeTypePack* freeSuperTail = log.getMutable(*superTail); + if (freeSuperTail) + { + log.replace(*superTail, TypePack{}); + } + } +} + void Unifier::occursCheck(TypeId needle, TypeId haystack) { sharedState.tempSeenTy.clear(); @@ -2385,7 +2589,8 @@ void Unifier::occursCheck(TypeId needle, TypeId haystack) void Unifier::occursCheck(DenseHashSet& seen, TypeId needle, TypeId haystack) { - RecursionLimiter _ra(&sharedState.counters.recursionCount, FInt::LuauTypeInferRecursionLimit); + RecursionLimiter _ra(&sharedState.counters.recursionCount, + FFlag::LuauAutocompleteDynamicLimits ? sharedState.counters.recursionLimit : FInt::LuauTypeInferRecursionLimit, "occursCheck for TypeId"); auto check = [&](TypeId tv) { occursCheck(seen, needle, tv); @@ -2425,6 +2630,11 @@ void Unifier::occursCheck(DenseHashSet& seen, TypeId needle, TypeId hays for (TypeId ty : a->parts) check(ty); } + else if (auto a = log.getMutable(haystack)) + { + for (TypeId ty : a->parts) + check(ty); + } } void Unifier::occursCheck(TypePackId needle, TypePackId haystack) @@ -2450,7 +2660,8 @@ void Unifier::occursCheck(DenseHashSet& seen, TypePackId needle, Typ if (!log.getMutable(needle)) ice("Expected needle pack to be free"); - RecursionLimiter _ra(&sharedState.counters.recursionCount, FInt::LuauTypeInferRecursionLimit); + RecursionLimiter _ra(&sharedState.counters.recursionCount, + FFlag::LuauAutocompleteDynamicLimits ? sharedState.counters.recursionLimit : FInt::LuauTypeInferRecursionLimit, "occursCheck for TypePackId"); while (!log.getMutable(haystack)) { @@ -2474,7 +2685,23 @@ void Unifier::occursCheck(DenseHashSet& seen, TypePackId needle, Typ Unifier Unifier::makeChildUnifier() { - return Unifier{types, mode, log.sharedSeen, location, variance, sharedState, &log}; + if (FFlag::LuauTypecheckOptPass) + { + Unifier u = Unifier{types, mode, location, variance, sharedState, &log}; + u.anyIsTop = anyIsTop; + return u; + } + + Unifier u = Unifier{types, mode, log.sharedSeen, location, variance, sharedState, &log}; + u.anyIsTop = anyIsTop; + return u; +} + +// A utility function that appends the given error to the unifier's error log. +// This allows setting a breakpoint wherever the unifier reports an error. +void Unifier::reportError(TypeError err) +{ + errors.push_back(std::move(err)); } bool Unifier::isNonstrictMode() const diff --git a/Ast/include/Luau/DenseHash.h b/Ast/include/Luau/DenseHash.h index 65939bee..f8543111 100644 --- a/Ast/include/Luau/DenseHash.h +++ b/Ast/include/Luau/DenseHash.h @@ -32,6 +32,7 @@ class DenseHashTable { public: class const_iterator; + class iterator; DenseHashTable(const Key& empty_key, size_t buckets = 0) : count(0) @@ -43,7 +44,7 @@ public: // don't move this to initializer list! this works around an MSVC codegen issue on AMD CPUs: // https://developercommunity.visualstudio.com/t/stdvector-constructor-from-size-t-is-25-times-slow/1546547 if (buckets) - data.resize(buckets, ItemInterface::create(empty_key)); + resize_data(buckets); } void clear() @@ -125,7 +126,7 @@ public: if (data.empty() && data.capacity() >= newsize) { LUAU_ASSERT(count == 0); - data.resize(newsize, ItemInterface::create(empty_key)); + resize_data(newsize); return; } @@ -169,6 +170,21 @@ public: return const_iterator(this, data.size()); } + iterator begin() + { + size_t start = 0; + + while (start < data.size() && eq(ItemInterface::getKey(data[start]), empty_key)) + start++; + + return iterator(this, start); + } + + iterator end() + { + return iterator(this, data.size()); + } + size_t size() const { return count; @@ -233,7 +249,82 @@ public: size_t index; }; + class iterator + { + public: + iterator() + : set(0) + , index(0) + { + } + + iterator(DenseHashTable* set, size_t index) + : set(set) + , index(index) + { + } + + MutableItem& operator*() const + { + return *reinterpret_cast(&set->data[index]); + } + + MutableItem* operator->() const + { + return reinterpret_cast(&set->data[index]); + } + + bool operator==(const iterator& other) const + { + return set == other.set && index == other.index; + } + + bool operator!=(const iterator& other) const + { + return set != other.set || index != other.index; + } + + iterator& operator++() + { + size_t size = set->data.size(); + + do + { + index++; + } while (index < size && set->eq(ItemInterface::getKey(set->data[index]), set->empty_key)); + + return *this; + } + + iterator operator++(int) + { + iterator res = *this; + ++*this; + return res; + } + + private: + DenseHashTable* set; + size_t index; + }; + private: + template + void resize_data(size_t count, typename std::enable_if_t>* dummy = nullptr) + { + data.resize(count, ItemInterface::create(empty_key)); + } + + template + void resize_data(size_t count, typename std::enable_if_t>* dummy = nullptr) + { + size_t size = data.size(); + data.resize(count); + + for (size_t i = size; i < count; i++) + data[i].first = empty_key; + } + std::vector data; size_t count; Key empty_key; @@ -290,6 +381,7 @@ class DenseHashSet public: typedef typename Impl::const_iterator const_iterator; + typedef typename Impl::iterator iterator; DenseHashSet(const Key& empty_key, size_t buckets = 0) : impl(empty_key, buckets) @@ -336,6 +428,16 @@ public: { return impl.end(); } + + iterator begin() + { + return impl.begin(); + } + + iterator end() + { + return impl.end(); + } }; // This is a faster alternative of unordered_map, but it does not implement the same interface (i.e. it does not support erasing and has @@ -348,6 +450,7 @@ class DenseHashMap public: typedef typename Impl::const_iterator const_iterator; + typedef typename Impl::iterator iterator; DenseHashMap(const Key& empty_key, size_t buckets = 0) : impl(empty_key, buckets) @@ -401,10 +504,21 @@ public: { return impl.begin(); } + const_iterator end() const { return impl.end(); } + + iterator begin() + { + return impl.begin(); + } + + iterator end() + { + return impl.end(); + } }; } // namespace Luau diff --git a/Ast/include/Luau/Lexer.h b/Ast/include/Luau/Lexer.h index d7d867f4..4f3dbbd5 100644 --- a/Ast/include/Luau/Lexer.h +++ b/Ast/include/Luau/Lexer.h @@ -173,7 +173,7 @@ public: } const Lexeme& next(); - const Lexeme& next(bool skipComments); + const Lexeme& next(bool skipComments, bool updatePrevLocation); void nextline(); Lexeme lookahead(); diff --git a/Ast/src/Lexer.cpp b/Ast/src/Lexer.cpp index 70c6c78d..5dd4f04e 100644 --- a/Ast/src/Lexer.cpp +++ b/Ast/src/Lexer.cpp @@ -349,13 +349,11 @@ void Lexer::setReadNames(bool read) const Lexeme& Lexer::next() { - return next(this->skipComments); + return next(this->skipComments, true); } -const Lexeme& Lexer::next(bool skipComments) +const Lexeme& Lexer::next(bool skipComments, bool updatePrevLocation) { - bool first = true; - // in skipComments mode we reject valid comments do { @@ -363,11 +361,11 @@ const Lexeme& Lexer::next(bool skipComments) while (isSpace(peekch())) consume(); - if (!FFlag::LuauParseLocationIgnoreCommentSkip || first) + if (!FFlag::LuauParseLocationIgnoreCommentSkip || updatePrevLocation) prevLocation = lexeme.location; lexeme = readNext(); - first = false; + updatePrevLocation = false; } while (skipComments && (lexeme.type == Lexeme::Comment || lexeme.type == Lexeme::BlockComment)); return lexeme; diff --git a/Ast/src/Parser.cpp b/Ast/src/Parser.cpp index f9d32178..badd3fd3 100644 --- a/Ast/src/Parser.cpp +++ b/Ast/src/Parser.cpp @@ -11,6 +11,7 @@ LUAU_FASTINTVARIABLE(LuauRecursionLimit, 1000) LUAU_FASTINTVARIABLE(LuauParseErrorLimit, 100) LUAU_FASTFLAGVARIABLE(LuauParseRecoverUnexpectedPack, false) +LUAU_FASTFLAGVARIABLE(LuauParseLocationIgnoreCommentSkipInCapture, false) namespace Luau { @@ -2789,7 +2790,7 @@ void Parser::nextLexeme() { if (options.captureComments) { - Lexeme::Type type = lexer.next(/* skipComments= */ false).type; + Lexeme::Type type = lexer.next(/* skipComments= */ false, true).type; while (type == Lexeme::BrokenComment || type == Lexeme::Comment || type == Lexeme::BlockComment) { @@ -2813,7 +2814,7 @@ void Parser::nextLexeme() hotcomments.push_back({hotcommentHeader, lexeme.location, std::string(text + 1, text + end)}); } - type = lexer.next(/* skipComments= */ false).type; + type = lexer.next(/* skipComments= */ false, !FFlag::LuauParseLocationIgnoreCommentSkipInCapture).type; } } else diff --git a/Compiler/src/Compiler.cpp b/Compiler/src/Compiler.cpp index 6330bf1f..8ef69e75 100644 --- a/Compiler/src/Compiler.cpp +++ b/Compiler/src/Compiler.cpp @@ -1386,8 +1386,8 @@ struct Compiler const Constant* cv = constants.find(expr->index); - if (cv && cv->type == Constant::Type_Number && double(int(cv->valueNumber)) == cv->valueNumber && cv->valueNumber >= 1 && - cv->valueNumber <= 256) + if (cv && cv->type == Constant::Type_Number && cv->valueNumber >= 1 && cv->valueNumber <= 256 && + double(int(cv->valueNumber)) == cv->valueNumber) { uint8_t rt = compileExprAuto(expr->expr, rs); uint8_t i = uint8_t(int(cv->valueNumber) - 1); diff --git a/Compiler/src/CostModel.cpp b/Compiler/src/CostModel.cpp new file mode 100644 index 00000000..d8511bdb --- /dev/null +++ b/Compiler/src/CostModel.cpp @@ -0,0 +1,258 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +#include "CostModel.h" + +#include "Luau/Common.h" +#include "Luau/DenseHash.h" + +namespace Luau +{ +namespace Compile +{ + +inline uint64_t parallelAddSat(uint64_t x, uint64_t y) +{ + uint64_t s = x + y; + uint64_t m = s & 0x8080808080808080ull; // saturation mask + + return (s ^ m) | (m - (m >> 7)); +} + +struct Cost +{ + static const uint64_t kLiteral = ~0ull; + + // cost model: 8 bytes, where first byte is the baseline cost, and the next 7 bytes are discounts for when variable #i is constant + uint64_t model; + // constant mask: 8-byte 0xff mask; equal to all ff's for literals, for variables only byte #i (1+) is set to align with model + uint64_t constant; + + Cost(int cost = 0, uint64_t constant = 0) + : model(cost < 0x7f ? cost : 0x7f) + , constant(constant) + { + } + + Cost operator+(const Cost& other) const + { + Cost result; + result.model = parallelAddSat(model, other.model); + return result; + } + + Cost& operator+=(const Cost& other) + { + model = parallelAddSat(model, other.model); + constant = 0; + return *this; + } + + static Cost fold(const Cost& x, const Cost& y) + { + uint64_t newmodel = parallelAddSat(x.model, y.model); + uint64_t newconstant = x.constant & y.constant; + + // the extra cost for folding is 1; the discount is 1 for the variable that is shared by x&y (or whichever one is used in x/y if the other is + // literal) + uint64_t extra = (newconstant == kLiteral) ? 0 : (1 | (0x0101010101010101ull & newconstant)); + + Cost result; + result.model = parallelAddSat(newmodel, extra); + result.constant = newconstant; + + return result; + } +}; + +struct CostVisitor : AstVisitor +{ + DenseHashMap vars; + Cost result; + + CostVisitor() + : vars(nullptr) + { + } + + Cost model(AstExpr* node) + { + if (AstExprGroup* expr = node->as()) + { + return model(expr->expr); + } + else if (node->is() || node->is() || node->is() || + node->is()) + { + return Cost(0, Cost::kLiteral); + } + else if (AstExprLocal* expr = node->as()) + { + const uint64_t* i = vars.find(expr->local); + + return Cost(0, i ? *i : 0); // locals typically don't require extra instructions to compute + } + else if (node->is()) + { + return 1; + } + else if (node->is()) + { + return 3; + } + else if (AstExprCall* expr = node->as()) + { + Cost cost = 3; + cost += model(expr->func); + + for (size_t i = 0; i < expr->args.size; ++i) + { + Cost ac = model(expr->args.data[i]); + // for constants/locals we still need to copy them to the argument list + cost += ac.model == 0 ? Cost(1) : ac; + } + + return cost; + } + else if (AstExprIndexName* expr = node->as()) + { + return model(expr->expr) + 1; + } + else if (AstExprIndexExpr* expr = node->as()) + { + return model(expr->expr) + model(expr->index) + 1; + } + else if (AstExprFunction* expr = node->as()) + { + return 10; // high baseline cost due to allocation + } + else if (AstExprTable* expr = node->as()) + { + Cost cost = 10; // high baseline cost due to allocation + + for (size_t i = 0; i < expr->items.size; ++i) + { + const AstExprTable::Item& item = expr->items.data[i]; + + if (item.key) + cost += model(item.key); + + cost += model(item.value); + cost += 1; + } + + return cost; + } + else if (AstExprUnary* expr = node->as()) + { + return Cost::fold(model(expr->expr), Cost(0, Cost::kLiteral)); + } + else if (AstExprBinary* expr = node->as()) + { + return Cost::fold(model(expr->left), model(expr->right)); + } + else if (AstExprTypeAssertion* expr = node->as()) + { + return model(expr->expr); + } + else if (AstExprIfElse* expr = node->as()) + { + return model(expr->condition) + model(expr->trueExpr) + model(expr->falseExpr) + 2; + } + else + { + LUAU_ASSERT(!"Unknown expression type"); + return {}; + } + } + + void assign(AstExpr* expr) + { + // variable assignments reset variable mask, so that further uses of this variable aren't discounted + // this doesn't work perfectly with backwards control flow like loops, but is good enough for a single pass + if (AstExprLocal* lv = expr->as()) + if (uint64_t* i = vars.find(lv->local)) + *i = 0; + } + + bool visit(AstExpr* node) override + { + // note: we short-circuit the visitor traversal through any expression trees by returning false + // recursive traversal is happening inside model() which makes it easier to get the resulting value of the subexpression + result += model(node); + + return false; + } + + bool visit(AstStat* node) override + { + if (node->is()) + result += 2; + else if (node->is() || node->is() || node->is() || node->is()) + result += 2; + else if (node->is() || node->is()) + result += 1; + + return true; + } + + bool visit(AstStatLocal* node) override + { + for (size_t i = 0; i < node->values.size; ++i) + { + Cost arg = model(node->values.data[i]); + + // propagate constant mask from expression through variables + if (arg.constant && i < node->vars.size) + vars[node->vars.data[i]] = arg.constant; + + result += arg; + } + + return false; + } + + bool visit(AstStatAssign* node) override + { + for (size_t i = 0; i < node->vars.size; ++i) + assign(node->vars.data[i]); + + return true; + } + + bool visit(AstStatCompoundAssign* node) override + { + assign(node->var); + + // if lhs is not a local, setting it requires an extra table operation + result += node->var->is() ? 1 : 2; + + return true; + } +}; + +uint64_t modelCost(AstNode* root, AstLocal* const* vars, size_t varCount) +{ + CostVisitor visitor; + for (size_t i = 0; i < varCount && i < 7; ++i) + visitor.vars[vars[i]] = 0xffull << (i * 8 + 8); + + root->visit(&visitor); + + return visitor.result.model; +} + +int computeCost(uint64_t model, const bool* varsConst, size_t varCount) +{ + int cost = int(model & 0x7f); + + // don't apply discounts to what is likely a saturated sum + if (cost == 0x7f) + return cost; + + for (size_t i = 0; i < varCount && i < 7; ++i) + cost -= int((model >> (8 * i + 8)) & 0x7f) * varsConst[i]; + + return cost; +} + +} // namespace Compile +} // namespace Luau diff --git a/Compiler/src/CostModel.h b/Compiler/src/CostModel.h new file mode 100644 index 00000000..c27861ec --- /dev/null +++ b/Compiler/src/CostModel.h @@ -0,0 +1,18 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +#pragma once + +#include "Luau/Ast.h" + +namespace Luau +{ +namespace Compile +{ + +// cost model: 8 bytes, where first byte is the baseline cost, and the next 7 bytes are discounts for when variable #i is constant +uint64_t modelCost(AstNode* root, AstLocal* const* vars, size_t varCount); + +// cost is computed as B - sum(Di * Ci), where B is baseline cost, Di is the discount for each variable and Ci is 1 when variable #i is constant +int computeCost(uint64_t model, const bool* varsConst, size_t varCount); + +} // namespace Compile +} // namespace Luau diff --git a/Sources.cmake b/Sources.cmake index 6f110f1f..60e5dfda 100644 --- a/Sources.cmake +++ b/Sources.cmake @@ -32,11 +32,13 @@ target_sources(Luau.Compiler PRIVATE Compiler/src/Compiler.cpp Compiler/src/Builtins.cpp Compiler/src/ConstantFolding.cpp + Compiler/src/CostModel.cpp Compiler/src/TableShape.cpp Compiler/src/ValueTracking.cpp Compiler/src/lcode.cpp Compiler/src/Builtins.h Compiler/src/ConstantFolding.h + Compiler/src/CostModel.h Compiler/src/TableShape.h Compiler/src/ValueTracking.h ) @@ -58,6 +60,7 @@ target_sources(Luau.Analysis PRIVATE Analysis/include/Luau/LValue.h Analysis/include/Luau/Module.h Analysis/include/Luau/ModuleResolver.h + Analysis/include/Luau/Normalize.h Analysis/include/Luau/Predicate.h Analysis/include/Luau/Quantify.h Analysis/include/Luau/RecursionCounter.h @@ -94,6 +97,7 @@ target_sources(Luau.Analysis PRIVATE Analysis/src/Linter.cpp Analysis/src/LValue.cpp Analysis/src/Module.cpp + Analysis/src/Normalize.cpp Analysis/src/Quantify.cpp Analysis/src/RequireTracer.cpp Analysis/src/Scope.cpp @@ -216,6 +220,7 @@ if(TARGET Luau.UnitTest) tests/Autocomplete.test.cpp tests/BuiltinDefinitions.test.cpp tests/Compiler.test.cpp + tests/CostModel.test.cpp tests/Config.test.cpp tests/Error.test.cpp tests/Frontend.test.cpp @@ -224,6 +229,7 @@ if(TARGET Luau.UnitTest) tests/LValue.test.cpp tests/Module.test.cpp tests/NonstrictMode.test.cpp + tests/Normalize.test.cpp tests/Parser.test.cpp tests/RequireTracer.test.cpp tests/StringUtils.test.cpp diff --git a/VM/src/ltable.cpp b/VM/src/ltable.cpp index 1c75c0b0..dc40b6ef 100644 --- a/VM/src/ltable.cpp +++ b/VM/src/ltable.cpp @@ -34,7 +34,7 @@ #include LUAU_FASTFLAGVARIABLE(LuauTableRehashRework, false) -LUAU_FASTFLAGVARIABLE(LuauTableNewBoundary, false) +LUAU_FASTFLAGVARIABLE(LuauTableNewBoundary2, false) // max size of both array and hash part is 2^MAXBITS #define MAXBITS 26 @@ -390,6 +390,8 @@ static void resize(lua_State* L, Table* t, int nasize, int nhsize) setarrayvector(L, t, nasize); /* create new hash part with appropriate size */ setnodevector(L, t, nhsize); + /* used for the migration check at the end */ + LuaNode* nnew = t->node; if (nasize < oldasize) { /* array part must shrink? */ t->sizearray = nasize; @@ -413,6 +415,8 @@ static void resize(lua_State* L, Table* t, int nasize, int nhsize) /* shrink array */ luaM_reallocarray(L, t->array, oldasize, nasize, TValue, t->memcat); } + /* used for the migration check at the end */ + TValue* anew = t->array; /* re-insert elements from hash part */ if (FFlag::LuauTableRehashRework) { @@ -441,14 +445,30 @@ static void resize(lua_State* L, Table* t, int nasize, int nhsize) } } + /* make sure we haven't recursively rehashed during element migration */ + LUAU_ASSERT(nnew == t->node); + LUAU_ASSERT(anew == t->array); + if (nold != dummynode) luaM_freearray(L, nold, twoto(oldhsize), LuaNode, t->memcat); /* free old array */ } +static int adjustasize(Table* t, int size, const TValue* ek) +{ + LUAU_ASSERT(FFlag::LuauTableNewBoundary2); + bool tbound = t->node != dummynode || size < t->sizearray; + int ekindex = ek && ttisnumber(ek) ? arrayindex(nvalue(ek)) : -1; + /* move the array size up until the boundary is guaranteed to be inside the array part */ + while (size + 1 == ekindex || (tbound && !ttisnil(luaH_getnum(t, size + 1)))) + size++; + return size; +} + void luaH_resizearray(lua_State* L, Table* t, int nasize) { int nsize = (t->node == dummynode) ? 0 : sizenode(t); - resize(L, t, nasize, nsize); + int asize = FFlag::LuauTableNewBoundary2 ? adjustasize(t, nasize, NULL) : nasize; + resize(L, t, asize, nsize); } void luaH_resizehash(lua_State* L, Table* t, int nhsize) @@ -470,21 +490,12 @@ static void rehash(lua_State* L, Table* t, const TValue* ek) totaluse++; /* compute new size for array part */ int na = computesizes(nums, &nasize); + int nh = totaluse - na; /* enforce the boundary invariant; for performance, only do hash lookups if we must */ - if (FFlag::LuauTableNewBoundary) - { - bool tbound = t->node != dummynode || nasize < t->sizearray; - int ekindex = ttisnumber(ek) ? arrayindex(nvalue(ek)) : -1; - /* move the array size up until the boundary is guaranteed to be inside the array part */ - while (nasize + 1 == ekindex || (tbound && !ttisnil(luaH_getnum(t, nasize + 1)))) - { - nasize++; - na++; - } - } + if (FFlag::LuauTableNewBoundary2) + nasize = adjustasize(t, nasize, ek); /* resize the table to new computed sizes */ - LUAU_ASSERT(na <= totaluse); - resize(L, t, nasize, totaluse - na); + resize(L, t, nasize, nh); } /* @@ -544,7 +555,7 @@ static LuaNode* getfreepos(Table* t) static TValue* newkey(lua_State* L, Table* t, const TValue* key) { /* enforce boundary invariant */ - if (FFlag::LuauTableNewBoundary && ttisnumber(key) && nvalue(key) == t->sizearray + 1) + if (FFlag::LuauTableNewBoundary2 && ttisnumber(key) && nvalue(key) == t->sizearray + 1) { rehash(L, t, key); /* grow table */ @@ -735,7 +746,7 @@ TValue* luaH_setstr(lua_State* L, Table* t, TString* key) static LUAU_NOINLINE int unbound_search(Table* t, unsigned int j) { - LUAU_ASSERT(!FFlag::LuauTableNewBoundary); + LUAU_ASSERT(!FFlag::LuauTableNewBoundary2); unsigned int i = j; /* i is zero or a present index */ j++; /* find `i' and `j' such that i is present and j is not */ @@ -820,7 +831,7 @@ int luaH_getn(Table* t) maybesetaboundary(t, boundary); return boundary; } - else if (FFlag::LuauTableNewBoundary) + else if (FFlag::LuauTableNewBoundary2) { /* validate boundary invariant */ LUAU_ASSERT(t->node == dummynode || ttisnil(luaH_getnum(t, j + 1))); diff --git a/VM/src/ltablib.cpp b/VM/src/ltablib.cpp index 41887f4b..9c1f387e 100644 --- a/VM/src/ltablib.cpp +++ b/VM/src/ltablib.cpp @@ -199,7 +199,7 @@ static int tmove(lua_State* L) int tt = !lua_isnoneornil(L, 5) ? 5 : 1; /* destination table */ luaL_checktype(L, tt, LUA_TTABLE); - void (*telemetrycb)(lua_State* L, int f, int e, int t, int nf, int nt) = lua_table_move_telemetry; + void (*telemetrycb)(lua_State * L, int f, int e, int t, int nf, int nt) = lua_table_move_telemetry; if (DFFlag::LuauTableMoveTelemetry2 && telemetrycb && e >= f) { diff --git a/VM/src/lvmexecute.cpp b/VM/src/lvmexecute.cpp index 34949efb..39c60eac 100644 --- a/VM/src/lvmexecute.cpp +++ b/VM/src/lvmexecute.cpp @@ -16,7 +16,7 @@ #include -LUAU_FASTFLAG(LuauTableNewBoundary) +LUAU_FASTFLAG(LuauTableNewBoundary2) // Disable c99-designator to avoid the warning in CGOTO dispatch table #ifdef __clang__ @@ -2268,7 +2268,7 @@ static void luau_execute(lua_State* L) VM_NEXT(); } } - else if (FFlag::LuauTableNewBoundary || (h->lsizenode == 0 && ttisnil(gval(h->node)))) + else if (FFlag::LuauTableNewBoundary2 || (h->lsizenode == 0 && ttisnil(gval(h->node)))) { // fallthrough to exit VM_NEXT(); diff --git a/tests/CostModel.test.cpp b/tests/CostModel.test.cpp new file mode 100644 index 00000000..ec04932f --- /dev/null +++ b/tests/CostModel.test.cpp @@ -0,0 +1,101 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +#include "Luau/Parser.h" + +#include "doctest.h" + +using namespace Luau; + +namespace Luau +{ +namespace Compile +{ + +uint64_t modelCost(AstNode* root, AstLocal* const* vars, size_t varCount); +int computeCost(uint64_t model, const bool* varsConst, size_t varCount); + +} // namespace Compile +} // namespace Luau + +TEST_SUITE_BEGIN("CostModel"); + +static uint64_t modelFunction(const char* source) +{ + Allocator allocator; + AstNameTable names(allocator); + + ParseResult result = Parser::parse(source, strlen(source), names, allocator); + REQUIRE(result.root != nullptr); + + AstStatFunction* func = result.root->body.data[0]->as(); + REQUIRE(func); + + return Luau::Compile::modelCost(func->func->body, func->func->args.data, func->func->args.size); +} + +TEST_CASE("Expression") +{ + uint64_t model = modelFunction(R"( +function test(a, b, c) + return a + (b + 1) * (b + 1) - c +end +)"); + + const bool args1[] = {false, false, false}; + const bool args2[] = {false, true, false}; + + CHECK_EQ(5, Luau::Compile::computeCost(model, args1, 3)); + CHECK_EQ(2, Luau::Compile::computeCost(model, args2, 3)); +} + +TEST_CASE("PropagateVariable") +{ + uint64_t model = modelFunction(R"( +function test(a) + local b = a * a * a + return b * b +end +)"); + + const bool args1[] = {false}; + const bool args2[] = {true}; + + CHECK_EQ(3, Luau::Compile::computeCost(model, args1, 1)); + CHECK_EQ(0, Luau::Compile::computeCost(model, args2, 1)); +} + +TEST_CASE("LoopAssign") +{ + uint64_t model = modelFunction(R"( +function test(a) + for i=1,3 do + a[i] = i + end +end +)"); + + const bool args1[] = {false}; + const bool args2[] = {true}; + + // loop baseline cost is 2 + CHECK_EQ(3, Luau::Compile::computeCost(model, args1, 1)); + CHECK_EQ(3, Luau::Compile::computeCost(model, args2, 1)); +} + +TEST_CASE("MutableVariable") +{ + uint64_t model = modelFunction(R"( +function test(a, b) + local x = a * a + x += b + return x * x +end +)"); + + const bool args1[] = {false}; + const bool args2[] = {true}; + + CHECK_EQ(3, Luau::Compile::computeCost(model, args1, 1)); + CHECK_EQ(2, Luau::Compile::computeCost(model, args2, 1)); +} + +TEST_SUITE_END(); diff --git a/tests/Fixture.cpp b/tests/Fixture.cpp index 9dc9feee..d8b37a65 100644 --- a/tests/Fixture.cpp +++ b/tests/Fixture.cpp @@ -231,7 +231,7 @@ ModulePtr Fixture::getMainModule() SourceModule* Fixture::getMainSourceModule() { - return frontend.getSourceModule(fromString("MainModule")); + return frontend.getSourceModule(fromString(mainModuleName)); } std::optional Fixture::getPrimitiveType(TypeId ty) @@ -259,7 +259,7 @@ std::optional Fixture::getType(const std::string& name) TypeId Fixture::requireType(const std::string& name) { std::optional ty = getType(name); - REQUIRE(bool(ty)); + REQUIRE_MESSAGE(bool(ty), "Unable to requireType \"" << name << "\""); return follow(*ty); } diff --git a/tests/JsonEncoder.test.cpp b/tests/JsonEncoder.test.cpp index cb508072..1d2ad645 100644 --- a/tests/JsonEncoder.test.cpp +++ b/tests/JsonEncoder.test.cpp @@ -1,6 +1,7 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #include "Luau/Ast.h" #include "Luau/JsonEncoder.h" +#include "Luau/Parser.h" #include "doctest.h" @@ -50,4 +51,26 @@ TEST_CASE("encode_AstStatBlock") toJson(&block)); } +TEST_CASE("encode_tables") +{ + std::string src = R"( + local x: { + foo: number + } = { + foo = 123, + } + )"; + + Allocator allocator; + AstNameTable names(allocator); + ParseResult parseResult = Parser::parse(src.c_str(), src.length(), names, allocator); + + REQUIRE(parseResult.errors.size() == 0); + std::string json = toJson(parseResult.root); + + CHECK( + json == + R"({"type":"AstStatBlock","location":"0,0 - 6,4","body":[{"type":"AstStatLocal","location":"1,8 - 5,9","vars":[{"type":{"type":"AstTypeTable","location":"1,17 - 3,9","props":[{"name":"foo","location":"2,12 - 2,15","type":{"type":"AstTypeReference","location":"2,17 - 2,23","name":"number","parameters":[]}}],"indexer":false},"name":"x","location":"1,14 - 1,15"}],"values":[{"type":"AstExprTable","location":"3,12 - 5,9","items":[{"kind":"record","key":{"type":"AstExprConstantString","location":"4,12 - 4,15","value":"foo"},"value":{"type":"AstExprConstantNumber","location":"4,18 - 4,21","value":123}}]}]}]})"); +} + TEST_SUITE_END(); diff --git a/tests/Linter.test.cpp b/tests/Linter.test.cpp index 9ce9a4c2..05ee9a7b 100644 --- a/tests/Linter.test.cpp +++ b/tests/Linter.test.cpp @@ -597,8 +597,6 @@ return foo1 TEST_CASE_FIXTURE(Fixture, "UnknownType") { - ScopedFastFlag sff("LuauLintNoRobloxBits", true); - unfreeze(typeChecker.globalTypes); TableTypeVar::Props instanceProps{ {"ClassName", {typeChecker.anyType}}, @@ -1439,6 +1437,7 @@ TEST_CASE_FIXTURE(Fixture, "DeprecatedApi") { unfreeze(typeChecker.globalTypes); TypeId instanceType = typeChecker.globalTypes.addType(ClassTypeVar{"Instance", {}, std::nullopt, std::nullopt, {}, {}}); + persist(instanceType); typeChecker.globalScope->exportedTypeBindings["Instance"] = TypeFun{{}, instanceType}; getMutable(instanceType)->props = { diff --git a/tests/Module.test.cpp b/tests/Module.test.cpp index de063121..738893db 100644 --- a/tests/Module.test.cpp +++ b/tests/Module.test.cpp @@ -2,6 +2,7 @@ #include "Luau/Clone.h" #include "Luau/Module.h" #include "Luau/Scope.h" +#include "Luau/RecursionCounter.h" #include "Fixture.h" @@ -9,6 +10,8 @@ using namespace Luau; +LUAU_FASTFLAG(LuauLowerBoundsCalculation); + TEST_SUITE_BEGIN("ModuleTests"); TEST_CASE_FIXTURE(Fixture, "is_within_comment") @@ -42,29 +45,23 @@ TEST_CASE_FIXTURE(Fixture, "is_within_comment") TEST_CASE_FIXTURE(Fixture, "dont_clone_persistent_primitive") { TypeArena dest; - - SeenTypes seenTypes; - SeenTypePacks seenTypePacks; CloneState cloneState; // numberType is persistent. We leave it as-is. - TypeId newNumber = clone(typeChecker.numberType, dest, seenTypes, seenTypePacks, cloneState); + TypeId newNumber = clone(typeChecker.numberType, dest, cloneState); CHECK_EQ(newNumber, typeChecker.numberType); } TEST_CASE_FIXTURE(Fixture, "deepClone_non_persistent_primitive") { TypeArena dest; - - SeenTypes seenTypes; - SeenTypePacks seenTypePacks; CloneState cloneState; // Create a new number type that isn't persistent unfreeze(typeChecker.globalTypes); TypeId oldNumber = typeChecker.globalTypes.addType(PrimitiveTypeVar{PrimitiveTypeVar::Number}); freeze(typeChecker.globalTypes); - TypeId newNumber = clone(oldNumber, dest, seenTypes, seenTypePacks, cloneState); + TypeId newNumber = clone(oldNumber, dest, cloneState); CHECK_NE(newNumber, oldNumber); CHECK_EQ(*oldNumber, *newNumber); @@ -90,12 +87,9 @@ TEST_CASE_FIXTURE(Fixture, "deepClone_cyclic_table") TypeId counterType = requireType("Cyclic"); - SeenTypes seenTypes; - SeenTypePacks seenTypePacks; - CloneState cloneState; - TypeArena dest; - TypeId counterCopy = clone(counterType, dest, seenTypes, seenTypePacks, cloneState); + CloneState cloneState; + TypeId counterCopy = clone(counterType, dest, cloneState); TableTypeVar* ttv = getMutable(counterCopy); REQUIRE(ttv != nullptr); @@ -112,8 +106,11 @@ TEST_CASE_FIXTURE(Fixture, "deepClone_cyclic_table") REQUIRE(methodReturnType); CHECK_EQ(methodReturnType, counterCopy); - CHECK_EQ(2, dest.typePacks.size()); // one for the function args, and another for its return type - CHECK_EQ(2, dest.typeVars.size()); // One table and one function + if (FFlag::LuauLowerBoundsCalculation) + CHECK_EQ(3, dest.typePacks.size()); // function args, its return type, and the hidden any... pack + else + CHECK_EQ(2, dest.typePacks.size()); // one for the function args, and another for its return type + CHECK_EQ(2, dest.typeVars.size()); // One table and one function } TEST_CASE_FIXTURE(Fixture, "builtin_types_point_into_globalTypes_arena") @@ -143,15 +140,12 @@ TEST_CASE_FIXTURE(Fixture, "builtin_types_point_into_globalTypes_arena") TEST_CASE_FIXTURE(Fixture, "deepClone_union") { TypeArena dest; - - SeenTypes seenTypes; - SeenTypePacks seenTypePacks; CloneState cloneState; unfreeze(typeChecker.globalTypes); TypeId oldUnion = typeChecker.globalTypes.addType(UnionTypeVar{{typeChecker.numberType, typeChecker.stringType}}); freeze(typeChecker.globalTypes); - TypeId newUnion = clone(oldUnion, dest, seenTypes, seenTypePacks, cloneState); + TypeId newUnion = clone(oldUnion, dest, cloneState); CHECK_NE(newUnion, oldUnion); CHECK_EQ("number | string", toString(newUnion)); @@ -161,15 +155,12 @@ TEST_CASE_FIXTURE(Fixture, "deepClone_union") TEST_CASE_FIXTURE(Fixture, "deepClone_intersection") { TypeArena dest; - - SeenTypes seenTypes; - SeenTypePacks seenTypePacks; CloneState cloneState; unfreeze(typeChecker.globalTypes); TypeId oldIntersection = typeChecker.globalTypes.addType(IntersectionTypeVar{{typeChecker.numberType, typeChecker.stringType}}); freeze(typeChecker.globalTypes); - TypeId newIntersection = clone(oldIntersection, dest, seenTypes, seenTypePacks, cloneState); + TypeId newIntersection = clone(oldIntersection, dest, cloneState); CHECK_NE(newIntersection, oldIntersection); CHECK_EQ("number & string", toString(newIntersection)); @@ -191,12 +182,9 @@ TEST_CASE_FIXTURE(Fixture, "clone_class") std::nullopt, &exampleMetaClass, {}, {}}}; TypeArena dest; - - SeenTypes seenTypes; - SeenTypePacks seenTypePacks; CloneState cloneState; - TypeId cloned = clone(&exampleClass, dest, seenTypes, seenTypePacks, cloneState); + TypeId cloned = clone(&exampleClass, dest, cloneState); const ClassTypeVar* ctv = get(cloned); REQUIRE(ctv != nullptr); @@ -216,16 +204,14 @@ TEST_CASE_FIXTURE(Fixture, "clone_sanitize_free_types") TypePackVar freeTp(FreeTypePack{TypeLevel{}}); TypeArena dest; - SeenTypes seenTypes; - SeenTypePacks seenTypePacks; CloneState cloneState; - TypeId clonedTy = clone(&freeTy, dest, seenTypes, seenTypePacks, cloneState); + TypeId clonedTy = clone(&freeTy, dest, cloneState); CHECK_EQ("any", toString(clonedTy)); CHECK(cloneState.encounteredFreeType); cloneState = {}; - TypePackId clonedTp = clone(&freeTp, dest, seenTypes, seenTypePacks, cloneState); + TypePackId clonedTp = clone(&freeTp, dest, cloneState); CHECK_EQ("...any", toString(clonedTp)); CHECK(cloneState.encounteredFreeType); } @@ -237,16 +223,32 @@ TEST_CASE_FIXTURE(Fixture, "clone_seal_free_tables") ttv->state = TableState::Free; TypeArena dest; - SeenTypes seenTypes; - SeenTypePacks seenTypePacks; CloneState cloneState; - TypeId cloned = clone(&tableTy, dest, seenTypes, seenTypePacks, cloneState); + TypeId cloned = clone(&tableTy, dest, cloneState); const TableTypeVar* clonedTtv = get(cloned); CHECK_EQ(clonedTtv->state, TableState::Sealed); CHECK(cloneState.encounteredFreeType); } +TEST_CASE_FIXTURE(Fixture, "clone_constrained_intersection") +{ + TypeArena src; + + TypeId constrained = src.addType(ConstrainedTypeVar{TypeLevel{}, {getSingletonTypes().numberType, getSingletonTypes().stringType}}); + + TypeArena dest; + CloneState cloneState; + + TypeId cloned = clone(constrained, dest, cloneState); + CHECK_NE(constrained, cloned); + + const ConstrainedTypeVar* ctv = get(cloned); + REQUIRE_EQ(2, ctv->parts.size()); + CHECK_EQ(getSingletonTypes().numberType, ctv->parts[0]); + CHECK_EQ(getSingletonTypes().stringType, ctv->parts[1]); +} + TEST_CASE_FIXTURE(Fixture, "clone_self_property") { ScopedFastFlag sff{"LuauAnyInIsOptionalIsOptional", true}; @@ -284,6 +286,7 @@ TEST_CASE_FIXTURE(Fixture, "clone_recursion_limit") int limit = 400; #endif ScopedFastInt luauTypeCloneRecursionLimit{"LuauTypeCloneRecursionLimit", limit}; + ScopedFastFlag sff{"LuauRecursionLimitException", true}; TypeArena src; @@ -299,11 +302,9 @@ TEST_CASE_FIXTURE(Fixture, "clone_recursion_limit") } TypeArena dest; - SeenTypes seenTypes; - SeenTypePacks seenTypePacks; CloneState cloneState; - CHECK_THROWS_AS(clone(table, dest, seenTypes, seenTypePacks, cloneState), std::runtime_error); + CHECK_THROWS_AS(clone(table, dest, cloneState), RecursionLimitException); } TEST_SUITE_END(); diff --git a/tests/NonstrictMode.test.cpp b/tests/NonstrictMode.test.cpp index d3faea2a..a8a12b69 100644 --- a/tests/NonstrictMode.test.cpp +++ b/tests/NonstrictMode.test.cpp @@ -275,4 +275,38 @@ TEST_CASE_FIXTURE(Fixture, "inconsistent_module_return_types_are_ok") REQUIRE_EQ("any", toString(getMainModule()->getModuleScope()->returnType)); } +TEST_CASE_FIXTURE(Fixture, "returning_insufficient_return_values") +{ + CheckResult result = check(R"( + --!nonstrict + + function foo(): (boolean, string?) + if true then + return true, "hello" + else + return false + end + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "returning_too_many_values") +{ + CheckResult result = check(R"( + --!nonstrict + + function foo(): boolean + if true then + return true, "hello" + else + return false + end + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + TEST_SUITE_END(); diff --git a/tests/Normalize.test.cpp b/tests/Normalize.test.cpp new file mode 100644 index 00000000..5a84201a --- /dev/null +++ b/tests/Normalize.test.cpp @@ -0,0 +1,967 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details + +#include "Fixture.h" + +#include "doctest.h" + +#include "Luau/Normalize.h" +#include "Luau/BuiltinDefinitions.h" + +using namespace Luau; + +struct NormalizeFixture : Fixture +{ + ScopedFastFlag sff1{"LuauLowerBoundsCalculation", true}; + ScopedFastFlag sff2{"LuauTableSubtypingVariance2", true}; +}; + +void createSomeClasses(TypeChecker& typeChecker) +{ + auto& arena = typeChecker.globalTypes; + + unfreeze(arena); + + TypeId parentType = arena.addType(ClassTypeVar{"Parent", {}, std::nullopt, std::nullopt, {}, nullptr}); + + ClassTypeVar* parentClass = getMutable(parentType); + parentClass->props["method"] = {makeFunction(arena, parentType, {}, {})}; + + parentClass->props["virtual_method"] = {makeFunction(arena, parentType, {}, {})}; + + addGlobalBinding(typeChecker, "Parent", {parentType}); + typeChecker.globalScope->exportedTypeBindings["Parent"] = TypeFun{{}, parentType}; + + TypeId childType = arena.addType(ClassTypeVar{"Child", {}, parentType, std::nullopt, {}, nullptr}); + + ClassTypeVar* childClass = getMutable(childType); + childClass->props["virtual_method"] = {makeFunction(arena, childType, {}, {})}; + + addGlobalBinding(typeChecker, "Child", {childType}); + typeChecker.globalScope->exportedTypeBindings["Child"] = TypeFun{{}, childType}; + + TypeId unrelatedType = arena.addType(ClassTypeVar{"Unrelated", {}, std::nullopt, std::nullopt, {}, nullptr}); + + addGlobalBinding(typeChecker, "Unrelated", {unrelatedType}); + typeChecker.globalScope->exportedTypeBindings["Unrelated"] = TypeFun{{}, unrelatedType}; + + freeze(arena); +} + +static bool isSubtype(TypeId a, TypeId b) +{ + InternalErrorReporter ice; + return isSubtype(a, b, ice); +} + +TEST_SUITE_BEGIN("isSubtype"); + +TEST_CASE_FIXTURE(NormalizeFixture, "primitives") +{ + check(R"( + local a = 41 + local b = 32 + + local c = "hello" + local d = "world" + )"); + + TypeId a = requireType("a"); + TypeId b = requireType("b"); + TypeId c = requireType("c"); + TypeId d = requireType("d"); + + CHECK(isSubtype(b, a)); + CHECK(isSubtype(d, c)); + CHECK(!isSubtype(d, a)); +} + +TEST_CASE_FIXTURE(NormalizeFixture, "functions") +{ + check(R"( + function a(x: number): number return x end + function b(x: number): number return x end + + function c(x: number?): number return x end + function d(x: number): number? return x end + )"); + + TypeId a = requireType("a"); + TypeId b = requireType("b"); + TypeId c = requireType("c"); + TypeId d = requireType("d"); + + CHECK(isSubtype(b, a)); + CHECK(isSubtype(c, a)); + CHECK(!isSubtype(d, a)); + CHECK(isSubtype(a, d)); +} + +TEST_CASE_FIXTURE(NormalizeFixture, "functions_and_any") +{ + check(R"( + function a(n: number) return "string" end + function b(q: any) return 5 :: any end + )"); + + TypeId a = requireType("a"); + TypeId b = requireType("b"); + + // Intuition: + // We cannot use b where a is required because we cannot rely on b to return a string. + // We cannot use a where b is required because we cannot rely on a to accept non-number arguments. + + CHECK(!isSubtype(b, a)); + CHECK(!isSubtype(a, b)); +} + +TEST_CASE_FIXTURE(NormalizeFixture, "intersection_of_functions_of_different_arities") +{ + check(R"( + type A = (any) -> () + type B = (any, any) -> () + type T = A & B + + local a: A + local b: B + local t: T + )"); + + TypeId a = requireType("a"); + TypeId b = requireType("b"); + + CHECK(!isSubtype(a, b)); // !! + CHECK(!isSubtype(b, a)); + + CHECK("((any) -> ()) & ((any, any) -> ())" == toString(requireType("t"))); +} + +TEST_CASE_FIXTURE(NormalizeFixture, "functions_with_mismatching_arity") +{ + check(R"( + local a: (number) -> () + local b: () -> () + + local c: () -> number + )"); + + TypeId a = requireType("a"); + TypeId b = requireType("b"); + TypeId c = requireType("c"); + + CHECK(!isSubtype(b, a)); + CHECK(!isSubtype(c, a)); + + CHECK(!isSubtype(a, b)); + CHECK(!isSubtype(c, b)); + + CHECK(!isSubtype(a, c)); + CHECK(!isSubtype(b, c)); +} + +TEST_CASE_FIXTURE(NormalizeFixture, "functions_with_mismatching_arity_but_optional_parameters") +{ + /* + * (T0..TN) <: (T0..TN, A?) + * (T0..TN) <: (T0..TN, any) + * (T0..TN, A?) R <: U -> S if U <: T and R <: S + * A | B <: T if A <: T and B <: T + * T <: A | B if T <: A or T <: B + */ + check(R"( + local a: (number?) -> () + local b: (number) -> () + local c: (number, number?) -> () + )"); + + TypeId a = requireType("a"); + TypeId b = requireType("b"); + TypeId c = requireType("c"); + + /* + * (number) -> () () + * because number? () + * because number? () <: (number) -> () + * because number <: number? (because number <: number) + */ + CHECK(isSubtype(a, b)); + + /* + * (number, number?) -> () <: (number) -> (number) + * The packs have inequal lengths, but (number) <: (number, number?) + * and number <: number + */ + CHECK(!isSubtype(c, b)); + + /* + * (number?) -> () () + * because (number, number?) () () + * because (number, number?) () + local b: (number) -> () + local c: (number, any) -> () + )"); + + TypeId a = requireType("a"); + TypeId b = requireType("b"); + TypeId c = requireType("c"); + + /* + * (number) -> () () + * because number? () + * because number? () <: (number) -> () + * because number <: number? (because number <: number) + */ + CHECK(isSubtype(a, b)); + + /* + * (number, any) -> () (number) + * The packs have inequal lengths + */ + CHECK(!isSubtype(c, b)); + + /* + * (number?) -> () () + * The packs have inequal lengths + */ + CHECK(!isSubtype(a, c)); + + /* + * (number) -> () () + * The packs have inequal lengths + */ + CHECK(!isSubtype(b, c)); +} + +TEST_CASE_FIXTURE(NormalizeFixture, "variadic_functions_with_no_head") +{ + check(R"( + local a: (...number) -> () + local b: (...number?) -> () + )"); + + TypeId a = requireType("a"); + TypeId b = requireType("b"); + + CHECK(isSubtype(b, a)); + CHECK(!isSubtype(a, b)); +} + +#if 0 +TEST_CASE_FIXTURE(NormalizeFixture, "variadic_function_with_head") +{ + check(R"( + local a: (...number) -> () + local b: (number, number) -> () + )"); + + TypeId a = requireType("a"); + TypeId b = requireType("b"); + + CHECK(!isSubtype(b, a)); + CHECK(isSubtype(a, b)); +} +#endif + +TEST_CASE_FIXTURE(NormalizeFixture, "union") +{ + check(R"( + local a: number | string + local b: number + local c: string + local d: number? + )"); + + TypeId a = requireType("a"); + TypeId b = requireType("b"); + TypeId c = requireType("c"); + TypeId d = requireType("d"); + + CHECK(isSubtype(b, a)); + CHECK(!isSubtype(a, b)); + + CHECK(isSubtype(c, a)); + CHECK(!isSubtype(a, c)); + + CHECK(!isSubtype(d, a)); + CHECK(!isSubtype(a, d)); + + CHECK(isSubtype(b, d)); + CHECK(!isSubtype(d, b)); +} + +TEST_CASE_FIXTURE(NormalizeFixture, "table_with_union_prop") +{ + check(R"( + local a: {x: number} + local b: {x: number?} + )"); + + TypeId a = requireType("a"); + TypeId b = requireType("b"); + + CHECK(isSubtype(a, b)); + CHECK(!isSubtype(b, a)); +} + +TEST_CASE_FIXTURE(NormalizeFixture, "table_with_any_prop") +{ + check(R"( + local a: {x: number} + local b: {x: any} + )"); + + TypeId a = requireType("a"); + TypeId b = requireType("b"); + + CHECK(isSubtype(a, b)); + CHECK(!isSubtype(b, a)); +} + +TEST_CASE_FIXTURE(NormalizeFixture, "intersection") +{ + check(R"( + local a: number & string + local b: number + local c: string + local d: number & nil + )"); + + TypeId a = requireType("a"); + TypeId b = requireType("b"); + TypeId c = requireType("c"); + TypeId d = requireType("d"); + + CHECK(!isSubtype(b, a)); + CHECK(isSubtype(a, b)); + + CHECK(!isSubtype(c, a)); + CHECK(isSubtype(a, c)); + + CHECK(!isSubtype(d, a)); + CHECK(!isSubtype(a, d)); +} + +TEST_CASE_FIXTURE(NormalizeFixture, "union_and_intersection") +{ + check(R"( + local a: number & string + local b: number | nil + )"); + + TypeId a = requireType("a"); + TypeId b = requireType("b"); + + CHECK(!isSubtype(b, a)); + CHECK(isSubtype(a, b)); +} + +TEST_CASE_FIXTURE(NormalizeFixture, "table_with_table_prop") +{ + check(R"( + type T = {x: {y: number}} & {x: {y: string}} + local a: T + )"); + + CHECK_EQ("{| x: {| y: number & string |} |}", toString(requireType("a"))); +} + +#if 0 +TEST_CASE_FIXTURE(NormalizeFixture, "tables") +{ + check(R"( + local a: {x: number} + local b: {x: any} + local c: {y: number} + local d: {x: number, y: number} + )"); + + TypeId a = requireType("a"); + TypeId b = requireType("b"); + TypeId c = requireType("c"); + TypeId d = requireType("d"); + + CHECK(isSubtype(a, b)); + CHECK(!isSubtype(b, a)); + + CHECK(!isSubtype(c, a)); + CHECK(!isSubtype(a, c)); + + CHECK(isSubtype(d, a)); + CHECK(!isSubtype(a, d)); + + CHECK(isSubtype(d, b)); + CHECK(!isSubtype(b, d)); +} + +TEST_CASE_FIXTURE(NormalizeFixture, "table_indexers_are_invariant") +{ + check(R"( + local a: {[string]: number} + local b: {[string]: any} + local c: {[string]: number} + )"); + + TypeId a = requireType("a"); + TypeId b = requireType("b"); + TypeId c = requireType("c"); + + CHECK(!isSubtype(b, a)); + CHECK(!isSubtype(a, b)); + + CHECK(isSubtype(c, a)); + CHECK(isSubtype(a, c)); +} + +TEST_CASE_FIXTURE(NormalizeFixture, "mismatched_indexers") +{ + check(R"( + local a: {x: number} + local b: {[string]: number} + local c: {} + )"); + + TypeId a = requireType("a"); + TypeId b = requireType("b"); + TypeId c = requireType("c"); + + CHECK(isSubtype(b, a)); + CHECK(!isSubtype(a, b)); + + CHECK(!isSubtype(c, b)); + CHECK(isSubtype(b, c)); +} + +TEST_CASE_FIXTURE(NormalizeFixture, "cyclic_table") +{ + check(R"( + type A = {method: (A) -> ()} + local a: A + + type B = {method: (any) -> ()} + local b: B + + type C = {method: (C) -> ()} + local c: C + + type D = {method: (D) -> (), another: (D) -> ()} + local d: D + + type E = {method: (A) -> (), another: (E) -> ()} + local e: E + )"); + + TypeId a = requireType("a"); + TypeId b = requireType("b"); + TypeId c = requireType("c"); + TypeId d = requireType("d"); + TypeId e = requireType("e"); + + CHECK(isSubtype(b, a)); + CHECK(!isSubtype(a, b)); + + CHECK(isSubtype(c, a)); + CHECK(isSubtype(a, c)); + + CHECK(!isSubtype(d, a)); + CHECK(!isSubtype(a, d)); + + CHECK(isSubtype(e, a)); + CHECK(!isSubtype(a, e)); +} +#endif + +TEST_CASE_FIXTURE(NormalizeFixture, "classes") +{ + createSomeClasses(typeChecker); + + TypeId p = typeChecker.globalScope->lookupType("Parent")->type; + TypeId c = typeChecker.globalScope->lookupType("Child")->type; + TypeId u = typeChecker.globalScope->lookupType("Unrelated")->type; + + CHECK(isSubtype(c, p)); + CHECK(!isSubtype(p, c)); + CHECK(!isSubtype(u, p)); + CHECK(!isSubtype(p, u)); +} + +#if 0 +TEST_CASE_FIXTURE(NormalizeFixture, "metatable" * doctest::expected_failures{1}) +{ + check(R"( + local T = {} + T.__index = T + function T.new() + return setmetatable({}, T) + end + + function T:method() end + + local a: typeof(T.new) + local b: {method: (any) -> ()} + )"); + + TypeId a = requireType("a"); + TypeId b = requireType("b"); + + CHECK(isSubtype(a, b)); +} +#endif + +TEST_CASE_FIXTURE(NormalizeFixture, "intersection_of_tables") +{ + check(R"( + type T = {x: number} & ({x: number} & {y: string?}) + local t: T + )"); + + CHECK("{| x: number, y: string? |}" == toString(requireType("t"))); +} + +TEST_SUITE_END(); + +TEST_SUITE_BEGIN("Normalize"); + +TEST_CASE_FIXTURE(NormalizeFixture, "intersection_of_disjoint_tables") +{ + check(R"( + type T = {a: number} & {b: number} + local t: T + )"); + + CHECK_EQ("{| a: number, b: number |}", toString(requireType("t"))); +} + +TEST_CASE_FIXTURE(NormalizeFixture, "intersection_of_overlapping_tables") +{ + check(R"( + type T = {a: number, b: string} & {b: number, c: string} + local t: T + )"); + + CHECK_EQ("{| a: number, b: number & string, c: string |}", toString(requireType("t"))); +} + +TEST_CASE_FIXTURE(NormalizeFixture, "intersection_of_confluent_overlapping_tables") +{ + check(R"( + type T = {a: number, b: string} & {b: string, c: string} + local t: T + )"); + + CHECK_EQ("{| a: number, b: string, c: string |}", toString(requireType("t"))); +} + +TEST_CASE_FIXTURE(NormalizeFixture, "union_with_overlapping_field_that_has_a_subtype_relationship") +{ + check(R"( + local t: {x: number} | {x: number?} + )"); + + ModulePtr tempModule{new Module}; + + // HACK: Normalization is an in-place operation. We need to cheat a little here and unfreeze + // the arena that the type lives in. + ModulePtr mainModule = getMainModule(); + unfreeze(mainModule->internalTypes); + + TypeId tType = requireType("t"); + normalize(tType, tempModule, *typeChecker.iceHandler); + + CHECK_EQ("{| x: number? |}", toString(tType, {true})); +} + +TEST_CASE_FIXTURE(NormalizeFixture, "intersection_of_functions") +{ + check(R"( + type T = ((any) -> string) & ((number) -> string) + local t: T + )"); + + CHECK_EQ("(any) -> string", toString(requireType("t"))); +} + +TEST_CASE_FIXTURE(Fixture, "normalize_module_return_type") +{ + ScopedFastFlag sff[] = { + {"LuauLowerBoundsCalculation", true}, + }; + + check(R"( + --!nonstrict + + if Math.random() then + return function(initialState, handlers) + return function(state, action) + return state + end + end + else + return function(initialState, handlers) + return function(state, action) + return state + end + end + end + )"); + + CHECK_EQ("(any, any) -> (...any)", toString(getMainModule()->getModuleScope()->returnType)); +} + +TEST_CASE_FIXTURE(Fixture, "return_type_is_not_a_constrained_intersection") +{ + check(R"( + function foo(x:number, y:number) + return x + y + end + )"); + + CHECK_EQ("(number, number) -> number", toString(requireType("foo"))); +} + +TEST_CASE_FIXTURE(Fixture, "higher_order_function") +{ + check(R"( + function apply(f, x) + return f(x) + end + + local a = apply(function(x: number) return x + x end, 5) + )"); + + TypeId aType = requireType("a"); + CHECK_MESSAGE(isNumber(follow(aType)), "Expected a number but got ", toString(aType)); +} + +TEST_CASE_FIXTURE(Fixture, "higher_order_function_with_annotation") +{ + check(R"( + function apply(f: (a) -> b, x) + return f(x) + end + )"); + + CHECK_EQ("((a) -> b, a) -> b", toString(requireType("apply"))); +} + +TEST_CASE_FIXTURE(Fixture, "cyclic_table_is_marked_normal") +{ + ScopedFastFlag flags[] = { + {"LuauLowerBoundsCalculation", true}, + }; + + check(R"( + type Fiber = { + return_: Fiber? + } + + local f: Fiber + )"); + + TypeId t = requireType("f"); + CHECK(t->normal); +} + +TEST_CASE_FIXTURE(Fixture, "variadic_tail_is_marked_normal") +{ + ScopedFastFlag flags[] = { + {"LuauLowerBoundsCalculation", true}, + }; + + CheckResult result = check(R"( + type Weirdo = (...{x: number}) -> () + + local w: Weirdo + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + TypeId t = requireType("w"); + auto ftv = get(t); + REQUIRE(ftv); + + auto [argHead, argTail] = flatten(ftv->argTypes); + CHECK(argHead.empty()); + REQUIRE(argTail.has_value()); + + auto vtp = get(*argTail); + REQUIRE(vtp); + CHECK(vtp->ty->normal); +} + +TEST_CASE_FIXTURE(Fixture, "cyclic_table_normalizes_sensibly") +{ + CheckResult result = check(R"( + local Cyclic = {} + function Cyclic.get() + return Cyclic + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + TypeId ty = requireType("Cyclic"); + CHECK_EQ("t1 where t1 = { get: () -> t1 }", toString(ty, {true})); +} + +TEST_CASE_FIXTURE(Fixture, "union_of_distinct_free_types") +{ + ScopedFastFlag flags[] = { + {"LuauLowerBoundsCalculation", true}, + }; + + CheckResult result = check(R"( + function fussy(a, b) + if math.random() > 0.5 then + return a + else + return b + end + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK("(a, b) -> a | b" == toString(requireType("fussy"))); +} + +TEST_CASE_FIXTURE(Fixture, "constrained_intersection_of_intersections") +{ + ScopedFastFlag flags[] = { + {"LuauLowerBoundsCalculation", true}, + }; + + CheckResult result = check(R"( + local f : (() -> number) | ((number) -> number) + local g : (() -> number) | ((string) -> number) + + function h() + if math.random() then + return f + else + return g + end + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + TypeId h = requireType("h"); + + CHECK("() -> (() -> number) | ((number) -> number) | ((string) -> number)" == toString(h)); +} + +TEST_CASE_FIXTURE(Fixture, "intersection_inside_a_table_inside_another_intersection") +{ + ScopedFastFlag flags[] = { + {"LuauLowerBoundsCalculation", true}, + }; + + CheckResult result = check(R"( + type X = {} + type Y = {y: number} + type Z = {z: string} + type W = {w: boolean} + type T = {x: Y & X} & {x:Z & W} + + local x: X + local y: Y + local z: Z + local w: W + local t: T + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK("{| |}" == toString(requireType("x"), {true})); + CHECK("{| y: number |}" == toString(requireType("y"), {true})); + CHECK("{| z: string |}" == toString(requireType("z"), {true})); + CHECK("{| w: boolean |}" == toString(requireType("w"), {true})); + CHECK("{| x: {| w: boolean, y: number, z: string |} |}" == toString(requireType("t"), {true})); +} + +TEST_CASE_FIXTURE(Fixture, "intersection_inside_a_table_inside_another_intersection_2") +{ + ScopedFastFlag flags[] = { + {"LuauLowerBoundsCalculation", true}, + }; + + // We use a function and inferred parameter types to prevent intermediate normalizations from being performed. + // This exposes a bug where the type of y is mutated. + CheckResult result = check(R"( + function strange(w, x, y, z) + y.y = 5 + z.z = "five" + w.w = true + + type Z = {x: typeof(x) & typeof(y)} & {x: typeof(w) & typeof(z)} + + return ((nil :: any) :: Z) + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + TypeId t = requireType("strange"); + auto ftv = get(t); + REQUIRE(ftv != nullptr); + + std::vector args = flatten(ftv->argTypes).first; + + REQUIRE(4 == args.size()); + CHECK("{+ w: boolean +}" == toString(args[0])); + CHECK("a" == toString(args[1])); + CHECK("{+ y: number +}" == toString(args[2])); + CHECK("{+ z: string +}" == toString(args[3])); + + std::vector ret = flatten(ftv->retType).first; + + REQUIRE(1 == ret.size()); + CHECK("{| x: a & {- w: boolean, y: number, z: string -} |}" == toString(ret[0])); +} + +TEST_CASE_FIXTURE(Fixture, "intersection_inside_a_table_inside_another_intersection_3") +{ + ScopedFastFlag flags[] = { + {"LuauLowerBoundsCalculation", true}, + }; + + // We use a function and inferred parameter types to prevent intermediate normalizations from being performed. + // This exposes a bug where the type of y is mutated. + CheckResult result = check(R"( + function strange(x, y, z) + x.x = true + y.y = y + z.z = "five" + + type Z = {x: typeof(y)} & {x: typeof(x) & typeof(z)} + + return ((nil :: any) :: Z) + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + TypeId t = requireType("strange"); + auto ftv = get(t); + REQUIRE(ftv != nullptr); + + std::vector args = flatten(ftv->argTypes).first; + + REQUIRE(3 == args.size()); + CHECK("{+ x: boolean +}" == toString(args[0])); + CHECK("t1 where t1 = {+ y: t1 +}" == toString(args[1])); + CHECK("{+ z: string +}" == toString(args[2])); + + std::vector ret = flatten(ftv->retType).first; + + REQUIRE(1 == ret.size()); + CHECK("{| x: {- x: boolean, y: t1, z: string -} |} where t1 = {+ y: t1 +}" == toString(ret[0])); +} + +TEST_CASE_FIXTURE(Fixture, "intersection_inside_a_table_inside_another_intersection_4") +{ + ScopedFastFlag flags[] = { + {"LuauLowerBoundsCalculation", true}, + }; + + // We use a function and inferred parameter types to prevent intermediate normalizations from being performed. + // This exposes a bug where the type of y is mutated. + CheckResult result = check(R"( + function strange(x, y, z) + x.x = true + z.z = "five" + + type R = {x: typeof(y)} & {x: typeof(x) & typeof(z)} + local r: R + + y.y = r + + return r + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + TypeId t = requireType("strange"); + auto ftv = get(t); + REQUIRE(ftv != nullptr); + + std::vector args = flatten(ftv->argTypes).first; + + REQUIRE(3 == args.size()); + CHECK("{+ x: boolean +}" == toString(args[0])); + CHECK("{+ y: t1 +} where t1 = {| x: {- x: boolean, y: t1, z: string -} |}" == toString(args[1])); + CHECK("{+ z: string +}" == toString(args[2])); + + std::vector ret = flatten(ftv->retType).first; + + REQUIRE(1 == ret.size()); + CHECK("t1 where t1 = {| x: {- x: boolean, y: t1, z: string -} |}" == toString(ret[0])); +} + +TEST_CASE_FIXTURE(Fixture, "nested_table_normalization_with_non_table__no_ice") +{ + ScopedFastFlag flags[] = { + {"LuauLowerBoundsCalculation", true}, + {"LuauNormalizeCombineTableFix", true}, + }; + // CLI-52787 + // ends up combining {_:any} with any, recursively + // which used to ICE because this combines a table with a non-table. + CheckResult result = check(R"( + export type t0 = any & { _: {_:any} } & { _:any } + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "fuzz_failure_instersection_combine_must_follow") +{ + ScopedFastFlag flags[] = { + {"LuauLowerBoundsCalculation", true}, + {"LuauNormalizeCombineIntersectionFix", true}, + }; + + CheckResult result = check(R"( + export type t0 = {_:{_:any} & {_:any|string}} & {_:{_:{}}} + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_SUITE_END(); diff --git a/tests/Parser.test.cpp b/tests/Parser.test.cpp index 79f9ecab..b941103d 100644 --- a/tests/Parser.test.cpp +++ b/tests/Parser.test.cpp @@ -1618,6 +1618,26 @@ TEST_CASE_FIXTURE(Fixture, "end_extent_doesnt_consume_comments") CHECK_EQ((Position{1, 23}), block->body.data[0]->location.end); } +TEST_CASE_FIXTURE(Fixture, "end_extent_doesnt_consume_comments_even_with_capture") +{ + ScopedFastFlag luauParseLocationIgnoreCommentSkip{"LuauParseLocationIgnoreCommentSkip", true}; + ScopedFastFlag luauParseLocationIgnoreCommentSkipInCapture{"LuauParseLocationIgnoreCommentSkipInCapture", true}; + + // Same should hold when comments are captured + ParseOptions opts; + opts.captureComments = true; + + AstStatBlock* block = parse(R"( + type F = number + --comment + print('hello') + )", + opts); + + REQUIRE_EQ(2, block->body.size); + CHECK_EQ((Position{1, 23}), block->body.data[0]->location.end); +} + TEST_CASE_FIXTURE(Fixture, "parse_error_loop_control") { matchParseError("break", "break statement must be inside a loop"); diff --git a/tests/ToDot.test.cpp b/tests/ToDot.test.cpp index 29bdd866..f3fda54e 100644 --- a/tests/ToDot.test.cpp +++ b/tests/ToDot.test.cpp @@ -7,6 +7,8 @@ #include "doctest.h" +LUAU_FASTFLAG(LuauLowerBoundsCalculation) + using namespace Luau; struct ToDotClassFixture : Fixture @@ -101,9 +103,34 @@ local function f(a, ...: string) return a end )"); LUAU_REQUIRE_NO_ERRORS(result); + CHECK_EQ("(a, ...string) -> a", toString(requireType("f"))); + ToDotOptions opts; opts.showPointers = false; - CHECK_EQ(R"(digraph graphname { + + if (FFlag::LuauLowerBoundsCalculation) + { + CHECK_EQ(R"(digraph graphname { +n1 [label="FunctionTypeVar 1"]; +n1 -> n2 [label="arg"]; +n2 [label="TypePack 2"]; +n2 -> n3; +n3 [label="GenericTypeVar 3"]; +n2 -> n4 [label="tail"]; +n4 [label="VariadicTypePack 4"]; +n4 -> n5; +n5 [label="string"]; +n1 -> n6 [label="ret"]; +n6 [label="TypePack 6"]; +n6 -> n7; +n7 [label="BoundTypeVar 7"]; +n7 -> n3; +})", + toDot(requireType("f"), opts)); + } + else + { + CHECK_EQ(R"(digraph graphname { n1 [label="FunctionTypeVar 1"]; n1 -> n2 [label="arg"]; n2 [label="TypePack 2"]; @@ -119,7 +146,8 @@ n6 -> n7; n7 [label="TypePack 7"]; n7 -> n3; })", - toDot(requireType("f"), opts)); + toDot(requireType("f"), opts)); + } } TEST_CASE_FIXTURE(Fixture, "union") @@ -361,4 +389,49 @@ n3 [label="number"]; toDot(*ty, opts)); } +TEST_CASE_FIXTURE(Fixture, "constrained") +{ + // ConstrainedTypeVars never appear in the final type graph, so we have to create one directly + // to dotify it. + TypeVar t{ConstrainedTypeVar{TypeLevel{}, {typeChecker.numberType, typeChecker.stringType, typeChecker.nilType}}}; + + ToDotOptions opts; + opts.showPointers = false; + + CHECK_EQ(R"(digraph graphname { +n1 [label="ConstrainedTypeVar 1"]; +n1 -> n2; +n2 [label="number"]; +n1 -> n3; +n3 [label="string"]; +n1 -> n4; +n4 [label="nil"]; +})", + toDot(&t, opts)); +} + +TEST_CASE_FIXTURE(Fixture, "singletontypes") +{ + CheckResult result = check(R"( + local x: "hi" | "\"hello\"" | true | false + )"); + + ToDotOptions opts; + opts.showPointers = false; + + CHECK_EQ(R"(digraph graphname { +n1 [label="UnionTypeVar 1"]; +n1 -> n2; +n2 [label="SingletonTypeVar string: hi"]; +n1 -> n3; +)" +"n3 [label=\"SingletonTypeVar string: \\\"hello\\\"\"];" +R"( +n1 -> n4; +n4 [label="SingletonTypeVar boolean: true"]; +n1 -> n5; +n5 [label="SingletonTypeVar boolean: false"]; +})", toDot(requireType("x"), opts)); +} + TEST_SUITE_END(); diff --git a/tests/ToString.test.cpp b/tests/ToString.test.cpp index 3051e209..ccf5c583 100644 --- a/tests/ToString.test.cpp +++ b/tests/ToString.test.cpp @@ -9,6 +9,8 @@ using namespace Luau; +LUAU_FASTFLAG(LuauRecursiveTypeParameterRestriction); + TEST_SUITE_BEGIN("ToString"); TEST_CASE_FIXTURE(Fixture, "primitive") diff --git a/tests/TopoSort.test.cpp b/tests/TopoSort.test.cpp index 9b990866..1f14ae88 100644 --- a/tests/TopoSort.test.cpp +++ b/tests/TopoSort.test.cpp @@ -340,26 +340,28 @@ TEST_CASE_FIXTURE(Fixture, "nested_type_annotations_depends_on_later_typealiases TEST_CASE_FIXTURE(Fixture, "return_comes_last") { - CheckResult result = check(R"( -export type Module = { bar: (number) -> boolean, foo: () -> string } + AstStatBlock* program = parse(R"( + local module = {} -return function() : Module - local module = {} + local function confuseCompiler() return module.foo() end - local function confuseCompiler() return module.foo() end - - module.foo = function() return "" end + module.foo = function() return "" end - function module.bar(x:number) - confuseCompiler() - return true - end - - return module -end + function module.bar(x:number) + confuseCompiler() + return true + end + + return module )"); - LUAU_REQUIRE_NO_ERRORS(result); + auto sorted = toposort(*program); + + CHECK_EQ(sorted[0], program->body.data[0]); + CHECK_EQ(sorted[2], program->body.data[1]); + CHECK_EQ(sorted[1], program->body.data[2]); + CHECK_EQ(sorted[3], program->body.data[3]); + CHECK_EQ(sorted[4], program->body.data[4]); } TEST_CASE_FIXTURE(Fixture, "break_comes_last") diff --git a/tests/Transpiler.test.cpp b/tests/Transpiler.test.cpp index 5ac45ff2..0c324cd0 100644 --- a/tests/Transpiler.test.cpp +++ b/tests/Transpiler.test.cpp @@ -388,7 +388,7 @@ TEST_CASE_FIXTURE(Fixture, "type_lists_should_be_emitted_correctly") std::string actual = decorateWithTypes(code); - CHECK_EQ(expected, decorateWithTypes(code)); + CHECK_EQ(expected, actual); } TEST_CASE_FIXTURE(Fixture, "function_type_location") diff --git a/tests/TypeInfer.annotations.test.cpp b/tests/TypeInfer.annotations.test.cpp index 2ad11d01..e2971ad5 100644 --- a/tests/TypeInfer.annotations.test.cpp +++ b/tests/TypeInfer.annotations.test.cpp @@ -753,4 +753,14 @@ TEST_CASE_FIXTURE(Fixture, "occurs_check_on_cyclic_intersection_typevar") REQUIRE(ocf); } +TEST_CASE_FIXTURE(Fixture, "instantiation_clone_has_to_follow") +{ + CheckResult result = check(R"( + export type t8 = (t0)&(((true)|(any))->"") + export type t0 = ({})&({_:{[any]:number},}) + )"); + + LUAU_REQUIRE_ERRORS(result); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.builtins.test.cpp b/tests/TypeInfer.builtins.test.cpp index c6fbebed..1ae65947 100644 --- a/tests/TypeInfer.builtins.test.cpp +++ b/tests/TypeInfer.builtins.test.cpp @@ -8,6 +8,8 @@ using namespace Luau; +LUAU_FASTFLAG(LuauLowerBoundsCalculation); + TEST_SUITE_BEGIN("BuiltinTests"); TEST_CASE_FIXTURE(Fixture, "math_things_are_defined") @@ -557,9 +559,9 @@ TEST_CASE_FIXTURE(Fixture, "xpcall") )"); LUAU_REQUIRE_NO_ERRORS(result); - REQUIRE_EQ("boolean", toString(requireType("a"))); - REQUIRE_EQ("number", toString(requireType("b"))); - REQUIRE_EQ("boolean", toString(requireType("c"))); + CHECK_EQ("boolean", toString(requireType("a"))); + CHECK_EQ("number", toString(requireType("b"))); + CHECK_EQ("boolean", toString(requireType("c"))); } TEST_CASE_FIXTURE(Fixture, "see_thru_select") @@ -881,7 +883,10 @@ TEST_CASE_FIXTURE(Fixture, "assert_removes_falsy_types") )"); LUAU_REQUIRE_NO_ERRORS(result); - CHECK_EQ("((boolean | number)?) -> boolean | number", toString(requireType("f"))); + if (FFlag::LuauLowerBoundsCalculation) + CHECK_EQ("((boolean | number)?) -> number | true", toString(requireType("f"))); + else + CHECK_EQ("((boolean | number)?) -> boolean | number", toString(requireType("f"))); } TEST_CASE_FIXTURE(Fixture, "assert_removes_falsy_types2") diff --git a/tests/TypeInfer.classes.test.cpp b/tests/TypeInfer.classes.test.cpp index 98fa66eb..8e3629e7 100644 --- a/tests/TypeInfer.classes.test.cpp +++ b/tests/TypeInfer.classes.test.cpp @@ -91,6 +91,9 @@ struct ClassFixture : Fixture typeChecker.globalScope->exportedTypeBindings["Vector2"] = TypeFun{{}, vector2InstanceType}; addGlobalBinding(typeChecker, "Vector2", vector2Type, "@test"); + for (const auto& [name, tf] : typeChecker.globalScope->exportedTypeBindings) + persist(tf.type); + freeze(arena); } }; diff --git a/tests/TypeInfer.functions.test.cpp b/tests/TypeInfer.functions.test.cpp index 1713216a..65993681 100644 --- a/tests/TypeInfer.functions.test.cpp +++ b/tests/TypeInfer.functions.test.cpp @@ -13,6 +13,8 @@ using namespace Luau; +LUAU_FASTFLAG(LuauLowerBoundsCalculation); + TEST_SUITE_BEGIN("TypeInferFunctions"); TEST_CASE_FIXTURE(Fixture, "tc_function") @@ -98,7 +100,7 @@ TEST_CASE_FIXTURE(Fixture, "vararg_function_is_quantified") end return result - end + end return T )"); @@ -274,6 +276,10 @@ TEST_CASE_FIXTURE(Fixture, "cyclic_function_type_in_rets") TEST_CASE_FIXTURE(Fixture, "cyclic_function_type_in_args") { + ScopedFastFlag sff[] = { + {"LuauLowerBoundsCalculation", true}, + }; + CheckResult result = check(R"( function f(g) return f(f) @@ -281,7 +287,7 @@ TEST_CASE_FIXTURE(Fixture, "cyclic_function_type_in_args") )"); LUAU_REQUIRE_NO_ERRORS(result); - CHECK_EQ("t1 where t1 = (t1) -> ()", toString(requireType("f"))); + CHECK_EQ("t1 where t1 = (t1) -> (a...)", toString(requireType("f"))); } TEST_CASE_FIXTURE(Fixture, "another_higher_order_function") @@ -481,10 +487,10 @@ TEST_CASE_FIXTURE(Fixture, "infer_higher_order_function") std::vector fArgs = flatten(fType->argTypes).first; - TypeId xType = argVec[1]; + TypeId xType = follow(argVec[1]); CHECK_EQ(1, fArgs.size()); - CHECK_EQ(xType, fArgs[0]); + CHECK_EQ(xType, follow(fArgs[0])); } TEST_CASE_FIXTURE(Fixture, "higher_order_function_2") @@ -1043,13 +1049,16 @@ f(function(x) return x * 2 end) LUAU_REQUIRE_ERROR_COUNT(1, result); CHECK_EQ("Type 'number' could not be converted into 'Table'", toString(result.errors[0])); - // Return type doesn't inference 'nil' - result = check(R"( -function f(a: (number) -> nil) return a(4) end -f(function(x) print(x) end) - )"); + if (!FFlag::LuauLowerBoundsCalculation) + { + // Return type doesn't inference 'nil' + result = check(R"( + function f(a: (number) -> nil) return a(4) end + f(function(x) print(x) end) + )"); - LUAU_REQUIRE_NO_ERRORS(result); + LUAU_REQUIRE_NO_ERRORS(result); + } } TEST_CASE_FIXTURE(Fixture, "infer_anonymous_function_arguments") @@ -1142,13 +1151,16 @@ f(function(x) return x * 2 end) LUAU_REQUIRE_ERROR_COUNT(1, result); CHECK_EQ("Type 'number' could not be converted into 'Table'", toString(result.errors[0])); - // Return type doesn't inference 'nil' - result = check(R"( -function f(a: (number) -> nil) return a(4) end -f(function(x) print(x) end) - )"); + if (!FFlag::LuauLowerBoundsCalculation) + { + // Return type doesn't inference 'nil' + result = check(R"( + function f(a: (number) -> nil) return a(4) end + f(function(x) print(x) end) + )"); - LUAU_REQUIRE_NO_ERRORS(result); + LUAU_REQUIRE_NO_ERRORS(result); + } } TEST_CASE_FIXTURE(Fixture, "infer_anonymous_function_arguments_outside_call") @@ -1338,6 +1350,126 @@ end CHECK_EQ(toString(result.errors[1]), R"(Type 'string' could not be converted into 'number')"); } +TEST_CASE_FIXTURE(Fixture, "inconsistent_return_types") +{ + const ScopedFastFlag flags[] = { + {"LuauLowerBoundsCalculation", true}, + }; + + CheckResult result = check(R"( + function foo(a: boolean, b: number) + if a then + return nil + else + return b + end + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + CHECK_EQ("(boolean, number) -> number?", toString(requireType("foo"))); + + // TODO: Test multiple returns + // Think of various cases where typepacks need to grow. maybe consult other tests + // Basic normalization of ConstrainedTypeVars during quantification +} + +TEST_CASE_FIXTURE(Fixture, "inconsistent_higher_order_function") +{ + const ScopedFastFlag flags[] = { + {"LuauLowerBoundsCalculation", true}, + }; + + CheckResult result = check(R"( + function foo(f) + f(5) + f("six") + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ("((number | string) -> (a...)) -> ()", toString(requireType("foo"))); +} + + +/* The bug here is that we are using the same level 2.0 for both the body of resolveDispatcher and the + * lambda useCallback. + * + * I think what we want to do is, at each scope level, never reuse the same sublevel. + * + * We also adjust checkBlock to consider the syntax `local x = function() ... end` to be sortable + * in the same way as `local function x() ... end`. This causes the function `resolveDispatcher` to be + * checked before the lambda. + */ +TEST_CASE_FIXTURE(Fixture, "inferred_higher_order_functions_are_quantified_at_the_right_time") +{ + ScopedFastFlag sff[] = { + {"LuauLowerBoundsCalculation", true}, + }; + + CheckResult result = check(R"( + --!strict + + local function resolveDispatcher() + return (nil :: any) :: {useCallback: (any) -> any} + end + + local useCallback = function(deps: any) + return resolveDispatcher().useCallback(deps) + end + )"); + + // LUAU_REQUIRE_NO_ERRORS is particularly unhelpful when this test is broken. + // You get a TypeMismatch error where both types stringify the same. + + CHECK(result.errors.empty()); + if (!result.errors.empty()) + { + for (const auto& e : result.errors) + printf("%s: %s\n", toString(e.location).c_str(), toString(e).c_str()); + } +} + +TEST_CASE_FIXTURE(Fixture, "inferred_higher_order_functions_are_quantified_at_the_right_time2") +{ + CheckResult result = check(R"( + --!strict + + local function resolveDispatcher() + return (nil :: any) :: {useContext: (number?) -> any} + end + + local useContext + useContext = function(unstable_observedBits: number?) + resolveDispatcher().useContext(unstable_observedBits) + end + )"); + + // LUAU_REQUIRE_NO_ERRORS is particularly unhelpful when this test is broken. + // You get a TypeMismatch error where both types stringify the same. + + CHECK(result.errors.empty()); + if (!result.errors.empty()) + { + for (const auto& e : result.errors) + printf("%s %s: %s\n", e.moduleName.c_str(), toString(e.location).c_str(), toString(e).c_str()); + } +} + +TEST_CASE_FIXTURE(Fixture, "inferred_higher_order_functions_are_quantified_at_the_right_time3") +{ + CheckResult result = check(R"( + local foo + + foo():bar(function() + return foo() + end) + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + TEST_CASE_FIXTURE(Fixture, "function_decl_non_self_unsealed_overwrite") { ScopedFastFlag statFunctionSimplify{"LuauStatFunctionSimplify4", true}; @@ -1471,4 +1603,17 @@ pcall(wrapper, test) CHECK(acm->isVariadic); } +TEST_CASE_FIXTURE(Fixture, "occurs_check_failure_in_function_return_type") +{ + CheckResult result = check(R"( + function f() + return 5, f() + end + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + CHECK(nullptr != get(result.errors[0])); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.generics.test.cpp b/tests/TypeInfer.generics.test.cpp index f360a77c..49d31fc6 100644 --- a/tests/TypeInfer.generics.test.cpp +++ b/tests/TypeInfer.generics.test.cpp @@ -230,8 +230,8 @@ TEST_CASE_FIXTURE(Fixture, "infer_generic_function") CHECK_EQ(idFun->generics.size(), 1); CHECK_EQ(idFun->genericPacks.size(), 0); - CHECK_EQ(args[0], idFun->generics[0]); - CHECK_EQ(rets[0], idFun->generics[0]); + CHECK_EQ(follow(args[0]), follow(idFun->generics[0])); + CHECK_EQ(follow(rets[0]), follow(idFun->generics[0])); } TEST_CASE_FIXTURE(Fixture, "infer_generic_local_function") @@ -253,8 +253,8 @@ TEST_CASE_FIXTURE(Fixture, "infer_generic_local_function") CHECK_EQ(idFun->generics.size(), 1); CHECK_EQ(idFun->genericPacks.size(), 0); - CHECK_EQ(args[0], idFun->generics[0]); - CHECK_EQ(rets[0], idFun->generics[0]); + CHECK_EQ(follow(args[0]), follow(idFun->generics[0])); + CHECK_EQ(follow(rets[0]), follow(idFun->generics[0])); } TEST_CASE_FIXTURE(Fixture, "infer_nested_generic_function") @@ -705,10 +705,10 @@ end TEST_CASE_FIXTURE(Fixture, "generic_functions_should_be_memory_safe") { ScopedFastFlag sffs[] = { - { "LuauTableSubtypingVariance2", true }, - { "LuauUnsealedTableLiteral", true }, - { "LuauPropertiesGetExpectedType", true }, - { "LuauRecursiveTypeParameterRestriction", true }, + {"LuauTableSubtypingVariance2", true}, + {"LuauUnsealedTableLiteral", true}, + {"LuauPropertiesGetExpectedType", true}, + {"LuauRecursiveTypeParameterRestriction", true}, }; CheckResult result = check(R"( @@ -843,6 +843,7 @@ TEST_CASE_FIXTURE(Fixture, "generic_function") LUAU_REQUIRE_NO_ERRORS(result); + CHECK_EQ("(a) -> a", toString(requireType("id"))); CHECK_EQ(*typeChecker.numberType, *requireType("a")); CHECK_EQ(*typeChecker.nilType, *requireType("b")); } @@ -1037,25 +1038,39 @@ TEST_CASE_FIXTURE(Fixture, "infer_generic_function_function_argument") ScopedFastFlag sff{"LuauUnsealedTableLiteral", true}; CheckResult result = check(R"( -local function sum(x: a, y: a, f: (a, a) -> a) return f(x, y) end -return sum(2, 3, function(a, b) return a + b end) + local function sum(x: a, y: a, f: (a, a) -> a) + return f(x, y) + end + return sum(2, 3, function(a, b) return a + b end) )"); LUAU_REQUIRE_NO_ERRORS(result); result = check(R"( -local function map(arr: {a}, f: (a) -> b) local r = {} for i,v in ipairs(arr) do table.insert(r, f(v)) end return r end -local a = {1, 2, 3} -local r = map(a, function(a) return a + a > 100 end) + local function map(arr: {a}, f: (a) -> b) + local r = {} + for i,v in ipairs(arr) do + table.insert(r, f(v)) + end + return r + end + local a = {1, 2, 3} + local r = map(a, function(a) return a + a > 100 end) )"); LUAU_REQUIRE_NO_ERRORS(result); REQUIRE_EQ("{boolean}", toString(requireType("r"))); check(R"( -local function foldl(arr: {a}, init: b, f: (b, a) -> b) local r = init for i,v in ipairs(arr) do r = f(r, v) end return r end -local a = {1, 2, 3} -local r = foldl(a, {s=0,c=0}, function(a, b) return {s = a.s + b, c = a.c + 1} end) + local function foldl(arr: {a}, init: b, f: (b, a) -> b) + local r = init + for i,v in ipairs(arr) do + r = f(r, v) + end + return r + end + local a = {1, 2, 3} + local r = foldl(a, {s=0,c=0}, function(a, b) return {s = a.s + b, c = a.c + 1} end) )"); LUAU_REQUIRE_NO_ERRORS(result); @@ -1065,25 +1080,19 @@ local r = foldl(a, {s=0,c=0}, function(a, b) return {s = a.s + b, c = a.c + 1} e TEST_CASE_FIXTURE(Fixture, "infer_generic_function_function_argument_overloaded") { CheckResult result = check(R"( -local function g1(a: T, f: (T) -> T) return f(a) end -local function g2(a: T, b: T, f: (T, T) -> T) return f(a, b) end + local g12: ((T, (T) -> T) -> T) & ((T, T, (T, T) -> T) -> T) -local g12: typeof(g1) & typeof(g2) - -g12(1, function(x) return x + x end) -g12(1, 2, function(x, y) return x + y end) + g12(1, function(x) return x + x end) + g12(1, 2, function(x, y) return x + y end) )"); LUAU_REQUIRE_NO_ERRORS(result); result = check(R"( -local function g1(a: T, f: (T) -> T) return f(a) end -local function g2(a: T, b: T, f: (T, T) -> T) return f(a, b) end + local g12: ((T, (T) -> T) -> T) & ((T, T, (T, T) -> T) -> T) -local g12: typeof(g1) & typeof(g2) - -g12({x=1}, function(x) return {x=-x.x} end) -g12({x=1}, {x=2}, function(x, y) return {x=x.x + y.x} end) + g12({x=1}, function(x) return {x=-x.x} end) + g12({x=1}, {x=2}, function(x, y) return {x=x.x + y.x} end) )"); LUAU_REQUIRE_NO_ERRORS(result); @@ -1121,12 +1130,12 @@ local c = sumrec(function(x, y, f) return f(x, y) end) -- type binders are not i TEST_CASE_FIXTURE(Fixture, "substitution_with_bound_table") { CheckResult result = check(R"( -type A = { x: number } -local a: A = { x = 1 } -local b = a -type B = typeof(b) -type X = T -local c: X + type A = { x: number } + local a: A = { x = 1 } + local b = a + type B = typeof(b) + type X = T + local c: X )"); LUAU_REQUIRE_NO_ERRORS(result); diff --git a/tests/TypeInfer.intersectionTypes.test.cpp b/tests/TypeInfer.intersectionTypes.test.cpp index ac7a6532..3675919f 100644 --- a/tests/TypeInfer.intersectionTypes.test.cpp +++ b/tests/TypeInfer.intersectionTypes.test.cpp @@ -8,6 +8,8 @@ using namespace Luau; +LUAU_FASTFLAG(LuauLowerBoundsCalculation); + TEST_SUITE_BEGIN("IntersectionTypes"); TEST_CASE_FIXTURE(Fixture, "select_correct_union_fn") @@ -306,7 +308,10 @@ TEST_CASE_FIXTURE(Fixture, "table_intersection_write_sealed") )"); LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK_EQ(toString(result.errors[0]), "Cannot add property 'z' to table 'X & Y'"); + if (FFlag::LuauLowerBoundsCalculation) + CHECK_EQ(toString(result.errors[0]), "Cannot add property 'z' to table '{| x: number, y: number |}'"); + else + CHECK_EQ(toString(result.errors[0]), "Cannot add property 'z' to table 'X & Y'"); } TEST_CASE_FIXTURE(Fixture, "table_intersection_write_sealed_indirect") @@ -314,27 +319,34 @@ TEST_CASE_FIXTURE(Fixture, "table_intersection_write_sealed_indirect") ScopedFastFlag statFunctionSimplify{"LuauStatFunctionSimplify4", true}; CheckResult result = check(R"( - type X = { x: (number) -> number } - type Y = { y: (string) -> string } + type X = { x: (number) -> number } + type Y = { y: (string) -> string } - type XY = X & Y + type XY = X & Y - local xy : XY = { - x = function(a: number) return -a end, - y = function(a: string) return a .. "b" end - } - function xy.z(a:number) return a * 10 end - function xy:y(a:number) return a * 10 end - function xy:w(a:number) return a * 10 end + local xy : XY = { + x = function(a: number) return -a end, + y = function(a: string) return a .. "b" end + } + function xy.z(a:number) return a * 10 end + function xy:y(a:number) return a * 10 end + function xy:w(a:number) return a * 10 end )"); LUAU_REQUIRE_ERROR_COUNT(4, result); CHECK_EQ(toString(result.errors[0]), R"(Type '(string, number) -> string' could not be converted into '(string) -> string' caused by: Argument count mismatch. Function expects 2 arguments, but only 1 is specified)"); - CHECK_EQ(toString(result.errors[1]), "Cannot add property 'z' to table 'X & Y'"); + if (FFlag::LuauLowerBoundsCalculation) + CHECK_EQ(toString(result.errors[1]), "Cannot add property 'z' to table '{| x: (number) -> number, y: (string) -> string |}'"); + else + CHECK_EQ(toString(result.errors[1]), "Cannot add property 'z' to table 'X & Y'"); CHECK_EQ(toString(result.errors[2]), "Type 'number' could not be converted into 'string'"); - CHECK_EQ(toString(result.errors[3]), "Cannot add property 'w' to table 'X & Y'"); + + if (FFlag::LuauLowerBoundsCalculation) + CHECK_EQ(toString(result.errors[3]), "Cannot add property 'w' to table '{| x: (number) -> number, y: (string) -> string |}'"); + else + CHECK_EQ(toString(result.errors[3]), "Cannot add property 'w' to table 'X & Y'"); } TEST_CASE_FIXTURE(Fixture, "table_write_sealed_indirect") @@ -375,6 +387,8 @@ TEST_CASE_FIXTURE(Fixture, "table_intersection_setmetatable") TEST_CASE_FIXTURE(Fixture, "error_detailed_intersection_part") { + ScopedFastFlag flags[] = {{"LuauLowerBoundsCalculation", false}}; + CheckResult result = check(R"( type X = { x: number } type Y = { y: number } @@ -393,6 +407,8 @@ caused by: TEST_CASE_FIXTURE(Fixture, "error_detailed_intersection_all") { + ScopedFastFlag flags[] = {{"LuauLowerBoundsCalculation", false}}; + CheckResult result = check(R"( type X = { x: number } type Y = { y: number } @@ -427,8 +443,8 @@ TEST_CASE_FIXTURE(Fixture, "no_stack_overflow_from_flattenintersection") repeat type t0 = ((any)|((any)&((any)|((any)&((any)|(any))))))&(t0) function _(l0):(t0)&(t0) - while nil do - end + while nil do + end end until _(_)(_)._ )"); diff --git a/tests/TypeInfer.oop.test.cpp b/tests/TypeInfer.oop.test.cpp index 40831bf6..5cd3f3ba 100644 --- a/tests/TypeInfer.oop.test.cpp +++ b/tests/TypeInfer.oop.test.cpp @@ -199,16 +199,16 @@ end TEST_CASE_FIXTURE(Fixture, "nonstrict_self_mismatch_tail") { CheckResult result = check(R"( ---!nonstrict -local f = {} -function f:foo(a: number, b: number) end + --!nonstrict + local f = {} + function f:foo(a: number, b: number) end -function bar(...) - f.foo(f, 1, ...) -end + function bar(...) + f.foo(f, 1, ...) + end -bar(2) -)"); + bar(2) + )"); LUAU_REQUIRE_NO_ERRORS(result); } diff --git a/tests/TypeInfer.operators.test.cpp b/tests/TypeInfer.operators.test.cpp index 6a8a9d93..5f2e2404 100644 --- a/tests/TypeInfer.operators.test.cpp +++ b/tests/TypeInfer.operators.test.cpp @@ -91,7 +91,8 @@ TEST_CASE_FIXTURE(Fixture, "primitive_arith_no_metatable") const FunctionTypeVar* functionType = get(requireType("add")); std::optional retType = first(functionType->retType); - CHECK_EQ(std::optional(typeChecker.numberType), retType); + REQUIRE(retType.has_value()); + CHECK_EQ(typeChecker.numberType, follow(*retType)); CHECK_EQ(requireType("n"), typeChecker.numberType); CHECK_EQ(requireType("s"), typeChecker.stringType); } diff --git a/tests/TypeInfer.provisional.test.cpp b/tests/TypeInfer.provisional.test.cpp index 2e16b21e..8c8059db 100644 --- a/tests/TypeInfer.provisional.test.cpp +++ b/tests/TypeInfer.provisional.test.cpp @@ -8,6 +8,7 @@ #include LUAU_FASTFLAG(LuauEqConstraint) +LUAU_FASTFLAG(LuauLowerBoundsCalculation) using namespace Luau; @@ -527,6 +528,7 @@ TEST_CASE_FIXTURE(Fixture, "invariant_table_properties_means_instantiating_table LUAU_REQUIRE_NO_ERRORS(result); } +// FIXME: Move this test to another source file when removing FFlag::LuauLowerBoundsCalculation TEST_CASE_FIXTURE(Fixture, "do_not_ice_when_trying_to_pick_first_of_generic_type_pack") { ScopedFastFlag sff[]{ @@ -556,10 +558,19 @@ TEST_CASE_FIXTURE(Fixture, "do_not_ice_when_trying_to_pick_first_of_generic_type LUAU_REQUIRE_NO_ERRORS(result); - // f and g should have the type () -> () - CHECK_EQ("() -> (a...)", toString(requireType("f"))); - CHECK_EQ("() -> (a...)", toString(requireType("g"))); - CHECK_EQ("any", toString(requireType("x"))); // any is returned instead of ICE for now + if (FFlag::LuauLowerBoundsCalculation) + { + CHECK_EQ("() -> ()", toString(requireType("f"))); + CHECK_EQ("() -> ()", toString(requireType("g"))); + CHECK_EQ("nil", toString(requireType("x"))); + } + else + { + // f and g should have the type () -> () + CHECK_EQ("() -> (a...)", toString(requireType("f"))); + CHECK_EQ("() -> (a...)", toString(requireType("g"))); + CHECK_EQ("any", toString(requireType("x"))); // any is returned instead of ICE for now + } } TEST_CASE_FIXTURE(Fixture, "specialization_binds_with_prototypes_too_early") @@ -575,6 +586,10 @@ TEST_CASE_FIXTURE(Fixture, "specialization_binds_with_prototypes_too_early") TEST_CASE_FIXTURE(Fixture, "weird_fail_to_unify_type_pack") { + ScopedFastFlag sff[] = { + {"LuauLowerBoundsCalculation", false}, + }; + CheckResult result = check(R"( local function f() return end local g = function() return f() end @@ -585,6 +600,10 @@ TEST_CASE_FIXTURE(Fixture, "weird_fail_to_unify_type_pack") TEST_CASE_FIXTURE(Fixture, "weird_fail_to_unify_variadic_pack") { + ScopedFastFlag sff[] = { + {"LuauLowerBoundsCalculation", false}, + }; + CheckResult result = check(R"( --!strict local function f(...) return ... end @@ -594,4 +613,108 @@ TEST_CASE_FIXTURE(Fixture, "weird_fail_to_unify_variadic_pack") LUAU_REQUIRE_ERRORS(result); // Should not have any errors. } +TEST_CASE_FIXTURE(Fixture, "lower_bounds_calculation_is_too_permissive_with_overloaded_higher_order_functions") +{ + ScopedFastFlag sff[] = { + {"LuauLowerBoundsCalculation", true}, + }; + + CheckResult result = check(R"( + function foo(f) + f(5, 'a') + f('b', 6) + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + // We incorrectly infer that the argument to foo could be called with (number, number) or (string, string) + // even though that is strictly more permissive than the actual source text shows. + CHECK("((number | string, number | string) -> (a...)) -> ()" == toString(requireType("foo"))); +} + +// Once fixed, move this to Normalize.test.cpp +TEST_CASE_FIXTURE(Fixture, "normalization_fails_on_certain_kinds_of_cyclic_tables") +{ + ScopedFastFlag flags[] = { + {"LuauLowerBoundsCalculation", true}, + }; + + // We use a function and inferred parameter types to prevent intermediate normalizations from being performed. + // This exposes a bug where the type of y is mutated. + CheckResult result = check(R"( + function strange(x, y) + x.x = y + y.x = x + + type R = {x: typeof(x)} & {x: typeof(y)} + local r: R + + return r + end + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + CHECK(nullptr != get(result.errors[0])); +} + +// Belongs in TypeInfer.builtins.test.cpp. +TEST_CASE_FIXTURE(Fixture, "pcall_returns_at_least_two_value_but_function_returns_nothing") +{ + CheckResult result = check(R"( + local function f(): () end + local ok, res = pcall(f) + )"); + + LUAU_REQUIRE_ERRORS(result); + // LUAU_REQUIRE_NO_ERRORS(result); + // CHECK_EQ("boolean", toString(requireType("ok"))); + // CHECK_EQ("any", toString(requireType("res"))); +} + +// Belongs in TypeInfer.builtins.test.cpp. +TEST_CASE_FIXTURE(Fixture, "choose_the_right_overload_for_pcall") +{ + CheckResult result = check(R"( + local function f(): number + if math.random() > 0.5 then + return 5 + else + error("something") + end + end + + local ok, res = pcall(f) + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + CHECK_EQ("boolean", toString(requireType("ok"))); + CHECK_EQ("number", toString(requireType("res"))); + // CHECK_EQ("any", toString(requireType("res"))); +} + +// Belongs in TypeInfer.builtins.test.cpp. +TEST_CASE_FIXTURE(Fixture, "function_returns_many_things_but_first_of_it_is_forgotten") +{ + CheckResult result = check(R"( + local function f(): (number, string, boolean) + if math.random() > 0.5 then + return 5, "hello", true + else + error("something") + end + end + + local ok, res, s, b = pcall(f) + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + CHECK_EQ("boolean", toString(requireType("ok"))); + CHECK_EQ("number", toString(requireType("res"))); + // CHECK_EQ("any", toString(requireType("res"))); + CHECK_EQ("string", toString(requireType("s"))); + CHECK_EQ("boolean", toString(requireType("b"))); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.refinements.test.cpp b/tests/TypeInfer.refinements.test.cpp index cddeab6e..ce22bcb1 100644 --- a/tests/TypeInfer.refinements.test.cpp +++ b/tests/TypeInfer.refinements.test.cpp @@ -1,4 +1,5 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +#include "Luau/Normalize.h" #include "Luau/Scope.h" #include "Luau/TypeInfer.h" @@ -8,6 +9,7 @@ LUAU_FASTFLAG(LuauDiscriminableUnions2) LUAU_FASTFLAG(LuauWeakEqConstraint) +LUAU_FASTFLAG(LuauLowerBoundsCalculation) using namespace Luau; @@ -48,6 +50,7 @@ struct RefinementClassFixture : Fixture {"Y", Property{typeChecker.numberType}}, {"Z", Property{typeChecker.numberType}}, }; + normalize(vec3, arena, *typeChecker.iceHandler); TypeId inst = arena.addType(ClassTypeVar{"Instance", {}, std::nullopt, std::nullopt, {}, nullptr}); @@ -55,17 +58,21 @@ struct RefinementClassFixture : Fixture TypePackId isARets = arena.addTypePack({typeChecker.booleanType}); TypeId isA = arena.addType(FunctionTypeVar{isAParams, isARets}); getMutable(isA)->magicFunction = magicFunctionInstanceIsA; + normalize(isA, arena, *typeChecker.iceHandler); getMutable(inst)->props = { {"Name", Property{typeChecker.stringType}}, {"IsA", Property{isA}}, }; + normalize(inst, arena, *typeChecker.iceHandler); TypeId folder = typeChecker.globalTypes.addType(ClassTypeVar{"Folder", {}, inst, std::nullopt, {}, nullptr}); + normalize(folder, arena, *typeChecker.iceHandler); TypeId part = typeChecker.globalTypes.addType(ClassTypeVar{"Part", {}, inst, std::nullopt, {}, nullptr}); getMutable(part)->props = { {"Position", Property{vec3}}, }; + normalize(part, arena, *typeChecker.iceHandler); typeChecker.globalScope->exportedTypeBindings["Vector3"] = TypeFun{{}, vec3}; typeChecker.globalScope->exportedTypeBindings["Instance"] = TypeFun{{}, inst}; @@ -697,7 +704,10 @@ TEST_CASE_FIXTURE(Fixture, "type_guard_can_filter_for_intersection_of_tables") LUAU_REQUIRE_NO_ERRORS(result); - CHECK_EQ("{| x: number |} & {| y: number |}", toString(requireTypeAtPosition({4, 28}))); + if (FFlag::LuauLowerBoundsCalculation) + CHECK_EQ("{| x: number, y: number |}", toString(requireTypeAtPosition({4, 28}))); + else + CHECK_EQ("{| x: number |} & {| y: number |}", toString(requireTypeAtPosition({4, 28}))); CHECK_EQ("nil", toString(requireTypeAtPosition({6, 28}))); } diff --git a/tests/TypeInfer.singletons.test.cpp b/tests/TypeInfer.singletons.test.cpp index d39341ea..2b01c29e 100644 --- a/tests/TypeInfer.singletons.test.cpp +++ b/tests/TypeInfer.singletons.test.cpp @@ -5,8 +5,6 @@ #include "doctest.h" #include "Luau/BuiltinDefinitions.h" -LUAU_FASTFLAG(BetterDiagnosticCodesInStudio) - using namespace Luau; TEST_SUITE_BEGIN("TypeSingletons"); @@ -261,14 +259,7 @@ TEST_CASE_FIXTURE(Fixture, "table_properties_alias_or_parens_is_indexer") )"); LUAU_REQUIRE_ERROR_COUNT(1, result); - if (FFlag::BetterDiagnosticCodesInStudio) - { - CHECK_EQ("Cannot have more than one table indexer", toString(result.errors[0])); - } - else - { - CHECK_EQ("Syntax error: Cannot have more than one table indexer", toString(result.errors[0])); - } + CHECK_EQ("Cannot have more than one table indexer", toString(result.errors[0])); } TEST_CASE_FIXTURE(Fixture, "table_properties_type_error_escapes") diff --git a/tests/TypeInfer.tables.test.cpp b/tests/TypeInfer.tables.test.cpp index 0484351d..ca1b8de7 100644 --- a/tests/TypeInfer.tables.test.cpp +++ b/tests/TypeInfer.tables.test.cpp @@ -11,6 +11,8 @@ using namespace Luau; +LUAU_FASTFLAG(LuauLowerBoundsCalculation); + TEST_SUITE_BEGIN("TableTests"); TEST_CASE_FIXTURE(Fixture, "basic") @@ -1211,7 +1213,10 @@ TEST_CASE_FIXTURE(Fixture, "pass_incompatible_union_to_a_generic_table_without_c )"); LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK(get(result.errors[0])); + if (FFlag::LuauLowerBoundsCalculation) + CHECK(get(result.errors[0])); + else + CHECK(get(result.errors[0])); } // This unit test could be flaky if the fix has regressed. @@ -2922,6 +2927,60 @@ TEST_CASE_FIXTURE(Fixture, "inferred_properties_of_a_table_should_start_with_the LUAU_REQUIRE_NO_ERRORS(result); } +// The real bug here was that we weren't always uncondionally typechecking a trailing return statement last. +TEST_CASE_FIXTURE(Fixture, "dont_leak_free_table_props") +{ + CheckResult result = check(R"( + local function a(state) + print(state.blah) + end + + local function b(state) -- The bug was that we inferred state: {blah: any, gwar: any} + print(state.gwar) + end + + return function() + return function(state) + a(state) + b(state) + end + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ("({+ blah: a +}) -> ()", toString(requireType("a"))); + CHECK_EQ("({+ gwar: a +}) -> ()", toString(requireType("b"))); + CHECK_EQ("() -> ({+ blah: a, gwar: b +}) -> ()", toString(getMainModule()->getModuleScope()->returnType)); +} + +TEST_CASE_FIXTURE(Fixture, "inferred_return_type_of_free_table") +{ + ScopedFastFlag sff[] = { + {"LuauLowerBoundsCalculation", true}, + }; + + check(R"( + function Base64FileReader(data) + local reader = {} + local index: number + + function reader:PeekByte() + return data:byte(index) + end + + function reader:Byte() + return data:byte(index - 1) + end + + return reader + end + )"); + + CHECK_EQ("(t1) -> {| Byte: (b) -> (a...), PeekByte: (c) -> (a...) |} where t1 = {+ byte: (t1, number) -> (a...) +}", + toString(requireType("Base64FileReader"))); +} + TEST_CASE_FIXTURE(Fixture, "mixed_tables_with_implicit_numbered_keys") { ScopedFastFlag sff{"LuauCheckImplicitNumbericKeys", true}; diff --git a/tests/TypeInfer.test.cpp b/tests/TypeInfer.test.cpp index 660ddcfc..6abd96b9 100644 --- a/tests/TypeInfer.test.cpp +++ b/tests/TypeInfer.test.cpp @@ -13,6 +13,7 @@ #include +LUAU_FASTFLAG(LuauLowerBoundsCalculation) LUAU_FASTFLAG(LuauFixLocationSpanTableIndexExpr) LUAU_FASTFLAG(LuauEqConstraint) @@ -177,7 +178,6 @@ TEST_CASE_FIXTURE(Fixture, "weird_case") )"); LUAU_REQUIRE_NO_ERRORS(result); - dumpErrors(result); } TEST_CASE_FIXTURE(Fixture, "dont_ice_when_failing_the_occurs_check") @@ -293,7 +293,7 @@ TEST_CASE_FIXTURE(Fixture, "exponential_blowup_from_copying_types") // In these tests, a successful parse is required, so we need the parser to return the AST and then we can test the recursion depth limit in type // checker. We also want it to somewhat match up with production values, so we push up the parser recursion limit a little bit instead. -TEST_CASE_FIXTURE(Fixture, "check_type_infer_recursion_limit") +TEST_CASE_FIXTURE(Fixture, "check_type_infer_recursion_count") { #if defined(LUAU_ENABLE_ASAN) int limit = 250; @@ -302,12 +302,14 @@ TEST_CASE_FIXTURE(Fixture, "check_type_infer_recursion_limit") #else int limit = 600; #endif - ScopedFastInt luauRecursionLimit{"LuauRecursionLimit", limit + 100}; - ScopedFastInt luauTypeInferRecursionLimit{"LuauTypeInferRecursionLimit", limit - 100}; - ScopedFastInt luauCheckRecursionLimit{"LuauCheckRecursionLimit", 0}; - CHECK_NOTHROW(check("print('Hello!')")); - CHECK_THROWS_AS(check("function f() return " + rep("{a=", limit) + "'a'" + rep("}", limit) + " end"), std::runtime_error); + ScopedFastFlag sff{"LuauTableUseCounterInstead", true}; + ScopedFastInt sfi{"LuauCheckRecursionLimit", limit}; + + CheckResult result = check("function f() return " + rep("{a=", limit) + "'a'" + rep("}", limit) + " end"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK(nullptr != get(result.errors[0])); } TEST_CASE_FIXTURE(Fixture, "check_block_recursion_limit") @@ -721,9 +723,9 @@ TEST_CASE_FIXTURE(Fixture, "no_heap_use_after_free_error") local l0 do end while _ do - function _:_() - _ += _(_._(_:n0(xpcall,_))) - end + function _:_() + _ += _(_._(_:n0(xpcall,_))) + end end )"); @@ -978,4 +980,48 @@ TEST_CASE_FIXTURE(Fixture, "cli_50041_committing_txnlog_in_apollo_client_error") LUAU_REQUIRE_NO_ERRORS(result); } +TEST_CASE_FIXTURE(Fixture, "type_infer_recursion_limit_no_ice") +{ + ScopedFastInt sfi("LuauTypeInferRecursionLimit", 2); + ScopedFastFlag sff{"LuauRecursionLimitException", true}; + + CheckResult result = check(R"( + function complex() + function _(l0:t0): (any, ()->()) + return 0,_ + end + type t0 = t0 | {} + _(nil) + end + )"); + + LUAU_REQUIRE_ERRORS(result); + CHECK_EQ("Code is too complex to typecheck! Consider simplifying the code around this area", toString(result.errors[0])); +} + +TEST_CASE_FIXTURE(Fixture, "follow_on_new_types_in_substitution") +{ + ScopedFastFlag substituteFollowNewTypes{"LuauSubstituteFollowNewTypes", true}; + + CheckResult result = check(R"( + local obj = {} + + function obj:Method() + self.fieldA = function(object) + if object.a then + self.arr[object] = true + elseif object.b then + self.fieldB[object] = object:Connect(function(arg) + self.arr[arg] = nil + end) + end + end + end + + return obj + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.typePacks.cpp b/tests/TypeInfer.typePacks.cpp index 130f33d7..f141622f 100644 --- a/tests/TypeInfer.typePacks.cpp +++ b/tests/TypeInfer.typePacks.cpp @@ -9,6 +9,8 @@ using namespace Luau; +LUAU_FASTFLAG(LuauLowerBoundsCalculation); + TEST_SUITE_BEGIN("TypePackTests"); TEST_CASE_FIXTURE(Fixture, "infer_multi_return") @@ -27,8 +29,8 @@ TEST_CASE_FIXTURE(Fixture, "infer_multi_return") const auto& [returns, tail] = flatten(takeTwoType->retType); CHECK_EQ(2, returns.size()); - CHECK_EQ(typeChecker.numberType, returns[0]); - CHECK_EQ(typeChecker.numberType, returns[1]); + CHECK_EQ(typeChecker.numberType, follow(returns[0])); + CHECK_EQ(typeChecker.numberType, follow(returns[1])); CHECK(!tail); } @@ -74,9 +76,9 @@ TEST_CASE_FIXTURE(Fixture, "last_element_of_return_statement_can_itself_be_a_pac const auto& [rets, tail] = flatten(takeOneMoreType->retType); REQUIRE_EQ(3, rets.size()); - CHECK_EQ(typeChecker.numberType, rets[0]); - CHECK_EQ(typeChecker.numberType, rets[1]); - CHECK_EQ(typeChecker.numberType, rets[2]); + CHECK_EQ(typeChecker.numberType, follow(rets[0])); + CHECK_EQ(typeChecker.numberType, follow(rets[1])); + CHECK_EQ(typeChecker.numberType, follow(rets[2])); CHECK(!tail); } @@ -91,26 +93,7 @@ TEST_CASE_FIXTURE(Fixture, "higher_order_function") LUAU_REQUIRE_NO_ERRORS(result); - const FunctionTypeVar* applyType = get(requireType("apply")); - REQUIRE(applyType != nullptr); - - std::vector applyArgs = flatten(applyType->argTypes).first; - REQUIRE_EQ(3, applyArgs.size()); - - const FunctionTypeVar* fType = get(follow(applyArgs[0])); - REQUIRE(fType != nullptr); - - const FunctionTypeVar* gType = get(follow(applyArgs[1])); - REQUIRE(gType != nullptr); - - std::vector gArgs = flatten(gType->argTypes).first; - REQUIRE_EQ(1, gArgs.size()); - - // function(function(t1, T2...): (t3, T4...), function(t5): (t1, T2...), t5): (t3, T4...) - - REQUIRE_EQ(*gArgs[0], *applyArgs[2]); - REQUIRE_EQ(toString(fType->argTypes), toString(gType->retType)); - REQUIRE_EQ(toString(fType->retType), toString(applyType->retType)); + CHECK_EQ("((b...) -> (c...), (a) -> (b...), a) -> (c...)", toString(requireType("apply"))); } TEST_CASE_FIXTURE(Fixture, "return_type_should_be_empty_if_nothing_is_returned") @@ -328,7 +311,10 @@ local c: Packed auto ttvA = get(requireType("a")); REQUIRE(ttvA); CHECK_EQ(toString(requireType("a")), "Packed"); - CHECK_EQ(toString(requireType("a"), {true}), "{| f: (number) -> (number) |}"); + if (FFlag::LuauLowerBoundsCalculation) + CHECK_EQ(toString(requireType("a"), {true}), "{| f: (number) -> number |}"); + else + CHECK_EQ(toString(requireType("a"), {true}), "{| f: (number) -> (number) |}"); REQUIRE(ttvA->instantiatedTypeParams.size() == 1); REQUIRE(ttvA->instantiatedTypePackParams.size() == 1); CHECK_EQ(toString(ttvA->instantiatedTypeParams[0], {true}), "number"); diff --git a/tests/TypeInfer.unionTypes.test.cpp b/tests/TypeInfer.unionTypes.test.cpp index ff207a18..96bdd534 100644 --- a/tests/TypeInfer.unionTypes.test.cpp +++ b/tests/TypeInfer.unionTypes.test.cpp @@ -6,6 +6,7 @@ #include "doctest.h" +LUAU_FASTFLAG(LuauLowerBoundsCalculation) LUAU_FASTFLAG(LuauEqConstraint) using namespace Luau; @@ -254,11 +255,11 @@ local c = bf.a.y TEST_CASE_FIXTURE(Fixture, "optional_union_functions") { CheckResult result = check(R"( -local a = {} -function a.foo(x:number, y:number) return x + y end -type A = typeof(a) -local b: A? = a -local c = b.foo(1, 2) + local a = {} + function a.foo(x:number, y:number) return x + y end + type A = typeof(a) + local b: A? = a + local c = b.foo(1, 2) )"); LUAU_REQUIRE_ERROR_COUNT(1, result); @@ -356,7 +357,10 @@ a.x = 2 )"); LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK_EQ("Value of type '({| x: number |} & {| y: number |})?' could be nil", toString(result.errors[0])); + if (FFlag::LuauLowerBoundsCalculation) + CHECK_EQ("Value of type '{| x: number, y: number |}?' could be nil", toString(result.errors[0])); + else + CHECK_EQ("Value of type '({| x: number |} & {| y: number |})?' could be nil", toString(result.errors[0])); } TEST_CASE_FIXTURE(Fixture, "optional_length_error") @@ -533,8 +537,13 @@ TEST_CASE_FIXTURE(Fixture, "table_union_write_indirect") LUAU_REQUIRE_ERROR_COUNT(1, result); // NOTE: union normalization will improve this message - CHECK_EQ(toString(result.errors[0]), - R"(Type '(string) -> number' could not be converted into '((number) -> string) | ((number) -> string)'; none of the union options are compatible)"); + if (FFlag::LuauLowerBoundsCalculation) + CHECK_EQ(toString(result.errors[0]), "Type '(string) -> number' could not be converted into '(number) -> string'\n" + "caused by:\n" + " Argument #1 type is not compatible. Type 'number' could not be converted into 'string'"); + else + CHECK_EQ(toString(result.errors[0]), + R"(Type '(string) -> number' could not be converted into '((number) -> string) | ((number) -> string)'; none of the union options are compatible)"); } diff --git a/tests/conformance/nextvar.lua b/tests/conformance/nextvar.lua index ab9be42c..c8176456 100644 --- a/tests/conformance/nextvar.lua +++ b/tests/conformance/nextvar.lua @@ -581,4 +581,19 @@ do assert(#arr == 5) end +-- test boundary invariant maintenance when table is filled using SETLIST opcode +do + local arr = {[2]=2,1} + assert(#arr == 2) +end + +-- test boundary invariant maintenance when table is filled using table.move +do + local t1 = {1, 2, 3, 4, 5} + local t2 = {[6] = 6} + + table.move(t1, 1, 5, 1, t2) + assert(#t2 == 6) +end + return"OK" From 25f90eae7d0650aaf1509880d8b26ac3c4cbebdb Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Thu, 14 Apr 2022 16:48:36 -0700 Subject: [PATCH 223/399] Fix test in debug --- tests/TypeInfer.provisional.test.cpp | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/tests/TypeInfer.provisional.test.cpp b/tests/TypeInfer.provisional.test.cpp index 8c8059db..6b3741fa 100644 --- a/tests/TypeInfer.provisional.test.cpp +++ b/tests/TypeInfer.provisional.test.cpp @@ -636,6 +636,10 @@ TEST_CASE_FIXTURE(Fixture, "lower_bounds_calculation_is_too_permissive_with_over // Once fixed, move this to Normalize.test.cpp TEST_CASE_FIXTURE(Fixture, "normalization_fails_on_certain_kinds_of_cyclic_tables") { +#if defined(_DEBUG) || defined(_NOOPT) + ScopedFastInt sfi("LuauNormalizeIterationLimit", 500); +#endif + ScopedFastFlag flags[] = { {"LuauLowerBoundsCalculation", true}, }; From 8e7845076b240cd10ea73680d48cad9525b31c20 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Thu, 14 Apr 2022 16:57:43 -0700 Subject: [PATCH 224/399] Sync to upstream/release/523 (#459) --- Analysis/include/Luau/Clone.h | 9 +- Analysis/include/Luau/Error.h | 10 +- Analysis/include/Luau/Frontend.h | 1 + Analysis/include/Luau/LValue.h | 4 + Analysis/include/Luau/Module.h | 2 +- Analysis/include/Luau/Normalize.h | 19 + Analysis/include/Luau/RecursionCounter.h | 26 +- Analysis/include/Luau/Substitution.h | 1 + Analysis/include/Luau/ToString.h | 3 + Analysis/include/Luau/TxnLog.h | 32 +- Analysis/include/Luau/TypeInfer.h | 24 +- Analysis/include/Luau/TypePack.h | 14 +- Analysis/include/Luau/TypeVar.h | 31 +- Analysis/include/Luau/Unifiable.h | 17 +- Analysis/include/Luau/Unifier.h | 22 +- Analysis/include/Luau/UnifierSharedState.h | 2 + Analysis/include/Luau/VisitTypeVar.h | 9 + Analysis/src/Autocomplete.cpp | 8 +- Analysis/src/Clone.cpp | 116 ++- Analysis/src/Error.cpp | 28 +- Analysis/src/Frontend.cpp | 41 +- Analysis/src/IostreamHelpers.cpp | 2 + Analysis/src/LValue.cpp | 17 + Analysis/src/Linter.cpp | 93 +- Analysis/src/Module.cpp | 39 +- Analysis/src/Normalize.cpp | 814 +++++++++++++++++ Analysis/src/Quantify.cpp | 36 + Analysis/src/Substitution.cpp | 138 ++- Analysis/src/ToDot.cpp | 31 + Analysis/src/ToString.cpp | 165 +++- Analysis/src/TopoSortStatements.cpp | 1 + Analysis/src/TxnLog.cpp | 44 +- Analysis/src/TypeAttach.cpp | 13 + Analysis/src/TypeInfer.cpp | 488 +++++++++-- Analysis/src/TypePack.cpp | 46 +- Analysis/src/TypeVar.cpp | 28 +- Analysis/src/Unifier.cpp | 337 +++++-- Ast/include/Luau/DenseHash.h | 118 ++- Ast/include/Luau/Lexer.h | 2 +- Ast/src/Lexer.cpp | 10 +- Ast/src/Parser.cpp | 5 +- Compiler/src/Compiler.cpp | 4 +- Compiler/src/CostModel.cpp | 258 ++++++ Compiler/src/CostModel.h | 18 + Sources.cmake | 6 + VM/src/ltable.cpp | 47 +- VM/src/ltablib.cpp | 2 +- VM/src/lvmexecute.cpp | 4 +- tests/CostModel.test.cpp | 101 +++ tests/Fixture.cpp | 4 +- tests/JsonEncoder.test.cpp | 4 +- tests/Linter.test.cpp | 3 +- tests/Module.test.cpp | 75 +- tests/NonstrictMode.test.cpp | 34 + tests/Normalize.test.cpp | 967 +++++++++++++++++++++ tests/Parser.test.cpp | 20 + tests/ToDot.test.cpp | 77 +- tests/ToString.test.cpp | 2 + tests/TopoSort.test.cpp | 32 +- tests/Transpiler.test.cpp | 2 +- tests/TypeInfer.annotations.test.cpp | 10 + tests/TypeInfer.builtins.test.cpp | 13 +- tests/TypeInfer.classes.test.cpp | 3 + tests/TypeInfer.functions.test.cpp | 177 +++- tests/TypeInfer.generics.test.cpp | 77 +- tests/TypeInfer.intersectionTypes.test.cpp | 46 +- tests/TypeInfer.oop.test.cpp | 16 +- tests/TypeInfer.operators.test.cpp | 3 +- tests/TypeInfer.provisional.test.cpp | 135 ++- tests/TypeInfer.refinements.test.cpp | 12 +- tests/TypeInfer.singletons.test.cpp | 11 +- tests/TypeInfer.tables.test.cpp | 61 +- tests/TypeInfer.test.cpp | 66 +- tests/TypeInfer.typePacks.cpp | 38 +- tests/TypeInfer.unionTypes.test.cpp | 25 +- tests/conformance/nextvar.lua | 15 + 76 files changed, 4575 insertions(+), 639 deletions(-) create mode 100644 Analysis/include/Luau/Normalize.h create mode 100644 Analysis/src/Normalize.cpp create mode 100644 Compiler/src/CostModel.cpp create mode 100644 Compiler/src/CostModel.h create mode 100644 tests/CostModel.test.cpp create mode 100644 tests/Normalize.test.cpp diff --git a/Analysis/include/Luau/Clone.h b/Analysis/include/Luau/Clone.h index 917ef801..78aa92c7 100644 --- a/Analysis/include/Luau/Clone.h +++ b/Analysis/include/Luau/Clone.h @@ -14,12 +14,15 @@ using SeenTypePacks = std::unordered_map; struct CloneState { + SeenTypes seenTypes; + SeenTypePacks seenTypePacks; + int recursionCount = 0; bool encounteredFreeType = false; }; -TypePackId clone(TypePackId tp, TypeArena& dest, SeenTypes& seenTypes, SeenTypePacks& seenTypePacks, CloneState& cloneState); -TypeId clone(TypeId tp, TypeArena& dest, SeenTypes& seenTypes, SeenTypePacks& seenTypePacks, CloneState& cloneState); -TypeFun clone(const TypeFun& typeFun, TypeArena& dest, SeenTypes& seenTypes, SeenTypePacks& seenTypePacks, CloneState& cloneState); +TypePackId clone(TypePackId tp, TypeArena& dest, CloneState& cloneState); +TypeId clone(TypeId tp, TypeArena& dest, CloneState& cloneState); +TypeFun clone(const TypeFun& typeFun, TypeArena& dest, CloneState& cloneState); } // namespace Luau diff --git a/Analysis/include/Luau/Error.h b/Analysis/include/Luau/Error.h index 53b946a0..70683141 100644 --- a/Analysis/include/Luau/Error.h +++ b/Analysis/include/Luau/Error.h @@ -287,12 +287,20 @@ struct TypesAreUnrelated bool operator==(const TypesAreUnrelated& rhs) const; }; +struct NormalizationTooComplex +{ + bool operator==(const NormalizationTooComplex&) const + { + return true; + } +}; + using TypeErrorData = Variant; + MissingProperties, SwappedGenericTypeParameter, OptionalValueAccess, MissingUnionProperty, TypesAreUnrelated, NormalizationTooComplex>; struct TypeError { diff --git a/Analysis/include/Luau/Frontend.h b/Analysis/include/Luau/Frontend.h index 2266f548..e24e433c 100644 --- a/Analysis/include/Luau/Frontend.h +++ b/Analysis/include/Luau/Frontend.h @@ -70,6 +70,7 @@ struct SourceNode std::vector> requireLocations; bool dirty = true; bool dirtyAutocomplete = true; + double autocompleteLimitsMult = 1.0; }; struct FrontendOptions diff --git a/Analysis/include/Luau/LValue.h b/Analysis/include/Luau/LValue.h index 3d510d5f..afb71415 100644 --- a/Analysis/include/Luau/LValue.h +++ b/Analysis/include/Luau/LValue.h @@ -35,8 +35,12 @@ const LValue* baseof(const LValue& lvalue); std::optional tryGetLValue(const class AstExpr& expr); // Utility function: breaks down an LValue to get at the Symbol, and reverses the vector of keys. +// TODO: remove with FFlagLuauTypecheckOptPass std::pair> getFullName(const LValue& lvalue); +// Utility function: breaks down an LValue to get at the Symbol +Symbol getBaseSymbol(const LValue& lvalue); + template const T* get(const LValue& lvalue) { diff --git a/Analysis/include/Luau/Module.h b/Analysis/include/Luau/Module.h index 9a32f614..0dd44188 100644 --- a/Analysis/include/Luau/Module.h +++ b/Analysis/include/Luau/Module.h @@ -113,7 +113,7 @@ struct Module // This helps us to force TypeVar ownership into a DAG rather than a DCG. // Returns true if there were any free types encountered in the public interface. This // indicates a bug in the type checker that we want to surface. - bool clonePublicInterface(); + bool clonePublicInterface(InternalErrorReporter& ice); }; } // namespace Luau diff --git a/Analysis/include/Luau/Normalize.h b/Analysis/include/Luau/Normalize.h new file mode 100644 index 00000000..262b54b2 --- /dev/null +++ b/Analysis/include/Luau/Normalize.h @@ -0,0 +1,19 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details + +#include "Luau/Substitution.h" +#include "Luau/TypeVar.h" +#include "Luau/Module.h" + +namespace Luau +{ + +struct InternalErrorReporter; + +bool isSubtype(TypeId superTy, TypeId subTy, InternalErrorReporter& ice); + +std::pair normalize(TypeId ty, TypeArena& arena, InternalErrorReporter& ice); +std::pair normalize(TypeId ty, const ModulePtr& module, InternalErrorReporter& ice); +std::pair normalize(TypePackId ty, TypeArena& arena, InternalErrorReporter& ice); +std::pair normalize(TypePackId ty, const ModulePtr& module, InternalErrorReporter& ice); + +} // namespace Luau diff --git a/Analysis/include/Luau/RecursionCounter.h b/Analysis/include/Luau/RecursionCounter.h index 89632cea..03ae2c83 100644 --- a/Analysis/include/Luau/RecursionCounter.h +++ b/Analysis/include/Luau/RecursionCounter.h @@ -4,10 +4,21 @@ #include "Luau/Common.h" #include +#include + +LUAU_FASTFLAG(LuauRecursionLimitException); namespace Luau { +struct RecursionLimitException : public std::exception +{ + const char* what() const noexcept + { + return "Internal recursion counter limit exceeded"; + } +}; + struct RecursionCounter { RecursionCounter(int* count) @@ -28,11 +39,22 @@ private: struct RecursionLimiter : RecursionCounter { - RecursionLimiter(int* count, int limit) + // TODO: remove ctx after LuauRecursionLimitException is removed + RecursionLimiter(int* count, int limit, const char* ctx) : RecursionCounter(count) { + LUAU_ASSERT(ctx); if (limit > 0 && *count > limit) - throw std::runtime_error("Internal recursion counter limit exceeded"); + { + if (FFlag::LuauRecursionLimitException) + throw RecursionLimitException(); + else + { + std::string m = "Internal recursion counter limit exceeded: "; + m += ctx; + throw std::runtime_error(m); + } + } } }; diff --git a/Analysis/include/Luau/Substitution.h b/Analysis/include/Luau/Substitution.h index 9662d5b3..6f5931e1 100644 --- a/Analysis/include/Luau/Substitution.h +++ b/Analysis/include/Luau/Substitution.h @@ -90,6 +90,7 @@ struct Tarjan std::vector lowlink; int childCount = 0; + int childLimit = 0; // This should never be null; ensure you initialize it before calling // substitution methods. diff --git a/Analysis/include/Luau/ToString.h b/Analysis/include/Luau/ToString.h index 49ee82fe..f4db5e35 100644 --- a/Analysis/include/Luau/ToString.h +++ b/Analysis/include/Luau/ToString.h @@ -28,6 +28,7 @@ struct ToStringOptions bool functionTypeArguments = false; // If true, output function type argument names when they are available bool hideTableKind = false; // If true, all tables will be surrounded with plain '{}' bool hideNamedFunctionTypeParameters = false; // If true, type parameters of functions will be hidden at top-level. + bool indent = false; size_t maxTableLength = size_t(FInt::LuauTableTypeMaximumStringifierLength); // Only applied to TableTypeVars size_t maxTypeLength = size_t(FInt::LuauTypeMaximumStringifierLength); std::optional nameMap; @@ -73,6 +74,8 @@ std::string toStringNamedFunction(const std::string& funcName, const FunctionTyp std::string dump(TypeId ty); std::string dump(TypePackId ty); +std::string dump(const std::shared_ptr& scope, const char* name); + std::string generateName(size_t n); } // namespace Luau diff --git a/Analysis/include/Luau/TxnLog.h b/Analysis/include/Luau/TxnLog.h index c8ebaaeb..995ed6c6 100644 --- a/Analysis/include/Luau/TxnLog.h +++ b/Analysis/include/Luau/TxnLog.h @@ -7,7 +7,7 @@ #include "Luau/TypeVar.h" #include "Luau/TypePack.h" -LUAU_FASTFLAG(LuauShareTxnSeen); +LUAU_FASTFLAG(LuauTypecheckOptPass) namespace Luau { @@ -64,13 +64,17 @@ T* getMutable(PendingTypePack* pending) struct TxnLog { TxnLog() - : ownedSeen() + : typeVarChanges(nullptr) + , typePackChanges(nullptr) + , ownedSeen() , sharedSeen(&ownedSeen) { } explicit TxnLog(TxnLog* parent) - : parent(parent) + : typeVarChanges(nullptr) + , typePackChanges(nullptr) + , parent(parent) { if (parent) { @@ -83,14 +87,19 @@ struct TxnLog } explicit TxnLog(std::vector>* sharedSeen) - : sharedSeen(sharedSeen) + : typeVarChanges(nullptr) + , typePackChanges(nullptr) + , sharedSeen(sharedSeen) { } TxnLog(TxnLog* parent, std::vector>* sharedSeen) - : parent(parent) + : typeVarChanges(nullptr) + , typePackChanges(nullptr) + , parent(parent) , sharedSeen(sharedSeen) { + LUAU_ASSERT(!FFlag::LuauTypecheckOptPass); } TxnLog(const TxnLog&) = delete; @@ -243,6 +252,12 @@ struct TxnLog return Luau::getMutable(ty); } + template + const T* get(TID ty) const + { + return this->getMutable(ty); + } + // Returns whether a given type or type pack is a given state, respecting the // log's pending state. // @@ -263,11 +278,8 @@ private: // unique_ptr is used to give us stable pointers across insertions into the // map. Otherwise, it would be really easy to accidentally invalidate the // pointers returned from queue/pending. - // - // We can't use a DenseHashMap here because we need a non-const iterator - // over the map when we concatenate. - std::unordered_map, DenseHashPointer> typeVarChanges; - std::unordered_map, DenseHashPointer> typePackChanges; + DenseHashMap> typeVarChanges; + DenseHashMap> typePackChanges; TxnLog* parent = nullptr; diff --git a/Analysis/include/Luau/TypeInfer.h b/Analysis/include/Luau/TypeInfer.h index 215da67f..ac880135 100644 --- a/Analysis/include/Luau/TypeInfer.h +++ b/Analysis/include/Luau/TypeInfer.h @@ -76,19 +76,32 @@ struct Instantiation : Substitution // A substitution which replaces free types by any struct Anyification : Substitution { - Anyification(TypeArena* arena, TypeId anyType, TypePackId anyTypePack) + Anyification(TypeArena* arena, InternalErrorReporter* iceHandler, TypeId anyType, TypePackId anyTypePack) : Substitution(TxnLog::empty(), arena) + , iceHandler(iceHandler) , anyType(anyType) , anyTypePack(anyTypePack) { } + InternalErrorReporter* iceHandler; + TypeId anyType; TypePackId anyTypePack; + bool normalizationTooComplex = false; bool isDirty(TypeId ty) override; bool isDirty(TypePackId tp) override; TypeId clean(TypeId ty) override; TypePackId clean(TypePackId tp) override; + + bool ignoreChildren(TypeId ty) override + { + return ty->persistent; + } + bool ignoreChildren(TypePackId ty) override + { + return ty->persistent; + } }; // A substitution which replaces the type parameters of a type function by arguments @@ -139,6 +152,7 @@ struct TypeChecker TypeChecker& operator=(const TypeChecker&) = delete; ModulePtr check(const SourceModule& module, Mode mode, std::optional environmentScope = std::nullopt); + ModulePtr checkWithoutRecursionCheck(const SourceModule& module, Mode mode, std::optional environmentScope = std::nullopt); std::vector> getScopes() const; @@ -160,6 +174,7 @@ struct TypeChecker void check(const ScopePtr& scope, const AstStatDeclareFunction& declaredFunction); void checkBlock(const ScopePtr& scope, const AstStatBlock& statement); + void checkBlockWithoutRecursionCheck(const ScopePtr& scope, const AstStatBlock& statement); void checkBlockTypeAliases(const ScopePtr& scope, std::vector& sorted); ExprResult checkExpr( @@ -172,6 +187,7 @@ struct TypeChecker ExprResult checkExpr(const ScopePtr& scope, const AstExprIndexExpr& expr); ExprResult checkExpr(const ScopePtr& scope, const AstExprFunction& expr, std::optional expectedType = std::nullopt); ExprResult checkExpr(const ScopePtr& scope, const AstExprTable& expr, std::optional expectedType = std::nullopt); + ExprResult checkExpr_(const ScopePtr& scope, const AstExprTable& expr, std::optional expectedType = std::nullopt); ExprResult checkExpr(const ScopePtr& scope, const AstExprUnary& expr); TypeId checkRelationalOperation( const ScopePtr& scope, const AstExprBinary& expr, TypeId lhsType, TypeId rhsType, const PredicateVec& predicates = {}); @@ -258,6 +274,8 @@ struct TypeChecker ErrorVec canUnify(TypeId subTy, TypeId superTy, const Location& location); ErrorVec canUnify(TypePackId subTy, TypePackId superTy, const Location& location); + void unifyLowerBound(TypePackId subTy, TypePackId superTy, const Location& location); + std::optional findMetatableEntry(TypeId type, std::string entry, const Location& location); std::optional findTablePropertyRespectingMeta(TypeId lhsType, Name name, const Location& location); @@ -395,6 +413,7 @@ private: void resolve(const EqPredicate& eqP, ErrorVec& errVec, RefinementMap& refis, const ScopePtr& scope, bool sense); bool isNonstrictMode() const; + bool useConstrainedIntersections() const; public: /** Extract the types in a type pack, given the assumption that the pack must have some exact length. @@ -421,7 +440,10 @@ public: std::vector requireCycles; + // Type inference limits std::optional finishTime; + std::optional instantiationChildLimit; + std::optional unifierIterationLimit; public: const TypeId nilType; diff --git a/Analysis/include/Luau/TypePack.h b/Analysis/include/Luau/TypePack.h index 85fa467f..bbc65f94 100644 --- a/Analysis/include/Luau/TypePack.h +++ b/Analysis/include/Luau/TypePack.h @@ -40,6 +40,7 @@ struct TypePack struct VariadicTypePack { TypeId ty; + bool hidden = false; // if true, we don't display this when toString()ing a pack with this variadic as its tail. }; struct TypePackVar @@ -109,10 +110,10 @@ private: }; TypePackIterator begin(TypePackId tp); -TypePackIterator begin(TypePackId tp, TxnLog* log); +TypePackIterator begin(TypePackId tp, const TxnLog* log); TypePackIterator end(TypePackId tp); -using SeenSet = std::set>; +using SeenSet = std::set>; bool areEqual(SeenSet& seen, const TypePackVar& lhs, const TypePackVar& rhs); @@ -122,7 +123,7 @@ TypePackId follow(TypePackId tp, std::function mapper); size_t size(TypePackId tp, TxnLog* log = nullptr); bool finite(TypePackId tp, TxnLog* log = nullptr); size_t size(const TypePack& tp, TxnLog* log = nullptr); -std::optional first(TypePackId tp); +std::optional first(TypePackId tp, bool ignoreHiddenVariadics = true); TypePackVar* asMutable(TypePackId tp); TypePack* asMutable(const TypePack* tp); @@ -154,5 +155,12 @@ bool isEmpty(TypePackId tp); /// Flattens out a type pack. Also returns a valid TypePackId tail if the type pack's full size is not known std::pair, std::optional> flatten(TypePackId tp); +std::pair, std::optional> flatten(TypePackId tp, const TxnLog& log); + +/// Returs true if the type pack arose from a function that is declared to be variadic. +/// Returns *false* for function argument packs that are inferred to be safe to oversaturate! +bool isVariadic(TypePackId tp); +bool isVariadic(TypePackId tp, const TxnLog& log); + } // namespace Luau diff --git a/Analysis/include/Luau/TypeVar.h b/Analysis/include/Luau/TypeVar.h index f61e4044..ae7d1377 100644 --- a/Analysis/include/Luau/TypeVar.h +++ b/Analysis/include/Luau/TypeVar.h @@ -109,6 +109,23 @@ struct PrimitiveTypeVar } }; +struct ConstrainedTypeVar +{ + explicit ConstrainedTypeVar(TypeLevel level) + : level(level) + { + } + + explicit ConstrainedTypeVar(TypeLevel level, const std::vector& parts) + : parts(parts) + , level(level) + { + } + + std::vector parts; + TypeLevel level; +}; + // Singleton types https://github.com/Roblox/luau/blob/master/rfcs/syntax-singleton-types.md // Types for true and false struct BooleanSingleton @@ -248,6 +265,7 @@ struct FunctionTypeVar MagicFunction magicFunction = nullptr; // Function pointer, can be nullptr. bool hasSelf; Tags tags; + bool hasNoGenerics = false; }; enum class TableState @@ -418,8 +436,8 @@ struct LazyTypeVar using ErrorTypeVar = Unifiable::Error; -using TypeVariant = Unifiable::Variant; +using TypeVariant = Unifiable::Variant; struct TypeVar final { @@ -436,6 +454,7 @@ struct TypeVar final TypeVar(const TypeVariant& ty, bool persistent) : ty(ty) , persistent(persistent) + , normal(persistent) // We assume that all persistent types are irreducable. { } @@ -446,6 +465,10 @@ struct TypeVar final // Persistent TypeVars do not get cloned. bool persistent = false; + // Normalization sets this for types that are fully normalized. + // This implies that they are transitively immutable. + bool normal = false; + std::optional documentationSymbol; // Pointer to the type arena that allocated this type. @@ -458,7 +481,7 @@ struct TypeVar final TypeVar& operator=(TypeVariant&& rhs); }; -using SeenSet = std::set>; +using SeenSet = std::set>; bool areEqual(SeenSet& seen, const TypeVar& lhs, const TypeVar& rhs); // Follow BoundTypeVars until we get to something real @@ -545,6 +568,8 @@ void persist(TypePackId tp); const TypeLevel* getLevel(TypeId ty); TypeLevel* getMutableLevel(TypeId ty); +std::optional getLevel(TypePackId tp); + const Property* lookupClassProp(const ClassTypeVar* cls, const Name& name); bool isSubclass(const ClassTypeVar* cls, const ClassTypeVar* parent); diff --git a/Analysis/include/Luau/Unifiable.h b/Analysis/include/Luau/Unifiable.h index e8eafe68..64fa131d 100644 --- a/Analysis/include/Luau/Unifiable.h +++ b/Analysis/include/Luau/Unifiable.h @@ -56,6 +56,14 @@ struct TypeLevel } }; +inline TypeLevel max(const TypeLevel& a, const TypeLevel& b) +{ + if (a.subsumes(b)) + return b; + else + return a; +} + inline TypeLevel min(const TypeLevel& a, const TypeLevel& b) { if (a.subsumes(b)) @@ -64,7 +72,9 @@ inline TypeLevel min(const TypeLevel& a, const TypeLevel& b) return b; } -namespace Unifiable +} // namespace Luau + +namespace Luau::Unifiable { using Name = std::string; @@ -125,7 +135,6 @@ private: }; template -using Variant = Variant, Generic, Error, Value...>; +using Variant = Luau::Variant, Generic, Error, Value...>; -} // namespace Unifiable -} // namespace Luau +} // namespace Luau::Unifiable diff --git a/Analysis/include/Luau/Unifier.h b/Analysis/include/Luau/Unifier.h index 474af50c..340feb7f 100644 --- a/Analysis/include/Luau/Unifier.h +++ b/Analysis/include/Luau/Unifier.h @@ -49,14 +49,14 @@ struct Unifier ErrorVec errors; Location location; Variance variance = Covariant; + bool anyIsTop = false; // If true, we consider any to be a top type. If false, it is a familiar but weird mix of top and bottom all at once. CountMismatch::Context ctx = CountMismatch::Arg; UnifierSharedState& sharedState; - Unifier(TypeArena* types, Mode mode, const Location& location, Variance variance, UnifierSharedState& sharedState, - TxnLog* parentLog = nullptr); - Unifier(TypeArena* types, Mode mode, std::vector>* sharedSeen, const Location& location, - Variance variance, UnifierSharedState& sharedState, TxnLog* parentLog = nullptr); + Unifier(TypeArena* types, Mode mode, const Location& location, Variance variance, UnifierSharedState& sharedState, TxnLog* parentLog = nullptr); + Unifier(TypeArena* types, Mode mode, std::vector>* sharedSeen, const Location& location, Variance variance, + UnifierSharedState& sharedState, TxnLog* parentLog = nullptr); // Test whether the two type vars unify. Never commits the result. ErrorVec canUnify(TypeId subTy, TypeId superTy); @@ -106,7 +106,12 @@ private: std::optional findTablePropertyRespectingMeta(TypeId lhsType, Name name); + void tryUnifyWithConstrainedSubTypeVar(TypeId subTy, TypeId superTy); + void tryUnifyWithConstrainedSuperTypeVar(TypeId subTy, TypeId superTy); + public: + void unifyLowerBound(TypePackId subTy, TypePackId superTy); + // Report an "infinite type error" if the type "needle" already occurs within "haystack" void occursCheck(TypeId needle, TypeId haystack); void occursCheck(DenseHashSet& seen, TypeId needle, TypeId haystack); @@ -115,12 +120,7 @@ public: Unifier makeChildUnifier(); - // A utility function that appends the given error to the unifier's error log. - // This allows setting a breakpoint wherever the unifier reports an error. - void reportError(TypeError error) - { - errors.push_back(error); - } + void reportError(TypeError err); private: bool isNonstrictMode() const; @@ -135,4 +135,6 @@ private: std::optional firstPackErrorPos; }; +void promoteTypeLevels(TxnLog& log, const TypeArena* arena, TypeLevel minLevel, TypePackId tp); + } // namespace Luau diff --git a/Analysis/include/Luau/UnifierSharedState.h b/Analysis/include/Luau/UnifierSharedState.h index 9a3ba56d..1a0b8b76 100644 --- a/Analysis/include/Luau/UnifierSharedState.h +++ b/Analysis/include/Luau/UnifierSharedState.h @@ -28,7 +28,9 @@ struct TypeIdPairHash struct UnifierCounters { int recursionCount = 0; + int recursionLimit = 0; int iterationCount = 0; + int iterationLimit = 0; }; struct UnifierSharedState diff --git a/Analysis/include/Luau/VisitTypeVar.h b/Analysis/include/Luau/VisitTypeVar.h index 740854b3..d11cbd0d 100644 --- a/Analysis/include/Luau/VisitTypeVar.h +++ b/Analysis/include/Luau/VisitTypeVar.h @@ -82,6 +82,15 @@ void visit(TypeId ty, F& f, Set& seen) else if (auto etv = get(ty)) apply(ty, *etv, seen, f); + else if (auto ctv = get(ty)) + { + if (apply(ty, *ctv, seen, f)) + { + for (TypeId part : ctv->parts) + visit(part, f, seen); + } + } + else if (auto ptv = get(ty)) apply(ty, *ptv, seen, f); diff --git a/Analysis/src/Autocomplete.cpp b/Analysis/src/Autocomplete.cpp index b7201ab3..e0e79cb4 100644 --- a/Analysis/src/Autocomplete.cpp +++ b/Analysis/src/Autocomplete.cpp @@ -151,8 +151,12 @@ static ParenthesesRecommendation getParenRecommendationForFunc(const FunctionTyp auto idxExpr = nodes.back()->as(); bool hasImplicitSelf = idxExpr && idxExpr->op == ':'; - auto args = Luau::flatten(func->argTypes); - bool noArgFunction = (args.first.empty() || (hasImplicitSelf && args.first.size() == 1)) && !args.second.has_value(); + auto [argTypes, argVariadicPack] = Luau::flatten(func->argTypes); + + if (argVariadicPack.has_value() && isVariadic(*argVariadicPack)) + return ParenthesesRecommendation::CursorInside; + + bool noArgFunction = argTypes.empty() || (hasImplicitSelf && argTypes.size() == 1); return noArgFunction ? ParenthesesRecommendation::CursorAfter : ParenthesesRecommendation::CursorInside; } diff --git a/Analysis/src/Clone.cpp b/Analysis/src/Clone.cpp index ac9705a7..8e7f7c07 100644 --- a/Analysis/src/Clone.cpp +++ b/Analysis/src/Clone.cpp @@ -6,7 +6,10 @@ #include "Luau/TypePack.h" #include "Luau/Unifiable.h" +LUAU_FASTFLAG(DebugLuauCopyBeforeNormalizing) + LUAU_FASTINTVARIABLE(LuauTypeCloneRecursionLimit, 300) +LUAU_FASTFLAG(LuauTypecheckOptPass) namespace Luau { @@ -23,11 +26,11 @@ struct TypePackCloner; struct TypeCloner { - TypeCloner(TypeArena& dest, TypeId typeId, SeenTypes& seenTypes, SeenTypePacks& seenTypePacks, CloneState& cloneState) + TypeCloner(TypeArena& dest, TypeId typeId, CloneState& cloneState) : dest(dest) , typeId(typeId) - , seenTypes(seenTypes) - , seenTypePacks(seenTypePacks) + , seenTypes(cloneState.seenTypes) + , seenTypePacks(cloneState.seenTypePacks) , cloneState(cloneState) { } @@ -46,6 +49,7 @@ struct TypeCloner void operator()(const Unifiable::Bound& t); void operator()(const Unifiable::Error& t); void operator()(const PrimitiveTypeVar& t); + void operator()(const ConstrainedTypeVar& t); void operator()(const SingletonTypeVar& t); void operator()(const FunctionTypeVar& t); void operator()(const TableTypeVar& t); @@ -65,11 +69,11 @@ struct TypePackCloner SeenTypePacks& seenTypePacks; CloneState& cloneState; - TypePackCloner(TypeArena& dest, TypePackId typePackId, SeenTypes& seenTypes, SeenTypePacks& seenTypePacks, CloneState& cloneState) + TypePackCloner(TypeArena& dest, TypePackId typePackId, CloneState& cloneState) : dest(dest) , typePackId(typePackId) - , seenTypes(seenTypes) - , seenTypePacks(seenTypePacks) + , seenTypes(cloneState.seenTypes) + , seenTypePacks(cloneState.seenTypePacks) , cloneState(cloneState) { } @@ -103,13 +107,15 @@ struct TypePackCloner // We just need to be sure that we rewrite pointers both to the binder and the bindee to the same pointer. void operator()(const Unifiable::Bound& t) { - TypePackId cloned = clone(t.boundTo, dest, seenTypes, seenTypePacks, cloneState); + TypePackId cloned = clone(t.boundTo, dest, cloneState); + if (FFlag::DebugLuauCopyBeforeNormalizing) + cloned = dest.addTypePack(TypePackVar{BoundTypePack{cloned}}); seenTypePacks[typePackId] = cloned; } void operator()(const VariadicTypePack& t) { - TypePackId cloned = dest.addTypePack(TypePackVar{VariadicTypePack{clone(t.ty, dest, seenTypes, seenTypePacks, cloneState)}}); + TypePackId cloned = dest.addTypePack(TypePackVar{VariadicTypePack{clone(t.ty, dest, cloneState), /*hidden*/ t.hidden}}); seenTypePacks[typePackId] = cloned; } @@ -121,10 +127,10 @@ struct TypePackCloner seenTypePacks[typePackId] = cloned; for (TypeId ty : t.head) - destTp->head.push_back(clone(ty, dest, seenTypes, seenTypePacks, cloneState)); + destTp->head.push_back(clone(ty, dest, cloneState)); if (t.tail) - destTp->tail = clone(*t.tail, dest, seenTypes, seenTypePacks, cloneState); + destTp->tail = clone(*t.tail, dest, cloneState); } }; @@ -150,7 +156,9 @@ void TypeCloner::operator()(const Unifiable::Generic& t) void TypeCloner::operator()(const Unifiable::Bound& t) { - TypeId boundTo = clone(t.boundTo, dest, seenTypes, seenTypePacks, cloneState); + TypeId boundTo = clone(t.boundTo, dest, cloneState); + if (FFlag::DebugLuauCopyBeforeNormalizing) + boundTo = dest.addType(BoundTypeVar{boundTo}); seenTypes[typeId] = boundTo; } @@ -164,6 +172,23 @@ void TypeCloner::operator()(const PrimitiveTypeVar& t) defaultClone(t); } +void TypeCloner::operator()(const ConstrainedTypeVar& t) +{ + cloneState.encounteredFreeType = true; + + TypeId res = dest.addType(ConstrainedTypeVar{t.level}); + ConstrainedTypeVar* ctv = getMutable(res); + LUAU_ASSERT(ctv); + + seenTypes[typeId] = res; + + std::vector parts; + for (TypeId part : t.parts) + parts.push_back(clone(part, dest, cloneState)); + + ctv->parts = std::move(parts); +} + void TypeCloner::operator()(const SingletonTypeVar& t) { defaultClone(t); @@ -178,23 +203,26 @@ void TypeCloner::operator()(const FunctionTypeVar& t) seenTypes[typeId] = result; for (TypeId generic : t.generics) - ftv->generics.push_back(clone(generic, dest, seenTypes, seenTypePacks, cloneState)); + ftv->generics.push_back(clone(generic, dest, cloneState)); for (TypePackId genericPack : t.genericPacks) - ftv->genericPacks.push_back(clone(genericPack, dest, seenTypes, seenTypePacks, cloneState)); + ftv->genericPacks.push_back(clone(genericPack, dest, cloneState)); ftv->tags = t.tags; - ftv->argTypes = clone(t.argTypes, dest, seenTypes, seenTypePacks, cloneState); + ftv->argTypes = clone(t.argTypes, dest, cloneState); ftv->argNames = t.argNames; - ftv->retType = clone(t.retType, dest, seenTypes, seenTypePacks, cloneState); + ftv->retType = clone(t.retType, dest, cloneState); + + if (FFlag::LuauTypecheckOptPass) + ftv->hasNoGenerics = t.hasNoGenerics; } void TypeCloner::operator()(const TableTypeVar& t) { // If table is now bound to another one, we ignore the content of the original - if (t.boundTo) + if (!FFlag::DebugLuauCopyBeforeNormalizing && t.boundTo) { - TypeId boundTo = clone(*t.boundTo, dest, seenTypes, seenTypePacks, cloneState); + TypeId boundTo = clone(*t.boundTo, dest, cloneState); seenTypes[typeId] = boundTo; return; } @@ -209,18 +237,20 @@ void TypeCloner::operator()(const TableTypeVar& t) ttv->level = TypeLevel{0, 0}; + if (FFlag::DebugLuauCopyBeforeNormalizing && t.boundTo) + ttv->boundTo = clone(*t.boundTo, dest, cloneState); + for (const auto& [name, prop] : t.props) - ttv->props[name] = {clone(prop.type, dest, seenTypes, seenTypePacks, cloneState), prop.deprecated, {}, prop.location, prop.tags}; + ttv->props[name] = {clone(prop.type, dest, cloneState), prop.deprecated, {}, prop.location, prop.tags}; if (t.indexer) - ttv->indexer = TableIndexer{clone(t.indexer->indexType, dest, seenTypes, seenTypePacks, cloneState), - clone(t.indexer->indexResultType, dest, seenTypes, seenTypePacks, cloneState)}; + ttv->indexer = TableIndexer{clone(t.indexer->indexType, dest, cloneState), clone(t.indexer->indexResultType, dest, cloneState)}; for (TypeId& arg : ttv->instantiatedTypeParams) - arg = clone(arg, dest, seenTypes, seenTypePacks, cloneState); + arg = clone(arg, dest, cloneState); for (TypePackId& arg : ttv->instantiatedTypePackParams) - arg = clone(arg, dest, seenTypes, seenTypePacks, cloneState); + arg = clone(arg, dest, cloneState); if (ttv->state == TableState::Free) { @@ -240,8 +270,8 @@ void TypeCloner::operator()(const MetatableTypeVar& t) MetatableTypeVar* mtv = getMutable(result); seenTypes[typeId] = result; - mtv->table = clone(t.table, dest, seenTypes, seenTypePacks, cloneState); - mtv->metatable = clone(t.metatable, dest, seenTypes, seenTypePacks, cloneState); + mtv->table = clone(t.table, dest, cloneState); + mtv->metatable = clone(t.metatable, dest, cloneState); } void TypeCloner::operator()(const ClassTypeVar& t) @@ -252,13 +282,13 @@ void TypeCloner::operator()(const ClassTypeVar& t) seenTypes[typeId] = result; for (const auto& [name, prop] : t.props) - ctv->props[name] = {clone(prop.type, dest, seenTypes, seenTypePacks, cloneState), prop.deprecated, {}, prop.location, prop.tags}; + ctv->props[name] = {clone(prop.type, dest, cloneState), prop.deprecated, {}, prop.location, prop.tags}; if (t.parent) - ctv->parent = clone(*t.parent, dest, seenTypes, seenTypePacks, cloneState); + ctv->parent = clone(*t.parent, dest, cloneState); if (t.metatable) - ctv->metatable = clone(*t.metatable, dest, seenTypes, seenTypePacks, cloneState); + ctv->metatable = clone(*t.metatable, dest, cloneState); } void TypeCloner::operator()(const AnyTypeVar& t) @@ -272,7 +302,7 @@ void TypeCloner::operator()(const UnionTypeVar& t) options.reserve(t.options.size()); for (TypeId ty : t.options) - options.push_back(clone(ty, dest, seenTypes, seenTypePacks, cloneState)); + options.push_back(clone(ty, dest, cloneState)); TypeId result = dest.addType(UnionTypeVar{std::move(options)}); seenTypes[typeId] = result; @@ -287,7 +317,7 @@ void TypeCloner::operator()(const IntersectionTypeVar& t) LUAU_ASSERT(option != nullptr); for (TypeId ty : t.parts) - option->parts.push_back(clone(ty, dest, seenTypes, seenTypePacks, cloneState)); + option->parts.push_back(clone(ty, dest, cloneState)); } void TypeCloner::operator()(const LazyTypeVar& t) @@ -297,36 +327,36 @@ void TypeCloner::operator()(const LazyTypeVar& t) } // anonymous namespace -TypePackId clone(TypePackId tp, TypeArena& dest, SeenTypes& seenTypes, SeenTypePacks& seenTypePacks, CloneState& cloneState) +TypePackId clone(TypePackId tp, TypeArena& dest, CloneState& cloneState) { if (tp->persistent) return tp; - RecursionLimiter _ra(&cloneState.recursionCount, FInt::LuauTypeCloneRecursionLimit); + RecursionLimiter _ra(&cloneState.recursionCount, FInt::LuauTypeCloneRecursionLimit, "cloning TypePackId"); - TypePackId& res = seenTypePacks[tp]; + TypePackId& res = cloneState.seenTypePacks[tp]; if (res == nullptr) { - TypePackCloner cloner{dest, tp, seenTypes, seenTypePacks, cloneState}; + TypePackCloner cloner{dest, tp, cloneState}; Luau::visit(cloner, tp->ty); // Mutates the storage that 'res' points into. } return res; } -TypeId clone(TypeId typeId, TypeArena& dest, SeenTypes& seenTypes, SeenTypePacks& seenTypePacks, CloneState& cloneState) +TypeId clone(TypeId typeId, TypeArena& dest, CloneState& cloneState) { if (typeId->persistent) return typeId; - RecursionLimiter _ra(&cloneState.recursionCount, FInt::LuauTypeCloneRecursionLimit); + RecursionLimiter _ra(&cloneState.recursionCount, FInt::LuauTypeCloneRecursionLimit, "cloning TypeId"); - TypeId& res = seenTypes[typeId]; + TypeId& res = cloneState.seenTypes[typeId]; if (res == nullptr) { - TypeCloner cloner{dest, typeId, seenTypes, seenTypePacks, cloneState}; + TypeCloner cloner{dest, typeId, cloneState}; Luau::visit(cloner, typeId->ty); // Mutates the storage that 'res' points into. // Persistent types are not being cloned and we get the original type back which might be read-only @@ -337,33 +367,33 @@ TypeId clone(TypeId typeId, TypeArena& dest, SeenTypes& seenTypes, SeenTypePacks return res; } -TypeFun clone(const TypeFun& typeFun, TypeArena& dest, SeenTypes& seenTypes, SeenTypePacks& seenTypePacks, CloneState& cloneState) +TypeFun clone(const TypeFun& typeFun, TypeArena& dest, CloneState& cloneState) { TypeFun result; for (auto param : typeFun.typeParams) { - TypeId ty = clone(param.ty, dest, seenTypes, seenTypePacks, cloneState); + TypeId ty = clone(param.ty, dest, cloneState); std::optional defaultValue; if (param.defaultValue) - defaultValue = clone(*param.defaultValue, dest, seenTypes, seenTypePacks, cloneState); + defaultValue = clone(*param.defaultValue, dest, cloneState); result.typeParams.push_back({ty, defaultValue}); } for (auto param : typeFun.typePackParams) { - TypePackId tp = clone(param.tp, dest, seenTypes, seenTypePacks, cloneState); + TypePackId tp = clone(param.tp, dest, cloneState); std::optional defaultValue; if (param.defaultValue) - defaultValue = clone(*param.defaultValue, dest, seenTypes, seenTypePacks, cloneState); + defaultValue = clone(*param.defaultValue, dest, cloneState); result.typePackParams.push_back({tp, defaultValue}); } - result.type = clone(typeFun.type, dest, seenTypes, seenTypePacks, cloneState); + result.type = clone(typeFun.type, dest, cloneState); return result; } diff --git a/Analysis/src/Error.cpp b/Analysis/src/Error.cpp index 5eb2ea2a..cbec0b15 100644 --- a/Analysis/src/Error.cpp +++ b/Analysis/src/Error.cpp @@ -8,7 +8,6 @@ #include -LUAU_FASTFLAGVARIABLE(BetterDiagnosticCodesInStudio, false); LUAU_FASTFLAGVARIABLE(LuauTypeMismatchModuleName, false); static std::string wrongNumberOfArgsString(size_t expectedCount, size_t actualCount, const char* argPrefix = nullptr, bool isVariadic = false) @@ -252,14 +251,7 @@ struct ErrorConverter std::string operator()(const Luau::SyntaxError& e) const { - if (FFlag::BetterDiagnosticCodesInStudio) - { - return e.message; - } - else - { - return "Syntax error: " + e.message; - } + return e.message; } std::string operator()(const Luau::CodeTooComplex&) const @@ -451,6 +443,11 @@ struct ErrorConverter { return "Cannot cast '" + toString(e.left) + "' into '" + toString(e.right) + "' because the types are unrelated"; } + + std::string operator()(const NormalizationTooComplex&) const + { + return "Code is too complex to typecheck! Consider simplifying the code around this area"; + } }; struct InvalidNameChecker @@ -716,14 +713,14 @@ bool containsParseErrorName(const TypeError& error) } template -void copyError(T& e, TypeArena& destArena, SeenTypes& seenTypes, SeenTypePacks& seenTypePacks, CloneState cloneState) +void copyError(T& e, TypeArena& destArena, CloneState cloneState) { auto clone = [&](auto&& ty) { - return ::Luau::clone(ty, destArena, seenTypes, seenTypePacks, cloneState); + return ::Luau::clone(ty, destArena, cloneState); }; auto visitErrorData = [&](auto&& e) { - copyError(e, destArena, seenTypes, seenTypePacks, cloneState); + copyError(e, destArena, cloneState); }; if constexpr (false) @@ -844,18 +841,19 @@ void copyError(T& e, TypeArena& destArena, SeenTypes& seenTypes, SeenTypePacks& e.left = clone(e.left); e.right = clone(e.right); } + else if constexpr (std::is_same_v) + { + } else static_assert(always_false_v, "Non-exhaustive type switch"); } void copyErrors(ErrorVec& errors, TypeArena& destArena) { - SeenTypes seenTypes; - SeenTypePacks seenTypePacks; CloneState cloneState; auto visitErrorData = [&](auto&& e) { - copyError(e, destArena, seenTypes, seenTypePacks, cloneState); + copyError(e, destArena, cloneState); }; LUAU_ASSERT(!destArena.typeVars.isFrozen()); diff --git a/Analysis/src/Frontend.cpp b/Analysis/src/Frontend.cpp index 000769fe..8b0b2210 100644 --- a/Analysis/src/Frontend.cpp +++ b/Analysis/src/Frontend.cpp @@ -11,16 +11,18 @@ #include "Luau/TimeTrace.h" #include "Luau/TypeInfer.h" #include "Luau/Variant.h" -#include "Luau/Common.h" #include #include #include +LUAU_FASTINT(LuauTypeInferIterationLimit) +LUAU_FASTINT(LuauTarjanChildLimit) LUAU_FASTFLAG(LuauCyclicModuleTypeSurface) LUAU_FASTFLAG(LuauInferInNoCheckMode) LUAU_FASTFLAGVARIABLE(LuauKnowsTheDataModel3, false) LUAU_FASTFLAGVARIABLE(LuauSeparateTypechecks, false) +LUAU_FASTFLAGVARIABLE(LuauAutocompleteDynamicLimits, false) LUAU_FASTINTVARIABLE(LuauAutocompleteCheckTimeoutMs, 0) namespace Luau @@ -97,13 +99,11 @@ LoadDefinitionFileResult loadDefinitionFile(TypeChecker& typeChecker, ScopePtr t if (checkedModule->errors.size() > 0) return LoadDefinitionFileResult{false, parseResult, checkedModule}; - SeenTypes seenTypes; - SeenTypePacks seenTypePacks; CloneState cloneState; for (const auto& [name, ty] : checkedModule->declaredGlobals) { - TypeId globalTy = clone(ty, typeChecker.globalTypes, seenTypes, seenTypePacks, cloneState); + TypeId globalTy = clone(ty, typeChecker.globalTypes, cloneState); std::string documentationSymbol = packageName + "/global/" + name; generateDocumentationSymbols(globalTy, documentationSymbol); targetScope->bindings[typeChecker.globalNames.names->getOrAdd(name.c_str())] = {globalTy, Location(), false, {}, documentationSymbol}; @@ -113,7 +113,7 @@ LoadDefinitionFileResult loadDefinitionFile(TypeChecker& typeChecker, ScopePtr t for (const auto& [name, ty] : checkedModule->getModuleScope()->exportedTypeBindings) { - TypeFun globalTy = clone(ty, typeChecker.globalTypes, seenTypes, seenTypePacks, cloneState); + TypeFun globalTy = clone(ty, typeChecker.globalTypes, cloneState); std::string documentationSymbol = packageName + "/globaltype/" + name; generateDocumentationSymbols(globalTy.type, documentationSymbol); targetScope->exportedTypeBindings[name] = globalTy; @@ -440,13 +440,42 @@ CheckResult Frontend::check(const ModuleName& name, std::optional 0) + typeCheckerForAutocomplete.instantiationChildLimit = + std::max(1, int(FInt::LuauTarjanChildLimit * sourceNode.autocompleteLimitsMult)); + else + typeCheckerForAutocomplete.instantiationChildLimit = std::nullopt; + + if (FInt::LuauTypeInferIterationLimit > 0) + typeCheckerForAutocomplete.unifierIterationLimit = + std::max(1, int(FInt::LuauTypeInferIterationLimit * sourceNode.autocompleteLimitsMult)); + else + typeCheckerForAutocomplete.unifierIterationLimit = std::nullopt; + } + ModulePtr moduleForAutocomplete = typeCheckerForAutocomplete.check(sourceModule, Mode::Strict); moduleResolverForAutocomplete.modules[moduleName] = moduleForAutocomplete; + double duration = getTimestamp() - timestamp; + if (moduleForAutocomplete->timeout) + { checkResult.timeoutHits.push_back(moduleName); - stats.timeCheck += getTimestamp() - timestamp; + if (FFlag::LuauAutocompleteDynamicLimits) + sourceNode.autocompleteLimitsMult = sourceNode.autocompleteLimitsMult / 2.0; + } + else if (FFlag::LuauAutocompleteDynamicLimits && duration < autocompleteTimeLimit / 2.0) + { + sourceNode.autocompleteLimitsMult = std::min(sourceNode.autocompleteLimitsMult * 2.0, 1.0); + } + + stats.timeCheck += duration; stats.filesStrict += 1; sourceNode.dirtyAutocomplete = false; diff --git a/Analysis/src/IostreamHelpers.cpp b/Analysis/src/IostreamHelpers.cpp index a8f67589..0eaa485e 100644 --- a/Analysis/src/IostreamHelpers.cpp +++ b/Analysis/src/IostreamHelpers.cpp @@ -184,6 +184,8 @@ static void errorToString(std::ostream& stream, const T& err) } else if constexpr (std::is_same_v) stream << "TypesAreUnrelated { left = '" + toString(err.left) + "', right = '" + toString(err.right) + "' }"; + else if constexpr (std::is_same_v) + stream << "NormalizationTooComplex { }"; else static_assert(always_false_v, "Non-exhaustive type switch"); } diff --git a/Analysis/src/LValue.cpp b/Analysis/src/LValue.cpp index c9466a40..72555ab4 100644 --- a/Analysis/src/LValue.cpp +++ b/Analysis/src/LValue.cpp @@ -5,6 +5,8 @@ #include +LUAU_FASTFLAG(LuauTypecheckOptPass) + namespace Luau { @@ -79,6 +81,8 @@ std::optional tryGetLValue(const AstExpr& node) std::pair> getFullName(const LValue& lvalue) { + LUAU_ASSERT(!FFlag::LuauTypecheckOptPass); + const LValue* current = &lvalue; std::vector keys; while (auto field = get(*current)) @@ -92,6 +96,19 @@ std::pair> getFullName(const LValue& lvalue) return {*symbol, std::vector(keys.rbegin(), keys.rend())}; } +Symbol getBaseSymbol(const LValue& lvalue) +{ + LUAU_ASSERT(FFlag::LuauTypecheckOptPass); + + const LValue* current = &lvalue; + while (auto field = get(*current)) + current = baseof(*current); + + const Symbol* symbol = get(*current); + LUAU_ASSERT(symbol); + return *symbol; +} + void merge(RefinementMap& l, const RefinementMap& r, std::function f) { for (const auto& [k, a] : r) diff --git a/Analysis/src/Linter.cpp b/Analysis/src/Linter.cpp index b7480e34..5608e4b3 100644 --- a/Analysis/src/Linter.cpp +++ b/Analysis/src/Linter.cpp @@ -14,7 +14,6 @@ LUAU_FASTINTVARIABLE(LuauSuggestionDistance, 4) LUAU_FASTFLAGVARIABLE(LuauLintGlobalNeverReadBeforeWritten, false) -LUAU_FASTFLAGVARIABLE(LuauLintNoRobloxBits, false) namespace Luau { @@ -1140,25 +1139,8 @@ private: Kind_Primitive, // primitive type supported by VM - boolean/userdata/etc. No differentiation between types of userdata. Kind_Vector, // 'vector' but only used when type is used Kind_Userdata, // custom userdata type - - // TODO: remove these with LuauLintNoRobloxBits - Kind_Class, // custom userdata type that reflects Roblox Instance-derived hierarchy - Part/etc. - Kind_Enum, // custom userdata type referring to an enum item of enum classes, e.g. Enum.NormalId.Back/Enum.Axis.X/etc. }; - bool containsPropName(TypeId ty, const std::string& propName) - { - LUAU_ASSERT(!FFlag::LuauLintNoRobloxBits); - - if (auto ctv = get(ty)) - return lookupClassProp(ctv, propName) != nullptr; - - if (auto ttv = get(ty)) - return ttv->props.find(propName) != ttv->props.end(); - - return false; - } - TypeKind getTypeKind(const std::string& name) { if (name == "nil" || name == "boolean" || name == "userdata" || name == "number" || name == "string" || name == "table" || @@ -1168,23 +1150,10 @@ private: if (name == "vector") return Kind_Vector; - if (FFlag::LuauLintNoRobloxBits) - { - if (std::optional maybeTy = context->scope->lookupType(name)) - return Kind_Userdata; + if (std::optional maybeTy = context->scope->lookupType(name)) + return Kind_Userdata; - return Kind_Unknown; - } - else - { - if (std::optional maybeTy = context->scope->lookupType(name)) - // Kind_Userdata is probably not 100% precise but is close enough - return containsPropName(maybeTy->type, "ClassName") ? Kind_Class : Kind_Userdata; - else if (std::optional maybeTy = context->scope->lookupImportedType("Enum", name)) - return Kind_Enum; - - return Kind_Unknown; - } + return Kind_Unknown; } void validateType(AstExprConstantString* expr, std::initializer_list expected, const char* expectedString) @@ -1202,67 +1171,11 @@ private: { if (kind == ek) return; - - // as a special case, Instance and EnumItem are both a userdata type (as returned by typeof) and a class type - if (!FFlag::LuauLintNoRobloxBits && ek == Kind_Userdata && (name == "Instance" || name == "EnumItem")) - return; } emitWarning(*context, LintWarning::Code_UnknownType, expr->location, "Unknown type '%s' (expected %s)", name.c_str(), expectedString); } - bool acceptsClassName(AstName method) - { - LUAU_ASSERT(!FFlag::LuauLintNoRobloxBits); - - return method.value[0] == 'F' && (method == "FindFirstChildOfClass" || method == "FindFirstChildWhichIsA" || - method == "FindFirstAncestorOfClass" || method == "FindFirstAncestorWhichIsA"); - } - - bool visit(AstExprCall* node) override - { - // TODO: Simply remove the override - if (FFlag::LuauLintNoRobloxBits) - return true; - - if (AstExprIndexName* index = node->func->as()) - { - AstExprConstantString* arg0 = node->args.size > 0 ? node->args.data[0]->as() : NULL; - - if (arg0) - { - if (node->self && index->index == "IsA" && node->args.size == 1) - { - validateType(arg0, {Kind_Class, Kind_Enum}, "class or enum type"); - } - else if (node->self && (index->index == "GetService" || index->index == "FindService") && node->args.size == 1) - { - AstExprGlobal* g = index->expr->as(); - - if (g && (g->name == "game" || g->name == "Game")) - { - validateType(arg0, {Kind_Class}, "class type"); - } - } - else if (node->self && acceptsClassName(index->index) && node->args.size == 1) - { - validateType(arg0, {Kind_Class}, "class type"); - } - else if (!node->self && index->index == "new" && node->args.size <= 2) - { - AstExprGlobal* g = index->expr->as(); - - if (g && g->name == "Instance") - { - validateType(arg0, {Kind_Class}, "class type"); - } - } - } - } - - return true; - } - bool visit(AstExprBinary* node) override { if (node->op == AstExprBinary::CompareNe || node->op == AstExprBinary::CompareEq) diff --git a/Analysis/src/Module.cpp b/Analysis/src/Module.cpp index 6bb45245..e2e3b436 100644 --- a/Analysis/src/Module.cpp +++ b/Analysis/src/Module.cpp @@ -1,8 +1,9 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #include "Luau/Module.h" -#include "Luau/Common.h" #include "Luau/Clone.h" +#include "Luau/Common.h" +#include "Luau/Normalize.h" #include "Luau/RecursionCounter.h" #include "Luau/Scope.h" #include "Luau/TypeInfer.h" @@ -14,6 +15,7 @@ LUAU_FASTFLAGVARIABLE(DebugLuauFreezeArena, false) LUAU_FASTFLAGVARIABLE(LuauCloneDeclaredGlobals, false) +LUAU_FASTFLAG(LuauLowerBoundsCalculation) namespace Luau { @@ -143,32 +145,51 @@ Module::~Module() unfreeze(internalTypes); } -bool Module::clonePublicInterface() +bool Module::clonePublicInterface(InternalErrorReporter& ice) { LUAU_ASSERT(interfaceTypes.typeVars.empty()); LUAU_ASSERT(interfaceTypes.typePacks.empty()); - SeenTypes seenTypes; - SeenTypePacks seenTypePacks; CloneState cloneState; ScopePtr moduleScope = getModuleScope(); - moduleScope->returnType = clone(moduleScope->returnType, interfaceTypes, seenTypes, seenTypePacks, cloneState); + moduleScope->returnType = clone(moduleScope->returnType, interfaceTypes, cloneState); if (moduleScope->varargPack) - moduleScope->varargPack = clone(*moduleScope->varargPack, interfaceTypes, seenTypes, seenTypePacks, cloneState); + moduleScope->varargPack = clone(*moduleScope->varargPack, interfaceTypes, cloneState); + + if (FFlag::LuauLowerBoundsCalculation) + { + normalize(moduleScope->returnType, interfaceTypes, ice); + if (moduleScope->varargPack) + normalize(*moduleScope->varargPack, interfaceTypes, ice); + } for (auto& [name, tf] : moduleScope->exportedTypeBindings) - tf = clone(tf, interfaceTypes, seenTypes, seenTypePacks, cloneState); + { + tf = clone(tf, interfaceTypes, cloneState); + if (FFlag::LuauLowerBoundsCalculation) + normalize(tf.type, interfaceTypes, ice); + } for (TypeId ty : moduleScope->returnType) + { if (get(follow(ty))) - *asMutable(ty) = AnyTypeVar{}; + { + auto t = asMutable(ty); + t->ty = AnyTypeVar{}; + t->normal = true; + } + } if (FFlag::LuauCloneDeclaredGlobals) { for (auto& [name, ty] : declaredGlobals) - ty = clone(ty, interfaceTypes, seenTypes, seenTypePacks, cloneState); + { + ty = clone(ty, interfaceTypes, cloneState); + if (FFlag::LuauLowerBoundsCalculation) + normalize(ty, interfaceTypes, ice); + } } freeze(internalTypes); diff --git a/Analysis/src/Normalize.cpp b/Analysis/src/Normalize.cpp new file mode 100644 index 00000000..40341ac1 --- /dev/null +++ b/Analysis/src/Normalize.cpp @@ -0,0 +1,814 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details + +#include "Luau/Normalize.h" + +#include + +#include "Luau/Clone.h" +#include "Luau/DenseHash.h" +#include "Luau/Substitution.h" +#include "Luau/Unifier.h" +#include "Luau/VisitTypeVar.h" + +LUAU_FASTFLAGVARIABLE(DebugLuauCopyBeforeNormalizing, false) + +// This could theoretically be 2000 on amd64, but x86 requires this. +LUAU_FASTINTVARIABLE(LuauNormalizeIterationLimit, 1200); +LUAU_FASTFLAGVARIABLE(LuauNormalizeCombineTableFix, false); +LUAU_FASTFLAGVARIABLE(LuauNormalizeCombineIntersectionFix, false); + +namespace Luau +{ + +namespace +{ + +struct Replacer : Substitution +{ + TypeId sourceType; + TypeId replacedType; + DenseHashMap replacedTypes{nullptr}; + DenseHashMap replacedPacks{nullptr}; + + Replacer(TypeArena* arena, TypeId sourceType, TypeId replacedType) + : Substitution(TxnLog::empty(), arena) + , sourceType(sourceType) + , replacedType(replacedType) + { + } + + bool isDirty(TypeId ty) override + { + if (!sourceType) + return false; + + auto vecHasSourceType = [sourceType = sourceType](const auto& vec) { + return end(vec) != std::find(begin(vec), end(vec), sourceType); + }; + + // Walk every kind of TypeVar and find pointers to sourceType + if (auto t = get(ty)) + return false; + else if (auto t = get(ty)) + return false; + else if (auto t = get(ty)) + return false; + else if (auto t = get(ty)) + return false; + else if (auto t = get(ty)) + return vecHasSourceType(t->parts); + else if (auto t = get(ty)) + return false; + else if (auto t = get(ty)) + { + if (vecHasSourceType(t->generics)) + return true; + + return false; + } + else if (auto t = get(ty)) + { + if (t->boundTo) + return *t->boundTo == sourceType; + + for (const auto& [_name, prop] : t->props) + { + if (prop.type == sourceType) + return true; + } + + if (auto indexer = t->indexer) + { + if (indexer->indexType == sourceType || indexer->indexResultType == sourceType) + return true; + } + + if (vecHasSourceType(t->instantiatedTypeParams)) + return true; + + return false; + } + else if (auto t = get(ty)) + return t->table == sourceType || t->metatable == sourceType; + else if (auto t = get(ty)) + return false; + else if (auto t = get(ty)) + return false; + else if (auto t = get(ty)) + return vecHasSourceType(t->options); + else if (auto t = get(ty)) + return vecHasSourceType(t->parts); + else if (auto t = get(ty)) + return false; + + LUAU_ASSERT(!"Luau::Replacer::isDirty internal error: Unknown TypeVar type"); + LUAU_UNREACHABLE(); + } + + bool isDirty(TypePackId tp) override + { + if (auto it = replacedPacks.find(tp)) + return false; + + if (auto pack = get(tp)) + { + for (TypeId ty : pack->head) + { + if (ty == sourceType) + return true; + } + return false; + } + else if (auto vtp = get(tp)) + return vtp->ty == sourceType; + else + return false; + } + + TypeId clean(TypeId ty) override + { + LUAU_ASSERT(sourceType && replacedType); + + // Walk every kind of TypeVar and create a copy with sourceType replaced by replacedType + // Before returning, memoize the result for later use. + + // Helpfully, Substitution::clone() only shallow-clones the kinds of types that we care to work with. This + // function returns the identity for things like primitives. + TypeId res = clone(ty); + + if (auto t = get(res)) + LUAU_ASSERT(!"Impossible"); + else if (auto t = get(res)) + LUAU_ASSERT(!"Impossible"); + else if (auto t = get(res)) + LUAU_ASSERT(!"Impossible"); + else if (auto t = get(res)) + LUAU_ASSERT(!"Impossible"); + else if (auto t = getMutable(res)) + { + for (TypeId& part : t->parts) + { + if (part == sourceType) + part = replacedType; + } + } + else if (auto t = get(res)) + LUAU_ASSERT(!"Impossible"); + else if (auto t = getMutable(res)) + { + // The constituent typepacks are cleaned separately. We just need to walk the generics array. + for (TypeId& g : t->generics) + { + if (g == sourceType) + g = replacedType; + } + } + else if (auto t = getMutable(res)) + { + for (auto& [_key, prop] : t->props) + { + if (prop.type == sourceType) + prop.type = replacedType; + } + } + else if (auto t = getMutable(res)) + { + if (t->table == sourceType) + t->table = replacedType; + if (t->metatable == sourceType) + t->table = replacedType; + } + else if (auto t = get(res)) + LUAU_ASSERT(!"Impossible"); + else if (auto t = get(res)) + LUAU_ASSERT(!"Impossible"); + else if (auto t = getMutable(res)) + { + for (TypeId& option : t->options) + { + if (option == sourceType) + option = replacedType; + } + } + else if (auto t = getMutable(res)) + { + for (TypeId& part : t->parts) + { + if (part == sourceType) + part = replacedType; + } + } + else if (auto t = get(res)) + LUAU_ASSERT(!"Impossible"); + else + LUAU_ASSERT(!"Luau::Replacer::clean internal error: Unknown TypeVar type"); + + replacedTypes[ty] = res; + return res; + } + + TypePackId clean(TypePackId tp) override + { + TypePackId res = clone(tp); + + if (auto pack = getMutable(res)) + { + for (TypeId& type : pack->head) + { + if (type == sourceType) + type = replacedType; + } + } + else if (auto vtp = getMutable(res)) + { + if (vtp->ty == sourceType) + vtp->ty = replacedType; + } + + replacedPacks[tp] = res; + return res; + } + + TypeId smartClone(TypeId t) + { + std::optional res = replace(t); + LUAU_ASSERT(res.has_value()); // TODO think about this + if (*res == t) + return clone(t); + return *res; + } +}; + +} // anonymous namespace + +bool isSubtype(TypeId subTy, TypeId superTy, InternalErrorReporter& ice) +{ + UnifierSharedState sharedState{&ice}; + TypeArena arena; + Unifier u{&arena, Mode::Strict, Location{}, Covariant, sharedState}; + u.anyIsTop = true; + + u.tryUnify(subTy, superTy); + const bool ok = u.errors.empty() && u.log.empty(); + return ok; +} + +template +static bool areNormal_(const T& t, const DenseHashSet& seen, InternalErrorReporter& ice) +{ + int count = 0; + auto isNormal = [&](TypeId ty) { + ++count; + if (count >= FInt::LuauNormalizeIterationLimit) + ice.ice("Luau::areNormal hit iteration limit"); + + return ty->normal || seen.find(asMutable(ty)); + }; + + return std::all_of(begin(t), end(t), isNormal); +} + +static bool areNormal(const std::vector& types, const DenseHashSet& seen, InternalErrorReporter& ice) +{ + return areNormal_(types, seen, ice); +} + +static bool areNormal(TypePackId tp, const DenseHashSet& seen, InternalErrorReporter& ice) +{ + tp = follow(tp); + if (get(tp)) + return false; + + auto [head, tail] = flatten(tp); + + if (!areNormal_(head, seen, ice)) + return false; + + if (!tail) + return true; + + if (auto vtp = get(*tail)) + return vtp->ty->normal || seen.find(asMutable(vtp->ty)); + + return true; +} + +#define CHECK_ITERATION_LIMIT(...) \ + do \ + { \ + if (iterationLimit > FInt::LuauNormalizeIterationLimit) \ + { \ + limitExceeded = true; \ + return __VA_ARGS__; \ + } \ + ++iterationLimit; \ + } while (false) + +struct Normalize +{ + TypeArena& arena; + InternalErrorReporter& ice; + + // Debug data. Types being normalized are invalidated but trying to see what's going on is painful. + // To actually see the original type, read it by using the pointer of the type being normalized. + // e.g. in lldb, `e dump(originalTys[ty])`. + SeenTypes originalTys; + SeenTypePacks originalTps; + + int iterationLimit = 0; + bool limitExceeded = false; + + template + bool operator()(TypePackId, const T&) + { + return true; + } + + template + void cycle(TID) + { + } + + bool operator()(TypeId ty, const FreeTypeVar&) + { + LUAU_ASSERT(!ty->normal); + return false; + } + + bool operator()(TypeId ty, const BoundTypeVar& btv) + { + // It should never be the case that this TypeVar is normal, but is bound to a non-normal type. + LUAU_ASSERT(!ty->normal || ty->normal == btv.boundTo->normal); + + asMutable(ty)->normal = btv.boundTo->normal; + return !ty->normal; + } + + bool operator()(TypeId ty, const PrimitiveTypeVar&) + { + LUAU_ASSERT(ty->normal); + return false; + } + + bool operator()(TypeId ty, const GenericTypeVar&) + { + if (!ty->normal) + asMutable(ty)->normal = true; + + return false; + } + + bool operator()(TypeId ty, const ErrorTypeVar&) + { + if (!ty->normal) + asMutable(ty)->normal = true; + return false; + } + + bool operator()(TypeId ty, const ConstrainedTypeVar& ctvRef, DenseHashSet& seen) + { + CHECK_ITERATION_LIMIT(false); + + ConstrainedTypeVar* ctv = const_cast(&ctvRef); + + std::vector parts = std::move(ctv->parts); + + // We might transmute, so it's not safe to rely on the builtin traversal logic of visitTypeVar + for (TypeId part : parts) + visit_detail::visit(part, *this, seen); + + std::vector newParts = normalizeUnion(parts); + + const bool normal = areNormal(newParts, seen, ice); + + if (newParts.size() == 1) + *asMutable(ty) = BoundTypeVar{newParts[0]}; + else + *asMutable(ty) = UnionTypeVar{std::move(newParts)}; + + asMutable(ty)->normal = normal; + + return false; + } + + bool operator()(TypeId ty, const FunctionTypeVar& ftv) = delete; + bool operator()(TypeId ty, const FunctionTypeVar& ftv, DenseHashSet& seen) + { + CHECK_ITERATION_LIMIT(false); + + if (ty->normal) + return false; + + visit_detail::visit(ftv.argTypes, *this, seen); + visit_detail::visit(ftv.retType, *this, seen); + + asMutable(ty)->normal = areNormal(ftv.argTypes, seen, ice) && areNormal(ftv.retType, seen, ice); + + return false; + } + + bool operator()(TypeId ty, const TableTypeVar& ttv, DenseHashSet& seen) + { + CHECK_ITERATION_LIMIT(false); + + if (ty->normal) + return false; + + bool normal = true; + + auto checkNormal = [&](TypeId t) { + // if t is on the stack, it is possible that this type is normal. + // If t is not normal and it is not on the stack, this type is definitely not normal. + if (!t->normal && !seen.find(asMutable(t))) + normal = false; + }; + + if (ttv.boundTo) + { + visit_detail::visit(*ttv.boundTo, *this, seen); + asMutable(ty)->normal = (*ttv.boundTo)->normal; + return false; + } + + for (const auto& [_name, prop] : ttv.props) + { + visit_detail::visit(prop.type, *this, seen); + checkNormal(prop.type); + } + + if (ttv.indexer) + { + visit_detail::visit(ttv.indexer->indexType, *this, seen); + checkNormal(ttv.indexer->indexType); + visit_detail::visit(ttv.indexer->indexResultType, *this, seen); + checkNormal(ttv.indexer->indexResultType); + } + + asMutable(ty)->normal = normal; + + return false; + } + + bool operator()(TypeId ty, const MetatableTypeVar& mtv, DenseHashSet& seen) + { + CHECK_ITERATION_LIMIT(false); + + if (ty->normal) + return false; + + visit_detail::visit(mtv.table, *this, seen); + visit_detail::visit(mtv.metatable, *this, seen); + + asMutable(ty)->normal = mtv.table->normal && mtv.metatable->normal; + + return false; + } + + bool operator()(TypeId ty, const ClassTypeVar& ctv) + { + if (!ty->normal) + asMutable(ty)->normal = true; + return false; + } + + bool operator()(TypeId ty, const AnyTypeVar&) + { + LUAU_ASSERT(ty->normal); + return false; + } + + bool operator()(TypeId ty, const UnionTypeVar& utvRef, DenseHashSet& seen) + { + CHECK_ITERATION_LIMIT(false); + + if (ty->normal) + return false; + + UnionTypeVar* utv = &const_cast(utvRef); + std::vector options = std::move(utv->options); + + // We might transmute, so it's not safe to rely on the builtin traversal logic of visitTypeVar + for (TypeId option : options) + visit_detail::visit(option, *this, seen); + + std::vector newOptions = normalizeUnion(options); + + const bool normal = areNormal(newOptions, seen, ice); + + LUAU_ASSERT(!newOptions.empty()); + + if (newOptions.size() == 1) + *asMutable(ty) = BoundTypeVar{newOptions[0]}; + else + utv->options = std::move(newOptions); + + asMutable(ty)->normal = normal; + + return false; + } + + bool operator()(TypeId ty, const IntersectionTypeVar& itvRef, DenseHashSet& seen) + { + CHECK_ITERATION_LIMIT(false); + + if (ty->normal) + return false; + + IntersectionTypeVar* itv = &const_cast(itvRef); + + std::vector oldParts = std::move(itv->parts); + + for (TypeId part : oldParts) + visit_detail::visit(part, *this, seen); + + std::vector tables; + for (TypeId part : oldParts) + { + part = follow(part); + if (get(part)) + tables.push_back(part); + else + { + Replacer replacer{&arena, nullptr, nullptr}; // FIXME this is super super WEIRD + combineIntoIntersection(replacer, itv, part); + } + } + + // Don't allocate a new table if there's just one in the intersection. + if (tables.size() == 1) + itv->parts.push_back(tables[0]); + else if (!tables.empty()) + { + const TableTypeVar* first = get(tables[0]); + LUAU_ASSERT(first); + + TypeId newTable = arena.addType(TableTypeVar{first->state, first->level}); + TableTypeVar* ttv = getMutable(newTable); + for (TypeId part : tables) + { + // Intuition: If combineIntoTable() needs to clone a table, any references to 'part' are cyclic and need + // to be rewritten to point at 'newTable' in the clone. + Replacer replacer{&arena, part, newTable}; + combineIntoTable(replacer, ttv, part); + } + + itv->parts.push_back(newTable); + } + + asMutable(ty)->normal = areNormal(itv->parts, seen, ice); + + if (itv->parts.size() == 1) + { + TypeId part = itv->parts[0]; + *asMutable(ty) = BoundTypeVar{part}; + } + + return false; + } + + bool operator()(TypeId ty, const LazyTypeVar&) + { + return false; + } + + std::vector normalizeUnion(const std::vector& options) + { + if (options.size() == 1) + return options; + + std::vector result; + + for (TypeId part : options) + combineIntoUnion(result, part); + + return result; + } + + void combineIntoUnion(std::vector& result, TypeId ty) + { + ty = follow(ty); + if (auto utv = get(ty)) + { + for (TypeId t : utv) + combineIntoUnion(result, t); + return; + } + + for (TypeId& part : result) + { + if (isSubtype(ty, part, ice)) + return; // no need to do anything + else if (isSubtype(part, ty, ice)) + { + part = ty; // replace the less general type by the more general one + return; + } + } + + result.push_back(ty); + } + + /** + * @param replacer knows how to clone a type such that any recursive references point at the new containing type. + * @param result is an intersection that is safe for us to mutate in-place. + */ + void combineIntoIntersection(Replacer& replacer, IntersectionTypeVar* result, TypeId ty) + { + // Note: this check guards against running out of stack space + // so if you increase the size of a stack frame, you'll need to decrease the limit. + CHECK_ITERATION_LIMIT(); + + ty = follow(ty); + if (auto itv = get(ty)) + { + for (TypeId part : itv->parts) + combineIntoIntersection(replacer, result, part); + return; + } + + // Let's say that the last part of our result intersection is always a table, if any table is part of this intersection + if (get(ty)) + { + if (result->parts.empty()) + result->parts.push_back(arena.addType(TableTypeVar{TableState::Sealed, TypeLevel{}})); + + TypeId theTable = result->parts.back(); + + if (!get(FFlag::LuauNormalizeCombineIntersectionFix ? follow(theTable) : theTable)) + { + result->parts.push_back(arena.addType(TableTypeVar{TableState::Sealed, TypeLevel{}})); + theTable = result->parts.back(); + } + + TypeId newTable = replacer.smartClone(theTable); + result->parts.back() = newTable; + + combineIntoTable(replacer, getMutable(newTable), ty); + } + else if (auto ftv = get(ty)) + { + bool merged = false; + for (TypeId& part : result->parts) + { + if (isSubtype(part, ty, ice)) + { + merged = true; + break; // no need to do anything + } + else if (isSubtype(ty, part, ice)) + { + merged = true; + part = ty; // replace the less general type by the more general one + break; + } + } + + if (!merged) + result->parts.push_back(ty); + } + else + result->parts.push_back(ty); + } + + TableState combineTableStates(TableState lhs, TableState rhs) + { + if (lhs == rhs) + return lhs; + + if (lhs == TableState::Free || rhs == TableState::Free) + return TableState::Free; + + if (lhs == TableState::Unsealed || rhs == TableState::Unsealed) + return TableState::Unsealed; + + return lhs; + } + + /** + * @param replacer gives us a way to clone a type such that recursive references are rewritten to the new + * "containing" type. + * @param table always points into a table that is safe for us to mutate. + */ + void combineIntoTable(Replacer& replacer, TableTypeVar* table, TypeId ty) + { + // Note: this check guards against running out of stack space + // so if you increase the size of a stack frame, you'll need to decrease the limit. + CHECK_ITERATION_LIMIT(); + + LUAU_ASSERT(table); + + ty = follow(ty); + + TableTypeVar* tyTable = getMutable(ty); + LUAU_ASSERT(tyTable); + + for (const auto& [propName, prop] : tyTable->props) + { + if (auto it = table->props.find(propName); it != table->props.end()) + { + /** + * If we are going to recursively merge intersections of tables, we need to ensure that we never mutate + * a table that comes from somewhere else in the type graph. + * + * smarClone() does some nice things for us: It will perform a clone that is as shallow as possible + * while still rewriting any cyclic references back to the new 'root' table. + * + * replacer also keeps a mapping of types that have previously been copied, so we have the added + * advantage here of knowing that, whether or not a new copy was actually made, the resulting TypeVar is + * safe for us to mutate in-place. + */ + TypeId clone = replacer.smartClone(it->second.type); + it->second.type = combine(replacer, clone, prop.type); + } + else + table->props.insert({propName, prop}); + } + + table->state = combineTableStates(table->state, tyTable->state); + table->level = max(table->level, tyTable->level); + } + + /** + * @param a is always cloned by the caller. It is safe to mutate in-place. + * @param b will never be mutated. + */ + TypeId combine(Replacer& replacer, TypeId a, TypeId b) + { + if (FFlag::LuauNormalizeCombineTableFix && a == b) + return a; + + if (!get(a) && !get(a)) + { + if (!FFlag::LuauNormalizeCombineTableFix && a == b) + return a; + else + return arena.addType(IntersectionTypeVar{{a, b}}); + } + + if (auto itv = getMutable(a)) + { + combineIntoIntersection(replacer, itv, b); + return a; + } + else if (auto ttv = getMutable(a)) + { + if (FFlag::LuauNormalizeCombineTableFix && !get(follow(b))) + return arena.addType(IntersectionTypeVar{{a, b}}); + combineIntoTable(replacer, ttv, b); + return a; + } + + LUAU_ASSERT(!"Impossible"); + LUAU_UNREACHABLE(); + } +}; + +#undef CHECK_ITERATION_LIMIT + +/** + * @returns A tuple of TypeId and a success indicator. (true indicates that the normalization completed successfully) + */ +std::pair normalize(TypeId ty, TypeArena& arena, InternalErrorReporter& ice) +{ + CloneState state; + if (FFlag::DebugLuauCopyBeforeNormalizing) + (void)clone(ty, arena, state); + + Normalize n{arena, ice, std::move(state.seenTypes), std::move(state.seenTypePacks)}; + DenseHashSet seen{nullptr}; + visitTypeVarOnce(ty, n, seen); + + return {ty, !n.limitExceeded}; +} + +// TODO: Think about using a temporary arena and cloning types out of it so that we +// reclaim memory used by wantonly allocated intermediate types here. +// The main wrinkle here is that we don't want clone() to copy a type if the source and dest +// arena are the same. +std::pair normalize(TypeId ty, const ModulePtr& module, InternalErrorReporter& ice) +{ + return normalize(ty, module->internalTypes, ice); +} + +/** + * @returns A tuple of TypeId and a success indicator. (true indicates that the normalization completed successfully) + */ +std::pair normalize(TypePackId tp, TypeArena& arena, InternalErrorReporter& ice) +{ + CloneState state; + if (FFlag::DebugLuauCopyBeforeNormalizing) + (void)clone(tp, arena, state); + + Normalize n{arena, ice, std::move(state.seenTypes), std::move(state.seenTypePacks)}; + DenseHashSet seen{nullptr}; + visitTypeVarOnce(tp, n, seen); + + return {tp, !n.limitExceeded}; +} + +std::pair normalize(TypePackId tp, const ModulePtr& module, InternalErrorReporter& ice) +{ + return normalize(tp, module->internalTypes, ice); +} + +} // namespace Luau diff --git a/Analysis/src/Quantify.cpp b/Analysis/src/Quantify.cpp index 94e169f1..305f83ce 100644 --- a/Analysis/src/Quantify.cpp +++ b/Analysis/src/Quantify.cpp @@ -4,6 +4,8 @@ #include "Luau/VisitTypeVar.h" +LUAU_FASTFLAG(LuauTypecheckOptPass) + namespace Luau { @@ -12,6 +14,8 @@ struct Quantifier TypeLevel level; std::vector generics; std::vector genericPacks; + bool seenGenericType = false; + bool seenMutableType = false; Quantifier(TypeLevel level) : level(level) @@ -23,6 +27,9 @@ struct Quantifier bool operator()(TypeId ty, const FreeTypeVar& ftv) { + if (FFlag::LuauTypecheckOptPass) + seenMutableType = true; + if (!level.subsumes(ftv.level)) return false; @@ -44,17 +51,40 @@ struct Quantifier return true; } + bool operator()(TypeId ty, const ConstrainedTypeVar&) + { + return true; + } + bool operator()(TypeId ty, const TableTypeVar&) { TableTypeVar& ttv = *getMutable(ty); + if (FFlag::LuauTypecheckOptPass) + { + if (ttv.state == TableState::Generic) + seenGenericType = true; + + if (ttv.state == TableState::Free) + seenMutableType = true; + } + if (ttv.state == TableState::Sealed || ttv.state == TableState::Generic) return false; if (!level.subsumes(ttv.level)) + { + if (FFlag::LuauTypecheckOptPass && ttv.state == TableState::Unsealed) + seenMutableType = true; return false; + } if (ttv.state == TableState::Free) + { ttv.state = TableState::Generic; + + if (FFlag::LuauTypecheckOptPass) + seenGenericType = true; + } else if (ttv.state == TableState::Unsealed) ttv.state = TableState::Sealed; @@ -65,6 +95,9 @@ struct Quantifier bool operator()(TypePackId tp, const FreeTypePack& ftp) { + if (FFlag::LuauTypecheckOptPass) + seenMutableType = true; + if (!level.subsumes(ftp.level)) return false; @@ -84,6 +117,9 @@ void quantify(TypeId ty, TypeLevel level) LUAU_ASSERT(ftv); ftv->generics = q.generics; ftv->genericPacks = q.genericPacks; + + if (FFlag::LuauTypecheckOptPass && ftv->generics.empty() && ftv->genericPacks.empty() && !q.seenMutableType && !q.seenGenericType) + ftv->hasNoGenerics = true; } } // namespace Luau diff --git a/Analysis/src/Substitution.cpp b/Analysis/src/Substitution.cpp index 770c7a47..8648b21e 100644 --- a/Analysis/src/Substitution.cpp +++ b/Analysis/src/Substitution.cpp @@ -7,24 +7,36 @@ #include #include +LUAU_FASTFLAG(LuauLowerBoundsCalculation) LUAU_FASTINTVARIABLE(LuauTarjanChildLimit, 1000) +LUAU_FASTFLAG(LuauTypecheckOptPass) +LUAU_FASTFLAGVARIABLE(LuauSubstituteFollowNewTypes, false) namespace Luau { void Tarjan::visitChildren(TypeId ty, int index) { - ty = log->follow(ty); + if (FFlag::LuauTypecheckOptPass) + LUAU_ASSERT(ty == log->follow(ty)); + else + ty = log->follow(ty); if (ignoreChildren(ty)) return; - if (const FunctionTypeVar* ftv = log->getMutable(ty)) + if (FFlag::LuauTypecheckOptPass) + { + if (auto pty = log->pending(ty)) + ty = &pty->pending; + } + + if (const FunctionTypeVar* ftv = FFlag::LuauTypecheckOptPass ? get(ty) : log->getMutable(ty)) { visitChild(ftv->argTypes); visitChild(ftv->retType); } - else if (const TableTypeVar* ttv = log->getMutable(ty)) + else if (const TableTypeVar* ttv = FFlag::LuauTypecheckOptPass ? get(ty) : log->getMutable(ty)) { LUAU_ASSERT(!ttv->boundTo); for (const auto& [name, prop] : ttv->props) @@ -41,38 +53,52 @@ void Tarjan::visitChildren(TypeId ty, int index) for (TypePackId itp : ttv->instantiatedTypePackParams) visitChild(itp); } - else if (const MetatableTypeVar* mtv = log->getMutable(ty)) + else if (const MetatableTypeVar* mtv = FFlag::LuauTypecheckOptPass ? get(ty) : log->getMutable(ty)) { visitChild(mtv->table); visitChild(mtv->metatable); } - else if (const UnionTypeVar* utv = log->getMutable(ty)) + else if (const UnionTypeVar* utv = FFlag::LuauTypecheckOptPass ? get(ty) : log->getMutable(ty)) { for (TypeId opt : utv->options) visitChild(opt); } - else if (const IntersectionTypeVar* itv = log->getMutable(ty)) + else if (const IntersectionTypeVar* itv = FFlag::LuauTypecheckOptPass ? get(ty) : log->getMutable(ty)) { for (TypeId part : itv->parts) visitChild(part); } + else if (const ConstrainedTypeVar* ctv = get(ty)) + { + for (TypeId part : ctv->parts) + visitChild(part); + } } void Tarjan::visitChildren(TypePackId tp, int index) { - tp = log->follow(tp); + if (FFlag::LuauTypecheckOptPass) + LUAU_ASSERT(tp == log->follow(tp)); + else + tp = log->follow(tp); if (ignoreChildren(tp)) return; - if (const TypePack* tpp = log->getMutable(tp)) + if (FFlag::LuauTypecheckOptPass) + { + if (auto ptp = log->pending(tp)) + tp = &ptp->pending; + } + + if (const TypePack* tpp = FFlag::LuauTypecheckOptPass ? get(tp) : log->getMutable(tp)) { for (TypeId tv : tpp->head) visitChild(tv); if (tpp->tail) visitChild(*tpp->tail); } - else if (const VariadicTypePack* vtp = log->getMutable(tp)) + else if (const VariadicTypePack* vtp = FFlag::LuauTypecheckOptPass ? get(tp) : log->getMutable(tp)) { visitChild(vtp->ty); } @@ -80,7 +106,10 @@ void Tarjan::visitChildren(TypePackId tp, int index) std::pair Tarjan::indexify(TypeId ty) { - ty = log->follow(ty); + if (FFlag::LuauTypecheckOptPass) + LUAU_ASSERT(ty == log->follow(ty)); + else + ty = log->follow(ty); bool fresh = !typeToIndex.contains(ty); int& index = typeToIndex[ty]; @@ -98,7 +127,10 @@ std::pair Tarjan::indexify(TypeId ty) std::pair Tarjan::indexify(TypePackId tp) { - tp = log->follow(tp); + if (FFlag::LuauTypecheckOptPass) + LUAU_ASSERT(tp == log->follow(tp)); + else + tp = log->follow(tp); bool fresh = !packToIndex.contains(tp); int& index = packToIndex[tp]; @@ -141,7 +173,7 @@ TarjanResult Tarjan::loop() if (currEdge == -1) { ++childCount; - if (FInt::LuauTarjanChildLimit > 0 && FInt::LuauTarjanChildLimit < childCount) + if (childLimit > 0 && childLimit < childCount) return TarjanResult::TooManyChildren; stack.push_back(index); @@ -229,6 +261,9 @@ TarjanResult Tarjan::loop() TarjanResult Tarjan::visitRoot(TypeId ty) { childCount = 0; + if (childLimit == 0) + childLimit = FInt::LuauTarjanChildLimit; + ty = log->follow(ty); auto [index, fresh] = indexify(ty); @@ -239,6 +274,9 @@ TarjanResult Tarjan::visitRoot(TypeId ty) TarjanResult Tarjan::visitRoot(TypePackId tp) { childCount = 0; + if (childLimit == 0) + childLimit = FInt::LuauTarjanChildLimit; + tp = log->follow(tp); auto [index, fresh] = indexify(tp); @@ -347,7 +385,13 @@ TypeId Substitution::clone(TypeId ty) TypeId result = ty; - if (const FunctionTypeVar* ftv = log->getMutable(ty)) + if (FFlag::LuauTypecheckOptPass) + { + if (auto pty = log->pending(ty)) + ty = &pty->pending; + } + + if (const FunctionTypeVar* ftv = FFlag::LuauTypecheckOptPass ? get(ty) : log->getMutable(ty)) { FunctionTypeVar clone = FunctionTypeVar{ftv->level, ftv->argTypes, ftv->retType, ftv->definition, ftv->hasSelf}; clone.generics = ftv->generics; @@ -357,7 +401,7 @@ TypeId Substitution::clone(TypeId ty) clone.argNames = ftv->argNames; result = addType(std::move(clone)); } - else if (const TableTypeVar* ttv = log->getMutable(ty)) + else if (const TableTypeVar* ttv = FFlag::LuauTypecheckOptPass ? get(ty) : log->getMutable(ty)) { LUAU_ASSERT(!ttv->boundTo); TableTypeVar clone = TableTypeVar{ttv->props, ttv->indexer, ttv->level, ttv->state}; @@ -370,24 +414,29 @@ TypeId Substitution::clone(TypeId ty) clone.tags = ttv->tags; result = addType(std::move(clone)); } - else if (const MetatableTypeVar* mtv = log->getMutable(ty)) + else if (const MetatableTypeVar* mtv = FFlag::LuauTypecheckOptPass ? get(ty) : log->getMutable(ty)) { MetatableTypeVar clone = MetatableTypeVar{mtv->table, mtv->metatable}; clone.syntheticName = mtv->syntheticName; result = addType(std::move(clone)); } - else if (const UnionTypeVar* utv = log->getMutable(ty)) + else if (const UnionTypeVar* utv = FFlag::LuauTypecheckOptPass ? get(ty) : log->getMutable(ty)) { UnionTypeVar clone; clone.options = utv->options; result = addType(std::move(clone)); } - else if (const IntersectionTypeVar* itv = log->getMutable(ty)) + else if (const IntersectionTypeVar* itv = FFlag::LuauTypecheckOptPass ? get(ty) : log->getMutable(ty)) { IntersectionTypeVar clone; clone.parts = itv->parts; result = addType(std::move(clone)); } + else if (const ConstrainedTypeVar* ctv = get(ty)) + { + ConstrainedTypeVar clone{ctv->level, ctv->parts}; + result = addType(std::move(clone)); + } asMutable(result)->documentationSymbol = ty->documentationSymbol; return result; @@ -396,14 +445,21 @@ TypeId Substitution::clone(TypeId ty) TypePackId Substitution::clone(TypePackId tp) { tp = log->follow(tp); - if (const TypePack* tpp = log->getMutable(tp)) + + if (FFlag::LuauTypecheckOptPass) + { + if (auto ptp = log->pending(tp)) + tp = &ptp->pending; + } + + if (const TypePack* tpp = FFlag::LuauTypecheckOptPass ? get(tp) : log->getMutable(tp)) { TypePack clone; clone.head = tpp->head; clone.tail = tpp->tail; return addTypePack(std::move(clone)); } - else if (const VariadicTypePack* vtp = log->getMutable(tp)) + else if (const VariadicTypePack* vtp = FFlag::LuauTypecheckOptPass ? get(tp) : log->getMutable(tp)) { VariadicTypePack clone; clone.ty = vtp->ty; @@ -415,25 +471,34 @@ TypePackId Substitution::clone(TypePackId tp) void Substitution::foundDirty(TypeId ty) { - ty = log->follow(ty); - if (isDirty(ty)) - newTypes[ty] = clean(ty); + if (FFlag::LuauTypecheckOptPass) + LUAU_ASSERT(ty == log->follow(ty)); else - newTypes[ty] = clone(ty); + ty = log->follow(ty); + + if (isDirty(ty)) + newTypes[ty] = FFlag::LuauSubstituteFollowNewTypes ? follow(clean(ty)) : clean(ty); + else + newTypes[ty] = FFlag::LuauSubstituteFollowNewTypes ? follow(clone(ty)) : clone(ty); } void Substitution::foundDirty(TypePackId tp) { - tp = log->follow(tp); - if (isDirty(tp)) - newPacks[tp] = clean(tp); + if (FFlag::LuauTypecheckOptPass) + LUAU_ASSERT(tp == log->follow(tp)); else - newPacks[tp] = clone(tp); + tp = log->follow(tp); + + if (isDirty(tp)) + newPacks[tp] = FFlag::LuauSubstituteFollowNewTypes ? follow(clean(tp)) : clean(tp); + else + newPacks[tp] = FFlag::LuauSubstituteFollowNewTypes ? follow(clone(tp)) : clone(tp); } TypeId Substitution::replace(TypeId ty) { ty = log->follow(ty); + if (TypeId* prevTy = newTypes.find(ty)) return *prevTy; else @@ -443,6 +508,7 @@ TypeId Substitution::replace(TypeId ty) TypePackId Substitution::replace(TypePackId tp) { tp = log->follow(tp); + if (TypePackId* prevTp = newPacks.find(tp)) return *prevTp; else @@ -451,7 +517,13 @@ TypePackId Substitution::replace(TypePackId tp) void Substitution::replaceChildren(TypeId ty) { - ty = log->follow(ty); + if (BoundTypeVar* btv = log->getMutable(ty); FFlag::LuauLowerBoundsCalculation && btv) + btv->boundTo = replace(btv->boundTo); + + if (FFlag::LuauTypecheckOptPass) + LUAU_ASSERT(ty == log->follow(ty)); + else + ty = log->follow(ty); if (ignoreChildren(ty)) return; @@ -493,11 +565,19 @@ void Substitution::replaceChildren(TypeId ty) for (TypeId& part : itv->parts) part = replace(part); } + else if (ConstrainedTypeVar* ctv = getMutable(ty)) + { + for (TypeId& part : ctv->parts) + part = replace(part); + } } void Substitution::replaceChildren(TypePackId tp) { - tp = log->follow(tp); + if (FFlag::LuauTypecheckOptPass) + LUAU_ASSERT(tp == log->follow(tp)); + else + tp = log->follow(tp); if (ignoreChildren(tp)) return; diff --git a/Analysis/src/ToDot.cpp b/Analysis/src/ToDot.cpp index df9d4188..cb54bfc1 100644 --- a/Analysis/src/ToDot.cpp +++ b/Analysis/src/ToDot.cpp @@ -237,6 +237,15 @@ void StateDot::visitChildren(TypeId ty, int index) finishNodeLabel(ty); finishNode(); } + else if (const ConstrainedTypeVar* ctv = get(ty)) + { + formatAppend(result, "ConstrainedTypeVar %d", index); + finishNodeLabel(ty); + finishNode(); + + for (TypeId part : ctv->parts) + visitChild(part, index); + } else if (get(ty)) { formatAppend(result, "ErrorTypeVar %d", index); @@ -258,6 +267,28 @@ void StateDot::visitChildren(TypeId ty, int index) if (ctv->metatable) visitChild(*ctv->metatable, index, "[metatable]"); } + else if (const SingletonTypeVar* stv = get(ty)) + { + std::string res; + + if (const StringSingleton* ss = get(stv)) + { + // Don't put in quotes anywhere. If it's outside of the call to escape, + // then it's invalid syntax. If it's inside, then escaping is super noisy. + res = "string: " + escape(ss->value); + } + else if (const BooleanSingleton* bs = get(stv)) + { + res = "boolean: "; + res += bs->value ? "true" : "false"; + } + else + LUAU_ASSERT(!"unknown singleton type"); + + formatAppend(result, "SingletonTypeVar %s", res.c_str()); + finishNodeLabel(ty); + finishNode(); + } else { LUAU_ASSERT(!"unknown type kind"); diff --git a/Analysis/src/ToString.cpp b/Analysis/src/ToString.cpp index 59ee6de2..610842da 100644 --- a/Analysis/src/ToString.cpp +++ b/Analysis/src/ToString.cpp @@ -10,6 +10,8 @@ #include #include +LUAU_FASTFLAG(LuauLowerBoundsCalculation) + /* * Prefix generic typenames with gen- * Additionally, free types will be prefixed with free- and suffixed with their level. eg free-a-4 @@ -33,8 +35,8 @@ struct FindCyclicTypes bool exhaustive = false; std::unordered_set visited; std::unordered_set visitedPacks; - std::unordered_set cycles; - std::unordered_set cycleTPs; + std::set cycles; + std::set cycleTPs; void cycle(TypeId ty) { @@ -86,7 +88,7 @@ struct FindCyclicTypes }; template -void findCyclicTypes(std::unordered_set& cycles, std::unordered_set& cycleTPs, TID ty, bool exhaustive) +void findCyclicTypes(std::set& cycles, std::set& cycleTPs, TID ty, bool exhaustive) { FindCyclicTypes fct; fct.exhaustive = exhaustive; @@ -124,6 +126,7 @@ struct StringifierState std::unordered_map cycleTpNames; std::unordered_set seen; std::unordered_set usedNames; + size_t indentation = 0; bool exhaustive; @@ -216,6 +219,34 @@ struct StringifierState result.name += s; } + + void indent() + { + indentation += 4; + } + + void dedent() + { + indentation -= 4; + } + + void newline() + { + if (!opts.useLineBreaks) + return emit(" "); + + emit("\n"); + emitIndentation(); + } + +private: + void emitIndentation() + { + if (!opts.indent) + return; + + emit(std::string(indentation, ' ')); + } }; struct TypeVarStringifier @@ -321,7 +352,7 @@ struct TypeVarStringifier stringify(btv.boundTo); } - void operator()(TypeId ty, const Unifiable::Generic& gtv) + void operator()(TypeId ty, const GenericTypeVar& gtv) { if (gtv.explicitName) { @@ -332,6 +363,26 @@ struct TypeVarStringifier state.emit(state.getName(ty)); } + void operator()(TypeId, const ConstrainedTypeVar& ctv) + { + state.result.invalid = true; + + state.emit("[["); + + bool first = true; + for (TypeId ty : ctv.parts) + { + if (first) + first = false; + else + state.emit("|"); + + stringify(ty); + } + + state.emit("]]"); + } + void operator()(TypeId, const PrimitiveTypeVar& ptv) { switch (ptv.type) @@ -415,10 +466,25 @@ struct TypeVarStringifier state.emit(") -> "); bool plural = true; - if (auto retPack = get(follow(ftv.retType))) + + if (FFlag::LuauLowerBoundsCalculation) { - if (retPack->head.size() == 1 && !retPack->tail) - plural = false; + auto retBegin = begin(ftv.retType); + auto retEnd = end(ftv.retType); + if (retBegin != retEnd) + { + ++retBegin; + if (retBegin == retEnd && !retBegin.tail()) + plural = false; + } + } + else + { + if (auto retPack = get(follow(ftv.retType))) + { + if (retPack->head.size() == 1 && !retPack->tail) + plural = false; + } } if (plural) @@ -511,6 +577,7 @@ struct TypeVarStringifier } state.emit(openbrace); + state.indent(); bool comma = false; if (ttv.indexer) @@ -527,7 +594,10 @@ struct TypeVarStringifier for (const auto& [name, prop] : ttv.props) { if (comma) - state.emit(state.opts.useLineBreaks ? ",\n" : ", "); + { + state.emit(","); + state.newline(); + } size_t length = state.result.name.length() - oldLength; @@ -553,6 +623,7 @@ struct TypeVarStringifier ++index; } + state.dedent(); state.emit(closedbrace); state.unsee(&ttv); @@ -563,7 +634,8 @@ struct TypeVarStringifier state.result.invalid = true; state.emit("{ @metatable "); stringify(mtv.metatable); - state.emit(state.opts.useLineBreaks ? ",\n" : ", "); + state.emit(","); + state.newline(); stringify(mtv.table); state.emit(" }"); } @@ -784,13 +856,16 @@ struct TypePackStringifier if (tp.tail && !isEmpty(*tp.tail)) { - const auto& tail = *tp.tail; - if (first) - first = false; - else - state.emit(", "); + TypePackId tail = follow(*tp.tail); + if (auto vtp = get(tail); !vtp || (!FFlag::DebugLuauVerboseTypeNames && !vtp->hidden)) + { + if (first) + first = false; + else + state.emit(", "); - stringify(tail); + stringify(tail); + } } state.unsee(&tp); @@ -805,6 +880,8 @@ struct TypePackStringifier void operator()(TypePackId, const VariadicTypePack& pack) { state.emit("..."); + if (FFlag::DebugLuauVerboseTypeNames && pack.hidden) + state.emit(""); stringify(pack.ty); } @@ -858,15 +935,12 @@ void TypeVarStringifier::stringify(TypePackId tpid, const std::vector& cycles, const std::unordered_set& cycleTPs, +static void assignCycleNames(const std::set& cycles, const std::set& cycleTPs, std::unordered_map& cycleNames, std::unordered_map& cycleTpNames, bool exhaustive) { int nextIndex = 1; - std::vector sortedCycles{cycles.begin(), cycles.end()}; - std::sort(sortedCycles.begin(), sortedCycles.end(), std::less{}); - - for (TypeId cycleTy : sortedCycles) + for (TypeId cycleTy : cycles) { std::string name; @@ -888,10 +962,7 @@ static void assignCycleNames(const std::unordered_set& cycles, const std cycleNames[cycleTy] = std::move(name); } - std::vector sortedCycleTps{cycleTPs.begin(), cycleTPs.end()}; - std::sort(sortedCycleTps.begin(), sortedCycleTps.end(), std::less()); - - for (TypePackId tp : sortedCycleTps) + for (TypePackId tp : cycleTPs) { std::string name = "tp" + std::to_string(nextIndex); ++nextIndex; @@ -913,8 +984,8 @@ ToStringResult toStringDetailed(TypeId ty, const ToStringOptions& opts) StringifierState state{opts, result, opts.nameMap}; - std::unordered_set cycles; - std::unordered_set cycleTPs; + std::set cycles; + std::set cycleTPs; findCyclicTypes(cycles, cycleTPs, ty, opts.exhaustive); @@ -1016,8 +1087,8 @@ ToStringResult toStringDetailed(TypePackId tp, const ToStringOptions& opts) ToStringResult result; StringifierState state{opts, result, opts.nameMap}; - std::unordered_set cycles; - std::unordered_set cycleTPs; + std::set cycles; + std::set cycleTPs; findCyclicTypes(cycles, cycleTPs, tp, opts.exhaustive); @@ -1058,7 +1129,7 @@ ToStringResult toStringDetailed(TypePackId tp, const ToStringOptions& opts) state.emit(name); state.emit(" = "); Luau::visit( - [&tvs, cycleTy = cycleTy](auto&& t) { + [&tvs, cycleTy = cycleTy](auto t) { return tvs(cycleTy, t); }, cycleTy->ty); @@ -1163,14 +1234,18 @@ std::string toStringNamedFunction(const std::string& funcName, const FunctionTyp if (argPackIter.tail()) { - if (!first) - state.emit(", "); + if (auto vtp = get(*argPackIter.tail()); !vtp || !vtp->hidden) + { + if (!first) + state.emit(", "); - state.emit("...: "); - if (auto vtp = get(*argPackIter.tail())) - tvs.stringify(vtp->ty); - else - tvs.stringify(*argPackIter.tail()); + state.emit("...: "); + + if (vtp) + tvs.stringify(vtp->ty); + else + tvs.stringify(*argPackIter.tail()); + } } state.emit("): "); @@ -1210,6 +1285,24 @@ std::string dump(TypePackId ty) return s; } +std::string dump(const ScopePtr& scope, const char* name) +{ + auto binding = scope->linearSearchForBinding(name); + if (!binding) + { + printf("No binding %s\n", name); + return {}; + } + + TypeId ty = binding->typeId; + ToStringOptions opts; + opts.exhaustive = true; + opts.functionTypeArguments = true; + std::string s = toString(ty, opts); + printf("%s\n", s.c_str()); + return s; +} + std::string generateName(size_t i) { std::string n; diff --git a/Analysis/src/TopoSortStatements.cpp b/Analysis/src/TopoSortStatements.cpp index 678001bf..1ea2e27d 100644 --- a/Analysis/src/TopoSortStatements.cpp +++ b/Analysis/src/TopoSortStatements.cpp @@ -215,6 +215,7 @@ struct ArcCollector : public AstVisitor } } + // Adds a dependency from the current node to the named node. void add(const Identifier& name) { Node** it = map.find(name); diff --git a/Analysis/src/TxnLog.cpp b/Analysis/src/TxnLog.cpp index 5fbb596d..a5f9d26c 100644 --- a/Analysis/src/TxnLog.cpp +++ b/Analysis/src/TxnLog.cpp @@ -8,6 +8,7 @@ #include LUAU_FASTFLAGVARIABLE(LuauTxnLogPreserveOwner, false) +LUAU_FASTFLAGVARIABLE(LuauJustOneCallFrameForHaveSeen, false) namespace Luau { @@ -161,18 +162,37 @@ void TxnLog::popSeen(TypePackId lhs, TypePackId rhs) bool TxnLog::haveSeen(TypeOrPackId lhs, TypeOrPackId rhs) const { - const std::pair sortedPair = (lhs > rhs) ? std::make_pair(lhs, rhs) : std::make_pair(rhs, lhs); - if (sharedSeen->end() != std::find(sharedSeen->begin(), sharedSeen->end(), sortedPair)) + if (FFlag::LuauJustOneCallFrameForHaveSeen && !FFlag::LuauTypecheckOptPass) { - return true; - } + // This function will technically work if `this` is nullptr, but this + // indicates a bug, so we explicitly assert. + LUAU_ASSERT(static_cast(this) != nullptr); - if (parent) + const std::pair sortedPair = (lhs > rhs) ? std::make_pair(lhs, rhs) : std::make_pair(rhs, lhs); + + for (const TxnLog* current = this; current; current = current->parent) + { + if (current->sharedSeen->end() != std::find(current->sharedSeen->begin(), current->sharedSeen->end(), sortedPair)) + return true; + } + + return false; + } + else { - return parent->haveSeen(lhs, rhs); - } + const std::pair sortedPair = (lhs > rhs) ? std::make_pair(lhs, rhs) : std::make_pair(rhs, lhs); + if (sharedSeen->end() != std::find(sharedSeen->begin(), sharedSeen->end(), sortedPair)) + { + return true; + } - return false; + if (!FFlag::LuauTypecheckOptPass && parent) + { + return parent->haveSeen(lhs, rhs); + } + + return false; + } } void TxnLog::pushSeen(TypeOrPackId lhs, TypeOrPackId rhs) @@ -222,8 +242,8 @@ PendingType* TxnLog::pending(TypeId ty) const for (const TxnLog* current = this; current; current = current->parent) { - if (auto it = current->typeVarChanges.find(ty); it != current->typeVarChanges.end()) - return it->second.get(); + if (auto it = current->typeVarChanges.find(ty)) + return it->get(); } return nullptr; @@ -237,8 +257,8 @@ PendingTypePack* TxnLog::pending(TypePackId tp) const for (const TxnLog* current = this; current; current = current->parent) { - if (auto it = current->typePackChanges.find(tp); it != current->typePackChanges.end()) - return it->second.get(); + if (auto it = current->typePackChanges.find(tp)) + return it->get(); } return nullptr; diff --git a/Analysis/src/TypeAttach.cpp b/Analysis/src/TypeAttach.cpp index d575e023..bc8d0d4e 100644 --- a/Analysis/src/TypeAttach.cpp +++ b/Analysis/src/TypeAttach.cpp @@ -94,6 +94,16 @@ public: } } + AstType* operator()(const ConstrainedTypeVar& ctv) + { + AstArray types; + types.size = ctv.parts.size(); + types.data = static_cast(allocator->allocate(sizeof(AstType*) * ctv.parts.size())); + for (size_t i = 0; i < ctv.parts.size(); ++i) + types.data[i] = Luau::visit(*this, ctv.parts[i]->ty); + return allocator->alloc(Location(), types); + } + AstType* operator()(const SingletonTypeVar& stv) { if (const BooleanSingleton* bs = get(&stv)) @@ -364,6 +374,9 @@ public: AstTypePack* operator()(const VariadicTypePack& vtp) const { + if (vtp.hidden) + return nullptr; + return allocator->alloc(Location(), Luau::visit(*typeVisitor, vtp.ty->ty)); } diff --git a/Analysis/src/TypeInfer.cpp b/Analysis/src/TypeInfer.cpp index 10930248..af42a4e6 100644 --- a/Analysis/src/TypeInfer.cpp +++ b/Analysis/src/TypeInfer.cpp @@ -3,12 +3,15 @@ #include "Luau/Common.h" #include "Luau/ModuleResolver.h" +#include "Luau/Normalize.h" +#include "Luau/Parser.h" #include "Luau/Quantify.h" #include "Luau/RecursionCounter.h" #include "Luau/Scope.h" #include "Luau/Substitution.h" #include "Luau/TopoSortStatements.h" #include "Luau/TypePack.h" +#include "Luau/ToString.h" #include "Luau/TypeUtils.h" #include "Luau/ToString.h" #include "Luau/TypeVar.h" @@ -19,14 +22,17 @@ LUAU_FASTFLAGVARIABLE(DebugLuauMagicTypes, false) LUAU_FASTINTVARIABLE(LuauTypeInferRecursionLimit, 500) +LUAU_FASTINTVARIABLE(LuauTypeInferIterationLimit, 2000) LUAU_FASTINTVARIABLE(LuauTypeInferTypePackLoopLimit, 5000) LUAU_FASTINTVARIABLE(LuauCheckRecursionLimit, 500) LUAU_FASTFLAG(LuauKnowsTheDataModel3) LUAU_FASTFLAG(LuauSeparateTypechecks) +LUAU_FASTFLAG(LuauAutocompleteDynamicLimits) LUAU_FASTFLAG(LuauAutocompleteSingletonTypes) LUAU_FASTFLAGVARIABLE(LuauCyclicModuleTypeSurface, false) LUAU_FASTFLAGVARIABLE(LuauEqConstraint, false) LUAU_FASTFLAGVARIABLE(LuauWeakEqConstraint, false) // Eventually removed as false. +LUAU_FASTFLAGVARIABLE(LuauLowerBoundsCalculation, false) LUAU_FASTFLAGVARIABLE(DebugLuauFreezeDuringUnification, false) LUAU_FASTFLAGVARIABLE(LuauRecursiveTypeParameterRestriction, false) LUAU_FASTFLAGVARIABLE(LuauGenericFunctionsDontCacheTypeParams, false) @@ -39,6 +45,7 @@ LUAU_FASTFLAGVARIABLE(LuauErrorRecoveryType, false) LUAU_FASTFLAGVARIABLE(LuauOnlyMutateInstantiatedTables, false) LUAU_FASTFLAGVARIABLE(LuauPropertiesGetExpectedType, false) LUAU_FASTFLAGVARIABLE(LuauStatFunctionSimplify4, false) +LUAU_FASTFLAGVARIABLE(LuauTypecheckOptPass, false) LUAU_FASTFLAGVARIABLE(LuauUnsealedTableLiteral, false) LUAU_FASTFLAG(LuauTypeMismatchModuleName) LUAU_FASTFLAGVARIABLE(LuauTwoPassAliasDefinitionFix, false) @@ -53,6 +60,8 @@ LUAU_FASTFLAGVARIABLE(LuauCheckImplicitNumbericKeys, false) LUAU_FASTFLAG(LuauAnyInIsOptionalIsOptional) LUAU_FASTFLAGVARIABLE(LuauDecoupleOperatorInferenceFromUnifiedTypeInference, false) LUAU_FASTFLAGVARIABLE(LuauArgCountMismatchSaysAtLeastWhenVariadic, false) +LUAU_FASTFLAGVARIABLE(LuauTableUseCounterInstead, false) +LUAU_FASTFLAGVARIABLE(LuauRecursionLimitException, false); namespace Luau { @@ -140,6 +149,34 @@ bool hasBreak(AstStat* node) } } +static bool hasReturn(const AstStat* node) +{ + struct Searcher : AstVisitor + { + bool result = false; + + bool visit(AstStat*) override + { + return !result; // if we've already found a return statement, don't bother to traverse inward anymore + } + + bool visit(AstStatReturn*) override + { + result = true; + return false; + } + + bool visit(AstExprFunction*) override + { + return false; // We don't care if the function uses a lambda that itself returns + } + }; + + Searcher searcher; + const_cast(node)->visit(&searcher); + return searcher.result; +} + // returns the last statement before the block exits, or nullptr if the block never exits const AstStat* getFallthrough(const AstStat* node) { @@ -253,6 +290,26 @@ TypeChecker::TypeChecker(ModuleResolver* resolver, InternalErrorReporter* iceHan } ModulePtr TypeChecker::check(const SourceModule& module, Mode mode, std::optional environmentScope) +{ + if (FFlag::LuauRecursionLimitException) + { + try + { + return checkWithoutRecursionCheck(module, mode, environmentScope); + } + catch (const RecursionLimitException&) + { + reportErrorCodeTooComplex(module.root->location); + return std::move(currentModule); + } + } + else + { + return checkWithoutRecursionCheck(module, mode, environmentScope); + } +} + +ModulePtr TypeChecker::checkWithoutRecursionCheck(const SourceModule& module, Mode mode, std::optional environmentScope) { LUAU_TIMETRACE_SCOPE("TypeChecker::check", "TypeChecker"); LUAU_TIMETRACE_ARGUMENT("module", module.name.c_str()); @@ -268,6 +325,12 @@ ModulePtr TypeChecker::check(const SourceModule& module, Mode mode, std::optiona iceHandler->moduleName = module.name; + if (FFlag::LuauAutocompleteDynamicLimits) + { + unifierState.counters.recursionLimit = FInt::LuauTypeInferRecursionLimit; + unifierState.counters.iterationLimit = unifierIterationLimit ? *unifierIterationLimit : FInt::LuauTypeInferIterationLimit; + } + ScopePtr parentScope = environmentScope.value_or(globalScope); ScopePtr moduleScope = std::make_shared(parentScope); @@ -312,7 +375,7 @@ ModulePtr TypeChecker::check(const SourceModule& module, Mode mode, std::optiona prepareErrorsForDisplay(currentModule->errors); - bool encounteredFreeType = currentModule->clonePublicInterface(); + bool encounteredFreeType = currentModule->clonePublicInterface(*iceHandler); if (encounteredFreeType) { reportError(TypeError{module.root->location, @@ -415,7 +478,26 @@ void TypeChecker::checkBlock(const ScopePtr& scope, const AstStatBlock& block) reportErrorCodeTooComplex(block.location); return; } + if (FFlag::LuauRecursionLimitException) + { + try + { + checkBlockWithoutRecursionCheck(scope, block); + } + catch (const RecursionLimitException&) + { + reportErrorCodeTooComplex(block.location); + return; + } + } + else + { + checkBlockWithoutRecursionCheck(scope, block); + } +} +void TypeChecker::checkBlockWithoutRecursionCheck(const ScopePtr& scope, const AstStatBlock& block) +{ int subLevel = 0; std::vector sorted(block.body.data, block.body.data + block.body.size); @@ -435,6 +517,16 @@ void TypeChecker::checkBlock(const ScopePtr& scope, const AstStatBlock& block) std::unordered_map> functionDecls; + auto isLocalLambda = [](AstStat* stat) -> AstStatLocal* { + AstStatLocal* local = stat->as(); + + if (FFlag::LuauLowerBoundsCalculation && local && local->vars.size == 1 && local->values.size == 1 && + local->values.data[0]->is()) + return local; + else + return nullptr; + }; + auto checkBody = [&](AstStat* stat) { if (auto fun = stat->as()) { @@ -482,7 +574,7 @@ void TypeChecker::checkBlock(const ScopePtr& scope, const AstStatBlock& block) // function f(x:a):a local x: number = g(37) return x end // function g(x:number):number return f(x) end // ``` - if (containsFunctionCallOrReturn(**protoIter)) + if (containsFunctionCallOrReturn(**protoIter) || (FFlag::LuauLowerBoundsCalculation && isLocalLambda(*protoIter))) { while (checkIter != protoIter) { @@ -513,7 +605,8 @@ void TypeChecker::checkBlock(const ScopePtr& scope, const AstStatBlock& block) functionDecls[*protoIter] = pair; ++subLevel; - TypeId leftType = checkFunctionName(scope, *fun->name, funScope->level); + TypeId leftType = follow(checkFunctionName(scope, *fun->name, funScope->level)); + unify(funTy, leftType, fun->location); } else if (auto fun = (*protoIter)->as()) @@ -658,6 +751,16 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatRepeat& statement) checkExpr(repScope, *statement.condition); } +void TypeChecker::unifyLowerBound(TypePackId subTy, TypePackId superTy, const Location& location) +{ + Unifier state = mkUnifier(location); + state.unifyLowerBound(subTy, superTy); + + state.log.commit(); + + reportErrors(state.errors); +} + void TypeChecker::check(const ScopePtr& scope, const AstStatReturn& return_) { std::vector> expectedTypes; @@ -682,6 +785,12 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatReturn& return_) TypePackId retPack = checkExprList(scope, return_.location, return_.list, false, {}, expectedTypes).type; + if (useConstrainedIntersections()) + { + unifyLowerBound(retPack, scope->returnType, return_.location); + return; + } + // HACK: Nonstrict mode gets a bit too smart and strict for us when we // start typechecking everything across module boundaries. if (isNonstrictMode() && follow(scope->returnType) == follow(currentModule->getModuleScope()->returnType)) @@ -1209,9 +1318,11 @@ void TypeChecker::check(const ScopePtr& scope, TypeId ty, const ScopePtr& funSco else if (tableSelf->state == TableState::Sealed) reportError(TypeError{function.location, CannotExtendTable{selfTy, CannotExtendTable::Property, indexName->index.value}}); + const bool tableIsExtendable = tableSelf && tableSelf->state != TableState::Sealed; + ty = follow(ty); - if (tableSelf && tableSelf->state != TableState::Sealed) + if (tableIsExtendable) tableSelf->props[indexName->index.value] = {ty, /* deprecated */ false, {}, indexName->indexLocation}; const FunctionTypeVar* funTy = get(ty); @@ -1224,7 +1335,7 @@ void TypeChecker::check(const ScopePtr& scope, TypeId ty, const ScopePtr& funSco checkFunctionBody(funScope, ty, *function.func); - if (tableSelf && tableSelf->state != TableState::Sealed) + if (tableIsExtendable) tableSelf->props[indexName->index.value] = { follow(quantify(funScope, ty, indexName->indexLocation)), /* deprecated */ false, {}, indexName->indexLocation}; } @@ -1372,7 +1483,11 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatTypeAlias& typealias for (auto param : binding->typePackParams) clone.instantiatedTypePackParams.push_back(param.tp); + bool isNormal = ty->normal; ty = addType(std::move(clone)); + + if (FFlag::LuauLowerBoundsCalculation) + asMutable(ty)->normal = isNormal; } } else @@ -1400,6 +1515,14 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatTypeAlias& typealias if (FFlag::LuauTwoPassAliasDefinitionFix && ok) bindingType = ty; + + if (FFlag::LuauLowerBoundsCalculation) + { + auto [t, ok] = normalize(bindingType, currentModule, *iceHandler); + bindingType = t; + if (!ok) + reportError(typealias.location, NormalizationTooComplex{}); + } } } @@ -1673,10 +1796,11 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprCa { return {pack->head.empty() ? nilType : pack->head[0], std::move(result.predicates)}; } - else if (get(retPack)) + else if (const FreeTypePack* ftp = get(retPack)) { - TypeId head = freshType(scope); - TypePackId pack = addTypePack(TypePackVar{TypePack{{head}, freshTypePack(scope)}}); + TypeLevel level = FFlag::LuauLowerBoundsCalculation ? ftp->level : scope->level; + TypeId head = freshType(level); + TypePackId pack = addTypePack(TypePackVar{TypePack{{head}, freshTypePack(level)}}); unify(pack, retPack, expr.location); return {head, std::move(result.predicates)}; } @@ -1793,7 +1917,7 @@ std::optional TypeChecker::getIndexTypeFromType( for (TypeId t : utv) { - RecursionLimiter _rl(&recursionCount, FInt::LuauTypeInferRecursionLimit); + RecursionLimiter _rl(&recursionCount, FInt::LuauTypeInferRecursionLimit, "getIndexTypeForType unions"); // Not needed when we normalize types. if (get(follow(t))) @@ -1817,12 +1941,25 @@ std::optional TypeChecker::getIndexTypeFromType( return std::nullopt; } - std::vector result = reduceUnion(goodOptions); + if (FFlag::LuauLowerBoundsCalculation) + { + auto [t, ok] = normalize(addType(UnionTypeVar{std::move(goodOptions)}), currentModule, + *iceHandler); // FIXME Inefficient. We craft a UnionTypeVar and immediately throw it away. - if (result.size() == 1) - return result[0]; + if (!ok) + reportError(location, NormalizationTooComplex{}); - return addType(UnionTypeVar{std::move(result)}); + return t; + } + else + { + std::vector result = reduceUnion(goodOptions); + + if (result.size() == 1) + return result[0]; + + return addType(UnionTypeVar{std::move(result)}); + } } else if (const IntersectionTypeVar* itv = get(type)) { @@ -1830,7 +1967,7 @@ std::optional TypeChecker::getIndexTypeFromType( for (TypeId t : itv->parts) { - RecursionLimiter _rl(&recursionCount, FInt::LuauTypeInferRecursionLimit); + RecursionLimiter _rl(&recursionCount, FInt::LuauTypeInferRecursionLimit, "getIndexTypeFromType intersections"); if (std::optional ty = getIndexTypeFromType(scope, t, name, location, false)) parts.push_back(*ty); @@ -1982,7 +2119,6 @@ TypeId TypeChecker::stripFromNilAndReport(TypeId ty, const Location& location) { if (!std::any_of(begin(utv), end(utv), isNil)) return ty; - } if (std::optional strippedUnion = tryStripUnionFromNil(ty)) @@ -2124,7 +2260,26 @@ TypeId TypeChecker::checkExprTable( ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprTable& expr, std::optional expectedType) { - RecursionLimiter _rl(&recursionCount, FInt::LuauTypeInferRecursionLimit); + if (FFlag::LuauTableUseCounterInstead) + { + RecursionCounter _rc(&checkRecursionCount); + if (FInt::LuauCheckRecursionLimit > 0 && checkRecursionCount >= FInt::LuauCheckRecursionLimit) + { + reportErrorCodeTooComplex(expr.location); + return {errorRecoveryType(scope)}; + } + + return checkExpr_(scope, expr, expectedType); + } + else + { + RecursionLimiter _rl(&recursionCount, FInt::LuauTypeInferRecursionLimit, "checkExpr for tables"); + return checkExpr_(scope, expr, expectedType); + } +} + +ExprResult TypeChecker::checkExpr_(const ScopePtr& scope, const AstExprTable& expr, std::optional expectedType) +{ std::vector> fieldTypes(expr.items.size); const TableTypeVar* expectedTable = nullptr; @@ -3176,6 +3331,10 @@ std::pair TypeChecker::checkFunctionSignature( funScope->varargPack = anyTypePack; } } + else if (FFlag::LuauLowerBoundsCalculation && !isNonstrictMode()) + { + funScope->varargPack = addTypePack(TypePackVar{VariadicTypePack{anyType, /*hidden*/ true}}); + } std::vector argTypes; @@ -3311,9 +3470,24 @@ void TypeChecker::checkFunctionBody(const ScopePtr& scope, TypeId ty, const AstE { check(scope, *function.body); - // We explicitly don't follow here to check if we have a 'true' free type instead of bound one - if (get_if(&funTy->retType->ty)) - *asMutable(funTy->retType) = TypePack{{}, std::nullopt}; + if (useConstrainedIntersections()) + { + TypePackId retPack = follow(funTy->retType); + // It is possible for a function to have no annotation and no return statement, and yet still have an ascribed return type + // if it is expected to conform to some other interface. (eg the function may be a lambda passed as a callback) + if (!hasReturn(function.body) && !function.returnAnnotation.has_value() && get(retPack)) + { + auto level = getLevel(retPack); + if (level && scope->level.subsumes(*level)) + *asMutable(retPack) = TypePack{{}, std::nullopt}; + } + } + else + { + // We explicitly don't follow here to check if we have a 'true' free type instead of bound one + if (get_if(&funTy->retType->ty)) + *asMutable(funTy->retType) = TypePack{{}, std::nullopt}; + } bool reachesImplicitReturn = getFallthrough(function.body) != nullptr; @@ -3418,6 +3592,19 @@ void TypeChecker::checkArgumentList( size_t minParams = FFlag::LuauFixIncorrectLineNumberDuplicateType ? 0 : getMinParameterCount_DEPRECATED(paramPack); + auto reportCountMismatchError = [&state, &argLocations, minParams, paramPack, argPack]() { + // For this case, we want the error span to cover every errant extra parameter + Location location = state.location; + if (!argLocations.empty()) + location = {state.location.begin, argLocations.back().end}; + + size_t mp = minParams; + if (FFlag::LuauFixArgumentCountMismatchAmountWithGenericTypes) + mp = getMinParameterCount(&state.log, paramPack); + + state.reportError(TypeError{location, CountMismatch{mp, std::distance(begin(argPack), end(argPack))}}); + }; + while (true) { state.location = paramIndex < argLocations.size() ? argLocations[paramIndex] : state.location; @@ -3472,6 +3659,8 @@ void TypeChecker::checkArgumentList( } else if (auto vtp = state.log.getMutable(tail)) { + // Function is variadic and requires that all subsequent parameters + // be compatible with a type. while (paramIter != endIter) { state.tryUnify(vtp->ty, *paramIter); @@ -3506,14 +3695,22 @@ void TypeChecker::checkArgumentList( else if (state.log.getMutable(t)) { } // ok - else if (!FFlag::LuauAnyInIsOptionalIsOptional && isNonstrictMode() && state.log.getMutable(t)) + else if (!FFlag::LuauAnyInIsOptionalIsOptional && isNonstrictMode() && state.log.get(t)) { } // ok else { if (FFlag::LuauFixArgumentCountMismatchAmountWithGenericTypes) minParams = getMinParameterCount(&state.log, paramPack); - bool isVariadic = FFlag::LuauArgCountMismatchSaysAtLeastWhenVariadic && !finite(paramPack, &state.log); + + bool isVariadic = false; + if (FFlag::LuauArgCountMismatchSaysAtLeastWhenVariadic) + { + std::optional tail = flatten(paramPack, state.log).second; + if (tail) + isVariadic = Luau::isVariadic(*tail); + } + state.reportError(TypeError{state.location, CountMismatch{minParams, paramIndex, CountMismatch::Context::Arg, isVariadic}}); return; } @@ -3532,14 +3729,7 @@ void TypeChecker::checkArgumentList( unify(errorRecoveryType(scope), *argIter, state.location); ++argIter; } - // For this case, we want the error span to cover every errant extra parameter - Location location = state.location; - if (!argLocations.empty()) - location = {state.location.begin, argLocations.back().end}; - - if (FFlag::LuauFixArgumentCountMismatchAmountWithGenericTypes) - minParams = getMinParameterCount(&state.log, paramPack); - state.reportError(TypeError{location, CountMismatch{minParams, std::distance(begin(argPack), end(argPack))}}); + reportCountMismatchError(); return; } TypePackId tail = state.log.follow(*paramIter.tail()); @@ -3551,6 +3741,21 @@ void TypeChecker::checkArgumentList( } else if (auto vtp = state.log.getMutable(tail)) { + if (FFlag::LuauLowerBoundsCalculation && vtp->hidden) + { + // We know that this function can technically be oversaturated, but we have its definition and we + // know that it's useless. + + TypeId e = errorRecoveryType(scope); + while (argIter != endIter) + { + unify(e, *argIter, state.location); + ++argIter; + } + + reportCountMismatchError(); + return; + } // Function is variadic and requires that all subsequent parameters // be compatible with a type. size_t argIndex = paramIndex; @@ -3595,14 +3800,7 @@ void TypeChecker::checkArgumentList( } else if (state.log.getMutable(tail)) { - // For this case, we want the error span to cover every errant extra parameter - Location location = state.location; - if (!argLocations.empty()) - location = {state.location.begin, argLocations.back().end}; - // TODO: Better error message? - if (FFlag::LuauFixArgumentCountMismatchAmountWithGenericTypes) - minParams = getMinParameterCount(&state.log, paramPack); - state.reportError(TypeError{location, CountMismatch{minParams, std::distance(begin(argPack), end(argPack))}}); + reportCountMismatchError(); return; } } @@ -3661,7 +3859,7 @@ ExprResult TypeChecker::checkExprPack(const ScopePtr& scope, const A actualFunctionType = follow(actualFunctionType); TypePackId retPack; - if (!FFlag::LuauWidenIfSupertypeIsFree2) + if (FFlag::LuauLowerBoundsCalculation || !FFlag::LuauWidenIfSupertypeIsFree2) { retPack = freshTypePack(scope->level); } @@ -3809,21 +4007,49 @@ std::optional> TypeChecker::checkCallOverload(const Scope return {{errorRecoveryTypePack(scope)}}; } - if (get(fn)) + if (auto ftv = get(fn)) { // fn is one of the overloads of actualFunctionType, which // has been instantiated, so is a monotype. We can therefore // unify it with a monomorphic function. - TypeId r = addType(FunctionTypeVar(scope->level, argPack, retPack)); - if (FFlag::LuauWidenIfSupertypeIsFree2) + if (useConstrainedIntersections()) { - UnifierOptions options; - options.isFunctionCall = true; - unify(r, fn, expr.location, options); + // This ternary is phrased deliberately. We need ties between sibling scopes to bias toward ftv->level. + const TypeLevel level = scope->level.subsumes(ftv->level) ? scope->level : ftv->level; + + std::vector adjustedArgTypes; + auto it = begin(argPack); + auto endIt = end(argPack); + Widen widen{¤tModule->internalTypes}; + for (; it != endIt; ++it) + { + TypeId t = *it; + TypeId widened = widen.substitute(t).value_or(t); // Surely widening is infallible + adjustedArgTypes.push_back(addType(ConstrainedTypeVar{level, {widened}})); + } + + TypePackId adjustedArgPack = addTypePack(TypePack{std::move(adjustedArgTypes), it.tail()}); + + TxnLog log; + promoteTypeLevels(log, ¤tModule->internalTypes, level, retPack); + log.commit(); + + *asMutable(fn) = FunctionTypeVar{level, adjustedArgPack, retPack}; + return {{retPack}}; } else - unify(fn, r, expr.location); - return {{retPack}}; + { + TypeId r = addType(FunctionTypeVar(scope->level, argPack, retPack)); + if (FFlag::LuauWidenIfSupertypeIsFree2) + { + UnifierOptions options; + options.isFunctionCall = true; + unify(r, fn, expr.location, options); + } + else + unify(fn, r, expr.location); + return {{retPack}}; + } } std::vector metaArgLocations; @@ -4363,10 +4589,17 @@ void TypeChecker::unifyWithInstantiationIfNeeded(const ScopePtr& scope, TypeId s bool Instantiation::isDirty(TypeId ty) { - if (log->getMutable(ty)) + if (const FunctionTypeVar* ftv = log->getMutable(ty)) + { + if (FFlag::LuauTypecheckOptPass && ftv->hasNoGenerics) + return false; + return true; + } else + { return false; + } } bool Instantiation::isDirty(TypePackId tp) @@ -4414,14 +4647,21 @@ TypePackId Instantiation::clean(TypePackId tp) bool ReplaceGenerics::ignoreChildren(TypeId ty) { if (const FunctionTypeVar* ftv = log->getMutable(ty)) + { + if (FFlag::LuauTypecheckOptPass && ftv->hasNoGenerics) + return true; + // We aren't recursing in the case of a generic function which // binds the same generics. This can happen if, for example, there's recursive types. // If T = (a,T)->T then instantiating T should produce T' = (X,T)->T not T' = (X,T')->T'. // It's OK to use vector equality here, since we always generate fresh generics // whenever we quantify, so the vectors overlap if and only if they are equal. return (!generics.empty() || !genericPacks.empty()) && (ftv->generics == generics) && (ftv->genericPacks == genericPacks); + } else + { return false; + } } bool ReplaceGenerics::isDirty(TypeId ty) @@ -4464,16 +4704,24 @@ TypePackId ReplaceGenerics::clean(TypePackId tp) bool Anyification::isDirty(TypeId ty) { + if (ty->persistent) + return false; + if (const TableTypeVar* ttv = log->getMutable(ty)) return (ttv->state == TableState::Free || (FFlag::LuauSealExports && ttv->state == TableState::Unsealed)); else if (log->getMutable(ty)) return true; + else if (get(ty)) + return true; else return false; } bool Anyification::isDirty(TypePackId tp) { + if (tp->persistent) + return false; + if (log->getMutable(tp)) return true; else @@ -4494,7 +4742,16 @@ TypeId Anyification::clean(TypeId ty) clone.syntheticName = ttv->syntheticName; clone.tags = ttv->tags; } - return addType(std::move(clone)); + TypeId res = addType(std::move(clone)); + asMutable(res)->normal = ty->normal; + return res; + } + else if (auto ctv = get(ty)) + { + auto [t, ok] = normalize(ty, *arena, *iceHandler); + if (!ok) + normalizationTooComplex = true; + return t; } else return anyType; @@ -4511,16 +4768,34 @@ TypeId TypeChecker::quantify(const ScopePtr& scope, TypeId ty, Location location ty = follow(ty); const FunctionTypeVar* ftv = get(ty); - if (!ftv || !ftv->generics.empty() || !ftv->genericPacks.empty()) - return ty; + if (ftv && ftv->generics.empty() && ftv->genericPacks.empty()) + Luau::quantify(ty, scope->level); + + if (FFlag::LuauLowerBoundsCalculation && ftv) + { + auto [t, ok] = Luau::normalize(ty, currentModule, *iceHandler); + if (!ok) + reportError(location, NormalizationTooComplex{}); + return t; + } - Luau::quantify(ty, scope->level); return ty; } TypeId TypeChecker::instantiate(const ScopePtr& scope, TypeId ty, Location location, const TxnLog* log) { + if (FFlag::LuauTypecheckOptPass) + { + const FunctionTypeVar* ftv = get(follow(ty)); + if (ftv && ftv->hasNoGenerics) + return ty; + } + Instantiation instantiation{log, ¤tModule->internalTypes, scope->level}; + + if (FFlag::LuauAutocompleteDynamicLimits && instantiationChildLimit) + instantiation.childLimit = *instantiationChildLimit; + std::optional instantiated = instantiation.substitute(ty); if (instantiated.has_value()) return *instantiated; @@ -4533,8 +4808,18 @@ TypeId TypeChecker::instantiate(const ScopePtr& scope, TypeId ty, Location locat TypeId TypeChecker::anyify(const ScopePtr& scope, TypeId ty, Location location) { - Anyification anyification{¤tModule->internalTypes, anyType, anyTypePack}; + if (FFlag::LuauLowerBoundsCalculation) + { + auto [t, ok] = normalize(ty, currentModule, *iceHandler); + if (!ok) + reportError(location, NormalizationTooComplex{}); + ty = t; + } + + Anyification anyification{¤tModule->internalTypes, iceHandler, anyType, anyTypePack}; std::optional any = anyification.substitute(ty); + if (anyification.normalizationTooComplex) + reportError(location, NormalizationTooComplex{}); if (any.has_value()) return *any; else @@ -4546,7 +4831,15 @@ TypeId TypeChecker::anyify(const ScopePtr& scope, TypeId ty, Location location) TypePackId TypeChecker::anyify(const ScopePtr& scope, TypePackId ty, Location location) { - Anyification anyification{¤tModule->internalTypes, anyType, anyTypePack}; + if (FFlag::LuauLowerBoundsCalculation) + { + auto [t, ok] = normalize(ty, currentModule, *iceHandler); + if (!ok) + reportError(location, NormalizationTooComplex{}); + ty = t; + } + + Anyification anyification{¤tModule->internalTypes, iceHandler, anyType, anyTypePack}; std::optional any = anyification.substitute(ty); if (any.has_value()) return *any; @@ -4830,6 +5123,7 @@ TypeId TypeChecker::resolveType(const ScopePtr& scope, const AstType& annotation ToStringOptions opts; opts.exhaustive = true; opts.maxTableLength = 0; + opts.useLineBreaks = true; TypeId param = resolveType(scope, *lit->parameters.data[0].type); luauPrintLine(format("_luau_print\t%s\t|\t%s", toString(param, opts).c_str(), toString(lit->location).c_str())); @@ -5283,7 +5577,7 @@ TypeId TypeChecker::instantiateTypeFun(const ScopePtr& scope, const TypeFun& tf, bool needsClone = follow(tf.type) == target; bool shouldMutate = (!FFlag::LuauOnlyMutateInstantiatedTables || getTableType(tf.type)); TableTypeVar* ttv = getMutableTableType(target); - + if (shouldMutate && ttv && needsClone) { // Substitution::clone is a shallow clone. If this is a metatable type, we @@ -5487,25 +5781,82 @@ std::optional TypeChecker::resolveLValue(const ScopePtr& scope, const LV // We need to search in the provided Scope. Find t.x.y first. // We fail to find t.x.y. Try t.x. We found it. Now we must return the type of the property y from the mapped-to type of t.x. // If we completely fail to find the Symbol t but the Scope has that entry, then we should walk that all the way through and terminate. - const auto& [symbol, keys] = getFullName(lvalue); + if (!FFlag::LuauTypecheckOptPass) + { + const auto& [symbol, keys] = getFullName(lvalue); + + ScopePtr currentScope = scope; + while (currentScope) + { + std::optional found; + + std::vector childKeys; + const LValue* currentLValue = &lvalue; + while (currentLValue) + { + if (auto it = currentScope->refinements.find(*currentLValue); it != currentScope->refinements.end()) + { + found = it->second; + break; + } + + childKeys.push_back(*currentLValue); + currentLValue = baseof(*currentLValue); + } + + if (!found) + { + // Should not be using scope->lookup. This is already recursive. + if (auto it = currentScope->bindings.find(symbol); it != currentScope->bindings.end()) + found = it->second.typeId; + else + { + // Nothing exists in this Scope. Just skip and try the parent one. + currentScope = currentScope->parent; + continue; + } + } + + for (auto it = childKeys.rbegin(); it != childKeys.rend(); ++it) + { + const LValue& key = *it; + + // Symbol can happen. Skip. + if (get(key)) + continue; + else if (auto field = get(key)) + { + found = getIndexTypeFromType(scope, *found, field->key, Location(), false); + if (!found) + return std::nullopt; // Turns out this type doesn't have the property at all. We're done. + } + else + LUAU_ASSERT(!"New LValue alternative not handled here."); + } + + return found; + } + + // No entry for it at all. Can happen when LValue root is a global. + return std::nullopt; + } + + const Symbol symbol = getBaseSymbol(lvalue); ScopePtr currentScope = scope; while (currentScope) { std::optional found; - std::vector childKeys; - const LValue* currentLValue = &lvalue; - while (currentLValue) + const LValue* topLValue = nullptr; + + for (topLValue = &lvalue; topLValue; topLValue = baseof(*topLValue)) { - if (auto it = currentScope->refinements.find(*currentLValue); it != currentScope->refinements.end()) + if (auto it = currentScope->refinements.find(*topLValue); it != currentScope->refinements.end()) { found = it->second; break; } - - childKeys.push_back(*currentLValue); - currentLValue = baseof(*currentLValue); } if (!found) @@ -5521,9 +5872,15 @@ std::optional TypeChecker::resolveLValue(const ScopePtr& scope, const LV } } + // We need to walk the l-value path in reverse, so we collect components into a vector + std::vector childKeys; + + for (const LValue* curr = &lvalue; curr != topLValue; curr = baseof(*curr)) + childKeys.push_back(curr); + for (auto it = childKeys.rbegin(); it != childKeys.rend(); ++it) { - const LValue& key = *it; + const LValue& key = **it; // Symbol can happen. Skip. if (get(key)) @@ -5938,6 +6295,11 @@ bool TypeChecker::isNonstrictMode() const return (currentModule->mode == Mode::Nonstrict) || (currentModule->mode == Mode::NoCheck); } +bool TypeChecker::useConstrainedIntersections() const +{ + return FFlag::LuauLowerBoundsCalculation && !isNonstrictMode(); +} + std::vector TypeChecker::unTypePack(const ScopePtr& scope, TypePackId tp, size_t expectedLength, const Location& location) { TypePackId expectedTypePack = addTypePack({}); diff --git a/Analysis/src/TypePack.cpp b/Analysis/src/TypePack.cpp index 5bb05234..30503233 100644 --- a/Analysis/src/TypePack.cpp +++ b/Analysis/src/TypePack.cpp @@ -104,7 +104,7 @@ TypePackIterator begin(TypePackId tp) return TypePackIterator{tp}; } -TypePackIterator begin(TypePackId tp, TxnLog* log) +TypePackIterator begin(TypePackId tp, const TxnLog* log) { return TypePackIterator{tp, log}; } @@ -256,7 +256,7 @@ size_t size(const TypePack& tp, TxnLog* log) return result; } -std::optional first(TypePackId tp) +std::optional first(TypePackId tp, bool ignoreHiddenVariadics) { auto it = begin(tp); auto endIter = end(tp); @@ -266,7 +266,7 @@ std::optional first(TypePackId tp) if (auto tail = it.tail()) { - if (auto vtp = get(*tail)) + if (auto vtp = get(*tail); vtp && (!vtp->hidden || !ignoreHiddenVariadics)) return vtp->ty; } @@ -299,6 +299,46 @@ std::pair, std::optional> flatten(TypePackId tp) return {res, iter.tail()}; } +std::pair, std::optional> flatten(TypePackId tp, const TxnLog& log) +{ + tp = log.follow(tp); + + std::vector flattened; + std::optional tail = std::nullopt; + + TypePackIterator it(tp, &log); + + for (; it != end(tp); ++it) + { + flattened.push_back(*it); + } + + tail = it.tail(); + + return {flattened, tail}; +} + +bool isVariadic(TypePackId tp) +{ + return isVariadic(tp, *TxnLog::empty()); +} + +bool isVariadic(TypePackId tp, const TxnLog& log) +{ + std::optional tail = flatten(tp, log).second; + + if (!tail) + return false; + + if (log.get(*tail)) + return true; + + if (auto vtp = log.get(*tail); vtp && !vtp->hidden) + return true; + + return false; +} + TypePackVar* asMutable(TypePackId tp) { return const_cast(tp); diff --git a/Analysis/src/TypeVar.cpp b/Analysis/src/TypeVar.cpp index dbc412fc..0fbfdbf0 100644 --- a/Analysis/src/TypeVar.cpp +++ b/Analysis/src/TypeVar.cpp @@ -177,7 +177,7 @@ bool maybeString(TypeId ty) if (FFlag::LuauSubtypingAddOptPropsToUnsealedTables) { ty = follow(ty); - + if (isPrim(ty, PrimitiveTypeVar::String) || get(ty)) return true; @@ -366,7 +366,7 @@ bool maybeSingleton(TypeId ty) bool hasLength(TypeId ty, DenseHashSet& seen, int* recursionCount) { - RecursionLimiter _rl(recursionCount, FInt::LuauTypeInferRecursionLimit); + RecursionLimiter _rl(recursionCount, FInt::LuauTypeInferRecursionLimit, "hasLength"); ty = follow(ty); @@ -654,9 +654,9 @@ static TypeVar booleanType_{PrimitiveTypeVar{PrimitiveTypeVar::Boolean}, /*persi static TypeVar threadType_{PrimitiveTypeVar{PrimitiveTypeVar::Thread}, /*persistent*/ true}; static TypeVar trueType_{SingletonTypeVar{BooleanSingleton{true}}, /*persistent*/ true}; static TypeVar falseType_{SingletonTypeVar{BooleanSingleton{false}}, /*persistent*/ true}; -static TypeVar anyType_{AnyTypeVar{}}; -static TypeVar errorType_{ErrorTypeVar{}}; -static TypeVar optionalNumberType_{UnionTypeVar{{&numberType_, &nilType_}}}; +static TypeVar anyType_{AnyTypeVar{}, /*persistent*/ true}; +static TypeVar errorType_{ErrorTypeVar{}, /*persistent*/ true}; +static TypeVar optionalNumberType_{UnionTypeVar{{&numberType_, &nilType_}}, /*persistent*/ true}; static TypePackVar anyTypePack_{VariadicTypePack{&anyType_}, true}; static TypePackVar errorTypePack_{Unifiable::Error{}}; @@ -698,7 +698,7 @@ TypeId SingletonTypes::makeStringMetatable() { const TypeId optionalNumber = arena->addType(UnionTypeVar{{nilType, numberType}}); const TypeId optionalString = arena->addType(UnionTypeVar{{nilType, stringType}}); - const TypeId optionalBoolean = arena->addType(UnionTypeVar{{nilType, &booleanType_}}); + const TypeId optionalBoolean = arena->addType(UnionTypeVar{{nilType, booleanType}}); const TypePackId oneStringPack = arena->addTypePack({stringType}); const TypePackId anyTypePack = arena->addTypePack(TypePackVar{VariadicTypePack{anyType}, true}); @@ -802,6 +802,7 @@ void persist(TypeId ty) continue; asMutable(t)->persistent = true; + asMutable(t)->normal = true; // all persistent types are assumed to be normal if (auto btv = get(t)) queue.push_back(btv->boundTo); @@ -838,6 +839,11 @@ void persist(TypeId ty) for (TypeId opt : itv->parts) queue.push_back(opt); } + else if (auto ctv = get(t)) + { + for (TypeId opt : ctv->parts) + queue.push_back(opt); + } else if (auto mtv = get(t)) { queue.push_back(mtv->table); @@ -899,6 +905,16 @@ TypeLevel* getMutableLevel(TypeId ty) return const_cast(getLevel(ty)); } +std::optional getLevel(TypePackId tp) +{ + tp = follow(tp); + + if (auto ftv = get(tp)) + return ftv->level; + else + return std::nullopt; +} + const Property* lookupClassProp(const ClassTypeVar* cls, const Name& name) { while (cls) diff --git a/Analysis/src/Unifier.cpp b/Analysis/src/Unifier.cpp index 398dc9e2..f9ea58cc 100644 --- a/Analysis/src/Unifier.cpp +++ b/Analysis/src/Unifier.cpp @@ -14,9 +14,12 @@ LUAU_FASTINT(LuauTypeInferRecursionLimit); LUAU_FASTINT(LuauTypeInferTypePackLoopLimit); -LUAU_FASTINTVARIABLE(LuauTypeInferIterationLimit, 2000); +LUAU_FASTINT(LuauTypeInferIterationLimit); +LUAU_FASTFLAG(LuauAutocompleteDynamicLimits) +LUAU_FASTINTVARIABLE(LuauTypeInferLowerBoundsIterationLimit, 2000); LUAU_FASTFLAGVARIABLE(LuauExtendedIndexerError, false); LUAU_FASTFLAGVARIABLE(LuauTableSubtypingVariance2, false); +LUAU_FASTFLAG(LuauLowerBoundsCalculation); LUAU_FASTFLAG(LuauErrorRecoveryType); LUAU_FASTFLAGVARIABLE(LuauSubtypingAddOptPropsToUnsealedTables, false) LUAU_FASTFLAGVARIABLE(LuauWidenIfSupertypeIsFree2, false) @@ -27,6 +30,7 @@ LUAU_FASTFLAGVARIABLE(LuauTxnLogRefreshFunctionPointers, false) LUAU_FASTFLAGVARIABLE(LuauTxnLogDontRetryForIndexers, false) LUAU_FASTFLAGVARIABLE(LuauUnifierCacheErrors, false) LUAU_FASTFLAG(LuauAnyInIsOptionalIsOptional) +LUAU_FASTFLAG(LuauTypecheckOptPass) namespace Luau { @@ -126,7 +130,6 @@ static void promoteTypeLevels(TxnLog& log, const TypeArena* typeArena, TypeLevel visitTypeVarOnce(ty, ptl, seen); } -// TODO: use this and make it static. void promoteTypeLevels(TxnLog& log, const TypeArena* typeArena, TypeLevel minLevel, TypePackId tp) { // Type levels of types from other modules are already global, so we don't need to promote anything inside @@ -305,8 +308,7 @@ static std::optional> getTableMat return std::nullopt; } -Unifier::Unifier(TypeArena* types, Mode mode, const Location& location, Variance variance, UnifierSharedState& sharedState, - TxnLog* parentLog) +Unifier::Unifier(TypeArena* types, Mode mode, const Location& location, Variance variance, UnifierSharedState& sharedState, TxnLog* parentLog) : types(types) , mode(mode) , log(parentLog) @@ -326,6 +328,7 @@ Unifier::Unifier(TypeArena* types, Mode mode, std::vector 0 && FInt::LuauTypeInferIterationLimit < sharedState.counters.iterationCount) + if (FFlag::LuauAutocompleteDynamicLimits) { - reportError(TypeError{location, UnificationTooComplex{}}); - return; + if (sharedState.counters.iterationLimit > 0 && sharedState.counters.iterationLimit < sharedState.counters.iterationCount) + { + reportError(TypeError{location, UnificationTooComplex{}}); + return; + } + } + else + { + if (FInt::LuauTypeInferIterationLimit > 0 && FInt::LuauTypeInferIterationLimit < sharedState.counters.iterationCount) + { + reportError(TypeError{location, UnificationTooComplex{}}); + return; + } } superTy = log.follow(superTy); @@ -354,6 +369,9 @@ void Unifier::tryUnify_(TypeId subTy, TypeId superTy, bool isFunctionCall, bool if (superTy == subTy) return; + if (log.get(superTy)) + return tryUnifyWithConstrainedSuperTypeVar(subTy, superTy); + auto superFree = log.getMutable(superTy); auto subFree = log.getMutable(subTy); @@ -442,7 +460,18 @@ void Unifier::tryUnify_(TypeId subTy, TypeId superTy, bool isFunctionCall, bool if (get(superTy) || get(superTy)) return tryUnifyWithAny(subTy, superTy); - if (get(subTy) || get(subTy)) + if (get(subTy)) + { + if (anyIsTop) + { + reportError(TypeError{location, TypeMismatch{superTy, subTy}}); + return; + } + else + return tryUnifyWithAny(superTy, subTy); + } + + if (get(subTy)) return tryUnifyWithAny(superTy, subTy); bool cacheEnabled; @@ -484,7 +513,9 @@ void Unifier::tryUnify_(TypeId subTy, TypeId superTy, bool isFunctionCall, bool size_t errorCount = errors.size(); - if (const UnionTypeVar* uv = log.getMutable(subTy)) + if (log.get(subTy)) + tryUnifyWithConstrainedSubTypeVar(subTy, superTy); + else if (const UnionTypeVar* uv = log.getMutable(subTy)) { tryUnifyUnionWithType(subTy, uv, superTy); } @@ -946,7 +977,7 @@ struct WeirdIter LUAU_ASSERT(log.getMutable(newTail)); level = log.getMutable(packId)->level; - log.replace(packId, Unifiable::Bound(newTail)); + log.replace(packId, BoundTypePack(newTail)); packId = newTail; pack = log.getMutable(newTail); index = 0; @@ -994,39 +1025,32 @@ void Unifier::tryUnify(TypePackId subTp, TypePackId superTp, bool isFunctionCall tryUnify_(subTp, superTp, isFunctionCall); } -static std::pair, std::optional> logAwareFlatten(TypePackId tp, const TxnLog& log) -{ - tp = log.follow(tp); - - std::vector flattened; - std::optional tail = std::nullopt; - - TypePackIterator it(tp, &log); - - for (; it != end(tp); ++it) - { - flattened.push_back(*it); - } - - tail = it.tail(); - - return {flattened, tail}; -} - /* * This is quite tricky: we are walking two rope-like structures and unifying corresponding elements. * If one is longer than the other, but the short end is free, we grow it to the required length. */ void Unifier::tryUnify_(TypePackId subTp, TypePackId superTp, bool isFunctionCall) { - RecursionLimiter _ra(&sharedState.counters.recursionCount, FInt::LuauTypeInferRecursionLimit); + RecursionLimiter _ra(&sharedState.counters.recursionCount, + FFlag::LuauAutocompleteDynamicLimits ? sharedState.counters.recursionLimit : FInt::LuauTypeInferRecursionLimit, "TypePackId tryUnify_"); ++sharedState.counters.iterationCount; - if (FInt::LuauTypeInferIterationLimit > 0 && FInt::LuauTypeInferIterationLimit < sharedState.counters.iterationCount) + if (FFlag::LuauAutocompleteDynamicLimits) { - reportError(TypeError{location, UnificationTooComplex{}}); - return; + if (sharedState.counters.iterationLimit > 0 && sharedState.counters.iterationLimit < sharedState.counters.iterationCount) + { + reportError(TypeError{location, UnificationTooComplex{}}); + return; + } + } + else + { + if (FInt::LuauTypeInferIterationLimit > 0 && FInt::LuauTypeInferIterationLimit < sharedState.counters.iterationCount) + { + reportError(TypeError{location, UnificationTooComplex{}}); + return; + } } superTp = log.follow(superTp); @@ -1087,8 +1111,8 @@ void Unifier::tryUnify_(TypePackId subTp, TypePackId superTp, bool isFunctionCal // If the size of two heads does not match, but both packs have free tail // We set the sentinel variable to say so to avoid growing it forever. - auto [superTypes, superTail] = logAwareFlatten(superTp, log); - auto [subTypes, subTail] = logAwareFlatten(subTp, log); + auto [superTypes, superTail] = flatten(superTp, log); + auto [subTypes, subTail] = flatten(subTp, log); bool noInfiniteGrowth = (superTypes.size() != subTypes.size()) && (superTail && log.getMutable(*superTail)) && (subTail && log.getMutable(*subTail)); @@ -1165,19 +1189,20 @@ void Unifier::tryUnify_(TypePackId subTp, TypePackId superTp, bool isFunctionCal else { // A union type including nil marks an optional argument - if (superIter.good() && isOptional(*superIter)) + if ((!FFlag::LuauLowerBoundsCalculation || isNonstrictMode()) && superIter.good() && isOptional(*superIter)) { superIter.advance(); continue; } - else if (subIter.good() && isOptional(*subIter)) + else if ((!FFlag::LuauLowerBoundsCalculation || isNonstrictMode()) && subIter.good() && isOptional(*subIter)) { subIter.advance(); continue; } // In nonstrict mode, any also marks an optional argument. - else if (!FFlag::LuauAnyInIsOptionalIsOptional && superIter.good() && isNonstrictMode() && log.getMutable(log.follow(*superIter))) + else if (!FFlag::LuauAnyInIsOptionalIsOptional && superIter.good() && isNonstrictMode() && + log.getMutable(log.follow(*superIter))) { superIter.advance(); continue; @@ -1195,7 +1220,7 @@ void Unifier::tryUnify_(TypePackId subTp, TypePackId superTp, bool isFunctionCal return; } - if (!isFunctionCall && subIter.good()) + if ((!FFlag::LuauLowerBoundsCalculation || isNonstrictMode()) && !isFunctionCall && subIter.good()) { // Sometimes it is ok to pass too many arguments return; @@ -1418,14 +1443,17 @@ void Unifier::tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) if (FFlag::LuauAnyInIsOptionalIsOptional) { - if (subIter == subTable->props.end() && (!FFlag::LuauSubtypingAddOptPropsToUnsealedTables || subTable->state == TableState::Unsealed) && !isOptional(superProp.type)) + if (subIter == subTable->props.end() && + (!FFlag::LuauSubtypingAddOptPropsToUnsealedTables || subTable->state == TableState::Unsealed) && !isOptional(superProp.type)) missingProperties.push_back(propName); } else { bool isAny = log.getMutable(log.follow(superProp.type)); - if (subIter == subTable->props.end() && (!FFlag::LuauSubtypingAddOptPropsToUnsealedTables || subTable->state == TableState::Unsealed) && !isOptional(superProp.type) && !isAny) + if (subIter == subTable->props.end() && + (!FFlag::LuauSubtypingAddOptPropsToUnsealedTables || subTable->state == TableState::Unsealed) && !isOptional(superProp.type) && + !isAny) missingProperties.push_back(propName); } } @@ -1438,8 +1466,7 @@ void Unifier::tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) } // And vice versa if we're invariant - if (variance == Invariant && !superTable->indexer && superTable->state != TableState::Unsealed && - superTable->state != TableState::Free) + if (variance == Invariant && !superTable->indexer && superTable->state != TableState::Unsealed && superTable->state != TableState::Free) { for (const auto& [propName, subProp] : subTable->props) { @@ -1453,7 +1480,8 @@ void Unifier::tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) else { bool isAny = log.is(log.follow(subProp.type)); - if (superIter == superTable->props.end() && (FFlag::LuauSubtypingAddOptPropsToUnsealedTables || (!isOptional(subProp.type) && !isAny))) + if (superIter == superTable->props.end() && + (FFlag::LuauSubtypingAddOptPropsToUnsealedTables || (!isOptional(subProp.type) && !isAny))) extraProperties.push_back(propName); } } @@ -1499,13 +1527,15 @@ void Unifier::tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) if (innerState.errors.empty()) log.concat(std::move(innerState.log)); } - else if (FFlag::LuauAnyInIsOptionalIsOptional && (!FFlag::LuauSubtypingAddOptPropsToUnsealedTables || subTable->state == TableState::Unsealed) && isOptional(prop.type)) + else if (FFlag::LuauAnyInIsOptionalIsOptional && + (!FFlag::LuauSubtypingAddOptPropsToUnsealedTables || subTable->state == TableState::Unsealed) && isOptional(prop.type)) // This is sound because unsealed table types are precise, so `{ p : T } <: { p : T, q : U? }` // since if `t : { p : T }` then we are guaranteed that `t.q` is `nil`. // TODO: if the supertype is written to, the subtype may no longer be precise (alias analysis?) { } - else if ((!FFlag::LuauSubtypingAddOptPropsToUnsealedTables || subTable->state == TableState::Unsealed) && (isOptional(prop.type) || get(follow(prop.type)))) + else if ((!FFlag::LuauSubtypingAddOptPropsToUnsealedTables || subTable->state == TableState::Unsealed) && + (isOptional(prop.type) || get(follow(prop.type)))) // This is sound because unsealed table types are precise, so `{ p : T } <: { p : T, q : U? }` // since if `t : { p : T }` then we are guaranteed that `t.q` is `nil`. // TODO: should isOptional(anyType) be true? @@ -1664,9 +1694,9 @@ void Unifier::tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) if (FFlag::LuauTxnLogDontRetryForIndexers) { - // Changing the indexer can invalidate the table pointers. - superTable = log.getMutable(superTy); - subTable = log.getMutable(subTy); + // Changing the indexer can invalidate the table pointers. + superTable = log.getMutable(superTy); + subTable = log.getMutable(subTy); } else if (FFlag::LuauTxnLogCheckForInvalidation) { @@ -1921,8 +1951,6 @@ void Unifier::tryUnifySealedTables(TypeId subTy, TypeId superTy, bool isIntersec if (!superTable || !subTable) ice("passed non-table types to unifySealedTables"); - Unifier innerState = makeChildUnifier(); - std::vector missingPropertiesInSuper; bool isUnnamedTable = subTable->name == std::nullopt && subTable->syntheticName == std::nullopt; bool errorReported = false; @@ -1944,6 +1972,8 @@ void Unifier::tryUnifySealedTables(TypeId subTy, TypeId superTy, bool isIntersec } } + Unifier innerState = makeChildUnifier(); + // Tables must have exactly the same props and their types must all unify for (const auto& it : superTable->props) { @@ -2376,6 +2406,180 @@ std::optional Unifier::findTablePropertyRespectingMeta(TypeId lhsType, N return Luau::findTablePropertyRespectingMeta(errors, lhsType, name, location); } +void Unifier::tryUnifyWithConstrainedSubTypeVar(TypeId subTy, TypeId superTy) +{ + const ConstrainedTypeVar* subConstrained = get(subTy); + if (!subConstrained) + ice("tryUnifyWithConstrainedSubTypeVar received non-ConstrainedTypeVar subTy!"); + + const std::vector& subTyParts = subConstrained->parts; + + // A | B <: T if A <: T and B <: T + bool failed = false; + std::optional unificationTooComplex; + + const size_t count = subTyParts.size(); + + for (size_t i = 0; i < count; ++i) + { + TypeId type = subTyParts[i]; + Unifier innerState = makeChildUnifier(); + innerState.tryUnify_(type, superTy); + + if (i == count - 1) + log.concat(std::move(innerState.log)); + + ++i; + + if (auto e = hasUnificationTooComplex(innerState.errors)) + unificationTooComplex = e; + + if (!innerState.errors.empty()) + { + failed = true; + break; + } + } + + if (unificationTooComplex) + reportError(*unificationTooComplex); + else if (failed) + reportError(TypeError{location, TypeMismatch{superTy, subTy}}); + else + log.replace(subTy, BoundTypeVar{superTy}); +} + +void Unifier::tryUnifyWithConstrainedSuperTypeVar(TypeId subTy, TypeId superTy) +{ + ConstrainedTypeVar* superC = log.getMutable(superTy); + if (!superC) + ice("tryUnifyWithConstrainedSuperTypeVar received non-ConstrainedTypeVar superTy!"); + + // subTy could be a + // table + // metatable + // class + // function + // primitive + // free + // generic + // intersection + // union + // Do we really just tack it on? I think we might! + // We can certainly do some deduplication. + // Is there any point to deducing Player|Instance when we could just reduce to Instance? + // Is it actually ok to have multiple free types in a single intersection? What if they are later unified into the same type? + // Maybe we do a simplification step during quantification. + + auto it = std::find(superC->parts.begin(), superC->parts.end(), subTy); + if (it != superC->parts.end()) + return; + + superC->parts.push_back(subTy); +} + +void Unifier::unifyLowerBound(TypePackId subTy, TypePackId superTy) +{ + // The duplication between this and regular typepack unification is tragic. + + auto superIter = begin(superTy, &log); + auto superEndIter = end(superTy); + + auto subIter = begin(subTy, &log); + auto subEndIter = end(subTy); + + int count = FInt::LuauTypeInferLowerBoundsIterationLimit; + + for (; subIter != subEndIter; ++subIter) + { + if (0 >= --count) + ice("Internal recursion counter limit exceeded in Unifier::unifyLowerBound"); + + if (superIter != superEndIter) + { + tryUnify_(*subIter, *superIter); + ++superIter; + continue; + } + + if (auto t = superIter.tail()) + { + TypePackId tailPack = follow(*t); + + if (log.get(tailPack)) + occursCheck(tailPack, subTy); + + FreeTypePack* freeTailPack = log.getMutable(tailPack); + if (!freeTailPack) + return; + + TypeLevel level = freeTailPack->level; + + TypePack* tp = getMutable(log.replace(tailPack, TypePack{})); + + for (; subIter != subEndIter; ++subIter) + { + tp->head.push_back(types->addType(ConstrainedTypeVar{level, {follow(*subIter)}})); + } + + tp->tail = subIter.tail(); + } + + return; + } + + if (superIter != superEndIter) + { + if (auto subTail = subIter.tail()) + { + TypePackId subTailPack = follow(*subTail); + if (get(subTailPack)) + { + TypePack* tp = getMutable(log.replace(subTailPack, TypePack{})); + + for (; superIter != superEndIter; ++superIter) + tp->head.push_back(*superIter); + } + } + else + { + while (superIter != superEndIter) + { + if (!isOptional(*superIter)) + { + errors.push_back(TypeError{location, CountMismatch{size(superTy), size(subTy), CountMismatch::Return}}); + return; + } + ++superIter; + } + } + + return; + } + + // Both iters are at their respective tails + auto subTail = subIter.tail(); + auto superTail = superIter.tail(); + if (subTail && superTail) + tryUnify(*subTail, *superTail); + else if (subTail) + { + const FreeTypePack* freeSubTail = log.getMutable(*subTail); + if (freeSubTail) + { + log.replace(*subTail, TypePack{}); + } + } + else if (superTail) + { + const FreeTypePack* freeSuperTail = log.getMutable(*superTail); + if (freeSuperTail) + { + log.replace(*superTail, TypePack{}); + } + } +} + void Unifier::occursCheck(TypeId needle, TypeId haystack) { sharedState.tempSeenTy.clear(); @@ -2385,7 +2589,8 @@ void Unifier::occursCheck(TypeId needle, TypeId haystack) void Unifier::occursCheck(DenseHashSet& seen, TypeId needle, TypeId haystack) { - RecursionLimiter _ra(&sharedState.counters.recursionCount, FInt::LuauTypeInferRecursionLimit); + RecursionLimiter _ra(&sharedState.counters.recursionCount, + FFlag::LuauAutocompleteDynamicLimits ? sharedState.counters.recursionLimit : FInt::LuauTypeInferRecursionLimit, "occursCheck for TypeId"); auto check = [&](TypeId tv) { occursCheck(seen, needle, tv); @@ -2425,6 +2630,11 @@ void Unifier::occursCheck(DenseHashSet& seen, TypeId needle, TypeId hays for (TypeId ty : a->parts) check(ty); } + else if (auto a = log.getMutable(haystack)) + { + for (TypeId ty : a->parts) + check(ty); + } } void Unifier::occursCheck(TypePackId needle, TypePackId haystack) @@ -2450,7 +2660,8 @@ void Unifier::occursCheck(DenseHashSet& seen, TypePackId needle, Typ if (!log.getMutable(needle)) ice("Expected needle pack to be free"); - RecursionLimiter _ra(&sharedState.counters.recursionCount, FInt::LuauTypeInferRecursionLimit); + RecursionLimiter _ra(&sharedState.counters.recursionCount, + FFlag::LuauAutocompleteDynamicLimits ? sharedState.counters.recursionLimit : FInt::LuauTypeInferRecursionLimit, "occursCheck for TypePackId"); while (!log.getMutable(haystack)) { @@ -2474,7 +2685,23 @@ void Unifier::occursCheck(DenseHashSet& seen, TypePackId needle, Typ Unifier Unifier::makeChildUnifier() { - return Unifier{types, mode, log.sharedSeen, location, variance, sharedState, &log}; + if (FFlag::LuauTypecheckOptPass) + { + Unifier u = Unifier{types, mode, location, variance, sharedState, &log}; + u.anyIsTop = anyIsTop; + return u; + } + + Unifier u = Unifier{types, mode, log.sharedSeen, location, variance, sharedState, &log}; + u.anyIsTop = anyIsTop; + return u; +} + +// A utility function that appends the given error to the unifier's error log. +// This allows setting a breakpoint wherever the unifier reports an error. +void Unifier::reportError(TypeError err) +{ + errors.push_back(std::move(err)); } bool Unifier::isNonstrictMode() const diff --git a/Ast/include/Luau/DenseHash.h b/Ast/include/Luau/DenseHash.h index 65939bee..f8543111 100644 --- a/Ast/include/Luau/DenseHash.h +++ b/Ast/include/Luau/DenseHash.h @@ -32,6 +32,7 @@ class DenseHashTable { public: class const_iterator; + class iterator; DenseHashTable(const Key& empty_key, size_t buckets = 0) : count(0) @@ -43,7 +44,7 @@ public: // don't move this to initializer list! this works around an MSVC codegen issue on AMD CPUs: // https://developercommunity.visualstudio.com/t/stdvector-constructor-from-size-t-is-25-times-slow/1546547 if (buckets) - data.resize(buckets, ItemInterface::create(empty_key)); + resize_data(buckets); } void clear() @@ -125,7 +126,7 @@ public: if (data.empty() && data.capacity() >= newsize) { LUAU_ASSERT(count == 0); - data.resize(newsize, ItemInterface::create(empty_key)); + resize_data(newsize); return; } @@ -169,6 +170,21 @@ public: return const_iterator(this, data.size()); } + iterator begin() + { + size_t start = 0; + + while (start < data.size() && eq(ItemInterface::getKey(data[start]), empty_key)) + start++; + + return iterator(this, start); + } + + iterator end() + { + return iterator(this, data.size()); + } + size_t size() const { return count; @@ -233,7 +249,82 @@ public: size_t index; }; + class iterator + { + public: + iterator() + : set(0) + , index(0) + { + } + + iterator(DenseHashTable* set, size_t index) + : set(set) + , index(index) + { + } + + MutableItem& operator*() const + { + return *reinterpret_cast(&set->data[index]); + } + + MutableItem* operator->() const + { + return reinterpret_cast(&set->data[index]); + } + + bool operator==(const iterator& other) const + { + return set == other.set && index == other.index; + } + + bool operator!=(const iterator& other) const + { + return set != other.set || index != other.index; + } + + iterator& operator++() + { + size_t size = set->data.size(); + + do + { + index++; + } while (index < size && set->eq(ItemInterface::getKey(set->data[index]), set->empty_key)); + + return *this; + } + + iterator operator++(int) + { + iterator res = *this; + ++*this; + return res; + } + + private: + DenseHashTable* set; + size_t index; + }; + private: + template + void resize_data(size_t count, typename std::enable_if_t>* dummy = nullptr) + { + data.resize(count, ItemInterface::create(empty_key)); + } + + template + void resize_data(size_t count, typename std::enable_if_t>* dummy = nullptr) + { + size_t size = data.size(); + data.resize(count); + + for (size_t i = size; i < count; i++) + data[i].first = empty_key; + } + std::vector data; size_t count; Key empty_key; @@ -290,6 +381,7 @@ class DenseHashSet public: typedef typename Impl::const_iterator const_iterator; + typedef typename Impl::iterator iterator; DenseHashSet(const Key& empty_key, size_t buckets = 0) : impl(empty_key, buckets) @@ -336,6 +428,16 @@ public: { return impl.end(); } + + iterator begin() + { + return impl.begin(); + } + + iterator end() + { + return impl.end(); + } }; // This is a faster alternative of unordered_map, but it does not implement the same interface (i.e. it does not support erasing and has @@ -348,6 +450,7 @@ class DenseHashMap public: typedef typename Impl::const_iterator const_iterator; + typedef typename Impl::iterator iterator; DenseHashMap(const Key& empty_key, size_t buckets = 0) : impl(empty_key, buckets) @@ -401,10 +504,21 @@ public: { return impl.begin(); } + const_iterator end() const { return impl.end(); } + + iterator begin() + { + return impl.begin(); + } + + iterator end() + { + return impl.end(); + } }; } // namespace Luau diff --git a/Ast/include/Luau/Lexer.h b/Ast/include/Luau/Lexer.h index d7d867f4..4f3dbbd5 100644 --- a/Ast/include/Luau/Lexer.h +++ b/Ast/include/Luau/Lexer.h @@ -173,7 +173,7 @@ public: } const Lexeme& next(); - const Lexeme& next(bool skipComments); + const Lexeme& next(bool skipComments, bool updatePrevLocation); void nextline(); Lexeme lookahead(); diff --git a/Ast/src/Lexer.cpp b/Ast/src/Lexer.cpp index 70c6c78d..5dd4f04e 100644 --- a/Ast/src/Lexer.cpp +++ b/Ast/src/Lexer.cpp @@ -349,13 +349,11 @@ void Lexer::setReadNames(bool read) const Lexeme& Lexer::next() { - return next(this->skipComments); + return next(this->skipComments, true); } -const Lexeme& Lexer::next(bool skipComments) +const Lexeme& Lexer::next(bool skipComments, bool updatePrevLocation) { - bool first = true; - // in skipComments mode we reject valid comments do { @@ -363,11 +361,11 @@ const Lexeme& Lexer::next(bool skipComments) while (isSpace(peekch())) consume(); - if (!FFlag::LuauParseLocationIgnoreCommentSkip || first) + if (!FFlag::LuauParseLocationIgnoreCommentSkip || updatePrevLocation) prevLocation = lexeme.location; lexeme = readNext(); - first = false; + updatePrevLocation = false; } while (skipComments && (lexeme.type == Lexeme::Comment || lexeme.type == Lexeme::BlockComment)); return lexeme; diff --git a/Ast/src/Parser.cpp b/Ast/src/Parser.cpp index f9d32178..badd3fd3 100644 --- a/Ast/src/Parser.cpp +++ b/Ast/src/Parser.cpp @@ -11,6 +11,7 @@ LUAU_FASTINTVARIABLE(LuauRecursionLimit, 1000) LUAU_FASTINTVARIABLE(LuauParseErrorLimit, 100) LUAU_FASTFLAGVARIABLE(LuauParseRecoverUnexpectedPack, false) +LUAU_FASTFLAGVARIABLE(LuauParseLocationIgnoreCommentSkipInCapture, false) namespace Luau { @@ -2789,7 +2790,7 @@ void Parser::nextLexeme() { if (options.captureComments) { - Lexeme::Type type = lexer.next(/* skipComments= */ false).type; + Lexeme::Type type = lexer.next(/* skipComments= */ false, true).type; while (type == Lexeme::BrokenComment || type == Lexeme::Comment || type == Lexeme::BlockComment) { @@ -2813,7 +2814,7 @@ void Parser::nextLexeme() hotcomments.push_back({hotcommentHeader, lexeme.location, std::string(text + 1, text + end)}); } - type = lexer.next(/* skipComments= */ false).type; + type = lexer.next(/* skipComments= */ false, !FFlag::LuauParseLocationIgnoreCommentSkipInCapture).type; } } else diff --git a/Compiler/src/Compiler.cpp b/Compiler/src/Compiler.cpp index 6330bf1f..8ef69e75 100644 --- a/Compiler/src/Compiler.cpp +++ b/Compiler/src/Compiler.cpp @@ -1386,8 +1386,8 @@ struct Compiler const Constant* cv = constants.find(expr->index); - if (cv && cv->type == Constant::Type_Number && double(int(cv->valueNumber)) == cv->valueNumber && cv->valueNumber >= 1 && - cv->valueNumber <= 256) + if (cv && cv->type == Constant::Type_Number && cv->valueNumber >= 1 && cv->valueNumber <= 256 && + double(int(cv->valueNumber)) == cv->valueNumber) { uint8_t rt = compileExprAuto(expr->expr, rs); uint8_t i = uint8_t(int(cv->valueNumber) - 1); diff --git a/Compiler/src/CostModel.cpp b/Compiler/src/CostModel.cpp new file mode 100644 index 00000000..d8511bdb --- /dev/null +++ b/Compiler/src/CostModel.cpp @@ -0,0 +1,258 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +#include "CostModel.h" + +#include "Luau/Common.h" +#include "Luau/DenseHash.h" + +namespace Luau +{ +namespace Compile +{ + +inline uint64_t parallelAddSat(uint64_t x, uint64_t y) +{ + uint64_t s = x + y; + uint64_t m = s & 0x8080808080808080ull; // saturation mask + + return (s ^ m) | (m - (m >> 7)); +} + +struct Cost +{ + static const uint64_t kLiteral = ~0ull; + + // cost model: 8 bytes, where first byte is the baseline cost, and the next 7 bytes are discounts for when variable #i is constant + uint64_t model; + // constant mask: 8-byte 0xff mask; equal to all ff's for literals, for variables only byte #i (1+) is set to align with model + uint64_t constant; + + Cost(int cost = 0, uint64_t constant = 0) + : model(cost < 0x7f ? cost : 0x7f) + , constant(constant) + { + } + + Cost operator+(const Cost& other) const + { + Cost result; + result.model = parallelAddSat(model, other.model); + return result; + } + + Cost& operator+=(const Cost& other) + { + model = parallelAddSat(model, other.model); + constant = 0; + return *this; + } + + static Cost fold(const Cost& x, const Cost& y) + { + uint64_t newmodel = parallelAddSat(x.model, y.model); + uint64_t newconstant = x.constant & y.constant; + + // the extra cost for folding is 1; the discount is 1 for the variable that is shared by x&y (or whichever one is used in x/y if the other is + // literal) + uint64_t extra = (newconstant == kLiteral) ? 0 : (1 | (0x0101010101010101ull & newconstant)); + + Cost result; + result.model = parallelAddSat(newmodel, extra); + result.constant = newconstant; + + return result; + } +}; + +struct CostVisitor : AstVisitor +{ + DenseHashMap vars; + Cost result; + + CostVisitor() + : vars(nullptr) + { + } + + Cost model(AstExpr* node) + { + if (AstExprGroup* expr = node->as()) + { + return model(expr->expr); + } + else if (node->is() || node->is() || node->is() || + node->is()) + { + return Cost(0, Cost::kLiteral); + } + else if (AstExprLocal* expr = node->as()) + { + const uint64_t* i = vars.find(expr->local); + + return Cost(0, i ? *i : 0); // locals typically don't require extra instructions to compute + } + else if (node->is()) + { + return 1; + } + else if (node->is()) + { + return 3; + } + else if (AstExprCall* expr = node->as()) + { + Cost cost = 3; + cost += model(expr->func); + + for (size_t i = 0; i < expr->args.size; ++i) + { + Cost ac = model(expr->args.data[i]); + // for constants/locals we still need to copy them to the argument list + cost += ac.model == 0 ? Cost(1) : ac; + } + + return cost; + } + else if (AstExprIndexName* expr = node->as()) + { + return model(expr->expr) + 1; + } + else if (AstExprIndexExpr* expr = node->as()) + { + return model(expr->expr) + model(expr->index) + 1; + } + else if (AstExprFunction* expr = node->as()) + { + return 10; // high baseline cost due to allocation + } + else if (AstExprTable* expr = node->as()) + { + Cost cost = 10; // high baseline cost due to allocation + + for (size_t i = 0; i < expr->items.size; ++i) + { + const AstExprTable::Item& item = expr->items.data[i]; + + if (item.key) + cost += model(item.key); + + cost += model(item.value); + cost += 1; + } + + return cost; + } + else if (AstExprUnary* expr = node->as()) + { + return Cost::fold(model(expr->expr), Cost(0, Cost::kLiteral)); + } + else if (AstExprBinary* expr = node->as()) + { + return Cost::fold(model(expr->left), model(expr->right)); + } + else if (AstExprTypeAssertion* expr = node->as()) + { + return model(expr->expr); + } + else if (AstExprIfElse* expr = node->as()) + { + return model(expr->condition) + model(expr->trueExpr) + model(expr->falseExpr) + 2; + } + else + { + LUAU_ASSERT(!"Unknown expression type"); + return {}; + } + } + + void assign(AstExpr* expr) + { + // variable assignments reset variable mask, so that further uses of this variable aren't discounted + // this doesn't work perfectly with backwards control flow like loops, but is good enough for a single pass + if (AstExprLocal* lv = expr->as()) + if (uint64_t* i = vars.find(lv->local)) + *i = 0; + } + + bool visit(AstExpr* node) override + { + // note: we short-circuit the visitor traversal through any expression trees by returning false + // recursive traversal is happening inside model() which makes it easier to get the resulting value of the subexpression + result += model(node); + + return false; + } + + bool visit(AstStat* node) override + { + if (node->is()) + result += 2; + else if (node->is() || node->is() || node->is() || node->is()) + result += 2; + else if (node->is() || node->is()) + result += 1; + + return true; + } + + bool visit(AstStatLocal* node) override + { + for (size_t i = 0; i < node->values.size; ++i) + { + Cost arg = model(node->values.data[i]); + + // propagate constant mask from expression through variables + if (arg.constant && i < node->vars.size) + vars[node->vars.data[i]] = arg.constant; + + result += arg; + } + + return false; + } + + bool visit(AstStatAssign* node) override + { + for (size_t i = 0; i < node->vars.size; ++i) + assign(node->vars.data[i]); + + return true; + } + + bool visit(AstStatCompoundAssign* node) override + { + assign(node->var); + + // if lhs is not a local, setting it requires an extra table operation + result += node->var->is() ? 1 : 2; + + return true; + } +}; + +uint64_t modelCost(AstNode* root, AstLocal* const* vars, size_t varCount) +{ + CostVisitor visitor; + for (size_t i = 0; i < varCount && i < 7; ++i) + visitor.vars[vars[i]] = 0xffull << (i * 8 + 8); + + root->visit(&visitor); + + return visitor.result.model; +} + +int computeCost(uint64_t model, const bool* varsConst, size_t varCount) +{ + int cost = int(model & 0x7f); + + // don't apply discounts to what is likely a saturated sum + if (cost == 0x7f) + return cost; + + for (size_t i = 0; i < varCount && i < 7; ++i) + cost -= int((model >> (8 * i + 8)) & 0x7f) * varsConst[i]; + + return cost; +} + +} // namespace Compile +} // namespace Luau diff --git a/Compiler/src/CostModel.h b/Compiler/src/CostModel.h new file mode 100644 index 00000000..c27861ec --- /dev/null +++ b/Compiler/src/CostModel.h @@ -0,0 +1,18 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +#pragma once + +#include "Luau/Ast.h" + +namespace Luau +{ +namespace Compile +{ + +// cost model: 8 bytes, where first byte is the baseline cost, and the next 7 bytes are discounts for when variable #i is constant +uint64_t modelCost(AstNode* root, AstLocal* const* vars, size_t varCount); + +// cost is computed as B - sum(Di * Ci), where B is baseline cost, Di is the discount for each variable and Ci is 1 when variable #i is constant +int computeCost(uint64_t model, const bool* varsConst, size_t varCount); + +} // namespace Compile +} // namespace Luau diff --git a/Sources.cmake b/Sources.cmake index 6f110f1f..60e5dfda 100644 --- a/Sources.cmake +++ b/Sources.cmake @@ -32,11 +32,13 @@ target_sources(Luau.Compiler PRIVATE Compiler/src/Compiler.cpp Compiler/src/Builtins.cpp Compiler/src/ConstantFolding.cpp + Compiler/src/CostModel.cpp Compiler/src/TableShape.cpp Compiler/src/ValueTracking.cpp Compiler/src/lcode.cpp Compiler/src/Builtins.h Compiler/src/ConstantFolding.h + Compiler/src/CostModel.h Compiler/src/TableShape.h Compiler/src/ValueTracking.h ) @@ -58,6 +60,7 @@ target_sources(Luau.Analysis PRIVATE Analysis/include/Luau/LValue.h Analysis/include/Luau/Module.h Analysis/include/Luau/ModuleResolver.h + Analysis/include/Luau/Normalize.h Analysis/include/Luau/Predicate.h Analysis/include/Luau/Quantify.h Analysis/include/Luau/RecursionCounter.h @@ -94,6 +97,7 @@ target_sources(Luau.Analysis PRIVATE Analysis/src/Linter.cpp Analysis/src/LValue.cpp Analysis/src/Module.cpp + Analysis/src/Normalize.cpp Analysis/src/Quantify.cpp Analysis/src/RequireTracer.cpp Analysis/src/Scope.cpp @@ -216,6 +220,7 @@ if(TARGET Luau.UnitTest) tests/Autocomplete.test.cpp tests/BuiltinDefinitions.test.cpp tests/Compiler.test.cpp + tests/CostModel.test.cpp tests/Config.test.cpp tests/Error.test.cpp tests/Frontend.test.cpp @@ -224,6 +229,7 @@ if(TARGET Luau.UnitTest) tests/LValue.test.cpp tests/Module.test.cpp tests/NonstrictMode.test.cpp + tests/Normalize.test.cpp tests/Parser.test.cpp tests/RequireTracer.test.cpp tests/StringUtils.test.cpp diff --git a/VM/src/ltable.cpp b/VM/src/ltable.cpp index 1c75c0b0..dc40b6ef 100644 --- a/VM/src/ltable.cpp +++ b/VM/src/ltable.cpp @@ -34,7 +34,7 @@ #include LUAU_FASTFLAGVARIABLE(LuauTableRehashRework, false) -LUAU_FASTFLAGVARIABLE(LuauTableNewBoundary, false) +LUAU_FASTFLAGVARIABLE(LuauTableNewBoundary2, false) // max size of both array and hash part is 2^MAXBITS #define MAXBITS 26 @@ -390,6 +390,8 @@ static void resize(lua_State* L, Table* t, int nasize, int nhsize) setarrayvector(L, t, nasize); /* create new hash part with appropriate size */ setnodevector(L, t, nhsize); + /* used for the migration check at the end */ + LuaNode* nnew = t->node; if (nasize < oldasize) { /* array part must shrink? */ t->sizearray = nasize; @@ -413,6 +415,8 @@ static void resize(lua_State* L, Table* t, int nasize, int nhsize) /* shrink array */ luaM_reallocarray(L, t->array, oldasize, nasize, TValue, t->memcat); } + /* used for the migration check at the end */ + TValue* anew = t->array; /* re-insert elements from hash part */ if (FFlag::LuauTableRehashRework) { @@ -441,14 +445,30 @@ static void resize(lua_State* L, Table* t, int nasize, int nhsize) } } + /* make sure we haven't recursively rehashed during element migration */ + LUAU_ASSERT(nnew == t->node); + LUAU_ASSERT(anew == t->array); + if (nold != dummynode) luaM_freearray(L, nold, twoto(oldhsize), LuaNode, t->memcat); /* free old array */ } +static int adjustasize(Table* t, int size, const TValue* ek) +{ + LUAU_ASSERT(FFlag::LuauTableNewBoundary2); + bool tbound = t->node != dummynode || size < t->sizearray; + int ekindex = ek && ttisnumber(ek) ? arrayindex(nvalue(ek)) : -1; + /* move the array size up until the boundary is guaranteed to be inside the array part */ + while (size + 1 == ekindex || (tbound && !ttisnil(luaH_getnum(t, size + 1)))) + size++; + return size; +} + void luaH_resizearray(lua_State* L, Table* t, int nasize) { int nsize = (t->node == dummynode) ? 0 : sizenode(t); - resize(L, t, nasize, nsize); + int asize = FFlag::LuauTableNewBoundary2 ? adjustasize(t, nasize, NULL) : nasize; + resize(L, t, asize, nsize); } void luaH_resizehash(lua_State* L, Table* t, int nhsize) @@ -470,21 +490,12 @@ static void rehash(lua_State* L, Table* t, const TValue* ek) totaluse++; /* compute new size for array part */ int na = computesizes(nums, &nasize); + int nh = totaluse - na; /* enforce the boundary invariant; for performance, only do hash lookups if we must */ - if (FFlag::LuauTableNewBoundary) - { - bool tbound = t->node != dummynode || nasize < t->sizearray; - int ekindex = ttisnumber(ek) ? arrayindex(nvalue(ek)) : -1; - /* move the array size up until the boundary is guaranteed to be inside the array part */ - while (nasize + 1 == ekindex || (tbound && !ttisnil(luaH_getnum(t, nasize + 1)))) - { - nasize++; - na++; - } - } + if (FFlag::LuauTableNewBoundary2) + nasize = adjustasize(t, nasize, ek); /* resize the table to new computed sizes */ - LUAU_ASSERT(na <= totaluse); - resize(L, t, nasize, totaluse - na); + resize(L, t, nasize, nh); } /* @@ -544,7 +555,7 @@ static LuaNode* getfreepos(Table* t) static TValue* newkey(lua_State* L, Table* t, const TValue* key) { /* enforce boundary invariant */ - if (FFlag::LuauTableNewBoundary && ttisnumber(key) && nvalue(key) == t->sizearray + 1) + if (FFlag::LuauTableNewBoundary2 && ttisnumber(key) && nvalue(key) == t->sizearray + 1) { rehash(L, t, key); /* grow table */ @@ -735,7 +746,7 @@ TValue* luaH_setstr(lua_State* L, Table* t, TString* key) static LUAU_NOINLINE int unbound_search(Table* t, unsigned int j) { - LUAU_ASSERT(!FFlag::LuauTableNewBoundary); + LUAU_ASSERT(!FFlag::LuauTableNewBoundary2); unsigned int i = j; /* i is zero or a present index */ j++; /* find `i' and `j' such that i is present and j is not */ @@ -820,7 +831,7 @@ int luaH_getn(Table* t) maybesetaboundary(t, boundary); return boundary; } - else if (FFlag::LuauTableNewBoundary) + else if (FFlag::LuauTableNewBoundary2) { /* validate boundary invariant */ LUAU_ASSERT(t->node == dummynode || ttisnil(luaH_getnum(t, j + 1))); diff --git a/VM/src/ltablib.cpp b/VM/src/ltablib.cpp index 41887f4b..9c1f387e 100644 --- a/VM/src/ltablib.cpp +++ b/VM/src/ltablib.cpp @@ -199,7 +199,7 @@ static int tmove(lua_State* L) int tt = !lua_isnoneornil(L, 5) ? 5 : 1; /* destination table */ luaL_checktype(L, tt, LUA_TTABLE); - void (*telemetrycb)(lua_State* L, int f, int e, int t, int nf, int nt) = lua_table_move_telemetry; + void (*telemetrycb)(lua_State * L, int f, int e, int t, int nf, int nt) = lua_table_move_telemetry; if (DFFlag::LuauTableMoveTelemetry2 && telemetrycb && e >= f) { diff --git a/VM/src/lvmexecute.cpp b/VM/src/lvmexecute.cpp index 34949efb..39c60eac 100644 --- a/VM/src/lvmexecute.cpp +++ b/VM/src/lvmexecute.cpp @@ -16,7 +16,7 @@ #include -LUAU_FASTFLAG(LuauTableNewBoundary) +LUAU_FASTFLAG(LuauTableNewBoundary2) // Disable c99-designator to avoid the warning in CGOTO dispatch table #ifdef __clang__ @@ -2268,7 +2268,7 @@ static void luau_execute(lua_State* L) VM_NEXT(); } } - else if (FFlag::LuauTableNewBoundary || (h->lsizenode == 0 && ttisnil(gval(h->node)))) + else if (FFlag::LuauTableNewBoundary2 || (h->lsizenode == 0 && ttisnil(gval(h->node)))) { // fallthrough to exit VM_NEXT(); diff --git a/tests/CostModel.test.cpp b/tests/CostModel.test.cpp new file mode 100644 index 00000000..ec04932f --- /dev/null +++ b/tests/CostModel.test.cpp @@ -0,0 +1,101 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +#include "Luau/Parser.h" + +#include "doctest.h" + +using namespace Luau; + +namespace Luau +{ +namespace Compile +{ + +uint64_t modelCost(AstNode* root, AstLocal* const* vars, size_t varCount); +int computeCost(uint64_t model, const bool* varsConst, size_t varCount); + +} // namespace Compile +} // namespace Luau + +TEST_SUITE_BEGIN("CostModel"); + +static uint64_t modelFunction(const char* source) +{ + Allocator allocator; + AstNameTable names(allocator); + + ParseResult result = Parser::parse(source, strlen(source), names, allocator); + REQUIRE(result.root != nullptr); + + AstStatFunction* func = result.root->body.data[0]->as(); + REQUIRE(func); + + return Luau::Compile::modelCost(func->func->body, func->func->args.data, func->func->args.size); +} + +TEST_CASE("Expression") +{ + uint64_t model = modelFunction(R"( +function test(a, b, c) + return a + (b + 1) * (b + 1) - c +end +)"); + + const bool args1[] = {false, false, false}; + const bool args2[] = {false, true, false}; + + CHECK_EQ(5, Luau::Compile::computeCost(model, args1, 3)); + CHECK_EQ(2, Luau::Compile::computeCost(model, args2, 3)); +} + +TEST_CASE("PropagateVariable") +{ + uint64_t model = modelFunction(R"( +function test(a) + local b = a * a * a + return b * b +end +)"); + + const bool args1[] = {false}; + const bool args2[] = {true}; + + CHECK_EQ(3, Luau::Compile::computeCost(model, args1, 1)); + CHECK_EQ(0, Luau::Compile::computeCost(model, args2, 1)); +} + +TEST_CASE("LoopAssign") +{ + uint64_t model = modelFunction(R"( +function test(a) + for i=1,3 do + a[i] = i + end +end +)"); + + const bool args1[] = {false}; + const bool args2[] = {true}; + + // loop baseline cost is 2 + CHECK_EQ(3, Luau::Compile::computeCost(model, args1, 1)); + CHECK_EQ(3, Luau::Compile::computeCost(model, args2, 1)); +} + +TEST_CASE("MutableVariable") +{ + uint64_t model = modelFunction(R"( +function test(a, b) + local x = a * a + x += b + return x * x +end +)"); + + const bool args1[] = {false}; + const bool args2[] = {true}; + + CHECK_EQ(3, Luau::Compile::computeCost(model, args1, 1)); + CHECK_EQ(2, Luau::Compile::computeCost(model, args2, 1)); +} + +TEST_SUITE_END(); diff --git a/tests/Fixture.cpp b/tests/Fixture.cpp index 9dc9feee..d8b37a65 100644 --- a/tests/Fixture.cpp +++ b/tests/Fixture.cpp @@ -231,7 +231,7 @@ ModulePtr Fixture::getMainModule() SourceModule* Fixture::getMainSourceModule() { - return frontend.getSourceModule(fromString("MainModule")); + return frontend.getSourceModule(fromString(mainModuleName)); } std::optional Fixture::getPrimitiveType(TypeId ty) @@ -259,7 +259,7 @@ std::optional Fixture::getType(const std::string& name) TypeId Fixture::requireType(const std::string& name) { std::optional ty = getType(name); - REQUIRE(bool(ty)); + REQUIRE_MESSAGE(bool(ty), "Unable to requireType \"" << name << "\""); return follow(*ty); } diff --git a/tests/JsonEncoder.test.cpp b/tests/JsonEncoder.test.cpp index 6711d979..1d2ad645 100644 --- a/tests/JsonEncoder.test.cpp +++ b/tests/JsonEncoder.test.cpp @@ -68,7 +68,9 @@ TEST_CASE("encode_tables") REQUIRE(parseResult.errors.size() == 0); std::string json = toJson(parseResult.root); - CHECK(json == R"({"type":"AstStatBlock","location":"0,0 - 6,4","body":[{"type":"AstStatLocal","location":"1,8 - 5,9","vars":[{"type":{"type":"AstTypeTable","location":"1,17 - 3,9","props":[{"name":"foo","location":"2,12 - 2,15","type":{"type":"AstTypeReference","location":"2,17 - 2,23","name":"number","parameters":[]}}],"indexer":false},"name":"x","location":"1,14 - 1,15"}],"values":[{"type":"AstExprTable","location":"3,12 - 5,9","items":[{"kind":"record","key":{"type":"AstExprConstantString","location":"4,12 - 4,15","value":"foo"},"value":{"type":"AstExprConstantNumber","location":"4,18 - 4,21","value":123}}]}]}]})"); + CHECK( + json == + R"({"type":"AstStatBlock","location":"0,0 - 6,4","body":[{"type":"AstStatLocal","location":"1,8 - 5,9","vars":[{"type":{"type":"AstTypeTable","location":"1,17 - 3,9","props":[{"name":"foo","location":"2,12 - 2,15","type":{"type":"AstTypeReference","location":"2,17 - 2,23","name":"number","parameters":[]}}],"indexer":false},"name":"x","location":"1,14 - 1,15"}],"values":[{"type":"AstExprTable","location":"3,12 - 5,9","items":[{"kind":"record","key":{"type":"AstExprConstantString","location":"4,12 - 4,15","value":"foo"},"value":{"type":"AstExprConstantNumber","location":"4,18 - 4,21","value":123}}]}]}]})"); } TEST_SUITE_END(); diff --git a/tests/Linter.test.cpp b/tests/Linter.test.cpp index 9ce9a4c2..05ee9a7b 100644 --- a/tests/Linter.test.cpp +++ b/tests/Linter.test.cpp @@ -597,8 +597,6 @@ return foo1 TEST_CASE_FIXTURE(Fixture, "UnknownType") { - ScopedFastFlag sff("LuauLintNoRobloxBits", true); - unfreeze(typeChecker.globalTypes); TableTypeVar::Props instanceProps{ {"ClassName", {typeChecker.anyType}}, @@ -1439,6 +1437,7 @@ TEST_CASE_FIXTURE(Fixture, "DeprecatedApi") { unfreeze(typeChecker.globalTypes); TypeId instanceType = typeChecker.globalTypes.addType(ClassTypeVar{"Instance", {}, std::nullopt, std::nullopt, {}, {}}); + persist(instanceType); typeChecker.globalScope->exportedTypeBindings["Instance"] = TypeFun{{}, instanceType}; getMutable(instanceType)->props = { diff --git a/tests/Module.test.cpp b/tests/Module.test.cpp index de063121..738893db 100644 --- a/tests/Module.test.cpp +++ b/tests/Module.test.cpp @@ -2,6 +2,7 @@ #include "Luau/Clone.h" #include "Luau/Module.h" #include "Luau/Scope.h" +#include "Luau/RecursionCounter.h" #include "Fixture.h" @@ -9,6 +10,8 @@ using namespace Luau; +LUAU_FASTFLAG(LuauLowerBoundsCalculation); + TEST_SUITE_BEGIN("ModuleTests"); TEST_CASE_FIXTURE(Fixture, "is_within_comment") @@ -42,29 +45,23 @@ TEST_CASE_FIXTURE(Fixture, "is_within_comment") TEST_CASE_FIXTURE(Fixture, "dont_clone_persistent_primitive") { TypeArena dest; - - SeenTypes seenTypes; - SeenTypePacks seenTypePacks; CloneState cloneState; // numberType is persistent. We leave it as-is. - TypeId newNumber = clone(typeChecker.numberType, dest, seenTypes, seenTypePacks, cloneState); + TypeId newNumber = clone(typeChecker.numberType, dest, cloneState); CHECK_EQ(newNumber, typeChecker.numberType); } TEST_CASE_FIXTURE(Fixture, "deepClone_non_persistent_primitive") { TypeArena dest; - - SeenTypes seenTypes; - SeenTypePacks seenTypePacks; CloneState cloneState; // Create a new number type that isn't persistent unfreeze(typeChecker.globalTypes); TypeId oldNumber = typeChecker.globalTypes.addType(PrimitiveTypeVar{PrimitiveTypeVar::Number}); freeze(typeChecker.globalTypes); - TypeId newNumber = clone(oldNumber, dest, seenTypes, seenTypePacks, cloneState); + TypeId newNumber = clone(oldNumber, dest, cloneState); CHECK_NE(newNumber, oldNumber); CHECK_EQ(*oldNumber, *newNumber); @@ -90,12 +87,9 @@ TEST_CASE_FIXTURE(Fixture, "deepClone_cyclic_table") TypeId counterType = requireType("Cyclic"); - SeenTypes seenTypes; - SeenTypePacks seenTypePacks; - CloneState cloneState; - TypeArena dest; - TypeId counterCopy = clone(counterType, dest, seenTypes, seenTypePacks, cloneState); + CloneState cloneState; + TypeId counterCopy = clone(counterType, dest, cloneState); TableTypeVar* ttv = getMutable(counterCopy); REQUIRE(ttv != nullptr); @@ -112,8 +106,11 @@ TEST_CASE_FIXTURE(Fixture, "deepClone_cyclic_table") REQUIRE(methodReturnType); CHECK_EQ(methodReturnType, counterCopy); - CHECK_EQ(2, dest.typePacks.size()); // one for the function args, and another for its return type - CHECK_EQ(2, dest.typeVars.size()); // One table and one function + if (FFlag::LuauLowerBoundsCalculation) + CHECK_EQ(3, dest.typePacks.size()); // function args, its return type, and the hidden any... pack + else + CHECK_EQ(2, dest.typePacks.size()); // one for the function args, and another for its return type + CHECK_EQ(2, dest.typeVars.size()); // One table and one function } TEST_CASE_FIXTURE(Fixture, "builtin_types_point_into_globalTypes_arena") @@ -143,15 +140,12 @@ TEST_CASE_FIXTURE(Fixture, "builtin_types_point_into_globalTypes_arena") TEST_CASE_FIXTURE(Fixture, "deepClone_union") { TypeArena dest; - - SeenTypes seenTypes; - SeenTypePacks seenTypePacks; CloneState cloneState; unfreeze(typeChecker.globalTypes); TypeId oldUnion = typeChecker.globalTypes.addType(UnionTypeVar{{typeChecker.numberType, typeChecker.stringType}}); freeze(typeChecker.globalTypes); - TypeId newUnion = clone(oldUnion, dest, seenTypes, seenTypePacks, cloneState); + TypeId newUnion = clone(oldUnion, dest, cloneState); CHECK_NE(newUnion, oldUnion); CHECK_EQ("number | string", toString(newUnion)); @@ -161,15 +155,12 @@ TEST_CASE_FIXTURE(Fixture, "deepClone_union") TEST_CASE_FIXTURE(Fixture, "deepClone_intersection") { TypeArena dest; - - SeenTypes seenTypes; - SeenTypePacks seenTypePacks; CloneState cloneState; unfreeze(typeChecker.globalTypes); TypeId oldIntersection = typeChecker.globalTypes.addType(IntersectionTypeVar{{typeChecker.numberType, typeChecker.stringType}}); freeze(typeChecker.globalTypes); - TypeId newIntersection = clone(oldIntersection, dest, seenTypes, seenTypePacks, cloneState); + TypeId newIntersection = clone(oldIntersection, dest, cloneState); CHECK_NE(newIntersection, oldIntersection); CHECK_EQ("number & string", toString(newIntersection)); @@ -191,12 +182,9 @@ TEST_CASE_FIXTURE(Fixture, "clone_class") std::nullopt, &exampleMetaClass, {}, {}}}; TypeArena dest; - - SeenTypes seenTypes; - SeenTypePacks seenTypePacks; CloneState cloneState; - TypeId cloned = clone(&exampleClass, dest, seenTypes, seenTypePacks, cloneState); + TypeId cloned = clone(&exampleClass, dest, cloneState); const ClassTypeVar* ctv = get(cloned); REQUIRE(ctv != nullptr); @@ -216,16 +204,14 @@ TEST_CASE_FIXTURE(Fixture, "clone_sanitize_free_types") TypePackVar freeTp(FreeTypePack{TypeLevel{}}); TypeArena dest; - SeenTypes seenTypes; - SeenTypePacks seenTypePacks; CloneState cloneState; - TypeId clonedTy = clone(&freeTy, dest, seenTypes, seenTypePacks, cloneState); + TypeId clonedTy = clone(&freeTy, dest, cloneState); CHECK_EQ("any", toString(clonedTy)); CHECK(cloneState.encounteredFreeType); cloneState = {}; - TypePackId clonedTp = clone(&freeTp, dest, seenTypes, seenTypePacks, cloneState); + TypePackId clonedTp = clone(&freeTp, dest, cloneState); CHECK_EQ("...any", toString(clonedTp)); CHECK(cloneState.encounteredFreeType); } @@ -237,16 +223,32 @@ TEST_CASE_FIXTURE(Fixture, "clone_seal_free_tables") ttv->state = TableState::Free; TypeArena dest; - SeenTypes seenTypes; - SeenTypePacks seenTypePacks; CloneState cloneState; - TypeId cloned = clone(&tableTy, dest, seenTypes, seenTypePacks, cloneState); + TypeId cloned = clone(&tableTy, dest, cloneState); const TableTypeVar* clonedTtv = get(cloned); CHECK_EQ(clonedTtv->state, TableState::Sealed); CHECK(cloneState.encounteredFreeType); } +TEST_CASE_FIXTURE(Fixture, "clone_constrained_intersection") +{ + TypeArena src; + + TypeId constrained = src.addType(ConstrainedTypeVar{TypeLevel{}, {getSingletonTypes().numberType, getSingletonTypes().stringType}}); + + TypeArena dest; + CloneState cloneState; + + TypeId cloned = clone(constrained, dest, cloneState); + CHECK_NE(constrained, cloned); + + const ConstrainedTypeVar* ctv = get(cloned); + REQUIRE_EQ(2, ctv->parts.size()); + CHECK_EQ(getSingletonTypes().numberType, ctv->parts[0]); + CHECK_EQ(getSingletonTypes().stringType, ctv->parts[1]); +} + TEST_CASE_FIXTURE(Fixture, "clone_self_property") { ScopedFastFlag sff{"LuauAnyInIsOptionalIsOptional", true}; @@ -284,6 +286,7 @@ TEST_CASE_FIXTURE(Fixture, "clone_recursion_limit") int limit = 400; #endif ScopedFastInt luauTypeCloneRecursionLimit{"LuauTypeCloneRecursionLimit", limit}; + ScopedFastFlag sff{"LuauRecursionLimitException", true}; TypeArena src; @@ -299,11 +302,9 @@ TEST_CASE_FIXTURE(Fixture, "clone_recursion_limit") } TypeArena dest; - SeenTypes seenTypes; - SeenTypePacks seenTypePacks; CloneState cloneState; - CHECK_THROWS_AS(clone(table, dest, seenTypes, seenTypePacks, cloneState), std::runtime_error); + CHECK_THROWS_AS(clone(table, dest, cloneState), RecursionLimitException); } TEST_SUITE_END(); diff --git a/tests/NonstrictMode.test.cpp b/tests/NonstrictMode.test.cpp index d3faea2a..a8a12b69 100644 --- a/tests/NonstrictMode.test.cpp +++ b/tests/NonstrictMode.test.cpp @@ -275,4 +275,38 @@ TEST_CASE_FIXTURE(Fixture, "inconsistent_module_return_types_are_ok") REQUIRE_EQ("any", toString(getMainModule()->getModuleScope()->returnType)); } +TEST_CASE_FIXTURE(Fixture, "returning_insufficient_return_values") +{ + CheckResult result = check(R"( + --!nonstrict + + function foo(): (boolean, string?) + if true then + return true, "hello" + else + return false + end + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "returning_too_many_values") +{ + CheckResult result = check(R"( + --!nonstrict + + function foo(): boolean + if true then + return true, "hello" + else + return false + end + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + TEST_SUITE_END(); diff --git a/tests/Normalize.test.cpp b/tests/Normalize.test.cpp new file mode 100644 index 00000000..5a84201a --- /dev/null +++ b/tests/Normalize.test.cpp @@ -0,0 +1,967 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details + +#include "Fixture.h" + +#include "doctest.h" + +#include "Luau/Normalize.h" +#include "Luau/BuiltinDefinitions.h" + +using namespace Luau; + +struct NormalizeFixture : Fixture +{ + ScopedFastFlag sff1{"LuauLowerBoundsCalculation", true}; + ScopedFastFlag sff2{"LuauTableSubtypingVariance2", true}; +}; + +void createSomeClasses(TypeChecker& typeChecker) +{ + auto& arena = typeChecker.globalTypes; + + unfreeze(arena); + + TypeId parentType = arena.addType(ClassTypeVar{"Parent", {}, std::nullopt, std::nullopt, {}, nullptr}); + + ClassTypeVar* parentClass = getMutable(parentType); + parentClass->props["method"] = {makeFunction(arena, parentType, {}, {})}; + + parentClass->props["virtual_method"] = {makeFunction(arena, parentType, {}, {})}; + + addGlobalBinding(typeChecker, "Parent", {parentType}); + typeChecker.globalScope->exportedTypeBindings["Parent"] = TypeFun{{}, parentType}; + + TypeId childType = arena.addType(ClassTypeVar{"Child", {}, parentType, std::nullopt, {}, nullptr}); + + ClassTypeVar* childClass = getMutable(childType); + childClass->props["virtual_method"] = {makeFunction(arena, childType, {}, {})}; + + addGlobalBinding(typeChecker, "Child", {childType}); + typeChecker.globalScope->exportedTypeBindings["Child"] = TypeFun{{}, childType}; + + TypeId unrelatedType = arena.addType(ClassTypeVar{"Unrelated", {}, std::nullopt, std::nullopt, {}, nullptr}); + + addGlobalBinding(typeChecker, "Unrelated", {unrelatedType}); + typeChecker.globalScope->exportedTypeBindings["Unrelated"] = TypeFun{{}, unrelatedType}; + + freeze(arena); +} + +static bool isSubtype(TypeId a, TypeId b) +{ + InternalErrorReporter ice; + return isSubtype(a, b, ice); +} + +TEST_SUITE_BEGIN("isSubtype"); + +TEST_CASE_FIXTURE(NormalizeFixture, "primitives") +{ + check(R"( + local a = 41 + local b = 32 + + local c = "hello" + local d = "world" + )"); + + TypeId a = requireType("a"); + TypeId b = requireType("b"); + TypeId c = requireType("c"); + TypeId d = requireType("d"); + + CHECK(isSubtype(b, a)); + CHECK(isSubtype(d, c)); + CHECK(!isSubtype(d, a)); +} + +TEST_CASE_FIXTURE(NormalizeFixture, "functions") +{ + check(R"( + function a(x: number): number return x end + function b(x: number): number return x end + + function c(x: number?): number return x end + function d(x: number): number? return x end + )"); + + TypeId a = requireType("a"); + TypeId b = requireType("b"); + TypeId c = requireType("c"); + TypeId d = requireType("d"); + + CHECK(isSubtype(b, a)); + CHECK(isSubtype(c, a)); + CHECK(!isSubtype(d, a)); + CHECK(isSubtype(a, d)); +} + +TEST_CASE_FIXTURE(NormalizeFixture, "functions_and_any") +{ + check(R"( + function a(n: number) return "string" end + function b(q: any) return 5 :: any end + )"); + + TypeId a = requireType("a"); + TypeId b = requireType("b"); + + // Intuition: + // We cannot use b where a is required because we cannot rely on b to return a string. + // We cannot use a where b is required because we cannot rely on a to accept non-number arguments. + + CHECK(!isSubtype(b, a)); + CHECK(!isSubtype(a, b)); +} + +TEST_CASE_FIXTURE(NormalizeFixture, "intersection_of_functions_of_different_arities") +{ + check(R"( + type A = (any) -> () + type B = (any, any) -> () + type T = A & B + + local a: A + local b: B + local t: T + )"); + + TypeId a = requireType("a"); + TypeId b = requireType("b"); + + CHECK(!isSubtype(a, b)); // !! + CHECK(!isSubtype(b, a)); + + CHECK("((any) -> ()) & ((any, any) -> ())" == toString(requireType("t"))); +} + +TEST_CASE_FIXTURE(NormalizeFixture, "functions_with_mismatching_arity") +{ + check(R"( + local a: (number) -> () + local b: () -> () + + local c: () -> number + )"); + + TypeId a = requireType("a"); + TypeId b = requireType("b"); + TypeId c = requireType("c"); + + CHECK(!isSubtype(b, a)); + CHECK(!isSubtype(c, a)); + + CHECK(!isSubtype(a, b)); + CHECK(!isSubtype(c, b)); + + CHECK(!isSubtype(a, c)); + CHECK(!isSubtype(b, c)); +} + +TEST_CASE_FIXTURE(NormalizeFixture, "functions_with_mismatching_arity_but_optional_parameters") +{ + /* + * (T0..TN) <: (T0..TN, A?) + * (T0..TN) <: (T0..TN, any) + * (T0..TN, A?) R <: U -> S if U <: T and R <: S + * A | B <: T if A <: T and B <: T + * T <: A | B if T <: A or T <: B + */ + check(R"( + local a: (number?) -> () + local b: (number) -> () + local c: (number, number?) -> () + )"); + + TypeId a = requireType("a"); + TypeId b = requireType("b"); + TypeId c = requireType("c"); + + /* + * (number) -> () () + * because number? () + * because number? () <: (number) -> () + * because number <: number? (because number <: number) + */ + CHECK(isSubtype(a, b)); + + /* + * (number, number?) -> () <: (number) -> (number) + * The packs have inequal lengths, but (number) <: (number, number?) + * and number <: number + */ + CHECK(!isSubtype(c, b)); + + /* + * (number?) -> () () + * because (number, number?) () () + * because (number, number?) () + local b: (number) -> () + local c: (number, any) -> () + )"); + + TypeId a = requireType("a"); + TypeId b = requireType("b"); + TypeId c = requireType("c"); + + /* + * (number) -> () () + * because number? () + * because number? () <: (number) -> () + * because number <: number? (because number <: number) + */ + CHECK(isSubtype(a, b)); + + /* + * (number, any) -> () (number) + * The packs have inequal lengths + */ + CHECK(!isSubtype(c, b)); + + /* + * (number?) -> () () + * The packs have inequal lengths + */ + CHECK(!isSubtype(a, c)); + + /* + * (number) -> () () + * The packs have inequal lengths + */ + CHECK(!isSubtype(b, c)); +} + +TEST_CASE_FIXTURE(NormalizeFixture, "variadic_functions_with_no_head") +{ + check(R"( + local a: (...number) -> () + local b: (...number?) -> () + )"); + + TypeId a = requireType("a"); + TypeId b = requireType("b"); + + CHECK(isSubtype(b, a)); + CHECK(!isSubtype(a, b)); +} + +#if 0 +TEST_CASE_FIXTURE(NormalizeFixture, "variadic_function_with_head") +{ + check(R"( + local a: (...number) -> () + local b: (number, number) -> () + )"); + + TypeId a = requireType("a"); + TypeId b = requireType("b"); + + CHECK(!isSubtype(b, a)); + CHECK(isSubtype(a, b)); +} +#endif + +TEST_CASE_FIXTURE(NormalizeFixture, "union") +{ + check(R"( + local a: number | string + local b: number + local c: string + local d: number? + )"); + + TypeId a = requireType("a"); + TypeId b = requireType("b"); + TypeId c = requireType("c"); + TypeId d = requireType("d"); + + CHECK(isSubtype(b, a)); + CHECK(!isSubtype(a, b)); + + CHECK(isSubtype(c, a)); + CHECK(!isSubtype(a, c)); + + CHECK(!isSubtype(d, a)); + CHECK(!isSubtype(a, d)); + + CHECK(isSubtype(b, d)); + CHECK(!isSubtype(d, b)); +} + +TEST_CASE_FIXTURE(NormalizeFixture, "table_with_union_prop") +{ + check(R"( + local a: {x: number} + local b: {x: number?} + )"); + + TypeId a = requireType("a"); + TypeId b = requireType("b"); + + CHECK(isSubtype(a, b)); + CHECK(!isSubtype(b, a)); +} + +TEST_CASE_FIXTURE(NormalizeFixture, "table_with_any_prop") +{ + check(R"( + local a: {x: number} + local b: {x: any} + )"); + + TypeId a = requireType("a"); + TypeId b = requireType("b"); + + CHECK(isSubtype(a, b)); + CHECK(!isSubtype(b, a)); +} + +TEST_CASE_FIXTURE(NormalizeFixture, "intersection") +{ + check(R"( + local a: number & string + local b: number + local c: string + local d: number & nil + )"); + + TypeId a = requireType("a"); + TypeId b = requireType("b"); + TypeId c = requireType("c"); + TypeId d = requireType("d"); + + CHECK(!isSubtype(b, a)); + CHECK(isSubtype(a, b)); + + CHECK(!isSubtype(c, a)); + CHECK(isSubtype(a, c)); + + CHECK(!isSubtype(d, a)); + CHECK(!isSubtype(a, d)); +} + +TEST_CASE_FIXTURE(NormalizeFixture, "union_and_intersection") +{ + check(R"( + local a: number & string + local b: number | nil + )"); + + TypeId a = requireType("a"); + TypeId b = requireType("b"); + + CHECK(!isSubtype(b, a)); + CHECK(isSubtype(a, b)); +} + +TEST_CASE_FIXTURE(NormalizeFixture, "table_with_table_prop") +{ + check(R"( + type T = {x: {y: number}} & {x: {y: string}} + local a: T + )"); + + CHECK_EQ("{| x: {| y: number & string |} |}", toString(requireType("a"))); +} + +#if 0 +TEST_CASE_FIXTURE(NormalizeFixture, "tables") +{ + check(R"( + local a: {x: number} + local b: {x: any} + local c: {y: number} + local d: {x: number, y: number} + )"); + + TypeId a = requireType("a"); + TypeId b = requireType("b"); + TypeId c = requireType("c"); + TypeId d = requireType("d"); + + CHECK(isSubtype(a, b)); + CHECK(!isSubtype(b, a)); + + CHECK(!isSubtype(c, a)); + CHECK(!isSubtype(a, c)); + + CHECK(isSubtype(d, a)); + CHECK(!isSubtype(a, d)); + + CHECK(isSubtype(d, b)); + CHECK(!isSubtype(b, d)); +} + +TEST_CASE_FIXTURE(NormalizeFixture, "table_indexers_are_invariant") +{ + check(R"( + local a: {[string]: number} + local b: {[string]: any} + local c: {[string]: number} + )"); + + TypeId a = requireType("a"); + TypeId b = requireType("b"); + TypeId c = requireType("c"); + + CHECK(!isSubtype(b, a)); + CHECK(!isSubtype(a, b)); + + CHECK(isSubtype(c, a)); + CHECK(isSubtype(a, c)); +} + +TEST_CASE_FIXTURE(NormalizeFixture, "mismatched_indexers") +{ + check(R"( + local a: {x: number} + local b: {[string]: number} + local c: {} + )"); + + TypeId a = requireType("a"); + TypeId b = requireType("b"); + TypeId c = requireType("c"); + + CHECK(isSubtype(b, a)); + CHECK(!isSubtype(a, b)); + + CHECK(!isSubtype(c, b)); + CHECK(isSubtype(b, c)); +} + +TEST_CASE_FIXTURE(NormalizeFixture, "cyclic_table") +{ + check(R"( + type A = {method: (A) -> ()} + local a: A + + type B = {method: (any) -> ()} + local b: B + + type C = {method: (C) -> ()} + local c: C + + type D = {method: (D) -> (), another: (D) -> ()} + local d: D + + type E = {method: (A) -> (), another: (E) -> ()} + local e: E + )"); + + TypeId a = requireType("a"); + TypeId b = requireType("b"); + TypeId c = requireType("c"); + TypeId d = requireType("d"); + TypeId e = requireType("e"); + + CHECK(isSubtype(b, a)); + CHECK(!isSubtype(a, b)); + + CHECK(isSubtype(c, a)); + CHECK(isSubtype(a, c)); + + CHECK(!isSubtype(d, a)); + CHECK(!isSubtype(a, d)); + + CHECK(isSubtype(e, a)); + CHECK(!isSubtype(a, e)); +} +#endif + +TEST_CASE_FIXTURE(NormalizeFixture, "classes") +{ + createSomeClasses(typeChecker); + + TypeId p = typeChecker.globalScope->lookupType("Parent")->type; + TypeId c = typeChecker.globalScope->lookupType("Child")->type; + TypeId u = typeChecker.globalScope->lookupType("Unrelated")->type; + + CHECK(isSubtype(c, p)); + CHECK(!isSubtype(p, c)); + CHECK(!isSubtype(u, p)); + CHECK(!isSubtype(p, u)); +} + +#if 0 +TEST_CASE_FIXTURE(NormalizeFixture, "metatable" * doctest::expected_failures{1}) +{ + check(R"( + local T = {} + T.__index = T + function T.new() + return setmetatable({}, T) + end + + function T:method() end + + local a: typeof(T.new) + local b: {method: (any) -> ()} + )"); + + TypeId a = requireType("a"); + TypeId b = requireType("b"); + + CHECK(isSubtype(a, b)); +} +#endif + +TEST_CASE_FIXTURE(NormalizeFixture, "intersection_of_tables") +{ + check(R"( + type T = {x: number} & ({x: number} & {y: string?}) + local t: T + )"); + + CHECK("{| x: number, y: string? |}" == toString(requireType("t"))); +} + +TEST_SUITE_END(); + +TEST_SUITE_BEGIN("Normalize"); + +TEST_CASE_FIXTURE(NormalizeFixture, "intersection_of_disjoint_tables") +{ + check(R"( + type T = {a: number} & {b: number} + local t: T + )"); + + CHECK_EQ("{| a: number, b: number |}", toString(requireType("t"))); +} + +TEST_CASE_FIXTURE(NormalizeFixture, "intersection_of_overlapping_tables") +{ + check(R"( + type T = {a: number, b: string} & {b: number, c: string} + local t: T + )"); + + CHECK_EQ("{| a: number, b: number & string, c: string |}", toString(requireType("t"))); +} + +TEST_CASE_FIXTURE(NormalizeFixture, "intersection_of_confluent_overlapping_tables") +{ + check(R"( + type T = {a: number, b: string} & {b: string, c: string} + local t: T + )"); + + CHECK_EQ("{| a: number, b: string, c: string |}", toString(requireType("t"))); +} + +TEST_CASE_FIXTURE(NormalizeFixture, "union_with_overlapping_field_that_has_a_subtype_relationship") +{ + check(R"( + local t: {x: number} | {x: number?} + )"); + + ModulePtr tempModule{new Module}; + + // HACK: Normalization is an in-place operation. We need to cheat a little here and unfreeze + // the arena that the type lives in. + ModulePtr mainModule = getMainModule(); + unfreeze(mainModule->internalTypes); + + TypeId tType = requireType("t"); + normalize(tType, tempModule, *typeChecker.iceHandler); + + CHECK_EQ("{| x: number? |}", toString(tType, {true})); +} + +TEST_CASE_FIXTURE(NormalizeFixture, "intersection_of_functions") +{ + check(R"( + type T = ((any) -> string) & ((number) -> string) + local t: T + )"); + + CHECK_EQ("(any) -> string", toString(requireType("t"))); +} + +TEST_CASE_FIXTURE(Fixture, "normalize_module_return_type") +{ + ScopedFastFlag sff[] = { + {"LuauLowerBoundsCalculation", true}, + }; + + check(R"( + --!nonstrict + + if Math.random() then + return function(initialState, handlers) + return function(state, action) + return state + end + end + else + return function(initialState, handlers) + return function(state, action) + return state + end + end + end + )"); + + CHECK_EQ("(any, any) -> (...any)", toString(getMainModule()->getModuleScope()->returnType)); +} + +TEST_CASE_FIXTURE(Fixture, "return_type_is_not_a_constrained_intersection") +{ + check(R"( + function foo(x:number, y:number) + return x + y + end + )"); + + CHECK_EQ("(number, number) -> number", toString(requireType("foo"))); +} + +TEST_CASE_FIXTURE(Fixture, "higher_order_function") +{ + check(R"( + function apply(f, x) + return f(x) + end + + local a = apply(function(x: number) return x + x end, 5) + )"); + + TypeId aType = requireType("a"); + CHECK_MESSAGE(isNumber(follow(aType)), "Expected a number but got ", toString(aType)); +} + +TEST_CASE_FIXTURE(Fixture, "higher_order_function_with_annotation") +{ + check(R"( + function apply(f: (a) -> b, x) + return f(x) + end + )"); + + CHECK_EQ("((a) -> b, a) -> b", toString(requireType("apply"))); +} + +TEST_CASE_FIXTURE(Fixture, "cyclic_table_is_marked_normal") +{ + ScopedFastFlag flags[] = { + {"LuauLowerBoundsCalculation", true}, + }; + + check(R"( + type Fiber = { + return_: Fiber? + } + + local f: Fiber + )"); + + TypeId t = requireType("f"); + CHECK(t->normal); +} + +TEST_CASE_FIXTURE(Fixture, "variadic_tail_is_marked_normal") +{ + ScopedFastFlag flags[] = { + {"LuauLowerBoundsCalculation", true}, + }; + + CheckResult result = check(R"( + type Weirdo = (...{x: number}) -> () + + local w: Weirdo + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + TypeId t = requireType("w"); + auto ftv = get(t); + REQUIRE(ftv); + + auto [argHead, argTail] = flatten(ftv->argTypes); + CHECK(argHead.empty()); + REQUIRE(argTail.has_value()); + + auto vtp = get(*argTail); + REQUIRE(vtp); + CHECK(vtp->ty->normal); +} + +TEST_CASE_FIXTURE(Fixture, "cyclic_table_normalizes_sensibly") +{ + CheckResult result = check(R"( + local Cyclic = {} + function Cyclic.get() + return Cyclic + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + TypeId ty = requireType("Cyclic"); + CHECK_EQ("t1 where t1 = { get: () -> t1 }", toString(ty, {true})); +} + +TEST_CASE_FIXTURE(Fixture, "union_of_distinct_free_types") +{ + ScopedFastFlag flags[] = { + {"LuauLowerBoundsCalculation", true}, + }; + + CheckResult result = check(R"( + function fussy(a, b) + if math.random() > 0.5 then + return a + else + return b + end + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK("(a, b) -> a | b" == toString(requireType("fussy"))); +} + +TEST_CASE_FIXTURE(Fixture, "constrained_intersection_of_intersections") +{ + ScopedFastFlag flags[] = { + {"LuauLowerBoundsCalculation", true}, + }; + + CheckResult result = check(R"( + local f : (() -> number) | ((number) -> number) + local g : (() -> number) | ((string) -> number) + + function h() + if math.random() then + return f + else + return g + end + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + TypeId h = requireType("h"); + + CHECK("() -> (() -> number) | ((number) -> number) | ((string) -> number)" == toString(h)); +} + +TEST_CASE_FIXTURE(Fixture, "intersection_inside_a_table_inside_another_intersection") +{ + ScopedFastFlag flags[] = { + {"LuauLowerBoundsCalculation", true}, + }; + + CheckResult result = check(R"( + type X = {} + type Y = {y: number} + type Z = {z: string} + type W = {w: boolean} + type T = {x: Y & X} & {x:Z & W} + + local x: X + local y: Y + local z: Z + local w: W + local t: T + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK("{| |}" == toString(requireType("x"), {true})); + CHECK("{| y: number |}" == toString(requireType("y"), {true})); + CHECK("{| z: string |}" == toString(requireType("z"), {true})); + CHECK("{| w: boolean |}" == toString(requireType("w"), {true})); + CHECK("{| x: {| w: boolean, y: number, z: string |} |}" == toString(requireType("t"), {true})); +} + +TEST_CASE_FIXTURE(Fixture, "intersection_inside_a_table_inside_another_intersection_2") +{ + ScopedFastFlag flags[] = { + {"LuauLowerBoundsCalculation", true}, + }; + + // We use a function and inferred parameter types to prevent intermediate normalizations from being performed. + // This exposes a bug where the type of y is mutated. + CheckResult result = check(R"( + function strange(w, x, y, z) + y.y = 5 + z.z = "five" + w.w = true + + type Z = {x: typeof(x) & typeof(y)} & {x: typeof(w) & typeof(z)} + + return ((nil :: any) :: Z) + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + TypeId t = requireType("strange"); + auto ftv = get(t); + REQUIRE(ftv != nullptr); + + std::vector args = flatten(ftv->argTypes).first; + + REQUIRE(4 == args.size()); + CHECK("{+ w: boolean +}" == toString(args[0])); + CHECK("a" == toString(args[1])); + CHECK("{+ y: number +}" == toString(args[2])); + CHECK("{+ z: string +}" == toString(args[3])); + + std::vector ret = flatten(ftv->retType).first; + + REQUIRE(1 == ret.size()); + CHECK("{| x: a & {- w: boolean, y: number, z: string -} |}" == toString(ret[0])); +} + +TEST_CASE_FIXTURE(Fixture, "intersection_inside_a_table_inside_another_intersection_3") +{ + ScopedFastFlag flags[] = { + {"LuauLowerBoundsCalculation", true}, + }; + + // We use a function and inferred parameter types to prevent intermediate normalizations from being performed. + // This exposes a bug where the type of y is mutated. + CheckResult result = check(R"( + function strange(x, y, z) + x.x = true + y.y = y + z.z = "five" + + type Z = {x: typeof(y)} & {x: typeof(x) & typeof(z)} + + return ((nil :: any) :: Z) + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + TypeId t = requireType("strange"); + auto ftv = get(t); + REQUIRE(ftv != nullptr); + + std::vector args = flatten(ftv->argTypes).first; + + REQUIRE(3 == args.size()); + CHECK("{+ x: boolean +}" == toString(args[0])); + CHECK("t1 where t1 = {+ y: t1 +}" == toString(args[1])); + CHECK("{+ z: string +}" == toString(args[2])); + + std::vector ret = flatten(ftv->retType).first; + + REQUIRE(1 == ret.size()); + CHECK("{| x: {- x: boolean, y: t1, z: string -} |} where t1 = {+ y: t1 +}" == toString(ret[0])); +} + +TEST_CASE_FIXTURE(Fixture, "intersection_inside_a_table_inside_another_intersection_4") +{ + ScopedFastFlag flags[] = { + {"LuauLowerBoundsCalculation", true}, + }; + + // We use a function and inferred parameter types to prevent intermediate normalizations from being performed. + // This exposes a bug where the type of y is mutated. + CheckResult result = check(R"( + function strange(x, y, z) + x.x = true + z.z = "five" + + type R = {x: typeof(y)} & {x: typeof(x) & typeof(z)} + local r: R + + y.y = r + + return r + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + TypeId t = requireType("strange"); + auto ftv = get(t); + REQUIRE(ftv != nullptr); + + std::vector args = flatten(ftv->argTypes).first; + + REQUIRE(3 == args.size()); + CHECK("{+ x: boolean +}" == toString(args[0])); + CHECK("{+ y: t1 +} where t1 = {| x: {- x: boolean, y: t1, z: string -} |}" == toString(args[1])); + CHECK("{+ z: string +}" == toString(args[2])); + + std::vector ret = flatten(ftv->retType).first; + + REQUIRE(1 == ret.size()); + CHECK("t1 where t1 = {| x: {- x: boolean, y: t1, z: string -} |}" == toString(ret[0])); +} + +TEST_CASE_FIXTURE(Fixture, "nested_table_normalization_with_non_table__no_ice") +{ + ScopedFastFlag flags[] = { + {"LuauLowerBoundsCalculation", true}, + {"LuauNormalizeCombineTableFix", true}, + }; + // CLI-52787 + // ends up combining {_:any} with any, recursively + // which used to ICE because this combines a table with a non-table. + CheckResult result = check(R"( + export type t0 = any & { _: {_:any} } & { _:any } + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "fuzz_failure_instersection_combine_must_follow") +{ + ScopedFastFlag flags[] = { + {"LuauLowerBoundsCalculation", true}, + {"LuauNormalizeCombineIntersectionFix", true}, + }; + + CheckResult result = check(R"( + export type t0 = {_:{_:any} & {_:any|string}} & {_:{_:{}}} + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_SUITE_END(); diff --git a/tests/Parser.test.cpp b/tests/Parser.test.cpp index 79f9ecab..b941103d 100644 --- a/tests/Parser.test.cpp +++ b/tests/Parser.test.cpp @@ -1618,6 +1618,26 @@ TEST_CASE_FIXTURE(Fixture, "end_extent_doesnt_consume_comments") CHECK_EQ((Position{1, 23}), block->body.data[0]->location.end); } +TEST_CASE_FIXTURE(Fixture, "end_extent_doesnt_consume_comments_even_with_capture") +{ + ScopedFastFlag luauParseLocationIgnoreCommentSkip{"LuauParseLocationIgnoreCommentSkip", true}; + ScopedFastFlag luauParseLocationIgnoreCommentSkipInCapture{"LuauParseLocationIgnoreCommentSkipInCapture", true}; + + // Same should hold when comments are captured + ParseOptions opts; + opts.captureComments = true; + + AstStatBlock* block = parse(R"( + type F = number + --comment + print('hello') + )", + opts); + + REQUIRE_EQ(2, block->body.size); + CHECK_EQ((Position{1, 23}), block->body.data[0]->location.end); +} + TEST_CASE_FIXTURE(Fixture, "parse_error_loop_control") { matchParseError("break", "break statement must be inside a loop"); diff --git a/tests/ToDot.test.cpp b/tests/ToDot.test.cpp index 29bdd866..f3fda54e 100644 --- a/tests/ToDot.test.cpp +++ b/tests/ToDot.test.cpp @@ -7,6 +7,8 @@ #include "doctest.h" +LUAU_FASTFLAG(LuauLowerBoundsCalculation) + using namespace Luau; struct ToDotClassFixture : Fixture @@ -101,9 +103,34 @@ local function f(a, ...: string) return a end )"); LUAU_REQUIRE_NO_ERRORS(result); + CHECK_EQ("(a, ...string) -> a", toString(requireType("f"))); + ToDotOptions opts; opts.showPointers = false; - CHECK_EQ(R"(digraph graphname { + + if (FFlag::LuauLowerBoundsCalculation) + { + CHECK_EQ(R"(digraph graphname { +n1 [label="FunctionTypeVar 1"]; +n1 -> n2 [label="arg"]; +n2 [label="TypePack 2"]; +n2 -> n3; +n3 [label="GenericTypeVar 3"]; +n2 -> n4 [label="tail"]; +n4 [label="VariadicTypePack 4"]; +n4 -> n5; +n5 [label="string"]; +n1 -> n6 [label="ret"]; +n6 [label="TypePack 6"]; +n6 -> n7; +n7 [label="BoundTypeVar 7"]; +n7 -> n3; +})", + toDot(requireType("f"), opts)); + } + else + { + CHECK_EQ(R"(digraph graphname { n1 [label="FunctionTypeVar 1"]; n1 -> n2 [label="arg"]; n2 [label="TypePack 2"]; @@ -119,7 +146,8 @@ n6 -> n7; n7 [label="TypePack 7"]; n7 -> n3; })", - toDot(requireType("f"), opts)); + toDot(requireType("f"), opts)); + } } TEST_CASE_FIXTURE(Fixture, "union") @@ -361,4 +389,49 @@ n3 [label="number"]; toDot(*ty, opts)); } +TEST_CASE_FIXTURE(Fixture, "constrained") +{ + // ConstrainedTypeVars never appear in the final type graph, so we have to create one directly + // to dotify it. + TypeVar t{ConstrainedTypeVar{TypeLevel{}, {typeChecker.numberType, typeChecker.stringType, typeChecker.nilType}}}; + + ToDotOptions opts; + opts.showPointers = false; + + CHECK_EQ(R"(digraph graphname { +n1 [label="ConstrainedTypeVar 1"]; +n1 -> n2; +n2 [label="number"]; +n1 -> n3; +n3 [label="string"]; +n1 -> n4; +n4 [label="nil"]; +})", + toDot(&t, opts)); +} + +TEST_CASE_FIXTURE(Fixture, "singletontypes") +{ + CheckResult result = check(R"( + local x: "hi" | "\"hello\"" | true | false + )"); + + ToDotOptions opts; + opts.showPointers = false; + + CHECK_EQ(R"(digraph graphname { +n1 [label="UnionTypeVar 1"]; +n1 -> n2; +n2 [label="SingletonTypeVar string: hi"]; +n1 -> n3; +)" +"n3 [label=\"SingletonTypeVar string: \\\"hello\\\"\"];" +R"( +n1 -> n4; +n4 [label="SingletonTypeVar boolean: true"]; +n1 -> n5; +n5 [label="SingletonTypeVar boolean: false"]; +})", toDot(requireType("x"), opts)); +} + TEST_SUITE_END(); diff --git a/tests/ToString.test.cpp b/tests/ToString.test.cpp index 3051e209..ccf5c583 100644 --- a/tests/ToString.test.cpp +++ b/tests/ToString.test.cpp @@ -9,6 +9,8 @@ using namespace Luau; +LUAU_FASTFLAG(LuauRecursiveTypeParameterRestriction); + TEST_SUITE_BEGIN("ToString"); TEST_CASE_FIXTURE(Fixture, "primitive") diff --git a/tests/TopoSort.test.cpp b/tests/TopoSort.test.cpp index 9b990866..1f14ae88 100644 --- a/tests/TopoSort.test.cpp +++ b/tests/TopoSort.test.cpp @@ -340,26 +340,28 @@ TEST_CASE_FIXTURE(Fixture, "nested_type_annotations_depends_on_later_typealiases TEST_CASE_FIXTURE(Fixture, "return_comes_last") { - CheckResult result = check(R"( -export type Module = { bar: (number) -> boolean, foo: () -> string } + AstStatBlock* program = parse(R"( + local module = {} -return function() : Module - local module = {} + local function confuseCompiler() return module.foo() end - local function confuseCompiler() return module.foo() end - - module.foo = function() return "" end + module.foo = function() return "" end - function module.bar(x:number) - confuseCompiler() - return true - end - - return module -end + function module.bar(x:number) + confuseCompiler() + return true + end + + return module )"); - LUAU_REQUIRE_NO_ERRORS(result); + auto sorted = toposort(*program); + + CHECK_EQ(sorted[0], program->body.data[0]); + CHECK_EQ(sorted[2], program->body.data[1]); + CHECK_EQ(sorted[1], program->body.data[2]); + CHECK_EQ(sorted[3], program->body.data[3]); + CHECK_EQ(sorted[4], program->body.data[4]); } TEST_CASE_FIXTURE(Fixture, "break_comes_last") diff --git a/tests/Transpiler.test.cpp b/tests/Transpiler.test.cpp index 5ac45ff2..0c324cd0 100644 --- a/tests/Transpiler.test.cpp +++ b/tests/Transpiler.test.cpp @@ -388,7 +388,7 @@ TEST_CASE_FIXTURE(Fixture, "type_lists_should_be_emitted_correctly") std::string actual = decorateWithTypes(code); - CHECK_EQ(expected, decorateWithTypes(code)); + CHECK_EQ(expected, actual); } TEST_CASE_FIXTURE(Fixture, "function_type_location") diff --git a/tests/TypeInfer.annotations.test.cpp b/tests/TypeInfer.annotations.test.cpp index 2ad11d01..e2971ad5 100644 --- a/tests/TypeInfer.annotations.test.cpp +++ b/tests/TypeInfer.annotations.test.cpp @@ -753,4 +753,14 @@ TEST_CASE_FIXTURE(Fixture, "occurs_check_on_cyclic_intersection_typevar") REQUIRE(ocf); } +TEST_CASE_FIXTURE(Fixture, "instantiation_clone_has_to_follow") +{ + CheckResult result = check(R"( + export type t8 = (t0)&(((true)|(any))->"") + export type t0 = ({})&({_:{[any]:number},}) + )"); + + LUAU_REQUIRE_ERRORS(result); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.builtins.test.cpp b/tests/TypeInfer.builtins.test.cpp index c6fbebed..1ae65947 100644 --- a/tests/TypeInfer.builtins.test.cpp +++ b/tests/TypeInfer.builtins.test.cpp @@ -8,6 +8,8 @@ using namespace Luau; +LUAU_FASTFLAG(LuauLowerBoundsCalculation); + TEST_SUITE_BEGIN("BuiltinTests"); TEST_CASE_FIXTURE(Fixture, "math_things_are_defined") @@ -557,9 +559,9 @@ TEST_CASE_FIXTURE(Fixture, "xpcall") )"); LUAU_REQUIRE_NO_ERRORS(result); - REQUIRE_EQ("boolean", toString(requireType("a"))); - REQUIRE_EQ("number", toString(requireType("b"))); - REQUIRE_EQ("boolean", toString(requireType("c"))); + CHECK_EQ("boolean", toString(requireType("a"))); + CHECK_EQ("number", toString(requireType("b"))); + CHECK_EQ("boolean", toString(requireType("c"))); } TEST_CASE_FIXTURE(Fixture, "see_thru_select") @@ -881,7 +883,10 @@ TEST_CASE_FIXTURE(Fixture, "assert_removes_falsy_types") )"); LUAU_REQUIRE_NO_ERRORS(result); - CHECK_EQ("((boolean | number)?) -> boolean | number", toString(requireType("f"))); + if (FFlag::LuauLowerBoundsCalculation) + CHECK_EQ("((boolean | number)?) -> number | true", toString(requireType("f"))); + else + CHECK_EQ("((boolean | number)?) -> boolean | number", toString(requireType("f"))); } TEST_CASE_FIXTURE(Fixture, "assert_removes_falsy_types2") diff --git a/tests/TypeInfer.classes.test.cpp b/tests/TypeInfer.classes.test.cpp index 98fa66eb..8e3629e7 100644 --- a/tests/TypeInfer.classes.test.cpp +++ b/tests/TypeInfer.classes.test.cpp @@ -91,6 +91,9 @@ struct ClassFixture : Fixture typeChecker.globalScope->exportedTypeBindings["Vector2"] = TypeFun{{}, vector2InstanceType}; addGlobalBinding(typeChecker, "Vector2", vector2Type, "@test"); + for (const auto& [name, tf] : typeChecker.globalScope->exportedTypeBindings) + persist(tf.type); + freeze(arena); } }; diff --git a/tests/TypeInfer.functions.test.cpp b/tests/TypeInfer.functions.test.cpp index 1713216a..65993681 100644 --- a/tests/TypeInfer.functions.test.cpp +++ b/tests/TypeInfer.functions.test.cpp @@ -13,6 +13,8 @@ using namespace Luau; +LUAU_FASTFLAG(LuauLowerBoundsCalculation); + TEST_SUITE_BEGIN("TypeInferFunctions"); TEST_CASE_FIXTURE(Fixture, "tc_function") @@ -98,7 +100,7 @@ TEST_CASE_FIXTURE(Fixture, "vararg_function_is_quantified") end return result - end + end return T )"); @@ -274,6 +276,10 @@ TEST_CASE_FIXTURE(Fixture, "cyclic_function_type_in_rets") TEST_CASE_FIXTURE(Fixture, "cyclic_function_type_in_args") { + ScopedFastFlag sff[] = { + {"LuauLowerBoundsCalculation", true}, + }; + CheckResult result = check(R"( function f(g) return f(f) @@ -281,7 +287,7 @@ TEST_CASE_FIXTURE(Fixture, "cyclic_function_type_in_args") )"); LUAU_REQUIRE_NO_ERRORS(result); - CHECK_EQ("t1 where t1 = (t1) -> ()", toString(requireType("f"))); + CHECK_EQ("t1 where t1 = (t1) -> (a...)", toString(requireType("f"))); } TEST_CASE_FIXTURE(Fixture, "another_higher_order_function") @@ -481,10 +487,10 @@ TEST_CASE_FIXTURE(Fixture, "infer_higher_order_function") std::vector fArgs = flatten(fType->argTypes).first; - TypeId xType = argVec[1]; + TypeId xType = follow(argVec[1]); CHECK_EQ(1, fArgs.size()); - CHECK_EQ(xType, fArgs[0]); + CHECK_EQ(xType, follow(fArgs[0])); } TEST_CASE_FIXTURE(Fixture, "higher_order_function_2") @@ -1043,13 +1049,16 @@ f(function(x) return x * 2 end) LUAU_REQUIRE_ERROR_COUNT(1, result); CHECK_EQ("Type 'number' could not be converted into 'Table'", toString(result.errors[0])); - // Return type doesn't inference 'nil' - result = check(R"( -function f(a: (number) -> nil) return a(4) end -f(function(x) print(x) end) - )"); + if (!FFlag::LuauLowerBoundsCalculation) + { + // Return type doesn't inference 'nil' + result = check(R"( + function f(a: (number) -> nil) return a(4) end + f(function(x) print(x) end) + )"); - LUAU_REQUIRE_NO_ERRORS(result); + LUAU_REQUIRE_NO_ERRORS(result); + } } TEST_CASE_FIXTURE(Fixture, "infer_anonymous_function_arguments") @@ -1142,13 +1151,16 @@ f(function(x) return x * 2 end) LUAU_REQUIRE_ERROR_COUNT(1, result); CHECK_EQ("Type 'number' could not be converted into 'Table'", toString(result.errors[0])); - // Return type doesn't inference 'nil' - result = check(R"( -function f(a: (number) -> nil) return a(4) end -f(function(x) print(x) end) - )"); + if (!FFlag::LuauLowerBoundsCalculation) + { + // Return type doesn't inference 'nil' + result = check(R"( + function f(a: (number) -> nil) return a(4) end + f(function(x) print(x) end) + )"); - LUAU_REQUIRE_NO_ERRORS(result); + LUAU_REQUIRE_NO_ERRORS(result); + } } TEST_CASE_FIXTURE(Fixture, "infer_anonymous_function_arguments_outside_call") @@ -1338,6 +1350,126 @@ end CHECK_EQ(toString(result.errors[1]), R"(Type 'string' could not be converted into 'number')"); } +TEST_CASE_FIXTURE(Fixture, "inconsistent_return_types") +{ + const ScopedFastFlag flags[] = { + {"LuauLowerBoundsCalculation", true}, + }; + + CheckResult result = check(R"( + function foo(a: boolean, b: number) + if a then + return nil + else + return b + end + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + CHECK_EQ("(boolean, number) -> number?", toString(requireType("foo"))); + + // TODO: Test multiple returns + // Think of various cases where typepacks need to grow. maybe consult other tests + // Basic normalization of ConstrainedTypeVars during quantification +} + +TEST_CASE_FIXTURE(Fixture, "inconsistent_higher_order_function") +{ + const ScopedFastFlag flags[] = { + {"LuauLowerBoundsCalculation", true}, + }; + + CheckResult result = check(R"( + function foo(f) + f(5) + f("six") + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ("((number | string) -> (a...)) -> ()", toString(requireType("foo"))); +} + + +/* The bug here is that we are using the same level 2.0 for both the body of resolveDispatcher and the + * lambda useCallback. + * + * I think what we want to do is, at each scope level, never reuse the same sublevel. + * + * We also adjust checkBlock to consider the syntax `local x = function() ... end` to be sortable + * in the same way as `local function x() ... end`. This causes the function `resolveDispatcher` to be + * checked before the lambda. + */ +TEST_CASE_FIXTURE(Fixture, "inferred_higher_order_functions_are_quantified_at_the_right_time") +{ + ScopedFastFlag sff[] = { + {"LuauLowerBoundsCalculation", true}, + }; + + CheckResult result = check(R"( + --!strict + + local function resolveDispatcher() + return (nil :: any) :: {useCallback: (any) -> any} + end + + local useCallback = function(deps: any) + return resolveDispatcher().useCallback(deps) + end + )"); + + // LUAU_REQUIRE_NO_ERRORS is particularly unhelpful when this test is broken. + // You get a TypeMismatch error where both types stringify the same. + + CHECK(result.errors.empty()); + if (!result.errors.empty()) + { + for (const auto& e : result.errors) + printf("%s: %s\n", toString(e.location).c_str(), toString(e).c_str()); + } +} + +TEST_CASE_FIXTURE(Fixture, "inferred_higher_order_functions_are_quantified_at_the_right_time2") +{ + CheckResult result = check(R"( + --!strict + + local function resolveDispatcher() + return (nil :: any) :: {useContext: (number?) -> any} + end + + local useContext + useContext = function(unstable_observedBits: number?) + resolveDispatcher().useContext(unstable_observedBits) + end + )"); + + // LUAU_REQUIRE_NO_ERRORS is particularly unhelpful when this test is broken. + // You get a TypeMismatch error where both types stringify the same. + + CHECK(result.errors.empty()); + if (!result.errors.empty()) + { + for (const auto& e : result.errors) + printf("%s %s: %s\n", e.moduleName.c_str(), toString(e.location).c_str(), toString(e).c_str()); + } +} + +TEST_CASE_FIXTURE(Fixture, "inferred_higher_order_functions_are_quantified_at_the_right_time3") +{ + CheckResult result = check(R"( + local foo + + foo():bar(function() + return foo() + end) + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + TEST_CASE_FIXTURE(Fixture, "function_decl_non_self_unsealed_overwrite") { ScopedFastFlag statFunctionSimplify{"LuauStatFunctionSimplify4", true}; @@ -1471,4 +1603,17 @@ pcall(wrapper, test) CHECK(acm->isVariadic); } +TEST_CASE_FIXTURE(Fixture, "occurs_check_failure_in_function_return_type") +{ + CheckResult result = check(R"( + function f() + return 5, f() + end + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + CHECK(nullptr != get(result.errors[0])); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.generics.test.cpp b/tests/TypeInfer.generics.test.cpp index f360a77c..49d31fc6 100644 --- a/tests/TypeInfer.generics.test.cpp +++ b/tests/TypeInfer.generics.test.cpp @@ -230,8 +230,8 @@ TEST_CASE_FIXTURE(Fixture, "infer_generic_function") CHECK_EQ(idFun->generics.size(), 1); CHECK_EQ(idFun->genericPacks.size(), 0); - CHECK_EQ(args[0], idFun->generics[0]); - CHECK_EQ(rets[0], idFun->generics[0]); + CHECK_EQ(follow(args[0]), follow(idFun->generics[0])); + CHECK_EQ(follow(rets[0]), follow(idFun->generics[0])); } TEST_CASE_FIXTURE(Fixture, "infer_generic_local_function") @@ -253,8 +253,8 @@ TEST_CASE_FIXTURE(Fixture, "infer_generic_local_function") CHECK_EQ(idFun->generics.size(), 1); CHECK_EQ(idFun->genericPacks.size(), 0); - CHECK_EQ(args[0], idFun->generics[0]); - CHECK_EQ(rets[0], idFun->generics[0]); + CHECK_EQ(follow(args[0]), follow(idFun->generics[0])); + CHECK_EQ(follow(rets[0]), follow(idFun->generics[0])); } TEST_CASE_FIXTURE(Fixture, "infer_nested_generic_function") @@ -705,10 +705,10 @@ end TEST_CASE_FIXTURE(Fixture, "generic_functions_should_be_memory_safe") { ScopedFastFlag sffs[] = { - { "LuauTableSubtypingVariance2", true }, - { "LuauUnsealedTableLiteral", true }, - { "LuauPropertiesGetExpectedType", true }, - { "LuauRecursiveTypeParameterRestriction", true }, + {"LuauTableSubtypingVariance2", true}, + {"LuauUnsealedTableLiteral", true}, + {"LuauPropertiesGetExpectedType", true}, + {"LuauRecursiveTypeParameterRestriction", true}, }; CheckResult result = check(R"( @@ -843,6 +843,7 @@ TEST_CASE_FIXTURE(Fixture, "generic_function") LUAU_REQUIRE_NO_ERRORS(result); + CHECK_EQ("(a) -> a", toString(requireType("id"))); CHECK_EQ(*typeChecker.numberType, *requireType("a")); CHECK_EQ(*typeChecker.nilType, *requireType("b")); } @@ -1037,25 +1038,39 @@ TEST_CASE_FIXTURE(Fixture, "infer_generic_function_function_argument") ScopedFastFlag sff{"LuauUnsealedTableLiteral", true}; CheckResult result = check(R"( -local function sum(x: a, y: a, f: (a, a) -> a) return f(x, y) end -return sum(2, 3, function(a, b) return a + b end) + local function sum(x: a, y: a, f: (a, a) -> a) + return f(x, y) + end + return sum(2, 3, function(a, b) return a + b end) )"); LUAU_REQUIRE_NO_ERRORS(result); result = check(R"( -local function map(arr: {a}, f: (a) -> b) local r = {} for i,v in ipairs(arr) do table.insert(r, f(v)) end return r end -local a = {1, 2, 3} -local r = map(a, function(a) return a + a > 100 end) + local function map(arr: {a}, f: (a) -> b) + local r = {} + for i,v in ipairs(arr) do + table.insert(r, f(v)) + end + return r + end + local a = {1, 2, 3} + local r = map(a, function(a) return a + a > 100 end) )"); LUAU_REQUIRE_NO_ERRORS(result); REQUIRE_EQ("{boolean}", toString(requireType("r"))); check(R"( -local function foldl(arr: {a}, init: b, f: (b, a) -> b) local r = init for i,v in ipairs(arr) do r = f(r, v) end return r end -local a = {1, 2, 3} -local r = foldl(a, {s=0,c=0}, function(a, b) return {s = a.s + b, c = a.c + 1} end) + local function foldl(arr: {a}, init: b, f: (b, a) -> b) + local r = init + for i,v in ipairs(arr) do + r = f(r, v) + end + return r + end + local a = {1, 2, 3} + local r = foldl(a, {s=0,c=0}, function(a, b) return {s = a.s + b, c = a.c + 1} end) )"); LUAU_REQUIRE_NO_ERRORS(result); @@ -1065,25 +1080,19 @@ local r = foldl(a, {s=0,c=0}, function(a, b) return {s = a.s + b, c = a.c + 1} e TEST_CASE_FIXTURE(Fixture, "infer_generic_function_function_argument_overloaded") { CheckResult result = check(R"( -local function g1(a: T, f: (T) -> T) return f(a) end -local function g2(a: T, b: T, f: (T, T) -> T) return f(a, b) end + local g12: ((T, (T) -> T) -> T) & ((T, T, (T, T) -> T) -> T) -local g12: typeof(g1) & typeof(g2) - -g12(1, function(x) return x + x end) -g12(1, 2, function(x, y) return x + y end) + g12(1, function(x) return x + x end) + g12(1, 2, function(x, y) return x + y end) )"); LUAU_REQUIRE_NO_ERRORS(result); result = check(R"( -local function g1(a: T, f: (T) -> T) return f(a) end -local function g2(a: T, b: T, f: (T, T) -> T) return f(a, b) end + local g12: ((T, (T) -> T) -> T) & ((T, T, (T, T) -> T) -> T) -local g12: typeof(g1) & typeof(g2) - -g12({x=1}, function(x) return {x=-x.x} end) -g12({x=1}, {x=2}, function(x, y) return {x=x.x + y.x} end) + g12({x=1}, function(x) return {x=-x.x} end) + g12({x=1}, {x=2}, function(x, y) return {x=x.x + y.x} end) )"); LUAU_REQUIRE_NO_ERRORS(result); @@ -1121,12 +1130,12 @@ local c = sumrec(function(x, y, f) return f(x, y) end) -- type binders are not i TEST_CASE_FIXTURE(Fixture, "substitution_with_bound_table") { CheckResult result = check(R"( -type A = { x: number } -local a: A = { x = 1 } -local b = a -type B = typeof(b) -type X = T -local c: X + type A = { x: number } + local a: A = { x = 1 } + local b = a + type B = typeof(b) + type X = T + local c: X )"); LUAU_REQUIRE_NO_ERRORS(result); diff --git a/tests/TypeInfer.intersectionTypes.test.cpp b/tests/TypeInfer.intersectionTypes.test.cpp index ac7a6532..3675919f 100644 --- a/tests/TypeInfer.intersectionTypes.test.cpp +++ b/tests/TypeInfer.intersectionTypes.test.cpp @@ -8,6 +8,8 @@ using namespace Luau; +LUAU_FASTFLAG(LuauLowerBoundsCalculation); + TEST_SUITE_BEGIN("IntersectionTypes"); TEST_CASE_FIXTURE(Fixture, "select_correct_union_fn") @@ -306,7 +308,10 @@ TEST_CASE_FIXTURE(Fixture, "table_intersection_write_sealed") )"); LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK_EQ(toString(result.errors[0]), "Cannot add property 'z' to table 'X & Y'"); + if (FFlag::LuauLowerBoundsCalculation) + CHECK_EQ(toString(result.errors[0]), "Cannot add property 'z' to table '{| x: number, y: number |}'"); + else + CHECK_EQ(toString(result.errors[0]), "Cannot add property 'z' to table 'X & Y'"); } TEST_CASE_FIXTURE(Fixture, "table_intersection_write_sealed_indirect") @@ -314,27 +319,34 @@ TEST_CASE_FIXTURE(Fixture, "table_intersection_write_sealed_indirect") ScopedFastFlag statFunctionSimplify{"LuauStatFunctionSimplify4", true}; CheckResult result = check(R"( - type X = { x: (number) -> number } - type Y = { y: (string) -> string } + type X = { x: (number) -> number } + type Y = { y: (string) -> string } - type XY = X & Y + type XY = X & Y - local xy : XY = { - x = function(a: number) return -a end, - y = function(a: string) return a .. "b" end - } - function xy.z(a:number) return a * 10 end - function xy:y(a:number) return a * 10 end - function xy:w(a:number) return a * 10 end + local xy : XY = { + x = function(a: number) return -a end, + y = function(a: string) return a .. "b" end + } + function xy.z(a:number) return a * 10 end + function xy:y(a:number) return a * 10 end + function xy:w(a:number) return a * 10 end )"); LUAU_REQUIRE_ERROR_COUNT(4, result); CHECK_EQ(toString(result.errors[0]), R"(Type '(string, number) -> string' could not be converted into '(string) -> string' caused by: Argument count mismatch. Function expects 2 arguments, but only 1 is specified)"); - CHECK_EQ(toString(result.errors[1]), "Cannot add property 'z' to table 'X & Y'"); + if (FFlag::LuauLowerBoundsCalculation) + CHECK_EQ(toString(result.errors[1]), "Cannot add property 'z' to table '{| x: (number) -> number, y: (string) -> string |}'"); + else + CHECK_EQ(toString(result.errors[1]), "Cannot add property 'z' to table 'X & Y'"); CHECK_EQ(toString(result.errors[2]), "Type 'number' could not be converted into 'string'"); - CHECK_EQ(toString(result.errors[3]), "Cannot add property 'w' to table 'X & Y'"); + + if (FFlag::LuauLowerBoundsCalculation) + CHECK_EQ(toString(result.errors[3]), "Cannot add property 'w' to table '{| x: (number) -> number, y: (string) -> string |}'"); + else + CHECK_EQ(toString(result.errors[3]), "Cannot add property 'w' to table 'X & Y'"); } TEST_CASE_FIXTURE(Fixture, "table_write_sealed_indirect") @@ -375,6 +387,8 @@ TEST_CASE_FIXTURE(Fixture, "table_intersection_setmetatable") TEST_CASE_FIXTURE(Fixture, "error_detailed_intersection_part") { + ScopedFastFlag flags[] = {{"LuauLowerBoundsCalculation", false}}; + CheckResult result = check(R"( type X = { x: number } type Y = { y: number } @@ -393,6 +407,8 @@ caused by: TEST_CASE_FIXTURE(Fixture, "error_detailed_intersection_all") { + ScopedFastFlag flags[] = {{"LuauLowerBoundsCalculation", false}}; + CheckResult result = check(R"( type X = { x: number } type Y = { y: number } @@ -427,8 +443,8 @@ TEST_CASE_FIXTURE(Fixture, "no_stack_overflow_from_flattenintersection") repeat type t0 = ((any)|((any)&((any)|((any)&((any)|(any))))))&(t0) function _(l0):(t0)&(t0) - while nil do - end + while nil do + end end until _(_)(_)._ )"); diff --git a/tests/TypeInfer.oop.test.cpp b/tests/TypeInfer.oop.test.cpp index 40831bf6..5cd3f3ba 100644 --- a/tests/TypeInfer.oop.test.cpp +++ b/tests/TypeInfer.oop.test.cpp @@ -199,16 +199,16 @@ end TEST_CASE_FIXTURE(Fixture, "nonstrict_self_mismatch_tail") { CheckResult result = check(R"( ---!nonstrict -local f = {} -function f:foo(a: number, b: number) end + --!nonstrict + local f = {} + function f:foo(a: number, b: number) end -function bar(...) - f.foo(f, 1, ...) -end + function bar(...) + f.foo(f, 1, ...) + end -bar(2) -)"); + bar(2) + )"); LUAU_REQUIRE_NO_ERRORS(result); } diff --git a/tests/TypeInfer.operators.test.cpp b/tests/TypeInfer.operators.test.cpp index 6a8a9d93..5f2e2404 100644 --- a/tests/TypeInfer.operators.test.cpp +++ b/tests/TypeInfer.operators.test.cpp @@ -91,7 +91,8 @@ TEST_CASE_FIXTURE(Fixture, "primitive_arith_no_metatable") const FunctionTypeVar* functionType = get(requireType("add")); std::optional retType = first(functionType->retType); - CHECK_EQ(std::optional(typeChecker.numberType), retType); + REQUIRE(retType.has_value()); + CHECK_EQ(typeChecker.numberType, follow(*retType)); CHECK_EQ(requireType("n"), typeChecker.numberType); CHECK_EQ(requireType("s"), typeChecker.stringType); } diff --git a/tests/TypeInfer.provisional.test.cpp b/tests/TypeInfer.provisional.test.cpp index 2e16b21e..6b3741fa 100644 --- a/tests/TypeInfer.provisional.test.cpp +++ b/tests/TypeInfer.provisional.test.cpp @@ -8,6 +8,7 @@ #include LUAU_FASTFLAG(LuauEqConstraint) +LUAU_FASTFLAG(LuauLowerBoundsCalculation) using namespace Luau; @@ -527,6 +528,7 @@ TEST_CASE_FIXTURE(Fixture, "invariant_table_properties_means_instantiating_table LUAU_REQUIRE_NO_ERRORS(result); } +// FIXME: Move this test to another source file when removing FFlag::LuauLowerBoundsCalculation TEST_CASE_FIXTURE(Fixture, "do_not_ice_when_trying_to_pick_first_of_generic_type_pack") { ScopedFastFlag sff[]{ @@ -556,10 +558,19 @@ TEST_CASE_FIXTURE(Fixture, "do_not_ice_when_trying_to_pick_first_of_generic_type LUAU_REQUIRE_NO_ERRORS(result); - // f and g should have the type () -> () - CHECK_EQ("() -> (a...)", toString(requireType("f"))); - CHECK_EQ("() -> (a...)", toString(requireType("g"))); - CHECK_EQ("any", toString(requireType("x"))); // any is returned instead of ICE for now + if (FFlag::LuauLowerBoundsCalculation) + { + CHECK_EQ("() -> ()", toString(requireType("f"))); + CHECK_EQ("() -> ()", toString(requireType("g"))); + CHECK_EQ("nil", toString(requireType("x"))); + } + else + { + // f and g should have the type () -> () + CHECK_EQ("() -> (a...)", toString(requireType("f"))); + CHECK_EQ("() -> (a...)", toString(requireType("g"))); + CHECK_EQ("any", toString(requireType("x"))); // any is returned instead of ICE for now + } } TEST_CASE_FIXTURE(Fixture, "specialization_binds_with_prototypes_too_early") @@ -575,6 +586,10 @@ TEST_CASE_FIXTURE(Fixture, "specialization_binds_with_prototypes_too_early") TEST_CASE_FIXTURE(Fixture, "weird_fail_to_unify_type_pack") { + ScopedFastFlag sff[] = { + {"LuauLowerBoundsCalculation", false}, + }; + CheckResult result = check(R"( local function f() return end local g = function() return f() end @@ -585,6 +600,10 @@ TEST_CASE_FIXTURE(Fixture, "weird_fail_to_unify_type_pack") TEST_CASE_FIXTURE(Fixture, "weird_fail_to_unify_variadic_pack") { + ScopedFastFlag sff[] = { + {"LuauLowerBoundsCalculation", false}, + }; + CheckResult result = check(R"( --!strict local function f(...) return ... end @@ -594,4 +613,112 @@ TEST_CASE_FIXTURE(Fixture, "weird_fail_to_unify_variadic_pack") LUAU_REQUIRE_ERRORS(result); // Should not have any errors. } +TEST_CASE_FIXTURE(Fixture, "lower_bounds_calculation_is_too_permissive_with_overloaded_higher_order_functions") +{ + ScopedFastFlag sff[] = { + {"LuauLowerBoundsCalculation", true}, + }; + + CheckResult result = check(R"( + function foo(f) + f(5, 'a') + f('b', 6) + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + // We incorrectly infer that the argument to foo could be called with (number, number) or (string, string) + // even though that is strictly more permissive than the actual source text shows. + CHECK("((number | string, number | string) -> (a...)) -> ()" == toString(requireType("foo"))); +} + +// Once fixed, move this to Normalize.test.cpp +TEST_CASE_FIXTURE(Fixture, "normalization_fails_on_certain_kinds_of_cyclic_tables") +{ +#if defined(_DEBUG) || defined(_NOOPT) + ScopedFastInt sfi("LuauNormalizeIterationLimit", 500); +#endif + + ScopedFastFlag flags[] = { + {"LuauLowerBoundsCalculation", true}, + }; + + // We use a function and inferred parameter types to prevent intermediate normalizations from being performed. + // This exposes a bug where the type of y is mutated. + CheckResult result = check(R"( + function strange(x, y) + x.x = y + y.x = x + + type R = {x: typeof(x)} & {x: typeof(y)} + local r: R + + return r + end + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + CHECK(nullptr != get(result.errors[0])); +} + +// Belongs in TypeInfer.builtins.test.cpp. +TEST_CASE_FIXTURE(Fixture, "pcall_returns_at_least_two_value_but_function_returns_nothing") +{ + CheckResult result = check(R"( + local function f(): () end + local ok, res = pcall(f) + )"); + + LUAU_REQUIRE_ERRORS(result); + // LUAU_REQUIRE_NO_ERRORS(result); + // CHECK_EQ("boolean", toString(requireType("ok"))); + // CHECK_EQ("any", toString(requireType("res"))); +} + +// Belongs in TypeInfer.builtins.test.cpp. +TEST_CASE_FIXTURE(Fixture, "choose_the_right_overload_for_pcall") +{ + CheckResult result = check(R"( + local function f(): number + if math.random() > 0.5 then + return 5 + else + error("something") + end + end + + local ok, res = pcall(f) + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + CHECK_EQ("boolean", toString(requireType("ok"))); + CHECK_EQ("number", toString(requireType("res"))); + // CHECK_EQ("any", toString(requireType("res"))); +} + +// Belongs in TypeInfer.builtins.test.cpp. +TEST_CASE_FIXTURE(Fixture, "function_returns_many_things_but_first_of_it_is_forgotten") +{ + CheckResult result = check(R"( + local function f(): (number, string, boolean) + if math.random() > 0.5 then + return 5, "hello", true + else + error("something") + end + end + + local ok, res, s, b = pcall(f) + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + CHECK_EQ("boolean", toString(requireType("ok"))); + CHECK_EQ("number", toString(requireType("res"))); + // CHECK_EQ("any", toString(requireType("res"))); + CHECK_EQ("string", toString(requireType("s"))); + CHECK_EQ("boolean", toString(requireType("b"))); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.refinements.test.cpp b/tests/TypeInfer.refinements.test.cpp index cddeab6e..ce22bcb1 100644 --- a/tests/TypeInfer.refinements.test.cpp +++ b/tests/TypeInfer.refinements.test.cpp @@ -1,4 +1,5 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +#include "Luau/Normalize.h" #include "Luau/Scope.h" #include "Luau/TypeInfer.h" @@ -8,6 +9,7 @@ LUAU_FASTFLAG(LuauDiscriminableUnions2) LUAU_FASTFLAG(LuauWeakEqConstraint) +LUAU_FASTFLAG(LuauLowerBoundsCalculation) using namespace Luau; @@ -48,6 +50,7 @@ struct RefinementClassFixture : Fixture {"Y", Property{typeChecker.numberType}}, {"Z", Property{typeChecker.numberType}}, }; + normalize(vec3, arena, *typeChecker.iceHandler); TypeId inst = arena.addType(ClassTypeVar{"Instance", {}, std::nullopt, std::nullopt, {}, nullptr}); @@ -55,17 +58,21 @@ struct RefinementClassFixture : Fixture TypePackId isARets = arena.addTypePack({typeChecker.booleanType}); TypeId isA = arena.addType(FunctionTypeVar{isAParams, isARets}); getMutable(isA)->magicFunction = magicFunctionInstanceIsA; + normalize(isA, arena, *typeChecker.iceHandler); getMutable(inst)->props = { {"Name", Property{typeChecker.stringType}}, {"IsA", Property{isA}}, }; + normalize(inst, arena, *typeChecker.iceHandler); TypeId folder = typeChecker.globalTypes.addType(ClassTypeVar{"Folder", {}, inst, std::nullopt, {}, nullptr}); + normalize(folder, arena, *typeChecker.iceHandler); TypeId part = typeChecker.globalTypes.addType(ClassTypeVar{"Part", {}, inst, std::nullopt, {}, nullptr}); getMutable(part)->props = { {"Position", Property{vec3}}, }; + normalize(part, arena, *typeChecker.iceHandler); typeChecker.globalScope->exportedTypeBindings["Vector3"] = TypeFun{{}, vec3}; typeChecker.globalScope->exportedTypeBindings["Instance"] = TypeFun{{}, inst}; @@ -697,7 +704,10 @@ TEST_CASE_FIXTURE(Fixture, "type_guard_can_filter_for_intersection_of_tables") LUAU_REQUIRE_NO_ERRORS(result); - CHECK_EQ("{| x: number |} & {| y: number |}", toString(requireTypeAtPosition({4, 28}))); + if (FFlag::LuauLowerBoundsCalculation) + CHECK_EQ("{| x: number, y: number |}", toString(requireTypeAtPosition({4, 28}))); + else + CHECK_EQ("{| x: number |} & {| y: number |}", toString(requireTypeAtPosition({4, 28}))); CHECK_EQ("nil", toString(requireTypeAtPosition({6, 28}))); } diff --git a/tests/TypeInfer.singletons.test.cpp b/tests/TypeInfer.singletons.test.cpp index d39341ea..2b01c29e 100644 --- a/tests/TypeInfer.singletons.test.cpp +++ b/tests/TypeInfer.singletons.test.cpp @@ -5,8 +5,6 @@ #include "doctest.h" #include "Luau/BuiltinDefinitions.h" -LUAU_FASTFLAG(BetterDiagnosticCodesInStudio) - using namespace Luau; TEST_SUITE_BEGIN("TypeSingletons"); @@ -261,14 +259,7 @@ TEST_CASE_FIXTURE(Fixture, "table_properties_alias_or_parens_is_indexer") )"); LUAU_REQUIRE_ERROR_COUNT(1, result); - if (FFlag::BetterDiagnosticCodesInStudio) - { - CHECK_EQ("Cannot have more than one table indexer", toString(result.errors[0])); - } - else - { - CHECK_EQ("Syntax error: Cannot have more than one table indexer", toString(result.errors[0])); - } + CHECK_EQ("Cannot have more than one table indexer", toString(result.errors[0])); } TEST_CASE_FIXTURE(Fixture, "table_properties_type_error_escapes") diff --git a/tests/TypeInfer.tables.test.cpp b/tests/TypeInfer.tables.test.cpp index 0484351d..ca1b8de7 100644 --- a/tests/TypeInfer.tables.test.cpp +++ b/tests/TypeInfer.tables.test.cpp @@ -11,6 +11,8 @@ using namespace Luau; +LUAU_FASTFLAG(LuauLowerBoundsCalculation); + TEST_SUITE_BEGIN("TableTests"); TEST_CASE_FIXTURE(Fixture, "basic") @@ -1211,7 +1213,10 @@ TEST_CASE_FIXTURE(Fixture, "pass_incompatible_union_to_a_generic_table_without_c )"); LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK(get(result.errors[0])); + if (FFlag::LuauLowerBoundsCalculation) + CHECK(get(result.errors[0])); + else + CHECK(get(result.errors[0])); } // This unit test could be flaky if the fix has regressed. @@ -2922,6 +2927,60 @@ TEST_CASE_FIXTURE(Fixture, "inferred_properties_of_a_table_should_start_with_the LUAU_REQUIRE_NO_ERRORS(result); } +// The real bug here was that we weren't always uncondionally typechecking a trailing return statement last. +TEST_CASE_FIXTURE(Fixture, "dont_leak_free_table_props") +{ + CheckResult result = check(R"( + local function a(state) + print(state.blah) + end + + local function b(state) -- The bug was that we inferred state: {blah: any, gwar: any} + print(state.gwar) + end + + return function() + return function(state) + a(state) + b(state) + end + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ("({+ blah: a +}) -> ()", toString(requireType("a"))); + CHECK_EQ("({+ gwar: a +}) -> ()", toString(requireType("b"))); + CHECK_EQ("() -> ({+ blah: a, gwar: b +}) -> ()", toString(getMainModule()->getModuleScope()->returnType)); +} + +TEST_CASE_FIXTURE(Fixture, "inferred_return_type_of_free_table") +{ + ScopedFastFlag sff[] = { + {"LuauLowerBoundsCalculation", true}, + }; + + check(R"( + function Base64FileReader(data) + local reader = {} + local index: number + + function reader:PeekByte() + return data:byte(index) + end + + function reader:Byte() + return data:byte(index - 1) + end + + return reader + end + )"); + + CHECK_EQ("(t1) -> {| Byte: (b) -> (a...), PeekByte: (c) -> (a...) |} where t1 = {+ byte: (t1, number) -> (a...) +}", + toString(requireType("Base64FileReader"))); +} + TEST_CASE_FIXTURE(Fixture, "mixed_tables_with_implicit_numbered_keys") { ScopedFastFlag sff{"LuauCheckImplicitNumbericKeys", true}; diff --git a/tests/TypeInfer.test.cpp b/tests/TypeInfer.test.cpp index 660ddcfc..6abd96b9 100644 --- a/tests/TypeInfer.test.cpp +++ b/tests/TypeInfer.test.cpp @@ -13,6 +13,7 @@ #include +LUAU_FASTFLAG(LuauLowerBoundsCalculation) LUAU_FASTFLAG(LuauFixLocationSpanTableIndexExpr) LUAU_FASTFLAG(LuauEqConstraint) @@ -177,7 +178,6 @@ TEST_CASE_FIXTURE(Fixture, "weird_case") )"); LUAU_REQUIRE_NO_ERRORS(result); - dumpErrors(result); } TEST_CASE_FIXTURE(Fixture, "dont_ice_when_failing_the_occurs_check") @@ -293,7 +293,7 @@ TEST_CASE_FIXTURE(Fixture, "exponential_blowup_from_copying_types") // In these tests, a successful parse is required, so we need the parser to return the AST and then we can test the recursion depth limit in type // checker. We also want it to somewhat match up with production values, so we push up the parser recursion limit a little bit instead. -TEST_CASE_FIXTURE(Fixture, "check_type_infer_recursion_limit") +TEST_CASE_FIXTURE(Fixture, "check_type_infer_recursion_count") { #if defined(LUAU_ENABLE_ASAN) int limit = 250; @@ -302,12 +302,14 @@ TEST_CASE_FIXTURE(Fixture, "check_type_infer_recursion_limit") #else int limit = 600; #endif - ScopedFastInt luauRecursionLimit{"LuauRecursionLimit", limit + 100}; - ScopedFastInt luauTypeInferRecursionLimit{"LuauTypeInferRecursionLimit", limit - 100}; - ScopedFastInt luauCheckRecursionLimit{"LuauCheckRecursionLimit", 0}; - CHECK_NOTHROW(check("print('Hello!')")); - CHECK_THROWS_AS(check("function f() return " + rep("{a=", limit) + "'a'" + rep("}", limit) + " end"), std::runtime_error); + ScopedFastFlag sff{"LuauTableUseCounterInstead", true}; + ScopedFastInt sfi{"LuauCheckRecursionLimit", limit}; + + CheckResult result = check("function f() return " + rep("{a=", limit) + "'a'" + rep("}", limit) + " end"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK(nullptr != get(result.errors[0])); } TEST_CASE_FIXTURE(Fixture, "check_block_recursion_limit") @@ -721,9 +723,9 @@ TEST_CASE_FIXTURE(Fixture, "no_heap_use_after_free_error") local l0 do end while _ do - function _:_() - _ += _(_._(_:n0(xpcall,_))) - end + function _:_() + _ += _(_._(_:n0(xpcall,_))) + end end )"); @@ -978,4 +980,48 @@ TEST_CASE_FIXTURE(Fixture, "cli_50041_committing_txnlog_in_apollo_client_error") LUAU_REQUIRE_NO_ERRORS(result); } +TEST_CASE_FIXTURE(Fixture, "type_infer_recursion_limit_no_ice") +{ + ScopedFastInt sfi("LuauTypeInferRecursionLimit", 2); + ScopedFastFlag sff{"LuauRecursionLimitException", true}; + + CheckResult result = check(R"( + function complex() + function _(l0:t0): (any, ()->()) + return 0,_ + end + type t0 = t0 | {} + _(nil) + end + )"); + + LUAU_REQUIRE_ERRORS(result); + CHECK_EQ("Code is too complex to typecheck! Consider simplifying the code around this area", toString(result.errors[0])); +} + +TEST_CASE_FIXTURE(Fixture, "follow_on_new_types_in_substitution") +{ + ScopedFastFlag substituteFollowNewTypes{"LuauSubstituteFollowNewTypes", true}; + + CheckResult result = check(R"( + local obj = {} + + function obj:Method() + self.fieldA = function(object) + if object.a then + self.arr[object] = true + elseif object.b then + self.fieldB[object] = object:Connect(function(arg) + self.arr[arg] = nil + end) + end + end + end + + return obj + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.typePacks.cpp b/tests/TypeInfer.typePacks.cpp index 130f33d7..f141622f 100644 --- a/tests/TypeInfer.typePacks.cpp +++ b/tests/TypeInfer.typePacks.cpp @@ -9,6 +9,8 @@ using namespace Luau; +LUAU_FASTFLAG(LuauLowerBoundsCalculation); + TEST_SUITE_BEGIN("TypePackTests"); TEST_CASE_FIXTURE(Fixture, "infer_multi_return") @@ -27,8 +29,8 @@ TEST_CASE_FIXTURE(Fixture, "infer_multi_return") const auto& [returns, tail] = flatten(takeTwoType->retType); CHECK_EQ(2, returns.size()); - CHECK_EQ(typeChecker.numberType, returns[0]); - CHECK_EQ(typeChecker.numberType, returns[1]); + CHECK_EQ(typeChecker.numberType, follow(returns[0])); + CHECK_EQ(typeChecker.numberType, follow(returns[1])); CHECK(!tail); } @@ -74,9 +76,9 @@ TEST_CASE_FIXTURE(Fixture, "last_element_of_return_statement_can_itself_be_a_pac const auto& [rets, tail] = flatten(takeOneMoreType->retType); REQUIRE_EQ(3, rets.size()); - CHECK_EQ(typeChecker.numberType, rets[0]); - CHECK_EQ(typeChecker.numberType, rets[1]); - CHECK_EQ(typeChecker.numberType, rets[2]); + CHECK_EQ(typeChecker.numberType, follow(rets[0])); + CHECK_EQ(typeChecker.numberType, follow(rets[1])); + CHECK_EQ(typeChecker.numberType, follow(rets[2])); CHECK(!tail); } @@ -91,26 +93,7 @@ TEST_CASE_FIXTURE(Fixture, "higher_order_function") LUAU_REQUIRE_NO_ERRORS(result); - const FunctionTypeVar* applyType = get(requireType("apply")); - REQUIRE(applyType != nullptr); - - std::vector applyArgs = flatten(applyType->argTypes).first; - REQUIRE_EQ(3, applyArgs.size()); - - const FunctionTypeVar* fType = get(follow(applyArgs[0])); - REQUIRE(fType != nullptr); - - const FunctionTypeVar* gType = get(follow(applyArgs[1])); - REQUIRE(gType != nullptr); - - std::vector gArgs = flatten(gType->argTypes).first; - REQUIRE_EQ(1, gArgs.size()); - - // function(function(t1, T2...): (t3, T4...), function(t5): (t1, T2...), t5): (t3, T4...) - - REQUIRE_EQ(*gArgs[0], *applyArgs[2]); - REQUIRE_EQ(toString(fType->argTypes), toString(gType->retType)); - REQUIRE_EQ(toString(fType->retType), toString(applyType->retType)); + CHECK_EQ("((b...) -> (c...), (a) -> (b...), a) -> (c...)", toString(requireType("apply"))); } TEST_CASE_FIXTURE(Fixture, "return_type_should_be_empty_if_nothing_is_returned") @@ -328,7 +311,10 @@ local c: Packed auto ttvA = get(requireType("a")); REQUIRE(ttvA); CHECK_EQ(toString(requireType("a")), "Packed"); - CHECK_EQ(toString(requireType("a"), {true}), "{| f: (number) -> (number) |}"); + if (FFlag::LuauLowerBoundsCalculation) + CHECK_EQ(toString(requireType("a"), {true}), "{| f: (number) -> number |}"); + else + CHECK_EQ(toString(requireType("a"), {true}), "{| f: (number) -> (number) |}"); REQUIRE(ttvA->instantiatedTypeParams.size() == 1); REQUIRE(ttvA->instantiatedTypePackParams.size() == 1); CHECK_EQ(toString(ttvA->instantiatedTypeParams[0], {true}), "number"); diff --git a/tests/TypeInfer.unionTypes.test.cpp b/tests/TypeInfer.unionTypes.test.cpp index ff207a18..96bdd534 100644 --- a/tests/TypeInfer.unionTypes.test.cpp +++ b/tests/TypeInfer.unionTypes.test.cpp @@ -6,6 +6,7 @@ #include "doctest.h" +LUAU_FASTFLAG(LuauLowerBoundsCalculation) LUAU_FASTFLAG(LuauEqConstraint) using namespace Luau; @@ -254,11 +255,11 @@ local c = bf.a.y TEST_CASE_FIXTURE(Fixture, "optional_union_functions") { CheckResult result = check(R"( -local a = {} -function a.foo(x:number, y:number) return x + y end -type A = typeof(a) -local b: A? = a -local c = b.foo(1, 2) + local a = {} + function a.foo(x:number, y:number) return x + y end + type A = typeof(a) + local b: A? = a + local c = b.foo(1, 2) )"); LUAU_REQUIRE_ERROR_COUNT(1, result); @@ -356,7 +357,10 @@ a.x = 2 )"); LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK_EQ("Value of type '({| x: number |} & {| y: number |})?' could be nil", toString(result.errors[0])); + if (FFlag::LuauLowerBoundsCalculation) + CHECK_EQ("Value of type '{| x: number, y: number |}?' could be nil", toString(result.errors[0])); + else + CHECK_EQ("Value of type '({| x: number |} & {| y: number |})?' could be nil", toString(result.errors[0])); } TEST_CASE_FIXTURE(Fixture, "optional_length_error") @@ -533,8 +537,13 @@ TEST_CASE_FIXTURE(Fixture, "table_union_write_indirect") LUAU_REQUIRE_ERROR_COUNT(1, result); // NOTE: union normalization will improve this message - CHECK_EQ(toString(result.errors[0]), - R"(Type '(string) -> number' could not be converted into '((number) -> string) | ((number) -> string)'; none of the union options are compatible)"); + if (FFlag::LuauLowerBoundsCalculation) + CHECK_EQ(toString(result.errors[0]), "Type '(string) -> number' could not be converted into '(number) -> string'\n" + "caused by:\n" + " Argument #1 type is not compatible. Type 'number' could not be converted into 'string'"); + else + CHECK_EQ(toString(result.errors[0]), + R"(Type '(string) -> number' could not be converted into '((number) -> string) | ((number) -> string)'; none of the union options are compatible)"); } diff --git a/tests/conformance/nextvar.lua b/tests/conformance/nextvar.lua index ab9be42c..c8176456 100644 --- a/tests/conformance/nextvar.lua +++ b/tests/conformance/nextvar.lua @@ -581,4 +581,19 @@ do assert(#arr == 5) end +-- test boundary invariant maintenance when table is filled using SETLIST opcode +do + local arr = {[2]=2,1} + assert(#arr == 2) +end + +-- test boundary invariant maintenance when table is filled using table.move +do + local t1 = {1, 2, 3, 4, 5} + local t2 = {[6] = 6} + + table.move(t1, 1, 5, 1, t2) + assert(#t2 == 6) +end + return"OK" From 5bb9f379b07e378db0a170e7c4030e3a943b2f14 Mon Sep 17 00:00:00 2001 From: Alan Jeffrey <403333+asajeffrey@users.noreply.github.com> Date: Fri, 15 Apr 2022 19:19:42 -0500 Subject: [PATCH 225/399] Unified strict and nonstrict mode in the prototype (#458) --- prototyping/Luau/StrictMode.agda | 9 +++---- prototyping/Luau/StrictMode/ToString.agda | 4 ++-- prototyping/Luau/Type.agda | 29 ++++++++--------------- prototyping/Luau/TypeCheck.agda | 9 ++----- prototyping/Properties/StrictMode.agda | 8 +++---- prototyping/Properties/Subtyping.agda | 4 +--- prototyping/Properties/TypeCheck.agda | 11 +++------ 7 files changed, 24 insertions(+), 50 deletions(-) diff --git a/prototyping/Luau/StrictMode.agda b/prototyping/Luau/StrictMode.agda index b6769f01..1b028042 100644 --- a/prototyping/Luau/StrictMode.agda +++ b/prototyping/Luau/StrictMode.agda @@ -5,18 +5,15 @@ module Luau.StrictMode where open import Agda.Builtin.Equality using (_≡_) open import FFI.Data.Maybe using (just; nothing) open import Luau.Syntax using (Expr; Stat; Block; BinaryOperator; yes; nil; addr; var; binexp; var_∈_; _⟨_⟩∈_; function_is_end; _$_; block_is_end; local_←_; _∙_; done; return; name; +; -; *; /; <; >; <=; >=; ··) -open import Luau.Type using (Type; strict; nil; number; string; boolean; _⇒_; _∪_; _∩_; tgt) +open import Luau.Type using (Type; nil; number; string; boolean; _⇒_; _∪_; _∩_; src; tgt) open import Luau.Subtyping using (_≮:_) open import Luau.Heap using (Heap; function_is_end) renaming (_[_] to _[_]ᴴ) open import Luau.VarCtxt using (VarCtxt; ∅; _⋒_; _↦_; _⊕_↦_; _⊝_) renaming (_[_] to _[_]ⱽ) -open import Luau.TypeCheck(strict) using (_⊢ᴮ_∈_; _⊢ᴱ_∈_; ⊢ᴴ_; ⊢ᴼ_; _⊢ᴴᴱ_▷_∈_; _⊢ᴴᴮ_▷_∈_; var; addr; app; binexp; block; return; local; function; srcBinOp) +open import Luau.TypeCheck using (_⊢ᴮ_∈_; _⊢ᴱ_∈_; ⊢ᴴ_; ⊢ᴼ_; _⊢ᴴᴱ_▷_∈_; _⊢ᴴᴮ_▷_∈_; var; addr; app; binexp; block; return; local; function; srcBinOp) open import Properties.Contradiction using (¬) -open import Properties.TypeCheck(strict) using (typeCheckᴮ) +open import Properties.TypeCheck using (typeCheckᴮ) open import Properties.Product using (_,_) -src : Type → Type -src = Luau.Type.src strict - data Warningᴱ (H : Heap yes) {Γ} : ∀ {M T} → (Γ ⊢ᴱ M ∈ T) → Set data Warningᴮ (H : Heap yes) {Γ} : ∀ {B T} → (Γ ⊢ᴮ B ∈ T) → Set diff --git a/prototyping/Luau/StrictMode/ToString.agda b/prototyping/Luau/StrictMode/ToString.agda index 08ee13b8..eee5722e 100644 --- a/prototyping/Luau/StrictMode/ToString.agda +++ b/prototyping/Luau/StrictMode/ToString.agda @@ -7,8 +7,8 @@ open import FFI.Data.String using (String; _++_) open import Luau.Subtyping using (_≮:_; Tree; witness; scalar; function; function-ok; function-err) open import Luau.StrictMode using (Warningᴱ; Warningᴮ; UnallocatedAddress; UnboundVariable; FunctionCallMismatch; FunctionDefnMismatch; BlockMismatch; app₁; app₂; BinOpMismatch₁; BinOpMismatch₂; bin₁; bin₂; block₁; return; LocalVarMismatch; local₁; local₂; function₁; function₂; heap; expr; block; addr) open import Luau.Syntax using (Expr; val; yes; var; var_∈_; _⟨_⟩∈_; _$_; addr; number; binexp; nil; function_is_end; block_is_end; done; return; local_←_; _∙_; fun; arg; name) -open import Luau.Type using (strict; number; boolean; string; nil) -open import Luau.TypeCheck(strict) using (_⊢ᴮ_∈_; _⊢ᴱ_∈_) +open import Luau.Type using (number; boolean; string; nil) +open import Luau.TypeCheck using (_⊢ᴮ_∈_; _⊢ᴱ_∈_) open import Luau.Addr.ToString using (addrToString) open import Luau.Var.ToString using (varToString) open import Luau.Type.ToString using (typeToString) diff --git a/prototyping/Luau/Type.agda b/prototyping/Luau/Type.agda index 30c45388..59d1107f 100644 --- a/prototyping/Luau/Type.agda +++ b/prototyping/Luau/Type.agda @@ -146,25 +146,16 @@ just T ≡ᴹᵀ just U with T ≡ᵀ U (just T ≡ᴹᵀ just T) | yes refl = yes refl (just T ≡ᴹᵀ just U) | no p = no (λ q → p (just-inv q)) -data Mode : Set where - strict : Mode - nonstrict : Mode - -src : Mode → Type → Type -src m nil = never -src m number = never -src m boolean = never -src m string = never -src m (S ⇒ T) = S --- In nonstrict mode, functions are covaraiant, in strict mode they're contravariant -src strict (S ∪ T) = (src strict S) ∩ (src strict T) -src nonstrict (S ∪ T) = (src nonstrict S) ∪ (src nonstrict T) -src strict (S ∩ T) = (src strict S) ∪ (src strict T) -src nonstrict (S ∩ T) = (src nonstrict S) ∩ (src nonstrict T) -src strict never = unknown -src nonstrict never = never -src strict unknown = never -src nonstrict unknown = unknown +src : Type → Type +src nil = never +src number = never +src boolean = never +src string = never +src (S ⇒ T) = S +src (S ∪ T) = (src S) ∩ (src T) +src (S ∩ T) = (src S) ∪ (src T) +src never = unknown +src unknown = never tgt : Type → Type tgt nil = never diff --git a/prototyping/Luau/TypeCheck.agda b/prototyping/Luau/TypeCheck.agda index aea6507a..cabd27a8 100644 --- a/prototyping/Luau/TypeCheck.agda +++ b/prototyping/Luau/TypeCheck.agda @@ -1,8 +1,6 @@ {-# OPTIONS --rewriting #-} -open import Luau.Type using (Mode) - -module Luau.TypeCheck (m : Mode) where +module Luau.TypeCheck where open import Agda.Builtin.Equality using (_≡_) open import FFI.Data.Maybe using (Maybe; just) @@ -10,15 +8,12 @@ open import Luau.Syntax using (Expr; Stat; Block; BinaryOperator; yes; nil; addr open import Luau.Var using (Var) open import Luau.Addr using (Addr) open import Luau.Heap using (Heap; Object; function_is_end) renaming (_[_] to _[_]ᴴ) -open import Luau.Type using (Type; Mode; nil; unknown; number; boolean; string; _⇒_; tgt) +open import Luau.Type using (Type; nil; unknown; number; boolean; string; _⇒_; src; tgt) open import Luau.VarCtxt using (VarCtxt; ∅; _⋒_; _↦_; _⊕_↦_; _⊝_) renaming (_[_] to _[_]ⱽ) open import FFI.Data.Vector using (Vector) open import FFI.Data.Maybe using (Maybe; just; nothing) open import Properties.Product using (_×_; _,_) -src : Type → Type -src = Luau.Type.src m - orUnknown : Maybe Type → Type orUnknown nothing = unknown orUnknown (just T) = T diff --git a/prototyping/Properties/StrictMode.agda b/prototyping/Properties/StrictMode.agda index 1165fdaa..fd2cf2f2 100644 --- a/prototyping/Properties/StrictMode.agda +++ b/prototyping/Properties/StrictMode.agda @@ -11,8 +11,8 @@ open import Luau.StrictMode using (Warningᴱ; Warningᴮ; Warningᴼ; Warning open import Luau.Substitution using (_[_/_]ᴮ; _[_/_]ᴱ; _[_/_]ᴮunless_; var_[_/_]ᴱwhenever_) open import Luau.Subtyping using (_≮:_; witness; unknown; never; scalar; function; scalar-function; scalar-function-ok; scalar-function-err; scalar-scalar; function-scalar; function-ok; function-err; left; right; _,_; Tree; Language; ¬Language) open import Luau.Syntax using (Expr; yes; var; val; var_∈_; _⟨_⟩∈_; _$_; addr; number; bool; string; binexp; nil; function_is_end; block_is_end; done; return; local_←_; _∙_; fun; arg; name; ==; ~=) -open import Luau.Type using (Type; strict; nil; number; boolean; string; _⇒_; never; unknown; _∩_; _∪_; tgt; _≡ᵀ_; _≡ᴹᵀ_) -open import Luau.TypeCheck(strict) using (_⊢ᴮ_∈_; _⊢ᴱ_∈_; _⊢ᴴᴮ_▷_∈_; _⊢ᴴᴱ_▷_∈_; nil; var; addr; app; function; block; done; return; local; orUnknown; srcBinOp; tgtBinOp) +open import Luau.Type using (Type; nil; number; boolean; string; _⇒_; never; unknown; _∩_; _∪_; src; tgt; _≡ᵀ_; _≡ᴹᵀ_) +open import Luau.TypeCheck using (_⊢ᴮ_∈_; _⊢ᴱ_∈_; _⊢ᴴᴮ_▷_∈_; _⊢ᴴᴱ_▷_∈_; nil; var; addr; app; function; block; done; return; local; orUnknown; srcBinOp; tgtBinOp) open import Luau.Var using (_≡ⱽ_) open import Luau.Addr using (_≡ᴬ_) open import Luau.VarCtxt using (VarCtxt; ∅; _⋒_; _↦_; _⊕_↦_; _⊝_; ⊕-lookup-miss; ⊕-swap; ⊕-over) renaming (_[_] to _[_]ⱽ) @@ -23,13 +23,11 @@ open import Properties.Dec using (Dec; yes; no) open import Properties.Contradiction using (CONTRADICTION; ¬) open import Properties.Functions using (_∘_) open import Properties.Subtyping using (unknown-≮:; ≡-trans-≮:; ≮:-trans-≡; never-tgt-≮:; tgt-never-≮:; src-unknown-≮:; unknown-src-≮:; ≮:-trans; ≮:-refl; scalar-≢-impl-≮:; function-≮:-scalar; scalar-≮:-function; function-≮:-never; unknown-≮:-scalar; scalar-≮:-never; unknown-≮:-never) -open import Properties.TypeCheck(strict) using (typeOfᴼ; typeOfᴹᴼ; typeOfⱽ; typeOfᴱ; typeOfᴮ; typeCheckᴱ; typeCheckᴮ; typeCheckᴼ; typeCheckᴴ) +open import Properties.TypeCheck using (typeOfᴼ; typeOfᴹᴼ; typeOfⱽ; typeOfᴱ; typeOfᴮ; typeCheckᴱ; typeCheckᴮ; typeCheckᴼ; typeCheckᴴ) open import Luau.OpSem using (_⟦_⟧_⟶_; _⊢_⟶*_⊣_; _⊢_⟶ᴮ_⊣_; _⊢_⟶ᴱ_⊣_; app₁; app₂; function; beta; return; block; done; local; subst; binOp₀; binOp₁; binOp₂; refl; step; +; -; *; /; <; >; ==; ~=; <=; >=; ··) open import Luau.RuntimeError using (BinOpError; RuntimeErrorᴱ; RuntimeErrorᴮ; FunctionMismatch; BinOpMismatch₁; BinOpMismatch₂; UnboundVariable; SEGV; app₁; app₂; bin₁; bin₂; block; local; return; +; -; *; /; <; >; <=; >=; ··) open import Luau.RuntimeType using (RuntimeType; valueType; number; string; boolean; nil; function) -src = Luau.Type.src strict - data _⊑_ (H : Heap yes) : Heap yes → Set where refl : (H ⊑ H) snoc : ∀ {H′ a O} → (H′ ≡ᴴ H ⊕ a ↦ O) → (H ⊑ H′) diff --git a/prototyping/Properties/Subtyping.agda b/prototyping/Properties/Subtyping.agda index cc6bb5c1..b713eaf7 100644 --- a/prototyping/Properties/Subtyping.agda +++ b/prototyping/Properties/Subtyping.agda @@ -5,14 +5,12 @@ module Properties.Subtyping where open import Agda.Builtin.Equality using (_≡_; refl) open import FFI.Data.Either using (Either; Left; Right; mapLR; swapLR; cond) open import Luau.Subtyping using (_<:_; _≮:_; Tree; Language; ¬Language; witness; unknown; never; scalar; function; scalar-function; scalar-function-ok; scalar-function-err; scalar-scalar; function-scalar; function-ok; function-err; left; right; _,_) -open import Luau.Type using (Type; Scalar; strict; nil; number; string; boolean; never; unknown; _⇒_; _∪_; _∩_; tgt) +open import Luau.Type using (Type; Scalar; nil; number; string; boolean; never; unknown; _⇒_; _∪_; _∩_; src; tgt) open import Properties.Contradiction using (CONTRADICTION; ¬) open import Properties.Equality using (_≢_) open import Properties.Functions using (_∘_) open import Properties.Product using (_×_; _,_) -src = Luau.Type.src strict - -- Language membership is decidable dec-language : ∀ T t → Either (¬Language T t) (Language T t) dec-language nil (scalar number) = Left (scalar-scalar number nil (λ ())) diff --git a/prototyping/Properties/TypeCheck.agda b/prototyping/Properties/TypeCheck.agda index a5916a13..0726a4be 100644 --- a/prototyping/Properties/TypeCheck.agda +++ b/prototyping/Properties/TypeCheck.agda @@ -1,16 +1,14 @@ {-# OPTIONS --rewriting #-} -open import Luau.Type using (Mode) - -module Properties.TypeCheck (m : Mode) where +module Properties.TypeCheck where open import Agda.Builtin.Equality using (_≡_; refl) open import Agda.Builtin.Bool using (Bool; true; false) open import FFI.Data.Maybe using (Maybe; just; nothing) open import FFI.Data.Either using (Either) -open import Luau.TypeCheck(m) using (_⊢ᴱ_∈_; _⊢ᴮ_∈_; ⊢ᴼ_; ⊢ᴴ_; _⊢ᴴᴱ_▷_∈_; _⊢ᴴᴮ_▷_∈_; nil; var; addr; number; bool; string; app; function; block; binexp; done; return; local; nothing; orUnknown; tgtBinOp) +open import Luau.TypeCheck using (_⊢ᴱ_∈_; _⊢ᴮ_∈_; ⊢ᴼ_; ⊢ᴴ_; _⊢ᴴᴱ_▷_∈_; _⊢ᴴᴮ_▷_∈_; nil; var; addr; number; bool; string; app; function; block; binexp; done; return; local; nothing; orUnknown; tgtBinOp) open import Luau.Syntax using (Block; Expr; Value; BinaryOperator; yes; nil; addr; number; bool; string; val; var; binexp; _$_; function_is_end; block_is_end; _∙_; return; done; local_←_; _⟨_⟩; _⟨_⟩∈_; var_∈_; name; fun; arg; +; -; *; /; <; >; ==; ~=; <=; >=) -open import Luau.Type using (Type; nil; unknown; never; number; boolean; string; _⇒_; tgt) +open import Luau.Type using (Type; nil; unknown; never; number; boolean; string; _⇒_; src; tgt) open import Luau.RuntimeType using (RuntimeType; nil; number; function; string; valueType) open import Luau.VarCtxt using (VarCtxt; ∅; _↦_; _⊕_↦_; _⋒_; _⊝_) renaming (_[_] to _[_]ⱽ) open import Luau.Addr using (Addr) @@ -22,9 +20,6 @@ open import Properties.Equality using (_≢_; sym; trans; cong) open import Properties.Product using (_×_; _,_) open import Properties.Remember using (Remember; remember; _,_) -src : Type → Type -src = Luau.Type.src m - typeOfᴼ : Object yes → Type typeOfᴼ (function f ⟨ var x ∈ S ⟩∈ T is B end) = (S ⇒ T) From f2677f697574da4f634ba9c3c322f3a4a6541262 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Thu, 21 Apr 2022 14:04:22 -0700 Subject: [PATCH 226/399] Sync to upstream/release/524 --- Analysis/include/Luau/Clone.h | 2 +- Analysis/include/Luau/Frontend.h | 21 +- Analysis/include/Luau/TypeVar.h | 6 +- Analysis/include/Luau/Unifier.h | 1 - Analysis/include/Luau/VisitTypeVar.h | 2 +- Analysis/src/Autocomplete.cpp | 4 +- Analysis/src/Clone.cpp | 42 ++- Analysis/src/Error.cpp | 23 +- Analysis/src/Frontend.cpp | 50 ++-- Analysis/src/Module.cpp | 18 +- Analysis/src/Normalize.cpp | 44 +-- Analysis/src/Substitution.cpp | 15 +- Analysis/src/ToDot.cpp | 2 +- Analysis/src/Transpiler.cpp | 53 ++-- Analysis/src/TypeAttach.cpp | 14 + Analysis/src/TypeInfer.cpp | 37 ++- Analysis/src/TypeVar.cpp | 6 + Analysis/src/Unifier.cpp | 110 ++------ Ast/include/Luau/StringUtils.h | 1 + Ast/src/Parser.cpp | 8 + Ast/src/StringUtils.cpp | 2 +- CLI/Repl.cpp | 72 ++++- CMakeLists.txt | 6 + Compiler/include/Luau/BytecodeBuilder.h | 7 + Compiler/src/BytecodeBuilder.cpp | 70 ++++- Compiler/src/Compiler.cpp | 127 ++++++++- Compiler/src/ConstantFolding.cpp | 13 +- Compiler/src/ConstantFolding.h | 3 +- VM/src/lapi.cpp | 24 +- VM/src/lgc.cpp | 26 +- VM/src/lgc.h | 2 +- VM/src/lstate.h | 7 +- bench/bench.py | 13 +- fuzz/proto.cpp | 13 +- tests/Autocomplete.test.cpp | 36 +++ tests/Compiler.test.cpp | 359 +++++++++++++++++++++++- tests/CostModel.test.cpp | 125 +++++++++ tests/JsonEncoder.test.cpp | 332 +++++++++++++++++++++- tests/Linter.test.cpp | 2 +- tests/Module.test.cpp | 24 +- tests/NonstrictMode.test.cpp | 46 ++- tests/Normalize.test.cpp | 44 ++- tests/ToDot.test.cpp | 4 +- tests/Transpiler.test.cpp | 17 ++ tests/TypeInfer.classes.test.cpp | 34 ++- tests/TypeInfer.definitions.test.cpp | 2 - tests/TypeInfer.functions.test.cpp | 30 +- tests/TypeInfer.modules.test.cpp | 4 - tests/TypeInfer.provisional.test.cpp | 18 +- tests/TypeInfer.refinements.test.cpp | 8 +- tests/TypeInfer.tables.test.cpp | 2 - tests/TypeInfer.test.cpp | 18 +- tests/TypeVar.test.cpp | 6 +- 53 files changed, 1600 insertions(+), 355 deletions(-) diff --git a/Analysis/include/Luau/Clone.h b/Analysis/include/Luau/Clone.h index 78aa92c7..9b6ffa62 100644 --- a/Analysis/include/Luau/Clone.h +++ b/Analysis/include/Luau/Clone.h @@ -18,7 +18,7 @@ struct CloneState SeenTypePacks seenTypePacks; int recursionCount = 0; - bool encounteredFreeType = false; + bool encounteredFreeType = false; // TODO: Remove with LuauLosslessClone. }; TypePackId clone(TypePackId tp, TypeArena& dest, CloneState& cloneState); diff --git a/Analysis/include/Luau/Frontend.h b/Analysis/include/Luau/Frontend.h index e24e433c..59125470 100644 --- a/Analysis/include/Luau/Frontend.h +++ b/Analysis/include/Luau/Frontend.h @@ -13,6 +13,7 @@ #include LUAU_FASTFLAG(LuauSeparateTypechecks) +LUAU_FASTFLAG(LuauDirtySourceModule) namespace Luau { @@ -57,19 +58,27 @@ std::optional pathExprToModuleName(const ModuleName& currentModuleNa struct SourceNode { - bool isDirty(bool forAutocomplete) const + bool hasDirtySourceModule() const + { + LUAU_ASSERT(FFlag::LuauDirtySourceModule); + + return dirtySourceModule; + } + + bool hasDirtyModule(bool forAutocomplete) const { if (FFlag::LuauSeparateTypechecks) - return forAutocomplete ? dirtyAutocomplete : dirty; + return forAutocomplete ? dirtyModuleForAutocomplete : dirtyModule; else - return dirty; + return dirtyModule; } ModuleName name; std::unordered_set requires; std::vector> requireLocations; - bool dirty = true; - bool dirtyAutocomplete = true; + bool dirtySourceModule = true; + bool dirtyModule = true; + bool dirtyModuleForAutocomplete = true; double autocompleteLimitsMult = 1.0; }; @@ -163,7 +172,7 @@ struct Frontend void applyBuiltinDefinitionToEnvironment(const std::string& environmentName, const std::string& definitionName); private: - std::pair getSourceNode(CheckResult& checkResult, const ModuleName& name, bool forAutocomplete); + std::pair getSourceNode(CheckResult& checkResult, const ModuleName& name, bool forAutocomplete_DEPRECATED); SourceModule parse(const ModuleName& name, std::string_view src, const ParseOptions& parseOptions); bool parseGraph(std::vector& buildQueue, CheckResult& checkResult, const ModuleName& root, bool forAutocomplete); diff --git a/Analysis/include/Luau/TypeVar.h b/Analysis/include/Luau/TypeVar.h index ae7d1377..84576758 100644 --- a/Analysis/include/Luau/TypeVar.h +++ b/Analysis/include/Luau/TypeVar.h @@ -373,15 +373,17 @@ struct ClassTypeVar std::optional metatable; // metaclass? Tags tags; std::shared_ptr userData; + ModuleName definitionModuleName; - ClassTypeVar( - Name name, Props props, std::optional parent, std::optional metatable, Tags tags, std::shared_ptr userData) + ClassTypeVar(Name name, Props props, std::optional parent, std::optional metatable, Tags tags, + std::shared_ptr userData, ModuleName definitionModuleName) : name(name) , props(props) , parent(parent) , metatable(metatable) , tags(tags) , userData(userData) + , definitionModuleName(definitionModuleName) { } }; diff --git a/Analysis/include/Luau/Unifier.h b/Analysis/include/Luau/Unifier.h index 340feb7f..418d4ca4 100644 --- a/Analysis/include/Luau/Unifier.h +++ b/Analysis/include/Luau/Unifier.h @@ -92,7 +92,6 @@ private: bool canCacheResult(TypeId subTy, TypeId superTy); void cacheResult(TypeId subTy, TypeId superTy, size_t prevErrorCount); - void cacheResult_DEPRECATED(TypeId subTy, TypeId superTy); public: void tryUnify(TypePackId subTy, TypePackId superTy, bool isFunctionCall = false); diff --git a/Analysis/include/Luau/VisitTypeVar.h b/Analysis/include/Luau/VisitTypeVar.h index d11cbd0d..045190ea 100644 --- a/Analysis/include/Luau/VisitTypeVar.h +++ b/Analysis/include/Luau/VisitTypeVar.h @@ -52,7 +52,7 @@ inline void unsee(std::unordered_set& seen, const void* tv) inline void unsee(DenseHashSet& seen, const void* tv) { - // When DenseHashSet is used for 'visitOnce', where don't forget visited elements + // When DenseHashSet is used for 'visitTypeVarOnce', where don't forget visited elements } template diff --git a/Analysis/src/Autocomplete.cpp b/Analysis/src/Autocomplete.cpp index e0e79cb4..dec12d01 100644 --- a/Analysis/src/Autocomplete.cpp +++ b/Analysis/src/Autocomplete.cpp @@ -15,6 +15,7 @@ LUAU_FASTFLAGVARIABLE(LuauIfElseExprFixCompletionIssue, false); LUAU_FASTFLAGVARIABLE(LuauAutocompleteSingletonTypes, false); +LUAU_FASTFLAGVARIABLE(LuauFixAutocompleteClassSecurityLevel, false); LUAU_FASTFLAG(LuauSelfCallAutocompleteFix) static const std::unordered_set kStatementStartingKeywords = { @@ -462,7 +463,8 @@ static void autocompleteProps(const Module& module, TypeArena* typeArena, TypeId containingClass = containingClass.value_or(cls); fillProps(cls->props); if (cls->parent) - autocompleteProps(module, typeArena, rootTy, *cls->parent, indexType, nodes, result, seen, cls); + autocompleteProps(module, typeArena, rootTy, *cls->parent, indexType, nodes, result, seen, + FFlag::LuauFixAutocompleteClassSecurityLevel ? containingClass : cls); } else if (auto tbl = get(ty)) fillProps(tbl->props); diff --git a/Analysis/src/Clone.cpp b/Analysis/src/Clone.cpp index 8e7f7c07..d5bd9dab 100644 --- a/Analysis/src/Clone.cpp +++ b/Analysis/src/Clone.cpp @@ -10,6 +10,7 @@ LUAU_FASTFLAG(DebugLuauCopyBeforeNormalizing) LUAU_FASTINTVARIABLE(LuauTypeCloneRecursionLimit, 300) LUAU_FASTFLAG(LuauTypecheckOptPass) +LUAU_FASTFLAGVARIABLE(LuauLosslessClone, false) namespace Luau { @@ -87,11 +88,18 @@ struct TypePackCloner void operator()(const Unifiable::Free& t) { - cloneState.encounteredFreeType = true; + if (FFlag::LuauLosslessClone) + { + defaultClone(t); + } + else + { + cloneState.encounteredFreeType = true; - TypePackId err = getSingletonTypes().errorRecoveryTypePack(getSingletonTypes().anyTypePack); - TypePackId cloned = dest.addTypePack(*err); - seenTypePacks[typePackId] = cloned; + TypePackId err = getSingletonTypes().errorRecoveryTypePack(getSingletonTypes().anyTypePack); + TypePackId cloned = dest.addTypePack(*err); + seenTypePacks[typePackId] = cloned; + } } void operator()(const Unifiable::Generic& t) @@ -143,10 +151,18 @@ void TypeCloner::defaultClone(const T& t) void TypeCloner::operator()(const Unifiable::Free& t) { - cloneState.encounteredFreeType = true; - TypeId err = getSingletonTypes().errorRecoveryType(getSingletonTypes().anyType); - TypeId cloned = dest.addType(*err); - seenTypes[typeId] = cloned; + if (FFlag::LuauLosslessClone) + { + defaultClone(t); + } + else + { + cloneState.encounteredFreeType = true; + + TypeId err = getSingletonTypes().errorRecoveryType(getSingletonTypes().anyType); + TypeId cloned = dest.addType(*err); + seenTypes[typeId] = cloned; + } } void TypeCloner::operator()(const Unifiable::Generic& t) @@ -174,7 +190,8 @@ void TypeCloner::operator()(const PrimitiveTypeVar& t) void TypeCloner::operator()(const ConstrainedTypeVar& t) { - cloneState.encounteredFreeType = true; + if (!FFlag::LuauLosslessClone) + cloneState.encounteredFreeType = true; TypeId res = dest.addType(ConstrainedTypeVar{t.level}); ConstrainedTypeVar* ctv = getMutable(res); @@ -252,7 +269,7 @@ void TypeCloner::operator()(const TableTypeVar& t) for (TypePackId& arg : ttv->instantiatedTypePackParams) arg = clone(arg, dest, cloneState); - if (ttv->state == TableState::Free) + if (!FFlag::LuauLosslessClone && ttv->state == TableState::Free) { cloneState.encounteredFreeType = true; @@ -276,7 +293,7 @@ void TypeCloner::operator()(const MetatableTypeVar& t) void TypeCloner::operator()(const ClassTypeVar& t) { - TypeId result = dest.addType(ClassTypeVar{t.name, {}, std::nullopt, std::nullopt, t.tags, t.userData}); + TypeId result = dest.addType(ClassTypeVar{t.name, {}, std::nullopt, std::nullopt, t.tags, t.userData, t.definitionModuleName}); ClassTypeVar* ctv = getMutable(result); seenTypes[typeId] = result; @@ -361,7 +378,10 @@ TypeId clone(TypeId typeId, TypeArena& dest, CloneState& cloneState) // Persistent types are not being cloned and we get the original type back which might be read-only if (!res->persistent) + { asMutable(res)->documentationSymbol = typeId->documentationSymbol; + asMutable(res)->normal = typeId->normal; + } } return res; diff --git a/Analysis/src/Error.cpp b/Analysis/src/Error.cpp index cbec0b15..24ed4ac1 100644 --- a/Analysis/src/Error.cpp +++ b/Analysis/src/Error.cpp @@ -8,8 +8,6 @@ #include -LUAU_FASTFLAGVARIABLE(LuauTypeMismatchModuleName, false); - static std::string wrongNumberOfArgsString(size_t expectedCount, size_t actualCount, const char* argPrefix = nullptr, bool isVariadic = false) { std::string s = "expects "; @@ -59,27 +57,20 @@ struct ErrorConverter std::string result; - if (FFlag::LuauTypeMismatchModuleName) + if (givenTypeName == wantedTypeName) { - if (givenTypeName == wantedTypeName) + if (auto givenDefinitionModule = getDefinitionModuleName(tm.givenType)) { - if (auto givenDefinitionModule = getDefinitionModuleName(tm.givenType)) + if (auto wantedDefinitionModule = getDefinitionModuleName(tm.wantedType)) { - if (auto wantedDefinitionModule = getDefinitionModuleName(tm.wantedType)) - { - result = "Type '" + givenTypeName + "' from '" + *givenDefinitionModule + "' could not be converted into '" + wantedTypeName + - "' from '" + *wantedDefinitionModule + "'"; - } + result = "Type '" + givenTypeName + "' from '" + *givenDefinitionModule + "' could not be converted into '" + wantedTypeName + + "' from '" + *wantedDefinitionModule + "'"; } } + } - if (result.empty()) - result = "Type '" + givenTypeName + "' could not be converted into '" + wantedTypeName + "'"; - } - else - { + if (result.empty()) result = "Type '" + givenTypeName + "' could not be converted into '" + wantedTypeName + "'"; - } if (tm.error) { diff --git a/Analysis/src/Frontend.cpp b/Analysis/src/Frontend.cpp index 8b0b2210..34ccdac4 100644 --- a/Analysis/src/Frontend.cpp +++ b/Analysis/src/Frontend.cpp @@ -23,6 +23,7 @@ LUAU_FASTFLAG(LuauInferInNoCheckMode) LUAU_FASTFLAGVARIABLE(LuauKnowsTheDataModel3, false) LUAU_FASTFLAGVARIABLE(LuauSeparateTypechecks, false) LUAU_FASTFLAGVARIABLE(LuauAutocompleteDynamicLimits, false) +LUAU_FASTFLAGVARIABLE(LuauDirtySourceModule, false) LUAU_FASTINTVARIABLE(LuauAutocompleteCheckTimeoutMs, 0) namespace Luau @@ -358,7 +359,7 @@ CheckResult Frontend::check(const ModuleName& name, std::optionalsecond.isDirty(frontendOptions.forAutocomplete)) + if (it != sourceNodes.end() && !it->second.hasDirtyModule(frontendOptions.forAutocomplete)) { // No recheck required. if (FFlag::LuauSeparateTypechecks) @@ -402,7 +403,7 @@ CheckResult Frontend::check(const ModuleName& name, std::optionalerrors.begin(), module->errors.end()); moduleResolver.modules[moduleName] = std::move(module); - sourceNode.dirty = false; + sourceNode.dirtyModule = false; } return checkResult; @@ -618,7 +619,7 @@ bool Frontend::parseGraph(std::vector& buildQueue, CheckResult& chec // this relies on the fact that markDirty marks reverse-dependencies dirty as well // thus if a node is not dirty, all its transitive deps aren't dirty, which means that they won't ever need // to be built, *and* can't form a cycle with any nodes we did process. - if (!it->second.isDirty(forAutocomplete)) + if (!it->second.hasDirtyModule(forAutocomplete)) continue; // note: this check is technically redundant *except* that getSourceNode has somewhat broken memoization @@ -768,7 +769,7 @@ LintResult Frontend::lint(const SourceModule& module, std::optionalsecond.isDirty(forAutocomplete); + return it == sourceNodes.end() || it->second.hasDirtyModule(forAutocomplete); } /* @@ -810,20 +811,31 @@ void Frontend::markDirty(const ModuleName& name, std::vector* marked if (markedDirty) markedDirty->push_back(next); - if (FFlag::LuauSeparateTypechecks) + if (FFlag::LuauDirtySourceModule) { - if (sourceNode.dirty && sourceNode.dirtyAutocomplete) + LUAU_ASSERT(FFlag::LuauSeparateTypechecks); + + if (sourceNode.dirtySourceModule && sourceNode.dirtyModule && sourceNode.dirtyModuleForAutocomplete) continue; - sourceNode.dirty = true; - sourceNode.dirtyAutocomplete = true; + sourceNode.dirtySourceModule = true; + sourceNode.dirtyModule = true; + sourceNode.dirtyModuleForAutocomplete = true; + } + else if (FFlag::LuauSeparateTypechecks) + { + if (sourceNode.dirtyModule && sourceNode.dirtyModuleForAutocomplete) + continue; + + sourceNode.dirtyModule = true; + sourceNode.dirtyModuleForAutocomplete = true; } else { - if (sourceNode.dirty) + if (sourceNode.dirtyModule) continue; - sourceNode.dirty = true; + sourceNode.dirtyModule = true; } if (0 == reverseDeps.count(name)) @@ -851,13 +863,14 @@ const SourceModule* Frontend::getSourceModule(const ModuleName& moduleName) cons } // Read AST into sourceModules if necessary. Trace require()s. Report parse errors. -std::pair Frontend::getSourceNode(CheckResult& checkResult, const ModuleName& name, bool forAutocomplete) +std::pair Frontend::getSourceNode(CheckResult& checkResult, const ModuleName& name, bool forAutocomplete_DEPRECATED) { LUAU_TIMETRACE_SCOPE("Frontend::getSourceNode", "Frontend"); LUAU_TIMETRACE_ARGUMENT("name", name.c_str()); auto it = sourceNodes.find(name); - if (it != sourceNodes.end() && !it->second.isDirty(forAutocomplete)) + if (it != sourceNodes.end() && + (FFlag::LuauDirtySourceModule ? !it->second.hasDirtySourceModule() : !it->second.hasDirtyModule(forAutocomplete_DEPRECATED))) { auto moduleIt = sourceModules.find(name); if (moduleIt != sourceModules.end()) @@ -901,17 +914,20 @@ std::pair Frontend::getSourceNode(CheckResult& check sourceNode.requires.clear(); sourceNode.requireLocations.clear(); + if (FFlag::LuauDirtySourceModule) + sourceNode.dirtySourceModule = false; + if (FFlag::LuauSeparateTypechecks) { if (it == sourceNodes.end()) { - sourceNode.dirty = true; - sourceNode.dirtyAutocomplete = true; + sourceNode.dirtyModule = true; + sourceNode.dirtyModuleForAutocomplete = true; } } else { - sourceNode.dirty = true; + sourceNode.dirtyModule = true; } for (const auto& [moduleName, location] : requireTrace.requires) diff --git a/Analysis/src/Module.cpp b/Analysis/src/Module.cpp index e2e3b436..bafd4371 100644 --- a/Analysis/src/Module.cpp +++ b/Analysis/src/Module.cpp @@ -14,8 +14,8 @@ #include LUAU_FASTFLAGVARIABLE(DebugLuauFreezeArena, false) -LUAU_FASTFLAGVARIABLE(LuauCloneDeclaredGlobals, false) LUAU_FASTFLAG(LuauLowerBoundsCalculation) +LUAU_FASTFLAG(LuauLosslessClone) namespace Luau { @@ -182,20 +182,20 @@ bool Module::clonePublicInterface(InternalErrorReporter& ice) } } - if (FFlag::LuauCloneDeclaredGlobals) + for (auto& [name, ty] : declaredGlobals) { - for (auto& [name, ty] : declaredGlobals) - { - ty = clone(ty, interfaceTypes, cloneState); - if (FFlag::LuauLowerBoundsCalculation) - normalize(ty, interfaceTypes, ice); - } + ty = clone(ty, interfaceTypes, cloneState); + if (FFlag::LuauLowerBoundsCalculation) + normalize(ty, interfaceTypes, ice); } freeze(internalTypes); freeze(interfaceTypes); - return cloneState.encounteredFreeType; + if (FFlag::LuauLosslessClone) + return false; // TODO: make function return void. + else + return cloneState.encounteredFreeType; } } // namespace Luau diff --git a/Analysis/src/Normalize.cpp b/Analysis/src/Normalize.cpp index 40341ac1..043526ed 100644 --- a/Analysis/src/Normalize.cpp +++ b/Analysis/src/Normalize.cpp @@ -5,7 +5,6 @@ #include #include "Luau/Clone.h" -#include "Luau/DenseHash.h" #include "Luau/Substitution.h" #include "Luau/Unifier.h" #include "Luau/VisitTypeVar.h" @@ -254,7 +253,7 @@ bool isSubtype(TypeId subTy, TypeId superTy, InternalErrorReporter& ice) } template -static bool areNormal_(const T& t, const DenseHashSet& seen, InternalErrorReporter& ice) +static bool areNormal_(const T& t, const std::unordered_set& seen, InternalErrorReporter& ice) { int count = 0; auto isNormal = [&](TypeId ty) { @@ -262,18 +261,19 @@ static bool areNormal_(const T& t, const DenseHashSet& seen, InternalErro if (count >= FInt::LuauNormalizeIterationLimit) ice.ice("Luau::areNormal hit iteration limit"); - return ty->normal || seen.find(asMutable(ty)); + // The follow is here because a bound type may not be normal, but the bound type is normal. + return ty->normal || follow(ty)->normal || seen.find(asMutable(ty)) != seen.end(); }; return std::all_of(begin(t), end(t), isNormal); } -static bool areNormal(const std::vector& types, const DenseHashSet& seen, InternalErrorReporter& ice) +static bool areNormal(const std::vector& types, const std::unordered_set& seen, InternalErrorReporter& ice) { return areNormal_(types, seen, ice); } -static bool areNormal(TypePackId tp, const DenseHashSet& seen, InternalErrorReporter& ice) +static bool areNormal(TypePackId tp, const std::unordered_set& seen, InternalErrorReporter& ice) { tp = follow(tp); if (get(tp)) @@ -288,7 +288,7 @@ static bool areNormal(TypePackId tp, const DenseHashSet& seen, InternalEr return true; if (auto vtp = get(*tail)) - return vtp->ty->normal || seen.find(asMutable(vtp->ty)); + return vtp->ty->normal || follow(vtp->ty)->normal || seen.find(asMutable(vtp->ty)) != seen.end(); return true; } @@ -335,9 +335,14 @@ struct Normalize return false; } - bool operator()(TypeId ty, const BoundTypeVar& btv) + bool operator()(TypeId ty, const BoundTypeVar& btv, std::unordered_set& seen) { - // It should never be the case that this TypeVar is normal, but is bound to a non-normal type. + // A type could be considered normal when it is in the stack, but we will eventually find out it is not normal as normalization progresses. + // So we need to avoid eagerly saying that this bound type is normal if the thing it is bound to is in the stack. + if (seen.find(asMutable(btv.boundTo)) != seen.end()) + return false; + + // It should never be the case that this TypeVar is normal, but is bound to a non-normal type, except in nontrivial cases. LUAU_ASSERT(!ty->normal || ty->normal == btv.boundTo->normal); asMutable(ty)->normal = btv.boundTo->normal; @@ -365,7 +370,7 @@ struct Normalize return false; } - bool operator()(TypeId ty, const ConstrainedTypeVar& ctvRef, DenseHashSet& seen) + bool operator()(TypeId ty, const ConstrainedTypeVar& ctvRef, std::unordered_set& seen) { CHECK_ITERATION_LIMIT(false); @@ -391,8 +396,7 @@ struct Normalize return false; } - bool operator()(TypeId ty, const FunctionTypeVar& ftv) = delete; - bool operator()(TypeId ty, const FunctionTypeVar& ftv, DenseHashSet& seen) + bool operator()(TypeId ty, const FunctionTypeVar& ftv, std::unordered_set& seen) { CHECK_ITERATION_LIMIT(false); @@ -407,7 +411,7 @@ struct Normalize return false; } - bool operator()(TypeId ty, const TableTypeVar& ttv, DenseHashSet& seen) + bool operator()(TypeId ty, const TableTypeVar& ttv, std::unordered_set& seen) { CHECK_ITERATION_LIMIT(false); @@ -419,7 +423,7 @@ struct Normalize auto checkNormal = [&](TypeId t) { // if t is on the stack, it is possible that this type is normal. // If t is not normal and it is not on the stack, this type is definitely not normal. - if (!t->normal && !seen.find(asMutable(t))) + if (!t->normal && seen.find(asMutable(t)) == seen.end()) normal = false; }; @@ -449,7 +453,7 @@ struct Normalize return false; } - bool operator()(TypeId ty, const MetatableTypeVar& mtv, DenseHashSet& seen) + bool operator()(TypeId ty, const MetatableTypeVar& mtv, std::unordered_set& seen) { CHECK_ITERATION_LIMIT(false); @@ -477,7 +481,7 @@ struct Normalize return false; } - bool operator()(TypeId ty, const UnionTypeVar& utvRef, DenseHashSet& seen) + bool operator()(TypeId ty, const UnionTypeVar& utvRef, std::unordered_set& seen) { CHECK_ITERATION_LIMIT(false); @@ -507,7 +511,7 @@ struct Normalize return false; } - bool operator()(TypeId ty, const IntersectionTypeVar& itvRef, DenseHashSet& seen) + bool operator()(TypeId ty, const IntersectionTypeVar& itvRef, std::unordered_set& seen) { CHECK_ITERATION_LIMIT(false); @@ -775,8 +779,8 @@ std::pair normalize(TypeId ty, TypeArena& arena, InternalErrorRepo (void)clone(ty, arena, state); Normalize n{arena, ice, std::move(state.seenTypes), std::move(state.seenTypePacks)}; - DenseHashSet seen{nullptr}; - visitTypeVarOnce(ty, n, seen); + std::unordered_set seen; + visitTypeVar(ty, n, seen); return {ty, !n.limitExceeded}; } @@ -800,8 +804,8 @@ std::pair normalize(TypePackId tp, TypeArena& arena, InternalE (void)clone(tp, arena, state); Normalize n{arena, ice, std::move(state.seenTypes), std::move(state.seenTypePacks)}; - DenseHashSet seen{nullptr}; - visitTypeVarOnce(tp, n, seen); + std::unordered_set seen; + visitTypeVar(tp, n, seen); return {tp, !n.limitExceeded}; } diff --git a/Analysis/src/Substitution.cpp b/Analysis/src/Substitution.cpp index 8648b21e..1b51fa3d 100644 --- a/Analysis/src/Substitution.cpp +++ b/Analysis/src/Substitution.cpp @@ -11,6 +11,7 @@ LUAU_FASTFLAG(LuauLowerBoundsCalculation) LUAU_FASTINTVARIABLE(LuauTarjanChildLimit, 1000) LUAU_FASTFLAG(LuauTypecheckOptPass) LUAU_FASTFLAGVARIABLE(LuauSubstituteFollowNewTypes, false) +LUAU_FASTFLAGVARIABLE(LuauSubstituteFollowPossibleMutations, false) namespace Luau { @@ -106,7 +107,7 @@ void Tarjan::visitChildren(TypePackId tp, int index) std::pair Tarjan::indexify(TypeId ty) { - if (FFlag::LuauTypecheckOptPass) + if (FFlag::LuauTypecheckOptPass && !FFlag::LuauSubstituteFollowPossibleMutations) LUAU_ASSERT(ty == log->follow(ty)); else ty = log->follow(ty); @@ -127,7 +128,7 @@ std::pair Tarjan::indexify(TypeId ty) std::pair Tarjan::indexify(TypePackId tp) { - if (FFlag::LuauTypecheckOptPass) + if (FFlag::LuauTypecheckOptPass && !FFlag::LuauSubstituteFollowPossibleMutations) LUAU_ASSERT(tp == log->follow(tp)); else tp = log->follow(tp); @@ -148,7 +149,8 @@ std::pair Tarjan::indexify(TypePackId tp) void Tarjan::visitChild(TypeId ty) { - ty = log->follow(ty); + if (!FFlag::LuauSubstituteFollowPossibleMutations) + ty = log->follow(ty); edgesTy.push_back(ty); edgesTp.push_back(nullptr); @@ -156,7 +158,8 @@ void Tarjan::visitChild(TypeId ty) void Tarjan::visitChild(TypePackId tp) { - tp = log->follow(tp); + if (!FFlag::LuauSubstituteFollowPossibleMutations) + tp = log->follow(tp); edgesTy.push_back(nullptr); edgesTp.push_back(tp); @@ -471,7 +474,7 @@ TypePackId Substitution::clone(TypePackId tp) void Substitution::foundDirty(TypeId ty) { - if (FFlag::LuauTypecheckOptPass) + if (FFlag::LuauTypecheckOptPass && !FFlag::LuauSubstituteFollowPossibleMutations) LUAU_ASSERT(ty == log->follow(ty)); else ty = log->follow(ty); @@ -484,7 +487,7 @@ void Substitution::foundDirty(TypeId ty) void Substitution::foundDirty(TypePackId tp) { - if (FFlag::LuauTypecheckOptPass) + if (FFlag::LuauTypecheckOptPass && !FFlag::LuauSubstituteFollowPossibleMutations) LUAU_ASSERT(tp == log->follow(tp)); else tp = log->follow(tp); diff --git a/Analysis/src/ToDot.cpp b/Analysis/src/ToDot.cpp index cb54bfc1..9b396c80 100644 --- a/Analysis/src/ToDot.cpp +++ b/Analysis/src/ToDot.cpp @@ -327,7 +327,7 @@ void StateDot::visitChildren(TypePackId tp, int index) } else if (const VariadicTypePack* vtp = get(tp)) { - formatAppend(result, "VariadicTypePack %d", index); + formatAppend(result, "VariadicTypePack %s%d", vtp->hidden ? "hidden " : "", index); finishNodeLabel(tp); finishNode(); diff --git a/Analysis/src/Transpiler.cpp b/Analysis/src/Transpiler.cpp index 92ed241e..1577bd63 100644 --- a/Analysis/src/Transpiler.cpp +++ b/Analysis/src/Transpiler.cpp @@ -1025,31 +1025,42 @@ struct Printer } else if (const auto& a = typeAnnotation.as()) { - CommaSeparatorInserter comma(writer); + AstTypeReference* indexType = a->indexer ? a->indexer->indexType->as() : nullptr; - writer.symbol("{"); - - for (std::size_t i = 0; i < a->props.size; ++i) + if (a->props.size == 0 && indexType && indexType->name == "number") { - comma(); - advance(a->props.data[i].location.begin); - writer.identifier(a->props.data[i].name.value); - if (a->props.data[i].type) - { - writer.symbol(":"); - visualizeTypeAnnotation(*a->props.data[i].type); - } - } - if (a->indexer) - { - comma(); - writer.symbol("["); - visualizeTypeAnnotation(*a->indexer->indexType); - writer.symbol("]"); - writer.symbol(":"); + writer.symbol("{"); visualizeTypeAnnotation(*a->indexer->resultType); + writer.symbol("}"); + } + else + { + CommaSeparatorInserter comma(writer); + + writer.symbol("{"); + + for (std::size_t i = 0; i < a->props.size; ++i) + { + comma(); + advance(a->props.data[i].location.begin); + writer.identifier(a->props.data[i].name.value); + if (a->props.data[i].type) + { + writer.symbol(":"); + visualizeTypeAnnotation(*a->props.data[i].type); + } + } + if (a->indexer) + { + comma(); + writer.symbol("["); + visualizeTypeAnnotation(*a->indexer->indexType); + writer.symbol("]"); + writer.symbol(":"); + visualizeTypeAnnotation(*a->indexer->resultType); + } + writer.symbol("}"); } - writer.symbol("}"); } else if (auto a = typeAnnotation.as()) { diff --git a/Analysis/src/TypeAttach.cpp b/Analysis/src/TypeAttach.cpp index bc8d0d4e..0f4534b7 100644 --- a/Analysis/src/TypeAttach.cpp +++ b/Analysis/src/TypeAttach.cpp @@ -479,6 +479,20 @@ public: { return visitLocal(al->local); } + + virtual bool visit(AstStatFor* stat) override + { + visitLocal(stat->var); + return true; + } + + virtual bool visit(AstStatForIn* stat) override + { + for (size_t i = 0; i < stat->vars.size; ++i) + visitLocal(stat->vars.data[i]); + return true; + } + virtual bool visit(AstExprFunction* fn) override { // TODO: add generics if the inferred type of the function is generic CLI-39908 diff --git a/Analysis/src/TypeInfer.cpp b/Analysis/src/TypeInfer.cpp index af42a4e6..6411e2ab 100644 --- a/Analysis/src/TypeInfer.cpp +++ b/Analysis/src/TypeInfer.cpp @@ -1,6 +1,7 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #include "Luau/TypeInfer.h" +#include "Luau/Clone.h" #include "Luau/Common.h" #include "Luau/ModuleResolver.h" #include "Luau/Normalize.h" @@ -47,7 +48,6 @@ LUAU_FASTFLAGVARIABLE(LuauPropertiesGetExpectedType, false) LUAU_FASTFLAGVARIABLE(LuauStatFunctionSimplify4, false) LUAU_FASTFLAGVARIABLE(LuauTypecheckOptPass, false) LUAU_FASTFLAGVARIABLE(LuauUnsealedTableLiteral, false) -LUAU_FASTFLAG(LuauTypeMismatchModuleName) LUAU_FASTFLAGVARIABLE(LuauTwoPassAliasDefinitionFix, false) LUAU_FASTFLAGVARIABLE(LuauAssertStripsFalsyTypes, false) LUAU_FASTFLAGVARIABLE(LuauReturnAnyInsteadOfICE, false) // Eventually removed as false. @@ -61,7 +61,9 @@ LUAU_FASTFLAG(LuauAnyInIsOptionalIsOptional) LUAU_FASTFLAGVARIABLE(LuauDecoupleOperatorInferenceFromUnifiedTypeInference, false) LUAU_FASTFLAGVARIABLE(LuauArgCountMismatchSaysAtLeastWhenVariadic, false) LUAU_FASTFLAGVARIABLE(LuauTableUseCounterInstead, false) +LUAU_FASTFLAGVARIABLE(LuauReturnTypeInferenceInNonstrict, false) LUAU_FASTFLAGVARIABLE(LuauRecursionLimitException, false); +LUAU_FASTFLAG(LuauLosslessClone) namespace Luau { @@ -376,7 +378,7 @@ ModulePtr TypeChecker::checkWithoutRecursionCheck(const SourceModule& module, Mo prepareErrorsForDisplay(currentModule->errors); bool encounteredFreeType = currentModule->clonePublicInterface(*iceHandler); - if (encounteredFreeType) + if (!FFlag::LuauLosslessClone && encounteredFreeType) { reportError(TypeError{module.root->location, GenericError{"Free types leaked into this module's public interface. This is an internal Luau error; please report it."}}); @@ -785,7 +787,7 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatReturn& return_) TypePackId retPack = checkExprList(scope, return_.location, return_.list, false, {}, expectedTypes).type; - if (useConstrainedIntersections()) + if (FFlag::LuauReturnTypeInferenceInNonstrict ? FFlag::LuauLowerBoundsCalculation : useConstrainedIntersections()) { unifyLowerBound(retPack, scope->returnType, return_.location); return; @@ -1241,7 +1243,12 @@ void TypeChecker::check(const ScopePtr& scope, TypeId ty, const ScopePtr& funSco // If in nonstrict mode and allowing redefinition of global function, restore the previous definition type // in case this function has a differing signature. The signature discrepancy will be caught in checkBlock. if (previouslyDefined) + { + if (FFlag::LuauReturnTypeInferenceInNonstrict && FFlag::LuauLowerBoundsCalculation) + quantify(funScope, ty, exprName->location); + globalBindings[name] = oldBinding; + } else globalBindings[name] = {quantify(funScope, ty, exprName->location), exprName->location}; @@ -1555,7 +1562,7 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatDeclareClass& declar Name className(declaredClass.name.value); - TypeId classTy = addType(ClassTypeVar(className, {}, superTy, std::nullopt, {}, {})); + TypeId classTy = addType(ClassTypeVar(className, {}, superTy, std::nullopt, {}, {}, currentModuleName)); ClassTypeVar* ctv = getMutable(classTy); TypeId metaTy = addType(TableTypeVar{TableState::Sealed, scope->level}); @@ -3284,7 +3291,7 @@ std::pair TypeChecker::checkFunctionSignature( TypePackId retPack; if (expr.returnAnnotation) retPack = resolveTypePack(funScope, *expr.returnAnnotation); - else if (isNonstrictMode()) + else if (FFlag::LuauReturnTypeInferenceInNonstrict ? (!FFlag::LuauLowerBoundsCalculation && isNonstrictMode()) : isNonstrictMode()) retPack = anyTypePack; else if (expectedFunctionType) { @@ -5328,19 +5335,9 @@ TypeId TypeChecker::resolveType(const ScopePtr& scope, const AstType& annotation if (const auto& indexer = table->indexer) tableIndexer = TableIndexer(resolveType(scope, *indexer->indexType), resolveType(scope, *indexer->resultType)); - if (FFlag::LuauTypeMismatchModuleName) - { - TableTypeVar ttv{props, tableIndexer, scope->level, TableState::Sealed}; - ttv.definitionModuleName = currentModuleName; - return addType(std::move(ttv)); - } - else - { - return addType(TableTypeVar{ - props, tableIndexer, scope->level, - TableState::Sealed // FIXME: probably want a way to annotate other kinds of tables maybe - }); - } + TableTypeVar ttv{props, tableIndexer, scope->level, TableState::Sealed}; + ttv.definitionModuleName = currentModuleName; + return addType(std::move(ttv)); } else if (const auto& func = annotation.as()) { @@ -5602,9 +5599,7 @@ TypeId TypeChecker::instantiateTypeFun(const ScopePtr& scope, const TypeFun& tf, { ttv->instantiatedTypeParams = typeParams; ttv->instantiatedTypePackParams = typePackParams; - - if (FFlag::LuauTypeMismatchModuleName) - ttv->definitionModuleName = currentModuleName; + ttv->definitionModuleName = currentModuleName; } return instantiated; diff --git a/Analysis/src/TypeVar.cpp b/Analysis/src/TypeVar.cpp index 0fbfdbf0..4d42573c 100644 --- a/Analysis/src/TypeVar.cpp +++ b/Analysis/src/TypeVar.cpp @@ -27,6 +27,7 @@ LUAU_FASTFLAG(LuauErrorRecoveryType) LUAU_FASTFLAG(LuauSubtypingAddOptPropsToUnsealedTables) LUAU_FASTFLAG(LuauDiscriminableUnions2) LUAU_FASTFLAGVARIABLE(LuauAnyInIsOptionalIsOptional, false) +LUAU_FASTFLAGVARIABLE(LuauClassDefinitionModuleInError, false) namespace Luau { @@ -304,6 +305,11 @@ std::optional getDefinitionModuleName(TypeId type) if (ftv->definition) return ftv->definition->definitionModuleName; } + else if (auto ctv = get(type); ctv && FFlag::LuauClassDefinitionModuleInError) + { + if (!ctv->definitionModuleName.empty()) + return ctv->definitionModuleName; + } return std::nullopt; } diff --git a/Analysis/src/Unifier.cpp b/Analysis/src/Unifier.cpp index f9ea58cc..9862d7b3 100644 --- a/Analysis/src/Unifier.cpp +++ b/Analysis/src/Unifier.cpp @@ -17,7 +17,6 @@ LUAU_FASTINT(LuauTypeInferTypePackLoopLimit); LUAU_FASTINT(LuauTypeInferIterationLimit); LUAU_FASTFLAG(LuauAutocompleteDynamicLimits) LUAU_FASTINTVARIABLE(LuauTypeInferLowerBoundsIterationLimit, 2000); -LUAU_FASTFLAGVARIABLE(LuauExtendedIndexerError, false); LUAU_FASTFLAGVARIABLE(LuauTableSubtypingVariance2, false); LUAU_FASTFLAG(LuauLowerBoundsCalculation); LUAU_FASTFLAG(LuauErrorRecoveryType); @@ -28,7 +27,6 @@ LUAU_FASTFLAGVARIABLE(LuauTxnLogSeesTypePacks2, false) LUAU_FASTFLAGVARIABLE(LuauTxnLogCheckForInvalidation, false) LUAU_FASTFLAGVARIABLE(LuauTxnLogRefreshFunctionPointers, false) LUAU_FASTFLAGVARIABLE(LuauTxnLogDontRetryForIndexers, false) -LUAU_FASTFLAGVARIABLE(LuauUnifierCacheErrors, false) LUAU_FASTFLAG(LuauAnyInIsOptionalIsOptional) LUAU_FASTFLAG(LuauTypecheckOptPass) @@ -474,32 +472,21 @@ void Unifier::tryUnify_(TypeId subTy, TypeId superTy, bool isFunctionCall, bool if (get(subTy)) return tryUnifyWithAny(superTy, subTy); - bool cacheEnabled; auto& cache = sharedState.cachedUnify; // What if the types are immutable and we proved their relation before - if (FFlag::LuauUnifierCacheErrors) + bool cacheEnabled = !isFunctionCall && !isIntersection && variance == Invariant; + + if (cacheEnabled) { - cacheEnabled = !isFunctionCall && !isIntersection && variance == Invariant; - - if (cacheEnabled) - { - if (cache.contains({subTy, superTy})) - return; - - if (auto error = sharedState.cachedUnifyError.find({subTy, superTy})) - { - reportError(TypeError{location, *error}); - return; - } - } - } - else - { - cacheEnabled = !isFunctionCall && !isIntersection; - - if (cacheEnabled && cache.contains({superTy, subTy}) && (variance == Covariant || cache.contains({subTy, superTy}))) + if (cache.contains({subTy, superTy})) return; + + if (auto error = sharedState.cachedUnifyError.find({subTy, superTy})) + { + reportError(TypeError{location, *error}); + return; + } } // If we have seen this pair of types before, we are currently recursing into cyclic types. @@ -543,12 +530,6 @@ void Unifier::tryUnify_(TypeId subTy, TypeId superTy, bool isFunctionCall, bool else if (log.getMutable(superTy) && log.getMutable(subTy)) { tryUnifyTables(subTy, superTy, isIntersection); - - if (!FFlag::LuauUnifierCacheErrors) - { - if (cacheEnabled && errors.empty()) - cacheResult_DEPRECATED(subTy, superTy); - } } // tryUnifyWithMetatable assumes its first argument is a MetatableTypeVar. The check is otherwise symmetrical. @@ -568,7 +549,7 @@ void Unifier::tryUnify_(TypeId subTy, TypeId superTy, bool isFunctionCall, bool else reportError(TypeError{location, TypeMismatch{superTy, subTy}}); - if (FFlag::LuauUnifierCacheErrors && cacheEnabled) + if (cacheEnabled) cacheResult(subTy, superTy, errorCount); log.popSeen(superTy, subTy); @@ -705,21 +686,10 @@ void Unifier::tryUnifyTypeWithUnion(TypeId subTy, TypeId superTy, const UnionTyp { TypeId type = uv->options[i]; - if (FFlag::LuauUnifierCacheErrors) + if (cache.contains({subTy, type})) { - if (cache.contains({subTy, type})) - { - startIndex = i; - break; - } - } - else - { - if (cache.contains({type, subTy}) && (variance == Covariant || cache.contains({subTy, type}))) - { - startIndex = i; - break; - } + startIndex = i; + break; } } } @@ -807,21 +777,10 @@ void Unifier::tryUnifyIntersectionWithType(TypeId subTy, const IntersectionTypeV { TypeId type = uv->parts[i]; - if (FFlag::LuauUnifierCacheErrors) + if (cache.contains({type, superTy})) { - if (cache.contains({type, superTy})) - { - startIndex = i; - break; - } - } - else - { - if (cache.contains({superTy, type}) && (variance == Covariant || cache.contains({type, superTy}))) - { - startIndex = i; - break; - } + startIndex = i; + break; } } } @@ -896,19 +855,6 @@ void Unifier::cacheResult(TypeId subTy, TypeId superTy, size_t prevErrorCount) } } -void Unifier::cacheResult_DEPRECATED(TypeId subTy, TypeId superTy) -{ - LUAU_ASSERT(!FFlag::LuauUnifierCacheErrors); - - if (!canCacheResult(subTy, superTy)) - return; - - sharedState.cachedUnify.insert({superTy, subTy}); - - if (variance == Invariant) - sharedState.cachedUnify.insert({subTy, superTy}); -} - struct WeirdIter { TypePackId packId; @@ -1650,24 +1596,16 @@ void Unifier::tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) Unifier innerState = makeChildUnifier(); - if (FFlag::LuauExtendedIndexerError) - { - innerState.tryUnify_(subTable->indexer->indexType, superTable->indexer->indexType); + innerState.tryUnify_(subTable->indexer->indexType, superTable->indexer->indexType); - bool reported = !innerState.errors.empty(); + bool reported = !innerState.errors.empty(); - checkChildUnifierTypeMismatch(innerState.errors, "[indexer key]", superTy, subTy); + checkChildUnifierTypeMismatch(innerState.errors, "[indexer key]", superTy, subTy); - innerState.tryUnify_(subTable->indexer->indexResultType, superTable->indexer->indexResultType); + innerState.tryUnify_(subTable->indexer->indexResultType, superTable->indexer->indexResultType); - if (!reported) - checkChildUnifierTypeMismatch(innerState.errors, "[indexer value]", superTy, subTy); - } - else - { - innerState.tryUnifyIndexer(*subTable->indexer, *superTable->indexer); - checkChildUnifierTypeMismatch(innerState.errors, superTy, subTy); - } + if (!reported) + checkChildUnifierTypeMismatch(innerState.errors, "[indexer value]", superTy, subTy); if (innerState.errors.empty()) log.concat(std::move(innerState.log)); @@ -2225,7 +2163,7 @@ void Unifier::tryUnifyWithClass(TypeId subTy, TypeId superTy, bool reversed) void Unifier::tryUnifyIndexer(const TableIndexer& subIndexer, const TableIndexer& superIndexer) { - LUAU_ASSERT(!FFlag::LuauTableSubtypingVariance2 || !FFlag::LuauExtendedIndexerError); + LUAU_ASSERT(!FFlag::LuauTableSubtypingVariance2); tryUnify_(subIndexer.indexType, superIndexer.indexType); tryUnify_(subIndexer.indexResultType, superIndexer.indexResultType); diff --git a/Ast/include/Luau/StringUtils.h b/Ast/include/Luau/StringUtils.h index 6ecf0606..6ae9e977 100644 --- a/Ast/include/Luau/StringUtils.h +++ b/Ast/include/Luau/StringUtils.h @@ -19,6 +19,7 @@ std::string format(const char* fmt, ...) LUAU_PRINTF_ATTR(1, 2); std::string vformat(const char* fmt, va_list args); void formatAppend(std::string& str, const char* fmt, ...) LUAU_PRINTF_ATTR(2, 3); +void vformatAppend(std::string& ret, const char* fmt, va_list args); std::string join(const std::vector& segments, std::string_view delimiter); std::string join(const std::vector& segments, std::string_view delimiter); diff --git a/Ast/src/Parser.cpp b/Ast/src/Parser.cpp index badd3fd3..31ff3f77 100644 --- a/Ast/src/Parser.cpp +++ b/Ast/src/Parser.cpp @@ -167,6 +167,7 @@ Parser::Parser(const char* buffer, size_t bufferSize, AstNameTable& names, Alloc Function top; top.vararg = true; + functionStack.reserve(8); functionStack.push_back(top); nameSelf = names.addStatic("self"); @@ -186,6 +187,13 @@ Parser::Parser(const char* buffer, size_t bufferSize, AstNameTable& names, Alloc // all hot comments parsed after the first non-comment lexeme are special in that they don't affect type checking / linting mode hotcommentHeader = false; + + // preallocate some buffers that are very likely to grow anyway; this works around std::vector's inefficient growth policy for small arrays + localStack.reserve(16); + scratchStat.reserve(16); + scratchExpr.reserve(16); + scratchLocal.reserve(16); + scratchBinding.reserve(16); } bool Parser::blockFollow(const Lexeme& l) diff --git a/Ast/src/StringUtils.cpp b/Ast/src/StringUtils.cpp index 9c7fed31..0dc3f3f5 100644 --- a/Ast/src/StringUtils.cpp +++ b/Ast/src/StringUtils.cpp @@ -11,7 +11,7 @@ namespace Luau { -static void vformatAppend(std::string& ret, const char* fmt, va_list args) +void vformatAppend(std::string& ret, const char* fmt, va_list args) { va_list argscopy; va_copy(argscopy, args); diff --git a/CLI/Repl.cpp b/CLI/Repl.cpp index 5fd6d341..345cb7ac 100644 --- a/CLI/Repl.cpp +++ b/CLI/Repl.cpp @@ -579,7 +579,8 @@ static bool compileFile(const char* name, CompileFormat format) if (format == CompileFormat::Text) { - bcb.setDumpFlags(Luau::BytecodeBuilder::Dump_Code | Luau::BytecodeBuilder::Dump_Source | Luau::BytecodeBuilder::Dump_Locals); + bcb.setDumpFlags(Luau::BytecodeBuilder::Dump_Code | Luau::BytecodeBuilder::Dump_Source | Luau::BytecodeBuilder::Dump_Locals | + Luau::BytecodeBuilder::Dump_Remarks); bcb.setDumpSource(*source); } @@ -636,13 +637,60 @@ static int assertionHandler(const char* expr, const char* file, int line, const return 1; } +static void setLuauFlags(bool state) +{ + for (Luau::FValue* flag = Luau::FValue::list; flag; flag = flag->next) + { + if (strncmp(flag->name, "Luau", 4) == 0) + flag->value = state; + } +} + +static void setFlag(std::string_view name, bool state) +{ + for (Luau::FValue* flag = Luau::FValue::list; flag; flag = flag->next) + { + if (name == flag->name) + { + flag->value = state; + return; + } + } + + fprintf(stderr, "Warning: --fflag unrecognized flag '%.*s'.\n\n", int(name.length()), name.data()); +} + +static void applyFlagKeyValue(std::string_view element) +{ + if (size_t separator = element.find('='); separator != std::string_view::npos) + { + std::string_view key = element.substr(0, separator); + std::string_view value = element.substr(separator + 1); + + if (value == "true") + setFlag(key, true); + else if (value == "false") + setFlag(key, false); + else + fprintf(stderr, "Warning: --fflag unrecognized value '%.*s' for flag '%.*s'.\n\n", int(value.length()), value.data(), int(key.length()), + key.data()); + } + else + { + if (element == "true") + setLuauFlags(true); + else if (element == "false") + setLuauFlags(false); + else + setFlag(element, true); + } +} + int replMain(int argc, char** argv) { Luau::assertHandler() = assertionHandler; - for (Luau::FValue* flag = Luau::FValue::list; flag; flag = flag->next) - if (strncmp(flag->name, "Luau", 4) == 0) - flag->value = true; + setLuauFlags(true); CliMode mode = CliMode::Unknown; CompileFormat compileFormat{}; @@ -727,6 +775,22 @@ int replMain(int argc, char** argv) return 1; #endif } + else if (strncmp(argv[i], "--fflags=", 9) == 0) + { + std::string_view list = argv[i] + 9; + + while (!list.empty()) + { + size_t ending = list.find(","); + + applyFlagKeyValue(list.substr(0, ending)); + + if (ending != std::string_view::npos) + list.remove_prefix(ending + 1); + else + break; + } + } else if (argv[i][0] == '-') { fprintf(stderr, "Error: Unrecognized option '%s'.\n\n", argv[i]); diff --git a/CMakeLists.txt b/CMakeLists.txt index c6ccebc5..af03b33a 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -73,6 +73,12 @@ else() list(APPEND LUAU_OPTIONS -Wall) # All warnings endif() +if (CMAKE_CXX_COMPILER_ID STREQUAL "GNU") + # Some gcc versions treat var in `if (type var = val)` as unused + # Some gcc versions treat variables used in constexpr if blocks as unused + list(APPEND LUAU_OPTIONS -Wno-unused) +endif() + # Enabled in CI; we should be warning free on our main compiler versions but don't guarantee being warning free everywhere if(LUAU_WERROR) if(MSVC) diff --git a/Compiler/include/Luau/BytecodeBuilder.h b/Compiler/include/Luau/BytecodeBuilder.h index 287bf4ee..67b93028 100644 --- a/Compiler/include/Luau/BytecodeBuilder.h +++ b/Compiler/include/Luau/BytecodeBuilder.h @@ -3,6 +3,7 @@ #include "Luau/Bytecode.h" #include "Luau/DenseHash.h" +#include "Luau/StringUtils.h" #include @@ -80,6 +81,8 @@ public: void pushDebugUpval(StringRef name); uint32_t getDebugPC() const; + void addDebugRemark(const char* format, ...) LUAU_PRINTF_ATTR(2, 3); + void finalize(); enum DumpFlags @@ -88,6 +91,7 @@ public: Dump_Lines = 1 << 1, Dump_Source = 1 << 2, Dump_Locals = 1 << 3, + Dump_Remarks = 1 << 4, }; void setDumpFlags(uint32_t flags) @@ -228,6 +232,9 @@ private: DenseHashMap stringTable; + DenseHashMap debugRemarks; + std::string debugRemarkBuffer; + BytecodeEncoder* encoder = nullptr; std::string bytecode; diff --git a/Compiler/src/BytecodeBuilder.cpp b/Compiler/src/BytecodeBuilder.cpp index 6944de0f..6c6f1225 100644 --- a/Compiler/src/BytecodeBuilder.cpp +++ b/Compiler/src/BytecodeBuilder.cpp @@ -181,9 +181,17 @@ BytecodeBuilder::BytecodeBuilder(BytecodeEncoder* encoder) : constantMap({Constant::Type_Nil, ~0ull}) , tableShapeMap(TableShape()) , stringTable({nullptr, 0}) + , debugRemarks(~0u) , encoder(encoder) { LUAU_ASSERT(stringTable.find(StringRef{"", 0}) == nullptr); + + // preallocate some buffers that are very likely to grow anyway; this works around std::vector's inefficient growth policy for small arrays + insns.reserve(32); + lines.reserve(32); + constants.reserve(16); + protos.reserve(16); + functions.reserve(8); } uint32_t BytecodeBuilder::beginFunction(uint8_t numparams, bool isvararg) @@ -219,8 +227,8 @@ void BytecodeBuilder::endFunction(uint8_t maxstacksize, uint8_t numupvalues) validate(); #endif - // very approximate: 4 bytes per instruction for code, 1 byte for debug line, and 1-2 bytes for aux data like constants - func.data.reserve(insns.size() * 7); + // very approximate: 4 bytes per instruction for code, 1 byte for debug line, and 1-2 bytes for aux data like constants plus overhead + func.data.reserve(32 + insns.size() * 7); writeFunction(func.data, currentFunction); @@ -242,6 +250,9 @@ void BytecodeBuilder::endFunction(uint8_t maxstacksize, uint8_t numupvalues) constantMap.clear(); tableShapeMap.clear(); + + debugRemarks.clear(); + debugRemarkBuffer.clear(); } void BytecodeBuilder::setMainFunction(uint32_t fid) @@ -505,9 +516,40 @@ uint32_t BytecodeBuilder::getDebugPC() const return uint32_t(insns.size()); } +void BytecodeBuilder::addDebugRemark(const char* format, ...) +{ + if ((dumpFlags & Dump_Remarks) == 0) + return; + + size_t offset = debugRemarkBuffer.size(); + + va_list args; + va_start(args, format); + vformatAppend(debugRemarkBuffer, format, args); + va_end(args); + + // we null-terminate all remarks to avoid storing remark length + debugRemarkBuffer += '\0'; + + debugRemarks[uint32_t(insns.size())] = uint32_t(offset); +} + void BytecodeBuilder::finalize() { LUAU_ASSERT(bytecode.empty()); + + // preallocate space for bytecode blob + size_t capacity = 16; + + for (auto& p : stringTable) + capacity += p.first.length + 2; + + for (const Function& func : functions) + capacity += func.data.size(); + + bytecode.reserve(capacity); + + // assemble final bytecode blob bytecode = char(LBC_VERSION); writeStringTable(bytecode); @@ -663,6 +705,8 @@ void BytecodeBuilder::writeFunction(std::string& ss, uint32_t id) const void BytecodeBuilder::writeLineInfo(std::string& ss) const { + LUAU_ASSERT(!lines.empty()); + // this function encodes lines inside each span as a 8-bit delta to span baseline // span is always a power of two; depending on the line info input, it may need to be as low as 1 int span = 1 << 24; @@ -693,7 +737,17 @@ void BytecodeBuilder::writeLineInfo(std::string& ss) const } // second pass: compute span base - std::vector baseline((lines.size() - 1) / span + 1); + int baselineOne = 0; + std::vector baselineScratch; + int* baseline = &baselineOne; + size_t baselineSize = (lines.size() - 1) / span + 1; + + if (baselineSize > 1) + { + // avoid heap allocation for single-element baseline which is most functions (<256 lines) + baselineScratch.resize(baselineSize); + baseline = baselineScratch.data(); + } for (size_t offset = 0; offset < lines.size(); offset += span) { @@ -725,7 +779,7 @@ void BytecodeBuilder::writeLineInfo(std::string& ss) const int lastLine = 0; - for (size_t i = 0; i < baseline.size(); ++i) + for (size_t i = 0; i < baselineSize; ++i) { writeInt(ss, baseline[i] - lastLine); lastLine = baseline[i]; @@ -1695,6 +1749,14 @@ std::string BytecodeBuilder::dumpCurrentFunction() const continue; } + if (dumpFlags & Dump_Remarks) + { + const uint32_t* remark = debugRemarks.find(uint32_t(code - insns.data())); + + if (remark) + formatAppend(result, "REMARK %s\n", debugRemarkBuffer.c_str() + *remark); + } + if (dumpFlags & Dump_Source) { int line = lines[code - insns.data()]; diff --git a/Compiler/src/Compiler.cpp b/Compiler/src/Compiler.cpp index 8ef69e75..810caaee 100644 --- a/Compiler/src/Compiler.cpp +++ b/Compiler/src/Compiler.cpp @@ -8,12 +8,17 @@ #include "Builtins.h" #include "ConstantFolding.h" +#include "CostModel.h" #include "TableShape.h" #include "ValueTracking.h" #include #include #include +#include + +LUAU_FASTINTVARIABLE(LuauCompileLoopUnrollThreshold, 25) +LUAU_FASTINTVARIABLE(LuauCompileLoopUnrollThresholdMaxBoost, 300) namespace Luau { @@ -77,8 +82,12 @@ struct Compiler , globals(AstName()) , variables(nullptr) , constants(nullptr) + , locstants(nullptr) , tableShapes(nullptr) { + // preallocate some buffers that are very likely to grow anyway; this works around std::vector's inefficient growth policy for small arrays + localStack.reserve(16); + upvals.reserve(16); } uint8_t getLocal(AstLocal* local) @@ -209,7 +218,9 @@ struct Compiler Function& f = functions[func]; f.id = fid; - f.upvals = std::move(upvals); + f.upvals = upvals; + + upvals.clear(); // note: instead of std::move above, we copy & clear to preserve capacity for future pushes return fid; } @@ -2133,10 +2144,119 @@ struct Compiler pushLocal(stat->vars.data[i], uint8_t(vars + i)); } + int getConstantShort(AstExpr* expr) + { + const Constant* c = constants.find(expr); + + if (c && c->type == Constant::Type_Number) + { + double n = c->valueNumber; + + if (n >= -32767 && n <= 32767 && double(int(n)) == n) + return int(n); + } + + return INT_MIN; + } + + bool canUnrollForBody(AstStatFor* stat) + { + struct CanUnrollVisitor : AstVisitor + { + bool result = true; + + bool visit(AstExpr* node) override + { + // functions may capture loop variable, and our upval handling doesn't handle elided variables (constant) + result = result && !node->is(); + return result; + } + + bool visit(AstStat* node) override + { + // while we can easily unroll nested loops, our cost model doesn't take unrolling into account so this can result in code explosion + // we also avoid continue/break since they introduce control flow across iterations + result = result && !node->is() && !node->is() && !node->is(); + return result; + } + }; + + CanUnrollVisitor canUnroll; + stat->body->visit(&canUnroll); + + return canUnroll.result; + } + + bool tryCompileUnrolledFor(AstStatFor* stat, int thresholdBase, int thresholdMaxBoost) + { + int from = getConstantShort(stat->from); + int to = getConstantShort(stat->to); + int step = stat->step ? getConstantShort(stat->step) : 1; + + // check that limits are reasonably small and trip count can be computed + if (from == INT_MIN || to == INT_MIN || step == INT_MIN || step == 0 || (step < 0 && to > from) || (step > 0 && to < from)) + { + bytecode.addDebugRemark("loop unroll failed: invalid iteration count"); + return false; + } + + if (!canUnrollForBody(stat)) + { + bytecode.addDebugRemark("loop unroll failed: unsupported loop body"); + return false; + } + + int tripCount = (to - from) / step + 1; + + if (tripCount > thresholdBase * thresholdMaxBoost / 100) + { + bytecode.addDebugRemark("loop unroll failed: too many iterations (%d)", tripCount); + return false; + } + + AstLocal* var = stat->var; + uint64_t costModel = modelCost(stat->body, &var, 1); + + // we use a dynamic cost threshold that's based on the fixed limit boosted by the cost advantage we gain due to unrolling + bool varc = true; + int unrolledCost = computeCost(costModel, &varc, 1) * tripCount; + int baselineCost = (computeCost(costModel, nullptr, 0) + 1) * tripCount; + int unrollProfit = (unrolledCost == 0) ? thresholdMaxBoost : std::min(thresholdMaxBoost, 100 * baselineCost / unrolledCost); + + int threshold = thresholdBase * unrollProfit / 100; + + if (unrolledCost > threshold) + { + bytecode.addDebugRemark( + "loop unroll failed: too expensive (iterations %d, cost %d, profit %.2fx)", tripCount, unrolledCost, double(unrollProfit) / 100); + return false; + } + + bytecode.addDebugRemark("loop unroll succeeded (iterations %d, cost %d, profit %.2fx)", tripCount, unrolledCost, double(unrollProfit) / 100); + + for (int i = from; step > 0 ? i <= to : i >= to; i += step) + { + // we need to re-fold constants in the loop body with the new value; this reuses computed constant values elsewhere in the tree + locstants[var].type = Constant::Type_Number; + locstants[var].valueNumber = i; + + foldConstants(constants, variables, locstants, stat); + + compileStat(stat->body); + } + + return true; + } + void compileStatFor(AstStatFor* stat) { RegScope rs(this); + // Optimization: small loops can be unrolled when it is profitable + if (options.optimizationLevel >= 2 && isConstant(stat->to) && isConstant(stat->from) && (!stat->step || isConstant(stat->step))) + if (tryCompileUnrolledFor(stat, FInt::LuauCompileLoopUnrollThreshold, FInt::LuauCompileLoopUnrollThresholdMaxBoost)) + return; + size_t oldLocals = localStack.size(); size_t oldJumps = loopJumps.size(); @@ -2826,6 +2946,8 @@ struct Compiler : self(self) , functions(functions) { + // preallocate the result; this works around std::vector's inefficient growth policy for small arrays + functions.reserve(16); } bool visit(AstExprFunction* node) override @@ -2979,6 +3101,7 @@ struct Compiler DenseHashMap globals; DenseHashMap variables; DenseHashMap constants; + DenseHashMap locstants; DenseHashMap tableShapes; unsigned int regTop = 0; @@ -3008,7 +3131,7 @@ void compileOrThrow(BytecodeBuilder& bytecode, AstStatBlock* root, const AstName if (options.optimizationLevel >= 1) { // this pass analyzes constantness of expressions - foldConstants(compiler.constants, compiler.variables, root); + foldConstants(compiler.constants, compiler.variables, compiler.locstants, root); // this pass analyzes table assignments to estimate table shapes for initially empty tables predictTableShapes(compiler.tableShapes, root); diff --git a/Compiler/src/ConstantFolding.cpp b/Compiler/src/ConstantFolding.cpp index 35ea0bf0..7ad91d4b 100644 --- a/Compiler/src/ConstantFolding.cpp +++ b/Compiler/src/ConstantFolding.cpp @@ -191,13 +191,13 @@ struct ConstantVisitor : AstVisitor { DenseHashMap& constants; DenseHashMap& variables; + DenseHashMap& locals; - DenseHashMap locals; - - ConstantVisitor(DenseHashMap& constants, DenseHashMap& variables) + ConstantVisitor( + DenseHashMap& constants, DenseHashMap& variables, DenseHashMap& locals) : constants(constants) , variables(variables) - , locals(nullptr) + , locals(locals) { } @@ -385,9 +385,10 @@ struct ConstantVisitor : AstVisitor } }; -void foldConstants(DenseHashMap& constants, DenseHashMap& variables, AstNode* root) +void foldConstants(DenseHashMap& constants, DenseHashMap& variables, + DenseHashMap& locals, AstNode* root) { - ConstantVisitor visitor{constants, variables}; + ConstantVisitor visitor{constants, variables, locals}; root->visit(&visitor); } diff --git a/Compiler/src/ConstantFolding.h b/Compiler/src/ConstantFolding.h index c0e63539..0a995d75 100644 --- a/Compiler/src/ConstantFolding.h +++ b/Compiler/src/ConstantFolding.h @@ -42,7 +42,8 @@ struct Constant } }; -void foldConstants(DenseHashMap& constants, DenseHashMap& variables, AstNode* root); +void foldConstants(DenseHashMap& constants, DenseHashMap& variables, + DenseHashMap& locals, AstNode* root); } // namespace Compile } // namespace Luau diff --git a/VM/src/lapi.cpp b/VM/src/lapi.cpp index 46b10934..431f7e59 100644 --- a/VM/src/lapi.cpp +++ b/VM/src/lapi.cpp @@ -14,6 +14,8 @@ #include +LUAU_FASTFLAG(LuauGcWorkTrackFix) + const char* lua_ident = "$Lua: Lua 5.1.4 Copyright (C) 1994-2008 Lua.org, PUC-Rio $\n" "$Authors: R. Ierusalimschy, L. H. de Figueiredo & W. Celes $\n" "$URL: www.lua.org $\n"; @@ -1050,6 +1052,7 @@ int lua_gc(lua_State* L, int what, int data) { size_t prevthreshold = g->GCthreshold; size_t amount = (cast_to(size_t, data) << 10); + ptrdiff_t oldcredit = g->gcstate == GCSpause ? 0 : g->GCthreshold - g->totalbytes; // temporarily adjust the threshold so that we can perform GC work if (amount <= g->totalbytes) @@ -1069,9 +1072,9 @@ int lua_gc(lua_State* L, int what, int data) while (g->GCthreshold <= g->totalbytes) { - luaC_step(L, false); + size_t stepsize = luaC_step(L, false); - actualwork += g->gcstepsize; + actualwork += FFlag::LuauGcWorkTrackFix ? stepsize : g->gcstepsize; if (g->gcstate == GCSpause) { /* end of cycle? */ @@ -1107,11 +1110,20 @@ int lua_gc(lua_State* L, int what, int data) // if cycle hasn't finished, advance threshold forward for the amount of extra work performed if (g->gcstate != GCSpause) { - // if a new cycle was triggered by explicit step, we ignore old threshold as that shows an incorrect 'credit' of GC work - if (waspaused) - g->GCthreshold = g->totalbytes + actualwork; + if (FFlag::LuauGcWorkTrackFix) + { + // if a new cycle was triggered by explicit step, old 'credit' of GC work is 0 + ptrdiff_t newthreshold = g->totalbytes + actualwork + oldcredit; + g->GCthreshold = newthreshold < 0 ? 0 : newthreshold; + } else - g->GCthreshold = prevthreshold + actualwork; + { + // if a new cycle was triggered by explicit step, we ignore old threshold as that shows an incorrect 'credit' of GC work + if (waspaused) + g->GCthreshold = g->totalbytes + actualwork; + else + g->GCthreshold = prevthreshold + actualwork; + } } break; } diff --git a/VM/src/lgc.cpp b/VM/src/lgc.cpp index 8fc930d5..e7b73fe7 100644 --- a/VM/src/lgc.cpp +++ b/VM/src/lgc.cpp @@ -13,9 +13,10 @@ #include -#define GC_SWEEPMAX 40 -#define GC_SWEEPCOST 10 -#define GC_SWEEPPAGESTEPCOST 4 +LUAU_FASTFLAGVARIABLE(LuauGcWorkTrackFix, false) +LUAU_FASTFLAGVARIABLE(LuauGcSweepCostFix, false) + +#define GC_SWEEPPAGESTEPCOST (FFlag::LuauGcSweepCostFix ? 16 : 4) #define GC_INTERRUPT(state) \ { \ @@ -64,7 +65,7 @@ static void recordGcStateStep(global_State* g, int startgcstate, double seconds, case GCSpropagate: case GCSpropagateagain: g->gcmetrics.currcycle.marktime += seconds; - g->gcmetrics.currcycle.markrequests += g->gcstepsize; + g->gcmetrics.currcycle.markwork += work; if (assist) g->gcmetrics.currcycle.markassisttime += seconds; @@ -74,7 +75,7 @@ static void recordGcStateStep(global_State* g, int startgcstate, double seconds, break; case GCSsweep: g->gcmetrics.currcycle.sweeptime += seconds; - g->gcmetrics.currcycle.sweeprequests += g->gcstepsize; + g->gcmetrics.currcycle.sweepwork += work; if (assist) g->gcmetrics.currcycle.sweepassisttime += seconds; @@ -87,13 +88,11 @@ static void recordGcStateStep(global_State* g, int startgcstate, double seconds, { g->gcmetrics.stepassisttimeacc += seconds; g->gcmetrics.currcycle.assistwork += work; - g->gcmetrics.currcycle.assistrequests += g->gcstepsize; } else { g->gcmetrics.stepexplicittimeacc += seconds; g->gcmetrics.currcycle.explicitwork += work; - g->gcmetrics.currcycle.explicitrequests += g->gcstepsize; } } @@ -878,11 +877,11 @@ static size_t getheaptrigger(global_State* g, size_t heapgoal) return heaptrigger < int64_t(g->totalbytes) ? g->totalbytes : (heaptrigger > int64_t(heapgoal) ? heapgoal : size_t(heaptrigger)); } -void luaC_step(lua_State* L, bool assist) +size_t luaC_step(lua_State* L, bool assist) { global_State* g = L->global; - int lim = (g->gcstepsize / 100) * g->gcstepmul; /* how much to work */ + int lim = FFlag::LuauGcWorkTrackFix ? g->gcstepsize * g->gcstepmul / 100 : (g->gcstepsize / 100) * g->gcstepmul; /* how much to work */ LUAU_ASSERT(g->totalbytes >= g->GCthreshold); size_t debt = g->totalbytes - g->GCthreshold; @@ -902,12 +901,13 @@ void luaC_step(lua_State* L, bool assist) int lastgcstate = g->gcstate; size_t work = gcstep(L, lim); - (void)work; #ifdef LUAI_GCMETRICS recordGcStateStep(g, lastgcstate, lua_clock() - lasttimestamp, assist, work); #endif + size_t actualstepsize = work * 100 / g->gcstepmul; + // at the end of the last cycle if (g->gcstate == GCSpause) { @@ -927,14 +927,16 @@ void luaC_step(lua_State* L, bool assist) } else { - g->GCthreshold = g->totalbytes + g->gcstepsize; + g->GCthreshold = g->totalbytes + (FFlag::LuauGcWorkTrackFix ? actualstepsize : g->gcstepsize); // compensate if GC is "behind schedule" (has some debt to pay) - if (g->GCthreshold > debt) + if (FFlag::LuauGcWorkTrackFix ? g->GCthreshold >= debt : g->GCthreshold > debt) g->GCthreshold -= debt; } GC_INTERRUPT(lastgcstate); + + return actualstepsize; } void luaC_fullgc(lua_State* L) diff --git a/VM/src/lgc.h b/VM/src/lgc.h index dcd070b7..08d1ff5d 100644 --- a/VM/src/lgc.h +++ b/VM/src/lgc.h @@ -133,7 +133,7 @@ #define luaC_init(L, o, tt) luaC_initobj(L, cast_to(GCObject*, (o)), tt) LUAI_FUNC void luaC_freeall(lua_State* L); -LUAI_FUNC void luaC_step(lua_State* L, bool assist); +LUAI_FUNC size_t luaC_step(lua_State* L, bool assist); LUAI_FUNC void luaC_fullgc(lua_State* L); LUAI_FUNC void luaC_initobj(lua_State* L, GCObject* o, uint8_t tt); LUAI_FUNC void luaC_initupval(lua_State* L, UpVal* uv); diff --git a/VM/src/lstate.h b/VM/src/lstate.h index e7c37373..45d9ba2c 100644 --- a/VM/src/lstate.h +++ b/VM/src/lstate.h @@ -106,7 +106,7 @@ struct GCCycleMetrics double markassisttime = 0.0; double markmaxexplicittime = 0.0; size_t markexplicitsteps = 0; - size_t markrequests = 0; + size_t markwork = 0; double atomicstarttimestamp = 0.0; size_t atomicstarttotalsizebytes = 0; @@ -122,10 +122,7 @@ struct GCCycleMetrics double sweepassisttime = 0.0; double sweepmaxexplicittime = 0.0; size_t sweepexplicitsteps = 0; - size_t sweeprequests = 0; - - size_t assistrequests = 0; - size_t explicitrequests = 0; + size_t sweepwork = 0; size_t assistwork = 0; size_t explicitwork = 0; diff --git a/bench/bench.py b/bench/bench.py index 39f219f3..67fc8cf7 100644 --- a/bench/bench.py +++ b/bench/bench.py @@ -814,13 +814,12 @@ def run(args, argsubcb): analyzeResult('', mainResult, compareResults) else: - for subdir, dirs, files in os.walk(arguments.folder): - for filename in files: - filepath = subdir + os.sep + filename - - if filename.endswith(".lua"): - if arguments.run_test == None or re.match(arguments.run_test, filename[:-4]): - runTest(subdir, filename, filepath) + all_files = [subdir + os.sep + filename for subdir, dirs, files in os.walk(arguments.folder) for filename in files] + for filepath in sorted(all_files): + subdir, filename = os.path.split(filepath) + if filename.endswith(".lua"): + if arguments.run_test == None or re.match(arguments.run_test, filename[:-4]): + runTest(subdir, filename, filepath) if arguments.sort and len(plotValueLists) > 1: rearrange(rearrangeSortKeyForComparison) diff --git a/fuzz/proto.cpp b/fuzz/proto.cpp index 1022831b..a48f068b 100644 --- a/fuzz/proto.cpp +++ b/fuzz/proto.cpp @@ -103,7 +103,7 @@ int registerTypes(Luau::TypeChecker& env) // Vector3 stub TypeId vector3MetaType = arena.addType(TableTypeVar{}); - TypeId vector3InstanceType = arena.addType(ClassTypeVar{"Vector3", {}, nullopt, vector3MetaType, {}, {}}); + TypeId vector3InstanceType = arena.addType(ClassTypeVar{"Vector3", {}, nullopt, vector3MetaType, {}, {}, "Test"}); getMutable(vector3InstanceType)->props = { {"X", {env.numberType}}, {"Y", {env.numberType}}, @@ -117,7 +117,7 @@ int registerTypes(Luau::TypeChecker& env) env.globalScope->exportedTypeBindings["Vector3"] = TypeFun{{}, vector3InstanceType}; // Instance stub - TypeId instanceType = arena.addType(ClassTypeVar{"Instance", {}, nullopt, nullopt, {}, {}}); + TypeId instanceType = arena.addType(ClassTypeVar{"Instance", {}, nullopt, nullopt, {}, {}, "Test"}); getMutable(instanceType)->props = { {"Name", {env.stringType}}, }; @@ -125,7 +125,7 @@ int registerTypes(Luau::TypeChecker& env) env.globalScope->exportedTypeBindings["Instance"] = TypeFun{{}, instanceType}; // Part stub - TypeId partType = arena.addType(ClassTypeVar{"Part", {}, instanceType, nullopt, {}, {}}); + TypeId partType = arena.addType(ClassTypeVar{"Part", {}, instanceType, nullopt, {}, {}, "Test"}); getMutable(partType)->props = { {"Position", {vector3InstanceType}}, }; @@ -173,7 +173,7 @@ struct FuzzConfigResolver : Luau::ConfigResolver { FuzzConfigResolver() { - defaultConfig.mode = Luau::Mode::Nonstrict; // typecheckTwice option will cover Strict mode + defaultConfig.mode = Luau::Mode::Nonstrict; defaultConfig.enabledLint.warningMask = ~0ull; defaultConfig.parseOptions.captureComments = true; } @@ -275,6 +275,11 @@ DEFINE_PROTO_FUZZER(const luau::ModuleSet& message) // lint (note that we need access to types so we need to do this with typeck in scope) if (kFuzzLinter && result.errors.empty()) frontend.lint(name, std::nullopt); + + // Second pass in strict mode (forced by auto-complete) + Luau::FrontendOptions opts; + opts.forAutocomplete = true; + frontend.check(name, opts); } catch (std::exception&) { diff --git a/tests/Autocomplete.test.cpp b/tests/Autocomplete.test.cpp index 2e7902f5..f66e23ed 100644 --- a/tests/Autocomplete.test.cpp +++ b/tests/Autocomplete.test.cpp @@ -3034,4 +3034,40 @@ string:@1 CHECK(ac.entryMap["sub"].wrongIndexType == true); } +TEST_CASE_FIXTURE(ACFixture, "source_module_preservation_and_invalidation") +{ + check(R"( +local a = { x = 2, y = 4 } +a.@1 + )"); + + frontend.clear(); + + auto ac = autocomplete('1'); + + CHECK(ac.entryMap.count("x")); + CHECK(ac.entryMap.count("y")); + + frontend.check("MainModule", {}); + + ac = autocomplete('1'); + + CHECK(ac.entryMap.count("x")); + CHECK(ac.entryMap.count("y")); + + frontend.markDirty("MainModule", nullptr); + + ac = autocomplete('1'); + + CHECK(ac.entryMap.count("x")); + CHECK(ac.entryMap.count("y")); + + frontend.check("MainModule", {}); + + ac = autocomplete('1'); + + CHECK(ac.entryMap.count("x")); + CHECK(ac.entryMap.count("y")); +} + TEST_SUITE_END(); diff --git a/tests/Compiler.test.cpp b/tests/Compiler.test.cpp index 83dad729..f3e60690 100644 --- a/tests/Compiler.test.cpp +++ b/tests/Compiler.test.cpp @@ -17,11 +17,13 @@ std::string rep(const std::string& s, size_t n); using namespace Luau; -static std::string compileFunction(const char* source, uint32_t id) +static std::string compileFunction(const char* source, uint32_t id, int optimizationLevel = 1) { Luau::BytecodeBuilder bcb; bcb.setDumpFlags(Luau::BytecodeBuilder::Dump_Code); - Luau::compileOrThrow(bcb, source); + Luau::CompileOptions options; + options.optimizationLevel = optimizationLevel; + Luau::compileOrThrow(bcb, source, options); return bcb.dumpFunction(id); } @@ -2689,6 +2691,27 @@ local 8: reg 3, start pc 34 line 21, end pc 34 line 21 )"); } +TEST_CASE("DebugRemarks") +{ + Luau::BytecodeBuilder bcb; + bcb.setDumpFlags(Luau::BytecodeBuilder::Dump_Code | Luau::BytecodeBuilder::Dump_Remarks); + + uint32_t fid = bcb.beginFunction(0); + + bcb.addDebugRemark("test remark #%d", 42); + bcb.emitABC(LOP_RETURN, 0, 1, 0); + + bcb.endFunction(0, 0); + + bcb.setMainFunction(fid); + bcb.finalize(); + + CHECK_EQ("\n" + bcb.dumpFunction(0), R"( +REMARK test remark #42 +RETURN R0 0 +)"); +} + TEST_CASE("AssignmentConflict") { // assignments are left to right @@ -4076,4 +4099,336 @@ RETURN R1 6 )"); } +TEST_CASE("LoopUnrollBasic") +{ + // forward loops + CHECK_EQ("\n" + compileFunction(R"( +local t = {} +for i=1,2 do + t[i] = i +end +return t +)", + 0, 2), + R"( +NEWTABLE R0 0 2 +LOADN R1 1 +SETTABLEN R1 R0 1 +LOADN R1 2 +SETTABLEN R1 R0 2 +RETURN R0 1 +)"); + + // backward loops + CHECK_EQ("\n" + compileFunction(R"( +local t = {} +for i=2,1,-1 do + t[i] = i +end +return t +)", + 0, 2), + R"( +NEWTABLE R0 0 0 +LOADN R1 2 +SETTABLEN R1 R0 2 +LOADN R1 1 +SETTABLEN R1 R0 1 +RETURN R0 1 +)"); + + // loops with step that doesn't divide to-from + CHECK_EQ("\n" + compileFunction(R"( +local t = {} +for i=1,4,2 do + t[i] = i +end +return t +)", + 0, 2), + R"( +NEWTABLE R0 0 0 +LOADN R1 1 +SETTABLEN R1 R0 1 +LOADN R1 3 +SETTABLEN R1 R0 3 +RETURN R0 1 +)"); +} + +TEST_CASE("LoopUnrollUnsupported") +{ + // can't unroll loops with non-constant bounds + CHECK_EQ("\n" + compileFunction(R"( +for i=x,y,z do +end +)", + 0, 2), + R"( +GETIMPORT R2 1 +GETIMPORT R0 3 +GETIMPORT R1 5 +FORNPREP R0 +1 +FORNLOOP R0 -1 +RETURN R0 0 +)"); + + // can't unroll loops with bounds where we can't compute trip count + CHECK_EQ("\n" + compileFunction(R"( +for i=2,1 do +end +)", + 0, 2), + R"( +LOADN R2 2 +LOADN R0 1 +LOADN R1 1 +FORNPREP R0 +1 +FORNLOOP R0 -1 +RETURN R0 0 +)"); + + // can't unroll loops with bounds that might be imprecise (non-integer) + CHECK_EQ("\n" + compileFunction(R"( +for i=1,2,0.1 do +end +)", + 0, 2), + R"( +LOADN R2 1 +LOADN R0 2 +LOADK R1 K0 +FORNPREP R0 +1 +FORNLOOP R0 -1 +RETURN R0 0 +)"); + + // can't unroll loops if the bounds are too large, as it might overflow trip count math + CHECK_EQ("\n" + compileFunction(R"( +for i=4294967295,4294967296 do +end +)", + 0, 2), + R"( +LOADK R2 K0 +LOADK R0 K1 +LOADN R1 1 +FORNPREP R0 +1 +FORNLOOP R0 -1 +RETURN R0 0 +)"); + + // can't unroll loops if the body has loop control flow or nested loops + CHECK_EQ("\n" + compileFunction(R"( +for i=1,1 do + for j=1,1 do + if i == 1 then + continue + else + break + end + end +end +)", + 0, 2), + R"( +LOADN R2 1 +LOADN R0 1 +LOADN R1 1 +FORNPREP R0 +11 +LOADN R5 1 +LOADN R3 1 +LOADN R4 1 +FORNPREP R3 +6 +JUMPIFNOTEQK R2 K0 +5 +JUMP +2 +JUMP +1 +JUMP +1 +FORNLOOP R3 -6 +FORNLOOP R0 -11 +RETURN R0 0 +)"); + + // can't unroll loops if the body has functions that refer to loop variables + CHECK_EQ("\n" + compileFunction(R"( +for i=1,1 do + local x = function() return i end +end +)", + 1, 2), + R"( +LOADN R2 1 +LOADN R0 1 +LOADN R1 1 +FORNPREP R0 +3 +NEWCLOSURE R3 P0 +CAPTURE VAL R2 +FORNLOOP R0 -3 +RETURN R0 0 +)"); +} + +TEST_CASE("LoopUnrollCost") +{ + ScopedFastInt sfis[] = { + {"LuauCompileLoopUnrollThreshold", 25}, + {"LuauCompileLoopUnrollThresholdMaxBoost", 300}, + }; + + // loops with short body + CHECK_EQ("\n" + compileFunction(R"( +local t = {} +for i=1,10 do + t[i] = i +end +return t +)", + 0, 2), + R"( +NEWTABLE R0 0 10 +LOADN R1 1 +SETTABLEN R1 R0 1 +LOADN R1 2 +SETTABLEN R1 R0 2 +LOADN R1 3 +SETTABLEN R1 R0 3 +LOADN R1 4 +SETTABLEN R1 R0 4 +LOADN R1 5 +SETTABLEN R1 R0 5 +LOADN R1 6 +SETTABLEN R1 R0 6 +LOADN R1 7 +SETTABLEN R1 R0 7 +LOADN R1 8 +SETTABLEN R1 R0 8 +LOADN R1 9 +SETTABLEN R1 R0 9 +LOADN R1 10 +SETTABLEN R1 R0 10 +RETURN R0 1 +)"); + + // loops with body that's too long + CHECK_EQ("\n" + compileFunction(R"( +local t = {} +for i=1,100 do + t[i] = i +end +return t +)", + 0, 2), + R"( +NEWTABLE R0 0 0 +LOADN R3 1 +LOADN R1 100 +LOADN R2 1 +FORNPREP R1 +2 +SETTABLE R3 R0 R3 +FORNLOOP R1 -2 +RETURN R0 1 +)"); + + // loops with body that's long but has a high boost factor due to constant folding + CHECK_EQ("\n" + compileFunction(R"( +local t = {} +for i=1,30 do + t[i] = i * i * i +end +return t +)", + 0, 2), + R"( +NEWTABLE R0 0 0 +LOADN R1 1 +SETTABLEN R1 R0 1 +LOADN R1 8 +SETTABLEN R1 R0 2 +LOADN R1 27 +SETTABLEN R1 R0 3 +LOADN R1 64 +SETTABLEN R1 R0 4 +LOADN R1 125 +SETTABLEN R1 R0 5 +LOADN R1 216 +SETTABLEN R1 R0 6 +LOADN R1 343 +SETTABLEN R1 R0 7 +LOADN R1 512 +SETTABLEN R1 R0 8 +LOADN R1 729 +SETTABLEN R1 R0 9 +LOADN R1 1000 +SETTABLEN R1 R0 10 +LOADN R1 1331 +SETTABLEN R1 R0 11 +LOADN R1 1728 +SETTABLEN R1 R0 12 +LOADN R1 2197 +SETTABLEN R1 R0 13 +LOADN R1 2744 +SETTABLEN R1 R0 14 +LOADN R1 3375 +SETTABLEN R1 R0 15 +LOADN R1 4096 +SETTABLEN R1 R0 16 +LOADN R1 4913 +SETTABLEN R1 R0 17 +LOADN R1 5832 +SETTABLEN R1 R0 18 +LOADN R1 6859 +SETTABLEN R1 R0 19 +LOADN R1 8000 +SETTABLEN R1 R0 20 +LOADN R1 9261 +SETTABLEN R1 R0 21 +LOADN R1 10648 +SETTABLEN R1 R0 22 +LOADN R1 12167 +SETTABLEN R1 R0 23 +LOADN R1 13824 +SETTABLEN R1 R0 24 +LOADN R1 15625 +SETTABLEN R1 R0 25 +LOADN R1 17576 +SETTABLEN R1 R0 26 +LOADN R1 19683 +SETTABLEN R1 R0 27 +LOADN R1 21952 +SETTABLEN R1 R0 28 +LOADN R1 24389 +SETTABLEN R1 R0 29 +LOADN R1 27000 +SETTABLEN R1 R0 30 +RETURN R0 1 +)"); + + // loops with body that's long and doesn't have a high boost factor + CHECK_EQ("\n" + compileFunction(R"( +local t = {} +for i=1,10 do + t[i] = math.abs(math.sin(i)) +end +return t +)", + 0, 2), + R"( +NEWTABLE R0 0 10 +LOADN R3 1 +LOADN R1 10 +LOADN R2 1 +FORNPREP R1 +11 +FASTCALL1 24 R3 +3 +MOVE R6 R3 +GETIMPORT R5 2 +CALL R5 1 -1 +FASTCALL 2 +2 +GETIMPORT R4 4 +CALL R4 -1 1 +SETTABLE R4 R0 R3 +FORNLOOP R1 -11 +RETURN R0 1 +)"); +} + TEST_SUITE_END(); diff --git a/tests/CostModel.test.cpp b/tests/CostModel.test.cpp index ec04932f..aa5b7284 100644 --- a/tests/CostModel.test.cpp +++ b/tests/CostModel.test.cpp @@ -98,4 +98,129 @@ end CHECK_EQ(2, Luau::Compile::computeCost(model, args2, 1)); } +TEST_CASE("ImportCall") +{ + uint64_t model = modelFunction(R"( +function test(a) + return Instance.new(a) +end +)"); + + const bool args1[] = {false}; + const bool args2[] = {true}; + + CHECK_EQ(6, Luau::Compile::computeCost(model, args1, 1)); + CHECK_EQ(6, Luau::Compile::computeCost(model, args2, 1)); +} + +TEST_CASE("FastCall") +{ + uint64_t model = modelFunction(R"( +function test(a) + return math.abs(a + 1) +end +)"); + + const bool args1[] = {false}; + const bool args2[] = {true}; + + // note: we currently don't treat fast calls differently from cost model perspective + CHECK_EQ(6, Luau::Compile::computeCost(model, args1, 1)); + CHECK_EQ(5, Luau::Compile::computeCost(model, args2, 1)); +} + +TEST_CASE("ControlFlow") +{ + uint64_t model = modelFunction(R"( +function test(a) + while a < 0 do + a += 1 + end + for i=1,2 do + a += 1 + end + for i in pairs({}) do + a += 1 + if a % 2 == 0 then continue end + end + repeat + a += 1 + if a % 2 == 0 then break end + until a > 10 + return a +end +)"); + + const bool args1[] = {false}; + const bool args2[] = {true}; + + CHECK_EQ(38, Luau::Compile::computeCost(model, args1, 1)); + CHECK_EQ(37, Luau::Compile::computeCost(model, args2, 1)); +} + +TEST_CASE("Conditional") +{ + uint64_t model = modelFunction(R"( +function test(a) + return if a < 0 then -a else a +end +)"); + + const bool args1[] = {false}; + const bool args2[] = {true}; + + CHECK_EQ(4, Luau::Compile::computeCost(model, args1, 1)); + CHECK_EQ(2, Luau::Compile::computeCost(model, args2, 1)); +} + +TEST_CASE("VarArgs") +{ + uint64_t model = modelFunction(R"( +function test(...) + return select('#', ...) :: number +end +)"); + + CHECK_EQ(8, Luau::Compile::computeCost(model, nullptr, 0)); +} + +TEST_CASE("TablesFunctions") +{ + uint64_t model = modelFunction(R"( +function test() + return { 42, op = function() end } +end +)"); + + CHECK_EQ(22, Luau::Compile::computeCost(model, nullptr, 0)); +} + +TEST_CASE("CostOverflow") +{ + uint64_t model = modelFunction(R"( +function test() + return {{{{{{{{{{{{{{{}}}}}}}}}}}}}}} +end +)"); + + CHECK_EQ(127, Luau::Compile::computeCost(model, nullptr, 0)); +} + +TEST_CASE("TableAssign") +{ + uint64_t model = modelFunction(R"( +function test(a) + for i=1,#a do + a[i] = i + end +end +)"); + + const bool args1[] = {false}; + const bool args2[] = {true}; + + CHECK_EQ(4, Luau::Compile::computeCost(model, args1, 1)); + CHECK_EQ(3, Luau::Compile::computeCost(model, args2, 1)); +} + TEST_SUITE_END(); diff --git a/tests/JsonEncoder.test.cpp b/tests/JsonEncoder.test.cpp index 1d2ad645..6f1cebc6 100644 --- a/tests/JsonEncoder.test.cpp +++ b/tests/JsonEncoder.test.cpp @@ -9,6 +9,46 @@ using namespace Luau; +struct JsonEncoderFixture +{ + Allocator allocator; + AstNameTable names{allocator}; + + ParseResult parse(std::string_view src) + { + ParseOptions opts; + opts.allowDeclarationSyntax = true; + return Parser::parse(src.data(), src.size(), names, allocator, opts); + } + + AstStatBlock* expectParse(std::string_view src) + { + ParseResult res = parse(src); + REQUIRE(res.errors.size() == 0); + return res.root; + } + + AstStat* expectParseStatement(std::string_view src) + { + AstStatBlock* root = expectParse(src); + REQUIRE(1 == root->body.size); + return root->body.data[0]; + } + + AstExpr* expectParseExpr(std::string_view src) + { + std::string s = "a = "; + s.append(src); + AstStatBlock* root = expectParse(s); + + AstStatAssign* statAssign = root->body.data[0]->as(); + REQUIRE(statAssign != nullptr); + REQUIRE(statAssign->values.size == 1); + + return statAssign->values.data[0]; + } +}; + TEST_SUITE_BEGIN("JsonEncoderTests"); TEST_CASE("encode_constants") @@ -51,7 +91,7 @@ TEST_CASE("encode_AstStatBlock") toJson(&block)); } -TEST_CASE("encode_tables") +TEST_CASE_FIXTURE(JsonEncoderFixture, "encode_tables") { std::string src = R"( local x: { @@ -61,16 +101,294 @@ TEST_CASE("encode_tables") } )"; - Allocator allocator; - AstNameTable names(allocator); - ParseResult parseResult = Parser::parse(src.c_str(), src.length(), names, allocator); - - REQUIRE(parseResult.errors.size() == 0); - std::string json = toJson(parseResult.root); + AstStatBlock* root = expectParse(src); + std::string json = toJson(root); CHECK( json == R"({"type":"AstStatBlock","location":"0,0 - 6,4","body":[{"type":"AstStatLocal","location":"1,8 - 5,9","vars":[{"type":{"type":"AstTypeTable","location":"1,17 - 3,9","props":[{"name":"foo","location":"2,12 - 2,15","type":{"type":"AstTypeReference","location":"2,17 - 2,23","name":"number","parameters":[]}}],"indexer":false},"name":"x","location":"1,14 - 1,15"}],"values":[{"type":"AstExprTable","location":"3,12 - 5,9","items":[{"kind":"record","key":{"type":"AstExprConstantString","location":"4,12 - 4,15","value":"foo"},"value":{"type":"AstExprConstantNumber","location":"4,18 - 4,21","value":123}}]}]}]})"); } +TEST_CASE("encode_AstExprGroup") +{ + AstExprConstantNumber number{Location{}, 5.0}; + AstExprGroup group{Location{}, &number}; + + std::string json = toJson(&group); + + const std::string expected = R"({"type":"AstExprGroup","location":"0,0 - 0,0","expr":{"type":"AstExprConstantNumber","location":"0,0 - 0,0","value":5}})"; + + CHECK(json == expected); +} + +TEST_CASE("encode_AstExprGlobal") +{ + AstExprGlobal global{Location{}, AstName{"print"}}; + + std::string json = toJson(&global); + std::string expected = R"({"type":"AstExprGlobal","location":"0,0 - 0,0","global":"print"})"; + + CHECK(json == expected); +} + +TEST_CASE("encode_AstExprLocal") +{ + AstLocal local{AstName{"foo"}, Location{}, nullptr, 0, 0, nullptr}; + AstExprLocal exprLocal{Location{}, &local, false}; + + CHECK(toJson(&exprLocal) == R"({"type":"AstExprLocal","location":"0,0 - 0,0","local":{"type":null,"name":"foo","location":"0,0 - 0,0"}})"); +} + +TEST_CASE("encode_AstExprVarargs") +{ + AstExprVarargs varargs{Location{}}; + + CHECK(toJson(&varargs) == R"({"type":"AstExprVarargs","location":"0,0 - 0,0"})"); +} + +TEST_CASE_FIXTURE(JsonEncoderFixture, "encode_AstExprCall") +{ + AstExpr* expr = expectParseExpr("foo(1, 2, 3)"); + std::string_view expected = R"({"type":"AstExprCall","location":"0,4 - 0,16","func":{"type":"AstExprGlobal","location":"0,4 - 0,7","global":"foo"},"args":[{"type":"AstExprConstantNumber","location":"0,8 - 0,9","value":1},{"type":"AstExprConstantNumber","location":"0,11 - 0,12","value":2},{"type":"AstExprConstantNumber","location":"0,14 - 0,15","value":3}],"self":false,"argLocation":"0,8 - 0,16"})"; + + CHECK(toJson(expr) == expected); +} + +TEST_CASE_FIXTURE(JsonEncoderFixture, "encode_AstExprIndexName") +{ + AstExpr* expr = expectParseExpr("foo.bar"); + + std::string_view expected = R"({"type":"AstExprIndexName","location":"0,4 - 0,11","expr":{"type":"AstExprGlobal","location":"0,4 - 0,7","global":"foo"},"index":"bar","indexLocation":"0,8 - 0,11","op":"."})"; + + CHECK(toJson(expr) == expected); +} + +TEST_CASE_FIXTURE(JsonEncoderFixture, "encode_AstExprIndexExpr") +{ + AstExpr* expr = expectParseExpr("foo['bar']"); + + std::string_view expected = R"({"type":"AstExprIndexExpr","location":"0,4 - 0,14","expr":{"type":"AstExprGlobal","location":"0,4 - 0,7","global":"foo"},"index":{"type":"AstExprConstantString","location":"0,8 - 0,13","value":"bar"}})"; + + CHECK(toJson(expr) == expected); +} + +TEST_CASE_FIXTURE(JsonEncoderFixture, "encode_AstExprFunction") +{ + AstExpr* expr = expectParseExpr("function (a) return a end"); + + std::string_view expected = R"({"type":"AstExprFunction","location":"0,4 - 0,29","generics":[],"genericPacks":[],"args":[{"type":null,"name":"a","location":"0,14 - 0,15"}],"vararg":false,"varargLocation":"0,0 - 0,0","body":{"type":"AstStatBlock","location":"0,16 - 0,26","body":[{"type":"AstStatReturn","location":"0,17 - 0,25","list":[{"type":"AstExprLocal","location":"0,24 - 0,25","local":{"type":null,"name":"a","location":"0,14 - 0,15"}}]}]},"functionDepth":1,"debugname":"","hasEnd":true})"; + + CHECK(toJson(expr) == expected); +} + +TEST_CASE_FIXTURE(JsonEncoderFixture, "encode_AstExprTable") +{ + AstExpr* expr = expectParseExpr("{true, key=true, [key2]=true}"); + + std::string_view expected = R"({"type":"AstExprTable","location":"0,4 - 0,33","items":[{"kind":"item","value":{"type":"AstExprConstantBool","location":"0,5 - 0,9","value":true}},{"kind":"record","key":{"type":"AstExprConstantString","location":"0,11 - 0,14","value":"key"},"value":{"type":"AstExprConstantBool","location":"0,15 - 0,19","value":true}},{"kind":"general","key":{"type":"AstExprGlobal","location":"0,22 - 0,26","global":"key2"},"value":{"type":"AstExprConstantBool","location":"0,28 - 0,32","value":true}}]})"; + + CHECK(toJson(expr) == expected); +} + +TEST_CASE_FIXTURE(JsonEncoderFixture, "encode_AstExprUnary") +{ + AstExpr* expr = expectParseExpr("-b"); + + std::string_view expected = R"({"type":"AstExprUnary","location":"0,4 - 0,6","op":"minus","expr":{"type":"AstExprGlobal","location":"0,5 - 0,6","global":"b"}})"; + + CHECK(toJson(expr) == expected); +} + +TEST_CASE_FIXTURE(JsonEncoderFixture, "encode_AstExprBinary") +{ + AstExpr* expr = expectParseExpr("b + c"); + + std::string_view expected = R"({"type":"AstExprBinary","location":"0,4 - 0,9","op":"Add","left":{"type":"AstExprGlobal","location":"0,4 - 0,5","global":"b"},"right":{"type":"AstExprGlobal","location":"0,8 - 0,9","global":"c"}})"; + + CHECK(toJson(expr) == expected); +} + +TEST_CASE_FIXTURE(JsonEncoderFixture, "encode_AstExprTypeAssertion") +{ + AstExpr* expr = expectParseExpr("b :: any"); + + std::string_view expected = R"({"type":"AstExprTypeAssertion","location":"0,4 - 0,12","expr":{"type":"AstExprGlobal","location":"0,4 - 0,5","global":"b"},"annotation":{"type":"AstTypeReference","location":"0,9 - 0,12","name":"any","parameters":[]}})"; + + CHECK(toJson(expr) == expected); +} + +TEST_CASE_FIXTURE(JsonEncoderFixture, "encode_AstExprError") +{ + std::string_view src = "a = "; + ParseResult parseResult = Parser::parse(src.data(), src.size(), names, allocator); + + REQUIRE(1 == parseResult.root->body.size); + + AstStatAssign* statAssign = parseResult.root->body.data[0]->as(); + REQUIRE(statAssign != nullptr); + REQUIRE(1 == statAssign->values.size); + + AstExpr* expr = statAssign->values.data[0]; + + std::string_view expected = R"({"type":"AstExprError","location":"0,4 - 0,4","expressions":[],"messageIndex":0})"; + + CHECK(toJson(expr) == expected); +} + +TEST_CASE_FIXTURE(JsonEncoderFixture, "encode_AstStatIf") +{ + AstStat* statement = expectParseStatement("if true then else end"); + + std::string_view expected = R"({"type":"AstStatIf","location":"0,0 - 0,21","condition":{"type":"AstExprConstantBool","location":"0,3 - 0,7","value":true},"thenbody":{"type":"AstStatBlock","location":"0,12 - 0,13","body":[]},"elsebody":{"type":"AstStatBlock","location":"0,17 - 0,18","body":[]},"hasThen":true,"hasEnd":true})"; + + CHECK(toJson(statement) == expected); +} + +TEST_CASE_FIXTURE(JsonEncoderFixture, "encode_AstStatWhile") +{ + AstStat* statement = expectParseStatement("while true do end"); + + std::string_view expected = R"({"type":"AtStatWhile","location":"0,0 - 0,17","condition":{"type":"AstExprConstantBool","location":"0,6 - 0,10","value":true},"body":{"type":"AstStatBlock","location":"0,13 - 0,14","body":[]},"hasDo":true,"hasEnd":true})"; + + CHECK(toJson(statement) == expected); +} + +TEST_CASE_FIXTURE(JsonEncoderFixture, "encode_AstStatRepeat") +{ + AstStat* statement = expectParseStatement("repeat until true"); + + std::string_view expected = R"({"type":"AstStatRepeat","location":"0,0 - 0,17","condition":{"type":"AstExprConstantBool","location":"0,13 - 0,17","value":true},"body":{"type":"AstStatBlock","location":"0,6 - 0,7","body":[]},"hasUntil":true})"; + + CHECK(toJson(statement) == expected); +} + +TEST_CASE_FIXTURE(JsonEncoderFixture, "encode_AstStatBreak") +{ + AstStat* statement = expectParseStatement("while true do break end"); + + std::string_view expected = R"({"type":"AtStatWhile","location":"0,0 - 0,23","condition":{"type":"AstExprConstantBool","location":"0,6 - 0,10","value":true},"body":{"type":"AstStatBlock","location":"0,13 - 0,20","body":[{"type":"AstStatBreak","location":"0,14 - 0,19"}]},"hasDo":true,"hasEnd":true})"; + + CHECK(toJson(statement) == expected); +} + +TEST_CASE_FIXTURE(JsonEncoderFixture, "encode_AstStatContinue") +{ + AstStat* statement = expectParseStatement("while true do continue end"); + + std::string_view expected = R"({"type":"AtStatWhile","location":"0,0 - 0,26","condition":{"type":"AstExprConstantBool","location":"0,6 - 0,10","value":true},"body":{"type":"AstStatBlock","location":"0,13 - 0,23","body":[{"type":"AstStatContinue","location":"0,14 - 0,22"}]},"hasDo":true,"hasEnd":true})"; + + CHECK(toJson(statement) == expected); +} + +TEST_CASE_FIXTURE(JsonEncoderFixture, "encode_AstStatFor") +{ + AstStat* statement = expectParseStatement("for a=0,1 do end"); + + std::string_view expected = R"({"type":"AstStatFor","location":"0,0 - 0,16","var":{"type":null,"name":"a","location":"0,4 - 0,5"},"from":{"type":"AstExprConstantNumber","location":"0,6 - 0,7","value":0},"to":{"type":"AstExprConstantNumber","location":"0,8 - 0,9","value":1},"body":{"type":"AstStatBlock","location":"0,12 - 0,13","body":[]},"hasDo":true,"hasEnd":true})"; + + CHECK(toJson(statement) == expected); +} + +TEST_CASE_FIXTURE(JsonEncoderFixture, "encode_AstStatForIn") +{ + AstStat* statement = expectParseStatement("for a in b do end"); + + std::string_view expected = R"({"type":"AstStatForIn","location":"0,0 - 0,17","vars":[{"type":null,"name":"a","location":"0,4 - 0,5"}],"values":[{"type":"AstExprGlobal","location":"0,9 - 0,10","global":"b"}],"body":{"type":"AstStatBlock","location":"0,13 - 0,14","body":[]},"hasIn":true,"hasDo":true,"hasEnd":true})"; + + CHECK(toJson(statement) == expected); +} + +TEST_CASE_FIXTURE(JsonEncoderFixture, "encode_AstStatCompoundAssign") +{ + AstStat* statement = expectParseStatement("a += b"); + + std::string_view expected = R"({"type":"AstStatCompoundAssign","location":"0,0 - 0,6","op":"Add","var":{"type":"AstExprGlobal","location":"0,0 - 0,1","global":"a"},"value":{"type":"AstExprGlobal","location":"0,5 - 0,6","global":"b"}})"; + + CHECK(toJson(statement) == expected); +} + +TEST_CASE_FIXTURE(JsonEncoderFixture, "encode_AstStatLocalFunction") +{ + AstStat* statement = expectParseStatement("local function a(b) return end"); + + std::string_view expected = R"({"type":"AstStatLocalFunction","location":"0,0 - 0,30","name":{"type":null,"name":"a","location":"0,15 - 0,16"},"func":{"type":"AstExprFunction","location":"0,0 - 0,30","generics":[],"genericPacks":[],"args":[{"type":null,"name":"b","location":"0,17 - 0,18"}],"vararg":false,"varargLocation":"0,0 - 0,0","body":{"type":"AstStatBlock","location":"0,19 - 0,27","body":[{"type":"AstStatReturn","location":"0,20 - 0,26","list":[]}]},"functionDepth":1,"debugname":"a","hasEnd":true}})"; + + CHECK(toJson(statement) == expected); +} + +TEST_CASE_FIXTURE(JsonEncoderFixture, "encode_AstStatTypeAlias") +{ + AstStat* statement = expectParseStatement("type A = B"); + + std::string_view expected = R"({"type":"AstStatTypeAlias","location":"0,0 - 0,10","name":"A","generics":[],"genericPacks":[],"type":{"type":"AstTypeReference","location":"0,9 - 0,10","name":"B","parameters":[]},"exported":false})"; + + CHECK(toJson(statement) == expected); +} + +TEST_CASE_FIXTURE(JsonEncoderFixture, "encode_AstStatDeclareFunction") +{ + AstStat* statement = expectParseStatement("declare function foo(x: number): string"); + + std::string_view expected = R"({"type":"AstStatDeclareFunction","location":"0,0 - 0,39","name":"foo","params":{"types":[{"type":"AstTypeReference","location":"0,24 - 0,30","name":"number","parameters":[]}]},"retTypes":{"types":[{"type":"AstTypeReference","location":"0,33 - 0,39","name":"string","parameters":[]}]},"generics":[],"genericPacks":[]})"; + + CHECK(toJson(statement) == expected); +} + +TEST_CASE_FIXTURE(JsonEncoderFixture, "encode_AstStatDeclareClass") +{ + AstStatBlock* root = expectParse(R"( + declare class Foo + prop: number + function method(self, foo: number): string + end + + declare class Bar extends Foo + prop2: string + end + )"); + + REQUIRE(2 == root->body.size); + + std::string_view expected1 = R"({"type":"AstStatDeclareClass","location":"1,22 - 4,11","name":"Foo","props":[{"name":"prop","type":{"type":"AstTypeReference","location":"2,18 - 2,24","name":"number","parameters":[]}},{"name":"method","type":{"type":"AstTypeFunction","location":"3,21 - 4,11","generics":[],"genericPacks":[],"argTypes":{"types":[{"type":"AstTypeReference","location":"3,39 - 3,45","name":"number","parameters":[]}]},"returnTypes":{"types":[{"type":"AstTypeReference","location":"3,48 - 3,54","name":"string","parameters":[]}]}}}]})"; + CHECK(toJson(root->body.data[0]) == expected1); + + std::string_view expected2 = R"({"type":"AstStatDeclareClass","location":"6,22 - 8,11","name":"Bar","superName":"Foo","props":[{"name":"prop2","type":{"type":"AstTypeReference","location":"7,19 - 7,25","name":"string","parameters":[]}}]})"; + CHECK(toJson(root->body.data[1]) == expected2); +} + +TEST_CASE_FIXTURE(JsonEncoderFixture, "encode_annotation") +{ + AstStat* statement = expectParseStatement("type T = ((number) -> (string | nil)) & ((string) -> ())"); + + std::string_view expected = R"({"type":"AstStatTypeAlias","location":"0,0 - 0,55","name":"T","generics":[],"genericPacks":[],"type":{"type":"AstTypeIntersection","location":"0,9 - 0,55","types":[{"type":"AstTypeFunction","location":"0,10 - 0,35","generics":[],"genericPacks":[],"argTypes":{"types":[{"type":"AstTypeReference","location":"0,11 - 0,17","name":"number","parameters":[]}]},"returnTypes":{"types":[{"type":"AstTypeUnion","location":"0,23 - 0,35","types":[{"type":"AstTypeReference","location":"0,23 - 0,29","name":"string","parameters":[]},{"type":"AstTypeReference","location":"0,32 - 0,35","name":"nil","parameters":[]}]}]}},{"type":"AstTypeFunction","location":"0,41 - 0,55","generics":[],"genericPacks":[],"argTypes":{"types":[{"type":"AstTypeReference","location":"0,42 - 0,48","name":"string","parameters":[]}]},"returnTypes":{"types":[]}}]},"exported":false})"; + + CHECK(toJson(statement) == expected); +} + +TEST_CASE_FIXTURE(JsonEncoderFixture, "encode_AstTypeError") +{ + ParseResult parseResult = parse("type T = "); + REQUIRE(1 == parseResult.root->body.size); + + AstStat* statement = parseResult.root->body.data[0]; + + std::string_view expected = R"({"type":"AstStatTypeAlias","location":"0,0 - 0,9","name":"T","generics":[],"genericPacks":[],"type":{"type":"AstTypeError","location":"0,8 - 0,9","types":[],"messageIndex":0},"exported":false})"; + + CHECK(toJson(statement) == expected); +} + +TEST_CASE_FIXTURE(JsonEncoderFixture, "encode_AstTypePackExplicit") +{ + AstStatBlock* root = expectParse(R"( + type A = () -> T... + local a: A<(number, string)> + )"); + + CHECK(2 == root->body.size); + + std::string_view expected = R"({"type":"AstStatLocal","location":"2,8 - 2,36","vars":[{"type":{"type":"AstTypeReference","location":"2,17 - 2,36","name":"A","parameters":[{"type":"AstTypePackExplicit","location":"2,19 - 2,20","typeList":{"types":[{"type":"AstTypeReference","location":"2,20 - 2,26","name":"number","parameters":[]},{"type":"AstTypeReference","location":"2,28 - 2,34","name":"string","parameters":[]}]}}]},"name":"a","location":"2,14 - 2,15"}],"values":[]})"; + + CHECK(toJson(root->body.data[1]) == expected); +} + TEST_SUITE_END(); diff --git a/tests/Linter.test.cpp b/tests/Linter.test.cpp index 05ee9a7b..6649cb7f 100644 --- a/tests/Linter.test.cpp +++ b/tests/Linter.test.cpp @@ -1436,7 +1436,7 @@ TEST_CASE_FIXTURE(Fixture, "LintHygieneUAF") TEST_CASE_FIXTURE(Fixture, "DeprecatedApi") { unfreeze(typeChecker.globalTypes); - TypeId instanceType = typeChecker.globalTypes.addType(ClassTypeVar{"Instance", {}, std::nullopt, std::nullopt, {}, {}}); + TypeId instanceType = typeChecker.globalTypes.addType(ClassTypeVar{"Instance", {}, std::nullopt, std::nullopt, {}, {}, "Test"}); persist(instanceType); typeChecker.globalScope->exportedTypeBindings["Instance"] = TypeFun{{}, instanceType}; diff --git a/tests/Module.test.cpp b/tests/Module.test.cpp index 738893db..af7d76de 100644 --- a/tests/Module.test.cpp +++ b/tests/Module.test.cpp @@ -173,13 +173,13 @@ TEST_CASE_FIXTURE(Fixture, "clone_class") { {"__add", {typeChecker.anyType}}, }, - std::nullopt, std::nullopt, {}, {}}}; + std::nullopt, std::nullopt, {}, {}, "Test"}}; TypeVar exampleClass{ClassTypeVar{"ExampleClass", { {"PropOne", {typeChecker.numberType}}, {"PropTwo", {typeChecker.stringType}}, }, - std::nullopt, &exampleMetaClass, {}, {}}}; + std::nullopt, &exampleMetaClass, {}, {}, "Test"}}; TypeArena dest; CloneState cloneState; @@ -196,9 +196,12 @@ TEST_CASE_FIXTURE(Fixture, "clone_class") CHECK_EQ("ExampleClassMeta", metatable->name); } -TEST_CASE_FIXTURE(Fixture, "clone_sanitize_free_types") +TEST_CASE_FIXTURE(Fixture, "clone_free_types") { - ScopedFastFlag sff{"LuauErrorRecoveryType", true}; + ScopedFastFlag sff[]{ + {"LuauErrorRecoveryType", true}, + {"LuauLosslessClone", true}, + }; TypeVar freeTy(FreeTypeVar{TypeLevel{}}); TypePackVar freeTp(FreeTypePack{TypeLevel{}}); @@ -207,17 +210,17 @@ TEST_CASE_FIXTURE(Fixture, "clone_sanitize_free_types") CloneState cloneState; TypeId clonedTy = clone(&freeTy, dest, cloneState); - CHECK_EQ("any", toString(clonedTy)); - CHECK(cloneState.encounteredFreeType); + CHECK(get(clonedTy)); cloneState = {}; TypePackId clonedTp = clone(&freeTp, dest, cloneState); - CHECK_EQ("...any", toString(clonedTp)); - CHECK(cloneState.encounteredFreeType); + CHECK(get(clonedTp)); } -TEST_CASE_FIXTURE(Fixture, "clone_seal_free_tables") +TEST_CASE_FIXTURE(Fixture, "clone_free_tables") { + ScopedFastFlag sff{"LuauLosslessClone", true}; + TypeVar tableTy{TableTypeVar{}}; TableTypeVar* ttv = getMutable(&tableTy); ttv->state = TableState::Free; @@ -227,8 +230,7 @@ TEST_CASE_FIXTURE(Fixture, "clone_seal_free_tables") TypeId cloned = clone(&tableTy, dest, cloneState); const TableTypeVar* clonedTtv = get(cloned); - CHECK_EQ(clonedTtv->state, TableState::Sealed); - CHECK(cloneState.encounteredFreeType); + CHECK_EQ(clonedTtv->state, TableState::Free); } TEST_CASE_FIXTURE(Fixture, "clone_constrained_intersection") diff --git a/tests/NonstrictMode.test.cpp b/tests/NonstrictMode.test.cpp index a8a12b69..9748eb27 100644 --- a/tests/NonstrictMode.test.cpp +++ b/tests/NonstrictMode.test.cpp @@ -13,6 +13,29 @@ using namespace Luau; TEST_SUITE_BEGIN("NonstrictModeTests"); +TEST_CASE_FIXTURE(Fixture, "function_returns_number_or_string") +{ + ScopedFastFlag sff[]{ + {"LuauReturnTypeInferenceInNonstrict", true}, + {"LuauLowerBoundsCalculation", true} + }; + + CheckResult result = check(R"( + --!nonstrict + local function f() + if math.random() > 0.5 then + return 5 + else + return "hi" + end + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK("() -> number | string" == toString(requireType("f"))); +} + TEST_CASE_FIXTURE(Fixture, "infer_nullary_function") { CheckResult result = check(R"( @@ -35,8 +58,13 @@ TEST_CASE_FIXTURE(Fixture, "infer_nullary_function") REQUIRE_EQ(0, rets.size()); } -TEST_CASE_FIXTURE(Fixture, "infer_the_maximum_number_of_values_the_function_could_return") +TEST_CASE_FIXTURE(Fixture, "first_return_type_dictates_number_of_return_types") { + ScopedFastFlag sff[]{ + {"LuauReturnTypeInferenceInNonstrict", true}, + {"LuauLowerBoundsCalculation", true}, + }; + CheckResult result = check(R"( --!nonstrict function getMinCardCountForWidth(width) @@ -51,22 +79,18 @@ TEST_CASE_FIXTURE(Fixture, "infer_the_maximum_number_of_values_the_function_coul TypeId t = requireType("getMinCardCountForWidth"); REQUIRE(t); - REQUIRE_EQ("(any) -> (...any)", toString(t)); + REQUIRE_EQ("(any) -> number", toString(t)); } -#if 0 -// Maybe we want this? TEST_CASE_FIXTURE(Fixture, "return_annotation_is_still_checked") { CheckResult result = check(R"( + --!nonstrict function foo(x): number return 'hello' end )"); LUAU_REQUIRE_ERROR_COUNT(1, result); - - REQUIRE_NE(*typeChecker.anyType, *requireType("foo")); } -#endif TEST_CASE_FIXTURE(Fixture, "function_parameters_are_any") { @@ -256,6 +280,12 @@ TEST_CASE_FIXTURE(Fixture, "delay_function_does_not_require_its_argument_to_retu TEST_CASE_FIXTURE(Fixture, "inconsistent_module_return_types_are_ok") { + ScopedFastFlag sff[]{ + {"LuauReturnTypeInferenceInNonstrict", true}, + {"LuauLowerBoundsCalculation", true}, + {"LuauSealExports", true}, + }; + CheckResult result = check(R"( --!nonstrict @@ -272,7 +302,7 @@ TEST_CASE_FIXTURE(Fixture, "inconsistent_module_return_types_are_ok") LUAU_REQUIRE_NO_ERRORS(result); - REQUIRE_EQ("any", toString(getMainModule()->getModuleScope()->returnType)); + REQUIRE_EQ("((any) -> string) | {| foo: any |}", toString(getMainModule()->getModuleScope()->returnType)); } TEST_CASE_FIXTURE(Fixture, "returning_insufficient_return_values") diff --git a/tests/Normalize.test.cpp b/tests/Normalize.test.cpp index 5a84201a..d3778f67 100644 --- a/tests/Normalize.test.cpp +++ b/tests/Normalize.test.cpp @@ -21,7 +21,7 @@ void createSomeClasses(TypeChecker& typeChecker) unfreeze(arena); - TypeId parentType = arena.addType(ClassTypeVar{"Parent", {}, std::nullopt, std::nullopt, {}, nullptr}); + TypeId parentType = arena.addType(ClassTypeVar{"Parent", {}, std::nullopt, std::nullopt, {}, nullptr, "Test"}); ClassTypeVar* parentClass = getMutable(parentType); parentClass->props["method"] = {makeFunction(arena, parentType, {}, {})}; @@ -31,7 +31,7 @@ void createSomeClasses(TypeChecker& typeChecker) addGlobalBinding(typeChecker, "Parent", {parentType}); typeChecker.globalScope->exportedTypeBindings["Parent"] = TypeFun{{}, parentType}; - TypeId childType = arena.addType(ClassTypeVar{"Child", {}, parentType, std::nullopt, {}, nullptr}); + TypeId childType = arena.addType(ClassTypeVar{"Child", {}, parentType, std::nullopt, {}, nullptr, "Test"}); ClassTypeVar* childClass = getMutable(childType); childClass->props["virtual_method"] = {makeFunction(arena, childType, {}, {})}; @@ -39,7 +39,7 @@ void createSomeClasses(TypeChecker& typeChecker) addGlobalBinding(typeChecker, "Child", {childType}); typeChecker.globalScope->exportedTypeBindings["Child"] = TypeFun{{}, childType}; - TypeId unrelatedType = arena.addType(ClassTypeVar{"Unrelated", {}, std::nullopt, std::nullopt, {}, nullptr}); + TypeId unrelatedType = arena.addType(ClassTypeVar{"Unrelated", {}, std::nullopt, std::nullopt, {}, nullptr, "Test"}); addGlobalBinding(typeChecker, "Unrelated", {unrelatedType}); typeChecker.globalScope->exportedTypeBindings["Unrelated"] = TypeFun{{}, unrelatedType}; @@ -400,7 +400,6 @@ TEST_CASE_FIXTURE(NormalizeFixture, "table_with_table_prop") CHECK_EQ("{| x: {| y: number & string |} |}", toString(requireType("a"))); } -#if 0 TEST_CASE_FIXTURE(NormalizeFixture, "tables") { check(R"( @@ -428,6 +427,7 @@ TEST_CASE_FIXTURE(NormalizeFixture, "tables") CHECK(!isSubtype(b, d)); } +#if 0 TEST_CASE_FIXTURE(NormalizeFixture, "table_indexers_are_invariant") { check(R"( @@ -619,6 +619,7 @@ TEST_CASE_FIXTURE(Fixture, "normalize_module_return_type") { ScopedFastFlag sff[] = { {"LuauLowerBoundsCalculation", true}, + {"LuauReturnTypeInferenceInNonstrict", true}, }; check(R"( @@ -639,7 +640,7 @@ TEST_CASE_FIXTURE(Fixture, "normalize_module_return_type") end )"); - CHECK_EQ("(any, any) -> (...any)", toString(getMainModule()->getModuleScope()->returnType)); + CHECK_EQ("(any, any) -> (any, any) -> any", toString(getMainModule()->getModuleScope()->returnType)); } TEST_CASE_FIXTURE(Fixture, "return_type_is_not_a_constrained_intersection") @@ -950,6 +951,27 @@ TEST_CASE_FIXTURE(Fixture, "nested_table_normalization_with_non_table__no_ice") LUAU_REQUIRE_NO_ERRORS(result); } +TEST_CASE_FIXTURE(Fixture, "visiting_a_type_twice_is_not_considered_normal") +{ + ScopedFastFlag sff{"LuauLowerBoundsCalculation", true}; + + CheckResult result = check(R"( + --!strict + function f(a, b) + local function g() + if math.random() > 0.5 then + return a() + else + return b + end + end + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + CHECK_EQ("(() -> a, a) -> ()", toString(requireType("f"))); +} + TEST_CASE_FIXTURE(Fixture, "fuzz_failure_instersection_combine_must_follow") { ScopedFastFlag flags[] = { @@ -964,4 +986,16 @@ TEST_CASE_FIXTURE(Fixture, "fuzz_failure_instersection_combine_must_follow") LUAU_REQUIRE_NO_ERRORS(result); } +TEST_CASE_FIXTURE(Fixture, "fuzz_failure_bound_type_is_normal_but_not_its_bounded_to") +{ + ScopedFastFlag sff{"LuauLowerBoundsCalculation", true}; + + CheckResult result = check(R"( + type t252 = ((t0)|(any))|(any) + type t0 = t252,t24...> + )"); + + LUAU_REQUIRE_ERRORS(result); +} + TEST_SUITE_END(); diff --git a/tests/ToDot.test.cpp b/tests/ToDot.test.cpp index f3fda54e..332a4b22 100644 --- a/tests/ToDot.test.cpp +++ b/tests/ToDot.test.cpp @@ -21,13 +21,13 @@ struct ToDotClassFixture : Fixture TypeId baseClassMetaType = arena.addType(TableTypeVar{}); - TypeId baseClassInstanceType = arena.addType(ClassTypeVar{"BaseClass", {}, std::nullopt, baseClassMetaType, {}, {}}); + TypeId baseClassInstanceType = arena.addType(ClassTypeVar{"BaseClass", {}, std::nullopt, baseClassMetaType, {}, {}, "Test"}); getMutable(baseClassInstanceType)->props = { {"BaseField", {typeChecker.numberType}}, }; typeChecker.globalScope->exportedTypeBindings["BaseClass"] = TypeFun{{}, baseClassInstanceType}; - TypeId childClassInstanceType = arena.addType(ClassTypeVar{"ChildClass", {}, baseClassInstanceType, std::nullopt, {}, {}}); + TypeId childClassInstanceType = arena.addType(ClassTypeVar{"ChildClass", {}, baseClassInstanceType, std::nullopt, {}, {}, "Test"}); getMutable(childClassInstanceType)->props = { {"ChildField", {typeChecker.stringType}}, }; diff --git a/tests/Transpiler.test.cpp b/tests/Transpiler.test.cpp index 0c324cd0..b02a52b2 100644 --- a/tests/Transpiler.test.cpp +++ b/tests/Transpiler.test.cpp @@ -661,4 +661,21 @@ type t4 = false CHECK_EQ(code, transpile(code, {}, true).code); } +TEST_CASE_FIXTURE(Fixture, "transpile_array_types") +{ + std::string code = R"( +type t1 = {number} +type t2 = {[string]: number} + )"; + + CHECK_EQ(code, transpile(code, {}, true).code); +} + +TEST_CASE_FIXTURE(Fixture, "transpile_for_in_multiple_types") +{ + std::string code = "for k:string,v:boolean in next,{}do end"; + + CHECK_EQ(code, transpile(code, {}, true).code); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.classes.test.cpp b/tests/TypeInfer.classes.test.cpp index 8e3629e7..5a6e4032 100644 --- a/tests/TypeInfer.classes.test.cpp +++ b/tests/TypeInfer.classes.test.cpp @@ -19,13 +19,13 @@ struct ClassFixture : Fixture unfreeze(arena); - TypeId baseClassInstanceType = arena.addType(ClassTypeVar{"BaseClass", {}, nullopt, nullopt, {}, {}}); + TypeId baseClassInstanceType = arena.addType(ClassTypeVar{"BaseClass", {}, nullopt, nullopt, {}, {}, "Test"}); getMutable(baseClassInstanceType)->props = { {"BaseMethod", {makeFunction(arena, baseClassInstanceType, {numberType}, {})}}, {"BaseField", {numberType}}, }; - TypeId baseClassType = arena.addType(ClassTypeVar{"BaseClass", {}, nullopt, nullopt, {}, {}}); + TypeId baseClassType = arena.addType(ClassTypeVar{"BaseClass", {}, nullopt, nullopt, {}, {}, "Test"}); getMutable(baseClassType)->props = { {"StaticMethod", {makeFunction(arena, nullopt, {}, {numberType})}}, {"Clone", {makeFunction(arena, nullopt, {baseClassInstanceType}, {baseClassInstanceType})}}, @@ -34,39 +34,39 @@ struct ClassFixture : Fixture typeChecker.globalScope->exportedTypeBindings["BaseClass"] = TypeFun{{}, baseClassInstanceType}; addGlobalBinding(typeChecker, "BaseClass", baseClassType, "@test"); - TypeId childClassInstanceType = arena.addType(ClassTypeVar{"ChildClass", {}, baseClassInstanceType, nullopt, {}, {}}); + TypeId childClassInstanceType = arena.addType(ClassTypeVar{"ChildClass", {}, baseClassInstanceType, nullopt, {}, {}, "Test"}); getMutable(childClassInstanceType)->props = { {"Method", {makeFunction(arena, childClassInstanceType, {}, {typeChecker.stringType})}}, }; - TypeId childClassType = arena.addType(ClassTypeVar{"ChildClass", {}, baseClassType, nullopt, {}, {}}); + TypeId childClassType = arena.addType(ClassTypeVar{"ChildClass", {}, baseClassType, nullopt, {}, {}, "Test"}); getMutable(childClassType)->props = { {"New", {makeFunction(arena, nullopt, {}, {childClassInstanceType})}}, }; typeChecker.globalScope->exportedTypeBindings["ChildClass"] = TypeFun{{}, childClassInstanceType}; addGlobalBinding(typeChecker, "ChildClass", childClassType, "@test"); - TypeId grandChildInstanceType = arena.addType(ClassTypeVar{"GrandChild", {}, childClassInstanceType, nullopt, {}, {}}); + TypeId grandChildInstanceType = arena.addType(ClassTypeVar{"GrandChild", {}, childClassInstanceType, nullopt, {}, {}, "Test"}); getMutable(grandChildInstanceType)->props = { {"Method", {makeFunction(arena, grandChildInstanceType, {}, {typeChecker.stringType})}}, }; - TypeId grandChildType = arena.addType(ClassTypeVar{"GrandChild", {}, baseClassType, nullopt, {}, {}}); + TypeId grandChildType = arena.addType(ClassTypeVar{"GrandChild", {}, baseClassType, nullopt, {}, {}, "Test"}); getMutable(grandChildType)->props = { {"New", {makeFunction(arena, nullopt, {}, {grandChildInstanceType})}}, }; typeChecker.globalScope->exportedTypeBindings["GrandChild"] = TypeFun{{}, grandChildInstanceType}; addGlobalBinding(typeChecker, "GrandChild", childClassType, "@test"); - TypeId anotherChildInstanceType = arena.addType(ClassTypeVar{"AnotherChild", {}, baseClassInstanceType, nullopt, {}, {}}); + TypeId anotherChildInstanceType = arena.addType(ClassTypeVar{"AnotherChild", {}, baseClassInstanceType, nullopt, {}, {}, "Test"}); getMutable(anotherChildInstanceType)->props = { {"Method", {makeFunction(arena, anotherChildInstanceType, {}, {typeChecker.stringType})}}, }; - TypeId anotherChildType = arena.addType(ClassTypeVar{"AnotherChild", {}, baseClassType, nullopt, {}, {}}); + TypeId anotherChildType = arena.addType(ClassTypeVar{"AnotherChild", {}, baseClassType, nullopt, {}, {}, "Test"}); getMutable(anotherChildType)->props = { {"New", {makeFunction(arena, nullopt, {}, {anotherChildInstanceType})}}, }; @@ -75,13 +75,13 @@ struct ClassFixture : Fixture TypeId vector2MetaType = arena.addType(TableTypeVar{}); - TypeId vector2InstanceType = arena.addType(ClassTypeVar{"Vector2", {}, nullopt, vector2MetaType, {}, {}}); + TypeId vector2InstanceType = arena.addType(ClassTypeVar{"Vector2", {}, nullopt, vector2MetaType, {}, {}, "Test"}); getMutable(vector2InstanceType)->props = { {"X", {numberType}}, {"Y", {numberType}}, }; - TypeId vector2Type = arena.addType(ClassTypeVar{"Vector2", {}, nullopt, nullopt, {}, {}}); + TypeId vector2Type = arena.addType(ClassTypeVar{"Vector2", {}, nullopt, nullopt, {}, {}, "Test"}); getMutable(vector2Type)->props = { {"New", {makeFunction(arena, nullopt, {numberType, numberType}, {vector2InstanceType})}}, }; @@ -468,4 +468,18 @@ caused by: toString(result.errors[0])); } +TEST_CASE_FIXTURE(ClassFixture, "class_type_mismatch_with_name_conflict") +{ + ScopedFastFlag luauClassDefinitionModuleInError{"LuauClassDefinitionModuleInError", true}; + + CheckResult result = check(R"( +local i = ChildClass.New() +type ChildClass = { x: number } +local a: ChildClass = i + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ("Type 'ChildClass' from 'Test' could not be converted into 'ChildClass' from 'MainModule'", toString(result.errors[0])); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.definitions.test.cpp b/tests/TypeInfer.definitions.test.cpp index 898d8902..4545b8db 100644 --- a/tests/TypeInfer.definitions.test.cpp +++ b/tests/TypeInfer.definitions.test.cpp @@ -295,8 +295,6 @@ TEST_CASE_FIXTURE(Fixture, "documentation_symbols_dont_attach_to_persistent_type TEST_CASE_FIXTURE(Fixture, "single_class_type_identity_in_global_types") { - ScopedFastFlag luauCloneDeclaredGlobals{"LuauCloneDeclaredGlobals", true}; - loadDefinition(R"( declare class Cls end diff --git a/tests/TypeInfer.functions.test.cpp b/tests/TypeInfer.functions.test.cpp index 65993681..7cd7bec3 100644 --- a/tests/TypeInfer.functions.test.cpp +++ b/tests/TypeInfer.functions.test.cpp @@ -656,6 +656,11 @@ TEST_CASE_FIXTURE(Fixture, "toposort_doesnt_break_mutual_recursion") TEST_CASE_FIXTURE(Fixture, "check_function_before_lambda_that_uses_it") { + ScopedFastFlag sff[]{ + {"LuauReturnTypeInferenceInNonstrict", true}, + {"LuauLowerBoundsCalculation", true}, + }; + CheckResult result = check(R"( --!nonstrict @@ -664,7 +669,7 @@ TEST_CASE_FIXTURE(Fixture, "check_function_before_lambda_that_uses_it") end return function() - return f():andThen() + return f() end )"); @@ -791,14 +796,18 @@ TEST_CASE_FIXTURE(Fixture, "calling_function_with_incorrect_argument_type_yields TEST_CASE_FIXTURE(Fixture, "calling_function_with_anytypepack_doesnt_leak_free_types") { + ScopedFastFlag sff[]{ + {"LuauReturnTypeInferenceInNonstrict", true}, + {"LuauLowerBoundsCalculation", true}, + }; + CheckResult result = check(R"( --!nonstrict - function Test(a) + function Test(a): ...any return 1, "" end - local tab = {} table.insert(tab, Test(1)); )"); @@ -1616,4 +1625,19 @@ TEST_CASE_FIXTURE(Fixture, "occurs_check_failure_in_function_return_type") CHECK(nullptr != get(result.errors[0])); } +TEST_CASE_FIXTURE(Fixture, "weird_fail_to_unify_type_pack") +{ + ScopedFastFlag sff[]{ + {"LuauReturnTypeInferenceInNonstrict", true}, + {"LuauLowerBoundsCalculation", true}, + }; + + CheckResult result = check(R"( + local function f() return end + local g = function() return f() end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.modules.test.cpp b/tests/TypeInfer.modules.test.cpp index e5eeae31..fa1f519c 100644 --- a/tests/TypeInfer.modules.test.cpp +++ b/tests/TypeInfer.modules.test.cpp @@ -307,8 +307,6 @@ type Rename = typeof(x.x) TEST_CASE_FIXTURE(Fixture, "module_type_conflict") { - ScopedFastFlag luauTypeMismatchModuleName{"LuauTypeMismatchModuleName", true}; - fileResolver.source["game/A"] = R"( export type T = { x: number } return {} @@ -343,8 +341,6 @@ caused by: TEST_CASE_FIXTURE(Fixture, "module_type_conflict_instantiated") { - ScopedFastFlag luauTypeMismatchModuleName{"LuauTypeMismatchModuleName", true}; - fileResolver.source["game/A"] = R"( export type Wrap = { x: T } return {} diff --git a/tests/TypeInfer.provisional.test.cpp b/tests/TypeInfer.provisional.test.cpp index 8c8059db..4b5075d9 100644 --- a/tests/TypeInfer.provisional.test.cpp +++ b/tests/TypeInfer.provisional.test.cpp @@ -584,20 +584,6 @@ TEST_CASE_FIXTURE(Fixture, "specialization_binds_with_prototypes_too_early") LUAU_REQUIRE_ERRORS(result); // Should not have any errors. } -TEST_CASE_FIXTURE(Fixture, "weird_fail_to_unify_type_pack") -{ - ScopedFastFlag sff[] = { - {"LuauLowerBoundsCalculation", false}, - }; - - CheckResult result = check(R"( - local function f() return end - local g = function() return f() end - )"); - - LUAU_REQUIRE_ERRORS(result); // Should not have any errors. -} - TEST_CASE_FIXTURE(Fixture, "weird_fail_to_unify_variadic_pack") { ScopedFastFlag sff[] = { @@ -636,6 +622,10 @@ TEST_CASE_FIXTURE(Fixture, "lower_bounds_calculation_is_too_permissive_with_over // Once fixed, move this to Normalize.test.cpp TEST_CASE_FIXTURE(Fixture, "normalization_fails_on_certain_kinds_of_cyclic_tables") { +#if defined(_DEBUG) || defined(_NOOPT) + ScopedFastInt sfi("LuauNormalizeIterationLimit", 500); +#endif + ScopedFastFlag flags[] = { {"LuauLowerBoundsCalculation", true}, }; diff --git a/tests/TypeInfer.refinements.test.cpp b/tests/TypeInfer.refinements.test.cpp index ce22bcb1..136ca00a 100644 --- a/tests/TypeInfer.refinements.test.cpp +++ b/tests/TypeInfer.refinements.test.cpp @@ -44,7 +44,7 @@ struct RefinementClassFixture : Fixture TypeArena& arena = typeChecker.globalTypes; unfreeze(arena); - TypeId vec3 = arena.addType(ClassTypeVar{"Vector3", {}, std::nullopt, std::nullopt, {}, nullptr}); + TypeId vec3 = arena.addType(ClassTypeVar{"Vector3", {}, std::nullopt, std::nullopt, {}, nullptr, "Test"}); getMutable(vec3)->props = { {"X", Property{typeChecker.numberType}}, {"Y", Property{typeChecker.numberType}}, @@ -52,7 +52,7 @@ struct RefinementClassFixture : Fixture }; normalize(vec3, arena, *typeChecker.iceHandler); - TypeId inst = arena.addType(ClassTypeVar{"Instance", {}, std::nullopt, std::nullopt, {}, nullptr}); + TypeId inst = arena.addType(ClassTypeVar{"Instance", {}, std::nullopt, std::nullopt, {}, nullptr, "Test"}); TypePackId isAParams = arena.addTypePack({inst, typeChecker.stringType}); TypePackId isARets = arena.addTypePack({typeChecker.booleanType}); @@ -66,9 +66,9 @@ struct RefinementClassFixture : Fixture }; normalize(inst, arena, *typeChecker.iceHandler); - TypeId folder = typeChecker.globalTypes.addType(ClassTypeVar{"Folder", {}, inst, std::nullopt, {}, nullptr}); + TypeId folder = typeChecker.globalTypes.addType(ClassTypeVar{"Folder", {}, inst, std::nullopt, {}, nullptr, "Test"}); normalize(folder, arena, *typeChecker.iceHandler); - TypeId part = typeChecker.globalTypes.addType(ClassTypeVar{"Part", {}, inst, std::nullopt, {}, nullptr}); + TypeId part = typeChecker.globalTypes.addType(ClassTypeVar{"Part", {}, inst, std::nullopt, {}, nullptr, "Test"}); getMutable(part)->props = { {"Position", Property{vec3}}, }; diff --git a/tests/TypeInfer.tables.test.cpp b/tests/TypeInfer.tables.test.cpp index ca1b8de7..2a727bb3 100644 --- a/tests/TypeInfer.tables.test.cpp +++ b/tests/TypeInfer.tables.test.cpp @@ -2086,7 +2086,6 @@ caused by: TEST_CASE_FIXTURE(Fixture, "error_detailed_indexer_key") { ScopedFastFlag luauTableSubtypingVariance2{"LuauTableSubtypingVariance2", true}; // Only for new path - ScopedFastFlag luauExtendedIndexerError{"LuauExtendedIndexerError", true}; CheckResult result = check(R"( type A = { [number]: string } @@ -2105,7 +2104,6 @@ caused by: TEST_CASE_FIXTURE(Fixture, "error_detailed_indexer_value") { ScopedFastFlag luauTableSubtypingVariance2{"LuauTableSubtypingVariance2", true}; // Only for new path - ScopedFastFlag luauExtendedIndexerError{"LuauExtendedIndexerError", true}; CheckResult result = check(R"( type A = { [number]: number } diff --git a/tests/TypeInfer.test.cpp b/tests/TypeInfer.test.cpp index 6abd96b9..a578b1cf 100644 --- a/tests/TypeInfer.test.cpp +++ b/tests/TypeInfer.test.cpp @@ -86,16 +86,21 @@ TEST_CASE_FIXTURE(Fixture, "infer_locals_via_assignment_from_its_call_site") TEST_CASE_FIXTURE(Fixture, "infer_in_nocheck_mode") { + ScopedFastFlag sff[]{ + {"LuauReturnTypeInferenceInNonstrict", true}, + {"LuauLowerBoundsCalculation", true}, + }; + CheckResult result = check(R"( --!nocheck function f(x) - return x + return 5 end -- we get type information even if there's type errors f(1, 2) )"); - CHECK_EQ("(any) -> (...any)", toString(requireType("f"))); + CHECK_EQ("(any) -> number", toString(requireType("f"))); LUAU_REQUIRE_NO_ERRORS(result); } @@ -363,6 +368,11 @@ TEST_CASE_FIXTURE(Fixture, "globals") TEST_CASE_FIXTURE(Fixture, "globals2") { + ScopedFastFlag sff[]{ + {"LuauReturnTypeInferenceInNonstrict", true}, + {"LuauLowerBoundsCalculation", true}, + }; + CheckResult result = check(R"( --!nonstrict foo = function() return 1 end @@ -373,9 +383,9 @@ TEST_CASE_FIXTURE(Fixture, "globals2") TypeMismatch* tm = get(result.errors[0]); REQUIRE(tm); - CHECK_EQ("() -> (...any)", toString(tm->wantedType)); + CHECK_EQ("() -> number", toString(tm->wantedType)); CHECK_EQ("string", toString(tm->givenType)); - CHECK_EQ("() -> (...any)", toString(requireType("foo"))); + CHECK_EQ("() -> number", toString(requireType("foo"))); } TEST_CASE_FIXTURE(Fixture, "globals_are_banned_in_strict_mode") diff --git a/tests/TypeVar.test.cpp b/tests/TypeVar.test.cpp index fd5f4dbc..d03bb03c 100644 --- a/tests/TypeVar.test.cpp +++ b/tests/TypeVar.test.cpp @@ -275,7 +275,7 @@ TEST_CASE("tagging_tables") TEST_CASE("tagging_classes") { - TypeVar base{ClassTypeVar{"Base", {}, std::nullopt, std::nullopt, {}, nullptr}}; + TypeVar base{ClassTypeVar{"Base", {}, std::nullopt, std::nullopt, {}, nullptr, "Test"}}; CHECK(!Luau::hasTag(&base, "foo")); Luau::attachTag(&base, "foo"); CHECK(Luau::hasTag(&base, "foo")); @@ -283,8 +283,8 @@ TEST_CASE("tagging_classes") TEST_CASE("tagging_subclasses") { - TypeVar base{ClassTypeVar{"Base", {}, std::nullopt, std::nullopt, {}, nullptr}}; - TypeVar derived{ClassTypeVar{"Derived", {}, &base, std::nullopt, {}, nullptr}}; + TypeVar base{ClassTypeVar{"Base", {}, std::nullopt, std::nullopt, {}, nullptr, "Test"}}; + TypeVar derived{ClassTypeVar{"Derived", {}, &base, std::nullopt, {}, nullptr, "Test"}}; CHECK(!Luau::hasTag(&base, "foo")); CHECK(!Luau::hasTag(&derived, "foo")); From e0a6461173216b76bdf731410f50485b89c83aa4 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Thu, 21 Apr 2022 14:44:27 -0700 Subject: [PATCH 227/399] Sync to upstream/release/524 (#462) --- Analysis/include/Luau/Clone.h | 2 +- Analysis/include/Luau/Frontend.h | 21 +- Analysis/include/Luau/TypeVar.h | 6 +- Analysis/include/Luau/Unifier.h | 1 - Analysis/include/Luau/VisitTypeVar.h | 2 +- Analysis/src/Autocomplete.cpp | 4 +- Analysis/src/Clone.cpp | 42 ++- Analysis/src/Error.cpp | 23 +- Analysis/src/Frontend.cpp | 50 ++-- Analysis/src/Module.cpp | 18 +- Analysis/src/Normalize.cpp | 44 +-- Analysis/src/Substitution.cpp | 15 +- Analysis/src/ToDot.cpp | 2 +- Analysis/src/Transpiler.cpp | 53 ++-- Analysis/src/TypeAttach.cpp | 14 + Analysis/src/TypeInfer.cpp | 37 ++- Analysis/src/TypeVar.cpp | 6 + Analysis/src/Unifier.cpp | 110 ++------ Ast/include/Luau/StringUtils.h | 1 + Ast/src/Parser.cpp | 8 + Ast/src/StringUtils.cpp | 2 +- CLI/Repl.cpp | 72 ++++- CMakeLists.txt | 6 + Compiler/include/Luau/BytecodeBuilder.h | 7 + Compiler/src/BytecodeBuilder.cpp | 70 ++++- Compiler/src/Compiler.cpp | 127 ++++++++- Compiler/src/ConstantFolding.cpp | 13 +- Compiler/src/ConstantFolding.h | 3 +- VM/src/lapi.cpp | 24 +- VM/src/lgc.cpp | 26 +- VM/src/lgc.h | 2 +- VM/src/lstate.h | 7 +- bench/bench.py | 13 +- fuzz/proto.cpp | 13 +- tests/Autocomplete.test.cpp | 36 +++ tests/Compiler.test.cpp | 359 +++++++++++++++++++++++- tests/CostModel.test.cpp | 125 +++++++++ tests/JsonEncoder.test.cpp | 332 +++++++++++++++++++++- tests/Linter.test.cpp | 2 +- tests/Module.test.cpp | 24 +- tests/NonstrictMode.test.cpp | 46 ++- tests/Normalize.test.cpp | 44 ++- tests/ToDot.test.cpp | 4 +- tests/Transpiler.test.cpp | 17 ++ tests/TypeInfer.classes.test.cpp | 34 ++- tests/TypeInfer.definitions.test.cpp | 2 - tests/TypeInfer.functions.test.cpp | 30 +- tests/TypeInfer.modules.test.cpp | 4 - tests/TypeInfer.provisional.test.cpp | 14 - tests/TypeInfer.refinements.test.cpp | 8 +- tests/TypeInfer.tables.test.cpp | 2 - tests/TypeInfer.test.cpp | 18 +- tests/TypeVar.test.cpp | 6 +- 53 files changed, 1596 insertions(+), 355 deletions(-) diff --git a/Analysis/include/Luau/Clone.h b/Analysis/include/Luau/Clone.h index 78aa92c7..9b6ffa62 100644 --- a/Analysis/include/Luau/Clone.h +++ b/Analysis/include/Luau/Clone.h @@ -18,7 +18,7 @@ struct CloneState SeenTypePacks seenTypePacks; int recursionCount = 0; - bool encounteredFreeType = false; + bool encounteredFreeType = false; // TODO: Remove with LuauLosslessClone. }; TypePackId clone(TypePackId tp, TypeArena& dest, CloneState& cloneState); diff --git a/Analysis/include/Luau/Frontend.h b/Analysis/include/Luau/Frontend.h index e24e433c..59125470 100644 --- a/Analysis/include/Luau/Frontend.h +++ b/Analysis/include/Luau/Frontend.h @@ -13,6 +13,7 @@ #include LUAU_FASTFLAG(LuauSeparateTypechecks) +LUAU_FASTFLAG(LuauDirtySourceModule) namespace Luau { @@ -57,19 +58,27 @@ std::optional pathExprToModuleName(const ModuleName& currentModuleNa struct SourceNode { - bool isDirty(bool forAutocomplete) const + bool hasDirtySourceModule() const + { + LUAU_ASSERT(FFlag::LuauDirtySourceModule); + + return dirtySourceModule; + } + + bool hasDirtyModule(bool forAutocomplete) const { if (FFlag::LuauSeparateTypechecks) - return forAutocomplete ? dirtyAutocomplete : dirty; + return forAutocomplete ? dirtyModuleForAutocomplete : dirtyModule; else - return dirty; + return dirtyModule; } ModuleName name; std::unordered_set requires; std::vector> requireLocations; - bool dirty = true; - bool dirtyAutocomplete = true; + bool dirtySourceModule = true; + bool dirtyModule = true; + bool dirtyModuleForAutocomplete = true; double autocompleteLimitsMult = 1.0; }; @@ -163,7 +172,7 @@ struct Frontend void applyBuiltinDefinitionToEnvironment(const std::string& environmentName, const std::string& definitionName); private: - std::pair getSourceNode(CheckResult& checkResult, const ModuleName& name, bool forAutocomplete); + std::pair getSourceNode(CheckResult& checkResult, const ModuleName& name, bool forAutocomplete_DEPRECATED); SourceModule parse(const ModuleName& name, std::string_view src, const ParseOptions& parseOptions); bool parseGraph(std::vector& buildQueue, CheckResult& checkResult, const ModuleName& root, bool forAutocomplete); diff --git a/Analysis/include/Luau/TypeVar.h b/Analysis/include/Luau/TypeVar.h index ae7d1377..84576758 100644 --- a/Analysis/include/Luau/TypeVar.h +++ b/Analysis/include/Luau/TypeVar.h @@ -373,15 +373,17 @@ struct ClassTypeVar std::optional metatable; // metaclass? Tags tags; std::shared_ptr userData; + ModuleName definitionModuleName; - ClassTypeVar( - Name name, Props props, std::optional parent, std::optional metatable, Tags tags, std::shared_ptr userData) + ClassTypeVar(Name name, Props props, std::optional parent, std::optional metatable, Tags tags, + std::shared_ptr userData, ModuleName definitionModuleName) : name(name) , props(props) , parent(parent) , metatable(metatable) , tags(tags) , userData(userData) + , definitionModuleName(definitionModuleName) { } }; diff --git a/Analysis/include/Luau/Unifier.h b/Analysis/include/Luau/Unifier.h index 340feb7f..418d4ca4 100644 --- a/Analysis/include/Luau/Unifier.h +++ b/Analysis/include/Luau/Unifier.h @@ -92,7 +92,6 @@ private: bool canCacheResult(TypeId subTy, TypeId superTy); void cacheResult(TypeId subTy, TypeId superTy, size_t prevErrorCount); - void cacheResult_DEPRECATED(TypeId subTy, TypeId superTy); public: void tryUnify(TypePackId subTy, TypePackId superTy, bool isFunctionCall = false); diff --git a/Analysis/include/Luau/VisitTypeVar.h b/Analysis/include/Luau/VisitTypeVar.h index d11cbd0d..045190ea 100644 --- a/Analysis/include/Luau/VisitTypeVar.h +++ b/Analysis/include/Luau/VisitTypeVar.h @@ -52,7 +52,7 @@ inline void unsee(std::unordered_set& seen, const void* tv) inline void unsee(DenseHashSet& seen, const void* tv) { - // When DenseHashSet is used for 'visitOnce', where don't forget visited elements + // When DenseHashSet is used for 'visitTypeVarOnce', where don't forget visited elements } template diff --git a/Analysis/src/Autocomplete.cpp b/Analysis/src/Autocomplete.cpp index e0e79cb4..dec12d01 100644 --- a/Analysis/src/Autocomplete.cpp +++ b/Analysis/src/Autocomplete.cpp @@ -15,6 +15,7 @@ LUAU_FASTFLAGVARIABLE(LuauIfElseExprFixCompletionIssue, false); LUAU_FASTFLAGVARIABLE(LuauAutocompleteSingletonTypes, false); +LUAU_FASTFLAGVARIABLE(LuauFixAutocompleteClassSecurityLevel, false); LUAU_FASTFLAG(LuauSelfCallAutocompleteFix) static const std::unordered_set kStatementStartingKeywords = { @@ -462,7 +463,8 @@ static void autocompleteProps(const Module& module, TypeArena* typeArena, TypeId containingClass = containingClass.value_or(cls); fillProps(cls->props); if (cls->parent) - autocompleteProps(module, typeArena, rootTy, *cls->parent, indexType, nodes, result, seen, cls); + autocompleteProps(module, typeArena, rootTy, *cls->parent, indexType, nodes, result, seen, + FFlag::LuauFixAutocompleteClassSecurityLevel ? containingClass : cls); } else if (auto tbl = get(ty)) fillProps(tbl->props); diff --git a/Analysis/src/Clone.cpp b/Analysis/src/Clone.cpp index 8e7f7c07..d5bd9dab 100644 --- a/Analysis/src/Clone.cpp +++ b/Analysis/src/Clone.cpp @@ -10,6 +10,7 @@ LUAU_FASTFLAG(DebugLuauCopyBeforeNormalizing) LUAU_FASTINTVARIABLE(LuauTypeCloneRecursionLimit, 300) LUAU_FASTFLAG(LuauTypecheckOptPass) +LUAU_FASTFLAGVARIABLE(LuauLosslessClone, false) namespace Luau { @@ -87,11 +88,18 @@ struct TypePackCloner void operator()(const Unifiable::Free& t) { - cloneState.encounteredFreeType = true; + if (FFlag::LuauLosslessClone) + { + defaultClone(t); + } + else + { + cloneState.encounteredFreeType = true; - TypePackId err = getSingletonTypes().errorRecoveryTypePack(getSingletonTypes().anyTypePack); - TypePackId cloned = dest.addTypePack(*err); - seenTypePacks[typePackId] = cloned; + TypePackId err = getSingletonTypes().errorRecoveryTypePack(getSingletonTypes().anyTypePack); + TypePackId cloned = dest.addTypePack(*err); + seenTypePacks[typePackId] = cloned; + } } void operator()(const Unifiable::Generic& t) @@ -143,10 +151,18 @@ void TypeCloner::defaultClone(const T& t) void TypeCloner::operator()(const Unifiable::Free& t) { - cloneState.encounteredFreeType = true; - TypeId err = getSingletonTypes().errorRecoveryType(getSingletonTypes().anyType); - TypeId cloned = dest.addType(*err); - seenTypes[typeId] = cloned; + if (FFlag::LuauLosslessClone) + { + defaultClone(t); + } + else + { + cloneState.encounteredFreeType = true; + + TypeId err = getSingletonTypes().errorRecoveryType(getSingletonTypes().anyType); + TypeId cloned = dest.addType(*err); + seenTypes[typeId] = cloned; + } } void TypeCloner::operator()(const Unifiable::Generic& t) @@ -174,7 +190,8 @@ void TypeCloner::operator()(const PrimitiveTypeVar& t) void TypeCloner::operator()(const ConstrainedTypeVar& t) { - cloneState.encounteredFreeType = true; + if (!FFlag::LuauLosslessClone) + cloneState.encounteredFreeType = true; TypeId res = dest.addType(ConstrainedTypeVar{t.level}); ConstrainedTypeVar* ctv = getMutable(res); @@ -252,7 +269,7 @@ void TypeCloner::operator()(const TableTypeVar& t) for (TypePackId& arg : ttv->instantiatedTypePackParams) arg = clone(arg, dest, cloneState); - if (ttv->state == TableState::Free) + if (!FFlag::LuauLosslessClone && ttv->state == TableState::Free) { cloneState.encounteredFreeType = true; @@ -276,7 +293,7 @@ void TypeCloner::operator()(const MetatableTypeVar& t) void TypeCloner::operator()(const ClassTypeVar& t) { - TypeId result = dest.addType(ClassTypeVar{t.name, {}, std::nullopt, std::nullopt, t.tags, t.userData}); + TypeId result = dest.addType(ClassTypeVar{t.name, {}, std::nullopt, std::nullopt, t.tags, t.userData, t.definitionModuleName}); ClassTypeVar* ctv = getMutable(result); seenTypes[typeId] = result; @@ -361,7 +378,10 @@ TypeId clone(TypeId typeId, TypeArena& dest, CloneState& cloneState) // Persistent types are not being cloned and we get the original type back which might be read-only if (!res->persistent) + { asMutable(res)->documentationSymbol = typeId->documentationSymbol; + asMutable(res)->normal = typeId->normal; + } } return res; diff --git a/Analysis/src/Error.cpp b/Analysis/src/Error.cpp index cbec0b15..24ed4ac1 100644 --- a/Analysis/src/Error.cpp +++ b/Analysis/src/Error.cpp @@ -8,8 +8,6 @@ #include -LUAU_FASTFLAGVARIABLE(LuauTypeMismatchModuleName, false); - static std::string wrongNumberOfArgsString(size_t expectedCount, size_t actualCount, const char* argPrefix = nullptr, bool isVariadic = false) { std::string s = "expects "; @@ -59,27 +57,20 @@ struct ErrorConverter std::string result; - if (FFlag::LuauTypeMismatchModuleName) + if (givenTypeName == wantedTypeName) { - if (givenTypeName == wantedTypeName) + if (auto givenDefinitionModule = getDefinitionModuleName(tm.givenType)) { - if (auto givenDefinitionModule = getDefinitionModuleName(tm.givenType)) + if (auto wantedDefinitionModule = getDefinitionModuleName(tm.wantedType)) { - if (auto wantedDefinitionModule = getDefinitionModuleName(tm.wantedType)) - { - result = "Type '" + givenTypeName + "' from '" + *givenDefinitionModule + "' could not be converted into '" + wantedTypeName + - "' from '" + *wantedDefinitionModule + "'"; - } + result = "Type '" + givenTypeName + "' from '" + *givenDefinitionModule + "' could not be converted into '" + wantedTypeName + + "' from '" + *wantedDefinitionModule + "'"; } } + } - if (result.empty()) - result = "Type '" + givenTypeName + "' could not be converted into '" + wantedTypeName + "'"; - } - else - { + if (result.empty()) result = "Type '" + givenTypeName + "' could not be converted into '" + wantedTypeName + "'"; - } if (tm.error) { diff --git a/Analysis/src/Frontend.cpp b/Analysis/src/Frontend.cpp index 8b0b2210..34ccdac4 100644 --- a/Analysis/src/Frontend.cpp +++ b/Analysis/src/Frontend.cpp @@ -23,6 +23,7 @@ LUAU_FASTFLAG(LuauInferInNoCheckMode) LUAU_FASTFLAGVARIABLE(LuauKnowsTheDataModel3, false) LUAU_FASTFLAGVARIABLE(LuauSeparateTypechecks, false) LUAU_FASTFLAGVARIABLE(LuauAutocompleteDynamicLimits, false) +LUAU_FASTFLAGVARIABLE(LuauDirtySourceModule, false) LUAU_FASTINTVARIABLE(LuauAutocompleteCheckTimeoutMs, 0) namespace Luau @@ -358,7 +359,7 @@ CheckResult Frontend::check(const ModuleName& name, std::optionalsecond.isDirty(frontendOptions.forAutocomplete)) + if (it != sourceNodes.end() && !it->second.hasDirtyModule(frontendOptions.forAutocomplete)) { // No recheck required. if (FFlag::LuauSeparateTypechecks) @@ -402,7 +403,7 @@ CheckResult Frontend::check(const ModuleName& name, std::optionalerrors.begin(), module->errors.end()); moduleResolver.modules[moduleName] = std::move(module); - sourceNode.dirty = false; + sourceNode.dirtyModule = false; } return checkResult; @@ -618,7 +619,7 @@ bool Frontend::parseGraph(std::vector& buildQueue, CheckResult& chec // this relies on the fact that markDirty marks reverse-dependencies dirty as well // thus if a node is not dirty, all its transitive deps aren't dirty, which means that they won't ever need // to be built, *and* can't form a cycle with any nodes we did process. - if (!it->second.isDirty(forAutocomplete)) + if (!it->second.hasDirtyModule(forAutocomplete)) continue; // note: this check is technically redundant *except* that getSourceNode has somewhat broken memoization @@ -768,7 +769,7 @@ LintResult Frontend::lint(const SourceModule& module, std::optionalsecond.isDirty(forAutocomplete); + return it == sourceNodes.end() || it->second.hasDirtyModule(forAutocomplete); } /* @@ -810,20 +811,31 @@ void Frontend::markDirty(const ModuleName& name, std::vector* marked if (markedDirty) markedDirty->push_back(next); - if (FFlag::LuauSeparateTypechecks) + if (FFlag::LuauDirtySourceModule) { - if (sourceNode.dirty && sourceNode.dirtyAutocomplete) + LUAU_ASSERT(FFlag::LuauSeparateTypechecks); + + if (sourceNode.dirtySourceModule && sourceNode.dirtyModule && sourceNode.dirtyModuleForAutocomplete) continue; - sourceNode.dirty = true; - sourceNode.dirtyAutocomplete = true; + sourceNode.dirtySourceModule = true; + sourceNode.dirtyModule = true; + sourceNode.dirtyModuleForAutocomplete = true; + } + else if (FFlag::LuauSeparateTypechecks) + { + if (sourceNode.dirtyModule && sourceNode.dirtyModuleForAutocomplete) + continue; + + sourceNode.dirtyModule = true; + sourceNode.dirtyModuleForAutocomplete = true; } else { - if (sourceNode.dirty) + if (sourceNode.dirtyModule) continue; - sourceNode.dirty = true; + sourceNode.dirtyModule = true; } if (0 == reverseDeps.count(name)) @@ -851,13 +863,14 @@ const SourceModule* Frontend::getSourceModule(const ModuleName& moduleName) cons } // Read AST into sourceModules if necessary. Trace require()s. Report parse errors. -std::pair Frontend::getSourceNode(CheckResult& checkResult, const ModuleName& name, bool forAutocomplete) +std::pair Frontend::getSourceNode(CheckResult& checkResult, const ModuleName& name, bool forAutocomplete_DEPRECATED) { LUAU_TIMETRACE_SCOPE("Frontend::getSourceNode", "Frontend"); LUAU_TIMETRACE_ARGUMENT("name", name.c_str()); auto it = sourceNodes.find(name); - if (it != sourceNodes.end() && !it->second.isDirty(forAutocomplete)) + if (it != sourceNodes.end() && + (FFlag::LuauDirtySourceModule ? !it->second.hasDirtySourceModule() : !it->second.hasDirtyModule(forAutocomplete_DEPRECATED))) { auto moduleIt = sourceModules.find(name); if (moduleIt != sourceModules.end()) @@ -901,17 +914,20 @@ std::pair Frontend::getSourceNode(CheckResult& check sourceNode.requires.clear(); sourceNode.requireLocations.clear(); + if (FFlag::LuauDirtySourceModule) + sourceNode.dirtySourceModule = false; + if (FFlag::LuauSeparateTypechecks) { if (it == sourceNodes.end()) { - sourceNode.dirty = true; - sourceNode.dirtyAutocomplete = true; + sourceNode.dirtyModule = true; + sourceNode.dirtyModuleForAutocomplete = true; } } else { - sourceNode.dirty = true; + sourceNode.dirtyModule = true; } for (const auto& [moduleName, location] : requireTrace.requires) diff --git a/Analysis/src/Module.cpp b/Analysis/src/Module.cpp index e2e3b436..bafd4371 100644 --- a/Analysis/src/Module.cpp +++ b/Analysis/src/Module.cpp @@ -14,8 +14,8 @@ #include LUAU_FASTFLAGVARIABLE(DebugLuauFreezeArena, false) -LUAU_FASTFLAGVARIABLE(LuauCloneDeclaredGlobals, false) LUAU_FASTFLAG(LuauLowerBoundsCalculation) +LUAU_FASTFLAG(LuauLosslessClone) namespace Luau { @@ -182,20 +182,20 @@ bool Module::clonePublicInterface(InternalErrorReporter& ice) } } - if (FFlag::LuauCloneDeclaredGlobals) + for (auto& [name, ty] : declaredGlobals) { - for (auto& [name, ty] : declaredGlobals) - { - ty = clone(ty, interfaceTypes, cloneState); - if (FFlag::LuauLowerBoundsCalculation) - normalize(ty, interfaceTypes, ice); - } + ty = clone(ty, interfaceTypes, cloneState); + if (FFlag::LuauLowerBoundsCalculation) + normalize(ty, interfaceTypes, ice); } freeze(internalTypes); freeze(interfaceTypes); - return cloneState.encounteredFreeType; + if (FFlag::LuauLosslessClone) + return false; // TODO: make function return void. + else + return cloneState.encounteredFreeType; } } // namespace Luau diff --git a/Analysis/src/Normalize.cpp b/Analysis/src/Normalize.cpp index 40341ac1..043526ed 100644 --- a/Analysis/src/Normalize.cpp +++ b/Analysis/src/Normalize.cpp @@ -5,7 +5,6 @@ #include #include "Luau/Clone.h" -#include "Luau/DenseHash.h" #include "Luau/Substitution.h" #include "Luau/Unifier.h" #include "Luau/VisitTypeVar.h" @@ -254,7 +253,7 @@ bool isSubtype(TypeId subTy, TypeId superTy, InternalErrorReporter& ice) } template -static bool areNormal_(const T& t, const DenseHashSet& seen, InternalErrorReporter& ice) +static bool areNormal_(const T& t, const std::unordered_set& seen, InternalErrorReporter& ice) { int count = 0; auto isNormal = [&](TypeId ty) { @@ -262,18 +261,19 @@ static bool areNormal_(const T& t, const DenseHashSet& seen, InternalErro if (count >= FInt::LuauNormalizeIterationLimit) ice.ice("Luau::areNormal hit iteration limit"); - return ty->normal || seen.find(asMutable(ty)); + // The follow is here because a bound type may not be normal, but the bound type is normal. + return ty->normal || follow(ty)->normal || seen.find(asMutable(ty)) != seen.end(); }; return std::all_of(begin(t), end(t), isNormal); } -static bool areNormal(const std::vector& types, const DenseHashSet& seen, InternalErrorReporter& ice) +static bool areNormal(const std::vector& types, const std::unordered_set& seen, InternalErrorReporter& ice) { return areNormal_(types, seen, ice); } -static bool areNormal(TypePackId tp, const DenseHashSet& seen, InternalErrorReporter& ice) +static bool areNormal(TypePackId tp, const std::unordered_set& seen, InternalErrorReporter& ice) { tp = follow(tp); if (get(tp)) @@ -288,7 +288,7 @@ static bool areNormal(TypePackId tp, const DenseHashSet& seen, InternalEr return true; if (auto vtp = get(*tail)) - return vtp->ty->normal || seen.find(asMutable(vtp->ty)); + return vtp->ty->normal || follow(vtp->ty)->normal || seen.find(asMutable(vtp->ty)) != seen.end(); return true; } @@ -335,9 +335,14 @@ struct Normalize return false; } - bool operator()(TypeId ty, const BoundTypeVar& btv) + bool operator()(TypeId ty, const BoundTypeVar& btv, std::unordered_set& seen) { - // It should never be the case that this TypeVar is normal, but is bound to a non-normal type. + // A type could be considered normal when it is in the stack, but we will eventually find out it is not normal as normalization progresses. + // So we need to avoid eagerly saying that this bound type is normal if the thing it is bound to is in the stack. + if (seen.find(asMutable(btv.boundTo)) != seen.end()) + return false; + + // It should never be the case that this TypeVar is normal, but is bound to a non-normal type, except in nontrivial cases. LUAU_ASSERT(!ty->normal || ty->normal == btv.boundTo->normal); asMutable(ty)->normal = btv.boundTo->normal; @@ -365,7 +370,7 @@ struct Normalize return false; } - bool operator()(TypeId ty, const ConstrainedTypeVar& ctvRef, DenseHashSet& seen) + bool operator()(TypeId ty, const ConstrainedTypeVar& ctvRef, std::unordered_set& seen) { CHECK_ITERATION_LIMIT(false); @@ -391,8 +396,7 @@ struct Normalize return false; } - bool operator()(TypeId ty, const FunctionTypeVar& ftv) = delete; - bool operator()(TypeId ty, const FunctionTypeVar& ftv, DenseHashSet& seen) + bool operator()(TypeId ty, const FunctionTypeVar& ftv, std::unordered_set& seen) { CHECK_ITERATION_LIMIT(false); @@ -407,7 +411,7 @@ struct Normalize return false; } - bool operator()(TypeId ty, const TableTypeVar& ttv, DenseHashSet& seen) + bool operator()(TypeId ty, const TableTypeVar& ttv, std::unordered_set& seen) { CHECK_ITERATION_LIMIT(false); @@ -419,7 +423,7 @@ struct Normalize auto checkNormal = [&](TypeId t) { // if t is on the stack, it is possible that this type is normal. // If t is not normal and it is not on the stack, this type is definitely not normal. - if (!t->normal && !seen.find(asMutable(t))) + if (!t->normal && seen.find(asMutable(t)) == seen.end()) normal = false; }; @@ -449,7 +453,7 @@ struct Normalize return false; } - bool operator()(TypeId ty, const MetatableTypeVar& mtv, DenseHashSet& seen) + bool operator()(TypeId ty, const MetatableTypeVar& mtv, std::unordered_set& seen) { CHECK_ITERATION_LIMIT(false); @@ -477,7 +481,7 @@ struct Normalize return false; } - bool operator()(TypeId ty, const UnionTypeVar& utvRef, DenseHashSet& seen) + bool operator()(TypeId ty, const UnionTypeVar& utvRef, std::unordered_set& seen) { CHECK_ITERATION_LIMIT(false); @@ -507,7 +511,7 @@ struct Normalize return false; } - bool operator()(TypeId ty, const IntersectionTypeVar& itvRef, DenseHashSet& seen) + bool operator()(TypeId ty, const IntersectionTypeVar& itvRef, std::unordered_set& seen) { CHECK_ITERATION_LIMIT(false); @@ -775,8 +779,8 @@ std::pair normalize(TypeId ty, TypeArena& arena, InternalErrorRepo (void)clone(ty, arena, state); Normalize n{arena, ice, std::move(state.seenTypes), std::move(state.seenTypePacks)}; - DenseHashSet seen{nullptr}; - visitTypeVarOnce(ty, n, seen); + std::unordered_set seen; + visitTypeVar(ty, n, seen); return {ty, !n.limitExceeded}; } @@ -800,8 +804,8 @@ std::pair normalize(TypePackId tp, TypeArena& arena, InternalE (void)clone(tp, arena, state); Normalize n{arena, ice, std::move(state.seenTypes), std::move(state.seenTypePacks)}; - DenseHashSet seen{nullptr}; - visitTypeVarOnce(tp, n, seen); + std::unordered_set seen; + visitTypeVar(tp, n, seen); return {tp, !n.limitExceeded}; } diff --git a/Analysis/src/Substitution.cpp b/Analysis/src/Substitution.cpp index 8648b21e..1b51fa3d 100644 --- a/Analysis/src/Substitution.cpp +++ b/Analysis/src/Substitution.cpp @@ -11,6 +11,7 @@ LUAU_FASTFLAG(LuauLowerBoundsCalculation) LUAU_FASTINTVARIABLE(LuauTarjanChildLimit, 1000) LUAU_FASTFLAG(LuauTypecheckOptPass) LUAU_FASTFLAGVARIABLE(LuauSubstituteFollowNewTypes, false) +LUAU_FASTFLAGVARIABLE(LuauSubstituteFollowPossibleMutations, false) namespace Luau { @@ -106,7 +107,7 @@ void Tarjan::visitChildren(TypePackId tp, int index) std::pair Tarjan::indexify(TypeId ty) { - if (FFlag::LuauTypecheckOptPass) + if (FFlag::LuauTypecheckOptPass && !FFlag::LuauSubstituteFollowPossibleMutations) LUAU_ASSERT(ty == log->follow(ty)); else ty = log->follow(ty); @@ -127,7 +128,7 @@ std::pair Tarjan::indexify(TypeId ty) std::pair Tarjan::indexify(TypePackId tp) { - if (FFlag::LuauTypecheckOptPass) + if (FFlag::LuauTypecheckOptPass && !FFlag::LuauSubstituteFollowPossibleMutations) LUAU_ASSERT(tp == log->follow(tp)); else tp = log->follow(tp); @@ -148,7 +149,8 @@ std::pair Tarjan::indexify(TypePackId tp) void Tarjan::visitChild(TypeId ty) { - ty = log->follow(ty); + if (!FFlag::LuauSubstituteFollowPossibleMutations) + ty = log->follow(ty); edgesTy.push_back(ty); edgesTp.push_back(nullptr); @@ -156,7 +158,8 @@ void Tarjan::visitChild(TypeId ty) void Tarjan::visitChild(TypePackId tp) { - tp = log->follow(tp); + if (!FFlag::LuauSubstituteFollowPossibleMutations) + tp = log->follow(tp); edgesTy.push_back(nullptr); edgesTp.push_back(tp); @@ -471,7 +474,7 @@ TypePackId Substitution::clone(TypePackId tp) void Substitution::foundDirty(TypeId ty) { - if (FFlag::LuauTypecheckOptPass) + if (FFlag::LuauTypecheckOptPass && !FFlag::LuauSubstituteFollowPossibleMutations) LUAU_ASSERT(ty == log->follow(ty)); else ty = log->follow(ty); @@ -484,7 +487,7 @@ void Substitution::foundDirty(TypeId ty) void Substitution::foundDirty(TypePackId tp) { - if (FFlag::LuauTypecheckOptPass) + if (FFlag::LuauTypecheckOptPass && !FFlag::LuauSubstituteFollowPossibleMutations) LUAU_ASSERT(tp == log->follow(tp)); else tp = log->follow(tp); diff --git a/Analysis/src/ToDot.cpp b/Analysis/src/ToDot.cpp index cb54bfc1..9b396c80 100644 --- a/Analysis/src/ToDot.cpp +++ b/Analysis/src/ToDot.cpp @@ -327,7 +327,7 @@ void StateDot::visitChildren(TypePackId tp, int index) } else if (const VariadicTypePack* vtp = get(tp)) { - formatAppend(result, "VariadicTypePack %d", index); + formatAppend(result, "VariadicTypePack %s%d", vtp->hidden ? "hidden " : "", index); finishNodeLabel(tp); finishNode(); diff --git a/Analysis/src/Transpiler.cpp b/Analysis/src/Transpiler.cpp index 92ed241e..1577bd63 100644 --- a/Analysis/src/Transpiler.cpp +++ b/Analysis/src/Transpiler.cpp @@ -1025,31 +1025,42 @@ struct Printer } else if (const auto& a = typeAnnotation.as()) { - CommaSeparatorInserter comma(writer); + AstTypeReference* indexType = a->indexer ? a->indexer->indexType->as() : nullptr; - writer.symbol("{"); - - for (std::size_t i = 0; i < a->props.size; ++i) + if (a->props.size == 0 && indexType && indexType->name == "number") { - comma(); - advance(a->props.data[i].location.begin); - writer.identifier(a->props.data[i].name.value); - if (a->props.data[i].type) - { - writer.symbol(":"); - visualizeTypeAnnotation(*a->props.data[i].type); - } - } - if (a->indexer) - { - comma(); - writer.symbol("["); - visualizeTypeAnnotation(*a->indexer->indexType); - writer.symbol("]"); - writer.symbol(":"); + writer.symbol("{"); visualizeTypeAnnotation(*a->indexer->resultType); + writer.symbol("}"); + } + else + { + CommaSeparatorInserter comma(writer); + + writer.symbol("{"); + + for (std::size_t i = 0; i < a->props.size; ++i) + { + comma(); + advance(a->props.data[i].location.begin); + writer.identifier(a->props.data[i].name.value); + if (a->props.data[i].type) + { + writer.symbol(":"); + visualizeTypeAnnotation(*a->props.data[i].type); + } + } + if (a->indexer) + { + comma(); + writer.symbol("["); + visualizeTypeAnnotation(*a->indexer->indexType); + writer.symbol("]"); + writer.symbol(":"); + visualizeTypeAnnotation(*a->indexer->resultType); + } + writer.symbol("}"); } - writer.symbol("}"); } else if (auto a = typeAnnotation.as()) { diff --git a/Analysis/src/TypeAttach.cpp b/Analysis/src/TypeAttach.cpp index bc8d0d4e..0f4534b7 100644 --- a/Analysis/src/TypeAttach.cpp +++ b/Analysis/src/TypeAttach.cpp @@ -479,6 +479,20 @@ public: { return visitLocal(al->local); } + + virtual bool visit(AstStatFor* stat) override + { + visitLocal(stat->var); + return true; + } + + virtual bool visit(AstStatForIn* stat) override + { + for (size_t i = 0; i < stat->vars.size; ++i) + visitLocal(stat->vars.data[i]); + return true; + } + virtual bool visit(AstExprFunction* fn) override { // TODO: add generics if the inferred type of the function is generic CLI-39908 diff --git a/Analysis/src/TypeInfer.cpp b/Analysis/src/TypeInfer.cpp index af42a4e6..6411e2ab 100644 --- a/Analysis/src/TypeInfer.cpp +++ b/Analysis/src/TypeInfer.cpp @@ -1,6 +1,7 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #include "Luau/TypeInfer.h" +#include "Luau/Clone.h" #include "Luau/Common.h" #include "Luau/ModuleResolver.h" #include "Luau/Normalize.h" @@ -47,7 +48,6 @@ LUAU_FASTFLAGVARIABLE(LuauPropertiesGetExpectedType, false) LUAU_FASTFLAGVARIABLE(LuauStatFunctionSimplify4, false) LUAU_FASTFLAGVARIABLE(LuauTypecheckOptPass, false) LUAU_FASTFLAGVARIABLE(LuauUnsealedTableLiteral, false) -LUAU_FASTFLAG(LuauTypeMismatchModuleName) LUAU_FASTFLAGVARIABLE(LuauTwoPassAliasDefinitionFix, false) LUAU_FASTFLAGVARIABLE(LuauAssertStripsFalsyTypes, false) LUAU_FASTFLAGVARIABLE(LuauReturnAnyInsteadOfICE, false) // Eventually removed as false. @@ -61,7 +61,9 @@ LUAU_FASTFLAG(LuauAnyInIsOptionalIsOptional) LUAU_FASTFLAGVARIABLE(LuauDecoupleOperatorInferenceFromUnifiedTypeInference, false) LUAU_FASTFLAGVARIABLE(LuauArgCountMismatchSaysAtLeastWhenVariadic, false) LUAU_FASTFLAGVARIABLE(LuauTableUseCounterInstead, false) +LUAU_FASTFLAGVARIABLE(LuauReturnTypeInferenceInNonstrict, false) LUAU_FASTFLAGVARIABLE(LuauRecursionLimitException, false); +LUAU_FASTFLAG(LuauLosslessClone) namespace Luau { @@ -376,7 +378,7 @@ ModulePtr TypeChecker::checkWithoutRecursionCheck(const SourceModule& module, Mo prepareErrorsForDisplay(currentModule->errors); bool encounteredFreeType = currentModule->clonePublicInterface(*iceHandler); - if (encounteredFreeType) + if (!FFlag::LuauLosslessClone && encounteredFreeType) { reportError(TypeError{module.root->location, GenericError{"Free types leaked into this module's public interface. This is an internal Luau error; please report it."}}); @@ -785,7 +787,7 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatReturn& return_) TypePackId retPack = checkExprList(scope, return_.location, return_.list, false, {}, expectedTypes).type; - if (useConstrainedIntersections()) + if (FFlag::LuauReturnTypeInferenceInNonstrict ? FFlag::LuauLowerBoundsCalculation : useConstrainedIntersections()) { unifyLowerBound(retPack, scope->returnType, return_.location); return; @@ -1241,7 +1243,12 @@ void TypeChecker::check(const ScopePtr& scope, TypeId ty, const ScopePtr& funSco // If in nonstrict mode and allowing redefinition of global function, restore the previous definition type // in case this function has a differing signature. The signature discrepancy will be caught in checkBlock. if (previouslyDefined) + { + if (FFlag::LuauReturnTypeInferenceInNonstrict && FFlag::LuauLowerBoundsCalculation) + quantify(funScope, ty, exprName->location); + globalBindings[name] = oldBinding; + } else globalBindings[name] = {quantify(funScope, ty, exprName->location), exprName->location}; @@ -1555,7 +1562,7 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatDeclareClass& declar Name className(declaredClass.name.value); - TypeId classTy = addType(ClassTypeVar(className, {}, superTy, std::nullopt, {}, {})); + TypeId classTy = addType(ClassTypeVar(className, {}, superTy, std::nullopt, {}, {}, currentModuleName)); ClassTypeVar* ctv = getMutable(classTy); TypeId metaTy = addType(TableTypeVar{TableState::Sealed, scope->level}); @@ -3284,7 +3291,7 @@ std::pair TypeChecker::checkFunctionSignature( TypePackId retPack; if (expr.returnAnnotation) retPack = resolveTypePack(funScope, *expr.returnAnnotation); - else if (isNonstrictMode()) + else if (FFlag::LuauReturnTypeInferenceInNonstrict ? (!FFlag::LuauLowerBoundsCalculation && isNonstrictMode()) : isNonstrictMode()) retPack = anyTypePack; else if (expectedFunctionType) { @@ -5328,19 +5335,9 @@ TypeId TypeChecker::resolveType(const ScopePtr& scope, const AstType& annotation if (const auto& indexer = table->indexer) tableIndexer = TableIndexer(resolveType(scope, *indexer->indexType), resolveType(scope, *indexer->resultType)); - if (FFlag::LuauTypeMismatchModuleName) - { - TableTypeVar ttv{props, tableIndexer, scope->level, TableState::Sealed}; - ttv.definitionModuleName = currentModuleName; - return addType(std::move(ttv)); - } - else - { - return addType(TableTypeVar{ - props, tableIndexer, scope->level, - TableState::Sealed // FIXME: probably want a way to annotate other kinds of tables maybe - }); - } + TableTypeVar ttv{props, tableIndexer, scope->level, TableState::Sealed}; + ttv.definitionModuleName = currentModuleName; + return addType(std::move(ttv)); } else if (const auto& func = annotation.as()) { @@ -5602,9 +5599,7 @@ TypeId TypeChecker::instantiateTypeFun(const ScopePtr& scope, const TypeFun& tf, { ttv->instantiatedTypeParams = typeParams; ttv->instantiatedTypePackParams = typePackParams; - - if (FFlag::LuauTypeMismatchModuleName) - ttv->definitionModuleName = currentModuleName; + ttv->definitionModuleName = currentModuleName; } return instantiated; diff --git a/Analysis/src/TypeVar.cpp b/Analysis/src/TypeVar.cpp index 0fbfdbf0..4d42573c 100644 --- a/Analysis/src/TypeVar.cpp +++ b/Analysis/src/TypeVar.cpp @@ -27,6 +27,7 @@ LUAU_FASTFLAG(LuauErrorRecoveryType) LUAU_FASTFLAG(LuauSubtypingAddOptPropsToUnsealedTables) LUAU_FASTFLAG(LuauDiscriminableUnions2) LUAU_FASTFLAGVARIABLE(LuauAnyInIsOptionalIsOptional, false) +LUAU_FASTFLAGVARIABLE(LuauClassDefinitionModuleInError, false) namespace Luau { @@ -304,6 +305,11 @@ std::optional getDefinitionModuleName(TypeId type) if (ftv->definition) return ftv->definition->definitionModuleName; } + else if (auto ctv = get(type); ctv && FFlag::LuauClassDefinitionModuleInError) + { + if (!ctv->definitionModuleName.empty()) + return ctv->definitionModuleName; + } return std::nullopt; } diff --git a/Analysis/src/Unifier.cpp b/Analysis/src/Unifier.cpp index f9ea58cc..9862d7b3 100644 --- a/Analysis/src/Unifier.cpp +++ b/Analysis/src/Unifier.cpp @@ -17,7 +17,6 @@ LUAU_FASTINT(LuauTypeInferTypePackLoopLimit); LUAU_FASTINT(LuauTypeInferIterationLimit); LUAU_FASTFLAG(LuauAutocompleteDynamicLimits) LUAU_FASTINTVARIABLE(LuauTypeInferLowerBoundsIterationLimit, 2000); -LUAU_FASTFLAGVARIABLE(LuauExtendedIndexerError, false); LUAU_FASTFLAGVARIABLE(LuauTableSubtypingVariance2, false); LUAU_FASTFLAG(LuauLowerBoundsCalculation); LUAU_FASTFLAG(LuauErrorRecoveryType); @@ -28,7 +27,6 @@ LUAU_FASTFLAGVARIABLE(LuauTxnLogSeesTypePacks2, false) LUAU_FASTFLAGVARIABLE(LuauTxnLogCheckForInvalidation, false) LUAU_FASTFLAGVARIABLE(LuauTxnLogRefreshFunctionPointers, false) LUAU_FASTFLAGVARIABLE(LuauTxnLogDontRetryForIndexers, false) -LUAU_FASTFLAGVARIABLE(LuauUnifierCacheErrors, false) LUAU_FASTFLAG(LuauAnyInIsOptionalIsOptional) LUAU_FASTFLAG(LuauTypecheckOptPass) @@ -474,32 +472,21 @@ void Unifier::tryUnify_(TypeId subTy, TypeId superTy, bool isFunctionCall, bool if (get(subTy)) return tryUnifyWithAny(superTy, subTy); - bool cacheEnabled; auto& cache = sharedState.cachedUnify; // What if the types are immutable and we proved their relation before - if (FFlag::LuauUnifierCacheErrors) + bool cacheEnabled = !isFunctionCall && !isIntersection && variance == Invariant; + + if (cacheEnabled) { - cacheEnabled = !isFunctionCall && !isIntersection && variance == Invariant; - - if (cacheEnabled) - { - if (cache.contains({subTy, superTy})) - return; - - if (auto error = sharedState.cachedUnifyError.find({subTy, superTy})) - { - reportError(TypeError{location, *error}); - return; - } - } - } - else - { - cacheEnabled = !isFunctionCall && !isIntersection; - - if (cacheEnabled && cache.contains({superTy, subTy}) && (variance == Covariant || cache.contains({subTy, superTy}))) + if (cache.contains({subTy, superTy})) return; + + if (auto error = sharedState.cachedUnifyError.find({subTy, superTy})) + { + reportError(TypeError{location, *error}); + return; + } } // If we have seen this pair of types before, we are currently recursing into cyclic types. @@ -543,12 +530,6 @@ void Unifier::tryUnify_(TypeId subTy, TypeId superTy, bool isFunctionCall, bool else if (log.getMutable(superTy) && log.getMutable(subTy)) { tryUnifyTables(subTy, superTy, isIntersection); - - if (!FFlag::LuauUnifierCacheErrors) - { - if (cacheEnabled && errors.empty()) - cacheResult_DEPRECATED(subTy, superTy); - } } // tryUnifyWithMetatable assumes its first argument is a MetatableTypeVar. The check is otherwise symmetrical. @@ -568,7 +549,7 @@ void Unifier::tryUnify_(TypeId subTy, TypeId superTy, bool isFunctionCall, bool else reportError(TypeError{location, TypeMismatch{superTy, subTy}}); - if (FFlag::LuauUnifierCacheErrors && cacheEnabled) + if (cacheEnabled) cacheResult(subTy, superTy, errorCount); log.popSeen(superTy, subTy); @@ -705,21 +686,10 @@ void Unifier::tryUnifyTypeWithUnion(TypeId subTy, TypeId superTy, const UnionTyp { TypeId type = uv->options[i]; - if (FFlag::LuauUnifierCacheErrors) + if (cache.contains({subTy, type})) { - if (cache.contains({subTy, type})) - { - startIndex = i; - break; - } - } - else - { - if (cache.contains({type, subTy}) && (variance == Covariant || cache.contains({subTy, type}))) - { - startIndex = i; - break; - } + startIndex = i; + break; } } } @@ -807,21 +777,10 @@ void Unifier::tryUnifyIntersectionWithType(TypeId subTy, const IntersectionTypeV { TypeId type = uv->parts[i]; - if (FFlag::LuauUnifierCacheErrors) + if (cache.contains({type, superTy})) { - if (cache.contains({type, superTy})) - { - startIndex = i; - break; - } - } - else - { - if (cache.contains({superTy, type}) && (variance == Covariant || cache.contains({type, superTy}))) - { - startIndex = i; - break; - } + startIndex = i; + break; } } } @@ -896,19 +855,6 @@ void Unifier::cacheResult(TypeId subTy, TypeId superTy, size_t prevErrorCount) } } -void Unifier::cacheResult_DEPRECATED(TypeId subTy, TypeId superTy) -{ - LUAU_ASSERT(!FFlag::LuauUnifierCacheErrors); - - if (!canCacheResult(subTy, superTy)) - return; - - sharedState.cachedUnify.insert({superTy, subTy}); - - if (variance == Invariant) - sharedState.cachedUnify.insert({subTy, superTy}); -} - struct WeirdIter { TypePackId packId; @@ -1650,24 +1596,16 @@ void Unifier::tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) Unifier innerState = makeChildUnifier(); - if (FFlag::LuauExtendedIndexerError) - { - innerState.tryUnify_(subTable->indexer->indexType, superTable->indexer->indexType); + innerState.tryUnify_(subTable->indexer->indexType, superTable->indexer->indexType); - bool reported = !innerState.errors.empty(); + bool reported = !innerState.errors.empty(); - checkChildUnifierTypeMismatch(innerState.errors, "[indexer key]", superTy, subTy); + checkChildUnifierTypeMismatch(innerState.errors, "[indexer key]", superTy, subTy); - innerState.tryUnify_(subTable->indexer->indexResultType, superTable->indexer->indexResultType); + innerState.tryUnify_(subTable->indexer->indexResultType, superTable->indexer->indexResultType); - if (!reported) - checkChildUnifierTypeMismatch(innerState.errors, "[indexer value]", superTy, subTy); - } - else - { - innerState.tryUnifyIndexer(*subTable->indexer, *superTable->indexer); - checkChildUnifierTypeMismatch(innerState.errors, superTy, subTy); - } + if (!reported) + checkChildUnifierTypeMismatch(innerState.errors, "[indexer value]", superTy, subTy); if (innerState.errors.empty()) log.concat(std::move(innerState.log)); @@ -2225,7 +2163,7 @@ void Unifier::tryUnifyWithClass(TypeId subTy, TypeId superTy, bool reversed) void Unifier::tryUnifyIndexer(const TableIndexer& subIndexer, const TableIndexer& superIndexer) { - LUAU_ASSERT(!FFlag::LuauTableSubtypingVariance2 || !FFlag::LuauExtendedIndexerError); + LUAU_ASSERT(!FFlag::LuauTableSubtypingVariance2); tryUnify_(subIndexer.indexType, superIndexer.indexType); tryUnify_(subIndexer.indexResultType, superIndexer.indexResultType); diff --git a/Ast/include/Luau/StringUtils.h b/Ast/include/Luau/StringUtils.h index 6ecf0606..6ae9e977 100644 --- a/Ast/include/Luau/StringUtils.h +++ b/Ast/include/Luau/StringUtils.h @@ -19,6 +19,7 @@ std::string format(const char* fmt, ...) LUAU_PRINTF_ATTR(1, 2); std::string vformat(const char* fmt, va_list args); void formatAppend(std::string& str, const char* fmt, ...) LUAU_PRINTF_ATTR(2, 3); +void vformatAppend(std::string& ret, const char* fmt, va_list args); std::string join(const std::vector& segments, std::string_view delimiter); std::string join(const std::vector& segments, std::string_view delimiter); diff --git a/Ast/src/Parser.cpp b/Ast/src/Parser.cpp index badd3fd3..31ff3f77 100644 --- a/Ast/src/Parser.cpp +++ b/Ast/src/Parser.cpp @@ -167,6 +167,7 @@ Parser::Parser(const char* buffer, size_t bufferSize, AstNameTable& names, Alloc Function top; top.vararg = true; + functionStack.reserve(8); functionStack.push_back(top); nameSelf = names.addStatic("self"); @@ -186,6 +187,13 @@ Parser::Parser(const char* buffer, size_t bufferSize, AstNameTable& names, Alloc // all hot comments parsed after the first non-comment lexeme are special in that they don't affect type checking / linting mode hotcommentHeader = false; + + // preallocate some buffers that are very likely to grow anyway; this works around std::vector's inefficient growth policy for small arrays + localStack.reserve(16); + scratchStat.reserve(16); + scratchExpr.reserve(16); + scratchLocal.reserve(16); + scratchBinding.reserve(16); } bool Parser::blockFollow(const Lexeme& l) diff --git a/Ast/src/StringUtils.cpp b/Ast/src/StringUtils.cpp index 9c7fed31..0dc3f3f5 100644 --- a/Ast/src/StringUtils.cpp +++ b/Ast/src/StringUtils.cpp @@ -11,7 +11,7 @@ namespace Luau { -static void vformatAppend(std::string& ret, const char* fmt, va_list args) +void vformatAppend(std::string& ret, const char* fmt, va_list args) { va_list argscopy; va_copy(argscopy, args); diff --git a/CLI/Repl.cpp b/CLI/Repl.cpp index 5fd6d341..345cb7ac 100644 --- a/CLI/Repl.cpp +++ b/CLI/Repl.cpp @@ -579,7 +579,8 @@ static bool compileFile(const char* name, CompileFormat format) if (format == CompileFormat::Text) { - bcb.setDumpFlags(Luau::BytecodeBuilder::Dump_Code | Luau::BytecodeBuilder::Dump_Source | Luau::BytecodeBuilder::Dump_Locals); + bcb.setDumpFlags(Luau::BytecodeBuilder::Dump_Code | Luau::BytecodeBuilder::Dump_Source | Luau::BytecodeBuilder::Dump_Locals | + Luau::BytecodeBuilder::Dump_Remarks); bcb.setDumpSource(*source); } @@ -636,13 +637,60 @@ static int assertionHandler(const char* expr, const char* file, int line, const return 1; } +static void setLuauFlags(bool state) +{ + for (Luau::FValue* flag = Luau::FValue::list; flag; flag = flag->next) + { + if (strncmp(flag->name, "Luau", 4) == 0) + flag->value = state; + } +} + +static void setFlag(std::string_view name, bool state) +{ + for (Luau::FValue* flag = Luau::FValue::list; flag; flag = flag->next) + { + if (name == flag->name) + { + flag->value = state; + return; + } + } + + fprintf(stderr, "Warning: --fflag unrecognized flag '%.*s'.\n\n", int(name.length()), name.data()); +} + +static void applyFlagKeyValue(std::string_view element) +{ + if (size_t separator = element.find('='); separator != std::string_view::npos) + { + std::string_view key = element.substr(0, separator); + std::string_view value = element.substr(separator + 1); + + if (value == "true") + setFlag(key, true); + else if (value == "false") + setFlag(key, false); + else + fprintf(stderr, "Warning: --fflag unrecognized value '%.*s' for flag '%.*s'.\n\n", int(value.length()), value.data(), int(key.length()), + key.data()); + } + else + { + if (element == "true") + setLuauFlags(true); + else if (element == "false") + setLuauFlags(false); + else + setFlag(element, true); + } +} + int replMain(int argc, char** argv) { Luau::assertHandler() = assertionHandler; - for (Luau::FValue* flag = Luau::FValue::list; flag; flag = flag->next) - if (strncmp(flag->name, "Luau", 4) == 0) - flag->value = true; + setLuauFlags(true); CliMode mode = CliMode::Unknown; CompileFormat compileFormat{}; @@ -727,6 +775,22 @@ int replMain(int argc, char** argv) return 1; #endif } + else if (strncmp(argv[i], "--fflags=", 9) == 0) + { + std::string_view list = argv[i] + 9; + + while (!list.empty()) + { + size_t ending = list.find(","); + + applyFlagKeyValue(list.substr(0, ending)); + + if (ending != std::string_view::npos) + list.remove_prefix(ending + 1); + else + break; + } + } else if (argv[i][0] == '-') { fprintf(stderr, "Error: Unrecognized option '%s'.\n\n", argv[i]); diff --git a/CMakeLists.txt b/CMakeLists.txt index c6ccebc5..af03b33a 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -73,6 +73,12 @@ else() list(APPEND LUAU_OPTIONS -Wall) # All warnings endif() +if (CMAKE_CXX_COMPILER_ID STREQUAL "GNU") + # Some gcc versions treat var in `if (type var = val)` as unused + # Some gcc versions treat variables used in constexpr if blocks as unused + list(APPEND LUAU_OPTIONS -Wno-unused) +endif() + # Enabled in CI; we should be warning free on our main compiler versions but don't guarantee being warning free everywhere if(LUAU_WERROR) if(MSVC) diff --git a/Compiler/include/Luau/BytecodeBuilder.h b/Compiler/include/Luau/BytecodeBuilder.h index 287bf4ee..67b93028 100644 --- a/Compiler/include/Luau/BytecodeBuilder.h +++ b/Compiler/include/Luau/BytecodeBuilder.h @@ -3,6 +3,7 @@ #include "Luau/Bytecode.h" #include "Luau/DenseHash.h" +#include "Luau/StringUtils.h" #include @@ -80,6 +81,8 @@ public: void pushDebugUpval(StringRef name); uint32_t getDebugPC() const; + void addDebugRemark(const char* format, ...) LUAU_PRINTF_ATTR(2, 3); + void finalize(); enum DumpFlags @@ -88,6 +91,7 @@ public: Dump_Lines = 1 << 1, Dump_Source = 1 << 2, Dump_Locals = 1 << 3, + Dump_Remarks = 1 << 4, }; void setDumpFlags(uint32_t flags) @@ -228,6 +232,9 @@ private: DenseHashMap stringTable; + DenseHashMap debugRemarks; + std::string debugRemarkBuffer; + BytecodeEncoder* encoder = nullptr; std::string bytecode; diff --git a/Compiler/src/BytecodeBuilder.cpp b/Compiler/src/BytecodeBuilder.cpp index 6944de0f..6c6f1225 100644 --- a/Compiler/src/BytecodeBuilder.cpp +++ b/Compiler/src/BytecodeBuilder.cpp @@ -181,9 +181,17 @@ BytecodeBuilder::BytecodeBuilder(BytecodeEncoder* encoder) : constantMap({Constant::Type_Nil, ~0ull}) , tableShapeMap(TableShape()) , stringTable({nullptr, 0}) + , debugRemarks(~0u) , encoder(encoder) { LUAU_ASSERT(stringTable.find(StringRef{"", 0}) == nullptr); + + // preallocate some buffers that are very likely to grow anyway; this works around std::vector's inefficient growth policy for small arrays + insns.reserve(32); + lines.reserve(32); + constants.reserve(16); + protos.reserve(16); + functions.reserve(8); } uint32_t BytecodeBuilder::beginFunction(uint8_t numparams, bool isvararg) @@ -219,8 +227,8 @@ void BytecodeBuilder::endFunction(uint8_t maxstacksize, uint8_t numupvalues) validate(); #endif - // very approximate: 4 bytes per instruction for code, 1 byte for debug line, and 1-2 bytes for aux data like constants - func.data.reserve(insns.size() * 7); + // very approximate: 4 bytes per instruction for code, 1 byte for debug line, and 1-2 bytes for aux data like constants plus overhead + func.data.reserve(32 + insns.size() * 7); writeFunction(func.data, currentFunction); @@ -242,6 +250,9 @@ void BytecodeBuilder::endFunction(uint8_t maxstacksize, uint8_t numupvalues) constantMap.clear(); tableShapeMap.clear(); + + debugRemarks.clear(); + debugRemarkBuffer.clear(); } void BytecodeBuilder::setMainFunction(uint32_t fid) @@ -505,9 +516,40 @@ uint32_t BytecodeBuilder::getDebugPC() const return uint32_t(insns.size()); } +void BytecodeBuilder::addDebugRemark(const char* format, ...) +{ + if ((dumpFlags & Dump_Remarks) == 0) + return; + + size_t offset = debugRemarkBuffer.size(); + + va_list args; + va_start(args, format); + vformatAppend(debugRemarkBuffer, format, args); + va_end(args); + + // we null-terminate all remarks to avoid storing remark length + debugRemarkBuffer += '\0'; + + debugRemarks[uint32_t(insns.size())] = uint32_t(offset); +} + void BytecodeBuilder::finalize() { LUAU_ASSERT(bytecode.empty()); + + // preallocate space for bytecode blob + size_t capacity = 16; + + for (auto& p : stringTable) + capacity += p.first.length + 2; + + for (const Function& func : functions) + capacity += func.data.size(); + + bytecode.reserve(capacity); + + // assemble final bytecode blob bytecode = char(LBC_VERSION); writeStringTable(bytecode); @@ -663,6 +705,8 @@ void BytecodeBuilder::writeFunction(std::string& ss, uint32_t id) const void BytecodeBuilder::writeLineInfo(std::string& ss) const { + LUAU_ASSERT(!lines.empty()); + // this function encodes lines inside each span as a 8-bit delta to span baseline // span is always a power of two; depending on the line info input, it may need to be as low as 1 int span = 1 << 24; @@ -693,7 +737,17 @@ void BytecodeBuilder::writeLineInfo(std::string& ss) const } // second pass: compute span base - std::vector baseline((lines.size() - 1) / span + 1); + int baselineOne = 0; + std::vector baselineScratch; + int* baseline = &baselineOne; + size_t baselineSize = (lines.size() - 1) / span + 1; + + if (baselineSize > 1) + { + // avoid heap allocation for single-element baseline which is most functions (<256 lines) + baselineScratch.resize(baselineSize); + baseline = baselineScratch.data(); + } for (size_t offset = 0; offset < lines.size(); offset += span) { @@ -725,7 +779,7 @@ void BytecodeBuilder::writeLineInfo(std::string& ss) const int lastLine = 0; - for (size_t i = 0; i < baseline.size(); ++i) + for (size_t i = 0; i < baselineSize; ++i) { writeInt(ss, baseline[i] - lastLine); lastLine = baseline[i]; @@ -1695,6 +1749,14 @@ std::string BytecodeBuilder::dumpCurrentFunction() const continue; } + if (dumpFlags & Dump_Remarks) + { + const uint32_t* remark = debugRemarks.find(uint32_t(code - insns.data())); + + if (remark) + formatAppend(result, "REMARK %s\n", debugRemarkBuffer.c_str() + *remark); + } + if (dumpFlags & Dump_Source) { int line = lines[code - insns.data()]; diff --git a/Compiler/src/Compiler.cpp b/Compiler/src/Compiler.cpp index 8ef69e75..810caaee 100644 --- a/Compiler/src/Compiler.cpp +++ b/Compiler/src/Compiler.cpp @@ -8,12 +8,17 @@ #include "Builtins.h" #include "ConstantFolding.h" +#include "CostModel.h" #include "TableShape.h" #include "ValueTracking.h" #include #include #include +#include + +LUAU_FASTINTVARIABLE(LuauCompileLoopUnrollThreshold, 25) +LUAU_FASTINTVARIABLE(LuauCompileLoopUnrollThresholdMaxBoost, 300) namespace Luau { @@ -77,8 +82,12 @@ struct Compiler , globals(AstName()) , variables(nullptr) , constants(nullptr) + , locstants(nullptr) , tableShapes(nullptr) { + // preallocate some buffers that are very likely to grow anyway; this works around std::vector's inefficient growth policy for small arrays + localStack.reserve(16); + upvals.reserve(16); } uint8_t getLocal(AstLocal* local) @@ -209,7 +218,9 @@ struct Compiler Function& f = functions[func]; f.id = fid; - f.upvals = std::move(upvals); + f.upvals = upvals; + + upvals.clear(); // note: instead of std::move above, we copy & clear to preserve capacity for future pushes return fid; } @@ -2133,10 +2144,119 @@ struct Compiler pushLocal(stat->vars.data[i], uint8_t(vars + i)); } + int getConstantShort(AstExpr* expr) + { + const Constant* c = constants.find(expr); + + if (c && c->type == Constant::Type_Number) + { + double n = c->valueNumber; + + if (n >= -32767 && n <= 32767 && double(int(n)) == n) + return int(n); + } + + return INT_MIN; + } + + bool canUnrollForBody(AstStatFor* stat) + { + struct CanUnrollVisitor : AstVisitor + { + bool result = true; + + bool visit(AstExpr* node) override + { + // functions may capture loop variable, and our upval handling doesn't handle elided variables (constant) + result = result && !node->is(); + return result; + } + + bool visit(AstStat* node) override + { + // while we can easily unroll nested loops, our cost model doesn't take unrolling into account so this can result in code explosion + // we also avoid continue/break since they introduce control flow across iterations + result = result && !node->is() && !node->is() && !node->is(); + return result; + } + }; + + CanUnrollVisitor canUnroll; + stat->body->visit(&canUnroll); + + return canUnroll.result; + } + + bool tryCompileUnrolledFor(AstStatFor* stat, int thresholdBase, int thresholdMaxBoost) + { + int from = getConstantShort(stat->from); + int to = getConstantShort(stat->to); + int step = stat->step ? getConstantShort(stat->step) : 1; + + // check that limits are reasonably small and trip count can be computed + if (from == INT_MIN || to == INT_MIN || step == INT_MIN || step == 0 || (step < 0 && to > from) || (step > 0 && to < from)) + { + bytecode.addDebugRemark("loop unroll failed: invalid iteration count"); + return false; + } + + if (!canUnrollForBody(stat)) + { + bytecode.addDebugRemark("loop unroll failed: unsupported loop body"); + return false; + } + + int tripCount = (to - from) / step + 1; + + if (tripCount > thresholdBase * thresholdMaxBoost / 100) + { + bytecode.addDebugRemark("loop unroll failed: too many iterations (%d)", tripCount); + return false; + } + + AstLocal* var = stat->var; + uint64_t costModel = modelCost(stat->body, &var, 1); + + // we use a dynamic cost threshold that's based on the fixed limit boosted by the cost advantage we gain due to unrolling + bool varc = true; + int unrolledCost = computeCost(costModel, &varc, 1) * tripCount; + int baselineCost = (computeCost(costModel, nullptr, 0) + 1) * tripCount; + int unrollProfit = (unrolledCost == 0) ? thresholdMaxBoost : std::min(thresholdMaxBoost, 100 * baselineCost / unrolledCost); + + int threshold = thresholdBase * unrollProfit / 100; + + if (unrolledCost > threshold) + { + bytecode.addDebugRemark( + "loop unroll failed: too expensive (iterations %d, cost %d, profit %.2fx)", tripCount, unrolledCost, double(unrollProfit) / 100); + return false; + } + + bytecode.addDebugRemark("loop unroll succeeded (iterations %d, cost %d, profit %.2fx)", tripCount, unrolledCost, double(unrollProfit) / 100); + + for (int i = from; step > 0 ? i <= to : i >= to; i += step) + { + // we need to re-fold constants in the loop body with the new value; this reuses computed constant values elsewhere in the tree + locstants[var].type = Constant::Type_Number; + locstants[var].valueNumber = i; + + foldConstants(constants, variables, locstants, stat); + + compileStat(stat->body); + } + + return true; + } + void compileStatFor(AstStatFor* stat) { RegScope rs(this); + // Optimization: small loops can be unrolled when it is profitable + if (options.optimizationLevel >= 2 && isConstant(stat->to) && isConstant(stat->from) && (!stat->step || isConstant(stat->step))) + if (tryCompileUnrolledFor(stat, FInt::LuauCompileLoopUnrollThreshold, FInt::LuauCompileLoopUnrollThresholdMaxBoost)) + return; + size_t oldLocals = localStack.size(); size_t oldJumps = loopJumps.size(); @@ -2826,6 +2946,8 @@ struct Compiler : self(self) , functions(functions) { + // preallocate the result; this works around std::vector's inefficient growth policy for small arrays + functions.reserve(16); } bool visit(AstExprFunction* node) override @@ -2979,6 +3101,7 @@ struct Compiler DenseHashMap globals; DenseHashMap variables; DenseHashMap constants; + DenseHashMap locstants; DenseHashMap tableShapes; unsigned int regTop = 0; @@ -3008,7 +3131,7 @@ void compileOrThrow(BytecodeBuilder& bytecode, AstStatBlock* root, const AstName if (options.optimizationLevel >= 1) { // this pass analyzes constantness of expressions - foldConstants(compiler.constants, compiler.variables, root); + foldConstants(compiler.constants, compiler.variables, compiler.locstants, root); // this pass analyzes table assignments to estimate table shapes for initially empty tables predictTableShapes(compiler.tableShapes, root); diff --git a/Compiler/src/ConstantFolding.cpp b/Compiler/src/ConstantFolding.cpp index 35ea0bf0..7ad91d4b 100644 --- a/Compiler/src/ConstantFolding.cpp +++ b/Compiler/src/ConstantFolding.cpp @@ -191,13 +191,13 @@ struct ConstantVisitor : AstVisitor { DenseHashMap& constants; DenseHashMap& variables; + DenseHashMap& locals; - DenseHashMap locals; - - ConstantVisitor(DenseHashMap& constants, DenseHashMap& variables) + ConstantVisitor( + DenseHashMap& constants, DenseHashMap& variables, DenseHashMap& locals) : constants(constants) , variables(variables) - , locals(nullptr) + , locals(locals) { } @@ -385,9 +385,10 @@ struct ConstantVisitor : AstVisitor } }; -void foldConstants(DenseHashMap& constants, DenseHashMap& variables, AstNode* root) +void foldConstants(DenseHashMap& constants, DenseHashMap& variables, + DenseHashMap& locals, AstNode* root) { - ConstantVisitor visitor{constants, variables}; + ConstantVisitor visitor{constants, variables, locals}; root->visit(&visitor); } diff --git a/Compiler/src/ConstantFolding.h b/Compiler/src/ConstantFolding.h index c0e63539..0a995d75 100644 --- a/Compiler/src/ConstantFolding.h +++ b/Compiler/src/ConstantFolding.h @@ -42,7 +42,8 @@ struct Constant } }; -void foldConstants(DenseHashMap& constants, DenseHashMap& variables, AstNode* root); +void foldConstants(DenseHashMap& constants, DenseHashMap& variables, + DenseHashMap& locals, AstNode* root); } // namespace Compile } // namespace Luau diff --git a/VM/src/lapi.cpp b/VM/src/lapi.cpp index 46b10934..431f7e59 100644 --- a/VM/src/lapi.cpp +++ b/VM/src/lapi.cpp @@ -14,6 +14,8 @@ #include +LUAU_FASTFLAG(LuauGcWorkTrackFix) + const char* lua_ident = "$Lua: Lua 5.1.4 Copyright (C) 1994-2008 Lua.org, PUC-Rio $\n" "$Authors: R. Ierusalimschy, L. H. de Figueiredo & W. Celes $\n" "$URL: www.lua.org $\n"; @@ -1050,6 +1052,7 @@ int lua_gc(lua_State* L, int what, int data) { size_t prevthreshold = g->GCthreshold; size_t amount = (cast_to(size_t, data) << 10); + ptrdiff_t oldcredit = g->gcstate == GCSpause ? 0 : g->GCthreshold - g->totalbytes; // temporarily adjust the threshold so that we can perform GC work if (amount <= g->totalbytes) @@ -1069,9 +1072,9 @@ int lua_gc(lua_State* L, int what, int data) while (g->GCthreshold <= g->totalbytes) { - luaC_step(L, false); + size_t stepsize = luaC_step(L, false); - actualwork += g->gcstepsize; + actualwork += FFlag::LuauGcWorkTrackFix ? stepsize : g->gcstepsize; if (g->gcstate == GCSpause) { /* end of cycle? */ @@ -1107,11 +1110,20 @@ int lua_gc(lua_State* L, int what, int data) // if cycle hasn't finished, advance threshold forward for the amount of extra work performed if (g->gcstate != GCSpause) { - // if a new cycle was triggered by explicit step, we ignore old threshold as that shows an incorrect 'credit' of GC work - if (waspaused) - g->GCthreshold = g->totalbytes + actualwork; + if (FFlag::LuauGcWorkTrackFix) + { + // if a new cycle was triggered by explicit step, old 'credit' of GC work is 0 + ptrdiff_t newthreshold = g->totalbytes + actualwork + oldcredit; + g->GCthreshold = newthreshold < 0 ? 0 : newthreshold; + } else - g->GCthreshold = prevthreshold + actualwork; + { + // if a new cycle was triggered by explicit step, we ignore old threshold as that shows an incorrect 'credit' of GC work + if (waspaused) + g->GCthreshold = g->totalbytes + actualwork; + else + g->GCthreshold = prevthreshold + actualwork; + } } break; } diff --git a/VM/src/lgc.cpp b/VM/src/lgc.cpp index 8fc930d5..e7b73fe7 100644 --- a/VM/src/lgc.cpp +++ b/VM/src/lgc.cpp @@ -13,9 +13,10 @@ #include -#define GC_SWEEPMAX 40 -#define GC_SWEEPCOST 10 -#define GC_SWEEPPAGESTEPCOST 4 +LUAU_FASTFLAGVARIABLE(LuauGcWorkTrackFix, false) +LUAU_FASTFLAGVARIABLE(LuauGcSweepCostFix, false) + +#define GC_SWEEPPAGESTEPCOST (FFlag::LuauGcSweepCostFix ? 16 : 4) #define GC_INTERRUPT(state) \ { \ @@ -64,7 +65,7 @@ static void recordGcStateStep(global_State* g, int startgcstate, double seconds, case GCSpropagate: case GCSpropagateagain: g->gcmetrics.currcycle.marktime += seconds; - g->gcmetrics.currcycle.markrequests += g->gcstepsize; + g->gcmetrics.currcycle.markwork += work; if (assist) g->gcmetrics.currcycle.markassisttime += seconds; @@ -74,7 +75,7 @@ static void recordGcStateStep(global_State* g, int startgcstate, double seconds, break; case GCSsweep: g->gcmetrics.currcycle.sweeptime += seconds; - g->gcmetrics.currcycle.sweeprequests += g->gcstepsize; + g->gcmetrics.currcycle.sweepwork += work; if (assist) g->gcmetrics.currcycle.sweepassisttime += seconds; @@ -87,13 +88,11 @@ static void recordGcStateStep(global_State* g, int startgcstate, double seconds, { g->gcmetrics.stepassisttimeacc += seconds; g->gcmetrics.currcycle.assistwork += work; - g->gcmetrics.currcycle.assistrequests += g->gcstepsize; } else { g->gcmetrics.stepexplicittimeacc += seconds; g->gcmetrics.currcycle.explicitwork += work; - g->gcmetrics.currcycle.explicitrequests += g->gcstepsize; } } @@ -878,11 +877,11 @@ static size_t getheaptrigger(global_State* g, size_t heapgoal) return heaptrigger < int64_t(g->totalbytes) ? g->totalbytes : (heaptrigger > int64_t(heapgoal) ? heapgoal : size_t(heaptrigger)); } -void luaC_step(lua_State* L, bool assist) +size_t luaC_step(lua_State* L, bool assist) { global_State* g = L->global; - int lim = (g->gcstepsize / 100) * g->gcstepmul; /* how much to work */ + int lim = FFlag::LuauGcWorkTrackFix ? g->gcstepsize * g->gcstepmul / 100 : (g->gcstepsize / 100) * g->gcstepmul; /* how much to work */ LUAU_ASSERT(g->totalbytes >= g->GCthreshold); size_t debt = g->totalbytes - g->GCthreshold; @@ -902,12 +901,13 @@ void luaC_step(lua_State* L, bool assist) int lastgcstate = g->gcstate; size_t work = gcstep(L, lim); - (void)work; #ifdef LUAI_GCMETRICS recordGcStateStep(g, lastgcstate, lua_clock() - lasttimestamp, assist, work); #endif + size_t actualstepsize = work * 100 / g->gcstepmul; + // at the end of the last cycle if (g->gcstate == GCSpause) { @@ -927,14 +927,16 @@ void luaC_step(lua_State* L, bool assist) } else { - g->GCthreshold = g->totalbytes + g->gcstepsize; + g->GCthreshold = g->totalbytes + (FFlag::LuauGcWorkTrackFix ? actualstepsize : g->gcstepsize); // compensate if GC is "behind schedule" (has some debt to pay) - if (g->GCthreshold > debt) + if (FFlag::LuauGcWorkTrackFix ? g->GCthreshold >= debt : g->GCthreshold > debt) g->GCthreshold -= debt; } GC_INTERRUPT(lastgcstate); + + return actualstepsize; } void luaC_fullgc(lua_State* L) diff --git a/VM/src/lgc.h b/VM/src/lgc.h index dcd070b7..08d1ff5d 100644 --- a/VM/src/lgc.h +++ b/VM/src/lgc.h @@ -133,7 +133,7 @@ #define luaC_init(L, o, tt) luaC_initobj(L, cast_to(GCObject*, (o)), tt) LUAI_FUNC void luaC_freeall(lua_State* L); -LUAI_FUNC void luaC_step(lua_State* L, bool assist); +LUAI_FUNC size_t luaC_step(lua_State* L, bool assist); LUAI_FUNC void luaC_fullgc(lua_State* L); LUAI_FUNC void luaC_initobj(lua_State* L, GCObject* o, uint8_t tt); LUAI_FUNC void luaC_initupval(lua_State* L, UpVal* uv); diff --git a/VM/src/lstate.h b/VM/src/lstate.h index e7c37373..45d9ba2c 100644 --- a/VM/src/lstate.h +++ b/VM/src/lstate.h @@ -106,7 +106,7 @@ struct GCCycleMetrics double markassisttime = 0.0; double markmaxexplicittime = 0.0; size_t markexplicitsteps = 0; - size_t markrequests = 0; + size_t markwork = 0; double atomicstarttimestamp = 0.0; size_t atomicstarttotalsizebytes = 0; @@ -122,10 +122,7 @@ struct GCCycleMetrics double sweepassisttime = 0.0; double sweepmaxexplicittime = 0.0; size_t sweepexplicitsteps = 0; - size_t sweeprequests = 0; - - size_t assistrequests = 0; - size_t explicitrequests = 0; + size_t sweepwork = 0; size_t assistwork = 0; size_t explicitwork = 0; diff --git a/bench/bench.py b/bench/bench.py index 39f219f3..67fc8cf7 100644 --- a/bench/bench.py +++ b/bench/bench.py @@ -814,13 +814,12 @@ def run(args, argsubcb): analyzeResult('', mainResult, compareResults) else: - for subdir, dirs, files in os.walk(arguments.folder): - for filename in files: - filepath = subdir + os.sep + filename - - if filename.endswith(".lua"): - if arguments.run_test == None or re.match(arguments.run_test, filename[:-4]): - runTest(subdir, filename, filepath) + all_files = [subdir + os.sep + filename for subdir, dirs, files in os.walk(arguments.folder) for filename in files] + for filepath in sorted(all_files): + subdir, filename = os.path.split(filepath) + if filename.endswith(".lua"): + if arguments.run_test == None or re.match(arguments.run_test, filename[:-4]): + runTest(subdir, filename, filepath) if arguments.sort and len(plotValueLists) > 1: rearrange(rearrangeSortKeyForComparison) diff --git a/fuzz/proto.cpp b/fuzz/proto.cpp index 1022831b..a48f068b 100644 --- a/fuzz/proto.cpp +++ b/fuzz/proto.cpp @@ -103,7 +103,7 @@ int registerTypes(Luau::TypeChecker& env) // Vector3 stub TypeId vector3MetaType = arena.addType(TableTypeVar{}); - TypeId vector3InstanceType = arena.addType(ClassTypeVar{"Vector3", {}, nullopt, vector3MetaType, {}, {}}); + TypeId vector3InstanceType = arena.addType(ClassTypeVar{"Vector3", {}, nullopt, vector3MetaType, {}, {}, "Test"}); getMutable(vector3InstanceType)->props = { {"X", {env.numberType}}, {"Y", {env.numberType}}, @@ -117,7 +117,7 @@ int registerTypes(Luau::TypeChecker& env) env.globalScope->exportedTypeBindings["Vector3"] = TypeFun{{}, vector3InstanceType}; // Instance stub - TypeId instanceType = arena.addType(ClassTypeVar{"Instance", {}, nullopt, nullopt, {}, {}}); + TypeId instanceType = arena.addType(ClassTypeVar{"Instance", {}, nullopt, nullopt, {}, {}, "Test"}); getMutable(instanceType)->props = { {"Name", {env.stringType}}, }; @@ -125,7 +125,7 @@ int registerTypes(Luau::TypeChecker& env) env.globalScope->exportedTypeBindings["Instance"] = TypeFun{{}, instanceType}; // Part stub - TypeId partType = arena.addType(ClassTypeVar{"Part", {}, instanceType, nullopt, {}, {}}); + TypeId partType = arena.addType(ClassTypeVar{"Part", {}, instanceType, nullopt, {}, {}, "Test"}); getMutable(partType)->props = { {"Position", {vector3InstanceType}}, }; @@ -173,7 +173,7 @@ struct FuzzConfigResolver : Luau::ConfigResolver { FuzzConfigResolver() { - defaultConfig.mode = Luau::Mode::Nonstrict; // typecheckTwice option will cover Strict mode + defaultConfig.mode = Luau::Mode::Nonstrict; defaultConfig.enabledLint.warningMask = ~0ull; defaultConfig.parseOptions.captureComments = true; } @@ -275,6 +275,11 @@ DEFINE_PROTO_FUZZER(const luau::ModuleSet& message) // lint (note that we need access to types so we need to do this with typeck in scope) if (kFuzzLinter && result.errors.empty()) frontend.lint(name, std::nullopt); + + // Second pass in strict mode (forced by auto-complete) + Luau::FrontendOptions opts; + opts.forAutocomplete = true; + frontend.check(name, opts); } catch (std::exception&) { diff --git a/tests/Autocomplete.test.cpp b/tests/Autocomplete.test.cpp index 2e7902f5..f66e23ed 100644 --- a/tests/Autocomplete.test.cpp +++ b/tests/Autocomplete.test.cpp @@ -3034,4 +3034,40 @@ string:@1 CHECK(ac.entryMap["sub"].wrongIndexType == true); } +TEST_CASE_FIXTURE(ACFixture, "source_module_preservation_and_invalidation") +{ + check(R"( +local a = { x = 2, y = 4 } +a.@1 + )"); + + frontend.clear(); + + auto ac = autocomplete('1'); + + CHECK(ac.entryMap.count("x")); + CHECK(ac.entryMap.count("y")); + + frontend.check("MainModule", {}); + + ac = autocomplete('1'); + + CHECK(ac.entryMap.count("x")); + CHECK(ac.entryMap.count("y")); + + frontend.markDirty("MainModule", nullptr); + + ac = autocomplete('1'); + + CHECK(ac.entryMap.count("x")); + CHECK(ac.entryMap.count("y")); + + frontend.check("MainModule", {}); + + ac = autocomplete('1'); + + CHECK(ac.entryMap.count("x")); + CHECK(ac.entryMap.count("y")); +} + TEST_SUITE_END(); diff --git a/tests/Compiler.test.cpp b/tests/Compiler.test.cpp index 83dad729..f3e60690 100644 --- a/tests/Compiler.test.cpp +++ b/tests/Compiler.test.cpp @@ -17,11 +17,13 @@ std::string rep(const std::string& s, size_t n); using namespace Luau; -static std::string compileFunction(const char* source, uint32_t id) +static std::string compileFunction(const char* source, uint32_t id, int optimizationLevel = 1) { Luau::BytecodeBuilder bcb; bcb.setDumpFlags(Luau::BytecodeBuilder::Dump_Code); - Luau::compileOrThrow(bcb, source); + Luau::CompileOptions options; + options.optimizationLevel = optimizationLevel; + Luau::compileOrThrow(bcb, source, options); return bcb.dumpFunction(id); } @@ -2689,6 +2691,27 @@ local 8: reg 3, start pc 34 line 21, end pc 34 line 21 )"); } +TEST_CASE("DebugRemarks") +{ + Luau::BytecodeBuilder bcb; + bcb.setDumpFlags(Luau::BytecodeBuilder::Dump_Code | Luau::BytecodeBuilder::Dump_Remarks); + + uint32_t fid = bcb.beginFunction(0); + + bcb.addDebugRemark("test remark #%d", 42); + bcb.emitABC(LOP_RETURN, 0, 1, 0); + + bcb.endFunction(0, 0); + + bcb.setMainFunction(fid); + bcb.finalize(); + + CHECK_EQ("\n" + bcb.dumpFunction(0), R"( +REMARK test remark #42 +RETURN R0 0 +)"); +} + TEST_CASE("AssignmentConflict") { // assignments are left to right @@ -4076,4 +4099,336 @@ RETURN R1 6 )"); } +TEST_CASE("LoopUnrollBasic") +{ + // forward loops + CHECK_EQ("\n" + compileFunction(R"( +local t = {} +for i=1,2 do + t[i] = i +end +return t +)", + 0, 2), + R"( +NEWTABLE R0 0 2 +LOADN R1 1 +SETTABLEN R1 R0 1 +LOADN R1 2 +SETTABLEN R1 R0 2 +RETURN R0 1 +)"); + + // backward loops + CHECK_EQ("\n" + compileFunction(R"( +local t = {} +for i=2,1,-1 do + t[i] = i +end +return t +)", + 0, 2), + R"( +NEWTABLE R0 0 0 +LOADN R1 2 +SETTABLEN R1 R0 2 +LOADN R1 1 +SETTABLEN R1 R0 1 +RETURN R0 1 +)"); + + // loops with step that doesn't divide to-from + CHECK_EQ("\n" + compileFunction(R"( +local t = {} +for i=1,4,2 do + t[i] = i +end +return t +)", + 0, 2), + R"( +NEWTABLE R0 0 0 +LOADN R1 1 +SETTABLEN R1 R0 1 +LOADN R1 3 +SETTABLEN R1 R0 3 +RETURN R0 1 +)"); +} + +TEST_CASE("LoopUnrollUnsupported") +{ + // can't unroll loops with non-constant bounds + CHECK_EQ("\n" + compileFunction(R"( +for i=x,y,z do +end +)", + 0, 2), + R"( +GETIMPORT R2 1 +GETIMPORT R0 3 +GETIMPORT R1 5 +FORNPREP R0 +1 +FORNLOOP R0 -1 +RETURN R0 0 +)"); + + // can't unroll loops with bounds where we can't compute trip count + CHECK_EQ("\n" + compileFunction(R"( +for i=2,1 do +end +)", + 0, 2), + R"( +LOADN R2 2 +LOADN R0 1 +LOADN R1 1 +FORNPREP R0 +1 +FORNLOOP R0 -1 +RETURN R0 0 +)"); + + // can't unroll loops with bounds that might be imprecise (non-integer) + CHECK_EQ("\n" + compileFunction(R"( +for i=1,2,0.1 do +end +)", + 0, 2), + R"( +LOADN R2 1 +LOADN R0 2 +LOADK R1 K0 +FORNPREP R0 +1 +FORNLOOP R0 -1 +RETURN R0 0 +)"); + + // can't unroll loops if the bounds are too large, as it might overflow trip count math + CHECK_EQ("\n" + compileFunction(R"( +for i=4294967295,4294967296 do +end +)", + 0, 2), + R"( +LOADK R2 K0 +LOADK R0 K1 +LOADN R1 1 +FORNPREP R0 +1 +FORNLOOP R0 -1 +RETURN R0 0 +)"); + + // can't unroll loops if the body has loop control flow or nested loops + CHECK_EQ("\n" + compileFunction(R"( +for i=1,1 do + for j=1,1 do + if i == 1 then + continue + else + break + end + end +end +)", + 0, 2), + R"( +LOADN R2 1 +LOADN R0 1 +LOADN R1 1 +FORNPREP R0 +11 +LOADN R5 1 +LOADN R3 1 +LOADN R4 1 +FORNPREP R3 +6 +JUMPIFNOTEQK R2 K0 +5 +JUMP +2 +JUMP +1 +JUMP +1 +FORNLOOP R3 -6 +FORNLOOP R0 -11 +RETURN R0 0 +)"); + + // can't unroll loops if the body has functions that refer to loop variables + CHECK_EQ("\n" + compileFunction(R"( +for i=1,1 do + local x = function() return i end +end +)", + 1, 2), + R"( +LOADN R2 1 +LOADN R0 1 +LOADN R1 1 +FORNPREP R0 +3 +NEWCLOSURE R3 P0 +CAPTURE VAL R2 +FORNLOOP R0 -3 +RETURN R0 0 +)"); +} + +TEST_CASE("LoopUnrollCost") +{ + ScopedFastInt sfis[] = { + {"LuauCompileLoopUnrollThreshold", 25}, + {"LuauCompileLoopUnrollThresholdMaxBoost", 300}, + }; + + // loops with short body + CHECK_EQ("\n" + compileFunction(R"( +local t = {} +for i=1,10 do + t[i] = i +end +return t +)", + 0, 2), + R"( +NEWTABLE R0 0 10 +LOADN R1 1 +SETTABLEN R1 R0 1 +LOADN R1 2 +SETTABLEN R1 R0 2 +LOADN R1 3 +SETTABLEN R1 R0 3 +LOADN R1 4 +SETTABLEN R1 R0 4 +LOADN R1 5 +SETTABLEN R1 R0 5 +LOADN R1 6 +SETTABLEN R1 R0 6 +LOADN R1 7 +SETTABLEN R1 R0 7 +LOADN R1 8 +SETTABLEN R1 R0 8 +LOADN R1 9 +SETTABLEN R1 R0 9 +LOADN R1 10 +SETTABLEN R1 R0 10 +RETURN R0 1 +)"); + + // loops with body that's too long + CHECK_EQ("\n" + compileFunction(R"( +local t = {} +for i=1,100 do + t[i] = i +end +return t +)", + 0, 2), + R"( +NEWTABLE R0 0 0 +LOADN R3 1 +LOADN R1 100 +LOADN R2 1 +FORNPREP R1 +2 +SETTABLE R3 R0 R3 +FORNLOOP R1 -2 +RETURN R0 1 +)"); + + // loops with body that's long but has a high boost factor due to constant folding + CHECK_EQ("\n" + compileFunction(R"( +local t = {} +for i=1,30 do + t[i] = i * i * i +end +return t +)", + 0, 2), + R"( +NEWTABLE R0 0 0 +LOADN R1 1 +SETTABLEN R1 R0 1 +LOADN R1 8 +SETTABLEN R1 R0 2 +LOADN R1 27 +SETTABLEN R1 R0 3 +LOADN R1 64 +SETTABLEN R1 R0 4 +LOADN R1 125 +SETTABLEN R1 R0 5 +LOADN R1 216 +SETTABLEN R1 R0 6 +LOADN R1 343 +SETTABLEN R1 R0 7 +LOADN R1 512 +SETTABLEN R1 R0 8 +LOADN R1 729 +SETTABLEN R1 R0 9 +LOADN R1 1000 +SETTABLEN R1 R0 10 +LOADN R1 1331 +SETTABLEN R1 R0 11 +LOADN R1 1728 +SETTABLEN R1 R0 12 +LOADN R1 2197 +SETTABLEN R1 R0 13 +LOADN R1 2744 +SETTABLEN R1 R0 14 +LOADN R1 3375 +SETTABLEN R1 R0 15 +LOADN R1 4096 +SETTABLEN R1 R0 16 +LOADN R1 4913 +SETTABLEN R1 R0 17 +LOADN R1 5832 +SETTABLEN R1 R0 18 +LOADN R1 6859 +SETTABLEN R1 R0 19 +LOADN R1 8000 +SETTABLEN R1 R0 20 +LOADN R1 9261 +SETTABLEN R1 R0 21 +LOADN R1 10648 +SETTABLEN R1 R0 22 +LOADN R1 12167 +SETTABLEN R1 R0 23 +LOADN R1 13824 +SETTABLEN R1 R0 24 +LOADN R1 15625 +SETTABLEN R1 R0 25 +LOADN R1 17576 +SETTABLEN R1 R0 26 +LOADN R1 19683 +SETTABLEN R1 R0 27 +LOADN R1 21952 +SETTABLEN R1 R0 28 +LOADN R1 24389 +SETTABLEN R1 R0 29 +LOADN R1 27000 +SETTABLEN R1 R0 30 +RETURN R0 1 +)"); + + // loops with body that's long and doesn't have a high boost factor + CHECK_EQ("\n" + compileFunction(R"( +local t = {} +for i=1,10 do + t[i] = math.abs(math.sin(i)) +end +return t +)", + 0, 2), + R"( +NEWTABLE R0 0 10 +LOADN R3 1 +LOADN R1 10 +LOADN R2 1 +FORNPREP R1 +11 +FASTCALL1 24 R3 +3 +MOVE R6 R3 +GETIMPORT R5 2 +CALL R5 1 -1 +FASTCALL 2 +2 +GETIMPORT R4 4 +CALL R4 -1 1 +SETTABLE R4 R0 R3 +FORNLOOP R1 -11 +RETURN R0 1 +)"); +} + TEST_SUITE_END(); diff --git a/tests/CostModel.test.cpp b/tests/CostModel.test.cpp index ec04932f..aa5b7284 100644 --- a/tests/CostModel.test.cpp +++ b/tests/CostModel.test.cpp @@ -98,4 +98,129 @@ end CHECK_EQ(2, Luau::Compile::computeCost(model, args2, 1)); } +TEST_CASE("ImportCall") +{ + uint64_t model = modelFunction(R"( +function test(a) + return Instance.new(a) +end +)"); + + const bool args1[] = {false}; + const bool args2[] = {true}; + + CHECK_EQ(6, Luau::Compile::computeCost(model, args1, 1)); + CHECK_EQ(6, Luau::Compile::computeCost(model, args2, 1)); +} + +TEST_CASE("FastCall") +{ + uint64_t model = modelFunction(R"( +function test(a) + return math.abs(a + 1) +end +)"); + + const bool args1[] = {false}; + const bool args2[] = {true}; + + // note: we currently don't treat fast calls differently from cost model perspective + CHECK_EQ(6, Luau::Compile::computeCost(model, args1, 1)); + CHECK_EQ(5, Luau::Compile::computeCost(model, args2, 1)); +} + +TEST_CASE("ControlFlow") +{ + uint64_t model = modelFunction(R"( +function test(a) + while a < 0 do + a += 1 + end + for i=1,2 do + a += 1 + end + for i in pairs({}) do + a += 1 + if a % 2 == 0 then continue end + end + repeat + a += 1 + if a % 2 == 0 then break end + until a > 10 + return a +end +)"); + + const bool args1[] = {false}; + const bool args2[] = {true}; + + CHECK_EQ(38, Luau::Compile::computeCost(model, args1, 1)); + CHECK_EQ(37, Luau::Compile::computeCost(model, args2, 1)); +} + +TEST_CASE("Conditional") +{ + uint64_t model = modelFunction(R"( +function test(a) + return if a < 0 then -a else a +end +)"); + + const bool args1[] = {false}; + const bool args2[] = {true}; + + CHECK_EQ(4, Luau::Compile::computeCost(model, args1, 1)); + CHECK_EQ(2, Luau::Compile::computeCost(model, args2, 1)); +} + +TEST_CASE("VarArgs") +{ + uint64_t model = modelFunction(R"( +function test(...) + return select('#', ...) :: number +end +)"); + + CHECK_EQ(8, Luau::Compile::computeCost(model, nullptr, 0)); +} + +TEST_CASE("TablesFunctions") +{ + uint64_t model = modelFunction(R"( +function test() + return { 42, op = function() end } +end +)"); + + CHECK_EQ(22, Luau::Compile::computeCost(model, nullptr, 0)); +} + +TEST_CASE("CostOverflow") +{ + uint64_t model = modelFunction(R"( +function test() + return {{{{{{{{{{{{{{{}}}}}}}}}}}}}}} +end +)"); + + CHECK_EQ(127, Luau::Compile::computeCost(model, nullptr, 0)); +} + +TEST_CASE("TableAssign") +{ + uint64_t model = modelFunction(R"( +function test(a) + for i=1,#a do + a[i] = i + end +end +)"); + + const bool args1[] = {false}; + const bool args2[] = {true}; + + CHECK_EQ(4, Luau::Compile::computeCost(model, args1, 1)); + CHECK_EQ(3, Luau::Compile::computeCost(model, args2, 1)); +} + TEST_SUITE_END(); diff --git a/tests/JsonEncoder.test.cpp b/tests/JsonEncoder.test.cpp index 1d2ad645..6f1cebc6 100644 --- a/tests/JsonEncoder.test.cpp +++ b/tests/JsonEncoder.test.cpp @@ -9,6 +9,46 @@ using namespace Luau; +struct JsonEncoderFixture +{ + Allocator allocator; + AstNameTable names{allocator}; + + ParseResult parse(std::string_view src) + { + ParseOptions opts; + opts.allowDeclarationSyntax = true; + return Parser::parse(src.data(), src.size(), names, allocator, opts); + } + + AstStatBlock* expectParse(std::string_view src) + { + ParseResult res = parse(src); + REQUIRE(res.errors.size() == 0); + return res.root; + } + + AstStat* expectParseStatement(std::string_view src) + { + AstStatBlock* root = expectParse(src); + REQUIRE(1 == root->body.size); + return root->body.data[0]; + } + + AstExpr* expectParseExpr(std::string_view src) + { + std::string s = "a = "; + s.append(src); + AstStatBlock* root = expectParse(s); + + AstStatAssign* statAssign = root->body.data[0]->as(); + REQUIRE(statAssign != nullptr); + REQUIRE(statAssign->values.size == 1); + + return statAssign->values.data[0]; + } +}; + TEST_SUITE_BEGIN("JsonEncoderTests"); TEST_CASE("encode_constants") @@ -51,7 +91,7 @@ TEST_CASE("encode_AstStatBlock") toJson(&block)); } -TEST_CASE("encode_tables") +TEST_CASE_FIXTURE(JsonEncoderFixture, "encode_tables") { std::string src = R"( local x: { @@ -61,16 +101,294 @@ TEST_CASE("encode_tables") } )"; - Allocator allocator; - AstNameTable names(allocator); - ParseResult parseResult = Parser::parse(src.c_str(), src.length(), names, allocator); - - REQUIRE(parseResult.errors.size() == 0); - std::string json = toJson(parseResult.root); + AstStatBlock* root = expectParse(src); + std::string json = toJson(root); CHECK( json == R"({"type":"AstStatBlock","location":"0,0 - 6,4","body":[{"type":"AstStatLocal","location":"1,8 - 5,9","vars":[{"type":{"type":"AstTypeTable","location":"1,17 - 3,9","props":[{"name":"foo","location":"2,12 - 2,15","type":{"type":"AstTypeReference","location":"2,17 - 2,23","name":"number","parameters":[]}}],"indexer":false},"name":"x","location":"1,14 - 1,15"}],"values":[{"type":"AstExprTable","location":"3,12 - 5,9","items":[{"kind":"record","key":{"type":"AstExprConstantString","location":"4,12 - 4,15","value":"foo"},"value":{"type":"AstExprConstantNumber","location":"4,18 - 4,21","value":123}}]}]}]})"); } +TEST_CASE("encode_AstExprGroup") +{ + AstExprConstantNumber number{Location{}, 5.0}; + AstExprGroup group{Location{}, &number}; + + std::string json = toJson(&group); + + const std::string expected = R"({"type":"AstExprGroup","location":"0,0 - 0,0","expr":{"type":"AstExprConstantNumber","location":"0,0 - 0,0","value":5}})"; + + CHECK(json == expected); +} + +TEST_CASE("encode_AstExprGlobal") +{ + AstExprGlobal global{Location{}, AstName{"print"}}; + + std::string json = toJson(&global); + std::string expected = R"({"type":"AstExprGlobal","location":"0,0 - 0,0","global":"print"})"; + + CHECK(json == expected); +} + +TEST_CASE("encode_AstExprLocal") +{ + AstLocal local{AstName{"foo"}, Location{}, nullptr, 0, 0, nullptr}; + AstExprLocal exprLocal{Location{}, &local, false}; + + CHECK(toJson(&exprLocal) == R"({"type":"AstExprLocal","location":"0,0 - 0,0","local":{"type":null,"name":"foo","location":"0,0 - 0,0"}})"); +} + +TEST_CASE("encode_AstExprVarargs") +{ + AstExprVarargs varargs{Location{}}; + + CHECK(toJson(&varargs) == R"({"type":"AstExprVarargs","location":"0,0 - 0,0"})"); +} + +TEST_CASE_FIXTURE(JsonEncoderFixture, "encode_AstExprCall") +{ + AstExpr* expr = expectParseExpr("foo(1, 2, 3)"); + std::string_view expected = R"({"type":"AstExprCall","location":"0,4 - 0,16","func":{"type":"AstExprGlobal","location":"0,4 - 0,7","global":"foo"},"args":[{"type":"AstExprConstantNumber","location":"0,8 - 0,9","value":1},{"type":"AstExprConstantNumber","location":"0,11 - 0,12","value":2},{"type":"AstExprConstantNumber","location":"0,14 - 0,15","value":3}],"self":false,"argLocation":"0,8 - 0,16"})"; + + CHECK(toJson(expr) == expected); +} + +TEST_CASE_FIXTURE(JsonEncoderFixture, "encode_AstExprIndexName") +{ + AstExpr* expr = expectParseExpr("foo.bar"); + + std::string_view expected = R"({"type":"AstExprIndexName","location":"0,4 - 0,11","expr":{"type":"AstExprGlobal","location":"0,4 - 0,7","global":"foo"},"index":"bar","indexLocation":"0,8 - 0,11","op":"."})"; + + CHECK(toJson(expr) == expected); +} + +TEST_CASE_FIXTURE(JsonEncoderFixture, "encode_AstExprIndexExpr") +{ + AstExpr* expr = expectParseExpr("foo['bar']"); + + std::string_view expected = R"({"type":"AstExprIndexExpr","location":"0,4 - 0,14","expr":{"type":"AstExprGlobal","location":"0,4 - 0,7","global":"foo"},"index":{"type":"AstExprConstantString","location":"0,8 - 0,13","value":"bar"}})"; + + CHECK(toJson(expr) == expected); +} + +TEST_CASE_FIXTURE(JsonEncoderFixture, "encode_AstExprFunction") +{ + AstExpr* expr = expectParseExpr("function (a) return a end"); + + std::string_view expected = R"({"type":"AstExprFunction","location":"0,4 - 0,29","generics":[],"genericPacks":[],"args":[{"type":null,"name":"a","location":"0,14 - 0,15"}],"vararg":false,"varargLocation":"0,0 - 0,0","body":{"type":"AstStatBlock","location":"0,16 - 0,26","body":[{"type":"AstStatReturn","location":"0,17 - 0,25","list":[{"type":"AstExprLocal","location":"0,24 - 0,25","local":{"type":null,"name":"a","location":"0,14 - 0,15"}}]}]},"functionDepth":1,"debugname":"","hasEnd":true})"; + + CHECK(toJson(expr) == expected); +} + +TEST_CASE_FIXTURE(JsonEncoderFixture, "encode_AstExprTable") +{ + AstExpr* expr = expectParseExpr("{true, key=true, [key2]=true}"); + + std::string_view expected = R"({"type":"AstExprTable","location":"0,4 - 0,33","items":[{"kind":"item","value":{"type":"AstExprConstantBool","location":"0,5 - 0,9","value":true}},{"kind":"record","key":{"type":"AstExprConstantString","location":"0,11 - 0,14","value":"key"},"value":{"type":"AstExprConstantBool","location":"0,15 - 0,19","value":true}},{"kind":"general","key":{"type":"AstExprGlobal","location":"0,22 - 0,26","global":"key2"},"value":{"type":"AstExprConstantBool","location":"0,28 - 0,32","value":true}}]})"; + + CHECK(toJson(expr) == expected); +} + +TEST_CASE_FIXTURE(JsonEncoderFixture, "encode_AstExprUnary") +{ + AstExpr* expr = expectParseExpr("-b"); + + std::string_view expected = R"({"type":"AstExprUnary","location":"0,4 - 0,6","op":"minus","expr":{"type":"AstExprGlobal","location":"0,5 - 0,6","global":"b"}})"; + + CHECK(toJson(expr) == expected); +} + +TEST_CASE_FIXTURE(JsonEncoderFixture, "encode_AstExprBinary") +{ + AstExpr* expr = expectParseExpr("b + c"); + + std::string_view expected = R"({"type":"AstExprBinary","location":"0,4 - 0,9","op":"Add","left":{"type":"AstExprGlobal","location":"0,4 - 0,5","global":"b"},"right":{"type":"AstExprGlobal","location":"0,8 - 0,9","global":"c"}})"; + + CHECK(toJson(expr) == expected); +} + +TEST_CASE_FIXTURE(JsonEncoderFixture, "encode_AstExprTypeAssertion") +{ + AstExpr* expr = expectParseExpr("b :: any"); + + std::string_view expected = R"({"type":"AstExprTypeAssertion","location":"0,4 - 0,12","expr":{"type":"AstExprGlobal","location":"0,4 - 0,5","global":"b"},"annotation":{"type":"AstTypeReference","location":"0,9 - 0,12","name":"any","parameters":[]}})"; + + CHECK(toJson(expr) == expected); +} + +TEST_CASE_FIXTURE(JsonEncoderFixture, "encode_AstExprError") +{ + std::string_view src = "a = "; + ParseResult parseResult = Parser::parse(src.data(), src.size(), names, allocator); + + REQUIRE(1 == parseResult.root->body.size); + + AstStatAssign* statAssign = parseResult.root->body.data[0]->as(); + REQUIRE(statAssign != nullptr); + REQUIRE(1 == statAssign->values.size); + + AstExpr* expr = statAssign->values.data[0]; + + std::string_view expected = R"({"type":"AstExprError","location":"0,4 - 0,4","expressions":[],"messageIndex":0})"; + + CHECK(toJson(expr) == expected); +} + +TEST_CASE_FIXTURE(JsonEncoderFixture, "encode_AstStatIf") +{ + AstStat* statement = expectParseStatement("if true then else end"); + + std::string_view expected = R"({"type":"AstStatIf","location":"0,0 - 0,21","condition":{"type":"AstExprConstantBool","location":"0,3 - 0,7","value":true},"thenbody":{"type":"AstStatBlock","location":"0,12 - 0,13","body":[]},"elsebody":{"type":"AstStatBlock","location":"0,17 - 0,18","body":[]},"hasThen":true,"hasEnd":true})"; + + CHECK(toJson(statement) == expected); +} + +TEST_CASE_FIXTURE(JsonEncoderFixture, "encode_AstStatWhile") +{ + AstStat* statement = expectParseStatement("while true do end"); + + std::string_view expected = R"({"type":"AtStatWhile","location":"0,0 - 0,17","condition":{"type":"AstExprConstantBool","location":"0,6 - 0,10","value":true},"body":{"type":"AstStatBlock","location":"0,13 - 0,14","body":[]},"hasDo":true,"hasEnd":true})"; + + CHECK(toJson(statement) == expected); +} + +TEST_CASE_FIXTURE(JsonEncoderFixture, "encode_AstStatRepeat") +{ + AstStat* statement = expectParseStatement("repeat until true"); + + std::string_view expected = R"({"type":"AstStatRepeat","location":"0,0 - 0,17","condition":{"type":"AstExprConstantBool","location":"0,13 - 0,17","value":true},"body":{"type":"AstStatBlock","location":"0,6 - 0,7","body":[]},"hasUntil":true})"; + + CHECK(toJson(statement) == expected); +} + +TEST_CASE_FIXTURE(JsonEncoderFixture, "encode_AstStatBreak") +{ + AstStat* statement = expectParseStatement("while true do break end"); + + std::string_view expected = R"({"type":"AtStatWhile","location":"0,0 - 0,23","condition":{"type":"AstExprConstantBool","location":"0,6 - 0,10","value":true},"body":{"type":"AstStatBlock","location":"0,13 - 0,20","body":[{"type":"AstStatBreak","location":"0,14 - 0,19"}]},"hasDo":true,"hasEnd":true})"; + + CHECK(toJson(statement) == expected); +} + +TEST_CASE_FIXTURE(JsonEncoderFixture, "encode_AstStatContinue") +{ + AstStat* statement = expectParseStatement("while true do continue end"); + + std::string_view expected = R"({"type":"AtStatWhile","location":"0,0 - 0,26","condition":{"type":"AstExprConstantBool","location":"0,6 - 0,10","value":true},"body":{"type":"AstStatBlock","location":"0,13 - 0,23","body":[{"type":"AstStatContinue","location":"0,14 - 0,22"}]},"hasDo":true,"hasEnd":true})"; + + CHECK(toJson(statement) == expected); +} + +TEST_CASE_FIXTURE(JsonEncoderFixture, "encode_AstStatFor") +{ + AstStat* statement = expectParseStatement("for a=0,1 do end"); + + std::string_view expected = R"({"type":"AstStatFor","location":"0,0 - 0,16","var":{"type":null,"name":"a","location":"0,4 - 0,5"},"from":{"type":"AstExprConstantNumber","location":"0,6 - 0,7","value":0},"to":{"type":"AstExprConstantNumber","location":"0,8 - 0,9","value":1},"body":{"type":"AstStatBlock","location":"0,12 - 0,13","body":[]},"hasDo":true,"hasEnd":true})"; + + CHECK(toJson(statement) == expected); +} + +TEST_CASE_FIXTURE(JsonEncoderFixture, "encode_AstStatForIn") +{ + AstStat* statement = expectParseStatement("for a in b do end"); + + std::string_view expected = R"({"type":"AstStatForIn","location":"0,0 - 0,17","vars":[{"type":null,"name":"a","location":"0,4 - 0,5"}],"values":[{"type":"AstExprGlobal","location":"0,9 - 0,10","global":"b"}],"body":{"type":"AstStatBlock","location":"0,13 - 0,14","body":[]},"hasIn":true,"hasDo":true,"hasEnd":true})"; + + CHECK(toJson(statement) == expected); +} + +TEST_CASE_FIXTURE(JsonEncoderFixture, "encode_AstStatCompoundAssign") +{ + AstStat* statement = expectParseStatement("a += b"); + + std::string_view expected = R"({"type":"AstStatCompoundAssign","location":"0,0 - 0,6","op":"Add","var":{"type":"AstExprGlobal","location":"0,0 - 0,1","global":"a"},"value":{"type":"AstExprGlobal","location":"0,5 - 0,6","global":"b"}})"; + + CHECK(toJson(statement) == expected); +} + +TEST_CASE_FIXTURE(JsonEncoderFixture, "encode_AstStatLocalFunction") +{ + AstStat* statement = expectParseStatement("local function a(b) return end"); + + std::string_view expected = R"({"type":"AstStatLocalFunction","location":"0,0 - 0,30","name":{"type":null,"name":"a","location":"0,15 - 0,16"},"func":{"type":"AstExprFunction","location":"0,0 - 0,30","generics":[],"genericPacks":[],"args":[{"type":null,"name":"b","location":"0,17 - 0,18"}],"vararg":false,"varargLocation":"0,0 - 0,0","body":{"type":"AstStatBlock","location":"0,19 - 0,27","body":[{"type":"AstStatReturn","location":"0,20 - 0,26","list":[]}]},"functionDepth":1,"debugname":"a","hasEnd":true}})"; + + CHECK(toJson(statement) == expected); +} + +TEST_CASE_FIXTURE(JsonEncoderFixture, "encode_AstStatTypeAlias") +{ + AstStat* statement = expectParseStatement("type A = B"); + + std::string_view expected = R"({"type":"AstStatTypeAlias","location":"0,0 - 0,10","name":"A","generics":[],"genericPacks":[],"type":{"type":"AstTypeReference","location":"0,9 - 0,10","name":"B","parameters":[]},"exported":false})"; + + CHECK(toJson(statement) == expected); +} + +TEST_CASE_FIXTURE(JsonEncoderFixture, "encode_AstStatDeclareFunction") +{ + AstStat* statement = expectParseStatement("declare function foo(x: number): string"); + + std::string_view expected = R"({"type":"AstStatDeclareFunction","location":"0,0 - 0,39","name":"foo","params":{"types":[{"type":"AstTypeReference","location":"0,24 - 0,30","name":"number","parameters":[]}]},"retTypes":{"types":[{"type":"AstTypeReference","location":"0,33 - 0,39","name":"string","parameters":[]}]},"generics":[],"genericPacks":[]})"; + + CHECK(toJson(statement) == expected); +} + +TEST_CASE_FIXTURE(JsonEncoderFixture, "encode_AstStatDeclareClass") +{ + AstStatBlock* root = expectParse(R"( + declare class Foo + prop: number + function method(self, foo: number): string + end + + declare class Bar extends Foo + prop2: string + end + )"); + + REQUIRE(2 == root->body.size); + + std::string_view expected1 = R"({"type":"AstStatDeclareClass","location":"1,22 - 4,11","name":"Foo","props":[{"name":"prop","type":{"type":"AstTypeReference","location":"2,18 - 2,24","name":"number","parameters":[]}},{"name":"method","type":{"type":"AstTypeFunction","location":"3,21 - 4,11","generics":[],"genericPacks":[],"argTypes":{"types":[{"type":"AstTypeReference","location":"3,39 - 3,45","name":"number","parameters":[]}]},"returnTypes":{"types":[{"type":"AstTypeReference","location":"3,48 - 3,54","name":"string","parameters":[]}]}}}]})"; + CHECK(toJson(root->body.data[0]) == expected1); + + std::string_view expected2 = R"({"type":"AstStatDeclareClass","location":"6,22 - 8,11","name":"Bar","superName":"Foo","props":[{"name":"prop2","type":{"type":"AstTypeReference","location":"7,19 - 7,25","name":"string","parameters":[]}}]})"; + CHECK(toJson(root->body.data[1]) == expected2); +} + +TEST_CASE_FIXTURE(JsonEncoderFixture, "encode_annotation") +{ + AstStat* statement = expectParseStatement("type T = ((number) -> (string | nil)) & ((string) -> ())"); + + std::string_view expected = R"({"type":"AstStatTypeAlias","location":"0,0 - 0,55","name":"T","generics":[],"genericPacks":[],"type":{"type":"AstTypeIntersection","location":"0,9 - 0,55","types":[{"type":"AstTypeFunction","location":"0,10 - 0,35","generics":[],"genericPacks":[],"argTypes":{"types":[{"type":"AstTypeReference","location":"0,11 - 0,17","name":"number","parameters":[]}]},"returnTypes":{"types":[{"type":"AstTypeUnion","location":"0,23 - 0,35","types":[{"type":"AstTypeReference","location":"0,23 - 0,29","name":"string","parameters":[]},{"type":"AstTypeReference","location":"0,32 - 0,35","name":"nil","parameters":[]}]}]}},{"type":"AstTypeFunction","location":"0,41 - 0,55","generics":[],"genericPacks":[],"argTypes":{"types":[{"type":"AstTypeReference","location":"0,42 - 0,48","name":"string","parameters":[]}]},"returnTypes":{"types":[]}}]},"exported":false})"; + + CHECK(toJson(statement) == expected); +} + +TEST_CASE_FIXTURE(JsonEncoderFixture, "encode_AstTypeError") +{ + ParseResult parseResult = parse("type T = "); + REQUIRE(1 == parseResult.root->body.size); + + AstStat* statement = parseResult.root->body.data[0]; + + std::string_view expected = R"({"type":"AstStatTypeAlias","location":"0,0 - 0,9","name":"T","generics":[],"genericPacks":[],"type":{"type":"AstTypeError","location":"0,8 - 0,9","types":[],"messageIndex":0},"exported":false})"; + + CHECK(toJson(statement) == expected); +} + +TEST_CASE_FIXTURE(JsonEncoderFixture, "encode_AstTypePackExplicit") +{ + AstStatBlock* root = expectParse(R"( + type A = () -> T... + local a: A<(number, string)> + )"); + + CHECK(2 == root->body.size); + + std::string_view expected = R"({"type":"AstStatLocal","location":"2,8 - 2,36","vars":[{"type":{"type":"AstTypeReference","location":"2,17 - 2,36","name":"A","parameters":[{"type":"AstTypePackExplicit","location":"2,19 - 2,20","typeList":{"types":[{"type":"AstTypeReference","location":"2,20 - 2,26","name":"number","parameters":[]},{"type":"AstTypeReference","location":"2,28 - 2,34","name":"string","parameters":[]}]}}]},"name":"a","location":"2,14 - 2,15"}],"values":[]})"; + + CHECK(toJson(root->body.data[1]) == expected); +} + TEST_SUITE_END(); diff --git a/tests/Linter.test.cpp b/tests/Linter.test.cpp index 05ee9a7b..6649cb7f 100644 --- a/tests/Linter.test.cpp +++ b/tests/Linter.test.cpp @@ -1436,7 +1436,7 @@ TEST_CASE_FIXTURE(Fixture, "LintHygieneUAF") TEST_CASE_FIXTURE(Fixture, "DeprecatedApi") { unfreeze(typeChecker.globalTypes); - TypeId instanceType = typeChecker.globalTypes.addType(ClassTypeVar{"Instance", {}, std::nullopt, std::nullopt, {}, {}}); + TypeId instanceType = typeChecker.globalTypes.addType(ClassTypeVar{"Instance", {}, std::nullopt, std::nullopt, {}, {}, "Test"}); persist(instanceType); typeChecker.globalScope->exportedTypeBindings["Instance"] = TypeFun{{}, instanceType}; diff --git a/tests/Module.test.cpp b/tests/Module.test.cpp index 738893db..af7d76de 100644 --- a/tests/Module.test.cpp +++ b/tests/Module.test.cpp @@ -173,13 +173,13 @@ TEST_CASE_FIXTURE(Fixture, "clone_class") { {"__add", {typeChecker.anyType}}, }, - std::nullopt, std::nullopt, {}, {}}}; + std::nullopt, std::nullopt, {}, {}, "Test"}}; TypeVar exampleClass{ClassTypeVar{"ExampleClass", { {"PropOne", {typeChecker.numberType}}, {"PropTwo", {typeChecker.stringType}}, }, - std::nullopt, &exampleMetaClass, {}, {}}}; + std::nullopt, &exampleMetaClass, {}, {}, "Test"}}; TypeArena dest; CloneState cloneState; @@ -196,9 +196,12 @@ TEST_CASE_FIXTURE(Fixture, "clone_class") CHECK_EQ("ExampleClassMeta", metatable->name); } -TEST_CASE_FIXTURE(Fixture, "clone_sanitize_free_types") +TEST_CASE_FIXTURE(Fixture, "clone_free_types") { - ScopedFastFlag sff{"LuauErrorRecoveryType", true}; + ScopedFastFlag sff[]{ + {"LuauErrorRecoveryType", true}, + {"LuauLosslessClone", true}, + }; TypeVar freeTy(FreeTypeVar{TypeLevel{}}); TypePackVar freeTp(FreeTypePack{TypeLevel{}}); @@ -207,17 +210,17 @@ TEST_CASE_FIXTURE(Fixture, "clone_sanitize_free_types") CloneState cloneState; TypeId clonedTy = clone(&freeTy, dest, cloneState); - CHECK_EQ("any", toString(clonedTy)); - CHECK(cloneState.encounteredFreeType); + CHECK(get(clonedTy)); cloneState = {}; TypePackId clonedTp = clone(&freeTp, dest, cloneState); - CHECK_EQ("...any", toString(clonedTp)); - CHECK(cloneState.encounteredFreeType); + CHECK(get(clonedTp)); } -TEST_CASE_FIXTURE(Fixture, "clone_seal_free_tables") +TEST_CASE_FIXTURE(Fixture, "clone_free_tables") { + ScopedFastFlag sff{"LuauLosslessClone", true}; + TypeVar tableTy{TableTypeVar{}}; TableTypeVar* ttv = getMutable(&tableTy); ttv->state = TableState::Free; @@ -227,8 +230,7 @@ TEST_CASE_FIXTURE(Fixture, "clone_seal_free_tables") TypeId cloned = clone(&tableTy, dest, cloneState); const TableTypeVar* clonedTtv = get(cloned); - CHECK_EQ(clonedTtv->state, TableState::Sealed); - CHECK(cloneState.encounteredFreeType); + CHECK_EQ(clonedTtv->state, TableState::Free); } TEST_CASE_FIXTURE(Fixture, "clone_constrained_intersection") diff --git a/tests/NonstrictMode.test.cpp b/tests/NonstrictMode.test.cpp index a8a12b69..9748eb27 100644 --- a/tests/NonstrictMode.test.cpp +++ b/tests/NonstrictMode.test.cpp @@ -13,6 +13,29 @@ using namespace Luau; TEST_SUITE_BEGIN("NonstrictModeTests"); +TEST_CASE_FIXTURE(Fixture, "function_returns_number_or_string") +{ + ScopedFastFlag sff[]{ + {"LuauReturnTypeInferenceInNonstrict", true}, + {"LuauLowerBoundsCalculation", true} + }; + + CheckResult result = check(R"( + --!nonstrict + local function f() + if math.random() > 0.5 then + return 5 + else + return "hi" + end + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK("() -> number | string" == toString(requireType("f"))); +} + TEST_CASE_FIXTURE(Fixture, "infer_nullary_function") { CheckResult result = check(R"( @@ -35,8 +58,13 @@ TEST_CASE_FIXTURE(Fixture, "infer_nullary_function") REQUIRE_EQ(0, rets.size()); } -TEST_CASE_FIXTURE(Fixture, "infer_the_maximum_number_of_values_the_function_could_return") +TEST_CASE_FIXTURE(Fixture, "first_return_type_dictates_number_of_return_types") { + ScopedFastFlag sff[]{ + {"LuauReturnTypeInferenceInNonstrict", true}, + {"LuauLowerBoundsCalculation", true}, + }; + CheckResult result = check(R"( --!nonstrict function getMinCardCountForWidth(width) @@ -51,22 +79,18 @@ TEST_CASE_FIXTURE(Fixture, "infer_the_maximum_number_of_values_the_function_coul TypeId t = requireType("getMinCardCountForWidth"); REQUIRE(t); - REQUIRE_EQ("(any) -> (...any)", toString(t)); + REQUIRE_EQ("(any) -> number", toString(t)); } -#if 0 -// Maybe we want this? TEST_CASE_FIXTURE(Fixture, "return_annotation_is_still_checked") { CheckResult result = check(R"( + --!nonstrict function foo(x): number return 'hello' end )"); LUAU_REQUIRE_ERROR_COUNT(1, result); - - REQUIRE_NE(*typeChecker.anyType, *requireType("foo")); } -#endif TEST_CASE_FIXTURE(Fixture, "function_parameters_are_any") { @@ -256,6 +280,12 @@ TEST_CASE_FIXTURE(Fixture, "delay_function_does_not_require_its_argument_to_retu TEST_CASE_FIXTURE(Fixture, "inconsistent_module_return_types_are_ok") { + ScopedFastFlag sff[]{ + {"LuauReturnTypeInferenceInNonstrict", true}, + {"LuauLowerBoundsCalculation", true}, + {"LuauSealExports", true}, + }; + CheckResult result = check(R"( --!nonstrict @@ -272,7 +302,7 @@ TEST_CASE_FIXTURE(Fixture, "inconsistent_module_return_types_are_ok") LUAU_REQUIRE_NO_ERRORS(result); - REQUIRE_EQ("any", toString(getMainModule()->getModuleScope()->returnType)); + REQUIRE_EQ("((any) -> string) | {| foo: any |}", toString(getMainModule()->getModuleScope()->returnType)); } TEST_CASE_FIXTURE(Fixture, "returning_insufficient_return_values") diff --git a/tests/Normalize.test.cpp b/tests/Normalize.test.cpp index 5a84201a..d3778f67 100644 --- a/tests/Normalize.test.cpp +++ b/tests/Normalize.test.cpp @@ -21,7 +21,7 @@ void createSomeClasses(TypeChecker& typeChecker) unfreeze(arena); - TypeId parentType = arena.addType(ClassTypeVar{"Parent", {}, std::nullopt, std::nullopt, {}, nullptr}); + TypeId parentType = arena.addType(ClassTypeVar{"Parent", {}, std::nullopt, std::nullopt, {}, nullptr, "Test"}); ClassTypeVar* parentClass = getMutable(parentType); parentClass->props["method"] = {makeFunction(arena, parentType, {}, {})}; @@ -31,7 +31,7 @@ void createSomeClasses(TypeChecker& typeChecker) addGlobalBinding(typeChecker, "Parent", {parentType}); typeChecker.globalScope->exportedTypeBindings["Parent"] = TypeFun{{}, parentType}; - TypeId childType = arena.addType(ClassTypeVar{"Child", {}, parentType, std::nullopt, {}, nullptr}); + TypeId childType = arena.addType(ClassTypeVar{"Child", {}, parentType, std::nullopt, {}, nullptr, "Test"}); ClassTypeVar* childClass = getMutable(childType); childClass->props["virtual_method"] = {makeFunction(arena, childType, {}, {})}; @@ -39,7 +39,7 @@ void createSomeClasses(TypeChecker& typeChecker) addGlobalBinding(typeChecker, "Child", {childType}); typeChecker.globalScope->exportedTypeBindings["Child"] = TypeFun{{}, childType}; - TypeId unrelatedType = arena.addType(ClassTypeVar{"Unrelated", {}, std::nullopt, std::nullopt, {}, nullptr}); + TypeId unrelatedType = arena.addType(ClassTypeVar{"Unrelated", {}, std::nullopt, std::nullopt, {}, nullptr, "Test"}); addGlobalBinding(typeChecker, "Unrelated", {unrelatedType}); typeChecker.globalScope->exportedTypeBindings["Unrelated"] = TypeFun{{}, unrelatedType}; @@ -400,7 +400,6 @@ TEST_CASE_FIXTURE(NormalizeFixture, "table_with_table_prop") CHECK_EQ("{| x: {| y: number & string |} |}", toString(requireType("a"))); } -#if 0 TEST_CASE_FIXTURE(NormalizeFixture, "tables") { check(R"( @@ -428,6 +427,7 @@ TEST_CASE_FIXTURE(NormalizeFixture, "tables") CHECK(!isSubtype(b, d)); } +#if 0 TEST_CASE_FIXTURE(NormalizeFixture, "table_indexers_are_invariant") { check(R"( @@ -619,6 +619,7 @@ TEST_CASE_FIXTURE(Fixture, "normalize_module_return_type") { ScopedFastFlag sff[] = { {"LuauLowerBoundsCalculation", true}, + {"LuauReturnTypeInferenceInNonstrict", true}, }; check(R"( @@ -639,7 +640,7 @@ TEST_CASE_FIXTURE(Fixture, "normalize_module_return_type") end )"); - CHECK_EQ("(any, any) -> (...any)", toString(getMainModule()->getModuleScope()->returnType)); + CHECK_EQ("(any, any) -> (any, any) -> any", toString(getMainModule()->getModuleScope()->returnType)); } TEST_CASE_FIXTURE(Fixture, "return_type_is_not_a_constrained_intersection") @@ -950,6 +951,27 @@ TEST_CASE_FIXTURE(Fixture, "nested_table_normalization_with_non_table__no_ice") LUAU_REQUIRE_NO_ERRORS(result); } +TEST_CASE_FIXTURE(Fixture, "visiting_a_type_twice_is_not_considered_normal") +{ + ScopedFastFlag sff{"LuauLowerBoundsCalculation", true}; + + CheckResult result = check(R"( + --!strict + function f(a, b) + local function g() + if math.random() > 0.5 then + return a() + else + return b + end + end + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + CHECK_EQ("(() -> a, a) -> ()", toString(requireType("f"))); +} + TEST_CASE_FIXTURE(Fixture, "fuzz_failure_instersection_combine_must_follow") { ScopedFastFlag flags[] = { @@ -964,4 +986,16 @@ TEST_CASE_FIXTURE(Fixture, "fuzz_failure_instersection_combine_must_follow") LUAU_REQUIRE_NO_ERRORS(result); } +TEST_CASE_FIXTURE(Fixture, "fuzz_failure_bound_type_is_normal_but_not_its_bounded_to") +{ + ScopedFastFlag sff{"LuauLowerBoundsCalculation", true}; + + CheckResult result = check(R"( + type t252 = ((t0)|(any))|(any) + type t0 = t252,t24...> + )"); + + LUAU_REQUIRE_ERRORS(result); +} + TEST_SUITE_END(); diff --git a/tests/ToDot.test.cpp b/tests/ToDot.test.cpp index f3fda54e..332a4b22 100644 --- a/tests/ToDot.test.cpp +++ b/tests/ToDot.test.cpp @@ -21,13 +21,13 @@ struct ToDotClassFixture : Fixture TypeId baseClassMetaType = arena.addType(TableTypeVar{}); - TypeId baseClassInstanceType = arena.addType(ClassTypeVar{"BaseClass", {}, std::nullopt, baseClassMetaType, {}, {}}); + TypeId baseClassInstanceType = arena.addType(ClassTypeVar{"BaseClass", {}, std::nullopt, baseClassMetaType, {}, {}, "Test"}); getMutable(baseClassInstanceType)->props = { {"BaseField", {typeChecker.numberType}}, }; typeChecker.globalScope->exportedTypeBindings["BaseClass"] = TypeFun{{}, baseClassInstanceType}; - TypeId childClassInstanceType = arena.addType(ClassTypeVar{"ChildClass", {}, baseClassInstanceType, std::nullopt, {}, {}}); + TypeId childClassInstanceType = arena.addType(ClassTypeVar{"ChildClass", {}, baseClassInstanceType, std::nullopt, {}, {}, "Test"}); getMutable(childClassInstanceType)->props = { {"ChildField", {typeChecker.stringType}}, }; diff --git a/tests/Transpiler.test.cpp b/tests/Transpiler.test.cpp index 0c324cd0..b02a52b2 100644 --- a/tests/Transpiler.test.cpp +++ b/tests/Transpiler.test.cpp @@ -661,4 +661,21 @@ type t4 = false CHECK_EQ(code, transpile(code, {}, true).code); } +TEST_CASE_FIXTURE(Fixture, "transpile_array_types") +{ + std::string code = R"( +type t1 = {number} +type t2 = {[string]: number} + )"; + + CHECK_EQ(code, transpile(code, {}, true).code); +} + +TEST_CASE_FIXTURE(Fixture, "transpile_for_in_multiple_types") +{ + std::string code = "for k:string,v:boolean in next,{}do end"; + + CHECK_EQ(code, transpile(code, {}, true).code); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.classes.test.cpp b/tests/TypeInfer.classes.test.cpp index 8e3629e7..5a6e4032 100644 --- a/tests/TypeInfer.classes.test.cpp +++ b/tests/TypeInfer.classes.test.cpp @@ -19,13 +19,13 @@ struct ClassFixture : Fixture unfreeze(arena); - TypeId baseClassInstanceType = arena.addType(ClassTypeVar{"BaseClass", {}, nullopt, nullopt, {}, {}}); + TypeId baseClassInstanceType = arena.addType(ClassTypeVar{"BaseClass", {}, nullopt, nullopt, {}, {}, "Test"}); getMutable(baseClassInstanceType)->props = { {"BaseMethod", {makeFunction(arena, baseClassInstanceType, {numberType}, {})}}, {"BaseField", {numberType}}, }; - TypeId baseClassType = arena.addType(ClassTypeVar{"BaseClass", {}, nullopt, nullopt, {}, {}}); + TypeId baseClassType = arena.addType(ClassTypeVar{"BaseClass", {}, nullopt, nullopt, {}, {}, "Test"}); getMutable(baseClassType)->props = { {"StaticMethod", {makeFunction(arena, nullopt, {}, {numberType})}}, {"Clone", {makeFunction(arena, nullopt, {baseClassInstanceType}, {baseClassInstanceType})}}, @@ -34,39 +34,39 @@ struct ClassFixture : Fixture typeChecker.globalScope->exportedTypeBindings["BaseClass"] = TypeFun{{}, baseClassInstanceType}; addGlobalBinding(typeChecker, "BaseClass", baseClassType, "@test"); - TypeId childClassInstanceType = arena.addType(ClassTypeVar{"ChildClass", {}, baseClassInstanceType, nullopt, {}, {}}); + TypeId childClassInstanceType = arena.addType(ClassTypeVar{"ChildClass", {}, baseClassInstanceType, nullopt, {}, {}, "Test"}); getMutable(childClassInstanceType)->props = { {"Method", {makeFunction(arena, childClassInstanceType, {}, {typeChecker.stringType})}}, }; - TypeId childClassType = arena.addType(ClassTypeVar{"ChildClass", {}, baseClassType, nullopt, {}, {}}); + TypeId childClassType = arena.addType(ClassTypeVar{"ChildClass", {}, baseClassType, nullopt, {}, {}, "Test"}); getMutable(childClassType)->props = { {"New", {makeFunction(arena, nullopt, {}, {childClassInstanceType})}}, }; typeChecker.globalScope->exportedTypeBindings["ChildClass"] = TypeFun{{}, childClassInstanceType}; addGlobalBinding(typeChecker, "ChildClass", childClassType, "@test"); - TypeId grandChildInstanceType = arena.addType(ClassTypeVar{"GrandChild", {}, childClassInstanceType, nullopt, {}, {}}); + TypeId grandChildInstanceType = arena.addType(ClassTypeVar{"GrandChild", {}, childClassInstanceType, nullopt, {}, {}, "Test"}); getMutable(grandChildInstanceType)->props = { {"Method", {makeFunction(arena, grandChildInstanceType, {}, {typeChecker.stringType})}}, }; - TypeId grandChildType = arena.addType(ClassTypeVar{"GrandChild", {}, baseClassType, nullopt, {}, {}}); + TypeId grandChildType = arena.addType(ClassTypeVar{"GrandChild", {}, baseClassType, nullopt, {}, {}, "Test"}); getMutable(grandChildType)->props = { {"New", {makeFunction(arena, nullopt, {}, {grandChildInstanceType})}}, }; typeChecker.globalScope->exportedTypeBindings["GrandChild"] = TypeFun{{}, grandChildInstanceType}; addGlobalBinding(typeChecker, "GrandChild", childClassType, "@test"); - TypeId anotherChildInstanceType = arena.addType(ClassTypeVar{"AnotherChild", {}, baseClassInstanceType, nullopt, {}, {}}); + TypeId anotherChildInstanceType = arena.addType(ClassTypeVar{"AnotherChild", {}, baseClassInstanceType, nullopt, {}, {}, "Test"}); getMutable(anotherChildInstanceType)->props = { {"Method", {makeFunction(arena, anotherChildInstanceType, {}, {typeChecker.stringType})}}, }; - TypeId anotherChildType = arena.addType(ClassTypeVar{"AnotherChild", {}, baseClassType, nullopt, {}, {}}); + TypeId anotherChildType = arena.addType(ClassTypeVar{"AnotherChild", {}, baseClassType, nullopt, {}, {}, "Test"}); getMutable(anotherChildType)->props = { {"New", {makeFunction(arena, nullopt, {}, {anotherChildInstanceType})}}, }; @@ -75,13 +75,13 @@ struct ClassFixture : Fixture TypeId vector2MetaType = arena.addType(TableTypeVar{}); - TypeId vector2InstanceType = arena.addType(ClassTypeVar{"Vector2", {}, nullopt, vector2MetaType, {}, {}}); + TypeId vector2InstanceType = arena.addType(ClassTypeVar{"Vector2", {}, nullopt, vector2MetaType, {}, {}, "Test"}); getMutable(vector2InstanceType)->props = { {"X", {numberType}}, {"Y", {numberType}}, }; - TypeId vector2Type = arena.addType(ClassTypeVar{"Vector2", {}, nullopt, nullopt, {}, {}}); + TypeId vector2Type = arena.addType(ClassTypeVar{"Vector2", {}, nullopt, nullopt, {}, {}, "Test"}); getMutable(vector2Type)->props = { {"New", {makeFunction(arena, nullopt, {numberType, numberType}, {vector2InstanceType})}}, }; @@ -468,4 +468,18 @@ caused by: toString(result.errors[0])); } +TEST_CASE_FIXTURE(ClassFixture, "class_type_mismatch_with_name_conflict") +{ + ScopedFastFlag luauClassDefinitionModuleInError{"LuauClassDefinitionModuleInError", true}; + + CheckResult result = check(R"( +local i = ChildClass.New() +type ChildClass = { x: number } +local a: ChildClass = i + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ("Type 'ChildClass' from 'Test' could not be converted into 'ChildClass' from 'MainModule'", toString(result.errors[0])); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.definitions.test.cpp b/tests/TypeInfer.definitions.test.cpp index 898d8902..4545b8db 100644 --- a/tests/TypeInfer.definitions.test.cpp +++ b/tests/TypeInfer.definitions.test.cpp @@ -295,8 +295,6 @@ TEST_CASE_FIXTURE(Fixture, "documentation_symbols_dont_attach_to_persistent_type TEST_CASE_FIXTURE(Fixture, "single_class_type_identity_in_global_types") { - ScopedFastFlag luauCloneDeclaredGlobals{"LuauCloneDeclaredGlobals", true}; - loadDefinition(R"( declare class Cls end diff --git a/tests/TypeInfer.functions.test.cpp b/tests/TypeInfer.functions.test.cpp index 65993681..7cd7bec3 100644 --- a/tests/TypeInfer.functions.test.cpp +++ b/tests/TypeInfer.functions.test.cpp @@ -656,6 +656,11 @@ TEST_CASE_FIXTURE(Fixture, "toposort_doesnt_break_mutual_recursion") TEST_CASE_FIXTURE(Fixture, "check_function_before_lambda_that_uses_it") { + ScopedFastFlag sff[]{ + {"LuauReturnTypeInferenceInNonstrict", true}, + {"LuauLowerBoundsCalculation", true}, + }; + CheckResult result = check(R"( --!nonstrict @@ -664,7 +669,7 @@ TEST_CASE_FIXTURE(Fixture, "check_function_before_lambda_that_uses_it") end return function() - return f():andThen() + return f() end )"); @@ -791,14 +796,18 @@ TEST_CASE_FIXTURE(Fixture, "calling_function_with_incorrect_argument_type_yields TEST_CASE_FIXTURE(Fixture, "calling_function_with_anytypepack_doesnt_leak_free_types") { + ScopedFastFlag sff[]{ + {"LuauReturnTypeInferenceInNonstrict", true}, + {"LuauLowerBoundsCalculation", true}, + }; + CheckResult result = check(R"( --!nonstrict - function Test(a) + function Test(a): ...any return 1, "" end - local tab = {} table.insert(tab, Test(1)); )"); @@ -1616,4 +1625,19 @@ TEST_CASE_FIXTURE(Fixture, "occurs_check_failure_in_function_return_type") CHECK(nullptr != get(result.errors[0])); } +TEST_CASE_FIXTURE(Fixture, "weird_fail_to_unify_type_pack") +{ + ScopedFastFlag sff[]{ + {"LuauReturnTypeInferenceInNonstrict", true}, + {"LuauLowerBoundsCalculation", true}, + }; + + CheckResult result = check(R"( + local function f() return end + local g = function() return f() end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.modules.test.cpp b/tests/TypeInfer.modules.test.cpp index e5eeae31..fa1f519c 100644 --- a/tests/TypeInfer.modules.test.cpp +++ b/tests/TypeInfer.modules.test.cpp @@ -307,8 +307,6 @@ type Rename = typeof(x.x) TEST_CASE_FIXTURE(Fixture, "module_type_conflict") { - ScopedFastFlag luauTypeMismatchModuleName{"LuauTypeMismatchModuleName", true}; - fileResolver.source["game/A"] = R"( export type T = { x: number } return {} @@ -343,8 +341,6 @@ caused by: TEST_CASE_FIXTURE(Fixture, "module_type_conflict_instantiated") { - ScopedFastFlag luauTypeMismatchModuleName{"LuauTypeMismatchModuleName", true}; - fileResolver.source["game/A"] = R"( export type Wrap = { x: T } return {} diff --git a/tests/TypeInfer.provisional.test.cpp b/tests/TypeInfer.provisional.test.cpp index 6b3741fa..4b5075d9 100644 --- a/tests/TypeInfer.provisional.test.cpp +++ b/tests/TypeInfer.provisional.test.cpp @@ -584,20 +584,6 @@ TEST_CASE_FIXTURE(Fixture, "specialization_binds_with_prototypes_too_early") LUAU_REQUIRE_ERRORS(result); // Should not have any errors. } -TEST_CASE_FIXTURE(Fixture, "weird_fail_to_unify_type_pack") -{ - ScopedFastFlag sff[] = { - {"LuauLowerBoundsCalculation", false}, - }; - - CheckResult result = check(R"( - local function f() return end - local g = function() return f() end - )"); - - LUAU_REQUIRE_ERRORS(result); // Should not have any errors. -} - TEST_CASE_FIXTURE(Fixture, "weird_fail_to_unify_variadic_pack") { ScopedFastFlag sff[] = { diff --git a/tests/TypeInfer.refinements.test.cpp b/tests/TypeInfer.refinements.test.cpp index ce22bcb1..136ca00a 100644 --- a/tests/TypeInfer.refinements.test.cpp +++ b/tests/TypeInfer.refinements.test.cpp @@ -44,7 +44,7 @@ struct RefinementClassFixture : Fixture TypeArena& arena = typeChecker.globalTypes; unfreeze(arena); - TypeId vec3 = arena.addType(ClassTypeVar{"Vector3", {}, std::nullopt, std::nullopt, {}, nullptr}); + TypeId vec3 = arena.addType(ClassTypeVar{"Vector3", {}, std::nullopt, std::nullopt, {}, nullptr, "Test"}); getMutable(vec3)->props = { {"X", Property{typeChecker.numberType}}, {"Y", Property{typeChecker.numberType}}, @@ -52,7 +52,7 @@ struct RefinementClassFixture : Fixture }; normalize(vec3, arena, *typeChecker.iceHandler); - TypeId inst = arena.addType(ClassTypeVar{"Instance", {}, std::nullopt, std::nullopt, {}, nullptr}); + TypeId inst = arena.addType(ClassTypeVar{"Instance", {}, std::nullopt, std::nullopt, {}, nullptr, "Test"}); TypePackId isAParams = arena.addTypePack({inst, typeChecker.stringType}); TypePackId isARets = arena.addTypePack({typeChecker.booleanType}); @@ -66,9 +66,9 @@ struct RefinementClassFixture : Fixture }; normalize(inst, arena, *typeChecker.iceHandler); - TypeId folder = typeChecker.globalTypes.addType(ClassTypeVar{"Folder", {}, inst, std::nullopt, {}, nullptr}); + TypeId folder = typeChecker.globalTypes.addType(ClassTypeVar{"Folder", {}, inst, std::nullopt, {}, nullptr, "Test"}); normalize(folder, arena, *typeChecker.iceHandler); - TypeId part = typeChecker.globalTypes.addType(ClassTypeVar{"Part", {}, inst, std::nullopt, {}, nullptr}); + TypeId part = typeChecker.globalTypes.addType(ClassTypeVar{"Part", {}, inst, std::nullopt, {}, nullptr, "Test"}); getMutable(part)->props = { {"Position", Property{vec3}}, }; diff --git a/tests/TypeInfer.tables.test.cpp b/tests/TypeInfer.tables.test.cpp index ca1b8de7..2a727bb3 100644 --- a/tests/TypeInfer.tables.test.cpp +++ b/tests/TypeInfer.tables.test.cpp @@ -2086,7 +2086,6 @@ caused by: TEST_CASE_FIXTURE(Fixture, "error_detailed_indexer_key") { ScopedFastFlag luauTableSubtypingVariance2{"LuauTableSubtypingVariance2", true}; // Only for new path - ScopedFastFlag luauExtendedIndexerError{"LuauExtendedIndexerError", true}; CheckResult result = check(R"( type A = { [number]: string } @@ -2105,7 +2104,6 @@ caused by: TEST_CASE_FIXTURE(Fixture, "error_detailed_indexer_value") { ScopedFastFlag luauTableSubtypingVariance2{"LuauTableSubtypingVariance2", true}; // Only for new path - ScopedFastFlag luauExtendedIndexerError{"LuauExtendedIndexerError", true}; CheckResult result = check(R"( type A = { [number]: number } diff --git a/tests/TypeInfer.test.cpp b/tests/TypeInfer.test.cpp index 6abd96b9..a578b1cf 100644 --- a/tests/TypeInfer.test.cpp +++ b/tests/TypeInfer.test.cpp @@ -86,16 +86,21 @@ TEST_CASE_FIXTURE(Fixture, "infer_locals_via_assignment_from_its_call_site") TEST_CASE_FIXTURE(Fixture, "infer_in_nocheck_mode") { + ScopedFastFlag sff[]{ + {"LuauReturnTypeInferenceInNonstrict", true}, + {"LuauLowerBoundsCalculation", true}, + }; + CheckResult result = check(R"( --!nocheck function f(x) - return x + return 5 end -- we get type information even if there's type errors f(1, 2) )"); - CHECK_EQ("(any) -> (...any)", toString(requireType("f"))); + CHECK_EQ("(any) -> number", toString(requireType("f"))); LUAU_REQUIRE_NO_ERRORS(result); } @@ -363,6 +368,11 @@ TEST_CASE_FIXTURE(Fixture, "globals") TEST_CASE_FIXTURE(Fixture, "globals2") { + ScopedFastFlag sff[]{ + {"LuauReturnTypeInferenceInNonstrict", true}, + {"LuauLowerBoundsCalculation", true}, + }; + CheckResult result = check(R"( --!nonstrict foo = function() return 1 end @@ -373,9 +383,9 @@ TEST_CASE_FIXTURE(Fixture, "globals2") TypeMismatch* tm = get(result.errors[0]); REQUIRE(tm); - CHECK_EQ("() -> (...any)", toString(tm->wantedType)); + CHECK_EQ("() -> number", toString(tm->wantedType)); CHECK_EQ("string", toString(tm->givenType)); - CHECK_EQ("() -> (...any)", toString(requireType("foo"))); + CHECK_EQ("() -> number", toString(requireType("foo"))); } TEST_CASE_FIXTURE(Fixture, "globals_are_banned_in_strict_mode") diff --git a/tests/TypeVar.test.cpp b/tests/TypeVar.test.cpp index fd5f4dbc..d03bb03c 100644 --- a/tests/TypeVar.test.cpp +++ b/tests/TypeVar.test.cpp @@ -275,7 +275,7 @@ TEST_CASE("tagging_tables") TEST_CASE("tagging_classes") { - TypeVar base{ClassTypeVar{"Base", {}, std::nullopt, std::nullopt, {}, nullptr}}; + TypeVar base{ClassTypeVar{"Base", {}, std::nullopt, std::nullopt, {}, nullptr, "Test"}}; CHECK(!Luau::hasTag(&base, "foo")); Luau::attachTag(&base, "foo"); CHECK(Luau::hasTag(&base, "foo")); @@ -283,8 +283,8 @@ TEST_CASE("tagging_classes") TEST_CASE("tagging_subclasses") { - TypeVar base{ClassTypeVar{"Base", {}, std::nullopt, std::nullopt, {}, nullptr}}; - TypeVar derived{ClassTypeVar{"Derived", {}, &base, std::nullopt, {}, nullptr}}; + TypeVar base{ClassTypeVar{"Base", {}, std::nullopt, std::nullopt, {}, nullptr, "Test"}}; + TypeVar derived{ClassTypeVar{"Derived", {}, &base, std::nullopt, {}, nullptr, "Test"}}; CHECK(!Luau::hasTag(&base, "foo")); CHECK(!Luau::hasTag(&derived, "foo")); From 74c84815a0ca7f1799a800156833cfc44016f12f Mon Sep 17 00:00:00 2001 From: Alan Jeffrey <403333+asajeffrey@users.noreply.github.com> Date: Thu, 28 Apr 2022 15:00:55 -0500 Subject: [PATCH 228/399] Prototyping type normalizaton (#466) * Added type normalization --- prototyping/Luau/FunctionTypes.agda | 38 ++ prototyping/Luau/StrictMode.agda | 3 +- prototyping/Luau/Subtyping.agda | 2 +- prototyping/Luau/Type.agda | 24 +- prototyping/Luau/TypeCheck.agda | 3 +- prototyping/Luau/TypeNormalization.agda | 69 ++++ prototyping/Properties.agda | 3 + prototyping/Properties/DecSubtyping.agda | 70 ++++ prototyping/Properties/FunctionTypes.agda | 150 +++++++ prototyping/Properties/StrictMode.agda | 6 +- prototyping/Properties/Subtyping.agda | 331 ++++++++++----- prototyping/Properties/TypeCheck.agda | 3 +- prototyping/Properties/TypeNormalization.agda | 376 ++++++++++++++++++ 13 files changed, 940 insertions(+), 138 deletions(-) create mode 100644 prototyping/Luau/FunctionTypes.agda create mode 100644 prototyping/Luau/TypeNormalization.agda create mode 100644 prototyping/Properties/DecSubtyping.agda create mode 100644 prototyping/Properties/FunctionTypes.agda create mode 100644 prototyping/Properties/TypeNormalization.agda diff --git a/prototyping/Luau/FunctionTypes.agda b/prototyping/Luau/FunctionTypes.agda new file mode 100644 index 00000000..7607052b --- /dev/null +++ b/prototyping/Luau/FunctionTypes.agda @@ -0,0 +1,38 @@ +{-# OPTIONS --rewriting #-} + +open import FFI.Data.Either using (Either; Left; Right) +open import Luau.Type using (Type; nil; number; string; boolean; never; unknown; _⇒_; _∪_; _∩_) +open import Luau.TypeNormalization using (normalize) + +module Luau.FunctionTypes where + +-- The domain of a normalized type +srcⁿ : Type → Type +srcⁿ (S ⇒ T) = S +srcⁿ (S ∩ T) = srcⁿ S ∪ srcⁿ T +srcⁿ never = unknown +srcⁿ T = never + +-- To get the domain of a type, we normalize it first We need to do +-- this, since if we try to use it on non-normalized types, we get +-- +-- src(number ∩ string) = src(number) ∪ src(string) = never ∪ never +-- src(never) = unknown +-- +-- so src doesn't respect type equivalence. +src : Type → Type +src (S ⇒ T) = S +src T = srcⁿ(normalize T) + +-- The codomain of a type +tgt : Type → Type +tgt nil = never +tgt (S ⇒ T) = T +tgt never = never +tgt unknown = unknown +tgt number = never +tgt boolean = never +tgt string = never +tgt (S ∪ T) = (tgt S) ∪ (tgt T) +tgt (S ∩ T) = (tgt S) ∩ (tgt T) + diff --git a/prototyping/Luau/StrictMode.agda b/prototyping/Luau/StrictMode.agda index 1b028042..d3c0f153 100644 --- a/prototyping/Luau/StrictMode.agda +++ b/prototyping/Luau/StrictMode.agda @@ -5,7 +5,8 @@ module Luau.StrictMode where open import Agda.Builtin.Equality using (_≡_) open import FFI.Data.Maybe using (just; nothing) open import Luau.Syntax using (Expr; Stat; Block; BinaryOperator; yes; nil; addr; var; binexp; var_∈_; _⟨_⟩∈_; function_is_end; _$_; block_is_end; local_←_; _∙_; done; return; name; +; -; *; /; <; >; <=; >=; ··) -open import Luau.Type using (Type; nil; number; string; boolean; _⇒_; _∪_; _∩_; src; tgt) +open import Luau.FunctionTypes using (src; tgt) +open import Luau.Type using (Type; nil; number; string; boolean; _⇒_; _∪_; _∩_) open import Luau.Subtyping using (_≮:_) open import Luau.Heap using (Heap; function_is_end) renaming (_[_] to _[_]ᴴ) open import Luau.VarCtxt using (VarCtxt; ∅; _⋒_; _↦_; _⊕_↦_; _⊝_) renaming (_[_] to _[_]ⱽ) diff --git a/prototyping/Luau/Subtyping.agda b/prototyping/Luau/Subtyping.agda index 943f459b..624b6be4 100644 --- a/prototyping/Luau/Subtyping.agda +++ b/prototyping/Luau/Subtyping.agda @@ -25,7 +25,6 @@ data Language where function : ∀ {T U} → Language (T ⇒ U) function function-ok : ∀ {T U u} → (Language U u) → Language (T ⇒ U) (function-ok u) function-err : ∀ {T U t} → (¬Language T t) → Language (T ⇒ U) (function-err t) - scalar-function-err : ∀ {S t} → (Scalar S) → Language S (function-err t) left : ∀ {T U t} → Language T t → Language (T ∪ U) t right : ∀ {T U u} → Language U u → Language (T ∪ U) u _,_ : ∀ {T U t} → Language T t → Language U t → Language (T ∩ U) t @@ -36,6 +35,7 @@ data ¬Language where scalar-scalar : ∀ {S T} → (s : Scalar S) → (Scalar T) → (S ≢ T) → ¬Language T (scalar s) scalar-function : ∀ {S} → (Scalar S) → ¬Language S function scalar-function-ok : ∀ {S u} → (Scalar S) → ¬Language S (function-ok u) + scalar-function-err : ∀ {S t} → (Scalar S) → ¬Language S (function-err t) function-scalar : ∀ {S T U} (s : Scalar S) → ¬Language (T ⇒ U) (scalar s) function-ok : ∀ {T U u} → (¬Language U u) → ¬Language (T ⇒ U) (function-ok u) function-err : ∀ {T U t} → (Language T t) → ¬Language (T ⇒ U) (function-err t) diff --git a/prototyping/Luau/Type.agda b/prototyping/Luau/Type.agda index 59d1107f..1d0ec9e5 100644 --- a/prototyping/Luau/Type.agda +++ b/prototyping/Luau/Type.agda @@ -24,6 +24,8 @@ data Scalar : Type → Set where string : Scalar string nil : Scalar nil +skalar = number ∪ (string ∪ (nil ∪ boolean)) + lhs : Type → Type lhs (T ⇒ _) = T lhs (T ∪ _) = T @@ -146,28 +148,6 @@ just T ≡ᴹᵀ just U with T ≡ᵀ U (just T ≡ᴹᵀ just T) | yes refl = yes refl (just T ≡ᴹᵀ just U) | no p = no (λ q → p (just-inv q)) -src : Type → Type -src nil = never -src number = never -src boolean = never -src string = never -src (S ⇒ T) = S -src (S ∪ T) = (src S) ∩ (src T) -src (S ∩ T) = (src S) ∪ (src T) -src never = unknown -src unknown = never - -tgt : Type → Type -tgt nil = never -tgt (S ⇒ T) = T -tgt never = never -tgt unknown = unknown -tgt number = never -tgt boolean = never -tgt string = never -tgt (S ∪ T) = (tgt S) ∪ (tgt T) -tgt (S ∩ T) = (tgt S) ∩ (tgt T) - optional : Type → Type optional nil = nil optional (T ∪ nil) = (T ∪ nil) diff --git a/prototyping/Luau/TypeCheck.agda b/prototyping/Luau/TypeCheck.agda index cabd27a8..d4fabb90 100644 --- a/prototyping/Luau/TypeCheck.agda +++ b/prototyping/Luau/TypeCheck.agda @@ -7,8 +7,9 @@ open import FFI.Data.Maybe using (Maybe; just) open import Luau.Syntax using (Expr; Stat; Block; BinaryOperator; yes; nil; addr; number; bool; string; val; var; var_∈_; _⟨_⟩∈_; function_is_end; _$_; block_is_end; binexp; local_←_; _∙_; done; return; name; +; -; *; /; <; >; ==; ~=; <=; >=; ··) open import Luau.Var using (Var) open import Luau.Addr using (Addr) +open import Luau.FunctionTypes using (src; tgt) open import Luau.Heap using (Heap; Object; function_is_end) renaming (_[_] to _[_]ᴴ) -open import Luau.Type using (Type; nil; unknown; number; boolean; string; _⇒_; src; tgt) +open import Luau.Type using (Type; nil; unknown; number; boolean; string; _⇒_) open import Luau.VarCtxt using (VarCtxt; ∅; _⋒_; _↦_; _⊕_↦_; _⊝_) renaming (_[_] to _[_]ⱽ) open import FFI.Data.Vector using (Vector) open import FFI.Data.Maybe using (Maybe; just; nothing) diff --git a/prototyping/Luau/TypeNormalization.agda b/prototyping/Luau/TypeNormalization.agda new file mode 100644 index 00000000..341883ea --- /dev/null +++ b/prototyping/Luau/TypeNormalization.agda @@ -0,0 +1,69 @@ +module Luau.TypeNormalization where + +open import Luau.Type using (Type; nil; number; string; boolean; never; unknown; _⇒_; _∪_; _∩_) + +-- The top non-function type +¬function : Type +¬function = number ∪ (string ∪ (nil ∪ boolean)) + +-- Unions and intersections of normalized types +_∪ᶠ_ : Type → Type → Type +_∪ⁿˢ_ : Type → Type → Type +_∩ⁿˢ_ : Type → Type → Type +_∪ⁿ_ : Type → Type → Type +_∩ⁿ_ : Type → Type → Type + +-- Union of function types +(F₁ ∩ F₂) ∪ᶠ G = (F₁ ∪ᶠ G) ∩ (F₂ ∪ᶠ G) +F ∪ᶠ (G₁ ∩ G₂) = (F ∪ᶠ G₁) ∩ (F ∪ᶠ G₂) +(R ⇒ S) ∪ᶠ (T ⇒ U) = (R ∩ⁿ T) ⇒ (S ∪ⁿ U) +F ∪ᶠ G = F ∪ G + +-- Union of normalized types +S ∪ⁿ (T₁ ∪ T₂) = (S ∪ⁿ T₁) ∪ T₂ +S ∪ⁿ unknown = unknown +S ∪ⁿ never = S +unknown ∪ⁿ T = unknown +never ∪ⁿ T = T +(S₁ ∪ S₂) ∪ⁿ G = (S₁ ∪ⁿ G) ∪ S₂ +F ∪ⁿ G = F ∪ᶠ G + +-- Intersection of normalized types +S ∩ⁿ (T₁ ∪ T₂) = (S ∩ⁿ T₁) ∪ⁿˢ (S ∩ⁿˢ T₂) +S ∩ⁿ unknown = S +S ∩ⁿ never = never +(S₁ ∪ S₂) ∩ⁿ G = (S₁ ∩ⁿ G) +unknown ∩ⁿ G = G +never ∩ⁿ G = never +F ∩ⁿ G = F ∩ G + +-- Intersection of normalized types with a scalar +(S₁ ∪ nil) ∩ⁿˢ nil = nil +(S₁ ∪ boolean) ∩ⁿˢ boolean = boolean +(S₁ ∪ number) ∩ⁿˢ number = number +(S₁ ∪ string) ∩ⁿˢ string = string +(S₁ ∪ S₂) ∩ⁿˢ T = S₁ ∩ⁿˢ T +unknown ∩ⁿˢ T = T +F ∩ⁿˢ T = never + +-- Union of normalized types with an optional scalar +S ∪ⁿˢ never = S +unknown ∪ⁿˢ T = unknown +(S₁ ∪ nil) ∪ⁿˢ nil = S₁ ∪ nil +(S₁ ∪ boolean) ∪ⁿˢ boolean = S₁ ∪ boolean +(S₁ ∪ number) ∪ⁿˢ number = S₁ ∪ number +(S₁ ∪ string) ∪ⁿˢ string = S₁ ∪ string +(S₁ ∪ S₂) ∪ⁿˢ T = (S₁ ∪ⁿˢ T) ∪ S₂ +F ∪ⁿˢ T = F ∪ T + +-- Normalize! +normalize : Type → Type +normalize nil = never ∪ nil +normalize (S ⇒ T) = (normalize S ⇒ normalize T) +normalize never = never +normalize unknown = unknown +normalize boolean = never ∪ boolean +normalize number = never ∪ number +normalize string = never ∪ string +normalize (S ∪ T) = normalize S ∪ⁿ normalize T +normalize (S ∩ T) = normalize S ∩ⁿ normalize T diff --git a/prototyping/Properties.agda b/prototyping/Properties.agda index 5594812e..b696c0fa 100644 --- a/prototyping/Properties.agda +++ b/prototyping/Properties.agda @@ -4,10 +4,13 @@ module Properties where import Properties.Contradiction import Properties.Dec +import Properties.DecSubtyping import Properties.Equality import Properties.Functions +import Properties.FunctionTypes import Properties.Remember import Properties.Step import Properties.StrictMode import Properties.Subtyping import Properties.TypeCheck +import Properties.TypeNormalization diff --git a/prototyping/Properties/DecSubtyping.agda b/prototyping/Properties/DecSubtyping.agda new file mode 100644 index 00000000..332520a9 --- /dev/null +++ b/prototyping/Properties/DecSubtyping.agda @@ -0,0 +1,70 @@ +{-# OPTIONS --rewriting #-} + +module Properties.DecSubtyping where + +open import Agda.Builtin.Equality using (_≡_; refl) +open import FFI.Data.Either using (Either; Left; Right; mapLR; swapLR; cond) +open import Luau.FunctionTypes using (src; srcⁿ; tgt) +open import Luau.Subtyping using (_<:_; _≮:_; Tree; Language; ¬Language; witness; unknown; never; scalar; function; scalar-function; scalar-function-ok; scalar-function-err; scalar-scalar; function-scalar; function-ok; function-err; left; right; _,_) +open import Luau.Type using (Type; Scalar; nil; number; string; boolean; never; unknown; _⇒_; _∪_; _∩_) +open import Properties.Contradiction using (CONTRADICTION; ¬) +open import Properties.Functions using (_∘_) +open import Properties.Subtyping using (<:-refl; <:-trans; ≮:-trans-<:; <:-trans-≮:; <:-never; <:-unknown; <:-∪-left; <:-∪-right; <:-∪-lub; ≮:-∪-left; ≮:-∪-right; <:-∩-left; <:-∩-right; <:-∩-glb; ≮:-∩-left; ≮:-∩-right; dec-language; scalar-<:; <:-everything; <:-function; ≮:-function-left; ≮:-function-right) +open import Properties.TypeNormalization using (FunType; Normal; never; unknown; _∩_; _∪_; _⇒_; normal; <:-normalize; normalize-<:) +open import Properties.FunctionTypes using (fun-¬scalar; ¬fun-scalar; fun-function; src-unknown-≮:; tgt-never-≮:; src-tgtᶠ-<:) +open import Properties.Equality using (_≢_) + +-- Honest this terminates, since src and tgt reduce the depth of nested arrows +{-# TERMINATING #-} +dec-subtypingˢⁿ : ∀ {T U} → Scalar T → Normal U → Either (T ≮: U) (T <: U) +dec-subtypingᶠ : ∀ {T U} → FunType T → FunType U → Either (T ≮: U) (T <: U) +dec-subtypingᶠⁿ : ∀ {T U} → FunType T → Normal U → Either (T ≮: U) (T <: U) +dec-subtypingⁿ : ∀ {T U} → Normal T → Normal U → Either (T ≮: U) (T <: U) +dec-subtyping : ∀ T U → Either (T ≮: U) (T <: U) + +dec-subtypingˢⁿ T U with dec-language _ (scalar T) +dec-subtypingˢⁿ T U | Left p = Left (witness (scalar T) (scalar T) p) +dec-subtypingˢⁿ T U | Right p = Right (scalar-<: T p) + +dec-subtypingᶠ {T = T} _ (U ⇒ V) with dec-subtypingⁿ U (normal (src T)) | dec-subtypingⁿ (normal (tgt T)) V +dec-subtypingᶠ {T = T} _ (U ⇒ V) | Left p | q = Left (≮:-trans-<: (src-unknown-≮: (≮:-trans-<: p (<:-normalize (src T)))) (<:-function <:-refl <:-unknown)) +dec-subtypingᶠ {T = T} _ (U ⇒ V) | Right p | Left q = Left (≮:-trans-<: (tgt-never-≮: (<:-trans-≮: (normalize-<: (tgt T)) q)) (<:-trans (<:-function <:-never <:-refl) <:-∪-right)) +dec-subtypingᶠ T (U ⇒ V) | Right p | Right q = Right (src-tgtᶠ-<: T (<:-trans p (normalize-<: _)) (<:-trans (<:-normalize _) q)) + +dec-subtypingᶠ T (U ∩ V) with dec-subtypingᶠ T U | dec-subtypingᶠ T V +dec-subtypingᶠ T (U ∩ V) | Left p | q = Left (≮:-∩-left p) +dec-subtypingᶠ T (U ∩ V) | Right p | Left q = Left (≮:-∩-right q) +dec-subtypingᶠ T (U ∩ V) | Right p | Right q = Right (<:-∩-glb p q) + +dec-subtypingᶠⁿ T never = Left (witness function (fun-function T) never) +dec-subtypingᶠⁿ T unknown = Right <:-unknown +dec-subtypingᶠⁿ T (U ⇒ V) = dec-subtypingᶠ T (U ⇒ V) +dec-subtypingᶠⁿ T (U ∩ V) = dec-subtypingᶠ T (U ∩ V) +dec-subtypingᶠⁿ T (U ∪ V) with dec-subtypingᶠⁿ T U +dec-subtypingᶠⁿ T (U ∪ V) | Left (witness t p q) = Left (witness t p (q , ¬fun-scalar V T p)) +dec-subtypingᶠⁿ T (U ∪ V) | Right p = Right (<:-trans p <:-∪-left) + +dec-subtypingⁿ never U = Right <:-never +dec-subtypingⁿ unknown unknown = Right <:-refl +dec-subtypingⁿ unknown U with dec-subtypingᶠⁿ (never ⇒ unknown) U +dec-subtypingⁿ unknown U | Left p = Left (<:-trans-≮: <:-unknown p) +dec-subtypingⁿ unknown U | Right p₁ with dec-subtypingˢⁿ number U +dec-subtypingⁿ unknown U | Right p₁ | Left p = Left (<:-trans-≮: <:-unknown p) +dec-subtypingⁿ unknown U | Right p₁ | Right p₂ with dec-subtypingˢⁿ string U +dec-subtypingⁿ unknown U | Right p₁ | Right p₂ | Left p = Left (<:-trans-≮: <:-unknown p) +dec-subtypingⁿ unknown U | Right p₁ | Right p₂ | Right p₃ with dec-subtypingˢⁿ nil U +dec-subtypingⁿ unknown U | Right p₁ | Right p₂ | Right p₃ | Left p = Left (<:-trans-≮: <:-unknown p) +dec-subtypingⁿ unknown U | Right p₁ | Right p₂ | Right p₃ | Right p₄ with dec-subtypingˢⁿ boolean U +dec-subtypingⁿ unknown U | Right p₁ | Right p₂ | Right p₃ | Right p₄ | Left p = Left (<:-trans-≮: <:-unknown p) +dec-subtypingⁿ unknown U | Right p₁ | Right p₂ | Right p₃ | Right p₄ | Right p₅ = Right (<:-trans <:-everything (<:-∪-lub p₁ (<:-∪-lub p₂ (<:-∪-lub p₃ (<:-∪-lub p₄ p₅))))) +dec-subtypingⁿ (S ⇒ T) U = dec-subtypingᶠⁿ (S ⇒ T) U +dec-subtypingⁿ (S ∩ T) U = dec-subtypingᶠⁿ (S ∩ T) U +dec-subtypingⁿ (S ∪ T) U with dec-subtypingⁿ S U | dec-subtypingˢⁿ T U +dec-subtypingⁿ (S ∪ T) U | Left p | q = Left (≮:-∪-left p) +dec-subtypingⁿ (S ∪ T) U | Right p | Left q = Left (≮:-∪-right q) +dec-subtypingⁿ (S ∪ T) U | Right p | Right q = Right (<:-∪-lub p q) + +dec-subtyping T U with dec-subtypingⁿ (normal T) (normal U) +dec-subtyping T U | Left p = Left (<:-trans-≮: (normalize-<: T) (≮:-trans-<: p (<:-normalize U))) +dec-subtyping T U | Right p = Right (<:-trans (<:-normalize T) (<:-trans p (normalize-<: U))) + diff --git a/prototyping/Properties/FunctionTypes.agda b/prototyping/Properties/FunctionTypes.agda new file mode 100644 index 00000000..514477f1 --- /dev/null +++ b/prototyping/Properties/FunctionTypes.agda @@ -0,0 +1,150 @@ +{-# OPTIONS --rewriting #-} + +module Properties.FunctionTypes where + +open import FFI.Data.Either using (Either; Left; Right; mapLR; swapLR; cond) +open import Luau.FunctionTypes using (srcⁿ; src; tgt) +open import Luau.Subtyping using (_<:_; _≮:_; Tree; Language; ¬Language; witness; unknown; never; scalar; function; scalar-function; scalar-function-ok; scalar-function-err; scalar-scalar; function-scalar; function-ok; function-err; left; right; _,_) +open import Luau.Type using (Type; Scalar; nil; number; string; boolean; never; unknown; _⇒_; _∪_; _∩_; skalar) +open import Properties.Contradiction using (CONTRADICTION; ¬; ⊥) +open import Properties.Functions using (_∘_) +open import Properties.Subtyping using (<:-refl; ≮:-refl; <:-trans-≮:; skalar-scalar; <:-impl-⊇; skalar-function-ok; language-comp) +open import Properties.TypeNormalization using (FunType; Normal; never; unknown; _∩_; _∪_; _⇒_; normal; <:-normalize; normalize-<:) + +-- Properties of src +function-err-srcⁿ : ∀ {T t} → (FunType T) → (¬Language (srcⁿ T) t) → Language T (function-err t) +function-err-srcⁿ (S ⇒ T) p = function-err p +function-err-srcⁿ (S ∩ T) (p₁ , p₂) = (function-err-srcⁿ S p₁ , function-err-srcⁿ T p₂) + +¬function-err-srcᶠ : ∀ {T t} → (FunType T) → (Language (srcⁿ T) t) → ¬Language T (function-err t) +¬function-err-srcᶠ (S ⇒ T) p = function-err p +¬function-err-srcᶠ (S ∩ T) (left p) = left (¬function-err-srcᶠ S p) +¬function-err-srcᶠ (S ∩ T) (right p) = right (¬function-err-srcᶠ T p) + +¬function-err-srcⁿ : ∀ {T t} → (Normal T) → (Language (srcⁿ T) t) → ¬Language T (function-err t) +¬function-err-srcⁿ never p = never +¬function-err-srcⁿ unknown (scalar ()) +¬function-err-srcⁿ (S ⇒ T) p = function-err p +¬function-err-srcⁿ (S ∩ T) (left p) = left (¬function-err-srcᶠ S p) +¬function-err-srcⁿ (S ∩ T) (right p) = right (¬function-err-srcᶠ T p) +¬function-err-srcⁿ (S ∪ T) (scalar ()) + +¬function-err-src : ∀ {T t} → (Language (src T) t) → ¬Language T (function-err t) +¬function-err-src {T = S ⇒ T} p = function-err p +¬function-err-src {T = nil} p = scalar-function-err nil +¬function-err-src {T = never} p = never +¬function-err-src {T = unknown} (scalar ()) +¬function-err-src {T = boolean} p = scalar-function-err boolean +¬function-err-src {T = number} p = scalar-function-err number +¬function-err-src {T = string} p = scalar-function-err string +¬function-err-src {T = S ∪ T} p = <:-impl-⊇ (<:-normalize (S ∪ T)) _ (¬function-err-srcⁿ (normal (S ∪ T)) p) +¬function-err-src {T = S ∩ T} p = <:-impl-⊇ (<:-normalize (S ∩ T)) _ (¬function-err-srcⁿ (normal (S ∩ T)) p) + +src-¬function-errᶠ : ∀ {T t} → (FunType T) → Language T (function-err t) → (¬Language (srcⁿ T) t) +src-¬function-errᶠ (S ⇒ T) (function-err p) = p +src-¬function-errᶠ (S ∩ T) (p₁ , p₂) = (src-¬function-errᶠ S p₁ , src-¬function-errᶠ T p₂) + +src-¬function-errⁿ : ∀ {T t} → (Normal T) → Language T (function-err t) → (¬Language (srcⁿ T) t) +src-¬function-errⁿ unknown p = never +src-¬function-errⁿ (S ⇒ T) (function-err p) = p +src-¬function-errⁿ (S ∩ T) (p₁ , p₂) = (src-¬function-errᶠ S p₁ , src-¬function-errᶠ T p₂) +src-¬function-errⁿ (S ∪ T) p = never + +src-¬function-err : ∀ {T t} → Language T (function-err t) → (¬Language (src T) t) +src-¬function-err {T = S ⇒ T} (function-err p) = p +src-¬function-err {T = unknown} p = never +src-¬function-err {T = S ∪ T} p = src-¬function-errⁿ (normal (S ∪ T)) (<:-normalize (S ∪ T) _ p) +src-¬function-err {T = S ∩ T} p = src-¬function-errⁿ (normal (S ∩ T)) (<:-normalize (S ∩ T) _ p) + +fun-¬scalar : ∀ {S T} (s : Scalar S) → FunType T → ¬Language T (scalar s) +fun-¬scalar s (S ⇒ T) = function-scalar s +fun-¬scalar s (S ∩ T) = left (fun-¬scalar s S) + +¬fun-scalar : ∀ {S T t} (s : Scalar S) → FunType T → Language T t → ¬Language S t +¬fun-scalar s (S ⇒ T) function = scalar-function s +¬fun-scalar s (S ⇒ T) (function-ok p) = scalar-function-ok s +¬fun-scalar s (S ⇒ T) (function-err p) = scalar-function-err s +¬fun-scalar s (S ∩ T) (p₁ , p₂) = ¬fun-scalar s T p₂ + +fun-function : ∀ {T} → FunType T → Language T function +fun-function (S ⇒ T) = function +fun-function (S ∩ T) = (fun-function S , fun-function T) + +srcⁿ-¬scalar : ∀ {S T t} (s : Scalar S) → Normal T → Language T (scalar s) → (¬Language (srcⁿ T) t) +srcⁿ-¬scalar s never (scalar ()) +srcⁿ-¬scalar s unknown p = never +srcⁿ-¬scalar s (S ⇒ T) (scalar ()) +srcⁿ-¬scalar s (S ∩ T) (p₁ , p₂) = CONTRADICTION (language-comp (scalar s) (fun-¬scalar s S) p₁) +srcⁿ-¬scalar s (S ∪ T) p = never + +src-¬scalar : ∀ {S T t} (s : Scalar S) → Language T (scalar s) → (¬Language (src T) t) +src-¬scalar {T = nil} s p = never +src-¬scalar {T = T ⇒ U} s (scalar ()) +src-¬scalar {T = never} s (scalar ()) +src-¬scalar {T = unknown} s p = never +src-¬scalar {T = boolean} s p = never +src-¬scalar {T = number} s p = never +src-¬scalar {T = string} s p = never +src-¬scalar {T = T ∪ U} s p = srcⁿ-¬scalar s (normal (T ∪ U)) (<:-normalize (T ∪ U) (scalar s) p) +src-¬scalar {T = T ∩ U} s p = srcⁿ-¬scalar s (normal (T ∩ U)) (<:-normalize (T ∩ U) (scalar s) p) + +srcⁿ-unknown-≮: : ∀ {T U} → (Normal U) → (T ≮: srcⁿ U) → (U ≮: (T ⇒ unknown)) +srcⁿ-unknown-≮: never (witness t p q) = CONTRADICTION (language-comp t q unknown) +srcⁿ-unknown-≮: unknown (witness t p q) = witness (function-err t) unknown (function-err p) +srcⁿ-unknown-≮: (U ⇒ V) (witness t p q) = witness (function-err t) (function-err q) (function-err p) +srcⁿ-unknown-≮: (U ∩ V) (witness t p q) = witness (function-err t) (function-err-srcⁿ (U ∩ V) q) (function-err p) +srcⁿ-unknown-≮: (U ∪ V) (witness t p q) = witness (scalar V) (right (scalar V)) (function-scalar V) + +src-unknown-≮: : ∀ {T U} → (T ≮: src U) → (U ≮: (T ⇒ unknown)) +src-unknown-≮: {U = nil} (witness t p q) = witness (scalar nil) (scalar nil) (function-scalar nil) +src-unknown-≮: {U = T ⇒ U} (witness t p q) = witness (function-err t) (function-err q) (function-err p) +src-unknown-≮: {U = never} (witness t p q) = CONTRADICTION (language-comp t q unknown) +src-unknown-≮: {U = unknown} (witness t p q) = witness (function-err t) unknown (function-err p) +src-unknown-≮: {U = boolean} (witness t p q) = witness (scalar boolean) (scalar boolean) (function-scalar boolean) +src-unknown-≮: {U = number} (witness t p q) = witness (scalar number) (scalar number) (function-scalar number) +src-unknown-≮: {U = string} (witness t p q) = witness (scalar string) (scalar string) (function-scalar string) +src-unknown-≮: {U = T ∪ U} p = <:-trans-≮: (normalize-<: (T ∪ U)) (srcⁿ-unknown-≮: (normal (T ∪ U)) p) +src-unknown-≮: {U = T ∩ U} p = <:-trans-≮: (normalize-<: (T ∩ U)) (srcⁿ-unknown-≮: (normal (T ∩ U)) p) + +unknown-src-≮: : ∀ {S T U} → (U ≮: S) → (T ≮: (U ⇒ unknown)) → (U ≮: src T) +unknown-src-≮: (witness t x x₁) (witness (scalar s) p (function-scalar s)) = witness t x (src-¬scalar s p) +unknown-src-≮: r (witness (function-ok (scalar s)) p (function-ok (scalar-scalar s () q))) +unknown-src-≮: r (witness (function-ok (function-ok _)) p (function-ok (scalar-function-ok ()))) +unknown-src-≮: r (witness (function-err t) p (function-err q)) = witness t q (src-¬function-err p) + +-- Properties of tgt +tgt-function-ok : ∀ {T t} → (Language (tgt T) t) → Language T (function-ok t) +tgt-function-ok {T = nil} (scalar ()) +tgt-function-ok {T = T₁ ⇒ T₂} p = function-ok p +tgt-function-ok {T = never} (scalar ()) +tgt-function-ok {T = unknown} p = unknown +tgt-function-ok {T = boolean} (scalar ()) +tgt-function-ok {T = number} (scalar ()) +tgt-function-ok {T = string} (scalar ()) +tgt-function-ok {T = T₁ ∪ T₂} (left p) = left (tgt-function-ok p) +tgt-function-ok {T = T₁ ∪ T₂} (right p) = right (tgt-function-ok p) +tgt-function-ok {T = T₁ ∩ T₂} (p₁ , p₂) = (tgt-function-ok p₁ , tgt-function-ok p₂) + +function-ok-tgt : ∀ {T t} → Language T (function-ok t) → (Language (tgt T) t) +function-ok-tgt (function-ok p) = p +function-ok-tgt (left p) = left (function-ok-tgt p) +function-ok-tgt (right p) = right (function-ok-tgt p) +function-ok-tgt (p₁ , p₂) = (function-ok-tgt p₁ , function-ok-tgt p₂) +function-ok-tgt unknown = unknown + +tgt-never-≮: : ∀ {T U} → (tgt T ≮: U) → (T ≮: (skalar ∪ (never ⇒ U))) +tgt-never-≮: (witness t p q) = witness (function-ok t) (tgt-function-ok p) (skalar-function-ok , function-ok q) + +never-tgt-≮: : ∀ {T U} → (T ≮: (skalar ∪ (never ⇒ U))) → (tgt T ≮: U) +never-tgt-≮: (witness (scalar s) p (q₁ , q₂)) = CONTRADICTION (≮:-refl (witness (scalar s) (skalar-scalar s) q₁)) +never-tgt-≮: (witness function p (q₁ , scalar-function ())) +never-tgt-≮: (witness (function-ok t) p (q₁ , function-ok q₂)) = witness t (function-ok-tgt p) q₂ +never-tgt-≮: (witness (function-err (scalar s)) p (q₁ , function-err (scalar ()))) + +src-tgtᶠ-<: : ∀ {T U V} → (FunType T) → (U <: src T) → (tgt T <: V) → (T <: (U ⇒ V)) +src-tgtᶠ-<: T p q (scalar s) r = CONTRADICTION (language-comp (scalar s) (fun-¬scalar s T) r) +src-tgtᶠ-<: T p q function r = function +src-tgtᶠ-<: T p q (function-ok s) r = function-ok (q s (function-ok-tgt r)) +src-tgtᶠ-<: T p q (function-err s) r = function-err (<:-impl-⊇ p s (src-¬function-err r)) + + diff --git a/prototyping/Properties/StrictMode.agda b/prototyping/Properties/StrictMode.agda index fd2cf2f2..69e9131c 100644 --- a/prototyping/Properties/StrictMode.agda +++ b/prototyping/Properties/StrictMode.agda @@ -11,7 +11,8 @@ open import Luau.StrictMode using (Warningᴱ; Warningᴮ; Warningᴼ; Warning open import Luau.Substitution using (_[_/_]ᴮ; _[_/_]ᴱ; _[_/_]ᴮunless_; var_[_/_]ᴱwhenever_) open import Luau.Subtyping using (_≮:_; witness; unknown; never; scalar; function; scalar-function; scalar-function-ok; scalar-function-err; scalar-scalar; function-scalar; function-ok; function-err; left; right; _,_; Tree; Language; ¬Language) open import Luau.Syntax using (Expr; yes; var; val; var_∈_; _⟨_⟩∈_; _$_; addr; number; bool; string; binexp; nil; function_is_end; block_is_end; done; return; local_←_; _∙_; fun; arg; name; ==; ~=) -open import Luau.Type using (Type; nil; number; boolean; string; _⇒_; never; unknown; _∩_; _∪_; src; tgt; _≡ᵀ_; _≡ᴹᵀ_) +open import Luau.FunctionTypes using (src; tgt) +open import Luau.Type using (Type; nil; number; boolean; string; _⇒_; never; unknown; _∩_; _∪_; _≡ᵀ_; _≡ᴹᵀ_) open import Luau.TypeCheck using (_⊢ᴮ_∈_; _⊢ᴱ_∈_; _⊢ᴴᴮ_▷_∈_; _⊢ᴴᴱ_▷_∈_; nil; var; addr; app; function; block; done; return; local; orUnknown; srcBinOp; tgtBinOp) open import Luau.Var using (_≡ⱽ_) open import Luau.Addr using (_≡ᴬ_) @@ -22,7 +23,8 @@ open import Properties.Equality using (_≢_; sym; cong; trans; subst₁) open import Properties.Dec using (Dec; yes; no) open import Properties.Contradiction using (CONTRADICTION; ¬) open import Properties.Functions using (_∘_) -open import Properties.Subtyping using (unknown-≮:; ≡-trans-≮:; ≮:-trans-≡; never-tgt-≮:; tgt-never-≮:; src-unknown-≮:; unknown-src-≮:; ≮:-trans; ≮:-refl; scalar-≢-impl-≮:; function-≮:-scalar; scalar-≮:-function; function-≮:-never; unknown-≮:-scalar; scalar-≮:-never; unknown-≮:-never) +open import Properties.FunctionTypes using (never-tgt-≮:; tgt-never-≮:; src-unknown-≮:; unknown-src-≮:) +open import Properties.Subtyping using (unknown-≮:; ≡-trans-≮:; ≮:-trans-≡; ≮:-trans; ≮:-refl; scalar-≢-impl-≮:; function-≮:-scalar; scalar-≮:-function; function-≮:-never; unknown-≮:-scalar; scalar-≮:-never; unknown-≮:-never) open import Properties.TypeCheck using (typeOfᴼ; typeOfᴹᴼ; typeOfⱽ; typeOfᴱ; typeOfᴮ; typeCheckᴱ; typeCheckᴮ; typeCheckᴼ; typeCheckᴴ) open import Luau.OpSem using (_⟦_⟧_⟶_; _⊢_⟶*_⊣_; _⊢_⟶ᴮ_⊣_; _⊢_⟶ᴱ_⊣_; app₁; app₂; function; beta; return; block; done; local; subst; binOp₀; binOp₁; binOp₂; refl; step; +; -; *; /; <; >; ==; ~=; <=; >=; ··) open import Luau.RuntimeError using (BinOpError; RuntimeErrorᴱ; RuntimeErrorᴮ; FunctionMismatch; BinOpMismatch₁; BinOpMismatch₂; UnboundVariable; SEGV; app₁; app₂; bin₁; bin₂; block; local; return; +; -; *; /; <; >; <=; >=; ··) diff --git a/prototyping/Properties/Subtyping.agda b/prototyping/Properties/Subtyping.agda index b713eaf7..34e6691f 100644 --- a/prototyping/Properties/Subtyping.agda +++ b/prototyping/Properties/Subtyping.agda @@ -4,9 +4,10 @@ module Properties.Subtyping where open import Agda.Builtin.Equality using (_≡_; refl) open import FFI.Data.Either using (Either; Left; Right; mapLR; swapLR; cond) +open import FFI.Data.Maybe using (Maybe; just; nothing) open import Luau.Subtyping using (_<:_; _≮:_; Tree; Language; ¬Language; witness; unknown; never; scalar; function; scalar-function; scalar-function-ok; scalar-function-err; scalar-scalar; function-scalar; function-ok; function-err; left; right; _,_) -open import Luau.Type using (Type; Scalar; nil; number; string; boolean; never; unknown; _⇒_; _∪_; _∩_; src; tgt) -open import Properties.Contradiction using (CONTRADICTION; ¬) +open import Luau.Type using (Type; Scalar; nil; number; string; boolean; never; unknown; _⇒_; _∪_; _∩_; skalar) +open import Properties.Contradiction using (CONTRADICTION; ¬; ⊥) open import Properties.Equality using (_≢_) open import Properties.Functions using (_∘_) open import Properties.Product using (_×_; _,_) @@ -19,28 +20,28 @@ dec-language nil (scalar string) = Left (scalar-scalar string nil (λ ())) dec-language nil (scalar nil) = Right (scalar nil) dec-language nil function = Left (scalar-function nil) dec-language nil (function-ok t) = Left (scalar-function-ok nil) -dec-language nil (function-err t) = Right (scalar-function-err nil) +dec-language nil (function-err t) = Left (scalar-function-err nil) dec-language boolean (scalar number) = Left (scalar-scalar number boolean (λ ())) dec-language boolean (scalar boolean) = Right (scalar boolean) dec-language boolean (scalar string) = Left (scalar-scalar string boolean (λ ())) dec-language boolean (scalar nil) = Left (scalar-scalar nil boolean (λ ())) dec-language boolean function = Left (scalar-function boolean) dec-language boolean (function-ok t) = Left (scalar-function-ok boolean) -dec-language boolean (function-err t) = Right (scalar-function-err boolean) +dec-language boolean (function-err t) = Left (scalar-function-err boolean) dec-language number (scalar number) = Right (scalar number) dec-language number (scalar boolean) = Left (scalar-scalar boolean number (λ ())) dec-language number (scalar string) = Left (scalar-scalar string number (λ ())) dec-language number (scalar nil) = Left (scalar-scalar nil number (λ ())) dec-language number function = Left (scalar-function number) dec-language number (function-ok t) = Left (scalar-function-ok number) -dec-language number (function-err t) = Right (scalar-function-err number) +dec-language number (function-err t) = Left (scalar-function-err number) dec-language string (scalar number) = Left (scalar-scalar number string (λ ())) dec-language string (scalar boolean) = Left (scalar-scalar boolean string (λ ())) dec-language string (scalar string) = Right (scalar string) dec-language string (scalar nil) = Left (scalar-scalar nil string (λ ())) dec-language string function = Left (scalar-function string) dec-language string (function-ok t) = Left (scalar-function-ok string) -dec-language string (function-err t) = Right (scalar-function-err string) +dec-language string (function-err t) = Left (scalar-function-err string) dec-language (T₁ ⇒ T₂) (scalar s) = Left (function-scalar s) dec-language (T₁ ⇒ T₂) function = Right function dec-language (T₁ ⇒ T₂) (function-ok t) = mapLR function-ok function-ok (dec-language T₂ t) @@ -73,6 +74,11 @@ language-comp (function-err t) (function-err p) (function-err q) = language-comp <:-impl-¬≮: : ∀ {T U} → (T <: U) → ¬(T ≮: U) <:-impl-¬≮: p (witness t q r) = language-comp t r (p t q) +<:-impl-⊇ : ∀ {T U} → (T <: U) → ∀ t → ¬Language U t → ¬Language T t +<:-impl-⊇ {T} p t q with dec-language T t +<:-impl-⊇ {_} p t q | Left r = r +<:-impl-⊇ {_} p t q | Right r = CONTRADICTION (language-comp t q (p t r)) + -- reflexivity ≮:-refl : ∀ {T} → ¬(T ≮: T) ≮:-refl (witness t p q) = language-comp t q p @@ -91,10 +97,162 @@ language-comp (function-err t) (function-err p) (function-err q) = language-comp ≮:-trans {T = T} (witness t p q) = mapLR (witness t p) (λ z → witness t z q) (dec-language T t) <:-trans : ∀ {S T U} → (S <: T) → (T <: U) → (S <: U) -<:-trans p q = ¬≮:-impl-<: (cond (<:-impl-¬≮: p) (<:-impl-¬≮: q) ∘ ≮:-trans) +<:-trans p q t r = q t (p t r) + +<:-trans-≮: : ∀ {S T U} → (S <: T) → (S ≮: U) → (T ≮: U) +<:-trans-≮: p (witness t q r) = witness t (p t q) r + +≮:-trans-<: : ∀ {S T U} → (S ≮: U) → (T <: U) → (S ≮: T) +≮:-trans-<: (witness t p q) r = witness t p (<:-impl-⊇ r t q) + +-- Properties of union + +<:-union : ∀ {R S T U} → (R <: T) → (S <: U) → ((R ∪ S) <: (T ∪ U)) +<:-union p q t (left r) = left (p t r) +<:-union p q t (right r) = right (q t r) + +<:-∪-left : ∀ {S T} → S <: (S ∪ T) +<:-∪-left t p = left p + +<:-∪-right : ∀ {S T} → T <: (S ∪ T) +<:-∪-right t p = right p + +<:-∪-lub : ∀ {S T U} → (S <: U) → (T <: U) → ((S ∪ T) <: U) +<:-∪-lub p q t (left r) = p t r +<:-∪-lub p q t (right r) = q t r + +<:-∪-symm : ∀ {T U} → (T ∪ U) <: (U ∪ T) +<:-∪-symm t (left p) = right p +<:-∪-symm t (right p) = left p + +<:-∪-assocl : ∀ {S T U} → (S ∪ (T ∪ U)) <: ((S ∪ T) ∪ U) +<:-∪-assocl t (left p) = left (left p) +<:-∪-assocl t (right (left p)) = left (right p) +<:-∪-assocl t (right (right p)) = right p + +<:-∪-assocr : ∀ {S T U} → ((S ∪ T) ∪ U) <: (S ∪ (T ∪ U)) +<:-∪-assocr t (left (left p)) = left p +<:-∪-assocr t (left (right p)) = right (left p) +<:-∪-assocr t (right p) = right (right p) + +≮:-∪-left : ∀ {S T U} → (S ≮: U) → ((S ∪ T) ≮: U) +≮:-∪-left (witness t p q) = witness t (left p) q + +≮:-∪-right : ∀ {S T U} → (T ≮: U) → ((S ∪ T) ≮: U) +≮:-∪-right (witness t p q) = witness t (right p) q + +-- Properties of intersection + +<:-intersect : ∀ {R S T U} → (R <: T) → (S <: U) → ((R ∩ S) <: (T ∩ U)) +<:-intersect p q t (r₁ , r₂) = (p t r₁ , q t r₂) + +<:-∩-left : ∀ {S T} → (S ∩ T) <: S +<:-∩-left t (p , _) = p + +<:-∩-right : ∀ {S T} → (S ∩ T) <: T +<:-∩-right t (_ , p) = p + +<:-∩-glb : ∀ {S T U} → (S <: T) → (S <: U) → (S <: (T ∩ U)) +<:-∩-glb p q t r = (p t r , q t r) + +<:-∩-symm : ∀ {T U} → (T ∩ U) <: (U ∩ T) +<:-∩-symm t (p₁ , p₂) = (p₂ , p₁) + +≮:-∩-left : ∀ {S T U} → (S ≮: T) → (S ≮: (T ∩ U)) +≮:-∩-left (witness t p q) = witness t p (left q) + +≮:-∩-right : ∀ {S T U} → (S ≮: U) → (S ≮: (T ∩ U)) +≮:-∩-right (witness t p q) = witness t p (right q) + +-- Distribution properties +<:-∩-distl-∪ : ∀ {S T U} → (S ∩ (T ∪ U)) <: ((S ∩ T) ∪ (S ∩ U)) +<:-∩-distl-∪ t (p₁ , left p₂) = left (p₁ , p₂) +<:-∩-distl-∪ t (p₁ , right p₂) = right (p₁ , p₂) + +∩-distl-∪-<: : ∀ {S T U} → ((S ∩ T) ∪ (S ∩ U)) <: (S ∩ (T ∪ U)) +∩-distl-∪-<: t (left (p₁ , p₂)) = (p₁ , left p₂) +∩-distl-∪-<: t (right (p₁ , p₂)) = (p₁ , right p₂) + +<:-∩-distr-∪ : ∀ {S T U} → ((S ∪ T) ∩ U) <: ((S ∩ U) ∪ (T ∩ U)) +<:-∩-distr-∪ t (left p₁ , p₂) = left (p₁ , p₂) +<:-∩-distr-∪ t (right p₁ , p₂) = right (p₁ , p₂) + +∩-distr-∪-<: : ∀ {S T U} → ((S ∩ U) ∪ (T ∩ U)) <: ((S ∪ T) ∩ U) +∩-distr-∪-<: t (left (p₁ , p₂)) = (left p₁ , p₂) +∩-distr-∪-<: t (right (p₁ , p₂)) = (right p₁ , p₂) + +<:-∪-distl-∩ : ∀ {S T U} → (S ∪ (T ∩ U)) <: ((S ∪ T) ∩ (S ∪ U)) +<:-∪-distl-∩ t (left p) = (left p , left p) +<:-∪-distl-∩ t (right (p₁ , p₂)) = (right p₁ , right p₂) + +∪-distl-∩-<: : ∀ {S T U} → ((S ∪ T) ∩ (S ∪ U)) <: (S ∪ (T ∩ U)) +∪-distl-∩-<: t (left p₁ , p₂) = left p₁ +∪-distl-∩-<: t (right p₁ , left p₂) = left p₂ +∪-distl-∩-<: t (right p₁ , right p₂) = right (p₁ , p₂) + +<:-∪-distr-∩ : ∀ {S T U} → ((S ∩ T) ∪ U) <: ((S ∪ U) ∩ (T ∪ U)) +<:-∪-distr-∩ t (left (p₁ , p₂)) = left p₁ , left p₂ +<:-∪-distr-∩ t (right p) = (right p , right p) + +∪-distr-∩-<: : ∀ {S T U} → ((S ∪ U) ∩ (T ∪ U)) <: ((S ∩ T) ∪ U) +∪-distr-∩-<: t (left p₁ , left p₂) = left (p₁ , p₂) +∪-distr-∩-<: t (left p₁ , right p₂) = right p₂ +∪-distr-∩-<: t (right p₁ , p₂) = right p₁ + +-- Properties of functions +<:-function : ∀ {R S T U} → (R <: S) → (T <: U) → (S ⇒ T) <: (R ⇒ U) +<:-function p q function function = function +<:-function p q (function-ok t) (function-ok r) = function-ok (q t r) +<:-function p q (function-err s) (function-err r) = function-err (<:-impl-⊇ p s r) + +<:-function-∩-∪ : ∀ {R S T U} → ((R ⇒ T) ∩ (S ⇒ U)) <: ((R ∪ S) ⇒ (T ∪ U)) +<:-function-∩-∪ function (function , function) = function +<:-function-∩-∪ (function-ok t) (function-ok p₁ , function-ok p₂) = function-ok (right p₂) +<:-function-∩-∪ (function-err _) (function-err p₁ , function-err q₂) = function-err (p₁ , q₂) + +<:-function-∩ : ∀ {S T U} → ((S ⇒ T) ∩ (S ⇒ U)) <: (S ⇒ (T ∩ U)) +<:-function-∩ function (function , function) = function +<:-function-∩ (function-ok t) (function-ok p₁ , function-ok p₂) = function-ok (p₁ , p₂) +<:-function-∩ (function-err s) (function-err p₁ , function-err p₂) = function-err p₂ + +<:-function-∪ : ∀ {R S T U} → ((R ⇒ S) ∪ (T ⇒ U)) <: ((R ∩ T) ⇒ (S ∪ U)) +<:-function-∪ function (left function) = function +<:-function-∪ (function-ok t) (left (function-ok p)) = function-ok (left p) +<:-function-∪ (function-err s) (left (function-err p)) = function-err (left p) +<:-function-∪ (scalar s) (left (scalar ())) +<:-function-∪ function (right function) = function +<:-function-∪ (function-ok t) (right (function-ok p)) = function-ok (right p) +<:-function-∪ (function-err s) (right (function-err x)) = function-err (right x) +<:-function-∪ (scalar s) (right (scalar ())) + +<:-function-∪-∩ : ∀ {R S T U} → ((R ∩ S) ⇒ (T ∪ U)) <: ((R ⇒ T) ∪ (S ⇒ U)) +<:-function-∪-∩ function function = left function +<:-function-∪-∩ (function-ok t) (function-ok (left p)) = left (function-ok p) +<:-function-∪-∩ (function-ok t) (function-ok (right p)) = right (function-ok p) +<:-function-∪-∩ (function-err s) (function-err (left p)) = left (function-err p) +<:-function-∪-∩ (function-err s) (function-err (right p)) = right (function-err p) + +≮:-function-left : ∀ {R S T U} → (R ≮: S) → (S ⇒ T) ≮: (R ⇒ U) +≮:-function-left (witness t p q) = witness (function-err t) (function-err q) (function-err p) + +≮:-function-right : ∀ {R S T U} → (T ≮: U) → (S ⇒ T) ≮: (R ⇒ U) +≮:-function-right (witness t p q) = witness (function-ok t) (function-ok p) (function-ok q) -- Properties of scalars -skalar = number ∪ (string ∪ (nil ∪ boolean)) +skalar-function-ok : ∀ {t} → (¬Language skalar (function-ok t)) +skalar-function-ok = (scalar-function-ok number , (scalar-function-ok string , (scalar-function-ok nil , scalar-function-ok boolean))) + +scalar-<: : ∀ {S T} → (s : Scalar S) → Language T (scalar s) → (S <: T) +scalar-<: number p (scalar number) (scalar number) = p +scalar-<: boolean p (scalar boolean) (scalar boolean) = p +scalar-<: string p (scalar string) (scalar string) = p +scalar-<: nil p (scalar nil) (scalar nil) = p + +scalar-∩-function-<:-never : ∀ {S T U} → (Scalar S) → ((T ⇒ U) ∩ S) <: never +scalar-∩-function-<:-never number .(scalar number) (() , scalar number) +scalar-∩-function-<:-never boolean .(scalar boolean) (() , scalar boolean) +scalar-∩-function-<:-never string .(scalar string) (() , scalar string) +scalar-∩-function-<:-never nil .(scalar nil) (() , scalar nil) function-≮:-scalar : ∀ {S T U} → (Scalar U) → ((S ⇒ T) ≮: U) function-≮:-scalar s = witness function function (scalar-function s) @@ -111,28 +269,8 @@ scalar-≮:-never s = witness (scalar s) (scalar s) never scalar-≢-impl-≮: : ∀ {T U} → (Scalar T) → (Scalar U) → (T ≢ U) → (T ≮: U) scalar-≢-impl-≮: s₁ s₂ p = witness (scalar s₁) (scalar s₁) (scalar-scalar s₁ s₂ p) --- Properties of tgt -tgt-function-ok : ∀ {T t} → (Language (tgt T) t) → Language T (function-ok t) -tgt-function-ok {T = nil} (scalar ()) -tgt-function-ok {T = T₁ ⇒ T₂} p = function-ok p -tgt-function-ok {T = never} (scalar ()) -tgt-function-ok {T = unknown} p = unknown -tgt-function-ok {T = boolean} (scalar ()) -tgt-function-ok {T = number} (scalar ()) -tgt-function-ok {T = string} (scalar ()) -tgt-function-ok {T = T₁ ∪ T₂} (left p) = left (tgt-function-ok p) -tgt-function-ok {T = T₁ ∪ T₂} (right p) = right (tgt-function-ok p) -tgt-function-ok {T = T₁ ∩ T₂} (p₁ , p₂) = (tgt-function-ok p₁ , tgt-function-ok p₂) - -function-ok-tgt : ∀ {T t} → Language T (function-ok t) → (Language (tgt T) t) -function-ok-tgt (function-ok p) = p -function-ok-tgt (left p) = left (function-ok-tgt p) -function-ok-tgt (right p) = right (function-ok-tgt p) -function-ok-tgt (p₁ , p₂) = (function-ok-tgt p₁ , function-ok-tgt p₂) -function-ok-tgt unknown = unknown - -skalar-function-ok : ∀ {t} → (¬Language skalar (function-ok t)) -skalar-function-ok = (scalar-function-ok number , (scalar-function-ok string , (scalar-function-ok nil , scalar-function-ok boolean))) +scalar-≢-∩-<:-never : ∀ {T U V} → (Scalar T) → (Scalar U) → (T ≢ U) → (T ∩ U) <: V +scalar-≢-∩-<:-never s t p u (scalar s₁ , scalar s₂) = CONTRADICTION (p refl) skalar-scalar : ∀ {T} (s : Scalar T) → (Language skalar (scalar s)) skalar-scalar number = left (scalar number) @@ -140,72 +278,6 @@ skalar-scalar boolean = right (right (right (scalar boolean))) skalar-scalar string = right (left (scalar string)) skalar-scalar nil = right (right (left (scalar nil))) -tgt-never-≮: : ∀ {T U} → (tgt T ≮: U) → (T ≮: (skalar ∪ (never ⇒ U))) -tgt-never-≮: (witness t p q) = witness (function-ok t) (tgt-function-ok p) (skalar-function-ok , function-ok q) - -never-tgt-≮: : ∀ {T U} → (T ≮: (skalar ∪ (never ⇒ U))) → (tgt T ≮: U) -never-tgt-≮: (witness (scalar s) p (q₁ , q₂)) = CONTRADICTION (≮:-refl (witness (scalar s) (skalar-scalar s) q₁)) -never-tgt-≮: (witness function p (q₁ , scalar-function ())) -never-tgt-≮: (witness (function-ok t) p (q₁ , function-ok q₂)) = witness t (function-ok-tgt p) q₂ -never-tgt-≮: (witness (function-err (scalar s)) p (q₁ , function-err (scalar ()))) - --- Properties of src -function-err-src : ∀ {T t} → (¬Language (src T) t) → Language T (function-err t) -function-err-src {T = nil} never = scalar-function-err nil -function-err-src {T = T₁ ⇒ T₂} p = function-err p -function-err-src {T = never} (scalar-scalar number () p) -function-err-src {T = never} (scalar-function-ok ()) -function-err-src {T = unknown} never = unknown -function-err-src {T = boolean} p = scalar-function-err boolean -function-err-src {T = number} p = scalar-function-err number -function-err-src {T = string} p = scalar-function-err string -function-err-src {T = T₁ ∪ T₂} (left p) = left (function-err-src p) -function-err-src {T = T₁ ∪ T₂} (right p) = right (function-err-src p) -function-err-src {T = T₁ ∩ T₂} (p₁ , p₂) = function-err-src p₁ , function-err-src p₂ - -¬function-err-src : ∀ {T t} → (Language (src T) t) → ¬Language T (function-err t) -¬function-err-src {T = nil} (scalar ()) -¬function-err-src {T = T₁ ⇒ T₂} p = function-err p -¬function-err-src {T = never} unknown = never -¬function-err-src {T = unknown} (scalar ()) -¬function-err-src {T = boolean} (scalar ()) -¬function-err-src {T = number} (scalar ()) -¬function-err-src {T = string} (scalar ()) -¬function-err-src {T = T₁ ∪ T₂} (p₁ , p₂) = (¬function-err-src p₁ , ¬function-err-src p₂) -¬function-err-src {T = T₁ ∩ T₂} (left p) = left (¬function-err-src p) -¬function-err-src {T = T₁ ∩ T₂} (right p) = right (¬function-err-src p) - -src-¬function-err : ∀ {T t} → Language T (function-err t) → (¬Language (src T) t) -src-¬function-err {T = nil} p = never -src-¬function-err {T = T₁ ⇒ T₂} (function-err p) = p -src-¬function-err {T = never} (scalar-function-err ()) -src-¬function-err {T = unknown} p = never -src-¬function-err {T = boolean} p = never -src-¬function-err {T = number} p = never -src-¬function-err {T = string} p = never -src-¬function-err {T = T₁ ∪ T₂} (left p) = left (src-¬function-err p) -src-¬function-err {T = T₁ ∪ T₂} (right p) = right (src-¬function-err p) -src-¬function-err {T = T₁ ∩ T₂} (p₁ , p₂) = (src-¬function-err p₁ , src-¬function-err p₂) - -src-¬scalar : ∀ {S T t} (s : Scalar S) → Language T (scalar s) → (¬Language (src T) t) -src-¬scalar number (scalar number) = never -src-¬scalar boolean (scalar boolean) = never -src-¬scalar string (scalar string) = never -src-¬scalar nil (scalar nil) = never -src-¬scalar s (left p) = left (src-¬scalar s p) -src-¬scalar s (right p) = right (src-¬scalar s p) -src-¬scalar s (p₁ , p₂) = (src-¬scalar s p₁ , src-¬scalar s p₂) -src-¬scalar s unknown = never - -src-unknown-≮: : ∀ {T U} → (T ≮: src U) → (U ≮: (T ⇒ unknown)) -src-unknown-≮: (witness t p q) = witness (function-err t) (function-err-src q) (¬function-err-src p) - -unknown-src-≮: : ∀ {S T U} → (U ≮: S) → (T ≮: (U ⇒ unknown)) → (U ≮: src T) -unknown-src-≮: (witness t x x₁) (witness (scalar s) p (function-scalar s)) = witness t x (src-¬scalar s p) -unknown-src-≮: r (witness (function-ok (scalar s)) p (function-ok (scalar-scalar s () q))) -unknown-src-≮: r (witness (function-ok (function-ok _)) p (function-ok (scalar-function-ok ()))) -unknown-src-≮: r (witness (function-err t) p (function-err q)) = witness t q (src-¬function-err p) - -- Properties of unknown and never unknown-≮: : ∀ {T U} → (T ≮: U) → (unknown ≮: U) unknown-≮: (witness t p q) = witness t unknown q @@ -219,6 +291,28 @@ unknown-≮:-never = witness (scalar nil) unknown never function-≮:-never : ∀ {T U} → ((T ⇒ U) ≮: never) function-≮:-never = witness function function never +<:-never : ∀ {T} → (never <: T) +<:-never t (scalar ()) + +≮:-never-left : ∀ {S T U} → (S <: (T ∪ U)) → (S ≮: T) → (S ∩ U) ≮: never +≮:-never-left p (witness t q₁ q₂) with p t q₁ +≮:-never-left p (witness t q₁ q₂) | left r = CONTRADICTION (language-comp t q₂ r) +≮:-never-left p (witness t q₁ q₂) | right r = witness t (q₁ , r) never + +≮:-never-right : ∀ {S T U} → (S <: (T ∪ U)) → (S ≮: U) → (S ∩ T) ≮: never +≮:-never-right p (witness t q₁ q₂) with p t q₁ +≮:-never-right p (witness t q₁ q₂) | left r = witness t (q₁ , r) never +≮:-never-right p (witness t q₁ q₂) | right r = CONTRADICTION (language-comp t q₂ r) + +<:-unknown : ∀ {T} → (T <: unknown) +<:-unknown t p = unknown + +<:-everything : unknown <: ((never ⇒ unknown) ∪ skalar) +<:-everything (scalar s) p = right (skalar-scalar s) +<:-everything function p = left function +<:-everything (function-ok t) p = left (function-ok unknown) +<:-everything (function-err s) p = left (function-err never) + -- A Gentle Introduction To Semantic Subtyping (https://www.cduce.org/papers/gentle.pdf) -- defines a "set-theoretic" model (sec 2.5) -- Unfortunately we don't quite have this property, due to uninhabited types, @@ -234,13 +328,21 @@ _⊗_ : ∀ {A B : Set} → (A → Set) → (B → Set) → ((A × B) → Set) Comp : ∀ {A : Set} → (A → Set) → (A → Set) Comp P a = ¬(P a) +Lift : ∀ {A : Set} → (A → Set) → (Maybe A → Set) +Lift P nothing = ⊥ +Lift P (just a) = P a + set-theoretic-if : ∀ {S₁ T₁ S₂ T₂} → -- This is the "if" part of being a set-theoretic model + -- though it uses the definition from Frisch's thesis + -- rather than from the Gentle Introduction. The difference + -- being the presence of Lift, (written D_Ω in Defn 4.2 of + -- https://www.cduce.org/papers/frisch_phd.pdf). (Language (S₁ ⇒ T₁) ⊆ Language (S₂ ⇒ T₂)) → - (∀ Q → Q ⊆ Comp((Language S₁) ⊗ Comp(Language T₁)) → Q ⊆ Comp((Language S₂) ⊗ Comp(Language T₂))) + (∀ Q → Q ⊆ Comp((Language S₁) ⊗ Comp(Lift(Language T₁))) → Q ⊆ Comp((Language S₂) ⊗ Comp(Lift(Language T₂)))) -set-theoretic-if {S₁} {T₁} {S₂} {T₂} p Q q (t , u) Qtu (S₂t , ¬T₂u) = q (t , u) Qtu (S₁t , ¬T₁u) where +set-theoretic-if {S₁} {T₁} {S₂} {T₂} p Q q (t , just u) Qtu (S₂t , ¬T₂u) = q (t , just u) Qtu (S₁t , ¬T₁u) where S₁t : Language S₁ t S₁t with dec-language S₁ t @@ -252,6 +354,14 @@ set-theoretic-if {S₁} {T₁} {S₂} {T₂} p Q q (t , u) Qtu (S₂t , ¬T₂u) ¬T₁u T₁u with p (function-ok u) (function-ok T₁u) ¬T₁u T₁u | function-ok T₂u = ¬T₂u T₂u +set-theoretic-if {S₁} {T₁} {S₂} {T₂} p Q q (t , nothing) Qt- (S₂t , _) = q (t , nothing) Qt- (S₁t , λ ()) where + + S₁t : Language S₁ t + S₁t with dec-language S₁ t + S₁t | Left ¬S₁t with p (function-err t) (function-err ¬S₁t) + S₁t | Left ¬S₁t | function-err ¬S₂t = CONTRADICTION (language-comp t ¬S₂t S₂t) + S₁t | Right r = r + not-quite-set-theoretic-only-if : ∀ {S₁ T₁ S₂ T₂} → -- We don't quite have that this is a set-theoretic model @@ -260,32 +370,33 @@ not-quite-set-theoretic-only-if : ∀ {S₁ T₁ S₂ T₂} → ∀ s₂ t₂ → Language S₂ s₂ → ¬Language T₂ t₂ → -- This is the "only if" part of being a set-theoretic model - (∀ Q → Q ⊆ Comp((Language S₁) ⊗ Comp(Language T₁)) → Q ⊆ Comp((Language S₂) ⊗ Comp(Language T₂))) → + (∀ Q → Q ⊆ Comp((Language S₁) ⊗ Comp(Lift(Language T₁))) → Q ⊆ Comp((Language S₂) ⊗ Comp(Lift(Language T₂)))) → (Language (S₁ ⇒ T₁) ⊆ Language (S₂ ⇒ T₂)) not-quite-set-theoretic-only-if {S₁} {T₁} {S₂} {T₂} s₂ t₂ S₂s₂ ¬T₂t₂ p = r where - Q : (Tree × Tree) → Set - Q (t , u) = Either (¬Language S₁ t) (Language T₁ u) + Q : (Tree × Maybe Tree) → Set + Q (t , just u) = Either (¬Language S₁ t) (Language T₁ u) + Q (t , nothing) = ¬Language S₁ t - q : Q ⊆ Comp((Language S₁) ⊗ Comp(Language T₁)) - q (t , u) (Left ¬S₁t) (S₁t , ¬T₁u) = language-comp t ¬S₁t S₁t - q (t , u) (Right T₂u) (S₁t , ¬T₁u) = ¬T₁u T₂u + q : Q ⊆ Comp((Language S₁) ⊗ Comp(Lift(Language T₁))) + q (t , just u) (Left ¬S₁t) (S₁t , ¬T₁u) = language-comp t ¬S₁t S₁t + q (t , just u) (Right T₂u) (S₁t , ¬T₁u) = ¬T₁u T₂u + q (t , nothing) ¬S₁t (S₁t , _) = language-comp t ¬S₁t S₁t r : Language (S₁ ⇒ T₁) ⊆ Language (S₂ ⇒ T₂) r function function = function - r (function-err t) (function-err ¬S₁t) with dec-language S₂ t - r (function-err t) (function-err ¬S₁t) | Left ¬S₂t = function-err ¬S₂t - r (function-err t) (function-err ¬S₁t) | Right S₂t = CONTRADICTION (p Q q (t , t₂) (Left ¬S₁t) (S₂t , language-comp t₂ ¬T₂t₂)) + r (function-err s) (function-err ¬S₁s) with dec-language S₂ s + r (function-err s) (function-err ¬S₁s) | Left ¬S₂s = function-err ¬S₂s + r (function-err s) (function-err ¬S₁s) | Right S₂s = CONTRADICTION (p Q q (s , nothing) ¬S₁s (S₂s , λ ())) r (function-ok t) (function-ok T₁t) with dec-language T₂ t - r (function-ok t) (function-ok T₁t) | Left ¬T₂t = CONTRADICTION (p Q q (s₂ , t) (Right T₁t) (S₂s₂ , language-comp t ¬T₂t)) + r (function-ok t) (function-ok T₁t) | Left ¬T₂t = CONTRADICTION (p Q q (s₂ , just t) (Right T₁t) (S₂s₂ , language-comp t ¬T₂t)) r (function-ok t) (function-ok T₁t) | Right T₂t = function-ok T₂t -- A counterexample when the argument type is empty. -set-theoretic-counterexample-one : (∀ Q → Q ⊆ Comp((Language never) ⊗ Comp(Language number)) → Q ⊆ Comp((Language never) ⊗ Comp(Language string))) +set-theoretic-counterexample-one : (∀ Q → Q ⊆ Comp((Language never) ⊗ Comp(Lift(Language number))) → Q ⊆ Comp((Language never) ⊗ Comp(Lift(Language string)))) set-theoretic-counterexample-one Q q ((scalar s) , u) Qtu (scalar () , p) -set-theoretic-counterexample-one Q q ((function-err t) , u) Qtu (scalar-function-err () , p) set-theoretic-counterexample-two : (never ⇒ number) ≮: (never ⇒ string) set-theoretic-counterexample-two = witness diff --git a/prototyping/Properties/TypeCheck.agda b/prototyping/Properties/TypeCheck.agda index 0726a4be..37fbeda5 100644 --- a/prototyping/Properties/TypeCheck.agda +++ b/prototyping/Properties/TypeCheck.agda @@ -8,7 +8,8 @@ open import FFI.Data.Maybe using (Maybe; just; nothing) open import FFI.Data.Either using (Either) open import Luau.TypeCheck using (_⊢ᴱ_∈_; _⊢ᴮ_∈_; ⊢ᴼ_; ⊢ᴴ_; _⊢ᴴᴱ_▷_∈_; _⊢ᴴᴮ_▷_∈_; nil; var; addr; number; bool; string; app; function; block; binexp; done; return; local; nothing; orUnknown; tgtBinOp) open import Luau.Syntax using (Block; Expr; Value; BinaryOperator; yes; nil; addr; number; bool; string; val; var; binexp; _$_; function_is_end; block_is_end; _∙_; return; done; local_←_; _⟨_⟩; _⟨_⟩∈_; var_∈_; name; fun; arg; +; -; *; /; <; >; ==; ~=; <=; >=) -open import Luau.Type using (Type; nil; unknown; never; number; boolean; string; _⇒_; src; tgt) +open import Luau.FunctionTypes using (src; tgt) +open import Luau.Type using (Type; nil; unknown; never; number; boolean; string; _⇒_) open import Luau.RuntimeType using (RuntimeType; nil; number; function; string; valueType) open import Luau.VarCtxt using (VarCtxt; ∅; _↦_; _⊕_↦_; _⋒_; _⊝_) renaming (_[_] to _[_]ⱽ) open import Luau.Addr using (Addr) diff --git a/prototyping/Properties/TypeNormalization.agda b/prototyping/Properties/TypeNormalization.agda new file mode 100644 index 00000000..299f648c --- /dev/null +++ b/prototyping/Properties/TypeNormalization.agda @@ -0,0 +1,376 @@ +{-# OPTIONS --rewriting #-} + +module Properties.TypeNormalization where + +open import Luau.Type using (Type; Scalar; nil; number; string; boolean; never; unknown; _⇒_; _∪_; _∩_) +open import Luau.Subtyping using (scalar-function-err) +open import Luau.TypeNormalization using (_∪ⁿ_; _∩ⁿ_; _∪ᶠ_; _∪ⁿˢ_; _∩ⁿˢ_; normalize) +open import Luau.Subtyping using (_<:_) +open import Properties.Subtyping using (<:-trans; <:-refl; <:-unknown; <:-never; <:-∪-left; <:-∪-right; <:-∪-lub; <:-∩-left; <:-∩-right; <:-∩-glb; <:-∩-symm; <:-function; <:-function-∪-∩; <:-function-∩-∪; <:-function-∪; <:-everything; <:-union; <:-∪-assocl; <:-∪-assocr; <:-∪-symm; <:-intersect; ∪-distl-∩-<:; ∪-distr-∩-<:; <:-∪-distr-∩; <:-∪-distl-∩; ∩-distl-∪-<:; <:-∩-distl-∪; <:-∩-distr-∪; scalar-∩-function-<:-never; scalar-≢-∩-<:-never) + +-- Notmal forms for types +data FunType : Type → Set +data Normal : Type → Set + +data FunType where + _⇒_ : ∀ {S T} → Normal S → Normal T → FunType (S ⇒ T) + _∩_ : ∀ {F G} → FunType F → FunType G → FunType (F ∩ G) + +data Normal where + never : Normal never + unknown : Normal unknown + _⇒_ : ∀ {S T} → Normal S → Normal T → Normal (S ⇒ T) + _∩_ : ∀ {F G} → FunType F → FunType G → Normal (F ∩ G) + _∪_ : ∀ {S T} → Normal S → Scalar T → Normal (S ∪ T) + +data OptScalar : Type → Set where + never : OptScalar never + number : OptScalar number + boolean : OptScalar boolean + string : OptScalar string + nil : OptScalar nil + +-- Normalization produces normal types +normal : ∀ T → Normal (normalize T) +normalᶠ : ∀ {F} → FunType F → Normal F +normal-∪ⁿ : ∀ {S T} → Normal S → Normal T → Normal (S ∪ⁿ T) +normal-∩ⁿ : ∀ {S T} → Normal S → Normal T → Normal (S ∩ⁿ T) +normal-∪ⁿˢ : ∀ {S T} → Normal S → OptScalar T → Normal (S ∪ⁿˢ T) +normal-∩ⁿˢ : ∀ {S T} → Normal S → Scalar T → OptScalar (S ∩ⁿˢ T) +normal-∪ᶠ : ∀ {F G} → FunType F → FunType G → FunType (F ∪ᶠ G) + +normal nil = never ∪ nil +normal (S ⇒ T) = normalᶠ ((normal S) ⇒ (normal T)) +normal never = never +normal unknown = unknown +normal boolean = never ∪ boolean +normal number = never ∪ number +normal string = never ∪ string +normal (S ∪ T) = normal-∪ⁿ (normal S) (normal T) +normal (S ∩ T) = normal-∩ⁿ (normal S) (normal T) + +normalᶠ (S ⇒ T) = S ⇒ T +normalᶠ (F ∩ G) = F ∩ G + +normal-∪ⁿ S (T₁ ∪ T₂) = (normal-∪ⁿ S T₁) ∪ T₂ +normal-∪ⁿ S never = S +normal-∪ⁿ S unknown = unknown +normal-∪ⁿ never (T ⇒ U) = T ⇒ U +normal-∪ⁿ never (G₁ ∩ G₂) = G₁ ∩ G₂ +normal-∪ⁿ unknown (T ⇒ U) = unknown +normal-∪ⁿ unknown (G₁ ∩ G₂) = unknown +normal-∪ⁿ (R ⇒ S) (T ⇒ U) = normalᶠ (normal-∪ᶠ (R ⇒ S) (T ⇒ U)) +normal-∪ⁿ (R ⇒ S) (G₁ ∩ G₂) = normalᶠ (normal-∪ᶠ (R ⇒ S) (G₁ ∩ G₂)) +normal-∪ⁿ (F₁ ∩ F₂) (T ⇒ U) = normalᶠ (normal-∪ᶠ (F₁ ∩ F₂) (T ⇒ U)) +normal-∪ⁿ (F₁ ∩ F₂) (G₁ ∩ G₂) = normalᶠ (normal-∪ᶠ (F₁ ∩ F₂) (G₁ ∩ G₂)) +normal-∪ⁿ (S₁ ∪ S₂) (T₁ ⇒ T₂) = normal-∪ⁿ S₁ (T₁ ⇒ T₂) ∪ S₂ +normal-∪ⁿ (S₁ ∪ S₂) (G₁ ∩ G₂) = normal-∪ⁿ S₁ (G₁ ∩ G₂) ∪ S₂ + +normal-∩ⁿ S never = never +normal-∩ⁿ S unknown = S +normal-∩ⁿ S (T ∪ U) = normal-∪ⁿˢ (normal-∩ⁿ S T) (normal-∩ⁿˢ S U ) +normal-∩ⁿ never (T ⇒ U) = never +normal-∩ⁿ unknown (T ⇒ U) = T ⇒ U +normal-∩ⁿ (R ⇒ S) (T ⇒ U) = (R ⇒ S) ∩ (T ⇒ U) +normal-∩ⁿ (R ∩ S) (T ⇒ U) = (R ∩ S) ∩ (T ⇒ U) +normal-∩ⁿ (R ∪ S) (T ⇒ U) = normal-∩ⁿ R (T ⇒ U) +normal-∩ⁿ never (T ∩ U) = never +normal-∩ⁿ unknown (T ∩ U) = T ∩ U +normal-∩ⁿ (R ⇒ S) (T ∩ U) = (R ⇒ S) ∩ (T ∩ U) +normal-∩ⁿ (R ∩ S) (T ∩ U) = (R ∩ S) ∩ (T ∩ U) +normal-∩ⁿ (R ∪ S) (T ∩ U) = normal-∩ⁿ R (T ∩ U) + +normal-∪ⁿˢ S never = S +normal-∪ⁿˢ never number = never ∪ number +normal-∪ⁿˢ unknown number = unknown +normal-∪ⁿˢ (R ⇒ S) number = (R ⇒ S) ∪ number +normal-∪ⁿˢ (R ∩ S) number = (R ∩ S) ∪ number +normal-∪ⁿˢ (R ∪ number) number = R ∪ number +normal-∪ⁿˢ (R ∪ boolean) number = normal-∪ⁿˢ R number ∪ boolean +normal-∪ⁿˢ (R ∪ string) number = normal-∪ⁿˢ R number ∪ string +normal-∪ⁿˢ (R ∪ nil) number = normal-∪ⁿˢ R number ∪ nil +normal-∪ⁿˢ never boolean = never ∪ boolean +normal-∪ⁿˢ unknown boolean = unknown +normal-∪ⁿˢ (R ⇒ S) boolean = (R ⇒ S) ∪ boolean +normal-∪ⁿˢ (R ∩ S) boolean = (R ∩ S) ∪ boolean +normal-∪ⁿˢ (R ∪ number) boolean = normal-∪ⁿˢ R boolean ∪ number +normal-∪ⁿˢ (R ∪ boolean) boolean = R ∪ boolean +normal-∪ⁿˢ (R ∪ string) boolean = normal-∪ⁿˢ R boolean ∪ string +normal-∪ⁿˢ (R ∪ nil) boolean = normal-∪ⁿˢ R boolean ∪ nil +normal-∪ⁿˢ never string = never ∪ string +normal-∪ⁿˢ unknown string = unknown +normal-∪ⁿˢ (R ⇒ S) string = (R ⇒ S) ∪ string +normal-∪ⁿˢ (R ∩ S) string = (R ∩ S) ∪ string +normal-∪ⁿˢ (R ∪ number) string = normal-∪ⁿˢ R string ∪ number +normal-∪ⁿˢ (R ∪ boolean) string = normal-∪ⁿˢ R string ∪ boolean +normal-∪ⁿˢ (R ∪ string) string = R ∪ string +normal-∪ⁿˢ (R ∪ nil) string = normal-∪ⁿˢ R string ∪ nil +normal-∪ⁿˢ never nil = never ∪ nil +normal-∪ⁿˢ unknown nil = unknown +normal-∪ⁿˢ (R ⇒ S) nil = (R ⇒ S) ∪ nil +normal-∪ⁿˢ (R ∩ S) nil = (R ∩ S) ∪ nil +normal-∪ⁿˢ (R ∪ number) nil = normal-∪ⁿˢ R nil ∪ number +normal-∪ⁿˢ (R ∪ boolean) nil = normal-∪ⁿˢ R nil ∪ boolean +normal-∪ⁿˢ (R ∪ string) nil = normal-∪ⁿˢ R nil ∪ string +normal-∪ⁿˢ (R ∪ nil) nil = R ∪ nil + +normal-∩ⁿˢ never number = never +normal-∩ⁿˢ never boolean = never +normal-∩ⁿˢ never string = never +normal-∩ⁿˢ never nil = never +normal-∩ⁿˢ unknown number = number +normal-∩ⁿˢ unknown boolean = boolean +normal-∩ⁿˢ unknown string = string +normal-∩ⁿˢ unknown nil = nil +normal-∩ⁿˢ (R ⇒ S) number = never +normal-∩ⁿˢ (R ⇒ S) boolean = never +normal-∩ⁿˢ (R ⇒ S) string = never +normal-∩ⁿˢ (R ⇒ S) nil = never +normal-∩ⁿˢ (R ∩ S) number = never +normal-∩ⁿˢ (R ∩ S) boolean = never +normal-∩ⁿˢ (R ∩ S) string = never +normal-∩ⁿˢ (R ∩ S) nil = never +normal-∩ⁿˢ (R ∪ number) number = number +normal-∩ⁿˢ (R ∪ boolean) number = normal-∩ⁿˢ R number +normal-∩ⁿˢ (R ∪ string) number = normal-∩ⁿˢ R number +normal-∩ⁿˢ (R ∪ nil) number = normal-∩ⁿˢ R number +normal-∩ⁿˢ (R ∪ number) boolean = normal-∩ⁿˢ R boolean +normal-∩ⁿˢ (R ∪ boolean) boolean = boolean +normal-∩ⁿˢ (R ∪ string) boolean = normal-∩ⁿˢ R boolean +normal-∩ⁿˢ (R ∪ nil) boolean = normal-∩ⁿˢ R boolean +normal-∩ⁿˢ (R ∪ number) string = normal-∩ⁿˢ R string +normal-∩ⁿˢ (R ∪ boolean) string = normal-∩ⁿˢ R string +normal-∩ⁿˢ (R ∪ string) string = string +normal-∩ⁿˢ (R ∪ nil) string = normal-∩ⁿˢ R string +normal-∩ⁿˢ (R ∪ number) nil = normal-∩ⁿˢ R nil +normal-∩ⁿˢ (R ∪ boolean) nil = normal-∩ⁿˢ R nil +normal-∩ⁿˢ (R ∪ string) nil = normal-∩ⁿˢ R nil +normal-∩ⁿˢ (R ∪ nil) nil = nil + +normal-∪ᶠ (R ⇒ S) (T ⇒ U) = (normal-∩ⁿ R T) ⇒ (normal-∪ⁿ S U) +normal-∪ᶠ (R ⇒ S) (G ∩ H) = normal-∪ᶠ (R ⇒ S) G ∩ normal-∪ᶠ (R ⇒ S) H +normal-∪ᶠ (E ∩ F) G = normal-∪ᶠ E G ∩ normal-∪ᶠ F G + +scalar-∩-fun-<:-never : ∀ {F S} → FunType F → Scalar S → (F ∩ S) <: never +scalar-∩-fun-<:-never (T ⇒ U) S = scalar-∩-function-<:-never S +scalar-∩-fun-<:-never (F ∩ G) S = <:-trans (<:-intersect <:-∩-left <:-refl) (scalar-∩-fun-<:-never F S) + +flipper : ∀ {S T U} → ((S ∪ T) ∪ U) <: ((S ∪ U) ∪ T) +flipper = <:-trans <:-∪-assocr (<:-trans (<:-union <:-refl <:-∪-symm) <:-∪-assocl) + +∩-<:-∩ⁿ : ∀ {S T} → Normal S → Normal T → (S ∩ T) <: (S ∩ⁿ T) +∩ⁿ-<:-∩ : ∀ {S T} → Normal S → Normal T → (S ∩ⁿ T) <: (S ∩ T) +∩-<:-∩ⁿˢ : ∀ {S T} → Normal S → Scalar T → (S ∩ T) <: (S ∩ⁿˢ T) +∩ⁿˢ-<:-∩ : ∀ {S T} → Normal S → Scalar T → (S ∩ⁿˢ T) <: (S ∩ T) +∪ᶠ-<:-∪ : ∀ {F G} → FunType F → FunType G → (F ∪ᶠ G) <: (F ∪ G) +∪ⁿ-<:-∪ : ∀ {S T} → Normal S → Normal T → (S ∪ⁿ T) <: (S ∪ T) +∪-<:-∪ⁿ : ∀ {S T} → Normal S → Normal T → (S ∪ T) <: (S ∪ⁿ T) +∪ⁿˢ-<:-∪ : ∀ {S T} → Normal S → OptScalar T → (S ∪ⁿˢ T) <: (S ∪ T) +∪-<:-∪ⁿˢ : ∀ {S T} → Normal S → OptScalar T → (S ∪ T) <: (S ∪ⁿˢ T) + +∩-<:-∩ⁿ S never = <:-∩-right +∩-<:-∩ⁿ S unknown = <:-∩-left +∩-<:-∩ⁿ S (T ∪ U) = <:-trans <:-∩-distl-∪ (<:-trans (<:-union (∩-<:-∩ⁿ S T) (∩-<:-∩ⁿˢ S U)) (∪-<:-∪ⁿˢ (normal-∩ⁿ S T) (normal-∩ⁿˢ S U)) ) +∩-<:-∩ⁿ never (T ⇒ U) = <:-∩-left +∩-<:-∩ⁿ unknown (T ⇒ U) = <:-∩-right +∩-<:-∩ⁿ (R ⇒ S) (T ⇒ U) = <:-refl +∩-<:-∩ⁿ (R ∩ S) (T ⇒ U) = <:-refl +∩-<:-∩ⁿ (R ∪ S) (T ⇒ U) = <:-trans <:-∩-distr-∪ (<:-trans (<:-union (∩-<:-∩ⁿ R (T ⇒ U)) (<:-trans <:-∩-symm (∩-<:-∩ⁿˢ (T ⇒ U) S))) (<:-∪-lub <:-refl <:-never)) +∩-<:-∩ⁿ never (T ∩ U) = <:-∩-left +∩-<:-∩ⁿ unknown (T ∩ U) = <:-∩-right +∩-<:-∩ⁿ (R ⇒ S) (T ∩ U) = <:-refl +∩-<:-∩ⁿ (R ∩ S) (T ∩ U) = <:-refl +∩-<:-∩ⁿ (R ∪ S) (T ∩ U) = <:-trans <:-∩-distr-∪ (<:-trans (<:-union (∩-<:-∩ⁿ R (T ∩ U)) (<:-trans <:-∩-symm (∩-<:-∩ⁿˢ (T ∩ U) S))) (<:-∪-lub <:-refl <:-never)) + +∩ⁿ-<:-∩ S never = <:-never +∩ⁿ-<:-∩ S unknown = <:-∩-glb <:-refl <:-unknown +∩ⁿ-<:-∩ S (T ∪ U) = <:-trans (∪ⁿˢ-<:-∪ (normal-∩ⁿ S T) (normal-∩ⁿˢ S U)) (<:-trans (<:-union (∩ⁿ-<:-∩ S T) (∩ⁿˢ-<:-∩ S U)) ∩-distl-∪-<:) +∩ⁿ-<:-∩ never (T ⇒ U) = <:-never +∩ⁿ-<:-∩ unknown (T ⇒ U) = <:-∩-glb <:-unknown <:-refl +∩ⁿ-<:-∩ (R ⇒ S) (T ⇒ U) = <:-refl +∩ⁿ-<:-∩ (R ∩ S) (T ⇒ U) = <:-refl +∩ⁿ-<:-∩ (R ∪ S) (T ⇒ U) = <:-trans (∩ⁿ-<:-∩ R (T ⇒ U)) (<:-∩-glb (<:-trans <:-∩-left <:-∪-left) <:-∩-right) +∩ⁿ-<:-∩ never (T ∩ U) = <:-never +∩ⁿ-<:-∩ unknown (T ∩ U) = <:-∩-glb <:-unknown <:-refl +∩ⁿ-<:-∩ (R ⇒ S) (T ∩ U) = <:-refl +∩ⁿ-<:-∩ (R ∩ S) (T ∩ U) = <:-refl +∩ⁿ-<:-∩ (R ∪ S) (T ∩ U) = <:-trans (∩ⁿ-<:-∩ R (T ∩ U)) (<:-∩-glb (<:-trans <:-∩-left <:-∪-left) <:-∩-right) + +∩-<:-∩ⁿˢ never number = <:-∩-left +∩-<:-∩ⁿˢ never boolean = <:-∩-left +∩-<:-∩ⁿˢ never string = <:-∩-left +∩-<:-∩ⁿˢ never nil = <:-∩-left +∩-<:-∩ⁿˢ unknown T = <:-∩-right +∩-<:-∩ⁿˢ (R ⇒ S) T = scalar-∩-fun-<:-never (R ⇒ S) T +∩-<:-∩ⁿˢ (F ∩ G) T = scalar-∩-fun-<:-never (F ∩ G) T +∩-<:-∩ⁿˢ (R ∪ number) number = <:-∩-right +∩-<:-∩ⁿˢ (R ∪ boolean) number = <:-trans <:-∩-distr-∪ (<:-∪-lub (∩-<:-∩ⁿˢ R number) (scalar-≢-∩-<:-never boolean number (λ ()))) +∩-<:-∩ⁿˢ (R ∪ string) number = <:-trans <:-∩-distr-∪ (<:-∪-lub (∩-<:-∩ⁿˢ R number) (scalar-≢-∩-<:-never string number (λ ()))) +∩-<:-∩ⁿˢ (R ∪ nil) number = <:-trans <:-∩-distr-∪ (<:-∪-lub (∩-<:-∩ⁿˢ R number) (scalar-≢-∩-<:-never nil number (λ ()))) +∩-<:-∩ⁿˢ (R ∪ number) boolean = <:-trans <:-∩-distr-∪ (<:-∪-lub (∩-<:-∩ⁿˢ R boolean) (scalar-≢-∩-<:-never number boolean (λ ()))) +∩-<:-∩ⁿˢ (R ∪ boolean) boolean = <:-∩-right +∩-<:-∩ⁿˢ (R ∪ string) boolean = <:-trans <:-∩-distr-∪ (<:-∪-lub (∩-<:-∩ⁿˢ R boolean) (scalar-≢-∩-<:-never string boolean (λ ()))) +∩-<:-∩ⁿˢ (R ∪ nil) boolean = <:-trans <:-∩-distr-∪ (<:-∪-lub (∩-<:-∩ⁿˢ R boolean) (scalar-≢-∩-<:-never nil boolean (λ ()))) +∩-<:-∩ⁿˢ (R ∪ number) string = <:-trans <:-∩-distr-∪ (<:-∪-lub (∩-<:-∩ⁿˢ R string) (scalar-≢-∩-<:-never number string (λ ()))) +∩-<:-∩ⁿˢ (R ∪ boolean) string = <:-trans <:-∩-distr-∪ (<:-∪-lub (∩-<:-∩ⁿˢ R string) (scalar-≢-∩-<:-never boolean string (λ ()))) +∩-<:-∩ⁿˢ (R ∪ string) string = <:-∩-right +∩-<:-∩ⁿˢ (R ∪ nil) string = <:-trans <:-∩-distr-∪ (<:-∪-lub (∩-<:-∩ⁿˢ R string) (scalar-≢-∩-<:-never nil string (λ ()))) +∩-<:-∩ⁿˢ (R ∪ number) nil = <:-trans <:-∩-distr-∪ (<:-∪-lub (∩-<:-∩ⁿˢ R nil) (scalar-≢-∩-<:-never number nil (λ ()))) +∩-<:-∩ⁿˢ (R ∪ boolean) nil = <:-trans <:-∩-distr-∪ (<:-∪-lub (∩-<:-∩ⁿˢ R nil) (scalar-≢-∩-<:-never boolean nil (λ ()))) +∩-<:-∩ⁿˢ (R ∪ string) nil = <:-trans <:-∩-distr-∪ (<:-∪-lub (∩-<:-∩ⁿˢ R nil) (scalar-≢-∩-<:-never string nil (λ ()))) +∩-<:-∩ⁿˢ (R ∪ nil) nil = <:-∩-right + +∩ⁿˢ-<:-∩ never T = <:-never +∩ⁿˢ-<:-∩ unknown T = <:-∩-glb <:-unknown <:-refl +∩ⁿˢ-<:-∩ (R ⇒ S) T = <:-never +∩ⁿˢ-<:-∩ (F ∩ G) T = <:-never +∩ⁿˢ-<:-∩ (R ∪ number) number = <:-∩-glb <:-∪-right <:-refl +∩ⁿˢ-<:-∩ (R ∪ boolean) number = <:-trans (∩ⁿˢ-<:-∩ R number) (<:-intersect <:-∪-left <:-refl) +∩ⁿˢ-<:-∩ (R ∪ string) number = <:-trans (∩ⁿˢ-<:-∩ R number) (<:-intersect <:-∪-left <:-refl) +∩ⁿˢ-<:-∩ (R ∪ nil) number = <:-trans (∩ⁿˢ-<:-∩ R number) (<:-intersect <:-∪-left <:-refl) +∩ⁿˢ-<:-∩ (R ∪ number) boolean = <:-trans (∩ⁿˢ-<:-∩ R boolean) (<:-intersect <:-∪-left <:-refl) +∩ⁿˢ-<:-∩ (R ∪ boolean) boolean = <:-∩-glb <:-∪-right <:-refl +∩ⁿˢ-<:-∩ (R ∪ string) boolean = <:-trans (∩ⁿˢ-<:-∩ R boolean) (<:-intersect <:-∪-left <:-refl) +∩ⁿˢ-<:-∩ (R ∪ nil) boolean = <:-trans (∩ⁿˢ-<:-∩ R boolean) (<:-intersect <:-∪-left <:-refl) +∩ⁿˢ-<:-∩ (R ∪ number) string = <:-trans (∩ⁿˢ-<:-∩ R string) (<:-intersect <:-∪-left <:-refl) +∩ⁿˢ-<:-∩ (R ∪ boolean) string = <:-trans (∩ⁿˢ-<:-∩ R string) (<:-intersect <:-∪-left <:-refl) +∩ⁿˢ-<:-∩ (R ∪ string) string = <:-∩-glb <:-∪-right <:-refl +∩ⁿˢ-<:-∩ (R ∪ nil) string = <:-trans (∩ⁿˢ-<:-∩ R string) (<:-intersect <:-∪-left <:-refl) +∩ⁿˢ-<:-∩ (R ∪ number) nil = <:-trans (∩ⁿˢ-<:-∩ R nil) (<:-intersect <:-∪-left <:-refl) +∩ⁿˢ-<:-∩ (R ∪ boolean) nil = <:-trans (∩ⁿˢ-<:-∩ R nil) (<:-intersect <:-∪-left <:-refl) +∩ⁿˢ-<:-∩ (R ∪ string) nil = <:-trans (∩ⁿˢ-<:-∩ R nil) (<:-intersect <:-∪-left <:-refl) +∩ⁿˢ-<:-∩ (R ∪ nil) nil = <:-∩-glb <:-∪-right <:-refl + +∪ᶠ-<:-∪ (R ⇒ S) (T ⇒ U) = <:-trans (<:-function (∩-<:-∩ⁿ R T) (∪ⁿ-<:-∪ S U)) <:-function-∪-∩ +∪ᶠ-<:-∪ (R ⇒ S) (G ∩ H) = <:-trans (<:-intersect (∪ᶠ-<:-∪ (R ⇒ S) G) (∪ᶠ-<:-∪ (R ⇒ S) H)) ∪-distl-∩-<: +∪ᶠ-<:-∪ (E ∩ F) G = <:-trans (<:-intersect (∪ᶠ-<:-∪ E G) (∪ᶠ-<:-∪ F G)) ∪-distr-∩-<: + +∪-<:-∪ᶠ : ∀ {F G} → FunType F → FunType G → (F ∪ G) <: (F ∪ᶠ G) +∪-<:-∪ᶠ (R ⇒ S) (T ⇒ U) = <:-trans <:-function-∪ (<:-function (∩ⁿ-<:-∩ R T) (∪-<:-∪ⁿ S U)) +∪-<:-∪ᶠ (R ⇒ S) (G ∩ H) = <:-trans <:-∪-distl-∩ (<:-intersect (∪-<:-∪ᶠ (R ⇒ S) G) (∪-<:-∪ᶠ (R ⇒ S) H)) +∪-<:-∪ᶠ (E ∩ F) G = <:-trans <:-∪-distr-∩ (<:-intersect (∪-<:-∪ᶠ E G) (∪-<:-∪ᶠ F G)) + +∪ⁿˢ-<:-∪ S never = <:-∪-left +∪ⁿˢ-<:-∪ never number = <:-refl +∪ⁿˢ-<:-∪ never boolean = <:-refl +∪ⁿˢ-<:-∪ never string = <:-refl +∪ⁿˢ-<:-∪ never nil = <:-refl +∪ⁿˢ-<:-∪ unknown number = <:-∪-left +∪ⁿˢ-<:-∪ unknown boolean = <:-∪-left +∪ⁿˢ-<:-∪ unknown string = <:-∪-left +∪ⁿˢ-<:-∪ unknown nil = <:-∪-left +∪ⁿˢ-<:-∪ (R ⇒ S) number = <:-refl +∪ⁿˢ-<:-∪ (R ⇒ S) boolean = <:-refl +∪ⁿˢ-<:-∪ (R ⇒ S) string = <:-refl +∪ⁿˢ-<:-∪ (R ⇒ S) nil = <:-refl +∪ⁿˢ-<:-∪ (R ∩ S) number = <:-refl +∪ⁿˢ-<:-∪ (R ∩ S) boolean = <:-refl +∪ⁿˢ-<:-∪ (R ∩ S) string = <:-refl +∪ⁿˢ-<:-∪ (R ∩ S) nil = <:-refl +∪ⁿˢ-<:-∪ (R ∪ number) number = <:-union <:-∪-left <:-refl +∪ⁿˢ-<:-∪ (R ∪ boolean) number = <:-trans (<:-union (∪ⁿˢ-<:-∪ R number) <:-refl) flipper +∪ⁿˢ-<:-∪ (R ∪ string) number = <:-trans (<:-union (∪ⁿˢ-<:-∪ R number) <:-refl) flipper +∪ⁿˢ-<:-∪ (R ∪ nil) number = <:-trans (<:-union (∪ⁿˢ-<:-∪ R number) <:-refl) flipper +∪ⁿˢ-<:-∪ (R ∪ number) boolean = <:-trans (<:-union (∪ⁿˢ-<:-∪ R boolean) <:-refl) flipper +∪ⁿˢ-<:-∪ (R ∪ boolean) boolean = <:-union <:-∪-left <:-refl +∪ⁿˢ-<:-∪ (R ∪ string) boolean = <:-trans (<:-union (∪ⁿˢ-<:-∪ R boolean) <:-refl) flipper +∪ⁿˢ-<:-∪ (R ∪ nil) boolean = <:-trans (<:-union (∪ⁿˢ-<:-∪ R boolean) <:-refl) flipper +∪ⁿˢ-<:-∪ (R ∪ number) string = <:-trans (<:-union (∪ⁿˢ-<:-∪ R string) <:-refl) flipper +∪ⁿˢ-<:-∪ (R ∪ boolean) string = <:-trans (<:-union (∪ⁿˢ-<:-∪ R string) <:-refl) flipper +∪ⁿˢ-<:-∪ (R ∪ string) string = <:-union <:-∪-left <:-refl +∪ⁿˢ-<:-∪ (R ∪ nil) string = <:-trans (<:-union (∪ⁿˢ-<:-∪ R string) <:-refl) flipper +∪ⁿˢ-<:-∪ (R ∪ number) nil = <:-trans (<:-union (∪ⁿˢ-<:-∪ R nil) <:-refl) flipper +∪ⁿˢ-<:-∪ (R ∪ boolean) nil = <:-trans (<:-union (∪ⁿˢ-<:-∪ R nil) <:-refl) flipper +∪ⁿˢ-<:-∪ (R ∪ string) nil = <:-trans (<:-union (∪ⁿˢ-<:-∪ R nil) <:-refl) flipper +∪ⁿˢ-<:-∪ (R ∪ nil) nil = <:-union <:-∪-left <:-refl + +∪-<:-∪ⁿˢ T never = <:-∪-lub <:-refl <:-never +∪-<:-∪ⁿˢ never number = <:-refl +∪-<:-∪ⁿˢ never boolean = <:-refl +∪-<:-∪ⁿˢ never string = <:-refl +∪-<:-∪ⁿˢ never nil = <:-refl +∪-<:-∪ⁿˢ unknown number = <:-unknown +∪-<:-∪ⁿˢ unknown boolean = <:-unknown +∪-<:-∪ⁿˢ unknown string = <:-unknown +∪-<:-∪ⁿˢ unknown nil = <:-unknown +∪-<:-∪ⁿˢ (R ⇒ S) number = <:-refl +∪-<:-∪ⁿˢ (R ⇒ S) boolean = <:-refl +∪-<:-∪ⁿˢ (R ⇒ S) string = <:-refl +∪-<:-∪ⁿˢ (R ⇒ S) nil = <:-refl +∪-<:-∪ⁿˢ (R ∩ S) number = <:-refl +∪-<:-∪ⁿˢ (R ∩ S) boolean = <:-refl +∪-<:-∪ⁿˢ (R ∩ S) string = <:-refl +∪-<:-∪ⁿˢ (R ∩ S) nil = <:-refl +∪-<:-∪ⁿˢ (R ∪ number) number = <:-∪-lub <:-refl <:-∪-right +∪-<:-∪ⁿˢ (R ∪ boolean) number = <:-trans flipper (<:-union (∪-<:-∪ⁿˢ R number) <:-refl) +∪-<:-∪ⁿˢ (R ∪ string) number = <:-trans flipper (<:-union (∪-<:-∪ⁿˢ R number) <:-refl) +∪-<:-∪ⁿˢ (R ∪ nil) number = <:-trans flipper (<:-union (∪-<:-∪ⁿˢ R number) <:-refl) +∪-<:-∪ⁿˢ (R ∪ number) boolean = <:-trans flipper (<:-union (∪-<:-∪ⁿˢ R boolean) <:-refl) +∪-<:-∪ⁿˢ (R ∪ boolean) boolean = <:-∪-lub <:-refl <:-∪-right +∪-<:-∪ⁿˢ (R ∪ string) boolean = <:-trans flipper (<:-union (∪-<:-∪ⁿˢ R boolean) <:-refl) +∪-<:-∪ⁿˢ (R ∪ nil) boolean = <:-trans flipper (<:-union (∪-<:-∪ⁿˢ R boolean) <:-refl) +∪-<:-∪ⁿˢ (R ∪ number) string = <:-trans flipper (<:-union (∪-<:-∪ⁿˢ R string) <:-refl) +∪-<:-∪ⁿˢ (R ∪ boolean) string = <:-trans flipper (<:-union (∪-<:-∪ⁿˢ R string) <:-refl) +∪-<:-∪ⁿˢ (R ∪ string) string = <:-∪-lub <:-refl <:-∪-right +∪-<:-∪ⁿˢ (R ∪ nil) string = <:-trans flipper (<:-union (∪-<:-∪ⁿˢ R string) <:-refl) +∪-<:-∪ⁿˢ (R ∪ number) nil = <:-trans flipper (<:-union (∪-<:-∪ⁿˢ R nil) <:-refl) +∪-<:-∪ⁿˢ (R ∪ boolean) nil = <:-trans flipper (<:-union (∪-<:-∪ⁿˢ R nil) <:-refl) +∪-<:-∪ⁿˢ (R ∪ string) nil = <:-trans flipper (<:-union (∪-<:-∪ⁿˢ R nil) <:-refl) +∪-<:-∪ⁿˢ (R ∪ nil) nil = <:-∪-lub <:-refl <:-∪-right + +∪ⁿ-<:-∪ S never = <:-∪-left +∪ⁿ-<:-∪ S unknown = <:-∪-right +∪ⁿ-<:-∪ never (T ⇒ U) = <:-∪-right +∪ⁿ-<:-∪ unknown (T ⇒ U) = <:-∪-left +∪ⁿ-<:-∪ (R ⇒ S) (T ⇒ U) = ∪ᶠ-<:-∪ (R ⇒ S) (T ⇒ U) +∪ⁿ-<:-∪ (R ∩ S) (T ⇒ U) = ∪ᶠ-<:-∪ (R ∩ S) (T ⇒ U) +∪ⁿ-<:-∪ (R ∪ S) (T ⇒ U) = <:-trans (<:-union (∪ⁿ-<:-∪ R (T ⇒ U)) <:-refl) (<:-∪-lub (<:-∪-lub (<:-trans <:-∪-left <:-∪-left) <:-∪-right) (<:-trans <:-∪-right <:-∪-left)) +∪ⁿ-<:-∪ never (T ∩ U) = <:-∪-right +∪ⁿ-<:-∪ unknown (T ∩ U) = <:-∪-left +∪ⁿ-<:-∪ (R ⇒ S) (T ∩ U) = ∪ᶠ-<:-∪ (R ⇒ S) (T ∩ U) +∪ⁿ-<:-∪ (R ∩ S) (T ∩ U) = ∪ᶠ-<:-∪ (R ∩ S) (T ∩ U) +∪ⁿ-<:-∪ (R ∪ S) (T ∩ U) = <:-trans (<:-union (∪ⁿ-<:-∪ R (T ∩ U)) <:-refl) (<:-∪-lub (<:-∪-lub (<:-trans <:-∪-left <:-∪-left) <:-∪-right) (<:-trans <:-∪-right <:-∪-left)) +∪ⁿ-<:-∪ S (T ∪ U) = <:-∪-lub (<:-trans (∪ⁿ-<:-∪ S T) (<:-union <:-refl <:-∪-left)) (<:-trans <:-∪-right <:-∪-right) + +∪-<:-∪ⁿ S never = <:-∪-lub <:-refl <:-never +∪-<:-∪ⁿ S unknown = <:-unknown +∪-<:-∪ⁿ never (T ⇒ U) = <:-∪-lub <:-never <:-refl +∪-<:-∪ⁿ unknown (T ⇒ U) = <:-unknown +∪-<:-∪ⁿ (R ⇒ S) (T ⇒ U) = ∪-<:-∪ᶠ (R ⇒ S) (T ⇒ U) +∪-<:-∪ⁿ (R ∩ S) (T ⇒ U) = ∪-<:-∪ᶠ (R ∩ S) (T ⇒ U) +∪-<:-∪ⁿ (R ∪ S) (T ⇒ U) = <:-trans <:-∪-assocr (<:-trans (<:-union <:-refl <:-∪-symm) (<:-trans <:-∪-assocl (<:-union (∪-<:-∪ⁿ R (T ⇒ U)) <:-refl))) +∪-<:-∪ⁿ never (T ∩ U) = <:-∪-lub <:-never <:-refl +∪-<:-∪ⁿ unknown (T ∩ U) = <:-unknown +∪-<:-∪ⁿ (R ⇒ S) (T ∩ U) = ∪-<:-∪ᶠ (R ⇒ S) (T ∩ U) +∪-<:-∪ⁿ (R ∩ S) (T ∩ U) = ∪-<:-∪ᶠ (R ∩ S) (T ∩ U) +∪-<:-∪ⁿ (R ∪ S) (T ∩ U) = <:-trans <:-∪-assocr (<:-trans (<:-union <:-refl <:-∪-symm) (<:-trans <:-∪-assocl (<:-union (∪-<:-∪ⁿ R (T ∩ U)) <:-refl))) +∪-<:-∪ⁿ never (T ∪ U) = <:-trans <:-∪-assocl (<:-union (∪-<:-∪ⁿ never T) <:-refl) +∪-<:-∪ⁿ unknown (T ∪ U) = <:-trans <:-∪-assocl (<:-union (∪-<:-∪ⁿ unknown T) <:-refl) +∪-<:-∪ⁿ (R ⇒ S) (T ∪ U) = <:-trans <:-∪-assocl (<:-union (∪-<:-∪ⁿ (R ⇒ S) T) <:-refl) +∪-<:-∪ⁿ (R ∩ S) (T ∪ U) = <:-trans <:-∪-assocl (<:-union (∪-<:-∪ⁿ (R ∩ S) T) <:-refl) +∪-<:-∪ⁿ (R ∪ S) (T ∪ U) = <:-trans <:-∪-assocl (<:-union (∪-<:-∪ⁿ (R ∪ S) T) <:-refl) + +normalize-<: : ∀ T → normalize T <: T +<:-normalize : ∀ T → T <: normalize T + +<:-normalize nil = <:-∪-right +<:-normalize (S ⇒ T) = <:-function (normalize-<: S) (<:-normalize T) +<:-normalize never = <:-refl +<:-normalize unknown = <:-refl +<:-normalize boolean = <:-∪-right +<:-normalize number = <:-∪-right +<:-normalize string = <:-∪-right +<:-normalize (S ∪ T) = <:-trans (<:-union (<:-normalize S) (<:-normalize T)) (∪-<:-∪ⁿ (normal S) (normal T)) +<:-normalize (S ∩ T) = <:-trans (<:-intersect (<:-normalize S) (<:-normalize T)) (∩-<:-∩ⁿ (normal S) (normal T)) + +normalize-<: nil = <:-∪-lub <:-never <:-refl +normalize-<: (S ⇒ T) = <:-function (<:-normalize S) (normalize-<: T) +normalize-<: never = <:-refl +normalize-<: unknown = <:-refl +normalize-<: boolean = <:-∪-lub <:-never <:-refl +normalize-<: number = <:-∪-lub <:-never <:-refl +normalize-<: string = <:-∪-lub <:-never <:-refl +normalize-<: (S ∪ T) = <:-trans (∪ⁿ-<:-∪ (normal S) (normal T)) (<:-union (normalize-<: S) (normalize-<: T)) +normalize-<: (S ∩ T) = <:-trans (∩ⁿ-<:-∩ (normal S) (normal T)) (<:-intersect (normalize-<: S) (normalize-<: T)) + + From 4d9ac7db1e49a3bfdc96ed623d19fa966aa1920b Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Thu, 28 Apr 2022 18:04:52 -0700 Subject: [PATCH 229/399] Sync to upstream/release/525 --- Analysis/src/Frontend.cpp | 2 +- Analysis/src/Linter.cpp | 6 +- Analysis/src/Substitution.cpp | 2 +- Analysis/src/TypeInfer.cpp | 211 ++++++------------ Analysis/src/TypeVar.cpp | 11 +- Analysis/src/Unifier.cpp | 89 +++----- Ast/src/Lexer.cpp | 4 +- CLI/FileUtils.cpp | 4 +- CLI/Repl.cpp | 9 +- Compiler/include/Luau/BytecodeBuilder.h | 2 +- Compiler/src/BytecodeBuilder.cpp | 24 ++- Compiler/src/Compiler.cpp | 8 +- Compiler/src/CostModel.cpp | 2 +- Sources.cmake | 3 +- VM/include/lua.h | 2 +- VM/src/lapi.cpp | 2 +- VM/src/lstate.h | 2 +- VM/src/ltable.cpp | 55 ++--- VM/src/ludata.cpp | 13 +- fuzz/proto.cpp | 27 ++- tests/Autocomplete.test.cpp | 1 - tests/Compiler.test.cpp | 50 +++-- tests/Conformance.test.cpp | 2 +- tests/Frontend.test.cpp | 18 +- tests/Module.test.cpp | 1 - tests/NonstrictMode.test.cpp | 1 - tests/Parser.test.cpp | 3 - tests/RuntimeLimits.test.cpp | 270 ++++++++++++++++++++++++ tests/TypeInfer.aliases.test.cpp | 19 +- tests/TypeInfer.annotations.test.cpp | 4 - tests/TypeInfer.functions.test.cpp | 7 - tests/TypeInfer.generics.test.cpp | 24 +-- tests/TypeInfer.loops.test.cpp | 18 ++ tests/TypeInfer.operators.test.cpp | 4 - tests/TypeInfer.primitives.test.cpp | 2 - tests/TypeInfer.provisional.test.cpp | 236 --------------------- tests/TypeInfer.singletons.test.cpp | 16 -- tests/TypeInfer.tables.test.cpp | 10 - tests/TypeInfer.tryUnify.test.cpp | 2 - tests/TypeVar.test.cpp | 2 - 40 files changed, 527 insertions(+), 641 deletions(-) create mode 100644 tests/RuntimeLimits.test.cpp diff --git a/Analysis/src/Frontend.cpp b/Analysis/src/Frontend.cpp index 34ccdac4..b8f7836d 100644 --- a/Analysis/src/Frontend.cpp +++ b/Analysis/src/Frontend.cpp @@ -24,7 +24,7 @@ LUAU_FASTFLAGVARIABLE(LuauKnowsTheDataModel3, false) LUAU_FASTFLAGVARIABLE(LuauSeparateTypechecks, false) LUAU_FASTFLAGVARIABLE(LuauAutocompleteDynamicLimits, false) LUAU_FASTFLAGVARIABLE(LuauDirtySourceModule, false) -LUAU_FASTINTVARIABLE(LuauAutocompleteCheckTimeoutMs, 0) +LUAU_FASTINTVARIABLE(LuauAutocompleteCheckTimeoutMs, 100) namespace Luau { diff --git a/Analysis/src/Linter.cpp b/Analysis/src/Linter.cpp index 5608e4b3..200b7d1b 100644 --- a/Analysis/src/Linter.cpp +++ b/Analysis/src/Linter.cpp @@ -2653,12 +2653,12 @@ static void lintComments(LintContext& context, const std::vector& ho } else { - std::string::size_type space = hc.content.find_first_of(" \t"); + size_t space = hc.content.find_first_of(" \t"); std::string_view first = std::string_view(hc.content).substr(0, space); if (first == "nolint") { - std::string::size_type notspace = hc.content.find_first_not_of(" \t", space); + size_t notspace = hc.content.find_first_not_of(" \t", space); if (space == std::string::npos || notspace == std::string::npos) { @@ -2827,7 +2827,7 @@ uint64_t LintWarning::parseMask(const std::vector& hotcomments) if (hc.content.compare(0, 6, "nolint") != 0) continue; - std::string::size_type name = hc.content.find_first_not_of(" \t", 6); + size_t name = hc.content.find_first_not_of(" \t", 6); // --!nolint disables everything if (name == std::string::npos) diff --git a/Analysis/src/Substitution.cpp b/Analysis/src/Substitution.cpp index 1b51fa3d..30d8574a 100644 --- a/Analysis/src/Substitution.cpp +++ b/Analysis/src/Substitution.cpp @@ -8,7 +8,7 @@ #include LUAU_FASTFLAG(LuauLowerBoundsCalculation) -LUAU_FASTINTVARIABLE(LuauTarjanChildLimit, 1000) +LUAU_FASTINTVARIABLE(LuauTarjanChildLimit, 10000) LUAU_FASTFLAG(LuauTypecheckOptPass) LUAU_FASTFLAGVARIABLE(LuauSubstituteFollowNewTypes, false) LUAU_FASTFLAGVARIABLE(LuauSubstituteFollowPossibleMutations, false) diff --git a/Analysis/src/TypeInfer.cpp b/Analysis/src/TypeInfer.cpp index 6411e2ab..ba91ae1e 100644 --- a/Analysis/src/TypeInfer.cpp +++ b/Analysis/src/TypeInfer.cpp @@ -22,29 +22,25 @@ #include LUAU_FASTFLAGVARIABLE(DebugLuauMagicTypes, false) -LUAU_FASTINTVARIABLE(LuauTypeInferRecursionLimit, 500) -LUAU_FASTINTVARIABLE(LuauTypeInferIterationLimit, 2000) +LUAU_FASTINTVARIABLE(LuauTypeInferRecursionLimit, 165) +LUAU_FASTINTVARIABLE(LuauTypeInferIterationLimit, 20000) LUAU_FASTINTVARIABLE(LuauTypeInferTypePackLoopLimit, 5000) -LUAU_FASTINTVARIABLE(LuauCheckRecursionLimit, 500) +LUAU_FASTINTVARIABLE(LuauCheckRecursionLimit, 300) LUAU_FASTFLAG(LuauKnowsTheDataModel3) LUAU_FASTFLAG(LuauSeparateTypechecks) LUAU_FASTFLAG(LuauAutocompleteDynamicLimits) LUAU_FASTFLAG(LuauAutocompleteSingletonTypes) LUAU_FASTFLAGVARIABLE(LuauCyclicModuleTypeSurface, false) +LUAU_FASTFLAGVARIABLE(LuauDoNotRelyOnNextBinding, false) LUAU_FASTFLAGVARIABLE(LuauEqConstraint, false) LUAU_FASTFLAGVARIABLE(LuauWeakEqConstraint, false) // Eventually removed as false. LUAU_FASTFLAGVARIABLE(LuauLowerBoundsCalculation, false) LUAU_FASTFLAGVARIABLE(DebugLuauFreezeDuringUnification, false) -LUAU_FASTFLAGVARIABLE(LuauRecursiveTypeParameterRestriction, false) -LUAU_FASTFLAGVARIABLE(LuauGenericFunctionsDontCacheTypeParams, false) LUAU_FASTFLAGVARIABLE(LuauInferStatFunction, false) -LUAU_FASTFLAGVARIABLE(LuauSealExports, false) +LUAU_FASTFLAGVARIABLE(LuauInstantiateFollows, false) LUAU_FASTFLAGVARIABLE(LuauSelfCallAutocompleteFix, false) LUAU_FASTFLAGVARIABLE(LuauDiscriminableUnions2, false) -LUAU_FASTFLAGVARIABLE(LuauExpectedTypesOfProperties, false) -LUAU_FASTFLAGVARIABLE(LuauErrorRecoveryType, false) LUAU_FASTFLAGVARIABLE(LuauOnlyMutateInstantiatedTables, false) -LUAU_FASTFLAGVARIABLE(LuauPropertiesGetExpectedType, false) LUAU_FASTFLAGVARIABLE(LuauStatFunctionSimplify4, false) LUAU_FASTFLAGVARIABLE(LuauTypecheckOptPass, false) LUAU_FASTFLAGVARIABLE(LuauUnsealedTableLiteral, false) @@ -54,12 +50,9 @@ LUAU_FASTFLAGVARIABLE(LuauReturnAnyInsteadOfICE, false) // Eventually removed as LUAU_FASTFLAG(LuauWidenIfSupertypeIsFree2) LUAU_FASTFLAGVARIABLE(LuauDoNotTryToReduce, false) LUAU_FASTFLAGVARIABLE(LuauDoNotAccidentallyDependOnPointerOrdering, false) -LUAU_FASTFLAGVARIABLE(LuauFixArgumentCountMismatchAmountWithGenericTypes, false) -LUAU_FASTFLAGVARIABLE(LuauFixIncorrectLineNumberDuplicateType, false) LUAU_FASTFLAGVARIABLE(LuauCheckImplicitNumbericKeys, false) LUAU_FASTFLAG(LuauAnyInIsOptionalIsOptional) LUAU_FASTFLAGVARIABLE(LuauDecoupleOperatorInferenceFromUnifiedTypeInference, false) -LUAU_FASTFLAGVARIABLE(LuauArgCountMismatchSaysAtLeastWhenVariadic, false) LUAU_FASTFLAGVARIABLE(LuauTableUseCounterInstead, false) LUAU_FASTFLAGVARIABLE(LuauReturnTypeInferenceInNonstrict, false) LUAU_FASTFLAGVARIABLE(LuauRecursionLimitException, false); @@ -1160,7 +1153,10 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatForIn& forin) } else { - iterTy = follow(instantiate(scope, checkExpr(scope, *firstValue).type, firstValue->location)); + if (FFlag::LuauInstantiateFollows) + iterTy = instantiate(scope, checkExpr(scope, *firstValue).type, firstValue->location); + else + iterTy = follow(instantiate(scope, checkExpr(scope, *firstValue).type, firstValue->location)); } const FunctionTypeVar* iterFunc = get(iterTy); @@ -1172,7 +1168,12 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatForIn& forin) unify(varTy, var, forin.location); if (!get(iterTy) && !get(iterTy) && !get(iterTy)) - reportError(TypeError{firstValue->location, TypeMismatch{globalScope->bindings[AstName{"next"}].typeId, iterTy}}); + { + if (FFlag::LuauDoNotRelyOnNextBinding) + reportError(firstValue->location, CannotCallNonFunction{iterTy}); + else + reportError(TypeError{firstValue->location, TypeMismatch{globalScope->bindings[AstName{"next"}].typeId, iterTy}}); + } return check(loopScope, *forin.body); } @@ -1427,8 +1428,7 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatTypeAlias& typealias ftv->forwardedTypeAlias = true; bindingsMap[name] = {std::move(generics), std::move(genericPacks), ty}; - if (FFlag::LuauFixIncorrectLineNumberDuplicateType) - scope->typeAliasLocations[name] = typealias.location; + scope->typeAliasLocations[name] = typealias.location; } } else @@ -2217,7 +2217,7 @@ TypeId TypeChecker::checkExprTable( if (isNonstrictMode() && !getTableType(exprType) && !get(exprType)) exprType = anyType; - if (FFlag::LuauPropertiesGetExpectedType && expectedTable) + if (expectedTable) { auto it = expectedTable->props.find(key->value.data); if (it != expectedTable->props.end()) @@ -2309,9 +2309,8 @@ ExprResult TypeChecker::checkExpr_(const ScopePtr& scope, const AstExprT } } } - else if (FFlag::LuauExpectedTypesOfProperties) - if (const UnionTypeVar* utv = get(follow(*expectedType))) - expectedUnion = utv; + else if (const UnionTypeVar* utv = get(follow(*expectedType))) + expectedUnion = utv; } for (size_t i = 0; i < expr.items.size; ++i) @@ -2334,7 +2333,7 @@ ExprResult TypeChecker::checkExpr_(const ScopePtr& scope, const AstExprT if (auto prop = expectedTable->props.find(key->value.data); prop != expectedTable->props.end()) expectedResultType = prop->second.type; } - else if (FFlag::LuauExpectedTypesOfProperties && expectedUnion) + else if (expectedUnion) { std::vector expectedResultTypes; for (TypeId expectedOption : expectedUnion) @@ -2713,8 +2712,6 @@ TypeId TypeChecker::checkBinaryOperation( { auto name = getIdentifierOfBaseVar(expr.left); reportError(expr.location, CannotInferBinaryOperation{expr.op, name, CannotInferBinaryOperation::Operation}); - if (!FFlag::LuauErrorRecoveryType) - return errorRecoveryType(scope); } } @@ -2754,7 +2751,7 @@ TypeId TypeChecker::checkBinaryOperation( reportErrors(state.errors); bool hasErrors = !state.errors.empty(); - if (FFlag::LuauErrorRecoveryType && hasErrors) + if (hasErrors) { // If there are unification errors, the return type may still be unknown // so we loosen the argument types to see if that helps. @@ -2768,8 +2765,7 @@ TypeId TypeChecker::checkBinaryOperation( if (state.errors.empty()) state.log.commit(); } - - if (!hasErrors) + else { state.log.commit(); } @@ -3196,16 +3192,7 @@ TypeId TypeChecker::checkFunctionName(const ScopePtr& scope, AstExpr& funName, T } else { - if (!ttv) - { - if (!FFlag::LuauErrorRecoveryType && !isTableIntersection(lhsType)) - // This error now gets reported when we check the function body. - reportError(TypeError{funName.location, OnlyTablesCanHaveMethods{lhsType}}); - - return errorRecoveryType(scope); - } - - if (lhsType->persistent || ttv->state == TableState::Sealed) + if (!ttv || lhsType->persistent || ttv->state == TableState::Sealed) return errorRecoveryType(scope); } @@ -3532,32 +3519,6 @@ ExprResult TypeChecker::checkExprPack(const ScopePtr& scope, const A } // Returns the minimum number of arguments the argument list can accept. -static size_t getMinParameterCount_DEPRECATED(TypePackId tp) -{ - size_t minCount = 0; - size_t optionalCount = 0; - - auto it = begin(tp); - auto endIter = end(tp); - - while (it != endIter) - { - TypeId ty = *it; - if (isOptional(ty)) - ++optionalCount; - else - { - minCount += optionalCount; - optionalCount = 0; - minCount++; - } - - ++it; - } - - return minCount; -} - static size_t getMinParameterCount(TxnLog* log, TypePackId tp) { size_t minCount = 0; @@ -3597,19 +3558,14 @@ void TypeChecker::checkArgumentList( size_t paramIndex = 0; - size_t minParams = FFlag::LuauFixIncorrectLineNumberDuplicateType ? 0 : getMinParameterCount_DEPRECATED(paramPack); - - auto reportCountMismatchError = [&state, &argLocations, minParams, paramPack, argPack]() { + auto reportCountMismatchError = [&state, &argLocations, paramPack, argPack]() { // For this case, we want the error span to cover every errant extra parameter Location location = state.location; if (!argLocations.empty()) location = {state.location.begin, argLocations.back().end}; - size_t mp = minParams; - if (FFlag::LuauFixArgumentCountMismatchAmountWithGenericTypes) - mp = getMinParameterCount(&state.log, paramPack); - - state.reportError(TypeError{location, CountMismatch{mp, std::distance(begin(argPack), end(argPack))}}); + size_t minParams = getMinParameterCount(&state.log, paramPack); + state.reportError(TypeError{location, CountMismatch{minParams, std::distance(begin(argPack), end(argPack))}}); }; while (true) @@ -3707,16 +3663,10 @@ void TypeChecker::checkArgumentList( } // ok else { - if (FFlag::LuauFixArgumentCountMismatchAmountWithGenericTypes) - minParams = getMinParameterCount(&state.log, paramPack); + size_t minParams = getMinParameterCount(&state.log, paramPack); - bool isVariadic = false; - if (FFlag::LuauArgCountMismatchSaysAtLeastWhenVariadic) - { - std::optional tail = flatten(paramPack, state.log).second; - if (tail) - isVariadic = Luau::isVariadic(*tail); - } + std::optional tail = flatten(paramPack, state.log).second; + bool isVariadic = tail && Luau::isVariadic(*tail); state.reportError(TypeError{state.location, CountMismatch{minParams, paramIndex, CountMismatch::Context::Arg, isVariadic}}); return; @@ -3863,7 +3813,8 @@ ExprResult TypeChecker::checkExprPack(const ScopePtr& scope, const A actualFunctionType = instantiate(scope, functionType, expr.func->location); } - actualFunctionType = follow(actualFunctionType); + if (!FFlag::LuauInstantiateFollows) + actualFunctionType = follow(actualFunctionType); TypePackId retPack; if (FFlag::LuauLowerBoundsCalculation || !FFlag::LuauWidenIfSupertypeIsFree2) @@ -3930,16 +3881,13 @@ ExprResult TypeChecker::checkExprPack(const ScopePtr& scope, const A reportOverloadResolutionError(scope, expr, retPack, argPack, argLocations, overloads, overloadsThatMatchArgCount, errors); - if (FFlag::LuauErrorRecoveryType) - { - const FunctionTypeVar* overload = nullptr; - if (!overloadsThatMatchArgCount.empty()) - overload = get(overloadsThatMatchArgCount[0]); - if (!overload && !overloadsThatDont.empty()) - overload = get(overloadsThatDont[0]); - if (overload) - return {errorRecoveryTypePack(overload->retType)}; - } + const FunctionTypeVar* overload = nullptr; + if (!overloadsThatMatchArgCount.empty()) + overload = get(overloadsThatMatchArgCount[0]); + if (!overload && !overloadsThatDont.empty()) + overload = get(overloadsThatDont[0]); + if (overload) + return {errorRecoveryTypePack(overload->retType)}; return {errorRecoveryTypePack(retPack)}; } @@ -4129,7 +4077,7 @@ std::optional> TypeChecker::checkCallOverload(const Scope if (!argMismatch) overloadsThatMatchArgCount.push_back(fn); - else if (FFlag::LuauErrorRecoveryType) + else overloadsThatDont.push_back(fn); errors.emplace_back(std::move(state.errors), args->head, ftv); @@ -4715,7 +4663,7 @@ bool Anyification::isDirty(TypeId ty) return false; if (const TableTypeVar* ttv = log->getMutable(ty)) - return (ttv->state == TableState::Free || (FFlag::LuauSealExports && ttv->state == TableState::Unsealed)); + return (ttv->state == TableState::Free || ttv->state == TableState::Unsealed); else if (log->getMutable(ty)) return true; else if (get(ty)) @@ -4743,12 +4691,9 @@ TypeId Anyification::clean(TypeId ty) TableTypeVar clone = TableTypeVar{ttv->props, ttv->indexer, ttv->level, TableState::Sealed}; clone.methodDefinitionLocations = ttv->methodDefinitionLocations; clone.definitionModuleName = ttv->definitionModuleName; - if (FFlag::LuauSealExports) - { - clone.name = ttv->name; - clone.syntheticName = ttv->syntheticName; - clone.tags = ttv->tags; - } + clone.name = ttv->name; + clone.syntheticName = ttv->syntheticName; + clone.tags = ttv->tags; TypeId res = addType(std::move(clone)); asMutable(res)->normal = ty->normal; return res; @@ -4791,9 +4736,12 @@ TypeId TypeChecker::quantify(const ScopePtr& scope, TypeId ty, Location location TypeId TypeChecker::instantiate(const ScopePtr& scope, TypeId ty, Location location, const TxnLog* log) { + if (FFlag::LuauInstantiateFollows) + ty = follow(ty); + if (FFlag::LuauTypecheckOptPass) { - const FunctionTypeVar* ftv = get(follow(ty)); + const FunctionTypeVar* ftv = get(FFlag::LuauInstantiateFollows ? ty : follow(ty)); if (ftv && ftv->hasNoGenerics) return ty; } @@ -5175,8 +5123,6 @@ TypeId TypeChecker::resolveType(const ScopePtr& scope, const AstType& annotation { reportError(TypeError{annotation.location, GenericError{"Type parameter list is required"}}); parameterCountErrorReported = true; - if (!FFlag::LuauErrorRecoveryType) - return errorRecoveryType(scope); } } @@ -5294,33 +5240,25 @@ TypeId TypeChecker::resolveType(const ScopePtr& scope, const AstType& annotation reportError( TypeError{annotation.location, IncorrectGenericParameterCount{lit->name.value, *tf, typeParams.size(), typePackParams.size()}}); - if (FFlag::LuauErrorRecoveryType) - { - // Pad the types out with error recovery types - while (typeParams.size() < tf->typeParams.size()) - typeParams.push_back(errorRecoveryType(scope)); - while (typePackParams.size() < tf->typePackParams.size()) - typePackParams.push_back(errorRecoveryTypePack(scope)); - } - else - return errorRecoveryType(scope); + // Pad the types out with error recovery types + while (typeParams.size() < tf->typeParams.size()) + typeParams.push_back(errorRecoveryType(scope)); + while (typePackParams.size() < tf->typePackParams.size()) + typePackParams.push_back(errorRecoveryTypePack(scope)); } - if (FFlag::LuauRecursiveTypeParameterRestriction) - { - bool sameTys = std::equal(typeParams.begin(), typeParams.end(), tf->typeParams.begin(), tf->typeParams.end(), [](auto&& itp, auto&& tp) { - return itp == tp.ty; + bool sameTys = std::equal(typeParams.begin(), typeParams.end(), tf->typeParams.begin(), tf->typeParams.end(), [](auto&& itp, auto&& tp) { + return itp == tp.ty; + }); + bool sameTps = std::equal( + typePackParams.begin(), typePackParams.end(), tf->typePackParams.begin(), tf->typePackParams.end(), [](auto&& itpp, auto&& tpp) { + return itpp == tpp.tp; }); - bool sameTps = std::equal( - typePackParams.begin(), typePackParams.end(), tf->typePackParams.begin(), tf->typePackParams.end(), [](auto&& itpp, auto&& tpp) { - return itpp == tpp.tp; - }); - // If the generic parameters and the type arguments are the same, we are about to - // perform an identity substitution, which we can just short-circuit. - if (sameTys && sameTps) - return tf->type; - } + // If the generic parameters and the type arguments are the same, we are about to + // perform an identity substitution, which we can just short-circuit. + if (sameTys && sameTps) + return tf->type; return instantiateTypeFun(scope, *tf, typeParams, typePackParams, annotation.location); } @@ -5483,7 +5421,7 @@ bool ApplyTypeFunction::isDirty(TypeId ty) return true; else if (const FreeTypeVar* ftv = get(ty)) { - if (FFlag::LuauRecursiveTypeParameterRestriction && ftv->forwardedTypeAlias) + if (ftv->forwardedTypeAlias) encounteredForwardedType = true; return false; } @@ -5562,7 +5500,7 @@ TypeId TypeChecker::instantiateTypeFun(const ScopePtr& scope, const TypeFun& tf, reportError(location, UnificationTooComplex{}); return errorRecoveryType(scope); } - if (FFlag::LuauRecursiveTypeParameterRestriction && applyTypeFunction.encounteredForwardedType) + if (applyTypeFunction.encounteredForwardedType) { reportError(TypeError{location, GenericError{"Recursive type being used with different parameters"}}); return errorRecoveryType(scope); @@ -5632,7 +5570,7 @@ GenericTypeDefinitions TypeChecker::createGenericTypes(const ScopePtr& scope, st } TypeId g; - if (FFlag::LuauRecursiveTypeParameterRestriction && (!FFlag::LuauGenericFunctionsDontCacheTypeParams || useCache)) + if (useCache) { TypeId& cached = scope->parent->typeAliasTypeParameters[n]; if (!cached) @@ -5667,21 +5605,12 @@ GenericTypeDefinitions TypeChecker::createGenericTypes(const ScopePtr& scope, st reportError(TypeError{node.location, DuplicateGenericParameter{n}}); } - TypePackId g; - if (FFlag::LuauRecursiveTypeParameterRestriction) - { - TypePackId& cached = scope->parent->typeAliasTypePackParameters[n]; - if (!cached) - cached = addTypePack(TypePackVar{Unifiable::Generic{level, n}}); - g = cached; - } - else - { - g = addTypePack(TypePackVar{Unifiable::Generic{level, n}}); - } + TypePackId& cached = scope->parent->typeAliasTypePackParameters[n]; + if (!cached) + cached = addTypePack(TypePackVar{Unifiable::Generic{level, n}}); - genericPacks.push_back({g, defaultValue}); - scope->privateTypePackBindings[n] = g; + genericPacks.push_back({cached, defaultValue}); + scope->privateTypePackBindings[n] = cached; } return {generics, genericPacks}; diff --git a/Analysis/src/TypeVar.cpp b/Analysis/src/TypeVar.cpp index 4d42573c..463b4651 100644 --- a/Analysis/src/TypeVar.cpp +++ b/Analysis/src/TypeVar.cpp @@ -23,7 +23,6 @@ LUAU_FASTFLAG(DebugLuauFreezeArena) LUAU_FASTINTVARIABLE(LuauTypeMaximumStringifierLength, 500) LUAU_FASTINTVARIABLE(LuauTableTypeMaximumStringifierLength, 0) LUAU_FASTINT(LuauTypeInferRecursionLimit) -LUAU_FASTFLAG(LuauErrorRecoveryType) LUAU_FASTFLAG(LuauSubtypingAddOptPropsToUnsealedTables) LUAU_FASTFLAG(LuauDiscriminableUnions2) LUAU_FASTFLAGVARIABLE(LuauAnyInIsOptionalIsOptional, false) @@ -775,18 +774,12 @@ TypePackId SingletonTypes::errorRecoveryTypePack() TypeId SingletonTypes::errorRecoveryType(TypeId guess) { - if (FFlag::LuauErrorRecoveryType) - return guess; - else - return &errorType_; + return guess; } TypePackId SingletonTypes::errorRecoveryTypePack(TypePackId guess) { - if (FFlag::LuauErrorRecoveryType) - return guess; - else - return &errorTypePack_; + return guess; } SingletonTypes& getSingletonTypes() diff --git a/Analysis/src/Unifier.cpp b/Analysis/src/Unifier.cpp index 9862d7b3..334806ce 100644 --- a/Analysis/src/Unifier.cpp +++ b/Analysis/src/Unifier.cpp @@ -23,10 +23,7 @@ LUAU_FASTFLAG(LuauErrorRecoveryType); LUAU_FASTFLAGVARIABLE(LuauSubtypingAddOptPropsToUnsealedTables, false) LUAU_FASTFLAGVARIABLE(LuauWidenIfSupertypeIsFree2, false) LUAU_FASTFLAGVARIABLE(LuauDifferentOrderOfUnificationDoesntMatter, false) -LUAU_FASTFLAGVARIABLE(LuauTxnLogSeesTypePacks2, false) -LUAU_FASTFLAGVARIABLE(LuauTxnLogCheckForInvalidation, false) LUAU_FASTFLAGVARIABLE(LuauTxnLogRefreshFunctionPointers, false) -LUAU_FASTFLAGVARIABLE(LuauTxnLogDontRetryForIndexers, false) LUAU_FASTFLAG(LuauAnyInIsOptionalIsOptional) LUAU_FASTFLAG(LuauTypecheckOptPass) @@ -1021,7 +1018,7 @@ void Unifier::tryUnify_(TypePackId subTp, TypePackId superTp, bool isFunctionCal if (superTp == subTp) return; - if (FFlag::LuauTxnLogSeesTypePacks2 && log.haveSeen(superTp, subTp)) + if (log.haveSeen(superTp, subTp)) return; if (log.getMutable(superTp)) @@ -1265,12 +1262,9 @@ void Unifier::tryUnifyFunctions(TypeId subTy, TypeId superTy, bool isFunctionCal log.pushSeen(superFunction->generics[i], subFunction->generics[i]); } - if (FFlag::LuauTxnLogSeesTypePacks2) + for (size_t i = 0; i < numGenericPacks; i++) { - for (size_t i = 0; i < numGenericPacks; i++) - { - log.pushSeen(superFunction->genericPacks[i], subFunction->genericPacks[i]); - } + log.pushSeen(superFunction->genericPacks[i], subFunction->genericPacks[i]); } CountMismatch::Context context = ctx; @@ -1330,12 +1324,9 @@ void Unifier::tryUnifyFunctions(TypeId subTy, TypeId superTy, bool isFunctionCal ctx = context; - if (FFlag::LuauTxnLogSeesTypePacks2) + for (int i = int(numGenericPacks) - 1; 0 <= i; i--) { - for (int i = int(numGenericPacks) - 1; 0 <= i; i--) - { - log.popSeen(superFunction->genericPacks[i], subFunction->genericPacks[i]); - } + log.popSeen(superFunction->genericPacks[i], subFunction->genericPacks[i]); } for (int i = int(numGenerics) - 1; 0 <= i; i--) @@ -1499,20 +1490,17 @@ void Unifier::tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) else missingProperties.push_back(name); - if (FFlag::LuauTxnLogCheckForInvalidation) + // Recursive unification can change the txn log, and invalidate the old + // table. If we detect that this has happened, we start over, with the updated + // txn log. + TableTypeVar* newSuperTable = log.getMutable(superTy); + TableTypeVar* newSubTable = log.getMutable(subTy); + if (superTable != newSuperTable || subTable != newSubTable) { - // Recursive unification can change the txn log, and invalidate the old - // table. If we detect that this has happened, we start over, with the updated - // txn log. - TableTypeVar* newSuperTable = log.getMutable(superTy); - TableTypeVar* newSubTable = log.getMutable(subTy); - if (superTable != newSuperTable || subTable != newSubTable) - { - if (errors.empty()) - return tryUnifyTables(subTy, superTy, isIntersection); - else - return; - } + if (errors.empty()) + return tryUnifyTables(subTy, superTy, isIntersection); + else + return; } } @@ -1570,20 +1558,17 @@ void Unifier::tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) else extraProperties.push_back(name); - if (FFlag::LuauTxnLogCheckForInvalidation) + // Recursive unification can change the txn log, and invalidate the old + // table. If we detect that this has happened, we start over, with the updated + // txn log. + TableTypeVar* newSuperTable = log.getMutable(superTy); + TableTypeVar* newSubTable = log.getMutable(subTy); + if (superTable != newSuperTable || subTable != newSubTable) { - // Recursive unification can change the txn log, and invalidate the old - // table. If we detect that this has happened, we start over, with the updated - // txn log. - TableTypeVar* newSuperTable = log.getMutable(superTy); - TableTypeVar* newSubTable = log.getMutable(subTy); - if (superTable != newSuperTable || subTable != newSubTable) - { - if (errors.empty()) - return tryUnifyTables(subTy, superTy, isIntersection); - else - return; - } + if (errors.empty()) + return tryUnifyTables(subTy, superTy, isIntersection); + else + return; } } @@ -1630,27 +1615,9 @@ void Unifier::tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) } } - if (FFlag::LuauTxnLogDontRetryForIndexers) - { - // Changing the indexer can invalidate the table pointers. - superTable = log.getMutable(superTy); - subTable = log.getMutable(subTy); - } - else if (FFlag::LuauTxnLogCheckForInvalidation) - { - // Recursive unification can change the txn log, and invalidate the old - // table. If we detect that this has happened, we start over, with the updated - // txn log. - TableTypeVar* newSuperTable = log.getMutable(superTy); - TableTypeVar* newSubTable = log.getMutable(subTy); - if (superTable != newSuperTable || subTable != newSubTable) - { - if (errors.empty()) - return tryUnifyTables(subTy, superTy, isIntersection); - else - return; - } - } + // Changing the indexer can invalidate the table pointers. + superTable = log.getMutable(superTy); + subTable = log.getMutable(subTy); if (!missingProperties.empty()) { diff --git a/Ast/src/Lexer.cpp b/Ast/src/Lexer.cpp index 5dd4f04e..a1f1d469 100644 --- a/Ast/src/Lexer.cpp +++ b/Ast/src/Lexer.cpp @@ -6,8 +6,6 @@ #include -LUAU_FASTFLAGVARIABLE(LuauParseLocationIgnoreCommentSkip, false) - namespace Luau { @@ -361,7 +359,7 @@ const Lexeme& Lexer::next(bool skipComments, bool updatePrevLocation) while (isSpace(peekch())) consume(); - if (!FFlag::LuauParseLocationIgnoreCommentSkip || updatePrevLocation) + if (updatePrevLocation) prevLocation = lexeme.location; lexeme = readNext(); diff --git a/CLI/FileUtils.cpp b/CLI/FileUtils.cpp index fb6ac373..39a14ec7 100644 --- a/CLI/FileUtils.cpp +++ b/CLI/FileUtils.cpp @@ -240,7 +240,7 @@ std::optional getParentPath(const std::string& path) return std::nullopt; #endif - std::string::size_type slash = path.find_last_of("\\/", path.size() - 1); + size_t slash = path.find_last_of("\\/", path.size() - 1); if (slash == 0) return "/"; @@ -253,7 +253,7 @@ std::optional getParentPath(const std::string& path) static std::string getExtension(const std::string& path) { - std::string::size_type dot = path.find_last_of(".\\/"); + size_t dot = path.find_last_of(".\\/"); if (dot == std::string::npos || path[dot] != '.') return ""; diff --git a/CLI/Repl.cpp b/CLI/Repl.cpp index 345cb7ac..4cb22346 100644 --- a/CLI/Repl.cpp +++ b/CLI/Repl.cpp @@ -34,7 +34,8 @@ enum class CliMode enum class CompileFormat { Text, - Binary + Binary, + Null }; constexpr int MaxTraversalLimit = 50; @@ -594,6 +595,8 @@ static bool compileFile(const char* name, CompileFormat format) case CompileFormat::Binary: fwrite(bcb.getBytecode().data(), 1, bcb.getBytecode().size(), stdout); break; + case CompileFormat::Null: + break; } return true; @@ -716,6 +719,10 @@ int replMain(int argc, char** argv) { compileFormat = CompileFormat::Text; } + else if (strcmp(argv[1], "--compile=null") == 0) + { + compileFormat = CompileFormat::Null; + } else { fprintf(stderr, "Error: Unrecognized value for '--compile' specified.\n"); diff --git a/Compiler/include/Luau/BytecodeBuilder.h b/Compiler/include/Luau/BytecodeBuilder.h index 67b93028..b00440ae 100644 --- a/Compiler/include/Luau/BytecodeBuilder.h +++ b/Compiler/include/Luau/BytecodeBuilder.h @@ -232,7 +232,7 @@ private: DenseHashMap stringTable; - DenseHashMap debugRemarks; + std::vector> debugRemarks; std::string debugRemarkBuffer; BytecodeEncoder* encoder = nullptr; diff --git a/Compiler/src/BytecodeBuilder.cpp b/Compiler/src/BytecodeBuilder.cpp index 6c6f1225..871a1484 100644 --- a/Compiler/src/BytecodeBuilder.cpp +++ b/Compiler/src/BytecodeBuilder.cpp @@ -181,7 +181,6 @@ BytecodeBuilder::BytecodeBuilder(BytecodeEncoder* encoder) : constantMap({Constant::Type_Nil, ~0ull}) , tableShapeMap(TableShape()) , stringTable({nullptr, 0}) - , debugRemarks(~0u) , encoder(encoder) { LUAU_ASSERT(stringTable.find(StringRef{"", 0}) == nullptr); @@ -257,6 +256,8 @@ void BytecodeBuilder::endFunction(uint8_t maxstacksize, uint8_t numupvalues) void BytecodeBuilder::setMainFunction(uint32_t fid) { + LUAU_ASSERT(fid < functions.size()); + mainFunction = fid; } @@ -531,7 +532,7 @@ void BytecodeBuilder::addDebugRemark(const char* format, ...) // we null-terminate all remarks to avoid storing remark length debugRemarkBuffer += '\0'; - debugRemarks[uint32_t(insns.size())] = uint32_t(offset); + debugRemarks.emplace_back(uint32_t(insns.size()), uint32_t(offset)); } void BytecodeBuilder::finalize() @@ -1719,6 +1720,7 @@ std::string BytecodeBuilder::dumpCurrentFunction() const const uint32_t* codeEnd = insns.data() + insns.size(); int lastLine = -1; + size_t nextRemark = 0; std::string result; @@ -1741,6 +1743,7 @@ std::string BytecodeBuilder::dumpCurrentFunction() const while (code != codeEnd) { uint8_t op = LUAU_INSN_OP(*code); + uint32_t pc = uint32_t(code - insns.data()); if (op == LOP_PREPVARARGS) { @@ -1751,15 +1754,16 @@ std::string BytecodeBuilder::dumpCurrentFunction() const if (dumpFlags & Dump_Remarks) { - const uint32_t* remark = debugRemarks.find(uint32_t(code - insns.data())); - - if (remark) - formatAppend(result, "REMARK %s\n", debugRemarkBuffer.c_str() + *remark); + while (nextRemark < debugRemarks.size() && debugRemarks[nextRemark].first == pc) + { + formatAppend(result, "REMARK %s\n", debugRemarkBuffer.c_str() + debugRemarks[nextRemark].second); + nextRemark++; + } } if (dumpFlags & Dump_Source) { - int line = lines[code - insns.data()]; + int line = lines[pc]; if (line > 0 && line != lastLine) { @@ -1771,7 +1775,7 @@ std::string BytecodeBuilder::dumpCurrentFunction() const if (dumpFlags & Dump_Lines) { - formatAppend(result, "%d: ", lines[code - insns.data()]); + formatAppend(result, "%d: ", lines[pc]); } code = dumpInstruction(code, result); @@ -1784,11 +1788,11 @@ void BytecodeBuilder::setDumpSource(const std::string& source) { dumpSource.clear(); - std::string::size_type pos = 0; + size_t pos = 0; while (pos != std::string::npos) { - std::string::size_type next = source.find('\n', pos); + size_t next = source.find('\n', pos); if (next == std::string::npos) { diff --git a/Compiler/src/Compiler.cpp b/Compiler/src/Compiler.cpp index 810caaee..0f17ee02 100644 --- a/Compiler/src/Compiler.cpp +++ b/Compiler/src/Compiler.cpp @@ -2206,9 +2206,15 @@ struct Compiler return false; } + if (Variable* lv = variables.find(stat->var); lv && lv->written) + { + bytecode.addDebugRemark("loop unroll failed: mutable loop variable"); + return false; + } + int tripCount = (to - from) / step + 1; - if (tripCount > thresholdBase * thresholdMaxBoost / 100) + if (tripCount > thresholdBase) { bytecode.addDebugRemark("loop unroll failed: too many iterations (%d)", tripCount); return false; diff --git a/Compiler/src/CostModel.cpp b/Compiler/src/CostModel.cpp index d8511bdb..9afd09f6 100644 --- a/Compiler/src/CostModel.cpp +++ b/Compiler/src/CostModel.cpp @@ -249,7 +249,7 @@ int computeCost(uint64_t model, const bool* varsConst, size_t varCount) return cost; for (size_t i = 0; i < varCount && i < 7; ++i) - cost -= int((model >> (8 * i + 8)) & 0x7f) * varsConst[i]; + cost -= int((model >> (i * 8 + 8)) & 0x7f) * varsConst[i]; return cost; } diff --git a/Sources.cmake b/Sources.cmake index 60e5dfda..f9263b24 100644 --- a/Sources.cmake +++ b/Sources.cmake @@ -220,8 +220,8 @@ if(TARGET Luau.UnitTest) tests/Autocomplete.test.cpp tests/BuiltinDefinitions.test.cpp tests/Compiler.test.cpp - tests/CostModel.test.cpp tests/Config.test.cpp + tests/CostModel.test.cpp tests/Error.test.cpp tests/Frontend.test.cpp tests/JsonEncoder.test.cpp @@ -232,6 +232,7 @@ if(TARGET Luau.UnitTest) tests/Normalize.test.cpp tests/Parser.test.cpp tests/RequireTracer.test.cpp + tests/RuntimeLimits.test.cpp tests/StringUtils.test.cpp tests/Symbol.test.cpp tests/ToDot.test.cpp diff --git a/VM/include/lua.h b/VM/include/lua.h index d08b73ea..c3ebadb1 100644 --- a/VM/include/lua.h +++ b/VM/include/lua.h @@ -299,7 +299,7 @@ LUA_API uintptr_t lua_encodepointer(lua_State* L, uintptr_t p); LUA_API double lua_clock(); -LUA_API void lua_setuserdatadtor(lua_State* L, int tag, void (*dtor)(void*)); +LUA_API void lua_setuserdatadtor(lua_State* L, int tag, void (*dtor)(lua_State*, void*)); LUA_API void lua_clonefunction(lua_State* L, int idx); diff --git a/VM/src/lapi.cpp b/VM/src/lapi.cpp index 431f7e59..1f3b0943 100644 --- a/VM/src/lapi.cpp +++ b/VM/src/lapi.cpp @@ -1323,7 +1323,7 @@ void lua_unref(lua_State* L, int ref) return; } -void lua_setuserdatadtor(lua_State* L, int tag, void (*dtor)(void*)) +void lua_setuserdatadtor(lua_State* L, int tag, void (*dtor)(lua_State*, void*)) { api_check(L, unsigned(tag) < LUA_UTAG_LIMIT); L->global->udatagc[tag] = dtor; diff --git a/VM/src/lstate.h b/VM/src/lstate.h index 45d9ba2c..423514a7 100644 --- a/VM/src/lstate.h +++ b/VM/src/lstate.h @@ -200,7 +200,7 @@ typedef struct global_State uint64_t rngstate; /* PCG random number generator state */ uint64_t ptrenckey[4]; /* pointer encoding key for display */ - void (*udatagc[LUA_UTAG_LIMIT])(void*); /* for each userdata tag, a gc callback to be called immediately before freeing memory */ + void (*udatagc[LUA_UTAG_LIMIT])(lua_State*, void*); /* for each userdata tag, a gc callback to be called immediately before freeing memory */ lua_Callbacks cb; diff --git a/VM/src/ltable.cpp b/VM/src/ltable.cpp index dc40b6ef..3dc3bd1b 100644 --- a/VM/src/ltable.cpp +++ b/VM/src/ltable.cpp @@ -33,7 +33,6 @@ #include -LUAU_FASTFLAGVARIABLE(LuauTableRehashRework, false) LUAU_FASTFLAGVARIABLE(LuauTableNewBoundary2, false) // max size of both array and hash part is 2^MAXBITS @@ -400,16 +399,9 @@ static void resize(lua_State* L, Table* t, int nasize, int nhsize) { if (!ttisnil(&t->array[i])) { - if (FFlag::LuauTableRehashRework) - { - TValue ok; - setnvalue(&ok, cast_num(i + 1)); - setobjt2t(L, newkey(L, t, &ok), &t->array[i]); - } - else - { - setobjt2t(L, luaH_setnum(L, t, i + 1), &t->array[i]); - } + TValue ok; + setnvalue(&ok, cast_num(i + 1)); + setobjt2t(L, newkey(L, t, &ok), &t->array[i]); } } /* shrink array */ @@ -418,30 +410,14 @@ static void resize(lua_State* L, Table* t, int nasize, int nhsize) /* used for the migration check at the end */ TValue* anew = t->array; /* re-insert elements from hash part */ - if (FFlag::LuauTableRehashRework) + for (int i = twoto(oldhsize) - 1; i >= 0; i--) { - for (int i = twoto(oldhsize) - 1; i >= 0; i--) + LuaNode* old = nold + i; + if (!ttisnil(gval(old))) { - LuaNode* old = nold + i; - if (!ttisnil(gval(old))) - { - TValue ok; - getnodekey(L, &ok, old); - setobjt2t(L, arrayornewkey(L, t, &ok), gval(old)); - } - } - } - else - { - for (int i = twoto(oldhsize) - 1; i >= 0; i--) - { - LuaNode* old = nold + i; - if (!ttisnil(gval(old))) - { - TValue ok; - getnodekey(L, &ok, old); - setobjt2t(L, luaH_set(L, t, &ok), gval(old)); - } + TValue ok; + getnodekey(L, &ok, old); + setobjt2t(L, arrayornewkey(L, t, &ok), gval(old)); } } @@ -559,7 +535,7 @@ static TValue* newkey(lua_State* L, Table* t, const TValue* key) { rehash(L, t, key); /* grow table */ - // after rehash, numeric keys might be located in the new array part, but won't be found in the node part + /* after rehash, numeric keys might be located in the new array part, but won't be found in the node part */ return arrayornewkey(L, t, key); } @@ -571,15 +547,8 @@ static TValue* newkey(lua_State* L, Table* t, const TValue* key) { /* cannot find a free place? */ rehash(L, t, key); /* grow table */ - if (!FFlag::LuauTableRehashRework) - { - return luaH_set(L, t, key); /* re-insert key into grown table */ - } - else - { - // after rehash, numeric keys might be located in the new array part, but won't be found in the node part - return arrayornewkey(L, t, key); - } + /* after rehash, numeric keys might be located in the new array part, but won't be found in the node part */ + return arrayornewkey(L, t, key); } LUAU_ASSERT(n != dummynode); TValue mk; diff --git a/VM/src/ludata.cpp b/VM/src/ludata.cpp index 819d1863..28152689 100644 --- a/VM/src/ludata.cpp +++ b/VM/src/ludata.cpp @@ -22,14 +22,21 @@ Udata* luaU_newudata(lua_State* L, size_t s, int tag) void luaU_freeudata(lua_State* L, Udata* u, lua_Page* page) { - void (*dtor)(void*) = nullptr; if (u->tag < LUA_UTAG_LIMIT) + { + void (*dtor)(lua_State*, void*) = nullptr; dtor = L->global->udatagc[u->tag]; + if (dtor) + dtor(L, u->data); + } else if (u->tag == UTAG_IDTOR) + { + void (*dtor)(void*) = nullptr; memcpy(&dtor, &u->data + u->len - sizeof(dtor), sizeof(dtor)); + if (dtor) + dtor(u->data); + } - if (dtor) - dtor(u->data); luaM_freegco(L, u, sizeudata(u->len), u->memcat, page); } diff --git a/fuzz/proto.cpp b/fuzz/proto.cpp index a48f068b..22483f9e 100644 --- a/fuzz/proto.cpp +++ b/fuzz/proto.cpp @@ -137,6 +137,21 @@ int registerTypes(Luau::TypeChecker& env) return 0; } + +static void setupFrontend(Luau::Frontend& frontend) +{ + registerTypes(frontend.typeChecker); + Luau::freeze(frontend.typeChecker.globalTypes); + + registerTypes(frontend.typeCheckerForAutocomplete); + Luau::freeze(frontend.typeCheckerForAutocomplete.globalTypes); + + frontend.iceHandler.onInternalError = [](const char* error) { + printf("ICE: %s\n", error); + LUAU_ASSERT(!"ICE"); + }; +} + struct FuzzFileResolver : Luau::FileResolver { std::optional readSource(const Luau::ModuleName& name) override @@ -238,19 +253,11 @@ DEFINE_PROTO_FUZZER(const luau::ModuleSet& message) if (kFuzzTypeck) { static FuzzFileResolver fileResolver; - static Luau::NullConfigResolver configResolver; + static FuzzConfigResolver configResolver; static Luau::FrontendOptions options{true, true}; static Luau::Frontend frontend(&fileResolver, &configResolver, options); - static int once = registerTypes(frontend.typeChecker); - (void)once; - static int once2 = (Luau::freeze(frontend.typeChecker.globalTypes), 0); - (void)once2; - - frontend.iceHandler.onInternalError = [](const char* error) { - printf("ICE: %s\n", error); - LUAU_ASSERT(!"ICE"); - }; + static int once = (setupFrontend(frontend), 0); // restart frontend.clear(); diff --git a/tests/Autocomplete.test.cpp b/tests/Autocomplete.test.cpp index f66e23ed..5b70481b 100644 --- a/tests/Autocomplete.test.cpp +++ b/tests/Autocomplete.test.cpp @@ -2761,7 +2761,6 @@ TEST_CASE_FIXTURE(ACFixture, "autocomplete_on_string_singletons") TEST_CASE_FIXTURE(ACFixture, "autocomplete_string_singletons") { ScopedFastFlag luauAutocompleteSingletonTypes{"LuauAutocompleteSingletonTypes", true}; - ScopedFastFlag luauExpectedTypesOfProperties{"LuauExpectedTypesOfProperties", true}; check(R"( type tag = "cat" | "dog" diff --git a/tests/Compiler.test.cpp b/tests/Compiler.test.cpp index f3e60690..7b4bfc72 100644 --- a/tests/Compiler.test.cpp +++ b/tests/Compiler.test.cpp @@ -2698,16 +2698,22 @@ TEST_CASE("DebugRemarks") uint32_t fid = bcb.beginFunction(0); - bcb.addDebugRemark("test remark #%d", 42); + bcb.addDebugRemark("test remark #%d", 1); + bcb.emitABC(LOP_LOADNIL, 0, 0, 0); + bcb.addDebugRemark("test remark #%d", 2); + bcb.addDebugRemark("test remark #%d", 3); bcb.emitABC(LOP_RETURN, 0, 1, 0); - bcb.endFunction(0, 0); + bcb.endFunction(1, 0); bcb.setMainFunction(fid); bcb.finalize(); CHECK_EQ("\n" + bcb.dumpFunction(0), R"( -REMARK test remark #42 +REMARK test remark #1 +LOADNIL R0 +REMARK test remark #2 +REMARK test remark #3 RETURN R0 0 )"); } @@ -4332,7 +4338,7 @@ RETURN R0 1 // loops with body that's long but has a high boost factor due to constant folding CHECK_EQ("\n" + compileFunction(R"( local t = {} -for i=1,30 do +for i=1,25 do t[i] = i * i * i end return t @@ -4390,16 +4396,6 @@ LOADN R1 13824 SETTABLEN R1 R0 24 LOADN R1 15625 SETTABLEN R1 R0 25 -LOADN R1 17576 -SETTABLEN R1 R0 26 -LOADN R1 19683 -SETTABLEN R1 R0 27 -LOADN R1 21952 -SETTABLEN R1 R0 28 -LOADN R1 24389 -SETTABLEN R1 R0 29 -LOADN R1 27000 -SETTABLEN R1 R0 30 RETURN R0 1 )"); @@ -4431,4 +4427,30 @@ RETURN R0 1 )"); } +TEST_CASE("LoopUnrollMutable") +{ + // can't unroll loops that mutate iteration variable + CHECK_EQ("\n" + compileFunction(R"( +for i=1,3 do + i = 3 + print(i) -- should print 3 three times in a row +end +)", + 0, 2), + R"( +LOADN R2 1 +LOADN R0 3 +LOADN R1 1 +FORNPREP R0 +7 +MOVE R3 R2 +LOADN R3 3 +GETIMPORT R4 1 +MOVE R5 R3 +CALL R4 1 0 +FORNLOOP R0 -7 +RETURN R0 0 +)"); +} + + TEST_SUITE_END(); diff --git a/tests/Conformance.test.cpp b/tests/Conformance.test.cpp index 0ed7dc44..6f136d36 100644 --- a/tests/Conformance.test.cpp +++ b/tests/Conformance.test.cpp @@ -1056,7 +1056,7 @@ TEST_CASE("UserdataApi") lua_State* L = globalState.get(); // setup dtor for tag 42 (created later) - lua_setuserdatadtor(L, 42, [](void* data) { + lua_setuserdatadtor(L, 42, [](lua_State* l, void* data) { dtorhits += *(int*)data; }); diff --git a/tests/Frontend.test.cpp b/tests/Frontend.test.cpp index 9fc0a005..e771b6b1 100644 --- a/tests/Frontend.test.cpp +++ b/tests/Frontend.test.cpp @@ -975,8 +975,6 @@ TEST_CASE_FIXTURE(FrontendFixture, "typecheck_twice_for_ast_types") TEST_CASE_FIXTURE(FrontendFixture, "imported_table_modification_2") { - ScopedFastFlag sffs("LuauSealExports", true); - frontend.options.retainFullTypeGraphs = false; fileResolver.source["Module/A"] = R"( @@ -1035,4 +1033,20 @@ return false; fix.frontend.check("Module/B"); } +TEST_CASE("check_without_builtin_next") +{ + ScopedFastFlag luauDoNotRelyOnNextBinding{"LuauDoNotRelyOnNextBinding", true}; + + TestFileResolver fileResolver; + TestConfigResolver configResolver; + Frontend frontend(&fileResolver, &configResolver); + + fileResolver.source["Module/A"] = "for k,v in 2 do end"; + fileResolver.source["Module/B"] = "return next"; + + // We don't care about the result. That we haven't crashed is enough. + frontend.check("Module/A"); + frontend.check("Module/B"); +} + TEST_SUITE_END(); diff --git a/tests/Module.test.cpp b/tests/Module.test.cpp index af7d76de..44cc20a7 100644 --- a/tests/Module.test.cpp +++ b/tests/Module.test.cpp @@ -199,7 +199,6 @@ TEST_CASE_FIXTURE(Fixture, "clone_class") TEST_CASE_FIXTURE(Fixture, "clone_free_types") { ScopedFastFlag sff[]{ - {"LuauErrorRecoveryType", true}, {"LuauLosslessClone", true}, }; diff --git a/tests/NonstrictMode.test.cpp b/tests/NonstrictMode.test.cpp index 9748eb27..feeaf2c2 100644 --- a/tests/NonstrictMode.test.cpp +++ b/tests/NonstrictMode.test.cpp @@ -283,7 +283,6 @@ TEST_CASE_FIXTURE(Fixture, "inconsistent_module_return_types_are_ok") ScopedFastFlag sff[]{ {"LuauReturnTypeInferenceInNonstrict", true}, {"LuauLowerBoundsCalculation", true}, - {"LuauSealExports", true}, }; CheckResult result = check(R"( diff --git a/tests/Parser.test.cpp b/tests/Parser.test.cpp index b941103d..55eafe3c 100644 --- a/tests/Parser.test.cpp +++ b/tests/Parser.test.cpp @@ -1606,8 +1606,6 @@ TEST_CASE_FIXTURE(Fixture, "end_extent_of_functions_unions_and_intersections") TEST_CASE_FIXTURE(Fixture, "end_extent_doesnt_consume_comments") { - ScopedFastFlag luauParseLocationIgnoreCommentSkip{"LuauParseLocationIgnoreCommentSkip", true}; - AstStatBlock* block = parse(R"( type F = number --comment @@ -1620,7 +1618,6 @@ TEST_CASE_FIXTURE(Fixture, "end_extent_doesnt_consume_comments") TEST_CASE_FIXTURE(Fixture, "end_extent_doesnt_consume_comments_even_with_capture") { - ScopedFastFlag luauParseLocationIgnoreCommentSkip{"LuauParseLocationIgnoreCommentSkip", true}; ScopedFastFlag luauParseLocationIgnoreCommentSkipInCapture{"LuauParseLocationIgnoreCommentSkipInCapture", true}; // Same should hold when comments are captured diff --git a/tests/RuntimeLimits.test.cpp b/tests/RuntimeLimits.test.cpp new file mode 100644 index 00000000..0b615e16 --- /dev/null +++ b/tests/RuntimeLimits.test.cpp @@ -0,0 +1,270 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details + +/* Tests in this source file are meant to be a bellwether to verify that the numeric limits we've set are sufficient for + * most real-world scripts. + * + * If a change breaks a test in this source file, please don't adjust the flag values set in the fixture. Instead, + * consider it a latent performance problem by default. + * + * We should periodically revisit this to retest the limits. + */ + +#include "Fixture.h" + +#include "doctest.h" + +using namespace Luau; + +LUAU_FASTFLAG(LuauLowerBoundsCalculation); + +struct LimitFixture : Fixture +{ +#if defined(_NOOPT) + ScopedFastInt LuauTypeInferRecursionLimit{"LuauTypeInferRecursionLimit", 150}; +#endif + + ScopedFastFlag LuauJustOneCallFrameForHaveSeen{"LuauJustOneCallFrameForHaveSeen", true}; +}; + +template +bool hasError(const CheckResult& result, T* = nullptr) +{ + auto it = std::find_if(result.errors.begin(), result.errors.end(), [](const TypeError& a) { + return nullptr != get(a); + }); + return it != result.errors.end(); +} + +TEST_SUITE_BEGIN("RuntimeLimitTests"); + +TEST_CASE_FIXTURE(LimitFixture, "bail_early_on_typescript_port_of_Result_type" * doctest::timeout(1.0)) +{ + constexpr const char* src = R"LUA( + --!strict + local TS = _G[script] + local lazyGet = TS.import(script, script.Parent.Parent, "util", "lazyLoad").lazyGet + local unit = TS.import(script, script.Parent.Parent, "util", "Unit").unit + local Iterator + lazyGet("Iterator", function(c) + Iterator = c + end) + local Option + lazyGet("Option", function(c) + Option = c + end) + local Vec + lazyGet("Vec", function(c) + Vec = c + end) + local Result + do + Result = setmetatable({}, { + __tostring = function() + return "Result" + end, + }) + Result.__index = Result + function Result.new(...) + local self = setmetatable({}, Result) + self:constructor(...) + return self + end + function Result:constructor(okValue, errValue) + self.okValue = okValue + self.errValue = errValue + end + function Result:ok(val) + return Result.new(val, nil) + end + function Result:err(val) + return Result.new(nil, val) + end + function Result:fromCallback(c) + local _0 = c + local _1, _2 = pcall(_0) + local result = _1 and { + success = true, + value = _2, + } or { + success = false, + error = _2, + } + return result.success and Result:ok(result.value) or Result:err(Option:wrap(result.error)) + end + function Result:fromVoidCallback(c) + local _0 = c + local _1, _2 = pcall(_0) + local result = _1 and { + success = true, + value = _2, + } or { + success = false, + error = _2, + } + return result.success and Result:ok(unit()) or Result:err(Option:wrap(result.error)) + end + Result.fromPromise = TS.async(function(self, p) + local _0, _1 = TS.try(function() + return TS.TRY_RETURN, { Result:ok(TS.await(p)) } + end, function(e) + return TS.TRY_RETURN, { Result:err(Option:wrap(e)) } + end) + if _0 then + return unpack(_1) + end + end) + Result.fromVoidPromise = TS.async(function(self, p) + local _0, _1 = TS.try(function() + TS.await(p) + return TS.TRY_RETURN, { Result:ok(unit()) } + end, function(e) + return TS.TRY_RETURN, { Result:err(Option:wrap(e)) } + end) + if _0 then + return unpack(_1) + end + end) + function Result:isOk() + return self.okValue ~= nil + end + function Result:isErr() + return self.errValue ~= nil + end + function Result:contains(x) + return self.okValue == x + end + function Result:containsErr(x) + return self.errValue == x + end + function Result:okOption() + return Option:wrap(self.okValue) + end + function Result:errOption() + return Option:wrap(self.errValue) + end + function Result:map(func) + return self:isOk() and Result:ok(func(self.okValue)) or Result:err(self.errValue) + end + function Result:mapOr(def, func) + local _0 + if self:isOk() then + _0 = func(self.okValue) + else + _0 = def + end + return _0 + end + function Result:mapOrElse(def, func) + local _0 + if self:isOk() then + _0 = func(self.okValue) + else + _0 = def(self.errValue) + end + return _0 + end + function Result:mapErr(func) + return self:isErr() and Result:err(func(self.errValue)) or Result:ok(self.okValue) + end + Result["and"] = function(self, other) + return self:isErr() and Result:err(self.errValue) or other + end + function Result:andThen(func) + return self:isErr() and Result:err(self.errValue) or func(self.okValue) + end + Result["or"] = function(self, other) + return self:isOk() and Result:ok(self.okValue) or other + end + function Result:orElse(other) + return self:isOk() and Result:ok(self.okValue) or other(self.errValue) + end + function Result:expect(msg) + if self:isOk() then + return self.okValue + else + error(msg) + end + end + function Result:unwrap() + return self:expect("called `Result.unwrap()` on an `Err` value: " .. tostring(self.errValue)) + end + function Result:unwrapOr(def) + local _0 + if self:isOk() then + _0 = self.okValue + else + _0 = def + end + return _0 + end + function Result:unwrapOrElse(gen) + local _0 + if self:isOk() then + _0 = self.okValue + else + _0 = gen(self.errValue) + end + return _0 + end + function Result:expectErr(msg) + if self:isErr() then + return self.errValue + else + error(msg) + end + end + function Result:unwrapErr() + return self:expectErr("called `Result.unwrapErr()` on an `Ok` value: " .. tostring(self.okValue)) + end + function Result:transpose() + return self:isOk() and self.okValue:map(function(some) + return Result:ok(some) + end) or Option:some(Result:err(self.errValue)) + end + function Result:flatten() + return self:isOk() and Result.new(self.okValue.okValue, self.okValue.errValue) or Result:err(self.errValue) + end + function Result:match(ifOk, ifErr) + local _0 + if self:isOk() then + _0 = ifOk(self.okValue) + else + _0 = ifErr(self.errValue) + end + return _0 + end + function Result:asPtr() + local _0 = (self.okValue) + if _0 == nil then + _0 = (self.errValue) + end + return _0 + end + end + local resultMeta = Result + resultMeta.__eq = function(a, b) + return b:match(function(ok) + return a:contains(ok) + end, function(err) + return a:containsErr(err) + end) + end + resultMeta.__tostring = function(result) + return result:match(function(ok) + return "Result.ok(" .. tostring(ok) .. ")" + end, function(err) + return "Result.err(" .. tostring(err) .. ")" + end) + end + return { + Result = Result, + } + )LUA"; + + if (FFlag::LuauLowerBoundsCalculation) + (void)check(src); + else + CHECK_THROWS_AS(check(src), std::exception); +} + +TEST_SUITE_END(); diff --git a/tests/TypeInfer.aliases.test.cpp b/tests/TypeInfer.aliases.test.cpp index b2e76052..b0eb31ce 100644 --- a/tests/TypeInfer.aliases.test.cpp +++ b/tests/TypeInfer.aliases.test.cpp @@ -7,8 +7,6 @@ using namespace Luau; -LUAU_FASTFLAG(LuauFixIncorrectLineNumberDuplicateType) - TEST_SUITE_BEGIN("TypeAliases"); TEST_CASE_FIXTURE(Fixture, "cyclic_function_type_in_type_alias") @@ -257,11 +255,7 @@ TEST_CASE_FIXTURE(Fixture, "reported_location_is_correct_when_type_alias_are_dup auto dtd = get(result.errors[0]); REQUIRE(dtd); CHECK_EQ(dtd->name, "B"); - - if (FFlag::LuauFixIncorrectLineNumberDuplicateType) - CHECK_EQ(dtd->previousLocation.begin.line + 1, 3); - else - CHECK_EQ(dtd->previousLocation.begin.line + 1, 1); + CHECK_EQ(dtd->previousLocation.begin.line + 1, 3); } TEST_CASE_FIXTURE(Fixture, "stringify_optional_parameterized_alias") @@ -495,8 +489,6 @@ TEST_CASE_FIXTURE(Fixture, "mutually_recursive_types_restriction_ok") TEST_CASE_FIXTURE(Fixture, "mutually_recursive_types_restriction_not_ok_1") { - ScopedFastFlag sff{"LuauRecursiveTypeParameterRestriction", true}; - CheckResult result = check(R"( -- OK because forwarded types are used with their parameters. type Tree = { data: T, children: Forest } @@ -508,8 +500,6 @@ TEST_CASE_FIXTURE(Fixture, "mutually_recursive_types_restriction_not_ok_1") TEST_CASE_FIXTURE(Fixture, "mutually_recursive_types_restriction_not_ok_2") { - ScopedFastFlag sff{"LuauRecursiveTypeParameterRestriction", true}; - CheckResult result = check(R"( -- Not OK because forwarded types are used with different types than their parameters. type Forest = {Tree<{T}>} @@ -531,8 +521,6 @@ TEST_CASE_FIXTURE(Fixture, "mutually_recursive_types_swapsies_ok") TEST_CASE_FIXTURE(Fixture, "mutually_recursive_types_swapsies_not_ok") { - ScopedFastFlag sff{"LuauRecursiveTypeParameterRestriction", true}; - CheckResult result = check(R"( type Tree1 = { data: T, children: {Tree2} } type Tree2 = { data: U, children: {Tree1} } @@ -647,9 +635,6 @@ TEST_CASE_FIXTURE(Fixture, "forward_declared_alias_is_not_clobbered_by_prior_uni { ScopedFastFlag sff[] = { {"LuauTwoPassAliasDefinitionFix", true}, - - // We also force this flag because it surfaced an unfortunate interaction. - {"LuauErrorRecoveryType", true}, }; CheckResult result = check(R"( @@ -687,8 +672,6 @@ TEST_CASE_FIXTURE(Fixture, "recursive_types_restriction_ok") TEST_CASE_FIXTURE(Fixture, "recursive_types_restriction_not_ok") { - ScopedFastFlag sff{"LuauRecursiveTypeParameterRestriction", true}; - CheckResult result = check(R"( -- this would be an infinite type if we allowed it type Tree = { data: T, children: {Tree<{T}>} } diff --git a/tests/TypeInfer.annotations.test.cpp b/tests/TypeInfer.annotations.test.cpp index e2971ad5..7f1c757a 100644 --- a/tests/TypeInfer.annotations.test.cpp +++ b/tests/TypeInfer.annotations.test.cpp @@ -221,8 +221,6 @@ TEST_CASE_FIXTURE(Fixture, "as_expr_is_bidirectional") TEST_CASE_FIXTURE(Fixture, "as_expr_warns_on_unrelated_cast") { - ScopedFastFlag sff2{"LuauErrorRecoveryType", true}; - CheckResult result = check(R"( local a = 55 :: string )"); @@ -407,8 +405,6 @@ TEST_CASE_FIXTURE(Fixture, "typeof_expr") TEST_CASE_FIXTURE(Fixture, "corecursive_types_error_on_tight_loop") { - ScopedFastFlag sff{"LuauErrorRecoveryType", true}; - CheckResult result = check(R"( type A = B type B = A diff --git a/tests/TypeInfer.functions.test.cpp b/tests/TypeInfer.functions.test.cpp index 7cd7bec3..0e071217 100644 --- a/tests/TypeInfer.functions.test.cpp +++ b/tests/TypeInfer.functions.test.cpp @@ -951,8 +951,6 @@ TEST_CASE_FIXTURE(Fixture, "record_matching_overload") TEST_CASE_FIXTURE(Fixture, "return_type_by_overload") { - ScopedFastFlag sff{"LuauErrorRecoveryType", true}; - CheckResult result = check(R"( type Overload = ((string) -> string) & ((number, number) -> number) local abc: Overload @@ -1538,7 +1536,6 @@ caused by: TEST_CASE_FIXTURE(Fixture, "too_few_arguments_variadic") { - ScopedFastFlag sff{"LuauArgCountMismatchSaysAtLeastWhenVariadic", true}; CheckResult result = check(R"( function test(a: number, b: string, ...) end @@ -1560,8 +1557,6 @@ TEST_CASE_FIXTURE(Fixture, "too_few_arguments_variadic") TEST_CASE_FIXTURE(Fixture, "too_few_arguments_variadic_generic") { - ScopedFastFlag sff1{"LuauArgCountMismatchSaysAtLeastWhenVariadic", true}; - ScopedFastFlag sff2{"LuauFixArgumentCountMismatchAmountWithGenericTypes", true}; CheckResult result = check(R"( function test(a: number, b: string, ...) return 1 @@ -1587,8 +1582,6 @@ wrapper(test) TEST_CASE_FIXTURE(Fixture, "too_few_arguments_variadic_generic2") { - ScopedFastFlag sff1{"LuauArgCountMismatchSaysAtLeastWhenVariadic", true}; - ScopedFastFlag sff2{"LuauFixArgumentCountMismatchAmountWithGenericTypes", true}; CheckResult result = check(R"( function test(a: number, b: string, ...) return 1 diff --git a/tests/TypeInfer.generics.test.cpp b/tests/TypeInfer.generics.test.cpp index 49d31fc6..91be2c1c 100644 --- a/tests/TypeInfer.generics.test.cpp +++ b/tests/TypeInfer.generics.test.cpp @@ -11,8 +11,6 @@ using namespace Luau; -LUAU_FASTFLAG(LuauFixArgumentCountMismatchAmountWithGenericTypes) - TEST_SUITE_BEGIN("GenericsTests"); TEST_CASE_FIXTURE(Fixture, "check_generic_function") @@ -679,8 +677,6 @@ local d: D = c TEST_CASE_FIXTURE(Fixture, "generic_functions_dont_cache_type_parameters") { - ScopedFastFlag sff{"LuauGenericFunctionsDontCacheTypeParams", true}; - CheckResult result = check(R"( -- See https://github.com/Roblox/luau/issues/332 -- This function has a type parameter with the same name as clones, @@ -707,8 +703,6 @@ TEST_CASE_FIXTURE(Fixture, "generic_functions_should_be_memory_safe") ScopedFastFlag sffs[] = { {"LuauTableSubtypingVariance2", true}, {"LuauUnsealedTableLiteral", true}, - {"LuauPropertiesGetExpectedType", true}, - {"LuauRecursiveTypeParameterRestriction", true}, }; CheckResult result = check(R"( @@ -733,8 +727,6 @@ caused by: TEST_CASE_FIXTURE(Fixture, "generic_type_pack_unification1") { - ScopedFastFlag sff{"LuauTxnLogSeesTypePacks2", true}; - CheckResult result = check(R"( --!strict type Dispatcher = { @@ -753,8 +745,6 @@ local TheDispatcher: Dispatcher = { TEST_CASE_FIXTURE(Fixture, "generic_type_pack_unification2") { - ScopedFastFlag sff{"LuauTxnLogSeesTypePacks2", true}; - CheckResult result = check(R"( --!strict type Dispatcher = { @@ -773,8 +763,6 @@ local TheDispatcher: Dispatcher = { TEST_CASE_FIXTURE(Fixture, "generic_type_pack_unification3") { - ScopedFastFlag sff{"LuauTxnLogSeesTypePacks2", true}; - CheckResult result = check(R"( --!strict type Dispatcher = { @@ -805,11 +793,7 @@ wrapper(test) )"); LUAU_REQUIRE_ERROR_COUNT(1, result); - - if (FFlag::LuauFixArgumentCountMismatchAmountWithGenericTypes) - CHECK_EQ(toString(result.errors[0]), R"(Argument count mismatch. Function expects 2 arguments, but only 1 is specified)"); - else - CHECK_EQ(toString(result.errors[0]), R"(Argument count mismatch. Function expects 1 argument, but 1 is specified)"); + CHECK_EQ(toString(result.errors[0]), R"(Argument count mismatch. Function expects 2 arguments, but only 1 is specified)"); } TEST_CASE_FIXTURE(Fixture, "generic_argument_count_too_many") @@ -826,11 +810,7 @@ wrapper(test2, 1, "", 3) )"); LUAU_REQUIRE_ERROR_COUNT(1, result); - - if (FFlag::LuauFixArgumentCountMismatchAmountWithGenericTypes) - CHECK_EQ(toString(result.errors[0]), R"(Argument count mismatch. Function expects 3 arguments, but 4 are specified)"); - else - CHECK_EQ(toString(result.errors[0]), R"(Argument count mismatch. Function expects 1 argument, but 4 are specified)"); + CHECK_EQ(toString(result.errors[0]), R"(Argument count mismatch. Function expects 3 arguments, but 4 are specified)"); } TEST_CASE_FIXTURE(Fixture, "generic_function") diff --git a/tests/TypeInfer.loops.test.cpp b/tests/TypeInfer.loops.test.cpp index 30df717b..960c6edf 100644 --- a/tests/TypeInfer.loops.test.cpp +++ b/tests/TypeInfer.loops.test.cpp @@ -78,6 +78,8 @@ TEST_CASE_FIXTURE(Fixture, "for_in_with_an_iterator_of_type_any") TEST_CASE_FIXTURE(Fixture, "for_in_loop_should_fail_with_non_function_iterator") { + ScopedFastFlag luauDoNotRelyOnNextBinding{"LuauDoNotRelyOnNextBinding", true}; + CheckResult result = check(R"( local foo = "bar" for i, v in foo do @@ -85,6 +87,7 @@ TEST_CASE_FIXTURE(Fixture, "for_in_loop_should_fail_with_non_function_iterator") )"); LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ("Cannot call non-function string", toString(result.errors[0])); } TEST_CASE_FIXTURE(Fixture, "for_in_with_just_one_iterator_is_ok") @@ -470,4 +473,19 @@ TEST_CASE_FIXTURE(Fixture, "loop_typecheck_crash_on_empty_optional") LUAU_REQUIRE_ERROR_COUNT(2, result); } +TEST_CASE_FIXTURE(Fixture, "fuzz_fail_missing_instantitation_follow") +{ + ScopedFastFlag luauInstantiateFollows{"LuauInstantiateFollows", true}; + + // Just check that this doesn't assert + check(R"( + --!nonstrict + function _(l0:number) + return _ + end + for _ in _(8) do + end + )"); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.operators.test.cpp b/tests/TypeInfer.operators.test.cpp index 5f2e2404..a2787cad 100644 --- a/tests/TypeInfer.operators.test.cpp +++ b/tests/TypeInfer.operators.test.cpp @@ -142,8 +142,6 @@ TEST_CASE_FIXTURE(Fixture, "some_primitive_binary_ops") TEST_CASE_FIXTURE(Fixture, "typecheck_overloaded_multiply_that_is_an_intersection") { - ScopedFastFlag sff{"LuauErrorRecoveryType", true}; - CheckResult result = check(R"( --!strict local Vec3 = {} @@ -178,8 +176,6 @@ TEST_CASE_FIXTURE(Fixture, "typecheck_overloaded_multiply_that_is_an_intersectio TEST_CASE_FIXTURE(Fixture, "typecheck_overloaded_multiply_that_is_an_intersection_on_rhs") { - ScopedFastFlag sff{"LuauErrorRecoveryType", true}; - CheckResult result = check(R"( --!strict local Vec3 = {} diff --git a/tests/TypeInfer.primitives.test.cpp b/tests/TypeInfer.primitives.test.cpp index 3ddf9813..e1684df7 100644 --- a/tests/TypeInfer.primitives.test.cpp +++ b/tests/TypeInfer.primitives.test.cpp @@ -85,8 +85,6 @@ TEST_CASE_FIXTURE(Fixture, "string_function_other") TEST_CASE_FIXTURE(Fixture, "CheckMethodsOfNumber") { - ScopedFastFlag sff{"LuauErrorRecoveryType", true}; - CheckResult result = check(R"( local x: number = 9999 function x:y(z: number) diff --git a/tests/TypeInfer.provisional.test.cpp b/tests/TypeInfer.provisional.test.cpp index 4b5075d9..2ef77419 100644 --- a/tests/TypeInfer.provisional.test.cpp +++ b/tests/TypeInfer.provisional.test.cpp @@ -268,242 +268,6 @@ TEST_CASE_FIXTURE(Fixture, "bail_early_if_unification_is_too_complicated" * doct } } -TEST_CASE_FIXTURE(Fixture, "bail_early_on_typescript_port_of_Result_type" * doctest::timeout(1.0)) -{ - ScopedFastInt sffi{"LuauTarjanChildLimit", 400}; - - CheckResult result = check(R"LUA( - --!strict - local TS = _G[script] - local lazyGet = TS.import(script, script.Parent.Parent, "util", "lazyLoad").lazyGet - local unit = TS.import(script, script.Parent.Parent, "util", "Unit").unit - local Iterator - lazyGet("Iterator", function(c) - Iterator = c - end) - local Option - lazyGet("Option", function(c) - Option = c - end) - local Vec - lazyGet("Vec", function(c) - Vec = c - end) - local Result - do - Result = setmetatable({}, { - __tostring = function() - return "Result" - end, - }) - Result.__index = Result - function Result.new(...) - local self = setmetatable({}, Result) - self:constructor(...) - return self - end - function Result:constructor(okValue, errValue) - self.okValue = okValue - self.errValue = errValue - end - function Result:ok(val) - return Result.new(val, nil) - end - function Result:err(val) - return Result.new(nil, val) - end - function Result:fromCallback(c) - local _0 = c - local _1, _2 = pcall(_0) - local result = _1 and { - success = true, - value = _2, - } or { - success = false, - error = _2, - } - return result.success and Result:ok(result.value) or Result:err(Option:wrap(result.error)) - end - function Result:fromVoidCallback(c) - local _0 = c - local _1, _2 = pcall(_0) - local result = _1 and { - success = true, - value = _2, - } or { - success = false, - error = _2, - } - return result.success and Result:ok(unit()) or Result:err(Option:wrap(result.error)) - end - Result.fromPromise = TS.async(function(self, p) - local _0, _1 = TS.try(function() - return TS.TRY_RETURN, { Result:ok(TS.await(p)) } - end, function(e) - return TS.TRY_RETURN, { Result:err(Option:wrap(e)) } - end) - if _0 then - return unpack(_1) - end - end) - Result.fromVoidPromise = TS.async(function(self, p) - local _0, _1 = TS.try(function() - TS.await(p) - return TS.TRY_RETURN, { Result:ok(unit()) } - end, function(e) - return TS.TRY_RETURN, { Result:err(Option:wrap(e)) } - end) - if _0 then - return unpack(_1) - end - end) - function Result:isOk() - return self.okValue ~= nil - end - function Result:isErr() - return self.errValue ~= nil - end - function Result:contains(x) - return self.okValue == x - end - function Result:containsErr(x) - return self.errValue == x - end - function Result:okOption() - return Option:wrap(self.okValue) - end - function Result:errOption() - return Option:wrap(self.errValue) - end - function Result:map(func) - return self:isOk() and Result:ok(func(self.okValue)) or Result:err(self.errValue) - end - function Result:mapOr(def, func) - local _0 - if self:isOk() then - _0 = func(self.okValue) - else - _0 = def - end - return _0 - end - function Result:mapOrElse(def, func) - local _0 - if self:isOk() then - _0 = func(self.okValue) - else - _0 = def(self.errValue) - end - return _0 - end - function Result:mapErr(func) - return self:isErr() and Result:err(func(self.errValue)) or Result:ok(self.okValue) - end - Result["and"] = function(self, other) - return self:isErr() and Result:err(self.errValue) or other - end - function Result:andThen(func) - return self:isErr() and Result:err(self.errValue) or func(self.okValue) - end - Result["or"] = function(self, other) - return self:isOk() and Result:ok(self.okValue) or other - end - function Result:orElse(other) - return self:isOk() and Result:ok(self.okValue) or other(self.errValue) - end - function Result:expect(msg) - if self:isOk() then - return self.okValue - else - error(msg) - end - end - function Result:unwrap() - return self:expect("called `Result.unwrap()` on an `Err` value: " .. tostring(self.errValue)) - end - function Result:unwrapOr(def) - local _0 - if self:isOk() then - _0 = self.okValue - else - _0 = def - end - return _0 - end - function Result:unwrapOrElse(gen) - local _0 - if self:isOk() then - _0 = self.okValue - else - _0 = gen(self.errValue) - end - return _0 - end - function Result:expectErr(msg) - if self:isErr() then - return self.errValue - else - error(msg) - end - end - function Result:unwrapErr() - return self:expectErr("called `Result.unwrapErr()` on an `Ok` value: " .. tostring(self.okValue)) - end - function Result:transpose() - return self:isOk() and self.okValue:map(function(some) - return Result:ok(some) - end) or Option:some(Result:err(self.errValue)) - end - function Result:flatten() - return self:isOk() and Result.new(self.okValue.okValue, self.okValue.errValue) or Result:err(self.errValue) - end - function Result:match(ifOk, ifErr) - local _0 - if self:isOk() then - _0 = ifOk(self.okValue) - else - _0 = ifErr(self.errValue) - end - return _0 - end - function Result:asPtr() - local _0 = (self.okValue) - if _0 == nil then - _0 = (self.errValue) - end - return _0 - end - end - local resultMeta = Result - resultMeta.__eq = function(a, b) - return b:match(function(ok) - return a:contains(ok) - end, function(err) - return a:containsErr(err) - end) - end - resultMeta.__tostring = function(result) - return result:match(function(ok) - return "Result.ok(" .. tostring(ok) .. ")" - end, function(err) - return "Result.err(" .. tostring(err) .. ")" - end) - end - return { - Result = Result, - } - )LUA"); - - auto it = std::find_if(result.errors.begin(), result.errors.end(), [](TypeError& a) { - return nullptr != get(a); - }); - if (it == result.errors.end()) - { - dumpErrors(result); - FAIL("Expected a UnificationTooComplex error"); - } -} - // Should be in TypeInfer.tables.test.cpp // It's unsound to instantiate tables containing generic methods, // since mutating properties means table properties should be invariant. diff --git a/tests/TypeInfer.singletons.test.cpp b/tests/TypeInfer.singletons.test.cpp index 2b01c29e..8d6682b8 100644 --- a/tests/TypeInfer.singletons.test.cpp +++ b/tests/TypeInfer.singletons.test.cpp @@ -164,10 +164,6 @@ TEST_CASE_FIXTURE(Fixture, "enums_using_singletons_subtyping") TEST_CASE_FIXTURE(Fixture, "tagged_unions_using_singletons") { - ScopedFastFlag sffs[] = { - {"LuauExpectedTypesOfProperties", true}, - }; - CheckResult result = check(R"( type Dog = { tag: "Dog", howls: boolean } type Cat = { tag: "Cat", meows: boolean } @@ -281,10 +277,6 @@ TEST_CASE_FIXTURE(Fixture, "table_properties_type_error_escapes") TEST_CASE_FIXTURE(Fixture, "error_detailed_tagged_union_mismatch_string") { - ScopedFastFlag sffs[] = { - {"LuauExpectedTypesOfProperties", true}, - }; - CheckResult result = check(R"( type Cat = { tag: 'cat', catfood: string } type Dog = { tag: 'dog', dogfood: string } @@ -302,10 +294,6 @@ caused by: TEST_CASE_FIXTURE(Fixture, "error_detailed_tagged_union_mismatch_bool") { - ScopedFastFlag sffs[] = { - {"LuauExpectedTypesOfProperties", true}, - }; - CheckResult result = check(R"( type Good = { success: true, result: string } type Bad = { success: false, error: string } @@ -323,10 +311,6 @@ caused by: TEST_CASE_FIXTURE(Fixture, "if_then_else_expression_singleton_options") { - ScopedFastFlag sffs[] = { - {"LuauExpectedTypesOfProperties", true}, - }; - CheckResult result = check(R"( type Cat = { tag: 'cat', catfood: string } type Dog = { tag: 'dog', dogfood: string } diff --git a/tests/TypeInfer.tables.test.cpp b/tests/TypeInfer.tables.test.cpp index 2a727bb3..5bd522a3 100644 --- a/tests/TypeInfer.tables.test.cpp +++ b/tests/TypeInfer.tables.test.cpp @@ -2122,8 +2122,6 @@ caused by: TEST_CASE_FIXTURE(Fixture, "explicitly_typed_table") { ScopedFastFlag sffs[]{ - {"LuauPropertiesGetExpectedType", true}, - {"LuauExpectedTypesOfProperties", true}, {"LuauTableSubtypingVariance2", true}, }; @@ -2143,8 +2141,6 @@ a.p = { x = 9 } TEST_CASE_FIXTURE(Fixture, "explicitly_typed_table_error") { ScopedFastFlag sffs[]{ - {"LuauPropertiesGetExpectedType", true}, - {"LuauExpectedTypesOfProperties", true}, {"LuauTableSubtypingVariance2", true}, {"LuauUnsealedTableLiteral", true}, }; @@ -2171,8 +2167,6 @@ caused by: TEST_CASE_FIXTURE(Fixture, "explicitly_typed_table_with_indexer") { ScopedFastFlag sffs[]{ - {"LuauPropertiesGetExpectedType", true}, - {"LuauExpectedTypesOfProperties", true}, {"LuauTableSubtypingVariance2", true}, }; @@ -2377,8 +2371,6 @@ TEST_CASE_FIXTURE(Fixture, "pass_a_union_of_tables_to_a_function_that_requires_a TEST_CASE_FIXTURE(Fixture, "unifying_tables_shouldnt_uaf1") { - ScopedFastFlag sff{"LuauTxnLogCheckForInvalidation", true}; - CheckResult result = check(R"( -- This example produced a UAF at one point, caused by pointers to table types becoming -- invalidated by child unifiers. (Calling log.concat can cause pointers to become invalid.) @@ -2409,8 +2401,6 @@ end TEST_CASE_FIXTURE(Fixture, "unifying_tables_shouldnt_uaf2") { - ScopedFastFlag sff{"LuauTxnLogCheckForInvalidation", true}; - CheckResult result = check(R"( -- Another example that UAFd, this time found by fuzzing. local _ diff --git a/tests/TypeInfer.tryUnify.test.cpp b/tests/TypeInfer.tryUnify.test.cpp index c21e1625..b6e93265 100644 --- a/tests/TypeInfer.tryUnify.test.cpp +++ b/tests/TypeInfer.tryUnify.test.cpp @@ -126,8 +126,6 @@ TEST_CASE_FIXTURE(TryUnifyFixture, "members_of_failed_typepack_unification_are_u TEST_CASE_FIXTURE(TryUnifyFixture, "result_of_failed_typepack_unification_is_constrained") { - ScopedFastFlag sff{"LuauErrorRecoveryType", true}; - CheckResult result = check(R"( function f(arg: number) return arg end local a diff --git a/tests/TypeVar.test.cpp b/tests/TypeVar.test.cpp index d03bb03c..e033fe22 100644 --- a/tests/TypeVar.test.cpp +++ b/tests/TypeVar.test.cpp @@ -184,8 +184,6 @@ TEST_CASE_FIXTURE(Fixture, "UnionTypeVarIterator_with_empty_union") TEST_CASE_FIXTURE(Fixture, "substitution_skip_failure") { - ScopedFastFlag sff{"LuauSealExports", true}; - TypeVar ftv11{FreeTypeVar{TypeLevel{}}}; TypePackVar tp24{TypePack{{&ftv11}}}; From 0d6481b9df2472c1bd6267d3f6ea7cc2a0a38cab Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Thu, 28 Apr 2022 18:10:31 -0700 Subject: [PATCH 230/399] Fix tests in debug --- tests/RuntimeLimits.test.cpp | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/RuntimeLimits.test.cpp b/tests/RuntimeLimits.test.cpp index 0b615e16..d7e22465 100644 --- a/tests/RuntimeLimits.test.cpp +++ b/tests/RuntimeLimits.test.cpp @@ -19,7 +19,7 @@ LUAU_FASTFLAG(LuauLowerBoundsCalculation); struct LimitFixture : Fixture { -#if defined(_NOOPT) +#if defined(_NOOPT) || defined(_DEBUG) ScopedFastInt LuauTypeInferRecursionLimit{"LuauTypeInferRecursionLimit", 150}; #endif From 51ae97c211f6818aa59c1b9ab06e97b22817c388 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Thu, 28 Apr 2022 18:15:04 -0700 Subject: [PATCH 231/399] We also need to lower the limit --- tests/RuntimeLimits.test.cpp | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/RuntimeLimits.test.cpp b/tests/RuntimeLimits.test.cpp index d7e22465..dcbf0b61 100644 --- a/tests/RuntimeLimits.test.cpp +++ b/tests/RuntimeLimits.test.cpp @@ -20,7 +20,7 @@ LUAU_FASTFLAG(LuauLowerBoundsCalculation); struct LimitFixture : Fixture { #if defined(_NOOPT) || defined(_DEBUG) - ScopedFastInt LuauTypeInferRecursionLimit{"LuauTypeInferRecursionLimit", 150}; + ScopedFastInt LuauTypeInferRecursionLimit{"LuauTypeInferRecursionLimit", 100}; #endif ScopedFastFlag LuauJustOneCallFrameForHaveSeen{"LuauJustOneCallFrameForHaveSeen", true}; From bd6d44f5e33511e1a9241b7a57c0e3a00e3aa6eb Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Thu, 28 Apr 2022 18:24:24 -0700 Subject: [PATCH 232/399] Sync to upstream/release/525 (#467) --- Analysis/src/Frontend.cpp | 2 +- Analysis/src/Linter.cpp | 6 +- Analysis/src/Substitution.cpp | 2 +- Analysis/src/TypeInfer.cpp | 211 ++++++------------ Analysis/src/TypeVar.cpp | 11 +- Analysis/src/Unifier.cpp | 89 +++----- Ast/src/Lexer.cpp | 4 +- CLI/FileUtils.cpp | 4 +- CLI/Repl.cpp | 9 +- Compiler/include/Luau/BytecodeBuilder.h | 2 +- Compiler/src/BytecodeBuilder.cpp | 24 ++- Compiler/src/Compiler.cpp | 8 +- Compiler/src/CostModel.cpp | 2 +- Sources.cmake | 3 +- VM/include/lua.h | 2 +- VM/src/lapi.cpp | 2 +- VM/src/lstate.h | 2 +- VM/src/ltable.cpp | 55 ++--- VM/src/ludata.cpp | 13 +- fuzz/proto.cpp | 27 ++- tests/Autocomplete.test.cpp | 1 - tests/Compiler.test.cpp | 50 +++-- tests/Conformance.test.cpp | 2 +- tests/Frontend.test.cpp | 18 +- tests/Module.test.cpp | 1 - tests/NonstrictMode.test.cpp | 1 - tests/Parser.test.cpp | 3 - tests/RuntimeLimits.test.cpp | 270 ++++++++++++++++++++++++ tests/TypeInfer.aliases.test.cpp | 19 +- tests/TypeInfer.annotations.test.cpp | 4 - tests/TypeInfer.functions.test.cpp | 7 - tests/TypeInfer.generics.test.cpp | 24 +-- tests/TypeInfer.loops.test.cpp | 18 ++ tests/TypeInfer.operators.test.cpp | 4 - tests/TypeInfer.primitives.test.cpp | 2 - tests/TypeInfer.provisional.test.cpp | 236 --------------------- tests/TypeInfer.singletons.test.cpp | 16 -- tests/TypeInfer.tables.test.cpp | 10 - tests/TypeInfer.tryUnify.test.cpp | 2 - tests/TypeVar.test.cpp | 2 - 40 files changed, 527 insertions(+), 641 deletions(-) create mode 100644 tests/RuntimeLimits.test.cpp diff --git a/Analysis/src/Frontend.cpp b/Analysis/src/Frontend.cpp index 34ccdac4..b8f7836d 100644 --- a/Analysis/src/Frontend.cpp +++ b/Analysis/src/Frontend.cpp @@ -24,7 +24,7 @@ LUAU_FASTFLAGVARIABLE(LuauKnowsTheDataModel3, false) LUAU_FASTFLAGVARIABLE(LuauSeparateTypechecks, false) LUAU_FASTFLAGVARIABLE(LuauAutocompleteDynamicLimits, false) LUAU_FASTFLAGVARIABLE(LuauDirtySourceModule, false) -LUAU_FASTINTVARIABLE(LuauAutocompleteCheckTimeoutMs, 0) +LUAU_FASTINTVARIABLE(LuauAutocompleteCheckTimeoutMs, 100) namespace Luau { diff --git a/Analysis/src/Linter.cpp b/Analysis/src/Linter.cpp index 5608e4b3..200b7d1b 100644 --- a/Analysis/src/Linter.cpp +++ b/Analysis/src/Linter.cpp @@ -2653,12 +2653,12 @@ static void lintComments(LintContext& context, const std::vector& ho } else { - std::string::size_type space = hc.content.find_first_of(" \t"); + size_t space = hc.content.find_first_of(" \t"); std::string_view first = std::string_view(hc.content).substr(0, space); if (first == "nolint") { - std::string::size_type notspace = hc.content.find_first_not_of(" \t", space); + size_t notspace = hc.content.find_first_not_of(" \t", space); if (space == std::string::npos || notspace == std::string::npos) { @@ -2827,7 +2827,7 @@ uint64_t LintWarning::parseMask(const std::vector& hotcomments) if (hc.content.compare(0, 6, "nolint") != 0) continue; - std::string::size_type name = hc.content.find_first_not_of(" \t", 6); + size_t name = hc.content.find_first_not_of(" \t", 6); // --!nolint disables everything if (name == std::string::npos) diff --git a/Analysis/src/Substitution.cpp b/Analysis/src/Substitution.cpp index 1b51fa3d..30d8574a 100644 --- a/Analysis/src/Substitution.cpp +++ b/Analysis/src/Substitution.cpp @@ -8,7 +8,7 @@ #include LUAU_FASTFLAG(LuauLowerBoundsCalculation) -LUAU_FASTINTVARIABLE(LuauTarjanChildLimit, 1000) +LUAU_FASTINTVARIABLE(LuauTarjanChildLimit, 10000) LUAU_FASTFLAG(LuauTypecheckOptPass) LUAU_FASTFLAGVARIABLE(LuauSubstituteFollowNewTypes, false) LUAU_FASTFLAGVARIABLE(LuauSubstituteFollowPossibleMutations, false) diff --git a/Analysis/src/TypeInfer.cpp b/Analysis/src/TypeInfer.cpp index 6411e2ab..ba91ae1e 100644 --- a/Analysis/src/TypeInfer.cpp +++ b/Analysis/src/TypeInfer.cpp @@ -22,29 +22,25 @@ #include LUAU_FASTFLAGVARIABLE(DebugLuauMagicTypes, false) -LUAU_FASTINTVARIABLE(LuauTypeInferRecursionLimit, 500) -LUAU_FASTINTVARIABLE(LuauTypeInferIterationLimit, 2000) +LUAU_FASTINTVARIABLE(LuauTypeInferRecursionLimit, 165) +LUAU_FASTINTVARIABLE(LuauTypeInferIterationLimit, 20000) LUAU_FASTINTVARIABLE(LuauTypeInferTypePackLoopLimit, 5000) -LUAU_FASTINTVARIABLE(LuauCheckRecursionLimit, 500) +LUAU_FASTINTVARIABLE(LuauCheckRecursionLimit, 300) LUAU_FASTFLAG(LuauKnowsTheDataModel3) LUAU_FASTFLAG(LuauSeparateTypechecks) LUAU_FASTFLAG(LuauAutocompleteDynamicLimits) LUAU_FASTFLAG(LuauAutocompleteSingletonTypes) LUAU_FASTFLAGVARIABLE(LuauCyclicModuleTypeSurface, false) +LUAU_FASTFLAGVARIABLE(LuauDoNotRelyOnNextBinding, false) LUAU_FASTFLAGVARIABLE(LuauEqConstraint, false) LUAU_FASTFLAGVARIABLE(LuauWeakEqConstraint, false) // Eventually removed as false. LUAU_FASTFLAGVARIABLE(LuauLowerBoundsCalculation, false) LUAU_FASTFLAGVARIABLE(DebugLuauFreezeDuringUnification, false) -LUAU_FASTFLAGVARIABLE(LuauRecursiveTypeParameterRestriction, false) -LUAU_FASTFLAGVARIABLE(LuauGenericFunctionsDontCacheTypeParams, false) LUAU_FASTFLAGVARIABLE(LuauInferStatFunction, false) -LUAU_FASTFLAGVARIABLE(LuauSealExports, false) +LUAU_FASTFLAGVARIABLE(LuauInstantiateFollows, false) LUAU_FASTFLAGVARIABLE(LuauSelfCallAutocompleteFix, false) LUAU_FASTFLAGVARIABLE(LuauDiscriminableUnions2, false) -LUAU_FASTFLAGVARIABLE(LuauExpectedTypesOfProperties, false) -LUAU_FASTFLAGVARIABLE(LuauErrorRecoveryType, false) LUAU_FASTFLAGVARIABLE(LuauOnlyMutateInstantiatedTables, false) -LUAU_FASTFLAGVARIABLE(LuauPropertiesGetExpectedType, false) LUAU_FASTFLAGVARIABLE(LuauStatFunctionSimplify4, false) LUAU_FASTFLAGVARIABLE(LuauTypecheckOptPass, false) LUAU_FASTFLAGVARIABLE(LuauUnsealedTableLiteral, false) @@ -54,12 +50,9 @@ LUAU_FASTFLAGVARIABLE(LuauReturnAnyInsteadOfICE, false) // Eventually removed as LUAU_FASTFLAG(LuauWidenIfSupertypeIsFree2) LUAU_FASTFLAGVARIABLE(LuauDoNotTryToReduce, false) LUAU_FASTFLAGVARIABLE(LuauDoNotAccidentallyDependOnPointerOrdering, false) -LUAU_FASTFLAGVARIABLE(LuauFixArgumentCountMismatchAmountWithGenericTypes, false) -LUAU_FASTFLAGVARIABLE(LuauFixIncorrectLineNumberDuplicateType, false) LUAU_FASTFLAGVARIABLE(LuauCheckImplicitNumbericKeys, false) LUAU_FASTFLAG(LuauAnyInIsOptionalIsOptional) LUAU_FASTFLAGVARIABLE(LuauDecoupleOperatorInferenceFromUnifiedTypeInference, false) -LUAU_FASTFLAGVARIABLE(LuauArgCountMismatchSaysAtLeastWhenVariadic, false) LUAU_FASTFLAGVARIABLE(LuauTableUseCounterInstead, false) LUAU_FASTFLAGVARIABLE(LuauReturnTypeInferenceInNonstrict, false) LUAU_FASTFLAGVARIABLE(LuauRecursionLimitException, false); @@ -1160,7 +1153,10 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatForIn& forin) } else { - iterTy = follow(instantiate(scope, checkExpr(scope, *firstValue).type, firstValue->location)); + if (FFlag::LuauInstantiateFollows) + iterTy = instantiate(scope, checkExpr(scope, *firstValue).type, firstValue->location); + else + iterTy = follow(instantiate(scope, checkExpr(scope, *firstValue).type, firstValue->location)); } const FunctionTypeVar* iterFunc = get(iterTy); @@ -1172,7 +1168,12 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatForIn& forin) unify(varTy, var, forin.location); if (!get(iterTy) && !get(iterTy) && !get(iterTy)) - reportError(TypeError{firstValue->location, TypeMismatch{globalScope->bindings[AstName{"next"}].typeId, iterTy}}); + { + if (FFlag::LuauDoNotRelyOnNextBinding) + reportError(firstValue->location, CannotCallNonFunction{iterTy}); + else + reportError(TypeError{firstValue->location, TypeMismatch{globalScope->bindings[AstName{"next"}].typeId, iterTy}}); + } return check(loopScope, *forin.body); } @@ -1427,8 +1428,7 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatTypeAlias& typealias ftv->forwardedTypeAlias = true; bindingsMap[name] = {std::move(generics), std::move(genericPacks), ty}; - if (FFlag::LuauFixIncorrectLineNumberDuplicateType) - scope->typeAliasLocations[name] = typealias.location; + scope->typeAliasLocations[name] = typealias.location; } } else @@ -2217,7 +2217,7 @@ TypeId TypeChecker::checkExprTable( if (isNonstrictMode() && !getTableType(exprType) && !get(exprType)) exprType = anyType; - if (FFlag::LuauPropertiesGetExpectedType && expectedTable) + if (expectedTable) { auto it = expectedTable->props.find(key->value.data); if (it != expectedTable->props.end()) @@ -2309,9 +2309,8 @@ ExprResult TypeChecker::checkExpr_(const ScopePtr& scope, const AstExprT } } } - else if (FFlag::LuauExpectedTypesOfProperties) - if (const UnionTypeVar* utv = get(follow(*expectedType))) - expectedUnion = utv; + else if (const UnionTypeVar* utv = get(follow(*expectedType))) + expectedUnion = utv; } for (size_t i = 0; i < expr.items.size; ++i) @@ -2334,7 +2333,7 @@ ExprResult TypeChecker::checkExpr_(const ScopePtr& scope, const AstExprT if (auto prop = expectedTable->props.find(key->value.data); prop != expectedTable->props.end()) expectedResultType = prop->second.type; } - else if (FFlag::LuauExpectedTypesOfProperties && expectedUnion) + else if (expectedUnion) { std::vector expectedResultTypes; for (TypeId expectedOption : expectedUnion) @@ -2713,8 +2712,6 @@ TypeId TypeChecker::checkBinaryOperation( { auto name = getIdentifierOfBaseVar(expr.left); reportError(expr.location, CannotInferBinaryOperation{expr.op, name, CannotInferBinaryOperation::Operation}); - if (!FFlag::LuauErrorRecoveryType) - return errorRecoveryType(scope); } } @@ -2754,7 +2751,7 @@ TypeId TypeChecker::checkBinaryOperation( reportErrors(state.errors); bool hasErrors = !state.errors.empty(); - if (FFlag::LuauErrorRecoveryType && hasErrors) + if (hasErrors) { // If there are unification errors, the return type may still be unknown // so we loosen the argument types to see if that helps. @@ -2768,8 +2765,7 @@ TypeId TypeChecker::checkBinaryOperation( if (state.errors.empty()) state.log.commit(); } - - if (!hasErrors) + else { state.log.commit(); } @@ -3196,16 +3192,7 @@ TypeId TypeChecker::checkFunctionName(const ScopePtr& scope, AstExpr& funName, T } else { - if (!ttv) - { - if (!FFlag::LuauErrorRecoveryType && !isTableIntersection(lhsType)) - // This error now gets reported when we check the function body. - reportError(TypeError{funName.location, OnlyTablesCanHaveMethods{lhsType}}); - - return errorRecoveryType(scope); - } - - if (lhsType->persistent || ttv->state == TableState::Sealed) + if (!ttv || lhsType->persistent || ttv->state == TableState::Sealed) return errorRecoveryType(scope); } @@ -3532,32 +3519,6 @@ ExprResult TypeChecker::checkExprPack(const ScopePtr& scope, const A } // Returns the minimum number of arguments the argument list can accept. -static size_t getMinParameterCount_DEPRECATED(TypePackId tp) -{ - size_t minCount = 0; - size_t optionalCount = 0; - - auto it = begin(tp); - auto endIter = end(tp); - - while (it != endIter) - { - TypeId ty = *it; - if (isOptional(ty)) - ++optionalCount; - else - { - minCount += optionalCount; - optionalCount = 0; - minCount++; - } - - ++it; - } - - return minCount; -} - static size_t getMinParameterCount(TxnLog* log, TypePackId tp) { size_t minCount = 0; @@ -3597,19 +3558,14 @@ void TypeChecker::checkArgumentList( size_t paramIndex = 0; - size_t minParams = FFlag::LuauFixIncorrectLineNumberDuplicateType ? 0 : getMinParameterCount_DEPRECATED(paramPack); - - auto reportCountMismatchError = [&state, &argLocations, minParams, paramPack, argPack]() { + auto reportCountMismatchError = [&state, &argLocations, paramPack, argPack]() { // For this case, we want the error span to cover every errant extra parameter Location location = state.location; if (!argLocations.empty()) location = {state.location.begin, argLocations.back().end}; - size_t mp = minParams; - if (FFlag::LuauFixArgumentCountMismatchAmountWithGenericTypes) - mp = getMinParameterCount(&state.log, paramPack); - - state.reportError(TypeError{location, CountMismatch{mp, std::distance(begin(argPack), end(argPack))}}); + size_t minParams = getMinParameterCount(&state.log, paramPack); + state.reportError(TypeError{location, CountMismatch{minParams, std::distance(begin(argPack), end(argPack))}}); }; while (true) @@ -3707,16 +3663,10 @@ void TypeChecker::checkArgumentList( } // ok else { - if (FFlag::LuauFixArgumentCountMismatchAmountWithGenericTypes) - minParams = getMinParameterCount(&state.log, paramPack); + size_t minParams = getMinParameterCount(&state.log, paramPack); - bool isVariadic = false; - if (FFlag::LuauArgCountMismatchSaysAtLeastWhenVariadic) - { - std::optional tail = flatten(paramPack, state.log).second; - if (tail) - isVariadic = Luau::isVariadic(*tail); - } + std::optional tail = flatten(paramPack, state.log).second; + bool isVariadic = tail && Luau::isVariadic(*tail); state.reportError(TypeError{state.location, CountMismatch{minParams, paramIndex, CountMismatch::Context::Arg, isVariadic}}); return; @@ -3863,7 +3813,8 @@ ExprResult TypeChecker::checkExprPack(const ScopePtr& scope, const A actualFunctionType = instantiate(scope, functionType, expr.func->location); } - actualFunctionType = follow(actualFunctionType); + if (!FFlag::LuauInstantiateFollows) + actualFunctionType = follow(actualFunctionType); TypePackId retPack; if (FFlag::LuauLowerBoundsCalculation || !FFlag::LuauWidenIfSupertypeIsFree2) @@ -3930,16 +3881,13 @@ ExprResult TypeChecker::checkExprPack(const ScopePtr& scope, const A reportOverloadResolutionError(scope, expr, retPack, argPack, argLocations, overloads, overloadsThatMatchArgCount, errors); - if (FFlag::LuauErrorRecoveryType) - { - const FunctionTypeVar* overload = nullptr; - if (!overloadsThatMatchArgCount.empty()) - overload = get(overloadsThatMatchArgCount[0]); - if (!overload && !overloadsThatDont.empty()) - overload = get(overloadsThatDont[0]); - if (overload) - return {errorRecoveryTypePack(overload->retType)}; - } + const FunctionTypeVar* overload = nullptr; + if (!overloadsThatMatchArgCount.empty()) + overload = get(overloadsThatMatchArgCount[0]); + if (!overload && !overloadsThatDont.empty()) + overload = get(overloadsThatDont[0]); + if (overload) + return {errorRecoveryTypePack(overload->retType)}; return {errorRecoveryTypePack(retPack)}; } @@ -4129,7 +4077,7 @@ std::optional> TypeChecker::checkCallOverload(const Scope if (!argMismatch) overloadsThatMatchArgCount.push_back(fn); - else if (FFlag::LuauErrorRecoveryType) + else overloadsThatDont.push_back(fn); errors.emplace_back(std::move(state.errors), args->head, ftv); @@ -4715,7 +4663,7 @@ bool Anyification::isDirty(TypeId ty) return false; if (const TableTypeVar* ttv = log->getMutable(ty)) - return (ttv->state == TableState::Free || (FFlag::LuauSealExports && ttv->state == TableState::Unsealed)); + return (ttv->state == TableState::Free || ttv->state == TableState::Unsealed); else if (log->getMutable(ty)) return true; else if (get(ty)) @@ -4743,12 +4691,9 @@ TypeId Anyification::clean(TypeId ty) TableTypeVar clone = TableTypeVar{ttv->props, ttv->indexer, ttv->level, TableState::Sealed}; clone.methodDefinitionLocations = ttv->methodDefinitionLocations; clone.definitionModuleName = ttv->definitionModuleName; - if (FFlag::LuauSealExports) - { - clone.name = ttv->name; - clone.syntheticName = ttv->syntheticName; - clone.tags = ttv->tags; - } + clone.name = ttv->name; + clone.syntheticName = ttv->syntheticName; + clone.tags = ttv->tags; TypeId res = addType(std::move(clone)); asMutable(res)->normal = ty->normal; return res; @@ -4791,9 +4736,12 @@ TypeId TypeChecker::quantify(const ScopePtr& scope, TypeId ty, Location location TypeId TypeChecker::instantiate(const ScopePtr& scope, TypeId ty, Location location, const TxnLog* log) { + if (FFlag::LuauInstantiateFollows) + ty = follow(ty); + if (FFlag::LuauTypecheckOptPass) { - const FunctionTypeVar* ftv = get(follow(ty)); + const FunctionTypeVar* ftv = get(FFlag::LuauInstantiateFollows ? ty : follow(ty)); if (ftv && ftv->hasNoGenerics) return ty; } @@ -5175,8 +5123,6 @@ TypeId TypeChecker::resolveType(const ScopePtr& scope, const AstType& annotation { reportError(TypeError{annotation.location, GenericError{"Type parameter list is required"}}); parameterCountErrorReported = true; - if (!FFlag::LuauErrorRecoveryType) - return errorRecoveryType(scope); } } @@ -5294,33 +5240,25 @@ TypeId TypeChecker::resolveType(const ScopePtr& scope, const AstType& annotation reportError( TypeError{annotation.location, IncorrectGenericParameterCount{lit->name.value, *tf, typeParams.size(), typePackParams.size()}}); - if (FFlag::LuauErrorRecoveryType) - { - // Pad the types out with error recovery types - while (typeParams.size() < tf->typeParams.size()) - typeParams.push_back(errorRecoveryType(scope)); - while (typePackParams.size() < tf->typePackParams.size()) - typePackParams.push_back(errorRecoveryTypePack(scope)); - } - else - return errorRecoveryType(scope); + // Pad the types out with error recovery types + while (typeParams.size() < tf->typeParams.size()) + typeParams.push_back(errorRecoveryType(scope)); + while (typePackParams.size() < tf->typePackParams.size()) + typePackParams.push_back(errorRecoveryTypePack(scope)); } - if (FFlag::LuauRecursiveTypeParameterRestriction) - { - bool sameTys = std::equal(typeParams.begin(), typeParams.end(), tf->typeParams.begin(), tf->typeParams.end(), [](auto&& itp, auto&& tp) { - return itp == tp.ty; + bool sameTys = std::equal(typeParams.begin(), typeParams.end(), tf->typeParams.begin(), tf->typeParams.end(), [](auto&& itp, auto&& tp) { + return itp == tp.ty; + }); + bool sameTps = std::equal( + typePackParams.begin(), typePackParams.end(), tf->typePackParams.begin(), tf->typePackParams.end(), [](auto&& itpp, auto&& tpp) { + return itpp == tpp.tp; }); - bool sameTps = std::equal( - typePackParams.begin(), typePackParams.end(), tf->typePackParams.begin(), tf->typePackParams.end(), [](auto&& itpp, auto&& tpp) { - return itpp == tpp.tp; - }); - // If the generic parameters and the type arguments are the same, we are about to - // perform an identity substitution, which we can just short-circuit. - if (sameTys && sameTps) - return tf->type; - } + // If the generic parameters and the type arguments are the same, we are about to + // perform an identity substitution, which we can just short-circuit. + if (sameTys && sameTps) + return tf->type; return instantiateTypeFun(scope, *tf, typeParams, typePackParams, annotation.location); } @@ -5483,7 +5421,7 @@ bool ApplyTypeFunction::isDirty(TypeId ty) return true; else if (const FreeTypeVar* ftv = get(ty)) { - if (FFlag::LuauRecursiveTypeParameterRestriction && ftv->forwardedTypeAlias) + if (ftv->forwardedTypeAlias) encounteredForwardedType = true; return false; } @@ -5562,7 +5500,7 @@ TypeId TypeChecker::instantiateTypeFun(const ScopePtr& scope, const TypeFun& tf, reportError(location, UnificationTooComplex{}); return errorRecoveryType(scope); } - if (FFlag::LuauRecursiveTypeParameterRestriction && applyTypeFunction.encounteredForwardedType) + if (applyTypeFunction.encounteredForwardedType) { reportError(TypeError{location, GenericError{"Recursive type being used with different parameters"}}); return errorRecoveryType(scope); @@ -5632,7 +5570,7 @@ GenericTypeDefinitions TypeChecker::createGenericTypes(const ScopePtr& scope, st } TypeId g; - if (FFlag::LuauRecursiveTypeParameterRestriction && (!FFlag::LuauGenericFunctionsDontCacheTypeParams || useCache)) + if (useCache) { TypeId& cached = scope->parent->typeAliasTypeParameters[n]; if (!cached) @@ -5667,21 +5605,12 @@ GenericTypeDefinitions TypeChecker::createGenericTypes(const ScopePtr& scope, st reportError(TypeError{node.location, DuplicateGenericParameter{n}}); } - TypePackId g; - if (FFlag::LuauRecursiveTypeParameterRestriction) - { - TypePackId& cached = scope->parent->typeAliasTypePackParameters[n]; - if (!cached) - cached = addTypePack(TypePackVar{Unifiable::Generic{level, n}}); - g = cached; - } - else - { - g = addTypePack(TypePackVar{Unifiable::Generic{level, n}}); - } + TypePackId& cached = scope->parent->typeAliasTypePackParameters[n]; + if (!cached) + cached = addTypePack(TypePackVar{Unifiable::Generic{level, n}}); - genericPacks.push_back({g, defaultValue}); - scope->privateTypePackBindings[n] = g; + genericPacks.push_back({cached, defaultValue}); + scope->privateTypePackBindings[n] = cached; } return {generics, genericPacks}; diff --git a/Analysis/src/TypeVar.cpp b/Analysis/src/TypeVar.cpp index 4d42573c..463b4651 100644 --- a/Analysis/src/TypeVar.cpp +++ b/Analysis/src/TypeVar.cpp @@ -23,7 +23,6 @@ LUAU_FASTFLAG(DebugLuauFreezeArena) LUAU_FASTINTVARIABLE(LuauTypeMaximumStringifierLength, 500) LUAU_FASTINTVARIABLE(LuauTableTypeMaximumStringifierLength, 0) LUAU_FASTINT(LuauTypeInferRecursionLimit) -LUAU_FASTFLAG(LuauErrorRecoveryType) LUAU_FASTFLAG(LuauSubtypingAddOptPropsToUnsealedTables) LUAU_FASTFLAG(LuauDiscriminableUnions2) LUAU_FASTFLAGVARIABLE(LuauAnyInIsOptionalIsOptional, false) @@ -775,18 +774,12 @@ TypePackId SingletonTypes::errorRecoveryTypePack() TypeId SingletonTypes::errorRecoveryType(TypeId guess) { - if (FFlag::LuauErrorRecoveryType) - return guess; - else - return &errorType_; + return guess; } TypePackId SingletonTypes::errorRecoveryTypePack(TypePackId guess) { - if (FFlag::LuauErrorRecoveryType) - return guess; - else - return &errorTypePack_; + return guess; } SingletonTypes& getSingletonTypes() diff --git a/Analysis/src/Unifier.cpp b/Analysis/src/Unifier.cpp index 9862d7b3..334806ce 100644 --- a/Analysis/src/Unifier.cpp +++ b/Analysis/src/Unifier.cpp @@ -23,10 +23,7 @@ LUAU_FASTFLAG(LuauErrorRecoveryType); LUAU_FASTFLAGVARIABLE(LuauSubtypingAddOptPropsToUnsealedTables, false) LUAU_FASTFLAGVARIABLE(LuauWidenIfSupertypeIsFree2, false) LUAU_FASTFLAGVARIABLE(LuauDifferentOrderOfUnificationDoesntMatter, false) -LUAU_FASTFLAGVARIABLE(LuauTxnLogSeesTypePacks2, false) -LUAU_FASTFLAGVARIABLE(LuauTxnLogCheckForInvalidation, false) LUAU_FASTFLAGVARIABLE(LuauTxnLogRefreshFunctionPointers, false) -LUAU_FASTFLAGVARIABLE(LuauTxnLogDontRetryForIndexers, false) LUAU_FASTFLAG(LuauAnyInIsOptionalIsOptional) LUAU_FASTFLAG(LuauTypecheckOptPass) @@ -1021,7 +1018,7 @@ void Unifier::tryUnify_(TypePackId subTp, TypePackId superTp, bool isFunctionCal if (superTp == subTp) return; - if (FFlag::LuauTxnLogSeesTypePacks2 && log.haveSeen(superTp, subTp)) + if (log.haveSeen(superTp, subTp)) return; if (log.getMutable(superTp)) @@ -1265,12 +1262,9 @@ void Unifier::tryUnifyFunctions(TypeId subTy, TypeId superTy, bool isFunctionCal log.pushSeen(superFunction->generics[i], subFunction->generics[i]); } - if (FFlag::LuauTxnLogSeesTypePacks2) + for (size_t i = 0; i < numGenericPacks; i++) { - for (size_t i = 0; i < numGenericPacks; i++) - { - log.pushSeen(superFunction->genericPacks[i], subFunction->genericPacks[i]); - } + log.pushSeen(superFunction->genericPacks[i], subFunction->genericPacks[i]); } CountMismatch::Context context = ctx; @@ -1330,12 +1324,9 @@ void Unifier::tryUnifyFunctions(TypeId subTy, TypeId superTy, bool isFunctionCal ctx = context; - if (FFlag::LuauTxnLogSeesTypePacks2) + for (int i = int(numGenericPacks) - 1; 0 <= i; i--) { - for (int i = int(numGenericPacks) - 1; 0 <= i; i--) - { - log.popSeen(superFunction->genericPacks[i], subFunction->genericPacks[i]); - } + log.popSeen(superFunction->genericPacks[i], subFunction->genericPacks[i]); } for (int i = int(numGenerics) - 1; 0 <= i; i--) @@ -1499,20 +1490,17 @@ void Unifier::tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) else missingProperties.push_back(name); - if (FFlag::LuauTxnLogCheckForInvalidation) + // Recursive unification can change the txn log, and invalidate the old + // table. If we detect that this has happened, we start over, with the updated + // txn log. + TableTypeVar* newSuperTable = log.getMutable(superTy); + TableTypeVar* newSubTable = log.getMutable(subTy); + if (superTable != newSuperTable || subTable != newSubTable) { - // Recursive unification can change the txn log, and invalidate the old - // table. If we detect that this has happened, we start over, with the updated - // txn log. - TableTypeVar* newSuperTable = log.getMutable(superTy); - TableTypeVar* newSubTable = log.getMutable(subTy); - if (superTable != newSuperTable || subTable != newSubTable) - { - if (errors.empty()) - return tryUnifyTables(subTy, superTy, isIntersection); - else - return; - } + if (errors.empty()) + return tryUnifyTables(subTy, superTy, isIntersection); + else + return; } } @@ -1570,20 +1558,17 @@ void Unifier::tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) else extraProperties.push_back(name); - if (FFlag::LuauTxnLogCheckForInvalidation) + // Recursive unification can change the txn log, and invalidate the old + // table. If we detect that this has happened, we start over, with the updated + // txn log. + TableTypeVar* newSuperTable = log.getMutable(superTy); + TableTypeVar* newSubTable = log.getMutable(subTy); + if (superTable != newSuperTable || subTable != newSubTable) { - // Recursive unification can change the txn log, and invalidate the old - // table. If we detect that this has happened, we start over, with the updated - // txn log. - TableTypeVar* newSuperTable = log.getMutable(superTy); - TableTypeVar* newSubTable = log.getMutable(subTy); - if (superTable != newSuperTable || subTable != newSubTable) - { - if (errors.empty()) - return tryUnifyTables(subTy, superTy, isIntersection); - else - return; - } + if (errors.empty()) + return tryUnifyTables(subTy, superTy, isIntersection); + else + return; } } @@ -1630,27 +1615,9 @@ void Unifier::tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) } } - if (FFlag::LuauTxnLogDontRetryForIndexers) - { - // Changing the indexer can invalidate the table pointers. - superTable = log.getMutable(superTy); - subTable = log.getMutable(subTy); - } - else if (FFlag::LuauTxnLogCheckForInvalidation) - { - // Recursive unification can change the txn log, and invalidate the old - // table. If we detect that this has happened, we start over, with the updated - // txn log. - TableTypeVar* newSuperTable = log.getMutable(superTy); - TableTypeVar* newSubTable = log.getMutable(subTy); - if (superTable != newSuperTable || subTable != newSubTable) - { - if (errors.empty()) - return tryUnifyTables(subTy, superTy, isIntersection); - else - return; - } - } + // Changing the indexer can invalidate the table pointers. + superTable = log.getMutable(superTy); + subTable = log.getMutable(subTy); if (!missingProperties.empty()) { diff --git a/Ast/src/Lexer.cpp b/Ast/src/Lexer.cpp index 5dd4f04e..a1f1d469 100644 --- a/Ast/src/Lexer.cpp +++ b/Ast/src/Lexer.cpp @@ -6,8 +6,6 @@ #include -LUAU_FASTFLAGVARIABLE(LuauParseLocationIgnoreCommentSkip, false) - namespace Luau { @@ -361,7 +359,7 @@ const Lexeme& Lexer::next(bool skipComments, bool updatePrevLocation) while (isSpace(peekch())) consume(); - if (!FFlag::LuauParseLocationIgnoreCommentSkip || updatePrevLocation) + if (updatePrevLocation) prevLocation = lexeme.location; lexeme = readNext(); diff --git a/CLI/FileUtils.cpp b/CLI/FileUtils.cpp index fb6ac373..39a14ec7 100644 --- a/CLI/FileUtils.cpp +++ b/CLI/FileUtils.cpp @@ -240,7 +240,7 @@ std::optional getParentPath(const std::string& path) return std::nullopt; #endif - std::string::size_type slash = path.find_last_of("\\/", path.size() - 1); + size_t slash = path.find_last_of("\\/", path.size() - 1); if (slash == 0) return "/"; @@ -253,7 +253,7 @@ std::optional getParentPath(const std::string& path) static std::string getExtension(const std::string& path) { - std::string::size_type dot = path.find_last_of(".\\/"); + size_t dot = path.find_last_of(".\\/"); if (dot == std::string::npos || path[dot] != '.') return ""; diff --git a/CLI/Repl.cpp b/CLI/Repl.cpp index 345cb7ac..4cb22346 100644 --- a/CLI/Repl.cpp +++ b/CLI/Repl.cpp @@ -34,7 +34,8 @@ enum class CliMode enum class CompileFormat { Text, - Binary + Binary, + Null }; constexpr int MaxTraversalLimit = 50; @@ -594,6 +595,8 @@ static bool compileFile(const char* name, CompileFormat format) case CompileFormat::Binary: fwrite(bcb.getBytecode().data(), 1, bcb.getBytecode().size(), stdout); break; + case CompileFormat::Null: + break; } return true; @@ -716,6 +719,10 @@ int replMain(int argc, char** argv) { compileFormat = CompileFormat::Text; } + else if (strcmp(argv[1], "--compile=null") == 0) + { + compileFormat = CompileFormat::Null; + } else { fprintf(stderr, "Error: Unrecognized value for '--compile' specified.\n"); diff --git a/Compiler/include/Luau/BytecodeBuilder.h b/Compiler/include/Luau/BytecodeBuilder.h index 67b93028..b00440ae 100644 --- a/Compiler/include/Luau/BytecodeBuilder.h +++ b/Compiler/include/Luau/BytecodeBuilder.h @@ -232,7 +232,7 @@ private: DenseHashMap stringTable; - DenseHashMap debugRemarks; + std::vector> debugRemarks; std::string debugRemarkBuffer; BytecodeEncoder* encoder = nullptr; diff --git a/Compiler/src/BytecodeBuilder.cpp b/Compiler/src/BytecodeBuilder.cpp index 6c6f1225..871a1484 100644 --- a/Compiler/src/BytecodeBuilder.cpp +++ b/Compiler/src/BytecodeBuilder.cpp @@ -181,7 +181,6 @@ BytecodeBuilder::BytecodeBuilder(BytecodeEncoder* encoder) : constantMap({Constant::Type_Nil, ~0ull}) , tableShapeMap(TableShape()) , stringTable({nullptr, 0}) - , debugRemarks(~0u) , encoder(encoder) { LUAU_ASSERT(stringTable.find(StringRef{"", 0}) == nullptr); @@ -257,6 +256,8 @@ void BytecodeBuilder::endFunction(uint8_t maxstacksize, uint8_t numupvalues) void BytecodeBuilder::setMainFunction(uint32_t fid) { + LUAU_ASSERT(fid < functions.size()); + mainFunction = fid; } @@ -531,7 +532,7 @@ void BytecodeBuilder::addDebugRemark(const char* format, ...) // we null-terminate all remarks to avoid storing remark length debugRemarkBuffer += '\0'; - debugRemarks[uint32_t(insns.size())] = uint32_t(offset); + debugRemarks.emplace_back(uint32_t(insns.size()), uint32_t(offset)); } void BytecodeBuilder::finalize() @@ -1719,6 +1720,7 @@ std::string BytecodeBuilder::dumpCurrentFunction() const const uint32_t* codeEnd = insns.data() + insns.size(); int lastLine = -1; + size_t nextRemark = 0; std::string result; @@ -1741,6 +1743,7 @@ std::string BytecodeBuilder::dumpCurrentFunction() const while (code != codeEnd) { uint8_t op = LUAU_INSN_OP(*code); + uint32_t pc = uint32_t(code - insns.data()); if (op == LOP_PREPVARARGS) { @@ -1751,15 +1754,16 @@ std::string BytecodeBuilder::dumpCurrentFunction() const if (dumpFlags & Dump_Remarks) { - const uint32_t* remark = debugRemarks.find(uint32_t(code - insns.data())); - - if (remark) - formatAppend(result, "REMARK %s\n", debugRemarkBuffer.c_str() + *remark); + while (nextRemark < debugRemarks.size() && debugRemarks[nextRemark].first == pc) + { + formatAppend(result, "REMARK %s\n", debugRemarkBuffer.c_str() + debugRemarks[nextRemark].second); + nextRemark++; + } } if (dumpFlags & Dump_Source) { - int line = lines[code - insns.data()]; + int line = lines[pc]; if (line > 0 && line != lastLine) { @@ -1771,7 +1775,7 @@ std::string BytecodeBuilder::dumpCurrentFunction() const if (dumpFlags & Dump_Lines) { - formatAppend(result, "%d: ", lines[code - insns.data()]); + formatAppend(result, "%d: ", lines[pc]); } code = dumpInstruction(code, result); @@ -1784,11 +1788,11 @@ void BytecodeBuilder::setDumpSource(const std::string& source) { dumpSource.clear(); - std::string::size_type pos = 0; + size_t pos = 0; while (pos != std::string::npos) { - std::string::size_type next = source.find('\n', pos); + size_t next = source.find('\n', pos); if (next == std::string::npos) { diff --git a/Compiler/src/Compiler.cpp b/Compiler/src/Compiler.cpp index 810caaee..0f17ee02 100644 --- a/Compiler/src/Compiler.cpp +++ b/Compiler/src/Compiler.cpp @@ -2206,9 +2206,15 @@ struct Compiler return false; } + if (Variable* lv = variables.find(stat->var); lv && lv->written) + { + bytecode.addDebugRemark("loop unroll failed: mutable loop variable"); + return false; + } + int tripCount = (to - from) / step + 1; - if (tripCount > thresholdBase * thresholdMaxBoost / 100) + if (tripCount > thresholdBase) { bytecode.addDebugRemark("loop unroll failed: too many iterations (%d)", tripCount); return false; diff --git a/Compiler/src/CostModel.cpp b/Compiler/src/CostModel.cpp index d8511bdb..9afd09f6 100644 --- a/Compiler/src/CostModel.cpp +++ b/Compiler/src/CostModel.cpp @@ -249,7 +249,7 @@ int computeCost(uint64_t model, const bool* varsConst, size_t varCount) return cost; for (size_t i = 0; i < varCount && i < 7; ++i) - cost -= int((model >> (8 * i + 8)) & 0x7f) * varsConst[i]; + cost -= int((model >> (i * 8 + 8)) & 0x7f) * varsConst[i]; return cost; } diff --git a/Sources.cmake b/Sources.cmake index 60e5dfda..f9263b24 100644 --- a/Sources.cmake +++ b/Sources.cmake @@ -220,8 +220,8 @@ if(TARGET Luau.UnitTest) tests/Autocomplete.test.cpp tests/BuiltinDefinitions.test.cpp tests/Compiler.test.cpp - tests/CostModel.test.cpp tests/Config.test.cpp + tests/CostModel.test.cpp tests/Error.test.cpp tests/Frontend.test.cpp tests/JsonEncoder.test.cpp @@ -232,6 +232,7 @@ if(TARGET Luau.UnitTest) tests/Normalize.test.cpp tests/Parser.test.cpp tests/RequireTracer.test.cpp + tests/RuntimeLimits.test.cpp tests/StringUtils.test.cpp tests/Symbol.test.cpp tests/ToDot.test.cpp diff --git a/VM/include/lua.h b/VM/include/lua.h index d08b73ea..c3ebadb1 100644 --- a/VM/include/lua.h +++ b/VM/include/lua.h @@ -299,7 +299,7 @@ LUA_API uintptr_t lua_encodepointer(lua_State* L, uintptr_t p); LUA_API double lua_clock(); -LUA_API void lua_setuserdatadtor(lua_State* L, int tag, void (*dtor)(void*)); +LUA_API void lua_setuserdatadtor(lua_State* L, int tag, void (*dtor)(lua_State*, void*)); LUA_API void lua_clonefunction(lua_State* L, int idx); diff --git a/VM/src/lapi.cpp b/VM/src/lapi.cpp index 431f7e59..1f3b0943 100644 --- a/VM/src/lapi.cpp +++ b/VM/src/lapi.cpp @@ -1323,7 +1323,7 @@ void lua_unref(lua_State* L, int ref) return; } -void lua_setuserdatadtor(lua_State* L, int tag, void (*dtor)(void*)) +void lua_setuserdatadtor(lua_State* L, int tag, void (*dtor)(lua_State*, void*)) { api_check(L, unsigned(tag) < LUA_UTAG_LIMIT); L->global->udatagc[tag] = dtor; diff --git a/VM/src/lstate.h b/VM/src/lstate.h index 45d9ba2c..423514a7 100644 --- a/VM/src/lstate.h +++ b/VM/src/lstate.h @@ -200,7 +200,7 @@ typedef struct global_State uint64_t rngstate; /* PCG random number generator state */ uint64_t ptrenckey[4]; /* pointer encoding key for display */ - void (*udatagc[LUA_UTAG_LIMIT])(void*); /* for each userdata tag, a gc callback to be called immediately before freeing memory */ + void (*udatagc[LUA_UTAG_LIMIT])(lua_State*, void*); /* for each userdata tag, a gc callback to be called immediately before freeing memory */ lua_Callbacks cb; diff --git a/VM/src/ltable.cpp b/VM/src/ltable.cpp index dc40b6ef..3dc3bd1b 100644 --- a/VM/src/ltable.cpp +++ b/VM/src/ltable.cpp @@ -33,7 +33,6 @@ #include -LUAU_FASTFLAGVARIABLE(LuauTableRehashRework, false) LUAU_FASTFLAGVARIABLE(LuauTableNewBoundary2, false) // max size of both array and hash part is 2^MAXBITS @@ -400,16 +399,9 @@ static void resize(lua_State* L, Table* t, int nasize, int nhsize) { if (!ttisnil(&t->array[i])) { - if (FFlag::LuauTableRehashRework) - { - TValue ok; - setnvalue(&ok, cast_num(i + 1)); - setobjt2t(L, newkey(L, t, &ok), &t->array[i]); - } - else - { - setobjt2t(L, luaH_setnum(L, t, i + 1), &t->array[i]); - } + TValue ok; + setnvalue(&ok, cast_num(i + 1)); + setobjt2t(L, newkey(L, t, &ok), &t->array[i]); } } /* shrink array */ @@ -418,30 +410,14 @@ static void resize(lua_State* L, Table* t, int nasize, int nhsize) /* used for the migration check at the end */ TValue* anew = t->array; /* re-insert elements from hash part */ - if (FFlag::LuauTableRehashRework) + for (int i = twoto(oldhsize) - 1; i >= 0; i--) { - for (int i = twoto(oldhsize) - 1; i >= 0; i--) + LuaNode* old = nold + i; + if (!ttisnil(gval(old))) { - LuaNode* old = nold + i; - if (!ttisnil(gval(old))) - { - TValue ok; - getnodekey(L, &ok, old); - setobjt2t(L, arrayornewkey(L, t, &ok), gval(old)); - } - } - } - else - { - for (int i = twoto(oldhsize) - 1; i >= 0; i--) - { - LuaNode* old = nold + i; - if (!ttisnil(gval(old))) - { - TValue ok; - getnodekey(L, &ok, old); - setobjt2t(L, luaH_set(L, t, &ok), gval(old)); - } + TValue ok; + getnodekey(L, &ok, old); + setobjt2t(L, arrayornewkey(L, t, &ok), gval(old)); } } @@ -559,7 +535,7 @@ static TValue* newkey(lua_State* L, Table* t, const TValue* key) { rehash(L, t, key); /* grow table */ - // after rehash, numeric keys might be located in the new array part, but won't be found in the node part + /* after rehash, numeric keys might be located in the new array part, but won't be found in the node part */ return arrayornewkey(L, t, key); } @@ -571,15 +547,8 @@ static TValue* newkey(lua_State* L, Table* t, const TValue* key) { /* cannot find a free place? */ rehash(L, t, key); /* grow table */ - if (!FFlag::LuauTableRehashRework) - { - return luaH_set(L, t, key); /* re-insert key into grown table */ - } - else - { - // after rehash, numeric keys might be located in the new array part, but won't be found in the node part - return arrayornewkey(L, t, key); - } + /* after rehash, numeric keys might be located in the new array part, but won't be found in the node part */ + return arrayornewkey(L, t, key); } LUAU_ASSERT(n != dummynode); TValue mk; diff --git a/VM/src/ludata.cpp b/VM/src/ludata.cpp index 819d1863..28152689 100644 --- a/VM/src/ludata.cpp +++ b/VM/src/ludata.cpp @@ -22,14 +22,21 @@ Udata* luaU_newudata(lua_State* L, size_t s, int tag) void luaU_freeudata(lua_State* L, Udata* u, lua_Page* page) { - void (*dtor)(void*) = nullptr; if (u->tag < LUA_UTAG_LIMIT) + { + void (*dtor)(lua_State*, void*) = nullptr; dtor = L->global->udatagc[u->tag]; + if (dtor) + dtor(L, u->data); + } else if (u->tag == UTAG_IDTOR) + { + void (*dtor)(void*) = nullptr; memcpy(&dtor, &u->data + u->len - sizeof(dtor), sizeof(dtor)); + if (dtor) + dtor(u->data); + } - if (dtor) - dtor(u->data); luaM_freegco(L, u, sizeudata(u->len), u->memcat, page); } diff --git a/fuzz/proto.cpp b/fuzz/proto.cpp index a48f068b..22483f9e 100644 --- a/fuzz/proto.cpp +++ b/fuzz/proto.cpp @@ -137,6 +137,21 @@ int registerTypes(Luau::TypeChecker& env) return 0; } + +static void setupFrontend(Luau::Frontend& frontend) +{ + registerTypes(frontend.typeChecker); + Luau::freeze(frontend.typeChecker.globalTypes); + + registerTypes(frontend.typeCheckerForAutocomplete); + Luau::freeze(frontend.typeCheckerForAutocomplete.globalTypes); + + frontend.iceHandler.onInternalError = [](const char* error) { + printf("ICE: %s\n", error); + LUAU_ASSERT(!"ICE"); + }; +} + struct FuzzFileResolver : Luau::FileResolver { std::optional readSource(const Luau::ModuleName& name) override @@ -238,19 +253,11 @@ DEFINE_PROTO_FUZZER(const luau::ModuleSet& message) if (kFuzzTypeck) { static FuzzFileResolver fileResolver; - static Luau::NullConfigResolver configResolver; + static FuzzConfigResolver configResolver; static Luau::FrontendOptions options{true, true}; static Luau::Frontend frontend(&fileResolver, &configResolver, options); - static int once = registerTypes(frontend.typeChecker); - (void)once; - static int once2 = (Luau::freeze(frontend.typeChecker.globalTypes), 0); - (void)once2; - - frontend.iceHandler.onInternalError = [](const char* error) { - printf("ICE: %s\n", error); - LUAU_ASSERT(!"ICE"); - }; + static int once = (setupFrontend(frontend), 0); // restart frontend.clear(); diff --git a/tests/Autocomplete.test.cpp b/tests/Autocomplete.test.cpp index f66e23ed..5b70481b 100644 --- a/tests/Autocomplete.test.cpp +++ b/tests/Autocomplete.test.cpp @@ -2761,7 +2761,6 @@ TEST_CASE_FIXTURE(ACFixture, "autocomplete_on_string_singletons") TEST_CASE_FIXTURE(ACFixture, "autocomplete_string_singletons") { ScopedFastFlag luauAutocompleteSingletonTypes{"LuauAutocompleteSingletonTypes", true}; - ScopedFastFlag luauExpectedTypesOfProperties{"LuauExpectedTypesOfProperties", true}; check(R"( type tag = "cat" | "dog" diff --git a/tests/Compiler.test.cpp b/tests/Compiler.test.cpp index f3e60690..7b4bfc72 100644 --- a/tests/Compiler.test.cpp +++ b/tests/Compiler.test.cpp @@ -2698,16 +2698,22 @@ TEST_CASE("DebugRemarks") uint32_t fid = bcb.beginFunction(0); - bcb.addDebugRemark("test remark #%d", 42); + bcb.addDebugRemark("test remark #%d", 1); + bcb.emitABC(LOP_LOADNIL, 0, 0, 0); + bcb.addDebugRemark("test remark #%d", 2); + bcb.addDebugRemark("test remark #%d", 3); bcb.emitABC(LOP_RETURN, 0, 1, 0); - bcb.endFunction(0, 0); + bcb.endFunction(1, 0); bcb.setMainFunction(fid); bcb.finalize(); CHECK_EQ("\n" + bcb.dumpFunction(0), R"( -REMARK test remark #42 +REMARK test remark #1 +LOADNIL R0 +REMARK test remark #2 +REMARK test remark #3 RETURN R0 0 )"); } @@ -4332,7 +4338,7 @@ RETURN R0 1 // loops with body that's long but has a high boost factor due to constant folding CHECK_EQ("\n" + compileFunction(R"( local t = {} -for i=1,30 do +for i=1,25 do t[i] = i * i * i end return t @@ -4390,16 +4396,6 @@ LOADN R1 13824 SETTABLEN R1 R0 24 LOADN R1 15625 SETTABLEN R1 R0 25 -LOADN R1 17576 -SETTABLEN R1 R0 26 -LOADN R1 19683 -SETTABLEN R1 R0 27 -LOADN R1 21952 -SETTABLEN R1 R0 28 -LOADN R1 24389 -SETTABLEN R1 R0 29 -LOADN R1 27000 -SETTABLEN R1 R0 30 RETURN R0 1 )"); @@ -4431,4 +4427,30 @@ RETURN R0 1 )"); } +TEST_CASE("LoopUnrollMutable") +{ + // can't unroll loops that mutate iteration variable + CHECK_EQ("\n" + compileFunction(R"( +for i=1,3 do + i = 3 + print(i) -- should print 3 three times in a row +end +)", + 0, 2), + R"( +LOADN R2 1 +LOADN R0 3 +LOADN R1 1 +FORNPREP R0 +7 +MOVE R3 R2 +LOADN R3 3 +GETIMPORT R4 1 +MOVE R5 R3 +CALL R4 1 0 +FORNLOOP R0 -7 +RETURN R0 0 +)"); +} + + TEST_SUITE_END(); diff --git a/tests/Conformance.test.cpp b/tests/Conformance.test.cpp index 0ed7dc44..6f136d36 100644 --- a/tests/Conformance.test.cpp +++ b/tests/Conformance.test.cpp @@ -1056,7 +1056,7 @@ TEST_CASE("UserdataApi") lua_State* L = globalState.get(); // setup dtor for tag 42 (created later) - lua_setuserdatadtor(L, 42, [](void* data) { + lua_setuserdatadtor(L, 42, [](lua_State* l, void* data) { dtorhits += *(int*)data; }); diff --git a/tests/Frontend.test.cpp b/tests/Frontend.test.cpp index 9fc0a005..e771b6b1 100644 --- a/tests/Frontend.test.cpp +++ b/tests/Frontend.test.cpp @@ -975,8 +975,6 @@ TEST_CASE_FIXTURE(FrontendFixture, "typecheck_twice_for_ast_types") TEST_CASE_FIXTURE(FrontendFixture, "imported_table_modification_2") { - ScopedFastFlag sffs("LuauSealExports", true); - frontend.options.retainFullTypeGraphs = false; fileResolver.source["Module/A"] = R"( @@ -1035,4 +1033,20 @@ return false; fix.frontend.check("Module/B"); } +TEST_CASE("check_without_builtin_next") +{ + ScopedFastFlag luauDoNotRelyOnNextBinding{"LuauDoNotRelyOnNextBinding", true}; + + TestFileResolver fileResolver; + TestConfigResolver configResolver; + Frontend frontend(&fileResolver, &configResolver); + + fileResolver.source["Module/A"] = "for k,v in 2 do end"; + fileResolver.source["Module/B"] = "return next"; + + // We don't care about the result. That we haven't crashed is enough. + frontend.check("Module/A"); + frontend.check("Module/B"); +} + TEST_SUITE_END(); diff --git a/tests/Module.test.cpp b/tests/Module.test.cpp index af7d76de..44cc20a7 100644 --- a/tests/Module.test.cpp +++ b/tests/Module.test.cpp @@ -199,7 +199,6 @@ TEST_CASE_FIXTURE(Fixture, "clone_class") TEST_CASE_FIXTURE(Fixture, "clone_free_types") { ScopedFastFlag sff[]{ - {"LuauErrorRecoveryType", true}, {"LuauLosslessClone", true}, }; diff --git a/tests/NonstrictMode.test.cpp b/tests/NonstrictMode.test.cpp index 9748eb27..feeaf2c2 100644 --- a/tests/NonstrictMode.test.cpp +++ b/tests/NonstrictMode.test.cpp @@ -283,7 +283,6 @@ TEST_CASE_FIXTURE(Fixture, "inconsistent_module_return_types_are_ok") ScopedFastFlag sff[]{ {"LuauReturnTypeInferenceInNonstrict", true}, {"LuauLowerBoundsCalculation", true}, - {"LuauSealExports", true}, }; CheckResult result = check(R"( diff --git a/tests/Parser.test.cpp b/tests/Parser.test.cpp index b941103d..55eafe3c 100644 --- a/tests/Parser.test.cpp +++ b/tests/Parser.test.cpp @@ -1606,8 +1606,6 @@ TEST_CASE_FIXTURE(Fixture, "end_extent_of_functions_unions_and_intersections") TEST_CASE_FIXTURE(Fixture, "end_extent_doesnt_consume_comments") { - ScopedFastFlag luauParseLocationIgnoreCommentSkip{"LuauParseLocationIgnoreCommentSkip", true}; - AstStatBlock* block = parse(R"( type F = number --comment @@ -1620,7 +1618,6 @@ TEST_CASE_FIXTURE(Fixture, "end_extent_doesnt_consume_comments") TEST_CASE_FIXTURE(Fixture, "end_extent_doesnt_consume_comments_even_with_capture") { - ScopedFastFlag luauParseLocationIgnoreCommentSkip{"LuauParseLocationIgnoreCommentSkip", true}; ScopedFastFlag luauParseLocationIgnoreCommentSkipInCapture{"LuauParseLocationIgnoreCommentSkipInCapture", true}; // Same should hold when comments are captured diff --git a/tests/RuntimeLimits.test.cpp b/tests/RuntimeLimits.test.cpp new file mode 100644 index 00000000..dcbf0b61 --- /dev/null +++ b/tests/RuntimeLimits.test.cpp @@ -0,0 +1,270 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details + +/* Tests in this source file are meant to be a bellwether to verify that the numeric limits we've set are sufficient for + * most real-world scripts. + * + * If a change breaks a test in this source file, please don't adjust the flag values set in the fixture. Instead, + * consider it a latent performance problem by default. + * + * We should periodically revisit this to retest the limits. + */ + +#include "Fixture.h" + +#include "doctest.h" + +using namespace Luau; + +LUAU_FASTFLAG(LuauLowerBoundsCalculation); + +struct LimitFixture : Fixture +{ +#if defined(_NOOPT) || defined(_DEBUG) + ScopedFastInt LuauTypeInferRecursionLimit{"LuauTypeInferRecursionLimit", 100}; +#endif + + ScopedFastFlag LuauJustOneCallFrameForHaveSeen{"LuauJustOneCallFrameForHaveSeen", true}; +}; + +template +bool hasError(const CheckResult& result, T* = nullptr) +{ + auto it = std::find_if(result.errors.begin(), result.errors.end(), [](const TypeError& a) { + return nullptr != get(a); + }); + return it != result.errors.end(); +} + +TEST_SUITE_BEGIN("RuntimeLimitTests"); + +TEST_CASE_FIXTURE(LimitFixture, "bail_early_on_typescript_port_of_Result_type" * doctest::timeout(1.0)) +{ + constexpr const char* src = R"LUA( + --!strict + local TS = _G[script] + local lazyGet = TS.import(script, script.Parent.Parent, "util", "lazyLoad").lazyGet + local unit = TS.import(script, script.Parent.Parent, "util", "Unit").unit + local Iterator + lazyGet("Iterator", function(c) + Iterator = c + end) + local Option + lazyGet("Option", function(c) + Option = c + end) + local Vec + lazyGet("Vec", function(c) + Vec = c + end) + local Result + do + Result = setmetatable({}, { + __tostring = function() + return "Result" + end, + }) + Result.__index = Result + function Result.new(...) + local self = setmetatable({}, Result) + self:constructor(...) + return self + end + function Result:constructor(okValue, errValue) + self.okValue = okValue + self.errValue = errValue + end + function Result:ok(val) + return Result.new(val, nil) + end + function Result:err(val) + return Result.new(nil, val) + end + function Result:fromCallback(c) + local _0 = c + local _1, _2 = pcall(_0) + local result = _1 and { + success = true, + value = _2, + } or { + success = false, + error = _2, + } + return result.success and Result:ok(result.value) or Result:err(Option:wrap(result.error)) + end + function Result:fromVoidCallback(c) + local _0 = c + local _1, _2 = pcall(_0) + local result = _1 and { + success = true, + value = _2, + } or { + success = false, + error = _2, + } + return result.success and Result:ok(unit()) or Result:err(Option:wrap(result.error)) + end + Result.fromPromise = TS.async(function(self, p) + local _0, _1 = TS.try(function() + return TS.TRY_RETURN, { Result:ok(TS.await(p)) } + end, function(e) + return TS.TRY_RETURN, { Result:err(Option:wrap(e)) } + end) + if _0 then + return unpack(_1) + end + end) + Result.fromVoidPromise = TS.async(function(self, p) + local _0, _1 = TS.try(function() + TS.await(p) + return TS.TRY_RETURN, { Result:ok(unit()) } + end, function(e) + return TS.TRY_RETURN, { Result:err(Option:wrap(e)) } + end) + if _0 then + return unpack(_1) + end + end) + function Result:isOk() + return self.okValue ~= nil + end + function Result:isErr() + return self.errValue ~= nil + end + function Result:contains(x) + return self.okValue == x + end + function Result:containsErr(x) + return self.errValue == x + end + function Result:okOption() + return Option:wrap(self.okValue) + end + function Result:errOption() + return Option:wrap(self.errValue) + end + function Result:map(func) + return self:isOk() and Result:ok(func(self.okValue)) or Result:err(self.errValue) + end + function Result:mapOr(def, func) + local _0 + if self:isOk() then + _0 = func(self.okValue) + else + _0 = def + end + return _0 + end + function Result:mapOrElse(def, func) + local _0 + if self:isOk() then + _0 = func(self.okValue) + else + _0 = def(self.errValue) + end + return _0 + end + function Result:mapErr(func) + return self:isErr() and Result:err(func(self.errValue)) or Result:ok(self.okValue) + end + Result["and"] = function(self, other) + return self:isErr() and Result:err(self.errValue) or other + end + function Result:andThen(func) + return self:isErr() and Result:err(self.errValue) or func(self.okValue) + end + Result["or"] = function(self, other) + return self:isOk() and Result:ok(self.okValue) or other + end + function Result:orElse(other) + return self:isOk() and Result:ok(self.okValue) or other(self.errValue) + end + function Result:expect(msg) + if self:isOk() then + return self.okValue + else + error(msg) + end + end + function Result:unwrap() + return self:expect("called `Result.unwrap()` on an `Err` value: " .. tostring(self.errValue)) + end + function Result:unwrapOr(def) + local _0 + if self:isOk() then + _0 = self.okValue + else + _0 = def + end + return _0 + end + function Result:unwrapOrElse(gen) + local _0 + if self:isOk() then + _0 = self.okValue + else + _0 = gen(self.errValue) + end + return _0 + end + function Result:expectErr(msg) + if self:isErr() then + return self.errValue + else + error(msg) + end + end + function Result:unwrapErr() + return self:expectErr("called `Result.unwrapErr()` on an `Ok` value: " .. tostring(self.okValue)) + end + function Result:transpose() + return self:isOk() and self.okValue:map(function(some) + return Result:ok(some) + end) or Option:some(Result:err(self.errValue)) + end + function Result:flatten() + return self:isOk() and Result.new(self.okValue.okValue, self.okValue.errValue) or Result:err(self.errValue) + end + function Result:match(ifOk, ifErr) + local _0 + if self:isOk() then + _0 = ifOk(self.okValue) + else + _0 = ifErr(self.errValue) + end + return _0 + end + function Result:asPtr() + local _0 = (self.okValue) + if _0 == nil then + _0 = (self.errValue) + end + return _0 + end + end + local resultMeta = Result + resultMeta.__eq = function(a, b) + return b:match(function(ok) + return a:contains(ok) + end, function(err) + return a:containsErr(err) + end) + end + resultMeta.__tostring = function(result) + return result:match(function(ok) + return "Result.ok(" .. tostring(ok) .. ")" + end, function(err) + return "Result.err(" .. tostring(err) .. ")" + end) + end + return { + Result = Result, + } + )LUA"; + + if (FFlag::LuauLowerBoundsCalculation) + (void)check(src); + else + CHECK_THROWS_AS(check(src), std::exception); +} + +TEST_SUITE_END(); diff --git a/tests/TypeInfer.aliases.test.cpp b/tests/TypeInfer.aliases.test.cpp index b2e76052..b0eb31ce 100644 --- a/tests/TypeInfer.aliases.test.cpp +++ b/tests/TypeInfer.aliases.test.cpp @@ -7,8 +7,6 @@ using namespace Luau; -LUAU_FASTFLAG(LuauFixIncorrectLineNumberDuplicateType) - TEST_SUITE_BEGIN("TypeAliases"); TEST_CASE_FIXTURE(Fixture, "cyclic_function_type_in_type_alias") @@ -257,11 +255,7 @@ TEST_CASE_FIXTURE(Fixture, "reported_location_is_correct_when_type_alias_are_dup auto dtd = get(result.errors[0]); REQUIRE(dtd); CHECK_EQ(dtd->name, "B"); - - if (FFlag::LuauFixIncorrectLineNumberDuplicateType) - CHECK_EQ(dtd->previousLocation.begin.line + 1, 3); - else - CHECK_EQ(dtd->previousLocation.begin.line + 1, 1); + CHECK_EQ(dtd->previousLocation.begin.line + 1, 3); } TEST_CASE_FIXTURE(Fixture, "stringify_optional_parameterized_alias") @@ -495,8 +489,6 @@ TEST_CASE_FIXTURE(Fixture, "mutually_recursive_types_restriction_ok") TEST_CASE_FIXTURE(Fixture, "mutually_recursive_types_restriction_not_ok_1") { - ScopedFastFlag sff{"LuauRecursiveTypeParameterRestriction", true}; - CheckResult result = check(R"( -- OK because forwarded types are used with their parameters. type Tree = { data: T, children: Forest } @@ -508,8 +500,6 @@ TEST_CASE_FIXTURE(Fixture, "mutually_recursive_types_restriction_not_ok_1") TEST_CASE_FIXTURE(Fixture, "mutually_recursive_types_restriction_not_ok_2") { - ScopedFastFlag sff{"LuauRecursiveTypeParameterRestriction", true}; - CheckResult result = check(R"( -- Not OK because forwarded types are used with different types than their parameters. type Forest = {Tree<{T}>} @@ -531,8 +521,6 @@ TEST_CASE_FIXTURE(Fixture, "mutually_recursive_types_swapsies_ok") TEST_CASE_FIXTURE(Fixture, "mutually_recursive_types_swapsies_not_ok") { - ScopedFastFlag sff{"LuauRecursiveTypeParameterRestriction", true}; - CheckResult result = check(R"( type Tree1 = { data: T, children: {Tree2} } type Tree2 = { data: U, children: {Tree1} } @@ -647,9 +635,6 @@ TEST_CASE_FIXTURE(Fixture, "forward_declared_alias_is_not_clobbered_by_prior_uni { ScopedFastFlag sff[] = { {"LuauTwoPassAliasDefinitionFix", true}, - - // We also force this flag because it surfaced an unfortunate interaction. - {"LuauErrorRecoveryType", true}, }; CheckResult result = check(R"( @@ -687,8 +672,6 @@ TEST_CASE_FIXTURE(Fixture, "recursive_types_restriction_ok") TEST_CASE_FIXTURE(Fixture, "recursive_types_restriction_not_ok") { - ScopedFastFlag sff{"LuauRecursiveTypeParameterRestriction", true}; - CheckResult result = check(R"( -- this would be an infinite type if we allowed it type Tree = { data: T, children: {Tree<{T}>} } diff --git a/tests/TypeInfer.annotations.test.cpp b/tests/TypeInfer.annotations.test.cpp index e2971ad5..7f1c757a 100644 --- a/tests/TypeInfer.annotations.test.cpp +++ b/tests/TypeInfer.annotations.test.cpp @@ -221,8 +221,6 @@ TEST_CASE_FIXTURE(Fixture, "as_expr_is_bidirectional") TEST_CASE_FIXTURE(Fixture, "as_expr_warns_on_unrelated_cast") { - ScopedFastFlag sff2{"LuauErrorRecoveryType", true}; - CheckResult result = check(R"( local a = 55 :: string )"); @@ -407,8 +405,6 @@ TEST_CASE_FIXTURE(Fixture, "typeof_expr") TEST_CASE_FIXTURE(Fixture, "corecursive_types_error_on_tight_loop") { - ScopedFastFlag sff{"LuauErrorRecoveryType", true}; - CheckResult result = check(R"( type A = B type B = A diff --git a/tests/TypeInfer.functions.test.cpp b/tests/TypeInfer.functions.test.cpp index 7cd7bec3..0e071217 100644 --- a/tests/TypeInfer.functions.test.cpp +++ b/tests/TypeInfer.functions.test.cpp @@ -951,8 +951,6 @@ TEST_CASE_FIXTURE(Fixture, "record_matching_overload") TEST_CASE_FIXTURE(Fixture, "return_type_by_overload") { - ScopedFastFlag sff{"LuauErrorRecoveryType", true}; - CheckResult result = check(R"( type Overload = ((string) -> string) & ((number, number) -> number) local abc: Overload @@ -1538,7 +1536,6 @@ caused by: TEST_CASE_FIXTURE(Fixture, "too_few_arguments_variadic") { - ScopedFastFlag sff{"LuauArgCountMismatchSaysAtLeastWhenVariadic", true}; CheckResult result = check(R"( function test(a: number, b: string, ...) end @@ -1560,8 +1557,6 @@ TEST_CASE_FIXTURE(Fixture, "too_few_arguments_variadic") TEST_CASE_FIXTURE(Fixture, "too_few_arguments_variadic_generic") { - ScopedFastFlag sff1{"LuauArgCountMismatchSaysAtLeastWhenVariadic", true}; - ScopedFastFlag sff2{"LuauFixArgumentCountMismatchAmountWithGenericTypes", true}; CheckResult result = check(R"( function test(a: number, b: string, ...) return 1 @@ -1587,8 +1582,6 @@ wrapper(test) TEST_CASE_FIXTURE(Fixture, "too_few_arguments_variadic_generic2") { - ScopedFastFlag sff1{"LuauArgCountMismatchSaysAtLeastWhenVariadic", true}; - ScopedFastFlag sff2{"LuauFixArgumentCountMismatchAmountWithGenericTypes", true}; CheckResult result = check(R"( function test(a: number, b: string, ...) return 1 diff --git a/tests/TypeInfer.generics.test.cpp b/tests/TypeInfer.generics.test.cpp index 49d31fc6..91be2c1c 100644 --- a/tests/TypeInfer.generics.test.cpp +++ b/tests/TypeInfer.generics.test.cpp @@ -11,8 +11,6 @@ using namespace Luau; -LUAU_FASTFLAG(LuauFixArgumentCountMismatchAmountWithGenericTypes) - TEST_SUITE_BEGIN("GenericsTests"); TEST_CASE_FIXTURE(Fixture, "check_generic_function") @@ -679,8 +677,6 @@ local d: D = c TEST_CASE_FIXTURE(Fixture, "generic_functions_dont_cache_type_parameters") { - ScopedFastFlag sff{"LuauGenericFunctionsDontCacheTypeParams", true}; - CheckResult result = check(R"( -- See https://github.com/Roblox/luau/issues/332 -- This function has a type parameter with the same name as clones, @@ -707,8 +703,6 @@ TEST_CASE_FIXTURE(Fixture, "generic_functions_should_be_memory_safe") ScopedFastFlag sffs[] = { {"LuauTableSubtypingVariance2", true}, {"LuauUnsealedTableLiteral", true}, - {"LuauPropertiesGetExpectedType", true}, - {"LuauRecursiveTypeParameterRestriction", true}, }; CheckResult result = check(R"( @@ -733,8 +727,6 @@ caused by: TEST_CASE_FIXTURE(Fixture, "generic_type_pack_unification1") { - ScopedFastFlag sff{"LuauTxnLogSeesTypePacks2", true}; - CheckResult result = check(R"( --!strict type Dispatcher = { @@ -753,8 +745,6 @@ local TheDispatcher: Dispatcher = { TEST_CASE_FIXTURE(Fixture, "generic_type_pack_unification2") { - ScopedFastFlag sff{"LuauTxnLogSeesTypePacks2", true}; - CheckResult result = check(R"( --!strict type Dispatcher = { @@ -773,8 +763,6 @@ local TheDispatcher: Dispatcher = { TEST_CASE_FIXTURE(Fixture, "generic_type_pack_unification3") { - ScopedFastFlag sff{"LuauTxnLogSeesTypePacks2", true}; - CheckResult result = check(R"( --!strict type Dispatcher = { @@ -805,11 +793,7 @@ wrapper(test) )"); LUAU_REQUIRE_ERROR_COUNT(1, result); - - if (FFlag::LuauFixArgumentCountMismatchAmountWithGenericTypes) - CHECK_EQ(toString(result.errors[0]), R"(Argument count mismatch. Function expects 2 arguments, but only 1 is specified)"); - else - CHECK_EQ(toString(result.errors[0]), R"(Argument count mismatch. Function expects 1 argument, but 1 is specified)"); + CHECK_EQ(toString(result.errors[0]), R"(Argument count mismatch. Function expects 2 arguments, but only 1 is specified)"); } TEST_CASE_FIXTURE(Fixture, "generic_argument_count_too_many") @@ -826,11 +810,7 @@ wrapper(test2, 1, "", 3) )"); LUAU_REQUIRE_ERROR_COUNT(1, result); - - if (FFlag::LuauFixArgumentCountMismatchAmountWithGenericTypes) - CHECK_EQ(toString(result.errors[0]), R"(Argument count mismatch. Function expects 3 arguments, but 4 are specified)"); - else - CHECK_EQ(toString(result.errors[0]), R"(Argument count mismatch. Function expects 1 argument, but 4 are specified)"); + CHECK_EQ(toString(result.errors[0]), R"(Argument count mismatch. Function expects 3 arguments, but 4 are specified)"); } TEST_CASE_FIXTURE(Fixture, "generic_function") diff --git a/tests/TypeInfer.loops.test.cpp b/tests/TypeInfer.loops.test.cpp index 30df717b..960c6edf 100644 --- a/tests/TypeInfer.loops.test.cpp +++ b/tests/TypeInfer.loops.test.cpp @@ -78,6 +78,8 @@ TEST_CASE_FIXTURE(Fixture, "for_in_with_an_iterator_of_type_any") TEST_CASE_FIXTURE(Fixture, "for_in_loop_should_fail_with_non_function_iterator") { + ScopedFastFlag luauDoNotRelyOnNextBinding{"LuauDoNotRelyOnNextBinding", true}; + CheckResult result = check(R"( local foo = "bar" for i, v in foo do @@ -85,6 +87,7 @@ TEST_CASE_FIXTURE(Fixture, "for_in_loop_should_fail_with_non_function_iterator") )"); LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ("Cannot call non-function string", toString(result.errors[0])); } TEST_CASE_FIXTURE(Fixture, "for_in_with_just_one_iterator_is_ok") @@ -470,4 +473,19 @@ TEST_CASE_FIXTURE(Fixture, "loop_typecheck_crash_on_empty_optional") LUAU_REQUIRE_ERROR_COUNT(2, result); } +TEST_CASE_FIXTURE(Fixture, "fuzz_fail_missing_instantitation_follow") +{ + ScopedFastFlag luauInstantiateFollows{"LuauInstantiateFollows", true}; + + // Just check that this doesn't assert + check(R"( + --!nonstrict + function _(l0:number) + return _ + end + for _ in _(8) do + end + )"); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.operators.test.cpp b/tests/TypeInfer.operators.test.cpp index 5f2e2404..a2787cad 100644 --- a/tests/TypeInfer.operators.test.cpp +++ b/tests/TypeInfer.operators.test.cpp @@ -142,8 +142,6 @@ TEST_CASE_FIXTURE(Fixture, "some_primitive_binary_ops") TEST_CASE_FIXTURE(Fixture, "typecheck_overloaded_multiply_that_is_an_intersection") { - ScopedFastFlag sff{"LuauErrorRecoveryType", true}; - CheckResult result = check(R"( --!strict local Vec3 = {} @@ -178,8 +176,6 @@ TEST_CASE_FIXTURE(Fixture, "typecheck_overloaded_multiply_that_is_an_intersectio TEST_CASE_FIXTURE(Fixture, "typecheck_overloaded_multiply_that_is_an_intersection_on_rhs") { - ScopedFastFlag sff{"LuauErrorRecoveryType", true}; - CheckResult result = check(R"( --!strict local Vec3 = {} diff --git a/tests/TypeInfer.primitives.test.cpp b/tests/TypeInfer.primitives.test.cpp index 3ddf9813..e1684df7 100644 --- a/tests/TypeInfer.primitives.test.cpp +++ b/tests/TypeInfer.primitives.test.cpp @@ -85,8 +85,6 @@ TEST_CASE_FIXTURE(Fixture, "string_function_other") TEST_CASE_FIXTURE(Fixture, "CheckMethodsOfNumber") { - ScopedFastFlag sff{"LuauErrorRecoveryType", true}; - CheckResult result = check(R"( local x: number = 9999 function x:y(z: number) diff --git a/tests/TypeInfer.provisional.test.cpp b/tests/TypeInfer.provisional.test.cpp index 4b5075d9..2ef77419 100644 --- a/tests/TypeInfer.provisional.test.cpp +++ b/tests/TypeInfer.provisional.test.cpp @@ -268,242 +268,6 @@ TEST_CASE_FIXTURE(Fixture, "bail_early_if_unification_is_too_complicated" * doct } } -TEST_CASE_FIXTURE(Fixture, "bail_early_on_typescript_port_of_Result_type" * doctest::timeout(1.0)) -{ - ScopedFastInt sffi{"LuauTarjanChildLimit", 400}; - - CheckResult result = check(R"LUA( - --!strict - local TS = _G[script] - local lazyGet = TS.import(script, script.Parent.Parent, "util", "lazyLoad").lazyGet - local unit = TS.import(script, script.Parent.Parent, "util", "Unit").unit - local Iterator - lazyGet("Iterator", function(c) - Iterator = c - end) - local Option - lazyGet("Option", function(c) - Option = c - end) - local Vec - lazyGet("Vec", function(c) - Vec = c - end) - local Result - do - Result = setmetatable({}, { - __tostring = function() - return "Result" - end, - }) - Result.__index = Result - function Result.new(...) - local self = setmetatable({}, Result) - self:constructor(...) - return self - end - function Result:constructor(okValue, errValue) - self.okValue = okValue - self.errValue = errValue - end - function Result:ok(val) - return Result.new(val, nil) - end - function Result:err(val) - return Result.new(nil, val) - end - function Result:fromCallback(c) - local _0 = c - local _1, _2 = pcall(_0) - local result = _1 and { - success = true, - value = _2, - } or { - success = false, - error = _2, - } - return result.success and Result:ok(result.value) or Result:err(Option:wrap(result.error)) - end - function Result:fromVoidCallback(c) - local _0 = c - local _1, _2 = pcall(_0) - local result = _1 and { - success = true, - value = _2, - } or { - success = false, - error = _2, - } - return result.success and Result:ok(unit()) or Result:err(Option:wrap(result.error)) - end - Result.fromPromise = TS.async(function(self, p) - local _0, _1 = TS.try(function() - return TS.TRY_RETURN, { Result:ok(TS.await(p)) } - end, function(e) - return TS.TRY_RETURN, { Result:err(Option:wrap(e)) } - end) - if _0 then - return unpack(_1) - end - end) - Result.fromVoidPromise = TS.async(function(self, p) - local _0, _1 = TS.try(function() - TS.await(p) - return TS.TRY_RETURN, { Result:ok(unit()) } - end, function(e) - return TS.TRY_RETURN, { Result:err(Option:wrap(e)) } - end) - if _0 then - return unpack(_1) - end - end) - function Result:isOk() - return self.okValue ~= nil - end - function Result:isErr() - return self.errValue ~= nil - end - function Result:contains(x) - return self.okValue == x - end - function Result:containsErr(x) - return self.errValue == x - end - function Result:okOption() - return Option:wrap(self.okValue) - end - function Result:errOption() - return Option:wrap(self.errValue) - end - function Result:map(func) - return self:isOk() and Result:ok(func(self.okValue)) or Result:err(self.errValue) - end - function Result:mapOr(def, func) - local _0 - if self:isOk() then - _0 = func(self.okValue) - else - _0 = def - end - return _0 - end - function Result:mapOrElse(def, func) - local _0 - if self:isOk() then - _0 = func(self.okValue) - else - _0 = def(self.errValue) - end - return _0 - end - function Result:mapErr(func) - return self:isErr() and Result:err(func(self.errValue)) or Result:ok(self.okValue) - end - Result["and"] = function(self, other) - return self:isErr() and Result:err(self.errValue) or other - end - function Result:andThen(func) - return self:isErr() and Result:err(self.errValue) or func(self.okValue) - end - Result["or"] = function(self, other) - return self:isOk() and Result:ok(self.okValue) or other - end - function Result:orElse(other) - return self:isOk() and Result:ok(self.okValue) or other(self.errValue) - end - function Result:expect(msg) - if self:isOk() then - return self.okValue - else - error(msg) - end - end - function Result:unwrap() - return self:expect("called `Result.unwrap()` on an `Err` value: " .. tostring(self.errValue)) - end - function Result:unwrapOr(def) - local _0 - if self:isOk() then - _0 = self.okValue - else - _0 = def - end - return _0 - end - function Result:unwrapOrElse(gen) - local _0 - if self:isOk() then - _0 = self.okValue - else - _0 = gen(self.errValue) - end - return _0 - end - function Result:expectErr(msg) - if self:isErr() then - return self.errValue - else - error(msg) - end - end - function Result:unwrapErr() - return self:expectErr("called `Result.unwrapErr()` on an `Ok` value: " .. tostring(self.okValue)) - end - function Result:transpose() - return self:isOk() and self.okValue:map(function(some) - return Result:ok(some) - end) or Option:some(Result:err(self.errValue)) - end - function Result:flatten() - return self:isOk() and Result.new(self.okValue.okValue, self.okValue.errValue) or Result:err(self.errValue) - end - function Result:match(ifOk, ifErr) - local _0 - if self:isOk() then - _0 = ifOk(self.okValue) - else - _0 = ifErr(self.errValue) - end - return _0 - end - function Result:asPtr() - local _0 = (self.okValue) - if _0 == nil then - _0 = (self.errValue) - end - return _0 - end - end - local resultMeta = Result - resultMeta.__eq = function(a, b) - return b:match(function(ok) - return a:contains(ok) - end, function(err) - return a:containsErr(err) - end) - end - resultMeta.__tostring = function(result) - return result:match(function(ok) - return "Result.ok(" .. tostring(ok) .. ")" - end, function(err) - return "Result.err(" .. tostring(err) .. ")" - end) - end - return { - Result = Result, - } - )LUA"); - - auto it = std::find_if(result.errors.begin(), result.errors.end(), [](TypeError& a) { - return nullptr != get(a); - }); - if (it == result.errors.end()) - { - dumpErrors(result); - FAIL("Expected a UnificationTooComplex error"); - } -} - // Should be in TypeInfer.tables.test.cpp // It's unsound to instantiate tables containing generic methods, // since mutating properties means table properties should be invariant. diff --git a/tests/TypeInfer.singletons.test.cpp b/tests/TypeInfer.singletons.test.cpp index 2b01c29e..8d6682b8 100644 --- a/tests/TypeInfer.singletons.test.cpp +++ b/tests/TypeInfer.singletons.test.cpp @@ -164,10 +164,6 @@ TEST_CASE_FIXTURE(Fixture, "enums_using_singletons_subtyping") TEST_CASE_FIXTURE(Fixture, "tagged_unions_using_singletons") { - ScopedFastFlag sffs[] = { - {"LuauExpectedTypesOfProperties", true}, - }; - CheckResult result = check(R"( type Dog = { tag: "Dog", howls: boolean } type Cat = { tag: "Cat", meows: boolean } @@ -281,10 +277,6 @@ TEST_CASE_FIXTURE(Fixture, "table_properties_type_error_escapes") TEST_CASE_FIXTURE(Fixture, "error_detailed_tagged_union_mismatch_string") { - ScopedFastFlag sffs[] = { - {"LuauExpectedTypesOfProperties", true}, - }; - CheckResult result = check(R"( type Cat = { tag: 'cat', catfood: string } type Dog = { tag: 'dog', dogfood: string } @@ -302,10 +294,6 @@ caused by: TEST_CASE_FIXTURE(Fixture, "error_detailed_tagged_union_mismatch_bool") { - ScopedFastFlag sffs[] = { - {"LuauExpectedTypesOfProperties", true}, - }; - CheckResult result = check(R"( type Good = { success: true, result: string } type Bad = { success: false, error: string } @@ -323,10 +311,6 @@ caused by: TEST_CASE_FIXTURE(Fixture, "if_then_else_expression_singleton_options") { - ScopedFastFlag sffs[] = { - {"LuauExpectedTypesOfProperties", true}, - }; - CheckResult result = check(R"( type Cat = { tag: 'cat', catfood: string } type Dog = { tag: 'dog', dogfood: string } diff --git a/tests/TypeInfer.tables.test.cpp b/tests/TypeInfer.tables.test.cpp index 2a727bb3..5bd522a3 100644 --- a/tests/TypeInfer.tables.test.cpp +++ b/tests/TypeInfer.tables.test.cpp @@ -2122,8 +2122,6 @@ caused by: TEST_CASE_FIXTURE(Fixture, "explicitly_typed_table") { ScopedFastFlag sffs[]{ - {"LuauPropertiesGetExpectedType", true}, - {"LuauExpectedTypesOfProperties", true}, {"LuauTableSubtypingVariance2", true}, }; @@ -2143,8 +2141,6 @@ a.p = { x = 9 } TEST_CASE_FIXTURE(Fixture, "explicitly_typed_table_error") { ScopedFastFlag sffs[]{ - {"LuauPropertiesGetExpectedType", true}, - {"LuauExpectedTypesOfProperties", true}, {"LuauTableSubtypingVariance2", true}, {"LuauUnsealedTableLiteral", true}, }; @@ -2171,8 +2167,6 @@ caused by: TEST_CASE_FIXTURE(Fixture, "explicitly_typed_table_with_indexer") { ScopedFastFlag sffs[]{ - {"LuauPropertiesGetExpectedType", true}, - {"LuauExpectedTypesOfProperties", true}, {"LuauTableSubtypingVariance2", true}, }; @@ -2377,8 +2371,6 @@ TEST_CASE_FIXTURE(Fixture, "pass_a_union_of_tables_to_a_function_that_requires_a TEST_CASE_FIXTURE(Fixture, "unifying_tables_shouldnt_uaf1") { - ScopedFastFlag sff{"LuauTxnLogCheckForInvalidation", true}; - CheckResult result = check(R"( -- This example produced a UAF at one point, caused by pointers to table types becoming -- invalidated by child unifiers. (Calling log.concat can cause pointers to become invalid.) @@ -2409,8 +2401,6 @@ end TEST_CASE_FIXTURE(Fixture, "unifying_tables_shouldnt_uaf2") { - ScopedFastFlag sff{"LuauTxnLogCheckForInvalidation", true}; - CheckResult result = check(R"( -- Another example that UAFd, this time found by fuzzing. local _ diff --git a/tests/TypeInfer.tryUnify.test.cpp b/tests/TypeInfer.tryUnify.test.cpp index c21e1625..b6e93265 100644 --- a/tests/TypeInfer.tryUnify.test.cpp +++ b/tests/TypeInfer.tryUnify.test.cpp @@ -126,8 +126,6 @@ TEST_CASE_FIXTURE(TryUnifyFixture, "members_of_failed_typepack_unification_are_u TEST_CASE_FIXTURE(TryUnifyFixture, "result_of_failed_typepack_unification_is_constrained") { - ScopedFastFlag sff{"LuauErrorRecoveryType", true}; - CheckResult result = check(R"( function f(arg: number) return arg end local a diff --git a/tests/TypeVar.test.cpp b/tests/TypeVar.test.cpp index d03bb03c..e033fe22 100644 --- a/tests/TypeVar.test.cpp +++ b/tests/TypeVar.test.cpp @@ -184,8 +184,6 @@ TEST_CASE_FIXTURE(Fixture, "UnionTypeVarIterator_with_empty_union") TEST_CASE_FIXTURE(Fixture, "substitution_skip_failure") { - ScopedFastFlag sff{"LuauSealExports", true}; - TypeVar ftv11{FreeTypeVar{TypeLevel{}}}; TypePackVar tp24{TypePack{{&ftv11}}}; From 448f03218f8c42b1af06a724083252f5197a3653 Mon Sep 17 00:00:00 2001 From: Andy Friesen Date: Fri, 29 Apr 2022 09:33:30 -0700 Subject: [PATCH 233/399] Add attribution for Result.ts (#468) --- tests/RuntimeLimits.test.cpp | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/tests/RuntimeLimits.test.cpp b/tests/RuntimeLimits.test.cpp index dcbf0b61..42411de2 100644 --- a/tests/RuntimeLimits.test.cpp +++ b/tests/RuntimeLimits.test.cpp @@ -41,6 +41,11 @@ TEST_CASE_FIXTURE(LimitFixture, "bail_early_on_typescript_port_of_Result_type" * { constexpr const char* src = R"LUA( --!strict + + -- Big thanks to Dionysusnu by letting us use this code as part of our test suite! + -- https://github.com/Dionysusnu/rbxts-rust-classes + -- Licensed under the MPL 2.0: https://raw.githubusercontent.com/Dionysusnu/rbxts-rust-classes/master/LICENSE + local TS = _G[script] local lazyGet = TS.import(script, script.Parent.Parent, "util", "lazyLoad").lazyGet local unit = TS.import(script, script.Parent.Parent, "util", "Unit").unit From 9bc71c4b133aeb0e9734cfa5455f48d2829edf9d Mon Sep 17 00:00:00 2001 From: Andy Friesen Date: Tue, 3 May 2022 15:29:01 -0700 Subject: [PATCH 234/399] April 2022 recap (#470) --- .../2022-05-02-luau-recap-april-2022.md | 51 +++++++++++++++++++ 1 file changed, 51 insertions(+) create mode 100644 docs/_posts/2022-05-02-luau-recap-april-2022.md diff --git a/docs/_posts/2022-05-02-luau-recap-april-2022.md b/docs/_posts/2022-05-02-luau-recap-april-2022.md new file mode 100644 index 00000000..f5a14e3e --- /dev/null +++ b/docs/_posts/2022-05-02-luau-recap-april-2022.md @@ -0,0 +1,51 @@ +--- +layout: single +title: "Luau Recap: April 2022" +--- + +Luau is our new language that you can read more about at [https://luau-lang.org](https://luau-lang.org). + +[Cross-posted to the [Roblox Developer Forum](https://devforum.roblox.com/t/luau-recap-april-2022/).] + +It's been a bit of a quiet month. We mostly have small optimizations and bugfixes for you. + +It is now allowed to define functions on sealed tables that have string indexers. These functions will be typechecked against the indexer type. For example, the following is now valid: + +```lua +local a : {[string]: () -> number} = {} + +function b.y() return 4 end -- OK +``` + +Autocomplete will now provide string literal suggestions for singleton types. eg + +```lua +local function f(x: "a" | "b") end +f("_") -- suggest "a" and "b" +``` + +Improve error recovery in the case where we encounter a type pack variable in a place where one is not allowed. eg `type Foo = { value: A... }` + +When code does not pass enough arguments to a variadic function, the error feedback is now better. + +For example, the following script now produces a much nicer error message: +```lua +type A = { [number]: number } +type B = { [number]: string } + +local a: A = { 1, 2, 3 } + +-- ERROR: Type 'A' could not be converted into 'B' +-- caused by: +-- Property '[indexer value]' is not compatible. Type 'number' could not be converted into 'string' +local b: B = a +``` + +If the following code were to error because `Hello` was undefined, we would erroneously include the comment in the span of the error. This is now fixed. +```lua +type Foo = Hello -- some comment over here +``` + +Fix a crash that could occur when strict scripts have cyclic require() dependencies. + +Add an option to autocomplete to cause it to abort processing after a certain amount of time has elapsed. From 47a8d28aa92379a724a0a9646d176c8375a7f111 Mon Sep 17 00:00:00 2001 From: Alexander McCord <11488393+alexmccord@users.noreply.github.com> Date: Tue, 3 May 2022 16:12:59 -0700 Subject: [PATCH 235/399] Fix a typo in recap. (#472) --- docs/_posts/2022-05-02-luau-recap-april-2022.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/_posts/2022-05-02-luau-recap-april-2022.md b/docs/_posts/2022-05-02-luau-recap-april-2022.md index f5a14e3e..dd6b2c0c 100644 --- a/docs/_posts/2022-05-02-luau-recap-april-2022.md +++ b/docs/_posts/2022-05-02-luau-recap-april-2022.md @@ -14,7 +14,7 @@ It is now allowed to define functions on sealed tables that have string indexers ```lua local a : {[string]: () -> number} = {} -function b.y() return 4 end -- OK +function a.y() return 4 end -- OK ``` Autocomplete will now provide string literal suggestions for singleton types. eg From 9156b5ae6db17d937c75657674aa2470d6752b07 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?byte-chan=E2=84=A2?= Date: Wed, 4 May 2022 21:27:12 +0200 Subject: [PATCH 236/399] Fix non-C locale issues in REPL (#474) --- CLI/Repl.cpp | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/CLI/Repl.cpp b/CLI/Repl.cpp index 4cb22346..83060f5b 100644 --- a/CLI/Repl.cpp +++ b/CLI/Repl.cpp @@ -21,6 +21,8 @@ #include #endif +#include + LUAU_FASTFLAG(DebugLuauTimeTracing) enum class CliMode @@ -435,6 +437,9 @@ static void runReplImpl(lua_State* L) { ic_set_default_completer(completeRepl, L); + // Reset the locale to C + setlocale(LC_ALL, "C"); + // Make brace matching easier to see ic_style_def("ic-bracematch", "teal"); From 57016582a7f0ac07d1ce11767347ba5346fb004c Mon Sep 17 00:00:00 2001 From: phoebe <13684891+phoebethewitch@users.noreply.github.com> Date: Thu, 5 May 2022 17:37:27 -0400 Subject: [PATCH 237/399] fix feed link (#476) --- docs/_config.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/_config.yml b/docs/_config.yml index 71308686..33a85609 100644 --- a/docs/_config.yml +++ b/docs/_config.yml @@ -10,7 +10,7 @@ logo: /assets/images/luau-88.png plugins: ["jekyll-include-cache", "jekyll-feed"] include: ["_pages"] atom_feed: - path: feed.xml + path: "/feed.xml" defaults: # _docs From bb57bf96035b11a6afdf5bfb5055616aece55203 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Thu, 5 May 2022 16:52:48 -0700 Subject: [PATCH 238/399] Sync to upstream/release/526 --- Analysis/include/Luau/Frontend.h | 1 - Analysis/include/Luau/VisitTypeVar.h | 319 +++++++- Analysis/src/Autocomplete.cpp | 54 +- Analysis/src/Frontend.cpp | 34 +- Analysis/src/Normalize.cpp | 329 ++++++-- Analysis/src/Quantify.cpp | 50 +- Analysis/src/ToString.cpp | 50 +- Analysis/src/TxnLog.cpp | 34 +- Analysis/src/TypeInfer.cpp | 125 ++-- Analysis/src/Unifier.cpp | 134 +++- Ast/include/Luau/Ast.h | 2 +- Ast/src/Parser.cpp | 5 +- CLI/Repl.cpp | 5 + Compiler/include/Luau/Bytecode.h | 5 + Compiler/src/BytecodeBuilder.cpp | 10 + Compiler/src/Compiler.cpp | 338 ++++++++- Compiler/src/ConstantFolding.cpp | 53 +- Sources.cmake | 1 + VM/src/lapi.cpp | 2 +- VM/src/lbuiltins.cpp | 4 +- VM/src/lgc.h | 2 +- VM/src/ltable.cpp | 48 +- VM/src/ltm.cpp | 4 +- VM/src/ltm.h | 3 +- VM/src/lvmexecute.cpp | 196 ++++- .../test_LargeTableSum_loop_iter.lua | 17 + bench/tests/sunspider/3d-cube.lua | 30 +- bench/tests/sunspider/3d-morph.lua | 2 +- bench/tests/sunspider/3d-raytrace.lua | 44 +- bench/tests/sunspider/access-binary-trees.lua | 69 -- .../tests/sunspider/controlflow-recursive.lua | 8 +- bench/tests/sunspider/crypto-aes.lua | 148 ++-- bench/tests/sunspider/math-cordic.lua | 10 +- bench/tests/sunspider/math-partial-sums.lua | 2 +- bench/tests/sunspider/math-spectral-norm.lua | 72 -- tests/Autocomplete.test.cpp | 8 - tests/Compiler.test.cpp | 708 +++++++++++++++++- tests/Conformance.test.cpp | 12 + tests/Frontend.test.cpp | 4 - tests/Parser.test.cpp | 4 - tests/RuntimeLimits.test.cpp | 13 +- tests/TypeInfer.loops.test.cpp | 67 ++ tests/TypeInfer.modules.test.cpp | 1 - tests/TypeInfer.tables.test.cpp | 4 +- tests/TypeInfer.test.cpp | 41 + tests/TypeInfer.tryUnify.test.cpp | 4 - tests/TypeVar.test.cpp | 22 +- tests/VisitTypeVar.test.cpp | 48 ++ tests/conformance/iter.lua | 196 +++++ tests/conformance/nextvar.lua | 55 +- tools/lldb_formatters.py | 2 +- 51 files changed, 2670 insertions(+), 729 deletions(-) create mode 100644 bench/micro_tests/test_LargeTableSum_loop_iter.lua delete mode 100644 bench/tests/sunspider/access-binary-trees.lua delete mode 100644 bench/tests/sunspider/math-spectral-norm.lua create mode 100644 tests/VisitTypeVar.test.cpp create mode 100644 tests/conformance/iter.lua diff --git a/Analysis/include/Luau/Frontend.h b/Analysis/include/Luau/Frontend.h index 59125470..37e3cfdc 100644 --- a/Analysis/include/Luau/Frontend.h +++ b/Analysis/include/Luau/Frontend.h @@ -145,7 +145,6 @@ struct Frontend */ std::pair lintFragment(std::string_view source, std::optional enabledLintWarnings = {}); - CheckResult check(const SourceModule& module); // OLD. TODO KILL LintResult lint(const SourceModule& module, std::optional enabledLintWarnings = {}); bool isDirty(const ModuleName& name, bool forAutocomplete = false) const; diff --git a/Analysis/include/Luau/VisitTypeVar.h b/Analysis/include/Luau/VisitTypeVar.h index 045190ea..67fce5ed 100644 --- a/Analysis/include/Luau/VisitTypeVar.h +++ b/Analysis/include/Luau/VisitTypeVar.h @@ -1,9 +1,15 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #pragma once +#include + #include "Luau/DenseHash.h" -#include "Luau/TypeVar.h" +#include "Luau/RecursionCounter.h" #include "Luau/TypePack.h" +#include "Luau/TypeVar.h" + +LUAU_FASTFLAG(LuauUseVisitRecursionLimit) +LUAU_FASTINT(LuauVisitRecursionLimit) namespace Luau { @@ -219,24 +225,321 @@ void visit(TypePackId tp, F& f, Set& seen) } // namespace visit_detail +template +struct GenericTypeVarVisitor +{ + using Set = S; + + Set seen; + int recursionCounter = 0; + + GenericTypeVarVisitor() = default; + + explicit GenericTypeVarVisitor(Set seen) + : seen(std::move(seen)) + { + } + + virtual void cycle(TypeId) {} + virtual void cycle(TypePackId) {} + + virtual bool visit(TypeId ty) + { + return true; + } + virtual bool visit(TypeId ty, const BoundTypeVar& btv) + { + return visit(ty); + } + virtual bool visit(TypeId ty, const FreeTypeVar& ftv) + { + return visit(ty); + } + virtual bool visit(TypeId ty, const GenericTypeVar& gtv) + { + return visit(ty); + } + virtual bool visit(TypeId ty, const ErrorTypeVar& etv) + { + return visit(ty); + } + virtual bool visit(TypeId ty, const ConstrainedTypeVar& ctv) + { + return visit(ty); + } + virtual bool visit(TypeId ty, const PrimitiveTypeVar& ptv) + { + return visit(ty); + } + virtual bool visit(TypeId ty, const FunctionTypeVar& ftv) + { + return visit(ty); + } + virtual bool visit(TypeId ty, const TableTypeVar& ttv) + { + return visit(ty); + } + virtual bool visit(TypeId ty, const MetatableTypeVar& mtv) + { + return visit(ty); + } + virtual bool visit(TypeId ty, const ClassTypeVar& ctv) + { + return visit(ty); + } + virtual bool visit(TypeId ty, const AnyTypeVar& atv) + { + return visit(ty); + } + virtual bool visit(TypeId ty, const UnionTypeVar& utv) + { + return visit(ty); + } + virtual bool visit(TypeId ty, const IntersectionTypeVar& itv) + { + return visit(ty); + } + + virtual bool visit(TypePackId tp) + { + return true; + } + virtual bool visit(TypePackId tp, const BoundTypePack& btp) + { + return visit(tp); + } + virtual bool visit(TypePackId tp, const FreeTypePack& ftp) + { + return visit(tp); + } + virtual bool visit(TypePackId tp, const GenericTypePack& gtp) + { + return visit(tp); + } + virtual bool visit(TypePackId tp, const Unifiable::Error& etp) + { + return visit(tp); + } + virtual bool visit(TypePackId tp, const TypePack& pack) + { + return visit(tp); + } + virtual bool visit(TypePackId tp, const VariadicTypePack& vtp) + { + return visit(tp); + } + + void traverse(TypeId ty) + { + RecursionLimiter limiter{&recursionCounter, FInt::LuauVisitRecursionLimit, "TypeVarVisitor"}; + + if (visit_detail::hasSeen(seen, ty)) + { + cycle(ty); + return; + } + + if (auto btv = get(ty)) + { + if (visit(ty, *btv)) + traverse(btv->boundTo); + } + + else if (auto ftv = get(ty)) + visit(ty, *ftv); + + else if (auto gtv = get(ty)) + visit(ty, *gtv); + + else if (auto etv = get(ty)) + visit(ty, *etv); + + else if (auto ctv = get(ty)) + { + if (visit(ty, *ctv)) + { + for (TypeId part : ctv->parts) + traverse(part); + } + } + + else if (auto ptv = get(ty)) + visit(ty, *ptv); + + else if (auto ftv = get(ty)) + { + if (visit(ty, *ftv)) + { + traverse(ftv->argTypes); + traverse(ftv->retType); + } + } + + else if (auto ttv = get(ty)) + { + // Some visitors want to see bound tables, that's why we traverse the original type + if (visit(ty, *ttv)) + { + if (ttv->boundTo) + { + traverse(*ttv->boundTo); + } + else + { + for (auto& [_name, prop] : ttv->props) + traverse(prop.type); + + if (ttv->indexer) + { + traverse(ttv->indexer->indexType); + traverse(ttv->indexer->indexResultType); + } + } + } + } + + else if (auto mtv = get(ty)) + { + if (visit(ty, *mtv)) + { + traverse(mtv->table); + traverse(mtv->metatable); + } + } + + else if (auto ctv = get(ty)) + { + if (visit(ty, *ctv)) + { + for (const auto& [name, prop] : ctv->props) + traverse(prop.type); + + if (ctv->parent) + traverse(*ctv->parent); + + if (ctv->metatable) + traverse(*ctv->metatable); + } + } + + else if (auto atv = get(ty)) + visit(ty, *atv); + + else if (auto utv = get(ty)) + { + if (visit(ty, *utv)) + { + for (TypeId optTy : utv->options) + traverse(optTy); + } + } + + else if (auto itv = get(ty)) + { + if (visit(ty, *itv)) + { + for (TypeId partTy : itv->parts) + traverse(partTy); + } + } + + visit_detail::unsee(seen, ty); + } + + void traverse(TypePackId tp) + { + if (visit_detail::hasSeen(seen, tp)) + { + cycle(tp); + return; + } + + if (auto btv = get(tp)) + { + if (visit(tp, *btv)) + traverse(btv->boundTo); + } + + else if (auto ftv = get(tp)) + visit(tp, *ftv); + + else if (auto gtv = get(tp)) + visit(tp, *gtv); + + else if (auto etv = get(tp)) + visit(tp, *etv); + + else if (auto pack = get(tp)) + { + visit(tp, *pack); + + for (TypeId ty : pack->head) + traverse(ty); + + if (pack->tail) + traverse(*pack->tail); + } + else if (auto pack = get(tp)) + { + visit(tp, *pack); + traverse(pack->ty); + } + else + LUAU_ASSERT(!"GenericTypeVarVisitor::traverse(TypePackId) is not exhaustive!"); + + visit_detail::unsee(seen, tp); + } +}; + +/** Visit each type under a given type. Skips over cycles and keeps recursion depth under control. + * + * The same type may be visited multiple times if there are multiple distinct paths to it. If this is undesirable, use + * TypeVarOnceVisitor. + */ +struct TypeVarVisitor : GenericTypeVarVisitor> +{ +}; + +/// Visit each type under a given type. Each type will only be checked once even if there are multiple paths to it. +struct TypeVarOnceVisitor : GenericTypeVarVisitor> +{ + TypeVarOnceVisitor() + : GenericTypeVarVisitor{DenseHashSet{nullptr}} + { + } +}; + +// Clip with FFlagLuauUseVisitRecursionLimit template -void visitTypeVar(TID ty, F& f, std::unordered_set& seen) +void DEPRECATED_visitTypeVar(TID ty, F& f, std::unordered_set& seen) { visit_detail::visit(ty, f, seen); } +// Delete and inline when clipping FFlagLuauUseVisitRecursionLimit template -void visitTypeVar(TID ty, F& f) +void DEPRECATED_visitTypeVar(TID ty, F& f) { - std::unordered_set seen; - visit_detail::visit(ty, f, seen); + if (FFlag::LuauUseVisitRecursionLimit) + f.traverse(ty); + else + { + std::unordered_set seen; + visit_detail::visit(ty, f, seen); + } } +// Delete and inline when clipping FFlagLuauUseVisitRecursionLimit template -void visitTypeVarOnce(TID ty, F& f, DenseHashSet& seen) +void DEPRECATED_visitTypeVarOnce(TID ty, F& f, DenseHashSet& seen) { - seen.clear(); - visit_detail::visit(ty, f, seen); + if (FFlag::LuauUseVisitRecursionLimit) + f.traverse(ty); + else + { + seen.clear(); + visit_detail::visit(ty, f, seen); + } } } // namespace Luau diff --git a/Analysis/src/Autocomplete.cpp b/Analysis/src/Autocomplete.cpp index dec12d01..19d06cfc 100644 --- a/Analysis/src/Autocomplete.cpp +++ b/Analysis/src/Autocomplete.cpp @@ -14,7 +14,6 @@ #include LUAU_FASTFLAGVARIABLE(LuauIfElseExprFixCompletionIssue, false); -LUAU_FASTFLAGVARIABLE(LuauAutocompleteSingletonTypes, false); LUAU_FASTFLAGVARIABLE(LuauFixAutocompleteClassSecurityLevel, false); LUAU_FASTFLAG(LuauSelfCallAutocompleteFix) @@ -1341,38 +1340,21 @@ static void autocompleteExpression(const SourceModule& sourceModule, const Modul scope = scope->parent; } - if (FFlag::LuauAutocompleteSingletonTypes) - { - TypeCorrectKind correctForNil = checkTypeCorrectKind(module, typeArena, node, position, typeChecker.nilType); - TypeCorrectKind correctForTrue = checkTypeCorrectKind(module, typeArena, node, position, getSingletonTypes().trueType); - TypeCorrectKind correctForFalse = checkTypeCorrectKind(module, typeArena, node, position, getSingletonTypes().falseType); - TypeCorrectKind correctForFunction = - functionIsExpectedAt(module, node, position).value_or(false) ? TypeCorrectKind::Correct : TypeCorrectKind::None; + TypeCorrectKind correctForNil = checkTypeCorrectKind(module, typeArena, node, position, typeChecker.nilType); + TypeCorrectKind correctForTrue = checkTypeCorrectKind(module, typeArena, node, position, getSingletonTypes().trueType); + TypeCorrectKind correctForFalse = checkTypeCorrectKind(module, typeArena, node, position, getSingletonTypes().falseType); + TypeCorrectKind correctForFunction = + functionIsExpectedAt(module, node, position).value_or(false) ? TypeCorrectKind::Correct : TypeCorrectKind::None; - result["if"] = {AutocompleteEntryKind::Keyword, std::nullopt, false, false}; - result["true"] = {AutocompleteEntryKind::Keyword, typeChecker.booleanType, false, false, correctForTrue}; - result["false"] = {AutocompleteEntryKind::Keyword, typeChecker.booleanType, false, false, correctForFalse}; - result["nil"] = {AutocompleteEntryKind::Keyword, typeChecker.nilType, false, false, correctForNil}; - result["not"] = {AutocompleteEntryKind::Keyword}; - result["function"] = {AutocompleteEntryKind::Keyword, std::nullopt, false, false, correctForFunction}; + result["if"] = {AutocompleteEntryKind::Keyword, std::nullopt, false, false}; + result["true"] = {AutocompleteEntryKind::Keyword, typeChecker.booleanType, false, false, correctForTrue}; + result["false"] = {AutocompleteEntryKind::Keyword, typeChecker.booleanType, false, false, correctForFalse}; + result["nil"] = {AutocompleteEntryKind::Keyword, typeChecker.nilType, false, false, correctForNil}; + result["not"] = {AutocompleteEntryKind::Keyword}; + result["function"] = {AutocompleteEntryKind::Keyword, std::nullopt, false, false, correctForFunction}; - if (auto ty = findExpectedTypeAt(module, node, position)) - autocompleteStringSingleton(*ty, true, result); - } - else - { - TypeCorrectKind correctForNil = checkTypeCorrectKind(module, typeArena, node, position, typeChecker.nilType); - TypeCorrectKind correctForBoolean = checkTypeCorrectKind(module, typeArena, node, position, typeChecker.booleanType); - TypeCorrectKind correctForFunction = - functionIsExpectedAt(module, node, position).value_or(false) ? TypeCorrectKind::Correct : TypeCorrectKind::None; - - result["if"] = {AutocompleteEntryKind::Keyword, std::nullopt, false, false}; - result["true"] = {AutocompleteEntryKind::Keyword, typeChecker.booleanType, false, false, correctForBoolean}; - result["false"] = {AutocompleteEntryKind::Keyword, typeChecker.booleanType, false, false, correctForBoolean}; - result["nil"] = {AutocompleteEntryKind::Keyword, typeChecker.nilType, false, false, correctForNil}; - result["not"] = {AutocompleteEntryKind::Keyword}; - result["function"] = {AutocompleteEntryKind::Keyword, std::nullopt, false, false, correctForFunction}; - } + if (auto ty = findExpectedTypeAt(module, node, position)) + autocompleteStringSingleton(*ty, true, result); } } @@ -1680,11 +1662,8 @@ static AutocompleteResult autocomplete(const SourceModule& sourceModule, const M { AutocompleteEntryMap result; - if (FFlag::LuauAutocompleteSingletonTypes) - { - if (auto it = module->astExpectedTypes.find(node->asExpr())) - autocompleteStringSingleton(*it, false, result); - } + if (auto it = module->astExpectedTypes.find(node->asExpr())) + autocompleteStringSingleton(*it, false, result); if (finder.ancestry.size() >= 2) { @@ -1693,8 +1672,7 @@ static AutocompleteResult autocomplete(const SourceModule& sourceModule, const M if (auto it = module->astTypes.find(idxExpr->expr)) autocompleteProps(*module, typeArena, follow(*it), PropIndexType::Point, finder.ancestry, result); } - else if (auto binExpr = finder.ancestry.at(finder.ancestry.size() - 2)->as(); - binExpr && FFlag::LuauAutocompleteSingletonTypes) + else if (auto binExpr = finder.ancestry.at(finder.ancestry.size() - 2)->as()) { if (binExpr->op == AstExprBinary::CompareEq || binExpr->op == AstExprBinary::CompareNe) { diff --git a/Analysis/src/Frontend.cpp b/Analysis/src/Frontend.cpp index b8f7836d..56c0ac2c 100644 --- a/Analysis/src/Frontend.cpp +++ b/Analysis/src/Frontend.cpp @@ -18,7 +18,6 @@ LUAU_FASTINT(LuauTypeInferIterationLimit) LUAU_FASTINT(LuauTarjanChildLimit) -LUAU_FASTFLAG(LuauCyclicModuleTypeSurface) LUAU_FASTFLAG(LuauInferInNoCheckMode) LUAU_FASTFLAGVARIABLE(LuauKnowsTheDataModel3, false) LUAU_FASTFLAGVARIABLE(LuauSeparateTypechecks, false) @@ -433,8 +432,7 @@ CheckResult Frontend::check(const ModuleName& name, std::optional Frontend::lintFragment(std::string_view sour return {std::move(sourceModule), classifyLints(warnings, config)}; } -CheckResult Frontend::check(const SourceModule& module) -{ - LUAU_TIMETRACE_SCOPE("Frontend::check", "Frontend"); - LUAU_TIMETRACE_ARGUMENT("module", module.name.c_str()); - - const Config& config = configResolver->getConfig(module.name); - - Mode mode = module.mode.value_or(config.mode); - - double timestamp = getTimestamp(); - - ModulePtr checkedModule = typeChecker.check(module, mode); - - stats.timeCheck += getTimestamp() - timestamp; - stats.filesStrict += mode == Mode::Strict; - stats.filesNonstrict += mode == Mode::Nonstrict; - - if (checkedModule == nullptr) - throw std::runtime_error("Frontend::check produced a nullptr module for module " + module.name); - moduleResolver.modules[module.name] = checkedModule; - - return CheckResult{checkedModule->errors}; -} - LintResult Frontend::lint(const SourceModule& module, std::optional enabledLintWarnings) { LUAU_TIMETRACE_SCOPE("Frontend::lint", "Frontend"); diff --git a/Analysis/src/Normalize.cpp b/Analysis/src/Normalize.cpp index 043526ed..d8c11388 100644 --- a/Analysis/src/Normalize.cpp +++ b/Analysis/src/Normalize.cpp @@ -304,37 +304,23 @@ static bool areNormal(TypePackId tp, const std::unordered_set& seen, Inte ++iterationLimit; \ } while (false) -struct Normalize +struct Normalize final : TypeVarVisitor { + using TypeVarVisitor::Set; + + Normalize(TypeArena& arena, InternalErrorReporter& ice) + : arena(arena) + , ice(ice) + { + } + TypeArena& arena; InternalErrorReporter& ice; - // Debug data. Types being normalized are invalidated but trying to see what's going on is painful. - // To actually see the original type, read it by using the pointer of the type being normalized. - // e.g. in lldb, `e dump(originalTys[ty])`. - SeenTypes originalTys; - SeenTypePacks originalTps; - int iterationLimit = 0; bool limitExceeded = false; - template - bool operator()(TypePackId, const T&) - { - return true; - } - - template - void cycle(TID) - { - } - - bool operator()(TypeId ty, const FreeTypeVar&) - { - LUAU_ASSERT(!ty->normal); - return false; - } - + // TODO: Clip with FFlag::LuauUseVisitRecursionLimit bool operator()(TypeId ty, const BoundTypeVar& btv, std::unordered_set& seen) { // A type could be considered normal when it is in the stack, but we will eventually find out it is not normal as normalization progresses. @@ -349,27 +335,22 @@ struct Normalize return !ty->normal; } - bool operator()(TypeId ty, const PrimitiveTypeVar&) + bool operator()(TypeId ty, const FreeTypeVar& ftv) { - LUAU_ASSERT(ty->normal); - return false; + return visit(ty, ftv); } - - bool operator()(TypeId ty, const GenericTypeVar&) + bool operator()(TypeId ty, const PrimitiveTypeVar& ptv) { - if (!ty->normal) - asMutable(ty)->normal = true; - - return false; + return visit(ty, ptv); } - - bool operator()(TypeId ty, const ErrorTypeVar&) + bool operator()(TypeId ty, const GenericTypeVar& gtv) { - if (!ty->normal) - asMutable(ty)->normal = true; - return false; + return visit(ty, gtv); + } + bool operator()(TypeId ty, const ErrorTypeVar& etv) + { + return visit(ty, etv); } - bool operator()(TypeId ty, const ConstrainedTypeVar& ctvRef, std::unordered_set& seen) { CHECK_ITERATION_LIMIT(false); @@ -470,17 +451,12 @@ struct Normalize bool operator()(TypeId ty, const ClassTypeVar& ctv) { - if (!ty->normal) - asMutable(ty)->normal = true; - return false; + return visit(ty, ctv); } - - bool operator()(TypeId ty, const AnyTypeVar&) + bool operator()(TypeId ty, const AnyTypeVar& atv) { - LUAU_ASSERT(ty->normal); - return false; + return visit(ty, atv); } - bool operator()(TypeId ty, const UnionTypeVar& utvRef, std::unordered_set& seen) { CHECK_ITERATION_LIMIT(false); @@ -570,8 +546,257 @@ struct Normalize return false; } - bool operator()(TypeId ty, const LazyTypeVar&) + // TODO: Clip with FFlag::LuauUseVisitRecursionLimit + template + bool operator()(TypePackId, const T&) { + return true; + } + + // TODO: Clip with FFlag::LuauUseVisitRecursionLimit + template + void cycle(TID) + { + } + + bool visit(TypeId ty, const FreeTypeVar&) override + { + LUAU_ASSERT(!ty->normal); + return false; + } + + bool visit(TypeId ty, const BoundTypeVar& btv) override + { + // A type could be considered normal when it is in the stack, but we will eventually find out it is not normal as normalization progresses. + // So we need to avoid eagerly saying that this bound type is normal if the thing it is bound to is in the stack. + if (seen.find(asMutable(btv.boundTo)) != seen.end()) + return false; + + // It should never be the case that this TypeVar is normal, but is bound to a non-normal type, except in nontrivial cases. + LUAU_ASSERT(!ty->normal || ty->normal == btv.boundTo->normal); + + asMutable(ty)->normal = btv.boundTo->normal; + return !ty->normal; + } + + bool visit(TypeId ty, const PrimitiveTypeVar&) override + { + LUAU_ASSERT(ty->normal); + return false; + } + + bool visit(TypeId ty, const GenericTypeVar&) override + { + if (!ty->normal) + asMutable(ty)->normal = true; + + return false; + } + + bool visit(TypeId ty, const ErrorTypeVar&) override + { + if (!ty->normal) + asMutable(ty)->normal = true; + return false; + } + + bool visit(TypeId ty, const ConstrainedTypeVar& ctvRef) override + { + CHECK_ITERATION_LIMIT(false); + + ConstrainedTypeVar* ctv = const_cast(&ctvRef); + + std::vector parts = std::move(ctv->parts); + + // We might transmute, so it's not safe to rely on the builtin traversal logic of visitTypeVar + for (TypeId part : parts) + traverse(part); + + std::vector newParts = normalizeUnion(parts); + + const bool normal = areNormal(newParts, seen, ice); + + if (newParts.size() == 1) + *asMutable(ty) = BoundTypeVar{newParts[0]}; + else + *asMutable(ty) = UnionTypeVar{std::move(newParts)}; + + asMutable(ty)->normal = normal; + + return false; + } + + bool visit(TypeId ty, const FunctionTypeVar& ftv) override + { + CHECK_ITERATION_LIMIT(false); + + if (ty->normal) + return false; + + traverse(ftv.argTypes); + traverse(ftv.retType); + + asMutable(ty)->normal = areNormal(ftv.argTypes, seen, ice) && areNormal(ftv.retType, seen, ice); + + return false; + } + + bool visit(TypeId ty, const TableTypeVar& ttv) override + { + CHECK_ITERATION_LIMIT(false); + + if (ty->normal) + return false; + + bool normal = true; + + auto checkNormal = [&](TypeId t) { + // if t is on the stack, it is possible that this type is normal. + // If t is not normal and it is not on the stack, this type is definitely not normal. + if (!t->normal && seen.find(asMutable(t)) == seen.end()) + normal = false; + }; + + if (ttv.boundTo) + { + traverse(*ttv.boundTo); + asMutable(ty)->normal = (*ttv.boundTo)->normal; + return false; + } + + for (const auto& [_name, prop] : ttv.props) + { + traverse(prop.type); + checkNormal(prop.type); + } + + if (ttv.indexer) + { + traverse(ttv.indexer->indexType); + checkNormal(ttv.indexer->indexType); + traverse(ttv.indexer->indexResultType); + checkNormal(ttv.indexer->indexResultType); + } + + asMutable(ty)->normal = normal; + + return false; + } + + bool visit(TypeId ty, const MetatableTypeVar& mtv) override + { + CHECK_ITERATION_LIMIT(false); + + if (ty->normal) + return false; + + traverse(mtv.table); + traverse(mtv.metatable); + + asMutable(ty)->normal = mtv.table->normal && mtv.metatable->normal; + + return false; + } + + bool visit(TypeId ty, const ClassTypeVar& ctv) override + { + if (!ty->normal) + asMutable(ty)->normal = true; + return false; + } + + bool visit(TypeId ty, const AnyTypeVar&) override + { + LUAU_ASSERT(ty->normal); + return false; + } + + bool visit(TypeId ty, const UnionTypeVar& utvRef) override + { + CHECK_ITERATION_LIMIT(false); + + if (ty->normal) + return false; + + UnionTypeVar* utv = &const_cast(utvRef); + std::vector options = std::move(utv->options); + + // We might transmute, so it's not safe to rely on the builtin traversal logic of visitTypeVar + for (TypeId option : options) + traverse(option); + + std::vector newOptions = normalizeUnion(options); + + const bool normal = areNormal(newOptions, seen, ice); + + LUAU_ASSERT(!newOptions.empty()); + + if (newOptions.size() == 1) + *asMutable(ty) = BoundTypeVar{newOptions[0]}; + else + utv->options = std::move(newOptions); + + asMutable(ty)->normal = normal; + + return false; + } + + bool visit(TypeId ty, const IntersectionTypeVar& itvRef) override + { + CHECK_ITERATION_LIMIT(false); + + if (ty->normal) + return false; + + IntersectionTypeVar* itv = &const_cast(itvRef); + + std::vector oldParts = std::move(itv->parts); + + for (TypeId part : oldParts) + traverse(part); + + std::vector tables; + for (TypeId part : oldParts) + { + part = follow(part); + if (get(part)) + tables.push_back(part); + else + { + Replacer replacer{&arena, nullptr, nullptr}; // FIXME this is super super WEIRD + combineIntoIntersection(replacer, itv, part); + } + } + + // Don't allocate a new table if there's just one in the intersection. + if (tables.size() == 1) + itv->parts.push_back(tables[0]); + else if (!tables.empty()) + { + const TableTypeVar* first = get(tables[0]); + LUAU_ASSERT(first); + + TypeId newTable = arena.addType(TableTypeVar{first->state, first->level}); + TableTypeVar* ttv = getMutable(newTable); + for (TypeId part : tables) + { + // Intuition: If combineIntoTable() needs to clone a table, any references to 'part' are cyclic and need + // to be rewritten to point at 'newTable' in the clone. + Replacer replacer{&arena, part, newTable}; + combineIntoTable(replacer, ttv, part); + } + + itv->parts.push_back(newTable); + } + + asMutable(ty)->normal = areNormal(itv->parts, seen, ice); + + if (itv->parts.size() == 1) + { + TypeId part = itv->parts[0]; + *asMutable(ty) = BoundTypeVar{part}; + } + return false; } @@ -778,9 +1003,9 @@ std::pair normalize(TypeId ty, TypeArena& arena, InternalErrorRepo if (FFlag::DebugLuauCopyBeforeNormalizing) (void)clone(ty, arena, state); - Normalize n{arena, ice, std::move(state.seenTypes), std::move(state.seenTypePacks)}; + Normalize n{arena, ice}; std::unordered_set seen; - visitTypeVar(ty, n, seen); + DEPRECATED_visitTypeVar(ty, n, seen); return {ty, !n.limitExceeded}; } @@ -803,9 +1028,9 @@ std::pair normalize(TypePackId tp, TypeArena& arena, InternalE if (FFlag::DebugLuauCopyBeforeNormalizing) (void)clone(tp, arena, state); - Normalize n{arena, ice, std::move(state.seenTypes), std::move(state.seenTypePacks)}; + Normalize n{arena, ice}; std::unordered_set seen; - visitTypeVar(tp, n, seen); + DEPRECATED_visitTypeVar(tp, n, seen); return {tp, !n.limitExceeded}; } diff --git a/Analysis/src/Quantify.cpp b/Analysis/src/Quantify.cpp index 305f83ce..4f3e4469 100644 --- a/Analysis/src/Quantify.cpp +++ b/Analysis/src/Quantify.cpp @@ -9,7 +9,7 @@ LUAU_FASTFLAG(LuauTypecheckOptPass) namespace Luau { -struct Quantifier +struct Quantifier final : TypeVarOnceVisitor { TypeLevel level; std::vector generics; @@ -17,26 +17,17 @@ struct Quantifier bool seenGenericType = false; bool seenMutableType = false; - Quantifier(TypeLevel level) + explicit Quantifier(TypeLevel level) : level(level) { } - void cycle(TypeId) {} - void cycle(TypePackId) {} + void cycle(TypeId) override {} + void cycle(TypePackId) override {} bool operator()(TypeId ty, const FreeTypeVar& ftv) { - if (FFlag::LuauTypecheckOptPass) - seenMutableType = true; - - if (!level.subsumes(ftv.level)) - return false; - - *asMutable(ty) = GenericTypeVar{level}; - generics.push_back(ty); - - return false; + return visit(ty, ftv); } template @@ -56,8 +47,33 @@ struct Quantifier return true; } - bool operator()(TypeId ty, const TableTypeVar&) + bool operator()(TypeId ty, const TableTypeVar& ttv) { + return visit(ty, ttv); + } + + bool operator()(TypePackId tp, const FreeTypePack& ftp) + { + return visit(tp, ftp); + } + + bool visit(TypeId ty, const FreeTypeVar& ftv) override + { + if (FFlag::LuauTypecheckOptPass) + seenMutableType = true; + + if (!level.subsumes(ftv.level)) + return false; + + *asMutable(ty) = GenericTypeVar{level}; + generics.push_back(ty); + + return false; + } + + bool visit(TypeId ty, const TableTypeVar&) override + { + LUAU_ASSERT(getMutable(ty)); TableTypeVar& ttv = *getMutable(ty); if (FFlag::LuauTypecheckOptPass) @@ -93,7 +109,7 @@ struct Quantifier return true; } - bool operator()(TypePackId tp, const FreeTypePack& ftp) + bool visit(TypePackId tp, const FreeTypePack& ftp) override { if (FFlag::LuauTypecheckOptPass) seenMutableType = true; @@ -111,7 +127,7 @@ void quantify(TypeId ty, TypeLevel level) { Quantifier q{level}; DenseHashSet seen{nullptr}; - visitTypeVarOnce(ty, q, seen); + DEPRECATED_visitTypeVarOnce(ty, q, seen); FunctionTypeVar* ftv = getMutable(ty); LUAU_ASSERT(ftv); diff --git a/Analysis/src/ToString.cpp b/Analysis/src/ToString.cpp index 610842da..b5d6a550 100644 --- a/Analysis/src/ToString.cpp +++ b/Analysis/src/ToString.cpp @@ -26,7 +26,7 @@ namespace Luau namespace { -struct FindCyclicTypes +struct FindCyclicTypes final : TypeVarVisitor { FindCyclicTypes() = default; FindCyclicTypes(const FindCyclicTypes&) = delete; @@ -38,20 +38,22 @@ struct FindCyclicTypes std::set cycles; std::set cycleTPs; - void cycle(TypeId ty) + void cycle(TypeId ty) override { cycles.insert(ty); } - void cycle(TypePackId tp) + void cycle(TypePackId tp) override { cycleTPs.insert(tp); } + // TODO: Clip all the operator()s when we clip FFlagLuauUseVisitRecursionLimit + template bool operator()(TypeId ty, const T&) { - return visited.insert(ty).second; + return visit(ty); } bool operator()(TypeId ty, const TableTypeVar& ttv) = delete; @@ -64,10 +66,10 @@ struct FindCyclicTypes if (ttv.name || ttv.syntheticName) { for (TypeId itp : ttv.instantiatedTypeParams) - visitTypeVar(itp, *this, seen); + DEPRECATED_visitTypeVar(itp, *this, seen); for (TypePackId itp : ttv.instantiatedTypePackParams) - visitTypeVar(itp, *this, seen); + DEPRECATED_visitTypeVar(itp, *this, seen); return exhaustive; } @@ -82,9 +84,43 @@ struct FindCyclicTypes template bool operator()(TypePackId tp, const T&) + { + return visit(tp); + } + + bool visit(TypeId ty) override + { + return visited.insert(ty).second; + } + + bool visit(TypePackId tp) override { return visitedPacks.insert(tp).second; } + + bool visit(TypeId ty, const TableTypeVar& ttv) override + { + if (!visited.insert(ty).second) + return false; + + if (ttv.name || ttv.syntheticName) + { + for (TypeId itp : ttv.instantiatedTypeParams) + traverse(itp); + + for (TypePackId itp : ttv.instantiatedTypePackParams) + traverse(itp); + + return exhaustive; + } + + return true; + } + + bool visit(TypeId ty, const ClassTypeVar&) override + { + return false; + } }; template @@ -92,7 +128,7 @@ void findCyclicTypes(std::set& cycles, std::set& cycleTPs, T { FindCyclicTypes fct; fct.exhaustive = exhaustive; - visitTypeVar(ty, fct); + DEPRECATED_visitTypeVar(ty, fct); cycles = std::move(fct.cycles); cycleTPs = std::move(fct.cycleTPs); diff --git a/Analysis/src/TxnLog.cpp b/Analysis/src/TxnLog.cpp index a5f9d26c..1fb5a61a 100644 --- a/Analysis/src/TxnLog.cpp +++ b/Analysis/src/TxnLog.cpp @@ -7,7 +7,6 @@ #include #include -LUAU_FASTFLAGVARIABLE(LuauTxnLogPreserveOwner, false) LUAU_FASTFLAGVARIABLE(LuauJustOneCallFrameForHaveSeen, false) namespace Luau @@ -81,31 +80,20 @@ void TxnLog::concat(TxnLog rhs) void TxnLog::commit() { - if (FFlag::LuauTxnLogPreserveOwner) + for (auto& [ty, rep] : typeVarChanges) { - for (auto& [ty, rep] : typeVarChanges) - { - TypeArena* owningArena = ty->owningArena; - TypeVar* mtv = asMutable(ty); - *mtv = rep.get()->pending; - mtv->owningArena = owningArena; - } - - for (auto& [tp, rep] : typePackChanges) - { - TypeArena* owningArena = tp->owningArena; - TypePackVar* mpv = asMutable(tp); - *mpv = rep.get()->pending; - mpv->owningArena = owningArena; - } + TypeArena* owningArena = ty->owningArena; + TypeVar* mtv = asMutable(ty); + *mtv = rep.get()->pending; + mtv->owningArena = owningArena; } - else - { - for (auto& [ty, rep] : typeVarChanges) - *asMutable(ty) = rep.get()->pending; - for (auto& [tp, rep] : typePackChanges) - *asMutable(tp) = rep.get()->pending; + for (auto& [tp, rep] : typePackChanges) + { + TypeArena* owningArena = tp->owningArena; + TypePackVar* mpv = asMutable(tp); + *mpv = rep.get()->pending; + mpv->owningArena = owningArena; } clear(); diff --git a/Analysis/src/TypeInfer.cpp b/Analysis/src/TypeInfer.cpp index ba91ae1e..4466ede2 100644 --- a/Analysis/src/TypeInfer.cpp +++ b/Analysis/src/TypeInfer.cpp @@ -26,11 +26,11 @@ LUAU_FASTINTVARIABLE(LuauTypeInferRecursionLimit, 165) LUAU_FASTINTVARIABLE(LuauTypeInferIterationLimit, 20000) LUAU_FASTINTVARIABLE(LuauTypeInferTypePackLoopLimit, 5000) LUAU_FASTINTVARIABLE(LuauCheckRecursionLimit, 300) +LUAU_FASTFLAGVARIABLE(LuauUseVisitRecursionLimit, false) +LUAU_FASTINTVARIABLE(LuauVisitRecursionLimit, 500) LUAU_FASTFLAG(LuauKnowsTheDataModel3) LUAU_FASTFLAG(LuauSeparateTypechecks) LUAU_FASTFLAG(LuauAutocompleteDynamicLimits) -LUAU_FASTFLAG(LuauAutocompleteSingletonTypes) -LUAU_FASTFLAGVARIABLE(LuauCyclicModuleTypeSurface, false) LUAU_FASTFLAGVARIABLE(LuauDoNotRelyOnNextBinding, false) LUAU_FASTFLAGVARIABLE(LuauEqConstraint, false) LUAU_FASTFLAGVARIABLE(LuauWeakEqConstraint, false) // Eventually removed as false. @@ -40,6 +40,7 @@ LUAU_FASTFLAGVARIABLE(LuauInferStatFunction, false) LUAU_FASTFLAGVARIABLE(LuauInstantiateFollows, false) LUAU_FASTFLAGVARIABLE(LuauSelfCallAutocompleteFix, false) LUAU_FASTFLAGVARIABLE(LuauDiscriminableUnions2, false) +LUAU_FASTFLAGVARIABLE(LuauReduceUnionRecursion, false) LUAU_FASTFLAGVARIABLE(LuauOnlyMutateInstantiatedTables, false) LUAU_FASTFLAGVARIABLE(LuauStatFunctionSimplify4, false) LUAU_FASTFLAGVARIABLE(LuauTypecheckOptPass, false) @@ -57,6 +58,7 @@ LUAU_FASTFLAGVARIABLE(LuauTableUseCounterInstead, false) LUAU_FASTFLAGVARIABLE(LuauReturnTypeInferenceInNonstrict, false) LUAU_FASTFLAGVARIABLE(LuauRecursionLimitException, false); LUAU_FASTFLAG(LuauLosslessClone) +LUAU_FASTFLAGVARIABLE(LuauTypecheckIter, false); namespace Luau { @@ -1159,6 +1161,47 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatForIn& forin) iterTy = follow(instantiate(scope, checkExpr(scope, *firstValue).type, firstValue->location)); } + if (FFlag::LuauTypecheckIter) + { + if (std::optional iterMM = findMetatableEntry(iterTy, "__iter", firstValue->location)) + { + // if __iter metamethod is present, it will be called and the results are going to be called as if they are functions + // TODO: this needs to typecheck all returned values by __iter as if they were for loop arguments + // the structure of the function makes it difficult to do this especially since we don't have actual expressions, only types + for (TypeId var : varTypes) + unify(anyType, var, forin.location); + + return check(loopScope, *forin.body); + } + else if (const TableTypeVar* iterTable = get(iterTy)) + { + // TODO: note that this doesn't cleanly handle iteration over mixed tables and tables without an indexer + // this behavior is more or less consistent with what we do for pairs(), but really both are pretty wrong and need revisiting + if (iterTable->indexer) + { + if (varTypes.size() > 0) + unify(iterTable->indexer->indexType, varTypes[0], forin.location); + + if (varTypes.size() > 1) + unify(iterTable->indexer->indexResultType, varTypes[1], forin.location); + + for (size_t i = 2; i < varTypes.size(); ++i) + unify(nilType, varTypes[i], forin.location); + } + else + { + TypeId varTy = errorRecoveryType(loopScope); + + for (TypeId var : varTypes) + unify(varTy, var, forin.location); + + reportError(firstValue->location, GenericError{"Cannot iterate over a table without indexer"}); + } + + return check(loopScope, *forin.body); + } + } + const FunctionTypeVar* iterFunc = get(iterTy); if (!iterFunc) { @@ -2026,15 +2069,29 @@ std::vector TypeChecker::reduceUnion(const std::vector& types) if (const UnionTypeVar* utv = get(t)) { - std::vector r = reduceUnion(utv->options); - for (TypeId ty : r) + if (FFlag::LuauReduceUnionRecursion) { - ty = follow(ty); - if (get(ty) || get(ty)) - return {ty}; + for (TypeId ty : utv) + { + if (get(ty) || get(ty)) + return {ty}; - if (std::find(result.begin(), result.end(), ty) == result.end()) - result.push_back(ty); + if (result.end() == std::find(result.begin(), result.end(), ty)) + result.push_back(ty); + } + } + else + { + std::vector r = reduceUnion(utv->options); + for (TypeId ty : r) + { + ty = follow(ty); + if (get(ty) || get(ty)) + return {ty}; + + if (std::find(result.begin(), result.end(), ty) == result.end()) + result.push_back(ty); + } } } else if (std::find(result.begin(), result.end(), t) == result.end()) @@ -4372,17 +4429,12 @@ TypeId TypeChecker::checkRequire(const ScopePtr& scope, const ModuleInfo& module } // Types of requires that transitively refer to current module have to be replaced with 'any' - std::string humanReadableName; + std::string humanReadableName = resolver->getHumanReadableModuleName(moduleInfo.name); - if (FFlag::LuauCyclicModuleTypeSurface) + for (const auto& [location, path] : requireCycles) { - humanReadableName = resolver->getHumanReadableModuleName(moduleInfo.name); - - for (const auto& [location, path] : requireCycles) - { - if (!path.empty() && path.front() == humanReadableName) - return anyType; - } + if (!path.empty() && path.front() == humanReadableName) + return anyType; } ModulePtr module = resolver->getModule(moduleInfo.name); @@ -4392,32 +4444,14 @@ TypeId TypeChecker::checkRequire(const ScopePtr& scope, const ModuleInfo& module // either the file does not exist or there's a cycle. If there's a cycle // we will already have reported the error. if (!resolver->moduleExists(moduleInfo.name) && !moduleInfo.optional) - { - if (FFlag::LuauCyclicModuleTypeSurface) - { - reportError(TypeError{location, UnknownRequire{humanReadableName}}); - } - else - { - std::string reportedModulePath = resolver->getHumanReadableModuleName(moduleInfo.name); - reportError(TypeError{location, UnknownRequire{reportedModulePath}}); - } - } + reportError(TypeError{location, UnknownRequire{humanReadableName}}); return errorRecoveryType(scope); } if (module->type != SourceCode::Module) { - if (FFlag::LuauCyclicModuleTypeSurface) - { - reportError(location, IllegalRequire{humanReadableName, "Module is not a ModuleScript. It cannot be required."}); - } - else - { - std::string humanReadableName = resolver->getHumanReadableModuleName(moduleInfo.name); - reportError(location, IllegalRequire{humanReadableName, "Module is not a ModuleScript. It cannot be required."}); - } + reportError(location, IllegalRequire{humanReadableName, "Module is not a ModuleScript. It cannot be required."}); return errorRecoveryType(scope); } @@ -4429,15 +4463,7 @@ TypeId TypeChecker::checkRequire(const ScopePtr& scope, const ModuleInfo& module std::optional moduleType = first(modulePack); if (!moduleType) { - if (FFlag::LuauCyclicModuleTypeSurface) - { - reportError(location, IllegalRequire{humanReadableName, "Module does not return exactly 1 value. It cannot be required."}); - } - else - { - std::string humanReadableName = resolver->getHumanReadableModuleName(moduleInfo.name); - reportError(location, IllegalRequire{humanReadableName, "Module does not return exactly 1 value. It cannot be required."}); - } + reportError(location, IllegalRequire{humanReadableName, "Module does not return exactly 1 value. It cannot be required."}); return errorRecoveryType(scope); } @@ -4947,10 +4973,7 @@ TypeId TypeChecker::freshType(TypeLevel level) TypeId TypeChecker::singletonType(bool value) { - if (FFlag::LuauAutocompleteSingletonTypes) - return value ? getSingletonTypes().trueType : getSingletonTypes().falseType; - - return currentModule->internalTypes.addType(TypeVar(SingletonTypeVar(BooleanSingleton{value}))); + return value ? getSingletonTypes().trueType : getSingletonTypes().falseType; } TypeId TypeChecker::singletonType(std::string value) diff --git a/Analysis/src/Unifier.cpp b/Analysis/src/Unifier.cpp index 334806ce..f5c1dde9 100644 --- a/Analysis/src/Unifier.cpp +++ b/Analysis/src/Unifier.cpp @@ -22,7 +22,7 @@ LUAU_FASTFLAG(LuauLowerBoundsCalculation); LUAU_FASTFLAG(LuauErrorRecoveryType); LUAU_FASTFLAGVARIABLE(LuauSubtypingAddOptPropsToUnsealedTables, false) LUAU_FASTFLAGVARIABLE(LuauWidenIfSupertypeIsFree2, false) -LUAU_FASTFLAGVARIABLE(LuauDifferentOrderOfUnificationDoesntMatter, false) +LUAU_FASTFLAGVARIABLE(LuauDifferentOrderOfUnificationDoesntMatter2, false) LUAU_FASTFLAGVARIABLE(LuauTxnLogRefreshFunctionPointers, false) LUAU_FASTFLAG(LuauAnyInIsOptionalIsOptional) LUAU_FASTFLAG(LuauTypecheckOptPass) @@ -30,7 +30,7 @@ LUAU_FASTFLAG(LuauTypecheckOptPass) namespace Luau { -struct PromoteTypeLevels +struct PromoteTypeLevels final : TypeVarOnceVisitor { TxnLog& log; const TypeArena* typeArena = nullptr; @@ -53,13 +53,34 @@ struct PromoteTypeLevels } } + // TODO cycle and operator() need to be clipped when FFlagLuauUseVisitRecursionLimit is clipped template void cycle(TID) { } - template bool operator()(TID ty, const T&) + { + return visit(ty); + } + bool operator()(TypeId ty, const FreeTypeVar& ftv) + { + return visit(ty, ftv); + } + bool operator()(TypeId ty, const FunctionTypeVar& ftv) + { + return visit(ty, ftv); + } + bool operator()(TypeId ty, const TableTypeVar& ttv) + { + return visit(ty, ttv); + } + bool operator()(TypePackId tp, const FreeTypePack& ftp) + { + return visit(tp, ftp); + } + + bool visit(TypeId ty) override { // Type levels of types from other modules are already global, so we don't need to promote anything inside if (ty->owningArena != typeArena) @@ -68,7 +89,16 @@ struct PromoteTypeLevels return true; } - bool operator()(TypeId ty, const FreeTypeVar&) + bool visit(TypePackId tp) override + { + // Type levels of types from other modules are already global, so we don't need to promote anything inside + if (tp->owningArena != typeArena) + return false; + + return true; + } + + bool visit(TypeId ty, const FreeTypeVar&) override { // Surprise, it's actually a BoundTypeVar that hasn't been committed yet. // Calling getMutable on this will trigger an assertion. @@ -79,7 +109,7 @@ struct PromoteTypeLevels return true; } - bool operator()(TypeId ty, const FunctionTypeVar&) + bool visit(TypeId ty, const FunctionTypeVar&) override { // Type levels of types from other modules are already global, so we don't need to promote anything inside if (ty->owningArena != typeArena) @@ -89,7 +119,7 @@ struct PromoteTypeLevels return true; } - bool operator()(TypeId ty, const TableTypeVar& ttv) + bool visit(TypeId ty, const TableTypeVar& ttv) override { // Type levels of types from other modules are already global, so we don't need to promote anything inside if (ty->owningArena != typeArena) @@ -102,7 +132,7 @@ struct PromoteTypeLevels return true; } - bool operator()(TypePackId tp, const FreeTypePack&) + bool visit(TypePackId tp, const FreeTypePack&) override { // Surprise, it's actually a BoundTypePack that hasn't been committed yet. // Calling getMutable on this will trigger an assertion. @@ -122,7 +152,7 @@ static void promoteTypeLevels(TxnLog& log, const TypeArena* typeArena, TypeLevel PromoteTypeLevels ptl{log, typeArena, minLevel}; DenseHashSet seen{nullptr}; - visitTypeVarOnce(ty, ptl, seen); + DEPRECATED_visitTypeVarOnce(ty, ptl, seen); } void promoteTypeLevels(TxnLog& log, const TypeArena* typeArena, TypeLevel minLevel, TypePackId tp) @@ -133,10 +163,10 @@ void promoteTypeLevels(TxnLog& log, const TypeArena* typeArena, TypeLevel minLev PromoteTypeLevels ptl{log, typeArena, minLevel}; DenseHashSet seen{nullptr}; - visitTypeVarOnce(tp, ptl, seen); + DEPRECATED_visitTypeVarOnce(tp, ptl, seen); } -struct SkipCacheForType +struct SkipCacheForType final : TypeVarOnceVisitor { SkipCacheForType(const DenseHashMap& skipCacheForType, const TypeArena* typeArena) : skipCacheForType(skipCacheForType) @@ -144,28 +174,68 @@ struct SkipCacheForType { } - void cycle(TypeId) {} - void cycle(TypePackId) {} + // TODO cycle() and operator() can be clipped with FFlagLuauUseVisitRecursionLimit + void cycle(TypeId) override {} + void cycle(TypePackId) override {} bool operator()(TypeId ty, const FreeTypeVar& ftv) { - result = true; - return false; + return visit(ty, ftv); } - bool operator()(TypeId ty, const BoundTypeVar& btv) { - result = true; - return false; + return visit(ty, btv); + } + bool operator()(TypeId ty, const GenericTypeVar& gtv) + { + return visit(ty, gtv); + } + bool operator()(TypeId ty, const TableTypeVar& ttv) + { + return visit(ty, ttv); + } + bool operator()(TypePackId tp, const FreeTypePack& ftp) + { + return visit(tp, ftp); + } + bool operator()(TypePackId tp, const BoundTypePack& ftp) + { + return visit(tp, ftp); + } + bool operator()(TypePackId tp, const GenericTypePack& ftp) + { + return visit(tp, ftp); + } + template + bool operator()(TypeId ty, const T& t) + { + return visit(ty); + } + template + bool operator()(TypePackId tp, const T&) + { + return visit(tp); } - bool operator()(TypeId ty, const GenericTypeVar& btv) + bool visit(TypeId, const FreeTypeVar&) override { result = true; return false; } - bool operator()(TypeId ty, const TableTypeVar&) + bool visit(TypeId, const BoundTypeVar&) override + { + result = true; + return false; + } + + bool visit(TypeId, const GenericTypeVar&) override + { + result = true; + return false; + } + + bool visit(TypeId ty, const TableTypeVar&) override { // Types from other modules don't contain mutable elements and are ok to cache if (ty->owningArena != typeArena) @@ -188,8 +258,7 @@ struct SkipCacheForType return true; } - template - bool operator()(TypeId ty, const T& t) + bool visit(TypeId ty) override { // Types from other modules don't contain mutable elements and are ok to cache if (ty->owningArena != typeArena) @@ -206,8 +275,7 @@ struct SkipCacheForType return true; } - template - bool operator()(TypePackId tp, const T&) + bool visit(TypePackId tp) override { // Types from other modules don't contain mutable elements and are ok to cache if (tp->owningArena != typeArena) @@ -216,19 +284,19 @@ struct SkipCacheForType return true; } - bool operator()(TypePackId tp, const FreeTypePack& ftp) + bool visit(TypePackId tp, const FreeTypePack&) override { result = true; return false; } - bool operator()(TypePackId tp, const BoundTypePack& ftp) + bool visit(TypePackId tp, const BoundTypePack&) override { result = true; return false; } - bool operator()(TypePackId tp, const GenericTypePack& ftp) + bool visit(TypePackId tp, const GenericTypePack&) override { result = true; return false; @@ -578,7 +646,7 @@ void Unifier::tryUnifyUnionWithType(TypeId subTy, const UnionTypeVar* uv, TypeId failed = true; } - if (FFlag::LuauDifferentOrderOfUnificationDoesntMatter) + if (FFlag::LuauDifferentOrderOfUnificationDoesntMatter2) { } else @@ -593,7 +661,7 @@ void Unifier::tryUnifyUnionWithType(TypeId subTy, const UnionTypeVar* uv, TypeId } // even if A | B <: T fails, we want to bind some options of T with A | B iff A | B was a subtype of that option. - if (FFlag::LuauDifferentOrderOfUnificationDoesntMatter) + if (FFlag::LuauDifferentOrderOfUnificationDoesntMatter2) { auto tryBind = [this, subTy](TypeId superOption) { superOption = log.follow(superOption); @@ -603,6 +671,14 @@ void Unifier::tryUnifyUnionWithType(TypeId subTy, const UnionTypeVar* uv, TypeId if (!log.is(superOption) && (!ttv || ttv->state != TableState::Free)) return; + // If superOption is already present in subTy, do nothing. Nothing new has been learned, but the subtype + // test is successful. + if (auto subUnion = get(subTy)) + { + if (end(subUnion) != std::find(begin(subUnion), end(subUnion), superOption)) + return; + } + // Since we have already checked if S <: T, checking it again will not queue up the type for replacement. // So we'll have to do it ourselves. We assume they unified cleanly if they are still in the seen set. if (log.haveSeen(subTy, superOption)) @@ -822,7 +898,7 @@ bool Unifier::canCacheResult(TypeId subTy, TypeId superTy) auto skipCacheFor = [this](TypeId ty) { SkipCacheForType visitor{sharedState.skipCacheForType, types}; - visitTypeVarOnce(ty, visitor, sharedState.seenAny); + DEPRECATED_visitTypeVarOnce(ty, visitor, sharedState.seenAny); sharedState.skipCacheForType[ty] = visitor.result; diff --git a/Ast/include/Luau/Ast.h b/Ast/include/Luau/Ast.h index 31cd01cc..6f39e3fd 100644 --- a/Ast/include/Luau/Ast.h +++ b/Ast/include/Luau/Ast.h @@ -313,7 +313,7 @@ template struct AstArray { T* data; - std::size_t size; + size_t size; const T* begin() const { diff --git a/Ast/src/Parser.cpp b/Ast/src/Parser.cpp index 31ff3f77..91f5cd25 100644 --- a/Ast/src/Parser.cpp +++ b/Ast/src/Parser.cpp @@ -10,7 +10,6 @@ // See docs/SyntaxChanges.md for an explanation. LUAU_FASTINTVARIABLE(LuauRecursionLimit, 1000) LUAU_FASTINTVARIABLE(LuauParseErrorLimit, 100) -LUAU_FASTFLAGVARIABLE(LuauParseRecoverUnexpectedPack, false) LUAU_FASTFLAGVARIABLE(LuauParseLocationIgnoreCommentSkipInCapture, false) namespace Luau @@ -1430,7 +1429,7 @@ AstType* Parser::parseTypeAnnotation(TempVector& parts, const Location parts.push_back(parseSimpleTypeAnnotation(/* allowPack= */ false).type); isIntersection = true; } - else if (FFlag::LuauParseRecoverUnexpectedPack && c == Lexeme::Dot3) + else if (c == Lexeme::Dot3) { report(lexer.current().location, "Unexpected '...' after type annotation"); nextLexeme(); @@ -1551,7 +1550,7 @@ AstTypeOrPack Parser::parseSimpleTypeAnnotation(bool allowPack) prefix = name.name; name = parseIndexName("field name", pointPosition); } - else if (FFlag::LuauParseRecoverUnexpectedPack && lexer.current().type == Lexeme::Dot3) + else if (lexer.current().type == Lexeme::Dot3) { report(lexer.current().location, "Unexpected '...' after type name; type pack is not allowed in this context"); nextLexeme(); diff --git a/CLI/Repl.cpp b/CLI/Repl.cpp index 4cb22346..83060f5b 100644 --- a/CLI/Repl.cpp +++ b/CLI/Repl.cpp @@ -21,6 +21,8 @@ #include #endif +#include + LUAU_FASTFLAG(DebugLuauTimeTracing) enum class CliMode @@ -435,6 +437,9 @@ static void runReplImpl(lua_State* L) { ic_set_default_completer(completeRepl, L); + // Reset the locale to C + setlocale(LC_ALL, "C"); + // Make brace matching easier to see ic_style_def("ic-bracematch", "teal"); diff --git a/Compiler/include/Luau/Bytecode.h b/Compiler/include/Luau/Bytecode.h index c6e5a03b..f71d893c 100644 --- a/Compiler/include/Luau/Bytecode.h +++ b/Compiler/include/Luau/Bytecode.h @@ -353,6 +353,11 @@ enum LuauOpcode // AUX: constant index LOP_FASTCALL2K, + // FORGPREP: prepare loop variables for a generic for loop, jump to the loop backedge unconditionally + // A: target register; generic for loops assume a register layout [generator, state, index, variables...] + // D: jump offset (-32768..32767) + LOP_FORGPREP, + // Enum entry for number of opcodes, not a valid opcode by itself! LOP__COUNT }; diff --git a/Compiler/src/BytecodeBuilder.cpp b/Compiler/src/BytecodeBuilder.cpp index 871a1484..fb70392e 100644 --- a/Compiler/src/BytecodeBuilder.cpp +++ b/Compiler/src/BytecodeBuilder.cpp @@ -96,6 +96,7 @@ inline bool isJumpD(LuauOpcode op) case LOP_JUMPIFNOTLT: case LOP_FORNPREP: case LOP_FORNLOOP: + case LOP_FORGPREP: case LOP_FORGLOOP: case LOP_FORGPREP_INEXT: case LOP_FORGLOOP_INEXT: @@ -1269,6 +1270,11 @@ void BytecodeBuilder::validate() const VJUMP(LUAU_INSN_D(insn)); break; + case LOP_FORGPREP: + VREG(LUAU_INSN_A(insn) + 2 + 1); // forg loop protocol: A, A+1, A+2 are used for iteration protocol; A+3, ... are loop variables + VJUMP(LUAU_INSN_D(insn)); + break; + case LOP_FORGLOOP: VREG( LUAU_INSN_A(insn) + 2 + insns[i + 1]); // forg loop protocol: A, A+1, A+2 are used for iteration protocol; A+3, ... are loop variables @@ -1622,6 +1628,10 @@ const uint32_t* BytecodeBuilder::dumpInstruction(const uint32_t* code, std::stri formatAppend(result, "FORNLOOP R%d %+d\n", LUAU_INSN_A(insn), LUAU_INSN_D(insn)); break; + case LOP_FORGPREP: + formatAppend(result, "FORGPREP R%d %+d\n", LUAU_INSN_A(insn), LUAU_INSN_D(insn)); + break; + case LOP_FORGLOOP: formatAppend(result, "FORGLOOP R%d %+d %d\n", LUAU_INSN_A(insn), LUAU_INSN_D(insn), *code++); break; diff --git a/Compiler/src/Compiler.cpp b/Compiler/src/Compiler.cpp index 0f17ee02..4fe26222 100644 --- a/Compiler/src/Compiler.cpp +++ b/Compiler/src/Compiler.cpp @@ -17,9 +17,19 @@ #include #include +LUAU_FASTFLAGVARIABLE(LuauCompileSupportInlining, false) + +LUAU_FASTFLAGVARIABLE(LuauCompileIter, false) +LUAU_FASTFLAGVARIABLE(LuauCompileIterNoReserve, false) +LUAU_FASTFLAGVARIABLE(LuauCompileIterNoPairs, false) + LUAU_FASTINTVARIABLE(LuauCompileLoopUnrollThreshold, 25) LUAU_FASTINTVARIABLE(LuauCompileLoopUnrollThresholdMaxBoost, 300) +LUAU_FASTINTVARIABLE(LuauCompileInlineThreshold, 25) +LUAU_FASTINTVARIABLE(LuauCompileInlineThresholdMaxBoost, 300) +LUAU_FASTINTVARIABLE(LuauCompileInlineDepth, 5) + namespace Luau { @@ -147,6 +157,52 @@ struct Compiler } } + AstExprFunction* getFunctionExpr(AstExpr* node) + { + if (AstExprLocal* le = node->as()) + { + Variable* lv = variables.find(le->local); + + if (!lv || lv->written || !lv->init) + return nullptr; + + return getFunctionExpr(lv->init); + } + else if (AstExprGroup* ge = node->as()) + return getFunctionExpr(ge->expr); + else + return node->as(); + } + + bool canInlineFunctionBody(AstStat* stat) + { + struct CanInlineVisitor : AstVisitor + { + bool result = true; + + bool visit(AstExpr* node) override + { + // nested functions may capture function arguments, and our upval handling doesn't handle elided variables (constant) + // TODO: we could remove this case if we changed function compilation to create temporary locals for constant upvalues + // TODO: additionally we would need to change upvalue handling in compileExprFunction to handle upvalue->local migration + result = result && !node->is(); + return result; + } + + bool visit(AstStat* node) override + { + // loops may need to be unrolled which can result in cost amplification + result = result && !node->is(); + return result; + } + }; + + CanInlineVisitor canInline; + stat->visit(&canInline); + + return canInline.result; + } + uint32_t compileFunction(AstExprFunction* func) { LUAU_TIMETRACE_SCOPE("Compiler::compileFunction", "Compiler"); @@ -214,13 +270,21 @@ struct Compiler bytecode.endFunction(uint8_t(stackSize), uint8_t(upvals.size())); - stackSize = 0; - Function& f = functions[func]; f.id = fid; f.upvals = upvals; + // record information for inlining + if (FFlag::LuauCompileSupportInlining && options.optimizationLevel >= 2 && !func->vararg && canInlineFunctionBody(func->body) && + !getfenvUsed && !setfenvUsed) + { + f.canInline = true; + f.stackSize = stackSize; + f.costModel = modelCost(func->body, func->args.data, func->args.size); + } + upvals.clear(); // note: instead of std::move above, we copy & clear to preserve capacity for future pushes + stackSize = 0; return fid; } @@ -390,12 +454,183 @@ struct Compiler } } + bool tryCompileInlinedCall(AstExprCall* expr, AstExprFunction* func, uint8_t target, uint8_t targetCount, bool multRet, int thresholdBase, + int thresholdMaxBoost, int depthLimit) + { + Function* fi = functions.find(func); + LUAU_ASSERT(fi); + + // make sure we have enough register space + if (regTop > 128 || fi->stackSize > 32) + { + bytecode.addDebugRemark("inlining failed: high register pressure"); + return false; + } + + // we should ideally aggregate the costs during recursive inlining, but for now simply limit the depth + if (int(inlineFrames.size()) >= depthLimit) + { + bytecode.addDebugRemark("inlining failed: too many inlined frames"); + return false; + } + + // compiling recursive inlining is difficult because we share constant/variable state but need to bind variables to different registers + for (InlineFrame& frame : inlineFrames) + if (frame.func == func) + { + bytecode.addDebugRemark("inlining failed: can't inline recursive calls"); + return false; + } + + // TODO: we can compile multret functions if all returns of the function are multret as well + if (multRet) + { + bytecode.addDebugRemark("inlining failed: can't convert fixed returns to multret"); + return false; + } + + // TODO: we can compile functions with mismatching arity at call site but it's more annoying + if (func->args.size != expr->args.size) + { + bytecode.addDebugRemark("inlining failed: argument count mismatch (expected %d, got %d)", int(func->args.size), int(expr->args.size)); + return false; + } + + // we use a dynamic cost threshold that's based on the fixed limit boosted by the cost advantage we gain due to inlining + bool varc[8] = {}; + for (size_t i = 0; i < expr->args.size && i < 8; ++i) + varc[i] = isConstant(expr->args.data[i]); + + int inlinedCost = computeCost(fi->costModel, varc, std::min(int(expr->args.size), 8)); + int baselineCost = computeCost(fi->costModel, nullptr, 0) + 3; + int inlineProfit = (inlinedCost == 0) ? thresholdMaxBoost : std::min(thresholdMaxBoost, 100 * baselineCost / inlinedCost); + + int threshold = thresholdBase * inlineProfit / 100; + + if (inlinedCost > threshold) + { + bytecode.addDebugRemark("inlining failed: too expensive (cost %d, profit %.2fx)", inlinedCost, double(inlineProfit) / 100); + return false; + } + + bytecode.addDebugRemark( + "inlining succeeded (cost %d, profit %.2fx, depth %d)", inlinedCost, double(inlineProfit) / 100, int(inlineFrames.size())); + + compileInlinedCall(expr, func, target, targetCount); + return true; + } + + void compileInlinedCall(AstExprCall* expr, AstExprFunction* func, uint8_t target, uint8_t targetCount) + { + RegScope rs(this); + + size_t oldLocals = localStack.size(); + + // note that we push the frame early; this is needed to block recursive inline attempts + inlineFrames.push_back({func, target, targetCount}); + + // evaluate all arguments; note that we don't emit code for constant arguments (relying on constant folding) + for (size_t i = 0; i < func->args.size; ++i) + { + AstLocal* var = func->args.data[i]; + AstExpr* arg = expr->args.data[i]; + + if (Variable* vv = variables.find(var); vv && vv->written) + { + // if the argument is mutated, we need to allocate a fresh register even if it's a constant + uint8_t reg = allocReg(arg, 1); + compileExprTemp(arg, reg); + pushLocal(var, reg); + } + else if (const Constant* cv = constants.find(arg); cv && cv->type != Constant::Type_Unknown) + { + // since the argument is not mutated, we can simply fold the value into the expressions that need it + locstants[var] = *cv; + } + else + { + AstExprLocal* le = arg->as(); + Variable* lv = le ? variables.find(le->local) : nullptr; + + // if the argument is a local that isn't mutated, we will simply reuse the existing register + if (isExprLocalReg(arg) && (!lv || !lv->written)) + { + uint8_t reg = getLocal(le->local); + pushLocal(var, reg); + } + else + { + uint8_t reg = allocReg(arg, 1); + compileExprTemp(arg, reg); + pushLocal(var, reg); + } + } + } + + // fold constant values updated above into expressions in the function body + foldConstants(constants, variables, locstants, func->body); + + bool usedFallthrough = false; + + for (size_t i = 0; i < func->body->body.size; ++i) + { + AstStat* stat = func->body->body.data[i]; + + if (AstStatReturn* ret = stat->as()) + { + // Optimization: use fallthrough when compiling return at the end of the function to avoid an extra JUMP + compileInlineReturn(ret, /* fallthrough= */ true); + // TODO: This doesn't work when return is part of control flow; ideally we would track the state somehow and generalize this + usedFallthrough = true; + break; + } + else + compileStat(stat); + } + + // for the fallthrough path we need to ensure we clear out target registers + if (!usedFallthrough && !allPathsEndWithReturn(func->body)) + { + for (size_t i = 0; i < targetCount; ++i) + bytecode.emitABC(LOP_LOADNIL, uint8_t(target + i), 0, 0); + } + + popLocals(oldLocals); + + size_t returnLabel = bytecode.emitLabel(); + patchJumps(expr, inlineFrames.back().returnJumps, returnLabel); + + inlineFrames.pop_back(); + + // clean up constant state for future inlining attempts + for (size_t i = 0; i < func->args.size; ++i) + if (Constant* var = locstants.find(func->args.data[i])) + var->type = Constant::Type_Unknown; + + foldConstants(constants, variables, locstants, func->body); + } + void compileExprCall(AstExprCall* expr, uint8_t target, uint8_t targetCount, bool targetTop = false, bool multRet = false) { LUAU_ASSERT(!targetTop || unsigned(target + targetCount) == regTop); setDebugLine(expr); // normally compileExpr sets up line info, but compileExprCall can be called directly + // try inlining the function + if (options.optimizationLevel >= 2 && !expr->self) + { + AstExprFunction* func = getFunctionExpr(expr->func); + Function* fi = func ? functions.find(func) : nullptr; + + if (fi && fi->canInline && + tryCompileInlinedCall(expr, func, target, targetCount, multRet, FInt::LuauCompileInlineThreshold, + FInt::LuauCompileInlineThresholdMaxBoost, FInt::LuauCompileInlineDepth)) + return; + + if (fi && !fi->canInline) + bytecode.addDebugRemark("inlining failed: complex constructs in function body"); + } + RegScope rs(this); unsigned int regCount = std::max(unsigned(1 + expr->self + expr->args.size), unsigned(targetCount)); @@ -760,7 +995,7 @@ struct Compiler { const Constant* c = constants.find(node); - if (!c) + if (!c || c->type == Constant::Type_Unknown) return -1; int cid = -1; @@ -1395,27 +1630,29 @@ struct Compiler { RegScope rs(this); + // note: cv may be invalidated by compileExpr* so we stop using it before calling compile recursively const Constant* cv = constants.find(expr->index); if (cv && cv->type == Constant::Type_Number && cv->valueNumber >= 1 && cv->valueNumber <= 256 && double(int(cv->valueNumber)) == cv->valueNumber) { - uint8_t rt = compileExprAuto(expr->expr, rs); uint8_t i = uint8_t(int(cv->valueNumber) - 1); + uint8_t rt = compileExprAuto(expr->expr, rs); + setDebugLine(expr->index); bytecode.emitABC(LOP_GETTABLEN, target, rt, i); } else if (cv && cv->type == Constant::Type_String) { - uint8_t rt = compileExprAuto(expr->expr, rs); - BytecodeBuilder::StringRef iname = sref(cv->getString()); int32_t cid = bytecode.addConstantString(iname); if (cid < 0) CompileError::raise(expr->location, "Exceeded constant limit; simplify the code to compile"); + uint8_t rt = compileExprAuto(expr->expr, rs); + setDebugLine(expr->index); bytecode.emitABC(LOP_GETTABLEKS, target, rt, uint8_t(BytecodeBuilder::getStringHash(iname))); @@ -1561,8 +1798,9 @@ struct Compiler } else if (AstExprLocal* expr = node->as()) { - if (expr->upvalue) + if (FFlag::LuauCompileSupportInlining ? !isExprLocalReg(expr) : expr->upvalue) { + LUAU_ASSERT(expr->upvalue); uint8_t uid = getUpval(expr->local); bytecode.emitABC(LOP_GETUPVAL, target, uid, 0); @@ -1650,12 +1888,12 @@ struct Compiler // initializes target..target+targetCount-1 range using expressions from the list // if list has fewer expressions, and last expression is a call, we assume the call returns the rest of the values // if list has fewer expressions, and last expression isn't a call, we fill the rest with nil - // assumes target register range can be clobbered and is at the top of the register space - void compileExprListTop(const AstArray& list, uint8_t target, uint8_t targetCount) + // assumes target register range can be clobbered and is at the top of the register space if targetTop = true + void compileExprListTemp(const AstArray& list, uint8_t target, uint8_t targetCount, bool targetTop) { // we assume that target range is at the top of the register space and can be clobbered // this is what allows us to compile the last call expression - if it's a call - using targetTop=true - LUAU_ASSERT(unsigned(target + targetCount) == regTop); + LUAU_ASSERT(!targetTop || unsigned(target + targetCount) == regTop); if (list.size == targetCount) { @@ -1683,7 +1921,7 @@ struct Compiler if (AstExprCall* expr = last->as()) { - compileExprCall(expr, uint8_t(target + list.size - 1), uint8_t(targetCount - (list.size - 1)), /* targetTop= */ true); + compileExprCall(expr, uint8_t(target + list.size - 1), uint8_t(targetCount - (list.size - 1)), targetTop); } else if (AstExprVarargs* expr = last->as()) { @@ -1765,8 +2003,10 @@ struct Compiler if (AstExprLocal* expr = node->as()) { - if (expr->upvalue) + if (FFlag::LuauCompileSupportInlining ? !isExprLocalReg(expr) : expr->upvalue) { + LUAU_ASSERT(expr->upvalue); + LValue result = {LValue::Kind_Upvalue}; result.upval = getUpval(expr->local); result.location = node->location; @@ -1873,7 +2113,7 @@ struct Compiler bool isExprLocalReg(AstExpr* expr) { AstExprLocal* le = expr->as(); - if (!le || le->upvalue) + if (!le || (!FFlag::LuauCompileSupportInlining && le->upvalue)) return false; Local* l = locals.find(le->local); @@ -2080,6 +2320,23 @@ struct Compiler loops.pop_back(); } + void compileInlineReturn(AstStatReturn* stat, bool fallthrough) + { + setDebugLine(stat); // normally compileStat sets up line info, but compileInlineReturn can be called directly + + InlineFrame frame = inlineFrames.back(); + + compileExprListTemp(stat->list, frame.target, frame.targetCount, /* targetTop= */ false); + + if (!fallthrough) + { + size_t jumpLabel = bytecode.emitLabel(); + bytecode.emitAD(LOP_JUMP, 0, 0); + + inlineFrames.back().returnJumps.push_back(jumpLabel); + } + } + void compileStatReturn(AstStatReturn* stat) { RegScope rs(this); @@ -2138,7 +2395,7 @@ struct Compiler // note: allocReg in this case allocates into parent block register - note that we don't have RegScope here uint8_t vars = allocReg(stat, unsigned(stat->vars.size)); - compileExprListTop(stat->values, vars, uint8_t(stat->vars.size)); + compileExprListTemp(stat->values, vars, uint8_t(stat->vars.size), /* targetTop= */ true); for (size_t i = 0; i < stat->vars.size; ++i) pushLocal(stat->vars.data[i], uint8_t(vars + i)); @@ -2168,6 +2425,7 @@ struct Compiler bool visit(AstExpr* node) override { // functions may capture loop variable, and our upval handling doesn't handle elided variables (constant) + // TODO: we could remove this case if we changed function compilation to create temporary locals for constant upvalues result = result && !node->is(); return result; } @@ -2251,6 +2509,11 @@ struct Compiler compileStat(stat->body); } + // clean up fold state in case we need to recompile - normally we compile the loop body once, but due to inlining we may need to do it again + locstants[var].type = Constant::Type_Unknown; + + foldConstants(constants, variables, locstants, stat); + return true; } @@ -2336,12 +2599,17 @@ struct Compiler uint8_t regs = allocReg(stat, 3); // this puts initial values of (generator, state, index) into the loop registers - compileExprListTop(stat->values, regs, 3); + compileExprListTemp(stat->values, regs, 3, /* targetTop= */ true); - // for the general case, we will execute a CALL for every iteration that needs to evaluate "variables... = generator(state, index)" - // this requires at least extra 3 stack slots after index - // note that these stack slots overlap with the variables so we only need to reserve them to make sure stack frame is large enough - reserveReg(stat, 3); + // we don't need this because the extra stack space is just for calling the function with a loop protocol which is similar to calling + // metamethods - it should fit into the extra stack reservation + if (!FFlag::LuauCompileIterNoReserve) + { + // for the general case, we will execute a CALL for every iteration that needs to evaluate "variables... = generator(state, index)" + // this requires at least extra 3 stack slots after index + // note that these stack slots overlap with the variables so we only need to reserve them to make sure stack frame is large enough + reserveReg(stat, 3); + } // note that we reserve at least 2 variables; this allows our fast path to assume that we need 2 variables instead of 1 or 2 uint8_t vars = allocReg(stat, std::max(unsigned(stat->vars.size), 2u)); @@ -2350,7 +2618,7 @@ struct Compiler // Optimization: when we iterate through pairs/ipairs, we generate special bytecode that optimizes the traversal using internal iteration // index These instructions dynamically check if generator is equal to next/inext and bail out They assume that the generator produces 2 // variables, which is why we allocate at least 2 above (see vars assignment) - LuauOpcode skipOp = LOP_JUMP; + LuauOpcode skipOp = FFlag::LuauCompileIter ? LOP_FORGPREP : LOP_JUMP; LuauOpcode loopOp = LOP_FORGLOOP; if (options.optimizationLevel >= 1 && stat->vars.size <= 2) @@ -2367,7 +2635,7 @@ struct Compiler else if (builtin.isGlobal("pairs")) // for .. in pairs(t) { skipOp = LOP_FORGPREP_NEXT; - loopOp = LOP_FORGLOOP_NEXT; + loopOp = FFlag::LuauCompileIterNoPairs ? LOP_FORGLOOP : LOP_FORGLOOP_NEXT; } } else if (stat->values.size == 2) @@ -2377,7 +2645,7 @@ struct Compiler if (builtin.isGlobal("next")) // for .. in next,t { skipOp = LOP_FORGPREP_NEXT; - loopOp = LOP_FORGLOOP_NEXT; + loopOp = FFlag::LuauCompileIterNoPairs ? LOP_FORGLOOP : LOP_FORGLOOP_NEXT; } } } @@ -2514,10 +2782,10 @@ struct Compiler // compute values into temporaries uint8_t regs = allocReg(stat, unsigned(stat->vars.size)); - compileExprListTop(stat->values, regs, uint8_t(stat->vars.size)); + compileExprListTemp(stat->values, regs, uint8_t(stat->vars.size), /* targetTop= */ true); - // assign variables that have associated values; note that if we have fewer values than variables, we'll assign nil because compileExprListTop - // will generate nils + // assign variables that have associated values; note that if we have fewer values than variables, we'll assign nil because + // compileExprListTemp will generate nils for (size_t i = 0; i < stat->vars.size; ++i) { setDebugLine(stat->vars.data[i]); @@ -2675,7 +2943,10 @@ struct Compiler } else if (AstStatReturn* stat = node->as()) { - compileStatReturn(stat); + if (options.optimizationLevel >= 2 && !inlineFrames.empty()) + compileInlineReturn(stat, /* fallthrough= */ false); + else + compileStatReturn(stat); } else if (AstStatExpr* stat = node->as()) { @@ -3069,6 +3340,10 @@ struct Compiler { uint32_t id; std::vector upvals; + + uint64_t costModel = 0; + unsigned int stackSize = 0; + bool canInline = false; }; struct Local @@ -3098,6 +3373,16 @@ struct Compiler AstExpr* untilCondition; }; + struct InlineFrame + { + AstExprFunction* func; + + uint8_t target; + uint8_t targetCount; + + std::vector returnJumps; + }; + BytecodeBuilder& bytecode; CompileOptions options; @@ -3120,6 +3405,7 @@ struct Compiler std::vector upvals; std::vector loopJumps; std::vector loops; + std::vector inlineFrames; }; void compileOrThrow(BytecodeBuilder& bytecode, AstStatBlock* root, const AstNameTable& names, const CompileOptions& options) diff --git a/Compiler/src/ConstantFolding.cpp b/Compiler/src/ConstantFolding.cpp index 7ad91d4b..52ece73e 100644 --- a/Compiler/src/ConstantFolding.cpp +++ b/Compiler/src/ConstantFolding.cpp @@ -3,6 +3,8 @@ #include +LUAU_FASTFLAG(LuauCompileSupportInlining) + namespace Luau { namespace Compile @@ -314,12 +316,35 @@ struct ConstantVisitor : AstVisitor LUAU_ASSERT(!"Unknown expression type"); } - if (result.type != Constant::Type_Unknown) - constants[node] = result; + recordConstant(constants, node, result); return result; } + template + void recordConstant(DenseHashMap& map, T key, const Constant& value) + { + if (value.type != Constant::Type_Unknown) + map[key] = value; + else if (!FFlag::LuauCompileSupportInlining) + ; + else if (Constant* old = map.find(key)) + old->type = Constant::Type_Unknown; + } + + void recordValue(AstLocal* local, const Constant& value) + { + // note: we rely on trackValues to have been run before us + Variable* v = variables.find(local); + LUAU_ASSERT(v); + + if (!v->written) + { + v->constant = (value.type != Constant::Type_Unknown); + recordConstant(locals, local, value); + } + } + bool visit(AstExpr* node) override { // note: we short-circuit the visitor traversal through any expression trees by returning false @@ -336,18 +361,7 @@ struct ConstantVisitor : AstVisitor { Constant arg = analyze(node->values.data[i]); - if (arg.type != Constant::Type_Unknown) - { - // note: we rely on trackValues to have been run before us - Variable* v = variables.find(node->vars.data[i]); - LUAU_ASSERT(v); - - if (!v->written) - { - locals[node->vars.data[i]] = arg; - v->constant = true; - } - } + recordValue(node->vars.data[i], arg); } if (node->vars.size > node->values.size) @@ -361,15 +375,8 @@ struct ConstantVisitor : AstVisitor { for (size_t i = node->values.size; i < node->vars.size; ++i) { - // note: we rely on trackValues to have been run before us - Variable* v = variables.find(node->vars.data[i]); - LUAU_ASSERT(v); - - if (!v->written) - { - locals[node->vars.data[i]].type = Constant::Type_Nil; - v->constant = true; - } + Constant nil = {Constant::Type_Nil}; + recordValue(node->vars.data[i], nil); } } } diff --git a/Sources.cmake b/Sources.cmake index f9263b24..d2430cc9 100644 --- a/Sources.cmake +++ b/Sources.cmake @@ -264,6 +264,7 @@ if(TARGET Luau.UnitTest) tests/TypePack.test.cpp tests/TypeVar.test.cpp tests/Variant.test.cpp + tests/VisitTypeVar.test.cpp tests/main.cpp) endif() diff --git a/VM/src/lapi.cpp b/VM/src/lapi.cpp index 1f3b0943..f8baefaf 100644 --- a/VM/src/lapi.cpp +++ b/VM/src/lapi.cpp @@ -1270,7 +1270,7 @@ const char* lua_setupvalue(lua_State* L, int funcindex, int n) L->top--; setobj(L, val, L->top); luaC_barrier(L, clvalue(fi), L->top); - luaC_upvalbarrier(L, NULL, val); + luaC_upvalbarrier(L, cast_to(UpVal*, NULL), val); } return name; } diff --git a/VM/src/lbuiltins.cpp b/VM/src/lbuiltins.cpp index 718d387d..60149199 100644 --- a/VM/src/lbuiltins.cpp +++ b/VM/src/lbuiltins.cpp @@ -15,6 +15,8 @@ #include #endif +LUAU_FASTFLAGVARIABLE(LuauFixBuiltinsStackLimit, false) + // luauF functions implement FASTCALL instruction that performs a direct execution of some builtin functions from the VM // The rule of thumb is that FASTCALL functions can not call user code, yield, fail, or reallocate stack. // If types of the arguments mismatch, luauF_* needs to return -1 and the execution will fall back to the usual call path @@ -1003,7 +1005,7 @@ static int luauF_tunpack(lua_State* L, StkId res, TValue* arg0, int nresults, St else if (nparams == 3 && ttisnumber(args) && ttisnumber(args + 1) && nvalue(args) == 1.0) n = int(nvalue(args + 1)); - if (n >= 0 && n <= t->sizearray && cast_int(L->stack_last - res) >= n) + if (n >= 0 && n <= t->sizearray && cast_int(L->stack_last - res) >= n && (!FFlag::LuauFixBuiltinsStackLimit || n + nparams <= LUAI_MAXCSTACK)) { TValue* array = t->array; for (int i = 0; i < n; ++i) diff --git a/VM/src/lgc.h b/VM/src/lgc.h index 08d1ff5d..797284a2 100644 --- a/VM/src/lgc.h +++ b/VM/src/lgc.h @@ -120,7 +120,7 @@ #define luaC_upvalbarrier(L, uv, tv) \ { \ - if (iscollectable(tv) && iswhite(gcvalue(tv)) && (!(uv) || ((UpVal*)uv)->v != &((UpVal*)uv)->u.value)) \ + if (iscollectable(tv) && iswhite(gcvalue(tv)) && (!(uv) || (uv)->v != &(uv)->u.value)) \ luaC_barrierupval(L, gcvalue(tv)); \ } diff --git a/VM/src/ltable.cpp b/VM/src/ltable.cpp index 3dc3bd1b..8251b51c 100644 --- a/VM/src/ltable.cpp +++ b/VM/src/ltable.cpp @@ -33,8 +33,6 @@ #include -LUAU_FASTFLAGVARIABLE(LuauTableNewBoundary2, false) - // max size of both array and hash part is 2^MAXBITS #define MAXBITS 26 #define MAXSIZE (1 << MAXBITS) @@ -431,7 +429,6 @@ static void resize(lua_State* L, Table* t, int nasize, int nhsize) static int adjustasize(Table* t, int size, const TValue* ek) { - LUAU_ASSERT(FFlag::LuauTableNewBoundary2); bool tbound = t->node != dummynode || size < t->sizearray; int ekindex = ek && ttisnumber(ek) ? arrayindex(nvalue(ek)) : -1; /* move the array size up until the boundary is guaranteed to be inside the array part */ @@ -443,7 +440,7 @@ static int adjustasize(Table* t, int size, const TValue* ek) void luaH_resizearray(lua_State* L, Table* t, int nasize) { int nsize = (t->node == dummynode) ? 0 : sizenode(t); - int asize = FFlag::LuauTableNewBoundary2 ? adjustasize(t, nasize, NULL) : nasize; + int asize = adjustasize(t, nasize, NULL); resize(L, t, asize, nsize); } @@ -468,8 +465,7 @@ static void rehash(lua_State* L, Table* t, const TValue* ek) int na = computesizes(nums, &nasize); int nh = totaluse - na; /* enforce the boundary invariant; for performance, only do hash lookups if we must */ - if (FFlag::LuauTableNewBoundary2) - nasize = adjustasize(t, nasize, ek); + nasize = adjustasize(t, nasize, ek); /* resize the table to new computed sizes */ resize(L, t, nasize, nh); } @@ -531,7 +527,7 @@ static LuaNode* getfreepos(Table* t) static TValue* newkey(lua_State* L, Table* t, const TValue* key) { /* enforce boundary invariant */ - if (FFlag::LuauTableNewBoundary2 && ttisnumber(key) && nvalue(key) == t->sizearray + 1) + if (ttisnumber(key) && nvalue(key) == t->sizearray + 1) { rehash(L, t, key); /* grow table */ @@ -713,37 +709,6 @@ TValue* luaH_setstr(lua_State* L, Table* t, TString* key) } } -static LUAU_NOINLINE int unbound_search(Table* t, unsigned int j) -{ - LUAU_ASSERT(!FFlag::LuauTableNewBoundary2); - unsigned int i = j; /* i is zero or a present index */ - j++; - /* find `i' and `j' such that i is present and j is not */ - while (!ttisnil(luaH_getnum(t, j))) - { - i = j; - j *= 2; - if (j > cast_to(unsigned int, INT_MAX)) - { /* overflow? */ - /* table was built with bad purposes: resort to linear search */ - i = 1; - while (!ttisnil(luaH_getnum(t, i))) - i++; - return i - 1; - } - } - /* now do a binary search between them */ - while (j - i > 1) - { - unsigned int m = (i + j) / 2; - if (ttisnil(luaH_getnum(t, m))) - j = m; - else - i = m; - } - return i; -} - static int updateaboundary(Table* t, int boundary) { if (boundary < t->sizearray && ttisnil(&t->array[boundary - 1])) @@ -800,17 +765,12 @@ int luaH_getn(Table* t) maybesetaboundary(t, boundary); return boundary; } - else if (FFlag::LuauTableNewBoundary2) + else { /* validate boundary invariant */ LUAU_ASSERT(t->node == dummynode || ttisnil(luaH_getnum(t, j + 1))); return j; } - /* else must find a boundary in hash part */ - else if (t->node == dummynode) /* hash part is empty? */ - return j; /* that is easy... */ - else - return unbound_search(t, j); } Table* luaH_clone(lua_State* L, Table* tt) diff --git a/VM/src/ltm.cpp b/VM/src/ltm.cpp index 106efb2b..9b99506b 100644 --- a/VM/src/ltm.cpp +++ b/VM/src/ltm.cpp @@ -37,6 +37,8 @@ const char* const luaT_eventname[] = { "__newindex", "__mode", "__namecall", + "__call", + "__iter", "__eq", @@ -54,13 +56,13 @@ const char* const luaT_eventname[] = { "__lt", "__le", "__concat", - "__call", "__type", }; // clang-format on static_assert(sizeof(luaT_typenames) / sizeof(luaT_typenames[0]) == LUA_T_COUNT, "luaT_typenames size mismatch"); static_assert(sizeof(luaT_eventname) / sizeof(luaT_eventname[0]) == TM_N, "luaT_eventname size mismatch"); +static_assert(TM_EQ < 8, "fasttm optimization stores a bitfield with metamethods in a byte"); void luaT_init(lua_State* L) { diff --git a/VM/src/ltm.h b/VM/src/ltm.h index 0e4e915d..e1b95c21 100644 --- a/VM/src/ltm.h +++ b/VM/src/ltm.h @@ -16,6 +16,8 @@ typedef enum TM_NEWINDEX, TM_MODE, TM_NAMECALL, + TM_CALL, + TM_ITER, TM_EQ, /* last tag method with `fast' access */ @@ -33,7 +35,6 @@ typedef enum TM_LT, TM_LE, TM_CONCAT, - TM_CALL, TM_TYPE, TM_N /* number of elements in the enum */ diff --git a/VM/src/lvmexecute.cpp b/VM/src/lvmexecute.cpp index 39c60eac..3c7c276a 100644 --- a/VM/src/lvmexecute.cpp +++ b/VM/src/lvmexecute.cpp @@ -16,7 +16,10 @@ #include -LUAU_FASTFLAG(LuauTableNewBoundary2) +LUAU_FASTFLAGVARIABLE(LuauIter, false) +LUAU_DYNAMIC_FASTFLAGVARIABLE(LuauIterCallTelemetry, false) + +void (*lua_iter_call_telemetry)(lua_State* L); // Disable c99-designator to avoid the warning in CGOTO dispatch table #ifdef __clang__ @@ -110,7 +113,7 @@ LUAU_FASTFLAG(LuauTableNewBoundary2) VM_DISPATCH_OP(LOP_FORGLOOP_NEXT), VM_DISPATCH_OP(LOP_GETVARARGS), VM_DISPATCH_OP(LOP_DUPCLOSURE), VM_DISPATCH_OP(LOP_PREPVARARGS), \ VM_DISPATCH_OP(LOP_LOADKX), VM_DISPATCH_OP(LOP_JUMPX), VM_DISPATCH_OP(LOP_FASTCALL), VM_DISPATCH_OP(LOP_COVERAGE), \ VM_DISPATCH_OP(LOP_CAPTURE), VM_DISPATCH_OP(LOP_JUMPIFEQK), VM_DISPATCH_OP(LOP_JUMPIFNOTEQK), VM_DISPATCH_OP(LOP_FASTCALL1), \ - VM_DISPATCH_OP(LOP_FASTCALL2), VM_DISPATCH_OP(LOP_FASTCALL2K), + VM_DISPATCH_OP(LOP_FASTCALL2), VM_DISPATCH_OP(LOP_FASTCALL2K), VM_DISPATCH_OP(LOP_FORGPREP), #if defined(__GNUC__) || defined(__clang__) #define VM_USE_CGOTO 1 @@ -150,8 +153,20 @@ LUAU_NOINLINE static void luau_prepareFORN(lua_State* L, StkId plimit, StkId pst LUAU_NOINLINE static bool luau_loopFORG(lua_State* L, int a, int c) { + // note: it's safe to push arguments past top for complicated reasons (see top of the file) StkId ra = &L->base[a]; - LUAU_ASSERT(ra + 6 <= L->top); + LUAU_ASSERT(ra + 3 <= L->top); + + if (DFFlag::LuauIterCallTelemetry) + { + /* TODO: we might be able to stop supporting this depending on whether it's used in practice */ + void (*telemetrycb)(lua_State* L) = lua_iter_call_telemetry; + + if (telemetrycb && ttistable(ra) && fasttm(L, hvalue(ra)->metatable, TM_CALL)) + telemetrycb(L); + if (telemetrycb && ttisuserdata(ra) && fasttm(L, uvalue(ra)->metatable, TM_CALL)) + telemetrycb(L); + } setobjs2s(L, ra + 3 + 2, ra + 2); setobjs2s(L, ra + 3 + 1, ra + 1); @@ -2204,20 +2219,149 @@ static void luau_execute(lua_State* L) } } + VM_CASE(LOP_FORGPREP) + { + Instruction insn = *pc++; + StkId ra = VM_REG(LUAU_INSN_A(insn)); + + if (ttisfunction(ra)) + { + /* will be called during FORGLOOP */ + } + else if (FFlag::LuauIter) + { + Table* mt = ttistable(ra) ? hvalue(ra)->metatable : ttisuserdata(ra) ? uvalue(ra)->metatable : cast_to(Table*, NULL); + + if (const TValue* fn = fasttm(L, mt, TM_ITER)) + { + setobj2s(L, ra + 1, ra); + setobj2s(L, ra, fn); + + L->top = ra + 2; /* func + self arg */ + LUAU_ASSERT(L->top <= L->stack_last); + + VM_PROTECT(luaD_call(L, ra, 3)); + L->top = L->ci->top; + } + else if (fasttm(L, mt, TM_CALL)) + { + /* table or userdata with __call, will be called during FORGLOOP */ + /* TODO: we might be able to stop supporting this depending on whether it's used in practice */ + } + else if (ttistable(ra)) + { + /* set up registers for builtin iteration */ + setobj2s(L, ra + 1, ra); + setpvalue(ra + 2, reinterpret_cast(uintptr_t(0))); + setnilvalue(ra); + } + else + { + VM_PROTECT(luaG_typeerror(L, ra, "iterate over")); + } + } + + pc += LUAU_INSN_D(insn); + LUAU_ASSERT(unsigned(pc - cl->l.p->code) < unsigned(cl->l.p->sizecode)); + VM_NEXT(); + } + VM_CASE(LOP_FORGLOOP) { VM_INTERRUPT(); Instruction insn = *pc++; + StkId ra = VM_REG(LUAU_INSN_A(insn)); uint32_t aux = *pc; - // note: this is a slow generic path, fast-path is FORGLOOP_INEXT/NEXT - bool stop; - VM_PROTECT(stop = luau_loopFORG(L, LUAU_INSN_A(insn), aux)); + if (!FFlag::LuauIter) + { + bool stop; + VM_PROTECT(stop = luau_loopFORG(L, LUAU_INSN_A(insn), aux)); - // note that we need to increment pc by 1 to exit the loop since we need to skip over aux - pc += stop ? 1 : LUAU_INSN_D(insn); - LUAU_ASSERT(unsigned(pc - cl->l.p->code) < unsigned(cl->l.p->sizecode)); - VM_NEXT(); + // note that we need to increment pc by 1 to exit the loop since we need to skip over aux + pc += stop ? 1 : LUAU_INSN_D(insn); + LUAU_ASSERT(unsigned(pc - cl->l.p->code) < unsigned(cl->l.p->sizecode)); + VM_NEXT(); + } + + // fast-path: builtin table iteration + if (ttisnil(ra) && ttistable(ra + 1) && ttislightuserdata(ra + 2)) + { + Table* h = hvalue(ra + 1); + int index = int(reinterpret_cast(pvalue(ra + 2))); + + int sizearray = h->sizearray; + int sizenode = 1 << h->lsizenode; + + // clear extra variables since we might have more than two + if (LUAU_UNLIKELY(aux > 2)) + for (int i = 2; i < int(aux); ++i) + setnilvalue(ra + 3 + i); + + // first we advance index through the array portion + while (unsigned(index) < unsigned(sizearray)) + { + if (!ttisnil(&h->array[index])) + { + setpvalue(ra + 2, reinterpret_cast(uintptr_t(index + 1))); + setnvalue(ra + 3, double(index + 1)); + setobj2s(L, ra + 4, &h->array[index]); + + pc += LUAU_INSN_D(insn); + LUAU_ASSERT(unsigned(pc - cl->l.p->code) < unsigned(cl->l.p->sizecode)); + VM_NEXT(); + } + + index++; + } + + // then we advance index through the hash portion + while (unsigned(index - sizearray) < unsigned(sizenode)) + { + LuaNode* n = &h->node[index - sizearray]; + + if (!ttisnil(gval(n))) + { + setpvalue(ra + 2, reinterpret_cast(uintptr_t(index + 1))); + getnodekey(L, ra + 3, n); + setobj2s(L, ra + 4, gval(n)); + + pc += LUAU_INSN_D(insn); + LUAU_ASSERT(unsigned(pc - cl->l.p->code) < unsigned(cl->l.p->sizecode)); + VM_NEXT(); + } + + index++; + } + + // fallthrough to exit + pc++; + VM_NEXT(); + } + else + { + // note: it's safe to push arguments past top for complicated reasons (see top of the file) + setobjs2s(L, ra + 3 + 2, ra + 2); + setobjs2s(L, ra + 3 + 1, ra + 1); + setobjs2s(L, ra + 3, ra); + + L->top = ra + 3 + 3; /* func + 2 args (state and index) */ + LUAU_ASSERT(L->top <= L->stack_last); + + VM_PROTECT(luaD_call(L, ra + 3, aux)); + L->top = L->ci->top; + + // recompute ra since stack might have been reallocated + ra = VM_REG(LUAU_INSN_A(insn)); + + // copy first variable back into the iteration index + setobjs2s(L, ra + 2, ra + 3); + + // note that we need to increment pc by 1 to exit the loop since we need to skip over aux + pc += ttisnil(ra + 3) ? 1 : LUAU_INSN_D(insn); + LUAU_ASSERT(unsigned(pc - cl->l.p->code) < unsigned(cl->l.p->sizecode)); + VM_NEXT(); + } } VM_CASE(LOP_FORGPREP_INEXT) @@ -2228,8 +2372,15 @@ static void luau_execute(lua_State* L) // fast-path: ipairs/inext if (cl->env->safeenv && ttistable(ra + 1) && ttisnumber(ra + 2) && nvalue(ra + 2) == 0.0) { + if (FFlag::LuauIter) + setnilvalue(ra); + setpvalue(ra + 2, reinterpret_cast(uintptr_t(0))); } + else if (FFlag::LuauIter && !ttisfunction(ra)) + { + VM_PROTECT(luaG_typeerror(L, ra, "iterate over")); + } pc += LUAU_INSN_D(insn); LUAU_ASSERT(unsigned(pc - cl->l.p->code) < unsigned(cl->l.p->sizecode)); @@ -2268,23 +2419,9 @@ static void luau_execute(lua_State* L) VM_NEXT(); } } - else if (FFlag::LuauTableNewBoundary2 || (h->lsizenode == 0 && ttisnil(gval(h->node)))) - { - // fallthrough to exit - VM_NEXT(); - } else { - // the table has a hash part; index + 1 may appear in it in which case we need to iterate through the hash portion as well - const TValue* val = luaH_getnum(h, index + 1); - - setpvalue(ra + 2, reinterpret_cast(uintptr_t(index + 1))); - setnvalue(ra + 3, double(index + 1)); - setobj2s(L, ra + 4, val); - - // note that nil elements inside the array terminate the traversal - pc += ttisnil(ra + 4) ? 0 : LUAU_INSN_D(insn); - LUAU_ASSERT(unsigned(pc - cl->l.p->code) < unsigned(cl->l.p->sizecode)); + // fallthrough to exit VM_NEXT(); } } @@ -2308,8 +2445,15 @@ static void luau_execute(lua_State* L) // fast-path: pairs/next if (cl->env->safeenv && ttistable(ra + 1) && ttisnil(ra + 2)) { + if (FFlag::LuauIter) + setnilvalue(ra); + setpvalue(ra + 2, reinterpret_cast(uintptr_t(0))); } + else if (FFlag::LuauIter && !ttisfunction(ra)) + { + VM_PROTECT(luaG_typeerror(L, ra, "iterate over")); + } pc += LUAU_INSN_D(insn); LUAU_ASSERT(unsigned(pc - cl->l.p->code) < unsigned(cl->l.p->sizecode)); @@ -2704,7 +2848,7 @@ static void luau_execute(lua_State* L) { VM_PROTECT_PC(); - int n = f(L, ra, arg, nresults, nullptr, nparams); + int n = f(L, ra, arg, nresults, NULL, nparams); if (n >= 0) { diff --git a/bench/micro_tests/test_LargeTableSum_loop_iter.lua b/bench/micro_tests/test_LargeTableSum_loop_iter.lua new file mode 100644 index 00000000..057420f6 --- /dev/null +++ b/bench/micro_tests/test_LargeTableSum_loop_iter.lua @@ -0,0 +1,17 @@ +local bench = script and require(script.Parent.bench_support) or require("bench_support") + +function test() + + local t = {} + + for i=1,1000000 do t[i] = i end + + local ts0 = os.clock() + local sum = 0 + for k,v in t do sum = sum + v end + local ts1 = os.clock() + + return ts1-ts0 +end + +bench.runCode(test, "LargeTableSum: for k,v in {}") diff --git a/bench/tests/sunspider/3d-cube.lua b/bench/tests/sunspider/3d-cube.lua index 5d162ab9..77fa0854 100644 --- a/bench/tests/sunspider/3d-cube.lua +++ b/bench/tests/sunspider/3d-cube.lua @@ -25,7 +25,7 @@ local DisplArea = {} DisplArea.Width = 300; DisplArea.Height = 300; -function DrawLine(From, To) +local function DrawLine(From, To) local x1 = From.V[1]; local x2 = To.V[1]; local y1 = From.V[2]; @@ -81,7 +81,7 @@ function DrawLine(From, To) Q.LastPx = NumPix; end -function CalcCross(V0, V1) +local function CalcCross(V0, V1) local Cross = {}; Cross[1] = V0[2]*V1[3] - V0[3]*V1[2]; Cross[2] = V0[3]*V1[1] - V0[1]*V1[3]; @@ -89,7 +89,7 @@ function CalcCross(V0, V1) return Cross; end -function CalcNormal(V0, V1, V2) +local function CalcNormal(V0, V1, V2) local A = {}; local B = {}; for i = 1,3 do A[i] = V0[i] - V1[i]; @@ -102,14 +102,14 @@ function CalcNormal(V0, V1, V2) return A; end -function CreateP(X,Y,Z) +local function CreateP(X,Y,Z) local result = {} result.V = {X,Y,Z,1}; return result end -- multiplies two matrices -function MMulti(M1, M2) +local function MMulti(M1, M2) local M = {{},{},{},{}}; for i = 1,4 do for j = 1,4 do @@ -120,7 +120,7 @@ function MMulti(M1, M2) end -- multiplies matrix with vector -function VMulti(M, V) +local function VMulti(M, V) local Vect = {}; for i = 1,4 do Vect[i] = M[i][1] * V[1] + M[i][2] * V[2] + M[i][3] * V[3] + M[i][4] * V[4]; @@ -128,7 +128,7 @@ function VMulti(M, V) return Vect; end -function VMulti2(M, V) +local function VMulti2(M, V) local Vect = {}; for i = 1,3 do Vect[i] = M[i][1] * V[1] + M[i][2] * V[2] + M[i][3] * V[3]; @@ -137,7 +137,7 @@ function VMulti2(M, V) end -- add to matrices -function MAdd(M1, M2) +local function MAdd(M1, M2) local M = {{},{},{},{}}; for i = 1,4 do for j = 1,4 do @@ -147,7 +147,7 @@ function MAdd(M1, M2) return M; end -function Translate(M, Dx, Dy, Dz) +local function Translate(M, Dx, Dy, Dz) local T = { {1,0,0,Dx}, {0,1,0,Dy}, @@ -157,7 +157,7 @@ function Translate(M, Dx, Dy, Dz) return MMulti(T, M); end -function RotateX(M, Phi) +local function RotateX(M, Phi) local a = Phi; a = a * math.pi / 180; local Cos = math.cos(a); @@ -171,7 +171,7 @@ function RotateX(M, Phi) return MMulti(R, M); end -function RotateY(M, Phi) +local function RotateY(M, Phi) local a = Phi; a = a * math.pi / 180; local Cos = math.cos(a); @@ -185,7 +185,7 @@ function RotateY(M, Phi) return MMulti(R, M); end -function RotateZ(M, Phi) +local function RotateZ(M, Phi) local a = Phi; a = a * math.pi / 180; local Cos = math.cos(a); @@ -199,7 +199,7 @@ function RotateZ(M, Phi) return MMulti(R, M); end -function DrawQube() +local function DrawQube() -- calc current normals local CurN = {}; local i = 5; @@ -245,7 +245,7 @@ function DrawQube() Q.LastPx = 0; end -function Loop() +local function Loop() if (Testing.LoopCount > Testing.LoopMax) then return; end local TestingStr = tostring(Testing.LoopCount); while (#TestingStr < 3) do TestingStr = "0" .. TestingStr; end @@ -265,7 +265,7 @@ function Loop() Loop(); end -function Init(CubeSize) +local function Init(CubeSize) -- init/reset vars Origin.V = {150,150,20,1}; Testing.LoopCount = 0; diff --git a/bench/tests/sunspider/3d-morph.lua b/bench/tests/sunspider/3d-morph.lua index f73f173b..79e91419 100644 --- a/bench/tests/sunspider/3d-morph.lua +++ b/bench/tests/sunspider/3d-morph.lua @@ -31,7 +31,7 @@ local loops = 15 local nx = 120 local nz = 120 -function morph(a, f) +local function morph(a, f) local PI2nx = math.pi * 8/nx local sin = math.sin local f30 = -(50 * sin(f*math.pi*2)) diff --git a/bench/tests/sunspider/3d-raytrace.lua b/bench/tests/sunspider/3d-raytrace.lua index c8f6b5dc..3d5276c7 100644 --- a/bench/tests/sunspider/3d-raytrace.lua +++ b/bench/tests/sunspider/3d-raytrace.lua @@ -28,40 +28,40 @@ function test() local size = 30 -function createVector(x,y,z) +local function createVector(x,y,z) return { x,y,z }; end -function sqrLengthVector(self) +local function sqrLengthVector(self) return self[1] * self[1] + self[2] * self[2] + self[3] * self[3]; end -function lengthVector(self) +local function lengthVector(self) return math.sqrt(self[1] * self[1] + self[2] * self[2] + self[3] * self[3]); end -function addVector(self, v) +local function addVector(self, v) self[1] = self[1] + v[1]; self[2] = self[2] + v[2]; self[3] = self[3] + v[3]; return self; end -function subVector(self, v) +local function subVector(self, v) self[1] = self[1] - v[1]; self[2] = self[2] - v[2]; self[3] = self[3] - v[3]; return self; end -function scaleVector(self, scale) +local function scaleVector(self, scale) self[1] = self[1] * scale; self[2] = self[2] * scale; self[3] = self[3] * scale; return self; end -function normaliseVector(self) +local function normaliseVector(self) local len = math.sqrt(self[1] * self[1] + self[2] * self[2] + self[3] * self[3]); self[1] = self[1] / len; self[2] = self[2] / len; @@ -69,39 +69,39 @@ function normaliseVector(self) return self; end -function add(v1, v2) +local function add(v1, v2) return { v1[1] + v2[1], v1[2] + v2[2], v1[3] + v2[3] }; end -function sub(v1, v2) +local function sub(v1, v2) return { v1[1] - v2[1], v1[2] - v2[2], v1[3] - v2[3] }; end -function scalev(v1, v2) +local function scalev(v1, v2) return { v1[1] * v2[1], v1[2] * v2[2], v1[3] * v2[3] }; end -function dot(v1, v2) +local function dot(v1, v2) return v1[1] * v2[1] + v1[2] * v2[2] + v1[3] * v2[3]; end -function scale(v, scale) +local function scale(v, scale) return { v[1] * scale, v[2] * scale, v[3] * scale }; end -function cross(v1, v2) +local function cross(v1, v2) return { v1[2] * v2[3] - v1[3] * v2[2], v1[3] * v2[1] - v1[1] * v2[3], v1[1] * v2[2] - v1[2] * v2[1] }; end -function normalise(v) +local function normalise(v) local len = lengthVector(v); return { v[1] / len, v[2] / len, v[3] / len }; end -function transformMatrix(self, v) +local function transformMatrix(self, v) local vals = self; local x = vals[1] * v[1] + vals[2] * v[2] + vals[3] * v[3] + vals[4]; local y = vals[5] * v[1] + vals[6] * v[2] + vals[7] * v[3] + vals[8]; @@ -109,7 +109,7 @@ function transformMatrix(self, v) return { x, y, z }; end -function invertMatrix(self) +local function invertMatrix(self) local temp = {} local tx = -self[4]; local ty = -self[8]; @@ -131,7 +131,7 @@ function invertMatrix(self) end -- Triangle intersection using barycentric coord method -function Triangle(p1, p2, p3) +local function Triangle(p1, p2, p3) local this = {} local edge1 = sub(p3, p1); @@ -205,7 +205,7 @@ function Triangle(p1, p2, p3) return this end -function Scene(a_triangles) +local function Scene(a_triangles) local this = {} this.triangles = a_triangles; this.lights = {}; @@ -302,7 +302,7 @@ local zero = { 0,0,0 }; -- this camera code is from notes i made ages ago, it is from *somewhere* -- i cannot remember where -- that somewhere is -function Camera(origin, lookat, up) +local function Camera(origin, lookat, up) local this = {} local zaxis = normaliseVector(subVector(lookat, origin)); @@ -357,7 +357,7 @@ function Camera(origin, lookat, up) return this end -function raytraceScene() +local function raytraceScene() local startDate = 13154863; local numTriangles = 2 * 6; local triangles = {}; -- numTriangles); @@ -450,7 +450,7 @@ function raytraceScene() return pixels; end -function arrayToCanvasCommands(pixels) +local function arrayToCanvasCommands(pixels) local s = {}; table.insert(s, 'Test\nvar pixels = ['); for y = 0,size-1 do @@ -485,7 +485,7 @@ for (var y = 0; y < size; y++) {\n\ return table.concat(s); end -testOutput = arrayToCanvasCommands(raytraceScene()); +local testOutput = arrayToCanvasCommands(raytraceScene()); --local f = io.output("output.html") --f:write(testOutput) diff --git a/bench/tests/sunspider/access-binary-trees.lua b/bench/tests/sunspider/access-binary-trees.lua deleted file mode 100644 index 9eb93588..00000000 --- a/bench/tests/sunspider/access-binary-trees.lua +++ /dev/null @@ -1,69 +0,0 @@ ---[[ - The Great Computer Language Shootout - http://shootout.alioth.debian.org/ - contributed by Isaac Gouy -]] - -local bench = script and require(script.Parent.bench_support) or require("bench_support") - -function test() - -function TreeNode(left,right,item) - local this = {} - this.left = left; - this.right = right; - this.item = item; - - this.itemCheck = function(self) - if (self.left==nil) then return self.item; - else return self.item + self.left:itemCheck() - self.right:itemCheck(); end - end - - return this -end - -function bottomUpTree(item,depth) - if (depth>0) then - return TreeNode( - bottomUpTree(2*item-1, depth-1) - ,bottomUpTree(2*item, depth-1) - ,item - ); - else - return TreeNode(nil,nil,item); - end -end - -local ret = 0; - -for n = 4,7,1 do - local minDepth = 4; - local maxDepth = math.max(minDepth + 2, n); - local stretchDepth = maxDepth + 1; - - local check = bottomUpTree(0,stretchDepth):itemCheck(); - - local longLivedTree = bottomUpTree(0,maxDepth); - - for depth = minDepth,maxDepth,2 do - local iterations = 2.0 ^ (maxDepth - depth + minDepth - 1) -- 1 << (maxDepth - depth + minDepth); - - check = 0; - for i = 1,iterations do - check = check + bottomUpTree(i,depth):itemCheck(); - check = check + bottomUpTree(-i,depth):itemCheck(); - end - end - - ret = ret + longLivedTree:itemCheck(); -end - -local expected = -4; - -if (ret ~= expected) then - assert(false, "ERROR: bad result: expected " .. expected .. " but got " .. ret); -end - -end - -bench.runCode(test, "access-binary-trees") diff --git a/bench/tests/sunspider/controlflow-recursive.lua b/bench/tests/sunspider/controlflow-recursive.lua index d0791626..a2591b2f 100644 --- a/bench/tests/sunspider/controlflow-recursive.lua +++ b/bench/tests/sunspider/controlflow-recursive.lua @@ -7,18 +7,18 @@ local bench = script and require(script.Parent.bench_support) or require("bench_ function test() -function ack(m,n) +local function ack(m,n) if (m==0) then return n+1; end if (n==0) then return ack(m-1,1); end return ack(m-1, ack(m,n-1) ); end -function fib(n) +local function fib(n) if (n < 2) then return 1; end return fib(n-2) + fib(n-1); end -function tak(x,y,z) +local function tak(x,y,z) if (y >= x) then return z; end return tak(tak(x-1,y,z), tak(y-1,z,x), tak(z-1,x,y)); end @@ -27,7 +27,7 @@ local result = 0; for i = 3,5 do result = result + ack(3,i); - result = result + fib(17.0+i); + result = result + fib(17+i); result = result + tak(3*i+3,2*i+2,i+1); end diff --git a/bench/tests/sunspider/crypto-aes.lua b/bench/tests/sunspider/crypto-aes.lua index 3b289729..8dd0cec6 100644 --- a/bench/tests/sunspider/crypto-aes.lua +++ b/bench/tests/sunspider/crypto-aes.lua @@ -42,7 +42,68 @@ local Rcon = { { 0x00, 0x00, 0x00, 0x00 }, {0x1b, 0x00, 0x00, 0x00}, {0x36, 0x00, 0x00, 0x00} }; -function Cipher(input, w) -- main Cipher function [§5.1] +local function SubBytes(s, Nb) -- apply SBox to state S [§5.1.1] + for r = 0,3 do + for c = 0,Nb-1 do s[r + 1][c + 1] = Sbox[s[r + 1][c + 1] + 1]; end + end + return s; +end + + +local function ShiftRows(s, Nb) -- shift row r of state S left by r bytes [§5.1.2] + local t = {}; + for r = 1,3 do + for c = 0,3 do t[c + 1] = s[r + 1][((c + r) % Nb) + 1] end; -- shift into temp copy + for c = 0,3 do s[r + 1][c + 1] = t[c + 1]; end -- and copy back + end -- note that this will work for Nb=4,5,6, but not 7,8 (always 4 for AES): + return s; -- see fp.gladman.plus.com/cryptography_technology/rijndael/aes.spec.311.pdf +end + + +local function MixColumns(s, Nb) -- combine bytes of each col of state S [§5.1.3] + for c = 0,3 do + local a = {}; -- 'a' is a copy of the current column from 's' + local b = {}; -- 'b' is a•{02} in GF(2^8) + for i = 0,3 do + a[i + 1] = s[i + 1][c + 1]; + + if bit32.band(s[i + 1][c + 1], 0x80) ~= 0 then + b[i + 1] = bit32.bxor(bit32.lshift(s[i + 1][c + 1], 1), 0x011b); + else + b[i + 1] = bit32.lshift(s[i + 1][c + 1], 1); + end + end + -- a[n] ^ b[n] is a•{03} in GF(2^8) + s[1][c + 1] = bit32.bxor(b[1], a[2], b[2], a[3], a[4]); -- 2*a0 + 3*a1 + a2 + a3 + s[2][c + 1] = bit32.bxor(a[1], b[2], a[3], b[3], a[4]); -- a0 * 2*a1 + 3*a2 + a3 + s[3][c + 1] = bit32.bxor(a[1], a[2], b[3], a[4], b[4]); -- a0 + a1 + 2*a2 + 3*a3 + s[4][c + 1] = bit32.bxor(a[1], b[1], a[2], a[3], b[4]); -- 3*a0 + a1 + a2 + 2*a3 +end + return s; +end + + +local function SubWord(w) -- apply SBox to 4-byte word w + for i = 0,3 do w[i + 1] = Sbox[w[i + 1] + 1]; end + return w; +end + +local function RotWord(w) -- rotate 4-byte word w left by one byte + w[5] = w[1]; + for i = 0,3 do w[i + 1] = w[i + 2]; end + return w; +end + + + +local function AddRoundKey(state, w, rnd, Nb) -- xor Round Key into state S [§5.1.4] + for r = 0,3 do + for c = 0,Nb-1 do state[r + 1][c + 1] = bit32.bxor(state[r + 1][c + 1], w[rnd*4+c + 1][r + 1]); end + end + return state; +end + +local function Cipher(input, w) -- main Cipher function [§5.1] local Nb = 4; -- block size (in words): no of columns in state (fixed at 4 for AES) local Nr = #w / Nb - 1; -- no of rounds: 10/12/14 for 128/192/256-bit keys @@ -69,56 +130,7 @@ function Cipher(input, w) -- main Cipher function [§5.1] end -function SubBytes(s, Nb) -- apply SBox to state S [§5.1.1] - for r = 0,3 do - for c = 0,Nb-1 do s[r + 1][c + 1] = Sbox[s[r + 1][c + 1] + 1]; end - end - return s; -end - - -function ShiftRows(s, Nb) -- shift row r of state S left by r bytes [§5.1.2] - local t = {}; - for r = 1,3 do - for c = 0,3 do t[c + 1] = s[r + 1][((c + r) % Nb) + 1] end; -- shift into temp copy - for c = 0,3 do s[r + 1][c + 1] = t[c + 1]; end -- and copy back - end -- note that this will work for Nb=4,5,6, but not 7,8 (always 4 for AES): - return s; -- see fp.gladman.plus.com/cryptography_technology/rijndael/aes.spec.311.pdf -end - - -function MixColumns(s, Nb) -- combine bytes of each col of state S [§5.1.3] - for c = 0,3 do - local a = {}; -- 'a' is a copy of the current column from 's' - local b = {}; -- 'b' is a•{02} in GF(2^8) - for i = 0,3 do - a[i + 1] = s[i + 1][c + 1]; - - if bit32.band(s[i + 1][c + 1], 0x80) ~= 0 then - b[i + 1] = bit32.bxor(bit32.lshift(s[i + 1][c + 1], 1), 0x011b); - else - b[i + 1] = bit32.lshift(s[i + 1][c + 1], 1); - end - end - -- a[n] ^ b[n] is a•{03} in GF(2^8) - s[1][c + 1] = bit32.bxor(b[1], a[2], b[2], a[3], a[4]); -- 2*a0 + 3*a1 + a2 + a3 - s[2][c + 1] = bit32.bxor(a[1], b[2], a[3], b[3], a[4]); -- a0 * 2*a1 + 3*a2 + a3 - s[3][c + 1] = bit32.bxor(a[1], a[2], b[3], a[4], b[4]); -- a0 + a1 + 2*a2 + 3*a3 - s[4][c + 1] = bit32.bxor(a[1], b[1], a[2], a[3], b[4]); -- 3*a0 + a1 + a2 + 2*a3 -end - return s; -end - - -function AddRoundKey(state, w, rnd, Nb) -- xor Round Key into state S [§5.1.4] - for r = 0,3 do - for c = 0,Nb-1 do state[r + 1][c + 1] = bit32.bxor(state[r + 1][c + 1], w[rnd*4+c + 1][r + 1]); end - end - return state; -end - - -function KeyExpansion(key) -- generate Key Schedule (byte-array Nr+1 x Nb) from Key [§5.2] +local function KeyExpansion(key) -- generate Key Schedule (byte-array Nr+1 x Nb) from Key [§5.2] local Nb = 4; -- block size (in words): no of columns in state (fixed at 4 for AES) local Nk = #key / 4 -- key length (in words): 4/6/8 for 128/192/256-bit keys local Nr = Nk + 6; -- no of rounds: 10/12/14 for 128/192/256-bit keys @@ -146,17 +158,17 @@ function KeyExpansion(key) -- generate Key Schedule (byte-array Nr+1 x Nb) from return w; end -function SubWord(w) -- apply SBox to 4-byte word w - for i = 0,3 do w[i + 1] = Sbox[w[i + 1] + 1]; end - return w; +local function escCtrlChars(str) -- escape control chars which might cause problems handling ciphertext + return string.gsub(str, "[\0\t\n\v\f\r\'\"!-]", function(c) return '!' .. string.byte(c, 1) .. '!'; end); end -function RotWord(w) -- rotate 4-byte word w left by one byte - w[5] = w[1]; - for i = 0,3 do w[i + 1] = w[i + 2]; end - return w; -end +local function unescCtrlChars(str) -- unescape potentially problematic control characters + return string.gsub(str, "!%d%d?%d?!", function(c) + local sc = string.sub(c, 2,-2) + return string.char(tonumber(sc)); + end); +end --[[ * Use AES to encrypt 'plaintext' with 'password' using 'nBits' key, in 'Counter' mode of operation @@ -166,7 +178,7 @@ end * - cipherblock = plaintext xor outputblock ]] -function AESEncryptCtr(plaintext, password, nBits) +local function AESEncryptCtr(plaintext, password, nBits) if (not (nBits==128 or nBits==192 or nBits==256)) then return ''; end -- standard allows 128/192/256 bit keys -- for this example script, generate the key by applying Cipher to 1st 16/24/32 chars of password; @@ -243,7 +255,7 @@ end * - cipherblock = plaintext xor outputblock ]] -function AESDecryptCtr(ciphertext, password, nBits) +local function AESDecryptCtr(ciphertext, password, nBits) if (not (nBits==128 or nBits==192 or nBits==256)) then return ''; end -- standard allows 128/192/256 bit keys local nBytes = nBits/8; -- no bytes in key @@ -300,19 +312,7 @@ function AESDecryptCtr(ciphertext, password, nBits) return table.concat(plaintext) end -function escCtrlChars(str) -- escape control chars which might cause problems handling ciphertext - return string.gsub(str, "[\0\t\n\v\f\r\'\"!-]", function(c) return '!' .. string.byte(c, 1) .. '!'; end); -end - -function unescCtrlChars(str) -- unescape potentially problematic control characters - return string.gsub(str, "!%d%d?%d?!", function(c) - local sc = string.sub(c, 2,-2) - - return string.char(tonumber(sc)); - end); -end - -function test() +local function test() local plainText = "ROMEO: But, soft! what light through yonder window breaks?\n\ It is the east, and Juliet is the sun.\n\ diff --git a/bench/tests/sunspider/math-cordic.lua b/bench/tests/sunspider/math-cordic.lua index 94a64f45..cdb10fa2 100644 --- a/bench/tests/sunspider/math-cordic.lua +++ b/bench/tests/sunspider/math-cordic.lua @@ -31,15 +31,15 @@ function test() local AG_CONST = 0.6072529350; -function FIXED(X) +local function FIXED(X) return X * 65536.0; end -function FLOAT(X) +local function FLOAT(X) return X / 65536.0; end -function DEG2RAD(X) +local function DEG2RAD(X) return 0.017453 * (X); end @@ -52,7 +52,7 @@ local Angles = { local Target = 28.027; -function cordicsincos(Target) +local function cordicsincos(Target) local X; local Y; local TargetAngle; @@ -85,7 +85,7 @@ end local total = 0; -function cordic( runs ) +local function cordic( runs ) for i = 1,runs do total = total + cordicsincos(Target); end diff --git a/bench/tests/sunspider/math-partial-sums.lua b/bench/tests/sunspider/math-partial-sums.lua index 3c222876..9977ceff 100644 --- a/bench/tests/sunspider/math-partial-sums.lua +++ b/bench/tests/sunspider/math-partial-sums.lua @@ -7,7 +7,7 @@ local bench = script and require(script.Parent.bench_support) or require("bench_ function test() -function partial(n) +local function partial(n) local a1, a2, a3, a4, a5, a6, a7, a8, a9 = 0, 0, 0, 0, 0, 0, 0, 0, 0; local twothirds = 2.0/3.0; local alt = -1.0; diff --git a/bench/tests/sunspider/math-spectral-norm.lua b/bench/tests/sunspider/math-spectral-norm.lua deleted file mode 100644 index 7d7ec163..00000000 --- a/bench/tests/sunspider/math-spectral-norm.lua +++ /dev/null @@ -1,72 +0,0 @@ ---[[ -The Great Computer Language Shootout -http://shootout.alioth.debian.org/ - -contributed by Ian Osgood -]] -local bench = script and require(script.Parent.bench_support) or require("bench_support") - -function test() - -function A(i,j) - return 1/((i+j)*(i+j+1)/2+i+1); -end - -function Au(u,v) - for i = 0,#u-1 do - local t = 0; - for j = 0,#u-1 do - t = t + A(i,j) * u[j + 1]; - end - v[i + 1] = t; - end -end - -function Atu(u,v) - for i = 0,#u-1 do - local t = 0; - for j = 0,#u-1 do - t = t + A(j,i) * u[j + 1]; - end - v[i + 1] = t; - end -end - -function AtAu(u,v,w) - Au(u,w); - Atu(w,v); -end - -function spectralnorm(n) - local u, v, w, vv, vBv = {}, {}, {}, 0, 0; - for i = 1,n do - u[i] = 1; v[i] = 0; w[i] = 0; - end - for i = 0,9 do - AtAu(u,v,w); - AtAu(v,u,w); - end - for i = 1,n do - vBv = vBv + u[i]*v[i]; - vv = vv + v[i]*v[i]; - end - return math.sqrt(vBv/vv); -end - -local total = 0; -local i = 6 - -while i <= 48 do - total = total + spectralnorm(i); - i = i * 2 -end - -local expected = 5.086694231303284; - -if (total ~= expected) then - assert(false, "ERROR: bad result: expected " .. expected .. " but got " .. total) -end - -end - -bench.runCode(test, "math-spectral-norm") diff --git a/tests/Autocomplete.test.cpp b/tests/Autocomplete.test.cpp index 5b70481b..1c284f1f 100644 --- a/tests/Autocomplete.test.cpp +++ b/tests/Autocomplete.test.cpp @@ -2760,8 +2760,6 @@ TEST_CASE_FIXTURE(ACFixture, "autocomplete_on_string_singletons") TEST_CASE_FIXTURE(ACFixture, "autocomplete_string_singletons") { - ScopedFastFlag luauAutocompleteSingletonTypes{"LuauAutocompleteSingletonTypes", true}; - check(R"( type tag = "cat" | "dog" local function f(a: tag) end @@ -2798,8 +2796,6 @@ TEST_CASE_FIXTURE(ACFixture, "autocomplete_string_singletons") TEST_CASE_FIXTURE(ACFixture, "autocomplete_string_singleton_equality") { - ScopedFastFlag luauAutocompleteSingletonTypes{"LuauAutocompleteSingletonTypes", true}; - check(R"( type tagged = {tag:"cat", fieldx:number} | {tag:"dog", fieldy:number} local x: tagged = {tag="cat", fieldx=2} @@ -2821,8 +2817,6 @@ TEST_CASE_FIXTURE(ACFixture, "autocomplete_string_singleton_equality") TEST_CASE_FIXTURE(ACFixture, "autocomplete_boolean_singleton") { - ScopedFastFlag luauAutocompleteSingletonTypes{"LuauAutocompleteSingletonTypes", true}; - check(R"( local function f(x: true) end f(@1) @@ -2838,8 +2832,6 @@ f(@1) TEST_CASE_FIXTURE(ACFixture, "autocomplete_string_singleton_escape") { - ScopedFastFlag luauAutocompleteSingletonTypes{"LuauAutocompleteSingletonTypes", true}; - check(R"( type tag = "strange\t\"cat\"" | 'nice\t"dog"' local function f(x: tag) end diff --git a/tests/Compiler.test.cpp b/tests/Compiler.test.cpp index 7b4bfc72..f206438f 100644 --- a/tests/Compiler.test.cpp +++ b/tests/Compiler.test.cpp @@ -261,6 +261,9 @@ RETURN R0 0 TEST_CASE("ForBytecode") { + ScopedFastFlag sff("LuauCompileIter", true); + ScopedFastFlag sff2("LuauCompileIterNoPairs", false); + // basic for loop: variable directly refers to internal iteration index (R2) CHECK_EQ("\n" + compileFunction0("for i=1,5 do print(i) end"), R"( LOADN R2 1 @@ -295,7 +298,7 @@ GETIMPORT R0 2 LOADK R1 K3 LOADK R2 K4 CALL R0 2 3 -JUMP +4 +FORGPREP R0 +4 GETIMPORT R5 6 MOVE R6 R3 CALL R5 1 0 @@ -347,6 +350,8 @@ RETURN R0 0 TEST_CASE("ForBytecodeBuiltin") { + ScopedFastFlag sff("LuauCompileIter", true); + // we generally recognize builtins like pairs/ipairs and emit special opcodes CHECK_EQ("\n" + compileFunction0("for k,v in ipairs({}) do end"), R"( GETIMPORT R0 1 @@ -385,7 +390,7 @@ GETIMPORT R0 3 MOVE R1 R0 NEWTABLE R2 0 0 CALL R1 1 3 -JUMP +0 +FORGPREP R1 +0 FORGLOOP R1 -1 2 RETURN R0 0 )"); @@ -397,7 +402,7 @@ SETGLOBAL R0 K2 GETGLOBAL R0 K2 NEWTABLE R1 0 0 CALL R0 1 3 -JUMP +0 +FORGPREP R0 +0 FORGLOOP R0 -1 2 RETURN R0 0 )"); @@ -407,7 +412,7 @@ RETURN R0 0 GETIMPORT R0 1 NEWTABLE R1 0 0 CALL R0 1 3 -JUMP +0 +FORGPREP R0 +0 FORGLOOP R0 -1 2 RETURN R0 0 )"); @@ -2260,6 +2265,8 @@ TEST_CASE("TypeAliasing") TEST_CASE("DebugLineInfo") { + ScopedFastFlag sff("LuauCompileIterNoPairs", false); + Luau::BytecodeBuilder bcb; bcb.setDumpFlags(Luau::BytecodeBuilder::Dump_Code | Luau::BytecodeBuilder::Dump_Lines); Luau::compileOrThrow(bcb, R"( @@ -2316,6 +2323,8 @@ return result TEST_CASE("DebugLineInfoFor") { + ScopedFastFlag sff("LuauCompileIter", true); + Luau::BytecodeBuilder bcb; bcb.setDumpFlags(Luau::BytecodeBuilder::Dump_Code | Luau::BytecodeBuilder::Dump_Lines); Luau::compileOrThrow(bcb, R"( @@ -2336,7 +2345,7 @@ end 5: LOADN R0 1 7: LOADN R1 2 9: LOADN R2 3 -9: JUMP +4 +9: FORGPREP R0 +4 11: GETIMPORT R5 1 11: MOVE R6 R3 11: CALL R5 1 0 @@ -2541,6 +2550,8 @@ a TEST_CASE("DebugSource") { + ScopedFastFlag sff("LuauCompileIterNoPairs", false); + const char* source = R"( local kSelectedBiomes = { ['Mountains'] = true, @@ -2616,6 +2627,8 @@ RETURN R1 1 TEST_CASE("DebugLocals") { + ScopedFastFlag sff("LuauCompileIterNoPairs", false); + const char* source = R"( function foo(e, f) local a = 1 @@ -3767,6 +3780,8 @@ RETURN R0 1 TEST_CASE("SharedClosure") { + ScopedFastFlag sff("LuauCompileIterNoPairs", false); + // closures can be shared even if functions refer to upvalues, as long as upvalues are top-level CHECK_EQ("\n" + compileFunction(R"( local val = ... @@ -4452,5 +4467,688 @@ RETURN R0 0 )"); } +TEST_CASE("InlineBasic") +{ + ScopedFastFlag sff("LuauCompileSupportInlining", true); + + // inline function that returns a constant + CHECK_EQ("\n" + compileFunction(R"( +local function foo() + return 42 +end + +local x = foo() +return x +)", + 1, 2), + R"( +DUPCLOSURE R0 K0 +LOADN R1 42 +RETURN R1 1 +)"); + + // inline function that returns the argument + CHECK_EQ("\n" + compileFunction(R"( +local function foo(a) + return a +end + +local x = foo(42) +return x +)", + 1, 2), + R"( +DUPCLOSURE R0 K0 +LOADN R1 42 +RETURN R1 1 +)"); + + // inline function that returns one of the two arguments + CHECK_EQ("\n" + compileFunction(R"( +local function foo(a, b, c) + if a then + return b + else + return c + end +end + +local x = foo(true, math.random(), 5) +return x +)", + 1, 2), + R"( +DUPCLOSURE R0 K0 +GETIMPORT R2 3 +CALL R2 0 1 +MOVE R1 R2 +RETURN R1 1 +RETURN R1 1 +)"); + + // inline function that returns one of the two arguments + CHECK_EQ("\n" + compileFunction(R"( +local function foo(a, b, c) + if a then + return b + else + return c + end +end + +local x = foo(true, 5, math.random()) +return x +)", + 1, 2), + R"( +DUPCLOSURE R0 K0 +GETIMPORT R2 3 +CALL R2 0 1 +LOADN R1 5 +RETURN R1 1 +RETURN R1 1 +)"); +} + +TEST_CASE("InlineMutate") +{ + ScopedFastFlag sff("LuauCompileSupportInlining", true); + + // if the argument is mutated, it gets a register even if the value is constant + CHECK_EQ("\n" + compileFunction(R"( +local function foo(a) + a = a or 5 + return a +end + +local x = foo(42) +return x +)", + 1, 2), + R"( +DUPCLOSURE R0 K0 +LOADN R2 42 +ORK R2 R2 K1 +MOVE R1 R2 +RETURN R1 1 +)"); + + // if the argument is a local, it can be used directly + CHECK_EQ("\n" + compileFunction(R"( +local function foo(a) + return a +end + +local x = ... +local y = foo(x) +return y +)", + 1, 2), + R"( +DUPCLOSURE R0 K0 +GETVARARGS R1 1 +MOVE R2 R1 +RETURN R2 1 +)"); + + // ... but if it's mutated, we move it in case it is mutated through a capture during the inlined function + CHECK_EQ("\n" + compileFunction(R"( +local function foo(a) + return a +end + +local x = ... +x = nil +local y = foo(x) +return y +)", + 1, 2), + R"( +DUPCLOSURE R0 K0 +GETVARARGS R1 1 +LOADNIL R1 +MOVE R3 R1 +MOVE R2 R3 +RETURN R2 1 +)"); + + // we also don't inline functions if they have been assigned to + CHECK_EQ("\n" + compileFunction(R"( +local function foo(a) + return a +end + +foo = foo + +local x = foo(42) +return x +)", + 1, 2), + R"( +DUPCLOSURE R0 K0 +MOVE R0 R0 +MOVE R1 R0 +LOADN R2 42 +CALL R1 1 1 +RETURN R1 1 +)"); +} + +TEST_CASE("InlineUpval") +{ + ScopedFastFlag sff("LuauCompileSupportInlining", true); + + // if the argument is an upvalue, we naturally need to copy it to a local + CHECK_EQ("\n" + compileFunction(R"( +local function foo(a) + return a +end + +local b = ... + +function bar() + local x = foo(b) + return x +end +)", + 1, 2), + R"( +GETUPVAL R1 0 +MOVE R0 R1 +RETURN R0 1 +)"); + + // if the function uses an upvalue it's more complicated, because the lexical upvalue may become a local + CHECK_EQ("\n" + compileFunction(R"( +local b = ... + +local function foo(a) + return a + b +end + +local x = foo(42) +return x +)", + 1, 2), + R"( +GETVARARGS R0 1 +DUPCLOSURE R1 K0 +CAPTURE VAL R0 +LOADN R3 42 +ADD R2 R3 R0 +RETURN R2 1 +)"); + + // sometimes the lexical upvalue is deep enough that it's still an upvalue though + CHECK_EQ("\n" + compileFunction(R"( +local b = ... + +function bar() + local function foo(a) + return a + b + end + + local x = foo(42) + return x +end +)", + 1, 2), + R"( +DUPCLOSURE R0 K0 +CAPTURE UPVAL U0 +LOADN R2 42 +GETUPVAL R3 0 +ADD R1 R2 R3 +RETURN R1 1 +)"); +} + +TEST_CASE("InlineFallthrough") +{ + ScopedFastFlag sff("LuauCompileSupportInlining", true); + + // if the function doesn't return, we still fill the results with nil + CHECK_EQ("\n" + compileFunction(R"( +local function foo() +end + +local a, b = foo() + +return a, b +)", + 1, 2), + R"( +DUPCLOSURE R0 K0 +LOADNIL R1 +LOADNIL R2 +MOVE R3 R1 +MOVE R4 R2 +RETURN R3 2 +)"); + + // this happens even if the function returns conditionally + CHECK_EQ("\n" + compileFunction(R"( +local function foo(a) + if a then return 42 end +end + +local a, b = foo(false) + +return a, b +)", + 1, 2), + R"( +DUPCLOSURE R0 K0 +LOADNIL R1 +LOADNIL R2 +MOVE R3 R1 +MOVE R4 R2 +RETURN R3 2 +)"); + + // note though that we can't inline a function like this in multret context + // this is because we don't have a SETTOP instruction + CHECK_EQ("\n" + compileFunction(R"( +local function foo() +end + +return foo() +)", + 1, 2), + R"( +DUPCLOSURE R0 K0 +MOVE R1 R0 +CALL R1 0 -1 +RETURN R1 -1 +)"); +} + +TEST_CASE("InlineCapture") +{ + ScopedFastFlag sff("LuauCompileSupportInlining", true); + + // can't inline function with nested functions that capture locals because they might be constants + CHECK_EQ("\n" + compileFunction(R"( +local function foo(a) + local function bar() + return a + end + return bar() +end +)", + 1, 2), + R"( +NEWCLOSURE R1 P0 +CAPTURE VAL R0 +MOVE R2 R1 +CALL R2 0 -1 +RETURN R2 -1 +)"); +} + +TEST_CASE("InlineArgMismatch") +{ + ScopedFastFlag sff("LuauCompileSupportInlining", true); + + // when inlining a function, we must respect all the usual rules + + // caller might not have enough arguments + // TODO: we don't inline this atm + CHECK_EQ("\n" + compileFunction(R"( +local function foo(a) + return a +end + +local x = foo() +return x +)", + 1, 2), + R"( +DUPCLOSURE R0 K0 +MOVE R1 R0 +CALL R1 0 1 +RETURN R1 1 +)"); + + // caller might be using multret for arguments + // TODO: we don't inline this atm + CHECK_EQ("\n" + compileFunction(R"( +local function foo(a, b) + return a + b +end + +local x = foo(math.modf(1.5)) +return x +)", + 1, 2), + R"( +DUPCLOSURE R0 K0 +MOVE R1 R0 +LOADK R3 K1 +FASTCALL1 20 R3 +2 +GETIMPORT R2 4 +CALL R2 1 -1 +CALL R1 -1 1 +RETURN R1 1 +)"); + + // caller might have too many arguments, but we still need to compute them for side effects + // TODO: we don't inline this atm + CHECK_EQ("\n" + compileFunction(R"( +local function foo(a) + return a +end + +local x = foo(42, print()) +return x +)", + 1, 2), + R"( +DUPCLOSURE R0 K0 +MOVE R1 R0 +LOADN R2 42 +GETIMPORT R3 2 +CALL R3 0 -1 +CALL R1 -1 1 +RETURN R1 1 +)"); +} + +TEST_CASE("InlineMultiple") +{ + ScopedFastFlag sff("LuauCompileSupportInlining", true); + + // we call this with a different set of variable/constant args + CHECK_EQ("\n" + compileFunction(R"( +local function foo(a, b) + return a + b +end + +local x, y = ... +local a = foo(x, 1) +local b = foo(1, x) +local c = foo(1, 2) +local d = foo(x, y) +return a, b, c, d +)", + 1, 2), + R"( +DUPCLOSURE R0 K0 +GETVARARGS R1 2 +ADDK R3 R1 K1 +LOADN R5 1 +ADD R4 R5 R1 +LOADN R5 3 +ADD R6 R1 R2 +MOVE R7 R3 +MOVE R8 R4 +MOVE R9 R5 +MOVE R10 R6 +RETURN R7 4 +)"); +} + +TEST_CASE("InlineChain") +{ + ScopedFastFlag sff("LuauCompileSupportInlining", true); + + // inline a chain of functions + CHECK_EQ("\n" + compileFunction(R"( +local function foo(a, b) + return a + b +end + +local function bar(x) + return foo(x, 1) * foo(x, -1) +end + +local function baz() + return (bar(42)) +end + +return (baz()) +)", + 3, 2), + R"( +DUPCLOSURE R0 K0 +DUPCLOSURE R1 K1 +DUPCLOSURE R2 K2 +LOADN R4 43 +LOADN R5 41 +MUL R3 R4 R5 +RETURN R3 1 +)"); +} + +TEST_CASE("InlineThresholds") +{ + ScopedFastFlag sff("LuauCompileSupportInlining", true); + + ScopedFastInt sfis[] = { + {"LuauCompileInlineThreshold", 25}, + {"LuauCompileInlineThresholdMaxBoost", 300}, + {"LuauCompileInlineDepth", 2}, + }; + + // this function has enormous register pressure (50 regs) so we choose not to inline it + CHECK_EQ("\n" + compileFunction(R"( +local function foo() + return {{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}} +end + +return (foo()) +)", + 1, 2), + R"( +DUPCLOSURE R0 K0 +MOVE R1 R0 +CALL R1 0 1 +RETURN R1 1 +)"); + + // this function has less register pressure but a large cost + CHECK_EQ("\n" + compileFunction(R"( +local function foo() + return {},{},{},{},{} +end + +return (foo()) +)", + 1, 2), + R"( +DUPCLOSURE R0 K0 +MOVE R1 R0 +CALL R1 0 1 +RETURN R1 1 +)"); + + // this chain of function is of length 3 but our limit in this test is 2, so we call foo twice + CHECK_EQ("\n" + compileFunction(R"( +local function foo(a, b) + return a + b +end + +local function bar(x) + return foo(x, 1) * foo(x, -1) +end + +local function baz() + return (bar(42)) +end + +return (baz()) +)", + 3, 2), + R"( +DUPCLOSURE R0 K0 +DUPCLOSURE R1 K1 +DUPCLOSURE R2 K2 +MOVE R4 R0 +LOADN R5 42 +LOADN R6 1 +CALL R4 2 1 +MOVE R5 R0 +LOADN R6 42 +LOADN R7 -1 +CALL R5 2 1 +MUL R3 R4 R5 +RETURN R3 1 +)"); +} + +TEST_CASE("InlineIIFE") +{ + ScopedFastFlag sff("LuauCompileSupportInlining", true); + + // IIFE with arguments + CHECK_EQ("\n" + compileFunction(R"( +function choose(a, b, c) + return ((function(a, b, c) if a then return b else return c end end)(a, b, c)) +end +)", + 1, 2), + R"( +JUMPIFNOT R0 +2 +MOVE R3 R1 +RETURN R3 1 +MOVE R3 R2 +RETURN R3 1 +RETURN R3 1 +)"); + + // IIFE with upvalues + CHECK_EQ("\n" + compileFunction(R"( +function choose(a, b, c) + return ((function() if a then return b else return c end end)()) +end +)", + 1, 2), + R"( +JUMPIFNOT R0 +2 +MOVE R3 R1 +RETURN R3 1 +MOVE R3 R2 +RETURN R3 1 +RETURN R3 1 +)"); +} + +TEST_CASE("InlineRecurseArguments") +{ + ScopedFastFlag sff("LuauCompileSupportInlining", true); + + // we can't inline a function if it's used to compute its own arguments + CHECK_EQ("\n" + compileFunction(R"( +local function foo(a, b) +end +foo(foo(foo,foo(foo,foo))[foo]) +)", + 1, 2), + R"( +DUPCLOSURE R0 K0 +MOVE R1 R0 +MOVE R4 R0 +MOVE R5 R0 +MOVE R6 R0 +CALL R4 2 1 +LOADNIL R3 +GETTABLE R2 R3 R0 +CALL R1 1 0 +RETURN R0 0 +)"); +} + +TEST_CASE("InlineFastCallK") +{ + ScopedFastFlag sff("LuauCompileSupportInlining", true); + + CHECK_EQ("\n" + compileFunction(R"( +local function set(l0) + rawset({}, l0) +end + +set(false) +set({}) +)", + 1, 2), + R"( +DUPCLOSURE R0 K0 +NEWTABLE R2 0 0 +FASTCALL2K 49 R2 K1 +4 +LOADK R3 K1 +GETIMPORT R1 3 +CALL R1 2 0 +NEWTABLE R1 0 0 +NEWTABLE R3 0 0 +FASTCALL2 49 R3 R1 +4 +MOVE R4 R1 +GETIMPORT R2 3 +CALL R2 2 0 +RETURN R0 0 +)"); +} + +TEST_CASE("InlineExprIndexK") +{ + ScopedFastFlag sff("LuauCompileSupportInlining", true); + + CHECK_EQ("\n" + compileFunction(R"( +local _ = function(l0) +local _ = nil +while _(_)[_] do +end +end +local _ = _(0)[""] +if _ then +do +for l0=0,8 do +end +end +elseif _ then +_ = nil +do +for l0=0,8 do +return true +end +end +end +)", + 1, 2), + R"( +DUPCLOSURE R0 K0 +LOADNIL R4 +LOADNIL R5 +CALL R4 1 1 +LOADNIL R5 +GETTABLE R3 R4 R5 +JUMPIFNOT R3 +1 +JUMPBACK -7 +LOADNIL R2 +GETTABLEKS R1 R2 K1 +JUMPIFNOT R1 +1 +RETURN R0 0 +JUMPIFNOT R1 +19 +LOADNIL R1 +LOADB R2 1 +RETURN R2 1 +LOADB R2 1 +RETURN R2 1 +LOADB R2 1 +RETURN R2 1 +LOADB R2 1 +RETURN R2 1 +LOADB R2 1 +RETURN R2 1 +LOADB R2 1 +RETURN R2 1 +LOADB R2 1 +RETURN R2 1 +LOADB R2 1 +RETURN R2 1 +LOADB R2 1 +RETURN R2 1 +RETURN R0 0 +)"); +} TEST_SUITE_END(); diff --git a/tests/Conformance.test.cpp b/tests/Conformance.test.cpp index 6f136d36..a23ea470 100644 --- a/tests/Conformance.test.cpp +++ b/tests/Conformance.test.cpp @@ -241,6 +241,8 @@ TEST_CASE("Math") TEST_CASE("Table") { + ScopedFastFlag sff("LuauFixBuiltinsStackLimit", true); + runConformance("nextvar.lua"); } @@ -1099,4 +1101,14 @@ TEST_CASE("UserdataApi") CHECK(dtorhits == 42); } +TEST_CASE("Iter") +{ + ScopedFastFlag sffs[] = { + { "LuauCompileIter", true }, + { "LuauIter", true }, + }; + + runConformance("iter.lua"); +} + TEST_SUITE_END(); diff --git a/tests/Frontend.test.cpp b/tests/Frontend.test.cpp index e771b6b1..a10e8f7f 100644 --- a/tests/Frontend.test.cpp +++ b/tests/Frontend.test.cpp @@ -386,8 +386,6 @@ TEST_CASE_FIXTURE(FrontendFixture, "cycle_error_paths") TEST_CASE_FIXTURE(FrontendFixture, "cycle_incremental_type_surface") { - ScopedFastFlag luauCyclicModuleTypeSurface{"LuauCyclicModuleTypeSurface", true}; - fileResolver.source["game/A"] = R"( return {hello = 2} )"; @@ -410,8 +408,6 @@ TEST_CASE_FIXTURE(FrontendFixture, "cycle_incremental_type_surface") TEST_CASE_FIXTURE(FrontendFixture, "cycle_incremental_type_surface_longer") { - ScopedFastFlag luauCyclicModuleTypeSurface{"LuauCyclicModuleTypeSurface", true}; - fileResolver.source["game/A"] = R"( return {mod_a = 2} )"; diff --git a/tests/Parser.test.cpp b/tests/Parser.test.cpp index 55eafe3c..69ff73ad 100644 --- a/tests/Parser.test.cpp +++ b/tests/Parser.test.cpp @@ -2041,8 +2041,6 @@ TEST_CASE_FIXTURE(Fixture, "parse_type_alias_default_type_errors") TEST_CASE_FIXTURE(Fixture, "parse_type_pack_errors") { - ScopedFastFlag luauParseRecoverUnexpectedPack{"LuauParseRecoverUnexpectedPack", true}; - matchParseError("type Y = {a: T..., b: number}", "Unexpected '...' after type name; type pack is not allowed in this context", Location{{0, 20}, {0, 23}}); matchParseError("type Y = {a: (number | string)...", "Unexpected '...' after type annotation", Location{{0, 36}, {0, 39}}); @@ -2618,8 +2616,6 @@ type Y = (T...) -> U... TEST_CASE_FIXTURE(Fixture, "recover_unexpected_type_pack") { - ScopedFastFlag luauParseRecoverUnexpectedPack{"LuauParseRecoverUnexpectedPack", true}; - ParseResult result = tryParse(R"( type X = { a: T..., b: number } type Y = { a: T..., b: number } diff --git a/tests/RuntimeLimits.test.cpp b/tests/RuntimeLimits.test.cpp index 0b615e16..538f3576 100644 --- a/tests/RuntimeLimits.test.cpp +++ b/tests/RuntimeLimits.test.cpp @@ -19,8 +19,8 @@ LUAU_FASTFLAG(LuauLowerBoundsCalculation); struct LimitFixture : Fixture { -#if defined(_NOOPT) - ScopedFastInt LuauTypeInferRecursionLimit{"LuauTypeInferRecursionLimit", 150}; +#if defined(_NOOPT) || defined(_DEBUG) + ScopedFastInt LuauTypeInferRecursionLimit{"LuauTypeInferRecursionLimit", 100}; #endif ScopedFastFlag LuauJustOneCallFrameForHaveSeen{"LuauJustOneCallFrameForHaveSeen", true}; @@ -35,12 +35,17 @@ bool hasError(const CheckResult& result, T* = nullptr) return it != result.errors.end(); } -TEST_SUITE_BEGIN("RuntimeLimitTests"); +TEST_SUITE_BEGIN("RuntimeLimits"); -TEST_CASE_FIXTURE(LimitFixture, "bail_early_on_typescript_port_of_Result_type" * doctest::timeout(1.0)) +TEST_CASE_FIXTURE(LimitFixture, "typescript_port_of_Result_type") { constexpr const char* src = R"LUA( --!strict + + -- Big thanks to Dionysusnu by letting us use this code as part of our test suite! + -- https://github.com/Dionysusnu/rbxts-rust-classes + -- Licensed under the MPL 2.0: https://raw.githubusercontent.com/Dionysusnu/rbxts-rust-classes/master/LICENSE + local TS = _G[script] local lazyGet = TS.import(script, script.Parent.Parent, "util", "lazyLoad").lazyGet local unit = TS.import(script, script.Parent.Parent, "util", "Unit").unit diff --git a/tests/TypeInfer.loops.test.cpp b/tests/TypeInfer.loops.test.cpp index 960c6edf..f9b510c1 100644 --- a/tests/TypeInfer.loops.test.cpp +++ b/tests/TypeInfer.loops.test.cpp @@ -488,4 +488,71 @@ TEST_CASE_FIXTURE(Fixture, "fuzz_fail_missing_instantitation_follow") )"); } +TEST_CASE_FIXTURE(Fixture, "loop_iter_basic") +{ + ScopedFastFlag sff{"LuauTypecheckIter", true}; + + CheckResult result = check(R"( + local t: {string} = {} + local key + for k: number in t do + end + for k: number, v: string in t do + end + for k, v in t do + key = k + end + )"); + + LUAU_REQUIRE_ERROR_COUNT(0, result); + CHECK_EQ(*typeChecker.numberType, *requireType("key")); +} + +TEST_CASE_FIXTURE(Fixture, "loop_iter_trailing_nil") +{ + ScopedFastFlag sff{"LuauTypecheckIter", true}; + + CheckResult result = check(R"( + local t: {string} = {} + local extra + for k, v, e in t do + extra = e + end + )"); + + LUAU_REQUIRE_ERROR_COUNT(0, result); + CHECK_EQ(*typeChecker.nilType, *requireType("extra")); +} + +TEST_CASE_FIXTURE(Fixture, "loop_iter_no_indexer") +{ + ScopedFastFlag sff{"LuauTypecheckIter", true}; + + CheckResult result = check(R"( + local t = {} + for k, v in t do + end + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + GenericError* ge = get(result.errors[0]); + REQUIRE(ge); + CHECK_EQ("Cannot iterate over a table without indexer", ge->message); +} + +TEST_CASE_FIXTURE(Fixture, "loop_iter_iter_metamethod") +{ + ScopedFastFlag sff{"LuauTypecheckIter", true}; + + CheckResult result = check(R"( + local t = {} + setmetatable(t, { __iter = function(o) return next, o.children end }) + for k: number, v: string in t do + end + )"); + + LUAU_REQUIRE_ERROR_COUNT(0, result); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.modules.test.cpp b/tests/TypeInfer.modules.test.cpp index fa1f519c..b6f49f9f 100644 --- a/tests/TypeInfer.modules.test.cpp +++ b/tests/TypeInfer.modules.test.cpp @@ -5,7 +5,6 @@ #include "Luau/Scope.h" #include "Luau/TypeInfer.h" #include "Luau/TypeVar.h" -#include "Luau/VisitTypeVar.h" #include "Fixture.h" diff --git a/tests/TypeInfer.tables.test.cpp b/tests/TypeInfer.tables.test.cpp index 5bd522a3..8e535995 100644 --- a/tests/TypeInfer.tables.test.cpp +++ b/tests/TypeInfer.tables.test.cpp @@ -2331,7 +2331,7 @@ TEST_CASE_FIXTURE(Fixture, "confusing_indexing") TEST_CASE_FIXTURE(Fixture, "pass_a_union_of_tables_to_a_function_that_requires_a_table") { - ScopedFastFlag sff{"LuauDifferentOrderOfUnificationDoesntMatter", true}; + ScopedFastFlag sff{"LuauDifferentOrderOfUnificationDoesntMatter2", true}; CheckResult result = check(R"( local a: {x: number, y: number, [any]: any} | {y: number} @@ -2351,7 +2351,7 @@ TEST_CASE_FIXTURE(Fixture, "pass_a_union_of_tables_to_a_function_that_requires_a TEST_CASE_FIXTURE(Fixture, "pass_a_union_of_tables_to_a_function_that_requires_a_table_2") { - ScopedFastFlag sff{"LuauDifferentOrderOfUnificationDoesntMatter", true}; + ScopedFastFlag sff{"LuauDifferentOrderOfUnificationDoesntMatter2", true}; CheckResult result = check(R"( local a: {y: number} | {x: number, y: number, [any]: any} diff --git a/tests/TypeInfer.test.cpp b/tests/TypeInfer.test.cpp index a578b1cf..e81ef1a9 100644 --- a/tests/TypeInfer.test.cpp +++ b/tests/TypeInfer.test.cpp @@ -1034,4 +1034,45 @@ TEST_CASE_FIXTURE(Fixture, "follow_on_new_types_in_substitution") LUAU_REQUIRE_NO_ERRORS(result); } +/** + * The problem we had here was that the type of q in B.h was initially inferring to {} | {prop: free} before we bound + * that second table to the enclosing union. + */ +TEST_CASE_FIXTURE(Fixture, "do_not_bind_a_free_table_to_a_union_containing_that_table") +{ + ScopedFastFlag flag[] = { + {"LuauStatFunctionSimplify4", true}, + {"LuauLowerBoundsCalculation", true}, + {"LuauDifferentOrderOfUnificationDoesntMatter2", true}, + }; + + CheckResult result = check(R"( + --!strict + + local A = {} + + function A:f() + local t = {} + + for key, value in pairs(self) do + t[key] = value + end + + return t + end + + local B = A:f() + + function B.g(t) + assert(type(t) == "table") + assert(t.prop ~= nil) + end + + function B.h(q) + q = q or {} + return q or {} + end + )"); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.tryUnify.test.cpp b/tests/TypeInfer.tryUnify.test.cpp index b6e93265..87562644 100644 --- a/tests/TypeInfer.tryUnify.test.cpp +++ b/tests/TypeInfer.tryUnify.test.cpp @@ -242,8 +242,6 @@ TEST_CASE_FIXTURE(TryUnifyFixture, "cli_50320_follow_in_any_unification") TEST_CASE_FIXTURE(TryUnifyFixture, "txnlog_preserves_type_owner") { - ScopedFastFlag luauTxnLogPreserveOwner{"LuauTxnLogPreserveOwner", true}; - TypeId a = arena.addType(TypeVar{FreeTypeVar{TypeLevel{}}}); TypeId b = typeChecker.numberType; @@ -255,8 +253,6 @@ TEST_CASE_FIXTURE(TryUnifyFixture, "txnlog_preserves_type_owner") TEST_CASE_FIXTURE(TryUnifyFixture, "txnlog_preserves_pack_owner") { - ScopedFastFlag luauTxnLogPreserveOwner{"LuauTxnLogPreserveOwner", true}; - TypePackId a = arena.addTypePack(TypePackVar{FreeTypePack{TypeLevel{}}}); TypePackId b = typeChecker.anyTypePack; diff --git a/tests/TypeVar.test.cpp b/tests/TypeVar.test.cpp index e033fe22..a45af39c 100644 --- a/tests/TypeVar.test.cpp +++ b/tests/TypeVar.test.cpp @@ -313,23 +313,33 @@ TEST_CASE("tagging_props") CHECK(Luau::hasTag(prop, "foo")); } -struct VisitCountTracker +struct VisitCountTracker final : TypeVarOnceVisitor { std::unordered_map tyVisits; std::unordered_map tpVisits; - void cycle(TypeId) {} - void cycle(TypePackId) {} + void cycle(TypeId) override {} + void cycle(TypePackId) override {} template bool operator()(TypeId ty, const T& t) + { + return visit(ty); + } + + template + bool operator()(TypePackId tp, const T&) + { + return visit(tp); + } + + bool visit(TypeId ty) override { tyVisits[ty]++; return true; } - template - bool operator()(TypePackId tp, const T&) + bool visit(TypePackId tp) override { tpVisits[tp]++; return true; @@ -348,7 +358,7 @@ local b: (T, T, T) -> T VisitCountTracker tester; DenseHashSet seen{nullptr}; - visitTypeVarOnce(bType, tester, seen); + DEPRECATED_visitTypeVarOnce(bType, tester, seen); for (auto [_, count] : tester.tyVisits) CHECK_EQ(count, 1); diff --git a/tests/VisitTypeVar.test.cpp b/tests/VisitTypeVar.test.cpp new file mode 100644 index 00000000..3d426f10 --- /dev/null +++ b/tests/VisitTypeVar.test.cpp @@ -0,0 +1,48 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details + +#include "Fixture.h" + +#include "Luau/RecursionCounter.h" + +#include "doctest.h" + +using namespace Luau; + +LUAU_FASTFLAG(LuauUseVisitRecursionLimit) +LUAU_FASTINT(LuauVisitRecursionLimit) + +struct VisitTypeVarFixture : Fixture +{ + ScopedFastFlag flag1 = {"LuauUseVisitRecursionLimit", true}; + ScopedFastFlag flag2 = {"LuauRecursionLimitException", true}; +}; + +TEST_SUITE_BEGIN("VisitTypeVar"); + +TEST_CASE_FIXTURE(VisitTypeVarFixture, "throw_when_limit_is_exceeded") +{ + ScopedFastInt sfi{"LuauVisitRecursionLimit", 3}; + + CheckResult result = check(R"( + local t : {a: {b: {c: {d: {e: boolean}}}}} + )"); + + TypeId tType = requireType("t"); + + CHECK_THROWS_AS(toString(tType), RecursionLimitException); +} + +TEST_CASE_FIXTURE(VisitTypeVarFixture, "dont_throw_when_limit_is_high_enough") +{ + ScopedFastInt sfi{"LuauVisitRecursionLimit", 8}; + + CheckResult result = check(R"( + local t : {a: {b: {c: {d: {e: boolean}}}}} + )"); + + TypeId tType = requireType("t"); + + (void)toString(tType); +} + +TEST_SUITE_END(); diff --git a/tests/conformance/iter.lua b/tests/conformance/iter.lua new file mode 100644 index 00000000..468ffafb --- /dev/null +++ b/tests/conformance/iter.lua @@ -0,0 +1,196 @@ +-- This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +-- This file is based on Lua 5.x tests -- https://github.com/lua/lua/tree/master/testes +print('testing iteration') + +-- basic for loop tests +do + local a + for a,b in pairs{} do error("not here") end + for i=1,0 do error("not here") end + for i=0,1,-1 do error("not here") end + a = nil; for i=1,1 do assert(not a); a=1 end; assert(a) + a = nil; for i=1,1,-1 do assert(not a); a=1 end; assert(a) + a = 0; for i=0, 1, 0.1 do a=a+1 end; assert(a==11) +end + +-- precision tests for for loops +do + local a + --a = 0; for i=1, 0, -0.01 do a=a+1 end; assert(a==101) + a = 0; for i=0, 0.999999999, 0.1 do a=a+1 end; assert(a==10) + a = 0; for i=1, 1, 1 do a=a+1 end; assert(a==1) + a = 0; for i=1e10, 1e10, -1 do a=a+1 end; assert(a==1) + a = 0; for i=1, 0.99999, 1 do a=a+1 end; assert(a==0) + a = 0; for i=99999, 1e5, -1 do a=a+1 end; assert(a==0) + a = 0; for i=1, 0.99999, -1 do a=a+1 end; assert(a==1) +end + +-- for loops do string->number coercion +do + local a = 0; for i="10","1","-2" do a=a+1 end; assert(a==5) +end + +-- generic for with function iterators +do + local function f (n, p) + local t = {}; for i=1,p do t[i] = i*10 end + return function (_,n) + if n > 0 then + n = n-1 + return n, unpack(t) + end + end, nil, n + end + + local x = 0 + for n,a,b,c,d in f(5,3) do + x = x+1 + assert(a == 10 and b == 20 and c == 30 and d == nil) + end + assert(x == 5) +end + +-- generic for with __call (tables) +do + local f = {} + setmetatable(f, { __call = function(_, _, n) if n > 0 then return n - 1 end end }) + + local x = 0 + for n in f, nil, 5 do + x += n + end + assert(x == 10) +end + +-- generic for with __call (userdata) +do + local f = newproxy(true) + getmetatable(f).__call = function(_, _, n) if n > 0 then return n - 1 end end + + local x = 0 + for n in f, nil, 5 do + x += n + end + assert(x == 10) +end + +-- generic for with pairs +do + local x = 0 + for k, v in pairs({a = 1, b = 2, c = 3}) do + x += v + end + assert(x == 6) +end + +-- generic for with pairs with holes +do + local x = 0 + for k, v in pairs({1, 2, 3, nil, 5}) do + x += v + end + assert(x == 11) +end + +-- generic for with ipairs +do + local x = 0 + for k, v in ipairs({1, 2, 3, nil, 5}) do + x += v + end + assert(x == 6) +end + +-- generic for with __iter (tables) +do + local f = {} + setmetatable(f, { __iter = function(x) + assert(f == x) + return next, {1, 2, 3, 4} + end }) + + local x = 0 + for n in f do + x += n + end + assert(x == 10) +end + +-- generic for with __iter (userdata) +do + local f = newproxy(true) + getmetatable(f).__iter = function(x) + assert(f == x) + return next, {1, 2, 3, 4} + end + + local x = 0 + for n in f do + x += n + end + assert(x == 10) +end + +-- generic for with tables (dictionary) +do + local x = 0 + for k, v in {a = 1, b = 2, c = 3} do + print(k, v) + x += v + end + assert(x == 6) +end + +-- generic for with tables (arrays) +do + local x = '' + for k, v in {1, 2, 3, nil, 5} do + x ..= tostring(v) + end + assert(x == "1235") +end + +-- generic for with tables (mixed) +do + local x = 0 + for k, v in {1, 2, 3, nil, 5, a = 1, b = 2, c = 3} do + x += v + end + assert(x == 17) +end + +-- generic for over a non-iterable object +do + local ok, err = pcall(function() for x in 42 do end end) + assert(not ok and err:match("attempt to iterate")) +end + +-- generic for over an iterable object that doesn't return a function +do + local obj = {} + setmetatable(obj, { __iter = function() end }) + + local ok, err = pcall(function() for x in obj do end end) + assert(not ok and err:match("attempt to call a nil value")) +end + +-- it's okay to iterate through a table with a single variable +do + local x = 0 + for k in {1, 2, 3, 4, 5} do + x += k + end + assert(x == 15) +end + +-- all extra variables should be set to nil during builtin traversal +do + local x = 0 + for k,v,a,b,c,d,e in {1, 2, 3, 4, 5} do + x += k + assert(a == nil and b == nil and c == nil and d == nil and e == nil) + end + assert(x == 15) +end + +return"OK" diff --git a/tests/conformance/nextvar.lua b/tests/conformance/nextvar.lua index c8176456..0dba8fa6 100644 --- a/tests/conformance/nextvar.lua +++ b/tests/conformance/nextvar.lua @@ -368,48 +368,6 @@ assert(next(a,nil) == 1000 and next(a,1000) == nil) assert(next({}) == nil) assert(next({}, nil) == nil) -for a,b in pairs{} do error("not here") end -for i=1,0 do error("not here") end -for i=0,1,-1 do error("not here") end -a = nil; for i=1,1 do assert(not a); a=1 end; assert(a) -a = nil; for i=1,1,-1 do assert(not a); a=1 end; assert(a) - -a = 0; for i=0, 1, 0.1 do a=a+1 end; assert(a==11) --- precision problems ---a = 0; for i=1, 0, -0.01 do a=a+1 end; assert(a==101) -a = 0; for i=0, 0.999999999, 0.1 do a=a+1 end; assert(a==10) -a = 0; for i=1, 1, 1 do a=a+1 end; assert(a==1) -a = 0; for i=1e10, 1e10, -1 do a=a+1 end; assert(a==1) -a = 0; for i=1, 0.99999, 1 do a=a+1 end; assert(a==0) -a = 0; for i=99999, 1e5, -1 do a=a+1 end; assert(a==0) -a = 0; for i=1, 0.99999, -1 do a=a+1 end; assert(a==1) - --- conversion -a = 0; for i="10","1","-2" do a=a+1 end; assert(a==5) - - -collectgarbage() - - --- testing generic 'for' - -local function f (n, p) - local t = {}; for i=1,p do t[i] = i*10 end - return function (_,n) - if n > 0 then - n = n-1 - return n, unpack(t) - end - end, nil, n -end - -local x = 0 -for n,a,b,c,d in f(5,3) do - x = x+1 - assert(a == 10 and b == 20 and c == 30 and d == nil) -end -assert(x == 5) - -- testing table.create and table.find do local t = table.create(5) @@ -596,4 +554,17 @@ do assert(#t2 == 6) end +-- test table.unpack fastcall for rejecting large unpacks +do + local ok, res = pcall(function() + local a = table.create(7999, 0) + local b = table.create(8000, 0) + + local at = { table.unpack(a) } + local bt = { table.unpack(b) } + end) + + assert(not ok) +end + return"OK" diff --git a/tools/lldb_formatters.py b/tools/lldb_formatters.py index b3d2b4f5..ff610d09 100644 --- a/tools/lldb_formatters.py +++ b/tools/lldb_formatters.py @@ -97,7 +97,7 @@ class LuauVariantSyntheticChildrenProvider: if self.current_type: storage = self.valobj.GetChildMemberWithName("storage") - self.stored_value = storage.Cast(self.current_type.GetPointerType()).Dereference() + self.stored_value = storage.Cast(self.current_type) else: self.stored_value = None else: From e9cc76a3d5278533377eeff5b1de27a2b0f800e4 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Thu, 5 May 2022 17:03:43 -0700 Subject: [PATCH 239/399] Sync to upstream/release/526 (#477) --- Analysis/include/Luau/Frontend.h | 1 - Analysis/include/Luau/VisitTypeVar.h | 319 +++++++- Analysis/src/Autocomplete.cpp | 54 +- Analysis/src/Frontend.cpp | 34 +- Analysis/src/Normalize.cpp | 329 ++++++-- Analysis/src/Quantify.cpp | 50 +- Analysis/src/ToString.cpp | 50 +- Analysis/src/TxnLog.cpp | 34 +- Analysis/src/TypeInfer.cpp | 125 ++-- Analysis/src/Unifier.cpp | 134 +++- Ast/include/Luau/Ast.h | 2 +- Ast/src/Parser.cpp | 5 +- Compiler/include/Luau/Bytecode.h | 5 + Compiler/src/BytecodeBuilder.cpp | 10 + Compiler/src/Compiler.cpp | 338 ++++++++- Compiler/src/ConstantFolding.cpp | 53 +- Sources.cmake | 1 + VM/src/lapi.cpp | 2 +- VM/src/lbuiltins.cpp | 4 +- VM/src/lgc.h | 2 +- VM/src/ltable.cpp | 48 +- VM/src/ltm.cpp | 4 +- VM/src/ltm.h | 3 +- VM/src/lvmexecute.cpp | 196 ++++- .../test_LargeTableSum_loop_iter.lua | 17 + bench/tests/sunspider/3d-cube.lua | 30 +- bench/tests/sunspider/3d-morph.lua | 2 +- bench/tests/sunspider/3d-raytrace.lua | 44 +- bench/tests/sunspider/access-binary-trees.lua | 69 -- .../tests/sunspider/controlflow-recursive.lua | 8 +- bench/tests/sunspider/crypto-aes.lua | 148 ++-- bench/tests/sunspider/math-cordic.lua | 10 +- bench/tests/sunspider/math-partial-sums.lua | 2 +- bench/tests/sunspider/math-spectral-norm.lua | 72 -- tests/Autocomplete.test.cpp | 8 - tests/Compiler.test.cpp | 708 +++++++++++++++++- tests/Conformance.test.cpp | 12 + tests/Frontend.test.cpp | 4 - tests/Parser.test.cpp | 4 - tests/RuntimeLimits.test.cpp | 4 +- tests/TypeInfer.loops.test.cpp | 67 ++ tests/TypeInfer.modules.test.cpp | 1 - tests/TypeInfer.tables.test.cpp | 4 +- tests/TypeInfer.test.cpp | 41 + tests/TypeInfer.tryUnify.test.cpp | 4 - tests/TypeVar.test.cpp | 22 +- tests/VisitTypeVar.test.cpp | 48 ++ tests/conformance/iter.lua | 196 +++++ tests/conformance/nextvar.lua | 55 +- tools/lldb_formatters.py | 2 +- 50 files changed, 2658 insertions(+), 727 deletions(-) create mode 100644 bench/micro_tests/test_LargeTableSum_loop_iter.lua delete mode 100644 bench/tests/sunspider/access-binary-trees.lua delete mode 100644 bench/tests/sunspider/math-spectral-norm.lua create mode 100644 tests/VisitTypeVar.test.cpp create mode 100644 tests/conformance/iter.lua diff --git a/Analysis/include/Luau/Frontend.h b/Analysis/include/Luau/Frontend.h index 59125470..37e3cfdc 100644 --- a/Analysis/include/Luau/Frontend.h +++ b/Analysis/include/Luau/Frontend.h @@ -145,7 +145,6 @@ struct Frontend */ std::pair lintFragment(std::string_view source, std::optional enabledLintWarnings = {}); - CheckResult check(const SourceModule& module); // OLD. TODO KILL LintResult lint(const SourceModule& module, std::optional enabledLintWarnings = {}); bool isDirty(const ModuleName& name, bool forAutocomplete = false) const; diff --git a/Analysis/include/Luau/VisitTypeVar.h b/Analysis/include/Luau/VisitTypeVar.h index 045190ea..67fce5ed 100644 --- a/Analysis/include/Luau/VisitTypeVar.h +++ b/Analysis/include/Luau/VisitTypeVar.h @@ -1,9 +1,15 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #pragma once +#include + #include "Luau/DenseHash.h" -#include "Luau/TypeVar.h" +#include "Luau/RecursionCounter.h" #include "Luau/TypePack.h" +#include "Luau/TypeVar.h" + +LUAU_FASTFLAG(LuauUseVisitRecursionLimit) +LUAU_FASTINT(LuauVisitRecursionLimit) namespace Luau { @@ -219,24 +225,321 @@ void visit(TypePackId tp, F& f, Set& seen) } // namespace visit_detail +template +struct GenericTypeVarVisitor +{ + using Set = S; + + Set seen; + int recursionCounter = 0; + + GenericTypeVarVisitor() = default; + + explicit GenericTypeVarVisitor(Set seen) + : seen(std::move(seen)) + { + } + + virtual void cycle(TypeId) {} + virtual void cycle(TypePackId) {} + + virtual bool visit(TypeId ty) + { + return true; + } + virtual bool visit(TypeId ty, const BoundTypeVar& btv) + { + return visit(ty); + } + virtual bool visit(TypeId ty, const FreeTypeVar& ftv) + { + return visit(ty); + } + virtual bool visit(TypeId ty, const GenericTypeVar& gtv) + { + return visit(ty); + } + virtual bool visit(TypeId ty, const ErrorTypeVar& etv) + { + return visit(ty); + } + virtual bool visit(TypeId ty, const ConstrainedTypeVar& ctv) + { + return visit(ty); + } + virtual bool visit(TypeId ty, const PrimitiveTypeVar& ptv) + { + return visit(ty); + } + virtual bool visit(TypeId ty, const FunctionTypeVar& ftv) + { + return visit(ty); + } + virtual bool visit(TypeId ty, const TableTypeVar& ttv) + { + return visit(ty); + } + virtual bool visit(TypeId ty, const MetatableTypeVar& mtv) + { + return visit(ty); + } + virtual bool visit(TypeId ty, const ClassTypeVar& ctv) + { + return visit(ty); + } + virtual bool visit(TypeId ty, const AnyTypeVar& atv) + { + return visit(ty); + } + virtual bool visit(TypeId ty, const UnionTypeVar& utv) + { + return visit(ty); + } + virtual bool visit(TypeId ty, const IntersectionTypeVar& itv) + { + return visit(ty); + } + + virtual bool visit(TypePackId tp) + { + return true; + } + virtual bool visit(TypePackId tp, const BoundTypePack& btp) + { + return visit(tp); + } + virtual bool visit(TypePackId tp, const FreeTypePack& ftp) + { + return visit(tp); + } + virtual bool visit(TypePackId tp, const GenericTypePack& gtp) + { + return visit(tp); + } + virtual bool visit(TypePackId tp, const Unifiable::Error& etp) + { + return visit(tp); + } + virtual bool visit(TypePackId tp, const TypePack& pack) + { + return visit(tp); + } + virtual bool visit(TypePackId tp, const VariadicTypePack& vtp) + { + return visit(tp); + } + + void traverse(TypeId ty) + { + RecursionLimiter limiter{&recursionCounter, FInt::LuauVisitRecursionLimit, "TypeVarVisitor"}; + + if (visit_detail::hasSeen(seen, ty)) + { + cycle(ty); + return; + } + + if (auto btv = get(ty)) + { + if (visit(ty, *btv)) + traverse(btv->boundTo); + } + + else if (auto ftv = get(ty)) + visit(ty, *ftv); + + else if (auto gtv = get(ty)) + visit(ty, *gtv); + + else if (auto etv = get(ty)) + visit(ty, *etv); + + else if (auto ctv = get(ty)) + { + if (visit(ty, *ctv)) + { + for (TypeId part : ctv->parts) + traverse(part); + } + } + + else if (auto ptv = get(ty)) + visit(ty, *ptv); + + else if (auto ftv = get(ty)) + { + if (visit(ty, *ftv)) + { + traverse(ftv->argTypes); + traverse(ftv->retType); + } + } + + else if (auto ttv = get(ty)) + { + // Some visitors want to see bound tables, that's why we traverse the original type + if (visit(ty, *ttv)) + { + if (ttv->boundTo) + { + traverse(*ttv->boundTo); + } + else + { + for (auto& [_name, prop] : ttv->props) + traverse(prop.type); + + if (ttv->indexer) + { + traverse(ttv->indexer->indexType); + traverse(ttv->indexer->indexResultType); + } + } + } + } + + else if (auto mtv = get(ty)) + { + if (visit(ty, *mtv)) + { + traverse(mtv->table); + traverse(mtv->metatable); + } + } + + else if (auto ctv = get(ty)) + { + if (visit(ty, *ctv)) + { + for (const auto& [name, prop] : ctv->props) + traverse(prop.type); + + if (ctv->parent) + traverse(*ctv->parent); + + if (ctv->metatable) + traverse(*ctv->metatable); + } + } + + else if (auto atv = get(ty)) + visit(ty, *atv); + + else if (auto utv = get(ty)) + { + if (visit(ty, *utv)) + { + for (TypeId optTy : utv->options) + traverse(optTy); + } + } + + else if (auto itv = get(ty)) + { + if (visit(ty, *itv)) + { + for (TypeId partTy : itv->parts) + traverse(partTy); + } + } + + visit_detail::unsee(seen, ty); + } + + void traverse(TypePackId tp) + { + if (visit_detail::hasSeen(seen, tp)) + { + cycle(tp); + return; + } + + if (auto btv = get(tp)) + { + if (visit(tp, *btv)) + traverse(btv->boundTo); + } + + else if (auto ftv = get(tp)) + visit(tp, *ftv); + + else if (auto gtv = get(tp)) + visit(tp, *gtv); + + else if (auto etv = get(tp)) + visit(tp, *etv); + + else if (auto pack = get(tp)) + { + visit(tp, *pack); + + for (TypeId ty : pack->head) + traverse(ty); + + if (pack->tail) + traverse(*pack->tail); + } + else if (auto pack = get(tp)) + { + visit(tp, *pack); + traverse(pack->ty); + } + else + LUAU_ASSERT(!"GenericTypeVarVisitor::traverse(TypePackId) is not exhaustive!"); + + visit_detail::unsee(seen, tp); + } +}; + +/** Visit each type under a given type. Skips over cycles and keeps recursion depth under control. + * + * The same type may be visited multiple times if there are multiple distinct paths to it. If this is undesirable, use + * TypeVarOnceVisitor. + */ +struct TypeVarVisitor : GenericTypeVarVisitor> +{ +}; + +/// Visit each type under a given type. Each type will only be checked once even if there are multiple paths to it. +struct TypeVarOnceVisitor : GenericTypeVarVisitor> +{ + TypeVarOnceVisitor() + : GenericTypeVarVisitor{DenseHashSet{nullptr}} + { + } +}; + +// Clip with FFlagLuauUseVisitRecursionLimit template -void visitTypeVar(TID ty, F& f, std::unordered_set& seen) +void DEPRECATED_visitTypeVar(TID ty, F& f, std::unordered_set& seen) { visit_detail::visit(ty, f, seen); } +// Delete and inline when clipping FFlagLuauUseVisitRecursionLimit template -void visitTypeVar(TID ty, F& f) +void DEPRECATED_visitTypeVar(TID ty, F& f) { - std::unordered_set seen; - visit_detail::visit(ty, f, seen); + if (FFlag::LuauUseVisitRecursionLimit) + f.traverse(ty); + else + { + std::unordered_set seen; + visit_detail::visit(ty, f, seen); + } } +// Delete and inline when clipping FFlagLuauUseVisitRecursionLimit template -void visitTypeVarOnce(TID ty, F& f, DenseHashSet& seen) +void DEPRECATED_visitTypeVarOnce(TID ty, F& f, DenseHashSet& seen) { - seen.clear(); - visit_detail::visit(ty, f, seen); + if (FFlag::LuauUseVisitRecursionLimit) + f.traverse(ty); + else + { + seen.clear(); + visit_detail::visit(ty, f, seen); + } } } // namespace Luau diff --git a/Analysis/src/Autocomplete.cpp b/Analysis/src/Autocomplete.cpp index dec12d01..19d06cfc 100644 --- a/Analysis/src/Autocomplete.cpp +++ b/Analysis/src/Autocomplete.cpp @@ -14,7 +14,6 @@ #include LUAU_FASTFLAGVARIABLE(LuauIfElseExprFixCompletionIssue, false); -LUAU_FASTFLAGVARIABLE(LuauAutocompleteSingletonTypes, false); LUAU_FASTFLAGVARIABLE(LuauFixAutocompleteClassSecurityLevel, false); LUAU_FASTFLAG(LuauSelfCallAutocompleteFix) @@ -1341,38 +1340,21 @@ static void autocompleteExpression(const SourceModule& sourceModule, const Modul scope = scope->parent; } - if (FFlag::LuauAutocompleteSingletonTypes) - { - TypeCorrectKind correctForNil = checkTypeCorrectKind(module, typeArena, node, position, typeChecker.nilType); - TypeCorrectKind correctForTrue = checkTypeCorrectKind(module, typeArena, node, position, getSingletonTypes().trueType); - TypeCorrectKind correctForFalse = checkTypeCorrectKind(module, typeArena, node, position, getSingletonTypes().falseType); - TypeCorrectKind correctForFunction = - functionIsExpectedAt(module, node, position).value_or(false) ? TypeCorrectKind::Correct : TypeCorrectKind::None; + TypeCorrectKind correctForNil = checkTypeCorrectKind(module, typeArena, node, position, typeChecker.nilType); + TypeCorrectKind correctForTrue = checkTypeCorrectKind(module, typeArena, node, position, getSingletonTypes().trueType); + TypeCorrectKind correctForFalse = checkTypeCorrectKind(module, typeArena, node, position, getSingletonTypes().falseType); + TypeCorrectKind correctForFunction = + functionIsExpectedAt(module, node, position).value_or(false) ? TypeCorrectKind::Correct : TypeCorrectKind::None; - result["if"] = {AutocompleteEntryKind::Keyword, std::nullopt, false, false}; - result["true"] = {AutocompleteEntryKind::Keyword, typeChecker.booleanType, false, false, correctForTrue}; - result["false"] = {AutocompleteEntryKind::Keyword, typeChecker.booleanType, false, false, correctForFalse}; - result["nil"] = {AutocompleteEntryKind::Keyword, typeChecker.nilType, false, false, correctForNil}; - result["not"] = {AutocompleteEntryKind::Keyword}; - result["function"] = {AutocompleteEntryKind::Keyword, std::nullopt, false, false, correctForFunction}; + result["if"] = {AutocompleteEntryKind::Keyword, std::nullopt, false, false}; + result["true"] = {AutocompleteEntryKind::Keyword, typeChecker.booleanType, false, false, correctForTrue}; + result["false"] = {AutocompleteEntryKind::Keyword, typeChecker.booleanType, false, false, correctForFalse}; + result["nil"] = {AutocompleteEntryKind::Keyword, typeChecker.nilType, false, false, correctForNil}; + result["not"] = {AutocompleteEntryKind::Keyword}; + result["function"] = {AutocompleteEntryKind::Keyword, std::nullopt, false, false, correctForFunction}; - if (auto ty = findExpectedTypeAt(module, node, position)) - autocompleteStringSingleton(*ty, true, result); - } - else - { - TypeCorrectKind correctForNil = checkTypeCorrectKind(module, typeArena, node, position, typeChecker.nilType); - TypeCorrectKind correctForBoolean = checkTypeCorrectKind(module, typeArena, node, position, typeChecker.booleanType); - TypeCorrectKind correctForFunction = - functionIsExpectedAt(module, node, position).value_or(false) ? TypeCorrectKind::Correct : TypeCorrectKind::None; - - result["if"] = {AutocompleteEntryKind::Keyword, std::nullopt, false, false}; - result["true"] = {AutocompleteEntryKind::Keyword, typeChecker.booleanType, false, false, correctForBoolean}; - result["false"] = {AutocompleteEntryKind::Keyword, typeChecker.booleanType, false, false, correctForBoolean}; - result["nil"] = {AutocompleteEntryKind::Keyword, typeChecker.nilType, false, false, correctForNil}; - result["not"] = {AutocompleteEntryKind::Keyword}; - result["function"] = {AutocompleteEntryKind::Keyword, std::nullopt, false, false, correctForFunction}; - } + if (auto ty = findExpectedTypeAt(module, node, position)) + autocompleteStringSingleton(*ty, true, result); } } @@ -1680,11 +1662,8 @@ static AutocompleteResult autocomplete(const SourceModule& sourceModule, const M { AutocompleteEntryMap result; - if (FFlag::LuauAutocompleteSingletonTypes) - { - if (auto it = module->astExpectedTypes.find(node->asExpr())) - autocompleteStringSingleton(*it, false, result); - } + if (auto it = module->astExpectedTypes.find(node->asExpr())) + autocompleteStringSingleton(*it, false, result); if (finder.ancestry.size() >= 2) { @@ -1693,8 +1672,7 @@ static AutocompleteResult autocomplete(const SourceModule& sourceModule, const M if (auto it = module->astTypes.find(idxExpr->expr)) autocompleteProps(*module, typeArena, follow(*it), PropIndexType::Point, finder.ancestry, result); } - else if (auto binExpr = finder.ancestry.at(finder.ancestry.size() - 2)->as(); - binExpr && FFlag::LuauAutocompleteSingletonTypes) + else if (auto binExpr = finder.ancestry.at(finder.ancestry.size() - 2)->as()) { if (binExpr->op == AstExprBinary::CompareEq || binExpr->op == AstExprBinary::CompareNe) { diff --git a/Analysis/src/Frontend.cpp b/Analysis/src/Frontend.cpp index b8f7836d..56c0ac2c 100644 --- a/Analysis/src/Frontend.cpp +++ b/Analysis/src/Frontend.cpp @@ -18,7 +18,6 @@ LUAU_FASTINT(LuauTypeInferIterationLimit) LUAU_FASTINT(LuauTarjanChildLimit) -LUAU_FASTFLAG(LuauCyclicModuleTypeSurface) LUAU_FASTFLAG(LuauInferInNoCheckMode) LUAU_FASTFLAGVARIABLE(LuauKnowsTheDataModel3, false) LUAU_FASTFLAGVARIABLE(LuauSeparateTypechecks, false) @@ -433,8 +432,7 @@ CheckResult Frontend::check(const ModuleName& name, std::optional Frontend::lintFragment(std::string_view sour return {std::move(sourceModule), classifyLints(warnings, config)}; } -CheckResult Frontend::check(const SourceModule& module) -{ - LUAU_TIMETRACE_SCOPE("Frontend::check", "Frontend"); - LUAU_TIMETRACE_ARGUMENT("module", module.name.c_str()); - - const Config& config = configResolver->getConfig(module.name); - - Mode mode = module.mode.value_or(config.mode); - - double timestamp = getTimestamp(); - - ModulePtr checkedModule = typeChecker.check(module, mode); - - stats.timeCheck += getTimestamp() - timestamp; - stats.filesStrict += mode == Mode::Strict; - stats.filesNonstrict += mode == Mode::Nonstrict; - - if (checkedModule == nullptr) - throw std::runtime_error("Frontend::check produced a nullptr module for module " + module.name); - moduleResolver.modules[module.name] = checkedModule; - - return CheckResult{checkedModule->errors}; -} - LintResult Frontend::lint(const SourceModule& module, std::optional enabledLintWarnings) { LUAU_TIMETRACE_SCOPE("Frontend::lint", "Frontend"); diff --git a/Analysis/src/Normalize.cpp b/Analysis/src/Normalize.cpp index 043526ed..d8c11388 100644 --- a/Analysis/src/Normalize.cpp +++ b/Analysis/src/Normalize.cpp @@ -304,37 +304,23 @@ static bool areNormal(TypePackId tp, const std::unordered_set& seen, Inte ++iterationLimit; \ } while (false) -struct Normalize +struct Normalize final : TypeVarVisitor { + using TypeVarVisitor::Set; + + Normalize(TypeArena& arena, InternalErrorReporter& ice) + : arena(arena) + , ice(ice) + { + } + TypeArena& arena; InternalErrorReporter& ice; - // Debug data. Types being normalized are invalidated but trying to see what's going on is painful. - // To actually see the original type, read it by using the pointer of the type being normalized. - // e.g. in lldb, `e dump(originalTys[ty])`. - SeenTypes originalTys; - SeenTypePacks originalTps; - int iterationLimit = 0; bool limitExceeded = false; - template - bool operator()(TypePackId, const T&) - { - return true; - } - - template - void cycle(TID) - { - } - - bool operator()(TypeId ty, const FreeTypeVar&) - { - LUAU_ASSERT(!ty->normal); - return false; - } - + // TODO: Clip with FFlag::LuauUseVisitRecursionLimit bool operator()(TypeId ty, const BoundTypeVar& btv, std::unordered_set& seen) { // A type could be considered normal when it is in the stack, but we will eventually find out it is not normal as normalization progresses. @@ -349,27 +335,22 @@ struct Normalize return !ty->normal; } - bool operator()(TypeId ty, const PrimitiveTypeVar&) + bool operator()(TypeId ty, const FreeTypeVar& ftv) { - LUAU_ASSERT(ty->normal); - return false; + return visit(ty, ftv); } - - bool operator()(TypeId ty, const GenericTypeVar&) + bool operator()(TypeId ty, const PrimitiveTypeVar& ptv) { - if (!ty->normal) - asMutable(ty)->normal = true; - - return false; + return visit(ty, ptv); } - - bool operator()(TypeId ty, const ErrorTypeVar&) + bool operator()(TypeId ty, const GenericTypeVar& gtv) { - if (!ty->normal) - asMutable(ty)->normal = true; - return false; + return visit(ty, gtv); + } + bool operator()(TypeId ty, const ErrorTypeVar& etv) + { + return visit(ty, etv); } - bool operator()(TypeId ty, const ConstrainedTypeVar& ctvRef, std::unordered_set& seen) { CHECK_ITERATION_LIMIT(false); @@ -470,17 +451,12 @@ struct Normalize bool operator()(TypeId ty, const ClassTypeVar& ctv) { - if (!ty->normal) - asMutable(ty)->normal = true; - return false; + return visit(ty, ctv); } - - bool operator()(TypeId ty, const AnyTypeVar&) + bool operator()(TypeId ty, const AnyTypeVar& atv) { - LUAU_ASSERT(ty->normal); - return false; + return visit(ty, atv); } - bool operator()(TypeId ty, const UnionTypeVar& utvRef, std::unordered_set& seen) { CHECK_ITERATION_LIMIT(false); @@ -570,8 +546,257 @@ struct Normalize return false; } - bool operator()(TypeId ty, const LazyTypeVar&) + // TODO: Clip with FFlag::LuauUseVisitRecursionLimit + template + bool operator()(TypePackId, const T&) { + return true; + } + + // TODO: Clip with FFlag::LuauUseVisitRecursionLimit + template + void cycle(TID) + { + } + + bool visit(TypeId ty, const FreeTypeVar&) override + { + LUAU_ASSERT(!ty->normal); + return false; + } + + bool visit(TypeId ty, const BoundTypeVar& btv) override + { + // A type could be considered normal when it is in the stack, but we will eventually find out it is not normal as normalization progresses. + // So we need to avoid eagerly saying that this bound type is normal if the thing it is bound to is in the stack. + if (seen.find(asMutable(btv.boundTo)) != seen.end()) + return false; + + // It should never be the case that this TypeVar is normal, but is bound to a non-normal type, except in nontrivial cases. + LUAU_ASSERT(!ty->normal || ty->normal == btv.boundTo->normal); + + asMutable(ty)->normal = btv.boundTo->normal; + return !ty->normal; + } + + bool visit(TypeId ty, const PrimitiveTypeVar&) override + { + LUAU_ASSERT(ty->normal); + return false; + } + + bool visit(TypeId ty, const GenericTypeVar&) override + { + if (!ty->normal) + asMutable(ty)->normal = true; + + return false; + } + + bool visit(TypeId ty, const ErrorTypeVar&) override + { + if (!ty->normal) + asMutable(ty)->normal = true; + return false; + } + + bool visit(TypeId ty, const ConstrainedTypeVar& ctvRef) override + { + CHECK_ITERATION_LIMIT(false); + + ConstrainedTypeVar* ctv = const_cast(&ctvRef); + + std::vector parts = std::move(ctv->parts); + + // We might transmute, so it's not safe to rely on the builtin traversal logic of visitTypeVar + for (TypeId part : parts) + traverse(part); + + std::vector newParts = normalizeUnion(parts); + + const bool normal = areNormal(newParts, seen, ice); + + if (newParts.size() == 1) + *asMutable(ty) = BoundTypeVar{newParts[0]}; + else + *asMutable(ty) = UnionTypeVar{std::move(newParts)}; + + asMutable(ty)->normal = normal; + + return false; + } + + bool visit(TypeId ty, const FunctionTypeVar& ftv) override + { + CHECK_ITERATION_LIMIT(false); + + if (ty->normal) + return false; + + traverse(ftv.argTypes); + traverse(ftv.retType); + + asMutable(ty)->normal = areNormal(ftv.argTypes, seen, ice) && areNormal(ftv.retType, seen, ice); + + return false; + } + + bool visit(TypeId ty, const TableTypeVar& ttv) override + { + CHECK_ITERATION_LIMIT(false); + + if (ty->normal) + return false; + + bool normal = true; + + auto checkNormal = [&](TypeId t) { + // if t is on the stack, it is possible that this type is normal. + // If t is not normal and it is not on the stack, this type is definitely not normal. + if (!t->normal && seen.find(asMutable(t)) == seen.end()) + normal = false; + }; + + if (ttv.boundTo) + { + traverse(*ttv.boundTo); + asMutable(ty)->normal = (*ttv.boundTo)->normal; + return false; + } + + for (const auto& [_name, prop] : ttv.props) + { + traverse(prop.type); + checkNormal(prop.type); + } + + if (ttv.indexer) + { + traverse(ttv.indexer->indexType); + checkNormal(ttv.indexer->indexType); + traverse(ttv.indexer->indexResultType); + checkNormal(ttv.indexer->indexResultType); + } + + asMutable(ty)->normal = normal; + + return false; + } + + bool visit(TypeId ty, const MetatableTypeVar& mtv) override + { + CHECK_ITERATION_LIMIT(false); + + if (ty->normal) + return false; + + traverse(mtv.table); + traverse(mtv.metatable); + + asMutable(ty)->normal = mtv.table->normal && mtv.metatable->normal; + + return false; + } + + bool visit(TypeId ty, const ClassTypeVar& ctv) override + { + if (!ty->normal) + asMutable(ty)->normal = true; + return false; + } + + bool visit(TypeId ty, const AnyTypeVar&) override + { + LUAU_ASSERT(ty->normal); + return false; + } + + bool visit(TypeId ty, const UnionTypeVar& utvRef) override + { + CHECK_ITERATION_LIMIT(false); + + if (ty->normal) + return false; + + UnionTypeVar* utv = &const_cast(utvRef); + std::vector options = std::move(utv->options); + + // We might transmute, so it's not safe to rely on the builtin traversal logic of visitTypeVar + for (TypeId option : options) + traverse(option); + + std::vector newOptions = normalizeUnion(options); + + const bool normal = areNormal(newOptions, seen, ice); + + LUAU_ASSERT(!newOptions.empty()); + + if (newOptions.size() == 1) + *asMutable(ty) = BoundTypeVar{newOptions[0]}; + else + utv->options = std::move(newOptions); + + asMutable(ty)->normal = normal; + + return false; + } + + bool visit(TypeId ty, const IntersectionTypeVar& itvRef) override + { + CHECK_ITERATION_LIMIT(false); + + if (ty->normal) + return false; + + IntersectionTypeVar* itv = &const_cast(itvRef); + + std::vector oldParts = std::move(itv->parts); + + for (TypeId part : oldParts) + traverse(part); + + std::vector tables; + for (TypeId part : oldParts) + { + part = follow(part); + if (get(part)) + tables.push_back(part); + else + { + Replacer replacer{&arena, nullptr, nullptr}; // FIXME this is super super WEIRD + combineIntoIntersection(replacer, itv, part); + } + } + + // Don't allocate a new table if there's just one in the intersection. + if (tables.size() == 1) + itv->parts.push_back(tables[0]); + else if (!tables.empty()) + { + const TableTypeVar* first = get(tables[0]); + LUAU_ASSERT(first); + + TypeId newTable = arena.addType(TableTypeVar{first->state, first->level}); + TableTypeVar* ttv = getMutable(newTable); + for (TypeId part : tables) + { + // Intuition: If combineIntoTable() needs to clone a table, any references to 'part' are cyclic and need + // to be rewritten to point at 'newTable' in the clone. + Replacer replacer{&arena, part, newTable}; + combineIntoTable(replacer, ttv, part); + } + + itv->parts.push_back(newTable); + } + + asMutable(ty)->normal = areNormal(itv->parts, seen, ice); + + if (itv->parts.size() == 1) + { + TypeId part = itv->parts[0]; + *asMutable(ty) = BoundTypeVar{part}; + } + return false; } @@ -778,9 +1003,9 @@ std::pair normalize(TypeId ty, TypeArena& arena, InternalErrorRepo if (FFlag::DebugLuauCopyBeforeNormalizing) (void)clone(ty, arena, state); - Normalize n{arena, ice, std::move(state.seenTypes), std::move(state.seenTypePacks)}; + Normalize n{arena, ice}; std::unordered_set seen; - visitTypeVar(ty, n, seen); + DEPRECATED_visitTypeVar(ty, n, seen); return {ty, !n.limitExceeded}; } @@ -803,9 +1028,9 @@ std::pair normalize(TypePackId tp, TypeArena& arena, InternalE if (FFlag::DebugLuauCopyBeforeNormalizing) (void)clone(tp, arena, state); - Normalize n{arena, ice, std::move(state.seenTypes), std::move(state.seenTypePacks)}; + Normalize n{arena, ice}; std::unordered_set seen; - visitTypeVar(tp, n, seen); + DEPRECATED_visitTypeVar(tp, n, seen); return {tp, !n.limitExceeded}; } diff --git a/Analysis/src/Quantify.cpp b/Analysis/src/Quantify.cpp index 305f83ce..4f3e4469 100644 --- a/Analysis/src/Quantify.cpp +++ b/Analysis/src/Quantify.cpp @@ -9,7 +9,7 @@ LUAU_FASTFLAG(LuauTypecheckOptPass) namespace Luau { -struct Quantifier +struct Quantifier final : TypeVarOnceVisitor { TypeLevel level; std::vector generics; @@ -17,26 +17,17 @@ struct Quantifier bool seenGenericType = false; bool seenMutableType = false; - Quantifier(TypeLevel level) + explicit Quantifier(TypeLevel level) : level(level) { } - void cycle(TypeId) {} - void cycle(TypePackId) {} + void cycle(TypeId) override {} + void cycle(TypePackId) override {} bool operator()(TypeId ty, const FreeTypeVar& ftv) { - if (FFlag::LuauTypecheckOptPass) - seenMutableType = true; - - if (!level.subsumes(ftv.level)) - return false; - - *asMutable(ty) = GenericTypeVar{level}; - generics.push_back(ty); - - return false; + return visit(ty, ftv); } template @@ -56,8 +47,33 @@ struct Quantifier return true; } - bool operator()(TypeId ty, const TableTypeVar&) + bool operator()(TypeId ty, const TableTypeVar& ttv) { + return visit(ty, ttv); + } + + bool operator()(TypePackId tp, const FreeTypePack& ftp) + { + return visit(tp, ftp); + } + + bool visit(TypeId ty, const FreeTypeVar& ftv) override + { + if (FFlag::LuauTypecheckOptPass) + seenMutableType = true; + + if (!level.subsumes(ftv.level)) + return false; + + *asMutable(ty) = GenericTypeVar{level}; + generics.push_back(ty); + + return false; + } + + bool visit(TypeId ty, const TableTypeVar&) override + { + LUAU_ASSERT(getMutable(ty)); TableTypeVar& ttv = *getMutable(ty); if (FFlag::LuauTypecheckOptPass) @@ -93,7 +109,7 @@ struct Quantifier return true; } - bool operator()(TypePackId tp, const FreeTypePack& ftp) + bool visit(TypePackId tp, const FreeTypePack& ftp) override { if (FFlag::LuauTypecheckOptPass) seenMutableType = true; @@ -111,7 +127,7 @@ void quantify(TypeId ty, TypeLevel level) { Quantifier q{level}; DenseHashSet seen{nullptr}; - visitTypeVarOnce(ty, q, seen); + DEPRECATED_visitTypeVarOnce(ty, q, seen); FunctionTypeVar* ftv = getMutable(ty); LUAU_ASSERT(ftv); diff --git a/Analysis/src/ToString.cpp b/Analysis/src/ToString.cpp index 610842da..b5d6a550 100644 --- a/Analysis/src/ToString.cpp +++ b/Analysis/src/ToString.cpp @@ -26,7 +26,7 @@ namespace Luau namespace { -struct FindCyclicTypes +struct FindCyclicTypes final : TypeVarVisitor { FindCyclicTypes() = default; FindCyclicTypes(const FindCyclicTypes&) = delete; @@ -38,20 +38,22 @@ struct FindCyclicTypes std::set cycles; std::set cycleTPs; - void cycle(TypeId ty) + void cycle(TypeId ty) override { cycles.insert(ty); } - void cycle(TypePackId tp) + void cycle(TypePackId tp) override { cycleTPs.insert(tp); } + // TODO: Clip all the operator()s when we clip FFlagLuauUseVisitRecursionLimit + template bool operator()(TypeId ty, const T&) { - return visited.insert(ty).second; + return visit(ty); } bool operator()(TypeId ty, const TableTypeVar& ttv) = delete; @@ -64,10 +66,10 @@ struct FindCyclicTypes if (ttv.name || ttv.syntheticName) { for (TypeId itp : ttv.instantiatedTypeParams) - visitTypeVar(itp, *this, seen); + DEPRECATED_visitTypeVar(itp, *this, seen); for (TypePackId itp : ttv.instantiatedTypePackParams) - visitTypeVar(itp, *this, seen); + DEPRECATED_visitTypeVar(itp, *this, seen); return exhaustive; } @@ -82,9 +84,43 @@ struct FindCyclicTypes template bool operator()(TypePackId tp, const T&) + { + return visit(tp); + } + + bool visit(TypeId ty) override + { + return visited.insert(ty).second; + } + + bool visit(TypePackId tp) override { return visitedPacks.insert(tp).second; } + + bool visit(TypeId ty, const TableTypeVar& ttv) override + { + if (!visited.insert(ty).second) + return false; + + if (ttv.name || ttv.syntheticName) + { + for (TypeId itp : ttv.instantiatedTypeParams) + traverse(itp); + + for (TypePackId itp : ttv.instantiatedTypePackParams) + traverse(itp); + + return exhaustive; + } + + return true; + } + + bool visit(TypeId ty, const ClassTypeVar&) override + { + return false; + } }; template @@ -92,7 +128,7 @@ void findCyclicTypes(std::set& cycles, std::set& cycleTPs, T { FindCyclicTypes fct; fct.exhaustive = exhaustive; - visitTypeVar(ty, fct); + DEPRECATED_visitTypeVar(ty, fct); cycles = std::move(fct.cycles); cycleTPs = std::move(fct.cycleTPs); diff --git a/Analysis/src/TxnLog.cpp b/Analysis/src/TxnLog.cpp index a5f9d26c..1fb5a61a 100644 --- a/Analysis/src/TxnLog.cpp +++ b/Analysis/src/TxnLog.cpp @@ -7,7 +7,6 @@ #include #include -LUAU_FASTFLAGVARIABLE(LuauTxnLogPreserveOwner, false) LUAU_FASTFLAGVARIABLE(LuauJustOneCallFrameForHaveSeen, false) namespace Luau @@ -81,31 +80,20 @@ void TxnLog::concat(TxnLog rhs) void TxnLog::commit() { - if (FFlag::LuauTxnLogPreserveOwner) + for (auto& [ty, rep] : typeVarChanges) { - for (auto& [ty, rep] : typeVarChanges) - { - TypeArena* owningArena = ty->owningArena; - TypeVar* mtv = asMutable(ty); - *mtv = rep.get()->pending; - mtv->owningArena = owningArena; - } - - for (auto& [tp, rep] : typePackChanges) - { - TypeArena* owningArena = tp->owningArena; - TypePackVar* mpv = asMutable(tp); - *mpv = rep.get()->pending; - mpv->owningArena = owningArena; - } + TypeArena* owningArena = ty->owningArena; + TypeVar* mtv = asMutable(ty); + *mtv = rep.get()->pending; + mtv->owningArena = owningArena; } - else - { - for (auto& [ty, rep] : typeVarChanges) - *asMutable(ty) = rep.get()->pending; - for (auto& [tp, rep] : typePackChanges) - *asMutable(tp) = rep.get()->pending; + for (auto& [tp, rep] : typePackChanges) + { + TypeArena* owningArena = tp->owningArena; + TypePackVar* mpv = asMutable(tp); + *mpv = rep.get()->pending; + mpv->owningArena = owningArena; } clear(); diff --git a/Analysis/src/TypeInfer.cpp b/Analysis/src/TypeInfer.cpp index ba91ae1e..4466ede2 100644 --- a/Analysis/src/TypeInfer.cpp +++ b/Analysis/src/TypeInfer.cpp @@ -26,11 +26,11 @@ LUAU_FASTINTVARIABLE(LuauTypeInferRecursionLimit, 165) LUAU_FASTINTVARIABLE(LuauTypeInferIterationLimit, 20000) LUAU_FASTINTVARIABLE(LuauTypeInferTypePackLoopLimit, 5000) LUAU_FASTINTVARIABLE(LuauCheckRecursionLimit, 300) +LUAU_FASTFLAGVARIABLE(LuauUseVisitRecursionLimit, false) +LUAU_FASTINTVARIABLE(LuauVisitRecursionLimit, 500) LUAU_FASTFLAG(LuauKnowsTheDataModel3) LUAU_FASTFLAG(LuauSeparateTypechecks) LUAU_FASTFLAG(LuauAutocompleteDynamicLimits) -LUAU_FASTFLAG(LuauAutocompleteSingletonTypes) -LUAU_FASTFLAGVARIABLE(LuauCyclicModuleTypeSurface, false) LUAU_FASTFLAGVARIABLE(LuauDoNotRelyOnNextBinding, false) LUAU_FASTFLAGVARIABLE(LuauEqConstraint, false) LUAU_FASTFLAGVARIABLE(LuauWeakEqConstraint, false) // Eventually removed as false. @@ -40,6 +40,7 @@ LUAU_FASTFLAGVARIABLE(LuauInferStatFunction, false) LUAU_FASTFLAGVARIABLE(LuauInstantiateFollows, false) LUAU_FASTFLAGVARIABLE(LuauSelfCallAutocompleteFix, false) LUAU_FASTFLAGVARIABLE(LuauDiscriminableUnions2, false) +LUAU_FASTFLAGVARIABLE(LuauReduceUnionRecursion, false) LUAU_FASTFLAGVARIABLE(LuauOnlyMutateInstantiatedTables, false) LUAU_FASTFLAGVARIABLE(LuauStatFunctionSimplify4, false) LUAU_FASTFLAGVARIABLE(LuauTypecheckOptPass, false) @@ -57,6 +58,7 @@ LUAU_FASTFLAGVARIABLE(LuauTableUseCounterInstead, false) LUAU_FASTFLAGVARIABLE(LuauReturnTypeInferenceInNonstrict, false) LUAU_FASTFLAGVARIABLE(LuauRecursionLimitException, false); LUAU_FASTFLAG(LuauLosslessClone) +LUAU_FASTFLAGVARIABLE(LuauTypecheckIter, false); namespace Luau { @@ -1159,6 +1161,47 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatForIn& forin) iterTy = follow(instantiate(scope, checkExpr(scope, *firstValue).type, firstValue->location)); } + if (FFlag::LuauTypecheckIter) + { + if (std::optional iterMM = findMetatableEntry(iterTy, "__iter", firstValue->location)) + { + // if __iter metamethod is present, it will be called and the results are going to be called as if they are functions + // TODO: this needs to typecheck all returned values by __iter as if they were for loop arguments + // the structure of the function makes it difficult to do this especially since we don't have actual expressions, only types + for (TypeId var : varTypes) + unify(anyType, var, forin.location); + + return check(loopScope, *forin.body); + } + else if (const TableTypeVar* iterTable = get(iterTy)) + { + // TODO: note that this doesn't cleanly handle iteration over mixed tables and tables without an indexer + // this behavior is more or less consistent with what we do for pairs(), but really both are pretty wrong and need revisiting + if (iterTable->indexer) + { + if (varTypes.size() > 0) + unify(iterTable->indexer->indexType, varTypes[0], forin.location); + + if (varTypes.size() > 1) + unify(iterTable->indexer->indexResultType, varTypes[1], forin.location); + + for (size_t i = 2; i < varTypes.size(); ++i) + unify(nilType, varTypes[i], forin.location); + } + else + { + TypeId varTy = errorRecoveryType(loopScope); + + for (TypeId var : varTypes) + unify(varTy, var, forin.location); + + reportError(firstValue->location, GenericError{"Cannot iterate over a table without indexer"}); + } + + return check(loopScope, *forin.body); + } + } + const FunctionTypeVar* iterFunc = get(iterTy); if (!iterFunc) { @@ -2026,15 +2069,29 @@ std::vector TypeChecker::reduceUnion(const std::vector& types) if (const UnionTypeVar* utv = get(t)) { - std::vector r = reduceUnion(utv->options); - for (TypeId ty : r) + if (FFlag::LuauReduceUnionRecursion) { - ty = follow(ty); - if (get(ty) || get(ty)) - return {ty}; + for (TypeId ty : utv) + { + if (get(ty) || get(ty)) + return {ty}; - if (std::find(result.begin(), result.end(), ty) == result.end()) - result.push_back(ty); + if (result.end() == std::find(result.begin(), result.end(), ty)) + result.push_back(ty); + } + } + else + { + std::vector r = reduceUnion(utv->options); + for (TypeId ty : r) + { + ty = follow(ty); + if (get(ty) || get(ty)) + return {ty}; + + if (std::find(result.begin(), result.end(), ty) == result.end()) + result.push_back(ty); + } } } else if (std::find(result.begin(), result.end(), t) == result.end()) @@ -4372,17 +4429,12 @@ TypeId TypeChecker::checkRequire(const ScopePtr& scope, const ModuleInfo& module } // Types of requires that transitively refer to current module have to be replaced with 'any' - std::string humanReadableName; + std::string humanReadableName = resolver->getHumanReadableModuleName(moduleInfo.name); - if (FFlag::LuauCyclicModuleTypeSurface) + for (const auto& [location, path] : requireCycles) { - humanReadableName = resolver->getHumanReadableModuleName(moduleInfo.name); - - for (const auto& [location, path] : requireCycles) - { - if (!path.empty() && path.front() == humanReadableName) - return anyType; - } + if (!path.empty() && path.front() == humanReadableName) + return anyType; } ModulePtr module = resolver->getModule(moduleInfo.name); @@ -4392,32 +4444,14 @@ TypeId TypeChecker::checkRequire(const ScopePtr& scope, const ModuleInfo& module // either the file does not exist or there's a cycle. If there's a cycle // we will already have reported the error. if (!resolver->moduleExists(moduleInfo.name) && !moduleInfo.optional) - { - if (FFlag::LuauCyclicModuleTypeSurface) - { - reportError(TypeError{location, UnknownRequire{humanReadableName}}); - } - else - { - std::string reportedModulePath = resolver->getHumanReadableModuleName(moduleInfo.name); - reportError(TypeError{location, UnknownRequire{reportedModulePath}}); - } - } + reportError(TypeError{location, UnknownRequire{humanReadableName}}); return errorRecoveryType(scope); } if (module->type != SourceCode::Module) { - if (FFlag::LuauCyclicModuleTypeSurface) - { - reportError(location, IllegalRequire{humanReadableName, "Module is not a ModuleScript. It cannot be required."}); - } - else - { - std::string humanReadableName = resolver->getHumanReadableModuleName(moduleInfo.name); - reportError(location, IllegalRequire{humanReadableName, "Module is not a ModuleScript. It cannot be required."}); - } + reportError(location, IllegalRequire{humanReadableName, "Module is not a ModuleScript. It cannot be required."}); return errorRecoveryType(scope); } @@ -4429,15 +4463,7 @@ TypeId TypeChecker::checkRequire(const ScopePtr& scope, const ModuleInfo& module std::optional moduleType = first(modulePack); if (!moduleType) { - if (FFlag::LuauCyclicModuleTypeSurface) - { - reportError(location, IllegalRequire{humanReadableName, "Module does not return exactly 1 value. It cannot be required."}); - } - else - { - std::string humanReadableName = resolver->getHumanReadableModuleName(moduleInfo.name); - reportError(location, IllegalRequire{humanReadableName, "Module does not return exactly 1 value. It cannot be required."}); - } + reportError(location, IllegalRequire{humanReadableName, "Module does not return exactly 1 value. It cannot be required."}); return errorRecoveryType(scope); } @@ -4947,10 +4973,7 @@ TypeId TypeChecker::freshType(TypeLevel level) TypeId TypeChecker::singletonType(bool value) { - if (FFlag::LuauAutocompleteSingletonTypes) - return value ? getSingletonTypes().trueType : getSingletonTypes().falseType; - - return currentModule->internalTypes.addType(TypeVar(SingletonTypeVar(BooleanSingleton{value}))); + return value ? getSingletonTypes().trueType : getSingletonTypes().falseType; } TypeId TypeChecker::singletonType(std::string value) diff --git a/Analysis/src/Unifier.cpp b/Analysis/src/Unifier.cpp index 334806ce..f5c1dde9 100644 --- a/Analysis/src/Unifier.cpp +++ b/Analysis/src/Unifier.cpp @@ -22,7 +22,7 @@ LUAU_FASTFLAG(LuauLowerBoundsCalculation); LUAU_FASTFLAG(LuauErrorRecoveryType); LUAU_FASTFLAGVARIABLE(LuauSubtypingAddOptPropsToUnsealedTables, false) LUAU_FASTFLAGVARIABLE(LuauWidenIfSupertypeIsFree2, false) -LUAU_FASTFLAGVARIABLE(LuauDifferentOrderOfUnificationDoesntMatter, false) +LUAU_FASTFLAGVARIABLE(LuauDifferentOrderOfUnificationDoesntMatter2, false) LUAU_FASTFLAGVARIABLE(LuauTxnLogRefreshFunctionPointers, false) LUAU_FASTFLAG(LuauAnyInIsOptionalIsOptional) LUAU_FASTFLAG(LuauTypecheckOptPass) @@ -30,7 +30,7 @@ LUAU_FASTFLAG(LuauTypecheckOptPass) namespace Luau { -struct PromoteTypeLevels +struct PromoteTypeLevels final : TypeVarOnceVisitor { TxnLog& log; const TypeArena* typeArena = nullptr; @@ -53,13 +53,34 @@ struct PromoteTypeLevels } } + // TODO cycle and operator() need to be clipped when FFlagLuauUseVisitRecursionLimit is clipped template void cycle(TID) { } - template bool operator()(TID ty, const T&) + { + return visit(ty); + } + bool operator()(TypeId ty, const FreeTypeVar& ftv) + { + return visit(ty, ftv); + } + bool operator()(TypeId ty, const FunctionTypeVar& ftv) + { + return visit(ty, ftv); + } + bool operator()(TypeId ty, const TableTypeVar& ttv) + { + return visit(ty, ttv); + } + bool operator()(TypePackId tp, const FreeTypePack& ftp) + { + return visit(tp, ftp); + } + + bool visit(TypeId ty) override { // Type levels of types from other modules are already global, so we don't need to promote anything inside if (ty->owningArena != typeArena) @@ -68,7 +89,16 @@ struct PromoteTypeLevels return true; } - bool operator()(TypeId ty, const FreeTypeVar&) + bool visit(TypePackId tp) override + { + // Type levels of types from other modules are already global, so we don't need to promote anything inside + if (tp->owningArena != typeArena) + return false; + + return true; + } + + bool visit(TypeId ty, const FreeTypeVar&) override { // Surprise, it's actually a BoundTypeVar that hasn't been committed yet. // Calling getMutable on this will trigger an assertion. @@ -79,7 +109,7 @@ struct PromoteTypeLevels return true; } - bool operator()(TypeId ty, const FunctionTypeVar&) + bool visit(TypeId ty, const FunctionTypeVar&) override { // Type levels of types from other modules are already global, so we don't need to promote anything inside if (ty->owningArena != typeArena) @@ -89,7 +119,7 @@ struct PromoteTypeLevels return true; } - bool operator()(TypeId ty, const TableTypeVar& ttv) + bool visit(TypeId ty, const TableTypeVar& ttv) override { // Type levels of types from other modules are already global, so we don't need to promote anything inside if (ty->owningArena != typeArena) @@ -102,7 +132,7 @@ struct PromoteTypeLevels return true; } - bool operator()(TypePackId tp, const FreeTypePack&) + bool visit(TypePackId tp, const FreeTypePack&) override { // Surprise, it's actually a BoundTypePack that hasn't been committed yet. // Calling getMutable on this will trigger an assertion. @@ -122,7 +152,7 @@ static void promoteTypeLevels(TxnLog& log, const TypeArena* typeArena, TypeLevel PromoteTypeLevels ptl{log, typeArena, minLevel}; DenseHashSet seen{nullptr}; - visitTypeVarOnce(ty, ptl, seen); + DEPRECATED_visitTypeVarOnce(ty, ptl, seen); } void promoteTypeLevels(TxnLog& log, const TypeArena* typeArena, TypeLevel minLevel, TypePackId tp) @@ -133,10 +163,10 @@ void promoteTypeLevels(TxnLog& log, const TypeArena* typeArena, TypeLevel minLev PromoteTypeLevels ptl{log, typeArena, minLevel}; DenseHashSet seen{nullptr}; - visitTypeVarOnce(tp, ptl, seen); + DEPRECATED_visitTypeVarOnce(tp, ptl, seen); } -struct SkipCacheForType +struct SkipCacheForType final : TypeVarOnceVisitor { SkipCacheForType(const DenseHashMap& skipCacheForType, const TypeArena* typeArena) : skipCacheForType(skipCacheForType) @@ -144,28 +174,68 @@ struct SkipCacheForType { } - void cycle(TypeId) {} - void cycle(TypePackId) {} + // TODO cycle() and operator() can be clipped with FFlagLuauUseVisitRecursionLimit + void cycle(TypeId) override {} + void cycle(TypePackId) override {} bool operator()(TypeId ty, const FreeTypeVar& ftv) { - result = true; - return false; + return visit(ty, ftv); } - bool operator()(TypeId ty, const BoundTypeVar& btv) { - result = true; - return false; + return visit(ty, btv); + } + bool operator()(TypeId ty, const GenericTypeVar& gtv) + { + return visit(ty, gtv); + } + bool operator()(TypeId ty, const TableTypeVar& ttv) + { + return visit(ty, ttv); + } + bool operator()(TypePackId tp, const FreeTypePack& ftp) + { + return visit(tp, ftp); + } + bool operator()(TypePackId tp, const BoundTypePack& ftp) + { + return visit(tp, ftp); + } + bool operator()(TypePackId tp, const GenericTypePack& ftp) + { + return visit(tp, ftp); + } + template + bool operator()(TypeId ty, const T& t) + { + return visit(ty); + } + template + bool operator()(TypePackId tp, const T&) + { + return visit(tp); } - bool operator()(TypeId ty, const GenericTypeVar& btv) + bool visit(TypeId, const FreeTypeVar&) override { result = true; return false; } - bool operator()(TypeId ty, const TableTypeVar&) + bool visit(TypeId, const BoundTypeVar&) override + { + result = true; + return false; + } + + bool visit(TypeId, const GenericTypeVar&) override + { + result = true; + return false; + } + + bool visit(TypeId ty, const TableTypeVar&) override { // Types from other modules don't contain mutable elements and are ok to cache if (ty->owningArena != typeArena) @@ -188,8 +258,7 @@ struct SkipCacheForType return true; } - template - bool operator()(TypeId ty, const T& t) + bool visit(TypeId ty) override { // Types from other modules don't contain mutable elements and are ok to cache if (ty->owningArena != typeArena) @@ -206,8 +275,7 @@ struct SkipCacheForType return true; } - template - bool operator()(TypePackId tp, const T&) + bool visit(TypePackId tp) override { // Types from other modules don't contain mutable elements and are ok to cache if (tp->owningArena != typeArena) @@ -216,19 +284,19 @@ struct SkipCacheForType return true; } - bool operator()(TypePackId tp, const FreeTypePack& ftp) + bool visit(TypePackId tp, const FreeTypePack&) override { result = true; return false; } - bool operator()(TypePackId tp, const BoundTypePack& ftp) + bool visit(TypePackId tp, const BoundTypePack&) override { result = true; return false; } - bool operator()(TypePackId tp, const GenericTypePack& ftp) + bool visit(TypePackId tp, const GenericTypePack&) override { result = true; return false; @@ -578,7 +646,7 @@ void Unifier::tryUnifyUnionWithType(TypeId subTy, const UnionTypeVar* uv, TypeId failed = true; } - if (FFlag::LuauDifferentOrderOfUnificationDoesntMatter) + if (FFlag::LuauDifferentOrderOfUnificationDoesntMatter2) { } else @@ -593,7 +661,7 @@ void Unifier::tryUnifyUnionWithType(TypeId subTy, const UnionTypeVar* uv, TypeId } // even if A | B <: T fails, we want to bind some options of T with A | B iff A | B was a subtype of that option. - if (FFlag::LuauDifferentOrderOfUnificationDoesntMatter) + if (FFlag::LuauDifferentOrderOfUnificationDoesntMatter2) { auto tryBind = [this, subTy](TypeId superOption) { superOption = log.follow(superOption); @@ -603,6 +671,14 @@ void Unifier::tryUnifyUnionWithType(TypeId subTy, const UnionTypeVar* uv, TypeId if (!log.is(superOption) && (!ttv || ttv->state != TableState::Free)) return; + // If superOption is already present in subTy, do nothing. Nothing new has been learned, but the subtype + // test is successful. + if (auto subUnion = get(subTy)) + { + if (end(subUnion) != std::find(begin(subUnion), end(subUnion), superOption)) + return; + } + // Since we have already checked if S <: T, checking it again will not queue up the type for replacement. // So we'll have to do it ourselves. We assume they unified cleanly if they are still in the seen set. if (log.haveSeen(subTy, superOption)) @@ -822,7 +898,7 @@ bool Unifier::canCacheResult(TypeId subTy, TypeId superTy) auto skipCacheFor = [this](TypeId ty) { SkipCacheForType visitor{sharedState.skipCacheForType, types}; - visitTypeVarOnce(ty, visitor, sharedState.seenAny); + DEPRECATED_visitTypeVarOnce(ty, visitor, sharedState.seenAny); sharedState.skipCacheForType[ty] = visitor.result; diff --git a/Ast/include/Luau/Ast.h b/Ast/include/Luau/Ast.h index 31cd01cc..6f39e3fd 100644 --- a/Ast/include/Luau/Ast.h +++ b/Ast/include/Luau/Ast.h @@ -313,7 +313,7 @@ template struct AstArray { T* data; - std::size_t size; + size_t size; const T* begin() const { diff --git a/Ast/src/Parser.cpp b/Ast/src/Parser.cpp index 31ff3f77..91f5cd25 100644 --- a/Ast/src/Parser.cpp +++ b/Ast/src/Parser.cpp @@ -10,7 +10,6 @@ // See docs/SyntaxChanges.md for an explanation. LUAU_FASTINTVARIABLE(LuauRecursionLimit, 1000) LUAU_FASTINTVARIABLE(LuauParseErrorLimit, 100) -LUAU_FASTFLAGVARIABLE(LuauParseRecoverUnexpectedPack, false) LUAU_FASTFLAGVARIABLE(LuauParseLocationIgnoreCommentSkipInCapture, false) namespace Luau @@ -1430,7 +1429,7 @@ AstType* Parser::parseTypeAnnotation(TempVector& parts, const Location parts.push_back(parseSimpleTypeAnnotation(/* allowPack= */ false).type); isIntersection = true; } - else if (FFlag::LuauParseRecoverUnexpectedPack && c == Lexeme::Dot3) + else if (c == Lexeme::Dot3) { report(lexer.current().location, "Unexpected '...' after type annotation"); nextLexeme(); @@ -1551,7 +1550,7 @@ AstTypeOrPack Parser::parseSimpleTypeAnnotation(bool allowPack) prefix = name.name; name = parseIndexName("field name", pointPosition); } - else if (FFlag::LuauParseRecoverUnexpectedPack && lexer.current().type == Lexeme::Dot3) + else if (lexer.current().type == Lexeme::Dot3) { report(lexer.current().location, "Unexpected '...' after type name; type pack is not allowed in this context"); nextLexeme(); diff --git a/Compiler/include/Luau/Bytecode.h b/Compiler/include/Luau/Bytecode.h index c6e5a03b..f71d893c 100644 --- a/Compiler/include/Luau/Bytecode.h +++ b/Compiler/include/Luau/Bytecode.h @@ -353,6 +353,11 @@ enum LuauOpcode // AUX: constant index LOP_FASTCALL2K, + // FORGPREP: prepare loop variables for a generic for loop, jump to the loop backedge unconditionally + // A: target register; generic for loops assume a register layout [generator, state, index, variables...] + // D: jump offset (-32768..32767) + LOP_FORGPREP, + // Enum entry for number of opcodes, not a valid opcode by itself! LOP__COUNT }; diff --git a/Compiler/src/BytecodeBuilder.cpp b/Compiler/src/BytecodeBuilder.cpp index 871a1484..fb70392e 100644 --- a/Compiler/src/BytecodeBuilder.cpp +++ b/Compiler/src/BytecodeBuilder.cpp @@ -96,6 +96,7 @@ inline bool isJumpD(LuauOpcode op) case LOP_JUMPIFNOTLT: case LOP_FORNPREP: case LOP_FORNLOOP: + case LOP_FORGPREP: case LOP_FORGLOOP: case LOP_FORGPREP_INEXT: case LOP_FORGLOOP_INEXT: @@ -1269,6 +1270,11 @@ void BytecodeBuilder::validate() const VJUMP(LUAU_INSN_D(insn)); break; + case LOP_FORGPREP: + VREG(LUAU_INSN_A(insn) + 2 + 1); // forg loop protocol: A, A+1, A+2 are used for iteration protocol; A+3, ... are loop variables + VJUMP(LUAU_INSN_D(insn)); + break; + case LOP_FORGLOOP: VREG( LUAU_INSN_A(insn) + 2 + insns[i + 1]); // forg loop protocol: A, A+1, A+2 are used for iteration protocol; A+3, ... are loop variables @@ -1622,6 +1628,10 @@ const uint32_t* BytecodeBuilder::dumpInstruction(const uint32_t* code, std::stri formatAppend(result, "FORNLOOP R%d %+d\n", LUAU_INSN_A(insn), LUAU_INSN_D(insn)); break; + case LOP_FORGPREP: + formatAppend(result, "FORGPREP R%d %+d\n", LUAU_INSN_A(insn), LUAU_INSN_D(insn)); + break; + case LOP_FORGLOOP: formatAppend(result, "FORGLOOP R%d %+d %d\n", LUAU_INSN_A(insn), LUAU_INSN_D(insn), *code++); break; diff --git a/Compiler/src/Compiler.cpp b/Compiler/src/Compiler.cpp index 0f17ee02..4fe26222 100644 --- a/Compiler/src/Compiler.cpp +++ b/Compiler/src/Compiler.cpp @@ -17,9 +17,19 @@ #include #include +LUAU_FASTFLAGVARIABLE(LuauCompileSupportInlining, false) + +LUAU_FASTFLAGVARIABLE(LuauCompileIter, false) +LUAU_FASTFLAGVARIABLE(LuauCompileIterNoReserve, false) +LUAU_FASTFLAGVARIABLE(LuauCompileIterNoPairs, false) + LUAU_FASTINTVARIABLE(LuauCompileLoopUnrollThreshold, 25) LUAU_FASTINTVARIABLE(LuauCompileLoopUnrollThresholdMaxBoost, 300) +LUAU_FASTINTVARIABLE(LuauCompileInlineThreshold, 25) +LUAU_FASTINTVARIABLE(LuauCompileInlineThresholdMaxBoost, 300) +LUAU_FASTINTVARIABLE(LuauCompileInlineDepth, 5) + namespace Luau { @@ -147,6 +157,52 @@ struct Compiler } } + AstExprFunction* getFunctionExpr(AstExpr* node) + { + if (AstExprLocal* le = node->as()) + { + Variable* lv = variables.find(le->local); + + if (!lv || lv->written || !lv->init) + return nullptr; + + return getFunctionExpr(lv->init); + } + else if (AstExprGroup* ge = node->as()) + return getFunctionExpr(ge->expr); + else + return node->as(); + } + + bool canInlineFunctionBody(AstStat* stat) + { + struct CanInlineVisitor : AstVisitor + { + bool result = true; + + bool visit(AstExpr* node) override + { + // nested functions may capture function arguments, and our upval handling doesn't handle elided variables (constant) + // TODO: we could remove this case if we changed function compilation to create temporary locals for constant upvalues + // TODO: additionally we would need to change upvalue handling in compileExprFunction to handle upvalue->local migration + result = result && !node->is(); + return result; + } + + bool visit(AstStat* node) override + { + // loops may need to be unrolled which can result in cost amplification + result = result && !node->is(); + return result; + } + }; + + CanInlineVisitor canInline; + stat->visit(&canInline); + + return canInline.result; + } + uint32_t compileFunction(AstExprFunction* func) { LUAU_TIMETRACE_SCOPE("Compiler::compileFunction", "Compiler"); @@ -214,13 +270,21 @@ struct Compiler bytecode.endFunction(uint8_t(stackSize), uint8_t(upvals.size())); - stackSize = 0; - Function& f = functions[func]; f.id = fid; f.upvals = upvals; + // record information for inlining + if (FFlag::LuauCompileSupportInlining && options.optimizationLevel >= 2 && !func->vararg && canInlineFunctionBody(func->body) && + !getfenvUsed && !setfenvUsed) + { + f.canInline = true; + f.stackSize = stackSize; + f.costModel = modelCost(func->body, func->args.data, func->args.size); + } + upvals.clear(); // note: instead of std::move above, we copy & clear to preserve capacity for future pushes + stackSize = 0; return fid; } @@ -390,12 +454,183 @@ struct Compiler } } + bool tryCompileInlinedCall(AstExprCall* expr, AstExprFunction* func, uint8_t target, uint8_t targetCount, bool multRet, int thresholdBase, + int thresholdMaxBoost, int depthLimit) + { + Function* fi = functions.find(func); + LUAU_ASSERT(fi); + + // make sure we have enough register space + if (regTop > 128 || fi->stackSize > 32) + { + bytecode.addDebugRemark("inlining failed: high register pressure"); + return false; + } + + // we should ideally aggregate the costs during recursive inlining, but for now simply limit the depth + if (int(inlineFrames.size()) >= depthLimit) + { + bytecode.addDebugRemark("inlining failed: too many inlined frames"); + return false; + } + + // compiling recursive inlining is difficult because we share constant/variable state but need to bind variables to different registers + for (InlineFrame& frame : inlineFrames) + if (frame.func == func) + { + bytecode.addDebugRemark("inlining failed: can't inline recursive calls"); + return false; + } + + // TODO: we can compile multret functions if all returns of the function are multret as well + if (multRet) + { + bytecode.addDebugRemark("inlining failed: can't convert fixed returns to multret"); + return false; + } + + // TODO: we can compile functions with mismatching arity at call site but it's more annoying + if (func->args.size != expr->args.size) + { + bytecode.addDebugRemark("inlining failed: argument count mismatch (expected %d, got %d)", int(func->args.size), int(expr->args.size)); + return false; + } + + // we use a dynamic cost threshold that's based on the fixed limit boosted by the cost advantage we gain due to inlining + bool varc[8] = {}; + for (size_t i = 0; i < expr->args.size && i < 8; ++i) + varc[i] = isConstant(expr->args.data[i]); + + int inlinedCost = computeCost(fi->costModel, varc, std::min(int(expr->args.size), 8)); + int baselineCost = computeCost(fi->costModel, nullptr, 0) + 3; + int inlineProfit = (inlinedCost == 0) ? thresholdMaxBoost : std::min(thresholdMaxBoost, 100 * baselineCost / inlinedCost); + + int threshold = thresholdBase * inlineProfit / 100; + + if (inlinedCost > threshold) + { + bytecode.addDebugRemark("inlining failed: too expensive (cost %d, profit %.2fx)", inlinedCost, double(inlineProfit) / 100); + return false; + } + + bytecode.addDebugRemark( + "inlining succeeded (cost %d, profit %.2fx, depth %d)", inlinedCost, double(inlineProfit) / 100, int(inlineFrames.size())); + + compileInlinedCall(expr, func, target, targetCount); + return true; + } + + void compileInlinedCall(AstExprCall* expr, AstExprFunction* func, uint8_t target, uint8_t targetCount) + { + RegScope rs(this); + + size_t oldLocals = localStack.size(); + + // note that we push the frame early; this is needed to block recursive inline attempts + inlineFrames.push_back({func, target, targetCount}); + + // evaluate all arguments; note that we don't emit code for constant arguments (relying on constant folding) + for (size_t i = 0; i < func->args.size; ++i) + { + AstLocal* var = func->args.data[i]; + AstExpr* arg = expr->args.data[i]; + + if (Variable* vv = variables.find(var); vv && vv->written) + { + // if the argument is mutated, we need to allocate a fresh register even if it's a constant + uint8_t reg = allocReg(arg, 1); + compileExprTemp(arg, reg); + pushLocal(var, reg); + } + else if (const Constant* cv = constants.find(arg); cv && cv->type != Constant::Type_Unknown) + { + // since the argument is not mutated, we can simply fold the value into the expressions that need it + locstants[var] = *cv; + } + else + { + AstExprLocal* le = arg->as(); + Variable* lv = le ? variables.find(le->local) : nullptr; + + // if the argument is a local that isn't mutated, we will simply reuse the existing register + if (isExprLocalReg(arg) && (!lv || !lv->written)) + { + uint8_t reg = getLocal(le->local); + pushLocal(var, reg); + } + else + { + uint8_t reg = allocReg(arg, 1); + compileExprTemp(arg, reg); + pushLocal(var, reg); + } + } + } + + // fold constant values updated above into expressions in the function body + foldConstants(constants, variables, locstants, func->body); + + bool usedFallthrough = false; + + for (size_t i = 0; i < func->body->body.size; ++i) + { + AstStat* stat = func->body->body.data[i]; + + if (AstStatReturn* ret = stat->as()) + { + // Optimization: use fallthrough when compiling return at the end of the function to avoid an extra JUMP + compileInlineReturn(ret, /* fallthrough= */ true); + // TODO: This doesn't work when return is part of control flow; ideally we would track the state somehow and generalize this + usedFallthrough = true; + break; + } + else + compileStat(stat); + } + + // for the fallthrough path we need to ensure we clear out target registers + if (!usedFallthrough && !allPathsEndWithReturn(func->body)) + { + for (size_t i = 0; i < targetCount; ++i) + bytecode.emitABC(LOP_LOADNIL, uint8_t(target + i), 0, 0); + } + + popLocals(oldLocals); + + size_t returnLabel = bytecode.emitLabel(); + patchJumps(expr, inlineFrames.back().returnJumps, returnLabel); + + inlineFrames.pop_back(); + + // clean up constant state for future inlining attempts + for (size_t i = 0; i < func->args.size; ++i) + if (Constant* var = locstants.find(func->args.data[i])) + var->type = Constant::Type_Unknown; + + foldConstants(constants, variables, locstants, func->body); + } + void compileExprCall(AstExprCall* expr, uint8_t target, uint8_t targetCount, bool targetTop = false, bool multRet = false) { LUAU_ASSERT(!targetTop || unsigned(target + targetCount) == regTop); setDebugLine(expr); // normally compileExpr sets up line info, but compileExprCall can be called directly + // try inlining the function + if (options.optimizationLevel >= 2 && !expr->self) + { + AstExprFunction* func = getFunctionExpr(expr->func); + Function* fi = func ? functions.find(func) : nullptr; + + if (fi && fi->canInline && + tryCompileInlinedCall(expr, func, target, targetCount, multRet, FInt::LuauCompileInlineThreshold, + FInt::LuauCompileInlineThresholdMaxBoost, FInt::LuauCompileInlineDepth)) + return; + + if (fi && !fi->canInline) + bytecode.addDebugRemark("inlining failed: complex constructs in function body"); + } + RegScope rs(this); unsigned int regCount = std::max(unsigned(1 + expr->self + expr->args.size), unsigned(targetCount)); @@ -760,7 +995,7 @@ struct Compiler { const Constant* c = constants.find(node); - if (!c) + if (!c || c->type == Constant::Type_Unknown) return -1; int cid = -1; @@ -1395,27 +1630,29 @@ struct Compiler { RegScope rs(this); + // note: cv may be invalidated by compileExpr* so we stop using it before calling compile recursively const Constant* cv = constants.find(expr->index); if (cv && cv->type == Constant::Type_Number && cv->valueNumber >= 1 && cv->valueNumber <= 256 && double(int(cv->valueNumber)) == cv->valueNumber) { - uint8_t rt = compileExprAuto(expr->expr, rs); uint8_t i = uint8_t(int(cv->valueNumber) - 1); + uint8_t rt = compileExprAuto(expr->expr, rs); + setDebugLine(expr->index); bytecode.emitABC(LOP_GETTABLEN, target, rt, i); } else if (cv && cv->type == Constant::Type_String) { - uint8_t rt = compileExprAuto(expr->expr, rs); - BytecodeBuilder::StringRef iname = sref(cv->getString()); int32_t cid = bytecode.addConstantString(iname); if (cid < 0) CompileError::raise(expr->location, "Exceeded constant limit; simplify the code to compile"); + uint8_t rt = compileExprAuto(expr->expr, rs); + setDebugLine(expr->index); bytecode.emitABC(LOP_GETTABLEKS, target, rt, uint8_t(BytecodeBuilder::getStringHash(iname))); @@ -1561,8 +1798,9 @@ struct Compiler } else if (AstExprLocal* expr = node->as()) { - if (expr->upvalue) + if (FFlag::LuauCompileSupportInlining ? !isExprLocalReg(expr) : expr->upvalue) { + LUAU_ASSERT(expr->upvalue); uint8_t uid = getUpval(expr->local); bytecode.emitABC(LOP_GETUPVAL, target, uid, 0); @@ -1650,12 +1888,12 @@ struct Compiler // initializes target..target+targetCount-1 range using expressions from the list // if list has fewer expressions, and last expression is a call, we assume the call returns the rest of the values // if list has fewer expressions, and last expression isn't a call, we fill the rest with nil - // assumes target register range can be clobbered and is at the top of the register space - void compileExprListTop(const AstArray& list, uint8_t target, uint8_t targetCount) + // assumes target register range can be clobbered and is at the top of the register space if targetTop = true + void compileExprListTemp(const AstArray& list, uint8_t target, uint8_t targetCount, bool targetTop) { // we assume that target range is at the top of the register space and can be clobbered // this is what allows us to compile the last call expression - if it's a call - using targetTop=true - LUAU_ASSERT(unsigned(target + targetCount) == regTop); + LUAU_ASSERT(!targetTop || unsigned(target + targetCount) == regTop); if (list.size == targetCount) { @@ -1683,7 +1921,7 @@ struct Compiler if (AstExprCall* expr = last->as()) { - compileExprCall(expr, uint8_t(target + list.size - 1), uint8_t(targetCount - (list.size - 1)), /* targetTop= */ true); + compileExprCall(expr, uint8_t(target + list.size - 1), uint8_t(targetCount - (list.size - 1)), targetTop); } else if (AstExprVarargs* expr = last->as()) { @@ -1765,8 +2003,10 @@ struct Compiler if (AstExprLocal* expr = node->as()) { - if (expr->upvalue) + if (FFlag::LuauCompileSupportInlining ? !isExprLocalReg(expr) : expr->upvalue) { + LUAU_ASSERT(expr->upvalue); + LValue result = {LValue::Kind_Upvalue}; result.upval = getUpval(expr->local); result.location = node->location; @@ -1873,7 +2113,7 @@ struct Compiler bool isExprLocalReg(AstExpr* expr) { AstExprLocal* le = expr->as(); - if (!le || le->upvalue) + if (!le || (!FFlag::LuauCompileSupportInlining && le->upvalue)) return false; Local* l = locals.find(le->local); @@ -2080,6 +2320,23 @@ struct Compiler loops.pop_back(); } + void compileInlineReturn(AstStatReturn* stat, bool fallthrough) + { + setDebugLine(stat); // normally compileStat sets up line info, but compileInlineReturn can be called directly + + InlineFrame frame = inlineFrames.back(); + + compileExprListTemp(stat->list, frame.target, frame.targetCount, /* targetTop= */ false); + + if (!fallthrough) + { + size_t jumpLabel = bytecode.emitLabel(); + bytecode.emitAD(LOP_JUMP, 0, 0); + + inlineFrames.back().returnJumps.push_back(jumpLabel); + } + } + void compileStatReturn(AstStatReturn* stat) { RegScope rs(this); @@ -2138,7 +2395,7 @@ struct Compiler // note: allocReg in this case allocates into parent block register - note that we don't have RegScope here uint8_t vars = allocReg(stat, unsigned(stat->vars.size)); - compileExprListTop(stat->values, vars, uint8_t(stat->vars.size)); + compileExprListTemp(stat->values, vars, uint8_t(stat->vars.size), /* targetTop= */ true); for (size_t i = 0; i < stat->vars.size; ++i) pushLocal(stat->vars.data[i], uint8_t(vars + i)); @@ -2168,6 +2425,7 @@ struct Compiler bool visit(AstExpr* node) override { // functions may capture loop variable, and our upval handling doesn't handle elided variables (constant) + // TODO: we could remove this case if we changed function compilation to create temporary locals for constant upvalues result = result && !node->is(); return result; } @@ -2251,6 +2509,11 @@ struct Compiler compileStat(stat->body); } + // clean up fold state in case we need to recompile - normally we compile the loop body once, but due to inlining we may need to do it again + locstants[var].type = Constant::Type_Unknown; + + foldConstants(constants, variables, locstants, stat); + return true; } @@ -2336,12 +2599,17 @@ struct Compiler uint8_t regs = allocReg(stat, 3); // this puts initial values of (generator, state, index) into the loop registers - compileExprListTop(stat->values, regs, 3); + compileExprListTemp(stat->values, regs, 3, /* targetTop= */ true); - // for the general case, we will execute a CALL for every iteration that needs to evaluate "variables... = generator(state, index)" - // this requires at least extra 3 stack slots after index - // note that these stack slots overlap with the variables so we only need to reserve them to make sure stack frame is large enough - reserveReg(stat, 3); + // we don't need this because the extra stack space is just for calling the function with a loop protocol which is similar to calling + // metamethods - it should fit into the extra stack reservation + if (!FFlag::LuauCompileIterNoReserve) + { + // for the general case, we will execute a CALL for every iteration that needs to evaluate "variables... = generator(state, index)" + // this requires at least extra 3 stack slots after index + // note that these stack slots overlap with the variables so we only need to reserve them to make sure stack frame is large enough + reserveReg(stat, 3); + } // note that we reserve at least 2 variables; this allows our fast path to assume that we need 2 variables instead of 1 or 2 uint8_t vars = allocReg(stat, std::max(unsigned(stat->vars.size), 2u)); @@ -2350,7 +2618,7 @@ struct Compiler // Optimization: when we iterate through pairs/ipairs, we generate special bytecode that optimizes the traversal using internal iteration // index These instructions dynamically check if generator is equal to next/inext and bail out They assume that the generator produces 2 // variables, which is why we allocate at least 2 above (see vars assignment) - LuauOpcode skipOp = LOP_JUMP; + LuauOpcode skipOp = FFlag::LuauCompileIter ? LOP_FORGPREP : LOP_JUMP; LuauOpcode loopOp = LOP_FORGLOOP; if (options.optimizationLevel >= 1 && stat->vars.size <= 2) @@ -2367,7 +2635,7 @@ struct Compiler else if (builtin.isGlobal("pairs")) // for .. in pairs(t) { skipOp = LOP_FORGPREP_NEXT; - loopOp = LOP_FORGLOOP_NEXT; + loopOp = FFlag::LuauCompileIterNoPairs ? LOP_FORGLOOP : LOP_FORGLOOP_NEXT; } } else if (stat->values.size == 2) @@ -2377,7 +2645,7 @@ struct Compiler if (builtin.isGlobal("next")) // for .. in next,t { skipOp = LOP_FORGPREP_NEXT; - loopOp = LOP_FORGLOOP_NEXT; + loopOp = FFlag::LuauCompileIterNoPairs ? LOP_FORGLOOP : LOP_FORGLOOP_NEXT; } } } @@ -2514,10 +2782,10 @@ struct Compiler // compute values into temporaries uint8_t regs = allocReg(stat, unsigned(stat->vars.size)); - compileExprListTop(stat->values, regs, uint8_t(stat->vars.size)); + compileExprListTemp(stat->values, regs, uint8_t(stat->vars.size), /* targetTop= */ true); - // assign variables that have associated values; note that if we have fewer values than variables, we'll assign nil because compileExprListTop - // will generate nils + // assign variables that have associated values; note that if we have fewer values than variables, we'll assign nil because + // compileExprListTemp will generate nils for (size_t i = 0; i < stat->vars.size; ++i) { setDebugLine(stat->vars.data[i]); @@ -2675,7 +2943,10 @@ struct Compiler } else if (AstStatReturn* stat = node->as()) { - compileStatReturn(stat); + if (options.optimizationLevel >= 2 && !inlineFrames.empty()) + compileInlineReturn(stat, /* fallthrough= */ false); + else + compileStatReturn(stat); } else if (AstStatExpr* stat = node->as()) { @@ -3069,6 +3340,10 @@ struct Compiler { uint32_t id; std::vector upvals; + + uint64_t costModel = 0; + unsigned int stackSize = 0; + bool canInline = false; }; struct Local @@ -3098,6 +3373,16 @@ struct Compiler AstExpr* untilCondition; }; + struct InlineFrame + { + AstExprFunction* func; + + uint8_t target; + uint8_t targetCount; + + std::vector returnJumps; + }; + BytecodeBuilder& bytecode; CompileOptions options; @@ -3120,6 +3405,7 @@ struct Compiler std::vector upvals; std::vector loopJumps; std::vector loops; + std::vector inlineFrames; }; void compileOrThrow(BytecodeBuilder& bytecode, AstStatBlock* root, const AstNameTable& names, const CompileOptions& options) diff --git a/Compiler/src/ConstantFolding.cpp b/Compiler/src/ConstantFolding.cpp index 7ad91d4b..52ece73e 100644 --- a/Compiler/src/ConstantFolding.cpp +++ b/Compiler/src/ConstantFolding.cpp @@ -3,6 +3,8 @@ #include +LUAU_FASTFLAG(LuauCompileSupportInlining) + namespace Luau { namespace Compile @@ -314,12 +316,35 @@ struct ConstantVisitor : AstVisitor LUAU_ASSERT(!"Unknown expression type"); } - if (result.type != Constant::Type_Unknown) - constants[node] = result; + recordConstant(constants, node, result); return result; } + template + void recordConstant(DenseHashMap& map, T key, const Constant& value) + { + if (value.type != Constant::Type_Unknown) + map[key] = value; + else if (!FFlag::LuauCompileSupportInlining) + ; + else if (Constant* old = map.find(key)) + old->type = Constant::Type_Unknown; + } + + void recordValue(AstLocal* local, const Constant& value) + { + // note: we rely on trackValues to have been run before us + Variable* v = variables.find(local); + LUAU_ASSERT(v); + + if (!v->written) + { + v->constant = (value.type != Constant::Type_Unknown); + recordConstant(locals, local, value); + } + } + bool visit(AstExpr* node) override { // note: we short-circuit the visitor traversal through any expression trees by returning false @@ -336,18 +361,7 @@ struct ConstantVisitor : AstVisitor { Constant arg = analyze(node->values.data[i]); - if (arg.type != Constant::Type_Unknown) - { - // note: we rely on trackValues to have been run before us - Variable* v = variables.find(node->vars.data[i]); - LUAU_ASSERT(v); - - if (!v->written) - { - locals[node->vars.data[i]] = arg; - v->constant = true; - } - } + recordValue(node->vars.data[i], arg); } if (node->vars.size > node->values.size) @@ -361,15 +375,8 @@ struct ConstantVisitor : AstVisitor { for (size_t i = node->values.size; i < node->vars.size; ++i) { - // note: we rely on trackValues to have been run before us - Variable* v = variables.find(node->vars.data[i]); - LUAU_ASSERT(v); - - if (!v->written) - { - locals[node->vars.data[i]].type = Constant::Type_Nil; - v->constant = true; - } + Constant nil = {Constant::Type_Nil}; + recordValue(node->vars.data[i], nil); } } } diff --git a/Sources.cmake b/Sources.cmake index f9263b24..d2430cc9 100644 --- a/Sources.cmake +++ b/Sources.cmake @@ -264,6 +264,7 @@ if(TARGET Luau.UnitTest) tests/TypePack.test.cpp tests/TypeVar.test.cpp tests/Variant.test.cpp + tests/VisitTypeVar.test.cpp tests/main.cpp) endif() diff --git a/VM/src/lapi.cpp b/VM/src/lapi.cpp index 1f3b0943..f8baefaf 100644 --- a/VM/src/lapi.cpp +++ b/VM/src/lapi.cpp @@ -1270,7 +1270,7 @@ const char* lua_setupvalue(lua_State* L, int funcindex, int n) L->top--; setobj(L, val, L->top); luaC_barrier(L, clvalue(fi), L->top); - luaC_upvalbarrier(L, NULL, val); + luaC_upvalbarrier(L, cast_to(UpVal*, NULL), val); } return name; } diff --git a/VM/src/lbuiltins.cpp b/VM/src/lbuiltins.cpp index 718d387d..60149199 100644 --- a/VM/src/lbuiltins.cpp +++ b/VM/src/lbuiltins.cpp @@ -15,6 +15,8 @@ #include #endif +LUAU_FASTFLAGVARIABLE(LuauFixBuiltinsStackLimit, false) + // luauF functions implement FASTCALL instruction that performs a direct execution of some builtin functions from the VM // The rule of thumb is that FASTCALL functions can not call user code, yield, fail, or reallocate stack. // If types of the arguments mismatch, luauF_* needs to return -1 and the execution will fall back to the usual call path @@ -1003,7 +1005,7 @@ static int luauF_tunpack(lua_State* L, StkId res, TValue* arg0, int nresults, St else if (nparams == 3 && ttisnumber(args) && ttisnumber(args + 1) && nvalue(args) == 1.0) n = int(nvalue(args + 1)); - if (n >= 0 && n <= t->sizearray && cast_int(L->stack_last - res) >= n) + if (n >= 0 && n <= t->sizearray && cast_int(L->stack_last - res) >= n && (!FFlag::LuauFixBuiltinsStackLimit || n + nparams <= LUAI_MAXCSTACK)) { TValue* array = t->array; for (int i = 0; i < n; ++i) diff --git a/VM/src/lgc.h b/VM/src/lgc.h index 08d1ff5d..797284a2 100644 --- a/VM/src/lgc.h +++ b/VM/src/lgc.h @@ -120,7 +120,7 @@ #define luaC_upvalbarrier(L, uv, tv) \ { \ - if (iscollectable(tv) && iswhite(gcvalue(tv)) && (!(uv) || ((UpVal*)uv)->v != &((UpVal*)uv)->u.value)) \ + if (iscollectable(tv) && iswhite(gcvalue(tv)) && (!(uv) || (uv)->v != &(uv)->u.value)) \ luaC_barrierupval(L, gcvalue(tv)); \ } diff --git a/VM/src/ltable.cpp b/VM/src/ltable.cpp index 3dc3bd1b..8251b51c 100644 --- a/VM/src/ltable.cpp +++ b/VM/src/ltable.cpp @@ -33,8 +33,6 @@ #include -LUAU_FASTFLAGVARIABLE(LuauTableNewBoundary2, false) - // max size of both array and hash part is 2^MAXBITS #define MAXBITS 26 #define MAXSIZE (1 << MAXBITS) @@ -431,7 +429,6 @@ static void resize(lua_State* L, Table* t, int nasize, int nhsize) static int adjustasize(Table* t, int size, const TValue* ek) { - LUAU_ASSERT(FFlag::LuauTableNewBoundary2); bool tbound = t->node != dummynode || size < t->sizearray; int ekindex = ek && ttisnumber(ek) ? arrayindex(nvalue(ek)) : -1; /* move the array size up until the boundary is guaranteed to be inside the array part */ @@ -443,7 +440,7 @@ static int adjustasize(Table* t, int size, const TValue* ek) void luaH_resizearray(lua_State* L, Table* t, int nasize) { int nsize = (t->node == dummynode) ? 0 : sizenode(t); - int asize = FFlag::LuauTableNewBoundary2 ? adjustasize(t, nasize, NULL) : nasize; + int asize = adjustasize(t, nasize, NULL); resize(L, t, asize, nsize); } @@ -468,8 +465,7 @@ static void rehash(lua_State* L, Table* t, const TValue* ek) int na = computesizes(nums, &nasize); int nh = totaluse - na; /* enforce the boundary invariant; for performance, only do hash lookups if we must */ - if (FFlag::LuauTableNewBoundary2) - nasize = adjustasize(t, nasize, ek); + nasize = adjustasize(t, nasize, ek); /* resize the table to new computed sizes */ resize(L, t, nasize, nh); } @@ -531,7 +527,7 @@ static LuaNode* getfreepos(Table* t) static TValue* newkey(lua_State* L, Table* t, const TValue* key) { /* enforce boundary invariant */ - if (FFlag::LuauTableNewBoundary2 && ttisnumber(key) && nvalue(key) == t->sizearray + 1) + if (ttisnumber(key) && nvalue(key) == t->sizearray + 1) { rehash(L, t, key); /* grow table */ @@ -713,37 +709,6 @@ TValue* luaH_setstr(lua_State* L, Table* t, TString* key) } } -static LUAU_NOINLINE int unbound_search(Table* t, unsigned int j) -{ - LUAU_ASSERT(!FFlag::LuauTableNewBoundary2); - unsigned int i = j; /* i is zero or a present index */ - j++; - /* find `i' and `j' such that i is present and j is not */ - while (!ttisnil(luaH_getnum(t, j))) - { - i = j; - j *= 2; - if (j > cast_to(unsigned int, INT_MAX)) - { /* overflow? */ - /* table was built with bad purposes: resort to linear search */ - i = 1; - while (!ttisnil(luaH_getnum(t, i))) - i++; - return i - 1; - } - } - /* now do a binary search between them */ - while (j - i > 1) - { - unsigned int m = (i + j) / 2; - if (ttisnil(luaH_getnum(t, m))) - j = m; - else - i = m; - } - return i; -} - static int updateaboundary(Table* t, int boundary) { if (boundary < t->sizearray && ttisnil(&t->array[boundary - 1])) @@ -800,17 +765,12 @@ int luaH_getn(Table* t) maybesetaboundary(t, boundary); return boundary; } - else if (FFlag::LuauTableNewBoundary2) + else { /* validate boundary invariant */ LUAU_ASSERT(t->node == dummynode || ttisnil(luaH_getnum(t, j + 1))); return j; } - /* else must find a boundary in hash part */ - else if (t->node == dummynode) /* hash part is empty? */ - return j; /* that is easy... */ - else - return unbound_search(t, j); } Table* luaH_clone(lua_State* L, Table* tt) diff --git a/VM/src/ltm.cpp b/VM/src/ltm.cpp index 106efb2b..9b99506b 100644 --- a/VM/src/ltm.cpp +++ b/VM/src/ltm.cpp @@ -37,6 +37,8 @@ const char* const luaT_eventname[] = { "__newindex", "__mode", "__namecall", + "__call", + "__iter", "__eq", @@ -54,13 +56,13 @@ const char* const luaT_eventname[] = { "__lt", "__le", "__concat", - "__call", "__type", }; // clang-format on static_assert(sizeof(luaT_typenames) / sizeof(luaT_typenames[0]) == LUA_T_COUNT, "luaT_typenames size mismatch"); static_assert(sizeof(luaT_eventname) / sizeof(luaT_eventname[0]) == TM_N, "luaT_eventname size mismatch"); +static_assert(TM_EQ < 8, "fasttm optimization stores a bitfield with metamethods in a byte"); void luaT_init(lua_State* L) { diff --git a/VM/src/ltm.h b/VM/src/ltm.h index 0e4e915d..e1b95c21 100644 --- a/VM/src/ltm.h +++ b/VM/src/ltm.h @@ -16,6 +16,8 @@ typedef enum TM_NEWINDEX, TM_MODE, TM_NAMECALL, + TM_CALL, + TM_ITER, TM_EQ, /* last tag method with `fast' access */ @@ -33,7 +35,6 @@ typedef enum TM_LT, TM_LE, TM_CONCAT, - TM_CALL, TM_TYPE, TM_N /* number of elements in the enum */ diff --git a/VM/src/lvmexecute.cpp b/VM/src/lvmexecute.cpp index 39c60eac..3c7c276a 100644 --- a/VM/src/lvmexecute.cpp +++ b/VM/src/lvmexecute.cpp @@ -16,7 +16,10 @@ #include -LUAU_FASTFLAG(LuauTableNewBoundary2) +LUAU_FASTFLAGVARIABLE(LuauIter, false) +LUAU_DYNAMIC_FASTFLAGVARIABLE(LuauIterCallTelemetry, false) + +void (*lua_iter_call_telemetry)(lua_State* L); // Disable c99-designator to avoid the warning in CGOTO dispatch table #ifdef __clang__ @@ -110,7 +113,7 @@ LUAU_FASTFLAG(LuauTableNewBoundary2) VM_DISPATCH_OP(LOP_FORGLOOP_NEXT), VM_DISPATCH_OP(LOP_GETVARARGS), VM_DISPATCH_OP(LOP_DUPCLOSURE), VM_DISPATCH_OP(LOP_PREPVARARGS), \ VM_DISPATCH_OP(LOP_LOADKX), VM_DISPATCH_OP(LOP_JUMPX), VM_DISPATCH_OP(LOP_FASTCALL), VM_DISPATCH_OP(LOP_COVERAGE), \ VM_DISPATCH_OP(LOP_CAPTURE), VM_DISPATCH_OP(LOP_JUMPIFEQK), VM_DISPATCH_OP(LOP_JUMPIFNOTEQK), VM_DISPATCH_OP(LOP_FASTCALL1), \ - VM_DISPATCH_OP(LOP_FASTCALL2), VM_DISPATCH_OP(LOP_FASTCALL2K), + VM_DISPATCH_OP(LOP_FASTCALL2), VM_DISPATCH_OP(LOP_FASTCALL2K), VM_DISPATCH_OP(LOP_FORGPREP), #if defined(__GNUC__) || defined(__clang__) #define VM_USE_CGOTO 1 @@ -150,8 +153,20 @@ LUAU_NOINLINE static void luau_prepareFORN(lua_State* L, StkId plimit, StkId pst LUAU_NOINLINE static bool luau_loopFORG(lua_State* L, int a, int c) { + // note: it's safe to push arguments past top for complicated reasons (see top of the file) StkId ra = &L->base[a]; - LUAU_ASSERT(ra + 6 <= L->top); + LUAU_ASSERT(ra + 3 <= L->top); + + if (DFFlag::LuauIterCallTelemetry) + { + /* TODO: we might be able to stop supporting this depending on whether it's used in practice */ + void (*telemetrycb)(lua_State* L) = lua_iter_call_telemetry; + + if (telemetrycb && ttistable(ra) && fasttm(L, hvalue(ra)->metatable, TM_CALL)) + telemetrycb(L); + if (telemetrycb && ttisuserdata(ra) && fasttm(L, uvalue(ra)->metatable, TM_CALL)) + telemetrycb(L); + } setobjs2s(L, ra + 3 + 2, ra + 2); setobjs2s(L, ra + 3 + 1, ra + 1); @@ -2204,20 +2219,149 @@ static void luau_execute(lua_State* L) } } + VM_CASE(LOP_FORGPREP) + { + Instruction insn = *pc++; + StkId ra = VM_REG(LUAU_INSN_A(insn)); + + if (ttisfunction(ra)) + { + /* will be called during FORGLOOP */ + } + else if (FFlag::LuauIter) + { + Table* mt = ttistable(ra) ? hvalue(ra)->metatable : ttisuserdata(ra) ? uvalue(ra)->metatable : cast_to(Table*, NULL); + + if (const TValue* fn = fasttm(L, mt, TM_ITER)) + { + setobj2s(L, ra + 1, ra); + setobj2s(L, ra, fn); + + L->top = ra + 2; /* func + self arg */ + LUAU_ASSERT(L->top <= L->stack_last); + + VM_PROTECT(luaD_call(L, ra, 3)); + L->top = L->ci->top; + } + else if (fasttm(L, mt, TM_CALL)) + { + /* table or userdata with __call, will be called during FORGLOOP */ + /* TODO: we might be able to stop supporting this depending on whether it's used in practice */ + } + else if (ttistable(ra)) + { + /* set up registers for builtin iteration */ + setobj2s(L, ra + 1, ra); + setpvalue(ra + 2, reinterpret_cast(uintptr_t(0))); + setnilvalue(ra); + } + else + { + VM_PROTECT(luaG_typeerror(L, ra, "iterate over")); + } + } + + pc += LUAU_INSN_D(insn); + LUAU_ASSERT(unsigned(pc - cl->l.p->code) < unsigned(cl->l.p->sizecode)); + VM_NEXT(); + } + VM_CASE(LOP_FORGLOOP) { VM_INTERRUPT(); Instruction insn = *pc++; + StkId ra = VM_REG(LUAU_INSN_A(insn)); uint32_t aux = *pc; - // note: this is a slow generic path, fast-path is FORGLOOP_INEXT/NEXT - bool stop; - VM_PROTECT(stop = luau_loopFORG(L, LUAU_INSN_A(insn), aux)); + if (!FFlag::LuauIter) + { + bool stop; + VM_PROTECT(stop = luau_loopFORG(L, LUAU_INSN_A(insn), aux)); - // note that we need to increment pc by 1 to exit the loop since we need to skip over aux - pc += stop ? 1 : LUAU_INSN_D(insn); - LUAU_ASSERT(unsigned(pc - cl->l.p->code) < unsigned(cl->l.p->sizecode)); - VM_NEXT(); + // note that we need to increment pc by 1 to exit the loop since we need to skip over aux + pc += stop ? 1 : LUAU_INSN_D(insn); + LUAU_ASSERT(unsigned(pc - cl->l.p->code) < unsigned(cl->l.p->sizecode)); + VM_NEXT(); + } + + // fast-path: builtin table iteration + if (ttisnil(ra) && ttistable(ra + 1) && ttislightuserdata(ra + 2)) + { + Table* h = hvalue(ra + 1); + int index = int(reinterpret_cast(pvalue(ra + 2))); + + int sizearray = h->sizearray; + int sizenode = 1 << h->lsizenode; + + // clear extra variables since we might have more than two + if (LUAU_UNLIKELY(aux > 2)) + for (int i = 2; i < int(aux); ++i) + setnilvalue(ra + 3 + i); + + // first we advance index through the array portion + while (unsigned(index) < unsigned(sizearray)) + { + if (!ttisnil(&h->array[index])) + { + setpvalue(ra + 2, reinterpret_cast(uintptr_t(index + 1))); + setnvalue(ra + 3, double(index + 1)); + setobj2s(L, ra + 4, &h->array[index]); + + pc += LUAU_INSN_D(insn); + LUAU_ASSERT(unsigned(pc - cl->l.p->code) < unsigned(cl->l.p->sizecode)); + VM_NEXT(); + } + + index++; + } + + // then we advance index through the hash portion + while (unsigned(index - sizearray) < unsigned(sizenode)) + { + LuaNode* n = &h->node[index - sizearray]; + + if (!ttisnil(gval(n))) + { + setpvalue(ra + 2, reinterpret_cast(uintptr_t(index + 1))); + getnodekey(L, ra + 3, n); + setobj2s(L, ra + 4, gval(n)); + + pc += LUAU_INSN_D(insn); + LUAU_ASSERT(unsigned(pc - cl->l.p->code) < unsigned(cl->l.p->sizecode)); + VM_NEXT(); + } + + index++; + } + + // fallthrough to exit + pc++; + VM_NEXT(); + } + else + { + // note: it's safe to push arguments past top for complicated reasons (see top of the file) + setobjs2s(L, ra + 3 + 2, ra + 2); + setobjs2s(L, ra + 3 + 1, ra + 1); + setobjs2s(L, ra + 3, ra); + + L->top = ra + 3 + 3; /* func + 2 args (state and index) */ + LUAU_ASSERT(L->top <= L->stack_last); + + VM_PROTECT(luaD_call(L, ra + 3, aux)); + L->top = L->ci->top; + + // recompute ra since stack might have been reallocated + ra = VM_REG(LUAU_INSN_A(insn)); + + // copy first variable back into the iteration index + setobjs2s(L, ra + 2, ra + 3); + + // note that we need to increment pc by 1 to exit the loop since we need to skip over aux + pc += ttisnil(ra + 3) ? 1 : LUAU_INSN_D(insn); + LUAU_ASSERT(unsigned(pc - cl->l.p->code) < unsigned(cl->l.p->sizecode)); + VM_NEXT(); + } } VM_CASE(LOP_FORGPREP_INEXT) @@ -2228,8 +2372,15 @@ static void luau_execute(lua_State* L) // fast-path: ipairs/inext if (cl->env->safeenv && ttistable(ra + 1) && ttisnumber(ra + 2) && nvalue(ra + 2) == 0.0) { + if (FFlag::LuauIter) + setnilvalue(ra); + setpvalue(ra + 2, reinterpret_cast(uintptr_t(0))); } + else if (FFlag::LuauIter && !ttisfunction(ra)) + { + VM_PROTECT(luaG_typeerror(L, ra, "iterate over")); + } pc += LUAU_INSN_D(insn); LUAU_ASSERT(unsigned(pc - cl->l.p->code) < unsigned(cl->l.p->sizecode)); @@ -2268,23 +2419,9 @@ static void luau_execute(lua_State* L) VM_NEXT(); } } - else if (FFlag::LuauTableNewBoundary2 || (h->lsizenode == 0 && ttisnil(gval(h->node)))) - { - // fallthrough to exit - VM_NEXT(); - } else { - // the table has a hash part; index + 1 may appear in it in which case we need to iterate through the hash portion as well - const TValue* val = luaH_getnum(h, index + 1); - - setpvalue(ra + 2, reinterpret_cast(uintptr_t(index + 1))); - setnvalue(ra + 3, double(index + 1)); - setobj2s(L, ra + 4, val); - - // note that nil elements inside the array terminate the traversal - pc += ttisnil(ra + 4) ? 0 : LUAU_INSN_D(insn); - LUAU_ASSERT(unsigned(pc - cl->l.p->code) < unsigned(cl->l.p->sizecode)); + // fallthrough to exit VM_NEXT(); } } @@ -2308,8 +2445,15 @@ static void luau_execute(lua_State* L) // fast-path: pairs/next if (cl->env->safeenv && ttistable(ra + 1) && ttisnil(ra + 2)) { + if (FFlag::LuauIter) + setnilvalue(ra); + setpvalue(ra + 2, reinterpret_cast(uintptr_t(0))); } + else if (FFlag::LuauIter && !ttisfunction(ra)) + { + VM_PROTECT(luaG_typeerror(L, ra, "iterate over")); + } pc += LUAU_INSN_D(insn); LUAU_ASSERT(unsigned(pc - cl->l.p->code) < unsigned(cl->l.p->sizecode)); @@ -2704,7 +2848,7 @@ static void luau_execute(lua_State* L) { VM_PROTECT_PC(); - int n = f(L, ra, arg, nresults, nullptr, nparams); + int n = f(L, ra, arg, nresults, NULL, nparams); if (n >= 0) { diff --git a/bench/micro_tests/test_LargeTableSum_loop_iter.lua b/bench/micro_tests/test_LargeTableSum_loop_iter.lua new file mode 100644 index 00000000..057420f6 --- /dev/null +++ b/bench/micro_tests/test_LargeTableSum_loop_iter.lua @@ -0,0 +1,17 @@ +local bench = script and require(script.Parent.bench_support) or require("bench_support") + +function test() + + local t = {} + + for i=1,1000000 do t[i] = i end + + local ts0 = os.clock() + local sum = 0 + for k,v in t do sum = sum + v end + local ts1 = os.clock() + + return ts1-ts0 +end + +bench.runCode(test, "LargeTableSum: for k,v in {}") diff --git a/bench/tests/sunspider/3d-cube.lua b/bench/tests/sunspider/3d-cube.lua index 5d162ab9..77fa0854 100644 --- a/bench/tests/sunspider/3d-cube.lua +++ b/bench/tests/sunspider/3d-cube.lua @@ -25,7 +25,7 @@ local DisplArea = {} DisplArea.Width = 300; DisplArea.Height = 300; -function DrawLine(From, To) +local function DrawLine(From, To) local x1 = From.V[1]; local x2 = To.V[1]; local y1 = From.V[2]; @@ -81,7 +81,7 @@ function DrawLine(From, To) Q.LastPx = NumPix; end -function CalcCross(V0, V1) +local function CalcCross(V0, V1) local Cross = {}; Cross[1] = V0[2]*V1[3] - V0[3]*V1[2]; Cross[2] = V0[3]*V1[1] - V0[1]*V1[3]; @@ -89,7 +89,7 @@ function CalcCross(V0, V1) return Cross; end -function CalcNormal(V0, V1, V2) +local function CalcNormal(V0, V1, V2) local A = {}; local B = {}; for i = 1,3 do A[i] = V0[i] - V1[i]; @@ -102,14 +102,14 @@ function CalcNormal(V0, V1, V2) return A; end -function CreateP(X,Y,Z) +local function CreateP(X,Y,Z) local result = {} result.V = {X,Y,Z,1}; return result end -- multiplies two matrices -function MMulti(M1, M2) +local function MMulti(M1, M2) local M = {{},{},{},{}}; for i = 1,4 do for j = 1,4 do @@ -120,7 +120,7 @@ function MMulti(M1, M2) end -- multiplies matrix with vector -function VMulti(M, V) +local function VMulti(M, V) local Vect = {}; for i = 1,4 do Vect[i] = M[i][1] * V[1] + M[i][2] * V[2] + M[i][3] * V[3] + M[i][4] * V[4]; @@ -128,7 +128,7 @@ function VMulti(M, V) return Vect; end -function VMulti2(M, V) +local function VMulti2(M, V) local Vect = {}; for i = 1,3 do Vect[i] = M[i][1] * V[1] + M[i][2] * V[2] + M[i][3] * V[3]; @@ -137,7 +137,7 @@ function VMulti2(M, V) end -- add to matrices -function MAdd(M1, M2) +local function MAdd(M1, M2) local M = {{},{},{},{}}; for i = 1,4 do for j = 1,4 do @@ -147,7 +147,7 @@ function MAdd(M1, M2) return M; end -function Translate(M, Dx, Dy, Dz) +local function Translate(M, Dx, Dy, Dz) local T = { {1,0,0,Dx}, {0,1,0,Dy}, @@ -157,7 +157,7 @@ function Translate(M, Dx, Dy, Dz) return MMulti(T, M); end -function RotateX(M, Phi) +local function RotateX(M, Phi) local a = Phi; a = a * math.pi / 180; local Cos = math.cos(a); @@ -171,7 +171,7 @@ function RotateX(M, Phi) return MMulti(R, M); end -function RotateY(M, Phi) +local function RotateY(M, Phi) local a = Phi; a = a * math.pi / 180; local Cos = math.cos(a); @@ -185,7 +185,7 @@ function RotateY(M, Phi) return MMulti(R, M); end -function RotateZ(M, Phi) +local function RotateZ(M, Phi) local a = Phi; a = a * math.pi / 180; local Cos = math.cos(a); @@ -199,7 +199,7 @@ function RotateZ(M, Phi) return MMulti(R, M); end -function DrawQube() +local function DrawQube() -- calc current normals local CurN = {}; local i = 5; @@ -245,7 +245,7 @@ function DrawQube() Q.LastPx = 0; end -function Loop() +local function Loop() if (Testing.LoopCount > Testing.LoopMax) then return; end local TestingStr = tostring(Testing.LoopCount); while (#TestingStr < 3) do TestingStr = "0" .. TestingStr; end @@ -265,7 +265,7 @@ function Loop() Loop(); end -function Init(CubeSize) +local function Init(CubeSize) -- init/reset vars Origin.V = {150,150,20,1}; Testing.LoopCount = 0; diff --git a/bench/tests/sunspider/3d-morph.lua b/bench/tests/sunspider/3d-morph.lua index f73f173b..79e91419 100644 --- a/bench/tests/sunspider/3d-morph.lua +++ b/bench/tests/sunspider/3d-morph.lua @@ -31,7 +31,7 @@ local loops = 15 local nx = 120 local nz = 120 -function morph(a, f) +local function morph(a, f) local PI2nx = math.pi * 8/nx local sin = math.sin local f30 = -(50 * sin(f*math.pi*2)) diff --git a/bench/tests/sunspider/3d-raytrace.lua b/bench/tests/sunspider/3d-raytrace.lua index c8f6b5dc..3d5276c7 100644 --- a/bench/tests/sunspider/3d-raytrace.lua +++ b/bench/tests/sunspider/3d-raytrace.lua @@ -28,40 +28,40 @@ function test() local size = 30 -function createVector(x,y,z) +local function createVector(x,y,z) return { x,y,z }; end -function sqrLengthVector(self) +local function sqrLengthVector(self) return self[1] * self[1] + self[2] * self[2] + self[3] * self[3]; end -function lengthVector(self) +local function lengthVector(self) return math.sqrt(self[1] * self[1] + self[2] * self[2] + self[3] * self[3]); end -function addVector(self, v) +local function addVector(self, v) self[1] = self[1] + v[1]; self[2] = self[2] + v[2]; self[3] = self[3] + v[3]; return self; end -function subVector(self, v) +local function subVector(self, v) self[1] = self[1] - v[1]; self[2] = self[2] - v[2]; self[3] = self[3] - v[3]; return self; end -function scaleVector(self, scale) +local function scaleVector(self, scale) self[1] = self[1] * scale; self[2] = self[2] * scale; self[3] = self[3] * scale; return self; end -function normaliseVector(self) +local function normaliseVector(self) local len = math.sqrt(self[1] * self[1] + self[2] * self[2] + self[3] * self[3]); self[1] = self[1] / len; self[2] = self[2] / len; @@ -69,39 +69,39 @@ function normaliseVector(self) return self; end -function add(v1, v2) +local function add(v1, v2) return { v1[1] + v2[1], v1[2] + v2[2], v1[3] + v2[3] }; end -function sub(v1, v2) +local function sub(v1, v2) return { v1[1] - v2[1], v1[2] - v2[2], v1[3] - v2[3] }; end -function scalev(v1, v2) +local function scalev(v1, v2) return { v1[1] * v2[1], v1[2] * v2[2], v1[3] * v2[3] }; end -function dot(v1, v2) +local function dot(v1, v2) return v1[1] * v2[1] + v1[2] * v2[2] + v1[3] * v2[3]; end -function scale(v, scale) +local function scale(v, scale) return { v[1] * scale, v[2] * scale, v[3] * scale }; end -function cross(v1, v2) +local function cross(v1, v2) return { v1[2] * v2[3] - v1[3] * v2[2], v1[3] * v2[1] - v1[1] * v2[3], v1[1] * v2[2] - v1[2] * v2[1] }; end -function normalise(v) +local function normalise(v) local len = lengthVector(v); return { v[1] / len, v[2] / len, v[3] / len }; end -function transformMatrix(self, v) +local function transformMatrix(self, v) local vals = self; local x = vals[1] * v[1] + vals[2] * v[2] + vals[3] * v[3] + vals[4]; local y = vals[5] * v[1] + vals[6] * v[2] + vals[7] * v[3] + vals[8]; @@ -109,7 +109,7 @@ function transformMatrix(self, v) return { x, y, z }; end -function invertMatrix(self) +local function invertMatrix(self) local temp = {} local tx = -self[4]; local ty = -self[8]; @@ -131,7 +131,7 @@ function invertMatrix(self) end -- Triangle intersection using barycentric coord method -function Triangle(p1, p2, p3) +local function Triangle(p1, p2, p3) local this = {} local edge1 = sub(p3, p1); @@ -205,7 +205,7 @@ function Triangle(p1, p2, p3) return this end -function Scene(a_triangles) +local function Scene(a_triangles) local this = {} this.triangles = a_triangles; this.lights = {}; @@ -302,7 +302,7 @@ local zero = { 0,0,0 }; -- this camera code is from notes i made ages ago, it is from *somewhere* -- i cannot remember where -- that somewhere is -function Camera(origin, lookat, up) +local function Camera(origin, lookat, up) local this = {} local zaxis = normaliseVector(subVector(lookat, origin)); @@ -357,7 +357,7 @@ function Camera(origin, lookat, up) return this end -function raytraceScene() +local function raytraceScene() local startDate = 13154863; local numTriangles = 2 * 6; local triangles = {}; -- numTriangles); @@ -450,7 +450,7 @@ function raytraceScene() return pixels; end -function arrayToCanvasCommands(pixels) +local function arrayToCanvasCommands(pixels) local s = {}; table.insert(s, 'Test\nvar pixels = ['); for y = 0,size-1 do @@ -485,7 +485,7 @@ for (var y = 0; y < size; y++) {\n\ return table.concat(s); end -testOutput = arrayToCanvasCommands(raytraceScene()); +local testOutput = arrayToCanvasCommands(raytraceScene()); --local f = io.output("output.html") --f:write(testOutput) diff --git a/bench/tests/sunspider/access-binary-trees.lua b/bench/tests/sunspider/access-binary-trees.lua deleted file mode 100644 index 9eb93588..00000000 --- a/bench/tests/sunspider/access-binary-trees.lua +++ /dev/null @@ -1,69 +0,0 @@ ---[[ - The Great Computer Language Shootout - http://shootout.alioth.debian.org/ - contributed by Isaac Gouy -]] - -local bench = script and require(script.Parent.bench_support) or require("bench_support") - -function test() - -function TreeNode(left,right,item) - local this = {} - this.left = left; - this.right = right; - this.item = item; - - this.itemCheck = function(self) - if (self.left==nil) then return self.item; - else return self.item + self.left:itemCheck() - self.right:itemCheck(); end - end - - return this -end - -function bottomUpTree(item,depth) - if (depth>0) then - return TreeNode( - bottomUpTree(2*item-1, depth-1) - ,bottomUpTree(2*item, depth-1) - ,item - ); - else - return TreeNode(nil,nil,item); - end -end - -local ret = 0; - -for n = 4,7,1 do - local minDepth = 4; - local maxDepth = math.max(minDepth + 2, n); - local stretchDepth = maxDepth + 1; - - local check = bottomUpTree(0,stretchDepth):itemCheck(); - - local longLivedTree = bottomUpTree(0,maxDepth); - - for depth = minDepth,maxDepth,2 do - local iterations = 2.0 ^ (maxDepth - depth + minDepth - 1) -- 1 << (maxDepth - depth + minDepth); - - check = 0; - for i = 1,iterations do - check = check + bottomUpTree(i,depth):itemCheck(); - check = check + bottomUpTree(-i,depth):itemCheck(); - end - end - - ret = ret + longLivedTree:itemCheck(); -end - -local expected = -4; - -if (ret ~= expected) then - assert(false, "ERROR: bad result: expected " .. expected .. " but got " .. ret); -end - -end - -bench.runCode(test, "access-binary-trees") diff --git a/bench/tests/sunspider/controlflow-recursive.lua b/bench/tests/sunspider/controlflow-recursive.lua index d0791626..a2591b2f 100644 --- a/bench/tests/sunspider/controlflow-recursive.lua +++ b/bench/tests/sunspider/controlflow-recursive.lua @@ -7,18 +7,18 @@ local bench = script and require(script.Parent.bench_support) or require("bench_ function test() -function ack(m,n) +local function ack(m,n) if (m==0) then return n+1; end if (n==0) then return ack(m-1,1); end return ack(m-1, ack(m,n-1) ); end -function fib(n) +local function fib(n) if (n < 2) then return 1; end return fib(n-2) + fib(n-1); end -function tak(x,y,z) +local function tak(x,y,z) if (y >= x) then return z; end return tak(tak(x-1,y,z), tak(y-1,z,x), tak(z-1,x,y)); end @@ -27,7 +27,7 @@ local result = 0; for i = 3,5 do result = result + ack(3,i); - result = result + fib(17.0+i); + result = result + fib(17+i); result = result + tak(3*i+3,2*i+2,i+1); end diff --git a/bench/tests/sunspider/crypto-aes.lua b/bench/tests/sunspider/crypto-aes.lua index 3b289729..8dd0cec6 100644 --- a/bench/tests/sunspider/crypto-aes.lua +++ b/bench/tests/sunspider/crypto-aes.lua @@ -42,7 +42,68 @@ local Rcon = { { 0x00, 0x00, 0x00, 0x00 }, {0x1b, 0x00, 0x00, 0x00}, {0x36, 0x00, 0x00, 0x00} }; -function Cipher(input, w) -- main Cipher function [§5.1] +local function SubBytes(s, Nb) -- apply SBox to state S [§5.1.1] + for r = 0,3 do + for c = 0,Nb-1 do s[r + 1][c + 1] = Sbox[s[r + 1][c + 1] + 1]; end + end + return s; +end + + +local function ShiftRows(s, Nb) -- shift row r of state S left by r bytes [§5.1.2] + local t = {}; + for r = 1,3 do + for c = 0,3 do t[c + 1] = s[r + 1][((c + r) % Nb) + 1] end; -- shift into temp copy + for c = 0,3 do s[r + 1][c + 1] = t[c + 1]; end -- and copy back + end -- note that this will work for Nb=4,5,6, but not 7,8 (always 4 for AES): + return s; -- see fp.gladman.plus.com/cryptography_technology/rijndael/aes.spec.311.pdf +end + + +local function MixColumns(s, Nb) -- combine bytes of each col of state S [§5.1.3] + for c = 0,3 do + local a = {}; -- 'a' is a copy of the current column from 's' + local b = {}; -- 'b' is a•{02} in GF(2^8) + for i = 0,3 do + a[i + 1] = s[i + 1][c + 1]; + + if bit32.band(s[i + 1][c + 1], 0x80) ~= 0 then + b[i + 1] = bit32.bxor(bit32.lshift(s[i + 1][c + 1], 1), 0x011b); + else + b[i + 1] = bit32.lshift(s[i + 1][c + 1], 1); + end + end + -- a[n] ^ b[n] is a•{03} in GF(2^8) + s[1][c + 1] = bit32.bxor(b[1], a[2], b[2], a[3], a[4]); -- 2*a0 + 3*a1 + a2 + a3 + s[2][c + 1] = bit32.bxor(a[1], b[2], a[3], b[3], a[4]); -- a0 * 2*a1 + 3*a2 + a3 + s[3][c + 1] = bit32.bxor(a[1], a[2], b[3], a[4], b[4]); -- a0 + a1 + 2*a2 + 3*a3 + s[4][c + 1] = bit32.bxor(a[1], b[1], a[2], a[3], b[4]); -- 3*a0 + a1 + a2 + 2*a3 +end + return s; +end + + +local function SubWord(w) -- apply SBox to 4-byte word w + for i = 0,3 do w[i + 1] = Sbox[w[i + 1] + 1]; end + return w; +end + +local function RotWord(w) -- rotate 4-byte word w left by one byte + w[5] = w[1]; + for i = 0,3 do w[i + 1] = w[i + 2]; end + return w; +end + + + +local function AddRoundKey(state, w, rnd, Nb) -- xor Round Key into state S [§5.1.4] + for r = 0,3 do + for c = 0,Nb-1 do state[r + 1][c + 1] = bit32.bxor(state[r + 1][c + 1], w[rnd*4+c + 1][r + 1]); end + end + return state; +end + +local function Cipher(input, w) -- main Cipher function [§5.1] local Nb = 4; -- block size (in words): no of columns in state (fixed at 4 for AES) local Nr = #w / Nb - 1; -- no of rounds: 10/12/14 for 128/192/256-bit keys @@ -69,56 +130,7 @@ function Cipher(input, w) -- main Cipher function [§5.1] end -function SubBytes(s, Nb) -- apply SBox to state S [§5.1.1] - for r = 0,3 do - for c = 0,Nb-1 do s[r + 1][c + 1] = Sbox[s[r + 1][c + 1] + 1]; end - end - return s; -end - - -function ShiftRows(s, Nb) -- shift row r of state S left by r bytes [§5.1.2] - local t = {}; - for r = 1,3 do - for c = 0,3 do t[c + 1] = s[r + 1][((c + r) % Nb) + 1] end; -- shift into temp copy - for c = 0,3 do s[r + 1][c + 1] = t[c + 1]; end -- and copy back - end -- note that this will work for Nb=4,5,6, but not 7,8 (always 4 for AES): - return s; -- see fp.gladman.plus.com/cryptography_technology/rijndael/aes.spec.311.pdf -end - - -function MixColumns(s, Nb) -- combine bytes of each col of state S [§5.1.3] - for c = 0,3 do - local a = {}; -- 'a' is a copy of the current column from 's' - local b = {}; -- 'b' is a•{02} in GF(2^8) - for i = 0,3 do - a[i + 1] = s[i + 1][c + 1]; - - if bit32.band(s[i + 1][c + 1], 0x80) ~= 0 then - b[i + 1] = bit32.bxor(bit32.lshift(s[i + 1][c + 1], 1), 0x011b); - else - b[i + 1] = bit32.lshift(s[i + 1][c + 1], 1); - end - end - -- a[n] ^ b[n] is a•{03} in GF(2^8) - s[1][c + 1] = bit32.bxor(b[1], a[2], b[2], a[3], a[4]); -- 2*a0 + 3*a1 + a2 + a3 - s[2][c + 1] = bit32.bxor(a[1], b[2], a[3], b[3], a[4]); -- a0 * 2*a1 + 3*a2 + a3 - s[3][c + 1] = bit32.bxor(a[1], a[2], b[3], a[4], b[4]); -- a0 + a1 + 2*a2 + 3*a3 - s[4][c + 1] = bit32.bxor(a[1], b[1], a[2], a[3], b[4]); -- 3*a0 + a1 + a2 + 2*a3 -end - return s; -end - - -function AddRoundKey(state, w, rnd, Nb) -- xor Round Key into state S [§5.1.4] - for r = 0,3 do - for c = 0,Nb-1 do state[r + 1][c + 1] = bit32.bxor(state[r + 1][c + 1], w[rnd*4+c + 1][r + 1]); end - end - return state; -end - - -function KeyExpansion(key) -- generate Key Schedule (byte-array Nr+1 x Nb) from Key [§5.2] +local function KeyExpansion(key) -- generate Key Schedule (byte-array Nr+1 x Nb) from Key [§5.2] local Nb = 4; -- block size (in words): no of columns in state (fixed at 4 for AES) local Nk = #key / 4 -- key length (in words): 4/6/8 for 128/192/256-bit keys local Nr = Nk + 6; -- no of rounds: 10/12/14 for 128/192/256-bit keys @@ -146,17 +158,17 @@ function KeyExpansion(key) -- generate Key Schedule (byte-array Nr+1 x Nb) from return w; end -function SubWord(w) -- apply SBox to 4-byte word w - for i = 0,3 do w[i + 1] = Sbox[w[i + 1] + 1]; end - return w; +local function escCtrlChars(str) -- escape control chars which might cause problems handling ciphertext + return string.gsub(str, "[\0\t\n\v\f\r\'\"!-]", function(c) return '!' .. string.byte(c, 1) .. '!'; end); end -function RotWord(w) -- rotate 4-byte word w left by one byte - w[5] = w[1]; - for i = 0,3 do w[i + 1] = w[i + 2]; end - return w; -end +local function unescCtrlChars(str) -- unescape potentially problematic control characters + return string.gsub(str, "!%d%d?%d?!", function(c) + local sc = string.sub(c, 2,-2) + return string.char(tonumber(sc)); + end); +end --[[ * Use AES to encrypt 'plaintext' with 'password' using 'nBits' key, in 'Counter' mode of operation @@ -166,7 +178,7 @@ end * - cipherblock = plaintext xor outputblock ]] -function AESEncryptCtr(plaintext, password, nBits) +local function AESEncryptCtr(plaintext, password, nBits) if (not (nBits==128 or nBits==192 or nBits==256)) then return ''; end -- standard allows 128/192/256 bit keys -- for this example script, generate the key by applying Cipher to 1st 16/24/32 chars of password; @@ -243,7 +255,7 @@ end * - cipherblock = plaintext xor outputblock ]] -function AESDecryptCtr(ciphertext, password, nBits) +local function AESDecryptCtr(ciphertext, password, nBits) if (not (nBits==128 or nBits==192 or nBits==256)) then return ''; end -- standard allows 128/192/256 bit keys local nBytes = nBits/8; -- no bytes in key @@ -300,19 +312,7 @@ function AESDecryptCtr(ciphertext, password, nBits) return table.concat(plaintext) end -function escCtrlChars(str) -- escape control chars which might cause problems handling ciphertext - return string.gsub(str, "[\0\t\n\v\f\r\'\"!-]", function(c) return '!' .. string.byte(c, 1) .. '!'; end); -end - -function unescCtrlChars(str) -- unescape potentially problematic control characters - return string.gsub(str, "!%d%d?%d?!", function(c) - local sc = string.sub(c, 2,-2) - - return string.char(tonumber(sc)); - end); -end - -function test() +local function test() local plainText = "ROMEO: But, soft! what light through yonder window breaks?\n\ It is the east, and Juliet is the sun.\n\ diff --git a/bench/tests/sunspider/math-cordic.lua b/bench/tests/sunspider/math-cordic.lua index 94a64f45..cdb10fa2 100644 --- a/bench/tests/sunspider/math-cordic.lua +++ b/bench/tests/sunspider/math-cordic.lua @@ -31,15 +31,15 @@ function test() local AG_CONST = 0.6072529350; -function FIXED(X) +local function FIXED(X) return X * 65536.0; end -function FLOAT(X) +local function FLOAT(X) return X / 65536.0; end -function DEG2RAD(X) +local function DEG2RAD(X) return 0.017453 * (X); end @@ -52,7 +52,7 @@ local Angles = { local Target = 28.027; -function cordicsincos(Target) +local function cordicsincos(Target) local X; local Y; local TargetAngle; @@ -85,7 +85,7 @@ end local total = 0; -function cordic( runs ) +local function cordic( runs ) for i = 1,runs do total = total + cordicsincos(Target); end diff --git a/bench/tests/sunspider/math-partial-sums.lua b/bench/tests/sunspider/math-partial-sums.lua index 3c222876..9977ceff 100644 --- a/bench/tests/sunspider/math-partial-sums.lua +++ b/bench/tests/sunspider/math-partial-sums.lua @@ -7,7 +7,7 @@ local bench = script and require(script.Parent.bench_support) or require("bench_ function test() -function partial(n) +local function partial(n) local a1, a2, a3, a4, a5, a6, a7, a8, a9 = 0, 0, 0, 0, 0, 0, 0, 0, 0; local twothirds = 2.0/3.0; local alt = -1.0; diff --git a/bench/tests/sunspider/math-spectral-norm.lua b/bench/tests/sunspider/math-spectral-norm.lua deleted file mode 100644 index 7d7ec163..00000000 --- a/bench/tests/sunspider/math-spectral-norm.lua +++ /dev/null @@ -1,72 +0,0 @@ ---[[ -The Great Computer Language Shootout -http://shootout.alioth.debian.org/ - -contributed by Ian Osgood -]] -local bench = script and require(script.Parent.bench_support) or require("bench_support") - -function test() - -function A(i,j) - return 1/((i+j)*(i+j+1)/2+i+1); -end - -function Au(u,v) - for i = 0,#u-1 do - local t = 0; - for j = 0,#u-1 do - t = t + A(i,j) * u[j + 1]; - end - v[i + 1] = t; - end -end - -function Atu(u,v) - for i = 0,#u-1 do - local t = 0; - for j = 0,#u-1 do - t = t + A(j,i) * u[j + 1]; - end - v[i + 1] = t; - end -end - -function AtAu(u,v,w) - Au(u,w); - Atu(w,v); -end - -function spectralnorm(n) - local u, v, w, vv, vBv = {}, {}, {}, 0, 0; - for i = 1,n do - u[i] = 1; v[i] = 0; w[i] = 0; - end - for i = 0,9 do - AtAu(u,v,w); - AtAu(v,u,w); - end - for i = 1,n do - vBv = vBv + u[i]*v[i]; - vv = vv + v[i]*v[i]; - end - return math.sqrt(vBv/vv); -end - -local total = 0; -local i = 6 - -while i <= 48 do - total = total + spectralnorm(i); - i = i * 2 -end - -local expected = 5.086694231303284; - -if (total ~= expected) then - assert(false, "ERROR: bad result: expected " .. expected .. " but got " .. total) -end - -end - -bench.runCode(test, "math-spectral-norm") diff --git a/tests/Autocomplete.test.cpp b/tests/Autocomplete.test.cpp index 5b70481b..1c284f1f 100644 --- a/tests/Autocomplete.test.cpp +++ b/tests/Autocomplete.test.cpp @@ -2760,8 +2760,6 @@ TEST_CASE_FIXTURE(ACFixture, "autocomplete_on_string_singletons") TEST_CASE_FIXTURE(ACFixture, "autocomplete_string_singletons") { - ScopedFastFlag luauAutocompleteSingletonTypes{"LuauAutocompleteSingletonTypes", true}; - check(R"( type tag = "cat" | "dog" local function f(a: tag) end @@ -2798,8 +2796,6 @@ TEST_CASE_FIXTURE(ACFixture, "autocomplete_string_singletons") TEST_CASE_FIXTURE(ACFixture, "autocomplete_string_singleton_equality") { - ScopedFastFlag luauAutocompleteSingletonTypes{"LuauAutocompleteSingletonTypes", true}; - check(R"( type tagged = {tag:"cat", fieldx:number} | {tag:"dog", fieldy:number} local x: tagged = {tag="cat", fieldx=2} @@ -2821,8 +2817,6 @@ TEST_CASE_FIXTURE(ACFixture, "autocomplete_string_singleton_equality") TEST_CASE_FIXTURE(ACFixture, "autocomplete_boolean_singleton") { - ScopedFastFlag luauAutocompleteSingletonTypes{"LuauAutocompleteSingletonTypes", true}; - check(R"( local function f(x: true) end f(@1) @@ -2838,8 +2832,6 @@ f(@1) TEST_CASE_FIXTURE(ACFixture, "autocomplete_string_singleton_escape") { - ScopedFastFlag luauAutocompleteSingletonTypes{"LuauAutocompleteSingletonTypes", true}; - check(R"( type tag = "strange\t\"cat\"" | 'nice\t"dog"' local function f(x: tag) end diff --git a/tests/Compiler.test.cpp b/tests/Compiler.test.cpp index 7b4bfc72..f206438f 100644 --- a/tests/Compiler.test.cpp +++ b/tests/Compiler.test.cpp @@ -261,6 +261,9 @@ RETURN R0 0 TEST_CASE("ForBytecode") { + ScopedFastFlag sff("LuauCompileIter", true); + ScopedFastFlag sff2("LuauCompileIterNoPairs", false); + // basic for loop: variable directly refers to internal iteration index (R2) CHECK_EQ("\n" + compileFunction0("for i=1,5 do print(i) end"), R"( LOADN R2 1 @@ -295,7 +298,7 @@ GETIMPORT R0 2 LOADK R1 K3 LOADK R2 K4 CALL R0 2 3 -JUMP +4 +FORGPREP R0 +4 GETIMPORT R5 6 MOVE R6 R3 CALL R5 1 0 @@ -347,6 +350,8 @@ RETURN R0 0 TEST_CASE("ForBytecodeBuiltin") { + ScopedFastFlag sff("LuauCompileIter", true); + // we generally recognize builtins like pairs/ipairs and emit special opcodes CHECK_EQ("\n" + compileFunction0("for k,v in ipairs({}) do end"), R"( GETIMPORT R0 1 @@ -385,7 +390,7 @@ GETIMPORT R0 3 MOVE R1 R0 NEWTABLE R2 0 0 CALL R1 1 3 -JUMP +0 +FORGPREP R1 +0 FORGLOOP R1 -1 2 RETURN R0 0 )"); @@ -397,7 +402,7 @@ SETGLOBAL R0 K2 GETGLOBAL R0 K2 NEWTABLE R1 0 0 CALL R0 1 3 -JUMP +0 +FORGPREP R0 +0 FORGLOOP R0 -1 2 RETURN R0 0 )"); @@ -407,7 +412,7 @@ RETURN R0 0 GETIMPORT R0 1 NEWTABLE R1 0 0 CALL R0 1 3 -JUMP +0 +FORGPREP R0 +0 FORGLOOP R0 -1 2 RETURN R0 0 )"); @@ -2260,6 +2265,8 @@ TEST_CASE("TypeAliasing") TEST_CASE("DebugLineInfo") { + ScopedFastFlag sff("LuauCompileIterNoPairs", false); + Luau::BytecodeBuilder bcb; bcb.setDumpFlags(Luau::BytecodeBuilder::Dump_Code | Luau::BytecodeBuilder::Dump_Lines); Luau::compileOrThrow(bcb, R"( @@ -2316,6 +2323,8 @@ return result TEST_CASE("DebugLineInfoFor") { + ScopedFastFlag sff("LuauCompileIter", true); + Luau::BytecodeBuilder bcb; bcb.setDumpFlags(Luau::BytecodeBuilder::Dump_Code | Luau::BytecodeBuilder::Dump_Lines); Luau::compileOrThrow(bcb, R"( @@ -2336,7 +2345,7 @@ end 5: LOADN R0 1 7: LOADN R1 2 9: LOADN R2 3 -9: JUMP +4 +9: FORGPREP R0 +4 11: GETIMPORT R5 1 11: MOVE R6 R3 11: CALL R5 1 0 @@ -2541,6 +2550,8 @@ a TEST_CASE("DebugSource") { + ScopedFastFlag sff("LuauCompileIterNoPairs", false); + const char* source = R"( local kSelectedBiomes = { ['Mountains'] = true, @@ -2616,6 +2627,8 @@ RETURN R1 1 TEST_CASE("DebugLocals") { + ScopedFastFlag sff("LuauCompileIterNoPairs", false); + const char* source = R"( function foo(e, f) local a = 1 @@ -3767,6 +3780,8 @@ RETURN R0 1 TEST_CASE("SharedClosure") { + ScopedFastFlag sff("LuauCompileIterNoPairs", false); + // closures can be shared even if functions refer to upvalues, as long as upvalues are top-level CHECK_EQ("\n" + compileFunction(R"( local val = ... @@ -4452,5 +4467,688 @@ RETURN R0 0 )"); } +TEST_CASE("InlineBasic") +{ + ScopedFastFlag sff("LuauCompileSupportInlining", true); + + // inline function that returns a constant + CHECK_EQ("\n" + compileFunction(R"( +local function foo() + return 42 +end + +local x = foo() +return x +)", + 1, 2), + R"( +DUPCLOSURE R0 K0 +LOADN R1 42 +RETURN R1 1 +)"); + + // inline function that returns the argument + CHECK_EQ("\n" + compileFunction(R"( +local function foo(a) + return a +end + +local x = foo(42) +return x +)", + 1, 2), + R"( +DUPCLOSURE R0 K0 +LOADN R1 42 +RETURN R1 1 +)"); + + // inline function that returns one of the two arguments + CHECK_EQ("\n" + compileFunction(R"( +local function foo(a, b, c) + if a then + return b + else + return c + end +end + +local x = foo(true, math.random(), 5) +return x +)", + 1, 2), + R"( +DUPCLOSURE R0 K0 +GETIMPORT R2 3 +CALL R2 0 1 +MOVE R1 R2 +RETURN R1 1 +RETURN R1 1 +)"); + + // inline function that returns one of the two arguments + CHECK_EQ("\n" + compileFunction(R"( +local function foo(a, b, c) + if a then + return b + else + return c + end +end + +local x = foo(true, 5, math.random()) +return x +)", + 1, 2), + R"( +DUPCLOSURE R0 K0 +GETIMPORT R2 3 +CALL R2 0 1 +LOADN R1 5 +RETURN R1 1 +RETURN R1 1 +)"); +} + +TEST_CASE("InlineMutate") +{ + ScopedFastFlag sff("LuauCompileSupportInlining", true); + + // if the argument is mutated, it gets a register even if the value is constant + CHECK_EQ("\n" + compileFunction(R"( +local function foo(a) + a = a or 5 + return a +end + +local x = foo(42) +return x +)", + 1, 2), + R"( +DUPCLOSURE R0 K0 +LOADN R2 42 +ORK R2 R2 K1 +MOVE R1 R2 +RETURN R1 1 +)"); + + // if the argument is a local, it can be used directly + CHECK_EQ("\n" + compileFunction(R"( +local function foo(a) + return a +end + +local x = ... +local y = foo(x) +return y +)", + 1, 2), + R"( +DUPCLOSURE R0 K0 +GETVARARGS R1 1 +MOVE R2 R1 +RETURN R2 1 +)"); + + // ... but if it's mutated, we move it in case it is mutated through a capture during the inlined function + CHECK_EQ("\n" + compileFunction(R"( +local function foo(a) + return a +end + +local x = ... +x = nil +local y = foo(x) +return y +)", + 1, 2), + R"( +DUPCLOSURE R0 K0 +GETVARARGS R1 1 +LOADNIL R1 +MOVE R3 R1 +MOVE R2 R3 +RETURN R2 1 +)"); + + // we also don't inline functions if they have been assigned to + CHECK_EQ("\n" + compileFunction(R"( +local function foo(a) + return a +end + +foo = foo + +local x = foo(42) +return x +)", + 1, 2), + R"( +DUPCLOSURE R0 K0 +MOVE R0 R0 +MOVE R1 R0 +LOADN R2 42 +CALL R1 1 1 +RETURN R1 1 +)"); +} + +TEST_CASE("InlineUpval") +{ + ScopedFastFlag sff("LuauCompileSupportInlining", true); + + // if the argument is an upvalue, we naturally need to copy it to a local + CHECK_EQ("\n" + compileFunction(R"( +local function foo(a) + return a +end + +local b = ... + +function bar() + local x = foo(b) + return x +end +)", + 1, 2), + R"( +GETUPVAL R1 0 +MOVE R0 R1 +RETURN R0 1 +)"); + + // if the function uses an upvalue it's more complicated, because the lexical upvalue may become a local + CHECK_EQ("\n" + compileFunction(R"( +local b = ... + +local function foo(a) + return a + b +end + +local x = foo(42) +return x +)", + 1, 2), + R"( +GETVARARGS R0 1 +DUPCLOSURE R1 K0 +CAPTURE VAL R0 +LOADN R3 42 +ADD R2 R3 R0 +RETURN R2 1 +)"); + + // sometimes the lexical upvalue is deep enough that it's still an upvalue though + CHECK_EQ("\n" + compileFunction(R"( +local b = ... + +function bar() + local function foo(a) + return a + b + end + + local x = foo(42) + return x +end +)", + 1, 2), + R"( +DUPCLOSURE R0 K0 +CAPTURE UPVAL U0 +LOADN R2 42 +GETUPVAL R3 0 +ADD R1 R2 R3 +RETURN R1 1 +)"); +} + +TEST_CASE("InlineFallthrough") +{ + ScopedFastFlag sff("LuauCompileSupportInlining", true); + + // if the function doesn't return, we still fill the results with nil + CHECK_EQ("\n" + compileFunction(R"( +local function foo() +end + +local a, b = foo() + +return a, b +)", + 1, 2), + R"( +DUPCLOSURE R0 K0 +LOADNIL R1 +LOADNIL R2 +MOVE R3 R1 +MOVE R4 R2 +RETURN R3 2 +)"); + + // this happens even if the function returns conditionally + CHECK_EQ("\n" + compileFunction(R"( +local function foo(a) + if a then return 42 end +end + +local a, b = foo(false) + +return a, b +)", + 1, 2), + R"( +DUPCLOSURE R0 K0 +LOADNIL R1 +LOADNIL R2 +MOVE R3 R1 +MOVE R4 R2 +RETURN R3 2 +)"); + + // note though that we can't inline a function like this in multret context + // this is because we don't have a SETTOP instruction + CHECK_EQ("\n" + compileFunction(R"( +local function foo() +end + +return foo() +)", + 1, 2), + R"( +DUPCLOSURE R0 K0 +MOVE R1 R0 +CALL R1 0 -1 +RETURN R1 -1 +)"); +} + +TEST_CASE("InlineCapture") +{ + ScopedFastFlag sff("LuauCompileSupportInlining", true); + + // can't inline function with nested functions that capture locals because they might be constants + CHECK_EQ("\n" + compileFunction(R"( +local function foo(a) + local function bar() + return a + end + return bar() +end +)", + 1, 2), + R"( +NEWCLOSURE R1 P0 +CAPTURE VAL R0 +MOVE R2 R1 +CALL R2 0 -1 +RETURN R2 -1 +)"); +} + +TEST_CASE("InlineArgMismatch") +{ + ScopedFastFlag sff("LuauCompileSupportInlining", true); + + // when inlining a function, we must respect all the usual rules + + // caller might not have enough arguments + // TODO: we don't inline this atm + CHECK_EQ("\n" + compileFunction(R"( +local function foo(a) + return a +end + +local x = foo() +return x +)", + 1, 2), + R"( +DUPCLOSURE R0 K0 +MOVE R1 R0 +CALL R1 0 1 +RETURN R1 1 +)"); + + // caller might be using multret for arguments + // TODO: we don't inline this atm + CHECK_EQ("\n" + compileFunction(R"( +local function foo(a, b) + return a + b +end + +local x = foo(math.modf(1.5)) +return x +)", + 1, 2), + R"( +DUPCLOSURE R0 K0 +MOVE R1 R0 +LOADK R3 K1 +FASTCALL1 20 R3 +2 +GETIMPORT R2 4 +CALL R2 1 -1 +CALL R1 -1 1 +RETURN R1 1 +)"); + + // caller might have too many arguments, but we still need to compute them for side effects + // TODO: we don't inline this atm + CHECK_EQ("\n" + compileFunction(R"( +local function foo(a) + return a +end + +local x = foo(42, print()) +return x +)", + 1, 2), + R"( +DUPCLOSURE R0 K0 +MOVE R1 R0 +LOADN R2 42 +GETIMPORT R3 2 +CALL R3 0 -1 +CALL R1 -1 1 +RETURN R1 1 +)"); +} + +TEST_CASE("InlineMultiple") +{ + ScopedFastFlag sff("LuauCompileSupportInlining", true); + + // we call this with a different set of variable/constant args + CHECK_EQ("\n" + compileFunction(R"( +local function foo(a, b) + return a + b +end + +local x, y = ... +local a = foo(x, 1) +local b = foo(1, x) +local c = foo(1, 2) +local d = foo(x, y) +return a, b, c, d +)", + 1, 2), + R"( +DUPCLOSURE R0 K0 +GETVARARGS R1 2 +ADDK R3 R1 K1 +LOADN R5 1 +ADD R4 R5 R1 +LOADN R5 3 +ADD R6 R1 R2 +MOVE R7 R3 +MOVE R8 R4 +MOVE R9 R5 +MOVE R10 R6 +RETURN R7 4 +)"); +} + +TEST_CASE("InlineChain") +{ + ScopedFastFlag sff("LuauCompileSupportInlining", true); + + // inline a chain of functions + CHECK_EQ("\n" + compileFunction(R"( +local function foo(a, b) + return a + b +end + +local function bar(x) + return foo(x, 1) * foo(x, -1) +end + +local function baz() + return (bar(42)) +end + +return (baz()) +)", + 3, 2), + R"( +DUPCLOSURE R0 K0 +DUPCLOSURE R1 K1 +DUPCLOSURE R2 K2 +LOADN R4 43 +LOADN R5 41 +MUL R3 R4 R5 +RETURN R3 1 +)"); +} + +TEST_CASE("InlineThresholds") +{ + ScopedFastFlag sff("LuauCompileSupportInlining", true); + + ScopedFastInt sfis[] = { + {"LuauCompileInlineThreshold", 25}, + {"LuauCompileInlineThresholdMaxBoost", 300}, + {"LuauCompileInlineDepth", 2}, + }; + + // this function has enormous register pressure (50 regs) so we choose not to inline it + CHECK_EQ("\n" + compileFunction(R"( +local function foo() + return {{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{{}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}}} +end + +return (foo()) +)", + 1, 2), + R"( +DUPCLOSURE R0 K0 +MOVE R1 R0 +CALL R1 0 1 +RETURN R1 1 +)"); + + // this function has less register pressure but a large cost + CHECK_EQ("\n" + compileFunction(R"( +local function foo() + return {},{},{},{},{} +end + +return (foo()) +)", + 1, 2), + R"( +DUPCLOSURE R0 K0 +MOVE R1 R0 +CALL R1 0 1 +RETURN R1 1 +)"); + + // this chain of function is of length 3 but our limit in this test is 2, so we call foo twice + CHECK_EQ("\n" + compileFunction(R"( +local function foo(a, b) + return a + b +end + +local function bar(x) + return foo(x, 1) * foo(x, -1) +end + +local function baz() + return (bar(42)) +end + +return (baz()) +)", + 3, 2), + R"( +DUPCLOSURE R0 K0 +DUPCLOSURE R1 K1 +DUPCLOSURE R2 K2 +MOVE R4 R0 +LOADN R5 42 +LOADN R6 1 +CALL R4 2 1 +MOVE R5 R0 +LOADN R6 42 +LOADN R7 -1 +CALL R5 2 1 +MUL R3 R4 R5 +RETURN R3 1 +)"); +} + +TEST_CASE("InlineIIFE") +{ + ScopedFastFlag sff("LuauCompileSupportInlining", true); + + // IIFE with arguments + CHECK_EQ("\n" + compileFunction(R"( +function choose(a, b, c) + return ((function(a, b, c) if a then return b else return c end end)(a, b, c)) +end +)", + 1, 2), + R"( +JUMPIFNOT R0 +2 +MOVE R3 R1 +RETURN R3 1 +MOVE R3 R2 +RETURN R3 1 +RETURN R3 1 +)"); + + // IIFE with upvalues + CHECK_EQ("\n" + compileFunction(R"( +function choose(a, b, c) + return ((function() if a then return b else return c end end)()) +end +)", + 1, 2), + R"( +JUMPIFNOT R0 +2 +MOVE R3 R1 +RETURN R3 1 +MOVE R3 R2 +RETURN R3 1 +RETURN R3 1 +)"); +} + +TEST_CASE("InlineRecurseArguments") +{ + ScopedFastFlag sff("LuauCompileSupportInlining", true); + + // we can't inline a function if it's used to compute its own arguments + CHECK_EQ("\n" + compileFunction(R"( +local function foo(a, b) +end +foo(foo(foo,foo(foo,foo))[foo]) +)", + 1, 2), + R"( +DUPCLOSURE R0 K0 +MOVE R1 R0 +MOVE R4 R0 +MOVE R5 R0 +MOVE R6 R0 +CALL R4 2 1 +LOADNIL R3 +GETTABLE R2 R3 R0 +CALL R1 1 0 +RETURN R0 0 +)"); +} + +TEST_CASE("InlineFastCallK") +{ + ScopedFastFlag sff("LuauCompileSupportInlining", true); + + CHECK_EQ("\n" + compileFunction(R"( +local function set(l0) + rawset({}, l0) +end + +set(false) +set({}) +)", + 1, 2), + R"( +DUPCLOSURE R0 K0 +NEWTABLE R2 0 0 +FASTCALL2K 49 R2 K1 +4 +LOADK R3 K1 +GETIMPORT R1 3 +CALL R1 2 0 +NEWTABLE R1 0 0 +NEWTABLE R3 0 0 +FASTCALL2 49 R3 R1 +4 +MOVE R4 R1 +GETIMPORT R2 3 +CALL R2 2 0 +RETURN R0 0 +)"); +} + +TEST_CASE("InlineExprIndexK") +{ + ScopedFastFlag sff("LuauCompileSupportInlining", true); + + CHECK_EQ("\n" + compileFunction(R"( +local _ = function(l0) +local _ = nil +while _(_)[_] do +end +end +local _ = _(0)[""] +if _ then +do +for l0=0,8 do +end +end +elseif _ then +_ = nil +do +for l0=0,8 do +return true +end +end +end +)", + 1, 2), + R"( +DUPCLOSURE R0 K0 +LOADNIL R4 +LOADNIL R5 +CALL R4 1 1 +LOADNIL R5 +GETTABLE R3 R4 R5 +JUMPIFNOT R3 +1 +JUMPBACK -7 +LOADNIL R2 +GETTABLEKS R1 R2 K1 +JUMPIFNOT R1 +1 +RETURN R0 0 +JUMPIFNOT R1 +19 +LOADNIL R1 +LOADB R2 1 +RETURN R2 1 +LOADB R2 1 +RETURN R2 1 +LOADB R2 1 +RETURN R2 1 +LOADB R2 1 +RETURN R2 1 +LOADB R2 1 +RETURN R2 1 +LOADB R2 1 +RETURN R2 1 +LOADB R2 1 +RETURN R2 1 +LOADB R2 1 +RETURN R2 1 +LOADB R2 1 +RETURN R2 1 +RETURN R0 0 +)"); +} TEST_SUITE_END(); diff --git a/tests/Conformance.test.cpp b/tests/Conformance.test.cpp index 6f136d36..a23ea470 100644 --- a/tests/Conformance.test.cpp +++ b/tests/Conformance.test.cpp @@ -241,6 +241,8 @@ TEST_CASE("Math") TEST_CASE("Table") { + ScopedFastFlag sff("LuauFixBuiltinsStackLimit", true); + runConformance("nextvar.lua"); } @@ -1099,4 +1101,14 @@ TEST_CASE("UserdataApi") CHECK(dtorhits == 42); } +TEST_CASE("Iter") +{ + ScopedFastFlag sffs[] = { + { "LuauCompileIter", true }, + { "LuauIter", true }, + }; + + runConformance("iter.lua"); +} + TEST_SUITE_END(); diff --git a/tests/Frontend.test.cpp b/tests/Frontend.test.cpp index e771b6b1..a10e8f7f 100644 --- a/tests/Frontend.test.cpp +++ b/tests/Frontend.test.cpp @@ -386,8 +386,6 @@ TEST_CASE_FIXTURE(FrontendFixture, "cycle_error_paths") TEST_CASE_FIXTURE(FrontendFixture, "cycle_incremental_type_surface") { - ScopedFastFlag luauCyclicModuleTypeSurface{"LuauCyclicModuleTypeSurface", true}; - fileResolver.source["game/A"] = R"( return {hello = 2} )"; @@ -410,8 +408,6 @@ TEST_CASE_FIXTURE(FrontendFixture, "cycle_incremental_type_surface") TEST_CASE_FIXTURE(FrontendFixture, "cycle_incremental_type_surface_longer") { - ScopedFastFlag luauCyclicModuleTypeSurface{"LuauCyclicModuleTypeSurface", true}; - fileResolver.source["game/A"] = R"( return {mod_a = 2} )"; diff --git a/tests/Parser.test.cpp b/tests/Parser.test.cpp index 55eafe3c..69ff73ad 100644 --- a/tests/Parser.test.cpp +++ b/tests/Parser.test.cpp @@ -2041,8 +2041,6 @@ TEST_CASE_FIXTURE(Fixture, "parse_type_alias_default_type_errors") TEST_CASE_FIXTURE(Fixture, "parse_type_pack_errors") { - ScopedFastFlag luauParseRecoverUnexpectedPack{"LuauParseRecoverUnexpectedPack", true}; - matchParseError("type Y = {a: T..., b: number}", "Unexpected '...' after type name; type pack is not allowed in this context", Location{{0, 20}, {0, 23}}); matchParseError("type Y = {a: (number | string)...", "Unexpected '...' after type annotation", Location{{0, 36}, {0, 39}}); @@ -2618,8 +2616,6 @@ type Y = (T...) -> U... TEST_CASE_FIXTURE(Fixture, "recover_unexpected_type_pack") { - ScopedFastFlag luauParseRecoverUnexpectedPack{"LuauParseRecoverUnexpectedPack", true}; - ParseResult result = tryParse(R"( type X = { a: T..., b: number } type Y = { a: T..., b: number } diff --git a/tests/RuntimeLimits.test.cpp b/tests/RuntimeLimits.test.cpp index 42411de2..538f3576 100644 --- a/tests/RuntimeLimits.test.cpp +++ b/tests/RuntimeLimits.test.cpp @@ -35,9 +35,9 @@ bool hasError(const CheckResult& result, T* = nullptr) return it != result.errors.end(); } -TEST_SUITE_BEGIN("RuntimeLimitTests"); +TEST_SUITE_BEGIN("RuntimeLimits"); -TEST_CASE_FIXTURE(LimitFixture, "bail_early_on_typescript_port_of_Result_type" * doctest::timeout(1.0)) +TEST_CASE_FIXTURE(LimitFixture, "typescript_port_of_Result_type") { constexpr const char* src = R"LUA( --!strict diff --git a/tests/TypeInfer.loops.test.cpp b/tests/TypeInfer.loops.test.cpp index 960c6edf..f9b510c1 100644 --- a/tests/TypeInfer.loops.test.cpp +++ b/tests/TypeInfer.loops.test.cpp @@ -488,4 +488,71 @@ TEST_CASE_FIXTURE(Fixture, "fuzz_fail_missing_instantitation_follow") )"); } +TEST_CASE_FIXTURE(Fixture, "loop_iter_basic") +{ + ScopedFastFlag sff{"LuauTypecheckIter", true}; + + CheckResult result = check(R"( + local t: {string} = {} + local key + for k: number in t do + end + for k: number, v: string in t do + end + for k, v in t do + key = k + end + )"); + + LUAU_REQUIRE_ERROR_COUNT(0, result); + CHECK_EQ(*typeChecker.numberType, *requireType("key")); +} + +TEST_CASE_FIXTURE(Fixture, "loop_iter_trailing_nil") +{ + ScopedFastFlag sff{"LuauTypecheckIter", true}; + + CheckResult result = check(R"( + local t: {string} = {} + local extra + for k, v, e in t do + extra = e + end + )"); + + LUAU_REQUIRE_ERROR_COUNT(0, result); + CHECK_EQ(*typeChecker.nilType, *requireType("extra")); +} + +TEST_CASE_FIXTURE(Fixture, "loop_iter_no_indexer") +{ + ScopedFastFlag sff{"LuauTypecheckIter", true}; + + CheckResult result = check(R"( + local t = {} + for k, v in t do + end + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + GenericError* ge = get(result.errors[0]); + REQUIRE(ge); + CHECK_EQ("Cannot iterate over a table without indexer", ge->message); +} + +TEST_CASE_FIXTURE(Fixture, "loop_iter_iter_metamethod") +{ + ScopedFastFlag sff{"LuauTypecheckIter", true}; + + CheckResult result = check(R"( + local t = {} + setmetatable(t, { __iter = function(o) return next, o.children end }) + for k: number, v: string in t do + end + )"); + + LUAU_REQUIRE_ERROR_COUNT(0, result); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.modules.test.cpp b/tests/TypeInfer.modules.test.cpp index fa1f519c..b6f49f9f 100644 --- a/tests/TypeInfer.modules.test.cpp +++ b/tests/TypeInfer.modules.test.cpp @@ -5,7 +5,6 @@ #include "Luau/Scope.h" #include "Luau/TypeInfer.h" #include "Luau/TypeVar.h" -#include "Luau/VisitTypeVar.h" #include "Fixture.h" diff --git a/tests/TypeInfer.tables.test.cpp b/tests/TypeInfer.tables.test.cpp index 5bd522a3..8e535995 100644 --- a/tests/TypeInfer.tables.test.cpp +++ b/tests/TypeInfer.tables.test.cpp @@ -2331,7 +2331,7 @@ TEST_CASE_FIXTURE(Fixture, "confusing_indexing") TEST_CASE_FIXTURE(Fixture, "pass_a_union_of_tables_to_a_function_that_requires_a_table") { - ScopedFastFlag sff{"LuauDifferentOrderOfUnificationDoesntMatter", true}; + ScopedFastFlag sff{"LuauDifferentOrderOfUnificationDoesntMatter2", true}; CheckResult result = check(R"( local a: {x: number, y: number, [any]: any} | {y: number} @@ -2351,7 +2351,7 @@ TEST_CASE_FIXTURE(Fixture, "pass_a_union_of_tables_to_a_function_that_requires_a TEST_CASE_FIXTURE(Fixture, "pass_a_union_of_tables_to_a_function_that_requires_a_table_2") { - ScopedFastFlag sff{"LuauDifferentOrderOfUnificationDoesntMatter", true}; + ScopedFastFlag sff{"LuauDifferentOrderOfUnificationDoesntMatter2", true}; CheckResult result = check(R"( local a: {y: number} | {x: number, y: number, [any]: any} diff --git a/tests/TypeInfer.test.cpp b/tests/TypeInfer.test.cpp index a578b1cf..e81ef1a9 100644 --- a/tests/TypeInfer.test.cpp +++ b/tests/TypeInfer.test.cpp @@ -1034,4 +1034,45 @@ TEST_CASE_FIXTURE(Fixture, "follow_on_new_types_in_substitution") LUAU_REQUIRE_NO_ERRORS(result); } +/** + * The problem we had here was that the type of q in B.h was initially inferring to {} | {prop: free} before we bound + * that second table to the enclosing union. + */ +TEST_CASE_FIXTURE(Fixture, "do_not_bind_a_free_table_to_a_union_containing_that_table") +{ + ScopedFastFlag flag[] = { + {"LuauStatFunctionSimplify4", true}, + {"LuauLowerBoundsCalculation", true}, + {"LuauDifferentOrderOfUnificationDoesntMatter2", true}, + }; + + CheckResult result = check(R"( + --!strict + + local A = {} + + function A:f() + local t = {} + + for key, value in pairs(self) do + t[key] = value + end + + return t + end + + local B = A:f() + + function B.g(t) + assert(type(t) == "table") + assert(t.prop ~= nil) + end + + function B.h(q) + q = q or {} + return q or {} + end + )"); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.tryUnify.test.cpp b/tests/TypeInfer.tryUnify.test.cpp index b6e93265..87562644 100644 --- a/tests/TypeInfer.tryUnify.test.cpp +++ b/tests/TypeInfer.tryUnify.test.cpp @@ -242,8 +242,6 @@ TEST_CASE_FIXTURE(TryUnifyFixture, "cli_50320_follow_in_any_unification") TEST_CASE_FIXTURE(TryUnifyFixture, "txnlog_preserves_type_owner") { - ScopedFastFlag luauTxnLogPreserveOwner{"LuauTxnLogPreserveOwner", true}; - TypeId a = arena.addType(TypeVar{FreeTypeVar{TypeLevel{}}}); TypeId b = typeChecker.numberType; @@ -255,8 +253,6 @@ TEST_CASE_FIXTURE(TryUnifyFixture, "txnlog_preserves_type_owner") TEST_CASE_FIXTURE(TryUnifyFixture, "txnlog_preserves_pack_owner") { - ScopedFastFlag luauTxnLogPreserveOwner{"LuauTxnLogPreserveOwner", true}; - TypePackId a = arena.addTypePack(TypePackVar{FreeTypePack{TypeLevel{}}}); TypePackId b = typeChecker.anyTypePack; diff --git a/tests/TypeVar.test.cpp b/tests/TypeVar.test.cpp index e033fe22..a45af39c 100644 --- a/tests/TypeVar.test.cpp +++ b/tests/TypeVar.test.cpp @@ -313,23 +313,33 @@ TEST_CASE("tagging_props") CHECK(Luau::hasTag(prop, "foo")); } -struct VisitCountTracker +struct VisitCountTracker final : TypeVarOnceVisitor { std::unordered_map tyVisits; std::unordered_map tpVisits; - void cycle(TypeId) {} - void cycle(TypePackId) {} + void cycle(TypeId) override {} + void cycle(TypePackId) override {} template bool operator()(TypeId ty, const T& t) + { + return visit(ty); + } + + template + bool operator()(TypePackId tp, const T&) + { + return visit(tp); + } + + bool visit(TypeId ty) override { tyVisits[ty]++; return true; } - template - bool operator()(TypePackId tp, const T&) + bool visit(TypePackId tp) override { tpVisits[tp]++; return true; @@ -348,7 +358,7 @@ local b: (T, T, T) -> T VisitCountTracker tester; DenseHashSet seen{nullptr}; - visitTypeVarOnce(bType, tester, seen); + DEPRECATED_visitTypeVarOnce(bType, tester, seen); for (auto [_, count] : tester.tyVisits) CHECK_EQ(count, 1); diff --git a/tests/VisitTypeVar.test.cpp b/tests/VisitTypeVar.test.cpp new file mode 100644 index 00000000..3d426f10 --- /dev/null +++ b/tests/VisitTypeVar.test.cpp @@ -0,0 +1,48 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details + +#include "Fixture.h" + +#include "Luau/RecursionCounter.h" + +#include "doctest.h" + +using namespace Luau; + +LUAU_FASTFLAG(LuauUseVisitRecursionLimit) +LUAU_FASTINT(LuauVisitRecursionLimit) + +struct VisitTypeVarFixture : Fixture +{ + ScopedFastFlag flag1 = {"LuauUseVisitRecursionLimit", true}; + ScopedFastFlag flag2 = {"LuauRecursionLimitException", true}; +}; + +TEST_SUITE_BEGIN("VisitTypeVar"); + +TEST_CASE_FIXTURE(VisitTypeVarFixture, "throw_when_limit_is_exceeded") +{ + ScopedFastInt sfi{"LuauVisitRecursionLimit", 3}; + + CheckResult result = check(R"( + local t : {a: {b: {c: {d: {e: boolean}}}}} + )"); + + TypeId tType = requireType("t"); + + CHECK_THROWS_AS(toString(tType), RecursionLimitException); +} + +TEST_CASE_FIXTURE(VisitTypeVarFixture, "dont_throw_when_limit_is_high_enough") +{ + ScopedFastInt sfi{"LuauVisitRecursionLimit", 8}; + + CheckResult result = check(R"( + local t : {a: {b: {c: {d: {e: boolean}}}}} + )"); + + TypeId tType = requireType("t"); + + (void)toString(tType); +} + +TEST_SUITE_END(); diff --git a/tests/conformance/iter.lua b/tests/conformance/iter.lua new file mode 100644 index 00000000..468ffafb --- /dev/null +++ b/tests/conformance/iter.lua @@ -0,0 +1,196 @@ +-- This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +-- This file is based on Lua 5.x tests -- https://github.com/lua/lua/tree/master/testes +print('testing iteration') + +-- basic for loop tests +do + local a + for a,b in pairs{} do error("not here") end + for i=1,0 do error("not here") end + for i=0,1,-1 do error("not here") end + a = nil; for i=1,1 do assert(not a); a=1 end; assert(a) + a = nil; for i=1,1,-1 do assert(not a); a=1 end; assert(a) + a = 0; for i=0, 1, 0.1 do a=a+1 end; assert(a==11) +end + +-- precision tests for for loops +do + local a + --a = 0; for i=1, 0, -0.01 do a=a+1 end; assert(a==101) + a = 0; for i=0, 0.999999999, 0.1 do a=a+1 end; assert(a==10) + a = 0; for i=1, 1, 1 do a=a+1 end; assert(a==1) + a = 0; for i=1e10, 1e10, -1 do a=a+1 end; assert(a==1) + a = 0; for i=1, 0.99999, 1 do a=a+1 end; assert(a==0) + a = 0; for i=99999, 1e5, -1 do a=a+1 end; assert(a==0) + a = 0; for i=1, 0.99999, -1 do a=a+1 end; assert(a==1) +end + +-- for loops do string->number coercion +do + local a = 0; for i="10","1","-2" do a=a+1 end; assert(a==5) +end + +-- generic for with function iterators +do + local function f (n, p) + local t = {}; for i=1,p do t[i] = i*10 end + return function (_,n) + if n > 0 then + n = n-1 + return n, unpack(t) + end + end, nil, n + end + + local x = 0 + for n,a,b,c,d in f(5,3) do + x = x+1 + assert(a == 10 and b == 20 and c == 30 and d == nil) + end + assert(x == 5) +end + +-- generic for with __call (tables) +do + local f = {} + setmetatable(f, { __call = function(_, _, n) if n > 0 then return n - 1 end end }) + + local x = 0 + for n in f, nil, 5 do + x += n + end + assert(x == 10) +end + +-- generic for with __call (userdata) +do + local f = newproxy(true) + getmetatable(f).__call = function(_, _, n) if n > 0 then return n - 1 end end + + local x = 0 + for n in f, nil, 5 do + x += n + end + assert(x == 10) +end + +-- generic for with pairs +do + local x = 0 + for k, v in pairs({a = 1, b = 2, c = 3}) do + x += v + end + assert(x == 6) +end + +-- generic for with pairs with holes +do + local x = 0 + for k, v in pairs({1, 2, 3, nil, 5}) do + x += v + end + assert(x == 11) +end + +-- generic for with ipairs +do + local x = 0 + for k, v in ipairs({1, 2, 3, nil, 5}) do + x += v + end + assert(x == 6) +end + +-- generic for with __iter (tables) +do + local f = {} + setmetatable(f, { __iter = function(x) + assert(f == x) + return next, {1, 2, 3, 4} + end }) + + local x = 0 + for n in f do + x += n + end + assert(x == 10) +end + +-- generic for with __iter (userdata) +do + local f = newproxy(true) + getmetatable(f).__iter = function(x) + assert(f == x) + return next, {1, 2, 3, 4} + end + + local x = 0 + for n in f do + x += n + end + assert(x == 10) +end + +-- generic for with tables (dictionary) +do + local x = 0 + for k, v in {a = 1, b = 2, c = 3} do + print(k, v) + x += v + end + assert(x == 6) +end + +-- generic for with tables (arrays) +do + local x = '' + for k, v in {1, 2, 3, nil, 5} do + x ..= tostring(v) + end + assert(x == "1235") +end + +-- generic for with tables (mixed) +do + local x = 0 + for k, v in {1, 2, 3, nil, 5, a = 1, b = 2, c = 3} do + x += v + end + assert(x == 17) +end + +-- generic for over a non-iterable object +do + local ok, err = pcall(function() for x in 42 do end end) + assert(not ok and err:match("attempt to iterate")) +end + +-- generic for over an iterable object that doesn't return a function +do + local obj = {} + setmetatable(obj, { __iter = function() end }) + + local ok, err = pcall(function() for x in obj do end end) + assert(not ok and err:match("attempt to call a nil value")) +end + +-- it's okay to iterate through a table with a single variable +do + local x = 0 + for k in {1, 2, 3, 4, 5} do + x += k + end + assert(x == 15) +end + +-- all extra variables should be set to nil during builtin traversal +do + local x = 0 + for k,v,a,b,c,d,e in {1, 2, 3, 4, 5} do + x += k + assert(a == nil and b == nil and c == nil and d == nil and e == nil) + end + assert(x == 15) +end + +return"OK" diff --git a/tests/conformance/nextvar.lua b/tests/conformance/nextvar.lua index c8176456..0dba8fa6 100644 --- a/tests/conformance/nextvar.lua +++ b/tests/conformance/nextvar.lua @@ -368,48 +368,6 @@ assert(next(a,nil) == 1000 and next(a,1000) == nil) assert(next({}) == nil) assert(next({}, nil) == nil) -for a,b in pairs{} do error("not here") end -for i=1,0 do error("not here") end -for i=0,1,-1 do error("not here") end -a = nil; for i=1,1 do assert(not a); a=1 end; assert(a) -a = nil; for i=1,1,-1 do assert(not a); a=1 end; assert(a) - -a = 0; for i=0, 1, 0.1 do a=a+1 end; assert(a==11) --- precision problems ---a = 0; for i=1, 0, -0.01 do a=a+1 end; assert(a==101) -a = 0; for i=0, 0.999999999, 0.1 do a=a+1 end; assert(a==10) -a = 0; for i=1, 1, 1 do a=a+1 end; assert(a==1) -a = 0; for i=1e10, 1e10, -1 do a=a+1 end; assert(a==1) -a = 0; for i=1, 0.99999, 1 do a=a+1 end; assert(a==0) -a = 0; for i=99999, 1e5, -1 do a=a+1 end; assert(a==0) -a = 0; for i=1, 0.99999, -1 do a=a+1 end; assert(a==1) - --- conversion -a = 0; for i="10","1","-2" do a=a+1 end; assert(a==5) - - -collectgarbage() - - --- testing generic 'for' - -local function f (n, p) - local t = {}; for i=1,p do t[i] = i*10 end - return function (_,n) - if n > 0 then - n = n-1 - return n, unpack(t) - end - end, nil, n -end - -local x = 0 -for n,a,b,c,d in f(5,3) do - x = x+1 - assert(a == 10 and b == 20 and c == 30 and d == nil) -end -assert(x == 5) - -- testing table.create and table.find do local t = table.create(5) @@ -596,4 +554,17 @@ do assert(#t2 == 6) end +-- test table.unpack fastcall for rejecting large unpacks +do + local ok, res = pcall(function() + local a = table.create(7999, 0) + local b = table.create(8000, 0) + + local at = { table.unpack(a) } + local bt = { table.unpack(b) } + end) + + assert(not ok) +end + return"OK" diff --git a/tools/lldb_formatters.py b/tools/lldb_formatters.py index b3d2b4f5..ff610d09 100644 --- a/tools/lldb_formatters.py +++ b/tools/lldb_formatters.py @@ -97,7 +97,7 @@ class LuauVariantSyntheticChildrenProvider: if self.current_type: storage = self.valobj.GetChildMemberWithName("storage") - self.stored_value = storage.Cast(self.current_type.GetPointerType()).Dereference() + self.stored_value = storage.Cast(self.current_type) else: self.stored_value = None else: From 72d8d443431875607fd457a13fe36ea62804d327 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Thu, 5 May 2022 17:05:57 -0700 Subject: [PATCH 240/399] Add documentation for generalized iteration (#475) --- docs/_pages/syntax.md | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+) diff --git a/docs/_pages/syntax.md b/docs/_pages/syntax.md index 4d39e462..fe825fda 100644 --- a/docs/_pages/syntax.md +++ b/docs/_pages/syntax.md @@ -196,3 +196,26 @@ local sign = if x < 0 then -1 elseif x > 0 then 1 else 0 ``` **Note:** In Luau, the `if-then-else` expression is preferred vs the standard Lua idiom of writing `a and b or c` (which roughly simulates a ternary operator). However, the Lua idiom may return an unexpected result if `b` evaluates to false. The `if-then-else` expression will behave as expected in all situations. + +## Generalized iteration + +Luau uses the standard Lua syntax for iterating through containers, `for vars in values`, but extends the semantics with support for generalized iteration. In Lua, to iterate over a table you need to use an iterator like `next` or a function that returns one like `pairs` or `ipairs`. In Luau, you can simply iterate over a table: + +```lua +for k, v in {1, 4, 9} do + assert(k * k == v) +end +``` + +This works for tables but can also be extended for tables or userdata by implementing `__iter` metamethod that is called before the iteration begins, and should return an iterator function like `next` (or a custom one): + +```lua +local obj = { items = {1, 4, 9} } +setmetatable(obj, { __iter = function(o) return next, o.items end }) + +for k, v in obj do + assert(k * k == v) +end +``` + +The default iteration order for tables is specified to be consecutive for elements `1..#t` and unordered after that, visiting every element; similarly to iteration using `pairs`, modifying the table entries for keys other than the current one results in unspecified behavior. From 7935f9f8b66e81175ab2997e9c15b431fb1c1dfd Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Mon, 9 May 2022 18:33:53 -0700 Subject: [PATCH 241/399] Update sandbox.md Reword the GC docs to avoid back-referencing the thread identity mechanism, since it's entirely Roblox-side and isn't fully documented here anymore. --- docs/_pages/sandbox.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/_pages/sandbox.md b/docs/_pages/sandbox.md index 409a0929..04e72658 100644 --- a/docs/_pages/sandbox.md +++ b/docs/_pages/sandbox.md @@ -4,7 +4,7 @@ title: Sandboxing toc: true --- -Luau is safe to embed. Broadly speaking, this means that even in the face of untrusted (and in Roblox case, actively malicious) code, the language and the standard library don't allow any unsafe access to the underlying system, and don't have any bugs that allow escaping out of the sandbox (e.g. to gain native code execution through ROP gadgets et al). Additionally, the VM provides extra features to implement isolation of privileged code from unprivileged code and protect one from the other; this is important if the embedding environment (Roblox) decides to expose some APIs that may not be safe to call from untrusted code, for example because they do provide controlled access to the underlying system or risk PII exposure through fingerprinting etc. +Luau is safe to embed. Broadly speaking, this means that even in the face of untrusted (and in Roblox case, actively malicious) code, the language and the standard library don't allow any unsafe access to the underlying system, and don't have any bugs that allow escaping out of the sandbox (e.g. to gain native code execution through ROP gadgets et al). Additionally, the VM provides extra features to implement isolation of privileged code from unprivileged code and protect one from the other; this is important if the embedding environment decides to expose some APIs that may not be safe to call from untrusted code, for example because they do provide controlled access to the underlying system or risk PII exposure through fingerprinting etc. This safety is achieved through a combination of removing features from the standard library that are unsafe, adding features to the VM that make it possible to implement sandboxing and isolation, and making sure the implementation is safe from memory safety issues using fuzzing. @@ -54,7 +54,7 @@ This mechanism is bad for performance, memory safety and isolation: - In Lua 5.1, `__gc` support requires traversing userdata lists redundantly during garbage collection to filter out finalizable objects - In later versions of Lua, userdata that implement `__gc` are split into separate lists; however, finalization prolongs the lifetime of the finalized objects which results in less prompt memory reclamation, and two-step destruction results in extra cache misses for userdata -- `__gc` runs during garbage collection in context of an arbitrary thread which makes the thread identity mechanism described above invalid +- `__gc` runs during garbage collection in context of an arbitrary thread which makes the thread identity mechanism used in Roblox to support trusted Luau code invalid - Objects can be removed from weak tables *after* being finalized, which means that accessing these objects can result in memory safety bugs, unless all exposed userdata methods guard against use-after-gc. - If `__gc` method ever leaks to scripts, they can call it directly on an object and use any method exposed by that object after that. This means that `__gc` and all other exposed methods must support memory safety when called on a destroyed object. From be0b7d07e24549e18c54836d3c6e2c3f5f94cd57 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Mon, 9 May 2022 18:34:31 -0700 Subject: [PATCH 242/399] Update sandbox.md Replace debug.getinfo with debug.info --- docs/_pages/sandbox.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/_pages/sandbox.md b/docs/_pages/sandbox.md index 04e72658..d1d7d118 100644 --- a/docs/_pages/sandbox.md +++ b/docs/_pages/sandbox.md @@ -19,7 +19,7 @@ The following libraries and global functions have been removed as a result: - `io.` library has been removed entirely, as it gives access to files and allows running processes - `package.` library has been removed entirely, as it gives access to files and allows loading native modules - `os.` library has been cleaned up from file and environment access functions (`execute`, `exit`, etc.). The only supported functions in the library are `clock`, `date`, `difftime` and `time`. -- `debug.` library has been removed to a large extent, as it has functions that aren't memory safe and other functions break isolation; the only supported functions are `traceback` ~~and `getinfo` (with reduced functionality)~~. +- `debug.` library has been removed to a large extent, as it has functions that aren't memory safe and other functions break isolation; the only supported functions are `traceback` and `info` (which is similar to `debug.getinfo` but has a slightly different interface). - `dofile` and `loadfile` allowed access to file system and have been removed. To achieve memory safety, access to function bytecode has been removed. Bytecode is hard to validate and using untrusted bytecode may lead to exploits. Thus, `loadstring` doesn't work with bytecode inputs, and `string.dump`/`load` have been removed as they aren't necessary anymore. When embedding Luau, bytecode should be encrypted/signed to prevent MITM attacks as well, as the VM assumes that the bytecode was generated by the Luau compiler (which never produces invalid/unsafe bytecode). From f3f231ea6b1b063a511c7ef693ca46f02dc01087 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Mon, 9 May 2022 18:38:10 -0700 Subject: [PATCH 243/399] Update compatibility.md Update `__pairs` note with `__iter`, change `__len` to unsure as with `__iter` lack of `__len` on tables is the only issue preventing complete user created containers. --- docs/_pages/compatibility.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/_pages/compatibility.md b/docs/_pages/compatibility.md index 00d883e2..eab1aac8 100644 --- a/docs/_pages/compatibility.md +++ b/docs/_pages/compatibility.md @@ -54,7 +54,7 @@ Sandboxing challenges are [covered in the dedicated section](sandbox). | goto statement | ❌ | this complicates the compiler, makes control flow unstructured and doesn't address a significant need | | finalizers for tables | ❌ | no `__gc` support due to sandboxing and performance/complexity | | no more fenv for threads or functions | 😞 | we love this, but it breaks compatibility | -| tables honor the `__len` metamethod | ❌ | performance implications, no strong use cases +| tables honor the `__len` metamethod | 🤷‍♀️ | performance implications, no strong use cases | hex and `\z` escapes in strings | ✔️ | | | support for hexadecimal floats | 🤷‍♀️ | no strong use cases | | order metamethods work for different types | ❌ | no strong use cases and more complicated semantics + compat | @@ -63,7 +63,7 @@ Sandboxing challenges are [covered in the dedicated section](sandbox). | arguments for function called through `xpcall` | ✔️ | | | optional base in `math.log` | ✔️ | | | optional separator in `string.rep` | 🤷‍♀️ | no real use cases | -| new metamethods `__pairs` and `__ipairs` | ❌ | would like to reevaluate iteration design long term | +| new metamethods `__pairs` and `__ipairs` | ❌ | superseded by `__iter` | | frontier patterns | ✔️ | | | `%g` in patterns | ✔️ | | | `\0` in patterns | ✔️ | | @@ -72,7 +72,7 @@ Sandboxing challenges are [covered in the dedicated section](sandbox). Two things that are important to call out here are various new metamethods for tables and yielding in metamethods. In both cases, there are performance implications to supporting this - our implementation is *very* highly tuned for performance, so any changes that affect the core fundamentals of how Lua works have a price. To support yielding in metamethods we'd need to make the core of the VM more involved, since almost every single "interesting" opcode would need to learn how to be resumable - which also complicates future JIT/AOT story. Metamethods in general are important for extensibility, but very challenging to deal with in implementation, so we err on the side of not supporting any new metamethods unless a strong need arises. -For `__pairs`/`__ipairs`, we aren't sure that this is the right design choice - self-iterating tables via `__iter` are very appealing, and if we can resolve some challenges with array iteration order, that would make the language more accessible so we may go that route instead. +For `__pairs`/`__ipairs`, we felt that extending library functions to enable custom containers wasn't the right choice. Instead we revisited iteration design to allow for self-iterating objects via `__iter` metamethod, which results in a cleaner iteration design that also makes it easier to iterate over tables. As such, we have no plans to support `__pairs`/`__ipairs` as all use cases for it can now be solved by `__iter`. Ephemeron tables may be implemented at some point since they do have valid uses and they make weak tables semantically cleaner, however the cleanup mechanism for these is expensive and complicated, and as such this can only be considered after the pending GC rework is complete. From 105e74c7d940c2a5b416b5fdf0af718b3d040ed1 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Wed, 11 May 2022 15:14:51 -0700 Subject: [PATCH 244/399] Update STATUS.md Both generalized iteration and LBC are implemented but not fully enabled in Roblox yet. --- rfcs/STATUS.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/rfcs/STATUS.md b/rfcs/STATUS.md index e3e227a0..15a232ce 100644 --- a/rfcs/STATUS.md +++ b/rfcs/STATUS.md @@ -39,10 +39,10 @@ This document tracks unimplemented RFCs. [RFC: Generalized iteration](https://github.com/Roblox/luau/blob/master/rfcs/generalized-iteration.md) -**Status**: Needs implementation +**Status**: Implemented but not fully rolled out yet. ## Lower Bounds Calculation [RFC: Lower bounds calculation](https://github.com/Roblox/luau/blob/master/rfcs/lower-bounds-calculation.md) -**Status**: Needs implementation +**Status**: Implemented but not fully rolled out yet. From a775e6dc8e5261efd090e6a5b62a191b2f007055 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Thu, 12 May 2022 10:08:10 -0700 Subject: [PATCH 245/399] Mark last table subtyping RFC as implemented --- rfcs/unsealed-table-subtyping-strips-optional-properties.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/rfcs/unsealed-table-subtyping-strips-optional-properties.md b/rfcs/unsealed-table-subtyping-strips-optional-properties.md index deecfdb3..d99c1f81 100644 --- a/rfcs/unsealed-table-subtyping-strips-optional-properties.md +++ b/rfcs/unsealed-table-subtyping-strips-optional-properties.md @@ -1,5 +1,7 @@ # Only strip optional properties from unsealed tables during subtyping +**Status**: Implemented + ## Summary Currently subtyping allows optional properties to be stripped from table types during subtyping. From 87fe15ac510dbafa1263dad28e6bd1d41d11d1fd Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Thu, 12 May 2022 10:08:36 -0700 Subject: [PATCH 246/399] Update STATUS.md Mark last table subtyping RFC as implemented --- rfcs/STATUS.md | 6 ------ 1 file changed, 6 deletions(-) diff --git a/rfcs/STATUS.md b/rfcs/STATUS.md index 15a232ce..ef55b5c4 100644 --- a/rfcs/STATUS.md +++ b/rfcs/STATUS.md @@ -15,12 +15,6 @@ This document tracks unimplemented RFCs. **Status**: Needs implementation -## Sealed/unsealed typing changes - -[RFC: Only strip optional properties from unsealed tables during subtyping](https://github.com/Roblox/luau/blob/master/rfcs/unsealed-table-subtyping-strips-optional-properties.md) - -**Status**: Implemented but not fully rolled out yet. - ## Safe navigation operator [RFC: Safe navigation postfix operator (?)](https://github.com/Roblox/luau/blob/master/rfcs/syntax-safe-navigation-operator.md) From 298b33859b67014d76542b8a49c9be6b95730600 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Fri, 13 May 2022 12:16:50 -0700 Subject: [PATCH 247/399] Sync to upstream/release/527 --- Analysis/include/Luau/TypeVar.h | 2 +- Analysis/src/Clone.cpp | 4 +- Analysis/src/Normalize.cpp | 3 +- Analysis/src/Substitution.cpp | 4 +- Analysis/src/ToString.cpp | 16 +- Analysis/src/TypeInfer.cpp | 191 ++++++++------------- Ast/src/Parser.cpp | 3 +- CMakeLists.txt | 6 + Compiler/src/Compiler.cpp | 30 +++- Compiler/src/ConstantFolding.cpp | 6 +- Compiler/src/CostModel.cpp | 2 +- extern/isocline/src/bbcode.c | 1 + tests/AstQuery.test.cpp | 2 +- tests/Autocomplete.test.cpp | 44 +++-- tests/BuiltinDefinitions.test.cpp | 4 +- tests/Compiler.test.cpp | 71 ++++++-- tests/CostModel.test.cpp | 14 +- tests/Fixture.cpp | 19 +- tests/Fixture.h | 5 + tests/Frontend.test.cpp | 2 +- tests/Linter.test.cpp | 6 +- tests/Module.test.cpp | 4 +- tests/NonstrictMode.test.cpp | 6 +- tests/Normalize.test.cpp | 7 +- tests/Parser.test.cpp | 2 - tests/RuntimeLimits.test.cpp | 2 +- tests/ToDot.test.cpp | 2 +- tests/ToString.test.cpp | 4 +- tests/TypeInfer.aliases.test.cpp | 10 +- tests/TypeInfer.annotations.test.cpp | 8 +- tests/TypeInfer.anyerror.test.cpp | 4 +- tests/TypeInfer.builtins.test.cpp | 117 +++++++------ tests/TypeInfer.classes.test.cpp | 2 +- tests/TypeInfer.functions.test.cpp | 38 ++-- tests/TypeInfer.generics.test.cpp | 12 +- tests/TypeInfer.intersectionTypes.test.cpp | 6 +- tests/TypeInfer.loops.test.cpp | 28 +-- tests/TypeInfer.modules.test.cpp | 28 +-- tests/TypeInfer.oop.test.cpp | 4 +- tests/TypeInfer.operators.test.cpp | 71 ++++++-- tests/TypeInfer.provisional.test.cpp | 10 +- tests/TypeInfer.refinements.test.cpp | 20 +-- tests/TypeInfer.singletons.test.cpp | 2 +- tests/TypeInfer.tables.test.cpp | 74 ++++---- tests/TypeInfer.test.cpp | 17 +- tests/TypeInfer.tryUnify.test.cpp | 2 +- tests/TypeInfer.typePacks.cpp | 6 +- tests/TypeInfer.unionTypes.test.cpp | 6 +- 48 files changed, 511 insertions(+), 416 deletions(-) diff --git a/Analysis/include/Luau/TypeVar.h b/Analysis/include/Luau/TypeVar.h index 84576758..9cacbc6d 100644 --- a/Analysis/include/Luau/TypeVar.h +++ b/Analysis/include/Luau/TypeVar.h @@ -329,7 +329,7 @@ struct TableTypeVar // We need to know which is which when we stringify types. std::optional syntheticName; - std::map methodDefinitionLocations; + std::map methodDefinitionLocations; // TODO: Remove with FFlag::LuauNoMethodLocations std::vector instantiatedTypeParams; std::vector instantiatedTypePackParams; ModuleName definitionModuleName; diff --git a/Analysis/src/Clone.cpp b/Analysis/src/Clone.cpp index d5bd9dab..1aa556eb 100644 --- a/Analysis/src/Clone.cpp +++ b/Analysis/src/Clone.cpp @@ -11,6 +11,7 @@ LUAU_FASTFLAG(DebugLuauCopyBeforeNormalizing) LUAU_FASTINTVARIABLE(LuauTypeCloneRecursionLimit, 300) LUAU_FASTFLAG(LuauTypecheckOptPass) LUAU_FASTFLAGVARIABLE(LuauLosslessClone, false) +LUAU_FASTFLAG(LuauNoMethodLocations) namespace Luau { @@ -277,7 +278,8 @@ void TypeCloner::operator()(const TableTypeVar& t) } ttv->definitionModuleName = t.definitionModuleName; - ttv->methodDefinitionLocations = t.methodDefinitionLocations; + if (!FFlag::LuauNoMethodLocations) + ttv->methodDefinitionLocations = t.methodDefinitionLocations; ttv->tags = t.tags; } diff --git a/Analysis/src/Normalize.cpp b/Analysis/src/Normalize.cpp index d8c11388..ef5377a1 100644 --- a/Analysis/src/Normalize.cpp +++ b/Analysis/src/Normalize.cpp @@ -14,7 +14,6 @@ LUAU_FASTFLAGVARIABLE(DebugLuauCopyBeforeNormalizing, false) // This could theoretically be 2000 on amd64, but x86 requires this. LUAU_FASTINTVARIABLE(LuauNormalizeIterationLimit, 1200); LUAU_FASTFLAGVARIABLE(LuauNormalizeCombineTableFix, false); -LUAU_FASTFLAGVARIABLE(LuauNormalizeCombineIntersectionFix, false); namespace Luau { @@ -863,7 +862,7 @@ struct Normalize final : TypeVarVisitor TypeId theTable = result->parts.back(); - if (!get(FFlag::LuauNormalizeCombineIntersectionFix ? follow(theTable) : theTable)) + if (!get(follow(theTable))) { result->parts.push_back(arena.addType(TableTypeVar{TableState::Sealed, TypeLevel{}})); theTable = result->parts.back(); diff --git a/Analysis/src/Substitution.cpp b/Analysis/src/Substitution.cpp index 30d8574a..c5c7977a 100644 --- a/Analysis/src/Substitution.cpp +++ b/Analysis/src/Substitution.cpp @@ -12,6 +12,7 @@ LUAU_FASTINTVARIABLE(LuauTarjanChildLimit, 10000) LUAU_FASTFLAG(LuauTypecheckOptPass) LUAU_FASTFLAGVARIABLE(LuauSubstituteFollowNewTypes, false) LUAU_FASTFLAGVARIABLE(LuauSubstituteFollowPossibleMutations, false) +LUAU_FASTFLAG(LuauNoMethodLocations) namespace Luau { @@ -408,7 +409,8 @@ TypeId Substitution::clone(TypeId ty) { LUAU_ASSERT(!ttv->boundTo); TableTypeVar clone = TableTypeVar{ttv->props, ttv->indexer, ttv->level, ttv->state}; - clone.methodDefinitionLocations = ttv->methodDefinitionLocations; + if (!FFlag::LuauNoMethodLocations) + clone.methodDefinitionLocations = ttv->methodDefinitionLocations; clone.definitionModuleName = ttv->definitionModuleName; clone.name = ttv->name; clone.syntheticName = ttv->syntheticName; diff --git a/Analysis/src/ToString.cpp b/Analysis/src/ToString.cpp index b5d6a550..51665f7f 100644 --- a/Analysis/src/ToString.cpp +++ b/Analysis/src/ToString.cpp @@ -248,6 +248,13 @@ struct StringifierState result.name += s; } + void emit(TypeLevel level) + { + emit(std::to_string(level.level)); + emit("-"); + emit(std::to_string(level.subLevel)); + } + void emit(const char* s) { if (opts.maxTypeLength > 0 && result.name.length() > opts.maxTypeLength) @@ -379,7 +386,7 @@ struct TypeVarStringifier if (FFlag::DebugLuauVerboseTypeNames) { state.emit("-"); - state.emit(std::to_string(ftv.level.level)); + state.emit(ftv.level); } } @@ -403,7 +410,10 @@ struct TypeVarStringifier { state.result.invalid = true; - state.emit("[["); + state.emit("["); + if (FFlag::DebugLuauVerboseTypeNames) + state.emit(ctv.level); + state.emit("["); bool first = true; for (TypeId ty : ctv.parts) @@ -947,7 +957,7 @@ struct TypePackStringifier if (FFlag::DebugLuauVerboseTypeNames) { state.emit("-"); - state.emit(std::to_string(pack.level.level)); + state.emit(pack.level); } state.emit("..."); diff --git a/Analysis/src/TypeInfer.cpp b/Analysis/src/TypeInfer.cpp index 4466ede2..a13abd53 100644 --- a/Analysis/src/TypeInfer.cpp +++ b/Analysis/src/TypeInfer.cpp @@ -36,13 +36,11 @@ LUAU_FASTFLAGVARIABLE(LuauEqConstraint, false) LUAU_FASTFLAGVARIABLE(LuauWeakEqConstraint, false) // Eventually removed as false. LUAU_FASTFLAGVARIABLE(LuauLowerBoundsCalculation, false) LUAU_FASTFLAGVARIABLE(DebugLuauFreezeDuringUnification, false) -LUAU_FASTFLAGVARIABLE(LuauInferStatFunction, false) LUAU_FASTFLAGVARIABLE(LuauInstantiateFollows, false) LUAU_FASTFLAGVARIABLE(LuauSelfCallAutocompleteFix, false) LUAU_FASTFLAGVARIABLE(LuauDiscriminableUnions2, false) LUAU_FASTFLAGVARIABLE(LuauReduceUnionRecursion, false) LUAU_FASTFLAGVARIABLE(LuauOnlyMutateInstantiatedTables, false) -LUAU_FASTFLAGVARIABLE(LuauStatFunctionSimplify4, false) LUAU_FASTFLAGVARIABLE(LuauTypecheckOptPass, false) LUAU_FASTFLAGVARIABLE(LuauUnsealedTableLiteral, false) LUAU_FASTFLAGVARIABLE(LuauTwoPassAliasDefinitionFix, false) @@ -53,12 +51,13 @@ LUAU_FASTFLAGVARIABLE(LuauDoNotTryToReduce, false) LUAU_FASTFLAGVARIABLE(LuauDoNotAccidentallyDependOnPointerOrdering, false) LUAU_FASTFLAGVARIABLE(LuauCheckImplicitNumbericKeys, false) LUAU_FASTFLAG(LuauAnyInIsOptionalIsOptional) -LUAU_FASTFLAGVARIABLE(LuauDecoupleOperatorInferenceFromUnifiedTypeInference, false) LUAU_FASTFLAGVARIABLE(LuauTableUseCounterInstead, false) LUAU_FASTFLAGVARIABLE(LuauReturnTypeInferenceInNonstrict, false) LUAU_FASTFLAGVARIABLE(LuauRecursionLimitException, false); LUAU_FASTFLAG(LuauLosslessClone) LUAU_FASTFLAGVARIABLE(LuauTypecheckIter, false); +LUAU_FASTFLAGVARIABLE(LuauSuccessTypingForEqualityOperations, false) +LUAU_FASTFLAGVARIABLE(LuauNoMethodLocations, false); namespace Luau { @@ -587,7 +586,7 @@ void TypeChecker::checkBlockWithoutRecursionCheck(const ScopePtr& scope, const A { std::optional expectedType; - if (FFlag::LuauInferStatFunction && !fun->func->self) + if (!fun->func->self) { if (auto name = fun->name->as()) { @@ -1307,7 +1306,7 @@ void TypeChecker::check(const ScopePtr& scope, TypeId ty, const ScopePtr& funSco scope->bindings[name->local] = {anyIfNonstrict(quantify(funScope, ty, name->local->location)), name->local->location}; return; } - else if (auto name = function.name->as(); name && FFlag::LuauStatFunctionSimplify4) + else if (auto name = function.name->as()) { TypeId exprTy = checkExpr(scope, *name->expr).type; TableTypeVar* ttv = getMutableTableType(exprTy); @@ -1341,7 +1340,7 @@ void TypeChecker::check(const ScopePtr& scope, TypeId ty, const ScopePtr& funSco if (ttv && ttv->state != TableState::Sealed) ttv->props[name->index.value] = {follow(quantify(funScope, ty, name->indexLocation)), /* deprecated */ false, {}, name->indexLocation}; } - else if (FFlag::LuauStatFunctionSimplify4) + else { LUAU_ASSERT(function.name->is()); @@ -1349,71 +1348,6 @@ void TypeChecker::check(const ScopePtr& scope, TypeId ty, const ScopePtr& funSco checkFunctionBody(funScope, ty, *function.func); } - else if (function.func->self) - { - LUAU_ASSERT(!FFlag::LuauStatFunctionSimplify4); - - AstExprIndexName* indexName = function.name->as(); - if (!indexName) - ice("member function declaration has malformed name expression"); - - TypeId selfTy = checkExpr(scope, *indexName->expr).type; - TableTypeVar* tableSelf = getMutableTableType(selfTy); - if (!tableSelf) - { - if (isTableIntersection(selfTy)) - reportError(TypeError{function.location, CannotExtendTable{selfTy, CannotExtendTable::Property, indexName->index.value}}); - else if (!get(selfTy) && !get(selfTy)) - reportError(TypeError{function.location, OnlyTablesCanHaveMethods{selfTy}}); - } - else if (tableSelf->state == TableState::Sealed) - reportError(TypeError{function.location, CannotExtendTable{selfTy, CannotExtendTable::Property, indexName->index.value}}); - - const bool tableIsExtendable = tableSelf && tableSelf->state != TableState::Sealed; - - ty = follow(ty); - - if (tableIsExtendable) - tableSelf->props[indexName->index.value] = {ty, /* deprecated */ false, {}, indexName->indexLocation}; - - const FunctionTypeVar* funTy = get(ty); - if (!funTy) - ice("Methods should be functions"); - - std::optional arg0 = first(funTy->argTypes); - if (!arg0) - ice("Methods should always have at least 1 argument (self)"); - - checkFunctionBody(funScope, ty, *function.func); - - if (tableIsExtendable) - tableSelf->props[indexName->index.value] = { - follow(quantify(funScope, ty, indexName->indexLocation)), /* deprecated */ false, {}, indexName->indexLocation}; - } - else - { - LUAU_ASSERT(!FFlag::LuauStatFunctionSimplify4); - - TypeId leftType = checkLValueBinding(scope, *function.name); - - checkFunctionBody(funScope, ty, *function.func); - - unify(ty, leftType, function.location); - - LUAU_ASSERT(function.name->is() || function.name->is()); - - if (auto exprIndexName = function.name->as()) - { - if (auto typeIt = currentModule->astTypes.find(exprIndexName->expr)) - { - if (auto ttv = getMutableTableType(*typeIt)) - { - if (auto it = ttv->props.find(exprIndexName->index.value); it != ttv->props.end()) - it->second.type = follow(quantify(funScope, leftType, function.name->location)); - } - } - } - } } void TypeChecker::check(const ScopePtr& scope, TypeId ty, const ScopePtr& funScope, const AstStatLocalFunction& function) @@ -1523,7 +1457,8 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatTypeAlias& typealias // This is a shallow clone, original recursive links to self are not updated TableTypeVar clone = TableTypeVar{ttv->props, ttv->indexer, ttv->level, ttv->state}; - clone.methodDefinitionLocations = ttv->methodDefinitionLocations; + if (!FFlag::LuauNoMethodLocations) + clone.methodDefinitionLocations = ttv->methodDefinitionLocations; clone.definitionModuleName = ttv->definitionModuleName; clone.name = name; @@ -2652,13 +2587,58 @@ TypeId TypeChecker::checkRelationalOperation( std::optional leftMetatable = isString(lhsType) ? std::nullopt : getMetatable(follow(lhsType)); std::optional rightMetatable = isString(rhsType) ? std::nullopt : getMetatable(follow(rhsType)); - // TODO: this check seems odd, the second part is redundant - // is it meant to be if (leftMetatable && rightMetatable && leftMetatable != rightMetatable) - if (bool(leftMetatable) != bool(rightMetatable) && leftMetatable != rightMetatable) + if (FFlag::LuauSuccessTypingForEqualityOperations) { - reportError(expr.location, GenericError{format("Types %s and %s cannot be compared with %s because they do not have the same metatable", - toString(lhsType).c_str(), toString(rhsType).c_str(), toString(expr.op).c_str())}); - return errorRecoveryType(booleanType); + if (leftMetatable != rightMetatable) + { + bool matches = false; + if (isEquality) + { + if (const UnionTypeVar* utv = get(leftType); utv && rightMetatable) + { + for (TypeId leftOption : utv) + { + if (getMetatable(follow(leftOption)) == rightMetatable) + { + matches = true; + break; + } + } + } + + if (!matches) + { + if (const UnionTypeVar* utv = get(rhsType); utv && leftMetatable) + { + for (TypeId rightOption : utv) + { + if (getMetatable(follow(rightOption)) == leftMetatable) + { + matches = true; + break; + } + } + } + } + } + + + if (!matches) + { + reportError(expr.location, GenericError{format("Types %s and %s cannot be compared with %s because they do not have the same metatable", + toString(lhsType).c_str(), toString(rhsType).c_str(), toString(expr.op).c_str())}); + return errorRecoveryType(booleanType); + } + } + } + else + { + if (bool(leftMetatable) != bool(rightMetatable) && leftMetatable != rightMetatable) + { + reportError(expr.location, GenericError{format("Types %s and %s cannot be compared with %s because they do not have the same metatable", + toString(lhsType).c_str(), toString(rhsType).c_str(), toString(expr.op).c_str())}); + return errorRecoveryType(booleanType); + } } if (leftMetatable) @@ -2754,22 +2734,11 @@ TypeId TypeChecker::checkBinaryOperation( lhsType = follow(lhsType); rhsType = follow(rhsType); - if (FFlag::LuauDecoupleOperatorInferenceFromUnifiedTypeInference) + if (!isNonstrictMode() && get(lhsType)) { - if (!isNonstrictMode() && get(lhsType)) - { - auto name = getIdentifierOfBaseVar(expr.left); - reportError(expr.location, CannotInferBinaryOperation{expr.op, name, CannotInferBinaryOperation::Operation}); - // We will fall-through to the `return anyType` check below. - } - } - else - { - if (!isNonstrictMode() && get(lhsType)) - { - auto name = getIdentifierOfBaseVar(expr.left); - reportError(expr.location, CannotInferBinaryOperation{expr.op, name, CannotInferBinaryOperation::Operation}); - } + auto name = getIdentifierOfBaseVar(expr.left); + reportError(expr.location, CannotInferBinaryOperation{expr.op, name, CannotInferBinaryOperation::Operation}); + // We will fall-through to the `return anyType` check below. } // If we know nothing at all about the lhs type, we can usually say nothing about the result. @@ -3231,43 +3200,27 @@ TypeId TypeChecker::checkFunctionName(const ScopePtr& scope, AstExpr& funName, T else if (auto indexName = funName.as()) { TypeId lhsType = checkExpr(scope, *indexName->expr).type; - - if (!FFlag::LuauStatFunctionSimplify4 && (get(lhsType) || get(lhsType))) - return lhsType; - TableTypeVar* ttv = getMutableTableType(lhsType); - if (FFlag::LuauStatFunctionSimplify4) + if (!ttv || ttv->state == TableState::Sealed) { - if (!ttv || ttv->state == TableState::Sealed) - { - if (auto ty = getIndexTypeFromType(scope, lhsType, indexName->index.value, indexName->indexLocation, false)) - return *ty; + if (auto ty = getIndexTypeFromType(scope, lhsType, indexName->index.value, indexName->indexLocation, false)) + return *ty; - return errorRecoveryType(scope); - } - } - else - { - if (!ttv || lhsType->persistent || ttv->state == TableState::Sealed) - return errorRecoveryType(scope); + return errorRecoveryType(scope); } Name name = indexName->index.value; if (ttv->props.count(name)) - { - if (FFlag::LuauStatFunctionSimplify4) - return ttv->props[name].type; - else - return errorRecoveryType(scope); - } + return ttv->props[name].type; Property& property = ttv->props[name]; property.type = freshTy(); property.location = indexName->indexLocation; - ttv->methodDefinitionLocations[name] = funName.location; + if (!FFlag::LuauNoMethodLocations) + ttv->methodDefinitionLocations[name] = funName.location; return property.type; } else if (funName.is()) @@ -4669,7 +4622,8 @@ TypeId ReplaceGenerics::clean(TypeId ty) if (const TableTypeVar* ttv = log->getMutable(ty)) { TableTypeVar clone = TableTypeVar{ttv->props, ttv->indexer, level, TableState::Free}; - clone.methodDefinitionLocations = ttv->methodDefinitionLocations; + if (!FFlag::LuauNoMethodLocations) + clone.methodDefinitionLocations = ttv->methodDefinitionLocations; clone.definitionModuleName = ttv->definitionModuleName; return addType(std::move(clone)); } @@ -4715,7 +4669,8 @@ TypeId Anyification::clean(TypeId ty) if (const TableTypeVar* ttv = log->getMutable(ty)) { TableTypeVar clone = TableTypeVar{ttv->props, ttv->indexer, ttv->level, TableState::Sealed}; - clone.methodDefinitionLocations = ttv->methodDefinitionLocations; + if (!FFlag::LuauNoMethodLocations) + clone.methodDefinitionLocations = ttv->methodDefinitionLocations; clone.definitionModuleName = ttv->definitionModuleName; clone.name = ttv->name; clone.syntheticName = ttv->syntheticName; diff --git a/Ast/src/Parser.cpp b/Ast/src/Parser.cpp index 91f5cd25..c053e6bd 100644 --- a/Ast/src/Parser.cpp +++ b/Ast/src/Parser.cpp @@ -10,7 +10,6 @@ // See docs/SyntaxChanges.md for an explanation. LUAU_FASTINTVARIABLE(LuauRecursionLimit, 1000) LUAU_FASTINTVARIABLE(LuauParseErrorLimit, 100) -LUAU_FASTFLAGVARIABLE(LuauParseLocationIgnoreCommentSkipInCapture, false) namespace Luau { @@ -2821,7 +2820,7 @@ void Parser::nextLexeme() hotcomments.push_back({hotcommentHeader, lexeme.location, std::string(text + 1, text + end)}); } - type = lexer.next(/* skipComments= */ false, !FFlag::LuauParseLocationIgnoreCommentSkipInCapture).type; + type = lexer.next(/* skipComments= */ false, /* updatePrevLocation= */ false).type; } } else diff --git a/CMakeLists.txt b/CMakeLists.txt index af03b33a..ea352309 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -110,6 +110,12 @@ if (MSVC AND MSVC_VERSION GREATER_EQUAL 1924) set_source_files_properties(VM/src/lvmexecute.cpp PROPERTIES COMPILE_FLAGS /d2ssa-pre-) endif() +if(MSVC AND LUAU_BUILD_CLI) + # the default stack size that MSVC linker uses is 1 MB; we need more stack space in Debug because stack frames are larger + set_target_properties(Luau.Analyze.CLI PROPERTIES LINK_FLAGS_DEBUG /STACK:2097152) + set_target_properties(Luau.Repl.CLI PROPERTIES LINK_FLAGS_DEBUG /STACK:2097152) +endif() + # embed .natvis inside the library debug information if(MSVC) target_link_options(Luau.Ast INTERFACE /NATVIS:${CMAKE_CURRENT_SOURCE_DIR}/tools/natvis/Ast.natvis) diff --git a/Compiler/src/Compiler.cpp b/Compiler/src/Compiler.cpp index 4fe26222..e177e928 100644 --- a/Compiler/src/Compiler.cpp +++ b/Compiler/src/Compiler.cpp @@ -628,7 +628,12 @@ struct Compiler return; if (fi && !fi->canInline) - bytecode.addDebugRemark("inlining failed: complex constructs in function body"); + { + if (func->vararg) + bytecode.addDebugRemark("inlining failed: function is variadic"); + else + bytecode.addDebugRemark("inlining failed: complex constructs in function body"); + } } RegScope rs(this); @@ -2342,17 +2347,28 @@ struct Compiler RegScope rs(this); uint8_t temp = 0; + bool consecutive = false; bool multRet = false; - // Optimization: return local value directly instead of copying it into a temporary - if (stat->list.size == 1 && isExprLocalReg(stat->list.data[0])) + // Optimization: return locals directly instead of copying them into a temporary + // this is very important for a single return value and occasionally effective for multiple values + if (stat->list.size > 0 && isExprLocalReg(stat->list.data[0])) { - AstExprLocal* le = stat->list.data[0]->as(); - LUAU_ASSERT(le); + temp = getLocal(stat->list.data[0]->as()->local); + consecutive = true; - temp = getLocal(le->local); + for (size_t i = 1; i < stat->list.size; ++i) + { + AstExpr* v = stat->list.data[i]; + if (!isExprLocalReg(v) || getLocal(v->as()->local) != temp + i) + { + consecutive = false; + break; + } + } } - else if (stat->list.size > 0) + + if (!consecutive && stat->list.size > 0) { temp = allocReg(stat, unsigned(stat->list.size)); diff --git a/Compiler/src/ConstantFolding.cpp b/Compiler/src/ConstantFolding.cpp index 52ece73e..e4d59ea1 100644 --- a/Compiler/src/ConstantFolding.cpp +++ b/Compiler/src/ConstantFolding.cpp @@ -195,12 +195,16 @@ struct ConstantVisitor : AstVisitor DenseHashMap& variables; DenseHashMap& locals; + bool wasEmpty = false; + ConstantVisitor( DenseHashMap& constants, DenseHashMap& variables, DenseHashMap& locals) : constants(constants) , variables(variables) , locals(locals) { + // since we do a single pass over the tree, if the initial state was empty we don't need to clear out old entries + wasEmpty = constants.empty() && locals.empty(); } Constant analyze(AstExpr* node) @@ -326,7 +330,7 @@ struct ConstantVisitor : AstVisitor { if (value.type != Constant::Type_Unknown) map[key] = value; - else if (!FFlag::LuauCompileSupportInlining) + else if (!FFlag::LuauCompileSupportInlining || wasEmpty) ; else if (Constant* old = map.find(key)) old->type = Constant::Type_Unknown; diff --git a/Compiler/src/CostModel.cpp b/Compiler/src/CostModel.cpp index 9afd09f6..f804e9de 100644 --- a/Compiler/src/CostModel.cpp +++ b/Compiler/src/CostModel.cpp @@ -187,7 +187,7 @@ struct CostVisitor : AstVisitor if (node->is()) result += 2; else if (node->is() || node->is() || node->is() || node->is()) - result += 2; + result += 5; else if (node->is() || node->is()) result += 1; diff --git a/extern/isocline/src/bbcode.c b/extern/isocline/src/bbcode.c index 4d11ac38..8722cbd6 100644 --- a/extern/isocline/src/bbcode.c +++ b/extern/isocline/src/bbcode.c @@ -575,6 +575,7 @@ ic_private const char* parse_tag_value( tag_t* tag, char* idbuf, const char* s, } // limit name and attr to 128 bytes char valbuf[128]; + valbuf[0] = 0; // fixes gcc uninitialized warning ic_strncpy( idbuf, 128, id, idend - id); ic_strncpy( valbuf, 128, val, valend - val); ic_str_tolower(idbuf); diff --git a/tests/AstQuery.test.cpp b/tests/AstQuery.test.cpp index 292625b0..12c68450 100644 --- a/tests/AstQuery.test.cpp +++ b/tests/AstQuery.test.cpp @@ -7,7 +7,7 @@ using namespace Luau; -struct DocumentationSymbolFixture : Fixture +struct DocumentationSymbolFixture : BuiltinsFixture { std::optional getDocSymbol(const std::string& source, Position position) { diff --git a/tests/Autocomplete.test.cpp b/tests/Autocomplete.test.cpp index 1c284f1f..b4e9340c 100644 --- a/tests/Autocomplete.test.cpp +++ b/tests/Autocomplete.test.cpp @@ -27,7 +27,7 @@ template struct ACFixtureImpl : BaseType { ACFixtureImpl() - : Fixture(true, true) + : BaseType(true, true) { } @@ -111,6 +111,18 @@ struct ACFixtureImpl : BaseType }; struct ACFixture : ACFixtureImpl +{ + ACFixture() + : ACFixtureImpl() + { + addGlobalBinding(frontend.typeChecker, "table", Binding{typeChecker.anyType}); + addGlobalBinding(frontend.typeChecker, "math", Binding{typeChecker.anyType}); + addGlobalBinding(frontend.typeCheckerForAutocomplete, "table", Binding{typeChecker.anyType}); + addGlobalBinding(frontend.typeCheckerForAutocomplete, "math", Binding{typeChecker.anyType}); + } +}; + +struct ACBuiltinsFixture : ACFixtureImpl { }; @@ -277,7 +289,7 @@ TEST_CASE_FIXTURE(ACFixture, "function_parameters") CHECK(ac.entryMap.count("test")); } -TEST_CASE_FIXTURE(ACFixture, "get_member_completions") +TEST_CASE_FIXTURE(ACBuiltinsFixture, "get_member_completions") { check(R"( local a = table.@1 @@ -376,7 +388,7 @@ TEST_CASE_FIXTURE(ACFixture, "table_intersection") CHECK(ac.entryMap.count("c3")); } -TEST_CASE_FIXTURE(ACFixture, "get_string_completions") +TEST_CASE_FIXTURE(ACBuiltinsFixture, "get_string_completions") { check(R"( local a = ("foo"):@1 @@ -427,7 +439,7 @@ TEST_CASE_FIXTURE(ACFixture, "method_call_inside_function_body") CHECK(!ac.entryMap.count("math")); } -TEST_CASE_FIXTURE(ACFixture, "method_call_inside_if_conditional") +TEST_CASE_FIXTURE(ACBuiltinsFixture, "method_call_inside_if_conditional") { check(R"( if table: @1 @@ -1884,7 +1896,7 @@ ex.b(function(x: CHECK(!ac.entryMap.count("(done) -> number")); } -TEST_CASE_FIXTURE(ACFixture, "suggest_external_module_type") +TEST_CASE_FIXTURE(ACBuiltinsFixture, "suggest_external_module_type") { fileResolver.source["Module/A"] = R"( export type done = { x: number, y: number } @@ -2235,7 +2247,7 @@ local a: aaa.do CHECK(ac.entryMap.count("other")); } -TEST_CASE_FIXTURE(ACFixture, "autocompleteSource") +TEST_CASE_FIXTURE(ACBuiltinsFixture, "autocompleteSource") { std::string_view source = R"( local a = table. -- Line 1 @@ -2269,7 +2281,7 @@ TEST_CASE_FIXTURE(ACFixture, "autocompleteSource_comments") CHECK_EQ(0, ac.entryMap.size()); } -TEST_CASE_FIXTURE(ACFixture, "autocompleteProp_index_function_metamethod_is_variadic") +TEST_CASE_FIXTURE(ACBuiltinsFixture, "autocompleteProp_index_function_metamethod_is_variadic") { std::string_view source = R"( type Foo = {x: number} @@ -2720,7 +2732,7 @@ type A = () -> T CHECK(ac.entryMap.count("string")); } -TEST_CASE_FIXTURE(ACFixture, "autocomplete_oop_implicit_self") +TEST_CASE_FIXTURE(ACBuiltinsFixture, "autocomplete_oop_implicit_self") { check(R"( --!strict @@ -2728,15 +2740,15 @@ local Class = {} Class.__index = Class type Class = typeof(setmetatable({} :: { x: number }, Class)) function Class.new(x: number): Class - return setmetatable({x = x}, Class) + return setmetatable({x = x}, Class) end function Class.getx(self: Class) - return self.x + return self.x end function test() - local c = Class.new(42) - local n = c:@1 - print(n) + local c = Class.new(42) + local n = c:@1 + print(n) end )"); @@ -2745,7 +2757,7 @@ end CHECK(ac.entryMap.count("getx")); } -TEST_CASE_FIXTURE(ACFixture, "autocomplete_on_string_singletons") +TEST_CASE_FIXTURE(ACBuiltinsFixture, "autocomplete_on_string_singletons") { check(R"( --!strict @@ -2989,7 +3001,7 @@ s.@1 CHECK(ac.entryMap["sub"].wrongIndexType == true); } -TEST_CASE_FIXTURE(ACFixture, "string_library_non_self_calls_are_fine") +TEST_CASE_FIXTURE(ACBuiltinsFixture, "string_library_non_self_calls_are_fine") { ScopedFastFlag selfCallAutocompleteFix{"LuauSelfCallAutocompleteFix", true}; @@ -3007,7 +3019,7 @@ string.@1 CHECK(ac.entryMap["sub"].wrongIndexType == false); } -TEST_CASE_FIXTURE(ACFixture, "string_library_self_calls_are_invalid") +TEST_CASE_FIXTURE(ACBuiltinsFixture, "string_library_self_calls_are_invalid") { ScopedFastFlag selfCallAutocompleteFix{"LuauSelfCallAutocompleteFix", true}; diff --git a/tests/BuiltinDefinitions.test.cpp b/tests/BuiltinDefinitions.test.cpp index dbe80f2c..496df4b4 100644 --- a/tests/BuiltinDefinitions.test.cpp +++ b/tests/BuiltinDefinitions.test.cpp @@ -10,8 +10,10 @@ using namespace Luau; TEST_SUITE_BEGIN("BuiltinDefinitionsTest"); -TEST_CASE_FIXTURE(Fixture, "lib_documentation_symbols") +TEST_CASE_FIXTURE(BuiltinsFixture, "lib_documentation_symbols") { + CHECK(!typeChecker.globalScope->bindings.empty()); + for (const auto& [name, binding] : typeChecker.globalScope->bindings) { std::string nameString(name.c_str()); diff --git a/tests/Compiler.test.cpp b/tests/Compiler.test.cpp index f206438f..b032060e 100644 --- a/tests/Compiler.test.cpp +++ b/tests/Compiler.test.cpp @@ -4713,7 +4713,6 @@ local function foo() end local a, b = foo() - return a, b )", 1, 2), @@ -4721,9 +4720,7 @@ return a, b DUPCLOSURE R0 K0 LOADNIL R1 LOADNIL R2 -MOVE R3 R1 -MOVE R4 R2 -RETURN R3 2 +RETURN R1 2 )"); // this happens even if the function returns conditionally @@ -4733,7 +4730,6 @@ local function foo(a) end local a, b = foo(false) - return a, b )", 1, 2), @@ -4741,9 +4737,7 @@ return a, b DUPCLOSURE R0 K0 LOADNIL R1 LOADNIL R2 -MOVE R3 R1 -MOVE R4 R2 -RETURN R3 2 +RETURN R1 2 )"); // note though that we can't inline a function like this in multret context @@ -4880,11 +4874,7 @@ LOADN R5 1 ADD R4 R5 R1 LOADN R5 3 ADD R6 R1 R2 -MOVE R7 R3 -MOVE R8 R4 -MOVE R9 R5 -MOVE R10 R6 -RETURN R7 4 +RETURN R3 4 )"); } @@ -5151,4 +5141,59 @@ RETURN R0 0 )"); } +TEST_CASE("ReturnConsecutive") +{ + // we can return a single local directly + CHECK_EQ("\n" + compileFunction0(R"( +local x = ... +return x +)"), + R"( +GETVARARGS R0 1 +RETURN R0 1 +)"); + + // or multiple, when they are allocated in consecutive registers + CHECK_EQ("\n" + compileFunction0(R"( +local x, y = ... +return x, y +)"), + R"( +GETVARARGS R0 2 +RETURN R0 2 +)"); + + // but not if it's an expression + CHECK_EQ("\n" + compileFunction0(R"( +local x, y = ... +return x, y + 1 +)"), + R"( +GETVARARGS R0 2 +MOVE R2 R0 +ADDK R3 R1 K0 +RETURN R2 2 +)"); + + // or a local with wrong register number + CHECK_EQ("\n" + compileFunction0(R"( +local x, y = ... +return y, x +)"), + R"( +GETVARARGS R0 2 +MOVE R2 R1 +MOVE R3 R0 +RETURN R2 2 +)"); + + // also double check the optimization doesn't trip on no-argument return (these are rare) + CHECK_EQ("\n" + compileFunction0(R"( +return +)"), + R"( +RETURN R0 0 +)"); +} + TEST_SUITE_END(); diff --git a/tests/CostModel.test.cpp b/tests/CostModel.test.cpp index aa5b7284..2fa0659b 100644 --- a/tests/CostModel.test.cpp +++ b/tests/CostModel.test.cpp @@ -76,9 +76,9 @@ end const bool args1[] = {false}; const bool args2[] = {true}; - // loop baseline cost is 2 - CHECK_EQ(3, Luau::Compile::computeCost(model, args1, 1)); - CHECK_EQ(3, Luau::Compile::computeCost(model, args2, 1)); + // loop baseline cost is 5 + CHECK_EQ(6, Luau::Compile::computeCost(model, args1, 1)); + CHECK_EQ(6, Luau::Compile::computeCost(model, args2, 1)); } TEST_CASE("MutableVariable") @@ -154,8 +154,8 @@ end const bool args1[] = {false}; const bool args2[] = {true}; - CHECK_EQ(38, Luau::Compile::computeCost(model, args1, 1)); - CHECK_EQ(37, Luau::Compile::computeCost(model, args2, 1)); + CHECK_EQ(50, Luau::Compile::computeCost(model, args1, 1)); + CHECK_EQ(49, Luau::Compile::computeCost(model, args2, 1)); } TEST_CASE("Conditional") @@ -219,8 +219,8 @@ end const bool args1[] = {false}; const bool args2[] = {true}; - CHECK_EQ(4, Luau::Compile::computeCost(model, args1, 1)); - CHECK_EQ(3, Luau::Compile::computeCost(model, args2, 1)); + CHECK_EQ(7, Luau::Compile::computeCost(model, args1, 1)); + CHECK_EQ(6, Luau::Compile::computeCost(model, args2, 1)); } TEST_SUITE_END(); diff --git a/tests/Fixture.cpp b/tests/Fixture.cpp index d8b37a65..03f3e15c 100644 --- a/tests/Fixture.cpp +++ b/tests/Fixture.cpp @@ -92,10 +92,6 @@ Fixture::Fixture(bool freeze, bool prepareAutocomplete) configResolver.defaultConfig.enabledLint.warningMask = ~0ull; configResolver.defaultConfig.parseOptions.captureComments = true; - registerBuiltinTypes(frontend.typeChecker); - if (prepareAutocomplete) - registerBuiltinTypes(frontend.typeCheckerForAutocomplete); - registerTestTypes(); Luau::freeze(frontend.typeChecker.globalTypes); Luau::freeze(frontend.typeCheckerForAutocomplete.globalTypes); @@ -410,6 +406,21 @@ LoadDefinitionFileResult Fixture::loadDefinition(const std::string& source) return result; } +BuiltinsFixture::BuiltinsFixture(bool freeze, bool prepareAutocomplete) + : Fixture(freeze, prepareAutocomplete) +{ + Luau::unfreeze(frontend.typeChecker.globalTypes); + Luau::unfreeze(frontend.typeCheckerForAutocomplete.globalTypes); + + registerBuiltinTypes(frontend.typeChecker); + if (prepareAutocomplete) + registerBuiltinTypes(frontend.typeCheckerForAutocomplete); + registerTestTypes(); + + Luau::freeze(frontend.typeChecker.globalTypes); + Luau::freeze(frontend.typeCheckerForAutocomplete.globalTypes); +} + ModuleName fromString(std::string_view name) { return ModuleName(name); diff --git a/tests/Fixture.h b/tests/Fixture.h index 0d1233bf..901f7d42 100644 --- a/tests/Fixture.h +++ b/tests/Fixture.h @@ -151,6 +151,11 @@ struct Fixture LoadDefinitionFileResult loadDefinition(const std::string& source); }; +struct BuiltinsFixture : Fixture +{ + BuiltinsFixture(bool freeze = true, bool prepareAutocomplete = false); +}; + ModuleName fromString(std::string_view name); template diff --git a/tests/Frontend.test.cpp b/tests/Frontend.test.cpp index a10e8f7f..33b81be8 100644 --- a/tests/Frontend.test.cpp +++ b/tests/Frontend.test.cpp @@ -77,7 +77,7 @@ struct NaiveFileResolver : NullFileResolver } // namespace -struct FrontendFixture : Fixture +struct FrontendFixture : BuiltinsFixture { FrontendFixture() { diff --git a/tests/Linter.test.cpp b/tests/Linter.test.cpp index 6649cb7f..202aeceb 100644 --- a/tests/Linter.test.cpp +++ b/tests/Linter.test.cpp @@ -75,7 +75,7 @@ _ = 6 CHECK_EQ(result.warnings.size(), 0); } -TEST_CASE_FIXTURE(Fixture, "BuiltinGlobalWrite") +TEST_CASE_FIXTURE(BuiltinsFixture, "BuiltinGlobalWrite") { LintResult result = lint(R"( math = {} @@ -309,7 +309,7 @@ print(arg) CHECK_EQ(result.warnings[0].text, "Variable 'arg' shadows previous declaration at line 2"); } -TEST_CASE_FIXTURE(Fixture, "LocalShadowGlobal") +TEST_CASE_FIXTURE(BuiltinsFixture, "LocalShadowGlobal") { LintResult result = lint(R"( local math = math @@ -1470,7 +1470,7 @@ end CHECK_EQ(result.warnings[2].text, "Member 'Instance.DataCost' is deprecated"); } -TEST_CASE_FIXTURE(Fixture, "TableOperations") +TEST_CASE_FIXTURE(BuiltinsFixture, "TableOperations") { LintResult result = lintTyped(R"( local t = {} diff --git a/tests/Module.test.cpp b/tests/Module.test.cpp index 44cc20a7..4a999861 100644 --- a/tests/Module.test.cpp +++ b/tests/Module.test.cpp @@ -113,7 +113,7 @@ TEST_CASE_FIXTURE(Fixture, "deepClone_cyclic_table") CHECK_EQ(2, dest.typeVars.size()); // One table and one function } -TEST_CASE_FIXTURE(Fixture, "builtin_types_point_into_globalTypes_arena") +TEST_CASE_FIXTURE(BuiltinsFixture, "builtin_types_point_into_globalTypes_arena") { CheckResult result = check(R"( return {sign=math.sign} @@ -250,7 +250,7 @@ TEST_CASE_FIXTURE(Fixture, "clone_constrained_intersection") CHECK_EQ(getSingletonTypes().stringType, ctv->parts[1]); } -TEST_CASE_FIXTURE(Fixture, "clone_self_property") +TEST_CASE_FIXTURE(BuiltinsFixture, "clone_self_property") { ScopedFastFlag sff{"LuauAnyInIsOptionalIsOptional", true}; diff --git a/tests/NonstrictMode.test.cpp b/tests/NonstrictMode.test.cpp index feeaf2c2..69430b1c 100644 --- a/tests/NonstrictMode.test.cpp +++ b/tests/NonstrictMode.test.cpp @@ -13,7 +13,7 @@ using namespace Luau; TEST_SUITE_BEGIN("NonstrictModeTests"); -TEST_CASE_FIXTURE(Fixture, "function_returns_number_or_string") +TEST_CASE_FIXTURE(BuiltinsFixture, "function_returns_number_or_string") { ScopedFastFlag sff[]{ {"LuauReturnTypeInferenceInNonstrict", true}, @@ -224,7 +224,7 @@ TEST_CASE_FIXTURE(Fixture, "inline_table_props_are_also_any") CHECK_MESSAGE(get(ttv->props["three"].type), "Should be a function: " << *ttv->props["three"].type); } -TEST_CASE_FIXTURE(Fixture, "for_in_iterator_variables_are_any") +TEST_CASE_FIXTURE(BuiltinsFixture, "for_in_iterator_variables_are_any") { CheckResult result = check(R"( --!nonstrict @@ -243,7 +243,7 @@ TEST_CASE_FIXTURE(Fixture, "for_in_iterator_variables_are_any") LUAU_REQUIRE_NO_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "table_dot_insert_and_recursive_calls") +TEST_CASE_FIXTURE(BuiltinsFixture, "table_dot_insert_and_recursive_calls") { CheckResult result = check(R"( --!nonstrict diff --git a/tests/Normalize.test.cpp b/tests/Normalize.test.cpp index d3778f67..41830682 100644 --- a/tests/Normalize.test.cpp +++ b/tests/Normalize.test.cpp @@ -739,7 +739,7 @@ TEST_CASE_FIXTURE(Fixture, "cyclic_table_normalizes_sensibly") CHECK_EQ("t1 where t1 = { get: () -> t1 }", toString(ty, {true})); } -TEST_CASE_FIXTURE(Fixture, "union_of_distinct_free_types") +TEST_CASE_FIXTURE(BuiltinsFixture, "union_of_distinct_free_types") { ScopedFastFlag flags[] = { {"LuauLowerBoundsCalculation", true}, @@ -760,7 +760,7 @@ TEST_CASE_FIXTURE(Fixture, "union_of_distinct_free_types") CHECK("(a, b) -> a | b" == toString(requireType("fussy"))); } -TEST_CASE_FIXTURE(Fixture, "constrained_intersection_of_intersections") +TEST_CASE_FIXTURE(BuiltinsFixture, "constrained_intersection_of_intersections") { ScopedFastFlag flags[] = { {"LuauLowerBoundsCalculation", true}, @@ -951,7 +951,7 @@ TEST_CASE_FIXTURE(Fixture, "nested_table_normalization_with_non_table__no_ice") LUAU_REQUIRE_NO_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "visiting_a_type_twice_is_not_considered_normal") +TEST_CASE_FIXTURE(BuiltinsFixture, "visiting_a_type_twice_is_not_considered_normal") { ScopedFastFlag sff{"LuauLowerBoundsCalculation", true}; @@ -976,7 +976,6 @@ TEST_CASE_FIXTURE(Fixture, "fuzz_failure_instersection_combine_must_follow") { ScopedFastFlag flags[] = { {"LuauLowerBoundsCalculation", true}, - {"LuauNormalizeCombineIntersectionFix", true}, }; CheckResult result = check(R"( diff --git a/tests/Parser.test.cpp b/tests/Parser.test.cpp index 69ff73ad..c9d8d0b8 100644 --- a/tests/Parser.test.cpp +++ b/tests/Parser.test.cpp @@ -1618,8 +1618,6 @@ TEST_CASE_FIXTURE(Fixture, "end_extent_doesnt_consume_comments") TEST_CASE_FIXTURE(Fixture, "end_extent_doesnt_consume_comments_even_with_capture") { - ScopedFastFlag luauParseLocationIgnoreCommentSkipInCapture{"LuauParseLocationIgnoreCommentSkipInCapture", true}; - // Same should hold when comments are captured ParseOptions opts; opts.captureComments = true; diff --git a/tests/RuntimeLimits.test.cpp b/tests/RuntimeLimits.test.cpp index 538f3576..c16f60d5 100644 --- a/tests/RuntimeLimits.test.cpp +++ b/tests/RuntimeLimits.test.cpp @@ -17,7 +17,7 @@ using namespace Luau; LUAU_FASTFLAG(LuauLowerBoundsCalculation); -struct LimitFixture : Fixture +struct LimitFixture : BuiltinsFixture { #if defined(_NOOPT) || defined(_DEBUG) ScopedFastInt LuauTypeInferRecursionLimit{"LuauTypeInferRecursionLimit", 100}; diff --git a/tests/ToDot.test.cpp b/tests/ToDot.test.cpp index 332a4b22..e9fa5b26 100644 --- a/tests/ToDot.test.cpp +++ b/tests/ToDot.test.cpp @@ -224,7 +224,7 @@ n1 -> n4 [label="typePackParam"]; (void)toDot(requireType("a")); } -TEST_CASE_FIXTURE(Fixture, "metatable") +TEST_CASE_FIXTURE(BuiltinsFixture, "metatable") { CheckResult result = check(R"( local a: typeof(setmetatable({}, {})) diff --git a/tests/ToString.test.cpp b/tests/ToString.test.cpp index ccf5c583..50d0838e 100644 --- a/tests/ToString.test.cpp +++ b/tests/ToString.test.cpp @@ -60,7 +60,7 @@ TEST_CASE_FIXTURE(Fixture, "named_table") CHECK_EQ("TheTable", toString(&table)); } -TEST_CASE_FIXTURE(Fixture, "exhaustive_toString_of_cyclic_table") +TEST_CASE_FIXTURE(BuiltinsFixture, "exhaustive_toString_of_cyclic_table") { CheckResult result = check(R"( --!strict @@ -338,7 +338,7 @@ TEST_CASE_FIXTURE(Fixture, "toStringDetailed") REQUIRE_EQ("c", toString(params[2], opts)); } -TEST_CASE_FIXTURE(Fixture, "toStringDetailed2") +TEST_CASE_FIXTURE(BuiltinsFixture, "toStringDetailed2") { ScopedFastFlag sff{"LuauUnsealedTableLiteral", true}; diff --git a/tests/TypeInfer.aliases.test.cpp b/tests/TypeInfer.aliases.test.cpp index b0eb31ce..7562a4d7 100644 --- a/tests/TypeInfer.aliases.test.cpp +++ b/tests/TypeInfer.aliases.test.cpp @@ -279,7 +279,7 @@ TEST_CASE_FIXTURE(Fixture, "stringify_optional_parameterized_alias") CHECK_EQ("Node", toString(e->wantedType)); } -TEST_CASE_FIXTURE(Fixture, "general_require_multi_assign") +TEST_CASE_FIXTURE(BuiltinsFixture, "general_require_multi_assign") { fileResolver.source["workspace/A"] = R"( export type myvec2 = {x: number, y: number} @@ -317,7 +317,7 @@ TEST_CASE_FIXTURE(Fixture, "general_require_multi_assign") REQUIRE(bType->props.size() == 3); } -TEST_CASE_FIXTURE(Fixture, "type_alias_import_mutation") +TEST_CASE_FIXTURE(BuiltinsFixture, "type_alias_import_mutation") { CheckResult result = check("type t10 = typeof(table)"); LUAU_REQUIRE_NO_ERRORS(result); @@ -385,7 +385,7 @@ type Cool = typeof(c) CHECK_EQ(ttv->name, "Cool"); } -TEST_CASE_FIXTURE(Fixture, "type_alias_of_an_imported_recursive_type") +TEST_CASE_FIXTURE(BuiltinsFixture, "type_alias_of_an_imported_recursive_type") { fileResolver.source["game/A"] = R"( export type X = { a: number, b: X? } @@ -410,7 +410,7 @@ type X = Import.X CHECK_EQ(follow(*ty1), follow(*ty2)); } -TEST_CASE_FIXTURE(Fixture, "type_alias_of_an_imported_recursive_generic_type") +TEST_CASE_FIXTURE(BuiltinsFixture, "type_alias_of_an_imported_recursive_generic_type") { fileResolver.source["game/A"] = R"( export type X = { a: T, b: U, C: X? } @@ -564,7 +564,7 @@ TEST_CASE_FIXTURE(Fixture, "non_recursive_aliases_that_reuse_a_generic_name") * * We solved this by ascribing a unique subLevel to each prototyped alias. */ -TEST_CASE_FIXTURE(Fixture, "do_not_quantify_unresolved_aliases") +TEST_CASE_FIXTURE(BuiltinsFixture, "do_not_quantify_unresolved_aliases") { CheckResult result = check(R"( --!strict diff --git a/tests/TypeInfer.annotations.test.cpp b/tests/TypeInfer.annotations.test.cpp index 7f1c757a..b9e1ae96 100644 --- a/tests/TypeInfer.annotations.test.cpp +++ b/tests/TypeInfer.annotations.test.cpp @@ -528,7 +528,7 @@ TEST_CASE_FIXTURE(Fixture, "cloned_interface_maintains_pointers_between_definiti CHECK_EQ(recordType, bType); } -TEST_CASE_FIXTURE(Fixture, "use_type_required_from_another_file") +TEST_CASE_FIXTURE(BuiltinsFixture, "use_type_required_from_another_file") { addGlobalBinding(frontend.typeChecker, "script", frontend.typeChecker.anyType, "@test"); @@ -554,7 +554,7 @@ TEST_CASE_FIXTURE(Fixture, "use_type_required_from_another_file") LUAU_REQUIRE_NO_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "cannot_use_nonexported_type") +TEST_CASE_FIXTURE(BuiltinsFixture, "cannot_use_nonexported_type") { addGlobalBinding(frontend.typeChecker, "script", frontend.typeChecker.anyType, "@test"); @@ -580,7 +580,7 @@ TEST_CASE_FIXTURE(Fixture, "cannot_use_nonexported_type") LUAU_REQUIRE_ERROR_COUNT(1, result); } -TEST_CASE_FIXTURE(Fixture, "builtin_types_are_not_exported") +TEST_CASE_FIXTURE(BuiltinsFixture, "builtin_types_are_not_exported") { addGlobalBinding(frontend.typeChecker, "script", frontend.typeChecker.anyType, "@test"); @@ -676,7 +676,7 @@ TEST_CASE_FIXTURE(Fixture, "luau_ice_is_not_special_without_the_flag") )"); } -TEST_CASE_FIXTURE(Fixture, "luau_print_is_magic_if_the_flag_is_set") +TEST_CASE_FIXTURE(BuiltinsFixture, "luau_print_is_magic_if_the_flag_is_set") { // Luau::resetPrintLine(); ScopedFastFlag sffs{"DebugLuauMagicTypes", true}; diff --git a/tests/TypeInfer.anyerror.test.cpp b/tests/TypeInfer.anyerror.test.cpp index 5224b5d8..bc55940e 100644 --- a/tests/TypeInfer.anyerror.test.cpp +++ b/tests/TypeInfer.anyerror.test.cpp @@ -237,7 +237,7 @@ TEST_CASE_FIXTURE(Fixture, "chain_calling_error_type_yields_error") CHECK_EQ("*unknown*", toString(requireType("a"))); } -TEST_CASE_FIXTURE(Fixture, "replace_every_free_type_when_unifying_a_complex_function_with_any") +TEST_CASE_FIXTURE(BuiltinsFixture, "replace_every_free_type_when_unifying_a_complex_function_with_any") { CheckResult result = check(R"( local a: any @@ -285,7 +285,7 @@ end LUAU_REQUIRE_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "metatable_of_any_can_be_a_table") +TEST_CASE_FIXTURE(BuiltinsFixture, "metatable_of_any_can_be_a_table") { CheckResult result = check(R"( --!strict diff --git a/tests/TypeInfer.builtins.test.cpp b/tests/TypeInfer.builtins.test.cpp index 1ae65947..b710ea0d 100644 --- a/tests/TypeInfer.builtins.test.cpp +++ b/tests/TypeInfer.builtins.test.cpp @@ -12,7 +12,7 @@ LUAU_FASTFLAG(LuauLowerBoundsCalculation); TEST_SUITE_BEGIN("BuiltinTests"); -TEST_CASE_FIXTURE(Fixture, "math_things_are_defined") +TEST_CASE_FIXTURE(BuiltinsFixture, "math_things_are_defined") { CheckResult result = check(R"( local a00 = math.frexp @@ -50,7 +50,7 @@ TEST_CASE_FIXTURE(Fixture, "math_things_are_defined") LUAU_REQUIRE_NO_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "next_iterator_should_infer_types_and_type_check") +TEST_CASE_FIXTURE(BuiltinsFixture, "next_iterator_should_infer_types_and_type_check") { CheckResult result = check(R"( local a: string, b: number = next({ 1 }) @@ -63,7 +63,7 @@ TEST_CASE_FIXTURE(Fixture, "next_iterator_should_infer_types_and_type_check") LUAU_REQUIRE_ERROR_COUNT(1, result); } -TEST_CASE_FIXTURE(Fixture, "pairs_iterator_should_infer_types_and_type_check") +TEST_CASE_FIXTURE(BuiltinsFixture, "pairs_iterator_should_infer_types_and_type_check") { CheckResult result = check(R"( type Map = { [K]: V } @@ -75,7 +75,7 @@ TEST_CASE_FIXTURE(Fixture, "pairs_iterator_should_infer_types_and_type_check") LUAU_REQUIRE_NO_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "ipairs_iterator_should_infer_types_and_type_check") +TEST_CASE_FIXTURE(BuiltinsFixture, "ipairs_iterator_should_infer_types_and_type_check") { CheckResult result = check(R"( type Map = { [K]: V } @@ -87,7 +87,7 @@ TEST_CASE_FIXTURE(Fixture, "ipairs_iterator_should_infer_types_and_type_check") LUAU_REQUIRE_NO_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "table_dot_remove_optionally_returns_generic") +TEST_CASE_FIXTURE(BuiltinsFixture, "table_dot_remove_optionally_returns_generic") { CheckResult result = check(R"( local t = { 1 } @@ -98,7 +98,7 @@ TEST_CASE_FIXTURE(Fixture, "table_dot_remove_optionally_returns_generic") CHECK_EQ(toString(requireType("n")), "number?"); } -TEST_CASE_FIXTURE(Fixture, "table_concat_returns_string") +TEST_CASE_FIXTURE(BuiltinsFixture, "table_concat_returns_string") { CheckResult result = check(R"( local r = table.concat({1,2,3,4}, ",", 2); @@ -108,7 +108,7 @@ TEST_CASE_FIXTURE(Fixture, "table_concat_returns_string") CHECK_EQ(*typeChecker.stringType, *requireType("r")); } -TEST_CASE_FIXTURE(Fixture, "sort") +TEST_CASE_FIXTURE(BuiltinsFixture, "sort") { CheckResult result = check(R"( local t = {1, 2, 3}; @@ -118,7 +118,7 @@ TEST_CASE_FIXTURE(Fixture, "sort") LUAU_REQUIRE_NO_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "sort_with_predicate") +TEST_CASE_FIXTURE(BuiltinsFixture, "sort_with_predicate") { CheckResult result = check(R"( --!strict @@ -130,7 +130,7 @@ TEST_CASE_FIXTURE(Fixture, "sort_with_predicate") LUAU_REQUIRE_NO_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "sort_with_bad_predicate") +TEST_CASE_FIXTURE(BuiltinsFixture, "sort_with_bad_predicate") { CheckResult result = check(R"( --!strict @@ -140,6 +140,12 @@ TEST_CASE_FIXTURE(Fixture, "sort_with_bad_predicate") )"); LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ(R"(Type '(number, number) -> boolean' could not be converted into '((a, a) -> boolean)?' +caused by: + None of the union options are compatible. For example: Type '(number, number) -> boolean' could not be converted into '(a, a) -> boolean' +caused by: + Argument #1 type is not compatible. Type 'string' could not be converted into 'number')", + toString(result.errors[0])); } TEST_CASE_FIXTURE(Fixture, "strings_have_methods") @@ -152,7 +158,7 @@ TEST_CASE_FIXTURE(Fixture, "strings_have_methods") CHECK_EQ(*typeChecker.stringType, *requireType("s")); } -TEST_CASE_FIXTURE(Fixture, "math_max_variatic") +TEST_CASE_FIXTURE(BuiltinsFixture, "math_max_variatic") { CheckResult result = check(R"( local n = math.max(1,2,3,4,5,6,7,8,9,0) @@ -162,16 +168,17 @@ TEST_CASE_FIXTURE(Fixture, "math_max_variatic") CHECK_EQ(*typeChecker.numberType, *requireType("n")); } -TEST_CASE_FIXTURE(Fixture, "math_max_checks_for_numbers") +TEST_CASE_FIXTURE(BuiltinsFixture, "math_max_checks_for_numbers") { CheckResult result = check(R"( local n = math.max(1,2,"3") )"); CHECK(!result.errors.empty()); + CHECK_EQ("Type 'string' could not be converted into 'number'", toString(result.errors[0])); } -TEST_CASE_FIXTURE(Fixture, "builtin_tables_sealed") +TEST_CASE_FIXTURE(BuiltinsFixture, "builtin_tables_sealed") { CheckResult result = check(R"LUA( local b = bit32 @@ -183,7 +190,7 @@ TEST_CASE_FIXTURE(Fixture, "builtin_tables_sealed") CHECK_EQ(bit32t->state, TableState::Sealed); } -TEST_CASE_FIXTURE(Fixture, "lua_51_exported_globals_all_exist") +TEST_CASE_FIXTURE(BuiltinsFixture, "lua_51_exported_globals_all_exist") { // Extracted from lua5.1 CheckResult result = check(R"( @@ -340,7 +347,7 @@ TEST_CASE_FIXTURE(Fixture, "lua_51_exported_globals_all_exist") LUAU_REQUIRE_NO_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "setmetatable_unpacks_arg_types_correctly") +TEST_CASE_FIXTURE(BuiltinsFixture, "setmetatable_unpacks_arg_types_correctly") { CheckResult result = check(R"( setmetatable({}, setmetatable({}, {})) @@ -348,7 +355,7 @@ TEST_CASE_FIXTURE(Fixture, "setmetatable_unpacks_arg_types_correctly") LUAU_REQUIRE_NO_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "table_insert_correctly_infers_type_of_array_2_args_overload") +TEST_CASE_FIXTURE(BuiltinsFixture, "table_insert_correctly_infers_type_of_array_2_args_overload") { CheckResult result = check(R"( local t = {} @@ -360,7 +367,7 @@ TEST_CASE_FIXTURE(Fixture, "table_insert_correctly_infers_type_of_array_2_args_o CHECK_EQ(typeChecker.stringType, requireType("s")); } -TEST_CASE_FIXTURE(Fixture, "table_insert_correctly_infers_type_of_array_3_args_overload") +TEST_CASE_FIXTURE(BuiltinsFixture, "table_insert_correctly_infers_type_of_array_3_args_overload") { CheckResult result = check(R"( local t = {} @@ -372,7 +379,7 @@ TEST_CASE_FIXTURE(Fixture, "table_insert_correctly_infers_type_of_array_3_args_o CHECK_EQ("string", toString(requireType("s"))); } -TEST_CASE_FIXTURE(Fixture, "table_pack") +TEST_CASE_FIXTURE(BuiltinsFixture, "table_pack") { CheckResult result = check(R"( local t = table.pack(1, "foo", true) @@ -382,7 +389,7 @@ TEST_CASE_FIXTURE(Fixture, "table_pack") CHECK_EQ("{| [number]: boolean | number | string, n: number |}", toString(requireType("t"))); } -TEST_CASE_FIXTURE(Fixture, "table_pack_variadic") +TEST_CASE_FIXTURE(BuiltinsFixture, "table_pack_variadic") { CheckResult result = check(R"( --!strict @@ -397,7 +404,7 @@ local t = table.pack(f()) CHECK_EQ("{| [number]: number | string, n: number |}", toString(requireType("t"))); } -TEST_CASE_FIXTURE(Fixture, "table_pack_reduce") +TEST_CASE_FIXTURE(BuiltinsFixture, "table_pack_reduce") { CheckResult result = check(R"( local t = table.pack(1, 2, true) @@ -414,7 +421,7 @@ TEST_CASE_FIXTURE(Fixture, "table_pack_reduce") CHECK_EQ("{| [number]: string, n: number |}", toString(requireType("t"))); } -TEST_CASE_FIXTURE(Fixture, "gcinfo") +TEST_CASE_FIXTURE(BuiltinsFixture, "gcinfo") { CheckResult result = check(R"( local n = gcinfo() @@ -424,12 +431,12 @@ TEST_CASE_FIXTURE(Fixture, "gcinfo") CHECK_EQ(*typeChecker.numberType, *requireType("n")); } -TEST_CASE_FIXTURE(Fixture, "getfenv") +TEST_CASE_FIXTURE(BuiltinsFixture, "getfenv") { LUAU_REQUIRE_NO_ERRORS(check("getfenv(1)")); } -TEST_CASE_FIXTURE(Fixture, "os_time_takes_optional_date_table") +TEST_CASE_FIXTURE(BuiltinsFixture, "os_time_takes_optional_date_table") { CheckResult result = check(R"( local n1 = os.time() @@ -443,7 +450,7 @@ TEST_CASE_FIXTURE(Fixture, "os_time_takes_optional_date_table") CHECK_EQ(*typeChecker.numberType, *requireType("n3")); } -TEST_CASE_FIXTURE(Fixture, "thread_is_a_type") +TEST_CASE_FIXTURE(BuiltinsFixture, "thread_is_a_type") { CheckResult result = check(R"( local co = coroutine.create(function() end) @@ -453,7 +460,7 @@ TEST_CASE_FIXTURE(Fixture, "thread_is_a_type") CHECK_EQ(*typeChecker.threadType, *requireType("co")); } -TEST_CASE_FIXTURE(Fixture, "coroutine_resume_anything_goes") +TEST_CASE_FIXTURE(BuiltinsFixture, "coroutine_resume_anything_goes") { CheckResult result = check(R"( local function nifty(x, y) @@ -471,7 +478,7 @@ TEST_CASE_FIXTURE(Fixture, "coroutine_resume_anything_goes") LUAU_REQUIRE_NO_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "coroutine_wrap_anything_goes") +TEST_CASE_FIXTURE(BuiltinsFixture, "coroutine_wrap_anything_goes") { CheckResult result = check(R"( --!nonstrict @@ -490,7 +497,7 @@ TEST_CASE_FIXTURE(Fixture, "coroutine_wrap_anything_goes") LUAU_REQUIRE_NO_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "setmetatable_should_not_mutate_persisted_types") +TEST_CASE_FIXTURE(BuiltinsFixture, "setmetatable_should_not_mutate_persisted_types") { CheckResult result = check(R"( local string = string @@ -505,7 +512,7 @@ TEST_CASE_FIXTURE(Fixture, "setmetatable_should_not_mutate_persisted_types") REQUIRE(ttv); } -TEST_CASE_FIXTURE(Fixture, "string_format_arg_types_inference") +TEST_CASE_FIXTURE(BuiltinsFixture, "string_format_arg_types_inference") { CheckResult result = check(R"( --!strict @@ -518,7 +525,7 @@ TEST_CASE_FIXTURE(Fixture, "string_format_arg_types_inference") CHECK_EQ("(number, number, string) -> string", toString(requireType("f"))); } -TEST_CASE_FIXTURE(Fixture, "string_format_arg_count_mismatch") +TEST_CASE_FIXTURE(BuiltinsFixture, "string_format_arg_count_mismatch") { CheckResult result = check(R"( --!strict @@ -534,7 +541,7 @@ TEST_CASE_FIXTURE(Fixture, "string_format_arg_count_mismatch") CHECK_EQ(result.errors[2].location.begin.line, 4); } -TEST_CASE_FIXTURE(Fixture, "string_format_correctly_ordered_types") +TEST_CASE_FIXTURE(BuiltinsFixture, "string_format_correctly_ordered_types") { CheckResult result = check(R"( --!strict @@ -548,7 +555,7 @@ TEST_CASE_FIXTURE(Fixture, "string_format_correctly_ordered_types") CHECK_EQ(tm->givenType, typeChecker.numberType); } -TEST_CASE_FIXTURE(Fixture, "xpcall") +TEST_CASE_FIXTURE(BuiltinsFixture, "xpcall") { CheckResult result = check(R"( --!strict @@ -564,7 +571,7 @@ TEST_CASE_FIXTURE(Fixture, "xpcall") CHECK_EQ("boolean", toString(requireType("c"))); } -TEST_CASE_FIXTURE(Fixture, "see_thru_select") +TEST_CASE_FIXTURE(BuiltinsFixture, "see_thru_select") { CheckResult result = check(R"( local a:number, b:boolean = select(2,"hi", 10, true) @@ -573,7 +580,7 @@ TEST_CASE_FIXTURE(Fixture, "see_thru_select") LUAU_REQUIRE_NO_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "see_thru_select_count") +TEST_CASE_FIXTURE(BuiltinsFixture, "see_thru_select_count") { CheckResult result = check(R"( local a = select("#","hi", 10, true) @@ -583,7 +590,7 @@ TEST_CASE_FIXTURE(Fixture, "see_thru_select_count") LUAU_REQUIRE_NO_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "select_with_decimal_argument_is_rounded_down") +TEST_CASE_FIXTURE(BuiltinsFixture, "select_with_decimal_argument_is_rounded_down") { CheckResult result = check(R"( local a: number, b: boolean = select(2.9, "foo", 1, true) @@ -608,7 +615,7 @@ TEST_CASE_FIXTURE(Fixture, "bad_select_should_not_crash") CHECK_LE(0, result.errors.size()); } -TEST_CASE_FIXTURE(Fixture, "select_way_out_of_range") +TEST_CASE_FIXTURE(BuiltinsFixture, "select_way_out_of_range") { CheckResult result = check(R"( select(5432598430953240958) @@ -619,7 +626,7 @@ TEST_CASE_FIXTURE(Fixture, "select_way_out_of_range") CHECK_EQ("bad argument #1 to select (index out of range)", toString(result.errors[0])); } -TEST_CASE_FIXTURE(Fixture, "select_slightly_out_of_range") +TEST_CASE_FIXTURE(BuiltinsFixture, "select_slightly_out_of_range") { CheckResult result = check(R"( select(3, "a", 1) @@ -630,7 +637,7 @@ TEST_CASE_FIXTURE(Fixture, "select_slightly_out_of_range") CHECK_EQ("bad argument #1 to select (index out of range)", toString(result.errors[0])); } -TEST_CASE_FIXTURE(Fixture, "select_with_variadic_typepack_tail") +TEST_CASE_FIXTURE(BuiltinsFixture, "select_with_variadic_typepack_tail") { CheckResult result = check(R"( --!nonstrict @@ -649,7 +656,7 @@ TEST_CASE_FIXTURE(Fixture, "select_with_variadic_typepack_tail") CHECK_EQ("any", toString(requireType("quux"))); } -TEST_CASE_FIXTURE(Fixture, "select_with_variadic_typepack_tail_and_string_head") +TEST_CASE_FIXTURE(BuiltinsFixture, "select_with_variadic_typepack_tail_and_string_head") { CheckResult result = check(R"( --!nonstrict @@ -703,7 +710,7 @@ TEST_CASE_FIXTURE(Fixture, "string_format_use_correct_argument2") CHECK_EQ("Type 'number' could not be converted into 'string'", toString(result.errors[1])); } -TEST_CASE_FIXTURE(Fixture, "debug_traceback_is_crazy") +TEST_CASE_FIXTURE(BuiltinsFixture, "debug_traceback_is_crazy") { CheckResult result = check(R"( local co: thread = ... @@ -720,7 +727,7 @@ debug.traceback(co, "msg", 1) LUAU_REQUIRE_NO_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "debug_info_is_crazy") +TEST_CASE_FIXTURE(BuiltinsFixture, "debug_info_is_crazy") { CheckResult result = check(R"( local co: thread, f: ()->() = ... @@ -734,7 +741,7 @@ debug.info(f, "n") LUAU_REQUIRE_NO_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "aliased_string_format") +TEST_CASE_FIXTURE(BuiltinsFixture, "aliased_string_format") { CheckResult result = check(R"( local fmt = string.format @@ -745,7 +752,7 @@ TEST_CASE_FIXTURE(Fixture, "aliased_string_format") CHECK_EQ("Type 'string' could not be converted into 'number'", toString(result.errors[0])); } -TEST_CASE_FIXTURE(Fixture, "string_lib_self_noself") +TEST_CASE_FIXTURE(BuiltinsFixture, "string_lib_self_noself") { CheckResult result = check(R"( --!nonstrict @@ -764,7 +771,7 @@ TEST_CASE_FIXTURE(Fixture, "string_lib_self_noself") LUAU_REQUIRE_NO_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "gmatch_definition") +TEST_CASE_FIXTURE(BuiltinsFixture, "gmatch_definition") { CheckResult result = check(R"_( local a, b, c = ("hey"):gmatch("(.)(.)(.)")() @@ -777,7 +784,7 @@ end LUAU_REQUIRE_NO_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "select_on_variadic") +TEST_CASE_FIXTURE(BuiltinsFixture, "select_on_variadic") { CheckResult result = check(R"( local function f(): (number, ...(boolean | number)) @@ -793,7 +800,7 @@ TEST_CASE_FIXTURE(Fixture, "select_on_variadic") CHECK_EQ("any", toString(requireType("c"))); } -TEST_CASE_FIXTURE(Fixture, "string_format_report_all_type_errors_at_correct_positions") +TEST_CASE_FIXTURE(BuiltinsFixture, "string_format_report_all_type_errors_at_correct_positions") { CheckResult result = check(R"( ("%s%d%s"):format(1, "hello", true) @@ -825,7 +832,7 @@ TEST_CASE_FIXTURE(Fixture, "string_format_report_all_type_errors_at_correct_posi CHECK_EQ(TypeErrorData(TypeMismatch{stringType, booleanType}), result.errors[5].data); } -TEST_CASE_FIXTURE(Fixture, "tonumber_returns_optional_number_type") +TEST_CASE_FIXTURE(BuiltinsFixture, "tonumber_returns_optional_number_type") { CheckResult result = check(R"( --!strict @@ -836,7 +843,7 @@ TEST_CASE_FIXTURE(Fixture, "tonumber_returns_optional_number_type") CHECK_EQ("Type 'number?' could not be converted into 'number'", toString(result.errors[0])); } -TEST_CASE_FIXTURE(Fixture, "tonumber_returns_optional_number_type2") +TEST_CASE_FIXTURE(BuiltinsFixture, "tonumber_returns_optional_number_type2") { CheckResult result = check(R"( --!strict @@ -846,7 +853,7 @@ TEST_CASE_FIXTURE(Fixture, "tonumber_returns_optional_number_type2") LUAU_REQUIRE_NO_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "dont_add_definitions_to_persistent_types") +TEST_CASE_FIXTURE(BuiltinsFixture, "dont_add_definitions_to_persistent_types") { CheckResult result = check(R"( local f = math.sin @@ -868,7 +875,7 @@ TEST_CASE_FIXTURE(Fixture, "dont_add_definitions_to_persistent_types") REQUIRE(gtv->definition); } -TEST_CASE_FIXTURE(Fixture, "assert_removes_falsy_types") +TEST_CASE_FIXTURE(BuiltinsFixture, "assert_removes_falsy_types") { ScopedFastFlag sff[]{ {"LuauAssertStripsFalsyTypes", true}, @@ -889,7 +896,7 @@ TEST_CASE_FIXTURE(Fixture, "assert_removes_falsy_types") CHECK_EQ("((boolean | number)?) -> boolean | number", toString(requireType("f"))); } -TEST_CASE_FIXTURE(Fixture, "assert_removes_falsy_types2") +TEST_CASE_FIXTURE(BuiltinsFixture, "assert_removes_falsy_types2") { ScopedFastFlag sff[]{ {"LuauAssertStripsFalsyTypes", true}, @@ -907,7 +914,7 @@ TEST_CASE_FIXTURE(Fixture, "assert_removes_falsy_types2") CHECK_EQ("((boolean | number)?) -> number | true", toString(requireType("f"))); } -TEST_CASE_FIXTURE(Fixture, "assert_removes_falsy_types_even_from_type_pack_tail_but_only_for_the_first_type") +TEST_CASE_FIXTURE(BuiltinsFixture, "assert_removes_falsy_types_even_from_type_pack_tail_but_only_for_the_first_type") { ScopedFastFlag sff[]{ {"LuauAssertStripsFalsyTypes", true}, @@ -924,7 +931,7 @@ TEST_CASE_FIXTURE(Fixture, "assert_removes_falsy_types_even_from_type_pack_tail_ CHECK_EQ("(...number?) -> (number, ...number?)", toString(requireType("f"))); } -TEST_CASE_FIXTURE(Fixture, "assert_returns_false_and_string_iff_it_knows_the_first_argument_cannot_be_truthy") +TEST_CASE_FIXTURE(BuiltinsFixture, "assert_returns_false_and_string_iff_it_knows_the_first_argument_cannot_be_truthy") { ScopedFastFlag sff[]{ {"LuauAssertStripsFalsyTypes", true}, @@ -941,7 +948,7 @@ TEST_CASE_FIXTURE(Fixture, "assert_returns_false_and_string_iff_it_knows_the_fir CHECK_EQ("(nil) -> nil", toString(requireType("f"))); } -TEST_CASE_FIXTURE(Fixture, "table_freeze_is_generic") +TEST_CASE_FIXTURE(BuiltinsFixture, "table_freeze_is_generic") { CheckResult result = check(R"( local t1: {a: number} = {a = 42} @@ -968,7 +975,7 @@ TEST_CASE_FIXTURE(Fixture, "table_freeze_is_generic") CHECK_EQ("*unknown*", toString(requireType("d"))); } -TEST_CASE_FIXTURE(Fixture, "set_metatable_needs_arguments") +TEST_CASE_FIXTURE(BuiltinsFixture, "set_metatable_needs_arguments") { ScopedFastFlag sff{"LuauSetMetaTableArgsCheck", true}; CheckResult result = check(R"( @@ -991,7 +998,7 @@ local function f(a: typeof(f)) end CHECK_EQ("Unknown global 'f'", toString(result.errors[0])); } -TEST_CASE_FIXTURE(Fixture, "no_persistent_typelevel_change") +TEST_CASE_FIXTURE(BuiltinsFixture, "no_persistent_typelevel_change") { TypeId mathTy = requireType(typeChecker.globalScope, "math"); REQUIRE(mathTy); @@ -1008,7 +1015,7 @@ TEST_CASE_FIXTURE(Fixture, "no_persistent_typelevel_change") CHECK(ftv->level.subLevel == original.subLevel); } -TEST_CASE_FIXTURE(Fixture, "global_singleton_types_are_sealed") +TEST_CASE_FIXTURE(BuiltinsFixture, "global_singleton_types_are_sealed") { CheckResult result = check(R"( local function f(x: string) diff --git a/tests/TypeInfer.classes.test.cpp b/tests/TypeInfer.classes.test.cpp index 5a6e4032..d90129d7 100644 --- a/tests/TypeInfer.classes.test.cpp +++ b/tests/TypeInfer.classes.test.cpp @@ -10,7 +10,7 @@ using namespace Luau; using std::nullopt; -struct ClassFixture : Fixture +struct ClassFixture : BuiltinsFixture { ClassFixture() { diff --git a/tests/TypeInfer.functions.test.cpp b/tests/TypeInfer.functions.test.cpp index 0e071217..14f1f703 100644 --- a/tests/TypeInfer.functions.test.cpp +++ b/tests/TypeInfer.functions.test.cpp @@ -85,7 +85,7 @@ TEST_CASE_FIXTURE(Fixture, "vararg_functions_should_allow_calls_of_any_types_and LUAU_REQUIRE_NO_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "vararg_function_is_quantified") +TEST_CASE_FIXTURE(BuiltinsFixture, "vararg_function_is_quantified") { CheckResult result = check(R"( local T = {} @@ -555,7 +555,7 @@ TEST_CASE_FIXTURE(Fixture, "higher_order_function_3") CHECK(bool(argType->indexer)); } -TEST_CASE_FIXTURE(Fixture, "higher_order_function_4") +TEST_CASE_FIXTURE(BuiltinsFixture, "higher_order_function_4") { CheckResult result = check(R"( function bottomupmerge(comp, a, b, left, mid, right) @@ -620,7 +620,7 @@ TEST_CASE_FIXTURE(Fixture, "higher_order_function_4") CHECK_EQ(*arg0->indexer->indexResultType, *arg1Args[1]); } -TEST_CASE_FIXTURE(Fixture, "mutual_recursion") +TEST_CASE_FIXTURE(BuiltinsFixture, "mutual_recursion") { CheckResult result = check(R"( --!strict @@ -639,7 +639,7 @@ TEST_CASE_FIXTURE(Fixture, "mutual_recursion") dumpErrors(result); } -TEST_CASE_FIXTURE(Fixture, "toposort_doesnt_break_mutual_recursion") +TEST_CASE_FIXTURE(BuiltinsFixture, "toposort_doesnt_break_mutual_recursion") { CheckResult result = check(R"( --!strict @@ -676,7 +676,7 @@ TEST_CASE_FIXTURE(Fixture, "check_function_before_lambda_that_uses_it") LUAU_REQUIRE_NO_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "it_is_ok_to_oversaturate_a_higher_order_function_argument") +TEST_CASE_FIXTURE(BuiltinsFixture, "it_is_ok_to_oversaturate_a_higher_order_function_argument") { CheckResult result = check(R"( function onerror() end @@ -794,7 +794,7 @@ TEST_CASE_FIXTURE(Fixture, "calling_function_with_incorrect_argument_type_yields }})); } -TEST_CASE_FIXTURE(Fixture, "calling_function_with_anytypepack_doesnt_leak_free_types") +TEST_CASE_FIXTURE(BuiltinsFixture, "calling_function_with_anytypepack_doesnt_leak_free_types") { ScopedFastFlag sff[]{ {"LuauReturnTypeInferenceInNonstrict", true}, @@ -966,7 +966,7 @@ TEST_CASE_FIXTURE(Fixture, "return_type_by_overload") CHECK_EQ("string", toString(requireType("z"))); } -TEST_CASE_FIXTURE(Fixture, "infer_anonymous_function_arguments") +TEST_CASE_FIXTURE(BuiltinsFixture, "infer_anonymous_function_arguments") { // Simple direct arg to arg propagation CheckResult result = check(R"( @@ -1068,7 +1068,7 @@ f(function(x) return x * 2 end) } } -TEST_CASE_FIXTURE(Fixture, "infer_anonymous_function_arguments") +TEST_CASE_FIXTURE(BuiltinsFixture, "infer_anonymous_function_arguments") { // Simple direct arg to arg propagation CheckResult result = check(R"( @@ -1287,10 +1287,8 @@ caused by: Return #2 type is not compatible. Type 'string' could not be converted into 'boolean')"); } -TEST_CASE_FIXTURE(Fixture, "function_decl_quantify_right_type") +TEST_CASE_FIXTURE(BuiltinsFixture, "function_decl_quantify_right_type") { - ScopedFastFlag statFunctionSimplify{"LuauStatFunctionSimplify4", true}; - fileResolver.source["game/isAMagicMock"] = R"( --!nonstrict return function(value) @@ -1311,10 +1309,8 @@ end LUAU_REQUIRE_NO_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "function_decl_non_self_sealed_overwrite") +TEST_CASE_FIXTURE(BuiltinsFixture, "function_decl_non_self_sealed_overwrite") { - ScopedFastFlag statFunctionSimplify{"LuauStatFunctionSimplify4", true}; - CheckResult result = check(R"( function string.len(): number return 1 @@ -1333,11 +1329,8 @@ print(string.len('hello')) LUAU_REQUIRE_NO_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "function_decl_non_self_sealed_overwrite_2") +TEST_CASE_FIXTURE(BuiltinsFixture, "function_decl_non_self_sealed_overwrite_2") { - ScopedFastFlag statFunctionSimplify{"LuauStatFunctionSimplify4", true}; - ScopedFastFlag inferStatFunction{"LuauInferStatFunction", true}; - CheckResult result = check(R"( local t: { f: ((x: number) -> number)? } = {} @@ -1477,11 +1470,8 @@ TEST_CASE_FIXTURE(Fixture, "inferred_higher_order_functions_are_quantified_at_th LUAU_REQUIRE_NO_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "function_decl_non_self_unsealed_overwrite") +TEST_CASE_FIXTURE(BuiltinsFixture, "function_decl_non_self_unsealed_overwrite") { - ScopedFastFlag statFunctionSimplify{"LuauStatFunctionSimplify4", true}; - ScopedFastFlag inferStatFunction{"LuauInferStatFunction", true}; - CheckResult result = check(R"( local t = { f = nil :: ((x: number) -> number)? } @@ -1518,8 +1508,6 @@ TEST_CASE_FIXTURE(Fixture, "strict_mode_ok_with_missing_arguments") TEST_CASE_FIXTURE(Fixture, "function_statement_sealed_table_assignment_through_indexer") { - ScopedFastFlag statFunctionSimplify{"LuauStatFunctionSimplify4", true}; - CheckResult result = check(R"( local t: {[string]: () -> number} = {} @@ -1580,7 +1568,7 @@ wrapper(test) CHECK(acm->isVariadic); } -TEST_CASE_FIXTURE(Fixture, "too_few_arguments_variadic_generic2") +TEST_CASE_FIXTURE(BuiltinsFixture, "too_few_arguments_variadic_generic2") { CheckResult result = check(R"( function test(a: number, b: string, ...) diff --git a/tests/TypeInfer.generics.test.cpp b/tests/TypeInfer.generics.test.cpp index 91be2c1c..de0c9391 100644 --- a/tests/TypeInfer.generics.test.cpp +++ b/tests/TypeInfer.generics.test.cpp @@ -67,7 +67,7 @@ TEST_CASE_FIXTURE(Fixture, "local_vars_can_be_polytypes") LUAU_REQUIRE_NO_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "inferred_local_vars_can_be_polytypes") +TEST_CASE_FIXTURE(BuiltinsFixture, "inferred_local_vars_can_be_polytypes") { CheckResult result = check(R"( local function id(x) return x end @@ -79,7 +79,7 @@ TEST_CASE_FIXTURE(Fixture, "inferred_local_vars_can_be_polytypes") LUAU_REQUIRE_NO_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "local_vars_can_be_instantiated_polytypes") +TEST_CASE_FIXTURE(BuiltinsFixture, "local_vars_can_be_instantiated_polytypes") { CheckResult result = check(R"( local function id(x) return x end @@ -609,7 +609,7 @@ TEST_CASE_FIXTURE(Fixture, "typefuns_sharing_types") CHECK(requireType("y1") == requireType("y2")); } -TEST_CASE_FIXTURE(Fixture, "bound_tables_do_not_clone_original_fields") +TEST_CASE_FIXTURE(BuiltinsFixture, "bound_tables_do_not_clone_original_fields") { CheckResult result = check(R"( local exports = {} @@ -675,7 +675,7 @@ local d: D = c R"(Type '() -> ()' could not be converted into '() -> ()'; different number of generic type pack parameters)"); } -TEST_CASE_FIXTURE(Fixture, "generic_functions_dont_cache_type_parameters") +TEST_CASE_FIXTURE(BuiltinsFixture, "generic_functions_dont_cache_type_parameters") { CheckResult result = check(R"( -- See https://github.com/Roblox/luau/issues/332 @@ -1013,7 +1013,7 @@ TEST_CASE_FIXTURE(Fixture, "no_stack_overflow_from_quantifying") CHECK(it != result.errors.end()); } -TEST_CASE_FIXTURE(Fixture, "infer_generic_function_function_argument") +TEST_CASE_FIXTURE(BuiltinsFixture, "infer_generic_function_function_argument") { ScopedFastFlag sff{"LuauUnsealedTableLiteral", true}; @@ -1078,7 +1078,7 @@ TEST_CASE_FIXTURE(Fixture, "infer_generic_function_function_argument_overloaded" LUAU_REQUIRE_NO_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "infer_generic_lib_function_function_argument") +TEST_CASE_FIXTURE(BuiltinsFixture, "infer_generic_lib_function_function_argument") { CheckResult result = check(R"( local a = {{x=4}, {x=7}, {x=1}} diff --git a/tests/TypeInfer.intersectionTypes.test.cpp b/tests/TypeInfer.intersectionTypes.test.cpp index 3675919f..41bc0c21 100644 --- a/tests/TypeInfer.intersectionTypes.test.cpp +++ b/tests/TypeInfer.intersectionTypes.test.cpp @@ -316,8 +316,6 @@ TEST_CASE_FIXTURE(Fixture, "table_intersection_write_sealed") TEST_CASE_FIXTURE(Fixture, "table_intersection_write_sealed_indirect") { - ScopedFastFlag statFunctionSimplify{"LuauStatFunctionSimplify4", true}; - CheckResult result = check(R"( type X = { x: (number) -> number } type Y = { y: (string) -> string } @@ -351,8 +349,6 @@ caused by: TEST_CASE_FIXTURE(Fixture, "table_write_sealed_indirect") { - ScopedFastFlag statFunctionSimplify{"LuauStatFunctionSimplify4", true}; - // After normalization, previous 'table_intersection_write_sealed_indirect' is identical to this one CheckResult result = check(R"( type XY = { x: (number) -> number, y: (string) -> string } @@ -375,7 +371,7 @@ caused by: CHECK_EQ(toString(result.errors[3]), "Cannot add property 'w' to table 'XY'"); } -TEST_CASE_FIXTURE(Fixture, "table_intersection_setmetatable") +TEST_CASE_FIXTURE(BuiltinsFixture, "table_intersection_setmetatable") { CheckResult result = check(R"( local t: {} & {} diff --git a/tests/TypeInfer.loops.test.cpp b/tests/TypeInfer.loops.test.cpp index f9b510c1..765419c6 100644 --- a/tests/TypeInfer.loops.test.cpp +++ b/tests/TypeInfer.loops.test.cpp @@ -29,7 +29,7 @@ TEST_CASE_FIXTURE(Fixture, "for_loop") CHECK_EQ(*typeChecker.numberType, *requireType("q")); } -TEST_CASE_FIXTURE(Fixture, "for_in_loop") +TEST_CASE_FIXTURE(BuiltinsFixture, "for_in_loop") { CheckResult result = check(R"( local n @@ -46,7 +46,7 @@ TEST_CASE_FIXTURE(Fixture, "for_in_loop") CHECK_EQ(*typeChecker.stringType, *requireType("s")); } -TEST_CASE_FIXTURE(Fixture, "for_in_loop_with_next") +TEST_CASE_FIXTURE(BuiltinsFixture, "for_in_loop_with_next") { CheckResult result = check(R"( local n @@ -90,7 +90,7 @@ TEST_CASE_FIXTURE(Fixture, "for_in_loop_should_fail_with_non_function_iterator") CHECK_EQ("Cannot call non-function string", toString(result.errors[0])); } -TEST_CASE_FIXTURE(Fixture, "for_in_with_just_one_iterator_is_ok") +TEST_CASE_FIXTURE(BuiltinsFixture, "for_in_with_just_one_iterator_is_ok") { CheckResult result = check(R"( local function keys(dictionary) @@ -109,7 +109,7 @@ TEST_CASE_FIXTURE(Fixture, "for_in_with_just_one_iterator_is_ok") LUAU_REQUIRE_NO_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "for_in_with_a_custom_iterator_should_type_check") +TEST_CASE_FIXTURE(BuiltinsFixture, "for_in_with_a_custom_iterator_should_type_check") { CheckResult result = check(R"( local function range(l, h): () -> number @@ -161,7 +161,7 @@ TEST_CASE_FIXTURE(Fixture, "for_in_loop_on_non_function") REQUIRE(get(result.errors[0])); } -TEST_CASE_FIXTURE(Fixture, "for_in_loop_error_on_factory_not_returning_the_right_amount_of_values") +TEST_CASE_FIXTURE(BuiltinsFixture, "for_in_loop_error_on_factory_not_returning_the_right_amount_of_values") { CheckResult result = check(R"( local function hasDivisors(value: number, table) @@ -210,7 +210,7 @@ TEST_CASE_FIXTURE(Fixture, "for_in_loop_error_on_factory_not_returning_the_right CHECK_EQ(typeChecker.stringType, tm->givenType); } -TEST_CASE_FIXTURE(Fixture, "for_in_loop_error_on_iterator_requiring_args_but_none_given") +TEST_CASE_FIXTURE(BuiltinsFixture, "for_in_loop_error_on_iterator_requiring_args_but_none_given") { CheckResult result = check(R"( function prime_iter(state, index) @@ -288,7 +288,7 @@ TEST_CASE_FIXTURE(Fixture, "repeat_loop_condition_binds_to_its_block") LUAU_REQUIRE_NO_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "symbols_in_repeat_block_should_not_be_visible_beyond_until_condition") +TEST_CASE_FIXTURE(BuiltinsFixture, "symbols_in_repeat_block_should_not_be_visible_beyond_until_condition") { CheckResult result = check(R"( repeat @@ -301,7 +301,7 @@ TEST_CASE_FIXTURE(Fixture, "symbols_in_repeat_block_should_not_be_visible_beyond LUAU_REQUIRE_ERROR_COUNT(1, result); } -TEST_CASE_FIXTURE(Fixture, "varlist_declared_by_for_in_loop_should_be_free") +TEST_CASE_FIXTURE(BuiltinsFixture, "varlist_declared_by_for_in_loop_should_be_free") { CheckResult result = check(R"( local T = {} @@ -316,7 +316,7 @@ TEST_CASE_FIXTURE(Fixture, "varlist_declared_by_for_in_loop_should_be_free") LUAU_REQUIRE_NO_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "properly_infer_iteratee_is_a_free_table") +TEST_CASE_FIXTURE(BuiltinsFixture, "properly_infer_iteratee_is_a_free_table") { // In this case, we cannot know the element type of the table {}. It could be anything. // We therefore must initially ascribe a free typevar to iter. @@ -329,7 +329,7 @@ TEST_CASE_FIXTURE(Fixture, "properly_infer_iteratee_is_a_free_table") LUAU_REQUIRE_NO_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "correctly_scope_locals_while") +TEST_CASE_FIXTURE(BuiltinsFixture, "correctly_scope_locals_while") { CheckResult result = check(R"( while true do @@ -346,7 +346,7 @@ TEST_CASE_FIXTURE(Fixture, "correctly_scope_locals_while") CHECK_EQ(us->name, "a"); } -TEST_CASE_FIXTURE(Fixture, "ipairs_produces_integral_indices") +TEST_CASE_FIXTURE(BuiltinsFixture, "ipairs_produces_integral_indices") { CheckResult result = check(R"( local key @@ -378,7 +378,7 @@ TEST_CASE_FIXTURE(Fixture, "for_in_loop_where_iteratee_is_free") )"); } -TEST_CASE_FIXTURE(Fixture, "unreachable_code_after_infinite_loop") +TEST_CASE_FIXTURE(BuiltinsFixture, "unreachable_code_after_infinite_loop") { { CheckResult result = check(R"( @@ -460,7 +460,7 @@ TEST_CASE_FIXTURE(Fixture, "unreachable_code_after_infinite_loop") } } -TEST_CASE_FIXTURE(Fixture, "loop_typecheck_crash_on_empty_optional") +TEST_CASE_FIXTURE(BuiltinsFixture, "loop_typecheck_crash_on_empty_optional") { CheckResult result = check(R"( local t = {} @@ -541,7 +541,7 @@ TEST_CASE_FIXTURE(Fixture, "loop_iter_no_indexer") CHECK_EQ("Cannot iterate over a table without indexer", ge->message); } -TEST_CASE_FIXTURE(Fixture, "loop_iter_iter_metamethod") +TEST_CASE_FIXTURE(BuiltinsFixture, "loop_iter_iter_metamethod") { ScopedFastFlag sff{"LuauTypecheckIter", true}; diff --git a/tests/TypeInfer.modules.test.cpp b/tests/TypeInfer.modules.test.cpp index b6f49f9f..efa2a98d 100644 --- a/tests/TypeInfer.modules.test.cpp +++ b/tests/TypeInfer.modules.test.cpp @@ -16,7 +16,7 @@ LUAU_FASTFLAG(LuauTableSubtypingVariance2) TEST_SUITE_BEGIN("TypeInferModules"); -TEST_CASE_FIXTURE(Fixture, "require") +TEST_CASE_FIXTURE(BuiltinsFixture, "require") { fileResolver.source["game/A"] = R"( local function hooty(x: number): string @@ -54,7 +54,7 @@ TEST_CASE_FIXTURE(Fixture, "require") REQUIRE_EQ("number", toString(*hType)); } -TEST_CASE_FIXTURE(Fixture, "require_types") +TEST_CASE_FIXTURE(BuiltinsFixture, "require_types") { fileResolver.source["workspace/A"] = R"( export type Point = {x: number, y: number} @@ -69,7 +69,7 @@ TEST_CASE_FIXTURE(Fixture, "require_types") )"; CheckResult bResult = frontend.check("workspace/B"); - dumpErrors(bResult); + LUAU_REQUIRE_NO_ERRORS(bResult); ModulePtr b = frontend.moduleResolver.modules["workspace/B"]; REQUIRE(b != nullptr); @@ -78,7 +78,7 @@ TEST_CASE_FIXTURE(Fixture, "require_types") REQUIRE_MESSAGE(bool(get(hType)), "Expected table but got " << toString(hType)); } -TEST_CASE_FIXTURE(Fixture, "require_a_variadic_function") +TEST_CASE_FIXTURE(BuiltinsFixture, "require_a_variadic_function") { fileResolver.source["game/A"] = R"( local T = {} @@ -121,7 +121,7 @@ TEST_CASE_FIXTURE(Fixture, "type_error_of_unknown_qualified_type") REQUIRE_EQ(result.errors[0], (TypeError{Location{{1, 17}, {1, 40}}, UnknownSymbol{"SomeModule.DoesNotExist"}})); } -TEST_CASE_FIXTURE(Fixture, "require_module_that_does_not_export") +TEST_CASE_FIXTURE(BuiltinsFixture, "require_module_that_does_not_export") { const std::string sourceA = R"( )"; @@ -148,7 +148,7 @@ TEST_CASE_FIXTURE(Fixture, "require_module_that_does_not_export") CHECK_EQ("*unknown*", toString(hootyType)); } -TEST_CASE_FIXTURE(Fixture, "warn_if_you_try_to_require_a_non_modulescript") +TEST_CASE_FIXTURE(BuiltinsFixture, "warn_if_you_try_to_require_a_non_modulescript") { fileResolver.source["Modules/A"] = ""; fileResolver.sourceTypes["Modules/A"] = SourceCode::Local; @@ -164,7 +164,7 @@ TEST_CASE_FIXTURE(Fixture, "warn_if_you_try_to_require_a_non_modulescript") CHECK(get(result.errors[0])); } -TEST_CASE_FIXTURE(Fixture, "general_require_call_expression") +TEST_CASE_FIXTURE(BuiltinsFixture, "general_require_call_expression") { fileResolver.source["game/A"] = R"( --!strict @@ -183,7 +183,7 @@ a = tbl.abc.def CHECK_EQ("Type 'number' could not be converted into 'string'", toString(result.errors[0])); } -TEST_CASE_FIXTURE(Fixture, "general_require_type_mismatch") +TEST_CASE_FIXTURE(BuiltinsFixture, "general_require_type_mismatch") { fileResolver.source["game/A"] = R"( return { def = 4 } @@ -219,7 +219,7 @@ return m LUAU_REQUIRE_NO_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "custom_require_global") +TEST_CASE_FIXTURE(BuiltinsFixture, "custom_require_global") { CheckResult result = check(R"( --!nonstrict @@ -231,7 +231,7 @@ local crash = require(game.A) LUAU_REQUIRE_NO_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "require_failed_module") +TEST_CASE_FIXTURE(BuiltinsFixture, "require_failed_module") { fileResolver.source["game/A"] = R"( return unfortunately() @@ -267,7 +267,7 @@ function x:Destroy(): () end LUAU_REQUIRE_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "do_not_modify_imported_types_2") +TEST_CASE_FIXTURE(BuiltinsFixture, "do_not_modify_imported_types_2") { fileResolver.source["game/A"] = R"( export type Type = { x: { a: number } } @@ -285,7 +285,7 @@ type Rename = typeof(x.x) LUAU_REQUIRE_NO_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "do_not_modify_imported_types_3") +TEST_CASE_FIXTURE(BuiltinsFixture, "do_not_modify_imported_types_3") { fileResolver.source["game/A"] = R"( local y = setmetatable({}, {}) @@ -304,7 +304,7 @@ type Rename = typeof(x.x) LUAU_REQUIRE_NO_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "module_type_conflict") +TEST_CASE_FIXTURE(BuiltinsFixture, "module_type_conflict") { fileResolver.source["game/A"] = R"( export type T = { x: number } @@ -338,7 +338,7 @@ caused by: } } -TEST_CASE_FIXTURE(Fixture, "module_type_conflict_instantiated") +TEST_CASE_FIXTURE(BuiltinsFixture, "module_type_conflict_instantiated") { fileResolver.source["game/A"] = R"( export type Wrap = { x: T } diff --git a/tests/TypeInfer.oop.test.cpp b/tests/TypeInfer.oop.test.cpp index 5cd3f3ba..41690704 100644 --- a/tests/TypeInfer.oop.test.cpp +++ b/tests/TypeInfer.oop.test.cpp @@ -142,7 +142,7 @@ TEST_CASE_FIXTURE(Fixture, "inferring_hundreds_of_self_calls_should_not_suffocat CHECK_GE(50, module->internalTypes.typeVars.size()); } -TEST_CASE_FIXTURE(Fixture, "object_constructor_can_refer_to_method_of_self") +TEST_CASE_FIXTURE(BuiltinsFixture, "object_constructor_can_refer_to_method_of_self") { // CLI-30902 CheckResult result = check(R"( @@ -243,7 +243,7 @@ TEST_CASE_FIXTURE(Fixture, "inferred_methods_of_free_tables_have_the_same_level_ )"); } -TEST_CASE_FIXTURE(Fixture, "table_oop") +TEST_CASE_FIXTURE(BuiltinsFixture, "table_oop") { CheckResult result = check(R"( --!strict diff --git a/tests/TypeInfer.operators.test.cpp b/tests/TypeInfer.operators.test.cpp index a2787cad..51f6fdfb 100644 --- a/tests/TypeInfer.operators.test.cpp +++ b/tests/TypeInfer.operators.test.cpp @@ -77,7 +77,7 @@ TEST_CASE_FIXTURE(Fixture, "and_or_ternary") CHECK_EQ(toString(*requireType("s")), "number | string"); } -TEST_CASE_FIXTURE(Fixture, "primitive_arith_no_metatable") +TEST_CASE_FIXTURE(BuiltinsFixture, "primitive_arith_no_metatable") { CheckResult result = check(R"( function add(a: number, b: string) @@ -140,7 +140,7 @@ TEST_CASE_FIXTURE(Fixture, "some_primitive_binary_ops") CHECK_EQ("number", toString(requireType("c"))); } -TEST_CASE_FIXTURE(Fixture, "typecheck_overloaded_multiply_that_is_an_intersection") +TEST_CASE_FIXTURE(BuiltinsFixture, "typecheck_overloaded_multiply_that_is_an_intersection") { CheckResult result = check(R"( --!strict @@ -174,7 +174,7 @@ TEST_CASE_FIXTURE(Fixture, "typecheck_overloaded_multiply_that_is_an_intersectio CHECK_EQ("Vec3", toString(requireType("e"))); } -TEST_CASE_FIXTURE(Fixture, "typecheck_overloaded_multiply_that_is_an_intersection_on_rhs") +TEST_CASE_FIXTURE(BuiltinsFixture, "typecheck_overloaded_multiply_that_is_an_intersection_on_rhs") { CheckResult result = check(R"( --!strict @@ -245,7 +245,7 @@ TEST_CASE_FIXTURE(Fixture, "cannot_indirectly_compare_types_that_do_not_have_a_m REQUIRE_EQ(gen->message, "Type a cannot be compared with < because it has no metatable"); } -TEST_CASE_FIXTURE(Fixture, "cannot_indirectly_compare_types_that_do_not_offer_overloaded_ordering_operators") +TEST_CASE_FIXTURE(BuiltinsFixture, "cannot_indirectly_compare_types_that_do_not_offer_overloaded_ordering_operators") { CheckResult result = check(R"( local M = {} @@ -266,7 +266,7 @@ TEST_CASE_FIXTURE(Fixture, "cannot_indirectly_compare_types_that_do_not_offer_ov REQUIRE_EQ(gen->message, "Table M does not offer metamethod __lt"); } -TEST_CASE_FIXTURE(Fixture, "cannot_compare_tables_that_do_not_have_the_same_metatable") +TEST_CASE_FIXTURE(BuiltinsFixture, "cannot_compare_tables_that_do_not_have_the_same_metatable") { CheckResult result = check(R"( --!strict @@ -289,7 +289,7 @@ TEST_CASE_FIXTURE(Fixture, "cannot_compare_tables_that_do_not_have_the_same_meta REQUIRE_EQ((Location{{11, 18}, {11, 23}}), result.errors[1].location); } -TEST_CASE_FIXTURE(Fixture, "produce_the_correct_error_message_when_comparing_a_table_with_a_metatable_with_one_that_does_not") +TEST_CASE_FIXTURE(BuiltinsFixture, "produce_the_correct_error_message_when_comparing_a_table_with_a_metatable_with_one_that_does_not") { CheckResult result = check(R"( --!strict @@ -361,7 +361,7 @@ TEST_CASE_FIXTURE(Fixture, "compound_assign_mismatch_result") CHECK_EQ(result.errors[1], (TypeError{Location{{2, 8}, {2, 15}}, TypeMismatch{typeChecker.stringType, typeChecker.numberType}})); } -TEST_CASE_FIXTURE(Fixture, "compound_assign_metatable") +TEST_CASE_FIXTURE(BuiltinsFixture, "compound_assign_metatable") { CheckResult result = check(R"( --!strict @@ -381,7 +381,7 @@ TEST_CASE_FIXTURE(Fixture, "compound_assign_metatable") CHECK_EQ(0, result.errors.size()); } -TEST_CASE_FIXTURE(Fixture, "compound_assign_mismatch_metatable") +TEST_CASE_FIXTURE(BuiltinsFixture, "compound_assign_mismatch_metatable") { CheckResult result = check(R"( --!strict @@ -428,7 +428,7 @@ local x = false LUAU_REQUIRE_NO_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "typecheck_unary_minus") +TEST_CASE_FIXTURE(BuiltinsFixture, "typecheck_unary_minus") { CheckResult result = check(R"( --!strict @@ -461,7 +461,7 @@ TEST_CASE_FIXTURE(Fixture, "typecheck_unary_minus") REQUIRE_EQ(gen->message, "Unary operator '-' not supported by type 'bar'"); } -TEST_CASE_FIXTURE(Fixture, "unary_not_is_boolean") +TEST_CASE_FIXTURE(BuiltinsFixture, "unary_not_is_boolean") { CheckResult result = check(R"( local b = not "string" @@ -473,7 +473,7 @@ TEST_CASE_FIXTURE(Fixture, "unary_not_is_boolean") REQUIRE_EQ("boolean", toString(requireType("c"))); } -TEST_CASE_FIXTURE(Fixture, "disallow_string_and_types_without_metatables_from_arithmetic_binary_ops") +TEST_CASE_FIXTURE(BuiltinsFixture, "disallow_string_and_types_without_metatables_from_arithmetic_binary_ops") { CheckResult result = check(R"( --!strict @@ -573,7 +573,7 @@ TEST_CASE_FIXTURE(Fixture, "strict_binary_op_where_lhs_unknown") CHECK_EQ("Unknown type used in + operation; consider adding a type annotation to 'a'", toString(result.errors[0])); } -TEST_CASE_FIXTURE(Fixture, "and_binexps_dont_unify") +TEST_CASE_FIXTURE(BuiltinsFixture, "and_binexps_dont_unify") { CheckResult result = check(R"( --!strict @@ -628,7 +628,7 @@ TEST_CASE_FIXTURE(Fixture, "cli_38355_recursive_union") CHECK_EQ("Type contains a self-recursive construct that cannot be resolved", toString(result.errors[0])); } -TEST_CASE_FIXTURE(Fixture, "UnknownGlobalCompoundAssign") +TEST_CASE_FIXTURE(BuiltinsFixture, "UnknownGlobalCompoundAssign") { // In non-strict mode, global definition is still allowed { @@ -755,8 +755,6 @@ TEST_CASE_FIXTURE(Fixture, "refine_and_or") TEST_CASE_FIXTURE(Fixture, "infer_any_in_all_modes_when_lhs_is_unknown") { - ScopedFastFlag sff{"LuauDecoupleOperatorInferenceFromUnifiedTypeInference", true}; - CheckResult result = check(Mode::Strict, R"( local function f(x, y) return x + y @@ -779,4 +777,47 @@ TEST_CASE_FIXTURE(Fixture, "infer_any_in_all_modes_when_lhs_is_unknown") // the case right now, though. } +TEST_CASE_FIXTURE(BuiltinsFixture, "equality_operations_succeed_if_any_union_branch_succeeds") +{ + ScopedFastFlag sff("LuauSuccessTypingForEqualityOperations", true); + + CheckResult result = check(R"( + local mm = {} + type Foo = typeof(setmetatable({}, mm)) + local x: Foo + local y: Foo? + + local v1 = x == y + local v2 = y == x + local v3 = x ~= y + local v4 = y ~= x + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CheckResult result2 = check(R"( + local mm1 = { + x = "foo", + } + + local mm2 = { + y = "bar", + } + + type Foo = typeof(setmetatable({}, mm1)) + type Bar = typeof(setmetatable({}, mm2)) + + local x1: Foo + local x2: Foo? + local y1: Bar + local y2: Bar? + + local v1 = x1 == y1 + local v2 = x2 == y2 + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result2); + CHECK(toString(result2.errors[0]) == "Types Foo and Bar cannot be compared with == because they do not have the same metatable"); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.provisional.test.cpp b/tests/TypeInfer.provisional.test.cpp index 2ef77419..ee3ae972 100644 --- a/tests/TypeInfer.provisional.test.cpp +++ b/tests/TypeInfer.provisional.test.cpp @@ -53,7 +53,7 @@ TEST_CASE_FIXTURE(Fixture, "typeguard_inference_incomplete") CHECK_EQ(expected, decorateWithTypes(code)); } -TEST_CASE_FIXTURE(Fixture, "xpcall_returns_what_f_returns") +TEST_CASE_FIXTURE(BuiltinsFixture, "xpcall_returns_what_f_returns") { const std::string code = R"( local a, b, c = xpcall(function() return 1, "foo" end, function() return "foo", 1 end) @@ -105,7 +105,7 @@ TEST_CASE_FIXTURE(Fixture, "it_should_be_agnostic_of_actual_size") // Ideally setmetatable's second argument would be an optional free table. // For now, infer it as just a free table. -TEST_CASE_FIXTURE(Fixture, "setmetatable_constrains_free_type_into_free_table") +TEST_CASE_FIXTURE(BuiltinsFixture, "setmetatable_constrains_free_type_into_free_table") { CheckResult result = check(R"( local a = {} @@ -146,7 +146,7 @@ TEST_CASE_FIXTURE(Fixture, "while_body_are_also_refined") // Originally from TypeInfer.test.cpp. // I dont think type checking the metamethod at every site of == is the correct thing to do. // We should be type checking the metamethod at the call site of setmetatable. -TEST_CASE_FIXTURE(Fixture, "error_on_eq_metamethod_returning_a_type_other_than_boolean") +TEST_CASE_FIXTURE(BuiltinsFixture, "error_on_eq_metamethod_returning_a_type_other_than_boolean") { CheckResult result = check(R"( local tab = {a = 1} @@ -428,7 +428,7 @@ TEST_CASE_FIXTURE(Fixture, "pcall_returns_at_least_two_value_but_function_return } // Belongs in TypeInfer.builtins.test.cpp. -TEST_CASE_FIXTURE(Fixture, "choose_the_right_overload_for_pcall") +TEST_CASE_FIXTURE(BuiltinsFixture, "choose_the_right_overload_for_pcall") { CheckResult result = check(R"( local function f(): number @@ -449,7 +449,7 @@ TEST_CASE_FIXTURE(Fixture, "choose_the_right_overload_for_pcall") } // Belongs in TypeInfer.builtins.test.cpp. -TEST_CASE_FIXTURE(Fixture, "function_returns_many_things_but_first_of_it_is_forgotten") +TEST_CASE_FIXTURE(BuiltinsFixture, "function_returns_many_things_but_first_of_it_is_forgotten") { CheckResult result = check(R"( local function f(): (number, string, boolean) diff --git a/tests/TypeInfer.refinements.test.cpp b/tests/TypeInfer.refinements.test.cpp index 136ca00a..8c130490 100644 --- a/tests/TypeInfer.refinements.test.cpp +++ b/tests/TypeInfer.refinements.test.cpp @@ -240,7 +240,7 @@ TEST_CASE_FIXTURE(Fixture, "typeguard_in_if_condition_position") CHECK_EQ("number", toString(requireTypeAtPosition({3, 26}))); } -TEST_CASE_FIXTURE(Fixture, "typeguard_in_assert_position") +TEST_CASE_FIXTURE(BuiltinsFixture, "typeguard_in_assert_position") { CheckResult result = check(R"( local a @@ -300,7 +300,7 @@ TEST_CASE_FIXTURE(Fixture, "call_a_more_specific_function_using_typeguard") CHECK_EQ("Type 'string' could not be converted into 'number'", toString(result.errors[0])); } -TEST_CASE_FIXTURE(Fixture, "impossible_type_narrow_is_not_an_error") +TEST_CASE_FIXTURE(BuiltinsFixture, "impossible_type_narrow_is_not_an_error") { // This unit test serves as a reminder to not implement this warning until Luau is intelligent enough. // For instance, getting a value out of the indexer and checking whether the value exists is not an error. @@ -333,7 +333,7 @@ TEST_CASE_FIXTURE(Fixture, "truthy_constraint_on_properties") CHECK_EQ("number?", toString(requireType("bar"))); } -TEST_CASE_FIXTURE(Fixture, "index_on_a_refined_property") +TEST_CASE_FIXTURE(BuiltinsFixture, "index_on_a_refined_property") { CheckResult result = check(R"( local t: {x: {y: string}?} = {x = {y = "hello!"}} @@ -346,7 +346,7 @@ TEST_CASE_FIXTURE(Fixture, "index_on_a_refined_property") LUAU_REQUIRE_NO_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "assert_non_binary_expressions_actually_resolve_constraints") +TEST_CASE_FIXTURE(BuiltinsFixture, "assert_non_binary_expressions_actually_resolve_constraints") { CheckResult result = check(R"( local foo: string? = "hello" @@ -730,7 +730,7 @@ TEST_CASE_FIXTURE(Fixture, "type_guard_can_filter_for_overloaded_function") CHECK_EQ("nil", toString(requireTypeAtPosition({6, 28}))); } -TEST_CASE_FIXTURE(Fixture, "type_guard_warns_on_no_overlapping_types_only_when_sense_is_true") +TEST_CASE_FIXTURE(BuiltinsFixture, "type_guard_warns_on_no_overlapping_types_only_when_sense_is_true") { CheckResult result = check(R"( local function f(t: {x: number}) @@ -846,7 +846,7 @@ TEST_CASE_FIXTURE(Fixture, "not_t_or_some_prop_of_t") CHECK_EQ("{| x: boolean |}?", toString(requireTypeAtPosition({3, 28}))); } -TEST_CASE_FIXTURE(Fixture, "assert_a_to_be_truthy_then_assert_a_to_be_number") +TEST_CASE_FIXTURE(BuiltinsFixture, "assert_a_to_be_truthy_then_assert_a_to_be_number") { CheckResult result = check(R"( local a: (number | string)? @@ -862,7 +862,7 @@ TEST_CASE_FIXTURE(Fixture, "assert_a_to_be_truthy_then_assert_a_to_be_number") CHECK_EQ("number", toString(requireTypeAtPosition({5, 18}))); } -TEST_CASE_FIXTURE(Fixture, "merge_should_be_fully_agnostic_of_hashmap_ordering") +TEST_CASE_FIXTURE(BuiltinsFixture, "merge_should_be_fully_agnostic_of_hashmap_ordering") { // This bug came up because there was a mistake in Luau::merge where zipping on two maps would produce the wrong merged result. CheckResult result = check(R"( @@ -899,7 +899,7 @@ TEST_CASE_FIXTURE(Fixture, "refine_the_correct_types_opposite_of_when_a_is_not_n CHECK_EQ("number | string", toString(requireTypeAtPosition({5, 28}))); } -TEST_CASE_FIXTURE(Fixture, "is_truthy_constraint_ifelse_expression") +TEST_CASE_FIXTURE(BuiltinsFixture, "is_truthy_constraint_ifelse_expression") { CheckResult result = check(R"( function f(v:string?) @@ -913,7 +913,7 @@ TEST_CASE_FIXTURE(Fixture, "is_truthy_constraint_ifelse_expression") CHECK_EQ("nil", toString(requireTypeAtPosition({2, 45}))); } -TEST_CASE_FIXTURE(Fixture, "invert_is_truthy_constraint_ifelse_expression") +TEST_CASE_FIXTURE(BuiltinsFixture, "invert_is_truthy_constraint_ifelse_expression") { CheckResult result = check(R"( function f(v:string?) @@ -945,7 +945,7 @@ TEST_CASE_FIXTURE(Fixture, "type_comparison_ifelse_expression") CHECK_EQ("any", toString(requireTypeAtPosition({6, 66}))); } -TEST_CASE_FIXTURE(Fixture, "correctly_lookup_a_shadowed_local_that_which_was_previously_refined") +TEST_CASE_FIXTURE(BuiltinsFixture, "correctly_lookup_a_shadowed_local_that_which_was_previously_refined") { CheckResult result = check(R"( local foo: string? = "hi" diff --git a/tests/TypeInfer.singletons.test.cpp b/tests/TypeInfer.singletons.test.cpp index 8d6682b8..79eeb824 100644 --- a/tests/TypeInfer.singletons.test.cpp +++ b/tests/TypeInfer.singletons.test.cpp @@ -415,7 +415,7 @@ TEST_CASE_FIXTURE(Fixture, "widening_happens_almost_everywhere_except_for_tables LUAU_REQUIRE_NO_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "table_insert_with_a_singleton_argument") +TEST_CASE_FIXTURE(BuiltinsFixture, "table_insert_with_a_singleton_argument") { ScopedFastFlag sff[]{ {"LuauWidenIfSupertypeIsFree2", true}, diff --git a/tests/TypeInfer.tables.test.cpp b/tests/TypeInfer.tables.test.cpp index 8e535995..5078b0bf 100644 --- a/tests/TypeInfer.tables.test.cpp +++ b/tests/TypeInfer.tables.test.cpp @@ -201,7 +201,7 @@ TEST_CASE_FIXTURE(Fixture, "used_dot_instead_of_colon") REQUIRE(it != result.errors.end()); } -TEST_CASE_FIXTURE(Fixture, "used_colon_correctly") +TEST_CASE_FIXTURE(BuiltinsFixture, "used_colon_correctly") { CheckResult result = check(R"( --!nonstrict @@ -883,7 +883,7 @@ TEST_CASE_FIXTURE(Fixture, "assigning_to_an_unsealed_table_with_string_literal_s CHECK_EQ(*typeChecker.stringType, *propertyA); } -TEST_CASE_FIXTURE(Fixture, "oop_indexer_works") +TEST_CASE_FIXTURE(BuiltinsFixture, "oop_indexer_works") { CheckResult result = check(R"( local clazz = {} @@ -906,7 +906,7 @@ TEST_CASE_FIXTURE(Fixture, "oop_indexer_works") CHECK_EQ(*typeChecker.stringType, *requireType("words")); } -TEST_CASE_FIXTURE(Fixture, "indexer_table") +TEST_CASE_FIXTURE(BuiltinsFixture, "indexer_table") { CheckResult result = check(R"( local clazz = {a="hello"} @@ -919,7 +919,7 @@ TEST_CASE_FIXTURE(Fixture, "indexer_table") CHECK_EQ(*typeChecker.stringType, *requireType("b")); } -TEST_CASE_FIXTURE(Fixture, "indexer_fn") +TEST_CASE_FIXTURE(BuiltinsFixture, "indexer_fn") { CheckResult result = check(R"( local instanace = setmetatable({}, {__index=function() return 10 end}) @@ -930,7 +930,7 @@ TEST_CASE_FIXTURE(Fixture, "indexer_fn") CHECK_EQ(*typeChecker.numberType, *requireType("b")); } -TEST_CASE_FIXTURE(Fixture, "meta_add") +TEST_CASE_FIXTURE(BuiltinsFixture, "meta_add") { // Note: meta_add_inferred and this unit test are currently the same exact thing. // We'll want to change this one in particular when we add real syntax for metatables. @@ -947,7 +947,7 @@ TEST_CASE_FIXTURE(Fixture, "meta_add") CHECK_EQ(follow(requireType("a")), follow(requireType("c"))); } -TEST_CASE_FIXTURE(Fixture, "meta_add_inferred") +TEST_CASE_FIXTURE(BuiltinsFixture, "meta_add_inferred") { CheckResult result = check(R"( local a = {} @@ -960,7 +960,7 @@ TEST_CASE_FIXTURE(Fixture, "meta_add_inferred") CHECK_EQ(*requireType("a"), *requireType("c")); } -TEST_CASE_FIXTURE(Fixture, "meta_add_both_ways") +TEST_CASE_FIXTURE(BuiltinsFixture, "meta_add_both_ways") { CheckResult result = check(R"( type VectorMt = { __add: (Vector, number) -> Vector } @@ -980,7 +980,7 @@ TEST_CASE_FIXTURE(Fixture, "meta_add_both_ways") // This test exposed a bug where we let go of the "seen" stack while unifying table types // As a result, type inference crashed with a stack overflow. -TEST_CASE_FIXTURE(Fixture, "unification_of_unions_in_a_self_referential_type") +TEST_CASE_FIXTURE(BuiltinsFixture, "unification_of_unions_in_a_self_referential_type") { CheckResult result = check(R"( type A = {} @@ -1009,7 +1009,7 @@ TEST_CASE_FIXTURE(Fixture, "unification_of_unions_in_a_self_referential_type") CHECK_EQ(bmtv->metatable, requireType("bmt")); } -TEST_CASE_FIXTURE(Fixture, "oop_polymorphic") +TEST_CASE_FIXTURE(BuiltinsFixture, "oop_polymorphic") { CheckResult result = check(R"( local animal = {} @@ -1060,7 +1060,7 @@ TEST_CASE_FIXTURE(Fixture, "user_defined_table_types_are_named") CHECK_EQ("Vector3", toString(requireType("v"))); } -TEST_CASE_FIXTURE(Fixture, "result_is_always_any_if_lhs_is_any") +TEST_CASE_FIXTURE(BuiltinsFixture, "result_is_always_any_if_lhs_is_any") { CheckResult result = check(R"( type Vector3MT = { @@ -1133,7 +1133,7 @@ TEST_CASE_FIXTURE(Fixture, "nice_error_when_trying_to_fetch_property_of_boolean" CHECK_EQ("Type 'boolean' does not have key 'some_prop'", toString(result.errors[0])); } -TEST_CASE_FIXTURE(Fixture, "defining_a_method_for_a_builtin_sealed_table_must_fail") +TEST_CASE_FIXTURE(BuiltinsFixture, "defining_a_method_for_a_builtin_sealed_table_must_fail") { CheckResult result = check(R"( function string.m() end @@ -1142,7 +1142,7 @@ TEST_CASE_FIXTURE(Fixture, "defining_a_method_for_a_builtin_sealed_table_must_fa LUAU_REQUIRE_ERROR_COUNT(1, result); } -TEST_CASE_FIXTURE(Fixture, "defining_a_self_method_for_a_builtin_sealed_table_must_fail") +TEST_CASE_FIXTURE(BuiltinsFixture, "defining_a_self_method_for_a_builtin_sealed_table_must_fail") { CheckResult result = check(R"( function string:m() end @@ -1261,7 +1261,7 @@ TEST_CASE_FIXTURE(Fixture, "found_like_key_in_table_function_call") CHECK_EQ(toString(te), "Key 'fOo' not found in table 't'. Did you mean 'Foo'?"); } -TEST_CASE_FIXTURE(Fixture, "found_like_key_in_table_property_access") +TEST_CASE_FIXTURE(BuiltinsFixture, "found_like_key_in_table_property_access") { CheckResult result = check(R"( local t = {X = 1} @@ -1286,7 +1286,7 @@ TEST_CASE_FIXTURE(Fixture, "found_like_key_in_table_property_access") CHECK_EQ(toString(te), "Key 'x' not found in table 't'. Did you mean 'X'?"); } -TEST_CASE_FIXTURE(Fixture, "found_multiple_like_keys") +TEST_CASE_FIXTURE(BuiltinsFixture, "found_multiple_like_keys") { CheckResult result = check(R"( local t = {Foo = 1, foO = 2} @@ -1312,7 +1312,7 @@ TEST_CASE_FIXTURE(Fixture, "found_multiple_like_keys") CHECK_EQ(toString(te), "Key 'foo' not found in table 't'. Did you mean one of 'Foo', 'foO'?"); } -TEST_CASE_FIXTURE(Fixture, "dont_suggest_exact_match_keys") +TEST_CASE_FIXTURE(BuiltinsFixture, "dont_suggest_exact_match_keys") { CheckResult result = check(R"( local t = {} @@ -1339,7 +1339,7 @@ TEST_CASE_FIXTURE(Fixture, "dont_suggest_exact_match_keys") CHECK_EQ(toString(te), "Key 'Foo' not found in table 't'. Did you mean 'foO'?"); } -TEST_CASE_FIXTURE(Fixture, "getmetatable_returns_pointer_to_metatable") +TEST_CASE_FIXTURE(BuiltinsFixture, "getmetatable_returns_pointer_to_metatable") { CheckResult result = check(R"( local t = {x = 1} @@ -1352,7 +1352,7 @@ TEST_CASE_FIXTURE(Fixture, "getmetatable_returns_pointer_to_metatable") CHECK_EQ(*requireType("mt"), *requireType("returnedMT")); } -TEST_CASE_FIXTURE(Fixture, "metatable_mismatch_should_fail") +TEST_CASE_FIXTURE(BuiltinsFixture, "metatable_mismatch_should_fail") { CheckResult result = check(R"( local t1 = {x = 1} @@ -1374,7 +1374,7 @@ TEST_CASE_FIXTURE(Fixture, "metatable_mismatch_should_fail") CHECK_EQ(*tm->givenType, *requireType("t2")); } -TEST_CASE_FIXTURE(Fixture, "property_lookup_through_tabletypevar_metatable") +TEST_CASE_FIXTURE(BuiltinsFixture, "property_lookup_through_tabletypevar_metatable") { CheckResult result = check(R"( local t = {x = 1} @@ -1393,7 +1393,7 @@ TEST_CASE_FIXTURE(Fixture, "property_lookup_through_tabletypevar_metatable") CHECK_EQ(up->key, "z"); } -TEST_CASE_FIXTURE(Fixture, "missing_metatable_for_sealed_tables_do_not_get_inferred") +TEST_CASE_FIXTURE(BuiltinsFixture, "missing_metatable_for_sealed_tables_do_not_get_inferred") { CheckResult result = check(R"( local t = {x = 1} @@ -1742,7 +1742,7 @@ TEST_CASE_FIXTURE(Fixture, "hide_table_error_properties") CHECK_EQ("Cannot add property 'b' to table '{| x: number |}'", toString(result.errors[1])); } -TEST_CASE_FIXTURE(Fixture, "builtin_table_names") +TEST_CASE_FIXTURE(BuiltinsFixture, "builtin_table_names") { CheckResult result = check(R"( os.h = 2 @@ -1755,7 +1755,7 @@ TEST_CASE_FIXTURE(Fixture, "builtin_table_names") CHECK_EQ("Cannot add property 'k' to table 'string'", toString(result.errors[1])); } -TEST_CASE_FIXTURE(Fixture, "persistent_sealed_table_is_immutable") +TEST_CASE_FIXTURE(BuiltinsFixture, "persistent_sealed_table_is_immutable") { CheckResult result = check(R"( --!nonstrict @@ -1858,7 +1858,7 @@ local foos: {Foo} = { LUAU_REQUIRE_NO_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "quantifying_a_bound_var_works") +TEST_CASE_FIXTURE(BuiltinsFixture, "quantifying_a_bound_var_works") { CheckResult result = check(R"( local clazz = {} @@ -1983,7 +1983,7 @@ TEST_CASE_FIXTURE(Fixture, "invariant_table_properties_means_instantiating_table LUAU_REQUIRE_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "table_insert_should_cope_with_optional_properties_in_nonstrict") +TEST_CASE_FIXTURE(BuiltinsFixture, "table_insert_should_cope_with_optional_properties_in_nonstrict") { CheckResult result = check(R"( --!nonstrict @@ -1996,7 +1996,7 @@ TEST_CASE_FIXTURE(Fixture, "table_insert_should_cope_with_optional_properties_in LUAU_REQUIRE_NO_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "table_insert_should_cope_with_optional_properties_in_strict") +TEST_CASE_FIXTURE(BuiltinsFixture, "table_insert_should_cope_with_optional_properties_in_strict") { ScopedFastFlag sff{"LuauTableSubtypingVariance2", true}; @@ -2052,7 +2052,7 @@ caused by: Property 'y' is not compatible. Type 'number' could not be converted into 'string')"); } -TEST_CASE_FIXTURE(Fixture, "error_detailed_metatable_prop") +TEST_CASE_FIXTURE(BuiltinsFixture, "error_detailed_metatable_prop") { ScopedFastFlag sff[]{ {"LuauTableSubtypingVariance2", true}, @@ -2183,7 +2183,7 @@ a.p = { x = 9 } LUAU_REQUIRE_NO_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "recursive_metatable_type_call") +TEST_CASE_FIXTURE(BuiltinsFixture, "recursive_metatable_type_call") { ScopedFastFlag sff[]{ {"LuauUnsealedTableLiteral", true}, @@ -2277,7 +2277,7 @@ local y = #x LUAU_REQUIRE_ERROR_COUNT(1, result); } -TEST_CASE_FIXTURE(Fixture, "dont_hang_when_trying_to_look_up_in_cyclic_metatable_index") +TEST_CASE_FIXTURE(BuiltinsFixture, "dont_hang_when_trying_to_look_up_in_cyclic_metatable_index") { ScopedFastFlag sff{"LuauTerminateCyclicMetatableIndexLookup", true}; @@ -2296,7 +2296,7 @@ TEST_CASE_FIXTURE(Fixture, "dont_hang_when_trying_to_look_up_in_cyclic_metatable CHECK_EQ("Type 't' does not have key 'p'", toString(result.errors[0])); } -TEST_CASE_FIXTURE(Fixture, "give_up_after_one_metatable_index_look_up") +TEST_CASE_FIXTURE(BuiltinsFixture, "give_up_after_one_metatable_index_look_up") { CheckResult result = check(R"( local data = { x = 5 } @@ -2478,7 +2478,7 @@ TEST_CASE_FIXTURE(Fixture, "free_rhs_table_can_also_be_bound") )"); } -TEST_CASE_FIXTURE(Fixture, "table_unifies_into_map") +TEST_CASE_FIXTURE(BuiltinsFixture, "table_unifies_into_map") { CheckResult result = check(R"( local Instance: any @@ -2564,7 +2564,7 @@ TEST_CASE_FIXTURE(Fixture, "generalize_table_argument") * the generalization process), then it loses the knowledge that its metatable will have an :incr() * method. */ -TEST_CASE_FIXTURE(Fixture, "dont_quantify_table_that_belongs_to_outer_scope") +TEST_CASE_FIXTURE(BuiltinsFixture, "dont_quantify_table_that_belongs_to_outer_scope") { CheckResult result = check(R"( local Counter = {} @@ -2606,7 +2606,7 @@ TEST_CASE_FIXTURE(Fixture, "dont_quantify_table_that_belongs_to_outer_scope") } // TODO: CLI-39624 -TEST_CASE_FIXTURE(Fixture, "instantiate_tables_at_scope_level") +TEST_CASE_FIXTURE(BuiltinsFixture, "instantiate_tables_at_scope_level") { CheckResult result = check(R"( --!strict @@ -2690,7 +2690,7 @@ TEST_CASE_FIXTURE(Fixture, "dont_crash_when_setmetatable_does_not_produce_a_meta LUAU_REQUIRE_ERROR_COUNT(1, result); } -TEST_CASE_FIXTURE(Fixture, "instantiate_table_cloning") +TEST_CASE_FIXTURE(BuiltinsFixture, "instantiate_table_cloning") { CheckResult result = check(R"( --!nonstrict @@ -2711,7 +2711,7 @@ type t0 = any CHECK(ttv->instantiatedTypeParams.empty()); } -TEST_CASE_FIXTURE(Fixture, "instantiate_table_cloning_2") +TEST_CASE_FIXTURE(BuiltinsFixture, "instantiate_table_cloning_2") { ScopedFastFlag sff{"LuauOnlyMutateInstantiatedTables", true}; @@ -2767,7 +2767,7 @@ local baz = foo[bar] CHECK_EQ(result.errors[0].location, Location{Position{3, 16}, Position{3, 19}}); } -TEST_CASE_FIXTURE(Fixture, "table_simple_call") +TEST_CASE_FIXTURE(BuiltinsFixture, "table_simple_call") { CheckResult result = check(R"( local a = setmetatable({ x = 2 }, { @@ -2783,7 +2783,7 @@ local c = a(2) -- too many arguments CHECK_EQ("Argument count mismatch. Function expects 1 argument, but 2 are specified", toString(result.errors[0])); } -TEST_CASE_FIXTURE(Fixture, "access_index_metamethod_that_returns_variadic") +TEST_CASE_FIXTURE(BuiltinsFixture, "access_index_metamethod_that_returns_variadic") { CheckResult result = check(R"( type Foo = {x: string} @@ -2878,7 +2878,7 @@ TEST_CASE_FIXTURE(Fixture, "pairs_parameters_are_not_unsealed_tables") )"); } -TEST_CASE_FIXTURE(Fixture, "table_function_check_use_after_free") +TEST_CASE_FIXTURE(BuiltinsFixture, "table_function_check_use_after_free") { CheckResult result = check(R"( local t = {} @@ -2916,7 +2916,7 @@ TEST_CASE_FIXTURE(Fixture, "inferred_properties_of_a_table_should_start_with_the } // The real bug here was that we weren't always uncondionally typechecking a trailing return statement last. -TEST_CASE_FIXTURE(Fixture, "dont_leak_free_table_props") +TEST_CASE_FIXTURE(BuiltinsFixture, "dont_leak_free_table_props") { CheckResult result = check(R"( local function a(state) diff --git a/tests/TypeInfer.test.cpp b/tests/TypeInfer.test.cpp index e81ef1a9..48cd1c3d 100644 --- a/tests/TypeInfer.test.cpp +++ b/tests/TypeInfer.test.cpp @@ -161,7 +161,7 @@ TEST_CASE_FIXTURE(Fixture, "unify_nearly_identical_recursive_types") LUAU_REQUIRE_NO_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "warn_on_lowercase_parent_property") +TEST_CASE_FIXTURE(BuiltinsFixture, "warn_on_lowercase_parent_property") { CheckResult result = check(R"( local M = require(script.parent.DoesNotMatter) @@ -175,7 +175,7 @@ TEST_CASE_FIXTURE(Fixture, "warn_on_lowercase_parent_property") REQUIRE_EQ("parent", ed->symbol); } -TEST_CASE_FIXTURE(Fixture, "weird_case") +TEST_CASE_FIXTURE(BuiltinsFixture, "weird_case") { CheckResult result = check(R"( local function f() return 4 end @@ -419,7 +419,7 @@ TEST_CASE_FIXTURE(Fixture, "globals_everywhere") CHECK_EQ("any", toString(requireType("bar"))); } -TEST_CASE_FIXTURE(Fixture, "correctly_scope_locals_do") +TEST_CASE_FIXTURE(BuiltinsFixture, "correctly_scope_locals_do") { CheckResult result = check(R"( do @@ -534,7 +534,7 @@ TEST_CASE_FIXTURE(Fixture, "tc_after_error_recovery_no_assert") LUAU_REQUIRE_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "tc_after_error_recovery_no_replacement_name_in_error") +TEST_CASE_FIXTURE(BuiltinsFixture, "tc_after_error_recovery_no_replacement_name_in_error") { { CheckResult result = check(R"( @@ -587,7 +587,7 @@ TEST_CASE_FIXTURE(Fixture, "tc_after_error_recovery_no_replacement_name_in_error } } -TEST_CASE_FIXTURE(Fixture, "index_expr_should_be_checked") +TEST_CASE_FIXTURE(BuiltinsFixture, "index_expr_should_be_checked") { CheckResult result = check(R"( local foo: any @@ -768,7 +768,7 @@ b, c = {2, "s"}, {"b", 4} LUAU_REQUIRE_NO_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "infer_assignment_value_types_mutable_lval") +TEST_CASE_FIXTURE(BuiltinsFixture, "infer_assignment_value_types_mutable_lval") { CheckResult result = check(R"( local a = {} @@ -836,7 +836,7 @@ local a: number? = if true then 1 else nil LUAU_REQUIRE_NO_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "tc_if_else_expressions_expected_type_3") +TEST_CASE_FIXTURE(BuiltinsFixture, "tc_if_else_expressions_expected_type_3") { CheckResult result = check(R"( local function times(n: any, f: () -> T) @@ -907,7 +907,7 @@ TEST_CASE_FIXTURE(Fixture, "fuzzer_found_this") )"); } -TEST_CASE_FIXTURE(Fixture, "recursive_metatable_crash") +TEST_CASE_FIXTURE(BuiltinsFixture, "recursive_metatable_crash") { CheckResult result = check(R"( local function getIt() @@ -1041,7 +1041,6 @@ TEST_CASE_FIXTURE(Fixture, "follow_on_new_types_in_substitution") TEST_CASE_FIXTURE(Fixture, "do_not_bind_a_free_table_to_a_union_containing_that_table") { ScopedFastFlag flag[] = { - {"LuauStatFunctionSimplify4", true}, {"LuauLowerBoundsCalculation", true}, {"LuauDifferentOrderOfUnificationDoesntMatter2", true}, }; diff --git a/tests/TypeInfer.tryUnify.test.cpp b/tests/TypeInfer.tryUnify.test.cpp index 87562644..49deae71 100644 --- a/tests/TypeInfer.tryUnify.test.cpp +++ b/tests/TypeInfer.tryUnify.test.cpp @@ -196,7 +196,7 @@ TEST_CASE_FIXTURE(TryUnifyFixture, "variadics_should_use_reversed_properly") CHECK_EQ(toString(tm->wantedType), "string"); } -TEST_CASE_FIXTURE(TryUnifyFixture, "cli_41095_concat_log_in_sealed_table_unification") +TEST_CASE_FIXTURE(BuiltinsFixture, "cli_41095_concat_log_in_sealed_table_unification") { CheckResult result = check(R"( --!strict diff --git a/tests/TypeInfer.typePacks.cpp b/tests/TypeInfer.typePacks.cpp index f141622f..fd66b080 100644 --- a/tests/TypeInfer.typePacks.cpp +++ b/tests/TypeInfer.typePacks.cpp @@ -339,7 +339,7 @@ local c: Packed CHECK_EQ(toString(ttvC->instantiatedTypePackParams[0], {true}), "number, boolean"); } -TEST_CASE_FIXTURE(Fixture, "type_alias_type_packs_import") +TEST_CASE_FIXTURE(BuiltinsFixture, "type_alias_type_packs_import") { fileResolver.source["game/A"] = R"( export type Packed = { a: T, b: (U...) -> () } @@ -369,7 +369,7 @@ local d: { a: typeof(c) } CHECK_EQ(toString(requireType("d")), "{| a: Packed |}"); } -TEST_CASE_FIXTURE(Fixture, "type_pack_type_parameters") +TEST_CASE_FIXTURE(BuiltinsFixture, "type_pack_type_parameters") { fileResolver.source["game/A"] = R"( export type Packed = { a: T, b: (U...) -> () } @@ -784,7 +784,7 @@ local a: Y<...number> LUAU_REQUIRE_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "type_alias_default_export") +TEST_CASE_FIXTURE(BuiltinsFixture, "type_alias_default_export") { fileResolver.source["Module/Types"] = R"( export type A = { a: T, b: U } diff --git a/tests/TypeInfer.unionTypes.test.cpp b/tests/TypeInfer.unionTypes.test.cpp index 96bdd534..277f3887 100644 --- a/tests/TypeInfer.unionTypes.test.cpp +++ b/tests/TypeInfer.unionTypes.test.cpp @@ -104,7 +104,7 @@ TEST_CASE_FIXTURE(Fixture, "optional_arguments_table2") REQUIRE(!result.errors.empty()); } -TEST_CASE_FIXTURE(Fixture, "error_takes_optional_arguments") +TEST_CASE_FIXTURE(BuiltinsFixture, "error_takes_optional_arguments") { CheckResult result = check(R"( error("message") @@ -517,10 +517,8 @@ TEST_CASE_FIXTURE(Fixture, "dont_allow_cyclic_unions_to_be_inferred") LUAU_REQUIRE_NO_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "table_union_write_indirect") +TEST_CASE_FIXTURE(BuiltinsFixture, "table_union_write_indirect") { - ScopedFastFlag statFunctionSimplify{"LuauStatFunctionSimplify4", true}; - CheckResult result = check(R"( type A = { x: number, y: (number) -> string } | { z: number, y: (number) -> string } From a36b1eb29ba5c6c414305b4bbd2b6e2c58ed7d06 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Fri, 13 May 2022 12:36:37 -0700 Subject: [PATCH 248/399] Sync to upstream/release/527 (#481) --- Analysis/include/Luau/TypeVar.h | 2 +- Analysis/src/Clone.cpp | 4 +- Analysis/src/Normalize.cpp | 3 +- Analysis/src/Substitution.cpp | 4 +- Analysis/src/ToString.cpp | 16 +- Analysis/src/TypeInfer.cpp | 191 ++++++++------------- Ast/src/Parser.cpp | 3 +- CMakeLists.txt | 6 + Compiler/src/Compiler.cpp | 30 +++- Compiler/src/ConstantFolding.cpp | 6 +- Compiler/src/CostModel.cpp | 2 +- extern/isocline/src/bbcode.c | 1 + tests/AstQuery.test.cpp | 2 +- tests/Autocomplete.test.cpp | 44 +++-- tests/BuiltinDefinitions.test.cpp | 4 +- tests/Compiler.test.cpp | 71 ++++++-- tests/CostModel.test.cpp | 14 +- tests/Fixture.cpp | 19 +- tests/Fixture.h | 5 + tests/Frontend.test.cpp | 2 +- tests/Linter.test.cpp | 6 +- tests/Module.test.cpp | 4 +- tests/NonstrictMode.test.cpp | 6 +- tests/Normalize.test.cpp | 7 +- tests/Parser.test.cpp | 2 - tests/RuntimeLimits.test.cpp | 2 +- tests/ToDot.test.cpp | 2 +- tests/ToString.test.cpp | 4 +- tests/TypeInfer.aliases.test.cpp | 10 +- tests/TypeInfer.annotations.test.cpp | 8 +- tests/TypeInfer.anyerror.test.cpp | 4 +- tests/TypeInfer.builtins.test.cpp | 117 +++++++------ tests/TypeInfer.classes.test.cpp | 2 +- tests/TypeInfer.functions.test.cpp | 38 ++-- tests/TypeInfer.generics.test.cpp | 12 +- tests/TypeInfer.intersectionTypes.test.cpp | 6 +- tests/TypeInfer.loops.test.cpp | 28 +-- tests/TypeInfer.modules.test.cpp | 28 +-- tests/TypeInfer.oop.test.cpp | 4 +- tests/TypeInfer.operators.test.cpp | 71 ++++++-- tests/TypeInfer.provisional.test.cpp | 10 +- tests/TypeInfer.refinements.test.cpp | 20 +-- tests/TypeInfer.singletons.test.cpp | 2 +- tests/TypeInfer.tables.test.cpp | 74 ++++---- tests/TypeInfer.test.cpp | 17 +- tests/TypeInfer.tryUnify.test.cpp | 2 +- tests/TypeInfer.typePacks.cpp | 6 +- tests/TypeInfer.unionTypes.test.cpp | 6 +- 48 files changed, 511 insertions(+), 416 deletions(-) diff --git a/Analysis/include/Luau/TypeVar.h b/Analysis/include/Luau/TypeVar.h index 84576758..9cacbc6d 100644 --- a/Analysis/include/Luau/TypeVar.h +++ b/Analysis/include/Luau/TypeVar.h @@ -329,7 +329,7 @@ struct TableTypeVar // We need to know which is which when we stringify types. std::optional syntheticName; - std::map methodDefinitionLocations; + std::map methodDefinitionLocations; // TODO: Remove with FFlag::LuauNoMethodLocations std::vector instantiatedTypeParams; std::vector instantiatedTypePackParams; ModuleName definitionModuleName; diff --git a/Analysis/src/Clone.cpp b/Analysis/src/Clone.cpp index d5bd9dab..1aa556eb 100644 --- a/Analysis/src/Clone.cpp +++ b/Analysis/src/Clone.cpp @@ -11,6 +11,7 @@ LUAU_FASTFLAG(DebugLuauCopyBeforeNormalizing) LUAU_FASTINTVARIABLE(LuauTypeCloneRecursionLimit, 300) LUAU_FASTFLAG(LuauTypecheckOptPass) LUAU_FASTFLAGVARIABLE(LuauLosslessClone, false) +LUAU_FASTFLAG(LuauNoMethodLocations) namespace Luau { @@ -277,7 +278,8 @@ void TypeCloner::operator()(const TableTypeVar& t) } ttv->definitionModuleName = t.definitionModuleName; - ttv->methodDefinitionLocations = t.methodDefinitionLocations; + if (!FFlag::LuauNoMethodLocations) + ttv->methodDefinitionLocations = t.methodDefinitionLocations; ttv->tags = t.tags; } diff --git a/Analysis/src/Normalize.cpp b/Analysis/src/Normalize.cpp index d8c11388..ef5377a1 100644 --- a/Analysis/src/Normalize.cpp +++ b/Analysis/src/Normalize.cpp @@ -14,7 +14,6 @@ LUAU_FASTFLAGVARIABLE(DebugLuauCopyBeforeNormalizing, false) // This could theoretically be 2000 on amd64, but x86 requires this. LUAU_FASTINTVARIABLE(LuauNormalizeIterationLimit, 1200); LUAU_FASTFLAGVARIABLE(LuauNormalizeCombineTableFix, false); -LUAU_FASTFLAGVARIABLE(LuauNormalizeCombineIntersectionFix, false); namespace Luau { @@ -863,7 +862,7 @@ struct Normalize final : TypeVarVisitor TypeId theTable = result->parts.back(); - if (!get(FFlag::LuauNormalizeCombineIntersectionFix ? follow(theTable) : theTable)) + if (!get(follow(theTable))) { result->parts.push_back(arena.addType(TableTypeVar{TableState::Sealed, TypeLevel{}})); theTable = result->parts.back(); diff --git a/Analysis/src/Substitution.cpp b/Analysis/src/Substitution.cpp index 30d8574a..c5c7977a 100644 --- a/Analysis/src/Substitution.cpp +++ b/Analysis/src/Substitution.cpp @@ -12,6 +12,7 @@ LUAU_FASTINTVARIABLE(LuauTarjanChildLimit, 10000) LUAU_FASTFLAG(LuauTypecheckOptPass) LUAU_FASTFLAGVARIABLE(LuauSubstituteFollowNewTypes, false) LUAU_FASTFLAGVARIABLE(LuauSubstituteFollowPossibleMutations, false) +LUAU_FASTFLAG(LuauNoMethodLocations) namespace Luau { @@ -408,7 +409,8 @@ TypeId Substitution::clone(TypeId ty) { LUAU_ASSERT(!ttv->boundTo); TableTypeVar clone = TableTypeVar{ttv->props, ttv->indexer, ttv->level, ttv->state}; - clone.methodDefinitionLocations = ttv->methodDefinitionLocations; + if (!FFlag::LuauNoMethodLocations) + clone.methodDefinitionLocations = ttv->methodDefinitionLocations; clone.definitionModuleName = ttv->definitionModuleName; clone.name = ttv->name; clone.syntheticName = ttv->syntheticName; diff --git a/Analysis/src/ToString.cpp b/Analysis/src/ToString.cpp index b5d6a550..51665f7f 100644 --- a/Analysis/src/ToString.cpp +++ b/Analysis/src/ToString.cpp @@ -248,6 +248,13 @@ struct StringifierState result.name += s; } + void emit(TypeLevel level) + { + emit(std::to_string(level.level)); + emit("-"); + emit(std::to_string(level.subLevel)); + } + void emit(const char* s) { if (opts.maxTypeLength > 0 && result.name.length() > opts.maxTypeLength) @@ -379,7 +386,7 @@ struct TypeVarStringifier if (FFlag::DebugLuauVerboseTypeNames) { state.emit("-"); - state.emit(std::to_string(ftv.level.level)); + state.emit(ftv.level); } } @@ -403,7 +410,10 @@ struct TypeVarStringifier { state.result.invalid = true; - state.emit("[["); + state.emit("["); + if (FFlag::DebugLuauVerboseTypeNames) + state.emit(ctv.level); + state.emit("["); bool first = true; for (TypeId ty : ctv.parts) @@ -947,7 +957,7 @@ struct TypePackStringifier if (FFlag::DebugLuauVerboseTypeNames) { state.emit("-"); - state.emit(std::to_string(pack.level.level)); + state.emit(pack.level); } state.emit("..."); diff --git a/Analysis/src/TypeInfer.cpp b/Analysis/src/TypeInfer.cpp index 4466ede2..a13abd53 100644 --- a/Analysis/src/TypeInfer.cpp +++ b/Analysis/src/TypeInfer.cpp @@ -36,13 +36,11 @@ LUAU_FASTFLAGVARIABLE(LuauEqConstraint, false) LUAU_FASTFLAGVARIABLE(LuauWeakEqConstraint, false) // Eventually removed as false. LUAU_FASTFLAGVARIABLE(LuauLowerBoundsCalculation, false) LUAU_FASTFLAGVARIABLE(DebugLuauFreezeDuringUnification, false) -LUAU_FASTFLAGVARIABLE(LuauInferStatFunction, false) LUAU_FASTFLAGVARIABLE(LuauInstantiateFollows, false) LUAU_FASTFLAGVARIABLE(LuauSelfCallAutocompleteFix, false) LUAU_FASTFLAGVARIABLE(LuauDiscriminableUnions2, false) LUAU_FASTFLAGVARIABLE(LuauReduceUnionRecursion, false) LUAU_FASTFLAGVARIABLE(LuauOnlyMutateInstantiatedTables, false) -LUAU_FASTFLAGVARIABLE(LuauStatFunctionSimplify4, false) LUAU_FASTFLAGVARIABLE(LuauTypecheckOptPass, false) LUAU_FASTFLAGVARIABLE(LuauUnsealedTableLiteral, false) LUAU_FASTFLAGVARIABLE(LuauTwoPassAliasDefinitionFix, false) @@ -53,12 +51,13 @@ LUAU_FASTFLAGVARIABLE(LuauDoNotTryToReduce, false) LUAU_FASTFLAGVARIABLE(LuauDoNotAccidentallyDependOnPointerOrdering, false) LUAU_FASTFLAGVARIABLE(LuauCheckImplicitNumbericKeys, false) LUAU_FASTFLAG(LuauAnyInIsOptionalIsOptional) -LUAU_FASTFLAGVARIABLE(LuauDecoupleOperatorInferenceFromUnifiedTypeInference, false) LUAU_FASTFLAGVARIABLE(LuauTableUseCounterInstead, false) LUAU_FASTFLAGVARIABLE(LuauReturnTypeInferenceInNonstrict, false) LUAU_FASTFLAGVARIABLE(LuauRecursionLimitException, false); LUAU_FASTFLAG(LuauLosslessClone) LUAU_FASTFLAGVARIABLE(LuauTypecheckIter, false); +LUAU_FASTFLAGVARIABLE(LuauSuccessTypingForEqualityOperations, false) +LUAU_FASTFLAGVARIABLE(LuauNoMethodLocations, false); namespace Luau { @@ -587,7 +586,7 @@ void TypeChecker::checkBlockWithoutRecursionCheck(const ScopePtr& scope, const A { std::optional expectedType; - if (FFlag::LuauInferStatFunction && !fun->func->self) + if (!fun->func->self) { if (auto name = fun->name->as()) { @@ -1307,7 +1306,7 @@ void TypeChecker::check(const ScopePtr& scope, TypeId ty, const ScopePtr& funSco scope->bindings[name->local] = {anyIfNonstrict(quantify(funScope, ty, name->local->location)), name->local->location}; return; } - else if (auto name = function.name->as(); name && FFlag::LuauStatFunctionSimplify4) + else if (auto name = function.name->as()) { TypeId exprTy = checkExpr(scope, *name->expr).type; TableTypeVar* ttv = getMutableTableType(exprTy); @@ -1341,7 +1340,7 @@ void TypeChecker::check(const ScopePtr& scope, TypeId ty, const ScopePtr& funSco if (ttv && ttv->state != TableState::Sealed) ttv->props[name->index.value] = {follow(quantify(funScope, ty, name->indexLocation)), /* deprecated */ false, {}, name->indexLocation}; } - else if (FFlag::LuauStatFunctionSimplify4) + else { LUAU_ASSERT(function.name->is()); @@ -1349,71 +1348,6 @@ void TypeChecker::check(const ScopePtr& scope, TypeId ty, const ScopePtr& funSco checkFunctionBody(funScope, ty, *function.func); } - else if (function.func->self) - { - LUAU_ASSERT(!FFlag::LuauStatFunctionSimplify4); - - AstExprIndexName* indexName = function.name->as(); - if (!indexName) - ice("member function declaration has malformed name expression"); - - TypeId selfTy = checkExpr(scope, *indexName->expr).type; - TableTypeVar* tableSelf = getMutableTableType(selfTy); - if (!tableSelf) - { - if (isTableIntersection(selfTy)) - reportError(TypeError{function.location, CannotExtendTable{selfTy, CannotExtendTable::Property, indexName->index.value}}); - else if (!get(selfTy) && !get(selfTy)) - reportError(TypeError{function.location, OnlyTablesCanHaveMethods{selfTy}}); - } - else if (tableSelf->state == TableState::Sealed) - reportError(TypeError{function.location, CannotExtendTable{selfTy, CannotExtendTable::Property, indexName->index.value}}); - - const bool tableIsExtendable = tableSelf && tableSelf->state != TableState::Sealed; - - ty = follow(ty); - - if (tableIsExtendable) - tableSelf->props[indexName->index.value] = {ty, /* deprecated */ false, {}, indexName->indexLocation}; - - const FunctionTypeVar* funTy = get(ty); - if (!funTy) - ice("Methods should be functions"); - - std::optional arg0 = first(funTy->argTypes); - if (!arg0) - ice("Methods should always have at least 1 argument (self)"); - - checkFunctionBody(funScope, ty, *function.func); - - if (tableIsExtendable) - tableSelf->props[indexName->index.value] = { - follow(quantify(funScope, ty, indexName->indexLocation)), /* deprecated */ false, {}, indexName->indexLocation}; - } - else - { - LUAU_ASSERT(!FFlag::LuauStatFunctionSimplify4); - - TypeId leftType = checkLValueBinding(scope, *function.name); - - checkFunctionBody(funScope, ty, *function.func); - - unify(ty, leftType, function.location); - - LUAU_ASSERT(function.name->is() || function.name->is()); - - if (auto exprIndexName = function.name->as()) - { - if (auto typeIt = currentModule->astTypes.find(exprIndexName->expr)) - { - if (auto ttv = getMutableTableType(*typeIt)) - { - if (auto it = ttv->props.find(exprIndexName->index.value); it != ttv->props.end()) - it->second.type = follow(quantify(funScope, leftType, function.name->location)); - } - } - } - } } void TypeChecker::check(const ScopePtr& scope, TypeId ty, const ScopePtr& funScope, const AstStatLocalFunction& function) @@ -1523,7 +1457,8 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatTypeAlias& typealias // This is a shallow clone, original recursive links to self are not updated TableTypeVar clone = TableTypeVar{ttv->props, ttv->indexer, ttv->level, ttv->state}; - clone.methodDefinitionLocations = ttv->methodDefinitionLocations; + if (!FFlag::LuauNoMethodLocations) + clone.methodDefinitionLocations = ttv->methodDefinitionLocations; clone.definitionModuleName = ttv->definitionModuleName; clone.name = name; @@ -2652,13 +2587,58 @@ TypeId TypeChecker::checkRelationalOperation( std::optional leftMetatable = isString(lhsType) ? std::nullopt : getMetatable(follow(lhsType)); std::optional rightMetatable = isString(rhsType) ? std::nullopt : getMetatable(follow(rhsType)); - // TODO: this check seems odd, the second part is redundant - // is it meant to be if (leftMetatable && rightMetatable && leftMetatable != rightMetatable) - if (bool(leftMetatable) != bool(rightMetatable) && leftMetatable != rightMetatable) + if (FFlag::LuauSuccessTypingForEqualityOperations) { - reportError(expr.location, GenericError{format("Types %s and %s cannot be compared with %s because they do not have the same metatable", - toString(lhsType).c_str(), toString(rhsType).c_str(), toString(expr.op).c_str())}); - return errorRecoveryType(booleanType); + if (leftMetatable != rightMetatable) + { + bool matches = false; + if (isEquality) + { + if (const UnionTypeVar* utv = get(leftType); utv && rightMetatable) + { + for (TypeId leftOption : utv) + { + if (getMetatable(follow(leftOption)) == rightMetatable) + { + matches = true; + break; + } + } + } + + if (!matches) + { + if (const UnionTypeVar* utv = get(rhsType); utv && leftMetatable) + { + for (TypeId rightOption : utv) + { + if (getMetatable(follow(rightOption)) == leftMetatable) + { + matches = true; + break; + } + } + } + } + } + + + if (!matches) + { + reportError(expr.location, GenericError{format("Types %s and %s cannot be compared with %s because they do not have the same metatable", + toString(lhsType).c_str(), toString(rhsType).c_str(), toString(expr.op).c_str())}); + return errorRecoveryType(booleanType); + } + } + } + else + { + if (bool(leftMetatable) != bool(rightMetatable) && leftMetatable != rightMetatable) + { + reportError(expr.location, GenericError{format("Types %s and %s cannot be compared with %s because they do not have the same metatable", + toString(lhsType).c_str(), toString(rhsType).c_str(), toString(expr.op).c_str())}); + return errorRecoveryType(booleanType); + } } if (leftMetatable) @@ -2754,22 +2734,11 @@ TypeId TypeChecker::checkBinaryOperation( lhsType = follow(lhsType); rhsType = follow(rhsType); - if (FFlag::LuauDecoupleOperatorInferenceFromUnifiedTypeInference) + if (!isNonstrictMode() && get(lhsType)) { - if (!isNonstrictMode() && get(lhsType)) - { - auto name = getIdentifierOfBaseVar(expr.left); - reportError(expr.location, CannotInferBinaryOperation{expr.op, name, CannotInferBinaryOperation::Operation}); - // We will fall-through to the `return anyType` check below. - } - } - else - { - if (!isNonstrictMode() && get(lhsType)) - { - auto name = getIdentifierOfBaseVar(expr.left); - reportError(expr.location, CannotInferBinaryOperation{expr.op, name, CannotInferBinaryOperation::Operation}); - } + auto name = getIdentifierOfBaseVar(expr.left); + reportError(expr.location, CannotInferBinaryOperation{expr.op, name, CannotInferBinaryOperation::Operation}); + // We will fall-through to the `return anyType` check below. } // If we know nothing at all about the lhs type, we can usually say nothing about the result. @@ -3231,43 +3200,27 @@ TypeId TypeChecker::checkFunctionName(const ScopePtr& scope, AstExpr& funName, T else if (auto indexName = funName.as()) { TypeId lhsType = checkExpr(scope, *indexName->expr).type; - - if (!FFlag::LuauStatFunctionSimplify4 && (get(lhsType) || get(lhsType))) - return lhsType; - TableTypeVar* ttv = getMutableTableType(lhsType); - if (FFlag::LuauStatFunctionSimplify4) + if (!ttv || ttv->state == TableState::Sealed) { - if (!ttv || ttv->state == TableState::Sealed) - { - if (auto ty = getIndexTypeFromType(scope, lhsType, indexName->index.value, indexName->indexLocation, false)) - return *ty; + if (auto ty = getIndexTypeFromType(scope, lhsType, indexName->index.value, indexName->indexLocation, false)) + return *ty; - return errorRecoveryType(scope); - } - } - else - { - if (!ttv || lhsType->persistent || ttv->state == TableState::Sealed) - return errorRecoveryType(scope); + return errorRecoveryType(scope); } Name name = indexName->index.value; if (ttv->props.count(name)) - { - if (FFlag::LuauStatFunctionSimplify4) - return ttv->props[name].type; - else - return errorRecoveryType(scope); - } + return ttv->props[name].type; Property& property = ttv->props[name]; property.type = freshTy(); property.location = indexName->indexLocation; - ttv->methodDefinitionLocations[name] = funName.location; + if (!FFlag::LuauNoMethodLocations) + ttv->methodDefinitionLocations[name] = funName.location; return property.type; } else if (funName.is()) @@ -4669,7 +4622,8 @@ TypeId ReplaceGenerics::clean(TypeId ty) if (const TableTypeVar* ttv = log->getMutable(ty)) { TableTypeVar clone = TableTypeVar{ttv->props, ttv->indexer, level, TableState::Free}; - clone.methodDefinitionLocations = ttv->methodDefinitionLocations; + if (!FFlag::LuauNoMethodLocations) + clone.methodDefinitionLocations = ttv->methodDefinitionLocations; clone.definitionModuleName = ttv->definitionModuleName; return addType(std::move(clone)); } @@ -4715,7 +4669,8 @@ TypeId Anyification::clean(TypeId ty) if (const TableTypeVar* ttv = log->getMutable(ty)) { TableTypeVar clone = TableTypeVar{ttv->props, ttv->indexer, ttv->level, TableState::Sealed}; - clone.methodDefinitionLocations = ttv->methodDefinitionLocations; + if (!FFlag::LuauNoMethodLocations) + clone.methodDefinitionLocations = ttv->methodDefinitionLocations; clone.definitionModuleName = ttv->definitionModuleName; clone.name = ttv->name; clone.syntheticName = ttv->syntheticName; diff --git a/Ast/src/Parser.cpp b/Ast/src/Parser.cpp index 91f5cd25..c053e6bd 100644 --- a/Ast/src/Parser.cpp +++ b/Ast/src/Parser.cpp @@ -10,7 +10,6 @@ // See docs/SyntaxChanges.md for an explanation. LUAU_FASTINTVARIABLE(LuauRecursionLimit, 1000) LUAU_FASTINTVARIABLE(LuauParseErrorLimit, 100) -LUAU_FASTFLAGVARIABLE(LuauParseLocationIgnoreCommentSkipInCapture, false) namespace Luau { @@ -2821,7 +2820,7 @@ void Parser::nextLexeme() hotcomments.push_back({hotcommentHeader, lexeme.location, std::string(text + 1, text + end)}); } - type = lexer.next(/* skipComments= */ false, !FFlag::LuauParseLocationIgnoreCommentSkipInCapture).type; + type = lexer.next(/* skipComments= */ false, /* updatePrevLocation= */ false).type; } } else diff --git a/CMakeLists.txt b/CMakeLists.txt index af03b33a..ea352309 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -110,6 +110,12 @@ if (MSVC AND MSVC_VERSION GREATER_EQUAL 1924) set_source_files_properties(VM/src/lvmexecute.cpp PROPERTIES COMPILE_FLAGS /d2ssa-pre-) endif() +if(MSVC AND LUAU_BUILD_CLI) + # the default stack size that MSVC linker uses is 1 MB; we need more stack space in Debug because stack frames are larger + set_target_properties(Luau.Analyze.CLI PROPERTIES LINK_FLAGS_DEBUG /STACK:2097152) + set_target_properties(Luau.Repl.CLI PROPERTIES LINK_FLAGS_DEBUG /STACK:2097152) +endif() + # embed .natvis inside the library debug information if(MSVC) target_link_options(Luau.Ast INTERFACE /NATVIS:${CMAKE_CURRENT_SOURCE_DIR}/tools/natvis/Ast.natvis) diff --git a/Compiler/src/Compiler.cpp b/Compiler/src/Compiler.cpp index 4fe26222..e177e928 100644 --- a/Compiler/src/Compiler.cpp +++ b/Compiler/src/Compiler.cpp @@ -628,7 +628,12 @@ struct Compiler return; if (fi && !fi->canInline) - bytecode.addDebugRemark("inlining failed: complex constructs in function body"); + { + if (func->vararg) + bytecode.addDebugRemark("inlining failed: function is variadic"); + else + bytecode.addDebugRemark("inlining failed: complex constructs in function body"); + } } RegScope rs(this); @@ -2342,17 +2347,28 @@ struct Compiler RegScope rs(this); uint8_t temp = 0; + bool consecutive = false; bool multRet = false; - // Optimization: return local value directly instead of copying it into a temporary - if (stat->list.size == 1 && isExprLocalReg(stat->list.data[0])) + // Optimization: return locals directly instead of copying them into a temporary + // this is very important for a single return value and occasionally effective for multiple values + if (stat->list.size > 0 && isExprLocalReg(stat->list.data[0])) { - AstExprLocal* le = stat->list.data[0]->as(); - LUAU_ASSERT(le); + temp = getLocal(stat->list.data[0]->as()->local); + consecutive = true; - temp = getLocal(le->local); + for (size_t i = 1; i < stat->list.size; ++i) + { + AstExpr* v = stat->list.data[i]; + if (!isExprLocalReg(v) || getLocal(v->as()->local) != temp + i) + { + consecutive = false; + break; + } + } } - else if (stat->list.size > 0) + + if (!consecutive && stat->list.size > 0) { temp = allocReg(stat, unsigned(stat->list.size)); diff --git a/Compiler/src/ConstantFolding.cpp b/Compiler/src/ConstantFolding.cpp index 52ece73e..e4d59ea1 100644 --- a/Compiler/src/ConstantFolding.cpp +++ b/Compiler/src/ConstantFolding.cpp @@ -195,12 +195,16 @@ struct ConstantVisitor : AstVisitor DenseHashMap& variables; DenseHashMap& locals; + bool wasEmpty = false; + ConstantVisitor( DenseHashMap& constants, DenseHashMap& variables, DenseHashMap& locals) : constants(constants) , variables(variables) , locals(locals) { + // since we do a single pass over the tree, if the initial state was empty we don't need to clear out old entries + wasEmpty = constants.empty() && locals.empty(); } Constant analyze(AstExpr* node) @@ -326,7 +330,7 @@ struct ConstantVisitor : AstVisitor { if (value.type != Constant::Type_Unknown) map[key] = value; - else if (!FFlag::LuauCompileSupportInlining) + else if (!FFlag::LuauCompileSupportInlining || wasEmpty) ; else if (Constant* old = map.find(key)) old->type = Constant::Type_Unknown; diff --git a/Compiler/src/CostModel.cpp b/Compiler/src/CostModel.cpp index 9afd09f6..f804e9de 100644 --- a/Compiler/src/CostModel.cpp +++ b/Compiler/src/CostModel.cpp @@ -187,7 +187,7 @@ struct CostVisitor : AstVisitor if (node->is()) result += 2; else if (node->is() || node->is() || node->is() || node->is()) - result += 2; + result += 5; else if (node->is() || node->is()) result += 1; diff --git a/extern/isocline/src/bbcode.c b/extern/isocline/src/bbcode.c index 4d11ac38..8722cbd6 100644 --- a/extern/isocline/src/bbcode.c +++ b/extern/isocline/src/bbcode.c @@ -575,6 +575,7 @@ ic_private const char* parse_tag_value( tag_t* tag, char* idbuf, const char* s, } // limit name and attr to 128 bytes char valbuf[128]; + valbuf[0] = 0; // fixes gcc uninitialized warning ic_strncpy( idbuf, 128, id, idend - id); ic_strncpy( valbuf, 128, val, valend - val); ic_str_tolower(idbuf); diff --git a/tests/AstQuery.test.cpp b/tests/AstQuery.test.cpp index 292625b0..12c68450 100644 --- a/tests/AstQuery.test.cpp +++ b/tests/AstQuery.test.cpp @@ -7,7 +7,7 @@ using namespace Luau; -struct DocumentationSymbolFixture : Fixture +struct DocumentationSymbolFixture : BuiltinsFixture { std::optional getDocSymbol(const std::string& source, Position position) { diff --git a/tests/Autocomplete.test.cpp b/tests/Autocomplete.test.cpp index 1c284f1f..b4e9340c 100644 --- a/tests/Autocomplete.test.cpp +++ b/tests/Autocomplete.test.cpp @@ -27,7 +27,7 @@ template struct ACFixtureImpl : BaseType { ACFixtureImpl() - : Fixture(true, true) + : BaseType(true, true) { } @@ -111,6 +111,18 @@ struct ACFixtureImpl : BaseType }; struct ACFixture : ACFixtureImpl +{ + ACFixture() + : ACFixtureImpl() + { + addGlobalBinding(frontend.typeChecker, "table", Binding{typeChecker.anyType}); + addGlobalBinding(frontend.typeChecker, "math", Binding{typeChecker.anyType}); + addGlobalBinding(frontend.typeCheckerForAutocomplete, "table", Binding{typeChecker.anyType}); + addGlobalBinding(frontend.typeCheckerForAutocomplete, "math", Binding{typeChecker.anyType}); + } +}; + +struct ACBuiltinsFixture : ACFixtureImpl { }; @@ -277,7 +289,7 @@ TEST_CASE_FIXTURE(ACFixture, "function_parameters") CHECK(ac.entryMap.count("test")); } -TEST_CASE_FIXTURE(ACFixture, "get_member_completions") +TEST_CASE_FIXTURE(ACBuiltinsFixture, "get_member_completions") { check(R"( local a = table.@1 @@ -376,7 +388,7 @@ TEST_CASE_FIXTURE(ACFixture, "table_intersection") CHECK(ac.entryMap.count("c3")); } -TEST_CASE_FIXTURE(ACFixture, "get_string_completions") +TEST_CASE_FIXTURE(ACBuiltinsFixture, "get_string_completions") { check(R"( local a = ("foo"):@1 @@ -427,7 +439,7 @@ TEST_CASE_FIXTURE(ACFixture, "method_call_inside_function_body") CHECK(!ac.entryMap.count("math")); } -TEST_CASE_FIXTURE(ACFixture, "method_call_inside_if_conditional") +TEST_CASE_FIXTURE(ACBuiltinsFixture, "method_call_inside_if_conditional") { check(R"( if table: @1 @@ -1884,7 +1896,7 @@ ex.b(function(x: CHECK(!ac.entryMap.count("(done) -> number")); } -TEST_CASE_FIXTURE(ACFixture, "suggest_external_module_type") +TEST_CASE_FIXTURE(ACBuiltinsFixture, "suggest_external_module_type") { fileResolver.source["Module/A"] = R"( export type done = { x: number, y: number } @@ -2235,7 +2247,7 @@ local a: aaa.do CHECK(ac.entryMap.count("other")); } -TEST_CASE_FIXTURE(ACFixture, "autocompleteSource") +TEST_CASE_FIXTURE(ACBuiltinsFixture, "autocompleteSource") { std::string_view source = R"( local a = table. -- Line 1 @@ -2269,7 +2281,7 @@ TEST_CASE_FIXTURE(ACFixture, "autocompleteSource_comments") CHECK_EQ(0, ac.entryMap.size()); } -TEST_CASE_FIXTURE(ACFixture, "autocompleteProp_index_function_metamethod_is_variadic") +TEST_CASE_FIXTURE(ACBuiltinsFixture, "autocompleteProp_index_function_metamethod_is_variadic") { std::string_view source = R"( type Foo = {x: number} @@ -2720,7 +2732,7 @@ type A = () -> T CHECK(ac.entryMap.count("string")); } -TEST_CASE_FIXTURE(ACFixture, "autocomplete_oop_implicit_self") +TEST_CASE_FIXTURE(ACBuiltinsFixture, "autocomplete_oop_implicit_self") { check(R"( --!strict @@ -2728,15 +2740,15 @@ local Class = {} Class.__index = Class type Class = typeof(setmetatable({} :: { x: number }, Class)) function Class.new(x: number): Class - return setmetatable({x = x}, Class) + return setmetatable({x = x}, Class) end function Class.getx(self: Class) - return self.x + return self.x end function test() - local c = Class.new(42) - local n = c:@1 - print(n) + local c = Class.new(42) + local n = c:@1 + print(n) end )"); @@ -2745,7 +2757,7 @@ end CHECK(ac.entryMap.count("getx")); } -TEST_CASE_FIXTURE(ACFixture, "autocomplete_on_string_singletons") +TEST_CASE_FIXTURE(ACBuiltinsFixture, "autocomplete_on_string_singletons") { check(R"( --!strict @@ -2989,7 +3001,7 @@ s.@1 CHECK(ac.entryMap["sub"].wrongIndexType == true); } -TEST_CASE_FIXTURE(ACFixture, "string_library_non_self_calls_are_fine") +TEST_CASE_FIXTURE(ACBuiltinsFixture, "string_library_non_self_calls_are_fine") { ScopedFastFlag selfCallAutocompleteFix{"LuauSelfCallAutocompleteFix", true}; @@ -3007,7 +3019,7 @@ string.@1 CHECK(ac.entryMap["sub"].wrongIndexType == false); } -TEST_CASE_FIXTURE(ACFixture, "string_library_self_calls_are_invalid") +TEST_CASE_FIXTURE(ACBuiltinsFixture, "string_library_self_calls_are_invalid") { ScopedFastFlag selfCallAutocompleteFix{"LuauSelfCallAutocompleteFix", true}; diff --git a/tests/BuiltinDefinitions.test.cpp b/tests/BuiltinDefinitions.test.cpp index dbe80f2c..496df4b4 100644 --- a/tests/BuiltinDefinitions.test.cpp +++ b/tests/BuiltinDefinitions.test.cpp @@ -10,8 +10,10 @@ using namespace Luau; TEST_SUITE_BEGIN("BuiltinDefinitionsTest"); -TEST_CASE_FIXTURE(Fixture, "lib_documentation_symbols") +TEST_CASE_FIXTURE(BuiltinsFixture, "lib_documentation_symbols") { + CHECK(!typeChecker.globalScope->bindings.empty()); + for (const auto& [name, binding] : typeChecker.globalScope->bindings) { std::string nameString(name.c_str()); diff --git a/tests/Compiler.test.cpp b/tests/Compiler.test.cpp index f206438f..b032060e 100644 --- a/tests/Compiler.test.cpp +++ b/tests/Compiler.test.cpp @@ -4713,7 +4713,6 @@ local function foo() end local a, b = foo() - return a, b )", 1, 2), @@ -4721,9 +4720,7 @@ return a, b DUPCLOSURE R0 K0 LOADNIL R1 LOADNIL R2 -MOVE R3 R1 -MOVE R4 R2 -RETURN R3 2 +RETURN R1 2 )"); // this happens even if the function returns conditionally @@ -4733,7 +4730,6 @@ local function foo(a) end local a, b = foo(false) - return a, b )", 1, 2), @@ -4741,9 +4737,7 @@ return a, b DUPCLOSURE R0 K0 LOADNIL R1 LOADNIL R2 -MOVE R3 R1 -MOVE R4 R2 -RETURN R3 2 +RETURN R1 2 )"); // note though that we can't inline a function like this in multret context @@ -4880,11 +4874,7 @@ LOADN R5 1 ADD R4 R5 R1 LOADN R5 3 ADD R6 R1 R2 -MOVE R7 R3 -MOVE R8 R4 -MOVE R9 R5 -MOVE R10 R6 -RETURN R7 4 +RETURN R3 4 )"); } @@ -5151,4 +5141,59 @@ RETURN R0 0 )"); } +TEST_CASE("ReturnConsecutive") +{ + // we can return a single local directly + CHECK_EQ("\n" + compileFunction0(R"( +local x = ... +return x +)"), + R"( +GETVARARGS R0 1 +RETURN R0 1 +)"); + + // or multiple, when they are allocated in consecutive registers + CHECK_EQ("\n" + compileFunction0(R"( +local x, y = ... +return x, y +)"), + R"( +GETVARARGS R0 2 +RETURN R0 2 +)"); + + // but not if it's an expression + CHECK_EQ("\n" + compileFunction0(R"( +local x, y = ... +return x, y + 1 +)"), + R"( +GETVARARGS R0 2 +MOVE R2 R0 +ADDK R3 R1 K0 +RETURN R2 2 +)"); + + // or a local with wrong register number + CHECK_EQ("\n" + compileFunction0(R"( +local x, y = ... +return y, x +)"), + R"( +GETVARARGS R0 2 +MOVE R2 R1 +MOVE R3 R0 +RETURN R2 2 +)"); + + // also double check the optimization doesn't trip on no-argument return (these are rare) + CHECK_EQ("\n" + compileFunction0(R"( +return +)"), + R"( +RETURN R0 0 +)"); +} + TEST_SUITE_END(); diff --git a/tests/CostModel.test.cpp b/tests/CostModel.test.cpp index aa5b7284..2fa0659b 100644 --- a/tests/CostModel.test.cpp +++ b/tests/CostModel.test.cpp @@ -76,9 +76,9 @@ end const bool args1[] = {false}; const bool args2[] = {true}; - // loop baseline cost is 2 - CHECK_EQ(3, Luau::Compile::computeCost(model, args1, 1)); - CHECK_EQ(3, Luau::Compile::computeCost(model, args2, 1)); + // loop baseline cost is 5 + CHECK_EQ(6, Luau::Compile::computeCost(model, args1, 1)); + CHECK_EQ(6, Luau::Compile::computeCost(model, args2, 1)); } TEST_CASE("MutableVariable") @@ -154,8 +154,8 @@ end const bool args1[] = {false}; const bool args2[] = {true}; - CHECK_EQ(38, Luau::Compile::computeCost(model, args1, 1)); - CHECK_EQ(37, Luau::Compile::computeCost(model, args2, 1)); + CHECK_EQ(50, Luau::Compile::computeCost(model, args1, 1)); + CHECK_EQ(49, Luau::Compile::computeCost(model, args2, 1)); } TEST_CASE("Conditional") @@ -219,8 +219,8 @@ end const bool args1[] = {false}; const bool args2[] = {true}; - CHECK_EQ(4, Luau::Compile::computeCost(model, args1, 1)); - CHECK_EQ(3, Luau::Compile::computeCost(model, args2, 1)); + CHECK_EQ(7, Luau::Compile::computeCost(model, args1, 1)); + CHECK_EQ(6, Luau::Compile::computeCost(model, args2, 1)); } TEST_SUITE_END(); diff --git a/tests/Fixture.cpp b/tests/Fixture.cpp index d8b37a65..03f3e15c 100644 --- a/tests/Fixture.cpp +++ b/tests/Fixture.cpp @@ -92,10 +92,6 @@ Fixture::Fixture(bool freeze, bool prepareAutocomplete) configResolver.defaultConfig.enabledLint.warningMask = ~0ull; configResolver.defaultConfig.parseOptions.captureComments = true; - registerBuiltinTypes(frontend.typeChecker); - if (prepareAutocomplete) - registerBuiltinTypes(frontend.typeCheckerForAutocomplete); - registerTestTypes(); Luau::freeze(frontend.typeChecker.globalTypes); Luau::freeze(frontend.typeCheckerForAutocomplete.globalTypes); @@ -410,6 +406,21 @@ LoadDefinitionFileResult Fixture::loadDefinition(const std::string& source) return result; } +BuiltinsFixture::BuiltinsFixture(bool freeze, bool prepareAutocomplete) + : Fixture(freeze, prepareAutocomplete) +{ + Luau::unfreeze(frontend.typeChecker.globalTypes); + Luau::unfreeze(frontend.typeCheckerForAutocomplete.globalTypes); + + registerBuiltinTypes(frontend.typeChecker); + if (prepareAutocomplete) + registerBuiltinTypes(frontend.typeCheckerForAutocomplete); + registerTestTypes(); + + Luau::freeze(frontend.typeChecker.globalTypes); + Luau::freeze(frontend.typeCheckerForAutocomplete.globalTypes); +} + ModuleName fromString(std::string_view name) { return ModuleName(name); diff --git a/tests/Fixture.h b/tests/Fixture.h index 0d1233bf..901f7d42 100644 --- a/tests/Fixture.h +++ b/tests/Fixture.h @@ -151,6 +151,11 @@ struct Fixture LoadDefinitionFileResult loadDefinition(const std::string& source); }; +struct BuiltinsFixture : Fixture +{ + BuiltinsFixture(bool freeze = true, bool prepareAutocomplete = false); +}; + ModuleName fromString(std::string_view name); template diff --git a/tests/Frontend.test.cpp b/tests/Frontend.test.cpp index a10e8f7f..33b81be8 100644 --- a/tests/Frontend.test.cpp +++ b/tests/Frontend.test.cpp @@ -77,7 +77,7 @@ struct NaiveFileResolver : NullFileResolver } // namespace -struct FrontendFixture : Fixture +struct FrontendFixture : BuiltinsFixture { FrontendFixture() { diff --git a/tests/Linter.test.cpp b/tests/Linter.test.cpp index 6649cb7f..202aeceb 100644 --- a/tests/Linter.test.cpp +++ b/tests/Linter.test.cpp @@ -75,7 +75,7 @@ _ = 6 CHECK_EQ(result.warnings.size(), 0); } -TEST_CASE_FIXTURE(Fixture, "BuiltinGlobalWrite") +TEST_CASE_FIXTURE(BuiltinsFixture, "BuiltinGlobalWrite") { LintResult result = lint(R"( math = {} @@ -309,7 +309,7 @@ print(arg) CHECK_EQ(result.warnings[0].text, "Variable 'arg' shadows previous declaration at line 2"); } -TEST_CASE_FIXTURE(Fixture, "LocalShadowGlobal") +TEST_CASE_FIXTURE(BuiltinsFixture, "LocalShadowGlobal") { LintResult result = lint(R"( local math = math @@ -1470,7 +1470,7 @@ end CHECK_EQ(result.warnings[2].text, "Member 'Instance.DataCost' is deprecated"); } -TEST_CASE_FIXTURE(Fixture, "TableOperations") +TEST_CASE_FIXTURE(BuiltinsFixture, "TableOperations") { LintResult result = lintTyped(R"( local t = {} diff --git a/tests/Module.test.cpp b/tests/Module.test.cpp index 44cc20a7..4a999861 100644 --- a/tests/Module.test.cpp +++ b/tests/Module.test.cpp @@ -113,7 +113,7 @@ TEST_CASE_FIXTURE(Fixture, "deepClone_cyclic_table") CHECK_EQ(2, dest.typeVars.size()); // One table and one function } -TEST_CASE_FIXTURE(Fixture, "builtin_types_point_into_globalTypes_arena") +TEST_CASE_FIXTURE(BuiltinsFixture, "builtin_types_point_into_globalTypes_arena") { CheckResult result = check(R"( return {sign=math.sign} @@ -250,7 +250,7 @@ TEST_CASE_FIXTURE(Fixture, "clone_constrained_intersection") CHECK_EQ(getSingletonTypes().stringType, ctv->parts[1]); } -TEST_CASE_FIXTURE(Fixture, "clone_self_property") +TEST_CASE_FIXTURE(BuiltinsFixture, "clone_self_property") { ScopedFastFlag sff{"LuauAnyInIsOptionalIsOptional", true}; diff --git a/tests/NonstrictMode.test.cpp b/tests/NonstrictMode.test.cpp index feeaf2c2..69430b1c 100644 --- a/tests/NonstrictMode.test.cpp +++ b/tests/NonstrictMode.test.cpp @@ -13,7 +13,7 @@ using namespace Luau; TEST_SUITE_BEGIN("NonstrictModeTests"); -TEST_CASE_FIXTURE(Fixture, "function_returns_number_or_string") +TEST_CASE_FIXTURE(BuiltinsFixture, "function_returns_number_or_string") { ScopedFastFlag sff[]{ {"LuauReturnTypeInferenceInNonstrict", true}, @@ -224,7 +224,7 @@ TEST_CASE_FIXTURE(Fixture, "inline_table_props_are_also_any") CHECK_MESSAGE(get(ttv->props["three"].type), "Should be a function: " << *ttv->props["three"].type); } -TEST_CASE_FIXTURE(Fixture, "for_in_iterator_variables_are_any") +TEST_CASE_FIXTURE(BuiltinsFixture, "for_in_iterator_variables_are_any") { CheckResult result = check(R"( --!nonstrict @@ -243,7 +243,7 @@ TEST_CASE_FIXTURE(Fixture, "for_in_iterator_variables_are_any") LUAU_REQUIRE_NO_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "table_dot_insert_and_recursive_calls") +TEST_CASE_FIXTURE(BuiltinsFixture, "table_dot_insert_and_recursive_calls") { CheckResult result = check(R"( --!nonstrict diff --git a/tests/Normalize.test.cpp b/tests/Normalize.test.cpp index d3778f67..41830682 100644 --- a/tests/Normalize.test.cpp +++ b/tests/Normalize.test.cpp @@ -739,7 +739,7 @@ TEST_CASE_FIXTURE(Fixture, "cyclic_table_normalizes_sensibly") CHECK_EQ("t1 where t1 = { get: () -> t1 }", toString(ty, {true})); } -TEST_CASE_FIXTURE(Fixture, "union_of_distinct_free_types") +TEST_CASE_FIXTURE(BuiltinsFixture, "union_of_distinct_free_types") { ScopedFastFlag flags[] = { {"LuauLowerBoundsCalculation", true}, @@ -760,7 +760,7 @@ TEST_CASE_FIXTURE(Fixture, "union_of_distinct_free_types") CHECK("(a, b) -> a | b" == toString(requireType("fussy"))); } -TEST_CASE_FIXTURE(Fixture, "constrained_intersection_of_intersections") +TEST_CASE_FIXTURE(BuiltinsFixture, "constrained_intersection_of_intersections") { ScopedFastFlag flags[] = { {"LuauLowerBoundsCalculation", true}, @@ -951,7 +951,7 @@ TEST_CASE_FIXTURE(Fixture, "nested_table_normalization_with_non_table__no_ice") LUAU_REQUIRE_NO_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "visiting_a_type_twice_is_not_considered_normal") +TEST_CASE_FIXTURE(BuiltinsFixture, "visiting_a_type_twice_is_not_considered_normal") { ScopedFastFlag sff{"LuauLowerBoundsCalculation", true}; @@ -976,7 +976,6 @@ TEST_CASE_FIXTURE(Fixture, "fuzz_failure_instersection_combine_must_follow") { ScopedFastFlag flags[] = { {"LuauLowerBoundsCalculation", true}, - {"LuauNormalizeCombineIntersectionFix", true}, }; CheckResult result = check(R"( diff --git a/tests/Parser.test.cpp b/tests/Parser.test.cpp index 69ff73ad..c9d8d0b8 100644 --- a/tests/Parser.test.cpp +++ b/tests/Parser.test.cpp @@ -1618,8 +1618,6 @@ TEST_CASE_FIXTURE(Fixture, "end_extent_doesnt_consume_comments") TEST_CASE_FIXTURE(Fixture, "end_extent_doesnt_consume_comments_even_with_capture") { - ScopedFastFlag luauParseLocationIgnoreCommentSkipInCapture{"LuauParseLocationIgnoreCommentSkipInCapture", true}; - // Same should hold when comments are captured ParseOptions opts; opts.captureComments = true; diff --git a/tests/RuntimeLimits.test.cpp b/tests/RuntimeLimits.test.cpp index 538f3576..c16f60d5 100644 --- a/tests/RuntimeLimits.test.cpp +++ b/tests/RuntimeLimits.test.cpp @@ -17,7 +17,7 @@ using namespace Luau; LUAU_FASTFLAG(LuauLowerBoundsCalculation); -struct LimitFixture : Fixture +struct LimitFixture : BuiltinsFixture { #if defined(_NOOPT) || defined(_DEBUG) ScopedFastInt LuauTypeInferRecursionLimit{"LuauTypeInferRecursionLimit", 100}; diff --git a/tests/ToDot.test.cpp b/tests/ToDot.test.cpp index 332a4b22..e9fa5b26 100644 --- a/tests/ToDot.test.cpp +++ b/tests/ToDot.test.cpp @@ -224,7 +224,7 @@ n1 -> n4 [label="typePackParam"]; (void)toDot(requireType("a")); } -TEST_CASE_FIXTURE(Fixture, "metatable") +TEST_CASE_FIXTURE(BuiltinsFixture, "metatable") { CheckResult result = check(R"( local a: typeof(setmetatable({}, {})) diff --git a/tests/ToString.test.cpp b/tests/ToString.test.cpp index ccf5c583..50d0838e 100644 --- a/tests/ToString.test.cpp +++ b/tests/ToString.test.cpp @@ -60,7 +60,7 @@ TEST_CASE_FIXTURE(Fixture, "named_table") CHECK_EQ("TheTable", toString(&table)); } -TEST_CASE_FIXTURE(Fixture, "exhaustive_toString_of_cyclic_table") +TEST_CASE_FIXTURE(BuiltinsFixture, "exhaustive_toString_of_cyclic_table") { CheckResult result = check(R"( --!strict @@ -338,7 +338,7 @@ TEST_CASE_FIXTURE(Fixture, "toStringDetailed") REQUIRE_EQ("c", toString(params[2], opts)); } -TEST_CASE_FIXTURE(Fixture, "toStringDetailed2") +TEST_CASE_FIXTURE(BuiltinsFixture, "toStringDetailed2") { ScopedFastFlag sff{"LuauUnsealedTableLiteral", true}; diff --git a/tests/TypeInfer.aliases.test.cpp b/tests/TypeInfer.aliases.test.cpp index b0eb31ce..7562a4d7 100644 --- a/tests/TypeInfer.aliases.test.cpp +++ b/tests/TypeInfer.aliases.test.cpp @@ -279,7 +279,7 @@ TEST_CASE_FIXTURE(Fixture, "stringify_optional_parameterized_alias") CHECK_EQ("Node", toString(e->wantedType)); } -TEST_CASE_FIXTURE(Fixture, "general_require_multi_assign") +TEST_CASE_FIXTURE(BuiltinsFixture, "general_require_multi_assign") { fileResolver.source["workspace/A"] = R"( export type myvec2 = {x: number, y: number} @@ -317,7 +317,7 @@ TEST_CASE_FIXTURE(Fixture, "general_require_multi_assign") REQUIRE(bType->props.size() == 3); } -TEST_CASE_FIXTURE(Fixture, "type_alias_import_mutation") +TEST_CASE_FIXTURE(BuiltinsFixture, "type_alias_import_mutation") { CheckResult result = check("type t10 = typeof(table)"); LUAU_REQUIRE_NO_ERRORS(result); @@ -385,7 +385,7 @@ type Cool = typeof(c) CHECK_EQ(ttv->name, "Cool"); } -TEST_CASE_FIXTURE(Fixture, "type_alias_of_an_imported_recursive_type") +TEST_CASE_FIXTURE(BuiltinsFixture, "type_alias_of_an_imported_recursive_type") { fileResolver.source["game/A"] = R"( export type X = { a: number, b: X? } @@ -410,7 +410,7 @@ type X = Import.X CHECK_EQ(follow(*ty1), follow(*ty2)); } -TEST_CASE_FIXTURE(Fixture, "type_alias_of_an_imported_recursive_generic_type") +TEST_CASE_FIXTURE(BuiltinsFixture, "type_alias_of_an_imported_recursive_generic_type") { fileResolver.source["game/A"] = R"( export type X = { a: T, b: U, C: X? } @@ -564,7 +564,7 @@ TEST_CASE_FIXTURE(Fixture, "non_recursive_aliases_that_reuse_a_generic_name") * * We solved this by ascribing a unique subLevel to each prototyped alias. */ -TEST_CASE_FIXTURE(Fixture, "do_not_quantify_unresolved_aliases") +TEST_CASE_FIXTURE(BuiltinsFixture, "do_not_quantify_unresolved_aliases") { CheckResult result = check(R"( --!strict diff --git a/tests/TypeInfer.annotations.test.cpp b/tests/TypeInfer.annotations.test.cpp index 7f1c757a..b9e1ae96 100644 --- a/tests/TypeInfer.annotations.test.cpp +++ b/tests/TypeInfer.annotations.test.cpp @@ -528,7 +528,7 @@ TEST_CASE_FIXTURE(Fixture, "cloned_interface_maintains_pointers_between_definiti CHECK_EQ(recordType, bType); } -TEST_CASE_FIXTURE(Fixture, "use_type_required_from_another_file") +TEST_CASE_FIXTURE(BuiltinsFixture, "use_type_required_from_another_file") { addGlobalBinding(frontend.typeChecker, "script", frontend.typeChecker.anyType, "@test"); @@ -554,7 +554,7 @@ TEST_CASE_FIXTURE(Fixture, "use_type_required_from_another_file") LUAU_REQUIRE_NO_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "cannot_use_nonexported_type") +TEST_CASE_FIXTURE(BuiltinsFixture, "cannot_use_nonexported_type") { addGlobalBinding(frontend.typeChecker, "script", frontend.typeChecker.anyType, "@test"); @@ -580,7 +580,7 @@ TEST_CASE_FIXTURE(Fixture, "cannot_use_nonexported_type") LUAU_REQUIRE_ERROR_COUNT(1, result); } -TEST_CASE_FIXTURE(Fixture, "builtin_types_are_not_exported") +TEST_CASE_FIXTURE(BuiltinsFixture, "builtin_types_are_not_exported") { addGlobalBinding(frontend.typeChecker, "script", frontend.typeChecker.anyType, "@test"); @@ -676,7 +676,7 @@ TEST_CASE_FIXTURE(Fixture, "luau_ice_is_not_special_without_the_flag") )"); } -TEST_CASE_FIXTURE(Fixture, "luau_print_is_magic_if_the_flag_is_set") +TEST_CASE_FIXTURE(BuiltinsFixture, "luau_print_is_magic_if_the_flag_is_set") { // Luau::resetPrintLine(); ScopedFastFlag sffs{"DebugLuauMagicTypes", true}; diff --git a/tests/TypeInfer.anyerror.test.cpp b/tests/TypeInfer.anyerror.test.cpp index 5224b5d8..bc55940e 100644 --- a/tests/TypeInfer.anyerror.test.cpp +++ b/tests/TypeInfer.anyerror.test.cpp @@ -237,7 +237,7 @@ TEST_CASE_FIXTURE(Fixture, "chain_calling_error_type_yields_error") CHECK_EQ("*unknown*", toString(requireType("a"))); } -TEST_CASE_FIXTURE(Fixture, "replace_every_free_type_when_unifying_a_complex_function_with_any") +TEST_CASE_FIXTURE(BuiltinsFixture, "replace_every_free_type_when_unifying_a_complex_function_with_any") { CheckResult result = check(R"( local a: any @@ -285,7 +285,7 @@ end LUAU_REQUIRE_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "metatable_of_any_can_be_a_table") +TEST_CASE_FIXTURE(BuiltinsFixture, "metatable_of_any_can_be_a_table") { CheckResult result = check(R"( --!strict diff --git a/tests/TypeInfer.builtins.test.cpp b/tests/TypeInfer.builtins.test.cpp index 1ae65947..b710ea0d 100644 --- a/tests/TypeInfer.builtins.test.cpp +++ b/tests/TypeInfer.builtins.test.cpp @@ -12,7 +12,7 @@ LUAU_FASTFLAG(LuauLowerBoundsCalculation); TEST_SUITE_BEGIN("BuiltinTests"); -TEST_CASE_FIXTURE(Fixture, "math_things_are_defined") +TEST_CASE_FIXTURE(BuiltinsFixture, "math_things_are_defined") { CheckResult result = check(R"( local a00 = math.frexp @@ -50,7 +50,7 @@ TEST_CASE_FIXTURE(Fixture, "math_things_are_defined") LUAU_REQUIRE_NO_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "next_iterator_should_infer_types_and_type_check") +TEST_CASE_FIXTURE(BuiltinsFixture, "next_iterator_should_infer_types_and_type_check") { CheckResult result = check(R"( local a: string, b: number = next({ 1 }) @@ -63,7 +63,7 @@ TEST_CASE_FIXTURE(Fixture, "next_iterator_should_infer_types_and_type_check") LUAU_REQUIRE_ERROR_COUNT(1, result); } -TEST_CASE_FIXTURE(Fixture, "pairs_iterator_should_infer_types_and_type_check") +TEST_CASE_FIXTURE(BuiltinsFixture, "pairs_iterator_should_infer_types_and_type_check") { CheckResult result = check(R"( type Map = { [K]: V } @@ -75,7 +75,7 @@ TEST_CASE_FIXTURE(Fixture, "pairs_iterator_should_infer_types_and_type_check") LUAU_REQUIRE_NO_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "ipairs_iterator_should_infer_types_and_type_check") +TEST_CASE_FIXTURE(BuiltinsFixture, "ipairs_iterator_should_infer_types_and_type_check") { CheckResult result = check(R"( type Map = { [K]: V } @@ -87,7 +87,7 @@ TEST_CASE_FIXTURE(Fixture, "ipairs_iterator_should_infer_types_and_type_check") LUAU_REQUIRE_NO_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "table_dot_remove_optionally_returns_generic") +TEST_CASE_FIXTURE(BuiltinsFixture, "table_dot_remove_optionally_returns_generic") { CheckResult result = check(R"( local t = { 1 } @@ -98,7 +98,7 @@ TEST_CASE_FIXTURE(Fixture, "table_dot_remove_optionally_returns_generic") CHECK_EQ(toString(requireType("n")), "number?"); } -TEST_CASE_FIXTURE(Fixture, "table_concat_returns_string") +TEST_CASE_FIXTURE(BuiltinsFixture, "table_concat_returns_string") { CheckResult result = check(R"( local r = table.concat({1,2,3,4}, ",", 2); @@ -108,7 +108,7 @@ TEST_CASE_FIXTURE(Fixture, "table_concat_returns_string") CHECK_EQ(*typeChecker.stringType, *requireType("r")); } -TEST_CASE_FIXTURE(Fixture, "sort") +TEST_CASE_FIXTURE(BuiltinsFixture, "sort") { CheckResult result = check(R"( local t = {1, 2, 3}; @@ -118,7 +118,7 @@ TEST_CASE_FIXTURE(Fixture, "sort") LUAU_REQUIRE_NO_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "sort_with_predicate") +TEST_CASE_FIXTURE(BuiltinsFixture, "sort_with_predicate") { CheckResult result = check(R"( --!strict @@ -130,7 +130,7 @@ TEST_CASE_FIXTURE(Fixture, "sort_with_predicate") LUAU_REQUIRE_NO_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "sort_with_bad_predicate") +TEST_CASE_FIXTURE(BuiltinsFixture, "sort_with_bad_predicate") { CheckResult result = check(R"( --!strict @@ -140,6 +140,12 @@ TEST_CASE_FIXTURE(Fixture, "sort_with_bad_predicate") )"); LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ(R"(Type '(number, number) -> boolean' could not be converted into '((a, a) -> boolean)?' +caused by: + None of the union options are compatible. For example: Type '(number, number) -> boolean' could not be converted into '(a, a) -> boolean' +caused by: + Argument #1 type is not compatible. Type 'string' could not be converted into 'number')", + toString(result.errors[0])); } TEST_CASE_FIXTURE(Fixture, "strings_have_methods") @@ -152,7 +158,7 @@ TEST_CASE_FIXTURE(Fixture, "strings_have_methods") CHECK_EQ(*typeChecker.stringType, *requireType("s")); } -TEST_CASE_FIXTURE(Fixture, "math_max_variatic") +TEST_CASE_FIXTURE(BuiltinsFixture, "math_max_variatic") { CheckResult result = check(R"( local n = math.max(1,2,3,4,5,6,7,8,9,0) @@ -162,16 +168,17 @@ TEST_CASE_FIXTURE(Fixture, "math_max_variatic") CHECK_EQ(*typeChecker.numberType, *requireType("n")); } -TEST_CASE_FIXTURE(Fixture, "math_max_checks_for_numbers") +TEST_CASE_FIXTURE(BuiltinsFixture, "math_max_checks_for_numbers") { CheckResult result = check(R"( local n = math.max(1,2,"3") )"); CHECK(!result.errors.empty()); + CHECK_EQ("Type 'string' could not be converted into 'number'", toString(result.errors[0])); } -TEST_CASE_FIXTURE(Fixture, "builtin_tables_sealed") +TEST_CASE_FIXTURE(BuiltinsFixture, "builtin_tables_sealed") { CheckResult result = check(R"LUA( local b = bit32 @@ -183,7 +190,7 @@ TEST_CASE_FIXTURE(Fixture, "builtin_tables_sealed") CHECK_EQ(bit32t->state, TableState::Sealed); } -TEST_CASE_FIXTURE(Fixture, "lua_51_exported_globals_all_exist") +TEST_CASE_FIXTURE(BuiltinsFixture, "lua_51_exported_globals_all_exist") { // Extracted from lua5.1 CheckResult result = check(R"( @@ -340,7 +347,7 @@ TEST_CASE_FIXTURE(Fixture, "lua_51_exported_globals_all_exist") LUAU_REQUIRE_NO_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "setmetatable_unpacks_arg_types_correctly") +TEST_CASE_FIXTURE(BuiltinsFixture, "setmetatable_unpacks_arg_types_correctly") { CheckResult result = check(R"( setmetatable({}, setmetatable({}, {})) @@ -348,7 +355,7 @@ TEST_CASE_FIXTURE(Fixture, "setmetatable_unpacks_arg_types_correctly") LUAU_REQUIRE_NO_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "table_insert_correctly_infers_type_of_array_2_args_overload") +TEST_CASE_FIXTURE(BuiltinsFixture, "table_insert_correctly_infers_type_of_array_2_args_overload") { CheckResult result = check(R"( local t = {} @@ -360,7 +367,7 @@ TEST_CASE_FIXTURE(Fixture, "table_insert_correctly_infers_type_of_array_2_args_o CHECK_EQ(typeChecker.stringType, requireType("s")); } -TEST_CASE_FIXTURE(Fixture, "table_insert_correctly_infers_type_of_array_3_args_overload") +TEST_CASE_FIXTURE(BuiltinsFixture, "table_insert_correctly_infers_type_of_array_3_args_overload") { CheckResult result = check(R"( local t = {} @@ -372,7 +379,7 @@ TEST_CASE_FIXTURE(Fixture, "table_insert_correctly_infers_type_of_array_3_args_o CHECK_EQ("string", toString(requireType("s"))); } -TEST_CASE_FIXTURE(Fixture, "table_pack") +TEST_CASE_FIXTURE(BuiltinsFixture, "table_pack") { CheckResult result = check(R"( local t = table.pack(1, "foo", true) @@ -382,7 +389,7 @@ TEST_CASE_FIXTURE(Fixture, "table_pack") CHECK_EQ("{| [number]: boolean | number | string, n: number |}", toString(requireType("t"))); } -TEST_CASE_FIXTURE(Fixture, "table_pack_variadic") +TEST_CASE_FIXTURE(BuiltinsFixture, "table_pack_variadic") { CheckResult result = check(R"( --!strict @@ -397,7 +404,7 @@ local t = table.pack(f()) CHECK_EQ("{| [number]: number | string, n: number |}", toString(requireType("t"))); } -TEST_CASE_FIXTURE(Fixture, "table_pack_reduce") +TEST_CASE_FIXTURE(BuiltinsFixture, "table_pack_reduce") { CheckResult result = check(R"( local t = table.pack(1, 2, true) @@ -414,7 +421,7 @@ TEST_CASE_FIXTURE(Fixture, "table_pack_reduce") CHECK_EQ("{| [number]: string, n: number |}", toString(requireType("t"))); } -TEST_CASE_FIXTURE(Fixture, "gcinfo") +TEST_CASE_FIXTURE(BuiltinsFixture, "gcinfo") { CheckResult result = check(R"( local n = gcinfo() @@ -424,12 +431,12 @@ TEST_CASE_FIXTURE(Fixture, "gcinfo") CHECK_EQ(*typeChecker.numberType, *requireType("n")); } -TEST_CASE_FIXTURE(Fixture, "getfenv") +TEST_CASE_FIXTURE(BuiltinsFixture, "getfenv") { LUAU_REQUIRE_NO_ERRORS(check("getfenv(1)")); } -TEST_CASE_FIXTURE(Fixture, "os_time_takes_optional_date_table") +TEST_CASE_FIXTURE(BuiltinsFixture, "os_time_takes_optional_date_table") { CheckResult result = check(R"( local n1 = os.time() @@ -443,7 +450,7 @@ TEST_CASE_FIXTURE(Fixture, "os_time_takes_optional_date_table") CHECK_EQ(*typeChecker.numberType, *requireType("n3")); } -TEST_CASE_FIXTURE(Fixture, "thread_is_a_type") +TEST_CASE_FIXTURE(BuiltinsFixture, "thread_is_a_type") { CheckResult result = check(R"( local co = coroutine.create(function() end) @@ -453,7 +460,7 @@ TEST_CASE_FIXTURE(Fixture, "thread_is_a_type") CHECK_EQ(*typeChecker.threadType, *requireType("co")); } -TEST_CASE_FIXTURE(Fixture, "coroutine_resume_anything_goes") +TEST_CASE_FIXTURE(BuiltinsFixture, "coroutine_resume_anything_goes") { CheckResult result = check(R"( local function nifty(x, y) @@ -471,7 +478,7 @@ TEST_CASE_FIXTURE(Fixture, "coroutine_resume_anything_goes") LUAU_REQUIRE_NO_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "coroutine_wrap_anything_goes") +TEST_CASE_FIXTURE(BuiltinsFixture, "coroutine_wrap_anything_goes") { CheckResult result = check(R"( --!nonstrict @@ -490,7 +497,7 @@ TEST_CASE_FIXTURE(Fixture, "coroutine_wrap_anything_goes") LUAU_REQUIRE_NO_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "setmetatable_should_not_mutate_persisted_types") +TEST_CASE_FIXTURE(BuiltinsFixture, "setmetatable_should_not_mutate_persisted_types") { CheckResult result = check(R"( local string = string @@ -505,7 +512,7 @@ TEST_CASE_FIXTURE(Fixture, "setmetatable_should_not_mutate_persisted_types") REQUIRE(ttv); } -TEST_CASE_FIXTURE(Fixture, "string_format_arg_types_inference") +TEST_CASE_FIXTURE(BuiltinsFixture, "string_format_arg_types_inference") { CheckResult result = check(R"( --!strict @@ -518,7 +525,7 @@ TEST_CASE_FIXTURE(Fixture, "string_format_arg_types_inference") CHECK_EQ("(number, number, string) -> string", toString(requireType("f"))); } -TEST_CASE_FIXTURE(Fixture, "string_format_arg_count_mismatch") +TEST_CASE_FIXTURE(BuiltinsFixture, "string_format_arg_count_mismatch") { CheckResult result = check(R"( --!strict @@ -534,7 +541,7 @@ TEST_CASE_FIXTURE(Fixture, "string_format_arg_count_mismatch") CHECK_EQ(result.errors[2].location.begin.line, 4); } -TEST_CASE_FIXTURE(Fixture, "string_format_correctly_ordered_types") +TEST_CASE_FIXTURE(BuiltinsFixture, "string_format_correctly_ordered_types") { CheckResult result = check(R"( --!strict @@ -548,7 +555,7 @@ TEST_CASE_FIXTURE(Fixture, "string_format_correctly_ordered_types") CHECK_EQ(tm->givenType, typeChecker.numberType); } -TEST_CASE_FIXTURE(Fixture, "xpcall") +TEST_CASE_FIXTURE(BuiltinsFixture, "xpcall") { CheckResult result = check(R"( --!strict @@ -564,7 +571,7 @@ TEST_CASE_FIXTURE(Fixture, "xpcall") CHECK_EQ("boolean", toString(requireType("c"))); } -TEST_CASE_FIXTURE(Fixture, "see_thru_select") +TEST_CASE_FIXTURE(BuiltinsFixture, "see_thru_select") { CheckResult result = check(R"( local a:number, b:boolean = select(2,"hi", 10, true) @@ -573,7 +580,7 @@ TEST_CASE_FIXTURE(Fixture, "see_thru_select") LUAU_REQUIRE_NO_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "see_thru_select_count") +TEST_CASE_FIXTURE(BuiltinsFixture, "see_thru_select_count") { CheckResult result = check(R"( local a = select("#","hi", 10, true) @@ -583,7 +590,7 @@ TEST_CASE_FIXTURE(Fixture, "see_thru_select_count") LUAU_REQUIRE_NO_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "select_with_decimal_argument_is_rounded_down") +TEST_CASE_FIXTURE(BuiltinsFixture, "select_with_decimal_argument_is_rounded_down") { CheckResult result = check(R"( local a: number, b: boolean = select(2.9, "foo", 1, true) @@ -608,7 +615,7 @@ TEST_CASE_FIXTURE(Fixture, "bad_select_should_not_crash") CHECK_LE(0, result.errors.size()); } -TEST_CASE_FIXTURE(Fixture, "select_way_out_of_range") +TEST_CASE_FIXTURE(BuiltinsFixture, "select_way_out_of_range") { CheckResult result = check(R"( select(5432598430953240958) @@ -619,7 +626,7 @@ TEST_CASE_FIXTURE(Fixture, "select_way_out_of_range") CHECK_EQ("bad argument #1 to select (index out of range)", toString(result.errors[0])); } -TEST_CASE_FIXTURE(Fixture, "select_slightly_out_of_range") +TEST_CASE_FIXTURE(BuiltinsFixture, "select_slightly_out_of_range") { CheckResult result = check(R"( select(3, "a", 1) @@ -630,7 +637,7 @@ TEST_CASE_FIXTURE(Fixture, "select_slightly_out_of_range") CHECK_EQ("bad argument #1 to select (index out of range)", toString(result.errors[0])); } -TEST_CASE_FIXTURE(Fixture, "select_with_variadic_typepack_tail") +TEST_CASE_FIXTURE(BuiltinsFixture, "select_with_variadic_typepack_tail") { CheckResult result = check(R"( --!nonstrict @@ -649,7 +656,7 @@ TEST_CASE_FIXTURE(Fixture, "select_with_variadic_typepack_tail") CHECK_EQ("any", toString(requireType("quux"))); } -TEST_CASE_FIXTURE(Fixture, "select_with_variadic_typepack_tail_and_string_head") +TEST_CASE_FIXTURE(BuiltinsFixture, "select_with_variadic_typepack_tail_and_string_head") { CheckResult result = check(R"( --!nonstrict @@ -703,7 +710,7 @@ TEST_CASE_FIXTURE(Fixture, "string_format_use_correct_argument2") CHECK_EQ("Type 'number' could not be converted into 'string'", toString(result.errors[1])); } -TEST_CASE_FIXTURE(Fixture, "debug_traceback_is_crazy") +TEST_CASE_FIXTURE(BuiltinsFixture, "debug_traceback_is_crazy") { CheckResult result = check(R"( local co: thread = ... @@ -720,7 +727,7 @@ debug.traceback(co, "msg", 1) LUAU_REQUIRE_NO_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "debug_info_is_crazy") +TEST_CASE_FIXTURE(BuiltinsFixture, "debug_info_is_crazy") { CheckResult result = check(R"( local co: thread, f: ()->() = ... @@ -734,7 +741,7 @@ debug.info(f, "n") LUAU_REQUIRE_NO_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "aliased_string_format") +TEST_CASE_FIXTURE(BuiltinsFixture, "aliased_string_format") { CheckResult result = check(R"( local fmt = string.format @@ -745,7 +752,7 @@ TEST_CASE_FIXTURE(Fixture, "aliased_string_format") CHECK_EQ("Type 'string' could not be converted into 'number'", toString(result.errors[0])); } -TEST_CASE_FIXTURE(Fixture, "string_lib_self_noself") +TEST_CASE_FIXTURE(BuiltinsFixture, "string_lib_self_noself") { CheckResult result = check(R"( --!nonstrict @@ -764,7 +771,7 @@ TEST_CASE_FIXTURE(Fixture, "string_lib_self_noself") LUAU_REQUIRE_NO_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "gmatch_definition") +TEST_CASE_FIXTURE(BuiltinsFixture, "gmatch_definition") { CheckResult result = check(R"_( local a, b, c = ("hey"):gmatch("(.)(.)(.)")() @@ -777,7 +784,7 @@ end LUAU_REQUIRE_NO_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "select_on_variadic") +TEST_CASE_FIXTURE(BuiltinsFixture, "select_on_variadic") { CheckResult result = check(R"( local function f(): (number, ...(boolean | number)) @@ -793,7 +800,7 @@ TEST_CASE_FIXTURE(Fixture, "select_on_variadic") CHECK_EQ("any", toString(requireType("c"))); } -TEST_CASE_FIXTURE(Fixture, "string_format_report_all_type_errors_at_correct_positions") +TEST_CASE_FIXTURE(BuiltinsFixture, "string_format_report_all_type_errors_at_correct_positions") { CheckResult result = check(R"( ("%s%d%s"):format(1, "hello", true) @@ -825,7 +832,7 @@ TEST_CASE_FIXTURE(Fixture, "string_format_report_all_type_errors_at_correct_posi CHECK_EQ(TypeErrorData(TypeMismatch{stringType, booleanType}), result.errors[5].data); } -TEST_CASE_FIXTURE(Fixture, "tonumber_returns_optional_number_type") +TEST_CASE_FIXTURE(BuiltinsFixture, "tonumber_returns_optional_number_type") { CheckResult result = check(R"( --!strict @@ -836,7 +843,7 @@ TEST_CASE_FIXTURE(Fixture, "tonumber_returns_optional_number_type") CHECK_EQ("Type 'number?' could not be converted into 'number'", toString(result.errors[0])); } -TEST_CASE_FIXTURE(Fixture, "tonumber_returns_optional_number_type2") +TEST_CASE_FIXTURE(BuiltinsFixture, "tonumber_returns_optional_number_type2") { CheckResult result = check(R"( --!strict @@ -846,7 +853,7 @@ TEST_CASE_FIXTURE(Fixture, "tonumber_returns_optional_number_type2") LUAU_REQUIRE_NO_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "dont_add_definitions_to_persistent_types") +TEST_CASE_FIXTURE(BuiltinsFixture, "dont_add_definitions_to_persistent_types") { CheckResult result = check(R"( local f = math.sin @@ -868,7 +875,7 @@ TEST_CASE_FIXTURE(Fixture, "dont_add_definitions_to_persistent_types") REQUIRE(gtv->definition); } -TEST_CASE_FIXTURE(Fixture, "assert_removes_falsy_types") +TEST_CASE_FIXTURE(BuiltinsFixture, "assert_removes_falsy_types") { ScopedFastFlag sff[]{ {"LuauAssertStripsFalsyTypes", true}, @@ -889,7 +896,7 @@ TEST_CASE_FIXTURE(Fixture, "assert_removes_falsy_types") CHECK_EQ("((boolean | number)?) -> boolean | number", toString(requireType("f"))); } -TEST_CASE_FIXTURE(Fixture, "assert_removes_falsy_types2") +TEST_CASE_FIXTURE(BuiltinsFixture, "assert_removes_falsy_types2") { ScopedFastFlag sff[]{ {"LuauAssertStripsFalsyTypes", true}, @@ -907,7 +914,7 @@ TEST_CASE_FIXTURE(Fixture, "assert_removes_falsy_types2") CHECK_EQ("((boolean | number)?) -> number | true", toString(requireType("f"))); } -TEST_CASE_FIXTURE(Fixture, "assert_removes_falsy_types_even_from_type_pack_tail_but_only_for_the_first_type") +TEST_CASE_FIXTURE(BuiltinsFixture, "assert_removes_falsy_types_even_from_type_pack_tail_but_only_for_the_first_type") { ScopedFastFlag sff[]{ {"LuauAssertStripsFalsyTypes", true}, @@ -924,7 +931,7 @@ TEST_CASE_FIXTURE(Fixture, "assert_removes_falsy_types_even_from_type_pack_tail_ CHECK_EQ("(...number?) -> (number, ...number?)", toString(requireType("f"))); } -TEST_CASE_FIXTURE(Fixture, "assert_returns_false_and_string_iff_it_knows_the_first_argument_cannot_be_truthy") +TEST_CASE_FIXTURE(BuiltinsFixture, "assert_returns_false_and_string_iff_it_knows_the_first_argument_cannot_be_truthy") { ScopedFastFlag sff[]{ {"LuauAssertStripsFalsyTypes", true}, @@ -941,7 +948,7 @@ TEST_CASE_FIXTURE(Fixture, "assert_returns_false_and_string_iff_it_knows_the_fir CHECK_EQ("(nil) -> nil", toString(requireType("f"))); } -TEST_CASE_FIXTURE(Fixture, "table_freeze_is_generic") +TEST_CASE_FIXTURE(BuiltinsFixture, "table_freeze_is_generic") { CheckResult result = check(R"( local t1: {a: number} = {a = 42} @@ -968,7 +975,7 @@ TEST_CASE_FIXTURE(Fixture, "table_freeze_is_generic") CHECK_EQ("*unknown*", toString(requireType("d"))); } -TEST_CASE_FIXTURE(Fixture, "set_metatable_needs_arguments") +TEST_CASE_FIXTURE(BuiltinsFixture, "set_metatable_needs_arguments") { ScopedFastFlag sff{"LuauSetMetaTableArgsCheck", true}; CheckResult result = check(R"( @@ -991,7 +998,7 @@ local function f(a: typeof(f)) end CHECK_EQ("Unknown global 'f'", toString(result.errors[0])); } -TEST_CASE_FIXTURE(Fixture, "no_persistent_typelevel_change") +TEST_CASE_FIXTURE(BuiltinsFixture, "no_persistent_typelevel_change") { TypeId mathTy = requireType(typeChecker.globalScope, "math"); REQUIRE(mathTy); @@ -1008,7 +1015,7 @@ TEST_CASE_FIXTURE(Fixture, "no_persistent_typelevel_change") CHECK(ftv->level.subLevel == original.subLevel); } -TEST_CASE_FIXTURE(Fixture, "global_singleton_types_are_sealed") +TEST_CASE_FIXTURE(BuiltinsFixture, "global_singleton_types_are_sealed") { CheckResult result = check(R"( local function f(x: string) diff --git a/tests/TypeInfer.classes.test.cpp b/tests/TypeInfer.classes.test.cpp index 5a6e4032..d90129d7 100644 --- a/tests/TypeInfer.classes.test.cpp +++ b/tests/TypeInfer.classes.test.cpp @@ -10,7 +10,7 @@ using namespace Luau; using std::nullopt; -struct ClassFixture : Fixture +struct ClassFixture : BuiltinsFixture { ClassFixture() { diff --git a/tests/TypeInfer.functions.test.cpp b/tests/TypeInfer.functions.test.cpp index 0e071217..14f1f703 100644 --- a/tests/TypeInfer.functions.test.cpp +++ b/tests/TypeInfer.functions.test.cpp @@ -85,7 +85,7 @@ TEST_CASE_FIXTURE(Fixture, "vararg_functions_should_allow_calls_of_any_types_and LUAU_REQUIRE_NO_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "vararg_function_is_quantified") +TEST_CASE_FIXTURE(BuiltinsFixture, "vararg_function_is_quantified") { CheckResult result = check(R"( local T = {} @@ -555,7 +555,7 @@ TEST_CASE_FIXTURE(Fixture, "higher_order_function_3") CHECK(bool(argType->indexer)); } -TEST_CASE_FIXTURE(Fixture, "higher_order_function_4") +TEST_CASE_FIXTURE(BuiltinsFixture, "higher_order_function_4") { CheckResult result = check(R"( function bottomupmerge(comp, a, b, left, mid, right) @@ -620,7 +620,7 @@ TEST_CASE_FIXTURE(Fixture, "higher_order_function_4") CHECK_EQ(*arg0->indexer->indexResultType, *arg1Args[1]); } -TEST_CASE_FIXTURE(Fixture, "mutual_recursion") +TEST_CASE_FIXTURE(BuiltinsFixture, "mutual_recursion") { CheckResult result = check(R"( --!strict @@ -639,7 +639,7 @@ TEST_CASE_FIXTURE(Fixture, "mutual_recursion") dumpErrors(result); } -TEST_CASE_FIXTURE(Fixture, "toposort_doesnt_break_mutual_recursion") +TEST_CASE_FIXTURE(BuiltinsFixture, "toposort_doesnt_break_mutual_recursion") { CheckResult result = check(R"( --!strict @@ -676,7 +676,7 @@ TEST_CASE_FIXTURE(Fixture, "check_function_before_lambda_that_uses_it") LUAU_REQUIRE_NO_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "it_is_ok_to_oversaturate_a_higher_order_function_argument") +TEST_CASE_FIXTURE(BuiltinsFixture, "it_is_ok_to_oversaturate_a_higher_order_function_argument") { CheckResult result = check(R"( function onerror() end @@ -794,7 +794,7 @@ TEST_CASE_FIXTURE(Fixture, "calling_function_with_incorrect_argument_type_yields }})); } -TEST_CASE_FIXTURE(Fixture, "calling_function_with_anytypepack_doesnt_leak_free_types") +TEST_CASE_FIXTURE(BuiltinsFixture, "calling_function_with_anytypepack_doesnt_leak_free_types") { ScopedFastFlag sff[]{ {"LuauReturnTypeInferenceInNonstrict", true}, @@ -966,7 +966,7 @@ TEST_CASE_FIXTURE(Fixture, "return_type_by_overload") CHECK_EQ("string", toString(requireType("z"))); } -TEST_CASE_FIXTURE(Fixture, "infer_anonymous_function_arguments") +TEST_CASE_FIXTURE(BuiltinsFixture, "infer_anonymous_function_arguments") { // Simple direct arg to arg propagation CheckResult result = check(R"( @@ -1068,7 +1068,7 @@ f(function(x) return x * 2 end) } } -TEST_CASE_FIXTURE(Fixture, "infer_anonymous_function_arguments") +TEST_CASE_FIXTURE(BuiltinsFixture, "infer_anonymous_function_arguments") { // Simple direct arg to arg propagation CheckResult result = check(R"( @@ -1287,10 +1287,8 @@ caused by: Return #2 type is not compatible. Type 'string' could not be converted into 'boolean')"); } -TEST_CASE_FIXTURE(Fixture, "function_decl_quantify_right_type") +TEST_CASE_FIXTURE(BuiltinsFixture, "function_decl_quantify_right_type") { - ScopedFastFlag statFunctionSimplify{"LuauStatFunctionSimplify4", true}; - fileResolver.source["game/isAMagicMock"] = R"( --!nonstrict return function(value) @@ -1311,10 +1309,8 @@ end LUAU_REQUIRE_NO_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "function_decl_non_self_sealed_overwrite") +TEST_CASE_FIXTURE(BuiltinsFixture, "function_decl_non_self_sealed_overwrite") { - ScopedFastFlag statFunctionSimplify{"LuauStatFunctionSimplify4", true}; - CheckResult result = check(R"( function string.len(): number return 1 @@ -1333,11 +1329,8 @@ print(string.len('hello')) LUAU_REQUIRE_NO_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "function_decl_non_self_sealed_overwrite_2") +TEST_CASE_FIXTURE(BuiltinsFixture, "function_decl_non_self_sealed_overwrite_2") { - ScopedFastFlag statFunctionSimplify{"LuauStatFunctionSimplify4", true}; - ScopedFastFlag inferStatFunction{"LuauInferStatFunction", true}; - CheckResult result = check(R"( local t: { f: ((x: number) -> number)? } = {} @@ -1477,11 +1470,8 @@ TEST_CASE_FIXTURE(Fixture, "inferred_higher_order_functions_are_quantified_at_th LUAU_REQUIRE_NO_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "function_decl_non_self_unsealed_overwrite") +TEST_CASE_FIXTURE(BuiltinsFixture, "function_decl_non_self_unsealed_overwrite") { - ScopedFastFlag statFunctionSimplify{"LuauStatFunctionSimplify4", true}; - ScopedFastFlag inferStatFunction{"LuauInferStatFunction", true}; - CheckResult result = check(R"( local t = { f = nil :: ((x: number) -> number)? } @@ -1518,8 +1508,6 @@ TEST_CASE_FIXTURE(Fixture, "strict_mode_ok_with_missing_arguments") TEST_CASE_FIXTURE(Fixture, "function_statement_sealed_table_assignment_through_indexer") { - ScopedFastFlag statFunctionSimplify{"LuauStatFunctionSimplify4", true}; - CheckResult result = check(R"( local t: {[string]: () -> number} = {} @@ -1580,7 +1568,7 @@ wrapper(test) CHECK(acm->isVariadic); } -TEST_CASE_FIXTURE(Fixture, "too_few_arguments_variadic_generic2") +TEST_CASE_FIXTURE(BuiltinsFixture, "too_few_arguments_variadic_generic2") { CheckResult result = check(R"( function test(a: number, b: string, ...) diff --git a/tests/TypeInfer.generics.test.cpp b/tests/TypeInfer.generics.test.cpp index 91be2c1c..de0c9391 100644 --- a/tests/TypeInfer.generics.test.cpp +++ b/tests/TypeInfer.generics.test.cpp @@ -67,7 +67,7 @@ TEST_CASE_FIXTURE(Fixture, "local_vars_can_be_polytypes") LUAU_REQUIRE_NO_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "inferred_local_vars_can_be_polytypes") +TEST_CASE_FIXTURE(BuiltinsFixture, "inferred_local_vars_can_be_polytypes") { CheckResult result = check(R"( local function id(x) return x end @@ -79,7 +79,7 @@ TEST_CASE_FIXTURE(Fixture, "inferred_local_vars_can_be_polytypes") LUAU_REQUIRE_NO_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "local_vars_can_be_instantiated_polytypes") +TEST_CASE_FIXTURE(BuiltinsFixture, "local_vars_can_be_instantiated_polytypes") { CheckResult result = check(R"( local function id(x) return x end @@ -609,7 +609,7 @@ TEST_CASE_FIXTURE(Fixture, "typefuns_sharing_types") CHECK(requireType("y1") == requireType("y2")); } -TEST_CASE_FIXTURE(Fixture, "bound_tables_do_not_clone_original_fields") +TEST_CASE_FIXTURE(BuiltinsFixture, "bound_tables_do_not_clone_original_fields") { CheckResult result = check(R"( local exports = {} @@ -675,7 +675,7 @@ local d: D = c R"(Type '() -> ()' could not be converted into '() -> ()'; different number of generic type pack parameters)"); } -TEST_CASE_FIXTURE(Fixture, "generic_functions_dont_cache_type_parameters") +TEST_CASE_FIXTURE(BuiltinsFixture, "generic_functions_dont_cache_type_parameters") { CheckResult result = check(R"( -- See https://github.com/Roblox/luau/issues/332 @@ -1013,7 +1013,7 @@ TEST_CASE_FIXTURE(Fixture, "no_stack_overflow_from_quantifying") CHECK(it != result.errors.end()); } -TEST_CASE_FIXTURE(Fixture, "infer_generic_function_function_argument") +TEST_CASE_FIXTURE(BuiltinsFixture, "infer_generic_function_function_argument") { ScopedFastFlag sff{"LuauUnsealedTableLiteral", true}; @@ -1078,7 +1078,7 @@ TEST_CASE_FIXTURE(Fixture, "infer_generic_function_function_argument_overloaded" LUAU_REQUIRE_NO_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "infer_generic_lib_function_function_argument") +TEST_CASE_FIXTURE(BuiltinsFixture, "infer_generic_lib_function_function_argument") { CheckResult result = check(R"( local a = {{x=4}, {x=7}, {x=1}} diff --git a/tests/TypeInfer.intersectionTypes.test.cpp b/tests/TypeInfer.intersectionTypes.test.cpp index 3675919f..41bc0c21 100644 --- a/tests/TypeInfer.intersectionTypes.test.cpp +++ b/tests/TypeInfer.intersectionTypes.test.cpp @@ -316,8 +316,6 @@ TEST_CASE_FIXTURE(Fixture, "table_intersection_write_sealed") TEST_CASE_FIXTURE(Fixture, "table_intersection_write_sealed_indirect") { - ScopedFastFlag statFunctionSimplify{"LuauStatFunctionSimplify4", true}; - CheckResult result = check(R"( type X = { x: (number) -> number } type Y = { y: (string) -> string } @@ -351,8 +349,6 @@ caused by: TEST_CASE_FIXTURE(Fixture, "table_write_sealed_indirect") { - ScopedFastFlag statFunctionSimplify{"LuauStatFunctionSimplify4", true}; - // After normalization, previous 'table_intersection_write_sealed_indirect' is identical to this one CheckResult result = check(R"( type XY = { x: (number) -> number, y: (string) -> string } @@ -375,7 +371,7 @@ caused by: CHECK_EQ(toString(result.errors[3]), "Cannot add property 'w' to table 'XY'"); } -TEST_CASE_FIXTURE(Fixture, "table_intersection_setmetatable") +TEST_CASE_FIXTURE(BuiltinsFixture, "table_intersection_setmetatable") { CheckResult result = check(R"( local t: {} & {} diff --git a/tests/TypeInfer.loops.test.cpp b/tests/TypeInfer.loops.test.cpp index f9b510c1..765419c6 100644 --- a/tests/TypeInfer.loops.test.cpp +++ b/tests/TypeInfer.loops.test.cpp @@ -29,7 +29,7 @@ TEST_CASE_FIXTURE(Fixture, "for_loop") CHECK_EQ(*typeChecker.numberType, *requireType("q")); } -TEST_CASE_FIXTURE(Fixture, "for_in_loop") +TEST_CASE_FIXTURE(BuiltinsFixture, "for_in_loop") { CheckResult result = check(R"( local n @@ -46,7 +46,7 @@ TEST_CASE_FIXTURE(Fixture, "for_in_loop") CHECK_EQ(*typeChecker.stringType, *requireType("s")); } -TEST_CASE_FIXTURE(Fixture, "for_in_loop_with_next") +TEST_CASE_FIXTURE(BuiltinsFixture, "for_in_loop_with_next") { CheckResult result = check(R"( local n @@ -90,7 +90,7 @@ TEST_CASE_FIXTURE(Fixture, "for_in_loop_should_fail_with_non_function_iterator") CHECK_EQ("Cannot call non-function string", toString(result.errors[0])); } -TEST_CASE_FIXTURE(Fixture, "for_in_with_just_one_iterator_is_ok") +TEST_CASE_FIXTURE(BuiltinsFixture, "for_in_with_just_one_iterator_is_ok") { CheckResult result = check(R"( local function keys(dictionary) @@ -109,7 +109,7 @@ TEST_CASE_FIXTURE(Fixture, "for_in_with_just_one_iterator_is_ok") LUAU_REQUIRE_NO_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "for_in_with_a_custom_iterator_should_type_check") +TEST_CASE_FIXTURE(BuiltinsFixture, "for_in_with_a_custom_iterator_should_type_check") { CheckResult result = check(R"( local function range(l, h): () -> number @@ -161,7 +161,7 @@ TEST_CASE_FIXTURE(Fixture, "for_in_loop_on_non_function") REQUIRE(get(result.errors[0])); } -TEST_CASE_FIXTURE(Fixture, "for_in_loop_error_on_factory_not_returning_the_right_amount_of_values") +TEST_CASE_FIXTURE(BuiltinsFixture, "for_in_loop_error_on_factory_not_returning_the_right_amount_of_values") { CheckResult result = check(R"( local function hasDivisors(value: number, table) @@ -210,7 +210,7 @@ TEST_CASE_FIXTURE(Fixture, "for_in_loop_error_on_factory_not_returning_the_right CHECK_EQ(typeChecker.stringType, tm->givenType); } -TEST_CASE_FIXTURE(Fixture, "for_in_loop_error_on_iterator_requiring_args_but_none_given") +TEST_CASE_FIXTURE(BuiltinsFixture, "for_in_loop_error_on_iterator_requiring_args_but_none_given") { CheckResult result = check(R"( function prime_iter(state, index) @@ -288,7 +288,7 @@ TEST_CASE_FIXTURE(Fixture, "repeat_loop_condition_binds_to_its_block") LUAU_REQUIRE_NO_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "symbols_in_repeat_block_should_not_be_visible_beyond_until_condition") +TEST_CASE_FIXTURE(BuiltinsFixture, "symbols_in_repeat_block_should_not_be_visible_beyond_until_condition") { CheckResult result = check(R"( repeat @@ -301,7 +301,7 @@ TEST_CASE_FIXTURE(Fixture, "symbols_in_repeat_block_should_not_be_visible_beyond LUAU_REQUIRE_ERROR_COUNT(1, result); } -TEST_CASE_FIXTURE(Fixture, "varlist_declared_by_for_in_loop_should_be_free") +TEST_CASE_FIXTURE(BuiltinsFixture, "varlist_declared_by_for_in_loop_should_be_free") { CheckResult result = check(R"( local T = {} @@ -316,7 +316,7 @@ TEST_CASE_FIXTURE(Fixture, "varlist_declared_by_for_in_loop_should_be_free") LUAU_REQUIRE_NO_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "properly_infer_iteratee_is_a_free_table") +TEST_CASE_FIXTURE(BuiltinsFixture, "properly_infer_iteratee_is_a_free_table") { // In this case, we cannot know the element type of the table {}. It could be anything. // We therefore must initially ascribe a free typevar to iter. @@ -329,7 +329,7 @@ TEST_CASE_FIXTURE(Fixture, "properly_infer_iteratee_is_a_free_table") LUAU_REQUIRE_NO_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "correctly_scope_locals_while") +TEST_CASE_FIXTURE(BuiltinsFixture, "correctly_scope_locals_while") { CheckResult result = check(R"( while true do @@ -346,7 +346,7 @@ TEST_CASE_FIXTURE(Fixture, "correctly_scope_locals_while") CHECK_EQ(us->name, "a"); } -TEST_CASE_FIXTURE(Fixture, "ipairs_produces_integral_indices") +TEST_CASE_FIXTURE(BuiltinsFixture, "ipairs_produces_integral_indices") { CheckResult result = check(R"( local key @@ -378,7 +378,7 @@ TEST_CASE_FIXTURE(Fixture, "for_in_loop_where_iteratee_is_free") )"); } -TEST_CASE_FIXTURE(Fixture, "unreachable_code_after_infinite_loop") +TEST_CASE_FIXTURE(BuiltinsFixture, "unreachable_code_after_infinite_loop") { { CheckResult result = check(R"( @@ -460,7 +460,7 @@ TEST_CASE_FIXTURE(Fixture, "unreachable_code_after_infinite_loop") } } -TEST_CASE_FIXTURE(Fixture, "loop_typecheck_crash_on_empty_optional") +TEST_CASE_FIXTURE(BuiltinsFixture, "loop_typecheck_crash_on_empty_optional") { CheckResult result = check(R"( local t = {} @@ -541,7 +541,7 @@ TEST_CASE_FIXTURE(Fixture, "loop_iter_no_indexer") CHECK_EQ("Cannot iterate over a table without indexer", ge->message); } -TEST_CASE_FIXTURE(Fixture, "loop_iter_iter_metamethod") +TEST_CASE_FIXTURE(BuiltinsFixture, "loop_iter_iter_metamethod") { ScopedFastFlag sff{"LuauTypecheckIter", true}; diff --git a/tests/TypeInfer.modules.test.cpp b/tests/TypeInfer.modules.test.cpp index b6f49f9f..efa2a98d 100644 --- a/tests/TypeInfer.modules.test.cpp +++ b/tests/TypeInfer.modules.test.cpp @@ -16,7 +16,7 @@ LUAU_FASTFLAG(LuauTableSubtypingVariance2) TEST_SUITE_BEGIN("TypeInferModules"); -TEST_CASE_FIXTURE(Fixture, "require") +TEST_CASE_FIXTURE(BuiltinsFixture, "require") { fileResolver.source["game/A"] = R"( local function hooty(x: number): string @@ -54,7 +54,7 @@ TEST_CASE_FIXTURE(Fixture, "require") REQUIRE_EQ("number", toString(*hType)); } -TEST_CASE_FIXTURE(Fixture, "require_types") +TEST_CASE_FIXTURE(BuiltinsFixture, "require_types") { fileResolver.source["workspace/A"] = R"( export type Point = {x: number, y: number} @@ -69,7 +69,7 @@ TEST_CASE_FIXTURE(Fixture, "require_types") )"; CheckResult bResult = frontend.check("workspace/B"); - dumpErrors(bResult); + LUAU_REQUIRE_NO_ERRORS(bResult); ModulePtr b = frontend.moduleResolver.modules["workspace/B"]; REQUIRE(b != nullptr); @@ -78,7 +78,7 @@ TEST_CASE_FIXTURE(Fixture, "require_types") REQUIRE_MESSAGE(bool(get(hType)), "Expected table but got " << toString(hType)); } -TEST_CASE_FIXTURE(Fixture, "require_a_variadic_function") +TEST_CASE_FIXTURE(BuiltinsFixture, "require_a_variadic_function") { fileResolver.source["game/A"] = R"( local T = {} @@ -121,7 +121,7 @@ TEST_CASE_FIXTURE(Fixture, "type_error_of_unknown_qualified_type") REQUIRE_EQ(result.errors[0], (TypeError{Location{{1, 17}, {1, 40}}, UnknownSymbol{"SomeModule.DoesNotExist"}})); } -TEST_CASE_FIXTURE(Fixture, "require_module_that_does_not_export") +TEST_CASE_FIXTURE(BuiltinsFixture, "require_module_that_does_not_export") { const std::string sourceA = R"( )"; @@ -148,7 +148,7 @@ TEST_CASE_FIXTURE(Fixture, "require_module_that_does_not_export") CHECK_EQ("*unknown*", toString(hootyType)); } -TEST_CASE_FIXTURE(Fixture, "warn_if_you_try_to_require_a_non_modulescript") +TEST_CASE_FIXTURE(BuiltinsFixture, "warn_if_you_try_to_require_a_non_modulescript") { fileResolver.source["Modules/A"] = ""; fileResolver.sourceTypes["Modules/A"] = SourceCode::Local; @@ -164,7 +164,7 @@ TEST_CASE_FIXTURE(Fixture, "warn_if_you_try_to_require_a_non_modulescript") CHECK(get(result.errors[0])); } -TEST_CASE_FIXTURE(Fixture, "general_require_call_expression") +TEST_CASE_FIXTURE(BuiltinsFixture, "general_require_call_expression") { fileResolver.source["game/A"] = R"( --!strict @@ -183,7 +183,7 @@ a = tbl.abc.def CHECK_EQ("Type 'number' could not be converted into 'string'", toString(result.errors[0])); } -TEST_CASE_FIXTURE(Fixture, "general_require_type_mismatch") +TEST_CASE_FIXTURE(BuiltinsFixture, "general_require_type_mismatch") { fileResolver.source["game/A"] = R"( return { def = 4 } @@ -219,7 +219,7 @@ return m LUAU_REQUIRE_NO_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "custom_require_global") +TEST_CASE_FIXTURE(BuiltinsFixture, "custom_require_global") { CheckResult result = check(R"( --!nonstrict @@ -231,7 +231,7 @@ local crash = require(game.A) LUAU_REQUIRE_NO_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "require_failed_module") +TEST_CASE_FIXTURE(BuiltinsFixture, "require_failed_module") { fileResolver.source["game/A"] = R"( return unfortunately() @@ -267,7 +267,7 @@ function x:Destroy(): () end LUAU_REQUIRE_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "do_not_modify_imported_types_2") +TEST_CASE_FIXTURE(BuiltinsFixture, "do_not_modify_imported_types_2") { fileResolver.source["game/A"] = R"( export type Type = { x: { a: number } } @@ -285,7 +285,7 @@ type Rename = typeof(x.x) LUAU_REQUIRE_NO_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "do_not_modify_imported_types_3") +TEST_CASE_FIXTURE(BuiltinsFixture, "do_not_modify_imported_types_3") { fileResolver.source["game/A"] = R"( local y = setmetatable({}, {}) @@ -304,7 +304,7 @@ type Rename = typeof(x.x) LUAU_REQUIRE_NO_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "module_type_conflict") +TEST_CASE_FIXTURE(BuiltinsFixture, "module_type_conflict") { fileResolver.source["game/A"] = R"( export type T = { x: number } @@ -338,7 +338,7 @@ caused by: } } -TEST_CASE_FIXTURE(Fixture, "module_type_conflict_instantiated") +TEST_CASE_FIXTURE(BuiltinsFixture, "module_type_conflict_instantiated") { fileResolver.source["game/A"] = R"( export type Wrap = { x: T } diff --git a/tests/TypeInfer.oop.test.cpp b/tests/TypeInfer.oop.test.cpp index 5cd3f3ba..41690704 100644 --- a/tests/TypeInfer.oop.test.cpp +++ b/tests/TypeInfer.oop.test.cpp @@ -142,7 +142,7 @@ TEST_CASE_FIXTURE(Fixture, "inferring_hundreds_of_self_calls_should_not_suffocat CHECK_GE(50, module->internalTypes.typeVars.size()); } -TEST_CASE_FIXTURE(Fixture, "object_constructor_can_refer_to_method_of_self") +TEST_CASE_FIXTURE(BuiltinsFixture, "object_constructor_can_refer_to_method_of_self") { // CLI-30902 CheckResult result = check(R"( @@ -243,7 +243,7 @@ TEST_CASE_FIXTURE(Fixture, "inferred_methods_of_free_tables_have_the_same_level_ )"); } -TEST_CASE_FIXTURE(Fixture, "table_oop") +TEST_CASE_FIXTURE(BuiltinsFixture, "table_oop") { CheckResult result = check(R"( --!strict diff --git a/tests/TypeInfer.operators.test.cpp b/tests/TypeInfer.operators.test.cpp index a2787cad..51f6fdfb 100644 --- a/tests/TypeInfer.operators.test.cpp +++ b/tests/TypeInfer.operators.test.cpp @@ -77,7 +77,7 @@ TEST_CASE_FIXTURE(Fixture, "and_or_ternary") CHECK_EQ(toString(*requireType("s")), "number | string"); } -TEST_CASE_FIXTURE(Fixture, "primitive_arith_no_metatable") +TEST_CASE_FIXTURE(BuiltinsFixture, "primitive_arith_no_metatable") { CheckResult result = check(R"( function add(a: number, b: string) @@ -140,7 +140,7 @@ TEST_CASE_FIXTURE(Fixture, "some_primitive_binary_ops") CHECK_EQ("number", toString(requireType("c"))); } -TEST_CASE_FIXTURE(Fixture, "typecheck_overloaded_multiply_that_is_an_intersection") +TEST_CASE_FIXTURE(BuiltinsFixture, "typecheck_overloaded_multiply_that_is_an_intersection") { CheckResult result = check(R"( --!strict @@ -174,7 +174,7 @@ TEST_CASE_FIXTURE(Fixture, "typecheck_overloaded_multiply_that_is_an_intersectio CHECK_EQ("Vec3", toString(requireType("e"))); } -TEST_CASE_FIXTURE(Fixture, "typecheck_overloaded_multiply_that_is_an_intersection_on_rhs") +TEST_CASE_FIXTURE(BuiltinsFixture, "typecheck_overloaded_multiply_that_is_an_intersection_on_rhs") { CheckResult result = check(R"( --!strict @@ -245,7 +245,7 @@ TEST_CASE_FIXTURE(Fixture, "cannot_indirectly_compare_types_that_do_not_have_a_m REQUIRE_EQ(gen->message, "Type a cannot be compared with < because it has no metatable"); } -TEST_CASE_FIXTURE(Fixture, "cannot_indirectly_compare_types_that_do_not_offer_overloaded_ordering_operators") +TEST_CASE_FIXTURE(BuiltinsFixture, "cannot_indirectly_compare_types_that_do_not_offer_overloaded_ordering_operators") { CheckResult result = check(R"( local M = {} @@ -266,7 +266,7 @@ TEST_CASE_FIXTURE(Fixture, "cannot_indirectly_compare_types_that_do_not_offer_ov REQUIRE_EQ(gen->message, "Table M does not offer metamethod __lt"); } -TEST_CASE_FIXTURE(Fixture, "cannot_compare_tables_that_do_not_have_the_same_metatable") +TEST_CASE_FIXTURE(BuiltinsFixture, "cannot_compare_tables_that_do_not_have_the_same_metatable") { CheckResult result = check(R"( --!strict @@ -289,7 +289,7 @@ TEST_CASE_FIXTURE(Fixture, "cannot_compare_tables_that_do_not_have_the_same_meta REQUIRE_EQ((Location{{11, 18}, {11, 23}}), result.errors[1].location); } -TEST_CASE_FIXTURE(Fixture, "produce_the_correct_error_message_when_comparing_a_table_with_a_metatable_with_one_that_does_not") +TEST_CASE_FIXTURE(BuiltinsFixture, "produce_the_correct_error_message_when_comparing_a_table_with_a_metatable_with_one_that_does_not") { CheckResult result = check(R"( --!strict @@ -361,7 +361,7 @@ TEST_CASE_FIXTURE(Fixture, "compound_assign_mismatch_result") CHECK_EQ(result.errors[1], (TypeError{Location{{2, 8}, {2, 15}}, TypeMismatch{typeChecker.stringType, typeChecker.numberType}})); } -TEST_CASE_FIXTURE(Fixture, "compound_assign_metatable") +TEST_CASE_FIXTURE(BuiltinsFixture, "compound_assign_metatable") { CheckResult result = check(R"( --!strict @@ -381,7 +381,7 @@ TEST_CASE_FIXTURE(Fixture, "compound_assign_metatable") CHECK_EQ(0, result.errors.size()); } -TEST_CASE_FIXTURE(Fixture, "compound_assign_mismatch_metatable") +TEST_CASE_FIXTURE(BuiltinsFixture, "compound_assign_mismatch_metatable") { CheckResult result = check(R"( --!strict @@ -428,7 +428,7 @@ local x = false LUAU_REQUIRE_NO_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "typecheck_unary_minus") +TEST_CASE_FIXTURE(BuiltinsFixture, "typecheck_unary_minus") { CheckResult result = check(R"( --!strict @@ -461,7 +461,7 @@ TEST_CASE_FIXTURE(Fixture, "typecheck_unary_minus") REQUIRE_EQ(gen->message, "Unary operator '-' not supported by type 'bar'"); } -TEST_CASE_FIXTURE(Fixture, "unary_not_is_boolean") +TEST_CASE_FIXTURE(BuiltinsFixture, "unary_not_is_boolean") { CheckResult result = check(R"( local b = not "string" @@ -473,7 +473,7 @@ TEST_CASE_FIXTURE(Fixture, "unary_not_is_boolean") REQUIRE_EQ("boolean", toString(requireType("c"))); } -TEST_CASE_FIXTURE(Fixture, "disallow_string_and_types_without_metatables_from_arithmetic_binary_ops") +TEST_CASE_FIXTURE(BuiltinsFixture, "disallow_string_and_types_without_metatables_from_arithmetic_binary_ops") { CheckResult result = check(R"( --!strict @@ -573,7 +573,7 @@ TEST_CASE_FIXTURE(Fixture, "strict_binary_op_where_lhs_unknown") CHECK_EQ("Unknown type used in + operation; consider adding a type annotation to 'a'", toString(result.errors[0])); } -TEST_CASE_FIXTURE(Fixture, "and_binexps_dont_unify") +TEST_CASE_FIXTURE(BuiltinsFixture, "and_binexps_dont_unify") { CheckResult result = check(R"( --!strict @@ -628,7 +628,7 @@ TEST_CASE_FIXTURE(Fixture, "cli_38355_recursive_union") CHECK_EQ("Type contains a self-recursive construct that cannot be resolved", toString(result.errors[0])); } -TEST_CASE_FIXTURE(Fixture, "UnknownGlobalCompoundAssign") +TEST_CASE_FIXTURE(BuiltinsFixture, "UnknownGlobalCompoundAssign") { // In non-strict mode, global definition is still allowed { @@ -755,8 +755,6 @@ TEST_CASE_FIXTURE(Fixture, "refine_and_or") TEST_CASE_FIXTURE(Fixture, "infer_any_in_all_modes_when_lhs_is_unknown") { - ScopedFastFlag sff{"LuauDecoupleOperatorInferenceFromUnifiedTypeInference", true}; - CheckResult result = check(Mode::Strict, R"( local function f(x, y) return x + y @@ -779,4 +777,47 @@ TEST_CASE_FIXTURE(Fixture, "infer_any_in_all_modes_when_lhs_is_unknown") // the case right now, though. } +TEST_CASE_FIXTURE(BuiltinsFixture, "equality_operations_succeed_if_any_union_branch_succeeds") +{ + ScopedFastFlag sff("LuauSuccessTypingForEqualityOperations", true); + + CheckResult result = check(R"( + local mm = {} + type Foo = typeof(setmetatable({}, mm)) + local x: Foo + local y: Foo? + + local v1 = x == y + local v2 = y == x + local v3 = x ~= y + local v4 = y ~= x + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CheckResult result2 = check(R"( + local mm1 = { + x = "foo", + } + + local mm2 = { + y = "bar", + } + + type Foo = typeof(setmetatable({}, mm1)) + type Bar = typeof(setmetatable({}, mm2)) + + local x1: Foo + local x2: Foo? + local y1: Bar + local y2: Bar? + + local v1 = x1 == y1 + local v2 = x2 == y2 + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result2); + CHECK(toString(result2.errors[0]) == "Types Foo and Bar cannot be compared with == because they do not have the same metatable"); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.provisional.test.cpp b/tests/TypeInfer.provisional.test.cpp index 2ef77419..ee3ae972 100644 --- a/tests/TypeInfer.provisional.test.cpp +++ b/tests/TypeInfer.provisional.test.cpp @@ -53,7 +53,7 @@ TEST_CASE_FIXTURE(Fixture, "typeguard_inference_incomplete") CHECK_EQ(expected, decorateWithTypes(code)); } -TEST_CASE_FIXTURE(Fixture, "xpcall_returns_what_f_returns") +TEST_CASE_FIXTURE(BuiltinsFixture, "xpcall_returns_what_f_returns") { const std::string code = R"( local a, b, c = xpcall(function() return 1, "foo" end, function() return "foo", 1 end) @@ -105,7 +105,7 @@ TEST_CASE_FIXTURE(Fixture, "it_should_be_agnostic_of_actual_size") // Ideally setmetatable's second argument would be an optional free table. // For now, infer it as just a free table. -TEST_CASE_FIXTURE(Fixture, "setmetatable_constrains_free_type_into_free_table") +TEST_CASE_FIXTURE(BuiltinsFixture, "setmetatable_constrains_free_type_into_free_table") { CheckResult result = check(R"( local a = {} @@ -146,7 +146,7 @@ TEST_CASE_FIXTURE(Fixture, "while_body_are_also_refined") // Originally from TypeInfer.test.cpp. // I dont think type checking the metamethod at every site of == is the correct thing to do. // We should be type checking the metamethod at the call site of setmetatable. -TEST_CASE_FIXTURE(Fixture, "error_on_eq_metamethod_returning_a_type_other_than_boolean") +TEST_CASE_FIXTURE(BuiltinsFixture, "error_on_eq_metamethod_returning_a_type_other_than_boolean") { CheckResult result = check(R"( local tab = {a = 1} @@ -428,7 +428,7 @@ TEST_CASE_FIXTURE(Fixture, "pcall_returns_at_least_two_value_but_function_return } // Belongs in TypeInfer.builtins.test.cpp. -TEST_CASE_FIXTURE(Fixture, "choose_the_right_overload_for_pcall") +TEST_CASE_FIXTURE(BuiltinsFixture, "choose_the_right_overload_for_pcall") { CheckResult result = check(R"( local function f(): number @@ -449,7 +449,7 @@ TEST_CASE_FIXTURE(Fixture, "choose_the_right_overload_for_pcall") } // Belongs in TypeInfer.builtins.test.cpp. -TEST_CASE_FIXTURE(Fixture, "function_returns_many_things_but_first_of_it_is_forgotten") +TEST_CASE_FIXTURE(BuiltinsFixture, "function_returns_many_things_but_first_of_it_is_forgotten") { CheckResult result = check(R"( local function f(): (number, string, boolean) diff --git a/tests/TypeInfer.refinements.test.cpp b/tests/TypeInfer.refinements.test.cpp index 136ca00a..8c130490 100644 --- a/tests/TypeInfer.refinements.test.cpp +++ b/tests/TypeInfer.refinements.test.cpp @@ -240,7 +240,7 @@ TEST_CASE_FIXTURE(Fixture, "typeguard_in_if_condition_position") CHECK_EQ("number", toString(requireTypeAtPosition({3, 26}))); } -TEST_CASE_FIXTURE(Fixture, "typeguard_in_assert_position") +TEST_CASE_FIXTURE(BuiltinsFixture, "typeguard_in_assert_position") { CheckResult result = check(R"( local a @@ -300,7 +300,7 @@ TEST_CASE_FIXTURE(Fixture, "call_a_more_specific_function_using_typeguard") CHECK_EQ("Type 'string' could not be converted into 'number'", toString(result.errors[0])); } -TEST_CASE_FIXTURE(Fixture, "impossible_type_narrow_is_not_an_error") +TEST_CASE_FIXTURE(BuiltinsFixture, "impossible_type_narrow_is_not_an_error") { // This unit test serves as a reminder to not implement this warning until Luau is intelligent enough. // For instance, getting a value out of the indexer and checking whether the value exists is not an error. @@ -333,7 +333,7 @@ TEST_CASE_FIXTURE(Fixture, "truthy_constraint_on_properties") CHECK_EQ("number?", toString(requireType("bar"))); } -TEST_CASE_FIXTURE(Fixture, "index_on_a_refined_property") +TEST_CASE_FIXTURE(BuiltinsFixture, "index_on_a_refined_property") { CheckResult result = check(R"( local t: {x: {y: string}?} = {x = {y = "hello!"}} @@ -346,7 +346,7 @@ TEST_CASE_FIXTURE(Fixture, "index_on_a_refined_property") LUAU_REQUIRE_NO_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "assert_non_binary_expressions_actually_resolve_constraints") +TEST_CASE_FIXTURE(BuiltinsFixture, "assert_non_binary_expressions_actually_resolve_constraints") { CheckResult result = check(R"( local foo: string? = "hello" @@ -730,7 +730,7 @@ TEST_CASE_FIXTURE(Fixture, "type_guard_can_filter_for_overloaded_function") CHECK_EQ("nil", toString(requireTypeAtPosition({6, 28}))); } -TEST_CASE_FIXTURE(Fixture, "type_guard_warns_on_no_overlapping_types_only_when_sense_is_true") +TEST_CASE_FIXTURE(BuiltinsFixture, "type_guard_warns_on_no_overlapping_types_only_when_sense_is_true") { CheckResult result = check(R"( local function f(t: {x: number}) @@ -846,7 +846,7 @@ TEST_CASE_FIXTURE(Fixture, "not_t_or_some_prop_of_t") CHECK_EQ("{| x: boolean |}?", toString(requireTypeAtPosition({3, 28}))); } -TEST_CASE_FIXTURE(Fixture, "assert_a_to_be_truthy_then_assert_a_to_be_number") +TEST_CASE_FIXTURE(BuiltinsFixture, "assert_a_to_be_truthy_then_assert_a_to_be_number") { CheckResult result = check(R"( local a: (number | string)? @@ -862,7 +862,7 @@ TEST_CASE_FIXTURE(Fixture, "assert_a_to_be_truthy_then_assert_a_to_be_number") CHECK_EQ("number", toString(requireTypeAtPosition({5, 18}))); } -TEST_CASE_FIXTURE(Fixture, "merge_should_be_fully_agnostic_of_hashmap_ordering") +TEST_CASE_FIXTURE(BuiltinsFixture, "merge_should_be_fully_agnostic_of_hashmap_ordering") { // This bug came up because there was a mistake in Luau::merge where zipping on two maps would produce the wrong merged result. CheckResult result = check(R"( @@ -899,7 +899,7 @@ TEST_CASE_FIXTURE(Fixture, "refine_the_correct_types_opposite_of_when_a_is_not_n CHECK_EQ("number | string", toString(requireTypeAtPosition({5, 28}))); } -TEST_CASE_FIXTURE(Fixture, "is_truthy_constraint_ifelse_expression") +TEST_CASE_FIXTURE(BuiltinsFixture, "is_truthy_constraint_ifelse_expression") { CheckResult result = check(R"( function f(v:string?) @@ -913,7 +913,7 @@ TEST_CASE_FIXTURE(Fixture, "is_truthy_constraint_ifelse_expression") CHECK_EQ("nil", toString(requireTypeAtPosition({2, 45}))); } -TEST_CASE_FIXTURE(Fixture, "invert_is_truthy_constraint_ifelse_expression") +TEST_CASE_FIXTURE(BuiltinsFixture, "invert_is_truthy_constraint_ifelse_expression") { CheckResult result = check(R"( function f(v:string?) @@ -945,7 +945,7 @@ TEST_CASE_FIXTURE(Fixture, "type_comparison_ifelse_expression") CHECK_EQ("any", toString(requireTypeAtPosition({6, 66}))); } -TEST_CASE_FIXTURE(Fixture, "correctly_lookup_a_shadowed_local_that_which_was_previously_refined") +TEST_CASE_FIXTURE(BuiltinsFixture, "correctly_lookup_a_shadowed_local_that_which_was_previously_refined") { CheckResult result = check(R"( local foo: string? = "hi" diff --git a/tests/TypeInfer.singletons.test.cpp b/tests/TypeInfer.singletons.test.cpp index 8d6682b8..79eeb824 100644 --- a/tests/TypeInfer.singletons.test.cpp +++ b/tests/TypeInfer.singletons.test.cpp @@ -415,7 +415,7 @@ TEST_CASE_FIXTURE(Fixture, "widening_happens_almost_everywhere_except_for_tables LUAU_REQUIRE_NO_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "table_insert_with_a_singleton_argument") +TEST_CASE_FIXTURE(BuiltinsFixture, "table_insert_with_a_singleton_argument") { ScopedFastFlag sff[]{ {"LuauWidenIfSupertypeIsFree2", true}, diff --git a/tests/TypeInfer.tables.test.cpp b/tests/TypeInfer.tables.test.cpp index 8e535995..5078b0bf 100644 --- a/tests/TypeInfer.tables.test.cpp +++ b/tests/TypeInfer.tables.test.cpp @@ -201,7 +201,7 @@ TEST_CASE_FIXTURE(Fixture, "used_dot_instead_of_colon") REQUIRE(it != result.errors.end()); } -TEST_CASE_FIXTURE(Fixture, "used_colon_correctly") +TEST_CASE_FIXTURE(BuiltinsFixture, "used_colon_correctly") { CheckResult result = check(R"( --!nonstrict @@ -883,7 +883,7 @@ TEST_CASE_FIXTURE(Fixture, "assigning_to_an_unsealed_table_with_string_literal_s CHECK_EQ(*typeChecker.stringType, *propertyA); } -TEST_CASE_FIXTURE(Fixture, "oop_indexer_works") +TEST_CASE_FIXTURE(BuiltinsFixture, "oop_indexer_works") { CheckResult result = check(R"( local clazz = {} @@ -906,7 +906,7 @@ TEST_CASE_FIXTURE(Fixture, "oop_indexer_works") CHECK_EQ(*typeChecker.stringType, *requireType("words")); } -TEST_CASE_FIXTURE(Fixture, "indexer_table") +TEST_CASE_FIXTURE(BuiltinsFixture, "indexer_table") { CheckResult result = check(R"( local clazz = {a="hello"} @@ -919,7 +919,7 @@ TEST_CASE_FIXTURE(Fixture, "indexer_table") CHECK_EQ(*typeChecker.stringType, *requireType("b")); } -TEST_CASE_FIXTURE(Fixture, "indexer_fn") +TEST_CASE_FIXTURE(BuiltinsFixture, "indexer_fn") { CheckResult result = check(R"( local instanace = setmetatable({}, {__index=function() return 10 end}) @@ -930,7 +930,7 @@ TEST_CASE_FIXTURE(Fixture, "indexer_fn") CHECK_EQ(*typeChecker.numberType, *requireType("b")); } -TEST_CASE_FIXTURE(Fixture, "meta_add") +TEST_CASE_FIXTURE(BuiltinsFixture, "meta_add") { // Note: meta_add_inferred and this unit test are currently the same exact thing. // We'll want to change this one in particular when we add real syntax for metatables. @@ -947,7 +947,7 @@ TEST_CASE_FIXTURE(Fixture, "meta_add") CHECK_EQ(follow(requireType("a")), follow(requireType("c"))); } -TEST_CASE_FIXTURE(Fixture, "meta_add_inferred") +TEST_CASE_FIXTURE(BuiltinsFixture, "meta_add_inferred") { CheckResult result = check(R"( local a = {} @@ -960,7 +960,7 @@ TEST_CASE_FIXTURE(Fixture, "meta_add_inferred") CHECK_EQ(*requireType("a"), *requireType("c")); } -TEST_CASE_FIXTURE(Fixture, "meta_add_both_ways") +TEST_CASE_FIXTURE(BuiltinsFixture, "meta_add_both_ways") { CheckResult result = check(R"( type VectorMt = { __add: (Vector, number) -> Vector } @@ -980,7 +980,7 @@ TEST_CASE_FIXTURE(Fixture, "meta_add_both_ways") // This test exposed a bug where we let go of the "seen" stack while unifying table types // As a result, type inference crashed with a stack overflow. -TEST_CASE_FIXTURE(Fixture, "unification_of_unions_in_a_self_referential_type") +TEST_CASE_FIXTURE(BuiltinsFixture, "unification_of_unions_in_a_self_referential_type") { CheckResult result = check(R"( type A = {} @@ -1009,7 +1009,7 @@ TEST_CASE_FIXTURE(Fixture, "unification_of_unions_in_a_self_referential_type") CHECK_EQ(bmtv->metatable, requireType("bmt")); } -TEST_CASE_FIXTURE(Fixture, "oop_polymorphic") +TEST_CASE_FIXTURE(BuiltinsFixture, "oop_polymorphic") { CheckResult result = check(R"( local animal = {} @@ -1060,7 +1060,7 @@ TEST_CASE_FIXTURE(Fixture, "user_defined_table_types_are_named") CHECK_EQ("Vector3", toString(requireType("v"))); } -TEST_CASE_FIXTURE(Fixture, "result_is_always_any_if_lhs_is_any") +TEST_CASE_FIXTURE(BuiltinsFixture, "result_is_always_any_if_lhs_is_any") { CheckResult result = check(R"( type Vector3MT = { @@ -1133,7 +1133,7 @@ TEST_CASE_FIXTURE(Fixture, "nice_error_when_trying_to_fetch_property_of_boolean" CHECK_EQ("Type 'boolean' does not have key 'some_prop'", toString(result.errors[0])); } -TEST_CASE_FIXTURE(Fixture, "defining_a_method_for_a_builtin_sealed_table_must_fail") +TEST_CASE_FIXTURE(BuiltinsFixture, "defining_a_method_for_a_builtin_sealed_table_must_fail") { CheckResult result = check(R"( function string.m() end @@ -1142,7 +1142,7 @@ TEST_CASE_FIXTURE(Fixture, "defining_a_method_for_a_builtin_sealed_table_must_fa LUAU_REQUIRE_ERROR_COUNT(1, result); } -TEST_CASE_FIXTURE(Fixture, "defining_a_self_method_for_a_builtin_sealed_table_must_fail") +TEST_CASE_FIXTURE(BuiltinsFixture, "defining_a_self_method_for_a_builtin_sealed_table_must_fail") { CheckResult result = check(R"( function string:m() end @@ -1261,7 +1261,7 @@ TEST_CASE_FIXTURE(Fixture, "found_like_key_in_table_function_call") CHECK_EQ(toString(te), "Key 'fOo' not found in table 't'. Did you mean 'Foo'?"); } -TEST_CASE_FIXTURE(Fixture, "found_like_key_in_table_property_access") +TEST_CASE_FIXTURE(BuiltinsFixture, "found_like_key_in_table_property_access") { CheckResult result = check(R"( local t = {X = 1} @@ -1286,7 +1286,7 @@ TEST_CASE_FIXTURE(Fixture, "found_like_key_in_table_property_access") CHECK_EQ(toString(te), "Key 'x' not found in table 't'. Did you mean 'X'?"); } -TEST_CASE_FIXTURE(Fixture, "found_multiple_like_keys") +TEST_CASE_FIXTURE(BuiltinsFixture, "found_multiple_like_keys") { CheckResult result = check(R"( local t = {Foo = 1, foO = 2} @@ -1312,7 +1312,7 @@ TEST_CASE_FIXTURE(Fixture, "found_multiple_like_keys") CHECK_EQ(toString(te), "Key 'foo' not found in table 't'. Did you mean one of 'Foo', 'foO'?"); } -TEST_CASE_FIXTURE(Fixture, "dont_suggest_exact_match_keys") +TEST_CASE_FIXTURE(BuiltinsFixture, "dont_suggest_exact_match_keys") { CheckResult result = check(R"( local t = {} @@ -1339,7 +1339,7 @@ TEST_CASE_FIXTURE(Fixture, "dont_suggest_exact_match_keys") CHECK_EQ(toString(te), "Key 'Foo' not found in table 't'. Did you mean 'foO'?"); } -TEST_CASE_FIXTURE(Fixture, "getmetatable_returns_pointer_to_metatable") +TEST_CASE_FIXTURE(BuiltinsFixture, "getmetatable_returns_pointer_to_metatable") { CheckResult result = check(R"( local t = {x = 1} @@ -1352,7 +1352,7 @@ TEST_CASE_FIXTURE(Fixture, "getmetatable_returns_pointer_to_metatable") CHECK_EQ(*requireType("mt"), *requireType("returnedMT")); } -TEST_CASE_FIXTURE(Fixture, "metatable_mismatch_should_fail") +TEST_CASE_FIXTURE(BuiltinsFixture, "metatable_mismatch_should_fail") { CheckResult result = check(R"( local t1 = {x = 1} @@ -1374,7 +1374,7 @@ TEST_CASE_FIXTURE(Fixture, "metatable_mismatch_should_fail") CHECK_EQ(*tm->givenType, *requireType("t2")); } -TEST_CASE_FIXTURE(Fixture, "property_lookup_through_tabletypevar_metatable") +TEST_CASE_FIXTURE(BuiltinsFixture, "property_lookup_through_tabletypevar_metatable") { CheckResult result = check(R"( local t = {x = 1} @@ -1393,7 +1393,7 @@ TEST_CASE_FIXTURE(Fixture, "property_lookup_through_tabletypevar_metatable") CHECK_EQ(up->key, "z"); } -TEST_CASE_FIXTURE(Fixture, "missing_metatable_for_sealed_tables_do_not_get_inferred") +TEST_CASE_FIXTURE(BuiltinsFixture, "missing_metatable_for_sealed_tables_do_not_get_inferred") { CheckResult result = check(R"( local t = {x = 1} @@ -1742,7 +1742,7 @@ TEST_CASE_FIXTURE(Fixture, "hide_table_error_properties") CHECK_EQ("Cannot add property 'b' to table '{| x: number |}'", toString(result.errors[1])); } -TEST_CASE_FIXTURE(Fixture, "builtin_table_names") +TEST_CASE_FIXTURE(BuiltinsFixture, "builtin_table_names") { CheckResult result = check(R"( os.h = 2 @@ -1755,7 +1755,7 @@ TEST_CASE_FIXTURE(Fixture, "builtin_table_names") CHECK_EQ("Cannot add property 'k' to table 'string'", toString(result.errors[1])); } -TEST_CASE_FIXTURE(Fixture, "persistent_sealed_table_is_immutable") +TEST_CASE_FIXTURE(BuiltinsFixture, "persistent_sealed_table_is_immutable") { CheckResult result = check(R"( --!nonstrict @@ -1858,7 +1858,7 @@ local foos: {Foo} = { LUAU_REQUIRE_NO_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "quantifying_a_bound_var_works") +TEST_CASE_FIXTURE(BuiltinsFixture, "quantifying_a_bound_var_works") { CheckResult result = check(R"( local clazz = {} @@ -1983,7 +1983,7 @@ TEST_CASE_FIXTURE(Fixture, "invariant_table_properties_means_instantiating_table LUAU_REQUIRE_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "table_insert_should_cope_with_optional_properties_in_nonstrict") +TEST_CASE_FIXTURE(BuiltinsFixture, "table_insert_should_cope_with_optional_properties_in_nonstrict") { CheckResult result = check(R"( --!nonstrict @@ -1996,7 +1996,7 @@ TEST_CASE_FIXTURE(Fixture, "table_insert_should_cope_with_optional_properties_in LUAU_REQUIRE_NO_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "table_insert_should_cope_with_optional_properties_in_strict") +TEST_CASE_FIXTURE(BuiltinsFixture, "table_insert_should_cope_with_optional_properties_in_strict") { ScopedFastFlag sff{"LuauTableSubtypingVariance2", true}; @@ -2052,7 +2052,7 @@ caused by: Property 'y' is not compatible. Type 'number' could not be converted into 'string')"); } -TEST_CASE_FIXTURE(Fixture, "error_detailed_metatable_prop") +TEST_CASE_FIXTURE(BuiltinsFixture, "error_detailed_metatable_prop") { ScopedFastFlag sff[]{ {"LuauTableSubtypingVariance2", true}, @@ -2183,7 +2183,7 @@ a.p = { x = 9 } LUAU_REQUIRE_NO_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "recursive_metatable_type_call") +TEST_CASE_FIXTURE(BuiltinsFixture, "recursive_metatable_type_call") { ScopedFastFlag sff[]{ {"LuauUnsealedTableLiteral", true}, @@ -2277,7 +2277,7 @@ local y = #x LUAU_REQUIRE_ERROR_COUNT(1, result); } -TEST_CASE_FIXTURE(Fixture, "dont_hang_when_trying_to_look_up_in_cyclic_metatable_index") +TEST_CASE_FIXTURE(BuiltinsFixture, "dont_hang_when_trying_to_look_up_in_cyclic_metatable_index") { ScopedFastFlag sff{"LuauTerminateCyclicMetatableIndexLookup", true}; @@ -2296,7 +2296,7 @@ TEST_CASE_FIXTURE(Fixture, "dont_hang_when_trying_to_look_up_in_cyclic_metatable CHECK_EQ("Type 't' does not have key 'p'", toString(result.errors[0])); } -TEST_CASE_FIXTURE(Fixture, "give_up_after_one_metatable_index_look_up") +TEST_CASE_FIXTURE(BuiltinsFixture, "give_up_after_one_metatable_index_look_up") { CheckResult result = check(R"( local data = { x = 5 } @@ -2478,7 +2478,7 @@ TEST_CASE_FIXTURE(Fixture, "free_rhs_table_can_also_be_bound") )"); } -TEST_CASE_FIXTURE(Fixture, "table_unifies_into_map") +TEST_CASE_FIXTURE(BuiltinsFixture, "table_unifies_into_map") { CheckResult result = check(R"( local Instance: any @@ -2564,7 +2564,7 @@ TEST_CASE_FIXTURE(Fixture, "generalize_table_argument") * the generalization process), then it loses the knowledge that its metatable will have an :incr() * method. */ -TEST_CASE_FIXTURE(Fixture, "dont_quantify_table_that_belongs_to_outer_scope") +TEST_CASE_FIXTURE(BuiltinsFixture, "dont_quantify_table_that_belongs_to_outer_scope") { CheckResult result = check(R"( local Counter = {} @@ -2606,7 +2606,7 @@ TEST_CASE_FIXTURE(Fixture, "dont_quantify_table_that_belongs_to_outer_scope") } // TODO: CLI-39624 -TEST_CASE_FIXTURE(Fixture, "instantiate_tables_at_scope_level") +TEST_CASE_FIXTURE(BuiltinsFixture, "instantiate_tables_at_scope_level") { CheckResult result = check(R"( --!strict @@ -2690,7 +2690,7 @@ TEST_CASE_FIXTURE(Fixture, "dont_crash_when_setmetatable_does_not_produce_a_meta LUAU_REQUIRE_ERROR_COUNT(1, result); } -TEST_CASE_FIXTURE(Fixture, "instantiate_table_cloning") +TEST_CASE_FIXTURE(BuiltinsFixture, "instantiate_table_cloning") { CheckResult result = check(R"( --!nonstrict @@ -2711,7 +2711,7 @@ type t0 = any CHECK(ttv->instantiatedTypeParams.empty()); } -TEST_CASE_FIXTURE(Fixture, "instantiate_table_cloning_2") +TEST_CASE_FIXTURE(BuiltinsFixture, "instantiate_table_cloning_2") { ScopedFastFlag sff{"LuauOnlyMutateInstantiatedTables", true}; @@ -2767,7 +2767,7 @@ local baz = foo[bar] CHECK_EQ(result.errors[0].location, Location{Position{3, 16}, Position{3, 19}}); } -TEST_CASE_FIXTURE(Fixture, "table_simple_call") +TEST_CASE_FIXTURE(BuiltinsFixture, "table_simple_call") { CheckResult result = check(R"( local a = setmetatable({ x = 2 }, { @@ -2783,7 +2783,7 @@ local c = a(2) -- too many arguments CHECK_EQ("Argument count mismatch. Function expects 1 argument, but 2 are specified", toString(result.errors[0])); } -TEST_CASE_FIXTURE(Fixture, "access_index_metamethod_that_returns_variadic") +TEST_CASE_FIXTURE(BuiltinsFixture, "access_index_metamethod_that_returns_variadic") { CheckResult result = check(R"( type Foo = {x: string} @@ -2878,7 +2878,7 @@ TEST_CASE_FIXTURE(Fixture, "pairs_parameters_are_not_unsealed_tables") )"); } -TEST_CASE_FIXTURE(Fixture, "table_function_check_use_after_free") +TEST_CASE_FIXTURE(BuiltinsFixture, "table_function_check_use_after_free") { CheckResult result = check(R"( local t = {} @@ -2916,7 +2916,7 @@ TEST_CASE_FIXTURE(Fixture, "inferred_properties_of_a_table_should_start_with_the } // The real bug here was that we weren't always uncondionally typechecking a trailing return statement last. -TEST_CASE_FIXTURE(Fixture, "dont_leak_free_table_props") +TEST_CASE_FIXTURE(BuiltinsFixture, "dont_leak_free_table_props") { CheckResult result = check(R"( local function a(state) diff --git a/tests/TypeInfer.test.cpp b/tests/TypeInfer.test.cpp index e81ef1a9..48cd1c3d 100644 --- a/tests/TypeInfer.test.cpp +++ b/tests/TypeInfer.test.cpp @@ -161,7 +161,7 @@ TEST_CASE_FIXTURE(Fixture, "unify_nearly_identical_recursive_types") LUAU_REQUIRE_NO_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "warn_on_lowercase_parent_property") +TEST_CASE_FIXTURE(BuiltinsFixture, "warn_on_lowercase_parent_property") { CheckResult result = check(R"( local M = require(script.parent.DoesNotMatter) @@ -175,7 +175,7 @@ TEST_CASE_FIXTURE(Fixture, "warn_on_lowercase_parent_property") REQUIRE_EQ("parent", ed->symbol); } -TEST_CASE_FIXTURE(Fixture, "weird_case") +TEST_CASE_FIXTURE(BuiltinsFixture, "weird_case") { CheckResult result = check(R"( local function f() return 4 end @@ -419,7 +419,7 @@ TEST_CASE_FIXTURE(Fixture, "globals_everywhere") CHECK_EQ("any", toString(requireType("bar"))); } -TEST_CASE_FIXTURE(Fixture, "correctly_scope_locals_do") +TEST_CASE_FIXTURE(BuiltinsFixture, "correctly_scope_locals_do") { CheckResult result = check(R"( do @@ -534,7 +534,7 @@ TEST_CASE_FIXTURE(Fixture, "tc_after_error_recovery_no_assert") LUAU_REQUIRE_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "tc_after_error_recovery_no_replacement_name_in_error") +TEST_CASE_FIXTURE(BuiltinsFixture, "tc_after_error_recovery_no_replacement_name_in_error") { { CheckResult result = check(R"( @@ -587,7 +587,7 @@ TEST_CASE_FIXTURE(Fixture, "tc_after_error_recovery_no_replacement_name_in_error } } -TEST_CASE_FIXTURE(Fixture, "index_expr_should_be_checked") +TEST_CASE_FIXTURE(BuiltinsFixture, "index_expr_should_be_checked") { CheckResult result = check(R"( local foo: any @@ -768,7 +768,7 @@ b, c = {2, "s"}, {"b", 4} LUAU_REQUIRE_NO_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "infer_assignment_value_types_mutable_lval") +TEST_CASE_FIXTURE(BuiltinsFixture, "infer_assignment_value_types_mutable_lval") { CheckResult result = check(R"( local a = {} @@ -836,7 +836,7 @@ local a: number? = if true then 1 else nil LUAU_REQUIRE_NO_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "tc_if_else_expressions_expected_type_3") +TEST_CASE_FIXTURE(BuiltinsFixture, "tc_if_else_expressions_expected_type_3") { CheckResult result = check(R"( local function times(n: any, f: () -> T) @@ -907,7 +907,7 @@ TEST_CASE_FIXTURE(Fixture, "fuzzer_found_this") )"); } -TEST_CASE_FIXTURE(Fixture, "recursive_metatable_crash") +TEST_CASE_FIXTURE(BuiltinsFixture, "recursive_metatable_crash") { CheckResult result = check(R"( local function getIt() @@ -1041,7 +1041,6 @@ TEST_CASE_FIXTURE(Fixture, "follow_on_new_types_in_substitution") TEST_CASE_FIXTURE(Fixture, "do_not_bind_a_free_table_to_a_union_containing_that_table") { ScopedFastFlag flag[] = { - {"LuauStatFunctionSimplify4", true}, {"LuauLowerBoundsCalculation", true}, {"LuauDifferentOrderOfUnificationDoesntMatter2", true}, }; diff --git a/tests/TypeInfer.tryUnify.test.cpp b/tests/TypeInfer.tryUnify.test.cpp index 87562644..49deae71 100644 --- a/tests/TypeInfer.tryUnify.test.cpp +++ b/tests/TypeInfer.tryUnify.test.cpp @@ -196,7 +196,7 @@ TEST_CASE_FIXTURE(TryUnifyFixture, "variadics_should_use_reversed_properly") CHECK_EQ(toString(tm->wantedType), "string"); } -TEST_CASE_FIXTURE(TryUnifyFixture, "cli_41095_concat_log_in_sealed_table_unification") +TEST_CASE_FIXTURE(BuiltinsFixture, "cli_41095_concat_log_in_sealed_table_unification") { CheckResult result = check(R"( --!strict diff --git a/tests/TypeInfer.typePacks.cpp b/tests/TypeInfer.typePacks.cpp index f141622f..fd66b080 100644 --- a/tests/TypeInfer.typePacks.cpp +++ b/tests/TypeInfer.typePacks.cpp @@ -339,7 +339,7 @@ local c: Packed CHECK_EQ(toString(ttvC->instantiatedTypePackParams[0], {true}), "number, boolean"); } -TEST_CASE_FIXTURE(Fixture, "type_alias_type_packs_import") +TEST_CASE_FIXTURE(BuiltinsFixture, "type_alias_type_packs_import") { fileResolver.source["game/A"] = R"( export type Packed = { a: T, b: (U...) -> () } @@ -369,7 +369,7 @@ local d: { a: typeof(c) } CHECK_EQ(toString(requireType("d")), "{| a: Packed |}"); } -TEST_CASE_FIXTURE(Fixture, "type_pack_type_parameters") +TEST_CASE_FIXTURE(BuiltinsFixture, "type_pack_type_parameters") { fileResolver.source["game/A"] = R"( export type Packed = { a: T, b: (U...) -> () } @@ -784,7 +784,7 @@ local a: Y<...number> LUAU_REQUIRE_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "type_alias_default_export") +TEST_CASE_FIXTURE(BuiltinsFixture, "type_alias_default_export") { fileResolver.source["Module/Types"] = R"( export type A = { a: T, b: U } diff --git a/tests/TypeInfer.unionTypes.test.cpp b/tests/TypeInfer.unionTypes.test.cpp index 96bdd534..277f3887 100644 --- a/tests/TypeInfer.unionTypes.test.cpp +++ b/tests/TypeInfer.unionTypes.test.cpp @@ -104,7 +104,7 @@ TEST_CASE_FIXTURE(Fixture, "optional_arguments_table2") REQUIRE(!result.errors.empty()); } -TEST_CASE_FIXTURE(Fixture, "error_takes_optional_arguments") +TEST_CASE_FIXTURE(BuiltinsFixture, "error_takes_optional_arguments") { CheckResult result = check(R"( error("message") @@ -517,10 +517,8 @@ TEST_CASE_FIXTURE(Fixture, "dont_allow_cyclic_unions_to_be_inferred") LUAU_REQUIRE_NO_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "table_union_write_indirect") +TEST_CASE_FIXTURE(BuiltinsFixture, "table_union_write_indirect") { - ScopedFastFlag statFunctionSimplify{"LuauStatFunctionSimplify4", true}; - CheckResult result = check(R"( type A = { x: number, y: (number) -> string } | { z: number, y: (number) -> string } From ab4bb355a3f261d18ae8c0d09ce44a7af67ecaf9 Mon Sep 17 00:00:00 2001 From: JohnnyMorganz Date: Mon, 16 May 2022 17:50:15 +0100 Subject: [PATCH 249/399] Add `ToStringOptions.hideFunctionSelfArgument` (#486) Adds an option to hide the `self: type` argument as the first argument in the string representation of a named function type var if the ftv hasSelf. Also added in a test for the original output (i.e., if the option was disabled) I didn't apply this option in the normal `Luau::toString()` function, just the `Luau::toStringNamedFunction()` one (for my usecase, that is enough + I felt like a named function would include the method colon `:` to signify self). If this is unintuitive, I can also add it to the general `Luau::toString()` function. --- Analysis/include/Luau/ToString.h | 1 + Analysis/src/ToString.cpp | 8 +++++++ tests/ToString.test.cpp | 36 ++++++++++++++++++++++++++++++++ 3 files changed, 45 insertions(+) diff --git a/Analysis/include/Luau/ToString.h b/Analysis/include/Luau/ToString.h index f4db5e35..3b380a60 100644 --- a/Analysis/include/Luau/ToString.h +++ b/Analysis/include/Luau/ToString.h @@ -28,6 +28,7 @@ struct ToStringOptions bool functionTypeArguments = false; // If true, output function type argument names when they are available bool hideTableKind = false; // If true, all tables will be surrounded with plain '{}' bool hideNamedFunctionTypeParameters = false; // If true, type parameters of functions will be hidden at top-level. + bool hideFunctionSelfArgument = false; // If true, `self: X` will be omitted from the function signature if the function has self bool indent = false; size_t maxTableLength = size_t(FInt::LuauTableTypeMaximumStringifierLength); // Only applied to TableTypeVars size_t maxTypeLength = size_t(FInt::LuauTypeMaximumStringifierLength); diff --git a/Analysis/src/ToString.cpp b/Analysis/src/ToString.cpp index 51665f7f..51f3f69f 100644 --- a/Analysis/src/ToString.cpp +++ b/Analysis/src/ToString.cpp @@ -1230,6 +1230,14 @@ std::string toStringNamedFunction(const std::string& funcName, const FunctionTyp size_t idx = 0; while (argPackIter != end(ftv.argTypes)) { + // ftv takes a self parameter as the first argument, skip it if specified in option + if (idx == 0 && ftv.hasSelf && opts.hideFunctionSelfArgument) + { + ++argPackIter; + ++idx; + continue; + } + if (!first) state.emit(", "); first = false; diff --git a/tests/ToString.test.cpp b/tests/ToString.test.cpp index 50d0838e..4bdd45f7 100644 --- a/tests/ToString.test.cpp +++ b/tests/ToString.test.cpp @@ -617,4 +617,40 @@ TEST_CASE_FIXTURE(Fixture, "toStringNamedFunction_overrides_param_names") CHECK_EQ("test(first: a, second: string, ...: number): a", toStringNamedFunction("test", *ftv, opts)); } +TEST_CASE_FIXTURE(Fixture, "toStringNamedFunction_include_self_param") +{ + ScopedFastFlag flag{"LuauDocFuncParameters", true}; + CheckResult result = check(R"( + local foo = {} + function foo:method(arg: string): () + end + )"); + + TypeId parentTy = requireType("foo"); + auto ttv = get(follow(parentTy)); + auto ftv = get(ttv->props.at("method").type); + + CHECK_EQ("foo:method(self: a, arg: string): ()", toStringNamedFunction("foo:method", *ftv)); +} + + +TEST_CASE_FIXTURE(Fixture, "toStringNamedFunction_hide_self_param") +{ + ScopedFastFlag flag{"LuauDocFuncParameters", true}; + CheckResult result = check(R"( + local foo = {} + function foo:method(arg: string): () + end + )"); + + TypeId parentTy = requireType("foo"); + auto ttv = get(follow(parentTy)); + auto ftv = get(ttv->props.at("method").type); + + ToStringOptions opts; + opts.hideFunctionSelfArgument = true; + CHECK_EQ("foo:method(arg: string): ()", toStringNamedFunction("foo:method", *ftv, opts)); +} + + TEST_SUITE_END(); From f2191b9e4da6a4bb2d9d344ebd7941ec2f00844b Mon Sep 17 00:00:00 2001 From: JohnnyMorganz Date: Tue, 17 May 2022 19:22:54 +0100 Subject: [PATCH 250/399] Respect useLineBreaks for union/intersect toString (#487) * Respect useLineBreaks for union/intersect toString * Apply suggestions from code review Co-authored-by: Andy Friesen Co-authored-by: Andy Friesen --- Analysis/src/ToString.cpp | 10 ++++++++-- tests/ToString.test.cpp | 33 +++++++++++++++++++++++++++++++++ 2 files changed, 41 insertions(+), 2 deletions(-) diff --git a/Analysis/src/ToString.cpp b/Analysis/src/ToString.cpp index 51f3f69f..380ac456 100644 --- a/Analysis/src/ToString.cpp +++ b/Analysis/src/ToString.cpp @@ -745,7 +745,10 @@ struct TypeVarStringifier for (std::string& ss : results) { if (!first) - state.emit(" | "); + { + state.newline(); + state.emit("| "); + } state.emit(ss); first = false; } @@ -798,7 +801,10 @@ struct TypeVarStringifier for (std::string& ss : results) { if (!first) - state.emit(" & "); + { + state.newline(); + state.emit("& "); + } state.emit(ss); first = false; } diff --git a/tests/ToString.test.cpp b/tests/ToString.test.cpp index 4bdd45f7..f38dd10a 100644 --- a/tests/ToString.test.cpp +++ b/tests/ToString.test.cpp @@ -126,6 +126,39 @@ TEST_CASE_FIXTURE(Fixture, "functions_are_always_parenthesized_in_unions_or_inte CHECK_EQ(toString(&itv), "((number, string) -> (string, number)) & ((string, number) -> (number, string))"); } +TEST_CASE_FIXTURE(Fixture, "intersections_respects_use_line_breaks") +{ + CheckResult result = check(R"( + local a: ((string) -> string) & ((number) -> number) + )"); + + ToStringOptions opts; + opts.useLineBreaks = true; + + //clang-format off + CHECK_EQ("((number) -> number)\n" + "& ((string) -> string)", + toString(requireType("a"), opts)); + //clang-format on +} + +TEST_CASE_FIXTURE(Fixture, "unions_respects_use_line_breaks") +{ + CheckResult result = check(R"( + local a: string | number | boolean + )"); + + ToStringOptions opts; + opts.useLineBreaks = true; + + //clang-format off + CHECK_EQ("boolean\n" + "| number\n" + "| string", + toString(requireType("a"), opts)); + //clang-format on +} + TEST_CASE_FIXTURE(Fixture, "quit_stringifying_table_type_when_length_is_exceeded") { TableTypeVar ttv{}; From 8b4c6aabc271c8bee2910c5b752759b21a6bcca9 Mon Sep 17 00:00:00 2001 From: JohnnyMorganz Date: Thu, 19 May 2022 00:26:05 +0100 Subject: [PATCH 251/399] Fix findAstAncestry when position is at eof (#490) --- Analysis/src/AstQuery.cpp | 20 ++++++++++++++++++-- tests/AstQuery.test.cpp | 13 +++++++++++++ 2 files changed, 31 insertions(+), 2 deletions(-) diff --git a/Analysis/src/AstQuery.cpp b/Analysis/src/AstQuery.cpp index 0aed34c0..0522b1fa 100644 --- a/Analysis/src/AstQuery.cpp +++ b/Analysis/src/AstQuery.cpp @@ -71,9 +71,11 @@ struct FindFullAncestry final : public AstVisitor { std::vector nodes; Position pos; + Position documentEnd; - explicit FindFullAncestry(Position pos) + explicit FindFullAncestry(Position pos, Position documentEnd) : pos(pos) + , documentEnd(documentEnd) { } @@ -84,6 +86,16 @@ struct FindFullAncestry final : public AstVisitor nodes.push_back(node); return true; } + + // Edge case: If we ask for the node at the position that is the very end of the document + // return the innermost AST element that ends at that position. + + if (node->location.end == documentEnd && pos >= documentEnd) + { + nodes.push_back(node); + return true; + } + return false; } }; @@ -92,7 +104,11 @@ struct FindFullAncestry final : public AstVisitor std::vector findAstAncestryOfPosition(const SourceModule& source, Position pos) { - FindFullAncestry finder(pos); + const Position end = source.root->location.end; + if (pos > end) + pos = end; + + FindFullAncestry finder(pos, end); source.root->visit(&finder); return std::move(finder.nodes); } diff --git a/tests/AstQuery.test.cpp b/tests/AstQuery.test.cpp index 12c68450..f0017509 100644 --- a/tests/AstQuery.test.cpp +++ b/tests/AstQuery.test.cpp @@ -92,4 +92,17 @@ bar(foo()) CHECK_EQ("number", toString(*expectedOty)); } +TEST_CASE_FIXTURE(Fixture, "ast_ancestry_at_eof") +{ + check(R"( +if true then + )"); + + std::vector ancestry = findAstAncestryOfPosition(*getMainSourceModule(), Position(2, 4)); + REQUIRE_GE(ancestry.size(), 2); + AstStat* parentStat = ancestry[ancestry.size() - 2]->asStat(); + REQUIRE(bool(parentStat)); + REQUIRE(parentStat->is()); +} + TEST_SUITE_END(); From 7e9e697489c886773d289020afa210a1166ea7d6 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Thu, 19 May 2022 16:46:52 -0700 Subject: [PATCH 252/399] Sync to upstream/release/527 --- Analysis/include/Luau/Clone.h | 2 +- Analysis/include/Luau/Error.h | 4 +- Analysis/include/Luau/LValue.h | 4 - Analysis/include/Luau/Module.h | 36 +- Analysis/include/Luau/Substitution.h | 3 +- Analysis/include/Luau/ToString.h | 1 + Analysis/include/Luau/TxnLog.h | 11 - Analysis/include/Luau/TypeArena.h | 42 ++ Analysis/include/Luau/TypeInfer.h | 19 +- Analysis/include/Luau/Unifier.h | 4 +- Analysis/include/Luau/VisitTypeVar.h | 20 +- Analysis/src/AstQuery.cpp | 20 +- Analysis/src/BuiltinDefinitions.cpp | 53 +- Analysis/src/Clone.cpp | 44 +- Analysis/src/Error.cpp | 13 +- Analysis/src/IostreamHelpers.cpp | 2 +- Analysis/src/LValue.cpp | 21 - Analysis/src/Module.cpp | 123 ++-- Analysis/src/Normalize.cpp | 32 +- Analysis/src/Quantify.cpp | 39 +- Analysis/src/Substitution.cpp | 113 ++-- Analysis/src/ToString.cpp | 25 +- Analysis/src/TxnLog.cpp | 34 +- Analysis/src/TypeArena.cpp | 88 +++ Analysis/src/TypeInfer.cpp | 650 +++++---------------- Analysis/src/TypeUtils.cpp | 11 +- Analysis/src/TypeVar.cpp | 9 +- Analysis/src/Unifier.cpp | 84 +-- Compiler/include/Luau/BytecodeBuilder.h | 1 + Compiler/src/BytecodeBuilder.cpp | 10 + Compiler/src/Compiler.cpp | 289 ++++++--- Compiler/src/ConstantFolding.cpp | 4 +- Sources.cmake | 2 + VM/src/ltablib.cpp | 27 - VM/src/lvmexecute.cpp | 14 - tests/AstQuery.test.cpp | 13 + tests/Autocomplete.test.cpp | 4 + tests/Compiler.test.cpp | 235 ++++++-- tests/Module.test.cpp | 8 - tests/NonstrictMode.test.cpp | 4 - tests/Normalize.test.cpp | 45 ++ tests/RuntimeLimits.test.cpp | 2 - tests/ToString.test.cpp | 81 +++ tests/TypeInfer.builtins.test.cpp | 14 - tests/TypeInfer.functions.test.cpp | 2 - tests/TypeInfer.generics.test.cpp | 74 +++ tests/TypeInfer.intersectionTypes.test.cpp | 2 - tests/TypeInfer.loops.test.cpp | 2 - tests/TypeInfer.operators.test.cpp | 2 - tests/TypeInfer.provisional.test.cpp | 36 +- tests/TypeInfer.refinements.test.cpp | 143 +---- tests/TypeInfer.singletons.test.cpp | 25 +- tests/TypeInfer.tables.test.cpp | 34 +- tests/TypeInfer.test.cpp | 4 - tests/TypeInfer.unionTypes.test.cpp | 1 - 55 files changed, 1213 insertions(+), 1372 deletions(-) create mode 100644 Analysis/include/Luau/TypeArena.h create mode 100644 Analysis/src/TypeArena.cpp diff --git a/Analysis/include/Luau/Clone.h b/Analysis/include/Luau/Clone.h index 9b6ffa62..9fcbce04 100644 --- a/Analysis/include/Luau/Clone.h +++ b/Analysis/include/Luau/Clone.h @@ -1,6 +1,7 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #pragma once +#include "Luau/TypeArena.h" #include "Luau/TypeVar.h" #include @@ -18,7 +19,6 @@ struct CloneState SeenTypePacks seenTypePacks; int recursionCount = 0; - bool encounteredFreeType = false; // TODO: Remove with LuauLosslessClone. }; TypePackId clone(TypePackId tp, TypeArena& dest, CloneState& cloneState); diff --git a/Analysis/include/Luau/Error.h b/Analysis/include/Luau/Error.h index 70683141..b4530674 100644 --- a/Analysis/include/Luau/Error.h +++ b/Analysis/include/Luau/Error.h @@ -5,6 +5,7 @@ #include "Luau/Location.h" #include "Luau/TypeVar.h" #include "Luau/Variant.h" +#include "Luau/TypeArena.h" namespace Luau { @@ -108,9 +109,6 @@ struct FunctionDoesNotTakeSelf struct FunctionRequiresSelf { - // TODO: Delete with LuauAnyInIsOptionalIsOptional - int requiredExtraNils = 0; - bool operator==(const FunctionRequiresSelf& rhs) const; }; diff --git a/Analysis/include/Luau/LValue.h b/Analysis/include/Luau/LValue.h index afb71415..1a92d52d 100644 --- a/Analysis/include/Luau/LValue.h +++ b/Analysis/include/Luau/LValue.h @@ -34,10 +34,6 @@ const LValue* baseof(const LValue& lvalue); std::optional tryGetLValue(const class AstExpr& expr); -// Utility function: breaks down an LValue to get at the Symbol, and reverses the vector of keys. -// TODO: remove with FFlagLuauTypecheckOptPass -std::pair> getFullName(const LValue& lvalue); - // Utility function: breaks down an LValue to get at the Symbol Symbol getBaseSymbol(const LValue& lvalue); diff --git a/Analysis/include/Luau/Module.h b/Analysis/include/Luau/Module.h index 0dd44188..00e1e635 100644 --- a/Analysis/include/Luau/Module.h +++ b/Analysis/include/Luau/Module.h @@ -2,11 +2,10 @@ #pragma once #include "Luau/FileResolver.h" -#include "Luau/TypePack.h" -#include "Luau/TypedAllocator.h" #include "Luau/ParseOptions.h" #include "Luau/Error.h" #include "Luau/ParseResult.h" +#include "Luau/TypeArena.h" #include #include @@ -54,35 +53,6 @@ struct RequireCycle std::vector path; // one of the paths for a require() to go all the way back to the originating module }; -struct TypeArena -{ - TypedAllocator typeVars; - TypedAllocator typePacks; - - void clear(); - - template - TypeId addType(T tv) - { - if constexpr (std::is_same_v) - LUAU_ASSERT(tv.options.size() >= 2); - - return addTV(TypeVar(std::move(tv))); - } - - TypeId addTV(TypeVar&& tv); - - TypeId freshType(TypeLevel level); - - TypePackId addTypePack(std::initializer_list types); - TypePackId addTypePack(std::vector types); - TypePackId addTypePack(TypePack pack); - TypePackId addTypePack(TypePackVar pack); -}; - -void freeze(TypeArena& arena); -void unfreeze(TypeArena& arena); - struct Module { ~Module(); @@ -111,9 +81,7 @@ struct Module // Once a module has been typechecked, we clone its public interface into a separate arena. // This helps us to force TypeVar ownership into a DAG rather than a DCG. - // Returns true if there were any free types encountered in the public interface. This - // indicates a bug in the type checker that we want to surface. - bool clonePublicInterface(InternalErrorReporter& ice); + void clonePublicInterface(InternalErrorReporter& ice); }; } // namespace Luau diff --git a/Analysis/include/Luau/Substitution.h b/Analysis/include/Luau/Substitution.h index 6f5931e1..f3c3ae9a 100644 --- a/Analysis/include/Luau/Substitution.h +++ b/Analysis/include/Luau/Substitution.h @@ -1,8 +1,7 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #pragma once -#include "Luau/Module.h" -#include "Luau/ModuleResolver.h" +#include "Luau/TypeArena.h" #include "Luau/TypePack.h" #include "Luau/TypeVar.h" #include "Luau/DenseHash.h" diff --git a/Analysis/include/Luau/ToString.h b/Analysis/include/Luau/ToString.h index f4db5e35..3b380a60 100644 --- a/Analysis/include/Luau/ToString.h +++ b/Analysis/include/Luau/ToString.h @@ -28,6 +28,7 @@ struct ToStringOptions bool functionTypeArguments = false; // If true, output function type argument names when they are available bool hideTableKind = false; // If true, all tables will be surrounded with plain '{}' bool hideNamedFunctionTypeParameters = false; // If true, type parameters of functions will be hidden at top-level. + bool hideFunctionSelfArgument = false; // If true, `self: X` will be omitted from the function signature if the function has self bool indent = false; size_t maxTableLength = size_t(FInt::LuauTableTypeMaximumStringifierLength); // Only applied to TableTypeVars size_t maxTypeLength = size_t(FInt::LuauTypeMaximumStringifierLength); diff --git a/Analysis/include/Luau/TxnLog.h b/Analysis/include/Luau/TxnLog.h index 995ed6c6..cd115e3b 100644 --- a/Analysis/include/Luau/TxnLog.h +++ b/Analysis/include/Luau/TxnLog.h @@ -7,8 +7,6 @@ #include "Luau/TypeVar.h" #include "Luau/TypePack.h" -LUAU_FASTFLAG(LuauTypecheckOptPass) - namespace Luau { @@ -93,15 +91,6 @@ struct TxnLog { } - TxnLog(TxnLog* parent, std::vector>* sharedSeen) - : typeVarChanges(nullptr) - , typePackChanges(nullptr) - , parent(parent) - , sharedSeen(sharedSeen) - { - LUAU_ASSERT(!FFlag::LuauTypecheckOptPass); - } - TxnLog(const TxnLog&) = delete; TxnLog& operator=(const TxnLog&) = delete; diff --git a/Analysis/include/Luau/TypeArena.h b/Analysis/include/Luau/TypeArena.h new file mode 100644 index 00000000..7c74158b --- /dev/null +++ b/Analysis/include/Luau/TypeArena.h @@ -0,0 +1,42 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +#pragma once + +#include "Luau/TypedAllocator.h" +#include "Luau/TypeVar.h" +#include "Luau/TypePack.h" + +#include + +namespace Luau +{ + +struct TypeArena +{ + TypedAllocator typeVars; + TypedAllocator typePacks; + + void clear(); + + template + TypeId addType(T tv) + { + if constexpr (std::is_same_v) + LUAU_ASSERT(tv.options.size() >= 2); + + return addTV(TypeVar(std::move(tv))); + } + + TypeId addTV(TypeVar&& tv); + + TypeId freshType(TypeLevel level); + + TypePackId addTypePack(std::initializer_list types); + TypePackId addTypePack(std::vector types); + TypePackId addTypePack(TypePack pack); + TypePackId addTypePack(TypePackVar pack); +}; + +void freeze(TypeArena& arena); +void unfreeze(TypeArena& arena); + +} diff --git a/Analysis/include/Luau/TypeInfer.h b/Analysis/include/Luau/TypeInfer.h index ac880135..fcaf5baa 100644 --- a/Analysis/include/Luau/TypeInfer.h +++ b/Analysis/include/Luau/TypeInfer.h @@ -187,7 +187,6 @@ struct TypeChecker ExprResult checkExpr(const ScopePtr& scope, const AstExprIndexExpr& expr); ExprResult checkExpr(const ScopePtr& scope, const AstExprFunction& expr, std::optional expectedType = std::nullopt); ExprResult checkExpr(const ScopePtr& scope, const AstExprTable& expr, std::optional expectedType = std::nullopt); - ExprResult checkExpr_(const ScopePtr& scope, const AstExprTable& expr, std::optional expectedType = std::nullopt); ExprResult checkExpr(const ScopePtr& scope, const AstExprUnary& expr); TypeId checkRelationalOperation( const ScopePtr& scope, const AstExprBinary& expr, TypeId lhsType, TypeId rhsType, const PredicateVec& predicates = {}); @@ -395,7 +394,7 @@ private: const AstArray& genericNames, const AstArray& genericPackNames, bool useCache = false); public: - ErrorVec resolve(const PredicateVec& predicates, const ScopePtr& scope, bool sense); + void resolve(const PredicateVec& predicates, const ScopePtr& scope, bool sense); private: void refineLValue(const LValue& lvalue, RefinementMap& refis, const ScopePtr& scope, TypeIdPredicate predicate); @@ -403,14 +402,14 @@ private: std::optional resolveLValue(const ScopePtr& scope, const LValue& lvalue); std::optional resolveLValue(const RefinementMap& refis, const ScopePtr& scope, const LValue& lvalue); - void resolve(const PredicateVec& predicates, ErrorVec& errVec, RefinementMap& refis, const ScopePtr& scope, bool sense, bool fromOr = false); - void resolve(const Predicate& predicate, ErrorVec& errVec, RefinementMap& refis, const ScopePtr& scope, bool sense, bool fromOr); - void resolve(const TruthyPredicate& truthyP, ErrorVec& errVec, RefinementMap& refis, const ScopePtr& scope, bool sense, bool fromOr); - void resolve(const AndPredicate& andP, ErrorVec& errVec, RefinementMap& refis, const ScopePtr& scope, bool sense); - void resolve(const OrPredicate& orP, ErrorVec& errVec, RefinementMap& refis, const ScopePtr& scope, bool sense); - void resolve(const IsAPredicate& isaP, ErrorVec& errVec, RefinementMap& refis, const ScopePtr& scope, bool sense); - void resolve(const TypeGuardPredicate& typeguardP, ErrorVec& errVec, RefinementMap& refis, const ScopePtr& scope, bool sense); - void resolve(const EqPredicate& eqP, ErrorVec& errVec, RefinementMap& refis, const ScopePtr& scope, bool sense); + void resolve(const PredicateVec& predicates, RefinementMap& refis, const ScopePtr& scope, bool sense, bool fromOr = false); + void resolve(const Predicate& predicate, RefinementMap& refis, const ScopePtr& scope, bool sense, bool fromOr); + void resolve(const TruthyPredicate& truthyP, RefinementMap& refis, const ScopePtr& scope, bool sense, bool fromOr); + void resolve(const AndPredicate& andP, RefinementMap& refis, const ScopePtr& scope, bool sense); + void resolve(const OrPredicate& orP, RefinementMap& refis, const ScopePtr& scope, bool sense); + void resolve(const IsAPredicate& isaP, RefinementMap& refis, const ScopePtr& scope, bool sense); + void resolve(const TypeGuardPredicate& typeguardP, RefinementMap& refis, const ScopePtr& scope, bool sense); + void resolve(const EqPredicate& eqP, RefinementMap& refis, const ScopePtr& scope, bool sense); bool isNonstrictMode() const; bool useConstrainedIntersections() const; diff --git a/Analysis/include/Luau/Unifier.h b/Analysis/include/Luau/Unifier.h index 418d4ca4..0e24c8b0 100644 --- a/Analysis/include/Luau/Unifier.h +++ b/Analysis/include/Luau/Unifier.h @@ -5,7 +5,7 @@ #include "Luau/Location.h" #include "Luau/TxnLog.h" #include "Luau/TypeInfer.h" -#include "Luau/Module.h" // FIXME: For TypeArena. It merits breaking out into its own header. +#include "Luau/TypeArena.h" #include "Luau/UnifierSharedState.h" #include @@ -55,8 +55,6 @@ struct Unifier UnifierSharedState& sharedState; Unifier(TypeArena* types, Mode mode, const Location& location, Variance variance, UnifierSharedState& sharedState, TxnLog* parentLog = nullptr); - Unifier(TypeArena* types, Mode mode, std::vector>* sharedSeen, const Location& location, Variance variance, - UnifierSharedState& sharedState, TxnLog* parentLog = nullptr); // Test whether the two type vars unify. Never commits the result. ErrorVec canUnify(TypeId subTy, TypeId superTy); diff --git a/Analysis/include/Luau/VisitTypeVar.h b/Analysis/include/Luau/VisitTypeVar.h index 67fce5ed..2e98f526 100644 --- a/Analysis/include/Luau/VisitTypeVar.h +++ b/Analysis/include/Luau/VisitTypeVar.h @@ -10,6 +10,7 @@ LUAU_FASTFLAG(LuauUseVisitRecursionLimit) LUAU_FASTINT(LuauVisitRecursionLimit) +LUAU_FASTFLAG(LuauNormalizeFlagIsConservative) namespace Luau { @@ -471,18 +472,21 @@ struct GenericTypeVarVisitor else if (auto pack = get(tp)) { - visit(tp, *pack); + bool res = visit(tp, *pack); + if (!FFlag::LuauNormalizeFlagIsConservative || res) + { + for (TypeId ty : pack->head) + traverse(ty); - for (TypeId ty : pack->head) - traverse(ty); - - if (pack->tail) - traverse(*pack->tail); + if (pack->tail) + traverse(*pack->tail); + } } else if (auto pack = get(tp)) { - visit(tp, *pack); - traverse(pack->ty); + bool res = visit(tp, *pack); + if (!FFlag::LuauNormalizeFlagIsConservative || res) + traverse(pack->ty); } else LUAU_ASSERT(!"GenericTypeVarVisitor::traverse(TypePackId) is not exhaustive!"); diff --git a/Analysis/src/AstQuery.cpp b/Analysis/src/AstQuery.cpp index 0aed34c0..0522b1fa 100644 --- a/Analysis/src/AstQuery.cpp +++ b/Analysis/src/AstQuery.cpp @@ -71,9 +71,11 @@ struct FindFullAncestry final : public AstVisitor { std::vector nodes; Position pos; + Position documentEnd; - explicit FindFullAncestry(Position pos) + explicit FindFullAncestry(Position pos, Position documentEnd) : pos(pos) + , documentEnd(documentEnd) { } @@ -84,6 +86,16 @@ struct FindFullAncestry final : public AstVisitor nodes.push_back(node); return true; } + + // Edge case: If we ask for the node at the position that is the very end of the document + // return the innermost AST element that ends at that position. + + if (node->location.end == documentEnd && pos >= documentEnd) + { + nodes.push_back(node); + return true; + } + return false; } }; @@ -92,7 +104,11 @@ struct FindFullAncestry final : public AstVisitor std::vector findAstAncestryOfPosition(const SourceModule& source, Position pos) { - FindFullAncestry finder(pos); + const Position end = source.root->location.end; + if (pos > end) + pos = end; + + FindFullAncestry finder(pos, end); source.root->visit(&finder); return std::move(finder.nodes); } diff --git a/Analysis/src/BuiltinDefinitions.cpp b/Analysis/src/BuiltinDefinitions.cpp index 3895b01b..5ed6de67 100644 --- a/Analysis/src/BuiltinDefinitions.cpp +++ b/Analysis/src/BuiltinDefinitions.cpp @@ -8,7 +8,6 @@ #include -LUAU_FASTFLAG(LuauAssertStripsFalsyTypes) LUAU_FASTFLAGVARIABLE(LuauSetMetaTableArgsCheck, false) /** FIXME: Many of these type definitions are not quite completely accurate. @@ -408,41 +407,29 @@ static std::optional> magicFunctionAssert( { auto [paramPack, predicates] = exprResult; - if (FFlag::LuauAssertStripsFalsyTypes) + TypeArena& arena = typechecker.currentModule->internalTypes; + + auto [head, tail] = flatten(paramPack); + if (head.empty() && tail) { - TypeArena& arena = typechecker.currentModule->internalTypes; - - auto [head, tail] = flatten(paramPack); - if (head.empty() && tail) - { - std::optional fst = first(*tail); - if (!fst) - return ExprResult{paramPack}; - head.push_back(*fst); - } - - typechecker.reportErrors(typechecker.resolve(predicates, scope, true)); - - if (head.size() > 0) - { - std::optional newhead = typechecker.pickTypesFromSense(head[0], true); - if (!newhead) - head = {typechecker.nilType}; - else - head[0] = *newhead; - } - - return ExprResult{arena.addTypePack(TypePack{std::move(head), tail})}; - } - else - { - if (expr.args.size < 1) + std::optional fst = first(*tail); + if (!fst) return ExprResult{paramPack}; - - typechecker.reportErrors(typechecker.resolve(predicates, scope, true)); - - return ExprResult{paramPack}; + head.push_back(*fst); } + + typechecker.resolve(predicates, scope, true); + + if (head.size() > 0) + { + std::optional newhead = typechecker.pickTypesFromSense(head[0], true); + if (!newhead) + head = {typechecker.nilType}; + else + head[0] = *newhead; + } + + return ExprResult{arena.addTypePack(TypePack{std::move(head), tail})}; } static std::optional> magicFunctionPack( diff --git a/Analysis/src/Clone.cpp b/Analysis/src/Clone.cpp index 1aa556eb..a3611f53 100644 --- a/Analysis/src/Clone.cpp +++ b/Analysis/src/Clone.cpp @@ -1,7 +1,6 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #include "Luau/Clone.h" -#include "Luau/Module.h" #include "Luau/RecursionCounter.h" #include "Luau/TypePack.h" #include "Luau/Unifiable.h" @@ -9,8 +8,6 @@ LUAU_FASTFLAG(DebugLuauCopyBeforeNormalizing) LUAU_FASTINTVARIABLE(LuauTypeCloneRecursionLimit, 300) -LUAU_FASTFLAG(LuauTypecheckOptPass) -LUAU_FASTFLAGVARIABLE(LuauLosslessClone, false) LUAU_FASTFLAG(LuauNoMethodLocations) namespace Luau @@ -89,20 +86,8 @@ struct TypePackCloner void operator()(const Unifiable::Free& t) { - if (FFlag::LuauLosslessClone) - { - defaultClone(t); - } - else - { - cloneState.encounteredFreeType = true; - - TypePackId err = getSingletonTypes().errorRecoveryTypePack(getSingletonTypes().anyTypePack); - TypePackId cloned = dest.addTypePack(*err); - seenTypePacks[typePackId] = cloned; - } + defaultClone(t); } - void operator()(const Unifiable::Generic& t) { defaultClone(t); @@ -152,18 +137,7 @@ void TypeCloner::defaultClone(const T& t) void TypeCloner::operator()(const Unifiable::Free& t) { - if (FFlag::LuauLosslessClone) - { - defaultClone(t); - } - else - { - cloneState.encounteredFreeType = true; - - TypeId err = getSingletonTypes().errorRecoveryType(getSingletonTypes().anyType); - TypeId cloned = dest.addType(*err); - seenTypes[typeId] = cloned; - } + defaultClone(t); } void TypeCloner::operator()(const Unifiable::Generic& t) @@ -191,9 +165,6 @@ void TypeCloner::operator()(const PrimitiveTypeVar& t) void TypeCloner::operator()(const ConstrainedTypeVar& t) { - if (!FFlag::LuauLosslessClone) - cloneState.encounteredFreeType = true; - TypeId res = dest.addType(ConstrainedTypeVar{t.level}); ConstrainedTypeVar* ctv = getMutable(res); LUAU_ASSERT(ctv); @@ -230,9 +201,7 @@ void TypeCloner::operator()(const FunctionTypeVar& t) ftv->argTypes = clone(t.argTypes, dest, cloneState); ftv->argNames = t.argNames; ftv->retType = clone(t.retType, dest, cloneState); - - if (FFlag::LuauTypecheckOptPass) - ftv->hasNoGenerics = t.hasNoGenerics; + ftv->hasNoGenerics = t.hasNoGenerics; } void TypeCloner::operator()(const TableTypeVar& t) @@ -270,13 +239,6 @@ void TypeCloner::operator()(const TableTypeVar& t) for (TypePackId& arg : ttv->instantiatedTypePackParams) arg = clone(arg, dest, cloneState); - if (!FFlag::LuauLosslessClone && ttv->state == TableState::Free) - { - cloneState.encounteredFreeType = true; - - ttv->state = TableState::Sealed; - } - ttv->definitionModuleName = t.definitionModuleName; if (!FFlag::LuauNoMethodLocations) ttv->methodDefinitionLocations = t.methodDefinitionLocations; diff --git a/Analysis/src/Error.cpp b/Analysis/src/Error.cpp index 24ed4ac1..f443a3cc 100644 --- a/Analysis/src/Error.cpp +++ b/Analysis/src/Error.cpp @@ -2,7 +2,6 @@ #include "Luau/Error.h" #include "Luau/Clone.h" -#include "Luau/Module.h" #include "Luau/StringUtils.h" #include "Luau/ToString.h" @@ -178,15 +177,7 @@ struct ErrorConverter std::string operator()(const Luau::FunctionRequiresSelf& e) const { - if (e.requiredExtraNils) - { - const char* plural = e.requiredExtraNils == 1 ? "" : "s"; - return format("This function was declared to accept self, but you did not pass enough arguments. Use a colon instead of a dot or " - "pass %i extra nil%s to suppress this warning", - e.requiredExtraNils, plural); - } - else - return "This function must be called with self. Did you mean to use a colon instead of a dot?"; + return "This function must be called with self. Did you mean to use a colon instead of a dot?"; } std::string operator()(const Luau::OccursCheckFailed&) const @@ -539,7 +530,7 @@ bool FunctionDoesNotTakeSelf::operator==(const FunctionDoesNotTakeSelf&) const bool FunctionRequiresSelf::operator==(const FunctionRequiresSelf& e) const { - return requiredExtraNils == e.requiredExtraNils; + return true; } bool OccursCheckFailed::operator==(const OccursCheckFailed&) const diff --git a/Analysis/src/IostreamHelpers.cpp b/Analysis/src/IostreamHelpers.cpp index 0eaa485e..048167ae 100644 --- a/Analysis/src/IostreamHelpers.cpp +++ b/Analysis/src/IostreamHelpers.cpp @@ -48,7 +48,7 @@ static void errorToString(std::ostream& stream, const T& err) else if constexpr (std::is_same_v) stream << "FunctionDoesNotTakeSelf { }"; else if constexpr (std::is_same_v) - stream << "FunctionRequiresSelf { extraNils " << err.requiredExtraNils << " }"; + stream << "FunctionRequiresSelf { }"; else if constexpr (std::is_same_v) stream << "OccursCheckFailed { }"; else if constexpr (std::is_same_v) diff --git a/Analysis/src/LValue.cpp b/Analysis/src/LValue.cpp index 72555ab4..38dfe1ae 100644 --- a/Analysis/src/LValue.cpp +++ b/Analysis/src/LValue.cpp @@ -5,8 +5,6 @@ #include -LUAU_FASTFLAG(LuauTypecheckOptPass) - namespace Luau { @@ -79,27 +77,8 @@ std::optional tryGetLValue(const AstExpr& node) return std::nullopt; } -std::pair> getFullName(const LValue& lvalue) -{ - LUAU_ASSERT(!FFlag::LuauTypecheckOptPass); - - const LValue* current = &lvalue; - std::vector keys; - while (auto field = get(*current)) - { - keys.push_back(field->key); - current = baseof(*current); - } - - const Symbol* symbol = get(*current); - LUAU_ASSERT(symbol); - return {*symbol, std::vector(keys.rbegin(), keys.rend())}; -} - Symbol getBaseSymbol(const LValue& lvalue) { - LUAU_ASSERT(FFlag::LuauTypecheckOptPass); - const LValue* current = &lvalue; while (auto field = get(*current)) current = baseof(*current); diff --git a/Analysis/src/Module.cpp b/Analysis/src/Module.cpp index bafd4371..074a41e6 100644 --- a/Analysis/src/Module.cpp +++ b/Analysis/src/Module.cpp @@ -13,9 +13,8 @@ #include -LUAU_FASTFLAGVARIABLE(DebugLuauFreezeArena, false) -LUAU_FASTFLAG(LuauLowerBoundsCalculation) -LUAU_FASTFLAG(LuauLosslessClone) +LUAU_FASTFLAG(LuauLowerBoundsCalculation); +LUAU_FASTFLAG(LuauNormalizeFlagIsConservative); namespace Luau { @@ -55,89 +54,25 @@ bool isWithinComment(const SourceModule& sourceModule, Position pos) return contains(pos, *iter); } -void TypeArena::clear() +struct ForceNormal : TypeVarOnceVisitor { - typeVars.clear(); - typePacks.clear(); -} + bool visit(TypeId ty) override + { + asMutable(ty)->normal = true; + return true; + } -TypeId TypeArena::addTV(TypeVar&& tv) -{ - TypeId allocated = typeVars.allocate(std::move(tv)); + bool visit(TypeId ty, const FreeTypeVar& ftv) override + { + visit(ty); + return true; + } - asMutable(allocated)->owningArena = this; - - return allocated; -} - -TypeId TypeArena::freshType(TypeLevel level) -{ - TypeId allocated = typeVars.allocate(FreeTypeVar{level}); - - asMutable(allocated)->owningArena = this; - - return allocated; -} - -TypePackId TypeArena::addTypePack(std::initializer_list types) -{ - TypePackId allocated = typePacks.allocate(TypePack{std::move(types)}); - - asMutable(allocated)->owningArena = this; - - return allocated; -} - -TypePackId TypeArena::addTypePack(std::vector types) -{ - TypePackId allocated = typePacks.allocate(TypePack{std::move(types)}); - - asMutable(allocated)->owningArena = this; - - return allocated; -} - -TypePackId TypeArena::addTypePack(TypePack tp) -{ - TypePackId allocated = typePacks.allocate(std::move(tp)); - - asMutable(allocated)->owningArena = this; - - return allocated; -} - -TypePackId TypeArena::addTypePack(TypePackVar tp) -{ - TypePackId allocated = typePacks.allocate(std::move(tp)); - - asMutable(allocated)->owningArena = this; - - return allocated; -} - -ScopePtr Module::getModuleScope() const -{ - LUAU_ASSERT(!scopes.empty()); - return scopes.front().second; -} - -void freeze(TypeArena& arena) -{ - if (!FFlag::DebugLuauFreezeArena) - return; - - arena.typeVars.freeze(); - arena.typePacks.freeze(); -} - -void unfreeze(TypeArena& arena) -{ - if (!FFlag::DebugLuauFreezeArena) - return; - - arena.typeVars.unfreeze(); - arena.typePacks.unfreeze(); -} + bool visit(TypePackId tp, const FreeTypePack& ftp) override + { + return true; + } +}; Module::~Module() { @@ -145,7 +80,7 @@ Module::~Module() unfreeze(internalTypes); } -bool Module::clonePublicInterface(InternalErrorReporter& ice) +void Module::clonePublicInterface(InternalErrorReporter& ice) { LUAU_ASSERT(interfaceTypes.typeVars.empty()); LUAU_ASSERT(interfaceTypes.typePacks.empty()); @@ -165,11 +100,22 @@ bool Module::clonePublicInterface(InternalErrorReporter& ice) normalize(*moduleScope->varargPack, interfaceTypes, ice); } + ForceNormal forceNormal; + for (auto& [name, tf] : moduleScope->exportedTypeBindings) { tf = clone(tf, interfaceTypes, cloneState); if (FFlag::LuauLowerBoundsCalculation) + { normalize(tf.type, interfaceTypes, ice); + + if (FFlag::LuauNormalizeFlagIsConservative) + { + // We're about to freeze the memory. We know that the flag is conservative by design. Cyclic tables + // won't be marked normal. If the types aren't normal by now, they never will be. + forceNormal.traverse(tf.type); + } + } } for (TypeId ty : moduleScope->returnType) @@ -191,11 +137,12 @@ bool Module::clonePublicInterface(InternalErrorReporter& ice) freeze(internalTypes); freeze(interfaceTypes); +} - if (FFlag::LuauLosslessClone) - return false; // TODO: make function return void. - else - return cloneState.encounteredFreeType; +ScopePtr Module::getModuleScope() const +{ + LUAU_ASSERT(!scopes.empty()); + return scopes.front().second; } } // namespace Luau diff --git a/Analysis/src/Normalize.cpp b/Analysis/src/Normalize.cpp index ef5377a1..30fd4af2 100644 --- a/Analysis/src/Normalize.cpp +++ b/Analysis/src/Normalize.cpp @@ -14,6 +14,7 @@ LUAU_FASTFLAGVARIABLE(DebugLuauCopyBeforeNormalizing, false) // This could theoretically be 2000 on amd64, but x86 requires this. LUAU_FASTINTVARIABLE(LuauNormalizeIterationLimit, 1200); LUAU_FASTFLAGVARIABLE(LuauNormalizeCombineTableFix, false); +LUAU_FASTFLAGVARIABLE(LuauNormalizeFlagIsConservative, false); namespace Luau { @@ -260,8 +261,13 @@ static bool areNormal_(const T& t, const std::unordered_set& seen, Intern if (count >= FInt::LuauNormalizeIterationLimit) ice.ice("Luau::areNormal hit iteration limit"); - // The follow is here because a bound type may not be normal, but the bound type is normal. - return ty->normal || follow(ty)->normal || seen.find(asMutable(ty)) != seen.end(); + if (FFlag::LuauNormalizeFlagIsConservative) + return ty->normal; + else + { + // The follow is here because a bound type may not be normal, but the bound type is normal. + return ty->normal || follow(ty)->normal || seen.find(asMutable(ty)) != seen.end(); + } }; return std::all_of(begin(t), end(t), isNormal); @@ -1003,8 +1009,15 @@ std::pair normalize(TypeId ty, TypeArena& arena, InternalErrorRepo (void)clone(ty, arena, state); Normalize n{arena, ice}; - std::unordered_set seen; - DEPRECATED_visitTypeVar(ty, n, seen); + if (FFlag::LuauNormalizeFlagIsConservative) + { + DEPRECATED_visitTypeVar(ty, n); + } + else + { + std::unordered_set seen; + DEPRECATED_visitTypeVar(ty, n, seen); + } return {ty, !n.limitExceeded}; } @@ -1028,8 +1041,15 @@ std::pair normalize(TypePackId tp, TypeArena& arena, InternalE (void)clone(tp, arena, state); Normalize n{arena, ice}; - std::unordered_set seen; - DEPRECATED_visitTypeVar(tp, n, seen); + if (FFlag::LuauNormalizeFlagIsConservative) + { + DEPRECATED_visitTypeVar(tp, n); + } + else + { + std::unordered_set seen; + DEPRECATED_visitTypeVar(tp, n, seen); + } return {tp, !n.limitExceeded}; } diff --git a/Analysis/src/Quantify.cpp b/Analysis/src/Quantify.cpp index 4f3e4469..018d5632 100644 --- a/Analysis/src/Quantify.cpp +++ b/Analysis/src/Quantify.cpp @@ -4,7 +4,7 @@ #include "Luau/VisitTypeVar.h" -LUAU_FASTFLAG(LuauTypecheckOptPass) +LUAU_FASTFLAG(LuauAlwaysQuantify) namespace Luau { @@ -59,8 +59,7 @@ struct Quantifier final : TypeVarOnceVisitor bool visit(TypeId ty, const FreeTypeVar& ftv) override { - if (FFlag::LuauTypecheckOptPass) - seenMutableType = true; + seenMutableType = true; if (!level.subsumes(ftv.level)) return false; @@ -76,20 +75,17 @@ struct Quantifier final : TypeVarOnceVisitor LUAU_ASSERT(getMutable(ty)); TableTypeVar& ttv = *getMutable(ty); - if (FFlag::LuauTypecheckOptPass) - { - if (ttv.state == TableState::Generic) - seenGenericType = true; + if (ttv.state == TableState::Generic) + seenGenericType = true; - if (ttv.state == TableState::Free) - seenMutableType = true; - } + if (ttv.state == TableState::Free) + seenMutableType = true; if (ttv.state == TableState::Sealed || ttv.state == TableState::Generic) return false; if (!level.subsumes(ttv.level)) { - if (FFlag::LuauTypecheckOptPass && ttv.state == TableState::Unsealed) + if (ttv.state == TableState::Unsealed) seenMutableType = true; return false; } @@ -97,9 +93,7 @@ struct Quantifier final : TypeVarOnceVisitor if (ttv.state == TableState::Free) { ttv.state = TableState::Generic; - - if (FFlag::LuauTypecheckOptPass) - seenGenericType = true; + seenGenericType = true; } else if (ttv.state == TableState::Unsealed) ttv.state = TableState::Sealed; @@ -111,8 +105,7 @@ struct Quantifier final : TypeVarOnceVisitor bool visit(TypePackId tp, const FreeTypePack& ftp) override { - if (FFlag::LuauTypecheckOptPass) - seenMutableType = true; + seenMutableType = true; if (!level.subsumes(ftp.level)) return false; @@ -131,10 +124,18 @@ void quantify(TypeId ty, TypeLevel level) FunctionTypeVar* ftv = getMutable(ty); LUAU_ASSERT(ftv); - ftv->generics = q.generics; - ftv->genericPacks = q.genericPacks; + if (FFlag::LuauAlwaysQuantify) + { + ftv->generics.insert(ftv->generics.end(), q.generics.begin(), q.generics.end()); + ftv->genericPacks.insert(ftv->genericPacks.end(), q.genericPacks.begin(), q.genericPacks.end()); + } + else + { + ftv->generics = q.generics; + ftv->genericPacks = q.genericPacks; + } - if (FFlag::LuauTypecheckOptPass && ftv->generics.empty() && ftv->genericPacks.empty() && !q.seenMutableType && !q.seenGenericType) + if (ftv->generics.empty() && ftv->genericPacks.empty() && !q.seenMutableType && !q.seenGenericType) ftv->hasNoGenerics = true; } diff --git a/Analysis/src/Substitution.cpp b/Analysis/src/Substitution.cpp index c5c7977a..e40bedb0 100644 --- a/Analysis/src/Substitution.cpp +++ b/Analysis/src/Substitution.cpp @@ -9,9 +9,6 @@ LUAU_FASTFLAG(LuauLowerBoundsCalculation) LUAU_FASTINTVARIABLE(LuauTarjanChildLimit, 10000) -LUAU_FASTFLAG(LuauTypecheckOptPass) -LUAU_FASTFLAGVARIABLE(LuauSubstituteFollowNewTypes, false) -LUAU_FASTFLAGVARIABLE(LuauSubstituteFollowPossibleMutations, false) LUAU_FASTFLAG(LuauNoMethodLocations) namespace Luau @@ -19,26 +16,20 @@ namespace Luau void Tarjan::visitChildren(TypeId ty, int index) { - if (FFlag::LuauTypecheckOptPass) - LUAU_ASSERT(ty == log->follow(ty)); - else - ty = log->follow(ty); + LUAU_ASSERT(ty == log->follow(ty)); if (ignoreChildren(ty)) return; - if (FFlag::LuauTypecheckOptPass) - { - if (auto pty = log->pending(ty)) - ty = &pty->pending; - } + if (auto pty = log->pending(ty)) + ty = &pty->pending; - if (const FunctionTypeVar* ftv = FFlag::LuauTypecheckOptPass ? get(ty) : log->getMutable(ty)) + if (const FunctionTypeVar* ftv = get(ty)) { visitChild(ftv->argTypes); visitChild(ftv->retType); } - else if (const TableTypeVar* ttv = FFlag::LuauTypecheckOptPass ? get(ty) : log->getMutable(ty)) + else if (const TableTypeVar* ttv = get(ty)) { LUAU_ASSERT(!ttv->boundTo); for (const auto& [name, prop] : ttv->props) @@ -55,17 +46,17 @@ void Tarjan::visitChildren(TypeId ty, int index) for (TypePackId itp : ttv->instantiatedTypePackParams) visitChild(itp); } - else if (const MetatableTypeVar* mtv = FFlag::LuauTypecheckOptPass ? get(ty) : log->getMutable(ty)) + else if (const MetatableTypeVar* mtv = get(ty)) { visitChild(mtv->table); visitChild(mtv->metatable); } - else if (const UnionTypeVar* utv = FFlag::LuauTypecheckOptPass ? get(ty) : log->getMutable(ty)) + else if (const UnionTypeVar* utv = get(ty)) { for (TypeId opt : utv->options) visitChild(opt); } - else if (const IntersectionTypeVar* itv = FFlag::LuauTypecheckOptPass ? get(ty) : log->getMutable(ty)) + else if (const IntersectionTypeVar* itv = get(ty)) { for (TypeId part : itv->parts) visitChild(part); @@ -79,28 +70,22 @@ void Tarjan::visitChildren(TypeId ty, int index) void Tarjan::visitChildren(TypePackId tp, int index) { - if (FFlag::LuauTypecheckOptPass) - LUAU_ASSERT(tp == log->follow(tp)); - else - tp = log->follow(tp); + LUAU_ASSERT(tp == log->follow(tp)); if (ignoreChildren(tp)) return; - if (FFlag::LuauTypecheckOptPass) - { - if (auto ptp = log->pending(tp)) - tp = &ptp->pending; - } + if (auto ptp = log->pending(tp)) + tp = &ptp->pending; - if (const TypePack* tpp = FFlag::LuauTypecheckOptPass ? get(tp) : log->getMutable(tp)) + if (const TypePack* tpp = get(tp)) { for (TypeId tv : tpp->head) visitChild(tv); if (tpp->tail) visitChild(*tpp->tail); } - else if (const VariadicTypePack* vtp = FFlag::LuauTypecheckOptPass ? get(tp) : log->getMutable(tp)) + else if (const VariadicTypePack* vtp = get(tp)) { visitChild(vtp->ty); } @@ -108,10 +93,7 @@ void Tarjan::visitChildren(TypePackId tp, int index) std::pair Tarjan::indexify(TypeId ty) { - if (FFlag::LuauTypecheckOptPass && !FFlag::LuauSubstituteFollowPossibleMutations) - LUAU_ASSERT(ty == log->follow(ty)); - else - ty = log->follow(ty); + ty = log->follow(ty); bool fresh = !typeToIndex.contains(ty); int& index = typeToIndex[ty]; @@ -129,10 +111,7 @@ std::pair Tarjan::indexify(TypeId ty) std::pair Tarjan::indexify(TypePackId tp) { - if (FFlag::LuauTypecheckOptPass && !FFlag::LuauSubstituteFollowPossibleMutations) - LUAU_ASSERT(tp == log->follow(tp)); - else - tp = log->follow(tp); + tp = log->follow(tp); bool fresh = !packToIndex.contains(tp); int& index = packToIndex[tp]; @@ -150,8 +129,7 @@ std::pair Tarjan::indexify(TypePackId tp) void Tarjan::visitChild(TypeId ty) { - if (!FFlag::LuauSubstituteFollowPossibleMutations) - ty = log->follow(ty); + ty = log->follow(ty); edgesTy.push_back(ty); edgesTp.push_back(nullptr); @@ -159,8 +137,7 @@ void Tarjan::visitChild(TypeId ty) void Tarjan::visitChild(TypePackId tp) { - if (!FFlag::LuauSubstituteFollowPossibleMutations) - tp = log->follow(tp); + tp = log->follow(tp); edgesTy.push_back(nullptr); edgesTp.push_back(tp); @@ -389,13 +366,10 @@ TypeId Substitution::clone(TypeId ty) TypeId result = ty; - if (FFlag::LuauTypecheckOptPass) - { - if (auto pty = log->pending(ty)) - ty = &pty->pending; - } + if (auto pty = log->pending(ty)) + ty = &pty->pending; - if (const FunctionTypeVar* ftv = FFlag::LuauTypecheckOptPass ? get(ty) : log->getMutable(ty)) + if (const FunctionTypeVar* ftv = get(ty)) { FunctionTypeVar clone = FunctionTypeVar{ftv->level, ftv->argTypes, ftv->retType, ftv->definition, ftv->hasSelf}; clone.generics = ftv->generics; @@ -405,7 +379,7 @@ TypeId Substitution::clone(TypeId ty) clone.argNames = ftv->argNames; result = addType(std::move(clone)); } - else if (const TableTypeVar* ttv = FFlag::LuauTypecheckOptPass ? get(ty) : log->getMutable(ty)) + else if (const TableTypeVar* ttv = get(ty)) { LUAU_ASSERT(!ttv->boundTo); TableTypeVar clone = TableTypeVar{ttv->props, ttv->indexer, ttv->level, ttv->state}; @@ -419,19 +393,19 @@ TypeId Substitution::clone(TypeId ty) clone.tags = ttv->tags; result = addType(std::move(clone)); } - else if (const MetatableTypeVar* mtv = FFlag::LuauTypecheckOptPass ? get(ty) : log->getMutable(ty)) + else if (const MetatableTypeVar* mtv = get(ty)) { MetatableTypeVar clone = MetatableTypeVar{mtv->table, mtv->metatable}; clone.syntheticName = mtv->syntheticName; result = addType(std::move(clone)); } - else if (const UnionTypeVar* utv = FFlag::LuauTypecheckOptPass ? get(ty) : log->getMutable(ty)) + else if (const UnionTypeVar* utv = get(ty)) { UnionTypeVar clone; clone.options = utv->options; result = addType(std::move(clone)); } - else if (const IntersectionTypeVar* itv = FFlag::LuauTypecheckOptPass ? get(ty) : log->getMutable(ty)) + else if (const IntersectionTypeVar* itv = get(ty)) { IntersectionTypeVar clone; clone.parts = itv->parts; @@ -451,20 +425,17 @@ TypePackId Substitution::clone(TypePackId tp) { tp = log->follow(tp); - if (FFlag::LuauTypecheckOptPass) - { - if (auto ptp = log->pending(tp)) - tp = &ptp->pending; - } + if (auto ptp = log->pending(tp)) + tp = &ptp->pending; - if (const TypePack* tpp = FFlag::LuauTypecheckOptPass ? get(tp) : log->getMutable(tp)) + if (const TypePack* tpp = get(tp)) { TypePack clone; clone.head = tpp->head; clone.tail = tpp->tail; return addTypePack(std::move(clone)); } - else if (const VariadicTypePack* vtp = FFlag::LuauTypecheckOptPass ? get(tp) : log->getMutable(tp)) + else if (const VariadicTypePack* vtp = get(tp)) { VariadicTypePack clone; clone.ty = vtp->ty; @@ -476,28 +447,22 @@ TypePackId Substitution::clone(TypePackId tp) void Substitution::foundDirty(TypeId ty) { - if (FFlag::LuauTypecheckOptPass && !FFlag::LuauSubstituteFollowPossibleMutations) - LUAU_ASSERT(ty == log->follow(ty)); - else - ty = log->follow(ty); + ty = log->follow(ty); if (isDirty(ty)) - newTypes[ty] = FFlag::LuauSubstituteFollowNewTypes ? follow(clean(ty)) : clean(ty); + newTypes[ty] = follow(clean(ty)); else - newTypes[ty] = FFlag::LuauSubstituteFollowNewTypes ? follow(clone(ty)) : clone(ty); + newTypes[ty] = follow(clone(ty)); } void Substitution::foundDirty(TypePackId tp) { - if (FFlag::LuauTypecheckOptPass && !FFlag::LuauSubstituteFollowPossibleMutations) - LUAU_ASSERT(tp == log->follow(tp)); - else - tp = log->follow(tp); + tp = log->follow(tp); if (isDirty(tp)) - newPacks[tp] = FFlag::LuauSubstituteFollowNewTypes ? follow(clean(tp)) : clean(tp); + newPacks[tp] = follow(clean(tp)); else - newPacks[tp] = FFlag::LuauSubstituteFollowNewTypes ? follow(clone(tp)) : clone(tp); + newPacks[tp] = follow(clone(tp)); } TypeId Substitution::replace(TypeId ty) @@ -525,10 +490,7 @@ void Substitution::replaceChildren(TypeId ty) if (BoundTypeVar* btv = log->getMutable(ty); FFlag::LuauLowerBoundsCalculation && btv) btv->boundTo = replace(btv->boundTo); - if (FFlag::LuauTypecheckOptPass) - LUAU_ASSERT(ty == log->follow(ty)); - else - ty = log->follow(ty); + LUAU_ASSERT(ty == log->follow(ty)); if (ignoreChildren(ty)) return; @@ -579,10 +541,7 @@ void Substitution::replaceChildren(TypeId ty) void Substitution::replaceChildren(TypePackId tp) { - if (FFlag::LuauTypecheckOptPass) - LUAU_ASSERT(tp == log->follow(tp)); - else - tp = log->follow(tp); + LUAU_ASSERT(tp == log->follow(tp)); if (ignoreChildren(tp)) return; diff --git a/Analysis/src/ToString.cpp b/Analysis/src/ToString.cpp index 51665f7f..f90f7019 100644 --- a/Analysis/src/ToString.cpp +++ b/Analysis/src/ToString.cpp @@ -219,6 +219,8 @@ struct StringifierState return generateName(s); } + int previousNameIndex = 0; + std::string getName(TypePackId ty) { const size_t s = result.nameMap.typePacks.size(); @@ -228,9 +230,10 @@ struct StringifierState for (int count = 0; count < 256; ++count) { - std::string candidate = generateName(usedNames.size() + count); + std::string candidate = generateName(previousNameIndex + count); if (!usedNames.count(candidate)) { + previousNameIndex += count; usedNames.insert(candidate); n = candidate; return candidate; @@ -399,6 +402,7 @@ struct TypeVarStringifier { if (gtv.explicitName) { + state.usedNames.insert(gtv.name); state.result.nameMap.typeVars[ty] = gtv.name; state.emit(gtv.name); } @@ -745,7 +749,10 @@ struct TypeVarStringifier for (std::string& ss : results) { if (!first) - state.emit(" | "); + { + state.newline(); + state.emit("| "); + } state.emit(ss); first = false; } @@ -798,7 +805,10 @@ struct TypeVarStringifier for (std::string& ss : results) { if (!first) - state.emit(" & "); + { + state.newline(); + state.emit("& "); + } state.emit(ss); first = false; } @@ -937,6 +947,7 @@ struct TypePackStringifier state.emit("gen-"); if (pack.explicitName) { + state.usedNames.insert(pack.name); state.result.nameMap.typePacks[tp] = pack.name; state.emit(pack.name); } @@ -1230,6 +1241,14 @@ std::string toStringNamedFunction(const std::string& funcName, const FunctionTyp size_t idx = 0; while (argPackIter != end(ftv.argTypes)) { + // ftv takes a self parameter as the first argument, skip it if specified in option + if (idx == 0 && ftv.hasSelf && opts.hideFunctionSelfArgument) + { + ++argPackIter; + ++idx; + continue; + } + if (!first) state.emit(", "); first = false; diff --git a/Analysis/src/TxnLog.cpp b/Analysis/src/TxnLog.cpp index 1fb5a61a..e45c0cbd 100644 --- a/Analysis/src/TxnLog.cpp +++ b/Analysis/src/TxnLog.cpp @@ -7,8 +7,6 @@ #include #include -LUAU_FASTFLAGVARIABLE(LuauJustOneCallFrameForHaveSeen, false) - namespace Luau { @@ -150,37 +148,13 @@ void TxnLog::popSeen(TypePackId lhs, TypePackId rhs) bool TxnLog::haveSeen(TypeOrPackId lhs, TypeOrPackId rhs) const { - if (FFlag::LuauJustOneCallFrameForHaveSeen && !FFlag::LuauTypecheckOptPass) + const std::pair sortedPair = (lhs > rhs) ? std::make_pair(lhs, rhs) : std::make_pair(rhs, lhs); + if (sharedSeen->end() != std::find(sharedSeen->begin(), sharedSeen->end(), sortedPair)) { - // This function will technically work if `this` is nullptr, but this - // indicates a bug, so we explicitly assert. - LUAU_ASSERT(static_cast(this) != nullptr); - - const std::pair sortedPair = (lhs > rhs) ? std::make_pair(lhs, rhs) : std::make_pair(rhs, lhs); - - for (const TxnLog* current = this; current; current = current->parent) - { - if (current->sharedSeen->end() != std::find(current->sharedSeen->begin(), current->sharedSeen->end(), sortedPair)) - return true; - } - - return false; + return true; } - else - { - const std::pair sortedPair = (lhs > rhs) ? std::make_pair(lhs, rhs) : std::make_pair(rhs, lhs); - if (sharedSeen->end() != std::find(sharedSeen->begin(), sharedSeen->end(), sortedPair)) - { - return true; - } - if (!FFlag::LuauTypecheckOptPass && parent) - { - return parent->haveSeen(lhs, rhs); - } - - return false; - } + return false; } void TxnLog::pushSeen(TypeOrPackId lhs, TypeOrPackId rhs) diff --git a/Analysis/src/TypeArena.cpp b/Analysis/src/TypeArena.cpp new file mode 100644 index 00000000..673b002d --- /dev/null +++ b/Analysis/src/TypeArena.cpp @@ -0,0 +1,88 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details + +#include "Luau/TypeArena.h" + +LUAU_FASTFLAGVARIABLE(DebugLuauFreezeArena, false); + +namespace Luau +{ + +void TypeArena::clear() +{ + typeVars.clear(); + typePacks.clear(); +} + +TypeId TypeArena::addTV(TypeVar&& tv) +{ + TypeId allocated = typeVars.allocate(std::move(tv)); + + asMutable(allocated)->owningArena = this; + + return allocated; +} + +TypeId TypeArena::freshType(TypeLevel level) +{ + TypeId allocated = typeVars.allocate(FreeTypeVar{level}); + + asMutable(allocated)->owningArena = this; + + return allocated; +} + +TypePackId TypeArena::addTypePack(std::initializer_list types) +{ + TypePackId allocated = typePacks.allocate(TypePack{std::move(types)}); + + asMutable(allocated)->owningArena = this; + + return allocated; +} + +TypePackId TypeArena::addTypePack(std::vector types) +{ + TypePackId allocated = typePacks.allocate(TypePack{std::move(types)}); + + asMutable(allocated)->owningArena = this; + + return allocated; +} + +TypePackId TypeArena::addTypePack(TypePack tp) +{ + TypePackId allocated = typePacks.allocate(std::move(tp)); + + asMutable(allocated)->owningArena = this; + + return allocated; +} + +TypePackId TypeArena::addTypePack(TypePackVar tp) +{ + TypePackId allocated = typePacks.allocate(std::move(tp)); + + asMutable(allocated)->owningArena = this; + + return allocated; +} + +void freeze(TypeArena& arena) +{ + if (!FFlag::DebugLuauFreezeArena) + return; + + arena.typeVars.freeze(); + arena.typePacks.freeze(); +} + +void unfreeze(TypeArena& arena) +{ + if (!FFlag::DebugLuauFreezeArena) + return; + + arena.typeVars.unfreeze(); + arena.typePacks.unfreeze(); +} + +} diff --git a/Analysis/src/TypeInfer.cpp b/Analysis/src/TypeInfer.cpp index a13abd53..208b3f2f 100644 --- a/Analysis/src/TypeInfer.cpp +++ b/Analysis/src/TypeInfer.cpp @@ -32,32 +32,25 @@ LUAU_FASTFLAG(LuauKnowsTheDataModel3) LUAU_FASTFLAG(LuauSeparateTypechecks) LUAU_FASTFLAG(LuauAutocompleteDynamicLimits) LUAU_FASTFLAGVARIABLE(LuauDoNotRelyOnNextBinding, false) -LUAU_FASTFLAGVARIABLE(LuauEqConstraint, false) +LUAU_FASTFLAGVARIABLE(LuauExpectedPropTypeFromIndexer, false) LUAU_FASTFLAGVARIABLE(LuauWeakEqConstraint, false) // Eventually removed as false. LUAU_FASTFLAGVARIABLE(LuauLowerBoundsCalculation, false) LUAU_FASTFLAGVARIABLE(DebugLuauFreezeDuringUnification, false) -LUAU_FASTFLAGVARIABLE(LuauInstantiateFollows, false) LUAU_FASTFLAGVARIABLE(LuauSelfCallAutocompleteFix, false) -LUAU_FASTFLAGVARIABLE(LuauDiscriminableUnions2, false) LUAU_FASTFLAGVARIABLE(LuauReduceUnionRecursion, false) LUAU_FASTFLAGVARIABLE(LuauOnlyMutateInstantiatedTables, false) -LUAU_FASTFLAGVARIABLE(LuauTypecheckOptPass, false) LUAU_FASTFLAGVARIABLE(LuauUnsealedTableLiteral, false) LUAU_FASTFLAGVARIABLE(LuauTwoPassAliasDefinitionFix, false) -LUAU_FASTFLAGVARIABLE(LuauAssertStripsFalsyTypes, false) LUAU_FASTFLAGVARIABLE(LuauReturnAnyInsteadOfICE, false) // Eventually removed as false. +LUAU_FASTFLAG(LuauNormalizeFlagIsConservative) LUAU_FASTFLAG(LuauWidenIfSupertypeIsFree2) -LUAU_FASTFLAGVARIABLE(LuauDoNotTryToReduce, false) -LUAU_FASTFLAGVARIABLE(LuauDoNotAccidentallyDependOnPointerOrdering, false) -LUAU_FASTFLAGVARIABLE(LuauCheckImplicitNumbericKeys, false) -LUAU_FASTFLAG(LuauAnyInIsOptionalIsOptional) -LUAU_FASTFLAGVARIABLE(LuauTableUseCounterInstead, false) LUAU_FASTFLAGVARIABLE(LuauReturnTypeInferenceInNonstrict, false) LUAU_FASTFLAGVARIABLE(LuauRecursionLimitException, false); -LUAU_FASTFLAG(LuauLosslessClone) +LUAU_FASTFLAGVARIABLE(LuauApplyTypeFunctionFix, false); LUAU_FASTFLAGVARIABLE(LuauTypecheckIter, false); LUAU_FASTFLAGVARIABLE(LuauSuccessTypingForEqualityOperations, false) LUAU_FASTFLAGVARIABLE(LuauNoMethodLocations, false); +LUAU_FASTFLAGVARIABLE(LuauAlwaysQuantify, false); namespace Luau { @@ -371,12 +364,7 @@ ModulePtr TypeChecker::checkWithoutRecursionCheck(const SourceModule& module, Mo prepareErrorsForDisplay(currentModule->errors); - bool encounteredFreeType = currentModule->clonePublicInterface(*iceHandler); - if (!FFlag::LuauLosslessClone && encounteredFreeType) - { - reportError(TypeError{module.root->location, - GenericError{"Free types leaked into this module's public interface. This is an internal Luau error; please report it."}}); - } + currentModule->clonePublicInterface(*iceHandler); // Clear unifier cache since it's keyed off internal types that get deallocated // This avoids fake cross-module cache hits and keeps cache size at bay when typechecking large module graphs. @@ -701,7 +689,7 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatIf& statement) ExprResult result = checkExpr(scope, *statement.condition); ScopePtr ifScope = childScope(scope, statement.thenbody->location); - reportErrors(resolve(result.predicates, ifScope, true)); + resolve(result.predicates, ifScope, true); check(ifScope, *statement.thenbody); if (statement.elsebody) @@ -734,7 +722,7 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatWhile& statement) ExprResult result = checkExpr(scope, *statement.condition); ScopePtr whileScope = childScope(scope, statement.body->location); - reportErrors(resolve(result.predicates, whileScope, true)); + resolve(result.predicates, whileScope, true); check(whileScope, *statement.body); } @@ -1154,10 +1142,7 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatForIn& forin) } else { - if (FFlag::LuauInstantiateFollows) - iterTy = instantiate(scope, checkExpr(scope, *firstValue).type, firstValue->location); - else - iterTy = follow(instantiate(scope, checkExpr(scope, *firstValue).type, firstValue->location)); + iterTy = instantiate(scope, checkExpr(scope, *firstValue).type, firstValue->location); } if (FFlag::LuauTypecheckIter) @@ -1849,23 +1834,11 @@ std::optional TypeChecker::getIndexTypeFromType( tablify(type); - if (FFlag::LuauDiscriminableUnions2) + if (isString(type)) { - if (isString(type)) - { - std::optional mtIndex = findMetatableEntry(stringType, "__index", location); - LUAU_ASSERT(mtIndex); - type = *mtIndex; - } - } - else - { - const PrimitiveTypeVar* primitiveType = get(type); - if (primitiveType && primitiveType->type == PrimitiveTypeVar::String) - { - if (std::optional mtIndex = findMetatableEntry(type, "__index", location)) - type = *mtIndex; - } + std::optional mtIndex = findMetatableEntry(stringType, "__index", location); + LUAU_ASSERT(mtIndex); + type = *mtIndex; } if (TableTypeVar* tableType = getMutableTableType(type)) @@ -1966,23 +1939,10 @@ std::optional TypeChecker::getIndexTypeFromType( return std::nullopt; } - if (FFlag::LuauDoNotTryToReduce) - { - if (parts.size() == 1) - return parts[0]; + if (parts.size() == 1) + return parts[0]; - return addType(IntersectionTypeVar{std::move(parts)}); // Not at all correct. - } - else - { - // TODO(amccord): Write some logic to correctly handle intersections. CLI-34659 - std::vector result = reduceUnion(parts); - - if (result.size() == 1) - return result[0]; - - return addType(IntersectionTypeVar{result}); - } + return addType(IntersectionTypeVar{std::move(parts)}); // Not at all correct. } if (addErrors) @@ -1993,103 +1953,55 @@ std::optional TypeChecker::getIndexTypeFromType( std::vector TypeChecker::reduceUnion(const std::vector& types) { - if (FFlag::LuauDoNotAccidentallyDependOnPointerOrdering) + std::vector result; + for (TypeId t : types) { - std::vector result; - for (TypeId t : types) + t = follow(t); + if (get(t) || get(t)) + return {t}; + + if (const UnionTypeVar* utv = get(t)) { - t = follow(t); - if (get(t) || get(t)) - return {t}; - - if (const UnionTypeVar* utv = get(t)) + if (FFlag::LuauReduceUnionRecursion) { - if (FFlag::LuauReduceUnionRecursion) + for (TypeId ty : utv) { - for (TypeId ty : utv) - { - if (get(ty) || get(ty)) - return {ty}; - - if (result.end() == std::find(result.begin(), result.end(), ty)) - result.push_back(ty); - } - } - else - { - std::vector r = reduceUnion(utv->options); - for (TypeId ty : r) - { + if (FFlag::LuauNormalizeFlagIsConservative) ty = follow(ty); - if (get(ty) || get(ty)) - return {ty}; + if (get(ty) || get(ty)) + return {ty}; - if (std::find(result.begin(), result.end(), ty) == result.end()) - result.push_back(ty); - } + if (result.end() == std::find(result.begin(), result.end(), ty)) + result.push_back(ty); } } - else if (std::find(result.begin(), result.end(), t) == result.end()) - result.push_back(t); - } - - return result; - } - else - { - std::set s; - - for (TypeId t : types) - { - if (const UnionTypeVar* utv = get(follow(t))) + else { std::vector r = reduceUnion(utv->options); for (TypeId ty : r) - s.insert(ty); + { + ty = follow(ty); + if (get(ty) || get(ty)) + return {ty}; + + if (std::find(result.begin(), result.end(), ty) == result.end()) + result.push_back(ty); + } } - else - s.insert(t); } - - // If any of them are ErrorTypeVars/AnyTypeVars, decay into them. - for (TypeId t : s) - { - t = follow(t); - if (get(t) || get(t)) - return {t}; - } - - std::vector r(s.begin(), s.end()); - std::sort(r.begin(), r.end()); - return r; + else if (std::find(result.begin(), result.end(), t) == result.end()) + result.push_back(t); } + + return result; } std::optional TypeChecker::tryStripUnionFromNil(TypeId ty) { if (const UnionTypeVar* utv = get(ty)) { - if (FFlag::LuauAnyInIsOptionalIsOptional) - { - if (!std::any_of(begin(utv), end(utv), isNil)) - return ty; - } - else - { - bool hasNil = false; - - for (TypeId option : utv) - { - if (isNil(option)) - { - hasNil = true; - break; - } - } - - if (!hasNil) - return ty; - } + if (!std::any_of(begin(utv), end(utv), isNil)) + return ty; std::vector result; @@ -2110,32 +2022,18 @@ std::optional TypeChecker::tryStripUnionFromNil(TypeId ty) TypeId TypeChecker::stripFromNilAndReport(TypeId ty, const Location& location) { - if (FFlag::LuauAnyInIsOptionalIsOptional) + ty = follow(ty); + + if (auto utv = get(ty)) { - ty = follow(ty); - - if (auto utv = get(ty)) - { - if (!std::any_of(begin(utv), end(utv), isNil)) - return ty; - } - - if (std::optional strippedUnion = tryStripUnionFromNil(ty)) - { - reportError(location, OptionalValueAccess{ty}); - return follow(*strippedUnion); - } + if (!std::any_of(begin(utv), end(utv), isNil)) + return ty; } - else + + if (std::optional strippedUnion = tryStripUnionFromNil(ty)) { - if (isOptional(ty)) - { - if (std::optional strippedUnion = tryStripUnionFromNil(follow(ty))) - { - reportError(location, OptionalValueAccess{ty}); - return follow(*strippedUnion); - } - } + reportError(location, OptionalValueAccess{ty}); + return follow(*strippedUnion); } return ty; @@ -2194,8 +2092,7 @@ TypeId TypeChecker::checkExprTable( if (indexer) { - if (FFlag::LuauCheckImplicitNumbericKeys) - unify(numberType, indexer->indexType, value->location); + unify(numberType, indexer->indexType, value->location); unify(valueType, indexer->indexResultType, value->location); } else @@ -2219,7 +2116,8 @@ TypeId TypeChecker::checkExprTable( if (errors.empty()) exprType = expectedProp.type; } - else if (expectedTable->indexer && isString(expectedTable->indexer->indexType)) + else if (expectedTable->indexer && (FFlag::LuauExpectedPropTypeFromIndexer ? maybeString(expectedTable->indexer->indexType) + : isString(expectedTable->indexer->indexType))) { ErrorVec errors = tryUnify(exprType, expectedTable->indexer->indexResultType, k->location); if (errors.empty()) @@ -2259,26 +2157,13 @@ TypeId TypeChecker::checkExprTable( ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprTable& expr, std::optional expectedType) { - if (FFlag::LuauTableUseCounterInstead) + RecursionCounter _rc(&checkRecursionCount); + if (FInt::LuauCheckRecursionLimit > 0 && checkRecursionCount >= FInt::LuauCheckRecursionLimit) { - RecursionCounter _rc(&checkRecursionCount); - if (FInt::LuauCheckRecursionLimit > 0 && checkRecursionCount >= FInt::LuauCheckRecursionLimit) - { - reportErrorCodeTooComplex(expr.location); - return {errorRecoveryType(scope)}; - } - - return checkExpr_(scope, expr, expectedType); + reportErrorCodeTooComplex(expr.location); + return {errorRecoveryType(scope)}; } - else - { - RecursionLimiter _rl(&recursionCount, FInt::LuauTypeInferRecursionLimit, "checkExpr for tables"); - return checkExpr_(scope, expr, expectedType); - } -} -ExprResult TypeChecker::checkExpr_(const ScopePtr& scope, const AstExprTable& expr, std::optional expectedType) -{ std::vector> fieldTypes(expr.items.size); const TableTypeVar* expectedTable = nullptr; @@ -2324,6 +2209,8 @@ ExprResult TypeChecker::checkExpr_(const ScopePtr& scope, const AstExprT { if (auto prop = expectedTable->props.find(key->value.data); prop != expectedTable->props.end()) expectedResultType = prop->second.type; + else if (FFlag::LuauExpectedPropTypeFromIndexer && expectedIndexType && maybeString(*expectedIndexType)) + expectedResultType = expectedIndexResultType; } else if (expectedUnion) { @@ -2529,7 +2416,7 @@ TypeId TypeChecker::checkRelationalOperation( if (expr.op == AstExprBinary::Or && subexp->op == AstExprBinary::And) { ScopePtr subScope = childScope(scope, subexp->location); - reportErrors(resolve(predicates, subScope, true)); + resolve(predicates, subScope, true); return unionOfTypes(rhsType, stripNil(checkExpr(subScope, *subexp->right).type, true), expr.location); } } @@ -2851,8 +2738,7 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprBi auto [rhsTy, rhsPredicates] = checkExpr(innerScope, *expr.right); - return {checkBinaryOperation(FFlag::LuauDiscriminableUnions2 ? scope : innerScope, expr, lhsTy, rhsTy), - {AndPredicate{std::move(lhsPredicates), std::move(rhsPredicates)}}}; + return {checkBinaryOperation(scope, expr, lhsTy, rhsTy), {AndPredicate{std::move(lhsPredicates), std::move(rhsPredicates)}}}; } else if (expr.op == AstExprBinary::Or) { @@ -2864,7 +2750,7 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprBi auto [rhsTy, rhsPredicates] = checkExpr(innerScope, *expr.right); // Because of C++, I'm not sure if lhsPredicates was not moved out by the time we call checkBinaryOperation. - TypeId result = checkBinaryOperation(FFlag::LuauDiscriminableUnions2 ? scope : innerScope, expr, lhsTy, rhsTy, lhsPredicates); + TypeId result = checkBinaryOperation(scope, expr, lhsTy, rhsTy, lhsPredicates); return {result, {OrPredicate{std::move(lhsPredicates), std::move(rhsPredicates)}}}; } else if (expr.op == AstExprBinary::CompareEq || expr.op == AstExprBinary::CompareNe) @@ -2872,8 +2758,8 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprBi if (auto predicate = tryGetTypeGuardPredicate(expr)) return {booleanType, {std::move(*predicate)}}; - ExprResult lhs = checkExpr(scope, *expr.left, std::nullopt, /*forceSingleton=*/FFlag::LuauDiscriminableUnions2); - ExprResult rhs = checkExpr(scope, *expr.right, std::nullopt, /*forceSingleton=*/FFlag::LuauDiscriminableUnions2); + ExprResult lhs = checkExpr(scope, *expr.left, std::nullopt, /*forceSingleton=*/true); + ExprResult rhs = checkExpr(scope, *expr.right, std::nullopt, /*forceSingleton=*/true); PredicateVec predicates; @@ -2931,12 +2817,12 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprEr ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprIfElse& expr, std::optional expectedType) { ExprResult result = checkExpr(scope, *expr.condition); + ScopePtr trueScope = childScope(scope, expr.trueExpr->location); - reportErrors(resolve(result.predicates, trueScope, true)); + resolve(result.predicates, trueScope, true); ExprResult trueType = checkExpr(trueScope, *expr.trueExpr, expectedType); ScopePtr falseScope = childScope(scope, expr.falseExpr->location); - // Don't report errors for this scope to avoid potentially duplicating errors reported for the first scope. resolve(result.predicates, falseScope, false); ExprResult falseType = checkExpr(falseScope, *expr.falseExpr, expectedType); @@ -3668,9 +3554,6 @@ void TypeChecker::checkArgumentList( else if (state.log.getMutable(t)) { } // ok - else if (!FFlag::LuauAnyInIsOptionalIsOptional && isNonstrictMode() && state.log.get(t)) - { - } // ok else { size_t minParams = getMinParameterCount(&state.log, paramPack); @@ -3823,9 +3706,6 @@ ExprResult TypeChecker::checkExprPack(const ScopePtr& scope, const A actualFunctionType = instantiate(scope, functionType, expr.func->location); } - if (!FFlag::LuauInstantiateFollows) - actualFunctionType = follow(actualFunctionType); - TypePackId retPack; if (FFlag::LuauLowerBoundsCalculation || !FFlag::LuauWidenIfSupertypeIsFree2) { @@ -4096,32 +3976,6 @@ std::optional> TypeChecker::checkCallOverload(const Scope { state.log.commit(); - if (!FFlag::LuauAnyInIsOptionalIsOptional && isNonstrictMode() && !expr.self && expr.func->is() && ftv->hasSelf) - { - // If we are running in nonstrict mode, passing fewer arguments than the function is declared to take AND - // the function is declared with colon notation AND we use dot notation, warn. - auto [providedArgs, providedTail] = flatten(argPack); - - // If we have a variadic tail, we can't say how many arguments were actually provided - if (!providedTail) - { - std::vector actualArgs = flatten(ftv->argTypes).first; - - size_t providedCount = providedArgs.size(); - size_t requiredCount = actualArgs.size(); - - // Ignore optional arguments - while (providedCount < requiredCount && requiredCount != 0 && isOptional(actualArgs[requiredCount - 1])) - requiredCount--; - - if (providedCount < requiredCount) - { - int requiredExtraNils = int(requiredCount - providedCount); - reportError(TypeError{expr.func->location, FunctionRequiresSelf{requiredExtraNils}}); - } - } - } - currentModule->astOverloadResolvedTypes[&expr] = fn; // We select this overload @@ -4525,7 +4379,7 @@ bool Instantiation::isDirty(TypeId ty) { if (const FunctionTypeVar* ftv = log->getMutable(ty)) { - if (FFlag::LuauTypecheckOptPass && ftv->hasNoGenerics) + if (ftv->hasNoGenerics) return false; return true; @@ -4582,7 +4436,7 @@ bool ReplaceGenerics::ignoreChildren(TypeId ty) { if (const FunctionTypeVar* ftv = log->getMutable(ty)) { - if (FFlag::LuauTypecheckOptPass && ftv->hasNoGenerics) + if (ftv->hasNoGenerics) return true; // We aren't recursing in the case of a generic function which @@ -4701,8 +4555,17 @@ TypeId TypeChecker::quantify(const ScopePtr& scope, TypeId ty, Location location ty = follow(ty); const FunctionTypeVar* ftv = get(ty); - if (ftv && ftv->generics.empty() && ftv->genericPacks.empty()) - Luau::quantify(ty, scope->level); + + if (FFlag::LuauAlwaysQuantify) + { + if (ftv) + Luau::quantify(ty, scope->level); + } + else + { + if (ftv && ftv->generics.empty() && ftv->genericPacks.empty()) + Luau::quantify(ty, scope->level); + } if (FFlag::LuauLowerBoundsCalculation && ftv) { @@ -4717,15 +4580,11 @@ TypeId TypeChecker::quantify(const ScopePtr& scope, TypeId ty, Location location TypeId TypeChecker::instantiate(const ScopePtr& scope, TypeId ty, Location location, const TxnLog* log) { - if (FFlag::LuauInstantiateFollows) - ty = follow(ty); + ty = follow(ty); - if (FFlag::LuauTypecheckOptPass) - { - const FunctionTypeVar* ftv = get(FFlag::LuauInstantiateFollows ? ty : follow(ty)); - if (ftv && ftv->hasNoGenerics) - return ty; - } + const FunctionTypeVar* ftv = get(ty); + if (ftv && ftv->hasNoGenerics) + return ty; Instantiation instantiation{log, ¤tModule->internalTypes, scope->level}; @@ -5392,10 +5251,9 @@ TypePackId TypeChecker::resolveTypePack(const ScopePtr& scope, const AstTypePack bool ApplyTypeFunction::isDirty(TypeId ty) { - // Really this should just replace the arguments, - // but for bug-compatibility with existing code, we replace - // all generics. - if (get(ty)) + if (FFlag::LuauApplyTypeFunctionFix && typeArguments.count(ty)) + return true; + else if (!FFlag::LuauApplyTypeFunctionFix && get(ty)) return true; else if (const FreeTypeVar* ftv = get(ty)) { @@ -5409,10 +5267,9 @@ bool ApplyTypeFunction::isDirty(TypeId ty) bool ApplyTypeFunction::isDirty(TypePackId tp) { - // Really this should just replace the arguments, - // but for bug-compatibility with existing code, we replace - // all generics. - if (get(tp)) + if (FFlag::LuauApplyTypeFunctionFix && typePackArguments.count(tp)) + return true; + else if (!FFlag::LuauApplyTypeFunctionFix && get(tp)) return true; else return false; @@ -5436,11 +5293,13 @@ bool ApplyTypeFunction::ignoreChildren(TypePackId tp) TypeId ApplyTypeFunction::clean(TypeId ty) { - // Really this should just replace the arguments, - // but for bug-compatibility with existing code, we replace - // all generics by free type variables. TypeId& arg = typeArguments[ty]; - if (arg) + if (FFlag::LuauApplyTypeFunctionFix) + { + LUAU_ASSERT(arg); + return arg; + } + else if (arg) return arg; else return addType(FreeTypeVar{level}); @@ -5448,11 +5307,13 @@ TypeId ApplyTypeFunction::clean(TypeId ty) TypePackId ApplyTypeFunction::clean(TypePackId tp) { - // Really this should just replace the arguments, - // but for bug-compatibility with existing code, we replace - // all generics by free type variables. TypePackId& arg = typePackArguments[tp]; - if (arg) + if (FFlag::LuauApplyTypeFunctionFix) + { + LUAU_ASSERT(arg); + return arg; + } + else if (arg) return arg; else return addTypePack(FreeTypePack{level}); @@ -5596,8 +5457,6 @@ GenericTypeDefinitions TypeChecker::createGenericTypes(const ScopePtr& scope, st void TypeChecker::refineLValue(const LValue& lvalue, RefinementMap& refis, const ScopePtr& scope, TypeIdPredicate predicate) { - LUAU_ASSERT(FFlag::LuauDiscriminableUnions2 || FFlag::LuauAssertStripsFalsyTypes); - const LValue* target = &lvalue; std::optional key; // If set, we know we took the base of the lvalue path and should be walking down each option of the base's type. @@ -5683,66 +5542,6 @@ std::optional TypeChecker::resolveLValue(const ScopePtr& scope, const LV // We need to search in the provided Scope. Find t.x.y first. // We fail to find t.x.y. Try t.x. We found it. Now we must return the type of the property y from the mapped-to type of t.x. // If we completely fail to find the Symbol t but the Scope has that entry, then we should walk that all the way through and terminate. - if (!FFlag::LuauTypecheckOptPass) - { - const auto& [symbol, keys] = getFullName(lvalue); - - ScopePtr currentScope = scope; - while (currentScope) - { - std::optional found; - - std::vector childKeys; - const LValue* currentLValue = &lvalue; - while (currentLValue) - { - if (auto it = currentScope->refinements.find(*currentLValue); it != currentScope->refinements.end()) - { - found = it->second; - break; - } - - childKeys.push_back(*currentLValue); - currentLValue = baseof(*currentLValue); - } - - if (!found) - { - // Should not be using scope->lookup. This is already recursive. - if (auto it = currentScope->bindings.find(symbol); it != currentScope->bindings.end()) - found = it->second.typeId; - else - { - // Nothing exists in this Scope. Just skip and try the parent one. - currentScope = currentScope->parent; - continue; - } - } - - for (auto it = childKeys.rbegin(); it != childKeys.rend(); ++it) - { - const LValue& key = *it; - - // Symbol can happen. Skip. - if (get(key)) - continue; - else if (auto field = get(key)) - { - found = getIndexTypeFromType(scope, *found, field->key, Location(), false); - if (!found) - return std::nullopt; // Turns out this type doesn't have the property at all. We're done. - } - else - LUAU_ASSERT(!"New LValue alternative not handled here."); - } - - return found; - } - - // No entry for it at all. Can happen when LValue root is a global. - return std::nullopt; - } - const Symbol symbol = getBaseSymbol(lvalue); ScopePtr currentScope = scope; @@ -5820,85 +5619,47 @@ static bool isUndecidable(TypeId ty) return get(ty) || get(ty) || get(ty); } -ErrorVec TypeChecker::resolve(const PredicateVec& predicates, const ScopePtr& scope, bool sense) +void TypeChecker::resolve(const PredicateVec& predicates, const ScopePtr& scope, bool sense) { - ErrorVec errVec; - resolve(predicates, errVec, scope->refinements, scope, sense); - return errVec; + resolve(predicates, scope->refinements, scope, sense); } -void TypeChecker::resolve(const PredicateVec& predicates, ErrorVec& errVec, RefinementMap& refis, const ScopePtr& scope, bool sense, bool fromOr) +void TypeChecker::resolve(const PredicateVec& predicates, RefinementMap& refis, const ScopePtr& scope, bool sense, bool fromOr) { for (const Predicate& c : predicates) - resolve(c, errVec, refis, scope, sense, fromOr); + resolve(c, refis, scope, sense, fromOr); } -void TypeChecker::resolve(const Predicate& predicate, ErrorVec& errVec, RefinementMap& refis, const ScopePtr& scope, bool sense, bool fromOr) +void TypeChecker::resolve(const Predicate& predicate, RefinementMap& refis, const ScopePtr& scope, bool sense, bool fromOr) { if (auto truthyP = get(predicate)) - resolve(*truthyP, errVec, refis, scope, sense, fromOr); + resolve(*truthyP, refis, scope, sense, fromOr); else if (auto andP = get(predicate)) - resolve(*andP, errVec, refis, scope, sense); + resolve(*andP, refis, scope, sense); else if (auto orP = get(predicate)) - resolve(*orP, errVec, refis, scope, sense); + resolve(*orP, refis, scope, sense); else if (auto notP = get(predicate)) - resolve(notP->predicates, errVec, refis, scope, !sense, fromOr); + resolve(notP->predicates, refis, scope, !sense, fromOr); else if (auto isaP = get(predicate)) - resolve(*isaP, errVec, refis, scope, sense); + resolve(*isaP, refis, scope, sense); else if (auto typeguardP = get(predicate)) - resolve(*typeguardP, errVec, refis, scope, sense); + resolve(*typeguardP, refis, scope, sense); else if (auto eqP = get(predicate)) - resolve(*eqP, errVec, refis, scope, sense); + resolve(*eqP, refis, scope, sense); else ice("Unhandled predicate kind"); } -void TypeChecker::resolve(const TruthyPredicate& truthyP, ErrorVec& errVec, RefinementMap& refis, const ScopePtr& scope, bool sense, bool fromOr) +void TypeChecker::resolve(const TruthyPredicate& truthyP, RefinementMap& refis, const ScopePtr& scope, bool sense, bool fromOr) { - if (FFlag::LuauAssertStripsFalsyTypes) - { - std::optional ty = resolveLValue(refis, scope, truthyP.lvalue); - if (ty && fromOr) - return addRefinement(refis, truthyP.lvalue, *ty); + std::optional ty = resolveLValue(refis, scope, truthyP.lvalue); + if (ty && fromOr) + return addRefinement(refis, truthyP.lvalue, *ty); - refineLValue(truthyP.lvalue, refis, scope, mkTruthyPredicate(sense)); - } - else - { - auto predicate = [sense](TypeId option) -> std::optional { - if (isUndecidable(option) || isBoolean(option) || isNil(option) != sense) - return option; - - return std::nullopt; - }; - - if (FFlag::LuauDiscriminableUnions2) - { - std::optional ty = resolveLValue(refis, scope, truthyP.lvalue); - if (ty && fromOr) - return addRefinement(refis, truthyP.lvalue, *ty); - - refineLValue(truthyP.lvalue, refis, scope, predicate); - } - else - { - std::optional ty = resolveLValue(refis, scope, truthyP.lvalue); - if (!ty) - return; - - // This is a hack. :( - // Without this, the expression 'a or b' might refine 'b' to be falsy. - // I'm not yet sure how else to get this to do the right thing without this hack, so we'll do this for now in the meantime. - if (fromOr) - return addRefinement(refis, truthyP.lvalue, *ty); - - if (std::optional result = filterMap(*ty, predicate)) - addRefinement(refis, truthyP.lvalue, *result); - } - } + refineLValue(truthyP.lvalue, refis, scope, mkTruthyPredicate(sense)); } -void TypeChecker::resolve(const AndPredicate& andP, ErrorVec& errVec, RefinementMap& refis, const ScopePtr& scope, bool sense) +void TypeChecker::resolve(const AndPredicate& andP, RefinementMap& refis, const ScopePtr& scope, bool sense) { if (!sense) { @@ -5907,14 +5668,14 @@ void TypeChecker::resolve(const AndPredicate& andP, ErrorVec& errVec, Refinement {NotPredicate{std::move(andP.rhs)}}, }; - return resolve(orP, errVec, refis, scope, !sense); + return resolve(orP, refis, scope, !sense); } - resolve(andP.lhs, errVec, refis, scope, sense); - resolve(andP.rhs, errVec, refis, scope, sense); + resolve(andP.lhs, refis, scope, sense); + resolve(andP.rhs, refis, scope, sense); } -void TypeChecker::resolve(const OrPredicate& orP, ErrorVec& errVec, RefinementMap& refis, const ScopePtr& scope, bool sense) +void TypeChecker::resolve(const OrPredicate& orP, RefinementMap& refis, const ScopePtr& scope, bool sense) { if (!sense) { @@ -5923,28 +5684,24 @@ void TypeChecker::resolve(const OrPredicate& orP, ErrorVec& errVec, RefinementMa {NotPredicate{std::move(orP.rhs)}}, }; - return resolve(andP, errVec, refis, scope, !sense); + return resolve(andP, refis, scope, !sense); } - ErrorVec discarded; - RefinementMap leftRefis; - resolve(orP.lhs, errVec, leftRefis, scope, sense); + resolve(orP.lhs, leftRefis, scope, sense); RefinementMap rightRefis; - resolve(orP.lhs, discarded, rightRefis, scope, !sense); - resolve(orP.rhs, errVec, rightRefis, scope, sense, true); // :( + resolve(orP.lhs, rightRefis, scope, !sense); + resolve(orP.rhs, rightRefis, scope, sense, true); // :( merge(refis, leftRefis); merge(refis, rightRefis); } -void TypeChecker::resolve(const IsAPredicate& isaP, ErrorVec& errVec, RefinementMap& refis, const ScopePtr& scope, bool sense) +void TypeChecker::resolve(const IsAPredicate& isaP, RefinementMap& refis, const ScopePtr& scope, bool sense) { auto predicate = [&](TypeId option) -> std::optional { // This by itself is not truly enough to determine that A is stronger than B or vice versa. - // The best unambiguous way about this would be to have a function that returns the relationship ordering of a pair. - // i.e. TypeRelationship relationshipOf(TypeId superTy, TypeId subTy) bool optionIsSubtype = canUnify(option, isaP.ty, isaP.location).empty(); bool targetIsSubtype = canUnify(isaP.ty, option, isaP.location).empty(); @@ -5985,32 +5742,15 @@ void TypeChecker::resolve(const IsAPredicate& isaP, ErrorVec& errVec, Refinement return res; }; - if (FFlag::LuauDiscriminableUnions2) - { - refineLValue(isaP.lvalue, refis, scope, predicate); - } - else - { - std::optional ty = resolveLValue(refis, scope, isaP.lvalue); - if (!ty) - return; - - if (std::optional result = filterMap(*ty, predicate)) - addRefinement(refis, isaP.lvalue, *result); - else - { - addRefinement(refis, isaP.lvalue, errorRecoveryType(scope)); - errVec.push_back(TypeError{isaP.location, TypeMismatch{isaP.ty, *ty}}); - } - } + refineLValue(isaP.lvalue, refis, scope, predicate); } -void TypeChecker::resolve(const TypeGuardPredicate& typeguardP, ErrorVec& errVec, RefinementMap& refis, const ScopePtr& scope, bool sense) +void TypeChecker::resolve(const TypeGuardPredicate& typeguardP, RefinementMap& refis, const ScopePtr& scope, bool sense) { // Rewrite the predicate 'type(foo) == "vector"' to be 'typeof(foo) == "Vector3"'. They're exactly identical. // This allows us to avoid writing in edge cases. if (!typeguardP.isTypeof && typeguardP.kind == "vector") - return resolve(TypeGuardPredicate{std::move(typeguardP.lvalue), typeguardP.location, "Vector3", true}, errVec, refis, scope, sense); + return resolve(TypeGuardPredicate{std::move(typeguardP.lvalue), typeguardP.location, "Vector3", true}, refis, scope, sense); std::optional ty = resolveLValue(refis, scope, typeguardP.lvalue); if (!ty) @@ -6060,52 +5800,29 @@ void TypeChecker::resolve(const TypeGuardPredicate& typeguardP, ErrorVec& errVec if (auto it = primitives.find(typeguardP.kind); it != primitives.end()) { - if (FFlag::LuauDiscriminableUnions2) - { - refineLValue(typeguardP.lvalue, refis, scope, it->second(sense)); - return; - } - else - { - if (std::optional result = filterMap(*ty, it->second(sense))) - addRefinement(refis, typeguardP.lvalue, *result); - else - { - addRefinement(refis, typeguardP.lvalue, errorRecoveryType(scope)); - if (sense) - errVec.push_back( - TypeError{typeguardP.location, GenericError{"Type '" + toString(*ty) + "' has no overlap with '" + typeguardP.kind + "'"}}); - } - - return; - } + refineLValue(typeguardP.lvalue, refis, scope, it->second(sense)); + return; } - auto fail = [&](const TypeErrorData& err) { - if (!FFlag::LuauDiscriminableUnions2) - errVec.push_back(TypeError{typeguardP.location, err}); - addRefinement(refis, typeguardP.lvalue, errorRecoveryType(scope)); - }; - if (!typeguardP.isTypeof) - return fail(UnknownSymbol{typeguardP.kind, UnknownSymbol::Type}); + return addRefinement(refis, typeguardP.lvalue, errorRecoveryType(scope)); auto typeFun = globalScope->lookupType(typeguardP.kind); if (!typeFun || !typeFun->typeParams.empty() || !typeFun->typePackParams.empty()) - return fail(UnknownSymbol{typeguardP.kind, UnknownSymbol::Type}); + return addRefinement(refis, typeguardP.lvalue, errorRecoveryType(scope)); TypeId type = follow(typeFun->type); // We're only interested in the root class of any classes. if (auto ctv = get(type); !ctv || ctv->parent) - return fail(UnknownSymbol{typeguardP.kind, UnknownSymbol::Type}); + return addRefinement(refis, typeguardP.lvalue, errorRecoveryType(scope)); // This probably hints at breaking out type filtering functions from the predicate solver so that typeof is not tightly coupled with IsA. // Until then, we rewrite this to be the same as using IsA. - return resolve(IsAPredicate{std::move(typeguardP.lvalue), typeguardP.location, type}, errVec, refis, scope, sense); + return resolve(IsAPredicate{std::move(typeguardP.lvalue), typeguardP.location, type}, refis, scope, sense); } -void TypeChecker::resolve(const EqPredicate& eqP, ErrorVec& errVec, RefinementMap& refis, const ScopePtr& scope, bool sense) +void TypeChecker::resolve(const EqPredicate& eqP, RefinementMap& refis, const ScopePtr& scope, bool sense) { // This refinement will require success typing to do everything correctly. For now, we can get most of the way there. auto options = [](TypeId ty) -> std::vector { @@ -6114,82 +5831,33 @@ void TypeChecker::resolve(const EqPredicate& eqP, ErrorVec& errVec, RefinementMa return {ty}; }; - if (FFlag::LuauDiscriminableUnions2) - { - std::vector rhs = options(eqP.type); + std::vector rhs = options(eqP.type); - if (sense && std::any_of(rhs.begin(), rhs.end(), isUndecidable)) - return; // Optimization: the other side has unknown types, so there's probably an overlap. Refining is no-op here. + if (sense && std::any_of(rhs.begin(), rhs.end(), isUndecidable)) + return; // Optimization: the other side has unknown types, so there's probably an overlap. Refining is no-op here. - auto predicate = [&](TypeId option) -> std::optional { - if (sense && isUndecidable(option)) - return FFlag::LuauWeakEqConstraint ? option : eqP.type; + auto predicate = [&](TypeId option) -> std::optional { + if (sense && isUndecidable(option)) + return FFlag::LuauWeakEqConstraint ? option : eqP.type; - if (!sense && isNil(eqP.type)) - return (isUndecidable(option) || !isNil(option)) ? std::optional(option) : std::nullopt; + if (!sense && isNil(eqP.type)) + return (isUndecidable(option) || !isNil(option)) ? std::optional(option) : std::nullopt; - if (maybeSingleton(eqP.type)) - { - // Normally we'd write option <: eqP.type, but singletons are always the subtype, so we flip this. - if (!sense || canUnify(eqP.type, option, eqP.location).empty()) - return sense ? eqP.type : option; - - // local variable works around an odd gcc 9.3 warning: may be used uninitialized - std::optional res = std::nullopt; - return res; - } - - return option; - }; - - refineLValue(eqP.lvalue, refis, scope, predicate); - } - else - { - if (FFlag::LuauWeakEqConstraint) + if (maybeSingleton(eqP.type)) { - if (!sense && isNil(eqP.type)) - resolve(TruthyPredicate{std::move(eqP.lvalue), eqP.location}, errVec, refis, scope, true, /* fromOr= */ false); + // Normally we'd write option <: eqP.type, but singletons are always the subtype, so we flip this. + if (!sense || canUnify(eqP.type, option, eqP.location).empty()) + return sense ? eqP.type : option; - return; + // local variable works around an odd gcc 9.3 warning: may be used uninitialized + std::optional res = std::nullopt; + return res; } - if (FFlag::LuauEqConstraint) - { - std::optional ty = resolveLValue(refis, scope, eqP.lvalue); - if (!ty) - return; + return option; + }; - std::vector lhs = options(*ty); - std::vector rhs = options(eqP.type); - - if (sense && std::any_of(lhs.begin(), lhs.end(), isUndecidable)) - { - addRefinement(refis, eqP.lvalue, eqP.type); - return; - } - else if (sense && std::any_of(rhs.begin(), rhs.end(), isUndecidable)) - return; // Optimization: the other side has unknown types, so there's probably an overlap. Refining is no-op here. - - std::unordered_set set; - for (TypeId left : lhs) - { - for (TypeId right : rhs) - { - // When singleton types arrive, `isNil` here probably should be replaced with `isLiteral`. - if (canUnify(right, left, eqP.location).empty() == sense || (!sense && !isNil(left))) - set.insert(left); - } - } - - if (set.empty()) - return; - - std::vector viable(set.begin(), set.end()); - TypeId result = viable.size() == 1 ? viable[0] : addType(UnionTypeVar{std::move(viable)}); - addRefinement(refis, eqP.lvalue, result); - } - } + refineLValue(eqP.lvalue, refis, scope, predicate); } bool TypeChecker::isNonstrictMode() const diff --git a/Analysis/src/TypeUtils.cpp b/Analysis/src/TypeUtils.cpp index c2435890..ba09df5f 100644 --- a/Analysis/src/TypeUtils.cpp +++ b/Analysis/src/TypeUtils.cpp @@ -5,8 +5,6 @@ #include "Luau/ToString.h" #include "Luau/TypeInfer.h" -LUAU_FASTFLAGVARIABLE(LuauTerminateCyclicMetatableIndexLookup, false) - namespace Luau { @@ -55,13 +53,10 @@ std::optional findTablePropertyRespectingMeta(ErrorVec& errors, TypeId t { TypeId index = follow(*mtIndex); - if (FFlag::LuauTerminateCyclicMetatableIndexLookup) - { - if (count >= 100) - return std::nullopt; + if (count >= 100) + return std::nullopt; - ++count; - } + ++count; if (const auto& itt = getTableType(index)) { diff --git a/Analysis/src/TypeVar.cpp b/Analysis/src/TypeVar.cpp index 463b4651..2355dab2 100644 --- a/Analysis/src/TypeVar.cpp +++ b/Analysis/src/TypeVar.cpp @@ -24,8 +24,6 @@ LUAU_FASTINTVARIABLE(LuauTypeMaximumStringifierLength, 500) LUAU_FASTINTVARIABLE(LuauTableTypeMaximumStringifierLength, 0) LUAU_FASTINT(LuauTypeInferRecursionLimit) LUAU_FASTFLAG(LuauSubtypingAddOptPropsToUnsealedTables) -LUAU_FASTFLAG(LuauDiscriminableUnions2) -LUAU_FASTFLAGVARIABLE(LuauAnyInIsOptionalIsOptional, false) LUAU_FASTFLAGVARIABLE(LuauClassDefinitionModuleInError, false) namespace Luau @@ -204,14 +202,14 @@ bool isOptional(TypeId ty) ty = follow(ty); - if (FFlag::LuauAnyInIsOptionalIsOptional && get(ty)) + if (get(ty)) return true; auto utv = get(ty); if (!utv) return false; - return std::any_of(begin(utv), end(utv), FFlag::LuauAnyInIsOptionalIsOptional ? isOptional : isNil); + return std::any_of(begin(utv), end(utv), isOptional); } bool isTableIntersection(TypeId ty) @@ -378,8 +376,7 @@ bool hasLength(TypeId ty, DenseHashSet& seen, int* recursionCount) if (seen.contains(ty)) return true; - bool isStr = FFlag::LuauDiscriminableUnions2 ? isString(ty) : isPrim(ty, PrimitiveTypeVar::String); - if (isStr || get(ty) || get(ty) || get(ty)) + if (isString(ty) || get(ty) || get(ty) || get(ty)) return true; if (auto uty = get(ty)) diff --git a/Analysis/src/Unifier.cpp b/Analysis/src/Unifier.cpp index f5c1dde9..9308e9ff 100644 --- a/Analysis/src/Unifier.cpp +++ b/Analysis/src/Unifier.cpp @@ -24,8 +24,6 @@ LUAU_FASTFLAGVARIABLE(LuauSubtypingAddOptPropsToUnsealedTables, false) LUAU_FASTFLAGVARIABLE(LuauWidenIfSupertypeIsFree2, false) LUAU_FASTFLAGVARIABLE(LuauDifferentOrderOfUnificationDoesntMatter2, false) LUAU_FASTFLAGVARIABLE(LuauTxnLogRefreshFunctionPointers, false) -LUAU_FASTFLAG(LuauAnyInIsOptionalIsOptional) -LUAU_FASTFLAG(LuauTypecheckOptPass) namespace Luau { @@ -382,19 +380,6 @@ Unifier::Unifier(TypeArena* types, Mode mode, const Location& location, Variance LUAU_ASSERT(sharedState.iceHandler); } -Unifier::Unifier(TypeArena* types, Mode mode, std::vector>* sharedSeen, const Location& location, - Variance variance, UnifierSharedState& sharedState, TxnLog* parentLog) - : types(types) - , mode(mode) - , log(parentLog, sharedSeen) - , location(location) - , variance(variance) - , sharedState(sharedState) -{ - LUAU_ASSERT(!FFlag::LuauTypecheckOptPass); - LUAU_ASSERT(sharedState.iceHandler); -} - void Unifier::tryUnify(TypeId subTy, TypeId superTy, bool isFunctionCall, bool isIntersection) { sharedState.counters.iterationCount = 0; @@ -1219,14 +1204,6 @@ void Unifier::tryUnify_(TypePackId subTp, TypePackId superTp, bool isFunctionCal continue; } - // In nonstrict mode, any also marks an optional argument. - else if (!FFlag::LuauAnyInIsOptionalIsOptional && superIter.good() && isNonstrictMode() && - log.getMutable(log.follow(*superIter))) - { - superIter.advance(); - continue; - } - if (log.getMutable(superIter.packId)) { tryUnifyVariadics(subIter.packId, superIter.packId, false, int(subIter.index)); @@ -1454,21 +1431,9 @@ void Unifier::tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) { auto subIter = subTable->props.find(propName); - if (FFlag::LuauAnyInIsOptionalIsOptional) - { - if (subIter == subTable->props.end() && - (!FFlag::LuauSubtypingAddOptPropsToUnsealedTables || subTable->state == TableState::Unsealed) && !isOptional(superProp.type)) - missingProperties.push_back(propName); - } - else - { - bool isAny = log.getMutable(log.follow(superProp.type)); - - if (subIter == subTable->props.end() && - (!FFlag::LuauSubtypingAddOptPropsToUnsealedTables || subTable->state == TableState::Unsealed) && !isOptional(superProp.type) && - !isAny) - missingProperties.push_back(propName); - } + if (subIter == subTable->props.end() && (!FFlag::LuauSubtypingAddOptPropsToUnsealedTables || subTable->state == TableState::Unsealed) && + !isOptional(superProp.type)) + missingProperties.push_back(propName); } if (!missingProperties.empty()) @@ -1485,18 +1450,8 @@ void Unifier::tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) { auto superIter = superTable->props.find(propName); - if (FFlag::LuauAnyInIsOptionalIsOptional) - { - if (superIter == superTable->props.end() && (FFlag::LuauSubtypingAddOptPropsToUnsealedTables || !isOptional(subProp.type))) - extraProperties.push_back(propName); - } - else - { - bool isAny = log.is(log.follow(subProp.type)); - if (superIter == superTable->props.end() && - (FFlag::LuauSubtypingAddOptPropsToUnsealedTables || (!isOptional(subProp.type) && !isAny))) - extraProperties.push_back(propName); - } + if (superIter == superTable->props.end() && (FFlag::LuauSubtypingAddOptPropsToUnsealedTables || !isOptional(subProp.type))) + extraProperties.push_back(propName); } if (!extraProperties.empty()) @@ -1540,21 +1495,12 @@ void Unifier::tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) if (innerState.errors.empty()) log.concat(std::move(innerState.log)); } - else if (FFlag::LuauAnyInIsOptionalIsOptional && - (!FFlag::LuauSubtypingAddOptPropsToUnsealedTables || subTable->state == TableState::Unsealed) && isOptional(prop.type)) + else if ((!FFlag::LuauSubtypingAddOptPropsToUnsealedTables || subTable->state == TableState::Unsealed) && isOptional(prop.type)) // This is sound because unsealed table types are precise, so `{ p : T } <: { p : T, q : U? }` // since if `t : { p : T }` then we are guaranteed that `t.q` is `nil`. // TODO: if the supertype is written to, the subtype may no longer be precise (alias analysis?) { } - else if ((!FFlag::LuauSubtypingAddOptPropsToUnsealedTables || subTable->state == TableState::Unsealed) && - (isOptional(prop.type) || get(follow(prop.type)))) - // This is sound because unsealed table types are precise, so `{ p : T } <: { p : T, q : U? }` - // since if `t : { p : T }` then we are guaranteed that `t.q` is `nil`. - // TODO: should isOptional(anyType) be true? - // TODO: if the supertype is written to, the subtype may no longer be precise (alias analysis?) - { - } else if (subTable->state == TableState::Free) { PendingType* pendingSub = log.queue(subTy); @@ -1618,10 +1564,7 @@ void Unifier::tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) else if (variance == Covariant) { } - else if (FFlag::LuauAnyInIsOptionalIsOptional && !FFlag::LuauSubtypingAddOptPropsToUnsealedTables && isOptional(prop.type)) - { - } - else if (!FFlag::LuauSubtypingAddOptPropsToUnsealedTables && (isOptional(prop.type) || get(follow(prop.type)))) + else if (!FFlag::LuauSubtypingAddOptPropsToUnsealedTables && isOptional(prop.type)) { } else if (superTable->state == TableState::Free) @@ -1753,9 +1696,7 @@ TypePackId Unifier::widen(TypePackId tp) TypeId Unifier::deeplyOptional(TypeId ty, std::unordered_map seen) { ty = follow(ty); - if (!FFlag::LuauAnyInIsOptionalIsOptional && get(ty)) - return ty; - else if (isOptional(ty)) + if (isOptional(ty)) return ty; else if (const TableTypeVar* ttv = get(ty)) { @@ -2666,14 +2607,7 @@ void Unifier::occursCheck(DenseHashSet& seen, TypePackId needle, Typ Unifier Unifier::makeChildUnifier() { - if (FFlag::LuauTypecheckOptPass) - { - Unifier u = Unifier{types, mode, location, variance, sharedState, &log}; - u.anyIsTop = anyIsTop; - return u; - } - - Unifier u = Unifier{types, mode, log.sharedSeen, location, variance, sharedState, &log}; + Unifier u = Unifier{types, mode, location, variance, sharedState, &log}; u.anyIsTop = anyIsTop; return u; } diff --git a/Compiler/include/Luau/BytecodeBuilder.h b/Compiler/include/Luau/BytecodeBuilder.h index b00440ae..12465377 100644 --- a/Compiler/include/Luau/BytecodeBuilder.h +++ b/Compiler/include/Luau/BytecodeBuilder.h @@ -224,6 +224,7 @@ private: DenseHashMap constantMap; DenseHashMap tableShapeMap; + DenseHashMap protoMap; int debugLine = 0; diff --git a/Compiler/src/BytecodeBuilder.cpp b/Compiler/src/BytecodeBuilder.cpp index fb70392e..beeda295 100644 --- a/Compiler/src/BytecodeBuilder.cpp +++ b/Compiler/src/BytecodeBuilder.cpp @@ -6,6 +6,8 @@ #include #include +LUAU_FASTFLAG(LuauCompileNestedClosureO2) + namespace Luau { @@ -181,6 +183,7 @@ size_t BytecodeBuilder::TableShapeHash::operator()(const TableShape& v) const BytecodeBuilder::BytecodeBuilder(BytecodeEncoder* encoder) : constantMap({Constant::Type_Nil, ~0ull}) , tableShapeMap(TableShape()) + , protoMap(~0u) , stringTable({nullptr, 0}) , encoder(encoder) { @@ -250,6 +253,7 @@ void BytecodeBuilder::endFunction(uint8_t maxstacksize, uint8_t numupvalues) constantMap.clear(); tableShapeMap.clear(); + protoMap.clear(); debugRemarks.clear(); debugRemarkBuffer.clear(); @@ -372,11 +376,17 @@ int32_t BytecodeBuilder::addConstantClosure(uint32_t fid) int16_t BytecodeBuilder::addChildFunction(uint32_t fid) { + if (FFlag::LuauCompileNestedClosureO2) + if (int16_t* cache = protoMap.find(fid)) + return *cache; + uint32_t id = uint32_t(protos.size()); if (id >= kMaxClosureCount) return -1; + if (FFlag::LuauCompileNestedClosureO2) + protoMap[fid] = int16_t(id); protos.push_back(fid); return int16_t(id); diff --git a/Compiler/src/Compiler.cpp b/Compiler/src/Compiler.cpp index e177e928..4f26ceb9 100644 --- a/Compiler/src/Compiler.cpp +++ b/Compiler/src/Compiler.cpp @@ -17,8 +17,6 @@ #include #include -LUAU_FASTFLAGVARIABLE(LuauCompileSupportInlining, false) - LUAU_FASTFLAGVARIABLE(LuauCompileIter, false) LUAU_FASTFLAGVARIABLE(LuauCompileIterNoReserve, false) LUAU_FASTFLAGVARIABLE(LuauCompileIterNoPairs, false) @@ -30,6 +28,8 @@ LUAU_FASTINTVARIABLE(LuauCompileInlineThreshold, 25) LUAU_FASTINTVARIABLE(LuauCompileInlineThresholdMaxBoost, 300) LUAU_FASTINTVARIABLE(LuauCompileInlineDepth, 5) +LUAU_FASTFLAGVARIABLE(LuauCompileNestedClosureO2, false) + namespace Luau { @@ -100,13 +100,11 @@ struct Compiler upvals.reserve(16); } - uint8_t getLocal(AstLocal* local) + int getLocalReg(AstLocal* local) { Local* l = locals.find(local); - LUAU_ASSERT(l); - LUAU_ASSERT(l->allocated); - return l->reg; + return l && l->allocated ? l->reg : -1; } uint8_t getUpval(AstLocal* local) @@ -159,17 +157,19 @@ struct Compiler AstExprFunction* getFunctionExpr(AstExpr* node) { - if (AstExprLocal* le = node->as()) + if (AstExprLocal* expr = node->as()) { - Variable* lv = variables.find(le->local); + Variable* lv = variables.find(expr->local); if (!lv || lv->written || !lv->init) return nullptr; return getFunctionExpr(lv->init); } - else if (AstExprGroup* ge = node->as()) - return getFunctionExpr(ge->expr); + else if (AstExprGroup* expr = node->as()) + return getFunctionExpr(expr->expr); + else if (AstExprTypeAssertion* expr = node->as()) + return getFunctionExpr(expr->expr); else return node->as(); } @@ -180,13 +180,13 @@ struct Compiler { bool result = true; - bool visit(AstExpr* node) override + bool visit(AstExprFunction* node) override { - // nested functions may capture function arguments, and our upval handling doesn't handle elided variables (constant) - // TODO: we could remove this case if we changed function compilation to create temporary locals for constant upvalues - // TODO: additionally we would need to change upvalue handling in compileExprFunction to handle upvalue->local migration - result = result && !node->is(); - return result; + if (!FFlag::LuauCompileNestedClosureO2) + result = false; + + // short-circuit to avoid analyzing nested closure bodies + return false; } bool visit(AstStat* node) override @@ -275,8 +275,7 @@ struct Compiler f.upvals = upvals; // record information for inlining - if (FFlag::LuauCompileSupportInlining && options.optimizationLevel >= 2 && !func->vararg && canInlineFunctionBody(func->body) && - !getfenvUsed && !setfenvUsed) + if (options.optimizationLevel >= 2 && !func->vararg && canInlineFunctionBody(func->body) && !getfenvUsed && !setfenvUsed) { f.canInline = true; f.stackSize = stackSize; @@ -346,8 +345,8 @@ struct Compiler uint8_t argreg; - if (isExprLocalReg(arg)) - argreg = getLocal(arg->as()->local); + if (int reg = getExprLocalReg(arg); reg >= 0) + argreg = uint8_t(reg); else { argreg = uint8_t(regs + 1); @@ -403,8 +402,8 @@ struct Compiler } } - if (isExprLocalReg(expr->args.data[i])) - args[i] = getLocal(expr->args.data[i]->as()->local); + if (int reg = getExprLocalReg(expr->args.data[i]); reg >= 0) + args[i] = uint8_t(reg); else { args[i] = uint8_t(regs + 1 + i); @@ -489,19 +488,18 @@ struct Compiler return false; } - // TODO: we can compile functions with mismatching arity at call site but it's more annoying - if (func->args.size != expr->args.size) - { - bytecode.addDebugRemark("inlining failed: argument count mismatch (expected %d, got %d)", int(func->args.size), int(expr->args.size)); - return false; - } - - // we use a dynamic cost threshold that's based on the fixed limit boosted by the cost advantage we gain due to inlining + // compute constant bitvector for all arguments to feed the cost model bool varc[8] = {}; - for (size_t i = 0; i < expr->args.size && i < 8; ++i) + for (size_t i = 0; i < func->args.size && i < expr->args.size && i < 8; ++i) varc[i] = isConstant(expr->args.data[i]); - int inlinedCost = computeCost(fi->costModel, varc, std::min(int(expr->args.size), 8)); + // if the last argument only returns a single value, all following arguments are nil + if (expr->args.size != 0 && !(expr->args.data[expr->args.size - 1]->is() || expr->args.data[expr->args.size - 1]->is())) + for (size_t i = expr->args.size; i < func->args.size && i < 8; ++i) + varc[i] = true; + + // we use a dynamic cost threshold that's based on the fixed limit boosted by the cost advantage we gain due to inlining + int inlinedCost = computeCost(fi->costModel, varc, std::min(int(func->args.size), 8)); int baselineCost = computeCost(fi->costModel, nullptr, 0) + 3; int inlineProfit = (inlinedCost == 0) ? thresholdMaxBoost : std::min(thresholdMaxBoost, 100 * baselineCost / inlinedCost); @@ -533,15 +531,44 @@ struct Compiler for (size_t i = 0; i < func->args.size; ++i) { AstLocal* var = func->args.data[i]; - AstExpr* arg = expr->args.data[i]; + AstExpr* arg = i < expr->args.size ? expr->args.data[i] : nullptr; - if (Variable* vv = variables.find(var); vv && vv->written) + if (i + 1 == expr->args.size && func->args.size > expr->args.size && (arg->is() || arg->is())) + { + // if the last argument can return multiple values, we need to compute all of them into the remaining arguments + unsigned int tail = unsigned(func->args.size - expr->args.size) + 1; + uint8_t reg = allocReg(arg, tail); + + if (AstExprCall* expr = arg->as()) + compileExprCall(expr, reg, tail, /* targetTop= */ true); + else if (AstExprVarargs* expr = arg->as()) + compileExprVarargs(expr, reg, tail); + else + LUAU_ASSERT(!"Unexpected expression type"); + + for (size_t j = i; j < func->args.size; ++j) + pushLocal(func->args.data[j], uint8_t(reg + (j - i))); + + // all remaining function arguments have been allocated and assigned to + break; + } + else if (Variable* vv = variables.find(var); vv && vv->written) { // if the argument is mutated, we need to allocate a fresh register even if it's a constant uint8_t reg = allocReg(arg, 1); - compileExprTemp(arg, reg); + + if (arg) + compileExprTemp(arg, reg); + else + bytecode.emitABC(LOP_LOADNIL, reg, 0, 0); + pushLocal(var, reg); } + else if (arg == nullptr) + { + // since the argument is not mutated, we can simply fold the value into the expressions that need it + locstants[var] = {Constant::Type_Nil}; + } else if (const Constant* cv = constants.find(arg); cv && cv->type != Constant::Type_Unknown) { // since the argument is not mutated, we can simply fold the value into the expressions that need it @@ -553,20 +580,26 @@ struct Compiler Variable* lv = le ? variables.find(le->local) : nullptr; // if the argument is a local that isn't mutated, we will simply reuse the existing register - if (isExprLocalReg(arg) && (!lv || !lv->written)) + if (int reg = le ? getExprLocalReg(le) : -1; reg >= 0 && (!lv || !lv->written)) { - uint8_t reg = getLocal(le->local); - pushLocal(var, reg); + pushLocal(var, uint8_t(reg)); } else { - uint8_t reg = allocReg(arg, 1); - compileExprTemp(arg, reg); - pushLocal(var, reg); + uint8_t temp = allocReg(arg, 1); + compileExprTemp(arg, temp); + pushLocal(var, temp); } } } + // evaluate extra expressions for side effects + for (size_t i = func->args.size; i < expr->args.size; ++i) + { + RegScope rsi(this); + compileExprAuto(expr->args.data[i], rsi); + } + // fold constant values updated above into expressions in the function body foldConstants(constants, variables, locstants, func->body); @@ -627,12 +660,15 @@ struct Compiler FInt::LuauCompileInlineThresholdMaxBoost, FInt::LuauCompileInlineDepth)) return; - if (fi && !fi->canInline) + // add a debug remark for cases when we didn't even call tryCompileInlinedCall + if (func && !(fi && fi->canInline)) { if (func->vararg) bytecode.addDebugRemark("inlining failed: function is variadic"); - else + else if (fi) bytecode.addDebugRemark("inlining failed: complex constructs in function body"); + else + bytecode.addDebugRemark("inlining failed: can't inline recursive calls"); } } @@ -677,9 +713,9 @@ struct Compiler LUAU_ASSERT(fi); // Optimization: use local register directly in NAMECALL if possible - if (isExprLocalReg(fi->expr)) + if (int reg = getExprLocalReg(fi->expr); reg >= 0) { - selfreg = getLocal(fi->expr->as()->local); + selfreg = uint8_t(reg); } else { @@ -785,6 +821,8 @@ struct Compiler void compileExprFunction(AstExprFunction* expr, uint8_t target) { + RegScope rs(this); + const Function* f = functions.find(expr); LUAU_ASSERT(f); @@ -795,6 +833,67 @@ struct Compiler if (pid < 0) CompileError::raise(expr->location, "Exceeded closure limit; simplify the code to compile"); + if (FFlag::LuauCompileNestedClosureO2) + { + captures.clear(); + captures.reserve(f->upvals.size()); + + for (AstLocal* uv : f->upvals) + { + LUAU_ASSERT(uv->functionDepth < expr->functionDepth); + + if (int reg = getLocalReg(uv); reg >= 0) + { + // note: we can't check if uv is an upvalue in the current frame because inlining can migrate from upvalues to locals + Variable* ul = variables.find(uv); + bool immutable = !ul || !ul->written; + + captures.push_back({immutable ? LCT_VAL : LCT_REF, uint8_t(reg)}); + } + else if (const Constant* uc = locstants.find(uv); uc && uc->type != Constant::Type_Unknown) + { + // inlining can result in an upvalue capture of a constant, in which case we can't capture without a temporary register + uint8_t reg = allocReg(expr, 1); + compileExprConstant(expr, uc, reg); + + captures.push_back({LCT_VAL, reg}); + } + else + { + LUAU_ASSERT(uv->functionDepth < expr->functionDepth - 1); + + // get upvalue from parent frame + // note: this will add uv to the current upvalue list if necessary + uint8_t uid = getUpval(uv); + + captures.push_back({LCT_UPVAL, uid}); + } + } + + // Optimization: when closure has no upvalues, or upvalues are safe to share, instead of allocating it every time we can share closure + // objects (this breaks assumptions about function identity which can lead to setfenv not working as expected, so we disable this when it + // is used) + int16_t shared = -1; + + if (options.optimizationLevel >= 1 && shouldShareClosure(expr) && !setfenvUsed) + { + int32_t cid = bytecode.addConstantClosure(f->id); + + if (cid >= 0 && cid < 32768) + shared = int16_t(cid); + } + + if (shared >= 0) + bytecode.emitAD(LOP_DUPCLOSURE, target, shared); + else + bytecode.emitAD(LOP_NEWCLOSURE, target, pid); + + for (const Capture& c : captures) + bytecode.emitABC(LOP_CAPTURE, uint8_t(c.type), c.data, 0); + + return; + } + bool shared = false; // Optimization: when closure has no upvalues, or upvalues are safe to share, instead of allocating it every time we can share closure @@ -824,9 +923,10 @@ struct Compiler if (uv->functionDepth == expr->functionDepth - 1) { // get local variable - uint8_t reg = getLocal(uv); + int reg = getLocalReg(uv); + LUAU_ASSERT(reg >= 0); - bytecode.emitABC(LOP_CAPTURE, uint8_t(immutable ? LCT_VAL : LCT_REF), reg, 0); + bytecode.emitABC(LOP_CAPTURE, uint8_t(immutable ? LCT_VAL : LCT_REF), uint8_t(reg), 0); } else { @@ -1213,10 +1313,10 @@ struct Compiler if (!isConditionFast(expr->left)) { // Optimization: when right hand side is a local variable, we can use AND/OR - if (isExprLocalReg(expr->right)) + if (int reg = getExprLocalReg(expr->right); reg >= 0) { uint8_t lr = compileExprAuto(expr->left, rs); - uint8_t rr = getLocal(expr->right->as()->local); + uint8_t rr = uint8_t(reg); bytecode.emitABC(and_ ? LOP_AND : LOP_OR, target, lr, rr); return; @@ -1803,19 +1903,18 @@ struct Compiler } else if (AstExprLocal* expr = node->as()) { - if (FFlag::LuauCompileSupportInlining ? !isExprLocalReg(expr) : expr->upvalue) + // note: this can't check expr->upvalue because upvalues may be upgraded to locals during inlining + if (int reg = getExprLocalReg(expr); reg >= 0) + { + bytecode.emitABC(LOP_MOVE, target, uint8_t(reg), 0); + } + else { LUAU_ASSERT(expr->upvalue); uint8_t uid = getUpval(expr->local); bytecode.emitABC(LOP_GETUPVAL, target, uid, 0); } - else - { - uint8_t reg = getLocal(expr->local); - - bytecode.emitABC(LOP_MOVE, target, reg, 0); - } } else if (AstExprGlobal* expr = node->as()) { @@ -1879,8 +1978,8 @@ struct Compiler uint8_t compileExprAuto(AstExpr* node, RegScope&) { // Optimization: directly return locals instead of copying them to a temporary - if (isExprLocalReg(node)) - return getLocal(node->as()->local); + if (int reg = getExprLocalReg(node); reg >= 0) + return uint8_t(reg); // note: the register is owned by the parent scope uint8_t reg = allocReg(node, 1); @@ -1910,7 +2009,7 @@ struct Compiler for (size_t i = 0; i < targetCount; ++i) compileExprTemp(list.data[i], uint8_t(target + i)); - // compute expressions with values that go nowhere; this is required to run side-effecting code if any + // evaluate extra expressions for side effects for (size_t i = targetCount; i < list.size; ++i) { RegScope rsi(this); @@ -2008,20 +2107,21 @@ struct Compiler if (AstExprLocal* expr = node->as()) { - if (FFlag::LuauCompileSupportInlining ? !isExprLocalReg(expr) : expr->upvalue) + // note: this can't check expr->upvalue because upvalues may be upgraded to locals during inlining + if (int reg = getExprLocalReg(expr); reg >= 0) { - LUAU_ASSERT(expr->upvalue); - - LValue result = {LValue::Kind_Upvalue}; - result.upval = getUpval(expr->local); + LValue result = {LValue::Kind_Local}; + result.reg = uint8_t(reg); result.location = node->location; return result; } else { - LValue result = {LValue::Kind_Local}; - result.reg = getLocal(expr->local); + LUAU_ASSERT(expr->upvalue); + + LValue result = {LValue::Kind_Upvalue}; + result.upval = getUpval(expr->local); result.location = node->location; return result; @@ -2115,15 +2215,21 @@ struct Compiler compileLValueUse(lv, source, /* set= */ true); } - bool isExprLocalReg(AstExpr* expr) + int getExprLocalReg(AstExpr* node) { - AstExprLocal* le = expr->as(); - if (!le || (!FFlag::LuauCompileSupportInlining && le->upvalue)) - return false; + if (AstExprLocal* expr = node->as()) + { + // note: this can't check expr->upvalue because upvalues may be upgraded to locals during inlining + Local* l = locals.find(expr->local); - Local* l = locals.find(le->local); - - return l && l->allocated; + return l && l->allocated ? l->reg : -1; + } + else if (AstExprGroup* expr = node->as()) + return getExprLocalReg(expr->expr); + else if (AstExprTypeAssertion* expr = node->as()) + return getExprLocalReg(expr->expr); + else + return -1; } bool isStatBreak(AstStat* node) @@ -2352,20 +2458,17 @@ struct Compiler // Optimization: return locals directly instead of copying them into a temporary // this is very important for a single return value and occasionally effective for multiple values - if (stat->list.size > 0 && isExprLocalReg(stat->list.data[0])) + if (int reg = stat->list.size > 0 ? getExprLocalReg(stat->list.data[0]) : -1; reg >= 0) { - temp = getLocal(stat->list.data[0]->as()->local); + temp = uint8_t(reg); consecutive = true; for (size_t i = 1; i < stat->list.size; ++i) - { - AstExpr* v = stat->list.data[i]; - if (!isExprLocalReg(v) || getLocal(v->as()->local) != temp + i) + if (getExprLocalReg(stat->list.data[i]) != int(temp + i)) { consecutive = false; break; } - } } if (!consecutive && stat->list.size > 0) @@ -2438,12 +2541,13 @@ struct Compiler { bool result = true; - bool visit(AstExpr* node) override + bool visit(AstExprFunction* node) override { - // functions may capture loop variable, and our upval handling doesn't handle elided variables (constant) - // TODO: we could remove this case if we changed function compilation to create temporary locals for constant upvalues - result = result && !node->is(); - return result; + if (!FFlag::LuauCompileNestedClosureO2) + result = false; + + // short-circuit to avoid analyzing nested closure bodies + return false; } bool visit(AstStat* node) override @@ -2874,12 +2978,9 @@ struct Compiler void compileStatFunction(AstStatFunction* stat) { // Optimization: compile value expresion directly into target local register - if (isExprLocalReg(stat->name)) + if (int reg = getExprLocalReg(stat->name); reg >= 0) { - AstExprLocal* le = stat->name->as(); - LUAU_ASSERT(le); - - compileExpr(stat->func, getLocal(le->local)); + compileExpr(stat->func, uint8_t(reg)); return; } @@ -3399,6 +3500,12 @@ struct Compiler std::vector returnJumps; }; + struct Capture + { + LuauCaptureType type; + uint8_t data; + }; + BytecodeBuilder& bytecode; CompileOptions options; @@ -3422,6 +3529,7 @@ struct Compiler std::vector loopJumps; std::vector loops; std::vector inlineFrames; + std::vector captures; }; void compileOrThrow(BytecodeBuilder& bytecode, AstStatBlock* root, const AstNameTable& names, const CompileOptions& options) @@ -3465,6 +3573,9 @@ void compileOrThrow(BytecodeBuilder& bytecode, AstStatBlock* root, const AstName /* self= */ nullptr, AstArray(), /* vararg= */ Luau::Location(), root, /* functionDepth= */ 0, /* debugname= */ AstName()); uint32_t mainid = compiler.compileFunction(&main); + const Compiler::Function* mainf = compiler.functions.find(&main); + LUAU_ASSERT(mainf && mainf->upvals.empty()); + bytecode.setMainFunction(mainid); bytecode.finalize(); } diff --git a/Compiler/src/ConstantFolding.cpp b/Compiler/src/ConstantFolding.cpp index e4d59ea1..a62beeb1 100644 --- a/Compiler/src/ConstantFolding.cpp +++ b/Compiler/src/ConstantFolding.cpp @@ -3,8 +3,6 @@ #include -LUAU_FASTFLAG(LuauCompileSupportInlining) - namespace Luau { namespace Compile @@ -330,7 +328,7 @@ struct ConstantVisitor : AstVisitor { if (value.type != Constant::Type_Unknown) map[key] = value; - else if (!FFlag::LuauCompileSupportInlining || wasEmpty) + else if (wasEmpty) ; else if (Constant* old = map.find(key)) old->type = Constant::Type_Unknown; diff --git a/Sources.cmake b/Sources.cmake index d2430cc9..297f561a 100644 --- a/Sources.cmake +++ b/Sources.cmake @@ -73,6 +73,7 @@ target_sources(Luau.Analysis PRIVATE Analysis/include/Luau/ToString.h Analysis/include/Luau/Transpiler.h Analysis/include/Luau/TxnLog.h + Analysis/include/Luau/TypeArena.h Analysis/include/Luau/TypeAttach.h Analysis/include/Luau/TypedAllocator.h Analysis/include/Luau/TypeInfer.h @@ -108,6 +109,7 @@ target_sources(Luau.Analysis PRIVATE Analysis/src/ToString.cpp Analysis/src/Transpiler.cpp Analysis/src/TxnLog.cpp + Analysis/src/TypeArena.cpp Analysis/src/TypeAttach.cpp Analysis/src/TypedAllocator.cpp Analysis/src/TypeInfer.cpp diff --git a/VM/src/ltablib.cpp b/VM/src/ltablib.cpp index 9c1f387e..27187c61 100644 --- a/VM/src/ltablib.cpp +++ b/VM/src/ltablib.cpp @@ -10,10 +10,6 @@ #include "ldebug.h" #include "lvm.h" -LUAU_DYNAMIC_FASTFLAGVARIABLE(LuauTableMoveTelemetry2, false) - -void (*lua_table_move_telemetry)(lua_State* L, int f, int e, int t, int nf, int nt); - static int foreachi(lua_State* L) { luaL_checktype(L, 1, LUA_TTABLE); @@ -199,29 +195,6 @@ static int tmove(lua_State* L) int tt = !lua_isnoneornil(L, 5) ? 5 : 1; /* destination table */ luaL_checktype(L, tt, LUA_TTABLE); - void (*telemetrycb)(lua_State * L, int f, int e, int t, int nf, int nt) = lua_table_move_telemetry; - - if (DFFlag::LuauTableMoveTelemetry2 && telemetrycb && e >= f) - { - int nf = lua_objlen(L, 1); - int nt = lua_objlen(L, tt); - - bool report = false; - - // source index range must be in bounds in source table unless the table is empty (permits 1..#t moves) - if (!(f == 1 || (f >= 1 && f <= nf))) - report = true; - if (!(e == nf || (e >= 1 && e <= nf))) - report = true; - - // destination index must be in bounds in dest table or be exactly at the first empty element (permits concats) - if (!(t == nt + 1 || (t >= 1 && t <= nt))) - report = true; - - if (report) - telemetrycb(L, f, e, t, nf, nt); - } - if (e >= f) { /* otherwise, nothing to move */ luaL_argcheck(L, f > 0 || e < INT_MAX + f, 3, "too many elements to move"); diff --git a/VM/src/lvmexecute.cpp b/VM/src/lvmexecute.cpp index 3c7c276a..9e2eb268 100644 --- a/VM/src/lvmexecute.cpp +++ b/VM/src/lvmexecute.cpp @@ -17,9 +17,6 @@ #include LUAU_FASTFLAGVARIABLE(LuauIter, false) -LUAU_DYNAMIC_FASTFLAGVARIABLE(LuauIterCallTelemetry, false) - -void (*lua_iter_call_telemetry)(lua_State* L); // Disable c99-designator to avoid the warning in CGOTO dispatch table #ifdef __clang__ @@ -157,17 +154,6 @@ LUAU_NOINLINE static bool luau_loopFORG(lua_State* L, int a, int c) StkId ra = &L->base[a]; LUAU_ASSERT(ra + 3 <= L->top); - if (DFFlag::LuauIterCallTelemetry) - { - /* TODO: we might be able to stop supporting this depending on whether it's used in practice */ - void (*telemetrycb)(lua_State* L) = lua_iter_call_telemetry; - - if (telemetrycb && ttistable(ra) && fasttm(L, hvalue(ra)->metatable, TM_CALL)) - telemetrycb(L); - if (telemetrycb && ttisuserdata(ra) && fasttm(L, uvalue(ra)->metatable, TM_CALL)) - telemetrycb(L); - } - setobjs2s(L, ra + 3 + 2, ra + 2); setobjs2s(L, ra + 3 + 1, ra + 1); setobjs2s(L, ra + 3, ra); diff --git a/tests/AstQuery.test.cpp b/tests/AstQuery.test.cpp index 12c68450..f0017509 100644 --- a/tests/AstQuery.test.cpp +++ b/tests/AstQuery.test.cpp @@ -92,4 +92,17 @@ bar(foo()) CHECK_EQ("number", toString(*expectedOty)); } +TEST_CASE_FIXTURE(Fixture, "ast_ancestry_at_eof") +{ + check(R"( +if true then + )"); + + std::vector ancestry = findAstAncestryOfPosition(*getMainSourceModule(), Position(2, 4)); + REQUIRE_GE(ancestry.size(), 2); + AstStat* parentStat = ancestry[ancestry.size() - 2]->asStat(); + REQUIRE(bool(parentStat)); + REQUIRE(parentStat->is()); +} + TEST_SUITE_END(); diff --git a/tests/Autocomplete.test.cpp b/tests/Autocomplete.test.cpp index b4e9340c..caaccf4e 100644 --- a/tests/Autocomplete.test.cpp +++ b/tests/Autocomplete.test.cpp @@ -2772,6 +2772,8 @@ TEST_CASE_FIXTURE(ACBuiltinsFixture, "autocomplete_on_string_singletons") TEST_CASE_FIXTURE(ACFixture, "autocomplete_string_singletons") { + ScopedFastFlag sff{"LuauTwoPassAliasDefinitionFix", true}; + check(R"( type tag = "cat" | "dog" local function f(a: tag) end @@ -2844,6 +2846,8 @@ f(@1) TEST_CASE_FIXTURE(ACFixture, "autocomplete_string_singleton_escape") { + ScopedFastFlag sff{"LuauTwoPassAliasDefinitionFix", true}; + check(R"( type tag = "strange\t\"cat\"" | 'nice\t"dog"' local function f(x: tag) end diff --git a/tests/Compiler.test.cpp b/tests/Compiler.test.cpp index b032060e..cf27d191 100644 --- a/tests/Compiler.test.cpp +++ b/tests/Compiler.test.cpp @@ -4269,22 +4269,26 @@ FORNLOOP R3 -6 FORNLOOP R0 -11 RETURN R0 0 )"); +} - // can't unroll loops if the body has functions that refer to loop variables +TEST_CASE("LoopUnrollNestedClosure") +{ + ScopedFastFlag sff("LuauCompileNestedClosureO2", true); + + // if the body has functions that refer to loop variables, we unroll the loop and use MOVE+CAPTURE for upvalues CHECK_EQ("\n" + compileFunction(R"( -for i=1,1 do +for i=1,2 do local x = function() return i end end )", 1, 2), R"( -LOADN R2 1 -LOADN R0 1 LOADN R1 1 -FORNPREP R0 +3 -NEWCLOSURE R3 P0 -CAPTURE VAL R2 -FORNLOOP R0 -3 +NEWCLOSURE R0 P0 +CAPTURE VAL R1 +LOADN R1 2 +NEWCLOSURE R0 P0 +CAPTURE VAL R1 RETURN R0 0 )"); } @@ -4469,8 +4473,6 @@ RETURN R0 0 TEST_CASE("InlineBasic") { - ScopedFastFlag sff("LuauCompileSupportInlining", true); - // inline function that returns a constant CHECK_EQ("\n" + compileFunction(R"( local function foo() @@ -4550,10 +4552,72 @@ RETURN R1 1 )"); } +TEST_CASE("InlineBasicProhibited") +{ + ScopedFastFlag sff("LuauCompileNestedClosureO2", true); + + // we can't inline variadic functions + CHECK_EQ("\n" + compileFunction(R"( +local function foo(...) + return 42 +end + +local x = foo() +return x +)", + 1, 2), + R"( +DUPCLOSURE R0 K0 +MOVE R1 R0 +CALL R1 0 1 +RETURN R1 1 +)"); + + // we also can't inline functions that have internal loops + CHECK_EQ("\n" + compileFunction(R"( +local function foo() + for i=1,4 do end +end + +local x = foo() +return x +)", + 1, 2), + R"( +DUPCLOSURE R0 K0 +MOVE R1 R0 +CALL R1 0 1 +RETURN R1 1 +)"); +} + +TEST_CASE("InlineNestedClosures") +{ + ScopedFastFlag sff("LuauCompileNestedClosureO2", true); + + // we can inline functions that contain/return functions + CHECK_EQ("\n" + compileFunction(R"( +local function foo(x) + return function(y) return x + y end +end + +local x = foo(1)(2) +return x +)", + 2, 2), + R"( +DUPCLOSURE R0 K0 +LOADN R2 1 +NEWCLOSURE R1 P1 +CAPTURE VAL R2 +LOADN R2 2 +CALL R1 1 1 +RETURN R1 1 +)"); +} + TEST_CASE("InlineMutate") { - ScopedFastFlag sff("LuauCompileSupportInlining", true); - // if the argument is mutated, it gets a register even if the value is constant CHECK_EQ("\n" + compileFunction(R"( local function foo(a) @@ -4636,8 +4700,6 @@ RETURN R1 1 TEST_CASE("InlineUpval") { - ScopedFastFlag sff("LuauCompileSupportInlining", true); - // if the argument is an upvalue, we naturally need to copy it to a local CHECK_EQ("\n" + compileFunction(R"( local function foo(a) @@ -4705,8 +4767,6 @@ RETURN R1 1 TEST_CASE("InlineFallthrough") { - ScopedFastFlag sff("LuauCompileSupportInlining", true); - // if the function doesn't return, we still fill the results with nil CHECK_EQ("\n" + compileFunction(R"( local function foo() @@ -4759,8 +4819,6 @@ RETURN R1 -1 TEST_CASE("InlineCapture") { - ScopedFastFlag sff("LuauCompileSupportInlining", true); - // can't inline function with nested functions that capture locals because they might be constants CHECK_EQ("\n" + compileFunction(R"( local function foo(a) @@ -4782,12 +4840,9 @@ RETURN R2 -1 TEST_CASE("InlineArgMismatch") { - ScopedFastFlag sff("LuauCompileSupportInlining", true); - // when inlining a function, we must respect all the usual rules // caller might not have enough arguments - // TODO: we don't inline this atm CHECK_EQ("\n" + compileFunction(R"( local function foo(a) return a @@ -4799,13 +4854,11 @@ return x 1, 2), R"( DUPCLOSURE R0 K0 -MOVE R1 R0 -CALL R1 0 1 +LOADNIL R1 RETURN R1 1 )"); // caller might be using multret for arguments - // TODO: we don't inline this atm CHECK_EQ("\n" + compileFunction(R"( local function foo(a, b) return a + b @@ -4817,17 +4870,32 @@ return x 1, 2), R"( DUPCLOSURE R0 K0 -MOVE R1 R0 LOADK R3 K1 FASTCALL1 20 R3 +2 GETIMPORT R2 4 -CALL R2 1 -1 -CALL R1 -1 1 +CALL R2 1 2 +ADD R1 R2 R3 +RETURN R1 1 +)"); + + // caller might be using varargs for arguments + CHECK_EQ("\n" + compileFunction(R"( +local function foo(a, b) + return a + b +end + +local x = foo(...) +return x +)", + 1, 2), + R"( +DUPCLOSURE R0 K0 +GETVARARGS R2 2 +ADD R1 R2 R3 RETURN R1 1 )"); // caller might have too many arguments, but we still need to compute them for side effects - // TODO: we don't inline this atm CHECK_EQ("\n" + compileFunction(R"( local function foo(a) return a @@ -4839,19 +4907,34 @@ return x 1, 2), R"( DUPCLOSURE R0 K0 -MOVE R1 R0 +GETIMPORT R2 2 +CALL R2 0 1 +LOADN R1 42 +RETURN R1 1 +)"); + + // caller might not have enough arguments, and the arg might be mutated so it needs a register + CHECK_EQ("\n" + compileFunction(R"( +local function foo(a) + a = 42 + return a +end + +local x = foo() +return x +)", + 1, 2), + R"( +DUPCLOSURE R0 K0 +LOADNIL R2 LOADN R2 42 -GETIMPORT R3 2 -CALL R3 0 -1 -CALL R1 -1 1 +MOVE R1 R2 RETURN R1 1 )"); } TEST_CASE("InlineMultiple") { - ScopedFastFlag sff("LuauCompileSupportInlining", true); - // we call this with a different set of variable/constant args CHECK_EQ("\n" + compileFunction(R"( local function foo(a, b) @@ -4880,8 +4963,6 @@ RETURN R3 4 TEST_CASE("InlineChain") { - ScopedFastFlag sff("LuauCompileSupportInlining", true); - // inline a chain of functions CHECK_EQ("\n" + compileFunction(R"( local function foo(a, b) @@ -4912,8 +4993,6 @@ RETURN R3 1 TEST_CASE("InlineThresholds") { - ScopedFastFlag sff("LuauCompileSupportInlining", true); - ScopedFastInt sfis[] = { {"LuauCompileInlineThreshold", 25}, {"LuauCompileInlineThresholdMaxBoost", 300}, @@ -4988,8 +5067,6 @@ RETURN R3 1 TEST_CASE("InlineIIFE") { - ScopedFastFlag sff("LuauCompileSupportInlining", true); - // IIFE with arguments CHECK_EQ("\n" + compileFunction(R"( function choose(a, b, c) @@ -5025,8 +5102,6 @@ RETURN R3 1 TEST_CASE("InlineRecurseArguments") { - ScopedFastFlag sff("LuauCompileSupportInlining", true); - // we can't inline a function if it's used to compute its own arguments CHECK_EQ("\n" + compileFunction(R"( local function foo(a, b) @@ -5036,22 +5111,20 @@ foo(foo(foo,foo(foo,foo))[foo]) 1, 2), R"( DUPCLOSURE R0 K0 -MOVE R1 R0 +MOVE R2 R0 +MOVE R3 R0 MOVE R4 R0 MOVE R5 R0 MOVE R6 R0 -CALL R4 2 1 -LOADNIL R3 -GETTABLE R2 R3 R0 -CALL R1 1 0 +CALL R4 2 -1 +CALL R2 -1 1 +GETTABLE R1 R2 R0 RETURN R0 0 )"); } TEST_CASE("InlineFastCallK") { - ScopedFastFlag sff("LuauCompileSupportInlining", true); - CHECK_EQ("\n" + compileFunction(R"( local function set(l0) rawset({}, l0) @@ -5080,8 +5153,6 @@ RETURN R0 0 TEST_CASE("InlineExprIndexK") { - ScopedFastFlag sff("LuauCompileSupportInlining", true); - CHECK_EQ("\n" + compileFunction(R"( local _ = function(l0) local _ = nil @@ -5141,6 +5212,58 @@ RETURN R0 0 )"); } +TEST_CASE("InlineHiddenMutation") +{ + // when the argument is assigned inside the function, we can't reuse the local + CHECK_EQ("\n" + compileFunction(R"( +local function foo(a) + a = 42 + return a +end + +local x = ... +local y = foo(x :: number) +return y +)", + 1, 2), + R"( +DUPCLOSURE R0 K0 +GETVARARGS R1 1 +MOVE R3 R1 +LOADN R3 42 +MOVE R2 R3 +RETURN R2 1 +)"); + + // and neither can we do that when it's assigned outside the function + CHECK_EQ("\n" + compileFunction(R"( +local function foo(a) + mutator() + return a +end + +local x = ... +mutator = function() x = 42 end + +local y = foo(x :: number) +return y +)", + 2, 2), + R"( +DUPCLOSURE R0 K0 +GETVARARGS R1 1 +NEWCLOSURE R2 P1 +CAPTURE REF R1 +SETGLOBAL R2 K1 +MOVE R3 R1 +GETGLOBAL R4 K1 +CALL R4 0 0 +MOVE R2 R3 +CLOSEUPVALS R1 +RETURN R2 1 +)"); +} + TEST_CASE("ReturnConsecutive") { // we can return a single local directly @@ -5193,6 +5316,16 @@ return )"), R"( RETURN R0 0 +)"); + + // this optimization also works in presence of group / type casts + CHECK_EQ("\n" + compileFunction0(R"( +local x, y = ... +return (x), y :: number +)"), + R"( +GETVARARGS R0 2 +RETURN R0 2 )"); } diff --git a/tests/Module.test.cpp b/tests/Module.test.cpp index 4a999861..c7e18efd 100644 --- a/tests/Module.test.cpp +++ b/tests/Module.test.cpp @@ -198,10 +198,6 @@ TEST_CASE_FIXTURE(Fixture, "clone_class") TEST_CASE_FIXTURE(Fixture, "clone_free_types") { - ScopedFastFlag sff[]{ - {"LuauLosslessClone", true}, - }; - TypeVar freeTy(FreeTypeVar{TypeLevel{}}); TypePackVar freeTp(FreeTypePack{TypeLevel{}}); @@ -218,8 +214,6 @@ TEST_CASE_FIXTURE(Fixture, "clone_free_types") TEST_CASE_FIXTURE(Fixture, "clone_free_tables") { - ScopedFastFlag sff{"LuauLosslessClone", true}; - TypeVar tableTy{TableTypeVar{}}; TableTypeVar* ttv = getMutable(&tableTy); ttv->state = TableState::Free; @@ -252,8 +246,6 @@ TEST_CASE_FIXTURE(Fixture, "clone_constrained_intersection") TEST_CASE_FIXTURE(BuiltinsFixture, "clone_self_property") { - ScopedFastFlag sff{"LuauAnyInIsOptionalIsOptional", true}; - fileResolver.source["Module/A"] = R"( --!nonstrict local a = {} diff --git a/tests/NonstrictMode.test.cpp b/tests/NonstrictMode.test.cpp index 69430b1c..83c526ef 100644 --- a/tests/NonstrictMode.test.cpp +++ b/tests/NonstrictMode.test.cpp @@ -150,8 +150,6 @@ TEST_CASE_FIXTURE(Fixture, "parameters_having_type_any_are_optional") TEST_CASE_FIXTURE(Fixture, "local_tables_are_not_any") { - ScopedFastFlag sff{"LuauAnyInIsOptionalIsOptional", true}; - CheckResult result = check(R"( --!nonstrict local T = {} @@ -169,8 +167,6 @@ TEST_CASE_FIXTURE(Fixture, "local_tables_are_not_any") TEST_CASE_FIXTURE(Fixture, "offer_a_hint_if_you_use_a_dot_instead_of_a_colon") { - ScopedFastFlag sff{"LuauAnyInIsOptionalIsOptional", true}; - CheckResult result = check(R"( --!nonstrict local T = {} diff --git a/tests/Normalize.test.cpp b/tests/Normalize.test.cpp index 41830682..dd49eb01 100644 --- a/tests/Normalize.test.cpp +++ b/tests/Normalize.test.cpp @@ -683,6 +683,7 @@ TEST_CASE_FIXTURE(Fixture, "cyclic_table_is_marked_normal") { ScopedFastFlag flags[] = { {"LuauLowerBoundsCalculation", true}, + {"LuauNormalizeFlagIsConservative", false} }; check(R"( @@ -697,6 +698,26 @@ TEST_CASE_FIXTURE(Fixture, "cyclic_table_is_marked_normal") CHECK(t->normal); } +// Unfortunately, getting this right in the general case is difficult. +TEST_CASE_FIXTURE(Fixture, "cyclic_table_is_not_marked_normal") +{ + ScopedFastFlag flags[] = { + {"LuauLowerBoundsCalculation", true}, + {"LuauNormalizeFlagIsConservative", true} + }; + + check(R"( + type Fiber = { + return_: Fiber? + } + + local f: Fiber + )"); + + TypeId t = requireType("f"); + CHECK(!t->normal); +} + TEST_CASE_FIXTURE(Fixture, "variadic_tail_is_marked_normal") { ScopedFastFlag flags[] = { @@ -997,4 +1018,28 @@ TEST_CASE_FIXTURE(Fixture, "fuzz_failure_bound_type_is_normal_but_not_its_bounde LUAU_REQUIRE_ERRORS(result); } +// We had an issue where a normal BoundTypeVar might point at a non-normal BoundTypeVar if it in turn pointed to a +// normal TypeVar because we were calling follow() in an improper place. +TEST_CASE_FIXTURE(Fixture, "bound_typevars_should_only_be_marked_normal_if_their_pointee_is_normal") +{ + ScopedFastFlag sff[]{ + {"LuauLowerBoundsCalculation", true}, + {"LuauNormalizeFlagIsConservative", true}, + }; + + CheckResult result = check(R"( + local T = {} + + function T:M() + local function f(a) + print(self.prop) + self:g(a) + self.prop = a + end + end + + return T + )"); +} + TEST_SUITE_END(); diff --git a/tests/RuntimeLimits.test.cpp b/tests/RuntimeLimits.test.cpp index c16f60d5..14c17614 100644 --- a/tests/RuntimeLimits.test.cpp +++ b/tests/RuntimeLimits.test.cpp @@ -22,8 +22,6 @@ struct LimitFixture : BuiltinsFixture #if defined(_NOOPT) || defined(_DEBUG) ScopedFastInt LuauTypeInferRecursionLimit{"LuauTypeInferRecursionLimit", 100}; #endif - - ScopedFastFlag LuauJustOneCallFrameForHaveSeen{"LuauJustOneCallFrameForHaveSeen", true}; }; template diff --git a/tests/ToString.test.cpp b/tests/ToString.test.cpp index 50d0838e..b854bc51 100644 --- a/tests/ToString.test.cpp +++ b/tests/ToString.test.cpp @@ -126,6 +126,39 @@ TEST_CASE_FIXTURE(Fixture, "functions_are_always_parenthesized_in_unions_or_inte CHECK_EQ(toString(&itv), "((number, string) -> (string, number)) & ((string, number) -> (number, string))"); } +TEST_CASE_FIXTURE(Fixture, "intersections_respects_use_line_breaks") +{ + CheckResult result = check(R"( + local a: ((string) -> string) & ((number) -> number) + )"); + + ToStringOptions opts; + opts.useLineBreaks = true; + + //clang-format off + CHECK_EQ("((number) -> number)\n" + "& ((string) -> string)", + toString(requireType("a"), opts)); + //clang-format on +} + +TEST_CASE_FIXTURE(Fixture, "unions_respects_use_line_breaks") +{ + CheckResult result = check(R"( + local a: string | number | boolean + )"); + + ToStringOptions opts; + opts.useLineBreaks = true; + + //clang-format off + CHECK_EQ("boolean\n" + "| number\n" + "| string", + toString(requireType("a"), opts)); + //clang-format on +} + TEST_CASE_FIXTURE(Fixture, "quit_stringifying_table_type_when_length_is_exceeded") { TableTypeVar ttv{}; @@ -617,4 +650,52 @@ TEST_CASE_FIXTURE(Fixture, "toStringNamedFunction_overrides_param_names") CHECK_EQ("test(first: a, second: string, ...: number): a", toStringNamedFunction("test", *ftv, opts)); } +TEST_CASE_FIXTURE(Fixture, "pick_distinct_names_for_mixed_explicit_and_implicit_generics") +{ + ScopedFastFlag sff[] = { + {"LuauAlwaysQuantify", true}, + }; + + CheckResult result = check(R"( + function foo(x: a, y) end + )"); + + CHECK("(a, b) -> ()" == toString(requireType("foo"))); +} + +TEST_CASE_FIXTURE(Fixture, "toStringNamedFunction_include_self_param") +{ + ScopedFastFlag flag{"LuauDocFuncParameters", true}; + CheckResult result = check(R"( + local foo = {} + function foo:method(arg: string): () + end + )"); + + TypeId parentTy = requireType("foo"); + auto ttv = get(follow(parentTy)); + auto ftv = get(ttv->props.at("method").type); + + CHECK_EQ("foo:method(self: a, arg: string): ()", toStringNamedFunction("foo:method", *ftv)); +} + + +TEST_CASE_FIXTURE(Fixture, "toStringNamedFunction_hide_self_param") +{ + ScopedFastFlag flag{"LuauDocFuncParameters", true}; + CheckResult result = check(R"( + local foo = {} + function foo:method(arg: string): () + end + )"); + + TypeId parentTy = requireType("foo"); + auto ttv = get(follow(parentTy)); + auto ftv = get(ttv->props.at("method").type); + + ToStringOptions opts; + opts.hideFunctionSelfArgument = true; + CHECK_EQ("foo:method(arg: string): ()", toStringNamedFunction("foo:method", *ftv, opts)); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.builtins.test.cpp b/tests/TypeInfer.builtins.test.cpp index b710ea0d..aa4ca415 100644 --- a/tests/TypeInfer.builtins.test.cpp +++ b/tests/TypeInfer.builtins.test.cpp @@ -878,8 +878,6 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "dont_add_definitions_to_persistent_types") TEST_CASE_FIXTURE(BuiltinsFixture, "assert_removes_falsy_types") { ScopedFastFlag sff[]{ - {"LuauAssertStripsFalsyTypes", true}, - {"LuauDiscriminableUnions2", true}, {"LuauWidenIfSupertypeIsFree2", true}, }; @@ -899,8 +897,6 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "assert_removes_falsy_types") TEST_CASE_FIXTURE(BuiltinsFixture, "assert_removes_falsy_types2") { ScopedFastFlag sff[]{ - {"LuauAssertStripsFalsyTypes", true}, - {"LuauDiscriminableUnions2", true}, {"LuauWidenIfSupertypeIsFree2", true}, }; @@ -916,11 +912,6 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "assert_removes_falsy_types2") TEST_CASE_FIXTURE(BuiltinsFixture, "assert_removes_falsy_types_even_from_type_pack_tail_but_only_for_the_first_type") { - ScopedFastFlag sff[]{ - {"LuauAssertStripsFalsyTypes", true}, - {"LuauDiscriminableUnions2", true}, - }; - CheckResult result = check(R"( local function f(...: number?) return assert(...) @@ -933,11 +924,6 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "assert_removes_falsy_types_even_from_type_pa TEST_CASE_FIXTURE(BuiltinsFixture, "assert_returns_false_and_string_iff_it_knows_the_first_argument_cannot_be_truthy") { - ScopedFastFlag sff[]{ - {"LuauAssertStripsFalsyTypes", true}, - {"LuauDiscriminableUnions2", true}, - }; - CheckResult result = check(R"( local function f(x: nil) return assert(x, "hmm") diff --git a/tests/TypeInfer.functions.test.cpp b/tests/TypeInfer.functions.test.cpp index 14f1f703..a28ba49e 100644 --- a/tests/TypeInfer.functions.test.cpp +++ b/tests/TypeInfer.functions.test.cpp @@ -1496,8 +1496,6 @@ caused by: TEST_CASE_FIXTURE(Fixture, "strict_mode_ok_with_missing_arguments") { - ScopedFastFlag sff{"LuauAnyInIsOptionalIsOptional", true}; - CheckResult result = check(R"( local function f(x: any) end f() diff --git a/tests/TypeInfer.generics.test.cpp b/tests/TypeInfer.generics.test.cpp index de0c9391..78a5fee7 100644 --- a/tests/TypeInfer.generics.test.cpp +++ b/tests/TypeInfer.generics.test.cpp @@ -1121,4 +1121,78 @@ TEST_CASE_FIXTURE(Fixture, "substitution_with_bound_table") LUAU_REQUIRE_NO_ERRORS(result); } +TEST_CASE_FIXTURE(Fixture, "apply_type_function_nested_generics1") +{ + ScopedFastFlag sff{"LuauApplyTypeFunctionFix", true}; + + // https://github.com/Roblox/luau/issues/484 + CheckResult result = check(R"( +--!strict +type MyObject = { + getReturnValue: (cb: () -> V) -> V +} +local object: MyObject = { + getReturnValue = function(cb: () -> U): U + return cb() + end, +} + +type ComplexObject = { + id: T, + nested: MyObject +} + +local complex: ComplexObject = { + id = "Foo", + nested = object, +} + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "apply_type_function_nested_generics2") +{ + ScopedFastFlag sff{"LuauApplyTypeFunctionFix", true}; + + // https://github.com/Roblox/luau/issues/484 + CheckResult result = check(R"( +--!strict +type MyObject = { + getReturnValue: (cb: () -> V) -> V +} +type ComplexObject = { + id: T, + nested: MyObject +} + +local complex2: ComplexObject = nil + +local x = complex2.nested.getReturnValue(function(): string + return "" +end) + +local y = complex2.nested.getReturnValue(function() + return 3 +end) + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "quantify_functions_even_if_they_have_an_explicit_generic") +{ + ScopedFastFlag sff[] = { + {"LuauAlwaysQuantify", true}, + }; + + CheckResult result = check(R"( + function foo(f, x: X) + return f(x) + end + )"); + + CHECK("((X) -> (a...), X) -> (a...)" == toString(requireType("foo"))); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.intersectionTypes.test.cpp b/tests/TypeInfer.intersectionTypes.test.cpp index 41bc0c21..f75b2d11 100644 --- a/tests/TypeInfer.intersectionTypes.test.cpp +++ b/tests/TypeInfer.intersectionTypes.test.cpp @@ -177,8 +177,6 @@ TEST_CASE_FIXTURE(Fixture, "index_on_an_intersection_type_with_property_guarante TEST_CASE_FIXTURE(Fixture, "index_on_an_intersection_type_works_at_arbitrary_depth") { - ScopedFastFlag sff{"LuauDoNotTryToReduce", true}; - CheckResult result = check(R"( type A = {x: {y: {z: {thing: string}}}} type B = {x: {y: {z: {thing: string}}}} diff --git a/tests/TypeInfer.loops.test.cpp b/tests/TypeInfer.loops.test.cpp index 765419c6..a3cae3de 100644 --- a/tests/TypeInfer.loops.test.cpp +++ b/tests/TypeInfer.loops.test.cpp @@ -475,8 +475,6 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "loop_typecheck_crash_on_empty_optional") TEST_CASE_FIXTURE(Fixture, "fuzz_fail_missing_instantitation_follow") { - ScopedFastFlag luauInstantiateFollows{"LuauInstantiateFollows", true}; - // Just check that this doesn't assert check(R"( --!nonstrict diff --git a/tests/TypeInfer.operators.test.cpp b/tests/TypeInfer.operators.test.cpp index 51f6fdfb..03614938 100644 --- a/tests/TypeInfer.operators.test.cpp +++ b/tests/TypeInfer.operators.test.cpp @@ -728,8 +728,6 @@ TEST_CASE_FIXTURE(Fixture, "operator_eq_verifies_types_do_intersect") TEST_CASE_FIXTURE(Fixture, "operator_eq_operands_are_not_subtypes_of_each_other_but_has_overlap") { - ScopedFastFlag sff1{"LuauEqConstraint", true}; - CheckResult result = check(R"( local function f(a: string | number, b: boolean | number) return a == b diff --git a/tests/TypeInfer.provisional.test.cpp b/tests/TypeInfer.provisional.test.cpp index ee3ae972..9d227895 100644 --- a/tests/TypeInfer.provisional.test.cpp +++ b/tests/TypeInfer.provisional.test.cpp @@ -7,7 +7,6 @@ #include -LUAU_FASTFLAG(LuauEqConstraint) LUAU_FASTFLAG(LuauLowerBoundsCalculation) using namespace Luau; @@ -183,8 +182,6 @@ TEST_CASE_FIXTURE(Fixture, "operator_eq_completely_incompatible") // We'll need to not only report an error on `a == b`, but also to refine both operands as `never` in the `==` branch. TEST_CASE_FIXTURE(Fixture, "lvalue_equals_another_lvalue_with_no_overlap") { - ScopedFastFlag sff1{"LuauEqConstraint", true}; - CheckResult result = check(R"( local function f(a: string, b: boolean?) if a == b then @@ -208,8 +205,6 @@ TEST_CASE_FIXTURE(Fixture, "lvalue_equals_another_lvalue_with_no_overlap") // Just needs to fully support equality refinement. Which is annoying without type states. TEST_CASE_FIXTURE(Fixture, "discriminate_from_x_not_equal_to_nil") { - ScopedFastFlag sff{"LuauDiscriminableUnions2", true}; - CheckResult result = check(R"( type T = {x: string, y: number} | {x: nil, y: nil} @@ -471,4 +466,35 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "function_returns_many_things_but_first_of_it CHECK_EQ("boolean", toString(requireType("b"))); } +TEST_CASE_FIXTURE(Fixture, "constrained_is_level_dependent") +{ + ScopedFastFlag sff[]{ + {"LuauLowerBoundsCalculation", true}, + {"LuauNormalizeFlagIsConservative", true}, + }; + + CheckResult result = check(R"( + local function f(o) + local t = {} + t[o] = true + + local function foo(o) + o:m1() + t[o] = nil + end + + local function bar(o) + o:m2() + t[o] = true + end + + return t + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + // TODO: We're missing generics a... and b... + CHECK_EQ("(t1) -> {| [t1]: boolean |} where t1 = t2 ; t2 = {+ m1: (t1) -> (a...), m2: (t2) -> (b...) +}", toString(requireType("f"))); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.refinements.test.cpp b/tests/TypeInfer.refinements.test.cpp index 8c130490..85a3334c 100644 --- a/tests/TypeInfer.refinements.test.cpp +++ b/tests/TypeInfer.refinements.test.cpp @@ -7,7 +7,6 @@ #include "doctest.h" -LUAU_FASTFLAG(LuauDiscriminableUnions2) LUAU_FASTFLAG(LuauWeakEqConstraint) LUAU_FASTFLAG(LuauLowerBoundsCalculation) @@ -268,18 +267,10 @@ TEST_CASE_FIXTURE(Fixture, "typeguard_only_look_up_types_from_global_scope") end )"); - if (FFlag::LuauDiscriminableUnions2) - { - LUAU_REQUIRE_NO_ERRORS(result); + LUAU_REQUIRE_NO_ERRORS(result); - CHECK_EQ("*unknown*", toString(requireTypeAtPosition({8, 44}))); - CHECK_EQ("*unknown*", toString(requireTypeAtPosition({9, 38}))); - } - else - { - LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK_EQ("Type 'number' has no overlap with 'string'", toString(result.errors[0])); - } + CHECK_EQ("*unknown*", toString(requireTypeAtPosition({8, 44}))); + CHECK_EQ("*unknown*", toString(requireTypeAtPosition({9, 38}))); } TEST_CASE_FIXTURE(Fixture, "call_a_more_specific_function_using_typeguard") @@ -378,8 +369,6 @@ caused by: TEST_CASE_FIXTURE(Fixture, "lvalue_is_equal_to_another_lvalue") { - ScopedFastFlag sff1{"LuauEqConstraint", true}; - CheckResult result = check(R"( local function f(a: (string | number)?, b: boolean?) if a == b then @@ -392,28 +381,15 @@ TEST_CASE_FIXTURE(Fixture, "lvalue_is_equal_to_another_lvalue") LUAU_REQUIRE_NO_ERRORS(result); - if (FFlag::LuauWeakEqConstraint) - { - CHECK_EQ(toString(requireTypeAtPosition({3, 33})), "(number | string)?"); // a == b - CHECK_EQ(toString(requireTypeAtPosition({3, 36})), "boolean?"); // a == b + CHECK_EQ(toString(requireTypeAtPosition({3, 33})), "(number | string)?"); // a == b + CHECK_EQ(toString(requireTypeAtPosition({3, 36})), "boolean?"); // a == b - CHECK_EQ(toString(requireTypeAtPosition({5, 33})), "(number | string)?"); // a ~= b - CHECK_EQ(toString(requireTypeAtPosition({5, 36})), "boolean?"); // a ~= b - } - else - { - CHECK_EQ(toString(requireTypeAtPosition({3, 33})), "nil"); // a == b - CHECK_EQ(toString(requireTypeAtPosition({3, 36})), "nil"); // a == b - - CHECK_EQ(toString(requireTypeAtPosition({5, 33})), "(number | string)?"); // a ~= b - CHECK_EQ(toString(requireTypeAtPosition({5, 36})), "boolean?"); // a ~= b - } + CHECK_EQ(toString(requireTypeAtPosition({5, 33})), "(number | string)?"); // a ~= b + CHECK_EQ(toString(requireTypeAtPosition({5, 36})), "boolean?"); // a ~= b } TEST_CASE_FIXTURE(Fixture, "lvalue_is_equal_to_a_term") { - ScopedFastFlag sff1{"LuauEqConstraint", true}; - CheckResult result = check(R"( local function f(a: (string | number)?) if a == 1 then @@ -426,24 +402,12 @@ TEST_CASE_FIXTURE(Fixture, "lvalue_is_equal_to_a_term") LUAU_REQUIRE_NO_ERRORS(result); - if (FFlag::LuauWeakEqConstraint) - { - CHECK_EQ(toString(requireTypeAtPosition({3, 28})), "(number | string)?"); // a == 1 - CHECK_EQ(toString(requireTypeAtPosition({5, 28})), "(number | string)?"); // a ~= 1 - } - else - { - CHECK_EQ(toString(requireTypeAtPosition({3, 28})), "number"); // a == 1 - CHECK_EQ(toString(requireTypeAtPosition({5, 28})), "(number | string)?"); // a ~= 1 - } + CHECK_EQ(toString(requireTypeAtPosition({3, 28})), "(number | string)?"); // a == 1 + CHECK_EQ(toString(requireTypeAtPosition({5, 28})), "(number | string)?"); // a ~= 1 } TEST_CASE_FIXTURE(Fixture, "term_is_equal_to_an_lvalue") { - ScopedFastFlag sff[] = { - {"LuauDiscriminableUnions2", true}, - }; - CheckResult result = check(R"( local function f(a: (string | number)?) if "hello" == a then @@ -462,8 +426,6 @@ TEST_CASE_FIXTURE(Fixture, "term_is_equal_to_an_lvalue") TEST_CASE_FIXTURE(Fixture, "lvalue_is_not_nil") { - ScopedFastFlag sff1{"LuauEqConstraint", true}; - CheckResult result = check(R"( local function f(a: (string | number)?) if a ~= nil then @@ -476,21 +438,12 @@ TEST_CASE_FIXTURE(Fixture, "lvalue_is_not_nil") LUAU_REQUIRE_NO_ERRORS(result); - if (FFlag::LuauWeakEqConstraint) - { - CHECK_EQ(toString(requireTypeAtPosition({3, 28})), "number | string"); // a ~= nil - CHECK_EQ(toString(requireTypeAtPosition({5, 28})), "(number | string)?"); // a == nil - } - else - { - CHECK_EQ(toString(requireTypeAtPosition({3, 28})), "number | string"); // a ~= nil - CHECK_EQ(toString(requireTypeAtPosition({5, 28})), "nil"); // a == nil - } + CHECK_EQ(toString(requireTypeAtPosition({3, 28})), "number | string"); // a ~= nil + CHECK_EQ(toString(requireTypeAtPosition({5, 28})), "(number | string)?"); // a == nil } TEST_CASE_FIXTURE(Fixture, "free_type_is_equal_to_an_lvalue") { - ScopedFastFlag sff{"LuauDiscriminableUnions2", true}; ScopedFastFlag sff2{"LuauWeakEqConstraint", true}; CheckResult result = check(R"( @@ -509,8 +462,6 @@ TEST_CASE_FIXTURE(Fixture, "free_type_is_equal_to_an_lvalue") TEST_CASE_FIXTURE(Fixture, "unknown_lvalue_is_not_synonymous_with_other_on_not_equal") { - ScopedFastFlag sff1{"LuauEqConstraint", true}; - CheckResult result = check(R"( local function f(a: any, b: {x: number}?) if a ~= b then @@ -521,22 +472,12 @@ TEST_CASE_FIXTURE(Fixture, "unknown_lvalue_is_not_synonymous_with_other_on_not_e LUAU_REQUIRE_NO_ERRORS(result); - if (FFlag::LuauWeakEqConstraint) - { - CHECK_EQ(toString(requireTypeAtPosition({3, 33})), "any"); // a ~= b - CHECK_EQ(toString(requireTypeAtPosition({3, 36})), "{| x: number |}?"); // a ~= b - } - else - { - CHECK_EQ(toString(requireTypeAtPosition({3, 33})), "any"); // a ~= b - CHECK_EQ(toString(requireTypeAtPosition({3, 36})), "{| x: number |}"); // a ~= b - } + CHECK_EQ(toString(requireTypeAtPosition({3, 33})), "any"); // a ~= b + CHECK_EQ(toString(requireTypeAtPosition({3, 36})), "{| x: number |}?"); // a ~= b } TEST_CASE_FIXTURE(Fixture, "string_not_equal_to_string_or_nil") { - ScopedFastFlag sff1{"LuauEqConstraint", true}; - CheckResult result = check(R"( local t: {string} = {"hello"} @@ -554,18 +495,8 @@ TEST_CASE_FIXTURE(Fixture, "string_not_equal_to_string_or_nil") CHECK_EQ(toString(requireTypeAtPosition({6, 29})), "string"); // a ~= b CHECK_EQ(toString(requireTypeAtPosition({6, 32})), "string?"); // a ~= b - if (FFlag::LuauWeakEqConstraint) - { - CHECK_EQ(toString(requireTypeAtPosition({8, 29})), "string"); // a == b - CHECK_EQ(toString(requireTypeAtPosition({8, 32})), "string?"); // a == b - } - else - { - // This is technically not wrong, but it's also wrong at the same time. - // The refinement code is none the wiser about the fact we pulled a string out of an array, so it has no choice but to narrow as just string. - CHECK_EQ(toString(requireTypeAtPosition({8, 29})), "string"); // a == b - CHECK_EQ(toString(requireTypeAtPosition({8, 32})), "string"); // a == b - } + CHECK_EQ(toString(requireTypeAtPosition({8, 29})), "string"); // a == b + CHECK_EQ(toString(requireTypeAtPosition({8, 32})), "string?"); // a == b } TEST_CASE_FIXTURE(Fixture, "narrow_property_of_a_bounded_variable") @@ -594,16 +525,7 @@ TEST_CASE_FIXTURE(Fixture, "type_narrow_to_vector") end )"); - if (FFlag::LuauDiscriminableUnions2) - { - LUAU_REQUIRE_NO_ERRORS(result); - } - else - { - // This is kinda weird to see, but this actually only happens in Luau without Roblox type bindings because we don't have a Vector3 type. - LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK_EQ("Unknown type 'Vector3'", toString(result.errors[0])); - } + LUAU_REQUIRE_NO_ERRORS(result); CHECK_EQ("*unknown*", toString(requireTypeAtPosition({3, 28}))); } @@ -1009,10 +931,6 @@ TEST_CASE_FIXTURE(Fixture, "apply_refinements_on_astexprindexexpr_whose_subscrip TEST_CASE_FIXTURE(Fixture, "discriminate_from_truthiness_of_x") { - ScopedFastFlag sff[] = { - {"LuauDiscriminableUnions2", true}, - }; - CheckResult result = check(R"( type T = {tag: "missing", x: nil} | {tag: "exists", x: string} @@ -1033,10 +951,6 @@ TEST_CASE_FIXTURE(Fixture, "discriminate_from_truthiness_of_x") TEST_CASE_FIXTURE(Fixture, "discriminate_tag") { - ScopedFastFlag sff[] = { - {"LuauDiscriminableUnions2", true}, - }; - CheckResult result = check(R"( type Cat = {tag: "Cat", name: string, catfood: string} type Dog = {tag: "Dog", name: string, dogfood: string} @@ -1070,11 +984,6 @@ TEST_CASE_FIXTURE(Fixture, "and_or_peephole_refinement") TEST_CASE_FIXTURE(Fixture, "narrow_boolean_to_true_or_false") { - ScopedFastFlag sff[]{ - {"LuauDiscriminableUnions2", true}, - {"LuauAssertStripsFalsyTypes", true}, - }; - CheckResult result = check(R"( local function is_true(b: true) end local function is_false(b: false) end @@ -1093,11 +1002,6 @@ TEST_CASE_FIXTURE(Fixture, "narrow_boolean_to_true_or_false") TEST_CASE_FIXTURE(Fixture, "discriminate_on_properties_of_disjoint_tables_where_that_property_is_true_or_false") { - ScopedFastFlag sff[]{ - {"LuauDiscriminableUnions2", true}, - {"LuauAssertStripsFalsyTypes", true}, - }; - CheckResult result = check(R"( type Ok = { ok: true, value: T } type Err = { ok: false, error: E } @@ -1117,8 +1021,6 @@ TEST_CASE_FIXTURE(Fixture, "discriminate_on_properties_of_disjoint_tables_where_ TEST_CASE_FIXTURE(Fixture, "refine_a_property_not_to_be_nil_through_an_intersection_table") { - ScopedFastFlag sff{"LuauDoNotTryToReduce", true}; - CheckResult result = check(R"( type T = {} & {f: ((string) -> string)?} local function f(t: T, x) @@ -1133,10 +1035,6 @@ TEST_CASE_FIXTURE(Fixture, "refine_a_property_not_to_be_nil_through_an_intersect TEST_CASE_FIXTURE(RefinementClassFixture, "discriminate_from_isa_of_x") { - ScopedFastFlag sff[] = { - {"LuauDiscriminableUnions2", true}, - }; - CheckResult result = check(R"( type T = {tag: "Part", x: Part} | {tag: "Folder", x: Folder} @@ -1171,14 +1069,7 @@ TEST_CASE_FIXTURE(RefinementClassFixture, "typeguard_cast_free_table_to_vector") end )"); - if (FFlag::LuauDiscriminableUnions2) - LUAU_REQUIRE_NO_ERRORS(result); - else - { - LUAU_REQUIRE_ERROR_COUNT(1, result); - - CHECK_EQ("Type '{+ X: a, Y: b, Z: c +}' could not be converted into 'Instance'", toString(result.errors[0])); - } + LUAU_REQUIRE_NO_ERRORS(result); CHECK_EQ("Vector3", toString(requireTypeAtPosition({5, 28}))); // type(vec) == "vector" diff --git a/tests/TypeInfer.singletons.test.cpp b/tests/TypeInfer.singletons.test.cpp index 79eeb824..d90dfbb5 100644 --- a/tests/TypeInfer.singletons.test.cpp +++ b/tests/TypeInfer.singletons.test.cpp @@ -139,6 +139,8 @@ TEST_CASE_FIXTURE(Fixture, "enums_using_singletons") TEST_CASE_FIXTURE(Fixture, "enums_using_singletons_mismatch") { + ScopedFastFlag sff{"LuauTwoPassAliasDefinitionFix", true}; + CheckResult result = check(R"( type MyEnum = "foo" | "bar" | "baz" local a : MyEnum = "bang" @@ -325,8 +327,6 @@ local a: Animal = if true then { tag = 'cat', catfood = 'something' } else { tag TEST_CASE_FIXTURE(Fixture, "widen_the_supertype_if_it_is_free_and_subtype_has_singleton") { ScopedFastFlag sff[]{ - {"LuauEqConstraint", true}, - {"LuauDiscriminableUnions2", true}, {"LuauWidenIfSupertypeIsFree2", true}, {"LuauWeakEqConstraint", false}, }; @@ -350,11 +350,8 @@ TEST_CASE_FIXTURE(Fixture, "widen_the_supertype_if_it_is_free_and_subtype_has_si TEST_CASE_FIXTURE(Fixture, "return_type_of_f_is_not_widened") { ScopedFastFlag sff[]{ - {"LuauDiscriminableUnions2", true}, - {"LuauEqConstraint", true}, {"LuauWidenIfSupertypeIsFree2", true}, {"LuauWeakEqConstraint", false}, - {"LuauDoNotAccidentallyDependOnPointerOrdering", true}, }; CheckResult result = check(R"( @@ -390,7 +387,6 @@ TEST_CASE_FIXTURE(Fixture, "widening_happens_almost_everywhere") TEST_CASE_FIXTURE(Fixture, "widening_happens_almost_everywhere_except_for_tables") { ScopedFastFlag sff[]{ - {"LuauDiscriminableUnions2", true}, {"LuauWidenIfSupertypeIsFree2", true}, }; @@ -419,6 +415,7 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "table_insert_with_a_singleton_argument") { ScopedFastFlag sff[]{ {"LuauWidenIfSupertypeIsFree2", true}, + {"LuauWeakEqConstraint", true}, }; CheckResult result = check(R"( @@ -456,10 +453,6 @@ TEST_CASE_FIXTURE(Fixture, "functions_are_not_to_be_widened") TEST_CASE_FIXTURE(Fixture, "indexing_on_string_singletons") { - ScopedFastFlag sff[]{ - {"LuauDiscriminableUnions2", true}, - }; - CheckResult result = check(R"( local a: string = "hi" if a == "hi" then @@ -474,10 +467,6 @@ TEST_CASE_FIXTURE(Fixture, "indexing_on_string_singletons") TEST_CASE_FIXTURE(Fixture, "indexing_on_union_of_string_singletons") { - ScopedFastFlag sff[]{ - {"LuauDiscriminableUnions2", true}, - }; - CheckResult result = check(R"( local a: string = "hi" if a == "hi" or a == "bye" then @@ -492,10 +481,6 @@ TEST_CASE_FIXTURE(Fixture, "indexing_on_union_of_string_singletons") TEST_CASE_FIXTURE(Fixture, "taking_the_length_of_string_singleton") { - ScopedFastFlag sff[]{ - {"LuauDiscriminableUnions2", true}, - }; - CheckResult result = check(R"( local a: string = "hi" if a == "hi" then @@ -510,10 +495,6 @@ TEST_CASE_FIXTURE(Fixture, "taking_the_length_of_string_singleton") TEST_CASE_FIXTURE(Fixture, "taking_the_length_of_union_of_string_singleton") { - ScopedFastFlag sff[]{ - {"LuauDiscriminableUnions2", true}, - }; - CheckResult result = check(R"( local a: string = "hi" if a == "hi" or a == "bye" then diff --git a/tests/TypeInfer.tables.test.cpp b/tests/TypeInfer.tables.test.cpp index 5078b0bf..c924484a 100644 --- a/tests/TypeInfer.tables.test.cpp +++ b/tests/TypeInfer.tables.test.cpp @@ -2279,8 +2279,6 @@ local y = #x TEST_CASE_FIXTURE(BuiltinsFixture, "dont_hang_when_trying_to_look_up_in_cyclic_metatable_index") { - ScopedFastFlag sff{"LuauTerminateCyclicMetatableIndexLookup", true}; - // t :: t1 where t1 = {metatable {__index: t1, __tostring: (t1) -> string}} CheckResult result = check(R"( local mt = {} @@ -2313,8 +2311,6 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "give_up_after_one_metatable_index_look_up") TEST_CASE_FIXTURE(Fixture, "confusing_indexing") { - ScopedFastFlag sff{"LuauDoNotTryToReduce", true}; - CheckResult result = check(R"( type T = {} & {p: number | string} local function f(t: T) @@ -2971,8 +2967,6 @@ TEST_CASE_FIXTURE(Fixture, "inferred_return_type_of_free_table") TEST_CASE_FIXTURE(Fixture, "mixed_tables_with_implicit_numbered_keys") { - ScopedFastFlag sff{"LuauCheckImplicitNumbericKeys", true}; - CheckResult result = check(R"( local t: { [string]: number } = { 5, 6, 7 } )"); @@ -2984,4 +2978,32 @@ TEST_CASE_FIXTURE(Fixture, "mixed_tables_with_implicit_numbered_keys") CHECK_EQ("Type 'number' could not be converted into 'string'", toString(result.errors[2])); } +TEST_CASE_FIXTURE(Fixture, "expected_indexer_value_type_extra") +{ + ScopedFastFlag luauExpectedPropTypeFromIndexer{"LuauExpectedPropTypeFromIndexer", true}; + ScopedFastFlag luauSubtypingAddOptPropsToUnsealedTables{"LuauSubtypingAddOptPropsToUnsealedTables", true}; + + CheckResult result = check(R"( + type X = { { x: boolean?, y: boolean? } } + + local l1: {[string]: X} = { key = { { x = true }, { y = true } } } + local l2: {[any]: X} = { key = { { x = true }, { y = true } } } + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "expected_indexer_value_type_extra_2") +{ + ScopedFastFlag luauExpectedPropTypeFromIndexer{"LuauExpectedPropTypeFromIndexer", true}; + + CheckResult result = check(R"( + type X = {[any]: string | boolean} + + local x: X = { key = "str" } + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.test.cpp b/tests/TypeInfer.test.cpp index 48cd1c3d..1d144db7 100644 --- a/tests/TypeInfer.test.cpp +++ b/tests/TypeInfer.test.cpp @@ -15,7 +15,6 @@ LUAU_FASTFLAG(LuauLowerBoundsCalculation) LUAU_FASTFLAG(LuauFixLocationSpanTableIndexExpr) -LUAU_FASTFLAG(LuauEqConstraint) using namespace Luau; @@ -308,7 +307,6 @@ TEST_CASE_FIXTURE(Fixture, "check_type_infer_recursion_count") int limit = 600; #endif - ScopedFastFlag sff{"LuauTableUseCounterInstead", true}; ScopedFastInt sfi{"LuauCheckRecursionLimit", limit}; CheckResult result = check("function f() return " + rep("{a=", limit) + "'a'" + rep("}", limit) + " end"); @@ -1011,8 +1009,6 @@ TEST_CASE_FIXTURE(Fixture, "type_infer_recursion_limit_no_ice") TEST_CASE_FIXTURE(Fixture, "follow_on_new_types_in_substitution") { - ScopedFastFlag substituteFollowNewTypes{"LuauSubstituteFollowNewTypes", true}; - CheckResult result = check(R"( local obj = {} diff --git a/tests/TypeInfer.unionTypes.test.cpp b/tests/TypeInfer.unionTypes.test.cpp index 277f3887..d19d80cb 100644 --- a/tests/TypeInfer.unionTypes.test.cpp +++ b/tests/TypeInfer.unionTypes.test.cpp @@ -7,7 +7,6 @@ #include "doctest.h" LUAU_FASTFLAG(LuauLowerBoundsCalculation) -LUAU_FASTFLAG(LuauEqConstraint) using namespace Luau; From f5923aefeb66f8ea3e5193d328d8eb74400e0b3b Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Thu, 19 May 2022 17:02:24 -0700 Subject: [PATCH 253/399] Sync to upstream/release/527 (#491) --- Analysis/include/Luau/Clone.h | 2 +- Analysis/include/Luau/Error.h | 4 +- Analysis/include/Luau/LValue.h | 4 - Analysis/include/Luau/Module.h | 36 +- Analysis/include/Luau/Substitution.h | 3 +- Analysis/include/Luau/TxnLog.h | 11 - Analysis/include/Luau/TypeArena.h | 42 ++ Analysis/include/Luau/TypeInfer.h | 19 +- Analysis/include/Luau/Unifier.h | 4 +- Analysis/include/Luau/VisitTypeVar.h | 20 +- Analysis/src/BuiltinDefinitions.cpp | 53 +- Analysis/src/Clone.cpp | 44 +- Analysis/src/Error.cpp | 13 +- Analysis/src/IostreamHelpers.cpp | 2 +- Analysis/src/LValue.cpp | 21 - Analysis/src/Module.cpp | 123 ++-- Analysis/src/Normalize.cpp | 32 +- Analysis/src/Quantify.cpp | 39 +- Analysis/src/Substitution.cpp | 113 ++-- Analysis/src/ToString.cpp | 7 +- Analysis/src/TxnLog.cpp | 34 +- Analysis/src/TypeArena.cpp | 88 +++ Analysis/src/TypeInfer.cpp | 650 +++++---------------- Analysis/src/TypeUtils.cpp | 11 +- Analysis/src/TypeVar.cpp | 9 +- Analysis/src/Unifier.cpp | 84 +-- Compiler/include/Luau/BytecodeBuilder.h | 1 + Compiler/src/BytecodeBuilder.cpp | 10 + Compiler/src/Compiler.cpp | 289 ++++++--- Compiler/src/ConstantFolding.cpp | 4 +- Sources.cmake | 2 + VM/src/ltablib.cpp | 27 - VM/src/lvmexecute.cpp | 14 - tests/Autocomplete.test.cpp | 4 + tests/Compiler.test.cpp | 235 ++++++-- tests/Module.test.cpp | 8 - tests/NonstrictMode.test.cpp | 4 - tests/Normalize.test.cpp | 45 ++ tests/RuntimeLimits.test.cpp | 2 - tests/ToString.test.cpp | 14 +- tests/TypeInfer.builtins.test.cpp | 14 - tests/TypeInfer.functions.test.cpp | 2 - tests/TypeInfer.generics.test.cpp | 74 +++ tests/TypeInfer.intersectionTypes.test.cpp | 2 - tests/TypeInfer.loops.test.cpp | 2 - tests/TypeInfer.operators.test.cpp | 2 - tests/TypeInfer.provisional.test.cpp | 36 +- tests/TypeInfer.refinements.test.cpp | 143 +---- tests/TypeInfer.singletons.test.cpp | 25 +- tests/TypeInfer.tables.test.cpp | 34 +- tests/TypeInfer.test.cpp | 4 - tests/TypeInfer.unionTypes.test.cpp | 1 - 52 files changed, 1097 insertions(+), 1369 deletions(-) create mode 100644 Analysis/include/Luau/TypeArena.h create mode 100644 Analysis/src/TypeArena.cpp diff --git a/Analysis/include/Luau/Clone.h b/Analysis/include/Luau/Clone.h index 9b6ffa62..9fcbce04 100644 --- a/Analysis/include/Luau/Clone.h +++ b/Analysis/include/Luau/Clone.h @@ -1,6 +1,7 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #pragma once +#include "Luau/TypeArena.h" #include "Luau/TypeVar.h" #include @@ -18,7 +19,6 @@ struct CloneState SeenTypePacks seenTypePacks; int recursionCount = 0; - bool encounteredFreeType = false; // TODO: Remove with LuauLosslessClone. }; TypePackId clone(TypePackId tp, TypeArena& dest, CloneState& cloneState); diff --git a/Analysis/include/Luau/Error.h b/Analysis/include/Luau/Error.h index 70683141..b4530674 100644 --- a/Analysis/include/Luau/Error.h +++ b/Analysis/include/Luau/Error.h @@ -5,6 +5,7 @@ #include "Luau/Location.h" #include "Luau/TypeVar.h" #include "Luau/Variant.h" +#include "Luau/TypeArena.h" namespace Luau { @@ -108,9 +109,6 @@ struct FunctionDoesNotTakeSelf struct FunctionRequiresSelf { - // TODO: Delete with LuauAnyInIsOptionalIsOptional - int requiredExtraNils = 0; - bool operator==(const FunctionRequiresSelf& rhs) const; }; diff --git a/Analysis/include/Luau/LValue.h b/Analysis/include/Luau/LValue.h index afb71415..1a92d52d 100644 --- a/Analysis/include/Luau/LValue.h +++ b/Analysis/include/Luau/LValue.h @@ -34,10 +34,6 @@ const LValue* baseof(const LValue& lvalue); std::optional tryGetLValue(const class AstExpr& expr); -// Utility function: breaks down an LValue to get at the Symbol, and reverses the vector of keys. -// TODO: remove with FFlagLuauTypecheckOptPass -std::pair> getFullName(const LValue& lvalue); - // Utility function: breaks down an LValue to get at the Symbol Symbol getBaseSymbol(const LValue& lvalue); diff --git a/Analysis/include/Luau/Module.h b/Analysis/include/Luau/Module.h index 0dd44188..00e1e635 100644 --- a/Analysis/include/Luau/Module.h +++ b/Analysis/include/Luau/Module.h @@ -2,11 +2,10 @@ #pragma once #include "Luau/FileResolver.h" -#include "Luau/TypePack.h" -#include "Luau/TypedAllocator.h" #include "Luau/ParseOptions.h" #include "Luau/Error.h" #include "Luau/ParseResult.h" +#include "Luau/TypeArena.h" #include #include @@ -54,35 +53,6 @@ struct RequireCycle std::vector path; // one of the paths for a require() to go all the way back to the originating module }; -struct TypeArena -{ - TypedAllocator typeVars; - TypedAllocator typePacks; - - void clear(); - - template - TypeId addType(T tv) - { - if constexpr (std::is_same_v) - LUAU_ASSERT(tv.options.size() >= 2); - - return addTV(TypeVar(std::move(tv))); - } - - TypeId addTV(TypeVar&& tv); - - TypeId freshType(TypeLevel level); - - TypePackId addTypePack(std::initializer_list types); - TypePackId addTypePack(std::vector types); - TypePackId addTypePack(TypePack pack); - TypePackId addTypePack(TypePackVar pack); -}; - -void freeze(TypeArena& arena); -void unfreeze(TypeArena& arena); - struct Module { ~Module(); @@ -111,9 +81,7 @@ struct Module // Once a module has been typechecked, we clone its public interface into a separate arena. // This helps us to force TypeVar ownership into a DAG rather than a DCG. - // Returns true if there were any free types encountered in the public interface. This - // indicates a bug in the type checker that we want to surface. - bool clonePublicInterface(InternalErrorReporter& ice); + void clonePublicInterface(InternalErrorReporter& ice); }; } // namespace Luau diff --git a/Analysis/include/Luau/Substitution.h b/Analysis/include/Luau/Substitution.h index 6f5931e1..f3c3ae9a 100644 --- a/Analysis/include/Luau/Substitution.h +++ b/Analysis/include/Luau/Substitution.h @@ -1,8 +1,7 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #pragma once -#include "Luau/Module.h" -#include "Luau/ModuleResolver.h" +#include "Luau/TypeArena.h" #include "Luau/TypePack.h" #include "Luau/TypeVar.h" #include "Luau/DenseHash.h" diff --git a/Analysis/include/Luau/TxnLog.h b/Analysis/include/Luau/TxnLog.h index 995ed6c6..cd115e3b 100644 --- a/Analysis/include/Luau/TxnLog.h +++ b/Analysis/include/Luau/TxnLog.h @@ -7,8 +7,6 @@ #include "Luau/TypeVar.h" #include "Luau/TypePack.h" -LUAU_FASTFLAG(LuauTypecheckOptPass) - namespace Luau { @@ -93,15 +91,6 @@ struct TxnLog { } - TxnLog(TxnLog* parent, std::vector>* sharedSeen) - : typeVarChanges(nullptr) - , typePackChanges(nullptr) - , parent(parent) - , sharedSeen(sharedSeen) - { - LUAU_ASSERT(!FFlag::LuauTypecheckOptPass); - } - TxnLog(const TxnLog&) = delete; TxnLog& operator=(const TxnLog&) = delete; diff --git a/Analysis/include/Luau/TypeArena.h b/Analysis/include/Luau/TypeArena.h new file mode 100644 index 00000000..7c74158b --- /dev/null +++ b/Analysis/include/Luau/TypeArena.h @@ -0,0 +1,42 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +#pragma once + +#include "Luau/TypedAllocator.h" +#include "Luau/TypeVar.h" +#include "Luau/TypePack.h" + +#include + +namespace Luau +{ + +struct TypeArena +{ + TypedAllocator typeVars; + TypedAllocator typePacks; + + void clear(); + + template + TypeId addType(T tv) + { + if constexpr (std::is_same_v) + LUAU_ASSERT(tv.options.size() >= 2); + + return addTV(TypeVar(std::move(tv))); + } + + TypeId addTV(TypeVar&& tv); + + TypeId freshType(TypeLevel level); + + TypePackId addTypePack(std::initializer_list types); + TypePackId addTypePack(std::vector types); + TypePackId addTypePack(TypePack pack); + TypePackId addTypePack(TypePackVar pack); +}; + +void freeze(TypeArena& arena); +void unfreeze(TypeArena& arena); + +} diff --git a/Analysis/include/Luau/TypeInfer.h b/Analysis/include/Luau/TypeInfer.h index ac880135..fcaf5baa 100644 --- a/Analysis/include/Luau/TypeInfer.h +++ b/Analysis/include/Luau/TypeInfer.h @@ -187,7 +187,6 @@ struct TypeChecker ExprResult checkExpr(const ScopePtr& scope, const AstExprIndexExpr& expr); ExprResult checkExpr(const ScopePtr& scope, const AstExprFunction& expr, std::optional expectedType = std::nullopt); ExprResult checkExpr(const ScopePtr& scope, const AstExprTable& expr, std::optional expectedType = std::nullopt); - ExprResult checkExpr_(const ScopePtr& scope, const AstExprTable& expr, std::optional expectedType = std::nullopt); ExprResult checkExpr(const ScopePtr& scope, const AstExprUnary& expr); TypeId checkRelationalOperation( const ScopePtr& scope, const AstExprBinary& expr, TypeId lhsType, TypeId rhsType, const PredicateVec& predicates = {}); @@ -395,7 +394,7 @@ private: const AstArray& genericNames, const AstArray& genericPackNames, bool useCache = false); public: - ErrorVec resolve(const PredicateVec& predicates, const ScopePtr& scope, bool sense); + void resolve(const PredicateVec& predicates, const ScopePtr& scope, bool sense); private: void refineLValue(const LValue& lvalue, RefinementMap& refis, const ScopePtr& scope, TypeIdPredicate predicate); @@ -403,14 +402,14 @@ private: std::optional resolveLValue(const ScopePtr& scope, const LValue& lvalue); std::optional resolveLValue(const RefinementMap& refis, const ScopePtr& scope, const LValue& lvalue); - void resolve(const PredicateVec& predicates, ErrorVec& errVec, RefinementMap& refis, const ScopePtr& scope, bool sense, bool fromOr = false); - void resolve(const Predicate& predicate, ErrorVec& errVec, RefinementMap& refis, const ScopePtr& scope, bool sense, bool fromOr); - void resolve(const TruthyPredicate& truthyP, ErrorVec& errVec, RefinementMap& refis, const ScopePtr& scope, bool sense, bool fromOr); - void resolve(const AndPredicate& andP, ErrorVec& errVec, RefinementMap& refis, const ScopePtr& scope, bool sense); - void resolve(const OrPredicate& orP, ErrorVec& errVec, RefinementMap& refis, const ScopePtr& scope, bool sense); - void resolve(const IsAPredicate& isaP, ErrorVec& errVec, RefinementMap& refis, const ScopePtr& scope, bool sense); - void resolve(const TypeGuardPredicate& typeguardP, ErrorVec& errVec, RefinementMap& refis, const ScopePtr& scope, bool sense); - void resolve(const EqPredicate& eqP, ErrorVec& errVec, RefinementMap& refis, const ScopePtr& scope, bool sense); + void resolve(const PredicateVec& predicates, RefinementMap& refis, const ScopePtr& scope, bool sense, bool fromOr = false); + void resolve(const Predicate& predicate, RefinementMap& refis, const ScopePtr& scope, bool sense, bool fromOr); + void resolve(const TruthyPredicate& truthyP, RefinementMap& refis, const ScopePtr& scope, bool sense, bool fromOr); + void resolve(const AndPredicate& andP, RefinementMap& refis, const ScopePtr& scope, bool sense); + void resolve(const OrPredicate& orP, RefinementMap& refis, const ScopePtr& scope, bool sense); + void resolve(const IsAPredicate& isaP, RefinementMap& refis, const ScopePtr& scope, bool sense); + void resolve(const TypeGuardPredicate& typeguardP, RefinementMap& refis, const ScopePtr& scope, bool sense); + void resolve(const EqPredicate& eqP, RefinementMap& refis, const ScopePtr& scope, bool sense); bool isNonstrictMode() const; bool useConstrainedIntersections() const; diff --git a/Analysis/include/Luau/Unifier.h b/Analysis/include/Luau/Unifier.h index 418d4ca4..0e24c8b0 100644 --- a/Analysis/include/Luau/Unifier.h +++ b/Analysis/include/Luau/Unifier.h @@ -5,7 +5,7 @@ #include "Luau/Location.h" #include "Luau/TxnLog.h" #include "Luau/TypeInfer.h" -#include "Luau/Module.h" // FIXME: For TypeArena. It merits breaking out into its own header. +#include "Luau/TypeArena.h" #include "Luau/UnifierSharedState.h" #include @@ -55,8 +55,6 @@ struct Unifier UnifierSharedState& sharedState; Unifier(TypeArena* types, Mode mode, const Location& location, Variance variance, UnifierSharedState& sharedState, TxnLog* parentLog = nullptr); - Unifier(TypeArena* types, Mode mode, std::vector>* sharedSeen, const Location& location, Variance variance, - UnifierSharedState& sharedState, TxnLog* parentLog = nullptr); // Test whether the two type vars unify. Never commits the result. ErrorVec canUnify(TypeId subTy, TypeId superTy); diff --git a/Analysis/include/Luau/VisitTypeVar.h b/Analysis/include/Luau/VisitTypeVar.h index 67fce5ed..2e98f526 100644 --- a/Analysis/include/Luau/VisitTypeVar.h +++ b/Analysis/include/Luau/VisitTypeVar.h @@ -10,6 +10,7 @@ LUAU_FASTFLAG(LuauUseVisitRecursionLimit) LUAU_FASTINT(LuauVisitRecursionLimit) +LUAU_FASTFLAG(LuauNormalizeFlagIsConservative) namespace Luau { @@ -471,18 +472,21 @@ struct GenericTypeVarVisitor else if (auto pack = get(tp)) { - visit(tp, *pack); + bool res = visit(tp, *pack); + if (!FFlag::LuauNormalizeFlagIsConservative || res) + { + for (TypeId ty : pack->head) + traverse(ty); - for (TypeId ty : pack->head) - traverse(ty); - - if (pack->tail) - traverse(*pack->tail); + if (pack->tail) + traverse(*pack->tail); + } } else if (auto pack = get(tp)) { - visit(tp, *pack); - traverse(pack->ty); + bool res = visit(tp, *pack); + if (!FFlag::LuauNormalizeFlagIsConservative || res) + traverse(pack->ty); } else LUAU_ASSERT(!"GenericTypeVarVisitor::traverse(TypePackId) is not exhaustive!"); diff --git a/Analysis/src/BuiltinDefinitions.cpp b/Analysis/src/BuiltinDefinitions.cpp index 3895b01b..5ed6de67 100644 --- a/Analysis/src/BuiltinDefinitions.cpp +++ b/Analysis/src/BuiltinDefinitions.cpp @@ -8,7 +8,6 @@ #include -LUAU_FASTFLAG(LuauAssertStripsFalsyTypes) LUAU_FASTFLAGVARIABLE(LuauSetMetaTableArgsCheck, false) /** FIXME: Many of these type definitions are not quite completely accurate. @@ -408,41 +407,29 @@ static std::optional> magicFunctionAssert( { auto [paramPack, predicates] = exprResult; - if (FFlag::LuauAssertStripsFalsyTypes) + TypeArena& arena = typechecker.currentModule->internalTypes; + + auto [head, tail] = flatten(paramPack); + if (head.empty() && tail) { - TypeArena& arena = typechecker.currentModule->internalTypes; - - auto [head, tail] = flatten(paramPack); - if (head.empty() && tail) - { - std::optional fst = first(*tail); - if (!fst) - return ExprResult{paramPack}; - head.push_back(*fst); - } - - typechecker.reportErrors(typechecker.resolve(predicates, scope, true)); - - if (head.size() > 0) - { - std::optional newhead = typechecker.pickTypesFromSense(head[0], true); - if (!newhead) - head = {typechecker.nilType}; - else - head[0] = *newhead; - } - - return ExprResult{arena.addTypePack(TypePack{std::move(head), tail})}; - } - else - { - if (expr.args.size < 1) + std::optional fst = first(*tail); + if (!fst) return ExprResult{paramPack}; - - typechecker.reportErrors(typechecker.resolve(predicates, scope, true)); - - return ExprResult{paramPack}; + head.push_back(*fst); } + + typechecker.resolve(predicates, scope, true); + + if (head.size() > 0) + { + std::optional newhead = typechecker.pickTypesFromSense(head[0], true); + if (!newhead) + head = {typechecker.nilType}; + else + head[0] = *newhead; + } + + return ExprResult{arena.addTypePack(TypePack{std::move(head), tail})}; } static std::optional> magicFunctionPack( diff --git a/Analysis/src/Clone.cpp b/Analysis/src/Clone.cpp index 1aa556eb..a3611f53 100644 --- a/Analysis/src/Clone.cpp +++ b/Analysis/src/Clone.cpp @@ -1,7 +1,6 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #include "Luau/Clone.h" -#include "Luau/Module.h" #include "Luau/RecursionCounter.h" #include "Luau/TypePack.h" #include "Luau/Unifiable.h" @@ -9,8 +8,6 @@ LUAU_FASTFLAG(DebugLuauCopyBeforeNormalizing) LUAU_FASTINTVARIABLE(LuauTypeCloneRecursionLimit, 300) -LUAU_FASTFLAG(LuauTypecheckOptPass) -LUAU_FASTFLAGVARIABLE(LuauLosslessClone, false) LUAU_FASTFLAG(LuauNoMethodLocations) namespace Luau @@ -89,20 +86,8 @@ struct TypePackCloner void operator()(const Unifiable::Free& t) { - if (FFlag::LuauLosslessClone) - { - defaultClone(t); - } - else - { - cloneState.encounteredFreeType = true; - - TypePackId err = getSingletonTypes().errorRecoveryTypePack(getSingletonTypes().anyTypePack); - TypePackId cloned = dest.addTypePack(*err); - seenTypePacks[typePackId] = cloned; - } + defaultClone(t); } - void operator()(const Unifiable::Generic& t) { defaultClone(t); @@ -152,18 +137,7 @@ void TypeCloner::defaultClone(const T& t) void TypeCloner::operator()(const Unifiable::Free& t) { - if (FFlag::LuauLosslessClone) - { - defaultClone(t); - } - else - { - cloneState.encounteredFreeType = true; - - TypeId err = getSingletonTypes().errorRecoveryType(getSingletonTypes().anyType); - TypeId cloned = dest.addType(*err); - seenTypes[typeId] = cloned; - } + defaultClone(t); } void TypeCloner::operator()(const Unifiable::Generic& t) @@ -191,9 +165,6 @@ void TypeCloner::operator()(const PrimitiveTypeVar& t) void TypeCloner::operator()(const ConstrainedTypeVar& t) { - if (!FFlag::LuauLosslessClone) - cloneState.encounteredFreeType = true; - TypeId res = dest.addType(ConstrainedTypeVar{t.level}); ConstrainedTypeVar* ctv = getMutable(res); LUAU_ASSERT(ctv); @@ -230,9 +201,7 @@ void TypeCloner::operator()(const FunctionTypeVar& t) ftv->argTypes = clone(t.argTypes, dest, cloneState); ftv->argNames = t.argNames; ftv->retType = clone(t.retType, dest, cloneState); - - if (FFlag::LuauTypecheckOptPass) - ftv->hasNoGenerics = t.hasNoGenerics; + ftv->hasNoGenerics = t.hasNoGenerics; } void TypeCloner::operator()(const TableTypeVar& t) @@ -270,13 +239,6 @@ void TypeCloner::operator()(const TableTypeVar& t) for (TypePackId& arg : ttv->instantiatedTypePackParams) arg = clone(arg, dest, cloneState); - if (!FFlag::LuauLosslessClone && ttv->state == TableState::Free) - { - cloneState.encounteredFreeType = true; - - ttv->state = TableState::Sealed; - } - ttv->definitionModuleName = t.definitionModuleName; if (!FFlag::LuauNoMethodLocations) ttv->methodDefinitionLocations = t.methodDefinitionLocations; diff --git a/Analysis/src/Error.cpp b/Analysis/src/Error.cpp index 24ed4ac1..f443a3cc 100644 --- a/Analysis/src/Error.cpp +++ b/Analysis/src/Error.cpp @@ -2,7 +2,6 @@ #include "Luau/Error.h" #include "Luau/Clone.h" -#include "Luau/Module.h" #include "Luau/StringUtils.h" #include "Luau/ToString.h" @@ -178,15 +177,7 @@ struct ErrorConverter std::string operator()(const Luau::FunctionRequiresSelf& e) const { - if (e.requiredExtraNils) - { - const char* plural = e.requiredExtraNils == 1 ? "" : "s"; - return format("This function was declared to accept self, but you did not pass enough arguments. Use a colon instead of a dot or " - "pass %i extra nil%s to suppress this warning", - e.requiredExtraNils, plural); - } - else - return "This function must be called with self. Did you mean to use a colon instead of a dot?"; + return "This function must be called with self. Did you mean to use a colon instead of a dot?"; } std::string operator()(const Luau::OccursCheckFailed&) const @@ -539,7 +530,7 @@ bool FunctionDoesNotTakeSelf::operator==(const FunctionDoesNotTakeSelf&) const bool FunctionRequiresSelf::operator==(const FunctionRequiresSelf& e) const { - return requiredExtraNils == e.requiredExtraNils; + return true; } bool OccursCheckFailed::operator==(const OccursCheckFailed&) const diff --git a/Analysis/src/IostreamHelpers.cpp b/Analysis/src/IostreamHelpers.cpp index 0eaa485e..048167ae 100644 --- a/Analysis/src/IostreamHelpers.cpp +++ b/Analysis/src/IostreamHelpers.cpp @@ -48,7 +48,7 @@ static void errorToString(std::ostream& stream, const T& err) else if constexpr (std::is_same_v) stream << "FunctionDoesNotTakeSelf { }"; else if constexpr (std::is_same_v) - stream << "FunctionRequiresSelf { extraNils " << err.requiredExtraNils << " }"; + stream << "FunctionRequiresSelf { }"; else if constexpr (std::is_same_v) stream << "OccursCheckFailed { }"; else if constexpr (std::is_same_v) diff --git a/Analysis/src/LValue.cpp b/Analysis/src/LValue.cpp index 72555ab4..38dfe1ae 100644 --- a/Analysis/src/LValue.cpp +++ b/Analysis/src/LValue.cpp @@ -5,8 +5,6 @@ #include -LUAU_FASTFLAG(LuauTypecheckOptPass) - namespace Luau { @@ -79,27 +77,8 @@ std::optional tryGetLValue(const AstExpr& node) return std::nullopt; } -std::pair> getFullName(const LValue& lvalue) -{ - LUAU_ASSERT(!FFlag::LuauTypecheckOptPass); - - const LValue* current = &lvalue; - std::vector keys; - while (auto field = get(*current)) - { - keys.push_back(field->key); - current = baseof(*current); - } - - const Symbol* symbol = get(*current); - LUAU_ASSERT(symbol); - return {*symbol, std::vector(keys.rbegin(), keys.rend())}; -} - Symbol getBaseSymbol(const LValue& lvalue) { - LUAU_ASSERT(FFlag::LuauTypecheckOptPass); - const LValue* current = &lvalue; while (auto field = get(*current)) current = baseof(*current); diff --git a/Analysis/src/Module.cpp b/Analysis/src/Module.cpp index bafd4371..074a41e6 100644 --- a/Analysis/src/Module.cpp +++ b/Analysis/src/Module.cpp @@ -13,9 +13,8 @@ #include -LUAU_FASTFLAGVARIABLE(DebugLuauFreezeArena, false) -LUAU_FASTFLAG(LuauLowerBoundsCalculation) -LUAU_FASTFLAG(LuauLosslessClone) +LUAU_FASTFLAG(LuauLowerBoundsCalculation); +LUAU_FASTFLAG(LuauNormalizeFlagIsConservative); namespace Luau { @@ -55,89 +54,25 @@ bool isWithinComment(const SourceModule& sourceModule, Position pos) return contains(pos, *iter); } -void TypeArena::clear() +struct ForceNormal : TypeVarOnceVisitor { - typeVars.clear(); - typePacks.clear(); -} + bool visit(TypeId ty) override + { + asMutable(ty)->normal = true; + return true; + } -TypeId TypeArena::addTV(TypeVar&& tv) -{ - TypeId allocated = typeVars.allocate(std::move(tv)); + bool visit(TypeId ty, const FreeTypeVar& ftv) override + { + visit(ty); + return true; + } - asMutable(allocated)->owningArena = this; - - return allocated; -} - -TypeId TypeArena::freshType(TypeLevel level) -{ - TypeId allocated = typeVars.allocate(FreeTypeVar{level}); - - asMutable(allocated)->owningArena = this; - - return allocated; -} - -TypePackId TypeArena::addTypePack(std::initializer_list types) -{ - TypePackId allocated = typePacks.allocate(TypePack{std::move(types)}); - - asMutable(allocated)->owningArena = this; - - return allocated; -} - -TypePackId TypeArena::addTypePack(std::vector types) -{ - TypePackId allocated = typePacks.allocate(TypePack{std::move(types)}); - - asMutable(allocated)->owningArena = this; - - return allocated; -} - -TypePackId TypeArena::addTypePack(TypePack tp) -{ - TypePackId allocated = typePacks.allocate(std::move(tp)); - - asMutable(allocated)->owningArena = this; - - return allocated; -} - -TypePackId TypeArena::addTypePack(TypePackVar tp) -{ - TypePackId allocated = typePacks.allocate(std::move(tp)); - - asMutable(allocated)->owningArena = this; - - return allocated; -} - -ScopePtr Module::getModuleScope() const -{ - LUAU_ASSERT(!scopes.empty()); - return scopes.front().second; -} - -void freeze(TypeArena& arena) -{ - if (!FFlag::DebugLuauFreezeArena) - return; - - arena.typeVars.freeze(); - arena.typePacks.freeze(); -} - -void unfreeze(TypeArena& arena) -{ - if (!FFlag::DebugLuauFreezeArena) - return; - - arena.typeVars.unfreeze(); - arena.typePacks.unfreeze(); -} + bool visit(TypePackId tp, const FreeTypePack& ftp) override + { + return true; + } +}; Module::~Module() { @@ -145,7 +80,7 @@ Module::~Module() unfreeze(internalTypes); } -bool Module::clonePublicInterface(InternalErrorReporter& ice) +void Module::clonePublicInterface(InternalErrorReporter& ice) { LUAU_ASSERT(interfaceTypes.typeVars.empty()); LUAU_ASSERT(interfaceTypes.typePacks.empty()); @@ -165,11 +100,22 @@ bool Module::clonePublicInterface(InternalErrorReporter& ice) normalize(*moduleScope->varargPack, interfaceTypes, ice); } + ForceNormal forceNormal; + for (auto& [name, tf] : moduleScope->exportedTypeBindings) { tf = clone(tf, interfaceTypes, cloneState); if (FFlag::LuauLowerBoundsCalculation) + { normalize(tf.type, interfaceTypes, ice); + + if (FFlag::LuauNormalizeFlagIsConservative) + { + // We're about to freeze the memory. We know that the flag is conservative by design. Cyclic tables + // won't be marked normal. If the types aren't normal by now, they never will be. + forceNormal.traverse(tf.type); + } + } } for (TypeId ty : moduleScope->returnType) @@ -191,11 +137,12 @@ bool Module::clonePublicInterface(InternalErrorReporter& ice) freeze(internalTypes); freeze(interfaceTypes); +} - if (FFlag::LuauLosslessClone) - return false; // TODO: make function return void. - else - return cloneState.encounteredFreeType; +ScopePtr Module::getModuleScope() const +{ + LUAU_ASSERT(!scopes.empty()); + return scopes.front().second; } } // namespace Luau diff --git a/Analysis/src/Normalize.cpp b/Analysis/src/Normalize.cpp index ef5377a1..30fd4af2 100644 --- a/Analysis/src/Normalize.cpp +++ b/Analysis/src/Normalize.cpp @@ -14,6 +14,7 @@ LUAU_FASTFLAGVARIABLE(DebugLuauCopyBeforeNormalizing, false) // This could theoretically be 2000 on amd64, but x86 requires this. LUAU_FASTINTVARIABLE(LuauNormalizeIterationLimit, 1200); LUAU_FASTFLAGVARIABLE(LuauNormalizeCombineTableFix, false); +LUAU_FASTFLAGVARIABLE(LuauNormalizeFlagIsConservative, false); namespace Luau { @@ -260,8 +261,13 @@ static bool areNormal_(const T& t, const std::unordered_set& seen, Intern if (count >= FInt::LuauNormalizeIterationLimit) ice.ice("Luau::areNormal hit iteration limit"); - // The follow is here because a bound type may not be normal, but the bound type is normal. - return ty->normal || follow(ty)->normal || seen.find(asMutable(ty)) != seen.end(); + if (FFlag::LuauNormalizeFlagIsConservative) + return ty->normal; + else + { + // The follow is here because a bound type may not be normal, but the bound type is normal. + return ty->normal || follow(ty)->normal || seen.find(asMutable(ty)) != seen.end(); + } }; return std::all_of(begin(t), end(t), isNormal); @@ -1003,8 +1009,15 @@ std::pair normalize(TypeId ty, TypeArena& arena, InternalErrorRepo (void)clone(ty, arena, state); Normalize n{arena, ice}; - std::unordered_set seen; - DEPRECATED_visitTypeVar(ty, n, seen); + if (FFlag::LuauNormalizeFlagIsConservative) + { + DEPRECATED_visitTypeVar(ty, n); + } + else + { + std::unordered_set seen; + DEPRECATED_visitTypeVar(ty, n, seen); + } return {ty, !n.limitExceeded}; } @@ -1028,8 +1041,15 @@ std::pair normalize(TypePackId tp, TypeArena& arena, InternalE (void)clone(tp, arena, state); Normalize n{arena, ice}; - std::unordered_set seen; - DEPRECATED_visitTypeVar(tp, n, seen); + if (FFlag::LuauNormalizeFlagIsConservative) + { + DEPRECATED_visitTypeVar(tp, n); + } + else + { + std::unordered_set seen; + DEPRECATED_visitTypeVar(tp, n, seen); + } return {tp, !n.limitExceeded}; } diff --git a/Analysis/src/Quantify.cpp b/Analysis/src/Quantify.cpp index 4f3e4469..018d5632 100644 --- a/Analysis/src/Quantify.cpp +++ b/Analysis/src/Quantify.cpp @@ -4,7 +4,7 @@ #include "Luau/VisitTypeVar.h" -LUAU_FASTFLAG(LuauTypecheckOptPass) +LUAU_FASTFLAG(LuauAlwaysQuantify) namespace Luau { @@ -59,8 +59,7 @@ struct Quantifier final : TypeVarOnceVisitor bool visit(TypeId ty, const FreeTypeVar& ftv) override { - if (FFlag::LuauTypecheckOptPass) - seenMutableType = true; + seenMutableType = true; if (!level.subsumes(ftv.level)) return false; @@ -76,20 +75,17 @@ struct Quantifier final : TypeVarOnceVisitor LUAU_ASSERT(getMutable(ty)); TableTypeVar& ttv = *getMutable(ty); - if (FFlag::LuauTypecheckOptPass) - { - if (ttv.state == TableState::Generic) - seenGenericType = true; + if (ttv.state == TableState::Generic) + seenGenericType = true; - if (ttv.state == TableState::Free) - seenMutableType = true; - } + if (ttv.state == TableState::Free) + seenMutableType = true; if (ttv.state == TableState::Sealed || ttv.state == TableState::Generic) return false; if (!level.subsumes(ttv.level)) { - if (FFlag::LuauTypecheckOptPass && ttv.state == TableState::Unsealed) + if (ttv.state == TableState::Unsealed) seenMutableType = true; return false; } @@ -97,9 +93,7 @@ struct Quantifier final : TypeVarOnceVisitor if (ttv.state == TableState::Free) { ttv.state = TableState::Generic; - - if (FFlag::LuauTypecheckOptPass) - seenGenericType = true; + seenGenericType = true; } else if (ttv.state == TableState::Unsealed) ttv.state = TableState::Sealed; @@ -111,8 +105,7 @@ struct Quantifier final : TypeVarOnceVisitor bool visit(TypePackId tp, const FreeTypePack& ftp) override { - if (FFlag::LuauTypecheckOptPass) - seenMutableType = true; + seenMutableType = true; if (!level.subsumes(ftp.level)) return false; @@ -131,10 +124,18 @@ void quantify(TypeId ty, TypeLevel level) FunctionTypeVar* ftv = getMutable(ty); LUAU_ASSERT(ftv); - ftv->generics = q.generics; - ftv->genericPacks = q.genericPacks; + if (FFlag::LuauAlwaysQuantify) + { + ftv->generics.insert(ftv->generics.end(), q.generics.begin(), q.generics.end()); + ftv->genericPacks.insert(ftv->genericPacks.end(), q.genericPacks.begin(), q.genericPacks.end()); + } + else + { + ftv->generics = q.generics; + ftv->genericPacks = q.genericPacks; + } - if (FFlag::LuauTypecheckOptPass && ftv->generics.empty() && ftv->genericPacks.empty() && !q.seenMutableType && !q.seenGenericType) + if (ftv->generics.empty() && ftv->genericPacks.empty() && !q.seenMutableType && !q.seenGenericType) ftv->hasNoGenerics = true; } diff --git a/Analysis/src/Substitution.cpp b/Analysis/src/Substitution.cpp index c5c7977a..e40bedb0 100644 --- a/Analysis/src/Substitution.cpp +++ b/Analysis/src/Substitution.cpp @@ -9,9 +9,6 @@ LUAU_FASTFLAG(LuauLowerBoundsCalculation) LUAU_FASTINTVARIABLE(LuauTarjanChildLimit, 10000) -LUAU_FASTFLAG(LuauTypecheckOptPass) -LUAU_FASTFLAGVARIABLE(LuauSubstituteFollowNewTypes, false) -LUAU_FASTFLAGVARIABLE(LuauSubstituteFollowPossibleMutations, false) LUAU_FASTFLAG(LuauNoMethodLocations) namespace Luau @@ -19,26 +16,20 @@ namespace Luau void Tarjan::visitChildren(TypeId ty, int index) { - if (FFlag::LuauTypecheckOptPass) - LUAU_ASSERT(ty == log->follow(ty)); - else - ty = log->follow(ty); + LUAU_ASSERT(ty == log->follow(ty)); if (ignoreChildren(ty)) return; - if (FFlag::LuauTypecheckOptPass) - { - if (auto pty = log->pending(ty)) - ty = &pty->pending; - } + if (auto pty = log->pending(ty)) + ty = &pty->pending; - if (const FunctionTypeVar* ftv = FFlag::LuauTypecheckOptPass ? get(ty) : log->getMutable(ty)) + if (const FunctionTypeVar* ftv = get(ty)) { visitChild(ftv->argTypes); visitChild(ftv->retType); } - else if (const TableTypeVar* ttv = FFlag::LuauTypecheckOptPass ? get(ty) : log->getMutable(ty)) + else if (const TableTypeVar* ttv = get(ty)) { LUAU_ASSERT(!ttv->boundTo); for (const auto& [name, prop] : ttv->props) @@ -55,17 +46,17 @@ void Tarjan::visitChildren(TypeId ty, int index) for (TypePackId itp : ttv->instantiatedTypePackParams) visitChild(itp); } - else if (const MetatableTypeVar* mtv = FFlag::LuauTypecheckOptPass ? get(ty) : log->getMutable(ty)) + else if (const MetatableTypeVar* mtv = get(ty)) { visitChild(mtv->table); visitChild(mtv->metatable); } - else if (const UnionTypeVar* utv = FFlag::LuauTypecheckOptPass ? get(ty) : log->getMutable(ty)) + else if (const UnionTypeVar* utv = get(ty)) { for (TypeId opt : utv->options) visitChild(opt); } - else if (const IntersectionTypeVar* itv = FFlag::LuauTypecheckOptPass ? get(ty) : log->getMutable(ty)) + else if (const IntersectionTypeVar* itv = get(ty)) { for (TypeId part : itv->parts) visitChild(part); @@ -79,28 +70,22 @@ void Tarjan::visitChildren(TypeId ty, int index) void Tarjan::visitChildren(TypePackId tp, int index) { - if (FFlag::LuauTypecheckOptPass) - LUAU_ASSERT(tp == log->follow(tp)); - else - tp = log->follow(tp); + LUAU_ASSERT(tp == log->follow(tp)); if (ignoreChildren(tp)) return; - if (FFlag::LuauTypecheckOptPass) - { - if (auto ptp = log->pending(tp)) - tp = &ptp->pending; - } + if (auto ptp = log->pending(tp)) + tp = &ptp->pending; - if (const TypePack* tpp = FFlag::LuauTypecheckOptPass ? get(tp) : log->getMutable(tp)) + if (const TypePack* tpp = get(tp)) { for (TypeId tv : tpp->head) visitChild(tv); if (tpp->tail) visitChild(*tpp->tail); } - else if (const VariadicTypePack* vtp = FFlag::LuauTypecheckOptPass ? get(tp) : log->getMutable(tp)) + else if (const VariadicTypePack* vtp = get(tp)) { visitChild(vtp->ty); } @@ -108,10 +93,7 @@ void Tarjan::visitChildren(TypePackId tp, int index) std::pair Tarjan::indexify(TypeId ty) { - if (FFlag::LuauTypecheckOptPass && !FFlag::LuauSubstituteFollowPossibleMutations) - LUAU_ASSERT(ty == log->follow(ty)); - else - ty = log->follow(ty); + ty = log->follow(ty); bool fresh = !typeToIndex.contains(ty); int& index = typeToIndex[ty]; @@ -129,10 +111,7 @@ std::pair Tarjan::indexify(TypeId ty) std::pair Tarjan::indexify(TypePackId tp) { - if (FFlag::LuauTypecheckOptPass && !FFlag::LuauSubstituteFollowPossibleMutations) - LUAU_ASSERT(tp == log->follow(tp)); - else - tp = log->follow(tp); + tp = log->follow(tp); bool fresh = !packToIndex.contains(tp); int& index = packToIndex[tp]; @@ -150,8 +129,7 @@ std::pair Tarjan::indexify(TypePackId tp) void Tarjan::visitChild(TypeId ty) { - if (!FFlag::LuauSubstituteFollowPossibleMutations) - ty = log->follow(ty); + ty = log->follow(ty); edgesTy.push_back(ty); edgesTp.push_back(nullptr); @@ -159,8 +137,7 @@ void Tarjan::visitChild(TypeId ty) void Tarjan::visitChild(TypePackId tp) { - if (!FFlag::LuauSubstituteFollowPossibleMutations) - tp = log->follow(tp); + tp = log->follow(tp); edgesTy.push_back(nullptr); edgesTp.push_back(tp); @@ -389,13 +366,10 @@ TypeId Substitution::clone(TypeId ty) TypeId result = ty; - if (FFlag::LuauTypecheckOptPass) - { - if (auto pty = log->pending(ty)) - ty = &pty->pending; - } + if (auto pty = log->pending(ty)) + ty = &pty->pending; - if (const FunctionTypeVar* ftv = FFlag::LuauTypecheckOptPass ? get(ty) : log->getMutable(ty)) + if (const FunctionTypeVar* ftv = get(ty)) { FunctionTypeVar clone = FunctionTypeVar{ftv->level, ftv->argTypes, ftv->retType, ftv->definition, ftv->hasSelf}; clone.generics = ftv->generics; @@ -405,7 +379,7 @@ TypeId Substitution::clone(TypeId ty) clone.argNames = ftv->argNames; result = addType(std::move(clone)); } - else if (const TableTypeVar* ttv = FFlag::LuauTypecheckOptPass ? get(ty) : log->getMutable(ty)) + else if (const TableTypeVar* ttv = get(ty)) { LUAU_ASSERT(!ttv->boundTo); TableTypeVar clone = TableTypeVar{ttv->props, ttv->indexer, ttv->level, ttv->state}; @@ -419,19 +393,19 @@ TypeId Substitution::clone(TypeId ty) clone.tags = ttv->tags; result = addType(std::move(clone)); } - else if (const MetatableTypeVar* mtv = FFlag::LuauTypecheckOptPass ? get(ty) : log->getMutable(ty)) + else if (const MetatableTypeVar* mtv = get(ty)) { MetatableTypeVar clone = MetatableTypeVar{mtv->table, mtv->metatable}; clone.syntheticName = mtv->syntheticName; result = addType(std::move(clone)); } - else if (const UnionTypeVar* utv = FFlag::LuauTypecheckOptPass ? get(ty) : log->getMutable(ty)) + else if (const UnionTypeVar* utv = get(ty)) { UnionTypeVar clone; clone.options = utv->options; result = addType(std::move(clone)); } - else if (const IntersectionTypeVar* itv = FFlag::LuauTypecheckOptPass ? get(ty) : log->getMutable(ty)) + else if (const IntersectionTypeVar* itv = get(ty)) { IntersectionTypeVar clone; clone.parts = itv->parts; @@ -451,20 +425,17 @@ TypePackId Substitution::clone(TypePackId tp) { tp = log->follow(tp); - if (FFlag::LuauTypecheckOptPass) - { - if (auto ptp = log->pending(tp)) - tp = &ptp->pending; - } + if (auto ptp = log->pending(tp)) + tp = &ptp->pending; - if (const TypePack* tpp = FFlag::LuauTypecheckOptPass ? get(tp) : log->getMutable(tp)) + if (const TypePack* tpp = get(tp)) { TypePack clone; clone.head = tpp->head; clone.tail = tpp->tail; return addTypePack(std::move(clone)); } - else if (const VariadicTypePack* vtp = FFlag::LuauTypecheckOptPass ? get(tp) : log->getMutable(tp)) + else if (const VariadicTypePack* vtp = get(tp)) { VariadicTypePack clone; clone.ty = vtp->ty; @@ -476,28 +447,22 @@ TypePackId Substitution::clone(TypePackId tp) void Substitution::foundDirty(TypeId ty) { - if (FFlag::LuauTypecheckOptPass && !FFlag::LuauSubstituteFollowPossibleMutations) - LUAU_ASSERT(ty == log->follow(ty)); - else - ty = log->follow(ty); + ty = log->follow(ty); if (isDirty(ty)) - newTypes[ty] = FFlag::LuauSubstituteFollowNewTypes ? follow(clean(ty)) : clean(ty); + newTypes[ty] = follow(clean(ty)); else - newTypes[ty] = FFlag::LuauSubstituteFollowNewTypes ? follow(clone(ty)) : clone(ty); + newTypes[ty] = follow(clone(ty)); } void Substitution::foundDirty(TypePackId tp) { - if (FFlag::LuauTypecheckOptPass && !FFlag::LuauSubstituteFollowPossibleMutations) - LUAU_ASSERT(tp == log->follow(tp)); - else - tp = log->follow(tp); + tp = log->follow(tp); if (isDirty(tp)) - newPacks[tp] = FFlag::LuauSubstituteFollowNewTypes ? follow(clean(tp)) : clean(tp); + newPacks[tp] = follow(clean(tp)); else - newPacks[tp] = FFlag::LuauSubstituteFollowNewTypes ? follow(clone(tp)) : clone(tp); + newPacks[tp] = follow(clone(tp)); } TypeId Substitution::replace(TypeId ty) @@ -525,10 +490,7 @@ void Substitution::replaceChildren(TypeId ty) if (BoundTypeVar* btv = log->getMutable(ty); FFlag::LuauLowerBoundsCalculation && btv) btv->boundTo = replace(btv->boundTo); - if (FFlag::LuauTypecheckOptPass) - LUAU_ASSERT(ty == log->follow(ty)); - else - ty = log->follow(ty); + LUAU_ASSERT(ty == log->follow(ty)); if (ignoreChildren(ty)) return; @@ -579,10 +541,7 @@ void Substitution::replaceChildren(TypeId ty) void Substitution::replaceChildren(TypePackId tp) { - if (FFlag::LuauTypecheckOptPass) - LUAU_ASSERT(tp == log->follow(tp)); - else - tp = log->follow(tp); + LUAU_ASSERT(tp == log->follow(tp)); if (ignoreChildren(tp)) return; diff --git a/Analysis/src/ToString.cpp b/Analysis/src/ToString.cpp index 380ac456..f90f7019 100644 --- a/Analysis/src/ToString.cpp +++ b/Analysis/src/ToString.cpp @@ -219,6 +219,8 @@ struct StringifierState return generateName(s); } + int previousNameIndex = 0; + std::string getName(TypePackId ty) { const size_t s = result.nameMap.typePacks.size(); @@ -228,9 +230,10 @@ struct StringifierState for (int count = 0; count < 256; ++count) { - std::string candidate = generateName(usedNames.size() + count); + std::string candidate = generateName(previousNameIndex + count); if (!usedNames.count(candidate)) { + previousNameIndex += count; usedNames.insert(candidate); n = candidate; return candidate; @@ -399,6 +402,7 @@ struct TypeVarStringifier { if (gtv.explicitName) { + state.usedNames.insert(gtv.name); state.result.nameMap.typeVars[ty] = gtv.name; state.emit(gtv.name); } @@ -943,6 +947,7 @@ struct TypePackStringifier state.emit("gen-"); if (pack.explicitName) { + state.usedNames.insert(pack.name); state.result.nameMap.typePacks[tp] = pack.name; state.emit(pack.name); } diff --git a/Analysis/src/TxnLog.cpp b/Analysis/src/TxnLog.cpp index 1fb5a61a..e45c0cbd 100644 --- a/Analysis/src/TxnLog.cpp +++ b/Analysis/src/TxnLog.cpp @@ -7,8 +7,6 @@ #include #include -LUAU_FASTFLAGVARIABLE(LuauJustOneCallFrameForHaveSeen, false) - namespace Luau { @@ -150,37 +148,13 @@ void TxnLog::popSeen(TypePackId lhs, TypePackId rhs) bool TxnLog::haveSeen(TypeOrPackId lhs, TypeOrPackId rhs) const { - if (FFlag::LuauJustOneCallFrameForHaveSeen && !FFlag::LuauTypecheckOptPass) + const std::pair sortedPair = (lhs > rhs) ? std::make_pair(lhs, rhs) : std::make_pair(rhs, lhs); + if (sharedSeen->end() != std::find(sharedSeen->begin(), sharedSeen->end(), sortedPair)) { - // This function will technically work if `this` is nullptr, but this - // indicates a bug, so we explicitly assert. - LUAU_ASSERT(static_cast(this) != nullptr); - - const std::pair sortedPair = (lhs > rhs) ? std::make_pair(lhs, rhs) : std::make_pair(rhs, lhs); - - for (const TxnLog* current = this; current; current = current->parent) - { - if (current->sharedSeen->end() != std::find(current->sharedSeen->begin(), current->sharedSeen->end(), sortedPair)) - return true; - } - - return false; + return true; } - else - { - const std::pair sortedPair = (lhs > rhs) ? std::make_pair(lhs, rhs) : std::make_pair(rhs, lhs); - if (sharedSeen->end() != std::find(sharedSeen->begin(), sharedSeen->end(), sortedPair)) - { - return true; - } - if (!FFlag::LuauTypecheckOptPass && parent) - { - return parent->haveSeen(lhs, rhs); - } - - return false; - } + return false; } void TxnLog::pushSeen(TypeOrPackId lhs, TypeOrPackId rhs) diff --git a/Analysis/src/TypeArena.cpp b/Analysis/src/TypeArena.cpp new file mode 100644 index 00000000..673b002d --- /dev/null +++ b/Analysis/src/TypeArena.cpp @@ -0,0 +1,88 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details + +#include "Luau/TypeArena.h" + +LUAU_FASTFLAGVARIABLE(DebugLuauFreezeArena, false); + +namespace Luau +{ + +void TypeArena::clear() +{ + typeVars.clear(); + typePacks.clear(); +} + +TypeId TypeArena::addTV(TypeVar&& tv) +{ + TypeId allocated = typeVars.allocate(std::move(tv)); + + asMutable(allocated)->owningArena = this; + + return allocated; +} + +TypeId TypeArena::freshType(TypeLevel level) +{ + TypeId allocated = typeVars.allocate(FreeTypeVar{level}); + + asMutable(allocated)->owningArena = this; + + return allocated; +} + +TypePackId TypeArena::addTypePack(std::initializer_list types) +{ + TypePackId allocated = typePacks.allocate(TypePack{std::move(types)}); + + asMutable(allocated)->owningArena = this; + + return allocated; +} + +TypePackId TypeArena::addTypePack(std::vector types) +{ + TypePackId allocated = typePacks.allocate(TypePack{std::move(types)}); + + asMutable(allocated)->owningArena = this; + + return allocated; +} + +TypePackId TypeArena::addTypePack(TypePack tp) +{ + TypePackId allocated = typePacks.allocate(std::move(tp)); + + asMutable(allocated)->owningArena = this; + + return allocated; +} + +TypePackId TypeArena::addTypePack(TypePackVar tp) +{ + TypePackId allocated = typePacks.allocate(std::move(tp)); + + asMutable(allocated)->owningArena = this; + + return allocated; +} + +void freeze(TypeArena& arena) +{ + if (!FFlag::DebugLuauFreezeArena) + return; + + arena.typeVars.freeze(); + arena.typePacks.freeze(); +} + +void unfreeze(TypeArena& arena) +{ + if (!FFlag::DebugLuauFreezeArena) + return; + + arena.typeVars.unfreeze(); + arena.typePacks.unfreeze(); +} + +} diff --git a/Analysis/src/TypeInfer.cpp b/Analysis/src/TypeInfer.cpp index a13abd53..208b3f2f 100644 --- a/Analysis/src/TypeInfer.cpp +++ b/Analysis/src/TypeInfer.cpp @@ -32,32 +32,25 @@ LUAU_FASTFLAG(LuauKnowsTheDataModel3) LUAU_FASTFLAG(LuauSeparateTypechecks) LUAU_FASTFLAG(LuauAutocompleteDynamicLimits) LUAU_FASTFLAGVARIABLE(LuauDoNotRelyOnNextBinding, false) -LUAU_FASTFLAGVARIABLE(LuauEqConstraint, false) +LUAU_FASTFLAGVARIABLE(LuauExpectedPropTypeFromIndexer, false) LUAU_FASTFLAGVARIABLE(LuauWeakEqConstraint, false) // Eventually removed as false. LUAU_FASTFLAGVARIABLE(LuauLowerBoundsCalculation, false) LUAU_FASTFLAGVARIABLE(DebugLuauFreezeDuringUnification, false) -LUAU_FASTFLAGVARIABLE(LuauInstantiateFollows, false) LUAU_FASTFLAGVARIABLE(LuauSelfCallAutocompleteFix, false) -LUAU_FASTFLAGVARIABLE(LuauDiscriminableUnions2, false) LUAU_FASTFLAGVARIABLE(LuauReduceUnionRecursion, false) LUAU_FASTFLAGVARIABLE(LuauOnlyMutateInstantiatedTables, false) -LUAU_FASTFLAGVARIABLE(LuauTypecheckOptPass, false) LUAU_FASTFLAGVARIABLE(LuauUnsealedTableLiteral, false) LUAU_FASTFLAGVARIABLE(LuauTwoPassAliasDefinitionFix, false) -LUAU_FASTFLAGVARIABLE(LuauAssertStripsFalsyTypes, false) LUAU_FASTFLAGVARIABLE(LuauReturnAnyInsteadOfICE, false) // Eventually removed as false. +LUAU_FASTFLAG(LuauNormalizeFlagIsConservative) LUAU_FASTFLAG(LuauWidenIfSupertypeIsFree2) -LUAU_FASTFLAGVARIABLE(LuauDoNotTryToReduce, false) -LUAU_FASTFLAGVARIABLE(LuauDoNotAccidentallyDependOnPointerOrdering, false) -LUAU_FASTFLAGVARIABLE(LuauCheckImplicitNumbericKeys, false) -LUAU_FASTFLAG(LuauAnyInIsOptionalIsOptional) -LUAU_FASTFLAGVARIABLE(LuauTableUseCounterInstead, false) LUAU_FASTFLAGVARIABLE(LuauReturnTypeInferenceInNonstrict, false) LUAU_FASTFLAGVARIABLE(LuauRecursionLimitException, false); -LUAU_FASTFLAG(LuauLosslessClone) +LUAU_FASTFLAGVARIABLE(LuauApplyTypeFunctionFix, false); LUAU_FASTFLAGVARIABLE(LuauTypecheckIter, false); LUAU_FASTFLAGVARIABLE(LuauSuccessTypingForEqualityOperations, false) LUAU_FASTFLAGVARIABLE(LuauNoMethodLocations, false); +LUAU_FASTFLAGVARIABLE(LuauAlwaysQuantify, false); namespace Luau { @@ -371,12 +364,7 @@ ModulePtr TypeChecker::checkWithoutRecursionCheck(const SourceModule& module, Mo prepareErrorsForDisplay(currentModule->errors); - bool encounteredFreeType = currentModule->clonePublicInterface(*iceHandler); - if (!FFlag::LuauLosslessClone && encounteredFreeType) - { - reportError(TypeError{module.root->location, - GenericError{"Free types leaked into this module's public interface. This is an internal Luau error; please report it."}}); - } + currentModule->clonePublicInterface(*iceHandler); // Clear unifier cache since it's keyed off internal types that get deallocated // This avoids fake cross-module cache hits and keeps cache size at bay when typechecking large module graphs. @@ -701,7 +689,7 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatIf& statement) ExprResult result = checkExpr(scope, *statement.condition); ScopePtr ifScope = childScope(scope, statement.thenbody->location); - reportErrors(resolve(result.predicates, ifScope, true)); + resolve(result.predicates, ifScope, true); check(ifScope, *statement.thenbody); if (statement.elsebody) @@ -734,7 +722,7 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatWhile& statement) ExprResult result = checkExpr(scope, *statement.condition); ScopePtr whileScope = childScope(scope, statement.body->location); - reportErrors(resolve(result.predicates, whileScope, true)); + resolve(result.predicates, whileScope, true); check(whileScope, *statement.body); } @@ -1154,10 +1142,7 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatForIn& forin) } else { - if (FFlag::LuauInstantiateFollows) - iterTy = instantiate(scope, checkExpr(scope, *firstValue).type, firstValue->location); - else - iterTy = follow(instantiate(scope, checkExpr(scope, *firstValue).type, firstValue->location)); + iterTy = instantiate(scope, checkExpr(scope, *firstValue).type, firstValue->location); } if (FFlag::LuauTypecheckIter) @@ -1849,23 +1834,11 @@ std::optional TypeChecker::getIndexTypeFromType( tablify(type); - if (FFlag::LuauDiscriminableUnions2) + if (isString(type)) { - if (isString(type)) - { - std::optional mtIndex = findMetatableEntry(stringType, "__index", location); - LUAU_ASSERT(mtIndex); - type = *mtIndex; - } - } - else - { - const PrimitiveTypeVar* primitiveType = get(type); - if (primitiveType && primitiveType->type == PrimitiveTypeVar::String) - { - if (std::optional mtIndex = findMetatableEntry(type, "__index", location)) - type = *mtIndex; - } + std::optional mtIndex = findMetatableEntry(stringType, "__index", location); + LUAU_ASSERT(mtIndex); + type = *mtIndex; } if (TableTypeVar* tableType = getMutableTableType(type)) @@ -1966,23 +1939,10 @@ std::optional TypeChecker::getIndexTypeFromType( return std::nullopt; } - if (FFlag::LuauDoNotTryToReduce) - { - if (parts.size() == 1) - return parts[0]; + if (parts.size() == 1) + return parts[0]; - return addType(IntersectionTypeVar{std::move(parts)}); // Not at all correct. - } - else - { - // TODO(amccord): Write some logic to correctly handle intersections. CLI-34659 - std::vector result = reduceUnion(parts); - - if (result.size() == 1) - return result[0]; - - return addType(IntersectionTypeVar{result}); - } + return addType(IntersectionTypeVar{std::move(parts)}); // Not at all correct. } if (addErrors) @@ -1993,103 +1953,55 @@ std::optional TypeChecker::getIndexTypeFromType( std::vector TypeChecker::reduceUnion(const std::vector& types) { - if (FFlag::LuauDoNotAccidentallyDependOnPointerOrdering) + std::vector result; + for (TypeId t : types) { - std::vector result; - for (TypeId t : types) + t = follow(t); + if (get(t) || get(t)) + return {t}; + + if (const UnionTypeVar* utv = get(t)) { - t = follow(t); - if (get(t) || get(t)) - return {t}; - - if (const UnionTypeVar* utv = get(t)) + if (FFlag::LuauReduceUnionRecursion) { - if (FFlag::LuauReduceUnionRecursion) + for (TypeId ty : utv) { - for (TypeId ty : utv) - { - if (get(ty) || get(ty)) - return {ty}; - - if (result.end() == std::find(result.begin(), result.end(), ty)) - result.push_back(ty); - } - } - else - { - std::vector r = reduceUnion(utv->options); - for (TypeId ty : r) - { + if (FFlag::LuauNormalizeFlagIsConservative) ty = follow(ty); - if (get(ty) || get(ty)) - return {ty}; + if (get(ty) || get(ty)) + return {ty}; - if (std::find(result.begin(), result.end(), ty) == result.end()) - result.push_back(ty); - } + if (result.end() == std::find(result.begin(), result.end(), ty)) + result.push_back(ty); } } - else if (std::find(result.begin(), result.end(), t) == result.end()) - result.push_back(t); - } - - return result; - } - else - { - std::set s; - - for (TypeId t : types) - { - if (const UnionTypeVar* utv = get(follow(t))) + else { std::vector r = reduceUnion(utv->options); for (TypeId ty : r) - s.insert(ty); + { + ty = follow(ty); + if (get(ty) || get(ty)) + return {ty}; + + if (std::find(result.begin(), result.end(), ty) == result.end()) + result.push_back(ty); + } } - else - s.insert(t); } - - // If any of them are ErrorTypeVars/AnyTypeVars, decay into them. - for (TypeId t : s) - { - t = follow(t); - if (get(t) || get(t)) - return {t}; - } - - std::vector r(s.begin(), s.end()); - std::sort(r.begin(), r.end()); - return r; + else if (std::find(result.begin(), result.end(), t) == result.end()) + result.push_back(t); } + + return result; } std::optional TypeChecker::tryStripUnionFromNil(TypeId ty) { if (const UnionTypeVar* utv = get(ty)) { - if (FFlag::LuauAnyInIsOptionalIsOptional) - { - if (!std::any_of(begin(utv), end(utv), isNil)) - return ty; - } - else - { - bool hasNil = false; - - for (TypeId option : utv) - { - if (isNil(option)) - { - hasNil = true; - break; - } - } - - if (!hasNil) - return ty; - } + if (!std::any_of(begin(utv), end(utv), isNil)) + return ty; std::vector result; @@ -2110,32 +2022,18 @@ std::optional TypeChecker::tryStripUnionFromNil(TypeId ty) TypeId TypeChecker::stripFromNilAndReport(TypeId ty, const Location& location) { - if (FFlag::LuauAnyInIsOptionalIsOptional) + ty = follow(ty); + + if (auto utv = get(ty)) { - ty = follow(ty); - - if (auto utv = get(ty)) - { - if (!std::any_of(begin(utv), end(utv), isNil)) - return ty; - } - - if (std::optional strippedUnion = tryStripUnionFromNil(ty)) - { - reportError(location, OptionalValueAccess{ty}); - return follow(*strippedUnion); - } + if (!std::any_of(begin(utv), end(utv), isNil)) + return ty; } - else + + if (std::optional strippedUnion = tryStripUnionFromNil(ty)) { - if (isOptional(ty)) - { - if (std::optional strippedUnion = tryStripUnionFromNil(follow(ty))) - { - reportError(location, OptionalValueAccess{ty}); - return follow(*strippedUnion); - } - } + reportError(location, OptionalValueAccess{ty}); + return follow(*strippedUnion); } return ty; @@ -2194,8 +2092,7 @@ TypeId TypeChecker::checkExprTable( if (indexer) { - if (FFlag::LuauCheckImplicitNumbericKeys) - unify(numberType, indexer->indexType, value->location); + unify(numberType, indexer->indexType, value->location); unify(valueType, indexer->indexResultType, value->location); } else @@ -2219,7 +2116,8 @@ TypeId TypeChecker::checkExprTable( if (errors.empty()) exprType = expectedProp.type; } - else if (expectedTable->indexer && isString(expectedTable->indexer->indexType)) + else if (expectedTable->indexer && (FFlag::LuauExpectedPropTypeFromIndexer ? maybeString(expectedTable->indexer->indexType) + : isString(expectedTable->indexer->indexType))) { ErrorVec errors = tryUnify(exprType, expectedTable->indexer->indexResultType, k->location); if (errors.empty()) @@ -2259,26 +2157,13 @@ TypeId TypeChecker::checkExprTable( ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprTable& expr, std::optional expectedType) { - if (FFlag::LuauTableUseCounterInstead) + RecursionCounter _rc(&checkRecursionCount); + if (FInt::LuauCheckRecursionLimit > 0 && checkRecursionCount >= FInt::LuauCheckRecursionLimit) { - RecursionCounter _rc(&checkRecursionCount); - if (FInt::LuauCheckRecursionLimit > 0 && checkRecursionCount >= FInt::LuauCheckRecursionLimit) - { - reportErrorCodeTooComplex(expr.location); - return {errorRecoveryType(scope)}; - } - - return checkExpr_(scope, expr, expectedType); + reportErrorCodeTooComplex(expr.location); + return {errorRecoveryType(scope)}; } - else - { - RecursionLimiter _rl(&recursionCount, FInt::LuauTypeInferRecursionLimit, "checkExpr for tables"); - return checkExpr_(scope, expr, expectedType); - } -} -ExprResult TypeChecker::checkExpr_(const ScopePtr& scope, const AstExprTable& expr, std::optional expectedType) -{ std::vector> fieldTypes(expr.items.size); const TableTypeVar* expectedTable = nullptr; @@ -2324,6 +2209,8 @@ ExprResult TypeChecker::checkExpr_(const ScopePtr& scope, const AstExprT { if (auto prop = expectedTable->props.find(key->value.data); prop != expectedTable->props.end()) expectedResultType = prop->second.type; + else if (FFlag::LuauExpectedPropTypeFromIndexer && expectedIndexType && maybeString(*expectedIndexType)) + expectedResultType = expectedIndexResultType; } else if (expectedUnion) { @@ -2529,7 +2416,7 @@ TypeId TypeChecker::checkRelationalOperation( if (expr.op == AstExprBinary::Or && subexp->op == AstExprBinary::And) { ScopePtr subScope = childScope(scope, subexp->location); - reportErrors(resolve(predicates, subScope, true)); + resolve(predicates, subScope, true); return unionOfTypes(rhsType, stripNil(checkExpr(subScope, *subexp->right).type, true), expr.location); } } @@ -2851,8 +2738,7 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprBi auto [rhsTy, rhsPredicates] = checkExpr(innerScope, *expr.right); - return {checkBinaryOperation(FFlag::LuauDiscriminableUnions2 ? scope : innerScope, expr, lhsTy, rhsTy), - {AndPredicate{std::move(lhsPredicates), std::move(rhsPredicates)}}}; + return {checkBinaryOperation(scope, expr, lhsTy, rhsTy), {AndPredicate{std::move(lhsPredicates), std::move(rhsPredicates)}}}; } else if (expr.op == AstExprBinary::Or) { @@ -2864,7 +2750,7 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprBi auto [rhsTy, rhsPredicates] = checkExpr(innerScope, *expr.right); // Because of C++, I'm not sure if lhsPredicates was not moved out by the time we call checkBinaryOperation. - TypeId result = checkBinaryOperation(FFlag::LuauDiscriminableUnions2 ? scope : innerScope, expr, lhsTy, rhsTy, lhsPredicates); + TypeId result = checkBinaryOperation(scope, expr, lhsTy, rhsTy, lhsPredicates); return {result, {OrPredicate{std::move(lhsPredicates), std::move(rhsPredicates)}}}; } else if (expr.op == AstExprBinary::CompareEq || expr.op == AstExprBinary::CompareNe) @@ -2872,8 +2758,8 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprBi if (auto predicate = tryGetTypeGuardPredicate(expr)) return {booleanType, {std::move(*predicate)}}; - ExprResult lhs = checkExpr(scope, *expr.left, std::nullopt, /*forceSingleton=*/FFlag::LuauDiscriminableUnions2); - ExprResult rhs = checkExpr(scope, *expr.right, std::nullopt, /*forceSingleton=*/FFlag::LuauDiscriminableUnions2); + ExprResult lhs = checkExpr(scope, *expr.left, std::nullopt, /*forceSingleton=*/true); + ExprResult rhs = checkExpr(scope, *expr.right, std::nullopt, /*forceSingleton=*/true); PredicateVec predicates; @@ -2931,12 +2817,12 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprEr ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprIfElse& expr, std::optional expectedType) { ExprResult result = checkExpr(scope, *expr.condition); + ScopePtr trueScope = childScope(scope, expr.trueExpr->location); - reportErrors(resolve(result.predicates, trueScope, true)); + resolve(result.predicates, trueScope, true); ExprResult trueType = checkExpr(trueScope, *expr.trueExpr, expectedType); ScopePtr falseScope = childScope(scope, expr.falseExpr->location); - // Don't report errors for this scope to avoid potentially duplicating errors reported for the first scope. resolve(result.predicates, falseScope, false); ExprResult falseType = checkExpr(falseScope, *expr.falseExpr, expectedType); @@ -3668,9 +3554,6 @@ void TypeChecker::checkArgumentList( else if (state.log.getMutable(t)) { } // ok - else if (!FFlag::LuauAnyInIsOptionalIsOptional && isNonstrictMode() && state.log.get(t)) - { - } // ok else { size_t minParams = getMinParameterCount(&state.log, paramPack); @@ -3823,9 +3706,6 @@ ExprResult TypeChecker::checkExprPack(const ScopePtr& scope, const A actualFunctionType = instantiate(scope, functionType, expr.func->location); } - if (!FFlag::LuauInstantiateFollows) - actualFunctionType = follow(actualFunctionType); - TypePackId retPack; if (FFlag::LuauLowerBoundsCalculation || !FFlag::LuauWidenIfSupertypeIsFree2) { @@ -4096,32 +3976,6 @@ std::optional> TypeChecker::checkCallOverload(const Scope { state.log.commit(); - if (!FFlag::LuauAnyInIsOptionalIsOptional && isNonstrictMode() && !expr.self && expr.func->is() && ftv->hasSelf) - { - // If we are running in nonstrict mode, passing fewer arguments than the function is declared to take AND - // the function is declared with colon notation AND we use dot notation, warn. - auto [providedArgs, providedTail] = flatten(argPack); - - // If we have a variadic tail, we can't say how many arguments were actually provided - if (!providedTail) - { - std::vector actualArgs = flatten(ftv->argTypes).first; - - size_t providedCount = providedArgs.size(); - size_t requiredCount = actualArgs.size(); - - // Ignore optional arguments - while (providedCount < requiredCount && requiredCount != 0 && isOptional(actualArgs[requiredCount - 1])) - requiredCount--; - - if (providedCount < requiredCount) - { - int requiredExtraNils = int(requiredCount - providedCount); - reportError(TypeError{expr.func->location, FunctionRequiresSelf{requiredExtraNils}}); - } - } - } - currentModule->astOverloadResolvedTypes[&expr] = fn; // We select this overload @@ -4525,7 +4379,7 @@ bool Instantiation::isDirty(TypeId ty) { if (const FunctionTypeVar* ftv = log->getMutable(ty)) { - if (FFlag::LuauTypecheckOptPass && ftv->hasNoGenerics) + if (ftv->hasNoGenerics) return false; return true; @@ -4582,7 +4436,7 @@ bool ReplaceGenerics::ignoreChildren(TypeId ty) { if (const FunctionTypeVar* ftv = log->getMutable(ty)) { - if (FFlag::LuauTypecheckOptPass && ftv->hasNoGenerics) + if (ftv->hasNoGenerics) return true; // We aren't recursing in the case of a generic function which @@ -4701,8 +4555,17 @@ TypeId TypeChecker::quantify(const ScopePtr& scope, TypeId ty, Location location ty = follow(ty); const FunctionTypeVar* ftv = get(ty); - if (ftv && ftv->generics.empty() && ftv->genericPacks.empty()) - Luau::quantify(ty, scope->level); + + if (FFlag::LuauAlwaysQuantify) + { + if (ftv) + Luau::quantify(ty, scope->level); + } + else + { + if (ftv && ftv->generics.empty() && ftv->genericPacks.empty()) + Luau::quantify(ty, scope->level); + } if (FFlag::LuauLowerBoundsCalculation && ftv) { @@ -4717,15 +4580,11 @@ TypeId TypeChecker::quantify(const ScopePtr& scope, TypeId ty, Location location TypeId TypeChecker::instantiate(const ScopePtr& scope, TypeId ty, Location location, const TxnLog* log) { - if (FFlag::LuauInstantiateFollows) - ty = follow(ty); + ty = follow(ty); - if (FFlag::LuauTypecheckOptPass) - { - const FunctionTypeVar* ftv = get(FFlag::LuauInstantiateFollows ? ty : follow(ty)); - if (ftv && ftv->hasNoGenerics) - return ty; - } + const FunctionTypeVar* ftv = get(ty); + if (ftv && ftv->hasNoGenerics) + return ty; Instantiation instantiation{log, ¤tModule->internalTypes, scope->level}; @@ -5392,10 +5251,9 @@ TypePackId TypeChecker::resolveTypePack(const ScopePtr& scope, const AstTypePack bool ApplyTypeFunction::isDirty(TypeId ty) { - // Really this should just replace the arguments, - // but for bug-compatibility with existing code, we replace - // all generics. - if (get(ty)) + if (FFlag::LuauApplyTypeFunctionFix && typeArguments.count(ty)) + return true; + else if (!FFlag::LuauApplyTypeFunctionFix && get(ty)) return true; else if (const FreeTypeVar* ftv = get(ty)) { @@ -5409,10 +5267,9 @@ bool ApplyTypeFunction::isDirty(TypeId ty) bool ApplyTypeFunction::isDirty(TypePackId tp) { - // Really this should just replace the arguments, - // but for bug-compatibility with existing code, we replace - // all generics. - if (get(tp)) + if (FFlag::LuauApplyTypeFunctionFix && typePackArguments.count(tp)) + return true; + else if (!FFlag::LuauApplyTypeFunctionFix && get(tp)) return true; else return false; @@ -5436,11 +5293,13 @@ bool ApplyTypeFunction::ignoreChildren(TypePackId tp) TypeId ApplyTypeFunction::clean(TypeId ty) { - // Really this should just replace the arguments, - // but for bug-compatibility with existing code, we replace - // all generics by free type variables. TypeId& arg = typeArguments[ty]; - if (arg) + if (FFlag::LuauApplyTypeFunctionFix) + { + LUAU_ASSERT(arg); + return arg; + } + else if (arg) return arg; else return addType(FreeTypeVar{level}); @@ -5448,11 +5307,13 @@ TypeId ApplyTypeFunction::clean(TypeId ty) TypePackId ApplyTypeFunction::clean(TypePackId tp) { - // Really this should just replace the arguments, - // but for bug-compatibility with existing code, we replace - // all generics by free type variables. TypePackId& arg = typePackArguments[tp]; - if (arg) + if (FFlag::LuauApplyTypeFunctionFix) + { + LUAU_ASSERT(arg); + return arg; + } + else if (arg) return arg; else return addTypePack(FreeTypePack{level}); @@ -5596,8 +5457,6 @@ GenericTypeDefinitions TypeChecker::createGenericTypes(const ScopePtr& scope, st void TypeChecker::refineLValue(const LValue& lvalue, RefinementMap& refis, const ScopePtr& scope, TypeIdPredicate predicate) { - LUAU_ASSERT(FFlag::LuauDiscriminableUnions2 || FFlag::LuauAssertStripsFalsyTypes); - const LValue* target = &lvalue; std::optional key; // If set, we know we took the base of the lvalue path and should be walking down each option of the base's type. @@ -5683,66 +5542,6 @@ std::optional TypeChecker::resolveLValue(const ScopePtr& scope, const LV // We need to search in the provided Scope. Find t.x.y first. // We fail to find t.x.y. Try t.x. We found it. Now we must return the type of the property y from the mapped-to type of t.x. // If we completely fail to find the Symbol t but the Scope has that entry, then we should walk that all the way through and terminate. - if (!FFlag::LuauTypecheckOptPass) - { - const auto& [symbol, keys] = getFullName(lvalue); - - ScopePtr currentScope = scope; - while (currentScope) - { - std::optional found; - - std::vector childKeys; - const LValue* currentLValue = &lvalue; - while (currentLValue) - { - if (auto it = currentScope->refinements.find(*currentLValue); it != currentScope->refinements.end()) - { - found = it->second; - break; - } - - childKeys.push_back(*currentLValue); - currentLValue = baseof(*currentLValue); - } - - if (!found) - { - // Should not be using scope->lookup. This is already recursive. - if (auto it = currentScope->bindings.find(symbol); it != currentScope->bindings.end()) - found = it->second.typeId; - else - { - // Nothing exists in this Scope. Just skip and try the parent one. - currentScope = currentScope->parent; - continue; - } - } - - for (auto it = childKeys.rbegin(); it != childKeys.rend(); ++it) - { - const LValue& key = *it; - - // Symbol can happen. Skip. - if (get(key)) - continue; - else if (auto field = get(key)) - { - found = getIndexTypeFromType(scope, *found, field->key, Location(), false); - if (!found) - return std::nullopt; // Turns out this type doesn't have the property at all. We're done. - } - else - LUAU_ASSERT(!"New LValue alternative not handled here."); - } - - return found; - } - - // No entry for it at all. Can happen when LValue root is a global. - return std::nullopt; - } - const Symbol symbol = getBaseSymbol(lvalue); ScopePtr currentScope = scope; @@ -5820,85 +5619,47 @@ static bool isUndecidable(TypeId ty) return get(ty) || get(ty) || get(ty); } -ErrorVec TypeChecker::resolve(const PredicateVec& predicates, const ScopePtr& scope, bool sense) +void TypeChecker::resolve(const PredicateVec& predicates, const ScopePtr& scope, bool sense) { - ErrorVec errVec; - resolve(predicates, errVec, scope->refinements, scope, sense); - return errVec; + resolve(predicates, scope->refinements, scope, sense); } -void TypeChecker::resolve(const PredicateVec& predicates, ErrorVec& errVec, RefinementMap& refis, const ScopePtr& scope, bool sense, bool fromOr) +void TypeChecker::resolve(const PredicateVec& predicates, RefinementMap& refis, const ScopePtr& scope, bool sense, bool fromOr) { for (const Predicate& c : predicates) - resolve(c, errVec, refis, scope, sense, fromOr); + resolve(c, refis, scope, sense, fromOr); } -void TypeChecker::resolve(const Predicate& predicate, ErrorVec& errVec, RefinementMap& refis, const ScopePtr& scope, bool sense, bool fromOr) +void TypeChecker::resolve(const Predicate& predicate, RefinementMap& refis, const ScopePtr& scope, bool sense, bool fromOr) { if (auto truthyP = get(predicate)) - resolve(*truthyP, errVec, refis, scope, sense, fromOr); + resolve(*truthyP, refis, scope, sense, fromOr); else if (auto andP = get(predicate)) - resolve(*andP, errVec, refis, scope, sense); + resolve(*andP, refis, scope, sense); else if (auto orP = get(predicate)) - resolve(*orP, errVec, refis, scope, sense); + resolve(*orP, refis, scope, sense); else if (auto notP = get(predicate)) - resolve(notP->predicates, errVec, refis, scope, !sense, fromOr); + resolve(notP->predicates, refis, scope, !sense, fromOr); else if (auto isaP = get(predicate)) - resolve(*isaP, errVec, refis, scope, sense); + resolve(*isaP, refis, scope, sense); else if (auto typeguardP = get(predicate)) - resolve(*typeguardP, errVec, refis, scope, sense); + resolve(*typeguardP, refis, scope, sense); else if (auto eqP = get(predicate)) - resolve(*eqP, errVec, refis, scope, sense); + resolve(*eqP, refis, scope, sense); else ice("Unhandled predicate kind"); } -void TypeChecker::resolve(const TruthyPredicate& truthyP, ErrorVec& errVec, RefinementMap& refis, const ScopePtr& scope, bool sense, bool fromOr) +void TypeChecker::resolve(const TruthyPredicate& truthyP, RefinementMap& refis, const ScopePtr& scope, bool sense, bool fromOr) { - if (FFlag::LuauAssertStripsFalsyTypes) - { - std::optional ty = resolveLValue(refis, scope, truthyP.lvalue); - if (ty && fromOr) - return addRefinement(refis, truthyP.lvalue, *ty); + std::optional ty = resolveLValue(refis, scope, truthyP.lvalue); + if (ty && fromOr) + return addRefinement(refis, truthyP.lvalue, *ty); - refineLValue(truthyP.lvalue, refis, scope, mkTruthyPredicate(sense)); - } - else - { - auto predicate = [sense](TypeId option) -> std::optional { - if (isUndecidable(option) || isBoolean(option) || isNil(option) != sense) - return option; - - return std::nullopt; - }; - - if (FFlag::LuauDiscriminableUnions2) - { - std::optional ty = resolveLValue(refis, scope, truthyP.lvalue); - if (ty && fromOr) - return addRefinement(refis, truthyP.lvalue, *ty); - - refineLValue(truthyP.lvalue, refis, scope, predicate); - } - else - { - std::optional ty = resolveLValue(refis, scope, truthyP.lvalue); - if (!ty) - return; - - // This is a hack. :( - // Without this, the expression 'a or b' might refine 'b' to be falsy. - // I'm not yet sure how else to get this to do the right thing without this hack, so we'll do this for now in the meantime. - if (fromOr) - return addRefinement(refis, truthyP.lvalue, *ty); - - if (std::optional result = filterMap(*ty, predicate)) - addRefinement(refis, truthyP.lvalue, *result); - } - } + refineLValue(truthyP.lvalue, refis, scope, mkTruthyPredicate(sense)); } -void TypeChecker::resolve(const AndPredicate& andP, ErrorVec& errVec, RefinementMap& refis, const ScopePtr& scope, bool sense) +void TypeChecker::resolve(const AndPredicate& andP, RefinementMap& refis, const ScopePtr& scope, bool sense) { if (!sense) { @@ -5907,14 +5668,14 @@ void TypeChecker::resolve(const AndPredicate& andP, ErrorVec& errVec, Refinement {NotPredicate{std::move(andP.rhs)}}, }; - return resolve(orP, errVec, refis, scope, !sense); + return resolve(orP, refis, scope, !sense); } - resolve(andP.lhs, errVec, refis, scope, sense); - resolve(andP.rhs, errVec, refis, scope, sense); + resolve(andP.lhs, refis, scope, sense); + resolve(andP.rhs, refis, scope, sense); } -void TypeChecker::resolve(const OrPredicate& orP, ErrorVec& errVec, RefinementMap& refis, const ScopePtr& scope, bool sense) +void TypeChecker::resolve(const OrPredicate& orP, RefinementMap& refis, const ScopePtr& scope, bool sense) { if (!sense) { @@ -5923,28 +5684,24 @@ void TypeChecker::resolve(const OrPredicate& orP, ErrorVec& errVec, RefinementMa {NotPredicate{std::move(orP.rhs)}}, }; - return resolve(andP, errVec, refis, scope, !sense); + return resolve(andP, refis, scope, !sense); } - ErrorVec discarded; - RefinementMap leftRefis; - resolve(orP.lhs, errVec, leftRefis, scope, sense); + resolve(orP.lhs, leftRefis, scope, sense); RefinementMap rightRefis; - resolve(orP.lhs, discarded, rightRefis, scope, !sense); - resolve(orP.rhs, errVec, rightRefis, scope, sense, true); // :( + resolve(orP.lhs, rightRefis, scope, !sense); + resolve(orP.rhs, rightRefis, scope, sense, true); // :( merge(refis, leftRefis); merge(refis, rightRefis); } -void TypeChecker::resolve(const IsAPredicate& isaP, ErrorVec& errVec, RefinementMap& refis, const ScopePtr& scope, bool sense) +void TypeChecker::resolve(const IsAPredicate& isaP, RefinementMap& refis, const ScopePtr& scope, bool sense) { auto predicate = [&](TypeId option) -> std::optional { // This by itself is not truly enough to determine that A is stronger than B or vice versa. - // The best unambiguous way about this would be to have a function that returns the relationship ordering of a pair. - // i.e. TypeRelationship relationshipOf(TypeId superTy, TypeId subTy) bool optionIsSubtype = canUnify(option, isaP.ty, isaP.location).empty(); bool targetIsSubtype = canUnify(isaP.ty, option, isaP.location).empty(); @@ -5985,32 +5742,15 @@ void TypeChecker::resolve(const IsAPredicate& isaP, ErrorVec& errVec, Refinement return res; }; - if (FFlag::LuauDiscriminableUnions2) - { - refineLValue(isaP.lvalue, refis, scope, predicate); - } - else - { - std::optional ty = resolveLValue(refis, scope, isaP.lvalue); - if (!ty) - return; - - if (std::optional result = filterMap(*ty, predicate)) - addRefinement(refis, isaP.lvalue, *result); - else - { - addRefinement(refis, isaP.lvalue, errorRecoveryType(scope)); - errVec.push_back(TypeError{isaP.location, TypeMismatch{isaP.ty, *ty}}); - } - } + refineLValue(isaP.lvalue, refis, scope, predicate); } -void TypeChecker::resolve(const TypeGuardPredicate& typeguardP, ErrorVec& errVec, RefinementMap& refis, const ScopePtr& scope, bool sense) +void TypeChecker::resolve(const TypeGuardPredicate& typeguardP, RefinementMap& refis, const ScopePtr& scope, bool sense) { // Rewrite the predicate 'type(foo) == "vector"' to be 'typeof(foo) == "Vector3"'. They're exactly identical. // This allows us to avoid writing in edge cases. if (!typeguardP.isTypeof && typeguardP.kind == "vector") - return resolve(TypeGuardPredicate{std::move(typeguardP.lvalue), typeguardP.location, "Vector3", true}, errVec, refis, scope, sense); + return resolve(TypeGuardPredicate{std::move(typeguardP.lvalue), typeguardP.location, "Vector3", true}, refis, scope, sense); std::optional ty = resolveLValue(refis, scope, typeguardP.lvalue); if (!ty) @@ -6060,52 +5800,29 @@ void TypeChecker::resolve(const TypeGuardPredicate& typeguardP, ErrorVec& errVec if (auto it = primitives.find(typeguardP.kind); it != primitives.end()) { - if (FFlag::LuauDiscriminableUnions2) - { - refineLValue(typeguardP.lvalue, refis, scope, it->second(sense)); - return; - } - else - { - if (std::optional result = filterMap(*ty, it->second(sense))) - addRefinement(refis, typeguardP.lvalue, *result); - else - { - addRefinement(refis, typeguardP.lvalue, errorRecoveryType(scope)); - if (sense) - errVec.push_back( - TypeError{typeguardP.location, GenericError{"Type '" + toString(*ty) + "' has no overlap with '" + typeguardP.kind + "'"}}); - } - - return; - } + refineLValue(typeguardP.lvalue, refis, scope, it->second(sense)); + return; } - auto fail = [&](const TypeErrorData& err) { - if (!FFlag::LuauDiscriminableUnions2) - errVec.push_back(TypeError{typeguardP.location, err}); - addRefinement(refis, typeguardP.lvalue, errorRecoveryType(scope)); - }; - if (!typeguardP.isTypeof) - return fail(UnknownSymbol{typeguardP.kind, UnknownSymbol::Type}); + return addRefinement(refis, typeguardP.lvalue, errorRecoveryType(scope)); auto typeFun = globalScope->lookupType(typeguardP.kind); if (!typeFun || !typeFun->typeParams.empty() || !typeFun->typePackParams.empty()) - return fail(UnknownSymbol{typeguardP.kind, UnknownSymbol::Type}); + return addRefinement(refis, typeguardP.lvalue, errorRecoveryType(scope)); TypeId type = follow(typeFun->type); // We're only interested in the root class of any classes. if (auto ctv = get(type); !ctv || ctv->parent) - return fail(UnknownSymbol{typeguardP.kind, UnknownSymbol::Type}); + return addRefinement(refis, typeguardP.lvalue, errorRecoveryType(scope)); // This probably hints at breaking out type filtering functions from the predicate solver so that typeof is not tightly coupled with IsA. // Until then, we rewrite this to be the same as using IsA. - return resolve(IsAPredicate{std::move(typeguardP.lvalue), typeguardP.location, type}, errVec, refis, scope, sense); + return resolve(IsAPredicate{std::move(typeguardP.lvalue), typeguardP.location, type}, refis, scope, sense); } -void TypeChecker::resolve(const EqPredicate& eqP, ErrorVec& errVec, RefinementMap& refis, const ScopePtr& scope, bool sense) +void TypeChecker::resolve(const EqPredicate& eqP, RefinementMap& refis, const ScopePtr& scope, bool sense) { // This refinement will require success typing to do everything correctly. For now, we can get most of the way there. auto options = [](TypeId ty) -> std::vector { @@ -6114,82 +5831,33 @@ void TypeChecker::resolve(const EqPredicate& eqP, ErrorVec& errVec, RefinementMa return {ty}; }; - if (FFlag::LuauDiscriminableUnions2) - { - std::vector rhs = options(eqP.type); + std::vector rhs = options(eqP.type); - if (sense && std::any_of(rhs.begin(), rhs.end(), isUndecidable)) - return; // Optimization: the other side has unknown types, so there's probably an overlap. Refining is no-op here. + if (sense && std::any_of(rhs.begin(), rhs.end(), isUndecidable)) + return; // Optimization: the other side has unknown types, so there's probably an overlap. Refining is no-op here. - auto predicate = [&](TypeId option) -> std::optional { - if (sense && isUndecidable(option)) - return FFlag::LuauWeakEqConstraint ? option : eqP.type; + auto predicate = [&](TypeId option) -> std::optional { + if (sense && isUndecidable(option)) + return FFlag::LuauWeakEqConstraint ? option : eqP.type; - if (!sense && isNil(eqP.type)) - return (isUndecidable(option) || !isNil(option)) ? std::optional(option) : std::nullopt; + if (!sense && isNil(eqP.type)) + return (isUndecidable(option) || !isNil(option)) ? std::optional(option) : std::nullopt; - if (maybeSingleton(eqP.type)) - { - // Normally we'd write option <: eqP.type, but singletons are always the subtype, so we flip this. - if (!sense || canUnify(eqP.type, option, eqP.location).empty()) - return sense ? eqP.type : option; - - // local variable works around an odd gcc 9.3 warning: may be used uninitialized - std::optional res = std::nullopt; - return res; - } - - return option; - }; - - refineLValue(eqP.lvalue, refis, scope, predicate); - } - else - { - if (FFlag::LuauWeakEqConstraint) + if (maybeSingleton(eqP.type)) { - if (!sense && isNil(eqP.type)) - resolve(TruthyPredicate{std::move(eqP.lvalue), eqP.location}, errVec, refis, scope, true, /* fromOr= */ false); + // Normally we'd write option <: eqP.type, but singletons are always the subtype, so we flip this. + if (!sense || canUnify(eqP.type, option, eqP.location).empty()) + return sense ? eqP.type : option; - return; + // local variable works around an odd gcc 9.3 warning: may be used uninitialized + std::optional res = std::nullopt; + return res; } - if (FFlag::LuauEqConstraint) - { - std::optional ty = resolveLValue(refis, scope, eqP.lvalue); - if (!ty) - return; + return option; + }; - std::vector lhs = options(*ty); - std::vector rhs = options(eqP.type); - - if (sense && std::any_of(lhs.begin(), lhs.end(), isUndecidable)) - { - addRefinement(refis, eqP.lvalue, eqP.type); - return; - } - else if (sense && std::any_of(rhs.begin(), rhs.end(), isUndecidable)) - return; // Optimization: the other side has unknown types, so there's probably an overlap. Refining is no-op here. - - std::unordered_set set; - for (TypeId left : lhs) - { - for (TypeId right : rhs) - { - // When singleton types arrive, `isNil` here probably should be replaced with `isLiteral`. - if (canUnify(right, left, eqP.location).empty() == sense || (!sense && !isNil(left))) - set.insert(left); - } - } - - if (set.empty()) - return; - - std::vector viable(set.begin(), set.end()); - TypeId result = viable.size() == 1 ? viable[0] : addType(UnionTypeVar{std::move(viable)}); - addRefinement(refis, eqP.lvalue, result); - } - } + refineLValue(eqP.lvalue, refis, scope, predicate); } bool TypeChecker::isNonstrictMode() const diff --git a/Analysis/src/TypeUtils.cpp b/Analysis/src/TypeUtils.cpp index c2435890..ba09df5f 100644 --- a/Analysis/src/TypeUtils.cpp +++ b/Analysis/src/TypeUtils.cpp @@ -5,8 +5,6 @@ #include "Luau/ToString.h" #include "Luau/TypeInfer.h" -LUAU_FASTFLAGVARIABLE(LuauTerminateCyclicMetatableIndexLookup, false) - namespace Luau { @@ -55,13 +53,10 @@ std::optional findTablePropertyRespectingMeta(ErrorVec& errors, TypeId t { TypeId index = follow(*mtIndex); - if (FFlag::LuauTerminateCyclicMetatableIndexLookup) - { - if (count >= 100) - return std::nullopt; + if (count >= 100) + return std::nullopt; - ++count; - } + ++count; if (const auto& itt = getTableType(index)) { diff --git a/Analysis/src/TypeVar.cpp b/Analysis/src/TypeVar.cpp index 463b4651..2355dab2 100644 --- a/Analysis/src/TypeVar.cpp +++ b/Analysis/src/TypeVar.cpp @@ -24,8 +24,6 @@ LUAU_FASTINTVARIABLE(LuauTypeMaximumStringifierLength, 500) LUAU_FASTINTVARIABLE(LuauTableTypeMaximumStringifierLength, 0) LUAU_FASTINT(LuauTypeInferRecursionLimit) LUAU_FASTFLAG(LuauSubtypingAddOptPropsToUnsealedTables) -LUAU_FASTFLAG(LuauDiscriminableUnions2) -LUAU_FASTFLAGVARIABLE(LuauAnyInIsOptionalIsOptional, false) LUAU_FASTFLAGVARIABLE(LuauClassDefinitionModuleInError, false) namespace Luau @@ -204,14 +202,14 @@ bool isOptional(TypeId ty) ty = follow(ty); - if (FFlag::LuauAnyInIsOptionalIsOptional && get(ty)) + if (get(ty)) return true; auto utv = get(ty); if (!utv) return false; - return std::any_of(begin(utv), end(utv), FFlag::LuauAnyInIsOptionalIsOptional ? isOptional : isNil); + return std::any_of(begin(utv), end(utv), isOptional); } bool isTableIntersection(TypeId ty) @@ -378,8 +376,7 @@ bool hasLength(TypeId ty, DenseHashSet& seen, int* recursionCount) if (seen.contains(ty)) return true; - bool isStr = FFlag::LuauDiscriminableUnions2 ? isString(ty) : isPrim(ty, PrimitiveTypeVar::String); - if (isStr || get(ty) || get(ty) || get(ty)) + if (isString(ty) || get(ty) || get(ty) || get(ty)) return true; if (auto uty = get(ty)) diff --git a/Analysis/src/Unifier.cpp b/Analysis/src/Unifier.cpp index f5c1dde9..9308e9ff 100644 --- a/Analysis/src/Unifier.cpp +++ b/Analysis/src/Unifier.cpp @@ -24,8 +24,6 @@ LUAU_FASTFLAGVARIABLE(LuauSubtypingAddOptPropsToUnsealedTables, false) LUAU_FASTFLAGVARIABLE(LuauWidenIfSupertypeIsFree2, false) LUAU_FASTFLAGVARIABLE(LuauDifferentOrderOfUnificationDoesntMatter2, false) LUAU_FASTFLAGVARIABLE(LuauTxnLogRefreshFunctionPointers, false) -LUAU_FASTFLAG(LuauAnyInIsOptionalIsOptional) -LUAU_FASTFLAG(LuauTypecheckOptPass) namespace Luau { @@ -382,19 +380,6 @@ Unifier::Unifier(TypeArena* types, Mode mode, const Location& location, Variance LUAU_ASSERT(sharedState.iceHandler); } -Unifier::Unifier(TypeArena* types, Mode mode, std::vector>* sharedSeen, const Location& location, - Variance variance, UnifierSharedState& sharedState, TxnLog* parentLog) - : types(types) - , mode(mode) - , log(parentLog, sharedSeen) - , location(location) - , variance(variance) - , sharedState(sharedState) -{ - LUAU_ASSERT(!FFlag::LuauTypecheckOptPass); - LUAU_ASSERT(sharedState.iceHandler); -} - void Unifier::tryUnify(TypeId subTy, TypeId superTy, bool isFunctionCall, bool isIntersection) { sharedState.counters.iterationCount = 0; @@ -1219,14 +1204,6 @@ void Unifier::tryUnify_(TypePackId subTp, TypePackId superTp, bool isFunctionCal continue; } - // In nonstrict mode, any also marks an optional argument. - else if (!FFlag::LuauAnyInIsOptionalIsOptional && superIter.good() && isNonstrictMode() && - log.getMutable(log.follow(*superIter))) - { - superIter.advance(); - continue; - } - if (log.getMutable(superIter.packId)) { tryUnifyVariadics(subIter.packId, superIter.packId, false, int(subIter.index)); @@ -1454,21 +1431,9 @@ void Unifier::tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) { auto subIter = subTable->props.find(propName); - if (FFlag::LuauAnyInIsOptionalIsOptional) - { - if (subIter == subTable->props.end() && - (!FFlag::LuauSubtypingAddOptPropsToUnsealedTables || subTable->state == TableState::Unsealed) && !isOptional(superProp.type)) - missingProperties.push_back(propName); - } - else - { - bool isAny = log.getMutable(log.follow(superProp.type)); - - if (subIter == subTable->props.end() && - (!FFlag::LuauSubtypingAddOptPropsToUnsealedTables || subTable->state == TableState::Unsealed) && !isOptional(superProp.type) && - !isAny) - missingProperties.push_back(propName); - } + if (subIter == subTable->props.end() && (!FFlag::LuauSubtypingAddOptPropsToUnsealedTables || subTable->state == TableState::Unsealed) && + !isOptional(superProp.type)) + missingProperties.push_back(propName); } if (!missingProperties.empty()) @@ -1485,18 +1450,8 @@ void Unifier::tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) { auto superIter = superTable->props.find(propName); - if (FFlag::LuauAnyInIsOptionalIsOptional) - { - if (superIter == superTable->props.end() && (FFlag::LuauSubtypingAddOptPropsToUnsealedTables || !isOptional(subProp.type))) - extraProperties.push_back(propName); - } - else - { - bool isAny = log.is(log.follow(subProp.type)); - if (superIter == superTable->props.end() && - (FFlag::LuauSubtypingAddOptPropsToUnsealedTables || (!isOptional(subProp.type) && !isAny))) - extraProperties.push_back(propName); - } + if (superIter == superTable->props.end() && (FFlag::LuauSubtypingAddOptPropsToUnsealedTables || !isOptional(subProp.type))) + extraProperties.push_back(propName); } if (!extraProperties.empty()) @@ -1540,21 +1495,12 @@ void Unifier::tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) if (innerState.errors.empty()) log.concat(std::move(innerState.log)); } - else if (FFlag::LuauAnyInIsOptionalIsOptional && - (!FFlag::LuauSubtypingAddOptPropsToUnsealedTables || subTable->state == TableState::Unsealed) && isOptional(prop.type)) + else if ((!FFlag::LuauSubtypingAddOptPropsToUnsealedTables || subTable->state == TableState::Unsealed) && isOptional(prop.type)) // This is sound because unsealed table types are precise, so `{ p : T } <: { p : T, q : U? }` // since if `t : { p : T }` then we are guaranteed that `t.q` is `nil`. // TODO: if the supertype is written to, the subtype may no longer be precise (alias analysis?) { } - else if ((!FFlag::LuauSubtypingAddOptPropsToUnsealedTables || subTable->state == TableState::Unsealed) && - (isOptional(prop.type) || get(follow(prop.type)))) - // This is sound because unsealed table types are precise, so `{ p : T } <: { p : T, q : U? }` - // since if `t : { p : T }` then we are guaranteed that `t.q` is `nil`. - // TODO: should isOptional(anyType) be true? - // TODO: if the supertype is written to, the subtype may no longer be precise (alias analysis?) - { - } else if (subTable->state == TableState::Free) { PendingType* pendingSub = log.queue(subTy); @@ -1618,10 +1564,7 @@ void Unifier::tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) else if (variance == Covariant) { } - else if (FFlag::LuauAnyInIsOptionalIsOptional && !FFlag::LuauSubtypingAddOptPropsToUnsealedTables && isOptional(prop.type)) - { - } - else if (!FFlag::LuauSubtypingAddOptPropsToUnsealedTables && (isOptional(prop.type) || get(follow(prop.type)))) + else if (!FFlag::LuauSubtypingAddOptPropsToUnsealedTables && isOptional(prop.type)) { } else if (superTable->state == TableState::Free) @@ -1753,9 +1696,7 @@ TypePackId Unifier::widen(TypePackId tp) TypeId Unifier::deeplyOptional(TypeId ty, std::unordered_map seen) { ty = follow(ty); - if (!FFlag::LuauAnyInIsOptionalIsOptional && get(ty)) - return ty; - else if (isOptional(ty)) + if (isOptional(ty)) return ty; else if (const TableTypeVar* ttv = get(ty)) { @@ -2666,14 +2607,7 @@ void Unifier::occursCheck(DenseHashSet& seen, TypePackId needle, Typ Unifier Unifier::makeChildUnifier() { - if (FFlag::LuauTypecheckOptPass) - { - Unifier u = Unifier{types, mode, location, variance, sharedState, &log}; - u.anyIsTop = anyIsTop; - return u; - } - - Unifier u = Unifier{types, mode, log.sharedSeen, location, variance, sharedState, &log}; + Unifier u = Unifier{types, mode, location, variance, sharedState, &log}; u.anyIsTop = anyIsTop; return u; } diff --git a/Compiler/include/Luau/BytecodeBuilder.h b/Compiler/include/Luau/BytecodeBuilder.h index b00440ae..12465377 100644 --- a/Compiler/include/Luau/BytecodeBuilder.h +++ b/Compiler/include/Luau/BytecodeBuilder.h @@ -224,6 +224,7 @@ private: DenseHashMap constantMap; DenseHashMap tableShapeMap; + DenseHashMap protoMap; int debugLine = 0; diff --git a/Compiler/src/BytecodeBuilder.cpp b/Compiler/src/BytecodeBuilder.cpp index fb70392e..beeda295 100644 --- a/Compiler/src/BytecodeBuilder.cpp +++ b/Compiler/src/BytecodeBuilder.cpp @@ -6,6 +6,8 @@ #include #include +LUAU_FASTFLAG(LuauCompileNestedClosureO2) + namespace Luau { @@ -181,6 +183,7 @@ size_t BytecodeBuilder::TableShapeHash::operator()(const TableShape& v) const BytecodeBuilder::BytecodeBuilder(BytecodeEncoder* encoder) : constantMap({Constant::Type_Nil, ~0ull}) , tableShapeMap(TableShape()) + , protoMap(~0u) , stringTable({nullptr, 0}) , encoder(encoder) { @@ -250,6 +253,7 @@ void BytecodeBuilder::endFunction(uint8_t maxstacksize, uint8_t numupvalues) constantMap.clear(); tableShapeMap.clear(); + protoMap.clear(); debugRemarks.clear(); debugRemarkBuffer.clear(); @@ -372,11 +376,17 @@ int32_t BytecodeBuilder::addConstantClosure(uint32_t fid) int16_t BytecodeBuilder::addChildFunction(uint32_t fid) { + if (FFlag::LuauCompileNestedClosureO2) + if (int16_t* cache = protoMap.find(fid)) + return *cache; + uint32_t id = uint32_t(protos.size()); if (id >= kMaxClosureCount) return -1; + if (FFlag::LuauCompileNestedClosureO2) + protoMap[fid] = int16_t(id); protos.push_back(fid); return int16_t(id); diff --git a/Compiler/src/Compiler.cpp b/Compiler/src/Compiler.cpp index e177e928..4f26ceb9 100644 --- a/Compiler/src/Compiler.cpp +++ b/Compiler/src/Compiler.cpp @@ -17,8 +17,6 @@ #include #include -LUAU_FASTFLAGVARIABLE(LuauCompileSupportInlining, false) - LUAU_FASTFLAGVARIABLE(LuauCompileIter, false) LUAU_FASTFLAGVARIABLE(LuauCompileIterNoReserve, false) LUAU_FASTFLAGVARIABLE(LuauCompileIterNoPairs, false) @@ -30,6 +28,8 @@ LUAU_FASTINTVARIABLE(LuauCompileInlineThreshold, 25) LUAU_FASTINTVARIABLE(LuauCompileInlineThresholdMaxBoost, 300) LUAU_FASTINTVARIABLE(LuauCompileInlineDepth, 5) +LUAU_FASTFLAGVARIABLE(LuauCompileNestedClosureO2, false) + namespace Luau { @@ -100,13 +100,11 @@ struct Compiler upvals.reserve(16); } - uint8_t getLocal(AstLocal* local) + int getLocalReg(AstLocal* local) { Local* l = locals.find(local); - LUAU_ASSERT(l); - LUAU_ASSERT(l->allocated); - return l->reg; + return l && l->allocated ? l->reg : -1; } uint8_t getUpval(AstLocal* local) @@ -159,17 +157,19 @@ struct Compiler AstExprFunction* getFunctionExpr(AstExpr* node) { - if (AstExprLocal* le = node->as()) + if (AstExprLocal* expr = node->as()) { - Variable* lv = variables.find(le->local); + Variable* lv = variables.find(expr->local); if (!lv || lv->written || !lv->init) return nullptr; return getFunctionExpr(lv->init); } - else if (AstExprGroup* ge = node->as()) - return getFunctionExpr(ge->expr); + else if (AstExprGroup* expr = node->as()) + return getFunctionExpr(expr->expr); + else if (AstExprTypeAssertion* expr = node->as()) + return getFunctionExpr(expr->expr); else return node->as(); } @@ -180,13 +180,13 @@ struct Compiler { bool result = true; - bool visit(AstExpr* node) override + bool visit(AstExprFunction* node) override { - // nested functions may capture function arguments, and our upval handling doesn't handle elided variables (constant) - // TODO: we could remove this case if we changed function compilation to create temporary locals for constant upvalues - // TODO: additionally we would need to change upvalue handling in compileExprFunction to handle upvalue->local migration - result = result && !node->is(); - return result; + if (!FFlag::LuauCompileNestedClosureO2) + result = false; + + // short-circuit to avoid analyzing nested closure bodies + return false; } bool visit(AstStat* node) override @@ -275,8 +275,7 @@ struct Compiler f.upvals = upvals; // record information for inlining - if (FFlag::LuauCompileSupportInlining && options.optimizationLevel >= 2 && !func->vararg && canInlineFunctionBody(func->body) && - !getfenvUsed && !setfenvUsed) + if (options.optimizationLevel >= 2 && !func->vararg && canInlineFunctionBody(func->body) && !getfenvUsed && !setfenvUsed) { f.canInline = true; f.stackSize = stackSize; @@ -346,8 +345,8 @@ struct Compiler uint8_t argreg; - if (isExprLocalReg(arg)) - argreg = getLocal(arg->as()->local); + if (int reg = getExprLocalReg(arg); reg >= 0) + argreg = uint8_t(reg); else { argreg = uint8_t(regs + 1); @@ -403,8 +402,8 @@ struct Compiler } } - if (isExprLocalReg(expr->args.data[i])) - args[i] = getLocal(expr->args.data[i]->as()->local); + if (int reg = getExprLocalReg(expr->args.data[i]); reg >= 0) + args[i] = uint8_t(reg); else { args[i] = uint8_t(regs + 1 + i); @@ -489,19 +488,18 @@ struct Compiler return false; } - // TODO: we can compile functions with mismatching arity at call site but it's more annoying - if (func->args.size != expr->args.size) - { - bytecode.addDebugRemark("inlining failed: argument count mismatch (expected %d, got %d)", int(func->args.size), int(expr->args.size)); - return false; - } - - // we use a dynamic cost threshold that's based on the fixed limit boosted by the cost advantage we gain due to inlining + // compute constant bitvector for all arguments to feed the cost model bool varc[8] = {}; - for (size_t i = 0; i < expr->args.size && i < 8; ++i) + for (size_t i = 0; i < func->args.size && i < expr->args.size && i < 8; ++i) varc[i] = isConstant(expr->args.data[i]); - int inlinedCost = computeCost(fi->costModel, varc, std::min(int(expr->args.size), 8)); + // if the last argument only returns a single value, all following arguments are nil + if (expr->args.size != 0 && !(expr->args.data[expr->args.size - 1]->is() || expr->args.data[expr->args.size - 1]->is())) + for (size_t i = expr->args.size; i < func->args.size && i < 8; ++i) + varc[i] = true; + + // we use a dynamic cost threshold that's based on the fixed limit boosted by the cost advantage we gain due to inlining + int inlinedCost = computeCost(fi->costModel, varc, std::min(int(func->args.size), 8)); int baselineCost = computeCost(fi->costModel, nullptr, 0) + 3; int inlineProfit = (inlinedCost == 0) ? thresholdMaxBoost : std::min(thresholdMaxBoost, 100 * baselineCost / inlinedCost); @@ -533,15 +531,44 @@ struct Compiler for (size_t i = 0; i < func->args.size; ++i) { AstLocal* var = func->args.data[i]; - AstExpr* arg = expr->args.data[i]; + AstExpr* arg = i < expr->args.size ? expr->args.data[i] : nullptr; - if (Variable* vv = variables.find(var); vv && vv->written) + if (i + 1 == expr->args.size && func->args.size > expr->args.size && (arg->is() || arg->is())) + { + // if the last argument can return multiple values, we need to compute all of them into the remaining arguments + unsigned int tail = unsigned(func->args.size - expr->args.size) + 1; + uint8_t reg = allocReg(arg, tail); + + if (AstExprCall* expr = arg->as()) + compileExprCall(expr, reg, tail, /* targetTop= */ true); + else if (AstExprVarargs* expr = arg->as()) + compileExprVarargs(expr, reg, tail); + else + LUAU_ASSERT(!"Unexpected expression type"); + + for (size_t j = i; j < func->args.size; ++j) + pushLocal(func->args.data[j], uint8_t(reg + (j - i))); + + // all remaining function arguments have been allocated and assigned to + break; + } + else if (Variable* vv = variables.find(var); vv && vv->written) { // if the argument is mutated, we need to allocate a fresh register even if it's a constant uint8_t reg = allocReg(arg, 1); - compileExprTemp(arg, reg); + + if (arg) + compileExprTemp(arg, reg); + else + bytecode.emitABC(LOP_LOADNIL, reg, 0, 0); + pushLocal(var, reg); } + else if (arg == nullptr) + { + // since the argument is not mutated, we can simply fold the value into the expressions that need it + locstants[var] = {Constant::Type_Nil}; + } else if (const Constant* cv = constants.find(arg); cv && cv->type != Constant::Type_Unknown) { // since the argument is not mutated, we can simply fold the value into the expressions that need it @@ -553,20 +580,26 @@ struct Compiler Variable* lv = le ? variables.find(le->local) : nullptr; // if the argument is a local that isn't mutated, we will simply reuse the existing register - if (isExprLocalReg(arg) && (!lv || !lv->written)) + if (int reg = le ? getExprLocalReg(le) : -1; reg >= 0 && (!lv || !lv->written)) { - uint8_t reg = getLocal(le->local); - pushLocal(var, reg); + pushLocal(var, uint8_t(reg)); } else { - uint8_t reg = allocReg(arg, 1); - compileExprTemp(arg, reg); - pushLocal(var, reg); + uint8_t temp = allocReg(arg, 1); + compileExprTemp(arg, temp); + pushLocal(var, temp); } } } + // evaluate extra expressions for side effects + for (size_t i = func->args.size; i < expr->args.size; ++i) + { + RegScope rsi(this); + compileExprAuto(expr->args.data[i], rsi); + } + // fold constant values updated above into expressions in the function body foldConstants(constants, variables, locstants, func->body); @@ -627,12 +660,15 @@ struct Compiler FInt::LuauCompileInlineThresholdMaxBoost, FInt::LuauCompileInlineDepth)) return; - if (fi && !fi->canInline) + // add a debug remark for cases when we didn't even call tryCompileInlinedCall + if (func && !(fi && fi->canInline)) { if (func->vararg) bytecode.addDebugRemark("inlining failed: function is variadic"); - else + else if (fi) bytecode.addDebugRemark("inlining failed: complex constructs in function body"); + else + bytecode.addDebugRemark("inlining failed: can't inline recursive calls"); } } @@ -677,9 +713,9 @@ struct Compiler LUAU_ASSERT(fi); // Optimization: use local register directly in NAMECALL if possible - if (isExprLocalReg(fi->expr)) + if (int reg = getExprLocalReg(fi->expr); reg >= 0) { - selfreg = getLocal(fi->expr->as()->local); + selfreg = uint8_t(reg); } else { @@ -785,6 +821,8 @@ struct Compiler void compileExprFunction(AstExprFunction* expr, uint8_t target) { + RegScope rs(this); + const Function* f = functions.find(expr); LUAU_ASSERT(f); @@ -795,6 +833,67 @@ struct Compiler if (pid < 0) CompileError::raise(expr->location, "Exceeded closure limit; simplify the code to compile"); + if (FFlag::LuauCompileNestedClosureO2) + { + captures.clear(); + captures.reserve(f->upvals.size()); + + for (AstLocal* uv : f->upvals) + { + LUAU_ASSERT(uv->functionDepth < expr->functionDepth); + + if (int reg = getLocalReg(uv); reg >= 0) + { + // note: we can't check if uv is an upvalue in the current frame because inlining can migrate from upvalues to locals + Variable* ul = variables.find(uv); + bool immutable = !ul || !ul->written; + + captures.push_back({immutable ? LCT_VAL : LCT_REF, uint8_t(reg)}); + } + else if (const Constant* uc = locstants.find(uv); uc && uc->type != Constant::Type_Unknown) + { + // inlining can result in an upvalue capture of a constant, in which case we can't capture without a temporary register + uint8_t reg = allocReg(expr, 1); + compileExprConstant(expr, uc, reg); + + captures.push_back({LCT_VAL, reg}); + } + else + { + LUAU_ASSERT(uv->functionDepth < expr->functionDepth - 1); + + // get upvalue from parent frame + // note: this will add uv to the current upvalue list if necessary + uint8_t uid = getUpval(uv); + + captures.push_back({LCT_UPVAL, uid}); + } + } + + // Optimization: when closure has no upvalues, or upvalues are safe to share, instead of allocating it every time we can share closure + // objects (this breaks assumptions about function identity which can lead to setfenv not working as expected, so we disable this when it + // is used) + int16_t shared = -1; + + if (options.optimizationLevel >= 1 && shouldShareClosure(expr) && !setfenvUsed) + { + int32_t cid = bytecode.addConstantClosure(f->id); + + if (cid >= 0 && cid < 32768) + shared = int16_t(cid); + } + + if (shared >= 0) + bytecode.emitAD(LOP_DUPCLOSURE, target, shared); + else + bytecode.emitAD(LOP_NEWCLOSURE, target, pid); + + for (const Capture& c : captures) + bytecode.emitABC(LOP_CAPTURE, uint8_t(c.type), c.data, 0); + + return; + } + bool shared = false; // Optimization: when closure has no upvalues, or upvalues are safe to share, instead of allocating it every time we can share closure @@ -824,9 +923,10 @@ struct Compiler if (uv->functionDepth == expr->functionDepth - 1) { // get local variable - uint8_t reg = getLocal(uv); + int reg = getLocalReg(uv); + LUAU_ASSERT(reg >= 0); - bytecode.emitABC(LOP_CAPTURE, uint8_t(immutable ? LCT_VAL : LCT_REF), reg, 0); + bytecode.emitABC(LOP_CAPTURE, uint8_t(immutable ? LCT_VAL : LCT_REF), uint8_t(reg), 0); } else { @@ -1213,10 +1313,10 @@ struct Compiler if (!isConditionFast(expr->left)) { // Optimization: when right hand side is a local variable, we can use AND/OR - if (isExprLocalReg(expr->right)) + if (int reg = getExprLocalReg(expr->right); reg >= 0) { uint8_t lr = compileExprAuto(expr->left, rs); - uint8_t rr = getLocal(expr->right->as()->local); + uint8_t rr = uint8_t(reg); bytecode.emitABC(and_ ? LOP_AND : LOP_OR, target, lr, rr); return; @@ -1803,19 +1903,18 @@ struct Compiler } else if (AstExprLocal* expr = node->as()) { - if (FFlag::LuauCompileSupportInlining ? !isExprLocalReg(expr) : expr->upvalue) + // note: this can't check expr->upvalue because upvalues may be upgraded to locals during inlining + if (int reg = getExprLocalReg(expr); reg >= 0) + { + bytecode.emitABC(LOP_MOVE, target, uint8_t(reg), 0); + } + else { LUAU_ASSERT(expr->upvalue); uint8_t uid = getUpval(expr->local); bytecode.emitABC(LOP_GETUPVAL, target, uid, 0); } - else - { - uint8_t reg = getLocal(expr->local); - - bytecode.emitABC(LOP_MOVE, target, reg, 0); - } } else if (AstExprGlobal* expr = node->as()) { @@ -1879,8 +1978,8 @@ struct Compiler uint8_t compileExprAuto(AstExpr* node, RegScope&) { // Optimization: directly return locals instead of copying them to a temporary - if (isExprLocalReg(node)) - return getLocal(node->as()->local); + if (int reg = getExprLocalReg(node); reg >= 0) + return uint8_t(reg); // note: the register is owned by the parent scope uint8_t reg = allocReg(node, 1); @@ -1910,7 +2009,7 @@ struct Compiler for (size_t i = 0; i < targetCount; ++i) compileExprTemp(list.data[i], uint8_t(target + i)); - // compute expressions with values that go nowhere; this is required to run side-effecting code if any + // evaluate extra expressions for side effects for (size_t i = targetCount; i < list.size; ++i) { RegScope rsi(this); @@ -2008,20 +2107,21 @@ struct Compiler if (AstExprLocal* expr = node->as()) { - if (FFlag::LuauCompileSupportInlining ? !isExprLocalReg(expr) : expr->upvalue) + // note: this can't check expr->upvalue because upvalues may be upgraded to locals during inlining + if (int reg = getExprLocalReg(expr); reg >= 0) { - LUAU_ASSERT(expr->upvalue); - - LValue result = {LValue::Kind_Upvalue}; - result.upval = getUpval(expr->local); + LValue result = {LValue::Kind_Local}; + result.reg = uint8_t(reg); result.location = node->location; return result; } else { - LValue result = {LValue::Kind_Local}; - result.reg = getLocal(expr->local); + LUAU_ASSERT(expr->upvalue); + + LValue result = {LValue::Kind_Upvalue}; + result.upval = getUpval(expr->local); result.location = node->location; return result; @@ -2115,15 +2215,21 @@ struct Compiler compileLValueUse(lv, source, /* set= */ true); } - bool isExprLocalReg(AstExpr* expr) + int getExprLocalReg(AstExpr* node) { - AstExprLocal* le = expr->as(); - if (!le || (!FFlag::LuauCompileSupportInlining && le->upvalue)) - return false; + if (AstExprLocal* expr = node->as()) + { + // note: this can't check expr->upvalue because upvalues may be upgraded to locals during inlining + Local* l = locals.find(expr->local); - Local* l = locals.find(le->local); - - return l && l->allocated; + return l && l->allocated ? l->reg : -1; + } + else if (AstExprGroup* expr = node->as()) + return getExprLocalReg(expr->expr); + else if (AstExprTypeAssertion* expr = node->as()) + return getExprLocalReg(expr->expr); + else + return -1; } bool isStatBreak(AstStat* node) @@ -2352,20 +2458,17 @@ struct Compiler // Optimization: return locals directly instead of copying them into a temporary // this is very important for a single return value and occasionally effective for multiple values - if (stat->list.size > 0 && isExprLocalReg(stat->list.data[0])) + if (int reg = stat->list.size > 0 ? getExprLocalReg(stat->list.data[0]) : -1; reg >= 0) { - temp = getLocal(stat->list.data[0]->as()->local); + temp = uint8_t(reg); consecutive = true; for (size_t i = 1; i < stat->list.size; ++i) - { - AstExpr* v = stat->list.data[i]; - if (!isExprLocalReg(v) || getLocal(v->as()->local) != temp + i) + if (getExprLocalReg(stat->list.data[i]) != int(temp + i)) { consecutive = false; break; } - } } if (!consecutive && stat->list.size > 0) @@ -2438,12 +2541,13 @@ struct Compiler { bool result = true; - bool visit(AstExpr* node) override + bool visit(AstExprFunction* node) override { - // functions may capture loop variable, and our upval handling doesn't handle elided variables (constant) - // TODO: we could remove this case if we changed function compilation to create temporary locals for constant upvalues - result = result && !node->is(); - return result; + if (!FFlag::LuauCompileNestedClosureO2) + result = false; + + // short-circuit to avoid analyzing nested closure bodies + return false; } bool visit(AstStat* node) override @@ -2874,12 +2978,9 @@ struct Compiler void compileStatFunction(AstStatFunction* stat) { // Optimization: compile value expresion directly into target local register - if (isExprLocalReg(stat->name)) + if (int reg = getExprLocalReg(stat->name); reg >= 0) { - AstExprLocal* le = stat->name->as(); - LUAU_ASSERT(le); - - compileExpr(stat->func, getLocal(le->local)); + compileExpr(stat->func, uint8_t(reg)); return; } @@ -3399,6 +3500,12 @@ struct Compiler std::vector returnJumps; }; + struct Capture + { + LuauCaptureType type; + uint8_t data; + }; + BytecodeBuilder& bytecode; CompileOptions options; @@ -3422,6 +3529,7 @@ struct Compiler std::vector loopJumps; std::vector loops; std::vector inlineFrames; + std::vector captures; }; void compileOrThrow(BytecodeBuilder& bytecode, AstStatBlock* root, const AstNameTable& names, const CompileOptions& options) @@ -3465,6 +3573,9 @@ void compileOrThrow(BytecodeBuilder& bytecode, AstStatBlock* root, const AstName /* self= */ nullptr, AstArray(), /* vararg= */ Luau::Location(), root, /* functionDepth= */ 0, /* debugname= */ AstName()); uint32_t mainid = compiler.compileFunction(&main); + const Compiler::Function* mainf = compiler.functions.find(&main); + LUAU_ASSERT(mainf && mainf->upvals.empty()); + bytecode.setMainFunction(mainid); bytecode.finalize(); } diff --git a/Compiler/src/ConstantFolding.cpp b/Compiler/src/ConstantFolding.cpp index e4d59ea1..a62beeb1 100644 --- a/Compiler/src/ConstantFolding.cpp +++ b/Compiler/src/ConstantFolding.cpp @@ -3,8 +3,6 @@ #include -LUAU_FASTFLAG(LuauCompileSupportInlining) - namespace Luau { namespace Compile @@ -330,7 +328,7 @@ struct ConstantVisitor : AstVisitor { if (value.type != Constant::Type_Unknown) map[key] = value; - else if (!FFlag::LuauCompileSupportInlining || wasEmpty) + else if (wasEmpty) ; else if (Constant* old = map.find(key)) old->type = Constant::Type_Unknown; diff --git a/Sources.cmake b/Sources.cmake index d2430cc9..297f561a 100644 --- a/Sources.cmake +++ b/Sources.cmake @@ -73,6 +73,7 @@ target_sources(Luau.Analysis PRIVATE Analysis/include/Luau/ToString.h Analysis/include/Luau/Transpiler.h Analysis/include/Luau/TxnLog.h + Analysis/include/Luau/TypeArena.h Analysis/include/Luau/TypeAttach.h Analysis/include/Luau/TypedAllocator.h Analysis/include/Luau/TypeInfer.h @@ -108,6 +109,7 @@ target_sources(Luau.Analysis PRIVATE Analysis/src/ToString.cpp Analysis/src/Transpiler.cpp Analysis/src/TxnLog.cpp + Analysis/src/TypeArena.cpp Analysis/src/TypeAttach.cpp Analysis/src/TypedAllocator.cpp Analysis/src/TypeInfer.cpp diff --git a/VM/src/ltablib.cpp b/VM/src/ltablib.cpp index 9c1f387e..27187c61 100644 --- a/VM/src/ltablib.cpp +++ b/VM/src/ltablib.cpp @@ -10,10 +10,6 @@ #include "ldebug.h" #include "lvm.h" -LUAU_DYNAMIC_FASTFLAGVARIABLE(LuauTableMoveTelemetry2, false) - -void (*lua_table_move_telemetry)(lua_State* L, int f, int e, int t, int nf, int nt); - static int foreachi(lua_State* L) { luaL_checktype(L, 1, LUA_TTABLE); @@ -199,29 +195,6 @@ static int tmove(lua_State* L) int tt = !lua_isnoneornil(L, 5) ? 5 : 1; /* destination table */ luaL_checktype(L, tt, LUA_TTABLE); - void (*telemetrycb)(lua_State * L, int f, int e, int t, int nf, int nt) = lua_table_move_telemetry; - - if (DFFlag::LuauTableMoveTelemetry2 && telemetrycb && e >= f) - { - int nf = lua_objlen(L, 1); - int nt = lua_objlen(L, tt); - - bool report = false; - - // source index range must be in bounds in source table unless the table is empty (permits 1..#t moves) - if (!(f == 1 || (f >= 1 && f <= nf))) - report = true; - if (!(e == nf || (e >= 1 && e <= nf))) - report = true; - - // destination index must be in bounds in dest table or be exactly at the first empty element (permits concats) - if (!(t == nt + 1 || (t >= 1 && t <= nt))) - report = true; - - if (report) - telemetrycb(L, f, e, t, nf, nt); - } - if (e >= f) { /* otherwise, nothing to move */ luaL_argcheck(L, f > 0 || e < INT_MAX + f, 3, "too many elements to move"); diff --git a/VM/src/lvmexecute.cpp b/VM/src/lvmexecute.cpp index 3c7c276a..9e2eb268 100644 --- a/VM/src/lvmexecute.cpp +++ b/VM/src/lvmexecute.cpp @@ -17,9 +17,6 @@ #include LUAU_FASTFLAGVARIABLE(LuauIter, false) -LUAU_DYNAMIC_FASTFLAGVARIABLE(LuauIterCallTelemetry, false) - -void (*lua_iter_call_telemetry)(lua_State* L); // Disable c99-designator to avoid the warning in CGOTO dispatch table #ifdef __clang__ @@ -157,17 +154,6 @@ LUAU_NOINLINE static bool luau_loopFORG(lua_State* L, int a, int c) StkId ra = &L->base[a]; LUAU_ASSERT(ra + 3 <= L->top); - if (DFFlag::LuauIterCallTelemetry) - { - /* TODO: we might be able to stop supporting this depending on whether it's used in practice */ - void (*telemetrycb)(lua_State* L) = lua_iter_call_telemetry; - - if (telemetrycb && ttistable(ra) && fasttm(L, hvalue(ra)->metatable, TM_CALL)) - telemetrycb(L); - if (telemetrycb && ttisuserdata(ra) && fasttm(L, uvalue(ra)->metatable, TM_CALL)) - telemetrycb(L); - } - setobjs2s(L, ra + 3 + 2, ra + 2); setobjs2s(L, ra + 3 + 1, ra + 1); setobjs2s(L, ra + 3, ra); diff --git a/tests/Autocomplete.test.cpp b/tests/Autocomplete.test.cpp index b4e9340c..caaccf4e 100644 --- a/tests/Autocomplete.test.cpp +++ b/tests/Autocomplete.test.cpp @@ -2772,6 +2772,8 @@ TEST_CASE_FIXTURE(ACBuiltinsFixture, "autocomplete_on_string_singletons") TEST_CASE_FIXTURE(ACFixture, "autocomplete_string_singletons") { + ScopedFastFlag sff{"LuauTwoPassAliasDefinitionFix", true}; + check(R"( type tag = "cat" | "dog" local function f(a: tag) end @@ -2844,6 +2846,8 @@ f(@1) TEST_CASE_FIXTURE(ACFixture, "autocomplete_string_singleton_escape") { + ScopedFastFlag sff{"LuauTwoPassAliasDefinitionFix", true}; + check(R"( type tag = "strange\t\"cat\"" | 'nice\t"dog"' local function f(x: tag) end diff --git a/tests/Compiler.test.cpp b/tests/Compiler.test.cpp index b032060e..cf27d191 100644 --- a/tests/Compiler.test.cpp +++ b/tests/Compiler.test.cpp @@ -4269,22 +4269,26 @@ FORNLOOP R3 -6 FORNLOOP R0 -11 RETURN R0 0 )"); +} - // can't unroll loops if the body has functions that refer to loop variables +TEST_CASE("LoopUnrollNestedClosure") +{ + ScopedFastFlag sff("LuauCompileNestedClosureO2", true); + + // if the body has functions that refer to loop variables, we unroll the loop and use MOVE+CAPTURE for upvalues CHECK_EQ("\n" + compileFunction(R"( -for i=1,1 do +for i=1,2 do local x = function() return i end end )", 1, 2), R"( -LOADN R2 1 -LOADN R0 1 LOADN R1 1 -FORNPREP R0 +3 -NEWCLOSURE R3 P0 -CAPTURE VAL R2 -FORNLOOP R0 -3 +NEWCLOSURE R0 P0 +CAPTURE VAL R1 +LOADN R1 2 +NEWCLOSURE R0 P0 +CAPTURE VAL R1 RETURN R0 0 )"); } @@ -4469,8 +4473,6 @@ RETURN R0 0 TEST_CASE("InlineBasic") { - ScopedFastFlag sff("LuauCompileSupportInlining", true); - // inline function that returns a constant CHECK_EQ("\n" + compileFunction(R"( local function foo() @@ -4550,10 +4552,72 @@ RETURN R1 1 )"); } +TEST_CASE("InlineBasicProhibited") +{ + ScopedFastFlag sff("LuauCompileNestedClosureO2", true); + + // we can't inline variadic functions + CHECK_EQ("\n" + compileFunction(R"( +local function foo(...) + return 42 +end + +local x = foo() +return x +)", + 1, 2), + R"( +DUPCLOSURE R0 K0 +MOVE R1 R0 +CALL R1 0 1 +RETURN R1 1 +)"); + + // we also can't inline functions that have internal loops + CHECK_EQ("\n" + compileFunction(R"( +local function foo() + for i=1,4 do end +end + +local x = foo() +return x +)", + 1, 2), + R"( +DUPCLOSURE R0 K0 +MOVE R1 R0 +CALL R1 0 1 +RETURN R1 1 +)"); +} + +TEST_CASE("InlineNestedClosures") +{ + ScopedFastFlag sff("LuauCompileNestedClosureO2", true); + + // we can inline functions that contain/return functions + CHECK_EQ("\n" + compileFunction(R"( +local function foo(x) + return function(y) return x + y end +end + +local x = foo(1)(2) +return x +)", + 2, 2), + R"( +DUPCLOSURE R0 K0 +LOADN R2 1 +NEWCLOSURE R1 P1 +CAPTURE VAL R2 +LOADN R2 2 +CALL R1 1 1 +RETURN R1 1 +)"); +} + TEST_CASE("InlineMutate") { - ScopedFastFlag sff("LuauCompileSupportInlining", true); - // if the argument is mutated, it gets a register even if the value is constant CHECK_EQ("\n" + compileFunction(R"( local function foo(a) @@ -4636,8 +4700,6 @@ RETURN R1 1 TEST_CASE("InlineUpval") { - ScopedFastFlag sff("LuauCompileSupportInlining", true); - // if the argument is an upvalue, we naturally need to copy it to a local CHECK_EQ("\n" + compileFunction(R"( local function foo(a) @@ -4705,8 +4767,6 @@ RETURN R1 1 TEST_CASE("InlineFallthrough") { - ScopedFastFlag sff("LuauCompileSupportInlining", true); - // if the function doesn't return, we still fill the results with nil CHECK_EQ("\n" + compileFunction(R"( local function foo() @@ -4759,8 +4819,6 @@ RETURN R1 -1 TEST_CASE("InlineCapture") { - ScopedFastFlag sff("LuauCompileSupportInlining", true); - // can't inline function with nested functions that capture locals because they might be constants CHECK_EQ("\n" + compileFunction(R"( local function foo(a) @@ -4782,12 +4840,9 @@ RETURN R2 -1 TEST_CASE("InlineArgMismatch") { - ScopedFastFlag sff("LuauCompileSupportInlining", true); - // when inlining a function, we must respect all the usual rules // caller might not have enough arguments - // TODO: we don't inline this atm CHECK_EQ("\n" + compileFunction(R"( local function foo(a) return a @@ -4799,13 +4854,11 @@ return x 1, 2), R"( DUPCLOSURE R0 K0 -MOVE R1 R0 -CALL R1 0 1 +LOADNIL R1 RETURN R1 1 )"); // caller might be using multret for arguments - // TODO: we don't inline this atm CHECK_EQ("\n" + compileFunction(R"( local function foo(a, b) return a + b @@ -4817,17 +4870,32 @@ return x 1, 2), R"( DUPCLOSURE R0 K0 -MOVE R1 R0 LOADK R3 K1 FASTCALL1 20 R3 +2 GETIMPORT R2 4 -CALL R2 1 -1 -CALL R1 -1 1 +CALL R2 1 2 +ADD R1 R2 R3 +RETURN R1 1 +)"); + + // caller might be using varargs for arguments + CHECK_EQ("\n" + compileFunction(R"( +local function foo(a, b) + return a + b +end + +local x = foo(...) +return x +)", + 1, 2), + R"( +DUPCLOSURE R0 K0 +GETVARARGS R2 2 +ADD R1 R2 R3 RETURN R1 1 )"); // caller might have too many arguments, but we still need to compute them for side effects - // TODO: we don't inline this atm CHECK_EQ("\n" + compileFunction(R"( local function foo(a) return a @@ -4839,19 +4907,34 @@ return x 1, 2), R"( DUPCLOSURE R0 K0 -MOVE R1 R0 +GETIMPORT R2 2 +CALL R2 0 1 +LOADN R1 42 +RETURN R1 1 +)"); + + // caller might not have enough arguments, and the arg might be mutated so it needs a register + CHECK_EQ("\n" + compileFunction(R"( +local function foo(a) + a = 42 + return a +end + +local x = foo() +return x +)", + 1, 2), + R"( +DUPCLOSURE R0 K0 +LOADNIL R2 LOADN R2 42 -GETIMPORT R3 2 -CALL R3 0 -1 -CALL R1 -1 1 +MOVE R1 R2 RETURN R1 1 )"); } TEST_CASE("InlineMultiple") { - ScopedFastFlag sff("LuauCompileSupportInlining", true); - // we call this with a different set of variable/constant args CHECK_EQ("\n" + compileFunction(R"( local function foo(a, b) @@ -4880,8 +4963,6 @@ RETURN R3 4 TEST_CASE("InlineChain") { - ScopedFastFlag sff("LuauCompileSupportInlining", true); - // inline a chain of functions CHECK_EQ("\n" + compileFunction(R"( local function foo(a, b) @@ -4912,8 +4993,6 @@ RETURN R3 1 TEST_CASE("InlineThresholds") { - ScopedFastFlag sff("LuauCompileSupportInlining", true); - ScopedFastInt sfis[] = { {"LuauCompileInlineThreshold", 25}, {"LuauCompileInlineThresholdMaxBoost", 300}, @@ -4988,8 +5067,6 @@ RETURN R3 1 TEST_CASE("InlineIIFE") { - ScopedFastFlag sff("LuauCompileSupportInlining", true); - // IIFE with arguments CHECK_EQ("\n" + compileFunction(R"( function choose(a, b, c) @@ -5025,8 +5102,6 @@ RETURN R3 1 TEST_CASE("InlineRecurseArguments") { - ScopedFastFlag sff("LuauCompileSupportInlining", true); - // we can't inline a function if it's used to compute its own arguments CHECK_EQ("\n" + compileFunction(R"( local function foo(a, b) @@ -5036,22 +5111,20 @@ foo(foo(foo,foo(foo,foo))[foo]) 1, 2), R"( DUPCLOSURE R0 K0 -MOVE R1 R0 +MOVE R2 R0 +MOVE R3 R0 MOVE R4 R0 MOVE R5 R0 MOVE R6 R0 -CALL R4 2 1 -LOADNIL R3 -GETTABLE R2 R3 R0 -CALL R1 1 0 +CALL R4 2 -1 +CALL R2 -1 1 +GETTABLE R1 R2 R0 RETURN R0 0 )"); } TEST_CASE("InlineFastCallK") { - ScopedFastFlag sff("LuauCompileSupportInlining", true); - CHECK_EQ("\n" + compileFunction(R"( local function set(l0) rawset({}, l0) @@ -5080,8 +5153,6 @@ RETURN R0 0 TEST_CASE("InlineExprIndexK") { - ScopedFastFlag sff("LuauCompileSupportInlining", true); - CHECK_EQ("\n" + compileFunction(R"( local _ = function(l0) local _ = nil @@ -5141,6 +5212,58 @@ RETURN R0 0 )"); } +TEST_CASE("InlineHiddenMutation") +{ + // when the argument is assigned inside the function, we can't reuse the local + CHECK_EQ("\n" + compileFunction(R"( +local function foo(a) + a = 42 + return a +end + +local x = ... +local y = foo(x :: number) +return y +)", + 1, 2), + R"( +DUPCLOSURE R0 K0 +GETVARARGS R1 1 +MOVE R3 R1 +LOADN R3 42 +MOVE R2 R3 +RETURN R2 1 +)"); + + // and neither can we do that when it's assigned outside the function + CHECK_EQ("\n" + compileFunction(R"( +local function foo(a) + mutator() + return a +end + +local x = ... +mutator = function() x = 42 end + +local y = foo(x :: number) +return y +)", + 2, 2), + R"( +DUPCLOSURE R0 K0 +GETVARARGS R1 1 +NEWCLOSURE R2 P1 +CAPTURE REF R1 +SETGLOBAL R2 K1 +MOVE R3 R1 +GETGLOBAL R4 K1 +CALL R4 0 0 +MOVE R2 R3 +CLOSEUPVALS R1 +RETURN R2 1 +)"); +} + TEST_CASE("ReturnConsecutive") { // we can return a single local directly @@ -5193,6 +5316,16 @@ return )"), R"( RETURN R0 0 +)"); + + // this optimization also works in presence of group / type casts + CHECK_EQ("\n" + compileFunction0(R"( +local x, y = ... +return (x), y :: number +)"), + R"( +GETVARARGS R0 2 +RETURN R0 2 )"); } diff --git a/tests/Module.test.cpp b/tests/Module.test.cpp index 4a999861..c7e18efd 100644 --- a/tests/Module.test.cpp +++ b/tests/Module.test.cpp @@ -198,10 +198,6 @@ TEST_CASE_FIXTURE(Fixture, "clone_class") TEST_CASE_FIXTURE(Fixture, "clone_free_types") { - ScopedFastFlag sff[]{ - {"LuauLosslessClone", true}, - }; - TypeVar freeTy(FreeTypeVar{TypeLevel{}}); TypePackVar freeTp(FreeTypePack{TypeLevel{}}); @@ -218,8 +214,6 @@ TEST_CASE_FIXTURE(Fixture, "clone_free_types") TEST_CASE_FIXTURE(Fixture, "clone_free_tables") { - ScopedFastFlag sff{"LuauLosslessClone", true}; - TypeVar tableTy{TableTypeVar{}}; TableTypeVar* ttv = getMutable(&tableTy); ttv->state = TableState::Free; @@ -252,8 +246,6 @@ TEST_CASE_FIXTURE(Fixture, "clone_constrained_intersection") TEST_CASE_FIXTURE(BuiltinsFixture, "clone_self_property") { - ScopedFastFlag sff{"LuauAnyInIsOptionalIsOptional", true}; - fileResolver.source["Module/A"] = R"( --!nonstrict local a = {} diff --git a/tests/NonstrictMode.test.cpp b/tests/NonstrictMode.test.cpp index 69430b1c..83c526ef 100644 --- a/tests/NonstrictMode.test.cpp +++ b/tests/NonstrictMode.test.cpp @@ -150,8 +150,6 @@ TEST_CASE_FIXTURE(Fixture, "parameters_having_type_any_are_optional") TEST_CASE_FIXTURE(Fixture, "local_tables_are_not_any") { - ScopedFastFlag sff{"LuauAnyInIsOptionalIsOptional", true}; - CheckResult result = check(R"( --!nonstrict local T = {} @@ -169,8 +167,6 @@ TEST_CASE_FIXTURE(Fixture, "local_tables_are_not_any") TEST_CASE_FIXTURE(Fixture, "offer_a_hint_if_you_use_a_dot_instead_of_a_colon") { - ScopedFastFlag sff{"LuauAnyInIsOptionalIsOptional", true}; - CheckResult result = check(R"( --!nonstrict local T = {} diff --git a/tests/Normalize.test.cpp b/tests/Normalize.test.cpp index 41830682..dd49eb01 100644 --- a/tests/Normalize.test.cpp +++ b/tests/Normalize.test.cpp @@ -683,6 +683,7 @@ TEST_CASE_FIXTURE(Fixture, "cyclic_table_is_marked_normal") { ScopedFastFlag flags[] = { {"LuauLowerBoundsCalculation", true}, + {"LuauNormalizeFlagIsConservative", false} }; check(R"( @@ -697,6 +698,26 @@ TEST_CASE_FIXTURE(Fixture, "cyclic_table_is_marked_normal") CHECK(t->normal); } +// Unfortunately, getting this right in the general case is difficult. +TEST_CASE_FIXTURE(Fixture, "cyclic_table_is_not_marked_normal") +{ + ScopedFastFlag flags[] = { + {"LuauLowerBoundsCalculation", true}, + {"LuauNormalizeFlagIsConservative", true} + }; + + check(R"( + type Fiber = { + return_: Fiber? + } + + local f: Fiber + )"); + + TypeId t = requireType("f"); + CHECK(!t->normal); +} + TEST_CASE_FIXTURE(Fixture, "variadic_tail_is_marked_normal") { ScopedFastFlag flags[] = { @@ -997,4 +1018,28 @@ TEST_CASE_FIXTURE(Fixture, "fuzz_failure_bound_type_is_normal_but_not_its_bounde LUAU_REQUIRE_ERRORS(result); } +// We had an issue where a normal BoundTypeVar might point at a non-normal BoundTypeVar if it in turn pointed to a +// normal TypeVar because we were calling follow() in an improper place. +TEST_CASE_FIXTURE(Fixture, "bound_typevars_should_only_be_marked_normal_if_their_pointee_is_normal") +{ + ScopedFastFlag sff[]{ + {"LuauLowerBoundsCalculation", true}, + {"LuauNormalizeFlagIsConservative", true}, + }; + + CheckResult result = check(R"( + local T = {} + + function T:M() + local function f(a) + print(self.prop) + self:g(a) + self.prop = a + end + end + + return T + )"); +} + TEST_SUITE_END(); diff --git a/tests/RuntimeLimits.test.cpp b/tests/RuntimeLimits.test.cpp index c16f60d5..14c17614 100644 --- a/tests/RuntimeLimits.test.cpp +++ b/tests/RuntimeLimits.test.cpp @@ -22,8 +22,6 @@ struct LimitFixture : BuiltinsFixture #if defined(_NOOPT) || defined(_DEBUG) ScopedFastInt LuauTypeInferRecursionLimit{"LuauTypeInferRecursionLimit", 100}; #endif - - ScopedFastFlag LuauJustOneCallFrameForHaveSeen{"LuauJustOneCallFrameForHaveSeen", true}; }; template diff --git a/tests/ToString.test.cpp b/tests/ToString.test.cpp index f38dd10a..b854bc51 100644 --- a/tests/ToString.test.cpp +++ b/tests/ToString.test.cpp @@ -650,6 +650,19 @@ TEST_CASE_FIXTURE(Fixture, "toStringNamedFunction_overrides_param_names") CHECK_EQ("test(first: a, second: string, ...: number): a", toStringNamedFunction("test", *ftv, opts)); } +TEST_CASE_FIXTURE(Fixture, "pick_distinct_names_for_mixed_explicit_and_implicit_generics") +{ + ScopedFastFlag sff[] = { + {"LuauAlwaysQuantify", true}, + }; + + CheckResult result = check(R"( + function foo(x: a, y) end + )"); + + CHECK("(a, b) -> ()" == toString(requireType("foo"))); +} + TEST_CASE_FIXTURE(Fixture, "toStringNamedFunction_include_self_param") { ScopedFastFlag flag{"LuauDocFuncParameters", true}; @@ -685,5 +698,4 @@ TEST_CASE_FIXTURE(Fixture, "toStringNamedFunction_hide_self_param") CHECK_EQ("foo:method(arg: string): ()", toStringNamedFunction("foo:method", *ftv, opts)); } - TEST_SUITE_END(); diff --git a/tests/TypeInfer.builtins.test.cpp b/tests/TypeInfer.builtins.test.cpp index b710ea0d..aa4ca415 100644 --- a/tests/TypeInfer.builtins.test.cpp +++ b/tests/TypeInfer.builtins.test.cpp @@ -878,8 +878,6 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "dont_add_definitions_to_persistent_types") TEST_CASE_FIXTURE(BuiltinsFixture, "assert_removes_falsy_types") { ScopedFastFlag sff[]{ - {"LuauAssertStripsFalsyTypes", true}, - {"LuauDiscriminableUnions2", true}, {"LuauWidenIfSupertypeIsFree2", true}, }; @@ -899,8 +897,6 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "assert_removes_falsy_types") TEST_CASE_FIXTURE(BuiltinsFixture, "assert_removes_falsy_types2") { ScopedFastFlag sff[]{ - {"LuauAssertStripsFalsyTypes", true}, - {"LuauDiscriminableUnions2", true}, {"LuauWidenIfSupertypeIsFree2", true}, }; @@ -916,11 +912,6 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "assert_removes_falsy_types2") TEST_CASE_FIXTURE(BuiltinsFixture, "assert_removes_falsy_types_even_from_type_pack_tail_but_only_for_the_first_type") { - ScopedFastFlag sff[]{ - {"LuauAssertStripsFalsyTypes", true}, - {"LuauDiscriminableUnions2", true}, - }; - CheckResult result = check(R"( local function f(...: number?) return assert(...) @@ -933,11 +924,6 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "assert_removes_falsy_types_even_from_type_pa TEST_CASE_FIXTURE(BuiltinsFixture, "assert_returns_false_and_string_iff_it_knows_the_first_argument_cannot_be_truthy") { - ScopedFastFlag sff[]{ - {"LuauAssertStripsFalsyTypes", true}, - {"LuauDiscriminableUnions2", true}, - }; - CheckResult result = check(R"( local function f(x: nil) return assert(x, "hmm") diff --git a/tests/TypeInfer.functions.test.cpp b/tests/TypeInfer.functions.test.cpp index 14f1f703..a28ba49e 100644 --- a/tests/TypeInfer.functions.test.cpp +++ b/tests/TypeInfer.functions.test.cpp @@ -1496,8 +1496,6 @@ caused by: TEST_CASE_FIXTURE(Fixture, "strict_mode_ok_with_missing_arguments") { - ScopedFastFlag sff{"LuauAnyInIsOptionalIsOptional", true}; - CheckResult result = check(R"( local function f(x: any) end f() diff --git a/tests/TypeInfer.generics.test.cpp b/tests/TypeInfer.generics.test.cpp index de0c9391..78a5fee7 100644 --- a/tests/TypeInfer.generics.test.cpp +++ b/tests/TypeInfer.generics.test.cpp @@ -1121,4 +1121,78 @@ TEST_CASE_FIXTURE(Fixture, "substitution_with_bound_table") LUAU_REQUIRE_NO_ERRORS(result); } +TEST_CASE_FIXTURE(Fixture, "apply_type_function_nested_generics1") +{ + ScopedFastFlag sff{"LuauApplyTypeFunctionFix", true}; + + // https://github.com/Roblox/luau/issues/484 + CheckResult result = check(R"( +--!strict +type MyObject = { + getReturnValue: (cb: () -> V) -> V +} +local object: MyObject = { + getReturnValue = function(cb: () -> U): U + return cb() + end, +} + +type ComplexObject = { + id: T, + nested: MyObject +} + +local complex: ComplexObject = { + id = "Foo", + nested = object, +} + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "apply_type_function_nested_generics2") +{ + ScopedFastFlag sff{"LuauApplyTypeFunctionFix", true}; + + // https://github.com/Roblox/luau/issues/484 + CheckResult result = check(R"( +--!strict +type MyObject = { + getReturnValue: (cb: () -> V) -> V +} +type ComplexObject = { + id: T, + nested: MyObject +} + +local complex2: ComplexObject = nil + +local x = complex2.nested.getReturnValue(function(): string + return "" +end) + +local y = complex2.nested.getReturnValue(function() + return 3 +end) + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "quantify_functions_even_if_they_have_an_explicit_generic") +{ + ScopedFastFlag sff[] = { + {"LuauAlwaysQuantify", true}, + }; + + CheckResult result = check(R"( + function foo(f, x: X) + return f(x) + end + )"); + + CHECK("((X) -> (a...), X) -> (a...)" == toString(requireType("foo"))); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.intersectionTypes.test.cpp b/tests/TypeInfer.intersectionTypes.test.cpp index 41bc0c21..f75b2d11 100644 --- a/tests/TypeInfer.intersectionTypes.test.cpp +++ b/tests/TypeInfer.intersectionTypes.test.cpp @@ -177,8 +177,6 @@ TEST_CASE_FIXTURE(Fixture, "index_on_an_intersection_type_with_property_guarante TEST_CASE_FIXTURE(Fixture, "index_on_an_intersection_type_works_at_arbitrary_depth") { - ScopedFastFlag sff{"LuauDoNotTryToReduce", true}; - CheckResult result = check(R"( type A = {x: {y: {z: {thing: string}}}} type B = {x: {y: {z: {thing: string}}}} diff --git a/tests/TypeInfer.loops.test.cpp b/tests/TypeInfer.loops.test.cpp index 765419c6..a3cae3de 100644 --- a/tests/TypeInfer.loops.test.cpp +++ b/tests/TypeInfer.loops.test.cpp @@ -475,8 +475,6 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "loop_typecheck_crash_on_empty_optional") TEST_CASE_FIXTURE(Fixture, "fuzz_fail_missing_instantitation_follow") { - ScopedFastFlag luauInstantiateFollows{"LuauInstantiateFollows", true}; - // Just check that this doesn't assert check(R"( --!nonstrict diff --git a/tests/TypeInfer.operators.test.cpp b/tests/TypeInfer.operators.test.cpp index 51f6fdfb..03614938 100644 --- a/tests/TypeInfer.operators.test.cpp +++ b/tests/TypeInfer.operators.test.cpp @@ -728,8 +728,6 @@ TEST_CASE_FIXTURE(Fixture, "operator_eq_verifies_types_do_intersect") TEST_CASE_FIXTURE(Fixture, "operator_eq_operands_are_not_subtypes_of_each_other_but_has_overlap") { - ScopedFastFlag sff1{"LuauEqConstraint", true}; - CheckResult result = check(R"( local function f(a: string | number, b: boolean | number) return a == b diff --git a/tests/TypeInfer.provisional.test.cpp b/tests/TypeInfer.provisional.test.cpp index ee3ae972..9d227895 100644 --- a/tests/TypeInfer.provisional.test.cpp +++ b/tests/TypeInfer.provisional.test.cpp @@ -7,7 +7,6 @@ #include -LUAU_FASTFLAG(LuauEqConstraint) LUAU_FASTFLAG(LuauLowerBoundsCalculation) using namespace Luau; @@ -183,8 +182,6 @@ TEST_CASE_FIXTURE(Fixture, "operator_eq_completely_incompatible") // We'll need to not only report an error on `a == b`, but also to refine both operands as `never` in the `==` branch. TEST_CASE_FIXTURE(Fixture, "lvalue_equals_another_lvalue_with_no_overlap") { - ScopedFastFlag sff1{"LuauEqConstraint", true}; - CheckResult result = check(R"( local function f(a: string, b: boolean?) if a == b then @@ -208,8 +205,6 @@ TEST_CASE_FIXTURE(Fixture, "lvalue_equals_another_lvalue_with_no_overlap") // Just needs to fully support equality refinement. Which is annoying without type states. TEST_CASE_FIXTURE(Fixture, "discriminate_from_x_not_equal_to_nil") { - ScopedFastFlag sff{"LuauDiscriminableUnions2", true}; - CheckResult result = check(R"( type T = {x: string, y: number} | {x: nil, y: nil} @@ -471,4 +466,35 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "function_returns_many_things_but_first_of_it CHECK_EQ("boolean", toString(requireType("b"))); } +TEST_CASE_FIXTURE(Fixture, "constrained_is_level_dependent") +{ + ScopedFastFlag sff[]{ + {"LuauLowerBoundsCalculation", true}, + {"LuauNormalizeFlagIsConservative", true}, + }; + + CheckResult result = check(R"( + local function f(o) + local t = {} + t[o] = true + + local function foo(o) + o:m1() + t[o] = nil + end + + local function bar(o) + o:m2() + t[o] = true + end + + return t + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + // TODO: We're missing generics a... and b... + CHECK_EQ("(t1) -> {| [t1]: boolean |} where t1 = t2 ; t2 = {+ m1: (t1) -> (a...), m2: (t2) -> (b...) +}", toString(requireType("f"))); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.refinements.test.cpp b/tests/TypeInfer.refinements.test.cpp index 8c130490..85a3334c 100644 --- a/tests/TypeInfer.refinements.test.cpp +++ b/tests/TypeInfer.refinements.test.cpp @@ -7,7 +7,6 @@ #include "doctest.h" -LUAU_FASTFLAG(LuauDiscriminableUnions2) LUAU_FASTFLAG(LuauWeakEqConstraint) LUAU_FASTFLAG(LuauLowerBoundsCalculation) @@ -268,18 +267,10 @@ TEST_CASE_FIXTURE(Fixture, "typeguard_only_look_up_types_from_global_scope") end )"); - if (FFlag::LuauDiscriminableUnions2) - { - LUAU_REQUIRE_NO_ERRORS(result); + LUAU_REQUIRE_NO_ERRORS(result); - CHECK_EQ("*unknown*", toString(requireTypeAtPosition({8, 44}))); - CHECK_EQ("*unknown*", toString(requireTypeAtPosition({9, 38}))); - } - else - { - LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK_EQ("Type 'number' has no overlap with 'string'", toString(result.errors[0])); - } + CHECK_EQ("*unknown*", toString(requireTypeAtPosition({8, 44}))); + CHECK_EQ("*unknown*", toString(requireTypeAtPosition({9, 38}))); } TEST_CASE_FIXTURE(Fixture, "call_a_more_specific_function_using_typeguard") @@ -378,8 +369,6 @@ caused by: TEST_CASE_FIXTURE(Fixture, "lvalue_is_equal_to_another_lvalue") { - ScopedFastFlag sff1{"LuauEqConstraint", true}; - CheckResult result = check(R"( local function f(a: (string | number)?, b: boolean?) if a == b then @@ -392,28 +381,15 @@ TEST_CASE_FIXTURE(Fixture, "lvalue_is_equal_to_another_lvalue") LUAU_REQUIRE_NO_ERRORS(result); - if (FFlag::LuauWeakEqConstraint) - { - CHECK_EQ(toString(requireTypeAtPosition({3, 33})), "(number | string)?"); // a == b - CHECK_EQ(toString(requireTypeAtPosition({3, 36})), "boolean?"); // a == b + CHECK_EQ(toString(requireTypeAtPosition({3, 33})), "(number | string)?"); // a == b + CHECK_EQ(toString(requireTypeAtPosition({3, 36})), "boolean?"); // a == b - CHECK_EQ(toString(requireTypeAtPosition({5, 33})), "(number | string)?"); // a ~= b - CHECK_EQ(toString(requireTypeAtPosition({5, 36})), "boolean?"); // a ~= b - } - else - { - CHECK_EQ(toString(requireTypeAtPosition({3, 33})), "nil"); // a == b - CHECK_EQ(toString(requireTypeAtPosition({3, 36})), "nil"); // a == b - - CHECK_EQ(toString(requireTypeAtPosition({5, 33})), "(number | string)?"); // a ~= b - CHECK_EQ(toString(requireTypeAtPosition({5, 36})), "boolean?"); // a ~= b - } + CHECK_EQ(toString(requireTypeAtPosition({5, 33})), "(number | string)?"); // a ~= b + CHECK_EQ(toString(requireTypeAtPosition({5, 36})), "boolean?"); // a ~= b } TEST_CASE_FIXTURE(Fixture, "lvalue_is_equal_to_a_term") { - ScopedFastFlag sff1{"LuauEqConstraint", true}; - CheckResult result = check(R"( local function f(a: (string | number)?) if a == 1 then @@ -426,24 +402,12 @@ TEST_CASE_FIXTURE(Fixture, "lvalue_is_equal_to_a_term") LUAU_REQUIRE_NO_ERRORS(result); - if (FFlag::LuauWeakEqConstraint) - { - CHECK_EQ(toString(requireTypeAtPosition({3, 28})), "(number | string)?"); // a == 1 - CHECK_EQ(toString(requireTypeAtPosition({5, 28})), "(number | string)?"); // a ~= 1 - } - else - { - CHECK_EQ(toString(requireTypeAtPosition({3, 28})), "number"); // a == 1 - CHECK_EQ(toString(requireTypeAtPosition({5, 28})), "(number | string)?"); // a ~= 1 - } + CHECK_EQ(toString(requireTypeAtPosition({3, 28})), "(number | string)?"); // a == 1 + CHECK_EQ(toString(requireTypeAtPosition({5, 28})), "(number | string)?"); // a ~= 1 } TEST_CASE_FIXTURE(Fixture, "term_is_equal_to_an_lvalue") { - ScopedFastFlag sff[] = { - {"LuauDiscriminableUnions2", true}, - }; - CheckResult result = check(R"( local function f(a: (string | number)?) if "hello" == a then @@ -462,8 +426,6 @@ TEST_CASE_FIXTURE(Fixture, "term_is_equal_to_an_lvalue") TEST_CASE_FIXTURE(Fixture, "lvalue_is_not_nil") { - ScopedFastFlag sff1{"LuauEqConstraint", true}; - CheckResult result = check(R"( local function f(a: (string | number)?) if a ~= nil then @@ -476,21 +438,12 @@ TEST_CASE_FIXTURE(Fixture, "lvalue_is_not_nil") LUAU_REQUIRE_NO_ERRORS(result); - if (FFlag::LuauWeakEqConstraint) - { - CHECK_EQ(toString(requireTypeAtPosition({3, 28})), "number | string"); // a ~= nil - CHECK_EQ(toString(requireTypeAtPosition({5, 28})), "(number | string)?"); // a == nil - } - else - { - CHECK_EQ(toString(requireTypeAtPosition({3, 28})), "number | string"); // a ~= nil - CHECK_EQ(toString(requireTypeAtPosition({5, 28})), "nil"); // a == nil - } + CHECK_EQ(toString(requireTypeAtPosition({3, 28})), "number | string"); // a ~= nil + CHECK_EQ(toString(requireTypeAtPosition({5, 28})), "(number | string)?"); // a == nil } TEST_CASE_FIXTURE(Fixture, "free_type_is_equal_to_an_lvalue") { - ScopedFastFlag sff{"LuauDiscriminableUnions2", true}; ScopedFastFlag sff2{"LuauWeakEqConstraint", true}; CheckResult result = check(R"( @@ -509,8 +462,6 @@ TEST_CASE_FIXTURE(Fixture, "free_type_is_equal_to_an_lvalue") TEST_CASE_FIXTURE(Fixture, "unknown_lvalue_is_not_synonymous_with_other_on_not_equal") { - ScopedFastFlag sff1{"LuauEqConstraint", true}; - CheckResult result = check(R"( local function f(a: any, b: {x: number}?) if a ~= b then @@ -521,22 +472,12 @@ TEST_CASE_FIXTURE(Fixture, "unknown_lvalue_is_not_synonymous_with_other_on_not_e LUAU_REQUIRE_NO_ERRORS(result); - if (FFlag::LuauWeakEqConstraint) - { - CHECK_EQ(toString(requireTypeAtPosition({3, 33})), "any"); // a ~= b - CHECK_EQ(toString(requireTypeAtPosition({3, 36})), "{| x: number |}?"); // a ~= b - } - else - { - CHECK_EQ(toString(requireTypeAtPosition({3, 33})), "any"); // a ~= b - CHECK_EQ(toString(requireTypeAtPosition({3, 36})), "{| x: number |}"); // a ~= b - } + CHECK_EQ(toString(requireTypeAtPosition({3, 33})), "any"); // a ~= b + CHECK_EQ(toString(requireTypeAtPosition({3, 36})), "{| x: number |}?"); // a ~= b } TEST_CASE_FIXTURE(Fixture, "string_not_equal_to_string_or_nil") { - ScopedFastFlag sff1{"LuauEqConstraint", true}; - CheckResult result = check(R"( local t: {string} = {"hello"} @@ -554,18 +495,8 @@ TEST_CASE_FIXTURE(Fixture, "string_not_equal_to_string_or_nil") CHECK_EQ(toString(requireTypeAtPosition({6, 29})), "string"); // a ~= b CHECK_EQ(toString(requireTypeAtPosition({6, 32})), "string?"); // a ~= b - if (FFlag::LuauWeakEqConstraint) - { - CHECK_EQ(toString(requireTypeAtPosition({8, 29})), "string"); // a == b - CHECK_EQ(toString(requireTypeAtPosition({8, 32})), "string?"); // a == b - } - else - { - // This is technically not wrong, but it's also wrong at the same time. - // The refinement code is none the wiser about the fact we pulled a string out of an array, so it has no choice but to narrow as just string. - CHECK_EQ(toString(requireTypeAtPosition({8, 29})), "string"); // a == b - CHECK_EQ(toString(requireTypeAtPosition({8, 32})), "string"); // a == b - } + CHECK_EQ(toString(requireTypeAtPosition({8, 29})), "string"); // a == b + CHECK_EQ(toString(requireTypeAtPosition({8, 32})), "string?"); // a == b } TEST_CASE_FIXTURE(Fixture, "narrow_property_of_a_bounded_variable") @@ -594,16 +525,7 @@ TEST_CASE_FIXTURE(Fixture, "type_narrow_to_vector") end )"); - if (FFlag::LuauDiscriminableUnions2) - { - LUAU_REQUIRE_NO_ERRORS(result); - } - else - { - // This is kinda weird to see, but this actually only happens in Luau without Roblox type bindings because we don't have a Vector3 type. - LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK_EQ("Unknown type 'Vector3'", toString(result.errors[0])); - } + LUAU_REQUIRE_NO_ERRORS(result); CHECK_EQ("*unknown*", toString(requireTypeAtPosition({3, 28}))); } @@ -1009,10 +931,6 @@ TEST_CASE_FIXTURE(Fixture, "apply_refinements_on_astexprindexexpr_whose_subscrip TEST_CASE_FIXTURE(Fixture, "discriminate_from_truthiness_of_x") { - ScopedFastFlag sff[] = { - {"LuauDiscriminableUnions2", true}, - }; - CheckResult result = check(R"( type T = {tag: "missing", x: nil} | {tag: "exists", x: string} @@ -1033,10 +951,6 @@ TEST_CASE_FIXTURE(Fixture, "discriminate_from_truthiness_of_x") TEST_CASE_FIXTURE(Fixture, "discriminate_tag") { - ScopedFastFlag sff[] = { - {"LuauDiscriminableUnions2", true}, - }; - CheckResult result = check(R"( type Cat = {tag: "Cat", name: string, catfood: string} type Dog = {tag: "Dog", name: string, dogfood: string} @@ -1070,11 +984,6 @@ TEST_CASE_FIXTURE(Fixture, "and_or_peephole_refinement") TEST_CASE_FIXTURE(Fixture, "narrow_boolean_to_true_or_false") { - ScopedFastFlag sff[]{ - {"LuauDiscriminableUnions2", true}, - {"LuauAssertStripsFalsyTypes", true}, - }; - CheckResult result = check(R"( local function is_true(b: true) end local function is_false(b: false) end @@ -1093,11 +1002,6 @@ TEST_CASE_FIXTURE(Fixture, "narrow_boolean_to_true_or_false") TEST_CASE_FIXTURE(Fixture, "discriminate_on_properties_of_disjoint_tables_where_that_property_is_true_or_false") { - ScopedFastFlag sff[]{ - {"LuauDiscriminableUnions2", true}, - {"LuauAssertStripsFalsyTypes", true}, - }; - CheckResult result = check(R"( type Ok = { ok: true, value: T } type Err = { ok: false, error: E } @@ -1117,8 +1021,6 @@ TEST_CASE_FIXTURE(Fixture, "discriminate_on_properties_of_disjoint_tables_where_ TEST_CASE_FIXTURE(Fixture, "refine_a_property_not_to_be_nil_through_an_intersection_table") { - ScopedFastFlag sff{"LuauDoNotTryToReduce", true}; - CheckResult result = check(R"( type T = {} & {f: ((string) -> string)?} local function f(t: T, x) @@ -1133,10 +1035,6 @@ TEST_CASE_FIXTURE(Fixture, "refine_a_property_not_to_be_nil_through_an_intersect TEST_CASE_FIXTURE(RefinementClassFixture, "discriminate_from_isa_of_x") { - ScopedFastFlag sff[] = { - {"LuauDiscriminableUnions2", true}, - }; - CheckResult result = check(R"( type T = {tag: "Part", x: Part} | {tag: "Folder", x: Folder} @@ -1171,14 +1069,7 @@ TEST_CASE_FIXTURE(RefinementClassFixture, "typeguard_cast_free_table_to_vector") end )"); - if (FFlag::LuauDiscriminableUnions2) - LUAU_REQUIRE_NO_ERRORS(result); - else - { - LUAU_REQUIRE_ERROR_COUNT(1, result); - - CHECK_EQ("Type '{+ X: a, Y: b, Z: c +}' could not be converted into 'Instance'", toString(result.errors[0])); - } + LUAU_REQUIRE_NO_ERRORS(result); CHECK_EQ("Vector3", toString(requireTypeAtPosition({5, 28}))); // type(vec) == "vector" diff --git a/tests/TypeInfer.singletons.test.cpp b/tests/TypeInfer.singletons.test.cpp index 79eeb824..d90dfbb5 100644 --- a/tests/TypeInfer.singletons.test.cpp +++ b/tests/TypeInfer.singletons.test.cpp @@ -139,6 +139,8 @@ TEST_CASE_FIXTURE(Fixture, "enums_using_singletons") TEST_CASE_FIXTURE(Fixture, "enums_using_singletons_mismatch") { + ScopedFastFlag sff{"LuauTwoPassAliasDefinitionFix", true}; + CheckResult result = check(R"( type MyEnum = "foo" | "bar" | "baz" local a : MyEnum = "bang" @@ -325,8 +327,6 @@ local a: Animal = if true then { tag = 'cat', catfood = 'something' } else { tag TEST_CASE_FIXTURE(Fixture, "widen_the_supertype_if_it_is_free_and_subtype_has_singleton") { ScopedFastFlag sff[]{ - {"LuauEqConstraint", true}, - {"LuauDiscriminableUnions2", true}, {"LuauWidenIfSupertypeIsFree2", true}, {"LuauWeakEqConstraint", false}, }; @@ -350,11 +350,8 @@ TEST_CASE_FIXTURE(Fixture, "widen_the_supertype_if_it_is_free_and_subtype_has_si TEST_CASE_FIXTURE(Fixture, "return_type_of_f_is_not_widened") { ScopedFastFlag sff[]{ - {"LuauDiscriminableUnions2", true}, - {"LuauEqConstraint", true}, {"LuauWidenIfSupertypeIsFree2", true}, {"LuauWeakEqConstraint", false}, - {"LuauDoNotAccidentallyDependOnPointerOrdering", true}, }; CheckResult result = check(R"( @@ -390,7 +387,6 @@ TEST_CASE_FIXTURE(Fixture, "widening_happens_almost_everywhere") TEST_CASE_FIXTURE(Fixture, "widening_happens_almost_everywhere_except_for_tables") { ScopedFastFlag sff[]{ - {"LuauDiscriminableUnions2", true}, {"LuauWidenIfSupertypeIsFree2", true}, }; @@ -419,6 +415,7 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "table_insert_with_a_singleton_argument") { ScopedFastFlag sff[]{ {"LuauWidenIfSupertypeIsFree2", true}, + {"LuauWeakEqConstraint", true}, }; CheckResult result = check(R"( @@ -456,10 +453,6 @@ TEST_CASE_FIXTURE(Fixture, "functions_are_not_to_be_widened") TEST_CASE_FIXTURE(Fixture, "indexing_on_string_singletons") { - ScopedFastFlag sff[]{ - {"LuauDiscriminableUnions2", true}, - }; - CheckResult result = check(R"( local a: string = "hi" if a == "hi" then @@ -474,10 +467,6 @@ TEST_CASE_FIXTURE(Fixture, "indexing_on_string_singletons") TEST_CASE_FIXTURE(Fixture, "indexing_on_union_of_string_singletons") { - ScopedFastFlag sff[]{ - {"LuauDiscriminableUnions2", true}, - }; - CheckResult result = check(R"( local a: string = "hi" if a == "hi" or a == "bye" then @@ -492,10 +481,6 @@ TEST_CASE_FIXTURE(Fixture, "indexing_on_union_of_string_singletons") TEST_CASE_FIXTURE(Fixture, "taking_the_length_of_string_singleton") { - ScopedFastFlag sff[]{ - {"LuauDiscriminableUnions2", true}, - }; - CheckResult result = check(R"( local a: string = "hi" if a == "hi" then @@ -510,10 +495,6 @@ TEST_CASE_FIXTURE(Fixture, "taking_the_length_of_string_singleton") TEST_CASE_FIXTURE(Fixture, "taking_the_length_of_union_of_string_singleton") { - ScopedFastFlag sff[]{ - {"LuauDiscriminableUnions2", true}, - }; - CheckResult result = check(R"( local a: string = "hi" if a == "hi" or a == "bye" then diff --git a/tests/TypeInfer.tables.test.cpp b/tests/TypeInfer.tables.test.cpp index 5078b0bf..c924484a 100644 --- a/tests/TypeInfer.tables.test.cpp +++ b/tests/TypeInfer.tables.test.cpp @@ -2279,8 +2279,6 @@ local y = #x TEST_CASE_FIXTURE(BuiltinsFixture, "dont_hang_when_trying_to_look_up_in_cyclic_metatable_index") { - ScopedFastFlag sff{"LuauTerminateCyclicMetatableIndexLookup", true}; - // t :: t1 where t1 = {metatable {__index: t1, __tostring: (t1) -> string}} CheckResult result = check(R"( local mt = {} @@ -2313,8 +2311,6 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "give_up_after_one_metatable_index_look_up") TEST_CASE_FIXTURE(Fixture, "confusing_indexing") { - ScopedFastFlag sff{"LuauDoNotTryToReduce", true}; - CheckResult result = check(R"( type T = {} & {p: number | string} local function f(t: T) @@ -2971,8 +2967,6 @@ TEST_CASE_FIXTURE(Fixture, "inferred_return_type_of_free_table") TEST_CASE_FIXTURE(Fixture, "mixed_tables_with_implicit_numbered_keys") { - ScopedFastFlag sff{"LuauCheckImplicitNumbericKeys", true}; - CheckResult result = check(R"( local t: { [string]: number } = { 5, 6, 7 } )"); @@ -2984,4 +2978,32 @@ TEST_CASE_FIXTURE(Fixture, "mixed_tables_with_implicit_numbered_keys") CHECK_EQ("Type 'number' could not be converted into 'string'", toString(result.errors[2])); } +TEST_CASE_FIXTURE(Fixture, "expected_indexer_value_type_extra") +{ + ScopedFastFlag luauExpectedPropTypeFromIndexer{"LuauExpectedPropTypeFromIndexer", true}; + ScopedFastFlag luauSubtypingAddOptPropsToUnsealedTables{"LuauSubtypingAddOptPropsToUnsealedTables", true}; + + CheckResult result = check(R"( + type X = { { x: boolean?, y: boolean? } } + + local l1: {[string]: X} = { key = { { x = true }, { y = true } } } + local l2: {[any]: X} = { key = { { x = true }, { y = true } } } + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "expected_indexer_value_type_extra_2") +{ + ScopedFastFlag luauExpectedPropTypeFromIndexer{"LuauExpectedPropTypeFromIndexer", true}; + + CheckResult result = check(R"( + type X = {[any]: string | boolean} + + local x: X = { key = "str" } + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.test.cpp b/tests/TypeInfer.test.cpp index 48cd1c3d..1d144db7 100644 --- a/tests/TypeInfer.test.cpp +++ b/tests/TypeInfer.test.cpp @@ -15,7 +15,6 @@ LUAU_FASTFLAG(LuauLowerBoundsCalculation) LUAU_FASTFLAG(LuauFixLocationSpanTableIndexExpr) -LUAU_FASTFLAG(LuauEqConstraint) using namespace Luau; @@ -308,7 +307,6 @@ TEST_CASE_FIXTURE(Fixture, "check_type_infer_recursion_count") int limit = 600; #endif - ScopedFastFlag sff{"LuauTableUseCounterInstead", true}; ScopedFastInt sfi{"LuauCheckRecursionLimit", limit}; CheckResult result = check("function f() return " + rep("{a=", limit) + "'a'" + rep("}", limit) + " end"); @@ -1011,8 +1009,6 @@ TEST_CASE_FIXTURE(Fixture, "type_infer_recursion_limit_no_ice") TEST_CASE_FIXTURE(Fixture, "follow_on_new_types_in_substitution") { - ScopedFastFlag substituteFollowNewTypes{"LuauSubstituteFollowNewTypes", true}; - CheckResult result = check(R"( local obj = {} diff --git a/tests/TypeInfer.unionTypes.test.cpp b/tests/TypeInfer.unionTypes.test.cpp index 277f3887..d19d80cb 100644 --- a/tests/TypeInfer.unionTypes.test.cpp +++ b/tests/TypeInfer.unionTypes.test.cpp @@ -7,7 +7,6 @@ #include "doctest.h" LUAU_FASTFLAG(LuauLowerBoundsCalculation) -LUAU_FASTFLAG(LuauEqConstraint) using namespace Luau; From 70ff6b434704bd336ea33fbf219d483f5a19ecee Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Fri, 20 May 2022 13:00:53 -0700 Subject: [PATCH 254/399] Update performance.md (#494) Add a section on table length optimizations and reword the table iteration section a bit to account for generalized iteration. --- docs/_pages/performance.md | 16 +++++++++++++--- 1 file changed, 13 insertions(+), 3 deletions(-) diff --git a/docs/_pages/performance.md b/docs/_pages/performance.md index b4fd3a7b..10e2341c 100644 --- a/docs/_pages/performance.md +++ b/docs/_pages/performance.md @@ -92,12 +92,22 @@ As a result, builtin calls are very fast in Luau - they are still slightly slowe ## Optimized table iteration -Luau implements a fully generic iteration protocol; however, for iteration through tables it recognizes three common idioms (`for .. in ipairs(t)`, `for .. in pairs(t)` and `for .. in next, t`) and emits specialized bytecode that is carefully optimized using custom internal iterators. +Luau implements a fully generic iteration protocol; however, for iteration through tables in addition to generalized iteration (`for .. in t`) it recognizes three common idioms (`for .. in ipairs(t)`, `for .. in pairs(t)` and `for .. in next, t`) and emits specialized bytecode that is carefully optimized using custom internal iterators. -As a result, iteration through tables typically doesn't result in function calls for every iteration; the performance of iteration using `pairs` and `ipairs` is comparable, so it's recommended to pick the iteration style based on readability instead of performance. +As a result, iteration through tables typically doesn't result in function calls for every iteration; the performance of iteration using generalized iteration, `pairs` and `ipairs` is comparable, so generalized iteration (without the use of `pairs`/`ipairs`) is recommended unless the code needs to be compatible with vanilla Lua or the specific semantics of `ipairs` (which stops at the first `nil` element) is required. Additionally, using generalized iteration avoids calling `pairs` when the loop starts which can be noticeable when the table is very short. Iterating through array-like tables using `for i=1,#t` tends to be slightly slower because of extra cost incurred when reading elements from the table. +## Optimized table length + +Luau tables use a hybrid array/hash storage, like in Lua; in some sense "arrays" don't truly exist and are an internal optimization, but some operations, notably `#t` and functions that depend on it, like `table.insert`, are defined by the Luau/Lua language to allow internal optimizations. Luau takes advantage of that fact. + +Unlike Lua, Luau guarantees that the element at index `#t` is stored in the array part of the table. This can accelerate various table operations that use indices limited by `#t`, and this makes `#t` worst-case complexity O(logN), unlike Lua where the worst case complexity is O(N). This also accelerates computation of this value for small tables like `{ [1] = 1 }` since we never need to look at the hash part. + +The "default" implementation of `#t` in both Lua and Luau is a binary search. Luau uses a special branch-free (depending on the compiler...) implementation of the binary search which results in 50+% faster computation of table length when it needs to be computed from scratch. + +Additionally, Luau can cache the length of the table and adjust it following operations like `table.insert`/`table.remove`; this means that in practice, `#t` is almost always a constant time operation. + ## Creating and modifying tables Luau implements several optimizations for table creation. When creating object-like tables, it's recommended to use table literals (`{ ... }`) and to specify all table fields in the literal in one go instead of assigning fields later; this triggers an optimization inspired by LuaJIT's "table templates" and results in higher performance when creating objects. When creating array-like tables, if the maximum size of the table is known up front, it's recommended to use `table.create` function which can create an empty table with preallocated storage, and optionally fill it with a given value. @@ -112,7 +122,7 @@ v.z = 3 return v ``` -When appending elements to tables, it's recommended to use `table.insert` (which is the fastest method to append an element to a table if the table size is not known). In cases when a table is filled sequentially, however, it's much more efficient to use a known index for insertion - together with preallocating tables using `table.create` this can result in much faster code, for example this is the fastest way to build a table of squares: +When appending elements to tables, it's recommended to use `table.insert` (which is the fastest method to append an element to a table if the table size is not known). In cases when a table is filled sequentially, however, it can be more efficient to use a known index for insertion - together with preallocating tables using `table.create` this can result in much faster code, for example this is the fastest way to build a table of squares: ```lua local t = table.create(N) From fb9c4311d8645fbce50186b4455992ad84194846 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Petri=20H=C3=A4kkinen?= Date: Tue, 24 May 2022 18:59:12 +0300 Subject: [PATCH 255/399] Add lua_tolightuserdata, optimized lua_topointer (#496) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: Petri Häkkinen --- VM/include/lua.h | 1 + VM/src/lapi.cpp | 19 ++++++++++++------- tests/Conformance.test.cpp | 2 ++ 3 files changed, 15 insertions(+), 7 deletions(-) diff --git a/VM/include/lua.h b/VM/include/lua.h index c3ebadb1..7f9647c8 100644 --- a/VM/include/lua.h +++ b/VM/include/lua.h @@ -148,6 +148,7 @@ LUA_API const char* lua_tostringatom(lua_State* L, int idx, int* atom); LUA_API const char* lua_namecallatom(lua_State* L, int* atom); LUA_API int lua_objlen(lua_State* L, int idx); LUA_API lua_CFunction lua_tocfunction(lua_State* L, int idx); +LUA_API void* lua_tolightuserdata(lua_State* L, int idx); LUA_API void* lua_touserdata(lua_State* L, int idx); LUA_API void* lua_touserdatatagged(lua_State* L, int idx, int tag); LUA_API int lua_userdatatag(lua_State* L, int idx); diff --git a/VM/src/lapi.cpp b/VM/src/lapi.cpp index f8baefaf..df144a15 100644 --- a/VM/src/lapi.cpp +++ b/VM/src/lapi.cpp @@ -478,18 +478,22 @@ lua_CFunction lua_tocfunction(lua_State* L, int idx) return (!iscfunction(o)) ? NULL : cast_to(lua_CFunction, clvalue(o)->c.f); } +void* lua_tolightuserdata(lua_State* L, int idx) +{ + StkId o = index2addr(L, idx); + return (!ttislightuserdata(o)) ? NULL : pvalue(o); +} + void* lua_touserdata(lua_State* L, int idx) { StkId o = index2addr(L, idx); - switch (ttype(o)) - { - case LUA_TUSERDATA: + // fast-path: check userdata first since it is most likely the expected result + if (ttisuserdata(o)) return uvalue(o)->data; - case LUA_TLIGHTUSERDATA: + else if (ttislightuserdata(o)) return pvalue(o); - default: + else return NULL; - } } void* lua_touserdatatagged(lua_State* L, int idx, int tag) @@ -524,8 +528,9 @@ const void* lua_topointer(lua_State* L, int idx) case LUA_TTHREAD: return thvalue(o); case LUA_TUSERDATA: + return uvalue(o)->data; case LUA_TLIGHTUSERDATA: - return lua_touserdata(L, idx); + return pvalue(o); default: return NULL; } diff --git a/tests/Conformance.test.cpp b/tests/Conformance.test.cpp index a23ea470..4282bd78 100644 --- a/tests/Conformance.test.cpp +++ b/tests/Conformance.test.cpp @@ -1066,6 +1066,7 @@ TEST_CASE("UserdataApi") int lud; lua_pushlightuserdata(L, &lud); + CHECK(lua_tolightuserdata(L, -1) == &lud); CHECK(lua_touserdata(L, -1) == &lud); CHECK(lua_topointer(L, -1) == &lud); @@ -1073,6 +1074,7 @@ TEST_CASE("UserdataApi") int* ud1 = (int*)lua_newuserdata(L, 4); *ud1 = 42; + CHECK(lua_tolightuserdata(L, -1) == nullptr); CHECK(lua_touserdata(L, -1) == ud1); CHECK(lua_topointer(L, -1) == ud1); From 69acf5ac07fbb0517fece9cd0df75d75cd1184cb Mon Sep 17 00:00:00 2001 From: T 'Filtered' C Date: Tue, 24 May 2022 19:29:17 +0100 Subject: [PATCH 256/399] Make coroutine.status use a string literal (#500) Implements #495 C++ isn't a language im very familliar with so this might be completely wrong --- Analysis/src/EmbeddedBuiltinDefinitions.cpp | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Analysis/src/EmbeddedBuiltinDefinitions.cpp b/Analysis/src/EmbeddedBuiltinDefinitions.cpp index be3fcd7d..9a2259f1 100644 --- a/Analysis/src/EmbeddedBuiltinDefinitions.cpp +++ b/Analysis/src/EmbeddedBuiltinDefinitions.cpp @@ -143,7 +143,7 @@ declare coroutine: { create: ((A...) -> R...) -> thread, resume: (thread, A...) -> (boolean, R...), running: () -> thread, - status: (thread) -> string, + status: (thread) -> "dead" | "running" | "normal" | "suspended", -- FIXME: This technically returns a function, but we can't represent this yet. wrap: ((A...) -> R...) -> any, yield: (A...) -> R..., From e13f17e2251e1fc3db03593cd139280fc84b2aca Mon Sep 17 00:00:00 2001 From: Austin <6193474+axstin@users.noreply.github.com> Date: Tue, 24 May 2022 13:32:03 -0500 Subject: [PATCH 257/399] Fix VM inconsistency caused by userdata C TM fast paths (#497) This fixes usage of userdata C functions in xpcall handler following call stack overflow --- VM/src/ldo.cpp | 16 +++++--- VM/src/ldo.h | 1 + VM/src/lvmexecute.cpp | 2 +- tests/conformance/errors.lua | 75 ++++++++++++++++++++++++++++++++++++ 4 files changed, 87 insertions(+), 7 deletions(-) diff --git a/VM/src/ldo.cpp b/VM/src/ldo.cpp index c133a59e..c7904dde 100644 --- a/VM/src/ldo.cpp +++ b/VM/src/ldo.cpp @@ -213,6 +213,14 @@ CallInfo* luaD_growCI(lua_State* L) return ++L->ci; } +void luaD_checkCstack(lua_State *L) +{ + if (L->nCcalls == LUAI_MAXCCALLS) + luaG_runerror(L, "C stack overflow"); + else if (L->nCcalls >= (LUAI_MAXCCALLS + (LUAI_MAXCCALLS >> 3))) + luaD_throw(L, LUA_ERRERR); /* error while handling stack error */ +} + /* ** Call a function (C or Lua). The function to be called is at *func. ** The arguments are on the stack, right after the function. @@ -222,12 +230,8 @@ CallInfo* luaD_growCI(lua_State* L) void luaD_call(lua_State* L, StkId func, int nResults) { if (++L->nCcalls >= LUAI_MAXCCALLS) - { - if (L->nCcalls == LUAI_MAXCCALLS) - luaG_runerror(L, "C stack overflow"); - else if (L->nCcalls >= (LUAI_MAXCCALLS + (LUAI_MAXCCALLS >> 3))) - luaD_throw(L, LUA_ERRERR); /* error while handing stack error */ - } + luaD_checkCstack(L); + if (luau_precall(L, func, nResults) == PCRLUA) { /* is a Lua function? */ L->ci->flags |= LUA_CALLINFO_RETURN; /* luau_execute will stop after returning from the stack frame */ diff --git a/VM/src/ldo.h b/VM/src/ldo.h index 6e16e6f1..5e9472bf 100644 --- a/VM/src/ldo.h +++ b/VM/src/ldo.h @@ -49,6 +49,7 @@ LUAI_FUNC int luaD_pcall(lua_State* L, Pfunc func, void* u, ptrdiff_t oldtop, pt LUAI_FUNC void luaD_reallocCI(lua_State* L, int newsize); LUAI_FUNC void luaD_reallocstack(lua_State* L, int newsize); LUAI_FUNC void luaD_growstack(lua_State* L, int n); +LUAI_FUNC void luaD_checkCstack(lua_State* L); LUAI_FUNC l_noret luaD_throw(lua_State* L, int errcode); LUAI_FUNC int luaD_rawrunprotected(lua_State* L, Pfunc f, void* ud); diff --git a/VM/src/lvmexecute.cpp b/VM/src/lvmexecute.cpp index 9e2eb268..3c505fe5 100644 --- a/VM/src/lvmexecute.cpp +++ b/VM/src/lvmexecute.cpp @@ -181,7 +181,7 @@ LUAU_NOINLINE static void luau_callTM(lua_State* L, int nparams, int res) ++L->nCcalls; if (L->nCcalls >= LUAI_MAXCCALLS) - luaG_runerror(L, "C stack overflow"); + luaD_checkCstack(L); luaD_checkstack(L, LUA_MINSTACK); diff --git a/tests/conformance/errors.lua b/tests/conformance/errors.lua index 297cf011..d8dc9bd2 100644 --- a/tests/conformance/errors.lua +++ b/tests/conformance/errors.lua @@ -167,6 +167,81 @@ if not limitedstack then end end +-- C stack overflow +if not limitedstack then + local count = 1 + local cso = setmetatable({}, { + __index = function(self, i) + count = count + 1 + return self[i] + end, + __newindex = function(self, i, v) + count = count + 1 + self[i] = v + end, + __tostring = function(self) + count = count + 1 + return tostring(self) + end + }) + + local ehline + local function ehassert(cond) + if not cond then + ehline = debug.info(2, "l") + error() + end + end + + local userdata = newproxy(true) + getmetatable(userdata).__index = print + assert(debug.info(print, "s") == "[C]") + + local s, e = xpcall(tostring, function(e) + ehassert(string.find(e, "C stack overflow")) + print("after __tostring C stack overflow", count) -- 198: 1 resume + 1 xpcall + 198 luaB_tostring calls (which runs our __tostring successfully 197 times, erroring on the last attempt) + ehassert(count > 1) + + local ps, pe + + -- __tostring overflow (lua_call) + count = 1 + ps, pe = pcall(tostring, cso) + print("after __tostring overflow in handler", count) -- 23: xpcall error handler + pcall + 23 luaB_tostring calls + ehassert(not ps and string.find(pe, "error in error handling")) + ehassert(count > 1) + + -- __index overflow (callTMres) + count = 1 + ps, pe = pcall(function() return cso[cso] end) + print("after __index overflow in handler", count) -- 23: xpcall error handler + pcall + 23 __index calls + ehassert(not ps and string.find(pe, "error in error handling")) + ehassert(count > 1) + + -- __newindex overflow (callTM) + count = 1 + ps, pe = pcall(function() cso[cso] = "kohuke" end) + print("after __newindex overflow in handler", count) -- 23: xpcall error handler + pcall + 23 __newindex calls + ehassert(not ps and string.find(pe, "error in error handling")) + ehassert(count > 1) + + -- test various C __index invocations on userdata + ehassert(pcall(function() return userdata[userdata] end)) -- LOP_GETTABLE + ehassert(pcall(function() return userdata[1] end)) -- LOP_GETTABLEN + ehassert(pcall(function() return userdata.StringConstant end)) -- LOP_GETTABLEKS (luau_callTM) + + -- lua_resume test + local coro = coroutine.create(function() end) + ps, pe = coroutine.resume(coro) + ehassert(not ps and string.find(pe, "C stack overflow")) + + return true + end, cso) + + assert(not s) + assert(e == true, "error in xpcall eh, line " .. tostring(ehline)) +end + --[[ local i=1 while stack[i] ~= l1 do From c4e05eb7c1d1a7753a54b01f2738c48c19a026d7 Mon Sep 17 00:00:00 2001 From: Rob Blanckaert Date: Thu, 26 May 2022 13:33:48 -0700 Subject: [PATCH 258/399] Sync to upstream/release/529 --- Analysis/include/Luau/Frontend.h | 16 +- Analysis/include/Luau/Instantiation.h | 53 + Analysis/include/Luau/TypeArena.h | 2 +- Analysis/include/Luau/TypeInfer.h | 39 - Analysis/include/Luau/Unifier.h | 3 + Analysis/include/Luau/UnifierSharedState.h | 1 - Analysis/include/Luau/VisitTypeVar.h | 196 --- Analysis/src/Autocomplete.cpp | 31 +- Analysis/src/EmbeddedBuiltinDefinitions.cpp | 2 +- Analysis/src/Frontend.cpp | 108 +- Analysis/src/Instantiation.cpp | 128 ++ Analysis/src/Module.cpp | 12 +- Analysis/src/Normalize.cpp | 265 +--- Analysis/src/Quantify.cpp | 3 +- Analysis/src/ToString.cpp | 42 +- Analysis/src/TypeArena.cpp | 2 +- Analysis/src/TypeInfer.cpp | 233 +--- Analysis/src/Unifier.cpp | 172 +-- Ast/include/Luau/TimeTrace.h | 2 +- Ast/src/Parser.cpp | 13 + CMakeLists.txt | 15 +- CodeGen/include/Luau/AssemblyBuilderX64.h | 169 +++ CodeGen/include/Luau/Condition.h | 46 + CodeGen/include/Luau/Label.h | 18 + CodeGen/include/Luau/OperandX64.h | 136 +++ CodeGen/include/Luau/RegisterX64.h | 116 ++ CodeGen/src/AssemblyBuilderX64.cpp | 1003 ++++++++++++++++ {Compiler => Common}/include/Luau/Bytecode.h | 0 {Ast => Common}/include/Luau/Common.h | 0 Compiler/include/Luau/BytecodeBuilder.h | 2 +- Compiler/src/BytecodeBuilder.cpp | 124 +- Compiler/src/Compiler.cpp | 179 ++- Compiler/src/CostModel.cpp | 124 +- Compiler/src/CostModel.h | 3 + Makefile | 30 +- Sources.cmake | 25 +- VM/include/lua.h | 1 + VM/src/lapi.cpp | 24 +- VM/src/lbuiltins.cpp | 4 +- VM/src/lbytecode.h | 5 +- VM/src/lcommon.h | 6 +- VM/src/ldo.cpp | 17 +- VM/src/ldo.h | 1 + VM/src/lvmexecute.cpp | 22 +- tests/AssemblyBuilderX64.test.cpp | 410 +++++++ tests/Autocomplete.test.cpp | 22 +- tests/Compiler.test.cpp | 1129 +++++++++++------- tests/Conformance.test.cpp | 176 ++- tests/CostModel.test.cpp | 6 +- tests/JsonEncoder.test.cpp | 77 +- tests/NonstrictMode.test.cpp | 5 +- tests/Normalize.test.cpp | 36 +- tests/Parser.test.cpp | 11 + tests/RuntimeLimits.test.cpp | 2 +- tests/ToDot.test.cpp | 10 +- tests/TypeInfer.builtins.test.cpp | 14 +- tests/TypeInfer.generics.test.cpp | 2 +- tests/TypeInfer.intersectionTypes.test.cpp | 2 +- tests/TypeInfer.modules.test.cpp | 4 +- tests/TypeInfer.provisional.test.cpp | 5 +- tests/TypeInfer.refinements.test.cpp | 7 +- tests/TypeInfer.singletons.test.cpp | 27 +- tests/TypeInfer.tables.test.cpp | 37 +- tests/TypeInfer.test.cpp | 11 +- tests/TypeInfer.typePacks.cpp | 8 +- tests/TypeVar.test.cpp | 3 +- tests/VisitTypeVar.test.cpp | 2 - tests/conformance/errors.lua | 80 ++ tests/conformance/nextvar.lua | 25 + tests/conformance/userdata.lua | 45 + tools/natvis/CodeGen.natvis | 50 + tools/patchtests.py | 76 ++ 72 files changed, 3937 insertions(+), 1738 deletions(-) create mode 100644 Analysis/include/Luau/Instantiation.h create mode 100644 Analysis/src/Instantiation.cpp create mode 100644 CodeGen/include/Luau/AssemblyBuilderX64.h create mode 100644 CodeGen/include/Luau/Condition.h create mode 100644 CodeGen/include/Luau/Label.h create mode 100644 CodeGen/include/Luau/OperandX64.h create mode 100644 CodeGen/include/Luau/RegisterX64.h create mode 100644 CodeGen/src/AssemblyBuilderX64.cpp rename {Compiler => Common}/include/Luau/Bytecode.h (100%) rename {Ast => Common}/include/Luau/Common.h (100%) create mode 100644 tests/AssemblyBuilderX64.test.cpp create mode 100644 tests/conformance/userdata.lua create mode 100644 tools/natvis/CodeGen.natvis create mode 100644 tools/patchtests.py diff --git a/Analysis/include/Luau/Frontend.h b/Analysis/include/Luau/Frontend.h index 37e3cfdc..d7c9ca40 100644 --- a/Analysis/include/Luau/Frontend.h +++ b/Analysis/include/Luau/Frontend.h @@ -12,9 +12,6 @@ #include #include -LUAU_FASTFLAG(LuauSeparateTypechecks) -LUAU_FASTFLAG(LuauDirtySourceModule) - namespace Luau { @@ -60,17 +57,12 @@ struct SourceNode { bool hasDirtySourceModule() const { - LUAU_ASSERT(FFlag::LuauDirtySourceModule); - return dirtySourceModule; } bool hasDirtyModule(bool forAutocomplete) const { - if (FFlag::LuauSeparateTypechecks) - return forAutocomplete ? dirtyModuleForAutocomplete : dirtyModule; - else - return dirtyModule; + return forAutocomplete ? dirtyModuleForAutocomplete : dirtyModule; } ModuleName name; @@ -90,10 +82,6 @@ struct FrontendOptions // is complete. bool retainFullTypeGraphs = false; - // When true, we run typechecking twice, once in the regular mode, and once in strict mode - // in order to get more precise type information (e.g. for autocomplete). - bool typecheckTwice_DEPRECATED = false; - // Run typechecking only in mode required for autocomplete (strict mode in order to get more precise type information) bool forAutocomplete = false; }; @@ -171,7 +159,7 @@ struct Frontend void applyBuiltinDefinitionToEnvironment(const std::string& environmentName, const std::string& definitionName); private: - std::pair getSourceNode(CheckResult& checkResult, const ModuleName& name, bool forAutocomplete_DEPRECATED); + std::pair getSourceNode(CheckResult& checkResult, const ModuleName& name); SourceModule parse(const ModuleName& name, std::string_view src, const ParseOptions& parseOptions); bool parseGraph(std::vector& buildQueue, CheckResult& checkResult, const ModuleName& root, bool forAutocomplete); diff --git a/Analysis/include/Luau/Instantiation.h b/Analysis/include/Luau/Instantiation.h new file mode 100644 index 00000000..e05ceebe --- /dev/null +++ b/Analysis/include/Luau/Instantiation.h @@ -0,0 +1,53 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +#pragma once + +#include "Luau/Substitution.h" +#include "Luau/TypeVar.h" +#include "Luau/Unifiable.h" + +namespace Luau +{ + +struct TypeArena; +struct TxnLog; + +// A substitution which replaces generic types in a given set by free types. +struct ReplaceGenerics : Substitution +{ + ReplaceGenerics( + const TxnLog* log, TypeArena* arena, TypeLevel level, const std::vector& generics, const std::vector& genericPacks) + : Substitution(log, arena) + , level(level) + , generics(generics) + , genericPacks(genericPacks) + { + } + + TypeLevel level; + std::vector generics; + std::vector genericPacks; + bool ignoreChildren(TypeId ty) override; + bool isDirty(TypeId ty) override; + bool isDirty(TypePackId tp) override; + TypeId clean(TypeId ty) override; + TypePackId clean(TypePackId tp) override; +}; + +// A substitution which replaces generic functions by monomorphic functions +struct Instantiation : Substitution +{ + Instantiation(const TxnLog* log, TypeArena* arena, TypeLevel level) + : Substitution(log, arena) + , level(level) + { + } + + TypeLevel level; + bool ignoreChildren(TypeId ty) override; + bool isDirty(TypeId ty) override; + bool isDirty(TypePackId tp) override; + TypeId clean(TypeId ty) override; + TypePackId clean(TypePackId tp) override; +}; + +} // namespace Luau diff --git a/Analysis/include/Luau/TypeArena.h b/Analysis/include/Luau/TypeArena.h index 7c74158b..559c55c8 100644 --- a/Analysis/include/Luau/TypeArena.h +++ b/Analysis/include/Luau/TypeArena.h @@ -39,4 +39,4 @@ struct TypeArena void freeze(TypeArena& arena); void unfreeze(TypeArena& arena); -} +} // namespace Luau diff --git a/Analysis/include/Luau/TypeInfer.h b/Analysis/include/Luau/TypeInfer.h index fcaf5baa..183cc053 100644 --- a/Analysis/include/Luau/TypeInfer.h +++ b/Analysis/include/Luau/TypeInfer.h @@ -34,45 +34,6 @@ const AstStat* getFallthrough(const AstStat* node); struct UnifierOptions; struct Unifier; -// A substitution which replaces generic types in a given set by free types. -struct ReplaceGenerics : Substitution -{ - ReplaceGenerics( - const TxnLog* log, TypeArena* arena, TypeLevel level, const std::vector& generics, const std::vector& genericPacks) - : Substitution(log, arena) - , level(level) - , generics(generics) - , genericPacks(genericPacks) - { - } - - TypeLevel level; - std::vector generics; - std::vector genericPacks; - bool ignoreChildren(TypeId ty) override; - bool isDirty(TypeId ty) override; - bool isDirty(TypePackId tp) override; - TypeId clean(TypeId ty) override; - TypePackId clean(TypePackId tp) override; -}; - -// A substitution which replaces generic functions by monomorphic functions -struct Instantiation : Substitution -{ - Instantiation(const TxnLog* log, TypeArena* arena, TypeLevel level) - : Substitution(log, arena) - , level(level) - { - } - - TypeLevel level; - bool ignoreChildren(TypeId ty) override; - bool isDirty(TypeId ty) override; - bool isDirty(TypePackId tp) override; - TypeId clean(TypeId ty) override; - TypePackId clean(TypePackId tp) override; -}; - // A substitution which replaces free types by any struct Anyification : Substitution { diff --git a/Analysis/include/Luau/Unifier.h b/Analysis/include/Luau/Unifier.h index 0e24c8b0..627b52ca 100644 --- a/Analysis/include/Luau/Unifier.h +++ b/Analysis/include/Luau/Unifier.h @@ -32,6 +32,9 @@ struct Widen : Substitution TypeId clean(TypeId ty) override; TypePackId clean(TypePackId ty) override; bool ignoreChildren(TypeId ty) override; + + TypeId operator()(TypeId ty); + TypePackId operator()(TypePackId ty); }; // TODO: Use this more widely. diff --git a/Analysis/include/Luau/UnifierSharedState.h b/Analysis/include/Luau/UnifierSharedState.h index 1a0b8b76..d4315d47 100644 --- a/Analysis/include/Luau/UnifierSharedState.h +++ b/Analysis/include/Luau/UnifierSharedState.h @@ -42,7 +42,6 @@ struct UnifierSharedState InternalErrorReporter* iceHandler; - DenseHashSet seenAny{nullptr}; DenseHashMap skipCacheForType{nullptr}; DenseHashSet, TypeIdPairHash> cachedUnify{{nullptr, nullptr}}; DenseHashMap, TypeErrorData, TypeIdPairHash> cachedUnifyError{{nullptr, nullptr}}; diff --git a/Analysis/include/Luau/VisitTypeVar.h b/Analysis/include/Luau/VisitTypeVar.h index 2e98f526..f3839915 100644 --- a/Analysis/include/Luau/VisitTypeVar.h +++ b/Analysis/include/Luau/VisitTypeVar.h @@ -8,7 +8,6 @@ #include "Luau/TypePack.h" #include "Luau/TypeVar.h" -LUAU_FASTFLAG(LuauUseVisitRecursionLimit) LUAU_FASTINT(LuauVisitRecursionLimit) LUAU_FASTFLAG(LuauNormalizeFlagIsConservative) @@ -62,168 +61,6 @@ inline void unsee(DenseHashSet& seen, const void* tv) // When DenseHashSet is used for 'visitTypeVarOnce', where don't forget visited elements } -template -void visit(TypePackId tp, F& f, Set& seen); - -template -void visit(TypeId ty, F& f, Set& seen) -{ - if (visit_detail::hasSeen(seen, ty)) - { - f.cycle(ty); - return; - } - - if (auto btv = get(ty)) - { - if (apply(ty, *btv, seen, f)) - visit(btv->boundTo, f, seen); - } - - else if (auto ftv = get(ty)) - apply(ty, *ftv, seen, f); - - else if (auto gtv = get(ty)) - apply(ty, *gtv, seen, f); - - else if (auto etv = get(ty)) - apply(ty, *etv, seen, f); - - else if (auto ctv = get(ty)) - { - if (apply(ty, *ctv, seen, f)) - { - for (TypeId part : ctv->parts) - visit(part, f, seen); - } - } - - else if (auto ptv = get(ty)) - apply(ty, *ptv, seen, f); - - else if (auto ftv = get(ty)) - { - if (apply(ty, *ftv, seen, f)) - { - visit(ftv->argTypes, f, seen); - visit(ftv->retType, f, seen); - } - } - - else if (auto ttv = get(ty)) - { - // Some visitors want to see bound tables, that's why we visit the original type - if (apply(ty, *ttv, seen, f)) - { - if (ttv->boundTo) - { - visit(*ttv->boundTo, f, seen); - } - else - { - for (auto& [_name, prop] : ttv->props) - visit(prop.type, f, seen); - - if (ttv->indexer) - { - visit(ttv->indexer->indexType, f, seen); - visit(ttv->indexer->indexResultType, f, seen); - } - } - } - } - - else if (auto mtv = get(ty)) - { - if (apply(ty, *mtv, seen, f)) - { - visit(mtv->table, f, seen); - visit(mtv->metatable, f, seen); - } - } - - else if (auto ctv = get(ty)) - { - if (apply(ty, *ctv, seen, f)) - { - for (const auto& [name, prop] : ctv->props) - visit(prop.type, f, seen); - - if (ctv->parent) - visit(*ctv->parent, f, seen); - - if (ctv->metatable) - visit(*ctv->metatable, f, seen); - } - } - - else if (auto atv = get(ty)) - apply(ty, *atv, seen, f); - - else if (auto utv = get(ty)) - { - if (apply(ty, *utv, seen, f)) - { - for (TypeId optTy : utv->options) - visit(optTy, f, seen); - } - } - - else if (auto itv = get(ty)) - { - if (apply(ty, *itv, seen, f)) - { - for (TypeId partTy : itv->parts) - visit(partTy, f, seen); - } - } - - visit_detail::unsee(seen, ty); -} - -template -void visit(TypePackId tp, F& f, Set& seen) -{ - if (visit_detail::hasSeen(seen, tp)) - { - f.cycle(tp); - return; - } - - if (auto btv = get(tp)) - { - if (apply(tp, *btv, seen, f)) - visit(btv->boundTo, f, seen); - } - - else if (auto ftv = get(tp)) - apply(tp, *ftv, seen, f); - - else if (auto gtv = get(tp)) - apply(tp, *gtv, seen, f); - - else if (auto etv = get(tp)) - apply(tp, *etv, seen, f); - - else if (auto pack = get(tp)) - { - apply(tp, *pack, seen, f); - - for (TypeId ty : pack->head) - visit(ty, f, seen); - - if (pack->tail) - visit(*pack->tail, f, seen); - } - else if (auto pack = get(tp)) - { - apply(tp, *pack, seen, f); - visit(pack->ty, f, seen); - } - - visit_detail::unsee(seen, tp); -} - } // namespace visit_detail template @@ -513,37 +350,4 @@ struct TypeVarOnceVisitor : GenericTypeVarVisitor> } }; -// Clip with FFlagLuauUseVisitRecursionLimit -template -void DEPRECATED_visitTypeVar(TID ty, F& f, std::unordered_set& seen) -{ - visit_detail::visit(ty, f, seen); -} - -// Delete and inline when clipping FFlagLuauUseVisitRecursionLimit -template -void DEPRECATED_visitTypeVar(TID ty, F& f) -{ - if (FFlag::LuauUseVisitRecursionLimit) - f.traverse(ty); - else - { - std::unordered_set seen; - visit_detail::visit(ty, f, seen); - } -} - -// Delete and inline when clipping FFlagLuauUseVisitRecursionLimit -template -void DEPRECATED_visitTypeVarOnce(TID ty, F& f, DenseHashSet& seen) -{ - if (FFlag::LuauUseVisitRecursionLimit) - f.traverse(ty); - else - { - seen.clear(); - visit_detail::visit(ty, f, seen); - } -} - } // namespace Luau diff --git a/Analysis/src/Autocomplete.cpp b/Analysis/src/Autocomplete.cpp index 19d06cfc..b988ed35 100644 --- a/Analysis/src/Autocomplete.cpp +++ b/Analysis/src/Autocomplete.cpp @@ -1700,31 +1700,18 @@ static AutocompleteResult autocomplete(const SourceModule& sourceModule, const M AutocompleteResult autocomplete(Frontend& frontend, const ModuleName& moduleName, Position position, StringCompletionCallback callback) { - if (FFlag::LuauSeparateTypechecks) - { - // FIXME: We can improve performance here by parsing without checking. - // The old type graph is probably fine. (famous last words!) - FrontendOptions opts; - opts.forAutocomplete = true; - frontend.check(moduleName, opts); - } - else - { - // FIXME: We can improve performance here by parsing without checking. - // The old type graph is probably fine. (famous last words!) - // FIXME: We don't need to typecheck for script analysis here, just for autocomplete. - frontend.check(moduleName); - } + // FIXME: We can improve performance here by parsing without checking. + // The old type graph is probably fine. (famous last words!) + FrontendOptions opts; + opts.forAutocomplete = true; + frontend.check(moduleName, opts); const SourceModule* sourceModule = frontend.getSourceModule(moduleName); if (!sourceModule) return {}; - TypeChecker& typeChecker = - (frontend.options.typecheckTwice_DEPRECATED || FFlag::LuauSeparateTypechecks ? frontend.typeCheckerForAutocomplete : frontend.typeChecker); - ModulePtr module = - (frontend.options.typecheckTwice_DEPRECATED || FFlag::LuauSeparateTypechecks ? frontend.moduleResolverForAutocomplete.getModule(moduleName) - : frontend.moduleResolver.getModule(moduleName)); + TypeChecker& typeChecker = frontend.typeCheckerForAutocomplete; + ModulePtr module = frontend.moduleResolverForAutocomplete.getModule(moduleName); if (!module) return {}; @@ -1752,9 +1739,7 @@ OwningAutocompleteResult autocompleteSource(Frontend& frontend, std::string_view sourceModule->mode = Mode::Strict; sourceModule->commentLocations = std::move(result.commentLocations); - TypeChecker& typeChecker = - (frontend.options.typecheckTwice_DEPRECATED || FFlag::LuauSeparateTypechecks ? frontend.typeCheckerForAutocomplete : frontend.typeChecker); - + TypeChecker& typeChecker = frontend.typeCheckerForAutocomplete; ModulePtr module = typeChecker.check(*sourceModule, Mode::Strict); OwningAutocompleteResult autocompleteResult = { diff --git a/Analysis/src/EmbeddedBuiltinDefinitions.cpp b/Analysis/src/EmbeddedBuiltinDefinitions.cpp index be3fcd7d..9a2259f1 100644 --- a/Analysis/src/EmbeddedBuiltinDefinitions.cpp +++ b/Analysis/src/EmbeddedBuiltinDefinitions.cpp @@ -143,7 +143,7 @@ declare coroutine: { create: ((A...) -> R...) -> thread, resume: (thread, A...) -> (boolean, R...), running: () -> thread, - status: (thread) -> string, + status: (thread) -> "dead" | "running" | "normal" | "suspended", -- FIXME: This technically returns a function, but we can't represent this yet. wrap: ((A...) -> R...) -> any, yield: (A...) -> R..., diff --git a/Analysis/src/Frontend.cpp b/Analysis/src/Frontend.cpp index 56c0ac2c..1d33f131 100644 --- a/Analysis/src/Frontend.cpp +++ b/Analysis/src/Frontend.cpp @@ -20,9 +20,7 @@ LUAU_FASTINT(LuauTypeInferIterationLimit) LUAU_FASTINT(LuauTarjanChildLimit) LUAU_FASTFLAG(LuauInferInNoCheckMode) LUAU_FASTFLAGVARIABLE(LuauKnowsTheDataModel3, false) -LUAU_FASTFLAGVARIABLE(LuauSeparateTypechecks, false) LUAU_FASTFLAGVARIABLE(LuauAutocompleteDynamicLimits, false) -LUAU_FASTFLAGVARIABLE(LuauDirtySourceModule, false) LUAU_FASTINTVARIABLE(LuauAutocompleteCheckTimeoutMs, 100) namespace Luau @@ -361,32 +359,21 @@ CheckResult Frontend::check(const ModuleName& name, std::optionalsecond.hasDirtyModule(frontendOptions.forAutocomplete)) { // No recheck required. - if (FFlag::LuauSeparateTypechecks) + if (frontendOptions.forAutocomplete) { - if (frontendOptions.forAutocomplete) - { - auto it2 = moduleResolverForAutocomplete.modules.find(name); - if (it2 == moduleResolverForAutocomplete.modules.end() || it2->second == nullptr) - throw std::runtime_error("Frontend::modules does not have data for " + name); - } - else - { - auto it2 = moduleResolver.modules.find(name); - if (it2 == moduleResolver.modules.end() || it2->second == nullptr) - throw std::runtime_error("Frontend::modules does not have data for " + name); - } - - return CheckResult{accumulateErrors( - sourceNodes, frontendOptions.forAutocomplete ? moduleResolverForAutocomplete.modules : moduleResolver.modules, name)}; + auto it2 = moduleResolverForAutocomplete.modules.find(name); + if (it2 == moduleResolverForAutocomplete.modules.end() || it2->second == nullptr) + throw std::runtime_error("Frontend::modules does not have data for " + name); } else { auto it2 = moduleResolver.modules.find(name); if (it2 == moduleResolver.modules.end() || it2->second == nullptr) throw std::runtime_error("Frontend::modules does not have data for " + name); - - return CheckResult{accumulateErrors(sourceNodes, moduleResolver.modules, name)}; } + + return CheckResult{ + accumulateErrors(sourceNodes, frontendOptions.forAutocomplete ? moduleResolverForAutocomplete.modules : moduleResolver.modules, name)}; } std::vector buildQueue; @@ -428,7 +415,7 @@ CheckResult Frontend::check(const ModuleName& name, std::optional& buildQueue, CheckResult& chec bool cyclic = false; { - auto [sourceNode, _] = getSourceNode(checkResult, root, forAutocomplete); + auto [sourceNode, _] = getSourceNode(checkResult, root); if (sourceNode) stack.push_back(sourceNode); } @@ -627,7 +603,7 @@ bool Frontend::parseGraph(std::vector& buildQueue, CheckResult& chec } } - auto [sourceNode, _] = getSourceNode(checkResult, dep, forAutocomplete); + auto [sourceNode, _] = getSourceNode(checkResult, dep); if (sourceNode) { stack.push_back(sourceNode); @@ -671,7 +647,7 @@ LintResult Frontend::lint(const ModuleName& name, std::optional* markedDirty) { - if (FFlag::LuauSeparateTypechecks) - { - if (!moduleResolver.modules.count(name) && !moduleResolverForAutocomplete.modules.count(name)) - return; - } - else - { - if (!moduleResolver.modules.count(name)) - return; - } + if (!moduleResolver.modules.count(name) && !moduleResolverForAutocomplete.modules.count(name)) + return; std::unordered_map> reverseDeps; for (const auto& module : sourceNodes) @@ -783,32 +751,12 @@ void Frontend::markDirty(const ModuleName& name, std::vector* marked if (markedDirty) markedDirty->push_back(next); - if (FFlag::LuauDirtySourceModule) - { - LUAU_ASSERT(FFlag::LuauSeparateTypechecks); + if (sourceNode.dirtySourceModule && sourceNode.dirtyModule && sourceNode.dirtyModuleForAutocomplete) + continue; - if (sourceNode.dirtySourceModule && sourceNode.dirtyModule && sourceNode.dirtyModuleForAutocomplete) - continue; - - sourceNode.dirtySourceModule = true; - sourceNode.dirtyModule = true; - sourceNode.dirtyModuleForAutocomplete = true; - } - else if (FFlag::LuauSeparateTypechecks) - { - if (sourceNode.dirtyModule && sourceNode.dirtyModuleForAutocomplete) - continue; - - sourceNode.dirtyModule = true; - sourceNode.dirtyModuleForAutocomplete = true; - } - else - { - if (sourceNode.dirtyModule) - continue; - - sourceNode.dirtyModule = true; - } + sourceNode.dirtySourceModule = true; + sourceNode.dirtyModule = true; + sourceNode.dirtyModuleForAutocomplete = true; if (0 == reverseDeps.count(name)) continue; @@ -835,14 +783,13 @@ const SourceModule* Frontend::getSourceModule(const ModuleName& moduleName) cons } // Read AST into sourceModules if necessary. Trace require()s. Report parse errors. -std::pair Frontend::getSourceNode(CheckResult& checkResult, const ModuleName& name, bool forAutocomplete_DEPRECATED) +std::pair Frontend::getSourceNode(CheckResult& checkResult, const ModuleName& name) { LUAU_TIMETRACE_SCOPE("Frontend::getSourceNode", "Frontend"); LUAU_TIMETRACE_ARGUMENT("name", name.c_str()); auto it = sourceNodes.find(name); - if (it != sourceNodes.end() && - (FFlag::LuauDirtySourceModule ? !it->second.hasDirtySourceModule() : !it->second.hasDirtyModule(forAutocomplete_DEPRECATED))) + if (it != sourceNodes.end() && !it->second.hasDirtySourceModule()) { auto moduleIt = sourceModules.find(name); if (moduleIt != sourceModules.end()) @@ -885,21 +832,12 @@ std::pair Frontend::getSourceNode(CheckResult& check sourceNode.name = name; sourceNode.requires.clear(); sourceNode.requireLocations.clear(); + sourceNode.dirtySourceModule = false; - if (FFlag::LuauDirtySourceModule) - sourceNode.dirtySourceModule = false; - - if (FFlag::LuauSeparateTypechecks) - { - if (it == sourceNodes.end()) - { - sourceNode.dirtyModule = true; - sourceNode.dirtyModuleForAutocomplete = true; - } - } - else + if (it == sourceNodes.end()) { sourceNode.dirtyModule = true; + sourceNode.dirtyModuleForAutocomplete = true; } for (const auto& [moduleName, location] : requireTrace.requires) diff --git a/Analysis/src/Instantiation.cpp b/Analysis/src/Instantiation.cpp new file mode 100644 index 00000000..4a12027d --- /dev/null +++ b/Analysis/src/Instantiation.cpp @@ -0,0 +1,128 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +#include "Luau/Common.h" +#include "Luau/Instantiation.h" +#include "Luau/TxnLog.h" +#include "Luau/TypeArena.h" + +LUAU_FASTFLAG(LuauNoMethodLocations) + +namespace Luau +{ + +bool Instantiation::isDirty(TypeId ty) +{ + if (const FunctionTypeVar* ftv = log->getMutable(ty)) + { + if (ftv->hasNoGenerics) + return false; + + return true; + } + else + { + return false; + } +} + +bool Instantiation::isDirty(TypePackId tp) +{ + return false; +} + +bool Instantiation::ignoreChildren(TypeId ty) +{ + if (log->getMutable(ty)) + return true; + else + return false; +} + +TypeId Instantiation::clean(TypeId ty) +{ + const FunctionTypeVar* ftv = log->getMutable(ty); + LUAU_ASSERT(ftv); + + FunctionTypeVar clone = FunctionTypeVar{level, ftv->argTypes, ftv->retType, ftv->definition, ftv->hasSelf}; + clone.magicFunction = ftv->magicFunction; + clone.tags = ftv->tags; + clone.argNames = ftv->argNames; + TypeId result = addType(std::move(clone)); + + // Annoyingly, we have to do this even if there are no generics, + // to replace any generic tables. + ReplaceGenerics replaceGenerics{log, arena, level, ftv->generics, ftv->genericPacks}; + + // TODO: What to do if this returns nullopt? + // We don't have access to the error-reporting machinery + result = replaceGenerics.substitute(result).value_or(result); + + asMutable(result)->documentationSymbol = ty->documentationSymbol; + return result; +} + +TypePackId Instantiation::clean(TypePackId tp) +{ + LUAU_ASSERT(false); + return tp; +} + +bool ReplaceGenerics::ignoreChildren(TypeId ty) +{ + if (const FunctionTypeVar* ftv = log->getMutable(ty)) + { + if (ftv->hasNoGenerics) + return true; + + // We aren't recursing in the case of a generic function which + // binds the same generics. This can happen if, for example, there's recursive types. + // If T = (a,T)->T then instantiating T should produce T' = (X,T)->T not T' = (X,T')->T'. + // It's OK to use vector equality here, since we always generate fresh generics + // whenever we quantify, so the vectors overlap if and only if they are equal. + return (!generics.empty() || !genericPacks.empty()) && (ftv->generics == generics) && (ftv->genericPacks == genericPacks); + } + else + { + return false; + } +} + +bool ReplaceGenerics::isDirty(TypeId ty) +{ + if (const TableTypeVar* ttv = log->getMutable(ty)) + return ttv->state == TableState::Generic; + else if (log->getMutable(ty)) + return std::find(generics.begin(), generics.end(), ty) != generics.end(); + else + return false; +} + +bool ReplaceGenerics::isDirty(TypePackId tp) +{ + if (log->getMutable(tp)) + return std::find(genericPacks.begin(), genericPacks.end(), tp) != genericPacks.end(); + else + return false; +} + +TypeId ReplaceGenerics::clean(TypeId ty) +{ + LUAU_ASSERT(isDirty(ty)); + if (const TableTypeVar* ttv = log->getMutable(ty)) + { + TableTypeVar clone = TableTypeVar{ttv->props, ttv->indexer, level, TableState::Free}; + if (!FFlag::LuauNoMethodLocations) + clone.methodDefinitionLocations = ttv->methodDefinitionLocations; + clone.definitionModuleName = ttv->definitionModuleName; + return addType(std::move(clone)); + } + else + return addType(FreeTypeVar{level}); +} + +TypePackId ReplaceGenerics::clean(TypePackId tp) +{ + LUAU_ASSERT(isDirty(tp)); + return addTypePack(TypePackVar(FreeTypePack{level})); +} + +} // namespace Luau diff --git a/Analysis/src/Module.cpp b/Analysis/src/Module.cpp index 074a41e6..6591d60a 100644 --- a/Analysis/src/Module.cpp +++ b/Analysis/src/Module.cpp @@ -56,8 +56,18 @@ bool isWithinComment(const SourceModule& sourceModule, Position pos) struct ForceNormal : TypeVarOnceVisitor { + const TypeArena* typeArena = nullptr; + + ForceNormal(const TypeArena* typeArena) + : typeArena(typeArena) + { + } + bool visit(TypeId ty) override { + if (ty->owningArena != typeArena) + return false; + asMutable(ty)->normal = true; return true; } @@ -100,7 +110,7 @@ void Module::clonePublicInterface(InternalErrorReporter& ice) normalize(*moduleScope->varargPack, interfaceTypes, ice); } - ForceNormal forceNormal; + ForceNormal forceNormal{&interfaceTypes}; for (auto& [name, tf] : moduleScope->exportedTypeBindings) { diff --git a/Analysis/src/Normalize.cpp b/Analysis/src/Normalize.cpp index 30fd4af2..fb31df1e 100644 --- a/Analysis/src/Normalize.cpp +++ b/Analysis/src/Normalize.cpp @@ -15,6 +15,7 @@ LUAU_FASTFLAGVARIABLE(DebugLuauCopyBeforeNormalizing, false) LUAU_FASTINTVARIABLE(LuauNormalizeIterationLimit, 1200); LUAU_FASTFLAGVARIABLE(LuauNormalizeCombineTableFix, false); LUAU_FASTFLAGVARIABLE(LuauNormalizeFlagIsConservative, false); +LUAU_FASTFLAGVARIABLE(LuauNormalizeCombineEqFix, false); namespace Luau { @@ -325,245 +326,6 @@ struct Normalize final : TypeVarVisitor int iterationLimit = 0; bool limitExceeded = false; - // TODO: Clip with FFlag::LuauUseVisitRecursionLimit - bool operator()(TypeId ty, const BoundTypeVar& btv, std::unordered_set& seen) - { - // A type could be considered normal when it is in the stack, but we will eventually find out it is not normal as normalization progresses. - // So we need to avoid eagerly saying that this bound type is normal if the thing it is bound to is in the stack. - if (seen.find(asMutable(btv.boundTo)) != seen.end()) - return false; - - // It should never be the case that this TypeVar is normal, but is bound to a non-normal type, except in nontrivial cases. - LUAU_ASSERT(!ty->normal || ty->normal == btv.boundTo->normal); - - asMutable(ty)->normal = btv.boundTo->normal; - return !ty->normal; - } - - bool operator()(TypeId ty, const FreeTypeVar& ftv) - { - return visit(ty, ftv); - } - bool operator()(TypeId ty, const PrimitiveTypeVar& ptv) - { - return visit(ty, ptv); - } - bool operator()(TypeId ty, const GenericTypeVar& gtv) - { - return visit(ty, gtv); - } - bool operator()(TypeId ty, const ErrorTypeVar& etv) - { - return visit(ty, etv); - } - bool operator()(TypeId ty, const ConstrainedTypeVar& ctvRef, std::unordered_set& seen) - { - CHECK_ITERATION_LIMIT(false); - - ConstrainedTypeVar* ctv = const_cast(&ctvRef); - - std::vector parts = std::move(ctv->parts); - - // We might transmute, so it's not safe to rely on the builtin traversal logic of visitTypeVar - for (TypeId part : parts) - visit_detail::visit(part, *this, seen); - - std::vector newParts = normalizeUnion(parts); - - const bool normal = areNormal(newParts, seen, ice); - - if (newParts.size() == 1) - *asMutable(ty) = BoundTypeVar{newParts[0]}; - else - *asMutable(ty) = UnionTypeVar{std::move(newParts)}; - - asMutable(ty)->normal = normal; - - return false; - } - - bool operator()(TypeId ty, const FunctionTypeVar& ftv, std::unordered_set& seen) - { - CHECK_ITERATION_LIMIT(false); - - if (ty->normal) - return false; - - visit_detail::visit(ftv.argTypes, *this, seen); - visit_detail::visit(ftv.retType, *this, seen); - - asMutable(ty)->normal = areNormal(ftv.argTypes, seen, ice) && areNormal(ftv.retType, seen, ice); - - return false; - } - - bool operator()(TypeId ty, const TableTypeVar& ttv, std::unordered_set& seen) - { - CHECK_ITERATION_LIMIT(false); - - if (ty->normal) - return false; - - bool normal = true; - - auto checkNormal = [&](TypeId t) { - // if t is on the stack, it is possible that this type is normal. - // If t is not normal and it is not on the stack, this type is definitely not normal. - if (!t->normal && seen.find(asMutable(t)) == seen.end()) - normal = false; - }; - - if (ttv.boundTo) - { - visit_detail::visit(*ttv.boundTo, *this, seen); - asMutable(ty)->normal = (*ttv.boundTo)->normal; - return false; - } - - for (const auto& [_name, prop] : ttv.props) - { - visit_detail::visit(prop.type, *this, seen); - checkNormal(prop.type); - } - - if (ttv.indexer) - { - visit_detail::visit(ttv.indexer->indexType, *this, seen); - checkNormal(ttv.indexer->indexType); - visit_detail::visit(ttv.indexer->indexResultType, *this, seen); - checkNormal(ttv.indexer->indexResultType); - } - - asMutable(ty)->normal = normal; - - return false; - } - - bool operator()(TypeId ty, const MetatableTypeVar& mtv, std::unordered_set& seen) - { - CHECK_ITERATION_LIMIT(false); - - if (ty->normal) - return false; - - visit_detail::visit(mtv.table, *this, seen); - visit_detail::visit(mtv.metatable, *this, seen); - - asMutable(ty)->normal = mtv.table->normal && mtv.metatable->normal; - - return false; - } - - bool operator()(TypeId ty, const ClassTypeVar& ctv) - { - return visit(ty, ctv); - } - bool operator()(TypeId ty, const AnyTypeVar& atv) - { - return visit(ty, atv); - } - bool operator()(TypeId ty, const UnionTypeVar& utvRef, std::unordered_set& seen) - { - CHECK_ITERATION_LIMIT(false); - - if (ty->normal) - return false; - - UnionTypeVar* utv = &const_cast(utvRef); - std::vector options = std::move(utv->options); - - // We might transmute, so it's not safe to rely on the builtin traversal logic of visitTypeVar - for (TypeId option : options) - visit_detail::visit(option, *this, seen); - - std::vector newOptions = normalizeUnion(options); - - const bool normal = areNormal(newOptions, seen, ice); - - LUAU_ASSERT(!newOptions.empty()); - - if (newOptions.size() == 1) - *asMutable(ty) = BoundTypeVar{newOptions[0]}; - else - utv->options = std::move(newOptions); - - asMutable(ty)->normal = normal; - - return false; - } - - bool operator()(TypeId ty, const IntersectionTypeVar& itvRef, std::unordered_set& seen) - { - CHECK_ITERATION_LIMIT(false); - - if (ty->normal) - return false; - - IntersectionTypeVar* itv = &const_cast(itvRef); - - std::vector oldParts = std::move(itv->parts); - - for (TypeId part : oldParts) - visit_detail::visit(part, *this, seen); - - std::vector tables; - for (TypeId part : oldParts) - { - part = follow(part); - if (get(part)) - tables.push_back(part); - else - { - Replacer replacer{&arena, nullptr, nullptr}; // FIXME this is super super WEIRD - combineIntoIntersection(replacer, itv, part); - } - } - - // Don't allocate a new table if there's just one in the intersection. - if (tables.size() == 1) - itv->parts.push_back(tables[0]); - else if (!tables.empty()) - { - const TableTypeVar* first = get(tables[0]); - LUAU_ASSERT(first); - - TypeId newTable = arena.addType(TableTypeVar{first->state, first->level}); - TableTypeVar* ttv = getMutable(newTable); - for (TypeId part : tables) - { - // Intuition: If combineIntoTable() needs to clone a table, any references to 'part' are cyclic and need - // to be rewritten to point at 'newTable' in the clone. - Replacer replacer{&arena, part, newTable}; - combineIntoTable(replacer, ttv, part); - } - - itv->parts.push_back(newTable); - } - - asMutable(ty)->normal = areNormal(itv->parts, seen, ice); - - if (itv->parts.size() == 1) - { - TypeId part = itv->parts[0]; - *asMutable(ty) = BoundTypeVar{part}; - } - - return false; - } - - // TODO: Clip with FFlag::LuauUseVisitRecursionLimit - template - bool operator()(TypePackId, const T&) - { - return true; - } - - // TODO: Clip with FFlag::LuauUseVisitRecursionLimit - template - void cycle(TID) - { - } - bool visit(TypeId ty, const FreeTypeVar&) override { LUAU_ASSERT(!ty->normal); @@ -968,6 +730,9 @@ struct Normalize final : TypeVarVisitor */ TypeId combine(Replacer& replacer, TypeId a, TypeId b) { + if (FFlag::LuauNormalizeCombineEqFix) + b = follow(b); + if (FFlag::LuauNormalizeCombineTableFix && a == b) return a; @@ -986,7 +751,7 @@ struct Normalize final : TypeVarVisitor } else if (auto ttv = getMutable(a)) { - if (FFlag::LuauNormalizeCombineTableFix && !get(follow(b))) + if (FFlag::LuauNormalizeCombineTableFix && !get(FFlag::LuauNormalizeCombineEqFix ? b : follow(b))) return arena.addType(IntersectionTypeVar{{a, b}}); combineIntoTable(replacer, ttv, b); return a; @@ -1009,15 +774,7 @@ std::pair normalize(TypeId ty, TypeArena& arena, InternalErrorRepo (void)clone(ty, arena, state); Normalize n{arena, ice}; - if (FFlag::LuauNormalizeFlagIsConservative) - { - DEPRECATED_visitTypeVar(ty, n); - } - else - { - std::unordered_set seen; - DEPRECATED_visitTypeVar(ty, n, seen); - } + n.traverse(ty); return {ty, !n.limitExceeded}; } @@ -1041,15 +798,7 @@ std::pair normalize(TypePackId tp, TypeArena& arena, InternalE (void)clone(tp, arena, state); Normalize n{arena, ice}; - if (FFlag::LuauNormalizeFlagIsConservative) - { - DEPRECATED_visitTypeVar(tp, n); - } - else - { - std::unordered_set seen; - DEPRECATED_visitTypeVar(tp, n, seen); - } + n.traverse(tp); return {tp, !n.limitExceeded}; } diff --git a/Analysis/src/Quantify.cpp b/Analysis/src/Quantify.cpp index 018d5632..c0f677d7 100644 --- a/Analysis/src/Quantify.cpp +++ b/Analysis/src/Quantify.cpp @@ -119,8 +119,7 @@ struct Quantifier final : TypeVarOnceVisitor void quantify(TypeId ty, TypeLevel level) { Quantifier q{level}; - DenseHashSet seen{nullptr}; - DEPRECATED_visitTypeVarOnce(ty, q, seen); + q.traverse(ty); FunctionTypeVar* ftv = getMutable(ty); LUAU_ASSERT(ftv); diff --git a/Analysis/src/ToString.cpp b/Analysis/src/ToString.cpp index f90f7019..a4a3ec49 100644 --- a/Analysis/src/ToString.cpp +++ b/Analysis/src/ToString.cpp @@ -48,46 +48,6 @@ struct FindCyclicTypes final : TypeVarVisitor cycleTPs.insert(tp); } - // TODO: Clip all the operator()s when we clip FFlagLuauUseVisitRecursionLimit - - template - bool operator()(TypeId ty, const T&) - { - return visit(ty); - } - - bool operator()(TypeId ty, const TableTypeVar& ttv) = delete; - - bool operator()(TypeId ty, const TableTypeVar& ttv, std::unordered_set& seen) - { - if (!visited.insert(ty).second) - return false; - - if (ttv.name || ttv.syntheticName) - { - for (TypeId itp : ttv.instantiatedTypeParams) - DEPRECATED_visitTypeVar(itp, *this, seen); - - for (TypePackId itp : ttv.instantiatedTypePackParams) - DEPRECATED_visitTypeVar(itp, *this, seen); - - return exhaustive; - } - - return true; - } - - bool operator()(TypeId, const ClassTypeVar&) - { - return false; - } - - template - bool operator()(TypePackId tp, const T&) - { - return visit(tp); - } - bool visit(TypeId ty) override { return visited.insert(ty).second; @@ -128,7 +88,7 @@ void findCyclicTypes(std::set& cycles, std::set& cycleTPs, T { FindCyclicTypes fct; fct.exhaustive = exhaustive; - DEPRECATED_visitTypeVar(ty, fct); + fct.traverse(ty); cycles = std::move(fct.cycles); cycleTPs = std::move(fct.cycleTPs); diff --git a/Analysis/src/TypeArena.cpp b/Analysis/src/TypeArena.cpp index 673b002d..0c89d130 100644 --- a/Analysis/src/TypeArena.cpp +++ b/Analysis/src/TypeArena.cpp @@ -85,4 +85,4 @@ void unfreeze(TypeArena& arena) arena.typePacks.unfreeze(); } -} +} // namespace Luau diff --git a/Analysis/src/TypeInfer.cpp b/Analysis/src/TypeInfer.cpp index 208b3f2f..11813c76 100644 --- a/Analysis/src/TypeInfer.cpp +++ b/Analysis/src/TypeInfer.cpp @@ -3,6 +3,7 @@ #include "Luau/Clone.h" #include "Luau/Common.h" +#include "Luau/Instantiation.h" #include "Luau/ModuleResolver.h" #include "Luau/Normalize.h" #include "Luau/Parser.h" @@ -10,13 +11,13 @@ #include "Luau/RecursionCounter.h" #include "Luau/Scope.h" #include "Luau/Substitution.h" -#include "Luau/TopoSortStatements.h" -#include "Luau/TypePack.h" -#include "Luau/ToString.h" -#include "Luau/TypeUtils.h" -#include "Luau/ToString.h" -#include "Luau/TypeVar.h" #include "Luau/TimeTrace.h" +#include "Luau/TopoSortStatements.h" +#include "Luau/ToString.h" +#include "Luau/ToString.h" +#include "Luau/TypePack.h" +#include "Luau/TypeUtils.h" +#include "Luau/TypeVar.h" #include #include @@ -26,14 +27,11 @@ LUAU_FASTINTVARIABLE(LuauTypeInferRecursionLimit, 165) LUAU_FASTINTVARIABLE(LuauTypeInferIterationLimit, 20000) LUAU_FASTINTVARIABLE(LuauTypeInferTypePackLoopLimit, 5000) LUAU_FASTINTVARIABLE(LuauCheckRecursionLimit, 300) -LUAU_FASTFLAGVARIABLE(LuauUseVisitRecursionLimit, false) LUAU_FASTINTVARIABLE(LuauVisitRecursionLimit, 500) LUAU_FASTFLAG(LuauKnowsTheDataModel3) -LUAU_FASTFLAG(LuauSeparateTypechecks) LUAU_FASTFLAG(LuauAutocompleteDynamicLimits) LUAU_FASTFLAGVARIABLE(LuauDoNotRelyOnNextBinding, false) LUAU_FASTFLAGVARIABLE(LuauExpectedPropTypeFromIndexer, false) -LUAU_FASTFLAGVARIABLE(LuauWeakEqConstraint, false) // Eventually removed as false. LUAU_FASTFLAGVARIABLE(LuauLowerBoundsCalculation, false) LUAU_FASTFLAGVARIABLE(DebugLuauFreezeDuringUnification, false) LUAU_FASTFLAGVARIABLE(LuauSelfCallAutocompleteFix, false) @@ -43,7 +41,6 @@ LUAU_FASTFLAGVARIABLE(LuauUnsealedTableLiteral, false) LUAU_FASTFLAGVARIABLE(LuauTwoPassAliasDefinitionFix, false) LUAU_FASTFLAGVARIABLE(LuauReturnAnyInsteadOfICE, false) // Eventually removed as false. LUAU_FASTFLAG(LuauNormalizeFlagIsConservative) -LUAU_FASTFLAG(LuauWidenIfSupertypeIsFree2) LUAU_FASTFLAGVARIABLE(LuauReturnTypeInferenceInNonstrict, false) LUAU_FASTFLAGVARIABLE(LuauRecursionLimitException, false); LUAU_FASTFLAGVARIABLE(LuauApplyTypeFunctionFix, false); @@ -51,6 +48,7 @@ LUAU_FASTFLAGVARIABLE(LuauTypecheckIter, false); LUAU_FASTFLAGVARIABLE(LuauSuccessTypingForEqualityOperations, false) LUAU_FASTFLAGVARIABLE(LuauNoMethodLocations, false); LUAU_FASTFLAGVARIABLE(LuauAlwaysQuantify, false); +LUAU_FASTFLAGVARIABLE(LuauReportErrorsOnIndexerKeyMismatch, false) namespace Luau { @@ -305,12 +303,8 @@ ModulePtr TypeChecker::checkWithoutRecursionCheck(const SourceModule& module, Mo currentModule.reset(new Module()); currentModule->type = module.type; - - if (FFlag::LuauSeparateTypechecks) - { - currentModule->allocator = module.allocator; - currentModule->names = module.names; - } + currentModule->allocator = module.allocator; + currentModule->names = module.names; iceHandler->moduleName = module.name; @@ -338,21 +332,14 @@ ModulePtr TypeChecker::checkWithoutRecursionCheck(const SourceModule& module, Mo if (prepareModuleScope) prepareModuleScope(module.name, currentModule->getModuleScope()); - if (FFlag::LuauSeparateTypechecks) - { - try - { - checkBlock(moduleScope, *module.root); - } - catch (const TimeLimitError&) - { - currentModule->timeout = true; - } - } - else + try { checkBlock(moduleScope, *module.root); } + catch (const TimeLimitError&) + { + currentModule->timeout = true; + } if (get(follow(moduleScope->returnType))) moduleScope->returnType = addTypePack(TypePack{{}, std::nullopt}); @@ -443,7 +430,7 @@ void TypeChecker::check(const ScopePtr& scope, const AstStat& program) else ice("Unknown AstStat"); - if (FFlag::LuauSeparateTypechecks && finishTime && TimeTrace::getClock() > *finishTime) + if (finishTime && TimeTrace::getClock() > *finishTime) throw TimeLimitError(); } @@ -868,9 +855,9 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatAssign& assign) TypeId right = nullptr; - Location loc = 0 == assign.values.size - ? assign.location - : i < assign.values.size ? assign.values.data[i]->location : assign.values.data[assign.values.size - 1]->location; + Location loc = 0 == assign.values.size ? assign.location + : i < assign.values.size ? assign.values.data[i]->location + : assign.values.data[assign.values.size - 1]->location; if (valueIter != valueEnd) { @@ -1825,7 +1812,7 @@ std::optional TypeChecker::findMetatableEntry(TypeId type, std::string e } std::optional TypeChecker::getIndexTypeFromType( - const ScopePtr& scope, TypeId type, const Name& name, const Location& location, bool addErrors) + const ScopePtr& scope, TypeId type, const std::string& name, const Location& location, bool addErrors) { type = follow(type); @@ -1843,13 +1830,25 @@ std::optional TypeChecker::getIndexTypeFromType( if (TableTypeVar* tableType = getMutableTableType(type)) { - const auto& it = tableType->props.find(name); - if (it != tableType->props.end()) + if (auto it = tableType->props.find(name); it != tableType->props.end()) return it->second.type; else if (auto indexer = tableType->indexer) { - tryUnify(stringType, indexer->indexType, location); - return indexer->indexResultType; + // TODO: Property lookup should work with string singletons or unions thereof as the indexer key type. + ErrorVec errors = tryUnify(stringType, indexer->indexType, location); + + if (FFlag::LuauReportErrorsOnIndexerKeyMismatch) + { + if (errors.empty()) + return indexer->indexResultType; + + if (addErrors) + reportError(location, UnknownProperty{type, name}); + + return std::nullopt; + } + else + return indexer->indexResultType; } else if (tableType->state == TableState::Free) { @@ -1858,8 +1857,7 @@ std::optional TypeChecker::getIndexTypeFromType( return result; } - auto found = findTablePropertyRespectingMeta(type, name, location); - if (found) + if (auto found = findTablePropertyRespectingMeta(type, name, location)) return *found; } else if (const ClassTypeVar* cls = get(type)) @@ -2512,8 +2510,9 @@ TypeId TypeChecker::checkRelationalOperation( if (!matches) { - reportError(expr.location, GenericError{format("Types %s and %s cannot be compared with %s because they do not have the same metatable", - toString(lhsType).c_str(), toString(rhsType).c_str(), toString(expr.op).c_str())}); + reportError( + expr.location, GenericError{format("Types %s and %s cannot be compared with %s because they do not have the same metatable", + toString(lhsType).c_str(), toString(rhsType).c_str(), toString(expr.op).c_str())}); return errorRecoveryType(booleanType); } } @@ -2522,8 +2521,9 @@ TypeId TypeChecker::checkRelationalOperation( { if (bool(leftMetatable) != bool(rightMetatable) && leftMetatable != rightMetatable) { - reportError(expr.location, GenericError{format("Types %s and %s cannot be compared with %s because they do not have the same metatable", - toString(lhsType).c_str(), toString(rhsType).c_str(), toString(expr.op).c_str())}); + reportError( + expr.location, GenericError{format("Types %s and %s cannot be compared with %s because they do not have the same metatable", + toString(lhsType).c_str(), toString(rhsType).c_str(), toString(expr.op).c_str())}); return errorRecoveryType(booleanType); } } @@ -3636,10 +3636,7 @@ void TypeChecker::checkArgumentList( } TypePackId varPack = addTypePack(TypePackVar{TypePack{rest, argIter.tail()}}); - if (FFlag::LuauWidenIfSupertypeIsFree2) - state.tryUnify(varPack, tail); - else - state.tryUnify(tail, varPack); + state.tryUnify(varPack, tail); return; } @@ -3707,7 +3704,7 @@ ExprResult TypeChecker::checkExprPack(const ScopePtr& scope, const A } TypePackId retPack; - if (FFlag::LuauLowerBoundsCalculation || !FFlag::LuauWidenIfSupertypeIsFree2) + if (FFlag::LuauLowerBoundsCalculation) { retPack = freshTypePack(scope->level); } @@ -3868,9 +3865,7 @@ std::optional> TypeChecker::checkCallOverload(const Scope Widen widen{¤tModule->internalTypes}; for (; it != endIt; ++it) { - TypeId t = *it; - TypeId widened = widen.substitute(t).value_or(t); // Surely widening is infallible - adjustedArgTypes.push_back(addType(ConstrainedTypeVar{level, {widened}})); + adjustedArgTypes.push_back(addType(ConstrainedTypeVar{level, {widen(*it)}})); } TypePackId adjustedArgPack = addTypePack(TypePack{std::move(adjustedArgTypes), it.tail()}); @@ -3885,14 +3880,11 @@ std::optional> TypeChecker::checkCallOverload(const Scope else { TypeId r = addType(FunctionTypeVar(scope->level, argPack, retPack)); - if (FFlag::LuauWidenIfSupertypeIsFree2) - { - UnifierOptions options; - options.isFunctionCall = true; - unify(r, fn, expr.location, options); - } - else - unify(fn, r, expr.location); + + UnifierOptions options; + options.isFunctionCall = true; + unify(r, fn, expr.location, options); + return {{retPack}}; } } @@ -4375,122 +4367,6 @@ void TypeChecker::unifyWithInstantiationIfNeeded(const ScopePtr& scope, TypeId s } } -bool Instantiation::isDirty(TypeId ty) -{ - if (const FunctionTypeVar* ftv = log->getMutable(ty)) - { - if (ftv->hasNoGenerics) - return false; - - return true; - } - else - { - return false; - } -} - -bool Instantiation::isDirty(TypePackId tp) -{ - return false; -} - -bool Instantiation::ignoreChildren(TypeId ty) -{ - if (log->getMutable(ty)) - return true; - else - return false; -} - -TypeId Instantiation::clean(TypeId ty) -{ - const FunctionTypeVar* ftv = log->getMutable(ty); - LUAU_ASSERT(ftv); - - FunctionTypeVar clone = FunctionTypeVar{level, ftv->argTypes, ftv->retType, ftv->definition, ftv->hasSelf}; - clone.magicFunction = ftv->magicFunction; - clone.tags = ftv->tags; - clone.argNames = ftv->argNames; - TypeId result = addType(std::move(clone)); - - // Annoyingly, we have to do this even if there are no generics, - // to replace any generic tables. - ReplaceGenerics replaceGenerics{log, arena, level, ftv->generics, ftv->genericPacks}; - - // TODO: What to do if this returns nullopt? - // We don't have access to the error-reporting machinery - result = replaceGenerics.substitute(result).value_or(result); - - asMutable(result)->documentationSymbol = ty->documentationSymbol; - return result; -} - -TypePackId Instantiation::clean(TypePackId tp) -{ - LUAU_ASSERT(false); - return tp; -} - -bool ReplaceGenerics::ignoreChildren(TypeId ty) -{ - if (const FunctionTypeVar* ftv = log->getMutable(ty)) - { - if (ftv->hasNoGenerics) - return true; - - // We aren't recursing in the case of a generic function which - // binds the same generics. This can happen if, for example, there's recursive types. - // If T = (a,T)->T then instantiating T should produce T' = (X,T)->T not T' = (X,T')->T'. - // It's OK to use vector equality here, since we always generate fresh generics - // whenever we quantify, so the vectors overlap if and only if they are equal. - return (!generics.empty() || !genericPacks.empty()) && (ftv->generics == generics) && (ftv->genericPacks == genericPacks); - } - else - { - return false; - } -} - -bool ReplaceGenerics::isDirty(TypeId ty) -{ - if (const TableTypeVar* ttv = log->getMutable(ty)) - return ttv->state == TableState::Generic; - else if (log->getMutable(ty)) - return std::find(generics.begin(), generics.end(), ty) != generics.end(); - else - return false; -} - -bool ReplaceGenerics::isDirty(TypePackId tp) -{ - if (log->getMutable(tp)) - return std::find(genericPacks.begin(), genericPacks.end(), tp) != genericPacks.end(); - else - return false; -} - -TypeId ReplaceGenerics::clean(TypeId ty) -{ - LUAU_ASSERT(isDirty(ty)); - if (const TableTypeVar* ttv = log->getMutable(ty)) - { - TableTypeVar clone = TableTypeVar{ttv->props, ttv->indexer, level, TableState::Free}; - if (!FFlag::LuauNoMethodLocations) - clone.methodDefinitionLocations = ttv->methodDefinitionLocations; - clone.definitionModuleName = ttv->definitionModuleName; - return addType(std::move(clone)); - } - else - return addType(FreeTypeVar{level}); -} - -TypePackId ReplaceGenerics::clean(TypePackId tp) -{ - LUAU_ASSERT(isDirty(tp)); - return addTypePack(TypePackVar(FreeTypePack{level})); -} - bool Anyification::isDirty(TypeId ty) { if (ty->persistent) @@ -5295,7 +5171,7 @@ TypeId ApplyTypeFunction::clean(TypeId ty) { TypeId& arg = typeArguments[ty]; if (FFlag::LuauApplyTypeFunctionFix) - { + { LUAU_ASSERT(arg); return arg; } @@ -5309,7 +5185,7 @@ TypePackId ApplyTypeFunction::clean(TypePackId tp) { TypePackId& arg = typePackArguments[tp]; if (FFlag::LuauApplyTypeFunctionFix) - { + { LUAU_ASSERT(arg); return arg; } @@ -5837,9 +5713,6 @@ void TypeChecker::resolve(const EqPredicate& eqP, RefinementMap& refis, const Sc return; // Optimization: the other side has unknown types, so there's probably an overlap. Refining is no-op here. auto predicate = [&](TypeId option) -> std::optional { - if (sense && isUndecidable(option)) - return FFlag::LuauWeakEqConstraint ? option : eqP.type; - if (!sense && isNil(eqP.type)) return (isUndecidable(option) || !isNil(option)) ? std::optional(option) : std::nullopt; diff --git a/Analysis/src/Unifier.cpp b/Analysis/src/Unifier.cpp index 9308e9ff..414b05f4 100644 --- a/Analysis/src/Unifier.cpp +++ b/Analysis/src/Unifier.cpp @@ -21,8 +21,6 @@ LUAU_FASTFLAGVARIABLE(LuauTableSubtypingVariance2, false); LUAU_FASTFLAG(LuauLowerBoundsCalculation); LUAU_FASTFLAG(LuauErrorRecoveryType); LUAU_FASTFLAGVARIABLE(LuauSubtypingAddOptPropsToUnsealedTables, false) -LUAU_FASTFLAGVARIABLE(LuauWidenIfSupertypeIsFree2, false) -LUAU_FASTFLAGVARIABLE(LuauDifferentOrderOfUnificationDoesntMatter2, false) LUAU_FASTFLAGVARIABLE(LuauTxnLogRefreshFunctionPointers, false) namespace Luau @@ -149,8 +147,7 @@ static void promoteTypeLevels(TxnLog& log, const TypeArena* typeArena, TypeLevel return; PromoteTypeLevels ptl{log, typeArena, minLevel}; - DenseHashSet seen{nullptr}; - DEPRECATED_visitTypeVarOnce(ty, ptl, seen); + ptl.traverse(ty); } void promoteTypeLevels(TxnLog& log, const TypeArena* typeArena, TypeLevel minLevel, TypePackId tp) @@ -160,8 +157,7 @@ void promoteTypeLevels(TxnLog& log, const TypeArena* typeArena, TypeLevel minLev return; PromoteTypeLevels ptl{log, typeArena, minLevel}; - DenseHashSet seen{nullptr}; - DEPRECATED_visitTypeVarOnce(tp, ptl, seen); + ptl.traverse(tp); } struct SkipCacheForType final : TypeVarOnceVisitor @@ -172,49 +168,6 @@ struct SkipCacheForType final : TypeVarOnceVisitor { } - // TODO cycle() and operator() can be clipped with FFlagLuauUseVisitRecursionLimit - void cycle(TypeId) override {} - void cycle(TypePackId) override {} - - bool operator()(TypeId ty, const FreeTypeVar& ftv) - { - return visit(ty, ftv); - } - bool operator()(TypeId ty, const BoundTypeVar& btv) - { - return visit(ty, btv); - } - bool operator()(TypeId ty, const GenericTypeVar& gtv) - { - return visit(ty, gtv); - } - bool operator()(TypeId ty, const TableTypeVar& ttv) - { - return visit(ty, ttv); - } - bool operator()(TypePackId tp, const FreeTypePack& ftp) - { - return visit(tp, ftp); - } - bool operator()(TypePackId tp, const BoundTypePack& ftp) - { - return visit(tp, ftp); - } - bool operator()(TypePackId tp, const GenericTypePack& ftp) - { - return visit(tp, ftp); - } - template - bool operator()(TypeId ty, const T& t) - { - return visit(ty); - } - template - bool operator()(TypePackId tp, const T&) - { - return visit(tp); - } - bool visit(TypeId, const FreeTypeVar&) override { result = true; @@ -341,6 +294,16 @@ bool Widen::ignoreChildren(TypeId ty) return !log->is(ty); } +TypeId Widen::operator()(TypeId ty) +{ + return substitute(ty).value_or(ty); +} + +TypePackId Widen::operator()(TypePackId tp) +{ + return substitute(tp).value_or(tp); +} + static std::optional hasUnificationTooComplex(const ErrorVec& errors) { auto isUnificationTooComplex = [](const TypeError& te) { @@ -475,6 +438,8 @@ void Unifier::tryUnify_(TypeId subTy, TypeId superTy, bool isFunctionCall, bool if (!occursFailed) { promoteTypeLevels(log, types, superLevel, subTy); + + Widen widen{types}; log.replace(superTy, BoundTypeVar(widen(subTy))); } @@ -612,9 +577,6 @@ void Unifier::tryUnifyUnionWithType(TypeId subTy, const UnionTypeVar* uv, TypeId std::optional unificationTooComplex; std::optional firstFailedOption; - size_t count = uv->options.size(); - size_t i = 0; - for (TypeId type : uv->options) { Unifier innerState = makeChildUnifier(); @@ -630,60 +592,44 @@ void Unifier::tryUnifyUnionWithType(TypeId subTy, const UnionTypeVar* uv, TypeId failed = true; } - - if (FFlag::LuauDifferentOrderOfUnificationDoesntMatter2) - { - } - else - { - if (i == count - 1) - { - log.concat(std::move(innerState.log)); - } - - ++i; - } } // even if A | B <: T fails, we want to bind some options of T with A | B iff A | B was a subtype of that option. - if (FFlag::LuauDifferentOrderOfUnificationDoesntMatter2) - { - auto tryBind = [this, subTy](TypeId superOption) { - superOption = log.follow(superOption); + auto tryBind = [this, subTy](TypeId superOption) { + superOption = log.follow(superOption); - // just skip if the superOption is not free-ish. - auto ttv = log.getMutable(superOption); - if (!log.is(superOption) && (!ttv || ttv->state != TableState::Free)) - return; + // just skip if the superOption is not free-ish. + auto ttv = log.getMutable(superOption); + if (!log.is(superOption) && (!ttv || ttv->state != TableState::Free)) + return; - // If superOption is already present in subTy, do nothing. Nothing new has been learned, but the subtype - // test is successful. - if (auto subUnion = get(subTy)) - { - if (end(subUnion) != std::find(begin(subUnion), end(subUnion), superOption)) - return; - } - - // Since we have already checked if S <: T, checking it again will not queue up the type for replacement. - // So we'll have to do it ourselves. We assume they unified cleanly if they are still in the seen set. - if (log.haveSeen(subTy, superOption)) - { - // TODO: would it be nice for TxnLog::replace to do this? - if (log.is(superOption)) - log.bindTable(superOption, subTy); - else - log.replace(superOption, *subTy); - } - }; - - if (auto utv = log.getMutable(superTy)) + // If superOption is already present in subTy, do nothing. Nothing new has been learned, but the subtype + // test is successful. + if (auto subUnion = get(subTy)) { - for (TypeId ty : utv) - tryBind(ty); + if (end(subUnion) != std::find(begin(subUnion), end(subUnion), superOption)) + return; } - else - tryBind(superTy); + + // Since we have already checked if S <: T, checking it again will not queue up the type for replacement. + // So we'll have to do it ourselves. We assume they unified cleanly if they are still in the seen set. + if (log.haveSeen(subTy, superOption)) + { + // TODO: would it be nice for TxnLog::replace to do this? + if (log.is(superOption)) + log.bindTable(superOption, subTy); + else + log.replace(superOption, *subTy); + } + }; + + if (auto utv = log.getMutable(superTy)) + { + for (TypeId ty : utv) + tryBind(ty); } + else + tryBind(superTy); if (unificationTooComplex) reportError(*unificationTooComplex); @@ -883,7 +829,7 @@ bool Unifier::canCacheResult(TypeId subTy, TypeId superTy) auto skipCacheFor = [this](TypeId ty) { SkipCacheForType visitor{sharedState.skipCacheForType, types}; - DEPRECATED_visitTypeVarOnce(ty, visitor, sharedState.seenAny); + visitor.traverse(ty); sharedState.skipCacheForType[ty] = visitor.result; @@ -1088,6 +1034,7 @@ void Unifier::tryUnify_(TypePackId subTp, TypePackId superTp, bool isFunctionCal if (!log.getMutable(superTp)) { + Widen widen{types}; log.replace(superTp, Unifiable::Bound(widen(subTp))); } } @@ -1671,28 +1618,6 @@ void Unifier::tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) } } -TypeId Unifier::widen(TypeId ty) -{ - if (!FFlag::LuauWidenIfSupertypeIsFree2) - return ty; - - Widen widen{types}; - std::optional result = widen.substitute(ty); - // TODO: what does it mean for substitution to fail to widen? - return result.value_or(ty); -} - -TypePackId Unifier::widen(TypePackId tp) -{ - if (!FFlag::LuauWidenIfSupertypeIsFree2) - return tp; - - Widen widen{types}; - std::optional result = widen.substitute(tp); - // TODO: what does it mean for substitution to fail to widen? - return result.value_or(tp); -} - TypeId Unifier::deeplyOptional(TypeId ty, std::unordered_map seen) { ty = follow(ty); @@ -1809,10 +1734,7 @@ void Unifier::tryUnifyFreeTable(TypeId subTy, TypeId superTy) { if (auto subProp = findTablePropertyRespectingMeta(subTy, freeName)) { - if (FFlag::LuauWidenIfSupertypeIsFree2) - tryUnify_(*subProp, freeProp.type); - else - tryUnify_(freeProp.type, *subProp); + tryUnify_(*subProp, freeProp.type); /* * TypeVars are commonly cyclic, so it is entirely possible diff --git a/Ast/include/Luau/TimeTrace.h b/Ast/include/Luau/TimeTrace.h index 9f7b2bdf..be282827 100644 --- a/Ast/include/Luau/TimeTrace.h +++ b/Ast/include/Luau/TimeTrace.h @@ -1,7 +1,7 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #pragma once -#include "Common.h" +#include "Luau/Common.h" #include diff --git a/Ast/src/Parser.cpp b/Ast/src/Parser.cpp index c053e6bd..eaf19914 100644 --- a/Ast/src/Parser.cpp +++ b/Ast/src/Parser.cpp @@ -11,6 +11,8 @@ LUAU_FASTINTVARIABLE(LuauRecursionLimit, 1000) LUAU_FASTINTVARIABLE(LuauParseErrorLimit, 100) +LUAU_FASTFLAGVARIABLE(LuauParserFunctionKeywordAsTypeHelp, false) + namespace Luau { @@ -1589,6 +1591,17 @@ AstTypeOrPack Parser::parseSimpleTypeAnnotation(bool allowPack) { return parseFunctionTypeAnnotation(allowPack); } + else if (FFlag::LuauParserFunctionKeywordAsTypeHelp && lexer.current().type == Lexeme::ReservedFunction) + { + Location location = lexer.current().location; + + nextLexeme(); + + return {reportTypeAnnotationError(location, {}, /*isMissing*/ false, + "Using 'function' as a type annotation is not supported, consider replacing with a function type annotation e.g. '(...any) -> " + "...any'"), + {}}; + } else { Location location = lexer.current().location; diff --git a/CMakeLists.txt b/CMakeLists.txt index ea352309..c624a132 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -19,9 +19,11 @@ if(LUAU_STATIC_CRT) endif() project(Luau LANGUAGES CXX C) +add_library(Luau.Common INTERFACE) add_library(Luau.Ast STATIC) add_library(Luau.Compiler STATIC) add_library(Luau.Analysis STATIC) +add_library(Luau.CodeGen STATIC) add_library(Luau.VM STATIC) add_library(isocline STATIC) @@ -48,8 +50,11 @@ endif() include(Sources.cmake) +target_include_directories(Luau.Common INTERFACE Common/include) + target_compile_features(Luau.Ast PUBLIC cxx_std_17) target_include_directories(Luau.Ast PUBLIC Ast/include) +target_link_libraries(Luau.Ast PUBLIC Luau.Common) target_compile_features(Luau.Compiler PUBLIC cxx_std_17) target_include_directories(Luau.Compiler PUBLIC Compiler/include) @@ -59,8 +64,13 @@ target_compile_features(Luau.Analysis PUBLIC cxx_std_17) target_include_directories(Luau.Analysis PUBLIC Analysis/include) target_link_libraries(Luau.Analysis PUBLIC Luau.Ast) +target_compile_features(Luau.CodeGen PRIVATE cxx_std_17) +target_include_directories(Luau.CodeGen PUBLIC CodeGen/include) +target_link_libraries(Luau.CodeGen PUBLIC Luau.Common) + target_compile_features(Luau.VM PRIVATE cxx_std_11) target_include_directories(Luau.VM PUBLIC VM/include) +target_link_libraries(Luau.VM PUBLIC Luau.Common) target_include_directories(isocline PUBLIC extern/isocline/include) @@ -101,6 +111,7 @@ endif() target_compile_options(Luau.Ast PRIVATE ${LUAU_OPTIONS}) target_compile_options(Luau.Analysis PRIVATE ${LUAU_OPTIONS}) +target_compile_options(Luau.CodeGen PRIVATE ${LUAU_OPTIONS}) target_compile_options(Luau.VM PRIVATE ${LUAU_OPTIONS}) target_compile_options(isocline PRIVATE ${LUAU_OPTIONS} ${ISOCLINE_OPTIONS}) @@ -120,6 +131,7 @@ endif() if(MSVC) target_link_options(Luau.Ast INTERFACE /NATVIS:${CMAKE_CURRENT_SOURCE_DIR}/tools/natvis/Ast.natvis) target_link_options(Luau.Analysis INTERFACE /NATVIS:${CMAKE_CURRENT_SOURCE_DIR}/tools/natvis/Analysis.natvis) + target_link_options(Luau.CodeGen INTERFACE /NATVIS:${CMAKE_CURRENT_SOURCE_DIR}/tools/natvis/CodeGen.natvis) target_link_options(Luau.VM INTERFACE /NATVIS:${CMAKE_CURRENT_SOURCE_DIR}/tools/natvis/VM.natvis) endif() @@ -127,6 +139,7 @@ endif() if(MSVC_IDE) target_sources(Luau.Ast PRIVATE tools/natvis/Ast.natvis) target_sources(Luau.Analysis PRIVATE tools/natvis/Analysis.natvis) + target_sources(Luau.CodeGen PRIVATE tools/natvis/CodeGen.natvis) target_sources(Luau.VM PRIVATE tools/natvis/VM.natvis) endif() @@ -154,7 +167,7 @@ endif() if(LUAU_BUILD_TESTS) target_compile_options(Luau.UnitTest PRIVATE ${LUAU_OPTIONS}) target_include_directories(Luau.UnitTest PRIVATE extern) - target_link_libraries(Luau.UnitTest PRIVATE Luau.Analysis Luau.Compiler) + target_link_libraries(Luau.UnitTest PRIVATE Luau.Analysis Luau.Compiler Luau.CodeGen) target_compile_options(Luau.Conformance PRIVATE ${LUAU_OPTIONS}) target_include_directories(Luau.Conformance PRIVATE extern) diff --git a/CodeGen/include/Luau/AssemblyBuilderX64.h b/CodeGen/include/Luau/AssemblyBuilderX64.h new file mode 100644 index 00000000..c5979d3c --- /dev/null +++ b/CodeGen/include/Luau/AssemblyBuilderX64.h @@ -0,0 +1,169 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +#pragma once + +#include "Luau/Common.h" +#include "Luau/Condition.h" +#include "Luau/Label.h" +#include "Luau/OperandX64.h" +#include "Luau/RegisterX64.h" + +#include +#include + +namespace Luau +{ +namespace CodeGen +{ + +class AssemblyBuilderX64 +{ +public: + explicit AssemblyBuilderX64(bool logText); + ~AssemblyBuilderX64(); + + // Base two operand instructions with 9 opcode selection + void add(OperandX64 lhs, OperandX64 rhs); + void sub(OperandX64 lhs, OperandX64 rhs); + void cmp(OperandX64 lhs, OperandX64 rhs); + void and_(OperandX64 lhs, OperandX64 rhs); + void or_(OperandX64 lhs, OperandX64 rhs); + void xor_(OperandX64 lhs, OperandX64 rhs); + + // Binary shift instructions with special rhs handling + void sal(OperandX64 lhs, OperandX64 rhs); + void sar(OperandX64 lhs, OperandX64 rhs); + void shl(OperandX64 lhs, OperandX64 rhs); + void shr(OperandX64 lhs, OperandX64 rhs); + + // Two operand mov instruction has additional specialized encodings + void mov(OperandX64 lhs, OperandX64 rhs); + void mov64(RegisterX64 lhs, int64_t imm); + + // Base one operand instruction with 2 opcode selection + void div(OperandX64 op); + void idiv(OperandX64 op); + void mul(OperandX64 op); + void neg(OperandX64 op); + void not_(OperandX64 op); + + void test(OperandX64 lhs, OperandX64 rhs); + void lea(OperandX64 lhs, OperandX64 rhs); + + void push(OperandX64 op); + void pop(OperandX64 op); + void ret(); + + // Control flow + void jcc(Condition cond, Label& label); + void jmp(Label& label); + void jmp(OperandX64 op); + + // AVX + void vaddpd(OperandX64 dst, OperandX64 src1, OperandX64 src2); + void vaddps(OperandX64 dst, OperandX64 src1, OperandX64 src2); + void vaddsd(OperandX64 dst, OperandX64 src1, OperandX64 src2); + void vaddss(OperandX64 dst, OperandX64 src1, OperandX64 src2); + + void vsqrtpd(OperandX64 dst, OperandX64 src); + void vsqrtps(OperandX64 dst, OperandX64 src); + void vsqrtsd(OperandX64 dst, OperandX64 src1, OperandX64 src2); + void vsqrtss(OperandX64 dst, OperandX64 src1, OperandX64 src2); + + void vmovsd(OperandX64 dst, OperandX64 src); + void vmovsd(OperandX64 dst, OperandX64 src1, OperandX64 src2); + void vmovss(OperandX64 dst, OperandX64 src); + void vmovss(OperandX64 dst, OperandX64 src1, OperandX64 src2); + void vmovapd(OperandX64 dst, OperandX64 src); + void vmovaps(OperandX64 dst, OperandX64 src); + void vmovupd(OperandX64 dst, OperandX64 src); + void vmovups(OperandX64 dst, OperandX64 src); + + // Run final checks + void finalize(); + + // Places a label at current location and returns it + Label setLabel(); + + // Assigns label position to the current location + void setLabel(Label& label); + + // Constant allocation (uses rip-relative addressing) + OperandX64 i64(int64_t value); + OperandX64 f32(float value); + OperandX64 f64(double value); + OperandX64 f32x4(float x, float y, float z, float w); + + // Resulting data and code that need to be copied over one after the other + // The *end* of 'data' has to be aligned to 16 bytes, this will also align 'code' + std::vector data; + std::vector code; + + std::string text; + +private: + // Instruction archetypes + void placeBinary(const char* name, OperandX64 lhs, OperandX64 rhs, uint8_t codeimm8, uint8_t codeimm, uint8_t codeimmImm8, uint8_t code8rev, + uint8_t coderev, uint8_t code8, uint8_t code, uint8_t opreg); + void placeBinaryRegMemAndImm(OperandX64 lhs, OperandX64 rhs, uint8_t code8, uint8_t code, uint8_t codeImm8, uint8_t opreg); + void placeBinaryRegAndRegMem(OperandX64 lhs, OperandX64 rhs, uint8_t code8, uint8_t code); + void placeBinaryRegMemAndReg(OperandX64 lhs, OperandX64 rhs, uint8_t code8, uint8_t code); + + void placeUnaryModRegMem(const char* name, OperandX64 op, uint8_t code8, uint8_t code, uint8_t opreg); + + void placeShift(const char* name, OperandX64 lhs, OperandX64 rhs, uint8_t opreg); + + void placeJcc(const char* name, Label& label, uint8_t cc); + + void placeAvx(const char* name, OperandX64 dst, OperandX64 src, uint8_t code, bool setW, uint8_t mode, uint8_t prefix); + void placeAvx(const char* name, OperandX64 dst, OperandX64 src, uint8_t code, uint8_t coderev, bool setW, uint8_t mode, uint8_t prefix); + void placeAvx(const char* name, OperandX64 dst, OperandX64 src1, OperandX64 src2, uint8_t code, bool setW, uint8_t mode, uint8_t prefix); + + // Instruction components + void placeRegAndModRegMem(OperandX64 lhs, OperandX64 rhs); + void placeModRegMem(OperandX64 rhs, uint8_t regop); + void placeRex(RegisterX64 op); + void placeRex(OperandX64 op); + void placeRex(RegisterX64 lhs, OperandX64 rhs); + void placeVex(OperandX64 dst, OperandX64 src1, OperandX64 src2, bool setW, uint8_t mode, uint8_t prefix); + void placeImm8Or32(int32_t imm); + void placeImm8(int32_t imm); + void placeImm32(int32_t imm); + void placeImm64(int64_t imm); + void placeLabel(Label& label); + void place(uint8_t byte); + + void commit(); + LUAU_NOINLINE void extend(); + uint32_t getCodeSize(); + + // Data + size_t allocateData(size_t size, size_t align); + + // Logging of assembly in text form (Intel asm with VS disassembly formatting) + LUAU_NOINLINE void log(const char* opcode); + LUAU_NOINLINE void log(const char* opcode, OperandX64 op); + LUAU_NOINLINE void log(const char* opcode, OperandX64 op1, OperandX64 op2); + LUAU_NOINLINE void log(const char* opcode, OperandX64 op1, OperandX64 op2, OperandX64 op3); + LUAU_NOINLINE void log(Label label); + LUAU_NOINLINE void log(const char* opcode, Label label); + void log(OperandX64 op); + void logAppend(const char* fmt, ...); + + const char* getSizeName(SizeX64 size); + const char* getRegisterName(RegisterX64 reg); + + uint32_t nextLabel = 1; + std::vector(a,T)->T then instantiating T should produce T' = (X,T)->T not T' = (X,T')->T'. + // It's OK to use vector equality here, since we always generate fresh generics + // whenever we quantify, so the vectors overlap if and only if they are equal. + return (!generics.empty() || !genericPacks.empty()) && (ftv->generics == generics) && (ftv->genericPacks == genericPacks); + } + else + { + return false; + } +} + +bool ReplaceGenerics::isDirty(TypeId ty) +{ + if (const TableTypeVar* ttv = log->getMutable(ty)) + return ttv->state == TableState::Generic; + else if (log->getMutable(ty)) + return std::find(generics.begin(), generics.end(), ty) != generics.end(); + else + return false; +} + +bool ReplaceGenerics::isDirty(TypePackId tp) +{ + if (log->getMutable(tp)) + return std::find(genericPacks.begin(), genericPacks.end(), tp) != genericPacks.end(); + else + return false; +} + +TypeId ReplaceGenerics::clean(TypeId ty) +{ + LUAU_ASSERT(isDirty(ty)); + if (const TableTypeVar* ttv = log->getMutable(ty)) + { + TableTypeVar clone = TableTypeVar{ttv->props, ttv->indexer, level, TableState::Free}; + if (!FFlag::LuauNoMethodLocations) + clone.methodDefinitionLocations = ttv->methodDefinitionLocations; + clone.definitionModuleName = ttv->definitionModuleName; + return addType(std::move(clone)); + } + else + return addType(FreeTypeVar{level}); +} + +TypePackId ReplaceGenerics::clean(TypePackId tp) +{ + LUAU_ASSERT(isDirty(tp)); + return addTypePack(TypePackVar(FreeTypePack{level})); +} + +} // namespace Luau diff --git a/Analysis/src/Module.cpp b/Analysis/src/Module.cpp index 074a41e6..6591d60a 100644 --- a/Analysis/src/Module.cpp +++ b/Analysis/src/Module.cpp @@ -56,8 +56,18 @@ bool isWithinComment(const SourceModule& sourceModule, Position pos) struct ForceNormal : TypeVarOnceVisitor { + const TypeArena* typeArena = nullptr; + + ForceNormal(const TypeArena* typeArena) + : typeArena(typeArena) + { + } + bool visit(TypeId ty) override { + if (ty->owningArena != typeArena) + return false; + asMutable(ty)->normal = true; return true; } @@ -100,7 +110,7 @@ void Module::clonePublicInterface(InternalErrorReporter& ice) normalize(*moduleScope->varargPack, interfaceTypes, ice); } - ForceNormal forceNormal; + ForceNormal forceNormal{&interfaceTypes}; for (auto& [name, tf] : moduleScope->exportedTypeBindings) { diff --git a/Analysis/src/Normalize.cpp b/Analysis/src/Normalize.cpp index 30fd4af2..fb31df1e 100644 --- a/Analysis/src/Normalize.cpp +++ b/Analysis/src/Normalize.cpp @@ -15,6 +15,7 @@ LUAU_FASTFLAGVARIABLE(DebugLuauCopyBeforeNormalizing, false) LUAU_FASTINTVARIABLE(LuauNormalizeIterationLimit, 1200); LUAU_FASTFLAGVARIABLE(LuauNormalizeCombineTableFix, false); LUAU_FASTFLAGVARIABLE(LuauNormalizeFlagIsConservative, false); +LUAU_FASTFLAGVARIABLE(LuauNormalizeCombineEqFix, false); namespace Luau { @@ -325,245 +326,6 @@ struct Normalize final : TypeVarVisitor int iterationLimit = 0; bool limitExceeded = false; - // TODO: Clip with FFlag::LuauUseVisitRecursionLimit - bool operator()(TypeId ty, const BoundTypeVar& btv, std::unordered_set& seen) - { - // A type could be considered normal when it is in the stack, but we will eventually find out it is not normal as normalization progresses. - // So we need to avoid eagerly saying that this bound type is normal if the thing it is bound to is in the stack. - if (seen.find(asMutable(btv.boundTo)) != seen.end()) - return false; - - // It should never be the case that this TypeVar is normal, but is bound to a non-normal type, except in nontrivial cases. - LUAU_ASSERT(!ty->normal || ty->normal == btv.boundTo->normal); - - asMutable(ty)->normal = btv.boundTo->normal; - return !ty->normal; - } - - bool operator()(TypeId ty, const FreeTypeVar& ftv) - { - return visit(ty, ftv); - } - bool operator()(TypeId ty, const PrimitiveTypeVar& ptv) - { - return visit(ty, ptv); - } - bool operator()(TypeId ty, const GenericTypeVar& gtv) - { - return visit(ty, gtv); - } - bool operator()(TypeId ty, const ErrorTypeVar& etv) - { - return visit(ty, etv); - } - bool operator()(TypeId ty, const ConstrainedTypeVar& ctvRef, std::unordered_set& seen) - { - CHECK_ITERATION_LIMIT(false); - - ConstrainedTypeVar* ctv = const_cast(&ctvRef); - - std::vector parts = std::move(ctv->parts); - - // We might transmute, so it's not safe to rely on the builtin traversal logic of visitTypeVar - for (TypeId part : parts) - visit_detail::visit(part, *this, seen); - - std::vector newParts = normalizeUnion(parts); - - const bool normal = areNormal(newParts, seen, ice); - - if (newParts.size() == 1) - *asMutable(ty) = BoundTypeVar{newParts[0]}; - else - *asMutable(ty) = UnionTypeVar{std::move(newParts)}; - - asMutable(ty)->normal = normal; - - return false; - } - - bool operator()(TypeId ty, const FunctionTypeVar& ftv, std::unordered_set& seen) - { - CHECK_ITERATION_LIMIT(false); - - if (ty->normal) - return false; - - visit_detail::visit(ftv.argTypes, *this, seen); - visit_detail::visit(ftv.retType, *this, seen); - - asMutable(ty)->normal = areNormal(ftv.argTypes, seen, ice) && areNormal(ftv.retType, seen, ice); - - return false; - } - - bool operator()(TypeId ty, const TableTypeVar& ttv, std::unordered_set& seen) - { - CHECK_ITERATION_LIMIT(false); - - if (ty->normal) - return false; - - bool normal = true; - - auto checkNormal = [&](TypeId t) { - // if t is on the stack, it is possible that this type is normal. - // If t is not normal and it is not on the stack, this type is definitely not normal. - if (!t->normal && seen.find(asMutable(t)) == seen.end()) - normal = false; - }; - - if (ttv.boundTo) - { - visit_detail::visit(*ttv.boundTo, *this, seen); - asMutable(ty)->normal = (*ttv.boundTo)->normal; - return false; - } - - for (const auto& [_name, prop] : ttv.props) - { - visit_detail::visit(prop.type, *this, seen); - checkNormal(prop.type); - } - - if (ttv.indexer) - { - visit_detail::visit(ttv.indexer->indexType, *this, seen); - checkNormal(ttv.indexer->indexType); - visit_detail::visit(ttv.indexer->indexResultType, *this, seen); - checkNormal(ttv.indexer->indexResultType); - } - - asMutable(ty)->normal = normal; - - return false; - } - - bool operator()(TypeId ty, const MetatableTypeVar& mtv, std::unordered_set& seen) - { - CHECK_ITERATION_LIMIT(false); - - if (ty->normal) - return false; - - visit_detail::visit(mtv.table, *this, seen); - visit_detail::visit(mtv.metatable, *this, seen); - - asMutable(ty)->normal = mtv.table->normal && mtv.metatable->normal; - - return false; - } - - bool operator()(TypeId ty, const ClassTypeVar& ctv) - { - return visit(ty, ctv); - } - bool operator()(TypeId ty, const AnyTypeVar& atv) - { - return visit(ty, atv); - } - bool operator()(TypeId ty, const UnionTypeVar& utvRef, std::unordered_set& seen) - { - CHECK_ITERATION_LIMIT(false); - - if (ty->normal) - return false; - - UnionTypeVar* utv = &const_cast(utvRef); - std::vector options = std::move(utv->options); - - // We might transmute, so it's not safe to rely on the builtin traversal logic of visitTypeVar - for (TypeId option : options) - visit_detail::visit(option, *this, seen); - - std::vector newOptions = normalizeUnion(options); - - const bool normal = areNormal(newOptions, seen, ice); - - LUAU_ASSERT(!newOptions.empty()); - - if (newOptions.size() == 1) - *asMutable(ty) = BoundTypeVar{newOptions[0]}; - else - utv->options = std::move(newOptions); - - asMutable(ty)->normal = normal; - - return false; - } - - bool operator()(TypeId ty, const IntersectionTypeVar& itvRef, std::unordered_set& seen) - { - CHECK_ITERATION_LIMIT(false); - - if (ty->normal) - return false; - - IntersectionTypeVar* itv = &const_cast(itvRef); - - std::vector oldParts = std::move(itv->parts); - - for (TypeId part : oldParts) - visit_detail::visit(part, *this, seen); - - std::vector tables; - for (TypeId part : oldParts) - { - part = follow(part); - if (get(part)) - tables.push_back(part); - else - { - Replacer replacer{&arena, nullptr, nullptr}; // FIXME this is super super WEIRD - combineIntoIntersection(replacer, itv, part); - } - } - - // Don't allocate a new table if there's just one in the intersection. - if (tables.size() == 1) - itv->parts.push_back(tables[0]); - else if (!tables.empty()) - { - const TableTypeVar* first = get(tables[0]); - LUAU_ASSERT(first); - - TypeId newTable = arena.addType(TableTypeVar{first->state, first->level}); - TableTypeVar* ttv = getMutable(newTable); - for (TypeId part : tables) - { - // Intuition: If combineIntoTable() needs to clone a table, any references to 'part' are cyclic and need - // to be rewritten to point at 'newTable' in the clone. - Replacer replacer{&arena, part, newTable}; - combineIntoTable(replacer, ttv, part); - } - - itv->parts.push_back(newTable); - } - - asMutable(ty)->normal = areNormal(itv->parts, seen, ice); - - if (itv->parts.size() == 1) - { - TypeId part = itv->parts[0]; - *asMutable(ty) = BoundTypeVar{part}; - } - - return false; - } - - // TODO: Clip with FFlag::LuauUseVisitRecursionLimit - template - bool operator()(TypePackId, const T&) - { - return true; - } - - // TODO: Clip with FFlag::LuauUseVisitRecursionLimit - template - void cycle(TID) - { - } - bool visit(TypeId ty, const FreeTypeVar&) override { LUAU_ASSERT(!ty->normal); @@ -968,6 +730,9 @@ struct Normalize final : TypeVarVisitor */ TypeId combine(Replacer& replacer, TypeId a, TypeId b) { + if (FFlag::LuauNormalizeCombineEqFix) + b = follow(b); + if (FFlag::LuauNormalizeCombineTableFix && a == b) return a; @@ -986,7 +751,7 @@ struct Normalize final : TypeVarVisitor } else if (auto ttv = getMutable(a)) { - if (FFlag::LuauNormalizeCombineTableFix && !get(follow(b))) + if (FFlag::LuauNormalizeCombineTableFix && !get(FFlag::LuauNormalizeCombineEqFix ? b : follow(b))) return arena.addType(IntersectionTypeVar{{a, b}}); combineIntoTable(replacer, ttv, b); return a; @@ -1009,15 +774,7 @@ std::pair normalize(TypeId ty, TypeArena& arena, InternalErrorRepo (void)clone(ty, arena, state); Normalize n{arena, ice}; - if (FFlag::LuauNormalizeFlagIsConservative) - { - DEPRECATED_visitTypeVar(ty, n); - } - else - { - std::unordered_set seen; - DEPRECATED_visitTypeVar(ty, n, seen); - } + n.traverse(ty); return {ty, !n.limitExceeded}; } @@ -1041,15 +798,7 @@ std::pair normalize(TypePackId tp, TypeArena& arena, InternalE (void)clone(tp, arena, state); Normalize n{arena, ice}; - if (FFlag::LuauNormalizeFlagIsConservative) - { - DEPRECATED_visitTypeVar(tp, n); - } - else - { - std::unordered_set seen; - DEPRECATED_visitTypeVar(tp, n, seen); - } + n.traverse(tp); return {tp, !n.limitExceeded}; } diff --git a/Analysis/src/Quantify.cpp b/Analysis/src/Quantify.cpp index 018d5632..c0f677d7 100644 --- a/Analysis/src/Quantify.cpp +++ b/Analysis/src/Quantify.cpp @@ -119,8 +119,7 @@ struct Quantifier final : TypeVarOnceVisitor void quantify(TypeId ty, TypeLevel level) { Quantifier q{level}; - DenseHashSet seen{nullptr}; - DEPRECATED_visitTypeVarOnce(ty, q, seen); + q.traverse(ty); FunctionTypeVar* ftv = getMutable(ty); LUAU_ASSERT(ftv); diff --git a/Analysis/src/ToString.cpp b/Analysis/src/ToString.cpp index f90f7019..a4a3ec49 100644 --- a/Analysis/src/ToString.cpp +++ b/Analysis/src/ToString.cpp @@ -48,46 +48,6 @@ struct FindCyclicTypes final : TypeVarVisitor cycleTPs.insert(tp); } - // TODO: Clip all the operator()s when we clip FFlagLuauUseVisitRecursionLimit - - template - bool operator()(TypeId ty, const T&) - { - return visit(ty); - } - - bool operator()(TypeId ty, const TableTypeVar& ttv) = delete; - - bool operator()(TypeId ty, const TableTypeVar& ttv, std::unordered_set& seen) - { - if (!visited.insert(ty).second) - return false; - - if (ttv.name || ttv.syntheticName) - { - for (TypeId itp : ttv.instantiatedTypeParams) - DEPRECATED_visitTypeVar(itp, *this, seen); - - for (TypePackId itp : ttv.instantiatedTypePackParams) - DEPRECATED_visitTypeVar(itp, *this, seen); - - return exhaustive; - } - - return true; - } - - bool operator()(TypeId, const ClassTypeVar&) - { - return false; - } - - template - bool operator()(TypePackId tp, const T&) - { - return visit(tp); - } - bool visit(TypeId ty) override { return visited.insert(ty).second; @@ -128,7 +88,7 @@ void findCyclicTypes(std::set& cycles, std::set& cycleTPs, T { FindCyclicTypes fct; fct.exhaustive = exhaustive; - DEPRECATED_visitTypeVar(ty, fct); + fct.traverse(ty); cycles = std::move(fct.cycles); cycleTPs = std::move(fct.cycleTPs); diff --git a/Analysis/src/TypeArena.cpp b/Analysis/src/TypeArena.cpp index 673b002d..0c89d130 100644 --- a/Analysis/src/TypeArena.cpp +++ b/Analysis/src/TypeArena.cpp @@ -85,4 +85,4 @@ void unfreeze(TypeArena& arena) arena.typePacks.unfreeze(); } -} +} // namespace Luau diff --git a/Analysis/src/TypeInfer.cpp b/Analysis/src/TypeInfer.cpp index 208b3f2f..11813c76 100644 --- a/Analysis/src/TypeInfer.cpp +++ b/Analysis/src/TypeInfer.cpp @@ -3,6 +3,7 @@ #include "Luau/Clone.h" #include "Luau/Common.h" +#include "Luau/Instantiation.h" #include "Luau/ModuleResolver.h" #include "Luau/Normalize.h" #include "Luau/Parser.h" @@ -10,13 +11,13 @@ #include "Luau/RecursionCounter.h" #include "Luau/Scope.h" #include "Luau/Substitution.h" -#include "Luau/TopoSortStatements.h" -#include "Luau/TypePack.h" -#include "Luau/ToString.h" -#include "Luau/TypeUtils.h" -#include "Luau/ToString.h" -#include "Luau/TypeVar.h" #include "Luau/TimeTrace.h" +#include "Luau/TopoSortStatements.h" +#include "Luau/ToString.h" +#include "Luau/ToString.h" +#include "Luau/TypePack.h" +#include "Luau/TypeUtils.h" +#include "Luau/TypeVar.h" #include #include @@ -26,14 +27,11 @@ LUAU_FASTINTVARIABLE(LuauTypeInferRecursionLimit, 165) LUAU_FASTINTVARIABLE(LuauTypeInferIterationLimit, 20000) LUAU_FASTINTVARIABLE(LuauTypeInferTypePackLoopLimit, 5000) LUAU_FASTINTVARIABLE(LuauCheckRecursionLimit, 300) -LUAU_FASTFLAGVARIABLE(LuauUseVisitRecursionLimit, false) LUAU_FASTINTVARIABLE(LuauVisitRecursionLimit, 500) LUAU_FASTFLAG(LuauKnowsTheDataModel3) -LUAU_FASTFLAG(LuauSeparateTypechecks) LUAU_FASTFLAG(LuauAutocompleteDynamicLimits) LUAU_FASTFLAGVARIABLE(LuauDoNotRelyOnNextBinding, false) LUAU_FASTFLAGVARIABLE(LuauExpectedPropTypeFromIndexer, false) -LUAU_FASTFLAGVARIABLE(LuauWeakEqConstraint, false) // Eventually removed as false. LUAU_FASTFLAGVARIABLE(LuauLowerBoundsCalculation, false) LUAU_FASTFLAGVARIABLE(DebugLuauFreezeDuringUnification, false) LUAU_FASTFLAGVARIABLE(LuauSelfCallAutocompleteFix, false) @@ -43,7 +41,6 @@ LUAU_FASTFLAGVARIABLE(LuauUnsealedTableLiteral, false) LUAU_FASTFLAGVARIABLE(LuauTwoPassAliasDefinitionFix, false) LUAU_FASTFLAGVARIABLE(LuauReturnAnyInsteadOfICE, false) // Eventually removed as false. LUAU_FASTFLAG(LuauNormalizeFlagIsConservative) -LUAU_FASTFLAG(LuauWidenIfSupertypeIsFree2) LUAU_FASTFLAGVARIABLE(LuauReturnTypeInferenceInNonstrict, false) LUAU_FASTFLAGVARIABLE(LuauRecursionLimitException, false); LUAU_FASTFLAGVARIABLE(LuauApplyTypeFunctionFix, false); @@ -51,6 +48,7 @@ LUAU_FASTFLAGVARIABLE(LuauTypecheckIter, false); LUAU_FASTFLAGVARIABLE(LuauSuccessTypingForEqualityOperations, false) LUAU_FASTFLAGVARIABLE(LuauNoMethodLocations, false); LUAU_FASTFLAGVARIABLE(LuauAlwaysQuantify, false); +LUAU_FASTFLAGVARIABLE(LuauReportErrorsOnIndexerKeyMismatch, false) namespace Luau { @@ -305,12 +303,8 @@ ModulePtr TypeChecker::checkWithoutRecursionCheck(const SourceModule& module, Mo currentModule.reset(new Module()); currentModule->type = module.type; - - if (FFlag::LuauSeparateTypechecks) - { - currentModule->allocator = module.allocator; - currentModule->names = module.names; - } + currentModule->allocator = module.allocator; + currentModule->names = module.names; iceHandler->moduleName = module.name; @@ -338,21 +332,14 @@ ModulePtr TypeChecker::checkWithoutRecursionCheck(const SourceModule& module, Mo if (prepareModuleScope) prepareModuleScope(module.name, currentModule->getModuleScope()); - if (FFlag::LuauSeparateTypechecks) - { - try - { - checkBlock(moduleScope, *module.root); - } - catch (const TimeLimitError&) - { - currentModule->timeout = true; - } - } - else + try { checkBlock(moduleScope, *module.root); } + catch (const TimeLimitError&) + { + currentModule->timeout = true; + } if (get(follow(moduleScope->returnType))) moduleScope->returnType = addTypePack(TypePack{{}, std::nullopt}); @@ -443,7 +430,7 @@ void TypeChecker::check(const ScopePtr& scope, const AstStat& program) else ice("Unknown AstStat"); - if (FFlag::LuauSeparateTypechecks && finishTime && TimeTrace::getClock() > *finishTime) + if (finishTime && TimeTrace::getClock() > *finishTime) throw TimeLimitError(); } @@ -868,9 +855,9 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatAssign& assign) TypeId right = nullptr; - Location loc = 0 == assign.values.size - ? assign.location - : i < assign.values.size ? assign.values.data[i]->location : assign.values.data[assign.values.size - 1]->location; + Location loc = 0 == assign.values.size ? assign.location + : i < assign.values.size ? assign.values.data[i]->location + : assign.values.data[assign.values.size - 1]->location; if (valueIter != valueEnd) { @@ -1825,7 +1812,7 @@ std::optional TypeChecker::findMetatableEntry(TypeId type, std::string e } std::optional TypeChecker::getIndexTypeFromType( - const ScopePtr& scope, TypeId type, const Name& name, const Location& location, bool addErrors) + const ScopePtr& scope, TypeId type, const std::string& name, const Location& location, bool addErrors) { type = follow(type); @@ -1843,13 +1830,25 @@ std::optional TypeChecker::getIndexTypeFromType( if (TableTypeVar* tableType = getMutableTableType(type)) { - const auto& it = tableType->props.find(name); - if (it != tableType->props.end()) + if (auto it = tableType->props.find(name); it != tableType->props.end()) return it->second.type; else if (auto indexer = tableType->indexer) { - tryUnify(stringType, indexer->indexType, location); - return indexer->indexResultType; + // TODO: Property lookup should work with string singletons or unions thereof as the indexer key type. + ErrorVec errors = tryUnify(stringType, indexer->indexType, location); + + if (FFlag::LuauReportErrorsOnIndexerKeyMismatch) + { + if (errors.empty()) + return indexer->indexResultType; + + if (addErrors) + reportError(location, UnknownProperty{type, name}); + + return std::nullopt; + } + else + return indexer->indexResultType; } else if (tableType->state == TableState::Free) { @@ -1858,8 +1857,7 @@ std::optional TypeChecker::getIndexTypeFromType( return result; } - auto found = findTablePropertyRespectingMeta(type, name, location); - if (found) + if (auto found = findTablePropertyRespectingMeta(type, name, location)) return *found; } else if (const ClassTypeVar* cls = get(type)) @@ -2512,8 +2510,9 @@ TypeId TypeChecker::checkRelationalOperation( if (!matches) { - reportError(expr.location, GenericError{format("Types %s and %s cannot be compared with %s because they do not have the same metatable", - toString(lhsType).c_str(), toString(rhsType).c_str(), toString(expr.op).c_str())}); + reportError( + expr.location, GenericError{format("Types %s and %s cannot be compared with %s because they do not have the same metatable", + toString(lhsType).c_str(), toString(rhsType).c_str(), toString(expr.op).c_str())}); return errorRecoveryType(booleanType); } } @@ -2522,8 +2521,9 @@ TypeId TypeChecker::checkRelationalOperation( { if (bool(leftMetatable) != bool(rightMetatable) && leftMetatable != rightMetatable) { - reportError(expr.location, GenericError{format("Types %s and %s cannot be compared with %s because they do not have the same metatable", - toString(lhsType).c_str(), toString(rhsType).c_str(), toString(expr.op).c_str())}); + reportError( + expr.location, GenericError{format("Types %s and %s cannot be compared with %s because they do not have the same metatable", + toString(lhsType).c_str(), toString(rhsType).c_str(), toString(expr.op).c_str())}); return errorRecoveryType(booleanType); } } @@ -3636,10 +3636,7 @@ void TypeChecker::checkArgumentList( } TypePackId varPack = addTypePack(TypePackVar{TypePack{rest, argIter.tail()}}); - if (FFlag::LuauWidenIfSupertypeIsFree2) - state.tryUnify(varPack, tail); - else - state.tryUnify(tail, varPack); + state.tryUnify(varPack, tail); return; } @@ -3707,7 +3704,7 @@ ExprResult TypeChecker::checkExprPack(const ScopePtr& scope, const A } TypePackId retPack; - if (FFlag::LuauLowerBoundsCalculation || !FFlag::LuauWidenIfSupertypeIsFree2) + if (FFlag::LuauLowerBoundsCalculation) { retPack = freshTypePack(scope->level); } @@ -3868,9 +3865,7 @@ std::optional> TypeChecker::checkCallOverload(const Scope Widen widen{¤tModule->internalTypes}; for (; it != endIt; ++it) { - TypeId t = *it; - TypeId widened = widen.substitute(t).value_or(t); // Surely widening is infallible - adjustedArgTypes.push_back(addType(ConstrainedTypeVar{level, {widened}})); + adjustedArgTypes.push_back(addType(ConstrainedTypeVar{level, {widen(*it)}})); } TypePackId adjustedArgPack = addTypePack(TypePack{std::move(adjustedArgTypes), it.tail()}); @@ -3885,14 +3880,11 @@ std::optional> TypeChecker::checkCallOverload(const Scope else { TypeId r = addType(FunctionTypeVar(scope->level, argPack, retPack)); - if (FFlag::LuauWidenIfSupertypeIsFree2) - { - UnifierOptions options; - options.isFunctionCall = true; - unify(r, fn, expr.location, options); - } - else - unify(fn, r, expr.location); + + UnifierOptions options; + options.isFunctionCall = true; + unify(r, fn, expr.location, options); + return {{retPack}}; } } @@ -4375,122 +4367,6 @@ void TypeChecker::unifyWithInstantiationIfNeeded(const ScopePtr& scope, TypeId s } } -bool Instantiation::isDirty(TypeId ty) -{ - if (const FunctionTypeVar* ftv = log->getMutable(ty)) - { - if (ftv->hasNoGenerics) - return false; - - return true; - } - else - { - return false; - } -} - -bool Instantiation::isDirty(TypePackId tp) -{ - return false; -} - -bool Instantiation::ignoreChildren(TypeId ty) -{ - if (log->getMutable(ty)) - return true; - else - return false; -} - -TypeId Instantiation::clean(TypeId ty) -{ - const FunctionTypeVar* ftv = log->getMutable(ty); - LUAU_ASSERT(ftv); - - FunctionTypeVar clone = FunctionTypeVar{level, ftv->argTypes, ftv->retType, ftv->definition, ftv->hasSelf}; - clone.magicFunction = ftv->magicFunction; - clone.tags = ftv->tags; - clone.argNames = ftv->argNames; - TypeId result = addType(std::move(clone)); - - // Annoyingly, we have to do this even if there are no generics, - // to replace any generic tables. - ReplaceGenerics replaceGenerics{log, arena, level, ftv->generics, ftv->genericPacks}; - - // TODO: What to do if this returns nullopt? - // We don't have access to the error-reporting machinery - result = replaceGenerics.substitute(result).value_or(result); - - asMutable(result)->documentationSymbol = ty->documentationSymbol; - return result; -} - -TypePackId Instantiation::clean(TypePackId tp) -{ - LUAU_ASSERT(false); - return tp; -} - -bool ReplaceGenerics::ignoreChildren(TypeId ty) -{ - if (const FunctionTypeVar* ftv = log->getMutable(ty)) - { - if (ftv->hasNoGenerics) - return true; - - // We aren't recursing in the case of a generic function which - // binds the same generics. This can happen if, for example, there's recursive types. - // If T = (a,T)->T then instantiating T should produce T' = (X,T)->T not T' = (X,T')->T'. - // It's OK to use vector equality here, since we always generate fresh generics - // whenever we quantify, so the vectors overlap if and only if they are equal. - return (!generics.empty() || !genericPacks.empty()) && (ftv->generics == generics) && (ftv->genericPacks == genericPacks); - } - else - { - return false; - } -} - -bool ReplaceGenerics::isDirty(TypeId ty) -{ - if (const TableTypeVar* ttv = log->getMutable(ty)) - return ttv->state == TableState::Generic; - else if (log->getMutable(ty)) - return std::find(generics.begin(), generics.end(), ty) != generics.end(); - else - return false; -} - -bool ReplaceGenerics::isDirty(TypePackId tp) -{ - if (log->getMutable(tp)) - return std::find(genericPacks.begin(), genericPacks.end(), tp) != genericPacks.end(); - else - return false; -} - -TypeId ReplaceGenerics::clean(TypeId ty) -{ - LUAU_ASSERT(isDirty(ty)); - if (const TableTypeVar* ttv = log->getMutable(ty)) - { - TableTypeVar clone = TableTypeVar{ttv->props, ttv->indexer, level, TableState::Free}; - if (!FFlag::LuauNoMethodLocations) - clone.methodDefinitionLocations = ttv->methodDefinitionLocations; - clone.definitionModuleName = ttv->definitionModuleName; - return addType(std::move(clone)); - } - else - return addType(FreeTypeVar{level}); -} - -TypePackId ReplaceGenerics::clean(TypePackId tp) -{ - LUAU_ASSERT(isDirty(tp)); - return addTypePack(TypePackVar(FreeTypePack{level})); -} - bool Anyification::isDirty(TypeId ty) { if (ty->persistent) @@ -5295,7 +5171,7 @@ TypeId ApplyTypeFunction::clean(TypeId ty) { TypeId& arg = typeArguments[ty]; if (FFlag::LuauApplyTypeFunctionFix) - { + { LUAU_ASSERT(arg); return arg; } @@ -5309,7 +5185,7 @@ TypePackId ApplyTypeFunction::clean(TypePackId tp) { TypePackId& arg = typePackArguments[tp]; if (FFlag::LuauApplyTypeFunctionFix) - { + { LUAU_ASSERT(arg); return arg; } @@ -5837,9 +5713,6 @@ void TypeChecker::resolve(const EqPredicate& eqP, RefinementMap& refis, const Sc return; // Optimization: the other side has unknown types, so there's probably an overlap. Refining is no-op here. auto predicate = [&](TypeId option) -> std::optional { - if (sense && isUndecidable(option)) - return FFlag::LuauWeakEqConstraint ? option : eqP.type; - if (!sense && isNil(eqP.type)) return (isUndecidable(option) || !isNil(option)) ? std::optional(option) : std::nullopt; diff --git a/Analysis/src/Unifier.cpp b/Analysis/src/Unifier.cpp index 9308e9ff..414b05f4 100644 --- a/Analysis/src/Unifier.cpp +++ b/Analysis/src/Unifier.cpp @@ -21,8 +21,6 @@ LUAU_FASTFLAGVARIABLE(LuauTableSubtypingVariance2, false); LUAU_FASTFLAG(LuauLowerBoundsCalculation); LUAU_FASTFLAG(LuauErrorRecoveryType); LUAU_FASTFLAGVARIABLE(LuauSubtypingAddOptPropsToUnsealedTables, false) -LUAU_FASTFLAGVARIABLE(LuauWidenIfSupertypeIsFree2, false) -LUAU_FASTFLAGVARIABLE(LuauDifferentOrderOfUnificationDoesntMatter2, false) LUAU_FASTFLAGVARIABLE(LuauTxnLogRefreshFunctionPointers, false) namespace Luau @@ -149,8 +147,7 @@ static void promoteTypeLevels(TxnLog& log, const TypeArena* typeArena, TypeLevel return; PromoteTypeLevels ptl{log, typeArena, minLevel}; - DenseHashSet seen{nullptr}; - DEPRECATED_visitTypeVarOnce(ty, ptl, seen); + ptl.traverse(ty); } void promoteTypeLevels(TxnLog& log, const TypeArena* typeArena, TypeLevel minLevel, TypePackId tp) @@ -160,8 +157,7 @@ void promoteTypeLevels(TxnLog& log, const TypeArena* typeArena, TypeLevel minLev return; PromoteTypeLevels ptl{log, typeArena, minLevel}; - DenseHashSet seen{nullptr}; - DEPRECATED_visitTypeVarOnce(tp, ptl, seen); + ptl.traverse(tp); } struct SkipCacheForType final : TypeVarOnceVisitor @@ -172,49 +168,6 @@ struct SkipCacheForType final : TypeVarOnceVisitor { } - // TODO cycle() and operator() can be clipped with FFlagLuauUseVisitRecursionLimit - void cycle(TypeId) override {} - void cycle(TypePackId) override {} - - bool operator()(TypeId ty, const FreeTypeVar& ftv) - { - return visit(ty, ftv); - } - bool operator()(TypeId ty, const BoundTypeVar& btv) - { - return visit(ty, btv); - } - bool operator()(TypeId ty, const GenericTypeVar& gtv) - { - return visit(ty, gtv); - } - bool operator()(TypeId ty, const TableTypeVar& ttv) - { - return visit(ty, ttv); - } - bool operator()(TypePackId tp, const FreeTypePack& ftp) - { - return visit(tp, ftp); - } - bool operator()(TypePackId tp, const BoundTypePack& ftp) - { - return visit(tp, ftp); - } - bool operator()(TypePackId tp, const GenericTypePack& ftp) - { - return visit(tp, ftp); - } - template - bool operator()(TypeId ty, const T& t) - { - return visit(ty); - } - template - bool operator()(TypePackId tp, const T&) - { - return visit(tp); - } - bool visit(TypeId, const FreeTypeVar&) override { result = true; @@ -341,6 +294,16 @@ bool Widen::ignoreChildren(TypeId ty) return !log->is(ty); } +TypeId Widen::operator()(TypeId ty) +{ + return substitute(ty).value_or(ty); +} + +TypePackId Widen::operator()(TypePackId tp) +{ + return substitute(tp).value_or(tp); +} + static std::optional hasUnificationTooComplex(const ErrorVec& errors) { auto isUnificationTooComplex = [](const TypeError& te) { @@ -475,6 +438,8 @@ void Unifier::tryUnify_(TypeId subTy, TypeId superTy, bool isFunctionCall, bool if (!occursFailed) { promoteTypeLevels(log, types, superLevel, subTy); + + Widen widen{types}; log.replace(superTy, BoundTypeVar(widen(subTy))); } @@ -612,9 +577,6 @@ void Unifier::tryUnifyUnionWithType(TypeId subTy, const UnionTypeVar* uv, TypeId std::optional unificationTooComplex; std::optional firstFailedOption; - size_t count = uv->options.size(); - size_t i = 0; - for (TypeId type : uv->options) { Unifier innerState = makeChildUnifier(); @@ -630,60 +592,44 @@ void Unifier::tryUnifyUnionWithType(TypeId subTy, const UnionTypeVar* uv, TypeId failed = true; } - - if (FFlag::LuauDifferentOrderOfUnificationDoesntMatter2) - { - } - else - { - if (i == count - 1) - { - log.concat(std::move(innerState.log)); - } - - ++i; - } } // even if A | B <: T fails, we want to bind some options of T with A | B iff A | B was a subtype of that option. - if (FFlag::LuauDifferentOrderOfUnificationDoesntMatter2) - { - auto tryBind = [this, subTy](TypeId superOption) { - superOption = log.follow(superOption); + auto tryBind = [this, subTy](TypeId superOption) { + superOption = log.follow(superOption); - // just skip if the superOption is not free-ish. - auto ttv = log.getMutable(superOption); - if (!log.is(superOption) && (!ttv || ttv->state != TableState::Free)) - return; + // just skip if the superOption is not free-ish. + auto ttv = log.getMutable(superOption); + if (!log.is(superOption) && (!ttv || ttv->state != TableState::Free)) + return; - // If superOption is already present in subTy, do nothing. Nothing new has been learned, but the subtype - // test is successful. - if (auto subUnion = get(subTy)) - { - if (end(subUnion) != std::find(begin(subUnion), end(subUnion), superOption)) - return; - } - - // Since we have already checked if S <: T, checking it again will not queue up the type for replacement. - // So we'll have to do it ourselves. We assume they unified cleanly if they are still in the seen set. - if (log.haveSeen(subTy, superOption)) - { - // TODO: would it be nice for TxnLog::replace to do this? - if (log.is(superOption)) - log.bindTable(superOption, subTy); - else - log.replace(superOption, *subTy); - } - }; - - if (auto utv = log.getMutable(superTy)) + // If superOption is already present in subTy, do nothing. Nothing new has been learned, but the subtype + // test is successful. + if (auto subUnion = get(subTy)) { - for (TypeId ty : utv) - tryBind(ty); + if (end(subUnion) != std::find(begin(subUnion), end(subUnion), superOption)) + return; } - else - tryBind(superTy); + + // Since we have already checked if S <: T, checking it again will not queue up the type for replacement. + // So we'll have to do it ourselves. We assume they unified cleanly if they are still in the seen set. + if (log.haveSeen(subTy, superOption)) + { + // TODO: would it be nice for TxnLog::replace to do this? + if (log.is(superOption)) + log.bindTable(superOption, subTy); + else + log.replace(superOption, *subTy); + } + }; + + if (auto utv = log.getMutable(superTy)) + { + for (TypeId ty : utv) + tryBind(ty); } + else + tryBind(superTy); if (unificationTooComplex) reportError(*unificationTooComplex); @@ -883,7 +829,7 @@ bool Unifier::canCacheResult(TypeId subTy, TypeId superTy) auto skipCacheFor = [this](TypeId ty) { SkipCacheForType visitor{sharedState.skipCacheForType, types}; - DEPRECATED_visitTypeVarOnce(ty, visitor, sharedState.seenAny); + visitor.traverse(ty); sharedState.skipCacheForType[ty] = visitor.result; @@ -1088,6 +1034,7 @@ void Unifier::tryUnify_(TypePackId subTp, TypePackId superTp, bool isFunctionCal if (!log.getMutable(superTp)) { + Widen widen{types}; log.replace(superTp, Unifiable::Bound(widen(subTp))); } } @@ -1671,28 +1618,6 @@ void Unifier::tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) } } -TypeId Unifier::widen(TypeId ty) -{ - if (!FFlag::LuauWidenIfSupertypeIsFree2) - return ty; - - Widen widen{types}; - std::optional result = widen.substitute(ty); - // TODO: what does it mean for substitution to fail to widen? - return result.value_or(ty); -} - -TypePackId Unifier::widen(TypePackId tp) -{ - if (!FFlag::LuauWidenIfSupertypeIsFree2) - return tp; - - Widen widen{types}; - std::optional result = widen.substitute(tp); - // TODO: what does it mean for substitution to fail to widen? - return result.value_or(tp); -} - TypeId Unifier::deeplyOptional(TypeId ty, std::unordered_map seen) { ty = follow(ty); @@ -1809,10 +1734,7 @@ void Unifier::tryUnifyFreeTable(TypeId subTy, TypeId superTy) { if (auto subProp = findTablePropertyRespectingMeta(subTy, freeName)) { - if (FFlag::LuauWidenIfSupertypeIsFree2) - tryUnify_(*subProp, freeProp.type); - else - tryUnify_(freeProp.type, *subProp); + tryUnify_(*subProp, freeProp.type); /* * TypeVars are commonly cyclic, so it is entirely possible diff --git a/Ast/include/Luau/TimeTrace.h b/Ast/include/Luau/TimeTrace.h index 9f7b2bdf..be282827 100644 --- a/Ast/include/Luau/TimeTrace.h +++ b/Ast/include/Luau/TimeTrace.h @@ -1,7 +1,7 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #pragma once -#include "Common.h" +#include "Luau/Common.h" #include diff --git a/Ast/src/Parser.cpp b/Ast/src/Parser.cpp index c053e6bd..eaf19914 100644 --- a/Ast/src/Parser.cpp +++ b/Ast/src/Parser.cpp @@ -11,6 +11,8 @@ LUAU_FASTINTVARIABLE(LuauRecursionLimit, 1000) LUAU_FASTINTVARIABLE(LuauParseErrorLimit, 100) +LUAU_FASTFLAGVARIABLE(LuauParserFunctionKeywordAsTypeHelp, false) + namespace Luau { @@ -1589,6 +1591,17 @@ AstTypeOrPack Parser::parseSimpleTypeAnnotation(bool allowPack) { return parseFunctionTypeAnnotation(allowPack); } + else if (FFlag::LuauParserFunctionKeywordAsTypeHelp && lexer.current().type == Lexeme::ReservedFunction) + { + Location location = lexer.current().location; + + nextLexeme(); + + return {reportTypeAnnotationError(location, {}, /*isMissing*/ false, + "Using 'function' as a type annotation is not supported, consider replacing with a function type annotation e.g. '(...any) -> " + "...any'"), + {}}; + } else { Location location = lexer.current().location; diff --git a/CMakeLists.txt b/CMakeLists.txt index ea352309..c624a132 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -19,9 +19,11 @@ if(LUAU_STATIC_CRT) endif() project(Luau LANGUAGES CXX C) +add_library(Luau.Common INTERFACE) add_library(Luau.Ast STATIC) add_library(Luau.Compiler STATIC) add_library(Luau.Analysis STATIC) +add_library(Luau.CodeGen STATIC) add_library(Luau.VM STATIC) add_library(isocline STATIC) @@ -48,8 +50,11 @@ endif() include(Sources.cmake) +target_include_directories(Luau.Common INTERFACE Common/include) + target_compile_features(Luau.Ast PUBLIC cxx_std_17) target_include_directories(Luau.Ast PUBLIC Ast/include) +target_link_libraries(Luau.Ast PUBLIC Luau.Common) target_compile_features(Luau.Compiler PUBLIC cxx_std_17) target_include_directories(Luau.Compiler PUBLIC Compiler/include) @@ -59,8 +64,13 @@ target_compile_features(Luau.Analysis PUBLIC cxx_std_17) target_include_directories(Luau.Analysis PUBLIC Analysis/include) target_link_libraries(Luau.Analysis PUBLIC Luau.Ast) +target_compile_features(Luau.CodeGen PRIVATE cxx_std_17) +target_include_directories(Luau.CodeGen PUBLIC CodeGen/include) +target_link_libraries(Luau.CodeGen PUBLIC Luau.Common) + target_compile_features(Luau.VM PRIVATE cxx_std_11) target_include_directories(Luau.VM PUBLIC VM/include) +target_link_libraries(Luau.VM PUBLIC Luau.Common) target_include_directories(isocline PUBLIC extern/isocline/include) @@ -101,6 +111,7 @@ endif() target_compile_options(Luau.Ast PRIVATE ${LUAU_OPTIONS}) target_compile_options(Luau.Analysis PRIVATE ${LUAU_OPTIONS}) +target_compile_options(Luau.CodeGen PRIVATE ${LUAU_OPTIONS}) target_compile_options(Luau.VM PRIVATE ${LUAU_OPTIONS}) target_compile_options(isocline PRIVATE ${LUAU_OPTIONS} ${ISOCLINE_OPTIONS}) @@ -120,6 +131,7 @@ endif() if(MSVC) target_link_options(Luau.Ast INTERFACE /NATVIS:${CMAKE_CURRENT_SOURCE_DIR}/tools/natvis/Ast.natvis) target_link_options(Luau.Analysis INTERFACE /NATVIS:${CMAKE_CURRENT_SOURCE_DIR}/tools/natvis/Analysis.natvis) + target_link_options(Luau.CodeGen INTERFACE /NATVIS:${CMAKE_CURRENT_SOURCE_DIR}/tools/natvis/CodeGen.natvis) target_link_options(Luau.VM INTERFACE /NATVIS:${CMAKE_CURRENT_SOURCE_DIR}/tools/natvis/VM.natvis) endif() @@ -127,6 +139,7 @@ endif() if(MSVC_IDE) target_sources(Luau.Ast PRIVATE tools/natvis/Ast.natvis) target_sources(Luau.Analysis PRIVATE tools/natvis/Analysis.natvis) + target_sources(Luau.CodeGen PRIVATE tools/natvis/CodeGen.natvis) target_sources(Luau.VM PRIVATE tools/natvis/VM.natvis) endif() @@ -154,7 +167,7 @@ endif() if(LUAU_BUILD_TESTS) target_compile_options(Luau.UnitTest PRIVATE ${LUAU_OPTIONS}) target_include_directories(Luau.UnitTest PRIVATE extern) - target_link_libraries(Luau.UnitTest PRIVATE Luau.Analysis Luau.Compiler) + target_link_libraries(Luau.UnitTest PRIVATE Luau.Analysis Luau.Compiler Luau.CodeGen) target_compile_options(Luau.Conformance PRIVATE ${LUAU_OPTIONS}) target_include_directories(Luau.Conformance PRIVATE extern) diff --git a/CodeGen/include/Luau/AssemblyBuilderX64.h b/CodeGen/include/Luau/AssemblyBuilderX64.h new file mode 100644 index 00000000..c5979d3c --- /dev/null +++ b/CodeGen/include/Luau/AssemblyBuilderX64.h @@ -0,0 +1,169 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +#pragma once + +#include "Luau/Common.h" +#include "Luau/Condition.h" +#include "Luau/Label.h" +#include "Luau/OperandX64.h" +#include "Luau/RegisterX64.h" + +#include +#include + +namespace Luau +{ +namespace CodeGen +{ + +class AssemblyBuilderX64 +{ +public: + explicit AssemblyBuilderX64(bool logText); + ~AssemblyBuilderX64(); + + // Base two operand instructions with 9 opcode selection + void add(OperandX64 lhs, OperandX64 rhs); + void sub(OperandX64 lhs, OperandX64 rhs); + void cmp(OperandX64 lhs, OperandX64 rhs); + void and_(OperandX64 lhs, OperandX64 rhs); + void or_(OperandX64 lhs, OperandX64 rhs); + void xor_(OperandX64 lhs, OperandX64 rhs); + + // Binary shift instructions with special rhs handling + void sal(OperandX64 lhs, OperandX64 rhs); + void sar(OperandX64 lhs, OperandX64 rhs); + void shl(OperandX64 lhs, OperandX64 rhs); + void shr(OperandX64 lhs, OperandX64 rhs); + + // Two operand mov instruction has additional specialized encodings + void mov(OperandX64 lhs, OperandX64 rhs); + void mov64(RegisterX64 lhs, int64_t imm); + + // Base one operand instruction with 2 opcode selection + void div(OperandX64 op); + void idiv(OperandX64 op); + void mul(OperandX64 op); + void neg(OperandX64 op); + void not_(OperandX64 op); + + void test(OperandX64 lhs, OperandX64 rhs); + void lea(OperandX64 lhs, OperandX64 rhs); + + void push(OperandX64 op); + void pop(OperandX64 op); + void ret(); + + // Control flow + void jcc(Condition cond, Label& label); + void jmp(Label& label); + void jmp(OperandX64 op); + + // AVX + void vaddpd(OperandX64 dst, OperandX64 src1, OperandX64 src2); + void vaddps(OperandX64 dst, OperandX64 src1, OperandX64 src2); + void vaddsd(OperandX64 dst, OperandX64 src1, OperandX64 src2); + void vaddss(OperandX64 dst, OperandX64 src1, OperandX64 src2); + + void vsqrtpd(OperandX64 dst, OperandX64 src); + void vsqrtps(OperandX64 dst, OperandX64 src); + void vsqrtsd(OperandX64 dst, OperandX64 src1, OperandX64 src2); + void vsqrtss(OperandX64 dst, OperandX64 src1, OperandX64 src2); + + void vmovsd(OperandX64 dst, OperandX64 src); + void vmovsd(OperandX64 dst, OperandX64 src1, OperandX64 src2); + void vmovss(OperandX64 dst, OperandX64 src); + void vmovss(OperandX64 dst, OperandX64 src1, OperandX64 src2); + void vmovapd(OperandX64 dst, OperandX64 src); + void vmovaps(OperandX64 dst, OperandX64 src); + void vmovupd(OperandX64 dst, OperandX64 src); + void vmovups(OperandX64 dst, OperandX64 src); + + // Run final checks + void finalize(); + + // Places a label at current location and returns it + Label setLabel(); + + // Assigns label position to the current location + void setLabel(Label& label); + + // Constant allocation (uses rip-relative addressing) + OperandX64 i64(int64_t value); + OperandX64 f32(float value); + OperandX64 f64(double value); + OperandX64 f32x4(float x, float y, float z, float w); + + // Resulting data and code that need to be copied over one after the other + // The *end* of 'data' has to be aligned to 16 bytes, this will also align 'code' + std::vector data; + std::vector code; + + std::string text; + +private: + // Instruction archetypes + void placeBinary(const char* name, OperandX64 lhs, OperandX64 rhs, uint8_t codeimm8, uint8_t codeimm, uint8_t codeimmImm8, uint8_t code8rev, + uint8_t coderev, uint8_t code8, uint8_t code, uint8_t opreg); + void placeBinaryRegMemAndImm(OperandX64 lhs, OperandX64 rhs, uint8_t code8, uint8_t code, uint8_t codeImm8, uint8_t opreg); + void placeBinaryRegAndRegMem(OperandX64 lhs, OperandX64 rhs, uint8_t code8, uint8_t code); + void placeBinaryRegMemAndReg(OperandX64 lhs, OperandX64 rhs, uint8_t code8, uint8_t code); + + void placeUnaryModRegMem(const char* name, OperandX64 op, uint8_t code8, uint8_t code, uint8_t opreg); + + void placeShift(const char* name, OperandX64 lhs, OperandX64 rhs, uint8_t opreg); + + void placeJcc(const char* name, Label& label, uint8_t cc); + + void placeAvx(const char* name, OperandX64 dst, OperandX64 src, uint8_t code, bool setW, uint8_t mode, uint8_t prefix); + void placeAvx(const char* name, OperandX64 dst, OperandX64 src, uint8_t code, uint8_t coderev, bool setW, uint8_t mode, uint8_t prefix); + void placeAvx(const char* name, OperandX64 dst, OperandX64 src1, OperandX64 src2, uint8_t code, bool setW, uint8_t mode, uint8_t prefix); + + // Instruction components + void placeRegAndModRegMem(OperandX64 lhs, OperandX64 rhs); + void placeModRegMem(OperandX64 rhs, uint8_t regop); + void placeRex(RegisterX64 op); + void placeRex(OperandX64 op); + void placeRex(RegisterX64 lhs, OperandX64 rhs); + void placeVex(OperandX64 dst, OperandX64 src1, OperandX64 src2, bool setW, uint8_t mode, uint8_t prefix); + void placeImm8Or32(int32_t imm); + void placeImm8(int32_t imm); + void placeImm32(int32_t imm); + void placeImm64(int64_t imm); + void placeLabel(Label& label); + void place(uint8_t byte); + + void commit(); + LUAU_NOINLINE void extend(); + uint32_t getCodeSize(); + + // Data + size_t allocateData(size_t size, size_t align); + + // Logging of assembly in text form (Intel asm with VS disassembly formatting) + LUAU_NOINLINE void log(const char* opcode); + LUAU_NOINLINE void log(const char* opcode, OperandX64 op); + LUAU_NOINLINE void log(const char* opcode, OperandX64 op1, OperandX64 op2); + LUAU_NOINLINE void log(const char* opcode, OperandX64 op1, OperandX64 op2, OperandX64 op3); + LUAU_NOINLINE void log(Label label); + LUAU_NOINLINE void log(const char* opcode, Label label); + void log(OperandX64 op); + void logAppend(const char* fmt, ...); + + const char* getSizeName(SizeX64 size); + const char* getRegisterName(RegisterX64 reg); + + uint32_t nextLabel = 1; + std::vector(a) -> a" == toString(idType)); +} + +#if 1 +TEST_CASE_FIXTURE(ConstraintGraphBuilderFixture, "proper_let_generalization") +{ + AstStatBlock* block = parse(R"( + local function a(c) + local function d(e) + return c + end + + return d + end + + local b = a(5) + )"); + + cgb.visit(block); + + ToStringOptions opts; + + ConstraintSolver cs{&arena, cgb.rootScope}; + + cs.run(); + + TypeId idType = requireBinding(cgb.rootScope, "b"); + + CHECK("(a) -> number" == toString(idType, opts)); +} +#endif + +TEST_SUITE_END(); diff --git a/tests/Fixture.cpp b/tests/Fixture.cpp index 03f3e15c..232ec2de 100644 --- a/tests/Fixture.cpp +++ b/tests/Fixture.cpp @@ -17,6 +17,8 @@ static const char* mainModuleName = "MainModule"; +LUAU_FASTFLAG(DebugLuauDeferredConstraintResolution); + namespace Luau { @@ -249,7 +251,10 @@ std::optional Fixture::getType(const std::string& name) ModulePtr module = getMainModule(); REQUIRE(module); - return lookupName(module->getModuleScope(), name); + if (FFlag::DebugLuauDeferredConstraintResolution) + return linearSearchForBinding(module->getModuleScope2(), name.c_str()); + else + return lookupName(module->getModuleScope(), name); } TypeId Fixture::requireType(const std::string& name) @@ -421,6 +426,12 @@ BuiltinsFixture::BuiltinsFixture(bool freeze, bool prepareAutocomplete) Luau::freeze(frontend.typeCheckerForAutocomplete.globalTypes); } +ConstraintGraphBuilderFixture::ConstraintGraphBuilderFixture() + : Fixture() + , forceTheFlag{"DebugLuauDeferredConstraintResolution", true} +{ +} + ModuleName fromString(std::string_view name) { return ModuleName(name); @@ -460,4 +471,27 @@ std::optional lookupName(ScopePtr scope, const std::string& name) return std::nullopt; } +std::optional linearSearchForBinding(Scope2* scope, const char* name) +{ + while (scope) + { + for (const auto& [n, ty] : scope->bindings) + { + if (n.astName() == name) + return ty; + } + + scope = scope->parent; + } + + return std::nullopt; +} + +void dump(const std::vector& constraints) +{ + ToStringOptions opts; + for (const auto& c : constraints) + printf("%s\n", toString(c, opts).c_str()); +} + } // namespace Luau diff --git a/tests/Fixture.h b/tests/Fixture.h index 901f7d42..ffcd4b9e 100644 --- a/tests/Fixture.h +++ b/tests/Fixture.h @@ -2,6 +2,7 @@ #pragma once #include "Luau/Config.h" +#include "Luau/ConstraintGraphBuilder.h" #include "Luau/FileResolver.h" #include "Luau/Frontend.h" #include "Luau/IostreamHelpers.h" @@ -156,6 +157,16 @@ struct BuiltinsFixture : Fixture BuiltinsFixture(bool freeze = true, bool prepareAutocomplete = false); }; +struct ConstraintGraphBuilderFixture : Fixture +{ + TypeArena arena; + ConstraintGraphBuilder cgb{&arena}; + + ScopedFastFlag forceTheFlag; + + ConstraintGraphBuilderFixture(); +}; + ModuleName fromString(std::string_view name); template @@ -175,9 +186,12 @@ bool isInArena(TypeId t, const TypeArena& arena); void dumpErrors(const ModulePtr& module); void dumpErrors(const Module& module); void dump(const std::string& name, TypeId ty); +void dump(const std::vector& constraints); std::optional lookupName(ScopePtr scope, const std::string& name); // Warning: This function runs in O(n**2) +std::optional linearSearchForBinding(Scope2* scope, const char* name); + } // namespace Luau #define LUAU_REQUIRE_ERRORS(result) \ diff --git a/tests/Frontend.test.cpp b/tests/Frontend.test.cpp index 33b81be8..c0554669 100644 --- a/tests/Frontend.test.cpp +++ b/tests/Frontend.test.cpp @@ -1031,8 +1031,6 @@ return false; TEST_CASE("check_without_builtin_next") { - ScopedFastFlag luauDoNotRelyOnNextBinding{"LuauDoNotRelyOnNextBinding", true}; - TestFileResolver fileResolver; TestConfigResolver configResolver; Frontend frontend(&fileResolver, &configResolver); diff --git a/tests/NotNull.test.cpp b/tests/NotNull.test.cpp new file mode 100644 index 00000000..1a323c85 --- /dev/null +++ b/tests/NotNull.test.cpp @@ -0,0 +1,116 @@ +#include "Luau/NotNull.h" + +#include "doctest.h" + +#include +#include +#include + +using Luau::NotNull; + +namespace +{ + +struct Test +{ + int x; + float y; + + static int count; + Test() + { + ++count; + } + + ~Test() + { + --count; + } +}; + +int Test::count = 0; + +} + +int foo(NotNull p) +{ + return *p; +} + +void bar(int* q) +{} + +TEST_SUITE_BEGIN("NotNull"); + +TEST_CASE("basic_stuff") +{ + NotNull a = NotNull{new int(55)}; // Does runtime test + NotNull b{new int(55)}; // As above + // NotNull c = new int(55); // Nope. Mildly regrettable, but implicit conversion from T* to NotNull in the general case is not good. + + // a = nullptr; // nope + + NotNull d = a; // No runtime test. a is known not to be null. + + int e = *d; + *d = 1; + CHECK(e == 55); + + const NotNull f = d; + *f = 5; // valid: there is a difference between const NotNull and NotNull + // f = a; // nope + + CHECK_EQ(a, d); + CHECK(a != b); + + NotNull g(a); + CHECK(g == a); + + // *g = 123; // nope + + (void)f; + + NotNull t{new Test}; + t->x = 5; + t->y = 3.14f; + + const NotNull u = t; + // u->x = 44; // nope + int v = u->x; + CHECK(v == 5); + + bar(a); + + // a++; // nope + // a[41]; // nope + // a + 41; // nope + // a - 41; // nope + + delete a; + delete b; + delete t; + + CHECK_EQ(0, Test::count); +} + +TEST_CASE("hashable") +{ + std::unordered_map, const char*> map; + NotNull a{new int(8)}; + NotNull b{new int(10)}; + + std::string hello = "hello"; + std::string world = "world"; + + map[a] = hello.c_str(); + map[b] = world.c_str(); + + CHECK_EQ(2, map.size()); + CHECK_EQ(hello.c_str(), map[a]); + CHECK_EQ(world.c_str(), map[b]); + + delete a; + delete b; +} + +TEST_SUITE_END(); diff --git a/tests/ToString.test.cpp b/tests/ToString.test.cpp index b854bc51..4d9fad14 100644 --- a/tests/ToString.test.cpp +++ b/tests/ToString.test.cpp @@ -505,7 +505,6 @@ TEST_CASE_FIXTURE(Fixture, "self_recursive_instantiated_param") TEST_CASE_FIXTURE(Fixture, "toStringNamedFunction_id") { - ScopedFastFlag flag{"LuauDocFuncParameters", true}; CheckResult result = check(R"( local function id(x) return x end )"); @@ -518,7 +517,6 @@ TEST_CASE_FIXTURE(Fixture, "toStringNamedFunction_id") TEST_CASE_FIXTURE(Fixture, "toStringNamedFunction_map") { - ScopedFastFlag flag{"LuauDocFuncParameters", true}; CheckResult result = check(R"( local function map(arr, fn) local t = {} @@ -537,7 +535,6 @@ TEST_CASE_FIXTURE(Fixture, "toStringNamedFunction_map") TEST_CASE_FIXTURE(Fixture, "toStringNamedFunction_generic_pack") { - ScopedFastFlag flag{"LuauDocFuncParameters", true}; CheckResult result = check(R"( local function f(a: number, b: string) end local function test(...: T...): U... @@ -554,7 +551,6 @@ TEST_CASE_FIXTURE(Fixture, "toStringNamedFunction_generic_pack") TEST_CASE("toStringNamedFunction_unit_f") { - ScopedFastFlag flag{"LuauDocFuncParameters", true}; TypePackVar empty{TypePack{}}; FunctionTypeVar ftv{&empty, &empty, {}, false}; CHECK_EQ("f(): ()", toStringNamedFunction("f", ftv)); @@ -562,7 +558,6 @@ TEST_CASE("toStringNamedFunction_unit_f") TEST_CASE_FIXTURE(Fixture, "toStringNamedFunction_variadics") { - ScopedFastFlag flag{"LuauDocFuncParameters", true}; CheckResult result = check(R"( local function f(x: a, ...): (a, a, b...) return x, x, ... @@ -577,7 +572,6 @@ TEST_CASE_FIXTURE(Fixture, "toStringNamedFunction_variadics") TEST_CASE_FIXTURE(Fixture, "toStringNamedFunction_variadics2") { - ScopedFastFlag flag{"LuauDocFuncParameters", true}; CheckResult result = check(R"( local function f(): ...number return 1, 2, 3 @@ -592,7 +586,6 @@ TEST_CASE_FIXTURE(Fixture, "toStringNamedFunction_variadics2") TEST_CASE_FIXTURE(Fixture, "toStringNamedFunction_variadics3") { - ScopedFastFlag flag{"LuauDocFuncParameters", true}; CheckResult result = check(R"( local function f(): (string, ...number) return 'a', 1, 2, 3 @@ -607,7 +600,6 @@ TEST_CASE_FIXTURE(Fixture, "toStringNamedFunction_variadics3") TEST_CASE_FIXTURE(Fixture, "toStringNamedFunction_type_annotation_has_partial_argnames") { - ScopedFastFlag flag{"LuauDocFuncParameters", true}; CheckResult result = check(R"( local f: (number, y: number) -> number )"); @@ -620,7 +612,6 @@ TEST_CASE_FIXTURE(Fixture, "toStringNamedFunction_type_annotation_has_partial_ar TEST_CASE_FIXTURE(Fixture, "toStringNamedFunction_hide_type_params") { - ScopedFastFlag flag{"LuauDocFuncParameters", true}; CheckResult result = check(R"( local function f(x: T, g: (T) -> U)): () end @@ -636,8 +627,6 @@ TEST_CASE_FIXTURE(Fixture, "toStringNamedFunction_hide_type_params") TEST_CASE_FIXTURE(Fixture, "toStringNamedFunction_overrides_param_names") { - ScopedFastFlag flag{"LuauDocFuncParameters", true}; - CheckResult result = check(R"( local function test(a, b : string, ... : number) return a end )"); @@ -665,7 +654,6 @@ TEST_CASE_FIXTURE(Fixture, "pick_distinct_names_for_mixed_explicit_and_implicit_ TEST_CASE_FIXTURE(Fixture, "toStringNamedFunction_include_self_param") { - ScopedFastFlag flag{"LuauDocFuncParameters", true}; CheckResult result = check(R"( local foo = {} function foo:method(arg: string): () @@ -682,7 +670,6 @@ TEST_CASE_FIXTURE(Fixture, "toStringNamedFunction_include_self_param") TEST_CASE_FIXTURE(Fixture, "toStringNamedFunction_hide_self_param") { - ScopedFastFlag flag{"LuauDocFuncParameters", true}; CheckResult result = check(R"( local foo = {} function foo:method(arg: string): () diff --git a/tests/TypeInfer.classes.test.cpp b/tests/TypeInfer.classes.test.cpp index d90129d7..6f4191e3 100644 --- a/tests/TypeInfer.classes.test.cpp +++ b/tests/TypeInfer.classes.test.cpp @@ -470,8 +470,6 @@ caused by: TEST_CASE_FIXTURE(ClassFixture, "class_type_mismatch_with_name_conflict") { - ScopedFastFlag luauClassDefinitionModuleInError{"LuauClassDefinitionModuleInError", true}; - CheckResult result = check(R"( local i = ChildClass.New() type ChildClass = { x: number } diff --git a/tests/TypeInfer.loops.test.cpp b/tests/TypeInfer.loops.test.cpp index a3cae3de..4444cd66 100644 --- a/tests/TypeInfer.loops.test.cpp +++ b/tests/TypeInfer.loops.test.cpp @@ -78,8 +78,6 @@ TEST_CASE_FIXTURE(Fixture, "for_in_with_an_iterator_of_type_any") TEST_CASE_FIXTURE(Fixture, "for_in_loop_should_fail_with_non_function_iterator") { - ScopedFastFlag luauDoNotRelyOnNextBinding{"LuauDoNotRelyOnNextBinding", true}; - CheckResult result = check(R"( local foo = "bar" for i, v in foo do diff --git a/tests/Variant.test.cpp b/tests/Variant.test.cpp index fcf37875..aa0731ca 100644 --- a/tests/Variant.test.cpp +++ b/tests/Variant.test.cpp @@ -13,6 +13,25 @@ struct Foo int x = 42; }; +struct Bar +{ + explicit Bar(int x) + : prop(x * 2) + { + ++count; + } + + ~Bar() + { + --count; + } + + int prop; + static int count; +}; + +int Bar::count = 0; + TEST_SUITE_BEGIN("Variant"); TEST_CASE("DefaultCtor") @@ -46,6 +65,29 @@ TEST_CASE("Create") CHECK(get_if(&v3)->x == 3); } +TEST_CASE("Emplace") +{ + { + Variant v1; + + CHECK(0 == Bar::count); + int& i = v1.emplace(5); + CHECK(5 == i); + + CHECK(0 == Bar::count); + + CHECK(get_if(&v1) == &i); + + Bar& bar = v1.emplace(11); + CHECK(22 == bar.prop); + CHECK(1 == Bar::count); + + CHECK(get_if(&v1) == &bar); + } + + CHECK(0 == Bar::count); +} + TEST_CASE("NonPOD") { // initialize (copy) diff --git a/tools/natvis/CodeGen.natvis b/tools/natvis/CodeGen.natvis index 47ff0db1..5ff6e143 100644 --- a/tools/natvis/CodeGen.natvis +++ b/tools/natvis/CodeGen.natvis @@ -2,7 +2,7 @@ - noreg + noreg rip al @@ -36,14 +36,20 @@ - {reg} - {mem.size,en} ptr[{mem.base} + {mem.index}*{(int)mem.scale,d} + {disp}] + {base} + {memSize,en} ptr[{base} + {index}*{(int)scale,d} + {imm}] + {memSize,en} ptr[{index}*{(int)scale,d} + {imm}] + {memSize,en} ptr[{base} + {imm}] + {memSize,en} ptr[{imm}] {imm} - reg - mem + base imm - disp + memSize + base + index + scale + imm From 55a026811ae6c9edc1695a192a577810dbb0e628 Mon Sep 17 00:00:00 2001 From: rblanckaert <63755228+rblanckaert@users.noreply.github.com> Date: Fri, 3 Jun 2022 15:15:45 -0700 Subject: [PATCH 264/399] Sync to upstream/release/530 (#517) * Run clang-format * Contains a preliminary implementation of deferred constraint resolution * Reduce stack usage by some recursive functions * Fix a bug when smartCloning a BoundTypeVar * Remove some GC related flags from VM --- Analysis/include/Luau/Clone.h | 2 + .../include/Luau/ConstraintGraphBuilder.h | 162 ++++++++++ Analysis/include/Luau/ConstraintSolver.h | 106 ++++++ Analysis/include/Luau/Frontend.h | 2 + Analysis/include/Luau/Module.h | 3 + Analysis/include/Luau/NotNull.h | 75 +++++ Analysis/include/Luau/Quantify.h | 3 + Analysis/include/Luau/Symbol.h | 3 + Analysis/include/Luau/ToString.h | 8 + Analysis/include/Luau/TypeVar.h | 5 + Analysis/include/Luau/TypedAllocator.h | 6 + Analysis/include/Luau/Unifiable.h | 8 +- Analysis/include/Luau/Variant.h | 14 + Analysis/src/Clone.cpp | 64 ++++ Analysis/src/ConstraintGraphBuilder.cpp | 300 +++++++++++++++++ Analysis/src/ConstraintSolver.cpp | 306 ++++++++++++++++++ Analysis/src/EmbeddedBuiltinDefinitions.cpp | 2 +- Analysis/src/Frontend.cpp | 23 +- Analysis/src/Module.cpp | 65 +++- Analysis/src/Normalize.cpp | 30 +- Analysis/src/Quantify.cpp | 63 +++- Analysis/src/Substitution.cpp | 59 +--- Analysis/src/ToString.cpp | 129 +++++--- Analysis/src/TypeInfer.cpp | 10 +- Analysis/src/TypeVar.cpp | 5 +- Analysis/src/Unifiable.cpp | 13 +- Sources.cmake | 12 +- VM/src/lapi.cpp | 30 +- VM/src/ldo.cpp | 2 +- VM/src/lgc.cpp | 11 +- tests/Compiler.test.cpp | 1 - tests/Conformance.test.cpp | 211 +++++++----- tests/ConstraintGraphBuilder.test.cpp | 107 ++++++ tests/ConstraintSolver.test.cpp | 87 +++++ tests/Fixture.cpp | 36 ++- tests/Fixture.h | 14 + tests/Frontend.test.cpp | 2 - tests/NotNull.test.cpp | 116 +++++++ tests/ToString.test.cpp | 13 - tests/TypeInfer.classes.test.cpp | 2 - tests/TypeInfer.loops.test.cpp | 2 - tests/Variant.test.cpp | 42 +++ tools/natvis/CodeGen.natvis | 18 +- 43 files changed, 1876 insertions(+), 296 deletions(-) create mode 100644 Analysis/include/Luau/ConstraintGraphBuilder.h create mode 100644 Analysis/include/Luau/ConstraintSolver.h create mode 100644 Analysis/include/Luau/NotNull.h create mode 100644 Analysis/src/ConstraintGraphBuilder.cpp create mode 100644 Analysis/src/ConstraintSolver.cpp create mode 100644 tests/ConstraintGraphBuilder.test.cpp create mode 100644 tests/ConstraintSolver.test.cpp create mode 100644 tests/NotNull.test.cpp diff --git a/Analysis/include/Luau/Clone.h b/Analysis/include/Luau/Clone.h index 9fcbce04..548a58f5 100644 --- a/Analysis/include/Luau/Clone.h +++ b/Analysis/include/Luau/Clone.h @@ -25,4 +25,6 @@ TypePackId clone(TypePackId tp, TypeArena& dest, CloneState& cloneState); TypeId clone(TypeId tp, TypeArena& dest, CloneState& cloneState); TypeFun clone(const TypeFun& typeFun, TypeArena& dest, CloneState& cloneState); +TypeId shallowClone(TypeId ty, TypeArena& dest, const TxnLog* log); + } // namespace Luau diff --git a/Analysis/include/Luau/ConstraintGraphBuilder.h b/Analysis/include/Luau/ConstraintGraphBuilder.h new file mode 100644 index 00000000..4234f2f6 --- /dev/null +++ b/Analysis/include/Luau/ConstraintGraphBuilder.h @@ -0,0 +1,162 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details + +#pragma once + +#include +#include + +#include "Luau/Ast.h" +#include "Luau/Module.h" +#include "Luau/Symbol.h" +#include "Luau/TypeVar.h" +#include "Luau/Variant.h" + +namespace Luau +{ + +struct Scope2; + +// subType <: superType +struct SubtypeConstraint +{ + TypeId subType; + TypeId superType; +}; + +// subPack <: superPack +struct PackSubtypeConstraint +{ + TypePackId subPack; + TypePackId superPack; +}; + +// subType ~ gen superType +struct GeneralizationConstraint +{ + TypeId subType; + TypeId superType; + Scope2* scope; +}; + +// subType ~ inst superType +struct InstantiationConstraint +{ + TypeId subType; + TypeId superType; +}; + +using ConstraintV = Variant; +using ConstraintPtr = std::unique_ptr; + +struct Constraint +{ + Constraint(ConstraintV&& c); + Constraint(ConstraintV&& c, std::vector dependencies); + + Constraint(const Constraint&) = delete; + Constraint& operator=(const Constraint&) = delete; + + ConstraintV c; + std::vector dependencies; +}; + +inline Constraint& asMutable(const Constraint& c) +{ + return const_cast(c); +} + +template +T* getMutable(Constraint& c) +{ + return ::Luau::get_if(&c.c); +} + +template +const T* get(const Constraint& c) +{ + return getMutable(asMutable(c)); +} + +struct Scope2 +{ + // The parent scope of this scope. Null if there is no parent (i.e. this + // is the module-level scope). + Scope2* parent = nullptr; + // All the children of this scope. + std::vector children; + std::unordered_map bindings; // TODO: I think this can be a DenseHashMap + TypePackId returnType; + // All constraints belonging to this scope. + std::vector constraints; + + std::optional lookup(Symbol sym); +}; + +struct ConstraintGraphBuilder +{ + // A list of all the scopes in the module. This vector holds ownership of the + // scope pointers; the scopes themselves borrow pointers to other scopes to + // define the scope hierarchy. + std::vector>> scopes; + SingletonTypes& singletonTypes; + TypeArena* const arena; + // The root scope of the module we're generating constraints for. + Scope2* rootScope; + + explicit ConstraintGraphBuilder(TypeArena* arena); + + /** + * Fabricates a new free type belonging to a given scope. + * @param scope the scope the free type belongs to. Must not be null. + */ + TypeId freshType(Scope2* scope); + + /** + * Fabricates a new free type pack belonging to a given scope. + * @param scope the scope the free type pack belongs to. Must not be null. + */ + TypePackId freshTypePack(Scope2* scope); + + /** + * Fabricates a scope that is a child of another scope. + * @param location the lexical extent of the scope in the source code. + * @param parent the parent scope of the new scope. Must not be null. + */ + Scope2* childScope(Location location, Scope2* parent); + + /** + * Adds a new constraint with no dependencies to a given scope. + * @param scope the scope to add the constraint to. Must not be null. + * @param cv the constraint variant to add. + */ + void addConstraint(Scope2* scope, ConstraintV cv); + + /** + * Adds a constraint to a given scope. + * @param scope the scope to add the constraint to. Must not be null. + * @param c the constraint to add. + */ + void addConstraint(Scope2* scope, std::unique_ptr c); + + /** + * The entry point to the ConstraintGraphBuilder. This will construct a set + * of scopes, constraints, and free types that can be solved later. + * @param block the root block to generate constraints for. + */ + void visit(AstStatBlock* block); + + void visit(Scope2* scope, AstStat* stat); + void visit(Scope2* scope, AstStatBlock* block); + void visit(Scope2* scope, AstStatLocal* local); + void visit(Scope2* scope, AstStatLocalFunction* local); + void visit(Scope2* scope, AstStatReturn* local); + + TypePackId checkPack(Scope2* scope, AstArray exprs); + TypePackId checkPack(Scope2* scope, AstExpr* expr); + + TypeId check(Scope2* scope, AstExpr* expr); +}; + +std::vector collectConstraints(Scope2* rootScope); + +} // namespace Luau diff --git a/Analysis/include/Luau/ConstraintSolver.h b/Analysis/include/Luau/ConstraintSolver.h new file mode 100644 index 00000000..85006e68 --- /dev/null +++ b/Analysis/include/Luau/ConstraintSolver.h @@ -0,0 +1,106 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details + +#pragma once + +#include "Luau/Error.h" +#include "Luau/Variant.h" +#include "Luau/ConstraintGraphBuilder.h" +#include "Luau/TypeVar.h" + +#include + +namespace Luau +{ + +// TypeId, TypePackId, or Constraint*. It is impossible to know which, but we +// never dereference this pointer. +using BlockedConstraintId = const void*; + +struct ConstraintSolver +{ + TypeArena* arena; + InternalErrorReporter iceReporter; + // The entire set of constraints that the solver is trying to resolve. + std::vector constraints; + Scope2* rootScope; + std::vector errors; + + // This includes every constraint that has not been fully solved. + // A constraint can be both blocked and unsolved, for instance. + std::unordered_set unsolvedConstraints; + + // A mapping of constraint pointer to how many things the constraint is + // blocked on. Can be empty or 0 for constraints that are not blocked on + // anything. + std::unordered_map blockedConstraints; + // A mapping of type/pack pointers to the constraints they block. + std::unordered_map> blocked; + + explicit ConstraintSolver(TypeArena* arena, Scope2* rootScope); + + /** + * Attempts to dispatch all pending constraints and reach a type solution + * that satisfies all of the constraints, recording any errors that are + * encountered. + **/ + void run(); + + bool done(); + + bool tryDispatch(const Constraint* c); + bool tryDispatch(const SubtypeConstraint& c); + bool tryDispatch(const PackSubtypeConstraint& c); + bool tryDispatch(const GeneralizationConstraint& c); + bool tryDispatch(const InstantiationConstraint& c, const Constraint* constraint); + + /** + * Marks a constraint as being blocked on a type or type pack. The constraint + * solver will not attempt to dispatch blocked constraints until their + * dependencies have made progress. + * @param target the type or type pack pointer that the constraint is blocked on. + * @param constraint the constraint to block. + **/ + void block_(BlockedConstraintId target, const Constraint* constraint); + void block(const Constraint* target, const Constraint* constraint); + void block(TypeId target, const Constraint* constraint); + void block(TypePackId target, const Constraint* constraint); + + /** + * Informs the solver that progress has been made on a type or type pack. The + * solver will wake up all constraints that are blocked on the type or type pack, + * and will resume attempting to dispatch them. + * @param progressed the type or type pack pointer that has progressed. + **/ + void unblock_(BlockedConstraintId progressed); + void unblock(const Constraint* progressed); + void unblock(TypeId progressed); + void unblock(TypePackId progressed); + + /** + * Returns whether the constraint is blocked on anything. + * @param constraint the constraint to check. + */ + bool isBlocked(const Constraint* constraint); + + void reportErrors(const std::vector& errors); + + /** + * Creates a new Unifier and performs a single unification operation. Commits + * the result and reports errors if necessary. + * @param subType the sub-type to unify. + * @param superType the super-type to unify. + */ + void unify(TypeId subType, TypeId superType); + + /** + * Creates a new Unifier and performs a single unification operation. Commits + * the result and reports errors if necessary. + * @param subPack the sub-type pack to unify. + * @param superPack the super-type pack to unify. + */ + void unify(TypePackId subPack, TypePackId superPack); +}; + +void dump(Scope2* rootScope, struct ToStringOptions& opts); + +} // namespace Luau diff --git a/Analysis/include/Luau/Frontend.h b/Analysis/include/Luau/Frontend.h index d7c9ca40..58be0ffe 100644 --- a/Analysis/include/Luau/Frontend.h +++ b/Analysis/include/Luau/Frontend.h @@ -159,6 +159,8 @@ struct Frontend void applyBuiltinDefinitionToEnvironment(const std::string& environmentName, const std::string& definitionName); private: + ModulePtr check(const SourceModule& sourceModule, Mode mode, const ScopePtr& environmentScope); + std::pair getSourceNode(CheckResult& checkResult, const ModuleName& name); SourceModule parse(const ModuleName& name, std::string_view src, const ParseOptions& parseOptions); diff --git a/Analysis/include/Luau/Module.h b/Analysis/include/Luau/Module.h index 00e1e635..f6e077dc 100644 --- a/Analysis/include/Luau/Module.h +++ b/Analysis/include/Luau/Module.h @@ -19,6 +19,7 @@ struct Module; using ScopePtr = std::shared_ptr; using ModulePtr = std::shared_ptr; +struct Scope2; /// Root of the AST of a parsed source file struct SourceModule @@ -65,6 +66,7 @@ struct Module std::shared_ptr names; std::vector> scopes; // never empty + std::vector>> scope2s; // never empty DenseHashMap astTypes{nullptr}; DenseHashMap astExpectedTypes{nullptr}; @@ -78,6 +80,7 @@ struct Module bool timeout = false; ScopePtr getModuleScope() const; + Scope2* getModuleScope2() const; // Once a module has been typechecked, we clone its public interface into a separate arena. // This helps us to force TypeVar ownership into a DAG rather than a DCG. diff --git a/Analysis/include/Luau/NotNull.h b/Analysis/include/Luau/NotNull.h new file mode 100644 index 00000000..3d05fdea --- /dev/null +++ b/Analysis/include/Luau/NotNull.h @@ -0,0 +1,75 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +#pragma once + +#include "Luau/Common.h" + +#include + +namespace Luau +{ + +/** A non-owning, non-null pointer to a T. + * + * A NotNull is notionally identical to a T* with the added restriction that it + * can never store nullptr. + * + * The sole conversion rule from T* to NotNull is the single-argument constructor, which + * is intentionally marked explicit. This constructor performs a runtime test to verify + * that the passed pointer is never nullptr. + * + * Pointer arithmetic, increment, decrement, and array indexing are all forbidden. + * + * An implicit coersion from NotNull to T* is afforded, as are the pointer indirection and member + * access operators. (*p and p->prop) + * + * The explicit delete statement is permitted on a NotNull through this implicit conversion. + */ +template +struct NotNull +{ + explicit NotNull(T* t) + : ptr(t) + { + LUAU_ASSERT(t); + } + + explicit NotNull(std::nullptr_t) = delete; + void operator=(std::nullptr_t) = delete; + + operator T*() const noexcept + { + return ptr; + } + + T& operator*() const noexcept + { + return *ptr; + } + + T* operator->() const noexcept + { + return ptr; + } + + T& operator[](int) = delete; + + T& operator+(int) = delete; + T& operator-(int) = delete; + + T* ptr; +}; + +} + +namespace std +{ + +template struct hash> +{ + size_t operator()(const Luau::NotNull& p) const + { + return std::hash()(p.ptr); + } +}; + +} diff --git a/Analysis/include/Luau/Quantify.h b/Analysis/include/Luau/Quantify.h index e48cad40..b32d684e 100644 --- a/Analysis/include/Luau/Quantify.h +++ b/Analysis/include/Luau/Quantify.h @@ -6,6 +6,9 @@ namespace Luau { +struct Scope2; + void quantify(TypeId ty, TypeLevel level); +void quantify(TypeId ty, Scope2* scope); } // namespace Luau diff --git a/Analysis/include/Luau/Symbol.h b/Analysis/include/Luau/Symbol.h index b5dd9c89..1fe037e5 100644 --- a/Analysis/include/Luau/Symbol.h +++ b/Analysis/include/Luau/Symbol.h @@ -30,6 +30,9 @@ struct Symbol { } + template + Symbol(const T&) = delete; + AstLocal* local; AstName global; diff --git a/Analysis/include/Luau/ToString.h b/Analysis/include/Luau/ToString.h index 3b380a60..a50fef78 100644 --- a/Analysis/include/Luau/ToString.h +++ b/Analysis/include/Luau/ToString.h @@ -3,6 +3,7 @@ #include "Luau/Common.h" #include "Luau/TypeVar.h" +#include "Luau/ConstraintGraphBuilder.h" #include #include @@ -53,6 +54,7 @@ ToStringResult toStringDetailed(TypePackId ty, const ToStringOptions& opts = {}) std::string toString(TypeId ty, const ToStringOptions& opts); std::string toString(TypePackId ty, const ToStringOptions& opts); +std::string toString(const Constraint& c, ToStringOptions& opts); // These are offered as overloads rather than a default parameter so that they can be easily invoked from within the MSVC debugger. // You can use them in watch expressions! @@ -64,6 +66,11 @@ inline std::string toString(TypePackId ty) { return toString(ty, ToStringOptions{}); } +inline std::string toString(const Constraint& c) +{ + ToStringOptions opts; + return toString(c, opts); +} std::string toString(const TypeVar& tv, const ToStringOptions& opts = {}); std::string toString(const TypePackVar& tp, const ToStringOptions& opts = {}); @@ -74,6 +81,7 @@ std::string toStringNamedFunction(const std::string& funcName, const FunctionTyp // These functions will dump the type to stdout and can be evaluated in Watch/Immediate windows or as gdb/lldb expression std::string dump(TypeId ty); std::string dump(TypePackId ty); +std::string dump(const Constraint& c); std::string dump(const std::shared_ptr& scope, const char* name); diff --git a/Analysis/include/Luau/TypeVar.h b/Analysis/include/Luau/TypeVar.h index 9cacbc6d..b3c455cf 100644 --- a/Analysis/include/Luau/TypeVar.h +++ b/Analysis/include/Luau/TypeVar.h @@ -24,6 +24,7 @@ namespace Luau { struct TypeArena; +struct Scope2; /** * There are three kinds of type variables: @@ -124,6 +125,7 @@ struct ConstrainedTypeVar std::vector parts; TypeLevel level; + Scope2* scope = nullptr; }; // Singleton types https://github.com/Roblox/luau/blob/master/rfcs/syntax-singleton-types.md @@ -255,6 +257,7 @@ struct FunctionTypeVar std::optional defn = {}, bool hasSelf = false); TypeLevel level; + Scope2* scope = nullptr; /// These should all be generic std::vector generics; std::vector genericPacks; @@ -266,6 +269,7 @@ struct FunctionTypeVar bool hasSelf; Tags tags; bool hasNoGenerics = false; + bool generalized = false; }; enum class TableState @@ -323,6 +327,7 @@ struct TableTypeVar TableState state = TableState::Unsealed; TypeLevel level; + Scope2* scope = nullptr; std::optional name; // Sometimes we throw a type on a name to make for nicer error messages, but without creating any entry in the type namespace diff --git a/Analysis/include/Luau/TypedAllocator.h b/Analysis/include/Luau/TypedAllocator.h index c1c04d10..f67e3d8e 100644 --- a/Analysis/include/Luau/TypedAllocator.h +++ b/Analysis/include/Luau/TypedAllocator.h @@ -23,6 +23,12 @@ public: currentBlockSize = kBlockSize; } + TypedAllocator(const TypedAllocator&) = delete; + TypedAllocator& operator=(const TypedAllocator&) = delete; + + TypedAllocator(TypedAllocator&&) = default; + TypedAllocator& operator=(TypedAllocator&&) = default; + ~TypedAllocator() { if (frozen) diff --git a/Analysis/include/Luau/Unifiable.h b/Analysis/include/Luau/Unifiable.h index 64fa131d..fdc39481 100644 --- a/Analysis/include/Luau/Unifiable.h +++ b/Analysis/include/Luau/Unifiable.h @@ -8,6 +8,8 @@ namespace Luau { +struct Scope2; + /** * The 'level' of a TypeVar is an indirect way to talk about the scope that it 'belongs' too. * To start, read http://okmij.org/ftp/ML/generalization.html @@ -82,9 +84,11 @@ using Name = std::string; struct Free { explicit Free(TypeLevel level); + explicit Free(Scope2* scope); int index; TypeLevel level; + Scope2* scope = nullptr; // True if this free type variable is part of a mutually // recursive type alias whose definitions haven't been // resolved yet. @@ -111,12 +115,14 @@ struct Generic Generic(); explicit Generic(TypeLevel level); explicit Generic(const Name& name); + explicit Generic(Scope2* scope); Generic(TypeLevel level, const Name& name); int index; TypeLevel level; + Scope2* scope = nullptr; Name name; - bool explicitName; + bool explicitName = false; private: static int nextIndex; diff --git a/Analysis/include/Luau/Variant.h b/Analysis/include/Luau/Variant.h index 5efe89ed..c9c97c92 100644 --- a/Analysis/include/Luau/Variant.h +++ b/Analysis/include/Luau/Variant.h @@ -95,6 +95,20 @@ public: return *this; } + template + T& emplace(Args&&... args) + { + using TT = std::decay_t; + constexpr int tid = getTypeId(); + static_assert(tid >= 0, "unsupported T"); + + tableDtor[typeId](&storage); + typeId = tid; + new (&storage) TT(std::forward(args)...); + + return *reinterpret_cast(&storage); + } + template const T* get_if() const { diff --git a/Analysis/src/Clone.cpp b/Analysis/src/Clone.cpp index a3611f53..19e3383e 100644 --- a/Analysis/src/Clone.cpp +++ b/Analysis/src/Clone.cpp @@ -2,6 +2,7 @@ #include "Luau/Clone.h" #include "Luau/RecursionCounter.h" +#include "Luau/TxnLog.h" #include "Luau/TypePack.h" #include "Luau/Unifiable.h" @@ -382,4 +383,67 @@ TypeFun clone(const TypeFun& typeFun, TypeArena& dest, CloneState& cloneState) return result; } +TypeId shallowClone(TypeId ty, TypeArena& dest, const TxnLog* log) +{ + ty = log->follow(ty); + + TypeId result = ty; + + if (auto pty = log->pending(ty)) + ty = &pty->pending; + + if (const FunctionTypeVar* ftv = get(ty)) + { + FunctionTypeVar clone = FunctionTypeVar{ftv->level, ftv->argTypes, ftv->retType, ftv->definition, ftv->hasSelf}; + clone.generics = ftv->generics; + clone.genericPacks = ftv->genericPacks; + clone.magicFunction = ftv->magicFunction; + clone.tags = ftv->tags; + clone.argNames = ftv->argNames; + result = dest.addType(std::move(clone)); + } + else if (const TableTypeVar* ttv = get(ty)) + { + LUAU_ASSERT(!ttv->boundTo); + TableTypeVar clone = TableTypeVar{ttv->props, ttv->indexer, ttv->level, ttv->state}; + if (!FFlag::LuauNoMethodLocations) + clone.methodDefinitionLocations = ttv->methodDefinitionLocations; + clone.definitionModuleName = ttv->definitionModuleName; + clone.name = ttv->name; + clone.syntheticName = ttv->syntheticName; + clone.instantiatedTypeParams = ttv->instantiatedTypeParams; + clone.instantiatedTypePackParams = ttv->instantiatedTypePackParams; + clone.tags = ttv->tags; + result = dest.addType(std::move(clone)); + } + else if (const MetatableTypeVar* mtv = get(ty)) + { + MetatableTypeVar clone = MetatableTypeVar{mtv->table, mtv->metatable}; + clone.syntheticName = mtv->syntheticName; + result = dest.addType(std::move(clone)); + } + else if (const UnionTypeVar* utv = get(ty)) + { + UnionTypeVar clone; + clone.options = utv->options; + result = dest.addType(std::move(clone)); + } + else if (const IntersectionTypeVar* itv = get(ty)) + { + IntersectionTypeVar clone; + clone.parts = itv->parts; + result = dest.addType(std::move(clone)); + } + else if (const ConstrainedTypeVar* ctv = get(ty)) + { + ConstrainedTypeVar clone{ctv->level, ctv->parts}; + result = dest.addType(std::move(clone)); + } + else + return result; + + asMutable(result)->documentationSymbol = ty->documentationSymbol; + return result; +} + } // namespace Luau diff --git a/Analysis/src/ConstraintGraphBuilder.cpp b/Analysis/src/ConstraintGraphBuilder.cpp new file mode 100644 index 00000000..c8f77ddf --- /dev/null +++ b/Analysis/src/ConstraintGraphBuilder.cpp @@ -0,0 +1,300 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details + +#include "Luau/ConstraintGraphBuilder.h" + +namespace Luau +{ + +Constraint::Constraint(ConstraintV&& c) + : c(std::move(c)) +{ +} + +Constraint::Constraint(ConstraintV&& c, std::vector dependencies) + : c(std::move(c)) + , dependencies(dependencies) +{ +} + +std::optional Scope2::lookup(Symbol sym) +{ + Scope2* s = this; + + while (true) + { + auto it = s->bindings.find(sym); + if (it != s->bindings.end()) + return it->second; + + if (s->parent) + s = s->parent; + else + return std::nullopt; + } +} + +ConstraintGraphBuilder::ConstraintGraphBuilder(TypeArena* arena) + : singletonTypes(getSingletonTypes()) + , arena(arena) + , rootScope(nullptr) +{ + LUAU_ASSERT(arena); +} + +TypeId ConstraintGraphBuilder::freshType(Scope2* scope) +{ + LUAU_ASSERT(scope); + return arena->addType(FreeTypeVar{scope}); +} + +TypePackId ConstraintGraphBuilder::freshTypePack(Scope2* scope) +{ + LUAU_ASSERT(scope); + FreeTypePack f{scope}; + return arena->addTypePack(TypePackVar{std::move(f)}); +} + +Scope2* ConstraintGraphBuilder::childScope(Location location, Scope2* parent) +{ + LUAU_ASSERT(parent); + auto scope = std::make_unique(); + Scope2* borrow = scope.get(); + scopes.emplace_back(location, std::move(scope)); + + borrow->parent = parent; + borrow->returnType = parent->returnType; + parent->children.push_back(borrow); + + return borrow; +} + +void ConstraintGraphBuilder::addConstraint(Scope2* scope, ConstraintV cv) +{ + LUAU_ASSERT(scope); + scope->constraints.emplace_back(new Constraint{std::move(cv)}); +} + +void ConstraintGraphBuilder::addConstraint(Scope2* scope, std::unique_ptr c) +{ + LUAU_ASSERT(scope); + scope->constraints.emplace_back(std::move(c)); +} + +void ConstraintGraphBuilder::visit(AstStatBlock* block) +{ + LUAU_ASSERT(scopes.empty()); + LUAU_ASSERT(rootScope == nullptr); + scopes.emplace_back(block->location, std::make_unique()); + rootScope = scopes.back().second.get(); + rootScope->returnType = freshTypePack(rootScope); + + visit(rootScope, block); +} + +void ConstraintGraphBuilder::visit(Scope2* scope, AstStat* stat) +{ + LUAU_ASSERT(scope); + + if (auto s = stat->as()) + visit(scope, s); + else if (auto s = stat->as()) + visit(scope, s); + else if (auto f = stat->as()) + visit(scope, f); + else if (auto r = stat->as()) + visit(scope, r); + else + LUAU_ASSERT(0); +} + +void ConstraintGraphBuilder::visit(Scope2* scope, AstStatLocal* local) +{ + LUAU_ASSERT(scope); + + std::vector varTypes; + + for (AstLocal* local : local->vars) + { + // TODO annotations + TypeId ty = freshType(scope); + varTypes.push_back(ty); + scope->bindings[local] = ty; + } + + for (size_t i = 0; i < local->vars.size; ++i) + { + if (i < local->values.size) + { + TypeId exprType = check(scope, local->values.data[i]); + addConstraint(scope, SubtypeConstraint{varTypes[i], exprType}); + } + } +} + +void addConstraints(Constraint* constraint, Scope2* scope) +{ + LUAU_ASSERT(scope); + + scope->constraints.reserve(scope->constraints.size() + scope->constraints.size()); + + for (const auto& c : scope->constraints) + constraint->dependencies.push_back(c.get()); + + for (Scope2* childScope : scope->children) + addConstraints(constraint, childScope); +} + +void ConstraintGraphBuilder::visit(Scope2* scope, AstStatLocalFunction* function) +{ + LUAU_ASSERT(scope); + + // Local + // Global + // Dotted path + // Self? + + TypeId functionType = nullptr; + auto ty = scope->lookup(function->name); + LUAU_ASSERT(!ty.has_value()); // The parser ensures that every local function has a distinct Symbol for its name. + + functionType = freshType(scope); + scope->bindings[function->name] = functionType; + + Scope2* innerScope = childScope(function->func->body->location, scope); + TypePackId returnType = freshTypePack(scope); + innerScope->returnType = returnType; + + std::vector argTypes; + + for (AstLocal* local : function->func->args) + { + TypeId t = freshType(innerScope); + argTypes.push_back(t); + innerScope->bindings[local] = t; // TODO annotations + } + + for (AstStat* stat : function->func->body->body) + visit(innerScope, stat); + + FunctionTypeVar actualFunction{arena->addTypePack(argTypes), returnType}; + TypeId actualFunctionType = arena->addType(std::move(actualFunction)); + + std::unique_ptr c{new Constraint{GeneralizationConstraint{functionType, actualFunctionType, innerScope}}}; + addConstraints(c.get(), innerScope); + + addConstraint(scope, std::move(c)); +} + +void ConstraintGraphBuilder::visit(Scope2* scope, AstStatReturn* ret) +{ + LUAU_ASSERT(scope); + + TypePackId exprTypes = checkPack(scope, ret->list); + addConstraint(scope, PackSubtypeConstraint{exprTypes, scope->returnType}); +} + +void ConstraintGraphBuilder::visit(Scope2* scope, AstStatBlock* block) +{ + LUAU_ASSERT(scope); + + for (AstStat* stat : block->body) + visit(scope, stat); +} + +TypePackId ConstraintGraphBuilder::checkPack(Scope2* scope, AstArray exprs) +{ + LUAU_ASSERT(scope); + + if (exprs.size == 0) + return arena->addTypePack({}); + + std::vector types; + TypePackId last = nullptr; + + for (size_t i = 0; i < exprs.size; ++i) + { + if (i < exprs.size - 1) + types.push_back(check(scope, exprs.data[i])); + else + last = checkPack(scope, exprs.data[i]); + } + + LUAU_ASSERT(last != nullptr); + + return arena->addTypePack(TypePack{std::move(types), last}); +} + +TypePackId ConstraintGraphBuilder::checkPack(Scope2* scope, AstExpr* expr) +{ + LUAU_ASSERT(scope); + + // TEMP TEMP TEMP HACK HACK HACK FIXME FIXME + TypeId t = check(scope, expr); + return arena->addTypePack({t}); +} + +TypeId ConstraintGraphBuilder::check(Scope2* scope, AstExpr* expr) +{ + LUAU_ASSERT(scope); + + if (auto a = expr->as()) + return singletonTypes.stringType; + else if (auto a = expr->as()) + return singletonTypes.numberType; + else if (auto a = expr->as()) + return singletonTypes.booleanType; + else if (auto a = expr->as()) + return singletonTypes.nilType; + else if (auto a = expr->as()) + { + std::optional ty = scope->lookup(a->local); + if (ty) + return *ty; + else + return singletonTypes.errorRecoveryType(singletonTypes.anyType); // FIXME? Record an error at this point? + } + else if (auto a = expr->as()) + { + std::vector args; + + for (AstExpr* arg : a->args) + { + args.push_back(check(scope, arg)); + } + + TypeId fnType = check(scope, a->func); + TypeId instantiatedType = freshType(scope); + addConstraint(scope, InstantiationConstraint{instantiatedType, fnType}); + + TypeId firstRet = freshType(scope); + TypePackId rets = arena->addTypePack(TypePack{{firstRet}, arena->addTypePack(TypePackVar{FreeTypePack{TypeLevel{}}})}); + FunctionTypeVar ftv(arena->addTypePack(TypePack{args, {}}), rets); + TypeId inferredFnType = arena->addType(ftv); + + addConstraint(scope, SubtypeConstraint{inferredFnType, instantiatedType}); + return firstRet; + } + else + { + LUAU_ASSERT(0); + return freshType(scope); + } +} + +static void collectConstraints(std::vector& result, Scope2* scope) +{ + for (const auto& c : scope->constraints) + result.push_back(c.get()); + + for (Scope2* child : scope->children) + collectConstraints(result, child); +} + +std::vector collectConstraints(Scope2* rootScope) +{ + std::vector result; + collectConstraints(result, rootScope); + return result; +} + +} // namespace Luau diff --git a/Analysis/src/ConstraintSolver.cpp b/Analysis/src/ConstraintSolver.cpp new file mode 100644 index 00000000..f40cd4b3 --- /dev/null +++ b/Analysis/src/ConstraintSolver.cpp @@ -0,0 +1,306 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details + +#include "Luau/ConstraintSolver.h" +#include "Luau/Instantiation.h" +#include "Luau/Quantify.h" +#include "Luau/ToString.h" +#include "Luau/Unifier.h" + +LUAU_FASTFLAGVARIABLE(DebugLuauLogSolver, false); + +namespace Luau +{ + +[[maybe_unused]] static void dumpBindings(Scope2* scope, ToStringOptions& opts) +{ + for (const auto& [k, v] : scope->bindings) + { + auto d = toStringDetailed(v, opts); + opts.nameMap = d.nameMap; + printf("\t%s : %s\n", k.c_str(), d.name.c_str()); + } + + for (Scope2* child : scope->children) + dumpBindings(child, opts); +} + +static void dumpConstraints(Scope2* scope, ToStringOptions& opts) +{ + for (const ConstraintPtr& c : scope->constraints) + { + printf("\t%s\n", toString(*c, opts).c_str()); + } + + for (Scope2* child : scope->children) + dumpConstraints(child, opts); +} + +void dump(Scope2* rootScope, ToStringOptions& opts) +{ + printf("constraints:\n"); + dumpConstraints(rootScope, opts); +} + +void dump(ConstraintSolver* cs, ToStringOptions& opts) +{ + printf("constraints:\n"); + for (const Constraint* c : cs->unsolvedConstraints) + { + printf("\t%s\n", toString(*c, opts).c_str()); + + for (const Constraint* dep : c->dependencies) + printf("\t\t%s\n", toString(*dep, opts).c_str()); + } +} + +ConstraintSolver::ConstraintSolver(TypeArena* arena, Scope2* rootScope) + : arena(arena) + , constraints(collectConstraints(rootScope)) + , rootScope(rootScope) +{ + for (const Constraint* c : constraints) + { + unsolvedConstraints.insert(c); + + for (const Constraint* dep : c->dependencies) + { + block(dep, c); + } + } +} + +void ConstraintSolver::run() +{ + if (done()) + return; + + bool progress = false; + + ToStringOptions opts; + + if (FFlag::DebugLuauLogSolver) + { + printf("Starting solver\n"); + dump(this, opts); + } + + do + { + progress = false; + + auto it = begin(unsolvedConstraints); + auto endIt = end(unsolvedConstraints); + + while (it != endIt) + { + if (isBlocked(*it)) + { + ++it; + continue; + } + + std::string saveMe = FFlag::DebugLuauLogSolver ? toString(**it, opts) : std::string{}; + + bool success = tryDispatch(*it); + progress = progress || success; + + auto saveIt = it; + ++it; + if (success) + { + unsolvedConstraints.erase(saveIt); + if (FFlag::DebugLuauLogSolver) + { + printf("Dispatched\n\t%s\n", saveMe.c_str()); + dump(this, opts); + } + } + } + } while (progress); + + if (FFlag::DebugLuauLogSolver) + dumpBindings(rootScope, opts); + + LUAU_ASSERT(done()); +} + +bool ConstraintSolver::done() +{ + return unsolvedConstraints.empty(); +} + +bool ConstraintSolver::tryDispatch(const Constraint* constraint) +{ + if (isBlocked(constraint)) + return false; + + bool success = false; + + if (auto sc = get(*constraint)) + success = tryDispatch(*sc); + else if (auto psc = get(*constraint)) + success = tryDispatch(*psc); + else if (auto gc = get(*constraint)) + success = tryDispatch(*gc); + else if (auto ic = get(*constraint)) + success = tryDispatch(*ic, constraint); + else + LUAU_ASSERT(0); + + if (success) + { + unblock(constraint); + } + + return success; +} + +bool ConstraintSolver::tryDispatch(const SubtypeConstraint& c) +{ + unify(c.subType, c.superType); + unblock(c.subType); + unblock(c.superType); + + return true; +} + +bool ConstraintSolver::tryDispatch(const PackSubtypeConstraint& c) +{ + unify(c.subPack, c.superPack); + unblock(c.subPack); + unblock(c.superPack); + + return true; +} + +bool ConstraintSolver::tryDispatch(const GeneralizationConstraint& constraint) +{ + unify(constraint.subType, constraint.superType); + + quantify(constraint.superType, constraint.scope); + unblock(constraint.subType); + unblock(constraint.superType); + + return true; +} + +bool ConstraintSolver::tryDispatch(const InstantiationConstraint& c, const Constraint* constraint) +{ + TypeId superType = follow(c.superType); + if (const FunctionTypeVar* ftv = get(superType)) + { + if (!ftv->generalized) + { + block(superType, constraint); + return false; + } + } + else if (get(superType)) + { + block(superType, constraint); + return false; + } + // TODO: Error if it's a primitive or something + + Instantiation inst(TxnLog::empty(), arena, TypeLevel{}); + + std::optional instantiated = inst.substitute(c.superType); + LUAU_ASSERT(instantiated); // TODO FIXME HANDLE THIS + + unify(c.subType, *instantiated); + unblock(c.subType); + + return true; +} + +void ConstraintSolver::block_(BlockedConstraintId target, const Constraint* constraint) +{ + blocked[target].push_back(constraint); + + auto& count = blockedConstraints[constraint]; + count += 1; +} + +void ConstraintSolver::block(const Constraint* target, const Constraint* constraint) +{ + block_(target, constraint); +} + +void ConstraintSolver::block(TypeId target, const Constraint* constraint) +{ + block_(target, constraint); +} + +void ConstraintSolver::block(TypePackId target, const Constraint* constraint) +{ + block_(target, constraint); +} + +void ConstraintSolver::unblock_(BlockedConstraintId progressed) +{ + auto it = blocked.find(progressed); + if (it == blocked.end()) + return; + + // unblocked should contain a value always, because of the above check + for (const Constraint* unblockedConstraint : it->second) + { + auto& count = blockedConstraints[unblockedConstraint]; + // This assertion being hit indicates that `blocked` and + // `blockedConstraints` desynchronized at some point. This is problematic + // because we rely on this count being correct to skip over blocked + // constraints. + LUAU_ASSERT(count > 0); + count -= 1; + } + + blocked.erase(it); +} + +void ConstraintSolver::unblock(const Constraint* progressed) +{ + return unblock_(progressed); +} + +void ConstraintSolver::unblock(TypeId progressed) +{ + return unblock_(progressed); +} + +void ConstraintSolver::unblock(TypePackId progressed) +{ + return unblock_(progressed); +} + +bool ConstraintSolver::isBlocked(const Constraint* constraint) +{ + auto blockedIt = blockedConstraints.find(constraint); + return blockedIt != blockedConstraints.end() && blockedIt->second > 0; +} + +void ConstraintSolver::reportErrors(const std::vector& errors) +{ + this->errors.insert(end(this->errors), begin(errors), end(errors)); +} + +void ConstraintSolver::unify(TypeId subType, TypeId superType) +{ + UnifierSharedState sharedState{&iceReporter}; + Unifier u{arena, Mode::Strict, Location{}, Covariant, sharedState}; + + u.tryUnify(subType, superType); + u.log.commit(); + reportErrors(u.errors); +} + +void ConstraintSolver::unify(TypePackId subPack, TypePackId superPack) +{ + UnifierSharedState sharedState{&iceReporter}; + Unifier u{arena, Mode::Strict, Location{}, Covariant, sharedState}; + + u.tryUnify(subPack, superPack); + u.log.commit(); + reportErrors(u.errors); +} + +} // namespace Luau diff --git a/Analysis/src/EmbeddedBuiltinDefinitions.cpp b/Analysis/src/EmbeddedBuiltinDefinitions.cpp index 9a2259f1..f184b74e 100644 --- a/Analysis/src/EmbeddedBuiltinDefinitions.cpp +++ b/Analysis/src/EmbeddedBuiltinDefinitions.cpp @@ -179,7 +179,7 @@ declare debug: { } declare utf8: { - char: (number, ...number) -> string, + char: (...number) -> string, charpattern: string, codes: (string) -> ((string, number) -> (number, number), string, number), -- FIXME diff --git a/Analysis/src/Frontend.cpp b/Analysis/src/Frontend.cpp index 1d33f131..741a35cf 100644 --- a/Analysis/src/Frontend.cpp +++ b/Analysis/src/Frontend.cpp @@ -5,6 +5,8 @@ #include "Luau/Clone.h" #include "Luau/Config.h" #include "Luau/FileResolver.h" +#include "Luau/ConstraintGraphBuilder.h" +#include "Luau/ConstraintSolver.h" #include "Luau/Parser.h" #include "Luau/Scope.h" #include "Luau/StringUtils.h" @@ -22,6 +24,7 @@ LUAU_FASTFLAG(LuauInferInNoCheckMode) LUAU_FASTFLAGVARIABLE(LuauKnowsTheDataModel3, false) LUAU_FASTFLAGVARIABLE(LuauAutocompleteDynamicLimits, false) LUAU_FASTINTVARIABLE(LuauAutocompleteCheckTimeoutMs, 100) +LUAU_FASTFLAGVARIABLE(DebugLuauDeferredConstraintResolution, false) namespace Luau { @@ -470,7 +473,8 @@ CheckResult Frontend::check(const ModuleName& name, std::optional(this)->getSourceModule(moduleName); } +ModulePtr Frontend::check(const SourceModule& sourceModule, Mode mode, const ScopePtr& environmentScope) +{ + ModulePtr result = std::make_shared(); + + ConstraintGraphBuilder cgb{&result->internalTypes}; + cgb.visit(sourceModule.root); + + ConstraintSolver cs{&result->internalTypes, cgb.rootScope}; + cs.run(); + + result->scope2s = std::move(cgb.scopes); + + result->clonePublicInterface(iceHandler); + + return result; +} + // Read AST into sourceModules if necessary. Trace require()s. Report parse errors. std::pair Frontend::getSourceNode(CheckResult& checkResult, const ModuleName& name) { diff --git a/Analysis/src/Module.cpp b/Analysis/src/Module.cpp index 6591d60a..4d157e6f 100644 --- a/Analysis/src/Module.cpp +++ b/Analysis/src/Module.cpp @@ -3,6 +3,7 @@ #include "Luau/Clone.h" #include "Luau/Common.h" +#include "Luau/ConstraintGraphBuilder.h" #include "Luau/Normalize.h" #include "Luau/RecursionCounter.h" #include "Luau/Scope.h" @@ -10,11 +11,13 @@ #include "Luau/TypePack.h" #include "Luau/TypeVar.h" #include "Luau/VisitTypeVar.h" +#include "Luau/ConstraintGraphBuilder.h" // FIXME: For Scope2 TODO pull out into its own header #include LUAU_FASTFLAG(LuauLowerBoundsCalculation); LUAU_FASTFLAG(LuauNormalizeFlagIsConservative); +LUAU_FASTFLAG(DebugLuauDeferredConstraintResolution); namespace Luau { @@ -97,38 +100,60 @@ void Module::clonePublicInterface(InternalErrorReporter& ice) CloneState cloneState; - ScopePtr moduleScope = getModuleScope(); + ScopePtr moduleScope = FFlag::DebugLuauDeferredConstraintResolution ? nullptr : getModuleScope(); + Scope2* moduleScope2 = FFlag::DebugLuauDeferredConstraintResolution ? getModuleScope2() : nullptr; - moduleScope->returnType = clone(moduleScope->returnType, interfaceTypes, cloneState); - if (moduleScope->varargPack) - moduleScope->varargPack = clone(*moduleScope->varargPack, interfaceTypes, cloneState); + TypePackId returnType = FFlag::DebugLuauDeferredConstraintResolution ? moduleScope2->returnType : moduleScope->returnType; + std::optional varargPack = FFlag::DebugLuauDeferredConstraintResolution ? std::nullopt : moduleScope->varargPack; + std::unordered_map* exportedTypeBindings = + FFlag::DebugLuauDeferredConstraintResolution ? nullptr : &moduleScope->exportedTypeBindings; + + returnType = clone(returnType, interfaceTypes, cloneState); + + if (moduleScope) + { + moduleScope->returnType = returnType; + if (varargPack) + { + varargPack = clone(*varargPack, interfaceTypes, cloneState); + moduleScope->varargPack = varargPack; + } + } + else + { + LUAU_ASSERT(moduleScope2); + moduleScope2->returnType = returnType; // TODO varargPack + } if (FFlag::LuauLowerBoundsCalculation) { - normalize(moduleScope->returnType, interfaceTypes, ice); - if (moduleScope->varargPack) - normalize(*moduleScope->varargPack, interfaceTypes, ice); + normalize(returnType, interfaceTypes, ice); + if (varargPack) + normalize(*varargPack, interfaceTypes, ice); } ForceNormal forceNormal{&interfaceTypes}; - for (auto& [name, tf] : moduleScope->exportedTypeBindings) + if (exportedTypeBindings) { - tf = clone(tf, interfaceTypes, cloneState); - if (FFlag::LuauLowerBoundsCalculation) + for (auto& [name, tf] : *exportedTypeBindings) { - normalize(tf.type, interfaceTypes, ice); - - if (FFlag::LuauNormalizeFlagIsConservative) + tf = clone(tf, interfaceTypes, cloneState); + if (FFlag::LuauLowerBoundsCalculation) { - // We're about to freeze the memory. We know that the flag is conservative by design. Cyclic tables - // won't be marked normal. If the types aren't normal by now, they never will be. - forceNormal.traverse(tf.type); + normalize(tf.type, interfaceTypes, ice); + + if (FFlag::LuauNormalizeFlagIsConservative) + { + // We're about to freeze the memory. We know that the flag is conservative by design. Cyclic tables + // won't be marked normal. If the types aren't normal by now, they never will be. + forceNormal.traverse(tf.type); + } } } } - for (TypeId ty : moduleScope->returnType) + for (TypeId ty : returnType) { if (get(follow(ty))) { @@ -155,4 +180,10 @@ ScopePtr Module::getModuleScope() const return scopes.front().second; } +Scope2* Module::getModuleScope2() const +{ + LUAU_ASSERT(!scope2s.empty()); + return scope2s.front().second.get(); +} + } // namespace Luau diff --git a/Analysis/src/Normalize.cpp b/Analysis/src/Normalize.cpp index fb31df1e..11403be5 100644 --- a/Analysis/src/Normalize.cpp +++ b/Analysis/src/Normalize.cpp @@ -16,6 +16,7 @@ LUAU_FASTINTVARIABLE(LuauNormalizeIterationLimit, 1200); LUAU_FASTFLAGVARIABLE(LuauNormalizeCombineTableFix, false); LUAU_FASTFLAGVARIABLE(LuauNormalizeFlagIsConservative, false); LUAU_FASTFLAGVARIABLE(LuauNormalizeCombineEqFix, false); +LUAU_FASTFLAGVARIABLE(LuauReplaceReplacer, false); namespace Luau { @@ -231,11 +232,30 @@ struct Replacer : Substitution TypeId smartClone(TypeId t) { - std::optional res = replace(t); - LUAU_ASSERT(res.has_value()); // TODO think about this - if (*res == t) - return clone(t); - return *res; + if (FFlag::LuauReplaceReplacer) + { + // The new smartClone is just a memoized clone() + // TODO: Remove the Substitution base class and all other methods from this struct. + // Add DenseHashMap newTypes; + t = log->follow(t); + TypeId* res = newTypes.find(t); + if (res) + return *res; + + TypeId result = shallowClone(t, *arena, TxnLog::empty()); + newTypes[t] = result; + newTypes[result] = result; + + return result; + } + else + { + std::optional res = replace(t); + LUAU_ASSERT(res.has_value()); // TODO think about this + if (*res == t) + return clone(t); + return *res; + } } }; diff --git a/Analysis/src/Quantify.cpp b/Analysis/src/Quantify.cpp index c0f677d7..8f2cc8e3 100644 --- a/Analysis/src/Quantify.cpp +++ b/Analysis/src/Quantify.cpp @@ -3,8 +3,10 @@ #include "Luau/Quantify.h" #include "Luau/VisitTypeVar.h" +#include "Luau/ConstraintGraphBuilder.h" // TODO for Scope2; move to separate header -LUAU_FASTFLAG(LuauAlwaysQuantify) +LUAU_FASTFLAG(LuauAlwaysQuantify); +LUAU_FASTFLAG(DebugLuauDeferredConstraintResolution); namespace Luau { @@ -14,12 +16,20 @@ struct Quantifier final : TypeVarOnceVisitor TypeLevel level; std::vector generics; std::vector genericPacks; + Scope2* scope = nullptr; bool seenGenericType = false; bool seenMutableType = false; explicit Quantifier(TypeLevel level) : level(level) { + LUAU_ASSERT(!FFlag::DebugLuauDeferredConstraintResolution); + } + + explicit Quantifier(Scope2* scope) + : scope(scope) + { + LUAU_ASSERT(FFlag::DebugLuauDeferredConstraintResolution); } void cycle(TypeId) override {} @@ -57,14 +67,31 @@ struct Quantifier final : TypeVarOnceVisitor return visit(tp, ftp); } + /// @return true if outer encloses inner + bool subsumes(Scope2* outer, Scope2* inner) + { + while (inner) + { + if (inner == outer) + return true; + inner = inner->parent; + } + + return false; + } + bool visit(TypeId ty, const FreeTypeVar& ftv) override { seenMutableType = true; - if (!level.subsumes(ftv.level)) + if (FFlag::DebugLuauDeferredConstraintResolution ? !subsumes(scope, ftv.scope) : !level.subsumes(ftv.level)) return false; - *asMutable(ty) = GenericTypeVar{level}; + if (FFlag::DebugLuauDeferredConstraintResolution) + *asMutable(ty) = GenericTypeVar{scope}; + else + *asMutable(ty) = GenericTypeVar{level}; + generics.push_back(ty); return false; @@ -83,7 +110,7 @@ struct Quantifier final : TypeVarOnceVisitor if (ttv.state == TableState::Sealed || ttv.state == TableState::Generic) return false; - if (!level.subsumes(ttv.level)) + if (FFlag::DebugLuauDeferredConstraintResolution ? !subsumes(scope, ttv.scope) : !level.subsumes(ttv.level)) { if (ttv.state == TableState::Unsealed) seenMutableType = true; @@ -107,7 +134,7 @@ struct Quantifier final : TypeVarOnceVisitor { seenMutableType = true; - if (!level.subsumes(ftp.level)) + if (FFlag::DebugLuauDeferredConstraintResolution ? !subsumes(scope, ftp.scope) : !level.subsumes(ftp.level)) return false; *asMutable(tp) = GenericTypePack{level}; @@ -136,6 +163,32 @@ void quantify(TypeId ty, TypeLevel level) if (ftv->generics.empty() && ftv->genericPacks.empty() && !q.seenMutableType && !q.seenGenericType) ftv->hasNoGenerics = true; + + ftv->generalized = true; +} + +void quantify(TypeId ty, Scope2* scope) +{ + Quantifier q{scope}; + q.traverse(ty); + + FunctionTypeVar* ftv = getMutable(ty); + LUAU_ASSERT(ftv); + if (FFlag::LuauAlwaysQuantify) + { + ftv->generics.insert(ftv->generics.end(), q.generics.begin(), q.generics.end()); + ftv->genericPacks.insert(ftv->genericPacks.end(), q.genericPacks.begin(), q.genericPacks.end()); + } + else + { + ftv->generics = q.generics; + ftv->genericPacks = q.genericPacks; + } + + if (ftv->generics.empty() && ftv->genericPacks.empty() && !q.seenMutableType && !q.seenGenericType) + ftv->hasNoGenerics = true; + + ftv->generalized = true; } } // namespace Luau diff --git a/Analysis/src/Substitution.cpp b/Analysis/src/Substitution.cpp index e40bedb0..50c516db 100644 --- a/Analysis/src/Substitution.cpp +++ b/Analysis/src/Substitution.cpp @@ -2,6 +2,7 @@ #include "Luau/Substitution.h" #include "Luau/Common.h" +#include "Luau/Clone.h" #include "Luau/TxnLog.h" #include @@ -362,63 +363,7 @@ std::optional Substitution::substitute(TypePackId tp) TypeId Substitution::clone(TypeId ty) { - ty = log->follow(ty); - - TypeId result = ty; - - if (auto pty = log->pending(ty)) - ty = &pty->pending; - - if (const FunctionTypeVar* ftv = get(ty)) - { - FunctionTypeVar clone = FunctionTypeVar{ftv->level, ftv->argTypes, ftv->retType, ftv->definition, ftv->hasSelf}; - clone.generics = ftv->generics; - clone.genericPacks = ftv->genericPacks; - clone.magicFunction = ftv->magicFunction; - clone.tags = ftv->tags; - clone.argNames = ftv->argNames; - result = addType(std::move(clone)); - } - else if (const TableTypeVar* ttv = get(ty)) - { - LUAU_ASSERT(!ttv->boundTo); - TableTypeVar clone = TableTypeVar{ttv->props, ttv->indexer, ttv->level, ttv->state}; - if (!FFlag::LuauNoMethodLocations) - clone.methodDefinitionLocations = ttv->methodDefinitionLocations; - clone.definitionModuleName = ttv->definitionModuleName; - clone.name = ttv->name; - clone.syntheticName = ttv->syntheticName; - clone.instantiatedTypeParams = ttv->instantiatedTypeParams; - clone.instantiatedTypePackParams = ttv->instantiatedTypePackParams; - clone.tags = ttv->tags; - result = addType(std::move(clone)); - } - else if (const MetatableTypeVar* mtv = get(ty)) - { - MetatableTypeVar clone = MetatableTypeVar{mtv->table, mtv->metatable}; - clone.syntheticName = mtv->syntheticName; - result = addType(std::move(clone)); - } - else if (const UnionTypeVar* utv = get(ty)) - { - UnionTypeVar clone; - clone.options = utv->options; - result = addType(std::move(clone)); - } - else if (const IntersectionTypeVar* itv = get(ty)) - { - IntersectionTypeVar clone; - clone.parts = itv->parts; - result = addType(std::move(clone)); - } - else if (const ConstrainedTypeVar* ctv = get(ty)) - { - ConstrainedTypeVar clone{ctv->level, ctv->parts}; - result = addType(std::move(clone)); - } - - asMutable(result)->documentationSymbol = ty->documentationSymbol; - return result; + return shallowClone(ty, *arena, log); } TypePackId Substitution::clone(TypePackId tp) diff --git a/Analysis/src/ToString.cpp b/Analysis/src/ToString.cpp index a4a3ec49..8490350d 100644 --- a/Analysis/src/ToString.cpp +++ b/Analysis/src/ToString.cpp @@ -18,7 +18,6 @@ LUAU_FASTFLAG(LuauLowerBoundsCalculation) * Fair warning: Setting this will break a lot of Luau unit tests. */ LUAU_FASTFLAGVARIABLE(DebugLuauVerboseTypeNames, false) -LUAU_FASTFLAGVARIABLE(LuauDocFuncParameters, false) namespace Luau { @@ -1196,65 +1195,38 @@ std::string toStringNamedFunction(const std::string& funcName, const FunctionTyp auto argPackIter = begin(ftv.argTypes); bool first = true; - if (FFlag::LuauDocFuncParameters) + size_t idx = 0; + while (argPackIter != end(ftv.argTypes)) { - size_t idx = 0; - while (argPackIter != end(ftv.argTypes)) + // ftv takes a self parameter as the first argument, skip it if specified in option + if (idx == 0 && ftv.hasSelf && opts.hideFunctionSelfArgument) { - // ftv takes a self parameter as the first argument, skip it if specified in option - if (idx == 0 && ftv.hasSelf && opts.hideFunctionSelfArgument) - { - ++argPackIter; - ++idx; - continue; - } - - if (!first) - state.emit(", "); - first = false; - - // We don't respect opts.functionTypeArguments - if (idx < opts.namedFunctionOverrideArgNames.size()) - { - state.emit(opts.namedFunctionOverrideArgNames[idx] + ": "); - } - else if (idx < ftv.argNames.size() && ftv.argNames[idx]) - { - state.emit(ftv.argNames[idx]->name + ": "); - } - else - { - state.emit("_: "); - } - tvs.stringify(*argPackIter); - ++argPackIter; ++idx; + continue; } - } - else - { - auto argNameIter = ftv.argNames.begin(); - while (argPackIter != end(ftv.argTypes)) + + if (!first) + state.emit(", "); + first = false; + + // We don't respect opts.functionTypeArguments + if (idx < opts.namedFunctionOverrideArgNames.size()) { - if (!first) - state.emit(", "); - first = false; - - // We don't currently respect opts.functionTypeArguments. I don't think this function should. - if (argNameIter != ftv.argNames.end()) - { - state.emit((*argNameIter ? (*argNameIter)->name : "_") + ": "); - ++argNameIter; - } - else - { - state.emit("_: "); - } - - tvs.stringify(*argPackIter); - ++argPackIter; + state.emit(opts.namedFunctionOverrideArgNames[idx] + ": "); } + else if (idx < ftv.argNames.size() && ftv.argNames[idx]) + { + state.emit(ftv.argNames[idx]->name + ": "); + } + else + { + state.emit("_: "); + } + tvs.stringify(*argPackIter); + + ++argPackIter; + ++idx; } if (argPackIter.tail()) @@ -1337,4 +1309,55 @@ std::string generateName(size_t i) return n; } +std::string toString(const Constraint& c, ToStringOptions& opts) +{ + if (const SubtypeConstraint* sc = Luau::get_if(&c.c)) + { + ToStringResult subStr = toStringDetailed(sc->subType, opts); + opts.nameMap = std::move(subStr.nameMap); + ToStringResult superStr = toStringDetailed(sc->superType, opts); + opts.nameMap = std::move(superStr.nameMap); + return subStr.name + " <: " + superStr.name; + } + else if (const PackSubtypeConstraint* psc = Luau::get_if(&c.c)) + { + ToStringResult subStr = toStringDetailed(psc->subPack, opts); + opts.nameMap = std::move(subStr.nameMap); + ToStringResult superStr = toStringDetailed(psc->superPack, opts); + opts.nameMap = std::move(superStr.nameMap); + return subStr.name + " <: " + superStr.name; + } + else if (const GeneralizationConstraint* gc = Luau::get_if(&c.c)) + { + ToStringResult subStr = toStringDetailed(gc->subType, opts); + opts.nameMap = std::move(subStr.nameMap); + ToStringResult superStr = toStringDetailed(gc->superType, opts); + opts.nameMap = std::move(superStr.nameMap); + return subStr.name + " ~ gen " + superStr.name; + } + else if (const InstantiationConstraint* ic = Luau::get_if(&c.c)) + { + ToStringResult subStr = toStringDetailed(ic->subType, opts); + opts.nameMap = std::move(subStr.nameMap); + ToStringResult superStr = toStringDetailed(ic->superType, opts); + opts.nameMap = std::move(superStr.nameMap); + return subStr.name + " ~ inst " + superStr.name; + } + else + { + LUAU_ASSERT(false); + return ""; + } +} + +std::string dump(const Constraint& c) +{ + ToStringOptions opts; + opts.exhaustive = true; + opts.functionTypeArguments = true; + std::string s = toString(c, opts); + printf("%s\n", s.c_str()); + return s; +} + } // namespace Luau diff --git a/Analysis/src/TypeInfer.cpp b/Analysis/src/TypeInfer.cpp index 11813c76..4931bc59 100644 --- a/Analysis/src/TypeInfer.cpp +++ b/Analysis/src/TypeInfer.cpp @@ -30,7 +30,6 @@ LUAU_FASTINTVARIABLE(LuauCheckRecursionLimit, 300) LUAU_FASTINTVARIABLE(LuauVisitRecursionLimit, 500) LUAU_FASTFLAG(LuauKnowsTheDataModel3) LUAU_FASTFLAG(LuauAutocompleteDynamicLimits) -LUAU_FASTFLAGVARIABLE(LuauDoNotRelyOnNextBinding, false) LUAU_FASTFLAGVARIABLE(LuauExpectedPropTypeFromIndexer, false) LUAU_FASTFLAGVARIABLE(LuauLowerBoundsCalculation, false) LUAU_FASTFLAGVARIABLE(DebugLuauFreezeDuringUnification, false) @@ -1182,12 +1181,7 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatForIn& forin) unify(varTy, var, forin.location); if (!get(iterTy) && !get(iterTy) && !get(iterTy)) - { - if (FFlag::LuauDoNotRelyOnNextBinding) - reportError(firstValue->location, CannotCallNonFunction{iterTy}); - else - reportError(TypeError{firstValue->location, TypeMismatch{globalScope->bindings[AstName{"next"}].typeId, iterTy}}); - } + reportError(firstValue->location, CannotCallNonFunction{iterTy}); return check(loopScope, *forin.body); } @@ -3714,7 +3708,7 @@ ExprResult TypeChecker::checkExprPack(const ScopePtr& scope, const A { retPack = freshTypePack(free->level); TypePackId freshArgPack = freshTypePack(free->level); - *asMutable(actualFunctionType) = FunctionTypeVar(free->level, freshArgPack, retPack); + asMutable(actualFunctionType)->ty.emplace(free->level, freshArgPack, retPack); } else retPack = freshTypePack(scope->level); diff --git a/Analysis/src/TypeVar.cpp b/Analysis/src/TypeVar.cpp index 2355dab2..12cbed91 100644 --- a/Analysis/src/TypeVar.cpp +++ b/Analysis/src/TypeVar.cpp @@ -24,7 +24,6 @@ LUAU_FASTINTVARIABLE(LuauTypeMaximumStringifierLength, 500) LUAU_FASTINTVARIABLE(LuauTableTypeMaximumStringifierLength, 0) LUAU_FASTINT(LuauTypeInferRecursionLimit) LUAU_FASTFLAG(LuauSubtypingAddOptPropsToUnsealedTables) -LUAU_FASTFLAGVARIABLE(LuauClassDefinitionModuleInError, false) namespace Luau { @@ -302,7 +301,7 @@ std::optional getDefinitionModuleName(TypeId type) if (ftv->definition) return ftv->definition->definitionModuleName; } - else if (auto ctv = get(type); ctv && FFlag::LuauClassDefinitionModuleInError) + else if (auto ctv = get(type)) { if (!ctv->definitionModuleName.empty()) return ctv->definitionModuleName; @@ -724,7 +723,7 @@ TypeId SingletonTypes::makeStringMetatable() TableTypeVar::Props stringLib = { {"byte", {arena->addType(FunctionTypeVar{arena->addTypePack({stringType, optionalNumber, optionalNumber}), numberVariadicList})}}, - {"char", {arena->addType(FunctionTypeVar{arena->addTypePack(TypePack{{numberType}, numberVariadicList}), arena->addTypePack({stringType})})}}, + {"char", {arena->addType(FunctionTypeVar{numberVariadicList, arena->addTypePack({stringType})})}}, {"find", {makeFunction(*arena, stringType, {}, {}, {stringType, optionalNumber, optionalBoolean}, {}, {optionalNumber, optionalNumber})}}, {"format", {formatFn}}, // FIXME {"gmatch", {gmatchFunc}}, diff --git a/Analysis/src/Unifiable.cpp b/Analysis/src/Unifiable.cpp index dc554664..fe878358 100644 --- a/Analysis/src/Unifiable.cpp +++ b/Analysis/src/Unifiable.cpp @@ -12,12 +12,16 @@ Free::Free(TypeLevel level) { } +Free::Free(Scope2* scope) + : scope(scope) +{ +} + int Free::nextIndex = 0; Generic::Generic() : index(++nextIndex) , name("g" + std::to_string(index)) - , explicitName(false) { } @@ -25,7 +29,6 @@ Generic::Generic(TypeLevel level) : index(++nextIndex) , level(level) , name("g" + std::to_string(index)) - , explicitName(false) { } @@ -36,6 +39,12 @@ Generic::Generic(const Name& name) { } +Generic::Generic(Scope2* scope) + : index(++nextIndex) + , scope(scope) +{ +} + Generic::Generic(TypeLevel level, const Name& name) : index(++nextIndex) , level(level) diff --git a/Sources.cmake b/Sources.cmake index 82993349..99007e89 100644 --- a/Sources.cmake +++ b/Sources.cmake @@ -65,9 +65,12 @@ target_sources(Luau.CodeGen PRIVATE target_sources(Luau.Analysis PRIVATE Analysis/include/Luau/AstQuery.h Analysis/include/Luau/Autocomplete.h + Analysis/include/Luau/NotNull.h Analysis/include/Luau/BuiltinDefinitions.h - Analysis/include/Luau/Config.h Analysis/include/Luau/Clone.h + Analysis/include/Luau/Config.h + Analysis/include/Luau/ConstraintGraphBuilder.h + Analysis/include/Luau/ConstraintSolver.h Analysis/include/Luau/Documentation.h Analysis/include/Luau/Error.h Analysis/include/Luau/FileResolver.h @@ -108,8 +111,10 @@ target_sources(Luau.Analysis PRIVATE Analysis/src/AstQuery.cpp Analysis/src/Autocomplete.cpp Analysis/src/BuiltinDefinitions.cpp - Analysis/src/Config.cpp Analysis/src/Clone.cpp + Analysis/src/Config.cpp + Analysis/src/ConstraintGraphBuilder.cpp + Analysis/src/ConstraintSolver.cpp Analysis/src/Error.cpp Analysis/src/Frontend.cpp Analysis/src/Instantiation.cpp @@ -240,6 +245,7 @@ if(TARGET Luau.UnitTest) tests/AstQuery.test.cpp tests/AstVisitor.test.cpp tests/Autocomplete.test.cpp + tests/NotNull.test.cpp tests/BuiltinDefinitions.test.cpp tests/Compiler.test.cpp tests/Config.test.cpp @@ -252,6 +258,8 @@ if(TARGET Luau.UnitTest) tests/Module.test.cpp tests/NonstrictMode.test.cpp tests/Normalize.test.cpp + tests/ConstraintGraphBuilder.test.cpp + tests/ConstraintSolver.test.cpp tests/Parser.test.cpp tests/RequireTracer.test.cpp tests/RuntimeLimits.test.cpp diff --git a/VM/src/lapi.cpp b/VM/src/lapi.cpp index f3be64b8..f86371da 100644 --- a/VM/src/lapi.cpp +++ b/VM/src/lapi.cpp @@ -14,8 +14,6 @@ #include -LUAU_FASTFLAG(LuauGcWorkTrackFix) - const char* lua_ident = "$Lua: Lua 5.1.4 Copyright (C) 1994-2008 Lua.org, PUC-Rio $\n" "$Authors: R. Ierusalimschy, L. H. de Figueiredo & W. Celes $\n" "$URL: www.lua.org $\n"; @@ -488,11 +486,11 @@ void* lua_touserdata(lua_State* L, int idx) { StkId o = index2addr(L, idx); if (ttisuserdata(o)) - return uvalue(o)->data; + return uvalue(o)->data; else if (ttislightuserdata(o)) - return pvalue(o); + return pvalue(o); else - return NULL; + return NULL; } void* lua_touserdatatagged(lua_State* L, int idx, int tag) @@ -1054,7 +1052,6 @@ int lua_gc(lua_State* L, int what, int data) } case LUA_GCSTEP: { - size_t prevthreshold = g->GCthreshold; size_t amount = (cast_to(size_t, data) << 10); ptrdiff_t oldcredit = g->gcstate == GCSpause ? 0 : g->GCthreshold - g->totalbytes; @@ -1064,8 +1061,6 @@ int lua_gc(lua_State* L, int what, int data) else g->GCthreshold = 0; - bool waspaused = g->gcstate == GCSpause; - #ifdef LUAI_GCMETRICS double startmarktime = g->gcmetrics.currcycle.marktime; double startsweeptime = g->gcmetrics.currcycle.sweeptime; @@ -1078,7 +1073,7 @@ int lua_gc(lua_State* L, int what, int data) { size_t stepsize = luaC_step(L, false); - actualwork += FFlag::LuauGcWorkTrackFix ? stepsize : g->gcstepsize; + actualwork += stepsize; if (g->gcstate == GCSpause) { /* end of cycle? */ @@ -1114,20 +1109,9 @@ int lua_gc(lua_State* L, int what, int data) // if cycle hasn't finished, advance threshold forward for the amount of extra work performed if (g->gcstate != GCSpause) { - if (FFlag::LuauGcWorkTrackFix) - { - // if a new cycle was triggered by explicit step, old 'credit' of GC work is 0 - ptrdiff_t newthreshold = g->totalbytes + actualwork + oldcredit; - g->GCthreshold = newthreshold < 0 ? 0 : newthreshold; - } - else - { - // if a new cycle was triggered by explicit step, we ignore old threshold as that shows an incorrect 'credit' of GC work - if (waspaused) - g->GCthreshold = g->totalbytes + actualwork; - else - g->GCthreshold = prevthreshold + actualwork; - } + // if a new cycle was triggered by explicit step, old 'credit' of GC work is 0 + ptrdiff_t newthreshold = g->totalbytes + actualwork + oldcredit; + g->GCthreshold = newthreshold < 0 ? 0 : newthreshold; } break; } diff --git a/VM/src/ldo.cpp b/VM/src/ldo.cpp index 4cab746a..a71fce52 100644 --- a/VM/src/ldo.cpp +++ b/VM/src/ldo.cpp @@ -213,7 +213,7 @@ CallInfo* luaD_growCI(lua_State* L) return ++L->ci; } -void luaD_checkCstack(lua_State *L) +void luaD_checkCstack(lua_State* L) { if (L->nCcalls == LUAI_MAXCCALLS) luaG_runerror(L, "C stack overflow"); diff --git a/VM/src/lgc.cpp b/VM/src/lgc.cpp index e7b73fe7..70b4dbf9 100644 --- a/VM/src/lgc.cpp +++ b/VM/src/lgc.cpp @@ -13,10 +13,7 @@ #include -LUAU_FASTFLAGVARIABLE(LuauGcWorkTrackFix, false) -LUAU_FASTFLAGVARIABLE(LuauGcSweepCostFix, false) - -#define GC_SWEEPPAGESTEPCOST (FFlag::LuauGcSweepCostFix ? 16 : 4) +#define GC_SWEEPPAGESTEPCOST 16 #define GC_INTERRUPT(state) \ { \ @@ -881,7 +878,7 @@ size_t luaC_step(lua_State* L, bool assist) { global_State* g = L->global; - int lim = FFlag::LuauGcWorkTrackFix ? g->gcstepsize * g->gcstepmul / 100 : (g->gcstepsize / 100) * g->gcstepmul; /* how much to work */ + int lim = g->gcstepsize * g->gcstepmul / 100; /* how much to work */ LUAU_ASSERT(g->totalbytes >= g->GCthreshold); size_t debt = g->totalbytes - g->GCthreshold; @@ -927,10 +924,10 @@ size_t luaC_step(lua_State* L, bool assist) } else { - g->GCthreshold = g->totalbytes + (FFlag::LuauGcWorkTrackFix ? actualstepsize : g->gcstepsize); + g->GCthreshold = g->totalbytes + actualstepsize; // compensate if GC is "behind schedule" (has some debt to pay) - if (FFlag::LuauGcWorkTrackFix ? g->GCthreshold >= debt : g->GCthreshold > debt) + if (g->GCthreshold >= debt) g->GCthreshold -= debt; } diff --git a/tests/Compiler.test.cpp b/tests/Compiler.test.cpp index f723e0d1..20139650 100644 --- a/tests/Compiler.test.cpp +++ b/tests/Compiler.test.cpp @@ -4471,7 +4471,6 @@ end RETURN R0 0 RETURN R0 0 )"); - } TEST_CASE("LoopUnrollNestedClosure") diff --git a/tests/Conformance.test.cpp b/tests/Conformance.test.cpp index 7b4e83ba..f7f2b4ac 100644 --- a/tests/Conformance.test.cpp +++ b/tests/Conformance.test.cpp @@ -242,11 +242,14 @@ TEST_CASE("Math") TEST_CASE("Table") { runConformance("nextvar.lua", [](lua_State* L) { - lua_pushcfunction(L, [](lua_State* L) { - unsigned v = luaL_checkunsigned(L, 1); - lua_pushlightuserdata(L, reinterpret_cast(uintptr_t(v))); - return 1; - }, "makelud"); + lua_pushcfunction( + L, + [](lua_State* L) { + unsigned v = luaL_checkunsigned(L, 1); + lua_pushlightuserdata(L, reinterpret_cast(uintptr_t(v))); + return 1; + }, + "makelud"); lua_setglobal(L, "makelud"); }); } @@ -1150,129 +1153,171 @@ TEST_CASE("Userdata") gInt64MT = lua_ref(L, -1); // __index - lua_pushcfunction(L, [](lua_State* L) { - void* p = lua_touserdatatagged(L, 1, kInt64Tag); - if (!p) - luaL_typeerror(L, 1, "int64"); + lua_pushcfunction( + L, + [](lua_State* L) { + void* p = lua_touserdatatagged(L, 1, kInt64Tag); + if (!p) + luaL_typeerror(L, 1, "int64"); - const char* name = luaL_checkstring(L, 2); + const char* name = luaL_checkstring(L, 2); - if (strcmp(name, "value") == 0) - { - lua_pushnumber(L, double(*static_cast(p))); - return 1; - } + if (strcmp(name, "value") == 0) + { + lua_pushnumber(L, double(*static_cast(p))); + return 1; + } - luaL_error(L, "unknown field %s", name); - }, nullptr); + luaL_error(L, "unknown field %s", name); + }, + nullptr); lua_setfield(L, -2, "__index"); // __newindex - lua_pushcfunction(L, [](lua_State* L) { - void* p = lua_touserdatatagged(L, 1, kInt64Tag); - if (!p) - luaL_typeerror(L, 1, "int64"); + lua_pushcfunction( + L, + [](lua_State* L) { + void* p = lua_touserdatatagged(L, 1, kInt64Tag); + if (!p) + luaL_typeerror(L, 1, "int64"); - const char* name = luaL_checkstring(L, 2); + const char* name = luaL_checkstring(L, 2); - if (strcmp(name, "value") == 0) - { - double value = luaL_checknumber(L, 3); - *static_cast(p) = int64_t(value); - return 0; - } + if (strcmp(name, "value") == 0) + { + double value = luaL_checknumber(L, 3); + *static_cast(p) = int64_t(value); + return 0; + } - luaL_error(L, "unknown field %s", name); - }, nullptr); + luaL_error(L, "unknown field %s", name); + }, + nullptr); lua_setfield(L, -2, "__newindex"); // __eq - lua_pushcfunction(L, [](lua_State* L) { - lua_pushboolean(L, getInt64(L, 1) == getInt64(L, 2)); - return 1; - }, nullptr); + lua_pushcfunction( + L, + [](lua_State* L) { + lua_pushboolean(L, getInt64(L, 1) == getInt64(L, 2)); + return 1; + }, + nullptr); lua_setfield(L, -2, "__eq"); // __lt - lua_pushcfunction(L, [](lua_State* L) { - lua_pushboolean(L, getInt64(L, 1) < getInt64(L, 2)); - return 1; - }, nullptr); + lua_pushcfunction( + L, + [](lua_State* L) { + lua_pushboolean(L, getInt64(L, 1) < getInt64(L, 2)); + return 1; + }, + nullptr); lua_setfield(L, -2, "__lt"); // __le - lua_pushcfunction(L, [](lua_State* L) { - lua_pushboolean(L, getInt64(L, 1) <= getInt64(L, 2)); - return 1; - }, nullptr); + lua_pushcfunction( + L, + [](lua_State* L) { + lua_pushboolean(L, getInt64(L, 1) <= getInt64(L, 2)); + return 1; + }, + nullptr); lua_setfield(L, -2, "__le"); // __add - lua_pushcfunction(L, [](lua_State* L) { - pushInt64(L, getInt64(L, 1) + getInt64(L, 2)); - return 1; - }, nullptr); + lua_pushcfunction( + L, + [](lua_State* L) { + pushInt64(L, getInt64(L, 1) + getInt64(L, 2)); + return 1; + }, + nullptr); lua_setfield(L, -2, "__add"); // __sub - lua_pushcfunction(L, [](lua_State* L) { - pushInt64(L, getInt64(L, 1) - getInt64(L, 2)); - return 1; - }, nullptr); + lua_pushcfunction( + L, + [](lua_State* L) { + pushInt64(L, getInt64(L, 1) - getInt64(L, 2)); + return 1; + }, + nullptr); lua_setfield(L, -2, "__sub"); // __mul - lua_pushcfunction(L, [](lua_State* L) { - pushInt64(L, getInt64(L, 1) * getInt64(L, 2)); - return 1; - }, nullptr); + lua_pushcfunction( + L, + [](lua_State* L) { + pushInt64(L, getInt64(L, 1) * getInt64(L, 2)); + return 1; + }, + nullptr); lua_setfield(L, -2, "__mul"); // __div - lua_pushcfunction(L, [](lua_State* L) { - // ideally we'd guard against 0 but it's a test so eh - pushInt64(L, getInt64(L, 1) / getInt64(L, 2)); - return 1; - }, nullptr); + lua_pushcfunction( + L, + [](lua_State* L) { + // ideally we'd guard against 0 but it's a test so eh + pushInt64(L, getInt64(L, 1) / getInt64(L, 2)); + return 1; + }, + nullptr); lua_setfield(L, -2, "__div"); // __mod - lua_pushcfunction(L, [](lua_State* L) { - // ideally we'd guard against 0 and INT64_MIN but it's a test so eh - pushInt64(L, getInt64(L, 1) % getInt64(L, 2)); - return 1; - }, nullptr); + lua_pushcfunction( + L, + [](lua_State* L) { + // ideally we'd guard against 0 and INT64_MIN but it's a test so eh + pushInt64(L, getInt64(L, 1) % getInt64(L, 2)); + return 1; + }, + nullptr); lua_setfield(L, -2, "__mod"); // __pow - lua_pushcfunction(L, [](lua_State* L) { - pushInt64(L, int64_t(pow(double(getInt64(L, 1)), double(getInt64(L, 2))))); - return 1; - }, nullptr); + lua_pushcfunction( + L, + [](lua_State* L) { + pushInt64(L, int64_t(pow(double(getInt64(L, 1)), double(getInt64(L, 2))))); + return 1; + }, + nullptr); lua_setfield(L, -2, "__pow"); // __unm - lua_pushcfunction(L, [](lua_State* L) { - pushInt64(L, -getInt64(L, 1)); - return 1; - }, nullptr); + lua_pushcfunction( + L, + [](lua_State* L) { + pushInt64(L, -getInt64(L, 1)); + return 1; + }, + nullptr); lua_setfield(L, -2, "__unm"); // __tostring - lua_pushcfunction(L, [](lua_State* L) { - int64_t value = getInt64(L, 1); - std::string str = std::to_string(value); - lua_pushlstring(L, str.c_str(), str.length()); - return 1; - }, nullptr); + lua_pushcfunction( + L, + [](lua_State* L) { + int64_t value = getInt64(L, 1); + std::string str = std::to_string(value); + lua_pushlstring(L, str.c_str(), str.length()); + return 1; + }, + nullptr); lua_setfield(L, -2, "__tostring"); // ctor - lua_pushcfunction(L, [](lua_State* L) { - double v = luaL_checknumber(L, 1); - pushInt64(L, int64_t(v)); - return 1; - }, "int64"); + lua_pushcfunction( + L, + [](lua_State* L) { + double v = luaL_checknumber(L, 1); + pushInt64(L, int64_t(v)); + return 1; + }, + "int64"); lua_setglobal(L, "int64"); }); } diff --git a/tests/ConstraintGraphBuilder.test.cpp b/tests/ConstraintGraphBuilder.test.cpp new file mode 100644 index 00000000..ab5af4f6 --- /dev/null +++ b/tests/ConstraintGraphBuilder.test.cpp @@ -0,0 +1,107 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details + +#include "Fixture.h" +#include "Luau/ConstraintGraphBuilder.h" + +#include "doctest.h" + +using namespace Luau; + +TEST_SUITE_BEGIN("ConstraintGraphBuilder"); + +TEST_CASE_FIXTURE(ConstraintGraphBuilderFixture, "hello_world") +{ + AstStatBlock* block = parse(R"( + local a = "hello" + local b = a + )"); + + cgb.visit(block); + std::vector constraints = collectConstraints(cgb.rootScope); + + REQUIRE(2 == constraints.size()); + + ToStringOptions opts; + CHECK("a <: string" == toString(*constraints[0], opts)); + CHECK("b <: a" == toString(*constraints[1], opts)); +} + +TEST_CASE_FIXTURE(ConstraintGraphBuilderFixture, "primitives") +{ + AstStatBlock* block = parse(R"( + local s = "hello" + local n = 555 + local b = true + local n2 = nil + )"); + + cgb.visit(block); + std::vector constraints = collectConstraints(cgb.rootScope); + + REQUIRE(4 == constraints.size()); + + ToStringOptions opts; + CHECK("a <: string" == toString(*constraints[0], opts)); + CHECK("b <: number" == toString(*constraints[1], opts)); + CHECK("c <: boolean" == toString(*constraints[2], opts)); + CHECK("d <: nil" == toString(*constraints[3], opts)); +} + +TEST_CASE_FIXTURE(ConstraintGraphBuilderFixture, "function_application") +{ + AstStatBlock* block = parse(R"( + local a = "hello" + local b = a("world") + )"); + + cgb.visit(block); + std::vector constraints = collectConstraints(cgb.rootScope); + + REQUIRE(4 == constraints.size()); + + ToStringOptions opts; + CHECK("a <: string" == toString(*constraints[0], opts)); + CHECK("b ~ inst a" == toString(*constraints[1], opts)); + CHECK("(string) -> (c, d...) <: b" == toString(*constraints[2], opts)); + CHECK("e <: c" == toString(*constraints[3], opts)); +} + +TEST_CASE_FIXTURE(ConstraintGraphBuilderFixture, "local_function_definition") +{ + AstStatBlock* block = parse(R"( + local function f(a) + return a + end + )"); + + cgb.visit(block); + std::vector constraints = collectConstraints(cgb.rootScope); + + REQUIRE(2 == constraints.size()); + + ToStringOptions opts; + CHECK("a ~ gen (b) -> (c...)" == toString(*constraints[0], opts)); + CHECK("b <: c..." == toString(*constraints[1], opts)); +} + +TEST_CASE_FIXTURE(ConstraintGraphBuilderFixture, "recursive_function") +{ + AstStatBlock* block = parse(R"( + local function f(a) + return f(a) + end + )"); + + cgb.visit(block); + std::vector constraints = collectConstraints(cgb.rootScope); + + REQUIRE(4 == constraints.size()); + + ToStringOptions opts; + CHECK("a ~ gen (b) -> (c...)" == toString(*constraints[0], opts)); + CHECK("d ~ inst a" == toString(*constraints[1], opts)); + CHECK("(b) -> (e, f...) <: d" == toString(*constraints[2], opts)); + CHECK("e <: c..." == toString(*constraints[3], opts)); +} + +TEST_SUITE_END(); diff --git a/tests/ConstraintSolver.test.cpp b/tests/ConstraintSolver.test.cpp new file mode 100644 index 00000000..5959f55c --- /dev/null +++ b/tests/ConstraintSolver.test.cpp @@ -0,0 +1,87 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details + +#include "Fixture.h" + +#include "doctest.h" + +#include "Luau/ConstraintGraphBuilder.h" +#include "Luau/ConstraintSolver.h" + +using namespace Luau; + +static TypeId requireBinding(Scope2* scope, const char* name) +{ + auto b = linearSearchForBinding(scope, name); + LUAU_ASSERT(b.has_value()); + return *b; +} + +TEST_SUITE_BEGIN("ConstraintSolver"); + +TEST_CASE_FIXTURE(ConstraintGraphBuilderFixture, "hello") +{ + AstStatBlock* block = parse(R"( + local a = 55 + local b = a + )"); + + cgb.visit(block); + + ConstraintSolver cs{&arena, cgb.rootScope}; + + cs.run(); + + TypeId bType = requireBinding(cgb.rootScope, "b"); + + CHECK("number" == toString(bType)); +} + +TEST_CASE_FIXTURE(ConstraintGraphBuilderFixture, "generic_function") +{ + AstStatBlock* block = parse(R"( + local function id(a) + return a + end + )"); + + cgb.visit(block); + + ConstraintSolver cs{&arena, cgb.rootScope}; + + cs.run(); + + TypeId idType = requireBinding(cgb.rootScope, "id"); + + CHECK("(a) -> a" == toString(idType)); +} + +#if 1 +TEST_CASE_FIXTURE(ConstraintGraphBuilderFixture, "proper_let_generalization") +{ + AstStatBlock* block = parse(R"( + local function a(c) + local function d(e) + return c + end + + return d + end + + local b = a(5) + )"); + + cgb.visit(block); + + ToStringOptions opts; + + ConstraintSolver cs{&arena, cgb.rootScope}; + + cs.run(); + + TypeId idType = requireBinding(cgb.rootScope, "b"); + + CHECK("(a) -> number" == toString(idType, opts)); +} +#endif + +TEST_SUITE_END(); diff --git a/tests/Fixture.cpp b/tests/Fixture.cpp index 03f3e15c..232ec2de 100644 --- a/tests/Fixture.cpp +++ b/tests/Fixture.cpp @@ -17,6 +17,8 @@ static const char* mainModuleName = "MainModule"; +LUAU_FASTFLAG(DebugLuauDeferredConstraintResolution); + namespace Luau { @@ -249,7 +251,10 @@ std::optional Fixture::getType(const std::string& name) ModulePtr module = getMainModule(); REQUIRE(module); - return lookupName(module->getModuleScope(), name); + if (FFlag::DebugLuauDeferredConstraintResolution) + return linearSearchForBinding(module->getModuleScope2(), name.c_str()); + else + return lookupName(module->getModuleScope(), name); } TypeId Fixture::requireType(const std::string& name) @@ -421,6 +426,12 @@ BuiltinsFixture::BuiltinsFixture(bool freeze, bool prepareAutocomplete) Luau::freeze(frontend.typeCheckerForAutocomplete.globalTypes); } +ConstraintGraphBuilderFixture::ConstraintGraphBuilderFixture() + : Fixture() + , forceTheFlag{"DebugLuauDeferredConstraintResolution", true} +{ +} + ModuleName fromString(std::string_view name) { return ModuleName(name); @@ -460,4 +471,27 @@ std::optional lookupName(ScopePtr scope, const std::string& name) return std::nullopt; } +std::optional linearSearchForBinding(Scope2* scope, const char* name) +{ + while (scope) + { + for (const auto& [n, ty] : scope->bindings) + { + if (n.astName() == name) + return ty; + } + + scope = scope->parent; + } + + return std::nullopt; +} + +void dump(const std::vector& constraints) +{ + ToStringOptions opts; + for (const auto& c : constraints) + printf("%s\n", toString(c, opts).c_str()); +} + } // namespace Luau diff --git a/tests/Fixture.h b/tests/Fixture.h index 901f7d42..ffcd4b9e 100644 --- a/tests/Fixture.h +++ b/tests/Fixture.h @@ -2,6 +2,7 @@ #pragma once #include "Luau/Config.h" +#include "Luau/ConstraintGraphBuilder.h" #include "Luau/FileResolver.h" #include "Luau/Frontend.h" #include "Luau/IostreamHelpers.h" @@ -156,6 +157,16 @@ struct BuiltinsFixture : Fixture BuiltinsFixture(bool freeze = true, bool prepareAutocomplete = false); }; +struct ConstraintGraphBuilderFixture : Fixture +{ + TypeArena arena; + ConstraintGraphBuilder cgb{&arena}; + + ScopedFastFlag forceTheFlag; + + ConstraintGraphBuilderFixture(); +}; + ModuleName fromString(std::string_view name); template @@ -175,9 +186,12 @@ bool isInArena(TypeId t, const TypeArena& arena); void dumpErrors(const ModulePtr& module); void dumpErrors(const Module& module); void dump(const std::string& name, TypeId ty); +void dump(const std::vector& constraints); std::optional lookupName(ScopePtr scope, const std::string& name); // Warning: This function runs in O(n**2) +std::optional linearSearchForBinding(Scope2* scope, const char* name); + } // namespace Luau #define LUAU_REQUIRE_ERRORS(result) \ diff --git a/tests/Frontend.test.cpp b/tests/Frontend.test.cpp index 33b81be8..c0554669 100644 --- a/tests/Frontend.test.cpp +++ b/tests/Frontend.test.cpp @@ -1031,8 +1031,6 @@ return false; TEST_CASE("check_without_builtin_next") { - ScopedFastFlag luauDoNotRelyOnNextBinding{"LuauDoNotRelyOnNextBinding", true}; - TestFileResolver fileResolver; TestConfigResolver configResolver; Frontend frontend(&fileResolver, &configResolver); diff --git a/tests/NotNull.test.cpp b/tests/NotNull.test.cpp new file mode 100644 index 00000000..1a323c85 --- /dev/null +++ b/tests/NotNull.test.cpp @@ -0,0 +1,116 @@ +#include "Luau/NotNull.h" + +#include "doctest.h" + +#include +#include +#include + +using Luau::NotNull; + +namespace +{ + +struct Test +{ + int x; + float y; + + static int count; + Test() + { + ++count; + } + + ~Test() + { + --count; + } +}; + +int Test::count = 0; + +} + +int foo(NotNull p) +{ + return *p; +} + +void bar(int* q) +{} + +TEST_SUITE_BEGIN("NotNull"); + +TEST_CASE("basic_stuff") +{ + NotNull a = NotNull{new int(55)}; // Does runtime test + NotNull b{new int(55)}; // As above + // NotNull c = new int(55); // Nope. Mildly regrettable, but implicit conversion from T* to NotNull in the general case is not good. + + // a = nullptr; // nope + + NotNull d = a; // No runtime test. a is known not to be null. + + int e = *d; + *d = 1; + CHECK(e == 55); + + const NotNull f = d; + *f = 5; // valid: there is a difference between const NotNull and NotNull + // f = a; // nope + + CHECK_EQ(a, d); + CHECK(a != b); + + NotNull g(a); + CHECK(g == a); + + // *g = 123; // nope + + (void)f; + + NotNull t{new Test}; + t->x = 5; + t->y = 3.14f; + + const NotNull u = t; + // u->x = 44; // nope + int v = u->x; + CHECK(v == 5); + + bar(a); + + // a++; // nope + // a[41]; // nope + // a + 41; // nope + // a - 41; // nope + + delete a; + delete b; + delete t; + + CHECK_EQ(0, Test::count); +} + +TEST_CASE("hashable") +{ + std::unordered_map, const char*> map; + NotNull a{new int(8)}; + NotNull b{new int(10)}; + + std::string hello = "hello"; + std::string world = "world"; + + map[a] = hello.c_str(); + map[b] = world.c_str(); + + CHECK_EQ(2, map.size()); + CHECK_EQ(hello.c_str(), map[a]); + CHECK_EQ(world.c_str(), map[b]); + + delete a; + delete b; +} + +TEST_SUITE_END(); diff --git a/tests/ToString.test.cpp b/tests/ToString.test.cpp index b854bc51..4d9fad14 100644 --- a/tests/ToString.test.cpp +++ b/tests/ToString.test.cpp @@ -505,7 +505,6 @@ TEST_CASE_FIXTURE(Fixture, "self_recursive_instantiated_param") TEST_CASE_FIXTURE(Fixture, "toStringNamedFunction_id") { - ScopedFastFlag flag{"LuauDocFuncParameters", true}; CheckResult result = check(R"( local function id(x) return x end )"); @@ -518,7 +517,6 @@ TEST_CASE_FIXTURE(Fixture, "toStringNamedFunction_id") TEST_CASE_FIXTURE(Fixture, "toStringNamedFunction_map") { - ScopedFastFlag flag{"LuauDocFuncParameters", true}; CheckResult result = check(R"( local function map(arr, fn) local t = {} @@ -537,7 +535,6 @@ TEST_CASE_FIXTURE(Fixture, "toStringNamedFunction_map") TEST_CASE_FIXTURE(Fixture, "toStringNamedFunction_generic_pack") { - ScopedFastFlag flag{"LuauDocFuncParameters", true}; CheckResult result = check(R"( local function f(a: number, b: string) end local function test(...: T...): U... @@ -554,7 +551,6 @@ TEST_CASE_FIXTURE(Fixture, "toStringNamedFunction_generic_pack") TEST_CASE("toStringNamedFunction_unit_f") { - ScopedFastFlag flag{"LuauDocFuncParameters", true}; TypePackVar empty{TypePack{}}; FunctionTypeVar ftv{&empty, &empty, {}, false}; CHECK_EQ("f(): ()", toStringNamedFunction("f", ftv)); @@ -562,7 +558,6 @@ TEST_CASE("toStringNamedFunction_unit_f") TEST_CASE_FIXTURE(Fixture, "toStringNamedFunction_variadics") { - ScopedFastFlag flag{"LuauDocFuncParameters", true}; CheckResult result = check(R"( local function f(x: a, ...): (a, a, b...) return x, x, ... @@ -577,7 +572,6 @@ TEST_CASE_FIXTURE(Fixture, "toStringNamedFunction_variadics") TEST_CASE_FIXTURE(Fixture, "toStringNamedFunction_variadics2") { - ScopedFastFlag flag{"LuauDocFuncParameters", true}; CheckResult result = check(R"( local function f(): ...number return 1, 2, 3 @@ -592,7 +586,6 @@ TEST_CASE_FIXTURE(Fixture, "toStringNamedFunction_variadics2") TEST_CASE_FIXTURE(Fixture, "toStringNamedFunction_variadics3") { - ScopedFastFlag flag{"LuauDocFuncParameters", true}; CheckResult result = check(R"( local function f(): (string, ...number) return 'a', 1, 2, 3 @@ -607,7 +600,6 @@ TEST_CASE_FIXTURE(Fixture, "toStringNamedFunction_variadics3") TEST_CASE_FIXTURE(Fixture, "toStringNamedFunction_type_annotation_has_partial_argnames") { - ScopedFastFlag flag{"LuauDocFuncParameters", true}; CheckResult result = check(R"( local f: (number, y: number) -> number )"); @@ -620,7 +612,6 @@ TEST_CASE_FIXTURE(Fixture, "toStringNamedFunction_type_annotation_has_partial_ar TEST_CASE_FIXTURE(Fixture, "toStringNamedFunction_hide_type_params") { - ScopedFastFlag flag{"LuauDocFuncParameters", true}; CheckResult result = check(R"( local function f(x: T, g: (T) -> U)): () end @@ -636,8 +627,6 @@ TEST_CASE_FIXTURE(Fixture, "toStringNamedFunction_hide_type_params") TEST_CASE_FIXTURE(Fixture, "toStringNamedFunction_overrides_param_names") { - ScopedFastFlag flag{"LuauDocFuncParameters", true}; - CheckResult result = check(R"( local function test(a, b : string, ... : number) return a end )"); @@ -665,7 +654,6 @@ TEST_CASE_FIXTURE(Fixture, "pick_distinct_names_for_mixed_explicit_and_implicit_ TEST_CASE_FIXTURE(Fixture, "toStringNamedFunction_include_self_param") { - ScopedFastFlag flag{"LuauDocFuncParameters", true}; CheckResult result = check(R"( local foo = {} function foo:method(arg: string): () @@ -682,7 +670,6 @@ TEST_CASE_FIXTURE(Fixture, "toStringNamedFunction_include_self_param") TEST_CASE_FIXTURE(Fixture, "toStringNamedFunction_hide_self_param") { - ScopedFastFlag flag{"LuauDocFuncParameters", true}; CheckResult result = check(R"( local foo = {} function foo:method(arg: string): () diff --git a/tests/TypeInfer.classes.test.cpp b/tests/TypeInfer.classes.test.cpp index d90129d7..6f4191e3 100644 --- a/tests/TypeInfer.classes.test.cpp +++ b/tests/TypeInfer.classes.test.cpp @@ -470,8 +470,6 @@ caused by: TEST_CASE_FIXTURE(ClassFixture, "class_type_mismatch_with_name_conflict") { - ScopedFastFlag luauClassDefinitionModuleInError{"LuauClassDefinitionModuleInError", true}; - CheckResult result = check(R"( local i = ChildClass.New() type ChildClass = { x: number } diff --git a/tests/TypeInfer.loops.test.cpp b/tests/TypeInfer.loops.test.cpp index a3cae3de..4444cd66 100644 --- a/tests/TypeInfer.loops.test.cpp +++ b/tests/TypeInfer.loops.test.cpp @@ -78,8 +78,6 @@ TEST_CASE_FIXTURE(Fixture, "for_in_with_an_iterator_of_type_any") TEST_CASE_FIXTURE(Fixture, "for_in_loop_should_fail_with_non_function_iterator") { - ScopedFastFlag luauDoNotRelyOnNextBinding{"LuauDoNotRelyOnNextBinding", true}; - CheckResult result = check(R"( local foo = "bar" for i, v in foo do diff --git a/tests/Variant.test.cpp b/tests/Variant.test.cpp index fcf37875..aa0731ca 100644 --- a/tests/Variant.test.cpp +++ b/tests/Variant.test.cpp @@ -13,6 +13,25 @@ struct Foo int x = 42; }; +struct Bar +{ + explicit Bar(int x) + : prop(x * 2) + { + ++count; + } + + ~Bar() + { + --count; + } + + int prop; + static int count; +}; + +int Bar::count = 0; + TEST_SUITE_BEGIN("Variant"); TEST_CASE("DefaultCtor") @@ -46,6 +65,29 @@ TEST_CASE("Create") CHECK(get_if(&v3)->x == 3); } +TEST_CASE("Emplace") +{ + { + Variant v1; + + CHECK(0 == Bar::count); + int& i = v1.emplace(5); + CHECK(5 == i); + + CHECK(0 == Bar::count); + + CHECK(get_if(&v1) == &i); + + Bar& bar = v1.emplace(11); + CHECK(22 == bar.prop); + CHECK(1 == Bar::count); + + CHECK(get_if(&v1) == &bar); + } + + CHECK(0 == Bar::count); +} + TEST_CASE("NonPOD") { // initialize (copy) diff --git a/tools/natvis/CodeGen.natvis b/tools/natvis/CodeGen.natvis index 47ff0db1..5ff6e143 100644 --- a/tools/natvis/CodeGen.natvis +++ b/tools/natvis/CodeGen.natvis @@ -2,7 +2,7 @@ - noreg + noreg rip al @@ -36,14 +36,20 @@ - {reg} - {mem.size,en} ptr[{mem.base} + {mem.index}*{(int)mem.scale,d} + {disp}] + {base} + {memSize,en} ptr[{base} + {index}*{(int)scale,d} + {imm}] + {memSize,en} ptr[{index}*{(int)scale,d} + {imm}] + {memSize,en} ptr[{base} + {imm}] + {memSize,en} ptr[{imm}] {imm} - reg - mem + base imm - disp + memSize + base + index + scale + imm From 9619f036ac39834d3db07a4b89a35e46fd269664 Mon Sep 17 00:00:00 2001 From: Daniel Nachun Date: Mon, 6 Jun 2022 15:52:55 -0700 Subject: [PATCH 265/399] fix build with newer GCC (#522) --- Analysis/include/Luau/Variant.h | 1 + 1 file changed, 1 insertion(+) diff --git a/Analysis/include/Luau/Variant.h b/Analysis/include/Luau/Variant.h index c9c97c92..f637222e 100644 --- a/Analysis/include/Luau/Variant.h +++ b/Analysis/include/Luau/Variant.h @@ -6,6 +6,7 @@ #include #include #include +#include namespace Luau { From b44912cd203c526bb470382cebf4b81ee737443a Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Petri=20H=C3=A4kkinen?= Date: Thu, 9 Jun 2022 19:41:52 +0300 Subject: [PATCH 266/399] Allow vector fastcall constructor to work with 3-4 arguments with 4-wide vectors (#511) --- VM/src/lbuiltins.cpp | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/VM/src/lbuiltins.cpp b/VM/src/lbuiltins.cpp index cc6e560a..deaf1407 100644 --- a/VM/src/lbuiltins.cpp +++ b/VM/src/lbuiltins.cpp @@ -1018,18 +1018,20 @@ static int luauF_tunpack(lua_State* L, StkId res, TValue* arg0, int nresults, St static int luauF_vector(lua_State* L, StkId res, TValue* arg0, int nresults, StkId args, int nparams) { -#if LUA_VECTOR_SIZE == 4 - if (nparams >= 4 && nresults <= 1 && ttisnumber(arg0) && ttisnumber(args) && ttisnumber(args + 1) && ttisnumber(args + 2)) -#else if (nparams >= 3 && nresults <= 1 && ttisnumber(arg0) && ttisnumber(args) && ttisnumber(args + 1)) -#endif { double x = nvalue(arg0); double y = nvalue(args); double z = nvalue(args + 1); #if LUA_VECTOR_SIZE == 4 - double w = nvalue(args + 2); + double w = 0.0; + if (nparams >= 4) + { + if (!ttisnumber(args + 2)) + return -1; + w = nvalue(args + 2); + } setvvalue(res, float(x), float(y), float(z), float(w)); #else setvvalue(res, float(x), float(y), float(z), 0.0f); From b8ef74372167580cc5a053dea21792ec7590a1d1 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Thu, 9 Jun 2022 10:03:37 -0700 Subject: [PATCH 267/399] RFC: Do not implement safe navigation operator (#501) This meta-RFC proposes removing the previously accepted RFC on safe navigation operator. This is probably going to be disappointing but is the best course of action that reflects our ideals in language evolution. See PR thread for rationale and discussion. --- rfcs/syntax-safe-navigation-operator.md | 104 ------------------------ 1 file changed, 104 deletions(-) delete mode 100644 rfcs/syntax-safe-navigation-operator.md diff --git a/rfcs/syntax-safe-navigation-operator.md b/rfcs/syntax-safe-navigation-operator.md deleted file mode 100644 index 11c4b37f..00000000 --- a/rfcs/syntax-safe-navigation-operator.md +++ /dev/null @@ -1,104 +0,0 @@ -# Safe navigation postfix operator (?) - -**Note**: We have unresolved issues with interaction between this feature and Roblox instance hierarchy. This may affect the viability of this proposal. - -## Summary - -Introduce syntax to navigate through `nil` values, or short-circuit with `nil` if it was encountered. - - -## Motivation - -nil values are very common in Lua, and take care to prevent runtime errors. - -Currently, attempting to index `dog.name` while caring for `dog` being nil requires some form of the following: - -```lua -local dogName = nil -if dog ~= nil then - dogName = dog.name -end -``` - -...or the unusual to read... - -```lua -local dogName = dog and dog.name -``` - -...which will return `false` if `dog` is `false`, instead of throwing an error because of the index of `false.name`. - -Luau provides the if...else expression making this turn into: - -```lua -local dogName = if dog == nil then nil else dog.name -``` - -...but this is fairly clunky for such a common expression. - -## Design - -The safe navigation operator will make all of these smooth, by supporting `x?.y` to safely index nil values. `dog?.name` would resolve to `nil` if `dog` was nil, or the name otherwise. - -The previous example turns into `local dogName = dog?.name` (or just using `dog?.name` elsewhere). - -Failing the nil-safety check early would make the entire expression nil, for instance `dog?.body.legs` would resolve to `nil` if `dog` is nil, rather than resolve `dog?.body` into nil, then turning into `nil.legs`. - -```lua -dog?.name --[[ is the same as ]] if dog == nil then nil else dog.name -``` - -The short-circuiting is limited within the expression. - -```lua -dog?.owner.name -- This will return nil if `dog` is nil -(dog?.owner).name -- `(dog?.owner)` resolves to nil, of which `name` is then indexed. This will error at runtime if `dog` is nil. - -dog?.legs + 3 -- `dog?.legs` is resolved on its own, meaning this will error at runtime if it is nil (`nil + 3`) -``` - -The operator must be used in the context of either a call or an index, and so: - -```lua -local value = x? -``` - -...would be invalid syntax. - -This syntax would be based on expressions, and not identifiers, meaning that `(x or y)?.call()` would be valid syntax. - -### Type -If the expression is typed as an optional, then the resulting type would be the final expression, also optional. Otherwise, it'll just be the resulting type if `?` wasn't used. - -```lua -local optionalObject: { name: string }? -local optionalObjectName = optionalObject?.name -- resolves to `string?` - -local nonOptionalObject: { name: string } -local nonOptionalObjectName = nonOptionalObject?.name -- resolves to `string` -``` - -### Calling - -This RFC only specifies `x?.y` as an index method. `x?:y()` is currently unspecified, and `x?.y(args)` as a syntax will be reserved (will error if you try to use it). - -While being able to support `dog?.getName()` is useful, it provides [some logistical issues for the language](https://github.com/Roblox/luau/pull/142#issuecomment-990563536). - -`x?.y(args)` will be reserved both so that this can potentially be resolved later down the line if something comes up, but also because it would be a guaranteed runtime error under this RFC: `dog?.getName()` will first index `dog?.getName`, which will return nil, then will attempt to call it. - -### Assignment -`x?.y = z` is not supported, and will be reported as a syntax error. - -## Drawbacks - -As with all syntax additions, this adds complexity to the parsing of expressions, and the execution of cancelling the rest of the expression could prove challenging. - -Furthermore, with the proposed syntax, it might lock off other uses of `?` within code (and not types) for the future as being ambiguous. - -## Alternatives - -Doing nothing is an option, as current standard if-checks already work, as well as the `and` trick in other use cases, but as shown before this can create some hard to read code, and nil values are common enough that the safe navigation operator is welcome. - -Supporting optional calls/indexes, such as `x?[1]` and `x?()`, while not out of scope, are likely too fringe to support, while adding on a significant amount of parsing difficulty, especially in the case of shorthand function calls, such as `x?{}` and `x?""`. - -It is possible to make `x?.y = z` resolve to only setting `x.y` if `x` is nil, but assignments silently failing can be seen as surprising. From bcab792e0d2eb91fc0618c733e8c66333293b813 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Thu, 9 Jun 2022 10:18:03 -0700 Subject: [PATCH 268/399] Update STATUS.md Generalized iteration got implemented, safe indexing got dropped. --- rfcs/STATUS.md | 14 -------------- 1 file changed, 14 deletions(-) diff --git a/rfcs/STATUS.md b/rfcs/STATUS.md index ef55b5c4..d2fe86f0 100644 --- a/rfcs/STATUS.md +++ b/rfcs/STATUS.md @@ -15,26 +15,12 @@ This document tracks unimplemented RFCs. **Status**: Needs implementation -## Safe navigation operator - -[RFC: Safe navigation postfix operator (?)](https://github.com/Roblox/luau/blob/master/rfcs/syntax-safe-navigation-operator.md) - -**Status**: Needs implementation. - -**Notes**: We have unresolved issues with interaction between this feature and Roblox instance hierarchy. This may affect the viability of this proposal. - ## String interpolation [RFC: String interpolation](https://github.com/Roblox/luau/blob/master/rfcs/syntax-string-interpolation.md) **Status**: Needs implementation -## Generalized iteration - -[RFC: Generalized iteration](https://github.com/Roblox/luau/blob/master/rfcs/generalized-iteration.md) - -**Status**: Implemented but not fully rolled out yet. - ## Lower Bounds Calculation [RFC: Lower bounds calculation](https://github.com/Roblox/luau/blob/master/rfcs/lower-bounds-calculation.md) From ca5fbbfc2467353e39799927425584c08a45b3d8 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Thu, 9 Jun 2022 10:18:29 -0700 Subject: [PATCH 269/399] Mark generalized iteration RFC as implemented --- rfcs/generalized-iteration.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/rfcs/generalized-iteration.md b/rfcs/generalized-iteration.md index 72bdd69e..99671090 100644 --- a/rfcs/generalized-iteration.md +++ b/rfcs/generalized-iteration.md @@ -1,5 +1,7 @@ # Generalized iteration +**Status**: Implemented + ## Summary Introduce support for iterating over tables without using `pairs`/`ipairs` as well as a generic customization point for iteration via `__iter` metamethod. From 7df94088512a660cad2fb2dcce443d72a72b33ef Mon Sep 17 00:00:00 2001 From: Rodactor Date: Thu, 9 Jun 2022 18:31:12 -0700 Subject: [PATCH 270/399] Sync to origin/release/531 --- Analysis/include/Luau/TypePack.h | 13 +- Analysis/include/Luau/TypeVar.h | 11 +- Analysis/include/Luau/Variant.h | 1 + Analysis/src/Autocomplete.cpp | 41 ++-- Analysis/src/BuiltinDefinitions.cpp | 36 +--- Analysis/src/Clone.cpp | 5 - Analysis/src/EmbeddedBuiltinDefinitions.cpp | 8 +- Analysis/src/Instantiation.cpp | 4 - Analysis/src/Quantify.cpp | 35 ---- Analysis/src/Scope.cpp | 5 +- Analysis/src/Substitution.cpp | 1 - Analysis/src/TxnLog.cpp | 56 +++++- Analysis/src/TypeInfer.cpp | 113 ++++++------ Analysis/src/TypePack.cpp | 21 +++ Analysis/src/TypeVar.cpp | 21 +++ Ast/src/Parser.cpp | 22 ++- Compiler/src/BytecodeBuilder.cpp | 10 +- Compiler/src/Compiler.cpp | 195 +++++--------------- VM/src/lapi.cpp | 106 +++++------ VM/src/lbuiltins.cpp | 12 +- VM/src/ldo.cpp | 25 ++- VM/src/lvmexecute.cpp | 19 +- tests/Autocomplete.test.cpp | 37 ++-- tests/Compiler.test.cpp | 13 -- tests/Conformance.test.cpp | 59 +++++- tests/Module.test.cpp | 20 ++ tests/Parser.test.cpp | 17 ++ tests/TypeInfer.aliases.test.cpp | 6 - tests/TypeInfer.loops.test.cpp | 8 - tests/TypeInfer.refinements.test.cpp | 32 +++- tests/TypeInfer.singletons.test.cpp | 2 - tests/TypePack.test.cpp | 16 ++ tests/TypeVar.test.cpp | 20 ++ tests/conformance/apicalls.lua | 11 ++ tests/conformance/pcall.lua | 17 ++ 35 files changed, 544 insertions(+), 474 deletions(-) diff --git a/Analysis/include/Luau/TypePack.h b/Analysis/include/Luau/TypePack.h index bbc65f94..c1de242f 100644 --- a/Analysis/include/Luau/TypePack.h +++ b/Analysis/include/Luau/TypePack.h @@ -48,13 +48,24 @@ struct TypePackVar explicit TypePackVar(const TypePackVariant& ty); explicit TypePackVar(TypePackVariant&& ty); TypePackVar(TypePackVariant&& ty, bool persistent); + bool operator==(const TypePackVar& rhs) const; + TypePackVar& operator=(TypePackVariant&& tp); + TypePackVar& operator=(const TypePackVar& rhs); + + // Re-assignes the content of the pack, but doesn't change the owning arena and can't make pack persistent. + void reassign(const TypePackVar& rhs) + { + ty = rhs.ty; + } + TypePackVariant ty; + bool persistent = false; - // Pointer to the type arena that allocated this type. + // Pointer to the type arena that allocated this pack. TypeArena* owningArena = nullptr; }; diff --git a/Analysis/include/Luau/TypeVar.h b/Analysis/include/Luau/TypeVar.h index b3c455cf..b59e7c64 100644 --- a/Analysis/include/Luau/TypeVar.h +++ b/Analysis/include/Luau/TypeVar.h @@ -334,7 +334,6 @@ struct TableTypeVar // We need to know which is which when we stringify types. std::optional syntheticName; - std::map methodDefinitionLocations; // TODO: Remove with FFlag::LuauNoMethodLocations std::vector instantiatedTypeParams; std::vector instantiatedTypePackParams; ModuleName definitionModuleName; @@ -465,6 +464,14 @@ struct TypeVar final { } + // Re-assignes the content of the type, but doesn't change the owning arena and can't make type persistent. + void reassign(const TypeVar& rhs) + { + ty = rhs.ty; + normal = rhs.normal; + documentationSymbol = rhs.documentationSymbol; + } + TypeVariant ty; // Kludge: A persistent TypeVar is one that belongs to the global scope. @@ -486,6 +493,8 @@ struct TypeVar final TypeVar& operator=(const TypeVariant& rhs); TypeVar& operator=(TypeVariant&& rhs); + + TypeVar& operator=(const TypeVar& rhs); }; using SeenSet = std::set>; diff --git a/Analysis/include/Luau/Variant.h b/Analysis/include/Luau/Variant.h index c9c97c92..f637222e 100644 --- a/Analysis/include/Luau/Variant.h +++ b/Analysis/include/Luau/Variant.h @@ -6,6 +6,7 @@ #include #include #include +#include namespace Luau { diff --git a/Analysis/src/Autocomplete.cpp b/Analysis/src/Autocomplete.cpp index b988ed35..a8319c59 100644 --- a/Analysis/src/Autocomplete.cpp +++ b/Analysis/src/Autocomplete.cpp @@ -14,8 +14,7 @@ #include LUAU_FASTFLAGVARIABLE(LuauIfElseExprFixCompletionIssue, false); -LUAU_FASTFLAGVARIABLE(LuauFixAutocompleteClassSecurityLevel, false); -LUAU_FASTFLAG(LuauSelfCallAutocompleteFix) +LUAU_FASTFLAG(LuauSelfCallAutocompleteFix2) static const std::unordered_set kStatementStartingKeywords = { "while", "if", "local", "repeat", "function", "do", "for", "return", "break", "continue", "type", "export"}; @@ -248,7 +247,7 @@ static TypeCorrectKind checkTypeCorrectKind(const Module& module, TypeArena* typ ty = follow(ty); auto canUnify = [&typeArena](TypeId subTy, TypeId superTy) { - LUAU_ASSERT(!FFlag::LuauSelfCallAutocompleteFix); + LUAU_ASSERT(!FFlag::LuauSelfCallAutocompleteFix2); InternalErrorReporter iceReporter; UnifierSharedState unifierState(&iceReporter); @@ -267,7 +266,7 @@ static TypeCorrectKind checkTypeCorrectKind(const Module& module, TypeArena* typ TypeId expectedType = follow(*typeAtPosition); auto checkFunctionType = [typeArena, &canUnify, &expectedType](const FunctionTypeVar* ftv) { - if (FFlag::LuauSelfCallAutocompleteFix) + if (FFlag::LuauSelfCallAutocompleteFix2) { if (std::optional firstRetTy = first(ftv->retType)) return checkTypeMatch(typeArena, *firstRetTy, expectedType); @@ -308,7 +307,7 @@ static TypeCorrectKind checkTypeCorrectKind(const Module& module, TypeArena* typ } } - if (FFlag::LuauSelfCallAutocompleteFix) + if (FFlag::LuauSelfCallAutocompleteFix2) return checkTypeMatch(typeArena, ty, expectedType) ? TypeCorrectKind::Correct : TypeCorrectKind::None; else return canUnify(ty, expectedType) ? TypeCorrectKind::Correct : TypeCorrectKind::None; @@ -325,7 +324,7 @@ static void autocompleteProps(const Module& module, TypeArena* typeArena, TypeId const std::vector& nodes, AutocompleteEntryMap& result, std::unordered_set& seen, std::optional containingClass = std::nullopt) { - if (FFlag::LuauSelfCallAutocompleteFix) + if (FFlag::LuauSelfCallAutocompleteFix2) rootTy = follow(rootTy); ty = follow(ty); @@ -335,7 +334,7 @@ static void autocompleteProps(const Module& module, TypeArena* typeArena, TypeId seen.insert(ty); auto isWrongIndexer_DEPRECATED = [indexType, useStrictFunctionIndexers = !!get(ty)](Luau::TypeId type) { - LUAU_ASSERT(!FFlag::LuauSelfCallAutocompleteFix); + LUAU_ASSERT(!FFlag::LuauSelfCallAutocompleteFix2); if (indexType == PropIndexType::Key) return false; @@ -368,7 +367,7 @@ static void autocompleteProps(const Module& module, TypeArena* typeArena, TypeId } }; auto isWrongIndexer = [typeArena, rootTy, indexType](Luau::TypeId type) { - LUAU_ASSERT(FFlag::LuauSelfCallAutocompleteFix); + LUAU_ASSERT(FFlag::LuauSelfCallAutocompleteFix2); if (indexType == PropIndexType::Key) return false; @@ -382,10 +381,15 @@ static void autocompleteProps(const Module& module, TypeArena* typeArena, TypeId return calledWithSelf == ftv->hasSelf; } - if (std::optional firstArgTy = first(ftv->argTypes)) + // If a call is made with ':', it is invalid if a function has incompatible first argument or no arguments at all + // If a call is made with '.', but it was declared with 'self', it is considered invalid if first argument is compatible + if (calledWithSelf || ftv->hasSelf) { - if (checkTypeMatch(typeArena, rootTy, *firstArgTy)) - return calledWithSelf; + if (std::optional firstArgTy = first(ftv->argTypes)) + { + if (checkTypeMatch(typeArena, rootTy, *firstArgTy)) + return calledWithSelf; + } } return !calledWithSelf; @@ -427,7 +431,7 @@ static void autocompleteProps(const Module& module, TypeArena* typeArena, TypeId AutocompleteEntryKind::Property, type, prop.deprecated, - FFlag::LuauSelfCallAutocompleteFix ? isWrongIndexer(type) : isWrongIndexer_DEPRECATED(type), + FFlag::LuauSelfCallAutocompleteFix2 ? isWrongIndexer(type) : isWrongIndexer_DEPRECATED(type), typeCorrect, containingClass, &prop, @@ -462,8 +466,7 @@ static void autocompleteProps(const Module& module, TypeArena* typeArena, TypeId containingClass = containingClass.value_or(cls); fillProps(cls->props); if (cls->parent) - autocompleteProps(module, typeArena, rootTy, *cls->parent, indexType, nodes, result, seen, - FFlag::LuauFixAutocompleteClassSecurityLevel ? containingClass : cls); + autocompleteProps(module, typeArena, rootTy, *cls->parent, indexType, nodes, result, seen, containingClass); } else if (auto tbl = get(ty)) fillProps(tbl->props); @@ -471,7 +474,7 @@ static void autocompleteProps(const Module& module, TypeArena* typeArena, TypeId { autocompleteProps(module, typeArena, rootTy, mt->table, indexType, nodes, result, seen); - if (FFlag::LuauSelfCallAutocompleteFix) + if (FFlag::LuauSelfCallAutocompleteFix2) { if (auto mtable = get(mt->metatable)) fillMetatableProps(mtable); @@ -537,7 +540,7 @@ static void autocompleteProps(const Module& module, TypeArena* typeArena, TypeId AutocompleteEntryMap inner; std::unordered_set innerSeen; - if (!FFlag::LuauSelfCallAutocompleteFix) + if (!FFlag::LuauSelfCallAutocompleteFix2) innerSeen = seen; if (isNil(*iter)) @@ -563,7 +566,7 @@ static void autocompleteProps(const Module& module, TypeArena* typeArena, TypeId ++iter; } } - else if (auto pt = get(ty); pt && FFlag::LuauSelfCallAutocompleteFix) + else if (auto pt = get(ty); pt && FFlag::LuauSelfCallAutocompleteFix2) { if (pt->metatable) { @@ -571,7 +574,7 @@ static void autocompleteProps(const Module& module, TypeArena* typeArena, TypeId fillMetatableProps(mtable); } } - else if (FFlag::LuauSelfCallAutocompleteFix && get(get(ty))) + else if (FFlag::LuauSelfCallAutocompleteFix2 && get(get(ty))) { autocompleteProps(module, typeArena, rootTy, getSingletonTypes().stringType, indexType, nodes, result, seen); } @@ -1501,7 +1504,7 @@ static AutocompleteResult autocomplete(const SourceModule& sourceModule, const M TypeId ty = follow(*it); PropIndexType indexType = indexName->op == ':' ? PropIndexType::Colon : PropIndexType::Point; - if (!FFlag::LuauSelfCallAutocompleteFix && isString(ty)) + if (!FFlag::LuauSelfCallAutocompleteFix2 && isString(ty)) return {autocompleteProps(*module, typeArena, typeChecker.globalScope->bindings[AstName{"string"}].typeId, indexType, finder.ancestry), finder.ancestry}; else diff --git a/Analysis/src/BuiltinDefinitions.cpp b/Analysis/src/BuiltinDefinitions.cpp index 5ed6de67..98737b43 100644 --- a/Analysis/src/BuiltinDefinitions.cpp +++ b/Analysis/src/BuiltinDefinitions.cpp @@ -179,44 +179,13 @@ void registerBuiltinTypes(TypeChecker& typeChecker) LUAU_ASSERT(!typeChecker.globalTypes.typeVars.isFrozen()); LUAU_ASSERT(!typeChecker.globalTypes.typePacks.isFrozen()); - TypeId numberType = typeChecker.numberType; - TypeId booleanType = typeChecker.booleanType; TypeId nilType = typeChecker.nilType; TypeArena& arena = typeChecker.globalTypes; - TypePackId oneNumberPack = arena.addTypePack({numberType}); - TypePackId oneBooleanPack = arena.addTypePack({booleanType}); - - TypePackId numberVariadicList = arena.addTypePack(TypePackVar{VariadicTypePack{numberType}}); - TypePackId listOfAtLeastOneNumber = arena.addTypePack(TypePack{{numberType}, numberVariadicList}); - - TypeId listOfAtLeastOneNumberToNumberType = arena.addType(FunctionTypeVar{ - listOfAtLeastOneNumber, - oneNumberPack, - }); - - TypeId listOfAtLeastZeroNumbersToNumberType = arena.addType(FunctionTypeVar{numberVariadicList, oneNumberPack}); - LoadDefinitionFileResult loadResult = Luau::loadDefinitionFile(typeChecker, typeChecker.globalScope, getBuiltinDefinitionSource(), "@luau"); LUAU_ASSERT(loadResult.success); - TypeId mathLibType = getGlobalBinding(typeChecker, "math"); - if (TableTypeVar* ttv = getMutable(mathLibType)) - { - ttv->props["min"] = makeProperty(listOfAtLeastOneNumberToNumberType, "@luau/global/math.min"); - ttv->props["max"] = makeProperty(listOfAtLeastOneNumberToNumberType, "@luau/global/math.max"); - } - - TypeId bit32LibType = getGlobalBinding(typeChecker, "bit32"); - if (TableTypeVar* ttv = getMutable(bit32LibType)) - { - ttv->props["band"] = makeProperty(listOfAtLeastZeroNumbersToNumberType, "@luau/global/bit32.band"); - ttv->props["bor"] = makeProperty(listOfAtLeastZeroNumbersToNumberType, "@luau/global/bit32.bor"); - ttv->props["bxor"] = makeProperty(listOfAtLeastZeroNumbersToNumberType, "@luau/global/bit32.bxor"); - ttv->props["btest"] = makeProperty(arena.addType(FunctionTypeVar{listOfAtLeastOneNumber, oneBooleanPack}), "@luau/global/bit32.btest"); - } - TypeId genericK = arena.addType(GenericTypeVar{"K"}); TypeId genericV = arena.addType(GenericTypeVar{"V"}); TypeId mapOfKtoV = arena.addType(TableTypeVar{{}, TableIndexer(genericK, genericV), typeChecker.globalScope->level, TableState::Generic}); @@ -231,7 +200,7 @@ void registerBuiltinTypes(TypeChecker& typeChecker) addGlobalBinding(typeChecker, "string", it->second.type, "@luau"); - // next(t: Table, i: K | nil) -> (K, V) + // next(t: Table, i: K?) -> (K, V) TypePackId nextArgsTypePack = arena.addTypePack(TypePack{{mapOfKtoV, makeOption(typeChecker, arena, genericK)}}); addGlobalBinding(typeChecker, "next", arena.addType(FunctionTypeVar{{genericK, genericV}, {}, nextArgsTypePack, arena.addTypePack(TypePack{{genericK, genericV}})}), "@luau"); @@ -241,8 +210,7 @@ void registerBuiltinTypes(TypeChecker& typeChecker) TypeId pairsNext = arena.addType(FunctionTypeVar{nextArgsTypePack, arena.addTypePack(TypePack{{genericK, genericV}})}); TypePackId pairsReturnTypePack = arena.addTypePack(TypePack{{pairsNext, mapOfKtoV, nilType}}); - // NOTE we are missing 'i: K | nil' argument in the first return types' argument. - // pairs(t: Table) -> ((Table) -> (K, V), Table, nil) + // pairs(t: Table) -> ((Table, K?) -> (K, V), Table, nil) addGlobalBinding(typeChecker, "pairs", arena.addType(FunctionTypeVar{{genericK, genericV}, {}, pairsArgsTypePack, pairsReturnTypePack}), "@luau"); TypeId genericMT = arena.addType(GenericTypeVar{"MT"}); diff --git a/Analysis/src/Clone.cpp b/Analysis/src/Clone.cpp index 19e3383e..9180f309 100644 --- a/Analysis/src/Clone.cpp +++ b/Analysis/src/Clone.cpp @@ -9,7 +9,6 @@ LUAU_FASTFLAG(DebugLuauCopyBeforeNormalizing) LUAU_FASTINTVARIABLE(LuauTypeCloneRecursionLimit, 300) -LUAU_FASTFLAG(LuauNoMethodLocations) namespace Luau { @@ -241,8 +240,6 @@ void TypeCloner::operator()(const TableTypeVar& t) arg = clone(arg, dest, cloneState); ttv->definitionModuleName = t.definitionModuleName; - if (!FFlag::LuauNoMethodLocations) - ttv->methodDefinitionLocations = t.methodDefinitionLocations; ttv->tags = t.tags; } @@ -406,8 +403,6 @@ TypeId shallowClone(TypeId ty, TypeArena& dest, const TxnLog* log) { LUAU_ASSERT(!ttv->boundTo); TableTypeVar clone = TableTypeVar{ttv->props, ttv->indexer, ttv->level, ttv->state}; - if (!FFlag::LuauNoMethodLocations) - clone.methodDefinitionLocations = ttv->methodDefinitionLocations; clone.definitionModuleName = ttv->definitionModuleName; clone.name = ttv->name; clone.syntheticName = ttv->syntheticName; diff --git a/Analysis/src/EmbeddedBuiltinDefinitions.cpp b/Analysis/src/EmbeddedBuiltinDefinitions.cpp index f184b74e..2407e3ef 100644 --- a/Analysis/src/EmbeddedBuiltinDefinitions.cpp +++ b/Analysis/src/EmbeddedBuiltinDefinitions.cpp @@ -7,7 +7,10 @@ namespace Luau static const std::string kBuiltinDefinitionLuaSrc = R"BUILTIN_SRC( declare bit32: { - -- band, bor, bxor, and btest are declared in C++ + band: (...number) -> number, + bor: (...number) -> number, + bxor: (...number) -> number, + btest: (number, ...number) -> boolean, rrotate: (number, number) -> number, lrotate: (number, number) -> number, lshift: (number, number) -> number, @@ -50,7 +53,8 @@ declare math: { asin: (number) -> number, atan2: (number, number) -> number, - -- min and max are declared in C++. + min: (number, ...number) -> number, + max: (number, ...number) -> number, pi: number, huge: number, diff --git a/Analysis/src/Instantiation.cpp b/Analysis/src/Instantiation.cpp index 4a12027d..f145a511 100644 --- a/Analysis/src/Instantiation.cpp +++ b/Analysis/src/Instantiation.cpp @@ -4,8 +4,6 @@ #include "Luau/TxnLog.h" #include "Luau/TypeArena.h" -LUAU_FASTFLAG(LuauNoMethodLocations) - namespace Luau { @@ -110,8 +108,6 @@ TypeId ReplaceGenerics::clean(TypeId ty) if (const TableTypeVar* ttv = log->getMutable(ty)) { TableTypeVar clone = TableTypeVar{ttv->props, ttv->indexer, level, TableState::Free}; - if (!FFlag::LuauNoMethodLocations) - clone.methodDefinitionLocations = ttv->methodDefinitionLocations; clone.definitionModuleName = ttv->definitionModuleName; return addType(std::move(clone)); } diff --git a/Analysis/src/Quantify.cpp b/Analysis/src/Quantify.cpp index 8f2cc8e3..21775373 100644 --- a/Analysis/src/Quantify.cpp +++ b/Analysis/src/Quantify.cpp @@ -32,41 +32,6 @@ struct Quantifier final : TypeVarOnceVisitor LUAU_ASSERT(FFlag::DebugLuauDeferredConstraintResolution); } - void cycle(TypeId) override {} - void cycle(TypePackId) override {} - - bool operator()(TypeId ty, const FreeTypeVar& ftv) - { - return visit(ty, ftv); - } - - template - bool operator()(TypeId ty, const T& t) - { - return true; - } - - template - bool operator()(TypePackId, const T&) - { - return true; - } - - bool operator()(TypeId ty, const ConstrainedTypeVar&) - { - return true; - } - - bool operator()(TypeId ty, const TableTypeVar& ttv) - { - return visit(ty, ttv); - } - - bool operator()(TypePackId tp, const FreeTypePack& ftp) - { - return visit(tp, ftp); - } - /// @return true if outer encloses inner bool subsumes(Scope2* outer, Scope2* inner) { diff --git a/Analysis/src/Scope.cpp b/Analysis/src/Scope.cpp index 0a362a5e..011e28d4 100644 --- a/Analysis/src/Scope.cpp +++ b/Analysis/src/Scope.cpp @@ -2,8 +2,6 @@ #include "Luau/Scope.h" -LUAU_FASTFLAG(LuauTwoPassAliasDefinitionFix); - namespace Luau { @@ -19,8 +17,7 @@ Scope::Scope(const ScopePtr& parent, int subLevel) , returnType(parent->returnType) , level(parent->level.incr()) { - if (FFlag::LuauTwoPassAliasDefinitionFix) - level = level.incr(); + level = level.incr(); level.subLevel = subLevel; } diff --git a/Analysis/src/Substitution.cpp b/Analysis/src/Substitution.cpp index 50c516db..5a22deeb 100644 --- a/Analysis/src/Substitution.cpp +++ b/Analysis/src/Substitution.cpp @@ -10,7 +10,6 @@ LUAU_FASTFLAG(LuauLowerBoundsCalculation) LUAU_FASTINTVARIABLE(LuauTarjanChildLimit, 10000) -LUAU_FASTFLAG(LuauNoMethodLocations) namespace Luau { diff --git a/Analysis/src/TxnLog.cpp b/Analysis/src/TxnLog.cpp index e45c0cbd..4c6d54e0 100644 --- a/Analysis/src/TxnLog.cpp +++ b/Analysis/src/TxnLog.cpp @@ -7,6 +7,8 @@ #include #include +LUAU_FASTFLAG(LuauNonCopyableTypeVarFields) + namespace Luau { @@ -80,18 +82,32 @@ void TxnLog::commit() { for (auto& [ty, rep] : typeVarChanges) { - TypeArena* owningArena = ty->owningArena; - TypeVar* mtv = asMutable(ty); - *mtv = rep.get()->pending; - mtv->owningArena = owningArena; + if (FFlag::LuauNonCopyableTypeVarFields) + { + asMutable(ty)->reassign(rep.get()->pending); + } + else + { + TypeArena* owningArena = ty->owningArena; + TypeVar* mtv = asMutable(ty); + *mtv = rep.get()->pending; + mtv->owningArena = owningArena; + } } for (auto& [tp, rep] : typePackChanges) { - TypeArena* owningArena = tp->owningArena; - TypePackVar* mpv = asMutable(tp); - *mpv = rep.get()->pending; - mpv->owningArena = owningArena; + if (FFlag::LuauNonCopyableTypeVarFields) + { + asMutable(tp)->reassign(rep.get()->pending); + } + else + { + TypeArena* owningArena = tp->owningArena; + TypePackVar* mpv = asMutable(tp); + *mpv = rep.get()->pending; + mpv->owningArena = owningArena; + } } clear(); @@ -178,8 +194,13 @@ PendingType* TxnLog::queue(TypeId ty) // about this type, we don't want to mutate the parent's state. auto& pending = typeVarChanges[ty]; if (!pending) + { pending = std::make_unique(*ty); + if (FFlag::LuauNonCopyableTypeVarFields) + pending->pending.owningArena = nullptr; + } + return pending.get(); } @@ -191,8 +212,13 @@ PendingTypePack* TxnLog::queue(TypePackId tp) // about this type, we don't want to mutate the parent's state. auto& pending = typePackChanges[tp]; if (!pending) + { pending = std::make_unique(*tp); + if (FFlag::LuauNonCopyableTypeVarFields) + pending->pending.owningArena = nullptr; + } + return pending.get(); } @@ -229,14 +255,24 @@ PendingTypePack* TxnLog::pending(TypePackId tp) const PendingType* TxnLog::replace(TypeId ty, TypeVar replacement) { PendingType* newTy = queue(ty); - newTy->pending = replacement; + + if (FFlag::LuauNonCopyableTypeVarFields) + newTy->pending.reassign(replacement); + else + newTy->pending = replacement; + return newTy; } PendingTypePack* TxnLog::replace(TypePackId tp, TypePackVar replacement) { PendingTypePack* newTp = queue(tp); - newTp->pending = replacement; + + if (FFlag::LuauNonCopyableTypeVarFields) + newTp->pending.reassign(replacement); + else + newTp->pending = replacement; + return newTp; } diff --git a/Analysis/src/TypeInfer.cpp b/Analysis/src/TypeInfer.cpp index 4931bc59..447cd029 100644 --- a/Analysis/src/TypeInfer.cpp +++ b/Analysis/src/TypeInfer.cpp @@ -33,21 +33,20 @@ LUAU_FASTFLAG(LuauAutocompleteDynamicLimits) LUAU_FASTFLAGVARIABLE(LuauExpectedPropTypeFromIndexer, false) LUAU_FASTFLAGVARIABLE(LuauLowerBoundsCalculation, false) LUAU_FASTFLAGVARIABLE(DebugLuauFreezeDuringUnification, false) -LUAU_FASTFLAGVARIABLE(LuauSelfCallAutocompleteFix, false) +LUAU_FASTFLAGVARIABLE(LuauSelfCallAutocompleteFix2, false) LUAU_FASTFLAGVARIABLE(LuauReduceUnionRecursion, false) LUAU_FASTFLAGVARIABLE(LuauOnlyMutateInstantiatedTables, false) LUAU_FASTFLAGVARIABLE(LuauUnsealedTableLiteral, false) -LUAU_FASTFLAGVARIABLE(LuauTwoPassAliasDefinitionFix, false) LUAU_FASTFLAGVARIABLE(LuauReturnAnyInsteadOfICE, false) // Eventually removed as false. LUAU_FASTFLAG(LuauNormalizeFlagIsConservative) LUAU_FASTFLAGVARIABLE(LuauReturnTypeInferenceInNonstrict, false) LUAU_FASTFLAGVARIABLE(LuauRecursionLimitException, false); LUAU_FASTFLAGVARIABLE(LuauApplyTypeFunctionFix, false); -LUAU_FASTFLAGVARIABLE(LuauTypecheckIter, false); LUAU_FASTFLAGVARIABLE(LuauSuccessTypingForEqualityOperations, false) -LUAU_FASTFLAGVARIABLE(LuauNoMethodLocations, false); LUAU_FASTFLAGVARIABLE(LuauAlwaysQuantify, false); LUAU_FASTFLAGVARIABLE(LuauReportErrorsOnIndexerKeyMismatch, false) +LUAU_FASTFLAGVARIABLE(LuauFalsyPredicateReturnsNilInstead, false) +LUAU_FASTFLAGVARIABLE(LuauNonCopyableTypeVarFields, false) namespace Luau { @@ -358,8 +357,7 @@ ModulePtr TypeChecker::checkWithoutRecursionCheck(const SourceModule& module, Mo unifierState.cachedUnifyError.clear(); unifierState.skipCacheForType.clear(); - if (FFlag::LuauTwoPassAliasDefinitionFix) - duplicateTypeAliases.clear(); + duplicateTypeAliases.clear(); return std::move(currentModule); } @@ -610,7 +608,7 @@ LUAU_NOINLINE void TypeChecker::checkBlockTypeAliases(const ScopePtr& scope, std { if (const auto& typealias = stat->as()) { - if (FFlag::LuauTwoPassAliasDefinitionFix && typealias->name == kParseNameError) + if (typealias->name == kParseNameError) continue; auto& bindings = typealias->exported ? scope->exportedTypeBindings : scope->privateTypeBindings; @@ -619,7 +617,16 @@ LUAU_NOINLINE void TypeChecker::checkBlockTypeAliases(const ScopePtr& scope, std TypeId type = bindings[name].type; if (get(follow(type))) { - *asMutable(type) = *errorRecoveryType(anyType); + if (FFlag::LuauNonCopyableTypeVarFields) + { + TypeVar* mty = asMutable(follow(type)); + mty->reassign(*errorRecoveryType(anyType)); + } + else + { + *asMutable(type) = *errorRecoveryType(anyType); + } + reportError(TypeError{typealias->location, OccursCheckFailed{}}); } } @@ -1131,45 +1138,43 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatForIn& forin) iterTy = instantiate(scope, checkExpr(scope, *firstValue).type, firstValue->location); } - if (FFlag::LuauTypecheckIter) + if (std::optional iterMM = findMetatableEntry(iterTy, "__iter", firstValue->location)) { - if (std::optional iterMM = findMetatableEntry(iterTy, "__iter", firstValue->location)) + // if __iter metamethod is present, it will be called and the results are going to be called as if they are functions + // TODO: this needs to typecheck all returned values by __iter as if they were for loop arguments + // the structure of the function makes it difficult to do this especially since we don't have actual expressions, only types + for (TypeId var : varTypes) + unify(anyType, var, forin.location); + + return check(loopScope, *forin.body); + } + + if (const TableTypeVar* iterTable = get(iterTy)) + { + // TODO: note that this doesn't cleanly handle iteration over mixed tables and tables without an indexer + // this behavior is more or less consistent with what we do for pairs(), but really both are pretty wrong and need revisiting + if (iterTable->indexer) { - // if __iter metamethod is present, it will be called and the results are going to be called as if they are functions - // TODO: this needs to typecheck all returned values by __iter as if they were for loop arguments - // the structure of the function makes it difficult to do this especially since we don't have actual expressions, only types + if (varTypes.size() > 0) + unify(iterTable->indexer->indexType, varTypes[0], forin.location); + + if (varTypes.size() > 1) + unify(iterTable->indexer->indexResultType, varTypes[1], forin.location); + + for (size_t i = 2; i < varTypes.size(); ++i) + unify(nilType, varTypes[i], forin.location); + } + else + { + TypeId varTy = errorRecoveryType(loopScope); + for (TypeId var : varTypes) - unify(anyType, var, forin.location); + unify(varTy, var, forin.location); - return check(loopScope, *forin.body); + reportError(firstValue->location, GenericError{"Cannot iterate over a table without indexer"}); } - else if (const TableTypeVar* iterTable = get(iterTy)) - { - // TODO: note that this doesn't cleanly handle iteration over mixed tables and tables without an indexer - // this behavior is more or less consistent with what we do for pairs(), but really both are pretty wrong and need revisiting - if (iterTable->indexer) - { - if (varTypes.size() > 0) - unify(iterTable->indexer->indexType, varTypes[0], forin.location); - if (varTypes.size() > 1) - unify(iterTable->indexer->indexResultType, varTypes[1], forin.location); - - for (size_t i = 2; i < varTypes.size(); ++i) - unify(nilType, varTypes[i], forin.location); - } - else - { - TypeId varTy = errorRecoveryType(loopScope); - - for (TypeId var : varTypes) - unify(varTy, var, forin.location); - - reportError(firstValue->location, GenericError{"Cannot iterate over a table without indexer"}); - } - - return check(loopScope, *forin.body); - } + return check(loopScope, *forin.body); } const FunctionTypeVar* iterFunc = get(iterTy); @@ -1334,7 +1339,7 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatTypeAlias& typealias Name name = typealias.name.value; // If the alias is missing a name, we can't do anything with it. Ignore it. - if (FFlag::LuauTwoPassAliasDefinitionFix && name == kParseNameError) + if (name == kParseNameError) return; std::optional binding; @@ -1353,8 +1358,7 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatTypeAlias& typealias reportError(TypeError{typealias.location, DuplicateTypeDefinition{name, location}}); bindingsMap[name] = TypeFun{binding->typeParams, binding->typePackParams, errorRecoveryType(anyType)}; - if (FFlag::LuauTwoPassAliasDefinitionFix) - duplicateTypeAliases.insert({typealias.exported, name}); + duplicateTypeAliases.insert({typealias.exported, name}); } else { @@ -1378,7 +1382,7 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatTypeAlias& typealias { // If the first pass failed (this should mean a duplicate definition), the second pass isn't going to be // interesting. - if (FFlag::LuauTwoPassAliasDefinitionFix && duplicateTypeAliases.find({typealias.exported, name})) + if (duplicateTypeAliases.find({typealias.exported, name})) return; if (!binding) @@ -1422,9 +1426,6 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatTypeAlias& typealias { // This is a shallow clone, original recursive links to self are not updated TableTypeVar clone = TableTypeVar{ttv->props, ttv->indexer, ttv->level, ttv->state}; - - if (!FFlag::LuauNoMethodLocations) - clone.methodDefinitionLocations = ttv->methodDefinitionLocations; clone.definitionModuleName = ttv->definitionModuleName; clone.name = name; @@ -1462,9 +1463,8 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatTypeAlias& typealias } TypeId& bindingType = bindingsMap[name].type; - bool ok = unify(ty, bindingType, typealias.location); - if (FFlag::LuauTwoPassAliasDefinitionFix && ok) + if (unify(ty, bindingType, typealias.location)) bindingType = ty; if (FFlag::LuauLowerBoundsCalculation) @@ -1532,7 +1532,7 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatDeclareClass& declar ftv->argNames.insert(ftv->argNames.begin(), FunctionArgument{"self", {}}); ftv->argTypes = addTypePack(TypePack{{classTy}, ftv->argTypes}); - if (FFlag::LuauSelfCallAutocompleteFix) + if (FFlag::LuauSelfCallAutocompleteFix2) ftv->hasSelf = true; } } @@ -3099,8 +3099,6 @@ TypeId TypeChecker::checkFunctionName(const ScopePtr& scope, AstExpr& funName, T property.type = freshTy(); property.location = indexName->indexLocation; - if (!FFlag::LuauNoMethodLocations) - ttv->methodDefinitionLocations[name] = funName.location; return property.type; } else if (funName.is()) @@ -4393,8 +4391,6 @@ TypeId Anyification::clean(TypeId ty) if (const TableTypeVar* ttv = log->getMutable(ty)) { TableTypeVar clone = TableTypeVar{ttv->props, ttv->indexer, ttv->level, TableState::Sealed}; - if (!FFlag::LuauNoMethodLocations) - clone.methodDefinitionLocations = ttv->methodDefinitionLocations; clone.definitionModuleName = ttv->definitionModuleName; clone.name = ttv->name; clone.syntheticName = ttv->syntheticName; @@ -4705,8 +4701,11 @@ TypeIdPredicate TypeChecker::mkTruthyPredicate(bool sense) if (isNil(ty)) return sense ? std::nullopt : std::optional(ty); - // at this point, anything else is kept if sense is true, or eliminated otherwise - return sense ? std::optional(ty) : std::nullopt; + // at this point, anything else is kept if sense is true, or replaced by nil + if (FFlag::LuauFalsyPredicateReturnsNilInstead) + return sense ? ty : nilType; + else + return sense ? std::optional(ty) : std::nullopt; }; } diff --git a/Analysis/src/TypePack.cpp b/Analysis/src/TypePack.cpp index 30503233..82451bd1 100644 --- a/Analysis/src/TypePack.cpp +++ b/Analysis/src/TypePack.cpp @@ -5,6 +5,8 @@ #include +LUAU_FASTFLAG(LuauNonCopyableTypeVarFields) + namespace Luau { @@ -36,6 +38,25 @@ TypePackVar& TypePackVar::operator=(TypePackVariant&& tp) return *this; } +TypePackVar& TypePackVar::operator=(const TypePackVar& rhs) +{ + if (FFlag::LuauNonCopyableTypeVarFields) + { + LUAU_ASSERT(owningArena == rhs.owningArena); + LUAU_ASSERT(!rhs.persistent); + + reassign(rhs); + } + else + { + ty = rhs.ty; + persistent = rhs.persistent; + owningArena = rhs.owningArena; + } + + return *this; +} + TypePackIterator::TypePackIterator(TypePackId typePack) : TypePackIterator(typePack, TxnLog::empty()) { diff --git a/Analysis/src/TypeVar.cpp b/Analysis/src/TypeVar.cpp index 12cbed91..33bfe254 100644 --- a/Analysis/src/TypeVar.cpp +++ b/Analysis/src/TypeVar.cpp @@ -24,6 +24,7 @@ LUAU_FASTINTVARIABLE(LuauTypeMaximumStringifierLength, 500) LUAU_FASTINTVARIABLE(LuauTableTypeMaximumStringifierLength, 0) LUAU_FASTINT(LuauTypeInferRecursionLimit) LUAU_FASTFLAG(LuauSubtypingAddOptPropsToUnsealedTables) +LUAU_FASTFLAG(LuauNonCopyableTypeVarFields) namespace Luau { @@ -644,6 +645,26 @@ TypeVar& TypeVar::operator=(TypeVariant&& rhs) return *this; } +TypeVar& TypeVar::operator=(const TypeVar& rhs) +{ + if (FFlag::LuauNonCopyableTypeVarFields) + { + LUAU_ASSERT(owningArena == rhs.owningArena); + LUAU_ASSERT(!rhs.persistent); + + reassign(rhs); + } + else + { + ty = rhs.ty; + persistent = rhs.persistent; + normal = rhs.normal; + owningArena = rhs.owningArena; + } + + return *this; +} + TypeId makeFunction(TypeArena& arena, std::optional selfType, std::initializer_list generics, std::initializer_list genericPacks, std::initializer_list paramTypes, std::initializer_list paramNames, std::initializer_list retTypes); diff --git a/Ast/src/Parser.cpp b/Ast/src/Parser.cpp index eaf19914..95bce3ee 100644 --- a/Ast/src/Parser.cpp +++ b/Ast/src/Parser.cpp @@ -12,6 +12,7 @@ LUAU_FASTINTVARIABLE(LuauRecursionLimit, 1000) LUAU_FASTINTVARIABLE(LuauParseErrorLimit, 100) LUAU_FASTFLAGVARIABLE(LuauParserFunctionKeywordAsTypeHelp, false) +LUAU_FASTFLAGVARIABLE(LuauReturnTypeTokenConfusion, false) namespace Luau { @@ -1118,8 +1119,12 @@ AstTypePack* Parser::parseTypeList(TempVector& result, TempVector Parser::parseOptionalReturnTypeAnnotation() { - if (options.allowTypeAnnotations && lexer.current().type == ':') + if (options.allowTypeAnnotations && + (lexer.current().type == ':' || (FFlag::LuauReturnTypeTokenConfusion && lexer.current().type == Lexeme::SkinnyArrow))) { + if (FFlag::LuauReturnTypeTokenConfusion && lexer.current().type == Lexeme::SkinnyArrow) + report(lexer.current().location, "Function return type annotations are written after ':' instead of '->'"); + nextLexeme(); unsigned int oldRecursionCount = recursionCounter; @@ -1350,8 +1355,12 @@ AstTypeOrPack Parser::parseFunctionTypeAnnotation(bool allowPack) AstArray paramTypes = copy(params); + bool returnTypeIntroducer = + FFlag::LuauReturnTypeTokenConfusion ? lexer.current().type == Lexeme::SkinnyArrow || lexer.current().type == ':' : false; + // Not a function at all. Just a parenthesized type. Or maybe a type pack with a single element - if (params.size() == 1 && !varargAnnotation && monomorphic && lexer.current().type != Lexeme::SkinnyArrow) + if (params.size() == 1 && !varargAnnotation && monomorphic && + (FFlag::LuauReturnTypeTokenConfusion ? !returnTypeIntroducer : lexer.current().type != Lexeme::SkinnyArrow)) { if (allowPack) return {{}, allocator.alloc(begin.location, AstTypeList{paramTypes, nullptr})}; @@ -1359,7 +1368,7 @@ AstTypeOrPack Parser::parseFunctionTypeAnnotation(bool allowPack) return {params[0], {}}; } - if (lexer.current().type != Lexeme::SkinnyArrow && monomorphic && allowPack) + if ((FFlag::LuauReturnTypeTokenConfusion ? !returnTypeIntroducer : lexer.current().type != Lexeme::SkinnyArrow) && monomorphic && allowPack) return {{}, allocator.alloc(begin.location, AstTypeList{paramTypes, varargAnnotation})}; AstArray> paramNames = copy(names); @@ -1373,8 +1382,13 @@ AstType* Parser::parseFunctionTypeAnnotationTail(const Lexeme& begin, AstArray' instead of ':'"); + lexer.next(); + } // Users occasionally write '()' as the 'unit' type when they actually want to use 'nil', here we'll try to give a more specific error - if (lexer.current().type != Lexeme::SkinnyArrow && generics.size == 0 && genericPacks.size == 0 && params.size == 0) + else if (lexer.current().type != Lexeme::SkinnyArrow && generics.size == 0 && genericPacks.size == 0 && params.size == 0) { report(Location(begin.location, lexer.previousLocation()), "Expected '->' after '()' when parsing function type; did you mean 'nil'?"); diff --git a/Compiler/src/BytecodeBuilder.cpp b/Compiler/src/BytecodeBuilder.cpp index 3aa12d99..597b2f0a 100644 --- a/Compiler/src/BytecodeBuilder.cpp +++ b/Compiler/src/BytecodeBuilder.cpp @@ -6,8 +6,6 @@ #include #include -LUAU_FASTFLAG(LuauCompileNestedClosureO2) - namespace Luau { @@ -390,17 +388,15 @@ int32_t BytecodeBuilder::addConstantClosure(uint32_t fid) int16_t BytecodeBuilder::addChildFunction(uint32_t fid) { - if (FFlag::LuauCompileNestedClosureO2) - if (int16_t* cache = protoMap.find(fid)) - return *cache; + if (int16_t* cache = protoMap.find(fid)) + return *cache; uint32_t id = uint32_t(protos.size()); if (id >= kMaxClosureCount) return -1; - if (FFlag::LuauCompileNestedClosureO2) - protoMap[fid] = int16_t(id); + protoMap[fid] = int16_t(id); protos.push_back(fid); return int16_t(id); diff --git a/Compiler/src/Compiler.cpp b/Compiler/src/Compiler.cpp index eea56c60..7431cde4 100644 --- a/Compiler/src/Compiler.cpp +++ b/Compiler/src/Compiler.cpp @@ -16,7 +16,6 @@ #include #include -LUAU_FASTFLAGVARIABLE(LuauCompileIter, false) LUAU_FASTFLAGVARIABLE(LuauCompileIterNoPairs, false) LUAU_FASTINTVARIABLE(LuauCompileLoopUnrollThreshold, 25) @@ -26,8 +25,6 @@ LUAU_FASTINTVARIABLE(LuauCompileInlineThreshold, 25) LUAU_FASTINTVARIABLE(LuauCompileInlineThresholdMaxBoost, 300) LUAU_FASTINTVARIABLE(LuauCompileInlineDepth, 5) -LUAU_FASTFLAGVARIABLE(LuauCompileNestedClosureO2, false) - namespace Luau { @@ -172,30 +169,6 @@ struct Compiler return node->as(); } - bool canInlineFunctionBody(AstStat* stat) - { - if (FFlag::LuauCompileNestedClosureO2) - return true; // TODO: remove this function - - struct CanInlineVisitor : AstVisitor - { - bool result = true; - - bool visit(AstExprFunction* node) override - { - result = false; - - // short-circuit to avoid analyzing nested closure bodies - return false; - } - }; - - CanInlineVisitor canInline; - stat->visit(&canInline); - - return canInline.result; - } - uint32_t compileFunction(AstExprFunction* func) { LUAU_TIMETRACE_SCOPE("Compiler::compileFunction", "Compiler"); @@ -268,7 +241,7 @@ struct Compiler f.upvals = upvals; // record information for inlining - if (options.optimizationLevel >= 2 && !func->vararg && canInlineFunctionBody(func->body) && !getfenvUsed && !setfenvUsed) + if (options.optimizationLevel >= 2 && !func->vararg && !getfenvUsed && !setfenvUsed) { f.canInline = true; f.stackSize = stackSize; @@ -827,110 +800,62 @@ struct Compiler if (pid < 0) CompileError::raise(expr->location, "Exceeded closure limit; simplify the code to compile"); - if (FFlag::LuauCompileNestedClosureO2) - { - captures.clear(); - captures.reserve(f->upvals.size()); - - for (AstLocal* uv : f->upvals) - { - LUAU_ASSERT(uv->functionDepth < expr->functionDepth); - - if (int reg = getLocalReg(uv); reg >= 0) - { - // note: we can't check if uv is an upvalue in the current frame because inlining can migrate from upvalues to locals - Variable* ul = variables.find(uv); - bool immutable = !ul || !ul->written; - - captures.push_back({immutable ? LCT_VAL : LCT_REF, uint8_t(reg)}); - } - else if (const Constant* uc = locstants.find(uv); uc && uc->type != Constant::Type_Unknown) - { - // inlining can result in an upvalue capture of a constant, in which case we can't capture without a temporary register - uint8_t reg = allocReg(expr, 1); - compileExprConstant(expr, uc, reg); - - captures.push_back({LCT_VAL, reg}); - } - else - { - LUAU_ASSERT(uv->functionDepth < expr->functionDepth - 1); - - // get upvalue from parent frame - // note: this will add uv to the current upvalue list if necessary - uint8_t uid = getUpval(uv); - - captures.push_back({LCT_UPVAL, uid}); - } - } - - // Optimization: when closure has no upvalues, or upvalues are safe to share, instead of allocating it every time we can share closure - // objects (this breaks assumptions about function identity which can lead to setfenv not working as expected, so we disable this when it - // is used) - int16_t shared = -1; - - if (options.optimizationLevel >= 1 && shouldShareClosure(expr) && !setfenvUsed) - { - int32_t cid = bytecode.addConstantClosure(f->id); - - if (cid >= 0 && cid < 32768) - shared = int16_t(cid); - } - - if (shared >= 0) - bytecode.emitAD(LOP_DUPCLOSURE, target, shared); - else - bytecode.emitAD(LOP_NEWCLOSURE, target, pid); - - for (const Capture& c : captures) - bytecode.emitABC(LOP_CAPTURE, uint8_t(c.type), c.data, 0); - - return; - } - - bool shared = false; - - // Optimization: when closure has no upvalues, or upvalues are safe to share, instead of allocating it every time we can share closure - // objects (this breaks assumptions about function identity which can lead to setfenv not working as expected, so we disable this when it - // is used) - if (options.optimizationLevel >= 1 && shouldShareClosure(expr) && !setfenvUsed) - { - int32_t cid = bytecode.addConstantClosure(f->id); - - if (cid >= 0 && cid < 32768) - { - bytecode.emitAD(LOP_DUPCLOSURE, target, int16_t(cid)); - shared = true; - } - } - - if (!shared) - bytecode.emitAD(LOP_NEWCLOSURE, target, pid); + // we use a scratch vector to reduce allocations; this is safe since compileExprFunction is not reentrant + captures.clear(); + captures.reserve(f->upvals.size()); for (AstLocal* uv : f->upvals) { LUAU_ASSERT(uv->functionDepth < expr->functionDepth); - Variable* ul = variables.find(uv); - bool immutable = !ul || !ul->written; - - if (uv->functionDepth == expr->functionDepth - 1) + if (int reg = getLocalReg(uv); reg >= 0) { - // get local variable - int reg = getLocalReg(uv); - LUAU_ASSERT(reg >= 0); + // note: we can't check if uv is an upvalue in the current frame because inlining can migrate from upvalues to locals + Variable* ul = variables.find(uv); + bool immutable = !ul || !ul->written; - bytecode.emitABC(LOP_CAPTURE, uint8_t(immutable ? LCT_VAL : LCT_REF), uint8_t(reg), 0); + captures.push_back({immutable ? LCT_VAL : LCT_REF, uint8_t(reg)}); + } + else if (const Constant* uc = locstants.find(uv); uc && uc->type != Constant::Type_Unknown) + { + // inlining can result in an upvalue capture of a constant, in which case we can't capture without a temporary register + uint8_t reg = allocReg(expr, 1); + compileExprConstant(expr, uc, reg); + + captures.push_back({LCT_VAL, reg}); } else { + LUAU_ASSERT(uv->functionDepth < expr->functionDepth - 1); + // get upvalue from parent frame // note: this will add uv to the current upvalue list if necessary uint8_t uid = getUpval(uv); - bytecode.emitABC(LOP_CAPTURE, LCT_UPVAL, uid, 0); + captures.push_back({LCT_UPVAL, uid}); } } + + // Optimization: when closure has no upvalues, or upvalues are safe to share, instead of allocating it every time we can share closure + // objects (this breaks assumptions about function identity which can lead to setfenv not working as expected, so we disable this when it + // is used) + int16_t shared = -1; + + if (options.optimizationLevel >= 1 && shouldShareClosure(expr) && !setfenvUsed) + { + int32_t cid = bytecode.addConstantClosure(f->id); + + if (cid >= 0 && cid < 32768) + shared = int16_t(cid); + } + + if (shared >= 0) + bytecode.emitAD(LOP_DUPCLOSURE, target, shared); + else + bytecode.emitAD(LOP_NEWCLOSURE, target, pid); + + for (const Capture& c : captures) + bytecode.emitABC(LOP_CAPTURE, uint8_t(c.type), c.data, 0); } LuauOpcode getUnaryOp(AstExprUnary::Op op) @@ -2511,30 +2436,6 @@ struct Compiler pushLocal(stat->vars.data[i], uint8_t(vars + i)); } - bool canUnrollForBody(AstStatFor* stat) - { - if (FFlag::LuauCompileNestedClosureO2) - return true; // TODO: remove this function - - struct CanUnrollVisitor : AstVisitor - { - bool result = true; - - bool visit(AstExprFunction* node) override - { - result = false; - - // short-circuit to avoid analyzing nested closure bodies - return false; - } - }; - - CanUnrollVisitor canUnroll; - stat->body->visit(&canUnroll); - - return canUnroll.result; - } - bool tryCompileUnrolledFor(AstStatFor* stat, int thresholdBase, int thresholdMaxBoost) { Constant one = {Constant::Type_Number}; @@ -2560,12 +2461,6 @@ struct Compiler return false; } - if (!canUnrollForBody(stat)) - { - bytecode.addDebugRemark("loop unroll failed: unsupported loop body"); - return false; - } - if (Variable* lv = variables.find(stat->var); lv && lv->written) { bytecode.addDebugRemark("loop unroll failed: mutable loop variable"); @@ -2730,12 +2625,12 @@ struct Compiler uint8_t vars = allocReg(stat, std::max(unsigned(stat->vars.size), 2u)); LUAU_ASSERT(vars == regs + 3); - // Optimization: when we iterate through pairs/ipairs, we generate special bytecode that optimizes the traversal using internal iteration - // index These instructions dynamically check if generator is equal to next/inext and bail out They assume that the generator produces 2 - // variables, which is why we allocate at least 2 above (see vars assignment) - LuauOpcode skipOp = FFlag::LuauCompileIter ? LOP_FORGPREP : LOP_JUMP; + LuauOpcode skipOp = LOP_FORGPREP; LuauOpcode loopOp = LOP_FORGLOOP; + // Optimization: when we iterate via pairs/ipairs, we generate special bytecode that optimizes the traversal using internal iteration index + // These instructions dynamically check if generator is equal to next/inext and bail out + // They assume that the generator produces 2 variables, which is why we allocate at least 2 above (see vars assignment) if (options.optimizationLevel >= 1 && stat->vars.size <= 2) { if (stat->values.size == 1 && stat->values.data[0]->is()) diff --git a/VM/src/lapi.cpp b/VM/src/lapi.cpp index f86371da..3c3b7bd0 100644 --- a/VM/src/lapi.cpp +++ b/VM/src/lapi.cpp @@ -14,6 +14,26 @@ #include +/* + * This file contains most implementations of core Lua APIs from lua.h. + * + * These implementations should use api_check macros to verify that stack and type contracts hold; it's the callers + * responsibility to, for example, pass a valid table index to lua_rawgetfield. Generally errors should only be raised + * for conditions caller can't predict such as an out-of-memory error. + * + * The caller is expected to handle stack reservation (by using less than LUA_MINSTACK slots or by calling lua_checkstack). + * To ensure this is handled correctly, use api_incr_top(L) when pushing values to the stack. + * + * Functions that push any collectable objects to the stack *should* call luaC_checkthreadsleep. Failure to do this can result + * in stack references that point to dead objects since sleeping threads don't get rescanned. + * + * Functions that push newly created objects to the stack *should* call luaC_checkGC in addition to luaC_checkthreadsleep. + * Failure to do this can result in OOM since GC may never run. + * + * Note that luaC_checkGC may scan the thread and put it back to sleep; functions that call both before pushing objects must + * therefore call luaC_checkGC before luaC_checkthreadsleep to guarantee the object is pushed to an awake thread. + */ + const char* lua_ident = "$Lua: Lua 5.1.4 Copyright (C) 1994-2008 Lua.org, PUC-Rio $\n" "$Authors: R. Ierusalimschy, L. H. de Figueiredo & W. Celes $\n" "$URL: www.lua.org $\n"; @@ -221,15 +241,13 @@ void lua_insert(lua_State* L, int idx) void lua_replace(lua_State* L, int idx) { - /* explicit test for incompatible code */ - if (idx == LUA_ENVIRONINDEX && L->ci == L->base_ci) - luaG_runerror(L, "no calling environment"); api_checknelems(L, 1); luaC_checkthreadsleep(L); StkId o = index2addr(L, idx); api_checkvalidindex(L, o); if (idx == LUA_ENVIRONINDEX) { + api_check(L, L->ci != L->base_ci); Closure* func = curr_func(L); api_check(L, ttistable(L->top - 1)); func->env = hvalue(L->top - 1); @@ -443,9 +461,7 @@ const float* lua_tovector(lua_State* L, int idx) { StkId o = index2addr(L, idx); if (!ttisvector(o)) - { return NULL; - } return vvalue(o); } @@ -460,11 +476,6 @@ int lua_objlen(lua_State* L, int idx) return uvalue(o)->len; case LUA_TTABLE: return luaH_getn(hvalue(o)); - case LUA_TNUMBER: - { - int l = (luaV_tostring(L, o) ? tsvalue(o)->len : 0); - return l; - } default: return 0; } @@ -752,10 +763,9 @@ void lua_setsafeenv(lua_State* L, int objindex, int enabled) int lua_getmetatable(lua_State* L, int objindex) { - const TValue* obj; + luaC_checkthreadsleep(L); Table* mt = NULL; - int res; - obj = index2addr(L, objindex); + const TValue* obj = index2addr(L, objindex); switch (ttype(obj)) { case LUA_TTABLE: @@ -768,21 +778,18 @@ int lua_getmetatable(lua_State* L, int objindex) mt = L->global->mt[ttype(obj)]; break; } - if (mt == NULL) - res = 0; - else + if (mt) { sethvalue(L, L->top, mt); api_incr_top(L); - res = 1; } - return res; + return mt != NULL; } void lua_getfenv(lua_State* L, int idx) { - StkId o; - o = index2addr(L, idx); + luaC_checkthreadsleep(L); + StkId o = index2addr(L, idx); api_checkvalidindex(L, o); switch (ttype(o)) { @@ -806,9 +813,8 @@ void lua_getfenv(lua_State* L, int idx) void lua_settable(lua_State* L, int idx) { - StkId t; api_checknelems(L, 2); - t = index2addr(L, idx); + StkId t = index2addr(L, idx); api_checkvalidindex(L, t); luaV_settable(L, t, L->top - 2, L->top - 1); L->top -= 2; /* pop index and value */ @@ -817,22 +823,20 @@ void lua_settable(lua_State* L, int idx) void lua_setfield(lua_State* L, int idx, const char* k) { - StkId t; - TValue key; api_checknelems(L, 1); - t = index2addr(L, idx); + StkId t = index2addr(L, idx); api_checkvalidindex(L, t); + TValue key; setsvalue(L, &key, luaS_new(L, k)); luaV_settable(L, t, &key, L->top - 1); - L->top--; /* pop value */ + L->top--; return; } void lua_rawset(lua_State* L, int idx) { - StkId t; api_checknelems(L, 2); - t = index2addr(L, idx); + StkId t = index2addr(L, idx); api_check(L, ttistable(t)); if (hvalue(t)->readonly) luaG_runerror(L, "Attempt to modify a readonly table"); @@ -844,9 +848,8 @@ void lua_rawset(lua_State* L, int idx) void lua_rawseti(lua_State* L, int idx, int n) { - StkId o; api_checknelems(L, 1); - o = index2addr(L, idx); + StkId o = index2addr(L, idx); api_check(L, ttistable(o)); if (hvalue(o)->readonly) luaG_runerror(L, "Attempt to modify a readonly table"); @@ -858,14 +861,11 @@ void lua_rawseti(lua_State* L, int idx, int n) int lua_setmetatable(lua_State* L, int objindex) { - TValue* obj; - Table* mt; api_checknelems(L, 1); - obj = index2addr(L, objindex); + TValue* obj = index2addr(L, objindex); api_checkvalidindex(L, obj); - if (ttisnil(L->top - 1)) - mt = NULL; - else + Table* mt = NULL; + if (!ttisnil(L->top - 1)) { api_check(L, ttistable(L->top - 1)); mt = hvalue(L->top - 1); @@ -900,10 +900,9 @@ int lua_setmetatable(lua_State* L, int objindex) int lua_setfenv(lua_State* L, int idx) { - StkId o; int res = 1; api_checknelems(L, 1); - o = index2addr(L, idx); + StkId o = index2addr(L, idx); api_checkvalidindex(L, o); api_check(L, ttistable(L->top - 1)); switch (ttype(o)) @@ -970,24 +969,21 @@ static void f_call(lua_State* L, void* ud) int lua_pcall(lua_State* L, int nargs, int nresults, int errfunc) { - struct CallS c; - int status; - ptrdiff_t func; api_checknelems(L, nargs + 1); api_check(L, L->status == 0); checkresults(L, nargs, nresults); - if (errfunc == 0) - func = 0; - else + ptrdiff_t func = 0; + if (errfunc != 0) { StkId o = index2addr(L, errfunc); api_checkvalidindex(L, o); func = savestack(L, o); } + struct CallS c; c.func = L->top - (nargs + 1); /* function to be called */ c.nresults = nresults; - status = luaD_pcall(L, f_call, &c, savestack(L, c.func), func); + int status = luaD_pcall(L, f_call, &c, savestack(L, c.func), func); adjustresults(L, nresults); return status; @@ -1247,12 +1243,10 @@ const char* lua_getupvalue(lua_State* L, int funcindex, int n) const char* lua_setupvalue(lua_State* L, int funcindex, int n) { - const char* name; - TValue* val; - StkId fi; - fi = index2addr(L, funcindex); api_checknelems(L, 1); - name = aux_upvalue(fi, n, &val); + StkId fi = index2addr(L, funcindex); + TValue* val; + const char* name = aux_upvalue(fi, n, &val); if (name) { L->top--; @@ -1319,14 +1313,16 @@ void lua_setuserdatadtor(lua_State* L, int tag, void (*dtor)(lua_State*, void*)) void lua_clonefunction(lua_State* L, int idx) { + luaC_checkGC(L); + luaC_checkthreadsleep(L); StkId p = index2addr(L, idx); api_check(L, isLfunction(p)); - - luaC_checkthreadsleep(L); - Closure* cl = clvalue(p); - Closure* newcl = luaF_newLclosure(L, 0, L->gt, cl->l.p); - setclvalue(L, L->top - 1, newcl); + Closure* newcl = luaF_newLclosure(L, cl->nupvalues, L->gt, cl->l.p); + for (int i = 0; i < cl->nupvalues; ++i) + setobj2n(L, &newcl->l.uprefs[i], &cl->l.uprefs[i]); + setclvalue(L, L->top, newcl); + api_incr_top(L); } lua_Callbacks* lua_callbacks(lua_State* L) diff --git a/VM/src/lbuiltins.cpp b/VM/src/lbuiltins.cpp index cc6e560a..deaf1407 100644 --- a/VM/src/lbuiltins.cpp +++ b/VM/src/lbuiltins.cpp @@ -1018,18 +1018,20 @@ static int luauF_tunpack(lua_State* L, StkId res, TValue* arg0, int nresults, St static int luauF_vector(lua_State* L, StkId res, TValue* arg0, int nresults, StkId args, int nparams) { -#if LUA_VECTOR_SIZE == 4 - if (nparams >= 4 && nresults <= 1 && ttisnumber(arg0) && ttisnumber(args) && ttisnumber(args + 1) && ttisnumber(args + 2)) -#else if (nparams >= 3 && nresults <= 1 && ttisnumber(arg0) && ttisnumber(args) && ttisnumber(args + 1)) -#endif { double x = nvalue(arg0); double y = nvalue(args); double z = nvalue(args + 1); #if LUA_VECTOR_SIZE == 4 - double w = nvalue(args + 2); + double w = 0.0; + if (nparams >= 4) + { + if (!ttisnumber(args + 2)) + return -1; + w = nvalue(args + 2); + } setvvalue(res, float(x), float(y), float(z), float(w)); #else setvvalue(res, float(x), float(y), float(z), 0.0f); diff --git a/VM/src/ldo.cpp b/VM/src/ldo.cpp index a71fce52..0642cb6d 100644 --- a/VM/src/ldo.cpp +++ b/VM/src/ldo.cpp @@ -202,22 +202,29 @@ void luaD_growstack(lua_State* L, int n) CallInfo* luaD_growCI(lua_State* L) { - if (L->size_ci > LUAI_MAXCALLS) /* overflow while handling overflow? */ - luaD_throw(L, LUA_ERRERR); - else - { - luaD_reallocCI(L, 2 * L->size_ci); - if (L->size_ci > LUAI_MAXCALLS) - luaG_runerror(L, "stack overflow"); - } + /* allow extra stack space to handle stack overflow in xpcall */ + const int hardlimit = LUAI_MAXCALLS + (LUAI_MAXCALLS >> 3); + + if (L->size_ci >= hardlimit) + luaD_throw(L, LUA_ERRERR); /* error while handling stack error */ + + int request = L->size_ci * 2; + luaD_reallocCI(L, L->size_ci >= LUAI_MAXCALLS ? hardlimit : request < LUAI_MAXCALLS ? request : LUAI_MAXCALLS); + + if (L->size_ci > LUAI_MAXCALLS) + luaG_runerror(L, "stack overflow"); + return ++L->ci; } void luaD_checkCstack(lua_State* L) { + /* allow extra stack space to handle stack overflow in xpcall */ + const int hardlimit = LUAI_MAXCCALLS + (LUAI_MAXCCALLS >> 3); + if (L->nCcalls == LUAI_MAXCCALLS) luaG_runerror(L, "C stack overflow"); - else if (L->nCcalls >= (LUAI_MAXCCALLS + (LUAI_MAXCCALLS >> 3))) + else if (L->nCcalls >= hardlimit) luaD_throw(L, LUA_ERRERR); /* error while handling stack error */ } diff --git a/VM/src/lvmexecute.cpp b/VM/src/lvmexecute.cpp index f9fd6574..e0a96474 100644 --- a/VM/src/lvmexecute.cpp +++ b/VM/src/lvmexecute.cpp @@ -16,8 +16,6 @@ #include -LUAU_FASTFLAGVARIABLE(LuauIter, false) - // Disable c99-designator to avoid the warning in CGOTO dispatch table #ifdef __clang__ #if __has_warning("-Wc99-designator") @@ -2214,7 +2212,7 @@ static void luau_execute(lua_State* L) { /* will be called during FORGLOOP */ } - else if (FFlag::LuauIter) + else { Table* mt = ttistable(ra) ? hvalue(ra)->metatable : ttisuserdata(ra) ? uvalue(ra)->metatable : cast_to(Table*, NULL); @@ -2259,17 +2257,6 @@ static void luau_execute(lua_State* L) StkId ra = VM_REG(LUAU_INSN_A(insn)); uint32_t aux = *pc; - if (!FFlag::LuauIter) - { - bool stop; - VM_PROTECT(stop = luau_loopFORG(L, LUAU_INSN_A(insn), aux)); - - // note that we need to increment pc by 1 to exit the loop since we need to skip over aux - pc += stop ? 1 : LUAU_INSN_D(insn); - LUAU_ASSERT(unsigned(pc - cl->l.p->code) < unsigned(cl->l.p->sizecode)); - VM_NEXT(); - } - // fast-path: builtin table iteration if (ttisnil(ra) && ttistable(ra + 1) && ttislightuserdata(ra + 2)) { @@ -2362,7 +2349,7 @@ static void luau_execute(lua_State* L) /* ra+1 is already the table */ setpvalue(ra + 2, reinterpret_cast(uintptr_t(0))); } - else if (FFlag::LuauIter && !ttisfunction(ra)) + else if (!ttisfunction(ra)) { VM_PROTECT(luaG_typeerror(L, ra, "iterate over")); } @@ -2434,7 +2421,7 @@ static void luau_execute(lua_State* L) /* ra+1 is already the table */ setpvalue(ra + 2, reinterpret_cast(uintptr_t(0))); } - else if (FFlag::LuauIter && !ttisfunction(ra)) + else if (!ttisfunction(ra)) { VM_PROTECT(luaG_typeerror(L, ra, "iterate over")); } diff --git a/tests/Autocomplete.test.cpp b/tests/Autocomplete.test.cpp index cc5b31c9..dea1ab19 100644 --- a/tests/Autocomplete.test.cpp +++ b/tests/Autocomplete.test.cpp @@ -2764,8 +2764,6 @@ TEST_CASE_FIXTURE(ACBuiltinsFixture, "autocomplete_on_string_singletons") TEST_CASE_FIXTURE(ACFixture, "autocomplete_string_singletons") { - ScopedFastFlag sff{"LuauTwoPassAliasDefinitionFix", true}; - check(R"( type tag = "cat" | "dog" local function f(a: tag) end @@ -2838,8 +2836,6 @@ f(@1) TEST_CASE_FIXTURE(ACFixture, "autocomplete_string_singleton_escape") { - ScopedFastFlag sff{"LuauTwoPassAliasDefinitionFix", true}; - check(R"( type tag = "strange\t\"cat\"" | 'nice\t"dog"' local function f(x: tag) end @@ -2873,7 +2869,7 @@ local abc = b@1 TEST_CASE_FIXTURE(ACFixture, "no_incompatible_self_calls_on_class") { - ScopedFastFlag selfCallAutocompleteFix{"LuauSelfCallAutocompleteFix", true}; + ScopedFastFlag selfCallAutocompleteFix2{"LuauSelfCallAutocompleteFix2", true}; loadDefinition(R"( declare class Foo @@ -2913,7 +2909,7 @@ t.@1 TEST_CASE_FIXTURE(ACFixture, "no_incompatible_self_calls") { - ScopedFastFlag selfCallAutocompleteFix{"LuauSelfCallAutocompleteFix", true}; + ScopedFastFlag selfCallAutocompleteFix2{"LuauSelfCallAutocompleteFix2", true}; check(R"( local t = {} @@ -2929,7 +2925,7 @@ t:@1 TEST_CASE_FIXTURE(ACFixture, "no_incompatible_self_calls_2") { - ScopedFastFlag selfCallAutocompleteFix{"LuauSelfCallAutocompleteFix", true}; + ScopedFastFlag selfCallAutocompleteFix2{"LuauSelfCallAutocompleteFix2", true}; check(R"( local f: (() -> number) & ((number) -> number) = function(x: number?) return 2 end @@ -2961,7 +2957,7 @@ t:@1 TEST_CASE_FIXTURE(ACFixture, "string_prim_self_calls_are_fine") { - ScopedFastFlag selfCallAutocompleteFix{"LuauSelfCallAutocompleteFix", true}; + ScopedFastFlag selfCallAutocompleteFix2{"LuauSelfCallAutocompleteFix2", true}; check(R"( local s = "hello" @@ -2980,7 +2976,7 @@ s:@1 TEST_CASE_FIXTURE(ACFixture, "string_prim_non_self_calls_are_avoided") { - ScopedFastFlag selfCallAutocompleteFix{"LuauSelfCallAutocompleteFix", true}; + ScopedFastFlag selfCallAutocompleteFix2{"LuauSelfCallAutocompleteFix2", true}; check(R"( local s = "hello" @@ -2989,17 +2985,15 @@ s.@1 auto ac = autocomplete('1'); - REQUIRE(ac.entryMap.count("byte")); - CHECK(ac.entryMap["byte"].wrongIndexType == true); REQUIRE(ac.entryMap.count("char")); CHECK(ac.entryMap["char"].wrongIndexType == false); REQUIRE(ac.entryMap.count("sub")); CHECK(ac.entryMap["sub"].wrongIndexType == true); } -TEST_CASE_FIXTURE(ACBuiltinsFixture, "string_library_non_self_calls_are_fine") +TEST_CASE_FIXTURE(ACBuiltinsFixture, "library_non_self_calls_are_fine") { - ScopedFastFlag selfCallAutocompleteFix{"LuauSelfCallAutocompleteFix", true}; + ScopedFastFlag selfCallAutocompleteFix2{"LuauSelfCallAutocompleteFix2", true}; check(R"( string.@1 @@ -3013,11 +3007,24 @@ string.@1 CHECK(ac.entryMap["char"].wrongIndexType == false); REQUIRE(ac.entryMap.count("sub")); CHECK(ac.entryMap["sub"].wrongIndexType == false); + + check(R"( +table.@1 + )"); + + ac = autocomplete('1'); + + REQUIRE(ac.entryMap.count("remove")); + CHECK(ac.entryMap["remove"].wrongIndexType == false); + REQUIRE(ac.entryMap.count("getn")); + CHECK(ac.entryMap["getn"].wrongIndexType == false); + REQUIRE(ac.entryMap.count("insert")); + CHECK(ac.entryMap["insert"].wrongIndexType == false); } -TEST_CASE_FIXTURE(ACBuiltinsFixture, "string_library_self_calls_are_invalid") +TEST_CASE_FIXTURE(ACBuiltinsFixture, "library_self_calls_are_invalid") { - ScopedFastFlag selfCallAutocompleteFix{"LuauSelfCallAutocompleteFix", true}; + ScopedFastFlag selfCallAutocompleteFix2{"LuauSelfCallAutocompleteFix2", true}; check(R"( string:@1 diff --git a/tests/Compiler.test.cpp b/tests/Compiler.test.cpp index 20139650..6eee254e 100644 --- a/tests/Compiler.test.cpp +++ b/tests/Compiler.test.cpp @@ -261,7 +261,6 @@ L1: RETURN R0 0 TEST_CASE("ForBytecode") { - ScopedFastFlag sff("LuauCompileIter", true); ScopedFastFlag sff2("LuauCompileIterNoPairs", false); // basic for loop: variable directly refers to internal iteration index (R2) @@ -350,8 +349,6 @@ RETURN R0 0 TEST_CASE("ForBytecodeBuiltin") { - ScopedFastFlag sff("LuauCompileIter", true); - // we generally recognize builtins like pairs/ipairs and emit special opcodes CHECK_EQ("\n" + compileFunction0("for k,v in ipairs({}) do end"), R"( GETIMPORT R0 1 @@ -2323,8 +2320,6 @@ return result TEST_CASE("DebugLineInfoFor") { - ScopedFastFlag sff("LuauCompileIter", true); - Luau::BytecodeBuilder bcb; bcb.setDumpFlags(Luau::BytecodeBuilder::Dump_Code | Luau::BytecodeBuilder::Dump_Lines); Luau::compileOrThrow(bcb, R"( @@ -4355,8 +4350,6 @@ L1: RETURN R0 0 TEST_CASE("LoopUnrollControlFlow") { - ScopedFastFlag sff("LuauCompileNestedClosureO2", true); - ScopedFastInt sfis[] = { {"LuauCompileLoopUnrollThreshold", 50}, {"LuauCompileLoopUnrollThresholdMaxBoost", 300}, @@ -4475,8 +4468,6 @@ RETURN R0 0 TEST_CASE("LoopUnrollNestedClosure") { - ScopedFastFlag sff("LuauCompileNestedClosureO2", true); - // if the body has functions that refer to loop variables, we unroll the loop and use MOVE+CAPTURE for upvalues CHECK_EQ("\n" + compileFunction(R"( for i=1,2 do @@ -4756,8 +4747,6 @@ RETURN R1 1 TEST_CASE("InlineBasicProhibited") { - ScopedFastFlag sff("LuauCompileNestedClosureO2", true); - // we can't inline variadic functions CHECK_EQ("\n" + compileFunction(R"( local function foo(...) @@ -4833,8 +4822,6 @@ RETURN R1 1 TEST_CASE("InlineNestedClosures") { - ScopedFastFlag sff("LuauCompileNestedClosureO2", true); - // we can inline functions that contain/return functions CHECK_EQ("\n" + compileFunction(R"( local function foo(x) diff --git a/tests/Conformance.test.cpp b/tests/Conformance.test.cpp index f7f2b4ac..96a2775f 100644 --- a/tests/Conformance.test.cpp +++ b/tests/Conformance.test.cpp @@ -741,7 +741,7 @@ TEST_CASE("ApiTables") lua_pop(L, 1); } -TEST_CASE("ApiFunctionCalls") +TEST_CASE("ApiCalls") { StateRef globalState = runConformance("apicalls.lua"); lua_State* L = globalState.get(); @@ -790,6 +790,58 @@ TEST_CASE("ApiFunctionCalls") CHECK(lua_equal(L2, -1, -2) == 1); lua_pop(L2, 2); } + + // lua_clonefunction + fenv + { + lua_getfield(L, LUA_GLOBALSINDEX, "getpi"); + lua_call(L, 0, 1); + CHECK(lua_tonumber(L, -1) == 3.1415926); + lua_pop(L, 1); + + lua_getfield(L, LUA_GLOBALSINDEX, "getpi"); + + // clone & override env + lua_clonefunction(L, -1); + lua_newtable(L); + lua_pushnumber(L, 42); + lua_setfield(L, -2, "pi"); + lua_setfenv(L, -2); + + lua_call(L, 0, 1); + CHECK(lua_tonumber(L, -1) == 42); + lua_pop(L, 1); + + // this one calls original function again + lua_call(L, 0, 1); + CHECK(lua_tonumber(L, -1) == 3.1415926); + lua_pop(L, 1); + } + + // lua_clonefunction + upvalues + { + lua_getfield(L, LUA_GLOBALSINDEX, "incuv"); + lua_call(L, 0, 1); + CHECK(lua_tonumber(L, -1) == 1); + lua_pop(L, 1); + + lua_getfield(L, LUA_GLOBALSINDEX, "incuv"); + // two clones + lua_clonefunction(L, -1); + lua_clonefunction(L, -2); + + lua_call(L, 0, 1); + CHECK(lua_tonumber(L, -1) == 2); + lua_pop(L, 1); + + lua_call(L, 0, 1); + CHECK(lua_tonumber(L, -1) == 3); + lua_pop(L, 1); + + // this one calls original function again + lua_call(L, 0, 1); + CHECK(lua_tonumber(L, -1) == 4); + lua_pop(L, 1); + } } static bool endsWith(const std::string& str, const std::string& suffix) @@ -1113,11 +1165,6 @@ TEST_CASE("UserdataApi") TEST_CASE("Iter") { - ScopedFastFlag sffs[] = { - {"LuauCompileIter", true}, - {"LuauIter", true}, - }; - runConformance("iter.lua"); } diff --git a/tests/Module.test.cpp b/tests/Module.test.cpp index c7e18efd..89b13ab1 100644 --- a/tests/Module.test.cpp +++ b/tests/Module.test.cpp @@ -300,4 +300,24 @@ TEST_CASE_FIXTURE(Fixture, "clone_recursion_limit") CHECK_THROWS_AS(clone(table, dest, cloneState), RecursionLimitException); } +TEST_CASE_FIXTURE(Fixture, "any_persistance_does_not_leak") +{ + ScopedFastFlag luauNonCopyableTypeVarFields{"LuauNonCopyableTypeVarFields", true}; + + fileResolver.source["Module/A"] = R"( +export type A = B +type B = A + )"; + + FrontendOptions opts; + opts.retainFullTypeGraphs = false; + CheckResult result = frontend.check("Module/A", opts); + LUAU_REQUIRE_ERRORS(result); + + auto mod = frontend.moduleResolver.getModule("Module/A"); + auto it = mod->getModuleScope()->exportedTypeBindings.find("A"); + REQUIRE(it != mod->getModuleScope()->exportedTypeBindings.end()); + CHECK(toString(it->second.type) == "any"); +} + TEST_SUITE_END(); diff --git a/tests/Parser.test.cpp b/tests/Parser.test.cpp index 87b1263f..878023e3 100644 --- a/tests/Parser.test.cpp +++ b/tests/Parser.test.cpp @@ -2622,6 +2622,23 @@ type Z = { a: string | T..., b: number } REQUIRE_EQ(3, result.errors.size()); } +TEST_CASE_FIXTURE(Fixture, "recover_function_return_type_annotations") +{ + ScopedFastFlag sff{"LuauReturnTypeTokenConfusion", true}; + ParseResult result = tryParse(R"( +type Custom = { x: A, y: B, z: C } +type Packed = { x: (A...) -> () } +type F = (number): Custom +type G = Packed<(number): (string, number, boolean)> +local function f(x: number) -> Custom +end + )"); + REQUIRE_EQ(3, result.errors.size()); + CHECK_EQ(result.errors[0].getMessage(), "Return types in function type annotations are written after '->' instead of ':'"); + CHECK_EQ(result.errors[1].getMessage(), "Return types in function type annotations are written after '->' instead of ':'"); + CHECK_EQ(result.errors[2].getMessage(), "Function return type annotations are written after ':' instead of '->'"); +} + TEST_CASE_FIXTURE(Fixture, "error_message_for_using_function_as_type_annotation") { ScopedFastFlag sff{"LuauParserFunctionKeywordAsTypeHelp", true}; diff --git a/tests/TypeInfer.aliases.test.cpp b/tests/TypeInfer.aliases.test.cpp index 7562a4d7..86cc9701 100644 --- a/tests/TypeInfer.aliases.test.cpp +++ b/tests/TypeInfer.aliases.test.cpp @@ -615,8 +615,6 @@ TEST_CASE_FIXTURE(Fixture, "generic_typevars_are_not_considered_to_escape_their_ */ TEST_CASE_FIXTURE(Fixture, "forward_declared_alias_is_not_clobbered_by_prior_unification_with_any") { - ScopedFastFlag sff[] = {{"LuauTwoPassAliasDefinitionFix", true}}; - CheckResult result = check(R"( local function x() local y: FutureType = {}::any @@ -633,10 +631,6 @@ TEST_CASE_FIXTURE(Fixture, "forward_declared_alias_is_not_clobbered_by_prior_uni TEST_CASE_FIXTURE(Fixture, "forward_declared_alias_is_not_clobbered_by_prior_unification_with_any_2") { - ScopedFastFlag sff[] = { - {"LuauTwoPassAliasDefinitionFix", true}, - }; - CheckResult result = check(R"( local B = {} B.bar = 4 diff --git a/tests/TypeInfer.loops.test.cpp b/tests/TypeInfer.loops.test.cpp index 4444cd66..1c6fe1d8 100644 --- a/tests/TypeInfer.loops.test.cpp +++ b/tests/TypeInfer.loops.test.cpp @@ -486,8 +486,6 @@ TEST_CASE_FIXTURE(Fixture, "fuzz_fail_missing_instantitation_follow") TEST_CASE_FIXTURE(Fixture, "loop_iter_basic") { - ScopedFastFlag sff{"LuauTypecheckIter", true}; - CheckResult result = check(R"( local t: {string} = {} local key @@ -506,8 +504,6 @@ TEST_CASE_FIXTURE(Fixture, "loop_iter_basic") TEST_CASE_FIXTURE(Fixture, "loop_iter_trailing_nil") { - ScopedFastFlag sff{"LuauTypecheckIter", true}; - CheckResult result = check(R"( local t: {string} = {} local extra @@ -522,8 +518,6 @@ TEST_CASE_FIXTURE(Fixture, "loop_iter_trailing_nil") TEST_CASE_FIXTURE(Fixture, "loop_iter_no_indexer") { - ScopedFastFlag sff{"LuauTypecheckIter", true}; - CheckResult result = check(R"( local t = {} for k, v in t do @@ -539,8 +533,6 @@ TEST_CASE_FIXTURE(Fixture, "loop_iter_no_indexer") TEST_CASE_FIXTURE(BuiltinsFixture, "loop_iter_iter_metamethod") { - ScopedFastFlag sff{"LuauTypecheckIter", true}; - CheckResult result = check(R"( local t = {} setmetatable(t, { __iter = function(o) return next, o.children end }) diff --git a/tests/TypeInfer.refinements.test.cpp b/tests/TypeInfer.refinements.test.cpp index 6785f277..207b3cff 100644 --- a/tests/TypeInfer.refinements.test.cpp +++ b/tests/TypeInfer.refinements.test.cpp @@ -932,6 +932,8 @@ TEST_CASE_FIXTURE(Fixture, "apply_refinements_on_astexprindexexpr_whose_subscrip TEST_CASE_FIXTURE(Fixture, "discriminate_from_truthiness_of_x") { + ScopedFastFlag sff{"LuauFalsyPredicateReturnsNilInstead", true}; + CheckResult result = check(R"( type T = {tag: "missing", x: nil} | {tag: "exists", x: string} @@ -947,7 +949,7 @@ TEST_CASE_FIXTURE(Fixture, "discriminate_from_truthiness_of_x") LUAU_REQUIRE_NO_ERRORS(result); CHECK_EQ(R"({| tag: "exists", x: string |})", toString(requireTypeAtPosition({5, 28}))); - CHECK_EQ(R"({| tag: "missing", x: nil |})", toString(requireTypeAtPosition({7, 28}))); + CHECK_EQ(R"({| tag: "exists", x: string |} | {| tag: "missing", x: nil |})", toString(requireTypeAtPosition({7, 28}))); } TEST_CASE_FIXTURE(Fixture, "discriminate_tag") @@ -1191,7 +1193,7 @@ TEST_CASE_FIXTURE(RefinementClassFixture, "x_is_not_instance_or_else_not_part") TEST_CASE_FIXTURE(Fixture, "typeguard_doesnt_leak_to_elseif") { - const std::string code = R"( + CheckResult result = check(R"( function f(a) if type(a) == "boolean" then local a1 = a @@ -1201,10 +1203,30 @@ TEST_CASE_FIXTURE(Fixture, "typeguard_doesnt_leak_to_elseif") local a3 = a end end - )"; - CheckResult result = check(code); + )"); + LUAU_REQUIRE_NO_ERRORS(result); - dumpErrors(result); +} + +TEST_CASE_FIXTURE(BuiltinsFixture, "falsiness_of_TruthyPredicate_narrows_into_nil") +{ + ScopedFastFlag sff{"LuauFalsyPredicateReturnsNilInstead", true}; + + CheckResult result = check(R"( + local function f(t: {number}) + local x = t[1] + if not x then + local foo = x + else + local bar = x + end + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ("nil", toString(requireTypeAtPosition({4, 28}))); + CHECK_EQ("number", toString(requireTypeAtPosition({6, 28}))); } TEST_SUITE_END(); diff --git a/tests/TypeInfer.singletons.test.cpp b/tests/TypeInfer.singletons.test.cpp index 14a5a6ae..a90f434f 100644 --- a/tests/TypeInfer.singletons.test.cpp +++ b/tests/TypeInfer.singletons.test.cpp @@ -139,8 +139,6 @@ TEST_CASE_FIXTURE(Fixture, "enums_using_singletons") TEST_CASE_FIXTURE(Fixture, "enums_using_singletons_mismatch") { - ScopedFastFlag sff{"LuauTwoPassAliasDefinitionFix", true}; - CheckResult result = check(R"( type MyEnum = "foo" | "bar" | "baz" local a : MyEnum = "bang" diff --git a/tests/TypePack.test.cpp b/tests/TypePack.test.cpp index c4931578..8a5a65fe 100644 --- a/tests/TypePack.test.cpp +++ b/tests/TypePack.test.cpp @@ -197,4 +197,20 @@ TEST_CASE_FIXTURE(TypePackFixture, "std_distance") CHECK_EQ(4, std::distance(b, e)); } +TEST_CASE("content_reassignment") +{ + ScopedFastFlag luauNonCopyableTypeVarFields{"LuauNonCopyableTypeVarFields", true}; + + TypePackVar myError{Unifiable::Error{}, /*presistent*/ true}; + + TypeArena arena; + + TypePackId futureError = arena.addTypePack(TypePackVar{FreeTypePack{TypeLevel{}}}); + asMutable(futureError)->reassign(myError); + + CHECK(get(futureError) != nullptr); + CHECK(!futureError->persistent); + CHECK(futureError->owningArena == &arena); +} + TEST_SUITE_END(); diff --git a/tests/TypeVar.test.cpp b/tests/TypeVar.test.cpp index bb2d94ba..4f8fc502 100644 --- a/tests/TypeVar.test.cpp +++ b/tests/TypeVar.test.cpp @@ -416,4 +416,24 @@ TEST_CASE("proof_that_isBoolean_uses_all_of") CHECK(!isBoolean(&union_)); } +TEST_CASE("content_reassignment") +{ + ScopedFastFlag luauNonCopyableTypeVarFields{"LuauNonCopyableTypeVarFields", true}; + + TypeVar myAny{AnyTypeVar{}, /*presistent*/ true}; + myAny.normal = true; + myAny.documentationSymbol = "@global/any"; + + TypeArena arena; + + TypeId futureAny = arena.addType(FreeTypeVar{TypeLevel{}}); + asMutable(futureAny)->reassign(myAny); + + CHECK(get(futureAny) != nullptr); + CHECK(!futureAny->persistent); + CHECK(futureAny->normal); + CHECK(futureAny->documentationSymbol == "@global/any"); + CHECK(futureAny->owningArena == &arena); +} + TEST_SUITE_END(); diff --git a/tests/conformance/apicalls.lua b/tests/conformance/apicalls.lua index 7a4058b5..27416623 100644 --- a/tests/conformance/apicalls.lua +++ b/tests/conformance/apicalls.lua @@ -11,4 +11,15 @@ function create_with_tm(x) return setmetatable({ a = x }, m) end +local gen = 0 +function incuv() + gen += 1 + return gen +end + +pi = 3.1415926 +function getpi() + return pi +end + return('OK') diff --git a/tests/conformance/pcall.lua b/tests/conformance/pcall.lua index 84ac2ba1..969209fc 100644 --- a/tests/conformance/pcall.lua +++ b/tests/conformance/pcall.lua @@ -144,4 +144,21 @@ coroutine.resume(co) resumeerror(co, "fail") checkresults({ true, false, "fail" }, coroutine.resume(co)) +-- stack overflow needs to happen at the call limit +local calllimit = 20000 +function recurse(n) return n <= 1 and 1 or recurse(n-1) + 1 end + +-- we use one frame for top-level function and one frame is the service frame for coroutines +assert(recurse(calllimit - 2) == calllimit - 2) + +-- note that when calling through pcall, pcall eats one more frame +checkresults({ true, calllimit - 3 }, pcall(recurse, calllimit - 3)) +checkerror(pcall(recurse, calllimit - 2)) + +-- xpcall handler runs in context of the stack frame, but this works just fine since we allow extra stack consumption past stack overflow +checkresults({ false, "ok" }, xpcall(recurse, function() return string.reverse("ko") end, calllimit - 2)) + +-- however, if xpcall handler itself runs out of extra stack space, we get "error in error handling" +checkresults({ false, "error in error handling" }, xpcall(recurse, function() return recurse(calllimit) end, calllimit - 2)) + return 'OK' From b066e4c8f8851d9727d079935cfa2e0f9b108531 Mon Sep 17 00:00:00 2001 From: rblanckaert <63755228+rblanckaert@users.noreply.github.com> Date: Fri, 10 Jun 2022 09:58:21 -0700 Subject: [PATCH 271/399] 0.531 (#532) * Fix free Luau type being fully overwritten by 'any' and causing UAF * Fix lua_clonefunction implementation replacing top instead of pushing * Falsey values other than false can now narrow refinements * Fix lua_getmetatable, lua_getfenv not waking thread up * FIx a case where lua_objlen could push a new string without thread wakeup or GC * Moved Luau math and bit32 definitions to definition file * Improve Luau parse recovery of incorrect return type token --- Analysis/include/Luau/TypePack.h | 13 +- Analysis/include/Luau/TypeVar.h | 11 +- Analysis/src/Autocomplete.cpp | 41 ++-- Analysis/src/BuiltinDefinitions.cpp | 36 +--- Analysis/src/Clone.cpp | 5 - Analysis/src/EmbeddedBuiltinDefinitions.cpp | 8 +- Analysis/src/Instantiation.cpp | 4 - Analysis/src/Quantify.cpp | 35 ---- Analysis/src/Scope.cpp | 5 +- Analysis/src/Substitution.cpp | 1 - Analysis/src/TxnLog.cpp | 56 +++++- Analysis/src/TypeInfer.cpp | 113 ++++++------ Analysis/src/TypePack.cpp | 21 +++ Analysis/src/TypeVar.cpp | 21 +++ Ast/src/Parser.cpp | 22 ++- Compiler/src/BytecodeBuilder.cpp | 10 +- Compiler/src/Compiler.cpp | 195 +++++--------------- VM/src/lapi.cpp | 106 +++++------ VM/src/ldo.cpp | 25 ++- VM/src/lvmexecute.cpp | 19 +- tests/Autocomplete.test.cpp | 37 ++-- tests/Compiler.test.cpp | 13 -- tests/Conformance.test.cpp | 59 +++++- tests/Module.test.cpp | 20 ++ tests/Parser.test.cpp | 17 ++ tests/TypeInfer.aliases.test.cpp | 6 - tests/TypeInfer.loops.test.cpp | 8 - tests/TypeInfer.refinements.test.cpp | 32 +++- tests/TypeInfer.singletons.test.cpp | 2 - tests/TypePack.test.cpp | 16 ++ tests/TypeVar.test.cpp | 20 ++ tests/conformance/apicalls.lua | 11 ++ tests/conformance/pcall.lua | 17 ++ 33 files changed, 536 insertions(+), 469 deletions(-) diff --git a/Analysis/include/Luau/TypePack.h b/Analysis/include/Luau/TypePack.h index bbc65f94..c1de242f 100644 --- a/Analysis/include/Luau/TypePack.h +++ b/Analysis/include/Luau/TypePack.h @@ -48,13 +48,24 @@ struct TypePackVar explicit TypePackVar(const TypePackVariant& ty); explicit TypePackVar(TypePackVariant&& ty); TypePackVar(TypePackVariant&& ty, bool persistent); + bool operator==(const TypePackVar& rhs) const; + TypePackVar& operator=(TypePackVariant&& tp); + TypePackVar& operator=(const TypePackVar& rhs); + + // Re-assignes the content of the pack, but doesn't change the owning arena and can't make pack persistent. + void reassign(const TypePackVar& rhs) + { + ty = rhs.ty; + } + TypePackVariant ty; + bool persistent = false; - // Pointer to the type arena that allocated this type. + // Pointer to the type arena that allocated this pack. TypeArena* owningArena = nullptr; }; diff --git a/Analysis/include/Luau/TypeVar.h b/Analysis/include/Luau/TypeVar.h index b3c455cf..b59e7c64 100644 --- a/Analysis/include/Luau/TypeVar.h +++ b/Analysis/include/Luau/TypeVar.h @@ -334,7 +334,6 @@ struct TableTypeVar // We need to know which is which when we stringify types. std::optional syntheticName; - std::map methodDefinitionLocations; // TODO: Remove with FFlag::LuauNoMethodLocations std::vector instantiatedTypeParams; std::vector instantiatedTypePackParams; ModuleName definitionModuleName; @@ -465,6 +464,14 @@ struct TypeVar final { } + // Re-assignes the content of the type, but doesn't change the owning arena and can't make type persistent. + void reassign(const TypeVar& rhs) + { + ty = rhs.ty; + normal = rhs.normal; + documentationSymbol = rhs.documentationSymbol; + } + TypeVariant ty; // Kludge: A persistent TypeVar is one that belongs to the global scope. @@ -486,6 +493,8 @@ struct TypeVar final TypeVar& operator=(const TypeVariant& rhs); TypeVar& operator=(TypeVariant&& rhs); + + TypeVar& operator=(const TypeVar& rhs); }; using SeenSet = std::set>; diff --git a/Analysis/src/Autocomplete.cpp b/Analysis/src/Autocomplete.cpp index b988ed35..a8319c59 100644 --- a/Analysis/src/Autocomplete.cpp +++ b/Analysis/src/Autocomplete.cpp @@ -14,8 +14,7 @@ #include LUAU_FASTFLAGVARIABLE(LuauIfElseExprFixCompletionIssue, false); -LUAU_FASTFLAGVARIABLE(LuauFixAutocompleteClassSecurityLevel, false); -LUAU_FASTFLAG(LuauSelfCallAutocompleteFix) +LUAU_FASTFLAG(LuauSelfCallAutocompleteFix2) static const std::unordered_set kStatementStartingKeywords = { "while", "if", "local", "repeat", "function", "do", "for", "return", "break", "continue", "type", "export"}; @@ -248,7 +247,7 @@ static TypeCorrectKind checkTypeCorrectKind(const Module& module, TypeArena* typ ty = follow(ty); auto canUnify = [&typeArena](TypeId subTy, TypeId superTy) { - LUAU_ASSERT(!FFlag::LuauSelfCallAutocompleteFix); + LUAU_ASSERT(!FFlag::LuauSelfCallAutocompleteFix2); InternalErrorReporter iceReporter; UnifierSharedState unifierState(&iceReporter); @@ -267,7 +266,7 @@ static TypeCorrectKind checkTypeCorrectKind(const Module& module, TypeArena* typ TypeId expectedType = follow(*typeAtPosition); auto checkFunctionType = [typeArena, &canUnify, &expectedType](const FunctionTypeVar* ftv) { - if (FFlag::LuauSelfCallAutocompleteFix) + if (FFlag::LuauSelfCallAutocompleteFix2) { if (std::optional firstRetTy = first(ftv->retType)) return checkTypeMatch(typeArena, *firstRetTy, expectedType); @@ -308,7 +307,7 @@ static TypeCorrectKind checkTypeCorrectKind(const Module& module, TypeArena* typ } } - if (FFlag::LuauSelfCallAutocompleteFix) + if (FFlag::LuauSelfCallAutocompleteFix2) return checkTypeMatch(typeArena, ty, expectedType) ? TypeCorrectKind::Correct : TypeCorrectKind::None; else return canUnify(ty, expectedType) ? TypeCorrectKind::Correct : TypeCorrectKind::None; @@ -325,7 +324,7 @@ static void autocompleteProps(const Module& module, TypeArena* typeArena, TypeId const std::vector& nodes, AutocompleteEntryMap& result, std::unordered_set& seen, std::optional containingClass = std::nullopt) { - if (FFlag::LuauSelfCallAutocompleteFix) + if (FFlag::LuauSelfCallAutocompleteFix2) rootTy = follow(rootTy); ty = follow(ty); @@ -335,7 +334,7 @@ static void autocompleteProps(const Module& module, TypeArena* typeArena, TypeId seen.insert(ty); auto isWrongIndexer_DEPRECATED = [indexType, useStrictFunctionIndexers = !!get(ty)](Luau::TypeId type) { - LUAU_ASSERT(!FFlag::LuauSelfCallAutocompleteFix); + LUAU_ASSERT(!FFlag::LuauSelfCallAutocompleteFix2); if (indexType == PropIndexType::Key) return false; @@ -368,7 +367,7 @@ static void autocompleteProps(const Module& module, TypeArena* typeArena, TypeId } }; auto isWrongIndexer = [typeArena, rootTy, indexType](Luau::TypeId type) { - LUAU_ASSERT(FFlag::LuauSelfCallAutocompleteFix); + LUAU_ASSERT(FFlag::LuauSelfCallAutocompleteFix2); if (indexType == PropIndexType::Key) return false; @@ -382,10 +381,15 @@ static void autocompleteProps(const Module& module, TypeArena* typeArena, TypeId return calledWithSelf == ftv->hasSelf; } - if (std::optional firstArgTy = first(ftv->argTypes)) + // If a call is made with ':', it is invalid if a function has incompatible first argument or no arguments at all + // If a call is made with '.', but it was declared with 'self', it is considered invalid if first argument is compatible + if (calledWithSelf || ftv->hasSelf) { - if (checkTypeMatch(typeArena, rootTy, *firstArgTy)) - return calledWithSelf; + if (std::optional firstArgTy = first(ftv->argTypes)) + { + if (checkTypeMatch(typeArena, rootTy, *firstArgTy)) + return calledWithSelf; + } } return !calledWithSelf; @@ -427,7 +431,7 @@ static void autocompleteProps(const Module& module, TypeArena* typeArena, TypeId AutocompleteEntryKind::Property, type, prop.deprecated, - FFlag::LuauSelfCallAutocompleteFix ? isWrongIndexer(type) : isWrongIndexer_DEPRECATED(type), + FFlag::LuauSelfCallAutocompleteFix2 ? isWrongIndexer(type) : isWrongIndexer_DEPRECATED(type), typeCorrect, containingClass, &prop, @@ -462,8 +466,7 @@ static void autocompleteProps(const Module& module, TypeArena* typeArena, TypeId containingClass = containingClass.value_or(cls); fillProps(cls->props); if (cls->parent) - autocompleteProps(module, typeArena, rootTy, *cls->parent, indexType, nodes, result, seen, - FFlag::LuauFixAutocompleteClassSecurityLevel ? containingClass : cls); + autocompleteProps(module, typeArena, rootTy, *cls->parent, indexType, nodes, result, seen, containingClass); } else if (auto tbl = get(ty)) fillProps(tbl->props); @@ -471,7 +474,7 @@ static void autocompleteProps(const Module& module, TypeArena* typeArena, TypeId { autocompleteProps(module, typeArena, rootTy, mt->table, indexType, nodes, result, seen); - if (FFlag::LuauSelfCallAutocompleteFix) + if (FFlag::LuauSelfCallAutocompleteFix2) { if (auto mtable = get(mt->metatable)) fillMetatableProps(mtable); @@ -537,7 +540,7 @@ static void autocompleteProps(const Module& module, TypeArena* typeArena, TypeId AutocompleteEntryMap inner; std::unordered_set innerSeen; - if (!FFlag::LuauSelfCallAutocompleteFix) + if (!FFlag::LuauSelfCallAutocompleteFix2) innerSeen = seen; if (isNil(*iter)) @@ -563,7 +566,7 @@ static void autocompleteProps(const Module& module, TypeArena* typeArena, TypeId ++iter; } } - else if (auto pt = get(ty); pt && FFlag::LuauSelfCallAutocompleteFix) + else if (auto pt = get(ty); pt && FFlag::LuauSelfCallAutocompleteFix2) { if (pt->metatable) { @@ -571,7 +574,7 @@ static void autocompleteProps(const Module& module, TypeArena* typeArena, TypeId fillMetatableProps(mtable); } } - else if (FFlag::LuauSelfCallAutocompleteFix && get(get(ty))) + else if (FFlag::LuauSelfCallAutocompleteFix2 && get(get(ty))) { autocompleteProps(module, typeArena, rootTy, getSingletonTypes().stringType, indexType, nodes, result, seen); } @@ -1501,7 +1504,7 @@ static AutocompleteResult autocomplete(const SourceModule& sourceModule, const M TypeId ty = follow(*it); PropIndexType indexType = indexName->op == ':' ? PropIndexType::Colon : PropIndexType::Point; - if (!FFlag::LuauSelfCallAutocompleteFix && isString(ty)) + if (!FFlag::LuauSelfCallAutocompleteFix2 && isString(ty)) return {autocompleteProps(*module, typeArena, typeChecker.globalScope->bindings[AstName{"string"}].typeId, indexType, finder.ancestry), finder.ancestry}; else diff --git a/Analysis/src/BuiltinDefinitions.cpp b/Analysis/src/BuiltinDefinitions.cpp index 5ed6de67..98737b43 100644 --- a/Analysis/src/BuiltinDefinitions.cpp +++ b/Analysis/src/BuiltinDefinitions.cpp @@ -179,44 +179,13 @@ void registerBuiltinTypes(TypeChecker& typeChecker) LUAU_ASSERT(!typeChecker.globalTypes.typeVars.isFrozen()); LUAU_ASSERT(!typeChecker.globalTypes.typePacks.isFrozen()); - TypeId numberType = typeChecker.numberType; - TypeId booleanType = typeChecker.booleanType; TypeId nilType = typeChecker.nilType; TypeArena& arena = typeChecker.globalTypes; - TypePackId oneNumberPack = arena.addTypePack({numberType}); - TypePackId oneBooleanPack = arena.addTypePack({booleanType}); - - TypePackId numberVariadicList = arena.addTypePack(TypePackVar{VariadicTypePack{numberType}}); - TypePackId listOfAtLeastOneNumber = arena.addTypePack(TypePack{{numberType}, numberVariadicList}); - - TypeId listOfAtLeastOneNumberToNumberType = arena.addType(FunctionTypeVar{ - listOfAtLeastOneNumber, - oneNumberPack, - }); - - TypeId listOfAtLeastZeroNumbersToNumberType = arena.addType(FunctionTypeVar{numberVariadicList, oneNumberPack}); - LoadDefinitionFileResult loadResult = Luau::loadDefinitionFile(typeChecker, typeChecker.globalScope, getBuiltinDefinitionSource(), "@luau"); LUAU_ASSERT(loadResult.success); - TypeId mathLibType = getGlobalBinding(typeChecker, "math"); - if (TableTypeVar* ttv = getMutable(mathLibType)) - { - ttv->props["min"] = makeProperty(listOfAtLeastOneNumberToNumberType, "@luau/global/math.min"); - ttv->props["max"] = makeProperty(listOfAtLeastOneNumberToNumberType, "@luau/global/math.max"); - } - - TypeId bit32LibType = getGlobalBinding(typeChecker, "bit32"); - if (TableTypeVar* ttv = getMutable(bit32LibType)) - { - ttv->props["band"] = makeProperty(listOfAtLeastZeroNumbersToNumberType, "@luau/global/bit32.band"); - ttv->props["bor"] = makeProperty(listOfAtLeastZeroNumbersToNumberType, "@luau/global/bit32.bor"); - ttv->props["bxor"] = makeProperty(listOfAtLeastZeroNumbersToNumberType, "@luau/global/bit32.bxor"); - ttv->props["btest"] = makeProperty(arena.addType(FunctionTypeVar{listOfAtLeastOneNumber, oneBooleanPack}), "@luau/global/bit32.btest"); - } - TypeId genericK = arena.addType(GenericTypeVar{"K"}); TypeId genericV = arena.addType(GenericTypeVar{"V"}); TypeId mapOfKtoV = arena.addType(TableTypeVar{{}, TableIndexer(genericK, genericV), typeChecker.globalScope->level, TableState::Generic}); @@ -231,7 +200,7 @@ void registerBuiltinTypes(TypeChecker& typeChecker) addGlobalBinding(typeChecker, "string", it->second.type, "@luau"); - // next(t: Table, i: K | nil) -> (K, V) + // next(t: Table, i: K?) -> (K, V) TypePackId nextArgsTypePack = arena.addTypePack(TypePack{{mapOfKtoV, makeOption(typeChecker, arena, genericK)}}); addGlobalBinding(typeChecker, "next", arena.addType(FunctionTypeVar{{genericK, genericV}, {}, nextArgsTypePack, arena.addTypePack(TypePack{{genericK, genericV}})}), "@luau"); @@ -241,8 +210,7 @@ void registerBuiltinTypes(TypeChecker& typeChecker) TypeId pairsNext = arena.addType(FunctionTypeVar{nextArgsTypePack, arena.addTypePack(TypePack{{genericK, genericV}})}); TypePackId pairsReturnTypePack = arena.addTypePack(TypePack{{pairsNext, mapOfKtoV, nilType}}); - // NOTE we are missing 'i: K | nil' argument in the first return types' argument. - // pairs(t: Table) -> ((Table) -> (K, V), Table, nil) + // pairs(t: Table) -> ((Table, K?) -> (K, V), Table, nil) addGlobalBinding(typeChecker, "pairs", arena.addType(FunctionTypeVar{{genericK, genericV}, {}, pairsArgsTypePack, pairsReturnTypePack}), "@luau"); TypeId genericMT = arena.addType(GenericTypeVar{"MT"}); diff --git a/Analysis/src/Clone.cpp b/Analysis/src/Clone.cpp index 19e3383e..9180f309 100644 --- a/Analysis/src/Clone.cpp +++ b/Analysis/src/Clone.cpp @@ -9,7 +9,6 @@ LUAU_FASTFLAG(DebugLuauCopyBeforeNormalizing) LUAU_FASTINTVARIABLE(LuauTypeCloneRecursionLimit, 300) -LUAU_FASTFLAG(LuauNoMethodLocations) namespace Luau { @@ -241,8 +240,6 @@ void TypeCloner::operator()(const TableTypeVar& t) arg = clone(arg, dest, cloneState); ttv->definitionModuleName = t.definitionModuleName; - if (!FFlag::LuauNoMethodLocations) - ttv->methodDefinitionLocations = t.methodDefinitionLocations; ttv->tags = t.tags; } @@ -406,8 +403,6 @@ TypeId shallowClone(TypeId ty, TypeArena& dest, const TxnLog* log) { LUAU_ASSERT(!ttv->boundTo); TableTypeVar clone = TableTypeVar{ttv->props, ttv->indexer, ttv->level, ttv->state}; - if (!FFlag::LuauNoMethodLocations) - clone.methodDefinitionLocations = ttv->methodDefinitionLocations; clone.definitionModuleName = ttv->definitionModuleName; clone.name = ttv->name; clone.syntheticName = ttv->syntheticName; diff --git a/Analysis/src/EmbeddedBuiltinDefinitions.cpp b/Analysis/src/EmbeddedBuiltinDefinitions.cpp index f184b74e..2407e3ef 100644 --- a/Analysis/src/EmbeddedBuiltinDefinitions.cpp +++ b/Analysis/src/EmbeddedBuiltinDefinitions.cpp @@ -7,7 +7,10 @@ namespace Luau static const std::string kBuiltinDefinitionLuaSrc = R"BUILTIN_SRC( declare bit32: { - -- band, bor, bxor, and btest are declared in C++ + band: (...number) -> number, + bor: (...number) -> number, + bxor: (...number) -> number, + btest: (number, ...number) -> boolean, rrotate: (number, number) -> number, lrotate: (number, number) -> number, lshift: (number, number) -> number, @@ -50,7 +53,8 @@ declare math: { asin: (number) -> number, atan2: (number, number) -> number, - -- min and max are declared in C++. + min: (number, ...number) -> number, + max: (number, ...number) -> number, pi: number, huge: number, diff --git a/Analysis/src/Instantiation.cpp b/Analysis/src/Instantiation.cpp index 4a12027d..f145a511 100644 --- a/Analysis/src/Instantiation.cpp +++ b/Analysis/src/Instantiation.cpp @@ -4,8 +4,6 @@ #include "Luau/TxnLog.h" #include "Luau/TypeArena.h" -LUAU_FASTFLAG(LuauNoMethodLocations) - namespace Luau { @@ -110,8 +108,6 @@ TypeId ReplaceGenerics::clean(TypeId ty) if (const TableTypeVar* ttv = log->getMutable(ty)) { TableTypeVar clone = TableTypeVar{ttv->props, ttv->indexer, level, TableState::Free}; - if (!FFlag::LuauNoMethodLocations) - clone.methodDefinitionLocations = ttv->methodDefinitionLocations; clone.definitionModuleName = ttv->definitionModuleName; return addType(std::move(clone)); } diff --git a/Analysis/src/Quantify.cpp b/Analysis/src/Quantify.cpp index 8f2cc8e3..21775373 100644 --- a/Analysis/src/Quantify.cpp +++ b/Analysis/src/Quantify.cpp @@ -32,41 +32,6 @@ struct Quantifier final : TypeVarOnceVisitor LUAU_ASSERT(FFlag::DebugLuauDeferredConstraintResolution); } - void cycle(TypeId) override {} - void cycle(TypePackId) override {} - - bool operator()(TypeId ty, const FreeTypeVar& ftv) - { - return visit(ty, ftv); - } - - template - bool operator()(TypeId ty, const T& t) - { - return true; - } - - template - bool operator()(TypePackId, const T&) - { - return true; - } - - bool operator()(TypeId ty, const ConstrainedTypeVar&) - { - return true; - } - - bool operator()(TypeId ty, const TableTypeVar& ttv) - { - return visit(ty, ttv); - } - - bool operator()(TypePackId tp, const FreeTypePack& ftp) - { - return visit(tp, ftp); - } - /// @return true if outer encloses inner bool subsumes(Scope2* outer, Scope2* inner) { diff --git a/Analysis/src/Scope.cpp b/Analysis/src/Scope.cpp index 0a362a5e..011e28d4 100644 --- a/Analysis/src/Scope.cpp +++ b/Analysis/src/Scope.cpp @@ -2,8 +2,6 @@ #include "Luau/Scope.h" -LUAU_FASTFLAG(LuauTwoPassAliasDefinitionFix); - namespace Luau { @@ -19,8 +17,7 @@ Scope::Scope(const ScopePtr& parent, int subLevel) , returnType(parent->returnType) , level(parent->level.incr()) { - if (FFlag::LuauTwoPassAliasDefinitionFix) - level = level.incr(); + level = level.incr(); level.subLevel = subLevel; } diff --git a/Analysis/src/Substitution.cpp b/Analysis/src/Substitution.cpp index 50c516db..5a22deeb 100644 --- a/Analysis/src/Substitution.cpp +++ b/Analysis/src/Substitution.cpp @@ -10,7 +10,6 @@ LUAU_FASTFLAG(LuauLowerBoundsCalculation) LUAU_FASTINTVARIABLE(LuauTarjanChildLimit, 10000) -LUAU_FASTFLAG(LuauNoMethodLocations) namespace Luau { diff --git a/Analysis/src/TxnLog.cpp b/Analysis/src/TxnLog.cpp index e45c0cbd..4c6d54e0 100644 --- a/Analysis/src/TxnLog.cpp +++ b/Analysis/src/TxnLog.cpp @@ -7,6 +7,8 @@ #include #include +LUAU_FASTFLAG(LuauNonCopyableTypeVarFields) + namespace Luau { @@ -80,18 +82,32 @@ void TxnLog::commit() { for (auto& [ty, rep] : typeVarChanges) { - TypeArena* owningArena = ty->owningArena; - TypeVar* mtv = asMutable(ty); - *mtv = rep.get()->pending; - mtv->owningArena = owningArena; + if (FFlag::LuauNonCopyableTypeVarFields) + { + asMutable(ty)->reassign(rep.get()->pending); + } + else + { + TypeArena* owningArena = ty->owningArena; + TypeVar* mtv = asMutable(ty); + *mtv = rep.get()->pending; + mtv->owningArena = owningArena; + } } for (auto& [tp, rep] : typePackChanges) { - TypeArena* owningArena = tp->owningArena; - TypePackVar* mpv = asMutable(tp); - *mpv = rep.get()->pending; - mpv->owningArena = owningArena; + if (FFlag::LuauNonCopyableTypeVarFields) + { + asMutable(tp)->reassign(rep.get()->pending); + } + else + { + TypeArena* owningArena = tp->owningArena; + TypePackVar* mpv = asMutable(tp); + *mpv = rep.get()->pending; + mpv->owningArena = owningArena; + } } clear(); @@ -178,8 +194,13 @@ PendingType* TxnLog::queue(TypeId ty) // about this type, we don't want to mutate the parent's state. auto& pending = typeVarChanges[ty]; if (!pending) + { pending = std::make_unique(*ty); + if (FFlag::LuauNonCopyableTypeVarFields) + pending->pending.owningArena = nullptr; + } + return pending.get(); } @@ -191,8 +212,13 @@ PendingTypePack* TxnLog::queue(TypePackId tp) // about this type, we don't want to mutate the parent's state. auto& pending = typePackChanges[tp]; if (!pending) + { pending = std::make_unique(*tp); + if (FFlag::LuauNonCopyableTypeVarFields) + pending->pending.owningArena = nullptr; + } + return pending.get(); } @@ -229,14 +255,24 @@ PendingTypePack* TxnLog::pending(TypePackId tp) const PendingType* TxnLog::replace(TypeId ty, TypeVar replacement) { PendingType* newTy = queue(ty); - newTy->pending = replacement; + + if (FFlag::LuauNonCopyableTypeVarFields) + newTy->pending.reassign(replacement); + else + newTy->pending = replacement; + return newTy; } PendingTypePack* TxnLog::replace(TypePackId tp, TypePackVar replacement) { PendingTypePack* newTp = queue(tp); - newTp->pending = replacement; + + if (FFlag::LuauNonCopyableTypeVarFields) + newTp->pending.reassign(replacement); + else + newTp->pending = replacement; + return newTp; } diff --git a/Analysis/src/TypeInfer.cpp b/Analysis/src/TypeInfer.cpp index 4931bc59..447cd029 100644 --- a/Analysis/src/TypeInfer.cpp +++ b/Analysis/src/TypeInfer.cpp @@ -33,21 +33,20 @@ LUAU_FASTFLAG(LuauAutocompleteDynamicLimits) LUAU_FASTFLAGVARIABLE(LuauExpectedPropTypeFromIndexer, false) LUAU_FASTFLAGVARIABLE(LuauLowerBoundsCalculation, false) LUAU_FASTFLAGVARIABLE(DebugLuauFreezeDuringUnification, false) -LUAU_FASTFLAGVARIABLE(LuauSelfCallAutocompleteFix, false) +LUAU_FASTFLAGVARIABLE(LuauSelfCallAutocompleteFix2, false) LUAU_FASTFLAGVARIABLE(LuauReduceUnionRecursion, false) LUAU_FASTFLAGVARIABLE(LuauOnlyMutateInstantiatedTables, false) LUAU_FASTFLAGVARIABLE(LuauUnsealedTableLiteral, false) -LUAU_FASTFLAGVARIABLE(LuauTwoPassAliasDefinitionFix, false) LUAU_FASTFLAGVARIABLE(LuauReturnAnyInsteadOfICE, false) // Eventually removed as false. LUAU_FASTFLAG(LuauNormalizeFlagIsConservative) LUAU_FASTFLAGVARIABLE(LuauReturnTypeInferenceInNonstrict, false) LUAU_FASTFLAGVARIABLE(LuauRecursionLimitException, false); LUAU_FASTFLAGVARIABLE(LuauApplyTypeFunctionFix, false); -LUAU_FASTFLAGVARIABLE(LuauTypecheckIter, false); LUAU_FASTFLAGVARIABLE(LuauSuccessTypingForEqualityOperations, false) -LUAU_FASTFLAGVARIABLE(LuauNoMethodLocations, false); LUAU_FASTFLAGVARIABLE(LuauAlwaysQuantify, false); LUAU_FASTFLAGVARIABLE(LuauReportErrorsOnIndexerKeyMismatch, false) +LUAU_FASTFLAGVARIABLE(LuauFalsyPredicateReturnsNilInstead, false) +LUAU_FASTFLAGVARIABLE(LuauNonCopyableTypeVarFields, false) namespace Luau { @@ -358,8 +357,7 @@ ModulePtr TypeChecker::checkWithoutRecursionCheck(const SourceModule& module, Mo unifierState.cachedUnifyError.clear(); unifierState.skipCacheForType.clear(); - if (FFlag::LuauTwoPassAliasDefinitionFix) - duplicateTypeAliases.clear(); + duplicateTypeAliases.clear(); return std::move(currentModule); } @@ -610,7 +608,7 @@ LUAU_NOINLINE void TypeChecker::checkBlockTypeAliases(const ScopePtr& scope, std { if (const auto& typealias = stat->as()) { - if (FFlag::LuauTwoPassAliasDefinitionFix && typealias->name == kParseNameError) + if (typealias->name == kParseNameError) continue; auto& bindings = typealias->exported ? scope->exportedTypeBindings : scope->privateTypeBindings; @@ -619,7 +617,16 @@ LUAU_NOINLINE void TypeChecker::checkBlockTypeAliases(const ScopePtr& scope, std TypeId type = bindings[name].type; if (get(follow(type))) { - *asMutable(type) = *errorRecoveryType(anyType); + if (FFlag::LuauNonCopyableTypeVarFields) + { + TypeVar* mty = asMutable(follow(type)); + mty->reassign(*errorRecoveryType(anyType)); + } + else + { + *asMutable(type) = *errorRecoveryType(anyType); + } + reportError(TypeError{typealias->location, OccursCheckFailed{}}); } } @@ -1131,45 +1138,43 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatForIn& forin) iterTy = instantiate(scope, checkExpr(scope, *firstValue).type, firstValue->location); } - if (FFlag::LuauTypecheckIter) + if (std::optional iterMM = findMetatableEntry(iterTy, "__iter", firstValue->location)) { - if (std::optional iterMM = findMetatableEntry(iterTy, "__iter", firstValue->location)) + // if __iter metamethod is present, it will be called and the results are going to be called as if they are functions + // TODO: this needs to typecheck all returned values by __iter as if they were for loop arguments + // the structure of the function makes it difficult to do this especially since we don't have actual expressions, only types + for (TypeId var : varTypes) + unify(anyType, var, forin.location); + + return check(loopScope, *forin.body); + } + + if (const TableTypeVar* iterTable = get(iterTy)) + { + // TODO: note that this doesn't cleanly handle iteration over mixed tables and tables without an indexer + // this behavior is more or less consistent with what we do for pairs(), but really both are pretty wrong and need revisiting + if (iterTable->indexer) { - // if __iter metamethod is present, it will be called and the results are going to be called as if they are functions - // TODO: this needs to typecheck all returned values by __iter as if they were for loop arguments - // the structure of the function makes it difficult to do this especially since we don't have actual expressions, only types + if (varTypes.size() > 0) + unify(iterTable->indexer->indexType, varTypes[0], forin.location); + + if (varTypes.size() > 1) + unify(iterTable->indexer->indexResultType, varTypes[1], forin.location); + + for (size_t i = 2; i < varTypes.size(); ++i) + unify(nilType, varTypes[i], forin.location); + } + else + { + TypeId varTy = errorRecoveryType(loopScope); + for (TypeId var : varTypes) - unify(anyType, var, forin.location); + unify(varTy, var, forin.location); - return check(loopScope, *forin.body); + reportError(firstValue->location, GenericError{"Cannot iterate over a table without indexer"}); } - else if (const TableTypeVar* iterTable = get(iterTy)) - { - // TODO: note that this doesn't cleanly handle iteration over mixed tables and tables without an indexer - // this behavior is more or less consistent with what we do for pairs(), but really both are pretty wrong and need revisiting - if (iterTable->indexer) - { - if (varTypes.size() > 0) - unify(iterTable->indexer->indexType, varTypes[0], forin.location); - if (varTypes.size() > 1) - unify(iterTable->indexer->indexResultType, varTypes[1], forin.location); - - for (size_t i = 2; i < varTypes.size(); ++i) - unify(nilType, varTypes[i], forin.location); - } - else - { - TypeId varTy = errorRecoveryType(loopScope); - - for (TypeId var : varTypes) - unify(varTy, var, forin.location); - - reportError(firstValue->location, GenericError{"Cannot iterate over a table without indexer"}); - } - - return check(loopScope, *forin.body); - } + return check(loopScope, *forin.body); } const FunctionTypeVar* iterFunc = get(iterTy); @@ -1334,7 +1339,7 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatTypeAlias& typealias Name name = typealias.name.value; // If the alias is missing a name, we can't do anything with it. Ignore it. - if (FFlag::LuauTwoPassAliasDefinitionFix && name == kParseNameError) + if (name == kParseNameError) return; std::optional binding; @@ -1353,8 +1358,7 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatTypeAlias& typealias reportError(TypeError{typealias.location, DuplicateTypeDefinition{name, location}}); bindingsMap[name] = TypeFun{binding->typeParams, binding->typePackParams, errorRecoveryType(anyType)}; - if (FFlag::LuauTwoPassAliasDefinitionFix) - duplicateTypeAliases.insert({typealias.exported, name}); + duplicateTypeAliases.insert({typealias.exported, name}); } else { @@ -1378,7 +1382,7 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatTypeAlias& typealias { // If the first pass failed (this should mean a duplicate definition), the second pass isn't going to be // interesting. - if (FFlag::LuauTwoPassAliasDefinitionFix && duplicateTypeAliases.find({typealias.exported, name})) + if (duplicateTypeAliases.find({typealias.exported, name})) return; if (!binding) @@ -1422,9 +1426,6 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatTypeAlias& typealias { // This is a shallow clone, original recursive links to self are not updated TableTypeVar clone = TableTypeVar{ttv->props, ttv->indexer, ttv->level, ttv->state}; - - if (!FFlag::LuauNoMethodLocations) - clone.methodDefinitionLocations = ttv->methodDefinitionLocations; clone.definitionModuleName = ttv->definitionModuleName; clone.name = name; @@ -1462,9 +1463,8 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatTypeAlias& typealias } TypeId& bindingType = bindingsMap[name].type; - bool ok = unify(ty, bindingType, typealias.location); - if (FFlag::LuauTwoPassAliasDefinitionFix && ok) + if (unify(ty, bindingType, typealias.location)) bindingType = ty; if (FFlag::LuauLowerBoundsCalculation) @@ -1532,7 +1532,7 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatDeclareClass& declar ftv->argNames.insert(ftv->argNames.begin(), FunctionArgument{"self", {}}); ftv->argTypes = addTypePack(TypePack{{classTy}, ftv->argTypes}); - if (FFlag::LuauSelfCallAutocompleteFix) + if (FFlag::LuauSelfCallAutocompleteFix2) ftv->hasSelf = true; } } @@ -3099,8 +3099,6 @@ TypeId TypeChecker::checkFunctionName(const ScopePtr& scope, AstExpr& funName, T property.type = freshTy(); property.location = indexName->indexLocation; - if (!FFlag::LuauNoMethodLocations) - ttv->methodDefinitionLocations[name] = funName.location; return property.type; } else if (funName.is()) @@ -4393,8 +4391,6 @@ TypeId Anyification::clean(TypeId ty) if (const TableTypeVar* ttv = log->getMutable(ty)) { TableTypeVar clone = TableTypeVar{ttv->props, ttv->indexer, ttv->level, TableState::Sealed}; - if (!FFlag::LuauNoMethodLocations) - clone.methodDefinitionLocations = ttv->methodDefinitionLocations; clone.definitionModuleName = ttv->definitionModuleName; clone.name = ttv->name; clone.syntheticName = ttv->syntheticName; @@ -4705,8 +4701,11 @@ TypeIdPredicate TypeChecker::mkTruthyPredicate(bool sense) if (isNil(ty)) return sense ? std::nullopt : std::optional(ty); - // at this point, anything else is kept if sense is true, or eliminated otherwise - return sense ? std::optional(ty) : std::nullopt; + // at this point, anything else is kept if sense is true, or replaced by nil + if (FFlag::LuauFalsyPredicateReturnsNilInstead) + return sense ? ty : nilType; + else + return sense ? std::optional(ty) : std::nullopt; }; } diff --git a/Analysis/src/TypePack.cpp b/Analysis/src/TypePack.cpp index 30503233..82451bd1 100644 --- a/Analysis/src/TypePack.cpp +++ b/Analysis/src/TypePack.cpp @@ -5,6 +5,8 @@ #include +LUAU_FASTFLAG(LuauNonCopyableTypeVarFields) + namespace Luau { @@ -36,6 +38,25 @@ TypePackVar& TypePackVar::operator=(TypePackVariant&& tp) return *this; } +TypePackVar& TypePackVar::operator=(const TypePackVar& rhs) +{ + if (FFlag::LuauNonCopyableTypeVarFields) + { + LUAU_ASSERT(owningArena == rhs.owningArena); + LUAU_ASSERT(!rhs.persistent); + + reassign(rhs); + } + else + { + ty = rhs.ty; + persistent = rhs.persistent; + owningArena = rhs.owningArena; + } + + return *this; +} + TypePackIterator::TypePackIterator(TypePackId typePack) : TypePackIterator(typePack, TxnLog::empty()) { diff --git a/Analysis/src/TypeVar.cpp b/Analysis/src/TypeVar.cpp index 12cbed91..33bfe254 100644 --- a/Analysis/src/TypeVar.cpp +++ b/Analysis/src/TypeVar.cpp @@ -24,6 +24,7 @@ LUAU_FASTINTVARIABLE(LuauTypeMaximumStringifierLength, 500) LUAU_FASTINTVARIABLE(LuauTableTypeMaximumStringifierLength, 0) LUAU_FASTINT(LuauTypeInferRecursionLimit) LUAU_FASTFLAG(LuauSubtypingAddOptPropsToUnsealedTables) +LUAU_FASTFLAG(LuauNonCopyableTypeVarFields) namespace Luau { @@ -644,6 +645,26 @@ TypeVar& TypeVar::operator=(TypeVariant&& rhs) return *this; } +TypeVar& TypeVar::operator=(const TypeVar& rhs) +{ + if (FFlag::LuauNonCopyableTypeVarFields) + { + LUAU_ASSERT(owningArena == rhs.owningArena); + LUAU_ASSERT(!rhs.persistent); + + reassign(rhs); + } + else + { + ty = rhs.ty; + persistent = rhs.persistent; + normal = rhs.normal; + owningArena = rhs.owningArena; + } + + return *this; +} + TypeId makeFunction(TypeArena& arena, std::optional selfType, std::initializer_list generics, std::initializer_list genericPacks, std::initializer_list paramTypes, std::initializer_list paramNames, std::initializer_list retTypes); diff --git a/Ast/src/Parser.cpp b/Ast/src/Parser.cpp index eaf19914..95bce3ee 100644 --- a/Ast/src/Parser.cpp +++ b/Ast/src/Parser.cpp @@ -12,6 +12,7 @@ LUAU_FASTINTVARIABLE(LuauRecursionLimit, 1000) LUAU_FASTINTVARIABLE(LuauParseErrorLimit, 100) LUAU_FASTFLAGVARIABLE(LuauParserFunctionKeywordAsTypeHelp, false) +LUAU_FASTFLAGVARIABLE(LuauReturnTypeTokenConfusion, false) namespace Luau { @@ -1118,8 +1119,12 @@ AstTypePack* Parser::parseTypeList(TempVector& result, TempVector Parser::parseOptionalReturnTypeAnnotation() { - if (options.allowTypeAnnotations && lexer.current().type == ':') + if (options.allowTypeAnnotations && + (lexer.current().type == ':' || (FFlag::LuauReturnTypeTokenConfusion && lexer.current().type == Lexeme::SkinnyArrow))) { + if (FFlag::LuauReturnTypeTokenConfusion && lexer.current().type == Lexeme::SkinnyArrow) + report(lexer.current().location, "Function return type annotations are written after ':' instead of '->'"); + nextLexeme(); unsigned int oldRecursionCount = recursionCounter; @@ -1350,8 +1355,12 @@ AstTypeOrPack Parser::parseFunctionTypeAnnotation(bool allowPack) AstArray paramTypes = copy(params); + bool returnTypeIntroducer = + FFlag::LuauReturnTypeTokenConfusion ? lexer.current().type == Lexeme::SkinnyArrow || lexer.current().type == ':' : false; + // Not a function at all. Just a parenthesized type. Or maybe a type pack with a single element - if (params.size() == 1 && !varargAnnotation && monomorphic && lexer.current().type != Lexeme::SkinnyArrow) + if (params.size() == 1 && !varargAnnotation && monomorphic && + (FFlag::LuauReturnTypeTokenConfusion ? !returnTypeIntroducer : lexer.current().type != Lexeme::SkinnyArrow)) { if (allowPack) return {{}, allocator.alloc(begin.location, AstTypeList{paramTypes, nullptr})}; @@ -1359,7 +1368,7 @@ AstTypeOrPack Parser::parseFunctionTypeAnnotation(bool allowPack) return {params[0], {}}; } - if (lexer.current().type != Lexeme::SkinnyArrow && monomorphic && allowPack) + if ((FFlag::LuauReturnTypeTokenConfusion ? !returnTypeIntroducer : lexer.current().type != Lexeme::SkinnyArrow) && monomorphic && allowPack) return {{}, allocator.alloc(begin.location, AstTypeList{paramTypes, varargAnnotation})}; AstArray> paramNames = copy(names); @@ -1373,8 +1382,13 @@ AstType* Parser::parseFunctionTypeAnnotationTail(const Lexeme& begin, AstArray' instead of ':'"); + lexer.next(); + } // Users occasionally write '()' as the 'unit' type when they actually want to use 'nil', here we'll try to give a more specific error - if (lexer.current().type != Lexeme::SkinnyArrow && generics.size == 0 && genericPacks.size == 0 && params.size == 0) + else if (lexer.current().type != Lexeme::SkinnyArrow && generics.size == 0 && genericPacks.size == 0 && params.size == 0) { report(Location(begin.location, lexer.previousLocation()), "Expected '->' after '()' when parsing function type; did you mean 'nil'?"); diff --git a/Compiler/src/BytecodeBuilder.cpp b/Compiler/src/BytecodeBuilder.cpp index 3aa12d99..597b2f0a 100644 --- a/Compiler/src/BytecodeBuilder.cpp +++ b/Compiler/src/BytecodeBuilder.cpp @@ -6,8 +6,6 @@ #include #include -LUAU_FASTFLAG(LuauCompileNestedClosureO2) - namespace Luau { @@ -390,17 +388,15 @@ int32_t BytecodeBuilder::addConstantClosure(uint32_t fid) int16_t BytecodeBuilder::addChildFunction(uint32_t fid) { - if (FFlag::LuauCompileNestedClosureO2) - if (int16_t* cache = protoMap.find(fid)) - return *cache; + if (int16_t* cache = protoMap.find(fid)) + return *cache; uint32_t id = uint32_t(protos.size()); if (id >= kMaxClosureCount) return -1; - if (FFlag::LuauCompileNestedClosureO2) - protoMap[fid] = int16_t(id); + protoMap[fid] = int16_t(id); protos.push_back(fid); return int16_t(id); diff --git a/Compiler/src/Compiler.cpp b/Compiler/src/Compiler.cpp index eea56c60..7431cde4 100644 --- a/Compiler/src/Compiler.cpp +++ b/Compiler/src/Compiler.cpp @@ -16,7 +16,6 @@ #include #include -LUAU_FASTFLAGVARIABLE(LuauCompileIter, false) LUAU_FASTFLAGVARIABLE(LuauCompileIterNoPairs, false) LUAU_FASTINTVARIABLE(LuauCompileLoopUnrollThreshold, 25) @@ -26,8 +25,6 @@ LUAU_FASTINTVARIABLE(LuauCompileInlineThreshold, 25) LUAU_FASTINTVARIABLE(LuauCompileInlineThresholdMaxBoost, 300) LUAU_FASTINTVARIABLE(LuauCompileInlineDepth, 5) -LUAU_FASTFLAGVARIABLE(LuauCompileNestedClosureO2, false) - namespace Luau { @@ -172,30 +169,6 @@ struct Compiler return node->as(); } - bool canInlineFunctionBody(AstStat* stat) - { - if (FFlag::LuauCompileNestedClosureO2) - return true; // TODO: remove this function - - struct CanInlineVisitor : AstVisitor - { - bool result = true; - - bool visit(AstExprFunction* node) override - { - result = false; - - // short-circuit to avoid analyzing nested closure bodies - return false; - } - }; - - CanInlineVisitor canInline; - stat->visit(&canInline); - - return canInline.result; - } - uint32_t compileFunction(AstExprFunction* func) { LUAU_TIMETRACE_SCOPE("Compiler::compileFunction", "Compiler"); @@ -268,7 +241,7 @@ struct Compiler f.upvals = upvals; // record information for inlining - if (options.optimizationLevel >= 2 && !func->vararg && canInlineFunctionBody(func->body) && !getfenvUsed && !setfenvUsed) + if (options.optimizationLevel >= 2 && !func->vararg && !getfenvUsed && !setfenvUsed) { f.canInline = true; f.stackSize = stackSize; @@ -827,110 +800,62 @@ struct Compiler if (pid < 0) CompileError::raise(expr->location, "Exceeded closure limit; simplify the code to compile"); - if (FFlag::LuauCompileNestedClosureO2) - { - captures.clear(); - captures.reserve(f->upvals.size()); - - for (AstLocal* uv : f->upvals) - { - LUAU_ASSERT(uv->functionDepth < expr->functionDepth); - - if (int reg = getLocalReg(uv); reg >= 0) - { - // note: we can't check if uv is an upvalue in the current frame because inlining can migrate from upvalues to locals - Variable* ul = variables.find(uv); - bool immutable = !ul || !ul->written; - - captures.push_back({immutable ? LCT_VAL : LCT_REF, uint8_t(reg)}); - } - else if (const Constant* uc = locstants.find(uv); uc && uc->type != Constant::Type_Unknown) - { - // inlining can result in an upvalue capture of a constant, in which case we can't capture without a temporary register - uint8_t reg = allocReg(expr, 1); - compileExprConstant(expr, uc, reg); - - captures.push_back({LCT_VAL, reg}); - } - else - { - LUAU_ASSERT(uv->functionDepth < expr->functionDepth - 1); - - // get upvalue from parent frame - // note: this will add uv to the current upvalue list if necessary - uint8_t uid = getUpval(uv); - - captures.push_back({LCT_UPVAL, uid}); - } - } - - // Optimization: when closure has no upvalues, or upvalues are safe to share, instead of allocating it every time we can share closure - // objects (this breaks assumptions about function identity which can lead to setfenv not working as expected, so we disable this when it - // is used) - int16_t shared = -1; - - if (options.optimizationLevel >= 1 && shouldShareClosure(expr) && !setfenvUsed) - { - int32_t cid = bytecode.addConstantClosure(f->id); - - if (cid >= 0 && cid < 32768) - shared = int16_t(cid); - } - - if (shared >= 0) - bytecode.emitAD(LOP_DUPCLOSURE, target, shared); - else - bytecode.emitAD(LOP_NEWCLOSURE, target, pid); - - for (const Capture& c : captures) - bytecode.emitABC(LOP_CAPTURE, uint8_t(c.type), c.data, 0); - - return; - } - - bool shared = false; - - // Optimization: when closure has no upvalues, or upvalues are safe to share, instead of allocating it every time we can share closure - // objects (this breaks assumptions about function identity which can lead to setfenv not working as expected, so we disable this when it - // is used) - if (options.optimizationLevel >= 1 && shouldShareClosure(expr) && !setfenvUsed) - { - int32_t cid = bytecode.addConstantClosure(f->id); - - if (cid >= 0 && cid < 32768) - { - bytecode.emitAD(LOP_DUPCLOSURE, target, int16_t(cid)); - shared = true; - } - } - - if (!shared) - bytecode.emitAD(LOP_NEWCLOSURE, target, pid); + // we use a scratch vector to reduce allocations; this is safe since compileExprFunction is not reentrant + captures.clear(); + captures.reserve(f->upvals.size()); for (AstLocal* uv : f->upvals) { LUAU_ASSERT(uv->functionDepth < expr->functionDepth); - Variable* ul = variables.find(uv); - bool immutable = !ul || !ul->written; - - if (uv->functionDepth == expr->functionDepth - 1) + if (int reg = getLocalReg(uv); reg >= 0) { - // get local variable - int reg = getLocalReg(uv); - LUAU_ASSERT(reg >= 0); + // note: we can't check if uv is an upvalue in the current frame because inlining can migrate from upvalues to locals + Variable* ul = variables.find(uv); + bool immutable = !ul || !ul->written; - bytecode.emitABC(LOP_CAPTURE, uint8_t(immutable ? LCT_VAL : LCT_REF), uint8_t(reg), 0); + captures.push_back({immutable ? LCT_VAL : LCT_REF, uint8_t(reg)}); + } + else if (const Constant* uc = locstants.find(uv); uc && uc->type != Constant::Type_Unknown) + { + // inlining can result in an upvalue capture of a constant, in which case we can't capture without a temporary register + uint8_t reg = allocReg(expr, 1); + compileExprConstant(expr, uc, reg); + + captures.push_back({LCT_VAL, reg}); } else { + LUAU_ASSERT(uv->functionDepth < expr->functionDepth - 1); + // get upvalue from parent frame // note: this will add uv to the current upvalue list if necessary uint8_t uid = getUpval(uv); - bytecode.emitABC(LOP_CAPTURE, LCT_UPVAL, uid, 0); + captures.push_back({LCT_UPVAL, uid}); } } + + // Optimization: when closure has no upvalues, or upvalues are safe to share, instead of allocating it every time we can share closure + // objects (this breaks assumptions about function identity which can lead to setfenv not working as expected, so we disable this when it + // is used) + int16_t shared = -1; + + if (options.optimizationLevel >= 1 && shouldShareClosure(expr) && !setfenvUsed) + { + int32_t cid = bytecode.addConstantClosure(f->id); + + if (cid >= 0 && cid < 32768) + shared = int16_t(cid); + } + + if (shared >= 0) + bytecode.emitAD(LOP_DUPCLOSURE, target, shared); + else + bytecode.emitAD(LOP_NEWCLOSURE, target, pid); + + for (const Capture& c : captures) + bytecode.emitABC(LOP_CAPTURE, uint8_t(c.type), c.data, 0); } LuauOpcode getUnaryOp(AstExprUnary::Op op) @@ -2511,30 +2436,6 @@ struct Compiler pushLocal(stat->vars.data[i], uint8_t(vars + i)); } - bool canUnrollForBody(AstStatFor* stat) - { - if (FFlag::LuauCompileNestedClosureO2) - return true; // TODO: remove this function - - struct CanUnrollVisitor : AstVisitor - { - bool result = true; - - bool visit(AstExprFunction* node) override - { - result = false; - - // short-circuit to avoid analyzing nested closure bodies - return false; - } - }; - - CanUnrollVisitor canUnroll; - stat->body->visit(&canUnroll); - - return canUnroll.result; - } - bool tryCompileUnrolledFor(AstStatFor* stat, int thresholdBase, int thresholdMaxBoost) { Constant one = {Constant::Type_Number}; @@ -2560,12 +2461,6 @@ struct Compiler return false; } - if (!canUnrollForBody(stat)) - { - bytecode.addDebugRemark("loop unroll failed: unsupported loop body"); - return false; - } - if (Variable* lv = variables.find(stat->var); lv && lv->written) { bytecode.addDebugRemark("loop unroll failed: mutable loop variable"); @@ -2730,12 +2625,12 @@ struct Compiler uint8_t vars = allocReg(stat, std::max(unsigned(stat->vars.size), 2u)); LUAU_ASSERT(vars == regs + 3); - // Optimization: when we iterate through pairs/ipairs, we generate special bytecode that optimizes the traversal using internal iteration - // index These instructions dynamically check if generator is equal to next/inext and bail out They assume that the generator produces 2 - // variables, which is why we allocate at least 2 above (see vars assignment) - LuauOpcode skipOp = FFlag::LuauCompileIter ? LOP_FORGPREP : LOP_JUMP; + LuauOpcode skipOp = LOP_FORGPREP; LuauOpcode loopOp = LOP_FORGLOOP; + // Optimization: when we iterate via pairs/ipairs, we generate special bytecode that optimizes the traversal using internal iteration index + // These instructions dynamically check if generator is equal to next/inext and bail out + // They assume that the generator produces 2 variables, which is why we allocate at least 2 above (see vars assignment) if (options.optimizationLevel >= 1 && stat->vars.size <= 2) { if (stat->values.size == 1 && stat->values.data[0]->is()) diff --git a/VM/src/lapi.cpp b/VM/src/lapi.cpp index f86371da..3c3b7bd0 100644 --- a/VM/src/lapi.cpp +++ b/VM/src/lapi.cpp @@ -14,6 +14,26 @@ #include +/* + * This file contains most implementations of core Lua APIs from lua.h. + * + * These implementations should use api_check macros to verify that stack and type contracts hold; it's the callers + * responsibility to, for example, pass a valid table index to lua_rawgetfield. Generally errors should only be raised + * for conditions caller can't predict such as an out-of-memory error. + * + * The caller is expected to handle stack reservation (by using less than LUA_MINSTACK slots or by calling lua_checkstack). + * To ensure this is handled correctly, use api_incr_top(L) when pushing values to the stack. + * + * Functions that push any collectable objects to the stack *should* call luaC_checkthreadsleep. Failure to do this can result + * in stack references that point to dead objects since sleeping threads don't get rescanned. + * + * Functions that push newly created objects to the stack *should* call luaC_checkGC in addition to luaC_checkthreadsleep. + * Failure to do this can result in OOM since GC may never run. + * + * Note that luaC_checkGC may scan the thread and put it back to sleep; functions that call both before pushing objects must + * therefore call luaC_checkGC before luaC_checkthreadsleep to guarantee the object is pushed to an awake thread. + */ + const char* lua_ident = "$Lua: Lua 5.1.4 Copyright (C) 1994-2008 Lua.org, PUC-Rio $\n" "$Authors: R. Ierusalimschy, L. H. de Figueiredo & W. Celes $\n" "$URL: www.lua.org $\n"; @@ -221,15 +241,13 @@ void lua_insert(lua_State* L, int idx) void lua_replace(lua_State* L, int idx) { - /* explicit test for incompatible code */ - if (idx == LUA_ENVIRONINDEX && L->ci == L->base_ci) - luaG_runerror(L, "no calling environment"); api_checknelems(L, 1); luaC_checkthreadsleep(L); StkId o = index2addr(L, idx); api_checkvalidindex(L, o); if (idx == LUA_ENVIRONINDEX) { + api_check(L, L->ci != L->base_ci); Closure* func = curr_func(L); api_check(L, ttistable(L->top - 1)); func->env = hvalue(L->top - 1); @@ -443,9 +461,7 @@ const float* lua_tovector(lua_State* L, int idx) { StkId o = index2addr(L, idx); if (!ttisvector(o)) - { return NULL; - } return vvalue(o); } @@ -460,11 +476,6 @@ int lua_objlen(lua_State* L, int idx) return uvalue(o)->len; case LUA_TTABLE: return luaH_getn(hvalue(o)); - case LUA_TNUMBER: - { - int l = (luaV_tostring(L, o) ? tsvalue(o)->len : 0); - return l; - } default: return 0; } @@ -752,10 +763,9 @@ void lua_setsafeenv(lua_State* L, int objindex, int enabled) int lua_getmetatable(lua_State* L, int objindex) { - const TValue* obj; + luaC_checkthreadsleep(L); Table* mt = NULL; - int res; - obj = index2addr(L, objindex); + const TValue* obj = index2addr(L, objindex); switch (ttype(obj)) { case LUA_TTABLE: @@ -768,21 +778,18 @@ int lua_getmetatable(lua_State* L, int objindex) mt = L->global->mt[ttype(obj)]; break; } - if (mt == NULL) - res = 0; - else + if (mt) { sethvalue(L, L->top, mt); api_incr_top(L); - res = 1; } - return res; + return mt != NULL; } void lua_getfenv(lua_State* L, int idx) { - StkId o; - o = index2addr(L, idx); + luaC_checkthreadsleep(L); + StkId o = index2addr(L, idx); api_checkvalidindex(L, o); switch (ttype(o)) { @@ -806,9 +813,8 @@ void lua_getfenv(lua_State* L, int idx) void lua_settable(lua_State* L, int idx) { - StkId t; api_checknelems(L, 2); - t = index2addr(L, idx); + StkId t = index2addr(L, idx); api_checkvalidindex(L, t); luaV_settable(L, t, L->top - 2, L->top - 1); L->top -= 2; /* pop index and value */ @@ -817,22 +823,20 @@ void lua_settable(lua_State* L, int idx) void lua_setfield(lua_State* L, int idx, const char* k) { - StkId t; - TValue key; api_checknelems(L, 1); - t = index2addr(L, idx); + StkId t = index2addr(L, idx); api_checkvalidindex(L, t); + TValue key; setsvalue(L, &key, luaS_new(L, k)); luaV_settable(L, t, &key, L->top - 1); - L->top--; /* pop value */ + L->top--; return; } void lua_rawset(lua_State* L, int idx) { - StkId t; api_checknelems(L, 2); - t = index2addr(L, idx); + StkId t = index2addr(L, idx); api_check(L, ttistable(t)); if (hvalue(t)->readonly) luaG_runerror(L, "Attempt to modify a readonly table"); @@ -844,9 +848,8 @@ void lua_rawset(lua_State* L, int idx) void lua_rawseti(lua_State* L, int idx, int n) { - StkId o; api_checknelems(L, 1); - o = index2addr(L, idx); + StkId o = index2addr(L, idx); api_check(L, ttistable(o)); if (hvalue(o)->readonly) luaG_runerror(L, "Attempt to modify a readonly table"); @@ -858,14 +861,11 @@ void lua_rawseti(lua_State* L, int idx, int n) int lua_setmetatable(lua_State* L, int objindex) { - TValue* obj; - Table* mt; api_checknelems(L, 1); - obj = index2addr(L, objindex); + TValue* obj = index2addr(L, objindex); api_checkvalidindex(L, obj); - if (ttisnil(L->top - 1)) - mt = NULL; - else + Table* mt = NULL; + if (!ttisnil(L->top - 1)) { api_check(L, ttistable(L->top - 1)); mt = hvalue(L->top - 1); @@ -900,10 +900,9 @@ int lua_setmetatable(lua_State* L, int objindex) int lua_setfenv(lua_State* L, int idx) { - StkId o; int res = 1; api_checknelems(L, 1); - o = index2addr(L, idx); + StkId o = index2addr(L, idx); api_checkvalidindex(L, o); api_check(L, ttistable(L->top - 1)); switch (ttype(o)) @@ -970,24 +969,21 @@ static void f_call(lua_State* L, void* ud) int lua_pcall(lua_State* L, int nargs, int nresults, int errfunc) { - struct CallS c; - int status; - ptrdiff_t func; api_checknelems(L, nargs + 1); api_check(L, L->status == 0); checkresults(L, nargs, nresults); - if (errfunc == 0) - func = 0; - else + ptrdiff_t func = 0; + if (errfunc != 0) { StkId o = index2addr(L, errfunc); api_checkvalidindex(L, o); func = savestack(L, o); } + struct CallS c; c.func = L->top - (nargs + 1); /* function to be called */ c.nresults = nresults; - status = luaD_pcall(L, f_call, &c, savestack(L, c.func), func); + int status = luaD_pcall(L, f_call, &c, savestack(L, c.func), func); adjustresults(L, nresults); return status; @@ -1247,12 +1243,10 @@ const char* lua_getupvalue(lua_State* L, int funcindex, int n) const char* lua_setupvalue(lua_State* L, int funcindex, int n) { - const char* name; - TValue* val; - StkId fi; - fi = index2addr(L, funcindex); api_checknelems(L, 1); - name = aux_upvalue(fi, n, &val); + StkId fi = index2addr(L, funcindex); + TValue* val; + const char* name = aux_upvalue(fi, n, &val); if (name) { L->top--; @@ -1319,14 +1313,16 @@ void lua_setuserdatadtor(lua_State* L, int tag, void (*dtor)(lua_State*, void*)) void lua_clonefunction(lua_State* L, int idx) { + luaC_checkGC(L); + luaC_checkthreadsleep(L); StkId p = index2addr(L, idx); api_check(L, isLfunction(p)); - - luaC_checkthreadsleep(L); - Closure* cl = clvalue(p); - Closure* newcl = luaF_newLclosure(L, 0, L->gt, cl->l.p); - setclvalue(L, L->top - 1, newcl); + Closure* newcl = luaF_newLclosure(L, cl->nupvalues, L->gt, cl->l.p); + for (int i = 0; i < cl->nupvalues; ++i) + setobj2n(L, &newcl->l.uprefs[i], &cl->l.uprefs[i]); + setclvalue(L, L->top, newcl); + api_incr_top(L); } lua_Callbacks* lua_callbacks(lua_State* L) diff --git a/VM/src/ldo.cpp b/VM/src/ldo.cpp index a71fce52..0642cb6d 100644 --- a/VM/src/ldo.cpp +++ b/VM/src/ldo.cpp @@ -202,22 +202,29 @@ void luaD_growstack(lua_State* L, int n) CallInfo* luaD_growCI(lua_State* L) { - if (L->size_ci > LUAI_MAXCALLS) /* overflow while handling overflow? */ - luaD_throw(L, LUA_ERRERR); - else - { - luaD_reallocCI(L, 2 * L->size_ci); - if (L->size_ci > LUAI_MAXCALLS) - luaG_runerror(L, "stack overflow"); - } + /* allow extra stack space to handle stack overflow in xpcall */ + const int hardlimit = LUAI_MAXCALLS + (LUAI_MAXCALLS >> 3); + + if (L->size_ci >= hardlimit) + luaD_throw(L, LUA_ERRERR); /* error while handling stack error */ + + int request = L->size_ci * 2; + luaD_reallocCI(L, L->size_ci >= LUAI_MAXCALLS ? hardlimit : request < LUAI_MAXCALLS ? request : LUAI_MAXCALLS); + + if (L->size_ci > LUAI_MAXCALLS) + luaG_runerror(L, "stack overflow"); + return ++L->ci; } void luaD_checkCstack(lua_State* L) { + /* allow extra stack space to handle stack overflow in xpcall */ + const int hardlimit = LUAI_MAXCCALLS + (LUAI_MAXCCALLS >> 3); + if (L->nCcalls == LUAI_MAXCCALLS) luaG_runerror(L, "C stack overflow"); - else if (L->nCcalls >= (LUAI_MAXCCALLS + (LUAI_MAXCCALLS >> 3))) + else if (L->nCcalls >= hardlimit) luaD_throw(L, LUA_ERRERR); /* error while handling stack error */ } diff --git a/VM/src/lvmexecute.cpp b/VM/src/lvmexecute.cpp index f9fd6574..e0a96474 100644 --- a/VM/src/lvmexecute.cpp +++ b/VM/src/lvmexecute.cpp @@ -16,8 +16,6 @@ #include -LUAU_FASTFLAGVARIABLE(LuauIter, false) - // Disable c99-designator to avoid the warning in CGOTO dispatch table #ifdef __clang__ #if __has_warning("-Wc99-designator") @@ -2214,7 +2212,7 @@ static void luau_execute(lua_State* L) { /* will be called during FORGLOOP */ } - else if (FFlag::LuauIter) + else { Table* mt = ttistable(ra) ? hvalue(ra)->metatable : ttisuserdata(ra) ? uvalue(ra)->metatable : cast_to(Table*, NULL); @@ -2259,17 +2257,6 @@ static void luau_execute(lua_State* L) StkId ra = VM_REG(LUAU_INSN_A(insn)); uint32_t aux = *pc; - if (!FFlag::LuauIter) - { - bool stop; - VM_PROTECT(stop = luau_loopFORG(L, LUAU_INSN_A(insn), aux)); - - // note that we need to increment pc by 1 to exit the loop since we need to skip over aux - pc += stop ? 1 : LUAU_INSN_D(insn); - LUAU_ASSERT(unsigned(pc - cl->l.p->code) < unsigned(cl->l.p->sizecode)); - VM_NEXT(); - } - // fast-path: builtin table iteration if (ttisnil(ra) && ttistable(ra + 1) && ttislightuserdata(ra + 2)) { @@ -2362,7 +2349,7 @@ static void luau_execute(lua_State* L) /* ra+1 is already the table */ setpvalue(ra + 2, reinterpret_cast(uintptr_t(0))); } - else if (FFlag::LuauIter && !ttisfunction(ra)) + else if (!ttisfunction(ra)) { VM_PROTECT(luaG_typeerror(L, ra, "iterate over")); } @@ -2434,7 +2421,7 @@ static void luau_execute(lua_State* L) /* ra+1 is already the table */ setpvalue(ra + 2, reinterpret_cast(uintptr_t(0))); } - else if (FFlag::LuauIter && !ttisfunction(ra)) + else if (!ttisfunction(ra)) { VM_PROTECT(luaG_typeerror(L, ra, "iterate over")); } diff --git a/tests/Autocomplete.test.cpp b/tests/Autocomplete.test.cpp index cc5b31c9..dea1ab19 100644 --- a/tests/Autocomplete.test.cpp +++ b/tests/Autocomplete.test.cpp @@ -2764,8 +2764,6 @@ TEST_CASE_FIXTURE(ACBuiltinsFixture, "autocomplete_on_string_singletons") TEST_CASE_FIXTURE(ACFixture, "autocomplete_string_singletons") { - ScopedFastFlag sff{"LuauTwoPassAliasDefinitionFix", true}; - check(R"( type tag = "cat" | "dog" local function f(a: tag) end @@ -2838,8 +2836,6 @@ f(@1) TEST_CASE_FIXTURE(ACFixture, "autocomplete_string_singleton_escape") { - ScopedFastFlag sff{"LuauTwoPassAliasDefinitionFix", true}; - check(R"( type tag = "strange\t\"cat\"" | 'nice\t"dog"' local function f(x: tag) end @@ -2873,7 +2869,7 @@ local abc = b@1 TEST_CASE_FIXTURE(ACFixture, "no_incompatible_self_calls_on_class") { - ScopedFastFlag selfCallAutocompleteFix{"LuauSelfCallAutocompleteFix", true}; + ScopedFastFlag selfCallAutocompleteFix2{"LuauSelfCallAutocompleteFix2", true}; loadDefinition(R"( declare class Foo @@ -2913,7 +2909,7 @@ t.@1 TEST_CASE_FIXTURE(ACFixture, "no_incompatible_self_calls") { - ScopedFastFlag selfCallAutocompleteFix{"LuauSelfCallAutocompleteFix", true}; + ScopedFastFlag selfCallAutocompleteFix2{"LuauSelfCallAutocompleteFix2", true}; check(R"( local t = {} @@ -2929,7 +2925,7 @@ t:@1 TEST_CASE_FIXTURE(ACFixture, "no_incompatible_self_calls_2") { - ScopedFastFlag selfCallAutocompleteFix{"LuauSelfCallAutocompleteFix", true}; + ScopedFastFlag selfCallAutocompleteFix2{"LuauSelfCallAutocompleteFix2", true}; check(R"( local f: (() -> number) & ((number) -> number) = function(x: number?) return 2 end @@ -2961,7 +2957,7 @@ t:@1 TEST_CASE_FIXTURE(ACFixture, "string_prim_self_calls_are_fine") { - ScopedFastFlag selfCallAutocompleteFix{"LuauSelfCallAutocompleteFix", true}; + ScopedFastFlag selfCallAutocompleteFix2{"LuauSelfCallAutocompleteFix2", true}; check(R"( local s = "hello" @@ -2980,7 +2976,7 @@ s:@1 TEST_CASE_FIXTURE(ACFixture, "string_prim_non_self_calls_are_avoided") { - ScopedFastFlag selfCallAutocompleteFix{"LuauSelfCallAutocompleteFix", true}; + ScopedFastFlag selfCallAutocompleteFix2{"LuauSelfCallAutocompleteFix2", true}; check(R"( local s = "hello" @@ -2989,17 +2985,15 @@ s.@1 auto ac = autocomplete('1'); - REQUIRE(ac.entryMap.count("byte")); - CHECK(ac.entryMap["byte"].wrongIndexType == true); REQUIRE(ac.entryMap.count("char")); CHECK(ac.entryMap["char"].wrongIndexType == false); REQUIRE(ac.entryMap.count("sub")); CHECK(ac.entryMap["sub"].wrongIndexType == true); } -TEST_CASE_FIXTURE(ACBuiltinsFixture, "string_library_non_self_calls_are_fine") +TEST_CASE_FIXTURE(ACBuiltinsFixture, "library_non_self_calls_are_fine") { - ScopedFastFlag selfCallAutocompleteFix{"LuauSelfCallAutocompleteFix", true}; + ScopedFastFlag selfCallAutocompleteFix2{"LuauSelfCallAutocompleteFix2", true}; check(R"( string.@1 @@ -3013,11 +3007,24 @@ string.@1 CHECK(ac.entryMap["char"].wrongIndexType == false); REQUIRE(ac.entryMap.count("sub")); CHECK(ac.entryMap["sub"].wrongIndexType == false); + + check(R"( +table.@1 + )"); + + ac = autocomplete('1'); + + REQUIRE(ac.entryMap.count("remove")); + CHECK(ac.entryMap["remove"].wrongIndexType == false); + REQUIRE(ac.entryMap.count("getn")); + CHECK(ac.entryMap["getn"].wrongIndexType == false); + REQUIRE(ac.entryMap.count("insert")); + CHECK(ac.entryMap["insert"].wrongIndexType == false); } -TEST_CASE_FIXTURE(ACBuiltinsFixture, "string_library_self_calls_are_invalid") +TEST_CASE_FIXTURE(ACBuiltinsFixture, "library_self_calls_are_invalid") { - ScopedFastFlag selfCallAutocompleteFix{"LuauSelfCallAutocompleteFix", true}; + ScopedFastFlag selfCallAutocompleteFix2{"LuauSelfCallAutocompleteFix2", true}; check(R"( string:@1 diff --git a/tests/Compiler.test.cpp b/tests/Compiler.test.cpp index 20139650..6eee254e 100644 --- a/tests/Compiler.test.cpp +++ b/tests/Compiler.test.cpp @@ -261,7 +261,6 @@ L1: RETURN R0 0 TEST_CASE("ForBytecode") { - ScopedFastFlag sff("LuauCompileIter", true); ScopedFastFlag sff2("LuauCompileIterNoPairs", false); // basic for loop: variable directly refers to internal iteration index (R2) @@ -350,8 +349,6 @@ RETURN R0 0 TEST_CASE("ForBytecodeBuiltin") { - ScopedFastFlag sff("LuauCompileIter", true); - // we generally recognize builtins like pairs/ipairs and emit special opcodes CHECK_EQ("\n" + compileFunction0("for k,v in ipairs({}) do end"), R"( GETIMPORT R0 1 @@ -2323,8 +2320,6 @@ return result TEST_CASE("DebugLineInfoFor") { - ScopedFastFlag sff("LuauCompileIter", true); - Luau::BytecodeBuilder bcb; bcb.setDumpFlags(Luau::BytecodeBuilder::Dump_Code | Luau::BytecodeBuilder::Dump_Lines); Luau::compileOrThrow(bcb, R"( @@ -4355,8 +4350,6 @@ L1: RETURN R0 0 TEST_CASE("LoopUnrollControlFlow") { - ScopedFastFlag sff("LuauCompileNestedClosureO2", true); - ScopedFastInt sfis[] = { {"LuauCompileLoopUnrollThreshold", 50}, {"LuauCompileLoopUnrollThresholdMaxBoost", 300}, @@ -4475,8 +4468,6 @@ RETURN R0 0 TEST_CASE("LoopUnrollNestedClosure") { - ScopedFastFlag sff("LuauCompileNestedClosureO2", true); - // if the body has functions that refer to loop variables, we unroll the loop and use MOVE+CAPTURE for upvalues CHECK_EQ("\n" + compileFunction(R"( for i=1,2 do @@ -4756,8 +4747,6 @@ RETURN R1 1 TEST_CASE("InlineBasicProhibited") { - ScopedFastFlag sff("LuauCompileNestedClosureO2", true); - // we can't inline variadic functions CHECK_EQ("\n" + compileFunction(R"( local function foo(...) @@ -4833,8 +4822,6 @@ RETURN R1 1 TEST_CASE("InlineNestedClosures") { - ScopedFastFlag sff("LuauCompileNestedClosureO2", true); - // we can inline functions that contain/return functions CHECK_EQ("\n" + compileFunction(R"( local function foo(x) diff --git a/tests/Conformance.test.cpp b/tests/Conformance.test.cpp index f7f2b4ac..96a2775f 100644 --- a/tests/Conformance.test.cpp +++ b/tests/Conformance.test.cpp @@ -741,7 +741,7 @@ TEST_CASE("ApiTables") lua_pop(L, 1); } -TEST_CASE("ApiFunctionCalls") +TEST_CASE("ApiCalls") { StateRef globalState = runConformance("apicalls.lua"); lua_State* L = globalState.get(); @@ -790,6 +790,58 @@ TEST_CASE("ApiFunctionCalls") CHECK(lua_equal(L2, -1, -2) == 1); lua_pop(L2, 2); } + + // lua_clonefunction + fenv + { + lua_getfield(L, LUA_GLOBALSINDEX, "getpi"); + lua_call(L, 0, 1); + CHECK(lua_tonumber(L, -1) == 3.1415926); + lua_pop(L, 1); + + lua_getfield(L, LUA_GLOBALSINDEX, "getpi"); + + // clone & override env + lua_clonefunction(L, -1); + lua_newtable(L); + lua_pushnumber(L, 42); + lua_setfield(L, -2, "pi"); + lua_setfenv(L, -2); + + lua_call(L, 0, 1); + CHECK(lua_tonumber(L, -1) == 42); + lua_pop(L, 1); + + // this one calls original function again + lua_call(L, 0, 1); + CHECK(lua_tonumber(L, -1) == 3.1415926); + lua_pop(L, 1); + } + + // lua_clonefunction + upvalues + { + lua_getfield(L, LUA_GLOBALSINDEX, "incuv"); + lua_call(L, 0, 1); + CHECK(lua_tonumber(L, -1) == 1); + lua_pop(L, 1); + + lua_getfield(L, LUA_GLOBALSINDEX, "incuv"); + // two clones + lua_clonefunction(L, -1); + lua_clonefunction(L, -2); + + lua_call(L, 0, 1); + CHECK(lua_tonumber(L, -1) == 2); + lua_pop(L, 1); + + lua_call(L, 0, 1); + CHECK(lua_tonumber(L, -1) == 3); + lua_pop(L, 1); + + // this one calls original function again + lua_call(L, 0, 1); + CHECK(lua_tonumber(L, -1) == 4); + lua_pop(L, 1); + } } static bool endsWith(const std::string& str, const std::string& suffix) @@ -1113,11 +1165,6 @@ TEST_CASE("UserdataApi") TEST_CASE("Iter") { - ScopedFastFlag sffs[] = { - {"LuauCompileIter", true}, - {"LuauIter", true}, - }; - runConformance("iter.lua"); } diff --git a/tests/Module.test.cpp b/tests/Module.test.cpp index c7e18efd..89b13ab1 100644 --- a/tests/Module.test.cpp +++ b/tests/Module.test.cpp @@ -300,4 +300,24 @@ TEST_CASE_FIXTURE(Fixture, "clone_recursion_limit") CHECK_THROWS_AS(clone(table, dest, cloneState), RecursionLimitException); } +TEST_CASE_FIXTURE(Fixture, "any_persistance_does_not_leak") +{ + ScopedFastFlag luauNonCopyableTypeVarFields{"LuauNonCopyableTypeVarFields", true}; + + fileResolver.source["Module/A"] = R"( +export type A = B +type B = A + )"; + + FrontendOptions opts; + opts.retainFullTypeGraphs = false; + CheckResult result = frontend.check("Module/A", opts); + LUAU_REQUIRE_ERRORS(result); + + auto mod = frontend.moduleResolver.getModule("Module/A"); + auto it = mod->getModuleScope()->exportedTypeBindings.find("A"); + REQUIRE(it != mod->getModuleScope()->exportedTypeBindings.end()); + CHECK(toString(it->second.type) == "any"); +} + TEST_SUITE_END(); diff --git a/tests/Parser.test.cpp b/tests/Parser.test.cpp index 87b1263f..878023e3 100644 --- a/tests/Parser.test.cpp +++ b/tests/Parser.test.cpp @@ -2622,6 +2622,23 @@ type Z = { a: string | T..., b: number } REQUIRE_EQ(3, result.errors.size()); } +TEST_CASE_FIXTURE(Fixture, "recover_function_return_type_annotations") +{ + ScopedFastFlag sff{"LuauReturnTypeTokenConfusion", true}; + ParseResult result = tryParse(R"( +type Custom = { x: A, y: B, z: C } +type Packed = { x: (A...) -> () } +type F = (number): Custom +type G = Packed<(number): (string, number, boolean)> +local function f(x: number) -> Custom +end + )"); + REQUIRE_EQ(3, result.errors.size()); + CHECK_EQ(result.errors[0].getMessage(), "Return types in function type annotations are written after '->' instead of ':'"); + CHECK_EQ(result.errors[1].getMessage(), "Return types in function type annotations are written after '->' instead of ':'"); + CHECK_EQ(result.errors[2].getMessage(), "Function return type annotations are written after ':' instead of '->'"); +} + TEST_CASE_FIXTURE(Fixture, "error_message_for_using_function_as_type_annotation") { ScopedFastFlag sff{"LuauParserFunctionKeywordAsTypeHelp", true}; diff --git a/tests/TypeInfer.aliases.test.cpp b/tests/TypeInfer.aliases.test.cpp index 7562a4d7..86cc9701 100644 --- a/tests/TypeInfer.aliases.test.cpp +++ b/tests/TypeInfer.aliases.test.cpp @@ -615,8 +615,6 @@ TEST_CASE_FIXTURE(Fixture, "generic_typevars_are_not_considered_to_escape_their_ */ TEST_CASE_FIXTURE(Fixture, "forward_declared_alias_is_not_clobbered_by_prior_unification_with_any") { - ScopedFastFlag sff[] = {{"LuauTwoPassAliasDefinitionFix", true}}; - CheckResult result = check(R"( local function x() local y: FutureType = {}::any @@ -633,10 +631,6 @@ TEST_CASE_FIXTURE(Fixture, "forward_declared_alias_is_not_clobbered_by_prior_uni TEST_CASE_FIXTURE(Fixture, "forward_declared_alias_is_not_clobbered_by_prior_unification_with_any_2") { - ScopedFastFlag sff[] = { - {"LuauTwoPassAliasDefinitionFix", true}, - }; - CheckResult result = check(R"( local B = {} B.bar = 4 diff --git a/tests/TypeInfer.loops.test.cpp b/tests/TypeInfer.loops.test.cpp index 4444cd66..1c6fe1d8 100644 --- a/tests/TypeInfer.loops.test.cpp +++ b/tests/TypeInfer.loops.test.cpp @@ -486,8 +486,6 @@ TEST_CASE_FIXTURE(Fixture, "fuzz_fail_missing_instantitation_follow") TEST_CASE_FIXTURE(Fixture, "loop_iter_basic") { - ScopedFastFlag sff{"LuauTypecheckIter", true}; - CheckResult result = check(R"( local t: {string} = {} local key @@ -506,8 +504,6 @@ TEST_CASE_FIXTURE(Fixture, "loop_iter_basic") TEST_CASE_FIXTURE(Fixture, "loop_iter_trailing_nil") { - ScopedFastFlag sff{"LuauTypecheckIter", true}; - CheckResult result = check(R"( local t: {string} = {} local extra @@ -522,8 +518,6 @@ TEST_CASE_FIXTURE(Fixture, "loop_iter_trailing_nil") TEST_CASE_FIXTURE(Fixture, "loop_iter_no_indexer") { - ScopedFastFlag sff{"LuauTypecheckIter", true}; - CheckResult result = check(R"( local t = {} for k, v in t do @@ -539,8 +533,6 @@ TEST_CASE_FIXTURE(Fixture, "loop_iter_no_indexer") TEST_CASE_FIXTURE(BuiltinsFixture, "loop_iter_iter_metamethod") { - ScopedFastFlag sff{"LuauTypecheckIter", true}; - CheckResult result = check(R"( local t = {} setmetatable(t, { __iter = function(o) return next, o.children end }) diff --git a/tests/TypeInfer.refinements.test.cpp b/tests/TypeInfer.refinements.test.cpp index 6785f277..207b3cff 100644 --- a/tests/TypeInfer.refinements.test.cpp +++ b/tests/TypeInfer.refinements.test.cpp @@ -932,6 +932,8 @@ TEST_CASE_FIXTURE(Fixture, "apply_refinements_on_astexprindexexpr_whose_subscrip TEST_CASE_FIXTURE(Fixture, "discriminate_from_truthiness_of_x") { + ScopedFastFlag sff{"LuauFalsyPredicateReturnsNilInstead", true}; + CheckResult result = check(R"( type T = {tag: "missing", x: nil} | {tag: "exists", x: string} @@ -947,7 +949,7 @@ TEST_CASE_FIXTURE(Fixture, "discriminate_from_truthiness_of_x") LUAU_REQUIRE_NO_ERRORS(result); CHECK_EQ(R"({| tag: "exists", x: string |})", toString(requireTypeAtPosition({5, 28}))); - CHECK_EQ(R"({| tag: "missing", x: nil |})", toString(requireTypeAtPosition({7, 28}))); + CHECK_EQ(R"({| tag: "exists", x: string |} | {| tag: "missing", x: nil |})", toString(requireTypeAtPosition({7, 28}))); } TEST_CASE_FIXTURE(Fixture, "discriminate_tag") @@ -1191,7 +1193,7 @@ TEST_CASE_FIXTURE(RefinementClassFixture, "x_is_not_instance_or_else_not_part") TEST_CASE_FIXTURE(Fixture, "typeguard_doesnt_leak_to_elseif") { - const std::string code = R"( + CheckResult result = check(R"( function f(a) if type(a) == "boolean" then local a1 = a @@ -1201,10 +1203,30 @@ TEST_CASE_FIXTURE(Fixture, "typeguard_doesnt_leak_to_elseif") local a3 = a end end - )"; - CheckResult result = check(code); + )"); + LUAU_REQUIRE_NO_ERRORS(result); - dumpErrors(result); +} + +TEST_CASE_FIXTURE(BuiltinsFixture, "falsiness_of_TruthyPredicate_narrows_into_nil") +{ + ScopedFastFlag sff{"LuauFalsyPredicateReturnsNilInstead", true}; + + CheckResult result = check(R"( + local function f(t: {number}) + local x = t[1] + if not x then + local foo = x + else + local bar = x + end + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ("nil", toString(requireTypeAtPosition({4, 28}))); + CHECK_EQ("number", toString(requireTypeAtPosition({6, 28}))); } TEST_SUITE_END(); diff --git a/tests/TypeInfer.singletons.test.cpp b/tests/TypeInfer.singletons.test.cpp index 14a5a6ae..a90f434f 100644 --- a/tests/TypeInfer.singletons.test.cpp +++ b/tests/TypeInfer.singletons.test.cpp @@ -139,8 +139,6 @@ TEST_CASE_FIXTURE(Fixture, "enums_using_singletons") TEST_CASE_FIXTURE(Fixture, "enums_using_singletons_mismatch") { - ScopedFastFlag sff{"LuauTwoPassAliasDefinitionFix", true}; - CheckResult result = check(R"( type MyEnum = "foo" | "bar" | "baz" local a : MyEnum = "bang" diff --git a/tests/TypePack.test.cpp b/tests/TypePack.test.cpp index c4931578..8a5a65fe 100644 --- a/tests/TypePack.test.cpp +++ b/tests/TypePack.test.cpp @@ -197,4 +197,20 @@ TEST_CASE_FIXTURE(TypePackFixture, "std_distance") CHECK_EQ(4, std::distance(b, e)); } +TEST_CASE("content_reassignment") +{ + ScopedFastFlag luauNonCopyableTypeVarFields{"LuauNonCopyableTypeVarFields", true}; + + TypePackVar myError{Unifiable::Error{}, /*presistent*/ true}; + + TypeArena arena; + + TypePackId futureError = arena.addTypePack(TypePackVar{FreeTypePack{TypeLevel{}}}); + asMutable(futureError)->reassign(myError); + + CHECK(get(futureError) != nullptr); + CHECK(!futureError->persistent); + CHECK(futureError->owningArena == &arena); +} + TEST_SUITE_END(); diff --git a/tests/TypeVar.test.cpp b/tests/TypeVar.test.cpp index bb2d94ba..4f8fc502 100644 --- a/tests/TypeVar.test.cpp +++ b/tests/TypeVar.test.cpp @@ -416,4 +416,24 @@ TEST_CASE("proof_that_isBoolean_uses_all_of") CHECK(!isBoolean(&union_)); } +TEST_CASE("content_reassignment") +{ + ScopedFastFlag luauNonCopyableTypeVarFields{"LuauNonCopyableTypeVarFields", true}; + + TypeVar myAny{AnyTypeVar{}, /*presistent*/ true}; + myAny.normal = true; + myAny.documentationSymbol = "@global/any"; + + TypeArena arena; + + TypeId futureAny = arena.addType(FreeTypeVar{TypeLevel{}}); + asMutable(futureAny)->reassign(myAny); + + CHECK(get(futureAny) != nullptr); + CHECK(!futureAny->persistent); + CHECK(futureAny->normal); + CHECK(futureAny->documentationSymbol == "@global/any"); + CHECK(futureAny->owningArena == &arena); +} + TEST_SUITE_END(); diff --git a/tests/conformance/apicalls.lua b/tests/conformance/apicalls.lua index 7a4058b5..27416623 100644 --- a/tests/conformance/apicalls.lua +++ b/tests/conformance/apicalls.lua @@ -11,4 +11,15 @@ function create_with_tm(x) return setmetatable({ a = x }, m) end +local gen = 0 +function incuv() + gen += 1 + return gen +end + +pi = 3.1415926 +function getpi() + return pi +end + return('OK') diff --git a/tests/conformance/pcall.lua b/tests/conformance/pcall.lua index 84ac2ba1..969209fc 100644 --- a/tests/conformance/pcall.lua +++ b/tests/conformance/pcall.lua @@ -144,4 +144,21 @@ coroutine.resume(co) resumeerror(co, "fail") checkresults({ true, false, "fail" }, coroutine.resume(co)) +-- stack overflow needs to happen at the call limit +local calllimit = 20000 +function recurse(n) return n <= 1 and 1 or recurse(n-1) + 1 end + +-- we use one frame for top-level function and one frame is the service frame for coroutines +assert(recurse(calllimit - 2) == calllimit - 2) + +-- note that when calling through pcall, pcall eats one more frame +checkresults({ true, calllimit - 3 }, pcall(recurse, calllimit - 3)) +checkerror(pcall(recurse, calllimit - 2)) + +-- xpcall handler runs in context of the stack frame, but this works just fine since we allow extra stack consumption past stack overflow +checkresults({ false, "ok" }, xpcall(recurse, function() return string.reverse("ko") end, calllimit - 2)) + +-- however, if xpcall handler itself runs out of extra stack space, we get "error in error handling" +checkresults({ false, "error in error handling" }, xpcall(recurse, function() return recurse(calllimit) end, calllimit - 2)) + return 'OK' From c30ab0647b88d42b872aabf3b7d142e331c7e8ab Mon Sep 17 00:00:00 2001 From: JohnnyMorganz Date: Tue, 14 Jun 2022 16:39:25 +0100 Subject: [PATCH 272/399] Improve table stringifier when line breaks enabled (#488) * Improve table stringifier when line breaks enabled * Add FFlag * Fix FFlags --- Analysis/src/ToString.cpp | 98 ++++++++++++++++++++++++++++++--------- tests/ToString.test.cpp | 36 ++++++++++++++ 2 files changed, 113 insertions(+), 21 deletions(-) diff --git a/Analysis/src/ToString.cpp b/Analysis/src/ToString.cpp index 8490350d..04d15cf7 100644 --- a/Analysis/src/ToString.cpp +++ b/Analysis/src/ToString.cpp @@ -18,6 +18,7 @@ LUAU_FASTFLAG(LuauLowerBoundsCalculation) * Fair warning: Setting this will break a lot of Luau unit tests. */ LUAU_FASTFLAGVARIABLE(DebugLuauVerboseTypeNames, false) +LUAU_FASTFLAGVARIABLE(LuauToStringTableBracesNewlines, false) namespace Luau { @@ -283,7 +284,8 @@ struct TypeVarStringifier } Luau::visit( - [this, tv](auto&& t) { + [this, tv](auto&& t) + { return (*this)(tv, t); }, tv->ty); @@ -557,22 +559,54 @@ struct TypeVarStringifier { case TableState::Sealed: state.result.invalid = true; - openbrace = "{| "; - closedbrace = " |}"; + if (FFlag::LuauToStringTableBracesNewlines) + { + openbrace = "{|"; + closedbrace = "|}"; + } + else + { + openbrace = "{| "; + closedbrace = " |}"; + } break; case TableState::Unsealed: - openbrace = "{ "; - closedbrace = " }"; + if (FFlag::LuauToStringTableBracesNewlines) + { + openbrace = "{"; + closedbrace = "}"; + } + else + { + openbrace = "{ "; + closedbrace = " }"; + } break; case TableState::Free: state.result.invalid = true; - openbrace = "{- "; - closedbrace = " -}"; + if (FFlag::LuauToStringTableBracesNewlines) + { + openbrace = "{-"; + closedbrace = "-}"; + } + else + { + openbrace = "{- "; + closedbrace = " -}"; + } break; case TableState::Generic: state.result.invalid = true; - openbrace = "{+ "; - closedbrace = " +}"; + if (FFlag::LuauToStringTableBracesNewlines) + { + openbrace = "{+"; + closedbrace = "+}"; + } + else + { + openbrace = "{+ "; + closedbrace = " +}"; + } break; } @@ -591,6 +625,8 @@ struct TypeVarStringifier bool comma = false; if (ttv.indexer) { + if (FFlag::LuauToStringTableBracesNewlines) + state.newline(); state.emit("["); stringify(ttv.indexer->indexType); state.emit("]: "); @@ -607,6 +643,10 @@ struct TypeVarStringifier state.emit(","); state.newline(); } + else if (FFlag::LuauToStringTableBracesNewlines) + { + state.newline(); + } size_t length = state.result.name.length() - oldLength; @@ -633,6 +673,13 @@ struct TypeVarStringifier } state.dedent(); + if (FFlag::LuauToStringTableBracesNewlines) + { + if (comma) + state.newline(); + else + state.emit(" "); + } state.emit(closedbrace); state.unsee(&ttv); @@ -833,7 +880,8 @@ struct TypePackStringifier } Luau::visit( - [this, tp](auto&& t) { + [this, tp](auto&& t) + { return (*this)(tp, t); }, tp->ty); @@ -964,9 +1012,11 @@ static void assignCycleNames(const std::set& cycles, const std::set(follow(cycleTy)); !exhaustive && ttv && (ttv->syntheticName || ttv->name)) { // If we have a cycle type in type parameters, assign a cycle name for this named table - if (std::find_if(ttv->instantiatedTypeParams.begin(), ttv->instantiatedTypeParams.end(), [&](auto&& el) { - return cycles.count(follow(el)); - }) != ttv->instantiatedTypeParams.end()) + if (std::find_if(ttv->instantiatedTypeParams.begin(), ttv->instantiatedTypeParams.end(), + [&](auto&& el) + { + return cycles.count(follow(el)); + }) != ttv->instantiatedTypeParams.end()) cycleNames[cycleTy] = ttv->name ? *ttv->name : *ttv->syntheticName; continue; @@ -1062,9 +1112,11 @@ ToStringResult toStringDetailed(TypeId ty, const ToStringOptions& opts) state.exhaustive = true; std::vector> sortedCycleNames{state.cycleNames.begin(), state.cycleNames.end()}; - std::sort(sortedCycleNames.begin(), sortedCycleNames.end(), [](const auto& a, const auto& b) { - return a.second < b.second; - }); + std::sort(sortedCycleNames.begin(), sortedCycleNames.end(), + [](const auto& a, const auto& b) + { + return a.second < b.second; + }); bool semi = false; for (const auto& [cycleTy, name] : sortedCycleNames) @@ -1075,7 +1127,8 @@ ToStringResult toStringDetailed(TypeId ty, const ToStringOptions& opts) state.emit(name); state.emit(" = "); Luau::visit( - [&tvs, cycleTy = cycleTy](auto&& t) { + [&tvs, cycleTy = cycleTy](auto&& t) + { return tvs(cycleTy, t); }, cycleTy->ty); @@ -1132,9 +1185,11 @@ ToStringResult toStringDetailed(TypePackId tp, const ToStringOptions& opts) state.exhaustive = true; std::vector> sortedCycleNames{state.cycleNames.begin(), state.cycleNames.end()}; - std::sort(sortedCycleNames.begin(), sortedCycleNames.end(), [](const auto& a, const auto& b) { - return a.second < b.second; - }); + std::sort(sortedCycleNames.begin(), sortedCycleNames.end(), + [](const auto& a, const auto& b) + { + return a.second < b.second; + }); bool semi = false; for (const auto& [cycleTy, name] : sortedCycleNames) @@ -1145,7 +1200,8 @@ ToStringResult toStringDetailed(TypePackId tp, const ToStringOptions& opts) state.emit(name); state.emit(" = "); Luau::visit( - [&tvs, cycleTy = cycleTy](auto t) { + [&tvs, cycleTy = cycleTy](auto t) + { return tvs(cycleTy, t); }, cycleTy->ty); diff --git a/tests/ToString.test.cpp b/tests/ToString.test.cpp index 4d9fad14..4d2e94ee 100644 --- a/tests/ToString.test.cpp +++ b/tests/ToString.test.cpp @@ -60,6 +60,42 @@ TEST_CASE_FIXTURE(Fixture, "named_table") CHECK_EQ("TheTable", toString(&table)); } +TEST_CASE_FIXTURE(Fixture, "empty_table") +{ + ScopedFastFlag LuauToStringTableBracesNewlines("LuauToStringTableBracesNewlines", true); + CheckResult result = check(R"( + local a: {} + )"); + + CHECK_EQ("{| |}", toString(requireType("a"))); + + // Should stay the same with useLineBreaks enabled + ToStringOptions opts; + opts.useLineBreaks = true; + CHECK_EQ("{| |}", toString(requireType("a"), opts)); +} + +TEST_CASE_FIXTURE(Fixture, "table_respects_use_line_break") +{ + ScopedFastFlag LuauToStringTableBracesNewlines("LuauToStringTableBracesNewlines", true); + CheckResult result = check(R"( + local a: { prop: string, anotherProp: number, thirdProp: boolean } + )"); + + ToStringOptions opts; + opts.useLineBreaks = true; + opts.indent = true; + + //clang-format off + CHECK_EQ("{|\n" + " anotherProp: number,\n" + " prop: string,\n" + " thirdProp: boolean\n" + "|}", + toString(requireType("a"), opts)); + //clang-format on +} + TEST_CASE_FIXTURE(BuiltinsFixture, "exhaustive_toString_of_cyclic_table") { CheckResult result = check(R"( From da01056022ce7e85f402932488bdb7f4ef495a65 Mon Sep 17 00:00:00 2001 From: Allan N Jeremy Date: Tue, 14 Jun 2022 18:48:07 +0300 Subject: [PATCH 273/399] Added Luau Benchmark Workflows (#530) --- .github/workflows/benchmark.yml | 109 ++++++++++++++++++++++++++++++++ scripts/run-with-cachegrind.sh | 102 ++++++++++++++++++++++++++++++ 2 files changed, 211 insertions(+) create mode 100644 .github/workflows/benchmark.yml create mode 100644 scripts/run-with-cachegrind.sh diff --git a/.github/workflows/benchmark.yml b/.github/workflows/benchmark.yml new file mode 100644 index 00000000..68f63006 --- /dev/null +++ b/.github/workflows/benchmark.yml @@ -0,0 +1,109 @@ +name: Luau Benchmarks + +on: + push: + branches: + - master + + paths-ignore: + - "docs/**" + - "papers/**" + - "rfcs/**" + - "*.md" + - "prototyping/**" + +jobs: + benchmarks-run: + name: Run ${{ matrix.bench.title }} + strategy: + fail-fast: false + matrix: + os: [ubuntu-latest] + bench: + - { + script: "run-benchmarks", + timeout: 12, + title: "Luau Benchmarks", + cachegrindTitle: "Performance", + cachegrindIterCount: 20, + } + benchResultsRepo: + - { name: "luau-lang/benchmark-data", branch: "main" } + + runs-on: ${{ matrix.os }} + steps: + - name: Checkout Luau + uses: actions/checkout@v3 + + - name: Build Luau + run: make config=release luau luau-analyze + + - uses: actions/setup-python@v3 + with: + python-version: "3.9" + architecture: "x64" + + - name: Install python dependencies + run: | + python -m pip install requests + python -m pip install --user numpy scipy matplotlib ipython jupyter pandas sympy nose + + - name: Install valgrind + run: | + sudo apt-get install valgrind + + - name: Run benchmark + run: | + python bench/bench.py | tee ${{ matrix.bench.script }}-output.txt + + - name: Run ${{ matrix.bench.title }} (Cold Cachegrind) + run: sudo bash ./scripts/run-with-cachegrind.sh python ./bench/bench.py "${{ matrix.bench.cachegrindTitle}}Cold" 1 | tee -a ${{ matrix.bench.script }}-output.txt + + - name: Run ${{ matrix.bench.title }} (Warm Cachegrind) + run: sudo bash ./scripts/run-with-cachegrind.sh python ./bench/bench.py "${{ matrix.bench.cachegrindTitle }}" ${{ matrix.bench.cachegrindIterCount }} | tee -a ${{ matrix.bench.script }}-output.txt + + - name: Checkout Benchmark Results repository + uses: actions/checkout@v3 + with: + repository: ${{ matrix.benchResultsRepo.name }} + ref: ${{ matrix.benchResultsRepo.branch }} + token: ${{ secrets.BENCH_GITHUB_TOKEN }} + path: "./gh-pages" + + - name: Store ${{ matrix.bench.title }} result + uses: Roblox/rhysd-github-action-benchmark@v-luau + with: + name: ${{ matrix.bench.title }} + tool: "benchmarkluau" + output-file-path: ./${{ matrix.bench.script }}-output.txt + external-data-json-path: ./gh-pages/dev/bench/data.json + alert-threshold: 150% + fail-threshold: 1000% + fail-on-alert: false + comment-on-alert: true + github-token: ${{ secrets.GITHUB_TOKEN }} + + - name: Store ${{ matrix.bench.title }} result + uses: Roblox/rhysd-github-action-benchmark@v-luau + with: + name: ${{ matrix.bench.title }} (CacheGrind) + tool: "roblox" + output-file-path: ./${{ matrix.bench.script }}-output.txt + external-data-json-path: ./gh-pages/dev/bench/data.json + alert-threshold: 150% + fail-threshold: 1000% + fail-on-alert: false + comment-on-alert: true + github-token: ${{ secrets.GITHUB_TOKEN }} + + - name: Push benchmark results + + run: | + echo "Pushing benchmark results..." + cd gh-pages + git config user.name github-actions + git config user.email github@users.noreply.github.com + git add ./dev/bench/data.json + git commit -m "Add benchmarks results for ${{ github.sha }}" + git push + cd .. diff --git a/scripts/run-with-cachegrind.sh b/scripts/run-with-cachegrind.sh new file mode 100644 index 00000000..eb4a8c3f --- /dev/null +++ b/scripts/run-with-cachegrind.sh @@ -0,0 +1,102 @@ +#!/bin/bash +set -euo pipefail +IFS=$'\n\t' + +declare -A event_map +event_map[Ir]="TotalInstructionsExecuted,executions\n" +event_map[I1mr]="L1_InstrReadCacheMisses,misses/op\n" +event_map[ILmr]="LL_InstrReadCacheMisses,misses/op\n" +event_map[Dr]="TotalMemoryReads,reads\n" +event_map[D1mr]="L1_DataReadCacheMisses,misses/op\n" +event_map[DLmr]="LL_DataReadCacheMisses,misses/op\n" +event_map[Dw]="TotalMemoryWrites,writes\n" +event_map[D1mw]="L1_DataWriteCacheMisses,misses/op\n" +event_map[DLmw]="LL_DataWriteCacheMisses,misses/op\n" +event_map[Bc]="ConditionalBranchesExecuted,executions\n" +event_map[Bcm]="ConditionalBranchMispredictions,mispredictions/op\n" +event_map[Bi]="IndirectBranchesExecuted,executions\n" +event_map[Bim]="IndirectBranchMispredictions,mispredictions/op\n" + +now_ms() { + echo -n $(date +%s%N | cut -b1-13) +} + +# Run cachegrind on a given benchmark and echo the results. +ITERATION_COUNT=$4 +START_TIME=$(now_ms) + +valgrind \ + --quiet \ + --tool=cachegrind \ + "$1" "$2" >/dev/null + +TIME_ELAPSED=$(bc <<< "$(now_ms) - ${START_TIME}") + +# Generate report using cg_annotate and extract the header and totals of the +# recorded events valgrind was configured to record. +CG_RESULTS=$(cg_annotate $(ls -t cachegrind.out.* | head -1)) +CG_HEADERS=$(grep -B2 'PROGRAM TOTALS$' <<< "$CG_RESULTS" | head -1 | sed -E 's/\s+/\n/g' | sed '/^$/d') +CG_TOTALS=$(grep 'PROGRAM TOTALS$' <<< "$CG_RESULTS" | head -1 | grep -Po '[0-9,]+\s' | tr -d ', ') + +TOTALS_ARRAY=($CG_TOTALS) +HEADERS_ARRAY=($CG_HEADERS) + +declare -A header_map +for i in "${!TOTALS_ARRAY[@]}"; do + header_map[${HEADERS_ARRAY[$i]}]=$i +done + +# Map the results to the format that the benchmark script expects. +for i in "${!TOTALS_ARRAY[@]}"; do + TOTAL=${TOTALS_ARRAY[$i]} + + # Labels and unit descriptions are packed together in the map. + EVENT_TUPLE=${event_map[${HEADERS_ARRAY[$i]}]} + IFS=$',' read -d '\n' -ra EVENT_VALUES < <(printf "%s" "$EVENT_TUPLE") + EVENT_NAME="${EVENT_VALUES[0]}" + UNIT="${EVENT_VALUES[1]}" + + case ${HEADERS_ARRAY[$i]} in + I1mr | ILmr) + REF=${TOTALS_ARRAY[header_map["Ir"]]} + OPS_PER_SEC=$(bc -l <<< "$TOTAL / $REF") + ;; + + D1mr | DLmr) + REF=${TOTALS_ARRAY[header_map["Dr"]]} + OPS_PER_SEC=$(bc -l <<< "$TOTAL / $REF") + ;; + + D1mw | DLmw) + REF=${TOTALS_ARRAY[header_map["Dw"]]} + OPS_PER_SEC=$(bc -l <<< "$TOTAL / $REF") + ;; + + Bcm) + REF=${TOTALS_ARRAY[header_map["Bc"]]} + OPS_PER_SEC=$(bc -l <<< "$TOTAL / $REF") + ;; + + Bim) + REF=${TOTALS_ARRAY[header_map["Bi"]]} + OPS_PER_SEC=$(bc -l <<< "$TOTAL / $REF") + ;; + + *) + OPS_PER_SEC=$(bc -l <<< "$TOTAL") + ;; + esac + + STD_DEV="0%" + RUNS="1" + + if [[ $OPS_PER_SEC =~ ^[+-]?[0-9]*$ ]] + then # $OPS_PER_SEC is integer + printf "%s#%s x %.0f %s ±%s (%d runs sampled)\n" \ + "$3" "$EVENT_NAME" "$OPS_PER_SEC" "$UNIT" "$STD_DEV" "$RUNS" + else # $OPS_PER_SEC is float + printf "%s#%s x %.10f %s ±%s (%d runs sampled)\n" \ + "$3" "$EVENT_NAME" "$OPS_PER_SEC" "$UNIT" "$STD_DEV" "$RUNS" + fi + +done From 948f678f935eeff9484468bcc3c9edfd0417d61d Mon Sep 17 00:00:00 2001 From: Alan Jeffrey <403333+asajeffrey@users.noreply.github.com> Date: Tue, 14 Jun 2022 22:03:43 -0500 Subject: [PATCH 274/399] Prototyping function overload resolution (#508) --- prototyping/Luau/FunctionTypes.agda | 38 -- prototyping/Luau/ResolveOverloads.agda | 98 ++++ prototyping/Luau/StrictMode.agda | 2 +- prototyping/Luau/StrictMode/ToString.agda | 5 +- prototyping/Luau/Subtyping.agda | 13 +- prototyping/Luau/TypeCheck.agda | 8 +- prototyping/Luau/TypeNormalization.agda | 8 +- prototyping/Luau/TypeSaturation.agda | 66 +++ prototyping/Properties.agda | 1 - prototyping/Properties/DecSubtyping.agda | 138 +++++- prototyping/Properties/FunctionTypes.agda | 150 ------ prototyping/Properties/ResolveOverloads.agda | 189 ++++++++ prototyping/Properties/StrictMode.agda | 215 +++++---- prototyping/Properties/Subtyping.agda | 165 ++++--- prototyping/Properties/TypeCheck.agda | 12 +- prototyping/Properties/TypeNormalization.agda | 46 +- prototyping/Properties/TypeSaturation.agda | 433 ++++++++++++++++++ 17 files changed, 1207 insertions(+), 380 deletions(-) delete mode 100644 prototyping/Luau/FunctionTypes.agda create mode 100644 prototyping/Luau/ResolveOverloads.agda create mode 100644 prototyping/Luau/TypeSaturation.agda delete mode 100644 prototyping/Properties/FunctionTypes.agda create mode 100644 prototyping/Properties/ResolveOverloads.agda create mode 100644 prototyping/Properties/TypeSaturation.agda diff --git a/prototyping/Luau/FunctionTypes.agda b/prototyping/Luau/FunctionTypes.agda deleted file mode 100644 index 7607052b..00000000 --- a/prototyping/Luau/FunctionTypes.agda +++ /dev/null @@ -1,38 +0,0 @@ -{-# OPTIONS --rewriting #-} - -open import FFI.Data.Either using (Either; Left; Right) -open import Luau.Type using (Type; nil; number; string; boolean; never; unknown; _⇒_; _∪_; _∩_) -open import Luau.TypeNormalization using (normalize) - -module Luau.FunctionTypes where - --- The domain of a normalized type -srcⁿ : Type → Type -srcⁿ (S ⇒ T) = S -srcⁿ (S ∩ T) = srcⁿ S ∪ srcⁿ T -srcⁿ never = unknown -srcⁿ T = never - --- To get the domain of a type, we normalize it first We need to do --- this, since if we try to use it on non-normalized types, we get --- --- src(number ∩ string) = src(number) ∪ src(string) = never ∪ never --- src(never) = unknown --- --- so src doesn't respect type equivalence. -src : Type → Type -src (S ⇒ T) = S -src T = srcⁿ(normalize T) - --- The codomain of a type -tgt : Type → Type -tgt nil = never -tgt (S ⇒ T) = T -tgt never = never -tgt unknown = unknown -tgt number = never -tgt boolean = never -tgt string = never -tgt (S ∪ T) = (tgt S) ∪ (tgt T) -tgt (S ∩ T) = (tgt S) ∩ (tgt T) - diff --git a/prototyping/Luau/ResolveOverloads.agda b/prototyping/Luau/ResolveOverloads.agda new file mode 100644 index 00000000..67175176 --- /dev/null +++ b/prototyping/Luau/ResolveOverloads.agda @@ -0,0 +1,98 @@ +{-# OPTIONS --rewriting #-} + +module Luau.ResolveOverloads where + +open import FFI.Data.Either using (Left; Right) +open import Luau.Subtyping using (_<:_; _≮:_; Language; witness; scalar; unknown; never; function-ok) +open import Luau.Type using (Type ; _⇒_; _∩_; _∪_; unknown; never) +open import Luau.TypeSaturation using (saturate) +open import Luau.TypeNormalization using (normalize) +open import Properties.Contradiction using (CONTRADICTION) +open import Properties.DecSubtyping using (dec-subtyping; dec-subtypingⁿ; <:-impl-<:ᵒ) +open import Properties.Functions using (_∘_) +open import Properties.Subtyping using (<:-refl; <:-trans; <:-trans-≮:; ≮:-trans-<:; <:-∩-left; <:-∩-right; <:-∩-glb; <:-impl-¬≮:; <:-unknown; <:-function; function-≮:-never; <:-never; unknown-≮:-function; scalar-≮:-function; ≮:-∪-right; scalar-≮:-never; <:-∪-left; <:-∪-right) +open import Properties.TypeNormalization using (Normal; FunType; normal; _⇒_; _∩_; _∪_; never; unknown; <:-normalize; normalize-<:; fun-≮:-never; unknown-≮:-fun; scalar-≮:-fun) +open import Properties.TypeSaturation using (Overloads; Saturated; _⊆ᵒ_; _<:ᵒ_; normal-saturate; saturated; <:-saturate; saturate-<:; defn; here; left; right) + +-- The domain of a normalized type +srcⁿ : Type → Type +srcⁿ (S ⇒ T) = S +srcⁿ (S ∩ T) = srcⁿ S ∪ srcⁿ T +srcⁿ never = unknown +srcⁿ T = never + +-- To get the domain of a type, we normalize it first We need to do +-- this, since if we try to use it on non-normalized types, we get +-- +-- src(number ∩ string) = src(number) ∪ src(string) = never ∪ never +-- src(never) = unknown +-- +-- so src doesn't respect type equivalence. +src : Type → Type +src (S ⇒ T) = S +src T = srcⁿ(normalize T) + +-- Calculate the result of applying a function type `F` to an argument type `V`. +-- We do this by finding an overload of `F` that has the most precise type, +-- that is an overload `(Sʳ ⇒ Tʳ)` where `V <: Sʳ` and moreover +-- for any other such overload `(S ⇒ T)` we have that `Tʳ <: T`. + +-- For example if `F` is `(number -> number) & (nil -> nil) & (number? -> number?)` +-- then to resolve `F` with argument type `number`, we pick the `number -> number` +-- overload, but if the argument is `number?`, we pick `number? -> number?`./ + +-- Not all types have such a most precise overload, but saturated ones do. + +data ResolvedTo F G V : Set where + + yes : ∀ Sʳ Tʳ → + + Overloads F (Sʳ ⇒ Tʳ) → + (V <: Sʳ) → + (∀ {S T} → Overloads G (S ⇒ T) → (V <: S) → (Tʳ <: T)) → + -------------------------------------------- + ResolvedTo F G V + + no : + + (∀ {S T} → Overloads G (S ⇒ T) → (V ≮: S)) → + -------------------------------------------- + ResolvedTo F G V + +Resolved : Type → Type → Set +Resolved F V = ResolvedTo F F V + +target : ∀ {F V} → Resolved F V → Type +target (yes _ T _ _ _) = T +target (no _) = unknown + +-- We can resolve any saturated function type +resolveˢ : ∀ {F G V} → FunType G → Saturated F → Normal V → (G ⊆ᵒ F) → ResolvedTo F G V +resolveˢ (Sⁿ ⇒ Tⁿ) (defn sat-∩ sat-∪) Vⁿ G⊆F with dec-subtypingⁿ Vⁿ Sⁿ +resolveˢ (Sⁿ ⇒ Tⁿ) (defn sat-∩ sat-∪) Vⁿ G⊆F | Left V≮:S = no (λ { here → V≮:S }) +resolveˢ (Sⁿ ⇒ Tⁿ) (defn sat-∩ sat-∪) Vⁿ G⊆F | Right V<:S = yes _ _ (G⊆F here) V<:S (λ { here _ → <:-refl }) +resolveˢ (Gᶠ ∩ Hᶠ) (defn sat-∩ sat-∪) Vⁿ G⊆F with resolveˢ Gᶠ (defn sat-∩ sat-∪) Vⁿ (G⊆F ∘ left) | resolveˢ Hᶠ (defn sat-∩ sat-∪) Vⁿ (G⊆F ∘ right) +resolveˢ (Gᶠ ∩ Hᶠ) (defn sat-∩ sat-∪) Vⁿ G⊆F | yes S₁ T₁ o₁ V<:S₁ tgt₁ | yes S₂ T₂ o₂ V<:S₂ tgt₂ with sat-∩ o₁ o₂ +resolveˢ (Gᶠ ∩ Hᶠ) (defn sat-∩ sat-∪) Vⁿ G⊆F | yes S₁ T₁ o₁ V<:S₁ tgt₁ | yes S₂ T₂ o₂ V<:S₂ tgt₂ | defn o p₁ p₂ = + yes _ _ o (<:-trans (<:-∩-glb V<:S₁ V<:S₂) p₁) (λ { (left o) p → <:-trans p₂ (<:-trans <:-∩-left (tgt₁ o p)) ; (right o) p → <:-trans p₂ (<:-trans <:-∩-right (tgt₂ o p)) }) +resolveˢ (Gᶠ ∩ Hᶠ) (defn sat-∩ sat-∪) Vⁿ G⊆F | yes S₁ T₁ o₁ V<:S₁ tgt₁ | no src₂ = + yes _ _ o₁ V<:S₁ (λ { (left o) p → tgt₁ o p ; (right o) p → CONTRADICTION (<:-impl-¬≮: p (src₂ o)) }) +resolveˢ (Gᶠ ∩ Hᶠ) (defn sat-∩ sat-∪) Vⁿ G⊆F | no src₁ | yes S₂ T₂ o₂ V<:S₂ tgt₂ = + yes _ _ o₂ V<:S₂ (λ { (left o) p → CONTRADICTION (<:-impl-¬≮: p (src₁ o)) ; (right o) p → tgt₂ o p }) +resolveˢ (Gᶠ ∩ Hᶠ) (defn sat-∩ sat-∪) Vⁿ G⊆F | no src₁ | no src₂ = + no (λ { (left o) → src₁ o ; (right o) → src₂ o }) + +-- Which means we can resolve any normalized type, by saturating it first +resolveᶠ : ∀ {F V} → FunType F → Normal V → Type +resolveᶠ Fᶠ Vⁿ = target (resolveˢ (normal-saturate Fᶠ) (saturated Fᶠ) Vⁿ (λ o → o)) + +resolveⁿ : ∀ {F V} → Normal F → Normal V → Type +resolveⁿ (Sⁿ ⇒ Tⁿ) Vⁿ = resolveᶠ (Sⁿ ⇒ Tⁿ) Vⁿ +resolveⁿ (Fᶠ ∩ Gᶠ) Vⁿ = resolveᶠ (Fᶠ ∩ Gᶠ) Vⁿ +resolveⁿ (Sⁿ ∪ Tˢ) Vⁿ = unknown +resolveⁿ unknown Vⁿ = unknown +resolveⁿ never Vⁿ = never + +-- Which means we can resolve any type, by normalizing it first +resolve : Type → Type → Type +resolve F V = resolveⁿ (normal F) (normal V) diff --git a/prototyping/Luau/StrictMode.agda b/prototyping/Luau/StrictMode.agda index d3c0f153..0628951b 100644 --- a/prototyping/Luau/StrictMode.agda +++ b/prototyping/Luau/StrictMode.agda @@ -5,8 +5,8 @@ module Luau.StrictMode where open import Agda.Builtin.Equality using (_≡_) open import FFI.Data.Maybe using (just; nothing) open import Luau.Syntax using (Expr; Stat; Block; BinaryOperator; yes; nil; addr; var; binexp; var_∈_; _⟨_⟩∈_; function_is_end; _$_; block_is_end; local_←_; _∙_; done; return; name; +; -; *; /; <; >; <=; >=; ··) -open import Luau.FunctionTypes using (src; tgt) open import Luau.Type using (Type; nil; number; string; boolean; _⇒_; _∪_; _∩_) +open import Luau.ResolveOverloads using (src; resolve) open import Luau.Subtyping using (_≮:_) open import Luau.Heap using (Heap; function_is_end) renaming (_[_] to _[_]ᴴ) open import Luau.VarCtxt using (VarCtxt; ∅; _⋒_; _↦_; _⊕_↦_; _⊝_) renaming (_[_] to _[_]ⱽ) diff --git a/prototyping/Luau/StrictMode/ToString.agda b/prototyping/Luau/StrictMode/ToString.agda index eee5722e..7c5f0253 100644 --- a/prototyping/Luau/StrictMode/ToString.agda +++ b/prototyping/Luau/StrictMode/ToString.agda @@ -4,7 +4,7 @@ module Luau.StrictMode.ToString where open import Agda.Builtin.Nat using (Nat; suc) open import FFI.Data.String using (String; _++_) -open import Luau.Subtyping using (_≮:_; Tree; witness; scalar; function; function-ok; function-err) +open import Luau.Subtyping using (_≮:_; Tree; witness; scalar; function; function-ok; function-err; function-tgt) open import Luau.StrictMode using (Warningᴱ; Warningᴮ; UnallocatedAddress; UnboundVariable; FunctionCallMismatch; FunctionDefnMismatch; BlockMismatch; app₁; app₂; BinOpMismatch₁; BinOpMismatch₂; bin₁; bin₂; block₁; return; LocalVarMismatch; local₁; local₂; function₁; function₂; heap; expr; block; addr) open import Luau.Syntax using (Expr; val; yes; var; var_∈_; _⟨_⟩∈_; _$_; addr; number; binexp; nil; function_is_end; block_is_end; done; return; local_←_; _∙_; fun; arg; name) open import Luau.Type using (number; boolean; string; nil) @@ -27,8 +27,9 @@ treeToString (scalar boolean) n v = v ++ " is a boolean" treeToString (scalar string) n v = v ++ " is a string" treeToString (scalar nil) n v = v ++ " is nil" treeToString function n v = v ++ " is a function" -treeToString (function-ok t) n v = treeToString t n (v ++ "()") +treeToString (function-ok s t) n v = treeToString t (suc n) (v ++ "(" ++ w ++ ")") ++ " when\n " ++ treeToString s (suc n) w where w = tmp n treeToString (function-err t) n v = v ++ "(" ++ w ++ ") can error when\n " ++ treeToString t (suc n) w where w = tmp n +treeToString (function-tgt t) n v = treeToString t n (v ++ "()") subtypeWarningToString : ∀ {T U} → (T ≮: U) → String subtypeWarningToString (witness t p q) = "\n because provided type contains v, where " ++ treeToString t 0 "v" diff --git a/prototyping/Luau/Subtyping.agda b/prototyping/Luau/Subtyping.agda index 624b6be4..dc2abed0 100644 --- a/prototyping/Luau/Subtyping.agda +++ b/prototyping/Luau/Subtyping.agda @@ -13,8 +13,9 @@ data Tree : Set where scalar : ∀ {T} → Scalar T → Tree function : Tree - function-ok : Tree → Tree + function-ok : Tree → Tree → Tree function-err : Tree → Tree + function-tgt : Tree → Tree data Language : Type → Tree → Set data ¬Language : Type → Tree → Set @@ -23,8 +24,10 @@ data Language where scalar : ∀ {T} → (s : Scalar T) → Language T (scalar s) function : ∀ {T U} → Language (T ⇒ U) function - function-ok : ∀ {T U u} → (Language U u) → Language (T ⇒ U) (function-ok u) + function-ok₁ : ∀ {T U t u} → (¬Language T t) → Language (T ⇒ U) (function-ok t u) + function-ok₂ : ∀ {T U t u} → (Language U u) → Language (T ⇒ U) (function-ok t u) function-err : ∀ {T U t} → (¬Language T t) → Language (T ⇒ U) (function-err t) + function-tgt : ∀ {T U t} → (Language U t) → Language (T ⇒ U) (function-tgt t) left : ∀ {T U t} → Language T t → Language (T ∪ U) t right : ∀ {T U u} → Language U u → Language (T ∪ U) u _,_ : ∀ {T U t} → Language T t → Language U t → Language (T ∩ U) t @@ -34,11 +37,13 @@ data ¬Language where scalar-scalar : ∀ {S T} → (s : Scalar S) → (Scalar T) → (S ≢ T) → ¬Language T (scalar s) scalar-function : ∀ {S} → (Scalar S) → ¬Language S function - scalar-function-ok : ∀ {S u} → (Scalar S) → ¬Language S (function-ok u) + scalar-function-ok : ∀ {S t u} → (Scalar S) → ¬Language S (function-ok t u) scalar-function-err : ∀ {S t} → (Scalar S) → ¬Language S (function-err t) + scalar-function-tgt : ∀ {S t} → (Scalar S) → ¬Language S (function-tgt t) function-scalar : ∀ {S T U} (s : Scalar S) → ¬Language (T ⇒ U) (scalar s) - function-ok : ∀ {T U u} → (¬Language U u) → ¬Language (T ⇒ U) (function-ok u) + function-ok : ∀ {T U t u} → (Language T t) → (¬Language U u) → ¬Language (T ⇒ U) (function-ok t u) function-err : ∀ {T U t} → (Language T t) → ¬Language (T ⇒ U) (function-err t) + function-tgt : ∀ {T U t} → (¬Language U t) → ¬Language (T ⇒ U) (function-tgt t) _,_ : ∀ {T U t} → ¬Language T t → ¬Language U t → ¬Language (T ∪ U) t left : ∀ {T U t} → ¬Language T t → ¬Language (T ∩ U) t right : ∀ {T U u} → ¬Language U u → ¬Language (T ∩ U) u diff --git a/prototyping/Luau/TypeCheck.agda b/prototyping/Luau/TypeCheck.agda index d4fabb90..1abc1eda 100644 --- a/prototyping/Luau/TypeCheck.agda +++ b/prototyping/Luau/TypeCheck.agda @@ -3,16 +3,18 @@ module Luau.TypeCheck where open import Agda.Builtin.Equality using (_≡_) +open import FFI.Data.Either using (Either; Left; Right) open import FFI.Data.Maybe using (Maybe; just) +open import Luau.ResolveOverloads using (resolve) open import Luau.Syntax using (Expr; Stat; Block; BinaryOperator; yes; nil; addr; number; bool; string; val; var; var_∈_; _⟨_⟩∈_; function_is_end; _$_; block_is_end; binexp; local_←_; _∙_; done; return; name; +; -; *; /; <; >; ==; ~=; <=; >=; ··) open import Luau.Var using (Var) open import Luau.Addr using (Addr) -open import Luau.FunctionTypes using (src; tgt) open import Luau.Heap using (Heap; Object; function_is_end) renaming (_[_] to _[_]ᴴ) open import Luau.Type using (Type; nil; unknown; number; boolean; string; _⇒_) open import Luau.VarCtxt using (VarCtxt; ∅; _⋒_; _↦_; _⊕_↦_; _⊝_) renaming (_[_] to _[_]ⱽ) open import FFI.Data.Vector using (Vector) open import FFI.Data.Maybe using (Maybe; just; nothing) +open import Properties.DecSubtyping using (dec-subtyping) open import Properties.Product using (_×_; _,_) orUnknown : Maybe Type → Type @@ -113,8 +115,8 @@ data _⊢ᴱ_∈_ where Γ ⊢ᴱ M ∈ T → Γ ⊢ᴱ N ∈ U → - ---------------------- - Γ ⊢ᴱ (M $ N) ∈ (tgt T) + ---------------------------- + Γ ⊢ᴱ (M $ N) ∈ (resolve T U) function : ∀ {f x B T U V Γ} → diff --git a/prototyping/Luau/TypeNormalization.agda b/prototyping/Luau/TypeNormalization.agda index 341883ea..08f14474 100644 --- a/prototyping/Luau/TypeNormalization.agda +++ b/prototyping/Luau/TypeNormalization.agda @@ -2,11 +2,7 @@ module Luau.TypeNormalization where open import Luau.Type using (Type; nil; number; string; boolean; never; unknown; _⇒_; _∪_; _∩_) --- The top non-function type -¬function : Type -¬function = number ∪ (string ∪ (nil ∪ boolean)) - --- Unions and intersections of normalized types +-- Operations on normalized types _∪ᶠ_ : Type → Type → Type _∪ⁿˢ_ : Type → Type → Type _∩ⁿˢ_ : Type → Type → Type @@ -23,8 +19,8 @@ F ∪ᶠ G = F ∪ G S ∪ⁿ (T₁ ∪ T₂) = (S ∪ⁿ T₁) ∪ T₂ S ∪ⁿ unknown = unknown S ∪ⁿ never = S -unknown ∪ⁿ T = unknown never ∪ⁿ T = T +unknown ∪ⁿ T = unknown (S₁ ∪ S₂) ∪ⁿ G = (S₁ ∪ⁿ G) ∪ S₂ F ∪ⁿ G = F ∪ᶠ G diff --git a/prototyping/Luau/TypeSaturation.agda b/prototyping/Luau/TypeSaturation.agda new file mode 100644 index 00000000..fa24ff73 --- /dev/null +++ b/prototyping/Luau/TypeSaturation.agda @@ -0,0 +1,66 @@ +module Luau.TypeSaturation where + +open import Luau.Type using (Type; _⇒_; _∩_; _∪_) +open import Luau.TypeNormalization using (_∪ⁿ_; _∩ⁿ_) + +-- So, there's a problem with overloaded functions +-- (of the form (S_1 ⇒ T_1) ∩⋯∩ (S_n ⇒ T_n)) +-- which is that it's not good enough to compare them +-- for subtyping by comparing all of their overloads. + +-- For example (nil → nil) is a subtype of (number? → number?) ∩ (string? → string?) +-- but not a subtype of any of its overloads. + +-- To fix this, we adapt the semantic subtyping algorithm for +-- function types, given in +-- https://www.irif.fr/~gc/papers/covcon-again.pdf and +-- https://pnwamk.github.io/sst-tutorial/ + +-- A function type is *intersection-saturated* if for any overloads +-- (S₁ ⇒ T₁) and (S₂ ⇒ T₂), there exists an overload which is a subtype +-- of ((S₁ ∩ S₂) ⇒ (T₁ ∩ T₂)). + +-- A function type is *union-saturated* if for any overloads +-- (S₁ ⇒ T₁) and (S₂ ⇒ T₂), there exists an overload which is a subtype +-- of ((S₁ ∪ S₂) ⇒ (T₁ ∪ T₂)). + +-- A function type is *saturated* if it is both intersection- and +-- union-saturated. + +-- For example (number? → number?) ∩ (string? → string?) +-- is not saturated, but (number? → number?) ∩ (string? → string?) ∩ (nil → nil) ∩ ((number ∪ string)? → (number ∪ string)?) +-- is. + +-- Saturated function types have the nice property that they can ber +-- compared by just comparing their overloads: F <: G whenever for any +-- overload of G, there is an overload os F which is a subtype of it. + +-- Forunately every function type can be saturated! +_⋓_ : Type → Type → Type +(S₁ ⇒ T₁) ⋓ (S₂ ⇒ T₂) = (S₁ ∪ⁿ S₂) ⇒ (T₁ ∪ⁿ T₂) +(F₁ ∩ G₁) ⋓ F₂ = (F₁ ⋓ F₂) ∩ (G₁ ⋓ F₂) +F₁ ⋓ (F₂ ∩ G₂) = (F₁ ⋓ F₂) ∩ (F₁ ⋓ G₂) +F ⋓ G = F ∩ G + +_⋒_ : Type → Type → Type +(S₁ ⇒ T₁) ⋒ (S₂ ⇒ T₂) = (S₁ ∩ⁿ S₂) ⇒ (T₁ ∩ⁿ T₂) +(F₁ ∩ G₁) ⋒ F₂ = (F₁ ⋒ F₂) ∩ (G₁ ⋒ F₂) +F₁ ⋒ (F₂ ∩ G₂) = (F₁ ⋒ F₂) ∩ (F₁ ⋒ G₂) +F ⋒ G = F ∩ G + +_∩ᵘ_ : Type → Type → Type +F ∩ᵘ G = (F ∩ G) ∩ (F ⋓ G) + +_∩ⁱ_ : Type → Type → Type +F ∩ⁱ G = (F ∩ G) ∩ (F ⋒ G) + +∪-saturate : Type → Type +∪-saturate (F ∩ G) = (∪-saturate F ∩ᵘ ∪-saturate G) +∪-saturate F = F + +∩-saturate : Type → Type +∩-saturate (F ∩ G) = (∩-saturate F ∩ⁱ ∩-saturate G) +∩-saturate F = F + +saturate : Type → Type +saturate F = ∪-saturate (∩-saturate F) diff --git a/prototyping/Properties.agda b/prototyping/Properties.agda index b696c0fa..f883a3ea 100644 --- a/prototyping/Properties.agda +++ b/prototyping/Properties.agda @@ -7,7 +7,6 @@ import Properties.Dec import Properties.DecSubtyping import Properties.Equality import Properties.Functions -import Properties.FunctionTypes import Properties.Remember import Properties.Step import Properties.StrictMode diff --git a/prototyping/Properties/DecSubtyping.agda b/prototyping/Properties/DecSubtyping.agda index 332520a9..8dc7a446 100644 --- a/prototyping/Properties/DecSubtyping.agda +++ b/prototyping/Properties/DecSubtyping.agda @@ -4,21 +4,23 @@ module Properties.DecSubtyping where open import Agda.Builtin.Equality using (_≡_; refl) open import FFI.Data.Either using (Either; Left; Right; mapLR; swapLR; cond) -open import Luau.FunctionTypes using (src; srcⁿ; tgt) -open import Luau.Subtyping using (_<:_; _≮:_; Tree; Language; ¬Language; witness; unknown; never; scalar; function; scalar-function; scalar-function-ok; scalar-function-err; scalar-scalar; function-scalar; function-ok; function-err; left; right; _,_) +open import Luau.Subtyping using (_<:_; _≮:_; Tree; Language; ¬Language; witness; unknown; never; scalar; function; scalar-function; scalar-function-ok; scalar-function-err; scalar-function-tgt; scalar-scalar; function-scalar; function-ok; function-ok₁; function-ok₂; function-err; function-tgt; left; right; _,_) open import Luau.Type using (Type; Scalar; nil; number; string; boolean; never; unknown; _⇒_; _∪_; _∩_) +open import Luau.TypeNormalization using (_∪ⁿ_; _∩ⁿ_) +open import Luau.TypeSaturation using (saturate) open import Properties.Contradiction using (CONTRADICTION; ¬) open import Properties.Functions using (_∘_) -open import Properties.Subtyping using (<:-refl; <:-trans; ≮:-trans-<:; <:-trans-≮:; <:-never; <:-unknown; <:-∪-left; <:-∪-right; <:-∪-lub; ≮:-∪-left; ≮:-∪-right; <:-∩-left; <:-∩-right; <:-∩-glb; ≮:-∩-left; ≮:-∩-right; dec-language; scalar-<:; <:-everything; <:-function; ≮:-function-left; ≮:-function-right) -open import Properties.TypeNormalization using (FunType; Normal; never; unknown; _∩_; _∪_; _⇒_; normal; <:-normalize; normalize-<:) -open import Properties.FunctionTypes using (fun-¬scalar; ¬fun-scalar; fun-function; src-unknown-≮:; tgt-never-≮:; src-tgtᶠ-<:) +open import Properties.Subtyping using (<:-refl; <:-trans; ≮:-trans-<:; <:-trans-≮:; <:-never; <:-unknown; <:-∪-left; <:-∪-right; <:-∪-lub; ≮:-∪-left; ≮:-∪-right; <:-∩-left; <:-∩-right; <:-∩-glb; ≮:-∩-left; ≮:-∩-right; dec-language; scalar-<:; <:-everything; <:-function; ≮:-function-left; ≮:-function-right; <:-impl-¬≮:; <:-intersect; <:-function-∩-∪; <:-function-∩; <:-union; ≮:-left-∪; ≮:-right-∪; <:-∩-distr-∪; <:-impl-⊇; language-comp) +open import Properties.TypeNormalization using (FunType; Normal; never; unknown; _∩_; _∪_; _⇒_; normal; <:-normalize; normalize-<:; normal-∩ⁿ; normal-∪ⁿ; ∪-<:-∪ⁿ; ∪ⁿ-<:-∪; ∩ⁿ-<:-∩; ∩-<:-∩ⁿ; normalᶠ; fun-top; fun-function; fun-¬scalar) +open import Properties.TypeSaturation using (Overloads; Saturated; _⊆ᵒ_; _<:ᵒ_; defn; here; left; right; ov-language; ov-<:; saturated; normal-saturate; normal-overload-src; normal-overload-tgt; saturate-<:; <:-saturate; <:ᵒ-impl-<:; _>>=ˡ_; _>>=ʳ_) open import Properties.Equality using (_≢_) --- Honest this terminates, since src and tgt reduce the depth of nested arrows +-- Honest this terminates, since saturation maintains the depth of nested arrows {-# TERMINATING #-} dec-subtypingˢⁿ : ∀ {T U} → Scalar T → Normal U → Either (T ≮: U) (T <: U) -dec-subtypingᶠ : ∀ {T U} → FunType T → FunType U → Either (T ≮: U) (T <: U) -dec-subtypingᶠⁿ : ∀ {T U} → FunType T → Normal U → Either (T ≮: U) (T <: U) +dec-subtypingˢᶠ : ∀ {F G} → FunType F → Saturated F → FunType G → Either (F ≮: G) (F <:ᵒ G) +dec-subtypingᶠ : ∀ {F G} → FunType F → FunType G → Either (F ≮: G) (F <: G) +dec-subtypingᶠⁿ : ∀ {F U} → FunType F → Normal U → Either (F ≮: U) (F <: U) dec-subtypingⁿ : ∀ {T U} → Normal T → Normal U → Either (T ≮: U) (T <: U) dec-subtyping : ∀ T U → Either (T ≮: U) (T <: U) @@ -26,22 +28,116 @@ dec-subtypingˢⁿ T U with dec-language _ (scalar T) dec-subtypingˢⁿ T U | Left p = Left (witness (scalar T) (scalar T) p) dec-subtypingˢⁿ T U | Right p = Right (scalar-<: T p) -dec-subtypingᶠ {T = T} _ (U ⇒ V) with dec-subtypingⁿ U (normal (src T)) | dec-subtypingⁿ (normal (tgt T)) V -dec-subtypingᶠ {T = T} _ (U ⇒ V) | Left p | q = Left (≮:-trans-<: (src-unknown-≮: (≮:-trans-<: p (<:-normalize (src T)))) (<:-function <:-refl <:-unknown)) -dec-subtypingᶠ {T = T} _ (U ⇒ V) | Right p | Left q = Left (≮:-trans-<: (tgt-never-≮: (<:-trans-≮: (normalize-<: (tgt T)) q)) (<:-trans (<:-function <:-never <:-refl) <:-∪-right)) -dec-subtypingᶠ T (U ⇒ V) | Right p | Right q = Right (src-tgtᶠ-<: T (<:-trans p (normalize-<: _)) (<:-trans (<:-normalize _) q)) +dec-subtypingˢᶠ {F} {S ⇒ T} Fᶠ (defn sat-∩ sat-∪) (Sⁿ ⇒ Tⁿ) = result (top Fᶠ (λ o → o)) where -dec-subtypingᶠ T (U ∩ V) with dec-subtypingᶠ T U | dec-subtypingᶠ T V -dec-subtypingᶠ T (U ∩ V) | Left p | q = Left (≮:-∩-left p) -dec-subtypingᶠ T (U ∩ V) | Right p | Left q = Left (≮:-∩-right q) -dec-subtypingᶠ T (U ∩ V) | Right p | Right q = Right (<:-∩-glb p q) + data Top G : Set where + + defn : ∀ Sᵗ Tᵗ → + + Overloads F (Sᵗ ⇒ Tᵗ) → + (∀ {S′ T′} → Overloads G (S′ ⇒ T′) → (S′ <: Sᵗ)) → + ------------- + Top G + + top : ∀ {G} → (FunType G) → (G ⊆ᵒ F) → Top G + top {S′ ⇒ T′} _ G⊆F = defn S′ T′ (G⊆F here) (λ { here → <:-refl }) + top (Gᶠ ∩ Hᶠ) G⊆F with top Gᶠ (G⊆F ∘ left) | top Hᶠ (G⊆F ∘ right) + top (Gᶠ ∩ Hᶠ) G⊆F | defn Rᵗ Sᵗ p p₁ | defn Tᵗ Uᵗ q q₁ with sat-∪ p q + top (Gᶠ ∩ Hᶠ) G⊆F | defn Rᵗ Sᵗ p p₁ | defn Tᵗ Uᵗ q q₁ | defn n r r₁ = defn _ _ n + (λ { (left o) → <:-trans (<:-trans (p₁ o) <:-∪-left) r ; (right o) → <:-trans (<:-trans (q₁ o) <:-∪-right) r }) + + result : Top F → Either (F ≮: (S ⇒ T)) (F <:ᵒ (S ⇒ T)) + result (defn Sᵗ Tᵗ oᵗ srcᵗ) with dec-subtypingⁿ Sⁿ (normal-overload-src Fᶠ oᵗ) + result (defn Sᵗ Tᵗ oᵗ srcᵗ) | Left (witness s Ss ¬Sᵗs) = Left (witness (function-err s) (ov-language Fᶠ (λ o → function-err (<:-impl-⊇ (srcᵗ o) s ¬Sᵗs))) (function-err Ss)) + result (defn Sᵗ Tᵗ oᵗ srcᵗ) | Right S<:Sᵗ = result₀ (largest Fᶠ (λ o → o)) where + + data LargestSrc (G : Type) : Set where + + yes : ∀ S₀ T₀ → + + Overloads F (S₀ ⇒ T₀) → + T₀ <: T → + (∀ {S′ T′} → Overloads G (S′ ⇒ T′) → T′ <: T → (S′ <: S₀)) → + ----------------------- + LargestSrc G + + no : ∀ S₀ T₀ → + + Overloads F (S₀ ⇒ T₀) → + T₀ ≮: T → + (∀ {S′ T′} → Overloads G (S′ ⇒ T′) → T₀ <: T′) → + ----------------------- + LargestSrc G + + largest : ∀ {G} → (FunType G) → (G ⊆ᵒ F) → LargestSrc G + largest {S′ ⇒ T′} (S′ⁿ ⇒ T′ⁿ) G⊆F with dec-subtypingⁿ T′ⁿ Tⁿ + largest {S′ ⇒ T′} (S′ⁿ ⇒ T′ⁿ) G⊆F | Left T′≮:T = no S′ T′ (G⊆F here) T′≮:T λ { here → <:-refl } + largest {S′ ⇒ T′} (S′ⁿ ⇒ T′ⁿ) G⊆F | Right T′<:T = yes S′ T′ (G⊆F here) T′<:T (λ { here _ → <:-refl }) + largest (Gᶠ ∩ Hᶠ) GH⊆F with largest Gᶠ (GH⊆F ∘ left) | largest Hᶠ (GH⊆F ∘ right) + largest (Gᶠ ∩ Hᶠ) GH⊆F | no S₁ T₁ o₁ T₁≮:T tgt₁ | no S₂ T₂ o₂ T₂≮:T tgt₂ with sat-∩ o₁ o₂ + largest (Gᶠ ∩ Hᶠ) GH⊆F | no S₁ T₁ o₁ T₁≮:T tgt₁ | no S₂ T₂ o₂ T₂≮:T tgt₂ | defn o src tgt with dec-subtypingⁿ (normal-overload-tgt Fᶠ o) Tⁿ + largest (Gᶠ ∩ Hᶠ) GH⊆F | no S₁ T₁ o₁ T₁≮:T tgt₁ | no S₂ T₂ o₂ T₂≮:T tgt₂ | defn o src tgt | Left T₀≮:T = no _ _ o T₀≮:T (λ { (left o) → <:-trans tgt (<:-trans <:-∩-left (tgt₁ o)) ; (right o) → <:-trans tgt (<:-trans <:-∩-right (tgt₂ o)) }) + largest (Gᶠ ∩ Hᶠ) GH⊆F | no S₁ T₁ o₁ T₁≮:T tgt₁ | no S₂ T₂ o₂ T₂≮:T tgt₂ | defn o src tgt | Right T₀<:T = yes _ _ o T₀<:T (λ { (left o) p → CONTRADICTION (<:-impl-¬≮: p (<:-trans-≮: (tgt₁ o) T₁≮:T)) ; (right o) p → CONTRADICTION (<:-impl-¬≮: p (<:-trans-≮: (tgt₂ o) T₂≮:T)) }) + largest (Gᶠ ∩ Hᶠ) GH⊆F | no S₁ T₁ o₁ T₁≮:T tgt₁ | yes S₂ T₂ o₂ T₂<:T src₂ = yes S₂ T₂ o₂ T₂<:T (λ { (left o) p → CONTRADICTION (<:-impl-¬≮: p (<:-trans-≮: (tgt₁ o) T₁≮:T)) ; (right o) p → src₂ o p }) + largest (Gᶠ ∩ Hᶠ) GH⊆F | yes S₁ T₁ o₁ T₁<:T src₁ | no S₂ T₂ o₂ T₂≮:T tgt₂ = yes S₁ T₁ o₁ T₁<:T (λ { (left o) p → src₁ o p ; (right o) p → CONTRADICTION (<:-impl-¬≮: p (<:-trans-≮: (tgt₂ o) T₂≮:T)) }) + largest (Gᶠ ∩ Hᶠ) GH⊆F | yes S₁ T₁ o₁ T₁<:T src₁ | yes S₂ T₂ o₂ T₂<:T src₂ with sat-∪ o₁ o₂ + largest (Gᶠ ∩ Hᶠ) GH⊆F | yes S₁ T₁ o₁ T₁<:T src₁ | yes S₂ T₂ o₂ T₂<:T src₂ | defn o src tgt = yes _ _ o (<:-trans tgt (<:-∪-lub T₁<:T T₂<:T)) + (λ { (left o) T′<:T → <:-trans (src₁ o T′<:T) (<:-trans <:-∪-left src) + ; (right o) T′<:T → <:-trans (src₂ o T′<:T) (<:-trans <:-∪-right src) + }) + + result₀ : LargestSrc F → Either (F ≮: (S ⇒ T)) (F <:ᵒ (S ⇒ T)) + result₀ (no S₀ T₀ o₀ (witness t T₀t ¬Tt) tgt₀) = Left (witness (function-tgt t) (ov-language Fᶠ (λ o → function-tgt (tgt₀ o t T₀t))) (function-tgt ¬Tt)) + result₀ (yes S₀ T₀ o₀ T₀<:T src₀) with dec-subtypingⁿ Sⁿ (normal-overload-src Fᶠ o₀) + result₀ (yes S₀ T₀ o₀ T₀<:T src₀) | Right S<:S₀ = Right λ { here → defn o₀ S<:S₀ T₀<:T } + result₀ (yes S₀ T₀ o₀ T₀<:T src₀) | Left (witness s Ss ¬S₀s) = Left (result₁ (smallest Fᶠ (λ o → o))) where + + data SmallestTgt (G : Type) : Set where + + defn : ∀ S₁ T₁ → + + Overloads F (S₁ ⇒ T₁) → + Language S₁ s → + (∀ {S′ T′} → Overloads G (S′ ⇒ T′) → Language S′ s → (T₁ <: T′)) → + ----------------------- + SmallestTgt G + + smallest : ∀ {G} → (FunType G) → (G ⊆ᵒ F) → SmallestTgt G + smallest {S′ ⇒ T′} _ G⊆F with dec-language S′ s + smallest {S′ ⇒ T′} _ G⊆F | Left ¬S′s = defn Sᵗ Tᵗ oᵗ (S<:Sᵗ s Ss) λ { here S′s → CONTRADICTION (language-comp s ¬S′s S′s) } + smallest {S′ ⇒ T′} _ G⊆F | Right S′s = defn S′ T′ (G⊆F here) S′s (λ { here _ → <:-refl }) + smallest (Gᶠ ∩ Hᶠ) GH⊆F with smallest Gᶠ (GH⊆F ∘ left) | smallest Hᶠ (GH⊆F ∘ right) + smallest (Gᶠ ∩ Hᶠ) GH⊆F | defn S₁ T₁ o₁ R₁s tgt₁ | defn S₂ T₂ o₂ R₂s tgt₂ with sat-∩ o₁ o₂ + smallest (Gᶠ ∩ Hᶠ) GH⊆F | defn S₁ T₁ o₁ R₁s tgt₁ | defn S₂ T₂ o₂ R₂s tgt₂ | defn o src tgt = defn _ _ o (src s (R₁s , R₂s)) + (λ { (left o) S′s → <:-trans (<:-trans tgt <:-∩-left) (tgt₁ o S′s) + ; (right o) S′s → <:-trans (<:-trans tgt <:-∩-right) (tgt₂ o S′s) + }) + + result₁ : SmallestTgt F → (F ≮: (S ⇒ T)) + result₁ (defn S₁ T₁ o₁ S₁s tgt₁) with dec-subtypingⁿ (normal-overload-tgt Fᶠ o₁) Tⁿ + result₁ (defn S₁ T₁ o₁ S₁s tgt₁) | Right T₁<:T = CONTRADICTION (language-comp s ¬S₀s (src₀ o₁ T₁<:T s S₁s)) + result₁ (defn S₁ T₁ o₁ S₁s tgt₁) | Left (witness t T₁t ¬Tt) = witness (function-ok s t) (ov-language Fᶠ lemma) (function-ok Ss ¬Tt) where + + lemma : ∀ {S′ T′} → Overloads F (S′ ⇒ T′) → Language (S′ ⇒ T′) (function-ok s t) + lemma {S′} o with dec-language S′ s + lemma {S′} o | Left ¬S′s = function-ok₁ ¬S′s + lemma {S′} o | Right S′s = function-ok₂ (tgt₁ o S′s t T₁t) + +dec-subtypingˢᶠ F Fˢ (G ∩ H) with dec-subtypingˢᶠ F Fˢ G | dec-subtypingˢᶠ F Fˢ H +dec-subtypingˢᶠ F Fˢ (G ∩ H) | Left F≮:G | _ = Left (≮:-∩-left F≮:G) +dec-subtypingˢᶠ F Fˢ (G ∩ H) | _ | Left F≮:H = Left (≮:-∩-right F≮:H) +dec-subtypingˢᶠ F Fˢ (G ∩ H) | Right F<:G | Right F<:H = Right (λ { (left o) → F<:G o ; (right o) → F<:H o }) + +dec-subtypingᶠ F G with dec-subtypingˢᶠ (normal-saturate F) (saturated F) G +dec-subtypingᶠ F G | Left H≮:G = Left (<:-trans-≮: (saturate-<: F) H≮:G) +dec-subtypingᶠ F G | Right H<:G = Right (<:-trans (<:-saturate F) (<:ᵒ-impl-<: (normal-saturate F) G H<:G)) dec-subtypingᶠⁿ T never = Left (witness function (fun-function T) never) dec-subtypingᶠⁿ T unknown = Right <:-unknown dec-subtypingᶠⁿ T (U ⇒ V) = dec-subtypingᶠ T (U ⇒ V) dec-subtypingᶠⁿ T (U ∩ V) = dec-subtypingᶠ T (U ∩ V) dec-subtypingᶠⁿ T (U ∪ V) with dec-subtypingᶠⁿ T U -dec-subtypingᶠⁿ T (U ∪ V) | Left (witness t p q) = Left (witness t p (q , ¬fun-scalar V T p)) +dec-subtypingᶠⁿ T (U ∪ V) | Left (witness t p q) = Left (witness t p (q , fun-¬scalar V T p)) dec-subtypingᶠⁿ T (U ∪ V) | Right p = Right (<:-trans p <:-∪-left) dec-subtypingⁿ never U = Right <:-never @@ -68,3 +164,11 @@ dec-subtyping T U with dec-subtypingⁿ (normal T) (normal U) dec-subtyping T U | Left p = Left (<:-trans-≮: (normalize-<: T) (≮:-trans-<: p (<:-normalize U))) dec-subtyping T U | Right p = Right (<:-trans (<:-normalize T) (<:-trans p (normalize-<: U))) +-- As a corollary, for saturated functions +-- <:ᵒ coincides with <:, that is F is a subtype of (S ⇒ T) precisely +-- when one of its overloads is. + +<:-impl-<:ᵒ : ∀ {F G} → FunType F → Saturated F → FunType G → (F <: G) → (F <:ᵒ G) +<:-impl-<:ᵒ {F} {G} Fᶠ Fˢ Gᶠ F<:G with dec-subtypingˢᶠ Fᶠ Fˢ Gᶠ +<:-impl-<:ᵒ {F} {G} Fᶠ Fˢ Gᶠ F<:G | Left F≮:G = CONTRADICTION (<:-impl-¬≮: F<:G F≮:G) +<:-impl-<:ᵒ {F} {G} Fᶠ Fˢ Gᶠ F<:G | Right F<:ᵒG = F<:ᵒG diff --git a/prototyping/Properties/FunctionTypes.agda b/prototyping/Properties/FunctionTypes.agda deleted file mode 100644 index 514477f1..00000000 --- a/prototyping/Properties/FunctionTypes.agda +++ /dev/null @@ -1,150 +0,0 @@ -{-# OPTIONS --rewriting #-} - -module Properties.FunctionTypes where - -open import FFI.Data.Either using (Either; Left; Right; mapLR; swapLR; cond) -open import Luau.FunctionTypes using (srcⁿ; src; tgt) -open import Luau.Subtyping using (_<:_; _≮:_; Tree; Language; ¬Language; witness; unknown; never; scalar; function; scalar-function; scalar-function-ok; scalar-function-err; scalar-scalar; function-scalar; function-ok; function-err; left; right; _,_) -open import Luau.Type using (Type; Scalar; nil; number; string; boolean; never; unknown; _⇒_; _∪_; _∩_; skalar) -open import Properties.Contradiction using (CONTRADICTION; ¬; ⊥) -open import Properties.Functions using (_∘_) -open import Properties.Subtyping using (<:-refl; ≮:-refl; <:-trans-≮:; skalar-scalar; <:-impl-⊇; skalar-function-ok; language-comp) -open import Properties.TypeNormalization using (FunType; Normal; never; unknown; _∩_; _∪_; _⇒_; normal; <:-normalize; normalize-<:) - --- Properties of src -function-err-srcⁿ : ∀ {T t} → (FunType T) → (¬Language (srcⁿ T) t) → Language T (function-err t) -function-err-srcⁿ (S ⇒ T) p = function-err p -function-err-srcⁿ (S ∩ T) (p₁ , p₂) = (function-err-srcⁿ S p₁ , function-err-srcⁿ T p₂) - -¬function-err-srcᶠ : ∀ {T t} → (FunType T) → (Language (srcⁿ T) t) → ¬Language T (function-err t) -¬function-err-srcᶠ (S ⇒ T) p = function-err p -¬function-err-srcᶠ (S ∩ T) (left p) = left (¬function-err-srcᶠ S p) -¬function-err-srcᶠ (S ∩ T) (right p) = right (¬function-err-srcᶠ T p) - -¬function-err-srcⁿ : ∀ {T t} → (Normal T) → (Language (srcⁿ T) t) → ¬Language T (function-err t) -¬function-err-srcⁿ never p = never -¬function-err-srcⁿ unknown (scalar ()) -¬function-err-srcⁿ (S ⇒ T) p = function-err p -¬function-err-srcⁿ (S ∩ T) (left p) = left (¬function-err-srcᶠ S p) -¬function-err-srcⁿ (S ∩ T) (right p) = right (¬function-err-srcᶠ T p) -¬function-err-srcⁿ (S ∪ T) (scalar ()) - -¬function-err-src : ∀ {T t} → (Language (src T) t) → ¬Language T (function-err t) -¬function-err-src {T = S ⇒ T} p = function-err p -¬function-err-src {T = nil} p = scalar-function-err nil -¬function-err-src {T = never} p = never -¬function-err-src {T = unknown} (scalar ()) -¬function-err-src {T = boolean} p = scalar-function-err boolean -¬function-err-src {T = number} p = scalar-function-err number -¬function-err-src {T = string} p = scalar-function-err string -¬function-err-src {T = S ∪ T} p = <:-impl-⊇ (<:-normalize (S ∪ T)) _ (¬function-err-srcⁿ (normal (S ∪ T)) p) -¬function-err-src {T = S ∩ T} p = <:-impl-⊇ (<:-normalize (S ∩ T)) _ (¬function-err-srcⁿ (normal (S ∩ T)) p) - -src-¬function-errᶠ : ∀ {T t} → (FunType T) → Language T (function-err t) → (¬Language (srcⁿ T) t) -src-¬function-errᶠ (S ⇒ T) (function-err p) = p -src-¬function-errᶠ (S ∩ T) (p₁ , p₂) = (src-¬function-errᶠ S p₁ , src-¬function-errᶠ T p₂) - -src-¬function-errⁿ : ∀ {T t} → (Normal T) → Language T (function-err t) → (¬Language (srcⁿ T) t) -src-¬function-errⁿ unknown p = never -src-¬function-errⁿ (S ⇒ T) (function-err p) = p -src-¬function-errⁿ (S ∩ T) (p₁ , p₂) = (src-¬function-errᶠ S p₁ , src-¬function-errᶠ T p₂) -src-¬function-errⁿ (S ∪ T) p = never - -src-¬function-err : ∀ {T t} → Language T (function-err t) → (¬Language (src T) t) -src-¬function-err {T = S ⇒ T} (function-err p) = p -src-¬function-err {T = unknown} p = never -src-¬function-err {T = S ∪ T} p = src-¬function-errⁿ (normal (S ∪ T)) (<:-normalize (S ∪ T) _ p) -src-¬function-err {T = S ∩ T} p = src-¬function-errⁿ (normal (S ∩ T)) (<:-normalize (S ∩ T) _ p) - -fun-¬scalar : ∀ {S T} (s : Scalar S) → FunType T → ¬Language T (scalar s) -fun-¬scalar s (S ⇒ T) = function-scalar s -fun-¬scalar s (S ∩ T) = left (fun-¬scalar s S) - -¬fun-scalar : ∀ {S T t} (s : Scalar S) → FunType T → Language T t → ¬Language S t -¬fun-scalar s (S ⇒ T) function = scalar-function s -¬fun-scalar s (S ⇒ T) (function-ok p) = scalar-function-ok s -¬fun-scalar s (S ⇒ T) (function-err p) = scalar-function-err s -¬fun-scalar s (S ∩ T) (p₁ , p₂) = ¬fun-scalar s T p₂ - -fun-function : ∀ {T} → FunType T → Language T function -fun-function (S ⇒ T) = function -fun-function (S ∩ T) = (fun-function S , fun-function T) - -srcⁿ-¬scalar : ∀ {S T t} (s : Scalar S) → Normal T → Language T (scalar s) → (¬Language (srcⁿ T) t) -srcⁿ-¬scalar s never (scalar ()) -srcⁿ-¬scalar s unknown p = never -srcⁿ-¬scalar s (S ⇒ T) (scalar ()) -srcⁿ-¬scalar s (S ∩ T) (p₁ , p₂) = CONTRADICTION (language-comp (scalar s) (fun-¬scalar s S) p₁) -srcⁿ-¬scalar s (S ∪ T) p = never - -src-¬scalar : ∀ {S T t} (s : Scalar S) → Language T (scalar s) → (¬Language (src T) t) -src-¬scalar {T = nil} s p = never -src-¬scalar {T = T ⇒ U} s (scalar ()) -src-¬scalar {T = never} s (scalar ()) -src-¬scalar {T = unknown} s p = never -src-¬scalar {T = boolean} s p = never -src-¬scalar {T = number} s p = never -src-¬scalar {T = string} s p = never -src-¬scalar {T = T ∪ U} s p = srcⁿ-¬scalar s (normal (T ∪ U)) (<:-normalize (T ∪ U) (scalar s) p) -src-¬scalar {T = T ∩ U} s p = srcⁿ-¬scalar s (normal (T ∩ U)) (<:-normalize (T ∩ U) (scalar s) p) - -srcⁿ-unknown-≮: : ∀ {T U} → (Normal U) → (T ≮: srcⁿ U) → (U ≮: (T ⇒ unknown)) -srcⁿ-unknown-≮: never (witness t p q) = CONTRADICTION (language-comp t q unknown) -srcⁿ-unknown-≮: unknown (witness t p q) = witness (function-err t) unknown (function-err p) -srcⁿ-unknown-≮: (U ⇒ V) (witness t p q) = witness (function-err t) (function-err q) (function-err p) -srcⁿ-unknown-≮: (U ∩ V) (witness t p q) = witness (function-err t) (function-err-srcⁿ (U ∩ V) q) (function-err p) -srcⁿ-unknown-≮: (U ∪ V) (witness t p q) = witness (scalar V) (right (scalar V)) (function-scalar V) - -src-unknown-≮: : ∀ {T U} → (T ≮: src U) → (U ≮: (T ⇒ unknown)) -src-unknown-≮: {U = nil} (witness t p q) = witness (scalar nil) (scalar nil) (function-scalar nil) -src-unknown-≮: {U = T ⇒ U} (witness t p q) = witness (function-err t) (function-err q) (function-err p) -src-unknown-≮: {U = never} (witness t p q) = CONTRADICTION (language-comp t q unknown) -src-unknown-≮: {U = unknown} (witness t p q) = witness (function-err t) unknown (function-err p) -src-unknown-≮: {U = boolean} (witness t p q) = witness (scalar boolean) (scalar boolean) (function-scalar boolean) -src-unknown-≮: {U = number} (witness t p q) = witness (scalar number) (scalar number) (function-scalar number) -src-unknown-≮: {U = string} (witness t p q) = witness (scalar string) (scalar string) (function-scalar string) -src-unknown-≮: {U = T ∪ U} p = <:-trans-≮: (normalize-<: (T ∪ U)) (srcⁿ-unknown-≮: (normal (T ∪ U)) p) -src-unknown-≮: {U = T ∩ U} p = <:-trans-≮: (normalize-<: (T ∩ U)) (srcⁿ-unknown-≮: (normal (T ∩ U)) p) - -unknown-src-≮: : ∀ {S T U} → (U ≮: S) → (T ≮: (U ⇒ unknown)) → (U ≮: src T) -unknown-src-≮: (witness t x x₁) (witness (scalar s) p (function-scalar s)) = witness t x (src-¬scalar s p) -unknown-src-≮: r (witness (function-ok (scalar s)) p (function-ok (scalar-scalar s () q))) -unknown-src-≮: r (witness (function-ok (function-ok _)) p (function-ok (scalar-function-ok ()))) -unknown-src-≮: r (witness (function-err t) p (function-err q)) = witness t q (src-¬function-err p) - --- Properties of tgt -tgt-function-ok : ∀ {T t} → (Language (tgt T) t) → Language T (function-ok t) -tgt-function-ok {T = nil} (scalar ()) -tgt-function-ok {T = T₁ ⇒ T₂} p = function-ok p -tgt-function-ok {T = never} (scalar ()) -tgt-function-ok {T = unknown} p = unknown -tgt-function-ok {T = boolean} (scalar ()) -tgt-function-ok {T = number} (scalar ()) -tgt-function-ok {T = string} (scalar ()) -tgt-function-ok {T = T₁ ∪ T₂} (left p) = left (tgt-function-ok p) -tgt-function-ok {T = T₁ ∪ T₂} (right p) = right (tgt-function-ok p) -tgt-function-ok {T = T₁ ∩ T₂} (p₁ , p₂) = (tgt-function-ok p₁ , tgt-function-ok p₂) - -function-ok-tgt : ∀ {T t} → Language T (function-ok t) → (Language (tgt T) t) -function-ok-tgt (function-ok p) = p -function-ok-tgt (left p) = left (function-ok-tgt p) -function-ok-tgt (right p) = right (function-ok-tgt p) -function-ok-tgt (p₁ , p₂) = (function-ok-tgt p₁ , function-ok-tgt p₂) -function-ok-tgt unknown = unknown - -tgt-never-≮: : ∀ {T U} → (tgt T ≮: U) → (T ≮: (skalar ∪ (never ⇒ U))) -tgt-never-≮: (witness t p q) = witness (function-ok t) (tgt-function-ok p) (skalar-function-ok , function-ok q) - -never-tgt-≮: : ∀ {T U} → (T ≮: (skalar ∪ (never ⇒ U))) → (tgt T ≮: U) -never-tgt-≮: (witness (scalar s) p (q₁ , q₂)) = CONTRADICTION (≮:-refl (witness (scalar s) (skalar-scalar s) q₁)) -never-tgt-≮: (witness function p (q₁ , scalar-function ())) -never-tgt-≮: (witness (function-ok t) p (q₁ , function-ok q₂)) = witness t (function-ok-tgt p) q₂ -never-tgt-≮: (witness (function-err (scalar s)) p (q₁ , function-err (scalar ()))) - -src-tgtᶠ-<: : ∀ {T U V} → (FunType T) → (U <: src T) → (tgt T <: V) → (T <: (U ⇒ V)) -src-tgtᶠ-<: T p q (scalar s) r = CONTRADICTION (language-comp (scalar s) (fun-¬scalar s T) r) -src-tgtᶠ-<: T p q function r = function -src-tgtᶠ-<: T p q (function-ok s) r = function-ok (q s (function-ok-tgt r)) -src-tgtᶠ-<: T p q (function-err s) r = function-err (<:-impl-⊇ p s (src-¬function-err r)) - - diff --git a/prototyping/Properties/ResolveOverloads.agda b/prototyping/Properties/ResolveOverloads.agda new file mode 100644 index 00000000..8de4a875 --- /dev/null +++ b/prototyping/Properties/ResolveOverloads.agda @@ -0,0 +1,189 @@ +{-# OPTIONS --rewriting #-} + +module Properties.ResolveOverloads where + +open import FFI.Data.Either using (Left; Right) +open import Luau.ResolveOverloads using (Resolved; src; srcⁿ; resolve; resolveⁿ; resolveᶠ; resolveˢ; target; yes; no) +open import Luau.Subtyping using (_<:_; _≮:_; Language; ¬Language; witness; scalar; unknown; never; function; function-ok; function-err; function-tgt; function-scalar; function-ok₁; function-ok₂; scalar-scalar; scalar-function; scalar-function-ok; scalar-function-err; scalar-function-tgt; _,_; left; right) +open import Luau.Type using (Type ; Scalar; _⇒_; _∩_; _∪_; nil; boolean; number; string; unknown; never) +open import Luau.TypeSaturation using (saturate) +open import Luau.TypeNormalization using (normalize) +open import Properties.Contradiction using (CONTRADICTION) +open import Properties.DecSubtyping using (dec-subtyping; dec-subtypingⁿ; <:-impl-<:ᵒ) +open import Properties.Functions using (_∘_) +open import Properties.Subtyping using (<:-refl; <:-trans; <:-trans-≮:; ≮:-trans-<:; <:-∩-left; <:-∩-right; <:-∩-glb; <:-impl-¬≮:; <:-unknown; <:-function; function-≮:-never; <:-never; unknown-≮:-function; scalar-≮:-function; ≮:-∪-right; scalar-≮:-never; <:-∪-left; <:-∪-right; <:-impl-⊇; language-comp) +open import Properties.TypeNormalization using (Normal; FunType; normal; _⇒_; _∩_; _∪_; never; unknown; <:-normalize; normalize-<:; fun-≮:-never; unknown-≮:-fun; scalar-≮:-fun) +open import Properties.TypeSaturation using (Overloads; Saturated; _⊆ᵒ_; _<:ᵒ_; normal-saturate; saturated; <:-saturate; saturate-<:; defn; here; left; right) + +-- Properties of src +function-err-srcⁿ : ∀ {T t} → (FunType T) → (¬Language (srcⁿ T) t) → Language T (function-err t) +function-err-srcⁿ (S ⇒ T) p = function-err p +function-err-srcⁿ (S ∩ T) (p₁ , p₂) = (function-err-srcⁿ S p₁ , function-err-srcⁿ T p₂) + +¬function-err-srcᶠ : ∀ {T t} → (FunType T) → (Language (srcⁿ T) t) → ¬Language T (function-err t) +¬function-err-srcᶠ (S ⇒ T) p = function-err p +¬function-err-srcᶠ (S ∩ T) (left p) = left (¬function-err-srcᶠ S p) +¬function-err-srcᶠ (S ∩ T) (right p) = right (¬function-err-srcᶠ T p) + +¬function-err-srcⁿ : ∀ {T t} → (Normal T) → (Language (srcⁿ T) t) → ¬Language T (function-err t) +¬function-err-srcⁿ never p = never +¬function-err-srcⁿ unknown (scalar ()) +¬function-err-srcⁿ (S ⇒ T) p = function-err p +¬function-err-srcⁿ (S ∩ T) (left p) = left (¬function-err-srcᶠ S p) +¬function-err-srcⁿ (S ∩ T) (right p) = right (¬function-err-srcᶠ T p) +¬function-err-srcⁿ (S ∪ T) (scalar ()) + +¬function-err-src : ∀ {T t} → (Language (src T) t) → ¬Language T (function-err t) +¬function-err-src {T = S ⇒ T} p = function-err p +¬function-err-src {T = nil} p = scalar-function-err nil +¬function-err-src {T = never} p = never +¬function-err-src {T = unknown} (scalar ()) +¬function-err-src {T = boolean} p = scalar-function-err boolean +¬function-err-src {T = number} p = scalar-function-err number +¬function-err-src {T = string} p = scalar-function-err string +¬function-err-src {T = S ∪ T} p = <:-impl-⊇ (<:-normalize (S ∪ T)) _ (¬function-err-srcⁿ (normal (S ∪ T)) p) +¬function-err-src {T = S ∩ T} p = <:-impl-⊇ (<:-normalize (S ∩ T)) _ (¬function-err-srcⁿ (normal (S ∩ T)) p) + +src-¬function-errᶠ : ∀ {T t} → (FunType T) → Language T (function-err t) → (¬Language (srcⁿ T) t) +src-¬function-errᶠ (S ⇒ T) (function-err p) = p +src-¬function-errᶠ (S ∩ T) (p₁ , p₂) = (src-¬function-errᶠ S p₁ , src-¬function-errᶠ T p₂) + +src-¬function-errⁿ : ∀ {T t} → (Normal T) → Language T (function-err t) → (¬Language (srcⁿ T) t) +src-¬function-errⁿ unknown p = never +src-¬function-errⁿ (S ⇒ T) (function-err p) = p +src-¬function-errⁿ (S ∩ T) (p₁ , p₂) = (src-¬function-errᶠ S p₁ , src-¬function-errᶠ T p₂) +src-¬function-errⁿ (S ∪ T) p = never + +src-¬function-err : ∀ {T t} → Language T (function-err t) → (¬Language (src T) t) +src-¬function-err {T = S ⇒ T} (function-err p) = p +src-¬function-err {T = unknown} p = never +src-¬function-err {T = S ∪ T} p = src-¬function-errⁿ (normal (S ∪ T)) (<:-normalize (S ∪ T) _ p) +src-¬function-err {T = S ∩ T} p = src-¬function-errⁿ (normal (S ∩ T)) (<:-normalize (S ∩ T) _ p) + +fun-¬scalar : ∀ {S T} (s : Scalar S) → FunType T → ¬Language T (scalar s) +fun-¬scalar s (S ⇒ T) = function-scalar s +fun-¬scalar s (S ∩ T) = left (fun-¬scalar s S) + +¬fun-scalar : ∀ {S T t} (s : Scalar S) → FunType T → Language T t → ¬Language S t +¬fun-scalar s (S ⇒ T) function = scalar-function s +¬fun-scalar s (S ⇒ T) (function-ok₁ p) = scalar-function-ok s +¬fun-scalar s (S ⇒ T) (function-ok₂ p) = scalar-function-ok s +¬fun-scalar s (S ⇒ T) (function-err p) = scalar-function-err s +¬fun-scalar s (S ⇒ T) (function-tgt p) = scalar-function-tgt s +¬fun-scalar s (S ∩ T) (p₁ , p₂) = ¬fun-scalar s T p₂ + +fun-function : ∀ {T} → FunType T → Language T function +fun-function (S ⇒ T) = function +fun-function (S ∩ T) = (fun-function S , fun-function T) + +srcⁿ-¬scalar : ∀ {S T t} (s : Scalar S) → Normal T → Language T (scalar s) → (¬Language (srcⁿ T) t) +srcⁿ-¬scalar s never (scalar ()) +srcⁿ-¬scalar s unknown p = never +srcⁿ-¬scalar s (S ⇒ T) (scalar ()) +srcⁿ-¬scalar s (S ∩ T) (p₁ , p₂) = CONTRADICTION (language-comp (scalar s) (fun-¬scalar s S) p₁) +srcⁿ-¬scalar s (S ∪ T) p = never + +src-¬scalar : ∀ {S T t} (s : Scalar S) → Language T (scalar s) → (¬Language (src T) t) +src-¬scalar {T = nil} s p = never +src-¬scalar {T = T ⇒ U} s (scalar ()) +src-¬scalar {T = never} s (scalar ()) +src-¬scalar {T = unknown} s p = never +src-¬scalar {T = boolean} s p = never +src-¬scalar {T = number} s p = never +src-¬scalar {T = string} s p = never +src-¬scalar {T = T ∪ U} s p = srcⁿ-¬scalar s (normal (T ∪ U)) (<:-normalize (T ∪ U) (scalar s) p) +src-¬scalar {T = T ∩ U} s p = srcⁿ-¬scalar s (normal (T ∩ U)) (<:-normalize (T ∩ U) (scalar s) p) + +srcⁿ-unknown-≮: : ∀ {T U} → (Normal U) → (T ≮: srcⁿ U) → (U ≮: (T ⇒ unknown)) +srcⁿ-unknown-≮: never (witness t p q) = CONTRADICTION (language-comp t q unknown) +srcⁿ-unknown-≮: unknown (witness t p q) = witness (function-err t) unknown (function-err p) +srcⁿ-unknown-≮: (U ⇒ V) (witness t p q) = witness (function-err t) (function-err q) (function-err p) +srcⁿ-unknown-≮: (U ∩ V) (witness t p q) = witness (function-err t) (function-err-srcⁿ (U ∩ V) q) (function-err p) +srcⁿ-unknown-≮: (U ∪ V) (witness t p q) = witness (scalar V) (right (scalar V)) (function-scalar V) + +src-unknown-≮: : ∀ {T U} → (T ≮: src U) → (U ≮: (T ⇒ unknown)) +src-unknown-≮: {U = nil} (witness t p q) = witness (scalar nil) (scalar nil) (function-scalar nil) +src-unknown-≮: {U = T ⇒ U} (witness t p q) = witness (function-err t) (function-err q) (function-err p) +src-unknown-≮: {U = never} (witness t p q) = CONTRADICTION (language-comp t q unknown) +src-unknown-≮: {U = unknown} (witness t p q) = witness (function-err t) unknown (function-err p) +src-unknown-≮: {U = boolean} (witness t p q) = witness (scalar boolean) (scalar boolean) (function-scalar boolean) +src-unknown-≮: {U = number} (witness t p q) = witness (scalar number) (scalar number) (function-scalar number) +src-unknown-≮: {U = string} (witness t p q) = witness (scalar string) (scalar string) (function-scalar string) +src-unknown-≮: {U = T ∪ U} p = <:-trans-≮: (normalize-<: (T ∪ U)) (srcⁿ-unknown-≮: (normal (T ∪ U)) p) +src-unknown-≮: {U = T ∩ U} p = <:-trans-≮: (normalize-<: (T ∩ U)) (srcⁿ-unknown-≮: (normal (T ∩ U)) p) + +unknown-src-≮: : ∀ {S T U} → (U ≮: S) → (T ≮: (U ⇒ unknown)) → (U ≮: src T) +unknown-src-≮: (witness t x x₁) (witness (scalar s) p (function-scalar s)) = witness t x (src-¬scalar s p) +unknown-src-≮: r (witness (function-ok s .(scalar s₁)) p (function-ok x (scalar-scalar s₁ () x₂))) +unknown-src-≮: r (witness (function-ok s .function) p (function-ok x (scalar-function ()))) +unknown-src-≮: r (witness (function-ok s .(function-ok _ _)) p (function-ok x (scalar-function-ok ()))) +unknown-src-≮: r (witness (function-ok s .(function-err _)) p (function-ok x (scalar-function-err ()))) +unknown-src-≮: r (witness (function-err t) p (function-err q)) = witness t q (src-¬function-err p) +unknown-src-≮: r (witness (function-tgt t) p (function-tgt (scalar-function-tgt ()))) + +-- Properties of resolve +resolveˢ-<:-⇒ : ∀ {F V U} → (FunType F) → (Saturated F) → (FunType (V ⇒ U)) → (r : Resolved F V) → (F <: (V ⇒ U)) → (target r <: U) +resolveˢ-<:-⇒ Fᶠ Fˢ V⇒Uᶠ r F<:V⇒U with <:-impl-<:ᵒ Fᶠ Fˢ V⇒Uᶠ F<:V⇒U here +resolveˢ-<:-⇒ Fᶠ Fˢ V⇒Uᶠ (yes Sʳ Tʳ oʳ V<:Sʳ tgtʳ) F<:V⇒U | defn o o₁ o₂ = <:-trans (tgtʳ o o₁) o₂ +resolveˢ-<:-⇒ Fᶠ Fˢ V⇒Uᶠ (no tgtʳ) F<:V⇒U | defn o o₁ o₂ = CONTRADICTION (<:-impl-¬≮: o₁ (tgtʳ o)) + +resolveⁿ-<:-⇒ : ∀ {F V U} → (Fⁿ : Normal F) → (Vⁿ : Normal V) → (Uⁿ : Normal U) → (F <: (V ⇒ U)) → (resolveⁿ Fⁿ Vⁿ <: U) +resolveⁿ-<:-⇒ (Sⁿ ⇒ Tⁿ) Vⁿ Uⁿ F<:V⇒U = resolveˢ-<:-⇒ (normal-saturate (Sⁿ ⇒ Tⁿ)) (saturated (Sⁿ ⇒ Tⁿ)) (Vⁿ ⇒ Uⁿ) (resolveˢ (normal-saturate (Sⁿ ⇒ Tⁿ)) (saturated (Sⁿ ⇒ Tⁿ)) Vⁿ (λ o → o)) F<:V⇒U +resolveⁿ-<:-⇒ (Fⁿ ∩ Gⁿ) Vⁿ Uⁿ F<:V⇒U = resolveˢ-<:-⇒ (normal-saturate (Fⁿ ∩ Gⁿ)) (saturated (Fⁿ ∩ Gⁿ)) (Vⁿ ⇒ Uⁿ) (resolveˢ (normal-saturate (Fⁿ ∩ Gⁿ)) (saturated (Fⁿ ∩ Gⁿ)) Vⁿ (λ o → o)) (<:-trans (saturate-<: (Fⁿ ∩ Gⁿ)) F<:V⇒U) +resolveⁿ-<:-⇒ (Sⁿ ∪ Tˢ) Vⁿ Uⁿ F<:V⇒U = CONTRADICTION (<:-impl-¬≮: F<:V⇒U (<:-trans-≮: <:-∪-right (scalar-≮:-function Tˢ))) +resolveⁿ-<:-⇒ never Vⁿ Uⁿ F<:V⇒U = <:-never +resolveⁿ-<:-⇒ unknown Vⁿ Uⁿ F<:V⇒U = CONTRADICTION (<:-impl-¬≮: F<:V⇒U unknown-≮:-function) + +resolve-<:-⇒ : ∀ {F V U} → (F <: (V ⇒ U)) → (resolve F V <: U) +resolve-<:-⇒ {F} {V} {U} F<:V⇒U = <:-trans (resolveⁿ-<:-⇒ (normal F) (normal V) (normal U) (<:-trans (normalize-<: F) (<:-trans F<:V⇒U (<:-normalize (V ⇒ U))))) (normalize-<: U) + +resolve-≮:-⇒ : ∀ {F V U} → (resolve F V ≮: U) → (F ≮: (V ⇒ U)) +resolve-≮:-⇒ {F} {V} {U} FV≮:U with dec-subtyping F (V ⇒ U) +resolve-≮:-⇒ {F} {V} {U} FV≮:U | Left F≮:V⇒U = F≮:V⇒U +resolve-≮:-⇒ {F} {V} {U} FV≮:U | Right F<:V⇒U = CONTRADICTION (<:-impl-¬≮: (resolve-<:-⇒ F<:V⇒U) FV≮:U) + +<:-resolveˢ-⇒ : ∀ {S T V} → (r : Resolved (S ⇒ T) V) → (V <: S) → T <: target r +<:-resolveˢ-⇒ (yes S T here _ _) V<:S = <:-refl +<:-resolveˢ-⇒ (no _) V<:S = <:-unknown + +<:-resolveⁿ-⇒ : ∀ {S T V} → (Sⁿ : Normal S) → (Tⁿ : Normal T) → (Vⁿ : Normal V) → (V <: S) → T <: resolveⁿ (Sⁿ ⇒ Tⁿ) Vⁿ +<:-resolveⁿ-⇒ Sⁿ Tⁿ Vⁿ V<:S = <:-resolveˢ-⇒ (resolveˢ (Sⁿ ⇒ Tⁿ) (saturated (Sⁿ ⇒ Tⁿ)) Vⁿ (λ o → o)) V<:S + +<:-resolve-⇒ : ∀ {S T V} → (V <: S) → T <: resolve (S ⇒ T) V +<:-resolve-⇒ {S} {T} {V} V<:S = <:-trans (<:-normalize T) (<:-resolveⁿ-⇒ (normal S) (normal T) (normal V) (<:-trans (normalize-<: V) (<:-trans V<:S (<:-normalize S)))) + +<:-resolveˢ : ∀ {F G V W} → (r : Resolved F V) → (s : Resolved G W) → (F <:ᵒ G) → (V <: W) → target r <: target s +<:-resolveˢ (yes Sʳ Tʳ oʳ V<:Sʳ tgtʳ) (yes Sˢ Tˢ oˢ W<:Sˢ tgtˢ) F<:G V<:W with F<:G oˢ +<:-resolveˢ (yes Sʳ Tʳ oʳ V<:Sʳ tgtʳ) (yes Sˢ Tˢ oˢ W<:Sˢ tgtˢ) F<:G V<:W | defn o o₁ o₂ = <:-trans (tgtʳ o (<:-trans (<:-trans V<:W W<:Sˢ) o₁)) o₂ +<:-resolveˢ (no r) (yes Sˢ Tˢ oˢ W<:Sˢ tgtˢ) F<:G V<:W with F<:G oˢ +<:-resolveˢ (no r) (yes Sˢ Tˢ oˢ W<:Sˢ tgtˢ) F<:G V<:W | defn o o₁ o₂ = CONTRADICTION (<:-impl-¬≮: (<:-trans V<:W (<:-trans W<:Sˢ o₁)) (r o)) +<:-resolveˢ r (no s) F<:G V<:W = <:-unknown + +<:-resolveᶠ : ∀ {F G V W} → (Fᶠ : FunType F) → (Gᶠ : FunType G) → (Vⁿ : Normal V) → (Wⁿ : Normal W) → (F <: G) → (V <: W) → resolveᶠ Fᶠ Vⁿ <: resolveᶠ Gᶠ Wⁿ +<:-resolveᶠ Fᶠ Gᶠ Vⁿ Wⁿ F<:G V<:W = <:-resolveˢ + (resolveˢ (normal-saturate Fᶠ) (saturated Fᶠ) Vⁿ (λ o → o)) + (resolveˢ (normal-saturate Gᶠ) (saturated Gᶠ) Wⁿ (λ o → o)) + (<:-impl-<:ᵒ (normal-saturate Fᶠ) (saturated Fᶠ) (normal-saturate Gᶠ) (<:-trans (saturate-<: Fᶠ) (<:-trans F<:G (<:-saturate Gᶠ)))) + V<:W + +<:-resolveⁿ : ∀ {F G V W} → (Fⁿ : Normal F) → (Gⁿ : Normal G) → (Vⁿ : Normal V) → (Wⁿ : Normal W) → (F <: G) → (V <: W) → resolveⁿ Fⁿ Vⁿ <: resolveⁿ Gⁿ Wⁿ +<:-resolveⁿ (Rⁿ ⇒ Sⁿ) (Tⁿ ⇒ Uⁿ) Vⁿ Wⁿ F<:G V<:W = <:-resolveᶠ (Rⁿ ⇒ Sⁿ) (Tⁿ ⇒ Uⁿ) Vⁿ Wⁿ F<:G V<:W +<:-resolveⁿ (Rⁿ ⇒ Sⁿ) (Gⁿ ∩ Hⁿ) Vⁿ Wⁿ F<:G V<:W = <:-resolveᶠ (Rⁿ ⇒ Sⁿ) (Gⁿ ∩ Hⁿ) Vⁿ Wⁿ F<:G V<:W +<:-resolveⁿ (Eⁿ ∩ Fⁿ) (Tⁿ ⇒ Uⁿ) Vⁿ Wⁿ F<:G V<:W = <:-resolveᶠ (Eⁿ ∩ Fⁿ) (Tⁿ ⇒ Uⁿ) Vⁿ Wⁿ F<:G V<:W +<:-resolveⁿ (Eⁿ ∩ Fⁿ) (Gⁿ ∩ Hⁿ) Vⁿ Wⁿ F<:G V<:W = <:-resolveᶠ (Eⁿ ∩ Fⁿ) (Gⁿ ∩ Hⁿ) Vⁿ Wⁿ F<:G V<:W +<:-resolveⁿ (Fⁿ ∪ Sˢ) (Tⁿ ⇒ Uⁿ) Vⁿ Wⁿ F<:G V<:W = CONTRADICTION (<:-impl-¬≮: F<:G (≮:-∪-right (scalar-≮:-function Sˢ))) +<:-resolveⁿ unknown (Tⁿ ⇒ Uⁿ) Vⁿ Wⁿ F<:G V<:W = CONTRADICTION (<:-impl-¬≮: F<:G unknown-≮:-function) +<:-resolveⁿ (Fⁿ ∪ Sˢ) (Gⁿ ∩ Hⁿ) Vⁿ Wⁿ F<:G V<:W = CONTRADICTION (<:-impl-¬≮: F<:G (≮:-∪-right (scalar-≮:-fun (Gⁿ ∩ Hⁿ) Sˢ))) +<:-resolveⁿ unknown (Gⁿ ∩ Hⁿ) Vⁿ Wⁿ F<:G V<:W = CONTRADICTION (<:-impl-¬≮: F<:G (unknown-≮:-fun (Gⁿ ∩ Hⁿ))) +<:-resolveⁿ (Rⁿ ⇒ Sⁿ) never Vⁿ Wⁿ F<:G V<:W = CONTRADICTION (<:-impl-¬≮: F<:G (fun-≮:-never (Rⁿ ⇒ Sⁿ))) +<:-resolveⁿ (Eⁿ ∩ Fⁿ) never Vⁿ Wⁿ F<:G V<:W = CONTRADICTION (<:-impl-¬≮: F<:G (fun-≮:-never (Eⁿ ∩ Fⁿ))) +<:-resolveⁿ (Fⁿ ∪ Sˢ) never Vⁿ Wⁿ F<:G V<:W = CONTRADICTION (<:-impl-¬≮: F<:G (≮:-∪-right (scalar-≮:-never Sˢ))) +<:-resolveⁿ unknown never Vⁿ Wⁿ F<:G V<:W = F<:G +<:-resolveⁿ never Gⁿ Vⁿ Wⁿ F<:G V<:W = <:-never +<:-resolveⁿ Fⁿ (Gⁿ ∪ Uˢ) Vⁿ Wⁿ F<:G V<:W = <:-unknown +<:-resolveⁿ Fⁿ unknown Vⁿ Wⁿ F<:G V<:W = <:-unknown + +<:-resolve : ∀ {F G V W} → (F <: G) → (V <: W) → resolve F V <: resolve G W +<:-resolve {F} {G} {V} {W} F<:G V<:W = <:-resolveⁿ (normal F) (normal G) (normal V) (normal W) + (<:-trans (normalize-<: F) (<:-trans F<:G (<:-normalize G))) + (<:-trans (normalize-<: V) (<:-trans V<:W (<:-normalize W))) diff --git a/prototyping/Properties/StrictMode.agda b/prototyping/Properties/StrictMode.agda index 69e9131c..948674b9 100644 --- a/prototyping/Properties/StrictMode.agda +++ b/prototyping/Properties/StrictMode.agda @@ -7,11 +7,11 @@ open import Agda.Builtin.Equality using (_≡_; refl) open import FFI.Data.Either using (Either; Left; Right; mapL; mapR; mapLR; swapLR; cond) open import FFI.Data.Maybe using (Maybe; just; nothing) open import Luau.Heap using (Heap; Object; function_is_end; defn; alloc; ok; next; lookup-not-allocated) renaming (_≡_⊕_↦_ to _≡ᴴ_⊕_↦_; _[_] to _[_]ᴴ; ∅ to ∅ᴴ) +open import Luau.ResolveOverloads using (src; resolve) open import Luau.StrictMode using (Warningᴱ; Warningᴮ; Warningᴼ; Warningᴴ; UnallocatedAddress; UnboundVariable; FunctionCallMismatch; app₁; app₂; BinOpMismatch₁; BinOpMismatch₂; bin₁; bin₂; BlockMismatch; block₁; return; LocalVarMismatch; local₁; local₂; FunctionDefnMismatch; function₁; function₂; heap; expr; block; addr) open import Luau.Substitution using (_[_/_]ᴮ; _[_/_]ᴱ; _[_/_]ᴮunless_; var_[_/_]ᴱwhenever_) -open import Luau.Subtyping using (_≮:_; witness; unknown; never; scalar; function; scalar-function; scalar-function-ok; scalar-function-err; scalar-scalar; function-scalar; function-ok; function-err; left; right; _,_; Tree; Language; ¬Language) +open import Luau.Subtyping using (_<:_; _≮:_; witness; unknown; never; scalar; function; scalar-function; scalar-function-ok; scalar-function-err; scalar-scalar; function-scalar; function-ok; function-err; left; right; _,_; Tree; Language; ¬Language) open import Luau.Syntax using (Expr; yes; var; val; var_∈_; _⟨_⟩∈_; _$_; addr; number; bool; string; binexp; nil; function_is_end; block_is_end; done; return; local_←_; _∙_; fun; arg; name; ==; ~=) -open import Luau.FunctionTypes using (src; tgt) open import Luau.Type using (Type; nil; number; boolean; string; _⇒_; never; unknown; _∩_; _∪_; _≡ᵀ_; _≡ᴹᵀ_) open import Luau.TypeCheck using (_⊢ᴮ_∈_; _⊢ᴱ_∈_; _⊢ᴴᴮ_▷_∈_; _⊢ᴴᴱ_▷_∈_; nil; var; addr; app; function; block; done; return; local; orUnknown; srcBinOp; tgtBinOp) open import Luau.Var using (_≡ⱽ_) @@ -23,8 +23,10 @@ open import Properties.Equality using (_≢_; sym; cong; trans; subst₁) open import Properties.Dec using (Dec; yes; no) open import Properties.Contradiction using (CONTRADICTION; ¬) open import Properties.Functions using (_∘_) -open import Properties.FunctionTypes using (never-tgt-≮:; tgt-never-≮:; src-unknown-≮:; unknown-src-≮:) -open import Properties.Subtyping using (unknown-≮:; ≡-trans-≮:; ≮:-trans-≡; ≮:-trans; ≮:-refl; scalar-≢-impl-≮:; function-≮:-scalar; scalar-≮:-function; function-≮:-never; unknown-≮:-scalar; scalar-≮:-never; unknown-≮:-never) +open import Properties.DecSubtyping using (dec-subtyping) +open import Properties.Subtyping using (unknown-≮:; ≡-trans-≮:; ≮:-trans-≡; ≮:-trans; ≮:-refl; scalar-≢-impl-≮:; function-≮:-scalar; scalar-≮:-function; function-≮:-never; unknown-≮:-scalar; scalar-≮:-never; unknown-≮:-never; <:-refl; <:-unknown; <:-impl-¬≮:) +open import Properties.ResolveOverloads using (src-unknown-≮:; unknown-src-≮:; <:-resolve; resolve-<:-⇒; <:-resolve-⇒) +open import Properties.Subtyping using (unknown-≮:; ≡-trans-≮:; ≮:-trans-≡; ≮:-trans; <:-trans-≮:; ≮:-refl; scalar-≢-impl-≮:; function-≮:-scalar; scalar-≮:-function; function-≮:-never; unknown-≮:-scalar; scalar-≮:-never; unknown-≮:-never; ≡-impl-<:; ≡-trans-<:; <:-trans-≡; ≮:-trans-<:; <:-trans) open import Properties.TypeCheck using (typeOfᴼ; typeOfᴹᴼ; typeOfⱽ; typeOfᴱ; typeOfᴮ; typeCheckᴱ; typeCheckᴮ; typeCheckᴼ; typeCheckᴴ) open import Luau.OpSem using (_⟦_⟧_⟶_; _⊢_⟶*_⊣_; _⊢_⟶ᴮ_⊣_; _⊢_⟶ᴱ_⊣_; app₁; app₂; function; beta; return; block; done; local; subst; binOp₀; binOp₁; binOp₂; refl; step; +; -; *; /; <; >; ==; ~=; <=; >=; ··) open import Luau.RuntimeError using (BinOpError; RuntimeErrorᴱ; RuntimeErrorᴮ; FunctionMismatch; BinOpMismatch₁; BinOpMismatch₂; UnboundVariable; SEGV; app₁; app₂; bin₁; bin₂; block; local; return; +; -; *; /; <; >; <=; >=; ··) @@ -63,51 +65,32 @@ lookup-⊑-nothing {H} a (snoc defn) p with a ≡ᴬ next H lookup-⊑-nothing {H} a (snoc defn) p | yes refl = refl lookup-⊑-nothing {H} a (snoc o) p | no q = trans (lookup-not-allocated o q) p -heap-weakeningᴱ : ∀ Γ H M {H′ U} → (H ⊑ H′) → (typeOfᴱ H′ Γ M ≮: U) → (typeOfᴱ H Γ M ≮: U) -heap-weakeningᴱ Γ H (var x) h p = p -heap-weakeningᴱ Γ H (val nil) h p = p -heap-weakeningᴱ Γ H (val (addr a)) refl p = p -heap-weakeningᴱ Γ H (val (addr a)) (snoc {a = b} q) p with a ≡ᴬ b -heap-weakeningᴱ Γ H (val (addr a)) (snoc {a = a} defn) p | yes refl = unknown-≮: p -heap-weakeningᴱ Γ H (val (addr a)) (snoc {a = b} q) p | no r = ≡-trans-≮: (cong orUnknown (cong typeOfᴹᴼ (lookup-not-allocated q r))) p -heap-weakeningᴱ Γ H (val (number x)) h p = p -heap-weakeningᴱ Γ H (val (bool x)) h p = p -heap-weakeningᴱ Γ H (val (string x)) h p = p -heap-weakeningᴱ Γ H (M $ N) h p = never-tgt-≮: (heap-weakeningᴱ Γ H M h (tgt-never-≮: p)) -heap-weakeningᴱ Γ H (function f ⟨ var x ∈ T ⟩∈ U is B end) h p = p -heap-weakeningᴱ Γ H (block var b ∈ T is B end) h p = p -heap-weakeningᴱ Γ H (binexp M op N) h p = p +<:-heap-weakeningᴱ : ∀ Γ H M {H′} → (H ⊑ H′) → (typeOfᴱ H′ Γ M <: typeOfᴱ H Γ M) +<:-heap-weakeningᴱ Γ H (var x) h = <:-refl +<:-heap-weakeningᴱ Γ H (val nil) h = <:-refl +<:-heap-weakeningᴱ Γ H (val (addr a)) refl = <:-refl +<:-heap-weakeningᴱ Γ H (val (addr a)) (snoc {a = b} q) with a ≡ᴬ b +<:-heap-weakeningᴱ Γ H (val (addr a)) (snoc {a = a} defn) | yes refl = <:-unknown +<:-heap-weakeningᴱ Γ H (val (addr a)) (snoc {a = b} q) | no r = ≡-impl-<: (sym (cong orUnknown (cong typeOfᴹᴼ (lookup-not-allocated q r)))) +<:-heap-weakeningᴱ Γ H (val (number n)) h = <:-refl +<:-heap-weakeningᴱ Γ H (val (bool b)) h = <:-refl +<:-heap-weakeningᴱ Γ H (val (string s)) h = <:-refl +<:-heap-weakeningᴱ Γ H (M $ N) h = <:-resolve (<:-heap-weakeningᴱ Γ H M h) (<:-heap-weakeningᴱ Γ H N h) +<:-heap-weakeningᴱ Γ H (function f ⟨ var x ∈ S ⟩∈ T is B end) h = <:-refl +<:-heap-weakeningᴱ Γ H (block var b ∈ T is N end) h = <:-refl +<:-heap-weakeningᴱ Γ H (binexp M op N) h = <:-refl -heap-weakeningᴮ : ∀ Γ H B {H′ U} → (H ⊑ H′) → (typeOfᴮ H′ Γ B ≮: U) → (typeOfᴮ H Γ B ≮: U) -heap-weakeningᴮ Γ H (function f ⟨ var x ∈ T ⟩∈ U is C end ∙ B) h p = heap-weakeningᴮ (Γ ⊕ f ↦ (T ⇒ U)) H B h p -heap-weakeningᴮ Γ H (local var x ∈ T ← M ∙ B) h p = heap-weakeningᴮ (Γ ⊕ x ↦ T) H B h p -heap-weakeningᴮ Γ H (return M ∙ B) h p = heap-weakeningᴱ Γ H M h p -heap-weakeningᴮ Γ H done h p = p +<:-heap-weakeningᴮ : ∀ Γ H B {H′} → (H ⊑ H′) → (typeOfᴮ H′ Γ B <: typeOfᴮ H Γ B) +<:-heap-weakeningᴮ Γ H (function f ⟨ var x ∈ T ⟩∈ U is C end ∙ B) h = <:-heap-weakeningᴮ (Γ ⊕ f ↦ (T ⇒ U)) H B h +<:-heap-weakeningᴮ Γ H (local var x ∈ T ← M ∙ B) h = <:-heap-weakeningᴮ (Γ ⊕ x ↦ T) H B h +<:-heap-weakeningᴮ Γ H (return M ∙ B) h = <:-heap-weakeningᴱ Γ H M h +<:-heap-weakeningᴮ Γ H done h = <:-refl -substitutivityᴱ : ∀ {Γ T U} H M v x → (typeOfᴱ H Γ (M [ v / x ]ᴱ) ≮: U) → Either (typeOfᴱ H (Γ ⊕ x ↦ T) M ≮: U) (typeOfᴱ H ∅ (val v) ≮: T) -substitutivityᴱ-whenever : ∀ {Γ T U} H v x y (r : Dec(x ≡ y)) → (typeOfᴱ H Γ (var y [ v / x ]ᴱwhenever r) ≮: U) → Either (typeOfᴱ H (Γ ⊕ x ↦ T) (var y) ≮: U) (typeOfᴱ H ∅ (val v) ≮: T) -substitutivityᴮ : ∀ {Γ T U} H B v x → (typeOfᴮ H Γ (B [ v / x ]ᴮ) ≮: U) → Either (typeOfᴮ H (Γ ⊕ x ↦ T) B ≮: U) (typeOfᴱ H ∅ (val v) ≮: T) -substitutivityᴮ-unless : ∀ {Γ T U V} H B v x y (r : Dec(x ≡ y)) → (typeOfᴮ H (Γ ⊕ y ↦ U) (B [ v / x ]ᴮunless r) ≮: V) → Either (typeOfᴮ H ((Γ ⊕ x ↦ T) ⊕ y ↦ U) B ≮: V) (typeOfᴱ H ∅ (val v) ≮: T) -substitutivityᴮ-unless-yes : ∀ {Γ Γ′ T V} H B v x y (r : x ≡ y) → (Γ′ ≡ Γ) → (typeOfᴮ H Γ (B [ v / x ]ᴮunless yes r) ≮: V) → Either (typeOfᴮ H Γ′ B ≮: V) (typeOfᴱ H ∅ (val v) ≮: T) -substitutivityᴮ-unless-no : ∀ {Γ Γ′ T V} H B v x y (r : x ≢ y) → (Γ′ ≡ Γ ⊕ x ↦ T) → (typeOfᴮ H Γ (B [ v / x ]ᴮunless no r) ≮: V) → Either (typeOfᴮ H Γ′ B ≮: V) (typeOfᴱ H ∅ (val v) ≮: T) +≮:-heap-weakeningᴱ : ∀ Γ H M {H′ U} → (H ⊑ H′) → (typeOfᴱ H′ Γ M ≮: U) → (typeOfᴱ H Γ M ≮: U) +≮:-heap-weakeningᴱ Γ H M h p = <:-trans-≮: (<:-heap-weakeningᴱ Γ H M h) p -substitutivityᴱ H (var y) v x p = substitutivityᴱ-whenever H v x y (x ≡ⱽ y) p -substitutivityᴱ H (val w) v x p = Left p -substitutivityᴱ H (binexp M op N) v x p = Left p -substitutivityᴱ H (M $ N) v x p = mapL never-tgt-≮: (substitutivityᴱ H M v x (tgt-never-≮: p)) -substitutivityᴱ H (function f ⟨ var y ∈ T ⟩∈ U is B end) v x p = Left p -substitutivityᴱ H (block var b ∈ T is B end) v x p = Left p -substitutivityᴱ-whenever H v x x (yes refl) q = swapLR (≮:-trans q) -substitutivityᴱ-whenever H v x y (no p) q = Left (≡-trans-≮: (cong orUnknown (sym (⊕-lookup-miss x y _ _ p))) q) - -substitutivityᴮ H (function f ⟨ var y ∈ T ⟩∈ U is C end ∙ B) v x p = substitutivityᴮ-unless H B v x f (x ≡ⱽ f) p -substitutivityᴮ H (local var y ∈ T ← M ∙ B) v x p = substitutivityᴮ-unless H B v x y (x ≡ⱽ y) p -substitutivityᴮ H (return M ∙ B) v x p = substitutivityᴱ H M v x p -substitutivityᴮ H done v x p = Left p -substitutivityᴮ-unless H B v x y (yes p) q = substitutivityᴮ-unless-yes H B v x y p (⊕-over p) q -substitutivityᴮ-unless H B v x y (no p) q = substitutivityᴮ-unless-no H B v x y p (⊕-swap p) q -substitutivityᴮ-unless-yes H B v x y refl refl p = Left p -substitutivityᴮ-unless-no H B v x y p refl q = substitutivityᴮ H B v x q +≮:-heap-weakeningᴮ : ∀ Γ H B {H′ U} → (H ⊑ H′) → (typeOfᴮ H′ Γ B ≮: U) → (typeOfᴮ H Γ B ≮: U) +≮:-heap-weakeningᴮ Γ H B h p = <:-trans-≮: (<:-heap-weakeningᴮ Γ H B h) p binOpPreservation : ∀ H {op v w x} → (v ⟦ op ⟧ w ⟶ x) → (tgtBinOp op ≡ typeOfᴱ H ∅ (val x)) binOpPreservation H (+ m n) = refl @@ -122,24 +105,78 @@ binOpPreservation H (== v w) = refl binOpPreservation H (~= v w) = refl binOpPreservation H (·· v w) = refl -reflect-subtypingᴱ : ∀ H M {H′ M′ T} → (H ⊢ M ⟶ᴱ M′ ⊣ H′) → (typeOfᴱ H′ ∅ M′ ≮: T) → Either (typeOfᴱ H ∅ M ≮: T) (Warningᴱ H (typeCheckᴱ H ∅ M)) -reflect-subtypingᴮ : ∀ H B {H′ B′ T} → (H ⊢ B ⟶ᴮ B′ ⊣ H′) → (typeOfᴮ H′ ∅ B′ ≮: T) → Either (typeOfᴮ H ∅ B ≮: T) (Warningᴮ H (typeCheckᴮ H ∅ B)) +<:-substitutivityᴱ : ∀ {Γ T} H M v x → (typeOfᴱ H ∅ (val v) <: T) → (typeOfᴱ H Γ (M [ v / x ]ᴱ) <: typeOfᴱ H (Γ ⊕ x ↦ T) M) +<:-substitutivityᴱ-whenever : ∀ {Γ T} H v x y (r : Dec(x ≡ y)) → (typeOfᴱ H ∅ (val v) <: T) → (typeOfᴱ H Γ (var y [ v / x ]ᴱwhenever r) <: typeOfᴱ H (Γ ⊕ x ↦ T) (var y)) +<:-substitutivityᴮ : ∀ {Γ T} H B v x → (typeOfᴱ H ∅ (val v) <: T) → (typeOfᴮ H Γ (B [ v / x ]ᴮ) <: typeOfᴮ H (Γ ⊕ x ↦ T) B) +<:-substitutivityᴮ-unless : ∀ {Γ T U} H B v x y (r : Dec(x ≡ y)) → (typeOfᴱ H ∅ (val v) <: T) → (typeOfᴮ H (Γ ⊕ y ↦ U) (B [ v / x ]ᴮunless r) <: typeOfᴮ H ((Γ ⊕ x ↦ T) ⊕ y ↦ U) B) +<:-substitutivityᴮ-unless-yes : ∀ {Γ Γ′} H B v x y (r : x ≡ y) → (Γ′ ≡ Γ) → (typeOfᴮ H Γ (B [ v / x ]ᴮunless yes r) <: typeOfᴮ H Γ′ B) +<:-substitutivityᴮ-unless-no : ∀ {Γ Γ′ T} H B v x y (r : x ≢ y) → (Γ′ ≡ Γ ⊕ x ↦ T) → (typeOfᴱ H ∅ (val v) <: T) → (typeOfᴮ H Γ (B [ v / x ]ᴮunless no r) <: typeOfᴮ H Γ′ B) -reflect-subtypingᴱ H (M $ N) (app₁ s) p = mapLR never-tgt-≮: app₁ (reflect-subtypingᴱ H M s (tgt-never-≮: p)) -reflect-subtypingᴱ H (M $ N) (app₂ v s) p = Left (never-tgt-≮: (heap-weakeningᴱ ∅ H M (rednᴱ⊑ s) (tgt-never-≮: p))) -reflect-subtypingᴱ H (M $ N) (beta (function f ⟨ var y ∈ T ⟩∈ U is B end) v refl q) p = Left (≡-trans-≮: (cong tgt (cong orUnknown (cong typeOfᴹᴼ q))) p) -reflect-subtypingᴱ H (function f ⟨ var x ∈ T ⟩∈ U is B end) (function a defn) p = Left p -reflect-subtypingᴱ H (block var b ∈ T is B end) (block s) p = Left p -reflect-subtypingᴱ H (block var b ∈ T is return (val v) ∙ B end) (return v) p = mapR BlockMismatch (swapLR (≮:-trans p)) -reflect-subtypingᴱ H (block var b ∈ T is done end) done p = mapR BlockMismatch (swapLR (≮:-trans p)) -reflect-subtypingᴱ H (binexp M op N) (binOp₀ s) p = Left (≡-trans-≮: (binOpPreservation H s) p) -reflect-subtypingᴱ H (binexp M op N) (binOp₁ s) p = Left p -reflect-subtypingᴱ H (binexp M op N) (binOp₂ s) p = Left p +<:-substitutivityᴱ H (var y) v x p = <:-substitutivityᴱ-whenever H v x y (x ≡ⱽ y) p +<:-substitutivityᴱ H (val w) v x p = <:-refl +<:-substitutivityᴱ H (binexp M op N) v x p = <:-refl +<:-substitutivityᴱ H (M $ N) v x p = <:-resolve (<:-substitutivityᴱ H M v x p) (<:-substitutivityᴱ H N v x p) +<:-substitutivityᴱ H (function f ⟨ var y ∈ T ⟩∈ U is B end) v x p = <:-refl +<:-substitutivityᴱ H (block var b ∈ T is B end) v x p = <:-refl +<:-substitutivityᴱ-whenever H v x x (yes refl) p = p +<:-substitutivityᴱ-whenever H v x y (no o) p = (≡-impl-<: (cong orUnknown (⊕-lookup-miss x y _ _ o))) -reflect-subtypingᴮ H (function f ⟨ var x ∈ T ⟩∈ U is C end ∙ B) (function a defn) p = mapLR (heap-weakeningᴮ _ _ B (snoc defn)) (CONTRADICTION ∘ ≮:-refl) (substitutivityᴮ _ B (addr a) f p) -reflect-subtypingᴮ H (local var x ∈ T ← M ∙ B) (local s) p = Left (heap-weakeningᴮ (x ↦ T) H B (rednᴱ⊑ s) p) -reflect-subtypingᴮ H (local var x ∈ T ← M ∙ B) (subst v) p = mapR LocalVarMismatch (substitutivityᴮ H B v x p) -reflect-subtypingᴮ H (return M ∙ B) (return s) p = mapR return (reflect-subtypingᴱ H M s p) +<:-substitutivityᴮ H (function f ⟨ var y ∈ T ⟩∈ U is C end ∙ B) v x p = <:-substitutivityᴮ-unless H B v x f (x ≡ⱽ f) p +<:-substitutivityᴮ H (local var y ∈ T ← M ∙ B) v x p = <:-substitutivityᴮ-unless H B v x y (x ≡ⱽ y) p +<:-substitutivityᴮ H (return M ∙ B) v x p = <:-substitutivityᴱ H M v x p +<:-substitutivityᴮ H done v x p = <:-refl +<:-substitutivityᴮ-unless H B v x y (yes r) p = <:-substitutivityᴮ-unless-yes H B v x y r (⊕-over r) +<:-substitutivityᴮ-unless H B v x y (no r) p = <:-substitutivityᴮ-unless-no H B v x y r (⊕-swap r) p +<:-substitutivityᴮ-unless-yes H B v x y refl refl = <:-refl +<:-substitutivityᴮ-unless-no H B v x y r refl p = <:-substitutivityᴮ H B v x p + +≮:-substitutivityᴱ : ∀ {Γ T U} H M v x → (typeOfᴱ H Γ (M [ v / x ]ᴱ) ≮: U) → Either (typeOfᴱ H (Γ ⊕ x ↦ T) M ≮: U) (typeOfᴱ H ∅ (val v) ≮: T) +≮:-substitutivityᴱ {T = T} H M v x p with dec-subtyping (typeOfᴱ H ∅ (val v)) T +≮:-substitutivityᴱ H M v x p | Left q = Right q +≮:-substitutivityᴱ H M v x p | Right q = Left (<:-trans-≮: (<:-substitutivityᴱ H M v x q) p) + +≮:-substitutivityᴮ : ∀ {Γ T U} H B v x → (typeOfᴮ H Γ (B [ v / x ]ᴮ) ≮: U) → Either (typeOfᴮ H (Γ ⊕ x ↦ T) B ≮: U) (typeOfᴱ H ∅ (val v) ≮: T) +≮:-substitutivityᴮ {T = T} H M v x p with dec-subtyping (typeOfᴱ H ∅ (val v)) T +≮:-substitutivityᴮ H M v x p | Left q = Right q +≮:-substitutivityᴮ H M v x p | Right q = Left (<:-trans-≮: (<:-substitutivityᴮ H M v x q) p) + +≮:-substitutivityᴮ-unless : ∀ {Γ T U V} H B v x y (r : Dec(x ≡ y)) → (typeOfᴮ H (Γ ⊕ y ↦ U) (B [ v / x ]ᴮunless r) ≮: V) → Either (typeOfᴮ H ((Γ ⊕ x ↦ T) ⊕ y ↦ U) B ≮: V) (typeOfᴱ H ∅ (val v) ≮: T) +≮:-substitutivityᴮ-unless {T = T} H B v x y r p with dec-subtyping (typeOfᴱ H ∅ (val v)) T +≮:-substitutivityᴮ-unless H B v x y r p | Left q = Right q +≮:-substitutivityᴮ-unless H B v x y r p | Right q = Left (<:-trans-≮: (<:-substitutivityᴮ-unless H B v x y r q) p) + +<:-reductionᴱ : ∀ H M {H′ M′} → (H ⊢ M ⟶ᴱ M′ ⊣ H′) → Either (typeOfᴱ H′ ∅ M′ <: typeOfᴱ H ∅ M) (Warningᴱ H (typeCheckᴱ H ∅ M)) +<:-reductionᴮ : ∀ H B {H′ B′} → (H ⊢ B ⟶ᴮ B′ ⊣ H′) → Either (typeOfᴮ H′ ∅ B′ <: typeOfᴮ H ∅ B) (Warningᴮ H (typeCheckᴮ H ∅ B)) + +<:-reductionᴱ H (M $ N) (app₁ s) = mapLR (λ p → <:-resolve p (<:-heap-weakeningᴱ ∅ H N (rednᴱ⊑ s))) app₁ (<:-reductionᴱ H M s) +<:-reductionᴱ H (M $ N) (app₂ q s) = mapLR (λ p → <:-resolve (<:-heap-weakeningᴱ ∅ H M (rednᴱ⊑ s)) p) app₂ (<:-reductionᴱ H N s) +<:-reductionᴱ H (M $ N) (beta (function f ⟨ var y ∈ S ⟩∈ U is B end) v refl q) with dec-subtyping (typeOfᴱ H ∅ (val v)) S +<:-reductionᴱ H (M $ N) (beta (function f ⟨ var y ∈ S ⟩∈ U is B end) v refl q) | Left r = Right (FunctionCallMismatch (≮:-trans-≡ r (cong src (cong orUnknown (cong typeOfᴹᴼ (sym q)))))) +<:-reductionᴱ H (M $ N) (beta (function f ⟨ var y ∈ S ⟩∈ U is B end) v refl q) | Right r = Left (<:-trans-≡ (<:-resolve-⇒ r) (cong (λ F → resolve F (typeOfᴱ H ∅ N)) (cong orUnknown (cong typeOfᴹᴼ (sym q))))) +<:-reductionᴱ H (function f ⟨ var x ∈ T ⟩∈ U is B end) (function a defn) = Left <:-refl +<:-reductionᴱ H (block var b ∈ T is B end) (block s) = Left <:-refl +<:-reductionᴱ H (block var b ∈ T is return (val v) ∙ B end) (return v) with dec-subtyping (typeOfᴱ H ∅ (val v)) T +<:-reductionᴱ H (block var b ∈ T is return (val v) ∙ B end) (return v) | Left p = Right (BlockMismatch p) +<:-reductionᴱ H (block var b ∈ T is return (val v) ∙ B end) (return v) | Right p = Left p +<:-reductionᴱ H (block var b ∈ T is done end) done with dec-subtyping nil T +<:-reductionᴱ H (block var b ∈ T is done end) done | Left p = Right (BlockMismatch p) +<:-reductionᴱ H (block var b ∈ T is done end) done | Right p = Left p +<:-reductionᴱ H (binexp M op N) (binOp₀ s) = Left (≡-impl-<: (sym (binOpPreservation H s))) +<:-reductionᴱ H (binexp M op N) (binOp₁ s) = Left <:-refl +<:-reductionᴱ H (binexp M op N) (binOp₂ s) = Left <:-refl + +<:-reductionᴮ H (function f ⟨ var x ∈ T ⟩∈ U is C end ∙ B) (function a defn) = Left (<:-trans (<:-substitutivityᴮ _ B (addr a) f <:-refl) (<:-heap-weakeningᴮ (f ↦ (T ⇒ U)) H B (snoc defn))) +<:-reductionᴮ H (local var x ∈ T ← M ∙ B) (local s) = Left (<:-heap-weakeningᴮ (x ↦ T) H B (rednᴱ⊑ s)) +<:-reductionᴮ H (local var x ∈ T ← M ∙ B) (subst v) with dec-subtyping (typeOfᴱ H ∅ (val v)) T +<:-reductionᴮ H (local var x ∈ T ← M ∙ B) (subst v) | Left p = Right (LocalVarMismatch p) +<:-reductionᴮ H (local var x ∈ T ← M ∙ B) (subst v) | Right p = Left (<:-substitutivityᴮ H B v x p) +<:-reductionᴮ H (return M ∙ B) (return s) = mapR return (<:-reductionᴱ H M s) + +≮:-reductionᴱ : ∀ H M {H′ M′ T} → (H ⊢ M ⟶ᴱ M′ ⊣ H′) → (typeOfᴱ H′ ∅ M′ ≮: T) → Either (typeOfᴱ H ∅ M ≮: T) (Warningᴱ H (typeCheckᴱ H ∅ M)) +≮:-reductionᴱ H M s p = mapL (λ q → <:-trans-≮: q p) (<:-reductionᴱ H M s) + +≮:-reductionᴮ : ∀ H B {H′ B′ T} → (H ⊢ B ⟶ᴮ B′ ⊣ H′) → (typeOfᴮ H′ ∅ B′ ≮: T) → Either (typeOfᴮ H ∅ B ≮: T) (Warningᴮ H (typeCheckᴮ H ∅ B)) +≮:-reductionᴮ H B s p = mapL (λ q → <:-trans-≮: q p) (<:-reductionᴮ H B s) reflect-substitutionᴱ : ∀ {Γ T} H M v x → Warningᴱ H (typeCheckᴱ H Γ (M [ v / x ]ᴱ)) → Either (Warningᴱ H (typeCheckᴱ H (Γ ⊕ x ↦ T) M)) (Either (Warningᴱ H (typeCheckᴱ H ∅ (val v))) (typeOfᴱ H ∅ (val v) ≮: T)) reflect-substitutionᴱ-whenever : ∀ {Γ T} H v x y (p : Dec(x ≡ y)) → Warningᴱ H (typeCheckᴱ H Γ (var y [ v / x ]ᴱwhenever p)) → Either (Warningᴱ H (typeCheckᴱ H (Γ ⊕ x ↦ T) (var y))) (Either (Warningᴱ H (typeCheckᴱ H ∅ (val v))) (typeOfᴱ H ∅ (val v) ≮: T)) @@ -150,29 +187,29 @@ reflect-substitutionᴮ-unless-no : ∀ {Γ Γ′ T} H B v x y (r : x ≢ y) → reflect-substitutionᴱ H (var y) v x W = reflect-substitutionᴱ-whenever H v x y (x ≡ⱽ y) W reflect-substitutionᴱ H (val (addr a)) v x (UnallocatedAddress r) = Left (UnallocatedAddress r) -reflect-substitutionᴱ H (M $ N) v x (FunctionCallMismatch p) with substitutivityᴱ H N v x p +reflect-substitutionᴱ H (M $ N) v x (FunctionCallMismatch p) with ≮:-substitutivityᴱ H N v x p reflect-substitutionᴱ H (M $ N) v x (FunctionCallMismatch p) | Right W = Right (Right W) -reflect-substitutionᴱ H (M $ N) v x (FunctionCallMismatch p) | Left q with substitutivityᴱ H M v x (src-unknown-≮: q) +reflect-substitutionᴱ H (M $ N) v x (FunctionCallMismatch p) | Left q with ≮:-substitutivityᴱ H M v x (src-unknown-≮: q) reflect-substitutionᴱ H (M $ N) v x (FunctionCallMismatch p) | Left q | Left r = Left ((FunctionCallMismatch ∘ unknown-src-≮: q) r) reflect-substitutionᴱ H (M $ N) v x (FunctionCallMismatch p) | Left q | Right W = Right (Right W) reflect-substitutionᴱ H (M $ N) v x (app₁ W) = mapL app₁ (reflect-substitutionᴱ H M v x W) reflect-substitutionᴱ H (M $ N) v x (app₂ W) = mapL app₂ (reflect-substitutionᴱ H N v x W) -reflect-substitutionᴱ H (function f ⟨ var y ∈ T ⟩∈ U is B end) v x (FunctionDefnMismatch q) = mapLR FunctionDefnMismatch Right (substitutivityᴮ-unless H B v x y (x ≡ⱽ y) q) +reflect-substitutionᴱ H (function f ⟨ var y ∈ T ⟩∈ U is B end) v x (FunctionDefnMismatch q) = mapLR FunctionDefnMismatch Right (≮:-substitutivityᴮ-unless H B v x y (x ≡ⱽ y) q) reflect-substitutionᴱ H (function f ⟨ var y ∈ T ⟩∈ U is B end) v x (function₁ W) = mapL function₁ (reflect-substitutionᴮ-unless H B v x y (x ≡ⱽ y) W) -reflect-substitutionᴱ H (block var b ∈ T is B end) v x (BlockMismatch q) = mapLR BlockMismatch Right (substitutivityᴮ H B v x q) +reflect-substitutionᴱ H (block var b ∈ T is B end) v x (BlockMismatch q) = mapLR BlockMismatch Right (≮:-substitutivityᴮ H B v x q) reflect-substitutionᴱ H (block var b ∈ T is B end) v x (block₁ W′) = mapL block₁ (reflect-substitutionᴮ H B v x W′) -reflect-substitutionᴱ H (binexp M op N) v x (BinOpMismatch₁ q) = mapLR BinOpMismatch₁ Right (substitutivityᴱ H M v x q) -reflect-substitutionᴱ H (binexp M op N) v x (BinOpMismatch₂ q) = mapLR BinOpMismatch₂ Right (substitutivityᴱ H N v x q) +reflect-substitutionᴱ H (binexp M op N) v x (BinOpMismatch₁ q) = mapLR BinOpMismatch₁ Right (≮:-substitutivityᴱ H M v x q) +reflect-substitutionᴱ H (binexp M op N) v x (BinOpMismatch₂ q) = mapLR BinOpMismatch₂ Right (≮:-substitutivityᴱ H N v x q) reflect-substitutionᴱ H (binexp M op N) v x (bin₁ W) = mapL bin₁ (reflect-substitutionᴱ H M v x W) reflect-substitutionᴱ H (binexp M op N) v x (bin₂ W) = mapL bin₂ (reflect-substitutionᴱ H N v x W) reflect-substitutionᴱ-whenever H a x x (yes refl) (UnallocatedAddress p) = Right (Left (UnallocatedAddress p)) reflect-substitutionᴱ-whenever H v x y (no p) (UnboundVariable q) = Left (UnboundVariable (trans (sym (⊕-lookup-miss x y _ _ p)) q)) -reflect-substitutionᴮ H (function f ⟨ var y ∈ T ⟩∈ U is C end ∙ B) v x (FunctionDefnMismatch q) = mapLR FunctionDefnMismatch Right (substitutivityᴮ-unless H C v x y (x ≡ⱽ y) q) +reflect-substitutionᴮ H (function f ⟨ var y ∈ T ⟩∈ U is C end ∙ B) v x (FunctionDefnMismatch q) = mapLR FunctionDefnMismatch Right (≮:-substitutivityᴮ-unless H C v x y (x ≡ⱽ y) q) reflect-substitutionᴮ H (function f ⟨ var y ∈ T ⟩∈ U is C end ∙ B) v x (function₁ W) = mapL function₁ (reflect-substitutionᴮ-unless H C v x y (x ≡ⱽ y) W) reflect-substitutionᴮ H (function f ⟨ var y ∈ T ⟩∈ U is C end ∙ B) v x (function₂ W) = mapL function₂ (reflect-substitutionᴮ-unless H B v x f (x ≡ⱽ f) W) -reflect-substitutionᴮ H (local var y ∈ T ← M ∙ B) v x (LocalVarMismatch q) = mapLR LocalVarMismatch Right (substitutivityᴱ H M v x q) +reflect-substitutionᴮ H (local var y ∈ T ← M ∙ B) v x (LocalVarMismatch q) = mapLR LocalVarMismatch Right (≮:-substitutivityᴱ H M v x q) reflect-substitutionᴮ H (local var y ∈ T ← M ∙ B) v x (local₁ W) = mapL local₁ (reflect-substitutionᴱ H M v x W) reflect-substitutionᴮ H (local var y ∈ T ← M ∙ B) v x (local₂ W) = mapL local₂ (reflect-substitutionᴮ-unless H B v x y (x ≡ⱽ y) W) reflect-substitutionᴮ H (return M ∙ B) v x (return W) = mapL return (reflect-substitutionᴱ H M v x W) @@ -187,61 +224,61 @@ reflect-weakeningᴮ : ∀ Γ H B {H′} → (H ⊑ H′) → Warningᴮ H′ (t reflect-weakeningᴱ Γ H (var x) h (UnboundVariable p) = (UnboundVariable p) reflect-weakeningᴱ Γ H (val (addr a)) h (UnallocatedAddress p) = UnallocatedAddress (lookup-⊑-nothing a h p) -reflect-weakeningᴱ Γ H (M $ N) h (FunctionCallMismatch p) = FunctionCallMismatch (heap-weakeningᴱ Γ H N h (unknown-src-≮: p (heap-weakeningᴱ Γ H M h (src-unknown-≮: p)))) +reflect-weakeningᴱ Γ H (M $ N) h (FunctionCallMismatch p) = FunctionCallMismatch (≮:-heap-weakeningᴱ Γ H N h (unknown-src-≮: p (≮:-heap-weakeningᴱ Γ H M h (src-unknown-≮: p)))) reflect-weakeningᴱ Γ H (M $ N) h (app₁ W) = app₁ (reflect-weakeningᴱ Γ H M h W) reflect-weakeningᴱ Γ H (M $ N) h (app₂ W) = app₂ (reflect-weakeningᴱ Γ H N h W) -reflect-weakeningᴱ Γ H (binexp M op N) h (BinOpMismatch₁ p) = BinOpMismatch₁ (heap-weakeningᴱ Γ H M h p) -reflect-weakeningᴱ Γ H (binexp M op N) h (BinOpMismatch₂ p) = BinOpMismatch₂ (heap-weakeningᴱ Γ H N h p) +reflect-weakeningᴱ Γ H (binexp M op N) h (BinOpMismatch₁ p) = BinOpMismatch₁ (≮:-heap-weakeningᴱ Γ H M h p) +reflect-weakeningᴱ Γ H (binexp M op N) h (BinOpMismatch₂ p) = BinOpMismatch₂ (≮:-heap-weakeningᴱ Γ H N h p) reflect-weakeningᴱ Γ H (binexp M op N) h (bin₁ W′) = bin₁ (reflect-weakeningᴱ Γ H M h W′) reflect-weakeningᴱ Γ H (binexp M op N) h (bin₂ W′) = bin₂ (reflect-weakeningᴱ Γ H N h W′) -reflect-weakeningᴱ Γ H (function f ⟨ var y ∈ T ⟩∈ U is B end) h (FunctionDefnMismatch p) = FunctionDefnMismatch (heap-weakeningᴮ (Γ ⊕ y ↦ T) H B h p) +reflect-weakeningᴱ Γ H (function f ⟨ var y ∈ T ⟩∈ U is B end) h (FunctionDefnMismatch p) = FunctionDefnMismatch (≮:-heap-weakeningᴮ (Γ ⊕ y ↦ T) H B h p) reflect-weakeningᴱ Γ H (function f ⟨ var y ∈ T ⟩∈ U is B end) h (function₁ W) = function₁ (reflect-weakeningᴮ (Γ ⊕ y ↦ T) H B h W) -reflect-weakeningᴱ Γ H (block var b ∈ T is B end) h (BlockMismatch p) = BlockMismatch (heap-weakeningᴮ Γ H B h p) +reflect-weakeningᴱ Γ H (block var b ∈ T is B end) h (BlockMismatch p) = BlockMismatch (≮:-heap-weakeningᴮ Γ H B h p) reflect-weakeningᴱ Γ H (block var b ∈ T is B end) h (block₁ W) = block₁ (reflect-weakeningᴮ Γ H B h W) reflect-weakeningᴮ Γ H (return M ∙ B) h (return W) = return (reflect-weakeningᴱ Γ H M h W) -reflect-weakeningᴮ Γ H (local var y ∈ T ← M ∙ B) h (LocalVarMismatch p) = LocalVarMismatch (heap-weakeningᴱ Γ H M h p) +reflect-weakeningᴮ Γ H (local var y ∈ T ← M ∙ B) h (LocalVarMismatch p) = LocalVarMismatch (≮:-heap-weakeningᴱ Γ H M h p) reflect-weakeningᴮ Γ H (local var y ∈ T ← M ∙ B) h (local₁ W) = local₁ (reflect-weakeningᴱ Γ H M h W) reflect-weakeningᴮ Γ H (local var y ∈ T ← M ∙ B) h (local₂ W) = local₂ (reflect-weakeningᴮ (Γ ⊕ y ↦ T) H B h W) -reflect-weakeningᴮ Γ H (function f ⟨ var x ∈ T ⟩∈ U is C end ∙ B) h (FunctionDefnMismatch p) = FunctionDefnMismatch (heap-weakeningᴮ (Γ ⊕ x ↦ T) H C h p) +reflect-weakeningᴮ Γ H (function f ⟨ var x ∈ T ⟩∈ U is C end ∙ B) h (FunctionDefnMismatch p) = FunctionDefnMismatch (≮:-heap-weakeningᴮ (Γ ⊕ x ↦ T) H C h p) reflect-weakeningᴮ Γ H (function f ⟨ var x ∈ T ⟩∈ U is C end ∙ B) h (function₁ W) = function₁ (reflect-weakeningᴮ (Γ ⊕ x ↦ T) H C h W) reflect-weakeningᴮ Γ H (function f ⟨ var x ∈ T ⟩∈ U is C end ∙ B) h (function₂ W) = function₂ (reflect-weakeningᴮ (Γ ⊕ f ↦ (T ⇒ U)) H B h W) reflect-weakeningᴼ : ∀ H O {H′} → (H ⊑ H′) → Warningᴼ H′ (typeCheckᴼ H′ O) → Warningᴼ H (typeCheckᴼ H O) -reflect-weakeningᴼ H (just function f ⟨ var x ∈ T ⟩∈ U is B end) h (FunctionDefnMismatch p) = FunctionDefnMismatch (heap-weakeningᴮ (x ↦ T) H B h p) +reflect-weakeningᴼ H (just function f ⟨ var x ∈ T ⟩∈ U is B end) h (FunctionDefnMismatch p) = FunctionDefnMismatch (≮:-heap-weakeningᴮ (x ↦ T) H B h p) reflect-weakeningᴼ H (just function f ⟨ var x ∈ T ⟩∈ U is B end) h (function₁ W) = function₁ (reflect-weakeningᴮ (x ↦ T) H B h W) reflectᴱ : ∀ H M {H′ M′} → (H ⊢ M ⟶ᴱ M′ ⊣ H′) → Warningᴱ H′ (typeCheckᴱ H′ ∅ M′) → Either (Warningᴱ H (typeCheckᴱ H ∅ M)) (Warningᴴ H (typeCheckᴴ H)) reflectᴮ : ∀ H B {H′ B′} → (H ⊢ B ⟶ᴮ B′ ⊣ H′) → Warningᴮ H′ (typeCheckᴮ H′ ∅ B′) → Either (Warningᴮ H (typeCheckᴮ H ∅ B)) (Warningᴴ H (typeCheckᴴ H)) -reflectᴱ H (M $ N) (app₁ s) (FunctionCallMismatch p) = cond (Left ∘ FunctionCallMismatch ∘ heap-weakeningᴱ ∅ H N (rednᴱ⊑ s) ∘ unknown-src-≮: p) (Left ∘ app₁) (reflect-subtypingᴱ H M s (src-unknown-≮: p)) +reflectᴱ H (M $ N) (app₁ s) (FunctionCallMismatch p) = cond (Left ∘ FunctionCallMismatch ∘ ≮:-heap-weakeningᴱ ∅ H N (rednᴱ⊑ s) ∘ unknown-src-≮: p) (Left ∘ app₁) (≮:-reductionᴱ H M s (src-unknown-≮: p)) reflectᴱ H (M $ N) (app₁ s) (app₁ W′) = mapL app₁ (reflectᴱ H M s W′) reflectᴱ H (M $ N) (app₁ s) (app₂ W′) = Left (app₂ (reflect-weakeningᴱ ∅ H N (rednᴱ⊑ s) W′)) -reflectᴱ H (M $ N) (app₂ p s) (FunctionCallMismatch q) = cond (λ r → Left (FunctionCallMismatch (unknown-src-≮: r (heap-weakeningᴱ ∅ H M (rednᴱ⊑ s) (src-unknown-≮: r))))) (Left ∘ app₂) (reflect-subtypingᴱ H N s q) +reflectᴱ H (M $ N) (app₂ p s) (FunctionCallMismatch q) = cond (λ r → Left (FunctionCallMismatch (unknown-src-≮: r (≮:-heap-weakeningᴱ ∅ H M (rednᴱ⊑ s) (src-unknown-≮: r))))) (Left ∘ app₂) (≮:-reductionᴱ H N s q) reflectᴱ H (M $ N) (app₂ p s) (app₁ W′) = Left (app₁ (reflect-weakeningᴱ ∅ H M (rednᴱ⊑ s) W′)) reflectᴱ H (M $ N) (app₂ p s) (app₂ W′) = mapL app₂ (reflectᴱ H N s W′) -reflectᴱ H (val (addr a) $ N) (beta (function f ⟨ var x ∈ T ⟩∈ U is B end) v refl p) (BlockMismatch q) with substitutivityᴮ H B v x q +reflectᴱ H (val (addr a) $ N) (beta (function f ⟨ var x ∈ T ⟩∈ U is B end) v refl p) (BlockMismatch q) with ≮:-substitutivityᴮ H B v x q reflectᴱ H (val (addr a) $ N) (beta (function f ⟨ var x ∈ T ⟩∈ U is B end) v refl p) (BlockMismatch q) | Left r = Right (addr a p (FunctionDefnMismatch r)) reflectᴱ H (val (addr a) $ N) (beta (function f ⟨ var x ∈ T ⟩∈ U is B end) v refl p) (BlockMismatch q) | Right r = Left (FunctionCallMismatch (≮:-trans-≡ r ((cong src (cong orUnknown (cong typeOfᴹᴼ (sym p))))))) reflectᴱ H (val (addr a) $ N) (beta (function f ⟨ var x ∈ T ⟩∈ U is B end) v refl p) (block₁ W′) with reflect-substitutionᴮ _ B v x W′ reflectᴱ H (val (addr a) $ N) (beta (function f ⟨ var x ∈ T ⟩∈ U is B end) v refl p) (block₁ W′) | Left W = Right (addr a p (function₁ W)) reflectᴱ H (val (addr a) $ N) (beta (function f ⟨ var x ∈ T ⟩∈ U is B end) v refl p) (block₁ W′) | Right (Left W) = Left (app₂ W) reflectᴱ H (val (addr a) $ N) (beta (function f ⟨ var x ∈ T ⟩∈ U is B end) v refl p) (block₁ W′) | Right (Right q) = Left (FunctionCallMismatch (≮:-trans-≡ q (cong src (cong orUnknown (cong typeOfᴹᴼ (sym p)))))) -reflectᴱ H (block var b ∈ T is B end) (block s) (BlockMismatch p) = Left (cond BlockMismatch block₁ (reflect-subtypingᴮ H B s p)) +reflectᴱ H (block var b ∈ T is B end) (block s) (BlockMismatch p) = Left (cond BlockMismatch block₁ (≮:-reductionᴮ H B s p)) reflectᴱ H (block var b ∈ T is B end) (block s) (block₁ W′) = mapL block₁ (reflectᴮ H B s W′) reflectᴱ H (block var b ∈ T is B end) (return v) W′ = Left (block₁ (return W′)) reflectᴱ H (function f ⟨ var x ∈ T ⟩∈ U is B end) (function a defn) (UnallocatedAddress ()) reflectᴱ H (binexp M op N) (binOp₀ ()) (UnallocatedAddress p) -reflectᴱ H (binexp M op N) (binOp₁ s) (BinOpMismatch₁ p) = Left (cond BinOpMismatch₁ bin₁ (reflect-subtypingᴱ H M s p)) -reflectᴱ H (binexp M op N) (binOp₁ s) (BinOpMismatch₂ p) = Left (BinOpMismatch₂ (heap-weakeningᴱ ∅ H N (rednᴱ⊑ s) p)) +reflectᴱ H (binexp M op N) (binOp₁ s) (BinOpMismatch₁ p) = Left (cond BinOpMismatch₁ bin₁ (≮:-reductionᴱ H M s p)) +reflectᴱ H (binexp M op N) (binOp₁ s) (BinOpMismatch₂ p) = Left (BinOpMismatch₂ (≮:-heap-weakeningᴱ ∅ H N (rednᴱ⊑ s) p)) reflectᴱ H (binexp M op N) (binOp₁ s) (bin₁ W′) = mapL bin₁ (reflectᴱ H M s W′) reflectᴱ H (binexp M op N) (binOp₁ s) (bin₂ W′) = Left (bin₂ (reflect-weakeningᴱ ∅ H N (rednᴱ⊑ s) W′)) -reflectᴱ H (binexp M op N) (binOp₂ s) (BinOpMismatch₁ p) = Left (BinOpMismatch₁ (heap-weakeningᴱ ∅ H M (rednᴱ⊑ s) p)) -reflectᴱ H (binexp M op N) (binOp₂ s) (BinOpMismatch₂ p) = Left (cond BinOpMismatch₂ bin₂ (reflect-subtypingᴱ H N s p)) +reflectᴱ H (binexp M op N) (binOp₂ s) (BinOpMismatch₁ p) = Left (BinOpMismatch₁ (≮:-heap-weakeningᴱ ∅ H M (rednᴱ⊑ s) p)) +reflectᴱ H (binexp M op N) (binOp₂ s) (BinOpMismatch₂ p) = Left (cond BinOpMismatch₂ bin₂ (≮:-reductionᴱ H N s p)) reflectᴱ H (binexp M op N) (binOp₂ s) (bin₁ W′) = Left (bin₁ (reflect-weakeningᴱ ∅ H M (rednᴱ⊑ s) W′)) reflectᴱ H (binexp M op N) (binOp₂ s) (bin₂ W′) = mapL bin₂ (reflectᴱ H N s W′) -reflectᴮ H (local var x ∈ T ← M ∙ B) (local s) (LocalVarMismatch p) = Left (cond LocalVarMismatch local₁ (reflect-subtypingᴱ H M s p)) +reflectᴮ H (local var x ∈ T ← M ∙ B) (local s) (LocalVarMismatch p) = Left (cond LocalVarMismatch local₁ (≮:-reductionᴱ H M s p)) reflectᴮ H (local var x ∈ T ← M ∙ B) (local s) (local₁ W′) = mapL local₁ (reflectᴱ H M s W′) reflectᴮ H (local var x ∈ T ← M ∙ B) (local s) (local₂ W′) = Left (local₂ (reflect-weakeningᴮ (x ↦ T) H B (rednᴱ⊑ s) W′)) reflectᴮ H (local var x ∈ T ← M ∙ B) (subst v) W′ = Left (cond local₂ (cond local₁ LocalVarMismatch) (reflect-substitutionᴮ H B v x W′)) @@ -258,7 +295,7 @@ reflectᴴᴱ H (M $ N) (app₁ s) W = mapL app₁ (reflectᴴᴱ H M s W) reflectᴴᴱ H (M $ N) (app₂ v s) W = mapL app₂ (reflectᴴᴱ H N s W) reflectᴴᴱ H (M $ N) (beta O v refl p) W = Right W reflectᴴᴱ H (function f ⟨ var x ∈ T ⟩∈ U is B end) (function a p) (addr b refl W) with b ≡ᴬ a -reflectᴴᴱ H (function f ⟨ var x ∈ T ⟩∈ U is B end) (function a defn) (addr b refl (FunctionDefnMismatch p)) | yes refl = Left (FunctionDefnMismatch (heap-weakeningᴮ (x ↦ T) H B (snoc defn) p)) +reflectᴴᴱ H (function f ⟨ var x ∈ T ⟩∈ U is B end) (function a defn) (addr b refl (FunctionDefnMismatch p)) | yes refl = Left (FunctionDefnMismatch (≮:-heap-weakeningᴮ (x ↦ T) H B (snoc defn) p)) reflectᴴᴱ H (function f ⟨ var x ∈ T ⟩∈ U is B end) (function a defn) (addr b refl (function₁ W)) | yes refl = Left (function₁ (reflect-weakeningᴮ (x ↦ T) H B (snoc defn) W)) reflectᴴᴱ H (function f ⟨ var x ∈ T ⟩∈ U is B end) (function a p) (addr b refl W) | no q = Right (addr b (lookup-not-allocated p q) (reflect-weakeningᴼ H _ (snoc p) W)) reflectᴴᴱ H (block var b ∈ T is B end) (block s) W = mapL block₁ (reflectᴴᴮ H B s W) @@ -269,7 +306,7 @@ reflectᴴᴱ H (binexp M op N) (binOp₁ s) W = mapL bin₁ (reflectᴴᴱ H M reflectᴴᴱ H (binexp M op N) (binOp₂ s) W = mapL bin₂ (reflectᴴᴱ H N s W) reflectᴴᴮ H (function f ⟨ var x ∈ T ⟩∈ U is C end ∙ B) (function a p) (addr b refl W) with b ≡ᴬ a -reflectᴴᴮ H (function f ⟨ var x ∈ T ⟩∈ U is C end ∙ B) (function a defn) (addr b refl (FunctionDefnMismatch p)) | yes refl = Left (FunctionDefnMismatch (heap-weakeningᴮ (x ↦ T) H C (snoc defn) p)) +reflectᴴᴮ H (function f ⟨ var x ∈ T ⟩∈ U is C end ∙ B) (function a defn) (addr b refl (FunctionDefnMismatch p)) | yes refl = Left (FunctionDefnMismatch (≮:-heap-weakeningᴮ (x ↦ T) H C (snoc defn) p)) reflectᴴᴮ H (function f ⟨ var x ∈ T ⟩∈ U is C end ∙ B) (function a defn) (addr b refl (function₁ W)) | yes refl = Left (function₁ (reflect-weakeningᴮ (x ↦ T) H C (snoc defn) W)) reflectᴴᴮ H (function f ⟨ var x ∈ T ⟩∈ U is C end ∙ B) (function a p) (addr b refl W) | no q = Right (addr b (lookup-not-allocated p q) (reflect-weakeningᴼ H _ (snoc p) W)) reflectᴴᴮ H (local var x ∈ T ← M ∙ B) (local s) W = mapL local₁ (reflectᴴᴱ H M s W) diff --git a/prototyping/Properties/Subtyping.agda b/prototyping/Properties/Subtyping.agda index 34e6691f..73bf0e9a 100644 --- a/prototyping/Properties/Subtyping.agda +++ b/prototyping/Properties/Subtyping.agda @@ -5,7 +5,7 @@ module Properties.Subtyping where open import Agda.Builtin.Equality using (_≡_; refl) open import FFI.Data.Either using (Either; Left; Right; mapLR; swapLR; cond) open import FFI.Data.Maybe using (Maybe; just; nothing) -open import Luau.Subtyping using (_<:_; _≮:_; Tree; Language; ¬Language; witness; unknown; never; scalar; function; scalar-function; scalar-function-ok; scalar-function-err; scalar-scalar; function-scalar; function-ok; function-err; left; right; _,_) +open import Luau.Subtyping using (_<:_; _≮:_; Tree; Language; ¬Language; witness; unknown; never; scalar; function; scalar-function; scalar-function-ok; scalar-function-err; scalar-function-tgt; scalar-scalar; function-scalar; function-ok; function-ok₁; function-ok₂; function-err; function-tgt; left; right; _,_) open import Luau.Type using (Type; Scalar; nil; number; string; boolean; never; unknown; _⇒_; _∪_; _∩_; skalar) open import Properties.Contradiction using (CONTRADICTION; ¬; ⊥) open import Properties.Equality using (_≢_) @@ -19,37 +19,42 @@ dec-language nil (scalar boolean) = Left (scalar-scalar boolean nil (λ ())) dec-language nil (scalar string) = Left (scalar-scalar string nil (λ ())) dec-language nil (scalar nil) = Right (scalar nil) dec-language nil function = Left (scalar-function nil) -dec-language nil (function-ok t) = Left (scalar-function-ok nil) +dec-language nil (function-ok s t) = Left (scalar-function-ok nil) dec-language nil (function-err t) = Left (scalar-function-err nil) dec-language boolean (scalar number) = Left (scalar-scalar number boolean (λ ())) dec-language boolean (scalar boolean) = Right (scalar boolean) dec-language boolean (scalar string) = Left (scalar-scalar string boolean (λ ())) dec-language boolean (scalar nil) = Left (scalar-scalar nil boolean (λ ())) dec-language boolean function = Left (scalar-function boolean) -dec-language boolean (function-ok t) = Left (scalar-function-ok boolean) +dec-language boolean (function-ok s t) = Left (scalar-function-ok boolean) dec-language boolean (function-err t) = Left (scalar-function-err boolean) dec-language number (scalar number) = Right (scalar number) dec-language number (scalar boolean) = Left (scalar-scalar boolean number (λ ())) dec-language number (scalar string) = Left (scalar-scalar string number (λ ())) dec-language number (scalar nil) = Left (scalar-scalar nil number (λ ())) dec-language number function = Left (scalar-function number) -dec-language number (function-ok t) = Left (scalar-function-ok number) +dec-language number (function-ok s t) = Left (scalar-function-ok number) dec-language number (function-err t) = Left (scalar-function-err number) dec-language string (scalar number) = Left (scalar-scalar number string (λ ())) dec-language string (scalar boolean) = Left (scalar-scalar boolean string (λ ())) dec-language string (scalar string) = Right (scalar string) dec-language string (scalar nil) = Left (scalar-scalar nil string (λ ())) dec-language string function = Left (scalar-function string) -dec-language string (function-ok t) = Left (scalar-function-ok string) +dec-language string (function-ok s t) = Left (scalar-function-ok string) dec-language string (function-err t) = Left (scalar-function-err string) dec-language (T₁ ⇒ T₂) (scalar s) = Left (function-scalar s) dec-language (T₁ ⇒ T₂) function = Right function -dec-language (T₁ ⇒ T₂) (function-ok t) = mapLR function-ok function-ok (dec-language T₂ t) +dec-language (T₁ ⇒ T₂) (function-ok s t) = cond (Right ∘ function-ok₁) (λ p → mapLR (function-ok p) function-ok₂ (dec-language T₂ t)) (dec-language T₁ s) dec-language (T₁ ⇒ T₂) (function-err t) = mapLR function-err function-err (swapLR (dec-language T₁ t)) dec-language never t = Left never dec-language unknown t = Right unknown dec-language (T₁ ∪ T₂) t = cond (λ p → cond (Left ∘ _,_ p) (Right ∘ right) (dec-language T₂ t)) (Right ∘ left) (dec-language T₁ t) dec-language (T₁ ∩ T₂) t = cond (Left ∘ left) (λ p → cond (Left ∘ right) (Right ∘ _,_ p) (dec-language T₂ t)) (dec-language T₁ t) +dec-language nil (function-tgt t) = Left (scalar-function-tgt nil) +dec-language (T₁ ⇒ T₂) (function-tgt t) = mapLR function-tgt function-tgt (dec-language T₂ t) +dec-language boolean (function-tgt t) = Left (scalar-function-tgt boolean) +dec-language number (function-tgt t) = Left (scalar-function-tgt number) +dec-language string (function-tgt t) = Left (scalar-function-tgt string) -- ¬Language T is the complement of Language T language-comp : ∀ {T} t → ¬Language T t → ¬(Language T t) @@ -61,9 +66,12 @@ language-comp (scalar s) (scalar-scalar s p₁ p₂) (scalar s) = p₂ refl language-comp (scalar s) (function-scalar s) (scalar s) = language-comp function (scalar-function s) function language-comp (scalar s) never (scalar ()) language-comp function (scalar-function ()) function -language-comp (function-ok t) (scalar-function-ok ()) (function-ok q) -language-comp (function-ok t) (function-ok p) (function-ok q) = language-comp t p q -language-comp (function-err t) (function-err p) (function-err q) = language-comp t q p +language-comp (function-ok s t) (scalar-function-ok ()) (function-ok₁ p) +language-comp (function-ok s t) (function-ok p₁ p₂) (function-ok₁ q) = language-comp s q p₁ +language-comp (function-ok s t) (function-ok p₁ p₂) (function-ok₂ q) = language-comp t p₂ q +language-comp (function-err t) (function-err p) (function-err q) = language-comp t q p +language-comp (function-tgt t) (scalar-function-tgt ()) (function-tgt q) +language-comp (function-tgt t) (function-tgt p) (function-tgt q) = language-comp t p q -- ≮: is the complement of <: ¬≮:-impl-<: : ∀ {T U} → ¬(T ≮: U) → (T <: U) @@ -90,9 +98,18 @@ language-comp (function-err t) (function-err p) (function-err q) = language-comp ≮:-trans-≡ : ∀ {S T U} → (S ≮: T) → (T ≡ U) → (S ≮: U) ≮:-trans-≡ p refl = p +<:-trans-≡ : ∀ {S T U} → (S <: T) → (T ≡ U) → (S <: U) +<:-trans-≡ p refl = p + +≡-impl-<: : ∀ {T U} → (T ≡ U) → (T <: U) +≡-impl-<: refl = <:-refl + ≡-trans-≮: : ∀ {S T U} → (S ≡ T) → (T ≮: U) → (S ≮: U) ≡-trans-≮: refl p = p +≡-trans-<: : ∀ {S T U} → (S ≡ T) → (T <: U) → (S <: U) +≡-trans-<: refl p = p + ≮:-trans : ∀ {S T U} → (S ≮: U) → Either (S ≮: T) (T ≮: U) ≮:-trans {T = T} (witness t p q) = mapLR (witness t p) (λ z → witness t z q) (dec-language T t) @@ -141,6 +158,12 @@ language-comp (function-err t) (function-err p) (function-err q) = language-comp ≮:-∪-right : ∀ {S T U} → (T ≮: U) → ((S ∪ T) ≮: U) ≮:-∪-right (witness t p q) = witness t (right p) q +≮:-left-∪ : ∀ {S T U} → (S ≮: (T ∪ U)) → (S ≮: T) +≮:-left-∪ (witness t p (q₁ , q₂)) = witness t p q₁ + +≮:-right-∪ : ∀ {S T U} → (S ≮: (T ∪ U)) → (S ≮: U) +≮:-right-∪ (witness t p (q₁ , q₂)) = witness t p q₂ + -- Properties of intersection <:-intersect : ∀ {R S T U} → (R <: T) → (S <: U) → ((R ∩ S) <: (T ∩ U)) @@ -158,6 +181,12 @@ language-comp (function-err t) (function-err p) (function-err q) = language-comp <:-∩-symm : ∀ {T U} → (T ∩ U) <: (U ∩ T) <:-∩-symm t (p₁ , p₂) = (p₂ , p₁) +<:-∩-assocl : ∀ {S T U} → (S ∩ (T ∩ U)) <: ((S ∩ T) ∩ U) +<:-∩-assocl t (p , (p₁ , p₂)) = (p , p₁) , p₂ + +<:-∩-assocr : ∀ {S T U} → ((S ∩ T) ∩ U) <: (S ∩ (T ∩ U)) +<:-∩-assocr t ((p , p₁) , p₂) = p , (p₁ , p₂) + ≮:-∩-left : ∀ {S T U} → (S ≮: T) → (S ≮: (T ∩ U)) ≮:-∩-left (witness t p q) = witness t p (left q) @@ -199,47 +228,84 @@ language-comp (function-err t) (function-err p) (function-err q) = language-comp ∪-distr-∩-<: t (left p₁ , right p₂) = right p₂ ∪-distr-∩-<: t (right p₁ , p₂) = right p₁ +∩-<:-∪ : ∀ {S T} → (S ∩ T) <: (S ∪ T) +∩-<:-∪ t (p , _) = left p + -- Properties of functions <:-function : ∀ {R S T U} → (R <: S) → (T <: U) → (S ⇒ T) <: (R ⇒ U) <:-function p q function function = function -<:-function p q (function-ok t) (function-ok r) = function-ok (q t r) +<:-function p q (function-ok s t) (function-ok₁ r) = function-ok₁ (<:-impl-⊇ p s r) +<:-function p q (function-ok s t) (function-ok₂ r) = function-ok₂ (q t r) <:-function p q (function-err s) (function-err r) = function-err (<:-impl-⊇ p s r) +<:-function p q (function-tgt t) (function-tgt r) = function-tgt (q t r) + +<:-function-∩-∩ : ∀ {R S T U} → ((R ⇒ T) ∩ (S ⇒ U)) <: ((R ∩ S) ⇒ (T ∩ U)) +<:-function-∩-∩ function (function , function) = function +<:-function-∩-∩ (function-ok s t) (function-ok₁ p , q) = function-ok₁ (left p) +<:-function-∩-∩ (function-ok s t) (function-ok₂ p , function-ok₁ q) = function-ok₁ (right q) +<:-function-∩-∩ (function-ok s t) (function-ok₂ p , function-ok₂ q) = function-ok₂ (p , q) +<:-function-∩-∩ (function-err s) (function-err p , q) = function-err (left p) +<:-function-∩-∩ (function-tgt s) (function-tgt p , function-tgt q) = function-tgt (p , q) <:-function-∩-∪ : ∀ {R S T U} → ((R ⇒ T) ∩ (S ⇒ U)) <: ((R ∪ S) ⇒ (T ∪ U)) <:-function-∩-∪ function (function , function) = function -<:-function-∩-∪ (function-ok t) (function-ok p₁ , function-ok p₂) = function-ok (right p₂) -<:-function-∩-∪ (function-err _) (function-err p₁ , function-err q₂) = function-err (p₁ , q₂) +<:-function-∩-∪ (function-ok s t) (function-ok₁ p₁ , function-ok₁ p₂) = function-ok₁ (p₁ , p₂) +<:-function-∩-∪ (function-ok s t) (p₁ , function-ok₂ p₂) = function-ok₂ (right p₂) +<:-function-∩-∪ (function-ok s t) (function-ok₂ p₁ , p₂) = function-ok₂ (left p₁) +<:-function-∩-∪ (function-err s) (function-err p₁ , function-err q₂) = function-err (p₁ , q₂) +<:-function-∩-∪ (function-tgt t) (function-tgt p , q) = function-tgt (left p) <:-function-∩ : ∀ {S T U} → ((S ⇒ T) ∩ (S ⇒ U)) <: (S ⇒ (T ∩ U)) <:-function-∩ function (function , function) = function -<:-function-∩ (function-ok t) (function-ok p₁ , function-ok p₂) = function-ok (p₁ , p₂) +<:-function-∩ (function-ok s t) (p₁ , function-ok₁ p₂) = function-ok₁ p₂ +<:-function-∩ (function-ok s t) (function-ok₁ p₁ , p₂) = function-ok₁ p₁ +<:-function-∩ (function-ok s t) (function-ok₂ p₁ , function-ok₂ p₂) = function-ok₂ (p₁ , p₂) <:-function-∩ (function-err s) (function-err p₁ , function-err p₂) = function-err p₂ +<:-function-∩ (function-tgt t) (function-tgt p₁ , function-tgt p₂) = function-tgt (p₁ , p₂) <:-function-∪ : ∀ {R S T U} → ((R ⇒ S) ∪ (T ⇒ U)) <: ((R ∩ T) ⇒ (S ∪ U)) <:-function-∪ function (left function) = function -<:-function-∪ (function-ok t) (left (function-ok p)) = function-ok (left p) +<:-function-∪ (function-ok s t) (left (function-ok₁ p)) = function-ok₁ (left p) +<:-function-∪ (function-ok s t) (left (function-ok₂ p)) = function-ok₂ (left p) <:-function-∪ (function-err s) (left (function-err p)) = function-err (left p) <:-function-∪ (scalar s) (left (scalar ())) <:-function-∪ function (right function) = function -<:-function-∪ (function-ok t) (right (function-ok p)) = function-ok (right p) +<:-function-∪ (function-ok s t) (right (function-ok₁ p)) = function-ok₁ (right p) +<:-function-∪ (function-ok s t) (right (function-ok₂ p)) = function-ok₂ (right p) <:-function-∪ (function-err s) (right (function-err x)) = function-err (right x) <:-function-∪ (scalar s) (right (scalar ())) +<:-function-∪ (function-tgt t) (left (function-tgt p)) = function-tgt (left p) +<:-function-∪ (function-tgt t) (right (function-tgt p)) = function-tgt (right p) <:-function-∪-∩ : ∀ {R S T U} → ((R ∩ S) ⇒ (T ∪ U)) <: ((R ⇒ T) ∪ (S ⇒ U)) <:-function-∪-∩ function function = left function -<:-function-∪-∩ (function-ok t) (function-ok (left p)) = left (function-ok p) -<:-function-∪-∩ (function-ok t) (function-ok (right p)) = right (function-ok p) +<:-function-∪-∩ (function-ok s t) (function-ok₁ (left p)) = left (function-ok₁ p) +<:-function-∪-∩ (function-ok s t) (function-ok₂ (left p)) = left (function-ok₂ p) +<:-function-∪-∩ (function-ok s t) (function-ok₁ (right p)) = right (function-ok₁ p) +<:-function-∪-∩ (function-ok s t) (function-ok₂ (right p)) = right (function-ok₂ p) <:-function-∪-∩ (function-err s) (function-err (left p)) = left (function-err p) <:-function-∪-∩ (function-err s) (function-err (right p)) = right (function-err p) +<:-function-∪-∩ (function-tgt t) (function-tgt (left p)) = left (function-tgt p) +<:-function-∪-∩ (function-tgt t) (function-tgt (right p)) = right (function-tgt p) + +<:-function-left : ∀ {R S T U} → (S ⇒ T) <: (R ⇒ U) → (R <: S) +<:-function-left {R} {S} p s Rs with dec-language S s +<:-function-left p s Rs | Right Ss = Ss +<:-function-left p s Rs | Left ¬Ss with p (function-err s) (function-err ¬Ss) +<:-function-left p s Rs | Left ¬Ss | function-err ¬Rs = CONTRADICTION (language-comp s ¬Rs Rs) + +<:-function-right : ∀ {R S T U} → (S ⇒ T) <: (R ⇒ U) → (T <: U) +<:-function-right p t Tt with p (function-tgt t) (function-tgt Tt) +<:-function-right p t Tt | function-tgt St = St ≮:-function-left : ∀ {R S T U} → (R ≮: S) → (S ⇒ T) ≮: (R ⇒ U) ≮:-function-left (witness t p q) = witness (function-err t) (function-err q) (function-err p) ≮:-function-right : ∀ {R S T U} → (T ≮: U) → (S ⇒ T) ≮: (R ⇒ U) -≮:-function-right (witness t p q) = witness (function-ok t) (function-ok p) (function-ok q) +≮:-function-right (witness t p q) = witness (function-tgt t) (function-tgt p) (function-tgt q) -- Properties of scalars -skalar-function-ok : ∀ {t} → (¬Language skalar (function-ok t)) +skalar-function-ok : ∀ {s t} → (¬Language skalar (function-ok s t)) skalar-function-ok = (scalar-function-ok number , (scalar-function-ok string , (scalar-function-ok nil , scalar-function-ok boolean))) scalar-<: : ∀ {S T} → (s : Scalar S) → Language T (scalar s) → (S <: T) @@ -261,7 +327,7 @@ scalar-≮:-function : ∀ {S T U} → (Scalar U) → (U ≮: (S ⇒ T)) scalar-≮:-function s = witness (scalar s) (scalar s) (function-scalar s) unknown-≮:-scalar : ∀ {U} → (Scalar U) → (unknown ≮: U) -unknown-≮:-scalar s = witness (function-ok (scalar s)) unknown (scalar-function-ok s) +unknown-≮:-scalar s = witness function unknown (scalar-function s) scalar-≮:-never : ∀ {U} → (Scalar U) → (U ≮: never) scalar-≮:-never s = witness (scalar s) (scalar s) never @@ -288,6 +354,9 @@ never-≮: (witness t p q) = witness t p never unknown-≮:-never : (unknown ≮: never) unknown-≮:-never = witness (scalar nil) unknown never +unknown-≮:-function : ∀ {S T} → (unknown ≮: (S ⇒ T)) +unknown-≮:-function = witness (scalar nil) unknown (function-scalar nil) + function-≮:-never : ∀ {T U} → ((T ⇒ U) ≮: never) function-≮:-never = witness function function never @@ -310,8 +379,9 @@ function-≮:-never = witness function function never <:-everything : unknown <: ((never ⇒ unknown) ∪ skalar) <:-everything (scalar s) p = right (skalar-scalar s) <:-everything function p = left function -<:-everything (function-ok t) p = left (function-ok unknown) +<:-everything (function-ok s t) p = left (function-ok₁ never) <:-everything (function-err s) p = left (function-err never) +<:-everything (function-tgt t) p = left (function-tgt unknown) -- A Gentle Introduction To Semantic Subtyping (https://www.cduce.org/papers/gentle.pdf) -- defines a "set-theoretic" model (sec 2.5) @@ -351,8 +421,9 @@ set-theoretic-if {S₁} {T₁} {S₂} {T₂} p Q q (t , just u) Qtu (S₂t , ¬T S₁t | Right r = r ¬T₁u : ¬(Language T₁ u) - ¬T₁u T₁u with p (function-ok u) (function-ok T₁u) - ¬T₁u T₁u | function-ok T₂u = ¬T₂u T₂u + ¬T₁u T₁u with p (function-ok t u) (function-ok₂ T₁u) + ¬T₁u T₁u | function-ok₁ ¬S₂t = language-comp t ¬S₂t S₂t + ¬T₁u T₁u | function-ok₂ T₂u = ¬T₂u T₂u set-theoretic-if {S₁} {T₁} {S₂} {T₂} p Q q (t , nothing) Qt- (S₂t , _) = q (t , nothing) Qt- (S₁t , λ ()) where @@ -365,33 +436,41 @@ set-theoretic-if {S₁} {T₁} {S₂} {T₂} p Q q (t , nothing) Qt- (S₂t , _) not-quite-set-theoretic-only-if : ∀ {S₁ T₁ S₂ T₂} → -- We don't quite have that this is a set-theoretic model - -- it's only true when Language T₁ and ¬Language T₂ t₂ are inhabited - -- in particular it's not true when T₁ is never, or T₂ is unknown. - ∀ s₂ t₂ → Language S₂ s₂ → ¬Language T₂ t₂ → + -- it's only true when Language S₂ is inhabited + -- in particular it's not true when S₂ is never, + ∀ s₂ → Language S₂ s₂ → -- This is the "only if" part of being a set-theoretic model (∀ Q → Q ⊆ Comp((Language S₁) ⊗ Comp(Lift(Language T₁))) → Q ⊆ Comp((Language S₂) ⊗ Comp(Lift(Language T₂)))) → (Language (S₁ ⇒ T₁) ⊆ Language (S₂ ⇒ T₂)) -not-quite-set-theoretic-only-if {S₁} {T₁} {S₂} {T₂} s₂ t₂ S₂s₂ ¬T₂t₂ p = r where +not-quite-set-theoretic-only-if {S₁} {T₁} {S₂} {T₂} s₂ S₂s₂ p = r where Q : (Tree × Maybe Tree) → Set Q (t , just u) = Either (¬Language S₁ t) (Language T₁ u) Q (t , nothing) = ¬Language S₁ t - - q : Q ⊆ Comp((Language S₁) ⊗ Comp(Lift(Language T₁))) + + q : Q ⊆ Comp(Language S₁ ⊗ Comp(Lift(Language T₁))) q (t , just u) (Left ¬S₁t) (S₁t , ¬T₁u) = language-comp t ¬S₁t S₁t q (t , just u) (Right T₂u) (S₁t , ¬T₁u) = ¬T₁u T₂u q (t , nothing) ¬S₁t (S₁t , _) = language-comp t ¬S₁t S₁t - + r : Language (S₁ ⇒ T₁) ⊆ Language (S₂ ⇒ T₂) r function function = function r (function-err s) (function-err ¬S₁s) with dec-language S₂ s r (function-err s) (function-err ¬S₁s) | Left ¬S₂s = function-err ¬S₂s r (function-err s) (function-err ¬S₁s) | Right S₂s = CONTRADICTION (p Q q (s , nothing) ¬S₁s (S₂s , λ ())) - r (function-ok t) (function-ok T₁t) with dec-language T₂ t - r (function-ok t) (function-ok T₁t) | Left ¬T₂t = CONTRADICTION (p Q q (s₂ , just t) (Right T₁t) (S₂s₂ , language-comp t ¬T₂t)) - r (function-ok t) (function-ok T₁t) | Right T₂t = function-ok T₂t + r (function-ok s t) (function-ok₁ ¬S₁s) with dec-language S₂ s + r (function-ok s t) (function-ok₁ ¬S₁s) | Left ¬S₂s = function-ok₁ ¬S₂s + r (function-ok s t) (function-ok₁ ¬S₁s) | Right S₂s = CONTRADICTION (p Q q (s , nothing) ¬S₁s (S₂s , λ ())) + r (function-ok s t) (function-ok₂ T₁t) with dec-language T₂ t + r (function-ok s t) (function-ok₂ T₁t) | Left ¬T₂t with dec-language S₂ s + r (function-ok s t) (function-ok₂ T₁t) | Left ¬T₂t | Left ¬S₂s = function-ok₁ ¬S₂s + r (function-ok s t) (function-ok₂ T₁t) | Left ¬T₂t | Right S₂s = CONTRADICTION (p Q q (s , just t) (Right T₁t) (S₂s , language-comp t ¬T₂t)) + r (function-ok s t) (function-ok₂ T₁t) | Right T₂t = function-ok₂ T₂t + r (function-tgt t) (function-tgt T₁t) with dec-language T₂ t + r (function-tgt t) (function-tgt T₁t) | Left ¬T₂t = CONTRADICTION (p Q q (s₂ , just t) (Right T₁t) (S₂s₂ , language-comp t ¬T₂t)) + r (function-tgt t) (function-tgt T₁t) | Right T₂t = function-tgt T₂t -- A counterexample when the argument type is empty. @@ -399,22 +478,4 @@ set-theoretic-counterexample-one : (∀ Q → Q ⊆ Comp((Language never) ⊗ Co set-theoretic-counterexample-one Q q ((scalar s) , u) Qtu (scalar () , p) set-theoretic-counterexample-two : (never ⇒ number) ≮: (never ⇒ string) -set-theoretic-counterexample-two = witness - (function-ok (scalar number)) (function-ok (scalar number)) - (function-ok (scalar-scalar number string (λ ()))) - --- At some point we may deal with overloaded function resolution, which should fix this problem... --- The reason why this is connected to overloaded functions is that currently we have that the type of --- f(x) is (tgt T) where f:T. Really we should have the type depend on the type of x, that is use (tgt T U), --- where U is the type of x. In particular (tgt (S => T) (U & V)) should be the same as (tgt ((S&U) => T) V) --- and tgt(never => T) should be unknown. For example --- --- tgt((number => string) & (string => bool))(number) --- is tgt(number => string)(number) & tgt(string => bool)(number) --- is tgt(number => string)(number) & tgt(string => bool)(number&unknown) --- is tgt(number => string)(number) & tgt(string&number => bool)(unknown) --- is tgt(number => string)(number) & tgt(never => bool)(unknown) --- is string & unknown --- is string --- --- there's some discussion of this in the Gentle Introduction paper. +set-theoretic-counterexample-two = witness (function-tgt (scalar number)) (function-tgt (scalar number)) (function-tgt (scalar-scalar number string (λ ()))) diff --git a/prototyping/Properties/TypeCheck.agda b/prototyping/Properties/TypeCheck.agda index 37fbeda5..b53bbd04 100644 --- a/prototyping/Properties/TypeCheck.agda +++ b/prototyping/Properties/TypeCheck.agda @@ -6,9 +6,9 @@ open import Agda.Builtin.Equality using (_≡_; refl) open import Agda.Builtin.Bool using (Bool; true; false) open import FFI.Data.Maybe using (Maybe; just; nothing) open import FFI.Data.Either using (Either) +open import Luau.ResolveOverloads using (resolve) open import Luau.TypeCheck using (_⊢ᴱ_∈_; _⊢ᴮ_∈_; ⊢ᴼ_; ⊢ᴴ_; _⊢ᴴᴱ_▷_∈_; _⊢ᴴᴮ_▷_∈_; nil; var; addr; number; bool; string; app; function; block; binexp; done; return; local; nothing; orUnknown; tgtBinOp) open import Luau.Syntax using (Block; Expr; Value; BinaryOperator; yes; nil; addr; number; bool; string; val; var; binexp; _$_; function_is_end; block_is_end; _∙_; return; done; local_←_; _⟨_⟩; _⟨_⟩∈_; var_∈_; name; fun; arg; +; -; *; /; <; >; ==; ~=; <=; >=) -open import Luau.FunctionTypes using (src; tgt) open import Luau.Type using (Type; nil; unknown; never; number; boolean; string; _⇒_) open import Luau.RuntimeType using (RuntimeType; nil; number; function; string; valueType) open import Luau.VarCtxt using (VarCtxt; ∅; _↦_; _⊕_↦_; _⋒_; _⊝_) renaming (_[_] to _[_]ⱽ) @@ -40,7 +40,7 @@ typeOfᴮ : Heap yes → VarCtxt → (Block yes) → Type typeOfᴱ H Γ (var x) = orUnknown(Γ [ x ]ⱽ) typeOfᴱ H Γ (val v) = orUnknown(typeOfⱽ H v) -typeOfᴱ H Γ (M $ N) = tgt(typeOfᴱ H Γ M) +typeOfᴱ H Γ (M $ N) = resolve (typeOfᴱ H Γ M) (typeOfᴱ H Γ N) typeOfᴱ H Γ (function f ⟨ var x ∈ S ⟩∈ T is B end) = S ⇒ T typeOfᴱ H Γ (block var b ∈ T is B end) = T typeOfᴱ H Γ (binexp M op N) = tgtBinOp op @@ -50,14 +50,6 @@ typeOfᴮ H Γ (local var x ∈ T ← M ∙ B) = typeOfᴮ H (Γ ⊕ x ↦ T) B typeOfᴮ H Γ (return M ∙ B) = typeOfᴱ H Γ M typeOfᴮ H Γ done = nil -mustBeFunction : ∀ H Γ v → (never ≢ src (typeOfᴱ H Γ (val v))) → (function ≡ valueType(v)) -mustBeFunction H Γ nil p = CONTRADICTION (p refl) -mustBeFunction H Γ (addr a) p = refl -mustBeFunction H Γ (number n) p = CONTRADICTION (p refl) -mustBeFunction H Γ (bool true) p = CONTRADICTION (p refl) -mustBeFunction H Γ (bool false) p = CONTRADICTION (p refl) -mustBeFunction H Γ (string x) p = CONTRADICTION (p refl) - mustBeNumber : ∀ H Γ v → (typeOfᴱ H Γ (val v) ≡ number) → (valueType(v) ≡ number) mustBeNumber H Γ (addr a) p with remember (H [ a ]ᴴ) mustBeNumber H Γ (addr a) p | (just O , q) with trans (cong orUnknown (cong typeOfᴹᴼ (sym q))) p diff --git a/prototyping/Properties/TypeNormalization.agda b/prototyping/Properties/TypeNormalization.agda index 299f648c..cbd8139f 100644 --- a/prototyping/Properties/TypeNormalization.agda +++ b/prototyping/Properties/TypeNormalization.agda @@ -3,12 +3,12 @@ module Properties.TypeNormalization where open import Luau.Type using (Type; Scalar; nil; number; string; boolean; never; unknown; _⇒_; _∪_; _∩_) -open import Luau.Subtyping using (scalar-function-err) +open import Luau.Subtyping using (Tree; Language; ¬Language; function; scalar; unknown; left; right; function-ok₁; function-ok₂; function-err; function-tgt; scalar-function; scalar-function-ok; scalar-function-err; scalar-function-tgt; function-scalar; _,_) open import Luau.TypeNormalization using (_∪ⁿ_; _∩ⁿ_; _∪ᶠ_; _∪ⁿˢ_; _∩ⁿˢ_; normalize) -open import Luau.Subtyping using (_<:_) +open import Luau.Subtyping using (_<:_; _≮:_; witness; never) open import Properties.Subtyping using (<:-trans; <:-refl; <:-unknown; <:-never; <:-∪-left; <:-∪-right; <:-∪-lub; <:-∩-left; <:-∩-right; <:-∩-glb; <:-∩-symm; <:-function; <:-function-∪-∩; <:-function-∩-∪; <:-function-∪; <:-everything; <:-union; <:-∪-assocl; <:-∪-assocr; <:-∪-symm; <:-intersect; ∪-distl-∩-<:; ∪-distr-∩-<:; <:-∪-distr-∩; <:-∪-distl-∩; ∩-distl-∪-<:; <:-∩-distl-∪; <:-∩-distr-∪; scalar-∩-function-<:-never; scalar-≢-∩-<:-never) --- Notmal forms for types +-- Normal forms for types data FunType : Type → Set data Normal : Type → Set @@ -17,11 +17,11 @@ data FunType where _∩_ : ∀ {F G} → FunType F → FunType G → FunType (F ∩ G) data Normal where - never : Normal never - unknown : Normal unknown _⇒_ : ∀ {S T} → Normal S → Normal T → Normal (S ⇒ T) _∩_ : ∀ {F G} → FunType F → FunType G → Normal (F ∩ G) _∪_ : ∀ {S T} → Normal S → Scalar T → Normal (S ∪ T) + never : Normal never + unknown : Normal unknown data OptScalar : Type → Set where never : OptScalar never @@ -30,6 +30,38 @@ data OptScalar : Type → Set where string : OptScalar string nil : OptScalar nil +-- Top function type +fun-top : ∀ {F} → (FunType F) → (F <: (never ⇒ unknown)) +fun-top (S ⇒ T) = <:-function <:-never <:-unknown +fun-top (F ∩ G) = <:-trans <:-∩-left (fun-top F) + +-- function types are inhabited +fun-function : ∀ {F} → FunType F → Language F function +fun-function (S ⇒ T) = function +fun-function (F ∩ G) = (fun-function F , fun-function G) + +fun-≮:-never : ∀ {F} → FunType F → (F ≮: never) +fun-≮:-never F = witness function (fun-function F) never + +-- function types aren't scalars +fun-¬scalar : ∀ {F S t} → (s : Scalar S) → FunType F → Language F t → ¬Language S t +fun-¬scalar s (S ⇒ T) function = scalar-function s +fun-¬scalar s (S ⇒ T) (function-ok₁ p) = scalar-function-ok s +fun-¬scalar s (S ⇒ T) (function-ok₂ p) = scalar-function-ok s +fun-¬scalar s (S ⇒ T) (function-err p) = scalar-function-err s +fun-¬scalar s (S ⇒ T) (function-tgt p) = scalar-function-tgt s +fun-¬scalar s (F ∩ G) (p₁ , p₂) = fun-¬scalar s G p₂ + +¬scalar-fun : ∀ {F S} → FunType F → (s : Scalar S) → ¬Language F (scalar s) +¬scalar-fun (S ⇒ T) s = function-scalar s +¬scalar-fun (F ∩ G) s = left (¬scalar-fun F s) + +scalar-≮:-fun : ∀ {F S} → FunType F → Scalar S → S ≮: F +scalar-≮:-fun F s = witness (scalar s) (scalar s) (¬scalar-fun F s) + +unknown-≮:-fun : ∀ {F} → FunType F → unknown ≮: F +unknown-≮:-fun F = witness (scalar nil) unknown (¬scalar-fun F nil) + -- Normalization produces normal types normal : ∀ T → Normal (normalize T) normalᶠ : ∀ {F} → FunType F → Normal F @@ -40,7 +72,7 @@ normal-∩ⁿˢ : ∀ {S T} → Normal S → Scalar T → OptScalar (S ∩ⁿˢ normal-∪ᶠ : ∀ {F G} → FunType F → FunType G → FunType (F ∪ᶠ G) normal nil = never ∪ nil -normal (S ⇒ T) = normalᶠ ((normal S) ⇒ (normal T)) +normal (S ⇒ T) = (normal S) ⇒ (normal T) normal never = never normal unknown = unknown normal boolean = never ∪ boolean @@ -338,7 +370,7 @@ flipper = <:-trans <:-∪-assocr (<:-trans (<:-union <:-refl <:-∪-symm) <:-∪ ∪-<:-∪ⁿ unknown (T ⇒ U) = <:-unknown ∪-<:-∪ⁿ (R ⇒ S) (T ⇒ U) = ∪-<:-∪ᶠ (R ⇒ S) (T ⇒ U) ∪-<:-∪ⁿ (R ∩ S) (T ⇒ U) = ∪-<:-∪ᶠ (R ∩ S) (T ⇒ U) -∪-<:-∪ⁿ (R ∪ S) (T ⇒ U) = <:-trans <:-∪-assocr (<:-trans (<:-union <:-refl <:-∪-symm) (<:-trans <:-∪-assocl (<:-union (∪-<:-∪ⁿ R (T ⇒ U)) <:-refl))) +∪-<:-∪ⁿ (R ∪ S) (T ⇒ U) = <:-trans <:-∪-assocr (<:-trans (<:-union <:-refl <:-∪-symm) (<:-trans <:-∪-assocl (<:-union (∪-<:-∪ⁿ R (T ⇒ U)) <:-refl))) ∪-<:-∪ⁿ never (T ∩ U) = <:-∪-lub <:-never <:-refl ∪-<:-∪ⁿ unknown (T ∩ U) = <:-unknown ∪-<:-∪ⁿ (R ⇒ S) (T ∩ U) = ∪-<:-∪ᶠ (R ⇒ S) (T ∩ U) diff --git a/prototyping/Properties/TypeSaturation.agda b/prototyping/Properties/TypeSaturation.agda new file mode 100644 index 00000000..13f7d171 --- /dev/null +++ b/prototyping/Properties/TypeSaturation.agda @@ -0,0 +1,433 @@ +{-# OPTIONS --rewriting #-} + +module Properties.TypeSaturation where + +open import Agda.Builtin.Equality using (_≡_; refl) +open import FFI.Data.Either using (Either; Left; Right) +open import Luau.Subtyping using (Tree; Language; ¬Language; _<:_; _≮:_; witness; scalar; function; function-err; function-ok; function-ok₁; function-ok₂; scalar-function; _,_; never) +open import Luau.Type using (Type; _⇒_; _∩_; _∪_; never; unknown) +open import Luau.TypeNormalization using (_∩ⁿ_; _∪ⁿ_) +open import Luau.TypeSaturation using (_⋓_; _⋒_; _∩ᵘ_; _∩ⁱ_; ∪-saturate; ∩-saturate; saturate) +open import Properties.Subtyping using (dec-language; language-comp; <:-impl-⊇; <:-refl; <:-trans; <:-trans-≮:; <:-impl-¬≮: ; <:-never; <:-unknown; <:-function; <:-union; <:-∪-symm; <:-∪-left; <:-∪-right; <:-∪-lub; <:-∪-assocl; <:-∪-assocr; <:-intersect; <:-∩-symm; <:-∩-left; <:-∩-right; <:-∩-glb; ≮:-function-left; ≮:-function-right; <:-function-∩-∪; <:-function-∩-∩; <:-∩-assocl; <:-∩-assocr; ∩-<:-∪; <:-∩-distl-∪; ∩-distl-∪-<:; <:-∩-distr-∪; ∩-distr-∪-<:) +open import Properties.TypeNormalization using (Normal; FunType; _⇒_; _∩_; _∪_; never; unknown; normal-∪ⁿ; normal-∩ⁿ; ∪ⁿ-<:-∪; ∪-<:-∪ⁿ; ∩ⁿ-<:-∩; ∩-<:-∩ⁿ) +open import Properties.Contradiction using (CONTRADICTION) +open import Properties.Functions using (_∘_) + +-- Saturation preserves normalization +normal-⋒ : ∀ {F G} → FunType F → FunType G → FunType (F ⋒ G) +normal-⋒ (R ⇒ S) (T ⇒ U) = (normal-∩ⁿ R T) ⇒ (normal-∩ⁿ S U) +normal-⋒ (R ⇒ S) (G ∩ H) = normal-⋒ (R ⇒ S) G ∩ normal-⋒ (R ⇒ S) H +normal-⋒ (E ∩ F) G = normal-⋒ E G ∩ normal-⋒ F G + +normal-⋓ : ∀ {F G} → FunType F → FunType G → FunType (F ⋓ G) +normal-⋓ (R ⇒ S) (T ⇒ U) = (normal-∪ⁿ R T) ⇒ (normal-∪ⁿ S U) +normal-⋓ (R ⇒ S) (G ∩ H) = normal-⋓ (R ⇒ S) G ∩ normal-⋓ (R ⇒ S) H +normal-⋓ (E ∩ F) G = normal-⋓ E G ∩ normal-⋓ F G + +normal-∩-saturate : ∀ {F} → FunType F → FunType (∩-saturate F) +normal-∩-saturate (S ⇒ T) = S ⇒ T +normal-∩-saturate (F ∩ G) = (normal-∩-saturate F ∩ normal-∩-saturate G) ∩ normal-⋒ (normal-∩-saturate F) (normal-∩-saturate G) + +normal-∪-saturate : ∀ {F} → FunType F → FunType (∪-saturate F) +normal-∪-saturate (S ⇒ T) = S ⇒ T +normal-∪-saturate (F ∩ G) = (normal-∪-saturate F ∩ normal-∪-saturate G) ∩ normal-⋓ (normal-∪-saturate F) (normal-∪-saturate G) + +normal-saturate : ∀ {F} → FunType F → FunType (saturate F) +normal-saturate F = normal-∪-saturate (normal-∩-saturate F) + +-- Saturation resects subtyping +∪-saturate-<: : ∀ {F} → FunType F → ∪-saturate F <: F +∪-saturate-<: (S ⇒ T) = <:-refl +∪-saturate-<: (F ∩ G) = <:-trans <:-∩-left (<:-intersect (∪-saturate-<: F) (∪-saturate-<: G)) + +∩-saturate-<: : ∀ {F} → FunType F → ∩-saturate F <: F +∩-saturate-<: (S ⇒ T) = <:-refl +∩-saturate-<: (F ∩ G) = <:-trans <:-∩-left (<:-intersect (∩-saturate-<: F) (∩-saturate-<: G)) + +saturate-<: : ∀ {F} → FunType F → saturate F <: F +saturate-<: F = <:-trans (∪-saturate-<: (normal-∩-saturate F)) (∩-saturate-<: F) + +∩-<:-⋓ : ∀ {F G} → FunType F → FunType G → (F ∩ G) <: (F ⋓ G) +∩-<:-⋓ (R ⇒ S) (T ⇒ U) = <:-trans <:-function-∩-∪ (<:-function (∪ⁿ-<:-∪ R T) (∪-<:-∪ⁿ S U)) +∩-<:-⋓ (R ⇒ S) (G ∩ H) = <:-trans (<:-∩-glb (<:-intersect <:-refl <:-∩-left) (<:-intersect <:-refl <:-∩-right)) (<:-intersect (∩-<:-⋓ (R ⇒ S) G) (∩-<:-⋓ (R ⇒ S) H)) +∩-<:-⋓ (E ∩ F) G = <:-trans (<:-∩-glb (<:-intersect <:-∩-left <:-refl) (<:-intersect <:-∩-right <:-refl)) (<:-intersect (∩-<:-⋓ E G) (∩-<:-⋓ F G)) + +∩-<:-⋒ : ∀ {F G} → FunType F → FunType G → (F ∩ G) <: (F ⋒ G) +∩-<:-⋒ (R ⇒ S) (T ⇒ U) = <:-trans <:-function-∩-∩ (<:-function (∩ⁿ-<:-∩ R T) (∩-<:-∩ⁿ S U)) +∩-<:-⋒ (R ⇒ S) (G ∩ H) = <:-trans (<:-∩-glb (<:-intersect <:-refl <:-∩-left) (<:-intersect <:-refl <:-∩-right)) (<:-intersect (∩-<:-⋒ (R ⇒ S) G) (∩-<:-⋒ (R ⇒ S) H)) +∩-<:-⋒ (E ∩ F) G = <:-trans (<:-∩-glb (<:-intersect <:-∩-left <:-refl) (<:-intersect <:-∩-right <:-refl)) (<:-intersect (∩-<:-⋒ E G) (∩-<:-⋒ F G)) + +<:-∪-saturate : ∀ {F} → FunType F → F <: ∪-saturate F +<:-∪-saturate (S ⇒ T) = <:-refl +<:-∪-saturate (F ∩ G) = <:-∩-glb (<:-intersect (<:-∪-saturate F) (<:-∪-saturate G)) (<:-trans (<:-intersect (<:-∪-saturate F) (<:-∪-saturate G)) (∩-<:-⋓ (normal-∪-saturate F) (normal-∪-saturate G))) + +<:-∩-saturate : ∀ {F} → FunType F → F <: ∩-saturate F +<:-∩-saturate (S ⇒ T) = <:-refl +<:-∩-saturate (F ∩ G) = <:-∩-glb (<:-intersect (<:-∩-saturate F) (<:-∩-saturate G)) (<:-trans (<:-intersect (<:-∩-saturate F) (<:-∩-saturate G)) (∩-<:-⋒ (normal-∩-saturate F) (normal-∩-saturate G))) + +<:-saturate : ∀ {F} → FunType F → F <: saturate F +<:-saturate F = <:-trans (<:-∩-saturate F) (<:-∪-saturate (normal-∩-saturate F)) + +-- Overloads F is the set of overloads of F +data Overloads : Type → Type → Set where + + here : ∀ {S T} → Overloads (S ⇒ T) (S ⇒ T) + left : ∀ {S T F G} → Overloads F (S ⇒ T) → Overloads (F ∩ G) (S ⇒ T) + right : ∀ {S T F G} → Overloads G (S ⇒ T) → Overloads (F ∩ G) (S ⇒ T) + +normal-overload-src : ∀ {F S T} → FunType F → Overloads F (S ⇒ T) → Normal S +normal-overload-src (S ⇒ T) here = S +normal-overload-src (F ∩ G) (left o) = normal-overload-src F o +normal-overload-src (F ∩ G) (right o) = normal-overload-src G o + +normal-overload-tgt : ∀ {F S T} → FunType F → Overloads F (S ⇒ T) → Normal T +normal-overload-tgt (S ⇒ T) here = T +normal-overload-tgt (F ∩ G) (left o) = normal-overload-tgt F o +normal-overload-tgt (F ∩ G) (right o) = normal-overload-tgt G o + +-- An inductive presentation of the overloads of F ⋓ G +data ∪-Lift (P Q : Type → Set) : Type → Set where + + union : ∀ {R S T U} → + + P (R ⇒ S) → + Q (T ⇒ U) → + -------------------- + ∪-Lift P Q ((R ∪ T) ⇒ (S ∪ U)) + +-- An inductive presentation of the overloads of F ⋒ G +data ∩-Lift (P Q : Type → Set) : Type → Set where + + intersect : ∀ {R S T U} → + + P (R ⇒ S) → + Q (T ⇒ U) → + -------------------- + ∩-Lift P Q ((R ∩ T) ⇒ (S ∩ U)) + +-- An inductive presentation of the overloads of ∪-saturate F +data ∪-Saturate (P : Type → Set) : Type → Set where + + base : ∀ {S T} → + + P (S ⇒ T) → + -------------------- + ∪-Saturate P (S ⇒ T) + + union : ∀ {R S T U} → + + ∪-Saturate P (R ⇒ S) → + ∪-Saturate P (T ⇒ U) → + -------------------- + ∪-Saturate P ((R ∪ T) ⇒ (S ∪ U)) + +-- An inductive presentation of the overloads of ∩-saturate F +data ∩-Saturate (P : Type → Set) : Type → Set where + + base : ∀ {S T} → + + P (S ⇒ T) → + -------------------- + ∩-Saturate P (S ⇒ T) + + intersect : ∀ {R S T U} → + + ∩-Saturate P (R ⇒ S) → + ∩-Saturate P (T ⇒ U) → + -------------------- + ∩-Saturate P ((R ∩ T) ⇒ (S ∩ U)) + +-- The <:-up-closure of a set of function types +data <:-Close (P : Type → Set) : Type → Set where + + defn : ∀ {R S T U} → + + P (S ⇒ T) → + R <: S → + T <: U → + ------------------ + <:-Close P (R ⇒ U) + +-- F ⊆ᵒ G whenever every overload of F is an overload of G +_⊆ᵒ_ : Type → Type → Set +F ⊆ᵒ G = ∀ {S T} → Overloads F (S ⇒ T) → Overloads G (S ⇒ T) + +-- F <:ᵒ G when every overload of G is a supertype of an overload of F +_<:ᵒ_ : Type → Type → Set +_<:ᵒ_ F G = ∀ {S T} → Overloads G (S ⇒ T) → <:-Close (Overloads F) (S ⇒ T) + +-- P ⊂: Q when any type in P is a subtype of some type in Q +_⊂:_ : (Type → Set) → (Type → Set) → Set +P ⊂: Q = ∀ {S T} → P (S ⇒ T) → <:-Close Q (S ⇒ T) + +-- <:-Close is a monad +just : ∀ {P S T} → P (S ⇒ T) → <:-Close P (S ⇒ T) +just p = defn p <:-refl <:-refl + +infixl 5 _>>=_ _>>=ˡ_ _>>=ʳ_ +_>>=_ : ∀ {P Q S T} → <:-Close P (S ⇒ T) → (P ⊂: Q) → <:-Close Q (S ⇒ T) +(defn p p₁ p₂) >>= P⊂Q with P⊂Q p +(defn p p₁ p₂) >>= P⊂Q | defn q q₁ q₂ = defn q (<:-trans p₁ q₁) (<:-trans q₂ p₂) + +_>>=ˡ_ : ∀ {P R S T} → <:-Close P (S ⇒ T) → (R <: S) → <:-Close P (R ⇒ T) +(defn p p₁ p₂) >>=ˡ q = defn p (<:-trans q p₁) p₂ + +_>>=ʳ_ : ∀ {P S T U} → <:-Close P (S ⇒ T) → (T <: U) → <:-Close P (S ⇒ U) +(defn p p₁ p₂) >>=ʳ q = defn p p₁ (<:-trans p₂ q) + +-- Properties of ⊂: +⊂:-refl : ∀ {P} → P ⊂: P +⊂:-refl p = just p + +_[∪]_ : ∀ {P Q R S T U} → <:-Close P (R ⇒ S) → <:-Close Q (T ⇒ U) → <:-Close (∪-Lift P Q) ((R ∪ T) ⇒ (S ∪ U)) +(defn p p₁ p₂) [∪] (defn q q₁ q₂) = defn (union p q) (<:-union p₁ q₁) (<:-union p₂ q₂) + +_[∩]_ : ∀ {P Q R S T U} → <:-Close P (R ⇒ S) → <:-Close Q (T ⇒ U) → <:-Close (∩-Lift P Q) ((R ∩ T) ⇒ (S ∩ U)) +(defn p p₁ p₂) [∩] (defn q q₁ q₂) = defn (intersect p q) (<:-intersect p₁ q₁) (<:-intersect p₂ q₂) + +⊂:-∩-saturate-inj : ∀ {P} → P ⊂: ∩-Saturate P +⊂:-∩-saturate-inj p = defn (base p) <:-refl <:-refl + +⊂:-∪-saturate-inj : ∀ {P} → P ⊂: ∪-Saturate P +⊂:-∪-saturate-inj p = just (base p) + +⊂:-∩-lift-saturate : ∀ {P} → ∩-Lift (∩-Saturate P) (∩-Saturate P) ⊂: ∩-Saturate P +⊂:-∩-lift-saturate (intersect p q) = just (intersect p q) + +⊂:-∪-lift-saturate : ∀ {P} → ∪-Lift (∪-Saturate P) (∪-Saturate P) ⊂: ∪-Saturate P +⊂:-∪-lift-saturate (union p q) = just (union p q) + +⊂:-∩-lift : ∀ {P Q R S} → (P ⊂: Q) → (R ⊂: S) → (∩-Lift P R ⊂: ∩-Lift Q S) +⊂:-∩-lift P⊂Q R⊂S (intersect n o) = P⊂Q n [∩] R⊂S o + +⊂:-∪-lift : ∀ {P Q R S} → (P ⊂: Q) → (R ⊂: S) → (∪-Lift P R ⊂: ∪-Lift Q S) +⊂:-∪-lift P⊂Q R⊂S (union n o) = P⊂Q n [∪] R⊂S o + +⊂:-∩-saturate : ∀ {P Q} → (P ⊂: Q) → (∩-Saturate P ⊂: ∩-Saturate Q) +⊂:-∩-saturate P⊂Q (base p) = P⊂Q p >>= ⊂:-∩-saturate-inj +⊂:-∩-saturate P⊂Q (intersect p q) = (⊂:-∩-saturate P⊂Q p [∩] ⊂:-∩-saturate P⊂Q q) >>= ⊂:-∩-lift-saturate + +⊂:-∪-saturate : ∀ {P Q} → (P ⊂: Q) → (∪-Saturate P ⊂: ∪-Saturate Q) +⊂:-∪-saturate P⊂Q (base p) = P⊂Q p >>= ⊂:-∪-saturate-inj +⊂:-∪-saturate P⊂Q (union p q) = (⊂:-∪-saturate P⊂Q p [∪] ⊂:-∪-saturate P⊂Q q) >>= ⊂:-∪-lift-saturate + +⊂:-∩-saturate-indn : ∀ {P Q} → (P ⊂: Q) → (∩-Lift Q Q ⊂: Q) → (∩-Saturate P ⊂: Q) +⊂:-∩-saturate-indn P⊂Q QQ⊂Q (base p) = P⊂Q p +⊂:-∩-saturate-indn P⊂Q QQ⊂Q (intersect p q) = (⊂:-∩-saturate-indn P⊂Q QQ⊂Q p [∩] ⊂:-∩-saturate-indn P⊂Q QQ⊂Q q) >>= QQ⊂Q + +⊂:-∪-saturate-indn : ∀ {P Q} → (P ⊂: Q) → (∪-Lift Q Q ⊂: Q) → (∪-Saturate P ⊂: Q) +⊂:-∪-saturate-indn P⊂Q QQ⊂Q (base p) = P⊂Q p +⊂:-∪-saturate-indn P⊂Q QQ⊂Q (union p q) = (⊂:-∪-saturate-indn P⊂Q QQ⊂Q p [∪] ⊂:-∪-saturate-indn P⊂Q QQ⊂Q q) >>= QQ⊂Q + +∪-saturate-resp-∩-saturation : ∀ {P} → (∩-Lift P P ⊂: P) → (∩-Lift (∪-Saturate P) (∪-Saturate P) ⊂: ∪-Saturate P) +∪-saturate-resp-∩-saturation ∩P⊂P (intersect (base p) (base q)) = ∩P⊂P (intersect p q) >>= ⊂:-∪-saturate-inj +∪-saturate-resp-∩-saturation ∩P⊂P (intersect p (union q q₁)) = (∪-saturate-resp-∩-saturation ∩P⊂P (intersect p q) [∪] ∪-saturate-resp-∩-saturation ∩P⊂P (intersect p q₁)) >>= ⊂:-∪-lift-saturate >>=ˡ <:-∩-distl-∪ >>=ʳ ∩-distl-∪-<: +∪-saturate-resp-∩-saturation ∩P⊂P (intersect (union p p₁) q) = (∪-saturate-resp-∩-saturation ∩P⊂P (intersect p q) [∪] ∪-saturate-resp-∩-saturation ∩P⊂P (intersect p₁ q)) >>= ⊂:-∪-lift-saturate >>=ˡ <:-∩-distr-∪ >>=ʳ ∩-distr-∪-<: + +ov-language : ∀ {F t} → FunType F → (∀ {S T} → Overloads F (S ⇒ T) → Language (S ⇒ T) t) → Language F t +ov-language (S ⇒ T) p = p here +ov-language (F ∩ G) p = (ov-language F (p ∘ left) , ov-language G (p ∘ right)) + +ov-<: : ∀ {F R S T U} → FunType F → Overloads F (R ⇒ S) → ((R ⇒ S) <: (T ⇒ U)) → F <: (T ⇒ U) +ov-<: F here p = p +ov-<: (F ∩ G) (left o) p = <:-trans <:-∩-left (ov-<: F o p) +ov-<: (F ∩ G) (right o) p = <:-trans <:-∩-right (ov-<: G o p) + +<:ᵒ-impl-<: : ∀ {F G} → FunType F → FunType G → (F <:ᵒ G) → (F <: G) +<:ᵒ-impl-<: F (T ⇒ U) F>= ⊂:-overloads-left +⊂:-overloads-⋒ (R ⇒ S) (G ∩ H) (intersect here (right o)) = ⊂:-overloads-⋒ (R ⇒ S) H (intersect here o) >>= ⊂:-overloads-right +⊂:-overloads-⋒ (E ∩ F) G (intersect (left n) o) = ⊂:-overloads-⋒ E G (intersect n o) >>= ⊂:-overloads-left +⊂:-overloads-⋒ (E ∩ F) G (intersect (right n) o) = ⊂:-overloads-⋒ F G (intersect n o) >>= ⊂:-overloads-right + +⊂:-⋒-overloads : ∀ {F G} → FunType F → FunType G → Overloads (F ⋒ G) ⊂: ∩-Lift (Overloads F) (Overloads G) +⊂:-⋒-overloads (R ⇒ S) (T ⇒ U) here = defn (intersect here here) (∩ⁿ-<:-∩ R T) (∩-<:-∩ⁿ S U) +⊂:-⋒-overloads (R ⇒ S) (G ∩ H) (left o) = ⊂:-⋒-overloads (R ⇒ S) G o >>= ⊂:-∩-lift ⊂:-refl ⊂:-overloads-left +⊂:-⋒-overloads (R ⇒ S) (G ∩ H) (right o) = ⊂:-⋒-overloads (R ⇒ S) H o >>= ⊂:-∩-lift ⊂:-refl ⊂:-overloads-right +⊂:-⋒-overloads (E ∩ F) G (left o) = ⊂:-⋒-overloads E G o >>= ⊂:-∩-lift ⊂:-overloads-left ⊂:-refl +⊂:-⋒-overloads (E ∩ F) G (right o) = ⊂:-⋒-overloads F G o >>= ⊂:-∩-lift ⊂:-overloads-right ⊂:-refl + +⊂:-overloads-⋓ : ∀ {F G} → FunType F → FunType G → ∪-Lift (Overloads F) (Overloads G) ⊂: Overloads (F ⋓ G) +⊂:-overloads-⋓ (R ⇒ S) (T ⇒ U) (union here here) = defn here (∪-<:-∪ⁿ R T) (∪ⁿ-<:-∪ S U) +⊂:-overloads-⋓ (R ⇒ S) (G ∩ H) (union here (left o)) = ⊂:-overloads-⋓ (R ⇒ S) G (union here o) >>= ⊂:-overloads-left +⊂:-overloads-⋓ (R ⇒ S) (G ∩ H) (union here (right o)) = ⊂:-overloads-⋓ (R ⇒ S) H (union here o) >>= ⊂:-overloads-right +⊂:-overloads-⋓ (E ∩ F) G (union (left n) o) = ⊂:-overloads-⋓ E G (union n o) >>= ⊂:-overloads-left +⊂:-overloads-⋓ (E ∩ F) G (union (right n) o) = ⊂:-overloads-⋓ F G (union n o) >>= ⊂:-overloads-right + +⊂:-⋓-overloads : ∀ {F G} → FunType F → FunType G → Overloads (F ⋓ G) ⊂: ∪-Lift (Overloads F) (Overloads G) +⊂:-⋓-overloads (R ⇒ S) (T ⇒ U) here = defn (union here here) (∪ⁿ-<:-∪ R T) (∪-<:-∪ⁿ S U) +⊂:-⋓-overloads (R ⇒ S) (G ∩ H) (left o) = ⊂:-⋓-overloads (R ⇒ S) G o >>= ⊂:-∪-lift ⊂:-refl ⊂:-overloads-left +⊂:-⋓-overloads (R ⇒ S) (G ∩ H) (right o) = ⊂:-⋓-overloads (R ⇒ S) H o >>= ⊂:-∪-lift ⊂:-refl ⊂:-overloads-right +⊂:-⋓-overloads (E ∩ F) G (left o) = ⊂:-⋓-overloads E G o >>= ⊂:-∪-lift ⊂:-overloads-left ⊂:-refl +⊂:-⋓-overloads (E ∩ F) G (right o) = ⊂:-⋓-overloads F G o >>= ⊂:-∪-lift ⊂:-overloads-right ⊂:-refl + +∪-saturate-overloads : ∀ {F} → FunType F → Overloads (∪-saturate F) ⊂: ∪-Saturate (Overloads F) +∪-saturate-overloads (S ⇒ T) here = just (base here) +∪-saturate-overloads (F ∩ G) (left (left o)) = ∪-saturate-overloads F o >>= ⊂:-∪-saturate ⊂:-overloads-left +∪-saturate-overloads (F ∩ G) (left (right o)) = ∪-saturate-overloads G o >>= ⊂:-∪-saturate ⊂:-overloads-right +∪-saturate-overloads (F ∩ G) (right o) = + ⊂:-⋓-overloads (normal-∪-saturate F) (normal-∪-saturate G) o >>= + ⊂:-∪-lift (∪-saturate-overloads F) (∪-saturate-overloads G) >>= + ⊂:-∪-lift (⊂:-∪-saturate ⊂:-overloads-left) (⊂:-∪-saturate ⊂:-overloads-right) >>= + ⊂:-∪-lift-saturate + +overloads-∪-saturate : ∀ {F} → FunType F → ∪-Saturate (Overloads F) ⊂: Overloads (∪-saturate F) +overloads-∪-saturate F = ⊂:-∪-saturate-indn (inj F) (step F) where + + inj : ∀ {F} → FunType F → Overloads F ⊂: Overloads (∪-saturate F) + inj (S ⇒ T) here = just here + inj (F ∩ G) (left p) = inj F p >>= ⊂:-overloads-left >>= ⊂:-overloads-left + inj (F ∩ G) (right p) = inj G p >>= ⊂:-overloads-right >>= ⊂:-overloads-left + + step : ∀ {F} → FunType F → ∪-Lift (Overloads (∪-saturate F)) (Overloads (∪-saturate F)) ⊂: Overloads (∪-saturate F) + step (S ⇒ T) (union here here) = defn here (<:-∪-lub <:-refl <:-refl) <:-∪-left + step (F ∩ G) (union (left (left p)) (left (left q))) = step F (union p q) >>= ⊂:-overloads-left >>= ⊂:-overloads-left + step (F ∩ G) (union (left (left p)) (left (right q))) = ⊂:-overloads-⋓ (normal-∪-saturate F) (normal-∪-saturate G) (union p q) >>= ⊂:-overloads-right + step (F ∩ G) (union (left (right p)) (left (left q))) = ⊂:-overloads-⋓ (normal-∪-saturate F) (normal-∪-saturate G) (union q p) >>= ⊂:-overloads-right >>=ˡ <:-∪-symm >>=ʳ <:-∪-symm + step (F ∩ G) (union (left (right p)) (left (right q))) = step G (union p q) >>= ⊂:-overloads-right >>= ⊂:-overloads-left + step (F ∩ G) (union p (right q)) with ⊂:-⋓-overloads (normal-∪-saturate F) (normal-∪-saturate G) q + step (F ∩ G) (union (left (left p)) (right q)) | defn (union q₁ q₂) q₃ q₄ = + (step F (union p q₁) [∪] just q₂) >>= + ⊂:-overloads-⋓ (normal-∪-saturate F) (normal-∪-saturate G) >>= + ⊂:-overloads-right >>=ˡ + <:-trans (<:-union <:-refl q₃) <:-∪-assocl >>=ʳ + <:-trans <:-∪-assocr (<:-union <:-refl q₄) + step (F ∩ G) (union (left (right p)) (right q)) | defn (union q₁ q₂) q₃ q₄ = + (just q₁ [∪] step G (union p q₂)) >>= + ⊂:-overloads-⋓ (normal-∪-saturate F) (normal-∪-saturate G) >>= + ⊂:-overloads-right >>=ˡ + <:-trans (<:-union <:-refl q₃) (<:-∪-lub (<:-trans <:-∪-left <:-∪-right) (<:-∪-lub <:-∪-left (<:-trans <:-∪-right <:-∪-right))) >>=ʳ + <:-trans (<:-∪-lub (<:-trans <:-∪-left <:-∪-right) (<:-∪-lub <:-∪-left (<:-trans <:-∪-right <:-∪-right))) (<:-union <:-refl q₄) + step (F ∩ G) (union (right p) (right q)) | defn (union q₁ q₂) q₃ q₄ with ⊂:-⋓-overloads (normal-∪-saturate F) (normal-∪-saturate G) p + step (F ∩ G) (union (right p) (right q)) | defn (union q₁ q₂) q₃ q₄ | defn (union p₁ p₂) p₃ p₄ = + (step F (union p₁ q₁) [∪] step G (union p₂ q₂)) >>= + ⊂:-overloads-⋓ (normal-∪-saturate F) (normal-∪-saturate G) >>= + ⊂:-overloads-right >>=ˡ + <:-trans (<:-union p₃ q₃) (<:-∪-lub (<:-union <:-∪-left <:-∪-left) (<:-union <:-∪-right <:-∪-right)) >>=ʳ + <:-trans (<:-∪-lub (<:-union <:-∪-left <:-∪-left) (<:-union <:-∪-right <:-∪-right)) (<:-union p₄ q₄) + step (F ∩ G) (union (right p) q) with ⊂:-⋓-overloads (normal-∪-saturate F) (normal-∪-saturate G) p + step (F ∩ G) (union (right p) (left (left q))) | defn (union p₁ p₂) p₃ p₄ = + (step F (union p₁ q) [∪] just p₂) >>= + ⊂:-overloads-⋓ (normal-∪-saturate F) (normal-∪-saturate G) >>= + ⊂:-overloads-right >>=ˡ + <:-trans (<:-union p₃ <:-refl) (<:-∪-lub (<:-union <:-∪-left <:-refl) (<:-trans <:-∪-right <:-∪-left)) >>=ʳ + <:-trans (<:-∪-lub (<:-union <:-∪-left <:-refl) (<:-trans <:-∪-right <:-∪-left)) (<:-union p₄ <:-refl) + step (F ∩ G) (union (right p) (left (right q))) | defn (union p₁ p₂) p₃ p₄ = + (just p₁ [∪] step G (union p₂ q)) >>= + ⊂:-overloads-⋓ (normal-∪-saturate F) (normal-∪-saturate G) >>= + ⊂:-overloads-right >>=ˡ + <:-trans (<:-union p₃ <:-refl) <:-∪-assocr >>=ʳ + <:-trans <:-∪-assocl (<:-union p₄ <:-refl) + step (F ∩ G) (union (right p) (right q)) | defn (union p₁ p₂) p₃ p₄ with ⊂:-⋓-overloads (normal-∪-saturate F) (normal-∪-saturate G) q + step (F ∩ G) (union (right p) (right q)) | defn (union p₁ p₂) p₃ p₄ | defn (union q₁ q₂) q₃ q₄ = + (step F (union p₁ q₁) [∪] step G (union p₂ q₂)) >>= + ⊂:-overloads-⋓ (normal-∪-saturate F) (normal-∪-saturate G) >>= + ⊂:-overloads-right >>=ˡ + <:-trans (<:-union p₃ q₃) (<:-∪-lub (<:-union <:-∪-left <:-∪-left) (<:-union <:-∪-right <:-∪-right)) >>=ʳ + <:-trans (<:-∪-lub (<:-union <:-∪-left <:-∪-left) (<:-union <:-∪-right <:-∪-right)) (<:-union p₄ q₄) + +∪-saturated : ∀ {F} → FunType F → ∪-Lift (Overloads (∪-saturate F)) (Overloads (∪-saturate F)) ⊂: Overloads (∪-saturate F) +∪-saturated F o = + ⊂:-∪-lift (∪-saturate-overloads F) (∪-saturate-overloads F) o >>= + ⊂:-∪-lift-saturate >>= + overloads-∪-saturate F + +∩-saturate-overloads : ∀ {F} → FunType F → Overloads (∩-saturate F) ⊂: ∩-Saturate (Overloads F) +∩-saturate-overloads (S ⇒ T) here = just (base here) +∩-saturate-overloads (F ∩ G) (left (left o)) = ∩-saturate-overloads F o >>= ⊂:-∩-saturate ⊂:-overloads-left +∩-saturate-overloads (F ∩ G) (left (right o)) = ∩-saturate-overloads G o >>= ⊂:-∩-saturate ⊂:-overloads-right +∩-saturate-overloads (F ∩ G) (right o) = + ⊂:-⋒-overloads (normal-∩-saturate F) (normal-∩-saturate G) o >>= + ⊂:-∩-lift (∩-saturate-overloads F) (∩-saturate-overloads G) >>= + ⊂:-∩-lift (⊂:-∩-saturate ⊂:-overloads-left) (⊂:-∩-saturate ⊂:-overloads-right) >>= + ⊂:-∩-lift-saturate + +overloads-∩-saturate : ∀ {F} → FunType F → ∩-Saturate (Overloads F) ⊂: Overloads (∩-saturate F) +overloads-∩-saturate F = ⊂:-∩-saturate-indn (inj F) (step F) where + + inj : ∀ {F} → FunType F → Overloads F ⊂: Overloads (∩-saturate F) + inj (S ⇒ T) here = just here + inj (F ∩ G) (left p) = inj F p >>= ⊂:-overloads-left >>= ⊂:-overloads-left + inj (F ∩ G) (right p) = inj G p >>= ⊂:-overloads-right >>= ⊂:-overloads-left + + step : ∀ {F} → FunType F → ∩-Lift (Overloads (∩-saturate F)) (Overloads (∩-saturate F)) ⊂: Overloads (∩-saturate F) + step (S ⇒ T) (intersect here here) = defn here <:-∩-left (<:-∩-glb <:-refl <:-refl) + step (F ∩ G) (intersect (left (left p)) (left (left q))) = step F (intersect p q) >>= ⊂:-overloads-left >>= ⊂:-overloads-left + step (F ∩ G) (intersect (left (left p)) (left (right q))) = ⊂:-overloads-⋒ (normal-∩-saturate F) (normal-∩-saturate G) (intersect p q) >>= ⊂:-overloads-right + step (F ∩ G) (intersect (left (right p)) (left (left q))) = ⊂:-overloads-⋒ (normal-∩-saturate F) (normal-∩-saturate G) (intersect q p) >>= ⊂:-overloads-right >>=ˡ <:-∩-symm >>=ʳ <:-∩-symm + step (F ∩ G) (intersect (left (right p)) (left (right q))) = step G (intersect p q) >>= ⊂:-overloads-right >>= ⊂:-overloads-left + step (F ∩ G) (intersect (right p) q) with ⊂:-⋒-overloads (normal-∩-saturate F) (normal-∩-saturate G) p + step (F ∩ G) (intersect (right p) (left (left q))) | defn (intersect p₁ p₂) p₃ p₄ = + (step F (intersect p₁ q) [∩] just p₂) >>= + ⊂:-overloads-⋒ (normal-∩-saturate F) (normal-∩-saturate G) >>= + ⊂:-overloads-right >>=ˡ + <:-trans (<:-intersect p₃ <:-refl) (<:-∩-glb (<:-intersect <:-∩-left <:-refl) (<:-trans <:-∩-left <:-∩-right)) >>=ʳ + <:-trans (<:-∩-glb (<:-intersect <:-∩-left <:-refl) (<:-trans <:-∩-left <:-∩-right)) (<:-intersect p₄ <:-refl) + step (F ∩ G) (intersect (right p) (left (right q))) | defn (intersect p₁ p₂) p₃ p₄ = + (just p₁ [∩] step G (intersect p₂ q)) >>= + ⊂:-overloads-⋒ (normal-∩-saturate F) (normal-∩-saturate G) >>= + ⊂:-overloads-right >>=ˡ + <:-trans (<:-intersect p₃ <:-refl) <:-∩-assocr >>=ʳ + <:-trans <:-∩-assocl (<:-intersect p₄ <:-refl) + step (F ∩ G) (intersect (right p) (right q)) | defn (intersect p₁ p₂) p₃ p₄ with ⊂:-⋒-overloads (normal-∩-saturate F) (normal-∩-saturate G) q + step (F ∩ G) (intersect (right p) (right q)) | defn (intersect p₁ p₂) p₃ p₄ | defn (intersect q₁ q₂) q₃ q₄ = + (step F (intersect p₁ q₁) [∩] step G (intersect p₂ q₂)) >>= + ⊂:-overloads-⋒ (normal-∩-saturate F) (normal-∩-saturate G) >>= + ⊂:-overloads-right >>=ˡ + <:-trans (<:-intersect p₃ q₃) (<:-∩-glb (<:-intersect <:-∩-left <:-∩-left) (<:-intersect <:-∩-right <:-∩-right)) >>=ʳ + <:-trans (<:-∩-glb (<:-intersect <:-∩-left <:-∩-left) (<:-intersect <:-∩-right <:-∩-right)) (<:-intersect p₄ q₄) + step (F ∩ G) (intersect p (right q)) with ⊂:-⋒-overloads (normal-∩-saturate F) (normal-∩-saturate G) q + step (F ∩ G) (intersect (left (left p)) (right q)) | defn (intersect q₁ q₂) q₃ q₄ = + (step F (intersect p q₁) [∩] just q₂) >>= + ⊂:-overloads-⋒ (normal-∩-saturate F) (normal-∩-saturate G) >>= + ⊂:-overloads-right >>=ˡ + <:-trans (<:-intersect <:-refl q₃) <:-∩-assocl >>=ʳ + <:-trans <:-∩-assocr (<:-intersect <:-refl q₄) + step (F ∩ G) (intersect (left (right p)) (right q)) | defn (intersect q₁ q₂) q₃ q₄ = + (just q₁ [∩] step G (intersect p q₂) ) >>= + ⊂:-overloads-⋒ (normal-∩-saturate F) (normal-∩-saturate G) >>= + ⊂:-overloads-right >>=ˡ + <:-trans (<:-intersect <:-refl q₃) (<:-∩-glb (<:-trans <:-∩-right <:-∩-left) (<:-∩-glb <:-∩-left (<:-trans <:-∩-right <:-∩-right))) >>=ʳ + <:-∩-glb (<:-trans <:-∩-right <:-∩-left) (<:-trans (<:-∩-glb <:-∩-left (<:-trans <:-∩-right <:-∩-right)) q₄) + step (F ∩ G) (intersect (right p) (right q)) | defn (intersect q₁ q₂) q₃ q₄ with ⊂:-⋒-overloads (normal-∩-saturate F) (normal-∩-saturate G) p + step (F ∩ G) (intersect (right p) (right q)) | defn (intersect q₁ q₂) q₃ q₄ | defn (intersect p₁ p₂) p₃ p₄ = + (step F (intersect p₁ q₁) [∩] step G (intersect p₂ q₂)) >>= + ⊂:-overloads-⋒ (normal-∩-saturate F) (normal-∩-saturate G) >>= + ⊂:-overloads-right >>=ˡ + <:-trans (<:-intersect p₃ q₃) (<:-∩-glb (<:-intersect <:-∩-left <:-∩-left) (<:-intersect <:-∩-right <:-∩-right)) >>=ʳ + <:-trans (<:-∩-glb (<:-intersect <:-∩-left <:-∩-left) (<:-intersect <:-∩-right <:-∩-right)) (<:-intersect p₄ q₄) + +saturate-overloads : ∀ {F} → FunType F → Overloads (saturate F) ⊂: ∪-Saturate (∩-Saturate (Overloads F)) +saturate-overloads F o = ∪-saturate-overloads (normal-∩-saturate F) o >>= (⊂:-∪-saturate (∩-saturate-overloads F)) + +overloads-saturate : ∀ {F} → FunType F → ∪-Saturate (∩-Saturate (Overloads F)) ⊂: Overloads (saturate F) +overloads-saturate F o = ⊂:-∪-saturate (overloads-∩-saturate F) o >>= overloads-∪-saturate (normal-∩-saturate F) + +-- Saturated F whenever +-- * if F has overloads (R ⇒ S) and (T ⇒ U) then F has an overload which is a subtype of ((R ∩ T) ⇒ (S ∩ U)) +-- * ditto union +data Saturated (F : Type) : Set where + + defn : + + (∀ {R S T U} → Overloads F (R ⇒ S) → Overloads F (T ⇒ U) → <:-Close (Overloads F) ((R ∩ T) ⇒ (S ∩ U))) → + (∀ {R S T U} → Overloads F (R ⇒ S) → Overloads F (T ⇒ U) → <:-Close (Overloads F) ((R ∪ T) ⇒ (S ∪ U))) → + ----------- + Saturated F + +-- saturated F is saturated! +saturated : ∀ {F} → FunType F → Saturated (saturate F) +saturated F = defn + (λ n o → (saturate-overloads F n [∩] saturate-overloads F o) >>= ∪-saturate-resp-∩-saturation ⊂:-∩-lift-saturate >>= overloads-saturate F) + (λ n o → ∪-saturated (normal-∩-saturate F) (union n o)) From 316838f253b64a2e469649beb5dd67c51a948f27 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Thu, 16 Jun 2022 17:52:23 -0700 Subject: [PATCH 275/399] Sync to upstream/release/531 --- Analysis/include/Luau/TypePack.h | 13 +- Analysis/include/Luau/TypeVar.h | 11 +- Analysis/include/Luau/Variant.h | 1 + Analysis/src/Autocomplete.cpp | 41 ++-- Analysis/src/BuiltinDefinitions.cpp | 36 +--- Analysis/src/Clone.cpp | 5 - Analysis/src/EmbeddedBuiltinDefinitions.cpp | 8 +- Analysis/src/Instantiation.cpp | 4 - Analysis/src/Quantify.cpp | 35 ---- Analysis/src/Scope.cpp | 5 +- Analysis/src/Substitution.cpp | 1 - Analysis/src/TxnLog.cpp | 56 +++++- Analysis/src/TypeInfer.cpp | 113 ++++++------ Analysis/src/TypePack.cpp | 21 +++ Analysis/src/TypeVar.cpp | 21 +++ Ast/src/Parser.cpp | 22 ++- Compiler/src/BytecodeBuilder.cpp | 10 +- Compiler/src/Compiler.cpp | 195 +++++--------------- VM/src/lapi.cpp | 106 +++++------ VM/src/lbuiltins.cpp | 12 +- VM/src/ldo.cpp | 25 ++- VM/src/lvmexecute.cpp | 19 +- tests/Autocomplete.test.cpp | 37 ++-- tests/Compiler.test.cpp | 13 -- tests/Conformance.test.cpp | 59 +++++- tests/Module.test.cpp | 20 ++ tests/Parser.test.cpp | 17 ++ tests/TypeInfer.aliases.test.cpp | 6 - tests/TypeInfer.loops.test.cpp | 8 - tests/TypeInfer.refinements.test.cpp | 32 +++- tests/TypeInfer.singletons.test.cpp | 2 - tests/TypePack.test.cpp | 16 ++ tests/TypeVar.test.cpp | 20 ++ tests/conformance/apicalls.lua | 11 ++ tests/conformance/pcall.lua | 17 ++ 35 files changed, 544 insertions(+), 474 deletions(-) diff --git a/Analysis/include/Luau/TypePack.h b/Analysis/include/Luau/TypePack.h index bbc65f94..c1de242f 100644 --- a/Analysis/include/Luau/TypePack.h +++ b/Analysis/include/Luau/TypePack.h @@ -48,13 +48,24 @@ struct TypePackVar explicit TypePackVar(const TypePackVariant& ty); explicit TypePackVar(TypePackVariant&& ty); TypePackVar(TypePackVariant&& ty, bool persistent); + bool operator==(const TypePackVar& rhs) const; + TypePackVar& operator=(TypePackVariant&& tp); + TypePackVar& operator=(const TypePackVar& rhs); + + // Re-assignes the content of the pack, but doesn't change the owning arena and can't make pack persistent. + void reassign(const TypePackVar& rhs) + { + ty = rhs.ty; + } + TypePackVariant ty; + bool persistent = false; - // Pointer to the type arena that allocated this type. + // Pointer to the type arena that allocated this pack. TypeArena* owningArena = nullptr; }; diff --git a/Analysis/include/Luau/TypeVar.h b/Analysis/include/Luau/TypeVar.h index b3c455cf..b59e7c64 100644 --- a/Analysis/include/Luau/TypeVar.h +++ b/Analysis/include/Luau/TypeVar.h @@ -334,7 +334,6 @@ struct TableTypeVar // We need to know which is which when we stringify types. std::optional syntheticName; - std::map methodDefinitionLocations; // TODO: Remove with FFlag::LuauNoMethodLocations std::vector instantiatedTypeParams; std::vector instantiatedTypePackParams; ModuleName definitionModuleName; @@ -465,6 +464,14 @@ struct TypeVar final { } + // Re-assignes the content of the type, but doesn't change the owning arena and can't make type persistent. + void reassign(const TypeVar& rhs) + { + ty = rhs.ty; + normal = rhs.normal; + documentationSymbol = rhs.documentationSymbol; + } + TypeVariant ty; // Kludge: A persistent TypeVar is one that belongs to the global scope. @@ -486,6 +493,8 @@ struct TypeVar final TypeVar& operator=(const TypeVariant& rhs); TypeVar& operator=(TypeVariant&& rhs); + + TypeVar& operator=(const TypeVar& rhs); }; using SeenSet = std::set>; diff --git a/Analysis/include/Luau/Variant.h b/Analysis/include/Luau/Variant.h index c9c97c92..f637222e 100644 --- a/Analysis/include/Luau/Variant.h +++ b/Analysis/include/Luau/Variant.h @@ -6,6 +6,7 @@ #include #include #include +#include namespace Luau { diff --git a/Analysis/src/Autocomplete.cpp b/Analysis/src/Autocomplete.cpp index b988ed35..a8319c59 100644 --- a/Analysis/src/Autocomplete.cpp +++ b/Analysis/src/Autocomplete.cpp @@ -14,8 +14,7 @@ #include LUAU_FASTFLAGVARIABLE(LuauIfElseExprFixCompletionIssue, false); -LUAU_FASTFLAGVARIABLE(LuauFixAutocompleteClassSecurityLevel, false); -LUAU_FASTFLAG(LuauSelfCallAutocompleteFix) +LUAU_FASTFLAG(LuauSelfCallAutocompleteFix2) static const std::unordered_set kStatementStartingKeywords = { "while", "if", "local", "repeat", "function", "do", "for", "return", "break", "continue", "type", "export"}; @@ -248,7 +247,7 @@ static TypeCorrectKind checkTypeCorrectKind(const Module& module, TypeArena* typ ty = follow(ty); auto canUnify = [&typeArena](TypeId subTy, TypeId superTy) { - LUAU_ASSERT(!FFlag::LuauSelfCallAutocompleteFix); + LUAU_ASSERT(!FFlag::LuauSelfCallAutocompleteFix2); InternalErrorReporter iceReporter; UnifierSharedState unifierState(&iceReporter); @@ -267,7 +266,7 @@ static TypeCorrectKind checkTypeCorrectKind(const Module& module, TypeArena* typ TypeId expectedType = follow(*typeAtPosition); auto checkFunctionType = [typeArena, &canUnify, &expectedType](const FunctionTypeVar* ftv) { - if (FFlag::LuauSelfCallAutocompleteFix) + if (FFlag::LuauSelfCallAutocompleteFix2) { if (std::optional firstRetTy = first(ftv->retType)) return checkTypeMatch(typeArena, *firstRetTy, expectedType); @@ -308,7 +307,7 @@ static TypeCorrectKind checkTypeCorrectKind(const Module& module, TypeArena* typ } } - if (FFlag::LuauSelfCallAutocompleteFix) + if (FFlag::LuauSelfCallAutocompleteFix2) return checkTypeMatch(typeArena, ty, expectedType) ? TypeCorrectKind::Correct : TypeCorrectKind::None; else return canUnify(ty, expectedType) ? TypeCorrectKind::Correct : TypeCorrectKind::None; @@ -325,7 +324,7 @@ static void autocompleteProps(const Module& module, TypeArena* typeArena, TypeId const std::vector& nodes, AutocompleteEntryMap& result, std::unordered_set& seen, std::optional containingClass = std::nullopt) { - if (FFlag::LuauSelfCallAutocompleteFix) + if (FFlag::LuauSelfCallAutocompleteFix2) rootTy = follow(rootTy); ty = follow(ty); @@ -335,7 +334,7 @@ static void autocompleteProps(const Module& module, TypeArena* typeArena, TypeId seen.insert(ty); auto isWrongIndexer_DEPRECATED = [indexType, useStrictFunctionIndexers = !!get(ty)](Luau::TypeId type) { - LUAU_ASSERT(!FFlag::LuauSelfCallAutocompleteFix); + LUAU_ASSERT(!FFlag::LuauSelfCallAutocompleteFix2); if (indexType == PropIndexType::Key) return false; @@ -368,7 +367,7 @@ static void autocompleteProps(const Module& module, TypeArena* typeArena, TypeId } }; auto isWrongIndexer = [typeArena, rootTy, indexType](Luau::TypeId type) { - LUAU_ASSERT(FFlag::LuauSelfCallAutocompleteFix); + LUAU_ASSERT(FFlag::LuauSelfCallAutocompleteFix2); if (indexType == PropIndexType::Key) return false; @@ -382,10 +381,15 @@ static void autocompleteProps(const Module& module, TypeArena* typeArena, TypeId return calledWithSelf == ftv->hasSelf; } - if (std::optional firstArgTy = first(ftv->argTypes)) + // If a call is made with ':', it is invalid if a function has incompatible first argument or no arguments at all + // If a call is made with '.', but it was declared with 'self', it is considered invalid if first argument is compatible + if (calledWithSelf || ftv->hasSelf) { - if (checkTypeMatch(typeArena, rootTy, *firstArgTy)) - return calledWithSelf; + if (std::optional firstArgTy = first(ftv->argTypes)) + { + if (checkTypeMatch(typeArena, rootTy, *firstArgTy)) + return calledWithSelf; + } } return !calledWithSelf; @@ -427,7 +431,7 @@ static void autocompleteProps(const Module& module, TypeArena* typeArena, TypeId AutocompleteEntryKind::Property, type, prop.deprecated, - FFlag::LuauSelfCallAutocompleteFix ? isWrongIndexer(type) : isWrongIndexer_DEPRECATED(type), + FFlag::LuauSelfCallAutocompleteFix2 ? isWrongIndexer(type) : isWrongIndexer_DEPRECATED(type), typeCorrect, containingClass, &prop, @@ -462,8 +466,7 @@ static void autocompleteProps(const Module& module, TypeArena* typeArena, TypeId containingClass = containingClass.value_or(cls); fillProps(cls->props); if (cls->parent) - autocompleteProps(module, typeArena, rootTy, *cls->parent, indexType, nodes, result, seen, - FFlag::LuauFixAutocompleteClassSecurityLevel ? containingClass : cls); + autocompleteProps(module, typeArena, rootTy, *cls->parent, indexType, nodes, result, seen, containingClass); } else if (auto tbl = get(ty)) fillProps(tbl->props); @@ -471,7 +474,7 @@ static void autocompleteProps(const Module& module, TypeArena* typeArena, TypeId { autocompleteProps(module, typeArena, rootTy, mt->table, indexType, nodes, result, seen); - if (FFlag::LuauSelfCallAutocompleteFix) + if (FFlag::LuauSelfCallAutocompleteFix2) { if (auto mtable = get(mt->metatable)) fillMetatableProps(mtable); @@ -537,7 +540,7 @@ static void autocompleteProps(const Module& module, TypeArena* typeArena, TypeId AutocompleteEntryMap inner; std::unordered_set innerSeen; - if (!FFlag::LuauSelfCallAutocompleteFix) + if (!FFlag::LuauSelfCallAutocompleteFix2) innerSeen = seen; if (isNil(*iter)) @@ -563,7 +566,7 @@ static void autocompleteProps(const Module& module, TypeArena* typeArena, TypeId ++iter; } } - else if (auto pt = get(ty); pt && FFlag::LuauSelfCallAutocompleteFix) + else if (auto pt = get(ty); pt && FFlag::LuauSelfCallAutocompleteFix2) { if (pt->metatable) { @@ -571,7 +574,7 @@ static void autocompleteProps(const Module& module, TypeArena* typeArena, TypeId fillMetatableProps(mtable); } } - else if (FFlag::LuauSelfCallAutocompleteFix && get(get(ty))) + else if (FFlag::LuauSelfCallAutocompleteFix2 && get(get(ty))) { autocompleteProps(module, typeArena, rootTy, getSingletonTypes().stringType, indexType, nodes, result, seen); } @@ -1501,7 +1504,7 @@ static AutocompleteResult autocomplete(const SourceModule& sourceModule, const M TypeId ty = follow(*it); PropIndexType indexType = indexName->op == ':' ? PropIndexType::Colon : PropIndexType::Point; - if (!FFlag::LuauSelfCallAutocompleteFix && isString(ty)) + if (!FFlag::LuauSelfCallAutocompleteFix2 && isString(ty)) return {autocompleteProps(*module, typeArena, typeChecker.globalScope->bindings[AstName{"string"}].typeId, indexType, finder.ancestry), finder.ancestry}; else diff --git a/Analysis/src/BuiltinDefinitions.cpp b/Analysis/src/BuiltinDefinitions.cpp index 5ed6de67..98737b43 100644 --- a/Analysis/src/BuiltinDefinitions.cpp +++ b/Analysis/src/BuiltinDefinitions.cpp @@ -179,44 +179,13 @@ void registerBuiltinTypes(TypeChecker& typeChecker) LUAU_ASSERT(!typeChecker.globalTypes.typeVars.isFrozen()); LUAU_ASSERT(!typeChecker.globalTypes.typePacks.isFrozen()); - TypeId numberType = typeChecker.numberType; - TypeId booleanType = typeChecker.booleanType; TypeId nilType = typeChecker.nilType; TypeArena& arena = typeChecker.globalTypes; - TypePackId oneNumberPack = arena.addTypePack({numberType}); - TypePackId oneBooleanPack = arena.addTypePack({booleanType}); - - TypePackId numberVariadicList = arena.addTypePack(TypePackVar{VariadicTypePack{numberType}}); - TypePackId listOfAtLeastOneNumber = arena.addTypePack(TypePack{{numberType}, numberVariadicList}); - - TypeId listOfAtLeastOneNumberToNumberType = arena.addType(FunctionTypeVar{ - listOfAtLeastOneNumber, - oneNumberPack, - }); - - TypeId listOfAtLeastZeroNumbersToNumberType = arena.addType(FunctionTypeVar{numberVariadicList, oneNumberPack}); - LoadDefinitionFileResult loadResult = Luau::loadDefinitionFile(typeChecker, typeChecker.globalScope, getBuiltinDefinitionSource(), "@luau"); LUAU_ASSERT(loadResult.success); - TypeId mathLibType = getGlobalBinding(typeChecker, "math"); - if (TableTypeVar* ttv = getMutable(mathLibType)) - { - ttv->props["min"] = makeProperty(listOfAtLeastOneNumberToNumberType, "@luau/global/math.min"); - ttv->props["max"] = makeProperty(listOfAtLeastOneNumberToNumberType, "@luau/global/math.max"); - } - - TypeId bit32LibType = getGlobalBinding(typeChecker, "bit32"); - if (TableTypeVar* ttv = getMutable(bit32LibType)) - { - ttv->props["band"] = makeProperty(listOfAtLeastZeroNumbersToNumberType, "@luau/global/bit32.band"); - ttv->props["bor"] = makeProperty(listOfAtLeastZeroNumbersToNumberType, "@luau/global/bit32.bor"); - ttv->props["bxor"] = makeProperty(listOfAtLeastZeroNumbersToNumberType, "@luau/global/bit32.bxor"); - ttv->props["btest"] = makeProperty(arena.addType(FunctionTypeVar{listOfAtLeastOneNumber, oneBooleanPack}), "@luau/global/bit32.btest"); - } - TypeId genericK = arena.addType(GenericTypeVar{"K"}); TypeId genericV = arena.addType(GenericTypeVar{"V"}); TypeId mapOfKtoV = arena.addType(TableTypeVar{{}, TableIndexer(genericK, genericV), typeChecker.globalScope->level, TableState::Generic}); @@ -231,7 +200,7 @@ void registerBuiltinTypes(TypeChecker& typeChecker) addGlobalBinding(typeChecker, "string", it->second.type, "@luau"); - // next(t: Table, i: K | nil) -> (K, V) + // next(t: Table, i: K?) -> (K, V) TypePackId nextArgsTypePack = arena.addTypePack(TypePack{{mapOfKtoV, makeOption(typeChecker, arena, genericK)}}); addGlobalBinding(typeChecker, "next", arena.addType(FunctionTypeVar{{genericK, genericV}, {}, nextArgsTypePack, arena.addTypePack(TypePack{{genericK, genericV}})}), "@luau"); @@ -241,8 +210,7 @@ void registerBuiltinTypes(TypeChecker& typeChecker) TypeId pairsNext = arena.addType(FunctionTypeVar{nextArgsTypePack, arena.addTypePack(TypePack{{genericK, genericV}})}); TypePackId pairsReturnTypePack = arena.addTypePack(TypePack{{pairsNext, mapOfKtoV, nilType}}); - // NOTE we are missing 'i: K | nil' argument in the first return types' argument. - // pairs(t: Table) -> ((Table) -> (K, V), Table, nil) + // pairs(t: Table) -> ((Table, K?) -> (K, V), Table, nil) addGlobalBinding(typeChecker, "pairs", arena.addType(FunctionTypeVar{{genericK, genericV}, {}, pairsArgsTypePack, pairsReturnTypePack}), "@luau"); TypeId genericMT = arena.addType(GenericTypeVar{"MT"}); diff --git a/Analysis/src/Clone.cpp b/Analysis/src/Clone.cpp index 19e3383e..9180f309 100644 --- a/Analysis/src/Clone.cpp +++ b/Analysis/src/Clone.cpp @@ -9,7 +9,6 @@ LUAU_FASTFLAG(DebugLuauCopyBeforeNormalizing) LUAU_FASTINTVARIABLE(LuauTypeCloneRecursionLimit, 300) -LUAU_FASTFLAG(LuauNoMethodLocations) namespace Luau { @@ -241,8 +240,6 @@ void TypeCloner::operator()(const TableTypeVar& t) arg = clone(arg, dest, cloneState); ttv->definitionModuleName = t.definitionModuleName; - if (!FFlag::LuauNoMethodLocations) - ttv->methodDefinitionLocations = t.methodDefinitionLocations; ttv->tags = t.tags; } @@ -406,8 +403,6 @@ TypeId shallowClone(TypeId ty, TypeArena& dest, const TxnLog* log) { LUAU_ASSERT(!ttv->boundTo); TableTypeVar clone = TableTypeVar{ttv->props, ttv->indexer, ttv->level, ttv->state}; - if (!FFlag::LuauNoMethodLocations) - clone.methodDefinitionLocations = ttv->methodDefinitionLocations; clone.definitionModuleName = ttv->definitionModuleName; clone.name = ttv->name; clone.syntheticName = ttv->syntheticName; diff --git a/Analysis/src/EmbeddedBuiltinDefinitions.cpp b/Analysis/src/EmbeddedBuiltinDefinitions.cpp index f184b74e..2407e3ef 100644 --- a/Analysis/src/EmbeddedBuiltinDefinitions.cpp +++ b/Analysis/src/EmbeddedBuiltinDefinitions.cpp @@ -7,7 +7,10 @@ namespace Luau static const std::string kBuiltinDefinitionLuaSrc = R"BUILTIN_SRC( declare bit32: { - -- band, bor, bxor, and btest are declared in C++ + band: (...number) -> number, + bor: (...number) -> number, + bxor: (...number) -> number, + btest: (number, ...number) -> boolean, rrotate: (number, number) -> number, lrotate: (number, number) -> number, lshift: (number, number) -> number, @@ -50,7 +53,8 @@ declare math: { asin: (number) -> number, atan2: (number, number) -> number, - -- min and max are declared in C++. + min: (number, ...number) -> number, + max: (number, ...number) -> number, pi: number, huge: number, diff --git a/Analysis/src/Instantiation.cpp b/Analysis/src/Instantiation.cpp index 4a12027d..f145a511 100644 --- a/Analysis/src/Instantiation.cpp +++ b/Analysis/src/Instantiation.cpp @@ -4,8 +4,6 @@ #include "Luau/TxnLog.h" #include "Luau/TypeArena.h" -LUAU_FASTFLAG(LuauNoMethodLocations) - namespace Luau { @@ -110,8 +108,6 @@ TypeId ReplaceGenerics::clean(TypeId ty) if (const TableTypeVar* ttv = log->getMutable(ty)) { TableTypeVar clone = TableTypeVar{ttv->props, ttv->indexer, level, TableState::Free}; - if (!FFlag::LuauNoMethodLocations) - clone.methodDefinitionLocations = ttv->methodDefinitionLocations; clone.definitionModuleName = ttv->definitionModuleName; return addType(std::move(clone)); } diff --git a/Analysis/src/Quantify.cpp b/Analysis/src/Quantify.cpp index 8f2cc8e3..21775373 100644 --- a/Analysis/src/Quantify.cpp +++ b/Analysis/src/Quantify.cpp @@ -32,41 +32,6 @@ struct Quantifier final : TypeVarOnceVisitor LUAU_ASSERT(FFlag::DebugLuauDeferredConstraintResolution); } - void cycle(TypeId) override {} - void cycle(TypePackId) override {} - - bool operator()(TypeId ty, const FreeTypeVar& ftv) - { - return visit(ty, ftv); - } - - template - bool operator()(TypeId ty, const T& t) - { - return true; - } - - template - bool operator()(TypePackId, const T&) - { - return true; - } - - bool operator()(TypeId ty, const ConstrainedTypeVar&) - { - return true; - } - - bool operator()(TypeId ty, const TableTypeVar& ttv) - { - return visit(ty, ttv); - } - - bool operator()(TypePackId tp, const FreeTypePack& ftp) - { - return visit(tp, ftp); - } - /// @return true if outer encloses inner bool subsumes(Scope2* outer, Scope2* inner) { diff --git a/Analysis/src/Scope.cpp b/Analysis/src/Scope.cpp index 0a362a5e..011e28d4 100644 --- a/Analysis/src/Scope.cpp +++ b/Analysis/src/Scope.cpp @@ -2,8 +2,6 @@ #include "Luau/Scope.h" -LUAU_FASTFLAG(LuauTwoPassAliasDefinitionFix); - namespace Luau { @@ -19,8 +17,7 @@ Scope::Scope(const ScopePtr& parent, int subLevel) , returnType(parent->returnType) , level(parent->level.incr()) { - if (FFlag::LuauTwoPassAliasDefinitionFix) - level = level.incr(); + level = level.incr(); level.subLevel = subLevel; } diff --git a/Analysis/src/Substitution.cpp b/Analysis/src/Substitution.cpp index 50c516db..5a22deeb 100644 --- a/Analysis/src/Substitution.cpp +++ b/Analysis/src/Substitution.cpp @@ -10,7 +10,6 @@ LUAU_FASTFLAG(LuauLowerBoundsCalculation) LUAU_FASTINTVARIABLE(LuauTarjanChildLimit, 10000) -LUAU_FASTFLAG(LuauNoMethodLocations) namespace Luau { diff --git a/Analysis/src/TxnLog.cpp b/Analysis/src/TxnLog.cpp index e45c0cbd..4c6d54e0 100644 --- a/Analysis/src/TxnLog.cpp +++ b/Analysis/src/TxnLog.cpp @@ -7,6 +7,8 @@ #include #include +LUAU_FASTFLAG(LuauNonCopyableTypeVarFields) + namespace Luau { @@ -80,18 +82,32 @@ void TxnLog::commit() { for (auto& [ty, rep] : typeVarChanges) { - TypeArena* owningArena = ty->owningArena; - TypeVar* mtv = asMutable(ty); - *mtv = rep.get()->pending; - mtv->owningArena = owningArena; + if (FFlag::LuauNonCopyableTypeVarFields) + { + asMutable(ty)->reassign(rep.get()->pending); + } + else + { + TypeArena* owningArena = ty->owningArena; + TypeVar* mtv = asMutable(ty); + *mtv = rep.get()->pending; + mtv->owningArena = owningArena; + } } for (auto& [tp, rep] : typePackChanges) { - TypeArena* owningArena = tp->owningArena; - TypePackVar* mpv = asMutable(tp); - *mpv = rep.get()->pending; - mpv->owningArena = owningArena; + if (FFlag::LuauNonCopyableTypeVarFields) + { + asMutable(tp)->reassign(rep.get()->pending); + } + else + { + TypeArena* owningArena = tp->owningArena; + TypePackVar* mpv = asMutable(tp); + *mpv = rep.get()->pending; + mpv->owningArena = owningArena; + } } clear(); @@ -178,8 +194,13 @@ PendingType* TxnLog::queue(TypeId ty) // about this type, we don't want to mutate the parent's state. auto& pending = typeVarChanges[ty]; if (!pending) + { pending = std::make_unique(*ty); + if (FFlag::LuauNonCopyableTypeVarFields) + pending->pending.owningArena = nullptr; + } + return pending.get(); } @@ -191,8 +212,13 @@ PendingTypePack* TxnLog::queue(TypePackId tp) // about this type, we don't want to mutate the parent's state. auto& pending = typePackChanges[tp]; if (!pending) + { pending = std::make_unique(*tp); + if (FFlag::LuauNonCopyableTypeVarFields) + pending->pending.owningArena = nullptr; + } + return pending.get(); } @@ -229,14 +255,24 @@ PendingTypePack* TxnLog::pending(TypePackId tp) const PendingType* TxnLog::replace(TypeId ty, TypeVar replacement) { PendingType* newTy = queue(ty); - newTy->pending = replacement; + + if (FFlag::LuauNonCopyableTypeVarFields) + newTy->pending.reassign(replacement); + else + newTy->pending = replacement; + return newTy; } PendingTypePack* TxnLog::replace(TypePackId tp, TypePackVar replacement) { PendingTypePack* newTp = queue(tp); - newTp->pending = replacement; + + if (FFlag::LuauNonCopyableTypeVarFields) + newTp->pending.reassign(replacement); + else + newTp->pending = replacement; + return newTp; } diff --git a/Analysis/src/TypeInfer.cpp b/Analysis/src/TypeInfer.cpp index 4931bc59..447cd029 100644 --- a/Analysis/src/TypeInfer.cpp +++ b/Analysis/src/TypeInfer.cpp @@ -33,21 +33,20 @@ LUAU_FASTFLAG(LuauAutocompleteDynamicLimits) LUAU_FASTFLAGVARIABLE(LuauExpectedPropTypeFromIndexer, false) LUAU_FASTFLAGVARIABLE(LuauLowerBoundsCalculation, false) LUAU_FASTFLAGVARIABLE(DebugLuauFreezeDuringUnification, false) -LUAU_FASTFLAGVARIABLE(LuauSelfCallAutocompleteFix, false) +LUAU_FASTFLAGVARIABLE(LuauSelfCallAutocompleteFix2, false) LUAU_FASTFLAGVARIABLE(LuauReduceUnionRecursion, false) LUAU_FASTFLAGVARIABLE(LuauOnlyMutateInstantiatedTables, false) LUAU_FASTFLAGVARIABLE(LuauUnsealedTableLiteral, false) -LUAU_FASTFLAGVARIABLE(LuauTwoPassAliasDefinitionFix, false) LUAU_FASTFLAGVARIABLE(LuauReturnAnyInsteadOfICE, false) // Eventually removed as false. LUAU_FASTFLAG(LuauNormalizeFlagIsConservative) LUAU_FASTFLAGVARIABLE(LuauReturnTypeInferenceInNonstrict, false) LUAU_FASTFLAGVARIABLE(LuauRecursionLimitException, false); LUAU_FASTFLAGVARIABLE(LuauApplyTypeFunctionFix, false); -LUAU_FASTFLAGVARIABLE(LuauTypecheckIter, false); LUAU_FASTFLAGVARIABLE(LuauSuccessTypingForEqualityOperations, false) -LUAU_FASTFLAGVARIABLE(LuauNoMethodLocations, false); LUAU_FASTFLAGVARIABLE(LuauAlwaysQuantify, false); LUAU_FASTFLAGVARIABLE(LuauReportErrorsOnIndexerKeyMismatch, false) +LUAU_FASTFLAGVARIABLE(LuauFalsyPredicateReturnsNilInstead, false) +LUAU_FASTFLAGVARIABLE(LuauNonCopyableTypeVarFields, false) namespace Luau { @@ -358,8 +357,7 @@ ModulePtr TypeChecker::checkWithoutRecursionCheck(const SourceModule& module, Mo unifierState.cachedUnifyError.clear(); unifierState.skipCacheForType.clear(); - if (FFlag::LuauTwoPassAliasDefinitionFix) - duplicateTypeAliases.clear(); + duplicateTypeAliases.clear(); return std::move(currentModule); } @@ -610,7 +608,7 @@ LUAU_NOINLINE void TypeChecker::checkBlockTypeAliases(const ScopePtr& scope, std { if (const auto& typealias = stat->as()) { - if (FFlag::LuauTwoPassAliasDefinitionFix && typealias->name == kParseNameError) + if (typealias->name == kParseNameError) continue; auto& bindings = typealias->exported ? scope->exportedTypeBindings : scope->privateTypeBindings; @@ -619,7 +617,16 @@ LUAU_NOINLINE void TypeChecker::checkBlockTypeAliases(const ScopePtr& scope, std TypeId type = bindings[name].type; if (get(follow(type))) { - *asMutable(type) = *errorRecoveryType(anyType); + if (FFlag::LuauNonCopyableTypeVarFields) + { + TypeVar* mty = asMutable(follow(type)); + mty->reassign(*errorRecoveryType(anyType)); + } + else + { + *asMutable(type) = *errorRecoveryType(anyType); + } + reportError(TypeError{typealias->location, OccursCheckFailed{}}); } } @@ -1131,45 +1138,43 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatForIn& forin) iterTy = instantiate(scope, checkExpr(scope, *firstValue).type, firstValue->location); } - if (FFlag::LuauTypecheckIter) + if (std::optional iterMM = findMetatableEntry(iterTy, "__iter", firstValue->location)) { - if (std::optional iterMM = findMetatableEntry(iterTy, "__iter", firstValue->location)) + // if __iter metamethod is present, it will be called and the results are going to be called as if they are functions + // TODO: this needs to typecheck all returned values by __iter as if they were for loop arguments + // the structure of the function makes it difficult to do this especially since we don't have actual expressions, only types + for (TypeId var : varTypes) + unify(anyType, var, forin.location); + + return check(loopScope, *forin.body); + } + + if (const TableTypeVar* iterTable = get(iterTy)) + { + // TODO: note that this doesn't cleanly handle iteration over mixed tables and tables without an indexer + // this behavior is more or less consistent with what we do for pairs(), but really both are pretty wrong and need revisiting + if (iterTable->indexer) { - // if __iter metamethod is present, it will be called and the results are going to be called as if they are functions - // TODO: this needs to typecheck all returned values by __iter as if they were for loop arguments - // the structure of the function makes it difficult to do this especially since we don't have actual expressions, only types + if (varTypes.size() > 0) + unify(iterTable->indexer->indexType, varTypes[0], forin.location); + + if (varTypes.size() > 1) + unify(iterTable->indexer->indexResultType, varTypes[1], forin.location); + + for (size_t i = 2; i < varTypes.size(); ++i) + unify(nilType, varTypes[i], forin.location); + } + else + { + TypeId varTy = errorRecoveryType(loopScope); + for (TypeId var : varTypes) - unify(anyType, var, forin.location); + unify(varTy, var, forin.location); - return check(loopScope, *forin.body); + reportError(firstValue->location, GenericError{"Cannot iterate over a table without indexer"}); } - else if (const TableTypeVar* iterTable = get(iterTy)) - { - // TODO: note that this doesn't cleanly handle iteration over mixed tables and tables without an indexer - // this behavior is more or less consistent with what we do for pairs(), but really both are pretty wrong and need revisiting - if (iterTable->indexer) - { - if (varTypes.size() > 0) - unify(iterTable->indexer->indexType, varTypes[0], forin.location); - if (varTypes.size() > 1) - unify(iterTable->indexer->indexResultType, varTypes[1], forin.location); - - for (size_t i = 2; i < varTypes.size(); ++i) - unify(nilType, varTypes[i], forin.location); - } - else - { - TypeId varTy = errorRecoveryType(loopScope); - - for (TypeId var : varTypes) - unify(varTy, var, forin.location); - - reportError(firstValue->location, GenericError{"Cannot iterate over a table without indexer"}); - } - - return check(loopScope, *forin.body); - } + return check(loopScope, *forin.body); } const FunctionTypeVar* iterFunc = get(iterTy); @@ -1334,7 +1339,7 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatTypeAlias& typealias Name name = typealias.name.value; // If the alias is missing a name, we can't do anything with it. Ignore it. - if (FFlag::LuauTwoPassAliasDefinitionFix && name == kParseNameError) + if (name == kParseNameError) return; std::optional binding; @@ -1353,8 +1358,7 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatTypeAlias& typealias reportError(TypeError{typealias.location, DuplicateTypeDefinition{name, location}}); bindingsMap[name] = TypeFun{binding->typeParams, binding->typePackParams, errorRecoveryType(anyType)}; - if (FFlag::LuauTwoPassAliasDefinitionFix) - duplicateTypeAliases.insert({typealias.exported, name}); + duplicateTypeAliases.insert({typealias.exported, name}); } else { @@ -1378,7 +1382,7 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatTypeAlias& typealias { // If the first pass failed (this should mean a duplicate definition), the second pass isn't going to be // interesting. - if (FFlag::LuauTwoPassAliasDefinitionFix && duplicateTypeAliases.find({typealias.exported, name})) + if (duplicateTypeAliases.find({typealias.exported, name})) return; if (!binding) @@ -1422,9 +1426,6 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatTypeAlias& typealias { // This is a shallow clone, original recursive links to self are not updated TableTypeVar clone = TableTypeVar{ttv->props, ttv->indexer, ttv->level, ttv->state}; - - if (!FFlag::LuauNoMethodLocations) - clone.methodDefinitionLocations = ttv->methodDefinitionLocations; clone.definitionModuleName = ttv->definitionModuleName; clone.name = name; @@ -1462,9 +1463,8 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatTypeAlias& typealias } TypeId& bindingType = bindingsMap[name].type; - bool ok = unify(ty, bindingType, typealias.location); - if (FFlag::LuauTwoPassAliasDefinitionFix && ok) + if (unify(ty, bindingType, typealias.location)) bindingType = ty; if (FFlag::LuauLowerBoundsCalculation) @@ -1532,7 +1532,7 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatDeclareClass& declar ftv->argNames.insert(ftv->argNames.begin(), FunctionArgument{"self", {}}); ftv->argTypes = addTypePack(TypePack{{classTy}, ftv->argTypes}); - if (FFlag::LuauSelfCallAutocompleteFix) + if (FFlag::LuauSelfCallAutocompleteFix2) ftv->hasSelf = true; } } @@ -3099,8 +3099,6 @@ TypeId TypeChecker::checkFunctionName(const ScopePtr& scope, AstExpr& funName, T property.type = freshTy(); property.location = indexName->indexLocation; - if (!FFlag::LuauNoMethodLocations) - ttv->methodDefinitionLocations[name] = funName.location; return property.type; } else if (funName.is()) @@ -4393,8 +4391,6 @@ TypeId Anyification::clean(TypeId ty) if (const TableTypeVar* ttv = log->getMutable(ty)) { TableTypeVar clone = TableTypeVar{ttv->props, ttv->indexer, ttv->level, TableState::Sealed}; - if (!FFlag::LuauNoMethodLocations) - clone.methodDefinitionLocations = ttv->methodDefinitionLocations; clone.definitionModuleName = ttv->definitionModuleName; clone.name = ttv->name; clone.syntheticName = ttv->syntheticName; @@ -4705,8 +4701,11 @@ TypeIdPredicate TypeChecker::mkTruthyPredicate(bool sense) if (isNil(ty)) return sense ? std::nullopt : std::optional(ty); - // at this point, anything else is kept if sense is true, or eliminated otherwise - return sense ? std::optional(ty) : std::nullopt; + // at this point, anything else is kept if sense is true, or replaced by nil + if (FFlag::LuauFalsyPredicateReturnsNilInstead) + return sense ? ty : nilType; + else + return sense ? std::optional(ty) : std::nullopt; }; } diff --git a/Analysis/src/TypePack.cpp b/Analysis/src/TypePack.cpp index 30503233..82451bd1 100644 --- a/Analysis/src/TypePack.cpp +++ b/Analysis/src/TypePack.cpp @@ -5,6 +5,8 @@ #include +LUAU_FASTFLAG(LuauNonCopyableTypeVarFields) + namespace Luau { @@ -36,6 +38,25 @@ TypePackVar& TypePackVar::operator=(TypePackVariant&& tp) return *this; } +TypePackVar& TypePackVar::operator=(const TypePackVar& rhs) +{ + if (FFlag::LuauNonCopyableTypeVarFields) + { + LUAU_ASSERT(owningArena == rhs.owningArena); + LUAU_ASSERT(!rhs.persistent); + + reassign(rhs); + } + else + { + ty = rhs.ty; + persistent = rhs.persistent; + owningArena = rhs.owningArena; + } + + return *this; +} + TypePackIterator::TypePackIterator(TypePackId typePack) : TypePackIterator(typePack, TxnLog::empty()) { diff --git a/Analysis/src/TypeVar.cpp b/Analysis/src/TypeVar.cpp index 12cbed91..33bfe254 100644 --- a/Analysis/src/TypeVar.cpp +++ b/Analysis/src/TypeVar.cpp @@ -24,6 +24,7 @@ LUAU_FASTINTVARIABLE(LuauTypeMaximumStringifierLength, 500) LUAU_FASTINTVARIABLE(LuauTableTypeMaximumStringifierLength, 0) LUAU_FASTINT(LuauTypeInferRecursionLimit) LUAU_FASTFLAG(LuauSubtypingAddOptPropsToUnsealedTables) +LUAU_FASTFLAG(LuauNonCopyableTypeVarFields) namespace Luau { @@ -644,6 +645,26 @@ TypeVar& TypeVar::operator=(TypeVariant&& rhs) return *this; } +TypeVar& TypeVar::operator=(const TypeVar& rhs) +{ + if (FFlag::LuauNonCopyableTypeVarFields) + { + LUAU_ASSERT(owningArena == rhs.owningArena); + LUAU_ASSERT(!rhs.persistent); + + reassign(rhs); + } + else + { + ty = rhs.ty; + persistent = rhs.persistent; + normal = rhs.normal; + owningArena = rhs.owningArena; + } + + return *this; +} + TypeId makeFunction(TypeArena& arena, std::optional selfType, std::initializer_list generics, std::initializer_list genericPacks, std::initializer_list paramTypes, std::initializer_list paramNames, std::initializer_list retTypes); diff --git a/Ast/src/Parser.cpp b/Ast/src/Parser.cpp index eaf19914..95bce3ee 100644 --- a/Ast/src/Parser.cpp +++ b/Ast/src/Parser.cpp @@ -12,6 +12,7 @@ LUAU_FASTINTVARIABLE(LuauRecursionLimit, 1000) LUAU_FASTINTVARIABLE(LuauParseErrorLimit, 100) LUAU_FASTFLAGVARIABLE(LuauParserFunctionKeywordAsTypeHelp, false) +LUAU_FASTFLAGVARIABLE(LuauReturnTypeTokenConfusion, false) namespace Luau { @@ -1118,8 +1119,12 @@ AstTypePack* Parser::parseTypeList(TempVector& result, TempVector Parser::parseOptionalReturnTypeAnnotation() { - if (options.allowTypeAnnotations && lexer.current().type == ':') + if (options.allowTypeAnnotations && + (lexer.current().type == ':' || (FFlag::LuauReturnTypeTokenConfusion && lexer.current().type == Lexeme::SkinnyArrow))) { + if (FFlag::LuauReturnTypeTokenConfusion && lexer.current().type == Lexeme::SkinnyArrow) + report(lexer.current().location, "Function return type annotations are written after ':' instead of '->'"); + nextLexeme(); unsigned int oldRecursionCount = recursionCounter; @@ -1350,8 +1355,12 @@ AstTypeOrPack Parser::parseFunctionTypeAnnotation(bool allowPack) AstArray paramTypes = copy(params); + bool returnTypeIntroducer = + FFlag::LuauReturnTypeTokenConfusion ? lexer.current().type == Lexeme::SkinnyArrow || lexer.current().type == ':' : false; + // Not a function at all. Just a parenthesized type. Or maybe a type pack with a single element - if (params.size() == 1 && !varargAnnotation && monomorphic && lexer.current().type != Lexeme::SkinnyArrow) + if (params.size() == 1 && !varargAnnotation && monomorphic && + (FFlag::LuauReturnTypeTokenConfusion ? !returnTypeIntroducer : lexer.current().type != Lexeme::SkinnyArrow)) { if (allowPack) return {{}, allocator.alloc(begin.location, AstTypeList{paramTypes, nullptr})}; @@ -1359,7 +1368,7 @@ AstTypeOrPack Parser::parseFunctionTypeAnnotation(bool allowPack) return {params[0], {}}; } - if (lexer.current().type != Lexeme::SkinnyArrow && monomorphic && allowPack) + if ((FFlag::LuauReturnTypeTokenConfusion ? !returnTypeIntroducer : lexer.current().type != Lexeme::SkinnyArrow) && monomorphic && allowPack) return {{}, allocator.alloc(begin.location, AstTypeList{paramTypes, varargAnnotation})}; AstArray> paramNames = copy(names); @@ -1373,8 +1382,13 @@ AstType* Parser::parseFunctionTypeAnnotationTail(const Lexeme& begin, AstArray' instead of ':'"); + lexer.next(); + } // Users occasionally write '()' as the 'unit' type when they actually want to use 'nil', here we'll try to give a more specific error - if (lexer.current().type != Lexeme::SkinnyArrow && generics.size == 0 && genericPacks.size == 0 && params.size == 0) + else if (lexer.current().type != Lexeme::SkinnyArrow && generics.size == 0 && genericPacks.size == 0 && params.size == 0) { report(Location(begin.location, lexer.previousLocation()), "Expected '->' after '()' when parsing function type; did you mean 'nil'?"); diff --git a/Compiler/src/BytecodeBuilder.cpp b/Compiler/src/BytecodeBuilder.cpp index 3aa12d99..597b2f0a 100644 --- a/Compiler/src/BytecodeBuilder.cpp +++ b/Compiler/src/BytecodeBuilder.cpp @@ -6,8 +6,6 @@ #include #include -LUAU_FASTFLAG(LuauCompileNestedClosureO2) - namespace Luau { @@ -390,17 +388,15 @@ int32_t BytecodeBuilder::addConstantClosure(uint32_t fid) int16_t BytecodeBuilder::addChildFunction(uint32_t fid) { - if (FFlag::LuauCompileNestedClosureO2) - if (int16_t* cache = protoMap.find(fid)) - return *cache; + if (int16_t* cache = protoMap.find(fid)) + return *cache; uint32_t id = uint32_t(protos.size()); if (id >= kMaxClosureCount) return -1; - if (FFlag::LuauCompileNestedClosureO2) - protoMap[fid] = int16_t(id); + protoMap[fid] = int16_t(id); protos.push_back(fid); return int16_t(id); diff --git a/Compiler/src/Compiler.cpp b/Compiler/src/Compiler.cpp index eea56c60..7431cde4 100644 --- a/Compiler/src/Compiler.cpp +++ b/Compiler/src/Compiler.cpp @@ -16,7 +16,6 @@ #include #include -LUAU_FASTFLAGVARIABLE(LuauCompileIter, false) LUAU_FASTFLAGVARIABLE(LuauCompileIterNoPairs, false) LUAU_FASTINTVARIABLE(LuauCompileLoopUnrollThreshold, 25) @@ -26,8 +25,6 @@ LUAU_FASTINTVARIABLE(LuauCompileInlineThreshold, 25) LUAU_FASTINTVARIABLE(LuauCompileInlineThresholdMaxBoost, 300) LUAU_FASTINTVARIABLE(LuauCompileInlineDepth, 5) -LUAU_FASTFLAGVARIABLE(LuauCompileNestedClosureO2, false) - namespace Luau { @@ -172,30 +169,6 @@ struct Compiler return node->as(); } - bool canInlineFunctionBody(AstStat* stat) - { - if (FFlag::LuauCompileNestedClosureO2) - return true; // TODO: remove this function - - struct CanInlineVisitor : AstVisitor - { - bool result = true; - - bool visit(AstExprFunction* node) override - { - result = false; - - // short-circuit to avoid analyzing nested closure bodies - return false; - } - }; - - CanInlineVisitor canInline; - stat->visit(&canInline); - - return canInline.result; - } - uint32_t compileFunction(AstExprFunction* func) { LUAU_TIMETRACE_SCOPE("Compiler::compileFunction", "Compiler"); @@ -268,7 +241,7 @@ struct Compiler f.upvals = upvals; // record information for inlining - if (options.optimizationLevel >= 2 && !func->vararg && canInlineFunctionBody(func->body) && !getfenvUsed && !setfenvUsed) + if (options.optimizationLevel >= 2 && !func->vararg && !getfenvUsed && !setfenvUsed) { f.canInline = true; f.stackSize = stackSize; @@ -827,110 +800,62 @@ struct Compiler if (pid < 0) CompileError::raise(expr->location, "Exceeded closure limit; simplify the code to compile"); - if (FFlag::LuauCompileNestedClosureO2) - { - captures.clear(); - captures.reserve(f->upvals.size()); - - for (AstLocal* uv : f->upvals) - { - LUAU_ASSERT(uv->functionDepth < expr->functionDepth); - - if (int reg = getLocalReg(uv); reg >= 0) - { - // note: we can't check if uv is an upvalue in the current frame because inlining can migrate from upvalues to locals - Variable* ul = variables.find(uv); - bool immutable = !ul || !ul->written; - - captures.push_back({immutable ? LCT_VAL : LCT_REF, uint8_t(reg)}); - } - else if (const Constant* uc = locstants.find(uv); uc && uc->type != Constant::Type_Unknown) - { - // inlining can result in an upvalue capture of a constant, in which case we can't capture without a temporary register - uint8_t reg = allocReg(expr, 1); - compileExprConstant(expr, uc, reg); - - captures.push_back({LCT_VAL, reg}); - } - else - { - LUAU_ASSERT(uv->functionDepth < expr->functionDepth - 1); - - // get upvalue from parent frame - // note: this will add uv to the current upvalue list if necessary - uint8_t uid = getUpval(uv); - - captures.push_back({LCT_UPVAL, uid}); - } - } - - // Optimization: when closure has no upvalues, or upvalues are safe to share, instead of allocating it every time we can share closure - // objects (this breaks assumptions about function identity which can lead to setfenv not working as expected, so we disable this when it - // is used) - int16_t shared = -1; - - if (options.optimizationLevel >= 1 && shouldShareClosure(expr) && !setfenvUsed) - { - int32_t cid = bytecode.addConstantClosure(f->id); - - if (cid >= 0 && cid < 32768) - shared = int16_t(cid); - } - - if (shared >= 0) - bytecode.emitAD(LOP_DUPCLOSURE, target, shared); - else - bytecode.emitAD(LOP_NEWCLOSURE, target, pid); - - for (const Capture& c : captures) - bytecode.emitABC(LOP_CAPTURE, uint8_t(c.type), c.data, 0); - - return; - } - - bool shared = false; - - // Optimization: when closure has no upvalues, or upvalues are safe to share, instead of allocating it every time we can share closure - // objects (this breaks assumptions about function identity which can lead to setfenv not working as expected, so we disable this when it - // is used) - if (options.optimizationLevel >= 1 && shouldShareClosure(expr) && !setfenvUsed) - { - int32_t cid = bytecode.addConstantClosure(f->id); - - if (cid >= 0 && cid < 32768) - { - bytecode.emitAD(LOP_DUPCLOSURE, target, int16_t(cid)); - shared = true; - } - } - - if (!shared) - bytecode.emitAD(LOP_NEWCLOSURE, target, pid); + // we use a scratch vector to reduce allocations; this is safe since compileExprFunction is not reentrant + captures.clear(); + captures.reserve(f->upvals.size()); for (AstLocal* uv : f->upvals) { LUAU_ASSERT(uv->functionDepth < expr->functionDepth); - Variable* ul = variables.find(uv); - bool immutable = !ul || !ul->written; - - if (uv->functionDepth == expr->functionDepth - 1) + if (int reg = getLocalReg(uv); reg >= 0) { - // get local variable - int reg = getLocalReg(uv); - LUAU_ASSERT(reg >= 0); + // note: we can't check if uv is an upvalue in the current frame because inlining can migrate from upvalues to locals + Variable* ul = variables.find(uv); + bool immutable = !ul || !ul->written; - bytecode.emitABC(LOP_CAPTURE, uint8_t(immutable ? LCT_VAL : LCT_REF), uint8_t(reg), 0); + captures.push_back({immutable ? LCT_VAL : LCT_REF, uint8_t(reg)}); + } + else if (const Constant* uc = locstants.find(uv); uc && uc->type != Constant::Type_Unknown) + { + // inlining can result in an upvalue capture of a constant, in which case we can't capture without a temporary register + uint8_t reg = allocReg(expr, 1); + compileExprConstant(expr, uc, reg); + + captures.push_back({LCT_VAL, reg}); } else { + LUAU_ASSERT(uv->functionDepth < expr->functionDepth - 1); + // get upvalue from parent frame // note: this will add uv to the current upvalue list if necessary uint8_t uid = getUpval(uv); - bytecode.emitABC(LOP_CAPTURE, LCT_UPVAL, uid, 0); + captures.push_back({LCT_UPVAL, uid}); } } + + // Optimization: when closure has no upvalues, or upvalues are safe to share, instead of allocating it every time we can share closure + // objects (this breaks assumptions about function identity which can lead to setfenv not working as expected, so we disable this when it + // is used) + int16_t shared = -1; + + if (options.optimizationLevel >= 1 && shouldShareClosure(expr) && !setfenvUsed) + { + int32_t cid = bytecode.addConstantClosure(f->id); + + if (cid >= 0 && cid < 32768) + shared = int16_t(cid); + } + + if (shared >= 0) + bytecode.emitAD(LOP_DUPCLOSURE, target, shared); + else + bytecode.emitAD(LOP_NEWCLOSURE, target, pid); + + for (const Capture& c : captures) + bytecode.emitABC(LOP_CAPTURE, uint8_t(c.type), c.data, 0); } LuauOpcode getUnaryOp(AstExprUnary::Op op) @@ -2511,30 +2436,6 @@ struct Compiler pushLocal(stat->vars.data[i], uint8_t(vars + i)); } - bool canUnrollForBody(AstStatFor* stat) - { - if (FFlag::LuauCompileNestedClosureO2) - return true; // TODO: remove this function - - struct CanUnrollVisitor : AstVisitor - { - bool result = true; - - bool visit(AstExprFunction* node) override - { - result = false; - - // short-circuit to avoid analyzing nested closure bodies - return false; - } - }; - - CanUnrollVisitor canUnroll; - stat->body->visit(&canUnroll); - - return canUnroll.result; - } - bool tryCompileUnrolledFor(AstStatFor* stat, int thresholdBase, int thresholdMaxBoost) { Constant one = {Constant::Type_Number}; @@ -2560,12 +2461,6 @@ struct Compiler return false; } - if (!canUnrollForBody(stat)) - { - bytecode.addDebugRemark("loop unroll failed: unsupported loop body"); - return false; - } - if (Variable* lv = variables.find(stat->var); lv && lv->written) { bytecode.addDebugRemark("loop unroll failed: mutable loop variable"); @@ -2730,12 +2625,12 @@ struct Compiler uint8_t vars = allocReg(stat, std::max(unsigned(stat->vars.size), 2u)); LUAU_ASSERT(vars == regs + 3); - // Optimization: when we iterate through pairs/ipairs, we generate special bytecode that optimizes the traversal using internal iteration - // index These instructions dynamically check if generator is equal to next/inext and bail out They assume that the generator produces 2 - // variables, which is why we allocate at least 2 above (see vars assignment) - LuauOpcode skipOp = FFlag::LuauCompileIter ? LOP_FORGPREP : LOP_JUMP; + LuauOpcode skipOp = LOP_FORGPREP; LuauOpcode loopOp = LOP_FORGLOOP; + // Optimization: when we iterate via pairs/ipairs, we generate special bytecode that optimizes the traversal using internal iteration index + // These instructions dynamically check if generator is equal to next/inext and bail out + // They assume that the generator produces 2 variables, which is why we allocate at least 2 above (see vars assignment) if (options.optimizationLevel >= 1 && stat->vars.size <= 2) { if (stat->values.size == 1 && stat->values.data[0]->is()) diff --git a/VM/src/lapi.cpp b/VM/src/lapi.cpp index f86371da..3c3b7bd0 100644 --- a/VM/src/lapi.cpp +++ b/VM/src/lapi.cpp @@ -14,6 +14,26 @@ #include +/* + * This file contains most implementations of core Lua APIs from lua.h. + * + * These implementations should use api_check macros to verify that stack and type contracts hold; it's the callers + * responsibility to, for example, pass a valid table index to lua_rawgetfield. Generally errors should only be raised + * for conditions caller can't predict such as an out-of-memory error. + * + * The caller is expected to handle stack reservation (by using less than LUA_MINSTACK slots or by calling lua_checkstack). + * To ensure this is handled correctly, use api_incr_top(L) when pushing values to the stack. + * + * Functions that push any collectable objects to the stack *should* call luaC_checkthreadsleep. Failure to do this can result + * in stack references that point to dead objects since sleeping threads don't get rescanned. + * + * Functions that push newly created objects to the stack *should* call luaC_checkGC in addition to luaC_checkthreadsleep. + * Failure to do this can result in OOM since GC may never run. + * + * Note that luaC_checkGC may scan the thread and put it back to sleep; functions that call both before pushing objects must + * therefore call luaC_checkGC before luaC_checkthreadsleep to guarantee the object is pushed to an awake thread. + */ + const char* lua_ident = "$Lua: Lua 5.1.4 Copyright (C) 1994-2008 Lua.org, PUC-Rio $\n" "$Authors: R. Ierusalimschy, L. H. de Figueiredo & W. Celes $\n" "$URL: www.lua.org $\n"; @@ -221,15 +241,13 @@ void lua_insert(lua_State* L, int idx) void lua_replace(lua_State* L, int idx) { - /* explicit test for incompatible code */ - if (idx == LUA_ENVIRONINDEX && L->ci == L->base_ci) - luaG_runerror(L, "no calling environment"); api_checknelems(L, 1); luaC_checkthreadsleep(L); StkId o = index2addr(L, idx); api_checkvalidindex(L, o); if (idx == LUA_ENVIRONINDEX) { + api_check(L, L->ci != L->base_ci); Closure* func = curr_func(L); api_check(L, ttistable(L->top - 1)); func->env = hvalue(L->top - 1); @@ -443,9 +461,7 @@ const float* lua_tovector(lua_State* L, int idx) { StkId o = index2addr(L, idx); if (!ttisvector(o)) - { return NULL; - } return vvalue(o); } @@ -460,11 +476,6 @@ int lua_objlen(lua_State* L, int idx) return uvalue(o)->len; case LUA_TTABLE: return luaH_getn(hvalue(o)); - case LUA_TNUMBER: - { - int l = (luaV_tostring(L, o) ? tsvalue(o)->len : 0); - return l; - } default: return 0; } @@ -752,10 +763,9 @@ void lua_setsafeenv(lua_State* L, int objindex, int enabled) int lua_getmetatable(lua_State* L, int objindex) { - const TValue* obj; + luaC_checkthreadsleep(L); Table* mt = NULL; - int res; - obj = index2addr(L, objindex); + const TValue* obj = index2addr(L, objindex); switch (ttype(obj)) { case LUA_TTABLE: @@ -768,21 +778,18 @@ int lua_getmetatable(lua_State* L, int objindex) mt = L->global->mt[ttype(obj)]; break; } - if (mt == NULL) - res = 0; - else + if (mt) { sethvalue(L, L->top, mt); api_incr_top(L); - res = 1; } - return res; + return mt != NULL; } void lua_getfenv(lua_State* L, int idx) { - StkId o; - o = index2addr(L, idx); + luaC_checkthreadsleep(L); + StkId o = index2addr(L, idx); api_checkvalidindex(L, o); switch (ttype(o)) { @@ -806,9 +813,8 @@ void lua_getfenv(lua_State* L, int idx) void lua_settable(lua_State* L, int idx) { - StkId t; api_checknelems(L, 2); - t = index2addr(L, idx); + StkId t = index2addr(L, idx); api_checkvalidindex(L, t); luaV_settable(L, t, L->top - 2, L->top - 1); L->top -= 2; /* pop index and value */ @@ -817,22 +823,20 @@ void lua_settable(lua_State* L, int idx) void lua_setfield(lua_State* L, int idx, const char* k) { - StkId t; - TValue key; api_checknelems(L, 1); - t = index2addr(L, idx); + StkId t = index2addr(L, idx); api_checkvalidindex(L, t); + TValue key; setsvalue(L, &key, luaS_new(L, k)); luaV_settable(L, t, &key, L->top - 1); - L->top--; /* pop value */ + L->top--; return; } void lua_rawset(lua_State* L, int idx) { - StkId t; api_checknelems(L, 2); - t = index2addr(L, idx); + StkId t = index2addr(L, idx); api_check(L, ttistable(t)); if (hvalue(t)->readonly) luaG_runerror(L, "Attempt to modify a readonly table"); @@ -844,9 +848,8 @@ void lua_rawset(lua_State* L, int idx) void lua_rawseti(lua_State* L, int idx, int n) { - StkId o; api_checknelems(L, 1); - o = index2addr(L, idx); + StkId o = index2addr(L, idx); api_check(L, ttistable(o)); if (hvalue(o)->readonly) luaG_runerror(L, "Attempt to modify a readonly table"); @@ -858,14 +861,11 @@ void lua_rawseti(lua_State* L, int idx, int n) int lua_setmetatable(lua_State* L, int objindex) { - TValue* obj; - Table* mt; api_checknelems(L, 1); - obj = index2addr(L, objindex); + TValue* obj = index2addr(L, objindex); api_checkvalidindex(L, obj); - if (ttisnil(L->top - 1)) - mt = NULL; - else + Table* mt = NULL; + if (!ttisnil(L->top - 1)) { api_check(L, ttistable(L->top - 1)); mt = hvalue(L->top - 1); @@ -900,10 +900,9 @@ int lua_setmetatable(lua_State* L, int objindex) int lua_setfenv(lua_State* L, int idx) { - StkId o; int res = 1; api_checknelems(L, 1); - o = index2addr(L, idx); + StkId o = index2addr(L, idx); api_checkvalidindex(L, o); api_check(L, ttistable(L->top - 1)); switch (ttype(o)) @@ -970,24 +969,21 @@ static void f_call(lua_State* L, void* ud) int lua_pcall(lua_State* L, int nargs, int nresults, int errfunc) { - struct CallS c; - int status; - ptrdiff_t func; api_checknelems(L, nargs + 1); api_check(L, L->status == 0); checkresults(L, nargs, nresults); - if (errfunc == 0) - func = 0; - else + ptrdiff_t func = 0; + if (errfunc != 0) { StkId o = index2addr(L, errfunc); api_checkvalidindex(L, o); func = savestack(L, o); } + struct CallS c; c.func = L->top - (nargs + 1); /* function to be called */ c.nresults = nresults; - status = luaD_pcall(L, f_call, &c, savestack(L, c.func), func); + int status = luaD_pcall(L, f_call, &c, savestack(L, c.func), func); adjustresults(L, nresults); return status; @@ -1247,12 +1243,10 @@ const char* lua_getupvalue(lua_State* L, int funcindex, int n) const char* lua_setupvalue(lua_State* L, int funcindex, int n) { - const char* name; - TValue* val; - StkId fi; - fi = index2addr(L, funcindex); api_checknelems(L, 1); - name = aux_upvalue(fi, n, &val); + StkId fi = index2addr(L, funcindex); + TValue* val; + const char* name = aux_upvalue(fi, n, &val); if (name) { L->top--; @@ -1319,14 +1313,16 @@ void lua_setuserdatadtor(lua_State* L, int tag, void (*dtor)(lua_State*, void*)) void lua_clonefunction(lua_State* L, int idx) { + luaC_checkGC(L); + luaC_checkthreadsleep(L); StkId p = index2addr(L, idx); api_check(L, isLfunction(p)); - - luaC_checkthreadsleep(L); - Closure* cl = clvalue(p); - Closure* newcl = luaF_newLclosure(L, 0, L->gt, cl->l.p); - setclvalue(L, L->top - 1, newcl); + Closure* newcl = luaF_newLclosure(L, cl->nupvalues, L->gt, cl->l.p); + for (int i = 0; i < cl->nupvalues; ++i) + setobj2n(L, &newcl->l.uprefs[i], &cl->l.uprefs[i]); + setclvalue(L, L->top, newcl); + api_incr_top(L); } lua_Callbacks* lua_callbacks(lua_State* L) diff --git a/VM/src/lbuiltins.cpp b/VM/src/lbuiltins.cpp index cc6e560a..deaf1407 100644 --- a/VM/src/lbuiltins.cpp +++ b/VM/src/lbuiltins.cpp @@ -1018,18 +1018,20 @@ static int luauF_tunpack(lua_State* L, StkId res, TValue* arg0, int nresults, St static int luauF_vector(lua_State* L, StkId res, TValue* arg0, int nresults, StkId args, int nparams) { -#if LUA_VECTOR_SIZE == 4 - if (nparams >= 4 && nresults <= 1 && ttisnumber(arg0) && ttisnumber(args) && ttisnumber(args + 1) && ttisnumber(args + 2)) -#else if (nparams >= 3 && nresults <= 1 && ttisnumber(arg0) && ttisnumber(args) && ttisnumber(args + 1)) -#endif { double x = nvalue(arg0); double y = nvalue(args); double z = nvalue(args + 1); #if LUA_VECTOR_SIZE == 4 - double w = nvalue(args + 2); + double w = 0.0; + if (nparams >= 4) + { + if (!ttisnumber(args + 2)) + return -1; + w = nvalue(args + 2); + } setvvalue(res, float(x), float(y), float(z), float(w)); #else setvvalue(res, float(x), float(y), float(z), 0.0f); diff --git a/VM/src/ldo.cpp b/VM/src/ldo.cpp index a71fce52..0642cb6d 100644 --- a/VM/src/ldo.cpp +++ b/VM/src/ldo.cpp @@ -202,22 +202,29 @@ void luaD_growstack(lua_State* L, int n) CallInfo* luaD_growCI(lua_State* L) { - if (L->size_ci > LUAI_MAXCALLS) /* overflow while handling overflow? */ - luaD_throw(L, LUA_ERRERR); - else - { - luaD_reallocCI(L, 2 * L->size_ci); - if (L->size_ci > LUAI_MAXCALLS) - luaG_runerror(L, "stack overflow"); - } + /* allow extra stack space to handle stack overflow in xpcall */ + const int hardlimit = LUAI_MAXCALLS + (LUAI_MAXCALLS >> 3); + + if (L->size_ci >= hardlimit) + luaD_throw(L, LUA_ERRERR); /* error while handling stack error */ + + int request = L->size_ci * 2; + luaD_reallocCI(L, L->size_ci >= LUAI_MAXCALLS ? hardlimit : request < LUAI_MAXCALLS ? request : LUAI_MAXCALLS); + + if (L->size_ci > LUAI_MAXCALLS) + luaG_runerror(L, "stack overflow"); + return ++L->ci; } void luaD_checkCstack(lua_State* L) { + /* allow extra stack space to handle stack overflow in xpcall */ + const int hardlimit = LUAI_MAXCCALLS + (LUAI_MAXCCALLS >> 3); + if (L->nCcalls == LUAI_MAXCCALLS) luaG_runerror(L, "C stack overflow"); - else if (L->nCcalls >= (LUAI_MAXCCALLS + (LUAI_MAXCCALLS >> 3))) + else if (L->nCcalls >= hardlimit) luaD_throw(L, LUA_ERRERR); /* error while handling stack error */ } diff --git a/VM/src/lvmexecute.cpp b/VM/src/lvmexecute.cpp index f9fd6574..e0a96474 100644 --- a/VM/src/lvmexecute.cpp +++ b/VM/src/lvmexecute.cpp @@ -16,8 +16,6 @@ #include -LUAU_FASTFLAGVARIABLE(LuauIter, false) - // Disable c99-designator to avoid the warning in CGOTO dispatch table #ifdef __clang__ #if __has_warning("-Wc99-designator") @@ -2214,7 +2212,7 @@ static void luau_execute(lua_State* L) { /* will be called during FORGLOOP */ } - else if (FFlag::LuauIter) + else { Table* mt = ttistable(ra) ? hvalue(ra)->metatable : ttisuserdata(ra) ? uvalue(ra)->metatable : cast_to(Table*, NULL); @@ -2259,17 +2257,6 @@ static void luau_execute(lua_State* L) StkId ra = VM_REG(LUAU_INSN_A(insn)); uint32_t aux = *pc; - if (!FFlag::LuauIter) - { - bool stop; - VM_PROTECT(stop = luau_loopFORG(L, LUAU_INSN_A(insn), aux)); - - // note that we need to increment pc by 1 to exit the loop since we need to skip over aux - pc += stop ? 1 : LUAU_INSN_D(insn); - LUAU_ASSERT(unsigned(pc - cl->l.p->code) < unsigned(cl->l.p->sizecode)); - VM_NEXT(); - } - // fast-path: builtin table iteration if (ttisnil(ra) && ttistable(ra + 1) && ttislightuserdata(ra + 2)) { @@ -2362,7 +2349,7 @@ static void luau_execute(lua_State* L) /* ra+1 is already the table */ setpvalue(ra + 2, reinterpret_cast(uintptr_t(0))); } - else if (FFlag::LuauIter && !ttisfunction(ra)) + else if (!ttisfunction(ra)) { VM_PROTECT(luaG_typeerror(L, ra, "iterate over")); } @@ -2434,7 +2421,7 @@ static void luau_execute(lua_State* L) /* ra+1 is already the table */ setpvalue(ra + 2, reinterpret_cast(uintptr_t(0))); } - else if (FFlag::LuauIter && !ttisfunction(ra)) + else if (!ttisfunction(ra)) { VM_PROTECT(luaG_typeerror(L, ra, "iterate over")); } diff --git a/tests/Autocomplete.test.cpp b/tests/Autocomplete.test.cpp index cc5b31c9..dea1ab19 100644 --- a/tests/Autocomplete.test.cpp +++ b/tests/Autocomplete.test.cpp @@ -2764,8 +2764,6 @@ TEST_CASE_FIXTURE(ACBuiltinsFixture, "autocomplete_on_string_singletons") TEST_CASE_FIXTURE(ACFixture, "autocomplete_string_singletons") { - ScopedFastFlag sff{"LuauTwoPassAliasDefinitionFix", true}; - check(R"( type tag = "cat" | "dog" local function f(a: tag) end @@ -2838,8 +2836,6 @@ f(@1) TEST_CASE_FIXTURE(ACFixture, "autocomplete_string_singleton_escape") { - ScopedFastFlag sff{"LuauTwoPassAliasDefinitionFix", true}; - check(R"( type tag = "strange\t\"cat\"" | 'nice\t"dog"' local function f(x: tag) end @@ -2873,7 +2869,7 @@ local abc = b@1 TEST_CASE_FIXTURE(ACFixture, "no_incompatible_self_calls_on_class") { - ScopedFastFlag selfCallAutocompleteFix{"LuauSelfCallAutocompleteFix", true}; + ScopedFastFlag selfCallAutocompleteFix2{"LuauSelfCallAutocompleteFix2", true}; loadDefinition(R"( declare class Foo @@ -2913,7 +2909,7 @@ t.@1 TEST_CASE_FIXTURE(ACFixture, "no_incompatible_self_calls") { - ScopedFastFlag selfCallAutocompleteFix{"LuauSelfCallAutocompleteFix", true}; + ScopedFastFlag selfCallAutocompleteFix2{"LuauSelfCallAutocompleteFix2", true}; check(R"( local t = {} @@ -2929,7 +2925,7 @@ t:@1 TEST_CASE_FIXTURE(ACFixture, "no_incompatible_self_calls_2") { - ScopedFastFlag selfCallAutocompleteFix{"LuauSelfCallAutocompleteFix", true}; + ScopedFastFlag selfCallAutocompleteFix2{"LuauSelfCallAutocompleteFix2", true}; check(R"( local f: (() -> number) & ((number) -> number) = function(x: number?) return 2 end @@ -2961,7 +2957,7 @@ t:@1 TEST_CASE_FIXTURE(ACFixture, "string_prim_self_calls_are_fine") { - ScopedFastFlag selfCallAutocompleteFix{"LuauSelfCallAutocompleteFix", true}; + ScopedFastFlag selfCallAutocompleteFix2{"LuauSelfCallAutocompleteFix2", true}; check(R"( local s = "hello" @@ -2980,7 +2976,7 @@ s:@1 TEST_CASE_FIXTURE(ACFixture, "string_prim_non_self_calls_are_avoided") { - ScopedFastFlag selfCallAutocompleteFix{"LuauSelfCallAutocompleteFix", true}; + ScopedFastFlag selfCallAutocompleteFix2{"LuauSelfCallAutocompleteFix2", true}; check(R"( local s = "hello" @@ -2989,17 +2985,15 @@ s.@1 auto ac = autocomplete('1'); - REQUIRE(ac.entryMap.count("byte")); - CHECK(ac.entryMap["byte"].wrongIndexType == true); REQUIRE(ac.entryMap.count("char")); CHECK(ac.entryMap["char"].wrongIndexType == false); REQUIRE(ac.entryMap.count("sub")); CHECK(ac.entryMap["sub"].wrongIndexType == true); } -TEST_CASE_FIXTURE(ACBuiltinsFixture, "string_library_non_self_calls_are_fine") +TEST_CASE_FIXTURE(ACBuiltinsFixture, "library_non_self_calls_are_fine") { - ScopedFastFlag selfCallAutocompleteFix{"LuauSelfCallAutocompleteFix", true}; + ScopedFastFlag selfCallAutocompleteFix2{"LuauSelfCallAutocompleteFix2", true}; check(R"( string.@1 @@ -3013,11 +3007,24 @@ string.@1 CHECK(ac.entryMap["char"].wrongIndexType == false); REQUIRE(ac.entryMap.count("sub")); CHECK(ac.entryMap["sub"].wrongIndexType == false); + + check(R"( +table.@1 + )"); + + ac = autocomplete('1'); + + REQUIRE(ac.entryMap.count("remove")); + CHECK(ac.entryMap["remove"].wrongIndexType == false); + REQUIRE(ac.entryMap.count("getn")); + CHECK(ac.entryMap["getn"].wrongIndexType == false); + REQUIRE(ac.entryMap.count("insert")); + CHECK(ac.entryMap["insert"].wrongIndexType == false); } -TEST_CASE_FIXTURE(ACBuiltinsFixture, "string_library_self_calls_are_invalid") +TEST_CASE_FIXTURE(ACBuiltinsFixture, "library_self_calls_are_invalid") { - ScopedFastFlag selfCallAutocompleteFix{"LuauSelfCallAutocompleteFix", true}; + ScopedFastFlag selfCallAutocompleteFix2{"LuauSelfCallAutocompleteFix2", true}; check(R"( string:@1 diff --git a/tests/Compiler.test.cpp b/tests/Compiler.test.cpp index 20139650..6eee254e 100644 --- a/tests/Compiler.test.cpp +++ b/tests/Compiler.test.cpp @@ -261,7 +261,6 @@ L1: RETURN R0 0 TEST_CASE("ForBytecode") { - ScopedFastFlag sff("LuauCompileIter", true); ScopedFastFlag sff2("LuauCompileIterNoPairs", false); // basic for loop: variable directly refers to internal iteration index (R2) @@ -350,8 +349,6 @@ RETURN R0 0 TEST_CASE("ForBytecodeBuiltin") { - ScopedFastFlag sff("LuauCompileIter", true); - // we generally recognize builtins like pairs/ipairs and emit special opcodes CHECK_EQ("\n" + compileFunction0("for k,v in ipairs({}) do end"), R"( GETIMPORT R0 1 @@ -2323,8 +2320,6 @@ return result TEST_CASE("DebugLineInfoFor") { - ScopedFastFlag sff("LuauCompileIter", true); - Luau::BytecodeBuilder bcb; bcb.setDumpFlags(Luau::BytecodeBuilder::Dump_Code | Luau::BytecodeBuilder::Dump_Lines); Luau::compileOrThrow(bcb, R"( @@ -4355,8 +4350,6 @@ L1: RETURN R0 0 TEST_CASE("LoopUnrollControlFlow") { - ScopedFastFlag sff("LuauCompileNestedClosureO2", true); - ScopedFastInt sfis[] = { {"LuauCompileLoopUnrollThreshold", 50}, {"LuauCompileLoopUnrollThresholdMaxBoost", 300}, @@ -4475,8 +4468,6 @@ RETURN R0 0 TEST_CASE("LoopUnrollNestedClosure") { - ScopedFastFlag sff("LuauCompileNestedClosureO2", true); - // if the body has functions that refer to loop variables, we unroll the loop and use MOVE+CAPTURE for upvalues CHECK_EQ("\n" + compileFunction(R"( for i=1,2 do @@ -4756,8 +4747,6 @@ RETURN R1 1 TEST_CASE("InlineBasicProhibited") { - ScopedFastFlag sff("LuauCompileNestedClosureO2", true); - // we can't inline variadic functions CHECK_EQ("\n" + compileFunction(R"( local function foo(...) @@ -4833,8 +4822,6 @@ RETURN R1 1 TEST_CASE("InlineNestedClosures") { - ScopedFastFlag sff("LuauCompileNestedClosureO2", true); - // we can inline functions that contain/return functions CHECK_EQ("\n" + compileFunction(R"( local function foo(x) diff --git a/tests/Conformance.test.cpp b/tests/Conformance.test.cpp index f7f2b4ac..96a2775f 100644 --- a/tests/Conformance.test.cpp +++ b/tests/Conformance.test.cpp @@ -741,7 +741,7 @@ TEST_CASE("ApiTables") lua_pop(L, 1); } -TEST_CASE("ApiFunctionCalls") +TEST_CASE("ApiCalls") { StateRef globalState = runConformance("apicalls.lua"); lua_State* L = globalState.get(); @@ -790,6 +790,58 @@ TEST_CASE("ApiFunctionCalls") CHECK(lua_equal(L2, -1, -2) == 1); lua_pop(L2, 2); } + + // lua_clonefunction + fenv + { + lua_getfield(L, LUA_GLOBALSINDEX, "getpi"); + lua_call(L, 0, 1); + CHECK(lua_tonumber(L, -1) == 3.1415926); + lua_pop(L, 1); + + lua_getfield(L, LUA_GLOBALSINDEX, "getpi"); + + // clone & override env + lua_clonefunction(L, -1); + lua_newtable(L); + lua_pushnumber(L, 42); + lua_setfield(L, -2, "pi"); + lua_setfenv(L, -2); + + lua_call(L, 0, 1); + CHECK(lua_tonumber(L, -1) == 42); + lua_pop(L, 1); + + // this one calls original function again + lua_call(L, 0, 1); + CHECK(lua_tonumber(L, -1) == 3.1415926); + lua_pop(L, 1); + } + + // lua_clonefunction + upvalues + { + lua_getfield(L, LUA_GLOBALSINDEX, "incuv"); + lua_call(L, 0, 1); + CHECK(lua_tonumber(L, -1) == 1); + lua_pop(L, 1); + + lua_getfield(L, LUA_GLOBALSINDEX, "incuv"); + // two clones + lua_clonefunction(L, -1); + lua_clonefunction(L, -2); + + lua_call(L, 0, 1); + CHECK(lua_tonumber(L, -1) == 2); + lua_pop(L, 1); + + lua_call(L, 0, 1); + CHECK(lua_tonumber(L, -1) == 3); + lua_pop(L, 1); + + // this one calls original function again + lua_call(L, 0, 1); + CHECK(lua_tonumber(L, -1) == 4); + lua_pop(L, 1); + } } static bool endsWith(const std::string& str, const std::string& suffix) @@ -1113,11 +1165,6 @@ TEST_CASE("UserdataApi") TEST_CASE("Iter") { - ScopedFastFlag sffs[] = { - {"LuauCompileIter", true}, - {"LuauIter", true}, - }; - runConformance("iter.lua"); } diff --git a/tests/Module.test.cpp b/tests/Module.test.cpp index c7e18efd..89b13ab1 100644 --- a/tests/Module.test.cpp +++ b/tests/Module.test.cpp @@ -300,4 +300,24 @@ TEST_CASE_FIXTURE(Fixture, "clone_recursion_limit") CHECK_THROWS_AS(clone(table, dest, cloneState), RecursionLimitException); } +TEST_CASE_FIXTURE(Fixture, "any_persistance_does_not_leak") +{ + ScopedFastFlag luauNonCopyableTypeVarFields{"LuauNonCopyableTypeVarFields", true}; + + fileResolver.source["Module/A"] = R"( +export type A = B +type B = A + )"; + + FrontendOptions opts; + opts.retainFullTypeGraphs = false; + CheckResult result = frontend.check("Module/A", opts); + LUAU_REQUIRE_ERRORS(result); + + auto mod = frontend.moduleResolver.getModule("Module/A"); + auto it = mod->getModuleScope()->exportedTypeBindings.find("A"); + REQUIRE(it != mod->getModuleScope()->exportedTypeBindings.end()); + CHECK(toString(it->second.type) == "any"); +} + TEST_SUITE_END(); diff --git a/tests/Parser.test.cpp b/tests/Parser.test.cpp index 87b1263f..878023e3 100644 --- a/tests/Parser.test.cpp +++ b/tests/Parser.test.cpp @@ -2622,6 +2622,23 @@ type Z = { a: string | T..., b: number } REQUIRE_EQ(3, result.errors.size()); } +TEST_CASE_FIXTURE(Fixture, "recover_function_return_type_annotations") +{ + ScopedFastFlag sff{"LuauReturnTypeTokenConfusion", true}; + ParseResult result = tryParse(R"( +type Custom = { x: A, y: B, z: C } +type Packed = { x: (A...) -> () } +type F = (number): Custom +type G = Packed<(number): (string, number, boolean)> +local function f(x: number) -> Custom +end + )"); + REQUIRE_EQ(3, result.errors.size()); + CHECK_EQ(result.errors[0].getMessage(), "Return types in function type annotations are written after '->' instead of ':'"); + CHECK_EQ(result.errors[1].getMessage(), "Return types in function type annotations are written after '->' instead of ':'"); + CHECK_EQ(result.errors[2].getMessage(), "Function return type annotations are written after ':' instead of '->'"); +} + TEST_CASE_FIXTURE(Fixture, "error_message_for_using_function_as_type_annotation") { ScopedFastFlag sff{"LuauParserFunctionKeywordAsTypeHelp", true}; diff --git a/tests/TypeInfer.aliases.test.cpp b/tests/TypeInfer.aliases.test.cpp index 7562a4d7..86cc9701 100644 --- a/tests/TypeInfer.aliases.test.cpp +++ b/tests/TypeInfer.aliases.test.cpp @@ -615,8 +615,6 @@ TEST_CASE_FIXTURE(Fixture, "generic_typevars_are_not_considered_to_escape_their_ */ TEST_CASE_FIXTURE(Fixture, "forward_declared_alias_is_not_clobbered_by_prior_unification_with_any") { - ScopedFastFlag sff[] = {{"LuauTwoPassAliasDefinitionFix", true}}; - CheckResult result = check(R"( local function x() local y: FutureType = {}::any @@ -633,10 +631,6 @@ TEST_CASE_FIXTURE(Fixture, "forward_declared_alias_is_not_clobbered_by_prior_uni TEST_CASE_FIXTURE(Fixture, "forward_declared_alias_is_not_clobbered_by_prior_unification_with_any_2") { - ScopedFastFlag sff[] = { - {"LuauTwoPassAliasDefinitionFix", true}, - }; - CheckResult result = check(R"( local B = {} B.bar = 4 diff --git a/tests/TypeInfer.loops.test.cpp b/tests/TypeInfer.loops.test.cpp index 4444cd66..1c6fe1d8 100644 --- a/tests/TypeInfer.loops.test.cpp +++ b/tests/TypeInfer.loops.test.cpp @@ -486,8 +486,6 @@ TEST_CASE_FIXTURE(Fixture, "fuzz_fail_missing_instantitation_follow") TEST_CASE_FIXTURE(Fixture, "loop_iter_basic") { - ScopedFastFlag sff{"LuauTypecheckIter", true}; - CheckResult result = check(R"( local t: {string} = {} local key @@ -506,8 +504,6 @@ TEST_CASE_FIXTURE(Fixture, "loop_iter_basic") TEST_CASE_FIXTURE(Fixture, "loop_iter_trailing_nil") { - ScopedFastFlag sff{"LuauTypecheckIter", true}; - CheckResult result = check(R"( local t: {string} = {} local extra @@ -522,8 +518,6 @@ TEST_CASE_FIXTURE(Fixture, "loop_iter_trailing_nil") TEST_CASE_FIXTURE(Fixture, "loop_iter_no_indexer") { - ScopedFastFlag sff{"LuauTypecheckIter", true}; - CheckResult result = check(R"( local t = {} for k, v in t do @@ -539,8 +533,6 @@ TEST_CASE_FIXTURE(Fixture, "loop_iter_no_indexer") TEST_CASE_FIXTURE(BuiltinsFixture, "loop_iter_iter_metamethod") { - ScopedFastFlag sff{"LuauTypecheckIter", true}; - CheckResult result = check(R"( local t = {} setmetatable(t, { __iter = function(o) return next, o.children end }) diff --git a/tests/TypeInfer.refinements.test.cpp b/tests/TypeInfer.refinements.test.cpp index 6785f277..207b3cff 100644 --- a/tests/TypeInfer.refinements.test.cpp +++ b/tests/TypeInfer.refinements.test.cpp @@ -932,6 +932,8 @@ TEST_CASE_FIXTURE(Fixture, "apply_refinements_on_astexprindexexpr_whose_subscrip TEST_CASE_FIXTURE(Fixture, "discriminate_from_truthiness_of_x") { + ScopedFastFlag sff{"LuauFalsyPredicateReturnsNilInstead", true}; + CheckResult result = check(R"( type T = {tag: "missing", x: nil} | {tag: "exists", x: string} @@ -947,7 +949,7 @@ TEST_CASE_FIXTURE(Fixture, "discriminate_from_truthiness_of_x") LUAU_REQUIRE_NO_ERRORS(result); CHECK_EQ(R"({| tag: "exists", x: string |})", toString(requireTypeAtPosition({5, 28}))); - CHECK_EQ(R"({| tag: "missing", x: nil |})", toString(requireTypeAtPosition({7, 28}))); + CHECK_EQ(R"({| tag: "exists", x: string |} | {| tag: "missing", x: nil |})", toString(requireTypeAtPosition({7, 28}))); } TEST_CASE_FIXTURE(Fixture, "discriminate_tag") @@ -1191,7 +1193,7 @@ TEST_CASE_FIXTURE(RefinementClassFixture, "x_is_not_instance_or_else_not_part") TEST_CASE_FIXTURE(Fixture, "typeguard_doesnt_leak_to_elseif") { - const std::string code = R"( + CheckResult result = check(R"( function f(a) if type(a) == "boolean" then local a1 = a @@ -1201,10 +1203,30 @@ TEST_CASE_FIXTURE(Fixture, "typeguard_doesnt_leak_to_elseif") local a3 = a end end - )"; - CheckResult result = check(code); + )"); + LUAU_REQUIRE_NO_ERRORS(result); - dumpErrors(result); +} + +TEST_CASE_FIXTURE(BuiltinsFixture, "falsiness_of_TruthyPredicate_narrows_into_nil") +{ + ScopedFastFlag sff{"LuauFalsyPredicateReturnsNilInstead", true}; + + CheckResult result = check(R"( + local function f(t: {number}) + local x = t[1] + if not x then + local foo = x + else + local bar = x + end + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ("nil", toString(requireTypeAtPosition({4, 28}))); + CHECK_EQ("number", toString(requireTypeAtPosition({6, 28}))); } TEST_SUITE_END(); diff --git a/tests/TypeInfer.singletons.test.cpp b/tests/TypeInfer.singletons.test.cpp index 14a5a6ae..a90f434f 100644 --- a/tests/TypeInfer.singletons.test.cpp +++ b/tests/TypeInfer.singletons.test.cpp @@ -139,8 +139,6 @@ TEST_CASE_FIXTURE(Fixture, "enums_using_singletons") TEST_CASE_FIXTURE(Fixture, "enums_using_singletons_mismatch") { - ScopedFastFlag sff{"LuauTwoPassAliasDefinitionFix", true}; - CheckResult result = check(R"( type MyEnum = "foo" | "bar" | "baz" local a : MyEnum = "bang" diff --git a/tests/TypePack.test.cpp b/tests/TypePack.test.cpp index c4931578..8a5a65fe 100644 --- a/tests/TypePack.test.cpp +++ b/tests/TypePack.test.cpp @@ -197,4 +197,20 @@ TEST_CASE_FIXTURE(TypePackFixture, "std_distance") CHECK_EQ(4, std::distance(b, e)); } +TEST_CASE("content_reassignment") +{ + ScopedFastFlag luauNonCopyableTypeVarFields{"LuauNonCopyableTypeVarFields", true}; + + TypePackVar myError{Unifiable::Error{}, /*presistent*/ true}; + + TypeArena arena; + + TypePackId futureError = arena.addTypePack(TypePackVar{FreeTypePack{TypeLevel{}}}); + asMutable(futureError)->reassign(myError); + + CHECK(get(futureError) != nullptr); + CHECK(!futureError->persistent); + CHECK(futureError->owningArena == &arena); +} + TEST_SUITE_END(); diff --git a/tests/TypeVar.test.cpp b/tests/TypeVar.test.cpp index bb2d94ba..4f8fc502 100644 --- a/tests/TypeVar.test.cpp +++ b/tests/TypeVar.test.cpp @@ -416,4 +416,24 @@ TEST_CASE("proof_that_isBoolean_uses_all_of") CHECK(!isBoolean(&union_)); } +TEST_CASE("content_reassignment") +{ + ScopedFastFlag luauNonCopyableTypeVarFields{"LuauNonCopyableTypeVarFields", true}; + + TypeVar myAny{AnyTypeVar{}, /*presistent*/ true}; + myAny.normal = true; + myAny.documentationSymbol = "@global/any"; + + TypeArena arena; + + TypeId futureAny = arena.addType(FreeTypeVar{TypeLevel{}}); + asMutable(futureAny)->reassign(myAny); + + CHECK(get(futureAny) != nullptr); + CHECK(!futureAny->persistent); + CHECK(futureAny->normal); + CHECK(futureAny->documentationSymbol == "@global/any"); + CHECK(futureAny->owningArena == &arena); +} + TEST_SUITE_END(); diff --git a/tests/conformance/apicalls.lua b/tests/conformance/apicalls.lua index 7a4058b5..27416623 100644 --- a/tests/conformance/apicalls.lua +++ b/tests/conformance/apicalls.lua @@ -11,4 +11,15 @@ function create_with_tm(x) return setmetatable({ a = x }, m) end +local gen = 0 +function incuv() + gen += 1 + return gen +end + +pi = 3.1415926 +function getpi() + return pi +end + return('OK') diff --git a/tests/conformance/pcall.lua b/tests/conformance/pcall.lua index 84ac2ba1..969209fc 100644 --- a/tests/conformance/pcall.lua +++ b/tests/conformance/pcall.lua @@ -144,4 +144,21 @@ coroutine.resume(co) resumeerror(co, "fail") checkresults({ true, false, "fail" }, coroutine.resume(co)) +-- stack overflow needs to happen at the call limit +local calllimit = 20000 +function recurse(n) return n <= 1 and 1 or recurse(n-1) + 1 end + +-- we use one frame for top-level function and one frame is the service frame for coroutines +assert(recurse(calllimit - 2) == calllimit - 2) + +-- note that when calling through pcall, pcall eats one more frame +checkresults({ true, calllimit - 3 }, pcall(recurse, calllimit - 3)) +checkerror(pcall(recurse, calllimit - 2)) + +-- xpcall handler runs in context of the stack frame, but this works just fine since we allow extra stack consumption past stack overflow +checkresults({ false, "ok" }, xpcall(recurse, function() return string.reverse("ko") end, calllimit - 2)) + +-- however, if xpcall handler itself runs out of extra stack space, we get "error in error handling" +checkresults({ false, "error in error handling" }, xpcall(recurse, function() return recurse(calllimit) end, calllimit - 2)) + return 'OK' From 88b3984711dff98bf51f1a152dcee874f79bd4a0 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Thu, 16 Jun 2022 17:54:42 -0700 Subject: [PATCH 276/399] Sync to upstream/release/532 --- Analysis/include/Luau/Constraint.h | 82 ++++ .../include/Luau/ConstraintGraphBuilder.h | 113 ++--- Analysis/include/Luau/ConstraintSolver.h | 103 +++-- .../include/Luau/ConstraintSolverLogger.h | 26 ++ Analysis/include/Luau/Frontend.h | 4 +- Analysis/include/Luau/Module.h | 1 + Analysis/include/Luau/Normalize.h | 1 + Analysis/include/Luau/NotNull.h | 41 +- Analysis/include/Luau/Quantify.h | 3 +- Analysis/include/Luau/RequireTracer.h | 2 +- Analysis/include/Luau/TypeChecker2.h | 13 + Analysis/include/Luau/TypeInfer.h | 41 +- Analysis/include/Luau/TypeVar.h | 37 +- Analysis/include/Luau/Unifier.h | 2 +- Analysis/include/Luau/VisitTypeVar.h | 2 +- Analysis/src/Autocomplete.cpp | 19 +- Analysis/src/BuiltinDefinitions.cpp | 68 +-- Analysis/src/Clone.cpp | 10 +- Analysis/src/Constraint.cpp | 14 + Analysis/src/ConstraintGraphBuilder.cpp | 406 +++++++++++++++--- Analysis/src/ConstraintSolver.cpp | 174 +++++--- Analysis/src/ConstraintSolverLogger.cpp | 139 ++++++ Analysis/src/Frontend.cpp | 34 +- Analysis/src/Instantiation.cpp | 2 +- Analysis/src/Linter.cpp | 4 +- Analysis/src/Normalize.cpp | 46 +- Analysis/src/Quantify.cpp | 153 ++++++- Analysis/src/RequireTracer.cpp | 14 +- Analysis/src/Substitution.cpp | 4 +- Analysis/src/ToDot.cpp | 2 +- Analysis/src/ToString.cpp | 94 +++- Analysis/src/TypeAttach.cpp | 7 +- Analysis/src/TypeChecker2.cpp | 160 +++++++ Analysis/src/TypeInfer.cpp | 253 ++++++----- Analysis/src/TypeUtils.cpp | 2 +- Analysis/src/TypeVar.cpp | 41 +- Analysis/src/Unifier.cpp | 13 +- Compiler/src/BytecodeBuilder.cpp | 14 + Compiler/src/Compiler.cpp | 76 +++- Sources.cmake | 8 +- VM/src/lobject.h | 2 +- VM/src/ltable.cpp | 8 +- VM/src/ltm.cpp | 4 +- VM/src/ltm.h | 4 +- tests/Autocomplete.test.cpp | 2 +- tests/Compiler.test.cpp | 252 ++++++++++- tests/ConstraintGraphBuilder.test.cpp | 61 ++- tests/Fixture.cpp | 3 +- tests/Frontend.test.cpp | 8 +- tests/Module.test.cpp | 2 +- tests/NonstrictMode.test.cpp | 53 ++- tests/Normalize.test.cpp | 42 +- tests/NotNull.test.cpp | 53 ++- tests/ToString.test.cpp | 36 ++ tests/TypeInfer.annotations.test.cpp | 2 +- tests/TypeInfer.functions.test.cpp | 60 ++- tests/TypeInfer.generics.test.cpp | 6 +- tests/TypeInfer.operators.test.cpp | 4 +- tests/TypeInfer.provisional.test.cpp | 5 +- tests/TypeInfer.refinements.test.cpp | 6 +- tests/TypeInfer.tables.test.cpp | 11 +- tests/TypeInfer.test.cpp | 89 +--- tests/TypeInfer.typePacks.cpp | 14 +- tools/natvis/Analysis.natvis | 66 +-- 64 files changed, 2276 insertions(+), 745 deletions(-) create mode 100644 Analysis/include/Luau/Constraint.h create mode 100644 Analysis/include/Luau/ConstraintSolverLogger.h create mode 100644 Analysis/include/Luau/TypeChecker2.h create mode 100644 Analysis/src/Constraint.cpp create mode 100644 Analysis/src/ConstraintSolverLogger.cpp create mode 100644 Analysis/src/TypeChecker2.cpp diff --git a/Analysis/include/Luau/Constraint.h b/Analysis/include/Luau/Constraint.h new file mode 100644 index 00000000..c62166e2 --- /dev/null +++ b/Analysis/include/Luau/Constraint.h @@ -0,0 +1,82 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +#pragma once + +#include "Luau/Location.h" +#include "Luau/NotNull.h" +#include "Luau/Variant.h" + +#include +#include + +namespace Luau +{ + +struct Scope2; +struct TypeVar; +using TypeId = const TypeVar*; + +struct TypePackVar; +using TypePackId = const TypePackVar*; + +// subType <: superType +struct SubtypeConstraint +{ + TypeId subType; + TypeId superType; +}; + +// subPack <: superPack +struct PackSubtypeConstraint +{ + TypePackId subPack; + TypePackId superPack; +}; + +// subType ~ gen superType +struct GeneralizationConstraint +{ + TypeId generalizedType; + TypeId sourceType; + Scope2* scope; +}; + +// subType ~ inst superType +struct InstantiationConstraint +{ + TypeId subType; + TypeId superType; +}; + +using ConstraintV = Variant; +using ConstraintPtr = std::unique_ptr; + +struct Constraint +{ + Constraint(ConstraintV&& c, Location location); + + Constraint(const Constraint&) = delete; + Constraint& operator=(const Constraint&) = delete; + + ConstraintV c; + Location location; + std::vector> dependencies; +}; + +inline Constraint& asMutable(const Constraint& c) +{ + return const_cast(c); +} + +template +T* getMutable(Constraint& c) +{ + return ::Luau::get_if(&c.c); +} + +template +const T* get(const Constraint& c) +{ + return getMutable(asMutable(c)); +} + +} // namespace Luau diff --git a/Analysis/include/Luau/ConstraintGraphBuilder.h b/Analysis/include/Luau/ConstraintGraphBuilder.h index 4234f2f6..da774a2a 100644 --- a/Analysis/include/Luau/ConstraintGraphBuilder.h +++ b/Analysis/include/Luau/ConstraintGraphBuilder.h @@ -4,9 +4,12 @@ #include #include +#include #include "Luau/Ast.h" +#include "Luau/Constraint.h" #include "Luau/Module.h" +#include "Luau/NotNull.h" #include "Luau/Symbol.h" #include "Luau/TypeVar.h" #include "Luau/Variant.h" @@ -14,69 +17,6 @@ namespace Luau { -struct Scope2; - -// subType <: superType -struct SubtypeConstraint -{ - TypeId subType; - TypeId superType; -}; - -// subPack <: superPack -struct PackSubtypeConstraint -{ - TypePackId subPack; - TypePackId superPack; -}; - -// subType ~ gen superType -struct GeneralizationConstraint -{ - TypeId subType; - TypeId superType; - Scope2* scope; -}; - -// subType ~ inst superType -struct InstantiationConstraint -{ - TypeId subType; - TypeId superType; -}; - -using ConstraintV = Variant; -using ConstraintPtr = std::unique_ptr; - -struct Constraint -{ - Constraint(ConstraintV&& c); - Constraint(ConstraintV&& c, std::vector dependencies); - - Constraint(const Constraint&) = delete; - Constraint& operator=(const Constraint&) = delete; - - ConstraintV c; - std::vector dependencies; -}; - -inline Constraint& asMutable(const Constraint& c) -{ - return const_cast(c); -} - -template -T* getMutable(Constraint& c) -{ - return ::Luau::get_if(&c.c); -} - -template -const T* get(const Constraint& c) -{ - return getMutable(asMutable(c)); -} - struct Scope2 { // The parent scope of this scope. Null if there is no parent (i.e. this @@ -102,6 +42,11 @@ struct ConstraintGraphBuilder TypeArena* const arena; // The root scope of the module we're generating constraints for. Scope2* rootScope; + // A mapping of AST node to TypeId. + DenseHashMap astTypes{nullptr}; + // A mapping of AST node to TypePackId. + DenseHashMap astTypePacks{nullptr}; + DenseHashMap astOriginalCallTypes{nullptr}; explicit ConstraintGraphBuilder(TypeArena* arena); @@ -128,8 +73,9 @@ struct ConstraintGraphBuilder * Adds a new constraint with no dependencies to a given scope. * @param scope the scope to add the constraint to. Must not be null. * @param cv the constraint variant to add. + * @param location the location to attribute to the constraint. */ - void addConstraint(Scope2* scope, ConstraintV cv); + void addConstraint(Scope2* scope, ConstraintV cv, Location location); /** * Adds a constraint to a given scope. @@ -148,15 +94,48 @@ struct ConstraintGraphBuilder void visit(Scope2* scope, AstStat* stat); void visit(Scope2* scope, AstStatBlock* block); void visit(Scope2* scope, AstStatLocal* local); - void visit(Scope2* scope, AstStatLocalFunction* local); - void visit(Scope2* scope, AstStatReturn* local); + void visit(Scope2* scope, AstStatLocalFunction* function); + void visit(Scope2* scope, AstStatFunction* function); + void visit(Scope2* scope, AstStatReturn* ret); + void visit(Scope2* scope, AstStatAssign* assign); + void visit(Scope2* scope, AstStatIf* ifStatement); + + TypePackId checkExprList(Scope2* scope, const AstArray& exprs); TypePackId checkPack(Scope2* scope, AstArray exprs); TypePackId checkPack(Scope2* scope, AstExpr* expr); + /** + * Checks an expression that is expected to evaluate to one type. + * @param scope the scope the expression is contained within. + * @param expr the expression to check. + * @return the type of the expression. + */ TypeId check(Scope2* scope, AstExpr* expr); + + TypeId checkExprTable(Scope2* scope, AstExprTable* expr); + TypeId check(Scope2* scope, AstExprIndexName* indexName); + + std::pair checkFunctionSignature(Scope2* parent, AstExprFunction* fn); + + /** + * Checks the body of a function expression. + * @param scope the interior scope of the body of the function. + * @param fn the function expression to check. + */ + void checkFunctionBody(Scope2* scope, AstExprFunction* fn); }; -std::vector collectConstraints(Scope2* rootScope); +/** + * Collects a vector of borrowed constraints from the scope and all its child + * scopes. It is important to only call this function when you're done adding + * constraints to the scope or its descendants, lest the borrowed pointers + * become invalid due to a container reallocation. + * @param rootScope the root scope of the scope graph to collect constraints + * from. + * @return a list of pointers to constraints contained within the scope graph. + * None of these pointers should be null. + */ +std::vector> collectConstraints(Scope2* rootScope); } // namespace Luau diff --git a/Analysis/include/Luau/ConstraintSolver.h b/Analysis/include/Luau/ConstraintSolver.h index 85006e68..7e6d4461 100644 --- a/Analysis/include/Luau/ConstraintSolver.h +++ b/Analysis/include/Luau/ConstraintSolver.h @@ -4,7 +4,8 @@ #include "Luau/Error.h" #include "Luau/Variant.h" -#include "Luau/ConstraintGraphBuilder.h" +#include "Luau/Constraint.h" +#include "Luau/ConstraintSolverLogger.h" #include "Luau/TypeVar.h" #include @@ -20,39 +21,81 @@ struct ConstraintSolver { TypeArena* arena; InternalErrorReporter iceReporter; - // The entire set of constraints that the solver is trying to resolve. - std::vector constraints; + // The entire set of constraints that the solver is trying to resolve. It + // is important to not add elements to this vector, lest the underlying + // storage that we retain pointers to be mutated underneath us. + const std::vector> constraints; Scope2* rootScope; - std::vector errors; // This includes every constraint that has not been fully solved. // A constraint can be both blocked and unsolved, for instance. - std::unordered_set unsolvedConstraints; + std::vector> unsolvedConstraints; // A mapping of constraint pointer to how many things the constraint is // blocked on. Can be empty or 0 for constraints that are not blocked on // anything. - std::unordered_map blockedConstraints; + std::unordered_map, size_t> blockedConstraints; // A mapping of type/pack pointers to the constraints they block. - std::unordered_map> blocked; + std::unordered_map>> blocked; + + ConstraintSolverLogger logger; explicit ConstraintSolver(TypeArena* arena, Scope2* rootScope); /** * Attempts to dispatch all pending constraints and reach a type solution - * that satisfies all of the constraints, recording any errors that are - * encountered. + * that satisfies all of the constraints. **/ void run(); bool done(); - bool tryDispatch(const Constraint* c); - bool tryDispatch(const SubtypeConstraint& c); - bool tryDispatch(const PackSubtypeConstraint& c); - bool tryDispatch(const GeneralizationConstraint& c); - bool tryDispatch(const InstantiationConstraint& c, const Constraint* constraint); + bool tryDispatch(NotNull c, bool force); + bool tryDispatch(const SubtypeConstraint& c, NotNull constraint, bool force); + bool tryDispatch(const PackSubtypeConstraint& c, NotNull constraint, bool force); + bool tryDispatch(const GeneralizationConstraint& c, NotNull constraint, bool force); + bool tryDispatch(const InstantiationConstraint& c, NotNull constraint, bool force); + void block(NotNull target, NotNull constraint); + /** + * Block a constraint on the resolution of a TypeVar. + * @returns false always. This is just to allow tryDispatch to return the result of block() + */ + bool block(TypeId target, NotNull constraint); + bool block(TypePackId target, NotNull constraint); + + void unblock(NotNull progressed); + void unblock(TypeId progressed); + void unblock(TypePackId progressed); + + /** + * @returns true if the TypeId is in a blocked state. + */ + bool isBlocked(TypeId ty); + + /** + * Returns whether the constraint is blocked on anything. + * @param constraint the constraint to check. + */ + bool isBlocked(NotNull constraint); + + /** + * Creates a new Unifier and performs a single unification operation. Commits + * the result. + * @param subType the sub-type to unify. + * @param superType the super-type to unify. + */ + void unify(TypeId subType, TypeId superType, Location location); + + /** + * Creates a new Unifier and performs a single unification operation. Commits + * the result. + * @param subPack the sub-type pack to unify. + * @param superPack the super-type pack to unify. + */ + void unify(TypePackId subPack, TypePackId superPack, Location location); + +private: /** * Marks a constraint as being blocked on a type or type pack. The constraint * solver will not attempt to dispatch blocked constraints until their @@ -60,10 +103,7 @@ struct ConstraintSolver * @param target the type or type pack pointer that the constraint is blocked on. * @param constraint the constraint to block. **/ - void block_(BlockedConstraintId target, const Constraint* constraint); - void block(const Constraint* target, const Constraint* constraint); - void block(TypeId target, const Constraint* constraint); - void block(TypePackId target, const Constraint* constraint); + void block_(BlockedConstraintId target, NotNull constraint); /** * Informs the solver that progress has been made on a type or type pack. The @@ -72,33 +112,6 @@ struct ConstraintSolver * @param progressed the type or type pack pointer that has progressed. **/ void unblock_(BlockedConstraintId progressed); - void unblock(const Constraint* progressed); - void unblock(TypeId progressed); - void unblock(TypePackId progressed); - - /** - * Returns whether the constraint is blocked on anything. - * @param constraint the constraint to check. - */ - bool isBlocked(const Constraint* constraint); - - void reportErrors(const std::vector& errors); - - /** - * Creates a new Unifier and performs a single unification operation. Commits - * the result and reports errors if necessary. - * @param subType the sub-type to unify. - * @param superType the super-type to unify. - */ - void unify(TypeId subType, TypeId superType); - - /** - * Creates a new Unifier and performs a single unification operation. Commits - * the result and reports errors if necessary. - * @param subPack the sub-type pack to unify. - * @param superPack the super-type pack to unify. - */ - void unify(TypePackId subPack, TypePackId superPack); }; void dump(Scope2* rootScope, struct ToStringOptions& opts); diff --git a/Analysis/include/Luau/ConstraintSolverLogger.h b/Analysis/include/Luau/ConstraintSolverLogger.h new file mode 100644 index 00000000..2b195d71 --- /dev/null +++ b/Analysis/include/Luau/ConstraintSolverLogger.h @@ -0,0 +1,26 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details + +#include "Luau/ConstraintGraphBuilder.h" +#include "Luau/ToString.h" + +#include +#include +#include + +namespace Luau +{ + +struct ConstraintSolverLogger +{ + std::string compileOutput(); + void captureBoundarySnapshot(const Scope2* rootScope, std::vector>& unsolvedConstraints); + void prepareStepSnapshot(const Scope2* rootScope, NotNull current, std::vector>& unsolvedConstraints); + void commitPreparedStepSnapshot(); + +private: + std::vector snapshots; + std::optional preparedSnapshot; + ToStringOptions opts; +}; + +} // namespace Luau diff --git a/Analysis/include/Luau/Frontend.h b/Analysis/include/Luau/Frontend.h index 58be0ffe..f4226cc1 100644 --- a/Analysis/include/Luau/Frontend.h +++ b/Analysis/include/Luau/Frontend.h @@ -66,7 +66,7 @@ struct SourceNode } ModuleName name; - std::unordered_set requires; + std::unordered_set requireSet; std::vector> requireLocations; bool dirtySourceModule = true; bool dirtyModule = true; @@ -186,7 +186,7 @@ public: std::unordered_map sourceNodes; std::unordered_map sourceModules; - std::unordered_map requires; + std::unordered_map requireTrace; Stats stats = {}; }; diff --git a/Analysis/include/Luau/Module.h b/Analysis/include/Luau/Module.h index f6e077dc..e979b3f0 100644 --- a/Analysis/include/Luau/Module.h +++ b/Analysis/include/Luau/Module.h @@ -69,6 +69,7 @@ struct Module std::vector>> scope2s; // never empty DenseHashMap astTypes{nullptr}; + DenseHashMap astTypePacks{nullptr}; DenseHashMap astExpectedTypes{nullptr}; DenseHashMap astOriginalCallTypes{nullptr}; DenseHashMap astOverloadResolvedTypes{nullptr}; diff --git a/Analysis/include/Luau/Normalize.h b/Analysis/include/Luau/Normalize.h index 262b54b2..d4c7698b 100644 --- a/Analysis/include/Luau/Normalize.h +++ b/Analysis/include/Luau/Normalize.h @@ -10,6 +10,7 @@ namespace Luau struct InternalErrorReporter; bool isSubtype(TypeId superTy, TypeId subTy, InternalErrorReporter& ice); +bool isSubtype(TypePackId superTy, TypePackId subTy, InternalErrorReporter& ice); std::pair normalize(TypeId ty, TypeArena& arena, InternalErrorReporter& ice); std::pair normalize(TypeId ty, const ModulePtr& module, InternalErrorReporter& ice); diff --git a/Analysis/include/Luau/NotNull.h b/Analysis/include/Luau/NotNull.h index 3d05fdea..f6043e9c 100644 --- a/Analysis/include/Luau/NotNull.h +++ b/Analysis/include/Luau/NotNull.h @@ -9,20 +9,22 @@ namespace Luau { /** A non-owning, non-null pointer to a T. - * - * A NotNull is notionally identical to a T* with the added restriction that it - * can never store nullptr. - * - * The sole conversion rule from T* to NotNull is the single-argument constructor, which - * is intentionally marked explicit. This constructor performs a runtime test to verify - * that the passed pointer is never nullptr. - * - * Pointer arithmetic, increment, decrement, and array indexing are all forbidden. - * - * An implicit coersion from NotNull to T* is afforded, as are the pointer indirection and member - * access operators. (*p and p->prop) * - * The explicit delete statement is permitted on a NotNull through this implicit conversion. + * A NotNull is notionally identical to a T* with the added restriction that + * it can never store nullptr. + * + * The sole conversion rule from T* to NotNull is the single-argument + * constructor, which is intentionally marked explicit. This constructor + * performs a runtime test to verify that the passed pointer is never nullptr. + * + * Pointer arithmetic, increment, decrement, and array indexing are all + * forbidden. + * + * An implicit coersion from NotNull to T* is afforded, as are the pointer + * indirection and member access operators. (*p and p->prop) + * + * The explicit delete statement is permitted (but not recommended) on a + * NotNull through this implicit conversion. */ template struct NotNull @@ -36,6 +38,11 @@ struct NotNull explicit NotNull(std::nullptr_t) = delete; void operator=(std::nullptr_t) = delete; + template + NotNull(NotNull other) + : ptr(other.get()) + {} + operator T*() const noexcept { return ptr; @@ -56,6 +63,12 @@ struct NotNull T& operator+(int) = delete; T& operator-(int) = delete; + T* get() const noexcept + { + return ptr; + } + +private: T* ptr; }; @@ -68,7 +81,7 @@ template struct hash> { size_t operator()(const Luau::NotNull& p) const { - return std::hash()(p.ptr); + return std::hash()(p.get()); } }; diff --git a/Analysis/include/Luau/Quantify.h b/Analysis/include/Luau/Quantify.h index b32d684e..f46f0cb5 100644 --- a/Analysis/include/Luau/Quantify.h +++ b/Analysis/include/Luau/Quantify.h @@ -6,9 +6,10 @@ namespace Luau { +struct TypeArena; struct Scope2; void quantify(TypeId ty, TypeLevel level); -void quantify(TypeId ty, Scope2* scope); +TypeId quantify(TypeArena* arena, TypeId ty, Scope2* scope); } // namespace Luau diff --git a/Analysis/include/Luau/RequireTracer.h b/Analysis/include/Luau/RequireTracer.h index c25545f5..f69d133e 100644 --- a/Analysis/include/Luau/RequireTracer.h +++ b/Analysis/include/Luau/RequireTracer.h @@ -19,7 +19,7 @@ struct RequireTraceResult { DenseHashMap exprs{nullptr}; - std::vector> requires; + std::vector> requireList; }; RequireTraceResult traceRequires(FileResolver* fileResolver, AstStatBlock* root, const ModuleName& currentModuleName); diff --git a/Analysis/include/Luau/TypeChecker2.h b/Analysis/include/Luau/TypeChecker2.h new file mode 100644 index 00000000..a6c7a3e3 --- /dev/null +++ b/Analysis/include/Luau/TypeChecker2.h @@ -0,0 +1,13 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details + +#pragma once + +#include "Luau/Ast.h" +#include "Luau/Module.h" + +namespace Luau +{ + +void check(const SourceModule& sourceModule, Module* module); + +} // namespace Luau diff --git a/Analysis/include/Luau/TypeInfer.h b/Analysis/include/Luau/TypeInfer.h index 183cc053..28adc9d9 100644 --- a/Analysis/include/Luau/TypeInfer.h +++ b/Analysis/include/Luau/TypeInfer.h @@ -138,25 +138,25 @@ struct TypeChecker void checkBlockWithoutRecursionCheck(const ScopePtr& scope, const AstStatBlock& statement); void checkBlockTypeAliases(const ScopePtr& scope, std::vector& sorted); - ExprResult checkExpr( + WithPredicate checkExpr( const ScopePtr& scope, const AstExpr& expr, std::optional expectedType = std::nullopt, bool forceSingleton = false); - ExprResult checkExpr(const ScopePtr& scope, const AstExprLocal& expr); - ExprResult checkExpr(const ScopePtr& scope, const AstExprGlobal& expr); - ExprResult checkExpr(const ScopePtr& scope, const AstExprVarargs& expr); - ExprResult checkExpr(const ScopePtr& scope, const AstExprCall& expr); - ExprResult checkExpr(const ScopePtr& scope, const AstExprIndexName& expr); - ExprResult checkExpr(const ScopePtr& scope, const AstExprIndexExpr& expr); - ExprResult checkExpr(const ScopePtr& scope, const AstExprFunction& expr, std::optional expectedType = std::nullopt); - ExprResult checkExpr(const ScopePtr& scope, const AstExprTable& expr, std::optional expectedType = std::nullopt); - ExprResult checkExpr(const ScopePtr& scope, const AstExprUnary& expr); + WithPredicate checkExpr(const ScopePtr& scope, const AstExprLocal& expr); + WithPredicate checkExpr(const ScopePtr& scope, const AstExprGlobal& expr); + WithPredicate checkExpr(const ScopePtr& scope, const AstExprVarargs& expr); + WithPredicate checkExpr(const ScopePtr& scope, const AstExprCall& expr); + WithPredicate checkExpr(const ScopePtr& scope, const AstExprIndexName& expr); + WithPredicate checkExpr(const ScopePtr& scope, const AstExprIndexExpr& expr); + WithPredicate checkExpr(const ScopePtr& scope, const AstExprFunction& expr, std::optional expectedType = std::nullopt); + WithPredicate checkExpr(const ScopePtr& scope, const AstExprTable& expr, std::optional expectedType = std::nullopt); + WithPredicate checkExpr(const ScopePtr& scope, const AstExprUnary& expr); TypeId checkRelationalOperation( const ScopePtr& scope, const AstExprBinary& expr, TypeId lhsType, TypeId rhsType, const PredicateVec& predicates = {}); TypeId checkBinaryOperation( const ScopePtr& scope, const AstExprBinary& expr, TypeId lhsType, TypeId rhsType, const PredicateVec& predicates = {}); - ExprResult checkExpr(const ScopePtr& scope, const AstExprBinary& expr); - ExprResult checkExpr(const ScopePtr& scope, const AstExprTypeAssertion& expr); - ExprResult checkExpr(const ScopePtr& scope, const AstExprError& expr); - ExprResult checkExpr(const ScopePtr& scope, const AstExprIfElse& expr, std::optional expectedType = std::nullopt); + WithPredicate checkExpr(const ScopePtr& scope, const AstExprBinary& expr); + WithPredicate checkExpr(const ScopePtr& scope, const AstExprTypeAssertion& expr); + WithPredicate checkExpr(const ScopePtr& scope, const AstExprError& expr); + WithPredicate checkExpr(const ScopePtr& scope, const AstExprIfElse& expr, std::optional expectedType = std::nullopt); TypeId checkExprTable(const ScopePtr& scope, const AstExprTable& expr, const std::vector>& fieldTypes, std::optional expectedType); @@ -179,11 +179,11 @@ struct TypeChecker void checkArgumentList( const ScopePtr& scope, Unifier& state, TypePackId paramPack, TypePackId argPack, const std::vector& argLocations); - ExprResult checkExprPack(const ScopePtr& scope, const AstExpr& expr); - ExprResult checkExprPack(const ScopePtr& scope, const AstExprCall& expr); + WithPredicate checkExprPack(const ScopePtr& scope, const AstExpr& expr); + WithPredicate checkExprPack(const ScopePtr& scope, const AstExprCall& expr); std::vector> getExpectedTypesForCall(const std::vector& overloads, size_t argumentCount, bool selfCall); - std::optional> checkCallOverload(const ScopePtr& scope, const AstExprCall& expr, TypeId fn, TypePackId retPack, - TypePackId argPack, TypePack* args, const std::vector* argLocations, const ExprResult& argListResult, + std::optional> checkCallOverload(const ScopePtr& scope, const AstExprCall& expr, TypeId fn, TypePackId retPack, + TypePackId argPack, TypePack* args, const std::vector* argLocations, const WithPredicate& argListResult, std::vector& overloadsThatMatchArgCount, std::vector& overloadsThatDont, std::vector& errors); bool handleSelfCallMismatch(const ScopePtr& scope, const AstExprCall& expr, TypePack* args, const std::vector& argLocations, const std::vector& errors); @@ -191,7 +191,7 @@ struct TypeChecker const std::vector& argLocations, const std::vector& overloads, const std::vector& overloadsThatMatchArgCount, const std::vector& errors); - ExprResult checkExprList(const ScopePtr& scope, const Location& location, const AstArray& exprs, + WithPredicate checkExprList(const ScopePtr& scope, const Location& location, const AstArray& exprs, bool substituteFreeForNil = false, const std::vector& lhsAnnotations = {}, const std::vector>& expectedTypes = {}); @@ -234,7 +234,7 @@ struct TypeChecker ErrorVec canUnify(TypeId subTy, TypeId superTy, const Location& location); ErrorVec canUnify(TypePackId subTy, TypePackId superTy, const Location& location); - void unifyLowerBound(TypePackId subTy, TypePackId superTy, const Location& location); + void unifyLowerBound(TypePackId subTy, TypePackId superTy, TypeLevel demotedLevel, const Location& location); std::optional findMetatableEntry(TypeId type, std::string entry, const Location& location); std::optional findTablePropertyRespectingMeta(TypeId lhsType, Name name, const Location& location); @@ -412,7 +412,6 @@ public: const TypeId booleanType; const TypeId threadType; const TypeId anyType; - const TypeId optionalNumberType; const TypePackId anyTypePack; diff --git a/Analysis/include/Luau/TypeVar.h b/Analysis/include/Luau/TypeVar.h index b59e7c64..ff7708d4 100644 --- a/Analysis/include/Luau/TypeVar.h +++ b/Analysis/include/Luau/TypeVar.h @@ -84,6 +84,24 @@ using Tags = std::vector; using ModuleName = std::string; +/** A TypeVar that cannot be computed. + * + * BlockedTypeVars essentially serve as a way to encode partial ordering on the + * constraint graph. Until a BlockedTypeVar is unblocked by its owning + * constraint, nothing at all can be said about it. Constraints that need to + * process a BlockedTypeVar cannot be dispatched. + * + * Whenever a BlockedTypeVar is added to the graph, we also record a constraint + * that will eventually unblock it. + */ +struct BlockedTypeVar +{ + BlockedTypeVar(); + int index; + + static int nextIndex; +}; + struct PrimitiveTypeVar { enum Type @@ -231,29 +249,29 @@ struct FunctionDefinition // TODO: Do we actually need this? We'll find out later if we can delete this. // Does not exactly belong in TypeVar.h, but this is the only way to appease the compiler. template -struct ExprResult +struct WithPredicate { T type; PredicateVec predicates; }; -using MagicFunction = std::function>( - struct TypeChecker&, const std::shared_ptr&, const class AstExprCall&, ExprResult)>; +using MagicFunction = std::function>( + struct TypeChecker&, const std::shared_ptr&, const class AstExprCall&, WithPredicate)>; struct FunctionTypeVar { // Global monomorphic function - FunctionTypeVar(TypePackId argTypes, TypePackId retType, std::optional defn = {}, bool hasSelf = false); + FunctionTypeVar(TypePackId argTypes, TypePackId retTypes, std::optional defn = {}, bool hasSelf = false); // Global polymorphic function - FunctionTypeVar(std::vector generics, std::vector genericPacks, TypePackId argTypes, TypePackId retType, + FunctionTypeVar(std::vector generics, std::vector genericPacks, TypePackId argTypes, TypePackId retTypes, std::optional defn = {}, bool hasSelf = false); // Local monomorphic function - FunctionTypeVar(TypeLevel level, TypePackId argTypes, TypePackId retType, std::optional defn = {}, bool hasSelf = false); + FunctionTypeVar(TypeLevel level, TypePackId argTypes, TypePackId retTypes, std::optional defn = {}, bool hasSelf = false); // Local polymorphic function - FunctionTypeVar(TypeLevel level, std::vector generics, std::vector genericPacks, TypePackId argTypes, TypePackId retType, + FunctionTypeVar(TypeLevel level, std::vector generics, std::vector genericPacks, TypePackId argTypes, TypePackId retTypes, std::optional defn = {}, bool hasSelf = false); TypeLevel level; @@ -263,7 +281,7 @@ struct FunctionTypeVar std::vector genericPacks; TypePackId argTypes; std::vector> argNames; - TypePackId retType; + TypePackId retTypes; std::optional definition; MagicFunction magicFunction = nullptr; // Function pointer, can be nullptr. bool hasSelf; @@ -442,7 +460,7 @@ struct LazyTypeVar using ErrorTypeVar = Unifiable::Error; -using TypeVariant = Unifiable::Variant; struct TypeVar final @@ -555,7 +573,6 @@ struct SingletonTypes const TypeId trueType; const TypeId falseType; const TypeId anyType; - const TypeId optionalNumberType; const TypePackId anyTypePack; diff --git a/Analysis/include/Luau/Unifier.h b/Analysis/include/Luau/Unifier.h index 627b52ca..b51a485e 100644 --- a/Analysis/include/Luau/Unifier.h +++ b/Analysis/include/Luau/Unifier.h @@ -110,7 +110,7 @@ private: void tryUnifyWithConstrainedSuperTypeVar(TypeId subTy, TypeId superTy); public: - void unifyLowerBound(TypePackId subTy, TypePackId superTy); + void unifyLowerBound(TypePackId subTy, TypePackId superTy, TypeLevel demotedLevel); // Report an "infinite type error" if the type "needle" already occurs within "haystack" void occursCheck(TypeId needle, TypeId haystack); diff --git a/Analysis/include/Luau/VisitTypeVar.h b/Analysis/include/Luau/VisitTypeVar.h index f3839915..642522c9 100644 --- a/Analysis/include/Luau/VisitTypeVar.h +++ b/Analysis/include/Luau/VisitTypeVar.h @@ -209,7 +209,7 @@ struct GenericTypeVarVisitor if (visit(ty, *ftv)) { traverse(ftv->argTypes); - traverse(ftv->retType); + traverse(ftv->retTypes); } } diff --git a/Analysis/src/Autocomplete.cpp b/Analysis/src/Autocomplete.cpp index a8319c59..8a63901f 100644 --- a/Analysis/src/Autocomplete.cpp +++ b/Analysis/src/Autocomplete.cpp @@ -13,7 +13,6 @@ #include #include -LUAU_FASTFLAGVARIABLE(LuauIfElseExprFixCompletionIssue, false); LUAU_FASTFLAG(LuauSelfCallAutocompleteFix2) static const std::unordered_set kStatementStartingKeywords = { @@ -268,14 +267,14 @@ static TypeCorrectKind checkTypeCorrectKind(const Module& module, TypeArena* typ auto checkFunctionType = [typeArena, &canUnify, &expectedType](const FunctionTypeVar* ftv) { if (FFlag::LuauSelfCallAutocompleteFix2) { - if (std::optional firstRetTy = first(ftv->retType)) + if (std::optional firstRetTy = first(ftv->retTypes)) return checkTypeMatch(typeArena, *firstRetTy, expectedType); return false; } else { - auto [retHead, retTail] = flatten(ftv->retType); + auto [retHead, retTail] = flatten(ftv->retTypes); if (!retHead.empty() && canUnify(retHead.front(), expectedType)) return true; @@ -454,7 +453,7 @@ static void autocompleteProps(const Module& module, TypeArena* typeArena, TypeId } else if (auto indexFunction = get(followed)) { - std::optional indexFunctionResult = first(indexFunction->retType); + std::optional indexFunctionResult = first(indexFunction->retTypes); if (indexFunctionResult) autocompleteProps(module, typeArena, rootTy, *indexFunctionResult, indexType, nodes, result, seen); } @@ -493,7 +492,7 @@ static void autocompleteProps(const Module& module, TypeArena* typeArena, TypeId autocompleteProps(module, typeArena, rootTy, followed, indexType, nodes, result, seen); else if (auto indexFunction = get(followed)) { - std::optional indexFunctionResult = first(indexFunction->retType); + std::optional indexFunctionResult = first(indexFunction->retTypes); if (indexFunctionResult) autocompleteProps(module, typeArena, rootTy, *indexFunctionResult, indexType, nodes, result, seen); } @@ -742,7 +741,7 @@ static std::optional findTypeElementAt(AstType* astType, TypeId ty, Posi if (auto element = findTypeElementAt(type->argTypes, ftv->argTypes, position)) return element; - if (auto element = findTypeElementAt(type->returnTypes, ftv->retType, position)) + if (auto element = findTypeElementAt(type->returnTypes, ftv->retTypes, position)) return element; } @@ -958,7 +957,7 @@ AutocompleteEntryMap autocompleteTypeNames(const Module& module, Position positi { if (const FunctionTypeVar* ftv = get(follow(*it))) { - if (auto ty = tryGetTypePackTypeAt(ftv->retType, tailPos)) + if (auto ty = tryGetTypePackTypeAt(ftv->retTypes, tailPos)) inferredType = *ty; } } @@ -1050,7 +1049,7 @@ AutocompleteEntryMap autocompleteTypeNames(const Module& module, Position positi { if (const FunctionTypeVar* ftv = tryGetExpectedFunctionType(module, node)) { - if (auto ty = tryGetTypePackTypeAt(ftv->retType, i)) + if (auto ty = tryGetTypePackTypeAt(ftv->retTypes, i)) tryAddTypeCorrectSuggestion(result, startScope, topType, *ty, position); } @@ -1067,7 +1066,7 @@ AutocompleteEntryMap autocompleteTypeNames(const Module& module, Position positi { if (const FunctionTypeVar* ftv = tryGetExpectedFunctionType(module, node)) { - if (auto ty = tryGetTypePackTypeAt(ftv->retType, ~0u)) + if (auto ty = tryGetTypePackTypeAt(ftv->retTypes, ~0u)) tryAddTypeCorrectSuggestion(result, startScope, topType, *ty, position); } } @@ -1266,7 +1265,7 @@ static bool autocompleteIfElseExpression( if (!parent) return false; - if (FFlag::LuauIfElseExprFixCompletionIssue && node->is()) + if (node->is()) { // Don't try to complete when the current node is an if-else expression (i.e. only try to complete when the node is a child of an if-else // expression. diff --git a/Analysis/src/BuiltinDefinitions.cpp b/Analysis/src/BuiltinDefinitions.cpp index 98737b43..2f57e23c 100644 --- a/Analysis/src/BuiltinDefinitions.cpp +++ b/Analysis/src/BuiltinDefinitions.cpp @@ -19,16 +19,16 @@ LUAU_FASTFLAGVARIABLE(LuauSetMetaTableArgsCheck, false) namespace Luau { -static std::optional> magicFunctionSelect( - TypeChecker& typechecker, const ScopePtr& scope, const AstExprCall& expr, ExprResult exprResult); -static std::optional> magicFunctionSetMetaTable( - TypeChecker& typechecker, const ScopePtr& scope, const AstExprCall& expr, ExprResult exprResult); -static std::optional> magicFunctionAssert( - TypeChecker& typechecker, const ScopePtr& scope, const AstExprCall& expr, ExprResult exprResult); -static std::optional> magicFunctionPack( - TypeChecker& typechecker, const ScopePtr& scope, const AstExprCall& expr, ExprResult exprResult); -static std::optional> magicFunctionRequire( - TypeChecker& typechecker, const ScopePtr& scope, const AstExprCall& expr, ExprResult exprResult); +static std::optional> magicFunctionSelect( + TypeChecker& typechecker, const ScopePtr& scope, const AstExprCall& expr, WithPredicate withPredicate); +static std::optional> magicFunctionSetMetaTable( + TypeChecker& typechecker, const ScopePtr& scope, const AstExprCall& expr, WithPredicate withPredicate); +static std::optional> magicFunctionAssert( + TypeChecker& typechecker, const ScopePtr& scope, const AstExprCall& expr, WithPredicate withPredicate); +static std::optional> magicFunctionPack( + TypeChecker& typechecker, const ScopePtr& scope, const AstExprCall& expr, WithPredicate withPredicate); +static std::optional> magicFunctionRequire( + TypeChecker& typechecker, const ScopePtr& scope, const AstExprCall& expr, WithPredicate withPredicate); TypeId makeUnion(TypeArena& arena, std::vector&& types) { @@ -263,10 +263,10 @@ void registerBuiltinTypes(TypeChecker& typeChecker) attachMagicFunction(getGlobalBinding(typeChecker, "require"), magicFunctionRequire); } -static std::optional> magicFunctionSelect( - TypeChecker& typechecker, const ScopePtr& scope, const AstExprCall& expr, ExprResult exprResult) +static std::optional> magicFunctionSelect( + TypeChecker& typechecker, const ScopePtr& scope, const AstExprCall& expr, WithPredicate withPredicate) { - auto [paramPack, _predicates] = exprResult; + auto [paramPack, _predicates] = withPredicate; (void)scope; @@ -287,10 +287,10 @@ static std::optional> magicFunctionSelect( if (size_t(offset) < v.size()) { std::vector result(v.begin() + offset, v.end()); - return ExprResult{typechecker.currentModule->internalTypes.addTypePack(TypePack{std::move(result), tail})}; + return WithPredicate{typechecker.currentModule->internalTypes.addTypePack(TypePack{std::move(result), tail})}; } else if (tail) - return ExprResult{*tail}; + return WithPredicate{*tail}; } typechecker.reportError(TypeError{arg1->location, GenericError{"bad argument #1 to select (index out of range)"}}); @@ -298,16 +298,16 @@ static std::optional> magicFunctionSelect( else if (AstExprConstantString* str = arg1->as()) { if (str->value.size == 1 && str->value.data[0] == '#') - return ExprResult{typechecker.currentModule->internalTypes.addTypePack({typechecker.numberType})}; + return WithPredicate{typechecker.currentModule->internalTypes.addTypePack({typechecker.numberType})}; } return std::nullopt; } -static std::optional> magicFunctionSetMetaTable( - TypeChecker& typechecker, const ScopePtr& scope, const AstExprCall& expr, ExprResult exprResult) +static std::optional> magicFunctionSetMetaTable( + TypeChecker& typechecker, const ScopePtr& scope, const AstExprCall& expr, WithPredicate withPredicate) { - auto [paramPack, _predicates] = exprResult; + auto [paramPack, _predicates] = withPredicate; TypeArena& arena = typechecker.currentModule->internalTypes; @@ -343,7 +343,7 @@ static std::optional> magicFunctionSetMetaTable( if (FFlag::LuauSetMetaTableArgsCheck && expr.args.size < 1) { - return ExprResult{}; + return WithPredicate{}; } if (!FFlag::LuauSetMetaTableArgsCheck || !expr.self) @@ -356,7 +356,7 @@ static std::optional> magicFunctionSetMetaTable( } } - return ExprResult{arena.addTypePack({mtTy})}; + return WithPredicate{arena.addTypePack({mtTy})}; } } else if (get(target) || get(target) || isTableIntersection(target)) @@ -367,13 +367,13 @@ static std::optional> magicFunctionSetMetaTable( typechecker.reportError(TypeError{expr.location, GenericError{"setmetatable should take a table"}}); } - return ExprResult{arena.addTypePack({target})}; + return WithPredicate{arena.addTypePack({target})}; } -static std::optional> magicFunctionAssert( - TypeChecker& typechecker, const ScopePtr& scope, const AstExprCall& expr, ExprResult exprResult) +static std::optional> magicFunctionAssert( + TypeChecker& typechecker, const ScopePtr& scope, const AstExprCall& expr, WithPredicate withPredicate) { - auto [paramPack, predicates] = exprResult; + auto [paramPack, predicates] = withPredicate; TypeArena& arena = typechecker.currentModule->internalTypes; @@ -382,7 +382,7 @@ static std::optional> magicFunctionAssert( { std::optional fst = first(*tail); if (!fst) - return ExprResult{paramPack}; + return WithPredicate{paramPack}; head.push_back(*fst); } @@ -397,13 +397,13 @@ static std::optional> magicFunctionAssert( head[0] = *newhead; } - return ExprResult{arena.addTypePack(TypePack{std::move(head), tail})}; + return WithPredicate{arena.addTypePack(TypePack{std::move(head), tail})}; } -static std::optional> magicFunctionPack( - TypeChecker& typechecker, const ScopePtr& scope, const AstExprCall& expr, ExprResult exprResult) +static std::optional> magicFunctionPack( + TypeChecker& typechecker, const ScopePtr& scope, const AstExprCall& expr, WithPredicate withPredicate) { - auto [paramPack, _predicates] = exprResult; + auto [paramPack, _predicates] = withPredicate; TypeArena& arena = typechecker.currentModule->internalTypes; @@ -436,7 +436,7 @@ static std::optional> magicFunctionPack( TypeId packedTable = arena.addType( TableTypeVar{{{"n", {typechecker.numberType}}}, TableIndexer(typechecker.numberType, result), scope->level, TableState::Sealed}); - return ExprResult{arena.addTypePack({packedTable})}; + return WithPredicate{arena.addTypePack({packedTable})}; } static bool checkRequirePath(TypeChecker& typechecker, AstExpr* expr) @@ -461,8 +461,8 @@ static bool checkRequirePath(TypeChecker& typechecker, AstExpr* expr) return good; } -static std::optional> magicFunctionRequire( - TypeChecker& typechecker, const ScopePtr& scope, const AstExprCall& expr, ExprResult exprResult) +static std::optional> magicFunctionRequire( + TypeChecker& typechecker, const ScopePtr& scope, const AstExprCall& expr, WithPredicate withPredicate) { TypeArena& arena = typechecker.currentModule->internalTypes; @@ -476,7 +476,7 @@ static std::optional> magicFunctionRequire( return std::nullopt; if (auto moduleInfo = typechecker.resolver->resolveModuleInfo(typechecker.currentModuleName, expr)) - return ExprResult{arena.addTypePack({typechecker.checkRequire(scope, *moduleInfo, expr.location)})}; + return WithPredicate{arena.addTypePack({typechecker.checkRequire(scope, *moduleInfo, expr.location)})}; return std::nullopt; } diff --git a/Analysis/src/Clone.cpp b/Analysis/src/Clone.cpp index 9180f309..248262ce 100644 --- a/Analysis/src/Clone.cpp +++ b/Analysis/src/Clone.cpp @@ -47,6 +47,7 @@ struct TypeCloner void operator()(const Unifiable::Generic& t); void operator()(const Unifiable::Bound& t); void operator()(const Unifiable::Error& t); + void operator()(const BlockedTypeVar& t); void operator()(const PrimitiveTypeVar& t); void operator()(const ConstrainedTypeVar& t); void operator()(const SingletonTypeVar& t); @@ -158,6 +159,11 @@ void TypeCloner::operator()(const Unifiable::Error& t) defaultClone(t); } +void TypeCloner::operator()(const BlockedTypeVar& t) +{ + defaultClone(t); +} + void TypeCloner::operator()(const PrimitiveTypeVar& t) { defaultClone(t); @@ -200,7 +206,7 @@ void TypeCloner::operator()(const FunctionTypeVar& t) ftv->tags = t.tags; ftv->argTypes = clone(t.argTypes, dest, cloneState); ftv->argNames = t.argNames; - ftv->retType = clone(t.retType, dest, cloneState); + ftv->retTypes = clone(t.retTypes, dest, cloneState); ftv->hasNoGenerics = t.hasNoGenerics; } @@ -391,7 +397,7 @@ TypeId shallowClone(TypeId ty, TypeArena& dest, const TxnLog* log) if (const FunctionTypeVar* ftv = get(ty)) { - FunctionTypeVar clone = FunctionTypeVar{ftv->level, ftv->argTypes, ftv->retType, ftv->definition, ftv->hasSelf}; + FunctionTypeVar clone = FunctionTypeVar{ftv->level, ftv->argTypes, ftv->retTypes, ftv->definition, ftv->hasSelf}; clone.generics = ftv->generics; clone.genericPacks = ftv->genericPacks; clone.magicFunction = ftv->magicFunction; diff --git a/Analysis/src/Constraint.cpp b/Analysis/src/Constraint.cpp new file mode 100644 index 00000000..6cb0e4ee --- /dev/null +++ b/Analysis/src/Constraint.cpp @@ -0,0 +1,14 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details + +#include "Luau/Constraint.h" + +namespace Luau +{ + +Constraint::Constraint(ConstraintV&& c, Location location) + : c(std::move(c)) + , location(location) +{ +} + +} // namespace Luau diff --git a/Analysis/src/ConstraintGraphBuilder.cpp b/Analysis/src/ConstraintGraphBuilder.cpp index c8f77ddf..fa627e7a 100644 --- a/Analysis/src/ConstraintGraphBuilder.cpp +++ b/Analysis/src/ConstraintGraphBuilder.cpp @@ -5,16 +5,7 @@ namespace Luau { -Constraint::Constraint(ConstraintV&& c) - : c(std::move(c)) -{ -} - -Constraint::Constraint(ConstraintV&& c, std::vector dependencies) - : c(std::move(c)) - , dependencies(dependencies) -{ -} +const AstStat* getFallthrough(const AstStat* node); // TypeInfer.cpp std::optional Scope2::lookup(Symbol sym) { @@ -68,10 +59,10 @@ Scope2* ConstraintGraphBuilder::childScope(Location location, Scope2* parent) return borrow; } -void ConstraintGraphBuilder::addConstraint(Scope2* scope, ConstraintV cv) +void ConstraintGraphBuilder::addConstraint(Scope2* scope, ConstraintV cv, Location location) { LUAU_ASSERT(scope); - scope->constraints.emplace_back(new Constraint{std::move(cv)}); + scope->constraints.emplace_back(new Constraint{std::move(cv), location}); } void ConstraintGraphBuilder::addConstraint(Scope2* scope, std::unique_ptr c) @@ -99,10 +90,18 @@ void ConstraintGraphBuilder::visit(Scope2* scope, AstStat* stat) visit(scope, s); else if (auto s = stat->as()) visit(scope, s); + else if (auto f = stat->as()) + visit(scope, f); else if (auto f = stat->as()) visit(scope, f); else if (auto r = stat->as()) visit(scope, r); + else if (auto a = stat->as()) + visit(scope, a); + else if (auto e = stat->as()) + checkPack(scope, e->expr); + else if (auto i = stat->as()) + visit(scope, i); else LUAU_ASSERT(0); } @@ -121,12 +120,30 @@ void ConstraintGraphBuilder::visit(Scope2* scope, AstStatLocal* local) scope->bindings[local] = ty; } - for (size_t i = 0; i < local->vars.size; ++i) + for (size_t i = 0; i < local->values.size; ++i) { - if (i < local->values.size) + if (local->values.data[i]->is()) + { + // HACK: we leave nil-initialized things floating under the assumption that they will later be populated. + // See the test TypeInfer/infer_locals_with_nil_value. + // Better flow awareness should make this obsolete. + } + else if (i == local->values.size - 1) + { + TypePackId exprPack = checkPack(scope, local->values.data[i]); + + if (i < local->vars.size) + { + std::vector tailValues{varTypes.begin() + i, varTypes.end()}; + TypePackId tailPack = arena->addTypePack(std::move(tailValues)); + addConstraint(scope, PackSubtypeConstraint{exprPack, tailPack}, local->location); + } + } + else { TypeId exprType = check(scope, local->values.data[i]); - addConstraint(scope, SubtypeConstraint{varTypes[i], exprType}); + if (i < varTypes.size()) + addConstraint(scope, SubtypeConstraint{varTypes[i], exprType}, local->vars.data[i]->location); } } } @@ -138,7 +155,7 @@ void addConstraints(Constraint* constraint, Scope2* scope) scope->constraints.reserve(scope->constraints.size() + scope->constraints.size()); for (const auto& c : scope->constraints) - constraint->dependencies.push_back(c.get()); + constraint->dependencies.push_back(NotNull{c.get()}); for (Scope2* childScope : scope->children) addConstraints(constraint, childScope); @@ -155,31 +172,75 @@ void ConstraintGraphBuilder::visit(Scope2* scope, AstStatLocalFunction* function TypeId functionType = nullptr; auto ty = scope->lookup(function->name); - LUAU_ASSERT(!ty.has_value()); // The parser ensures that every local function has a distinct Symbol for its name. - - functionType = freshType(scope); - scope->bindings[function->name] = functionType; - - Scope2* innerScope = childScope(function->func->body->location, scope); - TypePackId returnType = freshTypePack(scope); - innerScope->returnType = returnType; - - std::vector argTypes; - - for (AstLocal* local : function->func->args) + if (ty.has_value()) { - TypeId t = freshType(innerScope); - argTypes.push_back(t); - innerScope->bindings[local] = t; // TODO annotations + // TODO: This is duplicate definition of a local function. Is this allowed? + functionType = *ty; + } + else + { + functionType = arena->addType(BlockedTypeVar{}); + scope->bindings[function->name] = functionType; } - for (AstStat* stat : function->func->body->body) - visit(innerScope, stat); + auto [actualFunctionType, innerScope] = checkFunctionSignature(scope, function->func); + innerScope->bindings[function->name] = actualFunctionType; - FunctionTypeVar actualFunction{arena->addTypePack(argTypes), returnType}; - TypeId actualFunctionType = arena->addType(std::move(actualFunction)); + checkFunctionBody(innerScope, function->func); - std::unique_ptr c{new Constraint{GeneralizationConstraint{functionType, actualFunctionType, innerScope}}}; + std::unique_ptr c{new Constraint{GeneralizationConstraint{functionType, actualFunctionType, innerScope}, function->location}}; + addConstraints(c.get(), innerScope); + + addConstraint(scope, std::move(c)); +} + +void ConstraintGraphBuilder::visit(Scope2* scope, AstStatFunction* function) +{ + // Name could be AstStatLocal, AstStatGlobal, AstStatIndexName. + // With or without self + + TypeId functionType = nullptr; + + auto [actualFunctionType, innerScope] = checkFunctionSignature(scope, function->func); + + if (AstExprLocal* localName = function->name->as()) + { + std::optional existingFunctionTy = scope->lookup(localName->local); + if (existingFunctionTy) + { + // Duplicate definition + functionType = *existingFunctionTy; + } + else + { + functionType = arena->addType(BlockedTypeVar{}); + scope->bindings[localName->local] = functionType; + } + innerScope->bindings[localName->local] = actualFunctionType; + } + else if (AstExprGlobal* globalName = function->name->as()) + { + std::optional existingFunctionTy = scope->lookup(globalName->name); + if (existingFunctionTy) + { + // Duplicate definition + functionType = *existingFunctionTy; + } + else + { + functionType = arena->addType(BlockedTypeVar{}); + rootScope->bindings[globalName->name] = functionType; + } + innerScope->bindings[globalName->name] = actualFunctionType; + } + else if (AstExprIndexName* indexName = function->name->as()) + { + LUAU_ASSERT(0); // not yet implemented + } + + checkFunctionBody(innerScope, function->func); + + std::unique_ptr c{new Constraint{GeneralizationConstraint{functionType, actualFunctionType, innerScope}, function->location}}; addConstraints(c.get(), innerScope); addConstraint(scope, std::move(c)); @@ -190,7 +251,7 @@ void ConstraintGraphBuilder::visit(Scope2* scope, AstStatReturn* ret) LUAU_ASSERT(scope); TypePackId exprTypes = checkPack(scope, ret->list); - addConstraint(scope, PackSubtypeConstraint{exprTypes, scope->returnType}); + addConstraint(scope, PackSubtypeConstraint{exprTypes, scope->returnType}, ret->location); } void ConstraintGraphBuilder::visit(Scope2* scope, AstStatBlock* block) @@ -201,6 +262,28 @@ void ConstraintGraphBuilder::visit(Scope2* scope, AstStatBlock* block) visit(scope, stat); } +void ConstraintGraphBuilder::visit(Scope2* scope, AstStatAssign* assign) +{ + TypePackId varPackId = checkExprList(scope, assign->vars); + TypePackId valuePack = checkPack(scope, assign->values); + + addConstraint(scope, PackSubtypeConstraint{valuePack, varPackId}, assign->location); +} + +void ConstraintGraphBuilder::visit(Scope2* scope, AstStatIf* ifStatement) +{ + check(scope, ifStatement->condition); + + Scope2* thenScope = childScope(ifStatement->thenbody->location, scope); + visit(thenScope, ifStatement->thenbody); + + if (ifStatement->elsebody) + { + Scope2* elseScope = childScope(ifStatement->elsebody->location, scope); + visit(elseScope, ifStatement->elsebody); + } +} + TypePackId ConstraintGraphBuilder::checkPack(Scope2* scope, AstArray exprs) { LUAU_ASSERT(scope); @@ -224,75 +307,256 @@ TypePackId ConstraintGraphBuilder::checkPack(Scope2* scope, AstArray e return arena->addTypePack(TypePack{std::move(types), last}); } +TypePackId ConstraintGraphBuilder::checkExprList(Scope2* scope, const AstArray& exprs) +{ + TypePackId result = arena->addTypePack({}); + TypePack* resultPack = getMutable(result); + LUAU_ASSERT(resultPack); + + for (size_t i = 0; i < exprs.size; ++i) + { + AstExpr* expr = exprs.data[i]; + if (i < exprs.size - 1) + resultPack->head.push_back(check(scope, expr)); + else + resultPack->tail = checkPack(scope, expr); + } + + if (resultPack->head.empty() && resultPack->tail) + return *resultPack->tail; + else + return result; +} + TypePackId ConstraintGraphBuilder::checkPack(Scope2* scope, AstExpr* expr) { LUAU_ASSERT(scope); - // TEMP TEMP TEMP HACK HACK HACK FIXME FIXME - TypeId t = check(scope, expr); - return arena->addTypePack({t}); + TypePackId result = nullptr; + + if (AstExprCall* call = expr->as()) + { + std::vector args; + + for (AstExpr* arg : call->args) + { + args.push_back(check(scope, arg)); + } + + // TODO self + + TypeId fnType = check(scope, call->func); + + astOriginalCallTypes[call->func] = fnType; + + TypeId instantiatedType = freshType(scope); + addConstraint(scope, InstantiationConstraint{instantiatedType, fnType}, expr->location); + + TypePackId rets = freshTypePack(scope); + FunctionTypeVar ftv(arena->addTypePack(TypePack{args, {}}), rets); + TypeId inferredFnType = arena->addType(ftv); + + addConstraint(scope, SubtypeConstraint{inferredFnType, instantiatedType}, expr->location); + result = rets; + } + else + { + TypeId t = check(scope, expr); + result = arena->addTypePack({t}); + } + + LUAU_ASSERT(result); + astTypePacks[expr] = result; + return result; } TypeId ConstraintGraphBuilder::check(Scope2* scope, AstExpr* expr) { LUAU_ASSERT(scope); - if (auto a = expr->as()) - return singletonTypes.stringType; - else if (auto a = expr->as()) - return singletonTypes.numberType; - else if (auto a = expr->as()) - return singletonTypes.booleanType; - else if (auto a = expr->as()) - return singletonTypes.nilType; + TypeId result = nullptr; + + if (auto group = expr->as()) + result = check(scope, group->expr); + else if (expr->is()) + result = singletonTypes.stringType; + else if (expr->is()) + result = singletonTypes.numberType; + else if (expr->is()) + result = singletonTypes.booleanType; + else if (expr->is()) + result = singletonTypes.nilType; else if (auto a = expr->as()) { std::optional ty = scope->lookup(a->local); if (ty) - return *ty; + result = *ty; else - return singletonTypes.errorRecoveryType(singletonTypes.anyType); // FIXME? Record an error at this point? + result = singletonTypes.errorRecoveryType(); // FIXME? Record an error at this point? + } + else if (auto g = expr->as()) + { + std::optional ty = scope->lookup(g->name); + if (ty) + result = *ty; + else + result = singletonTypes.errorRecoveryType(); // FIXME? Record an error at this point? } else if (auto a = expr->as()) { - std::vector args; - - for (AstExpr* arg : a->args) + TypePackId packResult = checkPack(scope, expr); + if (auto f = first(packResult)) + return *f; + else if (get(packResult)) { - args.push_back(check(scope, arg)); + TypeId typeResult = freshType(scope); + TypePack onePack{{typeResult}, freshTypePack(scope)}; + TypePackId oneTypePack = arena->addTypePack(std::move(onePack)); + + addConstraint(scope, PackSubtypeConstraint{packResult, oneTypePack}, expr->location); + + return typeResult; } - - TypeId fnType = check(scope, a->func); - TypeId instantiatedType = freshType(scope); - addConstraint(scope, InstantiationConstraint{instantiatedType, fnType}); - - TypeId firstRet = freshType(scope); - TypePackId rets = arena->addTypePack(TypePack{{firstRet}, arena->addTypePack(TypePackVar{FreeTypePack{TypeLevel{}}})}); - FunctionTypeVar ftv(arena->addTypePack(TypePack{args, {}}), rets); - TypeId inferredFnType = arena->addType(ftv); - - addConstraint(scope, SubtypeConstraint{inferredFnType, instantiatedType}); - return firstRet; + } + else if (auto a = expr->as()) + { + auto [fnType, functionScope] = checkFunctionSignature(scope, a); + checkFunctionBody(functionScope, a); + return fnType; + } + else if (auto indexName = expr->as()) + { + result = check(scope, indexName); + } + else if (auto table = expr->as()) + { + result = checkExprTable(scope, table); } else { LUAU_ASSERT(0); - return freshType(scope); + result = freshType(scope); + } + + LUAU_ASSERT(result); + astTypes[expr] = result; + return result; +} + +TypeId ConstraintGraphBuilder::check(Scope2* scope, AstExprIndexName* indexName) +{ + TypeId obj = check(scope, indexName->expr); + TypeId result = freshType(scope); + + TableTypeVar::Props props{{indexName->index.value, Property{result}}}; + const std::optional indexer; + TableTypeVar ttv{std::move(props), indexer, TypeLevel{}, TableState::Free}; + + TypeId expectedTableType = arena->addType(std::move(ttv)); + + addConstraint(scope, SubtypeConstraint{obj, expectedTableType}, indexName->location); + + return result; +} + +TypeId ConstraintGraphBuilder::checkExprTable(Scope2* scope, AstExprTable* expr) +{ + TypeId ty = arena->addType(TableTypeVar{}); + TableTypeVar* ttv = getMutable(ty); + LUAU_ASSERT(ttv); + + auto createIndexer = [this, scope, ttv]( + TypeId currentIndexType, TypeId currentResultType, Location itemLocation, std::optional keyLocation) { + if (!ttv->indexer) + { + TypeId indexType = this->freshType(scope); + TypeId resultType = this->freshType(scope); + ttv->indexer = TableIndexer{indexType, resultType}; + } + + addConstraint(scope, SubtypeConstraint{ttv->indexer->indexType, currentIndexType}, keyLocation ? *keyLocation : itemLocation); + addConstraint(scope, SubtypeConstraint{ttv->indexer->indexResultType, currentResultType}, itemLocation); + }; + + for (const AstExprTable::Item& item : expr->items) + { + TypeId itemTy = check(scope, item.value); + + if (item.key) + { + // Even though we don't need to use the type of the item's key if + // it's a string constant, we still want to check it to populate + // astTypes. + TypeId keyTy = check(scope, item.key); + + if (AstExprConstantString* key = item.key->as()) + { + ttv->props[key->value.begin()] = {itemTy}; + } + else + { + createIndexer(keyTy, itemTy, item.value->location, item.key->location); + } + } + else + { + TypeId numberType = singletonTypes.numberType; + createIndexer(numberType, itemTy, item.value->location, std::nullopt); + } + } + + return ty; +} + +std::pair ConstraintGraphBuilder::checkFunctionSignature(Scope2* parent, AstExprFunction* fn) +{ + Scope2* innerScope = childScope(fn->body->location, parent); + TypePackId returnType = freshTypePack(innerScope); + innerScope->returnType = returnType; + + std::vector argTypes; + + for (AstLocal* local : fn->args) + { + TypeId t = freshType(innerScope); + argTypes.push_back(t); + innerScope->bindings[local] = t; // TODO annotations + } + + FunctionTypeVar actualFunction{arena->addTypePack(argTypes), returnType}; + TypeId actualFunctionType = arena->addType(std::move(actualFunction)); + LUAU_ASSERT(actualFunctionType); + astTypes[fn] = actualFunctionType; + + return {actualFunctionType, innerScope}; +} + +void ConstraintGraphBuilder::checkFunctionBody(Scope2* scope, AstExprFunction* fn) +{ + for (AstStat* stat : fn->body->body) + visit(scope, stat); + + // If it is possible for execution to reach the end of the function, the return type must be compatible with () + + if (nullptr != getFallthrough(fn->body)) + { + TypePackId empty = arena->addTypePack({}); // TODO we could have CSG retain one of these forever + addConstraint(scope, PackSubtypeConstraint{scope->returnType, empty}, fn->body->location); } } -static void collectConstraints(std::vector& result, Scope2* scope) +void collectConstraints(std::vector>& result, Scope2* scope) { for (const auto& c : scope->constraints) - result.push_back(c.get()); + result.push_back(NotNull{c.get()}); for (Scope2* child : scope->children) collectConstraints(result, child); } -std::vector collectConstraints(Scope2* rootScope) +std::vector> collectConstraints(Scope2* rootScope) { - std::vector result; + std::vector> result; collectConstraints(result, rootScope); return result; } diff --git a/Analysis/src/ConstraintSolver.cpp b/Analysis/src/ConstraintSolver.cpp index f40cd4b3..41dfd892 100644 --- a/Analysis/src/ConstraintSolver.cpp +++ b/Analysis/src/ConstraintSolver.cpp @@ -7,6 +7,7 @@ #include "Luau/Unifier.h" LUAU_FASTFLAGVARIABLE(DebugLuauLogSolver, false); +LUAU_FASTFLAGVARIABLE(DebugLuauLogSolverToJson, false); namespace Luau { @@ -58,11 +59,11 @@ ConstraintSolver::ConstraintSolver(TypeArena* arena, Scope2* rootScope) , constraints(collectConstraints(rootScope)) , rootScope(rootScope) { - for (const Constraint* c : constraints) + for (NotNull c : constraints) { - unsolvedConstraints.insert(c); + unsolvedConstraints.push_back(c); - for (const Constraint* dep : c->dependencies) + for (NotNull dep : c->dependencies) { block(dep, c); } @@ -74,8 +75,6 @@ void ConstraintSolver::run() if (done()) return; - bool progress = false; - ToStringOptions opts; if (FFlag::DebugLuauLogSolver) @@ -84,44 +83,80 @@ void ConstraintSolver::run() dump(this, opts); } - do + if (FFlag::DebugLuauLogSolverToJson) { - progress = false; + logger.captureBoundarySnapshot(rootScope, unsolvedConstraints); + } - auto it = begin(unsolvedConstraints); - auto endIt = end(unsolvedConstraints); + auto runSolverPass = [&](bool force) { + bool progress = false; - while (it != endIt) + size_t i = 0; + while (i < unsolvedConstraints.size()) { - if (isBlocked(*it)) + NotNull c = unsolvedConstraints[i]; + if (!force && isBlocked(c)) { - ++it; + ++i; continue; } - std::string saveMe = FFlag::DebugLuauLogSolver ? toString(**it, opts) : std::string{}; + std::string saveMe = FFlag::DebugLuauLogSolver ? toString(*c, opts) : std::string{}; - bool success = tryDispatch(*it); - progress = progress || success; + if (FFlag::DebugLuauLogSolverToJson) + { + logger.prepareStepSnapshot(rootScope, c, unsolvedConstraints); + } + + bool success = tryDispatch(c, force); + + progress |= success; - auto saveIt = it; - ++it; if (success) { - unsolvedConstraints.erase(saveIt); + unsolvedConstraints.erase(unsolvedConstraints.begin() + i); + + if (FFlag::DebugLuauLogSolverToJson) + { + logger.commitPreparedStepSnapshot(); + } + if (FFlag::DebugLuauLogSolver) { + if (force) + printf("Force "); printf("Dispatched\n\t%s\n", saveMe.c_str()); dump(this, opts); } } + else + ++i; + + if (force && success) + return true; } + + return progress; + }; + + bool progress = false; + do + { + progress = runSolverPass(false); + if (!progress) + progress |= runSolverPass(true); } while (progress); if (FFlag::DebugLuauLogSolver) + { dumpBindings(rootScope, opts); + } - LUAU_ASSERT(done()); + if (FFlag::DebugLuauLogSolverToJson) + { + logger.captureBoundarySnapshot(rootScope, unsolvedConstraints); + printf("Logger output:\n%s\n", logger.compileOutput().c_str()); + } } bool ConstraintSolver::done() @@ -129,21 +164,21 @@ bool ConstraintSolver::done() return unsolvedConstraints.empty(); } -bool ConstraintSolver::tryDispatch(const Constraint* constraint) +bool ConstraintSolver::tryDispatch(NotNull constraint, bool force) { - if (isBlocked(constraint)) + if (!force && isBlocked(constraint)) return false; bool success = false; if (auto sc = get(*constraint)) - success = tryDispatch(*sc); + success = tryDispatch(*sc, constraint, force); else if (auto psc = get(*constraint)) - success = tryDispatch(*psc); + success = tryDispatch(*psc, constraint, force); else if (auto gc = get(*constraint)) - success = tryDispatch(*gc); + success = tryDispatch(*gc, constraint, force); else if (auto ic = get(*constraint)) - success = tryDispatch(*ic, constraint); + success = tryDispatch(*ic, constraint, force); else LUAU_ASSERT(0); @@ -155,65 +190,66 @@ bool ConstraintSolver::tryDispatch(const Constraint* constraint) return success; } -bool ConstraintSolver::tryDispatch(const SubtypeConstraint& c) +bool ConstraintSolver::tryDispatch(const SubtypeConstraint& c, NotNull constraint, bool force) { - unify(c.subType, c.superType); + if (isBlocked(c.subType)) + return block(c.subType, constraint); + else if (isBlocked(c.superType)) + return block(c.superType, constraint); + + unify(c.subType, c.superType, constraint->location); + unblock(c.subType); unblock(c.superType); return true; } -bool ConstraintSolver::tryDispatch(const PackSubtypeConstraint& c) +bool ConstraintSolver::tryDispatch(const PackSubtypeConstraint& c, NotNull constraint, bool force) { - unify(c.subPack, c.superPack); + unify(c.subPack, c.superPack, constraint->location); unblock(c.subPack); unblock(c.superPack); return true; } -bool ConstraintSolver::tryDispatch(const GeneralizationConstraint& constraint) +bool ConstraintSolver::tryDispatch(const GeneralizationConstraint& c, NotNull constraint, bool force) { - unify(constraint.subType, constraint.superType); + if (isBlocked(c.sourceType)) + return block(c.sourceType, constraint); - quantify(constraint.superType, constraint.scope); - unblock(constraint.subType); - unblock(constraint.superType); + if (isBlocked(c.generalizedType)) + asMutable(c.generalizedType)->ty.emplace(c.sourceType); + else + unify(c.generalizedType, c.sourceType, constraint->location); + + TypeId generalized = quantify(arena, c.sourceType, c.scope); + *asMutable(c.sourceType) = *generalized; + + unblock(c.generalizedType); + unblock(c.sourceType); return true; } -bool ConstraintSolver::tryDispatch(const InstantiationConstraint& c, const Constraint* constraint) +bool ConstraintSolver::tryDispatch(const InstantiationConstraint& c, NotNull constraint, bool force) { - TypeId superType = follow(c.superType); - if (const FunctionTypeVar* ftv = get(superType)) - { - if (!ftv->generalized) - { - block(superType, constraint); - return false; - } - } - else if (get(superType)) - { - block(superType, constraint); - return false; - } - // TODO: Error if it's a primitive or something + if (isBlocked(c.superType)) + return block(c.superType, constraint); Instantiation inst(TxnLog::empty(), arena, TypeLevel{}); std::optional instantiated = inst.substitute(c.superType); LUAU_ASSERT(instantiated); // TODO FIXME HANDLE THIS - unify(c.subType, *instantiated); + unify(c.subType, *instantiated, constraint->location); unblock(c.subType); return true; } -void ConstraintSolver::block_(BlockedConstraintId target, const Constraint* constraint) +void ConstraintSolver::block_(BlockedConstraintId target, NotNull constraint) { blocked[target].push_back(constraint); @@ -221,19 +257,21 @@ void ConstraintSolver::block_(BlockedConstraintId target, const Constraint* cons count += 1; } -void ConstraintSolver::block(const Constraint* target, const Constraint* constraint) +void ConstraintSolver::block(NotNull target, NotNull constraint) { block_(target, constraint); } -void ConstraintSolver::block(TypeId target, const Constraint* constraint) +bool ConstraintSolver::block(TypeId target, NotNull constraint) { block_(target, constraint); + return false; } -void ConstraintSolver::block(TypePackId target, const Constraint* constraint) +bool ConstraintSolver::block(TypePackId target, NotNull constraint) { block_(target, constraint); + return false; } void ConstraintSolver::unblock_(BlockedConstraintId progressed) @@ -243,7 +281,7 @@ void ConstraintSolver::unblock_(BlockedConstraintId progressed) return; // unblocked should contain a value always, because of the above check - for (const Constraint* unblockedConstraint : it->second) + for (NotNull unblockedConstraint : it->second) { auto& count = blockedConstraints[unblockedConstraint]; // This assertion being hit indicates that `blocked` and @@ -257,7 +295,7 @@ void ConstraintSolver::unblock_(BlockedConstraintId progressed) blocked.erase(it); } -void ConstraintSolver::unblock(const Constraint* progressed) +void ConstraintSolver::unblock(NotNull progressed) { return unblock_(progressed); } @@ -272,35 +310,33 @@ void ConstraintSolver::unblock(TypePackId progressed) return unblock_(progressed); } -bool ConstraintSolver::isBlocked(const Constraint* constraint) +bool ConstraintSolver::isBlocked(TypeId ty) +{ + return nullptr != get(follow(ty)); +} + +bool ConstraintSolver::isBlocked(NotNull constraint) { auto blockedIt = blockedConstraints.find(constraint); return blockedIt != blockedConstraints.end() && blockedIt->second > 0; } -void ConstraintSolver::reportErrors(const std::vector& errors) -{ - this->errors.insert(end(this->errors), begin(errors), end(errors)); -} - -void ConstraintSolver::unify(TypeId subType, TypeId superType) +void ConstraintSolver::unify(TypeId subType, TypeId superType, Location location) { UnifierSharedState sharedState{&iceReporter}; - Unifier u{arena, Mode::Strict, Location{}, Covariant, sharedState}; + Unifier u{arena, Mode::Strict, location, Covariant, sharedState}; u.tryUnify(subType, superType); u.log.commit(); - reportErrors(u.errors); } -void ConstraintSolver::unify(TypePackId subPack, TypePackId superPack) +void ConstraintSolver::unify(TypePackId subPack, TypePackId superPack, Location location) { UnifierSharedState sharedState{&iceReporter}; - Unifier u{arena, Mode::Strict, Location{}, Covariant, sharedState}; + Unifier u{arena, Mode::Strict, location, Covariant, sharedState}; u.tryUnify(subPack, superPack); u.log.commit(); - reportErrors(u.errors); } } // namespace Luau diff --git a/Analysis/src/ConstraintSolverLogger.cpp b/Analysis/src/ConstraintSolverLogger.cpp new file mode 100644 index 00000000..2f93c280 --- /dev/null +++ b/Analysis/src/ConstraintSolverLogger.cpp @@ -0,0 +1,139 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details + +#include "Luau/ConstraintSolverLogger.h" + +namespace Luau +{ + +static std::string dumpScopeAndChildren(const Scope2* scope, ToStringOptions& opts) +{ + std::string output = "{\"bindings\":{"; + + bool comma = false; + for (const auto& [name, type] : scope->bindings) + { + if (comma) + output += ","; + + output += "\""; + output += name.c_str(); + output += "\": \""; + + ToStringResult result = toStringDetailed(type, opts); + opts.nameMap = std::move(result.nameMap); + output += result.name; + output += "\""; + + comma = true; + } + + output += "},\"children\":["; + comma = false; + + for (const Scope2* child : scope->children) + { + if (comma) + output += ","; + + output += dumpScopeAndChildren(child, opts); + comma = true; + } + + output += "]}"; + return output; +} + +static std::string dumpConstraintsToDot(std::vector>& constraints, ToStringOptions& opts) +{ + std::string result = "digraph Constraints {\\n"; + + std::unordered_set> contained; + for (NotNull c : constraints) + { + contained.insert(c); + } + + for (NotNull c : constraints) + { + std::string id = std::to_string(reinterpret_cast(c.get())); + result += id; + result += " [label=\\\""; + result += toString(*c, opts).c_str(); + result += "\\\"];\\n"; + + for (NotNull dep : c->dependencies) + { + if (contained.count(dep) == 0) + continue; + + result += std::to_string(reinterpret_cast(dep.get())); + result += " -> "; + result += id; + result += ";\\n"; + } + } + + result += "}"; + + return result; +} + +std::string ConstraintSolverLogger::compileOutput() +{ + std::string output = "["; + bool comma = false; + + for (const std::string& snapshot : snapshots) + { + if (comma) + output += ","; + output += snapshot; + + comma = true; + } + + output += "]"; + return output; +} + +void ConstraintSolverLogger::captureBoundarySnapshot(const Scope2* rootScope, std::vector>& unsolvedConstraints) +{ + std::string snapshot = "{\"type\":\"boundary\",\"rootScope\":"; + + snapshot += dumpScopeAndChildren(rootScope, opts); + snapshot += ",\"constraintGraph\":\""; + snapshot += dumpConstraintsToDot(unsolvedConstraints, opts); + snapshot += "\"}"; + + snapshots.push_back(std::move(snapshot)); +} + +void ConstraintSolverLogger::prepareStepSnapshot( + const Scope2* rootScope, NotNull current, std::vector>& unsolvedConstraints) +{ + // LUAU_ASSERT(!preparedSnapshot); + + std::string snapshot = "{\"type\":\"step\",\"rootScope\":"; + + snapshot += dumpScopeAndChildren(rootScope, opts); + snapshot += ",\"constraintGraph\":\""; + snapshot += dumpConstraintsToDot(unsolvedConstraints, opts); + snapshot += "\",\"currentId\":\""; + snapshot += std::to_string(reinterpret_cast(current.get())); + snapshot += "\",\"current\":\""; + snapshot += toString(*current, opts); + snapshot += "\"}"; + + preparedSnapshot = std::move(snapshot); +} + +void ConstraintSolverLogger::commitPreparedStepSnapshot() +{ + if (preparedSnapshot) + { + snapshots.push_back(std::move(*preparedSnapshot)); + preparedSnapshot = std::nullopt; + } +} + +} // namespace Luau diff --git a/Analysis/src/Frontend.cpp b/Analysis/src/Frontend.cpp index 741a35cf..9e025062 100644 --- a/Analysis/src/Frontend.cpp +++ b/Analysis/src/Frontend.cpp @@ -1,16 +1,17 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #include "Luau/Frontend.h" -#include "Luau/Common.h" #include "Luau/Clone.h" +#include "Luau/Common.h" #include "Luau/Config.h" -#include "Luau/FileResolver.h" #include "Luau/ConstraintGraphBuilder.h" #include "Luau/ConstraintSolver.h" +#include "Luau/FileResolver.h" #include "Luau/Parser.h" #include "Luau/Scope.h" #include "Luau/StringUtils.h" #include "Luau/TimeTrace.h" +#include "Luau/TypeChecker2.h" #include "Luau/TypeInfer.h" #include "Luau/Variant.h" @@ -216,7 +217,7 @@ ErrorVec accumulateErrors( continue; const SourceNode& sourceNode = it->second; - queue.insert(queue.end(), sourceNode.requires.begin(), sourceNode.requires.end()); + queue.insert(queue.end(), sourceNode.requireSet.begin(), sourceNode.requireSet.end()); // FIXME: If a module has a syntax error, we won't be able to re-report it here. // The solution is probably to move errors from Module to SourceNode @@ -586,7 +587,7 @@ bool Frontend::parseGraph(std::vector& buildQueue, CheckResult& chec path.push_back(top); // push children - for (const ModuleName& dep : top->requires) + for (const ModuleName& dep : top->requireSet) { auto it = sourceNodes.find(dep); if (it != sourceNodes.end()) @@ -738,7 +739,7 @@ void Frontend::markDirty(const ModuleName& name, std::vector* marked std::unordered_map> reverseDeps; for (const auto& module : sourceNodes) { - for (const auto& dep : module.second.requires) + for (const auto& dep : module.second.requireSet) reverseDeps[dep].push_back(module.first); } @@ -797,9 +798,14 @@ ModulePtr Frontend::check(const SourceModule& sourceModule, Mode mode, const Sco cs.run(); result->scope2s = std::move(cgb.scopes); + result->astTypes = std::move(cgb.astTypes); + result->astTypePacks = std::move(cgb.astTypePacks); + result->astOriginalCallTypes = std::move(cgb.astOriginalCallTypes); result->clonePublicInterface(iceHandler); + Luau::check(sourceModule, result.get()); + return result; } @@ -841,8 +847,8 @@ std::pair Frontend::getSourceNode(CheckResult& check SourceModule result = parse(name, source->source, opts); result.type = source->type; - RequireTraceResult& requireTrace = requires[name]; - requireTrace = traceRequires(fileResolver, result.root, name); + RequireTraceResult& require = requireTrace[name]; + require = traceRequires(fileResolver, result.root, name); SourceNode& sourceNode = sourceNodes[name]; SourceModule& sourceModule = sourceModules[name]; @@ -851,7 +857,7 @@ std::pair Frontend::getSourceNode(CheckResult& check sourceModule.environmentName = environmentName; sourceNode.name = name; - sourceNode.requires.clear(); + sourceNode.requireSet.clear(); sourceNode.requireLocations.clear(); sourceNode.dirtySourceModule = false; @@ -861,10 +867,10 @@ std::pair Frontend::getSourceNode(CheckResult& check sourceNode.dirtyModuleForAutocomplete = true; } - for (const auto& [moduleName, location] : requireTrace.requires) - sourceNode.requires.insert(moduleName); + for (const auto& [moduleName, location] : require.requireList) + sourceNode.requireSet.insert(moduleName); - sourceNode.requireLocations = requireTrace.requires; + sourceNode.requireLocations = require.requireList; return {&sourceNode, &sourceModule}; } @@ -925,8 +931,8 @@ SourceModule Frontend::parse(const ModuleName& name, std::string_view src, const std::optional FrontendModuleResolver::resolveModuleInfo(const ModuleName& currentModuleName, const AstExpr& pathExpr) { // FIXME I think this can be pushed into the FileResolver. - auto it = frontend->requires.find(currentModuleName); - if (it == frontend->requires.end()) + auto it = frontend->requireTrace.find(currentModuleName); + if (it == frontend->requireTrace.end()) { // CLI-43699 // If we can't find the current module name, that's because we bypassed the frontend's initializer @@ -1025,7 +1031,7 @@ void Frontend::clear() sourceModules.clear(); moduleResolver.modules.clear(); moduleResolverForAutocomplete.modules.clear(); - requires.clear(); + requireTrace.clear(); } } // namespace Luau diff --git a/Analysis/src/Instantiation.cpp b/Analysis/src/Instantiation.cpp index f145a511..77c62422 100644 --- a/Analysis/src/Instantiation.cpp +++ b/Analysis/src/Instantiation.cpp @@ -40,7 +40,7 @@ TypeId Instantiation::clean(TypeId ty) const FunctionTypeVar* ftv = log->getMutable(ty); LUAU_ASSERT(ftv); - FunctionTypeVar clone = FunctionTypeVar{level, ftv->argTypes, ftv->retType, ftv->definition, ftv->hasSelf}; + FunctionTypeVar clone = FunctionTypeVar{level, ftv->argTypes, ftv->retTypes, ftv->definition, ftv->hasSelf}; clone.magicFunction = ftv->magicFunction; clone.tags = ftv->tags; clone.argNames = ftv->argNames; diff --git a/Analysis/src/Linter.cpp b/Analysis/src/Linter.cpp index 200b7d1b..50868e56 100644 --- a/Analysis/src/Linter.cpp +++ b/Analysis/src/Linter.cpp @@ -2282,7 +2282,7 @@ private: size_t getReturnCount(TypeId ty) { if (auto ftv = get(ty)) - return size(ftv->retType); + return size(ftv->retTypes); if (auto itv = get(ty)) { @@ -2291,7 +2291,7 @@ private: for (TypeId part : itv->parts) if (auto ftv = get(follow(part))) - result = std::max(result, size(ftv->retType)); + result = std::max(result, size(ftv->retTypes)); return result; } diff --git a/Analysis/src/Normalize.cpp b/Analysis/src/Normalize.cpp index 11403be5..d36665e2 100644 --- a/Analysis/src/Normalize.cpp +++ b/Analysis/src/Normalize.cpp @@ -17,6 +17,7 @@ LUAU_FASTFLAGVARIABLE(LuauNormalizeCombineTableFix, false); LUAU_FASTFLAGVARIABLE(LuauNormalizeFlagIsConservative, false); LUAU_FASTFLAGVARIABLE(LuauNormalizeCombineEqFix, false); LUAU_FASTFLAGVARIABLE(LuauReplaceReplacer, false); +LUAU_FASTFLAG(LuauQuantifyConstrained) namespace Luau { @@ -273,6 +274,18 @@ bool isSubtype(TypeId subTy, TypeId superTy, InternalErrorReporter& ice) return ok; } +bool isSubtype(TypePackId subPack, TypePackId superPack, InternalErrorReporter& ice) +{ + UnifierSharedState sharedState{&ice}; + TypeArena arena; + Unifier u{&arena, Mode::Strict, Location{}, Covariant, sharedState}; + u.anyIsTop = true; + + u.tryUnify(subPack, superPack); + const bool ok = u.errors.empty() && u.log.empty(); + return ok; +} + template static bool areNormal_(const T& t, const std::unordered_set& seen, InternalErrorReporter& ice) { @@ -390,6 +403,7 @@ struct Normalize final : TypeVarVisitor bool visit(TypeId ty, const ConstrainedTypeVar& ctvRef) override { CHECK_ITERATION_LIMIT(false); + LUAU_ASSERT(!ty->normal); ConstrainedTypeVar* ctv = const_cast(&ctvRef); @@ -401,14 +415,21 @@ struct Normalize final : TypeVarVisitor std::vector newParts = normalizeUnion(parts); - const bool normal = areNormal(newParts, seen, ice); - - if (newParts.size() == 1) - *asMutable(ty) = BoundTypeVar{newParts[0]}; + if (FFlag::LuauQuantifyConstrained) + { + ctv->parts = std::move(newParts); + } else - *asMutable(ty) = UnionTypeVar{std::move(newParts)}; + { + const bool normal = areNormal(newParts, seen, ice); - asMutable(ty)->normal = normal; + if (newParts.size() == 1) + *asMutable(ty) = BoundTypeVar{newParts[0]}; + else + *asMutable(ty) = UnionTypeVar{std::move(newParts)}; + + asMutable(ty)->normal = normal; + } return false; } @@ -421,9 +442,9 @@ struct Normalize final : TypeVarVisitor return false; traverse(ftv.argTypes); - traverse(ftv.retType); + traverse(ftv.retTypes); - asMutable(ty)->normal = areNormal(ftv.argTypes, seen, ice) && areNormal(ftv.retType, seen, ice); + asMutable(ty)->normal = areNormal(ftv.argTypes, seen, ice) && areNormal(ftv.retTypes, seen, ice); return false; } @@ -465,7 +486,14 @@ struct Normalize final : TypeVarVisitor checkNormal(ttv.indexer->indexResultType); } - asMutable(ty)->normal = normal; + // An unsealed table can never be normal, ditto for free tables iff the type it is bound to is also not normal. + if (FFlag::LuauQuantifyConstrained) + { + if (ttv.state == TableState::Generic || ttv.state == TableState::Sealed || (ttv.state == TableState::Free && follow(ty)->normal)) + asMutable(ty)->normal = normal; + } + else + asMutable(ty)->normal = normal; return false; } diff --git a/Analysis/src/Quantify.cpp b/Analysis/src/Quantify.cpp index 21775373..2004d153 100644 --- a/Analysis/src/Quantify.cpp +++ b/Analysis/src/Quantify.cpp @@ -2,15 +2,32 @@ #include "Luau/Quantify.h" +#include "Luau/ConstraintGraphBuilder.h" // TODO for Scope2; move to separate header +#include "Luau/TxnLog.h" +#include "Luau/Substitution.h" #include "Luau/VisitTypeVar.h" #include "Luau/ConstraintGraphBuilder.h" // TODO for Scope2; move to separate header LUAU_FASTFLAG(LuauAlwaysQuantify); LUAU_FASTFLAG(DebugLuauDeferredConstraintResolution); +LUAU_FASTFLAGVARIABLE(LuauQuantifyConstrained, false) namespace Luau { +/// @return true if outer encloses inner +static bool subsumes(Scope2* outer, Scope2* inner) +{ + while (inner) + { + if (inner == outer) + return true; + inner = inner->parent; + } + + return false; +} + struct Quantifier final : TypeVarOnceVisitor { TypeLevel level; @@ -62,6 +79,34 @@ struct Quantifier final : TypeVarOnceVisitor return false; } + bool visit(TypeId ty, const ConstrainedTypeVar&) override + { + if (FFlag::LuauQuantifyConstrained) + { + ConstrainedTypeVar* ctv = getMutable(ty); + + seenMutableType = true; + + if (FFlag::DebugLuauDeferredConstraintResolution ? !subsumes(scope, ctv->scope) : !level.subsumes(ctv->level)) + return false; + + std::vector opts = std::move(ctv->parts); + + // We might transmute, so it's not safe to rely on the builtin traversal logic + for (TypeId opt : opts) + traverse(opt); + + if (opts.size() == 1) + *asMutable(ty) = BoundTypeVar{opts[0]}; + else + *asMutable(ty) = UnionTypeVar{std::move(opts)}; + + return false; + } + else + return true; + } + bool visit(TypeId ty, const TableTypeVar&) override { LUAU_ASSERT(getMutable(ty)); @@ -73,8 +118,12 @@ struct Quantifier final : TypeVarOnceVisitor if (ttv.state == TableState::Free) seenMutableType = true; - if (ttv.state == TableState::Sealed || ttv.state == TableState::Generic) - return false; + if (!FFlag::LuauQuantifyConstrained) + { + if (ttv.state == TableState::Sealed || ttv.state == TableState::Generic) + return false; + } + if (FFlag::DebugLuauDeferredConstraintResolution ? !subsumes(scope, ttv.scope) : !level.subsumes(ttv.level)) { if (ttv.state == TableState::Unsealed) @@ -156,4 +205,104 @@ void quantify(TypeId ty, Scope2* scope) ftv->generalized = true; } +struct PureQuantifier : Substitution +{ + Scope2* scope; + std::vector insertedGenerics; + std::vector insertedGenericPacks; + + PureQuantifier(const TxnLog* log, TypeArena* arena, Scope2* scope) + : Substitution(log, arena) + , scope(scope) + { + } + + bool isDirty(TypeId ty) override + { + LUAU_ASSERT(ty == follow(ty)); + + if (auto ftv = get(ty)) + { + return subsumes(scope, ftv->scope); + } + else if (auto ttv = get(ty)) + { + return ttv->state == TableState::Free && subsumes(scope, ttv->scope); + } + + return false; + } + + bool isDirty(TypePackId tp) override + { + if (auto ftp = get(tp)) + { + return subsumes(scope, ftp->scope); + } + + return false; + } + + TypeId clean(TypeId ty) override + { + if (auto ftv = get(ty)) + { + TypeId result = arena->addType(GenericTypeVar{}); + insertedGenerics.push_back(result); + return result; + } + else if (auto ttv = get(ty)) + { + TypeId result = arena->addType(TableTypeVar{}); + TableTypeVar* resultTable = getMutable(result); + LUAU_ASSERT(resultTable); + + *resultTable = *ttv; + resultTable->scope = nullptr; + resultTable->state = TableState::Generic; + + return result; + } + + return ty; + } + + TypePackId clean(TypePackId tp) override + { + if (auto ftp = get(tp)) + { + TypePackId result = arena->addTypePack(TypePackVar{GenericTypePack{}}); + insertedGenericPacks.push_back(result); + return result; + } + + return tp; + } + + bool ignoreChildren(TypeId ty) override + { + return ty->persistent; + } + bool ignoreChildren(TypePackId ty) override + { + return ty->persistent; + } +}; + +TypeId quantify(TypeArena* arena, TypeId ty, Scope2* scope) +{ + PureQuantifier quantifier{TxnLog::empty(), arena, scope}; + std::optional result = quantifier.substitute(ty); + LUAU_ASSERT(result); + + FunctionTypeVar* ftv = getMutable(*result); + LUAU_ASSERT(ftv); + ftv->generics.insert(ftv->generics.end(), quantifier.insertedGenerics.begin(), quantifier.insertedGenerics.end()); + ftv->genericPacks.insert(ftv->genericPacks.end(), quantifier.insertedGenericPacks.begin(), quantifier.insertedGenericPacks.end()); + + // TODO: Set hasNoGenerics. + + return *result; +} + } // namespace Luau diff --git a/Analysis/src/RequireTracer.cpp b/Analysis/src/RequireTracer.cpp index 8ed245fb..c036a7a5 100644 --- a/Analysis/src/RequireTracer.cpp +++ b/Analysis/src/RequireTracer.cpp @@ -28,7 +28,7 @@ struct RequireTracer : AstVisitor AstExprGlobal* global = expr->func->as(); if (global && global->name == "require" && expr->args.size >= 1) - requires.push_back(expr); + requireCalls.push_back(expr); return true; } @@ -84,9 +84,9 @@ struct RequireTracer : AstVisitor ModuleInfo moduleContext{currentModuleName}; // seed worklist with require arguments - work.reserve(requires.size()); + work.reserve(requireCalls.size()); - for (AstExprCall* require : requires) + for (AstExprCall* require : requireCalls) work.push_back(require->args.data[0]); // push all dependent expressions to the work stack; note that the vector is modified during traversal @@ -125,15 +125,15 @@ struct RequireTracer : AstVisitor } // resolve all requires according to their argument - result.requires.reserve(requires.size()); + result.requireList.reserve(requireCalls.size()); - for (AstExprCall* require : requires) + for (AstExprCall* require : requireCalls) { AstExpr* arg = require->args.data[0]; if (const ModuleInfo* info = result.exprs.find(arg)) { - result.requires.push_back({info->name, require->location}); + result.requireList.push_back({info->name, require->location}); ModuleInfo infoCopy = *info; // copy *info out since next line invalidates info! result.exprs[require] = std::move(infoCopy); @@ -151,7 +151,7 @@ struct RequireTracer : AstVisitor DenseHashMap locals; std::vector work; - std::vector requires; + std::vector requireCalls; }; RequireTraceResult traceRequires(FileResolver* fileResolver, AstStatBlock* root, const ModuleName& currentModuleName) diff --git a/Analysis/src/Substitution.cpp b/Analysis/src/Substitution.cpp index 5a22deeb..9c4ce829 100644 --- a/Analysis/src/Substitution.cpp +++ b/Analysis/src/Substitution.cpp @@ -27,7 +27,7 @@ void Tarjan::visitChildren(TypeId ty, int index) if (const FunctionTypeVar* ftv = get(ty)) { visitChild(ftv->argTypes); - visitChild(ftv->retType); + visitChild(ftv->retTypes); } else if (const TableTypeVar* ttv = get(ty)) { @@ -442,7 +442,7 @@ void Substitution::replaceChildren(TypeId ty) if (FunctionTypeVar* ftv = getMutable(ty)) { ftv->argTypes = replace(ftv->argTypes); - ftv->retType = replace(ftv->retType); + ftv->retTypes = replace(ftv->retTypes); } else if (TableTypeVar* ttv = getMutable(ty)) { diff --git a/Analysis/src/ToDot.cpp b/Analysis/src/ToDot.cpp index 9b396c80..6b677bb8 100644 --- a/Analysis/src/ToDot.cpp +++ b/Analysis/src/ToDot.cpp @@ -154,7 +154,7 @@ void StateDot::visitChildren(TypeId ty, int index) finishNode(); visitChild(ftv->argTypes, index, "arg"); - visitChild(ftv->retType, index, "ret"); + visitChild(ftv->retTypes, index, "ret"); } else if (const TableTypeVar* ttv = get(ty)) { diff --git a/Analysis/src/ToString.cpp b/Analysis/src/ToString.cpp index 8490350d..eee0deee 100644 --- a/Analysis/src/ToString.cpp +++ b/Analysis/src/ToString.cpp @@ -18,6 +18,7 @@ LUAU_FASTFLAG(LuauLowerBoundsCalculation) * Fair warning: Setting this will break a lot of Luau unit tests. */ LUAU_FASTFLAGVARIABLE(DebugLuauVerboseTypeNames, false) +LUAU_FASTFLAGVARIABLE(LuauToStringTableBracesNewlines, false) namespace Luau { @@ -225,6 +226,11 @@ struct StringifierState result.name += s; } + void emit(int i) + { + emit(std::to_string(i).c_str()); + } + void indent() { indentation += 4; @@ -392,6 +398,13 @@ struct TypeVarStringifier state.emit("]]"); } + void operator()(TypeId, const BlockedTypeVar& btv) + { + state.emit("*blocked-"); + state.emit(btv.index); + state.emit("*"); + } + void operator()(TypeId, const PrimitiveTypeVar& ptv) { switch (ptv.type) @@ -478,8 +491,8 @@ struct TypeVarStringifier if (FFlag::LuauLowerBoundsCalculation) { - auto retBegin = begin(ftv.retType); - auto retEnd = end(ftv.retType); + auto retBegin = begin(ftv.retTypes); + auto retEnd = end(ftv.retTypes); if (retBegin != retEnd) { ++retBegin; @@ -489,7 +502,7 @@ struct TypeVarStringifier } else { - if (auto retPack = get(follow(ftv.retType))) + if (auto retPack = get(follow(ftv.retTypes))) { if (retPack->head.size() == 1 && !retPack->tail) plural = false; @@ -499,7 +512,7 @@ struct TypeVarStringifier if (plural) state.emit("("); - stringify(ftv.retType); + stringify(ftv.retTypes); if (plural) state.emit(")"); @@ -557,22 +570,54 @@ struct TypeVarStringifier { case TableState::Sealed: state.result.invalid = true; - openbrace = "{| "; - closedbrace = " |}"; + if (FFlag::LuauToStringTableBracesNewlines) + { + openbrace = "{|"; + closedbrace = "|}"; + } + else + { + openbrace = "{| "; + closedbrace = " |}"; + } break; case TableState::Unsealed: - openbrace = "{ "; - closedbrace = " }"; + if (FFlag::LuauToStringTableBracesNewlines) + { + openbrace = "{"; + closedbrace = "}"; + } + else + { + openbrace = "{ "; + closedbrace = " }"; + } break; case TableState::Free: state.result.invalid = true; - openbrace = "{- "; - closedbrace = " -}"; + if (FFlag::LuauToStringTableBracesNewlines) + { + openbrace = "{-"; + closedbrace = "-}"; + } + else + { + openbrace = "{- "; + closedbrace = " -}"; + } break; case TableState::Generic: state.result.invalid = true; - openbrace = "{+ "; - closedbrace = " +}"; + if (FFlag::LuauToStringTableBracesNewlines) + { + openbrace = "{+"; + closedbrace = "+}"; + } + else + { + openbrace = "{+ "; + closedbrace = " +}"; + } break; } @@ -591,6 +636,8 @@ struct TypeVarStringifier bool comma = false; if (ttv.indexer) { + if (FFlag::LuauToStringTableBracesNewlines) + state.newline(); state.emit("["); stringify(ttv.indexer->indexType); state.emit("]: "); @@ -607,6 +654,10 @@ struct TypeVarStringifier state.emit(","); state.newline(); } + else if (FFlag::LuauToStringTableBracesNewlines) + { + state.newline(); + } size_t length = state.result.name.length() - oldLength; @@ -633,6 +684,13 @@ struct TypeVarStringifier } state.dedent(); + if (FFlag::LuauToStringTableBracesNewlines) + { + if (comma) + state.newline(); + else + state.emit(" "); + } state.emit(closedbrace); state.unsee(&ttv); @@ -1247,14 +1305,14 @@ std::string toStringNamedFunction(const std::string& funcName, const FunctionTyp state.emit("): "); - size_t retSize = size(ftv.retType); - bool hasTail = !finite(ftv.retType); - bool wrap = get(follow(ftv.retType)) && (hasTail ? retSize != 0 : retSize != 1); + size_t retSize = size(ftv.retTypes); + bool hasTail = !finite(ftv.retTypes); + bool wrap = get(follow(ftv.retTypes)) && (hasTail ? retSize != 0 : retSize != 1); if (wrap) state.emit("("); - tvs.stringify(ftv.retType); + tvs.stringify(ftv.retTypes); if (wrap) state.emit(")"); @@ -1329,9 +1387,9 @@ std::string toString(const Constraint& c, ToStringOptions& opts) } else if (const GeneralizationConstraint* gc = Luau::get_if(&c.c)) { - ToStringResult subStr = toStringDetailed(gc->subType, opts); + ToStringResult subStr = toStringDetailed(gc->generalizedType, opts); opts.nameMap = std::move(subStr.nameMap); - ToStringResult superStr = toStringDetailed(gc->superType, opts); + ToStringResult superStr = toStringDetailed(gc->sourceType, opts); opts.nameMap = std::move(superStr.nameMap); return subStr.name + " ~ gen " + superStr.name; } diff --git a/Analysis/src/TypeAttach.cpp b/Analysis/src/TypeAttach.cpp index 0f4534b7..6cca7127 100644 --- a/Analysis/src/TypeAttach.cpp +++ b/Analysis/src/TypeAttach.cpp @@ -94,6 +94,11 @@ public: } } + AstType* operator()(const BlockedTypeVar& btv) + { + return allocator->alloc(Location(), std::nullopt, AstName("*blocked*")); + } + AstType* operator()(const ConstrainedTypeVar& ctv) { AstArray types; @@ -271,7 +276,7 @@ public: } AstArray returnTypes; - const auto& [retVector, retTail] = flatten(ftv.retType); + const auto& [retVector, retTail] = flatten(ftv.retTypes); returnTypes.size = retVector.size(); returnTypes.data = static_cast(allocator->allocate(sizeof(AstType*) * returnTypes.size)); for (size_t i = 0; i < returnTypes.size; ++i) diff --git a/Analysis/src/TypeChecker2.cpp b/Analysis/src/TypeChecker2.cpp new file mode 100644 index 00000000..7f5ba683 --- /dev/null +++ b/Analysis/src/TypeChecker2.cpp @@ -0,0 +1,160 @@ + +#include "Luau/TypeChecker2.h" + +#include + +#include "Luau/Ast.h" +#include "Luau/AstQuery.h" +#include "Luau/Clone.h" +#include "Luau/Normalize.h" + +namespace Luau +{ + +struct TypeChecker2 : public AstVisitor +{ + const SourceModule* sourceModule; + Module* module; + InternalErrorReporter ice; // FIXME accept a pointer from Frontend + + TypeChecker2(const SourceModule* sourceModule, Module* module) + : sourceModule(sourceModule) + , module(module) + { + } + + using AstVisitor::visit; + + TypePackId lookupPack(AstExpr* expr) + { + TypePackId* tp = module->astTypePacks.find(expr); + LUAU_ASSERT(tp); + return follow(*tp); + } + + TypeId lookupType(AstExpr* expr) + { + TypeId* ty = module->astTypes.find(expr); + LUAU_ASSERT(ty); + return follow(*ty); + } + + bool visit(AstStatAssign* assign) override + { + size_t count = std::min(assign->vars.size, assign->values.size); + + for (size_t i = 0; i < count; ++i) + { + AstExpr* lhs = assign->vars.data[i]; + TypeId* lhsType = module->astTypes.find(lhs); + LUAU_ASSERT(lhsType); + + AstExpr* rhs = assign->values.data[i]; + TypeId* rhsType = module->astTypes.find(rhs); + LUAU_ASSERT(rhsType); + + if (!isSubtype(*rhsType, *lhsType, ice)) + { + reportError(TypeMismatch{*lhsType, *rhsType}, rhs->location); + } + } + + return true; + } + + bool visit(AstExprCall* call) override + { + TypePackId expectedRetType = lookupPack(call); + TypeId functionType = lookupType(call->func); + + TypeArena arena; + TypePack args; + for (const auto& arg : call->args) + { + TypeId argTy = module->astTypes[arg]; + LUAU_ASSERT(argTy); + args.head.push_back(argTy); + } + + TypePackId argsTp = arena.addTypePack(args); + FunctionTypeVar ftv{argsTp, expectedRetType}; + TypeId expectedType = arena.addType(ftv); + if (!isSubtype(expectedType, functionType, ice)) + { + unfreeze(module->interfaceTypes); + CloneState cloneState; + expectedType = clone(expectedType, module->interfaceTypes, cloneState); + freeze(module->interfaceTypes); + reportError(TypeMismatch{expectedType, functionType}, call->location); + } + + return true; + } + + bool visit(AstExprIndexName* indexName) override + { + TypeId leftType = lookupType(indexName->expr); + TypeId resultType = lookupType(indexName); + + // leftType must have a property called indexName->index + + if (auto ttv = get(leftType)) + { + auto it = ttv->props.find(indexName->index.value); + if (it == ttv->props.end()) + { + reportError(UnknownProperty{leftType, indexName->index.value}, indexName->location); + } + else if (!isSubtype(resultType, it->second.type, ice)) + { + reportError(TypeMismatch{resultType, it->second.type}, indexName->location); + } + } + else + { + reportError(UnknownProperty{leftType, indexName->index.value}, indexName->location); + } + + return true; + } + + bool visit(AstExprConstantNumber* number) override + { + TypeId actualType = lookupType(number); + TypeId numberType = getSingletonTypes().numberType; + + if (!isSubtype(actualType, numberType, ice)) + { + reportError(TypeMismatch{actualType, numberType}, number->location); + } + + return true; + } + + bool visit(AstExprConstantString* string) override + { + TypeId actualType = lookupType(string); + TypeId stringType = getSingletonTypes().stringType; + + if (!isSubtype(actualType, stringType, ice)) + { + reportError(TypeMismatch{actualType, stringType}, string->location); + } + + return true; + } + + void reportError(TypeErrorData&& data, const Location& location) + { + module->errors.emplace_back(location, sourceModule->name, std::move(data)); + } +}; + +void check(const SourceModule& sourceModule, Module* module) +{ + TypeChecker2 typeChecker{&sourceModule, module}; + + sourceModule.root->visit(&typeChecker); +} + +} // namespace Luau diff --git a/Analysis/src/TypeInfer.cpp b/Analysis/src/TypeInfer.cpp index 447cd029..fd1b3b85 100644 --- a/Analysis/src/TypeInfer.cpp +++ b/Analysis/src/TypeInfer.cpp @@ -18,6 +18,7 @@ #include "Luau/TypePack.h" #include "Luau/TypeUtils.h" #include "Luau/TypeVar.h" +#include "Luau/VisitTypeVar.h" #include #include @@ -30,7 +31,6 @@ LUAU_FASTINTVARIABLE(LuauCheckRecursionLimit, 300) LUAU_FASTINTVARIABLE(LuauVisitRecursionLimit, 500) LUAU_FASTFLAG(LuauKnowsTheDataModel3) LUAU_FASTFLAG(LuauAutocompleteDynamicLimits) -LUAU_FASTFLAGVARIABLE(LuauExpectedPropTypeFromIndexer, false) LUAU_FASTFLAGVARIABLE(LuauLowerBoundsCalculation, false) LUAU_FASTFLAGVARIABLE(DebugLuauFreezeDuringUnification, false) LUAU_FASTFLAGVARIABLE(LuauSelfCallAutocompleteFix2, false) @@ -42,9 +42,9 @@ LUAU_FASTFLAG(LuauNormalizeFlagIsConservative) LUAU_FASTFLAGVARIABLE(LuauReturnTypeInferenceInNonstrict, false) LUAU_FASTFLAGVARIABLE(LuauRecursionLimitException, false); LUAU_FASTFLAGVARIABLE(LuauApplyTypeFunctionFix, false); -LUAU_FASTFLAGVARIABLE(LuauSuccessTypingForEqualityOperations, false) LUAU_FASTFLAGVARIABLE(LuauAlwaysQuantify, false); LUAU_FASTFLAGVARIABLE(LuauReportErrorsOnIndexerKeyMismatch, false) +LUAU_FASTFLAG(LuauQuantifyConstrained) LUAU_FASTFLAGVARIABLE(LuauFalsyPredicateReturnsNilInstead, false) LUAU_FASTFLAGVARIABLE(LuauNonCopyableTypeVarFields, false) @@ -260,7 +260,6 @@ TypeChecker::TypeChecker(ModuleResolver* resolver, InternalErrorReporter* iceHan , booleanType(getSingletonTypes().booleanType) , threadType(getSingletonTypes().threadType) , anyType(getSingletonTypes().anyType) - , optionalNumberType(getSingletonTypes().optionalNumberType) , anyTypePack(getSingletonTypes().anyTypePack) , duplicateTypeAliases{{false, {}}} { @@ -679,7 +678,7 @@ static std::optional tryGetTypeGuardPredicate(const AstExprBinary& ex void TypeChecker::check(const ScopePtr& scope, const AstStatIf& statement) { - ExprResult result = checkExpr(scope, *statement.condition); + WithPredicate result = checkExpr(scope, *statement.condition); ScopePtr ifScope = childScope(scope, statement.thenbody->location); resolve(result.predicates, ifScope, true); @@ -712,7 +711,7 @@ ErrorVec TypeChecker::canUnify(TypePackId subTy, TypePackId superTy, const Locat void TypeChecker::check(const ScopePtr& scope, const AstStatWhile& statement) { - ExprResult result = checkExpr(scope, *statement.condition); + WithPredicate result = checkExpr(scope, *statement.condition); ScopePtr whileScope = childScope(scope, statement.body->location); resolve(result.predicates, whileScope, true); @@ -728,16 +727,64 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatRepeat& statement) checkExpr(repScope, *statement.condition); } -void TypeChecker::unifyLowerBound(TypePackId subTy, TypePackId superTy, const Location& location) +void TypeChecker::unifyLowerBound(TypePackId subTy, TypePackId superTy, TypeLevel demotedLevel, const Location& location) { Unifier state = mkUnifier(location); - state.unifyLowerBound(subTy, superTy); + state.unifyLowerBound(subTy, superTy, demotedLevel); state.log.commit(); reportErrors(state.errors); } +struct Demoter : Substitution +{ + Demoter(TypeArena* arena) + : Substitution(TxnLog::empty(), arena) + { + } + + bool isDirty(TypeId ty) override + { + return get(ty); + } + + bool isDirty(TypePackId tp) override + { + return get(tp); + } + + TypeId clean(TypeId ty) override + { + auto ftv = get(ty); + LUAU_ASSERT(ftv); + return addType(FreeTypeVar{demotedLevel(ftv->level)}); + } + + TypePackId clean(TypePackId tp) override + { + auto ftp = get(tp); + LUAU_ASSERT(ftp); + return addTypePack(TypePackVar{FreeTypePack{demotedLevel(ftp->level)}}); + } + + TypeLevel demotedLevel(TypeLevel level) + { + return TypeLevel{level.level + 5000, level.subLevel}; + } + + void demote(std::vector>& expectedTypes) + { + if (!FFlag::LuauQuantifyConstrained) + return; + for (std::optional& ty : expectedTypes) + { + if (ty) + ty = substitute(*ty); + } + } +}; + void TypeChecker::check(const ScopePtr& scope, const AstStatReturn& return_) { std::vector> expectedTypes; @@ -760,11 +807,14 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatReturn& return_) } } + Demoter demoter{¤tModule->internalTypes}; + demoter.demote(expectedTypes); + TypePackId retPack = checkExprList(scope, return_.location, return_.list, false, {}, expectedTypes).type; if (FFlag::LuauReturnTypeInferenceInNonstrict ? FFlag::LuauLowerBoundsCalculation : useConstrainedIntersections()) { - unifyLowerBound(retPack, scope->returnType, return_.location); + unifyLowerBound(retPack, scope->returnType, demoter.demotedLevel(scope->level), return_.location); return; } @@ -1230,7 +1280,7 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatForIn& forin) unify(retPack, varPack, forin.location); } else - unify(iterFunc->retType, varPack, forin.location); + unify(iterFunc->retTypes, varPack, forin.location); check(loopScope, *forin.body); } @@ -1611,7 +1661,7 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatDeclareFunction& glo currentModule->getModuleScope()->bindings[global.name] = Binding{fnType, global.location}; } -ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExpr& expr, std::optional expectedType, bool forceSingleton) +WithPredicate TypeChecker::checkExpr(const ScopePtr& scope, const AstExpr& expr, std::optional expectedType, bool forceSingleton) { RecursionCounter _rc(&checkRecursionCount); if (FInt::LuauCheckRecursionLimit > 0 && checkRecursionCount >= FInt::LuauCheckRecursionLimit) @@ -1620,7 +1670,7 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExpr& return {errorRecoveryType(scope)}; } - ExprResult result; + WithPredicate result; if (auto a = expr.as()) result = checkExpr(scope, *a->expr, expectedType); @@ -1682,7 +1732,7 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExpr& return result; } -ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprLocal& expr) +WithPredicate TypeChecker::checkExpr(const ScopePtr& scope, const AstExprLocal& expr) { std::optional lvalue = tryGetLValue(expr); LUAU_ASSERT(lvalue); // Guaranteed to not be nullopt - AstExprLocal is an LValue. @@ -1696,7 +1746,7 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprLo return {errorRecoveryType(scope)}; } -ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprGlobal& expr) +WithPredicate TypeChecker::checkExpr(const ScopePtr& scope, const AstExprGlobal& expr) { std::optional lvalue = tryGetLValue(expr); LUAU_ASSERT(lvalue); // Guaranteed to not be nullopt - AstExprGlobal is an LValue. @@ -1708,7 +1758,7 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprGl return {errorRecoveryType(scope)}; } -ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprVarargs& expr) +WithPredicate TypeChecker::checkExpr(const ScopePtr& scope, const AstExprVarargs& expr) { TypePackId varargPack = checkExprPack(scope, expr).type; @@ -1738,9 +1788,9 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprVa ice("Unknown TypePack type in checkExpr(AstExprVarargs)!"); } -ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprCall& expr) +WithPredicate TypeChecker::checkExpr(const ScopePtr& scope, const AstExprCall& expr) { - ExprResult result = checkExprPack(scope, expr); + WithPredicate result = checkExprPack(scope, expr); TypePackId retPack = follow(result.type); if (auto pack = get(retPack)) @@ -1770,7 +1820,7 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprCa ice("Unknown TypePack type!", expr.location); } -ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprIndexName& expr) +WithPredicate TypeChecker::checkExpr(const ScopePtr& scope, const AstExprIndexName& expr) { Name name = expr.index.value; @@ -2031,7 +2081,7 @@ TypeId TypeChecker::stripFromNilAndReport(TypeId ty, const Location& location) return ty; } -ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprIndexExpr& expr) +WithPredicate TypeChecker::checkExpr(const ScopePtr& scope, const AstExprIndexExpr& expr) { TypeId ty = checkLValue(scope, expr); @@ -2042,7 +2092,7 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprIn return {ty}; } -ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprFunction& expr, std::optional expectedType) +WithPredicate TypeChecker::checkExpr(const ScopePtr& scope, const AstExprFunction& expr, std::optional expectedType) { auto [funTy, funScope] = checkFunctionSignature(scope, 0, expr, std::nullopt, expectedType); @@ -2108,8 +2158,7 @@ TypeId TypeChecker::checkExprTable( if (errors.empty()) exprType = expectedProp.type; } - else if (expectedTable->indexer && (FFlag::LuauExpectedPropTypeFromIndexer ? maybeString(expectedTable->indexer->indexType) - : isString(expectedTable->indexer->indexType))) + else if (expectedTable->indexer && maybeString(expectedTable->indexer->indexType)) { ErrorVec errors = tryUnify(exprType, expectedTable->indexer->indexResultType, k->location); if (errors.empty()) @@ -2147,7 +2196,7 @@ TypeId TypeChecker::checkExprTable( return addType(table); } -ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprTable& expr, std::optional expectedType) +WithPredicate TypeChecker::checkExpr(const ScopePtr& scope, const AstExprTable& expr, std::optional expectedType) { RecursionCounter _rc(&checkRecursionCount); if (FInt::LuauCheckRecursionLimit > 0 && checkRecursionCount >= FInt::LuauCheckRecursionLimit) @@ -2201,7 +2250,7 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprTa { if (auto prop = expectedTable->props.find(key->value.data); prop != expectedTable->props.end()) expectedResultType = prop->second.type; - else if (FFlag::LuauExpectedPropTypeFromIndexer && expectedIndexType && maybeString(*expectedIndexType)) + else if (expectedIndexType && maybeString(*expectedIndexType)) expectedResultType = expectedIndexResultType; } else if (expectedUnion) @@ -2236,9 +2285,9 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprTa return {checkExprTable(scope, expr, fieldTypes, expectedType)}; } -ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprUnary& expr) +WithPredicate TypeChecker::checkExpr(const ScopePtr& scope, const AstExprUnary& expr) { - ExprResult result = checkExpr(scope, *expr.expr); + WithPredicate result = checkExpr(scope, *expr.expr); TypeId operandType = follow(result.type); switch (expr.op) @@ -2466,62 +2515,50 @@ TypeId TypeChecker::checkRelationalOperation( std::optional leftMetatable = isString(lhsType) ? std::nullopt : getMetatable(follow(lhsType)); std::optional rightMetatable = isString(rhsType) ? std::nullopt : getMetatable(follow(rhsType)); - if (FFlag::LuauSuccessTypingForEqualityOperations) + if (leftMetatable != rightMetatable) { - if (leftMetatable != rightMetatable) + bool matches = false; + if (isEquality) { - bool matches = false; - if (isEquality) + if (const UnionTypeVar* utv = get(leftType); utv && rightMetatable) { - if (const UnionTypeVar* utv = get(leftType); utv && rightMetatable) + for (TypeId leftOption : utv) { - for (TypeId leftOption : utv) + if (getMetatable(follow(leftOption)) == rightMetatable) { - if (getMetatable(follow(leftOption)) == rightMetatable) + matches = true; + break; + } + } + } + + if (!matches) + { + if (const UnionTypeVar* utv = get(rhsType); utv && leftMetatable) + { + for (TypeId rightOption : utv) + { + if (getMetatable(follow(rightOption)) == leftMetatable) { matches = true; break; } } } - - if (!matches) - { - if (const UnionTypeVar* utv = get(rhsType); utv && leftMetatable) - { - for (TypeId rightOption : utv) - { - if (getMetatable(follow(rightOption)) == leftMetatable) - { - matches = true; - break; - } - } - } - } - } - - - if (!matches) - { - reportError( - expr.location, GenericError{format("Types %s and %s cannot be compared with %s because they do not have the same metatable", - toString(lhsType).c_str(), toString(rhsType).c_str(), toString(expr.op).c_str())}); - return errorRecoveryType(booleanType); } } - } - else - { - if (bool(leftMetatable) != bool(rightMetatable) && leftMetatable != rightMetatable) + + + if (!matches) { reportError( expr.location, GenericError{format("Types %s and %s cannot be compared with %s because they do not have the same metatable", - toString(lhsType).c_str(), toString(rhsType).c_str(), toString(expr.op).c_str())}); + toString(lhsType).c_str(), toString(rhsType).c_str(), toString(expr.op).c_str())}); return errorRecoveryType(booleanType); } } + if (leftMetatable) { std::optional metamethod = findMetatableEntry(lhsType, metamethodName, expr.location); @@ -2532,7 +2569,7 @@ TypeId TypeChecker::checkRelationalOperation( if (isEquality) { Unifier state = mkUnifier(expr.location); - state.tryUnify(addTypePack({booleanType}), ftv->retType); + state.tryUnify(addTypePack({booleanType}), ftv->retTypes); if (!state.errors.empty()) { @@ -2721,7 +2758,7 @@ TypeId TypeChecker::checkBinaryOperation( } } -ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprBinary& expr) +WithPredicate TypeChecker::checkExpr(const ScopePtr& scope, const AstExprBinary& expr) { if (expr.op == AstExprBinary::And) { @@ -2752,8 +2789,8 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprBi if (auto predicate = tryGetTypeGuardPredicate(expr)) return {booleanType, {std::move(*predicate)}}; - ExprResult lhs = checkExpr(scope, *expr.left, std::nullopt, /*forceSingleton=*/true); - ExprResult rhs = checkExpr(scope, *expr.right, std::nullopt, /*forceSingleton=*/true); + WithPredicate lhs = checkExpr(scope, *expr.left, std::nullopt, /*forceSingleton=*/true); + WithPredicate rhs = checkExpr(scope, *expr.right, std::nullopt, /*forceSingleton=*/true); PredicateVec predicates; @@ -2770,18 +2807,18 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprBi } else { - ExprResult lhs = checkExpr(scope, *expr.left); - ExprResult rhs = checkExpr(scope, *expr.right); + WithPredicate lhs = checkExpr(scope, *expr.left); + WithPredicate rhs = checkExpr(scope, *expr.right); // Intentionally discarding predicates with other operators. return {checkBinaryOperation(scope, expr, lhs.type, rhs.type, lhs.predicates)}; } } -ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprTypeAssertion& expr) +WithPredicate TypeChecker::checkExpr(const ScopePtr& scope, const AstExprTypeAssertion& expr) { TypeId annotationType = resolveType(scope, *expr.annotation); - ExprResult result = checkExpr(scope, *expr.expr, annotationType); + WithPredicate result = checkExpr(scope, *expr.expr, annotationType); // Note: As an optimization, we try 'number <: number | string' first, as that is the more likely case. if (canUnify(annotationType, result.type, expr.location).empty()) @@ -2794,7 +2831,7 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprTy return {errorRecoveryType(annotationType), std::move(result.predicates)}; } -ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprError& expr) +WithPredicate TypeChecker::checkExpr(const ScopePtr& scope, const AstExprError& expr) { const size_t oldSize = currentModule->errors.size(); @@ -2808,17 +2845,17 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprEr return {errorRecoveryType(scope)}; } -ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprIfElse& expr, std::optional expectedType) +WithPredicate TypeChecker::checkExpr(const ScopePtr& scope, const AstExprIfElse& expr, std::optional expectedType) { - ExprResult result = checkExpr(scope, *expr.condition); + WithPredicate result = checkExpr(scope, *expr.condition); ScopePtr trueScope = childScope(scope, expr.trueExpr->location); resolve(result.predicates, trueScope, true); - ExprResult trueType = checkExpr(trueScope, *expr.trueExpr, expectedType); + WithPredicate trueType = checkExpr(trueScope, *expr.trueExpr, expectedType); ScopePtr falseScope = childScope(scope, expr.falseExpr->location); resolve(result.predicates, falseScope, false); - ExprResult falseType = checkExpr(falseScope, *expr.falseExpr, expectedType); + WithPredicate falseType = checkExpr(falseScope, *expr.falseExpr, expectedType); if (falseType.type == trueType.type) return {trueType.type}; @@ -3170,7 +3207,7 @@ std::pair TypeChecker::checkFunctionSignature( retPack = anyTypePack; else if (expectedFunctionType) { - auto [head, tail] = flatten(expectedFunctionType->retType); + auto [head, tail] = flatten(expectedFunctionType->retTypes); // Do not infer 'nil' as function return type if (!tail && head.size() == 1 && isNil(head[0])) @@ -3354,7 +3391,7 @@ void TypeChecker::checkFunctionBody(const ScopePtr& scope, TypeId ty, const AstE if (useConstrainedIntersections()) { - TypePackId retPack = follow(funTy->retType); + TypePackId retPack = follow(funTy->retTypes); // It is possible for a function to have no annotation and no return statement, and yet still have an ascribed return type // if it is expected to conform to some other interface. (eg the function may be a lambda passed as a callback) if (!hasReturn(function.body) && !function.returnAnnotation.has_value() && get(retPack)) @@ -3367,20 +3404,20 @@ void TypeChecker::checkFunctionBody(const ScopePtr& scope, TypeId ty, const AstE else { // We explicitly don't follow here to check if we have a 'true' free type instead of bound one - if (get_if(&funTy->retType->ty)) - *asMutable(funTy->retType) = TypePack{{}, std::nullopt}; + if (get_if(&funTy->retTypes->ty)) + *asMutable(funTy->retTypes) = TypePack{{}, std::nullopt}; } bool reachesImplicitReturn = getFallthrough(function.body) != nullptr; - if (reachesImplicitReturn && !allowsNoReturnValues(follow(funTy->retType))) + if (reachesImplicitReturn && !allowsNoReturnValues(follow(funTy->retTypes))) { // If we're in nonstrict mode we want to only report this missing return // statement if there are type annotations on the function. In strict mode // we report it regardless. if (!isNonstrictMode() || function.returnAnnotation) { - reportError(getEndLocation(function), FunctionExitsWithoutReturning{funTy->retType}); + reportError(getEndLocation(function), FunctionExitsWithoutReturning{funTy->retTypes}); } } } @@ -3388,7 +3425,7 @@ void TypeChecker::checkFunctionBody(const ScopePtr& scope, TypeId ty, const AstE ice("Checking non functional type"); } -ExprResult TypeChecker::checkExprPack(const ScopePtr& scope, const AstExpr& expr) +WithPredicate TypeChecker::checkExprPack(const ScopePtr& scope, const AstExpr& expr) { if (auto a = expr.as()) return checkExprPack(scope, *a); @@ -3654,7 +3691,7 @@ void TypeChecker::checkArgumentList( } } -ExprResult TypeChecker::checkExprPack(const ScopePtr& scope, const AstExprCall& expr) +WithPredicate TypeChecker::checkExprPack(const ScopePtr& scope, const AstExprCall& expr) { // evaluate type of function // decompose an intersection into its component overloads @@ -3722,7 +3759,7 @@ ExprResult TypeChecker::checkExprPack(const ScopePtr& scope, const A std::vector> expectedTypes = getExpectedTypesForCall(overloads, expr.args.size, expr.self); - ExprResult argListResult = checkExprList(scope, expr.location, expr.args, false, {}, expectedTypes); + WithPredicate argListResult = checkExprList(scope, expr.location, expr.args, false, {}, expectedTypes); TypePackId argPack = argListResult.type; if (get(argPack)) @@ -3766,7 +3803,7 @@ ExprResult TypeChecker::checkExprPack(const ScopePtr& scope, const A if (!overload && !overloadsThatDont.empty()) overload = get(overloadsThatDont[0]); if (overload) - return {errorRecoveryTypePack(overload->retType)}; + return {errorRecoveryTypePack(overload->retTypes)}; return {errorRecoveryTypePack(retPack)}; } @@ -3775,7 +3812,7 @@ std::vector> TypeChecker::getExpectedTypesForCall(const st { std::vector> expectedTypes; - auto assignOption = [this, &expectedTypes](size_t index, std::optional ty) { + auto assignOption = [this, &expectedTypes](size_t index, TypeId ty) { if (index == expectedTypes.size()) { expectedTypes.push_back(ty); @@ -3790,7 +3827,7 @@ std::vector> TypeChecker::getExpectedTypesForCall(const st } else { - std::vector result = reduceUnion({*el, *ty}); + std::vector result = reduceUnion({*el, ty}); el = result.size() == 1 ? result[0] : addType(UnionTypeVar{std::move(result)}); } } @@ -3810,7 +3847,8 @@ std::vector> TypeChecker::getExpectedTypesForCall(const st if (argsTail) { - if (const VariadicTypePack* vtp = get(follow(*argsTail))) + argsTail = follow(*argsTail); + if (const VariadicTypePack* vtp = get(*argsTail)) { while (index < argumentCount) assignOption(index++, vtp->ty); @@ -3819,11 +3857,14 @@ std::vector> TypeChecker::getExpectedTypesForCall(const st } } + Demoter demoter{¤tModule->internalTypes}; + demoter.demote(expectedTypes); + return expectedTypes; } -std::optional> TypeChecker::checkCallOverload(const ScopePtr& scope, const AstExprCall& expr, TypeId fn, TypePackId retPack, - TypePackId argPack, TypePack* args, const std::vector* argLocations, const ExprResult& argListResult, +std::optional> TypeChecker::checkCallOverload(const ScopePtr& scope, const AstExprCall& expr, TypeId fn, TypePackId retPack, + TypePackId argPack, TypePack* args, const std::vector* argLocations, const WithPredicate& argListResult, std::vector& overloadsThatMatchArgCount, std::vector& overloadsThatDont, std::vector& errors) { LUAU_ASSERT(argLocations); @@ -3918,14 +3959,14 @@ std::optional> TypeChecker::checkCallOverload(const Scope if (ftv->magicFunction) { // TODO: We're passing in the wrong TypePackId. Should be argPack, but a unit test fails otherwise. CLI-40458 - if (std::optional> ret = ftv->magicFunction(*this, scope, expr, argListResult)) + if (std::optional> ret = ftv->magicFunction(*this, scope, expr, argListResult)) return *ret; } Unifier state = mkUnifier(expr.location); // Unify return types - checkArgumentList(scope, state, retPack, ftv->retType, /*argLocations*/ {}); + checkArgumentList(scope, state, retPack, ftv->retTypes, /*argLocations*/ {}); if (!state.errors.empty()) { return {}; @@ -3996,7 +4037,7 @@ bool TypeChecker::handleSelfCallMismatch(const ScopePtr& scope, const AstExprCal // we eagerly assume that that's what you actually meant and we commit to it. // This could be incorrect if the function has an additional overload that // actually works. - // checkArgumentList(scope, editedState, retPack, ftv->retType, retLocations, CountMismatch::Return); + // checkArgumentList(scope, editedState, retPack, ftv->retTypes, retLocations, CountMismatch::Return); return true; } } @@ -4027,7 +4068,7 @@ bool TypeChecker::handleSelfCallMismatch(const ScopePtr& scope, const AstExprCal // we eagerly assume that that's what you actually meant and we commit to it. // This could be incorrect if the function has an additional overload that // actually works. - // checkArgumentList(scope, editedState, retPack, ftv->retType, retLocations, CountMismatch::Return); + // checkArgumentList(scope, editedState, retPack, ftv->retTypes, retLocations, CountMismatch::Return); return true; } } @@ -4085,7 +4126,7 @@ void TypeChecker::reportOverloadResolutionError(const ScopePtr& scope, const Ast // Unify return types if (const FunctionTypeVar* ftv = get(overload)) { - checkArgumentList(scope, state, retPack, ftv->retType, {}); + checkArgumentList(scope, state, retPack, ftv->retTypes, {}); checkArgumentList(scope, state, argPack, ftv->argTypes, argLocations); } @@ -4110,7 +4151,7 @@ void TypeChecker::reportOverloadResolutionError(const ScopePtr& scope, const Ast return; } -ExprResult TypeChecker::checkExprList(const ScopePtr& scope, const Location& location, const AstArray& exprs, +WithPredicate TypeChecker::checkExprList(const ScopePtr& scope, const Location& location, const AstArray& exprs, bool substituteFreeForNil, const std::vector& instantiateGenerics, const std::vector>& expectedTypes) { TypePackId pack = addTypePack(TypePack{}); @@ -4401,10 +4442,24 @@ TypeId Anyification::clean(TypeId ty) } else if (auto ctv = get(ty)) { - auto [t, ok] = normalize(ty, *arena, *iceHandler); - if (!ok) - normalizationTooComplex = true; - return t; + if (FFlag::LuauQuantifyConstrained) + { + std::vector copy = ctv->parts; + for (TypeId& ty : copy) + ty = replace(ty); + TypeId res = copy.size() == 1 ? copy[0] : addType(UnionTypeVar{std::move(copy)}); + auto [t, ok] = normalize(res, *arena, *iceHandler); + if (!ok) + normalizationTooComplex = true; + return t; + } + else + { + auto [t, ok] = normalize(ty, *arena, *iceHandler); + if (!ok) + normalizationTooComplex = true; + return t; + } } else return anyType; diff --git a/Analysis/src/TypeUtils.cpp b/Analysis/src/TypeUtils.cpp index ba09df5f..3d97e6eb 100644 --- a/Analysis/src/TypeUtils.cpp +++ b/Analysis/src/TypeUtils.cpp @@ -66,7 +66,7 @@ std::optional findTablePropertyRespectingMeta(ErrorVec& errors, TypeId t } else if (const auto& itf = get(index)) { - std::optional r = first(follow(itf->retType)); + std::optional r = first(follow(itf->retTypes)); if (!r) return getSingletonTypes().nilType; else diff --git a/Analysis/src/TypeVar.cpp b/Analysis/src/TypeVar.cpp index 33bfe254..57762937 100644 --- a/Analysis/src/TypeVar.cpp +++ b/Analysis/src/TypeVar.cpp @@ -29,8 +29,8 @@ LUAU_FASTFLAG(LuauNonCopyableTypeVarFields) namespace Luau { -std::optional> magicFunctionFormat( - TypeChecker& typechecker, const ScopePtr& scope, const AstExprCall& expr, ExprResult exprResult); +std::optional> magicFunctionFormat( + TypeChecker& typechecker, const ScopePtr& scope, const AstExprCall& expr, WithPredicate withPredicate); TypeId follow(TypeId t) { @@ -408,41 +408,48 @@ bool hasLength(TypeId ty, DenseHashSet& seen, int* recursionCount) return false; } -FunctionTypeVar::FunctionTypeVar(TypePackId argTypes, TypePackId retType, std::optional defn, bool hasSelf) +BlockedTypeVar::BlockedTypeVar() + : index(++nextIndex) +{ +} + +int BlockedTypeVar::nextIndex = 0; + +FunctionTypeVar::FunctionTypeVar(TypePackId argTypes, TypePackId retTypes, std::optional defn, bool hasSelf) : argTypes(argTypes) - , retType(retType) + , retTypes(retTypes) , definition(std::move(defn)) , hasSelf(hasSelf) { } -FunctionTypeVar::FunctionTypeVar(TypeLevel level, TypePackId argTypes, TypePackId retType, std::optional defn, bool hasSelf) +FunctionTypeVar::FunctionTypeVar(TypeLevel level, TypePackId argTypes, TypePackId retTypes, std::optional defn, bool hasSelf) : level(level) , argTypes(argTypes) - , retType(retType) + , retTypes(retTypes) , definition(std::move(defn)) , hasSelf(hasSelf) { } -FunctionTypeVar::FunctionTypeVar(std::vector generics, std::vector genericPacks, TypePackId argTypes, TypePackId retType, +FunctionTypeVar::FunctionTypeVar(std::vector generics, std::vector genericPacks, TypePackId argTypes, TypePackId retTypes, std::optional defn, bool hasSelf) : generics(generics) , genericPacks(genericPacks) , argTypes(argTypes) - , retType(retType) + , retTypes(retTypes) , definition(std::move(defn)) , hasSelf(hasSelf) { } FunctionTypeVar::FunctionTypeVar(TypeLevel level, std::vector generics, std::vector genericPacks, TypePackId argTypes, - TypePackId retType, std::optional defn, bool hasSelf) + TypePackId retTypes, std::optional defn, bool hasSelf) : level(level) , generics(generics) , genericPacks(genericPacks) , argTypes(argTypes) - , retType(retType) + , retTypes(retTypes) , definition(std::move(defn)) , hasSelf(hasSelf) { @@ -488,7 +495,7 @@ bool areEqual(SeenSet& seen, const FunctionTypeVar& lhs, const FunctionTypeVar& if (!areEqual(seen, *lhs.argTypes, *rhs.argTypes)) return false; - if (!areEqual(seen, *lhs.retType, *rhs.retType)) + if (!areEqual(seen, *lhs.retTypes, *rhs.retTypes)) return false; return true; @@ -678,7 +685,6 @@ static TypeVar trueType_{SingletonTypeVar{BooleanSingleton{true}}, /*persistent* static TypeVar falseType_{SingletonTypeVar{BooleanSingleton{false}}, /*persistent*/ true}; static TypeVar anyType_{AnyTypeVar{}, /*persistent*/ true}; static TypeVar errorType_{ErrorTypeVar{}, /*persistent*/ true}; -static TypeVar optionalNumberType_{UnionTypeVar{{&numberType_, &nilType_}}, /*persistent*/ true}; static TypePackVar anyTypePack_{VariadicTypePack{&anyType_}, true}; static TypePackVar errorTypePack_{Unifiable::Error{}}; @@ -692,7 +698,6 @@ SingletonTypes::SingletonTypes() , trueType(&trueType_) , falseType(&falseType_) , anyType(&anyType_) - , optionalNumberType(&optionalNumberType_) , anyTypePack(&anyTypePack_) , arena(new TypeArena) { @@ -825,7 +830,7 @@ void persist(TypeId ty) else if (auto ftv = get(t)) { persist(ftv->argTypes); - persist(ftv->retType); + persist(ftv->retTypes); } else if (auto ttv = get(t)) { @@ -1100,10 +1105,10 @@ static std::vector parseFormatString(TypeChecker& typechecker, const cha return result; } -std::optional> magicFunctionFormat( - TypeChecker& typechecker, const ScopePtr& scope, const AstExprCall& expr, ExprResult exprResult) +std::optional> magicFunctionFormat( + TypeChecker& typechecker, const ScopePtr& scope, const AstExprCall& expr, WithPredicate withPredicate) { - auto [paramPack, _predicates] = exprResult; + auto [paramPack, _predicates] = withPredicate; TypeArena& arena = typechecker.currentModule->internalTypes; @@ -1142,7 +1147,7 @@ std::optional> magicFunctionFormat( if (expected.size() != actualParamSize && (!tail || expected.size() < actualParamSize)) typechecker.reportError(TypeError{expr.location, CountMismatch{expected.size(), actualParamSize}}); - return ExprResult{arena.addTypePack({typechecker.stringType})}; + return WithPredicate{arena.addTypePack({typechecker.stringType})}; } std::vector filterMap(TypeId type, TypeIdPredicate predicate) diff --git a/Analysis/src/Unifier.cpp b/Analysis/src/Unifier.cpp index 414b05f4..877663de 100644 --- a/Analysis/src/Unifier.cpp +++ b/Analysis/src/Unifier.cpp @@ -22,6 +22,7 @@ LUAU_FASTFLAG(LuauLowerBoundsCalculation); LUAU_FASTFLAG(LuauErrorRecoveryType); LUAU_FASTFLAGVARIABLE(LuauSubtypingAddOptPropsToUnsealedTables, false) LUAU_FASTFLAGVARIABLE(LuauTxnLogRefreshFunctionPointers, false) +LUAU_FASTFLAG(LuauQuantifyConstrained) namespace Luau { @@ -1288,13 +1289,13 @@ void Unifier::tryUnifyFunctions(TypeId subTy, TypeId superTy, bool isFunctionCal reportError(TypeError{location, TypeMismatch{superTy, subTy, "", innerState.errors.front()}}); innerState.ctx = CountMismatch::Result; - innerState.tryUnify_(subFunction->retType, superFunction->retType); + innerState.tryUnify_(subFunction->retTypes, superFunction->retTypes); if (!reported) { if (auto e = hasUnificationTooComplex(innerState.errors)) reportError(*e); - else if (!innerState.errors.empty() && size(superFunction->retType) == 1 && finite(superFunction->retType)) + else if (!innerState.errors.empty() && size(superFunction->retTypes) == 1 && finite(superFunction->retTypes)) reportError(TypeError{location, TypeMismatch{superTy, subTy, "Return type is not compatible.", innerState.errors.front()}}); else if (!innerState.errors.empty() && innerState.firstPackErrorPos) reportError( @@ -1312,7 +1313,7 @@ void Unifier::tryUnifyFunctions(TypeId subTy, TypeId superTy, bool isFunctionCal tryUnify_(superFunction->argTypes, subFunction->argTypes, isFunctionCall); ctx = CountMismatch::Result; - tryUnify_(subFunction->retType, superFunction->retType); + tryUnify_(subFunction->retTypes, superFunction->retTypes); } if (FFlag::LuauTxnLogRefreshFunctionPointers) @@ -2177,7 +2178,7 @@ static void tryUnifyWithAny(std::vector& queue, Unifier& state, DenseHas else if (auto fun = state.log.getMutable(ty)) { queueTypePack(queue, seenTypePacks, state, fun->argTypes, anyTypePack); - queueTypePack(queue, seenTypePacks, state, fun->retType, anyTypePack); + queueTypePack(queue, seenTypePacks, state, fun->retTypes, anyTypePack); } else if (auto table = state.log.getMutable(ty)) { @@ -2322,7 +2323,7 @@ void Unifier::tryUnifyWithConstrainedSuperTypeVar(TypeId subTy, TypeId superTy) superC->parts.push_back(subTy); } -void Unifier::unifyLowerBound(TypePackId subTy, TypePackId superTy) +void Unifier::unifyLowerBound(TypePackId subTy, TypePackId superTy, TypeLevel demotedLevel) { // The duplication between this and regular typepack unification is tragic. @@ -2357,7 +2358,7 @@ void Unifier::unifyLowerBound(TypePackId subTy, TypePackId superTy) if (!freeTailPack) return; - TypeLevel level = freeTailPack->level; + TypeLevel level = FFlag::LuauQuantifyConstrained ? demotedLevel : freeTailPack->level; TypePack* tp = getMutable(log.replace(tailPack, TypePack{})); diff --git a/Compiler/src/BytecodeBuilder.cpp b/Compiler/src/BytecodeBuilder.cpp index 597b2f0a..a34f7603 100644 --- a/Compiler/src/BytecodeBuilder.cpp +++ b/Compiler/src/BytecodeBuilder.cpp @@ -1075,6 +1075,8 @@ void BytecodeBuilder::validate() const LUAU_ASSERT(i <= insns.size()); } + std::vector openCaptures; + // second pass: validate the rest of the bytecode for (size_t i = 0; i < insns.size();) { @@ -1121,6 +1123,8 @@ void BytecodeBuilder::validate() const case LOP_CLOSEUPVALS: VREG(LUAU_INSN_A(insn)); + while (openCaptures.size() && openCaptures.back() >= LUAU_INSN_A(insn)) + openCaptures.pop_back(); break; case LOP_GETIMPORT: @@ -1388,8 +1392,12 @@ void BytecodeBuilder::validate() const switch (LUAU_INSN_A(insn)) { case LCT_VAL: + VREG(LUAU_INSN_B(insn)); + break; + case LCT_REF: VREG(LUAU_INSN_B(insn)); + openCaptures.push_back(LUAU_INSN_B(insn)); break; case LCT_UPVAL: @@ -1409,6 +1417,12 @@ void BytecodeBuilder::validate() const LUAU_ASSERT(i <= insns.size()); } + // all CAPTURE REF instructions must have a CLOSEUPVALS instruction after them in the bytecode stream + // this doesn't guarantee safety as it doesn't perform basic block based analysis, but if this fails + // then the bytecode is definitely unsafe to run since the compiler won't generate backwards branches + // except for loop edges + LUAU_ASSERT(openCaptures.empty()); + #undef VREG #undef VREGEND #undef VUPVAL diff --git a/Compiler/src/Compiler.cpp b/Compiler/src/Compiler.cpp index 7431cde4..52dc9242 100644 --- a/Compiler/src/Compiler.cpp +++ b/Compiler/src/Compiler.cpp @@ -246,6 +246,14 @@ struct Compiler f.canInline = true; f.stackSize = stackSize; f.costModel = modelCost(func->body, func->args.data, func->args.size); + + // track functions that only ever return a single value so that we can convert multret calls to fixedret calls + if (allPathsEndWithReturn(func->body)) + { + ReturnVisitor returnVisitor(this); + stat->visit(&returnVisitor); + f.returnsOne = returnVisitor.returnsOne; + } } upvals.clear(); // note: instead of std::move above, we copy & clear to preserve capacity for future pushes @@ -260,6 +268,19 @@ struct Compiler { if (AstExprCall* expr = node->as()) { + // Optimization: convert multret calls to functions that always return one value to fixedret calls; this facilitates inlining + if (options.optimizationLevel >= 2) + { + AstExprFunction* func = getFunctionExpr(expr->func); + Function* fi = func ? functions.find(func) : nullptr; + + if (fi && fi->returnsOne) + { + compileExprTemp(node, target); + return false; + } + } + // We temporarily swap out regTop to have targetTop work correctly... // This is a crude hack but it's necessary for correctness :( RegScope rs(this, target); @@ -447,7 +468,9 @@ struct Compiler return false; } - // TODO: we can compile multret functions if all returns of the function are multret as well + // we can't inline multret functions because the caller expects L->top to be adjusted: + // - inlined return compiles to a JUMP, and we don't have an instruction that adjusts L->top arbitrarily + // - even if we did, right now all L->top adjustments are immediately consumed by the next instruction, and for now we want to preserve that if (multRet) { bytecode.addDebugRemark("inlining failed: can't convert fixed returns to multret"); @@ -492,7 +515,7 @@ struct Compiler size_t oldLocals = localStack.size(); // note that we push the frame early; this is needed to block recursive inline attempts - inlineFrames.push_back({func, target, targetCount}); + inlineFrames.push_back({func, oldLocals, target, targetCount}); // evaluate all arguments; note that we don't emit code for constant arguments (relying on constant folding) for (size_t i = 0; i < func->args.size; ++i) @@ -593,6 +616,8 @@ struct Compiler { for (size_t i = 0; i < targetCount; ++i) bytecode.emitABC(LOP_LOADNIL, uint8_t(target + i), 0, 0); + + closeLocals(oldLocals); } popLocals(oldLocals); @@ -2355,6 +2380,8 @@ struct Compiler compileExprListTemp(stat->list, frame.target, frame.targetCount, /* targetTop= */ false); + closeLocals(frame.localOffset); + if (!fallthrough) { size_t jumpLabel = bytecode.emitLabel(); @@ -3316,6 +3343,48 @@ struct Compiler std::vector upvals; }; + struct ReturnVisitor: AstVisitor + { + Compiler* self; + bool returnsOne = true; + + ReturnVisitor(Compiler* self) + : self(self) + { + } + + bool visit(AstExpr* expr) override + { + return false; + } + + bool visit(AstStatReturn* stat) override + { + if (stat->list.size == 1) + { + AstExpr* value = stat->list.data[0]; + + if (AstExprCall* expr = value->as()) + { + AstExprFunction* func = self->getFunctionExpr(expr->func); + Function* fi = func ? self->functions.find(func) : nullptr; + + returnsOne &= fi && fi->returnsOne; + } + else if (value->is()) + { + returnsOne = false; + } + } + else + { + returnsOne = false; + } + + return false; + } + }; + struct RegScope { RegScope(Compiler* self) @@ -3351,6 +3420,7 @@ struct Compiler uint64_t costModel = 0; unsigned int stackSize = 0; bool canInline = false; + bool returnsOne = false; }; struct Local @@ -3384,6 +3454,8 @@ struct Compiler { AstExprFunction* func; + size_t localOffset; + uint8_t target; uint8_t targetCount; diff --git a/Sources.cmake b/Sources.cmake index 99007e89..f261cba6 100644 --- a/Sources.cmake +++ b/Sources.cmake @@ -65,12 +65,13 @@ target_sources(Luau.CodeGen PRIVATE target_sources(Luau.Analysis PRIVATE Analysis/include/Luau/AstQuery.h Analysis/include/Luau/Autocomplete.h - Analysis/include/Luau/NotNull.h Analysis/include/Luau/BuiltinDefinitions.h Analysis/include/Luau/Clone.h Analysis/include/Luau/Config.h + Analysis/include/Luau/Constraint.h Analysis/include/Luau/ConstraintGraphBuilder.h Analysis/include/Luau/ConstraintSolver.h + Analysis/include/Luau/ConstraintSolverLogger.h Analysis/include/Luau/Documentation.h Analysis/include/Luau/Error.h Analysis/include/Luau/FileResolver.h @@ -97,6 +98,7 @@ target_sources(Luau.Analysis PRIVATE Analysis/include/Luau/TxnLog.h Analysis/include/Luau/TypeArena.h Analysis/include/Luau/TypeAttach.h + Analysis/include/Luau/TypeChecker2.h Analysis/include/Luau/TypedAllocator.h Analysis/include/Luau/TypeInfer.h Analysis/include/Luau/TypePack.h @@ -113,8 +115,10 @@ target_sources(Luau.Analysis PRIVATE Analysis/src/BuiltinDefinitions.cpp Analysis/src/Clone.cpp Analysis/src/Config.cpp + Analysis/src/Constraint.cpp Analysis/src/ConstraintGraphBuilder.cpp Analysis/src/ConstraintSolver.cpp + Analysis/src/ConstraintSolverLogger.cpp Analysis/src/Error.cpp Analysis/src/Frontend.cpp Analysis/src/Instantiation.cpp @@ -136,6 +140,7 @@ target_sources(Luau.Analysis PRIVATE Analysis/src/TxnLog.cpp Analysis/src/TypeArena.cpp Analysis/src/TypeAttach.cpp + Analysis/src/TypeChecker2.cpp Analysis/src/TypedAllocator.cpp Analysis/src/TypeInfer.cpp Analysis/src/TypePack.cpp @@ -245,7 +250,6 @@ if(TARGET Luau.UnitTest) tests/AstQuery.test.cpp tests/AstVisitor.test.cpp tests/Autocomplete.test.cpp - tests/NotNull.test.cpp tests/BuiltinDefinitions.test.cpp tests/Compiler.test.cpp tests/Config.test.cpp diff --git a/VM/src/lobject.h b/VM/src/lobject.h index 5e02c2ea..bdcb85cb 100644 --- a/VM/src/lobject.h +++ b/VM/src/lobject.h @@ -418,7 +418,7 @@ typedef struct Table CommonHeader; - uint8_t flags; /* 1<

flags = 0 +#define invalidateTMcache(t) t->tmcache = 0 // empty hash data points to dummynode so that we can always dereference it const LuaNode luaH_dummynode = { @@ -479,7 +479,7 @@ Table* luaH_new(lua_State* L, int narray, int nhash) Table* t = luaM_newgco(L, Table, sizeof(Table), L->activememcat); luaC_init(L, t, LUA_TTABLE); t->metatable = NULL; - t->flags = cast_byte(~0); + t->tmcache = cast_byte(~0); t->array = NULL; t->sizearray = 0; t->lastfree = 0; @@ -778,7 +778,7 @@ Table* luaH_clone(lua_State* L, Table* tt) Table* t = luaM_newgco(L, Table, sizeof(Table), L->activememcat); luaC_init(L, t, LUA_TTABLE); t->metatable = tt->metatable; - t->flags = tt->flags; + t->tmcache = tt->tmcache; t->array = NULL; t->sizearray = 0; t->lsizenode = 0; @@ -835,5 +835,5 @@ void luaH_clear(Table* tt) } /* back to empty -> no tag methods present */ - tt->flags = cast_byte(~0); + tt->tmcache = cast_byte(~0); } diff --git a/VM/src/ltm.cpp b/VM/src/ltm.cpp index 9b99506b..e7df4e53 100644 --- a/VM/src/ltm.cpp +++ b/VM/src/ltm.cpp @@ -88,8 +88,8 @@ const TValue* luaT_gettm(Table* events, TMS event, TString* ename) const TValue* tm = luaH_getstr(events, ename); LUAU_ASSERT(event <= TM_EQ); if (ttisnil(tm)) - { /* no tag method? */ - events->flags |= cast_byte(1u << event); /* cache this fact */ + { /* no tag method? */ + events->tmcache |= cast_byte(1u << event); /* cache this fact */ return NULL; } else diff --git a/VM/src/ltm.h b/VM/src/ltm.h index e1b95c21..a5223941 100644 --- a/VM/src/ltm.h +++ b/VM/src/ltm.h @@ -41,10 +41,10 @@ typedef enum } TMS; // clang-format on -#define gfasttm(g, et, e) ((et) == NULL ? NULL : ((et)->flags & (1u << (e))) ? NULL : luaT_gettm(et, e, (g)->tmname[e])) +#define gfasttm(g, et, e) ((et) == NULL ? NULL : ((et)->tmcache & (1u << (e))) ? NULL : luaT_gettm(et, e, (g)->tmname[e])) #define fasttm(l, et, e) gfasttm(l->global, et, e) -#define fastnotm(et, e) ((et) == NULL || ((et)->flags & (1u << (e)))) +#define fastnotm(et, e) ((et) == NULL || ((et)->tmcache & (1u << (e)))) LUAI_DATA const char* const luaT_typenames[]; LUAI_DATA const char* const luaT_eventname[]; diff --git a/tests/Autocomplete.test.cpp b/tests/Autocomplete.test.cpp index dea1ab19..f3b0bcad 100644 --- a/tests/Autocomplete.test.cpp +++ b/tests/Autocomplete.test.cpp @@ -1992,6 +1992,7 @@ local fp: @1= f auto ac = autocomplete('1'); + REQUIRE_EQ("({| x: number, y: number |}) -> number", toString(requireType("f"))); CHECK(ac.entryMap.count("({ x: number, y: number }) -> number")); } @@ -2620,7 +2621,6 @@ a = if temp then even elseif true then temp else e@9 TEST_CASE_FIXTURE(ACFixture, "autocomplete_if_else_regression") { - ScopedFastFlag FFlagLuauIfElseExprFixCompletionIssue("LuauIfElseExprFixCompletionIssue", true); check(R"( local abcdef = 0; local temp = false diff --git a/tests/Compiler.test.cpp b/tests/Compiler.test.cpp index 6eee254e..036bf124 100644 --- a/tests/Compiler.test.cpp +++ b/tests/Compiler.test.cpp @@ -4992,6 +4992,147 @@ RETURN R1 1 )"); } +TEST_CASE("InlineCapture") +{ + // if the argument is captured by a nested closure, normally we can rely on capture by value + CHECK_EQ("\n" + compileFunction(R"( +local function foo(a) + return function() return a end +end + +local x = ... +local y = foo(x) +return y +)", + 2, 2), + R"( +DUPCLOSURE R0 K0 +GETVARARGS R1 1 +NEWCLOSURE R2 P1 +CAPTURE VAL R1 +RETURN R2 1 +)"); + + // if the argument is a constant, we move it to a register so that capture by value can happen + CHECK_EQ("\n" + compileFunction(R"( +local function foo(a) + return function() return a end +end + +local y = foo(42) +return y +)", + 2, 2), + R"( +DUPCLOSURE R0 K0 +LOADN R2 42 +NEWCLOSURE R1 P1 +CAPTURE VAL R2 +RETURN R1 1 +)"); + + // if the argument is an externally mutated variable, we copy it to an argument and capture it by value + CHECK_EQ("\n" + compileFunction(R"( +local function foo(a) + return function() return a end +end + +local x x = 42 +local y = foo(x) +return y +)", + 2, 2), + R"( +DUPCLOSURE R0 K0 +LOADNIL R1 +LOADN R1 42 +MOVE R3 R1 +NEWCLOSURE R2 P1 +CAPTURE VAL R3 +RETURN R2 1 +)"); + + // finally, if the argument is mutated internally, we must capture it by reference and close the upvalue + CHECK_EQ("\n" + compileFunction(R"( +local function foo(a) + a = a or 42 + return function() return a end +end + +local y = foo() +return y +)", + 2, 2), + R"( +DUPCLOSURE R0 K0 +LOADNIL R2 +ORK R2 R2 K1 +NEWCLOSURE R1 P1 +CAPTURE REF R2 +CLOSEUPVALS R2 +RETURN R1 1 +)"); + + // note that capture might need to be performed during the fallthrough block + CHECK_EQ("\n" + compileFunction(R"( +local function foo(a) + a = a or 42 + print(function() return a end) +end + +local x = ... +local y = foo(x) +return y +)", + 2, 2), + R"( +DUPCLOSURE R0 K0 +GETVARARGS R1 1 +MOVE R3 R1 +ORK R3 R3 K1 +GETIMPORT R4 3 +NEWCLOSURE R5 P1 +CAPTURE REF R3 +CALL R4 1 0 +LOADNIL R2 +CLOSEUPVALS R3 +RETURN R2 1 +)"); + + // note that mutation and capture might be inside internal control flow + // TODO: this has an oddly redundant CLOSEUPVALS after JUMP; it's not due to inlining, and is an artifact of how StatBlock/StatReturn interact + // fixing this would reduce the number of redundant CLOSEUPVALS a bit but it only affects bytecode size as these instructions aren't executed + CHECK_EQ("\n" + compileFunction(R"( +local function foo(a) + if not a then + local b b = 42 + return function() return b end + end +end + +local x = ... +local y = foo(x) +return y, x +)", + 2, 2), + R"( +DUPCLOSURE R0 K0 +GETVARARGS R1 1 +JUMPIF R1 L0 +LOADNIL R3 +LOADN R3 42 +NEWCLOSURE R2 P1 +CAPTURE REF R3 +CLOSEUPVALS R3 +JUMP L1 +CLOSEUPVALS R3 +L0: LOADNIL R2 +L1: MOVE R3 R2 +MOVE R4 R1 +RETURN R3 2 +)"); +} + TEST_CASE("InlineFallthrough") { // if the function doesn't return, we still fill the results with nil @@ -5044,27 +5185,6 @@ RETURN R1 -1 )"); } -TEST_CASE("InlineCapture") -{ - // can't inline function with nested functions that capture locals because they might be constants - CHECK_EQ("\n" + compileFunction(R"( -local function foo(a) - local function bar() - return a - end - return bar() -end -)", - 1, 2), - R"( -NEWCLOSURE R1 P0 -CAPTURE VAL R0 -MOVE R2 R1 -CALL R2 0 -1 -RETURN R2 -1 -)"); -} - TEST_CASE("InlineArgMismatch") { // when inlining a function, we must respect all the usual rules @@ -5491,6 +5611,96 @@ RETURN R2 1 )"); } +TEST_CASE("InlineMultret") +{ + // inlining a function in multret context is prohibited since we can't adjust L->top outside of CALL/GETVARARGS + CHECK_EQ("\n" + compileFunction(R"( +local function foo(a) + return a() +end + +return foo(42) +)", + 1, 2), + R"( +DUPCLOSURE R0 K0 +MOVE R1 R0 +LOADN R2 42 +CALL R1 1 -1 +RETURN R1 -1 +)"); + + // however, if we can deduce statically that a function always returns a single value, the inlining will work + CHECK_EQ("\n" + compileFunction(R"( +local function foo(a) + return a +end + +return foo(42) +)", + 1, 2), + R"( +DUPCLOSURE R0 K0 +LOADN R1 42 +RETURN R1 1 +)"); + + // this analysis will also propagate through other functions + CHECK_EQ("\n" + compileFunction(R"( +local function foo(a) + return a +end + +local function bar(a) + return foo(a) +end + +return bar(42) +)", + 2, 2), + R"( +DUPCLOSURE R0 K0 +DUPCLOSURE R1 K1 +LOADN R2 42 +RETURN R2 1 +)"); + + // we currently don't do this analysis fully for recursive functions since they can't be inlined anyway + CHECK_EQ("\n" + compileFunction(R"( +local function foo(a) + return foo(a) +end + +return foo(42) +)", + 1, 2), + R"( +DUPCLOSURE R0 K0 +CAPTURE VAL R0 +MOVE R1 R0 +LOADN R2 42 +CALL R1 1 -1 +RETURN R1 -1 +)"); + + // and unfortunately we can't do this analysis for builtins or method calls due to getfenv + CHECK_EQ("\n" + compileFunction(R"( +local function foo(a) + return math.abs(a) +end + +return foo(42) +)", + 1, 2), + R"( +DUPCLOSURE R0 K0 +MOVE R1 R0 +LOADN R2 42 +CALL R1 1 -1 +RETURN R1 -1 +)"); +} + TEST_CASE("ReturnConsecutive") { // we can return a single local directly diff --git a/tests/ConstraintGraphBuilder.test.cpp b/tests/ConstraintGraphBuilder.test.cpp index ab5af4f6..96b21613 100644 --- a/tests/ConstraintGraphBuilder.test.cpp +++ b/tests/ConstraintGraphBuilder.test.cpp @@ -17,13 +17,13 @@ TEST_CASE_FIXTURE(ConstraintGraphBuilderFixture, "hello_world") )"); cgb.visit(block); - std::vector constraints = collectConstraints(cgb.rootScope); + auto constraints = collectConstraints(cgb.rootScope); REQUIRE(2 == constraints.size()); ToStringOptions opts; - CHECK("a <: string" == toString(*constraints[0], opts)); - CHECK("b <: a" == toString(*constraints[1], opts)); + CHECK("string <: a" == toString(*constraints[0], opts)); + CHECK("a <: b" == toString(*constraints[1], opts)); } TEST_CASE_FIXTURE(ConstraintGraphBuilderFixture, "primitives") @@ -36,15 +36,34 @@ TEST_CASE_FIXTURE(ConstraintGraphBuilderFixture, "primitives") )"); cgb.visit(block); - std::vector constraints = collectConstraints(cgb.rootScope); + auto constraints = collectConstraints(cgb.rootScope); - REQUIRE(4 == constraints.size()); + REQUIRE(3 == constraints.size()); ToStringOptions opts; - CHECK("a <: string" == toString(*constraints[0], opts)); - CHECK("b <: number" == toString(*constraints[1], opts)); - CHECK("c <: boolean" == toString(*constraints[2], opts)); - CHECK("d <: nil" == toString(*constraints[3], opts)); + CHECK("string <: a" == toString(*constraints[0], opts)); + CHECK("number <: b" == toString(*constraints[1], opts)); + CHECK("boolean <: c" == toString(*constraints[2], opts)); +} + +TEST_CASE_FIXTURE(ConstraintGraphBuilderFixture, "nil_primitive") +{ + AstStatBlock* block = parse(R"( + local function a() return nil end + local b = a() + )"); + + cgb.visit(block); + auto constraints = collectConstraints(cgb.rootScope); + + ToStringOptions opts; + REQUIRE(5 <= constraints.size()); + + CHECK("*blocked-1* ~ gen () -> (a...)" == toString(*constraints[0], opts)); + CHECK("b ~ inst *blocked-1*" == toString(*constraints[1], opts)); + CHECK("() -> (c...) <: b" == toString(*constraints[2], opts)); + CHECK("c... <: d" == toString(*constraints[3], opts)); + CHECK("nil <: a..." == toString(*constraints[4], opts)); } TEST_CASE_FIXTURE(ConstraintGraphBuilderFixture, "function_application") @@ -55,15 +74,15 @@ TEST_CASE_FIXTURE(ConstraintGraphBuilderFixture, "function_application") )"); cgb.visit(block); - std::vector constraints = collectConstraints(cgb.rootScope); + auto constraints = collectConstraints(cgb.rootScope); REQUIRE(4 == constraints.size()); ToStringOptions opts; - CHECK("a <: string" == toString(*constraints[0], opts)); + CHECK("string <: a" == toString(*constraints[0], opts)); CHECK("b ~ inst a" == toString(*constraints[1], opts)); - CHECK("(string) -> (c, d...) <: b" == toString(*constraints[2], opts)); - CHECK("e <: c" == toString(*constraints[3], opts)); + CHECK("(string) -> (c...) <: b" == toString(*constraints[2], opts)); + CHECK("c... <: d" == toString(*constraints[3], opts)); } TEST_CASE_FIXTURE(ConstraintGraphBuilderFixture, "local_function_definition") @@ -75,13 +94,13 @@ TEST_CASE_FIXTURE(ConstraintGraphBuilderFixture, "local_function_definition") )"); cgb.visit(block); - std::vector constraints = collectConstraints(cgb.rootScope); + auto constraints = collectConstraints(cgb.rootScope); REQUIRE(2 == constraints.size()); ToStringOptions opts; - CHECK("a ~ gen (b) -> (c...)" == toString(*constraints[0], opts)); - CHECK("b <: c..." == toString(*constraints[1], opts)); + CHECK("*blocked-1* ~ gen (a) -> (b...)" == toString(*constraints[0], opts)); + CHECK("a <: b..." == toString(*constraints[1], opts)); } TEST_CASE_FIXTURE(ConstraintGraphBuilderFixture, "recursive_function") @@ -93,15 +112,15 @@ TEST_CASE_FIXTURE(ConstraintGraphBuilderFixture, "recursive_function") )"); cgb.visit(block); - std::vector constraints = collectConstraints(cgb.rootScope); + auto constraints = collectConstraints(cgb.rootScope); REQUIRE(4 == constraints.size()); ToStringOptions opts; - CHECK("a ~ gen (b) -> (c...)" == toString(*constraints[0], opts)); - CHECK("d ~ inst a" == toString(*constraints[1], opts)); - CHECK("(b) -> (e, f...) <: d" == toString(*constraints[2], opts)); - CHECK("e <: c..." == toString(*constraints[3], opts)); + CHECK("*blocked-1* ~ gen (a) -> (b...)" == toString(*constraints[0], opts)); + CHECK("c ~ inst (a) -> (b...)" == toString(*constraints[1], opts)); + CHECK("(a) -> (d...) <: c" == toString(*constraints[2], opts)); + CHECK("d... <: b..." == toString(*constraints[3], opts)); } TEST_SUITE_END(); diff --git a/tests/Fixture.cpp b/tests/Fixture.cpp index 232ec2de..ac22f65b 100644 --- a/tests/Fixture.cpp +++ b/tests/Fixture.cpp @@ -345,7 +345,7 @@ void Fixture::dumpErrors(std::ostream& os, const std::vector& errors) if (error.location.begin.line >= lines.size()) { os << "\tSource not available?" << std::endl; - return; + continue; } std::string_view theLine = lines[error.location.begin.line]; @@ -430,6 +430,7 @@ ConstraintGraphBuilderFixture::ConstraintGraphBuilderFixture() : Fixture() , forceTheFlag{"DebugLuauDeferredConstraintResolution", true} { + BlockedTypeVar::nextIndex = 0; } ModuleName fromString(std::string_view name) diff --git a/tests/Frontend.test.cpp b/tests/Frontend.test.cpp index c0554669..b9c24704 100644 --- a/tests/Frontend.test.cpp +++ b/tests/Frontend.test.cpp @@ -97,8 +97,8 @@ TEST_CASE_FIXTURE(FrontendFixture, "find_a_require") NaiveFileResolver naiveFileResolver; auto res = traceRequires(&naiveFileResolver, program, ""); - CHECK_EQ(1, res.requires.size()); - CHECK_EQ(res.requires[0].first, "Modules/Foo/Bar"); + CHECK_EQ(1, res.requireList.size()); + CHECK_EQ(res.requireList[0].first, "Modules/Foo/Bar"); } // It could be argued that this should not work. @@ -113,7 +113,7 @@ TEST_CASE_FIXTURE(FrontendFixture, "find_a_require_inside_a_function") NaiveFileResolver naiveFileResolver; auto res = traceRequires(&naiveFileResolver, program, ""); - CHECK_EQ(1, res.requires.size()); + CHECK_EQ(1, res.requireList.size()); } TEST_CASE_FIXTURE(FrontendFixture, "real_source") @@ -138,7 +138,7 @@ TEST_CASE_FIXTURE(FrontendFixture, "real_source") NaiveFileResolver naiveFileResolver; auto res = traceRequires(&naiveFileResolver, program, ""); - CHECK_EQ(8, res.requires.size()); + CHECK_EQ(8, res.requireList.size()); } TEST_CASE_FIXTURE(FrontendFixture, "automatically_check_dependent_scripts") diff --git a/tests/Module.test.cpp b/tests/Module.test.cpp index 89b13ab1..d585b731 100644 --- a/tests/Module.test.cpp +++ b/tests/Module.test.cpp @@ -102,7 +102,7 @@ TEST_CASE_FIXTURE(Fixture, "deepClone_cyclic_table") const FunctionTypeVar* ftv = get(methodType); REQUIRE(ftv != nullptr); - std::optional methodReturnType = first(ftv->retType); + std::optional methodReturnType = first(ftv->retTypes); REQUIRE(methodReturnType); CHECK_EQ(methodReturnType, counterCopy); diff --git a/tests/NonstrictMode.test.cpp b/tests/NonstrictMode.test.cpp index c0556103..50dcbad0 100644 --- a/tests/NonstrictMode.test.cpp +++ b/tests/NonstrictMode.test.cpp @@ -13,6 +13,57 @@ using namespace Luau; TEST_SUITE_BEGIN("NonstrictModeTests"); +TEST_CASE_FIXTURE(Fixture, "globals") +{ + CheckResult result = check(R"( + --!nonstrict + foo = true + foo = "now i'm a string!" + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + CHECK_EQ("any", toString(requireType("foo"))); +} + +TEST_CASE_FIXTURE(Fixture, "globals2") +{ + ScopedFastFlag sff[]{ + {"LuauReturnTypeInferenceInNonstrict", true}, + {"LuauLowerBoundsCalculation", true}, + }; + + CheckResult result = check(R"( + --!nonstrict + foo = function() return 1 end + foo = "now i'm a string!" + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + TypeMismatch* tm = get(result.errors[0]); + REQUIRE(tm); + CHECK_EQ("() -> number", toString(tm->wantedType)); + CHECK_EQ("string", toString(tm->givenType)); + CHECK_EQ("() -> number", toString(requireType("foo"))); +} + +TEST_CASE_FIXTURE(Fixture, "globals_everywhere") +{ + CheckResult result = check(R"( + --!nonstrict + foo = 1 + + if true then + bar = 2 + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ("any", toString(requireType("foo"))); + CHECK_EQ("any", toString(requireType("bar"))); +} + TEST_CASE_FIXTURE(BuiltinsFixture, "function_returns_number_or_string") { ScopedFastFlag sff[]{{"LuauReturnTypeInferenceInNonstrict", true}, {"LuauLowerBoundsCalculation", true}}; @@ -51,7 +102,7 @@ TEST_CASE_FIXTURE(Fixture, "infer_nullary_function") REQUIRE_EQ("any", toString(args[0])); REQUIRE_EQ("any", toString(args[1])); - auto rets = flatten(ftv->retType).first; + auto rets = flatten(ftv->retTypes).first; REQUIRE_EQ(0, rets.size()); } diff --git a/tests/Normalize.test.cpp b/tests/Normalize.test.cpp index 2876175d..284230c9 100644 --- a/tests/Normalize.test.cpp +++ b/tests/Normalize.test.cpp @@ -837,6 +837,7 @@ TEST_CASE_FIXTURE(Fixture, "intersection_inside_a_table_inside_another_intersect { ScopedFastFlag flags[] = { {"LuauLowerBoundsCalculation", true}, + {"LuauQuantifyConstrained", true}, }; // We use a function and inferred parameter types to prevent intermediate normalizations from being performed. @@ -867,16 +868,17 @@ TEST_CASE_FIXTURE(Fixture, "intersection_inside_a_table_inside_another_intersect CHECK("{+ y: number +}" == toString(args[2])); CHECK("{+ z: string +}" == toString(args[3])); - std::vector ret = flatten(ftv->retType).first; + std::vector ret = flatten(ftv->retTypes).first; REQUIRE(1 == ret.size()); - CHECK("{| x: a & {- w: boolean, y: number, z: string -} |}" == toString(ret[0])); + CHECK("{| x: a & {+ w: boolean, y: number, z: string +} |}" == toString(ret[0])); } TEST_CASE_FIXTURE(Fixture, "intersection_inside_a_table_inside_another_intersection_3") { ScopedFastFlag flags[] = { {"LuauLowerBoundsCalculation", true}, + {"LuauQuantifyConstrained", true}, }; // We use a function and inferred parameter types to prevent intermediate normalizations from being performed. @@ -906,16 +908,17 @@ TEST_CASE_FIXTURE(Fixture, "intersection_inside_a_table_inside_another_intersect CHECK("t1 where t1 = {+ y: t1 +}" == toString(args[1])); CHECK("{+ z: string +}" == toString(args[2])); - std::vector ret = flatten(ftv->retType).first; + std::vector ret = flatten(ftv->retTypes).first; REQUIRE(1 == ret.size()); - CHECK("{| x: {- x: boolean, y: t1, z: string -} |} where t1 = {+ y: t1 +}" == toString(ret[0])); + CHECK("{| x: {+ x: boolean, y: t1, z: string +} |} where t1 = {+ y: t1 +}" == toString(ret[0])); } TEST_CASE_FIXTURE(Fixture, "intersection_inside_a_table_inside_another_intersection_4") { ScopedFastFlag flags[] = { {"LuauLowerBoundsCalculation", true}, + {"LuauQuantifyConstrained", true}, }; // We use a function and inferred parameter types to prevent intermediate normalizations from being performed. @@ -944,13 +947,13 @@ TEST_CASE_FIXTURE(Fixture, "intersection_inside_a_table_inside_another_intersect REQUIRE(3 == args.size()); CHECK("{+ x: boolean +}" == toString(args[0])); - CHECK("{+ y: t1 +} where t1 = {| x: {- x: boolean, y: t1, z: string -} |}" == toString(args[1])); + CHECK("{+ y: t1 +} where t1 = {| x: {+ x: boolean, y: t1, z: string +} |}" == toString(args[1])); CHECK("{+ z: string +}" == toString(args[2])); - std::vector ret = flatten(ftv->retType).first; + std::vector ret = flatten(ftv->retTypes).first; REQUIRE(1 == ret.size()); - CHECK("t1 where t1 = {| x: {- x: boolean, y: t1, z: string -} |}" == toString(ret[0])); + CHECK("t1 where t1 = {| x: {+ x: boolean, y: t1, z: string +} |}" == toString(ret[0])); } TEST_CASE_FIXTURE(Fixture, "nested_table_normalization_with_non_table__no_ice") @@ -1062,4 +1065,29 @@ export type t0 = (((any)&({_:l0.t0,n0:t0,_G:any,}))&({_:any,}))&(((any)&({_:l0.t LUAU_REQUIRE_ERRORS(result); } +TEST_CASE_FIXTURE(BuiltinsFixture, "normalization_does_not_convert_ever") +{ + ScopedFastFlag sff[]{ + {"LuauLowerBoundsCalculation", true}, + {"LuauQuantifyConstrained", true}, + }; + + CheckResult result = check(R"( + --!strict + local function f() + if math.random() > 0.5 then + return true + end + type Ret = typeof(f()) + if math.random() > 0.5 then + return "something" + end + return "something" :: Ret + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + CHECK_EQ("() -> boolean | string", toString(requireType("f"))); +} + TEST_SUITE_END(); diff --git a/tests/NotNull.test.cpp b/tests/NotNull.test.cpp index 1a323c85..ed1c25ec 100644 --- a/tests/NotNull.test.cpp +++ b/tests/NotNull.test.cpp @@ -75,9 +75,9 @@ TEST_CASE("basic_stuff") t->y = 3.14f; const NotNull u = t; - // u->x = 44; // nope + u->x = 44; int v = u->x; - CHECK(v == 5); + CHECK(v == 44); bar(a); @@ -96,8 +96,11 @@ TEST_CASE("basic_stuff") TEST_CASE("hashable") { std::unordered_map, const char*> map; - NotNull a{new int(8)}; - NotNull b{new int(10)}; + int a_ = 8; + int b_ = 10; + + NotNull a{&a_}; + NotNull b{&b_}; std::string hello = "hello"; std::string world = "world"; @@ -108,9 +111,47 @@ TEST_CASE("hashable") CHECK_EQ(2, map.size()); CHECK_EQ(hello.c_str(), map[a]); CHECK_EQ(world.c_str(), map[b]); +} - delete a; - delete b; +TEST_CASE("const") +{ + int p = 0; + int q = 0; + + NotNull n{&p}; + + *n = 123; + + NotNull m = n; // Conversion from NotNull to NotNull is allowed + + CHECK(123 == *m); // readonly access of m is ok + + // *m = 321; // nope. m points at const data. + + // NotNull o = m; // nope. Conversion from NotNull to NotNull is forbidden + + NotNull n2{&q}; + m = n2; // ok. m points to const data, but is not itself const + + const NotNull m2 = n; + // m2 = n2; // nope. m2 is const. + *m2 = 321; // ok. m2 is const, but points to mutable data + + CHECK(321 == *n); +} + +TEST_CASE("const_compatibility") +{ + int* raw = new int(8); + + NotNull a(raw); + NotNull b(raw); + NotNull c = a; + // NotNull d = c; // nope - no conversion from const to non-const + + CHECK_EQ(*c, 8); + + delete raw; } TEST_SUITE_END(); diff --git a/tests/ToString.test.cpp b/tests/ToString.test.cpp index 4d9fad14..4d2e94ee 100644 --- a/tests/ToString.test.cpp +++ b/tests/ToString.test.cpp @@ -60,6 +60,42 @@ TEST_CASE_FIXTURE(Fixture, "named_table") CHECK_EQ("TheTable", toString(&table)); } +TEST_CASE_FIXTURE(Fixture, "empty_table") +{ + ScopedFastFlag LuauToStringTableBracesNewlines("LuauToStringTableBracesNewlines", true); + CheckResult result = check(R"( + local a: {} + )"); + + CHECK_EQ("{| |}", toString(requireType("a"))); + + // Should stay the same with useLineBreaks enabled + ToStringOptions opts; + opts.useLineBreaks = true; + CHECK_EQ("{| |}", toString(requireType("a"), opts)); +} + +TEST_CASE_FIXTURE(Fixture, "table_respects_use_line_break") +{ + ScopedFastFlag LuauToStringTableBracesNewlines("LuauToStringTableBracesNewlines", true); + CheckResult result = check(R"( + local a: { prop: string, anotherProp: number, thirdProp: boolean } + )"); + + ToStringOptions opts; + opts.useLineBreaks = true; + opts.indent = true; + + //clang-format off + CHECK_EQ("{|\n" + " anotherProp: number,\n" + " prop: string,\n" + " thirdProp: boolean\n" + "|}", + toString(requireType("a"), opts)); + //clang-format on +} + TEST_CASE_FIXTURE(BuiltinsFixture, "exhaustive_toString_of_cyclic_table") { CheckResult result = check(R"( diff --git a/tests/TypeInfer.annotations.test.cpp b/tests/TypeInfer.annotations.test.cpp index b9e1ae96..ccdd2b37 100644 --- a/tests/TypeInfer.annotations.test.cpp +++ b/tests/TypeInfer.annotations.test.cpp @@ -70,7 +70,7 @@ TEST_CASE_FIXTURE(Fixture, "function_return_annotations_are_checked") const FunctionTypeVar* ftv = get(fiftyType); REQUIRE(ftv != nullptr); - TypePackId retPack = ftv->retType; + TypePackId retPack = ftv->retTypes; const TypePack* tp = get(retPack); REQUIRE(tp != nullptr); diff --git a/tests/TypeInfer.functions.test.cpp b/tests/TypeInfer.functions.test.cpp index a28ba49e..036a667a 100644 --- a/tests/TypeInfer.functions.test.cpp +++ b/tests/TypeInfer.functions.test.cpp @@ -45,7 +45,7 @@ TEST_CASE_FIXTURE(Fixture, "infer_return_type") const FunctionTypeVar* takeFiveType = get(requireType("take_five")); REQUIRE(takeFiveType != nullptr); - std::vector retVec = flatten(takeFiveType->retType).first; + std::vector retVec = flatten(takeFiveType->retTypes).first; REQUIRE(!retVec.empty()); REQUIRE_EQ(*follow(retVec[0]), *typeChecker.numberType); @@ -345,7 +345,7 @@ TEST_CASE_FIXTURE(Fixture, "local_function") const FunctionTypeVar* ftv = get(h); REQUIRE(ftv != nullptr); - std::optional rt = first(ftv->retType); + std::optional rt = first(ftv->retTypes); REQUIRE(bool(rt)); TypeId retType = follow(*rt); @@ -361,7 +361,7 @@ TEST_CASE_FIXTURE(Fixture, "func_expr_doesnt_leak_free") LUAU_REQUIRE_NO_ERRORS(result); const Luau::FunctionTypeVar* fn = get(requireType("p")); REQUIRE(fn); - auto ret = first(fn->retType); + auto ret = first(fn->retTypes); REQUIRE(ret); REQUIRE(get(follow(*ret))); } @@ -460,7 +460,7 @@ TEST_CASE_FIXTURE(Fixture, "complicated_return_types_require_an_explicit_annotat const FunctionTypeVar* functionType = get(requireType("most_of_the_natural_numbers")); - std::optional retType = first(functionType->retType); + std::optional retType = first(functionType->retTypes); REQUIRE(retType); CHECK(get(*retType)); } @@ -1619,4 +1619,56 @@ TEST_CASE_FIXTURE(Fixture, "weird_fail_to_unify_type_pack") LUAU_REQUIRE_NO_ERRORS(result); } +TEST_CASE_FIXTURE(Fixture, "quantify_constrained_types") +{ + ScopedFastFlag sff[]{ + {"LuauLowerBoundsCalculation", true}, + {"LuauQuantifyConstrained", true}, + }; + + CheckResult result = check(R"( + --!strict + local function foo(f) + f(5) + f("hi") + local function g() + return f + end + local h = g() + h(true) + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ("((boolean | number | string) -> (a...)) -> ()", toString(requireType("foo"))); +} + +TEST_CASE_FIXTURE(Fixture, "call_o_with_another_argument_after_foo_was_quantified") +{ + ScopedFastFlag sff[]{ + {"LuauLowerBoundsCalculation", true}, + {"LuauQuantifyConstrained", true}, + }; + + CheckResult result = check(R"( + local function f(o) + local t = {} + t[o] = true + + local function foo(o) + o.m1(5) + t[o] = nil + end + + o.m1("hi") + + return t + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + // TODO: check the normalized type of f +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.generics.test.cpp b/tests/TypeInfer.generics.test.cpp index fbda8bec..edb5adcf 100644 --- a/tests/TypeInfer.generics.test.cpp +++ b/tests/TypeInfer.generics.test.cpp @@ -224,7 +224,7 @@ TEST_CASE_FIXTURE(Fixture, "infer_generic_function") const FunctionTypeVar* idFun = get(idType); REQUIRE(idFun); auto [args, varargs] = flatten(idFun->argTypes); - auto [rets, varrets] = flatten(idFun->retType); + auto [rets, varrets] = flatten(idFun->retTypes); CHECK_EQ(idFun->generics.size(), 1); CHECK_EQ(idFun->genericPacks.size(), 0); @@ -247,7 +247,7 @@ TEST_CASE_FIXTURE(Fixture, "infer_generic_local_function") const FunctionTypeVar* idFun = get(idType); REQUIRE(idFun); auto [args, varargs] = flatten(idFun->argTypes); - auto [rets, varrets] = flatten(idFun->retType); + auto [rets, varrets] = flatten(idFun->retTypes); CHECK_EQ(idFun->generics.size(), 1); CHECK_EQ(idFun->genericPacks.size(), 0); @@ -882,7 +882,7 @@ TEST_CASE_FIXTURE(Fixture, "correctly_instantiate_polymorphic_member_functions") const FunctionTypeVar* foo = get(follow(fooProp->type)); REQUIRE(bool(foo)); - std::optional ret_ = first(foo->retType); + std::optional ret_ = first(foo->retTypes); REQUIRE(bool(ret_)); TypeId ret = follow(*ret_); diff --git a/tests/TypeInfer.operators.test.cpp b/tests/TypeInfer.operators.test.cpp index 03614938..fd9b1dd4 100644 --- a/tests/TypeInfer.operators.test.cpp +++ b/tests/TypeInfer.operators.test.cpp @@ -90,7 +90,7 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "primitive_arith_no_metatable") const FunctionTypeVar* functionType = get(requireType("add")); - std::optional retType = first(functionType->retType); + std::optional retType = first(functionType->retTypes); REQUIRE(retType.has_value()); CHECK_EQ(typeChecker.numberType, follow(*retType)); CHECK_EQ(requireType("n"), typeChecker.numberType); @@ -777,8 +777,6 @@ TEST_CASE_FIXTURE(Fixture, "infer_any_in_all_modes_when_lhs_is_unknown") TEST_CASE_FIXTURE(BuiltinsFixture, "equality_operations_succeed_if_any_union_branch_succeeds") { - ScopedFastFlag sff("LuauSuccessTypingForEqualityOperations", true); - CheckResult result = check(R"( local mm = {} type Foo = typeof(setmetatable({}, mm)) diff --git a/tests/TypeInfer.provisional.test.cpp b/tests/TypeInfer.provisional.test.cpp index 22fb3b69..487e5979 100644 --- a/tests/TypeInfer.provisional.test.cpp +++ b/tests/TypeInfer.provisional.test.cpp @@ -472,6 +472,7 @@ TEST_CASE_FIXTURE(Fixture, "constrained_is_level_dependent") ScopedFastFlag sff[]{ {"LuauLowerBoundsCalculation", true}, {"LuauNormalizeFlagIsConservative", true}, + {"LuauQuantifyConstrained", true}, }; CheckResult result = check(R"( @@ -494,8 +495,8 @@ TEST_CASE_FIXTURE(Fixture, "constrained_is_level_dependent") )"); LUAU_REQUIRE_NO_ERRORS(result); - // TODO: We're missing generics a... and b... - CHECK_EQ("(t1) -> {| [t1]: boolean |} where t1 = t2 ; t2 = {+ m1: (t1) -> (a...), m2: (t2) -> (b...) +}", toString(requireType("f"))); + // TODO: We're missing generics b... + CHECK_EQ("(t1) -> {| [t1]: boolean |} where t1 = t2 ; t2 = {+ m1: (t1) -> (a...), m2: (t2) -> (b...) +}", toString(requireType("f"))); } TEST_SUITE_END(); diff --git a/tests/TypeInfer.refinements.test.cpp b/tests/TypeInfer.refinements.test.cpp index 207b3cff..cefba4b2 100644 --- a/tests/TypeInfer.refinements.test.cpp +++ b/tests/TypeInfer.refinements.test.cpp @@ -13,8 +13,8 @@ using namespace Luau; namespace { -std::optional> magicFunctionInstanceIsA( - TypeChecker& typeChecker, const ScopePtr& scope, const AstExprCall& expr, ExprResult exprResult) +std::optional> magicFunctionInstanceIsA( + TypeChecker& typeChecker, const ScopePtr& scope, const AstExprCall& expr, WithPredicate withPredicate) { if (expr.args.size != 1) return std::nullopt; @@ -32,7 +32,7 @@ std::optional> magicFunctionInstanceIsA( unfreeze(typeChecker.globalTypes); TypePackId booleanPack = typeChecker.globalTypes.addTypePack({typeChecker.booleanType}); freeze(typeChecker.globalTypes); - return ExprResult{booleanPack, {IsAPredicate{std::move(*lvalue), expr.location, tfun->type}}}; + return WithPredicate{booleanPack, {IsAPredicate{std::move(*lvalue), expr.location, tfun->type}}}; } struct RefinementClassFixture : Fixture diff --git a/tests/TypeInfer.tables.test.cpp b/tests/TypeInfer.tables.test.cpp index d622d4af..87d49651 100644 --- a/tests/TypeInfer.tables.test.cpp +++ b/tests/TypeInfer.tables.test.cpp @@ -642,7 +642,7 @@ TEST_CASE_FIXTURE(Fixture, "indexers_quantification_2") const TableTypeVar* argType = get(follow(argVec[0])); REQUIRE(argType != nullptr); - std::vector retVec = flatten(ftv->retType).first; + std::vector retVec = flatten(ftv->retTypes).first; const TableTypeVar* retType = get(follow(retVec[0])); REQUIRE(retType != nullptr); @@ -691,7 +691,7 @@ TEST_CASE_FIXTURE(Fixture, "infer_indexer_from_value_property_in_literal") const FunctionTypeVar* fType = get(requireType("f")); REQUIRE(fType != nullptr); - auto retType_ = first(fType->retType); + auto retType_ = first(fType->retTypes); REQUIRE(bool(retType_)); auto retType = get(follow(*retType_)); @@ -1881,7 +1881,7 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "quantifying_a_bound_var_works") REQUIRE(prop.type); const FunctionTypeVar* ftv = get(follow(prop.type)); REQUIRE(ftv); - const TypePack* res = get(follow(ftv->retType)); + const TypePack* res = get(follow(ftv->retTypes)); REQUIRE(res); REQUIRE(res->head.size() == 1); const MetatableTypeVar* mtv = get(follow(res->head[0])); @@ -2584,7 +2584,7 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "dont_quantify_table_that_belongs_to_outer_sc const FunctionTypeVar* newType = get(follow(counterType->props["new"].type)); REQUIRE(newType); - std::optional newRetType = *first(newType->retType); + std::optional newRetType = *first(newType->retTypes); REQUIRE(newRetType); const MetatableTypeVar* newRet = get(follow(*newRetType)); @@ -2977,7 +2977,6 @@ TEST_CASE_FIXTURE(Fixture, "mixed_tables_with_implicit_numbered_keys") TEST_CASE_FIXTURE(Fixture, "expected_indexer_value_type_extra") { - ScopedFastFlag luauExpectedPropTypeFromIndexer{"LuauExpectedPropTypeFromIndexer", true}; ScopedFastFlag luauSubtypingAddOptPropsToUnsealedTables{"LuauSubtypingAddOptPropsToUnsealedTables", true}; CheckResult result = check(R"( @@ -2992,8 +2991,6 @@ TEST_CASE_FIXTURE(Fixture, "expected_indexer_value_type_extra") TEST_CASE_FIXTURE(Fixture, "expected_indexer_value_type_extra_2") { - ScopedFastFlag luauExpectedPropTypeFromIndexer{"LuauExpectedPropTypeFromIndexer", true}; - CheckResult result = check(R"( type X = {[any]: string | boolean} diff --git a/tests/TypeInfer.test.cpp b/tests/TypeInfer.test.cpp index cf0c9881..6257cda6 100644 --- a/tests/TypeInfer.test.cpp +++ b/tests/TypeInfer.test.cpp @@ -13,8 +13,9 @@ #include -LUAU_FASTFLAG(LuauLowerBoundsCalculation) -LUAU_FASTFLAG(LuauFixLocationSpanTableIndexExpr) +LUAU_FASTFLAG(LuauLowerBoundsCalculation); +LUAU_FASTFLAG(LuauFixLocationSpanTableIndexExpr); +LUAU_FASTFLAG(DebugLuauDeferredConstraintResolution); using namespace Luau; @@ -43,10 +44,7 @@ TEST_CASE_FIXTURE(Fixture, "tc_error") CheckResult result = check("local a = 7 local b = 'hi' a = b"); LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK_EQ(result.errors[0], (TypeError{Location{Position{0, 35}, Position{0, 36}}, TypeMismatch{ - requireType("a"), - requireType("b"), - }})); + CHECK_EQ(result.errors[0], (TypeError{Location{Position{0, 35}, Position{0, 36}}, TypeMismatch{typeChecker.numberType, typeChecker.stringType}})); } TEST_CASE_FIXTURE(Fixture, "tc_error_2") @@ -86,6 +84,7 @@ TEST_CASE_FIXTURE(Fixture, "infer_locals_via_assignment_from_its_call_site") TEST_CASE_FIXTURE(Fixture, "infer_in_nocheck_mode") { ScopedFastFlag sff[]{ + {"DebugLuauDeferredConstraintResolution", false}, {"LuauReturnTypeInferenceInNonstrict", true}, {"LuauLowerBoundsCalculation", true}, }; @@ -236,10 +235,14 @@ TEST_CASE_FIXTURE(Fixture, "type_errors_infer_types") CHECK_EQ("boolean", toString(err->table)); CHECK_EQ("x", err->key); - CHECK_EQ("*unknown*", toString(requireType("c"))); - CHECK_EQ("*unknown*", toString(requireType("d"))); - CHECK_EQ("*unknown*", toString(requireType("e"))); - CHECK_EQ("*unknown*", toString(requireType("f"))); + // TODO: Should we assert anything about these tests when DCR is being used? + if (!FFlag::DebugLuauDeferredConstraintResolution) + { + CHECK_EQ("*unknown*", toString(requireType("c"))); + CHECK_EQ("*unknown*", toString(requireType("d"))); + CHECK_EQ("*unknown*", toString(requireType("e"))); + CHECK_EQ("*unknown*", toString(requireType("f"))); + } } TEST_CASE_FIXTURE(Fixture, "should_be_able_to_infer_this_without_stack_overflowing") @@ -352,40 +355,6 @@ TEST_CASE_FIXTURE(Fixture, "check_expr_recursion_limit") CHECK(nullptr != get(result.errors[0])); } -TEST_CASE_FIXTURE(Fixture, "globals") -{ - CheckResult result = check(R"( - --!nonstrict - foo = true - foo = "now i'm a string!" - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - CHECK_EQ("any", toString(requireType("foo"))); -} - -TEST_CASE_FIXTURE(Fixture, "globals2") -{ - ScopedFastFlag sff[]{ - {"LuauReturnTypeInferenceInNonstrict", true}, - {"LuauLowerBoundsCalculation", true}, - }; - - CheckResult result = check(R"( - --!nonstrict - foo = function() return 1 end - foo = "now i'm a string!" - )"); - - LUAU_REQUIRE_ERROR_COUNT(1, result); - - TypeMismatch* tm = get(result.errors[0]); - REQUIRE(tm); - CHECK_EQ("() -> number", toString(tm->wantedType)); - CHECK_EQ("string", toString(tm->givenType)); - CHECK_EQ("() -> number", toString(requireType("foo"))); -} - TEST_CASE_FIXTURE(Fixture, "globals_are_banned_in_strict_mode") { CheckResult result = check(R"( @@ -400,23 +369,6 @@ TEST_CASE_FIXTURE(Fixture, "globals_are_banned_in_strict_mode") CHECK_EQ("foo", us->name); } -TEST_CASE_FIXTURE(Fixture, "globals_everywhere") -{ - CheckResult result = check(R"( - --!nonstrict - foo = 1 - - if true then - bar = 2 - end - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - - CHECK_EQ("any", toString(requireType("foo"))); - CHECK_EQ("any", toString(requireType("bar"))); -} - TEST_CASE_FIXTURE(BuiltinsFixture, "correctly_scope_locals_do") { CheckResult result = check(R"( @@ -447,21 +399,6 @@ TEST_CASE_FIXTURE(Fixture, "checking_should_not_ice") CHECK_EQ("any", toString(requireType("value"))); } -// TEST_CASE_FIXTURE(Fixture, "infer_method_signature_of_argument") -// { -// CheckResult result = check(R"( -// function f(a) -// if a.cond then -// return a.method() -// end -// end -// )"); - -// LUAU_REQUIRE_NO_ERRORS(result); - -// CHECK_EQ("A", toString(requireType("f"))); -// } - TEST_CASE_FIXTURE(Fixture, "cyclic_follow") { check(R"( diff --git a/tests/TypeInfer.typePacks.cpp b/tests/TypeInfer.typePacks.cpp index 118863fe..bcd30498 100644 --- a/tests/TypeInfer.typePacks.cpp +++ b/tests/TypeInfer.typePacks.cpp @@ -26,7 +26,7 @@ TEST_CASE_FIXTURE(Fixture, "infer_multi_return") const FunctionTypeVar* takeTwoType = get(requireType("take_two")); REQUIRE(takeTwoType != nullptr); - const auto& [returns, tail] = flatten(takeTwoType->retType); + const auto& [returns, tail] = flatten(takeTwoType->retTypes); CHECK_EQ(2, returns.size()); CHECK_EQ(typeChecker.numberType, follow(returns[0])); @@ -73,7 +73,7 @@ TEST_CASE_FIXTURE(Fixture, "last_element_of_return_statement_can_itself_be_a_pac const FunctionTypeVar* takeOneMoreType = get(requireType("take_three")); REQUIRE(takeOneMoreType != nullptr); - const auto& [rets, tail] = flatten(takeOneMoreType->retType); + const auto& [rets, tail] = flatten(takeOneMoreType->retTypes); REQUIRE_EQ(3, rets.size()); CHECK_EQ(typeChecker.numberType, follow(rets[0])); @@ -105,10 +105,10 @@ TEST_CASE_FIXTURE(Fixture, "return_type_should_be_empty_if_nothing_is_returned") LUAU_REQUIRE_NO_ERRORS(result); const FunctionTypeVar* fTy = get(requireType("f")); REQUIRE(fTy != nullptr); - CHECK_EQ(0, size(fTy->retType)); + CHECK_EQ(0, size(fTy->retTypes)); const FunctionTypeVar* gTy = get(requireType("g")); REQUIRE(gTy != nullptr); - CHECK_EQ(0, size(gTy->retType)); + CHECK_EQ(0, size(gTy->retTypes)); } TEST_CASE_FIXTURE(Fixture, "no_return_size_should_be_zero") @@ -125,15 +125,15 @@ TEST_CASE_FIXTURE(Fixture, "no_return_size_should_be_zero") const FunctionTypeVar* fTy = get(requireType("f")); REQUIRE(fTy != nullptr); - CHECK_EQ(1, size(follow(fTy->retType))); + CHECK_EQ(1, size(follow(fTy->retTypes))); const FunctionTypeVar* gTy = get(requireType("g")); REQUIRE(gTy != nullptr); - CHECK_EQ(0, size(gTy->retType)); + CHECK_EQ(0, size(gTy->retTypes)); const FunctionTypeVar* hTy = get(requireType("h")); REQUIRE(hTy != nullptr); - CHECK_EQ(0, size(hTy->retType)); + CHECK_EQ(0, size(hTy->retTypes)); } TEST_CASE_FIXTURE(Fixture, "varargs_inference_through_multiple_scopes") diff --git a/tools/natvis/Analysis.natvis b/tools/natvis/Analysis.natvis index 5de0140e..b9ea3141 100644 --- a/tools/natvis/Analysis.natvis +++ b/tools/natvis/Analysis.natvis @@ -6,40 +6,40 @@ - {{ index=0, value={*($T1*)storage} }} - {{ index=1, value={*($T2*)storage} }} - {{ index=2, value={*($T3*)storage} }} - {{ index=3, value={*($T4*)storage} }} - {{ index=4, value={*($T5*)storage} }} - {{ index=5, value={*($T6*)storage} }} - {{ index=6, value={*($T7*)storage} }} - {{ index=7, value={*($T8*)storage} }} - {{ index=8, value={*($T9*)storage} }} - {{ index=9, value={*($T10*)storage} }} - {{ index=10, value={*($T11*)storage} }} - {{ index=11, value={*($T12*)storage} }} - {{ index=12, value={*($T13*)storage} }} - {{ index=13, value={*($T14*)storage} }} - {{ index=14, value={*($T15*)storage} }} - {{ index=15, value={*($T16*)storage} }} - {{ index=16, value={*($T17*)storage} }} - {{ index=17, value={*($T18*)storage} }} - {{ index=18, value={*($T19*)storage} }} - {{ index=19, value={*($T20*)storage} }} - {{ index=20, value={*($T21*)storage} }} - {{ index=21, value={*($T22*)storage} }} - {{ index=22, value={*($T23*)storage} }} - {{ index=23, value={*($T24*)storage} }} - {{ index=24, value={*($T25*)storage} }} - {{ index=25, value={*($T26*)storage} }} - {{ index=26, value={*($T27*)storage} }} - {{ index=27, value={*($T28*)storage} }} - {{ index=28, value={*($T29*)storage} }} - {{ index=29, value={*($T30*)storage} }} - {{ index=30, value={*($T31*)storage} }} - {{ index=31, value={*($T32*)storage} }} + {{ typeId=0, value={*($T1*)storage} }} + {{ typeId=1, value={*($T2*)storage} }} + {{ typeId=2, value={*($T3*)storage} }} + {{ typeId=3, value={*($T4*)storage} }} + {{ typeId=4, value={*($T5*)storage} }} + {{ typeId=5, value={*($T6*)storage} }} + {{ typeId=6, value={*($T7*)storage} }} + {{ typeId=7, value={*($T8*)storage} }} + {{ typeId=8, value={*($T9*)storage} }} + {{ typeId=9, value={*($T10*)storage} }} + {{ typeId=10, value={*($T11*)storage} }} + {{ typeId=11, value={*($T12*)storage} }} + {{ typeId=12, value={*($T13*)storage} }} + {{ typeId=13, value={*($T14*)storage} }} + {{ typeId=14, value={*($T15*)storage} }} + {{ typeId=15, value={*($T16*)storage} }} + {{ typeId=16, value={*($T17*)storage} }} + {{ typeId=17, value={*($T18*)storage} }} + {{ typeId=18, value={*($T19*)storage} }} + {{ typeId=19, value={*($T20*)storage} }} + {{ typeId=20, value={*($T21*)storage} }} + {{ typeId=21, value={*($T22*)storage} }} + {{ typeId=22, value={*($T23*)storage} }} + {{ typeId=23, value={*($T24*)storage} }} + {{ typeId=24, value={*($T25*)storage} }} + {{ typeId=25, value={*($T26*)storage} }} + {{ typeId=26, value={*($T27*)storage} }} + {{ typeId=27, value={*($T28*)storage} }} + {{ typeId=28, value={*($T29*)storage} }} + {{ typeId=29, value={*($T30*)storage} }} + {{ typeId=30, value={*($T31*)storage} }} + {{ typeId=31, value={*($T32*)storage} }} - typeId + typeId *($T1*)storage *($T2*)storage *($T3*)storage From f1b46f4b967f11fabe666da1de0e71b225368260 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Thu, 16 Jun 2022 18:05:14 -0700 Subject: [PATCH 277/399] Sync to upstream/release/532 (#545) --- Analysis/include/Luau/Constraint.h | 82 ++++ .../include/Luau/ConstraintGraphBuilder.h | 113 ++--- Analysis/include/Luau/ConstraintSolver.h | 103 +++-- .../include/Luau/ConstraintSolverLogger.h | 26 ++ Analysis/include/Luau/Frontend.h | 4 +- Analysis/include/Luau/Module.h | 1 + Analysis/include/Luau/Normalize.h | 1 + Analysis/include/Luau/NotNull.h | 41 +- Analysis/include/Luau/Quantify.h | 3 +- Analysis/include/Luau/RequireTracer.h | 2 +- Analysis/include/Luau/TypeChecker2.h | 13 + Analysis/include/Luau/TypeInfer.h | 41 +- Analysis/include/Luau/TypeVar.h | 37 +- Analysis/include/Luau/Unifier.h | 2 +- Analysis/include/Luau/VisitTypeVar.h | 2 +- Analysis/src/Autocomplete.cpp | 19 +- Analysis/src/BuiltinDefinitions.cpp | 68 +-- Analysis/src/Clone.cpp | 10 +- Analysis/src/Constraint.cpp | 14 + Analysis/src/ConstraintGraphBuilder.cpp | 406 +++++++++++++++--- Analysis/src/ConstraintSolver.cpp | 174 +++++--- Analysis/src/ConstraintSolverLogger.cpp | 139 ++++++ Analysis/src/Frontend.cpp | 34 +- Analysis/src/Instantiation.cpp | 2 +- Analysis/src/Linter.cpp | 4 +- Analysis/src/Normalize.cpp | 46 +- Analysis/src/Quantify.cpp | 153 ++++++- Analysis/src/RequireTracer.cpp | 14 +- Analysis/src/Substitution.cpp | 4 +- Analysis/src/ToDot.cpp | 2 +- Analysis/src/ToString.cpp | 32 +- Analysis/src/TypeAttach.cpp | 7 +- Analysis/src/TypeChecker2.cpp | 160 +++++++ Analysis/src/TypeInfer.cpp | 253 ++++++----- Analysis/src/TypeUtils.cpp | 2 +- Analysis/src/TypeVar.cpp | 41 +- Analysis/src/Unifier.cpp | 13 +- Compiler/src/BytecodeBuilder.cpp | 14 + Compiler/src/Compiler.cpp | 76 +++- Sources.cmake | 8 +- VM/src/lobject.h | 2 +- VM/src/ltable.cpp | 8 +- VM/src/ltm.cpp | 4 +- VM/src/ltm.h | 4 +- tests/Autocomplete.test.cpp | 2 +- tests/Compiler.test.cpp | 252 ++++++++++- tests/ConstraintGraphBuilder.test.cpp | 61 ++- tests/Fixture.cpp | 3 +- tests/Frontend.test.cpp | 8 +- tests/Module.test.cpp | 2 +- tests/NonstrictMode.test.cpp | 53 ++- tests/Normalize.test.cpp | 42 +- tests/NotNull.test.cpp | 53 ++- tests/TypeInfer.annotations.test.cpp | 2 +- tests/TypeInfer.functions.test.cpp | 60 ++- tests/TypeInfer.generics.test.cpp | 6 +- tests/TypeInfer.operators.test.cpp | 4 +- tests/TypeInfer.provisional.test.cpp | 5 +- tests/TypeInfer.refinements.test.cpp | 6 +- tests/TypeInfer.tables.test.cpp | 11 +- tests/TypeInfer.test.cpp | 89 +--- tests/TypeInfer.typePacks.cpp | 14 +- tools/natvis/Analysis.natvis | 66 +-- 63 files changed, 2186 insertions(+), 737 deletions(-) create mode 100644 Analysis/include/Luau/Constraint.h create mode 100644 Analysis/include/Luau/ConstraintSolverLogger.h create mode 100644 Analysis/include/Luau/TypeChecker2.h create mode 100644 Analysis/src/Constraint.cpp create mode 100644 Analysis/src/ConstraintSolverLogger.cpp create mode 100644 Analysis/src/TypeChecker2.cpp diff --git a/Analysis/include/Luau/Constraint.h b/Analysis/include/Luau/Constraint.h new file mode 100644 index 00000000..c62166e2 --- /dev/null +++ b/Analysis/include/Luau/Constraint.h @@ -0,0 +1,82 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +#pragma once + +#include "Luau/Location.h" +#include "Luau/NotNull.h" +#include "Luau/Variant.h" + +#include +#include + +namespace Luau +{ + +struct Scope2; +struct TypeVar; +using TypeId = const TypeVar*; + +struct TypePackVar; +using TypePackId = const TypePackVar*; + +// subType <: superType +struct SubtypeConstraint +{ + TypeId subType; + TypeId superType; +}; + +// subPack <: superPack +struct PackSubtypeConstraint +{ + TypePackId subPack; + TypePackId superPack; +}; + +// subType ~ gen superType +struct GeneralizationConstraint +{ + TypeId generalizedType; + TypeId sourceType; + Scope2* scope; +}; + +// subType ~ inst superType +struct InstantiationConstraint +{ + TypeId subType; + TypeId superType; +}; + +using ConstraintV = Variant; +using ConstraintPtr = std::unique_ptr; + +struct Constraint +{ + Constraint(ConstraintV&& c, Location location); + + Constraint(const Constraint&) = delete; + Constraint& operator=(const Constraint&) = delete; + + ConstraintV c; + Location location; + std::vector> dependencies; +}; + +inline Constraint& asMutable(const Constraint& c) +{ + return const_cast(c); +} + +template +T* getMutable(Constraint& c) +{ + return ::Luau::get_if(&c.c); +} + +template +const T* get(const Constraint& c) +{ + return getMutable(asMutable(c)); +} + +} // namespace Luau diff --git a/Analysis/include/Luau/ConstraintGraphBuilder.h b/Analysis/include/Luau/ConstraintGraphBuilder.h index 4234f2f6..da774a2a 100644 --- a/Analysis/include/Luau/ConstraintGraphBuilder.h +++ b/Analysis/include/Luau/ConstraintGraphBuilder.h @@ -4,9 +4,12 @@ #include #include +#include #include "Luau/Ast.h" +#include "Luau/Constraint.h" #include "Luau/Module.h" +#include "Luau/NotNull.h" #include "Luau/Symbol.h" #include "Luau/TypeVar.h" #include "Luau/Variant.h" @@ -14,69 +17,6 @@ namespace Luau { -struct Scope2; - -// subType <: superType -struct SubtypeConstraint -{ - TypeId subType; - TypeId superType; -}; - -// subPack <: superPack -struct PackSubtypeConstraint -{ - TypePackId subPack; - TypePackId superPack; -}; - -// subType ~ gen superType -struct GeneralizationConstraint -{ - TypeId subType; - TypeId superType; - Scope2* scope; -}; - -// subType ~ inst superType -struct InstantiationConstraint -{ - TypeId subType; - TypeId superType; -}; - -using ConstraintV = Variant; -using ConstraintPtr = std::unique_ptr; - -struct Constraint -{ - Constraint(ConstraintV&& c); - Constraint(ConstraintV&& c, std::vector dependencies); - - Constraint(const Constraint&) = delete; - Constraint& operator=(const Constraint&) = delete; - - ConstraintV c; - std::vector dependencies; -}; - -inline Constraint& asMutable(const Constraint& c) -{ - return const_cast(c); -} - -template -T* getMutable(Constraint& c) -{ - return ::Luau::get_if(&c.c); -} - -template -const T* get(const Constraint& c) -{ - return getMutable(asMutable(c)); -} - struct Scope2 { // The parent scope of this scope. Null if there is no parent (i.e. this @@ -102,6 +42,11 @@ struct ConstraintGraphBuilder TypeArena* const arena; // The root scope of the module we're generating constraints for. Scope2* rootScope; + // A mapping of AST node to TypeId. + DenseHashMap astTypes{nullptr}; + // A mapping of AST node to TypePackId. + DenseHashMap astTypePacks{nullptr}; + DenseHashMap astOriginalCallTypes{nullptr}; explicit ConstraintGraphBuilder(TypeArena* arena); @@ -128,8 +73,9 @@ struct ConstraintGraphBuilder * Adds a new constraint with no dependencies to a given scope. * @param scope the scope to add the constraint to. Must not be null. * @param cv the constraint variant to add. + * @param location the location to attribute to the constraint. */ - void addConstraint(Scope2* scope, ConstraintV cv); + void addConstraint(Scope2* scope, ConstraintV cv, Location location); /** * Adds a constraint to a given scope. @@ -148,15 +94,48 @@ struct ConstraintGraphBuilder void visit(Scope2* scope, AstStat* stat); void visit(Scope2* scope, AstStatBlock* block); void visit(Scope2* scope, AstStatLocal* local); - void visit(Scope2* scope, AstStatLocalFunction* local); - void visit(Scope2* scope, AstStatReturn* local); + void visit(Scope2* scope, AstStatLocalFunction* function); + void visit(Scope2* scope, AstStatFunction* function); + void visit(Scope2* scope, AstStatReturn* ret); + void visit(Scope2* scope, AstStatAssign* assign); + void visit(Scope2* scope, AstStatIf* ifStatement); + + TypePackId checkExprList(Scope2* scope, const AstArray& exprs); TypePackId checkPack(Scope2* scope, AstArray exprs); TypePackId checkPack(Scope2* scope, AstExpr* expr); + /** + * Checks an expression that is expected to evaluate to one type. + * @param scope the scope the expression is contained within. + * @param expr the expression to check. + * @return the type of the expression. + */ TypeId check(Scope2* scope, AstExpr* expr); + + TypeId checkExprTable(Scope2* scope, AstExprTable* expr); + TypeId check(Scope2* scope, AstExprIndexName* indexName); + + std::pair checkFunctionSignature(Scope2* parent, AstExprFunction* fn); + + /** + * Checks the body of a function expression. + * @param scope the interior scope of the body of the function. + * @param fn the function expression to check. + */ + void checkFunctionBody(Scope2* scope, AstExprFunction* fn); }; -std::vector collectConstraints(Scope2* rootScope); +/** + * Collects a vector of borrowed constraints from the scope and all its child + * scopes. It is important to only call this function when you're done adding + * constraints to the scope or its descendants, lest the borrowed pointers + * become invalid due to a container reallocation. + * @param rootScope the root scope of the scope graph to collect constraints + * from. + * @return a list of pointers to constraints contained within the scope graph. + * None of these pointers should be null. + */ +std::vector> collectConstraints(Scope2* rootScope); } // namespace Luau diff --git a/Analysis/include/Luau/ConstraintSolver.h b/Analysis/include/Luau/ConstraintSolver.h index 85006e68..7e6d4461 100644 --- a/Analysis/include/Luau/ConstraintSolver.h +++ b/Analysis/include/Luau/ConstraintSolver.h @@ -4,7 +4,8 @@ #include "Luau/Error.h" #include "Luau/Variant.h" -#include "Luau/ConstraintGraphBuilder.h" +#include "Luau/Constraint.h" +#include "Luau/ConstraintSolverLogger.h" #include "Luau/TypeVar.h" #include @@ -20,39 +21,81 @@ struct ConstraintSolver { TypeArena* arena; InternalErrorReporter iceReporter; - // The entire set of constraints that the solver is trying to resolve. - std::vector constraints; + // The entire set of constraints that the solver is trying to resolve. It + // is important to not add elements to this vector, lest the underlying + // storage that we retain pointers to be mutated underneath us. + const std::vector> constraints; Scope2* rootScope; - std::vector errors; // This includes every constraint that has not been fully solved. // A constraint can be both blocked and unsolved, for instance. - std::unordered_set unsolvedConstraints; + std::vector> unsolvedConstraints; // A mapping of constraint pointer to how many things the constraint is // blocked on. Can be empty or 0 for constraints that are not blocked on // anything. - std::unordered_map blockedConstraints; + std::unordered_map, size_t> blockedConstraints; // A mapping of type/pack pointers to the constraints they block. - std::unordered_map> blocked; + std::unordered_map>> blocked; + + ConstraintSolverLogger logger; explicit ConstraintSolver(TypeArena* arena, Scope2* rootScope); /** * Attempts to dispatch all pending constraints and reach a type solution - * that satisfies all of the constraints, recording any errors that are - * encountered. + * that satisfies all of the constraints. **/ void run(); bool done(); - bool tryDispatch(const Constraint* c); - bool tryDispatch(const SubtypeConstraint& c); - bool tryDispatch(const PackSubtypeConstraint& c); - bool tryDispatch(const GeneralizationConstraint& c); - bool tryDispatch(const InstantiationConstraint& c, const Constraint* constraint); + bool tryDispatch(NotNull c, bool force); + bool tryDispatch(const SubtypeConstraint& c, NotNull constraint, bool force); + bool tryDispatch(const PackSubtypeConstraint& c, NotNull constraint, bool force); + bool tryDispatch(const GeneralizationConstraint& c, NotNull constraint, bool force); + bool tryDispatch(const InstantiationConstraint& c, NotNull constraint, bool force); + void block(NotNull target, NotNull constraint); + /** + * Block a constraint on the resolution of a TypeVar. + * @returns false always. This is just to allow tryDispatch to return the result of block() + */ + bool block(TypeId target, NotNull constraint); + bool block(TypePackId target, NotNull constraint); + + void unblock(NotNull progressed); + void unblock(TypeId progressed); + void unblock(TypePackId progressed); + + /** + * @returns true if the TypeId is in a blocked state. + */ + bool isBlocked(TypeId ty); + + /** + * Returns whether the constraint is blocked on anything. + * @param constraint the constraint to check. + */ + bool isBlocked(NotNull constraint); + + /** + * Creates a new Unifier and performs a single unification operation. Commits + * the result. + * @param subType the sub-type to unify. + * @param superType the super-type to unify. + */ + void unify(TypeId subType, TypeId superType, Location location); + + /** + * Creates a new Unifier and performs a single unification operation. Commits + * the result. + * @param subPack the sub-type pack to unify. + * @param superPack the super-type pack to unify. + */ + void unify(TypePackId subPack, TypePackId superPack, Location location); + +private: /** * Marks a constraint as being blocked on a type or type pack. The constraint * solver will not attempt to dispatch blocked constraints until their @@ -60,10 +103,7 @@ struct ConstraintSolver * @param target the type or type pack pointer that the constraint is blocked on. * @param constraint the constraint to block. **/ - void block_(BlockedConstraintId target, const Constraint* constraint); - void block(const Constraint* target, const Constraint* constraint); - void block(TypeId target, const Constraint* constraint); - void block(TypePackId target, const Constraint* constraint); + void block_(BlockedConstraintId target, NotNull constraint); /** * Informs the solver that progress has been made on a type or type pack. The @@ -72,33 +112,6 @@ struct ConstraintSolver * @param progressed the type or type pack pointer that has progressed. **/ void unblock_(BlockedConstraintId progressed); - void unblock(const Constraint* progressed); - void unblock(TypeId progressed); - void unblock(TypePackId progressed); - - /** - * Returns whether the constraint is blocked on anything. - * @param constraint the constraint to check. - */ - bool isBlocked(const Constraint* constraint); - - void reportErrors(const std::vector& errors); - - /** - * Creates a new Unifier and performs a single unification operation. Commits - * the result and reports errors if necessary. - * @param subType the sub-type to unify. - * @param superType the super-type to unify. - */ - void unify(TypeId subType, TypeId superType); - - /** - * Creates a new Unifier and performs a single unification operation. Commits - * the result and reports errors if necessary. - * @param subPack the sub-type pack to unify. - * @param superPack the super-type pack to unify. - */ - void unify(TypePackId subPack, TypePackId superPack); }; void dump(Scope2* rootScope, struct ToStringOptions& opts); diff --git a/Analysis/include/Luau/ConstraintSolverLogger.h b/Analysis/include/Luau/ConstraintSolverLogger.h new file mode 100644 index 00000000..2b195d71 --- /dev/null +++ b/Analysis/include/Luau/ConstraintSolverLogger.h @@ -0,0 +1,26 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details + +#include "Luau/ConstraintGraphBuilder.h" +#include "Luau/ToString.h" + +#include +#include +#include + +namespace Luau +{ + +struct ConstraintSolverLogger +{ + std::string compileOutput(); + void captureBoundarySnapshot(const Scope2* rootScope, std::vector>& unsolvedConstraints); + void prepareStepSnapshot(const Scope2* rootScope, NotNull current, std::vector>& unsolvedConstraints); + void commitPreparedStepSnapshot(); + +private: + std::vector snapshots; + std::optional preparedSnapshot; + ToStringOptions opts; +}; + +} // namespace Luau diff --git a/Analysis/include/Luau/Frontend.h b/Analysis/include/Luau/Frontend.h index 58be0ffe..f4226cc1 100644 --- a/Analysis/include/Luau/Frontend.h +++ b/Analysis/include/Luau/Frontend.h @@ -66,7 +66,7 @@ struct SourceNode } ModuleName name; - std::unordered_set requires; + std::unordered_set requireSet; std::vector> requireLocations; bool dirtySourceModule = true; bool dirtyModule = true; @@ -186,7 +186,7 @@ public: std::unordered_map sourceNodes; std::unordered_map sourceModules; - std::unordered_map requires; + std::unordered_map requireTrace; Stats stats = {}; }; diff --git a/Analysis/include/Luau/Module.h b/Analysis/include/Luau/Module.h index f6e077dc..e979b3f0 100644 --- a/Analysis/include/Luau/Module.h +++ b/Analysis/include/Luau/Module.h @@ -69,6 +69,7 @@ struct Module std::vector>> scope2s; // never empty DenseHashMap astTypes{nullptr}; + DenseHashMap astTypePacks{nullptr}; DenseHashMap astExpectedTypes{nullptr}; DenseHashMap astOriginalCallTypes{nullptr}; DenseHashMap astOverloadResolvedTypes{nullptr}; diff --git a/Analysis/include/Luau/Normalize.h b/Analysis/include/Luau/Normalize.h index 262b54b2..d4c7698b 100644 --- a/Analysis/include/Luau/Normalize.h +++ b/Analysis/include/Luau/Normalize.h @@ -10,6 +10,7 @@ namespace Luau struct InternalErrorReporter; bool isSubtype(TypeId superTy, TypeId subTy, InternalErrorReporter& ice); +bool isSubtype(TypePackId superTy, TypePackId subTy, InternalErrorReporter& ice); std::pair normalize(TypeId ty, TypeArena& arena, InternalErrorReporter& ice); std::pair normalize(TypeId ty, const ModulePtr& module, InternalErrorReporter& ice); diff --git a/Analysis/include/Luau/NotNull.h b/Analysis/include/Luau/NotNull.h index 3d05fdea..f6043e9c 100644 --- a/Analysis/include/Luau/NotNull.h +++ b/Analysis/include/Luau/NotNull.h @@ -9,20 +9,22 @@ namespace Luau { /** A non-owning, non-null pointer to a T. - * - * A NotNull is notionally identical to a T* with the added restriction that it - * can never store nullptr. - * - * The sole conversion rule from T* to NotNull is the single-argument constructor, which - * is intentionally marked explicit. This constructor performs a runtime test to verify - * that the passed pointer is never nullptr. - * - * Pointer arithmetic, increment, decrement, and array indexing are all forbidden. - * - * An implicit coersion from NotNull to T* is afforded, as are the pointer indirection and member - * access operators. (*p and p->prop) * - * The explicit delete statement is permitted on a NotNull through this implicit conversion. + * A NotNull is notionally identical to a T* with the added restriction that + * it can never store nullptr. + * + * The sole conversion rule from T* to NotNull is the single-argument + * constructor, which is intentionally marked explicit. This constructor + * performs a runtime test to verify that the passed pointer is never nullptr. + * + * Pointer arithmetic, increment, decrement, and array indexing are all + * forbidden. + * + * An implicit coersion from NotNull to T* is afforded, as are the pointer + * indirection and member access operators. (*p and p->prop) + * + * The explicit delete statement is permitted (but not recommended) on a + * NotNull through this implicit conversion. */ template struct NotNull @@ -36,6 +38,11 @@ struct NotNull explicit NotNull(std::nullptr_t) = delete; void operator=(std::nullptr_t) = delete; + template + NotNull(NotNull other) + : ptr(other.get()) + {} + operator T*() const noexcept { return ptr; @@ -56,6 +63,12 @@ struct NotNull T& operator+(int) = delete; T& operator-(int) = delete; + T* get() const noexcept + { + return ptr; + } + +private: T* ptr; }; @@ -68,7 +81,7 @@ template struct hash> { size_t operator()(const Luau::NotNull& p) const { - return std::hash()(p.ptr); + return std::hash()(p.get()); } }; diff --git a/Analysis/include/Luau/Quantify.h b/Analysis/include/Luau/Quantify.h index b32d684e..f46f0cb5 100644 --- a/Analysis/include/Luau/Quantify.h +++ b/Analysis/include/Luau/Quantify.h @@ -6,9 +6,10 @@ namespace Luau { +struct TypeArena; struct Scope2; void quantify(TypeId ty, TypeLevel level); -void quantify(TypeId ty, Scope2* scope); +TypeId quantify(TypeArena* arena, TypeId ty, Scope2* scope); } // namespace Luau diff --git a/Analysis/include/Luau/RequireTracer.h b/Analysis/include/Luau/RequireTracer.h index c25545f5..f69d133e 100644 --- a/Analysis/include/Luau/RequireTracer.h +++ b/Analysis/include/Luau/RequireTracer.h @@ -19,7 +19,7 @@ struct RequireTraceResult { DenseHashMap exprs{nullptr}; - std::vector> requires; + std::vector> requireList; }; RequireTraceResult traceRequires(FileResolver* fileResolver, AstStatBlock* root, const ModuleName& currentModuleName); diff --git a/Analysis/include/Luau/TypeChecker2.h b/Analysis/include/Luau/TypeChecker2.h new file mode 100644 index 00000000..a6c7a3e3 --- /dev/null +++ b/Analysis/include/Luau/TypeChecker2.h @@ -0,0 +1,13 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details + +#pragma once + +#include "Luau/Ast.h" +#include "Luau/Module.h" + +namespace Luau +{ + +void check(const SourceModule& sourceModule, Module* module); + +} // namespace Luau diff --git a/Analysis/include/Luau/TypeInfer.h b/Analysis/include/Luau/TypeInfer.h index 183cc053..28adc9d9 100644 --- a/Analysis/include/Luau/TypeInfer.h +++ b/Analysis/include/Luau/TypeInfer.h @@ -138,25 +138,25 @@ struct TypeChecker void checkBlockWithoutRecursionCheck(const ScopePtr& scope, const AstStatBlock& statement); void checkBlockTypeAliases(const ScopePtr& scope, std::vector& sorted); - ExprResult checkExpr( + WithPredicate checkExpr( const ScopePtr& scope, const AstExpr& expr, std::optional expectedType = std::nullopt, bool forceSingleton = false); - ExprResult checkExpr(const ScopePtr& scope, const AstExprLocal& expr); - ExprResult checkExpr(const ScopePtr& scope, const AstExprGlobal& expr); - ExprResult checkExpr(const ScopePtr& scope, const AstExprVarargs& expr); - ExprResult checkExpr(const ScopePtr& scope, const AstExprCall& expr); - ExprResult checkExpr(const ScopePtr& scope, const AstExprIndexName& expr); - ExprResult checkExpr(const ScopePtr& scope, const AstExprIndexExpr& expr); - ExprResult checkExpr(const ScopePtr& scope, const AstExprFunction& expr, std::optional expectedType = std::nullopt); - ExprResult checkExpr(const ScopePtr& scope, const AstExprTable& expr, std::optional expectedType = std::nullopt); - ExprResult checkExpr(const ScopePtr& scope, const AstExprUnary& expr); + WithPredicate checkExpr(const ScopePtr& scope, const AstExprLocal& expr); + WithPredicate checkExpr(const ScopePtr& scope, const AstExprGlobal& expr); + WithPredicate checkExpr(const ScopePtr& scope, const AstExprVarargs& expr); + WithPredicate checkExpr(const ScopePtr& scope, const AstExprCall& expr); + WithPredicate checkExpr(const ScopePtr& scope, const AstExprIndexName& expr); + WithPredicate checkExpr(const ScopePtr& scope, const AstExprIndexExpr& expr); + WithPredicate checkExpr(const ScopePtr& scope, const AstExprFunction& expr, std::optional expectedType = std::nullopt); + WithPredicate checkExpr(const ScopePtr& scope, const AstExprTable& expr, std::optional expectedType = std::nullopt); + WithPredicate checkExpr(const ScopePtr& scope, const AstExprUnary& expr); TypeId checkRelationalOperation( const ScopePtr& scope, const AstExprBinary& expr, TypeId lhsType, TypeId rhsType, const PredicateVec& predicates = {}); TypeId checkBinaryOperation( const ScopePtr& scope, const AstExprBinary& expr, TypeId lhsType, TypeId rhsType, const PredicateVec& predicates = {}); - ExprResult checkExpr(const ScopePtr& scope, const AstExprBinary& expr); - ExprResult checkExpr(const ScopePtr& scope, const AstExprTypeAssertion& expr); - ExprResult checkExpr(const ScopePtr& scope, const AstExprError& expr); - ExprResult checkExpr(const ScopePtr& scope, const AstExprIfElse& expr, std::optional expectedType = std::nullopt); + WithPredicate checkExpr(const ScopePtr& scope, const AstExprBinary& expr); + WithPredicate checkExpr(const ScopePtr& scope, const AstExprTypeAssertion& expr); + WithPredicate checkExpr(const ScopePtr& scope, const AstExprError& expr); + WithPredicate checkExpr(const ScopePtr& scope, const AstExprIfElse& expr, std::optional expectedType = std::nullopt); TypeId checkExprTable(const ScopePtr& scope, const AstExprTable& expr, const std::vector>& fieldTypes, std::optional expectedType); @@ -179,11 +179,11 @@ struct TypeChecker void checkArgumentList( const ScopePtr& scope, Unifier& state, TypePackId paramPack, TypePackId argPack, const std::vector& argLocations); - ExprResult checkExprPack(const ScopePtr& scope, const AstExpr& expr); - ExprResult checkExprPack(const ScopePtr& scope, const AstExprCall& expr); + WithPredicate checkExprPack(const ScopePtr& scope, const AstExpr& expr); + WithPredicate checkExprPack(const ScopePtr& scope, const AstExprCall& expr); std::vector> getExpectedTypesForCall(const std::vector& overloads, size_t argumentCount, bool selfCall); - std::optional> checkCallOverload(const ScopePtr& scope, const AstExprCall& expr, TypeId fn, TypePackId retPack, - TypePackId argPack, TypePack* args, const std::vector* argLocations, const ExprResult& argListResult, + std::optional> checkCallOverload(const ScopePtr& scope, const AstExprCall& expr, TypeId fn, TypePackId retPack, + TypePackId argPack, TypePack* args, const std::vector* argLocations, const WithPredicate& argListResult, std::vector& overloadsThatMatchArgCount, std::vector& overloadsThatDont, std::vector& errors); bool handleSelfCallMismatch(const ScopePtr& scope, const AstExprCall& expr, TypePack* args, const std::vector& argLocations, const std::vector& errors); @@ -191,7 +191,7 @@ struct TypeChecker const std::vector& argLocations, const std::vector& overloads, const std::vector& overloadsThatMatchArgCount, const std::vector& errors); - ExprResult checkExprList(const ScopePtr& scope, const Location& location, const AstArray& exprs, + WithPredicate checkExprList(const ScopePtr& scope, const Location& location, const AstArray& exprs, bool substituteFreeForNil = false, const std::vector& lhsAnnotations = {}, const std::vector>& expectedTypes = {}); @@ -234,7 +234,7 @@ struct TypeChecker ErrorVec canUnify(TypeId subTy, TypeId superTy, const Location& location); ErrorVec canUnify(TypePackId subTy, TypePackId superTy, const Location& location); - void unifyLowerBound(TypePackId subTy, TypePackId superTy, const Location& location); + void unifyLowerBound(TypePackId subTy, TypePackId superTy, TypeLevel demotedLevel, const Location& location); std::optional findMetatableEntry(TypeId type, std::string entry, const Location& location); std::optional findTablePropertyRespectingMeta(TypeId lhsType, Name name, const Location& location); @@ -412,7 +412,6 @@ public: const TypeId booleanType; const TypeId threadType; const TypeId anyType; - const TypeId optionalNumberType; const TypePackId anyTypePack; diff --git a/Analysis/include/Luau/TypeVar.h b/Analysis/include/Luau/TypeVar.h index b59e7c64..ff7708d4 100644 --- a/Analysis/include/Luau/TypeVar.h +++ b/Analysis/include/Luau/TypeVar.h @@ -84,6 +84,24 @@ using Tags = std::vector; using ModuleName = std::string; +/** A TypeVar that cannot be computed. + * + * BlockedTypeVars essentially serve as a way to encode partial ordering on the + * constraint graph. Until a BlockedTypeVar is unblocked by its owning + * constraint, nothing at all can be said about it. Constraints that need to + * process a BlockedTypeVar cannot be dispatched. + * + * Whenever a BlockedTypeVar is added to the graph, we also record a constraint + * that will eventually unblock it. + */ +struct BlockedTypeVar +{ + BlockedTypeVar(); + int index; + + static int nextIndex; +}; + struct PrimitiveTypeVar { enum Type @@ -231,29 +249,29 @@ struct FunctionDefinition // TODO: Do we actually need this? We'll find out later if we can delete this. // Does not exactly belong in TypeVar.h, but this is the only way to appease the compiler. template -struct ExprResult +struct WithPredicate { T type; PredicateVec predicates; }; -using MagicFunction = std::function>( - struct TypeChecker&, const std::shared_ptr&, const class AstExprCall&, ExprResult)>; +using MagicFunction = std::function>( + struct TypeChecker&, const std::shared_ptr&, const class AstExprCall&, WithPredicate)>; struct FunctionTypeVar { // Global monomorphic function - FunctionTypeVar(TypePackId argTypes, TypePackId retType, std::optional defn = {}, bool hasSelf = false); + FunctionTypeVar(TypePackId argTypes, TypePackId retTypes, std::optional defn = {}, bool hasSelf = false); // Global polymorphic function - FunctionTypeVar(std::vector generics, std::vector genericPacks, TypePackId argTypes, TypePackId retType, + FunctionTypeVar(std::vector generics, std::vector genericPacks, TypePackId argTypes, TypePackId retTypes, std::optional defn = {}, bool hasSelf = false); // Local monomorphic function - FunctionTypeVar(TypeLevel level, TypePackId argTypes, TypePackId retType, std::optional defn = {}, bool hasSelf = false); + FunctionTypeVar(TypeLevel level, TypePackId argTypes, TypePackId retTypes, std::optional defn = {}, bool hasSelf = false); // Local polymorphic function - FunctionTypeVar(TypeLevel level, std::vector generics, std::vector genericPacks, TypePackId argTypes, TypePackId retType, + FunctionTypeVar(TypeLevel level, std::vector generics, std::vector genericPacks, TypePackId argTypes, TypePackId retTypes, std::optional defn = {}, bool hasSelf = false); TypeLevel level; @@ -263,7 +281,7 @@ struct FunctionTypeVar std::vector genericPacks; TypePackId argTypes; std::vector> argNames; - TypePackId retType; + TypePackId retTypes; std::optional definition; MagicFunction magicFunction = nullptr; // Function pointer, can be nullptr. bool hasSelf; @@ -442,7 +460,7 @@ struct LazyTypeVar using ErrorTypeVar = Unifiable::Error; -using TypeVariant = Unifiable::Variant; struct TypeVar final @@ -555,7 +573,6 @@ struct SingletonTypes const TypeId trueType; const TypeId falseType; const TypeId anyType; - const TypeId optionalNumberType; const TypePackId anyTypePack; diff --git a/Analysis/include/Luau/Unifier.h b/Analysis/include/Luau/Unifier.h index 627b52ca..b51a485e 100644 --- a/Analysis/include/Luau/Unifier.h +++ b/Analysis/include/Luau/Unifier.h @@ -110,7 +110,7 @@ private: void tryUnifyWithConstrainedSuperTypeVar(TypeId subTy, TypeId superTy); public: - void unifyLowerBound(TypePackId subTy, TypePackId superTy); + void unifyLowerBound(TypePackId subTy, TypePackId superTy, TypeLevel demotedLevel); // Report an "infinite type error" if the type "needle" already occurs within "haystack" void occursCheck(TypeId needle, TypeId haystack); diff --git a/Analysis/include/Luau/VisitTypeVar.h b/Analysis/include/Luau/VisitTypeVar.h index f3839915..642522c9 100644 --- a/Analysis/include/Luau/VisitTypeVar.h +++ b/Analysis/include/Luau/VisitTypeVar.h @@ -209,7 +209,7 @@ struct GenericTypeVarVisitor if (visit(ty, *ftv)) { traverse(ftv->argTypes); - traverse(ftv->retType); + traverse(ftv->retTypes); } } diff --git a/Analysis/src/Autocomplete.cpp b/Analysis/src/Autocomplete.cpp index a8319c59..8a63901f 100644 --- a/Analysis/src/Autocomplete.cpp +++ b/Analysis/src/Autocomplete.cpp @@ -13,7 +13,6 @@ #include #include -LUAU_FASTFLAGVARIABLE(LuauIfElseExprFixCompletionIssue, false); LUAU_FASTFLAG(LuauSelfCallAutocompleteFix2) static const std::unordered_set kStatementStartingKeywords = { @@ -268,14 +267,14 @@ static TypeCorrectKind checkTypeCorrectKind(const Module& module, TypeArena* typ auto checkFunctionType = [typeArena, &canUnify, &expectedType](const FunctionTypeVar* ftv) { if (FFlag::LuauSelfCallAutocompleteFix2) { - if (std::optional firstRetTy = first(ftv->retType)) + if (std::optional firstRetTy = first(ftv->retTypes)) return checkTypeMatch(typeArena, *firstRetTy, expectedType); return false; } else { - auto [retHead, retTail] = flatten(ftv->retType); + auto [retHead, retTail] = flatten(ftv->retTypes); if (!retHead.empty() && canUnify(retHead.front(), expectedType)) return true; @@ -454,7 +453,7 @@ static void autocompleteProps(const Module& module, TypeArena* typeArena, TypeId } else if (auto indexFunction = get(followed)) { - std::optional indexFunctionResult = first(indexFunction->retType); + std::optional indexFunctionResult = first(indexFunction->retTypes); if (indexFunctionResult) autocompleteProps(module, typeArena, rootTy, *indexFunctionResult, indexType, nodes, result, seen); } @@ -493,7 +492,7 @@ static void autocompleteProps(const Module& module, TypeArena* typeArena, TypeId autocompleteProps(module, typeArena, rootTy, followed, indexType, nodes, result, seen); else if (auto indexFunction = get(followed)) { - std::optional indexFunctionResult = first(indexFunction->retType); + std::optional indexFunctionResult = first(indexFunction->retTypes); if (indexFunctionResult) autocompleteProps(module, typeArena, rootTy, *indexFunctionResult, indexType, nodes, result, seen); } @@ -742,7 +741,7 @@ static std::optional findTypeElementAt(AstType* astType, TypeId ty, Posi if (auto element = findTypeElementAt(type->argTypes, ftv->argTypes, position)) return element; - if (auto element = findTypeElementAt(type->returnTypes, ftv->retType, position)) + if (auto element = findTypeElementAt(type->returnTypes, ftv->retTypes, position)) return element; } @@ -958,7 +957,7 @@ AutocompleteEntryMap autocompleteTypeNames(const Module& module, Position positi { if (const FunctionTypeVar* ftv = get(follow(*it))) { - if (auto ty = tryGetTypePackTypeAt(ftv->retType, tailPos)) + if (auto ty = tryGetTypePackTypeAt(ftv->retTypes, tailPos)) inferredType = *ty; } } @@ -1050,7 +1049,7 @@ AutocompleteEntryMap autocompleteTypeNames(const Module& module, Position positi { if (const FunctionTypeVar* ftv = tryGetExpectedFunctionType(module, node)) { - if (auto ty = tryGetTypePackTypeAt(ftv->retType, i)) + if (auto ty = tryGetTypePackTypeAt(ftv->retTypes, i)) tryAddTypeCorrectSuggestion(result, startScope, topType, *ty, position); } @@ -1067,7 +1066,7 @@ AutocompleteEntryMap autocompleteTypeNames(const Module& module, Position positi { if (const FunctionTypeVar* ftv = tryGetExpectedFunctionType(module, node)) { - if (auto ty = tryGetTypePackTypeAt(ftv->retType, ~0u)) + if (auto ty = tryGetTypePackTypeAt(ftv->retTypes, ~0u)) tryAddTypeCorrectSuggestion(result, startScope, topType, *ty, position); } } @@ -1266,7 +1265,7 @@ static bool autocompleteIfElseExpression( if (!parent) return false; - if (FFlag::LuauIfElseExprFixCompletionIssue && node->is()) + if (node->is()) { // Don't try to complete when the current node is an if-else expression (i.e. only try to complete when the node is a child of an if-else // expression. diff --git a/Analysis/src/BuiltinDefinitions.cpp b/Analysis/src/BuiltinDefinitions.cpp index 98737b43..2f57e23c 100644 --- a/Analysis/src/BuiltinDefinitions.cpp +++ b/Analysis/src/BuiltinDefinitions.cpp @@ -19,16 +19,16 @@ LUAU_FASTFLAGVARIABLE(LuauSetMetaTableArgsCheck, false) namespace Luau { -static std::optional> magicFunctionSelect( - TypeChecker& typechecker, const ScopePtr& scope, const AstExprCall& expr, ExprResult exprResult); -static std::optional> magicFunctionSetMetaTable( - TypeChecker& typechecker, const ScopePtr& scope, const AstExprCall& expr, ExprResult exprResult); -static std::optional> magicFunctionAssert( - TypeChecker& typechecker, const ScopePtr& scope, const AstExprCall& expr, ExprResult exprResult); -static std::optional> magicFunctionPack( - TypeChecker& typechecker, const ScopePtr& scope, const AstExprCall& expr, ExprResult exprResult); -static std::optional> magicFunctionRequire( - TypeChecker& typechecker, const ScopePtr& scope, const AstExprCall& expr, ExprResult exprResult); +static std::optional> magicFunctionSelect( + TypeChecker& typechecker, const ScopePtr& scope, const AstExprCall& expr, WithPredicate withPredicate); +static std::optional> magicFunctionSetMetaTable( + TypeChecker& typechecker, const ScopePtr& scope, const AstExprCall& expr, WithPredicate withPredicate); +static std::optional> magicFunctionAssert( + TypeChecker& typechecker, const ScopePtr& scope, const AstExprCall& expr, WithPredicate withPredicate); +static std::optional> magicFunctionPack( + TypeChecker& typechecker, const ScopePtr& scope, const AstExprCall& expr, WithPredicate withPredicate); +static std::optional> magicFunctionRequire( + TypeChecker& typechecker, const ScopePtr& scope, const AstExprCall& expr, WithPredicate withPredicate); TypeId makeUnion(TypeArena& arena, std::vector&& types) { @@ -263,10 +263,10 @@ void registerBuiltinTypes(TypeChecker& typeChecker) attachMagicFunction(getGlobalBinding(typeChecker, "require"), magicFunctionRequire); } -static std::optional> magicFunctionSelect( - TypeChecker& typechecker, const ScopePtr& scope, const AstExprCall& expr, ExprResult exprResult) +static std::optional> magicFunctionSelect( + TypeChecker& typechecker, const ScopePtr& scope, const AstExprCall& expr, WithPredicate withPredicate) { - auto [paramPack, _predicates] = exprResult; + auto [paramPack, _predicates] = withPredicate; (void)scope; @@ -287,10 +287,10 @@ static std::optional> magicFunctionSelect( if (size_t(offset) < v.size()) { std::vector result(v.begin() + offset, v.end()); - return ExprResult{typechecker.currentModule->internalTypes.addTypePack(TypePack{std::move(result), tail})}; + return WithPredicate{typechecker.currentModule->internalTypes.addTypePack(TypePack{std::move(result), tail})}; } else if (tail) - return ExprResult{*tail}; + return WithPredicate{*tail}; } typechecker.reportError(TypeError{arg1->location, GenericError{"bad argument #1 to select (index out of range)"}}); @@ -298,16 +298,16 @@ static std::optional> magicFunctionSelect( else if (AstExprConstantString* str = arg1->as()) { if (str->value.size == 1 && str->value.data[0] == '#') - return ExprResult{typechecker.currentModule->internalTypes.addTypePack({typechecker.numberType})}; + return WithPredicate{typechecker.currentModule->internalTypes.addTypePack({typechecker.numberType})}; } return std::nullopt; } -static std::optional> magicFunctionSetMetaTable( - TypeChecker& typechecker, const ScopePtr& scope, const AstExprCall& expr, ExprResult exprResult) +static std::optional> magicFunctionSetMetaTable( + TypeChecker& typechecker, const ScopePtr& scope, const AstExprCall& expr, WithPredicate withPredicate) { - auto [paramPack, _predicates] = exprResult; + auto [paramPack, _predicates] = withPredicate; TypeArena& arena = typechecker.currentModule->internalTypes; @@ -343,7 +343,7 @@ static std::optional> magicFunctionSetMetaTable( if (FFlag::LuauSetMetaTableArgsCheck && expr.args.size < 1) { - return ExprResult{}; + return WithPredicate{}; } if (!FFlag::LuauSetMetaTableArgsCheck || !expr.self) @@ -356,7 +356,7 @@ static std::optional> magicFunctionSetMetaTable( } } - return ExprResult{arena.addTypePack({mtTy})}; + return WithPredicate{arena.addTypePack({mtTy})}; } } else if (get(target) || get(target) || isTableIntersection(target)) @@ -367,13 +367,13 @@ static std::optional> magicFunctionSetMetaTable( typechecker.reportError(TypeError{expr.location, GenericError{"setmetatable should take a table"}}); } - return ExprResult{arena.addTypePack({target})}; + return WithPredicate{arena.addTypePack({target})}; } -static std::optional> magicFunctionAssert( - TypeChecker& typechecker, const ScopePtr& scope, const AstExprCall& expr, ExprResult exprResult) +static std::optional> magicFunctionAssert( + TypeChecker& typechecker, const ScopePtr& scope, const AstExprCall& expr, WithPredicate withPredicate) { - auto [paramPack, predicates] = exprResult; + auto [paramPack, predicates] = withPredicate; TypeArena& arena = typechecker.currentModule->internalTypes; @@ -382,7 +382,7 @@ static std::optional> magicFunctionAssert( { std::optional fst = first(*tail); if (!fst) - return ExprResult{paramPack}; + return WithPredicate{paramPack}; head.push_back(*fst); } @@ -397,13 +397,13 @@ static std::optional> magicFunctionAssert( head[0] = *newhead; } - return ExprResult{arena.addTypePack(TypePack{std::move(head), tail})}; + return WithPredicate{arena.addTypePack(TypePack{std::move(head), tail})}; } -static std::optional> magicFunctionPack( - TypeChecker& typechecker, const ScopePtr& scope, const AstExprCall& expr, ExprResult exprResult) +static std::optional> magicFunctionPack( + TypeChecker& typechecker, const ScopePtr& scope, const AstExprCall& expr, WithPredicate withPredicate) { - auto [paramPack, _predicates] = exprResult; + auto [paramPack, _predicates] = withPredicate; TypeArena& arena = typechecker.currentModule->internalTypes; @@ -436,7 +436,7 @@ static std::optional> magicFunctionPack( TypeId packedTable = arena.addType( TableTypeVar{{{"n", {typechecker.numberType}}}, TableIndexer(typechecker.numberType, result), scope->level, TableState::Sealed}); - return ExprResult{arena.addTypePack({packedTable})}; + return WithPredicate{arena.addTypePack({packedTable})}; } static bool checkRequirePath(TypeChecker& typechecker, AstExpr* expr) @@ -461,8 +461,8 @@ static bool checkRequirePath(TypeChecker& typechecker, AstExpr* expr) return good; } -static std::optional> magicFunctionRequire( - TypeChecker& typechecker, const ScopePtr& scope, const AstExprCall& expr, ExprResult exprResult) +static std::optional> magicFunctionRequire( + TypeChecker& typechecker, const ScopePtr& scope, const AstExprCall& expr, WithPredicate withPredicate) { TypeArena& arena = typechecker.currentModule->internalTypes; @@ -476,7 +476,7 @@ static std::optional> magicFunctionRequire( return std::nullopt; if (auto moduleInfo = typechecker.resolver->resolveModuleInfo(typechecker.currentModuleName, expr)) - return ExprResult{arena.addTypePack({typechecker.checkRequire(scope, *moduleInfo, expr.location)})}; + return WithPredicate{arena.addTypePack({typechecker.checkRequire(scope, *moduleInfo, expr.location)})}; return std::nullopt; } diff --git a/Analysis/src/Clone.cpp b/Analysis/src/Clone.cpp index 9180f309..248262ce 100644 --- a/Analysis/src/Clone.cpp +++ b/Analysis/src/Clone.cpp @@ -47,6 +47,7 @@ struct TypeCloner void operator()(const Unifiable::Generic& t); void operator()(const Unifiable::Bound& t); void operator()(const Unifiable::Error& t); + void operator()(const BlockedTypeVar& t); void operator()(const PrimitiveTypeVar& t); void operator()(const ConstrainedTypeVar& t); void operator()(const SingletonTypeVar& t); @@ -158,6 +159,11 @@ void TypeCloner::operator()(const Unifiable::Error& t) defaultClone(t); } +void TypeCloner::operator()(const BlockedTypeVar& t) +{ + defaultClone(t); +} + void TypeCloner::operator()(const PrimitiveTypeVar& t) { defaultClone(t); @@ -200,7 +206,7 @@ void TypeCloner::operator()(const FunctionTypeVar& t) ftv->tags = t.tags; ftv->argTypes = clone(t.argTypes, dest, cloneState); ftv->argNames = t.argNames; - ftv->retType = clone(t.retType, dest, cloneState); + ftv->retTypes = clone(t.retTypes, dest, cloneState); ftv->hasNoGenerics = t.hasNoGenerics; } @@ -391,7 +397,7 @@ TypeId shallowClone(TypeId ty, TypeArena& dest, const TxnLog* log) if (const FunctionTypeVar* ftv = get(ty)) { - FunctionTypeVar clone = FunctionTypeVar{ftv->level, ftv->argTypes, ftv->retType, ftv->definition, ftv->hasSelf}; + FunctionTypeVar clone = FunctionTypeVar{ftv->level, ftv->argTypes, ftv->retTypes, ftv->definition, ftv->hasSelf}; clone.generics = ftv->generics; clone.genericPacks = ftv->genericPacks; clone.magicFunction = ftv->magicFunction; diff --git a/Analysis/src/Constraint.cpp b/Analysis/src/Constraint.cpp new file mode 100644 index 00000000..6cb0e4ee --- /dev/null +++ b/Analysis/src/Constraint.cpp @@ -0,0 +1,14 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details + +#include "Luau/Constraint.h" + +namespace Luau +{ + +Constraint::Constraint(ConstraintV&& c, Location location) + : c(std::move(c)) + , location(location) +{ +} + +} // namespace Luau diff --git a/Analysis/src/ConstraintGraphBuilder.cpp b/Analysis/src/ConstraintGraphBuilder.cpp index c8f77ddf..fa627e7a 100644 --- a/Analysis/src/ConstraintGraphBuilder.cpp +++ b/Analysis/src/ConstraintGraphBuilder.cpp @@ -5,16 +5,7 @@ namespace Luau { -Constraint::Constraint(ConstraintV&& c) - : c(std::move(c)) -{ -} - -Constraint::Constraint(ConstraintV&& c, std::vector dependencies) - : c(std::move(c)) - , dependencies(dependencies) -{ -} +const AstStat* getFallthrough(const AstStat* node); // TypeInfer.cpp std::optional Scope2::lookup(Symbol sym) { @@ -68,10 +59,10 @@ Scope2* ConstraintGraphBuilder::childScope(Location location, Scope2* parent) return borrow; } -void ConstraintGraphBuilder::addConstraint(Scope2* scope, ConstraintV cv) +void ConstraintGraphBuilder::addConstraint(Scope2* scope, ConstraintV cv, Location location) { LUAU_ASSERT(scope); - scope->constraints.emplace_back(new Constraint{std::move(cv)}); + scope->constraints.emplace_back(new Constraint{std::move(cv), location}); } void ConstraintGraphBuilder::addConstraint(Scope2* scope, std::unique_ptr c) @@ -99,10 +90,18 @@ void ConstraintGraphBuilder::visit(Scope2* scope, AstStat* stat) visit(scope, s); else if (auto s = stat->as()) visit(scope, s); + else if (auto f = stat->as()) + visit(scope, f); else if (auto f = stat->as()) visit(scope, f); else if (auto r = stat->as()) visit(scope, r); + else if (auto a = stat->as()) + visit(scope, a); + else if (auto e = stat->as()) + checkPack(scope, e->expr); + else if (auto i = stat->as()) + visit(scope, i); else LUAU_ASSERT(0); } @@ -121,12 +120,30 @@ void ConstraintGraphBuilder::visit(Scope2* scope, AstStatLocal* local) scope->bindings[local] = ty; } - for (size_t i = 0; i < local->vars.size; ++i) + for (size_t i = 0; i < local->values.size; ++i) { - if (i < local->values.size) + if (local->values.data[i]->is()) + { + // HACK: we leave nil-initialized things floating under the assumption that they will later be populated. + // See the test TypeInfer/infer_locals_with_nil_value. + // Better flow awareness should make this obsolete. + } + else if (i == local->values.size - 1) + { + TypePackId exprPack = checkPack(scope, local->values.data[i]); + + if (i < local->vars.size) + { + std::vector tailValues{varTypes.begin() + i, varTypes.end()}; + TypePackId tailPack = arena->addTypePack(std::move(tailValues)); + addConstraint(scope, PackSubtypeConstraint{exprPack, tailPack}, local->location); + } + } + else { TypeId exprType = check(scope, local->values.data[i]); - addConstraint(scope, SubtypeConstraint{varTypes[i], exprType}); + if (i < varTypes.size()) + addConstraint(scope, SubtypeConstraint{varTypes[i], exprType}, local->vars.data[i]->location); } } } @@ -138,7 +155,7 @@ void addConstraints(Constraint* constraint, Scope2* scope) scope->constraints.reserve(scope->constraints.size() + scope->constraints.size()); for (const auto& c : scope->constraints) - constraint->dependencies.push_back(c.get()); + constraint->dependencies.push_back(NotNull{c.get()}); for (Scope2* childScope : scope->children) addConstraints(constraint, childScope); @@ -155,31 +172,75 @@ void ConstraintGraphBuilder::visit(Scope2* scope, AstStatLocalFunction* function TypeId functionType = nullptr; auto ty = scope->lookup(function->name); - LUAU_ASSERT(!ty.has_value()); // The parser ensures that every local function has a distinct Symbol for its name. - - functionType = freshType(scope); - scope->bindings[function->name] = functionType; - - Scope2* innerScope = childScope(function->func->body->location, scope); - TypePackId returnType = freshTypePack(scope); - innerScope->returnType = returnType; - - std::vector argTypes; - - for (AstLocal* local : function->func->args) + if (ty.has_value()) { - TypeId t = freshType(innerScope); - argTypes.push_back(t); - innerScope->bindings[local] = t; // TODO annotations + // TODO: This is duplicate definition of a local function. Is this allowed? + functionType = *ty; + } + else + { + functionType = arena->addType(BlockedTypeVar{}); + scope->bindings[function->name] = functionType; } - for (AstStat* stat : function->func->body->body) - visit(innerScope, stat); + auto [actualFunctionType, innerScope] = checkFunctionSignature(scope, function->func); + innerScope->bindings[function->name] = actualFunctionType; - FunctionTypeVar actualFunction{arena->addTypePack(argTypes), returnType}; - TypeId actualFunctionType = arena->addType(std::move(actualFunction)); + checkFunctionBody(innerScope, function->func); - std::unique_ptr c{new Constraint{GeneralizationConstraint{functionType, actualFunctionType, innerScope}}}; + std::unique_ptr c{new Constraint{GeneralizationConstraint{functionType, actualFunctionType, innerScope}, function->location}}; + addConstraints(c.get(), innerScope); + + addConstraint(scope, std::move(c)); +} + +void ConstraintGraphBuilder::visit(Scope2* scope, AstStatFunction* function) +{ + // Name could be AstStatLocal, AstStatGlobal, AstStatIndexName. + // With or without self + + TypeId functionType = nullptr; + + auto [actualFunctionType, innerScope] = checkFunctionSignature(scope, function->func); + + if (AstExprLocal* localName = function->name->as()) + { + std::optional existingFunctionTy = scope->lookup(localName->local); + if (existingFunctionTy) + { + // Duplicate definition + functionType = *existingFunctionTy; + } + else + { + functionType = arena->addType(BlockedTypeVar{}); + scope->bindings[localName->local] = functionType; + } + innerScope->bindings[localName->local] = actualFunctionType; + } + else if (AstExprGlobal* globalName = function->name->as()) + { + std::optional existingFunctionTy = scope->lookup(globalName->name); + if (existingFunctionTy) + { + // Duplicate definition + functionType = *existingFunctionTy; + } + else + { + functionType = arena->addType(BlockedTypeVar{}); + rootScope->bindings[globalName->name] = functionType; + } + innerScope->bindings[globalName->name] = actualFunctionType; + } + else if (AstExprIndexName* indexName = function->name->as()) + { + LUAU_ASSERT(0); // not yet implemented + } + + checkFunctionBody(innerScope, function->func); + + std::unique_ptr c{new Constraint{GeneralizationConstraint{functionType, actualFunctionType, innerScope}, function->location}}; addConstraints(c.get(), innerScope); addConstraint(scope, std::move(c)); @@ -190,7 +251,7 @@ void ConstraintGraphBuilder::visit(Scope2* scope, AstStatReturn* ret) LUAU_ASSERT(scope); TypePackId exprTypes = checkPack(scope, ret->list); - addConstraint(scope, PackSubtypeConstraint{exprTypes, scope->returnType}); + addConstraint(scope, PackSubtypeConstraint{exprTypes, scope->returnType}, ret->location); } void ConstraintGraphBuilder::visit(Scope2* scope, AstStatBlock* block) @@ -201,6 +262,28 @@ void ConstraintGraphBuilder::visit(Scope2* scope, AstStatBlock* block) visit(scope, stat); } +void ConstraintGraphBuilder::visit(Scope2* scope, AstStatAssign* assign) +{ + TypePackId varPackId = checkExprList(scope, assign->vars); + TypePackId valuePack = checkPack(scope, assign->values); + + addConstraint(scope, PackSubtypeConstraint{valuePack, varPackId}, assign->location); +} + +void ConstraintGraphBuilder::visit(Scope2* scope, AstStatIf* ifStatement) +{ + check(scope, ifStatement->condition); + + Scope2* thenScope = childScope(ifStatement->thenbody->location, scope); + visit(thenScope, ifStatement->thenbody); + + if (ifStatement->elsebody) + { + Scope2* elseScope = childScope(ifStatement->elsebody->location, scope); + visit(elseScope, ifStatement->elsebody); + } +} + TypePackId ConstraintGraphBuilder::checkPack(Scope2* scope, AstArray exprs) { LUAU_ASSERT(scope); @@ -224,75 +307,256 @@ TypePackId ConstraintGraphBuilder::checkPack(Scope2* scope, AstArray e return arena->addTypePack(TypePack{std::move(types), last}); } +TypePackId ConstraintGraphBuilder::checkExprList(Scope2* scope, const AstArray& exprs) +{ + TypePackId result = arena->addTypePack({}); + TypePack* resultPack = getMutable(result); + LUAU_ASSERT(resultPack); + + for (size_t i = 0; i < exprs.size; ++i) + { + AstExpr* expr = exprs.data[i]; + if (i < exprs.size - 1) + resultPack->head.push_back(check(scope, expr)); + else + resultPack->tail = checkPack(scope, expr); + } + + if (resultPack->head.empty() && resultPack->tail) + return *resultPack->tail; + else + return result; +} + TypePackId ConstraintGraphBuilder::checkPack(Scope2* scope, AstExpr* expr) { LUAU_ASSERT(scope); - // TEMP TEMP TEMP HACK HACK HACK FIXME FIXME - TypeId t = check(scope, expr); - return arena->addTypePack({t}); + TypePackId result = nullptr; + + if (AstExprCall* call = expr->as()) + { + std::vector args; + + for (AstExpr* arg : call->args) + { + args.push_back(check(scope, arg)); + } + + // TODO self + + TypeId fnType = check(scope, call->func); + + astOriginalCallTypes[call->func] = fnType; + + TypeId instantiatedType = freshType(scope); + addConstraint(scope, InstantiationConstraint{instantiatedType, fnType}, expr->location); + + TypePackId rets = freshTypePack(scope); + FunctionTypeVar ftv(arena->addTypePack(TypePack{args, {}}), rets); + TypeId inferredFnType = arena->addType(ftv); + + addConstraint(scope, SubtypeConstraint{inferredFnType, instantiatedType}, expr->location); + result = rets; + } + else + { + TypeId t = check(scope, expr); + result = arena->addTypePack({t}); + } + + LUAU_ASSERT(result); + astTypePacks[expr] = result; + return result; } TypeId ConstraintGraphBuilder::check(Scope2* scope, AstExpr* expr) { LUAU_ASSERT(scope); - if (auto a = expr->as()) - return singletonTypes.stringType; - else if (auto a = expr->as()) - return singletonTypes.numberType; - else if (auto a = expr->as()) - return singletonTypes.booleanType; - else if (auto a = expr->as()) - return singletonTypes.nilType; + TypeId result = nullptr; + + if (auto group = expr->as()) + result = check(scope, group->expr); + else if (expr->is()) + result = singletonTypes.stringType; + else if (expr->is()) + result = singletonTypes.numberType; + else if (expr->is()) + result = singletonTypes.booleanType; + else if (expr->is()) + result = singletonTypes.nilType; else if (auto a = expr->as()) { std::optional ty = scope->lookup(a->local); if (ty) - return *ty; + result = *ty; else - return singletonTypes.errorRecoveryType(singletonTypes.anyType); // FIXME? Record an error at this point? + result = singletonTypes.errorRecoveryType(); // FIXME? Record an error at this point? + } + else if (auto g = expr->as()) + { + std::optional ty = scope->lookup(g->name); + if (ty) + result = *ty; + else + result = singletonTypes.errorRecoveryType(); // FIXME? Record an error at this point? } else if (auto a = expr->as()) { - std::vector args; - - for (AstExpr* arg : a->args) + TypePackId packResult = checkPack(scope, expr); + if (auto f = first(packResult)) + return *f; + else if (get(packResult)) { - args.push_back(check(scope, arg)); + TypeId typeResult = freshType(scope); + TypePack onePack{{typeResult}, freshTypePack(scope)}; + TypePackId oneTypePack = arena->addTypePack(std::move(onePack)); + + addConstraint(scope, PackSubtypeConstraint{packResult, oneTypePack}, expr->location); + + return typeResult; } - - TypeId fnType = check(scope, a->func); - TypeId instantiatedType = freshType(scope); - addConstraint(scope, InstantiationConstraint{instantiatedType, fnType}); - - TypeId firstRet = freshType(scope); - TypePackId rets = arena->addTypePack(TypePack{{firstRet}, arena->addTypePack(TypePackVar{FreeTypePack{TypeLevel{}}})}); - FunctionTypeVar ftv(arena->addTypePack(TypePack{args, {}}), rets); - TypeId inferredFnType = arena->addType(ftv); - - addConstraint(scope, SubtypeConstraint{inferredFnType, instantiatedType}); - return firstRet; + } + else if (auto a = expr->as()) + { + auto [fnType, functionScope] = checkFunctionSignature(scope, a); + checkFunctionBody(functionScope, a); + return fnType; + } + else if (auto indexName = expr->as()) + { + result = check(scope, indexName); + } + else if (auto table = expr->as()) + { + result = checkExprTable(scope, table); } else { LUAU_ASSERT(0); - return freshType(scope); + result = freshType(scope); + } + + LUAU_ASSERT(result); + astTypes[expr] = result; + return result; +} + +TypeId ConstraintGraphBuilder::check(Scope2* scope, AstExprIndexName* indexName) +{ + TypeId obj = check(scope, indexName->expr); + TypeId result = freshType(scope); + + TableTypeVar::Props props{{indexName->index.value, Property{result}}}; + const std::optional indexer; + TableTypeVar ttv{std::move(props), indexer, TypeLevel{}, TableState::Free}; + + TypeId expectedTableType = arena->addType(std::move(ttv)); + + addConstraint(scope, SubtypeConstraint{obj, expectedTableType}, indexName->location); + + return result; +} + +TypeId ConstraintGraphBuilder::checkExprTable(Scope2* scope, AstExprTable* expr) +{ + TypeId ty = arena->addType(TableTypeVar{}); + TableTypeVar* ttv = getMutable(ty); + LUAU_ASSERT(ttv); + + auto createIndexer = [this, scope, ttv]( + TypeId currentIndexType, TypeId currentResultType, Location itemLocation, std::optional keyLocation) { + if (!ttv->indexer) + { + TypeId indexType = this->freshType(scope); + TypeId resultType = this->freshType(scope); + ttv->indexer = TableIndexer{indexType, resultType}; + } + + addConstraint(scope, SubtypeConstraint{ttv->indexer->indexType, currentIndexType}, keyLocation ? *keyLocation : itemLocation); + addConstraint(scope, SubtypeConstraint{ttv->indexer->indexResultType, currentResultType}, itemLocation); + }; + + for (const AstExprTable::Item& item : expr->items) + { + TypeId itemTy = check(scope, item.value); + + if (item.key) + { + // Even though we don't need to use the type of the item's key if + // it's a string constant, we still want to check it to populate + // astTypes. + TypeId keyTy = check(scope, item.key); + + if (AstExprConstantString* key = item.key->as()) + { + ttv->props[key->value.begin()] = {itemTy}; + } + else + { + createIndexer(keyTy, itemTy, item.value->location, item.key->location); + } + } + else + { + TypeId numberType = singletonTypes.numberType; + createIndexer(numberType, itemTy, item.value->location, std::nullopt); + } + } + + return ty; +} + +std::pair ConstraintGraphBuilder::checkFunctionSignature(Scope2* parent, AstExprFunction* fn) +{ + Scope2* innerScope = childScope(fn->body->location, parent); + TypePackId returnType = freshTypePack(innerScope); + innerScope->returnType = returnType; + + std::vector argTypes; + + for (AstLocal* local : fn->args) + { + TypeId t = freshType(innerScope); + argTypes.push_back(t); + innerScope->bindings[local] = t; // TODO annotations + } + + FunctionTypeVar actualFunction{arena->addTypePack(argTypes), returnType}; + TypeId actualFunctionType = arena->addType(std::move(actualFunction)); + LUAU_ASSERT(actualFunctionType); + astTypes[fn] = actualFunctionType; + + return {actualFunctionType, innerScope}; +} + +void ConstraintGraphBuilder::checkFunctionBody(Scope2* scope, AstExprFunction* fn) +{ + for (AstStat* stat : fn->body->body) + visit(scope, stat); + + // If it is possible for execution to reach the end of the function, the return type must be compatible with () + + if (nullptr != getFallthrough(fn->body)) + { + TypePackId empty = arena->addTypePack({}); // TODO we could have CSG retain one of these forever + addConstraint(scope, PackSubtypeConstraint{scope->returnType, empty}, fn->body->location); } } -static void collectConstraints(std::vector& result, Scope2* scope) +void collectConstraints(std::vector>& result, Scope2* scope) { for (const auto& c : scope->constraints) - result.push_back(c.get()); + result.push_back(NotNull{c.get()}); for (Scope2* child : scope->children) collectConstraints(result, child); } -std::vector collectConstraints(Scope2* rootScope) +std::vector> collectConstraints(Scope2* rootScope) { - std::vector result; + std::vector> result; collectConstraints(result, rootScope); return result; } diff --git a/Analysis/src/ConstraintSolver.cpp b/Analysis/src/ConstraintSolver.cpp index f40cd4b3..41dfd892 100644 --- a/Analysis/src/ConstraintSolver.cpp +++ b/Analysis/src/ConstraintSolver.cpp @@ -7,6 +7,7 @@ #include "Luau/Unifier.h" LUAU_FASTFLAGVARIABLE(DebugLuauLogSolver, false); +LUAU_FASTFLAGVARIABLE(DebugLuauLogSolverToJson, false); namespace Luau { @@ -58,11 +59,11 @@ ConstraintSolver::ConstraintSolver(TypeArena* arena, Scope2* rootScope) , constraints(collectConstraints(rootScope)) , rootScope(rootScope) { - for (const Constraint* c : constraints) + for (NotNull c : constraints) { - unsolvedConstraints.insert(c); + unsolvedConstraints.push_back(c); - for (const Constraint* dep : c->dependencies) + for (NotNull dep : c->dependencies) { block(dep, c); } @@ -74,8 +75,6 @@ void ConstraintSolver::run() if (done()) return; - bool progress = false; - ToStringOptions opts; if (FFlag::DebugLuauLogSolver) @@ -84,44 +83,80 @@ void ConstraintSolver::run() dump(this, opts); } - do + if (FFlag::DebugLuauLogSolverToJson) { - progress = false; + logger.captureBoundarySnapshot(rootScope, unsolvedConstraints); + } - auto it = begin(unsolvedConstraints); - auto endIt = end(unsolvedConstraints); + auto runSolverPass = [&](bool force) { + bool progress = false; - while (it != endIt) + size_t i = 0; + while (i < unsolvedConstraints.size()) { - if (isBlocked(*it)) + NotNull c = unsolvedConstraints[i]; + if (!force && isBlocked(c)) { - ++it; + ++i; continue; } - std::string saveMe = FFlag::DebugLuauLogSolver ? toString(**it, opts) : std::string{}; + std::string saveMe = FFlag::DebugLuauLogSolver ? toString(*c, opts) : std::string{}; - bool success = tryDispatch(*it); - progress = progress || success; + if (FFlag::DebugLuauLogSolverToJson) + { + logger.prepareStepSnapshot(rootScope, c, unsolvedConstraints); + } + + bool success = tryDispatch(c, force); + + progress |= success; - auto saveIt = it; - ++it; if (success) { - unsolvedConstraints.erase(saveIt); + unsolvedConstraints.erase(unsolvedConstraints.begin() + i); + + if (FFlag::DebugLuauLogSolverToJson) + { + logger.commitPreparedStepSnapshot(); + } + if (FFlag::DebugLuauLogSolver) { + if (force) + printf("Force "); printf("Dispatched\n\t%s\n", saveMe.c_str()); dump(this, opts); } } + else + ++i; + + if (force && success) + return true; } + + return progress; + }; + + bool progress = false; + do + { + progress = runSolverPass(false); + if (!progress) + progress |= runSolverPass(true); } while (progress); if (FFlag::DebugLuauLogSolver) + { dumpBindings(rootScope, opts); + } - LUAU_ASSERT(done()); + if (FFlag::DebugLuauLogSolverToJson) + { + logger.captureBoundarySnapshot(rootScope, unsolvedConstraints); + printf("Logger output:\n%s\n", logger.compileOutput().c_str()); + } } bool ConstraintSolver::done() @@ -129,21 +164,21 @@ bool ConstraintSolver::done() return unsolvedConstraints.empty(); } -bool ConstraintSolver::tryDispatch(const Constraint* constraint) +bool ConstraintSolver::tryDispatch(NotNull constraint, bool force) { - if (isBlocked(constraint)) + if (!force && isBlocked(constraint)) return false; bool success = false; if (auto sc = get(*constraint)) - success = tryDispatch(*sc); + success = tryDispatch(*sc, constraint, force); else if (auto psc = get(*constraint)) - success = tryDispatch(*psc); + success = tryDispatch(*psc, constraint, force); else if (auto gc = get(*constraint)) - success = tryDispatch(*gc); + success = tryDispatch(*gc, constraint, force); else if (auto ic = get(*constraint)) - success = tryDispatch(*ic, constraint); + success = tryDispatch(*ic, constraint, force); else LUAU_ASSERT(0); @@ -155,65 +190,66 @@ bool ConstraintSolver::tryDispatch(const Constraint* constraint) return success; } -bool ConstraintSolver::tryDispatch(const SubtypeConstraint& c) +bool ConstraintSolver::tryDispatch(const SubtypeConstraint& c, NotNull constraint, bool force) { - unify(c.subType, c.superType); + if (isBlocked(c.subType)) + return block(c.subType, constraint); + else if (isBlocked(c.superType)) + return block(c.superType, constraint); + + unify(c.subType, c.superType, constraint->location); + unblock(c.subType); unblock(c.superType); return true; } -bool ConstraintSolver::tryDispatch(const PackSubtypeConstraint& c) +bool ConstraintSolver::tryDispatch(const PackSubtypeConstraint& c, NotNull constraint, bool force) { - unify(c.subPack, c.superPack); + unify(c.subPack, c.superPack, constraint->location); unblock(c.subPack); unblock(c.superPack); return true; } -bool ConstraintSolver::tryDispatch(const GeneralizationConstraint& constraint) +bool ConstraintSolver::tryDispatch(const GeneralizationConstraint& c, NotNull constraint, bool force) { - unify(constraint.subType, constraint.superType); + if (isBlocked(c.sourceType)) + return block(c.sourceType, constraint); - quantify(constraint.superType, constraint.scope); - unblock(constraint.subType); - unblock(constraint.superType); + if (isBlocked(c.generalizedType)) + asMutable(c.generalizedType)->ty.emplace(c.sourceType); + else + unify(c.generalizedType, c.sourceType, constraint->location); + + TypeId generalized = quantify(arena, c.sourceType, c.scope); + *asMutable(c.sourceType) = *generalized; + + unblock(c.generalizedType); + unblock(c.sourceType); return true; } -bool ConstraintSolver::tryDispatch(const InstantiationConstraint& c, const Constraint* constraint) +bool ConstraintSolver::tryDispatch(const InstantiationConstraint& c, NotNull constraint, bool force) { - TypeId superType = follow(c.superType); - if (const FunctionTypeVar* ftv = get(superType)) - { - if (!ftv->generalized) - { - block(superType, constraint); - return false; - } - } - else if (get(superType)) - { - block(superType, constraint); - return false; - } - // TODO: Error if it's a primitive or something + if (isBlocked(c.superType)) + return block(c.superType, constraint); Instantiation inst(TxnLog::empty(), arena, TypeLevel{}); std::optional instantiated = inst.substitute(c.superType); LUAU_ASSERT(instantiated); // TODO FIXME HANDLE THIS - unify(c.subType, *instantiated); + unify(c.subType, *instantiated, constraint->location); unblock(c.subType); return true; } -void ConstraintSolver::block_(BlockedConstraintId target, const Constraint* constraint) +void ConstraintSolver::block_(BlockedConstraintId target, NotNull constraint) { blocked[target].push_back(constraint); @@ -221,19 +257,21 @@ void ConstraintSolver::block_(BlockedConstraintId target, const Constraint* cons count += 1; } -void ConstraintSolver::block(const Constraint* target, const Constraint* constraint) +void ConstraintSolver::block(NotNull target, NotNull constraint) { block_(target, constraint); } -void ConstraintSolver::block(TypeId target, const Constraint* constraint) +bool ConstraintSolver::block(TypeId target, NotNull constraint) { block_(target, constraint); + return false; } -void ConstraintSolver::block(TypePackId target, const Constraint* constraint) +bool ConstraintSolver::block(TypePackId target, NotNull constraint) { block_(target, constraint); + return false; } void ConstraintSolver::unblock_(BlockedConstraintId progressed) @@ -243,7 +281,7 @@ void ConstraintSolver::unblock_(BlockedConstraintId progressed) return; // unblocked should contain a value always, because of the above check - for (const Constraint* unblockedConstraint : it->second) + for (NotNull unblockedConstraint : it->second) { auto& count = blockedConstraints[unblockedConstraint]; // This assertion being hit indicates that `blocked` and @@ -257,7 +295,7 @@ void ConstraintSolver::unblock_(BlockedConstraintId progressed) blocked.erase(it); } -void ConstraintSolver::unblock(const Constraint* progressed) +void ConstraintSolver::unblock(NotNull progressed) { return unblock_(progressed); } @@ -272,35 +310,33 @@ void ConstraintSolver::unblock(TypePackId progressed) return unblock_(progressed); } -bool ConstraintSolver::isBlocked(const Constraint* constraint) +bool ConstraintSolver::isBlocked(TypeId ty) +{ + return nullptr != get(follow(ty)); +} + +bool ConstraintSolver::isBlocked(NotNull constraint) { auto blockedIt = blockedConstraints.find(constraint); return blockedIt != blockedConstraints.end() && blockedIt->second > 0; } -void ConstraintSolver::reportErrors(const std::vector& errors) -{ - this->errors.insert(end(this->errors), begin(errors), end(errors)); -} - -void ConstraintSolver::unify(TypeId subType, TypeId superType) +void ConstraintSolver::unify(TypeId subType, TypeId superType, Location location) { UnifierSharedState sharedState{&iceReporter}; - Unifier u{arena, Mode::Strict, Location{}, Covariant, sharedState}; + Unifier u{arena, Mode::Strict, location, Covariant, sharedState}; u.tryUnify(subType, superType); u.log.commit(); - reportErrors(u.errors); } -void ConstraintSolver::unify(TypePackId subPack, TypePackId superPack) +void ConstraintSolver::unify(TypePackId subPack, TypePackId superPack, Location location) { UnifierSharedState sharedState{&iceReporter}; - Unifier u{arena, Mode::Strict, Location{}, Covariant, sharedState}; + Unifier u{arena, Mode::Strict, location, Covariant, sharedState}; u.tryUnify(subPack, superPack); u.log.commit(); - reportErrors(u.errors); } } // namespace Luau diff --git a/Analysis/src/ConstraintSolverLogger.cpp b/Analysis/src/ConstraintSolverLogger.cpp new file mode 100644 index 00000000..2f93c280 --- /dev/null +++ b/Analysis/src/ConstraintSolverLogger.cpp @@ -0,0 +1,139 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details + +#include "Luau/ConstraintSolverLogger.h" + +namespace Luau +{ + +static std::string dumpScopeAndChildren(const Scope2* scope, ToStringOptions& opts) +{ + std::string output = "{\"bindings\":{"; + + bool comma = false; + for (const auto& [name, type] : scope->bindings) + { + if (comma) + output += ","; + + output += "\""; + output += name.c_str(); + output += "\": \""; + + ToStringResult result = toStringDetailed(type, opts); + opts.nameMap = std::move(result.nameMap); + output += result.name; + output += "\""; + + comma = true; + } + + output += "},\"children\":["; + comma = false; + + for (const Scope2* child : scope->children) + { + if (comma) + output += ","; + + output += dumpScopeAndChildren(child, opts); + comma = true; + } + + output += "]}"; + return output; +} + +static std::string dumpConstraintsToDot(std::vector>& constraints, ToStringOptions& opts) +{ + std::string result = "digraph Constraints {\\n"; + + std::unordered_set> contained; + for (NotNull c : constraints) + { + contained.insert(c); + } + + for (NotNull c : constraints) + { + std::string id = std::to_string(reinterpret_cast(c.get())); + result += id; + result += " [label=\\\""; + result += toString(*c, opts).c_str(); + result += "\\\"];\\n"; + + for (NotNull dep : c->dependencies) + { + if (contained.count(dep) == 0) + continue; + + result += std::to_string(reinterpret_cast(dep.get())); + result += " -> "; + result += id; + result += ";\\n"; + } + } + + result += "}"; + + return result; +} + +std::string ConstraintSolverLogger::compileOutput() +{ + std::string output = "["; + bool comma = false; + + for (const std::string& snapshot : snapshots) + { + if (comma) + output += ","; + output += snapshot; + + comma = true; + } + + output += "]"; + return output; +} + +void ConstraintSolverLogger::captureBoundarySnapshot(const Scope2* rootScope, std::vector>& unsolvedConstraints) +{ + std::string snapshot = "{\"type\":\"boundary\",\"rootScope\":"; + + snapshot += dumpScopeAndChildren(rootScope, opts); + snapshot += ",\"constraintGraph\":\""; + snapshot += dumpConstraintsToDot(unsolvedConstraints, opts); + snapshot += "\"}"; + + snapshots.push_back(std::move(snapshot)); +} + +void ConstraintSolverLogger::prepareStepSnapshot( + const Scope2* rootScope, NotNull current, std::vector>& unsolvedConstraints) +{ + // LUAU_ASSERT(!preparedSnapshot); + + std::string snapshot = "{\"type\":\"step\",\"rootScope\":"; + + snapshot += dumpScopeAndChildren(rootScope, opts); + snapshot += ",\"constraintGraph\":\""; + snapshot += dumpConstraintsToDot(unsolvedConstraints, opts); + snapshot += "\",\"currentId\":\""; + snapshot += std::to_string(reinterpret_cast(current.get())); + snapshot += "\",\"current\":\""; + snapshot += toString(*current, opts); + snapshot += "\"}"; + + preparedSnapshot = std::move(snapshot); +} + +void ConstraintSolverLogger::commitPreparedStepSnapshot() +{ + if (preparedSnapshot) + { + snapshots.push_back(std::move(*preparedSnapshot)); + preparedSnapshot = std::nullopt; + } +} + +} // namespace Luau diff --git a/Analysis/src/Frontend.cpp b/Analysis/src/Frontend.cpp index 741a35cf..9e025062 100644 --- a/Analysis/src/Frontend.cpp +++ b/Analysis/src/Frontend.cpp @@ -1,16 +1,17 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #include "Luau/Frontend.h" -#include "Luau/Common.h" #include "Luau/Clone.h" +#include "Luau/Common.h" #include "Luau/Config.h" -#include "Luau/FileResolver.h" #include "Luau/ConstraintGraphBuilder.h" #include "Luau/ConstraintSolver.h" +#include "Luau/FileResolver.h" #include "Luau/Parser.h" #include "Luau/Scope.h" #include "Luau/StringUtils.h" #include "Luau/TimeTrace.h" +#include "Luau/TypeChecker2.h" #include "Luau/TypeInfer.h" #include "Luau/Variant.h" @@ -216,7 +217,7 @@ ErrorVec accumulateErrors( continue; const SourceNode& sourceNode = it->second; - queue.insert(queue.end(), sourceNode.requires.begin(), sourceNode.requires.end()); + queue.insert(queue.end(), sourceNode.requireSet.begin(), sourceNode.requireSet.end()); // FIXME: If a module has a syntax error, we won't be able to re-report it here. // The solution is probably to move errors from Module to SourceNode @@ -586,7 +587,7 @@ bool Frontend::parseGraph(std::vector& buildQueue, CheckResult& chec path.push_back(top); // push children - for (const ModuleName& dep : top->requires) + for (const ModuleName& dep : top->requireSet) { auto it = sourceNodes.find(dep); if (it != sourceNodes.end()) @@ -738,7 +739,7 @@ void Frontend::markDirty(const ModuleName& name, std::vector* marked std::unordered_map> reverseDeps; for (const auto& module : sourceNodes) { - for (const auto& dep : module.second.requires) + for (const auto& dep : module.second.requireSet) reverseDeps[dep].push_back(module.first); } @@ -797,9 +798,14 @@ ModulePtr Frontend::check(const SourceModule& sourceModule, Mode mode, const Sco cs.run(); result->scope2s = std::move(cgb.scopes); + result->astTypes = std::move(cgb.astTypes); + result->astTypePacks = std::move(cgb.astTypePacks); + result->astOriginalCallTypes = std::move(cgb.astOriginalCallTypes); result->clonePublicInterface(iceHandler); + Luau::check(sourceModule, result.get()); + return result; } @@ -841,8 +847,8 @@ std::pair Frontend::getSourceNode(CheckResult& check SourceModule result = parse(name, source->source, opts); result.type = source->type; - RequireTraceResult& requireTrace = requires[name]; - requireTrace = traceRequires(fileResolver, result.root, name); + RequireTraceResult& require = requireTrace[name]; + require = traceRequires(fileResolver, result.root, name); SourceNode& sourceNode = sourceNodes[name]; SourceModule& sourceModule = sourceModules[name]; @@ -851,7 +857,7 @@ std::pair Frontend::getSourceNode(CheckResult& check sourceModule.environmentName = environmentName; sourceNode.name = name; - sourceNode.requires.clear(); + sourceNode.requireSet.clear(); sourceNode.requireLocations.clear(); sourceNode.dirtySourceModule = false; @@ -861,10 +867,10 @@ std::pair Frontend::getSourceNode(CheckResult& check sourceNode.dirtyModuleForAutocomplete = true; } - for (const auto& [moduleName, location] : requireTrace.requires) - sourceNode.requires.insert(moduleName); + for (const auto& [moduleName, location] : require.requireList) + sourceNode.requireSet.insert(moduleName); - sourceNode.requireLocations = requireTrace.requires; + sourceNode.requireLocations = require.requireList; return {&sourceNode, &sourceModule}; } @@ -925,8 +931,8 @@ SourceModule Frontend::parse(const ModuleName& name, std::string_view src, const std::optional FrontendModuleResolver::resolveModuleInfo(const ModuleName& currentModuleName, const AstExpr& pathExpr) { // FIXME I think this can be pushed into the FileResolver. - auto it = frontend->requires.find(currentModuleName); - if (it == frontend->requires.end()) + auto it = frontend->requireTrace.find(currentModuleName); + if (it == frontend->requireTrace.end()) { // CLI-43699 // If we can't find the current module name, that's because we bypassed the frontend's initializer @@ -1025,7 +1031,7 @@ void Frontend::clear() sourceModules.clear(); moduleResolver.modules.clear(); moduleResolverForAutocomplete.modules.clear(); - requires.clear(); + requireTrace.clear(); } } // namespace Luau diff --git a/Analysis/src/Instantiation.cpp b/Analysis/src/Instantiation.cpp index f145a511..77c62422 100644 --- a/Analysis/src/Instantiation.cpp +++ b/Analysis/src/Instantiation.cpp @@ -40,7 +40,7 @@ TypeId Instantiation::clean(TypeId ty) const FunctionTypeVar* ftv = log->getMutable(ty); LUAU_ASSERT(ftv); - FunctionTypeVar clone = FunctionTypeVar{level, ftv->argTypes, ftv->retType, ftv->definition, ftv->hasSelf}; + FunctionTypeVar clone = FunctionTypeVar{level, ftv->argTypes, ftv->retTypes, ftv->definition, ftv->hasSelf}; clone.magicFunction = ftv->magicFunction; clone.tags = ftv->tags; clone.argNames = ftv->argNames; diff --git a/Analysis/src/Linter.cpp b/Analysis/src/Linter.cpp index 200b7d1b..50868e56 100644 --- a/Analysis/src/Linter.cpp +++ b/Analysis/src/Linter.cpp @@ -2282,7 +2282,7 @@ private: size_t getReturnCount(TypeId ty) { if (auto ftv = get(ty)) - return size(ftv->retType); + return size(ftv->retTypes); if (auto itv = get(ty)) { @@ -2291,7 +2291,7 @@ private: for (TypeId part : itv->parts) if (auto ftv = get(follow(part))) - result = std::max(result, size(ftv->retType)); + result = std::max(result, size(ftv->retTypes)); return result; } diff --git a/Analysis/src/Normalize.cpp b/Analysis/src/Normalize.cpp index 11403be5..d36665e2 100644 --- a/Analysis/src/Normalize.cpp +++ b/Analysis/src/Normalize.cpp @@ -17,6 +17,7 @@ LUAU_FASTFLAGVARIABLE(LuauNormalizeCombineTableFix, false); LUAU_FASTFLAGVARIABLE(LuauNormalizeFlagIsConservative, false); LUAU_FASTFLAGVARIABLE(LuauNormalizeCombineEqFix, false); LUAU_FASTFLAGVARIABLE(LuauReplaceReplacer, false); +LUAU_FASTFLAG(LuauQuantifyConstrained) namespace Luau { @@ -273,6 +274,18 @@ bool isSubtype(TypeId subTy, TypeId superTy, InternalErrorReporter& ice) return ok; } +bool isSubtype(TypePackId subPack, TypePackId superPack, InternalErrorReporter& ice) +{ + UnifierSharedState sharedState{&ice}; + TypeArena arena; + Unifier u{&arena, Mode::Strict, Location{}, Covariant, sharedState}; + u.anyIsTop = true; + + u.tryUnify(subPack, superPack); + const bool ok = u.errors.empty() && u.log.empty(); + return ok; +} + template static bool areNormal_(const T& t, const std::unordered_set& seen, InternalErrorReporter& ice) { @@ -390,6 +403,7 @@ struct Normalize final : TypeVarVisitor bool visit(TypeId ty, const ConstrainedTypeVar& ctvRef) override { CHECK_ITERATION_LIMIT(false); + LUAU_ASSERT(!ty->normal); ConstrainedTypeVar* ctv = const_cast(&ctvRef); @@ -401,14 +415,21 @@ struct Normalize final : TypeVarVisitor std::vector newParts = normalizeUnion(parts); - const bool normal = areNormal(newParts, seen, ice); - - if (newParts.size() == 1) - *asMutable(ty) = BoundTypeVar{newParts[0]}; + if (FFlag::LuauQuantifyConstrained) + { + ctv->parts = std::move(newParts); + } else - *asMutable(ty) = UnionTypeVar{std::move(newParts)}; + { + const bool normal = areNormal(newParts, seen, ice); - asMutable(ty)->normal = normal; + if (newParts.size() == 1) + *asMutable(ty) = BoundTypeVar{newParts[0]}; + else + *asMutable(ty) = UnionTypeVar{std::move(newParts)}; + + asMutable(ty)->normal = normal; + } return false; } @@ -421,9 +442,9 @@ struct Normalize final : TypeVarVisitor return false; traverse(ftv.argTypes); - traverse(ftv.retType); + traverse(ftv.retTypes); - asMutable(ty)->normal = areNormal(ftv.argTypes, seen, ice) && areNormal(ftv.retType, seen, ice); + asMutable(ty)->normal = areNormal(ftv.argTypes, seen, ice) && areNormal(ftv.retTypes, seen, ice); return false; } @@ -465,7 +486,14 @@ struct Normalize final : TypeVarVisitor checkNormal(ttv.indexer->indexResultType); } - asMutable(ty)->normal = normal; + // An unsealed table can never be normal, ditto for free tables iff the type it is bound to is also not normal. + if (FFlag::LuauQuantifyConstrained) + { + if (ttv.state == TableState::Generic || ttv.state == TableState::Sealed || (ttv.state == TableState::Free && follow(ty)->normal)) + asMutable(ty)->normal = normal; + } + else + asMutable(ty)->normal = normal; return false; } diff --git a/Analysis/src/Quantify.cpp b/Analysis/src/Quantify.cpp index 21775373..2004d153 100644 --- a/Analysis/src/Quantify.cpp +++ b/Analysis/src/Quantify.cpp @@ -2,15 +2,32 @@ #include "Luau/Quantify.h" +#include "Luau/ConstraintGraphBuilder.h" // TODO for Scope2; move to separate header +#include "Luau/TxnLog.h" +#include "Luau/Substitution.h" #include "Luau/VisitTypeVar.h" #include "Luau/ConstraintGraphBuilder.h" // TODO for Scope2; move to separate header LUAU_FASTFLAG(LuauAlwaysQuantify); LUAU_FASTFLAG(DebugLuauDeferredConstraintResolution); +LUAU_FASTFLAGVARIABLE(LuauQuantifyConstrained, false) namespace Luau { +/// @return true if outer encloses inner +static bool subsumes(Scope2* outer, Scope2* inner) +{ + while (inner) + { + if (inner == outer) + return true; + inner = inner->parent; + } + + return false; +} + struct Quantifier final : TypeVarOnceVisitor { TypeLevel level; @@ -62,6 +79,34 @@ struct Quantifier final : TypeVarOnceVisitor return false; } + bool visit(TypeId ty, const ConstrainedTypeVar&) override + { + if (FFlag::LuauQuantifyConstrained) + { + ConstrainedTypeVar* ctv = getMutable(ty); + + seenMutableType = true; + + if (FFlag::DebugLuauDeferredConstraintResolution ? !subsumes(scope, ctv->scope) : !level.subsumes(ctv->level)) + return false; + + std::vector opts = std::move(ctv->parts); + + // We might transmute, so it's not safe to rely on the builtin traversal logic + for (TypeId opt : opts) + traverse(opt); + + if (opts.size() == 1) + *asMutable(ty) = BoundTypeVar{opts[0]}; + else + *asMutable(ty) = UnionTypeVar{std::move(opts)}; + + return false; + } + else + return true; + } + bool visit(TypeId ty, const TableTypeVar&) override { LUAU_ASSERT(getMutable(ty)); @@ -73,8 +118,12 @@ struct Quantifier final : TypeVarOnceVisitor if (ttv.state == TableState::Free) seenMutableType = true; - if (ttv.state == TableState::Sealed || ttv.state == TableState::Generic) - return false; + if (!FFlag::LuauQuantifyConstrained) + { + if (ttv.state == TableState::Sealed || ttv.state == TableState::Generic) + return false; + } + if (FFlag::DebugLuauDeferredConstraintResolution ? !subsumes(scope, ttv.scope) : !level.subsumes(ttv.level)) { if (ttv.state == TableState::Unsealed) @@ -156,4 +205,104 @@ void quantify(TypeId ty, Scope2* scope) ftv->generalized = true; } +struct PureQuantifier : Substitution +{ + Scope2* scope; + std::vector insertedGenerics; + std::vector insertedGenericPacks; + + PureQuantifier(const TxnLog* log, TypeArena* arena, Scope2* scope) + : Substitution(log, arena) + , scope(scope) + { + } + + bool isDirty(TypeId ty) override + { + LUAU_ASSERT(ty == follow(ty)); + + if (auto ftv = get(ty)) + { + return subsumes(scope, ftv->scope); + } + else if (auto ttv = get(ty)) + { + return ttv->state == TableState::Free && subsumes(scope, ttv->scope); + } + + return false; + } + + bool isDirty(TypePackId tp) override + { + if (auto ftp = get(tp)) + { + return subsumes(scope, ftp->scope); + } + + return false; + } + + TypeId clean(TypeId ty) override + { + if (auto ftv = get(ty)) + { + TypeId result = arena->addType(GenericTypeVar{}); + insertedGenerics.push_back(result); + return result; + } + else if (auto ttv = get(ty)) + { + TypeId result = arena->addType(TableTypeVar{}); + TableTypeVar* resultTable = getMutable(result); + LUAU_ASSERT(resultTable); + + *resultTable = *ttv; + resultTable->scope = nullptr; + resultTable->state = TableState::Generic; + + return result; + } + + return ty; + } + + TypePackId clean(TypePackId tp) override + { + if (auto ftp = get(tp)) + { + TypePackId result = arena->addTypePack(TypePackVar{GenericTypePack{}}); + insertedGenericPacks.push_back(result); + return result; + } + + return tp; + } + + bool ignoreChildren(TypeId ty) override + { + return ty->persistent; + } + bool ignoreChildren(TypePackId ty) override + { + return ty->persistent; + } +}; + +TypeId quantify(TypeArena* arena, TypeId ty, Scope2* scope) +{ + PureQuantifier quantifier{TxnLog::empty(), arena, scope}; + std::optional result = quantifier.substitute(ty); + LUAU_ASSERT(result); + + FunctionTypeVar* ftv = getMutable(*result); + LUAU_ASSERT(ftv); + ftv->generics.insert(ftv->generics.end(), quantifier.insertedGenerics.begin(), quantifier.insertedGenerics.end()); + ftv->genericPacks.insert(ftv->genericPacks.end(), quantifier.insertedGenericPacks.begin(), quantifier.insertedGenericPacks.end()); + + // TODO: Set hasNoGenerics. + + return *result; +} + } // namespace Luau diff --git a/Analysis/src/RequireTracer.cpp b/Analysis/src/RequireTracer.cpp index 8ed245fb..c036a7a5 100644 --- a/Analysis/src/RequireTracer.cpp +++ b/Analysis/src/RequireTracer.cpp @@ -28,7 +28,7 @@ struct RequireTracer : AstVisitor AstExprGlobal* global = expr->func->as(); if (global && global->name == "require" && expr->args.size >= 1) - requires.push_back(expr); + requireCalls.push_back(expr); return true; } @@ -84,9 +84,9 @@ struct RequireTracer : AstVisitor ModuleInfo moduleContext{currentModuleName}; // seed worklist with require arguments - work.reserve(requires.size()); + work.reserve(requireCalls.size()); - for (AstExprCall* require : requires) + for (AstExprCall* require : requireCalls) work.push_back(require->args.data[0]); // push all dependent expressions to the work stack; note that the vector is modified during traversal @@ -125,15 +125,15 @@ struct RequireTracer : AstVisitor } // resolve all requires according to their argument - result.requires.reserve(requires.size()); + result.requireList.reserve(requireCalls.size()); - for (AstExprCall* require : requires) + for (AstExprCall* require : requireCalls) { AstExpr* arg = require->args.data[0]; if (const ModuleInfo* info = result.exprs.find(arg)) { - result.requires.push_back({info->name, require->location}); + result.requireList.push_back({info->name, require->location}); ModuleInfo infoCopy = *info; // copy *info out since next line invalidates info! result.exprs[require] = std::move(infoCopy); @@ -151,7 +151,7 @@ struct RequireTracer : AstVisitor DenseHashMap locals; std::vector work; - std::vector requires; + std::vector requireCalls; }; RequireTraceResult traceRequires(FileResolver* fileResolver, AstStatBlock* root, const ModuleName& currentModuleName) diff --git a/Analysis/src/Substitution.cpp b/Analysis/src/Substitution.cpp index 5a22deeb..9c4ce829 100644 --- a/Analysis/src/Substitution.cpp +++ b/Analysis/src/Substitution.cpp @@ -27,7 +27,7 @@ void Tarjan::visitChildren(TypeId ty, int index) if (const FunctionTypeVar* ftv = get(ty)) { visitChild(ftv->argTypes); - visitChild(ftv->retType); + visitChild(ftv->retTypes); } else if (const TableTypeVar* ttv = get(ty)) { @@ -442,7 +442,7 @@ void Substitution::replaceChildren(TypeId ty) if (FunctionTypeVar* ftv = getMutable(ty)) { ftv->argTypes = replace(ftv->argTypes); - ftv->retType = replace(ftv->retType); + ftv->retTypes = replace(ftv->retTypes); } else if (TableTypeVar* ttv = getMutable(ty)) { diff --git a/Analysis/src/ToDot.cpp b/Analysis/src/ToDot.cpp index 9b396c80..6b677bb8 100644 --- a/Analysis/src/ToDot.cpp +++ b/Analysis/src/ToDot.cpp @@ -154,7 +154,7 @@ void StateDot::visitChildren(TypeId ty, int index) finishNode(); visitChild(ftv->argTypes, index, "arg"); - visitChild(ftv->retType, index, "ret"); + visitChild(ftv->retTypes, index, "ret"); } else if (const TableTypeVar* ttv = get(ty)) { diff --git a/Analysis/src/ToString.cpp b/Analysis/src/ToString.cpp index 04d15cf7..81dc0467 100644 --- a/Analysis/src/ToString.cpp +++ b/Analysis/src/ToString.cpp @@ -226,6 +226,11 @@ struct StringifierState result.name += s; } + void emit(int i) + { + emit(std::to_string(i).c_str()); + } + void indent() { indentation += 4; @@ -394,6 +399,13 @@ struct TypeVarStringifier state.emit("]]"); } + void operator()(TypeId, const BlockedTypeVar& btv) + { + state.emit("*blocked-"); + state.emit(btv.index); + state.emit("*"); + } + void operator()(TypeId, const PrimitiveTypeVar& ptv) { switch (ptv.type) @@ -480,8 +492,8 @@ struct TypeVarStringifier if (FFlag::LuauLowerBoundsCalculation) { - auto retBegin = begin(ftv.retType); - auto retEnd = end(ftv.retType); + auto retBegin = begin(ftv.retTypes); + auto retEnd = end(ftv.retTypes); if (retBegin != retEnd) { ++retBegin; @@ -491,7 +503,7 @@ struct TypeVarStringifier } else { - if (auto retPack = get(follow(ftv.retType))) + if (auto retPack = get(follow(ftv.retTypes))) { if (retPack->head.size() == 1 && !retPack->tail) plural = false; @@ -501,7 +513,7 @@ struct TypeVarStringifier if (plural) state.emit("("); - stringify(ftv.retType); + stringify(ftv.retTypes); if (plural) state.emit(")"); @@ -1303,14 +1315,14 @@ std::string toStringNamedFunction(const std::string& funcName, const FunctionTyp state.emit("): "); - size_t retSize = size(ftv.retType); - bool hasTail = !finite(ftv.retType); - bool wrap = get(follow(ftv.retType)) && (hasTail ? retSize != 0 : retSize != 1); + size_t retSize = size(ftv.retTypes); + bool hasTail = !finite(ftv.retTypes); + bool wrap = get(follow(ftv.retTypes)) && (hasTail ? retSize != 0 : retSize != 1); if (wrap) state.emit("("); - tvs.stringify(ftv.retType); + tvs.stringify(ftv.retTypes); if (wrap) state.emit(")"); @@ -1385,9 +1397,9 @@ std::string toString(const Constraint& c, ToStringOptions& opts) } else if (const GeneralizationConstraint* gc = Luau::get_if(&c.c)) { - ToStringResult subStr = toStringDetailed(gc->subType, opts); + ToStringResult subStr = toStringDetailed(gc->generalizedType, opts); opts.nameMap = std::move(subStr.nameMap); - ToStringResult superStr = toStringDetailed(gc->superType, opts); + ToStringResult superStr = toStringDetailed(gc->sourceType, opts); opts.nameMap = std::move(superStr.nameMap); return subStr.name + " ~ gen " + superStr.name; } diff --git a/Analysis/src/TypeAttach.cpp b/Analysis/src/TypeAttach.cpp index 0f4534b7..6cca7127 100644 --- a/Analysis/src/TypeAttach.cpp +++ b/Analysis/src/TypeAttach.cpp @@ -94,6 +94,11 @@ public: } } + AstType* operator()(const BlockedTypeVar& btv) + { + return allocator->alloc(Location(), std::nullopt, AstName("*blocked*")); + } + AstType* operator()(const ConstrainedTypeVar& ctv) { AstArray types; @@ -271,7 +276,7 @@ public: } AstArray returnTypes; - const auto& [retVector, retTail] = flatten(ftv.retType); + const auto& [retVector, retTail] = flatten(ftv.retTypes); returnTypes.size = retVector.size(); returnTypes.data = static_cast(allocator->allocate(sizeof(AstType*) * returnTypes.size)); for (size_t i = 0; i < returnTypes.size; ++i) diff --git a/Analysis/src/TypeChecker2.cpp b/Analysis/src/TypeChecker2.cpp new file mode 100644 index 00000000..7f5ba683 --- /dev/null +++ b/Analysis/src/TypeChecker2.cpp @@ -0,0 +1,160 @@ + +#include "Luau/TypeChecker2.h" + +#include + +#include "Luau/Ast.h" +#include "Luau/AstQuery.h" +#include "Luau/Clone.h" +#include "Luau/Normalize.h" + +namespace Luau +{ + +struct TypeChecker2 : public AstVisitor +{ + const SourceModule* sourceModule; + Module* module; + InternalErrorReporter ice; // FIXME accept a pointer from Frontend + + TypeChecker2(const SourceModule* sourceModule, Module* module) + : sourceModule(sourceModule) + , module(module) + { + } + + using AstVisitor::visit; + + TypePackId lookupPack(AstExpr* expr) + { + TypePackId* tp = module->astTypePacks.find(expr); + LUAU_ASSERT(tp); + return follow(*tp); + } + + TypeId lookupType(AstExpr* expr) + { + TypeId* ty = module->astTypes.find(expr); + LUAU_ASSERT(ty); + return follow(*ty); + } + + bool visit(AstStatAssign* assign) override + { + size_t count = std::min(assign->vars.size, assign->values.size); + + for (size_t i = 0; i < count; ++i) + { + AstExpr* lhs = assign->vars.data[i]; + TypeId* lhsType = module->astTypes.find(lhs); + LUAU_ASSERT(lhsType); + + AstExpr* rhs = assign->values.data[i]; + TypeId* rhsType = module->astTypes.find(rhs); + LUAU_ASSERT(rhsType); + + if (!isSubtype(*rhsType, *lhsType, ice)) + { + reportError(TypeMismatch{*lhsType, *rhsType}, rhs->location); + } + } + + return true; + } + + bool visit(AstExprCall* call) override + { + TypePackId expectedRetType = lookupPack(call); + TypeId functionType = lookupType(call->func); + + TypeArena arena; + TypePack args; + for (const auto& arg : call->args) + { + TypeId argTy = module->astTypes[arg]; + LUAU_ASSERT(argTy); + args.head.push_back(argTy); + } + + TypePackId argsTp = arena.addTypePack(args); + FunctionTypeVar ftv{argsTp, expectedRetType}; + TypeId expectedType = arena.addType(ftv); + if (!isSubtype(expectedType, functionType, ice)) + { + unfreeze(module->interfaceTypes); + CloneState cloneState; + expectedType = clone(expectedType, module->interfaceTypes, cloneState); + freeze(module->interfaceTypes); + reportError(TypeMismatch{expectedType, functionType}, call->location); + } + + return true; + } + + bool visit(AstExprIndexName* indexName) override + { + TypeId leftType = lookupType(indexName->expr); + TypeId resultType = lookupType(indexName); + + // leftType must have a property called indexName->index + + if (auto ttv = get(leftType)) + { + auto it = ttv->props.find(indexName->index.value); + if (it == ttv->props.end()) + { + reportError(UnknownProperty{leftType, indexName->index.value}, indexName->location); + } + else if (!isSubtype(resultType, it->second.type, ice)) + { + reportError(TypeMismatch{resultType, it->second.type}, indexName->location); + } + } + else + { + reportError(UnknownProperty{leftType, indexName->index.value}, indexName->location); + } + + return true; + } + + bool visit(AstExprConstantNumber* number) override + { + TypeId actualType = lookupType(number); + TypeId numberType = getSingletonTypes().numberType; + + if (!isSubtype(actualType, numberType, ice)) + { + reportError(TypeMismatch{actualType, numberType}, number->location); + } + + return true; + } + + bool visit(AstExprConstantString* string) override + { + TypeId actualType = lookupType(string); + TypeId stringType = getSingletonTypes().stringType; + + if (!isSubtype(actualType, stringType, ice)) + { + reportError(TypeMismatch{actualType, stringType}, string->location); + } + + return true; + } + + void reportError(TypeErrorData&& data, const Location& location) + { + module->errors.emplace_back(location, sourceModule->name, std::move(data)); + } +}; + +void check(const SourceModule& sourceModule, Module* module) +{ + TypeChecker2 typeChecker{&sourceModule, module}; + + sourceModule.root->visit(&typeChecker); +} + +} // namespace Luau diff --git a/Analysis/src/TypeInfer.cpp b/Analysis/src/TypeInfer.cpp index 447cd029..fd1b3b85 100644 --- a/Analysis/src/TypeInfer.cpp +++ b/Analysis/src/TypeInfer.cpp @@ -18,6 +18,7 @@ #include "Luau/TypePack.h" #include "Luau/TypeUtils.h" #include "Luau/TypeVar.h" +#include "Luau/VisitTypeVar.h" #include #include @@ -30,7 +31,6 @@ LUAU_FASTINTVARIABLE(LuauCheckRecursionLimit, 300) LUAU_FASTINTVARIABLE(LuauVisitRecursionLimit, 500) LUAU_FASTFLAG(LuauKnowsTheDataModel3) LUAU_FASTFLAG(LuauAutocompleteDynamicLimits) -LUAU_FASTFLAGVARIABLE(LuauExpectedPropTypeFromIndexer, false) LUAU_FASTFLAGVARIABLE(LuauLowerBoundsCalculation, false) LUAU_FASTFLAGVARIABLE(DebugLuauFreezeDuringUnification, false) LUAU_FASTFLAGVARIABLE(LuauSelfCallAutocompleteFix2, false) @@ -42,9 +42,9 @@ LUAU_FASTFLAG(LuauNormalizeFlagIsConservative) LUAU_FASTFLAGVARIABLE(LuauReturnTypeInferenceInNonstrict, false) LUAU_FASTFLAGVARIABLE(LuauRecursionLimitException, false); LUAU_FASTFLAGVARIABLE(LuauApplyTypeFunctionFix, false); -LUAU_FASTFLAGVARIABLE(LuauSuccessTypingForEqualityOperations, false) LUAU_FASTFLAGVARIABLE(LuauAlwaysQuantify, false); LUAU_FASTFLAGVARIABLE(LuauReportErrorsOnIndexerKeyMismatch, false) +LUAU_FASTFLAG(LuauQuantifyConstrained) LUAU_FASTFLAGVARIABLE(LuauFalsyPredicateReturnsNilInstead, false) LUAU_FASTFLAGVARIABLE(LuauNonCopyableTypeVarFields, false) @@ -260,7 +260,6 @@ TypeChecker::TypeChecker(ModuleResolver* resolver, InternalErrorReporter* iceHan , booleanType(getSingletonTypes().booleanType) , threadType(getSingletonTypes().threadType) , anyType(getSingletonTypes().anyType) - , optionalNumberType(getSingletonTypes().optionalNumberType) , anyTypePack(getSingletonTypes().anyTypePack) , duplicateTypeAliases{{false, {}}} { @@ -679,7 +678,7 @@ static std::optional tryGetTypeGuardPredicate(const AstExprBinary& ex void TypeChecker::check(const ScopePtr& scope, const AstStatIf& statement) { - ExprResult result = checkExpr(scope, *statement.condition); + WithPredicate result = checkExpr(scope, *statement.condition); ScopePtr ifScope = childScope(scope, statement.thenbody->location); resolve(result.predicates, ifScope, true); @@ -712,7 +711,7 @@ ErrorVec TypeChecker::canUnify(TypePackId subTy, TypePackId superTy, const Locat void TypeChecker::check(const ScopePtr& scope, const AstStatWhile& statement) { - ExprResult result = checkExpr(scope, *statement.condition); + WithPredicate result = checkExpr(scope, *statement.condition); ScopePtr whileScope = childScope(scope, statement.body->location); resolve(result.predicates, whileScope, true); @@ -728,16 +727,64 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatRepeat& statement) checkExpr(repScope, *statement.condition); } -void TypeChecker::unifyLowerBound(TypePackId subTy, TypePackId superTy, const Location& location) +void TypeChecker::unifyLowerBound(TypePackId subTy, TypePackId superTy, TypeLevel demotedLevel, const Location& location) { Unifier state = mkUnifier(location); - state.unifyLowerBound(subTy, superTy); + state.unifyLowerBound(subTy, superTy, demotedLevel); state.log.commit(); reportErrors(state.errors); } +struct Demoter : Substitution +{ + Demoter(TypeArena* arena) + : Substitution(TxnLog::empty(), arena) + { + } + + bool isDirty(TypeId ty) override + { + return get(ty); + } + + bool isDirty(TypePackId tp) override + { + return get(tp); + } + + TypeId clean(TypeId ty) override + { + auto ftv = get(ty); + LUAU_ASSERT(ftv); + return addType(FreeTypeVar{demotedLevel(ftv->level)}); + } + + TypePackId clean(TypePackId tp) override + { + auto ftp = get(tp); + LUAU_ASSERT(ftp); + return addTypePack(TypePackVar{FreeTypePack{demotedLevel(ftp->level)}}); + } + + TypeLevel demotedLevel(TypeLevel level) + { + return TypeLevel{level.level + 5000, level.subLevel}; + } + + void demote(std::vector>& expectedTypes) + { + if (!FFlag::LuauQuantifyConstrained) + return; + for (std::optional& ty : expectedTypes) + { + if (ty) + ty = substitute(*ty); + } + } +}; + void TypeChecker::check(const ScopePtr& scope, const AstStatReturn& return_) { std::vector> expectedTypes; @@ -760,11 +807,14 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatReturn& return_) } } + Demoter demoter{¤tModule->internalTypes}; + demoter.demote(expectedTypes); + TypePackId retPack = checkExprList(scope, return_.location, return_.list, false, {}, expectedTypes).type; if (FFlag::LuauReturnTypeInferenceInNonstrict ? FFlag::LuauLowerBoundsCalculation : useConstrainedIntersections()) { - unifyLowerBound(retPack, scope->returnType, return_.location); + unifyLowerBound(retPack, scope->returnType, demoter.demotedLevel(scope->level), return_.location); return; } @@ -1230,7 +1280,7 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatForIn& forin) unify(retPack, varPack, forin.location); } else - unify(iterFunc->retType, varPack, forin.location); + unify(iterFunc->retTypes, varPack, forin.location); check(loopScope, *forin.body); } @@ -1611,7 +1661,7 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatDeclareFunction& glo currentModule->getModuleScope()->bindings[global.name] = Binding{fnType, global.location}; } -ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExpr& expr, std::optional expectedType, bool forceSingleton) +WithPredicate TypeChecker::checkExpr(const ScopePtr& scope, const AstExpr& expr, std::optional expectedType, bool forceSingleton) { RecursionCounter _rc(&checkRecursionCount); if (FInt::LuauCheckRecursionLimit > 0 && checkRecursionCount >= FInt::LuauCheckRecursionLimit) @@ -1620,7 +1670,7 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExpr& return {errorRecoveryType(scope)}; } - ExprResult result; + WithPredicate result; if (auto a = expr.as()) result = checkExpr(scope, *a->expr, expectedType); @@ -1682,7 +1732,7 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExpr& return result; } -ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprLocal& expr) +WithPredicate TypeChecker::checkExpr(const ScopePtr& scope, const AstExprLocal& expr) { std::optional lvalue = tryGetLValue(expr); LUAU_ASSERT(lvalue); // Guaranteed to not be nullopt - AstExprLocal is an LValue. @@ -1696,7 +1746,7 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprLo return {errorRecoveryType(scope)}; } -ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprGlobal& expr) +WithPredicate TypeChecker::checkExpr(const ScopePtr& scope, const AstExprGlobal& expr) { std::optional lvalue = tryGetLValue(expr); LUAU_ASSERT(lvalue); // Guaranteed to not be nullopt - AstExprGlobal is an LValue. @@ -1708,7 +1758,7 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprGl return {errorRecoveryType(scope)}; } -ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprVarargs& expr) +WithPredicate TypeChecker::checkExpr(const ScopePtr& scope, const AstExprVarargs& expr) { TypePackId varargPack = checkExprPack(scope, expr).type; @@ -1738,9 +1788,9 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprVa ice("Unknown TypePack type in checkExpr(AstExprVarargs)!"); } -ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprCall& expr) +WithPredicate TypeChecker::checkExpr(const ScopePtr& scope, const AstExprCall& expr) { - ExprResult result = checkExprPack(scope, expr); + WithPredicate result = checkExprPack(scope, expr); TypePackId retPack = follow(result.type); if (auto pack = get(retPack)) @@ -1770,7 +1820,7 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprCa ice("Unknown TypePack type!", expr.location); } -ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprIndexName& expr) +WithPredicate TypeChecker::checkExpr(const ScopePtr& scope, const AstExprIndexName& expr) { Name name = expr.index.value; @@ -2031,7 +2081,7 @@ TypeId TypeChecker::stripFromNilAndReport(TypeId ty, const Location& location) return ty; } -ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprIndexExpr& expr) +WithPredicate TypeChecker::checkExpr(const ScopePtr& scope, const AstExprIndexExpr& expr) { TypeId ty = checkLValue(scope, expr); @@ -2042,7 +2092,7 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprIn return {ty}; } -ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprFunction& expr, std::optional expectedType) +WithPredicate TypeChecker::checkExpr(const ScopePtr& scope, const AstExprFunction& expr, std::optional expectedType) { auto [funTy, funScope] = checkFunctionSignature(scope, 0, expr, std::nullopt, expectedType); @@ -2108,8 +2158,7 @@ TypeId TypeChecker::checkExprTable( if (errors.empty()) exprType = expectedProp.type; } - else if (expectedTable->indexer && (FFlag::LuauExpectedPropTypeFromIndexer ? maybeString(expectedTable->indexer->indexType) - : isString(expectedTable->indexer->indexType))) + else if (expectedTable->indexer && maybeString(expectedTable->indexer->indexType)) { ErrorVec errors = tryUnify(exprType, expectedTable->indexer->indexResultType, k->location); if (errors.empty()) @@ -2147,7 +2196,7 @@ TypeId TypeChecker::checkExprTable( return addType(table); } -ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprTable& expr, std::optional expectedType) +WithPredicate TypeChecker::checkExpr(const ScopePtr& scope, const AstExprTable& expr, std::optional expectedType) { RecursionCounter _rc(&checkRecursionCount); if (FInt::LuauCheckRecursionLimit > 0 && checkRecursionCount >= FInt::LuauCheckRecursionLimit) @@ -2201,7 +2250,7 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprTa { if (auto prop = expectedTable->props.find(key->value.data); prop != expectedTable->props.end()) expectedResultType = prop->second.type; - else if (FFlag::LuauExpectedPropTypeFromIndexer && expectedIndexType && maybeString(*expectedIndexType)) + else if (expectedIndexType && maybeString(*expectedIndexType)) expectedResultType = expectedIndexResultType; } else if (expectedUnion) @@ -2236,9 +2285,9 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprTa return {checkExprTable(scope, expr, fieldTypes, expectedType)}; } -ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprUnary& expr) +WithPredicate TypeChecker::checkExpr(const ScopePtr& scope, const AstExprUnary& expr) { - ExprResult result = checkExpr(scope, *expr.expr); + WithPredicate result = checkExpr(scope, *expr.expr); TypeId operandType = follow(result.type); switch (expr.op) @@ -2466,62 +2515,50 @@ TypeId TypeChecker::checkRelationalOperation( std::optional leftMetatable = isString(lhsType) ? std::nullopt : getMetatable(follow(lhsType)); std::optional rightMetatable = isString(rhsType) ? std::nullopt : getMetatable(follow(rhsType)); - if (FFlag::LuauSuccessTypingForEqualityOperations) + if (leftMetatable != rightMetatable) { - if (leftMetatable != rightMetatable) + bool matches = false; + if (isEquality) { - bool matches = false; - if (isEquality) + if (const UnionTypeVar* utv = get(leftType); utv && rightMetatable) { - if (const UnionTypeVar* utv = get(leftType); utv && rightMetatable) + for (TypeId leftOption : utv) { - for (TypeId leftOption : utv) + if (getMetatable(follow(leftOption)) == rightMetatable) { - if (getMetatable(follow(leftOption)) == rightMetatable) + matches = true; + break; + } + } + } + + if (!matches) + { + if (const UnionTypeVar* utv = get(rhsType); utv && leftMetatable) + { + for (TypeId rightOption : utv) + { + if (getMetatable(follow(rightOption)) == leftMetatable) { matches = true; break; } } } - - if (!matches) - { - if (const UnionTypeVar* utv = get(rhsType); utv && leftMetatable) - { - for (TypeId rightOption : utv) - { - if (getMetatable(follow(rightOption)) == leftMetatable) - { - matches = true; - break; - } - } - } - } - } - - - if (!matches) - { - reportError( - expr.location, GenericError{format("Types %s and %s cannot be compared with %s because they do not have the same metatable", - toString(lhsType).c_str(), toString(rhsType).c_str(), toString(expr.op).c_str())}); - return errorRecoveryType(booleanType); } } - } - else - { - if (bool(leftMetatable) != bool(rightMetatable) && leftMetatable != rightMetatable) + + + if (!matches) { reportError( expr.location, GenericError{format("Types %s and %s cannot be compared with %s because they do not have the same metatable", - toString(lhsType).c_str(), toString(rhsType).c_str(), toString(expr.op).c_str())}); + toString(lhsType).c_str(), toString(rhsType).c_str(), toString(expr.op).c_str())}); return errorRecoveryType(booleanType); } } + if (leftMetatable) { std::optional metamethod = findMetatableEntry(lhsType, metamethodName, expr.location); @@ -2532,7 +2569,7 @@ TypeId TypeChecker::checkRelationalOperation( if (isEquality) { Unifier state = mkUnifier(expr.location); - state.tryUnify(addTypePack({booleanType}), ftv->retType); + state.tryUnify(addTypePack({booleanType}), ftv->retTypes); if (!state.errors.empty()) { @@ -2721,7 +2758,7 @@ TypeId TypeChecker::checkBinaryOperation( } } -ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprBinary& expr) +WithPredicate TypeChecker::checkExpr(const ScopePtr& scope, const AstExprBinary& expr) { if (expr.op == AstExprBinary::And) { @@ -2752,8 +2789,8 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprBi if (auto predicate = tryGetTypeGuardPredicate(expr)) return {booleanType, {std::move(*predicate)}}; - ExprResult lhs = checkExpr(scope, *expr.left, std::nullopt, /*forceSingleton=*/true); - ExprResult rhs = checkExpr(scope, *expr.right, std::nullopt, /*forceSingleton=*/true); + WithPredicate lhs = checkExpr(scope, *expr.left, std::nullopt, /*forceSingleton=*/true); + WithPredicate rhs = checkExpr(scope, *expr.right, std::nullopt, /*forceSingleton=*/true); PredicateVec predicates; @@ -2770,18 +2807,18 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprBi } else { - ExprResult lhs = checkExpr(scope, *expr.left); - ExprResult rhs = checkExpr(scope, *expr.right); + WithPredicate lhs = checkExpr(scope, *expr.left); + WithPredicate rhs = checkExpr(scope, *expr.right); // Intentionally discarding predicates with other operators. return {checkBinaryOperation(scope, expr, lhs.type, rhs.type, lhs.predicates)}; } } -ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprTypeAssertion& expr) +WithPredicate TypeChecker::checkExpr(const ScopePtr& scope, const AstExprTypeAssertion& expr) { TypeId annotationType = resolveType(scope, *expr.annotation); - ExprResult result = checkExpr(scope, *expr.expr, annotationType); + WithPredicate result = checkExpr(scope, *expr.expr, annotationType); // Note: As an optimization, we try 'number <: number | string' first, as that is the more likely case. if (canUnify(annotationType, result.type, expr.location).empty()) @@ -2794,7 +2831,7 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprTy return {errorRecoveryType(annotationType), std::move(result.predicates)}; } -ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprError& expr) +WithPredicate TypeChecker::checkExpr(const ScopePtr& scope, const AstExprError& expr) { const size_t oldSize = currentModule->errors.size(); @@ -2808,17 +2845,17 @@ ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprEr return {errorRecoveryType(scope)}; } -ExprResult TypeChecker::checkExpr(const ScopePtr& scope, const AstExprIfElse& expr, std::optional expectedType) +WithPredicate TypeChecker::checkExpr(const ScopePtr& scope, const AstExprIfElse& expr, std::optional expectedType) { - ExprResult result = checkExpr(scope, *expr.condition); + WithPredicate result = checkExpr(scope, *expr.condition); ScopePtr trueScope = childScope(scope, expr.trueExpr->location); resolve(result.predicates, trueScope, true); - ExprResult trueType = checkExpr(trueScope, *expr.trueExpr, expectedType); + WithPredicate trueType = checkExpr(trueScope, *expr.trueExpr, expectedType); ScopePtr falseScope = childScope(scope, expr.falseExpr->location); resolve(result.predicates, falseScope, false); - ExprResult falseType = checkExpr(falseScope, *expr.falseExpr, expectedType); + WithPredicate falseType = checkExpr(falseScope, *expr.falseExpr, expectedType); if (falseType.type == trueType.type) return {trueType.type}; @@ -3170,7 +3207,7 @@ std::pair TypeChecker::checkFunctionSignature( retPack = anyTypePack; else if (expectedFunctionType) { - auto [head, tail] = flatten(expectedFunctionType->retType); + auto [head, tail] = flatten(expectedFunctionType->retTypes); // Do not infer 'nil' as function return type if (!tail && head.size() == 1 && isNil(head[0])) @@ -3354,7 +3391,7 @@ void TypeChecker::checkFunctionBody(const ScopePtr& scope, TypeId ty, const AstE if (useConstrainedIntersections()) { - TypePackId retPack = follow(funTy->retType); + TypePackId retPack = follow(funTy->retTypes); // It is possible for a function to have no annotation and no return statement, and yet still have an ascribed return type // if it is expected to conform to some other interface. (eg the function may be a lambda passed as a callback) if (!hasReturn(function.body) && !function.returnAnnotation.has_value() && get(retPack)) @@ -3367,20 +3404,20 @@ void TypeChecker::checkFunctionBody(const ScopePtr& scope, TypeId ty, const AstE else { // We explicitly don't follow here to check if we have a 'true' free type instead of bound one - if (get_if(&funTy->retType->ty)) - *asMutable(funTy->retType) = TypePack{{}, std::nullopt}; + if (get_if(&funTy->retTypes->ty)) + *asMutable(funTy->retTypes) = TypePack{{}, std::nullopt}; } bool reachesImplicitReturn = getFallthrough(function.body) != nullptr; - if (reachesImplicitReturn && !allowsNoReturnValues(follow(funTy->retType))) + if (reachesImplicitReturn && !allowsNoReturnValues(follow(funTy->retTypes))) { // If we're in nonstrict mode we want to only report this missing return // statement if there are type annotations on the function. In strict mode // we report it regardless. if (!isNonstrictMode() || function.returnAnnotation) { - reportError(getEndLocation(function), FunctionExitsWithoutReturning{funTy->retType}); + reportError(getEndLocation(function), FunctionExitsWithoutReturning{funTy->retTypes}); } } } @@ -3388,7 +3425,7 @@ void TypeChecker::checkFunctionBody(const ScopePtr& scope, TypeId ty, const AstE ice("Checking non functional type"); } -ExprResult TypeChecker::checkExprPack(const ScopePtr& scope, const AstExpr& expr) +WithPredicate TypeChecker::checkExprPack(const ScopePtr& scope, const AstExpr& expr) { if (auto a = expr.as()) return checkExprPack(scope, *a); @@ -3654,7 +3691,7 @@ void TypeChecker::checkArgumentList( } } -ExprResult TypeChecker::checkExprPack(const ScopePtr& scope, const AstExprCall& expr) +WithPredicate TypeChecker::checkExprPack(const ScopePtr& scope, const AstExprCall& expr) { // evaluate type of function // decompose an intersection into its component overloads @@ -3722,7 +3759,7 @@ ExprResult TypeChecker::checkExprPack(const ScopePtr& scope, const A std::vector> expectedTypes = getExpectedTypesForCall(overloads, expr.args.size, expr.self); - ExprResult argListResult = checkExprList(scope, expr.location, expr.args, false, {}, expectedTypes); + WithPredicate argListResult = checkExprList(scope, expr.location, expr.args, false, {}, expectedTypes); TypePackId argPack = argListResult.type; if (get(argPack)) @@ -3766,7 +3803,7 @@ ExprResult TypeChecker::checkExprPack(const ScopePtr& scope, const A if (!overload && !overloadsThatDont.empty()) overload = get(overloadsThatDont[0]); if (overload) - return {errorRecoveryTypePack(overload->retType)}; + return {errorRecoveryTypePack(overload->retTypes)}; return {errorRecoveryTypePack(retPack)}; } @@ -3775,7 +3812,7 @@ std::vector> TypeChecker::getExpectedTypesForCall(const st { std::vector> expectedTypes; - auto assignOption = [this, &expectedTypes](size_t index, std::optional ty) { + auto assignOption = [this, &expectedTypes](size_t index, TypeId ty) { if (index == expectedTypes.size()) { expectedTypes.push_back(ty); @@ -3790,7 +3827,7 @@ std::vector> TypeChecker::getExpectedTypesForCall(const st } else { - std::vector result = reduceUnion({*el, *ty}); + std::vector result = reduceUnion({*el, ty}); el = result.size() == 1 ? result[0] : addType(UnionTypeVar{std::move(result)}); } } @@ -3810,7 +3847,8 @@ std::vector> TypeChecker::getExpectedTypesForCall(const st if (argsTail) { - if (const VariadicTypePack* vtp = get(follow(*argsTail))) + argsTail = follow(*argsTail); + if (const VariadicTypePack* vtp = get(*argsTail)) { while (index < argumentCount) assignOption(index++, vtp->ty); @@ -3819,11 +3857,14 @@ std::vector> TypeChecker::getExpectedTypesForCall(const st } } + Demoter demoter{¤tModule->internalTypes}; + demoter.demote(expectedTypes); + return expectedTypes; } -std::optional> TypeChecker::checkCallOverload(const ScopePtr& scope, const AstExprCall& expr, TypeId fn, TypePackId retPack, - TypePackId argPack, TypePack* args, const std::vector* argLocations, const ExprResult& argListResult, +std::optional> TypeChecker::checkCallOverload(const ScopePtr& scope, const AstExprCall& expr, TypeId fn, TypePackId retPack, + TypePackId argPack, TypePack* args, const std::vector* argLocations, const WithPredicate& argListResult, std::vector& overloadsThatMatchArgCount, std::vector& overloadsThatDont, std::vector& errors) { LUAU_ASSERT(argLocations); @@ -3918,14 +3959,14 @@ std::optional> TypeChecker::checkCallOverload(const Scope if (ftv->magicFunction) { // TODO: We're passing in the wrong TypePackId. Should be argPack, but a unit test fails otherwise. CLI-40458 - if (std::optional> ret = ftv->magicFunction(*this, scope, expr, argListResult)) + if (std::optional> ret = ftv->magicFunction(*this, scope, expr, argListResult)) return *ret; } Unifier state = mkUnifier(expr.location); // Unify return types - checkArgumentList(scope, state, retPack, ftv->retType, /*argLocations*/ {}); + checkArgumentList(scope, state, retPack, ftv->retTypes, /*argLocations*/ {}); if (!state.errors.empty()) { return {}; @@ -3996,7 +4037,7 @@ bool TypeChecker::handleSelfCallMismatch(const ScopePtr& scope, const AstExprCal // we eagerly assume that that's what you actually meant and we commit to it. // This could be incorrect if the function has an additional overload that // actually works. - // checkArgumentList(scope, editedState, retPack, ftv->retType, retLocations, CountMismatch::Return); + // checkArgumentList(scope, editedState, retPack, ftv->retTypes, retLocations, CountMismatch::Return); return true; } } @@ -4027,7 +4068,7 @@ bool TypeChecker::handleSelfCallMismatch(const ScopePtr& scope, const AstExprCal // we eagerly assume that that's what you actually meant and we commit to it. // This could be incorrect if the function has an additional overload that // actually works. - // checkArgumentList(scope, editedState, retPack, ftv->retType, retLocations, CountMismatch::Return); + // checkArgumentList(scope, editedState, retPack, ftv->retTypes, retLocations, CountMismatch::Return); return true; } } @@ -4085,7 +4126,7 @@ void TypeChecker::reportOverloadResolutionError(const ScopePtr& scope, const Ast // Unify return types if (const FunctionTypeVar* ftv = get(overload)) { - checkArgumentList(scope, state, retPack, ftv->retType, {}); + checkArgumentList(scope, state, retPack, ftv->retTypes, {}); checkArgumentList(scope, state, argPack, ftv->argTypes, argLocations); } @@ -4110,7 +4151,7 @@ void TypeChecker::reportOverloadResolutionError(const ScopePtr& scope, const Ast return; } -ExprResult TypeChecker::checkExprList(const ScopePtr& scope, const Location& location, const AstArray& exprs, +WithPredicate TypeChecker::checkExprList(const ScopePtr& scope, const Location& location, const AstArray& exprs, bool substituteFreeForNil, const std::vector& instantiateGenerics, const std::vector>& expectedTypes) { TypePackId pack = addTypePack(TypePack{}); @@ -4401,10 +4442,24 @@ TypeId Anyification::clean(TypeId ty) } else if (auto ctv = get(ty)) { - auto [t, ok] = normalize(ty, *arena, *iceHandler); - if (!ok) - normalizationTooComplex = true; - return t; + if (FFlag::LuauQuantifyConstrained) + { + std::vector copy = ctv->parts; + for (TypeId& ty : copy) + ty = replace(ty); + TypeId res = copy.size() == 1 ? copy[0] : addType(UnionTypeVar{std::move(copy)}); + auto [t, ok] = normalize(res, *arena, *iceHandler); + if (!ok) + normalizationTooComplex = true; + return t; + } + else + { + auto [t, ok] = normalize(ty, *arena, *iceHandler); + if (!ok) + normalizationTooComplex = true; + return t; + } } else return anyType; diff --git a/Analysis/src/TypeUtils.cpp b/Analysis/src/TypeUtils.cpp index ba09df5f..3d97e6eb 100644 --- a/Analysis/src/TypeUtils.cpp +++ b/Analysis/src/TypeUtils.cpp @@ -66,7 +66,7 @@ std::optional findTablePropertyRespectingMeta(ErrorVec& errors, TypeId t } else if (const auto& itf = get(index)) { - std::optional r = first(follow(itf->retType)); + std::optional r = first(follow(itf->retTypes)); if (!r) return getSingletonTypes().nilType; else diff --git a/Analysis/src/TypeVar.cpp b/Analysis/src/TypeVar.cpp index 33bfe254..57762937 100644 --- a/Analysis/src/TypeVar.cpp +++ b/Analysis/src/TypeVar.cpp @@ -29,8 +29,8 @@ LUAU_FASTFLAG(LuauNonCopyableTypeVarFields) namespace Luau { -std::optional> magicFunctionFormat( - TypeChecker& typechecker, const ScopePtr& scope, const AstExprCall& expr, ExprResult exprResult); +std::optional> magicFunctionFormat( + TypeChecker& typechecker, const ScopePtr& scope, const AstExprCall& expr, WithPredicate withPredicate); TypeId follow(TypeId t) { @@ -408,41 +408,48 @@ bool hasLength(TypeId ty, DenseHashSet& seen, int* recursionCount) return false; } -FunctionTypeVar::FunctionTypeVar(TypePackId argTypes, TypePackId retType, std::optional defn, bool hasSelf) +BlockedTypeVar::BlockedTypeVar() + : index(++nextIndex) +{ +} + +int BlockedTypeVar::nextIndex = 0; + +FunctionTypeVar::FunctionTypeVar(TypePackId argTypes, TypePackId retTypes, std::optional defn, bool hasSelf) : argTypes(argTypes) - , retType(retType) + , retTypes(retTypes) , definition(std::move(defn)) , hasSelf(hasSelf) { } -FunctionTypeVar::FunctionTypeVar(TypeLevel level, TypePackId argTypes, TypePackId retType, std::optional defn, bool hasSelf) +FunctionTypeVar::FunctionTypeVar(TypeLevel level, TypePackId argTypes, TypePackId retTypes, std::optional defn, bool hasSelf) : level(level) , argTypes(argTypes) - , retType(retType) + , retTypes(retTypes) , definition(std::move(defn)) , hasSelf(hasSelf) { } -FunctionTypeVar::FunctionTypeVar(std::vector generics, std::vector genericPacks, TypePackId argTypes, TypePackId retType, +FunctionTypeVar::FunctionTypeVar(std::vector generics, std::vector genericPacks, TypePackId argTypes, TypePackId retTypes, std::optional defn, bool hasSelf) : generics(generics) , genericPacks(genericPacks) , argTypes(argTypes) - , retType(retType) + , retTypes(retTypes) , definition(std::move(defn)) , hasSelf(hasSelf) { } FunctionTypeVar::FunctionTypeVar(TypeLevel level, std::vector generics, std::vector genericPacks, TypePackId argTypes, - TypePackId retType, std::optional defn, bool hasSelf) + TypePackId retTypes, std::optional defn, bool hasSelf) : level(level) , generics(generics) , genericPacks(genericPacks) , argTypes(argTypes) - , retType(retType) + , retTypes(retTypes) , definition(std::move(defn)) , hasSelf(hasSelf) { @@ -488,7 +495,7 @@ bool areEqual(SeenSet& seen, const FunctionTypeVar& lhs, const FunctionTypeVar& if (!areEqual(seen, *lhs.argTypes, *rhs.argTypes)) return false; - if (!areEqual(seen, *lhs.retType, *rhs.retType)) + if (!areEqual(seen, *lhs.retTypes, *rhs.retTypes)) return false; return true; @@ -678,7 +685,6 @@ static TypeVar trueType_{SingletonTypeVar{BooleanSingleton{true}}, /*persistent* static TypeVar falseType_{SingletonTypeVar{BooleanSingleton{false}}, /*persistent*/ true}; static TypeVar anyType_{AnyTypeVar{}, /*persistent*/ true}; static TypeVar errorType_{ErrorTypeVar{}, /*persistent*/ true}; -static TypeVar optionalNumberType_{UnionTypeVar{{&numberType_, &nilType_}}, /*persistent*/ true}; static TypePackVar anyTypePack_{VariadicTypePack{&anyType_}, true}; static TypePackVar errorTypePack_{Unifiable::Error{}}; @@ -692,7 +698,6 @@ SingletonTypes::SingletonTypes() , trueType(&trueType_) , falseType(&falseType_) , anyType(&anyType_) - , optionalNumberType(&optionalNumberType_) , anyTypePack(&anyTypePack_) , arena(new TypeArena) { @@ -825,7 +830,7 @@ void persist(TypeId ty) else if (auto ftv = get(t)) { persist(ftv->argTypes); - persist(ftv->retType); + persist(ftv->retTypes); } else if (auto ttv = get(t)) { @@ -1100,10 +1105,10 @@ static std::vector parseFormatString(TypeChecker& typechecker, const cha return result; } -std::optional> magicFunctionFormat( - TypeChecker& typechecker, const ScopePtr& scope, const AstExprCall& expr, ExprResult exprResult) +std::optional> magicFunctionFormat( + TypeChecker& typechecker, const ScopePtr& scope, const AstExprCall& expr, WithPredicate withPredicate) { - auto [paramPack, _predicates] = exprResult; + auto [paramPack, _predicates] = withPredicate; TypeArena& arena = typechecker.currentModule->internalTypes; @@ -1142,7 +1147,7 @@ std::optional> magicFunctionFormat( if (expected.size() != actualParamSize && (!tail || expected.size() < actualParamSize)) typechecker.reportError(TypeError{expr.location, CountMismatch{expected.size(), actualParamSize}}); - return ExprResult{arena.addTypePack({typechecker.stringType})}; + return WithPredicate{arena.addTypePack({typechecker.stringType})}; } std::vector filterMap(TypeId type, TypeIdPredicate predicate) diff --git a/Analysis/src/Unifier.cpp b/Analysis/src/Unifier.cpp index 414b05f4..877663de 100644 --- a/Analysis/src/Unifier.cpp +++ b/Analysis/src/Unifier.cpp @@ -22,6 +22,7 @@ LUAU_FASTFLAG(LuauLowerBoundsCalculation); LUAU_FASTFLAG(LuauErrorRecoveryType); LUAU_FASTFLAGVARIABLE(LuauSubtypingAddOptPropsToUnsealedTables, false) LUAU_FASTFLAGVARIABLE(LuauTxnLogRefreshFunctionPointers, false) +LUAU_FASTFLAG(LuauQuantifyConstrained) namespace Luau { @@ -1288,13 +1289,13 @@ void Unifier::tryUnifyFunctions(TypeId subTy, TypeId superTy, bool isFunctionCal reportError(TypeError{location, TypeMismatch{superTy, subTy, "", innerState.errors.front()}}); innerState.ctx = CountMismatch::Result; - innerState.tryUnify_(subFunction->retType, superFunction->retType); + innerState.tryUnify_(subFunction->retTypes, superFunction->retTypes); if (!reported) { if (auto e = hasUnificationTooComplex(innerState.errors)) reportError(*e); - else if (!innerState.errors.empty() && size(superFunction->retType) == 1 && finite(superFunction->retType)) + else if (!innerState.errors.empty() && size(superFunction->retTypes) == 1 && finite(superFunction->retTypes)) reportError(TypeError{location, TypeMismatch{superTy, subTy, "Return type is not compatible.", innerState.errors.front()}}); else if (!innerState.errors.empty() && innerState.firstPackErrorPos) reportError( @@ -1312,7 +1313,7 @@ void Unifier::tryUnifyFunctions(TypeId subTy, TypeId superTy, bool isFunctionCal tryUnify_(superFunction->argTypes, subFunction->argTypes, isFunctionCall); ctx = CountMismatch::Result; - tryUnify_(subFunction->retType, superFunction->retType); + tryUnify_(subFunction->retTypes, superFunction->retTypes); } if (FFlag::LuauTxnLogRefreshFunctionPointers) @@ -2177,7 +2178,7 @@ static void tryUnifyWithAny(std::vector& queue, Unifier& state, DenseHas else if (auto fun = state.log.getMutable(ty)) { queueTypePack(queue, seenTypePacks, state, fun->argTypes, anyTypePack); - queueTypePack(queue, seenTypePacks, state, fun->retType, anyTypePack); + queueTypePack(queue, seenTypePacks, state, fun->retTypes, anyTypePack); } else if (auto table = state.log.getMutable(ty)) { @@ -2322,7 +2323,7 @@ void Unifier::tryUnifyWithConstrainedSuperTypeVar(TypeId subTy, TypeId superTy) superC->parts.push_back(subTy); } -void Unifier::unifyLowerBound(TypePackId subTy, TypePackId superTy) +void Unifier::unifyLowerBound(TypePackId subTy, TypePackId superTy, TypeLevel demotedLevel) { // The duplication between this and regular typepack unification is tragic. @@ -2357,7 +2358,7 @@ void Unifier::unifyLowerBound(TypePackId subTy, TypePackId superTy) if (!freeTailPack) return; - TypeLevel level = freeTailPack->level; + TypeLevel level = FFlag::LuauQuantifyConstrained ? demotedLevel : freeTailPack->level; TypePack* tp = getMutable(log.replace(tailPack, TypePack{})); diff --git a/Compiler/src/BytecodeBuilder.cpp b/Compiler/src/BytecodeBuilder.cpp index 597b2f0a..a34f7603 100644 --- a/Compiler/src/BytecodeBuilder.cpp +++ b/Compiler/src/BytecodeBuilder.cpp @@ -1075,6 +1075,8 @@ void BytecodeBuilder::validate() const LUAU_ASSERT(i <= insns.size()); } + std::vector openCaptures; + // second pass: validate the rest of the bytecode for (size_t i = 0; i < insns.size();) { @@ -1121,6 +1123,8 @@ void BytecodeBuilder::validate() const case LOP_CLOSEUPVALS: VREG(LUAU_INSN_A(insn)); + while (openCaptures.size() && openCaptures.back() >= LUAU_INSN_A(insn)) + openCaptures.pop_back(); break; case LOP_GETIMPORT: @@ -1388,8 +1392,12 @@ void BytecodeBuilder::validate() const switch (LUAU_INSN_A(insn)) { case LCT_VAL: + VREG(LUAU_INSN_B(insn)); + break; + case LCT_REF: VREG(LUAU_INSN_B(insn)); + openCaptures.push_back(LUAU_INSN_B(insn)); break; case LCT_UPVAL: @@ -1409,6 +1417,12 @@ void BytecodeBuilder::validate() const LUAU_ASSERT(i <= insns.size()); } + // all CAPTURE REF instructions must have a CLOSEUPVALS instruction after them in the bytecode stream + // this doesn't guarantee safety as it doesn't perform basic block based analysis, but if this fails + // then the bytecode is definitely unsafe to run since the compiler won't generate backwards branches + // except for loop edges + LUAU_ASSERT(openCaptures.empty()); + #undef VREG #undef VREGEND #undef VUPVAL diff --git a/Compiler/src/Compiler.cpp b/Compiler/src/Compiler.cpp index 7431cde4..52dc9242 100644 --- a/Compiler/src/Compiler.cpp +++ b/Compiler/src/Compiler.cpp @@ -246,6 +246,14 @@ struct Compiler f.canInline = true; f.stackSize = stackSize; f.costModel = modelCost(func->body, func->args.data, func->args.size); + + // track functions that only ever return a single value so that we can convert multret calls to fixedret calls + if (allPathsEndWithReturn(func->body)) + { + ReturnVisitor returnVisitor(this); + stat->visit(&returnVisitor); + f.returnsOne = returnVisitor.returnsOne; + } } upvals.clear(); // note: instead of std::move above, we copy & clear to preserve capacity for future pushes @@ -260,6 +268,19 @@ struct Compiler { if (AstExprCall* expr = node->as()) { + // Optimization: convert multret calls to functions that always return one value to fixedret calls; this facilitates inlining + if (options.optimizationLevel >= 2) + { + AstExprFunction* func = getFunctionExpr(expr->func); + Function* fi = func ? functions.find(func) : nullptr; + + if (fi && fi->returnsOne) + { + compileExprTemp(node, target); + return false; + } + } + // We temporarily swap out regTop to have targetTop work correctly... // This is a crude hack but it's necessary for correctness :( RegScope rs(this, target); @@ -447,7 +468,9 @@ struct Compiler return false; } - // TODO: we can compile multret functions if all returns of the function are multret as well + // we can't inline multret functions because the caller expects L->top to be adjusted: + // - inlined return compiles to a JUMP, and we don't have an instruction that adjusts L->top arbitrarily + // - even if we did, right now all L->top adjustments are immediately consumed by the next instruction, and for now we want to preserve that if (multRet) { bytecode.addDebugRemark("inlining failed: can't convert fixed returns to multret"); @@ -492,7 +515,7 @@ struct Compiler size_t oldLocals = localStack.size(); // note that we push the frame early; this is needed to block recursive inline attempts - inlineFrames.push_back({func, target, targetCount}); + inlineFrames.push_back({func, oldLocals, target, targetCount}); // evaluate all arguments; note that we don't emit code for constant arguments (relying on constant folding) for (size_t i = 0; i < func->args.size; ++i) @@ -593,6 +616,8 @@ struct Compiler { for (size_t i = 0; i < targetCount; ++i) bytecode.emitABC(LOP_LOADNIL, uint8_t(target + i), 0, 0); + + closeLocals(oldLocals); } popLocals(oldLocals); @@ -2355,6 +2380,8 @@ struct Compiler compileExprListTemp(stat->list, frame.target, frame.targetCount, /* targetTop= */ false); + closeLocals(frame.localOffset); + if (!fallthrough) { size_t jumpLabel = bytecode.emitLabel(); @@ -3316,6 +3343,48 @@ struct Compiler std::vector upvals; }; + struct ReturnVisitor: AstVisitor + { + Compiler* self; + bool returnsOne = true; + + ReturnVisitor(Compiler* self) + : self(self) + { + } + + bool visit(AstExpr* expr) override + { + return false; + } + + bool visit(AstStatReturn* stat) override + { + if (stat->list.size == 1) + { + AstExpr* value = stat->list.data[0]; + + if (AstExprCall* expr = value->as()) + { + AstExprFunction* func = self->getFunctionExpr(expr->func); + Function* fi = func ? self->functions.find(func) : nullptr; + + returnsOne &= fi && fi->returnsOne; + } + else if (value->is()) + { + returnsOne = false; + } + } + else + { + returnsOne = false; + } + + return false; + } + }; + struct RegScope { RegScope(Compiler* self) @@ -3351,6 +3420,7 @@ struct Compiler uint64_t costModel = 0; unsigned int stackSize = 0; bool canInline = false; + bool returnsOne = false; }; struct Local @@ -3384,6 +3454,8 @@ struct Compiler { AstExprFunction* func; + size_t localOffset; + uint8_t target; uint8_t targetCount; diff --git a/Sources.cmake b/Sources.cmake index 99007e89..f261cba6 100644 --- a/Sources.cmake +++ b/Sources.cmake @@ -65,12 +65,13 @@ target_sources(Luau.CodeGen PRIVATE target_sources(Luau.Analysis PRIVATE Analysis/include/Luau/AstQuery.h Analysis/include/Luau/Autocomplete.h - Analysis/include/Luau/NotNull.h Analysis/include/Luau/BuiltinDefinitions.h Analysis/include/Luau/Clone.h Analysis/include/Luau/Config.h + Analysis/include/Luau/Constraint.h Analysis/include/Luau/ConstraintGraphBuilder.h Analysis/include/Luau/ConstraintSolver.h + Analysis/include/Luau/ConstraintSolverLogger.h Analysis/include/Luau/Documentation.h Analysis/include/Luau/Error.h Analysis/include/Luau/FileResolver.h @@ -97,6 +98,7 @@ target_sources(Luau.Analysis PRIVATE Analysis/include/Luau/TxnLog.h Analysis/include/Luau/TypeArena.h Analysis/include/Luau/TypeAttach.h + Analysis/include/Luau/TypeChecker2.h Analysis/include/Luau/TypedAllocator.h Analysis/include/Luau/TypeInfer.h Analysis/include/Luau/TypePack.h @@ -113,8 +115,10 @@ target_sources(Luau.Analysis PRIVATE Analysis/src/BuiltinDefinitions.cpp Analysis/src/Clone.cpp Analysis/src/Config.cpp + Analysis/src/Constraint.cpp Analysis/src/ConstraintGraphBuilder.cpp Analysis/src/ConstraintSolver.cpp + Analysis/src/ConstraintSolverLogger.cpp Analysis/src/Error.cpp Analysis/src/Frontend.cpp Analysis/src/Instantiation.cpp @@ -136,6 +140,7 @@ target_sources(Luau.Analysis PRIVATE Analysis/src/TxnLog.cpp Analysis/src/TypeArena.cpp Analysis/src/TypeAttach.cpp + Analysis/src/TypeChecker2.cpp Analysis/src/TypedAllocator.cpp Analysis/src/TypeInfer.cpp Analysis/src/TypePack.cpp @@ -245,7 +250,6 @@ if(TARGET Luau.UnitTest) tests/AstQuery.test.cpp tests/AstVisitor.test.cpp tests/Autocomplete.test.cpp - tests/NotNull.test.cpp tests/BuiltinDefinitions.test.cpp tests/Compiler.test.cpp tests/Config.test.cpp diff --git a/VM/src/lobject.h b/VM/src/lobject.h index 5e02c2ea..bdcb85cb 100644 --- a/VM/src/lobject.h +++ b/VM/src/lobject.h @@ -418,7 +418,7 @@ typedef struct Table CommonHeader; - uint8_t flags; /* 1<

tm_sec); setfield(L, "min", stm->tm_min); setfield(L, "hour", stm->tm_hour); @@ -122,7 +122,7 @@ static int os_date(lua_State* L) luaL_buffinit(L, &b); for (; *s; s++) { - if (*s != '%' || *(s + 1) == '\0') /* no conversion specifier? */ + if (*s != '%' || *(s + 1) == '\0') // no conversion specifier? { luaL_addchar(&b, *s); } @@ -133,7 +133,7 @@ static int os_date(lua_State* L) else { size_t reslen; - char buff[200]; /* should be big enough for any conversion result */ + char buff[200]; // should be big enough for any conversion result cc[1] = *(++s); reslen = strftime(buff, sizeof(buff), cc, stm); luaL_addlstring(&b, buff, reslen); @@ -147,13 +147,13 @@ static int os_date(lua_State* L) static int os_time(lua_State* L) { time_t t; - if (lua_isnoneornil(L, 1)) /* called without args? */ - t = time(NULL); /* get current time */ + if (lua_isnoneornil(L, 1)) // called without args? + t = time(NULL); // get current time else { struct tm ts; luaL_checktype(L, 1, LUA_TTABLE); - lua_settop(L, 1); /* make sure table is at the top */ + lua_settop(L, 1); // make sure table is at the top ts.tm_sec = getfield(L, "sec", 0); ts.tm_min = getfield(L, "min", 0); ts.tm_hour = getfield(L, "hour", 12); diff --git a/VM/src/lstate.cpp b/VM/src/lstate.cpp index fbc6fb1e..4489f840 100644 --- a/VM/src/lstate.cpp +++ b/VM/src/lstate.cpp @@ -21,22 +21,22 @@ typedef struct LG static void stack_init(lua_State* L1, lua_State* L) { - /* initialize CallInfo array */ + // initialize CallInfo array L1->base_ci = luaM_newarray(L, BASIC_CI_SIZE, CallInfo, L1->memcat); L1->ci = L1->base_ci; L1->size_ci = BASIC_CI_SIZE; L1->end_ci = L1->base_ci + L1->size_ci - 1; - /* initialize stack array */ + // initialize stack array L1->stack = luaM_newarray(L, BASIC_STACK_SIZE + EXTRA_STACK, TValue, L1->memcat); L1->stacksize = BASIC_STACK_SIZE + EXTRA_STACK; TValue* stack = L1->stack; for (int i = 0; i < BASIC_STACK_SIZE + EXTRA_STACK; i++) - setnilvalue(stack + i); /* erase new stack */ + setnilvalue(stack + i); // erase new stack L1->top = stack; L1->stack_last = stack + (L1->stacksize - EXTRA_STACK); - /* initialize first ci */ + // initialize first ci L1->ci->func = L1->top; - setnilvalue(L1->top++); /* `function' entry for this `ci' */ + setnilvalue(L1->top++); // `function' entry for this `ci' L1->base = L1->ci->base = L1->top; L1->ci->top = L1->top + LUA_MINSTACK; } @@ -53,13 +53,13 @@ static void freestack(lua_State* L, lua_State* L1) static void f_luaopen(lua_State* L, void* ud) { global_State* g = L->global; - stack_init(L, L); /* init stack */ - L->gt = luaH_new(L, 0, 2); /* table of globals */ - sethvalue(L, registry(L), luaH_new(L, 0, 2)); /* registry */ - luaS_resize(L, LUA_MINSTRTABSIZE); /* initial size of string table */ + stack_init(L, L); // init stack + L->gt = luaH_new(L, 0, 2); // table of globals + sethvalue(L, registry(L), luaH_new(L, 0, 2)); // registry + luaS_resize(L, LUA_MINSTRTABSIZE); // initial size of string table luaT_init(L); - luaS_fix(luaS_newliteral(L, LUA_MEMERRMSG)); /* pin to make sure we can always throw this error */ - luaS_fix(luaS_newliteral(L, LUA_ERRERRMSG)); /* pin to make sure we can always throw this error */ + luaS_fix(luaS_newliteral(L, LUA_MEMERRMSG)); // pin to make sure we can always throw this error + luaS_fix(luaS_newliteral(L, LUA_ERRERRMSG)); // pin to make sure we can always throw this error g->GCthreshold = 4 * g->totalbytes; } @@ -85,8 +85,8 @@ static void preinit_state(lua_State* L, global_State* g) static void close_state(lua_State* L) { global_State* g = L->global; - luaF_close(L, L->stack); /* close all upvalues for this thread */ - luaC_freeall(L); /* collect all objects */ + luaF_close(L, L->stack); // close all upvalues for this thread + luaC_freeall(L); // collect all objects LUAU_ASSERT(g->strbufgc == NULL); LUAU_ASSERT(g->strt.nuse == 0); luaM_freearray(L, L->global->strt.hash, L->global->strt.size, TString*, 0); @@ -110,8 +110,8 @@ lua_State* luaE_newthread(lua_State* L) luaC_init(L, L1, LUA_TTHREAD); preinit_state(L1, L->global); L1->activememcat = L->activememcat; // inherit the active memory category - stack_init(L1, L); /* init stack */ - L1->gt = L->gt; /* share table of globals */ + stack_init(L1, L); // init stack + L1->gt = L->gt; // share table of globals L1->singlestep = L->singlestep; LUAU_ASSERT(iswhite(obj2gco(L1))); return L1; @@ -119,7 +119,7 @@ lua_State* luaE_newthread(lua_State* L) void luaE_freethread(lua_State* L, lua_State* L1, lua_Page* page) { - luaF_close(L1, L1->stack); /* close all upvalues for this thread */ + luaF_close(L1, L1->stack); // close all upvalues for this thread LUAU_ASSERT(L1->openupval == NULL); global_State* g = L->global; if (g->cb.userthread) @@ -130,9 +130,9 @@ void luaE_freethread(lua_State* L, lua_State* L1, lua_Page* page) void lua_resetthread(lua_State* L) { - /* close upvalues before clearing anything */ + // close upvalues before clearing anything luaF_close(L, L->stack); - /* clear call frames */ + // clear call frames CallInfo* ci = L->base_ci; ci->func = L->stack; ci->base = ci->func + 1; @@ -141,12 +141,12 @@ void lua_resetthread(lua_State* L) L->ci = ci; if (L->size_ci != BASIC_CI_SIZE) luaD_reallocCI(L, BASIC_CI_SIZE); - /* clear thread state */ + // clear thread state L->status = LUA_OK; L->base = L->ci->base; L->top = L->ci->base; L->nCcalls = L->baseCcalls = 0; - /* clear thread stack */ + // clear thread stack if (L->stacksize != BASIC_STACK_SIZE + EXTRA_STACK) luaD_reallocstack(L, BASIC_STACK_SIZE); for (int i = 0; i < L->stacksize; i++) @@ -177,7 +177,7 @@ lua_State* lua_newstate(lua_Alloc f, void* ud) g->mainthread = L; g->uvhead.u.l.prev = &g->uvhead; g->uvhead.u.l.next = &g->uvhead; - g->GCthreshold = 0; /* mark it as unfinished state */ + g->GCthreshold = 0; // mark it as unfinished state g->registryfree = 0; g->errorjmp = NULL; g->rngstate = 0; @@ -224,7 +224,7 @@ lua_State* lua_newstate(lua_Alloc f, void* ud) if (luaD_rawrunprotected(L, f_luaopen, NULL) != 0) { - /* memory allocation error: free partial state */ + // memory allocation error: free partial state close_state(L); L = NULL; } @@ -233,7 +233,7 @@ lua_State* lua_newstate(lua_Alloc f, void* ud) void lua_close(lua_State* L) { - L = L->global->mainthread; /* only the main thread can be closed */ - luaF_close(L, L->stack); /* close all upvalues for this thread */ + L = L->global->mainthread; // only the main thread can be closed + luaF_close(L, L->stack); // close all upvalues for this thread close_state(L); } diff --git a/VM/src/lstate.h b/VM/src/lstate.h index 423514a7..72a09713 100644 --- a/VM/src/lstate.h +++ b/VM/src/lstate.h @@ -5,10 +5,10 @@ #include "lobject.h" #include "ltm.h" -/* registry */ +// registry #define registry(L) (&L->global->registry) -/* extra stack space to handle TM calls and some other extras */ +// extra stack space to handle TM calls and some other extras #define EXTRA_STACK 5 #define BASIC_CI_SIZE 8 @@ -20,7 +20,7 @@ typedef struct stringtable { TString** hash; - uint32_t nuse; /* number of elements */ + uint32_t nuse; // number of elements int size; } stringtable; // clang-format on @@ -57,18 +57,18 @@ typedef struct stringtable typedef struct CallInfo { - StkId base; /* base for this function */ - StkId func; /* function index in the stack */ - StkId top; /* top for this function */ + StkId base; // base for this function + StkId func; // function index in the stack + StkId top; // top for this function const Instruction* savedpc; - int nresults; /* expected number of results from this function */ - unsigned int flags; /* call frame flags, see LUA_CALLINFO_* */ + int nresults; // expected number of results from this function + unsigned int flags; // call frame flags, see LUA_CALLINFO_* } CallInfo; // clang-format on -#define LUA_CALLINFO_RETURN (1 << 0) /* should the interpreter return after returning from this callinfo? first frame must have this set */ -#define LUA_CALLINFO_HANDLE (1 << 1) /* should the error thrown during execution get handled by continuation from this callinfo? func must be C */ +#define LUA_CALLINFO_RETURN (1 << 0) // should the interpreter return after returning from this callinfo? first frame must have this set +#define LUA_CALLINFO_HANDLE (1 << 1) // should the error thrown during execution get handled by continuation from this callinfo? func must be C #define curr_func(L) (clvalue(L->ci->func)) #define ci_func(ci) (clvalue((ci)->func)) @@ -152,55 +152,55 @@ struct GCMetrics // clang-format off typedef struct global_State { - stringtable strt; /* hash table for strings */ + stringtable strt; // hash table for strings - lua_Alloc frealloc; /* function to reallocate memory */ - void* ud; /* auxiliary data to `frealloc' */ + lua_Alloc frealloc; // function to reallocate memory + void* ud; // auxiliary data to `frealloc' uint8_t currentwhite; - uint8_t gcstate; /* state of garbage collector */ + uint8_t gcstate; // state of garbage collector - GCObject* gray; /* list of gray objects */ - GCObject* grayagain; /* list of objects to be traversed atomically */ - GCObject* weak; /* list of weak tables (to be cleared) */ + GCObject* gray; // list of gray objects + GCObject* grayagain; // list of objects to be traversed atomically + GCObject* weak; // list of weak tables (to be cleared) TString* strbufgc; // list of all string buffer objects - size_t GCthreshold; // when totalbytes > GCthreshold; run GC step + size_t GCthreshold; // when totalbytes > GCthreshold, run GC step size_t totalbytes; // number of bytes currently allocated int gcgoal; // see LUAI_GCGOAL int gcstepmul; // see LUAI_GCSTEPMUL int gcstepsize; // see LUAI_GCSTEPSIZE struct lua_Page* freepages[LUA_SIZECLASSES]; // free page linked list for each size class for non-collectable objects - struct lua_Page* freegcopages[LUA_SIZECLASSES]; // free page linked list for each size class for collectable objects + struct lua_Page* freegcopages[LUA_SIZECLASSES]; // free page linked list for each size class for collectable objects struct lua_Page* allgcopages; // page linked list with all pages for all classes struct lua_Page* sweepgcopage; // position of the sweep in `allgcopages' - size_t memcatbytes[LUA_MEMORY_CATEGORIES]; /* total amount of memory used by each memory category */ + size_t memcatbytes[LUA_MEMORY_CATEGORIES]; // total amount of memory used by each memory category struct lua_State* mainthread; - UpVal uvhead; /* head of double-linked list of all open upvalues */ - struct Table* mt[LUA_T_COUNT]; /* metatables for basic types */ - TString* ttname[LUA_T_COUNT]; /* names for basic types */ - TString* tmname[TM_N]; /* array with tag-method names */ + UpVal uvhead; // head of double-linked list of all open upvalues + struct Table* mt[LUA_T_COUNT]; // metatables for basic types + TString* ttname[LUA_T_COUNT]; // names for basic types + TString* tmname[TM_N]; // array with tag-method names - TValue pseudotemp; /* storage for temporary values used in pseudo2addr */ + TValue pseudotemp; // storage for temporary values used in pseudo2addr - TValue registry; /* registry table, used by lua_ref and LUA_REGISTRYINDEX */ - int registryfree; /* next free slot in registry */ + TValue registry; // registry table, used by lua_ref and LUA_REGISTRYINDEX + int registryfree; // next free slot in registry - struct lua_jmpbuf* errorjmp; /* jump buffer data for longjmp-style error handling */ + struct lua_jmpbuf* errorjmp; // jump buffer data for longjmp-style error handling - uint64_t rngstate; /* PCG random number generator state */ - uint64_t ptrenckey[4]; /* pointer encoding key for display */ + uint64_t rngstate; // PCG random number generator state + uint64_t ptrenckey[4]; // pointer encoding key for display - void (*udatagc[LUA_UTAG_LIMIT])(lua_State*, void*); /* for each userdata tag, a gc callback to be called immediately before freeing memory */ + void (*udatagc[LUA_UTAG_LIMIT])(lua_State*, void*); // for each userdata tag, a gc callback to be called immediately before freeing memory lua_Callbacks cb; @@ -221,39 +221,39 @@ struct lua_State CommonHeader; uint8_t status; - uint8_t activememcat; /* memory category that is used for new GC object allocations */ + uint8_t activememcat; // memory category that is used for new GC object allocations uint8_t stackstate; - bool singlestep; /* call debugstep hook after each instruction */ + bool singlestep; // call debugstep hook after each instruction - StkId top; /* first free slot in the stack */ - StkId base; /* base of current function */ + StkId top; // first free slot in the stack + StkId base; // base of current function global_State* global; - CallInfo* ci; /* call info for current function */ - StkId stack_last; /* last free slot in the stack */ - StkId stack; /* stack base */ + CallInfo* ci; // call info for current function + StkId stack_last; // last free slot in the stack + StkId stack; // stack base - CallInfo* end_ci; /* points after end of ci array*/ - CallInfo* base_ci; /* array of CallInfo's */ + CallInfo* end_ci; // points after end of ci array + CallInfo* base_ci; // array of CallInfo's int stacksize; - int size_ci; /* size of array `base_ci' */ + int size_ci; // size of array `base_ci' - unsigned short nCcalls; /* number of nested C calls */ - unsigned short baseCcalls; /* nested C calls when resuming coroutine */ + unsigned short nCcalls; // number of nested C calls + unsigned short baseCcalls; // nested C calls when resuming coroutine - int cachedslot; /* when table operations or INDEX/NEWINDEX is invoked from Luau, what is the expected slot for lookup? */ + int cachedslot; // when table operations or INDEX/NEWINDEX is invoked from Luau, what is the expected slot for lookup? - Table* gt; /* table of globals */ - UpVal* openupval; /* list of open upvalues in this stack */ + Table* gt; // table of globals + UpVal* openupval; // list of open upvalues in this stack GCObject* gclist; - TString* namecall; /* when invoked from Luau using NAMECALL, what method do we need to invoke? */ + TString* namecall; // when invoked from Luau using NAMECALL, what method do we need to invoke? void* userdata; }; @@ -271,10 +271,10 @@ union GCObject struct Table h; struct Proto p; struct UpVal uv; - struct lua_State th; /* thread */ + struct lua_State th; // thread }; -/* macros to convert a GCObject into a specific value */ +// macros to convert a GCObject into a specific value #define gco2ts(o) check_exp((o)->gch.tt == LUA_TSTRING, &((o)->ts)) #define gco2u(o) check_exp((o)->gch.tt == LUA_TUSERDATA, &((o)->u)) #define gco2cl(o) check_exp((o)->gch.tt == LUA_TFUNCTION, &((o)->cl)) @@ -283,7 +283,7 @@ union GCObject #define gco2uv(o) check_exp((o)->gch.tt == LUA_TUPVAL, &((o)->uv)) #define gco2th(o) check_exp((o)->gch.tt == LUA_TTHREAD, &((o)->th)) -/* macro to convert any Lua object into a GCObject */ +// macro to convert any Lua object into a GCObject #define obj2gco(v) check_exp(iscollectable(v), cast_to(GCObject*, (v) + 0)) LUAI_FUNC lua_State* luaE_newthread(lua_State* L); diff --git a/VM/src/lstring.cpp b/VM/src/lstring.cpp index d454308c..9c266031 100644 --- a/VM/src/lstring.cpp +++ b/VM/src/lstring.cpp @@ -48,17 +48,17 @@ void luaS_resize(lua_State* L, int newsize) stringtable* tb = &L->global->strt; for (int i = 0; i < newsize; i++) newhash[i] = NULL; - /* rehash */ + // rehash for (int i = 0; i < tb->size; i++) { TString* p = tb->hash[i]; while (p) - { /* for each node in the list */ - TString* next = p->next; /* save next */ + { // for each node in the list + TString* next = p->next; // save next unsigned int h = p->hash; - int h1 = lmod(h, newsize); /* new position */ + int h1 = lmod(h, newsize); // new position LUAU_ASSERT(cast_int(h % newsize) == lmod(h, newsize)); - p->next = newhash[h1]; /* chain it */ + p->next = newhash[h1]; // chain it newhash[h1] = p; p = next; } @@ -81,15 +81,15 @@ static TString* newlstr(lua_State* L, const char* str, size_t l, unsigned int h) ts->tt = LUA_TSTRING; ts->memcat = L->activememcat; memcpy(ts->data, str, l); - ts->data[l] = '\0'; /* ending 0 */ + ts->data[l] = '\0'; // ending 0 ts->atom = ATOM_UNDEF; tb = &L->global->strt; h = lmod(h, tb->size); - ts->next = tb->hash[h]; /* chain new entry */ + ts->next = tb->hash[h]; // chain new entry tb->hash[h] = ts; tb->nuse++; if (tb->nuse > cast_to(uint32_t, tb->size) && tb->size <= INT_MAX / 2) - luaS_resize(L, tb->size * 2); /* too crowded */ + luaS_resize(L, tb->size * 2); // too crowded return ts; } @@ -181,13 +181,13 @@ TString* luaS_newlstr(lua_State* L, const char* str, size_t l) { if (el->len == l && (memcmp(str, getstr(el), l) == 0)) { - /* string may be dead */ + // string may be dead if (isdead(L->global, obj2gco(el))) changewhite(obj2gco(el)); return el; } } - return newlstr(L, str, l, h); /* not found */ + return newlstr(L, str, l, h); // not found } static bool unlinkstr(lua_State* L, TString* ts) diff --git a/VM/src/lstring.h b/VM/src/lstring.h index acdf9a1e..41f9df9a 100644 --- a/VM/src/lstring.h +++ b/VM/src/lstring.h @@ -5,10 +5,10 @@ #include "lobject.h" #include "lstate.h" -/* string size limit */ +// string size limit #define MAXSSIZE (1 << 30) -/* string atoms are not defined by default; the storage is 16-bit integer */ +// string atoms are not defined by default; the storage is 16-bit integer #define ATOM_UNDEF -32768 #define sizestring(len) (offsetof(TString, data) + len + 1) diff --git a/VM/src/lstrlib.cpp b/VM/src/lstrlib.cpp index b0cd4dc2..b3ea1094 100644 --- a/VM/src/lstrlib.cpp +++ b/VM/src/lstrlib.cpp @@ -8,7 +8,9 @@ #include #include -/* macro to `unsign' a character */ +LUAU_FASTFLAGVARIABLE(LuauTostringFormatSpecifier, false); + +// macro to `unsign' a character #define uchar(c) ((unsigned char)(c)) static int str_len(lua_State* L) @@ -21,7 +23,7 @@ static int str_len(lua_State* L) static int posrelat(int pos, size_t len) { - /* relative string position: negative means back from end */ + // relative string position: negative means back from end if (pos < 0) pos += (int)len + 1; return (pos >= 0) ? pos : 0; @@ -137,9 +139,9 @@ static int str_byte(lua_State* L) if ((size_t)pose > l) pose = (int)l; if (posi > pose) - return 0; /* empty interval; return no values */ + return 0; // empty interval; return no values n = (int)(pose - posi + 1); - if (posi + n <= pose) /* overflow? */ + if (posi + n <= pose) // overflow? luaL_error(L, "string slice too long"); luaL_checkstack(L, n, "string slice too long"); for (i = 0; i < n; i++) @@ -149,7 +151,7 @@ static int str_byte(lua_State* L) static int str_char(lua_State* L) { - int n = lua_gettop(L); /* number of arguments */ + int n = lua_gettop(L); // number of arguments luaL_Buffer b; char* ptr = luaL_buffinitsize(L, &b, n); @@ -176,12 +178,12 @@ static int str_char(lua_State* L) typedef struct MatchState { - int matchdepth; /* control for recursive depth (to avoid C stack overflow) */ - const char* src_init; /* init of source string */ - const char* src_end; /* end ('\0') of source string */ - const char* p_end; /* end ('\0') of pattern */ + int matchdepth; // control for recursive depth (to avoid C stack overflow) + const char* src_init; // init of source string + const char* src_end; // end ('\0') of source string + const char* p_end; // end ('\0') of pattern lua_State* L; - int level; /* total number of captures (finished or unfinished) */ + int level; // total number of captures (finished or unfinished) struct { const char* init; @@ -189,7 +191,7 @@ typedef struct MatchState } capture[LUA_MAXCAPTURES]; } MatchState; -/* recursive function */ +// recursive function static const char* match(MatchState* ms, const char* s, const char* p); #define L_ESC '%' @@ -227,11 +229,11 @@ static const char* classend(MatchState* ms, const char* p) if (*p == '^') p++; do - { /* look for a `]' */ + { // look for a `]' if (p == ms->p_end) luaL_error(ms->L, "malformed pattern (missing ']')"); if (*(p++) == L_ESC && p < ms->p_end) - p++; /* skip escapes (e.g. `%]') */ + p++; // skip escapes (e.g. `%]') } while (*p != ']'); return p + 1; } @@ -279,7 +281,7 @@ static int match_class(int c, int cl) break; case 'z': res = (c == 0); - break; /* deprecated option */ + break; // deprecated option default: return (cl == c); } @@ -292,7 +294,7 @@ static int matchbracketclass(int c, const char* p, const char* ec) if (*(p + 1) == '^') { sig = 0; - p++; /* skip the `^' */ + p++; // skip the `^' } while (++p < ec) { @@ -324,7 +326,7 @@ static int singlematch(MatchState* ms, const char* s, const char* p, const char* switch (*p) { case '.': - return 1; /* matches any char */ + return 1; // matches any char case L_ESC: return match_class(c, uchar(*(p + 1))); case '[': @@ -357,21 +359,21 @@ static const char* matchbalance(MatchState* ms, const char* s, const char* p) cont++; } } - return NULL; /* string ends out of balance */ + return NULL; // string ends out of balance } static const char* max_expand(MatchState* ms, const char* s, const char* p, const char* ep) { - ptrdiff_t i = 0; /* counts maximum expand for item */ + ptrdiff_t i = 0; // counts maximum expand for item while (singlematch(ms, s + i, p, ep)) i++; - /* keeps trying to match with the maximum repetitions */ + // keeps trying to match with the maximum repetitions while (i >= 0) { const char* res = match(ms, (s + i), ep + 1); if (res) return res; - i--; /* else didn't match; reduce 1 repetition to try again */ + i--; // else didn't match; reduce 1 repetition to try again } return NULL; } @@ -384,7 +386,7 @@ static const char* min_expand(MatchState* ms, const char* s, const char* p, cons if (res != NULL) return res; else if (singlematch(ms, s, p, ep)) - s++; /* try with one more repetition */ + s++; // try with one more repetition else return NULL; } @@ -399,8 +401,8 @@ static const char* start_capture(MatchState* ms, const char* s, const char* p, i ms->capture[level].init = s; ms->capture[level].len = what; ms->level = level + 1; - if ((res = match(ms, s, p)) == NULL) /* match failed? */ - ms->level--; /* undo capture */ + if ((res = match(ms, s, p)) == NULL) // match failed? + ms->level--; // undo capture return res; } @@ -408,9 +410,9 @@ static const char* end_capture(MatchState* ms, const char* s, const char* p) { int l = capture_to_close(ms); const char* res; - ms->capture[l].len = s - ms->capture[l].init; /* close capture */ - if ((res = match(ms, s, p)) == NULL) /* match failed? */ - ms->capture[l].len = CAP_UNFINISHED; /* undo capture */ + ms->capture[l].len = s - ms->capture[l].init; // close capture + if ((res = match(ms, s, p)) == NULL) // match failed? + ms->capture[l].len = CAP_UNFINISHED; // undo capture return res; } @@ -429,60 +431,60 @@ static const char* match(MatchState* ms, const char* s, const char* p) { if (ms->matchdepth-- == 0) luaL_error(ms->L, "pattern too complex"); -init: /* using goto's to optimize tail recursion */ +init: // using goto's to optimize tail recursion if (p != ms->p_end) - { /* end of pattern? */ + { // end of pattern? switch (*p) { case '(': - { /* start capture */ - if (*(p + 1) == ')') /* position capture? */ + { // start capture + if (*(p + 1) == ')') // position capture? s = start_capture(ms, s, p + 2, CAP_POSITION); else s = start_capture(ms, s, p + 1, CAP_UNFINISHED); break; } case ')': - { /* end capture */ + { // end capture s = end_capture(ms, s, p + 1); break; } case '$': { - if ((p + 1) != ms->p_end) /* is the `$' the last char in pattern? */ - goto dflt; /* no; go to default */ - s = (s == ms->src_end) ? s : NULL; /* check end of string */ + if ((p + 1) != ms->p_end) // is the `$' the last char in pattern? + goto dflt; // no; go to default + s = (s == ms->src_end) ? s : NULL; // check end of string break; } case L_ESC: - { /* escaped sequences not in the format class[*+?-]? */ + { // escaped sequences not in the format class[*+?-]? switch (*(p + 1)) { case 'b': - { /* balanced string? */ + { // balanced string? s = matchbalance(ms, s, p + 2); if (s != NULL) { p += 4; - goto init; /* return match(ms, s, p + 4); */ - } /* else fail (s == NULL) */ + goto init; // return match(ms, s, p + 4); + } // else fail (s == NULL) break; } case 'f': - { /* frontier? */ + { // frontier? const char* ep; char previous; p += 2; if (*p != '[') luaL_error(ms->L, "missing '[' after '%%f' in pattern"); - ep = classend(ms, p); /* points to what is next */ + ep = classend(ms, p); // points to what is next previous = (s == ms->src_init) ? '\0' : *(s - 1); if (!matchbracketclass(uchar(previous), p, ep - 1) && matchbracketclass(uchar(*s), p, ep - 1)) { p = ep; - goto init; /* return match(ms, s, ep); */ + goto init; // return match(ms, s, ep); } - s = NULL; /* match failed */ + s = NULL; // match failed break; } case '0': @@ -495,12 +497,12 @@ init: /* using goto's to optimize tail recursion */ case '7': case '8': case '9': - { /* capture results (%0-%9)? */ + { // capture results (%0-%9)? s = match_capture(ms, s, uchar(*(p + 1))); if (s != NULL) { p += 2; - goto init; /* return match(ms, s, p + 2) */ + goto init; // return match(ms, s, p + 2) } break; } @@ -511,48 +513,48 @@ init: /* using goto's to optimize tail recursion */ } default: dflt: - { /* pattern class plus optional suffix */ - const char* ep = classend(ms, p); /* points to optional suffix */ - /* does not match at least once? */ + { // pattern class plus optional suffix + const char* ep = classend(ms, p); // points to optional suffix + // does not match at least once? if (!singlematch(ms, s, p, ep)) { if (*ep == '*' || *ep == '?' || *ep == '-') - { /* accept empty? */ + { // accept empty? p = ep + 1; - goto init; /* return match(ms, s, ep + 1); */ + goto init; // return match(ms, s, ep + 1); } - else /* '+' or no suffix */ - s = NULL; /* fail */ + else // '+' or no suffix + s = NULL; // fail } else - { /* matched once */ + { // matched once switch (*ep) - { /* handle optional suffix */ + { // handle optional suffix case '?': - { /* optional */ + { // optional const char* res; if ((res = match(ms, s + 1, ep + 1)) != NULL) s = res; else { p = ep + 1; - goto init; /* else return match(ms, s, ep + 1); */ + goto init; // else return match(ms, s, ep + 1); } break; } - case '+': /* 1 or more repetitions */ - s++; /* 1 match already done */ - /* go through */ - case '*': /* 0 or more repetitions */ + case '+': // 1 or more repetitions + s++; // 1 match already done + // go through + case '*': // 0 or more repetitions s = max_expand(ms, s, p, ep); break; - case '-': /* 0 or more repetitions (minimum) */ + case '-': // 0 or more repetitions (minimum) s = min_expand(ms, s, p, ep); break; - default: /* no suffix */ + default: // no suffix s++; p = ep; - goto init; /* return match(ms, s + 1, ep); */ + goto init; // return match(ms, s + 1, ep); } } break; @@ -566,26 +568,26 @@ init: /* using goto's to optimize tail recursion */ static const char* lmemfind(const char* s1, size_t l1, const char* s2, size_t l2) { if (l2 == 0) - return s1; /* empty strings are everywhere */ + return s1; // empty strings are everywhere else if (l2 > l1) - return NULL; /* avoids a negative `l1' */ + return NULL; // avoids a negative `l1' else { - const char* init; /* to search for a `*s2' inside `s1' */ - l2--; /* 1st char will be checked by `memchr' */ - l1 = l1 - l2; /* `s2' cannot be found after that */ + const char* init; // to search for a `*s2' inside `s1' + l2--; // 1st char will be checked by `memchr' + l1 = l1 - l2; // `s2' cannot be found after that while (l1 > 0 && (init = (const char*)memchr(s1, *s2, l1)) != NULL) { - init++; /* 1st char is already checked */ + init++; // 1st char is already checked if (memcmp(init, s2 + 1, l2) == 0) return init - 1; else - { /* correct `l1' and `s1' to try again */ + { // correct `l1' and `s1' to try again l1 -= init - s1; s1 = init; } } - return NULL; /* not found */ + return NULL; // not found } } @@ -593,8 +595,8 @@ static void push_onecapture(MatchState* ms, int i, const char* s, const char* e) { if (i >= ms->level) { - if (i == 0) /* ms->level == 0, too */ - lua_pushlstring(ms->L, s, e - s); /* add whole match */ + if (i == 0) // ms->level == 0, too + lua_pushlstring(ms->L, s, e - s); // add whole match else luaL_error(ms->L, "invalid capture index"); } @@ -617,20 +619,20 @@ static int push_captures(MatchState* ms, const char* s, const char* e) luaL_checkstack(ms->L, nlevels, "too many captures"); for (i = 0; i < nlevels; i++) push_onecapture(ms, i, s, e); - return nlevels; /* number of strings pushed */ + return nlevels; // number of strings pushed } -/* check whether pattern has no special characters */ +// check whether pattern has no special characters static int nospecials(const char* p, size_t l) { size_t upto = 0; do { if (strpbrk(p + upto, SPECIALS)) - return 0; /* pattern has a special character */ - upto += strlen(p + upto) + 1; /* may have more after \0 */ + return 0; // pattern has a special character + upto += strlen(p + upto) + 1; // may have more after \0 } while (upto <= l); - return 1; /* no special chars found */ + return 1; // no special chars found } static void prepstate(MatchState* ms, lua_State* L, const char* s, size_t ls, const char* p, size_t lp) @@ -657,14 +659,14 @@ static int str_find_aux(lua_State* L, int find) if (init < 1) init = 1; else if (init > (int)ls + 1) - { /* start after string's end? */ - lua_pushnil(L); /* cannot find anything */ + { // start after string's end? + lua_pushnil(L); // cannot find anything return 1; } - /* explicit request or no special characters? */ + // explicit request or no special characters? if (find && (lua_toboolean(L, 4) || nospecials(p, lp))) { - /* do a plain search */ + // do a plain search const char* s2 = lmemfind(s + init - 1, ls - init + 1, p, lp); if (s2) { @@ -681,7 +683,7 @@ static int str_find_aux(lua_State* L, int find) if (anchor) { p++; - lp--; /* skip anchor character */ + lp--; // skip anchor character } prepstate(&ms, L, s, ls, p, lp); do @@ -692,8 +694,8 @@ static int str_find_aux(lua_State* L, int find) { if (find) { - lua_pushinteger(L, (int)(s1 - s + 1)); /* start */ - lua_pushinteger(L, (int)(res - s)); /* end */ + lua_pushinteger(L, (int)(s1 - s + 1)); // start + lua_pushinteger(L, (int)(res - s)); // end return push_captures(&ms, NULL, 0) + 2; } else @@ -701,7 +703,7 @@ static int str_find_aux(lua_State* L, int find) } } while (s1++ < ms.src_end && !anchor); } - lua_pushnil(L); /* not found */ + lua_pushnil(L); // not found return 1; } @@ -731,13 +733,13 @@ static int gmatch_aux(lua_State* L) { int newstart = (int)(e - s); if (e == src) - newstart++; /* empty match? go at least one position */ + newstart++; // empty match? go at least one position lua_pushinteger(L, newstart); lua_replace(L, lua_upvalueindex(3)); return push_captures(&ms, src, e); } } - return 0; /* not found */ + return 0; // not found } static int gmatch(lua_State* L) @@ -763,7 +765,7 @@ static void add_s(MatchState* ms, luaL_Buffer* b, const char* s, const char* e) luaL_addchar(b, news[i]); else { - i++; /* skip ESC */ + i++; // skip ESC if (!isdigit(uchar(news[i]))) { if (news[i] != L_ESC) @@ -775,7 +777,7 @@ static void add_s(MatchState* ms, luaL_Buffer* b, const char* s, const char* e) else { push_onecapture(ms, news[i] - '1', s, e); - luaL_addvalue(b); /* add capture to accumulated result */ + luaL_addvalue(b); // add capture to accumulated result } } } @@ -801,19 +803,19 @@ static void add_value(MatchState* ms, luaL_Buffer* b, const char* s, const char* break; } default: - { /* LUA_TNUMBER or LUA_TSTRING */ + { // LUA_TNUMBER or LUA_TSTRING add_s(ms, b, s, e); return; } } if (!lua_toboolean(L, -1)) - { /* nil or false? */ + { // nil or false? lua_pop(L, 1); - lua_pushlstring(L, s, e - s); /* keep original text */ + lua_pushlstring(L, s, e - s); // keep original text } else if (!lua_isstring(L, -1)) luaL_error(L, "invalid replacement value (a %s)", luaL_typename(L, -1)); - luaL_addvalue(b); /* add result to accumulator */ + luaL_addvalue(b); // add result to accumulator } static int str_gsub(lua_State* L) @@ -832,7 +834,7 @@ static int str_gsub(lua_State* L) if (anchor) { p++; - lp--; /* skip anchor character */ + lp--; // skip anchor character } prepstate(&ms, L, src, srcl, p, lp); while (n < max_s) @@ -845,8 +847,8 @@ static int str_gsub(lua_State* L) n++; add_value(&ms, &b, src, e, tr); } - if (e && e > src) /* non empty match? */ - src = e; /* skip it */ + if (e && e > src) // non empty match? + src = e; // skip it else if (src < ms.src_end) luaL_addchar(&b, *src++); else @@ -856,17 +858,17 @@ static int str_gsub(lua_State* L) } luaL_addlstring(&b, src, ms.src_end - src); luaL_pushresult(&b); - lua_pushinteger(L, n); /* number of substitutions */ + lua_pushinteger(L, n); // number of substitutions return 2; } -/* }====================================================== */ +// }====================================================== -/* valid flags in a format specification */ +// valid flags in a format specification #define FLAGS "-+ #0" -/* maximum size of each formatted item (> len(format('%99.99f', -1e308))) */ +// maximum size of each formatted item (> len(format('%99.99f', -1e308))) #define MAX_ITEM 512 -/* maximum size of each format specification (such as '%-099.99d') */ +// maximum size of each format specification (such as '%-099.99d') #define MAX_FORMAT 32 static void addquoted(lua_State* L, luaL_Buffer* b, int arg) @@ -914,20 +916,20 @@ static const char* scanformat(lua_State* L, const char* strfrmt, char* form, siz { const char* p = strfrmt; while (*p != '\0' && strchr(FLAGS, *p) != NULL) - p++; /* skip flags */ + p++; // skip flags if ((size_t)(p - strfrmt) >= sizeof(FLAGS)) luaL_error(L, "invalid format (repeated flags)"); if (isdigit(uchar(*p))) - p++; /* skip width */ + p++; // skip width if (isdigit(uchar(*p))) - p++; /* (2 digits at most) */ + p++; // (2 digits at most) if (*p == '.') { p++; if (isdigit(uchar(*p))) - p++; /* skip precision */ + p++; // skip precision if (isdigit(uchar(*p))) - p++; /* (2 digits at most) */ + p++; // (2 digits at most) } if (isdigit(uchar(*p))) luaL_error(L, "invalid format (width or precision too long)"); @@ -965,11 +967,11 @@ static int str_format(lua_State* L) if (*strfrmt != L_ESC) luaL_addchar(&b, *strfrmt++); else if (*++strfrmt == L_ESC) - luaL_addchar(&b, *strfrmt++); /* %% */ + luaL_addchar(&b, *strfrmt++); // %% else - { /* format item */ - char form[MAX_FORMAT]; /* to store the format (`%...') */ - char buff[MAX_ITEM]; /* to store the formatted item */ + { // format item + char form[MAX_FORMAT]; // to store the format (`%...') + char buff[MAX_ITEM]; // to store the formatted item if (++arg > top) luaL_error(L, "missing argument #%d", arg); size_t formatItemSize = 0; @@ -1012,7 +1014,7 @@ static int str_format(lua_State* L) case 'q': { addquoted(L, &b, arg); - continue; /* skip the 'addsize' at the end */ + continue; // skip the 'addsize' at the end } case 's': { @@ -1024,7 +1026,7 @@ static int str_format(lua_State* L) keep original string */ lua_pushvalue(L, arg); luaL_addvalue(&b); - continue; /* skip the `addsize' at the end */ + continue; // skip the `addsize' at the end } else { @@ -1032,8 +1034,22 @@ static int str_format(lua_State* L) break; } } + case '*': + { + if (!FFlag::LuauTostringFormatSpecifier) + luaL_error(L, "invalid option '%%*' to 'format'"); + + if (formatItemSize != 1) + luaL_error(L, "'%%*' does not take a form"); + + size_t length; + const char* string = luaL_tolstring(L, arg, &length); + + luaL_addlstring(&b, string, length); + continue; // skip the `addsize' at the end + } default: - { /* also treat cases `pnLlh' */ + { // also treat cases `pnLlh' luaL_error(L, "invalid option '%%%c' to 'format'", *(strfrmt - 1)); } } @@ -1098,31 +1114,31 @@ static int str_split(lua_State* L) ** ======================================================= */ -/* value used for padding */ +// value used for padding #if !defined(LUAL_PACKPADBYTE) #define LUAL_PACKPADBYTE 0x00 #endif -/* maximum size for the binary representation of an integer */ +// maximum size for the binary representation of an integer #define MAXINTSIZE 16 -/* number of bits in a character */ +// number of bits in a character #define NB CHAR_BIT -/* mask for one character (NB 1's) */ +// mask for one character (NB 1's) #define MC ((1 << NB) - 1) -/* internal size of integers used for pack/unpack */ +// internal size of integers used for pack/unpack #define SZINT (int)sizeof(long long) -/* dummy union to get native endianness */ +// dummy union to get native endianness static const union { int dummy; - char little; /* true iff machine is little endian */ + char little; // true iff machine is little endian } nativeendian = {1}; -/* assume we need to align for double & pointers */ +// assume we need to align for double & pointers #define MAXALIGN 8 /* @@ -1133,7 +1149,7 @@ typedef union Ftypes float f; double d; double n; - char buff[5 * sizeof(double)]; /* enough for any float type */ + char buff[5 * sizeof(double)]; // enough for any float type } Ftypes; /* @@ -1151,15 +1167,15 @@ typedef struct Header */ typedef enum KOption { - Kint, /* signed integers */ - Kuint, /* unsigned integers */ - Kfloat, /* floating-point numbers */ - Kchar, /* fixed-length strings */ - Kstring, /* strings with prefixed length */ - Kzstr, /* zero-terminated strings */ - Kpadding, /* padding */ - Kpaddalign, /* padding for alignment */ - Knop /* no-op (configuration or spaces) */ + Kint, // signed integers + Kuint, // unsigned integers + Kfloat, // floating-point numbers + Kchar, // fixed-length strings + Kstring, // strings with prefixed length + Kzstr, // zero-terminated strings + Kpadding, // padding + Kpaddalign, // padding for alignment + Knop // no-op (configuration or spaces) } KOption; /* @@ -1173,8 +1189,8 @@ static int digit(int c) static int getnum(Header* h, const char** fmt, int df) { - if (!digit(**fmt)) /* no number? */ - return df; /* return default value */ + if (!digit(**fmt)) // no number? + return df; // return default value else { int a = 0; @@ -1216,7 +1232,7 @@ static void initheader(lua_State* L, Header* h) static KOption getoption(Header* h, const char** fmt, int* size) { int opt = *((*fmt)++); - *size = 0; /* default */ + *size = 0; // default switch (opt) { case 'b': @@ -1308,19 +1324,19 @@ static KOption getoption(Header* h, const char** fmt, int* size) static KOption getdetails(Header* h, size_t totalsize, const char** fmt, int* psize, int* ntoalign) { KOption opt = getoption(h, fmt, psize); - int align = *psize; /* usually, alignment follows size */ + int align = *psize; // usually, alignment follows size if (opt == Kpaddalign) - { /* 'X' gets alignment from following option */ + { // 'X' gets alignment from following option if (**fmt == '\0' || getoption(h, fmt, &align) == Kchar || align == 0) luaL_argerror(h->L, 1, "invalid next option for option 'X'"); } - if (align <= 1 || opt == Kchar) /* need no alignment? */ + if (align <= 1 || opt == Kchar) // need no alignment? *ntoalign = 0; else { - if (align > h->maxalign) /* enforce maximum alignment */ + if (align > h->maxalign) // enforce maximum alignment align = h->maxalign; - if ((align & (align - 1)) != 0) /* is 'align' not a power of 2? */ + if ((align & (align - 1)) != 0) // is 'align' not a power of 2? luaL_argerror(h->L, 1, "format asks for alignment not power of 2"); *ntoalign = (align - (int)(totalsize & (align - 1))) & (align - 1); } @@ -1338,18 +1354,18 @@ static void packint(luaL_Buffer* b, unsigned long long n, int islittle, int size LUAU_ASSERT(size <= MAXINTSIZE); char buff[MAXINTSIZE]; int i; - buff[islittle ? 0 : size - 1] = (char)(n & MC); /* first byte */ + buff[islittle ? 0 : size - 1] = (char)(n & MC); // first byte for (i = 1; i < size; i++) { n >>= NB; buff[islittle ? i : size - 1 - i] = (char)(n & MC); } if (neg && size > SZINT) - { /* negative number need sign extension? */ - for (i = SZINT; i < size; i++) /* correct extra bytes */ + { // negative number need sign extension? + for (i = SZINT; i < size; i++) // correct extra bytes buff[islittle ? i : size - 1 - i] = (char)MC; } - luaL_addlstring(b, buff, size); /* add result to buffer */ + luaL_addlstring(b, buff, size); // add result to buffer } /* @@ -1375,11 +1391,11 @@ static int str_pack(lua_State* L) { luaL_Buffer b; Header h; - const char* fmt = luaL_checkstring(L, 1); /* format string */ - int arg = 1; /* current argument to pack */ - size_t totalsize = 0; /* accumulate total size of result */ + const char* fmt = luaL_checkstring(L, 1); // format string + int arg = 1; // current argument to pack + size_t totalsize = 0; // accumulate total size of result initheader(L, &h); - lua_pushnil(L); /* mark to separate arguments from string buffer */ + lua_pushnil(L); // mark to separate arguments from string buffer luaL_buffinit(L, &b); while (*fmt != '\0') { @@ -1387,15 +1403,15 @@ static int str_pack(lua_State* L) KOption opt = getdetails(&h, totalsize, &fmt, &size, &ntoalign); totalsize += ntoalign + size; while (ntoalign-- > 0) - luaL_addchar(&b, LUAL_PACKPADBYTE); /* fill alignment */ + luaL_addchar(&b, LUAL_PACKPADBYTE); // fill alignment arg++; switch (opt) { case Kint: - { /* signed integers */ + { // signed integers long long n = (long long)luaL_checknumber(L, arg); if (size < SZINT) - { /* need overflow check? */ + { // need overflow check? long long lim = (long long)1 << ((size * NB) - 1); luaL_argcheck(L, -lim <= n && n < lim, arg, "integer overflow"); } @@ -1403,64 +1419,64 @@ static int str_pack(lua_State* L) break; } case Kuint: - { /* unsigned integers */ + { // unsigned integers long long n = (long long)luaL_checknumber(L, arg); - if (size < SZINT) /* need overflow check? */ + if (size < SZINT) // need overflow check? luaL_argcheck(L, (unsigned long long)n < ((unsigned long long)1 << (size * NB)), arg, "unsigned overflow"); packint(&b, (unsigned long long)n, h.islittle, size, 0); break; } case Kfloat: - { /* floating-point options */ + { // floating-point options volatile Ftypes u; char buff[MAXINTSIZE]; - double n = luaL_checknumber(L, arg); /* get argument */ + double n = luaL_checknumber(L, arg); // get argument if (size == sizeof(u.f)) - u.f = (float)n; /* copy it into 'u' */ + u.f = (float)n; // copy it into 'u' else if (size == sizeof(u.d)) u.d = (double)n; else u.n = n; - /* move 'u' to final result, correcting endianness if needed */ + // move 'u' to final result, correcting endianness if needed copywithendian(buff, u.buff, size, h.islittle); luaL_addlstring(&b, buff, size); break; } case Kchar: - { /* fixed-size string */ + { // fixed-size string size_t len; const char* s = luaL_checklstring(L, arg, &len); luaL_argcheck(L, len <= (size_t)size, arg, "string longer than given size"); - luaL_addlstring(&b, s, len); /* add string */ - while (len++ < (size_t)size) /* pad extra space */ + luaL_addlstring(&b, s, len); // add string + while (len++ < (size_t)size) // pad extra space luaL_addchar(&b, LUAL_PACKPADBYTE); break; } case Kstring: - { /* strings with length count */ + { // strings with length count size_t len; const char* s = luaL_checklstring(L, arg, &len); luaL_argcheck(L, size >= (int)sizeof(size_t) || len < ((size_t)1 << (size * NB)), arg, "string length does not fit in given size"); - packint(&b, len, h.islittle, size, 0); /* pack length */ + packint(&b, len, h.islittle, size, 0); // pack length luaL_addlstring(&b, s, len); totalsize += len; break; } case Kzstr: - { /* zero-terminated string */ + { // zero-terminated string size_t len; const char* s = luaL_checklstring(L, arg, &len); luaL_argcheck(L, strlen(s) == len, arg, "string contains zeros"); luaL_addlstring(&b, s, len); - luaL_addchar(&b, '\0'); /* add zero at the end */ + luaL_addchar(&b, '\0'); // add zero at the end totalsize += len + 1; break; } case Kpadding: - luaL_addchar(&b, LUAL_PACKPADBYTE); /* FALLTHROUGH */ + luaL_addchar(&b, LUAL_PACKPADBYTE); // FALLTHROUGH case Kpaddalign: case Knop: - arg--; /* undo increment */ + arg--; // undo increment break; } } @@ -1471,15 +1487,15 @@ static int str_pack(lua_State* L) static int str_packsize(lua_State* L) { Header h; - const char* fmt = luaL_checkstring(L, 1); /* format string */ - int totalsize = 0; /* accumulate total size of result */ + const char* fmt = luaL_checkstring(L, 1); // format string + int totalsize = 0; // accumulate total size of result initheader(L, &h); while (*fmt != '\0') { int size, ntoalign; KOption opt = getdetails(&h, totalsize, &fmt, &size, &ntoalign); luaL_argcheck(L, opt != Kstring && opt != Kzstr, 1, "variable-length format"); - size += ntoalign; /* total space used by option */ + size += ntoalign; // total space used by option luaL_argcheck(L, totalsize <= MAXSSIZE - size, 1, "format result too large"); totalsize += size; } @@ -1506,15 +1522,15 @@ static long long unpackint(lua_State* L, const char* str, int islittle, int size res |= (unsigned char)str[islittle ? i : size - 1 - i]; } if (size < SZINT) - { /* real size smaller than int? */ + { // real size smaller than int? if (issigned) - { /* needs sign extension? */ + { // needs sign extension? unsigned long long mask = (unsigned long long)1 << (size * NB - 1); - res = ((res ^ mask) - mask); /* do sign extension */ + res = ((res ^ mask) - mask); // do sign extension } } else if (size > SZINT) - { /* must check unread bytes */ + { // must check unread bytes int mask = (!issigned || (long long)res >= 0) ? 0 : MC; for (i = limit; i < size; i++) { @@ -1534,7 +1550,7 @@ static int str_unpack(lua_State* L) int pos = posrelat(luaL_optinteger(L, 3, 1), ld) - 1; if (pos < 0) pos = 0; - int n = 0; /* number of results */ + int n = 0; // number of results luaL_argcheck(L, size_t(pos) <= ld, 3, "initial position out of string"); initheader(L, &h); while (*fmt != '\0') @@ -1542,8 +1558,8 @@ static int str_unpack(lua_State* L) int size, ntoalign; KOption opt = getdetails(&h, pos, &fmt, &size, &ntoalign); luaL_argcheck(L, (size_t)ntoalign + size <= ld - pos, 2, "data string too short"); - pos += ntoalign; /* skip alignment */ - /* stack space for item + next position */ + pos += ntoalign; // skip alignment + // stack space for item + next position luaL_checkstack(L, 2, "too many results"); n++; switch (opt) @@ -1584,7 +1600,7 @@ static int str_unpack(lua_State* L) size_t len = (size_t)unpackint(L, data + pos, h.islittle, size, 0); luaL_argcheck(L, len <= ld - pos - size, 2, "data string too short"); lua_pushlstring(L, data + pos + size, len); - pos += (int)len; /* skip string */ + pos += (int)len; // skip string break; } case Kzstr: @@ -1592,22 +1608,22 @@ static int str_unpack(lua_State* L) size_t len = strlen(data + pos); luaL_argcheck(L, pos + len < ld, 2, "unfinished string for format 'z'"); lua_pushlstring(L, data + pos, len); - pos += (int)len + 1; /* skip string plus final '\0' */ + pos += (int)len + 1; // skip string plus final '\0' break; } case Kpaddalign: case Kpadding: case Knop: - n--; /* undo increment */ + n--; // undo increment break; } pos += size; } - lua_pushinteger(L, pos + 1); /* next position */ + lua_pushinteger(L, pos + 1); // next position return n + 1; } -/* }====================================================== */ +// }====================================================== static const luaL_Reg strlib[] = { {"byte", str_byte}, @@ -1632,14 +1648,14 @@ static const luaL_Reg strlib[] = { static void createmetatable(lua_State* L) { - lua_createtable(L, 0, 1); /* create metatable for strings */ - lua_pushliteral(L, ""); /* dummy string */ + lua_createtable(L, 0, 1); // create metatable for strings + lua_pushliteral(L, ""); // dummy string lua_pushvalue(L, -2); - lua_setmetatable(L, -2); /* set string metatable */ - lua_pop(L, 1); /* pop dummy string */ - lua_pushvalue(L, -2); /* string library... */ - lua_setfield(L, -2, "__index"); /* ...is the __index metamethod */ - lua_pop(L, 1); /* pop metatable */ + lua_setmetatable(L, -2); // set string metatable + lua_pop(L, 1); // pop dummy string + lua_pushvalue(L, -2); // string library... + lua_setfield(L, -2, "__index"); // ...is the __index metamethod + lua_pop(L, 1); // pop metatable } /* diff --git a/VM/src/ltable.cpp b/VM/src/ltable.cpp index 38693156..8d59ecbc 100644 --- a/VM/src/ltable.cpp +++ b/VM/src/ltable.cpp @@ -46,8 +46,8 @@ static_assert(TKey{{NULL}, {0}, LUA_TNIL, -(MAXSIZE - 1)}.next == -(MAXSIZE - 1) // empty hash data points to dummynode so that we can always dereference it const LuaNode luaH_dummynode = { - {{NULL}, {0}, LUA_TNIL}, /* value */ - {{NULL}, {0}, LUA_TNIL, 0} /* key */ + {{NULL}, {0}, LUA_TNIL}, // value + {{NULL}, {0}, LUA_TNIL, 0} // key }; #define dummynode (&luaH_dummynode) @@ -170,52 +170,52 @@ static int findindex(lua_State* L, Table* t, StkId key) { int i; if (ttisnil(key)) - return -1; /* first iteration */ + return -1; // first iteration i = ttisnumber(key) ? arrayindex(nvalue(key)) : -1; - if (0 < i && i <= t->sizearray) /* is `key' inside array part? */ - return i - 1; /* yes; that's the index (corrected to C) */ + if (0 < i && i <= t->sizearray) // is `key' inside array part? + return i - 1; // yes; that's the index (corrected to C) else { LuaNode* n = mainposition(t, key); for (;;) - { /* check whether `key' is somewhere in the chain */ - /* key may be dead already, but it is ok to use it in `next' */ + { // check whether `key' is somewhere in the chain + // key may be dead already, but it is ok to use it in `next' if (luaO_rawequalKey(gkey(n), key) || (ttype(gkey(n)) == LUA_TDEADKEY && iscollectable(key) && gcvalue(gkey(n)) == gcvalue(key))) { - i = cast_int(n - gnode(t, 0)); /* key index in hash table */ - /* hash elements are numbered after array ones */ + i = cast_int(n - gnode(t, 0)); // key index in hash table + // hash elements are numbered after array ones return i + t->sizearray; } if (gnext(n) == 0) break; n += gnext(n); } - luaG_runerror(L, "invalid key to 'next'"); /* key not found */ + luaG_runerror(L, "invalid key to 'next'"); // key not found } } int luaH_next(lua_State* L, Table* t, StkId key) { - int i = findindex(L, t, key); /* find original element */ + int i = findindex(L, t, key); // find original element for (i++; i < t->sizearray; i++) - { /* try first array part */ + { // try first array part if (!ttisnil(&t->array[i])) - { /* a non-nil value? */ + { // a non-nil value? setnvalue(key, cast_num(i + 1)); setobj2s(L, key + 1, &t->array[i]); return 1; } } for (i -= t->sizearray; i < sizenode(t); i++) - { /* then hash part */ + { // then hash part if (!ttisnil(gval(gnode(t, i)))) - { /* a non-nil value? */ + { // a non-nil value? getnodekey(L, key, gnode(t, i)); setobj2s(L, key + 1, gval(gnode(t, i))); return 1; } } - return 0; /* no more elements */ + return 0; // no more elements } /* @@ -235,23 +235,23 @@ int luaH_next(lua_State* L, Table* t, StkId key) static int computesizes(int nums[], int* narray) { int i; - int twotoi; /* 2^i */ - int a = 0; /* number of elements smaller than 2^i */ - int na = 0; /* number of elements to go to array part */ - int n = 0; /* optimal size for array part */ + int twotoi; // 2^i + int a = 0; // number of elements smaller than 2^i + int na = 0; // number of elements to go to array part + int n = 0; // optimal size for array part for (i = 0, twotoi = 1; twotoi / 2 < *narray; i++, twotoi *= 2) { if (nums[i] > 0) { a += nums[i]; if (a > twotoi / 2) - { /* more than half elements present? */ - n = twotoi; /* optimal size (till now) */ - na = a; /* all elements smaller than n will go to array part */ + { // more than half elements present? + n = twotoi; // optimal size (till now) + na = a; // all elements smaller than n will go to array part } } if (a == *narray) - break; /* all elements already counted */ + break; // all elements already counted } *narray = n; LUAU_ASSERT(*narray / 2 <= na && na <= *narray); @@ -262,8 +262,8 @@ static int countint(double key, int* nums) { int k = arrayindex(key); if (0 < k && k <= MAXSIZE) - { /* is `key' an appropriate array index? */ - nums[ceillog2(k)]++; /* count as such */ + { // is `key' an appropriate array index? + nums[ceillog2(k)]++; // count as such return 1; } else @@ -273,20 +273,20 @@ static int countint(double key, int* nums) static int numusearray(const Table* t, int* nums) { int lg; - int ttlg; /* 2^lg */ - int ause = 0; /* summation of `nums' */ - int i = 1; /* count to traverse all array keys */ + int ttlg; // 2^lg + int ause = 0; // summation of `nums' + int i = 1; // count to traverse all array keys for (lg = 0, ttlg = 1; lg <= MAXBITS; lg++, ttlg *= 2) - { /* for each slice */ - int lc = 0; /* counter */ + { // for each slice + int lc = 0; // counter int lim = ttlg; if (lim > t->sizearray) { - lim = t->sizearray; /* adjust upper limit */ + lim = t->sizearray; // adjust upper limit if (i > lim) - break; /* no more elements to count */ + break; // no more elements to count } - /* count elements in range (2^(lg-1), 2^lg] */ + // count elements in range (2^(lg-1), 2^lg] for (; i <= lim; i++) { if (!ttisnil(&t->array[i - 1])) @@ -300,8 +300,8 @@ static int numusearray(const Table* t, int* nums) static int numusehash(const Table* t, int* nums, int* pnasize) { - int totaluse = 0; /* total number of elements */ - int ause = 0; /* summation of `nums' */ + int totaluse = 0; // total number of elements + int ause = 0; // summation of `nums' int i = sizenode(t); while (i--) { @@ -332,8 +332,8 @@ static void setnodevector(lua_State* L, Table* t, int size) { int lsize; if (size == 0) - { /* no elements to hash part? */ - t->node = cast_to(LuaNode*, dummynode); /* use common `dummynode' */ + { // no elements to hash part? + t->node = cast_to(LuaNode*, dummynode); // use common `dummynode' lsize = 0; } else @@ -354,7 +354,7 @@ static void setnodevector(lua_State* L, Table* t, int size) } t->lsizenode = cast_byte(lsize); t->nodemask8 = cast_byte((1 << lsize) - 1); - t->lastfree = size; /* all positions are free */ + t->lastfree = size; // all positions are free } static TValue* newkey(lua_State* L, Table* t, const TValue* key); @@ -379,17 +379,17 @@ static void resize(lua_State* L, Table* t, int nasize, int nhsize) luaG_runerror(L, "table overflow"); int oldasize = t->sizearray; int oldhsize = t->lsizenode; - LuaNode* nold = t->node; /* save old hash ... */ - if (nasize > oldasize) /* array part must grow? */ + LuaNode* nold = t->node; // save old hash ... + if (nasize > oldasize) // array part must grow? setarrayvector(L, t, nasize); - /* create new hash part with appropriate size */ + // create new hash part with appropriate size setnodevector(L, t, nhsize); - /* used for the migration check at the end */ + // used for the migration check at the end LuaNode* nnew = t->node; if (nasize < oldasize) - { /* array part must shrink? */ + { // array part must shrink? t->sizearray = nasize; - /* re-insert elements from vanishing slice */ + // re-insert elements from vanishing slice for (int i = nasize; i < oldasize; i++) { if (!ttisnil(&t->array[i])) @@ -399,12 +399,12 @@ static void resize(lua_State* L, Table* t, int nasize, int nhsize) setobjt2t(L, newkey(L, t, &ok), &t->array[i]); } } - /* shrink array */ + // shrink array luaM_reallocarray(L, t->array, oldasize, nasize, TValue, t->memcat); } - /* used for the migration check at the end */ + // used for the migration check at the end TValue* anew = t->array; - /* re-insert elements from hash part */ + // re-insert elements from hash part for (int i = twoto(oldhsize) - 1; i >= 0; i--) { LuaNode* old = nold + i; @@ -416,19 +416,19 @@ static void resize(lua_State* L, Table* t, int nasize, int nhsize) } } - /* make sure we haven't recursively rehashed during element migration */ + // make sure we haven't recursively rehashed during element migration LUAU_ASSERT(nnew == t->node); LUAU_ASSERT(anew == t->array); if (nold != dummynode) - luaM_freearray(L, nold, twoto(oldhsize), LuaNode, t->memcat); /* free old array */ + luaM_freearray(L, nold, twoto(oldhsize), LuaNode, t->memcat); // free old array } static int adjustasize(Table* t, int size, const TValue* ek) { bool tbound = t->node != dummynode || size < t->sizearray; int ekindex = ek && ttisnumber(ek) ? arrayindex(nvalue(ek)) : -1; - /* move the array size up until the boundary is guaranteed to be inside the array part */ + // move the array size up until the boundary is guaranteed to be inside the array part while (size + 1 == ekindex || (tbound && !ttisnil(luaH_getnum(t, size + 1)))) size++; return size; @@ -448,22 +448,22 @@ void luaH_resizehash(lua_State* L, Table* t, int nhsize) static void rehash(lua_State* L, Table* t, const TValue* ek) { - int nums[MAXBITS + 1]; /* nums[i] = number of keys between 2^(i-1) and 2^i */ + int nums[MAXBITS + 1]; // nums[i] = number of keys between 2^(i-1) and 2^i for (int i = 0; i <= MAXBITS; i++) - nums[i] = 0; /* reset counts */ - int nasize = numusearray(t, nums); /* count keys in array part */ - int totaluse = nasize; /* all those keys are integer keys */ - totaluse += numusehash(t, nums, &nasize); /* count keys in hash part */ - /* count extra key */ + nums[i] = 0; // reset counts + int nasize = numusearray(t, nums); // count keys in array part + int totaluse = nasize; // all those keys are integer keys + totaluse += numusehash(t, nums, &nasize); // count keys in hash part + // count extra key if (ttisnumber(ek)) nasize += countint(nvalue(ek), nums); totaluse++; - /* compute new size for array part */ + // compute new size for array part int na = computesizes(nums, &nasize); int nh = totaluse - na; - /* enforce the boundary invariant; for performance, only do hash lookups if we must */ + // enforce the boundary invariant; for performance, only do hash lookups if we must nasize = adjustasize(t, nasize, ek); - /* resize the table to new computed sizes */ + // resize the table to new computed sizes resize(L, t, nasize, nh); } @@ -511,7 +511,7 @@ static LuaNode* getfreepos(Table* t) if (ttisnil(gkey(n))) return n; } - return NULL; /* could not find a free place */ + return NULL; // could not find a free place } /* @@ -523,24 +523,24 @@ static LuaNode* getfreepos(Table* t) */ static TValue* newkey(lua_State* L, Table* t, const TValue* key) { - /* enforce boundary invariant */ + // enforce boundary invariant if (ttisnumber(key) && nvalue(key) == t->sizearray + 1) { - rehash(L, t, key); /* grow table */ + rehash(L, t, key); // grow table - /* after rehash, numeric keys might be located in the new array part, but won't be found in the node part */ + // after rehash, numeric keys might be located in the new array part, but won't be found in the node part return arrayornewkey(L, t, key); } LuaNode* mp = mainposition(t, key); if (!ttisnil(gval(mp)) || mp == dummynode) { - LuaNode* n = getfreepos(t); /* get a free place */ + LuaNode* n = getfreepos(t); // get a free place if (n == NULL) - { /* cannot find a free place? */ - rehash(L, t, key); /* grow table */ + { // cannot find a free place? + rehash(L, t, key); // grow table - /* after rehash, numeric keys might be located in the new array part, but won't be found in the node part */ + // after rehash, numeric keys might be located in the new array part, but won't be found in the node part return arrayornewkey(L, t, key); } LUAU_ASSERT(n != dummynode); @@ -548,24 +548,24 @@ static TValue* newkey(lua_State* L, Table* t, const TValue* key) getnodekey(L, &mk, mp); LuaNode* othern = mainposition(t, &mk); if (othern != mp) - { /* is colliding node out of its main position? */ - /* yes; move colliding node into free position */ + { // is colliding node out of its main position? + // yes; move colliding node into free position while (othern + gnext(othern) != mp) - othern += gnext(othern); /* find previous */ - gnext(othern) = cast_int(n - othern); /* redo the chain with `n' in place of `mp' */ - *n = *mp; /* copy colliding node into free pos. (mp->next also goes) */ + othern += gnext(othern); // find previous + gnext(othern) = cast_int(n - othern); // redo the chain with `n' in place of `mp' + *n = *mp; // copy colliding node into free pos. (mp->next also goes) if (gnext(mp) != 0) { - gnext(n) += cast_int(mp - n); /* correct 'next' */ - gnext(mp) = 0; /* now 'mp' is free */ + gnext(n) += cast_int(mp - n); // correct 'next' + gnext(mp) = 0; // now 'mp' is free } setnilvalue(gval(mp)); } else - { /* colliding node is in its own main position */ - /* new node will go into free position */ + { // colliding node is in its own main position + // new node will go into free position if (gnext(mp) != 0) - gnext(n) = cast_int((mp + gnext(mp)) - n); /* chain new position */ + gnext(n) = cast_int((mp + gnext(mp)) - n); // chain new position else LUAU_ASSERT(gnext(n) == 0); gnext(mp) = cast_int(n - mp); @@ -583,7 +583,7 @@ static TValue* newkey(lua_State* L, Table* t, const TValue* key) */ const TValue* luaH_getnum(Table* t, int key) { - /* (1 <= key && key <= t->sizearray) */ + // (1 <= key && key <= t->sizearray) if (cast_to(unsigned int, key - 1) < cast_to(unsigned int, t->sizearray)) return &t->array[key - 1]; else if (t->node != dummynode) @@ -591,9 +591,9 @@ const TValue* luaH_getnum(Table* t, int key) double nk = cast_num(key); LuaNode* n = hashnum(t, nk); for (;;) - { /* check whether `key' is somewhere in the chain */ + { // check whether `key' is somewhere in the chain if (ttisnumber(gkey(n)) && luai_numeq(nvalue(gkey(n)), nk)) - return gval(n); /* that's it */ + return gval(n); // that's it if (gnext(n) == 0) break; n += gnext(n); @@ -611,9 +611,9 @@ const TValue* luaH_getstr(Table* t, TString* key) { LuaNode* n = hashstr(t, key); for (;;) - { /* check whether `key' is somewhere in the chain */ + { // check whether `key' is somewhere in the chain if (ttisstring(gkey(n)) && tsvalue(gkey(n)) == key) - return gval(n); /* that's it */ + return gval(n); // that's it if (gnext(n) == 0) break; n += gnext(n); @@ -637,17 +637,17 @@ const TValue* luaH_get(Table* t, const TValue* key) int k; double n = nvalue(key); luai_num2int(k, n); - if (luai_numeq(cast_num(k), nvalue(key))) /* index is int? */ - return luaH_getnum(t, k); /* use specialized version */ - /* else go through */ + if (luai_numeq(cast_num(k), nvalue(key))) // index is int? + return luaH_getnum(t, k); // use specialized version + // else go through } default: { LuaNode* n = mainposition(t, key); for (;;) - { /* check whether `key' is somewhere in the chain */ + { // check whether `key' is somewhere in the chain if (luaO_rawequalKey(gkey(n), key)) - return gval(n); /* that's it */ + return gval(n); // that's it if (gnext(n) == 0) break; n += gnext(n); @@ -680,10 +680,10 @@ TValue* luaH_newkey(lua_State* L, Table* t, const TValue* key) TValue* luaH_setnum(lua_State* L, Table* t, int key) { - /* (1 <= key && key <= t->sizearray) */ + // (1 <= key && key <= t->sizearray) if (cast_to(unsigned int, key - 1) < cast_to(unsigned int, t->sizearray)) return &t->array[key - 1]; - /* hash fallback */ + // hash fallback const TValue* p = luaH_getnum(t, key); if (p != luaO_nilobject) return cast_to(TValue*, p); @@ -739,9 +739,9 @@ int luaH_getn(Table* t) if (boundary > 0) { if (!ttisnil(&t->array[t->sizearray - 1]) && t->node == dummynode) - return t->sizearray; /* fast-path: the end of the array in `t' already refers to a boundary */ + return t->sizearray; // fast-path: the end of the array in `t' already refers to a boundary if (boundary < t->sizearray && !ttisnil(&t->array[boundary - 1]) && ttisnil(&t->array[boundary])) - return boundary; /* fast-path: boundary already refers to a boundary in `t' */ + return boundary; // fast-path: boundary already refers to a boundary in `t' int foundboundary = updateaboundary(t, boundary); if (foundboundary > 0) @@ -767,7 +767,7 @@ int luaH_getn(Table* t) } else { - /* validate boundary invariant */ + // validate boundary invariant LUAU_ASSERT(t->node == dummynode || ttisnil(luaH_getnum(t, j + 1))); return j; } @@ -812,7 +812,7 @@ Table* luaH_clone(lua_State* L, Table* tt) void luaH_clear(Table* tt) { - /* clear array part */ + // clear array part for (int i = 0; i < tt->sizearray; ++i) { setnilvalue(&tt->array[i]); @@ -820,7 +820,7 @@ void luaH_clear(Table* tt) maybesetaboundary(tt, 0); - /* clear hash part */ + // clear hash part if (tt->node != dummynode) { int size = sizenode(tt); @@ -834,6 +834,6 @@ void luaH_clear(Table* tt) } } - /* back to empty -> no tag methods present */ + // back to empty -> no tag methods present tt->tmcache = cast_byte(~0); } diff --git a/VM/src/ltablib.cpp b/VM/src/ltablib.cpp index dc653338..6dd94149 100644 --- a/VM/src/ltablib.cpp +++ b/VM/src/ltablib.cpp @@ -18,13 +18,13 @@ static int foreachi(lua_State* L) int n = lua_objlen(L, 1); for (i = 1; i <= n; i++) { - lua_pushvalue(L, 2); /* function */ - lua_pushinteger(L, i); /* 1st argument */ - lua_rawgeti(L, 1, i); /* 2nd argument */ + lua_pushvalue(L, 2); // function + lua_pushinteger(L, i); // 1st argument + lua_rawgeti(L, 1, i); // 2nd argument lua_call(L, 2, 1); if (!lua_isnil(L, -1)) return 1; - lua_pop(L, 1); /* remove nil result */ + lua_pop(L, 1); // remove nil result } return 0; } @@ -33,16 +33,16 @@ static int foreach (lua_State* L) { luaL_checktype(L, 1, LUA_TTABLE); luaL_checktype(L, 2, LUA_TFUNCTION); - lua_pushnil(L); /* first key */ + lua_pushnil(L); // first key while (lua_next(L, 1)) { - lua_pushvalue(L, 2); /* function */ - lua_pushvalue(L, -3); /* key */ - lua_pushvalue(L, -3); /* value */ + lua_pushvalue(L, 2); // function + lua_pushvalue(L, -3); // key + lua_pushvalue(L, -3); // value lua_call(L, 2, 1); if (!lua_isnil(L, -1)) return 1; - lua_pop(L, 2); /* remove value and result */ + lua_pop(L, 2); // remove value and result } return 0; } @@ -51,10 +51,10 @@ static int maxn(lua_State* L) { double max = 0; luaL_checktype(L, 1, LUA_TTABLE); - lua_pushnil(L); /* first key */ + lua_pushnil(L); // first key while (lua_next(L, 1)) { - lua_pop(L, 1); /* remove value */ + lua_pop(L, 1); // remove value if (lua_type(L, -1) == LUA_TNUMBER) { double v = lua_tonumber(L, -1); @@ -81,7 +81,7 @@ static void moveelements(lua_State* L, int srct, int dstt, int f, int e, int t) if (dst->readonly) luaG_readonlyerror(L); - int n = e - f + 1; /* number of elements to move */ + int n = e - f + 1; // number of elements to move if (cast_to(unsigned int, f - 1) < cast_to(unsigned int, src->sizearray) && cast_to(unsigned int, t - 1) < cast_to(unsigned int, dst->sizearray) && @@ -137,19 +137,19 @@ static int tinsert(lua_State* L) { luaL_checktype(L, 1, LUA_TTABLE); int n = lua_objlen(L, 1); - int pos; /* where to insert new element */ + int pos; // where to insert new element switch (lua_gettop(L)) { case 2: - { /* called with only 2 arguments */ - pos = n + 1; /* insert new element at the end */ + { // called with only 2 arguments + pos = n + 1; // insert new element at the end break; } case 3: { - pos = luaL_checkinteger(L, 2); /* 2nd argument is the position */ + pos = luaL_checkinteger(L, 2); // 2nd argument is the position - /* move up elements if necessary */ + // move up elements if necessary if (1 <= pos && pos <= n) moveelements(L, 1, 1, pos, n, pos + 1); break; @@ -159,7 +159,7 @@ static int tinsert(lua_State* L) luaL_error(L, "wrong number of arguments to 'insert'"); } } - lua_rawseti(L, 1, pos); /* t[pos] = v */ + lua_rawseti(L, 1, pos); // t[pos] = v return 0; } @@ -169,14 +169,14 @@ static int tremove(lua_State* L) int n = lua_objlen(L, 1); int pos = luaL_optinteger(L, 2, n); - if (!(1 <= pos && pos <= n)) /* position is outside bounds? */ - return 0; /* nothing to remove */ - lua_rawgeti(L, 1, pos); /* result = t[pos] */ + if (!(1 <= pos && pos <= n)) // position is outside bounds? + return 0; // nothing to remove + lua_rawgeti(L, 1, pos); // result = t[pos] moveelements(L, 1, 1, pos + 1, n, pos); lua_pushnil(L); - lua_rawseti(L, 1, n); /* t[n] = nil */ + lua_rawseti(L, 1, n); // t[n] = nil return 1; } @@ -192,28 +192,28 @@ static int tmove(lua_State* L) int f = luaL_checkinteger(L, 2); int e = luaL_checkinteger(L, 3); int t = luaL_checkinteger(L, 4); - int tt = !lua_isnoneornil(L, 5) ? 5 : 1; /* destination table */ + int tt = !lua_isnoneornil(L, 5) ? 5 : 1; // destination table luaL_checktype(L, tt, LUA_TTABLE); if (e >= f) - { /* otherwise, nothing to move */ + { // otherwise, nothing to move luaL_argcheck(L, f > 0 || e < INT_MAX + f, 3, "too many elements to move"); - int n = e - f + 1; /* number of elements to move */ + int n = e - f + 1; // number of elements to move luaL_argcheck(L, t <= INT_MAX - n + 1, 4, "destination wrap around"); Table* dst = hvalue(L->base + (tt - 1)); - if (dst->readonly) /* also checked in moveelements, but this blocks resizes of r/o tables */ + if (dst->readonly) // also checked in moveelements, but this blocks resizes of r/o tables luaG_readonlyerror(L); if (t > 0 && (t - 1) <= dst->sizearray && (t - 1 + n) > dst->sizearray) - { /* grow the destination table array */ + { // grow the destination table array luaH_resizearray(L, dst, t - 1 + n); } moveelements(L, 1, tt, f, e, t); } - lua_pushvalue(L, tt); /* return destination table */ + lua_pushvalue(L, tt); // return destination table return 1; } @@ -240,7 +240,7 @@ static int tconcat(lua_State* L) addfield(L, &b, i); luaL_addlstring(&b, sep, lsep); } - if (i == last) /* add last value (if interval was not empty) */ + if (i == last) // add last value (if interval was not empty) addfield(L, &b, i); luaL_pushresult(&b); return 1; @@ -248,8 +248,8 @@ static int tconcat(lua_State* L) static int tpack(lua_State* L) { - int n = lua_gettop(L); /* number of elements to pack */ - lua_createtable(L, n, 1); /* create result table */ + int n = lua_gettop(L); // number of elements to pack + lua_createtable(L, n, 1); // create result table Table* t = hvalue(L->top - 1); @@ -259,11 +259,11 @@ static int tpack(lua_State* L) setobj2t(L, e, L->base + i); } - /* t.n = number of elements */ + // t.n = number of elements TValue* nv = luaH_setstr(L, t, luaS_newliteral(L, "n")); setnvalue(nv, n); - return 1; /* return table */ + return 1; // return table } static int tunpack(lua_State* L) @@ -274,8 +274,8 @@ static int tunpack(lua_State* L) int i = luaL_optinteger(L, 2, 1); int e = luaL_opt(L, luaL_checkinteger, 3, lua_objlen(L, 1)); if (i > e) - return 0; /* empty range */ - unsigned n = (unsigned)e - i; /* number of elements minus 1 (avoid overflows) */ + return 0; // empty range + unsigned n = (unsigned)e - i; // number of elements minus 1 (avoid overflows) if (n >= (unsigned int)INT_MAX || !lua_checkstack(L, (int)(++n))) luaL_error(L, "too many results to unpack"); @@ -288,10 +288,10 @@ static int tunpack(lua_State* L) } else { - /* push arg[i..e - 1] (to avoid overflows) */ + // push arg[i..e - 1] (to avoid overflows) for (; i < e; i++) lua_rawgeti(L, 1, i); - lua_rawgeti(L, 1, e); /* push last element */ + lua_rawgeti(L, 1, e); // push last element } return (int)n; } @@ -312,85 +312,85 @@ static void set2(lua_State* L, int i, int j) static int sort_comp(lua_State* L, int a, int b) { if (!lua_isnil(L, 2)) - { /* function? */ + { // function? int res; lua_pushvalue(L, 2); - lua_pushvalue(L, a - 1); /* -1 to compensate function */ - lua_pushvalue(L, b - 2); /* -2 to compensate function and `a' */ + lua_pushvalue(L, a - 1); // -1 to compensate function + lua_pushvalue(L, b - 2); // -2 to compensate function and `a' lua_call(L, 2, 1); res = lua_toboolean(L, -1); lua_pop(L, 1); return res; } - else /* a < b? */ + else // a < b? return lua_lessthan(L, a, b); } static void auxsort(lua_State* L, int l, int u) { while (l < u) - { /* for tail recursion */ + { // for tail recursion int i, j; - /* sort elements a[l], a[(l+u)/2] and a[u] */ + // sort elements a[l], a[(l+u)/2] and a[u] lua_rawgeti(L, 1, l); lua_rawgeti(L, 1, u); - if (sort_comp(L, -1, -2)) /* a[u] < a[l]? */ - set2(L, l, u); /* swap a[l] - a[u] */ + if (sort_comp(L, -1, -2)) // a[u] < a[l]? + set2(L, l, u); // swap a[l] - a[u] else lua_pop(L, 2); if (u - l == 1) - break; /* only 2 elements */ + break; // only 2 elements i = (l + u) / 2; lua_rawgeti(L, 1, i); lua_rawgeti(L, 1, l); - if (sort_comp(L, -2, -1)) /* a[i]= P */ + { // invariant: a[l..i] <= P <= a[j..u] + // repeat ++i until a[i] >= P while (lua_rawgeti(L, 1, ++i), sort_comp(L, -1, -2)) { if (i >= u) luaL_error(L, "invalid order function for sorting"); - lua_pop(L, 1); /* remove a[i] */ + lua_pop(L, 1); // remove a[i] } - /* repeat --j until a[j] <= P */ + // repeat --j until a[j] <= P while (lua_rawgeti(L, 1, --j), sort_comp(L, -3, -1)) { if (j <= l) luaL_error(L, "invalid order function for sorting"); - lua_pop(L, 1); /* remove a[j] */ + lua_pop(L, 1); // remove a[j] } if (j < i) { - lua_pop(L, 3); /* pop pivot, a[i], a[j] */ + lua_pop(L, 3); // pop pivot, a[i], a[j] break; } set2(L, i, j); } lua_rawgeti(L, 1, u - 1); lua_rawgeti(L, 1, i); - set2(L, u - 1, i); /* swap pivot (a[u-1]) with a[i] */ - /* a[l..i-1] <= a[i] == P <= a[i+1..u] */ - /* adjust so that smaller half is in [j..i] and larger one in [l..u] */ + set2(L, u - 1, i); // swap pivot (a[u-1]) with a[i] + // a[l..i-1] <= a[i] == P <= a[i+1..u] + // adjust so that smaller half is in [j..i] and larger one in [l..u] if (i - l < u - i) { j = l; @@ -403,23 +403,23 @@ static void auxsort(lua_State* L, int l, int u) i = u; u = j - 2; } - auxsort(L, j, i); /* call recursively the smaller one */ - } /* repeat the routine for the larger one */ + auxsort(L, j, i); // call recursively the smaller one + } // repeat the routine for the larger one } static int sort(lua_State* L) { luaL_checktype(L, 1, LUA_TTABLE); int n = lua_objlen(L, 1); - luaL_checkstack(L, 40, ""); /* assume array is smaller than 2^40 */ - if (!lua_isnoneornil(L, 2)) /* is there a 2nd argument? */ + luaL_checkstack(L, 40, ""); // assume array is smaller than 2^40 + if (!lua_isnoneornil(L, 2)) // is there a 2nd argument? luaL_checktype(L, 2, LUA_TFUNCTION); - lua_settop(L, 2); /* make sure there is two arguments */ + lua_settop(L, 2); // make sure there is two arguments auxsort(L, 1, n); return 0; } -/* }====================================================== */ +// }====================================================== static int tcreate(lua_State* L) { diff --git a/VM/src/ltm.cpp b/VM/src/ltm.cpp index 49982b28..cb7ba097 100644 --- a/VM/src/ltm.cpp +++ b/VM/src/ltm.cpp @@ -12,7 +12,7 @@ // clang-format off const char* const luaT_typenames[] = { - /* ORDER TYPE */ + // ORDER TYPE "nil", "boolean", @@ -31,7 +31,7 @@ const char* const luaT_typenames[] = { }; const char* const luaT_eventname[] = { - /* ORDER TM */ + // ORDER TM "__index", "__newindex", @@ -70,12 +70,12 @@ void luaT_init(lua_State* L) for (i = 0; i < LUA_T_COUNT; i++) { L->global->ttname[i] = luaS_new(L, luaT_typenames[i]); - luaS_fix(L->global->ttname[i]); /* never collect these names */ + luaS_fix(L->global->ttname[i]); // never collect these names } for (i = 0; i < TM_N; i++) { L->global->tmname[i] = luaS_new(L, luaT_eventname[i]); - luaS_fix(L->global->tmname[i]); /* never collect these names */ + luaS_fix(L->global->tmname[i]); // never collect these names } } @@ -88,8 +88,8 @@ const TValue* luaT_gettm(Table* events, TMS event, TString* ename) const TValue* tm = luaH_getstr(events, ename); LUAU_ASSERT(event <= TM_EQ); if (ttisnil(tm)) - { /* no tag method? */ - events->tmcache |= cast_byte(1u << event); /* cache this fact */ + { // no tag method? + events->tmcache |= cast_byte(1u << event); // cache this fact return NULL; } else diff --git a/VM/src/ltm.h b/VM/src/ltm.h index e11ddb3a..f20ce1b2 100644 --- a/VM/src/ltm.h +++ b/VM/src/ltm.h @@ -20,7 +20,7 @@ typedef enum TM_ITER, TM_LEN, - TM_EQ, /* last tag method with `fast' access */ + TM_EQ, // last tag method with `fast' access TM_ADD, @@ -37,7 +37,7 @@ typedef enum TM_CONCAT, TM_TYPE, - TM_N /* number of elements in the enum */ + TM_N // number of elements in the enum } TMS; // clang-format on diff --git a/VM/src/ludata.h b/VM/src/ludata.h index f24e4a32..9b7ba26a 100644 --- a/VM/src/ludata.h +++ b/VM/src/ludata.h @@ -4,10 +4,10 @@ #include "lobject.h" -/* special tag value is used for user data with inline dtors */ +// special tag value is used for user data with inline dtors #define UTAG_IDTOR LUA_UTAG_LIMIT -/* special tag value is used for newproxy-created user data (all other user data objects are host-exposed) */ +// special tag value is used for newproxy-created user data (all other user data objects are host-exposed) #define UTAG_PROXY (LUA_UTAG_LIMIT + 1) #define sizeudata(len) (offsetof(Udata, data) + len) diff --git a/VM/src/lutf8lib.cpp b/VM/src/lutf8lib.cpp index 8bc8200a..837d0e12 100644 --- a/VM/src/lutf8lib.cpp +++ b/VM/src/lutf8lib.cpp @@ -8,8 +8,8 @@ #define iscont(p) ((*(p)&0xC0) == 0x80) -/* from strlib */ -/* translate a relative string position: negative means back from end */ +// from strlib +// translate a relative string position: negative means back from end static int u_posrelat(int pos, size_t len) { if (pos >= 0) @@ -28,28 +28,28 @@ static const char* utf8_decode(const char* o, int* val) static const unsigned int limits[] = {0xFF, 0x7F, 0x7FF, 0xFFFF}; const unsigned char* s = (const unsigned char*)o; unsigned int c = s[0]; - unsigned int res = 0; /* final result */ - if (c < 0x80) /* ascii? */ + unsigned int res = 0; // final result + if (c < 0x80) // ascii? res = c; else { - int count = 0; /* to count number of continuation bytes */ + int count = 0; // to count number of continuation bytes while (c & 0x40) - { /* still have continuation bytes? */ - int cc = s[++count]; /* read next byte */ - if ((cc & 0xC0) != 0x80) /* not a continuation byte? */ - return NULL; /* invalid byte sequence */ - res = (res << 6) | (cc & 0x3F); /* add lower 6 bits from cont. byte */ - c <<= 1; /* to test next bit */ + { // still have continuation bytes? + int cc = s[++count]; // read next byte + if ((cc & 0xC0) != 0x80) // not a continuation byte? + return NULL; // invalid byte sequence + res = (res << 6) | (cc & 0x3F); // add lower 6 bits from cont. byte + c <<= 1; // to test next bit } - res |= ((c & 0x7F) << (count * 5)); /* add first byte */ + res |= ((c & 0x7F) << (count * 5)); // add first byte if (count > 3 || res > MAXUNICODE || res <= limits[count]) - return NULL; /* invalid byte sequence */ - s += count; /* skip continuation bytes read */ + return NULL; // invalid byte sequence + s += count; // skip continuation bytes read } if (val) *val = res; - return (const char*)s + 1; /* +1 to include first byte */ + return (const char*)s + 1; // +1 to include first byte } /* @@ -70,9 +70,9 @@ static int utflen(lua_State* L) { const char* s1 = utf8_decode(s + posi, NULL); if (s1 == NULL) - { /* conversion error? */ - lua_pushnil(L); /* return nil ... */ - lua_pushinteger(L, posi + 1); /* ... and current position */ + { // conversion error? + lua_pushnil(L); // return nil ... + lua_pushinteger(L, posi + 1); // ... and current position return 2; } posi = (int)(s1 - s); @@ -97,8 +97,8 @@ static int codepoint(lua_State* L) luaL_argcheck(L, posi >= 1, 2, "out of range"); luaL_argcheck(L, pose <= (int)len, 3, "out of range"); if (posi > pose) - return 0; /* empty interval; return no values */ - if (pose - posi >= INT_MAX) /* (int -> int) overflow? */ + return 0; // empty interval; return no values + if (pose - posi >= INT_MAX) // (int -> int) overflow? luaL_error(L, "string slice too long"); n = (int)(pose - posi) + 1; luaL_checkstack(L, n, "string slice too long"); @@ -122,20 +122,20 @@ static int codepoint(lua_State* L) // from Lua 5.3 lobject.c, copied verbatim + static static int luaO_utf8esc(char* buff, unsigned long x) { - int n = 1; /* number of bytes put in buffer (backwards) */ + int n = 1; // number of bytes put in buffer (backwards) LUAU_ASSERT(x <= 0x10FFFF); - if (x < 0x80) /* ascii? */ + if (x < 0x80) // ascii? buff[UTF8BUFFSZ - 1] = cast_to(char, x); else - { /* need continuation bytes */ - unsigned int mfb = 0x3f; /* maximum that fits in first byte */ + { // need continuation bytes + unsigned int mfb = 0x3f; // maximum that fits in first byte do - { /* add continuation bytes */ + { // add continuation bytes buff[UTF8BUFFSZ - (n++)] = cast_to(char, 0x80 | (x & 0x3f)); - x >>= 6; /* remove added bits */ - mfb >>= 1; /* now there is one less bit available in first byte */ - } while (x > mfb); /* still needs continuation byte? */ - buff[UTF8BUFFSZ - n] = cast_to(char, (~mfb << 1) | x); /* add first byte */ + x >>= 6; // remove added bits + mfb >>= 1; // now there is one less bit available in first byte + } while (x > mfb); // still needs continuation byte? + buff[UTF8BUFFSZ - n] = cast_to(char, (~mfb << 1) | x); // add first byte } return n; } @@ -162,9 +162,9 @@ static int utfchar(lua_State* L) char buff[UTF8BUFFSZ]; const char* charstr; - int n = lua_gettop(L); /* number of arguments */ + int n = lua_gettop(L); // number of arguments if (n == 1) - { /* optimize common case of single char */ + { // optimize common case of single char int l = buffutfchar(L, 1, buff, &charstr); lua_pushlstring(L, charstr, l); } @@ -196,7 +196,7 @@ static int byteoffset(lua_State* L) luaL_argcheck(L, 1 <= posi && --posi <= (int)len, 3, "position out of range"); if (n == 0) { - /* find beginning of current byte sequence */ + // find beginning of current byte sequence while (posi > 0 && iscont(s + posi)) posi--; } @@ -207,9 +207,9 @@ static int byteoffset(lua_State* L) if (n < 0) { while (n < 0 && posi > 0) - { /* move back */ + { // move back do - { /* find beginning of previous character */ + { // find beginning of previous character posi--; } while (posi > 0 && iscont(s + posi)); n++; @@ -217,20 +217,20 @@ static int byteoffset(lua_State* L) } else { - n--; /* do not move for 1st character */ + n--; // do not move for 1st character while (n > 0 && posi < (int)len) { do - { /* find beginning of next character */ + { // find beginning of next character posi++; - } while (iscont(s + posi)); /* (cannot pass final '\0') */ + } while (iscont(s + posi)); // (cannot pass final '\0') n--; } } } - if (n == 0) /* did it find given character? */ + if (n == 0) // did it find given character? lua_pushinteger(L, posi + 1); - else /* no such character */ + else // no such character lua_pushnil(L); return 1; } @@ -240,16 +240,16 @@ static int iter_aux(lua_State* L) size_t len; const char* s = luaL_checklstring(L, 1, &len); int n = lua_tointeger(L, 2) - 1; - if (n < 0) /* first iteration? */ - n = 0; /* start from here */ + if (n < 0) // first iteration? + n = 0; // start from here else if (n < (int)len) { - n++; /* skip current byte */ + n++; // skip current byte while (iscont(s + n)) - n++; /* and its continuations */ + n++; // and its continuations } if (n >= (int)len) - return 0; /* no more codepoints */ + return 0; // no more codepoints else { int code; @@ -271,7 +271,7 @@ static int iter_codes(lua_State* L) return 3; } -/* pattern to match a single UTF-8 character */ +// pattern to match a single UTF-8 character #define UTF8PATT "[\0-\x7F\xC2-\xF4][\x80-\xBF]*" static const luaL_Reg funcs[] = { diff --git a/VM/src/lvmexecute.cpp b/VM/src/lvmexecute.cpp index 10d89aaf..376dd400 100644 --- a/VM/src/lvmexecute.cpp +++ b/VM/src/lvmexecute.cpp @@ -16,7 +16,7 @@ #include -LUAU_FASTFLAGVARIABLE(LuauLenTM, false) +LUAU_FASTFLAGVARIABLE(LuauNicerMethodErrors, false) // Disable c99-designator to avoid the warning in CGOTO dispatch table #ifdef __clang__ @@ -110,7 +110,8 @@ LUAU_FASTFLAGVARIABLE(LuauLenTM, false) VM_DISPATCH_OP(LOP_FORGLOOP_NEXT), VM_DISPATCH_OP(LOP_GETVARARGS), VM_DISPATCH_OP(LOP_DUPCLOSURE), VM_DISPATCH_OP(LOP_PREPVARARGS), \ VM_DISPATCH_OP(LOP_LOADKX), VM_DISPATCH_OP(LOP_JUMPX), VM_DISPATCH_OP(LOP_FASTCALL), VM_DISPATCH_OP(LOP_COVERAGE), \ VM_DISPATCH_OP(LOP_CAPTURE), VM_DISPATCH_OP(LOP_JUMPIFEQK), VM_DISPATCH_OP(LOP_JUMPIFNOTEQK), VM_DISPATCH_OP(LOP_FASTCALL1), \ - VM_DISPATCH_OP(LOP_FASTCALL2), VM_DISPATCH_OP(LOP_FASTCALL2K), VM_DISPATCH_OP(LOP_FORGPREP), + VM_DISPATCH_OP(LOP_FASTCALL2), VM_DISPATCH_OP(LOP_FASTCALL2K), VM_DISPATCH_OP(LOP_FORGPREP), VM_DISPATCH_OP(LOP_JUMPXEQKNIL), \ + VM_DISPATCH_OP(LOP_JUMPXEQKB), VM_DISPATCH_OP(LOP_JUMPXEQKN), VM_DISPATCH_OP(LOP_JUMPXEQKS), \ #if defined(__GNUC__) || defined(__clang__) #define VM_USE_CGOTO 1 @@ -158,7 +159,7 @@ LUAU_NOINLINE static bool luau_loopFORG(lua_State* L, int a, int c) setobjs2s(L, ra + 3 + 1, ra + 1); setobjs2s(L, ra + 3, ra); - L->top = ra + 3 + 3; /* func. + 2 args (state and index) */ + L->top = ra + 3 + 3; // func. + 2 args (state and index) LUAU_ASSERT(L->top <= L->stack_last); luaD_call(L, ra + 3, c); @@ -236,10 +237,10 @@ LUAU_NOINLINE static void luau_tryfuncTM(lua_State* L, StkId func) const TValue* tm = luaT_gettmbyobj(L, func, TM_CALL); if (!ttisfunction(tm)) luaG_typeerror(L, func, "call"); - for (StkId p = L->top; p > func; p--) /* open space for metamethod */ + for (StkId p = L->top; p > func; p--) // open space for metamethod setobjs2s(L, p, p - 1); - L->top++; /* stack space pre-allocated by the caller */ - setobj2s(L, func, tm); /* tag method is the new function to be called */ + L->top++; // stack space pre-allocated by the caller + setobj2s(L, func, tm); // tag method is the new function to be called } LUAU_NOINLINE void luau_callhook(lua_State* L, lua_Hook hook, void* userdata) @@ -256,7 +257,7 @@ LUAU_NOINLINE void luau_callhook(lua_State* L, lua_Hook hook, void* userdata) L->base = L->ci->base; } - luaD_checkstack(L, LUA_MINSTACK); /* ensure minimum stack size */ + luaD_checkstack(L, LUA_MINSTACK); // ensure minimum stack size L->ci->top = L->top + LUA_MINSTACK; LUAU_ASSERT(L->ci->top <= L->stack_last); @@ -929,6 +930,10 @@ static void luau_execute(lua_State* L) VM_PROTECT(luaV_gettable(L, rb, kv, ra)); // save cachedslot to accelerate future lookups; patches currently executing instruction since pc-2 rolls back two pc++ VM_PATCH_C(pc - 2, L->cachedslot); + // recompute ra since stack might have been reallocated + ra = VM_REG(LUAU_INSN_A(insn)); + if (FFlag::LuauNicerMethodErrors && ttisnil(ra)) + luaG_methoderror(L, ra + 1, tsvalue(kv)); } } else @@ -966,6 +971,10 @@ static void luau_execute(lua_State* L) VM_PROTECT(luaV_gettable(L, rb, kv, ra)); // save cachedslot to accelerate future lookups; patches currently executing instruction since pc-2 rolls back two pc++ VM_PATCH_C(pc - 2, L->cachedslot); + // recompute ra since stack might have been reallocated + ra = VM_REG(LUAU_INSN_A(insn)); + if (FFlag::LuauNicerMethodErrors && ttisnil(ra)) + luaG_methoderror(L, ra + 1, tsvalue(kv)); } } else @@ -973,6 +982,10 @@ static void luau_execute(lua_State* L) // slow-path: handles non-table __index setobj2s(L, ra + 1, rb); VM_PROTECT(luaV_gettable(L, rb, kv, ra)); + // recompute ra since stack might have been reallocated + ra = VM_REG(LUAU_INSN_A(insn)); + if (FFlag::LuauNicerMethodErrors && ttisnil(ra)) + luaG_methoderror(L, ra + 1, tsvalue(kv)); } } @@ -1028,7 +1041,7 @@ static void luau_execute(lua_State* L) StkId argi = L->top; StkId argend = L->base + p->numparams; while (argi < argend) - setnilvalue(argi++); /* complete missing arguments */ + setnilvalue(argi++); // complete missing arguments L->top = p->is_vararg ? argi : ci->top; // reentry @@ -2074,7 +2087,7 @@ static void luau_execute(lua_State* L) { Table* h = hvalue(rb); - if (!FFlag::LuauLenTM || fastnotm(h->metatable, TM_LEN)) + if (fastnotm(h->metatable, TM_LEN)) { setnvalue(ra, cast_num(luaH_getn(h))); VM_NEXT(); @@ -2214,7 +2227,7 @@ static void luau_execute(lua_State* L) if (ttisfunction(ra)) { - /* will be called during FORGLOOP */ + // will be called during FORGLOOP } else { @@ -2225,16 +2238,16 @@ static void luau_execute(lua_State* L) setobj2s(L, ra + 1, ra); setobj2s(L, ra, fn); - L->top = ra + 2; /* func + self arg */ + L->top = ra + 2; // func + self arg LUAU_ASSERT(L->top <= L->stack_last); VM_PROTECT(luaD_call(L, ra, 3)); L->top = L->ci->top; - /* recompute ra since stack might have been reallocated */ + // recompute ra since stack might have been reallocated ra = VM_REG(LUAU_INSN_A(insn)); - /* protect against __iter returning nil, since nil is used as a marker for builtin iteration in FORGLOOP */ + // protect against __iter returning nil, since nil is used as a marker for builtin iteration in FORGLOOP if (ttisnil(ra)) { VM_PROTECT(luaG_typeerror(L, ra, "call")); @@ -2242,12 +2255,12 @@ static void luau_execute(lua_State* L) } else if (fasttm(L, mt, TM_CALL)) { - /* table or userdata with __call, will be called during FORGLOOP */ - /* TODO: we might be able to stop supporting this depending on whether it's used in practice */ + // table or userdata with __call, will be called during FORGLOOP + // TODO: we might be able to stop supporting this depending on whether it's used in practice } else if (ttistable(ra)) { - /* set up registers for builtin iteration */ + // set up registers for builtin iteration setobj2s(L, ra + 1, ra); setpvalue(ra + 2, reinterpret_cast(uintptr_t(0))); setnilvalue(ra); @@ -2344,7 +2357,7 @@ static void luau_execute(lua_State* L) setobjs2s(L, ra + 3 + 1, ra + 1); setobjs2s(L, ra + 3, ra); - L->top = ra + 3 + 3; /* func + 2 args (state and index) */ + L->top = ra + 3 + 3; // func + 2 args (state and index) LUAU_ASSERT(L->top <= L->stack_last); VM_PROTECT(luaD_call(L, ra + 3, uint8_t(aux))); @@ -2372,7 +2385,7 @@ static void luau_execute(lua_State* L) if (cl->env->safeenv && ttistable(ra + 1) && ttisnumber(ra + 2) && nvalue(ra + 2) == 0.0) { setnilvalue(ra); - /* ra+1 is already the table */ + // ra+1 is already the table setpvalue(ra + 2, reinterpret_cast(uintptr_t(0))); } else if (!ttisfunction(ra)) @@ -2444,7 +2457,7 @@ static void luau_execute(lua_State* L) if (cl->env->safeenv && ttistable(ra + 1) && ttisnil(ra + 2)) { setnilvalue(ra); - /* ra+1 is already the table */ + // ra+1 is already the table setpvalue(ra + 2, reinterpret_cast(uintptr_t(0))); } else if (!ttisfunction(ra)) @@ -2619,8 +2632,8 @@ static void luau_execute(lua_State* L) LUAU_ASSERT(cast_int(L->top - base) >= numparams); // move fixed parameters to final position - StkId fixed = base; /* first fixed argument */ - base = L->top; /* final position of first argument */ + StkId fixed = base; // first fixed argument + base = L->top; // final position of first argument for (int i = 0; i < numparams; ++i) { @@ -2983,6 +2996,56 @@ static void luau_execute(lua_State* L) VM_CONTINUE(op); } + VM_CASE(LOP_JUMPXEQKNIL) + { + Instruction insn = *pc++; + uint32_t aux = *pc; + StkId ra = VM_REG(LUAU_INSN_A(insn)); + + static_assert(LUA_TNIL == 0, "we expect type-1 to be negative iff type is nil"); + // condition is equivalent to: int(ttisnil(ra)) != (aux >> 31) + pc += int((ttype(ra) - 1) ^ aux) < 0 ? LUAU_INSN_D(insn) : 1; + LUAU_ASSERT(unsigned(pc - cl->l.p->code) < unsigned(cl->l.p->sizecode)); + VM_NEXT(); + } + + VM_CASE(LOP_JUMPXEQKB) + { + Instruction insn = *pc++; + uint32_t aux = *pc; + StkId ra = VM_REG(LUAU_INSN_A(insn)); + + pc += int(ttisboolean(ra) && bvalue(ra) == int(aux & 1)) != (aux >> 31) ? LUAU_INSN_D(insn) : 1; + LUAU_ASSERT(unsigned(pc - cl->l.p->code) < unsigned(cl->l.p->sizecode)); + VM_NEXT(); + } + + VM_CASE(LOP_JUMPXEQKN) + { + Instruction insn = *pc++; + uint32_t aux = *pc; + StkId ra = VM_REG(LUAU_INSN_A(insn)); + TValue* kv = VM_KV(aux & 0xffffff); + LUAU_ASSERT(ttisnumber(kv)); + + pc += int(ttisnumber(ra) && nvalue(ra) == nvalue(kv)) != (aux >> 31) ? LUAU_INSN_D(insn) : 1; + LUAU_ASSERT(unsigned(pc - cl->l.p->code) < unsigned(cl->l.p->sizecode)); + VM_NEXT(); + } + + VM_CASE(LOP_JUMPXEQKS) + { + Instruction insn = *pc++; + uint32_t aux = *pc; + StkId ra = VM_REG(LUAU_INSN_A(insn)); + TValue* kv = VM_KV(aux & 0xffffff); + LUAU_ASSERT(ttisstring(kv)); + + pc += int(ttisstring(ra) && gcvalue(ra) == gcvalue(kv)) != (aux >> 31) ? LUAU_INSN_D(insn) : 1; + LUAU_ASSERT(unsigned(pc - cl->l.p->code) < unsigned(cl->l.p->sizecode)); + VM_NEXT(); + } + #if !VM_USE_CGOTO default: LUAU_ASSERT(!"Unknown opcode"); @@ -3032,7 +3095,7 @@ int luau_precall(lua_State* L, StkId func, int nresults) StkId argi = L->top; StkId argend = L->base + ccl->l.p->numparams; while (argi < argend) - setnilvalue(argi++); /* complete missing arguments */ + setnilvalue(argi++); // complete missing arguments L->top = ccl->l.p->is_vararg ? argi : ci->top; L->ci->savedpc = ccl->l.p->code; diff --git a/VM/src/lvmutils.cpp b/VM/src/lvmutils.cpp index b9267fb8..8be241e0 100644 --- a/VM/src/lvmutils.cpp +++ b/VM/src/lvmutils.cpp @@ -12,11 +12,9 @@ #include #include -LUAU_FASTFLAG(LuauLenTM) - LUAU_FASTFLAGVARIABLE(LuauBetterNewindex, false) -/* limit for table tag-method chains (to avoid loops) */ +// limit for table tag-method chains (to avoid loops) #define MAXTAGLOOP 100 const TValue* luaV_tonumber(const TValue* obj, TValue* n) @@ -67,9 +65,9 @@ static StkId callTMres(lua_State* L, StkId res, const TValue* f, const TValue* p // * during stack reallocation all of the allocated stack is copied (even beyond stack_last) so these // values will be preserved even if they go past stack_last LUAU_ASSERT((L->top + 3) < (L->stack + L->stacksize)); - setobj2s(L, L->top, f); /* push function */ - setobj2s(L, L->top + 1, p1); /* 1st argument */ - setobj2s(L, L->top + 2, p2); /* 2nd argument */ + setobj2s(L, L->top, f); // push function + setobj2s(L, L->top + 1, p1); // 1st argument + setobj2s(L, L->top + 2, p2); // 2nd argument luaD_checkstack(L, 3); L->top += 3; luaD_call(L, L->top - 3, 1); @@ -89,10 +87,10 @@ static void callTM(lua_State* L, const TValue* f, const TValue* p1, const TValue // * during stack reallocation all of the allocated stack is copied (even beyond stack_last) so these // values will be preserved even if they go past stack_last LUAU_ASSERT((L->top + 4) < (L->stack + L->stacksize)); - setobj2s(L, L->top, f); /* push function */ - setobj2s(L, L->top + 1, p1); /* 1st argument */ - setobj2s(L, L->top + 2, p2); /* 2nd argument */ - setobj2s(L, L->top + 3, p3); /* 3th argument */ + setobj2s(L, L->top, f); // push function + setobj2s(L, L->top + 1, p1); // 1st argument + setobj2s(L, L->top + 2, p2); // 2nd argument + setobj2s(L, L->top + 3, p3); // 3th argument luaD_checkstack(L, 4); L->top += 4; luaD_call(L, L->top - 4, 0); @@ -105,21 +103,21 @@ void luaV_gettable(lua_State* L, const TValue* t, TValue* key, StkId val) { const TValue* tm; if (ttistable(t)) - { /* `t' is a table? */ + { // `t' is a table? Table* h = hvalue(t); - const TValue* res = luaH_get(h, key); /* do a primitive get */ + const TValue* res = luaH_get(h, key); // do a primitive get if (res != luaO_nilobject) - L->cachedslot = gval2slot(h, res); /* remember slot to accelerate future lookups */ + L->cachedslot = gval2slot(h, res); // remember slot to accelerate future lookups - if (!ttisnil(res) /* result is no nil? */ + if (!ttisnil(res) // result is no nil? || (tm = fasttm(L, h->metatable, TM_INDEX)) == NULL) - { /* or no TM? */ + { // or no TM? setobj2s(L, val, res); return; } - /* t isn't a table, so see if it has an INDEX meta-method to look up the key with */ + // t isn't a table, so see if it has an INDEX meta-method to look up the key with } else if (ttisnil(tm = luaT_gettmbyobj(L, t, TM_INDEX))) luaG_indexerror(L, t, key); @@ -128,7 +126,7 @@ void luaV_gettable(lua_State* L, const TValue* t, TValue* key, StkId val) callTMres(L, val, tm, t, key); return; } - t = tm; /* else repeat with `tm' */ + t = tm; // else repeat with `tm' } luaG_runerror(L, "'__index' chain too long; possible loop"); } @@ -141,48 +139,48 @@ void luaV_settable(lua_State* L, const TValue* t, TValue* key, StkId val) { const TValue* tm; if (ttistable(t)) - { /* `t' is a table? */ + { // `t' is a table? Table* h = hvalue(t); if (FFlag::LuauBetterNewindex) { const TValue* oldval = luaH_get(h, key); - /* should we assign the key? (if key is valid or __newindex is not set) */ + // should we assign the key? (if key is valid or __newindex is not set) if (!ttisnil(oldval) || (tm = fasttm(L, h->metatable, TM_NEWINDEX)) == NULL) { if (h->readonly) luaG_readonlyerror(L); - /* luaH_set would work but would repeat the lookup so we use luaH_setslot that can reuse oldval if it's safe */ + // luaH_set would work but would repeat the lookup so we use luaH_setslot that can reuse oldval if it's safe TValue* newval = luaH_setslot(L, h, oldval, key); - L->cachedslot = gval2slot(h, newval); /* remember slot to accelerate future lookups */ + L->cachedslot = gval2slot(h, newval); // remember slot to accelerate future lookups setobj2t(L, newval, val); luaC_barriert(L, h, val); return; } - /* fallthrough to metamethod */ + // fallthrough to metamethod } else { if (h->readonly) luaG_readonlyerror(L); - TValue* oldval = luaH_set(L, h, key); /* do a primitive set */ + TValue* oldval = luaH_set(L, h, key); // do a primitive set - L->cachedslot = gval2slot(h, oldval); /* remember slot to accelerate future lookups */ + L->cachedslot = gval2slot(h, oldval); // remember slot to accelerate future lookups - if (!ttisnil(oldval) || /* result is no nil? */ + if (!ttisnil(oldval) || // result is no nil? (tm = fasttm(L, h->metatable, TM_NEWINDEX)) == NULL) - { /* or no TM? */ + { // or no TM? setobj2t(L, oldval, val); luaC_barriert(L, h, val); return; } - /* else will try the tag method */ + // else will try the tag method } } else if (ttisnil(tm = luaT_gettmbyobj(L, t, TM_NEWINDEX))) @@ -193,8 +191,8 @@ void luaV_settable(lua_State* L, const TValue* t, TValue* key, StkId val) callTM(L, tm, t, key, val); return; } - /* else repeat with `tm' */ - setobj(L, &temp, tm); /* avoid pointing inside table (may rehash) */ + // else repeat with `tm' + setobj(L, &temp, tm); // avoid pointing inside table (may rehash) t = &temp; } luaG_runerror(L, "'__newindex' chain too long; possible loop"); @@ -202,9 +200,9 @@ void luaV_settable(lua_State* L, const TValue* t, TValue* key, StkId val) static int call_binTM(lua_State* L, const TValue* p1, const TValue* p2, StkId res, TMS event) { - const TValue* tm = luaT_gettmbyobj(L, p1, event); /* try first operand */ + const TValue* tm = luaT_gettmbyobj(L, p1, event); // try first operand if (ttisnil(tm)) - tm = luaT_gettmbyobj(L, p2, event); /* try second operand */ + tm = luaT_gettmbyobj(L, p2, event); // try second operand if (ttisnil(tm)) return 0; callTMres(L, res, tm, p1, p2); @@ -216,13 +214,13 @@ static const TValue* get_compTM(lua_State* L, Table* mt1, Table* mt2, TMS event) const TValue* tm1 = fasttm(L, mt1, event); const TValue* tm2; if (tm1 == NULL) - return NULL; /* no metamethod */ + return NULL; // no metamethod if (mt1 == mt2) - return tm1; /* same metatables => same metamethods */ + return tm1; // same metatables => same metamethods tm2 = fasttm(L, mt2, event); if (tm2 == NULL) - return NULL; /* no metamethod */ - if (luaO_rawequalObj(tm1, tm2)) /* same metamethods? */ + return NULL; // no metamethod + if (luaO_rawequalObj(tm1, tm2)) // same metamethods? return tm1; return NULL; } @@ -232,9 +230,9 @@ static int call_orderTM(lua_State* L, const TValue* p1, const TValue* p2, TMS ev const TValue* tm1 = luaT_gettmbyobj(L, p1, event); const TValue* tm2; if (ttisnil(tm1)) - return -1; /* no metamethod? */ + return -1; // no metamethod? tm2 = luaT_gettmbyobj(L, p2, event); - if (!luaO_rawequalObj(tm1, tm2)) /* different metamethods? */ + if (!luaO_rawequalObj(tm1, tm2)) // different metamethods? return -1; callTMres(L, L->top, tm1, p1, p2); return !l_isfalse(L->top); @@ -281,9 +279,9 @@ int luaV_lessequal(lua_State* L, const TValue* l, const TValue* r) return luai_numle(nvalue(l), nvalue(r)); else if (ttisstring(l)) return luaV_strcmp(tsvalue(l), tsvalue(r)) <= 0; - else if ((res = call_orderTM(L, l, r, TM_LE)) != -1) /* first try `le' */ + else if ((res = call_orderTM(L, l, r, TM_LE)) != -1) // first try `le' return res; - else if ((res = call_orderTM(L, r, l, TM_LT)) == -1) /* error if not `lt' */ + else if ((res = call_orderTM(L, r, l, TM_LT)) == -1) // error if not `lt' luaG_ordererror(L, l, r, TM_LE); return !res; } @@ -301,7 +299,7 @@ int luaV_equalval(lua_State* L, const TValue* t1, const TValue* t2) case LUA_TVECTOR: return luai_veceq(vvalue(t1), vvalue(t2)); case LUA_TBOOLEAN: - return bvalue(t1) == bvalue(t2); /* true must be 1 !! */ + return bvalue(t1) == bvalue(t2); // true must be 1 !! case LUA_TLIGHTUSERDATA: return pvalue(t1) == pvalue(t2); case LUA_TUSERDATA: @@ -309,19 +307,19 @@ int luaV_equalval(lua_State* L, const TValue* t1, const TValue* t2) tm = get_compTM(L, uvalue(t1)->metatable, uvalue(t2)->metatable, TM_EQ); if (!tm) return uvalue(t1) == uvalue(t2); - break; /* will try TM */ + break; // will try TM } case LUA_TTABLE: { tm = get_compTM(L, hvalue(t1)->metatable, hvalue(t2)->metatable, TM_EQ); if (!tm) return hvalue(t1) == hvalue(t2); - break; /* will try TM */ + break; // will try TM } default: return gcvalue(t1) == gcvalue(t2); } - callTMres(L, L->top, tm, t1, t2); /* call TM */ + callTMres(L, L->top, tm, t1, t2); // call TM return !l_isfalse(L->top); } @@ -330,21 +328,21 @@ void luaV_concat(lua_State* L, int total, int last) do { StkId top = L->base + last + 1; - int n = 2; /* number of elements handled in this pass (at least 2) */ + int n = 2; // number of elements handled in this pass (at least 2) if (!(ttisstring(top - 2) || ttisnumber(top - 2)) || !tostring(L, top - 1)) { if (!call_binTM(L, top - 2, top - 1, top - 2, TM_CONCAT)) luaG_concaterror(L, top - 2, top - 1); } - else if (tsvalue(top - 1)->len == 0) /* second op is empty? */ - (void)tostring(L, top - 2); /* result is first op (as string) */ + else if (tsvalue(top - 1)->len == 0) // second op is empty? + (void)tostring(L, top - 2); // result is first op (as string) else { - /* at least two string values; get as many as possible */ + // at least two string values; get as many as possible size_t tl = tsvalue(top - 1)->len; char* buffer; int i; - /* collect total length */ + // collect total length for (n = 1; n < total && tostring(L, top - n - 1); n++) { size_t l = tsvalue(top - n - 1)->len; @@ -368,7 +366,7 @@ void luaV_concat(lua_State* L, int total, int last) tl = 0; for (i = n; i > 0; i--) - { /* concat all strings */ + { // concat all strings size_t l = tsvalue(top - i)->len; memcpy(buffer + tl, svalue(top - i), l); tl += l; @@ -383,9 +381,9 @@ void luaV_concat(lua_State* L, int total, int last) setsvalue2s(L, top - n, luaS_buffinish(L, ts)); } } - total -= n - 1; /* got `n' strings to create 1 new */ + total -= n - 1; // got `n' strings to create 1 new last -= n - 1; - } while (total > 1); /* repeat until only 1 result left */ + } while (total > 1); // repeat until only 1 result left } void luaV_doarith(lua_State* L, StkId ra, const TValue* rb, const TValue* rc, TMS op) @@ -504,29 +502,6 @@ void luaV_doarith(lua_State* L, StkId ra, const TValue* rb, const TValue* rc, TM void luaV_dolen(lua_State* L, StkId ra, const TValue* rb) { - if (!FFlag::LuauLenTM) - { - switch (ttype(rb)) - { - case LUA_TTABLE: - { - setnvalue(ra, cast_num(luaH_getn(hvalue(rb)))); - break; - } - case LUA_TSTRING: - { - setnvalue(ra, cast_num(tsvalue(rb)->len)); - break; - } - default: - { /* try metamethod */ - if (!call_binTM(L, rb, luaO_nilobject, ra, TM_LEN)) - luaG_typeerror(L, rb, "get length of"); - } - } - return; - } - const TValue* tm = NULL; switch (ttype(rb)) { @@ -555,5 +530,5 @@ void luaV_dolen(lua_State* L, StkId ra, const TValue* rb) StkId res = callTMres(L, ra, tm, rb, luaO_nilobject); if (!ttisnumber(res)) - luaG_runerror(L, "'__len' must return a number"); /* note, we can't access rb since stack may have been reallocated */ + luaG_runerror(L, "'__len' must return a number"); // note, we can't access rb since stack may have been reallocated } diff --git a/tests/JsonEncoder.test.cpp b/tests/AstJsonEncoder.test.cpp similarity index 99% rename from tests/JsonEncoder.test.cpp rename to tests/AstJsonEncoder.test.cpp index b16ad3e2..3ff36741 100644 --- a/tests/JsonEncoder.test.cpp +++ b/tests/AstJsonEncoder.test.cpp @@ -1,6 +1,6 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #include "Luau/Ast.h" -#include "Luau/JsonEncoder.h" +#include "Luau/AstJsonEncoder.h" #include "Luau/Parser.h" #include "doctest.h" @@ -181,7 +181,8 @@ TEST_CASE("encode_AstExprLocal") AstLocal local{AstName{"foo"}, Location{}, nullptr, 0, 0, nullptr}; AstExprLocal exprLocal{Location{}, &local, false}; - CHECK(toJson(&exprLocal) == R"({"type":"AstExprLocal","location":"0,0 - 0,0","local":{"luauType":null,"name":"foo","type":"AstLocal","location":"0,0 - 0,0"}})"); + CHECK(toJson(&exprLocal) == + R"({"type":"AstExprLocal","location":"0,0 - 0,0","local":{"luauType":null,"name":"foo","type":"AstLocal","location":"0,0 - 0,0"}})"); } TEST_CASE("encode_AstExprVarargs") diff --git a/tests/Autocomplete.test.cpp b/tests/Autocomplete.test.cpp index 0a5603d6..75c5a606 100644 --- a/tests/Autocomplete.test.cpp +++ b/tests/Autocomplete.test.cpp @@ -1974,7 +1974,7 @@ TEST_CASE_FIXTURE(ACFixture, "function_result_passed_to_function_has_parentheses check(R"( local function foo() return 1 end local function bar(a: number) return -a end -local abc = bar(@1) +local abc = bar(@1) )"); auto ac = autocomplete('1'); diff --git a/tests/Compiler.test.cpp b/tests/Compiler.test.cpp index 4b87c003..0a1c5a8b 100644 --- a/tests/Compiler.test.cpp +++ b/tests/Compiler.test.cpp @@ -2382,11 +2382,13 @@ end TEST_CASE("DebugLineInfoRepeatUntil") { + ScopedFastFlag sff("LuauCompileXEQ", true); + CHECK_EQ("\n" + compileFunction0Coverage(R"( local f = 0 repeat f += 1 - if f == 1 then + if f == 1 then print(f) else f = 0 @@ -2397,13 +2399,13 @@ until f == 0 R"( 2: LOADN R0 0 4: L0: ADDK R0 R0 K0 -5: JUMPIFNOTEQK R0 K0 L1 +5: JUMPXEQKN R0 K0 L1 NOT 6: GETIMPORT R1 2 6: MOVE R2 R0 6: CALL R1 1 0 6: JUMP L2 8: L1: LOADN R0 0 -10: L2: JUMPIFEQK R0 K3 L3 +10: L2: JUMPXEQKN R0 K3 L3 10: JUMPBACK L0 11: L3: RETURN R0 0 )"); @@ -3509,13 +3511,15 @@ RETURN R0 1 TEST_CASE("ConstantJumpCompare") { + ScopedFastFlag sff("LuauCompileXEQ", true); + CHECK_EQ("\n" + compileFunction0(R"( local obj = ... local b = obj == 1 )"), R"( GETVARARGS R0 1 -JUMPIFEQK R0 K0 L0 +JUMPXEQKN R0 K0 L0 LOADB R1 0 +1 L0: LOADB R1 1 L1: RETURN R0 0 @@ -3527,7 +3531,7 @@ local b = 1 == obj )"), R"( GETVARARGS R0 1 -JUMPIFEQK R0 K0 L0 +JUMPXEQKN R0 K0 L0 LOADB R1 0 +1 L0: LOADB R1 1 L1: RETURN R0 0 @@ -3539,7 +3543,7 @@ local b = "Hello, Sailor!" == obj )"), R"( GETVARARGS R0 1 -JUMPIFEQK R0 K0 L0 +JUMPXEQKS R0 K0 L0 LOADB R1 0 +1 L0: LOADB R1 1 L1: RETURN R0 0 @@ -3551,7 +3555,7 @@ local b = nil == obj )"), R"( GETVARARGS R0 1 -JUMPIFEQK R0 K0 L0 +JUMPXEQKNIL R0 L0 LOADB R1 0 +1 L0: LOADB R1 1 L1: RETURN R0 0 @@ -3563,7 +3567,7 @@ local b = true == obj )"), R"( GETVARARGS R0 1 -JUMPIFEQK R0 K0 L0 +JUMPXEQKB R0 1 L0 LOADB R1 0 +1 L0: LOADB R1 1 L1: RETURN R0 0 @@ -3575,7 +3579,7 @@ local b = nil ~= obj )"), R"( GETVARARGS R0 1 -JUMPIFNOTEQK R0 K0 L0 +JUMPXEQKNIL R0 L0 NOT LOADB R1 0 +1 L0: LOADB R1 1 L1: RETURN R0 0 @@ -6098,6 +6102,8 @@ L3: RETURN R0 -1 TEST_CASE("BuiltinFoldingMultret") { + ScopedFastFlag sff("LuauCompileXEQ", true); + CHECK_EQ("\n" + compileFunction(R"( local NoLanes: Lanes = --[[ ]] 0b0000000000000000000000000000000 local OffscreenLane: Lane = --[[ ]] 0b1000000000000000000000000000000 @@ -6120,14 +6126,14 @@ FASTCALL2K 29 R2 K1 L0 LOADK R3 K1 GETIMPORT R1 4 CALL R1 2 1 -L0: JUMPIFEQK R1 K5 L1 +L0: JUMPXEQKN R1 K5 L1 RETURN R1 1 L1: FASTCALL2K 29 R1 K6 L2 MOVE R3 R1 LOADK R4 K6 GETIMPORT R2 4 CALL R2 2 1 -L2: JUMPIFEQK R2 K5 L3 +L2: JUMPXEQKN R2 K5 L3 LOADK R2 K6 RETURN R2 1 L3: LOADN R2 0 @@ -6155,7 +6161,8 @@ local function test(a, b) local c = a return c + b end -)"), R"( +)"), + R"( ADD R2 R0 R1 RETURN R2 1 )"); @@ -6166,7 +6173,8 @@ local function test(a, b) local c = (a :: number) return c + b end -)"), R"( +)"), + R"( ADD R2 R0 R1 RETURN R2 1 )"); @@ -6180,7 +6188,8 @@ local function test(a, b) b += 0 return c + d end -)"), R"( +)"), + R"( MOVE R2 R0 ADDK R2 R2 K0 MOVE R3 R1 @@ -6196,7 +6205,8 @@ local function test(a, b) local d = b return c + d end -)"), R"( +)"), + R"( ADD R2 R0 R1 RETURN R2 1 )"); @@ -6207,7 +6217,8 @@ local function test(a, b) local c, d = a, b return c + d end -)"), R"( +)"), + R"( MOVE R2 R0 MOVE R3 R1 ADD R4 R2 R3 @@ -6221,7 +6232,9 @@ local function test(a, b) local d = b return function() return c + d end end -)", 1), R"( +)", + 1), + R"( NEWCLOSURE R2 P0 CAPTURE VAL R0 CAPTURE VAL R1 diff --git a/tests/Conformance.test.cpp b/tests/Conformance.test.cpp index 1339fa0c..e07ba12a 100644 --- a/tests/Conformance.test.cpp +++ b/tests/Conformance.test.cpp @@ -70,8 +70,8 @@ static int lua_loadstring(lua_State* L) return 1; lua_pushnil(L); - lua_insert(L, -2); /* put before error message */ - return 2; /* return nil plus error message */ + lua_insert(L, -2); // put before error message + return 2; // return nil plus error message } static int lua_vector(lua_State* L) @@ -244,8 +244,6 @@ TEST_CASE("Assert") TEST_CASE("Basic") { - ScopedFastFlag sff("LuauLenTM", true); - runConformance("basic.lua"); } @@ -291,6 +289,8 @@ TEST_CASE("Clear") TEST_CASE("Strings") { + ScopedFastFlag sff{"LuauTostringFormatSpecifier", true}; + runConformance("strings.lua"); } @@ -311,13 +311,14 @@ TEST_CASE("Literals") TEST_CASE("Errors") { + ScopedFastFlag sff("LuauNicerMethodErrors", true); + runConformance("errors.lua"); } TEST_CASE("Events") { - ScopedFastFlag sff1("LuauLenTM", true); - ScopedFastFlag sff2("LuauBetterNewindex", true); + ScopedFastFlag sff("LuauBetterNewindex", true); runConformance("events.lua"); } diff --git a/tests/Fixture.cpp b/tests/Fixture.cpp index 90aa6236..f51a9d1b 100644 --- a/tests/Fixture.cpp +++ b/tests/Fixture.cpp @@ -410,7 +410,7 @@ void Fixture::validateErrors(const std::vector& errors) LoadDefinitionFileResult Fixture::loadDefinition(const std::string& source) { unfreeze(typeChecker.globalTypes); - LoadDefinitionFileResult result = loadDefinitionFile(typeChecker, typeChecker.globalScope, source, "@test"); + LoadDefinitionFileResult result = frontend.loadDefinitionFile(source, "@test"); freeze(typeChecker.globalTypes); REQUIRE_MESSAGE(result.success, "loadDefinition: unable to load definition file"); diff --git a/tests/Frontend.test.cpp b/tests/Frontend.test.cpp index 0382f227..3c27ad54 100644 --- a/tests/Frontend.test.cpp +++ b/tests/Frontend.test.cpp @@ -1030,7 +1030,6 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "reexport_cyclic_type") ScopedFastFlag sff[] = { {"LuauForceExportSurfacesToBeNormal", true}, {"LuauLowerBoundsCalculation", true}, - {"LuauNormalizeFlagIsConservative", true}, }; fileResolver.source["Module/A"] = R"( @@ -1067,7 +1066,6 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "reexport_type_alias") ScopedFastFlag sff[] = { {"LuauForceExportSurfacesToBeNormal", true}, {"LuauLowerBoundsCalculation", true}, - {"LuauNormalizeFlagIsConservative", true}, }; fileResolver.source["Module/A"] = R"( diff --git a/tests/JsonEmitter.test.cpp b/tests/JsonEmitter.test.cpp new file mode 100644 index 00000000..ebe83209 --- /dev/null +++ b/tests/JsonEmitter.test.cpp @@ -0,0 +1,195 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +#include "Luau/JsonEmitter.h" + +#include "doctest.h" + +using namespace Luau::Json; + +TEST_SUITE_BEGIN("JsonEmitter"); + +TEST_CASE("write_array") +{ + JsonEmitter emitter; + ArrayEmitter a = emitter.writeArray(); + a.writeValue(123); + a.writeValue("foo"); + a.finish(); + + std::string result = emitter.str(); + CHECK(result == "[123,\"foo\"]"); +} + +TEST_CASE("write_object") +{ + JsonEmitter emitter; + ObjectEmitter o = emitter.writeObject(); + o.writePair("foo", "bar"); + o.writePair("bar", "baz"); + o.finish(); + + std::string result = emitter.str(); + CHECK(result == "{\"foo\":\"bar\",\"bar\":\"baz\"}"); +} + +TEST_CASE("write_bool") +{ + JsonEmitter emitter; + write(emitter, false); + CHECK(emitter.str() == "false"); + + emitter = JsonEmitter{}; + write(emitter, true); + CHECK(emitter.str() == "true"); +} + +TEST_CASE("write_null") +{ + JsonEmitter emitter; + write(emitter, nullptr); + CHECK(emitter.str() == "null"); +} + +TEST_CASE("write_string") +{ + JsonEmitter emitter; + write(emitter, R"(foo,bar,baz, +"this should be escaped")"); + CHECK(emitter.str() == "\"foo,bar,baz,\\n\\\"this should be escaped\\\"\""); +} + +TEST_CASE("write_comma") +{ + JsonEmitter emitter; + emitter.writeComma(); + write(emitter, true); + emitter.writeComma(); + write(emitter, false); + CHECK(emitter.str() == "true,false"); +} + +TEST_CASE("push_and_pop_comma") +{ + JsonEmitter emitter; + emitter.writeComma(); + write(emitter, true); + emitter.writeComma(); + emitter.writeRaw('['); + bool comma = emitter.pushComma(); + emitter.writeComma(); + write(emitter, true); + emitter.writeComma(); + write(emitter, false); + emitter.writeRaw(']'); + emitter.popComma(comma); + emitter.writeComma(); + write(emitter, false); + + CHECK(emitter.str() == "true,[true,false],false"); +} + +TEST_CASE("write_optional") +{ + JsonEmitter emitter; + emitter.writeComma(); + write(emitter, std::optional{true}); + emitter.writeComma(); + write(emitter, std::nullopt); + + CHECK(emitter.str() == "true,null"); +} + +TEST_CASE("write_vector") +{ + std::vector values{1, 2, 3, 4}; + JsonEmitter emitter; + write(emitter, values); + CHECK(emitter.str() == "[1,2,3,4]"); +} + +TEST_CASE("prevent_multiple_object_finish") +{ + JsonEmitter emitter; + ObjectEmitter o = emitter.writeObject(); + o.writePair("a", "b"); + o.finish(); + o.finish(); + + CHECK(emitter.str() == "{\"a\":\"b\"}"); +} + +TEST_CASE("prevent_multiple_array_finish") +{ + JsonEmitter emitter; + ArrayEmitter a = emitter.writeArray(); + a.writeValue(1); + a.finish(); + a.finish(); + + CHECK(emitter.str() == "[1]"); +} + +TEST_CASE("cannot_write_pair_after_finished") +{ + JsonEmitter emitter; + ObjectEmitter o = emitter.writeObject(); + o.finish(); + o.writePair("a", "b"); + + CHECK(emitter.str() == "{}"); +} + +TEST_CASE("cannot_write_value_after_finished") +{ + JsonEmitter emitter; + ArrayEmitter a = emitter.writeArray(); + a.finish(); + a.writeValue(1); + + CHECK(emitter.str() == "[]"); +} + +TEST_CASE("finish_when_destructing_object") +{ + JsonEmitter emitter; + emitter.writeObject(); + + CHECK(emitter.str() == "{}"); +} + +TEST_CASE("finish_when_destructing_array") +{ + JsonEmitter emitter; + emitter.writeArray(); + + CHECK(emitter.str() == "[]"); +} + +namespace Luau::Json +{ + +struct Special +{ + int foo; + int bar; +}; + +void write(JsonEmitter& emitter, const Special& value) +{ + ObjectEmitter o = emitter.writeObject(); + o.writePair("foo", value.foo); + o.writePair("bar", value.bar); +} + +} // namespace Luau::Json + +TEST_CASE("afford_extensibility") +{ + std::vector vec{Special{1, 2}, Special{3, 4}}; + JsonEmitter e; + write(e, vec); + + std::string result = e.str(); + CHECK(result == R"([{"foo":1,"bar":2},{"foo":3,"bar":4}])"); +} + +TEST_SUITE_END(); diff --git a/tests/Linter.test.cpp b/tests/Linter.test.cpp index 488ade9f..35c40508 100644 --- a/tests/Linter.test.cpp +++ b/tests/Linter.test.cpp @@ -1675,4 +1675,36 @@ TEST_CASE_FIXTURE(Fixture, "WrongCommentOptimize") CHECK_EQ(result.warnings[3].text, "optimize directive uses unknown optimization level '100500', 0..2 expected"); } +TEST_CASE_FIXTURE(Fixture, "LintIntegerParsing") +{ + ScopedFastFlag luauLintParseIntegerIssues{"LuauLintParseIntegerIssues", true}; + + LintResult result = lint(R"( +local _ = 0b10000000000000000000000000000000000000000000000000000000000000000 +local _ = 0x10000000000000000 +)"); + + REQUIRE_EQ(result.warnings.size(), 2); + CHECK_EQ(result.warnings[0].text, "Binary number literal exceeded available precision and has been truncated to 2^64"); + CHECK_EQ(result.warnings[1].text, "Hexadecimal number literal exceeded available precision and has been truncated to 2^64"); +} + +// TODO: remove with FFlagLuauErrorDoubleHexPrefix +TEST_CASE_FIXTURE(Fixture, "LintIntegerParsingDoublePrefix") +{ + ScopedFastFlag luauLintParseIntegerIssues{"LuauLintParseIntegerIssues", true}; + ScopedFastFlag luauErrorDoubleHexPrefix{"LuauErrorDoubleHexPrefix", false}; // Lint will be available until we start rejecting code + + LintResult result = lint(R"( +local _ = 0x0x123 +local _ = 0x0xffffffffffffffffffffffffffffffffff +)"); + + REQUIRE_EQ(result.warnings.size(), 2); + CHECK_EQ(result.warnings[0].text, + "Hexadecimal number literal has a double prefix, which will fail to parse in the future; remove the extra 0x to fix"); + CHECK_EQ(result.warnings[1].text, + "Hexadecimal number literal has a double prefix, which will fail to parse in the future; remove the extra 0x to fix"); +} + TEST_SUITE_END(); diff --git a/tests/Normalize.test.cpp b/tests/Normalize.test.cpp index 84a5a380..c64c41c5 100644 --- a/tests/Normalize.test.cpp +++ b/tests/Normalize.test.cpp @@ -680,26 +680,12 @@ TEST_CASE_FIXTURE(Fixture, "higher_order_function_with_annotation") CHECK_EQ("((a) -> b, a) -> b", toString(requireType("apply"))); } -TEST_CASE_FIXTURE(Fixture, "cyclic_table_is_marked_normal") -{ - ScopedFastFlag flags[] = {{"LuauLowerBoundsCalculation", true}, {"LuauNormalizeFlagIsConservative", false}}; - - check(R"( - type Fiber = { - return_: Fiber? - } - - local f: Fiber - )"); - - TypeId t = requireType("f"); - CHECK(t->normal); -} - // Unfortunately, getting this right in the general case is difficult. TEST_CASE_FIXTURE(Fixture, "cyclic_table_is_not_marked_normal") { - ScopedFastFlag flags[] = {{"LuauLowerBoundsCalculation", true}, {"LuauNormalizeFlagIsConservative", true}}; + ScopedFastFlag flags[] = { + {"LuauLowerBoundsCalculation", true}, + }; check(R"( type Fiber = { @@ -1081,7 +1067,6 @@ TEST_CASE_FIXTURE(Fixture, "bound_typevars_should_only_be_marked_normal_if_their { ScopedFastFlag sff[]{ {"LuauLowerBoundsCalculation", true}, - {"LuauNormalizeFlagIsConservative", true}, }; CheckResult result = check(R"( diff --git a/tests/Parser.test.cpp b/tests/Parser.test.cpp index c5c019aa..5b86807c 100644 --- a/tests/Parser.test.cpp +++ b/tests/Parser.test.cpp @@ -682,20 +682,23 @@ TEST_CASE_FIXTURE(Fixture, "parse_numbers_binary") TEST_CASE_FIXTURE(Fixture, "parse_numbers_error") { - ScopedFastFlag luauErrorParseIntegerIssues{"LuauErrorParseIntegerIssues", true}; + ScopedFastFlag luauLintParseIntegerIssues{"LuauLintParseIntegerIssues", true}; + ScopedFastFlag luauErrorDoubleHexPrefix{"LuauErrorDoubleHexPrefix", true}; CHECK_EQ(getParseError("return 0b123"), "Malformed number"); CHECK_EQ(getParseError("return 123x"), "Malformed number"); CHECK_EQ(getParseError("return 0xg"), "Malformed number"); CHECK_EQ(getParseError("return 0x0x123"), "Malformed number"); + CHECK_EQ(getParseError("return 0xffffffffffffffffffffllllllg"), "Malformed number"); + CHECK_EQ(getParseError("return 0x0xffffffffffffffffffffffffffff"), "Malformed number"); } -TEST_CASE_FIXTURE(Fixture, "parse_numbers_range_error") +TEST_CASE_FIXTURE(Fixture, "parse_numbers_error_soft") { - ScopedFastFlag luauErrorParseIntegerIssues{"LuauErrorParseIntegerIssues", true}; + ScopedFastFlag luauLintParseIntegerIssues{"LuauLintParseIntegerIssues", true}; + ScopedFastFlag luauErrorDoubleHexPrefix{"LuauErrorDoubleHexPrefix", false}; - CHECK_EQ(getParseError("return 0x10000000000000000"), "Integer number value is out of range"); - CHECK_EQ(getParseError("return 0b10000000000000000000000000000000000000000000000000000000000000000"), "Integer number value is out of range"); + CHECK_EQ(getParseError("return 0x0x0x0x0x0x0x0"), "Malformed number"); } TEST_CASE_FIXTURE(Fixture, "break_return_not_last_error") diff --git a/tests/Repl.test.cpp b/tests/Repl.test.cpp index 87a1e1e2..18c243b0 100644 --- a/tests/Repl.test.cpp +++ b/tests/Repl.test.cpp @@ -82,7 +82,7 @@ private: capturedoutput = "" function arraytostring(arr) - local strings = {} + local strings = {} table.foreachi(arr, function(k,v) table.insert(strings, pptostring(v)) end ) return "{" .. table.concat(strings, ", ") .. "}" end diff --git a/tests/ToString.test.cpp b/tests/ToString.test.cpp index 8c03fe55..fe376d86 100644 --- a/tests/ToString.test.cpp +++ b/tests/ToString.test.cpp @@ -305,7 +305,6 @@ TEST_CASE_FIXTURE(Fixture, "stringifying_type_is_still_capped_when_exhaustive") CHECK_EQ(toString(requireType("f2"), o), "((() -> ()) -> () -> ()) -> (() -> ()) -> ... "); CHECK_EQ(toString(requireType("f3"), o), "(((() -> ()) -> () -> ()) -> (() -> ()) -> ... "); } - } TEST_CASE_FIXTURE(Fixture, "stringifying_table_type_correctly_use_matching_table_state_braces") @@ -726,10 +725,6 @@ TEST_CASE_FIXTURE(Fixture, "toStringNamedFunction_overrides_param_names") TEST_CASE_FIXTURE(Fixture, "pick_distinct_names_for_mixed_explicit_and_implicit_generics") { - ScopedFastFlag sff[] = { - {"LuauAlwaysQuantify", true}, - }; - CheckResult result = check(R"( function foo(x: a, y) end )"); diff --git a/tests/TypeInfer.aliases.test.cpp b/tests/TypeInfer.aliases.test.cpp index bdd4d6fd..e487fd48 100644 --- a/tests/TypeInfer.aliases.test.cpp +++ b/tests/TypeInfer.aliases.test.cpp @@ -94,6 +94,65 @@ TEST_CASE_FIXTURE(Fixture, "cannot_steal_hoisted_type_alias") } } +TEST_CASE_FIXTURE(Fixture, "mismatched_generic_type_param") +{ + CheckResult result = check(R"( + type T = (A...) -> () + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK(toString(result.errors[0]) == + "Generic type 'A' is used as a variadic type parameter; consider changing 'A' to 'A...' in the generic argument list"); + CHECK(result.errors[0].location == Location{{1, 21}, {1, 25}}); +} + +TEST_CASE_FIXTURE(Fixture, "mismatched_generic_pack_type_param") +{ + CheckResult result = check(R"( + type T = (A) -> () + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK(toString(result.errors[0]) == + "Variadic type parameter 'A...' is used as a regular generic type; consider changing 'A...' to 'A' in the generic argument list"); + CHECK(result.errors[0].location == Location{{1, 24}, {1, 25}}); +} + +TEST_CASE_FIXTURE(Fixture, "default_type_parameter") +{ + CheckResult result = check(R"( + type T = { a: A, b: B } + local x: T = { a = "foo", b = "bar" } + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + CHECK(toString(requireType("x")) == "T"); +} + +TEST_CASE_FIXTURE(Fixture, "default_pack_parameter") +{ + CheckResult result = check(R"( + type T = { fn: (A...) -> () } + local x: T + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + CHECK(toString(requireType("x")) == "T"); +} + +TEST_CASE_FIXTURE(Fixture, "saturate_to_first_type_pack") +{ + CheckResult result = check(R"( + type T = { fn: (A, B) -> C... } + local x: T + local f = x.fn + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + CHECK(toString(requireType("x")) == "T"); + CHECK(toString(requireType("f")) == "(string, number) -> (string, boolean)"); +} + TEST_CASE_FIXTURE(Fixture, "cyclic_types_of_named_table_fields_do_not_expand_when_stringified") { CheckResult result = check(R"( @@ -126,6 +185,40 @@ TEST_CASE_FIXTURE(Fixture, "mutually_recursive_aliases") LUAU_REQUIRE_NO_ERRORS(result); } +TEST_CASE_FIXTURE(Fixture, "generic_aliases") +{ + ScopedFastFlag sff_DebugLuauDeferredConstraintResolution{"DebugLuauDeferredConstraintResolution", true}; + + CheckResult result = check(R"( + type T = { v: a } + local x: T = { v = 123 } + local y: T = { v = "foo" } + local bad: T = { v = "foo" } + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + CHECK(result.errors[0].location == Location{{4, 31}, {4, 44}}); + CHECK(toString(result.errors[0]) == "Type '{ v: string }' could not be converted into 'T'"); +} + +TEST_CASE_FIXTURE(Fixture, "dependent_generic_aliases") +{ + ScopedFastFlag sff_DebugLuauDeferredConstraintResolution{"DebugLuauDeferredConstraintResolution", true}; + + CheckResult result = check(R"( + type T = { v: a } + type U = { t: T } + local x: U = { t = { v = 123 } } + local bad: U = { t = { v = "foo" } } + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + CHECK(result.errors[0].location == Location{{4, 31}, {4, 52}}); + CHECK(toString(result.errors[0]) == "Type '{ t: { v: string } }' could not be converted into 'U'"); +} + TEST_CASE_FIXTURE(Fixture, "mutually_recursive_generic_aliases") { CheckResult result = check(R"( @@ -360,6 +453,7 @@ TEST_CASE_FIXTURE(Fixture, "stringify_optional_parameterized_alias") LUAU_REQUIRE_ERROR_COUNT(1, result); auto e = get(result.errors[0]); + REQUIRE(e != nullptr); CHECK_EQ("Node?", toString(e->givenType)); CHECK_EQ("Node", toString(e->wantedType)); } diff --git a/tests/TypeInfer.builtins.test.cpp b/tests/TypeInfer.builtins.test.cpp index 0c878023..10da0efa 100644 --- a/tests/TypeInfer.builtins.test.cpp +++ b/tests/TypeInfer.builtins.test.cpp @@ -556,6 +556,29 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "string_format_correctly_ordered_types") CHECK_EQ(tm->givenType, typeChecker.numberType); } +TEST_CASE_FIXTURE(BuiltinsFixture, "string_format_tostring_specifier") +{ + CheckResult result = check(R"( + --!strict + string.format("%* %* %* %*", "string", 1, true, function() end) + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(BuiltinsFixture, "string_format_tostring_specifier_type_constraint") +{ + CheckResult result = check(R"( + local function f(x): string + local _ = string.format("%*", x) + return x + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + CHECK_EQ("(string) -> string", toString(requireType("f"))); +} + TEST_CASE_FIXTURE(BuiltinsFixture, "xpcall") { CheckResult result = check(R"( diff --git a/tests/TypeInfer.functions.test.cpp b/tests/TypeInfer.functions.test.cpp index 70773d95..074c86c3 100644 --- a/tests/TypeInfer.functions.test.cpp +++ b/tests/TypeInfer.functions.test.cpp @@ -1547,7 +1547,6 @@ caused by: Argument count mismatch. Function expects 1 argument, but none are specified)", toString(result.errors[0])); } - } TEST_CASE_FIXTURE(Fixture, "too_few_arguments_variadic") diff --git a/tests/TypeInfer.generics.test.cpp b/tests/TypeInfer.generics.test.cpp index 7a1af164..a8325727 100644 --- a/tests/TypeInfer.generics.test.cpp +++ b/tests/TypeInfer.generics.test.cpp @@ -1186,10 +1186,6 @@ end) TEST_CASE_FIXTURE(Fixture, "quantify_functions_even_if_they_have_an_explicit_generic") { - ScopedFastFlag sff[] = { - {"LuauAlwaysQuantify", true}, - }; - CheckResult result = check(R"( function foo(f, x: X) return f(x) diff --git a/tests/TypeInfer.provisional.test.cpp b/tests/TypeInfer.provisional.test.cpp index 34afd560..01923f38 100644 --- a/tests/TypeInfer.provisional.test.cpp +++ b/tests/TypeInfer.provisional.test.cpp @@ -485,7 +485,6 @@ TEST_CASE_FIXTURE(Fixture, "constrained_is_level_dependent") { ScopedFastFlag sff[]{ {"LuauLowerBoundsCalculation", true}, - {"LuauNormalizeFlagIsConservative", true}, {"LuauQuantifyConstrained", true}, }; diff --git a/tests/TypeInfer.singletons.test.cpp b/tests/TypeInfer.singletons.test.cpp index 5da4a340..0a130d49 100644 --- a/tests/TypeInfer.singletons.test.cpp +++ b/tests/TypeInfer.singletons.test.cpp @@ -262,7 +262,7 @@ TEST_CASE_FIXTURE(Fixture, "table_properties_type_error_escapes") { CheckResult result = check(R"( --!strict - local x: { ["<>"] : number } + local x: { ["<>"] : number } x = { ["\n"] = 5 } )"); diff --git a/tests/TypeInfer.test.cpp b/tests/TypeInfer.test.cpp index af6185ef..80889363 100644 --- a/tests/TypeInfer.test.cpp +++ b/tests/TypeInfer.test.cpp @@ -252,7 +252,6 @@ TEST_CASE_FIXTURE(Fixture, "type_errors_infer_types") CHECK_EQ("", toString(requireType("e"))); CHECK_EQ("", toString(requireType("f"))); } - } } diff --git a/tests/TypeInfer.typePacks.cpp b/tests/TypeInfer.typePacks.cpp index bcd30498..7aefa00d 100644 --- a/tests/TypeInfer.typePacks.cpp +++ b/tests/TypeInfer.typePacks.cpp @@ -203,7 +203,7 @@ TEST_CASE_FIXTURE(Fixture, "variadic_packs") ), "@test" ); - addGlobalBinding(typeChecker, "bar", + addGlobalBinding(typeChecker, "bar", arena.addType( FunctionTypeVar{ arena.addTypePack({{typeChecker.numberType}, listOfStrings}), diff --git a/tests/conformance/errors.lua b/tests/conformance/errors.lua index 6c0ac5e6..7ab7099e 100644 --- a/tests/conformance/errors.lua +++ b/tests/conformance/errors.lua @@ -388,4 +388,11 @@ assert(ecall(function() for i='a',2 do end end) == "invalid 'for' initial value assert(ecall(function() for i=1,'a' do end end) == "invalid 'for' limit (number expected, got string)") assert(ecall(function() for i=1,2,'a' do end end) == "invalid 'for' step (number expected, got string)") +-- method call errors +assert(ecall(function() ({}):foo() end) == "attempt to call missing method 'foo' of table") +assert(ecall(function() (""):foo() end) == "attempt to call missing method 'foo' of string") +assert(ecall(function() (42):foo() end) == "attempt to index number with 'foo'") +assert(ecall(function() ({foo=42}):foo() end) == "attempt to call a number value") +assert(ecall(function() local ud = newproxy(true) getmetatable(ud).__index = {} ud:foo() end) == "attempt to call missing method 'foo' of userdata") + return('OK') diff --git a/tests/conformance/strings.lua b/tests/conformance/strings.lua index 98a5721a..c87cf15c 100644 --- a/tests/conformance/strings.lua +++ b/tests/conformance/strings.lua @@ -130,6 +130,26 @@ assert(string.format('"-%20s.20s"', string.rep("%", 2000)) == -- longest number that can be formated assert(string.len(string.format('%99.99f', -1e308)) >= 100) +local function return_one_thing() + return "hi" +end +local function return_two_nils() + return nil, nil +end + +assert(string.format("%*", return_one_thing()) == "hi") +assert(string.format("%* %*", return_two_nils()) == "nil nil") +assert(pcall(function() + string.format("%* %* %*", return_two_nils()) +end) == false) + +assert(string.format("%*", "a\0b\0c") == "a\0b\0c") +assert(string.format("%*", string.rep("doge", 3000)) == string.rep("doge", 3000)) + +assert(pcall(function() + string.format("%#*", "bad form") +end) == false) + assert(loadstring("return 1\n--comentrio sem EOL no final")() == 1) diff --git a/tests/main.cpp b/tests/main.cpp index 40ccd0b3..3e480c9f 100644 --- a/tests/main.cpp +++ b/tests/main.cpp @@ -61,7 +61,7 @@ static bool debuggerPresent() static int testAssertionHandler(const char* expr, const char* file, int line, const char* function) { if (debuggerPresent()) - LUAU_DEBUGBREAK(); + LUAU_DEBUGBREAK(); ADD_FAIL_AT(file, line, "Assertion failed: ", std::string(expr)); return 1; diff --git a/tools/faillist.txt b/tools/faillist.txt index 40fc3c06..6e93345b 100644 --- a/tools/faillist.txt +++ b/tools/faillist.txt @@ -4,14 +4,11 @@ AnnotationTests.as_expr_warns_on_unrelated_cast AnnotationTests.builtin_types_are_not_exported AnnotationTests.cannot_use_nonexported_type AnnotationTests.cloned_interface_maintains_pointers_between_definitions -AnnotationTests.corecursive_types_error_on_tight_loop AnnotationTests.define_generic_type_alias AnnotationTests.duplicate_type_param_name AnnotationTests.for_loop_counter_annotation_is_checked AnnotationTests.function_return_annotations_are_checked AnnotationTests.generic_aliases_are_cloned_properly -AnnotationTests.instantiate_type_fun_should_not_trip_rbxassert -AnnotationTests.instantiation_clone_has_to_follow AnnotationTests.interface_types_belong_to_interface_arena AnnotationTests.luau_ice_triggers_an_ice AnnotationTests.luau_ice_triggers_an_ice_exception_with_flag @@ -24,14 +21,9 @@ AnnotationTests.occurs_check_on_cyclic_union_typevar AnnotationTests.self_referential_type_alias AnnotationTests.too_many_type_params AnnotationTests.two_type_params -AnnotationTests.type_alias_always_resolve_to_a_real_type -AnnotationTests.type_alias_B_should_check_with_another_aliases_until_a_non_aliased_type -AnnotationTests.type_alias_should_alias_to_number -AnnotationTests.type_aliasing_to_number_should_not_check_given_a_string AnnotationTests.type_annotations_inside_function_bodies AnnotationTests.type_assertion_expr -AnnotationTests.typeof_variable_type_annotation_should_return_its_type -AnnotationTests.use_generic_type_alias +AnnotationTests.unknown_type_reference_generates_error AnnotationTests.use_type_required_from_another_file AstQuery.last_argument_function_call_type AstQuery::getDocumentationSymbolAtPosition.binding @@ -42,11 +34,8 @@ AutocompleteTest.argument_types AutocompleteTest.arguments_to_global_lambda AutocompleteTest.as_types AutocompleteTest.autocomplete_boolean_singleton -AutocompleteTest.autocomplete_default_type_pack_parameters -AutocompleteTest.autocomplete_default_type_parameters AutocompleteTest.autocomplete_end_with_fn_exprs AutocompleteTest.autocomplete_end_with_lambda -AutocompleteTest.autocomplete_explicit_type_pack AutocompleteTest.autocomplete_first_function_arg_expected_type AutocompleteTest.autocomplete_for_in_middle_keywords AutocompleteTest.autocomplete_for_middle_keywords @@ -93,7 +82,6 @@ AutocompleteTest.local_function_params AutocompleteTest.local_functions_fall_out_of_scope AutocompleteTest.method_call_inside_function_body AutocompleteTest.module_type_members -AutocompleteTest.modules_with_types AutocompleteTest.nested_member_completions AutocompleteTest.nested_recursive_function AutocompleteTest.no_function_name_suggestions @@ -101,8 +89,6 @@ AutocompleteTest.no_incompatible_self_calls AutocompleteTest.no_incompatible_self_calls_2 AutocompleteTest.no_incompatible_self_calls_on_class AutocompleteTest.no_wrong_compatible_self_calls_with_generics -AutocompleteTest.optional_members -AutocompleteTest.private_types AutocompleteTest.recursive_function AutocompleteTest.recursive_function_global AutocompleteTest.recursive_function_local @@ -114,7 +100,6 @@ AutocompleteTest.stop_at_first_stat_when_recommending_keywords AutocompleteTest.string_prim_non_self_calls_are_avoided AutocompleteTest.string_prim_self_calls_are_fine AutocompleteTest.suggest_external_module_type -AutocompleteTest.suggest_table_keys AutocompleteTest.table_intersection AutocompleteTest.table_union AutocompleteTest.type_correct_argument_type_suggestion @@ -133,8 +118,6 @@ AutocompleteTest.type_correct_local_type_suggestion AutocompleteTest.type_correct_sealed_table AutocompleteTest.type_correct_suggestion_for_overloads AutocompleteTest.type_correct_suggestion_in_argument -AutocompleteTest.type_correct_suggestion_in_table -AutocompleteTest.type_scoping_easy AutocompleteTest.unsealed_table AutocompleteTest.unsealed_table_2 AutocompleteTest.user_defined_local_functions_in_own_definition @@ -259,7 +242,6 @@ GenericsTests.check_mutual_generic_functions GenericsTests.correctly_instantiate_polymorphic_member_functions GenericsTests.do_not_always_instantiate_generic_intersection_types GenericsTests.do_not_infer_generic_functions -GenericsTests.dont_substitute_bound_types GenericsTests.dont_unify_bound_types GenericsTests.duplicate_generic_type_packs GenericsTests.duplicate_generic_types @@ -275,6 +257,7 @@ GenericsTests.generic_functions_dont_cache_type_parameters GenericsTests.generic_functions_in_types GenericsTests.generic_functions_should_be_memory_safe GenericsTests.generic_table_method +GenericsTests.generic_type_pack_parentheses GenericsTests.generic_type_pack_syntax GenericsTests.generic_type_pack_unification1 GenericsTests.generic_type_pack_unification2 @@ -297,15 +280,12 @@ GenericsTests.properties_can_be_polytypes GenericsTests.rank_N_types_via_typeof GenericsTests.reject_clashing_generic_and_pack_names GenericsTests.self_recursive_instantiated_param -GenericsTests.substitution_with_bound_table -GenericsTests.typefuns_sharing_types GenericsTests.variadic_generics IntersectionTypes.argument_is_intersection IntersectionTypes.error_detailed_intersection_all IntersectionTypes.error_detailed_intersection_part IntersectionTypes.fx_intersection_as_argument IntersectionTypes.fx_union_as_argument_fails -IntersectionTypes.index_on_an_intersection_type_with_all_parts_missing_the_property IntersectionTypes.index_on_an_intersection_type_with_mixed_types IntersectionTypes.index_on_an_intersection_type_with_one_part_missing_the_property IntersectionTypes.index_on_an_intersection_type_with_one_property_of_type_any @@ -313,12 +293,8 @@ IntersectionTypes.index_on_an_intersection_type_with_property_guaranteed_to_exis IntersectionTypes.index_on_an_intersection_type_works_at_arbitrary_depth IntersectionTypes.no_stack_overflow_from_flattenintersection IntersectionTypes.overload_is_not_a_function -IntersectionTypes.propagates_name IntersectionTypes.select_correct_union_fn IntersectionTypes.should_still_pick_an_overload_whose_arguments_are_unions -IntersectionTypes.table_combines -IntersectionTypes.table_combines_missing -IntersectionTypes.table_extra_ok IntersectionTypes.table_intersection_setmetatable IntersectionTypes.table_intersection_write IntersectionTypes.table_intersection_write_sealed @@ -332,7 +308,6 @@ isSubtype.table_with_table_prop isSubtype.tables Linter.DeprecatedApi Linter.TableOperations -ModuleTests.any_persistance_does_not_leak ModuleTests.builtin_types_point_into_globalTypes_arena ModuleTests.clone_self_property ModuleTests.deepClone_cyclic_table @@ -355,8 +330,6 @@ NonstrictModeTests.table_props_are_any Normalize.any_wins_the_battle_over_unknown_in_unions Normalize.constrained_intersection_of_intersections Normalize.cyclic_intersection -Normalize.cyclic_table_is_marked_normal -Normalize.cyclic_table_is_not_marked_normal Normalize.cyclic_table_normalizes_sensibly Normalize.cyclic_union Normalize.fuzz_failure_bound_type_is_normal_but_not_its_bounded_to @@ -490,16 +463,9 @@ TableTests.cannot_change_type_of_unsealed_table_prop TableTests.casting_sealed_tables_with_props_into_table_with_indexer TableTests.casting_tables_with_props_into_table_with_indexer3 TableTests.casting_tables_with_props_into_table_with_indexer4 -TableTests.casting_unsealed_tables_with_props_into_table_with_indexer TableTests.checked_prop_too_early -TableTests.common_table_element_general -TableTests.common_table_element_inner_index -TableTests.common_table_element_inner_prop -TableTests.common_table_element_list -TableTests.common_table_element_union_assignment TableTests.common_table_element_union_in_call TableTests.common_table_element_union_in_call_tail -TableTests.common_table_element_union_in_prop TableTests.confusing_indexing TableTests.defining_a_method_for_a_builtin_sealed_table_must_fail TableTests.defining_a_method_for_a_local_sealed_table_must_fail @@ -583,7 +549,6 @@ TableTests.right_table_missing_key TableTests.right_table_missing_key2 TableTests.scalar_is_a_subtype_of_a_compatible_polymorphic_shape_type TableTests.scalar_is_not_a_subtype_of_a_compatible_polymorphic_shape_type -TableTests.setmetatable_cant_be_used_to_mutate_global_types TableTests.shared_selfs TableTests.shared_selfs_from_free_param TableTests.shared_selfs_through_metatables @@ -611,7 +576,6 @@ TableTests.used_colon_correctly TableTests.used_colon_instead_of_dot TableTests.used_dot_instead_of_colon TableTests.used_dot_instead_of_colon_but_correctly -TableTests.user_defined_table_types_are_named TableTests.width_subtyping ToDot.bound_table ToDot.class @@ -620,7 +584,6 @@ ToDot.metatable ToDot.primitive ToDot.table ToString.exhaustive_toString_of_cyclic_table -ToString.function_type_with_argument_names ToString.function_type_with_argument_names_and_self ToString.function_type_with_argument_names_generic ToString.named_metatable_toStringNamedFunction @@ -640,48 +603,27 @@ TryUnifyTests.members_of_failed_typepack_unification_are_unified_with_errorType TryUnifyTests.result_of_failed_typepack_unification_is_constrained TryUnifyTests.typepack_unification_should_trim_free_tails TryUnifyTests.variadics_should_use_reversed_properly -TypeAliases.basic_alias -TypeAliases.cannot_steal_hoisted_type_alias TypeAliases.cli_38393_recursive_intersection_oom -TypeAliases.corecursive_function_types TypeAliases.corecursive_types_generic -TypeAliases.cyclic_function_type_in_type_alias -TypeAliases.cyclic_types_of_named_table_fields_do_not_expand_when_stringified TypeAliases.do_not_quantify_unresolved_aliases -TypeAliases.dont_stop_typechecking_after_reporting_duplicate_type_definition -TypeAliases.export_type_and_type_alias_are_duplicates TypeAliases.forward_declared_alias_is_not_clobbered_by_prior_unification_with_any -TypeAliases.forward_declared_alias_is_not_clobbered_by_prior_unification_with_any_2 -TypeAliases.free_variables_from_typeof_in_aliases TypeAliases.general_require_multi_assign TypeAliases.generic_param_remap -TypeAliases.generic_typevars_are_not_considered_to_escape_their_scope_if_they_are_reused_in_multiple_aliases -TypeAliases.module_export_free_type_leak -TypeAliases.module_export_wrapped_free_type_leak -TypeAliases.mutually_recursive_aliases +TypeAliases.mismatched_generic_pack_type_param +TypeAliases.mismatched_generic_type_param TypeAliases.mutually_recursive_generic_aliases -TypeAliases.mutually_recursive_types_errors TypeAliases.mutually_recursive_types_restriction_not_ok_1 TypeAliases.mutually_recursive_types_restriction_not_ok_2 -TypeAliases.mutually_recursive_types_restriction_ok TypeAliases.mutually_recursive_types_swapsies_not_ok -TypeAliases.mutually_recursive_types_swapsies_ok -TypeAliases.names_are_ascribed -TypeAliases.non_recursive_aliases_that_reuse_a_generic_name TypeAliases.recursive_types_restriction_not_ok -TypeAliases.recursive_types_restriction_ok -TypeAliases.reported_location_is_correct_when_type_alias_are_duplicates TypeAliases.stringify_optional_parameterized_alias TypeAliases.stringify_type_alias_of_recursive_template_table_type TypeAliases.stringify_type_alias_of_recursive_template_table_type2 -TypeAliases.type_alias_fwd_declaration_is_precise TypeAliases.type_alias_import_mutation TypeAliases.type_alias_local_mutation TypeAliases.type_alias_local_rename -TypeAliases.type_alias_local_synthetic_mutation TypeAliases.type_alias_of_an_imported_recursive_generic_type TypeAliases.type_alias_of_an_imported_recursive_type -TypeAliases.use_table_name_and_generic_params_in_errors TypeInfer.check_expr_recursion_limit TypeInfer.checking_should_not_ice TypeInfer.cli_50041_committing_txnlog_in_apollo_client_error @@ -698,9 +640,7 @@ TypeInfer.infer_assignment_value_types_mutable_lval TypeInfer.infer_through_group_expr TypeInfer.infer_type_assertion_value_type TypeInfer.no_heap_use_after_free_error -TypeInfer.no_infinite_loop_when_trying_to_unify_uh_this TypeInfer.no_stack_overflow_from_isoptional -TypeInfer.no_stack_overflow_from_isoptional2 TypeInfer.recursive_metatable_crash TypeInfer.tc_after_error_recovery_no_replacement_name_in_error TypeInfer.tc_if_else_expressions1 @@ -946,6 +886,8 @@ TypeInferUnknownNever.length_of_never TypeInferUnknownNever.math_operators_and_never TypeInferUnknownNever.never_is_reflexive TypeInferUnknownNever.never_subtype_and_string_supertype +TypeInferUnknownNever.pick_never_from_variadic_type_pack +TypeInferUnknownNever.string_subtype_and_never_supertype TypeInferUnknownNever.string_subtype_and_unknown_supertype TypeInferUnknownNever.table_with_prop_of_type_never_is_also_reflexive TypeInferUnknownNever.table_with_prop_of_type_never_is_uninhabitable @@ -953,6 +895,7 @@ TypeInferUnknownNever.type_packs_containing_never_is_itself_uninhabitable TypeInferUnknownNever.type_packs_containing_never_is_itself_uninhabitable2 TypeInferUnknownNever.unary_minus_of_never TypeInferUnknownNever.unknown_is_reflexive +TypeInferUnknownNever.unknown_subtype_and_string_supertype TypePackTests.cyclic_type_packs TypePackTests.higher_order_function TypePackTests.multiple_varargs_inference_are_not_confused @@ -965,18 +908,13 @@ TypePackTests.type_alias_default_export TypePackTests.type_alias_default_mixed_self TypePackTests.type_alias_default_type_chained TypePackTests.type_alias_default_type_errors -TypePackTests.type_alias_default_type_explicit -TypePackTests.type_alias_default_type_pack_explicit TypePackTests.type_alias_default_type_pack_self_chained_tp TypePackTests.type_alias_default_type_pack_self_tp -TypePackTests.type_alias_default_type_pack_self_ty TypePackTests.type_alias_default_type_self -TypePackTests.type_alias_default_type_skip_brackets TypePackTests.type_alias_defaults_confusing_types TypePackTests.type_alias_defaults_recursive_type TypePackTests.type_alias_type_pack_explicit TypePackTests.type_alias_type_pack_explicit_multi -TypePackTests.type_alias_type_pack_explicit_multi_tostring TypePackTests.type_alias_type_pack_multi TypePackTests.type_alias_type_pack_variadic TypePackTests.type_alias_type_packs @@ -1010,19 +948,13 @@ TypeSingletons.string_singletons TypeSingletons.string_singletons_escape_chars TypeSingletons.string_singletons_mismatch TypeSingletons.table_insert_with_a_singleton_argument -TypeSingletons.table_properties_alias_or_parens_is_indexer -TypeSingletons.table_properties_singleton_strings -TypeSingletons.table_properties_singleton_strings_mismatch TypeSingletons.table_properties_type_error_escapes -TypeSingletons.tagged_unions_immutable_tag TypeSingletons.tagged_unions_using_singletons -TypeSingletons.tagged_unions_using_singletons_mismatch TypeSingletons.taking_the_length_of_string_singleton TypeSingletons.taking_the_length_of_union_of_string_singleton TypeSingletons.widen_the_supertype_if_it_is_free_and_subtype_has_singleton TypeSingletons.widening_happens_almost_everywhere TypeSingletons.widening_happens_almost_everywhere_except_for_tables -TypeVarTests.visit_once UnionTypes.error_detailed_optional UnionTypes.error_detailed_union_all UnionTypes.error_detailed_union_part @@ -1046,6 +978,5 @@ UnionTypes.optional_union_members UnionTypes.optional_union_methods UnionTypes.return_types_can_be_disjoint UnionTypes.table_union_write_indirect -UnionTypes.unify_sealed_table_union_check UnionTypes.unify_unsealed_table_union_check UnionTypes.union_equality_comparisons diff --git a/tools/perfgraph.py b/tools/perfgraph.py index 95baef9c..ae74b7d6 100644 --- a/tools/perfgraph.py +++ b/tools/perfgraph.py @@ -6,6 +6,12 @@ import sys import svg +import argparse +import json + +argumentParser = argparse.ArgumentParser(description='Generate flamegraph SVG from Luau sampling profiler dumps') +argumentParser.add_argument('source_file', type=open) +argumentParser.add_argument('--json', dest='useJson',action='store_const',const=1,default=0,help='Parse source_file as JSON') class Node(svg.Node): def __init__(self): @@ -27,26 +33,64 @@ class Node(svg.Node): def details(self, root): return "Function: {} [{}:{}] ({:,} usec, {:.1%}); self: {:,} usec".format(self.function, self.source, self.line, self.width, self.width / root.width, self.ticks) -with open(sys.argv[1]) as f: - dump = f.readlines() -root = Node() +def nodeFromCallstackListFile(source_file): + dump = source_file.readlines() + root = Node() -for l in dump: - ticks, stack = l.strip().split(" ", 1) - node = root + for l in dump: + ticks, stack = l.strip().split(" ", 1) + node = root - for f in reversed(stack.split(";")): - source, function, line = f.split(",") + for f in reversed(stack.split(";")): + source, function, line = f.split(",") - child = node.child(f) - child.function = function - child.source = source - child.line = int(line) if len(line) > 0 else 0 + child = node.child(f) + child.function = function + child.source = source + child.line = int(line) if len(line) > 0 else 0 + + node = child + + node.ticks += int(ticks) + + return root + + +def nodeFromJSONbject(node, key, obj): + source, function, line = key.split(",") + + node.function = function + node.source = source + node.line = int(line) if len(line) > 0 else 0 + + node.ticks = obj['Duration'] + + for key, obj in obj['Children'].items(): + nodeFromJSONbject(node.child(key), key, obj) + + return node + + +def nodeFromJSONFile(source_file): + dump = json.load(source_file) + + root = Node() + + for key, obj in dump['Children'].items(): + nodeFromJSONbject(root.child(key), key, obj) + + return root + + +arguments = argumentParser.parse_args() + +if arguments.useJson: + root = nodeFromJSONFile(arguments.source_file) +else: + root = nodeFromCallstackListFile(arguments.source_file) - node = child - node.ticks += int(ticks) svg.layout(root, lambda n: n.ticks) svg.display(root, "Flame Graph", "hot", flip = True) diff --git a/tools/test_dcr.py b/tools/test_dcr.py index 94ff5ca2..0efea3c4 100644 --- a/tools/test_dcr.py +++ b/tools/test_dcr.py @@ -37,17 +37,15 @@ class Handler(x.ContentHandler): elif name == "OverallResultsAsserts": if self.currentTest: - failed = 0 != safeParseInt(attrs["failures"]) + passed = 0 == safeParseInt(attrs["failures"]) dottedName = ".".join(self.currentTest) - shouldFail = dottedName in self.failList - if failed and not shouldFail: - print("UNEXPECTED: {} should have passed".format(dottedName)) - elif not failed and shouldFail: - print("UNEXPECTED: {} should have failed".format(dottedName)) - - self.results[dottedName] = not failed + # Sometimes we get multiple XML trees for the same test. All of + # them must report a pass in order for us to consider the test + # to have passed. + r = self.results.get(dottedName, True) + self.results[dottedName] = r and passed elif name == 'OverallResultsTestCases': self.numSkippedTests = safeParseInt(attrs.get("skipped", 0)) @@ -104,6 +102,12 @@ def main(): p.wait() + for testName, passed in handler.results.items(): + if passed and testName in failList: + print('UNEXPECTED: {} should have failed'.format(testName)) + elif not passed and testName not in failList: + print('UNEXPECTED: {} should have passed'.format(testName)) + if args.write: newFailList = sorted( ( From 6d6fd12fee289ef1892857450ba15f49badeee9e Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Thu, 4 Aug 2022 15:07:49 -0700 Subject: [PATCH 332/399] Let's try this --- tests/conformance/errors.lua | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/tests/conformance/errors.lua b/tests/conformance/errors.lua index 7ab7099e..b13e7a82 100644 --- a/tests/conformance/errors.lua +++ b/tests/conformance/errors.lua @@ -295,8 +295,9 @@ end -- testing syntax limits +local syntaxdepth = if limitedstack then 200 else 1000 local function testrep (init, rep) - local s = "local a; "..init .. string.rep(rep, 300) + local s = "local a; "..init .. string.rep(rep, syntaxdepth) local a,b = loadstring(s) assert(not a) -- and string.find(b, "syntax levels")) end From 1b20fcd43c3e387d8d5585d3a58f14bcf63a4aa6 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Thu, 4 Aug 2022 15:35:33 -0700 Subject: [PATCH 333/399] Sync to upstream/release/539 (#625) --- .gitignore | 2 + Analysis/include/Luau/ApplyTypeFunction.h | 32 ++ .../Luau/{JsonEncoder.h => AstJsonEncoder.h} | 0 Analysis/include/Luau/Constraint.h | 10 +- .../include/Luau/ConstraintGraphBuilder.h | 8 +- Analysis/include/Luau/ConstraintSolver.h | 36 +- Analysis/include/Luau/Frontend.h | 5 +- Analysis/include/Luau/JsonEmitter.h | 235 +++++++++++ Analysis/include/Luau/Linter.h | 1 + Analysis/include/Luau/Module.h | 2 +- Analysis/include/Luau/Scope.h | 4 +- Analysis/include/Luau/Substitution.h | 4 + Analysis/include/Luau/TypeInfer.h | 22 - Analysis/include/Luau/TypeVar.h | 37 +- Analysis/include/Luau/VisitTypeVar.h | 42 +- Analysis/src/ApplyTypeFunction.cpp | 60 +++ .../{JsonEncoder.cpp => AstJsonEncoder.cpp} | 5 +- Analysis/src/AstQuery.cpp | 2 +- Analysis/src/BuiltinDefinitions.cpp | 3 +- Analysis/src/Clone.cpp | 52 +++ Analysis/src/ConstraintGraphBuilder.cpp | 195 +++++++-- Analysis/src/ConstraintSolver.cpp | 371 ++++++++++++++++- Analysis/src/ConstraintSolverLogger.cpp | 98 +++-- Analysis/src/Frontend.cpp | 61 ++- Analysis/src/Instantiation.cpp | 4 + Analysis/src/JsonEmitter.cpp | 220 ++++++++++ Analysis/src/Linter.cpp | 43 ++ Analysis/src/Module.cpp | 22 +- Analysis/src/Normalize.cpp | 9 +- Analysis/src/Quantify.cpp | 25 +- Analysis/src/Scope.cpp | 2 +- Analysis/src/Substitution.cpp | 159 +++++++- Analysis/src/ToString.cpp | 18 + Analysis/src/TypeAttach.cpp | 5 + Analysis/src/TypeChecker2.cpp | 154 ++++++- Analysis/src/TypeInfer.cpp | 107 +---- Analysis/src/TypeVar.cpp | 25 ++ Ast/include/Luau/Ast.h | 12 +- Ast/src/Ast.cpp | 3 +- Ast/src/Parser.cpp | 105 ++++- CLI/Ast.cpp | 2 +- CLI/Repl.cpp | 4 +- Common/include/Luau/Bytecode.h | 24 +- Common/include/Luau/Common.h | 19 +- Common/include/Luau/ExperimentalFlags.h | 7 +- Compiler/include/luacode.h | 4 +- Compiler/src/Builtins.cpp | 4 +- Compiler/src/BytecodeBuilder.cpp | 51 +++ Compiler/src/Compiler.cpp | 124 ++++-- Makefile | 8 +- Sources.cmake | 11 +- VM/include/lua.h | 80 ++-- VM/include/luaconf.h | 32 +- VM/include/lualib.h | 8 +- VM/src/lapi.cpp | 52 +-- VM/src/laux.cpp | 70 ++-- VM/src/lbaselib.cpp | 69 ++-- VM/src/lbitlib.cpp | 18 +- VM/src/lcommon.h | 2 +- VM/src/lcorolib.cpp | 36 +- VM/src/ldblib.cpp | 4 +- VM/src/ldebug.cpp | 11 +- VM/src/ldebug.h | 1 + VM/src/ldo.cpp | 40 +- VM/src/ldo.h | 10 +- VM/src/lfunc.cpp | 20 +- VM/src/lgc.cpp | 130 +++--- VM/src/lgc.h | 6 +- VM/src/lgcdebug.cpp | 4 +- VM/src/lmathlib.cpp | 18 +- VM/src/lnumutils.h | 2 +- VM/src/lobject.cpp | 24 +- VM/src/lobject.h | 74 ++-- VM/src/loslib.cpp | 22 +- VM/src/lstate.cpp | 48 +-- VM/src/lstate.h | 100 ++--- VM/src/lstring.cpp | 20 +- VM/src/lstring.h | 4 +- VM/src/lstrlib.cpp | 382 +++++++++--------- VM/src/ltable.cpp | 192 ++++----- VM/src/ltablib.cpp | 134 +++--- VM/src/ltm.cpp | 12 +- VM/src/ltm.h | 4 +- VM/src/ludata.h | 4 +- VM/src/lutf8lib.cpp | 90 ++--- VM/src/lvmexecute.cpp | 107 ++++- VM/src/lvmutils.cpp | 125 +++--- ...coder.test.cpp => AstJsonEncoder.test.cpp} | 5 +- tests/Autocomplete.test.cpp | 2 +- tests/Compiler.test.cpp | 47 ++- tests/Conformance.test.cpp | 11 +- tests/Fixture.cpp | 2 +- tests/Frontend.test.cpp | 2 - tests/JsonEmitter.test.cpp | 195 +++++++++ tests/Linter.test.cpp | 32 ++ tests/Normalize.test.cpp | 21 +- tests/Parser.test.cpp | 13 +- tests/Repl.test.cpp | 2 +- tests/ToString.test.cpp | 5 - tests/TypeInfer.aliases.test.cpp | 94 +++++ tests/TypeInfer.functions.test.cpp | 1 - tests/TypeInfer.generics.test.cpp | 4 - tests/TypeInfer.provisional.test.cpp | 1 - tests/TypeInfer.singletons.test.cpp | 2 +- tests/TypeInfer.test.cpp | 1 - tests/TypeInfer.typePacks.cpp | 2 +- tests/conformance/errors.lua | 10 +- tests/main.cpp | 2 +- tools/faillist.txt | 83 +--- tools/perfgraph.py | 72 +++- tools/test_dcr.py | 20 +- 111 files changed, 3548 insertions(+), 1494 deletions(-) create mode 100644 Analysis/include/Luau/ApplyTypeFunction.h rename Analysis/include/Luau/{JsonEncoder.h => AstJsonEncoder.h} (100%) create mode 100644 Analysis/include/Luau/JsonEmitter.h create mode 100644 Analysis/src/ApplyTypeFunction.cpp rename Analysis/src/{JsonEncoder.cpp => AstJsonEncoder.cpp} (99%) create mode 100644 Analysis/src/JsonEmitter.cpp rename tests/{JsonEncoder.test.cpp => AstJsonEncoder.test.cpp} (99%) create mode 100644 tests/JsonEmitter.test.cpp diff --git a/.gitignore b/.gitignore index ec5e6578..af77f73c 100644 --- a/.gitignore +++ b/.gitignore @@ -8,4 +8,6 @@ /default.prof* /fuzz-* /luau +/luau-tests +/luau-analyze __pycache__ diff --git a/Analysis/include/Luau/ApplyTypeFunction.h b/Analysis/include/Luau/ApplyTypeFunction.h new file mode 100644 index 00000000..8da3bc42 --- /dev/null +++ b/Analysis/include/Luau/ApplyTypeFunction.h @@ -0,0 +1,32 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +#pragma once + +#include "Luau/Substitution.h" +#include "Luau/TxnLog.h" +#include "Luau/TypeVar.h" + +namespace Luau +{ + +// A substitution which replaces the type parameters of a type function by arguments +struct ApplyTypeFunction : Substitution +{ + ApplyTypeFunction(TypeArena* arena) + : Substitution(TxnLog::empty(), arena) + , encounteredForwardedType(false) + { + } + + // Never set under deferred constraint resolution. + bool encounteredForwardedType; + std::unordered_map typeArguments; + std::unordered_map typePackArguments; + bool ignoreChildren(TypeId ty) override; + bool ignoreChildren(TypePackId tp) override; + bool isDirty(TypeId ty) override; + bool isDirty(TypePackId tp) override; + TypeId clean(TypeId ty) override; + TypePackId clean(TypePackId tp) override; +}; + +} // namespace Luau diff --git a/Analysis/include/Luau/JsonEncoder.h b/Analysis/include/Luau/AstJsonEncoder.h similarity index 100% rename from Analysis/include/Luau/JsonEncoder.h rename to Analysis/include/Luau/AstJsonEncoder.h diff --git a/Analysis/include/Luau/Constraint.h b/Analysis/include/Luau/Constraint.h index 9911c4d3..5b737146 100644 --- a/Analysis/include/Luau/Constraint.h +++ b/Analysis/include/Luau/Constraint.h @@ -4,6 +4,7 @@ #include "Luau/Ast.h" // Used for some of the enumerations #include "Luau/NotNull.h" #include "Luau/Variant.h" +#include "Luau/TypeVar.h" #include #include @@ -71,8 +72,15 @@ struct NameConstraint std::string name; }; +// target ~ inst target +struct TypeAliasExpansionConstraint +{ + // Must be a PendingExpansionTypeVar. + TypeId target; +}; + using ConstraintV = Variant; + BinaryConstraint, NameConstraint, TypeAliasExpansionConstraint>; using ConstraintPtr = std::unique_ptr; struct Constraint diff --git a/Analysis/include/Luau/ConstraintGraphBuilder.h b/Analysis/include/Luau/ConstraintGraphBuilder.h index 695b6cd5..69f35d46 100644 --- a/Analysis/include/Luau/ConstraintGraphBuilder.h +++ b/Analysis/include/Luau/ConstraintGraphBuilder.h @@ -42,6 +42,8 @@ struct ConstraintGraphBuilder DenseHashMap astResolvedTypes{nullptr}; // Type packs resolved from type annotations. Analogous to astTypePacks. DenseHashMap astResolvedTypePacks{nullptr}; + // Defining scopes for AST nodes. + DenseHashMap astTypeAliasDefiningScopes{nullptr}; int recursionCount = 0; @@ -107,6 +109,9 @@ struct ConstraintGraphBuilder void visit(const ScopePtr& scope, AstStatAssign* assign); void visit(const ScopePtr& scope, AstStatIf* ifStatement); void visit(const ScopePtr& scope, AstStatTypeAlias* alias); + void visit(const ScopePtr& scope, AstStatDeclareGlobal* declareGlobal); + void visit(const ScopePtr& scope, AstStatDeclareClass* declareClass); + void visit(const ScopePtr& scope, AstStatDeclareFunction* declareFunction); TypePackId checkExprList(const ScopePtr& scope, const AstArray& exprs); @@ -153,9 +158,10 @@ struct ConstraintGraphBuilder * Resolves a type from its AST annotation. * @param scope the scope that the type annotation appears within. * @param ty the AST annotation to resolve. + * @param topLevel whether the annotation is a "top-level" annotation. * @return the type of the AST annotation. **/ - TypeId resolveType(const ScopePtr& scope, AstType* ty); + TypeId resolveType(const ScopePtr& scope, AstType* ty, bool topLevel = false); /** * Resolves a type pack from its AST annotation. diff --git a/Analysis/include/Luau/ConstraintSolver.h b/Analysis/include/Luau/ConstraintSolver.h index 0b9b4ae3..9cc0e4cb 100644 --- a/Analysis/include/Luau/ConstraintSolver.h +++ b/Analysis/include/Luau/ConstraintSolver.h @@ -17,16 +17,36 @@ namespace Luau // never dereference this pointer. using BlockedConstraintId = const void*; +struct InstantiationSignature +{ + TypeFun fn; + std::vector arguments; + std::vector packArguments; + + bool operator==(const InstantiationSignature& rhs) const; + bool operator!=(const InstantiationSignature& rhs) const + { + return !((*this) == rhs); + } +}; + +struct HashInstantiationSignature +{ + size_t operator()(const InstantiationSignature& signature) const; +}; + struct ConstraintSolver { TypeArena* arena; InternalErrorReporter iceReporter; - // The entire set of constraints that the solver is trying to resolve. It - // is important to not add elements to this vector, lest the underlying - // storage that we retain pointers to be mutated underneath us. - const std::vector> constraints; + // The entire set of constraints that the solver is trying to resolve. + std::vector> constraints; NotNull rootScope; + // Constraints that the solver has generated, rather than sourcing from the + // scope tree. + std::vector> solverConstraints; + // This includes every constraint that has not been fully solved. // A constraint can be both blocked and unsolved, for instance. std::vector> unsolvedConstraints; @@ -37,6 +57,8 @@ struct ConstraintSolver std::unordered_map, size_t> blockedConstraints; // A mapping of type/pack pointers to the constraints they block. std::unordered_map>> blocked; + // Memoized instantiations of type aliases. + DenseHashMap instantiatedAliases{{}}; ConstraintSolverLogger logger; @@ -62,6 +84,7 @@ struct ConstraintSolver bool tryDispatch(const UnaryConstraint& c, NotNull constraint, bool force); bool tryDispatch(const BinaryConstraint& c, NotNull constraint, bool force); bool tryDispatch(const NameConstraint& c, NotNull constraint); + bool tryDispatch(const TypeAliasExpansionConstraint& c, NotNull constraint); void block(NotNull target, NotNull constraint); /** @@ -102,6 +125,11 @@ struct ConstraintSolver */ void unify(TypePackId subPack, TypePackId superPack); + /** Pushes a new solver constraint to the solver. + * @param cv the body of the constraint. + **/ + void pushConstraint(ConstraintV cv); + private: /** * Marks a constraint as being blocked on a type or type pack. The constraint diff --git a/Analysis/include/Luau/Frontend.h b/Analysis/include/Luau/Frontend.h index 3b0164c2..9b8ec19e 100644 --- a/Analysis/include/Luau/Frontend.h +++ b/Analysis/include/Luau/Frontend.h @@ -152,6 +152,8 @@ struct Frontend void registerBuiltinDefinition(const std::string& name, std::function); void applyBuiltinDefinitionToEnvironment(const std::string& environmentName, const std::string& definitionName); + LoadDefinitionFileResult loadDefinitionFile(std::string_view source, const std::string& packageName); + NotNull getGlobalScope(); private: @@ -169,7 +171,7 @@ private: std::unordered_map environments; std::unordered_map> builtinDefinitions; - std::unique_ptr globalScope; + ScopePtr globalScope; public: FileResolver* fileResolver; @@ -180,6 +182,7 @@ public: ConfigResolver* configResolver; FrontendOptions options; InternalErrorReporter iceHandler; + TypeArena globalTypes; TypeArena arenaForAutocomplete; std::unordered_map sourceNodes; diff --git a/Analysis/include/Luau/JsonEmitter.h b/Analysis/include/Luau/JsonEmitter.h new file mode 100644 index 00000000..0bf3327a --- /dev/null +++ b/Analysis/include/Luau/JsonEmitter.h @@ -0,0 +1,235 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +#pragma once + +#include +#include +#include +#include + +#include "Luau/NotNull.h" + +namespace Luau::Json +{ + +struct JsonEmitter; + +/// Writes a value to the JsonEmitter. Note that this can produce invalid JSON +/// if you do not insert commas or appropriate object / array syntax. +template +void write(JsonEmitter&, T) = delete; + +/// Writes a boolean to a JsonEmitter. +/// @param emitter the emitter to write to. +/// @param b the boolean to write. +void write(JsonEmitter& emitter, bool b); + +/// Writes an integer to a JsonEmitter. +/// @param emitter the emitter to write to. +/// @param i the integer to write. +void write(JsonEmitter& emitter, int i); + +/// Writes an integer to a JsonEmitter. +/// @param emitter the emitter to write to. +/// @param i the integer to write. +void write(JsonEmitter& emitter, long i); + +/// Writes an integer to a JsonEmitter. +/// @param emitter the emitter to write to. +/// @param i the integer to write. +void write(JsonEmitter& emitter, long long i); + +/// Writes an integer to a JsonEmitter. +/// @param emitter the emitter to write to. +/// @param i the integer to write. +void write(JsonEmitter& emitter, unsigned int i); + +/// Writes an integer to a JsonEmitter. +/// @param emitter the emitter to write to. +/// @param i the integer to write. +void write(JsonEmitter& emitter, unsigned long i); + +/// Writes an integer to a JsonEmitter. +/// @param emitter the emitter to write to. +/// @param i the integer to write. +void write(JsonEmitter& emitter, unsigned long long i); + +/// Writes a double to a JsonEmitter. +/// @param emitter the emitter to write to. +/// @param d the double to write. +void write(JsonEmitter& emitter, double d); + +/// Writes a string to a JsonEmitter. The string will be escaped. +/// @param emitter the emitter to write to. +/// @param sv the string to write. +void write(JsonEmitter& emitter, std::string_view sv); + +/// Writes a character to a JsonEmitter as a single-character string. The +/// character will be escaped. +/// @param emitter the emitter to write to. +/// @param c the string to write. +void write(JsonEmitter& emitter, char c); + +/// Writes a string to a JsonEmitter. The string will be escaped. +/// @param emitter the emitter to write to. +/// @param str the string to write. +void write(JsonEmitter& emitter, const char* str); + +/// Writes a string to a JsonEmitter. The string will be escaped. +/// @param emitter the emitter to write to. +/// @param str the string to write. +void write(JsonEmitter& emitter, const std::string& str); + +/// Writes null to a JsonEmitter. +/// @param emitter the emitter to write to. +void write(JsonEmitter& emitter, std::nullptr_t); + +/// Writes null to a JsonEmitter. +/// @param emitter the emitter to write to. +void write(JsonEmitter& emitter, std::nullopt_t); + +struct ObjectEmitter; +struct ArrayEmitter; + +struct JsonEmitter +{ + JsonEmitter(); + + /// Converts the current contents of the JsonEmitter to a string value. This + /// does not invalidate the emitter, but it does not clear it either. + std::string str(); + + /// Returns the current comma state and resets it to false. Use popComma to + /// restore the old state. + /// @returns the previous comma state. + bool pushComma(); + + /// Restores a previous comma state. + /// @param c the comma state to restore. + void popComma(bool c); + + /// Writes a raw sequence of characters to the buffer, without escaping or + /// other processing. + /// @param sv the character sequence to write. + void writeRaw(std::string_view sv); + + /// Writes a character to the buffer, without escaping or other processing. + /// @param c the character to write. + void writeRaw(char c); + + /// Writes a comma if this wasn't the first time writeComma has been + /// invoked. Otherwise, sets the comma state to true. + /// @see pushComma + /// @see popComma + void writeComma(); + + /// Begins writing an object to the emitter. + /// @returns an ObjectEmitter that can be used to write key-value pairs. + ObjectEmitter writeObject(); + + /// Begins writing an array to the emitter. + /// @returns an ArrayEmitter that can be used to write values. + ArrayEmitter writeArray(); + +private: + bool comma = false; + std::vector chunks; + + void newChunk(); +}; + +/// An interface for writing an object into a JsonEmitter instance. +/// @see JsonEmitter::writeObject +struct ObjectEmitter +{ + ObjectEmitter(NotNull emitter); + ~ObjectEmitter(); + + NotNull emitter; + bool comma; + bool finished; + + /// Writes a key-value pair to the associated JsonEmitter. Keys will be escaped. + /// @param name the name of the key-value pair. + /// @param value the value to write. + template + void writePair(std::string_view name, T value) + { + if (finished) + { + return; + } + + emitter->writeComma(); + write(*emitter, name); + emitter->writeRaw(':'); + write(*emitter, value); + } + + /// Finishes writing the object, appending a closing `}` character and + /// resetting the comma state of the associated emitter. This can only be + /// called once, and once called will render the emitter unusable. This + /// method is also called when the ObjectEmitter is destructed. + void finish(); +}; + +/// An interface for writing an array into a JsonEmitter instance. Array values +/// do not need to be the same type. +/// @see JsonEmitter::writeArray +struct ArrayEmitter +{ + ArrayEmitter(NotNull emitter); + ~ArrayEmitter(); + + NotNull emitter; + bool comma; + bool finished; + + /// Writes a value to the array. + /// @param value the value to write. + template + void writeValue(T value) + { + if (finished) + { + return; + } + + emitter->writeComma(); + write(*emitter, value); + } + + /// Finishes writing the object, appending a closing `]` character and + /// resetting the comma state of the associated emitter. This can only be + /// called once, and once called will render the emitter unusable. This + /// method is also called when the ArrayEmitter is destructed. + void finish(); +}; + +/// Writes a vector as an array to a JsonEmitter. +/// @param emitter the emitter to write to. +/// @param vec the vector to write. +template +void write(JsonEmitter& emitter, const std::vector& vec) +{ + ArrayEmitter a = emitter.writeArray(); + + for (const T& value : vec) + a.writeValue(value); + + a.finish(); +} + +/// Writes an optional to a JsonEmitter. Will write the contained value, if +/// present, or null, if no value is present. +/// @param emitter the emitter to write to. +/// @param v the value to write. +template +void write(JsonEmitter& emitter, const std::optional& v) +{ + if (v.has_value()) + write(emitter, *v); + else + emitter.writeRaw("null"); +} + +} // namespace Luau::Json diff --git a/Analysis/include/Luau/Linter.h b/Analysis/include/Luau/Linter.h index 6c7ce47f..2cd91d54 100644 --- a/Analysis/include/Luau/Linter.h +++ b/Analysis/include/Luau/Linter.h @@ -52,6 +52,7 @@ struct LintWarning Code_DuplicateCondition = 24, Code_MisleadingAndOr = 25, Code_CommentDirective = 26, + Code_IntegerParsing = 27, Code__Count }; diff --git a/Analysis/include/Luau/Module.h b/Analysis/include/Luau/Module.h index 73f4c972..6f4c6098 100644 --- a/Analysis/include/Luau/Module.h +++ b/Analysis/include/Luau/Module.h @@ -68,7 +68,7 @@ struct Module std::shared_ptr allocator; std::shared_ptr names; - std::vector> scopes; // never empty + std::vector> scopes; // never empty DenseHashMap astTypes{nullptr}; DenseHashMap astTypePacks{nullptr}; diff --git a/Analysis/include/Luau/Scope.h b/Analysis/include/Luau/Scope.h index 85c1750a..55ca54c6 100644 --- a/Analysis/include/Luau/Scope.h +++ b/Analysis/include/Luau/Scope.h @@ -36,7 +36,7 @@ struct Scope // All the children of this scope. std::vector> children; std::unordered_map bindings; - std::unordered_map typeBindings; + std::unordered_map typeBindings; std::unordered_map typePackBindings; TypePackId returnType; std::optional varargPack; @@ -52,7 +52,7 @@ struct Scope std::unordered_map> importedTypeBindings; std::optional lookup(Symbol sym); - std::optional lookupTypeBinding(const Name& name); + std::optional lookupTypeBinding(const Name& name); std::optional lookupTypePackBinding(const Name& name); std::optional lookupType(const Name& name); diff --git a/Analysis/include/Luau/Substitution.h b/Analysis/include/Luau/Substitution.h index f3c3ae9a..6ad38f9d 100644 --- a/Analysis/include/Luau/Substitution.h +++ b/Analysis/include/Luau/Substitution.h @@ -139,6 +139,8 @@ struct FindDirty : Tarjan { std::vector dirty; + void clearTarjan(); + // Get/set the dirty bit for an index (grows the vector if needed) bool getDirty(int index); void setDirty(int index, bool d); @@ -176,6 +178,8 @@ public: TypeArena* arena; DenseHashMap newTypes{nullptr}; DenseHashMap newPacks{nullptr}; + DenseHashSet replacedTypes{nullptr}; + DenseHashSet replacedTypePacks{nullptr}; std::optional substitute(TypeId ty); std::optional substitute(TypePackId tp); diff --git a/Analysis/include/Luau/TypeInfer.h b/Analysis/include/Luau/TypeInfer.h index 3fb710bb..c50b2c8c 100644 --- a/Analysis/include/Luau/TypeInfer.h +++ b/Analysis/include/Luau/TypeInfer.h @@ -65,28 +65,6 @@ struct Anyification : Substitution } }; -// A substitution which replaces the type parameters of a type function by arguments -struct ApplyTypeFunction : Substitution -{ - ApplyTypeFunction(TypeArena* arena, TypeLevel level) - : Substitution(TxnLog::empty(), arena) - , level(level) - , encounteredForwardedType(false) - { - } - - TypeLevel level; - bool encounteredForwardedType; - std::unordered_map typeArguments; - std::unordered_map typePackArguments; - bool ignoreChildren(TypeId ty) override; - bool ignoreChildren(TypePackId tp) override; - bool isDirty(TypeId ty) override; - bool isDirty(TypePackId tp) override; - TypeId clean(TypeId ty) override; - TypePackId clean(TypePackId tp) override; -}; - struct GenericTypeDefinitions { std::vector genericTypes; diff --git a/Analysis/include/Luau/TypeVar.h b/Analysis/include/Luau/TypeVar.h index 052d4d88..6a13b11c 100644 --- a/Analysis/include/Luau/TypeVar.h +++ b/Analysis/include/Luau/TypeVar.h @@ -223,12 +223,16 @@ struct GenericTypeDefinition { TypeId ty; std::optional defaultValue; + + bool operator==(const GenericTypeDefinition& rhs) const; }; struct GenericTypePackDefinition { TypePackId tp; std::optional defaultValue; + + bool operator==(const GenericTypePackDefinition& rhs) const; }; struct FunctionArgument @@ -426,6 +430,12 @@ struct TypeFun TypeId type; TypeFun() = default; + + explicit TypeFun(TypeId ty) + : type(ty) + { + } + TypeFun(std::vector typeParams, TypeId type) : typeParams(std::move(typeParams)) , type(type) @@ -438,6 +448,27 @@ struct TypeFun , type(type) { } + + bool operator==(const TypeFun& rhs) const; +}; + +/** Represents a pending type alias instantiation. + * + * In order to afford (co)recursive type aliases, we need to reason about a + * partially-complete instantiation. This requires encoding more information in + * a type variable than a BlockedTypeVar affords, hence this. Each + * PendingExpansionTypeVar has a corresponding TypeAliasExpansionConstraint + * enqueued in the solver to convert it to an actual instantiated type + */ +struct PendingExpansionTypeVar +{ + PendingExpansionTypeVar(TypeFun fn, std::vector typeArguments, std::vector packArguments); + TypeFun fn; + std::vector typeArguments; + std::vector packArguments; + size_t index; + + static size_t nextIndex; }; // Anything! All static checking is off. @@ -470,8 +501,10 @@ struct NeverTypeVar using ErrorTypeVar = Unifiable::Error; -using TypeVariant = Unifiable::Variant; +using TypeVariant = + Unifiable::Variant; + struct TypeVar final { diff --git a/Analysis/include/Luau/VisitTypeVar.h b/Analysis/include/Luau/VisitTypeVar.h index 7229dafc..7e5d71d6 100644 --- a/Analysis/include/Luau/VisitTypeVar.h +++ b/Analysis/include/Luau/VisitTypeVar.h @@ -9,7 +9,6 @@ #include "Luau/TypeVar.h" LUAU_FASTINT(LuauVisitRecursionLimit) -LUAU_FASTFLAG(LuauNormalizeFlagIsConservative) LUAU_FASTFLAG(LuauCompleteVisitor); namespace Luau @@ -150,6 +149,10 @@ struct GenericTypeVarVisitor { return visit(ty); } + virtual bool visit(TypeId ty, const PendingExpansionTypeVar& petv) + { + return visit(ty); + } virtual bool visit(TypeId ty, const SingletonTypeVar& stv) { return visit(ty); @@ -285,8 +288,6 @@ struct GenericTypeVarVisitor traverse(partTy); } } - else if (!FFlag::LuauCompleteVisitor) - return visit_detail::unsee(seen, ty); else if (get(ty)) { // Visiting into LazyTypeVar may necessarily cause infinite expansion, so we don't do that on purpose. @@ -301,6 +302,37 @@ struct GenericTypeVarVisitor visit(ty, *utv); else if (auto ntv = get(ty)) visit(ty, *ntv); + else if (auto petv = get(ty)) + { + if (visit(ty, *petv)) + { + traverse(petv->fn.type); + + for (const GenericTypeDefinition& p : petv->fn.typeParams) + { + traverse(p.ty); + + if (p.defaultValue) + traverse(*p.defaultValue); + } + + for (const GenericTypePackDefinition& p : petv->fn.typePackParams) + { + traverse(p.tp); + + if (p.defaultValue) + traverse(*p.defaultValue); + } + + for (TypeId a : petv->typeArguments) + traverse(a); + + for (TypePackId a : petv->packArguments) + traverse(a); + } + } + else if (!FFlag::LuauCompleteVisitor) + return visit_detail::unsee(seen, ty); else LUAU_ASSERT(!"GenericTypeVarVisitor::traverse(TypeId) is not exhaustive!"); @@ -333,7 +365,7 @@ struct GenericTypeVarVisitor else if (auto pack = get(tp)) { bool res = visit(tp, *pack); - if (!FFlag::LuauNormalizeFlagIsConservative || res) + if (res) { for (TypeId ty : pack->head) traverse(ty); @@ -345,7 +377,7 @@ struct GenericTypeVarVisitor else if (auto pack = get(tp)) { bool res = visit(tp, *pack); - if (!FFlag::LuauNormalizeFlagIsConservative || res) + if (res) traverse(pack->ty); } else diff --git a/Analysis/src/ApplyTypeFunction.cpp b/Analysis/src/ApplyTypeFunction.cpp new file mode 100644 index 00000000..c6ac3e19 --- /dev/null +++ b/Analysis/src/ApplyTypeFunction.cpp @@ -0,0 +1,60 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details + +#include "Luau/ApplyTypeFunction.h" + +namespace Luau +{ + +bool ApplyTypeFunction::isDirty(TypeId ty) +{ + if (typeArguments.count(ty)) + return true; + else if (const FreeTypeVar* ftv = get(ty)) + { + if (ftv->forwardedTypeAlias) + encounteredForwardedType = true; + return false; + } + else + return false; +} + +bool ApplyTypeFunction::isDirty(TypePackId tp) +{ + if (typePackArguments.count(tp)) + return true; + else + return false; +} + +bool ApplyTypeFunction::ignoreChildren(TypeId ty) +{ + if (get(ty)) + return true; + else + return false; +} + +bool ApplyTypeFunction::ignoreChildren(TypePackId tp) +{ + if (get(tp)) + return true; + else + return false; +} + +TypeId ApplyTypeFunction::clean(TypeId ty) +{ + TypeId& arg = typeArguments[ty]; + LUAU_ASSERT(arg); + return arg; +} + +TypePackId ApplyTypeFunction::clean(TypePackId tp) +{ + TypePackId& arg = typePackArguments[tp]; + LUAU_ASSERT(arg); + return arg; +} + +} // namespace Luau diff --git a/Analysis/src/JsonEncoder.cpp b/Analysis/src/AstJsonEncoder.cpp similarity index 99% rename from Analysis/src/JsonEncoder.cpp rename to Analysis/src/AstJsonEncoder.cpp index ee7dadb3..2897875d 100644 --- a/Analysis/src/JsonEncoder.cpp +++ b/Analysis/src/AstJsonEncoder.cpp @@ -1,5 +1,5 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details -#include "Luau/JsonEncoder.h" +#include "Luau/AstJsonEncoder.h" #include "Luau/Ast.h" #include "Luau/ParseResult.h" @@ -773,7 +773,7 @@ struct AstJsonEncoder : public AstVisitor PROP(indexer); }); } - + void write(struct AstTableIndexer* indexer) { if (indexer) @@ -1178,7 +1178,6 @@ struct AstJsonEncoder : public AstVisitor write("location", comment.location); popComma(c); writeRaw("}"); - } } }; diff --git a/Analysis/src/AstQuery.cpp b/Analysis/src/AstQuery.cpp index 1124c29e..50299704 100644 --- a/Analysis/src/AstQuery.cpp +++ b/Analysis/src/AstQuery.cpp @@ -314,7 +314,7 @@ std::optional findBindingAtPosition(const Module& module, const SourceM auto iter = currentScope->bindings.find(name); if (iter != currentScope->bindings.end() && iter->second.location.begin <= pos) { - /* Ignore this binding if we're inside its definition. e.g. local abc = abc -- Will take the definition of abc from outer scope */ + // Ignore this binding if we're inside its definition. e.g. local abc = abc -- Will take the definition of abc from outer scope std::optional bindingStatement = findBindingLocalStatement(source, iter->second); if (!bindingStatement || !(*bindingStatement)->location.contains(pos)) return iter->second; diff --git a/Analysis/src/BuiltinDefinitions.cpp b/Analysis/src/BuiltinDefinitions.cpp index aeba2c13..826179b3 100644 --- a/Analysis/src/BuiltinDefinitions.cpp +++ b/Analysis/src/BuiltinDefinitions.cpp @@ -10,6 +10,7 @@ LUAU_FASTFLAGVARIABLE(LuauSetMetaTableArgsCheck, false) LUAU_FASTFLAG(LuauUnknownAndNeverType) +LUAU_FASTFLAGVARIABLE(LuauBuiltInMetatableNoBadSynthetic, false) /** FIXME: Many of these type definitions are not quite completely accurate. * @@ -349,7 +350,7 @@ static std::optional> magicFunctionSetMetaTable( if (tableName == metatableName) mtv.syntheticName = tableName; - else + else if (!FFlag::LuauBuiltInMetatableNoBadSynthetic) mtv.syntheticName = "{ @metatable: " + metatableName + ", " + tableName + " }"; } diff --git a/Analysis/src/Clone.cpp b/Analysis/src/Clone.cpp index 88c50318..51ad61d5 100644 --- a/Analysis/src/Clone.cpp +++ b/Analysis/src/Clone.cpp @@ -48,6 +48,7 @@ struct TypeCloner void operator()(const Unifiable::Bound& t); void operator()(const Unifiable::Error& t); void operator()(const BlockedTypeVar& t); + void operator()(const PendingExpansionTypeVar& t); void operator()(const PrimitiveTypeVar& t); void operator()(const ConstrainedTypeVar& t); void operator()(const SingletonTypeVar& t); @@ -166,6 +167,52 @@ void TypeCloner::operator()(const BlockedTypeVar& t) defaultClone(t); } +void TypeCloner::operator()(const PendingExpansionTypeVar& t) +{ + TypeId res = dest.addType(PendingExpansionTypeVar{t.fn, t.typeArguments, t.packArguments}); + PendingExpansionTypeVar* petv = getMutable(res); + LUAU_ASSERT(petv); + + seenTypes[typeId] = res; + + std::vector typeArguments; + for (TypeId arg : t.typeArguments) + typeArguments.push_back(clone(arg, dest, cloneState)); + + std::vector packArguments; + for (TypePackId arg : t.packArguments) + packArguments.push_back(clone(arg, dest, cloneState)); + + TypeFun fn; + fn.type = clone(t.fn.type, dest, cloneState); + + for (const GenericTypeDefinition& param : t.fn.typeParams) + { + TypeId ty = clone(param.ty, dest, cloneState); + std::optional defaultValue = param.defaultValue; + + if (defaultValue) + defaultValue = clone(*defaultValue, dest, cloneState); + + fn.typeParams.push_back(GenericTypeDefinition{ty, defaultValue}); + } + + for (const GenericTypePackDefinition& param : t.fn.typePackParams) + { + TypePackId tp = clone(param.tp, dest, cloneState); + std::optional defaultValue = param.defaultValue; + + if (defaultValue) + defaultValue = clone(*defaultValue, dest, cloneState); + + fn.typePackParams.push_back(GenericTypePackDefinition{tp, defaultValue}); + } + + petv->fn = std::move(fn); + petv->typeArguments = std::move(typeArguments); + petv->packArguments = std::move(packArguments); +} + void TypeCloner::operator()(const PrimitiveTypeVar& t) { defaultClone(t); @@ -452,6 +499,11 @@ TypeId shallowClone(TypeId ty, TypeArena& dest, const TxnLog* log) ConstrainedTypeVar clone{ctv->level, ctv->parts}; result = dest.addType(std::move(clone)); } + else if (const PendingExpansionTypeVar* petv = get(ty)) + { + PendingExpansionTypeVar clone{petv->fn, petv->typeArguments, petv->packArguments}; + result = dest.addType(std::move(clone)); + } else return result; diff --git a/Analysis/src/ConstraintGraphBuilder.cpp b/Analysis/src/ConstraintGraphBuilder.cpp index efaeff6c..ea7037bf 100644 --- a/Analysis/src/ConstraintGraphBuilder.cpp +++ b/Analysis/src/ConstraintGraphBuilder.cpp @@ -70,11 +70,11 @@ void ConstraintGraphBuilder::visit(AstStatBlock* block) prepopulateGlobalScope(scope, block); // TODO: We should share the global scope. - rootScope->typeBindings["nil"] = singletonTypes.nilType; - rootScope->typeBindings["number"] = singletonTypes.numberType; - rootScope->typeBindings["string"] = singletonTypes.stringType; - rootScope->typeBindings["boolean"] = singletonTypes.booleanType; - rootScope->typeBindings["thread"] = singletonTypes.threadType; + rootScope->typeBindings["nil"] = TypeFun{singletonTypes.nilType}; + rootScope->typeBindings["number"] = TypeFun{singletonTypes.numberType}; + rootScope->typeBindings["string"] = TypeFun{singletonTypes.stringType}; + rootScope->typeBindings["boolean"] = TypeFun{singletonTypes.booleanType}; + rootScope->typeBindings["thread"] = TypeFun{singletonTypes.threadType}; visitBlockWithoutChildScope(scope, block); } @@ -89,6 +89,53 @@ void ConstraintGraphBuilder::visitBlockWithoutChildScope(const ScopePtr& scope, return; } + std::unordered_map aliasDefinitionLocations; + + // In order to enable mutually-recursive type aliases, we need to + // populate the type bindings before we actually check any of the + // alias statements. Since we're not ready to actually resolve + // any of the annotations, we just use a fresh type for now. + for (AstStat* stat : block->body) + { + if (auto alias = stat->as()) + { + if (scope->typeBindings.count(alias->name.value) != 0) + { + auto it = aliasDefinitionLocations.find(alias->name.value); + LUAU_ASSERT(it != aliasDefinitionLocations.end()); + reportError(alias->location, DuplicateTypeDefinition{alias->name.value, it->second}); + continue; + } + + bool hasGenerics = alias->generics.size > 0 || alias->genericPacks.size > 0; + + ScopePtr defnScope = scope; + if (hasGenerics) + { + defnScope = childScope(alias->location, scope); + } + + TypeId initialType = freshType(scope); + TypeFun initialFun = TypeFun{initialType}; + + for (const auto& [name, gen] : createGenerics(defnScope, alias->generics)) + { + initialFun.typeParams.push_back(gen); + defnScope->typeBindings[name] = TypeFun{gen.ty}; + } + + for (const auto& [name, genPack] : createGenericPacks(defnScope, alias->genericPacks)) + { + initialFun.typePackParams.push_back(genPack); + defnScope->typePackBindings[name] = genPack.tp; + } + + scope->typeBindings[alias->name.value] = std::move(initialFun); + astTypeAliasDefiningScopes[alias] = defnScope; + aliasDefinitionLocations[alias->name.value] = alias->location; + } + } + for (AstStat* stat : block->body) visit(scope, stat); } @@ -117,6 +164,12 @@ void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStat* stat) visit(scope, i); else if (auto a = stat->as()) visit(scope, a); + else if (auto s = stat->as()) + visit(scope, s); + else if (auto s = stat->as()) + visit(scope, s); + else if (auto s = stat->as()) + visit(scope, s); else LUAU_ASSERT(0); } @@ -133,7 +186,7 @@ void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatLocal* local) if (local->annotation) { location = local->annotation->location; - TypeId annotation = resolveType(scope, local->annotation); + TypeId annotation = resolveType(scope, local->annotation, /* topLevel */ true); addConstraint(scope, SubtypeConstraint{ty, annotation}); } @@ -171,11 +224,10 @@ void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatLocal* local) void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatFor* for_) { - auto checkNumber = [&](AstExpr* expr) - { + auto checkNumber = [&](AstExpr* expr) { if (!expr) return; - + TypeId t = check(scope, expr); addConstraint(scope, SubtypeConstraint{t, singletonTypes.numberType}); }; @@ -307,19 +359,6 @@ void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatBlock* block) { ScopePtr innerScope = childScope(block->location, scope); - // In order to enable mutually-recursive type aliases, we need to - // populate the type bindings before we actually check any of the - // alias statements. Since we're not ready to actually resolve - // any of the annotations, we just use a fresh type for now. - for (AstStat* stat : block->body) - { - if (auto alias = stat->as()) - { - TypeId initialType = freshType(scope); - scope->typeBindings[alias->name.value] = initialType; - } - } - visitBlockWithoutChildScope(innerScope, block); } @@ -348,29 +387,48 @@ void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatIf* ifStatement void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatTypeAlias* alias) { // TODO: Exported type aliases - // TODO: Generic type aliases - auto it = scope->typeBindings.find(alias->name.value); - // This should always be here since we do a separate pass over the - // AST to set up typeBindings. If it's not, we've somehow skipped - // this alias in that first pass. - LUAU_ASSERT(it != scope->typeBindings.end()); - if (it == scope->typeBindings.end()) + auto bindingIt = scope->typeBindings.find(alias->name.value); + ScopePtr* defnIt = astTypeAliasDefiningScopes.find(alias); + // These will be undefined if the alias was a duplicate definition, in which + // case we just skip over it. + if (bindingIt == scope->typeBindings.end() || defnIt == nullptr) { - ice->ice("Type alias does not have a pre-populated binding", alias->location); + return; } - TypeId ty = resolveType(scope, alias->type); + ScopePtr resolvingScope = *defnIt; + TypeId ty = resolveType(resolvingScope, alias->type, /* topLevel */ true); + + LUAU_ASSERT(get(bindingIt->second.type)); // Rather than using a subtype constraint, we instead directly bind // the free type we generated in the first pass to the resolved type. // This prevents a case where you could cause another constraint to // bind the free alias type to an unrelated type, causing havoc. - asMutable(it->second)->ty.emplace(ty); + asMutable(bindingIt->second.type)->ty.emplace(ty); addConstraint(scope, NameConstraint{ty, alias->name.value}); } +void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatDeclareGlobal* global) +{ + LUAU_ASSERT(global->type); + + TypeId globalTy = resolveType(scope, global->type); + scope->bindings[global->name] = Binding{globalTy, global->location}; +} + +void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatDeclareClass* global) +{ + LUAU_ASSERT(false); // TODO: implement +} + +void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatDeclareFunction* global) +{ + LUAU_ASSERT(false); // TODO: implement +} + TypePackId ConstraintGraphBuilder::checkPack(const ScopePtr& scope, AstArray exprs) { if (exprs.size == 0) @@ -707,7 +765,7 @@ ConstraintGraphBuilder::FunctionSignature ConstraintGraphBuilder::checkFunctionS for (const auto& [name, g] : genericDefinitions) { genericTypes.push_back(g.ty); - signatureScope->typeBindings[name] = g.ty; + signatureScope->typeBindings[name] = TypeFun{g.ty}; } for (const auto& [name, g] : genericPackDefinitions) @@ -745,7 +803,7 @@ ConstraintGraphBuilder::FunctionSignature ConstraintGraphBuilder::checkFunctionS if (local->annotation) { - TypeId argAnnotation = resolveType(signatureScope, local->annotation); + TypeId argAnnotation = resolveType(signatureScope, local->annotation, /* topLevel */ true); addConstraint(signatureScope, SubtypeConstraint{t, argAnnotation}); } } @@ -784,20 +842,65 @@ void ConstraintGraphBuilder::checkFunctionBody(const ScopePtr& scope, AstExprFun } } -TypeId ConstraintGraphBuilder::resolveType(const ScopePtr& scope, AstType* ty) +TypeId ConstraintGraphBuilder::resolveType(const ScopePtr& scope, AstType* ty, bool topLevel) { TypeId result = nullptr; if (auto ref = ty->as()) { // TODO: Support imported types w/ require tracing. - // TODO: Support generic type references. LUAU_ASSERT(!ref->prefix); - LUAU_ASSERT(!ref->hasParameterList); - // TODO: If it doesn't exist, should we introduce a free binding? - // This is probably important for handling type aliases. - result = scope->lookupTypeBinding(ref->name.value).value_or(singletonTypes.errorRecoveryType()); + std::optional alias = scope->lookupTypeBinding(ref->name.value); + + if (alias.has_value()) + { + // If the alias is not generic, we don't need to set up a blocked + // type and an instantiation constraint. + if (alias->typeParams.empty() && alias->typePackParams.empty()) + { + result = alias->type; + } + else + { + std::vector parameters; + std::vector packParameters; + + for (const AstTypeOrPack& p : ref->parameters) + { + // We do not enforce the ordering of types vs. type packs here; + // that is done in the parser. + if (p.type) + { + parameters.push_back(resolveType(scope, p.type)); + } + else if (p.typePack) + { + packParameters.push_back(resolveTypePack(scope, p.typePack)); + } + else + { + // This indicates a parser bug: one of these two pointers + // should be set. + LUAU_ASSERT(false); + } + } + + result = arena->addType(PendingExpansionTypeVar{*alias, parameters, packParameters}); + + if (topLevel) + { + addConstraint(scope, TypeAliasExpansionConstraint{ + /* target */ result, + }); + } + } + } + else + { + reportError(ty->location, UnknownSymbol{ref->name.value, UnknownSymbol::Context::Type}); + result = singletonTypes.errorRecoveryType(); + } } else if (auto tab = ty->as()) { @@ -846,7 +949,7 @@ TypeId ConstraintGraphBuilder::resolveType(const ScopePtr& scope, AstType* ty) for (const auto& [name, g] : genericDefinitions) { genericTypes.push_back(g.ty); - signatureScope->typeBindings[name] = g.ty; + signatureScope->typeBindings[name] = TypeFun{g.ty}; } for (const auto& [name, g] : genericPackDefinitions) @@ -956,7 +1059,15 @@ TypePackId ConstraintGraphBuilder::resolveTypePack(const ScopePtr& scope, AstTyp } else if (auto gen = tp->as()) { - result = arena->addTypePack(TypePackVar{GenericTypePack{scope.get(), gen->genericName.value}}); + if (std::optional lookup = scope->lookupTypePackBinding(gen->genericName.value)) + { + result = *lookup; + } + else + { + reportError(tp->location, UnknownSymbol{gen->genericName.value, UnknownSymbol::Context::Type}); + result = singletonTypes.errorRecoveryTypePack(); + } } else { diff --git a/Analysis/src/ConstraintSolver.cpp b/Analysis/src/ConstraintSolver.cpp index 1cd29918..0898f9aa 100644 --- a/Analysis/src/ConstraintSolver.cpp +++ b/Analysis/src/ConstraintSolver.cpp @@ -1,11 +1,13 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +#include "Luau/ApplyTypeFunction.h" #include "Luau/ConstraintSolver.h" #include "Luau/Instantiation.h" #include "Luau/Location.h" #include "Luau/Quantify.h" #include "Luau/ToString.h" #include "Luau/Unifier.h" +#include "Luau/VisitTypeVar.h" LUAU_FASTFLAGVARIABLE(DebugLuauLogSolver, false); LUAU_FASTFLAGVARIABLE(DebugLuauLogSolverToJson, false); @@ -37,6 +39,170 @@ static void dumpConstraints(NotNull scope, ToStringOptions& opts) dumpConstraints(child, opts); } +static std::pair, std::vector> saturateArguments( + const TypeFun& fn, const std::vector& rawTypeArguments, const std::vector& rawPackArguments, TypeArena* arena) +{ + std::vector saturatedTypeArguments; + std::vector extraTypes; + std::vector saturatedPackArguments; + + for (size_t i = 0; i < rawTypeArguments.size(); ++i) + { + TypeId ty = rawTypeArguments[i]; + + if (i < fn.typeParams.size()) + saturatedTypeArguments.push_back(ty); + else + extraTypes.push_back(ty); + } + + // If we collected extra types, put them in a type pack now. This case is + // mutually exclusive with the type pack -> type conversion we do below: + // extraTypes will only have elements in it if we have more types than we + // have parameter slots for them to go into. + if (!extraTypes.empty()) + { + saturatedPackArguments.push_back(arena->addTypePack(extraTypes)); + } + + for (size_t i = 0; i < rawPackArguments.size(); ++i) + { + TypePackId tp = rawPackArguments[i]; + + // If we are short on regular type saturatedTypeArguments and we have a single + // element type pack, we can decompose that to the type it contains and + // use that as a type parameter. + if (saturatedTypeArguments.size() < fn.typeParams.size() && size(tp) == 1 && finite(tp) && first(tp) && saturatedPackArguments.empty()) + { + saturatedTypeArguments.push_back(*first(tp)); + } + else + { + saturatedPackArguments.push_back(tp); + } + } + + size_t typesProvided = saturatedTypeArguments.size(); + size_t typesRequired = fn.typeParams.size(); + + size_t packsProvided = saturatedPackArguments.size(); + size_t packsRequired = fn.typePackParams.size(); + + // Extra types should be accumulated in extraTypes, not saturatedTypeArguments. Extra + // packs will be accumulated in saturatedPackArguments, so we don't have an + // assertion for that. + LUAU_ASSERT(typesProvided <= typesRequired); + + // If we didn't provide enough types, but we did provide a type pack, we + // don't want to use defaults. The rationale for this is that if the user + // provides a pack but doesn't provide enough types, we want to report an + // error, rather than simply using the default saturatedTypeArguments, if they exist. If + // they did provide enough types, but not enough packs, we of course want to + // use the default packs. + bool needsDefaults = (typesProvided < typesRequired && packsProvided == 0) || (typesProvided == typesRequired && packsProvided < packsRequired); + + if (needsDefaults) + { + // Default types can reference earlier types. It's legal to write + // something like + // type T = (A, B) -> number + // and we need to respect that. We use an ApplyTypeFunction for this. + ApplyTypeFunction atf{arena}; + + for (size_t i = 0; i < typesProvided; ++i) + atf.typeArguments[fn.typeParams[i].ty] = saturatedTypeArguments[i]; + + for (size_t i = typesProvided; i < typesRequired; ++i) + { + TypeId defaultTy = fn.typeParams[i].defaultValue.value_or(nullptr); + + // We will fill this in with the error type later. + if (!defaultTy) + break; + + TypeId instantiatedDefault = atf.substitute(defaultTy).value_or(getSingletonTypes().errorRecoveryType()); + atf.typeArguments[fn.typeParams[i].ty] = instantiatedDefault; + saturatedTypeArguments.push_back(instantiatedDefault); + } + + for (size_t i = 0; i < packsProvided; ++i) + { + atf.typePackArguments[fn.typePackParams[i].tp] = saturatedPackArguments[i]; + } + + for (size_t i = packsProvided; i < packsRequired; ++i) + { + TypePackId defaultTp = fn.typePackParams[i].defaultValue.value_or(nullptr); + + // We will fill this in with the error type pack later. + if (!defaultTp) + break; + + TypePackId instantiatedDefault = atf.substitute(defaultTp).value_or(getSingletonTypes().errorRecoveryTypePack()); + atf.typePackArguments[fn.typePackParams[i].tp] = instantiatedDefault; + saturatedPackArguments.push_back(instantiatedDefault); + } + } + + // If we didn't create an extra type pack from overflowing parameter packs, + // and we're still missing a type pack, plug in an empty type pack as the + // value of the empty packs. + if (extraTypes.empty() && saturatedPackArguments.size() + 1 == fn.typePackParams.size()) + { + saturatedPackArguments.push_back(arena->addTypePack({})); + } + + // We need to have _something_ when we substitute the generic saturatedTypeArguments, + // even if they're missing, so we use the error type as a filler. + for (size_t i = saturatedTypeArguments.size(); i < typesRequired; ++i) + { + saturatedTypeArguments.push_back(getSingletonTypes().errorRecoveryType()); + } + + for (size_t i = saturatedPackArguments.size(); i < packsRequired; ++i) + { + saturatedPackArguments.push_back(getSingletonTypes().errorRecoveryTypePack()); + } + + // At this point, these two conditions should be true. If they aren't we + // will run into access violations. + LUAU_ASSERT(saturatedTypeArguments.size() == fn.typeParams.size()); + LUAU_ASSERT(saturatedPackArguments.size() == fn.typePackParams.size()); + + return {saturatedTypeArguments, saturatedPackArguments}; +} + +bool InstantiationSignature::operator==(const InstantiationSignature& rhs) const +{ + return fn == rhs.fn && arguments == rhs.arguments && packArguments == rhs.packArguments; +} + +size_t HashInstantiationSignature::operator()(const InstantiationSignature& signature) const +{ + size_t hash = std::hash{}(signature.fn.type); + for (const GenericTypeDefinition& p : signature.fn.typeParams) + { + hash ^= (std::hash{}(p.ty) << 1); + } + + for (const GenericTypePackDefinition& p : signature.fn.typePackParams) + { + hash ^= (std::hash{}(p.tp) << 1); + } + + for (const TypeId a : signature.arguments) + { + hash ^= (std::hash{}(a) << 1); + } + + for (const TypePackId a : signature.packArguments) + { + hash ^= (std::hash{}(a) << 1); + } + + return hash; +} + void dump(NotNull rootScope, ToStringOptions& opts) { printf("constraints:\n"); @@ -77,6 +243,7 @@ void ConstraintSolver::run() return; ToStringOptions opts; + opts.exhaustive = true; if (FFlag::DebugLuauLogSolver) { @@ -186,6 +353,8 @@ bool ConstraintSolver::tryDispatch(NotNull constraint, bool fo success = tryDispatch(*bc, constraint, force); else if (auto nc = get(*constraint)) success = tryDispatch(*nc, constraint); + else if (auto taec = get(*constraint)) + success = tryDispatch(*taec, constraint); else LUAU_ASSERT(0); @@ -325,6 +494,198 @@ bool ConstraintSolver::tryDispatch(const NameConstraint& c, NotNullarena); + + if (follow(petv.fn.type) == follow(signature.fn.type) && (signature.arguments != typeArguments || signature.packArguments != packArguments)) + { + foundInfiniteType = true; + return false; + } + + return true; + } +}; + +struct InstantiationQueuer : TypeVarOnceVisitor +{ + ConstraintSolver* solver; + const InstantiationSignature& signature; + + explicit InstantiationQueuer(ConstraintSolver* solver, const InstantiationSignature& signature) + : solver(solver) + , signature(signature) + { + } + + bool visit(TypeId ty, const PendingExpansionTypeVar& petv) override + { + solver->pushConstraint(TypeAliasExpansionConstraint{ty}); + return false; + } +}; + +bool ConstraintSolver::tryDispatch(const TypeAliasExpansionConstraint& c, NotNull constraint) +{ + const PendingExpansionTypeVar* petv = get(follow(c.target)); + if (!petv) + { + unblock(c.target); + return true; + } + + auto bindResult = [this, &c](TypeId result) { + asMutable(c.target)->ty.emplace(result); + unblock(c.target); + }; + + // If there are no parameters to the type function we can just use the type + // directly. + if (petv->fn.typeParams.empty() && petv->fn.typePackParams.empty()) + { + bindResult(petv->fn.type); + return true; + } + + auto [typeArguments, packArguments] = saturateArguments(petv->fn, petv->typeArguments, petv->packArguments, arena); + + bool sameTypes = + std::equal(typeArguments.begin(), typeArguments.end(), petv->fn.typeParams.begin(), petv->fn.typeParams.end(), [](auto&& itp, auto&& p) { + return itp == p.ty; + }); + + bool samePacks = std::equal( + packArguments.begin(), packArguments.end(), petv->fn.typePackParams.begin(), petv->fn.typePackParams.end(), [](auto&& itp, auto&& p) { + return itp == p.tp; + }); + + // If we're instantiating the type with its generic saturatedTypeArguments we are + // performing the identity substitution. We can just short-circuit and bind + // to the TypeFun's type. + if (sameTypes && samePacks) + { + bindResult(petv->fn.type); + return true; + } + + InstantiationSignature signature{ + petv->fn, + typeArguments, + packArguments, + }; + + // If we use the same signature, we don't need to bother trying to + // instantiate the alias again, since the instantiation should be + // deterministic. + if (TypeId* cached = instantiatedAliases.find(signature)) + { + bindResult(*cached); + return true; + } + + // In order to prevent infinite types from being expanded and causing us to + // cycle infinitely, we need to scan the type function for cases where we + // expand the same alias with different type saturatedTypeArguments. See + // https://github.com/Roblox/luau/pull/68 for the RFC responsible for this. + // This is a little nicer than using a recursion limit because we can catch + // the infinite expansion before actually trying to expand it. + InfiniteTypeFinder itf{this, signature}; + itf.traverse(petv->fn.type); + + if (itf.foundInfiniteType) + { + // TODO (CLI-56761): Report an error. + bindResult(getSingletonTypes().errorRecoveryType()); + return true; + } + + ApplyTypeFunction applyTypeFunction{arena}; + for (size_t i = 0; i < typeArguments.size(); ++i) + { + applyTypeFunction.typeArguments[petv->fn.typeParams[i].ty] = typeArguments[i]; + } + + for (size_t i = 0; i < packArguments.size(); ++i) + { + applyTypeFunction.typePackArguments[petv->fn.typePackParams[i].tp] = packArguments[i]; + } + + std::optional maybeInstantiated = applyTypeFunction.substitute(petv->fn.type); + // Note that ApplyTypeFunction::encounteredForwardedType is never set in + // DCR, because we do not use free types for forward-declared generic + // aliases. + + if (!maybeInstantiated.has_value()) + { + // TODO (CLI-56761): Report an error. + bindResult(getSingletonTypes().errorRecoveryType()); + return true; + } + + TypeId instantiated = *maybeInstantiated; + TypeId target = follow(instantiated); + // Type function application will happily give us the exact same type if + // there are e.g. generic saturatedTypeArguments that go unused. + bool needsClone = follow(petv->fn.type) == target; + // Only tables have the properties we're trying to set. + TableTypeVar* ttv = getMutableTableType(target); + + if (ttv) + { + if (needsClone) + { + // Substitution::clone is a shallow clone. If this is a + // metatable type, we want to mutate its table, so we need to + // explicitly clone that table as well. If we don't, we will + // mutate another module's type surface and cause a + // use-after-free. + if (get(target)) + { + instantiated = applyTypeFunction.clone(target); + MetatableTypeVar* mtv = getMutable(instantiated); + mtv->table = applyTypeFunction.clone(mtv->table); + ttv = getMutable(mtv->table); + } + else if (get(target)) + { + instantiated = applyTypeFunction.clone(target); + ttv = getMutable(instantiated); + } + + target = follow(instantiated); + } + + ttv->instantiatedTypeParams = typeArguments; + ttv->instantiatedTypePackParams = packArguments; + // TODO: Fill in definitionModuleName. + } + + bindResult(target); + + // The application is not recursive, so we need to queue up application of + // any child type function instantiations within the result in order for it + // to be complete. + InstantiationQueuer queuer{this, signature}; + queuer.traverse(target); + + instantiatedAliases[signature] = target; + + return true; +} + void ConstraintSolver::block_(BlockedConstraintId target, NotNull constraint) { blocked[target].push_back(constraint); @@ -388,7 +749,7 @@ void ConstraintSolver::unblock(TypePackId progressed) bool ConstraintSolver::isBlocked(TypeId ty) { - return nullptr != get(follow(ty)); + return nullptr != get(follow(ty)) || nullptr != get(follow(ty)); } bool ConstraintSolver::isBlocked(NotNull constraint) @@ -415,4 +776,12 @@ void ConstraintSolver::unify(TypePackId subPack, TypePackId superPack) u.log.commit(); } +void ConstraintSolver::pushConstraint(ConstraintV cv) +{ + std::unique_ptr c = std::make_unique(std::move(cv)); + NotNull borrow = NotNull(c.get()); + solverConstraints.push_back(std::move(c)); + unsolvedConstraints.push_back(borrow); +} + } // namespace Luau diff --git a/Analysis/src/ConstraintSolverLogger.cpp b/Analysis/src/ConstraintSolverLogger.cpp index 0c2517c0..adb9c54e 100644 --- a/Analysis/src/ConstraintSolverLogger.cpp +++ b/Analysis/src/ConstraintSolverLogger.cpp @@ -2,45 +2,39 @@ #include "Luau/ConstraintSolverLogger.h" +#include "Luau/JsonEmitter.h" + namespace Luau { -static std::string dumpScopeAndChildren(const Scope* scope, ToStringOptions& opts) +static void dumpScopeAndChildren(const Scope* scope, Json::JsonEmitter& emitter, ToStringOptions& opts) { - std::string output = "{\"bindings\":{"; + emitter.writeRaw("{"); + Json::write(emitter, "bindings"); + emitter.writeRaw(":"); + + Json::ObjectEmitter o = emitter.writeObject(); - bool comma = false; for (const auto& [name, binding] : scope->bindings) { - if (comma) - output += ","; - - output += "\""; - output += name.c_str(); - output += "\": \""; - ToStringResult result = toStringDetailed(binding.typeId, opts); opts.nameMap = std::move(result.nameMap); - output += result.name; - output += "\""; - - comma = true; + o.writePair(name.c_str(), result.name); } - output += "},\"children\":["; - comma = false; + o.finish(); + emitter.writeRaw(","); + Json::write(emitter, "children"); + emitter.writeRaw(":"); + Json::ArrayEmitter a = emitter.writeArray(); for (const Scope* child : scope->children) { - if (comma) - output += ","; - - output += dumpScopeAndChildren(child, opts); - comma = true; + dumpScopeAndChildren(child, emitter, opts); } - output += "]}"; - return output; + a.finish(); + emitter.writeRaw("}"); } static std::string dumpConstraintsToDot(std::vector>& constraints, ToStringOptions& opts) @@ -80,51 +74,49 @@ static std::string dumpConstraintsToDot(std::vector>& std::string ConstraintSolverLogger::compileOutput() { - std::string output = "["; - bool comma = false; - + Json::JsonEmitter emitter; + emitter.writeRaw("["); for (const std::string& snapshot : snapshots) { - if (comma) - output += ","; - output += snapshot; - - comma = true; + emitter.writeComma(); + emitter.writeRaw(snapshot); } - output += "]"; - return output; + emitter.writeRaw("]"); + return emitter.str(); } void ConstraintSolverLogger::captureBoundarySnapshot(const Scope* rootScope, std::vector>& unsolvedConstraints) { - std::string snapshot = "{\"type\":\"boundary\",\"rootScope\":"; + Json::JsonEmitter emitter; + Json::ObjectEmitter o = emitter.writeObject(); + o.writePair("type", "boundary"); + o.writePair("constraintGraph", dumpConstraintsToDot(unsolvedConstraints, opts)); + emitter.writeComma(); + Json::write(emitter, "rootScope"); + emitter.writeRaw(":"); + dumpScopeAndChildren(rootScope, emitter, opts); + o.finish(); - snapshot += dumpScopeAndChildren(rootScope, opts); - snapshot += ",\"constraintGraph\":\""; - snapshot += dumpConstraintsToDot(unsolvedConstraints, opts); - snapshot += "\"}"; - - snapshots.push_back(std::move(snapshot)); + snapshots.push_back(emitter.str()); } void ConstraintSolverLogger::prepareStepSnapshot( const Scope* rootScope, NotNull current, std::vector>& unsolvedConstraints) { - // LUAU_ASSERT(!preparedSnapshot); + Json::JsonEmitter emitter; + Json::ObjectEmitter o = emitter.writeObject(); + o.writePair("type", "step"); + o.writePair("constraintGraph", dumpConstraintsToDot(unsolvedConstraints, opts)); + o.writePair("currentId", std::to_string(reinterpret_cast(current.get()))); + o.writePair("current", toString(*current, opts)); + emitter.writeComma(); + Json::write(emitter, "rootScope"); + emitter.writeRaw(":"); + dumpScopeAndChildren(rootScope, emitter, opts); + o.finish(); - std::string snapshot = "{\"type\":\"step\",\"rootScope\":"; - - snapshot += dumpScopeAndChildren(rootScope, opts); - snapshot += ",\"constraintGraph\":\""; - snapshot += dumpConstraintsToDot(unsolvedConstraints, opts); - snapshot += "\",\"currentId\":\""; - snapshot += std::to_string(reinterpret_cast(current.get())); - snapshot += "\",\"current\":\""; - snapshot += toString(*current, opts); - snapshot += "\"}"; - - preparedSnapshot = std::move(snapshot); + preparedSnapshot = emitter.str(); } void ConstraintSolverLogger::commitPreparedStepSnapshot() diff --git a/Analysis/src/Frontend.cpp b/Analysis/src/Frontend.cpp index 222a17df..fe65853d 100644 --- a/Analysis/src/Frontend.cpp +++ b/Analysis/src/Frontend.cpp @@ -77,6 +77,58 @@ static void generateDocumentationSymbols(TypeId ty, const std::string& rootName) } } +LoadDefinitionFileResult Frontend::loadDefinitionFile(std::string_view source, const std::string& packageName) +{ + if (!FFlag::DebugLuauDeferredConstraintResolution) + return Luau::loadDefinitionFile(typeChecker, typeChecker.globalScope, source, packageName); + + LUAU_TIMETRACE_SCOPE("loadDefinitionFile", "Frontend"); + + Luau::Allocator allocator; + Luau::AstNameTable names(allocator); + + ParseOptions options; + options.allowDeclarationSyntax = true; + + Luau::ParseResult parseResult = Luau::Parser::parse(source.data(), source.size(), names, allocator, options); + + if (parseResult.errors.size() > 0) + return LoadDefinitionFileResult{false, parseResult, nullptr}; + + Luau::SourceModule module; + module.root = parseResult.root; + module.mode = Mode::Definition; + + ModulePtr checkedModule = check(module, Mode::Definition, globalScope); + + if (checkedModule->errors.size() > 0) + return LoadDefinitionFileResult{false, parseResult, checkedModule}; + + CloneState cloneState; + + for (const auto& [name, ty] : checkedModule->declaredGlobals) + { + TypeId globalTy = clone(ty, globalTypes, cloneState); + std::string documentationSymbol = packageName + "/global/" + name; + generateDocumentationSymbols(globalTy, documentationSymbol); + globalScope->bindings[typeChecker.globalNames.names->getOrAdd(name.c_str())] = {globalTy, Location(), false, {}, documentationSymbol}; + + persist(globalTy); + } + + for (const auto& [name, ty] : checkedModule->getModuleScope()->exportedTypeBindings) + { + TypeFun globalTy = clone(ty, globalTypes, cloneState); + std::string documentationSymbol = packageName + "/globaltype/" + name; + generateDocumentationSymbols(globalTy.type, documentationSymbol); + globalScope->exportedTypeBindings[name] = globalTy; + + persist(globalTy.type); + } + + return LoadDefinitionFileResult{true, parseResult, checkedModule}; +} + LoadDefinitionFileResult loadDefinitionFile(TypeChecker& typeChecker, ScopePtr targetScope, std::string_view source, const std::string& packageName) { LUAU_TIMETRACE_SCOPE("loadDefinitionFile", "Frontend"); @@ -770,14 +822,7 @@ NotNull Frontend::getGlobalScope() { if (!globalScope) { - const SingletonTypes& singletonTypes = getSingletonTypes(); - - globalScope = std::make_unique(singletonTypes.anyTypePack); - globalScope->typeBindings["nil"] = singletonTypes.nilType; - globalScope->typeBindings["number"] = singletonTypes.numberType; - globalScope->typeBindings["string"] = singletonTypes.stringType; - globalScope->typeBindings["boolean"] = singletonTypes.booleanType; - globalScope->typeBindings["thread"] = singletonTypes.threadType; + globalScope = typeChecker.globalScope; } return NotNull(globalScope.get()); diff --git a/Analysis/src/Instantiation.cpp b/Analysis/src/Instantiation.cpp index 77c62422..1a6013af 100644 --- a/Analysis/src/Instantiation.cpp +++ b/Analysis/src/Instantiation.cpp @@ -4,6 +4,8 @@ #include "Luau/TxnLog.h" #include "Luau/TypeArena.h" +LUAU_FASTFLAG(LuauClassTypeVarsInSubstitution) + namespace Luau { @@ -31,6 +33,8 @@ bool Instantiation::ignoreChildren(TypeId ty) { if (log->getMutable(ty)) return true; + else if (FFlag::LuauClassTypeVarsInSubstitution && get(ty)) + return true; else return false; } diff --git a/Analysis/src/JsonEmitter.cpp b/Analysis/src/JsonEmitter.cpp new file mode 100644 index 00000000..e99619ba --- /dev/null +++ b/Analysis/src/JsonEmitter.cpp @@ -0,0 +1,220 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +#include "Luau/JsonEmitter.h" + +#include "Luau/StringUtils.h" + +#include + +namespace Luau::Json +{ + +static constexpr int CHUNK_SIZE = 1024; + +ObjectEmitter::ObjectEmitter(NotNull emitter) + : emitter(emitter), finished(false) +{ + comma = emitter->pushComma(); + emitter->writeRaw('{'); +} + +ObjectEmitter::~ObjectEmitter() +{ + finish(); +} + +void ObjectEmitter::finish() +{ + if (finished) + return; + + emitter->writeRaw('}'); + emitter->popComma(comma); + finished = true; +} + +ArrayEmitter::ArrayEmitter(NotNull emitter) + : emitter(emitter), finished(false) +{ + comma = emitter->pushComma(); + emitter->writeRaw('['); +} + +ArrayEmitter::~ArrayEmitter() +{ + finish(); +} + +void ArrayEmitter::finish() +{ + if (finished) + return; + + emitter->writeRaw(']'); + emitter->popComma(comma); + finished = true; +} + +JsonEmitter::JsonEmitter() +{ + newChunk(); +} + +std::string JsonEmitter::str() +{ + return join(chunks, ""); +} + +bool JsonEmitter::pushComma() +{ + bool current = comma; + comma = false; + return current; +} + +void JsonEmitter::popComma(bool c) +{ + comma = c; +} + +void JsonEmitter::writeRaw(std::string_view sv) +{ + if (sv.size() > CHUNK_SIZE) + { + chunks.emplace_back(sv); + newChunk(); + return; + } + + auto& chunk = chunks.back(); + if (chunk.size() + sv.size() < CHUNK_SIZE) + { + chunk.append(sv.data(), sv.size()); + return; + } + + size_t prefix = CHUNK_SIZE - chunk.size(); + chunk.append(sv.data(), prefix); + newChunk(); + + chunks.back().append(sv.data() + prefix, sv.size() - prefix); +} + +void JsonEmitter::writeRaw(char c) +{ + writeRaw(std::string_view{&c, 1}); +} + +void write(JsonEmitter& emitter, bool b) +{ + if (b) + emitter.writeRaw("true"); + else + emitter.writeRaw("false"); +} + +void write(JsonEmitter& emitter, double d) +{ + emitter.writeRaw(std::to_string(d)); +} + +void write(JsonEmitter& emitter, int i) +{ + emitter.writeRaw(std::to_string(i)); +} + +void write(JsonEmitter& emitter, long i) +{ + emitter.writeRaw(std::to_string(i)); +} + +void write(JsonEmitter& emitter, long long i) +{ + emitter.writeRaw(std::to_string(i)); +} + +void write(JsonEmitter& emitter, unsigned int i) +{ + emitter.writeRaw(std::to_string(i)); +} + +void write(JsonEmitter& emitter, unsigned long i) +{ + emitter.writeRaw(std::to_string(i)); +} + +void write(JsonEmitter& emitter, unsigned long long i) +{ + emitter.writeRaw(std::to_string(i)); +} + +void write(JsonEmitter& emitter, std::string_view sv) +{ + emitter.writeRaw('\"'); + + for (char c : sv) + { + if (c == '"') + emitter.writeRaw("\\\""); + else if (c == '\\') + emitter.writeRaw("\\\\"); + else if (c == '\n') + emitter.writeRaw("\\n"); + else if (c < ' ') + emitter.writeRaw(format("\\u%04x", c)); + else + emitter.writeRaw(c); + } + + emitter.writeRaw('\"'); +} + +void write(JsonEmitter& emitter, char c) +{ + write(emitter, std::string_view{&c, 1}); +} + +void write(JsonEmitter& emitter, const char* str) +{ + write(emitter, std::string_view{str, strlen(str)}); +} + +void write(JsonEmitter& emitter, const std::string& str) +{ + write(emitter, std::string_view{str}); +} + +void write(JsonEmitter& emitter, std::nullptr_t) +{ + emitter.writeRaw("null"); +} + +void write(JsonEmitter& emitter, std::nullopt_t) +{ + emitter.writeRaw("null"); +} + +void JsonEmitter::writeComma() +{ + if (comma) + writeRaw(','); + else + comma = true; +} + +ObjectEmitter JsonEmitter::writeObject() +{ + return ObjectEmitter{NotNull(this)}; +} + +ArrayEmitter JsonEmitter::writeArray() +{ + return ArrayEmitter{NotNull(this)}; +} + +void JsonEmitter::newChunk() +{ + chunks.emplace_back(); + chunks.back().reserve(CHUNK_SIZE); +} + +} // namespace Luau::Json diff --git a/Analysis/src/Linter.cpp b/Analysis/src/Linter.cpp index 6e9e57f2..9fce79a2 100644 --- a/Analysis/src/Linter.cpp +++ b/Analysis/src/Linter.cpp @@ -48,6 +48,7 @@ static const char* kWarningNames[] = { "DuplicateCondition", "MisleadingAndOr", "CommentDirective", + "IntegerParsing", }; // clang-format on @@ -2589,6 +2590,45 @@ private: } }; +class LintIntegerParsing : AstVisitor +{ +public: + LUAU_NOINLINE static void process(LintContext& context) + { + LintIntegerParsing pass; + pass.context = &context; + + context.root->visit(&pass); + } + +private: + LintContext* context; + + bool visit(AstExprConstantNumber* node) override + { + switch (node->parseResult) + { + case ConstantNumberParseResult::Ok: + case ConstantNumberParseResult::Malformed: + break; + case ConstantNumberParseResult::BinOverflow: + emitWarning(*context, LintWarning::Code_IntegerParsing, node->location, + "Binary number literal exceeded available precision and has been truncated to 2^64"); + break; + case ConstantNumberParseResult::HexOverflow: + emitWarning(*context, LintWarning::Code_IntegerParsing, node->location, + "Hexadecimal number literal exceeded available precision and has been truncated to 2^64"); + break; + case ConstantNumberParseResult::DoublePrefix: + emitWarning(*context, LintWarning::Code_IntegerParsing, node->location, + "Hexadecimal number literal has a double prefix, which will fail to parse in the future; remove the extra 0x to fix"); + break; + } + + return true; + } +}; + static void fillBuiltinGlobals(LintContext& context, const AstNameTable& names, const ScopePtr& env) { ScopePtr current = env; @@ -2810,6 +2850,9 @@ std::vector lint(AstStat* root, const AstNameTable& names, const Sc if (context.warningEnabled(LintWarning::Code_CommentDirective)) lintComments(context, hotcomments); + if (context.warningEnabled(LintWarning::Code_IntegerParsing)) + LintIntegerParsing::process(context); + std::sort(context.result.begin(), context.result.end(), WarningComparator()); return context.result; diff --git a/Analysis/src/Module.cpp b/Analysis/src/Module.cpp index dca18027..2b46da87 100644 --- a/Analysis/src/Module.cpp +++ b/Analysis/src/Module.cpp @@ -15,7 +15,6 @@ #include LUAU_FASTFLAG(LuauLowerBoundsCalculation); -LUAU_FASTFLAG(LuauNormalizeFlagIsConservative); LUAU_FASTFLAG(DebugLuauDeferredConstraintResolution); LUAU_FASTFLAGVARIABLE(LuauForceExportSurfacesToBeNormal, false); @@ -140,20 +139,17 @@ void Module::clonePublicInterface(InternalErrorReporter& ice) { normalize(tf.type, interfaceTypes, ice); - if (FFlag::LuauNormalizeFlagIsConservative) + // We're about to freeze the memory. We know that the flag is conservative by design. Cyclic tables + // won't be marked normal. If the types aren't normal by now, they never will be. + forceNormal.traverse(tf.type); + for (GenericTypeDefinition param : tf.typeParams) { - // We're about to freeze the memory. We know that the flag is conservative by design. Cyclic tables - // won't be marked normal. If the types aren't normal by now, they never will be. - forceNormal.traverse(tf.type); - for (GenericTypeDefinition param : tf.typeParams) - { - forceNormal.traverse(param.ty); + forceNormal.traverse(param.ty); - if (param.defaultValue) - { - normalize(*param.defaultValue, interfaceTypes, ice); - forceNormal.traverse(*param.defaultValue); - } + if (param.defaultValue) + { + normalize(*param.defaultValue, interfaceTypes, ice); + forceNormal.traverse(*param.defaultValue); } } } diff --git a/Analysis/src/Normalize.cpp b/Analysis/src/Normalize.cpp index a96b557d..9ae3b404 100644 --- a/Analysis/src/Normalize.cpp +++ b/Analysis/src/Normalize.cpp @@ -13,7 +13,6 @@ LUAU_FASTFLAGVARIABLE(DebugLuauCopyBeforeNormalizing, false) // This could theoretically be 2000 on amd64, but x86 requires this. LUAU_FASTINTVARIABLE(LuauNormalizeIterationLimit, 1200); LUAU_FASTFLAGVARIABLE(LuauNormalizeCombineTableFix, false); -LUAU_FASTFLAGVARIABLE(LuauNormalizeFlagIsConservative, false); LUAU_FASTFLAGVARIABLE(LuauFixNormalizationOfCyclicUnions, false); LUAU_FASTFLAG(LuauUnknownAndNeverType) LUAU_FASTFLAG(LuauQuantifyConstrained) @@ -89,13 +88,7 @@ static bool areNormal_(const T& t, const std::unordered_set& seen, Intern if (count >= FInt::LuauNormalizeIterationLimit) ice.ice("Luau::areNormal hit iteration limit"); - if (FFlag::LuauNormalizeFlagIsConservative) - return ty->normal; - else - { - // The follow is here because a bound type may not be normal, but the bound type is normal. - return ty->normal || follow(ty)->normal || seen.find(asMutable(ty)) != seen.end(); - } + return ty->normal; }; return std::all_of(begin(t), end(t), isNormal); diff --git a/Analysis/src/Quantify.cpp b/Analysis/src/Quantify.cpp index 72eb4012..03049cca 100644 --- a/Analysis/src/Quantify.cpp +++ b/Analysis/src/Quantify.cpp @@ -7,7 +7,6 @@ #include "Luau/TxnLog.h" #include "Luau/VisitTypeVar.h" -LUAU_FASTFLAG(LuauAlwaysQuantify); LUAU_FASTFLAG(DebugLuauSharedSelf) LUAU_FASTFLAG(DebugLuauDeferredConstraintResolution); LUAU_FASTFLAGVARIABLE(LuauQuantifyConstrained, false) @@ -203,16 +202,8 @@ void quantify(TypeId ty, TypeLevel level) FunctionTypeVar* ftv = getMutable(ty); LUAU_ASSERT(ftv); - if (FFlag::LuauAlwaysQuantify) - { - ftv->generics.insert(ftv->generics.end(), q.generics.begin(), q.generics.end()); - ftv->genericPacks.insert(ftv->genericPacks.end(), q.genericPacks.begin(), q.genericPacks.end()); - } - else - { - ftv->generics = q.generics; - ftv->genericPacks = q.genericPacks; - } + ftv->generics.insert(ftv->generics.end(), q.generics.begin(), q.generics.end()); + ftv->genericPacks.insert(ftv->genericPacks.end(), q.genericPacks.begin(), q.genericPacks.end()); } } @@ -223,16 +214,8 @@ void quantify(TypeId ty, Scope* scope) FunctionTypeVar* ftv = getMutable(ty); LUAU_ASSERT(ftv); - if (FFlag::LuauAlwaysQuantify) - { - ftv->generics.insert(ftv->generics.end(), q.generics.begin(), q.generics.end()); - ftv->genericPacks.insert(ftv->genericPacks.end(), q.genericPacks.begin(), q.genericPacks.end()); - } - else - { - ftv->generics = q.generics; - ftv->genericPacks = q.genericPacks; - } + ftv->generics.insert(ftv->generics.end(), q.generics.begin(), q.generics.end()); + ftv->genericPacks.insert(ftv->genericPacks.end(), q.genericPacks.begin(), q.genericPacks.end()); if (ftv->generics.empty() && ftv->genericPacks.empty() && !q.seenMutableType && !q.seenGenericType) ftv->hasNoGenerics = true; diff --git a/Analysis/src/Scope.cpp b/Analysis/src/Scope.cpp index 083aa085..bee16908 100644 --- a/Analysis/src/Scope.cpp +++ b/Analysis/src/Scope.cpp @@ -122,7 +122,7 @@ std::optional Scope::lookup(Symbol sym) } } -std::optional Scope::lookupTypeBinding(const Name& name) +std::optional Scope::lookupTypeBinding(const Name& name) { Scope* s = this; while (s) diff --git a/Analysis/src/Substitution.cpp b/Analysis/src/Substitution.cpp index 7245403c..148c9ee2 100644 --- a/Analysis/src/Substitution.cpp +++ b/Analysis/src/Substitution.cpp @@ -9,9 +9,12 @@ #include LUAU_FASTFLAGVARIABLE(LuauAnyificationMustClone, false) +LUAU_FASTFLAGVARIABLE(LuauSubstitutionFixMissingFields, false) LUAU_FASTFLAG(LuauLowerBoundsCalculation) LUAU_FASTINTVARIABLE(LuauTarjanChildLimit, 10000) +LUAU_FASTFLAGVARIABLE(LuauClassTypeVarsInSubstitution, false) LUAU_FASTFLAG(LuauUnknownAndNeverType) +LUAU_FASTFLAGVARIABLE(LuauSubstitutionReentrant, false) namespace Luau { @@ -28,6 +31,14 @@ void Tarjan::visitChildren(TypeId ty, int index) if (const FunctionTypeVar* ftv = get(ty)) { + if (FFlag::LuauSubstitutionFixMissingFields) + { + for (TypeId generic : ftv->generics) + visitChild(generic); + for (TypePackId genericPack : ftv->genericPacks) + visitChild(genericPack); + } + visitChild(ftv->argTypes); visitChild(ftv->retTypes); } @@ -68,6 +79,25 @@ void Tarjan::visitChildren(TypeId ty, int index) for (TypeId part : ctv->parts) visitChild(part); } + else if (const PendingExpansionTypeVar* petv = get(ty)) + { + for (TypeId a : petv->typeArguments) + visitChild(a); + + for (TypePackId a : petv->packArguments) + visitChild(a); + } + else if (const ClassTypeVar* ctv = get(ty); FFlag::LuauClassTypeVarsInSubstitution && ctv) + { + for (auto [name, prop] : ctv->props) + visitChild(prop.type); + + if (ctv->parent) + visitChild(*ctv->parent); + + if (ctv->metatable) + visitChild(*ctv->metatable); + } } void Tarjan::visitChildren(TypePackId tp, int index) @@ -267,6 +297,24 @@ TarjanResult Tarjan::visitRoot(TypePackId tp) return loop(); } +void FindDirty::clearTarjan() +{ + dirty.clear(); + + typeToIndex.clear(); + packToIndex.clear(); + indexToType.clear(); + indexToPack.clear(); + + stack.clear(); + onStack.clear(); + lowlink.clear(); + + edgesTy.clear(); + edgesTp.clear(); + worklist.clear(); +} + bool FindDirty::getDirty(int index) { if (dirty.size() <= size_t(index)) @@ -330,16 +378,46 @@ std::optional Substitution::substitute(TypeId ty) { ty = log->follow(ty); + // clear algorithm state for reentrancy + if (FFlag::LuauSubstitutionReentrant) + clearTarjan(); + auto result = findDirty(ty); if (result != TarjanResult::Ok) return std::nullopt; for (auto [oldTy, newTy] : newTypes) - if (!ignoreChildren(oldTy)) - replaceChildren(newTy); + { + if (FFlag::LuauSubstitutionReentrant) + { + if (!ignoreChildren(oldTy) && !replacedTypes.contains(newTy)) + { + replaceChildren(newTy); + replacedTypes.insert(newTy); + } + } + else + { + if (!ignoreChildren(oldTy)) + replaceChildren(newTy); + } + } for (auto [oldTp, newTp] : newPacks) - if (!ignoreChildren(oldTp)) - replaceChildren(newTp); + { + if (FFlag::LuauSubstitutionReentrant) + { + if (!ignoreChildren(oldTp) && !replacedTypePacks.contains(newTp)) + { + replaceChildren(newTp); + replacedTypePacks.insert(newTp); + } + } + else + { + if (!ignoreChildren(oldTp)) + replaceChildren(newTp); + } + } TypeId newTy = replace(ty); return newTy; } @@ -348,16 +426,46 @@ std::optional Substitution::substitute(TypePackId tp) { tp = log->follow(tp); + // clear algorithm state for reentrancy + if (FFlag::LuauSubstitutionReentrant) + clearTarjan(); + auto result = findDirty(tp); if (result != TarjanResult::Ok) return std::nullopt; for (auto [oldTy, newTy] : newTypes) - if (!ignoreChildren(oldTy)) - replaceChildren(newTy); + { + if (FFlag::LuauSubstitutionReentrant) + { + if (!ignoreChildren(oldTy) && !replacedTypes.contains(newTy)) + { + replaceChildren(newTy); + replacedTypes.insert(newTy); + } + } + else + { + if (!ignoreChildren(oldTy)) + replaceChildren(newTy); + } + } for (auto [oldTp, newTp] : newPacks) - if (!ignoreChildren(oldTp)) - replaceChildren(newTp); + { + if (FFlag::LuauSubstitutionReentrant) + { + if (!ignoreChildren(oldTp) && !replacedTypePacks.contains(newTp)) + { + replaceChildren(newTp); + replacedTypePacks.insert(newTp); + } + } + else + { + if (!ignoreChildren(oldTp)) + replaceChildren(newTp); + } + } TypePackId newTp = replace(tp); return newTp; } @@ -385,6 +493,8 @@ TypePackId Substitution::clone(TypePackId tp) { VariadicTypePack clone; clone.ty = vtp->ty; + if (FFlag::LuauSubstitutionFixMissingFields) + clone.hidden = vtp->hidden; return addTypePack(std::move(clone)); } else @@ -395,6 +505,9 @@ void Substitution::foundDirty(TypeId ty) { ty = log->follow(ty); + if (FFlag::LuauSubstitutionReentrant && newTypes.contains(ty)) + return; + if (isDirty(ty)) newTypes[ty] = follow(clean(ty)); else @@ -405,6 +518,9 @@ void Substitution::foundDirty(TypePackId tp) { tp = log->follow(tp); + if (FFlag::LuauSubstitutionReentrant && newPacks.contains(tp)) + return; + if (isDirty(tp)) newPacks[tp] = follow(clean(tp)); else @@ -446,6 +562,14 @@ void Substitution::replaceChildren(TypeId ty) if (FunctionTypeVar* ftv = getMutable(ty)) { + if (FFlag::LuauSubstitutionFixMissingFields) + { + for (TypeId& generic : ftv->generics) + generic = replace(generic); + for (TypePackId& genericPack : ftv->genericPacks) + genericPack = replace(genericPack); + } + ftv->argTypes = replace(ftv->argTypes); ftv->retTypes = replace(ftv->retTypes); } @@ -486,6 +610,25 @@ void Substitution::replaceChildren(TypeId ty) for (TypeId& part : ctv->parts) part = replace(part); } + else if (PendingExpansionTypeVar* petv = getMutable(ty)) + { + for (TypeId& a : petv->typeArguments) + a = replace(a); + + for (TypePackId& a : petv->packArguments) + a = replace(a); + } + else if (ClassTypeVar* ctv = getMutable(ty); FFlag::LuauClassTypeVarsInSubstitution && ctv) + { + for (auto& [name, prop] : ctv->props) + prop.type = replace(prop.type); + + if (ctv->parent) + ctv->parent = replace(*ctv->parent); + + if (ctv->metatable) + ctv->metatable = replace(*ctv->metatable); + } } void Substitution::replaceChildren(TypePackId tp) diff --git a/Analysis/src/ToString.cpp b/Analysis/src/ToString.cpp index f1dc9211..79d2e125 100644 --- a/Analysis/src/ToString.cpp +++ b/Analysis/src/ToString.cpp @@ -232,6 +232,11 @@ struct StringifierState emit(std::to_string(i).c_str()); } + void emit(size_t i) + { + emit(std::to_string(i).c_str()); + } + void indent() { indentation += 4; @@ -410,6 +415,13 @@ struct TypeVarStringifier state.emit("*"); } + void operator()(TypeId ty, const PendingExpansionTypeVar& petv) + { + state.emit("*pending-expansion-"); + state.emit(petv.index); + state.emit("*"); + } + void operator()(TypeId, const PrimitiveTypeVar& ptv) { switch (ptv.type) @@ -1459,6 +1471,12 @@ std::string toString(const Constraint& constraint, ToStringOptions& opts) opts.nameMap = std::move(namedStr.nameMap); return "@name(" + namedStr.name + ") = " + c.name; } + else if constexpr (std::is_same_v) + { + ToStringResult targetStr = toStringDetailed(c.target, opts); + opts.nameMap = std::move(targetStr.nameMap); + return "expand " + targetStr.name; + } else static_assert(always_false_v, "Non-exhaustive constraint switch"); }; diff --git a/Analysis/src/TypeAttach.cpp b/Analysis/src/TypeAttach.cpp index 2bc89cf6..f21a4fa9 100644 --- a/Analysis/src/TypeAttach.cpp +++ b/Analysis/src/TypeAttach.cpp @@ -99,6 +99,11 @@ public: return allocator->alloc(Location(), std::nullopt, AstName("*blocked*")); } + AstType* operator()(const PendingExpansionTypeVar& petv) + { + return allocator->alloc(Location(), std::nullopt, AstName("*pending-expansion*")); + } + AstType* operator()(const ConstrainedTypeVar& ctv) { AstArray types; diff --git a/Analysis/src/TypeChecker2.cpp b/Analysis/src/TypeChecker2.cpp index 85749671..53b069cf 100644 --- a/Analysis/src/TypeChecker2.cpp +++ b/Analysis/src/TypeChecker2.cpp @@ -67,6 +67,13 @@ struct TypeChecker2 : public AstVisitor return follow(*ty); } + TypePackId lookupPackAnnotation(AstTypePack* annotation) + { + TypePackId* tp = module->astResolvedTypePacks.find(annotation); + LUAU_ASSERT(tp); + return follow(*tp); + } + TypePackId reconstructPack(AstArray exprs, TypeArena& arena) { if (exprs.size == 0) @@ -363,12 +370,153 @@ struct TypeChecker2 : public AstVisitor bool visit(AstTypeReference* ty) override { Scope* scope = findInnermostScope(ty->location); + LUAU_ASSERT(scope); // TODO: Imported types - // TODO: Generic types - if (!scope->lookupTypeBinding(ty->name.value)) + + std::optional alias = scope->lookupTypeBinding(ty->name.value); + + if (alias.has_value()) { - reportError(UnknownSymbol{ty->name.value, UnknownSymbol::Context::Type}, ty->location); + size_t typesRequired = alias->typeParams.size(); + size_t packsRequired = alias->typePackParams.size(); + + bool hasDefaultTypes = std::any_of(alias->typeParams.begin(), alias->typeParams.end(), [](auto&& el) { + return el.defaultValue.has_value(); + }); + + bool hasDefaultPacks = std::any_of(alias->typePackParams.begin(), alias->typePackParams.end(), [](auto&& el) { + return el.defaultValue.has_value(); + }); + + if (!ty->hasParameterList) + { + if ((!alias->typeParams.empty() && !hasDefaultTypes) || (!alias->typePackParams.empty() && !hasDefaultPacks)) + { + reportError(GenericError{"Type parameter list is required"}, ty->location); + } + } + + size_t typesProvided = 0; + size_t extraTypes = 0; + size_t packsProvided = 0; + + for (const AstTypeOrPack& p : ty->parameters) + { + if (p.type) + { + if (packsProvided != 0) + { + reportError(GenericError{"Type parameters must come before type pack parameters"}, ty->location); + } + + if (typesProvided < typesRequired) + { + typesProvided += 1; + } + else + { + extraTypes += 1; + } + } + else if (p.typePack) + { + TypePackId tp = lookupPackAnnotation(p.typePack); + + if (typesProvided < typesRequired && size(tp) == 1 && finite(tp) && first(tp)) + { + typesProvided += 1; + } + else + { + packsProvided += 1; + } + } + } + + if (extraTypes != 0 && packsProvided == 0) + { + packsProvided += 1; + } + + for (size_t i = typesProvided; i < typesRequired; ++i) + { + if (alias->typeParams[i].defaultValue) + { + typesProvided += 1; + } + } + + for (size_t i = packsProvided; i < packsProvided; ++i) + { + if (alias->typePackParams[i].defaultValue) + { + packsProvided += 1; + } + } + + if (extraTypes == 0 && packsProvided + 1 == packsRequired) + { + packsProvided += 1; + } + + if (typesProvided != typesRequired || packsProvided != packsRequired) + { + reportError(IncorrectGenericParameterCount{ + /* name */ ty->name.value, + /* typeFun */ *alias, + /* actualParameters */ typesProvided, + /* actualPackParameters */ packsProvided, + }, + ty->location); + } + } + else + { + if (scope->lookupTypePackBinding(ty->name.value)) + { + reportError( + SwappedGenericTypeParameter{ + ty->name.value, + SwappedGenericTypeParameter::Kind::Type, + }, + ty->location); + } + else + { + reportError(UnknownSymbol{ty->name.value, UnknownSymbol::Context::Type}, ty->location); + } + } + + return true; + } + + bool visit(AstTypePack*) override + { + return true; + } + + bool visit(AstTypePackGeneric* tp) override + { + Scope* scope = findInnermostScope(tp->location); + LUAU_ASSERT(scope); + + std::optional alias = scope->lookupTypePackBinding(tp->genericName.value); + if (!alias.has_value()) + { + if (scope->lookupTypeBinding(tp->genericName.value)) + { + reportError( + SwappedGenericTypeParameter{ + tp->genericName.value, + SwappedGenericTypeParameter::Kind::Pack, + }, + tp->location); + } + else + { + reportError(UnknownSymbol{tp->genericName.value, UnknownSymbol::Context::Type}, tp->location); + } } return true; diff --git a/Analysis/src/TypeInfer.cpp b/Analysis/src/TypeInfer.cpp index 371ef77a..bdda195c 100644 --- a/Analysis/src/TypeInfer.cpp +++ b/Analysis/src/TypeInfer.cpp @@ -1,6 +1,7 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #include "Luau/TypeInfer.h" +#include "Luau/ApplyTypeFunction.h" #include "Luau/Clone.h" #include "Luau/Common.h" #include "Luau/Instantiation.h" @@ -36,11 +37,8 @@ LUAU_FASTFLAGVARIABLE(LuauIndexSilenceErrors, false) LUAU_FASTFLAGVARIABLE(LuauLowerBoundsCalculation, false) LUAU_FASTFLAGVARIABLE(DebugLuauFreezeDuringUnification, false) LUAU_FASTFLAGVARIABLE(LuauSelfCallAutocompleteFix3, false) -LUAU_FASTFLAGVARIABLE(LuauReduceUnionRecursion, false) LUAU_FASTFLAGVARIABLE(LuauReturnAnyInsteadOfICE, false) // Eventually removed as false. -LUAU_FASTFLAG(LuauNormalizeFlagIsConservative) LUAU_FASTFLAGVARIABLE(DebugLuauSharedSelf, false); -LUAU_FASTFLAGVARIABLE(LuauAlwaysQuantify, false); LUAU_FASTFLAGVARIABLE(LuauReportErrorsOnIndexerKeyMismatch, false) LUAU_FASTFLAGVARIABLE(LuauUnknownAndNeverType, false) LUAU_FASTFLAG(LuauQuantifyConstrained) @@ -2108,35 +2106,16 @@ std::vector TypeChecker::reduceUnion(const std::vector& types) if (const UnionTypeVar* utv = get(t)) { - if (FFlag::LuauReduceUnionRecursion) + for (TypeId ty : utv) { - for (TypeId ty : utv) - { - if (FFlag::LuauNormalizeFlagIsConservative) - ty = follow(ty); - if (get(ty)) - continue; - if (get(ty) || get(ty)) - return {ty}; + ty = follow(ty); + if (get(ty)) + continue; + if (get(ty) || get(ty)) + return {ty}; - if (result.end() == std::find(result.begin(), result.end(), ty)) - result.push_back(ty); - } - } - else - { - std::vector r = reduceUnion(utv->options); - for (TypeId ty : r) - { - ty = follow(ty); - if (get(ty)) - continue; - if (get(ty) || get(ty)) - return {ty}; - - if (std::find(result.begin(), result.end(), ty) == result.end()) - result.push_back(ty); - } + if (result.end() == std::find(result.begin(), result.end(), ty)) + result.push_back(ty); } } else if (std::find(result.begin(), result.end(), t) == result.end()) @@ -4770,16 +4749,8 @@ TypeId TypeChecker::quantify(const ScopePtr& scope, TypeId ty, Location location { const FunctionTypeVar* ftv = get(ty); - if (FFlag::LuauAlwaysQuantify) - { - if (ftv) - Luau::quantify(ty, scope->level); - } - else - { - if (ftv && ftv->generics.empty() && ftv->genericPacks.empty()) - Luau::quantify(ty, scope->level); - } + if (ftv) + Luau::quantify(ty, scope->level); if (FFlag::LuauLowerBoundsCalculation && ftv) { @@ -5253,7 +5224,7 @@ TypeId TypeChecker::resolveTypeWorker(const ScopePtr& scope, const AstType& anno if (notEnoughParameters && hasDefaultParameters) { // 'applyTypeFunction' is used to substitute default types that reference previous generic types - ApplyTypeFunction applyTypeFunction{¤tModule->internalTypes, scope->level}; + ApplyTypeFunction applyTypeFunction{¤tModule->internalTypes}; for (size_t i = 0; i < typesProvided; ++i) applyTypeFunction.typeArguments[tf->typeParams[i].ty] = typeParams[i]; @@ -5494,65 +5465,13 @@ TypePackId TypeChecker::resolveTypePack(const ScopePtr& scope, const AstTypePack return result; } -bool ApplyTypeFunction::isDirty(TypeId ty) -{ - if (typeArguments.count(ty)) - return true; - else if (const FreeTypeVar* ftv = get(ty)) - { - if (ftv->forwardedTypeAlias) - encounteredForwardedType = true; - return false; - } - else - return false; -} - -bool ApplyTypeFunction::isDirty(TypePackId tp) -{ - if (typePackArguments.count(tp)) - return true; - else - return false; -} - -bool ApplyTypeFunction::ignoreChildren(TypeId ty) -{ - if (get(ty)) - return true; - else - return false; -} - -bool ApplyTypeFunction::ignoreChildren(TypePackId tp) -{ - if (get(tp)) - return true; - else - return false; -} - -TypeId ApplyTypeFunction::clean(TypeId ty) -{ - TypeId& arg = typeArguments[ty]; - LUAU_ASSERT(arg); - return arg; -} - -TypePackId ApplyTypeFunction::clean(TypePackId tp) -{ - TypePackId& arg = typePackArguments[tp]; - LUAU_ASSERT(arg); - return arg; -} - TypeId TypeChecker::instantiateTypeFun(const ScopePtr& scope, const TypeFun& tf, const std::vector& typeParams, const std::vector& typePackParams, const Location& location) { if (tf.typeParams.empty() && tf.typePackParams.empty()) return tf.type; - ApplyTypeFunction applyTypeFunction{¤tModule->internalTypes, scope->level}; + ApplyTypeFunction applyTypeFunction{¤tModule->internalTypes}; for (size_t i = 0; i < tf.typeParams.size(); ++i) applyTypeFunction.typeArguments[tf.typeParams[i].ty] = typeParams[i]; diff --git a/Analysis/src/TypeVar.cpp b/Analysis/src/TypeVar.cpp index 51517db9..ada2b012 100644 --- a/Analysis/src/TypeVar.cpp +++ b/Analysis/src/TypeVar.cpp @@ -445,6 +445,16 @@ BlockedTypeVar::BlockedTypeVar() int BlockedTypeVar::nextIndex = 0; +PendingExpansionTypeVar::PendingExpansionTypeVar(TypeFun fn, std::vector typeArguments, std::vector packArguments) + : fn(fn) + , typeArguments(typeArguments) + , packArguments(packArguments) + , index(++nextIndex) +{ +} + +size_t PendingExpansionTypeVar::nextIndex = 0; + FunctionTypeVar::FunctionTypeVar(TypePackId argTypes, TypePackId retTypes, std::optional defn, bool hasSelf) : argTypes(argTypes) , retTypes(retTypes) @@ -1412,4 +1422,19 @@ bool hasTag(const Property& prop, const std::string& tagName) return hasTag(prop.tags, tagName); } +bool TypeFun::operator==(const TypeFun& rhs) const +{ + return type == rhs.type && typeParams == rhs.typeParams && typePackParams == rhs.typePackParams; +} + +bool GenericTypeDefinition::operator==(const GenericTypeDefinition& rhs) const +{ + return ty == rhs.ty && defaultValue == rhs.defaultValue; +} + +bool GenericTypePackDefinition::operator==(const GenericTypePackDefinition& rhs) const +{ + return tp == rhs.tp && defaultValue == rhs.defaultValue; +} + } // namespace Luau diff --git a/Ast/include/Luau/Ast.h b/Ast/include/Luau/Ast.h index 6f39e3fd..1e164d04 100644 --- a/Ast/include/Luau/Ast.h +++ b/Ast/include/Luau/Ast.h @@ -474,16 +474,26 @@ public: bool value; }; +enum class ConstantNumberParseResult +{ + Ok, + Malformed, + BinOverflow, + HexOverflow, + DoublePrefix, +}; + class AstExprConstantNumber : public AstExpr { public: LUAU_RTTI(AstExprConstantNumber) - AstExprConstantNumber(const Location& location, double value); + AstExprConstantNumber(const Location& location, double value, ConstantNumberParseResult parseResult = ConstantNumberParseResult::Ok); void visit(AstVisitor* visitor) override; double value; + ConstantNumberParseResult parseResult; }; class AstExprConstantString : public AstExpr diff --git a/Ast/src/Ast.cpp b/Ast/src/Ast.cpp index 24a280da..3066b756 100644 --- a/Ast/src/Ast.cpp +++ b/Ast/src/Ast.cpp @@ -50,9 +50,10 @@ void AstExprConstantBool::visit(AstVisitor* visitor) visitor->visit(this); } -AstExprConstantNumber::AstExprConstantNumber(const Location& location, double value) +AstExprConstantNumber::AstExprConstantNumber(const Location& location, double value, ConstantNumberParseResult parseResult) : AstExpr(ClassIndex(), location) , value(value) + , parseResult(parseResult) { } diff --git a/Ast/src/Parser.cpp b/Ast/src/Parser.cpp index 8d54e367..1eb9565f 100644 --- a/Ast/src/Parser.cpp +++ b/Ast/src/Parser.cpp @@ -21,7 +21,8 @@ LUAU_DYNAMIC_FASTFLAGVARIABLE(LuaReportParseWrongNamedType, false) bool lua_telemetry_parsed_named_non_function_type = false; -LUAU_FASTFLAGVARIABLE(LuauErrorParseIntegerIssues, false) +LUAU_FASTFLAGVARIABLE(LuauErrorDoubleHexPrefix, false) +LUAU_FASTFLAGVARIABLE(LuauLintParseIntegerIssues, false) LUAU_DYNAMIC_FASTFLAGVARIABLE(LuaReportParseIntegerIssues, false) bool lua_telemetry_parsed_out_of_range_bin_integer = false; @@ -2032,8 +2033,10 @@ AstExpr* Parser::parseAssertionExpr() return expr; } -static const char* parseInteger(double& result, const char* data, int base) +static const char* parseInteger_DEPRECATED(double& result, const char* data, int base) { + LUAU_ASSERT(!FFlag::LuauLintParseIntegerIssues); + char* end = nullptr; unsigned long long value = strtoull(data, &end, base); @@ -2053,9 +2056,6 @@ static const char* parseInteger(double& result, const char* data, int base) else lua_telemetry_parsed_out_of_range_hex_integer = true; } - - if (FFlag::LuauErrorParseIntegerIssues) - return "Integer number value is out of range"; } } @@ -2063,11 +2063,13 @@ static const char* parseInteger(double& result, const char* data, int base) return *end == 0 ? nullptr : "Malformed number"; } -static const char* parseNumber(double& result, const char* data) +static const char* parseNumber_DEPRECATED2(double& result, const char* data) { + LUAU_ASSERT(!FFlag::LuauLintParseIntegerIssues); + // binary literal if (data[0] == '0' && (data[1] == 'b' || data[1] == 'B') && data[2]) - return parseInteger(result, data + 2, 2); + return parseInteger_DEPRECATED(result, data + 2, 2); // hexadecimal literal if (data[0] == '0' && (data[1] == 'x' || data[1] == 'X') && data[2]) @@ -2075,10 +2077,7 @@ static const char* parseNumber(double& result, const char* data) if (DFFlag::LuaReportParseIntegerIssues && data[2] == '0' && (data[3] == 'x' || data[3] == 'X')) lua_telemetry_parsed_double_prefix_hex_integer = true; - if (FFlag::LuauErrorParseIntegerIssues) - return parseInteger(result, data, 16); // keep prefix, it's handled by 'strtoull' - else - return parseInteger(result, data + 2, 16); + return parseInteger_DEPRECATED(result, data + 2, 16); } char* end = nullptr; @@ -2090,6 +2089,8 @@ static const char* parseNumber(double& result, const char* data) static bool parseNumber_DEPRECATED(double& result, const char* data) { + LUAU_ASSERT(!FFlag::LuauLintParseIntegerIssues); + // binary literal if (data[0] == '0' && (data[1] == 'b' || data[1] == 'B') && data[2]) { @@ -2118,6 +2119,73 @@ static bool parseNumber_DEPRECATED(double& result, const char* data) } } +static ConstantNumberParseResult parseInteger(double& result, const char* data, int base) +{ + LUAU_ASSERT(FFlag::LuauLintParseIntegerIssues); + LUAU_ASSERT(base == 2 || base == 16); + + char* end = nullptr; + unsigned long long value = strtoull(data, &end, base); + + if (*end != 0) + return ConstantNumberParseResult::Malformed; + + result = double(value); + + if (value == ULLONG_MAX && errno == ERANGE) + { + // 'errno' might have been set before we called 'strtoull', but we don't want the overhead of resetting a TLS variable on each call + // so we only reset it when we get a result that might be an out-of-range error and parse again to make sure + errno = 0; + value = strtoull(data, &end, base); + + if (errno == ERANGE) + { + if (DFFlag::LuaReportParseIntegerIssues) + { + if (base == 2) + lua_telemetry_parsed_out_of_range_bin_integer = true; + else + lua_telemetry_parsed_out_of_range_hex_integer = true; + } + + return base == 2 ? ConstantNumberParseResult::BinOverflow : ConstantNumberParseResult::HexOverflow; + } + } + + return ConstantNumberParseResult::Ok; +} + +static ConstantNumberParseResult parseNumber(double& result, const char* data) +{ + LUAU_ASSERT(FFlag::LuauLintParseIntegerIssues); + + // binary literal + if (data[0] == '0' && (data[1] == 'b' || data[1] == 'B') && data[2]) + return parseInteger(result, data + 2, 2); + + // hexadecimal literal + if (data[0] == '0' && (data[1] == 'x' || data[1] == 'X') && data[2]) + { + if (!FFlag::LuauErrorDoubleHexPrefix && data[2] == '0' && (data[3] == 'x' || data[3] == 'X')) + { + if (DFFlag::LuaReportParseIntegerIssues) + lua_telemetry_parsed_double_prefix_hex_integer = true; + + ConstantNumberParseResult parseResult = parseInteger(result, data + 2, 16); + return parseResult == ConstantNumberParseResult::Malformed ? parseResult : ConstantNumberParseResult::DoublePrefix; + } + + return parseInteger(result, data, 16); // pass in '0x' prefix, it's handled by 'strtoull' + } + + char* end = nullptr; + double value = strtod(data, &end); + + result = value; + return *end == 0 ? ConstantNumberParseResult::Ok : ConstantNumberParseResult::Malformed; +} + // simpleexp -> NUMBER | STRING | NIL | true | false | ... | constructor | FUNCTION body | primaryexp AstExpr* Parser::parseSimpleExpr() { @@ -2158,10 +2226,21 @@ AstExpr* Parser::parseSimpleExpr() scratchData.erase(std::remove(scratchData.begin(), scratchData.end(), '_'), scratchData.end()); } - if (DFFlag::LuaReportParseIntegerIssues || FFlag::LuauErrorParseIntegerIssues) + if (FFlag::LuauLintParseIntegerIssues) { double value = 0; - if (const char* error = parseNumber(value, scratchData.c_str())) + ConstantNumberParseResult result = parseNumber(value, scratchData.c_str()); + nextLexeme(); + + if (result == ConstantNumberParseResult::Malformed) + return reportExprError(start, {}, "Malformed number"); + + return allocator.alloc(start, value, result); + } + else if (DFFlag::LuaReportParseIntegerIssues) + { + double value = 0; + if (const char* error = parseNumber_DEPRECATED2(value, scratchData.c_str())) { nextLexeme(); diff --git a/CLI/Ast.cpp b/CLI/Ast.cpp index 6ee608c4..fd99d225 100644 --- a/CLI/Ast.cpp +++ b/CLI/Ast.cpp @@ -3,7 +3,7 @@ #include "Luau/Common.h" #include "Luau/Ast.h" -#include "Luau/JsonEncoder.h" +#include "Luau/AstJsonEncoder.h" #include "Luau/Parser.h" #include "Luau/ParseOptions.h" diff --git a/CLI/Repl.cpp b/CLI/Repl.cpp index 13033bb4..4d3beec9 100644 --- a/CLI/Repl.cpp +++ b/CLI/Repl.cpp @@ -109,8 +109,8 @@ static int lua_loadstring(lua_State* L) return 1; lua_pushnil(L); - lua_insert(L, -2); /* put before error message */ - return 2; /* return nil plus error message */ + lua_insert(L, -2); // put before error message + return 2; // return nil plus error message } static int finishrequire(lua_State* L) diff --git a/Common/include/Luau/Bytecode.h b/Common/include/Luau/Bytecode.h index 0cb7e1d9..1d6b18e5 100644 --- a/Common/include/Luau/Bytecode.h +++ b/Common/include/Luau/Bytecode.h @@ -37,6 +37,14 @@ // Note that Luau runtime doesn't provide indefinite bytecode compatibility: support for older versions gets removed over time. As such, bytecode isn't a durable storage format and it's expected // that Luau users can recompile bytecode from source on Luau version upgrades if necessary. +// # Bytecode version history +// +// Note: due to limitations of the versioning scheme, some bytecode blobs that carry version 2 are using features from version 3. Starting from version 3, version should be sufficient to indicate bytecode compatibility. +// +// Version 1: Baseline version for the open-source release. Supported until 0.521. +// Version 2: Adds Proto::linedefined. Currently supported. +// Version 3: Adds FORGPREP/JUMPXEQK* and enhances AUX encoding for FORGLOOP. Removes FORGLOOP_NEXT/INEXT and JUMPIFEQK/JUMPIFNOTEQK. Currently supported. + // Bytecode opcode, part of the instruction header enum LuauOpcode { @@ -367,6 +375,20 @@ enum LuauOpcode // D: jump offset (-32768..32767) LOP_FORGPREP, + // JUMPXEQKNIL, JUMPXEQKB: jumps to target offset if the comparison with constant is true (or false, see AUX) + // A: source register 1 + // D: jump offset (-32768..32767; 0 means "next instruction" aka "don't jump") + // AUX: constant value (for boolean) in low bit, NOT flag (that flips comparison result) in high bit + LOP_JUMPXEQKNIL, + LOP_JUMPXEQKB, + + // JUMPXEQKN, JUMPXEQKS: jumps to target offset if the comparison with constant is true (or false, see AUX) + // A: source register 1 + // D: jump offset (-32768..32767; 0 means "next instruction" aka "don't jump") + // AUX: constant table index in low 24 bits, NOT flag (that flips comparison result) in high bit + LOP_JUMPXEQKN, + LOP_JUMPXEQKS, + // Enum entry for number of opcodes, not a valid opcode by itself! LOP__COUNT }; @@ -391,7 +413,7 @@ enum LuauBytecodeTag { // Bytecode version; runtime supports [MIN, MAX], compiler emits TARGET by default but may emit a higher version when flags are enabled LBC_VERSION_MIN = 2, - LBC_VERSION_MAX = 2, + LBC_VERSION_MAX = 3, LBC_VERSION_TARGET = 2, // Types of constant table entries LBC_CONSTANT_NIL = 0, diff --git a/Common/include/Luau/Common.h b/Common/include/Luau/Common.h index fbb03a9e..f1846ac3 100644 --- a/Common/include/Luau/Common.h +++ b/Common/include/Luau/Common.h @@ -20,12 +20,6 @@ #define LUAU_DEBUGBREAK() __builtin_trap() #endif - - - - - - namespace Luau { @@ -67,16 +61,13 @@ struct FValue const char* name; FValue* next; - FValue(const char* name, T def, bool dynamic, void (*reg)(const char*, T*, bool) = nullptr) + FValue(const char* name, T def, bool dynamic) : value(def) , dynamic(dynamic) , name(name) , next(list) { list = this; - - if (reg) - reg(name, &value, dynamic); } operator T() const @@ -98,7 +89,7 @@ FValue* FValue::list = nullptr; #define LUAU_FASTFLAGVARIABLE(flag, def) \ namespace FFlag \ { \ - Luau::FValue flag(#flag, def, false, nullptr); \ + Luau::FValue flag(#flag, def, false); \ } #define LUAU_FASTINT(flag) \ namespace FInt \ @@ -108,7 +99,7 @@ FValue* FValue::list = nullptr; #define LUAU_FASTINTVARIABLE(flag, def) \ namespace FInt \ { \ - Luau::FValue flag(#flag, def, false, nullptr); \ + Luau::FValue flag(#flag, def, false); \ } #define LUAU_DYNAMIC_FASTFLAG(flag) \ @@ -119,7 +110,7 @@ FValue* FValue::list = nullptr; #define LUAU_DYNAMIC_FASTFLAGVARIABLE(flag, def) \ namespace DFFlag \ { \ - Luau::FValue flag(#flag, def, true, nullptr); \ + Luau::FValue flag(#flag, def, true); \ } #define LUAU_DYNAMIC_FASTINT(flag) \ namespace DFInt \ @@ -129,5 +120,5 @@ FValue* FValue::list = nullptr; #define LUAU_DYNAMIC_FASTINTVARIABLE(flag, def) \ namespace DFInt \ { \ - Luau::FValue flag(#flag, def, true, nullptr); \ + Luau::FValue flag(#flag, def, true); \ } diff --git a/Common/include/Luau/ExperimentalFlags.h b/Common/include/Luau/ExperimentalFlags.h index 3525259d..71e76ffe 100644 --- a/Common/include/Luau/ExperimentalFlags.h +++ b/Common/include/Luau/ExperimentalFlags.h @@ -10,17 +10,16 @@ inline bool isFlagExperimental(const char* flag) { // Flags in this list are disabled by default in various command-line tools. They may have behavior that is not fully final, // or critical bugs that are found after the code has been submitted. - static const char* kList[] = - { + static const char* kList[] = { "LuauLowerBoundsCalculation", nullptr, // makes sure we always have at least one entry }; - for (const char* item: kList) + for (const char* item : kList) if (item && strcmp(item, flag) == 0) return true; return false; } -} +} // namespace Luau diff --git a/Compiler/include/luacode.h b/Compiler/include/luacode.h index e235a2e7..5f69f69e 100644 --- a/Compiler/include/luacode.h +++ b/Compiler/include/luacode.h @@ -3,7 +3,7 @@ #include -/* Can be used to reconfigure visibility/exports for public APIs */ +// Can be used to reconfigure visibility/exports for public APIs #ifndef LUACODE_API #define LUACODE_API extern #endif @@ -35,5 +35,5 @@ struct lua_CompileOptions const char** mutableGlobals; }; -/* compile source to bytecode; when source compilation fails, the resulting bytecode contains the encoded error. use free() to destroy */ +// compile source to bytecode; when source compilation fails, the resulting bytecode contains the encoded error. use free() to destroy LUACODE_API char* luau_compile(const char* source, size_t size, lua_CompileOptions* options, size_t* outsize); diff --git a/Compiler/src/Builtins.cpp b/Compiler/src/Builtins.cpp index 3650c147..26933730 100644 --- a/Compiler/src/Builtins.cpp +++ b/Compiler/src/Builtins.cpp @@ -4,8 +4,6 @@ #include "Luau/Bytecode.h" #include "Luau/Compiler.h" -LUAU_FASTFLAGVARIABLE(LuauCompileRawlen, false) - namespace Luau { namespace Compile @@ -57,7 +55,7 @@ static int getBuiltinFunctionId(const Builtin& builtin, const CompileOptions& op return LBF_RAWGET; if (builtin.isGlobal("rawequal")) return LBF_RAWEQUAL; - if (FFlag::LuauCompileRawlen && builtin.isGlobal("rawlen")) + if (builtin.isGlobal("rawlen")) return LBF_RAWLEN; if (builtin.isGlobal("unpack")) diff --git a/Compiler/src/BytecodeBuilder.cpp b/Compiler/src/BytecodeBuilder.cpp index 64327bd0..46ab2648 100644 --- a/Compiler/src/BytecodeBuilder.cpp +++ b/Compiler/src/BytecodeBuilder.cpp @@ -6,6 +6,8 @@ #include #include +LUAU_FASTFLAGVARIABLE(LuauCompileBytecodeV3, false) + namespace Luau { @@ -77,6 +79,10 @@ static int getOpLength(LuauOpcode op) case LOP_JUMPIFNOTEQK: case LOP_FASTCALL2: case LOP_FASTCALL2K: + case LOP_JUMPXEQKNIL: + case LOP_JUMPXEQKB: + case LOP_JUMPXEQKN: + case LOP_JUMPXEQKS: return 2; default: @@ -108,6 +114,10 @@ inline bool isJumpD(LuauOpcode op) case LOP_JUMPBACK: case LOP_JUMPIFEQK: case LOP_JUMPIFNOTEQK: + case LOP_JUMPXEQKNIL: + case LOP_JUMPXEQKB: + case LOP_JUMPXEQKN: + case LOP_JUMPXEQKS: return true; default: @@ -1069,6 +1079,9 @@ std::string BytecodeBuilder::getError(const std::string& message) uint8_t BytecodeBuilder::getVersion() { + if (FFlag::LuauCompileBytecodeV3) + return 3; + // This function usually returns LBC_VERSION_TARGET but may sometimes return a higher number (within LBC_VERSION_MIN/MAX) under fast flags return LBC_VERSION_TARGET; } @@ -1246,6 +1259,24 @@ void BytecodeBuilder::validate() const VJUMP(LUAU_INSN_D(insn)); break; + case LOP_JUMPXEQKNIL: + case LOP_JUMPXEQKB: + VREG(LUAU_INSN_A(insn)); + VJUMP(LUAU_INSN_D(insn)); + break; + + case LOP_JUMPXEQKN: + VREG(LUAU_INSN_A(insn)); + VCONST(insns[i + 1] & 0xffffff, Number); + VJUMP(LUAU_INSN_D(insn)); + break; + + case LOP_JUMPXEQKS: + VREG(LUAU_INSN_A(insn)); + VCONST(insns[i + 1] & 0xffffff, String); + VJUMP(LUAU_INSN_D(insn)); + break; + case LOP_ADD: case LOP_SUB: case LOP_MUL: @@ -1779,6 +1810,26 @@ void BytecodeBuilder::dumpInstruction(const uint32_t* code, std::string& result, formatAppend(result, "JUMPIFNOTEQK R%d K%d L%d\n", LUAU_INSN_A(insn), *code++, targetLabel); break; + case LOP_JUMPXEQKNIL: + formatAppend(result, "JUMPXEQKNIL R%d L%d%s\n", LUAU_INSN_A(insn), targetLabel, *code >> 31 ? " NOT" : ""); + code++; + break; + + case LOP_JUMPXEQKB: + formatAppend(result, "JUMPXEQKB R%d %d L%d%s\n", LUAU_INSN_A(insn), *code & 1, targetLabel, *code >> 31 ? " NOT" : ""); + code++; + break; + + case LOP_JUMPXEQKN: + formatAppend(result, "JUMPXEQKN R%d K%d L%d%s\n", LUAU_INSN_A(insn), *code & 0xffffff, targetLabel, *code >> 31 ? " NOT" : ""); + code++; + break; + + case LOP_JUMPXEQKS: + formatAppend(result, "JUMPXEQKS R%d K%d L%d%s\n", LUAU_INSN_A(insn), *code & 0xffffff, targetLabel, *code >> 31 ? " NOT" : ""); + code++; + break; + default: LUAU_ASSERT(!"Unsupported opcode"); } diff --git a/Compiler/src/Compiler.cpp b/Compiler/src/Compiler.cpp index 4be54378..2ee20cab 100644 --- a/Compiler/src/Compiler.cpp +++ b/Compiler/src/Compiler.cpp @@ -26,6 +26,7 @@ LUAU_FASTINTVARIABLE(LuauCompileInlineDepth, 5) LUAU_FASTFLAGVARIABLE(LuauCompileNoIpairs, false) LUAU_FASTFLAGVARIABLE(LuauCompileFreeReassign, false) +LUAU_FASTFLAGVARIABLE(LuauCompileXEQ, false) namespace Luau { @@ -1008,9 +1009,8 @@ struct Compiler size_t compileCompareJump(AstExprBinary* expr, bool not_ = false) { RegScope rs(this); - LuauOpcode opc = getJumpOpCompare(expr->op, not_); - bool isEq = (opc == LOP_JUMPIFEQ || opc == LOP_JUMPIFNOTEQ); + bool isEq = (expr->op == AstExprBinary::CompareEq || expr->op == AstExprBinary::CompareNe); AstExpr* left = expr->left; AstExpr* right = expr->right; @@ -1022,36 +1022,112 @@ struct Compiler std::swap(left, right); } - uint8_t rl = compileExprAuto(left, rs); - int32_t rr = -1; - - if (isEq && operandIsConstant) + if (FFlag::LuauCompileXEQ) { - if (opc == LOP_JUMPIFEQ) - opc = LOP_JUMPIFEQK; - else if (opc == LOP_JUMPIFNOTEQ) - opc = LOP_JUMPIFNOTEQK; + uint8_t rl = compileExprAuto(left, rs); - rr = getConstantIndex(right); - LUAU_ASSERT(rr >= 0); - } - else - rr = compileExprAuto(right, rs); + if (isEq && operandIsConstant) + { + const Constant* cv = constants.find(right); + LUAU_ASSERT(cv && cv->type != Constant::Type_Unknown); - size_t jumpLabel = bytecode.emitLabel(); + LuauOpcode opc = LOP_NOP; + int32_t cid = -1; + uint32_t flip = (expr->op == AstExprBinary::CompareEq) == not_ ? 0x80000000 : 0; - if (expr->op == AstExprBinary::CompareGt || expr->op == AstExprBinary::CompareGe) - { - bytecode.emitAD(opc, uint8_t(rr), 0); - bytecode.emitAux(rl); + switch (cv->type) + { + case Constant::Type_Nil: + opc = LOP_JUMPXEQKNIL; + cid = 0; + break; + + case Constant::Type_Boolean: + opc = LOP_JUMPXEQKB; + cid = cv->valueBoolean; + break; + + case Constant::Type_Number: + opc = LOP_JUMPXEQKN; + cid = getConstantIndex(right); + break; + + case Constant::Type_String: + opc = LOP_JUMPXEQKS; + cid = getConstantIndex(right); + break; + + default: + LUAU_ASSERT(!"Unexpected constant type"); + } + + if (cid < 0) + CompileError::raise(expr->location, "Exceeded constant limit; simplify the code to compile"); + + size_t jumpLabel = bytecode.emitLabel(); + + bytecode.emitAD(opc, rl, 0); + bytecode.emitAux(cid | flip); + + return jumpLabel; + } + else + { + LuauOpcode opc = getJumpOpCompare(expr->op, not_); + + uint8_t rr = compileExprAuto(right, rs); + + size_t jumpLabel = bytecode.emitLabel(); + + if (expr->op == AstExprBinary::CompareGt || expr->op == AstExprBinary::CompareGe) + { + bytecode.emitAD(opc, rr, 0); + bytecode.emitAux(rl); + } + else + { + bytecode.emitAD(opc, rl, 0); + bytecode.emitAux(rr); + } + + return jumpLabel; + } } else { - bytecode.emitAD(opc, rl, 0); - bytecode.emitAux(rr); - } + LuauOpcode opc = getJumpOpCompare(expr->op, not_); - return jumpLabel; + uint8_t rl = compileExprAuto(left, rs); + int32_t rr = -1; + + if (isEq && operandIsConstant) + { + if (opc == LOP_JUMPIFEQ) + opc = LOP_JUMPIFEQK; + else if (opc == LOP_JUMPIFNOTEQ) + opc = LOP_JUMPIFNOTEQK; + + rr = getConstantIndex(right); + LUAU_ASSERT(rr >= 0); + } + else + rr = compileExprAuto(right, rs); + + size_t jumpLabel = bytecode.emitLabel(); + + if (expr->op == AstExprBinary::CompareGt || expr->op == AstExprBinary::CompareGe) + { + bytecode.emitAD(opc, uint8_t(rr), 0); + bytecode.emitAux(rl); + } + else + { + bytecode.emitAD(opc, rl, 0); + bytecode.emitAux(rr); + } + + return jumpLabel; + } } int32_t getConstantNumber(AstExpr* node) diff --git a/Makefile b/Makefile index 33a2e5e1..dd32d237 100644 --- a/Makefile +++ b/Makefile @@ -55,6 +55,7 @@ ifneq ($(opt),) endif OBJECTS=$(AST_OBJECTS) $(COMPILER_OBJECTS) $(ANALYSIS_OBJECTS) $(CODEGEN_OBJECTS) $(VM_OBJECTS) $(ISOCLINE_OBJECTS) $(TESTS_OBJECTS) $(CLI_OBJECTS) $(FUZZ_OBJECTS) +EXECUTABLE_ALIASES = luau luau-analyze luau-tests # common flags CXXFLAGS=-g -Wall @@ -121,14 +122,14 @@ fuzz-proto fuzz-prototest: LDFLAGS+=build/libprotobuf-mutator/src/libfuzzer/libp all: $(REPL_CLI_TARGET) $(ANALYZE_CLI_TARGET) $(TESTS_TARGET) aliases -aliases: luau luau-analyze +aliases: $(EXECUTABLE_ALIASES) test: $(TESTS_TARGET) $(TESTS_TARGET) $(TESTS_ARGS) clean: rm -rf $(BUILD) - rm -rf luau luau-analyze + rm -rf $(EXECUTABLE_ALIASES) coverage: $(TESTS_TARGET) $(TESTS_TARGET) --fflags=true @@ -154,6 +155,9 @@ luau: $(REPL_CLI_TARGET) luau-analyze: $(ANALYZE_CLI_TARGET) ln -fs $^ $@ +luau-tests: $(TESTS_TARGET) + ln -fs $^ $@ + # executable targets $(TESTS_TARGET): $(TESTS_OBJECTS) $(ANALYSIS_TARGET) $(COMPILER_TARGET) $(AST_TARGET) $(CODEGEN_TARGET) $(VM_TARGET) $(ISOCLINE_TARGET) $(REPL_CLI_TARGET): $(REPL_CLI_OBJECTS) $(COMPILER_TARGET) $(AST_TARGET) $(VM_TARGET) $(ISOCLINE_TARGET) diff --git a/Sources.cmake b/Sources.cmake index f745667f..9a6019a9 100644 --- a/Sources.cmake +++ b/Sources.cmake @@ -66,6 +66,8 @@ target_sources(Luau.CodeGen PRIVATE # Luau.Analysis Sources target_sources(Luau.Analysis PRIVATE + Analysis/include/Luau/ApplyTypeFunction.h + Analysis/include/Luau/AstJsonEncoder.h Analysis/include/Luau/AstQuery.h Analysis/include/Luau/Autocomplete.h Analysis/include/Luau/BuiltinDefinitions.h @@ -81,7 +83,7 @@ target_sources(Luau.Analysis PRIVATE Analysis/include/Luau/Frontend.h Analysis/include/Luau/Instantiation.h Analysis/include/Luau/IostreamHelpers.h - Analysis/include/Luau/JsonEncoder.h + Analysis/include/Luau/JsonEmitter.h Analysis/include/Luau/Linter.h Analysis/include/Luau/LValue.h Analysis/include/Luau/Module.h @@ -113,6 +115,8 @@ target_sources(Luau.Analysis PRIVATE Analysis/include/Luau/Variant.h Analysis/include/Luau/VisitTypeVar.h + Analysis/src/ApplyTypeFunction.cpp + Analysis/src/AstJsonEncoder.cpp Analysis/src/AstQuery.cpp Analysis/src/Autocomplete.cpp Analysis/src/BuiltinDefinitions.cpp @@ -126,7 +130,7 @@ target_sources(Luau.Analysis PRIVATE Analysis/src/Frontend.cpp Analysis/src/Instantiation.cpp Analysis/src/IostreamHelpers.cpp - Analysis/src/JsonEncoder.cpp + Analysis/src/JsonEmitter.cpp Analysis/src/Linter.cpp Analysis/src/LValue.cpp Analysis/src/Module.cpp @@ -255,6 +259,7 @@ if(TARGET Luau.UnitTest) tests/ScopedFlags.h tests/Fixture.cpp tests/AssemblyBuilderX64.test.cpp + tests/AstJsonEncoder.test.cpp tests/AstQuery.test.cpp tests/AstVisitor.test.cpp tests/Autocomplete.test.cpp @@ -266,7 +271,7 @@ if(TARGET Luau.UnitTest) tests/CostModel.test.cpp tests/Error.test.cpp tests/Frontend.test.cpp - tests/JsonEncoder.test.cpp + tests/JsonEmitter.test.cpp tests/Lexer.test.cpp tests/Linter.test.cpp tests/LValue.test.cpp diff --git a/VM/include/lua.h b/VM/include/lua.h index 187dd5c2..f986c2b3 100644 --- a/VM/include/lua.h +++ b/VM/include/lua.h @@ -11,7 +11,7 @@ -/* option for multiple returns in `lua_pcall' and `lua_call' */ +// option for multiple returns in `lua_pcall' and `lua_call' #define LUA_MULTRET (-1) /* @@ -23,7 +23,7 @@ #define lua_upvalueindex(i) (LUA_GLOBALSINDEX - (i)) #define lua_ispseudo(i) ((i) <= LUA_REGISTRYINDEX) -/* thread status; 0 is OK */ +// thread status; 0 is OK enum lua_Status { LUA_OK = 0, @@ -32,7 +32,7 @@ enum lua_Status LUA_ERRSYNTAX, LUA_ERRMEM, LUA_ERRERR, - LUA_BREAK, /* yielded for a debug breakpoint */ + LUA_BREAK, // yielded for a debug breakpoint }; typedef struct lua_State lua_State; @@ -46,7 +46,7 @@ typedef int (*lua_Continuation)(lua_State* L, int status); typedef void* (*lua_Alloc)(void* ud, void* ptr, size_t osize, size_t nsize); -/* non-return type */ +// non-return type #define l_noret void LUA_NORETURN /* @@ -61,15 +61,15 @@ typedef void* (*lua_Alloc)(void* ud, void* ptr, size_t osize, size_t nsize); // clang-format off enum lua_Type { - LUA_TNIL = 0, /* must be 0 due to lua_isnoneornil */ - LUA_TBOOLEAN = 1, /* must be 1 due to l_isfalse */ + LUA_TNIL = 0, // must be 0 due to lua_isnoneornil + LUA_TBOOLEAN = 1, // must be 1 due to l_isfalse LUA_TLIGHTUSERDATA, LUA_TNUMBER, LUA_TVECTOR, - LUA_TSTRING, /* all types above this must be value types, all types below this must be GC types - see iscollectable */ + LUA_TSTRING, // all types above this must be value types, all types below this must be GC types - see iscollectable LUA_TTABLE, @@ -77,23 +77,23 @@ enum lua_Type LUA_TUSERDATA, LUA_TTHREAD, - /* values below this line are used in GCObject tags but may never show up in TValue type tags */ + // values below this line are used in GCObject tags but may never show up in TValue type tags LUA_TPROTO, LUA_TUPVAL, LUA_TDEADKEY, - /* the count of TValue type tags */ + // the count of TValue type tags LUA_T_COUNT = LUA_TPROTO }; // clang-format on -/* type of numbers in Luau */ +// type of numbers in Luau typedef double lua_Number; -/* type for integer functions */ +// type for integer functions typedef int lua_Integer; -/* unsigned integer type */ +// unsigned integer type typedef unsigned lua_Unsigned; /* @@ -117,7 +117,7 @@ LUA_API void lua_remove(lua_State* L, int idx); LUA_API void lua_insert(lua_State* L, int idx); LUA_API void lua_replace(lua_State* L, int idx); LUA_API int lua_checkstack(lua_State* L, int sz); -LUA_API void lua_rawcheckstack(lua_State* L, int sz); /* allows for unlimited stack frames */ +LUA_API void lua_rawcheckstack(lua_State* L, int sz); // allows for unlimited stack frames LUA_API void lua_xmove(lua_State* from, lua_State* to, int n); LUA_API void lua_xpush(lua_State* from, lua_State* to, int idx); @@ -231,18 +231,18 @@ LUA_API void lua_setthreaddata(lua_State* L, void* data); enum lua_GCOp { - /* stop and resume incremental garbage collection */ + // stop and resume incremental garbage collection LUA_GCSTOP, LUA_GCRESTART, - /* run a full GC cycle; not recommended for latency sensitive applications */ + // run a full GC cycle; not recommended for latency sensitive applications LUA_GCCOLLECT, - /* return the heap size in KB and the remainder in bytes */ + // return the heap size in KB and the remainder in bytes LUA_GCCOUNT, LUA_GCCOUNTB, - /* return 1 if GC is active (not stopped); note that GC may not be actively collecting even if it's running */ + // return 1 if GC is active (not stopped); note that GC may not be actively collecting even if it's running LUA_GCISRUNNING, /* @@ -359,9 +359,9 @@ LUA_API void lua_unref(lua_State* L, int ref); ** ======================================================================= */ -typedef struct lua_Debug lua_Debug; /* activation record */ +typedef struct lua_Debug lua_Debug; // activation record -/* Functions to be called by the debugger in specific events */ +// Functions to be called by the debugger in specific events typedef void (*lua_Hook)(lua_State* L, lua_Debug* ar); LUA_API int lua_stackdepth(lua_State* L); @@ -379,24 +379,24 @@ typedef void (*lua_Coverage)(void* context, const char* function, int linedefine LUA_API void lua_getcoverage(lua_State* L, int funcindex, void* context, lua_Coverage callback); -/* Warning: this function is not thread-safe since it stores the result in a shared global array! Only use for debugging. */ +// Warning: this function is not thread-safe since it stores the result in a shared global array! Only use for debugging. LUA_API const char* lua_debugtrace(lua_State* L); struct lua_Debug { - const char* name; /* (n) */ - const char* what; /* (s) `Lua', `C', `main', `tail' */ - const char* source; /* (s) */ - int linedefined; /* (s) */ - int currentline; /* (l) */ - unsigned char nupvals; /* (u) number of upvalues */ - unsigned char nparams; /* (a) number of parameters */ - char isvararg; /* (a) */ - char short_src[LUA_IDSIZE]; /* (s) */ - void* userdata; /* only valid in luau_callhook */ + const char* name; // (n) + const char* what; // (s) `Lua', `C', `main', `tail' + const char* source; // (s) + int linedefined; // (s) + int currentline; // (l) + unsigned char nupvals; // (u) number of upvalues + unsigned char nparams; // (a) number of parameters + char isvararg; // (a) + char short_src[LUA_IDSIZE]; // (s) + void* userdata; // only valid in luau_callhook }; -/* }====================================================================== */ +// }====================================================================== /* Callbacks that can be used to reconfigure behavior of the VM dynamically. * These are shared between all coroutines. @@ -405,18 +405,18 @@ struct lua_Debug * can only be changed when the VM is not running any code */ struct lua_Callbacks { - void* userdata; /* arbitrary userdata pointer that is never overwritten by Luau */ + void* userdata; // arbitrary userdata pointer that is never overwritten by Luau - void (*interrupt)(lua_State* L, int gc); /* gets called at safepoints (loop back edges, call/ret, gc) if set */ - void (*panic)(lua_State* L, int errcode); /* gets called when an unprotected error is raised (if longjmp is used) */ + void (*interrupt)(lua_State* L, int gc); // gets called at safepoints (loop back edges, call/ret, gc) if set + void (*panic)(lua_State* L, int errcode); // gets called when an unprotected error is raised (if longjmp is used) - void (*userthread)(lua_State* LP, lua_State* L); /* gets called when L is created (LP == parent) or destroyed (LP == NULL) */ - int16_t (*useratom)(const char* s, size_t l); /* gets called when a string is created; returned atom can be retrieved via tostringatom */ + void (*userthread)(lua_State* LP, lua_State* L); // gets called when L is created (LP == parent) or destroyed (LP == NULL) + int16_t (*useratom)(const char* s, size_t l); // gets called when a string is created; returned atom can be retrieved via tostringatom - void (*debugbreak)(lua_State* L, lua_Debug* ar); /* gets called when BREAK instruction is encountered */ - void (*debugstep)(lua_State* L, lua_Debug* ar); /* gets called after each instruction in single step mode */ - void (*debuginterrupt)(lua_State* L, lua_Debug* ar); /* gets called when thread execution is interrupted by break in another thread */ - void (*debugprotectederror)(lua_State* L); /* gets called when protected call results in an error */ + void (*debugbreak)(lua_State* L, lua_Debug* ar); // gets called when BREAK instruction is encountered + void (*debugstep)(lua_State* L, lua_Debug* ar); // gets called after each instruction in single step mode + void (*debuginterrupt)(lua_State* L, lua_Debug* ar); // gets called when thread execution is interrupted by break in another thread + void (*debugprotectederror)(lua_State* L); // gets called when protected call results in an error }; typedef struct lua_Callbacks lua_Callbacks; diff --git a/VM/include/luaconf.h b/VM/include/luaconf.h index b93cbf7c..7b0f4c30 100644 --- a/VM/include/luaconf.h +++ b/VM/include/luaconf.h @@ -33,14 +33,14 @@ #define LUA_NORETURN __attribute__((__noreturn__)) #endif -/* Can be used to reconfigure visibility/exports for public APIs */ +// Can be used to reconfigure visibility/exports for public APIs #ifndef LUA_API #define LUA_API extern #endif #define LUALIB_API LUA_API -/* Can be used to reconfigure visibility for internal APIs */ +// Can be used to reconfigure visibility for internal APIs #if defined(__GNUC__) #define LUAI_FUNC __attribute__((visibility("hidden"))) extern #define LUAI_DATA LUAI_FUNC @@ -49,67 +49,67 @@ #define LUAI_DATA extern #endif -/* Can be used to reconfigure internal error handling to use longjmp instead of C++ EH */ +// Can be used to reconfigure internal error handling to use longjmp instead of C++ EH #ifndef LUA_USE_LONGJMP #define LUA_USE_LONGJMP 0 #endif -/* LUA_IDSIZE gives the maximum size for the description of the source */ +// LUA_IDSIZE gives the maximum size for the description of the source #ifndef LUA_IDSIZE #define LUA_IDSIZE 256 #endif -/* LUA_MINSTACK is the guaranteed number of Lua stack slots available to a C function */ +// LUA_MINSTACK is the guaranteed number of Lua stack slots available to a C function #ifndef LUA_MINSTACK #define LUA_MINSTACK 20 #endif -/* LUAI_MAXCSTACK limits the number of Lua stack slots that a C function can use */ +// LUAI_MAXCSTACK limits the number of Lua stack slots that a C function can use #ifndef LUAI_MAXCSTACK #define LUAI_MAXCSTACK 8000 #endif -/* LUAI_MAXCALLS limits the number of nested calls */ +// LUAI_MAXCALLS limits the number of nested calls #ifndef LUAI_MAXCALLS #define LUAI_MAXCALLS 20000 #endif -/* LUAI_MAXCCALLS is the maximum depth for nested C calls; this limit depends on native stack size */ +// LUAI_MAXCCALLS is the maximum depth for nested C calls; this limit depends on native stack size #ifndef LUAI_MAXCCALLS #define LUAI_MAXCCALLS 200 #endif -/* buffer size used for on-stack string operations; this limit depends on native stack size */ +// buffer size used for on-stack string operations; this limit depends on native stack size #ifndef LUA_BUFFERSIZE #define LUA_BUFFERSIZE 512 #endif -/* number of valid Lua userdata tags */ +// number of valid Lua userdata tags #ifndef LUA_UTAG_LIMIT #define LUA_UTAG_LIMIT 128 #endif -/* upper bound for number of size classes used by page allocator */ +// upper bound for number of size classes used by page allocator #ifndef LUA_SIZECLASSES #define LUA_SIZECLASSES 32 #endif -/* available number of separate memory categories */ +// available number of separate memory categories #ifndef LUA_MEMORY_CATEGORIES #define LUA_MEMORY_CATEGORIES 256 #endif -/* minimum size for the string table (must be power of 2) */ +// minimum size for the string table (must be power of 2) #ifndef LUA_MINSTRTABSIZE #define LUA_MINSTRTABSIZE 32 #endif -/* maximum number of captures supported by pattern matching */ +// maximum number of captures supported by pattern matching #ifndef LUA_MAXCAPTURES #define LUA_MAXCAPTURES 32 #endif -/* }================================================================== */ +// }================================================================== /* @@ LUAI_USER_ALIGNMENT_T is a type that requires maximum alignment. @@ -126,6 +126,6 @@ long l; \ } -#define LUA_VECTOR_SIZE 3 /* must be 3 or 4 */ +#define LUA_VECTOR_SIZE 3 // must be 3 or 4 #define LUA_EXTRA_SIZE LUA_VECTOR_SIZE - 2 diff --git a/VM/include/lualib.h b/VM/include/lualib.h index bebd0a0f..955604de 100644 --- a/VM/include/lualib.h +++ b/VM/include/lualib.h @@ -72,7 +72,7 @@ LUALIB_API const char* luaL_typename(lua_State* L, int idx); #define luaL_opt(L, f, n, d) (lua_isnoneornil(L, (n)) ? (d) : f(L, (n))) -/* generic buffer manipulation */ +// generic buffer manipulation struct luaL_Buffer { @@ -102,7 +102,7 @@ LUALIB_API void luaL_addvalue(luaL_Buffer* B); LUALIB_API void luaL_pushresult(luaL_Buffer* B); LUALIB_API void luaL_pushresultsize(luaL_Buffer* B, size_t size); -/* builtin libraries */ +// builtin libraries LUALIB_API int luaopen_base(lua_State* L); #define LUA_COLIBNAME "coroutine" @@ -129,9 +129,9 @@ LUALIB_API int luaopen_math(lua_State* L); #define LUA_DBLIBNAME "debug" LUALIB_API int luaopen_debug(lua_State* L); -/* open all builtin libraries */ +// open all builtin libraries LUALIB_API void luaL_openlibs(lua_State* L); -/* sandbox libraries and globals */ +// sandbox libraries and globals LUALIB_API void luaL_sandbox(lua_State* L); LUALIB_API void luaL_sandboxthread(lua_State* L); diff --git a/VM/src/lapi.cpp b/VM/src/lapi.cpp index e59f914f..bb994fb4 100644 --- a/VM/src/lapi.cpp +++ b/VM/src/lapi.cpp @@ -59,8 +59,8 @@ const char* luau_ident = "$Luau: Copyright (C) 2019-2022 Roblox Corporation $\n" static Table* getcurrenv(lua_State* L) { - if (L->ci == L->base_ci) /* no enclosing function? */ - return L->gt; /* use global table as environment */ + if (L->ci == L->base_ci) // no enclosing function? + return L->gt; // use global table as environment else return curr_func(L)->env; } @@ -69,7 +69,7 @@ static LUAU_NOINLINE TValue* pseudo2addr(lua_State* L, int idx) { api_check(L, lua_ispseudo(idx)); switch (idx) - { /* pseudo-indices */ + { // pseudo-indices case LUA_REGISTRYINDEX: return registry(L); case LUA_ENVIRONINDEX: @@ -129,7 +129,7 @@ int lua_checkstack(lua_State* L, int size) { int res = 1; if (size > LUAI_MAXCSTACK || (L->top - L->base + size) > LUAI_MAXCSTACK) - res = 0; /* stack overflow */ + res = 0; // stack overflow else if (size > 0) { luaD_checkstack(L, size); @@ -219,7 +219,7 @@ void lua_settop(lua_State* L, int idx) else { api_check(L, -(idx + 1) <= (L->top - L->base)); - L->top += idx + 1; /* `subtract' index (index is negative) */ + L->top += idx + 1; // `subtract' index (index is negative) } return; } @@ -267,7 +267,7 @@ void lua_replace(lua_State* L, int idx) else { setobj(L, o, L->top - 1); - if (idx < LUA_GLOBALSINDEX) /* function upvalue? */ + if (idx < LUA_GLOBALSINDEX) // function upvalue? luaC_barrier(L, curr_func(L), L->top - 1); } L->top--; @@ -429,13 +429,13 @@ const char* lua_tolstring(lua_State* L, int idx, size_t* len) { luaC_checkthreadsleep(L); if (!luaV_tostring(L, o)) - { /* conversion failed? */ + { // conversion failed? if (len != NULL) *len = 0; return NULL; } luaC_checkGC(L); - o = index2addr(L, idx); /* previous call may reallocate the stack */ + o = index2addr(L, idx); // previous call may reallocate the stack } if (len != NULL) *len = tsvalue(o)->len; @@ -660,7 +660,7 @@ void lua_pushcclosurek(lua_State* L, lua_CFunction fn, const char* debugname, in void lua_pushboolean(lua_State* L, int b) { - setbvalue(L->top, (b != 0)); /* ensure that true is 1 */ + setbvalue(L->top, (b != 0)); // ensure that true is 1 api_incr_top(L); return; } @@ -829,7 +829,7 @@ void lua_settable(lua_State* L, int idx) StkId t = index2addr(L, idx); api_checkvalidindex(L, t); luaV_settable(L, t, L->top - 2, L->top - 1); - L->top -= 2; /* pop index and value */ + L->top -= 2; // pop index and value return; } @@ -967,7 +967,7 @@ void lua_call(lua_State* L, int nargs, int nresults) ** Execute a protected call. */ struct CallS -{ /* data to `f_call' */ +{ // data to `f_call' StkId func; int nresults; }; @@ -992,7 +992,7 @@ int lua_pcall(lua_State* L, int nargs, int nresults, int errfunc) func = savestack(L, o); } struct CallS c; - c.func = L->top - (nargs + 1); /* function to be called */ + c.func = L->top - (nargs + 1); // function to be called c.nresults = nresults; int status = luaD_pcall(L, f_call, &c, savestack(L, c.func), func); @@ -1044,7 +1044,7 @@ int lua_gc(lua_State* L, int what, int data) } case LUA_GCCOUNT: { - /* GC values are expressed in Kbytes: #bytes/2^10 */ + // GC values are expressed in Kbytes: #bytes/2^10 res = cast_int(g->totalbytes >> 10); break; } @@ -1084,8 +1084,8 @@ int lua_gc(lua_State* L, int what, int data) actualwork += stepsize; if (g->gcstate == GCSpause) - { /* end of cycle? */ - res = 1; /* signal it */ + { // end of cycle? + res = 1; // signal it break; } } @@ -1137,13 +1137,13 @@ int lua_gc(lua_State* L, int what, int data) } case LUA_GCSETSTEPSIZE: { - /* GC values are expressed in Kbytes: #bytes/2^10 */ + // GC values are expressed in Kbytes: #bytes/2^10 res = g->gcstepsize >> 10; g->gcstepsize = data << 10; break; } default: - res = -1; /* invalid option */ + res = -1; // invalid option } return res; } @@ -1169,8 +1169,8 @@ int lua_next(lua_State* L, int idx) { api_incr_top(L); } - else /* no more elements */ - L->top -= 1; /* remove key */ + else // no more elements + L->top -= 1; // remove key return more; } @@ -1185,12 +1185,12 @@ void lua_concat(lua_State* L, int n) L->top -= (n - 1); } else if (n == 0) - { /* push empty string */ + { // push empty string luaC_checkthreadsleep(L); setsvalue2s(L, L->top, luaS_newlstr(L, "", 0)); api_incr_top(L); } - /* else n == 1; nothing to do */ + // else n == 1; nothing to do return; } @@ -1277,7 +1277,7 @@ uintptr_t lua_encodepointer(lua_State* L, uintptr_t p) int lua_ref(lua_State* L, int idx) { - api_check(L, idx != LUA_REGISTRYINDEX); /* idx is a stack index for value */ + api_check(L, idx != LUA_REGISTRYINDEX); // idx is a stack index for value int ref = LUA_REFNIL; global_State* g = L->global; StkId p = index2addr(L, idx); @@ -1286,13 +1286,13 @@ int lua_ref(lua_State* L, int idx) Table* reg = hvalue(registry(L)); if (g->registryfree != 0) - { /* reuse existing slot */ + { // reuse existing slot ref = g->registryfree; } else - { /* no free elements */ + { // no free elements ref = luaH_getn(reg); - ref++; /* create new reference */ + ref++; // create new reference } TValue* slot = luaH_setnum(L, reg, ref); @@ -1312,7 +1312,7 @@ void lua_unref(lua_State* L, int ref) global_State* g = L->global; Table* reg = hvalue(registry(L)); TValue* slot = luaH_setnum(L, reg, ref); - setnvalue(slot, g->registryfree); /* NB: no barrier needed because value isn't collectable */ + setnvalue(slot, g->registryfree); // NB: no barrier needed because value isn't collectable g->registryfree = ref; return; } diff --git a/VM/src/laux.cpp b/VM/src/laux.cpp index 72169a86..c42e5ccc 100644 --- a/VM/src/laux.cpp +++ b/VM/src/laux.cpp @@ -11,7 +11,7 @@ #include -/* convert a stack index to positive */ +// convert a stack index to positive #define abs_index(L, i) ((i) > 0 || (i) <= LUA_REGISTRYINDEX ? (i) : lua_gettop(L) + (i) + 1) /* @@ -75,7 +75,7 @@ void luaL_where(lua_State* L, int level) lua_pushfstring(L, "%s:%d: ", ar.short_src, ar.currentline); return; } - lua_pushliteral(L, ""); /* else, no information available... */ + lua_pushliteral(L, ""); // else, no information available... } l_noret luaL_errorL(lua_State* L, const char* fmt, ...) @@ -89,7 +89,7 @@ l_noret luaL_errorL(lua_State* L, const char* fmt, ...) lua_error(L); } -/* }====================================================== */ +// }====================================================== int luaL_checkoption(lua_State* L, int narg, const char* def, const char* const lst[]) { @@ -104,13 +104,13 @@ int luaL_checkoption(lua_State* L, int narg, const char* def, const char* const int luaL_newmetatable(lua_State* L, const char* tname) { - lua_getfield(L, LUA_REGISTRYINDEX, tname); /* get registry.name */ - if (!lua_isnil(L, -1)) /* name already in use? */ - return 0; /* leave previous value on top, but return 0 */ + lua_getfield(L, LUA_REGISTRYINDEX, tname); // get registry.name + if (!lua_isnil(L, -1)) // name already in use? + return 0; // leave previous value on top, but return 0 lua_pop(L, 1); - lua_newtable(L); /* create metatable */ + lua_newtable(L); // create metatable lua_pushvalue(L, -1); - lua_setfield(L, LUA_REGISTRYINDEX, tname); /* registry.name = metatable */ + lua_setfield(L, LUA_REGISTRYINDEX, tname); // registry.name = metatable return 1; } @@ -118,18 +118,18 @@ void* luaL_checkudata(lua_State* L, int ud, const char* tname) { void* p = lua_touserdata(L, ud); if (p != NULL) - { /* value is a userdata? */ + { // value is a userdata? if (lua_getmetatable(L, ud)) - { /* does it have a metatable? */ - lua_getfield(L, LUA_REGISTRYINDEX, tname); /* get correct metatable */ + { // does it have a metatable? + lua_getfield(L, LUA_REGISTRYINDEX, tname); // get correct metatable if (lua_rawequal(L, -1, -2)) - { /* does it have the correct mt? */ - lua_pop(L, 2); /* remove both metatables */ + { // does it have the correct mt? + lua_pop(L, 2); // remove both metatables return p; } } } - luaL_typeerrorL(L, ud, tname); /* else error */ + luaL_typeerrorL(L, ud, tname); // else error } void luaL_checkstack(lua_State* L, int space, const char* mes) @@ -243,18 +243,18 @@ const float* luaL_optvector(lua_State* L, int narg, const float* def) int luaL_getmetafield(lua_State* L, int obj, const char* event) { - if (!lua_getmetatable(L, obj)) /* no metatable? */ + if (!lua_getmetatable(L, obj)) // no metatable? return 0; lua_pushstring(L, event); lua_rawget(L, -2); if (lua_isnil(L, -1)) { - lua_pop(L, 2); /* remove metatable and metafield */ + lua_pop(L, 2); // remove metatable and metafield return 0; } else { - lua_remove(L, -2); /* remove only metatable */ + lua_remove(L, -2); // remove only metatable return 1; } } @@ -262,7 +262,7 @@ int luaL_getmetafield(lua_State* L, int obj, const char* event) int luaL_callmeta(lua_State* L, int obj, const char* event) { obj = abs_index(L, obj); - if (!luaL_getmetafield(L, obj, event)) /* no metafield? */ + if (!luaL_getmetafield(L, obj, event)) // no metafield? return 0; lua_pushvalue(L, obj); lua_call(L, 1, 1); @@ -282,19 +282,19 @@ void luaL_register(lua_State* L, const char* libname, const luaL_Reg* l) if (libname) { int size = libsize(l); - /* check whether lib already exists */ + // check whether lib already exists luaL_findtable(L, LUA_REGISTRYINDEX, "_LOADED", 1); - lua_getfield(L, -1, libname); /* get _LOADED[libname] */ + lua_getfield(L, -1, libname); // get _LOADED[libname] if (!lua_istable(L, -1)) - { /* not found? */ - lua_pop(L, 1); /* remove previous result */ - /* try global variable (and create one if it does not exist) */ + { // not found? + lua_pop(L, 1); // remove previous result + // try global variable (and create one if it does not exist) if (luaL_findtable(L, LUA_GLOBALSINDEX, libname, size) != NULL) luaL_error(L, "name conflict for module '%s'", libname); lua_pushvalue(L, -1); - lua_setfield(L, -3, libname); /* _LOADED[libname] = new table */ + lua_setfield(L, -3, libname); // _LOADED[libname] = new table } - lua_remove(L, -2); /* remove _LOADED table */ + lua_remove(L, -2); // remove _LOADED table } for (; l->name; l++) { @@ -315,19 +315,19 @@ const char* luaL_findtable(lua_State* L, int idx, const char* fname, int szhint) lua_pushlstring(L, fname, e - fname); lua_rawget(L, -2); if (lua_isnil(L, -1)) - { /* no such field? */ - lua_pop(L, 1); /* remove this nil */ - lua_createtable(L, 0, (*e == '.' ? 1 : szhint)); /* new table for field */ + { // no such field? + lua_pop(L, 1); // remove this nil + lua_createtable(L, 0, (*e == '.' ? 1 : szhint)); // new table for field lua_pushlstring(L, fname, e - fname); lua_pushvalue(L, -2); - lua_settable(L, -4); /* set new table into field */ + lua_settable(L, -4); // set new table into field } else if (!lua_istable(L, -1)) - { /* field has a non-table value? */ - lua_pop(L, 2); /* remove table and value */ - return fname; /* return problematic part of the name */ + { // field has a non-table value? + lua_pop(L, 2); // remove table and value + return fname; // return problematic part of the name } - lua_remove(L, -2); /* remove previous table */ + lua_remove(L, -2); // remove previous table fname = e + 1; } while (*e == '.'); return NULL; @@ -470,11 +470,11 @@ void luaL_pushresultsize(luaL_Buffer* B, size_t size) luaL_pushresult(B); } -/* }====================================================== */ +// }====================================================== const char* luaL_tolstring(lua_State* L, int idx, size_t* len) { - if (luaL_callmeta(L, idx, "__tostring")) /* is there a metafield? */ + if (luaL_callmeta(L, idx, "__tostring")) // is there a metafield? { if (!lua_isstring(L, -1)) luaL_error(L, "'__tostring' must return a string"); diff --git a/VM/src/lbaselib.cpp b/VM/src/lbaselib.cpp index 4fc5033e..f4dac61f 100644 --- a/VM/src/lbaselib.cpp +++ b/VM/src/lbaselib.cpp @@ -11,8 +11,6 @@ #include #include -LUAU_FASTFLAG(LuauLenTM) - static void writestring(const char* s, size_t l) { fwrite(s, 1, l, stdout); @@ -20,15 +18,15 @@ static void writestring(const char* s, size_t l) static int luaB_print(lua_State* L) { - int n = lua_gettop(L); /* number of arguments */ + int n = lua_gettop(L); // number of arguments for (int i = 1; i <= n; i++) { size_t l; - const char* s = luaL_tolstring(L, i, &l); /* convert to string using __tostring et al */ + const char* s = luaL_tolstring(L, i, &l); // convert to string using __tostring et al if (i > 1) writestring("\t", 1); writestring(s, l); - lua_pop(L, 1); /* pop result */ + lua_pop(L, 1); // pop result } writestring("\n", 1); return 0; @@ -38,7 +36,7 @@ static int luaB_tonumber(lua_State* L) { int base = luaL_optinteger(L, 2, 10); if (base == 10) - { /* standard conversion */ + { // standard conversion int isnum = 0; double n = lua_tonumberx(L, 1, &isnum); if (isnum) @@ -46,7 +44,7 @@ static int luaB_tonumber(lua_State* L) lua_pushnumber(L, n); return 1; } - luaL_checkany(L, 1); /* error if we don't have any argument */ + luaL_checkany(L, 1); // error if we don't have any argument } else { @@ -56,17 +54,17 @@ static int luaB_tonumber(lua_State* L) unsigned long long n; n = strtoull(s1, &s2, base); if (s1 != s2) - { /* at least one valid digit? */ + { // at least one valid digit? while (isspace((unsigned char)(*s2))) - s2++; /* skip trailing spaces */ + s2++; // skip trailing spaces if (*s2 == '\0') - { /* no invalid trailing characters? */ + { // no invalid trailing characters? lua_pushnumber(L, (double)n); return 1; } } } - lua_pushnil(L); /* else not a number */ + lua_pushnil(L); // else not a number return 1; } @@ -75,7 +73,7 @@ static int luaB_error(lua_State* L) int level = luaL_optinteger(L, 2, 1); lua_settop(L, 1); if (lua_isstring(L, 1) && level > 0) - { /* add extra information? */ + { // add extra information? luaL_where(L, level); lua_pushvalue(L, 1); lua_concat(L, 2); @@ -89,10 +87,10 @@ static int luaB_getmetatable(lua_State* L) if (!lua_getmetatable(L, 1)) { lua_pushnil(L); - return 1; /* no metatable */ + return 1; // no metatable } luaL_getmetafield(L, 1, "__metatable"); - return 1; /* returns either __metatable field (if present) or metatable */ + return 1; // returns either __metatable field (if present) or metatable } static int luaB_setmetatable(lua_State* L) @@ -126,8 +124,8 @@ static void getfunc(lua_State* L, int opt) static int luaB_getfenv(lua_State* L) { getfunc(L, 1); - if (lua_iscfunction(L, -1)) /* is a C function? */ - lua_pushvalue(L, LUA_GLOBALSINDEX); /* return the thread's global env. */ + if (lua_iscfunction(L, -1)) // is a C function? + lua_pushvalue(L, LUA_GLOBALSINDEX); // return the thread's global env. else lua_getfenv(L, -1); lua_setsafeenv(L, -1, false); @@ -142,7 +140,7 @@ static int luaB_setfenv(lua_State* L) lua_setsafeenv(L, -1, false); if (lua_isnumber(L, 1) && lua_tonumber(L, 1) == 0) { - /* change environment of current thread */ + // change environment of current thread lua_pushthread(L); lua_insert(L, -2); lua_setfenv(L, -2); @@ -182,9 +180,6 @@ static int luaB_rawset(lua_State* L) static int luaB_rawlen(lua_State* L) { - if (!FFlag::LuauLenTM) - luaL_error(L, "'rawlen' is not available"); - int tt = lua_type(L, 1); luaL_argcheck(L, tt == LUA_TTABLE || tt == LUA_TSTRING, 1, "table or string expected"); int len = lua_objlen(L, 1); @@ -201,7 +196,7 @@ static int luaB_gcinfo(lua_State* L) static int luaB_type(lua_State* L) { luaL_checkany(L, 1); - /* resulting name doesn't differentiate between userdata types */ + // resulting name doesn't differentiate between userdata types lua_pushstring(L, lua_typename(L, lua_type(L, 1))); return 1; } @@ -209,7 +204,7 @@ static int luaB_type(lua_State* L) static int luaB_typeof(lua_State* L) { luaL_checkany(L, 1); - /* resulting name returns __type if specified unless the input is a newproxy-created userdata */ + // resulting name returns __type if specified unless the input is a newproxy-created userdata lua_pushstring(L, luaL_typename(L, 1)); return 1; } @@ -217,7 +212,7 @@ static int luaB_typeof(lua_State* L) int luaB_next(lua_State* L) { luaL_checktype(L, 1, LUA_TTABLE); - lua_settop(L, 2); /* create a 2nd argument if there isn't one */ + lua_settop(L, 2); // create a 2nd argument if there isn't one if (lua_next(L, 1)) return 2; else @@ -230,9 +225,9 @@ int luaB_next(lua_State* L) static int luaB_pairs(lua_State* L) { luaL_checktype(L, 1, LUA_TTABLE); - lua_pushvalue(L, lua_upvalueindex(1)); /* return generator, */ - lua_pushvalue(L, 1); /* state, */ - lua_pushnil(L); /* and initial value */ + lua_pushvalue(L, lua_upvalueindex(1)); // return generator, + lua_pushvalue(L, 1); // state, + lua_pushnil(L); // and initial value return 3; } @@ -240,7 +235,7 @@ int luaB_inext(lua_State* L) { int i = luaL_checkinteger(L, 2); luaL_checktype(L, 1, LUA_TTABLE); - i++; /* next value */ + i++; // next value lua_pushinteger(L, i); lua_rawgeti(L, 1, i); return (lua_isnil(L, -1)) ? 0 : 2; @@ -249,9 +244,9 @@ int luaB_inext(lua_State* L) static int luaB_ipairs(lua_State* L) { luaL_checktype(L, 1, LUA_TTABLE); - lua_pushvalue(L, lua_upvalueindex(1)); /* return generator, */ - lua_pushvalue(L, 1); /* state, */ - lua_pushinteger(L, 0); /* and initial value */ + lua_pushvalue(L, lua_upvalueindex(1)); // return generator, + lua_pushvalue(L, 1); // state, + lua_pushinteger(L, 0); // and initial value return 3; } @@ -340,12 +335,12 @@ static int luaB_xpcally(lua_State* L) { luaL_checktype(L, 2, LUA_TFUNCTION); - /* swap function & error function */ + // swap function & error function lua_pushvalue(L, 1); lua_pushvalue(L, 2); lua_replace(L, 1); lua_replace(L, 2); - /* at this point the stack looks like err, f, args */ + // at this point the stack looks like err, f, args // any errors from this point on are handled by continuation L->ci->flags |= LUA_CALLINFO_HANDLE; @@ -386,7 +381,7 @@ static int luaB_xpcallcont(lua_State* L, int status) lua_rawcheckstack(L, 1); lua_pushboolean(L, true); lua_replace(L, 1); // replace error function with status - return lua_gettop(L); /* return status + all results */ + return lua_gettop(L); // return status + all results } else { @@ -462,16 +457,16 @@ static void auxopen(lua_State* L, const char* name, lua_CFunction f, lua_CFuncti int luaopen_base(lua_State* L) { - /* set global _G */ + // set global _G lua_pushvalue(L, LUA_GLOBALSINDEX); lua_setglobal(L, "_G"); - /* open lib into global table */ + // open lib into global table luaL_register(L, "_G", base_funcs); lua_pushliteral(L, "Luau"); - lua_setglobal(L, "_VERSION"); /* set global _VERSION */ + lua_setglobal(L, "_VERSION"); // set global _VERSION - /* `ipairs' and `pairs' need auxiliary functions as upvalues */ + // `ipairs' and `pairs' need auxiliary functions as upvalues auxopen(L, "ipairs", luaB_ipairs, luaB_inext); auxopen(L, "pairs", luaB_pairs, luaB_next); diff --git a/VM/src/lbitlib.cpp b/VM/src/lbitlib.cpp index 093400f2..47445b80 100644 --- a/VM/src/lbitlib.cpp +++ b/VM/src/lbitlib.cpp @@ -8,10 +8,10 @@ #define ALLONES ~0u #define NBITS int(8 * sizeof(unsigned)) -/* macro to trim extra bits */ +// macro to trim extra bits #define trim(x) ((x)&ALLONES) -/* builds a number with 'n' ones (1 <= n <= NBITS) */ +// builds a number with 'n' ones (1 <= n <= NBITS) #define mask(n) (~((ALLONES << 1) << ((n)-1))) typedef unsigned b_uint; @@ -69,7 +69,7 @@ static int b_not(lua_State* L) static int b_shift(lua_State* L, b_uint r, int i) { if (i < 0) - { /* shift right? */ + { // shift right? i = -i; r = trim(r); if (i >= NBITS) @@ -78,7 +78,7 @@ static int b_shift(lua_State* L, b_uint r, int i) r >>= i; } else - { /* shift left */ + { // shift left if (i >= NBITS) r = 0; else @@ -106,11 +106,11 @@ static int b_arshift(lua_State* L) if (i < 0 || !(r & ((b_uint)1 << (NBITS - 1)))) return b_shift(L, r, -i); else - { /* arithmetic shift for 'negative' number */ + { // arithmetic shift for 'negative' number if (i >= NBITS) r = ALLONES; else - r = trim((r >> i) | ~(~(b_uint)0 >> i)); /* add signal bit */ + r = trim((r >> i) | ~(~(b_uint)0 >> i)); // add signal bit lua_pushunsigned(L, r); return 1; } @@ -119,9 +119,9 @@ static int b_arshift(lua_State* L) static int b_rot(lua_State* L, int i) { b_uint r = luaL_checkunsigned(L, 1); - i &= (NBITS - 1); /* i = i % NBITS */ + i &= (NBITS - 1); // i = i % NBITS r = trim(r); - if (i != 0) /* avoid undefined shift of NBITS when i == 0 */ + if (i != 0) // avoid undefined shift of NBITS when i == 0 r = (r << i) | (r >> (NBITS - i)); lua_pushunsigned(L, trim(r)); return 1; @@ -172,7 +172,7 @@ static int b_replace(lua_State* L) b_uint v = luaL_checkunsigned(L, 2); int f = fieldargs(L, 3, &w); int m = mask(w); - v &= m; /* erase bits outside given width */ + v &= m; // erase bits outside given width r = (r & ~(m << f)) | (v << f); lua_pushunsigned(L, r); return 1; diff --git a/VM/src/lcommon.h b/VM/src/lcommon.h index ac79cd97..c9d95c77 100644 --- a/VM/src/lcommon.h +++ b/VM/src/lcommon.h @@ -11,7 +11,7 @@ typedef LUAI_USER_ALIGNMENT_T L_Umaxalign; -/* internal assertions for in-house debugging */ +// internal assertions for in-house debugging #define check_exp(c, e) (LUAU_ASSERT(c), (e)) #define api_check(l, e) LUAU_ASSERT(e) diff --git a/VM/src/lcorolib.cpp b/VM/src/lcorolib.cpp index 7592a14c..7b967e34 100644 --- a/VM/src/lcorolib.cpp +++ b/VM/src/lcorolib.cpp @@ -5,9 +5,9 @@ #include "lstate.h" #include "lvm.h" -#define CO_RUN 0 /* running */ -#define CO_SUS 1 /* suspended */ -#define CO_NOR 2 /* 'normal' (it resumed another coroutine) */ +#define CO_RUN 0 // running +#define CO_SUS 1 // suspended +#define CO_NOR 2 // 'normal' (it resumed another coroutine) #define CO_DEAD 3 #define CO_STATUS_ERROR -1 @@ -23,13 +23,13 @@ static int auxstatus(lua_State* L, lua_State* co) return CO_SUS; if (co->status == LUA_BREAK) return CO_NOR; - if (co->status != 0) /* some error occurred */ + if (co->status != 0) // some error occurred return CO_DEAD; - if (co->ci != co->base_ci) /* does it have frames? */ + if (co->ci != co->base_ci) // does it have frames? return CO_NOR; if (co->top == co->base) return CO_DEAD; - return CO_SUS; /* initial state */ + return CO_SUS; // initial state } static int costatus(lua_State* L) @@ -68,10 +68,10 @@ static int auxresume(lua_State* L, lua_State* co, int narg) int nres = cast_int(co->top - co->base); if (nres) { - /* +1 accounts for true/false status in resumefinish */ + // +1 accounts for true/false status in resumefinish if (nres + 1 > LUA_MINSTACK && !lua_checkstack(L, nres + 1)) luaL_error(L, "too many results to resume"); - lua_xmove(co, L, nres); /* move yielded values */ + lua_xmove(co, L, nres); // move yielded values } return nres; } @@ -81,7 +81,7 @@ static int auxresume(lua_State* L, lua_State* co, int narg) } else { - lua_xmove(co, L, 1); /* move error message */ + lua_xmove(co, L, 1); // move error message return CO_STATUS_ERROR; } } @@ -102,13 +102,13 @@ static int auxresumecont(lua_State* L, lua_State* co) int nres = cast_int(co->top - co->base); if (!lua_checkstack(L, nres + 1)) luaL_error(L, "too many results to resume"); - lua_xmove(co, L, nres); /* move yielded values */ + lua_xmove(co, L, nres); // move yielded values return nres; } else { lua_rawcheckstack(L, 2); - lua_xmove(co, L, 1); /* move error message */ + lua_xmove(co, L, 1); // move error message return CO_STATUS_ERROR; } } @@ -119,13 +119,13 @@ static int coresumefinish(lua_State* L, int r) { lua_pushboolean(L, 0); lua_insert(L, -2); - return 2; /* return false + error message */ + return 2; // return false + error message } else { lua_pushboolean(L, 1); lua_insert(L, -(r + 1)); - return r + 1; /* return true + `resume' returns */ + return r + 1; // return true + `resume' returns } } @@ -161,12 +161,12 @@ static int auxwrapfinish(lua_State* L, int r) if (r < 0) { if (lua_isstring(L, -1)) - { /* error object is a string? */ - luaL_where(L, 1); /* add extra info */ + { // error object is a string? + luaL_where(L, 1); // add extra info lua_insert(L, -2); lua_concat(L, 2); } - lua_error(L); /* propagate error */ + lua_error(L); // propagate error } return r; } @@ -221,7 +221,7 @@ static int coyield(lua_State* L) static int corunning(lua_State* L) { if (lua_pushthread(L)) - lua_pushnil(L); /* main thread is not a coroutine */ + lua_pushnil(L); // main thread is not a coroutine return 1; } @@ -250,7 +250,7 @@ static int coclose(lua_State* L) { lua_pushboolean(L, false); if (lua_gettop(co)) - lua_xmove(co, L, 1); /* move error message */ + lua_xmove(co, L, 1); // move error message lua_resetthread(co); return 2; } diff --git a/VM/src/ldblib.cpp b/VM/src/ldblib.cpp index 8246f818..ece4f551 100644 --- a/VM/src/ldblib.cpp +++ b/VM/src/ldblib.cpp @@ -82,9 +82,9 @@ static int db_info(lua_State* L) case 'f': if (L1 == L) - lua_pushvalue(L, -1 - results); /* function is right before results */ + lua_pushvalue(L, -1 - results); // function is right before results else - lua_xmove(L1, L, 1); /* function is at top of L1 */ + lua_xmove(L1, L, 1); // function is at top of L1 results++; break; diff --git a/VM/src/ldebug.cpp b/VM/src/ldebug.cpp index 5ef41b78..c44ccbed 100644 --- a/VM/src/ldebug.cpp +++ b/VM/src/ldebug.cpp @@ -86,7 +86,7 @@ const char* lua_setlocal(lua_State* L, int level, int n) const LocVar* var = fp ? luaF_getlocal(fp, n, currentpc(L, ci)) : NULL; if (var) setobjs2s(L, ci->base + var->reg, L->top - 1); - L->top--; /* pop value */ + L->top--; // pop value const char* name = var ? getstr(var->varname) : NULL; return name; } @@ -269,6 +269,13 @@ l_noret luaG_indexerror(lua_State* L, const TValue* p1, const TValue* p2) luaG_runerror(L, "attempt to index %s with %s", t1, t2); } +l_noret luaG_methoderror(lua_State* L, const TValue* p1, const TString* p2) +{ + const char* t1 = luaT_objtypename(L, p1); + + luaG_runerror(L, "attempt to call missing method '%s' of %s", getstr(p2), t1); +} + l_noret luaG_readonlyerror(lua_State* L) { luaG_runerror(L, "attempt to modify a readonly table"); @@ -279,7 +286,7 @@ static void pusherror(lua_State* L, const char* msg) CallInfo* ci = L->ci; if (isLua(ci)) { - char buff[LUA_IDSIZE]; /* add file:line information */ + char buff[LUA_IDSIZE]; // add file:line information luaO_chunkid(buff, getstr(getluaproto(ci)->source), LUA_IDSIZE); int line = currentline(L, ci); luaO_pushfstring(L, "%s:%d: %s", buff, line, msg); diff --git a/VM/src/ldebug.h b/VM/src/ldebug.h index 8e03db36..a93e412f 100644 --- a/VM/src/ldebug.h +++ b/VM/src/ldebug.h @@ -19,6 +19,7 @@ LUAI_FUNC l_noret luaG_concaterror(lua_State* L, StkId p1, StkId p2); LUAI_FUNC l_noret luaG_aritherror(lua_State* L, const TValue* p1, const TValue* p2, TMS op); LUAI_FUNC l_noret luaG_ordererror(lua_State* L, const TValue* p1, const TValue* p2, TMS op); LUAI_FUNC l_noret luaG_indexerror(lua_State* L, const TValue* p1, const TValue* p2); +LUAI_FUNC l_noret luaG_methoderror(lua_State* L, const TValue* p1, const TString* p2); LUAI_FUNC l_noret luaG_readonlyerror(lua_State* L); LUAI_FUNC LUA_PRINTF_ATTR(2, 3) l_noret luaG_runerrorL(lua_State* L, const char* fmt, ...); diff --git a/VM/src/ldo.cpp b/VM/src/ldo.cpp index 0642cb6d..6016e41f 100644 --- a/VM/src/ldo.cpp +++ b/VM/src/ldo.cpp @@ -31,7 +31,7 @@ struct lua_jmpbuf jmp_buf buf; }; -/* use POSIX versions of setjmp/longjmp if possible: they don't save/restore signal mask and are therefore faster */ +// use POSIX versions of setjmp/longjmp if possible: they don't save/restore signal mask and are therefore faster #if defined(__linux__) || defined(__APPLE__) #define LUAU_SETJMP(buf) _setjmp(buf) #define LUAU_LONGJMP(buf, code) _longjmp(buf, code) @@ -153,7 +153,7 @@ l_noret luaD_throw(lua_State* L, int errcode) } #endif -/* }====================================================== */ +// }====================================================== static void correctstack(lua_State* L, TValue* oldstack) { @@ -177,7 +177,7 @@ void luaD_reallocstack(lua_State* L, int newsize) luaM_reallocarray(L, L->stack, L->stacksize, realsize, TValue, L->memcat); TValue* newstack = L->stack; for (int i = L->stacksize; i < realsize; i++) - setnilvalue(newstack + i); /* erase new segment */ + setnilvalue(newstack + i); // erase new segment L->stacksize = realsize; L->stack_last = newstack + newsize; correctstack(L, oldstack); @@ -194,7 +194,7 @@ void luaD_reallocCI(lua_State* L, int newsize) void luaD_growstack(lua_State* L, int n) { - if (n <= L->stacksize) /* double size is enough? */ + if (n <= L->stacksize) // double size is enough? luaD_reallocstack(L, 2 * L->stacksize); else luaD_reallocstack(L, L->stacksize + n); @@ -202,11 +202,11 @@ void luaD_growstack(lua_State* L, int n) CallInfo* luaD_growCI(lua_State* L) { - /* allow extra stack space to handle stack overflow in xpcall */ + // allow extra stack space to handle stack overflow in xpcall const int hardlimit = LUAI_MAXCALLS + (LUAI_MAXCALLS >> 3); if (L->size_ci >= hardlimit) - luaD_throw(L, LUA_ERRERR); /* error while handling stack error */ + luaD_throw(L, LUA_ERRERR); // error while handling stack error int request = L->size_ci * 2; luaD_reallocCI(L, L->size_ci >= LUAI_MAXCALLS ? hardlimit : request < LUAI_MAXCALLS ? request : LUAI_MAXCALLS); @@ -219,13 +219,13 @@ CallInfo* luaD_growCI(lua_State* L) void luaD_checkCstack(lua_State* L) { - /* allow extra stack space to handle stack overflow in xpcall */ + // allow extra stack space to handle stack overflow in xpcall const int hardlimit = LUAI_MAXCCALLS + (LUAI_MAXCCALLS >> 3); if (L->nCcalls == LUAI_MAXCCALLS) luaG_runerror(L, "C stack overflow"); else if (L->nCcalls >= hardlimit) - luaD_throw(L, LUA_ERRERR); /* error while handling stack error */ + luaD_throw(L, LUA_ERRERR); // error while handling stack error } /* @@ -240,14 +240,14 @@ void luaD_call(lua_State* L, StkId func, int nResults) luaD_checkCstack(L); if (luau_precall(L, func, nResults) == PCRLUA) - { /* is a Lua function? */ - L->ci->flags |= LUA_CALLINFO_RETURN; /* luau_execute will stop after returning from the stack frame */ + { // is a Lua function? + L->ci->flags |= LUA_CALLINFO_RETURN; // luau_execute will stop after returning from the stack frame int oldactive = luaC_threadactive(L); l_setbit(L->stackstate, THREAD_ACTIVEBIT); luaC_checkthreadsleep(L); - luau_execute(L); /* call it */ + luau_execute(L); // call it if (!oldactive) resetbit(L->stackstate, THREAD_ACTIVEBIT); @@ -263,18 +263,18 @@ static void seterrorobj(lua_State* L, int errcode, StkId oldtop) { case LUA_ERRMEM: { - setsvalue2s(L, oldtop, luaS_newliteral(L, LUA_MEMERRMSG)); /* can not fail because string is pinned in luaopen */ + setsvalue2s(L, oldtop, luaS_newliteral(L, LUA_MEMERRMSG)); // can not fail because string is pinned in luaopen break; } case LUA_ERRERR: { - setsvalue2s(L, oldtop, luaS_newliteral(L, LUA_ERRERRMSG)); /* can not fail because string is pinned in luaopen */ + setsvalue2s(L, oldtop, luaS_newliteral(L, LUA_ERRERRMSG)); // can not fail because string is pinned in luaopen break; } case LUA_ERRSYNTAX: case LUA_ERRRUN: { - setobjs2s(L, oldtop, L->top - 1); /* error message on current top */ + setobjs2s(L, oldtop, L->top - 1); // error message on current top break; } } @@ -430,8 +430,8 @@ static void resume_finish(lua_State* L, int status) resetbit(L->stackstate, THREAD_ACTIVEBIT); if (status != 0) - { /* error? */ - L->status = cast_byte(status); /* mark thread as `dead' */ + { // error? + L->status = cast_byte(status); // mark thread as `dead' seterrorobj(L, status, L->top); L->ci->top = L->top; } @@ -503,7 +503,7 @@ int lua_yield(lua_State* L, int nresults) { if (L->nCcalls > L->baseCcalls) luaG_runerror(L, "attempt to yield across metamethod/C-call boundary"); - L->base = L->top - nresults; /* protect stack slots below */ + L->base = L->top - nresults; // protect stack slots below L->status = LUA_YIELD; return -1; } @@ -535,9 +535,9 @@ static void restore_stack_limit(lua_State* L) { LUAU_ASSERT(L->stack_last - L->stack == L->stacksize - EXTRA_STACK); if (L->size_ci > LUAI_MAXCALLS) - { /* there was an overflow? */ + { // there was an overflow? int inuse = cast_int(L->ci - L->base_ci); - if (inuse + 1 < LUAI_MAXCALLS) /* can `undo' overflow? */ + if (inuse + 1 < LUAI_MAXCALLS) // can `undo' overflow? luaD_reallocCI(L, LUAI_MAXCALLS); } } @@ -576,7 +576,7 @@ int luaD_pcall(lua_State* L, Pfunc func, void* u, ptrdiff_t old_top, ptrdiff_t e } StkId oldtop = restorestack(L, old_top); - luaF_close(L, oldtop); /* close eventual pending closures */ + luaF_close(L, oldtop); // close eventual pending closures seterrorobj(L, status, oldtop); L->ci = restoreci(L, old_ci); L->base = L->ci->base; diff --git a/VM/src/ldo.h b/VM/src/ldo.h index 5e9472bf..eac9927c 100644 --- a/VM/src/ldo.h +++ b/VM/src/ldo.h @@ -34,12 +34,12 @@ #define saveci(L, p) ((char*)(p) - (char*)L->base_ci) #define restoreci(L, n) ((CallInfo*)((char*)L->base_ci + (n))) -/* results from luaD_precall */ -#define PCRLUA 0 /* initiated a call to a Lua function */ -#define PCRC 1 /* did a call to a C function */ -#define PCRYIELD 2 /* C function yielded */ +// results from luaD_precall +#define PCRLUA 0 // initiated a call to a Lua function +#define PCRC 1 // did a call to a C function +#define PCRYIELD 2 // C function yielded -/* type of protected functions, to be ran by `runprotected' */ +// type of protected functions, to be ran by `runprotected' typedef void (*Pfunc)(lua_State* L, void* ud); LUAI_FUNC CallInfo* luaD_growCI(lua_State* L); diff --git a/VM/src/lfunc.cpp b/VM/src/lfunc.cpp index 66447a95..dfde6dcb 100644 --- a/VM/src/lfunc.cpp +++ b/VM/src/lfunc.cpp @@ -73,20 +73,20 @@ UpVal* luaF_findupval(lua_State* L, StkId level) { LUAU_ASSERT(p->v != &p->u.value); if (p->v == level) - { /* found a corresponding upvalue? */ - if (isdead(g, obj2gco(p))) /* is it dead? */ - changewhite(obj2gco(p)); /* resurrect it */ + { // found a corresponding upvalue? + if (isdead(g, obj2gco(p))) // is it dead? + changewhite(obj2gco(p)); // resurrect it return p; } pp = &p->u.l.threadnext; } - UpVal* uv = luaM_newgco(L, UpVal, sizeof(UpVal), L->activememcat); /* not found: create a new one */ + UpVal* uv = luaM_newgco(L, UpVal, sizeof(UpVal), L->activememcat); // not found: create a new one uv->tt = LUA_TUPVAL; uv->marked = luaC_white(g); uv->memcat = L->activememcat; - uv->v = level; /* current value lives in the stack */ + uv->v = level; // current value lives in the stack // chain the upvalue in the threads open upvalue list at the proper position UpVal* next = *pp; @@ -121,9 +121,9 @@ void luaF_unlinkupval(UpVal* uv) void luaF_freeupval(lua_State* L, UpVal* uv, lua_Page* page) { - if (uv->v != &uv->u.value) /* is it open? */ - luaF_unlinkupval(uv); /* remove from open list */ - luaM_freegco(L, uv, sizeof(UpVal), uv->memcat, page); /* free upvalue */ + if (uv->v != &uv->u.value) // is it open? + luaF_unlinkupval(uv); // remove from open list + luaM_freegco(L, uv, sizeof(UpVal), uv->memcat, page); // free upvalue } void luaF_close(lua_State* L, StkId level) @@ -179,11 +179,11 @@ const LocVar* luaF_getlocal(const Proto* f, int local_number, int pc) for (i = 0; i < f->sizelocvars; i++) { if (pc >= f->locvars[i].startpc && pc < f->locvars[i].endpc) - { /* is variable active? */ + { // is variable active? local_number--; if (local_number == 0) return &f->locvars[i]; } } - return NULL; /* not found */ + return NULL; // not found } diff --git a/VM/src/lgc.cpp b/VM/src/lgc.cpp index 70b4dbf9..f7a851f4 100644 --- a/VM/src/lgc.cpp +++ b/VM/src/lgc.cpp @@ -125,7 +125,7 @@ static void removeentry(LuaNode* n) { LUAU_ASSERT(ttisnil(gval(n))); if (iscollectable(gkey(n))) - setttype(gkey(n), LUA_TDEADKEY); /* dead key; remove it */ + setttype(gkey(n), LUA_TDEADKEY); // dead key; remove it } static void reallymarkobject(global_State* g, GCObject* o) @@ -141,7 +141,7 @@ static void reallymarkobject(global_State* g, GCObject* o) case LUA_TUSERDATA: { Table* mt = gco2u(o)->metatable; - gray2black(o); /* udata are never gray */ + gray2black(o); // udata are never gray if (mt) markobject(g, mt); return; @@ -150,8 +150,8 @@ static void reallymarkobject(global_State* g, GCObject* o) { UpVal* uv = gco2uv(o); markvalue(g, uv->v); - if (uv->v == &uv->u.value) /* closed? */ - gray2black(o); /* open upvalues are never black */ + if (uv->v == &uv->u.value) // closed? + gray2black(o); // open upvalues are never black return; } case LUA_TFUNCTION: @@ -201,15 +201,15 @@ static int traversetable(global_State* g, Table* h) if (h->metatable) markobject(g, cast_to(Table*, h->metatable)); - /* is there a weak mode? */ + // is there a weak mode? if (const char* modev = gettablemode(g, h)) { weakkey = (strchr(modev, 'k') != NULL); weakvalue = (strchr(modev, 'v') != NULL); if (weakkey || weakvalue) - { /* is really weak? */ - h->gclist = g->weak; /* must be cleared after GC, ... */ - g->weak = obj2gco(h); /* ... so put in the appropriate list */ + { // is really weak? + h->gclist = g->weak; // must be cleared after GC, ... + g->weak = obj2gco(h); // ... so put in the appropriate list } } @@ -227,7 +227,7 @@ static int traversetable(global_State* g, Table* h) LuaNode* n = gnode(h, i); LUAU_ASSERT(ttype(gkey(n)) != LUA_TDEADKEY || ttisnil(gval(n))); if (ttisnil(gval(n))) - removeentry(n); /* remove empty entries */ + removeentry(n); // remove empty entries else { LUAU_ASSERT(!ttisnil(gkey(n))); @@ -251,20 +251,20 @@ static void traverseproto(global_State* g, Proto* f) stringmark(f->source); if (f->debugname) stringmark(f->debugname); - for (i = 0; i < f->sizek; i++) /* mark literals */ + for (i = 0; i < f->sizek; i++) // mark literals markvalue(g, &f->k[i]); for (i = 0; i < f->sizeupvalues; i++) - { /* mark upvalue names */ + { // mark upvalue names if (f->upvalues[i]) stringmark(f->upvalues[i]); } for (i = 0; i < f->sizep; i++) - { /* mark nested protos */ + { // mark nested protos if (f->p[i]) markobject(g, f->p[i]); } for (i = 0; i < f->sizelocvars; i++) - { /* mark local-variable names */ + { // mark local-variable names if (f->locvars[i].varname) stringmark(f->locvars[i].varname); } @@ -276,7 +276,7 @@ static void traverseclosure(global_State* g, Closure* cl) if (cl->isC) { int i; - for (i = 0; i < cl->nupvalues; i++) /* mark its upvalues */ + for (i = 0; i < cl->nupvalues; i++) // mark its upvalues markvalue(g, &cl->c.upvals[i]); } else @@ -284,7 +284,7 @@ static void traverseclosure(global_State* g, Closure* cl) int i; LUAU_ASSERT(cl->nupvalues == cl->l.p->nups); markobject(g, cast_to(Proto*, cl->l.p)); - for (i = 0; i < cl->nupvalues; i++) /* mark its upvalues */ + for (i = 0; i < cl->nupvalues; i++) // mark its upvalues markvalue(g, &cl->l.uprefs[i]); } } @@ -296,11 +296,11 @@ static void traversestack(global_State* g, lua_State* l, bool clearstack) stringmark(l->namecall); for (StkId o = l->stack; o < l->top; o++) markvalue(g, o); - /* final traversal? */ + // final traversal? if (g->gcstate == GCSatomic || clearstack) { StkId stack_end = l->stack + l->stacksize; - for (StkId o = l->top; o < stack_end; o++) /* clear not-marked stack slice */ + for (StkId o = l->top; o < stack_end; o++) // clear not-marked stack slice setnilvalue(o); } } @@ -320,8 +320,8 @@ static size_t propagatemark(global_State* g) { Table* h = gco2h(o); g->gray = h->gclist; - if (traversetable(g, h)) /* table is weak? */ - black2gray(o); /* keep it gray */ + if (traversetable(g, h)) // table is weak? + black2gray(o); // keep it gray return sizeof(Table) + sizeof(TValue) * h->sizearray + sizeof(LuaNode) * sizenode(h); } case LUA_TFUNCTION: @@ -393,7 +393,7 @@ static int isobjcleared(GCObject* o) { if (o->gch.tt == LUA_TSTRING) { - stringmark(&o->ts); /* strings are `values', so are never weak */ + stringmark(&o->ts); // strings are `values', so are never weak return 0; } @@ -417,8 +417,8 @@ static size_t cleartable(lua_State* L, GCObject* l) while (i--) { TValue* o = &h->array[i]; - if (iscleared(o)) /* value was collected? */ - setnilvalue(o); /* remove value */ + if (iscleared(o)) // value was collected? + setnilvalue(o); // remove value } i = sizenode(h); int activevalues = 0; @@ -432,8 +432,8 @@ static size_t cleartable(lua_State* L, GCObject* l) // can we clear key or value? if (iscleared(gkey(n)) || iscleared(gval(n))) { - setnilvalue(gval(n)); /* remove value ... */ - removeentry(n); /* remove entry from table */ + setnilvalue(gval(n)); // remove value ... + removeentry(n); // remove entry from table } else { @@ -460,7 +460,7 @@ static size_t cleartable(lua_State* L, GCObject* l) static void shrinkstack(lua_State* L) { - /* compute used stack - note that we can't use th->top if we're in the middle of vararg call */ + // compute used stack - note that we can't use th->top if we're in the middle of vararg call StkId lim = L->top; for (CallInfo* ci = L->base_ci; ci <= L->ci; ci++) { @@ -469,16 +469,16 @@ static void shrinkstack(lua_State* L) lim = ci->top; } - /* shrink stack and callinfo arrays if we aren't using most of the space */ - int ci_used = cast_int(L->ci - L->base_ci); /* number of `ci' in use */ - int s_used = cast_int(lim - L->stack); /* part of stack in use */ - if (L->size_ci > LUAI_MAXCALLS) /* handling overflow? */ - return; /* do not touch the stacks */ + // shrink stack and callinfo arrays if we aren't using most of the space + int ci_used = cast_int(L->ci - L->base_ci); // number of `ci' in use + int s_used = cast_int(lim - L->stack); // part of stack in use + if (L->size_ci > LUAI_MAXCALLS) // handling overflow? + return; // do not touch the stacks if (3 * ci_used < L->size_ci && 2 * BASIC_CI_SIZE < L->size_ci) - luaD_reallocCI(L, L->size_ci / 2); /* still big enough... */ + luaD_reallocCI(L, L->size_ci / 2); // still big enough... condhardstacktests(luaD_reallocCI(L, ci_used + 1)); if (3 * s_used < L->stacksize && 2 * (BASIC_STACK_SIZE + EXTRA_STACK) < L->stacksize) - luaD_reallocstack(L, L->stacksize / 2); /* still big enough... */ + luaD_reallocstack(L, L->stacksize / 2); // still big enough... condhardstacktests(luaD_reallocstack(L, s_used)); } @@ -516,20 +516,20 @@ static void freeobj(lua_State* L, GCObject* o, lua_Page* page) static void shrinkbuffers(lua_State* L) { global_State* g = L->global; - /* check size of string hash */ + // check size of string hash if (g->strt.nuse < cast_to(uint32_t, g->strt.size / 4) && g->strt.size > LUA_MINSTRTABSIZE * 2) - luaS_resize(L, g->strt.size / 2); /* table is too big */ + luaS_resize(L, g->strt.size / 2); // table is too big } static void shrinkbuffersfull(lua_State* L) { global_State* g = L->global; - /* check size of string hash */ + // check size of string hash int hashsize = g->strt.size; while (g->strt.nuse < cast_to(uint32_t, hashsize / 4) && hashsize > LUA_MINSTRTABSIZE * 2) hashsize /= 2; if (hashsize != g->strt.size) - luaS_resize(L, hashsize); /* table is too big */ + luaS_resize(L, hashsize); // table is too big } static bool deletegco(void* context, lua_Page* page, GCObject* gco) @@ -562,7 +562,7 @@ void luaC_freeall(lua_State* L) luaM_visitgco(L, L, deletegco); - for (int i = 0; i < g->strt.size; i++) /* free all string lists */ + for (int i = 0; i < g->strt.size; i++) // free all string lists LUAU_ASSERT(g->strt.hash[i] == NULL); LUAU_ASSERT(L->global->strt.nuse == 0); @@ -577,7 +577,7 @@ static void markmt(global_State* g) markobject(g, g->mt[i]); } -/* mark root set */ +// mark root set static void markroot(lua_State* L) { global_State* g = L->global; @@ -585,7 +585,7 @@ static void markroot(lua_State* L) g->grayagain = NULL; g->weak = NULL; markobject(g, g->mainthread); - /* make global table be traversed before main stack */ + // make global table be traversed before main stack markobject(g, g->mainthread->gt); markvalue(g, registry(L)); markmt(g); @@ -616,28 +616,28 @@ static size_t atomic(lua_State* L) double currts = lua_clock(); #endif - /* remark occasional upvalues of (maybe) dead threads */ + // remark occasional upvalues of (maybe) dead threads work += remarkupvals(g); - /* traverse objects caught by write barrier and by 'remarkupvals' */ + // traverse objects caught by write barrier and by 'remarkupvals' work += propagateall(g); #ifdef LUAI_GCMETRICS g->gcmetrics.currcycle.atomictimeupval += recordGcDeltaTime(currts); #endif - /* remark weak tables */ + // remark weak tables g->gray = g->weak; g->weak = NULL; LUAU_ASSERT(!iswhite(obj2gco(g->mainthread))); - markobject(g, L); /* mark running thread */ - markmt(g); /* mark basic metatables (again) */ + markobject(g, L); // mark running thread + markmt(g); // mark basic metatables (again) work += propagateall(g); #ifdef LUAI_GCMETRICS g->gcmetrics.currcycle.atomictimeweak += recordGcDeltaTime(currts); #endif - /* remark gray again */ + // remark gray again g->gray = g->grayagain; g->grayagain = NULL; work += propagateall(g); @@ -646,7 +646,7 @@ static size_t atomic(lua_State* L) g->gcmetrics.currcycle.atomictimegray += recordGcDeltaTime(currts); #endif - /* remove collected objects from weak tables */ + // remove collected objects from weak tables work += cleartable(L, g->weak); g->weak = NULL; @@ -654,7 +654,7 @@ static size_t atomic(lua_State* L) g->gcmetrics.currcycle.atomictimeclear += recordGcDeltaTime(currts); #endif - /* flip current white */ + // flip current white g->currentwhite = cast_byte(otherwhite(g)); g->sweepgcopage = g->allgcopages; g->gcstate = GCSsweep; @@ -733,7 +733,7 @@ static size_t gcstep(lua_State* L, size_t limit) { case GCSpause: { - markroot(L); /* start a new collection */ + markroot(L); // start a new collection LUAU_ASSERT(g->gcstate == GCSpropagate); break; } @@ -765,7 +765,7 @@ static size_t gcstep(lua_State* L, size_t limit) cost += propagatemark(g); } - if (!g->gray) /* no more `gray' objects */ + if (!g->gray) // no more `gray' objects { #ifdef LUAI_GCMETRICS g->gcmetrics.currcycle.propagateagainwork = @@ -786,7 +786,7 @@ static size_t gcstep(lua_State* L, size_t limit) g->gcstats.atomicstarttimestamp = lua_clock(); g->gcstats.atomicstarttotalsizebytes = g->totalbytes; - cost = atomic(L); /* finish mark phase */ + cost = atomic(L); // finish mark phase LUAU_ASSERT(g->gcstate == GCSsweep); break; @@ -810,7 +810,7 @@ static size_t gcstep(lua_State* L, size_t limit) sweepgco(L, NULL, obj2gco(g->mainthread)); shrinkbuffers(L); - g->gcstate = GCSpause; /* end collection */ + g->gcstate = GCSpause; // end collection } break; } @@ -878,7 +878,7 @@ size_t luaC_step(lua_State* L, bool assist) { global_State* g = L->global; - int lim = g->gcstepsize * g->gcstepmul / 100; /* how much to work */ + int lim = g->gcstepsize * g->gcstepmul / 100; // how much to work LUAU_ASSERT(g->totalbytes >= g->GCthreshold); size_t debt = g->totalbytes - g->GCthreshold; @@ -947,16 +947,16 @@ void luaC_fullgc(lua_State* L) if (g->gcstate <= GCSatomic) { - /* reset sweep marks to sweep all elements (returning them to white) */ + // reset sweep marks to sweep all elements (returning them to white) g->sweepgcopage = g->allgcopages; - /* reset other collector lists */ + // reset other collector lists g->gray = NULL; g->grayagain = NULL; g->weak = NULL; g->gcstate = GCSsweep; } LUAU_ASSERT(g->gcstate == GCSsweep); - /* finish any pending sweep phase */ + // finish any pending sweep phase while (g->gcstate != GCSpause) { LUAU_ASSERT(g->gcstate == GCSsweep); @@ -968,13 +968,13 @@ void luaC_fullgc(lua_State* L) startGcCycleMetrics(g); #endif - /* run a full collection cycle */ + // run a full collection cycle markroot(L); while (g->gcstate != GCSpause) { gcstep(L, SIZE_MAX); } - /* reclaim as much buffer memory as possible (shrinkbuffers() called during sweep is incremental) */ + // reclaim as much buffer memory as possible (shrinkbuffers() called during sweep is incremental) shrinkbuffersfull(L); size_t heapgoalsizebytes = (g->totalbytes / 100) * g->gcgoal; @@ -1011,11 +1011,11 @@ void luaC_barrierf(lua_State* L, GCObject* o, GCObject* v) global_State* g = L->global; LUAU_ASSERT(isblack(o) && iswhite(v) && !isdead(g, v) && !isdead(g, o)); LUAU_ASSERT(g->gcstate != GCSpause); - /* must keep invariant? */ + // must keep invariant? if (keepinvariant(g)) - reallymarkobject(g, v); /* restore invariant */ - else /* don't mind */ - makewhite(g, o); /* mark as white just to avoid other barriers */ + reallymarkobject(g, v); // restore invariant + else // don't mind + makewhite(g, o); // mark as white just to avoid other barriers } void luaC_barriertable(lua_State* L, Table* t, GCObject* v) @@ -1033,7 +1033,7 @@ void luaC_barriertable(lua_State* L, Table* t, GCObject* v) LUAU_ASSERT(isblack(o) && !isdead(g, o)); LUAU_ASSERT(g->gcstate != GCSpause); - black2gray(o); /* make table gray (again) */ + black2gray(o); // make table gray (again) t->gclist = g->grayagain; g->grayagain = o; } @@ -1044,7 +1044,7 @@ void luaC_barrierback(lua_State* L, Table* t) GCObject* o = obj2gco(t); LUAU_ASSERT(isblack(o) && !isdead(g, o)); LUAU_ASSERT(g->gcstate != GCSpause); - black2gray(o); /* make table gray (again) */ + black2gray(o); // make table gray (again) t->gclist = g->grayagain; g->grayagain = o; } @@ -1066,11 +1066,11 @@ void luaC_initupval(lua_State* L, UpVal* uv) { if (keepinvariant(g)) { - gray2black(o); /* closed upvalues need barrier */ + gray2black(o); // closed upvalues need barrier luaC_barrier(L, uv, uv->v); } else - { /* sweep phase: sweep it (turning it into white) */ + { // sweep phase: sweep it (turning it into white) makewhite(g, o); LUAU_ASSERT(g->gcstate != GCSpause); } diff --git a/VM/src/lgc.h b/VM/src/lgc.h index 797284a2..7b03a25d 100644 --- a/VM/src/lgc.h +++ b/VM/src/lgc.h @@ -9,9 +9,9 @@ /* ** Default settings for GC tunables (settable via lua_gc) */ -#define LUAI_GCGOAL 200 /* 200% (allow heap to double compared to live heap size) */ -#define LUAI_GCSTEPMUL 200 /* GC runs 'twice the speed' of memory allocation */ -#define LUAI_GCSTEPSIZE 1 /* GC runs every KB of memory allocation */ +#define LUAI_GCGOAL 200 // 200% (allow heap to double compared to live heap size) +#define LUAI_GCSTEPMUL 200 // GC runs 'twice the speed' of memory allocation +#define LUAI_GCSTEPSIZE 1 // GC runs every KB of memory allocation /* ** Possible states of the Garbage Collector diff --git a/VM/src/lgcdebug.cpp b/VM/src/lgcdebug.cpp index 2b38619b..bc997d44 100644 --- a/VM/src/lgcdebug.cpp +++ b/VM/src/lgcdebug.cpp @@ -19,7 +19,7 @@ static void validateobjref(global_State* g, GCObject* f, GCObject* t) if (keepinvariant(g)) { - /* basic incremental invariant: black can't point to white */ + // basic incremental invariant: black can't point to white LUAU_ASSERT(!(isblack(f) && iswhite(t))); } } @@ -135,7 +135,7 @@ static void validateproto(global_State* g, Proto* f) static void validateobj(global_State* g, GCObject* o) { - /* dead objects can only occur during sweep */ + // dead objects can only occur during sweep if (isdead(g, o)) { LUAU_ASSERT(g->gcstate == GCSsweep); diff --git a/VM/src/lmathlib.cpp b/VM/src/lmathlib.cpp index a6e7b494..0693b846 100644 --- a/VM/src/lmathlib.cpp +++ b/VM/src/lmathlib.cpp @@ -195,7 +195,7 @@ static int math_ldexp(lua_State* L) static int math_min(lua_State* L) { - int n = lua_gettop(L); /* number of arguments */ + int n = lua_gettop(L); // number of arguments double dmin = luaL_checknumber(L, 1); int i; for (i = 2; i <= n; i++) @@ -210,7 +210,7 @@ static int math_min(lua_State* L) static int math_max(lua_State* L) { - int n = lua_gettop(L); /* number of arguments */ + int n = lua_gettop(L); // number of arguments double dmax = luaL_checknumber(L, 1); int i; for (i = 2; i <= n; i++) @@ -227,29 +227,29 @@ static int math_random(lua_State* L) { global_State* g = L->global; switch (lua_gettop(L)) - { /* check number of arguments */ + { // check number of arguments case 0: - { /* no arguments */ + { // no arguments // Using ldexp instead of division for speed & clarity. // See http://mumble.net/~campbell/tmp/random_real.c for details on generating doubles from integer ranges. uint32_t rl = pcg32_random(&g->rngstate); uint32_t rh = pcg32_random(&g->rngstate); double rd = ldexp(double(rl | (uint64_t(rh) << 32)), -64); - lua_pushnumber(L, rd); /* number between 0 and 1 */ + lua_pushnumber(L, rd); // number between 0 and 1 break; } case 1: - { /* only upper limit */ + { // only upper limit int u = luaL_checkinteger(L, 1); luaL_argcheck(L, 1 <= u, 1, "interval is empty"); uint64_t x = uint64_t(u) * pcg32_random(&g->rngstate); int r = int(1 + (x >> 32)); - lua_pushinteger(L, r); /* int between 1 and `u' */ + lua_pushinteger(L, r); // int between 1 and `u' break; } case 2: - { /* lower and upper limits */ + { // lower and upper limits int l = luaL_checkinteger(L, 1); int u = luaL_checkinteger(L, 2); luaL_argcheck(L, l <= u, 2, "interval is empty"); @@ -258,7 +258,7 @@ static int math_random(lua_State* L) luaL_argcheck(L, ul < UINT_MAX, 2, "interval is too large"); // -INT_MIN..INT_MAX interval can result in integer overflow uint64_t x = uint64_t(ul + 1) * pcg32_random(&g->rngstate); int r = int(l + (x >> 32)); - lua_pushinteger(L, r); /* int between `l' and `u' */ + lua_pushinteger(L, r); // int between `l' and `u' break; } default: diff --git a/VM/src/lnumutils.h b/VM/src/lnumutils.h index 549b4630..5b27e2b8 100644 --- a/VM/src/lnumutils.h +++ b/VM/src/lnumutils.h @@ -42,7 +42,7 @@ LUAU_FASTMATH_END #define luai_num2int(i, d) ((i) = (int)(d)) -/* On MSVC in 32-bit, double to unsigned cast compiles into a call to __dtoui3, so we invoke x87->int64 conversion path manually */ +// On MSVC in 32-bit, double to unsigned cast compiles into a call to __dtoui3, so we invoke x87->int64 conversion path manually #if defined(_MSC_VER) && defined(_M_IX86) #define luai_num2unsigned(i, n) \ { \ diff --git a/VM/src/lobject.cpp b/VM/src/lobject.cpp index d5bd76a8..b6a40bb6 100644 --- a/VM/src/lobject.cpp +++ b/VM/src/lobject.cpp @@ -48,7 +48,7 @@ int luaO_rawequalObj(const TValue* t1, const TValue* t2) case LUA_TVECTOR: return luai_veceq(vvalue(t1), vvalue(t2)); case LUA_TBOOLEAN: - return bvalue(t1) == bvalue(t2); /* boolean true must be 1 !! */ + return bvalue(t1) == bvalue(t2); // boolean true must be 1 !! case LUA_TLIGHTUSERDATA: return pvalue(t1) == pvalue(t2); default: @@ -71,7 +71,7 @@ int luaO_rawequalKey(const TKey* t1, const TValue* t2) case LUA_TVECTOR: return luai_veceq(vvalue(t1), vvalue(t2)); case LUA_TBOOLEAN: - return bvalue(t1) == bvalue(t2); /* boolean true must be 1 !! */ + return bvalue(t1) == bvalue(t2); // boolean true must be 1 !! case LUA_TLIGHTUSERDATA: return pvalue(t1) == pvalue(t2); default: @@ -85,15 +85,15 @@ int luaO_str2d(const char* s, double* result) char* endptr; *result = luai_str2num(s, &endptr); if (endptr == s) - return 0; /* conversion failed */ - if (*endptr == 'x' || *endptr == 'X') /* maybe an hexadecimal constant? */ + return 0; // conversion failed + if (*endptr == 'x' || *endptr == 'X') // maybe an hexadecimal constant? *result = cast_num(strtoul(s, &endptr, 16)); if (*endptr == '\0') - return 1; /* most common case */ + return 1; // most common case while (isspace(cast_to(unsigned char, *endptr))) endptr++; if (*endptr != '\0') - return 0; /* invalid trailing characters? */ + return 0; // invalid trailing characters? return 1; } @@ -121,7 +121,7 @@ void luaO_chunkid(char* out, const char* source, size_t bufflen) { if (*source == '=') { - source++; /* skip the `=' */ + source++; // skip the `=' size_t srclen = strlen(source); size_t dstlen = srclen < bufflen ? srclen : bufflen - 1; memcpy(out, source, dstlen); @@ -130,26 +130,26 @@ void luaO_chunkid(char* out, const char* source, size_t bufflen) else if (*source == '@') { size_t l; - source++; /* skip the `@' */ + source++; // skip the `@' bufflen -= sizeof("..."); l = strlen(source); strcpy(out, ""); if (l > bufflen) { - source += (l - bufflen); /* get last part of file name */ + source += (l - bufflen); // get last part of file name strcat(out, "..."); } strcat(out, source); } else - { /* out = [string "string"] */ - size_t len = strcspn(source, "\n\r"); /* stop at first newline */ + { // out = [string "string"] + size_t len = strcspn(source, "\n\r"); // stop at first newline bufflen -= sizeof("[string \"...\"]"); if (len > bufflen) len = bufflen; strcpy(out, "[string \""); if (source[len] != '\0') - { /* must truncate? */ + { // must truncate? strncat(out, source, len); strcat(out, "..."); } diff --git a/VM/src/lobject.h b/VM/src/lobject.h index bdcb85cb..2097e335 100644 --- a/VM/src/lobject.h +++ b/VM/src/lobject.h @@ -49,7 +49,7 @@ typedef struct lua_TValue int tt; } TValue; -/* Macros to test type */ +// Macros to test type #define ttisnil(o) (ttype(o) == LUA_TNIL) #define ttisnumber(o) (ttype(o) == LUA_TNUMBER) #define ttisstring(o) (ttype(o) == LUA_TSTRING) @@ -62,7 +62,7 @@ typedef struct lua_TValue #define ttisvector(o) (ttype(o) == LUA_TVECTOR) #define ttisupval(o) (ttype(o) == LUA_TUPVAL) -/* Macros to access values */ +// Macros to access values #define ttype(o) ((o)->tt) #define gcvalue(o) check_exp(iscollectable(o), (o)->value.gc) #define pvalue(o) check_exp(ttislightuserdata(o), (o)->value.p) @@ -85,7 +85,7 @@ typedef struct lua_TValue #define checkliveness(g, obj) LUAU_ASSERT(!iscollectable(obj) || ((ttype(obj) == (obj)->value.gc->gch.tt) && !isdead(g, (obj)->value.gc))) -/* Macros to set values */ +// Macros to set values #define setnilvalue(obj) ((obj)->tt = LUA_TNIL) #define setnvalue(obj, x) \ @@ -200,18 +200,18 @@ typedef struct lua_TValue ** different types of sets, according to destination */ -/* from stack to (same) stack */ +// from stack to (same) stack #define setobjs2s setobj -/* to stack (not from same stack) */ +// to stack (not from same stack) #define setobj2s setobj #define setsvalue2s setsvalue #define sethvalue2s sethvalue #define setptvalue2s setptvalue -/* from table to same table */ +// from table to same table #define setobjt2t setobj -/* to table */ +// to table #define setobj2t setobj -/* to new object */ +// to new object #define setobj2n setobj #define setsvalue2n setsvalue @@ -219,7 +219,7 @@ typedef struct lua_TValue #define iscollectable(o) (ttype(o) >= LUA_TSTRING) -typedef TValue* StkId; /* index to stack elements */ +typedef TValue* StkId; // index to stack elements /* ** String headers for string table @@ -269,13 +269,13 @@ typedef struct Proto CommonHeader; - TValue* k; /* constants used by the function */ - Instruction* code; /* function bytecode */ - struct Proto** p; /* functions defined inside the function */ - uint8_t* lineinfo; /* for each instruction, line number as a delta from baseline */ - int* abslineinfo; /* baseline line info, one entry for each 1<global, i_o); \ } -/* copy a value from a key */ +// copy a value from a key #define getnodekey(L, obj, node) \ { \ TValue* i_o = (obj); \ @@ -418,22 +418,22 @@ typedef struct Table CommonHeader; - uint8_t tmcache; /* 1<

tm_sec); setfield(L, "min", stm->tm_min); setfield(L, "hour", stm->tm_hour); @@ -122,7 +122,7 @@ static int os_date(lua_State* L) luaL_buffinit(L, &b); for (; *s; s++) { - if (*s != '%' || *(s + 1) == '\0') /* no conversion specifier? */ + if (*s != '%' || *(s + 1) == '\0') // no conversion specifier? { luaL_addchar(&b, *s); } @@ -133,7 +133,7 @@ static int os_date(lua_State* L) else { size_t reslen; - char buff[200]; /* should be big enough for any conversion result */ + char buff[200]; // should be big enough for any conversion result cc[1] = *(++s); reslen = strftime(buff, sizeof(buff), cc, stm); luaL_addlstring(&b, buff, reslen); @@ -147,13 +147,13 @@ static int os_date(lua_State* L) static int os_time(lua_State* L) { time_t t; - if (lua_isnoneornil(L, 1)) /* called without args? */ - t = time(NULL); /* get current time */ + if (lua_isnoneornil(L, 1)) // called without args? + t = time(NULL); // get current time else { struct tm ts; luaL_checktype(L, 1, LUA_TTABLE); - lua_settop(L, 1); /* make sure table is at the top */ + lua_settop(L, 1); // make sure table is at the top ts.tm_sec = getfield(L, "sec", 0); ts.tm_min = getfield(L, "min", 0); ts.tm_hour = getfield(L, "hour", 12); diff --git a/VM/src/lstate.cpp b/VM/src/lstate.cpp index fbc6fb1e..4489f840 100644 --- a/VM/src/lstate.cpp +++ b/VM/src/lstate.cpp @@ -21,22 +21,22 @@ typedef struct LG static void stack_init(lua_State* L1, lua_State* L) { - /* initialize CallInfo array */ + // initialize CallInfo array L1->base_ci = luaM_newarray(L, BASIC_CI_SIZE, CallInfo, L1->memcat); L1->ci = L1->base_ci; L1->size_ci = BASIC_CI_SIZE; L1->end_ci = L1->base_ci + L1->size_ci - 1; - /* initialize stack array */ + // initialize stack array L1->stack = luaM_newarray(L, BASIC_STACK_SIZE + EXTRA_STACK, TValue, L1->memcat); L1->stacksize = BASIC_STACK_SIZE + EXTRA_STACK; TValue* stack = L1->stack; for (int i = 0; i < BASIC_STACK_SIZE + EXTRA_STACK; i++) - setnilvalue(stack + i); /* erase new stack */ + setnilvalue(stack + i); // erase new stack L1->top = stack; L1->stack_last = stack + (L1->stacksize - EXTRA_STACK); - /* initialize first ci */ + // initialize first ci L1->ci->func = L1->top; - setnilvalue(L1->top++); /* `function' entry for this `ci' */ + setnilvalue(L1->top++); // `function' entry for this `ci' L1->base = L1->ci->base = L1->top; L1->ci->top = L1->top + LUA_MINSTACK; } @@ -53,13 +53,13 @@ static void freestack(lua_State* L, lua_State* L1) static void f_luaopen(lua_State* L, void* ud) { global_State* g = L->global; - stack_init(L, L); /* init stack */ - L->gt = luaH_new(L, 0, 2); /* table of globals */ - sethvalue(L, registry(L), luaH_new(L, 0, 2)); /* registry */ - luaS_resize(L, LUA_MINSTRTABSIZE); /* initial size of string table */ + stack_init(L, L); // init stack + L->gt = luaH_new(L, 0, 2); // table of globals + sethvalue(L, registry(L), luaH_new(L, 0, 2)); // registry + luaS_resize(L, LUA_MINSTRTABSIZE); // initial size of string table luaT_init(L); - luaS_fix(luaS_newliteral(L, LUA_MEMERRMSG)); /* pin to make sure we can always throw this error */ - luaS_fix(luaS_newliteral(L, LUA_ERRERRMSG)); /* pin to make sure we can always throw this error */ + luaS_fix(luaS_newliteral(L, LUA_MEMERRMSG)); // pin to make sure we can always throw this error + luaS_fix(luaS_newliteral(L, LUA_ERRERRMSG)); // pin to make sure we can always throw this error g->GCthreshold = 4 * g->totalbytes; } @@ -85,8 +85,8 @@ static void preinit_state(lua_State* L, global_State* g) static void close_state(lua_State* L) { global_State* g = L->global; - luaF_close(L, L->stack); /* close all upvalues for this thread */ - luaC_freeall(L); /* collect all objects */ + luaF_close(L, L->stack); // close all upvalues for this thread + luaC_freeall(L); // collect all objects LUAU_ASSERT(g->strbufgc == NULL); LUAU_ASSERT(g->strt.nuse == 0); luaM_freearray(L, L->global->strt.hash, L->global->strt.size, TString*, 0); @@ -110,8 +110,8 @@ lua_State* luaE_newthread(lua_State* L) luaC_init(L, L1, LUA_TTHREAD); preinit_state(L1, L->global); L1->activememcat = L->activememcat; // inherit the active memory category - stack_init(L1, L); /* init stack */ - L1->gt = L->gt; /* share table of globals */ + stack_init(L1, L); // init stack + L1->gt = L->gt; // share table of globals L1->singlestep = L->singlestep; LUAU_ASSERT(iswhite(obj2gco(L1))); return L1; @@ -119,7 +119,7 @@ lua_State* luaE_newthread(lua_State* L) void luaE_freethread(lua_State* L, lua_State* L1, lua_Page* page) { - luaF_close(L1, L1->stack); /* close all upvalues for this thread */ + luaF_close(L1, L1->stack); // close all upvalues for this thread LUAU_ASSERT(L1->openupval == NULL); global_State* g = L->global; if (g->cb.userthread) @@ -130,9 +130,9 @@ void luaE_freethread(lua_State* L, lua_State* L1, lua_Page* page) void lua_resetthread(lua_State* L) { - /* close upvalues before clearing anything */ + // close upvalues before clearing anything luaF_close(L, L->stack); - /* clear call frames */ + // clear call frames CallInfo* ci = L->base_ci; ci->func = L->stack; ci->base = ci->func + 1; @@ -141,12 +141,12 @@ void lua_resetthread(lua_State* L) L->ci = ci; if (L->size_ci != BASIC_CI_SIZE) luaD_reallocCI(L, BASIC_CI_SIZE); - /* clear thread state */ + // clear thread state L->status = LUA_OK; L->base = L->ci->base; L->top = L->ci->base; L->nCcalls = L->baseCcalls = 0; - /* clear thread stack */ + // clear thread stack if (L->stacksize != BASIC_STACK_SIZE + EXTRA_STACK) luaD_reallocstack(L, BASIC_STACK_SIZE); for (int i = 0; i < L->stacksize; i++) @@ -177,7 +177,7 @@ lua_State* lua_newstate(lua_Alloc f, void* ud) g->mainthread = L; g->uvhead.u.l.prev = &g->uvhead; g->uvhead.u.l.next = &g->uvhead; - g->GCthreshold = 0; /* mark it as unfinished state */ + g->GCthreshold = 0; // mark it as unfinished state g->registryfree = 0; g->errorjmp = NULL; g->rngstate = 0; @@ -224,7 +224,7 @@ lua_State* lua_newstate(lua_Alloc f, void* ud) if (luaD_rawrunprotected(L, f_luaopen, NULL) != 0) { - /* memory allocation error: free partial state */ + // memory allocation error: free partial state close_state(L); L = NULL; } @@ -233,7 +233,7 @@ lua_State* lua_newstate(lua_Alloc f, void* ud) void lua_close(lua_State* L) { - L = L->global->mainthread; /* only the main thread can be closed */ - luaF_close(L, L->stack); /* close all upvalues for this thread */ + L = L->global->mainthread; // only the main thread can be closed + luaF_close(L, L->stack); // close all upvalues for this thread close_state(L); } diff --git a/VM/src/lstate.h b/VM/src/lstate.h index 423514a7..72a09713 100644 --- a/VM/src/lstate.h +++ b/VM/src/lstate.h @@ -5,10 +5,10 @@ #include "lobject.h" #include "ltm.h" -/* registry */ +// registry #define registry(L) (&L->global->registry) -/* extra stack space to handle TM calls and some other extras */ +// extra stack space to handle TM calls and some other extras #define EXTRA_STACK 5 #define BASIC_CI_SIZE 8 @@ -20,7 +20,7 @@ typedef struct stringtable { TString** hash; - uint32_t nuse; /* number of elements */ + uint32_t nuse; // number of elements int size; } stringtable; // clang-format on @@ -57,18 +57,18 @@ typedef struct stringtable typedef struct CallInfo { - StkId base; /* base for this function */ - StkId func; /* function index in the stack */ - StkId top; /* top for this function */ + StkId base; // base for this function + StkId func; // function index in the stack + StkId top; // top for this function const Instruction* savedpc; - int nresults; /* expected number of results from this function */ - unsigned int flags; /* call frame flags, see LUA_CALLINFO_* */ + int nresults; // expected number of results from this function + unsigned int flags; // call frame flags, see LUA_CALLINFO_* } CallInfo; // clang-format on -#define LUA_CALLINFO_RETURN (1 << 0) /* should the interpreter return after returning from this callinfo? first frame must have this set */ -#define LUA_CALLINFO_HANDLE (1 << 1) /* should the error thrown during execution get handled by continuation from this callinfo? func must be C */ +#define LUA_CALLINFO_RETURN (1 << 0) // should the interpreter return after returning from this callinfo? first frame must have this set +#define LUA_CALLINFO_HANDLE (1 << 1) // should the error thrown during execution get handled by continuation from this callinfo? func must be C #define curr_func(L) (clvalue(L->ci->func)) #define ci_func(ci) (clvalue((ci)->func)) @@ -152,55 +152,55 @@ struct GCMetrics // clang-format off typedef struct global_State { - stringtable strt; /* hash table for strings */ + stringtable strt; // hash table for strings - lua_Alloc frealloc; /* function to reallocate memory */ - void* ud; /* auxiliary data to `frealloc' */ + lua_Alloc frealloc; // function to reallocate memory + void* ud; // auxiliary data to `frealloc' uint8_t currentwhite; - uint8_t gcstate; /* state of garbage collector */ + uint8_t gcstate; // state of garbage collector - GCObject* gray; /* list of gray objects */ - GCObject* grayagain; /* list of objects to be traversed atomically */ - GCObject* weak; /* list of weak tables (to be cleared) */ + GCObject* gray; // list of gray objects + GCObject* grayagain; // list of objects to be traversed atomically + GCObject* weak; // list of weak tables (to be cleared) TString* strbufgc; // list of all string buffer objects - size_t GCthreshold; // when totalbytes > GCthreshold; run GC step + size_t GCthreshold; // when totalbytes > GCthreshold, run GC step size_t totalbytes; // number of bytes currently allocated int gcgoal; // see LUAI_GCGOAL int gcstepmul; // see LUAI_GCSTEPMUL int gcstepsize; // see LUAI_GCSTEPSIZE struct lua_Page* freepages[LUA_SIZECLASSES]; // free page linked list for each size class for non-collectable objects - struct lua_Page* freegcopages[LUA_SIZECLASSES]; // free page linked list for each size class for collectable objects + struct lua_Page* freegcopages[LUA_SIZECLASSES]; // free page linked list for each size class for collectable objects struct lua_Page* allgcopages; // page linked list with all pages for all classes struct lua_Page* sweepgcopage; // position of the sweep in `allgcopages' - size_t memcatbytes[LUA_MEMORY_CATEGORIES]; /* total amount of memory used by each memory category */ + size_t memcatbytes[LUA_MEMORY_CATEGORIES]; // total amount of memory used by each memory category struct lua_State* mainthread; - UpVal uvhead; /* head of double-linked list of all open upvalues */ - struct Table* mt[LUA_T_COUNT]; /* metatables for basic types */ - TString* ttname[LUA_T_COUNT]; /* names for basic types */ - TString* tmname[TM_N]; /* array with tag-method names */ + UpVal uvhead; // head of double-linked list of all open upvalues + struct Table* mt[LUA_T_COUNT]; // metatables for basic types + TString* ttname[LUA_T_COUNT]; // names for basic types + TString* tmname[TM_N]; // array with tag-method names - TValue pseudotemp; /* storage for temporary values used in pseudo2addr */ + TValue pseudotemp; // storage for temporary values used in pseudo2addr - TValue registry; /* registry table, used by lua_ref and LUA_REGISTRYINDEX */ - int registryfree; /* next free slot in registry */ + TValue registry; // registry table, used by lua_ref and LUA_REGISTRYINDEX + int registryfree; // next free slot in registry - struct lua_jmpbuf* errorjmp; /* jump buffer data for longjmp-style error handling */ + struct lua_jmpbuf* errorjmp; // jump buffer data for longjmp-style error handling - uint64_t rngstate; /* PCG random number generator state */ - uint64_t ptrenckey[4]; /* pointer encoding key for display */ + uint64_t rngstate; // PCG random number generator state + uint64_t ptrenckey[4]; // pointer encoding key for display - void (*udatagc[LUA_UTAG_LIMIT])(lua_State*, void*); /* for each userdata tag, a gc callback to be called immediately before freeing memory */ + void (*udatagc[LUA_UTAG_LIMIT])(lua_State*, void*); // for each userdata tag, a gc callback to be called immediately before freeing memory lua_Callbacks cb; @@ -221,39 +221,39 @@ struct lua_State CommonHeader; uint8_t status; - uint8_t activememcat; /* memory category that is used for new GC object allocations */ + uint8_t activememcat; // memory category that is used for new GC object allocations uint8_t stackstate; - bool singlestep; /* call debugstep hook after each instruction */ + bool singlestep; // call debugstep hook after each instruction - StkId top; /* first free slot in the stack */ - StkId base; /* base of current function */ + StkId top; // first free slot in the stack + StkId base; // base of current function global_State* global; - CallInfo* ci; /* call info for current function */ - StkId stack_last; /* last free slot in the stack */ - StkId stack; /* stack base */ + CallInfo* ci; // call info for current function + StkId stack_last; // last free slot in the stack + StkId stack; // stack base - CallInfo* end_ci; /* points after end of ci array*/ - CallInfo* base_ci; /* array of CallInfo's */ + CallInfo* end_ci; // points after end of ci array + CallInfo* base_ci; // array of CallInfo's int stacksize; - int size_ci; /* size of array `base_ci' */ + int size_ci; // size of array `base_ci' - unsigned short nCcalls; /* number of nested C calls */ - unsigned short baseCcalls; /* nested C calls when resuming coroutine */ + unsigned short nCcalls; // number of nested C calls + unsigned short baseCcalls; // nested C calls when resuming coroutine - int cachedslot; /* when table operations or INDEX/NEWINDEX is invoked from Luau, what is the expected slot for lookup? */ + int cachedslot; // when table operations or INDEX/NEWINDEX is invoked from Luau, what is the expected slot for lookup? - Table* gt; /* table of globals */ - UpVal* openupval; /* list of open upvalues in this stack */ + Table* gt; // table of globals + UpVal* openupval; // list of open upvalues in this stack GCObject* gclist; - TString* namecall; /* when invoked from Luau using NAMECALL, what method do we need to invoke? */ + TString* namecall; // when invoked from Luau using NAMECALL, what method do we need to invoke? void* userdata; }; @@ -271,10 +271,10 @@ union GCObject struct Table h; struct Proto p; struct UpVal uv; - struct lua_State th; /* thread */ + struct lua_State th; // thread }; -/* macros to convert a GCObject into a specific value */ +// macros to convert a GCObject into a specific value #define gco2ts(o) check_exp((o)->gch.tt == LUA_TSTRING, &((o)->ts)) #define gco2u(o) check_exp((o)->gch.tt == LUA_TUSERDATA, &((o)->u)) #define gco2cl(o) check_exp((o)->gch.tt == LUA_TFUNCTION, &((o)->cl)) @@ -283,7 +283,7 @@ union GCObject #define gco2uv(o) check_exp((o)->gch.tt == LUA_TUPVAL, &((o)->uv)) #define gco2th(o) check_exp((o)->gch.tt == LUA_TTHREAD, &((o)->th)) -/* macro to convert any Lua object into a GCObject */ +// macro to convert any Lua object into a GCObject #define obj2gco(v) check_exp(iscollectable(v), cast_to(GCObject*, (v) + 0)) LUAI_FUNC lua_State* luaE_newthread(lua_State* L); diff --git a/VM/src/lstring.cpp b/VM/src/lstring.cpp index d454308c..9c266031 100644 --- a/VM/src/lstring.cpp +++ b/VM/src/lstring.cpp @@ -48,17 +48,17 @@ void luaS_resize(lua_State* L, int newsize) stringtable* tb = &L->global->strt; for (int i = 0; i < newsize; i++) newhash[i] = NULL; - /* rehash */ + // rehash for (int i = 0; i < tb->size; i++) { TString* p = tb->hash[i]; while (p) - { /* for each node in the list */ - TString* next = p->next; /* save next */ + { // for each node in the list + TString* next = p->next; // save next unsigned int h = p->hash; - int h1 = lmod(h, newsize); /* new position */ + int h1 = lmod(h, newsize); // new position LUAU_ASSERT(cast_int(h % newsize) == lmod(h, newsize)); - p->next = newhash[h1]; /* chain it */ + p->next = newhash[h1]; // chain it newhash[h1] = p; p = next; } @@ -81,15 +81,15 @@ static TString* newlstr(lua_State* L, const char* str, size_t l, unsigned int h) ts->tt = LUA_TSTRING; ts->memcat = L->activememcat; memcpy(ts->data, str, l); - ts->data[l] = '\0'; /* ending 0 */ + ts->data[l] = '\0'; // ending 0 ts->atom = ATOM_UNDEF; tb = &L->global->strt; h = lmod(h, tb->size); - ts->next = tb->hash[h]; /* chain new entry */ + ts->next = tb->hash[h]; // chain new entry tb->hash[h] = ts; tb->nuse++; if (tb->nuse > cast_to(uint32_t, tb->size) && tb->size <= INT_MAX / 2) - luaS_resize(L, tb->size * 2); /* too crowded */ + luaS_resize(L, tb->size * 2); // too crowded return ts; } @@ -181,13 +181,13 @@ TString* luaS_newlstr(lua_State* L, const char* str, size_t l) { if (el->len == l && (memcmp(str, getstr(el), l) == 0)) { - /* string may be dead */ + // string may be dead if (isdead(L->global, obj2gco(el))) changewhite(obj2gco(el)); return el; } } - return newlstr(L, str, l, h); /* not found */ + return newlstr(L, str, l, h); // not found } static bool unlinkstr(lua_State* L, TString* ts) diff --git a/VM/src/lstring.h b/VM/src/lstring.h index acdf9a1e..41f9df9a 100644 --- a/VM/src/lstring.h +++ b/VM/src/lstring.h @@ -5,10 +5,10 @@ #include "lobject.h" #include "lstate.h" -/* string size limit */ +// string size limit #define MAXSSIZE (1 << 30) -/* string atoms are not defined by default; the storage is 16-bit integer */ +// string atoms are not defined by default; the storage is 16-bit integer #define ATOM_UNDEF -32768 #define sizestring(len) (offsetof(TString, data) + len + 1) diff --git a/VM/src/lstrlib.cpp b/VM/src/lstrlib.cpp index 98bbcd41..b3ea1094 100644 --- a/VM/src/lstrlib.cpp +++ b/VM/src/lstrlib.cpp @@ -10,7 +10,7 @@ LUAU_FASTFLAGVARIABLE(LuauTostringFormatSpecifier, false); -/* macro to `unsign' a character */ +// macro to `unsign' a character #define uchar(c) ((unsigned char)(c)) static int str_len(lua_State* L) @@ -23,7 +23,7 @@ static int str_len(lua_State* L) static int posrelat(int pos, size_t len) { - /* relative string position: negative means back from end */ + // relative string position: negative means back from end if (pos < 0) pos += (int)len + 1; return (pos >= 0) ? pos : 0; @@ -139,9 +139,9 @@ static int str_byte(lua_State* L) if ((size_t)pose > l) pose = (int)l; if (posi > pose) - return 0; /* empty interval; return no values */ + return 0; // empty interval; return no values n = (int)(pose - posi + 1); - if (posi + n <= pose) /* overflow? */ + if (posi + n <= pose) // overflow? luaL_error(L, "string slice too long"); luaL_checkstack(L, n, "string slice too long"); for (i = 0; i < n; i++) @@ -151,7 +151,7 @@ static int str_byte(lua_State* L) static int str_char(lua_State* L) { - int n = lua_gettop(L); /* number of arguments */ + int n = lua_gettop(L); // number of arguments luaL_Buffer b; char* ptr = luaL_buffinitsize(L, &b, n); @@ -178,12 +178,12 @@ static int str_char(lua_State* L) typedef struct MatchState { - int matchdepth; /* control for recursive depth (to avoid C stack overflow) */ - const char* src_init; /* init of source string */ - const char* src_end; /* end ('\0') of source string */ - const char* p_end; /* end ('\0') of pattern */ + int matchdepth; // control for recursive depth (to avoid C stack overflow) + const char* src_init; // init of source string + const char* src_end; // end ('\0') of source string + const char* p_end; // end ('\0') of pattern lua_State* L; - int level; /* total number of captures (finished or unfinished) */ + int level; // total number of captures (finished or unfinished) struct { const char* init; @@ -191,7 +191,7 @@ typedef struct MatchState } capture[LUA_MAXCAPTURES]; } MatchState; -/* recursive function */ +// recursive function static const char* match(MatchState* ms, const char* s, const char* p); #define L_ESC '%' @@ -229,11 +229,11 @@ static const char* classend(MatchState* ms, const char* p) if (*p == '^') p++; do - { /* look for a `]' */ + { // look for a `]' if (p == ms->p_end) luaL_error(ms->L, "malformed pattern (missing ']')"); if (*(p++) == L_ESC && p < ms->p_end) - p++; /* skip escapes (e.g. `%]') */ + p++; // skip escapes (e.g. `%]') } while (*p != ']'); return p + 1; } @@ -281,7 +281,7 @@ static int match_class(int c, int cl) break; case 'z': res = (c == 0); - break; /* deprecated option */ + break; // deprecated option default: return (cl == c); } @@ -294,7 +294,7 @@ static int matchbracketclass(int c, const char* p, const char* ec) if (*(p + 1) == '^') { sig = 0; - p++; /* skip the `^' */ + p++; // skip the `^' } while (++p < ec) { @@ -326,7 +326,7 @@ static int singlematch(MatchState* ms, const char* s, const char* p, const char* switch (*p) { case '.': - return 1; /* matches any char */ + return 1; // matches any char case L_ESC: return match_class(c, uchar(*(p + 1))); case '[': @@ -359,21 +359,21 @@ static const char* matchbalance(MatchState* ms, const char* s, const char* p) cont++; } } - return NULL; /* string ends out of balance */ + return NULL; // string ends out of balance } static const char* max_expand(MatchState* ms, const char* s, const char* p, const char* ep) { - ptrdiff_t i = 0; /* counts maximum expand for item */ + ptrdiff_t i = 0; // counts maximum expand for item while (singlematch(ms, s + i, p, ep)) i++; - /* keeps trying to match with the maximum repetitions */ + // keeps trying to match with the maximum repetitions while (i >= 0) { const char* res = match(ms, (s + i), ep + 1); if (res) return res; - i--; /* else didn't match; reduce 1 repetition to try again */ + i--; // else didn't match; reduce 1 repetition to try again } return NULL; } @@ -386,7 +386,7 @@ static const char* min_expand(MatchState* ms, const char* s, const char* p, cons if (res != NULL) return res; else if (singlematch(ms, s, p, ep)) - s++; /* try with one more repetition */ + s++; // try with one more repetition else return NULL; } @@ -401,8 +401,8 @@ static const char* start_capture(MatchState* ms, const char* s, const char* p, i ms->capture[level].init = s; ms->capture[level].len = what; ms->level = level + 1; - if ((res = match(ms, s, p)) == NULL) /* match failed? */ - ms->level--; /* undo capture */ + if ((res = match(ms, s, p)) == NULL) // match failed? + ms->level--; // undo capture return res; } @@ -410,9 +410,9 @@ static const char* end_capture(MatchState* ms, const char* s, const char* p) { int l = capture_to_close(ms); const char* res; - ms->capture[l].len = s - ms->capture[l].init; /* close capture */ - if ((res = match(ms, s, p)) == NULL) /* match failed? */ - ms->capture[l].len = CAP_UNFINISHED; /* undo capture */ + ms->capture[l].len = s - ms->capture[l].init; // close capture + if ((res = match(ms, s, p)) == NULL) // match failed? + ms->capture[l].len = CAP_UNFINISHED; // undo capture return res; } @@ -431,60 +431,60 @@ static const char* match(MatchState* ms, const char* s, const char* p) { if (ms->matchdepth-- == 0) luaL_error(ms->L, "pattern too complex"); -init: /* using goto's to optimize tail recursion */ +init: // using goto's to optimize tail recursion if (p != ms->p_end) - { /* end of pattern? */ + { // end of pattern? switch (*p) { case '(': - { /* start capture */ - if (*(p + 1) == ')') /* position capture? */ + { // start capture + if (*(p + 1) == ')') // position capture? s = start_capture(ms, s, p + 2, CAP_POSITION); else s = start_capture(ms, s, p + 1, CAP_UNFINISHED); break; } case ')': - { /* end capture */ + { // end capture s = end_capture(ms, s, p + 1); break; } case '$': { - if ((p + 1) != ms->p_end) /* is the `$' the last char in pattern? */ - goto dflt; /* no; go to default */ - s = (s == ms->src_end) ? s : NULL; /* check end of string */ + if ((p + 1) != ms->p_end) // is the `$' the last char in pattern? + goto dflt; // no; go to default + s = (s == ms->src_end) ? s : NULL; // check end of string break; } case L_ESC: - { /* escaped sequences not in the format class[*+?-]? */ + { // escaped sequences not in the format class[*+?-]? switch (*(p + 1)) { case 'b': - { /* balanced string? */ + { // balanced string? s = matchbalance(ms, s, p + 2); if (s != NULL) { p += 4; - goto init; /* return match(ms, s, p + 4); */ - } /* else fail (s == NULL) */ + goto init; // return match(ms, s, p + 4); + } // else fail (s == NULL) break; } case 'f': - { /* frontier? */ + { // frontier? const char* ep; char previous; p += 2; if (*p != '[') luaL_error(ms->L, "missing '[' after '%%f' in pattern"); - ep = classend(ms, p); /* points to what is next */ + ep = classend(ms, p); // points to what is next previous = (s == ms->src_init) ? '\0' : *(s - 1); if (!matchbracketclass(uchar(previous), p, ep - 1) && matchbracketclass(uchar(*s), p, ep - 1)) { p = ep; - goto init; /* return match(ms, s, ep); */ + goto init; // return match(ms, s, ep); } - s = NULL; /* match failed */ + s = NULL; // match failed break; } case '0': @@ -497,12 +497,12 @@ init: /* using goto's to optimize tail recursion */ case '7': case '8': case '9': - { /* capture results (%0-%9)? */ + { // capture results (%0-%9)? s = match_capture(ms, s, uchar(*(p + 1))); if (s != NULL) { p += 2; - goto init; /* return match(ms, s, p + 2) */ + goto init; // return match(ms, s, p + 2) } break; } @@ -513,48 +513,48 @@ init: /* using goto's to optimize tail recursion */ } default: dflt: - { /* pattern class plus optional suffix */ - const char* ep = classend(ms, p); /* points to optional suffix */ - /* does not match at least once? */ + { // pattern class plus optional suffix + const char* ep = classend(ms, p); // points to optional suffix + // does not match at least once? if (!singlematch(ms, s, p, ep)) { if (*ep == '*' || *ep == '?' || *ep == '-') - { /* accept empty? */ + { // accept empty? p = ep + 1; - goto init; /* return match(ms, s, ep + 1); */ + goto init; // return match(ms, s, ep + 1); } - else /* '+' or no suffix */ - s = NULL; /* fail */ + else // '+' or no suffix + s = NULL; // fail } else - { /* matched once */ + { // matched once switch (*ep) - { /* handle optional suffix */ + { // handle optional suffix case '?': - { /* optional */ + { // optional const char* res; if ((res = match(ms, s + 1, ep + 1)) != NULL) s = res; else { p = ep + 1; - goto init; /* else return match(ms, s, ep + 1); */ + goto init; // else return match(ms, s, ep + 1); } break; } - case '+': /* 1 or more repetitions */ - s++; /* 1 match already done */ - /* go through */ - case '*': /* 0 or more repetitions */ + case '+': // 1 or more repetitions + s++; // 1 match already done + // go through + case '*': // 0 or more repetitions s = max_expand(ms, s, p, ep); break; - case '-': /* 0 or more repetitions (minimum) */ + case '-': // 0 or more repetitions (minimum) s = min_expand(ms, s, p, ep); break; - default: /* no suffix */ + default: // no suffix s++; p = ep; - goto init; /* return match(ms, s + 1, ep); */ + goto init; // return match(ms, s + 1, ep); } } break; @@ -568,26 +568,26 @@ init: /* using goto's to optimize tail recursion */ static const char* lmemfind(const char* s1, size_t l1, const char* s2, size_t l2) { if (l2 == 0) - return s1; /* empty strings are everywhere */ + return s1; // empty strings are everywhere else if (l2 > l1) - return NULL; /* avoids a negative `l1' */ + return NULL; // avoids a negative `l1' else { - const char* init; /* to search for a `*s2' inside `s1' */ - l2--; /* 1st char will be checked by `memchr' */ - l1 = l1 - l2; /* `s2' cannot be found after that */ + const char* init; // to search for a `*s2' inside `s1' + l2--; // 1st char will be checked by `memchr' + l1 = l1 - l2; // `s2' cannot be found after that while (l1 > 0 && (init = (const char*)memchr(s1, *s2, l1)) != NULL) { - init++; /* 1st char is already checked */ + init++; // 1st char is already checked if (memcmp(init, s2 + 1, l2) == 0) return init - 1; else - { /* correct `l1' and `s1' to try again */ + { // correct `l1' and `s1' to try again l1 -= init - s1; s1 = init; } } - return NULL; /* not found */ + return NULL; // not found } } @@ -595,8 +595,8 @@ static void push_onecapture(MatchState* ms, int i, const char* s, const char* e) { if (i >= ms->level) { - if (i == 0) /* ms->level == 0, too */ - lua_pushlstring(ms->L, s, e - s); /* add whole match */ + if (i == 0) // ms->level == 0, too + lua_pushlstring(ms->L, s, e - s); // add whole match else luaL_error(ms->L, "invalid capture index"); } @@ -619,20 +619,20 @@ static int push_captures(MatchState* ms, const char* s, const char* e) luaL_checkstack(ms->L, nlevels, "too many captures"); for (i = 0; i < nlevels; i++) push_onecapture(ms, i, s, e); - return nlevels; /* number of strings pushed */ + return nlevels; // number of strings pushed } -/* check whether pattern has no special characters */ +// check whether pattern has no special characters static int nospecials(const char* p, size_t l) { size_t upto = 0; do { if (strpbrk(p + upto, SPECIALS)) - return 0; /* pattern has a special character */ - upto += strlen(p + upto) + 1; /* may have more after \0 */ + return 0; // pattern has a special character + upto += strlen(p + upto) + 1; // may have more after \0 } while (upto <= l); - return 1; /* no special chars found */ + return 1; // no special chars found } static void prepstate(MatchState* ms, lua_State* L, const char* s, size_t ls, const char* p, size_t lp) @@ -659,14 +659,14 @@ static int str_find_aux(lua_State* L, int find) if (init < 1) init = 1; else if (init > (int)ls + 1) - { /* start after string's end? */ - lua_pushnil(L); /* cannot find anything */ + { // start after string's end? + lua_pushnil(L); // cannot find anything return 1; } - /* explicit request or no special characters? */ + // explicit request or no special characters? if (find && (lua_toboolean(L, 4) || nospecials(p, lp))) { - /* do a plain search */ + // do a plain search const char* s2 = lmemfind(s + init - 1, ls - init + 1, p, lp); if (s2) { @@ -683,7 +683,7 @@ static int str_find_aux(lua_State* L, int find) if (anchor) { p++; - lp--; /* skip anchor character */ + lp--; // skip anchor character } prepstate(&ms, L, s, ls, p, lp); do @@ -694,8 +694,8 @@ static int str_find_aux(lua_State* L, int find) { if (find) { - lua_pushinteger(L, (int)(s1 - s + 1)); /* start */ - lua_pushinteger(L, (int)(res - s)); /* end */ + lua_pushinteger(L, (int)(s1 - s + 1)); // start + lua_pushinteger(L, (int)(res - s)); // end return push_captures(&ms, NULL, 0) + 2; } else @@ -703,7 +703,7 @@ static int str_find_aux(lua_State* L, int find) } } while (s1++ < ms.src_end && !anchor); } - lua_pushnil(L); /* not found */ + lua_pushnil(L); // not found return 1; } @@ -733,13 +733,13 @@ static int gmatch_aux(lua_State* L) { int newstart = (int)(e - s); if (e == src) - newstart++; /* empty match? go at least one position */ + newstart++; // empty match? go at least one position lua_pushinteger(L, newstart); lua_replace(L, lua_upvalueindex(3)); return push_captures(&ms, src, e); } } - return 0; /* not found */ + return 0; // not found } static int gmatch(lua_State* L) @@ -765,7 +765,7 @@ static void add_s(MatchState* ms, luaL_Buffer* b, const char* s, const char* e) luaL_addchar(b, news[i]); else { - i++; /* skip ESC */ + i++; // skip ESC if (!isdigit(uchar(news[i]))) { if (news[i] != L_ESC) @@ -777,7 +777,7 @@ static void add_s(MatchState* ms, luaL_Buffer* b, const char* s, const char* e) else { push_onecapture(ms, news[i] - '1', s, e); - luaL_addvalue(b); /* add capture to accumulated result */ + luaL_addvalue(b); // add capture to accumulated result } } } @@ -803,19 +803,19 @@ static void add_value(MatchState* ms, luaL_Buffer* b, const char* s, const char* break; } default: - { /* LUA_TNUMBER or LUA_TSTRING */ + { // LUA_TNUMBER or LUA_TSTRING add_s(ms, b, s, e); return; } } if (!lua_toboolean(L, -1)) - { /* nil or false? */ + { // nil or false? lua_pop(L, 1); - lua_pushlstring(L, s, e - s); /* keep original text */ + lua_pushlstring(L, s, e - s); // keep original text } else if (!lua_isstring(L, -1)) luaL_error(L, "invalid replacement value (a %s)", luaL_typename(L, -1)); - luaL_addvalue(b); /* add result to accumulator */ + luaL_addvalue(b); // add result to accumulator } static int str_gsub(lua_State* L) @@ -834,7 +834,7 @@ static int str_gsub(lua_State* L) if (anchor) { p++; - lp--; /* skip anchor character */ + lp--; // skip anchor character } prepstate(&ms, L, src, srcl, p, lp); while (n < max_s) @@ -847,8 +847,8 @@ static int str_gsub(lua_State* L) n++; add_value(&ms, &b, src, e, tr); } - if (e && e > src) /* non empty match? */ - src = e; /* skip it */ + if (e && e > src) // non empty match? + src = e; // skip it else if (src < ms.src_end) luaL_addchar(&b, *src++); else @@ -858,17 +858,17 @@ static int str_gsub(lua_State* L) } luaL_addlstring(&b, src, ms.src_end - src); luaL_pushresult(&b); - lua_pushinteger(L, n); /* number of substitutions */ + lua_pushinteger(L, n); // number of substitutions return 2; } -/* }====================================================== */ +// }====================================================== -/* valid flags in a format specification */ +// valid flags in a format specification #define FLAGS "-+ #0" -/* maximum size of each formatted item (> len(format('%99.99f', -1e308))) */ +// maximum size of each formatted item (> len(format('%99.99f', -1e308))) #define MAX_ITEM 512 -/* maximum size of each format specification (such as '%-099.99d') */ +// maximum size of each format specification (such as '%-099.99d') #define MAX_FORMAT 32 static void addquoted(lua_State* L, luaL_Buffer* b, int arg) @@ -916,20 +916,20 @@ static const char* scanformat(lua_State* L, const char* strfrmt, char* form, siz { const char* p = strfrmt; while (*p != '\0' && strchr(FLAGS, *p) != NULL) - p++; /* skip flags */ + p++; // skip flags if ((size_t)(p - strfrmt) >= sizeof(FLAGS)) luaL_error(L, "invalid format (repeated flags)"); if (isdigit(uchar(*p))) - p++; /* skip width */ + p++; // skip width if (isdigit(uchar(*p))) - p++; /* (2 digits at most) */ + p++; // (2 digits at most) if (*p == '.') { p++; if (isdigit(uchar(*p))) - p++; /* skip precision */ + p++; // skip precision if (isdigit(uchar(*p))) - p++; /* (2 digits at most) */ + p++; // (2 digits at most) } if (isdigit(uchar(*p))) luaL_error(L, "invalid format (width or precision too long)"); @@ -967,11 +967,11 @@ static int str_format(lua_State* L) if (*strfrmt != L_ESC) luaL_addchar(&b, *strfrmt++); else if (*++strfrmt == L_ESC) - luaL_addchar(&b, *strfrmt++); /* %% */ + luaL_addchar(&b, *strfrmt++); // %% else - { /* format item */ - char form[MAX_FORMAT]; /* to store the format (`%...') */ - char buff[MAX_ITEM]; /* to store the formatted item */ + { // format item + char form[MAX_FORMAT]; // to store the format (`%...') + char buff[MAX_ITEM]; // to store the formatted item if (++arg > top) luaL_error(L, "missing argument #%d", arg); size_t formatItemSize = 0; @@ -1014,7 +1014,7 @@ static int str_format(lua_State* L) case 'q': { addquoted(L, &b, arg); - continue; /* skip the 'addsize' at the end */ + continue; // skip the 'addsize' at the end } case 's': { @@ -1026,7 +1026,7 @@ static int str_format(lua_State* L) keep original string */ lua_pushvalue(L, arg); luaL_addvalue(&b); - continue; /* skip the `addsize' at the end */ + continue; // skip the `addsize' at the end } else { @@ -1037,25 +1037,19 @@ static int str_format(lua_State* L) case '*': { if (!FFlag::LuauTostringFormatSpecifier) - { luaL_error(L, "invalid option '%%*' to 'format'"); - break; - } if (formatItemSize != 1) - { luaL_error(L, "'%%*' does not take a form"); - } size_t length; const char* string = luaL_tolstring(L, arg, &length); luaL_addlstring(&b, string, length); - - continue; /* skip the `addsize' at the end */ + continue; // skip the `addsize' at the end } default: - { /* also treat cases `pnLlh' */ + { // also treat cases `pnLlh' luaL_error(L, "invalid option '%%%c' to 'format'", *(strfrmt - 1)); } } @@ -1120,31 +1114,31 @@ static int str_split(lua_State* L) ** ======================================================= */ -/* value used for padding */ +// value used for padding #if !defined(LUAL_PACKPADBYTE) #define LUAL_PACKPADBYTE 0x00 #endif -/* maximum size for the binary representation of an integer */ +// maximum size for the binary representation of an integer #define MAXINTSIZE 16 -/* number of bits in a character */ +// number of bits in a character #define NB CHAR_BIT -/* mask for one character (NB 1's) */ +// mask for one character (NB 1's) #define MC ((1 << NB) - 1) -/* internal size of integers used for pack/unpack */ +// internal size of integers used for pack/unpack #define SZINT (int)sizeof(long long) -/* dummy union to get native endianness */ +// dummy union to get native endianness static const union { int dummy; - char little; /* true iff machine is little endian */ + char little; // true iff machine is little endian } nativeendian = {1}; -/* assume we need to align for double & pointers */ +// assume we need to align for double & pointers #define MAXALIGN 8 /* @@ -1155,7 +1149,7 @@ typedef union Ftypes float f; double d; double n; - char buff[5 * sizeof(double)]; /* enough for any float type */ + char buff[5 * sizeof(double)]; // enough for any float type } Ftypes; /* @@ -1173,15 +1167,15 @@ typedef struct Header */ typedef enum KOption { - Kint, /* signed integers */ - Kuint, /* unsigned integers */ - Kfloat, /* floating-point numbers */ - Kchar, /* fixed-length strings */ - Kstring, /* strings with prefixed length */ - Kzstr, /* zero-terminated strings */ - Kpadding, /* padding */ - Kpaddalign, /* padding for alignment */ - Knop /* no-op (configuration or spaces) */ + Kint, // signed integers + Kuint, // unsigned integers + Kfloat, // floating-point numbers + Kchar, // fixed-length strings + Kstring, // strings with prefixed length + Kzstr, // zero-terminated strings + Kpadding, // padding + Kpaddalign, // padding for alignment + Knop // no-op (configuration or spaces) } KOption; /* @@ -1195,8 +1189,8 @@ static int digit(int c) static int getnum(Header* h, const char** fmt, int df) { - if (!digit(**fmt)) /* no number? */ - return df; /* return default value */ + if (!digit(**fmt)) // no number? + return df; // return default value else { int a = 0; @@ -1238,7 +1232,7 @@ static void initheader(lua_State* L, Header* h) static KOption getoption(Header* h, const char** fmt, int* size) { int opt = *((*fmt)++); - *size = 0; /* default */ + *size = 0; // default switch (opt) { case 'b': @@ -1330,19 +1324,19 @@ static KOption getoption(Header* h, const char** fmt, int* size) static KOption getdetails(Header* h, size_t totalsize, const char** fmt, int* psize, int* ntoalign) { KOption opt = getoption(h, fmt, psize); - int align = *psize; /* usually, alignment follows size */ + int align = *psize; // usually, alignment follows size if (opt == Kpaddalign) - { /* 'X' gets alignment from following option */ + { // 'X' gets alignment from following option if (**fmt == '\0' || getoption(h, fmt, &align) == Kchar || align == 0) luaL_argerror(h->L, 1, "invalid next option for option 'X'"); } - if (align <= 1 || opt == Kchar) /* need no alignment? */ + if (align <= 1 || opt == Kchar) // need no alignment? *ntoalign = 0; else { - if (align > h->maxalign) /* enforce maximum alignment */ + if (align > h->maxalign) // enforce maximum alignment align = h->maxalign; - if ((align & (align - 1)) != 0) /* is 'align' not a power of 2? */ + if ((align & (align - 1)) != 0) // is 'align' not a power of 2? luaL_argerror(h->L, 1, "format asks for alignment not power of 2"); *ntoalign = (align - (int)(totalsize & (align - 1))) & (align - 1); } @@ -1360,18 +1354,18 @@ static void packint(luaL_Buffer* b, unsigned long long n, int islittle, int size LUAU_ASSERT(size <= MAXINTSIZE); char buff[MAXINTSIZE]; int i; - buff[islittle ? 0 : size - 1] = (char)(n & MC); /* first byte */ + buff[islittle ? 0 : size - 1] = (char)(n & MC); // first byte for (i = 1; i < size; i++) { n >>= NB; buff[islittle ? i : size - 1 - i] = (char)(n & MC); } if (neg && size > SZINT) - { /* negative number need sign extension? */ - for (i = SZINT; i < size; i++) /* correct extra bytes */ + { // negative number need sign extension? + for (i = SZINT; i < size; i++) // correct extra bytes buff[islittle ? i : size - 1 - i] = (char)MC; } - luaL_addlstring(b, buff, size); /* add result to buffer */ + luaL_addlstring(b, buff, size); // add result to buffer } /* @@ -1397,11 +1391,11 @@ static int str_pack(lua_State* L) { luaL_Buffer b; Header h; - const char* fmt = luaL_checkstring(L, 1); /* format string */ - int arg = 1; /* current argument to pack */ - size_t totalsize = 0; /* accumulate total size of result */ + const char* fmt = luaL_checkstring(L, 1); // format string + int arg = 1; // current argument to pack + size_t totalsize = 0; // accumulate total size of result initheader(L, &h); - lua_pushnil(L); /* mark to separate arguments from string buffer */ + lua_pushnil(L); // mark to separate arguments from string buffer luaL_buffinit(L, &b); while (*fmt != '\0') { @@ -1409,15 +1403,15 @@ static int str_pack(lua_State* L) KOption opt = getdetails(&h, totalsize, &fmt, &size, &ntoalign); totalsize += ntoalign + size; while (ntoalign-- > 0) - luaL_addchar(&b, LUAL_PACKPADBYTE); /* fill alignment */ + luaL_addchar(&b, LUAL_PACKPADBYTE); // fill alignment arg++; switch (opt) { case Kint: - { /* signed integers */ + { // signed integers long long n = (long long)luaL_checknumber(L, arg); if (size < SZINT) - { /* need overflow check? */ + { // need overflow check? long long lim = (long long)1 << ((size * NB) - 1); luaL_argcheck(L, -lim <= n && n < lim, arg, "integer overflow"); } @@ -1425,64 +1419,64 @@ static int str_pack(lua_State* L) break; } case Kuint: - { /* unsigned integers */ + { // unsigned integers long long n = (long long)luaL_checknumber(L, arg); - if (size < SZINT) /* need overflow check? */ + if (size < SZINT) // need overflow check? luaL_argcheck(L, (unsigned long long)n < ((unsigned long long)1 << (size * NB)), arg, "unsigned overflow"); packint(&b, (unsigned long long)n, h.islittle, size, 0); break; } case Kfloat: - { /* floating-point options */ + { // floating-point options volatile Ftypes u; char buff[MAXINTSIZE]; - double n = luaL_checknumber(L, arg); /* get argument */ + double n = luaL_checknumber(L, arg); // get argument if (size == sizeof(u.f)) - u.f = (float)n; /* copy it into 'u' */ + u.f = (float)n; // copy it into 'u' else if (size == sizeof(u.d)) u.d = (double)n; else u.n = n; - /* move 'u' to final result, correcting endianness if needed */ + // move 'u' to final result, correcting endianness if needed copywithendian(buff, u.buff, size, h.islittle); luaL_addlstring(&b, buff, size); break; } case Kchar: - { /* fixed-size string */ + { // fixed-size string size_t len; const char* s = luaL_checklstring(L, arg, &len); luaL_argcheck(L, len <= (size_t)size, arg, "string longer than given size"); - luaL_addlstring(&b, s, len); /* add string */ - while (len++ < (size_t)size) /* pad extra space */ + luaL_addlstring(&b, s, len); // add string + while (len++ < (size_t)size) // pad extra space luaL_addchar(&b, LUAL_PACKPADBYTE); break; } case Kstring: - { /* strings with length count */ + { // strings with length count size_t len; const char* s = luaL_checklstring(L, arg, &len); luaL_argcheck(L, size >= (int)sizeof(size_t) || len < ((size_t)1 << (size * NB)), arg, "string length does not fit in given size"); - packint(&b, len, h.islittle, size, 0); /* pack length */ + packint(&b, len, h.islittle, size, 0); // pack length luaL_addlstring(&b, s, len); totalsize += len; break; } case Kzstr: - { /* zero-terminated string */ + { // zero-terminated string size_t len; const char* s = luaL_checklstring(L, arg, &len); luaL_argcheck(L, strlen(s) == len, arg, "string contains zeros"); luaL_addlstring(&b, s, len); - luaL_addchar(&b, '\0'); /* add zero at the end */ + luaL_addchar(&b, '\0'); // add zero at the end totalsize += len + 1; break; } case Kpadding: - luaL_addchar(&b, LUAL_PACKPADBYTE); /* FALLTHROUGH */ + luaL_addchar(&b, LUAL_PACKPADBYTE); // FALLTHROUGH case Kpaddalign: case Knop: - arg--; /* undo increment */ + arg--; // undo increment break; } } @@ -1493,15 +1487,15 @@ static int str_pack(lua_State* L) static int str_packsize(lua_State* L) { Header h; - const char* fmt = luaL_checkstring(L, 1); /* format string */ - int totalsize = 0; /* accumulate total size of result */ + const char* fmt = luaL_checkstring(L, 1); // format string + int totalsize = 0; // accumulate total size of result initheader(L, &h); while (*fmt != '\0') { int size, ntoalign; KOption opt = getdetails(&h, totalsize, &fmt, &size, &ntoalign); luaL_argcheck(L, opt != Kstring && opt != Kzstr, 1, "variable-length format"); - size += ntoalign; /* total space used by option */ + size += ntoalign; // total space used by option luaL_argcheck(L, totalsize <= MAXSSIZE - size, 1, "format result too large"); totalsize += size; } @@ -1528,15 +1522,15 @@ static long long unpackint(lua_State* L, const char* str, int islittle, int size res |= (unsigned char)str[islittle ? i : size - 1 - i]; } if (size < SZINT) - { /* real size smaller than int? */ + { // real size smaller than int? if (issigned) - { /* needs sign extension? */ + { // needs sign extension? unsigned long long mask = (unsigned long long)1 << (size * NB - 1); - res = ((res ^ mask) - mask); /* do sign extension */ + res = ((res ^ mask) - mask); // do sign extension } } else if (size > SZINT) - { /* must check unread bytes */ + { // must check unread bytes int mask = (!issigned || (long long)res >= 0) ? 0 : MC; for (i = limit; i < size; i++) { @@ -1556,7 +1550,7 @@ static int str_unpack(lua_State* L) int pos = posrelat(luaL_optinteger(L, 3, 1), ld) - 1; if (pos < 0) pos = 0; - int n = 0; /* number of results */ + int n = 0; // number of results luaL_argcheck(L, size_t(pos) <= ld, 3, "initial position out of string"); initheader(L, &h); while (*fmt != '\0') @@ -1564,8 +1558,8 @@ static int str_unpack(lua_State* L) int size, ntoalign; KOption opt = getdetails(&h, pos, &fmt, &size, &ntoalign); luaL_argcheck(L, (size_t)ntoalign + size <= ld - pos, 2, "data string too short"); - pos += ntoalign; /* skip alignment */ - /* stack space for item + next position */ + pos += ntoalign; // skip alignment + // stack space for item + next position luaL_checkstack(L, 2, "too many results"); n++; switch (opt) @@ -1606,7 +1600,7 @@ static int str_unpack(lua_State* L) size_t len = (size_t)unpackint(L, data + pos, h.islittle, size, 0); luaL_argcheck(L, len <= ld - pos - size, 2, "data string too short"); lua_pushlstring(L, data + pos + size, len); - pos += (int)len; /* skip string */ + pos += (int)len; // skip string break; } case Kzstr: @@ -1614,22 +1608,22 @@ static int str_unpack(lua_State* L) size_t len = strlen(data + pos); luaL_argcheck(L, pos + len < ld, 2, "unfinished string for format 'z'"); lua_pushlstring(L, data + pos, len); - pos += (int)len + 1; /* skip string plus final '\0' */ + pos += (int)len + 1; // skip string plus final '\0' break; } case Kpaddalign: case Kpadding: case Knop: - n--; /* undo increment */ + n--; // undo increment break; } pos += size; } - lua_pushinteger(L, pos + 1); /* next position */ + lua_pushinteger(L, pos + 1); // next position return n + 1; } -/* }====================================================== */ +// }====================================================== static const luaL_Reg strlib[] = { {"byte", str_byte}, @@ -1654,14 +1648,14 @@ static const luaL_Reg strlib[] = { static void createmetatable(lua_State* L) { - lua_createtable(L, 0, 1); /* create metatable for strings */ - lua_pushliteral(L, ""); /* dummy string */ + lua_createtable(L, 0, 1); // create metatable for strings + lua_pushliteral(L, ""); // dummy string lua_pushvalue(L, -2); - lua_setmetatable(L, -2); /* set string metatable */ - lua_pop(L, 1); /* pop dummy string */ - lua_pushvalue(L, -2); /* string library... */ - lua_setfield(L, -2, "__index"); /* ...is the __index metamethod */ - lua_pop(L, 1); /* pop metatable */ + lua_setmetatable(L, -2); // set string metatable + lua_pop(L, 1); // pop dummy string + lua_pushvalue(L, -2); // string library... + lua_setfield(L, -2, "__index"); // ...is the __index metamethod + lua_pop(L, 1); // pop metatable } /* diff --git a/VM/src/ltable.cpp b/VM/src/ltable.cpp index 38693156..8d59ecbc 100644 --- a/VM/src/ltable.cpp +++ b/VM/src/ltable.cpp @@ -46,8 +46,8 @@ static_assert(TKey{{NULL}, {0}, LUA_TNIL, -(MAXSIZE - 1)}.next == -(MAXSIZE - 1) // empty hash data points to dummynode so that we can always dereference it const LuaNode luaH_dummynode = { - {{NULL}, {0}, LUA_TNIL}, /* value */ - {{NULL}, {0}, LUA_TNIL, 0} /* key */ + {{NULL}, {0}, LUA_TNIL}, // value + {{NULL}, {0}, LUA_TNIL, 0} // key }; #define dummynode (&luaH_dummynode) @@ -170,52 +170,52 @@ static int findindex(lua_State* L, Table* t, StkId key) { int i; if (ttisnil(key)) - return -1; /* first iteration */ + return -1; // first iteration i = ttisnumber(key) ? arrayindex(nvalue(key)) : -1; - if (0 < i && i <= t->sizearray) /* is `key' inside array part? */ - return i - 1; /* yes; that's the index (corrected to C) */ + if (0 < i && i <= t->sizearray) // is `key' inside array part? + return i - 1; // yes; that's the index (corrected to C) else { LuaNode* n = mainposition(t, key); for (;;) - { /* check whether `key' is somewhere in the chain */ - /* key may be dead already, but it is ok to use it in `next' */ + { // check whether `key' is somewhere in the chain + // key may be dead already, but it is ok to use it in `next' if (luaO_rawequalKey(gkey(n), key) || (ttype(gkey(n)) == LUA_TDEADKEY && iscollectable(key) && gcvalue(gkey(n)) == gcvalue(key))) { - i = cast_int(n - gnode(t, 0)); /* key index in hash table */ - /* hash elements are numbered after array ones */ + i = cast_int(n - gnode(t, 0)); // key index in hash table + // hash elements are numbered after array ones return i + t->sizearray; } if (gnext(n) == 0) break; n += gnext(n); } - luaG_runerror(L, "invalid key to 'next'"); /* key not found */ + luaG_runerror(L, "invalid key to 'next'"); // key not found } } int luaH_next(lua_State* L, Table* t, StkId key) { - int i = findindex(L, t, key); /* find original element */ + int i = findindex(L, t, key); // find original element for (i++; i < t->sizearray; i++) - { /* try first array part */ + { // try first array part if (!ttisnil(&t->array[i])) - { /* a non-nil value? */ + { // a non-nil value? setnvalue(key, cast_num(i + 1)); setobj2s(L, key + 1, &t->array[i]); return 1; } } for (i -= t->sizearray; i < sizenode(t); i++) - { /* then hash part */ + { // then hash part if (!ttisnil(gval(gnode(t, i)))) - { /* a non-nil value? */ + { // a non-nil value? getnodekey(L, key, gnode(t, i)); setobj2s(L, key + 1, gval(gnode(t, i))); return 1; } } - return 0; /* no more elements */ + return 0; // no more elements } /* @@ -235,23 +235,23 @@ int luaH_next(lua_State* L, Table* t, StkId key) static int computesizes(int nums[], int* narray) { int i; - int twotoi; /* 2^i */ - int a = 0; /* number of elements smaller than 2^i */ - int na = 0; /* number of elements to go to array part */ - int n = 0; /* optimal size for array part */ + int twotoi; // 2^i + int a = 0; // number of elements smaller than 2^i + int na = 0; // number of elements to go to array part + int n = 0; // optimal size for array part for (i = 0, twotoi = 1; twotoi / 2 < *narray; i++, twotoi *= 2) { if (nums[i] > 0) { a += nums[i]; if (a > twotoi / 2) - { /* more than half elements present? */ - n = twotoi; /* optimal size (till now) */ - na = a; /* all elements smaller than n will go to array part */ + { // more than half elements present? + n = twotoi; // optimal size (till now) + na = a; // all elements smaller than n will go to array part } } if (a == *narray) - break; /* all elements already counted */ + break; // all elements already counted } *narray = n; LUAU_ASSERT(*narray / 2 <= na && na <= *narray); @@ -262,8 +262,8 @@ static int countint(double key, int* nums) { int k = arrayindex(key); if (0 < k && k <= MAXSIZE) - { /* is `key' an appropriate array index? */ - nums[ceillog2(k)]++; /* count as such */ + { // is `key' an appropriate array index? + nums[ceillog2(k)]++; // count as such return 1; } else @@ -273,20 +273,20 @@ static int countint(double key, int* nums) static int numusearray(const Table* t, int* nums) { int lg; - int ttlg; /* 2^lg */ - int ause = 0; /* summation of `nums' */ - int i = 1; /* count to traverse all array keys */ + int ttlg; // 2^lg + int ause = 0; // summation of `nums' + int i = 1; // count to traverse all array keys for (lg = 0, ttlg = 1; lg <= MAXBITS; lg++, ttlg *= 2) - { /* for each slice */ - int lc = 0; /* counter */ + { // for each slice + int lc = 0; // counter int lim = ttlg; if (lim > t->sizearray) { - lim = t->sizearray; /* adjust upper limit */ + lim = t->sizearray; // adjust upper limit if (i > lim) - break; /* no more elements to count */ + break; // no more elements to count } - /* count elements in range (2^(lg-1), 2^lg] */ + // count elements in range (2^(lg-1), 2^lg] for (; i <= lim; i++) { if (!ttisnil(&t->array[i - 1])) @@ -300,8 +300,8 @@ static int numusearray(const Table* t, int* nums) static int numusehash(const Table* t, int* nums, int* pnasize) { - int totaluse = 0; /* total number of elements */ - int ause = 0; /* summation of `nums' */ + int totaluse = 0; // total number of elements + int ause = 0; // summation of `nums' int i = sizenode(t); while (i--) { @@ -332,8 +332,8 @@ static void setnodevector(lua_State* L, Table* t, int size) { int lsize; if (size == 0) - { /* no elements to hash part? */ - t->node = cast_to(LuaNode*, dummynode); /* use common `dummynode' */ + { // no elements to hash part? + t->node = cast_to(LuaNode*, dummynode); // use common `dummynode' lsize = 0; } else @@ -354,7 +354,7 @@ static void setnodevector(lua_State* L, Table* t, int size) } t->lsizenode = cast_byte(lsize); t->nodemask8 = cast_byte((1 << lsize) - 1); - t->lastfree = size; /* all positions are free */ + t->lastfree = size; // all positions are free } static TValue* newkey(lua_State* L, Table* t, const TValue* key); @@ -379,17 +379,17 @@ static void resize(lua_State* L, Table* t, int nasize, int nhsize) luaG_runerror(L, "table overflow"); int oldasize = t->sizearray; int oldhsize = t->lsizenode; - LuaNode* nold = t->node; /* save old hash ... */ - if (nasize > oldasize) /* array part must grow? */ + LuaNode* nold = t->node; // save old hash ... + if (nasize > oldasize) // array part must grow? setarrayvector(L, t, nasize); - /* create new hash part with appropriate size */ + // create new hash part with appropriate size setnodevector(L, t, nhsize); - /* used for the migration check at the end */ + // used for the migration check at the end LuaNode* nnew = t->node; if (nasize < oldasize) - { /* array part must shrink? */ + { // array part must shrink? t->sizearray = nasize; - /* re-insert elements from vanishing slice */ + // re-insert elements from vanishing slice for (int i = nasize; i < oldasize; i++) { if (!ttisnil(&t->array[i])) @@ -399,12 +399,12 @@ static void resize(lua_State* L, Table* t, int nasize, int nhsize) setobjt2t(L, newkey(L, t, &ok), &t->array[i]); } } - /* shrink array */ + // shrink array luaM_reallocarray(L, t->array, oldasize, nasize, TValue, t->memcat); } - /* used for the migration check at the end */ + // used for the migration check at the end TValue* anew = t->array; - /* re-insert elements from hash part */ + // re-insert elements from hash part for (int i = twoto(oldhsize) - 1; i >= 0; i--) { LuaNode* old = nold + i; @@ -416,19 +416,19 @@ static void resize(lua_State* L, Table* t, int nasize, int nhsize) } } - /* make sure we haven't recursively rehashed during element migration */ + // make sure we haven't recursively rehashed during element migration LUAU_ASSERT(nnew == t->node); LUAU_ASSERT(anew == t->array); if (nold != dummynode) - luaM_freearray(L, nold, twoto(oldhsize), LuaNode, t->memcat); /* free old array */ + luaM_freearray(L, nold, twoto(oldhsize), LuaNode, t->memcat); // free old array } static int adjustasize(Table* t, int size, const TValue* ek) { bool tbound = t->node != dummynode || size < t->sizearray; int ekindex = ek && ttisnumber(ek) ? arrayindex(nvalue(ek)) : -1; - /* move the array size up until the boundary is guaranteed to be inside the array part */ + // move the array size up until the boundary is guaranteed to be inside the array part while (size + 1 == ekindex || (tbound && !ttisnil(luaH_getnum(t, size + 1)))) size++; return size; @@ -448,22 +448,22 @@ void luaH_resizehash(lua_State* L, Table* t, int nhsize) static void rehash(lua_State* L, Table* t, const TValue* ek) { - int nums[MAXBITS + 1]; /* nums[i] = number of keys between 2^(i-1) and 2^i */ + int nums[MAXBITS + 1]; // nums[i] = number of keys between 2^(i-1) and 2^i for (int i = 0; i <= MAXBITS; i++) - nums[i] = 0; /* reset counts */ - int nasize = numusearray(t, nums); /* count keys in array part */ - int totaluse = nasize; /* all those keys are integer keys */ - totaluse += numusehash(t, nums, &nasize); /* count keys in hash part */ - /* count extra key */ + nums[i] = 0; // reset counts + int nasize = numusearray(t, nums); // count keys in array part + int totaluse = nasize; // all those keys are integer keys + totaluse += numusehash(t, nums, &nasize); // count keys in hash part + // count extra key if (ttisnumber(ek)) nasize += countint(nvalue(ek), nums); totaluse++; - /* compute new size for array part */ + // compute new size for array part int na = computesizes(nums, &nasize); int nh = totaluse - na; - /* enforce the boundary invariant; for performance, only do hash lookups if we must */ + // enforce the boundary invariant; for performance, only do hash lookups if we must nasize = adjustasize(t, nasize, ek); - /* resize the table to new computed sizes */ + // resize the table to new computed sizes resize(L, t, nasize, nh); } @@ -511,7 +511,7 @@ static LuaNode* getfreepos(Table* t) if (ttisnil(gkey(n))) return n; } - return NULL; /* could not find a free place */ + return NULL; // could not find a free place } /* @@ -523,24 +523,24 @@ static LuaNode* getfreepos(Table* t) */ static TValue* newkey(lua_State* L, Table* t, const TValue* key) { - /* enforce boundary invariant */ + // enforce boundary invariant if (ttisnumber(key) && nvalue(key) == t->sizearray + 1) { - rehash(L, t, key); /* grow table */ + rehash(L, t, key); // grow table - /* after rehash, numeric keys might be located in the new array part, but won't be found in the node part */ + // after rehash, numeric keys might be located in the new array part, but won't be found in the node part return arrayornewkey(L, t, key); } LuaNode* mp = mainposition(t, key); if (!ttisnil(gval(mp)) || mp == dummynode) { - LuaNode* n = getfreepos(t); /* get a free place */ + LuaNode* n = getfreepos(t); // get a free place if (n == NULL) - { /* cannot find a free place? */ - rehash(L, t, key); /* grow table */ + { // cannot find a free place? + rehash(L, t, key); // grow table - /* after rehash, numeric keys might be located in the new array part, but won't be found in the node part */ + // after rehash, numeric keys might be located in the new array part, but won't be found in the node part return arrayornewkey(L, t, key); } LUAU_ASSERT(n != dummynode); @@ -548,24 +548,24 @@ static TValue* newkey(lua_State* L, Table* t, const TValue* key) getnodekey(L, &mk, mp); LuaNode* othern = mainposition(t, &mk); if (othern != mp) - { /* is colliding node out of its main position? */ - /* yes; move colliding node into free position */ + { // is colliding node out of its main position? + // yes; move colliding node into free position while (othern + gnext(othern) != mp) - othern += gnext(othern); /* find previous */ - gnext(othern) = cast_int(n - othern); /* redo the chain with `n' in place of `mp' */ - *n = *mp; /* copy colliding node into free pos. (mp->next also goes) */ + othern += gnext(othern); // find previous + gnext(othern) = cast_int(n - othern); // redo the chain with `n' in place of `mp' + *n = *mp; // copy colliding node into free pos. (mp->next also goes) if (gnext(mp) != 0) { - gnext(n) += cast_int(mp - n); /* correct 'next' */ - gnext(mp) = 0; /* now 'mp' is free */ + gnext(n) += cast_int(mp - n); // correct 'next' + gnext(mp) = 0; // now 'mp' is free } setnilvalue(gval(mp)); } else - { /* colliding node is in its own main position */ - /* new node will go into free position */ + { // colliding node is in its own main position + // new node will go into free position if (gnext(mp) != 0) - gnext(n) = cast_int((mp + gnext(mp)) - n); /* chain new position */ + gnext(n) = cast_int((mp + gnext(mp)) - n); // chain new position else LUAU_ASSERT(gnext(n) == 0); gnext(mp) = cast_int(n - mp); @@ -583,7 +583,7 @@ static TValue* newkey(lua_State* L, Table* t, const TValue* key) */ const TValue* luaH_getnum(Table* t, int key) { - /* (1 <= key && key <= t->sizearray) */ + // (1 <= key && key <= t->sizearray) if (cast_to(unsigned int, key - 1) < cast_to(unsigned int, t->sizearray)) return &t->array[key - 1]; else if (t->node != dummynode) @@ -591,9 +591,9 @@ const TValue* luaH_getnum(Table* t, int key) double nk = cast_num(key); LuaNode* n = hashnum(t, nk); for (;;) - { /* check whether `key' is somewhere in the chain */ + { // check whether `key' is somewhere in the chain if (ttisnumber(gkey(n)) && luai_numeq(nvalue(gkey(n)), nk)) - return gval(n); /* that's it */ + return gval(n); // that's it if (gnext(n) == 0) break; n += gnext(n); @@ -611,9 +611,9 @@ const TValue* luaH_getstr(Table* t, TString* key) { LuaNode* n = hashstr(t, key); for (;;) - { /* check whether `key' is somewhere in the chain */ + { // check whether `key' is somewhere in the chain if (ttisstring(gkey(n)) && tsvalue(gkey(n)) == key) - return gval(n); /* that's it */ + return gval(n); // that's it if (gnext(n) == 0) break; n += gnext(n); @@ -637,17 +637,17 @@ const TValue* luaH_get(Table* t, const TValue* key) int k; double n = nvalue(key); luai_num2int(k, n); - if (luai_numeq(cast_num(k), nvalue(key))) /* index is int? */ - return luaH_getnum(t, k); /* use specialized version */ - /* else go through */ + if (luai_numeq(cast_num(k), nvalue(key))) // index is int? + return luaH_getnum(t, k); // use specialized version + // else go through } default: { LuaNode* n = mainposition(t, key); for (;;) - { /* check whether `key' is somewhere in the chain */ + { // check whether `key' is somewhere in the chain if (luaO_rawequalKey(gkey(n), key)) - return gval(n); /* that's it */ + return gval(n); // that's it if (gnext(n) == 0) break; n += gnext(n); @@ -680,10 +680,10 @@ TValue* luaH_newkey(lua_State* L, Table* t, const TValue* key) TValue* luaH_setnum(lua_State* L, Table* t, int key) { - /* (1 <= key && key <= t->sizearray) */ + // (1 <= key && key <= t->sizearray) if (cast_to(unsigned int, key - 1) < cast_to(unsigned int, t->sizearray)) return &t->array[key - 1]; - /* hash fallback */ + // hash fallback const TValue* p = luaH_getnum(t, key); if (p != luaO_nilobject) return cast_to(TValue*, p); @@ -739,9 +739,9 @@ int luaH_getn(Table* t) if (boundary > 0) { if (!ttisnil(&t->array[t->sizearray - 1]) && t->node == dummynode) - return t->sizearray; /* fast-path: the end of the array in `t' already refers to a boundary */ + return t->sizearray; // fast-path: the end of the array in `t' already refers to a boundary if (boundary < t->sizearray && !ttisnil(&t->array[boundary - 1]) && ttisnil(&t->array[boundary])) - return boundary; /* fast-path: boundary already refers to a boundary in `t' */ + return boundary; // fast-path: boundary already refers to a boundary in `t' int foundboundary = updateaboundary(t, boundary); if (foundboundary > 0) @@ -767,7 +767,7 @@ int luaH_getn(Table* t) } else { - /* validate boundary invariant */ + // validate boundary invariant LUAU_ASSERT(t->node == dummynode || ttisnil(luaH_getnum(t, j + 1))); return j; } @@ -812,7 +812,7 @@ Table* luaH_clone(lua_State* L, Table* tt) void luaH_clear(Table* tt) { - /* clear array part */ + // clear array part for (int i = 0; i < tt->sizearray; ++i) { setnilvalue(&tt->array[i]); @@ -820,7 +820,7 @@ void luaH_clear(Table* tt) maybesetaboundary(tt, 0); - /* clear hash part */ + // clear hash part if (tt->node != dummynode) { int size = sizenode(tt); @@ -834,6 +834,6 @@ void luaH_clear(Table* tt) } } - /* back to empty -> no tag methods present */ + // back to empty -> no tag methods present tt->tmcache = cast_byte(~0); } diff --git a/VM/src/ltablib.cpp b/VM/src/ltablib.cpp index dc653338..6dd94149 100644 --- a/VM/src/ltablib.cpp +++ b/VM/src/ltablib.cpp @@ -18,13 +18,13 @@ static int foreachi(lua_State* L) int n = lua_objlen(L, 1); for (i = 1; i <= n; i++) { - lua_pushvalue(L, 2); /* function */ - lua_pushinteger(L, i); /* 1st argument */ - lua_rawgeti(L, 1, i); /* 2nd argument */ + lua_pushvalue(L, 2); // function + lua_pushinteger(L, i); // 1st argument + lua_rawgeti(L, 1, i); // 2nd argument lua_call(L, 2, 1); if (!lua_isnil(L, -1)) return 1; - lua_pop(L, 1); /* remove nil result */ + lua_pop(L, 1); // remove nil result } return 0; } @@ -33,16 +33,16 @@ static int foreach (lua_State* L) { luaL_checktype(L, 1, LUA_TTABLE); luaL_checktype(L, 2, LUA_TFUNCTION); - lua_pushnil(L); /* first key */ + lua_pushnil(L); // first key while (lua_next(L, 1)) { - lua_pushvalue(L, 2); /* function */ - lua_pushvalue(L, -3); /* key */ - lua_pushvalue(L, -3); /* value */ + lua_pushvalue(L, 2); // function + lua_pushvalue(L, -3); // key + lua_pushvalue(L, -3); // value lua_call(L, 2, 1); if (!lua_isnil(L, -1)) return 1; - lua_pop(L, 2); /* remove value and result */ + lua_pop(L, 2); // remove value and result } return 0; } @@ -51,10 +51,10 @@ static int maxn(lua_State* L) { double max = 0; luaL_checktype(L, 1, LUA_TTABLE); - lua_pushnil(L); /* first key */ + lua_pushnil(L); // first key while (lua_next(L, 1)) { - lua_pop(L, 1); /* remove value */ + lua_pop(L, 1); // remove value if (lua_type(L, -1) == LUA_TNUMBER) { double v = lua_tonumber(L, -1); @@ -81,7 +81,7 @@ static void moveelements(lua_State* L, int srct, int dstt, int f, int e, int t) if (dst->readonly) luaG_readonlyerror(L); - int n = e - f + 1; /* number of elements to move */ + int n = e - f + 1; // number of elements to move if (cast_to(unsigned int, f - 1) < cast_to(unsigned int, src->sizearray) && cast_to(unsigned int, t - 1) < cast_to(unsigned int, dst->sizearray) && @@ -137,19 +137,19 @@ static int tinsert(lua_State* L) { luaL_checktype(L, 1, LUA_TTABLE); int n = lua_objlen(L, 1); - int pos; /* where to insert new element */ + int pos; // where to insert new element switch (lua_gettop(L)) { case 2: - { /* called with only 2 arguments */ - pos = n + 1; /* insert new element at the end */ + { // called with only 2 arguments + pos = n + 1; // insert new element at the end break; } case 3: { - pos = luaL_checkinteger(L, 2); /* 2nd argument is the position */ + pos = luaL_checkinteger(L, 2); // 2nd argument is the position - /* move up elements if necessary */ + // move up elements if necessary if (1 <= pos && pos <= n) moveelements(L, 1, 1, pos, n, pos + 1); break; @@ -159,7 +159,7 @@ static int tinsert(lua_State* L) luaL_error(L, "wrong number of arguments to 'insert'"); } } - lua_rawseti(L, 1, pos); /* t[pos] = v */ + lua_rawseti(L, 1, pos); // t[pos] = v return 0; } @@ -169,14 +169,14 @@ static int tremove(lua_State* L) int n = lua_objlen(L, 1); int pos = luaL_optinteger(L, 2, n); - if (!(1 <= pos && pos <= n)) /* position is outside bounds? */ - return 0; /* nothing to remove */ - lua_rawgeti(L, 1, pos); /* result = t[pos] */ + if (!(1 <= pos && pos <= n)) // position is outside bounds? + return 0; // nothing to remove + lua_rawgeti(L, 1, pos); // result = t[pos] moveelements(L, 1, 1, pos + 1, n, pos); lua_pushnil(L); - lua_rawseti(L, 1, n); /* t[n] = nil */ + lua_rawseti(L, 1, n); // t[n] = nil return 1; } @@ -192,28 +192,28 @@ static int tmove(lua_State* L) int f = luaL_checkinteger(L, 2); int e = luaL_checkinteger(L, 3); int t = luaL_checkinteger(L, 4); - int tt = !lua_isnoneornil(L, 5) ? 5 : 1; /* destination table */ + int tt = !lua_isnoneornil(L, 5) ? 5 : 1; // destination table luaL_checktype(L, tt, LUA_TTABLE); if (e >= f) - { /* otherwise, nothing to move */ + { // otherwise, nothing to move luaL_argcheck(L, f > 0 || e < INT_MAX + f, 3, "too many elements to move"); - int n = e - f + 1; /* number of elements to move */ + int n = e - f + 1; // number of elements to move luaL_argcheck(L, t <= INT_MAX - n + 1, 4, "destination wrap around"); Table* dst = hvalue(L->base + (tt - 1)); - if (dst->readonly) /* also checked in moveelements, but this blocks resizes of r/o tables */ + if (dst->readonly) // also checked in moveelements, but this blocks resizes of r/o tables luaG_readonlyerror(L); if (t > 0 && (t - 1) <= dst->sizearray && (t - 1 + n) > dst->sizearray) - { /* grow the destination table array */ + { // grow the destination table array luaH_resizearray(L, dst, t - 1 + n); } moveelements(L, 1, tt, f, e, t); } - lua_pushvalue(L, tt); /* return destination table */ + lua_pushvalue(L, tt); // return destination table return 1; } @@ -240,7 +240,7 @@ static int tconcat(lua_State* L) addfield(L, &b, i); luaL_addlstring(&b, sep, lsep); } - if (i == last) /* add last value (if interval was not empty) */ + if (i == last) // add last value (if interval was not empty) addfield(L, &b, i); luaL_pushresult(&b); return 1; @@ -248,8 +248,8 @@ static int tconcat(lua_State* L) static int tpack(lua_State* L) { - int n = lua_gettop(L); /* number of elements to pack */ - lua_createtable(L, n, 1); /* create result table */ + int n = lua_gettop(L); // number of elements to pack + lua_createtable(L, n, 1); // create result table Table* t = hvalue(L->top - 1); @@ -259,11 +259,11 @@ static int tpack(lua_State* L) setobj2t(L, e, L->base + i); } - /* t.n = number of elements */ + // t.n = number of elements TValue* nv = luaH_setstr(L, t, luaS_newliteral(L, "n")); setnvalue(nv, n); - return 1; /* return table */ + return 1; // return table } static int tunpack(lua_State* L) @@ -274,8 +274,8 @@ static int tunpack(lua_State* L) int i = luaL_optinteger(L, 2, 1); int e = luaL_opt(L, luaL_checkinteger, 3, lua_objlen(L, 1)); if (i > e) - return 0; /* empty range */ - unsigned n = (unsigned)e - i; /* number of elements minus 1 (avoid overflows) */ + return 0; // empty range + unsigned n = (unsigned)e - i; // number of elements minus 1 (avoid overflows) if (n >= (unsigned int)INT_MAX || !lua_checkstack(L, (int)(++n))) luaL_error(L, "too many results to unpack"); @@ -288,10 +288,10 @@ static int tunpack(lua_State* L) } else { - /* push arg[i..e - 1] (to avoid overflows) */ + // push arg[i..e - 1] (to avoid overflows) for (; i < e; i++) lua_rawgeti(L, 1, i); - lua_rawgeti(L, 1, e); /* push last element */ + lua_rawgeti(L, 1, e); // push last element } return (int)n; } @@ -312,85 +312,85 @@ static void set2(lua_State* L, int i, int j) static int sort_comp(lua_State* L, int a, int b) { if (!lua_isnil(L, 2)) - { /* function? */ + { // function? int res; lua_pushvalue(L, 2); - lua_pushvalue(L, a - 1); /* -1 to compensate function */ - lua_pushvalue(L, b - 2); /* -2 to compensate function and `a' */ + lua_pushvalue(L, a - 1); // -1 to compensate function + lua_pushvalue(L, b - 2); // -2 to compensate function and `a' lua_call(L, 2, 1); res = lua_toboolean(L, -1); lua_pop(L, 1); return res; } - else /* a < b? */ + else // a < b? return lua_lessthan(L, a, b); } static void auxsort(lua_State* L, int l, int u) { while (l < u) - { /* for tail recursion */ + { // for tail recursion int i, j; - /* sort elements a[l], a[(l+u)/2] and a[u] */ + // sort elements a[l], a[(l+u)/2] and a[u] lua_rawgeti(L, 1, l); lua_rawgeti(L, 1, u); - if (sort_comp(L, -1, -2)) /* a[u] < a[l]? */ - set2(L, l, u); /* swap a[l] - a[u] */ + if (sort_comp(L, -1, -2)) // a[u] < a[l]? + set2(L, l, u); // swap a[l] - a[u] else lua_pop(L, 2); if (u - l == 1) - break; /* only 2 elements */ + break; // only 2 elements i = (l + u) / 2; lua_rawgeti(L, 1, i); lua_rawgeti(L, 1, l); - if (sort_comp(L, -2, -1)) /* a[i]= P */ + { // invariant: a[l..i] <= P <= a[j..u] + // repeat ++i until a[i] >= P while (lua_rawgeti(L, 1, ++i), sort_comp(L, -1, -2)) { if (i >= u) luaL_error(L, "invalid order function for sorting"); - lua_pop(L, 1); /* remove a[i] */ + lua_pop(L, 1); // remove a[i] } - /* repeat --j until a[j] <= P */ + // repeat --j until a[j] <= P while (lua_rawgeti(L, 1, --j), sort_comp(L, -3, -1)) { if (j <= l) luaL_error(L, "invalid order function for sorting"); - lua_pop(L, 1); /* remove a[j] */ + lua_pop(L, 1); // remove a[j] } if (j < i) { - lua_pop(L, 3); /* pop pivot, a[i], a[j] */ + lua_pop(L, 3); // pop pivot, a[i], a[j] break; } set2(L, i, j); } lua_rawgeti(L, 1, u - 1); lua_rawgeti(L, 1, i); - set2(L, u - 1, i); /* swap pivot (a[u-1]) with a[i] */ - /* a[l..i-1] <= a[i] == P <= a[i+1..u] */ - /* adjust so that smaller half is in [j..i] and larger one in [l..u] */ + set2(L, u - 1, i); // swap pivot (a[u-1]) with a[i] + // a[l..i-1] <= a[i] == P <= a[i+1..u] + // adjust so that smaller half is in [j..i] and larger one in [l..u] if (i - l < u - i) { j = l; @@ -403,23 +403,23 @@ static void auxsort(lua_State* L, int l, int u) i = u; u = j - 2; } - auxsort(L, j, i); /* call recursively the smaller one */ - } /* repeat the routine for the larger one */ + auxsort(L, j, i); // call recursively the smaller one + } // repeat the routine for the larger one } static int sort(lua_State* L) { luaL_checktype(L, 1, LUA_TTABLE); int n = lua_objlen(L, 1); - luaL_checkstack(L, 40, ""); /* assume array is smaller than 2^40 */ - if (!lua_isnoneornil(L, 2)) /* is there a 2nd argument? */ + luaL_checkstack(L, 40, ""); // assume array is smaller than 2^40 + if (!lua_isnoneornil(L, 2)) // is there a 2nd argument? luaL_checktype(L, 2, LUA_TFUNCTION); - lua_settop(L, 2); /* make sure there is two arguments */ + lua_settop(L, 2); // make sure there is two arguments auxsort(L, 1, n); return 0; } -/* }====================================================== */ +// }====================================================== static int tcreate(lua_State* L) { diff --git a/VM/src/ltm.cpp b/VM/src/ltm.cpp index 49982b28..cb7ba097 100644 --- a/VM/src/ltm.cpp +++ b/VM/src/ltm.cpp @@ -12,7 +12,7 @@ // clang-format off const char* const luaT_typenames[] = { - /* ORDER TYPE */ + // ORDER TYPE "nil", "boolean", @@ -31,7 +31,7 @@ const char* const luaT_typenames[] = { }; const char* const luaT_eventname[] = { - /* ORDER TM */ + // ORDER TM "__index", "__newindex", @@ -70,12 +70,12 @@ void luaT_init(lua_State* L) for (i = 0; i < LUA_T_COUNT; i++) { L->global->ttname[i] = luaS_new(L, luaT_typenames[i]); - luaS_fix(L->global->ttname[i]); /* never collect these names */ + luaS_fix(L->global->ttname[i]); // never collect these names } for (i = 0; i < TM_N; i++) { L->global->tmname[i] = luaS_new(L, luaT_eventname[i]); - luaS_fix(L->global->tmname[i]); /* never collect these names */ + luaS_fix(L->global->tmname[i]); // never collect these names } } @@ -88,8 +88,8 @@ const TValue* luaT_gettm(Table* events, TMS event, TString* ename) const TValue* tm = luaH_getstr(events, ename); LUAU_ASSERT(event <= TM_EQ); if (ttisnil(tm)) - { /* no tag method? */ - events->tmcache |= cast_byte(1u << event); /* cache this fact */ + { // no tag method? + events->tmcache |= cast_byte(1u << event); // cache this fact return NULL; } else diff --git a/VM/src/ltm.h b/VM/src/ltm.h index e11ddb3a..f20ce1b2 100644 --- a/VM/src/ltm.h +++ b/VM/src/ltm.h @@ -20,7 +20,7 @@ typedef enum TM_ITER, TM_LEN, - TM_EQ, /* last tag method with `fast' access */ + TM_EQ, // last tag method with `fast' access TM_ADD, @@ -37,7 +37,7 @@ typedef enum TM_CONCAT, TM_TYPE, - TM_N /* number of elements in the enum */ + TM_N // number of elements in the enum } TMS; // clang-format on diff --git a/VM/src/ludata.h b/VM/src/ludata.h index f24e4a32..9b7ba26a 100644 --- a/VM/src/ludata.h +++ b/VM/src/ludata.h @@ -4,10 +4,10 @@ #include "lobject.h" -/* special tag value is used for user data with inline dtors */ +// special tag value is used for user data with inline dtors #define UTAG_IDTOR LUA_UTAG_LIMIT -/* special tag value is used for newproxy-created user data (all other user data objects are host-exposed) */ +// special tag value is used for newproxy-created user data (all other user data objects are host-exposed) #define UTAG_PROXY (LUA_UTAG_LIMIT + 1) #define sizeudata(len) (offsetof(Udata, data) + len) diff --git a/VM/src/lutf8lib.cpp b/VM/src/lutf8lib.cpp index 8bc8200a..837d0e12 100644 --- a/VM/src/lutf8lib.cpp +++ b/VM/src/lutf8lib.cpp @@ -8,8 +8,8 @@ #define iscont(p) ((*(p)&0xC0) == 0x80) -/* from strlib */ -/* translate a relative string position: negative means back from end */ +// from strlib +// translate a relative string position: negative means back from end static int u_posrelat(int pos, size_t len) { if (pos >= 0) @@ -28,28 +28,28 @@ static const char* utf8_decode(const char* o, int* val) static const unsigned int limits[] = {0xFF, 0x7F, 0x7FF, 0xFFFF}; const unsigned char* s = (const unsigned char*)o; unsigned int c = s[0]; - unsigned int res = 0; /* final result */ - if (c < 0x80) /* ascii? */ + unsigned int res = 0; // final result + if (c < 0x80) // ascii? res = c; else { - int count = 0; /* to count number of continuation bytes */ + int count = 0; // to count number of continuation bytes while (c & 0x40) - { /* still have continuation bytes? */ - int cc = s[++count]; /* read next byte */ - if ((cc & 0xC0) != 0x80) /* not a continuation byte? */ - return NULL; /* invalid byte sequence */ - res = (res << 6) | (cc & 0x3F); /* add lower 6 bits from cont. byte */ - c <<= 1; /* to test next bit */ + { // still have continuation bytes? + int cc = s[++count]; // read next byte + if ((cc & 0xC0) != 0x80) // not a continuation byte? + return NULL; // invalid byte sequence + res = (res << 6) | (cc & 0x3F); // add lower 6 bits from cont. byte + c <<= 1; // to test next bit } - res |= ((c & 0x7F) << (count * 5)); /* add first byte */ + res |= ((c & 0x7F) << (count * 5)); // add first byte if (count > 3 || res > MAXUNICODE || res <= limits[count]) - return NULL; /* invalid byte sequence */ - s += count; /* skip continuation bytes read */ + return NULL; // invalid byte sequence + s += count; // skip continuation bytes read } if (val) *val = res; - return (const char*)s + 1; /* +1 to include first byte */ + return (const char*)s + 1; // +1 to include first byte } /* @@ -70,9 +70,9 @@ static int utflen(lua_State* L) { const char* s1 = utf8_decode(s + posi, NULL); if (s1 == NULL) - { /* conversion error? */ - lua_pushnil(L); /* return nil ... */ - lua_pushinteger(L, posi + 1); /* ... and current position */ + { // conversion error? + lua_pushnil(L); // return nil ... + lua_pushinteger(L, posi + 1); // ... and current position return 2; } posi = (int)(s1 - s); @@ -97,8 +97,8 @@ static int codepoint(lua_State* L) luaL_argcheck(L, posi >= 1, 2, "out of range"); luaL_argcheck(L, pose <= (int)len, 3, "out of range"); if (posi > pose) - return 0; /* empty interval; return no values */ - if (pose - posi >= INT_MAX) /* (int -> int) overflow? */ + return 0; // empty interval; return no values + if (pose - posi >= INT_MAX) // (int -> int) overflow? luaL_error(L, "string slice too long"); n = (int)(pose - posi) + 1; luaL_checkstack(L, n, "string slice too long"); @@ -122,20 +122,20 @@ static int codepoint(lua_State* L) // from Lua 5.3 lobject.c, copied verbatim + static static int luaO_utf8esc(char* buff, unsigned long x) { - int n = 1; /* number of bytes put in buffer (backwards) */ + int n = 1; // number of bytes put in buffer (backwards) LUAU_ASSERT(x <= 0x10FFFF); - if (x < 0x80) /* ascii? */ + if (x < 0x80) // ascii? buff[UTF8BUFFSZ - 1] = cast_to(char, x); else - { /* need continuation bytes */ - unsigned int mfb = 0x3f; /* maximum that fits in first byte */ + { // need continuation bytes + unsigned int mfb = 0x3f; // maximum that fits in first byte do - { /* add continuation bytes */ + { // add continuation bytes buff[UTF8BUFFSZ - (n++)] = cast_to(char, 0x80 | (x & 0x3f)); - x >>= 6; /* remove added bits */ - mfb >>= 1; /* now there is one less bit available in first byte */ - } while (x > mfb); /* still needs continuation byte? */ - buff[UTF8BUFFSZ - n] = cast_to(char, (~mfb << 1) | x); /* add first byte */ + x >>= 6; // remove added bits + mfb >>= 1; // now there is one less bit available in first byte + } while (x > mfb); // still needs continuation byte? + buff[UTF8BUFFSZ - n] = cast_to(char, (~mfb << 1) | x); // add first byte } return n; } @@ -162,9 +162,9 @@ static int utfchar(lua_State* L) char buff[UTF8BUFFSZ]; const char* charstr; - int n = lua_gettop(L); /* number of arguments */ + int n = lua_gettop(L); // number of arguments if (n == 1) - { /* optimize common case of single char */ + { // optimize common case of single char int l = buffutfchar(L, 1, buff, &charstr); lua_pushlstring(L, charstr, l); } @@ -196,7 +196,7 @@ static int byteoffset(lua_State* L) luaL_argcheck(L, 1 <= posi && --posi <= (int)len, 3, "position out of range"); if (n == 0) { - /* find beginning of current byte sequence */ + // find beginning of current byte sequence while (posi > 0 && iscont(s + posi)) posi--; } @@ -207,9 +207,9 @@ static int byteoffset(lua_State* L) if (n < 0) { while (n < 0 && posi > 0) - { /* move back */ + { // move back do - { /* find beginning of previous character */ + { // find beginning of previous character posi--; } while (posi > 0 && iscont(s + posi)); n++; @@ -217,20 +217,20 @@ static int byteoffset(lua_State* L) } else { - n--; /* do not move for 1st character */ + n--; // do not move for 1st character while (n > 0 && posi < (int)len) { do - { /* find beginning of next character */ + { // find beginning of next character posi++; - } while (iscont(s + posi)); /* (cannot pass final '\0') */ + } while (iscont(s + posi)); // (cannot pass final '\0') n--; } } } - if (n == 0) /* did it find given character? */ + if (n == 0) // did it find given character? lua_pushinteger(L, posi + 1); - else /* no such character */ + else // no such character lua_pushnil(L); return 1; } @@ -240,16 +240,16 @@ static int iter_aux(lua_State* L) size_t len; const char* s = luaL_checklstring(L, 1, &len); int n = lua_tointeger(L, 2) - 1; - if (n < 0) /* first iteration? */ - n = 0; /* start from here */ + if (n < 0) // first iteration? + n = 0; // start from here else if (n < (int)len) { - n++; /* skip current byte */ + n++; // skip current byte while (iscont(s + n)) - n++; /* and its continuations */ + n++; // and its continuations } if (n >= (int)len) - return 0; /* no more codepoints */ + return 0; // no more codepoints else { int code; @@ -271,7 +271,7 @@ static int iter_codes(lua_State* L) return 3; } -/* pattern to match a single UTF-8 character */ +// pattern to match a single UTF-8 character #define UTF8PATT "[\0-\x7F\xC2-\xF4][\x80-\xBF]*" static const luaL_Reg funcs[] = { diff --git a/VM/src/lvmexecute.cpp b/VM/src/lvmexecute.cpp index 10d89aaf..376dd400 100644 --- a/VM/src/lvmexecute.cpp +++ b/VM/src/lvmexecute.cpp @@ -16,7 +16,7 @@ #include -LUAU_FASTFLAGVARIABLE(LuauLenTM, false) +LUAU_FASTFLAGVARIABLE(LuauNicerMethodErrors, false) // Disable c99-designator to avoid the warning in CGOTO dispatch table #ifdef __clang__ @@ -110,7 +110,8 @@ LUAU_FASTFLAGVARIABLE(LuauLenTM, false) VM_DISPATCH_OP(LOP_FORGLOOP_NEXT), VM_DISPATCH_OP(LOP_GETVARARGS), VM_DISPATCH_OP(LOP_DUPCLOSURE), VM_DISPATCH_OP(LOP_PREPVARARGS), \ VM_DISPATCH_OP(LOP_LOADKX), VM_DISPATCH_OP(LOP_JUMPX), VM_DISPATCH_OP(LOP_FASTCALL), VM_DISPATCH_OP(LOP_COVERAGE), \ VM_DISPATCH_OP(LOP_CAPTURE), VM_DISPATCH_OP(LOP_JUMPIFEQK), VM_DISPATCH_OP(LOP_JUMPIFNOTEQK), VM_DISPATCH_OP(LOP_FASTCALL1), \ - VM_DISPATCH_OP(LOP_FASTCALL2), VM_DISPATCH_OP(LOP_FASTCALL2K), VM_DISPATCH_OP(LOP_FORGPREP), + VM_DISPATCH_OP(LOP_FASTCALL2), VM_DISPATCH_OP(LOP_FASTCALL2K), VM_DISPATCH_OP(LOP_FORGPREP), VM_DISPATCH_OP(LOP_JUMPXEQKNIL), \ + VM_DISPATCH_OP(LOP_JUMPXEQKB), VM_DISPATCH_OP(LOP_JUMPXEQKN), VM_DISPATCH_OP(LOP_JUMPXEQKS), \ #if defined(__GNUC__) || defined(__clang__) #define VM_USE_CGOTO 1 @@ -158,7 +159,7 @@ LUAU_NOINLINE static bool luau_loopFORG(lua_State* L, int a, int c) setobjs2s(L, ra + 3 + 1, ra + 1); setobjs2s(L, ra + 3, ra); - L->top = ra + 3 + 3; /* func. + 2 args (state and index) */ + L->top = ra + 3 + 3; // func. + 2 args (state and index) LUAU_ASSERT(L->top <= L->stack_last); luaD_call(L, ra + 3, c); @@ -236,10 +237,10 @@ LUAU_NOINLINE static void luau_tryfuncTM(lua_State* L, StkId func) const TValue* tm = luaT_gettmbyobj(L, func, TM_CALL); if (!ttisfunction(tm)) luaG_typeerror(L, func, "call"); - for (StkId p = L->top; p > func; p--) /* open space for metamethod */ + for (StkId p = L->top; p > func; p--) // open space for metamethod setobjs2s(L, p, p - 1); - L->top++; /* stack space pre-allocated by the caller */ - setobj2s(L, func, tm); /* tag method is the new function to be called */ + L->top++; // stack space pre-allocated by the caller + setobj2s(L, func, tm); // tag method is the new function to be called } LUAU_NOINLINE void luau_callhook(lua_State* L, lua_Hook hook, void* userdata) @@ -256,7 +257,7 @@ LUAU_NOINLINE void luau_callhook(lua_State* L, lua_Hook hook, void* userdata) L->base = L->ci->base; } - luaD_checkstack(L, LUA_MINSTACK); /* ensure minimum stack size */ + luaD_checkstack(L, LUA_MINSTACK); // ensure minimum stack size L->ci->top = L->top + LUA_MINSTACK; LUAU_ASSERT(L->ci->top <= L->stack_last); @@ -929,6 +930,10 @@ static void luau_execute(lua_State* L) VM_PROTECT(luaV_gettable(L, rb, kv, ra)); // save cachedslot to accelerate future lookups; patches currently executing instruction since pc-2 rolls back two pc++ VM_PATCH_C(pc - 2, L->cachedslot); + // recompute ra since stack might have been reallocated + ra = VM_REG(LUAU_INSN_A(insn)); + if (FFlag::LuauNicerMethodErrors && ttisnil(ra)) + luaG_methoderror(L, ra + 1, tsvalue(kv)); } } else @@ -966,6 +971,10 @@ static void luau_execute(lua_State* L) VM_PROTECT(luaV_gettable(L, rb, kv, ra)); // save cachedslot to accelerate future lookups; patches currently executing instruction since pc-2 rolls back two pc++ VM_PATCH_C(pc - 2, L->cachedslot); + // recompute ra since stack might have been reallocated + ra = VM_REG(LUAU_INSN_A(insn)); + if (FFlag::LuauNicerMethodErrors && ttisnil(ra)) + luaG_methoderror(L, ra + 1, tsvalue(kv)); } } else @@ -973,6 +982,10 @@ static void luau_execute(lua_State* L) // slow-path: handles non-table __index setobj2s(L, ra + 1, rb); VM_PROTECT(luaV_gettable(L, rb, kv, ra)); + // recompute ra since stack might have been reallocated + ra = VM_REG(LUAU_INSN_A(insn)); + if (FFlag::LuauNicerMethodErrors && ttisnil(ra)) + luaG_methoderror(L, ra + 1, tsvalue(kv)); } } @@ -1028,7 +1041,7 @@ static void luau_execute(lua_State* L) StkId argi = L->top; StkId argend = L->base + p->numparams; while (argi < argend) - setnilvalue(argi++); /* complete missing arguments */ + setnilvalue(argi++); // complete missing arguments L->top = p->is_vararg ? argi : ci->top; // reentry @@ -2074,7 +2087,7 @@ static void luau_execute(lua_State* L) { Table* h = hvalue(rb); - if (!FFlag::LuauLenTM || fastnotm(h->metatable, TM_LEN)) + if (fastnotm(h->metatable, TM_LEN)) { setnvalue(ra, cast_num(luaH_getn(h))); VM_NEXT(); @@ -2214,7 +2227,7 @@ static void luau_execute(lua_State* L) if (ttisfunction(ra)) { - /* will be called during FORGLOOP */ + // will be called during FORGLOOP } else { @@ -2225,16 +2238,16 @@ static void luau_execute(lua_State* L) setobj2s(L, ra + 1, ra); setobj2s(L, ra, fn); - L->top = ra + 2; /* func + self arg */ + L->top = ra + 2; // func + self arg LUAU_ASSERT(L->top <= L->stack_last); VM_PROTECT(luaD_call(L, ra, 3)); L->top = L->ci->top; - /* recompute ra since stack might have been reallocated */ + // recompute ra since stack might have been reallocated ra = VM_REG(LUAU_INSN_A(insn)); - /* protect against __iter returning nil, since nil is used as a marker for builtin iteration in FORGLOOP */ + // protect against __iter returning nil, since nil is used as a marker for builtin iteration in FORGLOOP if (ttisnil(ra)) { VM_PROTECT(luaG_typeerror(L, ra, "call")); @@ -2242,12 +2255,12 @@ static void luau_execute(lua_State* L) } else if (fasttm(L, mt, TM_CALL)) { - /* table or userdata with __call, will be called during FORGLOOP */ - /* TODO: we might be able to stop supporting this depending on whether it's used in practice */ + // table or userdata with __call, will be called during FORGLOOP + // TODO: we might be able to stop supporting this depending on whether it's used in practice } else if (ttistable(ra)) { - /* set up registers for builtin iteration */ + // set up registers for builtin iteration setobj2s(L, ra + 1, ra); setpvalue(ra + 2, reinterpret_cast(uintptr_t(0))); setnilvalue(ra); @@ -2344,7 +2357,7 @@ static void luau_execute(lua_State* L) setobjs2s(L, ra + 3 + 1, ra + 1); setobjs2s(L, ra + 3, ra); - L->top = ra + 3 + 3; /* func + 2 args (state and index) */ + L->top = ra + 3 + 3; // func + 2 args (state and index) LUAU_ASSERT(L->top <= L->stack_last); VM_PROTECT(luaD_call(L, ra + 3, uint8_t(aux))); @@ -2372,7 +2385,7 @@ static void luau_execute(lua_State* L) if (cl->env->safeenv && ttistable(ra + 1) && ttisnumber(ra + 2) && nvalue(ra + 2) == 0.0) { setnilvalue(ra); - /* ra+1 is already the table */ + // ra+1 is already the table setpvalue(ra + 2, reinterpret_cast(uintptr_t(0))); } else if (!ttisfunction(ra)) @@ -2444,7 +2457,7 @@ static void luau_execute(lua_State* L) if (cl->env->safeenv && ttistable(ra + 1) && ttisnil(ra + 2)) { setnilvalue(ra); - /* ra+1 is already the table */ + // ra+1 is already the table setpvalue(ra + 2, reinterpret_cast(uintptr_t(0))); } else if (!ttisfunction(ra)) @@ -2619,8 +2632,8 @@ static void luau_execute(lua_State* L) LUAU_ASSERT(cast_int(L->top - base) >= numparams); // move fixed parameters to final position - StkId fixed = base; /* first fixed argument */ - base = L->top; /* final position of first argument */ + StkId fixed = base; // first fixed argument + base = L->top; // final position of first argument for (int i = 0; i < numparams; ++i) { @@ -2983,6 +2996,56 @@ static void luau_execute(lua_State* L) VM_CONTINUE(op); } + VM_CASE(LOP_JUMPXEQKNIL) + { + Instruction insn = *pc++; + uint32_t aux = *pc; + StkId ra = VM_REG(LUAU_INSN_A(insn)); + + static_assert(LUA_TNIL == 0, "we expect type-1 to be negative iff type is nil"); + // condition is equivalent to: int(ttisnil(ra)) != (aux >> 31) + pc += int((ttype(ra) - 1) ^ aux) < 0 ? LUAU_INSN_D(insn) : 1; + LUAU_ASSERT(unsigned(pc - cl->l.p->code) < unsigned(cl->l.p->sizecode)); + VM_NEXT(); + } + + VM_CASE(LOP_JUMPXEQKB) + { + Instruction insn = *pc++; + uint32_t aux = *pc; + StkId ra = VM_REG(LUAU_INSN_A(insn)); + + pc += int(ttisboolean(ra) && bvalue(ra) == int(aux & 1)) != (aux >> 31) ? LUAU_INSN_D(insn) : 1; + LUAU_ASSERT(unsigned(pc - cl->l.p->code) < unsigned(cl->l.p->sizecode)); + VM_NEXT(); + } + + VM_CASE(LOP_JUMPXEQKN) + { + Instruction insn = *pc++; + uint32_t aux = *pc; + StkId ra = VM_REG(LUAU_INSN_A(insn)); + TValue* kv = VM_KV(aux & 0xffffff); + LUAU_ASSERT(ttisnumber(kv)); + + pc += int(ttisnumber(ra) && nvalue(ra) == nvalue(kv)) != (aux >> 31) ? LUAU_INSN_D(insn) : 1; + LUAU_ASSERT(unsigned(pc - cl->l.p->code) < unsigned(cl->l.p->sizecode)); + VM_NEXT(); + } + + VM_CASE(LOP_JUMPXEQKS) + { + Instruction insn = *pc++; + uint32_t aux = *pc; + StkId ra = VM_REG(LUAU_INSN_A(insn)); + TValue* kv = VM_KV(aux & 0xffffff); + LUAU_ASSERT(ttisstring(kv)); + + pc += int(ttisstring(ra) && gcvalue(ra) == gcvalue(kv)) != (aux >> 31) ? LUAU_INSN_D(insn) : 1; + LUAU_ASSERT(unsigned(pc - cl->l.p->code) < unsigned(cl->l.p->sizecode)); + VM_NEXT(); + } + #if !VM_USE_CGOTO default: LUAU_ASSERT(!"Unknown opcode"); @@ -3032,7 +3095,7 @@ int luau_precall(lua_State* L, StkId func, int nresults) StkId argi = L->top; StkId argend = L->base + ccl->l.p->numparams; while (argi < argend) - setnilvalue(argi++); /* complete missing arguments */ + setnilvalue(argi++); // complete missing arguments L->top = ccl->l.p->is_vararg ? argi : ci->top; L->ci->savedpc = ccl->l.p->code; diff --git a/VM/src/lvmutils.cpp b/VM/src/lvmutils.cpp index b9267fb8..8be241e0 100644 --- a/VM/src/lvmutils.cpp +++ b/VM/src/lvmutils.cpp @@ -12,11 +12,9 @@ #include #include -LUAU_FASTFLAG(LuauLenTM) - LUAU_FASTFLAGVARIABLE(LuauBetterNewindex, false) -/* limit for table tag-method chains (to avoid loops) */ +// limit for table tag-method chains (to avoid loops) #define MAXTAGLOOP 100 const TValue* luaV_tonumber(const TValue* obj, TValue* n) @@ -67,9 +65,9 @@ static StkId callTMres(lua_State* L, StkId res, const TValue* f, const TValue* p // * during stack reallocation all of the allocated stack is copied (even beyond stack_last) so these // values will be preserved even if they go past stack_last LUAU_ASSERT((L->top + 3) < (L->stack + L->stacksize)); - setobj2s(L, L->top, f); /* push function */ - setobj2s(L, L->top + 1, p1); /* 1st argument */ - setobj2s(L, L->top + 2, p2); /* 2nd argument */ + setobj2s(L, L->top, f); // push function + setobj2s(L, L->top + 1, p1); // 1st argument + setobj2s(L, L->top + 2, p2); // 2nd argument luaD_checkstack(L, 3); L->top += 3; luaD_call(L, L->top - 3, 1); @@ -89,10 +87,10 @@ static void callTM(lua_State* L, const TValue* f, const TValue* p1, const TValue // * during stack reallocation all of the allocated stack is copied (even beyond stack_last) so these // values will be preserved even if they go past stack_last LUAU_ASSERT((L->top + 4) < (L->stack + L->stacksize)); - setobj2s(L, L->top, f); /* push function */ - setobj2s(L, L->top + 1, p1); /* 1st argument */ - setobj2s(L, L->top + 2, p2); /* 2nd argument */ - setobj2s(L, L->top + 3, p3); /* 3th argument */ + setobj2s(L, L->top, f); // push function + setobj2s(L, L->top + 1, p1); // 1st argument + setobj2s(L, L->top + 2, p2); // 2nd argument + setobj2s(L, L->top + 3, p3); // 3th argument luaD_checkstack(L, 4); L->top += 4; luaD_call(L, L->top - 4, 0); @@ -105,21 +103,21 @@ void luaV_gettable(lua_State* L, const TValue* t, TValue* key, StkId val) { const TValue* tm; if (ttistable(t)) - { /* `t' is a table? */ + { // `t' is a table? Table* h = hvalue(t); - const TValue* res = luaH_get(h, key); /* do a primitive get */ + const TValue* res = luaH_get(h, key); // do a primitive get if (res != luaO_nilobject) - L->cachedslot = gval2slot(h, res); /* remember slot to accelerate future lookups */ + L->cachedslot = gval2slot(h, res); // remember slot to accelerate future lookups - if (!ttisnil(res) /* result is no nil? */ + if (!ttisnil(res) // result is no nil? || (tm = fasttm(L, h->metatable, TM_INDEX)) == NULL) - { /* or no TM? */ + { // or no TM? setobj2s(L, val, res); return; } - /* t isn't a table, so see if it has an INDEX meta-method to look up the key with */ + // t isn't a table, so see if it has an INDEX meta-method to look up the key with } else if (ttisnil(tm = luaT_gettmbyobj(L, t, TM_INDEX))) luaG_indexerror(L, t, key); @@ -128,7 +126,7 @@ void luaV_gettable(lua_State* L, const TValue* t, TValue* key, StkId val) callTMres(L, val, tm, t, key); return; } - t = tm; /* else repeat with `tm' */ + t = tm; // else repeat with `tm' } luaG_runerror(L, "'__index' chain too long; possible loop"); } @@ -141,48 +139,48 @@ void luaV_settable(lua_State* L, const TValue* t, TValue* key, StkId val) { const TValue* tm; if (ttistable(t)) - { /* `t' is a table? */ + { // `t' is a table? Table* h = hvalue(t); if (FFlag::LuauBetterNewindex) { const TValue* oldval = luaH_get(h, key); - /* should we assign the key? (if key is valid or __newindex is not set) */ + // should we assign the key? (if key is valid or __newindex is not set) if (!ttisnil(oldval) || (tm = fasttm(L, h->metatable, TM_NEWINDEX)) == NULL) { if (h->readonly) luaG_readonlyerror(L); - /* luaH_set would work but would repeat the lookup so we use luaH_setslot that can reuse oldval if it's safe */ + // luaH_set would work but would repeat the lookup so we use luaH_setslot that can reuse oldval if it's safe TValue* newval = luaH_setslot(L, h, oldval, key); - L->cachedslot = gval2slot(h, newval); /* remember slot to accelerate future lookups */ + L->cachedslot = gval2slot(h, newval); // remember slot to accelerate future lookups setobj2t(L, newval, val); luaC_barriert(L, h, val); return; } - /* fallthrough to metamethod */ + // fallthrough to metamethod } else { if (h->readonly) luaG_readonlyerror(L); - TValue* oldval = luaH_set(L, h, key); /* do a primitive set */ + TValue* oldval = luaH_set(L, h, key); // do a primitive set - L->cachedslot = gval2slot(h, oldval); /* remember slot to accelerate future lookups */ + L->cachedslot = gval2slot(h, oldval); // remember slot to accelerate future lookups - if (!ttisnil(oldval) || /* result is no nil? */ + if (!ttisnil(oldval) || // result is no nil? (tm = fasttm(L, h->metatable, TM_NEWINDEX)) == NULL) - { /* or no TM? */ + { // or no TM? setobj2t(L, oldval, val); luaC_barriert(L, h, val); return; } - /* else will try the tag method */ + // else will try the tag method } } else if (ttisnil(tm = luaT_gettmbyobj(L, t, TM_NEWINDEX))) @@ -193,8 +191,8 @@ void luaV_settable(lua_State* L, const TValue* t, TValue* key, StkId val) callTM(L, tm, t, key, val); return; } - /* else repeat with `tm' */ - setobj(L, &temp, tm); /* avoid pointing inside table (may rehash) */ + // else repeat with `tm' + setobj(L, &temp, tm); // avoid pointing inside table (may rehash) t = &temp; } luaG_runerror(L, "'__newindex' chain too long; possible loop"); @@ -202,9 +200,9 @@ void luaV_settable(lua_State* L, const TValue* t, TValue* key, StkId val) static int call_binTM(lua_State* L, const TValue* p1, const TValue* p2, StkId res, TMS event) { - const TValue* tm = luaT_gettmbyobj(L, p1, event); /* try first operand */ + const TValue* tm = luaT_gettmbyobj(L, p1, event); // try first operand if (ttisnil(tm)) - tm = luaT_gettmbyobj(L, p2, event); /* try second operand */ + tm = luaT_gettmbyobj(L, p2, event); // try second operand if (ttisnil(tm)) return 0; callTMres(L, res, tm, p1, p2); @@ -216,13 +214,13 @@ static const TValue* get_compTM(lua_State* L, Table* mt1, Table* mt2, TMS event) const TValue* tm1 = fasttm(L, mt1, event); const TValue* tm2; if (tm1 == NULL) - return NULL; /* no metamethod */ + return NULL; // no metamethod if (mt1 == mt2) - return tm1; /* same metatables => same metamethods */ + return tm1; // same metatables => same metamethods tm2 = fasttm(L, mt2, event); if (tm2 == NULL) - return NULL; /* no metamethod */ - if (luaO_rawequalObj(tm1, tm2)) /* same metamethods? */ + return NULL; // no metamethod + if (luaO_rawequalObj(tm1, tm2)) // same metamethods? return tm1; return NULL; } @@ -232,9 +230,9 @@ static int call_orderTM(lua_State* L, const TValue* p1, const TValue* p2, TMS ev const TValue* tm1 = luaT_gettmbyobj(L, p1, event); const TValue* tm2; if (ttisnil(tm1)) - return -1; /* no metamethod? */ + return -1; // no metamethod? tm2 = luaT_gettmbyobj(L, p2, event); - if (!luaO_rawequalObj(tm1, tm2)) /* different metamethods? */ + if (!luaO_rawequalObj(tm1, tm2)) // different metamethods? return -1; callTMres(L, L->top, tm1, p1, p2); return !l_isfalse(L->top); @@ -281,9 +279,9 @@ int luaV_lessequal(lua_State* L, const TValue* l, const TValue* r) return luai_numle(nvalue(l), nvalue(r)); else if (ttisstring(l)) return luaV_strcmp(tsvalue(l), tsvalue(r)) <= 0; - else if ((res = call_orderTM(L, l, r, TM_LE)) != -1) /* first try `le' */ + else if ((res = call_orderTM(L, l, r, TM_LE)) != -1) // first try `le' return res; - else if ((res = call_orderTM(L, r, l, TM_LT)) == -1) /* error if not `lt' */ + else if ((res = call_orderTM(L, r, l, TM_LT)) == -1) // error if not `lt' luaG_ordererror(L, l, r, TM_LE); return !res; } @@ -301,7 +299,7 @@ int luaV_equalval(lua_State* L, const TValue* t1, const TValue* t2) case LUA_TVECTOR: return luai_veceq(vvalue(t1), vvalue(t2)); case LUA_TBOOLEAN: - return bvalue(t1) == bvalue(t2); /* true must be 1 !! */ + return bvalue(t1) == bvalue(t2); // true must be 1 !! case LUA_TLIGHTUSERDATA: return pvalue(t1) == pvalue(t2); case LUA_TUSERDATA: @@ -309,19 +307,19 @@ int luaV_equalval(lua_State* L, const TValue* t1, const TValue* t2) tm = get_compTM(L, uvalue(t1)->metatable, uvalue(t2)->metatable, TM_EQ); if (!tm) return uvalue(t1) == uvalue(t2); - break; /* will try TM */ + break; // will try TM } case LUA_TTABLE: { tm = get_compTM(L, hvalue(t1)->metatable, hvalue(t2)->metatable, TM_EQ); if (!tm) return hvalue(t1) == hvalue(t2); - break; /* will try TM */ + break; // will try TM } default: return gcvalue(t1) == gcvalue(t2); } - callTMres(L, L->top, tm, t1, t2); /* call TM */ + callTMres(L, L->top, tm, t1, t2); // call TM return !l_isfalse(L->top); } @@ -330,21 +328,21 @@ void luaV_concat(lua_State* L, int total, int last) do { StkId top = L->base + last + 1; - int n = 2; /* number of elements handled in this pass (at least 2) */ + int n = 2; // number of elements handled in this pass (at least 2) if (!(ttisstring(top - 2) || ttisnumber(top - 2)) || !tostring(L, top - 1)) { if (!call_binTM(L, top - 2, top - 1, top - 2, TM_CONCAT)) luaG_concaterror(L, top - 2, top - 1); } - else if (tsvalue(top - 1)->len == 0) /* second op is empty? */ - (void)tostring(L, top - 2); /* result is first op (as string) */ + else if (tsvalue(top - 1)->len == 0) // second op is empty? + (void)tostring(L, top - 2); // result is first op (as string) else { - /* at least two string values; get as many as possible */ + // at least two string values; get as many as possible size_t tl = tsvalue(top - 1)->len; char* buffer; int i; - /* collect total length */ + // collect total length for (n = 1; n < total && tostring(L, top - n - 1); n++) { size_t l = tsvalue(top - n - 1)->len; @@ -368,7 +366,7 @@ void luaV_concat(lua_State* L, int total, int last) tl = 0; for (i = n; i > 0; i--) - { /* concat all strings */ + { // concat all strings size_t l = tsvalue(top - i)->len; memcpy(buffer + tl, svalue(top - i), l); tl += l; @@ -383,9 +381,9 @@ void luaV_concat(lua_State* L, int total, int last) setsvalue2s(L, top - n, luaS_buffinish(L, ts)); } } - total -= n - 1; /* got `n' strings to create 1 new */ + total -= n - 1; // got `n' strings to create 1 new last -= n - 1; - } while (total > 1); /* repeat until only 1 result left */ + } while (total > 1); // repeat until only 1 result left } void luaV_doarith(lua_State* L, StkId ra, const TValue* rb, const TValue* rc, TMS op) @@ -504,29 +502,6 @@ void luaV_doarith(lua_State* L, StkId ra, const TValue* rb, const TValue* rc, TM void luaV_dolen(lua_State* L, StkId ra, const TValue* rb) { - if (!FFlag::LuauLenTM) - { - switch (ttype(rb)) - { - case LUA_TTABLE: - { - setnvalue(ra, cast_num(luaH_getn(hvalue(rb)))); - break; - } - case LUA_TSTRING: - { - setnvalue(ra, cast_num(tsvalue(rb)->len)); - break; - } - default: - { /* try metamethod */ - if (!call_binTM(L, rb, luaO_nilobject, ra, TM_LEN)) - luaG_typeerror(L, rb, "get length of"); - } - } - return; - } - const TValue* tm = NULL; switch (ttype(rb)) { @@ -555,5 +530,5 @@ void luaV_dolen(lua_State* L, StkId ra, const TValue* rb) StkId res = callTMres(L, ra, tm, rb, luaO_nilobject); if (!ttisnumber(res)) - luaG_runerror(L, "'__len' must return a number"); /* note, we can't access rb since stack may have been reallocated */ + luaG_runerror(L, "'__len' must return a number"); // note, we can't access rb since stack may have been reallocated } diff --git a/tests/JsonEncoder.test.cpp b/tests/AstJsonEncoder.test.cpp similarity index 99% rename from tests/JsonEncoder.test.cpp rename to tests/AstJsonEncoder.test.cpp index b16ad3e2..3ff36741 100644 --- a/tests/JsonEncoder.test.cpp +++ b/tests/AstJsonEncoder.test.cpp @@ -1,6 +1,6 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #include "Luau/Ast.h" -#include "Luau/JsonEncoder.h" +#include "Luau/AstJsonEncoder.h" #include "Luau/Parser.h" #include "doctest.h" @@ -181,7 +181,8 @@ TEST_CASE("encode_AstExprLocal") AstLocal local{AstName{"foo"}, Location{}, nullptr, 0, 0, nullptr}; AstExprLocal exprLocal{Location{}, &local, false}; - CHECK(toJson(&exprLocal) == R"({"type":"AstExprLocal","location":"0,0 - 0,0","local":{"luauType":null,"name":"foo","type":"AstLocal","location":"0,0 - 0,0"}})"); + CHECK(toJson(&exprLocal) == + R"({"type":"AstExprLocal","location":"0,0 - 0,0","local":{"luauType":null,"name":"foo","type":"AstLocal","location":"0,0 - 0,0"}})"); } TEST_CASE("encode_AstExprVarargs") diff --git a/tests/Autocomplete.test.cpp b/tests/Autocomplete.test.cpp index 0a5603d6..75c5a606 100644 --- a/tests/Autocomplete.test.cpp +++ b/tests/Autocomplete.test.cpp @@ -1974,7 +1974,7 @@ TEST_CASE_FIXTURE(ACFixture, "function_result_passed_to_function_has_parentheses check(R"( local function foo() return 1 end local function bar(a: number) return -a end -local abc = bar(@1) +local abc = bar(@1) )"); auto ac = autocomplete('1'); diff --git a/tests/Compiler.test.cpp b/tests/Compiler.test.cpp index 4b87c003..0a1c5a8b 100644 --- a/tests/Compiler.test.cpp +++ b/tests/Compiler.test.cpp @@ -2382,11 +2382,13 @@ end TEST_CASE("DebugLineInfoRepeatUntil") { + ScopedFastFlag sff("LuauCompileXEQ", true); + CHECK_EQ("\n" + compileFunction0Coverage(R"( local f = 0 repeat f += 1 - if f == 1 then + if f == 1 then print(f) else f = 0 @@ -2397,13 +2399,13 @@ until f == 0 R"( 2: LOADN R0 0 4: L0: ADDK R0 R0 K0 -5: JUMPIFNOTEQK R0 K0 L1 +5: JUMPXEQKN R0 K0 L1 NOT 6: GETIMPORT R1 2 6: MOVE R2 R0 6: CALL R1 1 0 6: JUMP L2 8: L1: LOADN R0 0 -10: L2: JUMPIFEQK R0 K3 L3 +10: L2: JUMPXEQKN R0 K3 L3 10: JUMPBACK L0 11: L3: RETURN R0 0 )"); @@ -3509,13 +3511,15 @@ RETURN R0 1 TEST_CASE("ConstantJumpCompare") { + ScopedFastFlag sff("LuauCompileXEQ", true); + CHECK_EQ("\n" + compileFunction0(R"( local obj = ... local b = obj == 1 )"), R"( GETVARARGS R0 1 -JUMPIFEQK R0 K0 L0 +JUMPXEQKN R0 K0 L0 LOADB R1 0 +1 L0: LOADB R1 1 L1: RETURN R0 0 @@ -3527,7 +3531,7 @@ local b = 1 == obj )"), R"( GETVARARGS R0 1 -JUMPIFEQK R0 K0 L0 +JUMPXEQKN R0 K0 L0 LOADB R1 0 +1 L0: LOADB R1 1 L1: RETURN R0 0 @@ -3539,7 +3543,7 @@ local b = "Hello, Sailor!" == obj )"), R"( GETVARARGS R0 1 -JUMPIFEQK R0 K0 L0 +JUMPXEQKS R0 K0 L0 LOADB R1 0 +1 L0: LOADB R1 1 L1: RETURN R0 0 @@ -3551,7 +3555,7 @@ local b = nil == obj )"), R"( GETVARARGS R0 1 -JUMPIFEQK R0 K0 L0 +JUMPXEQKNIL R0 L0 LOADB R1 0 +1 L0: LOADB R1 1 L1: RETURN R0 0 @@ -3563,7 +3567,7 @@ local b = true == obj )"), R"( GETVARARGS R0 1 -JUMPIFEQK R0 K0 L0 +JUMPXEQKB R0 1 L0 LOADB R1 0 +1 L0: LOADB R1 1 L1: RETURN R0 0 @@ -3575,7 +3579,7 @@ local b = nil ~= obj )"), R"( GETVARARGS R0 1 -JUMPIFNOTEQK R0 K0 L0 +JUMPXEQKNIL R0 L0 NOT LOADB R1 0 +1 L0: LOADB R1 1 L1: RETURN R0 0 @@ -6098,6 +6102,8 @@ L3: RETURN R0 -1 TEST_CASE("BuiltinFoldingMultret") { + ScopedFastFlag sff("LuauCompileXEQ", true); + CHECK_EQ("\n" + compileFunction(R"( local NoLanes: Lanes = --[[ ]] 0b0000000000000000000000000000000 local OffscreenLane: Lane = --[[ ]] 0b1000000000000000000000000000000 @@ -6120,14 +6126,14 @@ FASTCALL2K 29 R2 K1 L0 LOADK R3 K1 GETIMPORT R1 4 CALL R1 2 1 -L0: JUMPIFEQK R1 K5 L1 +L0: JUMPXEQKN R1 K5 L1 RETURN R1 1 L1: FASTCALL2K 29 R1 K6 L2 MOVE R3 R1 LOADK R4 K6 GETIMPORT R2 4 CALL R2 2 1 -L2: JUMPIFEQK R2 K5 L3 +L2: JUMPXEQKN R2 K5 L3 LOADK R2 K6 RETURN R2 1 L3: LOADN R2 0 @@ -6155,7 +6161,8 @@ local function test(a, b) local c = a return c + b end -)"), R"( +)"), + R"( ADD R2 R0 R1 RETURN R2 1 )"); @@ -6166,7 +6173,8 @@ local function test(a, b) local c = (a :: number) return c + b end -)"), R"( +)"), + R"( ADD R2 R0 R1 RETURN R2 1 )"); @@ -6180,7 +6188,8 @@ local function test(a, b) b += 0 return c + d end -)"), R"( +)"), + R"( MOVE R2 R0 ADDK R2 R2 K0 MOVE R3 R1 @@ -6196,7 +6205,8 @@ local function test(a, b) local d = b return c + d end -)"), R"( +)"), + R"( ADD R2 R0 R1 RETURN R2 1 )"); @@ -6207,7 +6217,8 @@ local function test(a, b) local c, d = a, b return c + d end -)"), R"( +)"), + R"( MOVE R2 R0 MOVE R3 R1 ADD R4 R2 R3 @@ -6221,7 +6232,9 @@ local function test(a, b) local d = b return function() return c + d end end -)", 1), R"( +)", + 1), + R"( NEWCLOSURE R2 P0 CAPTURE VAL R0 CAPTURE VAL R1 diff --git a/tests/Conformance.test.cpp b/tests/Conformance.test.cpp index 5758f86c..e07ba12a 100644 --- a/tests/Conformance.test.cpp +++ b/tests/Conformance.test.cpp @@ -70,8 +70,8 @@ static int lua_loadstring(lua_State* L) return 1; lua_pushnil(L); - lua_insert(L, -2); /* put before error message */ - return 2; /* return nil plus error message */ + lua_insert(L, -2); // put before error message + return 2; // return nil plus error message } static int lua_vector(lua_State* L) @@ -244,8 +244,6 @@ TEST_CASE("Assert") TEST_CASE("Basic") { - ScopedFastFlag sff("LuauLenTM", true); - runConformance("basic.lua"); } @@ -313,13 +311,14 @@ TEST_CASE("Literals") TEST_CASE("Errors") { + ScopedFastFlag sff("LuauNicerMethodErrors", true); + runConformance("errors.lua"); } TEST_CASE("Events") { - ScopedFastFlag sff1("LuauLenTM", true); - ScopedFastFlag sff2("LuauBetterNewindex", true); + ScopedFastFlag sff("LuauBetterNewindex", true); runConformance("events.lua"); } diff --git a/tests/Fixture.cpp b/tests/Fixture.cpp index 90aa6236..f51a9d1b 100644 --- a/tests/Fixture.cpp +++ b/tests/Fixture.cpp @@ -410,7 +410,7 @@ void Fixture::validateErrors(const std::vector& errors) LoadDefinitionFileResult Fixture::loadDefinition(const std::string& source) { unfreeze(typeChecker.globalTypes); - LoadDefinitionFileResult result = loadDefinitionFile(typeChecker, typeChecker.globalScope, source, "@test"); + LoadDefinitionFileResult result = frontend.loadDefinitionFile(source, "@test"); freeze(typeChecker.globalTypes); REQUIRE_MESSAGE(result.success, "loadDefinition: unable to load definition file"); diff --git a/tests/Frontend.test.cpp b/tests/Frontend.test.cpp index 0382f227..3c27ad54 100644 --- a/tests/Frontend.test.cpp +++ b/tests/Frontend.test.cpp @@ -1030,7 +1030,6 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "reexport_cyclic_type") ScopedFastFlag sff[] = { {"LuauForceExportSurfacesToBeNormal", true}, {"LuauLowerBoundsCalculation", true}, - {"LuauNormalizeFlagIsConservative", true}, }; fileResolver.source["Module/A"] = R"( @@ -1067,7 +1066,6 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "reexport_type_alias") ScopedFastFlag sff[] = { {"LuauForceExportSurfacesToBeNormal", true}, {"LuauLowerBoundsCalculation", true}, - {"LuauNormalizeFlagIsConservative", true}, }; fileResolver.source["Module/A"] = R"( diff --git a/tests/JsonEmitter.test.cpp b/tests/JsonEmitter.test.cpp new file mode 100644 index 00000000..ebe83209 --- /dev/null +++ b/tests/JsonEmitter.test.cpp @@ -0,0 +1,195 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +#include "Luau/JsonEmitter.h" + +#include "doctest.h" + +using namespace Luau::Json; + +TEST_SUITE_BEGIN("JsonEmitter"); + +TEST_CASE("write_array") +{ + JsonEmitter emitter; + ArrayEmitter a = emitter.writeArray(); + a.writeValue(123); + a.writeValue("foo"); + a.finish(); + + std::string result = emitter.str(); + CHECK(result == "[123,\"foo\"]"); +} + +TEST_CASE("write_object") +{ + JsonEmitter emitter; + ObjectEmitter o = emitter.writeObject(); + o.writePair("foo", "bar"); + o.writePair("bar", "baz"); + o.finish(); + + std::string result = emitter.str(); + CHECK(result == "{\"foo\":\"bar\",\"bar\":\"baz\"}"); +} + +TEST_CASE("write_bool") +{ + JsonEmitter emitter; + write(emitter, false); + CHECK(emitter.str() == "false"); + + emitter = JsonEmitter{}; + write(emitter, true); + CHECK(emitter.str() == "true"); +} + +TEST_CASE("write_null") +{ + JsonEmitter emitter; + write(emitter, nullptr); + CHECK(emitter.str() == "null"); +} + +TEST_CASE("write_string") +{ + JsonEmitter emitter; + write(emitter, R"(foo,bar,baz, +"this should be escaped")"); + CHECK(emitter.str() == "\"foo,bar,baz,\\n\\\"this should be escaped\\\"\""); +} + +TEST_CASE("write_comma") +{ + JsonEmitter emitter; + emitter.writeComma(); + write(emitter, true); + emitter.writeComma(); + write(emitter, false); + CHECK(emitter.str() == "true,false"); +} + +TEST_CASE("push_and_pop_comma") +{ + JsonEmitter emitter; + emitter.writeComma(); + write(emitter, true); + emitter.writeComma(); + emitter.writeRaw('['); + bool comma = emitter.pushComma(); + emitter.writeComma(); + write(emitter, true); + emitter.writeComma(); + write(emitter, false); + emitter.writeRaw(']'); + emitter.popComma(comma); + emitter.writeComma(); + write(emitter, false); + + CHECK(emitter.str() == "true,[true,false],false"); +} + +TEST_CASE("write_optional") +{ + JsonEmitter emitter; + emitter.writeComma(); + write(emitter, std::optional{true}); + emitter.writeComma(); + write(emitter, std::nullopt); + + CHECK(emitter.str() == "true,null"); +} + +TEST_CASE("write_vector") +{ + std::vector values{1, 2, 3, 4}; + JsonEmitter emitter; + write(emitter, values); + CHECK(emitter.str() == "[1,2,3,4]"); +} + +TEST_CASE("prevent_multiple_object_finish") +{ + JsonEmitter emitter; + ObjectEmitter o = emitter.writeObject(); + o.writePair("a", "b"); + o.finish(); + o.finish(); + + CHECK(emitter.str() == "{\"a\":\"b\"}"); +} + +TEST_CASE("prevent_multiple_array_finish") +{ + JsonEmitter emitter; + ArrayEmitter a = emitter.writeArray(); + a.writeValue(1); + a.finish(); + a.finish(); + + CHECK(emitter.str() == "[1]"); +} + +TEST_CASE("cannot_write_pair_after_finished") +{ + JsonEmitter emitter; + ObjectEmitter o = emitter.writeObject(); + o.finish(); + o.writePair("a", "b"); + + CHECK(emitter.str() == "{}"); +} + +TEST_CASE("cannot_write_value_after_finished") +{ + JsonEmitter emitter; + ArrayEmitter a = emitter.writeArray(); + a.finish(); + a.writeValue(1); + + CHECK(emitter.str() == "[]"); +} + +TEST_CASE("finish_when_destructing_object") +{ + JsonEmitter emitter; + emitter.writeObject(); + + CHECK(emitter.str() == "{}"); +} + +TEST_CASE("finish_when_destructing_array") +{ + JsonEmitter emitter; + emitter.writeArray(); + + CHECK(emitter.str() == "[]"); +} + +namespace Luau::Json +{ + +struct Special +{ + int foo; + int bar; +}; + +void write(JsonEmitter& emitter, const Special& value) +{ + ObjectEmitter o = emitter.writeObject(); + o.writePair("foo", value.foo); + o.writePair("bar", value.bar); +} + +} // namespace Luau::Json + +TEST_CASE("afford_extensibility") +{ + std::vector vec{Special{1, 2}, Special{3, 4}}; + JsonEmitter e; + write(e, vec); + + std::string result = e.str(); + CHECK(result == R"([{"foo":1,"bar":2},{"foo":3,"bar":4}])"); +} + +TEST_SUITE_END(); diff --git a/tests/Linter.test.cpp b/tests/Linter.test.cpp index 488ade9f..35c40508 100644 --- a/tests/Linter.test.cpp +++ b/tests/Linter.test.cpp @@ -1675,4 +1675,36 @@ TEST_CASE_FIXTURE(Fixture, "WrongCommentOptimize") CHECK_EQ(result.warnings[3].text, "optimize directive uses unknown optimization level '100500', 0..2 expected"); } +TEST_CASE_FIXTURE(Fixture, "LintIntegerParsing") +{ + ScopedFastFlag luauLintParseIntegerIssues{"LuauLintParseIntegerIssues", true}; + + LintResult result = lint(R"( +local _ = 0b10000000000000000000000000000000000000000000000000000000000000000 +local _ = 0x10000000000000000 +)"); + + REQUIRE_EQ(result.warnings.size(), 2); + CHECK_EQ(result.warnings[0].text, "Binary number literal exceeded available precision and has been truncated to 2^64"); + CHECK_EQ(result.warnings[1].text, "Hexadecimal number literal exceeded available precision and has been truncated to 2^64"); +} + +// TODO: remove with FFlagLuauErrorDoubleHexPrefix +TEST_CASE_FIXTURE(Fixture, "LintIntegerParsingDoublePrefix") +{ + ScopedFastFlag luauLintParseIntegerIssues{"LuauLintParseIntegerIssues", true}; + ScopedFastFlag luauErrorDoubleHexPrefix{"LuauErrorDoubleHexPrefix", false}; // Lint will be available until we start rejecting code + + LintResult result = lint(R"( +local _ = 0x0x123 +local _ = 0x0xffffffffffffffffffffffffffffffffff +)"); + + REQUIRE_EQ(result.warnings.size(), 2); + CHECK_EQ(result.warnings[0].text, + "Hexadecimal number literal has a double prefix, which will fail to parse in the future; remove the extra 0x to fix"); + CHECK_EQ(result.warnings[1].text, + "Hexadecimal number literal has a double prefix, which will fail to parse in the future; remove the extra 0x to fix"); +} + TEST_SUITE_END(); diff --git a/tests/Normalize.test.cpp b/tests/Normalize.test.cpp index 84a5a380..c64c41c5 100644 --- a/tests/Normalize.test.cpp +++ b/tests/Normalize.test.cpp @@ -680,26 +680,12 @@ TEST_CASE_FIXTURE(Fixture, "higher_order_function_with_annotation") CHECK_EQ("((a) -> b, a) -> b", toString(requireType("apply"))); } -TEST_CASE_FIXTURE(Fixture, "cyclic_table_is_marked_normal") -{ - ScopedFastFlag flags[] = {{"LuauLowerBoundsCalculation", true}, {"LuauNormalizeFlagIsConservative", false}}; - - check(R"( - type Fiber = { - return_: Fiber? - } - - local f: Fiber - )"); - - TypeId t = requireType("f"); - CHECK(t->normal); -} - // Unfortunately, getting this right in the general case is difficult. TEST_CASE_FIXTURE(Fixture, "cyclic_table_is_not_marked_normal") { - ScopedFastFlag flags[] = {{"LuauLowerBoundsCalculation", true}, {"LuauNormalizeFlagIsConservative", true}}; + ScopedFastFlag flags[] = { + {"LuauLowerBoundsCalculation", true}, + }; check(R"( type Fiber = { @@ -1081,7 +1067,6 @@ TEST_CASE_FIXTURE(Fixture, "bound_typevars_should_only_be_marked_normal_if_their { ScopedFastFlag sff[]{ {"LuauLowerBoundsCalculation", true}, - {"LuauNormalizeFlagIsConservative", true}, }; CheckResult result = check(R"( diff --git a/tests/Parser.test.cpp b/tests/Parser.test.cpp index c5c019aa..5b86807c 100644 --- a/tests/Parser.test.cpp +++ b/tests/Parser.test.cpp @@ -682,20 +682,23 @@ TEST_CASE_FIXTURE(Fixture, "parse_numbers_binary") TEST_CASE_FIXTURE(Fixture, "parse_numbers_error") { - ScopedFastFlag luauErrorParseIntegerIssues{"LuauErrorParseIntegerIssues", true}; + ScopedFastFlag luauLintParseIntegerIssues{"LuauLintParseIntegerIssues", true}; + ScopedFastFlag luauErrorDoubleHexPrefix{"LuauErrorDoubleHexPrefix", true}; CHECK_EQ(getParseError("return 0b123"), "Malformed number"); CHECK_EQ(getParseError("return 123x"), "Malformed number"); CHECK_EQ(getParseError("return 0xg"), "Malformed number"); CHECK_EQ(getParseError("return 0x0x123"), "Malformed number"); + CHECK_EQ(getParseError("return 0xffffffffffffffffffffllllllg"), "Malformed number"); + CHECK_EQ(getParseError("return 0x0xffffffffffffffffffffffffffff"), "Malformed number"); } -TEST_CASE_FIXTURE(Fixture, "parse_numbers_range_error") +TEST_CASE_FIXTURE(Fixture, "parse_numbers_error_soft") { - ScopedFastFlag luauErrorParseIntegerIssues{"LuauErrorParseIntegerIssues", true}; + ScopedFastFlag luauLintParseIntegerIssues{"LuauLintParseIntegerIssues", true}; + ScopedFastFlag luauErrorDoubleHexPrefix{"LuauErrorDoubleHexPrefix", false}; - CHECK_EQ(getParseError("return 0x10000000000000000"), "Integer number value is out of range"); - CHECK_EQ(getParseError("return 0b10000000000000000000000000000000000000000000000000000000000000000"), "Integer number value is out of range"); + CHECK_EQ(getParseError("return 0x0x0x0x0x0x0x0"), "Malformed number"); } TEST_CASE_FIXTURE(Fixture, "break_return_not_last_error") diff --git a/tests/Repl.test.cpp b/tests/Repl.test.cpp index 87a1e1e2..18c243b0 100644 --- a/tests/Repl.test.cpp +++ b/tests/Repl.test.cpp @@ -82,7 +82,7 @@ private: capturedoutput = "" function arraytostring(arr) - local strings = {} + local strings = {} table.foreachi(arr, function(k,v) table.insert(strings, pptostring(v)) end ) return "{" .. table.concat(strings, ", ") .. "}" end diff --git a/tests/ToString.test.cpp b/tests/ToString.test.cpp index 8c03fe55..fe376d86 100644 --- a/tests/ToString.test.cpp +++ b/tests/ToString.test.cpp @@ -305,7 +305,6 @@ TEST_CASE_FIXTURE(Fixture, "stringifying_type_is_still_capped_when_exhaustive") CHECK_EQ(toString(requireType("f2"), o), "((() -> ()) -> () -> ()) -> (() -> ()) -> ... "); CHECK_EQ(toString(requireType("f3"), o), "(((() -> ()) -> () -> ()) -> (() -> ()) -> ... "); } - } TEST_CASE_FIXTURE(Fixture, "stringifying_table_type_correctly_use_matching_table_state_braces") @@ -726,10 +725,6 @@ TEST_CASE_FIXTURE(Fixture, "toStringNamedFunction_overrides_param_names") TEST_CASE_FIXTURE(Fixture, "pick_distinct_names_for_mixed_explicit_and_implicit_generics") { - ScopedFastFlag sff[] = { - {"LuauAlwaysQuantify", true}, - }; - CheckResult result = check(R"( function foo(x: a, y) end )"); diff --git a/tests/TypeInfer.aliases.test.cpp b/tests/TypeInfer.aliases.test.cpp index bdd4d6fd..e487fd48 100644 --- a/tests/TypeInfer.aliases.test.cpp +++ b/tests/TypeInfer.aliases.test.cpp @@ -94,6 +94,65 @@ TEST_CASE_FIXTURE(Fixture, "cannot_steal_hoisted_type_alias") } } +TEST_CASE_FIXTURE(Fixture, "mismatched_generic_type_param") +{ + CheckResult result = check(R"( + type T = (A...) -> () + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK(toString(result.errors[0]) == + "Generic type 'A' is used as a variadic type parameter; consider changing 'A' to 'A...' in the generic argument list"); + CHECK(result.errors[0].location == Location{{1, 21}, {1, 25}}); +} + +TEST_CASE_FIXTURE(Fixture, "mismatched_generic_pack_type_param") +{ + CheckResult result = check(R"( + type T = (A) -> () + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK(toString(result.errors[0]) == + "Variadic type parameter 'A...' is used as a regular generic type; consider changing 'A...' to 'A' in the generic argument list"); + CHECK(result.errors[0].location == Location{{1, 24}, {1, 25}}); +} + +TEST_CASE_FIXTURE(Fixture, "default_type_parameter") +{ + CheckResult result = check(R"( + type T = { a: A, b: B } + local x: T = { a = "foo", b = "bar" } + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + CHECK(toString(requireType("x")) == "T"); +} + +TEST_CASE_FIXTURE(Fixture, "default_pack_parameter") +{ + CheckResult result = check(R"( + type T = { fn: (A...) -> () } + local x: T + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + CHECK(toString(requireType("x")) == "T"); +} + +TEST_CASE_FIXTURE(Fixture, "saturate_to_first_type_pack") +{ + CheckResult result = check(R"( + type T = { fn: (A, B) -> C... } + local x: T + local f = x.fn + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + CHECK(toString(requireType("x")) == "T"); + CHECK(toString(requireType("f")) == "(string, number) -> (string, boolean)"); +} + TEST_CASE_FIXTURE(Fixture, "cyclic_types_of_named_table_fields_do_not_expand_when_stringified") { CheckResult result = check(R"( @@ -126,6 +185,40 @@ TEST_CASE_FIXTURE(Fixture, "mutually_recursive_aliases") LUAU_REQUIRE_NO_ERRORS(result); } +TEST_CASE_FIXTURE(Fixture, "generic_aliases") +{ + ScopedFastFlag sff_DebugLuauDeferredConstraintResolution{"DebugLuauDeferredConstraintResolution", true}; + + CheckResult result = check(R"( + type T = { v: a } + local x: T = { v = 123 } + local y: T = { v = "foo" } + local bad: T = { v = "foo" } + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + CHECK(result.errors[0].location == Location{{4, 31}, {4, 44}}); + CHECK(toString(result.errors[0]) == "Type '{ v: string }' could not be converted into 'T'"); +} + +TEST_CASE_FIXTURE(Fixture, "dependent_generic_aliases") +{ + ScopedFastFlag sff_DebugLuauDeferredConstraintResolution{"DebugLuauDeferredConstraintResolution", true}; + + CheckResult result = check(R"( + type T = { v: a } + type U = { t: T } + local x: U = { t = { v = 123 } } + local bad: U = { t = { v = "foo" } } + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + CHECK(result.errors[0].location == Location{{4, 31}, {4, 52}}); + CHECK(toString(result.errors[0]) == "Type '{ t: { v: string } }' could not be converted into 'U'"); +} + TEST_CASE_FIXTURE(Fixture, "mutually_recursive_generic_aliases") { CheckResult result = check(R"( @@ -360,6 +453,7 @@ TEST_CASE_FIXTURE(Fixture, "stringify_optional_parameterized_alias") LUAU_REQUIRE_ERROR_COUNT(1, result); auto e = get(result.errors[0]); + REQUIRE(e != nullptr); CHECK_EQ("Node?", toString(e->givenType)); CHECK_EQ("Node", toString(e->wantedType)); } diff --git a/tests/TypeInfer.functions.test.cpp b/tests/TypeInfer.functions.test.cpp index 70773d95..074c86c3 100644 --- a/tests/TypeInfer.functions.test.cpp +++ b/tests/TypeInfer.functions.test.cpp @@ -1547,7 +1547,6 @@ caused by: Argument count mismatch. Function expects 1 argument, but none are specified)", toString(result.errors[0])); } - } TEST_CASE_FIXTURE(Fixture, "too_few_arguments_variadic") diff --git a/tests/TypeInfer.generics.test.cpp b/tests/TypeInfer.generics.test.cpp index 7a1af164..a8325727 100644 --- a/tests/TypeInfer.generics.test.cpp +++ b/tests/TypeInfer.generics.test.cpp @@ -1186,10 +1186,6 @@ end) TEST_CASE_FIXTURE(Fixture, "quantify_functions_even_if_they_have_an_explicit_generic") { - ScopedFastFlag sff[] = { - {"LuauAlwaysQuantify", true}, - }; - CheckResult result = check(R"( function foo(f, x: X) return f(x) diff --git a/tests/TypeInfer.provisional.test.cpp b/tests/TypeInfer.provisional.test.cpp index 34afd560..01923f38 100644 --- a/tests/TypeInfer.provisional.test.cpp +++ b/tests/TypeInfer.provisional.test.cpp @@ -485,7 +485,6 @@ TEST_CASE_FIXTURE(Fixture, "constrained_is_level_dependent") { ScopedFastFlag sff[]{ {"LuauLowerBoundsCalculation", true}, - {"LuauNormalizeFlagIsConservative", true}, {"LuauQuantifyConstrained", true}, }; diff --git a/tests/TypeInfer.singletons.test.cpp b/tests/TypeInfer.singletons.test.cpp index 5da4a340..0a130d49 100644 --- a/tests/TypeInfer.singletons.test.cpp +++ b/tests/TypeInfer.singletons.test.cpp @@ -262,7 +262,7 @@ TEST_CASE_FIXTURE(Fixture, "table_properties_type_error_escapes") { CheckResult result = check(R"( --!strict - local x: { ["<>"] : number } + local x: { ["<>"] : number } x = { ["\n"] = 5 } )"); diff --git a/tests/TypeInfer.test.cpp b/tests/TypeInfer.test.cpp index af6185ef..80889363 100644 --- a/tests/TypeInfer.test.cpp +++ b/tests/TypeInfer.test.cpp @@ -252,7 +252,6 @@ TEST_CASE_FIXTURE(Fixture, "type_errors_infer_types") CHECK_EQ("", toString(requireType("e"))); CHECK_EQ("", toString(requireType("f"))); } - } } diff --git a/tests/TypeInfer.typePacks.cpp b/tests/TypeInfer.typePacks.cpp index bcd30498..7aefa00d 100644 --- a/tests/TypeInfer.typePacks.cpp +++ b/tests/TypeInfer.typePacks.cpp @@ -203,7 +203,7 @@ TEST_CASE_FIXTURE(Fixture, "variadic_packs") ), "@test" ); - addGlobalBinding(typeChecker, "bar", + addGlobalBinding(typeChecker, "bar", arena.addType( FunctionTypeVar{ arena.addTypePack({{typeChecker.numberType}, listOfStrings}), diff --git a/tests/conformance/errors.lua b/tests/conformance/errors.lua index 6c0ac5e6..b13e7a82 100644 --- a/tests/conformance/errors.lua +++ b/tests/conformance/errors.lua @@ -295,8 +295,9 @@ end -- testing syntax limits +local syntaxdepth = if limitedstack then 200 else 1000 local function testrep (init, rep) - local s = "local a; "..init .. string.rep(rep, 300) + local s = "local a; "..init .. string.rep(rep, syntaxdepth) local a,b = loadstring(s) assert(not a) -- and string.find(b, "syntax levels")) end @@ -388,4 +389,11 @@ assert(ecall(function() for i='a',2 do end end) == "invalid 'for' initial value assert(ecall(function() for i=1,'a' do end end) == "invalid 'for' limit (number expected, got string)") assert(ecall(function() for i=1,2,'a' do end end) == "invalid 'for' step (number expected, got string)") +-- method call errors +assert(ecall(function() ({}):foo() end) == "attempt to call missing method 'foo' of table") +assert(ecall(function() (""):foo() end) == "attempt to call missing method 'foo' of string") +assert(ecall(function() (42):foo() end) == "attempt to index number with 'foo'") +assert(ecall(function() ({foo=42}):foo() end) == "attempt to call a number value") +assert(ecall(function() local ud = newproxy(true) getmetatable(ud).__index = {} ud:foo() end) == "attempt to call missing method 'foo' of userdata") + return('OK') diff --git a/tests/main.cpp b/tests/main.cpp index 40ccd0b3..3e480c9f 100644 --- a/tests/main.cpp +++ b/tests/main.cpp @@ -61,7 +61,7 @@ static bool debuggerPresent() static int testAssertionHandler(const char* expr, const char* file, int line, const char* function) { if (debuggerPresent()) - LUAU_DEBUGBREAK(); + LUAU_DEBUGBREAK(); ADD_FAIL_AT(file, line, "Assertion failed: ", std::string(expr)); return 1; diff --git a/tools/faillist.txt b/tools/faillist.txt index 40fc3c06..6e93345b 100644 --- a/tools/faillist.txt +++ b/tools/faillist.txt @@ -4,14 +4,11 @@ AnnotationTests.as_expr_warns_on_unrelated_cast AnnotationTests.builtin_types_are_not_exported AnnotationTests.cannot_use_nonexported_type AnnotationTests.cloned_interface_maintains_pointers_between_definitions -AnnotationTests.corecursive_types_error_on_tight_loop AnnotationTests.define_generic_type_alias AnnotationTests.duplicate_type_param_name AnnotationTests.for_loop_counter_annotation_is_checked AnnotationTests.function_return_annotations_are_checked AnnotationTests.generic_aliases_are_cloned_properly -AnnotationTests.instantiate_type_fun_should_not_trip_rbxassert -AnnotationTests.instantiation_clone_has_to_follow AnnotationTests.interface_types_belong_to_interface_arena AnnotationTests.luau_ice_triggers_an_ice AnnotationTests.luau_ice_triggers_an_ice_exception_with_flag @@ -24,14 +21,9 @@ AnnotationTests.occurs_check_on_cyclic_union_typevar AnnotationTests.self_referential_type_alias AnnotationTests.too_many_type_params AnnotationTests.two_type_params -AnnotationTests.type_alias_always_resolve_to_a_real_type -AnnotationTests.type_alias_B_should_check_with_another_aliases_until_a_non_aliased_type -AnnotationTests.type_alias_should_alias_to_number -AnnotationTests.type_aliasing_to_number_should_not_check_given_a_string AnnotationTests.type_annotations_inside_function_bodies AnnotationTests.type_assertion_expr -AnnotationTests.typeof_variable_type_annotation_should_return_its_type -AnnotationTests.use_generic_type_alias +AnnotationTests.unknown_type_reference_generates_error AnnotationTests.use_type_required_from_another_file AstQuery.last_argument_function_call_type AstQuery::getDocumentationSymbolAtPosition.binding @@ -42,11 +34,8 @@ AutocompleteTest.argument_types AutocompleteTest.arguments_to_global_lambda AutocompleteTest.as_types AutocompleteTest.autocomplete_boolean_singleton -AutocompleteTest.autocomplete_default_type_pack_parameters -AutocompleteTest.autocomplete_default_type_parameters AutocompleteTest.autocomplete_end_with_fn_exprs AutocompleteTest.autocomplete_end_with_lambda -AutocompleteTest.autocomplete_explicit_type_pack AutocompleteTest.autocomplete_first_function_arg_expected_type AutocompleteTest.autocomplete_for_in_middle_keywords AutocompleteTest.autocomplete_for_middle_keywords @@ -93,7 +82,6 @@ AutocompleteTest.local_function_params AutocompleteTest.local_functions_fall_out_of_scope AutocompleteTest.method_call_inside_function_body AutocompleteTest.module_type_members -AutocompleteTest.modules_with_types AutocompleteTest.nested_member_completions AutocompleteTest.nested_recursive_function AutocompleteTest.no_function_name_suggestions @@ -101,8 +89,6 @@ AutocompleteTest.no_incompatible_self_calls AutocompleteTest.no_incompatible_self_calls_2 AutocompleteTest.no_incompatible_self_calls_on_class AutocompleteTest.no_wrong_compatible_self_calls_with_generics -AutocompleteTest.optional_members -AutocompleteTest.private_types AutocompleteTest.recursive_function AutocompleteTest.recursive_function_global AutocompleteTest.recursive_function_local @@ -114,7 +100,6 @@ AutocompleteTest.stop_at_first_stat_when_recommending_keywords AutocompleteTest.string_prim_non_self_calls_are_avoided AutocompleteTest.string_prim_self_calls_are_fine AutocompleteTest.suggest_external_module_type -AutocompleteTest.suggest_table_keys AutocompleteTest.table_intersection AutocompleteTest.table_union AutocompleteTest.type_correct_argument_type_suggestion @@ -133,8 +118,6 @@ AutocompleteTest.type_correct_local_type_suggestion AutocompleteTest.type_correct_sealed_table AutocompleteTest.type_correct_suggestion_for_overloads AutocompleteTest.type_correct_suggestion_in_argument -AutocompleteTest.type_correct_suggestion_in_table -AutocompleteTest.type_scoping_easy AutocompleteTest.unsealed_table AutocompleteTest.unsealed_table_2 AutocompleteTest.user_defined_local_functions_in_own_definition @@ -259,7 +242,6 @@ GenericsTests.check_mutual_generic_functions GenericsTests.correctly_instantiate_polymorphic_member_functions GenericsTests.do_not_always_instantiate_generic_intersection_types GenericsTests.do_not_infer_generic_functions -GenericsTests.dont_substitute_bound_types GenericsTests.dont_unify_bound_types GenericsTests.duplicate_generic_type_packs GenericsTests.duplicate_generic_types @@ -275,6 +257,7 @@ GenericsTests.generic_functions_dont_cache_type_parameters GenericsTests.generic_functions_in_types GenericsTests.generic_functions_should_be_memory_safe GenericsTests.generic_table_method +GenericsTests.generic_type_pack_parentheses GenericsTests.generic_type_pack_syntax GenericsTests.generic_type_pack_unification1 GenericsTests.generic_type_pack_unification2 @@ -297,15 +280,12 @@ GenericsTests.properties_can_be_polytypes GenericsTests.rank_N_types_via_typeof GenericsTests.reject_clashing_generic_and_pack_names GenericsTests.self_recursive_instantiated_param -GenericsTests.substitution_with_bound_table -GenericsTests.typefuns_sharing_types GenericsTests.variadic_generics IntersectionTypes.argument_is_intersection IntersectionTypes.error_detailed_intersection_all IntersectionTypes.error_detailed_intersection_part IntersectionTypes.fx_intersection_as_argument IntersectionTypes.fx_union_as_argument_fails -IntersectionTypes.index_on_an_intersection_type_with_all_parts_missing_the_property IntersectionTypes.index_on_an_intersection_type_with_mixed_types IntersectionTypes.index_on_an_intersection_type_with_one_part_missing_the_property IntersectionTypes.index_on_an_intersection_type_with_one_property_of_type_any @@ -313,12 +293,8 @@ IntersectionTypes.index_on_an_intersection_type_with_property_guaranteed_to_exis IntersectionTypes.index_on_an_intersection_type_works_at_arbitrary_depth IntersectionTypes.no_stack_overflow_from_flattenintersection IntersectionTypes.overload_is_not_a_function -IntersectionTypes.propagates_name IntersectionTypes.select_correct_union_fn IntersectionTypes.should_still_pick_an_overload_whose_arguments_are_unions -IntersectionTypes.table_combines -IntersectionTypes.table_combines_missing -IntersectionTypes.table_extra_ok IntersectionTypes.table_intersection_setmetatable IntersectionTypes.table_intersection_write IntersectionTypes.table_intersection_write_sealed @@ -332,7 +308,6 @@ isSubtype.table_with_table_prop isSubtype.tables Linter.DeprecatedApi Linter.TableOperations -ModuleTests.any_persistance_does_not_leak ModuleTests.builtin_types_point_into_globalTypes_arena ModuleTests.clone_self_property ModuleTests.deepClone_cyclic_table @@ -355,8 +330,6 @@ NonstrictModeTests.table_props_are_any Normalize.any_wins_the_battle_over_unknown_in_unions Normalize.constrained_intersection_of_intersections Normalize.cyclic_intersection -Normalize.cyclic_table_is_marked_normal -Normalize.cyclic_table_is_not_marked_normal Normalize.cyclic_table_normalizes_sensibly Normalize.cyclic_union Normalize.fuzz_failure_bound_type_is_normal_but_not_its_bounded_to @@ -490,16 +463,9 @@ TableTests.cannot_change_type_of_unsealed_table_prop TableTests.casting_sealed_tables_with_props_into_table_with_indexer TableTests.casting_tables_with_props_into_table_with_indexer3 TableTests.casting_tables_with_props_into_table_with_indexer4 -TableTests.casting_unsealed_tables_with_props_into_table_with_indexer TableTests.checked_prop_too_early -TableTests.common_table_element_general -TableTests.common_table_element_inner_index -TableTests.common_table_element_inner_prop -TableTests.common_table_element_list -TableTests.common_table_element_union_assignment TableTests.common_table_element_union_in_call TableTests.common_table_element_union_in_call_tail -TableTests.common_table_element_union_in_prop TableTests.confusing_indexing TableTests.defining_a_method_for_a_builtin_sealed_table_must_fail TableTests.defining_a_method_for_a_local_sealed_table_must_fail @@ -583,7 +549,6 @@ TableTests.right_table_missing_key TableTests.right_table_missing_key2 TableTests.scalar_is_a_subtype_of_a_compatible_polymorphic_shape_type TableTests.scalar_is_not_a_subtype_of_a_compatible_polymorphic_shape_type -TableTests.setmetatable_cant_be_used_to_mutate_global_types TableTests.shared_selfs TableTests.shared_selfs_from_free_param TableTests.shared_selfs_through_metatables @@ -611,7 +576,6 @@ TableTests.used_colon_correctly TableTests.used_colon_instead_of_dot TableTests.used_dot_instead_of_colon TableTests.used_dot_instead_of_colon_but_correctly -TableTests.user_defined_table_types_are_named TableTests.width_subtyping ToDot.bound_table ToDot.class @@ -620,7 +584,6 @@ ToDot.metatable ToDot.primitive ToDot.table ToString.exhaustive_toString_of_cyclic_table -ToString.function_type_with_argument_names ToString.function_type_with_argument_names_and_self ToString.function_type_with_argument_names_generic ToString.named_metatable_toStringNamedFunction @@ -640,48 +603,27 @@ TryUnifyTests.members_of_failed_typepack_unification_are_unified_with_errorType TryUnifyTests.result_of_failed_typepack_unification_is_constrained TryUnifyTests.typepack_unification_should_trim_free_tails TryUnifyTests.variadics_should_use_reversed_properly -TypeAliases.basic_alias -TypeAliases.cannot_steal_hoisted_type_alias TypeAliases.cli_38393_recursive_intersection_oom -TypeAliases.corecursive_function_types TypeAliases.corecursive_types_generic -TypeAliases.cyclic_function_type_in_type_alias -TypeAliases.cyclic_types_of_named_table_fields_do_not_expand_when_stringified TypeAliases.do_not_quantify_unresolved_aliases -TypeAliases.dont_stop_typechecking_after_reporting_duplicate_type_definition -TypeAliases.export_type_and_type_alias_are_duplicates TypeAliases.forward_declared_alias_is_not_clobbered_by_prior_unification_with_any -TypeAliases.forward_declared_alias_is_not_clobbered_by_prior_unification_with_any_2 -TypeAliases.free_variables_from_typeof_in_aliases TypeAliases.general_require_multi_assign TypeAliases.generic_param_remap -TypeAliases.generic_typevars_are_not_considered_to_escape_their_scope_if_they_are_reused_in_multiple_aliases -TypeAliases.module_export_free_type_leak -TypeAliases.module_export_wrapped_free_type_leak -TypeAliases.mutually_recursive_aliases +TypeAliases.mismatched_generic_pack_type_param +TypeAliases.mismatched_generic_type_param TypeAliases.mutually_recursive_generic_aliases -TypeAliases.mutually_recursive_types_errors TypeAliases.mutually_recursive_types_restriction_not_ok_1 TypeAliases.mutually_recursive_types_restriction_not_ok_2 -TypeAliases.mutually_recursive_types_restriction_ok TypeAliases.mutually_recursive_types_swapsies_not_ok -TypeAliases.mutually_recursive_types_swapsies_ok -TypeAliases.names_are_ascribed -TypeAliases.non_recursive_aliases_that_reuse_a_generic_name TypeAliases.recursive_types_restriction_not_ok -TypeAliases.recursive_types_restriction_ok -TypeAliases.reported_location_is_correct_when_type_alias_are_duplicates TypeAliases.stringify_optional_parameterized_alias TypeAliases.stringify_type_alias_of_recursive_template_table_type TypeAliases.stringify_type_alias_of_recursive_template_table_type2 -TypeAliases.type_alias_fwd_declaration_is_precise TypeAliases.type_alias_import_mutation TypeAliases.type_alias_local_mutation TypeAliases.type_alias_local_rename -TypeAliases.type_alias_local_synthetic_mutation TypeAliases.type_alias_of_an_imported_recursive_generic_type TypeAliases.type_alias_of_an_imported_recursive_type -TypeAliases.use_table_name_and_generic_params_in_errors TypeInfer.check_expr_recursion_limit TypeInfer.checking_should_not_ice TypeInfer.cli_50041_committing_txnlog_in_apollo_client_error @@ -698,9 +640,7 @@ TypeInfer.infer_assignment_value_types_mutable_lval TypeInfer.infer_through_group_expr TypeInfer.infer_type_assertion_value_type TypeInfer.no_heap_use_after_free_error -TypeInfer.no_infinite_loop_when_trying_to_unify_uh_this TypeInfer.no_stack_overflow_from_isoptional -TypeInfer.no_stack_overflow_from_isoptional2 TypeInfer.recursive_metatable_crash TypeInfer.tc_after_error_recovery_no_replacement_name_in_error TypeInfer.tc_if_else_expressions1 @@ -946,6 +886,8 @@ TypeInferUnknownNever.length_of_never TypeInferUnknownNever.math_operators_and_never TypeInferUnknownNever.never_is_reflexive TypeInferUnknownNever.never_subtype_and_string_supertype +TypeInferUnknownNever.pick_never_from_variadic_type_pack +TypeInferUnknownNever.string_subtype_and_never_supertype TypeInferUnknownNever.string_subtype_and_unknown_supertype TypeInferUnknownNever.table_with_prop_of_type_never_is_also_reflexive TypeInferUnknownNever.table_with_prop_of_type_never_is_uninhabitable @@ -953,6 +895,7 @@ TypeInferUnknownNever.type_packs_containing_never_is_itself_uninhabitable TypeInferUnknownNever.type_packs_containing_never_is_itself_uninhabitable2 TypeInferUnknownNever.unary_minus_of_never TypeInferUnknownNever.unknown_is_reflexive +TypeInferUnknownNever.unknown_subtype_and_string_supertype TypePackTests.cyclic_type_packs TypePackTests.higher_order_function TypePackTests.multiple_varargs_inference_are_not_confused @@ -965,18 +908,13 @@ TypePackTests.type_alias_default_export TypePackTests.type_alias_default_mixed_self TypePackTests.type_alias_default_type_chained TypePackTests.type_alias_default_type_errors -TypePackTests.type_alias_default_type_explicit -TypePackTests.type_alias_default_type_pack_explicit TypePackTests.type_alias_default_type_pack_self_chained_tp TypePackTests.type_alias_default_type_pack_self_tp -TypePackTests.type_alias_default_type_pack_self_ty TypePackTests.type_alias_default_type_self -TypePackTests.type_alias_default_type_skip_brackets TypePackTests.type_alias_defaults_confusing_types TypePackTests.type_alias_defaults_recursive_type TypePackTests.type_alias_type_pack_explicit TypePackTests.type_alias_type_pack_explicit_multi -TypePackTests.type_alias_type_pack_explicit_multi_tostring TypePackTests.type_alias_type_pack_multi TypePackTests.type_alias_type_pack_variadic TypePackTests.type_alias_type_packs @@ -1010,19 +948,13 @@ TypeSingletons.string_singletons TypeSingletons.string_singletons_escape_chars TypeSingletons.string_singletons_mismatch TypeSingletons.table_insert_with_a_singleton_argument -TypeSingletons.table_properties_alias_or_parens_is_indexer -TypeSingletons.table_properties_singleton_strings -TypeSingletons.table_properties_singleton_strings_mismatch TypeSingletons.table_properties_type_error_escapes -TypeSingletons.tagged_unions_immutable_tag TypeSingletons.tagged_unions_using_singletons -TypeSingletons.tagged_unions_using_singletons_mismatch TypeSingletons.taking_the_length_of_string_singleton TypeSingletons.taking_the_length_of_union_of_string_singleton TypeSingletons.widen_the_supertype_if_it_is_free_and_subtype_has_singleton TypeSingletons.widening_happens_almost_everywhere TypeSingletons.widening_happens_almost_everywhere_except_for_tables -TypeVarTests.visit_once UnionTypes.error_detailed_optional UnionTypes.error_detailed_union_all UnionTypes.error_detailed_union_part @@ -1046,6 +978,5 @@ UnionTypes.optional_union_members UnionTypes.optional_union_methods UnionTypes.return_types_can_be_disjoint UnionTypes.table_union_write_indirect -UnionTypes.unify_sealed_table_union_check UnionTypes.unify_unsealed_table_union_check UnionTypes.union_equality_comparisons diff --git a/tools/perfgraph.py b/tools/perfgraph.py index 95baef9c..ae74b7d6 100644 --- a/tools/perfgraph.py +++ b/tools/perfgraph.py @@ -6,6 +6,12 @@ import sys import svg +import argparse +import json + +argumentParser = argparse.ArgumentParser(description='Generate flamegraph SVG from Luau sampling profiler dumps') +argumentParser.add_argument('source_file', type=open) +argumentParser.add_argument('--json', dest='useJson',action='store_const',const=1,default=0,help='Parse source_file as JSON') class Node(svg.Node): def __init__(self): @@ -27,26 +33,64 @@ class Node(svg.Node): def details(self, root): return "Function: {} [{}:{}] ({:,} usec, {:.1%}); self: {:,} usec".format(self.function, self.source, self.line, self.width, self.width / root.width, self.ticks) -with open(sys.argv[1]) as f: - dump = f.readlines() -root = Node() +def nodeFromCallstackListFile(source_file): + dump = source_file.readlines() + root = Node() -for l in dump: - ticks, stack = l.strip().split(" ", 1) - node = root + for l in dump: + ticks, stack = l.strip().split(" ", 1) + node = root - for f in reversed(stack.split(";")): - source, function, line = f.split(",") + for f in reversed(stack.split(";")): + source, function, line = f.split(",") - child = node.child(f) - child.function = function - child.source = source - child.line = int(line) if len(line) > 0 else 0 + child = node.child(f) + child.function = function + child.source = source + child.line = int(line) if len(line) > 0 else 0 + + node = child + + node.ticks += int(ticks) + + return root + + +def nodeFromJSONbject(node, key, obj): + source, function, line = key.split(",") + + node.function = function + node.source = source + node.line = int(line) if len(line) > 0 else 0 + + node.ticks = obj['Duration'] + + for key, obj in obj['Children'].items(): + nodeFromJSONbject(node.child(key), key, obj) + + return node + + +def nodeFromJSONFile(source_file): + dump = json.load(source_file) + + root = Node() + + for key, obj in dump['Children'].items(): + nodeFromJSONbject(root.child(key), key, obj) + + return root + + +arguments = argumentParser.parse_args() + +if arguments.useJson: + root = nodeFromJSONFile(arguments.source_file) +else: + root = nodeFromCallstackListFile(arguments.source_file) - node = child - node.ticks += int(ticks) svg.layout(root, lambda n: n.ticks) svg.display(root, "Flame Graph", "hot", flip = True) diff --git a/tools/test_dcr.py b/tools/test_dcr.py index 94ff5ca2..0efea3c4 100644 --- a/tools/test_dcr.py +++ b/tools/test_dcr.py @@ -37,17 +37,15 @@ class Handler(x.ContentHandler): elif name == "OverallResultsAsserts": if self.currentTest: - failed = 0 != safeParseInt(attrs["failures"]) + passed = 0 == safeParseInt(attrs["failures"]) dottedName = ".".join(self.currentTest) - shouldFail = dottedName in self.failList - if failed and not shouldFail: - print("UNEXPECTED: {} should have passed".format(dottedName)) - elif not failed and shouldFail: - print("UNEXPECTED: {} should have failed".format(dottedName)) - - self.results[dottedName] = not failed + # Sometimes we get multiple XML trees for the same test. All of + # them must report a pass in order for us to consider the test + # to have passed. + r = self.results.get(dottedName, True) + self.results[dottedName] = r and passed elif name == 'OverallResultsTestCases': self.numSkippedTests = safeParseInt(attrs.get("skipped", 0)) @@ -104,6 +102,12 @@ def main(): p.wait() + for testName, passed in handler.results.items(): + if passed and testName in failList: + print('UNEXPECTED: {} should have failed'.format(testName)) + elif not passed and testName not in failList: + print('UNEXPECTED: {} should have passed'.format(testName)) + if args.write: newFailList = sorted( ( From e15b0728be64b380c0d6f04e305c2678493be268 Mon Sep 17 00:00:00 2001 From: JohnnyMorganz Date: Wed, 10 Aug 2022 21:04:08 +0100 Subject: [PATCH 334/399] Add autocomplete context to result (#611) Closes #599 Not sure if these context definitions are distinct enough, but they work for my use cases. I repurposed (a majority of) the existing unit tests for this, but if it should live in separate test cases let me know --- * Add autocomplete context to result * Update tests * Remove comments * Fix expression context issues --- Analysis/include/Luau/Autocomplete.h | 15 +++- Analysis/src/Autocomplete.cpp | 76 ++++++++++---------- tests/Autocomplete.test.cpp | 101 +++++++++++++++++++++++++++ 3 files changed, 154 insertions(+), 38 deletions(-) diff --git a/Analysis/include/Luau/Autocomplete.h b/Analysis/include/Luau/Autocomplete.h index 5e8d6605..f40f8b49 100644 --- a/Analysis/include/Luau/Autocomplete.h +++ b/Analysis/include/Luau/Autocomplete.h @@ -19,6 +19,17 @@ struct TypeChecker; using ModulePtr = std::shared_ptr; +enum class AutocompleteContext +{ + Unknown, + Expression, + Statement, + Property, + Type, + Keyword, + String, +}; + enum class AutocompleteEntryKind { Property, @@ -66,11 +77,13 @@ struct AutocompleteResult { AutocompleteEntryMap entryMap; std::vector ancestry; + AutocompleteContext context = AutocompleteContext::Unknown; AutocompleteResult() = default; - AutocompleteResult(AutocompleteEntryMap entryMap, std::vector ancestry) + AutocompleteResult(AutocompleteEntryMap entryMap, std::vector ancestry, AutocompleteContext context) : entryMap(std::move(entryMap)) , ancestry(std::move(ancestry)) + , context(context) { } }; diff --git a/Analysis/src/Autocomplete.cpp b/Analysis/src/Autocomplete.cpp index a57a789f..8d5cc723 100644 --- a/Analysis/src/Autocomplete.cpp +++ b/Analysis/src/Autocomplete.cpp @@ -1200,7 +1200,7 @@ static bool autocompleteIfElseExpression( } } -static void autocompleteExpression(const SourceModule& sourceModule, const Module& module, const TypeChecker& typeChecker, TypeArena* typeArena, +static AutocompleteContext autocompleteExpression(const SourceModule& sourceModule, const Module& module, const TypeChecker& typeChecker, TypeArena* typeArena, const std::vector& ancestry, Position position, AutocompleteEntryMap& result) { LUAU_ASSERT(!ancestry.empty()); @@ -1213,9 +1213,9 @@ static void autocompleteExpression(const SourceModule& sourceModule, const Modul autocompleteProps(module, typeArena, *it, PropIndexType::Point, ancestry, result); } else if (autocompleteIfElseExpression(node, ancestry, position, result)) - return; + return AutocompleteContext::Keyword; else if (node->is()) - return; + return AutocompleteContext::Unknown; else { // This is inefficient. :( @@ -1260,14 +1260,16 @@ static void autocompleteExpression(const SourceModule& sourceModule, const Modul if (auto ty = findExpectedTypeAt(module, node, position)) autocompleteStringSingleton(*ty, true, result); } + + return AutocompleteContext::Expression; } -static AutocompleteEntryMap autocompleteExpression(const SourceModule& sourceModule, const Module& module, const TypeChecker& typeChecker, +static AutocompleteResult autocompleteExpression(const SourceModule& sourceModule, const Module& module, const TypeChecker& typeChecker, TypeArena* typeArena, const std::vector& ancestry, Position position) { AutocompleteEntryMap result; - autocompleteExpression(sourceModule, module, typeChecker, typeArena, ancestry, position, result); - return result; + AutocompleteContext context = autocompleteExpression(sourceModule, module, typeChecker, typeArena, ancestry, position, result); + return {result, ancestry, context}; } static std::optional getMethodContainingClass(const ModulePtr& module, AstExpr* funcExpr) @@ -1406,27 +1408,27 @@ static AutocompleteResult autocomplete(const SourceModule& sourceModule, const M if (!FFlag::LuauSelfCallAutocompleteFix3 && isString(ty)) return { - autocompleteProps(*module, typeArena, typeChecker.globalScope->bindings[AstName{"string"}].typeId, indexType, ancestry), ancestry}; + autocompleteProps(*module, typeArena, typeChecker.globalScope->bindings[AstName{"string"}].typeId, indexType, ancestry), ancestry, AutocompleteContext::Property}; else - return {autocompleteProps(*module, typeArena, ty, indexType, ancestry), ancestry}; + return {autocompleteProps(*module, typeArena, ty, indexType, ancestry), ancestry, AutocompleteContext::Property}; } else if (auto typeReference = node->as()) { if (typeReference->prefix) - return {autocompleteModuleTypes(*module, position, typeReference->prefix->value), ancestry}; + return {autocompleteModuleTypes(*module, position, typeReference->prefix->value), ancestry, AutocompleteContext::Type}; else - return {autocompleteTypeNames(*module, position, ancestry), ancestry}; + return {autocompleteTypeNames(*module, position, ancestry), ancestry, AutocompleteContext::Type}; } else if (node->is()) { - return {autocompleteTypeNames(*module, position, ancestry), ancestry}; + return {autocompleteTypeNames(*module, position, ancestry), ancestry, AutocompleteContext::Type}; } else if (AstStatLocal* statLocal = node->as()) { if (statLocal->vars.size == 1 && (!statLocal->equalsSignLocation || position < statLocal->equalsSignLocation->begin)) - return {{{"function", AutocompleteEntry{AutocompleteEntryKind::Keyword}}}, ancestry}; + return {{{"function", AutocompleteEntry{AutocompleteEntryKind::Keyword}}}, ancestry, AutocompleteContext::Unknown}; else if (statLocal->equalsSignLocation && position >= statLocal->equalsSignLocation->end) - return {autocompleteExpression(sourceModule, *module, typeChecker, typeArena, ancestry, position), ancestry}; + return autocompleteExpression(sourceModule, *module, typeChecker, typeArena, ancestry, position); else return {}; } @@ -1436,16 +1438,16 @@ static AutocompleteResult autocomplete(const SourceModule& sourceModule, const M if (!statFor->hasDo || position < statFor->doLocation.begin) { if (!statFor->from->is() && !statFor->to->is() && (!statFor->step || !statFor->step->is())) - return {{{"do", AutocompleteEntry{AutocompleteEntryKind::Keyword}}}, ancestry}; + return {{{"do", AutocompleteEntry{AutocompleteEntryKind::Keyword}}}, ancestry, AutocompleteContext::Keyword}; if (statFor->from->location.containsClosed(position) || statFor->to->location.containsClosed(position) || (statFor->step && statFor->step->location.containsClosed(position))) - return {autocompleteExpression(sourceModule, *module, typeChecker, typeArena, ancestry, position), ancestry}; + return autocompleteExpression(sourceModule, *module, typeChecker, typeArena, ancestry, position); return {}; } - return {autocompleteStatement(sourceModule, *module, ancestry, position), ancestry}; + return {autocompleteStatement(sourceModule, *module, ancestry, position), ancestry, AutocompleteContext::Statement}; } else if (AstStatForIn* statForIn = parent->as(); statForIn && (node->is() || isIdentifier(node))) @@ -1461,7 +1463,7 @@ static AutocompleteResult autocomplete(const SourceModule& sourceModule, const M return {}; } - return {{{"in", AutocompleteEntry{AutocompleteEntryKind::Keyword}}}, ancestry}; + return {{{"in", AutocompleteEntry{AutocompleteEntryKind::Keyword}}}, ancestry, AutocompleteContext::Keyword}; } if (!statForIn->hasDo || position <= statForIn->doLocation.begin) @@ -1470,10 +1472,10 @@ static AutocompleteResult autocomplete(const SourceModule& sourceModule, const M AstExpr* lastExpr = statForIn->values.data[statForIn->values.size - 1]; if (lastExpr->location.containsClosed(position)) - return {autocompleteExpression(sourceModule, *module, typeChecker, typeArena, ancestry, position), ancestry}; + return autocompleteExpression(sourceModule, *module, typeChecker, typeArena, ancestry, position); if (position > lastExpr->location.end) - return {{{"do", AutocompleteEntry{AutocompleteEntryKind::Keyword}}}, ancestry}; + return {{{"do", AutocompleteEntry{AutocompleteEntryKind::Keyword}}}, ancestry, AutocompleteContext::Keyword}; return {}; // Not sure what this means } @@ -1483,45 +1485,45 @@ static AutocompleteResult autocomplete(const SourceModule& sourceModule, const M // The AST looks a bit differently if the cursor is at a position where only the "do" keyword is allowed. // ex "for f in f do" if (!statForIn->hasDo) - return {{{"do", AutocompleteEntry{AutocompleteEntryKind::Keyword}}}, ancestry}; + return {{{"do", AutocompleteEntry{AutocompleteEntryKind::Keyword}}}, ancestry, AutocompleteContext::Keyword}; - return {autocompleteStatement(sourceModule, *module, ancestry, position), ancestry}; + return {autocompleteStatement(sourceModule, *module, ancestry, position), ancestry, AutocompleteContext::Statement}; } else if (AstStatWhile* statWhile = parent->as(); node->is() && statWhile) { if (!statWhile->hasDo && !statWhile->condition->is() && position > statWhile->condition->location.end) - return {{{"do", AutocompleteEntry{AutocompleteEntryKind::Keyword}}}, ancestry}; + return {{{"do", AutocompleteEntry{AutocompleteEntryKind::Keyword}}}, ancestry, AutocompleteContext::Keyword}; if (!statWhile->hasDo || position < statWhile->doLocation.begin) - return {autocompleteExpression(sourceModule, *module, typeChecker, typeArena, ancestry, position), ancestry}; + return autocompleteExpression(sourceModule, *module, typeChecker, typeArena, ancestry, position); if (statWhile->hasDo && position > statWhile->doLocation.end) - return {autocompleteStatement(sourceModule, *module, ancestry, position), ancestry}; + return {autocompleteStatement(sourceModule, *module, ancestry, position), ancestry, AutocompleteContext::Statement}; } else if (AstStatWhile* statWhile = extractStat(ancestry); statWhile && !statWhile->hasDo) - return {{{"do", AutocompleteEntry{AutocompleteEntryKind::Keyword}}}, ancestry}; + return {{{"do", AutocompleteEntry{AutocompleteEntryKind::Keyword}}}, ancestry, AutocompleteContext::Keyword}; else if (AstStatIf* statIf = node->as(); statIf && !statIf->elseLocation.has_value()) { return { - {{"else", AutocompleteEntry{AutocompleteEntryKind::Keyword}}, {"elseif", AutocompleteEntry{AutocompleteEntryKind::Keyword}}}, ancestry}; + {{"else", AutocompleteEntry{AutocompleteEntryKind::Keyword}}, {"elseif", AutocompleteEntry{AutocompleteEntryKind::Keyword}}}, ancestry, AutocompleteContext::Keyword}; } else if (AstStatIf* statIf = parent->as(); statIf && node->is()) { if (statIf->condition->is()) - return {autocompleteExpression(sourceModule, *module, typeChecker, typeArena, ancestry, position), ancestry}; + return autocompleteExpression(sourceModule, *module, typeChecker, typeArena, ancestry, position); else if (!statIf->thenLocation || statIf->thenLocation->containsClosed(position)) - return {{{"then", AutocompleteEntry{AutocompleteEntryKind::Keyword}}}, ancestry}; + return {{{"then", AutocompleteEntry{AutocompleteEntryKind::Keyword}}}, ancestry, AutocompleteContext::Keyword}; } else if (AstStatIf* statIf = extractStat(ancestry); statIf && (!statIf->thenLocation || statIf->thenLocation->containsClosed(position))) - return {{{"then", AutocompleteEntry{AutocompleteEntryKind::Keyword}}}, ancestry}; + return {{{"then", AutocompleteEntry{AutocompleteEntryKind::Keyword}}}, ancestry, AutocompleteContext::Keyword}; else if (AstStatRepeat* statRepeat = node->as(); statRepeat && statRepeat->condition->is()) - return {autocompleteExpression(sourceModule, *module, typeChecker, typeArena, ancestry, position), ancestry}; + return autocompleteExpression(sourceModule, *module, typeChecker, typeArena, ancestry, position); else if (AstStatRepeat* statRepeat = extractStat(ancestry); statRepeat) - return {autocompleteStatement(sourceModule, *module, ancestry, position), ancestry}; + return {autocompleteStatement(sourceModule, *module, ancestry, position), ancestry, AutocompleteContext::Statement}; else if (AstExprTable* exprTable = parent->as(); exprTable && (node->is() || node->is())) { for (const auto& [kind, key, value] : exprTable->items) @@ -1547,7 +1549,7 @@ static AutocompleteResult autocomplete(const SourceModule& sourceModule, const M if (!key) autocompleteExpression(sourceModule, *module, typeChecker, typeArena, ancestry, position, result); - return {result, ancestry}; + return {result, ancestry, AutocompleteContext::Property}; } break; @@ -1555,11 +1557,11 @@ static AutocompleteResult autocomplete(const SourceModule& sourceModule, const M } } else if (isIdentifier(node) && (parent->is() || parent->is())) - return {autocompleteStatement(sourceModule, *module, ancestry, position), ancestry}; + return {autocompleteStatement(sourceModule, *module, ancestry, position), ancestry, AutocompleteContext::Statement}; if (std::optional ret = autocompleteStringParams(sourceModule, module, ancestry, position, callback)) { - return {*ret, ancestry}; + return {*ret, ancestry, AutocompleteContext::String}; } else if (node->is()) { @@ -1585,7 +1587,7 @@ static AutocompleteResult autocomplete(const SourceModule& sourceModule, const M } } - return {result, ancestry}; + return {result, ancestry, AutocompleteContext::String}; } if (node->is()) @@ -1594,9 +1596,9 @@ static AutocompleteResult autocomplete(const SourceModule& sourceModule, const M } if (node->asExpr()) - return {autocompleteExpression(sourceModule, *module, typeChecker, typeArena, ancestry, position), ancestry}; + return autocompleteExpression(sourceModule, *module, typeChecker, typeArena, ancestry, position); else if (node->asStat()) - return {autocompleteStatement(sourceModule, *module, ancestry, position), ancestry}; + return {autocompleteStatement(sourceModule, *module, ancestry, position), ancestry, AutocompleteContext::Statement}; return {}; } diff --git a/tests/Autocomplete.test.cpp b/tests/Autocomplete.test.cpp index 75c5a606..0f17531b 100644 --- a/tests/Autocomplete.test.cpp +++ b/tests/Autocomplete.test.cpp @@ -129,6 +129,7 @@ TEST_CASE_FIXTURE(ACFixture, "empty_program") CHECK(!ac.entryMap.empty()); CHECK(ac.entryMap.count("table")); CHECK(ac.entryMap.count("math")); + CHECK_EQ(ac.context, AutocompleteContext::Statement); } TEST_CASE_FIXTURE(ACFixture, "local_initializer") @@ -138,6 +139,7 @@ TEST_CASE_FIXTURE(ACFixture, "local_initializer") auto ac = autocomplete('1'); CHECK(ac.entryMap.count("table")); CHECK(ac.entryMap.count("math")); + CHECK_EQ(ac.context, AutocompleteContext::Expression); } TEST_CASE_FIXTURE(ACFixture, "leave_numbers_alone") @@ -146,6 +148,7 @@ TEST_CASE_FIXTURE(ACFixture, "leave_numbers_alone") auto ac = autocomplete('1'); CHECK(ac.entryMap.empty()); + CHECK_EQ(ac.context, AutocompleteContext::Unknown); } TEST_CASE_FIXTURE(ACFixture, "user_defined_globals") @@ -157,6 +160,7 @@ TEST_CASE_FIXTURE(ACFixture, "user_defined_globals") CHECK(ac.entryMap.count("myLocal")); CHECK(ac.entryMap.count("table")); CHECK(ac.entryMap.count("math")); + CHECK_EQ(ac.context, AutocompleteContext::Statement); } TEST_CASE_FIXTURE(ACFixture, "dont_suggest_local_before_its_definition") @@ -191,6 +195,7 @@ TEST_CASE_FIXTURE(ACFixture, "recursive_function") auto ac = autocomplete('1'); CHECK(ac.entryMap.count("foo")); + CHECK_EQ(ac.context, AutocompleteContext::Statement); } TEST_CASE_FIXTURE(ACFixture, "nested_recursive_function") @@ -293,6 +298,7 @@ TEST_CASE_FIXTURE(ACBuiltinsFixture, "get_member_completions") CHECK(ac.entryMap.count("find")); CHECK(ac.entryMap.count("pack")); CHECK(!ac.entryMap.count("math")); + CHECK_EQ(ac.context, AutocompleteContext::Property); } TEST_CASE_FIXTURE(ACFixture, "nested_member_completions") @@ -306,6 +312,7 @@ TEST_CASE_FIXTURE(ACFixture, "nested_member_completions") CHECK_EQ(2, ac.entryMap.size()); CHECK(ac.entryMap.count("def")); CHECK(ac.entryMap.count("egh")); + CHECK_EQ(ac.context, AutocompleteContext::Property); } TEST_CASE_FIXTURE(ACFixture, "unsealed_table") @@ -319,6 +326,7 @@ TEST_CASE_FIXTURE(ACFixture, "unsealed_table") auto ac = autocomplete('1'); CHECK_EQ(1, ac.entryMap.size()); CHECK(ac.entryMap.count("prop")); + CHECK_EQ(ac.context, AutocompleteContext::Property); } TEST_CASE_FIXTURE(ACFixture, "unsealed_table_2") @@ -333,6 +341,7 @@ TEST_CASE_FIXTURE(ACFixture, "unsealed_table_2") auto ac = autocomplete('1'); CHECK_EQ(1, ac.entryMap.size()); CHECK(ac.entryMap.count("prop")); + CHECK_EQ(ac.context, AutocompleteContext::Property); } TEST_CASE_FIXTURE(ACFixture, "cyclic_table") @@ -346,6 +355,7 @@ TEST_CASE_FIXTURE(ACFixture, "cyclic_table") auto ac = autocomplete('1'); CHECK(ac.entryMap.count("abc")); + CHECK_EQ(ac.context, AutocompleteContext::Property); } TEST_CASE_FIXTURE(ACFixture, "table_union") @@ -361,6 +371,7 @@ TEST_CASE_FIXTURE(ACFixture, "table_union") auto ac = autocomplete('1'); CHECK_EQ(1, ac.entryMap.size()); CHECK(ac.entryMap.count("b2")); + CHECK_EQ(ac.context, AutocompleteContext::Property); } TEST_CASE_FIXTURE(ACFixture, "table_intersection") @@ -378,6 +389,7 @@ TEST_CASE_FIXTURE(ACFixture, "table_intersection") CHECK(ac.entryMap.count("a1")); CHECK(ac.entryMap.count("b2")); CHECK(ac.entryMap.count("c3")); + CHECK_EQ(ac.context, AutocompleteContext::Property); } TEST_CASE_FIXTURE(ACBuiltinsFixture, "get_string_completions") @@ -389,6 +401,7 @@ TEST_CASE_FIXTURE(ACBuiltinsFixture, "get_string_completions") auto ac = autocomplete('1'); CHECK_EQ(17, ac.entryMap.size()); + CHECK_EQ(ac.context, AutocompleteContext::Property); } TEST_CASE_FIXTURE(ACFixture, "get_suggestions_for_new_statement") @@ -400,6 +413,7 @@ TEST_CASE_FIXTURE(ACFixture, "get_suggestions_for_new_statement") CHECK_NE(0, ac.entryMap.size()); CHECK(ac.entryMap.count("table")); + CHECK_EQ(ac.context, AutocompleteContext::Statement); } TEST_CASE_FIXTURE(ACFixture, "get_suggestions_for_the_very_start_of_the_script") @@ -412,6 +426,7 @@ TEST_CASE_FIXTURE(ACFixture, "get_suggestions_for_the_very_start_of_the_script") auto ac = autocomplete('1'); CHECK(ac.entryMap.count("table")); + CHECK_EQ(ac.context, AutocompleteContext::Statement); } TEST_CASE_FIXTURE(ACFixture, "method_call_inside_function_body") @@ -429,6 +444,7 @@ TEST_CASE_FIXTURE(ACFixture, "method_call_inside_function_body") CHECK_NE(0, ac.entryMap.size()); CHECK(!ac.entryMap.count("math")); + CHECK_EQ(ac.context, AutocompleteContext::Property); } TEST_CASE_FIXTURE(ACBuiltinsFixture, "method_call_inside_if_conditional") @@ -442,6 +458,7 @@ TEST_CASE_FIXTURE(ACBuiltinsFixture, "method_call_inside_if_conditional") CHECK_NE(0, ac.entryMap.size()); CHECK(ac.entryMap.count("concat")); CHECK(!ac.entryMap.count("math")); + CHECK_EQ(ac.context, AutocompleteContext::Property); } TEST_CASE_FIXTURE(ACFixture, "statement_between_two_statements") @@ -459,6 +476,8 @@ TEST_CASE_FIXTURE(ACFixture, "statement_between_two_statements") CHECK_NE(0, ac.entryMap.size()); CHECK(ac.entryMap.count("getmyscripts")); + + CHECK_EQ(ac.context, AutocompleteContext::Statement); } TEST_CASE_FIXTURE(ACFixture, "bias_toward_inner_scope") @@ -476,6 +495,7 @@ TEST_CASE_FIXTURE(ACFixture, "bias_toward_inner_scope") auto ac = autocomplete('1'); CHECK(ac.entryMap.count("A")); + CHECK_EQ(ac.context, AutocompleteContext::Statement); TypeId t = follow(*ac.entryMap["A"].type); const TableTypeVar* tt = get(t); @@ -489,10 +509,12 @@ TEST_CASE_FIXTURE(ACFixture, "recommend_statement_starting_keywords") check("@1"); auto ac = autocomplete('1'); CHECK(ac.entryMap.count("local")); + CHECK_EQ(ac.context, AutocompleteContext::Statement); check("local i = @1"); auto ac2 = autocomplete('1'); CHECK(!ac2.entryMap.count("local")); + CHECK_EQ(ac2.context, AutocompleteContext::Expression); } TEST_CASE_FIXTURE(ACFixture, "do_not_overwrite_context_sensitive_kws") @@ -508,6 +530,7 @@ TEST_CASE_FIXTURE(ACFixture, "do_not_overwrite_context_sensitive_kws") AutocompleteEntry entry = ac.entryMap["continue"]; CHECK(entry.kind == AutocompleteEntryKind::Binding); + CHECK_EQ(ac.context, AutocompleteContext::Statement); } TEST_CASE_FIXTURE(ACFixture, "dont_offer_any_suggestions_from_within_a_comment") @@ -525,6 +548,7 @@ TEST_CASE_FIXTURE(ACFixture, "dont_offer_any_suggestions_from_within_a_comment") auto ac = autocomplete('1'); CHECK_EQ(0, ac.entryMap.size()); + CHECK_EQ(ac.context, AutocompleteContext::Unknown); } TEST_CASE_FIXTURE(ACFixture, "dont_offer_any_suggestions_from_the_end_of_a_comment") @@ -536,6 +560,7 @@ TEST_CASE_FIXTURE(ACFixture, "dont_offer_any_suggestions_from_the_end_of_a_comme auto ac = autocomplete('1'); CHECK_EQ(0, ac.entryMap.size()); + CHECK_EQ(ac.context, AutocompleteContext::Unknown); } TEST_CASE_FIXTURE(ACFixture, "dont_offer_any_suggestions_from_within_a_broken_comment") @@ -547,6 +572,7 @@ TEST_CASE_FIXTURE(ACFixture, "dont_offer_any_suggestions_from_within_a_broken_co auto ac = autocomplete('1'); CHECK_EQ(0, ac.entryMap.size()); + CHECK_EQ(ac.context, AutocompleteContext::Unknown); } TEST_CASE_FIXTURE(ACFixture, "dont_offer_any_suggestions_from_within_a_broken_comment_at_the_very_end_of_the_file") @@ -555,6 +581,7 @@ TEST_CASE_FIXTURE(ACFixture, "dont_offer_any_suggestions_from_within_a_broken_co auto ac = autocomplete('1'); CHECK_EQ(0, ac.entryMap.size()); + CHECK_EQ(ac.context, AutocompleteContext::Unknown); } TEST_CASE_FIXTURE(ACFixture, "autocomplete_for_middle_keywords") @@ -566,6 +593,7 @@ TEST_CASE_FIXTURE(ACFixture, "autocomplete_for_middle_keywords") auto ac1 = autocomplete('1'); CHECK_EQ(ac1.entryMap.count("do"), 0); CHECK_EQ(ac1.entryMap.count("end"), 0); + CHECK_EQ(ac1.context, AutocompleteContext::Unknown); check(R"( for x =@1 1 @@ -574,6 +602,7 @@ TEST_CASE_FIXTURE(ACFixture, "autocomplete_for_middle_keywords") auto ac2 = autocomplete('1'); CHECK_EQ(ac2.entryMap.count("do"), 0); CHECK_EQ(ac2.entryMap.count("end"), 0); + CHECK_EQ(ac2.context, AutocompleteContext::Unknown); check(R"( for x = 1,@1 2 @@ -582,6 +611,7 @@ TEST_CASE_FIXTURE(ACFixture, "autocomplete_for_middle_keywords") auto ac3 = autocomplete('1'); CHECK_EQ(1, ac3.entryMap.size()); CHECK_EQ(ac3.entryMap.count("do"), 1); + CHECK_EQ(ac3.context, AutocompleteContext::Keyword); check(R"( for x = 1, @12, @@ -590,6 +620,7 @@ TEST_CASE_FIXTURE(ACFixture, "autocomplete_for_middle_keywords") auto ac4 = autocomplete('1'); CHECK_EQ(ac4.entryMap.count("do"), 0); CHECK_EQ(ac4.entryMap.count("end"), 0); + CHECK_EQ(ac4.context, AutocompleteContext::Expression); check(R"( for x = 1, 2, @15 @@ -598,6 +629,7 @@ TEST_CASE_FIXTURE(ACFixture, "autocomplete_for_middle_keywords") auto ac5 = autocomplete('1'); CHECK_EQ(ac5.entryMap.count("do"), 1); CHECK_EQ(ac5.entryMap.count("end"), 0); + CHECK_EQ(ac5.context, AutocompleteContext::Keyword); check(R"( for x = 1, 2, 5 f@1 @@ -606,6 +638,7 @@ TEST_CASE_FIXTURE(ACFixture, "autocomplete_for_middle_keywords") auto ac6 = autocomplete('1'); CHECK_EQ(ac6.entryMap.size(), 1); CHECK_EQ(ac6.entryMap.count("do"), 1); + CHECK_EQ(ac6.context, AutocompleteContext::Keyword); check(R"( for x = 1, 2, 5 do @1 @@ -613,6 +646,7 @@ TEST_CASE_FIXTURE(ACFixture, "autocomplete_for_middle_keywords") auto ac7 = autocomplete('1'); CHECK_EQ(ac7.entryMap.count("end"), 1); + CHECK_EQ(ac7.context, AutocompleteContext::Statement); } TEST_CASE_FIXTURE(ACFixture, "autocomplete_for_in_middle_keywords") @@ -623,6 +657,7 @@ TEST_CASE_FIXTURE(ACFixture, "autocomplete_for_in_middle_keywords") auto ac1 = autocomplete('1'); CHECK_EQ(0, ac1.entryMap.size()); + CHECK_EQ(ac1.context, AutocompleteContext::Unknown); check(R"( for x@1 @2 @@ -630,10 +665,12 @@ TEST_CASE_FIXTURE(ACFixture, "autocomplete_for_in_middle_keywords") auto ac2 = autocomplete('1'); CHECK_EQ(0, ac2.entryMap.size()); + CHECK_EQ(ac2.context, AutocompleteContext::Unknown); auto ac2a = autocomplete('2'); CHECK_EQ(1, ac2a.entryMap.size()); CHECK_EQ(1, ac2a.entryMap.count("in")); + CHECK_EQ(ac2a.context, AutocompleteContext::Keyword); check(R"( for x in y@1 @@ -642,6 +679,7 @@ TEST_CASE_FIXTURE(ACFixture, "autocomplete_for_in_middle_keywords") auto ac3 = autocomplete('1'); CHECK_EQ(ac3.entryMap.count("table"), 1); CHECK_EQ(ac3.entryMap.count("do"), 0); + CHECK_EQ(ac3.context, AutocompleteContext::Expression); check(R"( for x in y @1 @@ -650,6 +688,7 @@ TEST_CASE_FIXTURE(ACFixture, "autocomplete_for_in_middle_keywords") auto ac4 = autocomplete('1'); CHECK_EQ(ac4.entryMap.size(), 1); CHECK_EQ(ac4.entryMap.count("do"), 1); + CHECK_EQ(ac4.context, AutocompleteContext::Keyword); check(R"( for x in f f@1 @@ -658,6 +697,7 @@ TEST_CASE_FIXTURE(ACFixture, "autocomplete_for_in_middle_keywords") auto ac5 = autocomplete('1'); CHECK_EQ(ac5.entryMap.size(), 1); CHECK_EQ(ac5.entryMap.count("do"), 1); + CHECK_EQ(ac5.context, AutocompleteContext::Keyword); check(R"( for x in y do @1 @@ -668,6 +708,7 @@ TEST_CASE_FIXTURE(ACFixture, "autocomplete_for_in_middle_keywords") CHECK_EQ(ac6.entryMap.count("table"), 1); CHECK_EQ(ac6.entryMap.count("end"), 1); CHECK_EQ(ac6.entryMap.count("function"), 1); + CHECK_EQ(ac6.context, AutocompleteContext::Statement); check(R"( for x in y do e@1 @@ -678,6 +719,7 @@ TEST_CASE_FIXTURE(ACFixture, "autocomplete_for_in_middle_keywords") CHECK_EQ(ac7.entryMap.count("table"), 1); CHECK_EQ(ac7.entryMap.count("end"), 1); CHECK_EQ(ac7.entryMap.count("function"), 1); + CHECK_EQ(ac7.context, AutocompleteContext::Statement); } TEST_CASE_FIXTURE(ACFixture, "autocomplete_while_middle_keywords") @@ -689,6 +731,7 @@ TEST_CASE_FIXTURE(ACFixture, "autocomplete_while_middle_keywords") auto ac1 = autocomplete('1'); CHECK_EQ(ac1.entryMap.count("do"), 0); CHECK_EQ(ac1.entryMap.count("end"), 0); + CHECK_EQ(ac1.context, AutocompleteContext::Expression); check(R"( while true @1 @@ -697,6 +740,7 @@ TEST_CASE_FIXTURE(ACFixture, "autocomplete_while_middle_keywords") auto ac2 = autocomplete('1'); CHECK_EQ(1, ac2.entryMap.size()); CHECK_EQ(ac2.entryMap.count("do"), 1); + CHECK_EQ(ac2.context, AutocompleteContext::Keyword); check(R"( while true do @1 @@ -704,6 +748,7 @@ TEST_CASE_FIXTURE(ACFixture, "autocomplete_while_middle_keywords") auto ac3 = autocomplete('1'); CHECK_EQ(ac3.entryMap.count("end"), 1); + CHECK_EQ(ac3.context, AutocompleteContext::Statement); check(R"( while true d@1 @@ -712,6 +757,7 @@ TEST_CASE_FIXTURE(ACFixture, "autocomplete_while_middle_keywords") auto ac4 = autocomplete('1'); CHECK_EQ(1, ac4.entryMap.size()); CHECK_EQ(ac4.entryMap.count("do"), 1); + CHECK_EQ(ac4.context, AutocompleteContext::Keyword); } TEST_CASE_FIXTURE(ACFixture, "autocomplete_if_middle_keywords") @@ -728,6 +774,7 @@ TEST_CASE_FIXTURE(ACFixture, "autocomplete_if_middle_keywords") CHECK_EQ(ac1.entryMap.count("else"), 0); CHECK_EQ(ac1.entryMap.count("elseif"), 0); CHECK_EQ(ac1.entryMap.count("end"), 0); + CHECK_EQ(ac1.context, AutocompleteContext::Expression); check(R"( if x @1 @@ -739,6 +786,7 @@ TEST_CASE_FIXTURE(ACFixture, "autocomplete_if_middle_keywords") CHECK_EQ(ac2.entryMap.count("else"), 0); CHECK_EQ(ac2.entryMap.count("elseif"), 0); CHECK_EQ(ac2.entryMap.count("end"), 0); + CHECK_EQ(ac2.context, AutocompleteContext::Keyword); check(R"( if x t@1 @@ -747,6 +795,7 @@ TEST_CASE_FIXTURE(ACFixture, "autocomplete_if_middle_keywords") auto ac3 = autocomplete('1'); CHECK_EQ(1, ac3.entryMap.size()); CHECK_EQ(ac3.entryMap.count("then"), 1); + CHECK_EQ(ac3.context, AutocompleteContext::Keyword); check(R"( if x then @@ -760,6 +809,7 @@ TEST_CASE_FIXTURE(ACFixture, "autocomplete_if_middle_keywords") CHECK_EQ(ac4.entryMap.count("function"), 1); CHECK_EQ(ac4.entryMap.count("elseif"), 1); CHECK_EQ(ac4.entryMap.count("end"), 0); + CHECK_EQ(ac4.context, AutocompleteContext::Statement); check(R"( if x then @@ -772,6 +822,7 @@ TEST_CASE_FIXTURE(ACFixture, "autocomplete_if_middle_keywords") CHECK_EQ(ac4a.entryMap.count("table"), 1); CHECK_EQ(ac4a.entryMap.count("else"), 1); CHECK_EQ(ac4a.entryMap.count("elseif"), 1); + CHECK_EQ(ac4a.context, AutocompleteContext::Statement); check(R"( if x then @@ -786,6 +837,7 @@ TEST_CASE_FIXTURE(ACFixture, "autocomplete_if_middle_keywords") CHECK_EQ(ac5.entryMap.count("else"), 0); CHECK_EQ(ac5.entryMap.count("elseif"), 0); CHECK_EQ(ac5.entryMap.count("end"), 0); + CHECK_EQ(ac5.context, AutocompleteContext::Statement); } TEST_CASE_FIXTURE(ACFixture, "autocomplete_until_in_repeat") @@ -797,6 +849,7 @@ TEST_CASE_FIXTURE(ACFixture, "autocomplete_until_in_repeat") auto ac = autocomplete('1'); CHECK_EQ(ac.entryMap.count("table"), 1); CHECK_EQ(ac.entryMap.count("until"), 1); + CHECK_EQ(ac.context, AutocompleteContext::Statement); } TEST_CASE_FIXTURE(ACFixture, "autocomplete_until_expression") @@ -808,6 +861,7 @@ TEST_CASE_FIXTURE(ACFixture, "autocomplete_until_expression") auto ac = autocomplete('1'); CHECK_EQ(ac.entryMap.count("table"), 1); + CHECK_EQ(ac.context, AutocompleteContext::Expression); } TEST_CASE_FIXTURE(ACFixture, "local_names") @@ -819,6 +873,7 @@ TEST_CASE_FIXTURE(ACFixture, "local_names") auto ac1 = autocomplete('1'); CHECK_EQ(ac1.entryMap.size(), 1); CHECK_EQ(ac1.entryMap.count("function"), 1); + CHECK_EQ(ac1.context, AutocompleteContext::Unknown); check(R"( local ab, cd@1 @@ -826,6 +881,7 @@ TEST_CASE_FIXTURE(ACFixture, "local_names") auto ac2 = autocomplete('1'); CHECK(ac2.entryMap.empty()); + CHECK_EQ(ac2.context, AutocompleteContext::Unknown); } TEST_CASE_FIXTURE(ACFixture, "autocomplete_end_with_fn_exprs") @@ -836,6 +892,7 @@ TEST_CASE_FIXTURE(ACFixture, "autocomplete_end_with_fn_exprs") auto ac = autocomplete('1'); CHECK_EQ(ac.entryMap.count("end"), 1); + CHECK_EQ(ac.context, AutocompleteContext::Statement); } TEST_CASE_FIXTURE(ACFixture, "autocomplete_end_with_lambda") @@ -846,6 +903,7 @@ TEST_CASE_FIXTURE(ACFixture, "autocomplete_end_with_lambda") auto ac = autocomplete('1'); CHECK_EQ(ac.entryMap.count("end"), 1); + CHECK_EQ(ac.context, AutocompleteContext::Statement); } TEST_CASE_FIXTURE(ACFixture, "stop_at_first_stat_when_recommending_keywords") @@ -858,6 +916,7 @@ TEST_CASE_FIXTURE(ACFixture, "stop_at_first_stat_when_recommending_keywords") auto ac1 = autocomplete('1'); CHECK_EQ(ac1.entryMap.count("in"), 1); CHECK_EQ(ac1.entryMap.count("until"), 0); + CHECK_EQ(ac1.context, AutocompleteContext::Keyword); } TEST_CASE_FIXTURE(ACFixture, "autocomplete_repeat_middle_keyword") @@ -980,6 +1039,7 @@ TEST_CASE_FIXTURE(ACFixture, "local_function_params") auto ac2 = autocomplete('1'); CHECK_EQ(ac2.entryMap.count("abc"), 1); CHECK_EQ(ac2.entryMap.count("def"), 1); + CHECK_EQ(ac2.context, AutocompleteContext::Statement); check(R"( local function abc(def, ghi@1) @@ -988,6 +1048,7 @@ TEST_CASE_FIXTURE(ACFixture, "local_function_params") auto ac3 = autocomplete('1'); CHECK(ac3.entryMap.empty()); + CHECK_EQ(ac3.context, AutocompleteContext::Unknown); } TEST_CASE_FIXTURE(ACFixture, "global_function_params") @@ -1022,6 +1083,7 @@ TEST_CASE_FIXTURE(ACFixture, "global_function_params") auto ac2 = autocomplete('1'); CHECK_EQ(ac2.entryMap.count("abc"), 1); CHECK_EQ(ac2.entryMap.count("def"), 1); + CHECK_EQ(ac2.context, AutocompleteContext::Statement); check(R"( function abc(def, ghi@1) @@ -1030,6 +1092,7 @@ TEST_CASE_FIXTURE(ACFixture, "global_function_params") auto ac3 = autocomplete('1'); CHECK(ac3.entryMap.empty()); + CHECK_EQ(ac3.context, AutocompleteContext::Unknown); } TEST_CASE_FIXTURE(ACFixture, "arguments_to_global_lambda") @@ -1074,6 +1137,7 @@ TEST_CASE_FIXTURE(ACFixture, "function_expr_params") auto ac2 = autocomplete('1'); CHECK_EQ(ac2.entryMap.count("def"), 1); + CHECK_EQ(ac2.context, AutocompleteContext::Statement); } TEST_CASE_FIXTURE(ACFixture, "local_initializer") @@ -1135,6 +1199,7 @@ local b: string = "don't trip" CHECK(ac.entryMap.count("nil")); CHECK(ac.entryMap.count("number")); + CHECK_EQ(ac.context, AutocompleteContext::Type); } TEST_CASE_FIXTURE(ACFixture, "private_types") @@ -1203,6 +1268,7 @@ local a: aa auto ac = Luau::autocomplete(frontend, "Module/B", Position{2, 11}, nullCallback); CHECK(ac.entryMap.count("aaa")); + CHECK_EQ(ac.context, AutocompleteContext::Type); } TEST_CASE_FIXTURE(ACFixture, "module_type_members") @@ -1227,6 +1293,7 @@ local a: aaa. CHECK_EQ(2, ac.entryMap.size()); CHECK(ac.entryMap.count("A")); CHECK(ac.entryMap.count("B")); + CHECK_EQ(ac.context, AutocompleteContext::Type); } TEST_CASE_FIXTURE(ACFixture, "argument_types") @@ -1240,6 +1307,7 @@ local b: string = "don't trip" CHECK(ac.entryMap.count("nil")); CHECK(ac.entryMap.count("number")); + CHECK_EQ(ac.context, AutocompleteContext::Type); } TEST_CASE_FIXTURE(ACFixture, "return_types") @@ -1253,6 +1321,7 @@ local b: string = "don't trip" CHECK(ac.entryMap.count("nil")); CHECK(ac.entryMap.count("number")); + CHECK_EQ(ac.context, AutocompleteContext::Type); } TEST_CASE_FIXTURE(ACFixture, "as_types") @@ -1266,6 +1335,7 @@ local b: number = (a :: n@1 CHECK(ac.entryMap.count("nil")); CHECK(ac.entryMap.count("number")); + CHECK_EQ(ac.context, AutocompleteContext::Type); } TEST_CASE_FIXTURE(ACFixture, "function_type_types") @@ -1314,6 +1384,7 @@ local b: string = "don't trip" auto ac = autocomplete('1'); CHECK(ac.entryMap.count("Tee")); + CHECK_EQ(ac.context, AutocompleteContext::Type); } TEST_CASE_FIXTURE(ACFixture, "type_correct_suggestion_in_argument") @@ -1402,6 +1473,7 @@ local b: Foo = { a = a.@1 CHECK(ac.entryMap.count("one")); CHECK(ac.entryMap["one"].typeCorrect == TypeCorrectKind::Correct); CHECK(ac.entryMap["two"].typeCorrect == TypeCorrectKind::None); + CHECK_EQ(ac.context, AutocompleteContext::Property); check(R"( type Foo = { a: number, b: string } @@ -1414,6 +1486,7 @@ local b: Foo = { b = a.@1 CHECK(ac.entryMap.count("two")); CHECK(ac.entryMap["two"].typeCorrect == TypeCorrectKind::Correct); CHECK(ac.entryMap["one"].typeCorrect == TypeCorrectKind::None); + CHECK_EQ(ac.context, AutocompleteContext::Property); } TEST_CASE_FIXTURE(ACFixture, "type_correct_function_return_types") @@ -2395,6 +2468,7 @@ local t: Test = { f@1 } auto ac = autocomplete('1'); CHECK(ac.entryMap.count("first")); CHECK(ac.entryMap.count("second")); + CHECK_EQ(ac.context, AutocompleteContext::Property); // Intersection check(R"( @@ -2405,6 +2479,7 @@ local t: Test = { f@1 } ac = autocomplete('1'); CHECK(ac.entryMap.count("first")); CHECK(ac.entryMap.count("second")); + CHECK_EQ(ac.context, AutocompleteContext::Property); // Union check(R"( @@ -2416,6 +2491,7 @@ local t: Test = { s@1 } CHECK(ac.entryMap.count("second")); CHECK(!ac.entryMap.count("first")); CHECK(!ac.entryMap.count("third")); + CHECK_EQ(ac.context, AutocompleteContext::Property); // No parenthesis suggestion check(R"( @@ -2426,6 +2502,7 @@ local t: Test = { f@1 } ac = autocomplete('1'); CHECK(ac.entryMap.count("first")); CHECK(ac.entryMap["first"].parens == ParenthesesRecommendation::None); + CHECK_EQ(ac.context, AutocompleteContext::Property); // When key is changed check(R"( @@ -2436,6 +2513,7 @@ local t: Test = { f@1 = 2 } ac = autocomplete('1'); CHECK(ac.entryMap.count("first")); CHECK(ac.entryMap.count("second")); + CHECK_EQ(ac.context, AutocompleteContext::Property); // Alternative key syntax check(R"( @@ -2446,6 +2524,7 @@ local t: Test = { ["f@1"] } ac = autocomplete('1'); CHECK(ac.entryMap.count("first")); CHECK(ac.entryMap.count("second")); + CHECK_EQ(ac.context, AutocompleteContext::Property); // Not an alternative key syntax check(R"( @@ -2456,6 +2535,7 @@ local t: Test = { "f@1" } ac = autocomplete('1'); CHECK(!ac.entryMap.count("first")); CHECK(!ac.entryMap.count("second")); + CHECK_EQ(ac.context, AutocompleteContext::String); // Skip keys that are already defined check(R"( @@ -2466,6 +2546,7 @@ local t: Test = { first = 2, s@1 } ac = autocomplete('1'); CHECK(!ac.entryMap.count("first")); CHECK(ac.entryMap.count("second")); + CHECK_EQ(ac.context, AutocompleteContext::Property); // Don't skip active key check(R"( @@ -2476,6 +2557,7 @@ local t: Test = { first@1 } ac = autocomplete('1'); CHECK(ac.entryMap.count("first")); CHECK(ac.entryMap.count("second")); + CHECK_EQ(ac.context, AutocompleteContext::Property); // Inference after first key check(R"( @@ -2488,6 +2570,7 @@ local t = { ac = autocomplete('1'); CHECK(ac.entryMap.count("first")); CHECK(ac.entryMap.count("second")); + CHECK_EQ(ac.context, AutocompleteContext::Property); check(R"( local t = { @@ -2499,6 +2582,7 @@ local t = { ac = autocomplete('1'); CHECK(ac.entryMap.count("first")); CHECK(ac.entryMap.count("second")); + CHECK_EQ(ac.context, AutocompleteContext::Property); } TEST_CASE_FIXTURE(ACFixture, "autocomplete_documentation_symbols") @@ -2542,6 +2626,7 @@ a = if temp then even elseif true then temp else e@9 CHECK(ac.entryMap.count("then") == 0); CHECK(ac.entryMap.count("else") == 0); CHECK(ac.entryMap.count("elseif") == 0); + CHECK_EQ(ac.context, AutocompleteContext::Expression); ac = autocomplete('2'); CHECK(ac.entryMap.count("temp") == 0); @@ -2549,18 +2634,21 @@ a = if temp then even elseif true then temp else e@9 CHECK(ac.entryMap.count("then")); CHECK(ac.entryMap.count("else") == 0); CHECK(ac.entryMap.count("elseif") == 0); + CHECK_EQ(ac.context, AutocompleteContext::Keyword); ac = autocomplete('3'); CHECK(ac.entryMap.count("even")); CHECK(ac.entryMap.count("then") == 0); CHECK(ac.entryMap.count("else") == 0); CHECK(ac.entryMap.count("elseif") == 0); + CHECK_EQ(ac.context, AutocompleteContext::Expression); ac = autocomplete('4'); CHECK(ac.entryMap.count("even") == 0); CHECK(ac.entryMap.count("then") == 0); CHECK(ac.entryMap.count("else")); CHECK(ac.entryMap.count("elseif")); + CHECK_EQ(ac.context, AutocompleteContext::Keyword); ac = autocomplete('5'); CHECK(ac.entryMap.count("temp")); @@ -2568,6 +2656,7 @@ a = if temp then even elseif true then temp else e@9 CHECK(ac.entryMap.count("then") == 0); CHECK(ac.entryMap.count("else") == 0); CHECK(ac.entryMap.count("elseif") == 0); + CHECK_EQ(ac.context, AutocompleteContext::Expression); ac = autocomplete('6'); CHECK(ac.entryMap.count("temp") == 0); @@ -2575,6 +2664,7 @@ a = if temp then even elseif true then temp else e@9 CHECK(ac.entryMap.count("then")); CHECK(ac.entryMap.count("else") == 0); CHECK(ac.entryMap.count("elseif") == 0); + CHECK_EQ(ac.context, AutocompleteContext::Keyword); ac = autocomplete('7'); CHECK(ac.entryMap.count("temp")); @@ -2582,17 +2672,20 @@ a = if temp then even elseif true then temp else e@9 CHECK(ac.entryMap.count("then") == 0); CHECK(ac.entryMap.count("else") == 0); CHECK(ac.entryMap.count("elseif") == 0); + CHECK_EQ(ac.context, AutocompleteContext::Expression); ac = autocomplete('8'); CHECK(ac.entryMap.count("even") == 0); CHECK(ac.entryMap.count("then") == 0); CHECK(ac.entryMap.count("else")); CHECK(ac.entryMap.count("elseif")); + CHECK_EQ(ac.context, AutocompleteContext::Keyword); ac = autocomplete('9'); CHECK(ac.entryMap.count("then") == 0); CHECK(ac.entryMap.count("else") == 0); CHECK(ac.entryMap.count("elseif") == 0); + CHECK_EQ(ac.context, AutocompleteContext::Expression); } TEST_CASE_FIXTURE(ACFixture, "autocomplete_if_else_regression") @@ -2626,6 +2719,7 @@ local a: A<(number, s@1> CHECK(ac.entryMap.count("number")); CHECK(ac.entryMap.count("string")); + CHECK_EQ(ac.context, AutocompleteContext::Type); } TEST_CASE_FIXTURE(ACFixture, "autocomplete_first_function_arg_expected_type") @@ -2686,6 +2780,7 @@ type A = () -> T CHECK(ac.entryMap.count("number")); CHECK(ac.entryMap.count("string")); + CHECK_EQ(ac.context, AutocompleteContext::Type); } TEST_CASE_FIXTURE(ACFixture, "autocomplete_default_type_pack_parameters") @@ -2698,6 +2793,7 @@ type A = () -> T CHECK(ac.entryMap.count("number")); CHECK(ac.entryMap.count("string")); + CHECK_EQ(ac.context, AutocompleteContext::Type); } TEST_CASE_FIXTURE(ACBuiltinsFixture, "autocomplete_oop_implicit_self") @@ -2752,16 +2848,19 @@ TEST_CASE_FIXTURE(ACFixture, "autocomplete_string_singletons") CHECK(ac.entryMap.count("cat")); CHECK(ac.entryMap.count("dog")); + CHECK_EQ(ac.context, AutocompleteContext::String); ac = autocomplete('2'); CHECK(ac.entryMap.count("\"cat\"")); CHECK(ac.entryMap.count("\"dog\"")); + CHECK_EQ(ac.context, AutocompleteContext::Expression); ac = autocomplete('3'); CHECK(ac.entryMap.count("cat")); CHECK(ac.entryMap.count("dog")); + CHECK_EQ(ac.context, AutocompleteContext::String); check(R"( type tagged = {tag:"cat", fieldx:number} | {tag:"dog", fieldy:number} @@ -2772,6 +2871,7 @@ TEST_CASE_FIXTURE(ACFixture, "autocomplete_string_singletons") CHECK(ac.entryMap.count("cat")); CHECK(ac.entryMap.count("dog")); + CHECK_EQ(ac.context, AutocompleteContext::String); } TEST_CASE_FIXTURE(ACFixture, "autocomplete_string_singleton_equality") @@ -2808,6 +2908,7 @@ f(@1) CHECK(ac.entryMap["true"].typeCorrect == TypeCorrectKind::Correct); REQUIRE(ac.entryMap.count("false")); CHECK(ac.entryMap["false"].typeCorrect == TypeCorrectKind::None); + CHECK_EQ(ac.context, AutocompleteContext::Expression); } TEST_CASE_FIXTURE(ACFixture, "autocomplete_string_singleton_escape") From 2c40b7661c2fbb116f017626d3bc7874b146fe6f Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Thu, 11 Aug 2022 08:42:31 -0700 Subject: [PATCH 335/399] Update lint.md (#634) Add documentation for IntegerParsing and ComparisonPrecedence lints --- docs/_pages/lint.md | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+) diff --git a/docs/_pages/lint.md b/docs/_pages/lint.md index 340d35a2..fd3c595c 100644 --- a/docs/_pages/lint.md +++ b/docs/_pages/lint.md @@ -327,3 +327,26 @@ Luau uses comments that start from `!` to control certain aspects of analysis, f -- Unknown comment directive 'nostrict'; did you mean 'nonstrict'?" ``` ``` + +## IntegerParsing (27) + +Luau parses hexadecimal and binary literals as 64-bit integers before converting them to Luau numbers. As a result, numbers that exceed 2^64 are silently truncated to 2^64, which can result in unexpected program behavior. This warning flags literals that are truncated: + +``` +-- Hexadecimal number literal exceeded available precision and has been truncated to 2^64 +local x = 0x1111111111111111111111111111111111111 +``` + +## ComparisonPrecedence (28) + +Because of operator precedence rules, not X == Y parses as (not X) == Y; however, often the intent was to invert the result of the comparison. This warning flags erroneous conditions like that, as well as flagging cases where two comparisons happen in a row without any parentheses: + +``` +-- not X == Y is equivalent to (not X) == Y; consider using X ~= Y, or wrap one of the expressions in parentheses to silence +if not x == 5 then +end + +-- X <= Y <= Z is equivalent to (X <= Y) <= Z; wrap one of the expressions in parentheses to silence +if 1 <= x <= 3 then +end +``` From 106b26988540349e33cffa5a02ae1491f3a4d05d Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Thu, 11 Aug 2022 13:42:54 -0700 Subject: [PATCH 336/399] Sync to upstream/release/540 --- Analysis/include/Luau/Autocomplete.h | 15 +- Analysis/include/Luau/Clone.h | 2 +- .../include/Luau/ConstraintGraphBuilder.h | 7 +- Analysis/include/Luau/Frontend.h | 2 +- Analysis/include/Luau/Linter.h | 1 + Analysis/include/Luau/Scope.h | 4 - Analysis/include/Luau/TypeInfer.h | 5 + Analysis/include/Luau/TypeVar.h | 7 +- Analysis/include/Luau/Unifier.h | 10 +- Analysis/src/ApplyTypeFunction.cpp | 4 + Analysis/src/Autocomplete.cpp | 76 +-- Analysis/src/Clone.cpp | 12 +- Analysis/src/ConstraintGraphBuilder.cpp | 201 ++++++- Analysis/src/ConstraintSolver.cpp | 8 + Analysis/src/Frontend.cpp | 6 +- Analysis/src/Instantiation.cpp | 2 + Analysis/src/Linter.cpp | 64 +++ Analysis/src/Module.cpp | 139 ++++- Analysis/src/Normalize.cpp | 3 +- Analysis/src/Quantify.cpp | 5 + Analysis/src/Scope.cpp | 30 - Analysis/src/Substitution.cpp | 12 +- Analysis/src/TypeChecker2.cpp | 173 +++++- Analysis/src/TypeInfer.cpp | 15 +- Analysis/src/Unifier.cpp | 83 ++- CLI/Reduce.cpp | 511 ++++++++++++++++++ CMakeLists.txt | 7 + CodeGen/include/Luau/AssemblyBuilderX64.h | 17 + CodeGen/src/AssemblyBuilderX64.cpp | 194 ++++++- Compiler/src/Compiler.cpp | 256 +++++++-- Makefile | 4 +- Sources.cmake | 8 + bench/bench.py | 4 +- bench/influxbench.py | 7 +- tests/AssemblyBuilderX64.test.cpp | 92 +++- tests/Autocomplete.test.cpp | 101 ++++ tests/Compiler.test.cpp | 299 ++++++++-- tests/Fixture.cpp | 17 +- tests/Linter.test.cpp | 36 +- tests/Module.test.cpp | 74 +++ tests/NonstrictMode.test.cpp | 1 + tests/TypeInfer.definitions.test.cpp | 27 + tests/TypeInfer.generics.test.cpp | 1 + tests/TypeInfer.modules.test.cpp | 2 - tests/TypeInfer.tables.test.cpp | 4 +- tests/TypeVar.test.cpp | 11 + tests/conformance/basic.lua | 6 + tests/conformance/errors.lua | 3 +- tests/conformance/strings.lua | 8 + tools/faillist.txt | 150 +---- tools/heapgraph.py | 34 +- tools/heapstat.py | 18 +- 52 files changed, 2319 insertions(+), 459 deletions(-) create mode 100644 CLI/Reduce.cpp diff --git a/Analysis/include/Luau/Autocomplete.h b/Analysis/include/Luau/Autocomplete.h index 5e8d6605..f40f8b49 100644 --- a/Analysis/include/Luau/Autocomplete.h +++ b/Analysis/include/Luau/Autocomplete.h @@ -19,6 +19,17 @@ struct TypeChecker; using ModulePtr = std::shared_ptr; +enum class AutocompleteContext +{ + Unknown, + Expression, + Statement, + Property, + Type, + Keyword, + String, +}; + enum class AutocompleteEntryKind { Property, @@ -66,11 +77,13 @@ struct AutocompleteResult { AutocompleteEntryMap entryMap; std::vector ancestry; + AutocompleteContext context = AutocompleteContext::Unknown; AutocompleteResult() = default; - AutocompleteResult(AutocompleteEntryMap entryMap, std::vector ancestry) + AutocompleteResult(AutocompleteEntryMap entryMap, std::vector ancestry, AutocompleteContext context) : entryMap(std::move(entryMap)) , ancestry(std::move(ancestry)) + , context(context) { } }; diff --git a/Analysis/include/Luau/Clone.h b/Analysis/include/Luau/Clone.h index 548a58f5..217e1cc3 100644 --- a/Analysis/include/Luau/Clone.h +++ b/Analysis/include/Luau/Clone.h @@ -25,6 +25,6 @@ TypePackId clone(TypePackId tp, TypeArena& dest, CloneState& cloneState); TypeId clone(TypeId tp, TypeArena& dest, CloneState& cloneState); TypeFun clone(const TypeFun& typeFun, TypeArena& dest, CloneState& cloneState); -TypeId shallowClone(TypeId ty, TypeArena& dest, const TxnLog* log); +TypeId shallowClone(TypeId ty, TypeArena& dest, const TxnLog* log, bool alwaysClone = false); } // namespace Luau diff --git a/Analysis/include/Luau/ConstraintGraphBuilder.h b/Analysis/include/Luau/ConstraintGraphBuilder.h index 69f35d46..41d14326 100644 --- a/Analysis/include/Luau/ConstraintGraphBuilder.h +++ b/Analysis/include/Luau/ConstraintGraphBuilder.h @@ -28,6 +28,7 @@ struct ConstraintGraphBuilder std::vector> scopes; ModuleName moduleName; + ModulePtr module; SingletonTypes& singletonTypes; const NotNull arena; // The root scope of the module we're generating constraints for. @@ -53,9 +54,9 @@ struct ConstraintGraphBuilder // Occasionally constraint generation needs to produce an ICE. const NotNull ice; - NotNull globalScope; + ScopePtr globalScope; - ConstraintGraphBuilder(const ModuleName& moduleName, TypeArena* arena, NotNull ice, NotNull globalScope); + ConstraintGraphBuilder(const ModuleName& moduleName, ModulePtr module, TypeArena* arena, NotNull ice, const ScopePtr& globalScope); /** * Fabricates a new free type belonging to a given scope. @@ -103,6 +104,7 @@ struct ConstraintGraphBuilder void visit(const ScopePtr& scope, AstStatBlock* block); void visit(const ScopePtr& scope, AstStatLocal* local); void visit(const ScopePtr& scope, AstStatFor* for_); + void visit(const ScopePtr& scope, AstStatWhile* while_); void visit(const ScopePtr& scope, AstStatLocalFunction* function); void visit(const ScopePtr& scope, AstStatFunction* function); void visit(const ScopePtr& scope, AstStatReturn* ret); @@ -131,6 +133,7 @@ struct ConstraintGraphBuilder TypeId check(const ScopePtr& scope, AstExprIndexExpr* indexExpr); TypeId check(const ScopePtr& scope, AstExprUnary* unary); TypeId check(const ScopePtr& scope, AstExprBinary* binary); + TypeId check(const ScopePtr& scope, AstExprTypeAssertion* typeAssert); struct FunctionSignature { diff --git a/Analysis/include/Luau/Frontend.h b/Analysis/include/Luau/Frontend.h index 9b8ec19e..82df493a 100644 --- a/Analysis/include/Luau/Frontend.h +++ b/Analysis/include/Luau/Frontend.h @@ -154,7 +154,7 @@ struct Frontend LoadDefinitionFileResult loadDefinitionFile(std::string_view source, const std::string& packageName); - NotNull getGlobalScope(); + ScopePtr getGlobalScope(); private: ModulePtr check(const SourceModule& sourceModule, Mode mode, const ScopePtr& environmentScope); diff --git a/Analysis/include/Luau/Linter.h b/Analysis/include/Luau/Linter.h index 2cd91d54..0e3d9880 100644 --- a/Analysis/include/Luau/Linter.h +++ b/Analysis/include/Luau/Linter.h @@ -53,6 +53,7 @@ struct LintWarning Code_MisleadingAndOr = 25, Code_CommentDirective = 26, Code_IntegerParsing = 27, + Code_ComparisonPrecedence = 28, Code__Count }; diff --git a/Analysis/include/Luau/Scope.h b/Analysis/include/Luau/Scope.h index 55ca54c6..dc123335 100644 --- a/Analysis/include/Luau/Scope.h +++ b/Analysis/include/Luau/Scope.h @@ -36,8 +36,6 @@ struct Scope // All the children of this scope. std::vector> children; std::unordered_map bindings; - std::unordered_map typeBindings; - std::unordered_map typePackBindings; TypePackId returnType; std::optional varargPack; // All constraints belonging to this scope. @@ -52,8 +50,6 @@ struct Scope std::unordered_map> importedTypeBindings; std::optional lookup(Symbol sym); - std::optional lookupTypeBinding(const Name& name); - std::optional lookupTypePackBinding(const Name& name); std::optional lookupType(const Name& name); std::optional lookupImportedType(const Name& moduleAlias, const Name& name); diff --git a/Analysis/include/Luau/TypeInfer.h b/Analysis/include/Luau/TypeInfer.h index c50b2c8c..5c55ddb0 100644 --- a/Analysis/include/Luau/TypeInfer.h +++ b/Analysis/include/Luau/TypeInfer.h @@ -16,6 +16,8 @@ #include #include +LUAU_FASTFLAG(LuauClassTypeVarsInSubstitution) + namespace Luau { @@ -57,6 +59,9 @@ struct Anyification : Substitution bool ignoreChildren(TypeId ty) override { + if (FFlag::LuauClassTypeVarsInSubstitution && get(ty)) + return true; + return ty->persistent; } bool ignoreChildren(TypePackId ty) override diff --git a/Analysis/include/Luau/TypeVar.h b/Analysis/include/Luau/TypeVar.h index 6a13b11c..35b37394 100644 --- a/Analysis/include/Luau/TypeVar.h +++ b/Analysis/include/Luau/TypeVar.h @@ -717,6 +717,7 @@ struct TypeIterator stack.push_front({t, 0}); seen.insert(t); + descend(); } TypeIterator& operator++() @@ -748,17 +749,19 @@ struct TypeIterator const TypeId& operator*() { - LUAU_ASSERT(!stack.empty()); - descend(); + LUAU_ASSERT(!stack.empty()); + auto [t, currentIndex] = stack.front(); LUAU_ASSERT(t); + const std::vector& types = getTypes(t); LUAU_ASSERT(currentIndex < types.size()); const TypeId& ty = types[currentIndex]; LUAU_ASSERT(!get(follow(ty))); + return ty; } diff --git a/Analysis/include/Luau/Unifier.h b/Analysis/include/Luau/Unifier.h index f460dc87..9fa29076 100644 --- a/Analysis/include/Luau/Unifier.h +++ b/Analysis/include/Luau/Unifier.h @@ -109,11 +109,11 @@ private: public: void unifyLowerBound(TypePackId subTy, TypePackId superTy, TypeLevel demotedLevel); - // Report an "infinite type error" if the type "needle" already occurs within "haystack" - void occursCheck(TypeId needle, TypeId haystack); - void occursCheck(DenseHashSet& seen, TypeId needle, TypeId haystack); - void occursCheck(TypePackId needle, TypePackId haystack); - void occursCheck(DenseHashSet& seen, TypePackId needle, TypePackId haystack); + // Returns true if the type "needle" already occurs within "haystack" and reports an "infinite type error" + bool occursCheck(TypeId needle, TypeId haystack); + bool occursCheck(DenseHashSet& seen, TypeId needle, TypeId haystack); + bool occursCheck(TypePackId needle, TypePackId haystack); + bool occursCheck(DenseHashSet& seen, TypePackId needle, TypePackId haystack); Unifier makeChildUnifier(); diff --git a/Analysis/src/ApplyTypeFunction.cpp b/Analysis/src/ApplyTypeFunction.cpp index c6ac3e19..b293ed3d 100644 --- a/Analysis/src/ApplyTypeFunction.cpp +++ b/Analysis/src/ApplyTypeFunction.cpp @@ -2,6 +2,8 @@ #include "Luau/ApplyTypeFunction.h" +LUAU_FASTFLAG(LuauClassTypeVarsInSubstitution) + namespace Luau { @@ -31,6 +33,8 @@ bool ApplyTypeFunction::ignoreChildren(TypeId ty) { if (get(ty)) return true; + else if (FFlag::LuauClassTypeVarsInSubstitution && get(ty)) + return true; else return false; } diff --git a/Analysis/src/Autocomplete.cpp b/Analysis/src/Autocomplete.cpp index a57a789f..8d5cc723 100644 --- a/Analysis/src/Autocomplete.cpp +++ b/Analysis/src/Autocomplete.cpp @@ -1200,7 +1200,7 @@ static bool autocompleteIfElseExpression( } } -static void autocompleteExpression(const SourceModule& sourceModule, const Module& module, const TypeChecker& typeChecker, TypeArena* typeArena, +static AutocompleteContext autocompleteExpression(const SourceModule& sourceModule, const Module& module, const TypeChecker& typeChecker, TypeArena* typeArena, const std::vector& ancestry, Position position, AutocompleteEntryMap& result) { LUAU_ASSERT(!ancestry.empty()); @@ -1213,9 +1213,9 @@ static void autocompleteExpression(const SourceModule& sourceModule, const Modul autocompleteProps(module, typeArena, *it, PropIndexType::Point, ancestry, result); } else if (autocompleteIfElseExpression(node, ancestry, position, result)) - return; + return AutocompleteContext::Keyword; else if (node->is()) - return; + return AutocompleteContext::Unknown; else { // This is inefficient. :( @@ -1260,14 +1260,16 @@ static void autocompleteExpression(const SourceModule& sourceModule, const Modul if (auto ty = findExpectedTypeAt(module, node, position)) autocompleteStringSingleton(*ty, true, result); } + + return AutocompleteContext::Expression; } -static AutocompleteEntryMap autocompleteExpression(const SourceModule& sourceModule, const Module& module, const TypeChecker& typeChecker, +static AutocompleteResult autocompleteExpression(const SourceModule& sourceModule, const Module& module, const TypeChecker& typeChecker, TypeArena* typeArena, const std::vector& ancestry, Position position) { AutocompleteEntryMap result; - autocompleteExpression(sourceModule, module, typeChecker, typeArena, ancestry, position, result); - return result; + AutocompleteContext context = autocompleteExpression(sourceModule, module, typeChecker, typeArena, ancestry, position, result); + return {result, ancestry, context}; } static std::optional getMethodContainingClass(const ModulePtr& module, AstExpr* funcExpr) @@ -1406,27 +1408,27 @@ static AutocompleteResult autocomplete(const SourceModule& sourceModule, const M if (!FFlag::LuauSelfCallAutocompleteFix3 && isString(ty)) return { - autocompleteProps(*module, typeArena, typeChecker.globalScope->bindings[AstName{"string"}].typeId, indexType, ancestry), ancestry}; + autocompleteProps(*module, typeArena, typeChecker.globalScope->bindings[AstName{"string"}].typeId, indexType, ancestry), ancestry, AutocompleteContext::Property}; else - return {autocompleteProps(*module, typeArena, ty, indexType, ancestry), ancestry}; + return {autocompleteProps(*module, typeArena, ty, indexType, ancestry), ancestry, AutocompleteContext::Property}; } else if (auto typeReference = node->as()) { if (typeReference->prefix) - return {autocompleteModuleTypes(*module, position, typeReference->prefix->value), ancestry}; + return {autocompleteModuleTypes(*module, position, typeReference->prefix->value), ancestry, AutocompleteContext::Type}; else - return {autocompleteTypeNames(*module, position, ancestry), ancestry}; + return {autocompleteTypeNames(*module, position, ancestry), ancestry, AutocompleteContext::Type}; } else if (node->is()) { - return {autocompleteTypeNames(*module, position, ancestry), ancestry}; + return {autocompleteTypeNames(*module, position, ancestry), ancestry, AutocompleteContext::Type}; } else if (AstStatLocal* statLocal = node->as()) { if (statLocal->vars.size == 1 && (!statLocal->equalsSignLocation || position < statLocal->equalsSignLocation->begin)) - return {{{"function", AutocompleteEntry{AutocompleteEntryKind::Keyword}}}, ancestry}; + return {{{"function", AutocompleteEntry{AutocompleteEntryKind::Keyword}}}, ancestry, AutocompleteContext::Unknown}; else if (statLocal->equalsSignLocation && position >= statLocal->equalsSignLocation->end) - return {autocompleteExpression(sourceModule, *module, typeChecker, typeArena, ancestry, position), ancestry}; + return autocompleteExpression(sourceModule, *module, typeChecker, typeArena, ancestry, position); else return {}; } @@ -1436,16 +1438,16 @@ static AutocompleteResult autocomplete(const SourceModule& sourceModule, const M if (!statFor->hasDo || position < statFor->doLocation.begin) { if (!statFor->from->is() && !statFor->to->is() && (!statFor->step || !statFor->step->is())) - return {{{"do", AutocompleteEntry{AutocompleteEntryKind::Keyword}}}, ancestry}; + return {{{"do", AutocompleteEntry{AutocompleteEntryKind::Keyword}}}, ancestry, AutocompleteContext::Keyword}; if (statFor->from->location.containsClosed(position) || statFor->to->location.containsClosed(position) || (statFor->step && statFor->step->location.containsClosed(position))) - return {autocompleteExpression(sourceModule, *module, typeChecker, typeArena, ancestry, position), ancestry}; + return autocompleteExpression(sourceModule, *module, typeChecker, typeArena, ancestry, position); return {}; } - return {autocompleteStatement(sourceModule, *module, ancestry, position), ancestry}; + return {autocompleteStatement(sourceModule, *module, ancestry, position), ancestry, AutocompleteContext::Statement}; } else if (AstStatForIn* statForIn = parent->as(); statForIn && (node->is() || isIdentifier(node))) @@ -1461,7 +1463,7 @@ static AutocompleteResult autocomplete(const SourceModule& sourceModule, const M return {}; } - return {{{"in", AutocompleteEntry{AutocompleteEntryKind::Keyword}}}, ancestry}; + return {{{"in", AutocompleteEntry{AutocompleteEntryKind::Keyword}}}, ancestry, AutocompleteContext::Keyword}; } if (!statForIn->hasDo || position <= statForIn->doLocation.begin) @@ -1470,10 +1472,10 @@ static AutocompleteResult autocomplete(const SourceModule& sourceModule, const M AstExpr* lastExpr = statForIn->values.data[statForIn->values.size - 1]; if (lastExpr->location.containsClosed(position)) - return {autocompleteExpression(sourceModule, *module, typeChecker, typeArena, ancestry, position), ancestry}; + return autocompleteExpression(sourceModule, *module, typeChecker, typeArena, ancestry, position); if (position > lastExpr->location.end) - return {{{"do", AutocompleteEntry{AutocompleteEntryKind::Keyword}}}, ancestry}; + return {{{"do", AutocompleteEntry{AutocompleteEntryKind::Keyword}}}, ancestry, AutocompleteContext::Keyword}; return {}; // Not sure what this means } @@ -1483,45 +1485,45 @@ static AutocompleteResult autocomplete(const SourceModule& sourceModule, const M // The AST looks a bit differently if the cursor is at a position where only the "do" keyword is allowed. // ex "for f in f do" if (!statForIn->hasDo) - return {{{"do", AutocompleteEntry{AutocompleteEntryKind::Keyword}}}, ancestry}; + return {{{"do", AutocompleteEntry{AutocompleteEntryKind::Keyword}}}, ancestry, AutocompleteContext::Keyword}; - return {autocompleteStatement(sourceModule, *module, ancestry, position), ancestry}; + return {autocompleteStatement(sourceModule, *module, ancestry, position), ancestry, AutocompleteContext::Statement}; } else if (AstStatWhile* statWhile = parent->as(); node->is() && statWhile) { if (!statWhile->hasDo && !statWhile->condition->is() && position > statWhile->condition->location.end) - return {{{"do", AutocompleteEntry{AutocompleteEntryKind::Keyword}}}, ancestry}; + return {{{"do", AutocompleteEntry{AutocompleteEntryKind::Keyword}}}, ancestry, AutocompleteContext::Keyword}; if (!statWhile->hasDo || position < statWhile->doLocation.begin) - return {autocompleteExpression(sourceModule, *module, typeChecker, typeArena, ancestry, position), ancestry}; + return autocompleteExpression(sourceModule, *module, typeChecker, typeArena, ancestry, position); if (statWhile->hasDo && position > statWhile->doLocation.end) - return {autocompleteStatement(sourceModule, *module, ancestry, position), ancestry}; + return {autocompleteStatement(sourceModule, *module, ancestry, position), ancestry, AutocompleteContext::Statement}; } else if (AstStatWhile* statWhile = extractStat(ancestry); statWhile && !statWhile->hasDo) - return {{{"do", AutocompleteEntry{AutocompleteEntryKind::Keyword}}}, ancestry}; + return {{{"do", AutocompleteEntry{AutocompleteEntryKind::Keyword}}}, ancestry, AutocompleteContext::Keyword}; else if (AstStatIf* statIf = node->as(); statIf && !statIf->elseLocation.has_value()) { return { - {{"else", AutocompleteEntry{AutocompleteEntryKind::Keyword}}, {"elseif", AutocompleteEntry{AutocompleteEntryKind::Keyword}}}, ancestry}; + {{"else", AutocompleteEntry{AutocompleteEntryKind::Keyword}}, {"elseif", AutocompleteEntry{AutocompleteEntryKind::Keyword}}}, ancestry, AutocompleteContext::Keyword}; } else if (AstStatIf* statIf = parent->as(); statIf && node->is()) { if (statIf->condition->is()) - return {autocompleteExpression(sourceModule, *module, typeChecker, typeArena, ancestry, position), ancestry}; + return autocompleteExpression(sourceModule, *module, typeChecker, typeArena, ancestry, position); else if (!statIf->thenLocation || statIf->thenLocation->containsClosed(position)) - return {{{"then", AutocompleteEntry{AutocompleteEntryKind::Keyword}}}, ancestry}; + return {{{"then", AutocompleteEntry{AutocompleteEntryKind::Keyword}}}, ancestry, AutocompleteContext::Keyword}; } else if (AstStatIf* statIf = extractStat(ancestry); statIf && (!statIf->thenLocation || statIf->thenLocation->containsClosed(position))) - return {{{"then", AutocompleteEntry{AutocompleteEntryKind::Keyword}}}, ancestry}; + return {{{"then", AutocompleteEntry{AutocompleteEntryKind::Keyword}}}, ancestry, AutocompleteContext::Keyword}; else if (AstStatRepeat* statRepeat = node->as(); statRepeat && statRepeat->condition->is()) - return {autocompleteExpression(sourceModule, *module, typeChecker, typeArena, ancestry, position), ancestry}; + return autocompleteExpression(sourceModule, *module, typeChecker, typeArena, ancestry, position); else if (AstStatRepeat* statRepeat = extractStat(ancestry); statRepeat) - return {autocompleteStatement(sourceModule, *module, ancestry, position), ancestry}; + return {autocompleteStatement(sourceModule, *module, ancestry, position), ancestry, AutocompleteContext::Statement}; else if (AstExprTable* exprTable = parent->as(); exprTable && (node->is() || node->is())) { for (const auto& [kind, key, value] : exprTable->items) @@ -1547,7 +1549,7 @@ static AutocompleteResult autocomplete(const SourceModule& sourceModule, const M if (!key) autocompleteExpression(sourceModule, *module, typeChecker, typeArena, ancestry, position, result); - return {result, ancestry}; + return {result, ancestry, AutocompleteContext::Property}; } break; @@ -1555,11 +1557,11 @@ static AutocompleteResult autocomplete(const SourceModule& sourceModule, const M } } else if (isIdentifier(node) && (parent->is() || parent->is())) - return {autocompleteStatement(sourceModule, *module, ancestry, position), ancestry}; + return {autocompleteStatement(sourceModule, *module, ancestry, position), ancestry, AutocompleteContext::Statement}; if (std::optional ret = autocompleteStringParams(sourceModule, module, ancestry, position, callback)) { - return {*ret, ancestry}; + return {*ret, ancestry, AutocompleteContext::String}; } else if (node->is()) { @@ -1585,7 +1587,7 @@ static AutocompleteResult autocomplete(const SourceModule& sourceModule, const M } } - return {result, ancestry}; + return {result, ancestry, AutocompleteContext::String}; } if (node->is()) @@ -1594,9 +1596,9 @@ static AutocompleteResult autocomplete(const SourceModule& sourceModule, const M } if (node->asExpr()) - return {autocompleteExpression(sourceModule, *module, typeChecker, typeArena, ancestry, position), ancestry}; + return autocompleteExpression(sourceModule, *module, typeChecker, typeArena, ancestry, position); else if (node->asStat()) - return {autocompleteStatement(sourceModule, *module, ancestry, position), ancestry}; + return {autocompleteStatement(sourceModule, *module, ancestry, position), ancestry, AutocompleteContext::Statement}; return {}; } diff --git a/Analysis/src/Clone.cpp b/Analysis/src/Clone.cpp index 51ad61d5..2e04b527 100644 --- a/Analysis/src/Clone.cpp +++ b/Analysis/src/Clone.cpp @@ -7,6 +7,7 @@ #include "Luau/Unifiable.h" LUAU_FASTFLAG(DebugLuauCopyBeforeNormalizing) +LUAU_FASTFLAG(LuauClonePublicInterfaceLess) LUAU_FASTINTVARIABLE(LuauTypeCloneRecursionLimit, 300) @@ -445,7 +446,7 @@ TypeFun clone(const TypeFun& typeFun, TypeArena& dest, CloneState& cloneState) return result; } -TypeId shallowClone(TypeId ty, TypeArena& dest, const TxnLog* log) +TypeId shallowClone(TypeId ty, TypeArena& dest, const TxnLog* log, bool alwaysClone) { ty = log->follow(ty); @@ -504,6 +505,15 @@ TypeId shallowClone(TypeId ty, TypeArena& dest, const TxnLog* log) PendingExpansionTypeVar clone{petv->fn, petv->typeArguments, petv->packArguments}; result = dest.addType(std::move(clone)); } + else if (const ClassTypeVar* ctv = get(ty); FFlag::LuauClonePublicInterfaceLess && ctv && alwaysClone) + { + ClassTypeVar clone{ctv->name, ctv->props, ctv->parent, ctv->metatable, ctv->tags, ctv->userData, ctv->definitionModuleName}; + result = dest.addType(std::move(clone)); + } + else if (FFlag::LuauClonePublicInterfaceLess && alwaysClone) + { + result = dest.addType(*ty); + } else return result; diff --git a/Analysis/src/ConstraintGraphBuilder.cpp b/Analysis/src/ConstraintGraphBuilder.cpp index ea7037bf..2c36d422 100644 --- a/Analysis/src/ConstraintGraphBuilder.cpp +++ b/Analysis/src/ConstraintGraphBuilder.cpp @@ -1,6 +1,8 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #include "Luau/ConstraintGraphBuilder.h" +#include "Luau/Ast.h" +#include "Luau/Constraint.h" #include "Luau/RecursionCounter.h" #include "Luau/ToString.h" @@ -14,8 +16,9 @@ namespace Luau const AstStat* getFallthrough(const AstStat* node); // TypeInfer.cpp ConstraintGraphBuilder::ConstraintGraphBuilder( - const ModuleName& moduleName, TypeArena* arena, NotNull ice, NotNull globalScope) + const ModuleName& moduleName, ModulePtr module, TypeArena* arena, NotNull ice, const ScopePtr& globalScope) : moduleName(moduleName) + , module(module) , singletonTypes(getSingletonTypes()) , arena(arena) , rootScope(nullptr) @@ -61,7 +64,7 @@ void ConstraintGraphBuilder::visit(AstStatBlock* block) { LUAU_ASSERT(scopes.empty()); LUAU_ASSERT(rootScope == nullptr); - ScopePtr scope = std::make_shared(singletonTypes.anyTypePack); + ScopePtr scope = std::make_shared(globalScope); rootScope = scope.get(); scopes.emplace_back(block->location, scope); @@ -70,11 +73,11 @@ void ConstraintGraphBuilder::visit(AstStatBlock* block) prepopulateGlobalScope(scope, block); // TODO: We should share the global scope. - rootScope->typeBindings["nil"] = TypeFun{singletonTypes.nilType}; - rootScope->typeBindings["number"] = TypeFun{singletonTypes.numberType}; - rootScope->typeBindings["string"] = TypeFun{singletonTypes.stringType}; - rootScope->typeBindings["boolean"] = TypeFun{singletonTypes.booleanType}; - rootScope->typeBindings["thread"] = TypeFun{singletonTypes.threadType}; + rootScope->privateTypeBindings["nil"] = TypeFun{singletonTypes.nilType}; + rootScope->privateTypeBindings["number"] = TypeFun{singletonTypes.numberType}; + rootScope->privateTypeBindings["string"] = TypeFun{singletonTypes.stringType}; + rootScope->privateTypeBindings["boolean"] = TypeFun{singletonTypes.booleanType}; + rootScope->privateTypeBindings["thread"] = TypeFun{singletonTypes.threadType}; visitBlockWithoutChildScope(scope, block); } @@ -99,7 +102,7 @@ void ConstraintGraphBuilder::visitBlockWithoutChildScope(const ScopePtr& scope, { if (auto alias = stat->as()) { - if (scope->typeBindings.count(alias->name.value) != 0) + if (scope->privateTypeBindings.count(alias->name.value) != 0) { auto it = aliasDefinitionLocations.find(alias->name.value); LUAU_ASSERT(it != aliasDefinitionLocations.end()); @@ -121,16 +124,16 @@ void ConstraintGraphBuilder::visitBlockWithoutChildScope(const ScopePtr& scope, for (const auto& [name, gen] : createGenerics(defnScope, alias->generics)) { initialFun.typeParams.push_back(gen); - defnScope->typeBindings[name] = TypeFun{gen.ty}; + defnScope->privateTypeBindings[name] = TypeFun{gen.ty}; } for (const auto& [name, genPack] : createGenericPacks(defnScope, alias->genericPacks)) { initialFun.typePackParams.push_back(genPack); - defnScope->typePackBindings[name] = genPack.tp; + defnScope->privateTypePackBindings[name] = genPack.tp; } - scope->typeBindings[alias->name.value] = std::move(initialFun); + scope->privateTypeBindings[alias->name.value] = std::move(initialFun); astTypeAliasDefiningScopes[alias] = defnScope; aliasDefinitionLocations[alias->name.value] = alias->location; } @@ -150,6 +153,8 @@ void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStat* stat) visit(scope, s); else if (auto s = stat->as()) visit(scope, s); + else if (auto s = stat->as()) + visit(scope, s); else if (auto f = stat->as()) visit(scope, f); else if (auto f = stat->as()) @@ -242,6 +247,15 @@ void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatFor* for_) visit(forScope, for_->body); } +void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatWhile* while_) +{ + check(scope, while_->condition); + + ScopePtr whileScope = childScope(while_->location, scope); + + visit(whileScope, while_->body); +} + void addConstraints(Constraint* constraint, NotNull scope) { scope->constraints.reserve(scope->constraints.size() + scope->constraints.size()); @@ -388,11 +402,11 @@ void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatTypeAlias* alia { // TODO: Exported type aliases - auto bindingIt = scope->typeBindings.find(alias->name.value); + auto bindingIt = scope->privateTypeBindings.find(alias->name.value); ScopePtr* defnIt = astTypeAliasDefiningScopes.find(alias); // These will be undefined if the alias was a duplicate definition, in which // case we just skip over it. - if (bindingIt == scope->typeBindings.end() || defnIt == nullptr) + if (bindingIt == scope->privateTypeBindings.end() || defnIt == nullptr) { return; } @@ -416,17 +430,152 @@ void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatDeclareGlobal* LUAU_ASSERT(global->type); TypeId globalTy = resolveType(scope, global->type); + Name globalName(global->name.value); + + module->declaredGlobals[globalName] = globalTy; scope->bindings[global->name] = Binding{globalTy, global->location}; } -void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatDeclareClass* global) +static bool isMetamethod(const Name& name) { - LUAU_ASSERT(false); // TODO: implement + return name == "__index" || name == "__newindex" || name == "__call" || name == "__concat" || name == "__unm" || name == "__add" || + name == "__sub" || name == "__mul" || name == "__div" || name == "__mod" || name == "__pow" || name == "__tostring" || + name == "__metatable" || name == "__eq" || name == "__lt" || name == "__le" || name == "__mode" || name == "__iter" || name == "__len"; +} + +void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatDeclareClass* declaredClass) +{ + std::optional superTy = std::nullopt; + if (declaredClass->superName) + { + Name superName = Name(declaredClass->superName->value); + std::optional lookupType = scope->lookupType(superName); + + if (!lookupType) + { + reportError(declaredClass->location, UnknownSymbol{superName, UnknownSymbol::Type}); + return; + } + + // We don't have generic classes, so this assertion _should_ never be hit. + LUAU_ASSERT(lookupType->typeParams.size() == 0 && lookupType->typePackParams.size() == 0); + superTy = lookupType->type; + + if (!get(follow(*superTy))) + { + reportError(declaredClass->location, + GenericError{format("Cannot use non-class type '%s' as a superclass of class '%s'", superName.c_str(), declaredClass->name.value)}); + + return; + } + } + + Name className(declaredClass->name.value); + + TypeId classTy = arena->addType(ClassTypeVar(className, {}, superTy, std::nullopt, {}, {}, moduleName)); + ClassTypeVar* ctv = getMutable(classTy); + + TypeId metaTy = arena->addType(TableTypeVar{TableState::Sealed, scope->level}); + TableTypeVar* metatable = getMutable(metaTy); + + ctv->metatable = metaTy; + + scope->exportedTypeBindings[className] = TypeFun{{}, classTy}; + + for (const AstDeclaredClassProp& prop : declaredClass->props) + { + Name propName(prop.name.value); + TypeId propTy = resolveType(scope, prop.ty); + + bool assignToMetatable = isMetamethod(propName); + + // Function types always take 'self', but this isn't reflected in the + // parsed annotation. Add it here. + if (prop.isMethod) + { + if (FunctionTypeVar* ftv = getMutable(propTy)) + { + ftv->argNames.insert(ftv->argNames.begin(), FunctionArgument{"self", {}}); + ftv->argTypes = arena->addTypePack(TypePack{{classTy}, ftv->argTypes}); + + ftv->hasSelf = true; + } + } + + if (ctv->props.count(propName) == 0) + { + if (assignToMetatable) + metatable->props[propName] = {propTy}; + else + ctv->props[propName] = {propTy}; + } + else + { + TypeId currentTy = assignToMetatable ? metatable->props[propName].type : ctv->props[propName].type; + + // We special-case this logic to keep the intersection flat; otherwise we + // would create a ton of nested intersection types. + if (const IntersectionTypeVar* itv = get(currentTy)) + { + std::vector options = itv->parts; + options.push_back(propTy); + TypeId newItv = arena->addType(IntersectionTypeVar{std::move(options)}); + + if (assignToMetatable) + metatable->props[propName] = {newItv}; + else + ctv->props[propName] = {newItv}; + } + else if (get(currentTy)) + { + TypeId intersection = arena->addType(IntersectionTypeVar{{currentTy, propTy}}); + + if (assignToMetatable) + metatable->props[propName] = {intersection}; + else + ctv->props[propName] = {intersection}; + } + else + { + reportError(declaredClass->location, GenericError{format("Cannot overload non-function class member '%s'", propName.c_str())}); + } + } + } } void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatDeclareFunction* global) { - LUAU_ASSERT(false); // TODO: implement + + std::vector> generics = createGenerics(scope, global->generics); + std::vector> genericPacks = createGenericPacks(scope, global->genericPacks); + + std::vector genericTys; + genericTys.reserve(generics.size()); + for (auto& [name, generic] : generics) + genericTys.push_back(generic.ty); + + std::vector genericTps; + genericTps.reserve(genericPacks.size()); + for (auto& [name, generic] : genericPacks) + genericTps.push_back(generic.tp); + + ScopePtr funScope = scope; + if (!generics.empty() || !genericPacks.empty()) + funScope = childScope(global->location, scope); + + TypePackId paramPack = resolveTypePack(funScope, global->params); + TypePackId retPack = resolveTypePack(funScope, global->retTypes); + TypeId fnType = arena->addType(FunctionTypeVar{funScope->level, std::move(genericTys), std::move(genericTps), paramPack, retPack}); + FunctionTypeVar* ftv = getMutable(fnType); + + ftv->argNames.reserve(global->paramNames.size); + for (const auto& el : global->paramNames) + ftv->argNames.push_back(FunctionArgument{el.first.value, el.second}); + + Name fnName(global->name.value); + + module->declaredGlobals[fnName] = fnType; + scope->bindings[global->name] = Binding{fnType, global->location}; } TypePackId ConstraintGraphBuilder::checkPack(const ScopePtr& scope, AstArray exprs) @@ -590,6 +739,8 @@ TypeId ConstraintGraphBuilder::check(const ScopePtr& scope, AstExpr* expr) result = check(scope, unary); else if (auto binary = expr->as()) result = check(scope, binary); + else if (auto typeAssert = expr->as()) + result = check(scope, typeAssert); else if (auto err = expr->as()) { // Open question: Should we traverse into this? @@ -682,6 +833,12 @@ TypeId ConstraintGraphBuilder::check(const ScopePtr& scope, AstExprBinary* binar return nullptr; } +TypeId ConstraintGraphBuilder::check(const ScopePtr& scope, AstExprTypeAssertion* typeAssert) +{ + check(scope, typeAssert->expr); + return resolveType(scope, typeAssert->annotation); +} + TypeId ConstraintGraphBuilder::checkExprTable(const ScopePtr& scope, AstExprTable* expr) { TypeId ty = arena->addType(TableTypeVar{}); @@ -765,13 +922,13 @@ ConstraintGraphBuilder::FunctionSignature ConstraintGraphBuilder::checkFunctionS for (const auto& [name, g] : genericDefinitions) { genericTypes.push_back(g.ty); - signatureScope->typeBindings[name] = TypeFun{g.ty}; + signatureScope->privateTypeBindings[name] = TypeFun{g.ty}; } for (const auto& [name, g] : genericPackDefinitions) { genericTypePacks.push_back(g.tp); - signatureScope->typePackBindings[name] = g.tp; + signatureScope->privateTypePackBindings[name] = g.tp; } } else @@ -851,7 +1008,7 @@ TypeId ConstraintGraphBuilder::resolveType(const ScopePtr& scope, AstType* ty, b // TODO: Support imported types w/ require tracing. LUAU_ASSERT(!ref->prefix); - std::optional alias = scope->lookupTypeBinding(ref->name.value); + std::optional alias = scope->lookupType(ref->name.value); if (alias.has_value()) { @@ -949,13 +1106,13 @@ TypeId ConstraintGraphBuilder::resolveType(const ScopePtr& scope, AstType* ty, b for (const auto& [name, g] : genericDefinitions) { genericTypes.push_back(g.ty); - signatureScope->typeBindings[name] = TypeFun{g.ty}; + signatureScope->privateTypeBindings[name] = TypeFun{g.ty}; } for (const auto& [name, g] : genericPackDefinitions) { genericTypePacks.push_back(g.tp); - signatureScope->typePackBindings[name] = g.tp; + signatureScope->privateTypePackBindings[name] = g.tp; } } else @@ -1059,7 +1216,7 @@ TypePackId ConstraintGraphBuilder::resolveTypePack(const ScopePtr& scope, AstTyp } else if (auto gen = tp->as()) { - if (std::optional lookup = scope->lookupTypePackBinding(gen->genericName.value)) + if (std::optional lookup = scope->lookupPack(gen->genericName.value)) { result = *lookup; } diff --git a/Analysis/src/ConstraintSolver.cpp b/Analysis/src/ConstraintSolver.cpp index 0898f9aa..d8105ce3 100644 --- a/Analysis/src/ConstraintSolver.cpp +++ b/Analysis/src/ConstraintSolver.cpp @@ -484,6 +484,10 @@ bool ConstraintSolver::tryDispatch(const NameConstraint& c, NotNullpersistent) + return true; + if (TableTypeVar* ttv = getMutable(target)) ttv->name = c.name; else if (MetatableTypeVar* mtv = getMutable(target)) @@ -637,6 +641,10 @@ bool ConstraintSolver::tryDispatch(const TypeAliasExpansionConstraint& c, NotNul TypeId instantiated = *maybeInstantiated; TypeId target = follow(instantiated); + + if (target->persistent) + return true; + // Type function application will happily give us the exact same type if // there are e.g. generic saturatedTypeArguments that go unused. bool needsClone = follow(petv->fn.type) == target; diff --git a/Analysis/src/Frontend.cpp b/Analysis/src/Frontend.cpp index fe65853d..c450297f 100644 --- a/Analysis/src/Frontend.cpp +++ b/Analysis/src/Frontend.cpp @@ -818,21 +818,21 @@ const SourceModule* Frontend::getSourceModule(const ModuleName& moduleName) cons return const_cast(this)->getSourceModule(moduleName); } -NotNull Frontend::getGlobalScope() +ScopePtr Frontend::getGlobalScope() { if (!globalScope) { globalScope = typeChecker.globalScope; } - return NotNull(globalScope.get()); + return globalScope; } ModulePtr Frontend::check(const SourceModule& sourceModule, Mode mode, const ScopePtr& environmentScope) { ModulePtr result = std::make_shared(); - ConstraintGraphBuilder cgb{sourceModule.name, &result->internalTypes, NotNull(&iceHandler), getGlobalScope()}; + ConstraintGraphBuilder cgb{sourceModule.name, result, &result->internalTypes, NotNull(&iceHandler), getGlobalScope()}; cgb.visit(sourceModule.root); result->errors = std::move(cgb.errors); diff --git a/Analysis/src/Instantiation.cpp b/Analysis/src/Instantiation.cpp index 1a6013af..e98ab185 100644 --- a/Analysis/src/Instantiation.cpp +++ b/Analysis/src/Instantiation.cpp @@ -82,6 +82,8 @@ bool ReplaceGenerics::ignoreChildren(TypeId ty) // whenever we quantify, so the vectors overlap if and only if they are equal. return (!generics.empty() || !genericPacks.empty()) && (ftv->generics == generics) && (ftv->genericPacks == genericPacks); } + else if (FFlag::LuauClassTypeVarsInSubstitution && get(ty)) + return true; else { return false; diff --git a/Analysis/src/Linter.cpp b/Analysis/src/Linter.cpp index 9fce79a2..2d058376 100644 --- a/Analysis/src/Linter.cpp +++ b/Analysis/src/Linter.cpp @@ -14,6 +14,7 @@ LUAU_FASTINTVARIABLE(LuauSuggestionDistance, 4) LUAU_FASTFLAGVARIABLE(LuauLintGlobalNeverReadBeforeWritten, false) +LUAU_FASTFLAGVARIABLE(LuauLintComparisonPrecedence, false) namespace Luau { @@ -49,6 +50,7 @@ static const char* kWarningNames[] = { "MisleadingAndOr", "CommentDirective", "IntegerParsing", + "ComparisonPrecedence", }; // clang-format on @@ -2629,6 +2631,65 @@ private: } }; +class LintComparisonPrecedence : AstVisitor +{ +public: + LUAU_NOINLINE static void process(LintContext& context) + { + LintComparisonPrecedence pass; + pass.context = &context; + + context.root->visit(&pass); + } + +private: + LintContext* context; + + bool isComparison(AstExprBinary::Op op) + { + return op == AstExprBinary::CompareNe || op == AstExprBinary::CompareEq || op == AstExprBinary::CompareLt || op == AstExprBinary::CompareLe || + op == AstExprBinary::CompareGt || op == AstExprBinary::CompareGe; + } + + bool isNot(AstExpr* node) + { + AstExprUnary* expr = node->as(); + + return expr && expr->op == AstExprUnary::Not; + } + + bool visit(AstExprBinary* node) override + { + if (!isComparison(node->op)) + return true; + + // not X == Y; we silence this for not X == not Y as it's likely an intentional boolean comparison + if (isNot(node->left) && !isNot(node->right)) + { + std::string op = toString(node->op); + + if (node->op == AstExprBinary::CompareEq || node->op == AstExprBinary::CompareNe) + emitWarning(*context, LintWarning::Code_ComparisonPrecedence, node->location, + "not X %s Y is equivalent to (not X) %s Y; consider using X %s Y, or wrap one of the expressions in parentheses to silence", + op.c_str(), op.c_str(), node->op == AstExprBinary::CompareEq ? "~=" : "=="); + else + emitWarning(*context, LintWarning::Code_ComparisonPrecedence, node->location, + "not X %s Y is equivalent to (not X) %s Y; wrap one of the expressions in parentheses to silence", op.c_str(), op.c_str()); + } + else if (AstExprBinary* left = node->left->as(); left && isComparison(left->op)) + { + std::string lop = toString(left->op); + std::string rop = toString(node->op); + + emitWarning(*context, LintWarning::Code_ComparisonPrecedence, node->location, + "X %s Y %s Z is equivalent to (X %s Y) %s Z; wrap one of the expressions in parentheses to silence", lop.c_str(), rop.c_str(), + lop.c_str(), rop.c_str()); + } + + return true; + } +}; + static void fillBuiltinGlobals(LintContext& context, const AstNameTable& names, const ScopePtr& env) { ScopePtr current = env; @@ -2853,6 +2914,9 @@ std::vector lint(AstStat* root, const AstNameTable& names, const Sc if (context.warningEnabled(LintWarning::Code_IntegerParsing)) LintIntegerParsing::process(context); + if (context.warningEnabled(LintWarning::Code_ComparisonPrecedence) && FFlag::LuauLintComparisonPrecedence) + LintComparisonPrecedence::process(context); + std::sort(context.result.begin(), context.result.end(), WarningComparator()); return context.result; diff --git a/Analysis/src/Module.cpp b/Analysis/src/Module.cpp index 2b46da87..4e6b2585 100644 --- a/Analysis/src/Module.cpp +++ b/Analysis/src/Module.cpp @@ -17,6 +17,10 @@ LUAU_FASTFLAG(LuauLowerBoundsCalculation); LUAU_FASTFLAG(DebugLuauDeferredConstraintResolution); LUAU_FASTFLAGVARIABLE(LuauForceExportSurfacesToBeNormal, false); +LUAU_FASTFLAGVARIABLE(LuauClonePublicInterfaceLess, false); +LUAU_FASTFLAG(LuauSubstitutionReentrant); +LUAU_FASTFLAG(LuauClassTypeVarsInSubstitution); +LUAU_FASTFLAG(LuauSubstitutionFixMissingFields); namespace Luau { @@ -86,6 +90,118 @@ struct ForceNormal : TypeVarOnceVisitor } }; +struct ClonePublicInterface : Substitution +{ + NotNull module; + + ClonePublicInterface(const TxnLog* log, Module* module) + : Substitution(log, &module->interfaceTypes) + , module(module) + { + LUAU_ASSERT(module); + } + + bool isDirty(TypeId ty) override + { + if (ty->owningArena == &module->internalTypes) + return true; + + if (const FunctionTypeVar* ftv = get(ty)) + return ftv->level.level != 0; + if (const TableTypeVar* ttv = get(ty)) + return ttv->level.level != 0; + return false; + } + + bool isDirty(TypePackId tp) override + { + return tp->owningArena == &module->internalTypes; + } + + TypeId clean(TypeId ty) override + { + TypeId result = clone(ty); + + if (FunctionTypeVar* ftv = getMutable(result)) + ftv->level = TypeLevel{0, 0}; + else if (TableTypeVar* ttv = getMutable(result)) + ttv->level = TypeLevel{0, 0}; + + return result; + } + + TypePackId clean(TypePackId tp) override + { + return clone(tp); + } + + TypeId cloneType(TypeId ty) + { + LUAU_ASSERT(FFlag::LuauSubstitutionReentrant && FFlag::LuauSubstitutionFixMissingFields); + + std::optional result = substitute(ty); + if (result) + { + return *result; + } + else + { + module->errors.push_back(TypeError{module->scopes[0].first, UnificationTooComplex{}}); + return getSingletonTypes().errorRecoveryType(); + } + } + + TypePackId cloneTypePack(TypePackId tp) + { + LUAU_ASSERT(FFlag::LuauSubstitutionReentrant && FFlag::LuauSubstitutionFixMissingFields); + + std::optional result = substitute(tp); + if (result) + { + return *result; + } + else + { + module->errors.push_back(TypeError{module->scopes[0].first, UnificationTooComplex{}}); + return getSingletonTypes().errorRecoveryTypePack(); + } + } + + TypeFun cloneTypeFun(const TypeFun& tf) + { + LUAU_ASSERT(FFlag::LuauSubstitutionReentrant && FFlag::LuauSubstitutionFixMissingFields); + + std::vector typeParams; + std::vector typePackParams; + + for (GenericTypeDefinition typeParam : tf.typeParams) + { + TypeId ty = cloneType(typeParam.ty); + std::optional defaultValue; + + if (typeParam.defaultValue) + defaultValue = cloneType(*typeParam.defaultValue); + + typeParams.push_back(GenericTypeDefinition{ty, defaultValue}); + } + + for (GenericTypePackDefinition typePackParam : tf.typePackParams) + { + TypePackId tp = cloneTypePack(typePackParam.tp); + std::optional defaultValue; + + if (typePackParam.defaultValue) + defaultValue = cloneTypePack(*typePackParam.defaultValue); + + typePackParams.push_back(GenericTypePackDefinition{tp, defaultValue}); + } + + TypeId type = cloneType(tf.type); + + return TypeFun{typeParams, typePackParams, type}; + } +}; + Module::~Module() { unfreeze(interfaceTypes); @@ -106,12 +222,21 @@ void Module::clonePublicInterface(InternalErrorReporter& ice) std::unordered_map* exportedTypeBindings = FFlag::DebugLuauDeferredConstraintResolution ? nullptr : &moduleScope->exportedTypeBindings; - returnType = clone(returnType, interfaceTypes, cloneState); + TxnLog log; + ClonePublicInterface clonePublicInterface{&log, this}; + + if (FFlag::LuauClonePublicInterfaceLess) + returnType = clonePublicInterface.cloneTypePack(returnType); + else + returnType = clone(returnType, interfaceTypes, cloneState); moduleScope->returnType = returnType; if (varargPack) { - varargPack = clone(*varargPack, interfaceTypes, cloneState); + if (FFlag::LuauClonePublicInterfaceLess) + varargPack = clonePublicInterface.cloneTypePack(*varargPack); + else + varargPack = clone(*varargPack, interfaceTypes, cloneState); moduleScope->varargPack = varargPack; } @@ -134,7 +259,10 @@ void Module::clonePublicInterface(InternalErrorReporter& ice) { for (auto& [name, tf] : *exportedTypeBindings) { - tf = clone(tf, interfaceTypes, cloneState); + if (FFlag::LuauClonePublicInterfaceLess) + tf = clonePublicInterface.cloneTypeFun(tf); + else + tf = clone(tf, interfaceTypes, cloneState); if (FFlag::LuauLowerBoundsCalculation) { normalize(tf.type, interfaceTypes, ice); @@ -168,7 +296,10 @@ void Module::clonePublicInterface(InternalErrorReporter& ice) for (auto& [name, ty] : declaredGlobals) { - ty = clone(ty, interfaceTypes, cloneState); + if (FFlag::LuauClonePublicInterfaceLess) + ty = clonePublicInterface.cloneType(ty); + else + ty = clone(ty, interfaceTypes, cloneState); if (FFlag::LuauLowerBoundsCalculation) { normalize(ty, interfaceTypes, ice); diff --git a/Analysis/src/Normalize.cpp b/Analysis/src/Normalize.cpp index 9ae3b404..33f369a7 100644 --- a/Analysis/src/Normalize.cpp +++ b/Analysis/src/Normalize.cpp @@ -162,7 +162,8 @@ struct Normalize final : TypeVarVisitor // It should never be the case that this TypeVar is normal, but is bound to a non-normal type, except in nontrivial cases. LUAU_ASSERT(!ty->normal || ty->normal == btv.boundTo->normal); - asMutable(ty)->normal = btv.boundTo->normal; + if (!ty->normal) + asMutable(ty)->normal = btv.boundTo->normal; return !ty->normal; } diff --git a/Analysis/src/Quantify.cpp b/Analysis/src/Quantify.cpp index 03049cca..ce4afe22 100644 --- a/Analysis/src/Quantify.cpp +++ b/Analysis/src/Quantify.cpp @@ -5,11 +5,13 @@ #include "Luau/Scope.h" #include "Luau/Substitution.h" #include "Luau/TxnLog.h" +#include "Luau/TypeVar.h" #include "Luau/VisitTypeVar.h" LUAU_FASTFLAG(DebugLuauSharedSelf) LUAU_FASTFLAG(DebugLuauDeferredConstraintResolution); LUAU_FASTFLAGVARIABLE(LuauQuantifyConstrained, false) +LUAU_FASTFLAG(LuauClassTypeVarsInSubstitution) namespace Luau { @@ -297,6 +299,9 @@ struct PureQuantifier : Substitution bool ignoreChildren(TypeId ty) override { + if (FFlag::LuauClassTypeVarsInSubstitution && get(ty)) + return true; + return ty->persistent; } bool ignoreChildren(TypePackId ty) override diff --git a/Analysis/src/Scope.cpp b/Analysis/src/Scope.cpp index bee16908..c129b973 100644 --- a/Analysis/src/Scope.cpp +++ b/Analysis/src/Scope.cpp @@ -122,34 +122,4 @@ std::optional Scope::lookup(Symbol sym) } } -std::optional Scope::lookupTypeBinding(const Name& name) -{ - Scope* s = this; - while (s) - { - auto it = s->typeBindings.find(name); - if (it != s->typeBindings.end()) - return it->second; - - s = s->parent.get(); - } - - return std::nullopt; -} - -std::optional Scope::lookupTypePackBinding(const Name& name) -{ - Scope* s = this; - while (s) - { - auto it = s->typePackBindings.find(name); - if (it != s->typePackBindings.end()) - return it->second; - - s = s->parent.get(); - } - - return std::nullopt; -} - } // namespace Luau diff --git a/Analysis/src/Substitution.cpp b/Analysis/src/Substitution.cpp index 148c9ee2..0beeb58c 100644 --- a/Analysis/src/Substitution.cpp +++ b/Analysis/src/Substitution.cpp @@ -8,9 +8,9 @@ #include #include -LUAU_FASTFLAGVARIABLE(LuauAnyificationMustClone, false) LUAU_FASTFLAGVARIABLE(LuauSubstitutionFixMissingFields, false) LUAU_FASTFLAG(LuauLowerBoundsCalculation) +LUAU_FASTFLAG(LuauClonePublicInterfaceLess) LUAU_FASTINTVARIABLE(LuauTarjanChildLimit, 10000) LUAU_FASTFLAGVARIABLE(LuauClassTypeVarsInSubstitution, false) LUAU_FASTFLAG(LuauUnknownAndNeverType) @@ -472,7 +472,7 @@ std::optional Substitution::substitute(TypePackId tp) TypeId Substitution::clone(TypeId ty) { - return shallowClone(ty, *arena, log); + return shallowClone(ty, *arena, log, /* alwaysClone */ FFlag::LuauClonePublicInterfaceLess); } TypePackId Substitution::clone(TypePackId tp) @@ -497,6 +497,10 @@ TypePackId Substitution::clone(TypePackId tp) clone.hidden = vtp->hidden; return addTypePack(std::move(clone)); } + else if (FFlag::LuauClonePublicInterfaceLess) + { + return addTypePack(*tp); + } else return tp; } @@ -557,7 +561,7 @@ void Substitution::replaceChildren(TypeId ty) if (ignoreChildren(ty)) return; - if (FFlag::LuauAnyificationMustClone && ty->owningArena != arena) + if (ty->owningArena != arena) return; if (FunctionTypeVar* ftv = getMutable(ty)) @@ -638,7 +642,7 @@ void Substitution::replaceChildren(TypePackId tp) if (ignoreChildren(tp)) return; - if (FFlag::LuauAnyificationMustClone && tp->owningArena != arena) + if (tp->owningArena != arena) return; if (TypePack* tpp = getMutable(tp)) diff --git a/Analysis/src/TypeChecker2.cpp b/Analysis/src/TypeChecker2.cpp index 53b069cf..9d97e8d9 100644 --- a/Analysis/src/TypeChecker2.cpp +++ b/Analysis/src/TypeChecker2.cpp @@ -10,6 +10,7 @@ #include "Luau/Normalize.h" #include "Luau/TxnLog.h" #include "Luau/TypeUtils.h" +#include "Luau/TypeVar.h" #include "Luau/Unifier.h" #include "Luau/ToString.h" @@ -282,18 +283,14 @@ struct TypeChecker2 : public AstVisitor // leftType must have a property called indexName->index - std::optional t = findTablePropertyRespectingMeta(module->errors, leftType, indexName->index.value, indexName->location); - if (t) + std::optional ty = getIndexTypeFromType(module->getModuleScope(), leftType, indexName->index.value, indexName->location, /* addErrors */ true); + if (ty) { - if (!isSubtype(resultType, *t, ice)) + if (!isSubtype(resultType, *ty, ice)) { - reportError(TypeMismatch{resultType, *t}, indexName->location); + reportError(TypeMismatch{resultType, *ty}, indexName->location); } } - else - { - reportError(UnknownProperty{leftType, indexName->index.value}, indexName->location); - } return true; } @@ -324,6 +321,22 @@ struct TypeChecker2 : public AstVisitor return true; } + bool visit(AstExprTypeAssertion* expr) override + { + TypeId annotationType = lookupAnnotation(expr->annotation); + TypeId computedType = lookupType(expr->expr); + + // Note: As an optimization, we try 'number <: number | string' first, as that is the more likely case. + if (isSubtype(annotationType, computedType, ice)) + return true; + + if (isSubtype(computedType, annotationType, ice)) + return true; + + reportError(TypesAreUnrelated{computedType, annotationType}, expr->location); + return true; + } + /** Extract a TypeId for the first type of the provided pack. * * Note that this may require modifying some types. I hope this doesn't cause problems! @@ -374,7 +387,7 @@ struct TypeChecker2 : public AstVisitor // TODO: Imported types - std::optional alias = scope->lookupTypeBinding(ty->name.value); + std::optional alias = scope->lookupType(ty->name.value); if (alias.has_value()) { @@ -473,7 +486,7 @@ struct TypeChecker2 : public AstVisitor } else { - if (scope->lookupTypePackBinding(ty->name.value)) + if (scope->lookupPack(ty->name.value)) { reportError( SwappedGenericTypeParameter{ @@ -501,10 +514,10 @@ struct TypeChecker2 : public AstVisitor Scope* scope = findInnermostScope(tp->location); LUAU_ASSERT(scope); - std::optional alias = scope->lookupTypePackBinding(tp->genericName.value); + std::optional alias = scope->lookupPack(tp->genericName.value); if (!alias.has_value()) { - if (scope->lookupTypeBinding(tp->genericName.value)) + if (scope->lookupType(tp->genericName.value)) { reportError( SwappedGenericTypeParameter{ @@ -531,6 +544,142 @@ struct TypeChecker2 : public AstVisitor { module->errors.emplace_back(std::move(e)); } + + std::optional getIndexTypeFromType( + const ScopePtr& scope, TypeId type, const Name& name, const Location& location, bool addErrors) + { + type = follow(type); + + if (get(type) || get(type) || get(type)) + return type; + + if (auto f = get(type)) + *asMutable(type) = TableTypeVar{TableState::Free, f->level}; + + if (isString(type)) + { + std::optional mtIndex = Luau::findMetatableEntry(module->errors, singletonTypes.stringType, "__index", location); + LUAU_ASSERT(mtIndex); + type = *mtIndex; + } + + if (TableTypeVar* tableType = getMutableTableType(type)) + { + + return findTablePropertyRespectingMeta(module->errors, type, name, location); + } + else if (const ClassTypeVar* cls = get(type)) + { + const Property* prop = lookupClassProp(cls, name); + if (prop) + return prop->type; + } + else if (const UnionTypeVar* utv = get(type)) + { + std::vector goodOptions; + std::vector badOptions; + + for (TypeId t : utv) + { + // TODO: we should probably limit recursion here? + // RecursionLimiter _rl(&recursionCount, FInt::LuauTypeInferRecursionLimit); + + // Not needed when we normalize types. + if (get(follow(t))) + return t; + + if (std::optional ty = getIndexTypeFromType(scope, t, name, location, /* addErrors= */ false)) + goodOptions.push_back(*ty); + else + badOptions.push_back(t); + } + + if (!badOptions.empty()) + { + if (addErrors) + { + if (goodOptions.empty()) + reportError(UnknownProperty{type, name}, location); + else + reportError(MissingUnionProperty{type, badOptions, name}, location); + } + return std::nullopt; + } + + std::vector result = reduceUnion(goodOptions); + if (result.empty()) + return singletonTypes.neverType; + + if (result.size() == 1) + return result[0]; + + return module->internalTypes.addType(UnionTypeVar{std::move(result)}); + } + else if (const IntersectionTypeVar* itv = get(type)) + { + std::vector parts; + + for (TypeId t : itv->parts) + { + // TODO: we should probably limit recursion here? + // RecursionLimiter _rl(&recursionCount, FInt::LuauTypeInferRecursionLimit); + + if (std::optional ty = getIndexTypeFromType(scope, t, name, location, /* addErrors= */ false)) + parts.push_back(*ty); + } + + // If no parts of the intersection had the property we looked up for, it never existed at all. + if (parts.empty()) + { + if (addErrors) + reportError(UnknownProperty{type, name}, location); + return std::nullopt; + } + + if (parts.size() == 1) + return parts[0]; + + return module->internalTypes.addType(IntersectionTypeVar{std::move(parts)}); // Not at all correct. + } + + if (addErrors) + reportError(UnknownProperty{type, name}, location); + + return std::nullopt; + } + + std::vector reduceUnion(const std::vector& types) + { + std::vector result; + for (TypeId t : types) + { + t = follow(t); + if (get(t)) + continue; + + if (get(t) || get(t)) + return {t}; + + if (const UnionTypeVar* utv = get(t)) + { + for (TypeId ty : utv) + { + ty = follow(ty); + if (get(ty)) + continue; + if (get(ty) || get(ty)) + return {ty}; + + if (result.end() == std::find(result.begin(), result.end(), ty)) + result.push_back(ty); + } + } + else if (std::find(result.begin(), result.end(), t) == result.end()) + result.push_back(t); + } + + return result; + } }; void check(const SourceModule& sourceModule, Module* module) diff --git a/Analysis/src/TypeInfer.cpp b/Analysis/src/TypeInfer.cpp index bdda195c..7ab23360 100644 --- a/Analysis/src/TypeInfer.cpp +++ b/Analysis/src/TypeInfer.cpp @@ -33,7 +33,6 @@ LUAU_FASTINTVARIABLE(LuauVisitRecursionLimit, 500) LUAU_FASTFLAG(LuauKnowsTheDataModel3) LUAU_FASTFLAG(LuauAutocompleteDynamicLimits) LUAU_FASTFLAGVARIABLE(LuauExpectedTableUnionIndexerType, false) -LUAU_FASTFLAGVARIABLE(LuauIndexSilenceErrors, false) LUAU_FASTFLAGVARIABLE(LuauLowerBoundsCalculation, false) LUAU_FASTFLAGVARIABLE(DebugLuauFreezeDuringUnification, false) LUAU_FASTFLAGVARIABLE(LuauSelfCallAutocompleteFix3, false) @@ -830,6 +829,14 @@ struct Demoter : Substitution return get(tp); } + bool ignoreChildren(TypeId ty) override + { + if (FFlag::LuauClassTypeVarsInSubstitution && get(ty)) + return true; + + return false; + } + TypeId clean(TypeId ty) override { auto ftv = get(ty); @@ -1925,7 +1932,7 @@ std::optional TypeChecker::findTablePropertyRespectingMeta(TypeId lhsTyp { ErrorVec errors; auto result = Luau::findTablePropertyRespectingMeta(errors, lhsType, name, location); - if (!FFlag::LuauIndexSilenceErrors || addErrors) + if (addErrors) reportErrors(errors); return result; } @@ -1934,7 +1941,7 @@ std::optional TypeChecker::findMetatableEntry(TypeId type, std::string e { ErrorVec errors; auto result = Luau::findMetatableEntry(errors, type, entry, location); - if (!FFlag::LuauIndexSilenceErrors || addErrors) + if (addErrors) reportErrors(errors); return result; } @@ -1946,7 +1953,7 @@ std::optional TypeChecker::getIndexTypeFromType( std::optional result = getIndexTypeFromTypeImpl(scope, type, name, location, addErrors); - if (FFlag::LuauIndexSilenceErrors && !addErrors) + if (!addErrors) LUAU_ASSERT(errorCount == currentModule->errors.size()); return result; diff --git a/Analysis/src/Unifier.cpp b/Analysis/src/Unifier.cpp index e099817f..a0f4725d 100644 --- a/Analysis/src/Unifier.cpp +++ b/Analysis/src/Unifier.cpp @@ -22,6 +22,7 @@ LUAU_FASTFLAG(LuauErrorRecoveryType); LUAU_FASTFLAG(LuauUnknownAndNeverType) LUAU_FASTFLAG(LuauQuantifyConstrained) LUAU_FASTFLAGVARIABLE(LuauScalarShapeSubtyping, false) +LUAU_FASTFLAG(LuauClassTypeVarsInSubstitution) namespace Luau { @@ -273,6 +274,9 @@ TypePackId Widen::clean(TypePackId) bool Widen::ignoreChildren(TypeId ty) { + if (FFlag::LuauClassTypeVarsInSubstitution && get(ty)) + return true; + return !log->is(ty); } @@ -370,25 +374,14 @@ void Unifier::tryUnify_(TypeId subTy, TypeId superTy, bool isFunctionCall, bool if (superFree && subFree && superFree->level.subsumes(subFree->level)) { - occursCheck(subTy, superTy); - - // The occurrence check might have caused superTy no longer to be a free type - bool occursFailed = bool(log.getMutable(subTy)); - - if (!occursFailed) - { + if (!occursCheck(subTy, superTy)) log.replace(subTy, BoundTypeVar(superTy)); - } return; } else if (superFree && subFree) { - occursCheck(superTy, subTy); - - bool occursFailed = bool(log.getMutable(superTy)); - - if (!occursFailed) + if (!occursCheck(superTy, subTy)) { if (superFree->level.subsumes(subFree->level)) { @@ -402,24 +395,18 @@ void Unifier::tryUnify_(TypeId subTy, TypeId superTy, bool isFunctionCall, bool } else if (superFree) { - TypeLevel superLevel = superFree->level; - - occursCheck(superTy, subTy); - bool occursFailed = bool(log.getMutable(superTy)); - // Unification can't change the level of a generic. auto subGeneric = log.getMutable(subTy); - if (subGeneric && !subGeneric->level.subsumes(superLevel)) + if (subGeneric && !subGeneric->level.subsumes(superFree->level)) { // TODO: a more informative error message? CLI-39912 reportError(TypeError{location, GenericError{"Generic subtype escaping scope"}}); return; } - // The occurrence check might have caused superTy no longer to be a free type - if (!occursFailed) + if (!occursCheck(superTy, subTy)) { - promoteTypeLevels(log, types, superLevel, subTy); + promoteTypeLevels(log, types, superFree->level, subTy); Widen widen{types}; log.replace(superTy, BoundTypeVar(widen(subTy))); @@ -437,11 +424,6 @@ void Unifier::tryUnify_(TypeId subTy, TypeId superTy, bool isFunctionCall, bool return; } - TypeLevel subLevel = subFree->level; - - occursCheck(subTy, superTy); - bool occursFailed = bool(log.getMutable(subTy)); - // Unification can't change the level of a generic. auto superGeneric = log.getMutable(superTy); if (superGeneric && !superGeneric->level.subsumes(subFree->level)) @@ -451,9 +433,9 @@ void Unifier::tryUnify_(TypeId subTy, TypeId superTy, bool isFunctionCall, bool return; } - if (!occursFailed) + if (!occursCheck(subTy, superTy)) { - promoteTypeLevels(log, types, subLevel, superTy); + promoteTypeLevels(log, types, subFree->level, superTy); log.replace(subTy, BoundTypeVar(superTy)); } @@ -1033,9 +1015,7 @@ void Unifier::tryUnify_(TypePackId subTp, TypePackId superTp, bool isFunctionCal if (log.getMutable(superTp)) { - occursCheck(superTp, subTp); - - if (!log.getMutable(superTp)) + if (!occursCheck(superTp, subTp)) { Widen widen{types}; log.replace(superTp, Unifiable::Bound(widen(subTp))); @@ -1043,9 +1023,7 @@ void Unifier::tryUnify_(TypePackId subTp, TypePackId superTp, bool isFunctionCal } else if (log.getMutable(subTp)) { - occursCheck(subTp, superTp); - - if (!log.getMutable(subTp)) + if (!occursCheck(subTp, superTp)) { log.replace(subTp, Unifiable::Bound(superTp)); } @@ -2106,8 +2084,8 @@ void Unifier::unifyLowerBound(TypePackId subTy, TypePackId superTy, TypeLevel de { TypePackId tailPack = follow(*t); - if (log.get(tailPack)) - occursCheck(tailPack, subTy); + if (log.get(tailPack) && occursCheck(tailPack, subTy)) + return; FreeTypePack* freeTailPack = log.getMutable(tailPack); if (!freeTailPack) @@ -2180,32 +2158,35 @@ void Unifier::unifyLowerBound(TypePackId subTy, TypePackId superTy, TypeLevel de } } -void Unifier::occursCheck(TypeId needle, TypeId haystack) +bool Unifier::occursCheck(TypeId needle, TypeId haystack) { sharedState.tempSeenTy.clear(); return occursCheck(sharedState.tempSeenTy, needle, haystack); } -void Unifier::occursCheck(DenseHashSet& seen, TypeId needle, TypeId haystack) +bool Unifier::occursCheck(DenseHashSet& seen, TypeId needle, TypeId haystack) { RecursionLimiter _ra(&sharedState.counters.recursionCount, FFlag::LuauAutocompleteDynamicLimits ? sharedState.counters.recursionLimit : FInt::LuauTypeInferRecursionLimit); + bool occurrence = false; + auto check = [&](TypeId tv) { - occursCheck(seen, needle, tv); + if (occursCheck(seen, needle, tv)) + occurrence = true; }; needle = log.follow(needle); haystack = log.follow(haystack); if (seen.find(haystack)) - return; + return false; seen.insert(haystack); if (log.getMutable(needle)) - return; + return false; if (!log.getMutable(needle)) ice("Expected needle to be free"); @@ -2215,11 +2196,11 @@ void Unifier::occursCheck(DenseHashSet& seen, TypeId needle, TypeId hays reportError(TypeError{location, OccursCheckFailed{}}); log.replace(needle, *getSingletonTypes().errorRecoveryType()); - return; + return true; } if (log.getMutable(haystack)) - return; + return false; else if (auto a = log.getMutable(haystack)) { for (TypeId ty : a->options) @@ -2235,27 +2216,29 @@ void Unifier::occursCheck(DenseHashSet& seen, TypeId needle, TypeId hays for (TypeId ty : a->parts) check(ty); } + + return occurrence; } -void Unifier::occursCheck(TypePackId needle, TypePackId haystack) +bool Unifier::occursCheck(TypePackId needle, TypePackId haystack) { sharedState.tempSeenTp.clear(); return occursCheck(sharedState.tempSeenTp, needle, haystack); } -void Unifier::occursCheck(DenseHashSet& seen, TypePackId needle, TypePackId haystack) +bool Unifier::occursCheck(DenseHashSet& seen, TypePackId needle, TypePackId haystack) { needle = log.follow(needle); haystack = log.follow(haystack); if (seen.find(haystack)) - return; + return false; seen.insert(haystack); if (log.getMutable(needle)) - return; + return false; if (!log.getMutable(needle)) ice("Expected needle pack to be free"); @@ -2270,7 +2253,7 @@ void Unifier::occursCheck(DenseHashSet& seen, TypePackId needle, Typ reportError(TypeError{location, OccursCheckFailed{}}); log.replace(needle, *getSingletonTypes().errorRecoveryTypePack()); - return; + return true; } if (auto a = get(haystack); a && a->tail) @@ -2281,6 +2264,8 @@ void Unifier::occursCheck(DenseHashSet& seen, TypePackId needle, Typ break; } + + return false; } Unifier Unifier::makeChildUnifier() diff --git a/CLI/Reduce.cpp b/CLI/Reduce.cpp new file mode 100644 index 00000000..d24c9874 --- /dev/null +++ b/CLI/Reduce.cpp @@ -0,0 +1,511 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details + +#include "Luau/Ast.h" +#include "Luau/Common.h" +#include "Luau/Parser.h" +#include "Luau/Transpiler.h" + +#include "FileUtils.h" + +#include +#include +#include +#include +#include + +#define VERBOSE 0 // 1 - print out commandline invocations. 2 - print out stdout + +#ifdef _WIN32 + +const auto popen = &_popen; +const auto pclose = &_pclose; + +#endif + +using namespace Luau; + +enum class TestResult +{ + BugFound, // We encountered the bug we are trying to isolate + NoBug, // We did not encounter the bug we are trying to isolate +}; + +struct Enqueuer : public AstVisitor +{ + std::queue* queue; + + explicit Enqueuer(std::queue* queue) + : queue(queue) + { + LUAU_ASSERT(queue); + } + + bool visit(AstStatBlock* block) override + { + queue->push(block); + return false; + } +}; + +struct Reducer +{ + Allocator allocator; + AstNameTable nameTable{allocator}; + ParseOptions parseOptions; + + ParseResult parseResult; + AstStatBlock* root; + + std::string tempScriptName; + + std::string appName; + std::vector appArgs; + std::string_view searchText; + + Reducer() + { + parseOptions.captureComments = true; + } + + std::string readLine(FILE* f) + { + std::string line = ""; + char buffer[256]; + while (fgets(buffer, sizeof(buffer), f)) + { + auto len = strlen(buffer); + line += std::string(buffer, len); + if (buffer[len - 1] == '\n') + break; + } + + return line; + } + + void writeTempScript(bool minify = false) + { + std::string source = transpileWithTypes(*root); + + if (minify) + { + size_t pos = 0; + do + { + pos = source.find("\n\n", pos); + if (pos == std::string::npos) + break; + + source.erase(pos, 1); + } while (true); + } + + FILE* f = fopen(tempScriptName.c_str(), "w"); + if (!f) + { + printf("Unable to open temp script to %s\n", tempScriptName.c_str()); + exit(2); + } + + for (const HotComment& comment : parseResult.hotcomments) + fprintf(f, "--!%s\n", comment.content.c_str()); + + auto written = fwrite(source.data(), 1, source.size(), f); + if (written != source.size()) + { + printf("??? %zu %zu\n", written, source.size()); + printf("Unable to write to temp script %s\n", tempScriptName.c_str()); + exit(3); + } + + fclose(f); + } + + int step = 0; + + std::string escape(const std::string& s) + { + std::string result; + result.reserve(s.size() + 20); // guess + result += '"'; + for (char c : s) + { + if (c == '"') + result += '\\'; + result += c; + } + result += '"'; + + return result; + } + + TestResult run() + { + writeTempScript(); + + std::string command = appName + " " + escape(tempScriptName); + for (const auto& arg : appArgs) + command += " " + escape(arg); + +#if VERBOSE >= 1 + printf("running %s\n", command.c_str()); +#endif + + TestResult result = TestResult::NoBug; + + ++step; + printf("Step %4d...\n", step); + + FILE* p = popen(command.c_str(), "r"); + + while (!feof(p)) + { + std::string s = readLine(p); +#if VERBOSE >= 2 + printf("%s", s.c_str()); +#endif + if (std::string::npos != s.find(searchText)) + { + result = TestResult::BugFound; + break; + } + } + + pclose(p); + + return result; + } + + std::vector getNestedStats(AstStat* stat) + { + std::vector result; + + auto append = [&](AstStatBlock* block) { + if (block) + result.insert(result.end(), block->body.data, block->body.data + block->body.size); + }; + + if (auto block = stat->as()) + append(block); + else if (auto ifs = stat->as()) + { + append(ifs->thenbody); + if (ifs->elsebody) + { + if (AstStatBlock* elseBlock = ifs->elsebody->as()) + append(elseBlock); + else if (AstStatIf* elseIf = ifs->elsebody->as()) + { + auto innerStats = getNestedStats(elseIf); + result.insert(end(result), begin(innerStats), end(innerStats)); + } + else + { + printf("AstStatIf's else clause can have more statement types than I thought\n"); + LUAU_ASSERT(0); + } + } + } + else if (auto w = stat->as()) + append(w->body); + else if (auto r = stat->as()) + append(r->body); + else if (auto f = stat->as()) + append(f->body); + else if (auto f = stat->as()) + append(f->body); + else if (auto f = stat->as()) + append(f->func->body); + else if (auto f = stat->as()) + append(f->func->body); + + return result; + } + + // Move new body data into allocator-managed storage so that it's safe to keep around longterm. + AstStat** reallocateStatements(const std::vector& statements) + { + AstStat** newData = static_cast(allocator.allocate(sizeof(AstStat*) * statements.size())); + std::copy(statements.data(), statements.data() + statements.size(), newData); + + return newData; + } + + // Semiopen interval + using Span = std::pair; + + // Generates 'chunks' semiopen spans of equal-ish size to span the indeces running from 0 to 'size' + // Also inverses. + std::vector> generateSpans(size_t size, size_t chunks) + { + if (size <= 1) + return {}; + + LUAU_ASSERT(chunks > 0); + size_t chunkLength = std::max(1, size / chunks); + + std::vector> result; + + auto append = [&result](Span a, Span b) { + if (a.first == a.second && b.first == b.second) + return; + else + result.emplace_back(a, b); + }; + + size_t i = 0; + while (i < size) + { + size_t end = std::min(i + chunkLength, size); + append(Span{0, i}, Span{end, size}); + + i = end; + } + + i = 0; + while (i < size) + { + size_t end = std::min(i + chunkLength, size); + append(Span{i, end}, Span{size, size}); + + i = end; + } + + return result; + } + + // Returns the statements of block within span1 and span2 + // Also has the hokey restriction that span1 must come before span2 + std::vector prunedSpan(AstStatBlock* block, Span span1, Span span2) + { + std::vector result; + + for (size_t i = span1.first; i < span1.second; ++i) + result.push_back(block->body.data[i]); + + for (size_t i = span2.first; i < span2.second; ++i) + result.push_back(block->body.data[i]); + + return result; + } + + // returns true if anything was culled plus the chunk count + std::pair deleteChildStatements(AstStatBlock* block, size_t chunkCount) + { + if (block->body.size == 0) + return {false, chunkCount}; + + do + { + auto permutations = generateSpans(block->body.size, chunkCount); + for (const auto& [span1, span2] : permutations) + { + auto tempStatements = prunedSpan(block, span1, span2); + AstArray backupBody{tempStatements.data(), tempStatements.size()}; + + std::swap(block->body, backupBody); + TestResult result = run(); + if (result == TestResult::BugFound) + { + // The bug still reproduces without the statements we've culled. Commit. + block->body.data = reallocateStatements(tempStatements); + return {true, std::max(2, chunkCount - 1)}; + } + else + { + // The statements we've culled are critical for the reproduction of the bug. + // TODO try promoting its contents into this scope + std::swap(block->body, backupBody); + } + } + + chunkCount *= 2; + } while (chunkCount <= block->body.size); + + return {false, block->body.size}; + } + + bool deleteChildStatements(AstStatBlock* b) + { + bool result = false; + + size_t chunkCount = 2; + while (true) + { + auto [workDone, newChunkCount] = deleteChildStatements(b, chunkCount); + if (workDone) + { + result = true; + chunkCount = newChunkCount; + continue; + } + else + break; + } + + return result; + } + + bool tryPromotingChildStatements(AstStatBlock* b, size_t index) + { + std::vector tempStats(b->body.data, b->body.data + b->body.size); + AstStat* removed = tempStats.at(index); + tempStats.erase(begin(tempStats) + index); + + std::vector nestedStats = getNestedStats(removed); + tempStats.insert(begin(tempStats) + index, begin(nestedStats), end(nestedStats)); + + AstArray tempArray{tempStats.data(), tempStats.size()}; + std::swap(b->body, tempArray); + + TestResult result = run(); + + if (result == TestResult::BugFound) + { + b->body.data = reallocateStatements(tempStats); + return true; + } + else + { + std::swap(b->body, tempArray); + return false; + } + } + + // We live with some weirdness because I'm kind of lazy: If a statement's + // contents are promoted, we try promoting those prometed statements right + // away. I don't think it matters: If we can delete a statement and still + // exhibit the bug, we should do so. The order isn't so important. + bool tryPromotingChildStatements(AstStatBlock* b) + { + size_t i = 0; + while (i < b->body.size) + { + bool promoted = tryPromotingChildStatements(b, i); + if (!promoted) + ++i; + } + + return false; + } + + void walk(AstStatBlock* block) + { + std::queue queue; + Enqueuer enqueuer{&queue}; + + queue.push(block); + + while (!queue.empty()) + { + AstStatBlock* b = queue.front(); + queue.pop(); + + bool result = false; + do + { + result = deleteChildStatements(b); + + /* Try other reductions here before we walk into child statements + * Other reductions to try someday: + * + * Promoting a statement's children to the enclosing block. + * Deleting type annotations + * Deleting parts of type annotations + * Replacing subexpressions with ({} :: any) + * Inlining type aliases + * Inlining constants + * Inlining functions + */ + result |= tryPromotingChildStatements(b); + } while (result); + + for (AstStat* stat : b->body) + stat->visit(&enqueuer); + } + } + + void run(const std::string scriptName, const std::string appName, const std::vector& appArgs, std::string_view source, + std::string_view searchText) + { + tempScriptName = scriptName; + if (tempScriptName.substr(tempScriptName.size() - 4) == ".lua") + { + tempScriptName.erase(tempScriptName.size() - 4); + tempScriptName += "-reduced.lua"; + } + else + { + this->tempScriptName = scriptName + "-reduced"; + } + +#if 0 + // Handy debugging trick: VS Code will update its view of the file in realtime as it is edited. + std::string wheee = "code " + tempScriptName; + system(wheee.c_str()); +#endif + + printf("Temp script: %s\n", tempScriptName.c_str()); + + this->appName = appName; + this->appArgs = appArgs; + this->searchText = searchText; + + parseResult = Parser::parse(source.data(), source.size(), nameTable, allocator, parseOptions); + if (!parseResult.errors.empty()) + { + printf("Parse errors\n"); + exit(1); + } + + root = parseResult.root; + + const TestResult initialResult = run(); + if (initialResult == TestResult::NoBug) + { + printf("Could not find failure string in the unmodified script! Check your commandline arguments\n"); + exit(2); + } + + walk(root); + + writeTempScript(/* minify */ true); + + printf("Done! Check %s\n", tempScriptName.c_str()); + } +}; + +[[noreturn]] void help(const std::vector& args) +{ + printf("Syntax: %s script application \"search text\" [arguments]\n", args[0].data()); + exit(1); +} + +int main(int argc, char** argv) +{ + const std::vector args(argv, argv + argc); + + if (args.size() < 4) + help(args); + + for (int i = 1; i < args.size(); ++i) + { + if (args[i] == "--help") + help(args); + } + + const std::string scriptName = argv[1]; + const std::string appName = argv[2]; + const std::string searchText = argv[3]; + const std::vector appArgs(begin(args) + 4, end(args)); + + std::optional source = readFile(scriptName); + + if (!source) + { + printf("Could not read source %s\n", argv[1]); + exit(1); + } + + Reducer reducer; + reducer.run(scriptName, appName, appArgs, *source, searchText); +} diff --git a/CMakeLists.txt b/CMakeLists.txt index 92006344..3dafe5fe 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -32,11 +32,13 @@ if(LUAU_BUILD_CLI) add_executable(Luau.Repl.CLI) add_executable(Luau.Analyze.CLI) add_executable(Luau.Ast.CLI) + add_executable(Luau.Reduce.CLI) # This also adds target `name` on Linux/macOS and `name.exe` on Windows set_target_properties(Luau.Repl.CLI PROPERTIES OUTPUT_NAME luau) set_target_properties(Luau.Analyze.CLI PROPERTIES OUTPUT_NAME luau-analyze) set_target_properties(Luau.Ast.CLI PROPERTIES OUTPUT_NAME luau-ast) + set_target_properties(Luau.Reduce.CLI PROPERTIES OUTPUT_NAME luau-reduce) endif() if(LUAU_BUILD_TESTS) @@ -49,6 +51,7 @@ if(LUAU_BUILD_WEB) add_executable(Luau.Web) endif() + include(Sources.cmake) target_include_directories(Luau.Common INTERFACE Common/include) @@ -171,6 +174,10 @@ if(LUAU_BUILD_CLI) target_link_libraries(Luau.Analyze.CLI PRIVATE Luau.Analysis) target_link_libraries(Luau.Ast.CLI PRIVATE Luau.Ast Luau.Analysis) + + target_compile_features(Luau.Reduce.CLI PRIVATE cxx_std_17) + target_include_directories(Luau.Reduce.CLI PUBLIC Reduce/include) + target_link_libraries(Luau.Reduce.CLI PRIVATE Luau.Common Luau.Ast Luau.Analysis) endif() if(LUAU_BUILD_TESTS) diff --git a/CodeGen/include/Luau/AssemblyBuilderX64.h b/CodeGen/include/Luau/AssemblyBuilderX64.h index 028b2d16..cb799d31 100644 --- a/CodeGen/include/Luau/AssemblyBuilderX64.h +++ b/CodeGen/include/Luau/AssemblyBuilderX64.h @@ -38,14 +38,21 @@ public: // Two operand mov instruction has additional specialized encodings void mov(OperandX64 lhs, OperandX64 rhs); void mov64(RegisterX64 lhs, int64_t imm); + void movsx(RegisterX64 lhs, OperandX64 rhs); + void movzx(RegisterX64 lhs, OperandX64 rhs); // Base one operand instruction with 2 opcode selection void div(OperandX64 op); void idiv(OperandX64 op); void mul(OperandX64 op); + void imul(OperandX64 op); void neg(OperandX64 op); void not_(OperandX64 op); + // Additional forms of imul + void imul(OperandX64 lhs, OperandX64 rhs); + void imul(OperandX64 dst, OperandX64 lhs, int32_t rhs); + void test(OperandX64 lhs, OperandX64 rhs); void lea(OperandX64 lhs, OperandX64 rhs); @@ -76,6 +83,12 @@ public: void vxorpd(OperandX64 dst, OperandX64 src1, OperandX64 src2); void vcomisd(OperandX64 src1, OperandX64 src2); + void vucomisd(OperandX64 src1, OperandX64 src2); + + void vcvttsd2si(OperandX64 dst, OperandX64 src); + void vcvtsi2sd(OperandX64 dst, OperandX64 src1, OperandX64 src2); + + void vroundsd(OperandX64 dst, OperandX64 src1, OperandX64 src2, uint8_t mode); void vsqrtpd(OperandX64 dst, OperandX64 src); void vsqrtps(OperandX64 dst, OperandX64 src); @@ -105,6 +118,7 @@ public: OperandX64 f32(float value); OperandX64 f64(double value); OperandX64 f32x4(float x, float y, float z, float w); + OperandX64 bytes(const void* ptr, size_t size, size_t align = 8); // Resulting data and code that need to be copied over one after the other // The *end* of 'data' has to be aligned to 16 bytes, this will also align 'code' @@ -130,6 +144,8 @@ private: void placeAvx(const char* name, OperandX64 dst, OperandX64 src, uint8_t code, bool setW, uint8_t mode, uint8_t prefix); void placeAvx(const char* name, OperandX64 dst, OperandX64 src, uint8_t code, uint8_t coderev, bool setW, uint8_t mode, uint8_t prefix); void placeAvx(const char* name, OperandX64 dst, OperandX64 src1, OperandX64 src2, uint8_t code, bool setW, uint8_t mode, uint8_t prefix); + void placeAvx( + const char* name, OperandX64 dst, OperandX64 src1, OperandX64 src2, uint8_t imm8, uint8_t code, bool setW, uint8_t mode, uint8_t prefix); // Instruction components void placeRegAndModRegMem(OperandX64 lhs, OperandX64 rhs); @@ -157,6 +173,7 @@ private: LUAU_NOINLINE void log(const char* opcode, OperandX64 op); LUAU_NOINLINE void log(const char* opcode, OperandX64 op1, OperandX64 op2); LUAU_NOINLINE void log(const char* opcode, OperandX64 op1, OperandX64 op2, OperandX64 op3); + LUAU_NOINLINE void log(const char* opcode, OperandX64 op1, OperandX64 op2, OperandX64 op3, OperandX64 op4); LUAU_NOINLINE void log(Label label); LUAU_NOINLINE void log(const char* opcode, Label label); void log(OperandX64 op); diff --git a/CodeGen/src/AssemblyBuilderX64.cpp b/CodeGen/src/AssemblyBuilderX64.cpp index f88063cf..0fd10328 100644 --- a/CodeGen/src/AssemblyBuilderX64.cpp +++ b/CodeGen/src/AssemblyBuilderX64.cpp @@ -46,6 +46,44 @@ const unsigned AVX_F2 = 0b11; const unsigned kMaxAlign = 16; +// Utility functions to correctly write data on big endian machines +#if defined(__BYTE_ORDER__) && __BYTE_ORDER__ == __ORDER_BIG_ENDIAN__ +#include + +static void writeu32(uint8_t* target, uint32_t value) +{ + value = htole32(value); + memcpy(target, &value, sizeof(value)); +} + +static void writeu64(uint8_t* target, uint64_t value) +{ + value = htole64(value); + memcpy(target, &value, sizeof(value)); +} + +static void writef32(uint8_t* target, float value) +{ + static_assert(sizeof(float) == sizeof(uint32_t), "type size must match to reinterpret data"); + uint32_t data; + memcpy(&data, &value, sizeof(value)); + writeu32(target, data); +} + +static void writef64(uint8_t* target, double value) +{ + static_assert(sizeof(double) == sizeof(uint64_t), "type size must match to reinterpret data"); + uint64_t data; + memcpy(&data, &value, sizeof(value)); + writeu64(target, data); +} +#else +#define writeu32(target, value) memcpy(target, &value, sizeof(value)) +#define writeu64(target, value) memcpy(target, &value, sizeof(value)) +#define writef32(target, value) memcpy(target, &value, sizeof(value)) +#define writef64(target, value) memcpy(target, &value, sizeof(value)) +#endif + AssemblyBuilderX64::AssemblyBuilderX64(bool logText) : logText(logText) { @@ -195,6 +233,34 @@ void AssemblyBuilderX64::mov64(RegisterX64 lhs, int64_t imm) commit(); } +void AssemblyBuilderX64::movsx(RegisterX64 lhs, OperandX64 rhs) +{ + if (logText) + log("movsx", lhs, rhs); + + LUAU_ASSERT(rhs.memSize == SizeX64::byte || rhs.memSize == SizeX64::word); + + placeRex(lhs, rhs); + place(0x0f); + place(rhs.memSize == SizeX64::byte ? 0xbe : 0xbf); + placeRegAndModRegMem(lhs, rhs); + commit(); +} + +void AssemblyBuilderX64::movzx(RegisterX64 lhs, OperandX64 rhs) +{ + if (logText) + log("movzx", lhs, rhs); + + LUAU_ASSERT(rhs.memSize == SizeX64::byte || rhs.memSize == SizeX64::word); + + placeRex(lhs, rhs); + place(0x0f); + place(rhs.memSize == SizeX64::byte ? 0xb6 : 0xb7); + placeRegAndModRegMem(lhs, rhs); + commit(); +} + void AssemblyBuilderX64::div(OperandX64 op) { placeUnaryModRegMem("div", op, 0xf6, 0xf7, 6); @@ -210,6 +276,11 @@ void AssemblyBuilderX64::mul(OperandX64 op) placeUnaryModRegMem("mul", op, 0xf6, 0xf7, 4); } +void AssemblyBuilderX64::imul(OperandX64 op) +{ + placeUnaryModRegMem("imul", op, 0xf6, 0xf7, 5); +} + void AssemblyBuilderX64::neg(OperandX64 op) { placeUnaryModRegMem("neg", op, 0xf6, 0xf7, 3); @@ -220,6 +291,41 @@ void AssemblyBuilderX64::not_(OperandX64 op) placeUnaryModRegMem("not", op, 0xf6, 0xf7, 2); } +void AssemblyBuilderX64::imul(OperandX64 lhs, OperandX64 rhs) +{ + if (logText) + log("imul", lhs, rhs); + + placeRex(lhs.base, rhs); + place(0x0f); + place(0xaf); + placeRegAndModRegMem(lhs, rhs); + commit(); +} + +void AssemblyBuilderX64::imul(OperandX64 dst, OperandX64 lhs, int32_t rhs) +{ + if (logText) + log("imul", dst, lhs, rhs); + + placeRex(dst.base, lhs); + + if (int8_t(rhs) == rhs) + { + place(0x6b); + placeRegAndModRegMem(dst, lhs); + placeImm8(rhs); + } + else + { + place(0x69); + placeRegAndModRegMem(dst, lhs); + placeImm32(rhs); + } + + commit(); +} + void AssemblyBuilderX64::test(OperandX64 lhs, OperandX64 rhs) { // No forms for r/m*, imm8 and reg, r/m* @@ -368,6 +474,26 @@ void AssemblyBuilderX64::vcomisd(OperandX64 src1, OperandX64 src2) placeAvx("vcomisd", src1, src2, 0x2f, false, AVX_0F, AVX_66); } +void AssemblyBuilderX64::vucomisd(OperandX64 src1, OperandX64 src2) +{ + placeAvx("vucomisd", src1, src2, 0x2e, false, AVX_0F, AVX_66); +} + +void AssemblyBuilderX64::vcvttsd2si(OperandX64 dst, OperandX64 src) +{ + placeAvx("vcvttsd2si", dst, src, 0x2c, dst.base.size == SizeX64::dword, AVX_0F, AVX_F2); +} + +void AssemblyBuilderX64::vcvtsi2sd(OperandX64 dst, OperandX64 src1, OperandX64 src2) +{ + placeAvx("vcvtsi2sd", dst, src1, src2, 0x2a, (src2.cat == CategoryX64::reg ? src2.base.size : src2.memSize) == SizeX64::dword, AVX_0F, AVX_F2); +} + +void AssemblyBuilderX64::vroundsd(OperandX64 dst, OperandX64 src1, OperandX64 src2, uint8_t mode) +{ + placeAvx("vroundsd", dst, src1, src2, mode, 0x0b, false, AVX_0F3A, AVX_66); +} + void AssemblyBuilderX64::vsqrtpd(OperandX64 dst, OperandX64 src) { placeAvx("vsqrtpd", dst, src, 0x51, false, AVX_0F, AVX_66); @@ -436,7 +562,7 @@ void AssemblyBuilderX64::finalize() for (Label fixup : pendingLabels) { uint32_t value = labelLocations[fixup.id - 1] - (fixup.location + 4); - memcpy(&code[fixup.location], &value, sizeof(value)); + writeu32(&code[fixup.location], value); } size_t dataSize = data.size() - dataPos; @@ -479,34 +605,41 @@ void AssemblyBuilderX64::setLabel(Label& label) OperandX64 AssemblyBuilderX64::i64(int64_t value) { size_t pos = allocateData(8, 8); - memcpy(&data[pos], &value, sizeof(value)); + writeu64(&data[pos], value); return OperandX64(SizeX64::qword, noreg, 1, rip, int32_t(pos - data.size())); } OperandX64 AssemblyBuilderX64::f32(float value) { size_t pos = allocateData(4, 4); - memcpy(&data[pos], &value, sizeof(value)); + writef32(&data[pos], value); return OperandX64(SizeX64::dword, noreg, 1, rip, int32_t(pos - data.size())); } OperandX64 AssemblyBuilderX64::f64(double value) { size_t pos = allocateData(8, 8); - memcpy(&data[pos], &value, sizeof(value)); + writef64(&data[pos], value); return OperandX64(SizeX64::qword, noreg, 1, rip, int32_t(pos - data.size())); } OperandX64 AssemblyBuilderX64::f32x4(float x, float y, float z, float w) { size_t pos = allocateData(16, 16); - memcpy(&data[pos], &x, sizeof(x)); - memcpy(&data[pos + 4], &y, sizeof(y)); - memcpy(&data[pos + 8], &z, sizeof(z)); - memcpy(&data[pos + 12], &w, sizeof(w)); + writef32(&data[pos], x); + writef32(&data[pos + 4], y); + writef32(&data[pos + 8], z); + writef32(&data[pos + 12], w); return OperandX64(SizeX64::xmmword, noreg, 1, rip, int32_t(pos - data.size())); } +OperandX64 AssemblyBuilderX64::bytes(const void* ptr, size_t size, size_t align) +{ + size_t pos = allocateData(size, align); + memcpy(&data[pos], ptr, size); + return OperandX64(SizeX64::qword, noreg, 1, rip, int32_t(pos - data.size())); +} + void AssemblyBuilderX64::placeBinary(const char* name, OperandX64 lhs, OperandX64 rhs, uint8_t codeimm8, uint8_t codeimm, uint8_t codeimmImm8, uint8_t code8rev, uint8_t coderev, uint8_t code8, uint8_t code, uint8_t opreg) { @@ -700,6 +833,24 @@ void AssemblyBuilderX64::placeAvx( commit(); } +void AssemblyBuilderX64::placeAvx( + const char* name, OperandX64 dst, OperandX64 src1, OperandX64 src2, uint8_t imm8, uint8_t code, bool setW, uint8_t mode, uint8_t prefix) +{ + LUAU_ASSERT(dst.cat == CategoryX64::reg); + LUAU_ASSERT(src1.cat == CategoryX64::reg); + LUAU_ASSERT(src2.cat == CategoryX64::reg || src2.cat == CategoryX64::mem); + + if (logText) + log(name, dst, src1, src2, imm8); + + placeVex(dst, src1, src2, setW, mode, prefix); + place(code); + placeRegAndModRegMem(dst, src2); + placeImm8(imm8); + + commit(); +} + void AssemblyBuilderX64::placeRex(RegisterX64 op) { uint8_t code = REX_W(op.size == SizeX64::qword) | REX_B(op); @@ -861,16 +1012,18 @@ void AssemblyBuilderX64::placeImm8(int32_t imm) void AssemblyBuilderX64::placeImm32(int32_t imm) { - LUAU_ASSERT(codePos + sizeof(imm) < codeEnd); - memcpy(codePos, &imm, sizeof(imm)); - codePos += sizeof(imm); + uint8_t* pos = codePos; + LUAU_ASSERT(pos + sizeof(imm) < codeEnd); + writeu32(pos, imm); + codePos = pos + sizeof(imm); } void AssemblyBuilderX64::placeImm64(int64_t imm) { - LUAU_ASSERT(codePos + sizeof(imm) < codeEnd); - memcpy(codePos, &imm, sizeof(imm)); - codePos += sizeof(imm); + uint8_t* pos = codePos; + LUAU_ASSERT(pos + sizeof(imm) < codeEnd); + writeu64(pos, imm); + codePos = pos + sizeof(imm); } void AssemblyBuilderX64::placeLabel(Label& label) @@ -970,6 +1123,19 @@ void AssemblyBuilderX64::log(const char* opcode, OperandX64 op1, OperandX64 op2, text.append("\n"); } +void AssemblyBuilderX64::log(const char* opcode, OperandX64 op1, OperandX64 op2, OperandX64 op3, OperandX64 op4) +{ + logAppend(" %-12s", opcode); + log(op1); + text.append(","); + log(op2); + text.append(","); + log(op3); + text.append(","); + log(op4); + text.append("\n"); +} + void AssemblyBuilderX64::log(Label label) { logAppend(".L%d:\n", label.id); diff --git a/Compiler/src/Compiler.cpp b/Compiler/src/Compiler.cpp index 2ee20cab..eb815225 100644 --- a/Compiler/src/Compiler.cpp +++ b/Compiler/src/Compiler.cpp @@ -28,6 +28,8 @@ LUAU_FASTFLAGVARIABLE(LuauCompileNoIpairs, false) LUAU_FASTFLAGVARIABLE(LuauCompileFreeReassign, false) LUAU_FASTFLAGVARIABLE(LuauCompileXEQ, false) +LUAU_FASTFLAGVARIABLE(LuauCompileOptimalAssignment, false) + namespace Luau { @@ -37,6 +39,8 @@ static const uint32_t kMaxRegisterCount = 255; static const uint32_t kMaxUpvalueCount = 200; static const uint32_t kMaxLocalCount = 200; +static const uint8_t kInvalidReg = 255; + CompileError::CompileError(const Location& location, const std::string& message) : location(location) , message(message) @@ -2030,9 +2034,35 @@ struct Compiler return reg; } + // initializes target..target+targetCount-1 range using expression + // if expression is a call/vararg, we assume it returns all values, otherwise we fill the rest with nil + // assumes target register range can be clobbered and is at the top of the register space if targetTop = true + void compileExprTempN(AstExpr* node, uint8_t target, uint8_t targetCount, bool targetTop) + { + // we assume that target range is at the top of the register space and can be clobbered + // this is what allows us to compile the last call expression - if it's a call - using targetTop=true + LUAU_ASSERT(!targetTop || unsigned(target + targetCount) == regTop); + + if (AstExprCall* expr = node->as()) + { + compileExprCall(expr, target, targetCount, targetTop); + } + else if (AstExprVarargs* expr = node->as()) + { + compileExprVarargs(expr, target, targetCount); + } + else + { + compileExprTemp(node, target); + + for (size_t i = 1; i < targetCount; ++i) + bytecode.emitABC(LOP_LOADNIL, uint8_t(target + i), 0, 0); + } + } + // initializes target..target+targetCount-1 range using expressions from the list - // if list has fewer expressions, and last expression is a call, we assume the call returns the rest of the values - // if list has fewer expressions, and last expression isn't a call, we fill the rest with nil + // if list has fewer expressions, and last expression is multret, we assume it returns the rest of the values + // if list has fewer expressions, and last expression isn't multret, we fill the rest with nil // assumes target register range can be clobbered and is at the top of the register space if targetTop = true void compileExprListTemp(const AstArray& list, uint8_t target, uint8_t targetCount, bool targetTop) { @@ -2062,23 +2092,7 @@ struct Compiler for (size_t i = 0; i < list.size - 1; ++i) compileExprTemp(list.data[i], uint8_t(target + i)); - AstExpr* last = list.data[list.size - 1]; - - if (AstExprCall* expr = last->as()) - { - compileExprCall(expr, uint8_t(target + list.size - 1), uint8_t(targetCount - (list.size - 1)), targetTop); - } - else if (AstExprVarargs* expr = last->as()) - { - compileExprVarargs(expr, uint8_t(target + list.size - 1), uint8_t(targetCount - (list.size - 1))); - } - else - { - compileExprTemp(last, uint8_t(target + list.size - 1)); - - for (size_t i = list.size; i < targetCount; ++i) - bytecode.emitABC(LOP_LOADNIL, uint8_t(target + i), 0, 0); - } + compileExprTempN(list.data[list.size - 1], uint8_t(target + list.size - 1), uint8_t(targetCount - (list.size - 1)), targetTop); } else { @@ -2859,6 +2873,8 @@ struct Compiler void resolveAssignConflicts(AstStat* stat, std::vector& vars) { + LUAU_ASSERT(!FFlag::LuauCompileOptimalAssignment); + // regsUsed[i] is true if we have assigned the register during earlier assignments // regsRemap[i] is set to the register where the original (pre-assignment) copy was made // note: regsRemap is uninitialized intentionally to speed small assignments up; regsRemap[i] is valid iff regsUsed[i] @@ -2911,12 +2927,86 @@ struct Compiler } } + struct Assignment + { + LValue lvalue; + + uint8_t conflictReg = kInvalidReg; + uint8_t valueReg = kInvalidReg; + }; + + void resolveAssignConflicts(AstStat* stat, std::vector& vars, const AstArray& values) + { + struct Visitor : AstVisitor + { + Compiler* self; + + std::bitset<256> conflict; + std::bitset<256> assigned; + + Visitor(Compiler* self) + : self(self) + { + } + + bool visit(AstExprLocal* node) override + { + int reg = self->getLocalReg(node->local); + + if (reg >= 0 && assigned[reg]) + conflict[reg] = true; + + return true; + } + }; + + Visitor visitor(this); + + // mark any registers that are used *after* assignment as conflicting + for (size_t i = 0; i < vars.size(); ++i) + { + const LValue& li = vars[i].lvalue; + + if (i < values.size) + values.data[i]->visit(&visitor); + + if (li.kind == LValue::Kind_Local) + visitor.assigned[li.reg] = true; + } + + // mark any registers used in trailing expressions as conflicting as well + for (size_t i = vars.size(); i < values.size; ++i) + values.data[i]->visit(&visitor); + + // mark any registers used on left hand side that are also assigned anywhere as conflicting + // this is order-independent because we evaluate all right hand side arguments into registers before doing table assignments + for (const Assignment& var : vars) + { + const LValue& li = var.lvalue; + + if ((li.kind == LValue::Kind_IndexName || li.kind == LValue::Kind_IndexNumber || li.kind == LValue::Kind_IndexExpr) && + visitor.assigned[li.reg]) + visitor.conflict[li.reg] = true; + + if (li.kind == LValue::Kind_IndexExpr && visitor.assigned[li.index]) + visitor.conflict[li.index] = true; + } + + // for any conflicting var, we need to allocate a temporary register where the assignment is performed, so that we can move the value later + for (Assignment& var : vars) + { + const LValue& li = var.lvalue; + + if (li.kind == LValue::Kind_Local && visitor.conflict[li.reg]) + var.conflictReg = allocReg(stat, 1); + } + } + void compileStatAssign(AstStatAssign* stat) { RegScope rs(this); - // Optimization: one to one assignments don't require complex conflict resolution machinery and allow us to skip temporary registers for - // locals + // Optimization: one to one assignments don't require complex conflict resolution machinery if (stat->vars.size == 1 && stat->values.size == 1) { LValue var = compileLValue(stat->vars.data[0], rs); @@ -2936,28 +3026,110 @@ struct Compiler return; } - // compute all l-values: note that this doesn't assign anything yet but it allocates registers and computes complex expressions on the left - // hand side for example, in "a[expr] = foo" expr will get evaluated here - std::vector vars(stat->vars.size); - - for (size_t i = 0; i < stat->vars.size; ++i) - vars[i] = compileLValue(stat->vars.data[i], rs); - - // perform conflict resolution: if any lvalue refers to a local reg that will be reassigned before that, we save the local variable in a - // temporary reg - resolveAssignConflicts(stat, vars); - - // compute values into temporaries - uint8_t regs = allocReg(stat, unsigned(stat->vars.size)); - - compileExprListTemp(stat->values, regs, uint8_t(stat->vars.size), /* targetTop= */ true); - - // assign variables that have associated values; note that if we have fewer values than variables, we'll assign nil because - // compileExprListTemp will generate nils - for (size_t i = 0; i < stat->vars.size; ++i) + if (FFlag::LuauCompileOptimalAssignment) { - setDebugLine(stat->vars.data[i]); - compileAssign(vars[i], uint8_t(regs + i)); + // compute all l-values: note that this doesn't assign anything yet but it allocates registers and computes complex expressions on the + // left hand side - for example, in "a[expr] = foo" expr will get evaluated here + std::vector vars(stat->vars.size); + + for (size_t i = 0; i < stat->vars.size; ++i) + vars[i].lvalue = compileLValue(stat->vars.data[i], rs); + + // perform conflict resolution: if any expression refers to a local that is assigned before evaluating it, we assign to a temporary + // register after this, vars[i].conflictReg is set for locals that need to be assigned in the second pass + resolveAssignConflicts(stat, vars, stat->values); + + // compute rhs into (mostly) fresh registers + // note that when the lhs assigment is a local, we evaluate directly into that register + // this is possible because resolveAssignConflicts renamed conflicting locals into temporaries + // after this, vars[i].valueReg is set to a register with the value for *all* vars, but some have already been assigned + for (size_t i = 0; i < stat->vars.size && i < stat->values.size; ++i) + { + AstExpr* value = stat->values.data[i]; + + if (i + 1 == stat->values.size && stat->vars.size > stat->values.size) + { + // allocate a consecutive range of regs for all remaining vars and compute everything into temps + // note, this also handles trailing nils + uint8_t rest = uint8_t(stat->vars.size - stat->values.size + 1); + uint8_t temp = allocReg(stat, rest); + + compileExprTempN(value, temp, rest, /* targetTop= */ true); + + for (size_t j = i; j < stat->vars.size; ++j) + vars[j].valueReg = uint8_t(temp + (j - i)); + } + else + { + Assignment& var = vars[i]; + + // if target is a local, use compileExpr directly to target + if (var.lvalue.kind == LValue::Kind_Local) + { + var.valueReg = (var.conflictReg == kInvalidReg) ? var.lvalue.reg : var.conflictReg; + + compileExpr(stat->values.data[i], var.valueReg); + } + else + { + var.valueReg = compileExprAuto(stat->values.data[i], rs); + } + } + } + + // compute expressions with side effects for lulz + for (size_t i = stat->vars.size; i < stat->values.size; ++i) + { + RegScope rsi(this); + compileExprAuto(stat->values.data[i], rsi); + } + + // almost done... let's assign everything left to right, noting that locals were either written-to directly, or will be written-to in a + // separate pass to avoid conflicts + for (const Assignment& var : vars) + { + LUAU_ASSERT(var.valueReg != kInvalidReg); + + if (var.lvalue.kind != LValue::Kind_Local) + { + setDebugLine(var.lvalue.location); + compileAssign(var.lvalue, var.valueReg); + } + } + + // all regular local writes are done by the prior loops by computing result directly into target, so this just handles conflicts OR + // local copies from temporary registers in multret context, since in that case we have to allocate consecutive temporaries + for (const Assignment& var : vars) + { + if (var.lvalue.kind == LValue::Kind_Local && var.valueReg != var.lvalue.reg) + bytecode.emitABC(LOP_MOVE, var.lvalue.reg, var.valueReg, 0); + } + } + else + { + // compute all l-values: note that this doesn't assign anything yet but it allocates registers and computes complex expressions on the + // left hand side for example, in "a[expr] = foo" expr will get evaluated here + std::vector vars(stat->vars.size); + + for (size_t i = 0; i < stat->vars.size; ++i) + vars[i] = compileLValue(stat->vars.data[i], rs); + + // perform conflict resolution: if any lvalue refers to a local reg that will be reassigned before that, we save the local variable in a + // temporary reg + resolveAssignConflicts(stat, vars); + + // compute values into temporaries + uint8_t regs = allocReg(stat, unsigned(stat->vars.size)); + + compileExprListTemp(stat->values, regs, uint8_t(stat->vars.size), /* targetTop= */ true); + + // assign variables that have associated values; note that if we have fewer values than variables, we'll assign nil because + // compileExprListTemp will generate nils + for (size_t i = 0; i < stat->vars.size; ++i) + { + setDebugLine(stat->vars.data[i]); + compileAssign(vars[i], uint8_t(regs + i)); + } } } diff --git a/Makefile b/Makefile index 01fac077..8d95aac3 100644 --- a/Makefile +++ b/Makefile @@ -97,8 +97,8 @@ ifeq ($(config),fuzz) LDFLAGS+=-fsanitize=address,fuzzer endif -ifneq ($(CALLGRIND),) - CXXFLAGS+=-DCALLGRIND=$(CALLGRIND) +ifeq ($(config),profile) + CXXFLAGS+=-O2 -DNDEBUG -gdwarf-4 -DCALLGRIND=1 endif # target-specific flags diff --git a/Sources.cmake b/Sources.cmake index 9a6019a9..7a27684e 100644 --- a/Sources.cmake +++ b/Sources.cmake @@ -347,3 +347,11 @@ if(TARGET Luau.Web) target_sources(Luau.Web PRIVATE CLI/Web.cpp) endif() + +if(TARGET Luau.Reduce.CLI) + target_sources(Luau.Reduce.CLI PRIVATE + CLI/Reduce.cpp + CLI/FileUtils.cpp + CLI/FileUtils.h + ) +endif() diff --git a/bench/bench.py b/bench/bench.py index bb3ea5f7..42a0ac97 100644 --- a/bench/bench.py +++ b/bench/bench.py @@ -12,9 +12,6 @@ import json from color import colored, Color from tabulate import TablePrinter, Alignment -# Based on rotest, specialized for benchmark results -import influxbench - try: import matplotlib import matplotlib.pyplot as plt @@ -721,6 +718,7 @@ def run(args, argsubcb): argumentSubstituionCallback = argsubcb if arguments.report_metrics or arguments.print_influx_debugging: + import influxbench influxReporter = influxbench.InfluxReporter(arguments) else: influxReporter = None diff --git a/bench/influxbench.py b/bench/influxbench.py index adcddeba..20122456 100644 --- a/bench/influxbench.py +++ b/bench/influxbench.py @@ -4,12 +4,7 @@ import platform import shlex import socket import sys - -try: - import requests -except: - print("Please install 'requests' using using '{} -m pip install requests' command and try again".format(sys.executable)) - exit(-1) +import requests _hostname = socket.gethostname() diff --git a/tests/AssemblyBuilderX64.test.cpp b/tests/AssemblyBuilderX64.test.cpp index 28ce6a82..3f75ebac 100644 --- a/tests/AssemblyBuilderX64.test.cpp +++ b/tests/AssemblyBuilderX64.test.cpp @@ -22,7 +22,7 @@ std::string bytecodeAsArray(const std::vector& bytecode) class AssemblyBuilderX64Fixture { public: - void check(std::function f, std::vector result) + void check(std::function f, std::vector code, std::vector data = {}) { AssemblyBuilderX64 build(/* logText= */ false); @@ -30,9 +30,15 @@ public: build.finalize(); - if (build.code != result) + if (build.code != code) { - printf("Expected: %s\nReceived: %s\n", bytecodeAsArray(result).c_str(), bytecodeAsArray(build.code).c_str()); + printf("Expected code: %s\nReceived code: %s\n", bytecodeAsArray(code).c_str(), bytecodeAsArray(build.code).c_str()); + CHECK(false); + } + + if (build.data != data) + { + printf("Expected data: %s\nReceived data: %s\n", bytecodeAsArray(data).c_str(), bytecodeAsArray(build.data).c_str()); CHECK(false); } } @@ -169,6 +175,7 @@ TEST_CASE_FIXTURE(AssemblyBuilderX64Fixture, "BaseUnaryInstructionForms") SINGLE_COMPARE(div(rcx), 0x48, 0xf7, 0xf1); SINGLE_COMPARE(idiv(qword[rax]), 0x48, 0xf7, 0x38); SINGLE_COMPARE(mul(qword[rax + rbx]), 0x48, 0xf7, 0x24, 0x18); + SINGLE_COMPARE(imul(r9), 0x49, 0xf7, 0xe9); SINGLE_COMPARE(neg(r9), 0x49, 0xf7, 0xd9); SINGLE_COMPARE(not_(r12), 0x49, 0xf7, 0xd4); } @@ -191,6 +198,18 @@ TEST_CASE_FIXTURE(AssemblyBuilderX64Fixture, "FormsOfMov") SINGLE_COMPARE(mov(byte[rsi], al), 0x88, 0x06); } +TEST_CASE_FIXTURE(AssemblyBuilderX64Fixture, "FormsOfMovExtended") +{ + SINGLE_COMPARE(movsx(eax, byte[rcx]), 0x0f, 0xbe, 0x01); + SINGLE_COMPARE(movsx(r12, byte[r10]), 0x4d, 0x0f, 0xbe, 0x22); + SINGLE_COMPARE(movsx(ebx, word[r11]), 0x41, 0x0f, 0xbf, 0x1b); + SINGLE_COMPARE(movsx(rdx, word[rcx]), 0x48, 0x0f, 0xbf, 0x11); + SINGLE_COMPARE(movzx(eax, byte[rcx]), 0x0f, 0xb6, 0x01); + SINGLE_COMPARE(movzx(r12, byte[r10]), 0x4d, 0x0f, 0xb6, 0x22); + SINGLE_COMPARE(movzx(ebx, word[r11]), 0x41, 0x0f, 0xb7, 0x1b); + SINGLE_COMPARE(movzx(rdx, word[rcx]), 0x48, 0x0f, 0xb7, 0x11); +} + TEST_CASE_FIXTURE(AssemblyBuilderX64Fixture, "FormsOfTest") { SINGLE_COMPARE(test(al, 8), 0xf6, 0xc0, 0x08); @@ -230,6 +249,19 @@ TEST_CASE_FIXTURE(AssemblyBuilderX64Fixture, "FormsOfAbsoluteJumps") SINGLE_COMPARE(call(qword[r14 + rdx * 4]), 0x49, 0xff, 0x14, 0x96); } +TEST_CASE_FIXTURE(AssemblyBuilderX64Fixture, "FormsOfImul") +{ + SINGLE_COMPARE(imul(ecx, esi), 0x0f, 0xaf, 0xce); + SINGLE_COMPARE(imul(r12, rax), 0x4c, 0x0f, 0xaf, 0xe0); + SINGLE_COMPARE(imul(r12, qword[rdx + rdi]), 0x4c, 0x0f, 0xaf, 0x24, 0x3a); + SINGLE_COMPARE(imul(ecx, edx, 8), 0x6b, 0xca, 0x08); + SINGLE_COMPARE(imul(ecx, r9d, 0xabcd), 0x41, 0x69, 0xc9, 0xcd, 0xab, 0x00, 0x00); + SINGLE_COMPARE(imul(r8d, eax, -9), 0x44, 0x6b, 0xc0, 0xf7); + SINGLE_COMPARE(imul(rcx, rdx, 17), 0x48, 0x6b, 0xca, 0x11); + SINGLE_COMPARE(imul(rcx, r12, 0xabcd), 0x49, 0x69, 0xcc, 0xcd, 0xab, 0x00, 0x00); + SINGLE_COMPARE(imul(r12, rax, -13), 0x4c, 0x6b, 0xe0, 0xf3); +} + TEST_CASE_FIXTURE(AssemblyBuilderX64Fixture, "ControlFlow") { // Jump back @@ -335,6 +367,7 @@ TEST_CASE_FIXTURE(AssemblyBuilderX64Fixture, "AVXUnaryMergeInstructionForms") // Coverage for other instructions that follow the same pattern SINGLE_COMPARE(vcomisd(xmm8, xmm10), 0xc4, 0x41, 0xf9, 0x2f, 0xc2); + SINGLE_COMPARE(vucomisd(xmm1, xmm4), 0xc4, 0xe1, 0xf9, 0x2e, 0xcc); } TEST_CASE_FIXTURE(AssemblyBuilderX64Fixture, "AVXMoveInstructionForms") @@ -359,6 +392,25 @@ TEST_CASE_FIXTURE(AssemblyBuilderX64Fixture, "AVXMoveInstructionForms") SINGLE_COMPARE(vmovups(ymm8, ymmword[r9]), 0xc4, 0x41, 0xfc, 0x10, 0x01); } +TEST_CASE_FIXTURE(AssemblyBuilderX64Fixture, "AVXConversionInstructionForms") +{ + SINGLE_COMPARE(vcvttsd2si(ecx, xmm0), 0xc4, 0xe1, 0x7b, 0x2c, 0xc8); + SINGLE_COMPARE(vcvttsd2si(r9d, xmmword[rcx + rdx]), 0xc4, 0x61, 0x7b, 0x2c, 0x0c, 0x11); + SINGLE_COMPARE(vcvttsd2si(rdx, xmm0), 0xc4, 0xe1, 0xfb, 0x2c, 0xd0); + SINGLE_COMPARE(vcvttsd2si(r13, xmmword[rcx + rdx]), 0xc4, 0x61, 0xfb, 0x2c, 0x2c, 0x11); + SINGLE_COMPARE(vcvtsi2sd(xmm5, xmm10, ecx), 0xc4, 0xe1, 0x2b, 0x2a, 0xe9); + SINGLE_COMPARE(vcvtsi2sd(xmm6, xmm11, dword[rcx + rdx]), 0xc4, 0xe1, 0x23, 0x2a, 0x34, 0x11); + SINGLE_COMPARE(vcvtsi2sd(xmm5, xmm10, r13), 0xc4, 0xc1, 0xab, 0x2a, 0xed); + SINGLE_COMPARE(vcvtsi2sd(xmm6, xmm11, qword[rcx + rdx]), 0xc4, 0xe1, 0xa3, 0x2a, 0x34, 0x11); +} + +TEST_CASE_FIXTURE(AssemblyBuilderX64Fixture, "AVXTernaryInstructionForms") +{ + SINGLE_COMPARE(vroundsd(xmm7, xmm12, xmm3, 9), 0xc4, 0xe3, 0x99, 0x0b, 0xfb, 0x09); + SINGLE_COMPARE(vroundsd(xmm8, xmm13, xmmword[r13 + rdx], 9), 0xc4, 0x43, 0x91, 0x0b, 0x44, 0x15, 0x00, 0x09); + SINGLE_COMPARE(vroundsd(xmm9, xmm14, xmmword[rcx + r10], 1), 0xc4, 0x23, 0x89, 0x0b, 0x0c, 0x11, 0x01); +} + TEST_CASE_FIXTURE(AssemblyBuilderX64Fixture, "MiscInstructions") { SINGLE_COMPARE(int3(), 0xcc); @@ -386,6 +438,11 @@ TEST_CASE("LogTest") build.neg(qword[rbp + r12 * 2]); build.mov64(r10, 0x1234567812345678ll); build.vmovapd(xmmword[rax], xmm11); + build.movzx(eax, byte[rcx]); + build.movsx(rsi, word[r12]); + build.imul(rcx, rdx); + build.imul(rcx, rdx, 8); + build.vroundsd(xmm1, xmm2, xmm3, 5); build.pop(r12); build.ret(); build.int3(); @@ -409,6 +466,11 @@ TEST_CASE("LogTest") neg qword ptr [rbp+r12*2] mov r10,1234567812345678h vmovapd xmmword ptr [rax],xmm11 + movzx eax,byte ptr [rcx] + movsx rsi,word ptr [r12] + imul rcx,rdx + imul rcx,rdx,8 + vroundsd xmm1,xmm2,xmm3,5 pop r12 ret int3 @@ -426,6 +488,8 @@ TEST_CASE_FIXTURE(AssemblyBuilderX64Fixture, "Constants") build.vmovss(xmm2, build.f32(1.0f)); build.vmovsd(xmm3, build.f64(1.0)); build.vmovaps(xmm4, build.f32x4(1.0f, 2.0f, 4.0f, 8.0f)); + char arr[16] = "hello world!123"; + build.vmovupd(xmm5, build.bytes(arr, 16, 8)); build.ret(); }, { @@ -434,7 +498,20 @@ TEST_CASE_FIXTURE(AssemblyBuilderX64Fixture, "Constants") 0xc4, 0xe1, 0xfa, 0x10, 0x15, 0xe1, 0xff, 0xff, 0xff, 0xc4, 0xe1, 0xfb, 0x10, 0x1d, 0xcc, 0xff, 0xff, 0xff, 0xc4, 0xe1, 0xf8, 0x28, 0x25, 0xab, 0xff, 0xff, 0xff, + 0xc4, 0xe1, 0xf9, 0x10, 0x2d, 0x92, 0xff, 0xff, 0xff, 0xc3 + }, + { + 'h', 'e', 'l', 'l', 'o', ' ', 'w', 'o', 'r', 'l', 'd', '!', '1', '2', '3', 0x0, + 0x00, 0x00, 0x80, 0x3f, + 0x00, 0x00, 0x00, 0x40, + 0x00, 0x00, 0x80, 0x40, + 0x00, 0x00, 0x00, 0x41, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, // padding to align f32x4 + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0xf0, 0x3f, + 0x00, 0x00, 0x00, 0x00, // padding to align f64 + 0x00, 0x00, 0x80, 0x3f, + 0x21, 0x43, 0x65, 0x87, 0x78, 0x56, 0x34, 0x12, }); // clang-format on } @@ -444,7 +521,7 @@ TEST_CASE("ConstantStorage") AssemblyBuilderX64 build(/* logText= */ false); for (int i = 0; i <= 3000; i++) - build.vaddss(xmm0, xmm0, build.f32(float(i))); + build.vaddss(xmm0, xmm0, build.f32(1.0f)); build.finalize(); @@ -452,9 +529,10 @@ TEST_CASE("ConstantStorage") for (int i = 0; i <= 3000; i++) { - float v; - memcpy(&v, &build.data[build.data.size() - (i + 1) * sizeof(float)], sizeof(v)); - LUAU_ASSERT(v == float(i)); + LUAU_ASSERT(build.data[i * 4 + 0] == 0x00); + LUAU_ASSERT(build.data[i * 4 + 1] == 0x00); + LUAU_ASSERT(build.data[i * 4 + 2] == 0x80); + LUAU_ASSERT(build.data[i * 4 + 3] == 0x3f); } } diff --git a/tests/Autocomplete.test.cpp b/tests/Autocomplete.test.cpp index 75c5a606..0f17531b 100644 --- a/tests/Autocomplete.test.cpp +++ b/tests/Autocomplete.test.cpp @@ -129,6 +129,7 @@ TEST_CASE_FIXTURE(ACFixture, "empty_program") CHECK(!ac.entryMap.empty()); CHECK(ac.entryMap.count("table")); CHECK(ac.entryMap.count("math")); + CHECK_EQ(ac.context, AutocompleteContext::Statement); } TEST_CASE_FIXTURE(ACFixture, "local_initializer") @@ -138,6 +139,7 @@ TEST_CASE_FIXTURE(ACFixture, "local_initializer") auto ac = autocomplete('1'); CHECK(ac.entryMap.count("table")); CHECK(ac.entryMap.count("math")); + CHECK_EQ(ac.context, AutocompleteContext::Expression); } TEST_CASE_FIXTURE(ACFixture, "leave_numbers_alone") @@ -146,6 +148,7 @@ TEST_CASE_FIXTURE(ACFixture, "leave_numbers_alone") auto ac = autocomplete('1'); CHECK(ac.entryMap.empty()); + CHECK_EQ(ac.context, AutocompleteContext::Unknown); } TEST_CASE_FIXTURE(ACFixture, "user_defined_globals") @@ -157,6 +160,7 @@ TEST_CASE_FIXTURE(ACFixture, "user_defined_globals") CHECK(ac.entryMap.count("myLocal")); CHECK(ac.entryMap.count("table")); CHECK(ac.entryMap.count("math")); + CHECK_EQ(ac.context, AutocompleteContext::Statement); } TEST_CASE_FIXTURE(ACFixture, "dont_suggest_local_before_its_definition") @@ -191,6 +195,7 @@ TEST_CASE_FIXTURE(ACFixture, "recursive_function") auto ac = autocomplete('1'); CHECK(ac.entryMap.count("foo")); + CHECK_EQ(ac.context, AutocompleteContext::Statement); } TEST_CASE_FIXTURE(ACFixture, "nested_recursive_function") @@ -293,6 +298,7 @@ TEST_CASE_FIXTURE(ACBuiltinsFixture, "get_member_completions") CHECK(ac.entryMap.count("find")); CHECK(ac.entryMap.count("pack")); CHECK(!ac.entryMap.count("math")); + CHECK_EQ(ac.context, AutocompleteContext::Property); } TEST_CASE_FIXTURE(ACFixture, "nested_member_completions") @@ -306,6 +312,7 @@ TEST_CASE_FIXTURE(ACFixture, "nested_member_completions") CHECK_EQ(2, ac.entryMap.size()); CHECK(ac.entryMap.count("def")); CHECK(ac.entryMap.count("egh")); + CHECK_EQ(ac.context, AutocompleteContext::Property); } TEST_CASE_FIXTURE(ACFixture, "unsealed_table") @@ -319,6 +326,7 @@ TEST_CASE_FIXTURE(ACFixture, "unsealed_table") auto ac = autocomplete('1'); CHECK_EQ(1, ac.entryMap.size()); CHECK(ac.entryMap.count("prop")); + CHECK_EQ(ac.context, AutocompleteContext::Property); } TEST_CASE_FIXTURE(ACFixture, "unsealed_table_2") @@ -333,6 +341,7 @@ TEST_CASE_FIXTURE(ACFixture, "unsealed_table_2") auto ac = autocomplete('1'); CHECK_EQ(1, ac.entryMap.size()); CHECK(ac.entryMap.count("prop")); + CHECK_EQ(ac.context, AutocompleteContext::Property); } TEST_CASE_FIXTURE(ACFixture, "cyclic_table") @@ -346,6 +355,7 @@ TEST_CASE_FIXTURE(ACFixture, "cyclic_table") auto ac = autocomplete('1'); CHECK(ac.entryMap.count("abc")); + CHECK_EQ(ac.context, AutocompleteContext::Property); } TEST_CASE_FIXTURE(ACFixture, "table_union") @@ -361,6 +371,7 @@ TEST_CASE_FIXTURE(ACFixture, "table_union") auto ac = autocomplete('1'); CHECK_EQ(1, ac.entryMap.size()); CHECK(ac.entryMap.count("b2")); + CHECK_EQ(ac.context, AutocompleteContext::Property); } TEST_CASE_FIXTURE(ACFixture, "table_intersection") @@ -378,6 +389,7 @@ TEST_CASE_FIXTURE(ACFixture, "table_intersection") CHECK(ac.entryMap.count("a1")); CHECK(ac.entryMap.count("b2")); CHECK(ac.entryMap.count("c3")); + CHECK_EQ(ac.context, AutocompleteContext::Property); } TEST_CASE_FIXTURE(ACBuiltinsFixture, "get_string_completions") @@ -389,6 +401,7 @@ TEST_CASE_FIXTURE(ACBuiltinsFixture, "get_string_completions") auto ac = autocomplete('1'); CHECK_EQ(17, ac.entryMap.size()); + CHECK_EQ(ac.context, AutocompleteContext::Property); } TEST_CASE_FIXTURE(ACFixture, "get_suggestions_for_new_statement") @@ -400,6 +413,7 @@ TEST_CASE_FIXTURE(ACFixture, "get_suggestions_for_new_statement") CHECK_NE(0, ac.entryMap.size()); CHECK(ac.entryMap.count("table")); + CHECK_EQ(ac.context, AutocompleteContext::Statement); } TEST_CASE_FIXTURE(ACFixture, "get_suggestions_for_the_very_start_of_the_script") @@ -412,6 +426,7 @@ TEST_CASE_FIXTURE(ACFixture, "get_suggestions_for_the_very_start_of_the_script") auto ac = autocomplete('1'); CHECK(ac.entryMap.count("table")); + CHECK_EQ(ac.context, AutocompleteContext::Statement); } TEST_CASE_FIXTURE(ACFixture, "method_call_inside_function_body") @@ -429,6 +444,7 @@ TEST_CASE_FIXTURE(ACFixture, "method_call_inside_function_body") CHECK_NE(0, ac.entryMap.size()); CHECK(!ac.entryMap.count("math")); + CHECK_EQ(ac.context, AutocompleteContext::Property); } TEST_CASE_FIXTURE(ACBuiltinsFixture, "method_call_inside_if_conditional") @@ -442,6 +458,7 @@ TEST_CASE_FIXTURE(ACBuiltinsFixture, "method_call_inside_if_conditional") CHECK_NE(0, ac.entryMap.size()); CHECK(ac.entryMap.count("concat")); CHECK(!ac.entryMap.count("math")); + CHECK_EQ(ac.context, AutocompleteContext::Property); } TEST_CASE_FIXTURE(ACFixture, "statement_between_two_statements") @@ -459,6 +476,8 @@ TEST_CASE_FIXTURE(ACFixture, "statement_between_two_statements") CHECK_NE(0, ac.entryMap.size()); CHECK(ac.entryMap.count("getmyscripts")); + + CHECK_EQ(ac.context, AutocompleteContext::Statement); } TEST_CASE_FIXTURE(ACFixture, "bias_toward_inner_scope") @@ -476,6 +495,7 @@ TEST_CASE_FIXTURE(ACFixture, "bias_toward_inner_scope") auto ac = autocomplete('1'); CHECK(ac.entryMap.count("A")); + CHECK_EQ(ac.context, AutocompleteContext::Statement); TypeId t = follow(*ac.entryMap["A"].type); const TableTypeVar* tt = get(t); @@ -489,10 +509,12 @@ TEST_CASE_FIXTURE(ACFixture, "recommend_statement_starting_keywords") check("@1"); auto ac = autocomplete('1'); CHECK(ac.entryMap.count("local")); + CHECK_EQ(ac.context, AutocompleteContext::Statement); check("local i = @1"); auto ac2 = autocomplete('1'); CHECK(!ac2.entryMap.count("local")); + CHECK_EQ(ac2.context, AutocompleteContext::Expression); } TEST_CASE_FIXTURE(ACFixture, "do_not_overwrite_context_sensitive_kws") @@ -508,6 +530,7 @@ TEST_CASE_FIXTURE(ACFixture, "do_not_overwrite_context_sensitive_kws") AutocompleteEntry entry = ac.entryMap["continue"]; CHECK(entry.kind == AutocompleteEntryKind::Binding); + CHECK_EQ(ac.context, AutocompleteContext::Statement); } TEST_CASE_FIXTURE(ACFixture, "dont_offer_any_suggestions_from_within_a_comment") @@ -525,6 +548,7 @@ TEST_CASE_FIXTURE(ACFixture, "dont_offer_any_suggestions_from_within_a_comment") auto ac = autocomplete('1'); CHECK_EQ(0, ac.entryMap.size()); + CHECK_EQ(ac.context, AutocompleteContext::Unknown); } TEST_CASE_FIXTURE(ACFixture, "dont_offer_any_suggestions_from_the_end_of_a_comment") @@ -536,6 +560,7 @@ TEST_CASE_FIXTURE(ACFixture, "dont_offer_any_suggestions_from_the_end_of_a_comme auto ac = autocomplete('1'); CHECK_EQ(0, ac.entryMap.size()); + CHECK_EQ(ac.context, AutocompleteContext::Unknown); } TEST_CASE_FIXTURE(ACFixture, "dont_offer_any_suggestions_from_within_a_broken_comment") @@ -547,6 +572,7 @@ TEST_CASE_FIXTURE(ACFixture, "dont_offer_any_suggestions_from_within_a_broken_co auto ac = autocomplete('1'); CHECK_EQ(0, ac.entryMap.size()); + CHECK_EQ(ac.context, AutocompleteContext::Unknown); } TEST_CASE_FIXTURE(ACFixture, "dont_offer_any_suggestions_from_within_a_broken_comment_at_the_very_end_of_the_file") @@ -555,6 +581,7 @@ TEST_CASE_FIXTURE(ACFixture, "dont_offer_any_suggestions_from_within_a_broken_co auto ac = autocomplete('1'); CHECK_EQ(0, ac.entryMap.size()); + CHECK_EQ(ac.context, AutocompleteContext::Unknown); } TEST_CASE_FIXTURE(ACFixture, "autocomplete_for_middle_keywords") @@ -566,6 +593,7 @@ TEST_CASE_FIXTURE(ACFixture, "autocomplete_for_middle_keywords") auto ac1 = autocomplete('1'); CHECK_EQ(ac1.entryMap.count("do"), 0); CHECK_EQ(ac1.entryMap.count("end"), 0); + CHECK_EQ(ac1.context, AutocompleteContext::Unknown); check(R"( for x =@1 1 @@ -574,6 +602,7 @@ TEST_CASE_FIXTURE(ACFixture, "autocomplete_for_middle_keywords") auto ac2 = autocomplete('1'); CHECK_EQ(ac2.entryMap.count("do"), 0); CHECK_EQ(ac2.entryMap.count("end"), 0); + CHECK_EQ(ac2.context, AutocompleteContext::Unknown); check(R"( for x = 1,@1 2 @@ -582,6 +611,7 @@ TEST_CASE_FIXTURE(ACFixture, "autocomplete_for_middle_keywords") auto ac3 = autocomplete('1'); CHECK_EQ(1, ac3.entryMap.size()); CHECK_EQ(ac3.entryMap.count("do"), 1); + CHECK_EQ(ac3.context, AutocompleteContext::Keyword); check(R"( for x = 1, @12, @@ -590,6 +620,7 @@ TEST_CASE_FIXTURE(ACFixture, "autocomplete_for_middle_keywords") auto ac4 = autocomplete('1'); CHECK_EQ(ac4.entryMap.count("do"), 0); CHECK_EQ(ac4.entryMap.count("end"), 0); + CHECK_EQ(ac4.context, AutocompleteContext::Expression); check(R"( for x = 1, 2, @15 @@ -598,6 +629,7 @@ TEST_CASE_FIXTURE(ACFixture, "autocomplete_for_middle_keywords") auto ac5 = autocomplete('1'); CHECK_EQ(ac5.entryMap.count("do"), 1); CHECK_EQ(ac5.entryMap.count("end"), 0); + CHECK_EQ(ac5.context, AutocompleteContext::Keyword); check(R"( for x = 1, 2, 5 f@1 @@ -606,6 +638,7 @@ TEST_CASE_FIXTURE(ACFixture, "autocomplete_for_middle_keywords") auto ac6 = autocomplete('1'); CHECK_EQ(ac6.entryMap.size(), 1); CHECK_EQ(ac6.entryMap.count("do"), 1); + CHECK_EQ(ac6.context, AutocompleteContext::Keyword); check(R"( for x = 1, 2, 5 do @1 @@ -613,6 +646,7 @@ TEST_CASE_FIXTURE(ACFixture, "autocomplete_for_middle_keywords") auto ac7 = autocomplete('1'); CHECK_EQ(ac7.entryMap.count("end"), 1); + CHECK_EQ(ac7.context, AutocompleteContext::Statement); } TEST_CASE_FIXTURE(ACFixture, "autocomplete_for_in_middle_keywords") @@ -623,6 +657,7 @@ TEST_CASE_FIXTURE(ACFixture, "autocomplete_for_in_middle_keywords") auto ac1 = autocomplete('1'); CHECK_EQ(0, ac1.entryMap.size()); + CHECK_EQ(ac1.context, AutocompleteContext::Unknown); check(R"( for x@1 @2 @@ -630,10 +665,12 @@ TEST_CASE_FIXTURE(ACFixture, "autocomplete_for_in_middle_keywords") auto ac2 = autocomplete('1'); CHECK_EQ(0, ac2.entryMap.size()); + CHECK_EQ(ac2.context, AutocompleteContext::Unknown); auto ac2a = autocomplete('2'); CHECK_EQ(1, ac2a.entryMap.size()); CHECK_EQ(1, ac2a.entryMap.count("in")); + CHECK_EQ(ac2a.context, AutocompleteContext::Keyword); check(R"( for x in y@1 @@ -642,6 +679,7 @@ TEST_CASE_FIXTURE(ACFixture, "autocomplete_for_in_middle_keywords") auto ac3 = autocomplete('1'); CHECK_EQ(ac3.entryMap.count("table"), 1); CHECK_EQ(ac3.entryMap.count("do"), 0); + CHECK_EQ(ac3.context, AutocompleteContext::Expression); check(R"( for x in y @1 @@ -650,6 +688,7 @@ TEST_CASE_FIXTURE(ACFixture, "autocomplete_for_in_middle_keywords") auto ac4 = autocomplete('1'); CHECK_EQ(ac4.entryMap.size(), 1); CHECK_EQ(ac4.entryMap.count("do"), 1); + CHECK_EQ(ac4.context, AutocompleteContext::Keyword); check(R"( for x in f f@1 @@ -658,6 +697,7 @@ TEST_CASE_FIXTURE(ACFixture, "autocomplete_for_in_middle_keywords") auto ac5 = autocomplete('1'); CHECK_EQ(ac5.entryMap.size(), 1); CHECK_EQ(ac5.entryMap.count("do"), 1); + CHECK_EQ(ac5.context, AutocompleteContext::Keyword); check(R"( for x in y do @1 @@ -668,6 +708,7 @@ TEST_CASE_FIXTURE(ACFixture, "autocomplete_for_in_middle_keywords") CHECK_EQ(ac6.entryMap.count("table"), 1); CHECK_EQ(ac6.entryMap.count("end"), 1); CHECK_EQ(ac6.entryMap.count("function"), 1); + CHECK_EQ(ac6.context, AutocompleteContext::Statement); check(R"( for x in y do e@1 @@ -678,6 +719,7 @@ TEST_CASE_FIXTURE(ACFixture, "autocomplete_for_in_middle_keywords") CHECK_EQ(ac7.entryMap.count("table"), 1); CHECK_EQ(ac7.entryMap.count("end"), 1); CHECK_EQ(ac7.entryMap.count("function"), 1); + CHECK_EQ(ac7.context, AutocompleteContext::Statement); } TEST_CASE_FIXTURE(ACFixture, "autocomplete_while_middle_keywords") @@ -689,6 +731,7 @@ TEST_CASE_FIXTURE(ACFixture, "autocomplete_while_middle_keywords") auto ac1 = autocomplete('1'); CHECK_EQ(ac1.entryMap.count("do"), 0); CHECK_EQ(ac1.entryMap.count("end"), 0); + CHECK_EQ(ac1.context, AutocompleteContext::Expression); check(R"( while true @1 @@ -697,6 +740,7 @@ TEST_CASE_FIXTURE(ACFixture, "autocomplete_while_middle_keywords") auto ac2 = autocomplete('1'); CHECK_EQ(1, ac2.entryMap.size()); CHECK_EQ(ac2.entryMap.count("do"), 1); + CHECK_EQ(ac2.context, AutocompleteContext::Keyword); check(R"( while true do @1 @@ -704,6 +748,7 @@ TEST_CASE_FIXTURE(ACFixture, "autocomplete_while_middle_keywords") auto ac3 = autocomplete('1'); CHECK_EQ(ac3.entryMap.count("end"), 1); + CHECK_EQ(ac3.context, AutocompleteContext::Statement); check(R"( while true d@1 @@ -712,6 +757,7 @@ TEST_CASE_FIXTURE(ACFixture, "autocomplete_while_middle_keywords") auto ac4 = autocomplete('1'); CHECK_EQ(1, ac4.entryMap.size()); CHECK_EQ(ac4.entryMap.count("do"), 1); + CHECK_EQ(ac4.context, AutocompleteContext::Keyword); } TEST_CASE_FIXTURE(ACFixture, "autocomplete_if_middle_keywords") @@ -728,6 +774,7 @@ TEST_CASE_FIXTURE(ACFixture, "autocomplete_if_middle_keywords") CHECK_EQ(ac1.entryMap.count("else"), 0); CHECK_EQ(ac1.entryMap.count("elseif"), 0); CHECK_EQ(ac1.entryMap.count("end"), 0); + CHECK_EQ(ac1.context, AutocompleteContext::Expression); check(R"( if x @1 @@ -739,6 +786,7 @@ TEST_CASE_FIXTURE(ACFixture, "autocomplete_if_middle_keywords") CHECK_EQ(ac2.entryMap.count("else"), 0); CHECK_EQ(ac2.entryMap.count("elseif"), 0); CHECK_EQ(ac2.entryMap.count("end"), 0); + CHECK_EQ(ac2.context, AutocompleteContext::Keyword); check(R"( if x t@1 @@ -747,6 +795,7 @@ TEST_CASE_FIXTURE(ACFixture, "autocomplete_if_middle_keywords") auto ac3 = autocomplete('1'); CHECK_EQ(1, ac3.entryMap.size()); CHECK_EQ(ac3.entryMap.count("then"), 1); + CHECK_EQ(ac3.context, AutocompleteContext::Keyword); check(R"( if x then @@ -760,6 +809,7 @@ TEST_CASE_FIXTURE(ACFixture, "autocomplete_if_middle_keywords") CHECK_EQ(ac4.entryMap.count("function"), 1); CHECK_EQ(ac4.entryMap.count("elseif"), 1); CHECK_EQ(ac4.entryMap.count("end"), 0); + CHECK_EQ(ac4.context, AutocompleteContext::Statement); check(R"( if x then @@ -772,6 +822,7 @@ TEST_CASE_FIXTURE(ACFixture, "autocomplete_if_middle_keywords") CHECK_EQ(ac4a.entryMap.count("table"), 1); CHECK_EQ(ac4a.entryMap.count("else"), 1); CHECK_EQ(ac4a.entryMap.count("elseif"), 1); + CHECK_EQ(ac4a.context, AutocompleteContext::Statement); check(R"( if x then @@ -786,6 +837,7 @@ TEST_CASE_FIXTURE(ACFixture, "autocomplete_if_middle_keywords") CHECK_EQ(ac5.entryMap.count("else"), 0); CHECK_EQ(ac5.entryMap.count("elseif"), 0); CHECK_EQ(ac5.entryMap.count("end"), 0); + CHECK_EQ(ac5.context, AutocompleteContext::Statement); } TEST_CASE_FIXTURE(ACFixture, "autocomplete_until_in_repeat") @@ -797,6 +849,7 @@ TEST_CASE_FIXTURE(ACFixture, "autocomplete_until_in_repeat") auto ac = autocomplete('1'); CHECK_EQ(ac.entryMap.count("table"), 1); CHECK_EQ(ac.entryMap.count("until"), 1); + CHECK_EQ(ac.context, AutocompleteContext::Statement); } TEST_CASE_FIXTURE(ACFixture, "autocomplete_until_expression") @@ -808,6 +861,7 @@ TEST_CASE_FIXTURE(ACFixture, "autocomplete_until_expression") auto ac = autocomplete('1'); CHECK_EQ(ac.entryMap.count("table"), 1); + CHECK_EQ(ac.context, AutocompleteContext::Expression); } TEST_CASE_FIXTURE(ACFixture, "local_names") @@ -819,6 +873,7 @@ TEST_CASE_FIXTURE(ACFixture, "local_names") auto ac1 = autocomplete('1'); CHECK_EQ(ac1.entryMap.size(), 1); CHECK_EQ(ac1.entryMap.count("function"), 1); + CHECK_EQ(ac1.context, AutocompleteContext::Unknown); check(R"( local ab, cd@1 @@ -826,6 +881,7 @@ TEST_CASE_FIXTURE(ACFixture, "local_names") auto ac2 = autocomplete('1'); CHECK(ac2.entryMap.empty()); + CHECK_EQ(ac2.context, AutocompleteContext::Unknown); } TEST_CASE_FIXTURE(ACFixture, "autocomplete_end_with_fn_exprs") @@ -836,6 +892,7 @@ TEST_CASE_FIXTURE(ACFixture, "autocomplete_end_with_fn_exprs") auto ac = autocomplete('1'); CHECK_EQ(ac.entryMap.count("end"), 1); + CHECK_EQ(ac.context, AutocompleteContext::Statement); } TEST_CASE_FIXTURE(ACFixture, "autocomplete_end_with_lambda") @@ -846,6 +903,7 @@ TEST_CASE_FIXTURE(ACFixture, "autocomplete_end_with_lambda") auto ac = autocomplete('1'); CHECK_EQ(ac.entryMap.count("end"), 1); + CHECK_EQ(ac.context, AutocompleteContext::Statement); } TEST_CASE_FIXTURE(ACFixture, "stop_at_first_stat_when_recommending_keywords") @@ -858,6 +916,7 @@ TEST_CASE_FIXTURE(ACFixture, "stop_at_first_stat_when_recommending_keywords") auto ac1 = autocomplete('1'); CHECK_EQ(ac1.entryMap.count("in"), 1); CHECK_EQ(ac1.entryMap.count("until"), 0); + CHECK_EQ(ac1.context, AutocompleteContext::Keyword); } TEST_CASE_FIXTURE(ACFixture, "autocomplete_repeat_middle_keyword") @@ -980,6 +1039,7 @@ TEST_CASE_FIXTURE(ACFixture, "local_function_params") auto ac2 = autocomplete('1'); CHECK_EQ(ac2.entryMap.count("abc"), 1); CHECK_EQ(ac2.entryMap.count("def"), 1); + CHECK_EQ(ac2.context, AutocompleteContext::Statement); check(R"( local function abc(def, ghi@1) @@ -988,6 +1048,7 @@ TEST_CASE_FIXTURE(ACFixture, "local_function_params") auto ac3 = autocomplete('1'); CHECK(ac3.entryMap.empty()); + CHECK_EQ(ac3.context, AutocompleteContext::Unknown); } TEST_CASE_FIXTURE(ACFixture, "global_function_params") @@ -1022,6 +1083,7 @@ TEST_CASE_FIXTURE(ACFixture, "global_function_params") auto ac2 = autocomplete('1'); CHECK_EQ(ac2.entryMap.count("abc"), 1); CHECK_EQ(ac2.entryMap.count("def"), 1); + CHECK_EQ(ac2.context, AutocompleteContext::Statement); check(R"( function abc(def, ghi@1) @@ -1030,6 +1092,7 @@ TEST_CASE_FIXTURE(ACFixture, "global_function_params") auto ac3 = autocomplete('1'); CHECK(ac3.entryMap.empty()); + CHECK_EQ(ac3.context, AutocompleteContext::Unknown); } TEST_CASE_FIXTURE(ACFixture, "arguments_to_global_lambda") @@ -1074,6 +1137,7 @@ TEST_CASE_FIXTURE(ACFixture, "function_expr_params") auto ac2 = autocomplete('1'); CHECK_EQ(ac2.entryMap.count("def"), 1); + CHECK_EQ(ac2.context, AutocompleteContext::Statement); } TEST_CASE_FIXTURE(ACFixture, "local_initializer") @@ -1135,6 +1199,7 @@ local b: string = "don't trip" CHECK(ac.entryMap.count("nil")); CHECK(ac.entryMap.count("number")); + CHECK_EQ(ac.context, AutocompleteContext::Type); } TEST_CASE_FIXTURE(ACFixture, "private_types") @@ -1203,6 +1268,7 @@ local a: aa auto ac = Luau::autocomplete(frontend, "Module/B", Position{2, 11}, nullCallback); CHECK(ac.entryMap.count("aaa")); + CHECK_EQ(ac.context, AutocompleteContext::Type); } TEST_CASE_FIXTURE(ACFixture, "module_type_members") @@ -1227,6 +1293,7 @@ local a: aaa. CHECK_EQ(2, ac.entryMap.size()); CHECK(ac.entryMap.count("A")); CHECK(ac.entryMap.count("B")); + CHECK_EQ(ac.context, AutocompleteContext::Type); } TEST_CASE_FIXTURE(ACFixture, "argument_types") @@ -1240,6 +1307,7 @@ local b: string = "don't trip" CHECK(ac.entryMap.count("nil")); CHECK(ac.entryMap.count("number")); + CHECK_EQ(ac.context, AutocompleteContext::Type); } TEST_CASE_FIXTURE(ACFixture, "return_types") @@ -1253,6 +1321,7 @@ local b: string = "don't trip" CHECK(ac.entryMap.count("nil")); CHECK(ac.entryMap.count("number")); + CHECK_EQ(ac.context, AutocompleteContext::Type); } TEST_CASE_FIXTURE(ACFixture, "as_types") @@ -1266,6 +1335,7 @@ local b: number = (a :: n@1 CHECK(ac.entryMap.count("nil")); CHECK(ac.entryMap.count("number")); + CHECK_EQ(ac.context, AutocompleteContext::Type); } TEST_CASE_FIXTURE(ACFixture, "function_type_types") @@ -1314,6 +1384,7 @@ local b: string = "don't trip" auto ac = autocomplete('1'); CHECK(ac.entryMap.count("Tee")); + CHECK_EQ(ac.context, AutocompleteContext::Type); } TEST_CASE_FIXTURE(ACFixture, "type_correct_suggestion_in_argument") @@ -1402,6 +1473,7 @@ local b: Foo = { a = a.@1 CHECK(ac.entryMap.count("one")); CHECK(ac.entryMap["one"].typeCorrect == TypeCorrectKind::Correct); CHECK(ac.entryMap["two"].typeCorrect == TypeCorrectKind::None); + CHECK_EQ(ac.context, AutocompleteContext::Property); check(R"( type Foo = { a: number, b: string } @@ -1414,6 +1486,7 @@ local b: Foo = { b = a.@1 CHECK(ac.entryMap.count("two")); CHECK(ac.entryMap["two"].typeCorrect == TypeCorrectKind::Correct); CHECK(ac.entryMap["one"].typeCorrect == TypeCorrectKind::None); + CHECK_EQ(ac.context, AutocompleteContext::Property); } TEST_CASE_FIXTURE(ACFixture, "type_correct_function_return_types") @@ -2395,6 +2468,7 @@ local t: Test = { f@1 } auto ac = autocomplete('1'); CHECK(ac.entryMap.count("first")); CHECK(ac.entryMap.count("second")); + CHECK_EQ(ac.context, AutocompleteContext::Property); // Intersection check(R"( @@ -2405,6 +2479,7 @@ local t: Test = { f@1 } ac = autocomplete('1'); CHECK(ac.entryMap.count("first")); CHECK(ac.entryMap.count("second")); + CHECK_EQ(ac.context, AutocompleteContext::Property); // Union check(R"( @@ -2416,6 +2491,7 @@ local t: Test = { s@1 } CHECK(ac.entryMap.count("second")); CHECK(!ac.entryMap.count("first")); CHECK(!ac.entryMap.count("third")); + CHECK_EQ(ac.context, AutocompleteContext::Property); // No parenthesis suggestion check(R"( @@ -2426,6 +2502,7 @@ local t: Test = { f@1 } ac = autocomplete('1'); CHECK(ac.entryMap.count("first")); CHECK(ac.entryMap["first"].parens == ParenthesesRecommendation::None); + CHECK_EQ(ac.context, AutocompleteContext::Property); // When key is changed check(R"( @@ -2436,6 +2513,7 @@ local t: Test = { f@1 = 2 } ac = autocomplete('1'); CHECK(ac.entryMap.count("first")); CHECK(ac.entryMap.count("second")); + CHECK_EQ(ac.context, AutocompleteContext::Property); // Alternative key syntax check(R"( @@ -2446,6 +2524,7 @@ local t: Test = { ["f@1"] } ac = autocomplete('1'); CHECK(ac.entryMap.count("first")); CHECK(ac.entryMap.count("second")); + CHECK_EQ(ac.context, AutocompleteContext::Property); // Not an alternative key syntax check(R"( @@ -2456,6 +2535,7 @@ local t: Test = { "f@1" } ac = autocomplete('1'); CHECK(!ac.entryMap.count("first")); CHECK(!ac.entryMap.count("second")); + CHECK_EQ(ac.context, AutocompleteContext::String); // Skip keys that are already defined check(R"( @@ -2466,6 +2546,7 @@ local t: Test = { first = 2, s@1 } ac = autocomplete('1'); CHECK(!ac.entryMap.count("first")); CHECK(ac.entryMap.count("second")); + CHECK_EQ(ac.context, AutocompleteContext::Property); // Don't skip active key check(R"( @@ -2476,6 +2557,7 @@ local t: Test = { first@1 } ac = autocomplete('1'); CHECK(ac.entryMap.count("first")); CHECK(ac.entryMap.count("second")); + CHECK_EQ(ac.context, AutocompleteContext::Property); // Inference after first key check(R"( @@ -2488,6 +2570,7 @@ local t = { ac = autocomplete('1'); CHECK(ac.entryMap.count("first")); CHECK(ac.entryMap.count("second")); + CHECK_EQ(ac.context, AutocompleteContext::Property); check(R"( local t = { @@ -2499,6 +2582,7 @@ local t = { ac = autocomplete('1'); CHECK(ac.entryMap.count("first")); CHECK(ac.entryMap.count("second")); + CHECK_EQ(ac.context, AutocompleteContext::Property); } TEST_CASE_FIXTURE(ACFixture, "autocomplete_documentation_symbols") @@ -2542,6 +2626,7 @@ a = if temp then even elseif true then temp else e@9 CHECK(ac.entryMap.count("then") == 0); CHECK(ac.entryMap.count("else") == 0); CHECK(ac.entryMap.count("elseif") == 0); + CHECK_EQ(ac.context, AutocompleteContext::Expression); ac = autocomplete('2'); CHECK(ac.entryMap.count("temp") == 0); @@ -2549,18 +2634,21 @@ a = if temp then even elseif true then temp else e@9 CHECK(ac.entryMap.count("then")); CHECK(ac.entryMap.count("else") == 0); CHECK(ac.entryMap.count("elseif") == 0); + CHECK_EQ(ac.context, AutocompleteContext::Keyword); ac = autocomplete('3'); CHECK(ac.entryMap.count("even")); CHECK(ac.entryMap.count("then") == 0); CHECK(ac.entryMap.count("else") == 0); CHECK(ac.entryMap.count("elseif") == 0); + CHECK_EQ(ac.context, AutocompleteContext::Expression); ac = autocomplete('4'); CHECK(ac.entryMap.count("even") == 0); CHECK(ac.entryMap.count("then") == 0); CHECK(ac.entryMap.count("else")); CHECK(ac.entryMap.count("elseif")); + CHECK_EQ(ac.context, AutocompleteContext::Keyword); ac = autocomplete('5'); CHECK(ac.entryMap.count("temp")); @@ -2568,6 +2656,7 @@ a = if temp then even elseif true then temp else e@9 CHECK(ac.entryMap.count("then") == 0); CHECK(ac.entryMap.count("else") == 0); CHECK(ac.entryMap.count("elseif") == 0); + CHECK_EQ(ac.context, AutocompleteContext::Expression); ac = autocomplete('6'); CHECK(ac.entryMap.count("temp") == 0); @@ -2575,6 +2664,7 @@ a = if temp then even elseif true then temp else e@9 CHECK(ac.entryMap.count("then")); CHECK(ac.entryMap.count("else") == 0); CHECK(ac.entryMap.count("elseif") == 0); + CHECK_EQ(ac.context, AutocompleteContext::Keyword); ac = autocomplete('7'); CHECK(ac.entryMap.count("temp")); @@ -2582,17 +2672,20 @@ a = if temp then even elseif true then temp else e@9 CHECK(ac.entryMap.count("then") == 0); CHECK(ac.entryMap.count("else") == 0); CHECK(ac.entryMap.count("elseif") == 0); + CHECK_EQ(ac.context, AutocompleteContext::Expression); ac = autocomplete('8'); CHECK(ac.entryMap.count("even") == 0); CHECK(ac.entryMap.count("then") == 0); CHECK(ac.entryMap.count("else")); CHECK(ac.entryMap.count("elseif")); + CHECK_EQ(ac.context, AutocompleteContext::Keyword); ac = autocomplete('9'); CHECK(ac.entryMap.count("then") == 0); CHECK(ac.entryMap.count("else") == 0); CHECK(ac.entryMap.count("elseif") == 0); + CHECK_EQ(ac.context, AutocompleteContext::Expression); } TEST_CASE_FIXTURE(ACFixture, "autocomplete_if_else_regression") @@ -2626,6 +2719,7 @@ local a: A<(number, s@1> CHECK(ac.entryMap.count("number")); CHECK(ac.entryMap.count("string")); + CHECK_EQ(ac.context, AutocompleteContext::Type); } TEST_CASE_FIXTURE(ACFixture, "autocomplete_first_function_arg_expected_type") @@ -2686,6 +2780,7 @@ type A = () -> T CHECK(ac.entryMap.count("number")); CHECK(ac.entryMap.count("string")); + CHECK_EQ(ac.context, AutocompleteContext::Type); } TEST_CASE_FIXTURE(ACFixture, "autocomplete_default_type_pack_parameters") @@ -2698,6 +2793,7 @@ type A = () -> T CHECK(ac.entryMap.count("number")); CHECK(ac.entryMap.count("string")); + CHECK_EQ(ac.context, AutocompleteContext::Type); } TEST_CASE_FIXTURE(ACBuiltinsFixture, "autocomplete_oop_implicit_self") @@ -2752,16 +2848,19 @@ TEST_CASE_FIXTURE(ACFixture, "autocomplete_string_singletons") CHECK(ac.entryMap.count("cat")); CHECK(ac.entryMap.count("dog")); + CHECK_EQ(ac.context, AutocompleteContext::String); ac = autocomplete('2'); CHECK(ac.entryMap.count("\"cat\"")); CHECK(ac.entryMap.count("\"dog\"")); + CHECK_EQ(ac.context, AutocompleteContext::Expression); ac = autocomplete('3'); CHECK(ac.entryMap.count("cat")); CHECK(ac.entryMap.count("dog")); + CHECK_EQ(ac.context, AutocompleteContext::String); check(R"( type tagged = {tag:"cat", fieldx:number} | {tag:"dog", fieldy:number} @@ -2772,6 +2871,7 @@ TEST_CASE_FIXTURE(ACFixture, "autocomplete_string_singletons") CHECK(ac.entryMap.count("cat")); CHECK(ac.entryMap.count("dog")); + CHECK_EQ(ac.context, AutocompleteContext::String); } TEST_CASE_FIXTURE(ACFixture, "autocomplete_string_singleton_equality") @@ -2808,6 +2908,7 @@ f(@1) CHECK(ac.entryMap["true"].typeCorrect == TypeCorrectKind::Correct); REQUIRE(ac.entryMap.count("false")); CHECK(ac.entryMap["false"].typeCorrect == TypeCorrectKind::None); + CHECK_EQ(ac.context, AutocompleteContext::Expression); } TEST_CASE_FIXTURE(ACFixture, "autocomplete_string_singleton_escape") diff --git a/tests/Compiler.test.cpp b/tests/Compiler.test.cpp index 0a1c5a8b..a4ce88b1 100644 --- a/tests/Compiler.test.cpp +++ b/tests/Compiler.test.cpp @@ -2728,46 +2728,44 @@ RETURN R0 0 TEST_CASE("AssignmentConflict") { + ScopedFastFlag sff("LuauCompileOptimalAssignment", true); + // assignments are left to right CHECK_EQ("\n" + compileFunction0("local a, b a, b = 1, 2"), R"( LOADNIL R0 LOADNIL R1 -LOADN R2 1 -LOADN R3 2 -MOVE R0 R2 -MOVE R1 R3 +LOADN R0 1 +LOADN R1 2 RETURN R0 0 )"); - // if assignment of a local invalidates a direct register reference in later assignments, the value is evacuated to a temp register + // if assignment of a local invalidates a direct register reference in later assignments, the value is assigned to a temp register first CHECK_EQ("\n" + compileFunction0("local a a, a[1] = 1, 2"), R"( LOADNIL R0 -MOVE R1 R0 -LOADN R2 1 -LOADN R3 2 -MOVE R0 R2 -SETTABLEN R3 R1 1 +LOADN R1 1 +LOADN R2 2 +SETTABLEN R2 R0 1 +MOVE R0 R1 RETURN R0 0 )"); // note that this doesn't happen if the local assignment happens last naturally CHECK_EQ("\n" + compileFunction0("local a a[1], a = 1, 2"), R"( LOADNIL R0 -LOADN R1 1 -LOADN R2 2 -SETTABLEN R1 R0 1 -MOVE R0 R2 +LOADN R2 1 +LOADN R1 2 +SETTABLEN R2 R0 1 +MOVE R0 R1 RETURN R0 0 )"); // this will happen if assigned register is used in any table expression, including as an object... CHECK_EQ("\n" + compileFunction0("local a a, a.foo = 1, 2"), R"( LOADNIL R0 -MOVE R1 R0 -LOADN R2 1 -LOADN R3 2 -MOVE R0 R2 -SETTABLEKS R3 R1 K0 +LOADN R1 1 +LOADN R2 2 +SETTABLEKS R2 R0 K0 +MOVE R0 R1 RETURN R0 0 )"); @@ -2775,22 +2773,20 @@ RETURN R0 0 CHECK_EQ("\n" + compileFunction0("local a a, foo[a] = 1, 2"), R"( LOADNIL R0 GETIMPORT R1 1 -MOVE R2 R0 -LOADN R3 1 -LOADN R4 2 -MOVE R0 R3 -SETTABLE R4 R1 R2 +LOADN R2 1 +LOADN R3 2 +SETTABLE R3 R1 R0 +MOVE R0 R2 RETURN R0 0 )"); // ... or both ... CHECK_EQ("\n" + compileFunction0("local a a, a[a] = 1, 2"), R"( LOADNIL R0 -MOVE R1 R0 -LOADN R2 1 -LOADN R3 2 -MOVE R0 R2 -SETTABLE R3 R1 R1 +LOADN R1 1 +LOADN R2 2 +SETTABLE R2 R0 R0 +MOVE R0 R1 RETURN R0 0 )"); @@ -2798,14 +2794,12 @@ RETURN R0 0 CHECK_EQ("\n" + compileFunction0("local a, b a, b, a[b] = 1, 2, 3"), R"( LOADNIL R0 LOADNIL R1 -MOVE R2 R0 -MOVE R3 R1 -LOADN R4 1 -LOADN R5 2 -LOADN R6 3 -MOVE R0 R4 -MOVE R1 R5 -SETTABLE R6 R2 R3 +LOADN R2 1 +LOADN R3 2 +LOADN R4 3 +SETTABLE R4 R0 R1 +MOVE R0 R2 +MOVE R1 R3 RETURN R0 0 )"); @@ -2815,10 +2809,9 @@ RETURN R0 0 LOADNIL R0 GETIMPORT R1 1 ADDK R2 R0 K2 -LOADN R3 1 -LOADN R4 2 -MOVE R0 R3 -SETTABLE R4 R1 R2 +LOADN R0 1 +LOADN R3 2 +SETTABLE R3 R1 R2 RETURN R0 0 )"); } @@ -6242,4 +6235,228 @@ RETURN R2 1 )"); } +TEST_CASE("MultipleAssignments") +{ + ScopedFastFlag sff("LuauCompileOptimalAssignment", true); + + // order of assignments is left to right + CHECK_EQ("\n" + compileFunction0(R"( + local a, b + a, b = f(1), f(2) + )"), + R"( +LOADNIL R0 +LOADNIL R1 +GETIMPORT R2 1 +LOADN R3 1 +CALL R2 1 1 +MOVE R0 R2 +GETIMPORT R2 1 +LOADN R3 2 +CALL R2 1 1 +MOVE R1 R2 +RETURN R0 0 +)"); + + // this includes table assignments + CHECK_EQ("\n" + compileFunction0(R"( + local t + t[1], t[2] = 3, 4 + )"), + R"( +LOADNIL R0 +LOADNIL R1 +LOADN R2 3 +LOADN R3 4 +SETTABLEN R2 R0 1 +SETTABLEN R3 R1 2 +RETURN R0 0 +)"); + + // semantically, we evaluate the right hand side first; this allows us to e.g swap elements in a table easily + CHECK_EQ("\n" + compileFunction0(R"( + local t = ... + t[1], t[2] = t[2], t[1] + )"), + R"( +GETVARARGS R0 1 +GETTABLEN R1 R0 2 +GETTABLEN R2 R0 1 +SETTABLEN R1 R0 1 +SETTABLEN R2 R0 2 +RETURN R0 0 +)"); + + // however, we need to optimize local assignments; to do this well, we need to handle assignment conflicts + // let's first go through a few cases where there are no conflicts: + + // when multiple assignments have no conflicts (all local vars are read after being assigned), codegen is the same as a series of single + // assignments + CHECK_EQ("\n" + compileFunction0(R"( + local xm1, x, xp1, xi = ... + + xm1,x,xp1,xi = x,xp1,xp1+1,xi-1 + )"), + R"( +GETVARARGS R0 4 +MOVE R0 R1 +MOVE R1 R2 +ADDK R2 R2 K0 +SUBK R3 R3 K0 +RETURN R0 0 +)"); + + // similar example to above from a more complex case + CHECK_EQ("\n" + compileFunction0(R"( + local a, b, c, d, e, f, g, h, t1, t2 = ... + + h, g, f, e, d, c, b, a = g, f, e, d + t1, c, b, a, t1 + t2 + )"), + R"( +GETVARARGS R0 10 +MOVE R7 R6 +MOVE R6 R5 +MOVE R5 R4 +ADD R4 R3 R8 +MOVE R3 R2 +MOVE R2 R1 +MOVE R1 R0 +ADD R0 R8 R9 +RETURN R0 0 +)"); + + // when locals have a conflict, we assign temporaries instead of locals, and at the end copy the values back + // the basic example of this is a swap/rotate + CHECK_EQ("\n" + compileFunction0(R"( + local a, b = ... + a, b = b, a + )"), + R"( +GETVARARGS R0 2 +MOVE R2 R1 +MOVE R1 R0 +MOVE R0 R2 +RETURN R0 0 +)"); + + CHECK_EQ("\n" + compileFunction0(R"( + local a, b, c = ... + a, b, c = c, a, b + )"), + R"( +GETVARARGS R0 3 +MOVE R3 R2 +MOVE R4 R0 +MOVE R2 R1 +MOVE R0 R3 +MOVE R1 R4 +RETURN R0 0 +)"); + + CHECK_EQ("\n" + compileFunction0(R"( + local a, b, c = ... + a, b, c = b, c, a + )"), + R"( +GETVARARGS R0 3 +MOVE R3 R1 +MOVE R1 R2 +MOVE R2 R0 +MOVE R0 R3 +RETURN R0 0 +)"); + + // multiple assignments with multcall handling - foo() evalutes to temporary registers and they are copied out to target + CHECK_EQ("\n" + compileFunction0(R"( + local a, b, c, d = ... + a, b, c, d = 1, foo() + )"), + R"( +GETVARARGS R0 4 +LOADN R0 1 +GETIMPORT R4 1 +CALL R4 0 3 +MOVE R1 R4 +MOVE R2 R5 +MOVE R3 R6 +RETURN R0 0 +)"); + + // note that during this we still need to handle local reassignment, eg when table assignments are performed + CHECK_EQ("\n" + compileFunction0(R"( + local a, b, c, d = ... + a, b[a], c[d], d = 1, foo() + )"), + R"( +GETVARARGS R0 4 +LOADN R4 1 +GETIMPORT R6 1 +CALL R6 0 3 +SETTABLE R6 R1 R0 +SETTABLE R7 R2 R3 +MOVE R0 R4 +MOVE R3 R8 +RETURN R0 0 +)"); + + // multiple assignments with multcall handling - foo evaluates to a single argument so all remaining locals are assigned to nil + // note that here we don't assign the locals directly, as this case is very rare so we use the similar code path as above + CHECK_EQ("\n" + compileFunction0(R"( + local a, b, c, d = ... + a, b, c, d = 1, foo + )"), + R"( +GETVARARGS R0 4 +LOADN R0 1 +GETIMPORT R4 1 +LOADNIL R5 +LOADNIL R6 +MOVE R1 R4 +MOVE R2 R5 +MOVE R3 R6 +RETURN R0 0 +)"); + + // note that we also try to use locals as a source of assignment directly when assigning fields; this works using old local value when possible + CHECK_EQ("\n" + compileFunction0(R"( + local a, b = ... + a[1], a[2] = b, b + 1 + )"), + R"( +GETVARARGS R0 2 +ADDK R2 R1 K0 +SETTABLEN R1 R0 1 +SETTABLEN R2 R0 2 +RETURN R0 0 +)"); + + // ... of course if the local is reassigned, we defer the assignment until later + CHECK_EQ("\n" + compileFunction0(R"( + local a, b = ... + b, a[1] = 42, b + )"), + R"( +GETVARARGS R0 2 +LOADN R2 42 +SETTABLEN R1 R0 1 +MOVE R1 R2 +RETURN R0 0 +)"); + + // when there are more expressions when values, we evalute them for side effects, but they also participate in conflict handling + CHECK_EQ("\n" + compileFunction0(R"( + local a, b = ... + a, b = 1, 2, a + b + )"), + R"( +GETVARARGS R0 2 +LOADN R2 1 +LOADN R3 2 +ADD R4 R0 R1 +MOVE R0 R2 +MOVE R1 R3 +RETURN R0 0 +)"); +} + TEST_SUITE_END(); diff --git a/tests/Fixture.cpp b/tests/Fixture.cpp index f51a9d1b..f0146602 100644 --- a/tests/Fixture.cpp +++ b/tests/Fixture.cpp @@ -372,17 +372,25 @@ void Fixture::registerTestTypes() void Fixture::dumpErrors(const CheckResult& cr) { - dumpErrors(std::cout, cr.errors); + std::string error = getErrors(cr); + if (!error.empty()) + MESSAGE(error); } void Fixture::dumpErrors(const ModulePtr& module) { - dumpErrors(std::cout, module->errors); + std::stringstream ss; + dumpErrors(ss, module->errors); + if (!ss.str().empty()) + MESSAGE(ss.str()); } void Fixture::dumpErrors(const Module& module) { - dumpErrors(std::cout, module.errors); + std::stringstream ss; + dumpErrors(ss, module.errors); + if (!ss.str().empty()) + MESSAGE(ss.str()); } std::string Fixture::getErrors(const CheckResult& cr) @@ -413,6 +421,7 @@ LoadDefinitionFileResult Fixture::loadDefinition(const std::string& source) LoadDefinitionFileResult result = frontend.loadDefinitionFile(source, "@test"); freeze(typeChecker.globalTypes); + dumpErrors(result.module); REQUIRE_MESSAGE(result.success, "loadDefinition: unable to load definition file"); return result; } @@ -434,7 +443,7 @@ BuiltinsFixture::BuiltinsFixture(bool freeze, bool prepareAutocomplete) ConstraintGraphBuilderFixture::ConstraintGraphBuilderFixture() : Fixture() - , cgb(mainModuleName, &arena, NotNull(&ice), frontend.getGlobalScope()) + , cgb(mainModuleName, getMainModule(), &arena, NotNull(&ice), frontend.getGlobalScope()) , forceTheFlag{"DebugLuauDeferredConstraintResolution", true} { BlockedTypeVar::nextIndex = 0; diff --git a/tests/Linter.test.cpp b/tests/Linter.test.cpp index 35c40508..64c6d3e4 100644 --- a/tests/Linter.test.cpp +++ b/tests/Linter.test.cpp @@ -1675,7 +1675,7 @@ TEST_CASE_FIXTURE(Fixture, "WrongCommentOptimize") CHECK_EQ(result.warnings[3].text, "optimize directive uses unknown optimization level '100500', 0..2 expected"); } -TEST_CASE_FIXTURE(Fixture, "LintIntegerParsing") +TEST_CASE_FIXTURE(Fixture, "IntegerParsing") { ScopedFastFlag luauLintParseIntegerIssues{"LuauLintParseIntegerIssues", true}; @@ -1690,7 +1690,7 @@ local _ = 0x10000000000000000 } // TODO: remove with FFlagLuauErrorDoubleHexPrefix -TEST_CASE_FIXTURE(Fixture, "LintIntegerParsingDoublePrefix") +TEST_CASE_FIXTURE(Fixture, "IntegerParsingDoublePrefix") { ScopedFastFlag luauLintParseIntegerIssues{"LuauLintParseIntegerIssues", true}; ScopedFastFlag luauErrorDoubleHexPrefix{"LuauErrorDoubleHexPrefix", false}; // Lint will be available until we start rejecting code @@ -1707,4 +1707,36 @@ local _ = 0x0xffffffffffffffffffffffffffffffffff "Hexadecimal number literal has a double prefix, which will fail to parse in the future; remove the extra 0x to fix"); } +TEST_CASE_FIXTURE(Fixture, "ComparisonPrecedence") +{ + ScopedFastFlag sff("LuauLintComparisonPrecedence", true); + + LintResult result = lint(R"( +local a, b = ... + +local _ = not a == b +local _ = not a ~= b +local _ = not a <= b +local _ = a <= b == 0 + +local _ = not a == not b -- weird but ok + +-- silence tests for all of the above +local _ = not (a == b) +local _ = (not a) == b +local _ = not (a ~= b) +local _ = (not a) ~= b +local _ = not (a <= b) +local _ = (not a) <= b +local _ = (a <= b) == 0 +local _ = a <= (b == 0) +)"); + + REQUIRE_EQ(result.warnings.size(), 4); + CHECK_EQ(result.warnings[0].text, "not X == Y is equivalent to (not X) == Y; consider using X ~= Y, or wrap one of the expressions in parentheses to silence"); + CHECK_EQ(result.warnings[1].text, "not X ~= Y is equivalent to (not X) ~= Y; consider using X == Y, or wrap one of the expressions in parentheses to silence"); + CHECK_EQ(result.warnings[2].text, "not X <= Y is equivalent to (not X) <= Y; wrap one of the expressions in parentheses to silence"); + CHECK_EQ(result.warnings[3].text, "X <= Y == Z is equivalent to (X <= Y) == Z; wrap one of the expressions in parentheses to silence"); +} + TEST_SUITE_END(); diff --git a/tests/Module.test.cpp b/tests/Module.test.cpp index dd94e9d7..5ec375c1 100644 --- a/tests/Module.test.cpp +++ b/tests/Module.test.cpp @@ -317,4 +317,78 @@ type B = A CHECK(toString(it->second.type) == "any"); } +TEST_CASE_FIXTURE(BuiltinsFixture, "do_not_clone_reexports") +{ + ScopedFastFlag flags[] = { + {"LuauClonePublicInterfaceLess", true}, + {"LuauSubstitutionReentrant", true}, + {"LuauClassTypeVarsInSubstitution", true}, + {"LuauSubstitutionFixMissingFields", true}, + }; + + fileResolver.source["Module/A"] = R"( +export type A = {p : number} +return {} + )"; + + fileResolver.source["Module/B"] = R"( +local a = require(script.Parent.A) +export type B = {q : a.A} +return {} + )"; + + CheckResult result = frontend.check("Module/B"); + LUAU_REQUIRE_NO_ERRORS(result); + + ModulePtr modA = frontend.moduleResolver.getModule("Module/A"); + ModulePtr modB = frontend.moduleResolver.getModule("Module/B"); + REQUIRE(modA); + REQUIRE(modB); + auto modAiter = modA->getModuleScope()->exportedTypeBindings.find("A"); + auto modBiter = modB->getModuleScope()->exportedTypeBindings.find("B"); + REQUIRE(modAiter != modA->getModuleScope()->exportedTypeBindings.end()); + REQUIRE(modBiter != modB->getModuleScope()->exportedTypeBindings.end()); + TypeId typeA = modAiter->second.type; + TypeId typeB = modBiter->second.type; + TableTypeVar* tableB = getMutable(typeB); + REQUIRE(tableB); + CHECK(typeA == tableB->props["q"].type); +} + +TEST_CASE_FIXTURE(BuiltinsFixture, "do_not_clone_types_of_reexported_values") +{ + ScopedFastFlag flags[] = { + {"LuauClonePublicInterfaceLess", true}, + {"LuauSubstitutionReentrant", true}, + {"LuauClassTypeVarsInSubstitution", true}, + {"LuauSubstitutionFixMissingFields", true}, + }; + + fileResolver.source["Module/A"] = R"( +local exports = {a={p=5}} +return exports + )"; + + fileResolver.source["Module/B"] = R"( +local a = require(script.Parent.A) +local exports = {b=a.a} +return exports + )"; + + CheckResult result = frontend.check("Module/B"); + LUAU_REQUIRE_NO_ERRORS(result); + + ModulePtr modA = frontend.moduleResolver.getModule("Module/A"); + ModulePtr modB = frontend.moduleResolver.getModule("Module/B"); + REQUIRE(modA); + REQUIRE(modB); + std::optional typeA = first(modA->getModuleScope()->returnType); + std::optional typeB = first(modB->getModuleScope()->returnType); + REQUIRE(typeA); + REQUIRE(typeB); + TableTypeVar* tableA = getMutable(*typeA); + TableTypeVar* tableB = getMutable(*typeB); + CHECK(tableA->props["a"].type == tableB->props["b"].type); +} + TEST_SUITE_END(); diff --git a/tests/NonstrictMode.test.cpp b/tests/NonstrictMode.test.cpp index 02e02e6b..89aab5ee 100644 --- a/tests/NonstrictMode.test.cpp +++ b/tests/NonstrictMode.test.cpp @@ -169,6 +169,7 @@ TEST_CASE_FIXTURE(Fixture, "table_props_are_any") REQUIRE(ttv != nullptr); + REQUIRE(ttv->props.count("foo")); TypeId fooProp = ttv->props["foo"].type; REQUIRE(fooProp != nullptr); diff --git a/tests/TypeInfer.definitions.test.cpp b/tests/TypeInfer.definitions.test.cpp index 4545b8db..fd6fb83f 100644 --- a/tests/TypeInfer.definitions.test.cpp +++ b/tests/TypeInfer.definitions.test.cpp @@ -11,6 +11,33 @@ using namespace Luau; TEST_SUITE_BEGIN("DefinitionTests"); +TEST_CASE_FIXTURE(Fixture, "definition_file_simple") +{ + loadDefinition(R"( + declare foo: number + declare function bar(x: number): string + declare foo2: typeof(foo) + )"); + + TypeId globalFooTy = getGlobalBinding(frontend.typeChecker, "foo"); + CHECK_EQ(toString(globalFooTy), "number"); + + TypeId globalBarTy = getGlobalBinding(frontend.typeChecker, "bar"); + CHECK_EQ(toString(globalBarTy), "(number) -> string"); + + TypeId globalFoo2Ty = getGlobalBinding(frontend.typeChecker, "foo2"); + CHECK_EQ(toString(globalFoo2Ty), "number"); + + CheckResult result = check(R"( + local x: number = foo - 1 + local y: string = bar(x) + local z: number | string = x + z = y + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + TEST_CASE_FIXTURE(Fixture, "definition_file_loading") { loadDefinition(R"( diff --git a/tests/TypeInfer.generics.test.cpp b/tests/TypeInfer.generics.test.cpp index a8325727..9ac259cf 100644 --- a/tests/TypeInfer.generics.test.cpp +++ b/tests/TypeInfer.generics.test.cpp @@ -845,6 +845,7 @@ TEST_CASE_FIXTURE(Fixture, "generic_table_method") TableTypeVar* tTable = getMutable(tType); REQUIRE(tTable != nullptr); + REQUIRE(tTable->props.count("bar")); TypeId barType = tTable->props["bar"].type; REQUIRE(barType != nullptr); diff --git a/tests/TypeInfer.modules.test.cpp b/tests/TypeInfer.modules.test.cpp index a1d41339..754fb193 100644 --- a/tests/TypeInfer.modules.test.cpp +++ b/tests/TypeInfer.modules.test.cpp @@ -398,8 +398,6 @@ caused by: TEST_CASE_FIXTURE(BuiltinsFixture, "constrained_anyification_clone_immutable_types") { - ScopedFastFlag luauAnyificationMustClone{"LuauAnyificationMustClone", true}; - fileResolver.source["game/A"] = R"( return function(...) end )"; diff --git a/tests/TypeInfer.tables.test.cpp b/tests/TypeInfer.tables.test.cpp index d9bfc89d..8c7d8a53 100644 --- a/tests/TypeInfer.tables.test.cpp +++ b/tests/TypeInfer.tables.test.cpp @@ -1847,6 +1847,7 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "quantifying_a_bound_var_works") TypeId ty = requireType("clazz"); TableTypeVar* ttv = getMutable(ty); REQUIRE(ttv); + REQUIRE(ttv->props.count("new")); Property& prop = ttv->props["new"]; REQUIRE(prop.type); const FunctionTypeVar* ftv = get(follow(prop.type)); @@ -2516,6 +2517,7 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "dont_quantify_table_that_belongs_to_outer_sc TableTypeVar* counterType = getMutable(requireType("Counter")); REQUIRE(counterType); + REQUIRE(counterType->props.count("new")); const FunctionTypeVar* newType = get(follow(counterType->props["new"].type)); REQUIRE(newType); @@ -3081,8 +3083,6 @@ TEST_CASE_FIXTURE(Fixture, "quantify_even_that_table_was_never_exported_at_all") TEST_CASE_FIXTURE(BuiltinsFixture, "leaking_bad_metatable_errors") { - ScopedFastFlag luauIndexSilenceErrors{"LuauIndexSilenceErrors", true}; - CheckResult result = check(R"( local a = setmetatable({}, 1) local b = a.x diff --git a/tests/TypeVar.test.cpp b/tests/TypeVar.test.cpp index f4670048..edec8442 100644 --- a/tests/TypeVar.test.cpp +++ b/tests/TypeVar.test.cpp @@ -182,6 +182,17 @@ TEST_CASE_FIXTURE(Fixture, "UnionTypeVarIterator_with_empty_union") CHECK(actual.empty()); } +TEST_CASE_FIXTURE(Fixture, "UnionTypeVarIterator_with_only_cyclic_union") +{ + TypeVar tv{UnionTypeVar{}}; + auto utv = getMutable(&tv); + utv->options.push_back(&tv); + utv->options.push_back(&tv); + + std::vector actual(begin(utv), end(utv)); + CHECK(actual.empty()); +} + TEST_CASE_FIXTURE(Fixture, "substitution_skip_failure") { TypeVar ftv11{FreeTypeVar{TypeLevel{}}}; diff --git a/tests/conformance/basic.lua b/tests/conformance/basic.lua index b2dcaf94..d9274751 100644 --- a/tests/conformance/basic.lua +++ b/tests/conformance/basic.lua @@ -49,6 +49,12 @@ assert((function() _G.foo = 1 return _G['foo'] end)() == 1) assert((function() _G['bar'] = 1 return _G.bar end)() == 1) assert((function() local a = 1 (function () a = 2 end)() return a end)() == 2) +-- assignments with local conflicts +assert((function() local a, b = 1, {} a, b[a] = 43, -1 return a + b[1] end)() == 42) +assert((function() local a = {} local b = a a[1], a = 43, -1 return a + b[1] end)() == 42) +assert((function() local a, b = 1, {} a, b[a] = (function() return 43, -1 end)() return a + b[1] end)() == 42) +assert((function() local a = {} local b = a a[1], a = (function() return 43, -1 end)() return a + b[1] end)() == 42) + -- upvalues assert((function() local a = 1 function foo() return a end return foo() end)() == 1) diff --git a/tests/conformance/errors.lua b/tests/conformance/errors.lua index 7ab7099e..b69d437b 100644 --- a/tests/conformance/errors.lua +++ b/tests/conformance/errors.lua @@ -295,8 +295,9 @@ end -- testing syntax limits +local syntaxdepth = if limitedstack then 200 else 500 local function testrep (init, rep) - local s = "local a; "..init .. string.rep(rep, 300) + local s = "local a; "..init .. string.rep(rep, syntaxdepth) local a,b = loadstring(s) assert(not a) -- and string.find(b, "syntax levels")) end diff --git a/tests/conformance/strings.lua b/tests/conformance/strings.lua index c87cf15c..3d8fdd1f 100644 --- a/tests/conformance/strings.lua +++ b/tests/conformance/strings.lua @@ -145,6 +145,14 @@ end) == false) assert(string.format("%*", "a\0b\0c") == "a\0b\0c") assert(string.format("%*", string.rep("doge", 3000)) == string.rep("doge", 3000)) +assert(string.format("%*", 42) == "42") +assert(string.format("%*", true) == "true") + +assert(string.format("%*", setmetatable({}, { __tostring = function() return "ok" end })) == "ok") + +local ud = newproxy(true) +getmetatable(ud).__tostring = function() return "good" end +assert(string.format("%*", ud) == "good") assert(pcall(function() string.format("%#*", "bad form") diff --git a/tools/faillist.txt b/tools/faillist.txt index 6e93345b..d796236d 100644 --- a/tools/faillist.txt +++ b/tools/faillist.txt @@ -1,13 +1,8 @@ -AnnotationTests.as_expr_does_not_propagate_type_info -AnnotationTests.as_expr_is_bidirectional -AnnotationTests.as_expr_warns_on_unrelated_cast AnnotationTests.builtin_types_are_not_exported AnnotationTests.cannot_use_nonexported_type AnnotationTests.cloned_interface_maintains_pointers_between_definitions -AnnotationTests.define_generic_type_alias AnnotationTests.duplicate_type_param_name AnnotationTests.for_loop_counter_annotation_is_checked -AnnotationTests.function_return_annotations_are_checked AnnotationTests.generic_aliases_are_cloned_properly AnnotationTests.interface_types_belong_to_interface_arena AnnotationTests.luau_ice_triggers_an_ice @@ -18,21 +13,14 @@ AnnotationTests.luau_print_is_magic_if_the_flag_is_set AnnotationTests.luau_print_is_not_special_without_the_flag AnnotationTests.occurs_check_on_cyclic_intersection_typevar AnnotationTests.occurs_check_on_cyclic_union_typevar -AnnotationTests.self_referential_type_alias AnnotationTests.too_many_type_params AnnotationTests.two_type_params -AnnotationTests.type_annotations_inside_function_bodies -AnnotationTests.type_assertion_expr AnnotationTests.unknown_type_reference_generates_error AnnotationTests.use_type_required_from_another_file AstQuery.last_argument_function_call_type -AstQuery::getDocumentationSymbolAtPosition.binding -AstQuery::getDocumentationSymbolAtPosition.event_callback_arg AstQuery::getDocumentationSymbolAtPosition.overloaded_fn -AstQuery::getDocumentationSymbolAtPosition.prop AutocompleteTest.argument_types AutocompleteTest.arguments_to_global_lambda -AutocompleteTest.as_types AutocompleteTest.autocomplete_boolean_singleton AutocompleteTest.autocomplete_end_with_fn_exprs AutocompleteTest.autocomplete_end_with_lambda @@ -127,7 +115,6 @@ BuiltinTests.assert_removes_falsy_types2 BuiltinTests.assert_removes_falsy_types_even_from_type_pack_tail_but_only_for_the_first_type BuiltinTests.assert_returns_false_and_string_iff_it_knows_the_first_argument_cannot_be_truthy BuiltinTests.bad_select_should_not_crash -BuiltinTests.builtin_tables_sealed BuiltinTests.coroutine_resume_anything_goes BuiltinTests.coroutine_wrap_anything_goes BuiltinTests.debug_info_is_crazy @@ -136,28 +123,20 @@ BuiltinTests.dont_add_definitions_to_persistent_types BuiltinTests.find_capture_types BuiltinTests.find_capture_types2 BuiltinTests.find_capture_types3 -BuiltinTests.gcinfo BuiltinTests.getfenv BuiltinTests.global_singleton_types_are_sealed BuiltinTests.gmatch_capture_types BuiltinTests.gmatch_capture_types2 BuiltinTests.gmatch_capture_types_balanced_escaped_parens BuiltinTests.gmatch_capture_types_default_capture -BuiltinTests.gmatch_capture_types_invalid_pattern_fallback_to_builtin -BuiltinTests.gmatch_capture_types_invalid_pattern_fallback_to_builtin2 -BuiltinTests.gmatch_capture_types_leading_end_bracket_is_part_of_set BuiltinTests.gmatch_capture_types_parens_in_sets_are_ignored BuiltinTests.gmatch_capture_types_set_containing_lbracket BuiltinTests.gmatch_definition BuiltinTests.ipairs_iterator_should_infer_types_and_type_check -BuiltinTests.lua_51_exported_globals_all_exist BuiltinTests.match_capture_types BuiltinTests.match_capture_types2 BuiltinTests.math_max_checks_for_numbers -BuiltinTests.math_max_variatic -BuiltinTests.math_things_are_defined BuiltinTests.next_iterator_should_infer_types_and_type_check -BuiltinTests.no_persistent_typelevel_change BuiltinTests.os_time_takes_optional_date_table BuiltinTests.pairs_iterator_should_infer_types_and_type_check BuiltinTests.see_thru_select @@ -170,7 +149,6 @@ BuiltinTests.select_with_variadic_typepack_tail BuiltinTests.select_with_variadic_typepack_tail_and_string_head BuiltinTests.set_metatable_needs_arguments BuiltinTests.setmetatable_should_not_mutate_persisted_types -BuiltinTests.setmetatable_unpacks_arg_types_correctly BuiltinTests.sort BuiltinTests.sort_with_bad_predicate BuiltinTests.sort_with_predicate @@ -179,6 +157,8 @@ BuiltinTests.string_format_arg_types_inference BuiltinTests.string_format_as_method BuiltinTests.string_format_correctly_ordered_types BuiltinTests.string_format_report_all_type_errors_at_correct_positions +BuiltinTests.string_format_tostring_specifier +BuiltinTests.string_format_tostring_specifier_type_constraint BuiltinTests.string_format_use_correct_argument BuiltinTests.string_format_use_correct_argument2 BuiltinTests.string_lib_self_noself @@ -190,53 +170,38 @@ BuiltinTests.table_insert_correctly_infers_type_of_array_3_args_overload BuiltinTests.table_pack BuiltinTests.table_pack_reduce BuiltinTests.table_pack_variadic -BuiltinTests.thread_is_a_type BuiltinTests.tonumber_returns_optional_number_type BuiltinTests.tonumber_returns_optional_number_type2 -BuiltinTests.xpcall -DefinitionTests.class_definition_function_prop DefinitionTests.declaring_generic_functions -DefinitionTests.definition_file_class_function_args DefinitionTests.definition_file_classes DefinitionTests.definition_file_loading +DefinitionTests.definitions_documentation_symbols +DefinitionTests.documentation_symbols_dont_attach_to_persistent_types DefinitionTests.single_class_type_identity_in_global_types -FrontendTest.accumulate_cached_errors -FrontendTest.accumulate_cached_errors_in_consistent_order -FrontendTest.any_annotation_breaks_cycle FrontendTest.ast_node_at_position -FrontendTest.automatically_check_cyclically_dependent_scripts FrontendTest.automatically_check_dependent_scripts FrontendTest.check_without_builtin_next FrontendTest.clearStats -FrontendTest.cycle_detection_between_check_and_nocheck -FrontendTest.cycle_detection_disabled_in_nocheck -FrontendTest.cycle_error_paths -FrontendTest.cycle_errors_can_be_fixed -FrontendTest.cycle_incremental_type_surface -FrontendTest.cycle_incremental_type_surface_longer -FrontendTest.dont_recheck_script_that_hasnt_been_marked_dirty FrontendTest.dont_reparse_clean_file_when_linting FrontendTest.environments -FrontendTest.ignore_require_to_nonexistent_file FrontendTest.imported_table_modification_2 FrontendTest.it_should_be_safe_to_stringify_errors_when_full_type_graph_is_discarded FrontendTest.no_use_after_free_with_type_fun_instantiation FrontendTest.nocheck_cycle_used_by_checked FrontendTest.nocheck_modules_are_typed FrontendTest.produce_errors_for_unchanged_file_with_a_syntax_error -FrontendTest.re_report_type_error_in_required_file FrontendTest.recheck_if_dependent_script_is_dirty FrontendTest.reexport_cyclic_type FrontendTest.reexport_type_alias FrontendTest.report_require_to_nonexistent_file FrontendTest.report_syntax_error_in_required_file -FrontendTest.reports_errors_from_multiple_sources FrontendTest.stats_are_not_reset_between_checks FrontendTest.trace_requires_in_nonstrict_mode GenericsTests.apply_type_function_nested_generics1 GenericsTests.apply_type_function_nested_generics2 GenericsTests.better_mismatch_error_messages GenericsTests.bound_tables_do_not_clone_original_fields +GenericsTests.calling_self_generic_methods GenericsTests.check_generic_typepack_function GenericsTests.check_mutual_generic_functions GenericsTests.correctly_instantiate_polymorphic_member_functions @@ -265,8 +230,7 @@ GenericsTests.generic_type_pack_unification3 GenericsTests.infer_generic_function_function_argument GenericsTests.infer_generic_function_function_argument_overloaded GenericsTests.infer_generic_lib_function_function_argument -GenericsTests.infer_generic_property -GenericsTests.inferred_local_vars_can_be_polytypes +GenericsTests.infer_generic_methods GenericsTests.instantiate_cyclic_generic_function GenericsTests.instantiate_generic_function_in_assignments GenericsTests.instantiate_generic_function_in_assignments2 @@ -276,7 +240,6 @@ GenericsTests.local_vars_can_be_instantiated_polytypes GenericsTests.mutable_state_polymorphism GenericsTests.no_stack_overflow_from_quantifying GenericsTests.properties_can_be_instantiated_polytypes -GenericsTests.properties_can_be_polytypes GenericsTests.rank_N_types_via_typeof GenericsTests.reject_clashing_generic_and_pack_names GenericsTests.self_recursive_instantiated_param @@ -287,30 +250,23 @@ IntersectionTypes.error_detailed_intersection_part IntersectionTypes.fx_intersection_as_argument IntersectionTypes.fx_union_as_argument_fails IntersectionTypes.index_on_an_intersection_type_with_mixed_types -IntersectionTypes.index_on_an_intersection_type_with_one_part_missing_the_property -IntersectionTypes.index_on_an_intersection_type_with_one_property_of_type_any IntersectionTypes.index_on_an_intersection_type_with_property_guaranteed_to_exist IntersectionTypes.index_on_an_intersection_type_works_at_arbitrary_depth IntersectionTypes.no_stack_overflow_from_flattenintersection IntersectionTypes.overload_is_not_a_function IntersectionTypes.select_correct_union_fn IntersectionTypes.should_still_pick_an_overload_whose_arguments_are_unions -IntersectionTypes.table_intersection_setmetatable IntersectionTypes.table_intersection_write IntersectionTypes.table_intersection_write_sealed IntersectionTypes.table_intersection_write_sealed_indirect IntersectionTypes.table_write_sealed_indirect -isSubtype.functions_and_any -isSubtype.intersection_of_functions_of_different_arities isSubtype.intersection_of_tables -isSubtype.table_with_any_prop isSubtype.table_with_table_prop -isSubtype.tables -Linter.DeprecatedApi Linter.TableOperations -ModuleTests.builtin_types_point_into_globalTypes_arena ModuleTests.clone_self_property ModuleTests.deepClone_cyclic_table +ModuleTests.do_not_clone_reexports +ModuleTests.do_not_clone_types_of_reexported_values NonstrictModeTests.delay_function_does_not_require_its_argument_to_return_anything NonstrictModeTests.for_in_iterator_variables_are_any NonstrictModeTests.function_parameters_are_any @@ -333,7 +289,6 @@ Normalize.cyclic_intersection Normalize.cyclic_table_normalizes_sensibly Normalize.cyclic_union Normalize.fuzz_failure_bound_type_is_normal_but_not_its_bounded_to -Normalize.fuzz_failure_instersection_combine_must_follow Normalize.higher_order_function Normalize.intersection_combine_on_bound_self Normalize.intersection_inside_a_table_inside_another_intersection @@ -345,13 +300,11 @@ Normalize.intersection_of_disjoint_tables Normalize.intersection_of_functions Normalize.intersection_of_overlapping_tables Normalize.intersection_of_tables_with_indexers -Normalize.nested_table_normalization_with_non_table__no_ice Normalize.normalization_does_not_convert_ever Normalize.normalize_module_return_type Normalize.normalize_unions_containing_never Normalize.normalize_unions_containing_unknown Normalize.return_type_is_not_a_constrained_intersection -Normalize.skip_force_normal_on_external_types Normalize.union_of_distinct_free_types Normalize.variadic_tail_is_marked_normal Normalize.visiting_a_type_twice_is_not_considered_normal @@ -365,7 +318,6 @@ ProvisionalTests.constrained_is_level_dependent ProvisionalTests.discriminate_from_x_not_equal_to_nil ProvisionalTests.do_not_ice_when_trying_to_pick_first_of_generic_type_pack ProvisionalTests.error_on_eq_metamethod_returning_a_type_other_than_boolean -ProvisionalTests.free_is_not_bound_to_any ProvisionalTests.function_returns_many_things_but_first_of_it_is_forgotten ProvisionalTests.greedy_inference_with_shared_self_triggers_function_with_no_returns ProvisionalTests.invariant_table_properties_means_instantiating_tables_in_call_is_unsound @@ -380,7 +332,6 @@ ProvisionalTests.typeguard_inference_incomplete ProvisionalTests.weird_fail_to_unify_type_pack ProvisionalTests.weirditer_should_not_loop_forever ProvisionalTests.while_body_are_also_refined -ProvisionalTests.xpcall_returns_what_f_returns RefinementTest.and_constraint RefinementTest.and_or_peephole_refinement RefinementTest.apply_refinements_on_astexprindexexpr_whose_subscript_expr_is_constant_string @@ -420,7 +371,6 @@ RefinementTest.not_and_constraint RefinementTest.not_t_or_some_prop_of_t RefinementTest.or_predicate_with_truthy_predicates RefinementTest.parenthesized_expressions_are_followed_through -RefinementTest.refine_a_property_not_to_be_nil_through_an_intersection_table RefinementTest.refine_the_correct_types_opposite_of_when_a_is_not_number_or_string RefinementTest.refine_unknowns RefinementTest.string_not_equal_to_string_or_nil @@ -456,7 +406,6 @@ TableTests.augment_nested_table TableTests.augment_table TableTests.builtin_table_names TableTests.call_method -TableTests.call_method_with_explicit_self_argument TableTests.cannot_augment_sealed_table TableTests.cannot_call_tables TableTests.cannot_change_type_of_unsealed_table_prop @@ -469,16 +418,13 @@ TableTests.common_table_element_union_in_call_tail TableTests.confusing_indexing TableTests.defining_a_method_for_a_builtin_sealed_table_must_fail TableTests.defining_a_method_for_a_local_sealed_table_must_fail -TableTests.defining_a_method_for_a_local_unsealed_table_is_ok TableTests.defining_a_self_method_for_a_builtin_sealed_table_must_fail TableTests.defining_a_self_method_for_a_local_sealed_table_must_fail -TableTests.defining_a_self_method_for_a_local_unsealed_table_is_ok TableTests.dont_crash_when_setmetatable_does_not_produce_a_metatabletypevar TableTests.dont_hang_when_trying_to_look_up_in_cyclic_metatable_index TableTests.dont_invalidate_the_properties_iterator_of_free_table_when_rolled_back TableTests.dont_leak_free_table_props TableTests.dont_quantify_table_that_belongs_to_outer_scope -TableTests.dont_seal_an_unsealed_table_by_passing_it_to_a_function_that_takes_a_sealed_table TableTests.dont_suggest_exact_match_keys TableTests.error_detailed_indexer_key TableTests.error_detailed_indexer_value @@ -508,8 +454,6 @@ TableTests.infer_array_2 TableTests.infer_indexer_from_value_property_in_literal TableTests.inferred_return_type_of_free_table TableTests.inferring_crazy_table_should_also_be_quick -TableTests.instantiate_table_cloning -TableTests.instantiate_table_cloning_2 TableTests.instantiate_table_cloning_3 TableTests.instantiate_tables_at_scope_level TableTests.invariant_table_properties_means_instantiating_tables_in_assignment_is_unsound @@ -518,14 +462,12 @@ TableTests.length_operator_intersection TableTests.length_operator_non_table_union TableTests.length_operator_union TableTests.length_operator_union_errors -TableTests.less_exponential_blowup_please TableTests.meta_add TableTests.meta_add_both_ways TableTests.meta_add_inferred TableTests.metatable_mismatch_should_fail TableTests.missing_metatable_for_sealed_tables_do_not_get_inferred TableTests.mixed_tables_with_implicit_numbered_keys -TableTests.MixedPropertiesAndIndexers TableTests.nil_assign_doesnt_hit_indexer TableTests.okay_to_add_property_to_unsealed_tables_by_function_call TableTests.only_ascribe_synthetic_names_at_module_scope @@ -535,8 +477,8 @@ TableTests.open_table_unification_2 TableTests.pass_a_union_of_tables_to_a_function_that_requires_a_table TableTests.pass_a_union_of_tables_to_a_function_that_requires_a_table_2 TableTests.pass_incompatible_union_to_a_generic_table_without_crashing -TableTests.passing_compatible_unions_to_a_generic_table_without_crashing TableTests.persistent_sealed_table_is_immutable +TableTests.prop_access_on_key_whose_types_mismatches TableTests.property_lookup_through_tabletypevar_metatable TableTests.quantify_even_that_table_was_never_exported_at_all TableTests.quantify_metatables_of_metatables_of_table @@ -570,23 +512,17 @@ TableTests.tc_member_function_2 TableTests.top_table_type TableTests.type_mismatch_on_massive_table_is_cut_short TableTests.unification_of_unions_in_a_self_referential_type -TableTests.unifying_tables_shouldnt_uaf1 TableTests.unifying_tables_shouldnt_uaf2 -TableTests.used_colon_correctly TableTests.used_colon_instead_of_dot TableTests.used_dot_instead_of_colon -TableTests.used_dot_instead_of_colon_but_correctly TableTests.width_subtyping ToDot.bound_table -ToDot.class ToDot.function ToDot.metatable -ToDot.primitive ToDot.table ToString.exhaustive_toString_of_cyclic_table ToString.function_type_with_argument_names_and_self ToString.function_type_with_argument_names_generic -ToString.named_metatable_toStringNamedFunction ToString.no_parentheses_around_cyclic_function_type_in_union ToString.toStringDetailed2 ToString.toStringErrorPack @@ -605,13 +541,12 @@ TryUnifyTests.typepack_unification_should_trim_free_tails TryUnifyTests.variadics_should_use_reversed_properly TypeAliases.cli_38393_recursive_intersection_oom TypeAliases.corecursive_types_generic -TypeAliases.do_not_quantify_unresolved_aliases TypeAliases.forward_declared_alias_is_not_clobbered_by_prior_unification_with_any TypeAliases.general_require_multi_assign TypeAliases.generic_param_remap TypeAliases.mismatched_generic_pack_type_param TypeAliases.mismatched_generic_type_param -TypeAliases.mutually_recursive_generic_aliases +TypeAliases.mutually_recursive_types_errors TypeAliases.mutually_recursive_types_restriction_not_ok_1 TypeAliases.mutually_recursive_types_restriction_not_ok_2 TypeAliases.mutually_recursive_types_swapsies_not_ok @@ -619,29 +554,22 @@ TypeAliases.recursive_types_restriction_not_ok TypeAliases.stringify_optional_parameterized_alias TypeAliases.stringify_type_alias_of_recursive_template_table_type TypeAliases.stringify_type_alias_of_recursive_template_table_type2 -TypeAliases.type_alias_import_mutation +TypeAliases.type_alias_fwd_declaration_is_precise TypeAliases.type_alias_local_mutation TypeAliases.type_alias_local_rename TypeAliases.type_alias_of_an_imported_recursive_generic_type TypeAliases.type_alias_of_an_imported_recursive_type -TypeInfer.check_expr_recursion_limit TypeInfer.checking_should_not_ice -TypeInfer.cli_50041_committing_txnlog_in_apollo_client_error TypeInfer.cyclic_follow TypeInfer.do_not_bind_a_free_table_to_a_union_containing_that_table TypeInfer.dont_report_type_errors_within_an_AstStatError -TypeInfer.follow_on_new_types_in_substitution -TypeInfer.free_typevars_introduced_within_control_flow_constructs_do_not_get_an_elevated_TypeLevel TypeInfer.globals TypeInfer.globals2 -TypeInfer.index_expr_should_be_checked TypeInfer.infer_assignment_value_types TypeInfer.infer_assignment_value_types_mutable_lval TypeInfer.infer_through_group_expr -TypeInfer.infer_type_assertion_value_type TypeInfer.no_heap_use_after_free_error TypeInfer.no_stack_overflow_from_isoptional -TypeInfer.recursive_metatable_crash TypeInfer.tc_after_error_recovery_no_replacement_name_in_error TypeInfer.tc_if_else_expressions1 TypeInfer.tc_if_else_expressions2 @@ -650,56 +578,32 @@ TypeInfer.tc_if_else_expressions_expected_type_2 TypeInfer.tc_if_else_expressions_expected_type_3 TypeInfer.tc_if_else_expressions_type_union TypeInfer.type_infer_recursion_limit_no_ice -TypeInfer.types stored in astResolvedTypes TypeInfer.warn_on_lowercase_parent_property -TypeInfer.weird_case -TypeInferAnyError.any_type_propagates TypeInferAnyError.assign_prop_to_table_by_calling_any_yields_any -TypeInferAnyError.call_to_any_yields_any -TypeInferAnyError.calling_error_type_yields_error TypeInferAnyError.can_get_length_of_any -TypeInferAnyError.can_subscript_any -TypeInferAnyError.CheckMethodsOfAny TypeInferAnyError.for_in_loop_iterator_is_any TypeInferAnyError.for_in_loop_iterator_is_any2 TypeInferAnyError.for_in_loop_iterator_is_error TypeInferAnyError.for_in_loop_iterator_is_error2 TypeInferAnyError.for_in_loop_iterator_returns_any TypeInferAnyError.for_in_loop_iterator_returns_any2 -TypeInferAnyError.indexing_error_type_does_not_produce_an_error TypeInferAnyError.length_of_error_type_does_not_produce_an_error -TypeInferAnyError.metatable_of_any_can_be_a_table -TypeInferAnyError.prop_access_on_any_with_other_options -TypeInferAnyError.quantify_any_does_not_bind_to_itself TypeInferAnyError.replace_every_free_type_when_unifying_a_complex_function_with_any TypeInferAnyError.type_error_addition -TypeInferClasses.assign_to_prop_of_class TypeInferClasses.call_base_method TypeInferClasses.call_instance_method -TypeInferClasses.call_method_of_a_child_class -TypeInferClasses.call_method_of_a_class -TypeInferClasses.can_assign_to_prop_of_base_class -TypeInferClasses.can_assign_to_prop_of_base_class_using_string TypeInferClasses.can_read_prop_of_base_class TypeInferClasses.can_read_prop_of_base_class_using_string -TypeInferClasses.cannot_call_method_of_child_on_base_instance -TypeInferClasses.cannot_call_unknown_method_of_a_class -TypeInferClasses.cannot_unify_class_instance_with_primitive TypeInferClasses.class_type_mismatch_with_name_conflict -TypeInferClasses.class_unification_type_mismatch_is_correct_order TypeInferClasses.classes_can_have_overloaded_operators TypeInferClasses.classes_without_overloaded_operators_cannot_be_added TypeInferClasses.detailed_class_unification_error TypeInferClasses.function_arguments_are_covariant -TypeInferClasses.higher_order_function_arguments_are_contravariant TypeInferClasses.higher_order_function_return_type_is_not_contravariant TypeInferClasses.higher_order_function_return_values_are_covariant TypeInferClasses.optional_class_field_access_error TypeInferClasses.table_class_unification_reports_sane_errors_for_missing_properties -TypeInferClasses.table_indexers_are_invariant -TypeInferClasses.table_properties_are_invariant TypeInferClasses.warn_when_prop_almost_matches -TypeInferClasses.we_can_infer_that_a_parameter_must_be_a_particular_class TypeInferClasses.we_can_report_when_someone_is_trying_to_use_a_table_rather_than_a_class TypeInferFunctions.another_indirect_function_case_where_it_is_ok_to_provide_too_many_arguments TypeInferFunctions.another_recursive_local_function @@ -711,21 +615,18 @@ TypeInferFunctions.complicated_return_types_require_an_explicit_annotation TypeInferFunctions.cyclic_function_type_in_args TypeInferFunctions.dont_give_other_overloads_message_if_only_one_argument_matching_overload_exists TypeInferFunctions.dont_infer_parameter_types_for_functions_from_their_call_site -TypeInferFunctions.dont_mutate_the_underlying_head_of_typepack_when_calling_with_self TypeInferFunctions.duplicate_functions_with_different_signatures_not_allowed_in_nonstrict TypeInferFunctions.error_detailed_function_mismatch_arg TypeInferFunctions.error_detailed_function_mismatch_arg_count TypeInferFunctions.error_detailed_function_mismatch_ret TypeInferFunctions.error_detailed_function_mismatch_ret_count TypeInferFunctions.error_detailed_function_mismatch_ret_mult -TypeInferFunctions.first_argument_can_be_optional TypeInferFunctions.free_is_not_bound_to_unknown TypeInferFunctions.func_expr_doesnt_leak_free TypeInferFunctions.function_cast_error_uses_correct_language TypeInferFunctions.function_decl_non_self_sealed_overwrite TypeInferFunctions.function_decl_non_self_sealed_overwrite_2 TypeInferFunctions.function_decl_non_self_unsealed_overwrite -TypeInferFunctions.function_decl_quantify_right_type TypeInferFunctions.function_does_not_return_enough_values TypeInferFunctions.function_statement_sealed_table_assignment_through_indexer TypeInferFunctions.higher_order_function_2 @@ -737,13 +638,10 @@ TypeInferFunctions.infer_anonymous_function_arguments TypeInferFunctions.infer_anonymous_function_arguments_outside_call TypeInferFunctions.infer_return_type_from_selected_overload TypeInferFunctions.infer_that_function_does_not_return_a_table -TypeInferFunctions.inferred_higher_order_functions_are_quantified_at_the_right_time -TypeInferFunctions.inferred_higher_order_functions_are_quantified_at_the_right_time2 TypeInferFunctions.it_is_ok_not_to_supply_enough_retvals TypeInferFunctions.it_is_ok_to_oversaturate_a_higher_order_function_argument TypeInferFunctions.list_all_overloads_if_no_overload_takes_given_argument_count TypeInferFunctions.list_only_alternative_overloads_that_match_argument_count -TypeInferFunctions.mutual_recursion TypeInferFunctions.no_lossy_function_type TypeInferFunctions.occurs_check_failure_in_function_return_type TypeInferFunctions.quantify_constrained_types @@ -759,10 +657,8 @@ TypeInferFunctions.too_few_arguments_variadic_generic TypeInferFunctions.too_few_arguments_variadic_generic2 TypeInferFunctions.too_many_arguments TypeInferFunctions.too_many_return_values -TypeInferFunctions.toposort_doesnt_break_mutual_recursion TypeInferFunctions.vararg_function_is_quantified TypeInferFunctions.vararg_functions_should_allow_calls_of_any_types_and_size -TypeInferLoops.correctly_scope_locals_while TypeInferLoops.for_in_loop TypeInferLoops.for_in_loop_error_on_factory_not_returning_the_right_amount_of_values TypeInferLoops.for_in_loop_error_on_iterator_requiring_args_but_none_given @@ -789,14 +685,9 @@ TypeInferLoops.repeat_loop_condition_binds_to_its_block TypeInferLoops.symbols_in_repeat_block_should_not_be_visible_beyond_until_condition TypeInferLoops.unreachable_code_after_infinite_loop TypeInferLoops.varlist_declared_by_for_in_loop_should_be_free -TypeInferLoops.while_loop -TypeInferModules.bound_free_table_export_is_ok -TypeInferModules.constrained_anyification_clone_immutable_types -TypeInferModules.custom_require_global TypeInferModules.do_not_modify_imported_types TypeInferModules.do_not_modify_imported_types_2 TypeInferModules.do_not_modify_imported_types_3 -TypeInferModules.do_not_modify_imported_types_4 TypeInferModules.general_require_call_expression TypeInferModules.general_require_type_mismatch TypeInferModules.module_type_conflict @@ -808,16 +699,14 @@ TypeInferModules.require_module_that_does_not_export TypeInferModules.require_types TypeInferModules.type_error_of_unknown_qualified_type TypeInferModules.warn_if_you_try_to_require_a_non_modulescript +TypeInferOOP.CheckMethodsOfSealed TypeInferOOP.dont_suggest_using_colon_rather_than_dot_if_another_overload_works TypeInferOOP.dont_suggest_using_colon_rather_than_dot_if_it_wont_help_2 TypeInferOOP.dont_suggest_using_colon_rather_than_dot_if_not_defined_with_colon TypeInferOOP.inferred_methods_of_free_tables_have_the_same_level_as_the_enclosing_table TypeInferOOP.inferring_hundreds_of_self_calls_should_not_suffocate_memory -TypeInferOOP.method_depends_on_table TypeInferOOP.methods_are_topologically_sorted TypeInferOOP.nonstrict_self_mismatch_tail -TypeInferOOP.object_constructor_can_refer_to_method_of_self -TypeInferOOP.table_oop TypeInferOperators.and_adds_boolean TypeInferOperators.and_adds_boolean_no_superfluous_union TypeInferOperators.and_binexps_dont_unify @@ -872,37 +761,26 @@ TypeInferPrimitives.string_function_other TypeInferPrimitives.string_index TypeInferPrimitives.string_length TypeInferPrimitives.string_method -TypeInferUnknownNever.array_like_table_of_never_is_inhabitable TypeInferUnknownNever.assign_to_global_which_is_never TypeInferUnknownNever.assign_to_local_which_is_never TypeInferUnknownNever.assign_to_prop_which_is_never TypeInferUnknownNever.assign_to_subscript_which_is_never TypeInferUnknownNever.call_never TypeInferUnknownNever.dont_unify_operands_if_one_of_the_operand_is_never_in_any_ordering_operators -TypeInferUnknownNever.index_on_never TypeInferUnknownNever.index_on_union_of_tables_for_properties_that_is_never TypeInferUnknownNever.index_on_union_of_tables_for_properties_that_is_sorta_never TypeInferUnknownNever.length_of_never TypeInferUnknownNever.math_operators_and_never -TypeInferUnknownNever.never_is_reflexive -TypeInferUnknownNever.never_subtype_and_string_supertype -TypeInferUnknownNever.pick_never_from_variadic_type_pack -TypeInferUnknownNever.string_subtype_and_never_supertype -TypeInferUnknownNever.string_subtype_and_unknown_supertype -TypeInferUnknownNever.table_with_prop_of_type_never_is_also_reflexive -TypeInferUnknownNever.table_with_prop_of_type_never_is_uninhabitable TypeInferUnknownNever.type_packs_containing_never_is_itself_uninhabitable TypeInferUnknownNever.type_packs_containing_never_is_itself_uninhabitable2 TypeInferUnknownNever.unary_minus_of_never TypeInferUnknownNever.unknown_is_reflexive -TypeInferUnknownNever.unknown_subtype_and_string_supertype TypePackTests.cyclic_type_packs TypePackTests.higher_order_function TypePackTests.multiple_varargs_inference_are_not_confused TypePackTests.no_return_size_should_be_zero TypePackTests.pack_tail_unification_check TypePackTests.parenthesized_varargs_returns_any -TypePackTests.self_and_varargs_should_work TypePackTests.type_alias_backwards_compatible TypePackTests.type_alias_default_export TypePackTests.type_alias_default_mixed_self @@ -913,8 +791,6 @@ TypePackTests.type_alias_default_type_pack_self_tp TypePackTests.type_alias_default_type_self TypePackTests.type_alias_defaults_confusing_types TypePackTests.type_alias_defaults_recursive_type -TypePackTests.type_alias_type_pack_explicit -TypePackTests.type_alias_type_pack_explicit_multi TypePackTests.type_alias_type_pack_multi TypePackTests.type_alias_type_pack_variadic TypePackTests.type_alias_type_packs @@ -949,6 +825,7 @@ TypeSingletons.string_singletons_escape_chars TypeSingletons.string_singletons_mismatch TypeSingletons.table_insert_with_a_singleton_argument TypeSingletons.table_properties_type_error_escapes +TypeSingletons.tagged_unions_immutable_tag TypeSingletons.tagged_unions_using_singletons TypeSingletons.taking_the_length_of_string_singleton TypeSingletons.taking_the_length_of_union_of_string_singleton @@ -978,5 +855,4 @@ UnionTypes.optional_union_members UnionTypes.optional_union_methods UnionTypes.return_types_can_be_disjoint UnionTypes.table_union_write_indirect -UnionTypes.unify_unsealed_table_union_check UnionTypes.union_equality_comparisons diff --git a/tools/heapgraph.py b/tools/heapgraph.py index d4d29af1..2817c38f 100644 --- a/tools/heapgraph.py +++ b/tools/heapgraph.py @@ -39,6 +39,16 @@ class Node(svg.Node): def details(self, root): return "{} ({:,} bytes, {:.1%}); self: {:,} bytes in {:,} objects".format(self.name, self.width, self.width / root.width, self.size, self.count) +def getkey(heap, obj, key): + pairs = obj.get("pairs", []) + for i in range(0, len(pairs), 2): + if pairs[i] and heap[pairs[i]]["type"] == "string" and heap[pairs[i]]["data"] == key: + if pairs[i + 1] and heap[pairs[i + 1]]["type"] == "string": + return heap[pairs[i + 1]]["data"] + else: + return None + return None + # load files if arguments.snapshotnew == None: dumpold = None @@ -50,6 +60,8 @@ else: with open(arguments.snapshotnew) as f: dump = json.load(f) +heap = dump["objects"] + # reachability analysis: how much of the heap is reachable from roots? visited = set() queue = [] @@ -66,7 +78,7 @@ while offset < len(queue): continue visited.add(addr) - obj = dump["objects"][addr] + obj = heap[addr] if not dumpold or not addr in dumpold["objects"]: node.count += 1 @@ -75,17 +87,27 @@ while offset < len(queue): if obj["type"] == "table": pairs = obj.get("pairs", []) + weakkey = False + weakval = False + + if "metatable" in obj: + modemt = getkey(heap, heap[obj["metatable"]], "__mode") + if modemt: + weakkey = "k" in modemt + weakval = "v" in modemt for i in range(0, len(pairs), 2): key = pairs[i+0] val = pairs[i+1] - if key and val and dump["objects"][key]["type"] == "string": + if key and heap[key]["type"] == "string": + # string keys are always strong queue.append((key, node)) - queue.append((val, node.child(dump["objects"][key]["data"]))) + if val and not weakval: + queue.append((val, node.child(heap[key]["data"]))) else: - if key: + if key and not weakkey: queue.append((key, node)) - if val: + if val and not weakval: queue.append((val, node)) for a in obj.get("array", []): @@ -97,7 +119,7 @@ while offset < len(queue): source = "" if "proto" in obj: - proto = dump["objects"][obj["proto"]] + proto = heap[obj["proto"]] if "source" in proto: source = proto["source"] diff --git a/tools/heapstat.py b/tools/heapstat.py index 838521b6..4c0cb407 100644 --- a/tools/heapstat.py +++ b/tools/heapstat.py @@ -15,12 +15,20 @@ def updatesize(d, k, s): def sortedsize(p): return sorted(p, key = lambda s: s[1][1], reverse = True) +def getkey(heap, obj, key): + pairs = obj.get("pairs", []) + for i in range(0, len(pairs), 2): + if pairs[i] and heap[pairs[i]]["type"] == "string" and heap[pairs[i]]["data"] == key: + if pairs[i + 1] and heap[pairs[i + 1]]["type"] == "string": + return heap[pairs[i + 1]]["data"] + else: + return None + return None + with open(sys.argv[1]) as f: dump = json.load(f) heap = dump["objects"] -type_addr = next((addr for addr,obj in heap.items() if obj["type"] == "string" and obj["data"] == "__type"), None) - size_type = {} size_udata = {} size_category = {} @@ -33,11 +41,7 @@ for addr, obj in heap.items(): if obj["type"] == "userdata" and "metatable" in obj: metatable = heap[obj["metatable"]] - pairs = metatable.get("pairs", []) - typemt = "unknown" - for i in range(0, len(pairs), 2): - if type_addr and pairs[i] == type_addr and pairs[i + 1] and heap[pairs[i + 1]]["type"] == "string": - typemt = heap[pairs[i + 1]]["data"] + typemt = getkey(heap, metatable, "__type") or "unknown" updatesize(size_udata, typemt, obj["size"]) print("objects by type:") From f1928ddadeeaf1477bf85cf6750d57ded6fe9876 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Thu, 11 Aug 2022 13:48:35 -0700 Subject: [PATCH 337/399] Adjust benchmark runs to use config=profile and new file names --- .github/workflows/benchmark.yml | 40 ++++++++++++++++++++++++--------- 1 file changed, 29 insertions(+), 11 deletions(-) diff --git a/.github/workflows/benchmark.yml b/.github/workflows/benchmark.yml index 9d26186e..572e037f 100644 --- a/.github/workflows/benchmark.yml +++ b/.github/workflows/benchmark.yml @@ -13,12 +13,11 @@ on: jobs: callgrind: - name: callgrind ${{ matrix.compiler }} + name: callgrind strategy: fail-fast: false matrix: os: [ubuntu-22.04] - compiler: [g++] benchResultsRepo: - { name: "luau-lang/benchmark-data", branch: "main" } @@ -31,12 +30,23 @@ jobs: run: | sudo apt-get install valgrind - - name: Build Luau - run: CXX=${{ matrix.compiler }} make config=release CALLGRIND=1 luau luau-analyze - - - name: Run benchmark (bench) + - name: Build Luau (gcc) run: | - python bench/bench.py --callgrind --vm "./luau -O2" | tee -a bench-output.txt + CXX=g++ make config=profile luau + cp luau luau-gcc + + - name: Build Luau (clang) + run: | + make config=profile clean + CXX=clang++ make config=profile luau luau-analyze + + - name: Run benchmark (bench-gcc) + run: | + python bench/bench.py --callgrind --vm "./luau-gcc -O2" | tee -a bench-gcc-output.txt + + - name: Run benchmark (bench-clang) + run: | + python bench/bench.py --callgrind --vm "./luau -O2" | tee -a bench-clang-output.txt - name: Run benchmark (analyze) run: | @@ -68,13 +78,21 @@ jobs: token: ${{ secrets.BENCH_GITHUB_TOKEN }} path: "./gh-pages" - - name: Store results (bench) + - name: Store results (bench-clang) uses: Roblox/rhysd-github-action-benchmark@v-luau with: - name: callgrind ${{ matrix.compiler }} + name: callgrind clang tool: "benchmarkluau" - output-file-path: ./bench-output.txt - external-data-json-path: ./gh-pages/bench.json + output-file-path: ./bench-clang-output.txt + external-data-json-path: ./gh-pages/bench-clang.json + + - name: Store results (bench-gcc) + uses: Roblox/rhysd-github-action-benchmark@v-luau + with: + name: callgrind gcc + tool: "benchmarkluau" + output-file-path: ./bench-gcc-output.txt + external-data-json-path: ./gh-pages/bench-gcc.json - name: Store results (analyze) uses: Roblox/rhysd-github-action-benchmark@v-luau From f7d8ad077407e610b9d0cb2a6ede5249d365db08 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Thu, 11 Aug 2022 14:01:33 -0700 Subject: [PATCH 338/399] Sync to upstream/release/540 (#635) Also adjust benchmark runs to use config=profile and run clang for all benchmarks + gcc for runtime --- .github/workflows/benchmark.yml | 40 +- Analysis/include/Luau/Clone.h | 2 +- .../include/Luau/ConstraintGraphBuilder.h | 7 +- Analysis/include/Luau/Frontend.h | 2 +- Analysis/include/Luau/Linter.h | 1 + Analysis/include/Luau/Scope.h | 4 - Analysis/include/Luau/TypeInfer.h | 5 + Analysis/include/Luau/TypeVar.h | 7 +- Analysis/include/Luau/Unifier.h | 10 +- Analysis/src/ApplyTypeFunction.cpp | 4 + Analysis/src/Clone.cpp | 12 +- Analysis/src/ConstraintGraphBuilder.cpp | 201 ++++++- Analysis/src/ConstraintSolver.cpp | 8 + Analysis/src/Frontend.cpp | 6 +- Analysis/src/Instantiation.cpp | 2 + Analysis/src/Linter.cpp | 64 +++ Analysis/src/Module.cpp | 139 ++++- Analysis/src/Normalize.cpp | 3 +- Analysis/src/Quantify.cpp | 5 + Analysis/src/Scope.cpp | 30 - Analysis/src/Substitution.cpp | 12 +- Analysis/src/TypeChecker2.cpp | 173 +++++- Analysis/src/TypeInfer.cpp | 15 +- Analysis/src/Unifier.cpp | 83 ++- CLI/Reduce.cpp | 511 ++++++++++++++++++ CMakeLists.txt | 7 + CodeGen/include/Luau/AssemblyBuilderX64.h | 17 + CodeGen/src/AssemblyBuilderX64.cpp | 194 ++++++- Compiler/src/Compiler.cpp | 256 +++++++-- Makefile | 4 +- Sources.cmake | 8 + bench/bench.py | 4 +- bench/influxbench.py | 7 +- tests/AssemblyBuilderX64.test.cpp | 92 +++- tests/Compiler.test.cpp | 299 ++++++++-- tests/Fixture.cpp | 17 +- tests/Linter.test.cpp | 36 +- tests/Module.test.cpp | 74 +++ tests/NonstrictMode.test.cpp | 1 + tests/TypeInfer.definitions.test.cpp | 27 + tests/TypeInfer.generics.test.cpp | 1 + tests/TypeInfer.modules.test.cpp | 2 - tests/TypeInfer.tables.test.cpp | 4 +- tests/TypeVar.test.cpp | 11 + tests/conformance/basic.lua | 6 + tests/conformance/errors.lua | 2 +- tests/conformance/strings.lua | 8 + tools/faillist.txt | 150 +---- tools/heapgraph.py | 34 +- tools/heapstat.py | 18 +- 50 files changed, 2193 insertions(+), 432 deletions(-) create mode 100644 CLI/Reduce.cpp diff --git a/.github/workflows/benchmark.yml b/.github/workflows/benchmark.yml index 9d26186e..572e037f 100644 --- a/.github/workflows/benchmark.yml +++ b/.github/workflows/benchmark.yml @@ -13,12 +13,11 @@ on: jobs: callgrind: - name: callgrind ${{ matrix.compiler }} + name: callgrind strategy: fail-fast: false matrix: os: [ubuntu-22.04] - compiler: [g++] benchResultsRepo: - { name: "luau-lang/benchmark-data", branch: "main" } @@ -31,12 +30,23 @@ jobs: run: | sudo apt-get install valgrind - - name: Build Luau - run: CXX=${{ matrix.compiler }} make config=release CALLGRIND=1 luau luau-analyze - - - name: Run benchmark (bench) + - name: Build Luau (gcc) run: | - python bench/bench.py --callgrind --vm "./luau -O2" | tee -a bench-output.txt + CXX=g++ make config=profile luau + cp luau luau-gcc + + - name: Build Luau (clang) + run: | + make config=profile clean + CXX=clang++ make config=profile luau luau-analyze + + - name: Run benchmark (bench-gcc) + run: | + python bench/bench.py --callgrind --vm "./luau-gcc -O2" | tee -a bench-gcc-output.txt + + - name: Run benchmark (bench-clang) + run: | + python bench/bench.py --callgrind --vm "./luau -O2" | tee -a bench-clang-output.txt - name: Run benchmark (analyze) run: | @@ -68,13 +78,21 @@ jobs: token: ${{ secrets.BENCH_GITHUB_TOKEN }} path: "./gh-pages" - - name: Store results (bench) + - name: Store results (bench-clang) uses: Roblox/rhysd-github-action-benchmark@v-luau with: - name: callgrind ${{ matrix.compiler }} + name: callgrind clang tool: "benchmarkluau" - output-file-path: ./bench-output.txt - external-data-json-path: ./gh-pages/bench.json + output-file-path: ./bench-clang-output.txt + external-data-json-path: ./gh-pages/bench-clang.json + + - name: Store results (bench-gcc) + uses: Roblox/rhysd-github-action-benchmark@v-luau + with: + name: callgrind gcc + tool: "benchmarkluau" + output-file-path: ./bench-gcc-output.txt + external-data-json-path: ./gh-pages/bench-gcc.json - name: Store results (analyze) uses: Roblox/rhysd-github-action-benchmark@v-luau diff --git a/Analysis/include/Luau/Clone.h b/Analysis/include/Luau/Clone.h index 548a58f5..217e1cc3 100644 --- a/Analysis/include/Luau/Clone.h +++ b/Analysis/include/Luau/Clone.h @@ -25,6 +25,6 @@ TypePackId clone(TypePackId tp, TypeArena& dest, CloneState& cloneState); TypeId clone(TypeId tp, TypeArena& dest, CloneState& cloneState); TypeFun clone(const TypeFun& typeFun, TypeArena& dest, CloneState& cloneState); -TypeId shallowClone(TypeId ty, TypeArena& dest, const TxnLog* log); +TypeId shallowClone(TypeId ty, TypeArena& dest, const TxnLog* log, bool alwaysClone = false); } // namespace Luau diff --git a/Analysis/include/Luau/ConstraintGraphBuilder.h b/Analysis/include/Luau/ConstraintGraphBuilder.h index 69f35d46..41d14326 100644 --- a/Analysis/include/Luau/ConstraintGraphBuilder.h +++ b/Analysis/include/Luau/ConstraintGraphBuilder.h @@ -28,6 +28,7 @@ struct ConstraintGraphBuilder std::vector> scopes; ModuleName moduleName; + ModulePtr module; SingletonTypes& singletonTypes; const NotNull arena; // The root scope of the module we're generating constraints for. @@ -53,9 +54,9 @@ struct ConstraintGraphBuilder // Occasionally constraint generation needs to produce an ICE. const NotNull ice; - NotNull globalScope; + ScopePtr globalScope; - ConstraintGraphBuilder(const ModuleName& moduleName, TypeArena* arena, NotNull ice, NotNull globalScope); + ConstraintGraphBuilder(const ModuleName& moduleName, ModulePtr module, TypeArena* arena, NotNull ice, const ScopePtr& globalScope); /** * Fabricates a new free type belonging to a given scope. @@ -103,6 +104,7 @@ struct ConstraintGraphBuilder void visit(const ScopePtr& scope, AstStatBlock* block); void visit(const ScopePtr& scope, AstStatLocal* local); void visit(const ScopePtr& scope, AstStatFor* for_); + void visit(const ScopePtr& scope, AstStatWhile* while_); void visit(const ScopePtr& scope, AstStatLocalFunction* function); void visit(const ScopePtr& scope, AstStatFunction* function); void visit(const ScopePtr& scope, AstStatReturn* ret); @@ -131,6 +133,7 @@ struct ConstraintGraphBuilder TypeId check(const ScopePtr& scope, AstExprIndexExpr* indexExpr); TypeId check(const ScopePtr& scope, AstExprUnary* unary); TypeId check(const ScopePtr& scope, AstExprBinary* binary); + TypeId check(const ScopePtr& scope, AstExprTypeAssertion* typeAssert); struct FunctionSignature { diff --git a/Analysis/include/Luau/Frontend.h b/Analysis/include/Luau/Frontend.h index 9b8ec19e..82df493a 100644 --- a/Analysis/include/Luau/Frontend.h +++ b/Analysis/include/Luau/Frontend.h @@ -154,7 +154,7 @@ struct Frontend LoadDefinitionFileResult loadDefinitionFile(std::string_view source, const std::string& packageName); - NotNull getGlobalScope(); + ScopePtr getGlobalScope(); private: ModulePtr check(const SourceModule& sourceModule, Mode mode, const ScopePtr& environmentScope); diff --git a/Analysis/include/Luau/Linter.h b/Analysis/include/Luau/Linter.h index 2cd91d54..0e3d9880 100644 --- a/Analysis/include/Luau/Linter.h +++ b/Analysis/include/Luau/Linter.h @@ -53,6 +53,7 @@ struct LintWarning Code_MisleadingAndOr = 25, Code_CommentDirective = 26, Code_IntegerParsing = 27, + Code_ComparisonPrecedence = 28, Code__Count }; diff --git a/Analysis/include/Luau/Scope.h b/Analysis/include/Luau/Scope.h index 55ca54c6..dc123335 100644 --- a/Analysis/include/Luau/Scope.h +++ b/Analysis/include/Luau/Scope.h @@ -36,8 +36,6 @@ struct Scope // All the children of this scope. std::vector> children; std::unordered_map bindings; - std::unordered_map typeBindings; - std::unordered_map typePackBindings; TypePackId returnType; std::optional varargPack; // All constraints belonging to this scope. @@ -52,8 +50,6 @@ struct Scope std::unordered_map> importedTypeBindings; std::optional lookup(Symbol sym); - std::optional lookupTypeBinding(const Name& name); - std::optional lookupTypePackBinding(const Name& name); std::optional lookupType(const Name& name); std::optional lookupImportedType(const Name& moduleAlias, const Name& name); diff --git a/Analysis/include/Luau/TypeInfer.h b/Analysis/include/Luau/TypeInfer.h index c50b2c8c..5c55ddb0 100644 --- a/Analysis/include/Luau/TypeInfer.h +++ b/Analysis/include/Luau/TypeInfer.h @@ -16,6 +16,8 @@ #include #include +LUAU_FASTFLAG(LuauClassTypeVarsInSubstitution) + namespace Luau { @@ -57,6 +59,9 @@ struct Anyification : Substitution bool ignoreChildren(TypeId ty) override { + if (FFlag::LuauClassTypeVarsInSubstitution && get(ty)) + return true; + return ty->persistent; } bool ignoreChildren(TypePackId ty) override diff --git a/Analysis/include/Luau/TypeVar.h b/Analysis/include/Luau/TypeVar.h index 6a13b11c..35b37394 100644 --- a/Analysis/include/Luau/TypeVar.h +++ b/Analysis/include/Luau/TypeVar.h @@ -717,6 +717,7 @@ struct TypeIterator stack.push_front({t, 0}); seen.insert(t); + descend(); } TypeIterator& operator++() @@ -748,17 +749,19 @@ struct TypeIterator const TypeId& operator*() { - LUAU_ASSERT(!stack.empty()); - descend(); + LUAU_ASSERT(!stack.empty()); + auto [t, currentIndex] = stack.front(); LUAU_ASSERT(t); + const std::vector& types = getTypes(t); LUAU_ASSERT(currentIndex < types.size()); const TypeId& ty = types[currentIndex]; LUAU_ASSERT(!get(follow(ty))); + return ty; } diff --git a/Analysis/include/Luau/Unifier.h b/Analysis/include/Luau/Unifier.h index f460dc87..9fa29076 100644 --- a/Analysis/include/Luau/Unifier.h +++ b/Analysis/include/Luau/Unifier.h @@ -109,11 +109,11 @@ private: public: void unifyLowerBound(TypePackId subTy, TypePackId superTy, TypeLevel demotedLevel); - // Report an "infinite type error" if the type "needle" already occurs within "haystack" - void occursCheck(TypeId needle, TypeId haystack); - void occursCheck(DenseHashSet& seen, TypeId needle, TypeId haystack); - void occursCheck(TypePackId needle, TypePackId haystack); - void occursCheck(DenseHashSet& seen, TypePackId needle, TypePackId haystack); + // Returns true if the type "needle" already occurs within "haystack" and reports an "infinite type error" + bool occursCheck(TypeId needle, TypeId haystack); + bool occursCheck(DenseHashSet& seen, TypeId needle, TypeId haystack); + bool occursCheck(TypePackId needle, TypePackId haystack); + bool occursCheck(DenseHashSet& seen, TypePackId needle, TypePackId haystack); Unifier makeChildUnifier(); diff --git a/Analysis/src/ApplyTypeFunction.cpp b/Analysis/src/ApplyTypeFunction.cpp index c6ac3e19..b293ed3d 100644 --- a/Analysis/src/ApplyTypeFunction.cpp +++ b/Analysis/src/ApplyTypeFunction.cpp @@ -2,6 +2,8 @@ #include "Luau/ApplyTypeFunction.h" +LUAU_FASTFLAG(LuauClassTypeVarsInSubstitution) + namespace Luau { @@ -31,6 +33,8 @@ bool ApplyTypeFunction::ignoreChildren(TypeId ty) { if (get(ty)) return true; + else if (FFlag::LuauClassTypeVarsInSubstitution && get(ty)) + return true; else return false; } diff --git a/Analysis/src/Clone.cpp b/Analysis/src/Clone.cpp index 51ad61d5..2e04b527 100644 --- a/Analysis/src/Clone.cpp +++ b/Analysis/src/Clone.cpp @@ -7,6 +7,7 @@ #include "Luau/Unifiable.h" LUAU_FASTFLAG(DebugLuauCopyBeforeNormalizing) +LUAU_FASTFLAG(LuauClonePublicInterfaceLess) LUAU_FASTINTVARIABLE(LuauTypeCloneRecursionLimit, 300) @@ -445,7 +446,7 @@ TypeFun clone(const TypeFun& typeFun, TypeArena& dest, CloneState& cloneState) return result; } -TypeId shallowClone(TypeId ty, TypeArena& dest, const TxnLog* log) +TypeId shallowClone(TypeId ty, TypeArena& dest, const TxnLog* log, bool alwaysClone) { ty = log->follow(ty); @@ -504,6 +505,15 @@ TypeId shallowClone(TypeId ty, TypeArena& dest, const TxnLog* log) PendingExpansionTypeVar clone{petv->fn, petv->typeArguments, petv->packArguments}; result = dest.addType(std::move(clone)); } + else if (const ClassTypeVar* ctv = get(ty); FFlag::LuauClonePublicInterfaceLess && ctv && alwaysClone) + { + ClassTypeVar clone{ctv->name, ctv->props, ctv->parent, ctv->metatable, ctv->tags, ctv->userData, ctv->definitionModuleName}; + result = dest.addType(std::move(clone)); + } + else if (FFlag::LuauClonePublicInterfaceLess && alwaysClone) + { + result = dest.addType(*ty); + } else return result; diff --git a/Analysis/src/ConstraintGraphBuilder.cpp b/Analysis/src/ConstraintGraphBuilder.cpp index ea7037bf..2c36d422 100644 --- a/Analysis/src/ConstraintGraphBuilder.cpp +++ b/Analysis/src/ConstraintGraphBuilder.cpp @@ -1,6 +1,8 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #include "Luau/ConstraintGraphBuilder.h" +#include "Luau/Ast.h" +#include "Luau/Constraint.h" #include "Luau/RecursionCounter.h" #include "Luau/ToString.h" @@ -14,8 +16,9 @@ namespace Luau const AstStat* getFallthrough(const AstStat* node); // TypeInfer.cpp ConstraintGraphBuilder::ConstraintGraphBuilder( - const ModuleName& moduleName, TypeArena* arena, NotNull ice, NotNull globalScope) + const ModuleName& moduleName, ModulePtr module, TypeArena* arena, NotNull ice, const ScopePtr& globalScope) : moduleName(moduleName) + , module(module) , singletonTypes(getSingletonTypes()) , arena(arena) , rootScope(nullptr) @@ -61,7 +64,7 @@ void ConstraintGraphBuilder::visit(AstStatBlock* block) { LUAU_ASSERT(scopes.empty()); LUAU_ASSERT(rootScope == nullptr); - ScopePtr scope = std::make_shared(singletonTypes.anyTypePack); + ScopePtr scope = std::make_shared(globalScope); rootScope = scope.get(); scopes.emplace_back(block->location, scope); @@ -70,11 +73,11 @@ void ConstraintGraphBuilder::visit(AstStatBlock* block) prepopulateGlobalScope(scope, block); // TODO: We should share the global scope. - rootScope->typeBindings["nil"] = TypeFun{singletonTypes.nilType}; - rootScope->typeBindings["number"] = TypeFun{singletonTypes.numberType}; - rootScope->typeBindings["string"] = TypeFun{singletonTypes.stringType}; - rootScope->typeBindings["boolean"] = TypeFun{singletonTypes.booleanType}; - rootScope->typeBindings["thread"] = TypeFun{singletonTypes.threadType}; + rootScope->privateTypeBindings["nil"] = TypeFun{singletonTypes.nilType}; + rootScope->privateTypeBindings["number"] = TypeFun{singletonTypes.numberType}; + rootScope->privateTypeBindings["string"] = TypeFun{singletonTypes.stringType}; + rootScope->privateTypeBindings["boolean"] = TypeFun{singletonTypes.booleanType}; + rootScope->privateTypeBindings["thread"] = TypeFun{singletonTypes.threadType}; visitBlockWithoutChildScope(scope, block); } @@ -99,7 +102,7 @@ void ConstraintGraphBuilder::visitBlockWithoutChildScope(const ScopePtr& scope, { if (auto alias = stat->as()) { - if (scope->typeBindings.count(alias->name.value) != 0) + if (scope->privateTypeBindings.count(alias->name.value) != 0) { auto it = aliasDefinitionLocations.find(alias->name.value); LUAU_ASSERT(it != aliasDefinitionLocations.end()); @@ -121,16 +124,16 @@ void ConstraintGraphBuilder::visitBlockWithoutChildScope(const ScopePtr& scope, for (const auto& [name, gen] : createGenerics(defnScope, alias->generics)) { initialFun.typeParams.push_back(gen); - defnScope->typeBindings[name] = TypeFun{gen.ty}; + defnScope->privateTypeBindings[name] = TypeFun{gen.ty}; } for (const auto& [name, genPack] : createGenericPacks(defnScope, alias->genericPacks)) { initialFun.typePackParams.push_back(genPack); - defnScope->typePackBindings[name] = genPack.tp; + defnScope->privateTypePackBindings[name] = genPack.tp; } - scope->typeBindings[alias->name.value] = std::move(initialFun); + scope->privateTypeBindings[alias->name.value] = std::move(initialFun); astTypeAliasDefiningScopes[alias] = defnScope; aliasDefinitionLocations[alias->name.value] = alias->location; } @@ -150,6 +153,8 @@ void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStat* stat) visit(scope, s); else if (auto s = stat->as()) visit(scope, s); + else if (auto s = stat->as()) + visit(scope, s); else if (auto f = stat->as()) visit(scope, f); else if (auto f = stat->as()) @@ -242,6 +247,15 @@ void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatFor* for_) visit(forScope, for_->body); } +void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatWhile* while_) +{ + check(scope, while_->condition); + + ScopePtr whileScope = childScope(while_->location, scope); + + visit(whileScope, while_->body); +} + void addConstraints(Constraint* constraint, NotNull scope) { scope->constraints.reserve(scope->constraints.size() + scope->constraints.size()); @@ -388,11 +402,11 @@ void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatTypeAlias* alia { // TODO: Exported type aliases - auto bindingIt = scope->typeBindings.find(alias->name.value); + auto bindingIt = scope->privateTypeBindings.find(alias->name.value); ScopePtr* defnIt = astTypeAliasDefiningScopes.find(alias); // These will be undefined if the alias was a duplicate definition, in which // case we just skip over it. - if (bindingIt == scope->typeBindings.end() || defnIt == nullptr) + if (bindingIt == scope->privateTypeBindings.end() || defnIt == nullptr) { return; } @@ -416,17 +430,152 @@ void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatDeclareGlobal* LUAU_ASSERT(global->type); TypeId globalTy = resolveType(scope, global->type); + Name globalName(global->name.value); + + module->declaredGlobals[globalName] = globalTy; scope->bindings[global->name] = Binding{globalTy, global->location}; } -void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatDeclareClass* global) +static bool isMetamethod(const Name& name) { - LUAU_ASSERT(false); // TODO: implement + return name == "__index" || name == "__newindex" || name == "__call" || name == "__concat" || name == "__unm" || name == "__add" || + name == "__sub" || name == "__mul" || name == "__div" || name == "__mod" || name == "__pow" || name == "__tostring" || + name == "__metatable" || name == "__eq" || name == "__lt" || name == "__le" || name == "__mode" || name == "__iter" || name == "__len"; +} + +void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatDeclareClass* declaredClass) +{ + std::optional superTy = std::nullopt; + if (declaredClass->superName) + { + Name superName = Name(declaredClass->superName->value); + std::optional lookupType = scope->lookupType(superName); + + if (!lookupType) + { + reportError(declaredClass->location, UnknownSymbol{superName, UnknownSymbol::Type}); + return; + } + + // We don't have generic classes, so this assertion _should_ never be hit. + LUAU_ASSERT(lookupType->typeParams.size() == 0 && lookupType->typePackParams.size() == 0); + superTy = lookupType->type; + + if (!get(follow(*superTy))) + { + reportError(declaredClass->location, + GenericError{format("Cannot use non-class type '%s' as a superclass of class '%s'", superName.c_str(), declaredClass->name.value)}); + + return; + } + } + + Name className(declaredClass->name.value); + + TypeId classTy = arena->addType(ClassTypeVar(className, {}, superTy, std::nullopt, {}, {}, moduleName)); + ClassTypeVar* ctv = getMutable(classTy); + + TypeId metaTy = arena->addType(TableTypeVar{TableState::Sealed, scope->level}); + TableTypeVar* metatable = getMutable(metaTy); + + ctv->metatable = metaTy; + + scope->exportedTypeBindings[className] = TypeFun{{}, classTy}; + + for (const AstDeclaredClassProp& prop : declaredClass->props) + { + Name propName(prop.name.value); + TypeId propTy = resolveType(scope, prop.ty); + + bool assignToMetatable = isMetamethod(propName); + + // Function types always take 'self', but this isn't reflected in the + // parsed annotation. Add it here. + if (prop.isMethod) + { + if (FunctionTypeVar* ftv = getMutable(propTy)) + { + ftv->argNames.insert(ftv->argNames.begin(), FunctionArgument{"self", {}}); + ftv->argTypes = arena->addTypePack(TypePack{{classTy}, ftv->argTypes}); + + ftv->hasSelf = true; + } + } + + if (ctv->props.count(propName) == 0) + { + if (assignToMetatable) + metatable->props[propName] = {propTy}; + else + ctv->props[propName] = {propTy}; + } + else + { + TypeId currentTy = assignToMetatable ? metatable->props[propName].type : ctv->props[propName].type; + + // We special-case this logic to keep the intersection flat; otherwise we + // would create a ton of nested intersection types. + if (const IntersectionTypeVar* itv = get(currentTy)) + { + std::vector options = itv->parts; + options.push_back(propTy); + TypeId newItv = arena->addType(IntersectionTypeVar{std::move(options)}); + + if (assignToMetatable) + metatable->props[propName] = {newItv}; + else + ctv->props[propName] = {newItv}; + } + else if (get(currentTy)) + { + TypeId intersection = arena->addType(IntersectionTypeVar{{currentTy, propTy}}); + + if (assignToMetatable) + metatable->props[propName] = {intersection}; + else + ctv->props[propName] = {intersection}; + } + else + { + reportError(declaredClass->location, GenericError{format("Cannot overload non-function class member '%s'", propName.c_str())}); + } + } + } } void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatDeclareFunction* global) { - LUAU_ASSERT(false); // TODO: implement + + std::vector> generics = createGenerics(scope, global->generics); + std::vector> genericPacks = createGenericPacks(scope, global->genericPacks); + + std::vector genericTys; + genericTys.reserve(generics.size()); + for (auto& [name, generic] : generics) + genericTys.push_back(generic.ty); + + std::vector genericTps; + genericTps.reserve(genericPacks.size()); + for (auto& [name, generic] : genericPacks) + genericTps.push_back(generic.tp); + + ScopePtr funScope = scope; + if (!generics.empty() || !genericPacks.empty()) + funScope = childScope(global->location, scope); + + TypePackId paramPack = resolveTypePack(funScope, global->params); + TypePackId retPack = resolveTypePack(funScope, global->retTypes); + TypeId fnType = arena->addType(FunctionTypeVar{funScope->level, std::move(genericTys), std::move(genericTps), paramPack, retPack}); + FunctionTypeVar* ftv = getMutable(fnType); + + ftv->argNames.reserve(global->paramNames.size); + for (const auto& el : global->paramNames) + ftv->argNames.push_back(FunctionArgument{el.first.value, el.second}); + + Name fnName(global->name.value); + + module->declaredGlobals[fnName] = fnType; + scope->bindings[global->name] = Binding{fnType, global->location}; } TypePackId ConstraintGraphBuilder::checkPack(const ScopePtr& scope, AstArray exprs) @@ -590,6 +739,8 @@ TypeId ConstraintGraphBuilder::check(const ScopePtr& scope, AstExpr* expr) result = check(scope, unary); else if (auto binary = expr->as()) result = check(scope, binary); + else if (auto typeAssert = expr->as()) + result = check(scope, typeAssert); else if (auto err = expr->as()) { // Open question: Should we traverse into this? @@ -682,6 +833,12 @@ TypeId ConstraintGraphBuilder::check(const ScopePtr& scope, AstExprBinary* binar return nullptr; } +TypeId ConstraintGraphBuilder::check(const ScopePtr& scope, AstExprTypeAssertion* typeAssert) +{ + check(scope, typeAssert->expr); + return resolveType(scope, typeAssert->annotation); +} + TypeId ConstraintGraphBuilder::checkExprTable(const ScopePtr& scope, AstExprTable* expr) { TypeId ty = arena->addType(TableTypeVar{}); @@ -765,13 +922,13 @@ ConstraintGraphBuilder::FunctionSignature ConstraintGraphBuilder::checkFunctionS for (const auto& [name, g] : genericDefinitions) { genericTypes.push_back(g.ty); - signatureScope->typeBindings[name] = TypeFun{g.ty}; + signatureScope->privateTypeBindings[name] = TypeFun{g.ty}; } for (const auto& [name, g] : genericPackDefinitions) { genericTypePacks.push_back(g.tp); - signatureScope->typePackBindings[name] = g.tp; + signatureScope->privateTypePackBindings[name] = g.tp; } } else @@ -851,7 +1008,7 @@ TypeId ConstraintGraphBuilder::resolveType(const ScopePtr& scope, AstType* ty, b // TODO: Support imported types w/ require tracing. LUAU_ASSERT(!ref->prefix); - std::optional alias = scope->lookupTypeBinding(ref->name.value); + std::optional alias = scope->lookupType(ref->name.value); if (alias.has_value()) { @@ -949,13 +1106,13 @@ TypeId ConstraintGraphBuilder::resolveType(const ScopePtr& scope, AstType* ty, b for (const auto& [name, g] : genericDefinitions) { genericTypes.push_back(g.ty); - signatureScope->typeBindings[name] = TypeFun{g.ty}; + signatureScope->privateTypeBindings[name] = TypeFun{g.ty}; } for (const auto& [name, g] : genericPackDefinitions) { genericTypePacks.push_back(g.tp); - signatureScope->typePackBindings[name] = g.tp; + signatureScope->privateTypePackBindings[name] = g.tp; } } else @@ -1059,7 +1216,7 @@ TypePackId ConstraintGraphBuilder::resolveTypePack(const ScopePtr& scope, AstTyp } else if (auto gen = tp->as()) { - if (std::optional lookup = scope->lookupTypePackBinding(gen->genericName.value)) + if (std::optional lookup = scope->lookupPack(gen->genericName.value)) { result = *lookup; } diff --git a/Analysis/src/ConstraintSolver.cpp b/Analysis/src/ConstraintSolver.cpp index 0898f9aa..d8105ce3 100644 --- a/Analysis/src/ConstraintSolver.cpp +++ b/Analysis/src/ConstraintSolver.cpp @@ -484,6 +484,10 @@ bool ConstraintSolver::tryDispatch(const NameConstraint& c, NotNullpersistent) + return true; + if (TableTypeVar* ttv = getMutable(target)) ttv->name = c.name; else if (MetatableTypeVar* mtv = getMutable(target)) @@ -637,6 +641,10 @@ bool ConstraintSolver::tryDispatch(const TypeAliasExpansionConstraint& c, NotNul TypeId instantiated = *maybeInstantiated; TypeId target = follow(instantiated); + + if (target->persistent) + return true; + // Type function application will happily give us the exact same type if // there are e.g. generic saturatedTypeArguments that go unused. bool needsClone = follow(petv->fn.type) == target; diff --git a/Analysis/src/Frontend.cpp b/Analysis/src/Frontend.cpp index fe65853d..c450297f 100644 --- a/Analysis/src/Frontend.cpp +++ b/Analysis/src/Frontend.cpp @@ -818,21 +818,21 @@ const SourceModule* Frontend::getSourceModule(const ModuleName& moduleName) cons return const_cast(this)->getSourceModule(moduleName); } -NotNull Frontend::getGlobalScope() +ScopePtr Frontend::getGlobalScope() { if (!globalScope) { globalScope = typeChecker.globalScope; } - return NotNull(globalScope.get()); + return globalScope; } ModulePtr Frontend::check(const SourceModule& sourceModule, Mode mode, const ScopePtr& environmentScope) { ModulePtr result = std::make_shared(); - ConstraintGraphBuilder cgb{sourceModule.name, &result->internalTypes, NotNull(&iceHandler), getGlobalScope()}; + ConstraintGraphBuilder cgb{sourceModule.name, result, &result->internalTypes, NotNull(&iceHandler), getGlobalScope()}; cgb.visit(sourceModule.root); result->errors = std::move(cgb.errors); diff --git a/Analysis/src/Instantiation.cpp b/Analysis/src/Instantiation.cpp index 1a6013af..e98ab185 100644 --- a/Analysis/src/Instantiation.cpp +++ b/Analysis/src/Instantiation.cpp @@ -82,6 +82,8 @@ bool ReplaceGenerics::ignoreChildren(TypeId ty) // whenever we quantify, so the vectors overlap if and only if they are equal. return (!generics.empty() || !genericPacks.empty()) && (ftv->generics == generics) && (ftv->genericPacks == genericPacks); } + else if (FFlag::LuauClassTypeVarsInSubstitution && get(ty)) + return true; else { return false; diff --git a/Analysis/src/Linter.cpp b/Analysis/src/Linter.cpp index 9fce79a2..2d058376 100644 --- a/Analysis/src/Linter.cpp +++ b/Analysis/src/Linter.cpp @@ -14,6 +14,7 @@ LUAU_FASTINTVARIABLE(LuauSuggestionDistance, 4) LUAU_FASTFLAGVARIABLE(LuauLintGlobalNeverReadBeforeWritten, false) +LUAU_FASTFLAGVARIABLE(LuauLintComparisonPrecedence, false) namespace Luau { @@ -49,6 +50,7 @@ static const char* kWarningNames[] = { "MisleadingAndOr", "CommentDirective", "IntegerParsing", + "ComparisonPrecedence", }; // clang-format on @@ -2629,6 +2631,65 @@ private: } }; +class LintComparisonPrecedence : AstVisitor +{ +public: + LUAU_NOINLINE static void process(LintContext& context) + { + LintComparisonPrecedence pass; + pass.context = &context; + + context.root->visit(&pass); + } + +private: + LintContext* context; + + bool isComparison(AstExprBinary::Op op) + { + return op == AstExprBinary::CompareNe || op == AstExprBinary::CompareEq || op == AstExprBinary::CompareLt || op == AstExprBinary::CompareLe || + op == AstExprBinary::CompareGt || op == AstExprBinary::CompareGe; + } + + bool isNot(AstExpr* node) + { + AstExprUnary* expr = node->as(); + + return expr && expr->op == AstExprUnary::Not; + } + + bool visit(AstExprBinary* node) override + { + if (!isComparison(node->op)) + return true; + + // not X == Y; we silence this for not X == not Y as it's likely an intentional boolean comparison + if (isNot(node->left) && !isNot(node->right)) + { + std::string op = toString(node->op); + + if (node->op == AstExprBinary::CompareEq || node->op == AstExprBinary::CompareNe) + emitWarning(*context, LintWarning::Code_ComparisonPrecedence, node->location, + "not X %s Y is equivalent to (not X) %s Y; consider using X %s Y, or wrap one of the expressions in parentheses to silence", + op.c_str(), op.c_str(), node->op == AstExprBinary::CompareEq ? "~=" : "=="); + else + emitWarning(*context, LintWarning::Code_ComparisonPrecedence, node->location, + "not X %s Y is equivalent to (not X) %s Y; wrap one of the expressions in parentheses to silence", op.c_str(), op.c_str()); + } + else if (AstExprBinary* left = node->left->as(); left && isComparison(left->op)) + { + std::string lop = toString(left->op); + std::string rop = toString(node->op); + + emitWarning(*context, LintWarning::Code_ComparisonPrecedence, node->location, + "X %s Y %s Z is equivalent to (X %s Y) %s Z; wrap one of the expressions in parentheses to silence", lop.c_str(), rop.c_str(), + lop.c_str(), rop.c_str()); + } + + return true; + } +}; + static void fillBuiltinGlobals(LintContext& context, const AstNameTable& names, const ScopePtr& env) { ScopePtr current = env; @@ -2853,6 +2914,9 @@ std::vector lint(AstStat* root, const AstNameTable& names, const Sc if (context.warningEnabled(LintWarning::Code_IntegerParsing)) LintIntegerParsing::process(context); + if (context.warningEnabled(LintWarning::Code_ComparisonPrecedence) && FFlag::LuauLintComparisonPrecedence) + LintComparisonPrecedence::process(context); + std::sort(context.result.begin(), context.result.end(), WarningComparator()); return context.result; diff --git a/Analysis/src/Module.cpp b/Analysis/src/Module.cpp index 2b46da87..4e6b2585 100644 --- a/Analysis/src/Module.cpp +++ b/Analysis/src/Module.cpp @@ -17,6 +17,10 @@ LUAU_FASTFLAG(LuauLowerBoundsCalculation); LUAU_FASTFLAG(DebugLuauDeferredConstraintResolution); LUAU_FASTFLAGVARIABLE(LuauForceExportSurfacesToBeNormal, false); +LUAU_FASTFLAGVARIABLE(LuauClonePublicInterfaceLess, false); +LUAU_FASTFLAG(LuauSubstitutionReentrant); +LUAU_FASTFLAG(LuauClassTypeVarsInSubstitution); +LUAU_FASTFLAG(LuauSubstitutionFixMissingFields); namespace Luau { @@ -86,6 +90,118 @@ struct ForceNormal : TypeVarOnceVisitor } }; +struct ClonePublicInterface : Substitution +{ + NotNull module; + + ClonePublicInterface(const TxnLog* log, Module* module) + : Substitution(log, &module->interfaceTypes) + , module(module) + { + LUAU_ASSERT(module); + } + + bool isDirty(TypeId ty) override + { + if (ty->owningArena == &module->internalTypes) + return true; + + if (const FunctionTypeVar* ftv = get(ty)) + return ftv->level.level != 0; + if (const TableTypeVar* ttv = get(ty)) + return ttv->level.level != 0; + return false; + } + + bool isDirty(TypePackId tp) override + { + return tp->owningArena == &module->internalTypes; + } + + TypeId clean(TypeId ty) override + { + TypeId result = clone(ty); + + if (FunctionTypeVar* ftv = getMutable(result)) + ftv->level = TypeLevel{0, 0}; + else if (TableTypeVar* ttv = getMutable(result)) + ttv->level = TypeLevel{0, 0}; + + return result; + } + + TypePackId clean(TypePackId tp) override + { + return clone(tp); + } + + TypeId cloneType(TypeId ty) + { + LUAU_ASSERT(FFlag::LuauSubstitutionReentrant && FFlag::LuauSubstitutionFixMissingFields); + + std::optional result = substitute(ty); + if (result) + { + return *result; + } + else + { + module->errors.push_back(TypeError{module->scopes[0].first, UnificationTooComplex{}}); + return getSingletonTypes().errorRecoveryType(); + } + } + + TypePackId cloneTypePack(TypePackId tp) + { + LUAU_ASSERT(FFlag::LuauSubstitutionReentrant && FFlag::LuauSubstitutionFixMissingFields); + + std::optional result = substitute(tp); + if (result) + { + return *result; + } + else + { + module->errors.push_back(TypeError{module->scopes[0].first, UnificationTooComplex{}}); + return getSingletonTypes().errorRecoveryTypePack(); + } + } + + TypeFun cloneTypeFun(const TypeFun& tf) + { + LUAU_ASSERT(FFlag::LuauSubstitutionReentrant && FFlag::LuauSubstitutionFixMissingFields); + + std::vector typeParams; + std::vector typePackParams; + + for (GenericTypeDefinition typeParam : tf.typeParams) + { + TypeId ty = cloneType(typeParam.ty); + std::optional defaultValue; + + if (typeParam.defaultValue) + defaultValue = cloneType(*typeParam.defaultValue); + + typeParams.push_back(GenericTypeDefinition{ty, defaultValue}); + } + + for (GenericTypePackDefinition typePackParam : tf.typePackParams) + { + TypePackId tp = cloneTypePack(typePackParam.tp); + std::optional defaultValue; + + if (typePackParam.defaultValue) + defaultValue = cloneTypePack(*typePackParam.defaultValue); + + typePackParams.push_back(GenericTypePackDefinition{tp, defaultValue}); + } + + TypeId type = cloneType(tf.type); + + return TypeFun{typeParams, typePackParams, type}; + } +}; + Module::~Module() { unfreeze(interfaceTypes); @@ -106,12 +222,21 @@ void Module::clonePublicInterface(InternalErrorReporter& ice) std::unordered_map* exportedTypeBindings = FFlag::DebugLuauDeferredConstraintResolution ? nullptr : &moduleScope->exportedTypeBindings; - returnType = clone(returnType, interfaceTypes, cloneState); + TxnLog log; + ClonePublicInterface clonePublicInterface{&log, this}; + + if (FFlag::LuauClonePublicInterfaceLess) + returnType = clonePublicInterface.cloneTypePack(returnType); + else + returnType = clone(returnType, interfaceTypes, cloneState); moduleScope->returnType = returnType; if (varargPack) { - varargPack = clone(*varargPack, interfaceTypes, cloneState); + if (FFlag::LuauClonePublicInterfaceLess) + varargPack = clonePublicInterface.cloneTypePack(*varargPack); + else + varargPack = clone(*varargPack, interfaceTypes, cloneState); moduleScope->varargPack = varargPack; } @@ -134,7 +259,10 @@ void Module::clonePublicInterface(InternalErrorReporter& ice) { for (auto& [name, tf] : *exportedTypeBindings) { - tf = clone(tf, interfaceTypes, cloneState); + if (FFlag::LuauClonePublicInterfaceLess) + tf = clonePublicInterface.cloneTypeFun(tf); + else + tf = clone(tf, interfaceTypes, cloneState); if (FFlag::LuauLowerBoundsCalculation) { normalize(tf.type, interfaceTypes, ice); @@ -168,7 +296,10 @@ void Module::clonePublicInterface(InternalErrorReporter& ice) for (auto& [name, ty] : declaredGlobals) { - ty = clone(ty, interfaceTypes, cloneState); + if (FFlag::LuauClonePublicInterfaceLess) + ty = clonePublicInterface.cloneType(ty); + else + ty = clone(ty, interfaceTypes, cloneState); if (FFlag::LuauLowerBoundsCalculation) { normalize(ty, interfaceTypes, ice); diff --git a/Analysis/src/Normalize.cpp b/Analysis/src/Normalize.cpp index 9ae3b404..33f369a7 100644 --- a/Analysis/src/Normalize.cpp +++ b/Analysis/src/Normalize.cpp @@ -162,7 +162,8 @@ struct Normalize final : TypeVarVisitor // It should never be the case that this TypeVar is normal, but is bound to a non-normal type, except in nontrivial cases. LUAU_ASSERT(!ty->normal || ty->normal == btv.boundTo->normal); - asMutable(ty)->normal = btv.boundTo->normal; + if (!ty->normal) + asMutable(ty)->normal = btv.boundTo->normal; return !ty->normal; } diff --git a/Analysis/src/Quantify.cpp b/Analysis/src/Quantify.cpp index 03049cca..ce4afe22 100644 --- a/Analysis/src/Quantify.cpp +++ b/Analysis/src/Quantify.cpp @@ -5,11 +5,13 @@ #include "Luau/Scope.h" #include "Luau/Substitution.h" #include "Luau/TxnLog.h" +#include "Luau/TypeVar.h" #include "Luau/VisitTypeVar.h" LUAU_FASTFLAG(DebugLuauSharedSelf) LUAU_FASTFLAG(DebugLuauDeferredConstraintResolution); LUAU_FASTFLAGVARIABLE(LuauQuantifyConstrained, false) +LUAU_FASTFLAG(LuauClassTypeVarsInSubstitution) namespace Luau { @@ -297,6 +299,9 @@ struct PureQuantifier : Substitution bool ignoreChildren(TypeId ty) override { + if (FFlag::LuauClassTypeVarsInSubstitution && get(ty)) + return true; + return ty->persistent; } bool ignoreChildren(TypePackId ty) override diff --git a/Analysis/src/Scope.cpp b/Analysis/src/Scope.cpp index bee16908..c129b973 100644 --- a/Analysis/src/Scope.cpp +++ b/Analysis/src/Scope.cpp @@ -122,34 +122,4 @@ std::optional Scope::lookup(Symbol sym) } } -std::optional Scope::lookupTypeBinding(const Name& name) -{ - Scope* s = this; - while (s) - { - auto it = s->typeBindings.find(name); - if (it != s->typeBindings.end()) - return it->second; - - s = s->parent.get(); - } - - return std::nullopt; -} - -std::optional Scope::lookupTypePackBinding(const Name& name) -{ - Scope* s = this; - while (s) - { - auto it = s->typePackBindings.find(name); - if (it != s->typePackBindings.end()) - return it->second; - - s = s->parent.get(); - } - - return std::nullopt; -} - } // namespace Luau diff --git a/Analysis/src/Substitution.cpp b/Analysis/src/Substitution.cpp index 148c9ee2..0beeb58c 100644 --- a/Analysis/src/Substitution.cpp +++ b/Analysis/src/Substitution.cpp @@ -8,9 +8,9 @@ #include #include -LUAU_FASTFLAGVARIABLE(LuauAnyificationMustClone, false) LUAU_FASTFLAGVARIABLE(LuauSubstitutionFixMissingFields, false) LUAU_FASTFLAG(LuauLowerBoundsCalculation) +LUAU_FASTFLAG(LuauClonePublicInterfaceLess) LUAU_FASTINTVARIABLE(LuauTarjanChildLimit, 10000) LUAU_FASTFLAGVARIABLE(LuauClassTypeVarsInSubstitution, false) LUAU_FASTFLAG(LuauUnknownAndNeverType) @@ -472,7 +472,7 @@ std::optional Substitution::substitute(TypePackId tp) TypeId Substitution::clone(TypeId ty) { - return shallowClone(ty, *arena, log); + return shallowClone(ty, *arena, log, /* alwaysClone */ FFlag::LuauClonePublicInterfaceLess); } TypePackId Substitution::clone(TypePackId tp) @@ -497,6 +497,10 @@ TypePackId Substitution::clone(TypePackId tp) clone.hidden = vtp->hidden; return addTypePack(std::move(clone)); } + else if (FFlag::LuauClonePublicInterfaceLess) + { + return addTypePack(*tp); + } else return tp; } @@ -557,7 +561,7 @@ void Substitution::replaceChildren(TypeId ty) if (ignoreChildren(ty)) return; - if (FFlag::LuauAnyificationMustClone && ty->owningArena != arena) + if (ty->owningArena != arena) return; if (FunctionTypeVar* ftv = getMutable(ty)) @@ -638,7 +642,7 @@ void Substitution::replaceChildren(TypePackId tp) if (ignoreChildren(tp)) return; - if (FFlag::LuauAnyificationMustClone && tp->owningArena != arena) + if (tp->owningArena != arena) return; if (TypePack* tpp = getMutable(tp)) diff --git a/Analysis/src/TypeChecker2.cpp b/Analysis/src/TypeChecker2.cpp index 53b069cf..9d97e8d9 100644 --- a/Analysis/src/TypeChecker2.cpp +++ b/Analysis/src/TypeChecker2.cpp @@ -10,6 +10,7 @@ #include "Luau/Normalize.h" #include "Luau/TxnLog.h" #include "Luau/TypeUtils.h" +#include "Luau/TypeVar.h" #include "Luau/Unifier.h" #include "Luau/ToString.h" @@ -282,18 +283,14 @@ struct TypeChecker2 : public AstVisitor // leftType must have a property called indexName->index - std::optional t = findTablePropertyRespectingMeta(module->errors, leftType, indexName->index.value, indexName->location); - if (t) + std::optional ty = getIndexTypeFromType(module->getModuleScope(), leftType, indexName->index.value, indexName->location, /* addErrors */ true); + if (ty) { - if (!isSubtype(resultType, *t, ice)) + if (!isSubtype(resultType, *ty, ice)) { - reportError(TypeMismatch{resultType, *t}, indexName->location); + reportError(TypeMismatch{resultType, *ty}, indexName->location); } } - else - { - reportError(UnknownProperty{leftType, indexName->index.value}, indexName->location); - } return true; } @@ -324,6 +321,22 @@ struct TypeChecker2 : public AstVisitor return true; } + bool visit(AstExprTypeAssertion* expr) override + { + TypeId annotationType = lookupAnnotation(expr->annotation); + TypeId computedType = lookupType(expr->expr); + + // Note: As an optimization, we try 'number <: number | string' first, as that is the more likely case. + if (isSubtype(annotationType, computedType, ice)) + return true; + + if (isSubtype(computedType, annotationType, ice)) + return true; + + reportError(TypesAreUnrelated{computedType, annotationType}, expr->location); + return true; + } + /** Extract a TypeId for the first type of the provided pack. * * Note that this may require modifying some types. I hope this doesn't cause problems! @@ -374,7 +387,7 @@ struct TypeChecker2 : public AstVisitor // TODO: Imported types - std::optional alias = scope->lookupTypeBinding(ty->name.value); + std::optional alias = scope->lookupType(ty->name.value); if (alias.has_value()) { @@ -473,7 +486,7 @@ struct TypeChecker2 : public AstVisitor } else { - if (scope->lookupTypePackBinding(ty->name.value)) + if (scope->lookupPack(ty->name.value)) { reportError( SwappedGenericTypeParameter{ @@ -501,10 +514,10 @@ struct TypeChecker2 : public AstVisitor Scope* scope = findInnermostScope(tp->location); LUAU_ASSERT(scope); - std::optional alias = scope->lookupTypePackBinding(tp->genericName.value); + std::optional alias = scope->lookupPack(tp->genericName.value); if (!alias.has_value()) { - if (scope->lookupTypeBinding(tp->genericName.value)) + if (scope->lookupType(tp->genericName.value)) { reportError( SwappedGenericTypeParameter{ @@ -531,6 +544,142 @@ struct TypeChecker2 : public AstVisitor { module->errors.emplace_back(std::move(e)); } + + std::optional getIndexTypeFromType( + const ScopePtr& scope, TypeId type, const Name& name, const Location& location, bool addErrors) + { + type = follow(type); + + if (get(type) || get(type) || get(type)) + return type; + + if (auto f = get(type)) + *asMutable(type) = TableTypeVar{TableState::Free, f->level}; + + if (isString(type)) + { + std::optional mtIndex = Luau::findMetatableEntry(module->errors, singletonTypes.stringType, "__index", location); + LUAU_ASSERT(mtIndex); + type = *mtIndex; + } + + if (TableTypeVar* tableType = getMutableTableType(type)) + { + + return findTablePropertyRespectingMeta(module->errors, type, name, location); + } + else if (const ClassTypeVar* cls = get(type)) + { + const Property* prop = lookupClassProp(cls, name); + if (prop) + return prop->type; + } + else if (const UnionTypeVar* utv = get(type)) + { + std::vector goodOptions; + std::vector badOptions; + + for (TypeId t : utv) + { + // TODO: we should probably limit recursion here? + // RecursionLimiter _rl(&recursionCount, FInt::LuauTypeInferRecursionLimit); + + // Not needed when we normalize types. + if (get(follow(t))) + return t; + + if (std::optional ty = getIndexTypeFromType(scope, t, name, location, /* addErrors= */ false)) + goodOptions.push_back(*ty); + else + badOptions.push_back(t); + } + + if (!badOptions.empty()) + { + if (addErrors) + { + if (goodOptions.empty()) + reportError(UnknownProperty{type, name}, location); + else + reportError(MissingUnionProperty{type, badOptions, name}, location); + } + return std::nullopt; + } + + std::vector result = reduceUnion(goodOptions); + if (result.empty()) + return singletonTypes.neverType; + + if (result.size() == 1) + return result[0]; + + return module->internalTypes.addType(UnionTypeVar{std::move(result)}); + } + else if (const IntersectionTypeVar* itv = get(type)) + { + std::vector parts; + + for (TypeId t : itv->parts) + { + // TODO: we should probably limit recursion here? + // RecursionLimiter _rl(&recursionCount, FInt::LuauTypeInferRecursionLimit); + + if (std::optional ty = getIndexTypeFromType(scope, t, name, location, /* addErrors= */ false)) + parts.push_back(*ty); + } + + // If no parts of the intersection had the property we looked up for, it never existed at all. + if (parts.empty()) + { + if (addErrors) + reportError(UnknownProperty{type, name}, location); + return std::nullopt; + } + + if (parts.size() == 1) + return parts[0]; + + return module->internalTypes.addType(IntersectionTypeVar{std::move(parts)}); // Not at all correct. + } + + if (addErrors) + reportError(UnknownProperty{type, name}, location); + + return std::nullopt; + } + + std::vector reduceUnion(const std::vector& types) + { + std::vector result; + for (TypeId t : types) + { + t = follow(t); + if (get(t)) + continue; + + if (get(t) || get(t)) + return {t}; + + if (const UnionTypeVar* utv = get(t)) + { + for (TypeId ty : utv) + { + ty = follow(ty); + if (get(ty)) + continue; + if (get(ty) || get(ty)) + return {ty}; + + if (result.end() == std::find(result.begin(), result.end(), ty)) + result.push_back(ty); + } + } + else if (std::find(result.begin(), result.end(), t) == result.end()) + result.push_back(t); + } + + return result; + } }; void check(const SourceModule& sourceModule, Module* module) diff --git a/Analysis/src/TypeInfer.cpp b/Analysis/src/TypeInfer.cpp index bdda195c..7ab23360 100644 --- a/Analysis/src/TypeInfer.cpp +++ b/Analysis/src/TypeInfer.cpp @@ -33,7 +33,6 @@ LUAU_FASTINTVARIABLE(LuauVisitRecursionLimit, 500) LUAU_FASTFLAG(LuauKnowsTheDataModel3) LUAU_FASTFLAG(LuauAutocompleteDynamicLimits) LUAU_FASTFLAGVARIABLE(LuauExpectedTableUnionIndexerType, false) -LUAU_FASTFLAGVARIABLE(LuauIndexSilenceErrors, false) LUAU_FASTFLAGVARIABLE(LuauLowerBoundsCalculation, false) LUAU_FASTFLAGVARIABLE(DebugLuauFreezeDuringUnification, false) LUAU_FASTFLAGVARIABLE(LuauSelfCallAutocompleteFix3, false) @@ -830,6 +829,14 @@ struct Demoter : Substitution return get(tp); } + bool ignoreChildren(TypeId ty) override + { + if (FFlag::LuauClassTypeVarsInSubstitution && get(ty)) + return true; + + return false; + } + TypeId clean(TypeId ty) override { auto ftv = get(ty); @@ -1925,7 +1932,7 @@ std::optional TypeChecker::findTablePropertyRespectingMeta(TypeId lhsTyp { ErrorVec errors; auto result = Luau::findTablePropertyRespectingMeta(errors, lhsType, name, location); - if (!FFlag::LuauIndexSilenceErrors || addErrors) + if (addErrors) reportErrors(errors); return result; } @@ -1934,7 +1941,7 @@ std::optional TypeChecker::findMetatableEntry(TypeId type, std::string e { ErrorVec errors; auto result = Luau::findMetatableEntry(errors, type, entry, location); - if (!FFlag::LuauIndexSilenceErrors || addErrors) + if (addErrors) reportErrors(errors); return result; } @@ -1946,7 +1953,7 @@ std::optional TypeChecker::getIndexTypeFromType( std::optional result = getIndexTypeFromTypeImpl(scope, type, name, location, addErrors); - if (FFlag::LuauIndexSilenceErrors && !addErrors) + if (!addErrors) LUAU_ASSERT(errorCount == currentModule->errors.size()); return result; diff --git a/Analysis/src/Unifier.cpp b/Analysis/src/Unifier.cpp index e099817f..a0f4725d 100644 --- a/Analysis/src/Unifier.cpp +++ b/Analysis/src/Unifier.cpp @@ -22,6 +22,7 @@ LUAU_FASTFLAG(LuauErrorRecoveryType); LUAU_FASTFLAG(LuauUnknownAndNeverType) LUAU_FASTFLAG(LuauQuantifyConstrained) LUAU_FASTFLAGVARIABLE(LuauScalarShapeSubtyping, false) +LUAU_FASTFLAG(LuauClassTypeVarsInSubstitution) namespace Luau { @@ -273,6 +274,9 @@ TypePackId Widen::clean(TypePackId) bool Widen::ignoreChildren(TypeId ty) { + if (FFlag::LuauClassTypeVarsInSubstitution && get(ty)) + return true; + return !log->is(ty); } @@ -370,25 +374,14 @@ void Unifier::tryUnify_(TypeId subTy, TypeId superTy, bool isFunctionCall, bool if (superFree && subFree && superFree->level.subsumes(subFree->level)) { - occursCheck(subTy, superTy); - - // The occurrence check might have caused superTy no longer to be a free type - bool occursFailed = bool(log.getMutable(subTy)); - - if (!occursFailed) - { + if (!occursCheck(subTy, superTy)) log.replace(subTy, BoundTypeVar(superTy)); - } return; } else if (superFree && subFree) { - occursCheck(superTy, subTy); - - bool occursFailed = bool(log.getMutable(superTy)); - - if (!occursFailed) + if (!occursCheck(superTy, subTy)) { if (superFree->level.subsumes(subFree->level)) { @@ -402,24 +395,18 @@ void Unifier::tryUnify_(TypeId subTy, TypeId superTy, bool isFunctionCall, bool } else if (superFree) { - TypeLevel superLevel = superFree->level; - - occursCheck(superTy, subTy); - bool occursFailed = bool(log.getMutable(superTy)); - // Unification can't change the level of a generic. auto subGeneric = log.getMutable(subTy); - if (subGeneric && !subGeneric->level.subsumes(superLevel)) + if (subGeneric && !subGeneric->level.subsumes(superFree->level)) { // TODO: a more informative error message? CLI-39912 reportError(TypeError{location, GenericError{"Generic subtype escaping scope"}}); return; } - // The occurrence check might have caused superTy no longer to be a free type - if (!occursFailed) + if (!occursCheck(superTy, subTy)) { - promoteTypeLevels(log, types, superLevel, subTy); + promoteTypeLevels(log, types, superFree->level, subTy); Widen widen{types}; log.replace(superTy, BoundTypeVar(widen(subTy))); @@ -437,11 +424,6 @@ void Unifier::tryUnify_(TypeId subTy, TypeId superTy, bool isFunctionCall, bool return; } - TypeLevel subLevel = subFree->level; - - occursCheck(subTy, superTy); - bool occursFailed = bool(log.getMutable(subTy)); - // Unification can't change the level of a generic. auto superGeneric = log.getMutable(superTy); if (superGeneric && !superGeneric->level.subsumes(subFree->level)) @@ -451,9 +433,9 @@ void Unifier::tryUnify_(TypeId subTy, TypeId superTy, bool isFunctionCall, bool return; } - if (!occursFailed) + if (!occursCheck(subTy, superTy)) { - promoteTypeLevels(log, types, subLevel, superTy); + promoteTypeLevels(log, types, subFree->level, superTy); log.replace(subTy, BoundTypeVar(superTy)); } @@ -1033,9 +1015,7 @@ void Unifier::tryUnify_(TypePackId subTp, TypePackId superTp, bool isFunctionCal if (log.getMutable(superTp)) { - occursCheck(superTp, subTp); - - if (!log.getMutable(superTp)) + if (!occursCheck(superTp, subTp)) { Widen widen{types}; log.replace(superTp, Unifiable::Bound(widen(subTp))); @@ -1043,9 +1023,7 @@ void Unifier::tryUnify_(TypePackId subTp, TypePackId superTp, bool isFunctionCal } else if (log.getMutable(subTp)) { - occursCheck(subTp, superTp); - - if (!log.getMutable(subTp)) + if (!occursCheck(subTp, superTp)) { log.replace(subTp, Unifiable::Bound(superTp)); } @@ -2106,8 +2084,8 @@ void Unifier::unifyLowerBound(TypePackId subTy, TypePackId superTy, TypeLevel de { TypePackId tailPack = follow(*t); - if (log.get(tailPack)) - occursCheck(tailPack, subTy); + if (log.get(tailPack) && occursCheck(tailPack, subTy)) + return; FreeTypePack* freeTailPack = log.getMutable(tailPack); if (!freeTailPack) @@ -2180,32 +2158,35 @@ void Unifier::unifyLowerBound(TypePackId subTy, TypePackId superTy, TypeLevel de } } -void Unifier::occursCheck(TypeId needle, TypeId haystack) +bool Unifier::occursCheck(TypeId needle, TypeId haystack) { sharedState.tempSeenTy.clear(); return occursCheck(sharedState.tempSeenTy, needle, haystack); } -void Unifier::occursCheck(DenseHashSet& seen, TypeId needle, TypeId haystack) +bool Unifier::occursCheck(DenseHashSet& seen, TypeId needle, TypeId haystack) { RecursionLimiter _ra(&sharedState.counters.recursionCount, FFlag::LuauAutocompleteDynamicLimits ? sharedState.counters.recursionLimit : FInt::LuauTypeInferRecursionLimit); + bool occurrence = false; + auto check = [&](TypeId tv) { - occursCheck(seen, needle, tv); + if (occursCheck(seen, needle, tv)) + occurrence = true; }; needle = log.follow(needle); haystack = log.follow(haystack); if (seen.find(haystack)) - return; + return false; seen.insert(haystack); if (log.getMutable(needle)) - return; + return false; if (!log.getMutable(needle)) ice("Expected needle to be free"); @@ -2215,11 +2196,11 @@ void Unifier::occursCheck(DenseHashSet& seen, TypeId needle, TypeId hays reportError(TypeError{location, OccursCheckFailed{}}); log.replace(needle, *getSingletonTypes().errorRecoveryType()); - return; + return true; } if (log.getMutable(haystack)) - return; + return false; else if (auto a = log.getMutable(haystack)) { for (TypeId ty : a->options) @@ -2235,27 +2216,29 @@ void Unifier::occursCheck(DenseHashSet& seen, TypeId needle, TypeId hays for (TypeId ty : a->parts) check(ty); } + + return occurrence; } -void Unifier::occursCheck(TypePackId needle, TypePackId haystack) +bool Unifier::occursCheck(TypePackId needle, TypePackId haystack) { sharedState.tempSeenTp.clear(); return occursCheck(sharedState.tempSeenTp, needle, haystack); } -void Unifier::occursCheck(DenseHashSet& seen, TypePackId needle, TypePackId haystack) +bool Unifier::occursCheck(DenseHashSet& seen, TypePackId needle, TypePackId haystack) { needle = log.follow(needle); haystack = log.follow(haystack); if (seen.find(haystack)) - return; + return false; seen.insert(haystack); if (log.getMutable(needle)) - return; + return false; if (!log.getMutable(needle)) ice("Expected needle pack to be free"); @@ -2270,7 +2253,7 @@ void Unifier::occursCheck(DenseHashSet& seen, TypePackId needle, Typ reportError(TypeError{location, OccursCheckFailed{}}); log.replace(needle, *getSingletonTypes().errorRecoveryTypePack()); - return; + return true; } if (auto a = get(haystack); a && a->tail) @@ -2281,6 +2264,8 @@ void Unifier::occursCheck(DenseHashSet& seen, TypePackId needle, Typ break; } + + return false; } Unifier Unifier::makeChildUnifier() diff --git a/CLI/Reduce.cpp b/CLI/Reduce.cpp new file mode 100644 index 00000000..d24c9874 --- /dev/null +++ b/CLI/Reduce.cpp @@ -0,0 +1,511 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details + +#include "Luau/Ast.h" +#include "Luau/Common.h" +#include "Luau/Parser.h" +#include "Luau/Transpiler.h" + +#include "FileUtils.h" + +#include +#include +#include +#include +#include + +#define VERBOSE 0 // 1 - print out commandline invocations. 2 - print out stdout + +#ifdef _WIN32 + +const auto popen = &_popen; +const auto pclose = &_pclose; + +#endif + +using namespace Luau; + +enum class TestResult +{ + BugFound, // We encountered the bug we are trying to isolate + NoBug, // We did not encounter the bug we are trying to isolate +}; + +struct Enqueuer : public AstVisitor +{ + std::queue* queue; + + explicit Enqueuer(std::queue* queue) + : queue(queue) + { + LUAU_ASSERT(queue); + } + + bool visit(AstStatBlock* block) override + { + queue->push(block); + return false; + } +}; + +struct Reducer +{ + Allocator allocator; + AstNameTable nameTable{allocator}; + ParseOptions parseOptions; + + ParseResult parseResult; + AstStatBlock* root; + + std::string tempScriptName; + + std::string appName; + std::vector appArgs; + std::string_view searchText; + + Reducer() + { + parseOptions.captureComments = true; + } + + std::string readLine(FILE* f) + { + std::string line = ""; + char buffer[256]; + while (fgets(buffer, sizeof(buffer), f)) + { + auto len = strlen(buffer); + line += std::string(buffer, len); + if (buffer[len - 1] == '\n') + break; + } + + return line; + } + + void writeTempScript(bool minify = false) + { + std::string source = transpileWithTypes(*root); + + if (minify) + { + size_t pos = 0; + do + { + pos = source.find("\n\n", pos); + if (pos == std::string::npos) + break; + + source.erase(pos, 1); + } while (true); + } + + FILE* f = fopen(tempScriptName.c_str(), "w"); + if (!f) + { + printf("Unable to open temp script to %s\n", tempScriptName.c_str()); + exit(2); + } + + for (const HotComment& comment : parseResult.hotcomments) + fprintf(f, "--!%s\n", comment.content.c_str()); + + auto written = fwrite(source.data(), 1, source.size(), f); + if (written != source.size()) + { + printf("??? %zu %zu\n", written, source.size()); + printf("Unable to write to temp script %s\n", tempScriptName.c_str()); + exit(3); + } + + fclose(f); + } + + int step = 0; + + std::string escape(const std::string& s) + { + std::string result; + result.reserve(s.size() + 20); // guess + result += '"'; + for (char c : s) + { + if (c == '"') + result += '\\'; + result += c; + } + result += '"'; + + return result; + } + + TestResult run() + { + writeTempScript(); + + std::string command = appName + " " + escape(tempScriptName); + for (const auto& arg : appArgs) + command += " " + escape(arg); + +#if VERBOSE >= 1 + printf("running %s\n", command.c_str()); +#endif + + TestResult result = TestResult::NoBug; + + ++step; + printf("Step %4d...\n", step); + + FILE* p = popen(command.c_str(), "r"); + + while (!feof(p)) + { + std::string s = readLine(p); +#if VERBOSE >= 2 + printf("%s", s.c_str()); +#endif + if (std::string::npos != s.find(searchText)) + { + result = TestResult::BugFound; + break; + } + } + + pclose(p); + + return result; + } + + std::vector getNestedStats(AstStat* stat) + { + std::vector result; + + auto append = [&](AstStatBlock* block) { + if (block) + result.insert(result.end(), block->body.data, block->body.data + block->body.size); + }; + + if (auto block = stat->as()) + append(block); + else if (auto ifs = stat->as()) + { + append(ifs->thenbody); + if (ifs->elsebody) + { + if (AstStatBlock* elseBlock = ifs->elsebody->as()) + append(elseBlock); + else if (AstStatIf* elseIf = ifs->elsebody->as()) + { + auto innerStats = getNestedStats(elseIf); + result.insert(end(result), begin(innerStats), end(innerStats)); + } + else + { + printf("AstStatIf's else clause can have more statement types than I thought\n"); + LUAU_ASSERT(0); + } + } + } + else if (auto w = stat->as()) + append(w->body); + else if (auto r = stat->as()) + append(r->body); + else if (auto f = stat->as()) + append(f->body); + else if (auto f = stat->as()) + append(f->body); + else if (auto f = stat->as()) + append(f->func->body); + else if (auto f = stat->as()) + append(f->func->body); + + return result; + } + + // Move new body data into allocator-managed storage so that it's safe to keep around longterm. + AstStat** reallocateStatements(const std::vector& statements) + { + AstStat** newData = static_cast(allocator.allocate(sizeof(AstStat*) * statements.size())); + std::copy(statements.data(), statements.data() + statements.size(), newData); + + return newData; + } + + // Semiopen interval + using Span = std::pair; + + // Generates 'chunks' semiopen spans of equal-ish size to span the indeces running from 0 to 'size' + // Also inverses. + std::vector> generateSpans(size_t size, size_t chunks) + { + if (size <= 1) + return {}; + + LUAU_ASSERT(chunks > 0); + size_t chunkLength = std::max(1, size / chunks); + + std::vector> result; + + auto append = [&result](Span a, Span b) { + if (a.first == a.second && b.first == b.second) + return; + else + result.emplace_back(a, b); + }; + + size_t i = 0; + while (i < size) + { + size_t end = std::min(i + chunkLength, size); + append(Span{0, i}, Span{end, size}); + + i = end; + } + + i = 0; + while (i < size) + { + size_t end = std::min(i + chunkLength, size); + append(Span{i, end}, Span{size, size}); + + i = end; + } + + return result; + } + + // Returns the statements of block within span1 and span2 + // Also has the hokey restriction that span1 must come before span2 + std::vector prunedSpan(AstStatBlock* block, Span span1, Span span2) + { + std::vector result; + + for (size_t i = span1.first; i < span1.second; ++i) + result.push_back(block->body.data[i]); + + for (size_t i = span2.first; i < span2.second; ++i) + result.push_back(block->body.data[i]); + + return result; + } + + // returns true if anything was culled plus the chunk count + std::pair deleteChildStatements(AstStatBlock* block, size_t chunkCount) + { + if (block->body.size == 0) + return {false, chunkCount}; + + do + { + auto permutations = generateSpans(block->body.size, chunkCount); + for (const auto& [span1, span2] : permutations) + { + auto tempStatements = prunedSpan(block, span1, span2); + AstArray backupBody{tempStatements.data(), tempStatements.size()}; + + std::swap(block->body, backupBody); + TestResult result = run(); + if (result == TestResult::BugFound) + { + // The bug still reproduces without the statements we've culled. Commit. + block->body.data = reallocateStatements(tempStatements); + return {true, std::max(2, chunkCount - 1)}; + } + else + { + // The statements we've culled are critical for the reproduction of the bug. + // TODO try promoting its contents into this scope + std::swap(block->body, backupBody); + } + } + + chunkCount *= 2; + } while (chunkCount <= block->body.size); + + return {false, block->body.size}; + } + + bool deleteChildStatements(AstStatBlock* b) + { + bool result = false; + + size_t chunkCount = 2; + while (true) + { + auto [workDone, newChunkCount] = deleteChildStatements(b, chunkCount); + if (workDone) + { + result = true; + chunkCount = newChunkCount; + continue; + } + else + break; + } + + return result; + } + + bool tryPromotingChildStatements(AstStatBlock* b, size_t index) + { + std::vector tempStats(b->body.data, b->body.data + b->body.size); + AstStat* removed = tempStats.at(index); + tempStats.erase(begin(tempStats) + index); + + std::vector nestedStats = getNestedStats(removed); + tempStats.insert(begin(tempStats) + index, begin(nestedStats), end(nestedStats)); + + AstArray tempArray{tempStats.data(), tempStats.size()}; + std::swap(b->body, tempArray); + + TestResult result = run(); + + if (result == TestResult::BugFound) + { + b->body.data = reallocateStatements(tempStats); + return true; + } + else + { + std::swap(b->body, tempArray); + return false; + } + } + + // We live with some weirdness because I'm kind of lazy: If a statement's + // contents are promoted, we try promoting those prometed statements right + // away. I don't think it matters: If we can delete a statement and still + // exhibit the bug, we should do so. The order isn't so important. + bool tryPromotingChildStatements(AstStatBlock* b) + { + size_t i = 0; + while (i < b->body.size) + { + bool promoted = tryPromotingChildStatements(b, i); + if (!promoted) + ++i; + } + + return false; + } + + void walk(AstStatBlock* block) + { + std::queue queue; + Enqueuer enqueuer{&queue}; + + queue.push(block); + + while (!queue.empty()) + { + AstStatBlock* b = queue.front(); + queue.pop(); + + bool result = false; + do + { + result = deleteChildStatements(b); + + /* Try other reductions here before we walk into child statements + * Other reductions to try someday: + * + * Promoting a statement's children to the enclosing block. + * Deleting type annotations + * Deleting parts of type annotations + * Replacing subexpressions with ({} :: any) + * Inlining type aliases + * Inlining constants + * Inlining functions + */ + result |= tryPromotingChildStatements(b); + } while (result); + + for (AstStat* stat : b->body) + stat->visit(&enqueuer); + } + } + + void run(const std::string scriptName, const std::string appName, const std::vector& appArgs, std::string_view source, + std::string_view searchText) + { + tempScriptName = scriptName; + if (tempScriptName.substr(tempScriptName.size() - 4) == ".lua") + { + tempScriptName.erase(tempScriptName.size() - 4); + tempScriptName += "-reduced.lua"; + } + else + { + this->tempScriptName = scriptName + "-reduced"; + } + +#if 0 + // Handy debugging trick: VS Code will update its view of the file in realtime as it is edited. + std::string wheee = "code " + tempScriptName; + system(wheee.c_str()); +#endif + + printf("Temp script: %s\n", tempScriptName.c_str()); + + this->appName = appName; + this->appArgs = appArgs; + this->searchText = searchText; + + parseResult = Parser::parse(source.data(), source.size(), nameTable, allocator, parseOptions); + if (!parseResult.errors.empty()) + { + printf("Parse errors\n"); + exit(1); + } + + root = parseResult.root; + + const TestResult initialResult = run(); + if (initialResult == TestResult::NoBug) + { + printf("Could not find failure string in the unmodified script! Check your commandline arguments\n"); + exit(2); + } + + walk(root); + + writeTempScript(/* minify */ true); + + printf("Done! Check %s\n", tempScriptName.c_str()); + } +}; + +[[noreturn]] void help(const std::vector& args) +{ + printf("Syntax: %s script application \"search text\" [arguments]\n", args[0].data()); + exit(1); +} + +int main(int argc, char** argv) +{ + const std::vector args(argv, argv + argc); + + if (args.size() < 4) + help(args); + + for (int i = 1; i < args.size(); ++i) + { + if (args[i] == "--help") + help(args); + } + + const std::string scriptName = argv[1]; + const std::string appName = argv[2]; + const std::string searchText = argv[3]; + const std::vector appArgs(begin(args) + 4, end(args)); + + std::optional source = readFile(scriptName); + + if (!source) + { + printf("Could not read source %s\n", argv[1]); + exit(1); + } + + Reducer reducer; + reducer.run(scriptName, appName, appArgs, *source, searchText); +} diff --git a/CMakeLists.txt b/CMakeLists.txt index 92006344..3dafe5fe 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -32,11 +32,13 @@ if(LUAU_BUILD_CLI) add_executable(Luau.Repl.CLI) add_executable(Luau.Analyze.CLI) add_executable(Luau.Ast.CLI) + add_executable(Luau.Reduce.CLI) # This also adds target `name` on Linux/macOS and `name.exe` on Windows set_target_properties(Luau.Repl.CLI PROPERTIES OUTPUT_NAME luau) set_target_properties(Luau.Analyze.CLI PROPERTIES OUTPUT_NAME luau-analyze) set_target_properties(Luau.Ast.CLI PROPERTIES OUTPUT_NAME luau-ast) + set_target_properties(Luau.Reduce.CLI PROPERTIES OUTPUT_NAME luau-reduce) endif() if(LUAU_BUILD_TESTS) @@ -49,6 +51,7 @@ if(LUAU_BUILD_WEB) add_executable(Luau.Web) endif() + include(Sources.cmake) target_include_directories(Luau.Common INTERFACE Common/include) @@ -171,6 +174,10 @@ if(LUAU_BUILD_CLI) target_link_libraries(Luau.Analyze.CLI PRIVATE Luau.Analysis) target_link_libraries(Luau.Ast.CLI PRIVATE Luau.Ast Luau.Analysis) + + target_compile_features(Luau.Reduce.CLI PRIVATE cxx_std_17) + target_include_directories(Luau.Reduce.CLI PUBLIC Reduce/include) + target_link_libraries(Luau.Reduce.CLI PRIVATE Luau.Common Luau.Ast Luau.Analysis) endif() if(LUAU_BUILD_TESTS) diff --git a/CodeGen/include/Luau/AssemblyBuilderX64.h b/CodeGen/include/Luau/AssemblyBuilderX64.h index 028b2d16..cb799d31 100644 --- a/CodeGen/include/Luau/AssemblyBuilderX64.h +++ b/CodeGen/include/Luau/AssemblyBuilderX64.h @@ -38,14 +38,21 @@ public: // Two operand mov instruction has additional specialized encodings void mov(OperandX64 lhs, OperandX64 rhs); void mov64(RegisterX64 lhs, int64_t imm); + void movsx(RegisterX64 lhs, OperandX64 rhs); + void movzx(RegisterX64 lhs, OperandX64 rhs); // Base one operand instruction with 2 opcode selection void div(OperandX64 op); void idiv(OperandX64 op); void mul(OperandX64 op); + void imul(OperandX64 op); void neg(OperandX64 op); void not_(OperandX64 op); + // Additional forms of imul + void imul(OperandX64 lhs, OperandX64 rhs); + void imul(OperandX64 dst, OperandX64 lhs, int32_t rhs); + void test(OperandX64 lhs, OperandX64 rhs); void lea(OperandX64 lhs, OperandX64 rhs); @@ -76,6 +83,12 @@ public: void vxorpd(OperandX64 dst, OperandX64 src1, OperandX64 src2); void vcomisd(OperandX64 src1, OperandX64 src2); + void vucomisd(OperandX64 src1, OperandX64 src2); + + void vcvttsd2si(OperandX64 dst, OperandX64 src); + void vcvtsi2sd(OperandX64 dst, OperandX64 src1, OperandX64 src2); + + void vroundsd(OperandX64 dst, OperandX64 src1, OperandX64 src2, uint8_t mode); void vsqrtpd(OperandX64 dst, OperandX64 src); void vsqrtps(OperandX64 dst, OperandX64 src); @@ -105,6 +118,7 @@ public: OperandX64 f32(float value); OperandX64 f64(double value); OperandX64 f32x4(float x, float y, float z, float w); + OperandX64 bytes(const void* ptr, size_t size, size_t align = 8); // Resulting data and code that need to be copied over one after the other // The *end* of 'data' has to be aligned to 16 bytes, this will also align 'code' @@ -130,6 +144,8 @@ private: void placeAvx(const char* name, OperandX64 dst, OperandX64 src, uint8_t code, bool setW, uint8_t mode, uint8_t prefix); void placeAvx(const char* name, OperandX64 dst, OperandX64 src, uint8_t code, uint8_t coderev, bool setW, uint8_t mode, uint8_t prefix); void placeAvx(const char* name, OperandX64 dst, OperandX64 src1, OperandX64 src2, uint8_t code, bool setW, uint8_t mode, uint8_t prefix); + void placeAvx( + const char* name, OperandX64 dst, OperandX64 src1, OperandX64 src2, uint8_t imm8, uint8_t code, bool setW, uint8_t mode, uint8_t prefix); // Instruction components void placeRegAndModRegMem(OperandX64 lhs, OperandX64 rhs); @@ -157,6 +173,7 @@ private: LUAU_NOINLINE void log(const char* opcode, OperandX64 op); LUAU_NOINLINE void log(const char* opcode, OperandX64 op1, OperandX64 op2); LUAU_NOINLINE void log(const char* opcode, OperandX64 op1, OperandX64 op2, OperandX64 op3); + LUAU_NOINLINE void log(const char* opcode, OperandX64 op1, OperandX64 op2, OperandX64 op3, OperandX64 op4); LUAU_NOINLINE void log(Label label); LUAU_NOINLINE void log(const char* opcode, Label label); void log(OperandX64 op); diff --git a/CodeGen/src/AssemblyBuilderX64.cpp b/CodeGen/src/AssemblyBuilderX64.cpp index f88063cf..0fd10328 100644 --- a/CodeGen/src/AssemblyBuilderX64.cpp +++ b/CodeGen/src/AssemblyBuilderX64.cpp @@ -46,6 +46,44 @@ const unsigned AVX_F2 = 0b11; const unsigned kMaxAlign = 16; +// Utility functions to correctly write data on big endian machines +#if defined(__BYTE_ORDER__) && __BYTE_ORDER__ == __ORDER_BIG_ENDIAN__ +#include + +static void writeu32(uint8_t* target, uint32_t value) +{ + value = htole32(value); + memcpy(target, &value, sizeof(value)); +} + +static void writeu64(uint8_t* target, uint64_t value) +{ + value = htole64(value); + memcpy(target, &value, sizeof(value)); +} + +static void writef32(uint8_t* target, float value) +{ + static_assert(sizeof(float) == sizeof(uint32_t), "type size must match to reinterpret data"); + uint32_t data; + memcpy(&data, &value, sizeof(value)); + writeu32(target, data); +} + +static void writef64(uint8_t* target, double value) +{ + static_assert(sizeof(double) == sizeof(uint64_t), "type size must match to reinterpret data"); + uint64_t data; + memcpy(&data, &value, sizeof(value)); + writeu64(target, data); +} +#else +#define writeu32(target, value) memcpy(target, &value, sizeof(value)) +#define writeu64(target, value) memcpy(target, &value, sizeof(value)) +#define writef32(target, value) memcpy(target, &value, sizeof(value)) +#define writef64(target, value) memcpy(target, &value, sizeof(value)) +#endif + AssemblyBuilderX64::AssemblyBuilderX64(bool logText) : logText(logText) { @@ -195,6 +233,34 @@ void AssemblyBuilderX64::mov64(RegisterX64 lhs, int64_t imm) commit(); } +void AssemblyBuilderX64::movsx(RegisterX64 lhs, OperandX64 rhs) +{ + if (logText) + log("movsx", lhs, rhs); + + LUAU_ASSERT(rhs.memSize == SizeX64::byte || rhs.memSize == SizeX64::word); + + placeRex(lhs, rhs); + place(0x0f); + place(rhs.memSize == SizeX64::byte ? 0xbe : 0xbf); + placeRegAndModRegMem(lhs, rhs); + commit(); +} + +void AssemblyBuilderX64::movzx(RegisterX64 lhs, OperandX64 rhs) +{ + if (logText) + log("movzx", lhs, rhs); + + LUAU_ASSERT(rhs.memSize == SizeX64::byte || rhs.memSize == SizeX64::word); + + placeRex(lhs, rhs); + place(0x0f); + place(rhs.memSize == SizeX64::byte ? 0xb6 : 0xb7); + placeRegAndModRegMem(lhs, rhs); + commit(); +} + void AssemblyBuilderX64::div(OperandX64 op) { placeUnaryModRegMem("div", op, 0xf6, 0xf7, 6); @@ -210,6 +276,11 @@ void AssemblyBuilderX64::mul(OperandX64 op) placeUnaryModRegMem("mul", op, 0xf6, 0xf7, 4); } +void AssemblyBuilderX64::imul(OperandX64 op) +{ + placeUnaryModRegMem("imul", op, 0xf6, 0xf7, 5); +} + void AssemblyBuilderX64::neg(OperandX64 op) { placeUnaryModRegMem("neg", op, 0xf6, 0xf7, 3); @@ -220,6 +291,41 @@ void AssemblyBuilderX64::not_(OperandX64 op) placeUnaryModRegMem("not", op, 0xf6, 0xf7, 2); } +void AssemblyBuilderX64::imul(OperandX64 lhs, OperandX64 rhs) +{ + if (logText) + log("imul", lhs, rhs); + + placeRex(lhs.base, rhs); + place(0x0f); + place(0xaf); + placeRegAndModRegMem(lhs, rhs); + commit(); +} + +void AssemblyBuilderX64::imul(OperandX64 dst, OperandX64 lhs, int32_t rhs) +{ + if (logText) + log("imul", dst, lhs, rhs); + + placeRex(dst.base, lhs); + + if (int8_t(rhs) == rhs) + { + place(0x6b); + placeRegAndModRegMem(dst, lhs); + placeImm8(rhs); + } + else + { + place(0x69); + placeRegAndModRegMem(dst, lhs); + placeImm32(rhs); + } + + commit(); +} + void AssemblyBuilderX64::test(OperandX64 lhs, OperandX64 rhs) { // No forms for r/m*, imm8 and reg, r/m* @@ -368,6 +474,26 @@ void AssemblyBuilderX64::vcomisd(OperandX64 src1, OperandX64 src2) placeAvx("vcomisd", src1, src2, 0x2f, false, AVX_0F, AVX_66); } +void AssemblyBuilderX64::vucomisd(OperandX64 src1, OperandX64 src2) +{ + placeAvx("vucomisd", src1, src2, 0x2e, false, AVX_0F, AVX_66); +} + +void AssemblyBuilderX64::vcvttsd2si(OperandX64 dst, OperandX64 src) +{ + placeAvx("vcvttsd2si", dst, src, 0x2c, dst.base.size == SizeX64::dword, AVX_0F, AVX_F2); +} + +void AssemblyBuilderX64::vcvtsi2sd(OperandX64 dst, OperandX64 src1, OperandX64 src2) +{ + placeAvx("vcvtsi2sd", dst, src1, src2, 0x2a, (src2.cat == CategoryX64::reg ? src2.base.size : src2.memSize) == SizeX64::dword, AVX_0F, AVX_F2); +} + +void AssemblyBuilderX64::vroundsd(OperandX64 dst, OperandX64 src1, OperandX64 src2, uint8_t mode) +{ + placeAvx("vroundsd", dst, src1, src2, mode, 0x0b, false, AVX_0F3A, AVX_66); +} + void AssemblyBuilderX64::vsqrtpd(OperandX64 dst, OperandX64 src) { placeAvx("vsqrtpd", dst, src, 0x51, false, AVX_0F, AVX_66); @@ -436,7 +562,7 @@ void AssemblyBuilderX64::finalize() for (Label fixup : pendingLabels) { uint32_t value = labelLocations[fixup.id - 1] - (fixup.location + 4); - memcpy(&code[fixup.location], &value, sizeof(value)); + writeu32(&code[fixup.location], value); } size_t dataSize = data.size() - dataPos; @@ -479,34 +605,41 @@ void AssemblyBuilderX64::setLabel(Label& label) OperandX64 AssemblyBuilderX64::i64(int64_t value) { size_t pos = allocateData(8, 8); - memcpy(&data[pos], &value, sizeof(value)); + writeu64(&data[pos], value); return OperandX64(SizeX64::qword, noreg, 1, rip, int32_t(pos - data.size())); } OperandX64 AssemblyBuilderX64::f32(float value) { size_t pos = allocateData(4, 4); - memcpy(&data[pos], &value, sizeof(value)); + writef32(&data[pos], value); return OperandX64(SizeX64::dword, noreg, 1, rip, int32_t(pos - data.size())); } OperandX64 AssemblyBuilderX64::f64(double value) { size_t pos = allocateData(8, 8); - memcpy(&data[pos], &value, sizeof(value)); + writef64(&data[pos], value); return OperandX64(SizeX64::qword, noreg, 1, rip, int32_t(pos - data.size())); } OperandX64 AssemblyBuilderX64::f32x4(float x, float y, float z, float w) { size_t pos = allocateData(16, 16); - memcpy(&data[pos], &x, sizeof(x)); - memcpy(&data[pos + 4], &y, sizeof(y)); - memcpy(&data[pos + 8], &z, sizeof(z)); - memcpy(&data[pos + 12], &w, sizeof(w)); + writef32(&data[pos], x); + writef32(&data[pos + 4], y); + writef32(&data[pos + 8], z); + writef32(&data[pos + 12], w); return OperandX64(SizeX64::xmmword, noreg, 1, rip, int32_t(pos - data.size())); } +OperandX64 AssemblyBuilderX64::bytes(const void* ptr, size_t size, size_t align) +{ + size_t pos = allocateData(size, align); + memcpy(&data[pos], ptr, size); + return OperandX64(SizeX64::qword, noreg, 1, rip, int32_t(pos - data.size())); +} + void AssemblyBuilderX64::placeBinary(const char* name, OperandX64 lhs, OperandX64 rhs, uint8_t codeimm8, uint8_t codeimm, uint8_t codeimmImm8, uint8_t code8rev, uint8_t coderev, uint8_t code8, uint8_t code, uint8_t opreg) { @@ -700,6 +833,24 @@ void AssemblyBuilderX64::placeAvx( commit(); } +void AssemblyBuilderX64::placeAvx( + const char* name, OperandX64 dst, OperandX64 src1, OperandX64 src2, uint8_t imm8, uint8_t code, bool setW, uint8_t mode, uint8_t prefix) +{ + LUAU_ASSERT(dst.cat == CategoryX64::reg); + LUAU_ASSERT(src1.cat == CategoryX64::reg); + LUAU_ASSERT(src2.cat == CategoryX64::reg || src2.cat == CategoryX64::mem); + + if (logText) + log(name, dst, src1, src2, imm8); + + placeVex(dst, src1, src2, setW, mode, prefix); + place(code); + placeRegAndModRegMem(dst, src2); + placeImm8(imm8); + + commit(); +} + void AssemblyBuilderX64::placeRex(RegisterX64 op) { uint8_t code = REX_W(op.size == SizeX64::qword) | REX_B(op); @@ -861,16 +1012,18 @@ void AssemblyBuilderX64::placeImm8(int32_t imm) void AssemblyBuilderX64::placeImm32(int32_t imm) { - LUAU_ASSERT(codePos + sizeof(imm) < codeEnd); - memcpy(codePos, &imm, sizeof(imm)); - codePos += sizeof(imm); + uint8_t* pos = codePos; + LUAU_ASSERT(pos + sizeof(imm) < codeEnd); + writeu32(pos, imm); + codePos = pos + sizeof(imm); } void AssemblyBuilderX64::placeImm64(int64_t imm) { - LUAU_ASSERT(codePos + sizeof(imm) < codeEnd); - memcpy(codePos, &imm, sizeof(imm)); - codePos += sizeof(imm); + uint8_t* pos = codePos; + LUAU_ASSERT(pos + sizeof(imm) < codeEnd); + writeu64(pos, imm); + codePos = pos + sizeof(imm); } void AssemblyBuilderX64::placeLabel(Label& label) @@ -970,6 +1123,19 @@ void AssemblyBuilderX64::log(const char* opcode, OperandX64 op1, OperandX64 op2, text.append("\n"); } +void AssemblyBuilderX64::log(const char* opcode, OperandX64 op1, OperandX64 op2, OperandX64 op3, OperandX64 op4) +{ + logAppend(" %-12s", opcode); + log(op1); + text.append(","); + log(op2); + text.append(","); + log(op3); + text.append(","); + log(op4); + text.append("\n"); +} + void AssemblyBuilderX64::log(Label label) { logAppend(".L%d:\n", label.id); diff --git a/Compiler/src/Compiler.cpp b/Compiler/src/Compiler.cpp index 2ee20cab..eb815225 100644 --- a/Compiler/src/Compiler.cpp +++ b/Compiler/src/Compiler.cpp @@ -28,6 +28,8 @@ LUAU_FASTFLAGVARIABLE(LuauCompileNoIpairs, false) LUAU_FASTFLAGVARIABLE(LuauCompileFreeReassign, false) LUAU_FASTFLAGVARIABLE(LuauCompileXEQ, false) +LUAU_FASTFLAGVARIABLE(LuauCompileOptimalAssignment, false) + namespace Luau { @@ -37,6 +39,8 @@ static const uint32_t kMaxRegisterCount = 255; static const uint32_t kMaxUpvalueCount = 200; static const uint32_t kMaxLocalCount = 200; +static const uint8_t kInvalidReg = 255; + CompileError::CompileError(const Location& location, const std::string& message) : location(location) , message(message) @@ -2030,9 +2034,35 @@ struct Compiler return reg; } + // initializes target..target+targetCount-1 range using expression + // if expression is a call/vararg, we assume it returns all values, otherwise we fill the rest with nil + // assumes target register range can be clobbered and is at the top of the register space if targetTop = true + void compileExprTempN(AstExpr* node, uint8_t target, uint8_t targetCount, bool targetTop) + { + // we assume that target range is at the top of the register space and can be clobbered + // this is what allows us to compile the last call expression - if it's a call - using targetTop=true + LUAU_ASSERT(!targetTop || unsigned(target + targetCount) == regTop); + + if (AstExprCall* expr = node->as()) + { + compileExprCall(expr, target, targetCount, targetTop); + } + else if (AstExprVarargs* expr = node->as()) + { + compileExprVarargs(expr, target, targetCount); + } + else + { + compileExprTemp(node, target); + + for (size_t i = 1; i < targetCount; ++i) + bytecode.emitABC(LOP_LOADNIL, uint8_t(target + i), 0, 0); + } + } + // initializes target..target+targetCount-1 range using expressions from the list - // if list has fewer expressions, and last expression is a call, we assume the call returns the rest of the values - // if list has fewer expressions, and last expression isn't a call, we fill the rest with nil + // if list has fewer expressions, and last expression is multret, we assume it returns the rest of the values + // if list has fewer expressions, and last expression isn't multret, we fill the rest with nil // assumes target register range can be clobbered and is at the top of the register space if targetTop = true void compileExprListTemp(const AstArray& list, uint8_t target, uint8_t targetCount, bool targetTop) { @@ -2062,23 +2092,7 @@ struct Compiler for (size_t i = 0; i < list.size - 1; ++i) compileExprTemp(list.data[i], uint8_t(target + i)); - AstExpr* last = list.data[list.size - 1]; - - if (AstExprCall* expr = last->as()) - { - compileExprCall(expr, uint8_t(target + list.size - 1), uint8_t(targetCount - (list.size - 1)), targetTop); - } - else if (AstExprVarargs* expr = last->as()) - { - compileExprVarargs(expr, uint8_t(target + list.size - 1), uint8_t(targetCount - (list.size - 1))); - } - else - { - compileExprTemp(last, uint8_t(target + list.size - 1)); - - for (size_t i = list.size; i < targetCount; ++i) - bytecode.emitABC(LOP_LOADNIL, uint8_t(target + i), 0, 0); - } + compileExprTempN(list.data[list.size - 1], uint8_t(target + list.size - 1), uint8_t(targetCount - (list.size - 1)), targetTop); } else { @@ -2859,6 +2873,8 @@ struct Compiler void resolveAssignConflicts(AstStat* stat, std::vector& vars) { + LUAU_ASSERT(!FFlag::LuauCompileOptimalAssignment); + // regsUsed[i] is true if we have assigned the register during earlier assignments // regsRemap[i] is set to the register where the original (pre-assignment) copy was made // note: regsRemap is uninitialized intentionally to speed small assignments up; regsRemap[i] is valid iff regsUsed[i] @@ -2911,12 +2927,86 @@ struct Compiler } } + struct Assignment + { + LValue lvalue; + + uint8_t conflictReg = kInvalidReg; + uint8_t valueReg = kInvalidReg; + }; + + void resolveAssignConflicts(AstStat* stat, std::vector& vars, const AstArray& values) + { + struct Visitor : AstVisitor + { + Compiler* self; + + std::bitset<256> conflict; + std::bitset<256> assigned; + + Visitor(Compiler* self) + : self(self) + { + } + + bool visit(AstExprLocal* node) override + { + int reg = self->getLocalReg(node->local); + + if (reg >= 0 && assigned[reg]) + conflict[reg] = true; + + return true; + } + }; + + Visitor visitor(this); + + // mark any registers that are used *after* assignment as conflicting + for (size_t i = 0; i < vars.size(); ++i) + { + const LValue& li = vars[i].lvalue; + + if (i < values.size) + values.data[i]->visit(&visitor); + + if (li.kind == LValue::Kind_Local) + visitor.assigned[li.reg] = true; + } + + // mark any registers used in trailing expressions as conflicting as well + for (size_t i = vars.size(); i < values.size; ++i) + values.data[i]->visit(&visitor); + + // mark any registers used on left hand side that are also assigned anywhere as conflicting + // this is order-independent because we evaluate all right hand side arguments into registers before doing table assignments + for (const Assignment& var : vars) + { + const LValue& li = var.lvalue; + + if ((li.kind == LValue::Kind_IndexName || li.kind == LValue::Kind_IndexNumber || li.kind == LValue::Kind_IndexExpr) && + visitor.assigned[li.reg]) + visitor.conflict[li.reg] = true; + + if (li.kind == LValue::Kind_IndexExpr && visitor.assigned[li.index]) + visitor.conflict[li.index] = true; + } + + // for any conflicting var, we need to allocate a temporary register where the assignment is performed, so that we can move the value later + for (Assignment& var : vars) + { + const LValue& li = var.lvalue; + + if (li.kind == LValue::Kind_Local && visitor.conflict[li.reg]) + var.conflictReg = allocReg(stat, 1); + } + } + void compileStatAssign(AstStatAssign* stat) { RegScope rs(this); - // Optimization: one to one assignments don't require complex conflict resolution machinery and allow us to skip temporary registers for - // locals + // Optimization: one to one assignments don't require complex conflict resolution machinery if (stat->vars.size == 1 && stat->values.size == 1) { LValue var = compileLValue(stat->vars.data[0], rs); @@ -2936,28 +3026,110 @@ struct Compiler return; } - // compute all l-values: note that this doesn't assign anything yet but it allocates registers and computes complex expressions on the left - // hand side for example, in "a[expr] = foo" expr will get evaluated here - std::vector vars(stat->vars.size); - - for (size_t i = 0; i < stat->vars.size; ++i) - vars[i] = compileLValue(stat->vars.data[i], rs); - - // perform conflict resolution: if any lvalue refers to a local reg that will be reassigned before that, we save the local variable in a - // temporary reg - resolveAssignConflicts(stat, vars); - - // compute values into temporaries - uint8_t regs = allocReg(stat, unsigned(stat->vars.size)); - - compileExprListTemp(stat->values, regs, uint8_t(stat->vars.size), /* targetTop= */ true); - - // assign variables that have associated values; note that if we have fewer values than variables, we'll assign nil because - // compileExprListTemp will generate nils - for (size_t i = 0; i < stat->vars.size; ++i) + if (FFlag::LuauCompileOptimalAssignment) { - setDebugLine(stat->vars.data[i]); - compileAssign(vars[i], uint8_t(regs + i)); + // compute all l-values: note that this doesn't assign anything yet but it allocates registers and computes complex expressions on the + // left hand side - for example, in "a[expr] = foo" expr will get evaluated here + std::vector vars(stat->vars.size); + + for (size_t i = 0; i < stat->vars.size; ++i) + vars[i].lvalue = compileLValue(stat->vars.data[i], rs); + + // perform conflict resolution: if any expression refers to a local that is assigned before evaluating it, we assign to a temporary + // register after this, vars[i].conflictReg is set for locals that need to be assigned in the second pass + resolveAssignConflicts(stat, vars, stat->values); + + // compute rhs into (mostly) fresh registers + // note that when the lhs assigment is a local, we evaluate directly into that register + // this is possible because resolveAssignConflicts renamed conflicting locals into temporaries + // after this, vars[i].valueReg is set to a register with the value for *all* vars, but some have already been assigned + for (size_t i = 0; i < stat->vars.size && i < stat->values.size; ++i) + { + AstExpr* value = stat->values.data[i]; + + if (i + 1 == stat->values.size && stat->vars.size > stat->values.size) + { + // allocate a consecutive range of regs for all remaining vars and compute everything into temps + // note, this also handles trailing nils + uint8_t rest = uint8_t(stat->vars.size - stat->values.size + 1); + uint8_t temp = allocReg(stat, rest); + + compileExprTempN(value, temp, rest, /* targetTop= */ true); + + for (size_t j = i; j < stat->vars.size; ++j) + vars[j].valueReg = uint8_t(temp + (j - i)); + } + else + { + Assignment& var = vars[i]; + + // if target is a local, use compileExpr directly to target + if (var.lvalue.kind == LValue::Kind_Local) + { + var.valueReg = (var.conflictReg == kInvalidReg) ? var.lvalue.reg : var.conflictReg; + + compileExpr(stat->values.data[i], var.valueReg); + } + else + { + var.valueReg = compileExprAuto(stat->values.data[i], rs); + } + } + } + + // compute expressions with side effects for lulz + for (size_t i = stat->vars.size; i < stat->values.size; ++i) + { + RegScope rsi(this); + compileExprAuto(stat->values.data[i], rsi); + } + + // almost done... let's assign everything left to right, noting that locals were either written-to directly, or will be written-to in a + // separate pass to avoid conflicts + for (const Assignment& var : vars) + { + LUAU_ASSERT(var.valueReg != kInvalidReg); + + if (var.lvalue.kind != LValue::Kind_Local) + { + setDebugLine(var.lvalue.location); + compileAssign(var.lvalue, var.valueReg); + } + } + + // all regular local writes are done by the prior loops by computing result directly into target, so this just handles conflicts OR + // local copies from temporary registers in multret context, since in that case we have to allocate consecutive temporaries + for (const Assignment& var : vars) + { + if (var.lvalue.kind == LValue::Kind_Local && var.valueReg != var.lvalue.reg) + bytecode.emitABC(LOP_MOVE, var.lvalue.reg, var.valueReg, 0); + } + } + else + { + // compute all l-values: note that this doesn't assign anything yet but it allocates registers and computes complex expressions on the + // left hand side for example, in "a[expr] = foo" expr will get evaluated here + std::vector vars(stat->vars.size); + + for (size_t i = 0; i < stat->vars.size; ++i) + vars[i] = compileLValue(stat->vars.data[i], rs); + + // perform conflict resolution: if any lvalue refers to a local reg that will be reassigned before that, we save the local variable in a + // temporary reg + resolveAssignConflicts(stat, vars); + + // compute values into temporaries + uint8_t regs = allocReg(stat, unsigned(stat->vars.size)); + + compileExprListTemp(stat->values, regs, uint8_t(stat->vars.size), /* targetTop= */ true); + + // assign variables that have associated values; note that if we have fewer values than variables, we'll assign nil because + // compileExprListTemp will generate nils + for (size_t i = 0; i < stat->vars.size; ++i) + { + setDebugLine(stat->vars.data[i]); + compileAssign(vars[i], uint8_t(regs + i)); + } } } diff --git a/Makefile b/Makefile index dd32d237..4333a973 100644 --- a/Makefile +++ b/Makefile @@ -97,8 +97,8 @@ ifeq ($(config),fuzz) LDFLAGS+=-fsanitize=address,fuzzer endif -ifneq ($(CALLGRIND),) - CXXFLAGS+=-DCALLGRIND=$(CALLGRIND) +ifeq ($(config),profile) + CXXFLAGS+=-O2 -DNDEBUG -gdwarf-4 -DCALLGRIND=1 endif # target-specific flags diff --git a/Sources.cmake b/Sources.cmake index 9a6019a9..7a27684e 100644 --- a/Sources.cmake +++ b/Sources.cmake @@ -347,3 +347,11 @@ if(TARGET Luau.Web) target_sources(Luau.Web PRIVATE CLI/Web.cpp) endif() + +if(TARGET Luau.Reduce.CLI) + target_sources(Luau.Reduce.CLI PRIVATE + CLI/Reduce.cpp + CLI/FileUtils.cpp + CLI/FileUtils.h + ) +endif() diff --git a/bench/bench.py b/bench/bench.py index bb3ea5f7..42a0ac97 100644 --- a/bench/bench.py +++ b/bench/bench.py @@ -12,9 +12,6 @@ import json from color import colored, Color from tabulate import TablePrinter, Alignment -# Based on rotest, specialized for benchmark results -import influxbench - try: import matplotlib import matplotlib.pyplot as plt @@ -721,6 +718,7 @@ def run(args, argsubcb): argumentSubstituionCallback = argsubcb if arguments.report_metrics or arguments.print_influx_debugging: + import influxbench influxReporter = influxbench.InfluxReporter(arguments) else: influxReporter = None diff --git a/bench/influxbench.py b/bench/influxbench.py index adcddeba..20122456 100644 --- a/bench/influxbench.py +++ b/bench/influxbench.py @@ -4,12 +4,7 @@ import platform import shlex import socket import sys - -try: - import requests -except: - print("Please install 'requests' using using '{} -m pip install requests' command and try again".format(sys.executable)) - exit(-1) +import requests _hostname = socket.gethostname() diff --git a/tests/AssemblyBuilderX64.test.cpp b/tests/AssemblyBuilderX64.test.cpp index 28ce6a82..3f75ebac 100644 --- a/tests/AssemblyBuilderX64.test.cpp +++ b/tests/AssemblyBuilderX64.test.cpp @@ -22,7 +22,7 @@ std::string bytecodeAsArray(const std::vector& bytecode) class AssemblyBuilderX64Fixture { public: - void check(std::function f, std::vector result) + void check(std::function f, std::vector code, std::vector data = {}) { AssemblyBuilderX64 build(/* logText= */ false); @@ -30,9 +30,15 @@ public: build.finalize(); - if (build.code != result) + if (build.code != code) { - printf("Expected: %s\nReceived: %s\n", bytecodeAsArray(result).c_str(), bytecodeAsArray(build.code).c_str()); + printf("Expected code: %s\nReceived code: %s\n", bytecodeAsArray(code).c_str(), bytecodeAsArray(build.code).c_str()); + CHECK(false); + } + + if (build.data != data) + { + printf("Expected data: %s\nReceived data: %s\n", bytecodeAsArray(data).c_str(), bytecodeAsArray(build.data).c_str()); CHECK(false); } } @@ -169,6 +175,7 @@ TEST_CASE_FIXTURE(AssemblyBuilderX64Fixture, "BaseUnaryInstructionForms") SINGLE_COMPARE(div(rcx), 0x48, 0xf7, 0xf1); SINGLE_COMPARE(idiv(qword[rax]), 0x48, 0xf7, 0x38); SINGLE_COMPARE(mul(qword[rax + rbx]), 0x48, 0xf7, 0x24, 0x18); + SINGLE_COMPARE(imul(r9), 0x49, 0xf7, 0xe9); SINGLE_COMPARE(neg(r9), 0x49, 0xf7, 0xd9); SINGLE_COMPARE(not_(r12), 0x49, 0xf7, 0xd4); } @@ -191,6 +198,18 @@ TEST_CASE_FIXTURE(AssemblyBuilderX64Fixture, "FormsOfMov") SINGLE_COMPARE(mov(byte[rsi], al), 0x88, 0x06); } +TEST_CASE_FIXTURE(AssemblyBuilderX64Fixture, "FormsOfMovExtended") +{ + SINGLE_COMPARE(movsx(eax, byte[rcx]), 0x0f, 0xbe, 0x01); + SINGLE_COMPARE(movsx(r12, byte[r10]), 0x4d, 0x0f, 0xbe, 0x22); + SINGLE_COMPARE(movsx(ebx, word[r11]), 0x41, 0x0f, 0xbf, 0x1b); + SINGLE_COMPARE(movsx(rdx, word[rcx]), 0x48, 0x0f, 0xbf, 0x11); + SINGLE_COMPARE(movzx(eax, byte[rcx]), 0x0f, 0xb6, 0x01); + SINGLE_COMPARE(movzx(r12, byte[r10]), 0x4d, 0x0f, 0xb6, 0x22); + SINGLE_COMPARE(movzx(ebx, word[r11]), 0x41, 0x0f, 0xb7, 0x1b); + SINGLE_COMPARE(movzx(rdx, word[rcx]), 0x48, 0x0f, 0xb7, 0x11); +} + TEST_CASE_FIXTURE(AssemblyBuilderX64Fixture, "FormsOfTest") { SINGLE_COMPARE(test(al, 8), 0xf6, 0xc0, 0x08); @@ -230,6 +249,19 @@ TEST_CASE_FIXTURE(AssemblyBuilderX64Fixture, "FormsOfAbsoluteJumps") SINGLE_COMPARE(call(qword[r14 + rdx * 4]), 0x49, 0xff, 0x14, 0x96); } +TEST_CASE_FIXTURE(AssemblyBuilderX64Fixture, "FormsOfImul") +{ + SINGLE_COMPARE(imul(ecx, esi), 0x0f, 0xaf, 0xce); + SINGLE_COMPARE(imul(r12, rax), 0x4c, 0x0f, 0xaf, 0xe0); + SINGLE_COMPARE(imul(r12, qword[rdx + rdi]), 0x4c, 0x0f, 0xaf, 0x24, 0x3a); + SINGLE_COMPARE(imul(ecx, edx, 8), 0x6b, 0xca, 0x08); + SINGLE_COMPARE(imul(ecx, r9d, 0xabcd), 0x41, 0x69, 0xc9, 0xcd, 0xab, 0x00, 0x00); + SINGLE_COMPARE(imul(r8d, eax, -9), 0x44, 0x6b, 0xc0, 0xf7); + SINGLE_COMPARE(imul(rcx, rdx, 17), 0x48, 0x6b, 0xca, 0x11); + SINGLE_COMPARE(imul(rcx, r12, 0xabcd), 0x49, 0x69, 0xcc, 0xcd, 0xab, 0x00, 0x00); + SINGLE_COMPARE(imul(r12, rax, -13), 0x4c, 0x6b, 0xe0, 0xf3); +} + TEST_CASE_FIXTURE(AssemblyBuilderX64Fixture, "ControlFlow") { // Jump back @@ -335,6 +367,7 @@ TEST_CASE_FIXTURE(AssemblyBuilderX64Fixture, "AVXUnaryMergeInstructionForms") // Coverage for other instructions that follow the same pattern SINGLE_COMPARE(vcomisd(xmm8, xmm10), 0xc4, 0x41, 0xf9, 0x2f, 0xc2); + SINGLE_COMPARE(vucomisd(xmm1, xmm4), 0xc4, 0xe1, 0xf9, 0x2e, 0xcc); } TEST_CASE_FIXTURE(AssemblyBuilderX64Fixture, "AVXMoveInstructionForms") @@ -359,6 +392,25 @@ TEST_CASE_FIXTURE(AssemblyBuilderX64Fixture, "AVXMoveInstructionForms") SINGLE_COMPARE(vmovups(ymm8, ymmword[r9]), 0xc4, 0x41, 0xfc, 0x10, 0x01); } +TEST_CASE_FIXTURE(AssemblyBuilderX64Fixture, "AVXConversionInstructionForms") +{ + SINGLE_COMPARE(vcvttsd2si(ecx, xmm0), 0xc4, 0xe1, 0x7b, 0x2c, 0xc8); + SINGLE_COMPARE(vcvttsd2si(r9d, xmmword[rcx + rdx]), 0xc4, 0x61, 0x7b, 0x2c, 0x0c, 0x11); + SINGLE_COMPARE(vcvttsd2si(rdx, xmm0), 0xc4, 0xe1, 0xfb, 0x2c, 0xd0); + SINGLE_COMPARE(vcvttsd2si(r13, xmmword[rcx + rdx]), 0xc4, 0x61, 0xfb, 0x2c, 0x2c, 0x11); + SINGLE_COMPARE(vcvtsi2sd(xmm5, xmm10, ecx), 0xc4, 0xe1, 0x2b, 0x2a, 0xe9); + SINGLE_COMPARE(vcvtsi2sd(xmm6, xmm11, dword[rcx + rdx]), 0xc4, 0xe1, 0x23, 0x2a, 0x34, 0x11); + SINGLE_COMPARE(vcvtsi2sd(xmm5, xmm10, r13), 0xc4, 0xc1, 0xab, 0x2a, 0xed); + SINGLE_COMPARE(vcvtsi2sd(xmm6, xmm11, qword[rcx + rdx]), 0xc4, 0xe1, 0xa3, 0x2a, 0x34, 0x11); +} + +TEST_CASE_FIXTURE(AssemblyBuilderX64Fixture, "AVXTernaryInstructionForms") +{ + SINGLE_COMPARE(vroundsd(xmm7, xmm12, xmm3, 9), 0xc4, 0xe3, 0x99, 0x0b, 0xfb, 0x09); + SINGLE_COMPARE(vroundsd(xmm8, xmm13, xmmword[r13 + rdx], 9), 0xc4, 0x43, 0x91, 0x0b, 0x44, 0x15, 0x00, 0x09); + SINGLE_COMPARE(vroundsd(xmm9, xmm14, xmmword[rcx + r10], 1), 0xc4, 0x23, 0x89, 0x0b, 0x0c, 0x11, 0x01); +} + TEST_CASE_FIXTURE(AssemblyBuilderX64Fixture, "MiscInstructions") { SINGLE_COMPARE(int3(), 0xcc); @@ -386,6 +438,11 @@ TEST_CASE("LogTest") build.neg(qword[rbp + r12 * 2]); build.mov64(r10, 0x1234567812345678ll); build.vmovapd(xmmword[rax], xmm11); + build.movzx(eax, byte[rcx]); + build.movsx(rsi, word[r12]); + build.imul(rcx, rdx); + build.imul(rcx, rdx, 8); + build.vroundsd(xmm1, xmm2, xmm3, 5); build.pop(r12); build.ret(); build.int3(); @@ -409,6 +466,11 @@ TEST_CASE("LogTest") neg qword ptr [rbp+r12*2] mov r10,1234567812345678h vmovapd xmmword ptr [rax],xmm11 + movzx eax,byte ptr [rcx] + movsx rsi,word ptr [r12] + imul rcx,rdx + imul rcx,rdx,8 + vroundsd xmm1,xmm2,xmm3,5 pop r12 ret int3 @@ -426,6 +488,8 @@ TEST_CASE_FIXTURE(AssemblyBuilderX64Fixture, "Constants") build.vmovss(xmm2, build.f32(1.0f)); build.vmovsd(xmm3, build.f64(1.0)); build.vmovaps(xmm4, build.f32x4(1.0f, 2.0f, 4.0f, 8.0f)); + char arr[16] = "hello world!123"; + build.vmovupd(xmm5, build.bytes(arr, 16, 8)); build.ret(); }, { @@ -434,7 +498,20 @@ TEST_CASE_FIXTURE(AssemblyBuilderX64Fixture, "Constants") 0xc4, 0xe1, 0xfa, 0x10, 0x15, 0xe1, 0xff, 0xff, 0xff, 0xc4, 0xe1, 0xfb, 0x10, 0x1d, 0xcc, 0xff, 0xff, 0xff, 0xc4, 0xe1, 0xf8, 0x28, 0x25, 0xab, 0xff, 0xff, 0xff, + 0xc4, 0xe1, 0xf9, 0x10, 0x2d, 0x92, 0xff, 0xff, 0xff, 0xc3 + }, + { + 'h', 'e', 'l', 'l', 'o', ' ', 'w', 'o', 'r', 'l', 'd', '!', '1', '2', '3', 0x0, + 0x00, 0x00, 0x80, 0x3f, + 0x00, 0x00, 0x00, 0x40, + 0x00, 0x00, 0x80, 0x40, + 0x00, 0x00, 0x00, 0x41, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, // padding to align f32x4 + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0xf0, 0x3f, + 0x00, 0x00, 0x00, 0x00, // padding to align f64 + 0x00, 0x00, 0x80, 0x3f, + 0x21, 0x43, 0x65, 0x87, 0x78, 0x56, 0x34, 0x12, }); // clang-format on } @@ -444,7 +521,7 @@ TEST_CASE("ConstantStorage") AssemblyBuilderX64 build(/* logText= */ false); for (int i = 0; i <= 3000; i++) - build.vaddss(xmm0, xmm0, build.f32(float(i))); + build.vaddss(xmm0, xmm0, build.f32(1.0f)); build.finalize(); @@ -452,9 +529,10 @@ TEST_CASE("ConstantStorage") for (int i = 0; i <= 3000; i++) { - float v; - memcpy(&v, &build.data[build.data.size() - (i + 1) * sizeof(float)], sizeof(v)); - LUAU_ASSERT(v == float(i)); + LUAU_ASSERT(build.data[i * 4 + 0] == 0x00); + LUAU_ASSERT(build.data[i * 4 + 1] == 0x00); + LUAU_ASSERT(build.data[i * 4 + 2] == 0x80); + LUAU_ASSERT(build.data[i * 4 + 3] == 0x3f); } } diff --git a/tests/Compiler.test.cpp b/tests/Compiler.test.cpp index 0a1c5a8b..a4ce88b1 100644 --- a/tests/Compiler.test.cpp +++ b/tests/Compiler.test.cpp @@ -2728,46 +2728,44 @@ RETURN R0 0 TEST_CASE("AssignmentConflict") { + ScopedFastFlag sff("LuauCompileOptimalAssignment", true); + // assignments are left to right CHECK_EQ("\n" + compileFunction0("local a, b a, b = 1, 2"), R"( LOADNIL R0 LOADNIL R1 -LOADN R2 1 -LOADN R3 2 -MOVE R0 R2 -MOVE R1 R3 +LOADN R0 1 +LOADN R1 2 RETURN R0 0 )"); - // if assignment of a local invalidates a direct register reference in later assignments, the value is evacuated to a temp register + // if assignment of a local invalidates a direct register reference in later assignments, the value is assigned to a temp register first CHECK_EQ("\n" + compileFunction0("local a a, a[1] = 1, 2"), R"( LOADNIL R0 -MOVE R1 R0 -LOADN R2 1 -LOADN R3 2 -MOVE R0 R2 -SETTABLEN R3 R1 1 +LOADN R1 1 +LOADN R2 2 +SETTABLEN R2 R0 1 +MOVE R0 R1 RETURN R0 0 )"); // note that this doesn't happen if the local assignment happens last naturally CHECK_EQ("\n" + compileFunction0("local a a[1], a = 1, 2"), R"( LOADNIL R0 -LOADN R1 1 -LOADN R2 2 -SETTABLEN R1 R0 1 -MOVE R0 R2 +LOADN R2 1 +LOADN R1 2 +SETTABLEN R2 R0 1 +MOVE R0 R1 RETURN R0 0 )"); // this will happen if assigned register is used in any table expression, including as an object... CHECK_EQ("\n" + compileFunction0("local a a, a.foo = 1, 2"), R"( LOADNIL R0 -MOVE R1 R0 -LOADN R2 1 -LOADN R3 2 -MOVE R0 R2 -SETTABLEKS R3 R1 K0 +LOADN R1 1 +LOADN R2 2 +SETTABLEKS R2 R0 K0 +MOVE R0 R1 RETURN R0 0 )"); @@ -2775,22 +2773,20 @@ RETURN R0 0 CHECK_EQ("\n" + compileFunction0("local a a, foo[a] = 1, 2"), R"( LOADNIL R0 GETIMPORT R1 1 -MOVE R2 R0 -LOADN R3 1 -LOADN R4 2 -MOVE R0 R3 -SETTABLE R4 R1 R2 +LOADN R2 1 +LOADN R3 2 +SETTABLE R3 R1 R0 +MOVE R0 R2 RETURN R0 0 )"); // ... or both ... CHECK_EQ("\n" + compileFunction0("local a a, a[a] = 1, 2"), R"( LOADNIL R0 -MOVE R1 R0 -LOADN R2 1 -LOADN R3 2 -MOVE R0 R2 -SETTABLE R3 R1 R1 +LOADN R1 1 +LOADN R2 2 +SETTABLE R2 R0 R0 +MOVE R0 R1 RETURN R0 0 )"); @@ -2798,14 +2794,12 @@ RETURN R0 0 CHECK_EQ("\n" + compileFunction0("local a, b a, b, a[b] = 1, 2, 3"), R"( LOADNIL R0 LOADNIL R1 -MOVE R2 R0 -MOVE R3 R1 -LOADN R4 1 -LOADN R5 2 -LOADN R6 3 -MOVE R0 R4 -MOVE R1 R5 -SETTABLE R6 R2 R3 +LOADN R2 1 +LOADN R3 2 +LOADN R4 3 +SETTABLE R4 R0 R1 +MOVE R0 R2 +MOVE R1 R3 RETURN R0 0 )"); @@ -2815,10 +2809,9 @@ RETURN R0 0 LOADNIL R0 GETIMPORT R1 1 ADDK R2 R0 K2 -LOADN R3 1 -LOADN R4 2 -MOVE R0 R3 -SETTABLE R4 R1 R2 +LOADN R0 1 +LOADN R3 2 +SETTABLE R3 R1 R2 RETURN R0 0 )"); } @@ -6242,4 +6235,228 @@ RETURN R2 1 )"); } +TEST_CASE("MultipleAssignments") +{ + ScopedFastFlag sff("LuauCompileOptimalAssignment", true); + + // order of assignments is left to right + CHECK_EQ("\n" + compileFunction0(R"( + local a, b + a, b = f(1), f(2) + )"), + R"( +LOADNIL R0 +LOADNIL R1 +GETIMPORT R2 1 +LOADN R3 1 +CALL R2 1 1 +MOVE R0 R2 +GETIMPORT R2 1 +LOADN R3 2 +CALL R2 1 1 +MOVE R1 R2 +RETURN R0 0 +)"); + + // this includes table assignments + CHECK_EQ("\n" + compileFunction0(R"( + local t + t[1], t[2] = 3, 4 + )"), + R"( +LOADNIL R0 +LOADNIL R1 +LOADN R2 3 +LOADN R3 4 +SETTABLEN R2 R0 1 +SETTABLEN R3 R1 2 +RETURN R0 0 +)"); + + // semantically, we evaluate the right hand side first; this allows us to e.g swap elements in a table easily + CHECK_EQ("\n" + compileFunction0(R"( + local t = ... + t[1], t[2] = t[2], t[1] + )"), + R"( +GETVARARGS R0 1 +GETTABLEN R1 R0 2 +GETTABLEN R2 R0 1 +SETTABLEN R1 R0 1 +SETTABLEN R2 R0 2 +RETURN R0 0 +)"); + + // however, we need to optimize local assignments; to do this well, we need to handle assignment conflicts + // let's first go through a few cases where there are no conflicts: + + // when multiple assignments have no conflicts (all local vars are read after being assigned), codegen is the same as a series of single + // assignments + CHECK_EQ("\n" + compileFunction0(R"( + local xm1, x, xp1, xi = ... + + xm1,x,xp1,xi = x,xp1,xp1+1,xi-1 + )"), + R"( +GETVARARGS R0 4 +MOVE R0 R1 +MOVE R1 R2 +ADDK R2 R2 K0 +SUBK R3 R3 K0 +RETURN R0 0 +)"); + + // similar example to above from a more complex case + CHECK_EQ("\n" + compileFunction0(R"( + local a, b, c, d, e, f, g, h, t1, t2 = ... + + h, g, f, e, d, c, b, a = g, f, e, d + t1, c, b, a, t1 + t2 + )"), + R"( +GETVARARGS R0 10 +MOVE R7 R6 +MOVE R6 R5 +MOVE R5 R4 +ADD R4 R3 R8 +MOVE R3 R2 +MOVE R2 R1 +MOVE R1 R0 +ADD R0 R8 R9 +RETURN R0 0 +)"); + + // when locals have a conflict, we assign temporaries instead of locals, and at the end copy the values back + // the basic example of this is a swap/rotate + CHECK_EQ("\n" + compileFunction0(R"( + local a, b = ... + a, b = b, a + )"), + R"( +GETVARARGS R0 2 +MOVE R2 R1 +MOVE R1 R0 +MOVE R0 R2 +RETURN R0 0 +)"); + + CHECK_EQ("\n" + compileFunction0(R"( + local a, b, c = ... + a, b, c = c, a, b + )"), + R"( +GETVARARGS R0 3 +MOVE R3 R2 +MOVE R4 R0 +MOVE R2 R1 +MOVE R0 R3 +MOVE R1 R4 +RETURN R0 0 +)"); + + CHECK_EQ("\n" + compileFunction0(R"( + local a, b, c = ... + a, b, c = b, c, a + )"), + R"( +GETVARARGS R0 3 +MOVE R3 R1 +MOVE R1 R2 +MOVE R2 R0 +MOVE R0 R3 +RETURN R0 0 +)"); + + // multiple assignments with multcall handling - foo() evalutes to temporary registers and they are copied out to target + CHECK_EQ("\n" + compileFunction0(R"( + local a, b, c, d = ... + a, b, c, d = 1, foo() + )"), + R"( +GETVARARGS R0 4 +LOADN R0 1 +GETIMPORT R4 1 +CALL R4 0 3 +MOVE R1 R4 +MOVE R2 R5 +MOVE R3 R6 +RETURN R0 0 +)"); + + // note that during this we still need to handle local reassignment, eg when table assignments are performed + CHECK_EQ("\n" + compileFunction0(R"( + local a, b, c, d = ... + a, b[a], c[d], d = 1, foo() + )"), + R"( +GETVARARGS R0 4 +LOADN R4 1 +GETIMPORT R6 1 +CALL R6 0 3 +SETTABLE R6 R1 R0 +SETTABLE R7 R2 R3 +MOVE R0 R4 +MOVE R3 R8 +RETURN R0 0 +)"); + + // multiple assignments with multcall handling - foo evaluates to a single argument so all remaining locals are assigned to nil + // note that here we don't assign the locals directly, as this case is very rare so we use the similar code path as above + CHECK_EQ("\n" + compileFunction0(R"( + local a, b, c, d = ... + a, b, c, d = 1, foo + )"), + R"( +GETVARARGS R0 4 +LOADN R0 1 +GETIMPORT R4 1 +LOADNIL R5 +LOADNIL R6 +MOVE R1 R4 +MOVE R2 R5 +MOVE R3 R6 +RETURN R0 0 +)"); + + // note that we also try to use locals as a source of assignment directly when assigning fields; this works using old local value when possible + CHECK_EQ("\n" + compileFunction0(R"( + local a, b = ... + a[1], a[2] = b, b + 1 + )"), + R"( +GETVARARGS R0 2 +ADDK R2 R1 K0 +SETTABLEN R1 R0 1 +SETTABLEN R2 R0 2 +RETURN R0 0 +)"); + + // ... of course if the local is reassigned, we defer the assignment until later + CHECK_EQ("\n" + compileFunction0(R"( + local a, b = ... + b, a[1] = 42, b + )"), + R"( +GETVARARGS R0 2 +LOADN R2 42 +SETTABLEN R1 R0 1 +MOVE R1 R2 +RETURN R0 0 +)"); + + // when there are more expressions when values, we evalute them for side effects, but they also participate in conflict handling + CHECK_EQ("\n" + compileFunction0(R"( + local a, b = ... + a, b = 1, 2, a + b + )"), + R"( +GETVARARGS R0 2 +LOADN R2 1 +LOADN R3 2 +ADD R4 R0 R1 +MOVE R0 R2 +MOVE R1 R3 +RETURN R0 0 +)"); +} + TEST_SUITE_END(); diff --git a/tests/Fixture.cpp b/tests/Fixture.cpp index f51a9d1b..f0146602 100644 --- a/tests/Fixture.cpp +++ b/tests/Fixture.cpp @@ -372,17 +372,25 @@ void Fixture::registerTestTypes() void Fixture::dumpErrors(const CheckResult& cr) { - dumpErrors(std::cout, cr.errors); + std::string error = getErrors(cr); + if (!error.empty()) + MESSAGE(error); } void Fixture::dumpErrors(const ModulePtr& module) { - dumpErrors(std::cout, module->errors); + std::stringstream ss; + dumpErrors(ss, module->errors); + if (!ss.str().empty()) + MESSAGE(ss.str()); } void Fixture::dumpErrors(const Module& module) { - dumpErrors(std::cout, module.errors); + std::stringstream ss; + dumpErrors(ss, module.errors); + if (!ss.str().empty()) + MESSAGE(ss.str()); } std::string Fixture::getErrors(const CheckResult& cr) @@ -413,6 +421,7 @@ LoadDefinitionFileResult Fixture::loadDefinition(const std::string& source) LoadDefinitionFileResult result = frontend.loadDefinitionFile(source, "@test"); freeze(typeChecker.globalTypes); + dumpErrors(result.module); REQUIRE_MESSAGE(result.success, "loadDefinition: unable to load definition file"); return result; } @@ -434,7 +443,7 @@ BuiltinsFixture::BuiltinsFixture(bool freeze, bool prepareAutocomplete) ConstraintGraphBuilderFixture::ConstraintGraphBuilderFixture() : Fixture() - , cgb(mainModuleName, &arena, NotNull(&ice), frontend.getGlobalScope()) + , cgb(mainModuleName, getMainModule(), &arena, NotNull(&ice), frontend.getGlobalScope()) , forceTheFlag{"DebugLuauDeferredConstraintResolution", true} { BlockedTypeVar::nextIndex = 0; diff --git a/tests/Linter.test.cpp b/tests/Linter.test.cpp index 35c40508..64c6d3e4 100644 --- a/tests/Linter.test.cpp +++ b/tests/Linter.test.cpp @@ -1675,7 +1675,7 @@ TEST_CASE_FIXTURE(Fixture, "WrongCommentOptimize") CHECK_EQ(result.warnings[3].text, "optimize directive uses unknown optimization level '100500', 0..2 expected"); } -TEST_CASE_FIXTURE(Fixture, "LintIntegerParsing") +TEST_CASE_FIXTURE(Fixture, "IntegerParsing") { ScopedFastFlag luauLintParseIntegerIssues{"LuauLintParseIntegerIssues", true}; @@ -1690,7 +1690,7 @@ local _ = 0x10000000000000000 } // TODO: remove with FFlagLuauErrorDoubleHexPrefix -TEST_CASE_FIXTURE(Fixture, "LintIntegerParsingDoublePrefix") +TEST_CASE_FIXTURE(Fixture, "IntegerParsingDoublePrefix") { ScopedFastFlag luauLintParseIntegerIssues{"LuauLintParseIntegerIssues", true}; ScopedFastFlag luauErrorDoubleHexPrefix{"LuauErrorDoubleHexPrefix", false}; // Lint will be available until we start rejecting code @@ -1707,4 +1707,36 @@ local _ = 0x0xffffffffffffffffffffffffffffffffff "Hexadecimal number literal has a double prefix, which will fail to parse in the future; remove the extra 0x to fix"); } +TEST_CASE_FIXTURE(Fixture, "ComparisonPrecedence") +{ + ScopedFastFlag sff("LuauLintComparisonPrecedence", true); + + LintResult result = lint(R"( +local a, b = ... + +local _ = not a == b +local _ = not a ~= b +local _ = not a <= b +local _ = a <= b == 0 + +local _ = not a == not b -- weird but ok + +-- silence tests for all of the above +local _ = not (a == b) +local _ = (not a) == b +local _ = not (a ~= b) +local _ = (not a) ~= b +local _ = not (a <= b) +local _ = (not a) <= b +local _ = (a <= b) == 0 +local _ = a <= (b == 0) +)"); + + REQUIRE_EQ(result.warnings.size(), 4); + CHECK_EQ(result.warnings[0].text, "not X == Y is equivalent to (not X) == Y; consider using X ~= Y, or wrap one of the expressions in parentheses to silence"); + CHECK_EQ(result.warnings[1].text, "not X ~= Y is equivalent to (not X) ~= Y; consider using X == Y, or wrap one of the expressions in parentheses to silence"); + CHECK_EQ(result.warnings[2].text, "not X <= Y is equivalent to (not X) <= Y; wrap one of the expressions in parentheses to silence"); + CHECK_EQ(result.warnings[3].text, "X <= Y == Z is equivalent to (X <= Y) == Z; wrap one of the expressions in parentheses to silence"); +} + TEST_SUITE_END(); diff --git a/tests/Module.test.cpp b/tests/Module.test.cpp index dd94e9d7..5ec375c1 100644 --- a/tests/Module.test.cpp +++ b/tests/Module.test.cpp @@ -317,4 +317,78 @@ type B = A CHECK(toString(it->second.type) == "any"); } +TEST_CASE_FIXTURE(BuiltinsFixture, "do_not_clone_reexports") +{ + ScopedFastFlag flags[] = { + {"LuauClonePublicInterfaceLess", true}, + {"LuauSubstitutionReentrant", true}, + {"LuauClassTypeVarsInSubstitution", true}, + {"LuauSubstitutionFixMissingFields", true}, + }; + + fileResolver.source["Module/A"] = R"( +export type A = {p : number} +return {} + )"; + + fileResolver.source["Module/B"] = R"( +local a = require(script.Parent.A) +export type B = {q : a.A} +return {} + )"; + + CheckResult result = frontend.check("Module/B"); + LUAU_REQUIRE_NO_ERRORS(result); + + ModulePtr modA = frontend.moduleResolver.getModule("Module/A"); + ModulePtr modB = frontend.moduleResolver.getModule("Module/B"); + REQUIRE(modA); + REQUIRE(modB); + auto modAiter = modA->getModuleScope()->exportedTypeBindings.find("A"); + auto modBiter = modB->getModuleScope()->exportedTypeBindings.find("B"); + REQUIRE(modAiter != modA->getModuleScope()->exportedTypeBindings.end()); + REQUIRE(modBiter != modB->getModuleScope()->exportedTypeBindings.end()); + TypeId typeA = modAiter->second.type; + TypeId typeB = modBiter->second.type; + TableTypeVar* tableB = getMutable(typeB); + REQUIRE(tableB); + CHECK(typeA == tableB->props["q"].type); +} + +TEST_CASE_FIXTURE(BuiltinsFixture, "do_not_clone_types_of_reexported_values") +{ + ScopedFastFlag flags[] = { + {"LuauClonePublicInterfaceLess", true}, + {"LuauSubstitutionReentrant", true}, + {"LuauClassTypeVarsInSubstitution", true}, + {"LuauSubstitutionFixMissingFields", true}, + }; + + fileResolver.source["Module/A"] = R"( +local exports = {a={p=5}} +return exports + )"; + + fileResolver.source["Module/B"] = R"( +local a = require(script.Parent.A) +local exports = {b=a.a} +return exports + )"; + + CheckResult result = frontend.check("Module/B"); + LUAU_REQUIRE_NO_ERRORS(result); + + ModulePtr modA = frontend.moduleResolver.getModule("Module/A"); + ModulePtr modB = frontend.moduleResolver.getModule("Module/B"); + REQUIRE(modA); + REQUIRE(modB); + std::optional typeA = first(modA->getModuleScope()->returnType); + std::optional typeB = first(modB->getModuleScope()->returnType); + REQUIRE(typeA); + REQUIRE(typeB); + TableTypeVar* tableA = getMutable(*typeA); + TableTypeVar* tableB = getMutable(*typeB); + CHECK(tableA->props["a"].type == tableB->props["b"].type); +} + TEST_SUITE_END(); diff --git a/tests/NonstrictMode.test.cpp b/tests/NonstrictMode.test.cpp index 02e02e6b..89aab5ee 100644 --- a/tests/NonstrictMode.test.cpp +++ b/tests/NonstrictMode.test.cpp @@ -169,6 +169,7 @@ TEST_CASE_FIXTURE(Fixture, "table_props_are_any") REQUIRE(ttv != nullptr); + REQUIRE(ttv->props.count("foo")); TypeId fooProp = ttv->props["foo"].type; REQUIRE(fooProp != nullptr); diff --git a/tests/TypeInfer.definitions.test.cpp b/tests/TypeInfer.definitions.test.cpp index 4545b8db..fd6fb83f 100644 --- a/tests/TypeInfer.definitions.test.cpp +++ b/tests/TypeInfer.definitions.test.cpp @@ -11,6 +11,33 @@ using namespace Luau; TEST_SUITE_BEGIN("DefinitionTests"); +TEST_CASE_FIXTURE(Fixture, "definition_file_simple") +{ + loadDefinition(R"( + declare foo: number + declare function bar(x: number): string + declare foo2: typeof(foo) + )"); + + TypeId globalFooTy = getGlobalBinding(frontend.typeChecker, "foo"); + CHECK_EQ(toString(globalFooTy), "number"); + + TypeId globalBarTy = getGlobalBinding(frontend.typeChecker, "bar"); + CHECK_EQ(toString(globalBarTy), "(number) -> string"); + + TypeId globalFoo2Ty = getGlobalBinding(frontend.typeChecker, "foo2"); + CHECK_EQ(toString(globalFoo2Ty), "number"); + + CheckResult result = check(R"( + local x: number = foo - 1 + local y: string = bar(x) + local z: number | string = x + z = y + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + TEST_CASE_FIXTURE(Fixture, "definition_file_loading") { loadDefinition(R"( diff --git a/tests/TypeInfer.generics.test.cpp b/tests/TypeInfer.generics.test.cpp index a8325727..9ac259cf 100644 --- a/tests/TypeInfer.generics.test.cpp +++ b/tests/TypeInfer.generics.test.cpp @@ -845,6 +845,7 @@ TEST_CASE_FIXTURE(Fixture, "generic_table_method") TableTypeVar* tTable = getMutable(tType); REQUIRE(tTable != nullptr); + REQUIRE(tTable->props.count("bar")); TypeId barType = tTable->props["bar"].type; REQUIRE(barType != nullptr); diff --git a/tests/TypeInfer.modules.test.cpp b/tests/TypeInfer.modules.test.cpp index a1d41339..754fb193 100644 --- a/tests/TypeInfer.modules.test.cpp +++ b/tests/TypeInfer.modules.test.cpp @@ -398,8 +398,6 @@ caused by: TEST_CASE_FIXTURE(BuiltinsFixture, "constrained_anyification_clone_immutable_types") { - ScopedFastFlag luauAnyificationMustClone{"LuauAnyificationMustClone", true}; - fileResolver.source["game/A"] = R"( return function(...) end )"; diff --git a/tests/TypeInfer.tables.test.cpp b/tests/TypeInfer.tables.test.cpp index d9bfc89d..8c7d8a53 100644 --- a/tests/TypeInfer.tables.test.cpp +++ b/tests/TypeInfer.tables.test.cpp @@ -1847,6 +1847,7 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "quantifying_a_bound_var_works") TypeId ty = requireType("clazz"); TableTypeVar* ttv = getMutable(ty); REQUIRE(ttv); + REQUIRE(ttv->props.count("new")); Property& prop = ttv->props["new"]; REQUIRE(prop.type); const FunctionTypeVar* ftv = get(follow(prop.type)); @@ -2516,6 +2517,7 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "dont_quantify_table_that_belongs_to_outer_sc TableTypeVar* counterType = getMutable(requireType("Counter")); REQUIRE(counterType); + REQUIRE(counterType->props.count("new")); const FunctionTypeVar* newType = get(follow(counterType->props["new"].type)); REQUIRE(newType); @@ -3081,8 +3083,6 @@ TEST_CASE_FIXTURE(Fixture, "quantify_even_that_table_was_never_exported_at_all") TEST_CASE_FIXTURE(BuiltinsFixture, "leaking_bad_metatable_errors") { - ScopedFastFlag luauIndexSilenceErrors{"LuauIndexSilenceErrors", true}; - CheckResult result = check(R"( local a = setmetatable({}, 1) local b = a.x diff --git a/tests/TypeVar.test.cpp b/tests/TypeVar.test.cpp index f4670048..edec8442 100644 --- a/tests/TypeVar.test.cpp +++ b/tests/TypeVar.test.cpp @@ -182,6 +182,17 @@ TEST_CASE_FIXTURE(Fixture, "UnionTypeVarIterator_with_empty_union") CHECK(actual.empty()); } +TEST_CASE_FIXTURE(Fixture, "UnionTypeVarIterator_with_only_cyclic_union") +{ + TypeVar tv{UnionTypeVar{}}; + auto utv = getMutable(&tv); + utv->options.push_back(&tv); + utv->options.push_back(&tv); + + std::vector actual(begin(utv), end(utv)); + CHECK(actual.empty()); +} + TEST_CASE_FIXTURE(Fixture, "substitution_skip_failure") { TypeVar ftv11{FreeTypeVar{TypeLevel{}}}; diff --git a/tests/conformance/basic.lua b/tests/conformance/basic.lua index b2dcaf94..d9274751 100644 --- a/tests/conformance/basic.lua +++ b/tests/conformance/basic.lua @@ -49,6 +49,12 @@ assert((function() _G.foo = 1 return _G['foo'] end)() == 1) assert((function() _G['bar'] = 1 return _G.bar end)() == 1) assert((function() local a = 1 (function () a = 2 end)() return a end)() == 2) +-- assignments with local conflicts +assert((function() local a, b = 1, {} a, b[a] = 43, -1 return a + b[1] end)() == 42) +assert((function() local a = {} local b = a a[1], a = 43, -1 return a + b[1] end)() == 42) +assert((function() local a, b = 1, {} a, b[a] = (function() return 43, -1 end)() return a + b[1] end)() == 42) +assert((function() local a = {} local b = a a[1], a = (function() return 43, -1 end)() return a + b[1] end)() == 42) + -- upvalues assert((function() local a = 1 function foo() return a end return foo() end)() == 1) diff --git a/tests/conformance/errors.lua b/tests/conformance/errors.lua index b13e7a82..b69d437b 100644 --- a/tests/conformance/errors.lua +++ b/tests/conformance/errors.lua @@ -295,7 +295,7 @@ end -- testing syntax limits -local syntaxdepth = if limitedstack then 200 else 1000 +local syntaxdepth = if limitedstack then 200 else 500 local function testrep (init, rep) local s = "local a; "..init .. string.rep(rep, syntaxdepth) local a,b = loadstring(s) diff --git a/tests/conformance/strings.lua b/tests/conformance/strings.lua index c87cf15c..3d8fdd1f 100644 --- a/tests/conformance/strings.lua +++ b/tests/conformance/strings.lua @@ -145,6 +145,14 @@ end) == false) assert(string.format("%*", "a\0b\0c") == "a\0b\0c") assert(string.format("%*", string.rep("doge", 3000)) == string.rep("doge", 3000)) +assert(string.format("%*", 42) == "42") +assert(string.format("%*", true) == "true") + +assert(string.format("%*", setmetatable({}, { __tostring = function() return "ok" end })) == "ok") + +local ud = newproxy(true) +getmetatable(ud).__tostring = function() return "good" end +assert(string.format("%*", ud) == "good") assert(pcall(function() string.format("%#*", "bad form") diff --git a/tools/faillist.txt b/tools/faillist.txt index 6e93345b..d796236d 100644 --- a/tools/faillist.txt +++ b/tools/faillist.txt @@ -1,13 +1,8 @@ -AnnotationTests.as_expr_does_not_propagate_type_info -AnnotationTests.as_expr_is_bidirectional -AnnotationTests.as_expr_warns_on_unrelated_cast AnnotationTests.builtin_types_are_not_exported AnnotationTests.cannot_use_nonexported_type AnnotationTests.cloned_interface_maintains_pointers_between_definitions -AnnotationTests.define_generic_type_alias AnnotationTests.duplicate_type_param_name AnnotationTests.for_loop_counter_annotation_is_checked -AnnotationTests.function_return_annotations_are_checked AnnotationTests.generic_aliases_are_cloned_properly AnnotationTests.interface_types_belong_to_interface_arena AnnotationTests.luau_ice_triggers_an_ice @@ -18,21 +13,14 @@ AnnotationTests.luau_print_is_magic_if_the_flag_is_set AnnotationTests.luau_print_is_not_special_without_the_flag AnnotationTests.occurs_check_on_cyclic_intersection_typevar AnnotationTests.occurs_check_on_cyclic_union_typevar -AnnotationTests.self_referential_type_alias AnnotationTests.too_many_type_params AnnotationTests.two_type_params -AnnotationTests.type_annotations_inside_function_bodies -AnnotationTests.type_assertion_expr AnnotationTests.unknown_type_reference_generates_error AnnotationTests.use_type_required_from_another_file AstQuery.last_argument_function_call_type -AstQuery::getDocumentationSymbolAtPosition.binding -AstQuery::getDocumentationSymbolAtPosition.event_callback_arg AstQuery::getDocumentationSymbolAtPosition.overloaded_fn -AstQuery::getDocumentationSymbolAtPosition.prop AutocompleteTest.argument_types AutocompleteTest.arguments_to_global_lambda -AutocompleteTest.as_types AutocompleteTest.autocomplete_boolean_singleton AutocompleteTest.autocomplete_end_with_fn_exprs AutocompleteTest.autocomplete_end_with_lambda @@ -127,7 +115,6 @@ BuiltinTests.assert_removes_falsy_types2 BuiltinTests.assert_removes_falsy_types_even_from_type_pack_tail_but_only_for_the_first_type BuiltinTests.assert_returns_false_and_string_iff_it_knows_the_first_argument_cannot_be_truthy BuiltinTests.bad_select_should_not_crash -BuiltinTests.builtin_tables_sealed BuiltinTests.coroutine_resume_anything_goes BuiltinTests.coroutine_wrap_anything_goes BuiltinTests.debug_info_is_crazy @@ -136,28 +123,20 @@ BuiltinTests.dont_add_definitions_to_persistent_types BuiltinTests.find_capture_types BuiltinTests.find_capture_types2 BuiltinTests.find_capture_types3 -BuiltinTests.gcinfo BuiltinTests.getfenv BuiltinTests.global_singleton_types_are_sealed BuiltinTests.gmatch_capture_types BuiltinTests.gmatch_capture_types2 BuiltinTests.gmatch_capture_types_balanced_escaped_parens BuiltinTests.gmatch_capture_types_default_capture -BuiltinTests.gmatch_capture_types_invalid_pattern_fallback_to_builtin -BuiltinTests.gmatch_capture_types_invalid_pattern_fallback_to_builtin2 -BuiltinTests.gmatch_capture_types_leading_end_bracket_is_part_of_set BuiltinTests.gmatch_capture_types_parens_in_sets_are_ignored BuiltinTests.gmatch_capture_types_set_containing_lbracket BuiltinTests.gmatch_definition BuiltinTests.ipairs_iterator_should_infer_types_and_type_check -BuiltinTests.lua_51_exported_globals_all_exist BuiltinTests.match_capture_types BuiltinTests.match_capture_types2 BuiltinTests.math_max_checks_for_numbers -BuiltinTests.math_max_variatic -BuiltinTests.math_things_are_defined BuiltinTests.next_iterator_should_infer_types_and_type_check -BuiltinTests.no_persistent_typelevel_change BuiltinTests.os_time_takes_optional_date_table BuiltinTests.pairs_iterator_should_infer_types_and_type_check BuiltinTests.see_thru_select @@ -170,7 +149,6 @@ BuiltinTests.select_with_variadic_typepack_tail BuiltinTests.select_with_variadic_typepack_tail_and_string_head BuiltinTests.set_metatable_needs_arguments BuiltinTests.setmetatable_should_not_mutate_persisted_types -BuiltinTests.setmetatable_unpacks_arg_types_correctly BuiltinTests.sort BuiltinTests.sort_with_bad_predicate BuiltinTests.sort_with_predicate @@ -179,6 +157,8 @@ BuiltinTests.string_format_arg_types_inference BuiltinTests.string_format_as_method BuiltinTests.string_format_correctly_ordered_types BuiltinTests.string_format_report_all_type_errors_at_correct_positions +BuiltinTests.string_format_tostring_specifier +BuiltinTests.string_format_tostring_specifier_type_constraint BuiltinTests.string_format_use_correct_argument BuiltinTests.string_format_use_correct_argument2 BuiltinTests.string_lib_self_noself @@ -190,53 +170,38 @@ BuiltinTests.table_insert_correctly_infers_type_of_array_3_args_overload BuiltinTests.table_pack BuiltinTests.table_pack_reduce BuiltinTests.table_pack_variadic -BuiltinTests.thread_is_a_type BuiltinTests.tonumber_returns_optional_number_type BuiltinTests.tonumber_returns_optional_number_type2 -BuiltinTests.xpcall -DefinitionTests.class_definition_function_prop DefinitionTests.declaring_generic_functions -DefinitionTests.definition_file_class_function_args DefinitionTests.definition_file_classes DefinitionTests.definition_file_loading +DefinitionTests.definitions_documentation_symbols +DefinitionTests.documentation_symbols_dont_attach_to_persistent_types DefinitionTests.single_class_type_identity_in_global_types -FrontendTest.accumulate_cached_errors -FrontendTest.accumulate_cached_errors_in_consistent_order -FrontendTest.any_annotation_breaks_cycle FrontendTest.ast_node_at_position -FrontendTest.automatically_check_cyclically_dependent_scripts FrontendTest.automatically_check_dependent_scripts FrontendTest.check_without_builtin_next FrontendTest.clearStats -FrontendTest.cycle_detection_between_check_and_nocheck -FrontendTest.cycle_detection_disabled_in_nocheck -FrontendTest.cycle_error_paths -FrontendTest.cycle_errors_can_be_fixed -FrontendTest.cycle_incremental_type_surface -FrontendTest.cycle_incremental_type_surface_longer -FrontendTest.dont_recheck_script_that_hasnt_been_marked_dirty FrontendTest.dont_reparse_clean_file_when_linting FrontendTest.environments -FrontendTest.ignore_require_to_nonexistent_file FrontendTest.imported_table_modification_2 FrontendTest.it_should_be_safe_to_stringify_errors_when_full_type_graph_is_discarded FrontendTest.no_use_after_free_with_type_fun_instantiation FrontendTest.nocheck_cycle_used_by_checked FrontendTest.nocheck_modules_are_typed FrontendTest.produce_errors_for_unchanged_file_with_a_syntax_error -FrontendTest.re_report_type_error_in_required_file FrontendTest.recheck_if_dependent_script_is_dirty FrontendTest.reexport_cyclic_type FrontendTest.reexport_type_alias FrontendTest.report_require_to_nonexistent_file FrontendTest.report_syntax_error_in_required_file -FrontendTest.reports_errors_from_multiple_sources FrontendTest.stats_are_not_reset_between_checks FrontendTest.trace_requires_in_nonstrict_mode GenericsTests.apply_type_function_nested_generics1 GenericsTests.apply_type_function_nested_generics2 GenericsTests.better_mismatch_error_messages GenericsTests.bound_tables_do_not_clone_original_fields +GenericsTests.calling_self_generic_methods GenericsTests.check_generic_typepack_function GenericsTests.check_mutual_generic_functions GenericsTests.correctly_instantiate_polymorphic_member_functions @@ -265,8 +230,7 @@ GenericsTests.generic_type_pack_unification3 GenericsTests.infer_generic_function_function_argument GenericsTests.infer_generic_function_function_argument_overloaded GenericsTests.infer_generic_lib_function_function_argument -GenericsTests.infer_generic_property -GenericsTests.inferred_local_vars_can_be_polytypes +GenericsTests.infer_generic_methods GenericsTests.instantiate_cyclic_generic_function GenericsTests.instantiate_generic_function_in_assignments GenericsTests.instantiate_generic_function_in_assignments2 @@ -276,7 +240,6 @@ GenericsTests.local_vars_can_be_instantiated_polytypes GenericsTests.mutable_state_polymorphism GenericsTests.no_stack_overflow_from_quantifying GenericsTests.properties_can_be_instantiated_polytypes -GenericsTests.properties_can_be_polytypes GenericsTests.rank_N_types_via_typeof GenericsTests.reject_clashing_generic_and_pack_names GenericsTests.self_recursive_instantiated_param @@ -287,30 +250,23 @@ IntersectionTypes.error_detailed_intersection_part IntersectionTypes.fx_intersection_as_argument IntersectionTypes.fx_union_as_argument_fails IntersectionTypes.index_on_an_intersection_type_with_mixed_types -IntersectionTypes.index_on_an_intersection_type_with_one_part_missing_the_property -IntersectionTypes.index_on_an_intersection_type_with_one_property_of_type_any IntersectionTypes.index_on_an_intersection_type_with_property_guaranteed_to_exist IntersectionTypes.index_on_an_intersection_type_works_at_arbitrary_depth IntersectionTypes.no_stack_overflow_from_flattenintersection IntersectionTypes.overload_is_not_a_function IntersectionTypes.select_correct_union_fn IntersectionTypes.should_still_pick_an_overload_whose_arguments_are_unions -IntersectionTypes.table_intersection_setmetatable IntersectionTypes.table_intersection_write IntersectionTypes.table_intersection_write_sealed IntersectionTypes.table_intersection_write_sealed_indirect IntersectionTypes.table_write_sealed_indirect -isSubtype.functions_and_any -isSubtype.intersection_of_functions_of_different_arities isSubtype.intersection_of_tables -isSubtype.table_with_any_prop isSubtype.table_with_table_prop -isSubtype.tables -Linter.DeprecatedApi Linter.TableOperations -ModuleTests.builtin_types_point_into_globalTypes_arena ModuleTests.clone_self_property ModuleTests.deepClone_cyclic_table +ModuleTests.do_not_clone_reexports +ModuleTests.do_not_clone_types_of_reexported_values NonstrictModeTests.delay_function_does_not_require_its_argument_to_return_anything NonstrictModeTests.for_in_iterator_variables_are_any NonstrictModeTests.function_parameters_are_any @@ -333,7 +289,6 @@ Normalize.cyclic_intersection Normalize.cyclic_table_normalizes_sensibly Normalize.cyclic_union Normalize.fuzz_failure_bound_type_is_normal_but_not_its_bounded_to -Normalize.fuzz_failure_instersection_combine_must_follow Normalize.higher_order_function Normalize.intersection_combine_on_bound_self Normalize.intersection_inside_a_table_inside_another_intersection @@ -345,13 +300,11 @@ Normalize.intersection_of_disjoint_tables Normalize.intersection_of_functions Normalize.intersection_of_overlapping_tables Normalize.intersection_of_tables_with_indexers -Normalize.nested_table_normalization_with_non_table__no_ice Normalize.normalization_does_not_convert_ever Normalize.normalize_module_return_type Normalize.normalize_unions_containing_never Normalize.normalize_unions_containing_unknown Normalize.return_type_is_not_a_constrained_intersection -Normalize.skip_force_normal_on_external_types Normalize.union_of_distinct_free_types Normalize.variadic_tail_is_marked_normal Normalize.visiting_a_type_twice_is_not_considered_normal @@ -365,7 +318,6 @@ ProvisionalTests.constrained_is_level_dependent ProvisionalTests.discriminate_from_x_not_equal_to_nil ProvisionalTests.do_not_ice_when_trying_to_pick_first_of_generic_type_pack ProvisionalTests.error_on_eq_metamethod_returning_a_type_other_than_boolean -ProvisionalTests.free_is_not_bound_to_any ProvisionalTests.function_returns_many_things_but_first_of_it_is_forgotten ProvisionalTests.greedy_inference_with_shared_self_triggers_function_with_no_returns ProvisionalTests.invariant_table_properties_means_instantiating_tables_in_call_is_unsound @@ -380,7 +332,6 @@ ProvisionalTests.typeguard_inference_incomplete ProvisionalTests.weird_fail_to_unify_type_pack ProvisionalTests.weirditer_should_not_loop_forever ProvisionalTests.while_body_are_also_refined -ProvisionalTests.xpcall_returns_what_f_returns RefinementTest.and_constraint RefinementTest.and_or_peephole_refinement RefinementTest.apply_refinements_on_astexprindexexpr_whose_subscript_expr_is_constant_string @@ -420,7 +371,6 @@ RefinementTest.not_and_constraint RefinementTest.not_t_or_some_prop_of_t RefinementTest.or_predicate_with_truthy_predicates RefinementTest.parenthesized_expressions_are_followed_through -RefinementTest.refine_a_property_not_to_be_nil_through_an_intersection_table RefinementTest.refine_the_correct_types_opposite_of_when_a_is_not_number_or_string RefinementTest.refine_unknowns RefinementTest.string_not_equal_to_string_or_nil @@ -456,7 +406,6 @@ TableTests.augment_nested_table TableTests.augment_table TableTests.builtin_table_names TableTests.call_method -TableTests.call_method_with_explicit_self_argument TableTests.cannot_augment_sealed_table TableTests.cannot_call_tables TableTests.cannot_change_type_of_unsealed_table_prop @@ -469,16 +418,13 @@ TableTests.common_table_element_union_in_call_tail TableTests.confusing_indexing TableTests.defining_a_method_for_a_builtin_sealed_table_must_fail TableTests.defining_a_method_for_a_local_sealed_table_must_fail -TableTests.defining_a_method_for_a_local_unsealed_table_is_ok TableTests.defining_a_self_method_for_a_builtin_sealed_table_must_fail TableTests.defining_a_self_method_for_a_local_sealed_table_must_fail -TableTests.defining_a_self_method_for_a_local_unsealed_table_is_ok TableTests.dont_crash_when_setmetatable_does_not_produce_a_metatabletypevar TableTests.dont_hang_when_trying_to_look_up_in_cyclic_metatable_index TableTests.dont_invalidate_the_properties_iterator_of_free_table_when_rolled_back TableTests.dont_leak_free_table_props TableTests.dont_quantify_table_that_belongs_to_outer_scope -TableTests.dont_seal_an_unsealed_table_by_passing_it_to_a_function_that_takes_a_sealed_table TableTests.dont_suggest_exact_match_keys TableTests.error_detailed_indexer_key TableTests.error_detailed_indexer_value @@ -508,8 +454,6 @@ TableTests.infer_array_2 TableTests.infer_indexer_from_value_property_in_literal TableTests.inferred_return_type_of_free_table TableTests.inferring_crazy_table_should_also_be_quick -TableTests.instantiate_table_cloning -TableTests.instantiate_table_cloning_2 TableTests.instantiate_table_cloning_3 TableTests.instantiate_tables_at_scope_level TableTests.invariant_table_properties_means_instantiating_tables_in_assignment_is_unsound @@ -518,14 +462,12 @@ TableTests.length_operator_intersection TableTests.length_operator_non_table_union TableTests.length_operator_union TableTests.length_operator_union_errors -TableTests.less_exponential_blowup_please TableTests.meta_add TableTests.meta_add_both_ways TableTests.meta_add_inferred TableTests.metatable_mismatch_should_fail TableTests.missing_metatable_for_sealed_tables_do_not_get_inferred TableTests.mixed_tables_with_implicit_numbered_keys -TableTests.MixedPropertiesAndIndexers TableTests.nil_assign_doesnt_hit_indexer TableTests.okay_to_add_property_to_unsealed_tables_by_function_call TableTests.only_ascribe_synthetic_names_at_module_scope @@ -535,8 +477,8 @@ TableTests.open_table_unification_2 TableTests.pass_a_union_of_tables_to_a_function_that_requires_a_table TableTests.pass_a_union_of_tables_to_a_function_that_requires_a_table_2 TableTests.pass_incompatible_union_to_a_generic_table_without_crashing -TableTests.passing_compatible_unions_to_a_generic_table_without_crashing TableTests.persistent_sealed_table_is_immutable +TableTests.prop_access_on_key_whose_types_mismatches TableTests.property_lookup_through_tabletypevar_metatable TableTests.quantify_even_that_table_was_never_exported_at_all TableTests.quantify_metatables_of_metatables_of_table @@ -570,23 +512,17 @@ TableTests.tc_member_function_2 TableTests.top_table_type TableTests.type_mismatch_on_massive_table_is_cut_short TableTests.unification_of_unions_in_a_self_referential_type -TableTests.unifying_tables_shouldnt_uaf1 TableTests.unifying_tables_shouldnt_uaf2 -TableTests.used_colon_correctly TableTests.used_colon_instead_of_dot TableTests.used_dot_instead_of_colon -TableTests.used_dot_instead_of_colon_but_correctly TableTests.width_subtyping ToDot.bound_table -ToDot.class ToDot.function ToDot.metatable -ToDot.primitive ToDot.table ToString.exhaustive_toString_of_cyclic_table ToString.function_type_with_argument_names_and_self ToString.function_type_with_argument_names_generic -ToString.named_metatable_toStringNamedFunction ToString.no_parentheses_around_cyclic_function_type_in_union ToString.toStringDetailed2 ToString.toStringErrorPack @@ -605,13 +541,12 @@ TryUnifyTests.typepack_unification_should_trim_free_tails TryUnifyTests.variadics_should_use_reversed_properly TypeAliases.cli_38393_recursive_intersection_oom TypeAliases.corecursive_types_generic -TypeAliases.do_not_quantify_unresolved_aliases TypeAliases.forward_declared_alias_is_not_clobbered_by_prior_unification_with_any TypeAliases.general_require_multi_assign TypeAliases.generic_param_remap TypeAliases.mismatched_generic_pack_type_param TypeAliases.mismatched_generic_type_param -TypeAliases.mutually_recursive_generic_aliases +TypeAliases.mutually_recursive_types_errors TypeAliases.mutually_recursive_types_restriction_not_ok_1 TypeAliases.mutually_recursive_types_restriction_not_ok_2 TypeAliases.mutually_recursive_types_swapsies_not_ok @@ -619,29 +554,22 @@ TypeAliases.recursive_types_restriction_not_ok TypeAliases.stringify_optional_parameterized_alias TypeAliases.stringify_type_alias_of_recursive_template_table_type TypeAliases.stringify_type_alias_of_recursive_template_table_type2 -TypeAliases.type_alias_import_mutation +TypeAliases.type_alias_fwd_declaration_is_precise TypeAliases.type_alias_local_mutation TypeAliases.type_alias_local_rename TypeAliases.type_alias_of_an_imported_recursive_generic_type TypeAliases.type_alias_of_an_imported_recursive_type -TypeInfer.check_expr_recursion_limit TypeInfer.checking_should_not_ice -TypeInfer.cli_50041_committing_txnlog_in_apollo_client_error TypeInfer.cyclic_follow TypeInfer.do_not_bind_a_free_table_to_a_union_containing_that_table TypeInfer.dont_report_type_errors_within_an_AstStatError -TypeInfer.follow_on_new_types_in_substitution -TypeInfer.free_typevars_introduced_within_control_flow_constructs_do_not_get_an_elevated_TypeLevel TypeInfer.globals TypeInfer.globals2 -TypeInfer.index_expr_should_be_checked TypeInfer.infer_assignment_value_types TypeInfer.infer_assignment_value_types_mutable_lval TypeInfer.infer_through_group_expr -TypeInfer.infer_type_assertion_value_type TypeInfer.no_heap_use_after_free_error TypeInfer.no_stack_overflow_from_isoptional -TypeInfer.recursive_metatable_crash TypeInfer.tc_after_error_recovery_no_replacement_name_in_error TypeInfer.tc_if_else_expressions1 TypeInfer.tc_if_else_expressions2 @@ -650,56 +578,32 @@ TypeInfer.tc_if_else_expressions_expected_type_2 TypeInfer.tc_if_else_expressions_expected_type_3 TypeInfer.tc_if_else_expressions_type_union TypeInfer.type_infer_recursion_limit_no_ice -TypeInfer.types stored in astResolvedTypes TypeInfer.warn_on_lowercase_parent_property -TypeInfer.weird_case -TypeInferAnyError.any_type_propagates TypeInferAnyError.assign_prop_to_table_by_calling_any_yields_any -TypeInferAnyError.call_to_any_yields_any -TypeInferAnyError.calling_error_type_yields_error TypeInferAnyError.can_get_length_of_any -TypeInferAnyError.can_subscript_any -TypeInferAnyError.CheckMethodsOfAny TypeInferAnyError.for_in_loop_iterator_is_any TypeInferAnyError.for_in_loop_iterator_is_any2 TypeInferAnyError.for_in_loop_iterator_is_error TypeInferAnyError.for_in_loop_iterator_is_error2 TypeInferAnyError.for_in_loop_iterator_returns_any TypeInferAnyError.for_in_loop_iterator_returns_any2 -TypeInferAnyError.indexing_error_type_does_not_produce_an_error TypeInferAnyError.length_of_error_type_does_not_produce_an_error -TypeInferAnyError.metatable_of_any_can_be_a_table -TypeInferAnyError.prop_access_on_any_with_other_options -TypeInferAnyError.quantify_any_does_not_bind_to_itself TypeInferAnyError.replace_every_free_type_when_unifying_a_complex_function_with_any TypeInferAnyError.type_error_addition -TypeInferClasses.assign_to_prop_of_class TypeInferClasses.call_base_method TypeInferClasses.call_instance_method -TypeInferClasses.call_method_of_a_child_class -TypeInferClasses.call_method_of_a_class -TypeInferClasses.can_assign_to_prop_of_base_class -TypeInferClasses.can_assign_to_prop_of_base_class_using_string TypeInferClasses.can_read_prop_of_base_class TypeInferClasses.can_read_prop_of_base_class_using_string -TypeInferClasses.cannot_call_method_of_child_on_base_instance -TypeInferClasses.cannot_call_unknown_method_of_a_class -TypeInferClasses.cannot_unify_class_instance_with_primitive TypeInferClasses.class_type_mismatch_with_name_conflict -TypeInferClasses.class_unification_type_mismatch_is_correct_order TypeInferClasses.classes_can_have_overloaded_operators TypeInferClasses.classes_without_overloaded_operators_cannot_be_added TypeInferClasses.detailed_class_unification_error TypeInferClasses.function_arguments_are_covariant -TypeInferClasses.higher_order_function_arguments_are_contravariant TypeInferClasses.higher_order_function_return_type_is_not_contravariant TypeInferClasses.higher_order_function_return_values_are_covariant TypeInferClasses.optional_class_field_access_error TypeInferClasses.table_class_unification_reports_sane_errors_for_missing_properties -TypeInferClasses.table_indexers_are_invariant -TypeInferClasses.table_properties_are_invariant TypeInferClasses.warn_when_prop_almost_matches -TypeInferClasses.we_can_infer_that_a_parameter_must_be_a_particular_class TypeInferClasses.we_can_report_when_someone_is_trying_to_use_a_table_rather_than_a_class TypeInferFunctions.another_indirect_function_case_where_it_is_ok_to_provide_too_many_arguments TypeInferFunctions.another_recursive_local_function @@ -711,21 +615,18 @@ TypeInferFunctions.complicated_return_types_require_an_explicit_annotation TypeInferFunctions.cyclic_function_type_in_args TypeInferFunctions.dont_give_other_overloads_message_if_only_one_argument_matching_overload_exists TypeInferFunctions.dont_infer_parameter_types_for_functions_from_their_call_site -TypeInferFunctions.dont_mutate_the_underlying_head_of_typepack_when_calling_with_self TypeInferFunctions.duplicate_functions_with_different_signatures_not_allowed_in_nonstrict TypeInferFunctions.error_detailed_function_mismatch_arg TypeInferFunctions.error_detailed_function_mismatch_arg_count TypeInferFunctions.error_detailed_function_mismatch_ret TypeInferFunctions.error_detailed_function_mismatch_ret_count TypeInferFunctions.error_detailed_function_mismatch_ret_mult -TypeInferFunctions.first_argument_can_be_optional TypeInferFunctions.free_is_not_bound_to_unknown TypeInferFunctions.func_expr_doesnt_leak_free TypeInferFunctions.function_cast_error_uses_correct_language TypeInferFunctions.function_decl_non_self_sealed_overwrite TypeInferFunctions.function_decl_non_self_sealed_overwrite_2 TypeInferFunctions.function_decl_non_self_unsealed_overwrite -TypeInferFunctions.function_decl_quantify_right_type TypeInferFunctions.function_does_not_return_enough_values TypeInferFunctions.function_statement_sealed_table_assignment_through_indexer TypeInferFunctions.higher_order_function_2 @@ -737,13 +638,10 @@ TypeInferFunctions.infer_anonymous_function_arguments TypeInferFunctions.infer_anonymous_function_arguments_outside_call TypeInferFunctions.infer_return_type_from_selected_overload TypeInferFunctions.infer_that_function_does_not_return_a_table -TypeInferFunctions.inferred_higher_order_functions_are_quantified_at_the_right_time -TypeInferFunctions.inferred_higher_order_functions_are_quantified_at_the_right_time2 TypeInferFunctions.it_is_ok_not_to_supply_enough_retvals TypeInferFunctions.it_is_ok_to_oversaturate_a_higher_order_function_argument TypeInferFunctions.list_all_overloads_if_no_overload_takes_given_argument_count TypeInferFunctions.list_only_alternative_overloads_that_match_argument_count -TypeInferFunctions.mutual_recursion TypeInferFunctions.no_lossy_function_type TypeInferFunctions.occurs_check_failure_in_function_return_type TypeInferFunctions.quantify_constrained_types @@ -759,10 +657,8 @@ TypeInferFunctions.too_few_arguments_variadic_generic TypeInferFunctions.too_few_arguments_variadic_generic2 TypeInferFunctions.too_many_arguments TypeInferFunctions.too_many_return_values -TypeInferFunctions.toposort_doesnt_break_mutual_recursion TypeInferFunctions.vararg_function_is_quantified TypeInferFunctions.vararg_functions_should_allow_calls_of_any_types_and_size -TypeInferLoops.correctly_scope_locals_while TypeInferLoops.for_in_loop TypeInferLoops.for_in_loop_error_on_factory_not_returning_the_right_amount_of_values TypeInferLoops.for_in_loop_error_on_iterator_requiring_args_but_none_given @@ -789,14 +685,9 @@ TypeInferLoops.repeat_loop_condition_binds_to_its_block TypeInferLoops.symbols_in_repeat_block_should_not_be_visible_beyond_until_condition TypeInferLoops.unreachable_code_after_infinite_loop TypeInferLoops.varlist_declared_by_for_in_loop_should_be_free -TypeInferLoops.while_loop -TypeInferModules.bound_free_table_export_is_ok -TypeInferModules.constrained_anyification_clone_immutable_types -TypeInferModules.custom_require_global TypeInferModules.do_not_modify_imported_types TypeInferModules.do_not_modify_imported_types_2 TypeInferModules.do_not_modify_imported_types_3 -TypeInferModules.do_not_modify_imported_types_4 TypeInferModules.general_require_call_expression TypeInferModules.general_require_type_mismatch TypeInferModules.module_type_conflict @@ -808,16 +699,14 @@ TypeInferModules.require_module_that_does_not_export TypeInferModules.require_types TypeInferModules.type_error_of_unknown_qualified_type TypeInferModules.warn_if_you_try_to_require_a_non_modulescript +TypeInferOOP.CheckMethodsOfSealed TypeInferOOP.dont_suggest_using_colon_rather_than_dot_if_another_overload_works TypeInferOOP.dont_suggest_using_colon_rather_than_dot_if_it_wont_help_2 TypeInferOOP.dont_suggest_using_colon_rather_than_dot_if_not_defined_with_colon TypeInferOOP.inferred_methods_of_free_tables_have_the_same_level_as_the_enclosing_table TypeInferOOP.inferring_hundreds_of_self_calls_should_not_suffocate_memory -TypeInferOOP.method_depends_on_table TypeInferOOP.methods_are_topologically_sorted TypeInferOOP.nonstrict_self_mismatch_tail -TypeInferOOP.object_constructor_can_refer_to_method_of_self -TypeInferOOP.table_oop TypeInferOperators.and_adds_boolean TypeInferOperators.and_adds_boolean_no_superfluous_union TypeInferOperators.and_binexps_dont_unify @@ -872,37 +761,26 @@ TypeInferPrimitives.string_function_other TypeInferPrimitives.string_index TypeInferPrimitives.string_length TypeInferPrimitives.string_method -TypeInferUnknownNever.array_like_table_of_never_is_inhabitable TypeInferUnknownNever.assign_to_global_which_is_never TypeInferUnknownNever.assign_to_local_which_is_never TypeInferUnknownNever.assign_to_prop_which_is_never TypeInferUnknownNever.assign_to_subscript_which_is_never TypeInferUnknownNever.call_never TypeInferUnknownNever.dont_unify_operands_if_one_of_the_operand_is_never_in_any_ordering_operators -TypeInferUnknownNever.index_on_never TypeInferUnknownNever.index_on_union_of_tables_for_properties_that_is_never TypeInferUnknownNever.index_on_union_of_tables_for_properties_that_is_sorta_never TypeInferUnknownNever.length_of_never TypeInferUnknownNever.math_operators_and_never -TypeInferUnknownNever.never_is_reflexive -TypeInferUnknownNever.never_subtype_and_string_supertype -TypeInferUnknownNever.pick_never_from_variadic_type_pack -TypeInferUnknownNever.string_subtype_and_never_supertype -TypeInferUnknownNever.string_subtype_and_unknown_supertype -TypeInferUnknownNever.table_with_prop_of_type_never_is_also_reflexive -TypeInferUnknownNever.table_with_prop_of_type_never_is_uninhabitable TypeInferUnknownNever.type_packs_containing_never_is_itself_uninhabitable TypeInferUnknownNever.type_packs_containing_never_is_itself_uninhabitable2 TypeInferUnknownNever.unary_minus_of_never TypeInferUnknownNever.unknown_is_reflexive -TypeInferUnknownNever.unknown_subtype_and_string_supertype TypePackTests.cyclic_type_packs TypePackTests.higher_order_function TypePackTests.multiple_varargs_inference_are_not_confused TypePackTests.no_return_size_should_be_zero TypePackTests.pack_tail_unification_check TypePackTests.parenthesized_varargs_returns_any -TypePackTests.self_and_varargs_should_work TypePackTests.type_alias_backwards_compatible TypePackTests.type_alias_default_export TypePackTests.type_alias_default_mixed_self @@ -913,8 +791,6 @@ TypePackTests.type_alias_default_type_pack_self_tp TypePackTests.type_alias_default_type_self TypePackTests.type_alias_defaults_confusing_types TypePackTests.type_alias_defaults_recursive_type -TypePackTests.type_alias_type_pack_explicit -TypePackTests.type_alias_type_pack_explicit_multi TypePackTests.type_alias_type_pack_multi TypePackTests.type_alias_type_pack_variadic TypePackTests.type_alias_type_packs @@ -949,6 +825,7 @@ TypeSingletons.string_singletons_escape_chars TypeSingletons.string_singletons_mismatch TypeSingletons.table_insert_with_a_singleton_argument TypeSingletons.table_properties_type_error_escapes +TypeSingletons.tagged_unions_immutable_tag TypeSingletons.tagged_unions_using_singletons TypeSingletons.taking_the_length_of_string_singleton TypeSingletons.taking_the_length_of_union_of_string_singleton @@ -978,5 +855,4 @@ UnionTypes.optional_union_members UnionTypes.optional_union_methods UnionTypes.return_types_can_be_disjoint UnionTypes.table_union_write_indirect -UnionTypes.unify_unsealed_table_union_check UnionTypes.union_equality_comparisons diff --git a/tools/heapgraph.py b/tools/heapgraph.py index d4d29af1..2817c38f 100644 --- a/tools/heapgraph.py +++ b/tools/heapgraph.py @@ -39,6 +39,16 @@ class Node(svg.Node): def details(self, root): return "{} ({:,} bytes, {:.1%}); self: {:,} bytes in {:,} objects".format(self.name, self.width, self.width / root.width, self.size, self.count) +def getkey(heap, obj, key): + pairs = obj.get("pairs", []) + for i in range(0, len(pairs), 2): + if pairs[i] and heap[pairs[i]]["type"] == "string" and heap[pairs[i]]["data"] == key: + if pairs[i + 1] and heap[pairs[i + 1]]["type"] == "string": + return heap[pairs[i + 1]]["data"] + else: + return None + return None + # load files if arguments.snapshotnew == None: dumpold = None @@ -50,6 +60,8 @@ else: with open(arguments.snapshotnew) as f: dump = json.load(f) +heap = dump["objects"] + # reachability analysis: how much of the heap is reachable from roots? visited = set() queue = [] @@ -66,7 +78,7 @@ while offset < len(queue): continue visited.add(addr) - obj = dump["objects"][addr] + obj = heap[addr] if not dumpold or not addr in dumpold["objects"]: node.count += 1 @@ -75,17 +87,27 @@ while offset < len(queue): if obj["type"] == "table": pairs = obj.get("pairs", []) + weakkey = False + weakval = False + + if "metatable" in obj: + modemt = getkey(heap, heap[obj["metatable"]], "__mode") + if modemt: + weakkey = "k" in modemt + weakval = "v" in modemt for i in range(0, len(pairs), 2): key = pairs[i+0] val = pairs[i+1] - if key and val and dump["objects"][key]["type"] == "string": + if key and heap[key]["type"] == "string": + # string keys are always strong queue.append((key, node)) - queue.append((val, node.child(dump["objects"][key]["data"]))) + if val and not weakval: + queue.append((val, node.child(heap[key]["data"]))) else: - if key: + if key and not weakkey: queue.append((key, node)) - if val: + if val and not weakval: queue.append((val, node)) for a in obj.get("array", []): @@ -97,7 +119,7 @@ while offset < len(queue): source = "" if "proto" in obj: - proto = dump["objects"][obj["proto"]] + proto = heap[obj["proto"]] if "source" in proto: source = proto["source"] diff --git a/tools/heapstat.py b/tools/heapstat.py index 838521b6..4c0cb407 100644 --- a/tools/heapstat.py +++ b/tools/heapstat.py @@ -15,12 +15,20 @@ def updatesize(d, k, s): def sortedsize(p): return sorted(p, key = lambda s: s[1][1], reverse = True) +def getkey(heap, obj, key): + pairs = obj.get("pairs", []) + for i in range(0, len(pairs), 2): + if pairs[i] and heap[pairs[i]]["type"] == "string" and heap[pairs[i]]["data"] == key: + if pairs[i + 1] and heap[pairs[i + 1]]["type"] == "string": + return heap[pairs[i + 1]]["data"] + else: + return None + return None + with open(sys.argv[1]) as f: dump = json.load(f) heap = dump["objects"] -type_addr = next((addr for addr,obj in heap.items() if obj["type"] == "string" and obj["data"] == "__type"), None) - size_type = {} size_udata = {} size_category = {} @@ -33,11 +41,7 @@ for addr, obj in heap.items(): if obj["type"] == "userdata" and "metatable" in obj: metatable = heap[obj["metatable"]] - pairs = metatable.get("pairs", []) - typemt = "unknown" - for i in range(0, len(pairs), 2): - if type_addr and pairs[i] == type_addr and pairs[i + 1] and heap[pairs[i + 1]]["type"] == "string": - typemt = heap[pairs[i + 1]]["data"] + typemt = getkey(heap, metatable, "__type") or "unknown" updatesize(size_udata, typemt, obj["size"]) print("objects by type:") From 8b390a33aaf99eb1d570816502227cfbc14f00d6 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Thu, 11 Aug 2022 14:14:32 -0700 Subject: [PATCH 339/399] Update benchmark.yml Switch to bench.json file so that bench-gcc is the only outlier, as everything else is built with clang. --- .github/workflows/benchmark.yml | 12 +++++------- 1 file changed, 5 insertions(+), 7 deletions(-) diff --git a/.github/workflows/benchmark.yml b/.github/workflows/benchmark.yml index 572e037f..deb8be86 100644 --- a/.github/workflows/benchmark.yml +++ b/.github/workflows/benchmark.yml @@ -13,9 +13,7 @@ on: jobs: callgrind: - name: callgrind strategy: - fail-fast: false matrix: os: [ubuntu-22.04] benchResultsRepo: @@ -44,9 +42,9 @@ jobs: run: | python bench/bench.py --callgrind --vm "./luau-gcc -O2" | tee -a bench-gcc-output.txt - - name: Run benchmark (bench-clang) + - name: Run benchmark (bench) run: | - python bench/bench.py --callgrind --vm "./luau -O2" | tee -a bench-clang-output.txt + python bench/bench.py --callgrind --vm "./luau -O2" | tee -a bench-output.txt - name: Run benchmark (analyze) run: | @@ -78,13 +76,13 @@ jobs: token: ${{ secrets.BENCH_GITHUB_TOKEN }} path: "./gh-pages" - - name: Store results (bench-clang) + - name: Store results (bench) uses: Roblox/rhysd-github-action-benchmark@v-luau with: name: callgrind clang tool: "benchmarkluau" - output-file-path: ./bench-clang-output.txt - external-data-json-path: ./gh-pages/bench-clang.json + output-file-path: ./bench-output.txt + external-data-json-path: ./gh-pages/bench.json - name: Store results (bench-gcc) uses: Roblox/rhysd-github-action-benchmark@v-luau From b7d126bf993405e08f9ac4b6eb5438392b9f7474 Mon Sep 17 00:00:00 2001 From: Mactavsin <69454747+Mactavsin@users.noreply.github.com> Date: Fri, 12 Aug 2022 17:14:38 +0300 Subject: [PATCH 340/399] Fix lint.md formatting (#637) This pull requests fixes a mistake in lint.md file that causes incorrect formatting of the text: --- docs/_pages/lint.md | 1 - 1 file changed, 1 deletion(-) diff --git a/docs/_pages/lint.md b/docs/_pages/lint.md index fd3c595c..681a275a 100644 --- a/docs/_pages/lint.md +++ b/docs/_pages/lint.md @@ -326,7 +326,6 @@ Luau uses comments that start from `!` to control certain aspects of analysis, f --!nostrict -- Unknown comment directive 'nostrict'; did you mean 'nonstrict'?" ``` -``` ## IntegerParsing (27) From cd26f88d56d40e1f184745d3cce6ac3f5d21b0be Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Fri, 12 Aug 2022 09:10:42 -0700 Subject: [PATCH 341/399] Update compatibility.md Add NaN keys from Lua 5.2 --- docs/_pages/compatibility.md | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/_pages/compatibility.md b/docs/_pages/compatibility.md index bdb3d814..1c15e9b8 100644 --- a/docs/_pages/compatibility.md +++ b/docs/_pages/compatibility.md @@ -70,6 +70,7 @@ Sandboxing challenges are [covered in the dedicated section](sandbox). | `bit32` library | ✔️ | | | `string.gsub` is stricter about using `%` on special characters only | ✔️ | | | light C functions | 😞 | this changes semantics of fenv on C functions and has complex implications wrt runtime performance | +| NaN keys are supported for tables with `__newindex` | ✔️ | | Two things that are important to call out here are various new metamethods for tables and yielding in metamethods. In both cases, there are performance implications to supporting this - our implementation is *very* highly tuned for performance, so any changes that affect the core fundamentals of how Lua works have a price. To support yielding in metamethods we'd need to make the core of the VM more involved, since almost every single "interesting" opcode would need to learn how to be resumable - which also complicates future JIT/AOT story. Metamethods in general are important for extensibility, but very challenging to deal with in implementation, so we err on the side of not supporting any new metamethods unless a strong need arises. From 4ded555cc5f6fa9ab7cddb12c0b9e295058da68c Mon Sep 17 00:00:00 2001 From: XmiliaH <45106915+XmiliaH@users.noreply.github.com> Date: Wed, 17 Aug 2022 00:32:48 +0200 Subject: [PATCH 342/399] Prevent overflow in lua_newuserdatadtor (#639) In case a large userdata size is passed to lua_newuserdatadtor it might overflow the size resulting in luaU_newudata actually allocating the object without a memory error. This will then result in overwriting part of the metatable pointer of the userdata. This PR fixes this issue by checking for the overflow and in such cases pass a size value which will cause a memory error in luaU_newudata. --- VM/src/lapi.cpp | 5 ++++- tests/Conformance.test.cpp | 17 +++++++++++++++++ 2 files changed, 21 insertions(+), 1 deletion(-) diff --git a/VM/src/lapi.cpp b/VM/src/lapi.cpp index bb994fb4..77788e40 100644 --- a/VM/src/lapi.cpp +++ b/VM/src/lapi.cpp @@ -1209,7 +1209,10 @@ void* lua_newuserdatadtor(lua_State* L, size_t sz, void (*dtor)(void*)) { luaC_checkGC(L); luaC_checkthreadsleep(L); - Udata* u = luaU_newudata(L, sz + sizeof(dtor), UTAG_IDTOR); + size_t as = sz + sizeof(dtor); + if (as < sizeof(dtor)) + as = SIZE_MAX; // Will cause a memory error in luaU_newudata. + Udata* u = luaU_newudata(L, as, UTAG_IDTOR); memcpy(&u->data + sz, &dtor, sizeof(dtor)); setuvalue(L, L->top, u); api_incr_top(L); diff --git a/tests/Conformance.test.cpp b/tests/Conformance.test.cpp index e07ba12a..17c7ac58 100644 --- a/tests/Conformance.test.cpp +++ b/tests/Conformance.test.cpp @@ -716,6 +716,23 @@ TEST_CASE("Reference") CHECK(dtorhits == 2); } +TEST_CASE("NewUserdataOverflow") +{ + StateRef globalState(luaL_newstate(), lua_close); + lua_State* L = globalState.get(); + + lua_pushcfunction(L, [](lua_State* L1) { + // The following userdata request might cause an overflow. + lua_newuserdatadtor(L1, SIZE_MAX, [](void* d){}); + // The overflow might segfault in the following call. + lua_getmetatable(L1, -1); + return 0; + }, "PCall"); + + CHECK(lua_pcall(L, 0, 0, 0) == LUA_ERRRUN); + CHECK(strcmp(lua_tostring(L, -1), "memory allocation error: block too big") == 0); +} + TEST_CASE("ApiTables") { StateRef globalState(luaL_newstate(), lua_close); From 0e118b54bbb1d173fe96dffe009a80cfb846d6ea Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Wed, 17 Aug 2022 10:02:08 -0700 Subject: [PATCH 343/399] Update generalized-iteration.md (#640) Use a more precise pseudocode snippet for __iter handling which matches runtime semantics --- rfcs/generalized-iteration.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/rfcs/generalized-iteration.md b/rfcs/generalized-iteration.md index 99671090..c28156ff 100644 --- a/rfcs/generalized-iteration.md +++ b/rfcs/generalized-iteration.md @@ -51,8 +51,10 @@ end To support self-iterating objects, we modify the iteration protocol as follows: instead of simply expanding the result of expression `iter` into three variables (`gen`, `state` and `index`), we check if the first result has an `__iter` metamethod (which can be the case if it's a table, userdata or another composite object (e.g. a record in the future). If it does, the metamethod is called with `gen` as the first argument, and the returned three values replace `gen`/`state`/`index`. This happens *before* the loop: ```lua -if getmetatable(gen) and getmetatable(gen).__iter then - gen, state, index = getmetatable(gen).__iter(gen) +local genmt = rawgetmetatable(gen) -- pseudo code for getmetatable that bypasses __metatable +local iterf = genmt and rawget(genmt, "__iter") +if iterf then + gen, state, index = iterf(gen) end ``` From b3e6dcecfdd56c056b12cdbcc48ec93a33aa49f3 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Thu, 18 Aug 2022 14:04:33 -0700 Subject: [PATCH 344/399] Sync to upstream/release/541 --- Analysis/include/Luau/Anyification.h | 38 + Analysis/include/Luau/Constraint.h | 4 +- .../include/Luau/ConstraintGraphBuilder.h | 7 +- Analysis/include/Luau/ConstraintSolver.h | 11 +- Analysis/include/Luau/Module.h | 2 + Analysis/include/Luau/Normalize.h | 20 +- Analysis/include/Luau/TypeInfer.h | 63 +- Analysis/include/Luau/TypeUtils.h | 7 +- Analysis/include/Luau/TypedAllocator.h | 4 +- Analysis/include/Luau/Unifier.h | 6 +- Analysis/include/Luau/VisitTypeVar.h | 22 +- Analysis/src/Anyification.cpp | 96 +++ Analysis/src/Autocomplete.cpp | 45 +- Analysis/src/Constraint.cpp | 3 +- Analysis/src/ConstraintGraphBuilder.cpp | 98 ++- Analysis/src/ConstraintSolver.cpp | 42 +- Analysis/src/Frontend.cpp | 3 + Analysis/src/Module.cpp | 10 +- Analysis/src/Normalize.cpp | 67 +- Analysis/src/Quantify.cpp | 42 +- Analysis/src/TypeChecker2.cpp | 741 +++++++++++++----- Analysis/src/TypeInfer.cpp | 297 +++---- Analysis/src/TypeUtils.cpp | 111 ++- Analysis/src/TypeVar.cpp | 14 +- Analysis/src/TypedAllocator.cpp | 50 +- Analysis/src/Unifier.cpp | 10 +- Ast/include/Luau/Parser.h | 31 +- Ast/src/Parser.cpp | 322 ++++---- Common/include/Luau/Bytecode.h | 3 + Compiler/src/BuiltinFolding.cpp | 11 +- Compiler/src/Compiler.cpp | 108 ++- Sources.cmake | 4 +- VM/src/lapi.cpp | 4 +- VM/src/lbuiltins.cpp | 91 ++- VM/src/lfunc.cpp | 13 +- VM/src/lfunc.h | 1 + VM/src/lgcdebug.cpp | 57 +- tests/Autocomplete.test.cpp | 23 + tests/Compiler.test.cpp | 116 ++- tests/Conformance.test.cpp | 17 + tests/Fixture.cpp | 3 +- tests/Fixture.h | 1 + tests/Normalize.test.cpp | 18 +- tests/Parser.test.cpp | 1 - tests/TypeInfer.functions.test.cpp | 2 - tests/TypeInfer.modules.test.cpp | 29 + tests/TypeInfer.provisional.test.cpp | 1 - tests/TypeInfer.refinements.test.cpp | 15 +- tests/TypeInfer.tables.test.cpp | 4 - tests/TypeInfer.tryUnify.test.cpp | 3 +- tests/TypeVar.test.cpp | 6 + tests/conformance/bitwise.lua | 4 + tools/faillist.txt | 31 - tools/heapgraph.py | 12 +- tools/heapstat.py | 3 +- tools/perfgraph.py | 23 +- 56 files changed, 1820 insertions(+), 950 deletions(-) create mode 100644 Analysis/include/Luau/Anyification.h create mode 100644 Analysis/src/Anyification.cpp diff --git a/Analysis/include/Luau/Anyification.h b/Analysis/include/Luau/Anyification.h new file mode 100644 index 00000000..ee8d6689 --- /dev/null +++ b/Analysis/include/Luau/Anyification.h @@ -0,0 +1,38 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details + +#pragma once + +#include "Luau/NotNull.h" +#include "Luau/Substitution.h" +#include "Luau/TypeVar.h" + +#include + +namespace Luau +{ + +struct TypeArena; +struct Scope; +struct InternalErrorReporter; +using ScopePtr = std::shared_ptr; + +// A substitution which replaces free types by any +struct Anyification : Substitution +{ + Anyification(TypeArena* arena, const ScopePtr& scope, InternalErrorReporter* iceHandler, TypeId anyType, TypePackId anyTypePack); + NotNull scope; + InternalErrorReporter* iceHandler; + + TypeId anyType; + TypePackId anyTypePack; + bool normalizationTooComplex = false; + bool isDirty(TypeId ty) override; + bool isDirty(TypePackId tp) override; + TypeId clean(TypeId ty) override; + TypePackId clean(TypePackId tp) override; + + bool ignoreChildren(TypeId ty) override; + bool ignoreChildren(TypePackId ty) override; +}; + +} \ No newline at end of file diff --git a/Analysis/include/Luau/Constraint.h b/Analysis/include/Luau/Constraint.h index 5b737146..ce90f2c5 100644 --- a/Analysis/include/Luau/Constraint.h +++ b/Analysis/include/Luau/Constraint.h @@ -40,7 +40,6 @@ struct GeneralizationConstraint { TypeId generalizedType; TypeId sourceType; - Scope* scope; }; // subType ~ inst superType @@ -85,13 +84,14 @@ using ConstraintPtr = std::unique_ptr; struct Constraint { - explicit Constraint(ConstraintV&& c); + Constraint(ConstraintV&& c, NotNull scope); Constraint(const Constraint&) = delete; Constraint& operator=(const Constraint&) = delete; ConstraintV c; std::vector> dependencies; + NotNull scope; }; inline Constraint& asMutable(const Constraint& c) diff --git a/Analysis/include/Luau/ConstraintGraphBuilder.h b/Analysis/include/Luau/ConstraintGraphBuilder.h index 41d14326..0e41e1ee 100644 --- a/Analysis/include/Luau/ConstraintGraphBuilder.h +++ b/Analysis/include/Luau/ConstraintGraphBuilder.h @@ -72,10 +72,10 @@ struct ConstraintGraphBuilder /** * Fabricates a scope that is a child of another scope. - * @param location the lexical extent of the scope in the source code. + * @param node the lexical node that the scope belongs to. * @param parent the parent scope of the new scope. Must not be null. */ - ScopePtr childScope(Location location, const ScopePtr& parent); + ScopePtr childScope(AstNode* node, const ScopePtr& parent); /** * Adds a new constraint with no dependencies to a given scope. @@ -105,10 +105,12 @@ struct ConstraintGraphBuilder void visit(const ScopePtr& scope, AstStatLocal* local); void visit(const ScopePtr& scope, AstStatFor* for_); void visit(const ScopePtr& scope, AstStatWhile* while_); + void visit(const ScopePtr& scope, AstStatRepeat* repeat); void visit(const ScopePtr& scope, AstStatLocalFunction* function); void visit(const ScopePtr& scope, AstStatFunction* function); void visit(const ScopePtr& scope, AstStatReturn* ret); void visit(const ScopePtr& scope, AstStatAssign* assign); + void visit(const ScopePtr& scope, AstStatCompoundAssign* assign); void visit(const ScopePtr& scope, AstStatIf* ifStatement); void visit(const ScopePtr& scope, AstStatTypeAlias* alias); void visit(const ScopePtr& scope, AstStatDeclareGlobal* declareGlobal); @@ -133,6 +135,7 @@ struct ConstraintGraphBuilder TypeId check(const ScopePtr& scope, AstExprIndexExpr* indexExpr); TypeId check(const ScopePtr& scope, AstExprUnary* unary); TypeId check(const ScopePtr& scope, AstExprBinary* binary); + TypeId check(const ScopePtr& scope, AstExprIfElse* ifElse); TypeId check(const ScopePtr& scope, AstExprTypeAssertion* typeAssert); struct FunctionSignature diff --git a/Analysis/include/Luau/ConstraintSolver.h b/Analysis/include/Luau/ConstraintSolver.h index 9cc0e4cb..661d120e 100644 --- a/Analysis/include/Luau/ConstraintSolver.h +++ b/Analysis/include/Luau/ConstraintSolver.h @@ -60,6 +60,9 @@ struct ConstraintSolver // Memoized instantiations of type aliases. DenseHashMap instantiatedAliases{{}}; + // Recorded errors that take place within the solver. + ErrorVec errors; + ConstraintSolverLogger logger; explicit ConstraintSolver(TypeArena* arena, NotNull rootScope); @@ -115,7 +118,7 @@ struct ConstraintSolver * @param subType the sub-type to unify. * @param superType the super-type to unify. */ - void unify(TypeId subType, TypeId superType); + void unify(TypeId subType, TypeId superType, NotNull scope); /** * Creates a new Unifier and performs a single unification operation. Commits @@ -123,13 +126,15 @@ struct ConstraintSolver * @param subPack the sub-type pack to unify. * @param superPack the super-type pack to unify. */ - void unify(TypePackId subPack, TypePackId superPack); + void unify(TypePackId subPack, TypePackId superPack, NotNull scope); /** Pushes a new solver constraint to the solver. * @param cv the body of the constraint. **/ - void pushConstraint(ConstraintV cv); + void pushConstraint(ConstraintV cv, NotNull scope); + void reportError(TypeErrorData&& data, const Location& location); + void reportError(TypeError e); private: /** * Marks a constraint as being blocked on a type or type pack. The constraint diff --git a/Analysis/include/Luau/Module.h b/Analysis/include/Luau/Module.h index 6f4c6098..bec51b81 100644 --- a/Analysis/include/Luau/Module.h +++ b/Analysis/include/Luau/Module.h @@ -77,6 +77,8 @@ struct Module DenseHashMap astOverloadResolvedTypes{nullptr}; DenseHashMap astResolvedTypes{nullptr}; DenseHashMap astResolvedTypePacks{nullptr}; + // Map AST nodes to the scope they create. Cannot be NotNull because we need a sentinel value for the map. + DenseHashMap astScopes{nullptr}; std::unordered_map declaredGlobals; ErrorVec errors; diff --git a/Analysis/include/Luau/Normalize.h b/Analysis/include/Luau/Normalize.h index f5fd9886..78b241e4 100644 --- a/Analysis/include/Luau/Normalize.h +++ b/Analysis/include/Luau/Normalize.h @@ -1,20 +1,28 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details -#include "Luau/Substitution.h" -#include "Luau/TypeVar.h" #include "Luau/Module.h" +#include "Luau/NotNull.h" +#include "Luau/TypeVar.h" + +#include namespace Luau { struct InternalErrorReporter; +struct Module; +struct Scope; -bool isSubtype(TypeId subTy, TypeId superTy, InternalErrorReporter& ice); -bool isSubtype(TypePackId subTy, TypePackId superTy, InternalErrorReporter& ice); +using ModulePtr = std::shared_ptr; -std::pair normalize(TypeId ty, TypeArena& arena, InternalErrorReporter& ice); +bool isSubtype(TypeId subTy, TypeId superTy, NotNull scope, InternalErrorReporter& ice); +bool isSubtype(TypePackId subTy, TypePackId superTy, NotNull scope, InternalErrorReporter& ice); + +std::pair normalize(TypeId ty, NotNull scope, TypeArena& arena, InternalErrorReporter& ice); +std::pair normalize(TypeId ty, NotNull module, InternalErrorReporter& ice); std::pair normalize(TypeId ty, const ModulePtr& module, InternalErrorReporter& ice); -std::pair normalize(TypePackId ty, TypeArena& arena, InternalErrorReporter& ice); +std::pair normalize(TypePackId ty, NotNull scope, TypeArena& arena, InternalErrorReporter& ice); +std::pair normalize(TypePackId ty, NotNull module, InternalErrorReporter& ice); std::pair normalize(TypePackId ty, const ModulePtr& module, InternalErrorReporter& ice); } // namespace Luau diff --git a/Analysis/include/Luau/TypeInfer.h b/Analysis/include/Luau/TypeInfer.h index 5c55ddb0..80f9085f 100644 --- a/Analysis/include/Luau/TypeInfer.h +++ b/Analysis/include/Luau/TypeInfer.h @@ -1,6 +1,7 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #pragma once +#include "Luau/Anyification.h" #include "Luau/Predicate.h" #include "Luau/Error.h" #include "Luau/Module.h" @@ -36,40 +37,6 @@ const AstStat* getFallthrough(const AstStat* node); struct UnifierOptions; struct Unifier; -// A substitution which replaces free types by any -struct Anyification : Substitution -{ - Anyification(TypeArena* arena, InternalErrorReporter* iceHandler, TypeId anyType, TypePackId anyTypePack) - : Substitution(TxnLog::empty(), arena) - , iceHandler(iceHandler) - , anyType(anyType) - , anyTypePack(anyTypePack) - { - } - - InternalErrorReporter* iceHandler; - - TypeId anyType; - TypePackId anyTypePack; - bool normalizationTooComplex = false; - bool isDirty(TypeId ty) override; - bool isDirty(TypePackId tp) override; - TypeId clean(TypeId ty) override; - TypePackId clean(TypePackId tp) override; - - bool ignoreChildren(TypeId ty) override - { - if (FFlag::LuauClassTypeVarsInSubstitution && get(ty)) - return true; - - return ty->persistent; - } - bool ignoreChildren(TypePackId ty) override - { - return ty->persistent; - } -}; - struct GenericTypeDefinitions { std::vector genericTypes; @@ -196,32 +163,32 @@ struct TypeChecker /** Attempt to unify the types. * Treat any failures as type errors in the final typecheck report. */ - bool unify(TypeId subTy, TypeId superTy, const Location& location); - bool unify(TypeId subTy, TypeId superTy, const Location& location, const UnifierOptions& options); - bool unify(TypePackId subTy, TypePackId superTy, const Location& location, CountMismatch::Context ctx = CountMismatch::Context::Arg); + bool unify(TypeId subTy, TypeId superTy, const ScopePtr& scope, const Location& location); + bool unify(TypeId subTy, TypeId superTy, const ScopePtr& scope, const Location& location, const UnifierOptions& options); + bool unify(TypePackId subTy, TypePackId superTy, const ScopePtr& scope, const Location& location, CountMismatch::Context ctx = CountMismatch::Context::Arg); /** Attempt to unify the types. * If this fails, and the subTy type can be instantiated, do so and try unification again. */ - bool unifyWithInstantiationIfNeeded(const ScopePtr& scope, TypeId subTy, TypeId superTy, const Location& location); - void unifyWithInstantiationIfNeeded(const ScopePtr& scope, TypeId subTy, TypeId superTy, Unifier& state); + bool unifyWithInstantiationIfNeeded(TypeId subTy, TypeId superTy, const ScopePtr& scope, const Location& location); + void unifyWithInstantiationIfNeeded(TypeId subTy, TypeId superTy, const ScopePtr& scope, Unifier& state); /** Attempt to unify. * If there are errors, undo everything and return the errors. * If there are no errors, commit and return an empty error vector. */ template - ErrorVec tryUnify_(Id subTy, Id superTy, const Location& location); - ErrorVec tryUnify(TypeId subTy, TypeId superTy, const Location& location); - ErrorVec tryUnify(TypePackId subTy, TypePackId superTy, const Location& location); + ErrorVec tryUnify_(Id subTy, Id superTy, const ScopePtr& scope, const Location& location); + ErrorVec tryUnify(TypeId subTy, TypeId superTy, const ScopePtr& scope, const Location& location); + ErrorVec tryUnify(TypePackId subTy, TypePackId superTy, const ScopePtr& scope, const Location& location); // Test whether the two type vars unify. Never commits the result. template - ErrorVec canUnify_(Id subTy, Id superTy, const Location& location); - ErrorVec canUnify(TypeId subTy, TypeId superTy, const Location& location); - ErrorVec canUnify(TypePackId subTy, TypePackId superTy, const Location& location); + ErrorVec canUnify_(Id subTy, Id superTy, const ScopePtr& scope, const Location& location); + ErrorVec canUnify(TypeId subTy, TypeId superTy, const ScopePtr& scope, const Location& location); + ErrorVec canUnify(TypePackId subTy, TypePackId superTy, const ScopePtr& scope, const Location& location); - void unifyLowerBound(TypePackId subTy, TypePackId superTy, TypeLevel demotedLevel, const Location& location); + void unifyLowerBound(TypePackId subTy, TypePackId superTy, TypeLevel demotedLevel, const ScopePtr& scope, const Location& location); std::optional findMetatableEntry(TypeId type, std::string entry, const Location& location, bool addErrors); std::optional findTablePropertyRespectingMeta(TypeId lhsType, Name name, const Location& location, bool addErrors); @@ -290,7 +257,7 @@ private: void reportErrorCodeTooComplex(const Location& location); private: - Unifier mkUnifier(const Location& location); + Unifier mkUnifier(const ScopePtr& scope, const Location& location); // These functions are only safe to call when we are in the process of typechecking a module. @@ -312,7 +279,7 @@ public: std::pair, bool> pickTypesFromSense(TypeId type, bool sense); private: - TypeId unionOfTypes(TypeId a, TypeId b, const Location& location, bool unifyFreeTypes = true); + TypeId unionOfTypes(TypeId a, TypeId b, const ScopePtr& scope, const Location& location, bool unifyFreeTypes = true); // ex // TypeId id = addType(FreeTypeVar()); diff --git a/Analysis/include/Luau/TypeUtils.h b/Analysis/include/Luau/TypeUtils.h index 42c1bc0b..0aff5a7d 100644 --- a/Analysis/include/Luau/TypeUtils.h +++ b/Analysis/include/Luau/TypeUtils.h @@ -13,7 +13,10 @@ namespace Luau using ScopePtr = std::shared_ptr; -std::optional findMetatableEntry(ErrorVec& errors, TypeId type, std::string entry, Location location); -std::optional findTablePropertyRespectingMeta(ErrorVec& errors, TypeId ty, Name name, Location location); +std::optional findMetatableEntry(ErrorVec& errors, TypeId type, const std::string& entry, Location location); +std::optional findTablePropertyRespectingMeta(ErrorVec& errors, TypeId ty, const std::string& name, Location location); +std::optional getIndexTypeFromType( + const ScopePtr& scope, ErrorVec& errors, TypeArena* arena, TypeId type, const std::string& prop, const Location& location, bool addErrors, + InternalErrorReporter& handle); } // namespace Luau diff --git a/Analysis/include/Luau/TypedAllocator.h b/Analysis/include/Luau/TypedAllocator.h index f67e3d8e..a5dd17b6 100644 --- a/Analysis/include/Luau/TypedAllocator.h +++ b/Analysis/include/Luau/TypedAllocator.h @@ -10,7 +10,7 @@ namespace Luau { void* pagedAllocate(size_t size); -void pagedDeallocate(void* ptr); +void pagedDeallocate(void* ptr, size_t size); void pagedFreeze(void* ptr, size_t size); void pagedUnfreeze(void* ptr, size_t size); @@ -113,7 +113,7 @@ private: for (size_t i = 0; i < blockSize; ++i) block[i].~T(); - pagedDeallocate(block); + pagedDeallocate(block, kBlockSizeBytes); } stuff.clear(); diff --git a/Analysis/include/Luau/Unifier.h b/Analysis/include/Luau/Unifier.h index 9fa29076..312b0584 100644 --- a/Analysis/include/Luau/Unifier.h +++ b/Analysis/include/Luau/Unifier.h @@ -3,9 +3,10 @@ #include "Luau/Error.h" #include "Luau/Location.h" +#include "Luau/Scope.h" #include "Luau/TxnLog.h" -#include "Luau/TypeInfer.h" #include "Luau/TypeArena.h" +#include "Luau/TypeInfer.h" #include "Luau/UnifierSharedState.h" #include @@ -48,6 +49,7 @@ struct Unifier TypeArena* const types; Mode mode; + NotNull scope; // const Scope maybe TxnLog log; ErrorVec errors; Location location; @@ -57,7 +59,7 @@ struct Unifier UnifierSharedState& sharedState; - Unifier(TypeArena* types, Mode mode, const Location& location, Variance variance, UnifierSharedState& sharedState, TxnLog* parentLog = nullptr); + Unifier(TypeArena* types, Mode mode, NotNull scope, const Location& location, Variance variance, UnifierSharedState& sharedState, TxnLog* parentLog = nullptr); // Test whether the two type vars unify. Never commits the result. ErrorVec canUnify(TypeId subTy, TypeId superTy); diff --git a/Analysis/include/Luau/VisitTypeVar.h b/Analysis/include/Luau/VisitTypeVar.h index 7e5d71d6..9d7fa9fe 100644 --- a/Analysis/include/Luau/VisitTypeVar.h +++ b/Analysis/include/Luau/VisitTypeVar.h @@ -69,12 +69,14 @@ struct GenericTypeVarVisitor using Set = S; Set seen; + bool skipBoundTypes = false; int recursionCounter = 0; GenericTypeVarVisitor() = default; - explicit GenericTypeVarVisitor(Set seen) + explicit GenericTypeVarVisitor(Set seen, bool skipBoundTypes = false) : seen(std::move(seen)) + , skipBoundTypes(skipBoundTypes) { } @@ -199,7 +201,9 @@ struct GenericTypeVarVisitor if (auto btv = get(ty)) { - if (visit(ty, *btv)) + if (skipBoundTypes) + traverse(btv->boundTo); + else if (visit(ty, *btv)) traverse(btv->boundTo); } else if (auto ftv = get(ty)) @@ -229,7 +233,11 @@ struct GenericTypeVarVisitor else if (auto ttv = get(ty)) { // Some visitors want to see bound tables, that's why we traverse the original type - if (visit(ty, *ttv)) + if (skipBoundTypes && ttv->boundTo) + { + traverse(*ttv->boundTo); + } + else if (visit(ty, *ttv)) { if (ttv->boundTo) { @@ -394,13 +402,17 @@ struct GenericTypeVarVisitor */ struct TypeVarVisitor : GenericTypeVarVisitor> { + explicit TypeVarVisitor(bool skipBoundTypes = false) + : GenericTypeVarVisitor{{}, skipBoundTypes} + { + } }; /// Visit each type under a given type. Each type will only be checked once even if there are multiple paths to it. struct TypeVarOnceVisitor : GenericTypeVarVisitor> { - TypeVarOnceVisitor() - : GenericTypeVarVisitor{DenseHashSet{nullptr}} + explicit TypeVarOnceVisitor(bool skipBoundTypes = false) + : GenericTypeVarVisitor{DenseHashSet{nullptr}, skipBoundTypes} { } }; diff --git a/Analysis/src/Anyification.cpp b/Analysis/src/Anyification.cpp new file mode 100644 index 00000000..b6e58009 --- /dev/null +++ b/Analysis/src/Anyification.cpp @@ -0,0 +1,96 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details + +#include "Luau/Anyification.h" + +#include "Luau/Common.h" +#include "Luau/Normalize.h" +#include "Luau/TxnLog.h" + +LUAU_FASTFLAG(LuauClassTypeVarsInSubstitution) + +namespace Luau +{ + +Anyification::Anyification(TypeArena* arena, const ScopePtr& scope, InternalErrorReporter* iceHandler, TypeId anyType, TypePackId anyTypePack) + : Substitution(TxnLog::empty(), arena) + , scope(NotNull{scope.get()}) + , iceHandler(iceHandler) + , anyType(anyType) + , anyTypePack(anyTypePack) +{ +} + +bool Anyification::isDirty(TypeId ty) +{ + if (ty->persistent) + return false; + + if (const TableTypeVar* ttv = log->getMutable(ty)) + return (ttv->state == TableState::Free || ttv->state == TableState::Unsealed); + else if (log->getMutable(ty)) + return true; + else if (get(ty)) + return true; + else + return false; +} + +bool Anyification::isDirty(TypePackId tp) +{ + if (tp->persistent) + return false; + + if (log->getMutable(tp)) + return true; + else + return false; +} + +TypeId Anyification::clean(TypeId ty) +{ + LUAU_ASSERT(isDirty(ty)); + if (const TableTypeVar* ttv = log->getMutable(ty)) + { + TableTypeVar clone = TableTypeVar{ttv->props, ttv->indexer, ttv->level, TableState::Sealed}; + clone.definitionModuleName = ttv->definitionModuleName; + clone.name = ttv->name; + clone.syntheticName = ttv->syntheticName; + clone.tags = ttv->tags; + TypeId res = addType(std::move(clone)); + asMutable(res)->normal = ty->normal; + return res; + } + else if (auto ctv = get(ty)) + { + std::vector copy = ctv->parts; + for (TypeId& ty : copy) + ty = replace(ty); + TypeId res = copy.size() == 1 ? copy[0] : addType(UnionTypeVar{std::move(copy)}); + auto [t, ok] = normalize(res, scope, *arena, *iceHandler); + if (!ok) + normalizationTooComplex = true; + return t; + } + else + return anyType; +} + +TypePackId Anyification::clean(TypePackId tp) +{ + LUAU_ASSERT(isDirty(tp)); + return anyTypePack; +} + +bool Anyification::ignoreChildren(TypeId ty) +{ + if (FFlag::LuauClassTypeVarsInSubstitution && get(ty)) + return true; + + return ty->persistent; +} +bool Anyification::ignoreChildren(TypePackId ty) +{ + return ty->persistent; +} + +} diff --git a/Analysis/src/Autocomplete.cpp b/Analysis/src/Autocomplete.cpp index 8d5cc723..5c484899 100644 --- a/Analysis/src/Autocomplete.cpp +++ b/Analysis/src/Autocomplete.cpp @@ -14,6 +14,8 @@ LUAU_FASTFLAG(LuauSelfCallAutocompleteFix3) +LUAU_FASTFLAGVARIABLE(LuauAutocompleteFixGlobalOrder, false) + static const std::unordered_set kStatementStartingKeywords = { "while", "if", "local", "repeat", "function", "do", "for", "return", "break", "continue", "type", "export"}; @@ -135,11 +137,11 @@ static std::optional findExpectedTypeAt(const Module& module, AstNode* n return *it; } -static bool checkTypeMatch(TypeArena* typeArena, TypeId subTy, TypeId superTy) +static bool checkTypeMatch(TypeId subTy, TypeId superTy, NotNull scope, TypeArena* typeArena) { InternalErrorReporter iceReporter; UnifierSharedState unifierState(&iceReporter); - Unifier unifier(typeArena, Mode::Strict, Location(), Variance::Covariant, unifierState); + Unifier unifier(typeArena, Mode::Strict, scope, Location(), Variance::Covariant, unifierState); return unifier.canUnify(subTy, superTy).empty(); } @@ -148,12 +150,14 @@ static TypeCorrectKind checkTypeCorrectKind(const Module& module, TypeArena* typ { ty = follow(ty); - auto canUnify = [&typeArena](TypeId subTy, TypeId superTy) { + NotNull moduleScope{module.getModuleScope().get()}; + + auto canUnify = [&typeArena, moduleScope](TypeId subTy, TypeId superTy) { LUAU_ASSERT(!FFlag::LuauSelfCallAutocompleteFix3); InternalErrorReporter iceReporter; UnifierSharedState unifierState(&iceReporter); - Unifier unifier(typeArena, Mode::Strict, Location(), Variance::Covariant, unifierState); + Unifier unifier(typeArena, Mode::Strict, moduleScope, Location(), Variance::Covariant, unifierState); unifier.tryUnify(subTy, superTy); bool ok = unifier.errors.empty(); @@ -167,11 +171,11 @@ static TypeCorrectKind checkTypeCorrectKind(const Module& module, TypeArena* typ TypeId expectedType = follow(*typeAtPosition); - auto checkFunctionType = [typeArena, &canUnify, &expectedType](const FunctionTypeVar* ftv) { + auto checkFunctionType = [typeArena, moduleScope, &canUnify, &expectedType](const FunctionTypeVar* ftv) { if (FFlag::LuauSelfCallAutocompleteFix3) { if (std::optional firstRetTy = first(ftv->retTypes)) - return checkTypeMatch(typeArena, *firstRetTy, expectedType); + return checkTypeMatch(*firstRetTy, expectedType, moduleScope, typeArena); return false; } @@ -210,7 +214,7 @@ static TypeCorrectKind checkTypeCorrectKind(const Module& module, TypeArena* typ } if (FFlag::LuauSelfCallAutocompleteFix3) - return checkTypeMatch(typeArena, ty, expectedType) ? TypeCorrectKind::Correct : TypeCorrectKind::None; + return checkTypeMatch(ty, expectedType, NotNull{module.getModuleScope().get()}, typeArena) ? TypeCorrectKind::Correct : TypeCorrectKind::None; else return canUnify(ty, expectedType) ? TypeCorrectKind::Correct : TypeCorrectKind::None; } @@ -268,7 +272,7 @@ static void autocompleteProps(const Module& module, TypeArena* typeArena, TypeId return colonIndex; } }; - auto isWrongIndexer = [typeArena, rootTy, indexType](Luau::TypeId type) { + auto isWrongIndexer = [typeArena, &module, rootTy, indexType](Luau::TypeId type) { LUAU_ASSERT(FFlag::LuauSelfCallAutocompleteFix3); if (indexType == PropIndexType::Key) @@ -276,7 +280,7 @@ static void autocompleteProps(const Module& module, TypeArena* typeArena, TypeId bool calledWithSelf = indexType == PropIndexType::Colon; - auto isCompatibleCall = [typeArena, rootTy, calledWithSelf](const FunctionTypeVar* ftv) { + auto isCompatibleCall = [typeArena, &module, rootTy, calledWithSelf](const FunctionTypeVar* ftv) { // Strong match with definition is a success if (calledWithSelf == ftv->hasSelf) return true; @@ -289,7 +293,7 @@ static void autocompleteProps(const Module& module, TypeArena* typeArena, TypeId // When called with '.', but declared with 'self', it is considered invalid if first argument is compatible if (std::optional firstArgTy = first(ftv->argTypes)) { - if (checkTypeMatch(typeArena, rootTy, *firstArgTy)) + if (checkTypeMatch(rootTy, *firstArgTy, NotNull{module.getModuleScope().get()}, typeArena)) return calledWithSelf; } @@ -1073,10 +1077,21 @@ T* extractStat(const std::vector& ancestry) return nullptr; } -static bool isBindingLegalAtCurrentPosition(const Binding& binding, Position pos) +static bool isBindingLegalAtCurrentPosition(const Symbol& symbol, const Binding& binding, Position pos) { - // Default Location used for global bindings, which are always legal. - return binding.location == Location() || binding.location.end < pos; + if (FFlag::LuauAutocompleteFixGlobalOrder) + { + if (symbol.local) + return binding.location.end < pos; + + // Builtin globals have an empty location; for defined globals, we want pos to be outside of the definition range to suggest it + return binding.location == Location() || !binding.location.containsClosed(pos); + } + else + { + // Default Location used for global bindings, which are always legal. + return binding.location == Location() || binding.location.end < pos; + } } static AutocompleteEntryMap autocompleteStatement( @@ -1097,7 +1112,7 @@ static AutocompleteEntryMap autocompleteStatement( { for (const auto& [name, binding] : scope->bindings) { - if (!isBindingLegalAtCurrentPosition(binding, position)) + if (!isBindingLegalAtCurrentPosition(name, binding, position)) continue; std::string n = toString(name); @@ -1225,7 +1240,7 @@ static AutocompleteContext autocompleteExpression(const SourceModule& sourceModu { for (const auto& [name, binding] : scope->bindings) { - if (!isBindingLegalAtCurrentPosition(binding, position)) + if (!isBindingLegalAtCurrentPosition(name, binding, position)) continue; if (isBeingDefined(ancestry, name)) diff --git a/Analysis/src/Constraint.cpp b/Analysis/src/Constraint.cpp index 64e3a666..d272c027 100644 --- a/Analysis/src/Constraint.cpp +++ b/Analysis/src/Constraint.cpp @@ -5,8 +5,9 @@ namespace Luau { -Constraint::Constraint(ConstraintV&& c) +Constraint::Constraint(ConstraintV&& c, NotNull scope) : c(std::move(c)) + , scope(scope) { } diff --git a/Analysis/src/ConstraintGraphBuilder.cpp b/Analysis/src/ConstraintGraphBuilder.cpp index 2c36d422..c1e54df2 100644 --- a/Analysis/src/ConstraintGraphBuilder.cpp +++ b/Analysis/src/ConstraintGraphBuilder.cpp @@ -2,6 +2,7 @@ #include "Luau/ConstraintGraphBuilder.h" #include "Luau/Ast.h" +#include "Luau/Common.h" #include "Luau/Constraint.h" #include "Luau/RecursionCounter.h" #include "Luau/ToString.h" @@ -26,6 +27,7 @@ ConstraintGraphBuilder::ConstraintGraphBuilder( , globalScope(globalScope) { LUAU_ASSERT(arena); + LUAU_ASSERT(module); } TypeId ConstraintGraphBuilder::freshType(const ScopePtr& scope) @@ -39,20 +41,22 @@ TypePackId ConstraintGraphBuilder::freshTypePack(const ScopePtr& scope) return arena->addTypePack(TypePackVar{std::move(f)}); } -ScopePtr ConstraintGraphBuilder::childScope(Location location, const ScopePtr& parent) +ScopePtr ConstraintGraphBuilder::childScope(AstNode* node, const ScopePtr& parent) { auto scope = std::make_shared(parent); - scopes.emplace_back(location, scope); + scopes.emplace_back(node->location, scope); scope->returnType = parent->returnType; - parent->children.push_back(NotNull(scope.get())); + + parent->children.push_back(NotNull{scope.get()}); + module->astScopes[node] = scope.get(); return scope; } void ConstraintGraphBuilder::addConstraint(const ScopePtr& scope, ConstraintV cv) { - scope->constraints.emplace_back(new Constraint{std::move(cv)}); + scope->constraints.emplace_back(new Constraint{std::move(cv), NotNull{scope.get()}}); } void ConstraintGraphBuilder::addConstraint(const ScopePtr& scope, std::unique_ptr c) @@ -67,6 +71,7 @@ void ConstraintGraphBuilder::visit(AstStatBlock* block) ScopePtr scope = std::make_shared(globalScope); rootScope = scope.get(); scopes.emplace_back(block->location, scope); + module->astScopes[block] = NotNull{scope.get()}; rootScope->returnType = freshTypePack(scope); @@ -115,7 +120,7 @@ void ConstraintGraphBuilder::visitBlockWithoutChildScope(const ScopePtr& scope, ScopePtr defnScope = scope; if (hasGenerics) { - defnScope = childScope(alias->location, scope); + defnScope = childScope(alias, scope); } TypeId initialType = freshType(scope); @@ -155,6 +160,8 @@ void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStat* stat) visit(scope, s); else if (auto s = stat->as()) visit(scope, s); + else if (auto s = stat->as()) + visit(scope, s); else if (auto f = stat->as()) visit(scope, f); else if (auto f = stat->as()) @@ -163,6 +170,8 @@ void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStat* stat) visit(scope, r); else if (auto a = stat->as()) visit(scope, a); + else if (auto a = stat->as()) + visit(scope, a); else if (auto e = stat->as()) checkPack(scope, e->expr); else if (auto i = stat->as()) @@ -241,7 +250,7 @@ void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatFor* for_) checkNumber(for_->to); checkNumber(for_->step); - ScopePtr forScope = childScope(for_->location, scope); + ScopePtr forScope = childScope(for_, scope); forScope->bindings[for_->var] = Binding{singletonTypes.numberType, for_->var->location}; visit(forScope, for_->body); @@ -251,11 +260,22 @@ void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatWhile* while_) { check(scope, while_->condition); - ScopePtr whileScope = childScope(while_->location, scope); + ScopePtr whileScope = childScope(while_, scope); visit(whileScope, while_->body); } +void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatRepeat* repeat) +{ + ScopePtr repeatScope = childScope(repeat, scope); + + visit(repeatScope, repeat->body); + + // The condition does indeed have access to bindings from within the body of + // the loop. + check(repeatScope, repeat->condition); +} + void addConstraints(Constraint* constraint, NotNull scope) { scope->constraints.reserve(scope->constraints.size() + scope->constraints.size()); @@ -286,8 +306,8 @@ void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatLocalFunction* checkFunctionBody(sig.bodyScope, function->func); - std::unique_ptr c{ - new Constraint{GeneralizationConstraint{functionType, sig.signature, sig.signatureScope ? sig.signatureScope.get() : sig.bodyScope.get()}}}; + NotNull constraintScope{sig.signatureScope ? sig.signatureScope.get() : sig.bodyScope.get()}; + std::unique_ptr c = std::make_unique(GeneralizationConstraint{functionType, sig.signature}, constraintScope); addConstraints(c.get(), NotNull(sig.bodyScope.get())); addConstraint(scope, std::move(c)); @@ -356,8 +376,8 @@ void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatFunction* funct checkFunctionBody(sig.bodyScope, function->func); - std::unique_ptr c{ - new Constraint{GeneralizationConstraint{functionType, sig.signature, sig.signatureScope ? sig.signatureScope.get() : sig.bodyScope.get()}}}; + NotNull constraintScope{sig.signatureScope ? sig.signatureScope.get() : sig.bodyScope.get()}; + std::unique_ptr c = std::make_unique(GeneralizationConstraint{functionType, sig.signature}, constraintScope); addConstraints(c.get(), NotNull(sig.bodyScope.get())); addConstraint(scope, std::move(c)); @@ -371,7 +391,7 @@ void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatReturn* ret) void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatBlock* block) { - ScopePtr innerScope = childScope(block->location, scope); + ScopePtr innerScope = childScope(block, scope); visitBlockWithoutChildScope(innerScope, block); } @@ -384,16 +404,30 @@ void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatAssign* assign) addConstraint(scope, PackSubtypeConstraint{valuePack, varPackId}); } +void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatCompoundAssign* assign) +{ + // Synthesize A = A op B from A op= B and then build constraints for that instead. + + AstExprBinary exprBinary{assign->location, assign->op, assign->var, assign->value}; + AstExpr* exprBinaryPtr = &exprBinary; + + AstArray vars{&assign->var, 1}; + AstArray values{&exprBinaryPtr, 1}; + AstStatAssign syntheticAssign{assign->location, vars, values}; + + visit(scope, &syntheticAssign); +} + void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatIf* ifStatement) { check(scope, ifStatement->condition); - ScopePtr thenScope = childScope(ifStatement->thenbody->location, scope); + ScopePtr thenScope = childScope(ifStatement->thenbody, scope); visit(thenScope, ifStatement->thenbody); if (ifStatement->elsebody) { - ScopePtr elseScope = childScope(ifStatement->elsebody->location, scope); + ScopePtr elseScope = childScope(ifStatement->elsebody, scope); visit(elseScope, ifStatement->elsebody); } } @@ -561,7 +595,7 @@ void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatDeclareFunction ScopePtr funScope = scope; if (!generics.empty() || !genericPacks.empty()) - funScope = childScope(global->location, scope); + funScope = childScope(global, scope); TypePackId paramPack = resolveTypePack(funScope, global->params); TypePackId retPack = resolveTypePack(funScope, global->retTypes); @@ -739,6 +773,8 @@ TypeId ConstraintGraphBuilder::check(const ScopePtr& scope, AstExpr* expr) result = check(scope, unary); else if (auto binary = expr->as()) result = check(scope, binary); + else if (auto ifElse = expr->as()) + result = check(scope, ifElse); else if (auto typeAssert = expr->as()) result = check(scope, typeAssert); else if (auto err = expr->as()) @@ -819,6 +855,12 @@ TypeId ConstraintGraphBuilder::check(const ScopePtr& scope, AstExprBinary* binar addConstraint(scope, SubtypeConstraint{leftType, rightType}); return leftType; } + case AstExprBinary::Add: + { + TypeId resultType = arena->addType(BlockedTypeVar{}); + addConstraint(scope, BinaryConstraint{AstExprBinary::Add, leftType, rightType, resultType}); + return resultType; + } case AstExprBinary::Sub: { TypeId resultType = arena->addType(BlockedTypeVar{}); @@ -833,6 +875,24 @@ TypeId ConstraintGraphBuilder::check(const ScopePtr& scope, AstExprBinary* binar return nullptr; } +TypeId ConstraintGraphBuilder::check(const ScopePtr& scope, AstExprIfElse* ifElse) +{ + check(scope, ifElse->condition); + + TypeId thenType = check(scope, ifElse->trueExpr); + TypeId elseType = check(scope, ifElse->falseExpr); + + if (ifElse->hasElse) + { + TypeId resultType = arena->addType(BlockedTypeVar{}); + addConstraint(scope, SubtypeConstraint{thenType, resultType}); + addConstraint(scope, SubtypeConstraint{elseType, resultType}); + return resultType; + } + + return thenType; +} + TypeId ConstraintGraphBuilder::check(const ScopePtr& scope, AstExprTypeAssertion* typeAssert) { check(scope, typeAssert->expr); @@ -905,14 +965,14 @@ ConstraintGraphBuilder::FunctionSignature ConstraintGraphBuilder::checkFunctionS // generics properly. if (hasGenerics) { - signatureScope = childScope(fn->location, parent); + signatureScope = childScope(fn, parent); // We need to assign returnType before creating bodyScope so that the // return type gets propogated to bodyScope. returnType = freshTypePack(signatureScope); signatureScope->returnType = returnType; - bodyScope = childScope(fn->body->location, signatureScope); + bodyScope = childScope(fn->body, signatureScope); std::vector> genericDefinitions = createGenerics(signatureScope, fn->generics); std::vector> genericPackDefinitions = createGenericPacks(signatureScope, fn->genericPacks); @@ -933,7 +993,7 @@ ConstraintGraphBuilder::FunctionSignature ConstraintGraphBuilder::checkFunctionS } else { - bodyScope = childScope(fn->body->location, parent); + bodyScope = childScope(fn->body, parent); returnType = freshTypePack(bodyScope); bodyScope->returnType = returnType; @@ -1098,7 +1158,7 @@ TypeId ConstraintGraphBuilder::resolveType(const ScopePtr& scope, AstType* ty, b // for the generic bindings to live on. if (hasGenerics) { - signatureScope = childScope(fn->location, scope); + signatureScope = childScope(fn, scope); std::vector> genericDefinitions = createGenerics(signatureScope, fn->generics); std::vector> genericPackDefinitions = createGenericPacks(signatureScope, fn->genericPacks); diff --git a/Analysis/src/ConstraintSolver.cpp b/Analysis/src/ConstraintSolver.cpp index d8105ce3..6c6d2722 100644 --- a/Analysis/src/ConstraintSolver.cpp +++ b/Analysis/src/ConstraintSolver.cpp @@ -373,7 +373,7 @@ bool ConstraintSolver::tryDispatch(const SubtypeConstraint& c, NotNullscope); unblock(c.subType); unblock(c.superType); @@ -383,7 +383,7 @@ bool ConstraintSolver::tryDispatch(const SubtypeConstraint& c, NotNull constraint, bool force) { - unify(c.subPack, c.superPack); + unify(c.subPack, c.superPack, constraint->scope); unblock(c.subPack); unblock(c.superPack); @@ -398,9 +398,9 @@ bool ConstraintSolver::tryDispatch(const GeneralizationConstraint& c, NotNullty.emplace(c.sourceType); else - unify(c.generalizedType, c.sourceType); + unify(c.generalizedType, c.sourceType, constraint->scope); - TypeId generalized = quantify(arena, c.sourceType, c.scope); + TypeId generalized = quantify(arena, c.sourceType, constraint->scope); *asMutable(c.sourceType) = *generalized; unblock(c.generalizedType); @@ -422,7 +422,7 @@ bool ConstraintSolver::tryDispatch(const InstantiationConstraint& c, NotNullty.emplace(*instantiated); else - unify(c.subType, *instantiated); + unify(c.subType, *instantiated, constraint->scope); unblock(c.subType); @@ -465,7 +465,7 @@ bool ConstraintSolver::tryDispatch(const BinaryConstraint& c, NotNullscope); asMutable(c.resultType)->ty.emplace(leftType); return true; } @@ -528,16 +528,18 @@ struct InstantiationQueuer : TypeVarOnceVisitor { ConstraintSolver* solver; const InstantiationSignature& signature; + NotNull scope; - explicit InstantiationQueuer(ConstraintSolver* solver, const InstantiationSignature& signature) + explicit InstantiationQueuer(ConstraintSolver* solver, const InstantiationSignature& signature, NotNull scope) : solver(solver) , signature(signature) + , scope(scope) { } bool visit(TypeId ty, const PendingExpansionTypeVar& petv) override { - solver->pushConstraint(TypeAliasExpansionConstraint{ty}); + solver->pushConstraint(TypeAliasExpansionConstraint{ty}, scope); return false; } }; @@ -686,7 +688,7 @@ bool ConstraintSolver::tryDispatch(const TypeAliasExpansionConstraint& c, NotNul // The application is not recursive, so we need to queue up application of // any child type function instantiations within the result in order for it // to be complete. - InstantiationQueuer queuer{this, signature}; + InstantiationQueuer queuer{this, signature, constraint->scope}; queuer.traverse(target); instantiatedAliases[signature] = target; @@ -766,30 +768,40 @@ bool ConstraintSolver::isBlocked(NotNull constraint) return blockedIt != blockedConstraints.end() && blockedIt->second > 0; } -void ConstraintSolver::unify(TypeId subType, TypeId superType) +void ConstraintSolver::unify(TypeId subType, TypeId superType, NotNull scope) { UnifierSharedState sharedState{&iceReporter}; - Unifier u{arena, Mode::Strict, Location{}, Covariant, sharedState}; + Unifier u{arena, Mode::Strict, scope, Location{}, Covariant, sharedState}; u.tryUnify(subType, superType); u.log.commit(); } -void ConstraintSolver::unify(TypePackId subPack, TypePackId superPack) +void ConstraintSolver::unify(TypePackId subPack, TypePackId superPack, NotNull scope) { UnifierSharedState sharedState{&iceReporter}; - Unifier u{arena, Mode::Strict, Location{}, Covariant, sharedState}; + Unifier u{arena, Mode::Strict, scope, Location{}, Covariant, sharedState}; u.tryUnify(subPack, superPack); u.log.commit(); } -void ConstraintSolver::pushConstraint(ConstraintV cv) +void ConstraintSolver::pushConstraint(ConstraintV cv, NotNull scope) { - std::unique_ptr c = std::make_unique(std::move(cv)); + std::unique_ptr c = std::make_unique(std::move(cv), scope); NotNull borrow = NotNull(c.get()); solverConstraints.push_back(std::move(c)); unsolvedConstraints.push_back(borrow); } +void ConstraintSolver::reportError(TypeErrorData&& data, const Location& location) +{ + errors.emplace_back(location, std::move(data)); +} + +void ConstraintSolver::reportError(TypeError e) +{ + errors.emplace_back(std::move(e)); +} + } // namespace Luau diff --git a/Analysis/src/Frontend.cpp b/Analysis/src/Frontend.cpp index c450297f..8ab4e865 100644 --- a/Analysis/src/Frontend.cpp +++ b/Analysis/src/Frontend.cpp @@ -839,6 +839,9 @@ ModulePtr Frontend::check(const SourceModule& sourceModule, Mode mode, const Sco ConstraintSolver cs{&result->internalTypes, NotNull(cgb.rootScope)}; cs.run(); + for (TypeError& e : cs.errors) + result->errors.emplace_back(std::move(e)); + result->scopes = std::move(cgb.scopes); result->astTypes = std::move(cgb.astTypes); result->astTypePacks = std::move(cgb.astTypePacks); diff --git a/Analysis/src/Module.cpp b/Analysis/src/Module.cpp index 4e6b2585..de796c7a 100644 --- a/Analysis/src/Module.cpp +++ b/Analysis/src/Module.cpp @@ -244,12 +244,12 @@ void Module::clonePublicInterface(InternalErrorReporter& ice) if (FFlag::LuauLowerBoundsCalculation) { - normalize(returnType, interfaceTypes, ice); + normalize(returnType, NotNull{this}, ice); if (FFlag::LuauForceExportSurfacesToBeNormal) forceNormal.traverse(returnType); if (varargPack) { - normalize(*varargPack, interfaceTypes, ice); + normalize(*varargPack, NotNull{this}, ice); if (FFlag::LuauForceExportSurfacesToBeNormal) forceNormal.traverse(*varargPack); } @@ -265,7 +265,7 @@ void Module::clonePublicInterface(InternalErrorReporter& ice) tf = clone(tf, interfaceTypes, cloneState); if (FFlag::LuauLowerBoundsCalculation) { - normalize(tf.type, interfaceTypes, ice); + normalize(tf.type, NotNull{this}, ice); // We're about to freeze the memory. We know that the flag is conservative by design. Cyclic tables // won't be marked normal. If the types aren't normal by now, they never will be. @@ -276,7 +276,7 @@ void Module::clonePublicInterface(InternalErrorReporter& ice) if (param.defaultValue) { - normalize(*param.defaultValue, interfaceTypes, ice); + normalize(*param.defaultValue, NotNull{this}, ice); forceNormal.traverse(*param.defaultValue); } } @@ -302,7 +302,7 @@ void Module::clonePublicInterface(InternalErrorReporter& ice) ty = clone(ty, interfaceTypes, cloneState); if (FFlag::LuauLowerBoundsCalculation) { - normalize(ty, interfaceTypes, ice); + normalize(ty, NotNull{this}, ice); if (FFlag::LuauForceExportSurfacesToBeNormal) forceNormal.traverse(ty); diff --git a/Analysis/src/Normalize.cpp b/Analysis/src/Normalize.cpp index 33f369a7..94adaf5c 100644 --- a/Analysis/src/Normalize.cpp +++ b/Analysis/src/Normalize.cpp @@ -15,7 +15,6 @@ LUAU_FASTINTVARIABLE(LuauNormalizeIterationLimit, 1200); LUAU_FASTFLAGVARIABLE(LuauNormalizeCombineTableFix, false); LUAU_FASTFLAGVARIABLE(LuauFixNormalizationOfCyclicUnions, false); LUAU_FASTFLAG(LuauUnknownAndNeverType) -LUAU_FASTFLAG(LuauQuantifyConstrained) namespace Luau { @@ -55,11 +54,11 @@ struct Replacer } // anonymous namespace -bool isSubtype(TypeId subTy, TypeId superTy, InternalErrorReporter& ice) +bool isSubtype(TypeId subTy, TypeId superTy, NotNull scope, InternalErrorReporter& ice) { UnifierSharedState sharedState{&ice}; TypeArena arena; - Unifier u{&arena, Mode::Strict, Location{}, Covariant, sharedState}; + Unifier u{&arena, Mode::Strict, scope, Location{}, Covariant, sharedState}; u.anyIsTop = true; u.tryUnify(subTy, superTy); @@ -67,11 +66,11 @@ bool isSubtype(TypeId subTy, TypeId superTy, InternalErrorReporter& ice) return ok; } -bool isSubtype(TypePackId subPack, TypePackId superPack, InternalErrorReporter& ice) +bool isSubtype(TypePackId subPack, TypePackId superPack, NotNull scope, InternalErrorReporter& ice) { UnifierSharedState sharedState{&ice}; TypeArena arena; - Unifier u{&arena, Mode::Strict, Location{}, Covariant, sharedState}; + Unifier u{&arena, Mode::Strict, scope, Location{}, Covariant, sharedState}; u.anyIsTop = true; u.tryUnify(subPack, superPack); @@ -134,13 +133,15 @@ struct Normalize final : TypeVarVisitor { using TypeVarVisitor::Set; - Normalize(TypeArena& arena, InternalErrorReporter& ice) + Normalize(TypeArena& arena, NotNull scope, InternalErrorReporter& ice) : arena(arena) + , scope(scope) , ice(ice) { } TypeArena& arena; + NotNull scope; InternalErrorReporter& ice; int iterationLimit = 0; @@ -215,22 +216,7 @@ struct Normalize final : TypeVarVisitor traverse(part); std::vector newParts = normalizeUnion(parts); - - if (FFlag::LuauQuantifyConstrained) - { - ctv->parts = std::move(newParts); - } - else - { - const bool normal = areNormal(newParts, seen, ice); - - if (newParts.size() == 1) - *asMutable(ty) = BoundTypeVar{newParts[0]}; - else - *asMutable(ty) = UnionTypeVar{std::move(newParts)}; - - asMutable(ty)->normal = normal; - } + ctv->parts = std::move(newParts); return false; } @@ -288,12 +274,7 @@ struct Normalize final : TypeVarVisitor } // An unsealed table can never be normal, ditto for free tables iff the type it is bound to is also not normal. - if (FFlag::LuauQuantifyConstrained) - { - if (ttv.state == TableState::Generic || ttv.state == TableState::Sealed || (ttv.state == TableState::Free && follow(ty)->normal)) - asMutable(ty)->normal = normal; - } - else + if (ttv.state == TableState::Generic || ttv.state == TableState::Sealed || (ttv.state == TableState::Free && follow(ty)->normal)) asMutable(ty)->normal = normal; return false; @@ -518,9 +499,9 @@ struct Normalize final : TypeVarVisitor for (TypeId& part : result) { - if (isSubtype(ty, part, ice)) + if (isSubtype(ty, part, scope, ice)) return; // no need to do anything - else if (isSubtype(part, ty, ice)) + else if (isSubtype(part, ty, scope, ice)) { part = ty; // replace the less general type by the more general one return; @@ -572,12 +553,12 @@ struct Normalize final : TypeVarVisitor bool merged = false; for (TypeId& part : result->parts) { - if (isSubtype(part, ty, ice)) + if (isSubtype(part, ty, scope, ice)) { merged = true; break; // no need to do anything } - else if (isSubtype(ty, part, ice)) + else if (isSubtype(ty, part, scope, ice)) { merged = true; part = ty; // replace the less general type by the more general one @@ -710,13 +691,13 @@ struct Normalize final : TypeVarVisitor /** * @returns A tuple of TypeId and a success indicator. (true indicates that the normalization completed successfully) */ -std::pair normalize(TypeId ty, TypeArena& arena, InternalErrorReporter& ice) +std::pair normalize(TypeId ty, NotNull scope, TypeArena& arena, InternalErrorReporter& ice) { CloneState state; if (FFlag::DebugLuauCopyBeforeNormalizing) (void)clone(ty, arena, state); - Normalize n{arena, ice}; + Normalize n{arena, scope, ice}; n.traverse(ty); return {ty, !n.limitExceeded}; @@ -726,29 +707,39 @@ std::pair normalize(TypeId ty, TypeArena& arena, InternalErrorRepo // reclaim memory used by wantonly allocated intermediate types here. // The main wrinkle here is that we don't want clone() to copy a type if the source and dest // arena are the same. +std::pair normalize(TypeId ty, NotNull module, InternalErrorReporter& ice) +{ + return normalize(ty, NotNull{module->getModuleScope().get()}, module->internalTypes, ice); +} + std::pair normalize(TypeId ty, const ModulePtr& module, InternalErrorReporter& ice) { - return normalize(ty, module->internalTypes, ice); + return normalize(ty, NotNull{module.get()}, ice); } /** * @returns A tuple of TypeId and a success indicator. (true indicates that the normalization completed successfully) */ -std::pair normalize(TypePackId tp, TypeArena& arena, InternalErrorReporter& ice) +std::pair normalize(TypePackId tp, NotNull scope, TypeArena& arena, InternalErrorReporter& ice) { CloneState state; if (FFlag::DebugLuauCopyBeforeNormalizing) (void)clone(tp, arena, state); - Normalize n{arena, ice}; + Normalize n{arena, scope, ice}; n.traverse(tp); return {tp, !n.limitExceeded}; } +std::pair normalize(TypePackId tp, NotNull module, InternalErrorReporter& ice) +{ + return normalize(tp, NotNull{module->getModuleScope().get()}, module->internalTypes, ice); +} + std::pair normalize(TypePackId tp, const ModulePtr& module, InternalErrorReporter& ice) { - return normalize(tp, module->internalTypes, ice); + return normalize(tp, NotNull{module.get()}, ice); } } // namespace Luau diff --git a/Analysis/src/Quantify.cpp b/Analysis/src/Quantify.cpp index ce4afe22..7e6ff2f9 100644 --- a/Analysis/src/Quantify.cpp +++ b/Analysis/src/Quantify.cpp @@ -10,7 +10,6 @@ LUAU_FASTFLAG(DebugLuauSharedSelf) LUAU_FASTFLAG(DebugLuauDeferredConstraintResolution); -LUAU_FASTFLAGVARIABLE(LuauQuantifyConstrained, false) LUAU_FASTFLAG(LuauClassTypeVarsInSubstitution) namespace Luau @@ -82,30 +81,25 @@ struct Quantifier final : TypeVarOnceVisitor bool visit(TypeId ty, const ConstrainedTypeVar&) override { - if (FFlag::LuauQuantifyConstrained) - { - ConstrainedTypeVar* ctv = getMutable(ty); + ConstrainedTypeVar* ctv = getMutable(ty); - seenMutableType = true; - - if (FFlag::DebugLuauDeferredConstraintResolution ? !subsumes(scope, ctv->scope) : !level.subsumes(ctv->level)) - return false; - - std::vector opts = std::move(ctv->parts); - - // We might transmute, so it's not safe to rely on the builtin traversal logic - for (TypeId opt : opts) - traverse(opt); - - if (opts.size() == 1) - *asMutable(ty) = BoundTypeVar{opts[0]}; - else - *asMutable(ty) = UnionTypeVar{std::move(opts)}; + seenMutableType = true; + if (FFlag::DebugLuauDeferredConstraintResolution ? !subsumes(scope, ctv->scope) : !level.subsumes(ctv->level)) return false; - } + + std::vector opts = std::move(ctv->parts); + + // We might transmute, so it's not safe to rely on the builtin traversal logic + for (TypeId opt : opts) + traverse(opt); + + if (opts.size() == 1) + *asMutable(ty) = BoundTypeVar{opts[0]}; else - return true; + *asMutable(ty) = UnionTypeVar{std::move(opts)}; + + return false; } bool visit(TypeId ty, const TableTypeVar&) override @@ -119,12 +113,6 @@ struct Quantifier final : TypeVarOnceVisitor if (ttv.state == TableState::Free) seenMutableType = true; - if (!FFlag::LuauQuantifyConstrained) - { - if (ttv.state == TableState::Sealed || ttv.state == TableState::Generic) - return false; - } - if (FFlag::DebugLuauDeferredConstraintResolution ? !subsumes(scope, ttv.scope) : !level.subsumes(ttv.level)) { if (ttv.state == TableState::Unsealed) diff --git a/Analysis/src/TypeChecker2.cpp b/Analysis/src/TypeChecker2.cpp index 9d97e8d9..e5813cd2 100644 --- a/Analysis/src/TypeChecker2.cpp +++ b/Analysis/src/TypeChecker2.cpp @@ -8,22 +8,59 @@ #include "Luau/Clone.h" #include "Luau/Instantiation.h" #include "Luau/Normalize.h" +#include "Luau/ToString.h" #include "Luau/TxnLog.h" #include "Luau/TypeUtils.h" #include "Luau/TypeVar.h" #include "Luau/Unifier.h" -#include "Luau/ToString.h" namespace Luau { -struct TypeChecker2 : public AstVisitor +/* Push a scope onto the end of a stack for the lifetime of the StackPusher instance. + * TypeChecker2 uses this to maintain knowledge about which scope encloses every + * given AstNode. + */ +struct StackPusher +{ + std::vector>* stack; + NotNull scope; + + explicit StackPusher(std::vector>& stack, Scope* scope) + : stack(&stack) + , scope(scope) + { + stack.push_back(NotNull{scope}); + } + + ~StackPusher() + { + if (stack) + { + LUAU_ASSERT(stack->back() == scope); + stack->pop_back(); + } + } + + StackPusher(const StackPusher&) = delete; + StackPusher&& operator=(const StackPusher&) = delete; + + StackPusher(StackPusher&& other) + : stack(std::exchange(other.stack, nullptr)) + , scope(other.scope) + { + } +}; + +struct TypeChecker2 { const SourceModule* sourceModule; Module* module; InternalErrorReporter ice; // FIXME accept a pointer from Frontend SingletonTypes& singletonTypes; + std::vector> stack; + TypeChecker2(const SourceModule* sourceModule, Module* module) : sourceModule(sourceModule) , module(module) @@ -31,7 +68,13 @@ struct TypeChecker2 : public AstVisitor { } - using AstVisitor::visit; + std::optional pushStack(AstNode* node) + { + if (Scope** scope = module->astScopes.find(node)) + return StackPusher{stack, *scope}; + else + return std::nullopt; + } TypePackId lookupPack(AstExpr* expr) { @@ -118,11 +161,128 @@ struct TypeChecker2 : public AstVisitor return bestScope; } - bool visit(AstStatLocal* local) override + void visit(AstStat* stat) + { + auto pusher = pushStack(stat); + + if (0) + {} + else if (auto s = stat->as()) + return visit(s); + else if (auto s = stat->as()) + return visit(s); + else if (auto s = stat->as()) + return visit(s); + else if (auto s = stat->as()) + return visit(s); + else if (auto s = stat->as()) + return visit(s); + else if (auto s = stat->as()) + return visit(s); + else if (auto s = stat->as()) + return visit(s); + else if (auto s = stat->as()) + return visit(s); + else if (auto s = stat->as()) + return visit(s); + else if (auto s = stat->as()) + return visit(s); + else if (auto s = stat->as()) + return visit(s); + else if (auto s = stat->as()) + return visit(s); + else if (auto s = stat->as()) + return visit(s); + else if (auto s = stat->as()) + return visit(s); + else if (auto s = stat->as()) + return visit(s); + else if (auto s = stat->as()) + return visit(s); + else if (auto s = stat->as()) + return visit(s); + else if (auto s = stat->as()) + return visit(s); + else if (auto s = stat->as()) + return visit(s); + else if (auto s = stat->as()) + return visit(s); + else + LUAU_ASSERT(!"TypeChecker2 encountered an unknown node type"); + } + + void visit(AstStatBlock* block) + { + auto StackPusher = pushStack(block); + + for (AstStat* statement : block->body) + visit(statement); + } + + void visit(AstStatIf* ifStatement) + { + visit(ifStatement->condition); + visit(ifStatement->thenbody); + if (ifStatement->elsebody) + visit(ifStatement->elsebody); + } + + void visit(AstStatWhile* whileStatement) + { + visit(whileStatement->condition); + visit(whileStatement->body); + } + + void visit(AstStatRepeat* repeatStatement) + { + visit(repeatStatement->body); + visit(repeatStatement->condition); + } + + void visit(AstStatBreak*) + {} + + void visit(AstStatContinue*) + {} + + void visit(AstStatReturn* ret) + { + Scope* scope = findInnermostScope(ret->location); + TypePackId expectedRetType = scope->returnType; + + TypeArena arena; + TypePackId actualRetType = reconstructPack(ret->list, arena); + + UnifierSharedState sharedState{&ice}; + Unifier u{&arena, Mode::Strict, stack.back(), ret->location, Covariant, sharedState}; + u.anyIsTop = true; + + u.tryUnify(actualRetType, expectedRetType); + const bool ok = u.errors.empty() && u.log.empty(); + + if (!ok) + { + for (const TypeError& e : u.errors) + reportError(e); + } + + for (AstExpr* expr : ret->list) + visit(expr); + } + + void visit(AstStatExpr* expr) + { + visit(expr->expr); + } + + void visit(AstStatLocal* local) { for (size_t i = 0; i < local->values.size; ++i) { AstExpr* value = local->values.data[i]; + + visit(value); + if (i == local->values.size - 1) { if (i < local->values.size) @@ -140,7 +300,7 @@ struct TypeChecker2 : public AstVisitor if (var->annotation) { TypeId varType = lookupAnnotation(var->annotation); - if (!isSubtype(*it, varType, ice)) + if (!isSubtype(*it, varType, stack.back(), ice)) { reportError(TypeMismatch{varType, *it}, value->location); } @@ -158,64 +318,244 @@ struct TypeChecker2 : public AstVisitor if (var->annotation) { TypeId varType = lookupAnnotation(var->annotation); - if (!isSubtype(varType, valueType, ice)) + if (!isSubtype(varType, valueType, stack.back(), ice)) { reportError(TypeMismatch{varType, valueType}, value->location); } } } } - - return true; } - bool visit(AstStatAssign* assign) override + void visit(AstStatFor* forStatement) + { + if (forStatement->var->annotation) + visit(forStatement->var->annotation); + + visit(forStatement->from); + visit(forStatement->to); + if (forStatement->step) + visit(forStatement->step); + visit(forStatement->body); + } + + void visit(AstStatForIn* forInStatement) + { + for (AstLocal* local : forInStatement->vars) + { + if (local->annotation) + visit(local->annotation); + } + + for (AstExpr* expr : forInStatement->values) + visit(expr); + + visit(forInStatement->body); + } + + void visit(AstStatAssign* assign) { size_t count = std::min(assign->vars.size, assign->values.size); for (size_t i = 0; i < count; ++i) { AstExpr* lhs = assign->vars.data[i]; + visit(lhs); TypeId lhsType = lookupType(lhs); AstExpr* rhs = assign->values.data[i]; + visit(rhs); TypeId rhsType = lookupType(rhs); - if (!isSubtype(rhsType, lhsType, ice)) + if (!isSubtype(rhsType, lhsType, stack.back(), ice)) { reportError(TypeMismatch{lhsType, rhsType}, rhs->location); } } - - return true; } - bool visit(AstStatReturn* ret) override + void visit(AstStatCompoundAssign* stat) { - Scope* scope = findInnermostScope(ret->location); - TypePackId expectedRetType = scope->returnType; + visit(stat->var); + visit(stat->value); + } - TypeArena arena; - TypePackId actualRetType = reconstructPack(ret->list, arena); + void visit(AstStatFunction* stat) + { + visit(stat->name); + visit(stat->func); + } - UnifierSharedState sharedState{&ice}; - Unifier u{&arena, Mode::Strict, ret->location, Covariant, sharedState}; - u.anyIsTop = true; + void visit(AstStatLocalFunction* stat) + { + visit(stat->func); + } - u.tryUnify(actualRetType, expectedRetType); - const bool ok = u.errors.empty() && u.log.empty(); + void visit(const AstTypeList* typeList) + { + for (AstType* ty : typeList->types) + visit(ty); - if (!ok) + if (typeList->tailType) + visit(typeList->tailType); + } + + void visit(AstStatTypeAlias* stat) + { + for (const AstGenericType& el : stat->generics) { - for (const TypeError& e : u.errors) - reportError(e); + if (el.defaultValue) + visit(el.defaultValue); } - return true; + for (const AstGenericTypePack& el : stat->genericPacks) + { + if (el.defaultValue) + visit(el.defaultValue); + } + + visit(stat->type); } - bool visit(AstExprCall* call) override + void visit(AstTypeList types) { + for (AstType* type : types.types) + visit(type); + if (types.tailType) + visit(types.tailType); + } + + void visit(AstStatDeclareFunction* stat) + { + visit(stat->params); + visit(stat->retTypes); + } + + void visit(AstStatDeclareGlobal* stat) + { + visit(stat->type); + } + + void visit(AstStatDeclareClass* stat) + { + for (const AstDeclaredClassProp& prop : stat->props) + visit(prop.ty); + } + + void visit(AstStatError* stat) + { + for (AstExpr* expr : stat->expressions) + visit(expr); + + for (AstStat* s : stat->statements) + visit(s); + } + + void visit(AstExpr* expr) + { + auto StackPusher = pushStack(expr); + + if (0) + {} + else if (auto e = expr->as()) + return visit(e); + else if (auto e = expr->as()) + return visit(e); + else if (auto e = expr->as()) + return visit(e); + else if (auto e = expr->as()) + return visit(e); + else if (auto e = expr->as()) + return visit(e); + else if (auto e = expr->as()) + return visit(e); + else if (auto e = expr->as()) + return visit(e); + else if (auto e = expr->as()) + return visit(e); + else if (auto e = expr->as()) + return visit(e); + else if (auto e = expr->as()) + return visit(e); + else if (auto e = expr->as()) + return visit(e); + else if (auto e = expr->as()) + return visit(e); + else if (auto e = expr->as()) + return visit(e); + else if (auto e = expr->as()) + return visit(e); + else if (auto e = expr->as()) + return visit(e); + else if (auto e = expr->as()) + return visit(e); + else if (auto e = expr->as()) + return visit(e); + else if (auto e = expr->as()) + return visit(e); + else + LUAU_ASSERT(!"TypeChecker2 encountered an unknown expression type"); + } + + void visit(AstExprGroup* expr) + { + visit(expr->expr); + } + + void visit(AstExprConstantNil* expr) + { + // TODO! + } + + void visit(AstExprConstantBool* expr) + { + // TODO! + } + + void visit(AstExprConstantNumber* number) + { + TypeId actualType = lookupType(number); + TypeId numberType = getSingletonTypes().numberType; + + if (!isSubtype(numberType, actualType, stack.back(), ice)) + { + reportError(TypeMismatch{actualType, numberType}, number->location); + } + } + + void visit(AstExprConstantString* string) + { + TypeId actualType = lookupType(string); + TypeId stringType = getSingletonTypes().stringType; + + if (!isSubtype(stringType, actualType, stack.back(), ice)) + { + reportError(TypeMismatch{actualType, stringType}, string->location); + } + } + + void visit(AstExprLocal* expr) + { + // TODO! + } + + void visit(AstExprGlobal* expr) + { + // TODO! + } + + void visit(AstExprVarargs* expr) + { + // TODO! + } + + void visit(AstExprCall* call) + { + visit(call->func); + + for (AstExpr* arg : call->args) + visit(arg); + TypeArena arena; Instantiation instantiation{TxnLog::empty(), &arena, TypeLevel{}}; @@ -225,7 +565,7 @@ struct TypeChecker2 : public AstVisitor LUAU_ASSERT(functionType); TypePack args; - for (const auto& arg : call->args) + for (AstExpr* arg : call->args) { TypeId argTy = module->astTypes[arg]; LUAU_ASSERT(argTy); @@ -235,7 +575,7 @@ struct TypeChecker2 : public AstVisitor TypePackId argsTp = arena.addTypePack(args); FunctionTypeVar ftv{argsTp, expectedRetType}; TypeId expectedType = arena.addType(ftv); - if (!isSubtype(expectedType, instantiatedFunctionType, ice)) + if (!isSubtype(expectedType, instantiatedFunctionType, stack.back(), ice)) { unfreeze(module->interfaceTypes); CloneState cloneState; @@ -243,12 +583,36 @@ struct TypeChecker2 : public AstVisitor freeze(module->interfaceTypes); reportError(TypeMismatch{expectedType, functionType}, call->location); } - - return true; } - bool visit(AstExprFunction* fn) override + void visit(AstExprIndexName* indexName) { + TypeId leftType = lookupType(indexName->expr); + TypeId resultType = lookupType(indexName); + + // leftType must have a property called indexName->index + + std::optional ty = getIndexTypeFromType(module->getModuleScope(), leftType, indexName->index.value, indexName->location, /* addErrors */ true); + if (ty) + { + if (!isSubtype(resultType, *ty, stack.back(), ice)) + { + reportError(TypeMismatch{resultType, *ty}, indexName->location); + } + } + } + + void visit(AstExprIndexExpr* indexExpr) + { + // TODO! + visit(indexExpr->expr); + visit(indexExpr->index); + } + + void visit(AstExprFunction* fn) + { + auto StackPusher = pushStack(fn); + TypeId inferredFnTy = lookupType(fn); const FunctionTypeVar* inferredFtv = get(inferredFnTy); LUAU_ASSERT(inferredFtv); @@ -264,7 +628,7 @@ struct TypeChecker2 : public AstVisitor TypeId inferredArgTy = *argIt; TypeId annotatedArgTy = lookupAnnotation(arg->annotation); - if (!isSubtype(annotatedArgTy, inferredArgTy, ice)) + if (!isSubtype(annotatedArgTy, inferredArgTy, stack.back(), ice)) { reportError(TypeMismatch{annotatedArgTy, inferredArgTy}, arg->location); } @@ -273,68 +637,64 @@ struct TypeChecker2 : public AstVisitor ++argIt; } - return true; + visit(fn->body); } - bool visit(AstExprIndexName* indexName) override + void visit(AstExprTable* expr) { - TypeId leftType = lookupType(indexName->expr); - TypeId resultType = lookupType(indexName); - - // leftType must have a property called indexName->index - - std::optional ty = getIndexTypeFromType(module->getModuleScope(), leftType, indexName->index.value, indexName->location, /* addErrors */ true); - if (ty) + // TODO! + for (const AstExprTable::Item& item : expr->items) { - if (!isSubtype(resultType, *ty, ice)) - { - reportError(TypeMismatch{resultType, *ty}, indexName->location); - } + if (item.key) + visit(item.key); + visit(item.value); } - - return true; } - bool visit(AstExprConstantNumber* number) override + void visit(AstExprUnary* expr) { - TypeId actualType = lookupType(number); - TypeId numberType = getSingletonTypes().numberType; - - if (!isSubtype(numberType, actualType, ice)) - { - reportError(TypeMismatch{actualType, numberType}, number->location); - } - - return true; + // TODO! + visit(expr->expr); } - bool visit(AstExprConstantString* string) override + void visit(AstExprBinary* expr) { - TypeId actualType = lookupType(string); - TypeId stringType = getSingletonTypes().stringType; - - if (!isSubtype(stringType, actualType, ice)) - { - reportError(TypeMismatch{actualType, stringType}, string->location); - } - - return true; + // TODO! + visit(expr->left); + visit(expr->right); } - bool visit(AstExprTypeAssertion* expr) override + void visit(AstExprTypeAssertion* expr) { + visit(expr->expr); + visit(expr->annotation); + TypeId annotationType = lookupAnnotation(expr->annotation); TypeId computedType = lookupType(expr->expr); // Note: As an optimization, we try 'number <: number | string' first, as that is the more likely case. - if (isSubtype(annotationType, computedType, ice)) - return true; + if (isSubtype(annotationType, computedType, stack.back(), ice)) + return; - if (isSubtype(computedType, annotationType, ice)) - return true; + if (isSubtype(computedType, annotationType, stack.back(), ice)) + return; reportError(TypesAreUnrelated{computedType, annotationType}, expr->location); - return true; + } + + void visit(AstExprIfElse* expr) + { + // TODO! + visit(expr->condition); + visit(expr->trueExpr); + visit(expr->falseExpr); + } + + void visit(AstExprError* expr) + { + // TODO! + for (AstExpr* e : expr->expressions) + visit(e); } /** Extract a TypeId for the first type of the provided pack. @@ -375,13 +735,32 @@ struct TypeChecker2 : public AstVisitor ice.ice("flattenPack got a weird pack!"); } - bool visit(AstType* ty) override + void visit(AstType* ty) { - return true; + if (auto t = ty->as()) + return visit(t); + else if (auto t = ty->as()) + return visit(t); + else if (auto t = ty->as()) + return visit(t); + else if (auto t = ty->as()) + return visit(t); + else if (auto t = ty->as()) + return visit(t); + else if (auto t = ty->as()) + return visit(t); } - bool visit(AstTypeReference* ty) override + void visit(AstTypeReference* ty) { + for (const AstTypeOrPack& param : ty->parameters) + { + if (param.type) + visit(param.type); + else + visit(param.typePack); + } + Scope* scope = findInnermostScope(ty->location); LUAU_ASSERT(scope); @@ -500,16 +879,76 @@ struct TypeChecker2 : public AstVisitor reportError(UnknownSymbol{ty->name.value, UnknownSymbol::Context::Type}, ty->location); } } - - return true; } - bool visit(AstTypePack*) override + void visit(AstTypeTable* table) { - return true; + // TODO! + + for (const AstTableProp& prop : table->props) + visit(prop.type); + + if (table->indexer) + { + visit(table->indexer->indexType); + visit(table->indexer->resultType); + } } - bool visit(AstTypePackGeneric* tp) override + void visit(AstTypeFunction* ty) + { + // TODO! + + visit(ty->argTypes); + visit(ty->returnTypes); + } + + void visit(AstTypeTypeof* ty) + { + visit(ty->expr); + } + + void visit(AstTypeUnion* ty) + { + // TODO! + for (AstType* type : ty->types) + visit(type); + } + + void visit(AstTypeIntersection* ty) + { + // TODO! + for (AstType* type : ty->types) + visit(type); + } + + void visit(AstTypePack* pack) + { + if (auto p = pack->as()) + return visit(p); + else if (auto p = pack->as()) + return visit(p); + else if (auto p = pack->as()) + return visit(p); + } + + void visit(AstTypePackExplicit* tp) + { + // TODO! + for (AstType* type : tp->typeList.types) + visit(type); + + if (tp->typeList.tailType) + visit(tp->typeList.tailType); + } + + void visit(AstTypePackVariadic* tp) + { + // TODO! + visit(tp->variadicType); + } + + void visit(AstTypePackGeneric* tp) { Scope* scope = findInnermostScope(tp->location); LUAU_ASSERT(scope); @@ -531,8 +970,6 @@ struct TypeChecker2 : public AstVisitor reportError(UnknownSymbol{tp->genericName.value, UnknownSymbol::Context::Type}, tp->location); } } - - return true; } void reportError(TypeErrorData&& data, const Location& location) @@ -546,139 +983,9 @@ struct TypeChecker2 : public AstVisitor } std::optional getIndexTypeFromType( - const ScopePtr& scope, TypeId type, const Name& name, const Location& location, bool addErrors) + const ScopePtr& scope, TypeId type, const std::string& prop, const Location& location, bool addErrors) { - type = follow(type); - - if (get(type) || get(type) || get(type)) - return type; - - if (auto f = get(type)) - *asMutable(type) = TableTypeVar{TableState::Free, f->level}; - - if (isString(type)) - { - std::optional mtIndex = Luau::findMetatableEntry(module->errors, singletonTypes.stringType, "__index", location); - LUAU_ASSERT(mtIndex); - type = *mtIndex; - } - - if (TableTypeVar* tableType = getMutableTableType(type)) - { - - return findTablePropertyRespectingMeta(module->errors, type, name, location); - } - else if (const ClassTypeVar* cls = get(type)) - { - const Property* prop = lookupClassProp(cls, name); - if (prop) - return prop->type; - } - else if (const UnionTypeVar* utv = get(type)) - { - std::vector goodOptions; - std::vector badOptions; - - for (TypeId t : utv) - { - // TODO: we should probably limit recursion here? - // RecursionLimiter _rl(&recursionCount, FInt::LuauTypeInferRecursionLimit); - - // Not needed when we normalize types. - if (get(follow(t))) - return t; - - if (std::optional ty = getIndexTypeFromType(scope, t, name, location, /* addErrors= */ false)) - goodOptions.push_back(*ty); - else - badOptions.push_back(t); - } - - if (!badOptions.empty()) - { - if (addErrors) - { - if (goodOptions.empty()) - reportError(UnknownProperty{type, name}, location); - else - reportError(MissingUnionProperty{type, badOptions, name}, location); - } - return std::nullopt; - } - - std::vector result = reduceUnion(goodOptions); - if (result.empty()) - return singletonTypes.neverType; - - if (result.size() == 1) - return result[0]; - - return module->internalTypes.addType(UnionTypeVar{std::move(result)}); - } - else if (const IntersectionTypeVar* itv = get(type)) - { - std::vector parts; - - for (TypeId t : itv->parts) - { - // TODO: we should probably limit recursion here? - // RecursionLimiter _rl(&recursionCount, FInt::LuauTypeInferRecursionLimit); - - if (std::optional ty = getIndexTypeFromType(scope, t, name, location, /* addErrors= */ false)) - parts.push_back(*ty); - } - - // If no parts of the intersection had the property we looked up for, it never existed at all. - if (parts.empty()) - { - if (addErrors) - reportError(UnknownProperty{type, name}, location); - return std::nullopt; - } - - if (parts.size() == 1) - return parts[0]; - - return module->internalTypes.addType(IntersectionTypeVar{std::move(parts)}); // Not at all correct. - } - - if (addErrors) - reportError(UnknownProperty{type, name}, location); - - return std::nullopt; - } - - std::vector reduceUnion(const std::vector& types) - { - std::vector result; - for (TypeId t : types) - { - t = follow(t); - if (get(t)) - continue; - - if (get(t) || get(t)) - return {t}; - - if (const UnionTypeVar* utv = get(t)) - { - for (TypeId ty : utv) - { - ty = follow(ty); - if (get(ty)) - continue; - if (get(ty) || get(ty)) - return {ty}; - - if (result.end() == std::find(result.begin(), result.end(), ty)) - result.push_back(ty); - } - } - else if (std::find(result.begin(), result.end(), t) == result.end()) - result.push_back(t); - } - - return result; + return Luau::getIndexTypeFromType(scope, module->errors, &module->internalTypes, type, prop, location, addErrors, ice); } }; @@ -686,7 +993,7 @@ void check(const SourceModule& sourceModule, Module* module) { TypeChecker2 typeChecker{&sourceModule, module}; - sourceModule.root->visit(&typeChecker); + typeChecker.visit(sourceModule.root); } } // namespace Luau diff --git a/Analysis/src/TypeInfer.cpp b/Analysis/src/TypeInfer.cpp index 7ab23360..9886fb19 100644 --- a/Analysis/src/TypeInfer.cpp +++ b/Analysis/src/TypeInfer.cpp @@ -33,15 +33,13 @@ LUAU_FASTINTVARIABLE(LuauVisitRecursionLimit, 500) LUAU_FASTFLAG(LuauKnowsTheDataModel3) LUAU_FASTFLAG(LuauAutocompleteDynamicLimits) LUAU_FASTFLAGVARIABLE(LuauExpectedTableUnionIndexerType, false) +LUAU_FASTFLAGVARIABLE(LuauInplaceDemoteSkipAllBound, false) LUAU_FASTFLAGVARIABLE(LuauLowerBoundsCalculation, false) LUAU_FASTFLAGVARIABLE(DebugLuauFreezeDuringUnification, false) LUAU_FASTFLAGVARIABLE(LuauSelfCallAutocompleteFix3, false) LUAU_FASTFLAGVARIABLE(LuauReturnAnyInsteadOfICE, false) // Eventually removed as false. LUAU_FASTFLAGVARIABLE(DebugLuauSharedSelf, false); -LUAU_FASTFLAGVARIABLE(LuauReportErrorsOnIndexerKeyMismatch, false) LUAU_FASTFLAGVARIABLE(LuauUnknownAndNeverType, false) -LUAU_FASTFLAG(LuauQuantifyConstrained) -LUAU_FASTFLAGVARIABLE(LuauFalsyPredicateReturnsNilInstead, false) LUAU_FASTFLAGVARIABLE(LuauCheckGenericHOFTypes, false) LUAU_FASTFLAGVARIABLE(LuauBinaryNeedsExpectedTypesToo, false) LUAU_FASTFLAGVARIABLE(LuauNeverTypesAndOperatorsInference, false) @@ -473,7 +471,8 @@ struct InplaceDemoter : TypeVarOnceVisitor TypeArena* arena; InplaceDemoter(TypeLevel level, TypeArena* arena) - : newLevel(level) + : TypeVarOnceVisitor(/* skipBoundTypes= */ FFlag::LuauInplaceDemoteSkipAllBound) + , newLevel(level) , arena(arena) { } @@ -494,6 +493,7 @@ struct InplaceDemoter : TypeVarOnceVisitor bool visit(TypeId ty, const BoundTypeVar& btyRef) override { + LUAU_ASSERT(!FFlag::LuauInplaceDemoteSkipAllBound); return true; } @@ -656,7 +656,7 @@ void TypeChecker::checkBlockWithoutRecursionCheck(const ScopePtr& scope, const A TypeId leftType = follow(checkFunctionName(scope, *fun->name, funScope->level)); - unify(funTy, leftType, fun->location); + unify(funTy, leftType, scope, fun->location); } else if (auto fun = (*protoIter)->as()) { @@ -768,20 +768,20 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatIf& statement) } template -ErrorVec TypeChecker::canUnify_(Id subTy, Id superTy, const Location& location) +ErrorVec TypeChecker::canUnify_(Id subTy, Id superTy, const ScopePtr& scope, const Location& location) { - Unifier state = mkUnifier(location); + Unifier state = mkUnifier(scope, location); return state.canUnify(subTy, superTy); } -ErrorVec TypeChecker::canUnify(TypeId subTy, TypeId superTy, const Location& location) +ErrorVec TypeChecker::canUnify(TypeId subTy, TypeId superTy, const ScopePtr& scope, const Location& location) { - return canUnify_(subTy, superTy, location); + return canUnify_(subTy, superTy, scope, location); } -ErrorVec TypeChecker::canUnify(TypePackId subTy, TypePackId superTy, const Location& location) +ErrorVec TypeChecker::canUnify(TypePackId subTy, TypePackId superTy, const ScopePtr& scope, const Location& location) { - return canUnify_(subTy, superTy, location); + return canUnify_(subTy, superTy, scope, location); } void TypeChecker::check(const ScopePtr& scope, const AstStatWhile& statement) @@ -802,9 +802,9 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatRepeat& statement) checkExpr(repScope, *statement.condition); } -void TypeChecker::unifyLowerBound(TypePackId subTy, TypePackId superTy, TypeLevel demotedLevel, const Location& location) +void TypeChecker::unifyLowerBound(TypePackId subTy, TypePackId superTy, TypeLevel demotedLevel, const ScopePtr& scope, const Location& location) { - Unifier state = mkUnifier(location); + Unifier state = mkUnifier(scope, location); state.unifyLowerBound(subTy, superTy, demotedLevel); state.log.commit(); @@ -858,8 +858,6 @@ struct Demoter : Substitution void demote(std::vector>& expectedTypes) { - if (!FFlag::LuauQuantifyConstrained) - return; for (std::optional& ty : expectedTypes) { if (ty) @@ -897,7 +895,7 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatReturn& return_) if (useConstrainedIntersections()) { - unifyLowerBound(retPack, scope->returnType, demoter.demotedLevel(scope->level), return_.location); + unifyLowerBound(retPack, scope->returnType, demoter.demotedLevel(scope->level), scope, return_.location); return; } @@ -905,7 +903,7 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatReturn& return_) // start typechecking everything across module boundaries. if (isNonstrictMode() && follow(scope->returnType) == follow(currentModule->getModuleScope()->returnType)) { - ErrorVec errors = tryUnify(retPack, scope->returnType, return_.location); + ErrorVec errors = tryUnify(retPack, scope->returnType, scope, return_.location); if (!errors.empty()) currentModule->getModuleScope()->returnType = addTypePack({anyType}); @@ -913,13 +911,13 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatReturn& return_) return; } - unify(retPack, scope->returnType, return_.location, CountMismatch::Context::Return); + unify(retPack, scope->returnType, scope, return_.location, CountMismatch::Context::Return); } template -ErrorVec TypeChecker::tryUnify_(Id subTy, Id superTy, const Location& location) +ErrorVec TypeChecker::tryUnify_(Id subTy, Id superTy, const ScopePtr& scope, const Location& location) { - Unifier state = mkUnifier(location); + Unifier state = mkUnifier(scope, location); if (FFlag::DebugLuauFreezeDuringUnification) freeze(currentModule->internalTypes); @@ -935,14 +933,14 @@ ErrorVec TypeChecker::tryUnify_(Id subTy, Id superTy, const Location& location) return state.errors; } -ErrorVec TypeChecker::tryUnify(TypeId subTy, TypeId superTy, const Location& location) +ErrorVec TypeChecker::tryUnify(TypeId subTy, TypeId superTy, const ScopePtr& scope, const Location& location) { - return tryUnify_(subTy, superTy, location); + return tryUnify_(subTy, superTy, scope, location); } -ErrorVec TypeChecker::tryUnify(TypePackId subTy, TypePackId superTy, const Location& location) +ErrorVec TypeChecker::tryUnify(TypePackId subTy, TypePackId superTy, const ScopePtr& scope, const Location& location) { - return tryUnify_(subTy, superTy, location); + return tryUnify_(subTy, superTy, scope, location); } void TypeChecker::check(const ScopePtr& scope, const AstStatAssign& assign) @@ -1036,9 +1034,9 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatAssign& assign) { // In nonstrict mode, any assignments where the lhs is free and rhs isn't a function, we give it any typevar. if (isNonstrictMode() && get(follow(left)) && !get(follow(right))) - unify(anyType, left, loc); + unify(anyType, left, scope, loc); else - unify(right, left, loc); + unify(right, left, scope, loc); } } } @@ -1053,7 +1051,7 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatCompoundAssign& assi TypeId result = checkBinaryOperation(scope, expr, left, right); - unify(result, left, assign.location); + unify(result, left, scope, assign.location); } void TypeChecker::check(const ScopePtr& scope, const AstStatLocal& local) @@ -1108,7 +1106,7 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatLocal& local) TypePackId valuePack = checkExprList(scope, local.location, local.values, /* substituteFreeForNil= */ true, instantiateGenerics, expectedTypes).type; - Unifier state = mkUnifier(local.location); + Unifier state = mkUnifier(scope, local.location); state.ctx = CountMismatch::Result; state.tryUnify(valuePack, variablePack); reportErrors(state.errors); @@ -1184,7 +1182,7 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatFor& expr) TypeId loopVarType = numberType; if (expr.var->annotation) - unify(loopVarType, resolveType(scope, *expr.var->annotation), expr.location); + unify(loopVarType, resolveType(scope, *expr.var->annotation), scope, expr.location); loopScope->bindings[expr.var] = {loopVarType, expr.var->location}; @@ -1194,11 +1192,11 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatFor& expr) if (!expr.to) ice("Bad AstStatFor has no to expr"); - unify(checkExpr(loopScope, *expr.from).type, loopVarType, expr.from->location); - unify(checkExpr(loopScope, *expr.to).type, loopVarType, expr.to->location); + unify(checkExpr(loopScope, *expr.from).type, loopVarType, scope, expr.from->location); + unify(checkExpr(loopScope, *expr.to).type, loopVarType, scope, expr.to->location); if (expr.step) - unify(checkExpr(loopScope, *expr.step).type, loopVarType, expr.step->location); + unify(checkExpr(loopScope, *expr.step).type, loopVarType, scope, expr.step->location); check(loopScope, *expr.body); } @@ -1251,12 +1249,12 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatForIn& forin) if (get(callRetPack)) { iterTy = freshType(scope); - unify(callRetPack, addTypePack({{iterTy}, freshTypePack(scope)}), forin.location); + unify(callRetPack, addTypePack({{iterTy}, freshTypePack(scope)}), scope, forin.location); } else if (get(callRetPack) || !first(callRetPack)) { for (TypeId var : varTypes) - unify(errorRecoveryType(scope), var, forin.location); + unify(errorRecoveryType(scope), var, scope, forin.location); return check(loopScope, *forin.body); } @@ -1277,7 +1275,7 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatForIn& forin) // TODO: this needs to typecheck all returned values by __iter as if they were for loop arguments // the structure of the function makes it difficult to do this especially since we don't have actual expressions, only types for (TypeId var : varTypes) - unify(anyType, var, forin.location); + unify(anyType, var, scope, forin.location); return check(loopScope, *forin.body); } @@ -1289,25 +1287,25 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatForIn& forin) if (iterTable->indexer) { if (varTypes.size() > 0) - unify(iterTable->indexer->indexType, varTypes[0], forin.location); + unify(iterTable->indexer->indexType, varTypes[0], scope, forin.location); if (varTypes.size() > 1) - unify(iterTable->indexer->indexResultType, varTypes[1], forin.location); + unify(iterTable->indexer->indexResultType, varTypes[1], scope, forin.location); for (size_t i = 2; i < varTypes.size(); ++i) - unify(nilType, varTypes[i], forin.location); + unify(nilType, varTypes[i], scope, forin.location); } else if (isNonstrictMode()) { for (TypeId var : varTypes) - unify(anyType, var, forin.location); + unify(anyType, var, scope, forin.location); } else { TypeId varTy = errorRecoveryType(loopScope); for (TypeId var : varTypes) - unify(varTy, var, forin.location); + unify(varTy, var, scope, forin.location); reportError(firstValue->location, GenericError{"Cannot iterate over a table without indexer"}); } @@ -1321,7 +1319,7 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatForIn& forin) TypeId varTy = get(iterTy) ? anyType : errorRecoveryType(loopScope); for (TypeId var : varTypes) - unify(varTy, var, forin.location); + unify(varTy, var, scope, forin.location); if (!get(iterTy) && !get(iterTy) && !get(iterTy) && !get(iterTy)) reportError(firstValue->location, CannotCallNonFunction{iterTy}); @@ -1346,7 +1344,7 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatForIn& forin) argPack = addTypePack(TypePack{}); } - Unifier state = mkUnifier(firstValue->location); + Unifier state = mkUnifier(loopScope, firstValue->location); checkArgumentList(loopScope, state, argPack, iterFunc->argTypes, /*argLocations*/ {}); state.log.commit(); @@ -1365,10 +1363,10 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatForIn& forin) AstExprCall exprCall{Location(start, end), firstValue, arguments, /* self= */ false, Location()}; TypePackId retPack = checkExprPack(scope, exprCall).type; - unify(retPack, varPack, forin.location); + unify(retPack, varPack, scope, forin.location); } else - unify(iterFunc->retTypes, varPack, forin.location); + unify(iterFunc->retTypes, varPack, scope, forin.location); check(loopScope, *forin.body); } @@ -1603,7 +1601,7 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatTypeAlias& typealias TypeId& bindingType = bindingsMap[name].type; - if (unify(ty, bindingType, typealias.location)) + if (unify(ty, bindingType, aliasScope, typealias.location)) bindingType = ty; if (FFlag::LuauLowerBoundsCalculation) @@ -1891,7 +1889,7 @@ WithPredicate TypeChecker::checkExpr(const ScopePtr& scope, const AstExp TypeLevel level = FFlag::LuauLowerBoundsCalculation ? ftp->level : scope->level; TypeId head = freshType(level); TypePackId pack = addTypePack(TypePackVar{TypePack{{head}, freshTypePack(level)}}); - unify(pack, retPack, expr.location); + unify(pack, retPack, scope, expr.location); return {head, std::move(result.predicates)}; } if (get(retPack)) @@ -1983,20 +1981,15 @@ std::optional TypeChecker::getIndexTypeFromTypeImpl( else if (auto indexer = tableType->indexer) { // TODO: Property lookup should work with string singletons or unions thereof as the indexer key type. - ErrorVec errors = tryUnify(stringType, indexer->indexType, location); + ErrorVec errors = tryUnify(stringType, indexer->indexType, scope, location); - if (FFlag::LuauReportErrorsOnIndexerKeyMismatch) - { - if (errors.empty()) - return indexer->indexResultType; - - if (addErrors) - reportError(location, UnknownProperty{type, name}); - - return std::nullopt; - } - else + if (errors.empty()) return indexer->indexResultType; + + if (addErrors) + reportError(location, UnknownProperty{type, name}); + + return std::nullopt; } else if (tableType->state == TableState::Free) { @@ -2228,8 +2221,8 @@ TypeId TypeChecker::checkExprTable( if (indexer) { - unify(numberType, indexer->indexType, value->location); - unify(valueType, indexer->indexResultType, value->location); + unify(numberType, indexer->indexType, scope, value->location); + unify(valueType, indexer->indexResultType, scope, value->location); } else indexer = TableIndexer{numberType, anyIfNonstrict(valueType)}; @@ -2248,13 +2241,13 @@ TypeId TypeChecker::checkExprTable( if (it != expectedTable->props.end()) { Property expectedProp = it->second; - ErrorVec errors = tryUnify(exprType, expectedProp.type, k->location); + ErrorVec errors = tryUnify(exprType, expectedProp.type, scope, k->location); if (errors.empty()) exprType = expectedProp.type; } else if (expectedTable->indexer && maybeString(expectedTable->indexer->indexType)) { - ErrorVec errors = tryUnify(exprType, expectedTable->indexer->indexResultType, k->location); + ErrorVec errors = tryUnify(exprType, expectedTable->indexer->indexResultType, scope, k->location); if (errors.empty()) exprType = expectedTable->indexer->indexResultType; } @@ -2269,8 +2262,8 @@ TypeId TypeChecker::checkExprTable( if (indexer) { - unify(keyType, indexer->indexType, k->location); - unify(valueType, indexer->indexResultType, value->location); + unify(keyType, indexer->indexType, scope, k->location); + unify(valueType, indexer->indexResultType, scope, value->location); } else if (isNonstrictMode()) { @@ -2411,7 +2404,7 @@ WithPredicate TypeChecker::checkExpr(const ScopePtr& scope, const AstExp TypePackId retTypePack = freshTypePack(scope); TypeId expectedFunctionType = addType(FunctionTypeVar(scope->level, arguments, retTypePack)); - Unifier state = mkUnifier(expr.location); + Unifier state = mkUnifier(scope, expr.location); state.tryUnify(actualFunctionType, expectedFunctionType, /*isFunctionCall*/ true); state.log.commit(); @@ -2429,7 +2422,7 @@ WithPredicate TypeChecker::checkExpr(const ScopePtr& scope, const AstExp return {errorRecoveryType(scope)}; } - reportErrors(tryUnify(operandType, numberType, expr.location)); + reportErrors(tryUnify(operandType, numberType, scope, expr.location)); return {numberType}; } case AstExprUnary::Len: @@ -2459,7 +2452,7 @@ WithPredicate TypeChecker::checkExpr(const ScopePtr& scope, const AstExp TypePackId retTypePack = addTypePack({numberType}); TypeId expectedFunctionType = addType(FunctionTypeVar(scope->level, arguments, retTypePack)); - Unifier state = mkUnifier(expr.location); + Unifier state = mkUnifier(scope, expr.location); state.tryUnify(actualFunctionType, expectedFunctionType, /*isFunctionCall*/ true); state.log.commit(); @@ -2509,11 +2502,11 @@ std::string opToMetaTableEntry(const AstExprBinary::Op& op) } } -TypeId TypeChecker::unionOfTypes(TypeId a, TypeId b, const Location& location, bool unifyFreeTypes) +TypeId TypeChecker::unionOfTypes(TypeId a, TypeId b, const ScopePtr& scope, const Location& location, bool unifyFreeTypes) { if (unifyFreeTypes && (get(a) || get(b))) { - if (unify(b, a, location)) + if (unify(b, a, scope, location)) return a; return errorRecoveryType(anyType); @@ -2588,7 +2581,7 @@ TypeId TypeChecker::checkRelationalOperation( { ScopePtr subScope = childScope(scope, subexp->location); resolve(predicates, subScope, true); - return unionOfTypes(rhsType, stripNil(checkExpr(subScope, *subexp->right).type, true), expr.location); + return unionOfTypes(rhsType, stripNil(checkExpr(subScope, *subexp->right).type, true), subScope, expr.location); } } @@ -2624,7 +2617,7 @@ TypeId TypeChecker::checkRelationalOperation( * report any problems that might have been surfaced as a result of this step because we might already * have a better, more descriptive error teed up. */ - Unifier state = mkUnifier(expr.location); + Unifier state = mkUnifier(scope, expr.location); if (!isEquality) { state.tryUnify(rhsType, lhsType); @@ -2703,7 +2696,7 @@ TypeId TypeChecker::checkRelationalOperation( { if (isEquality) { - Unifier state = mkUnifier(expr.location); + Unifier state = mkUnifier(scope, expr.location); state.tryUnify(addTypePack({booleanType}), ftv->retTypes); if (!state.errors.empty()) @@ -2755,11 +2748,11 @@ TypeId TypeChecker::checkRelationalOperation( case AstExprBinary::And: if (lhsIsAny) return lhsType; - return unionOfTypes(rhsType, booleanType, expr.location, false); + return unionOfTypes(rhsType, booleanType, scope, expr.location, false); case AstExprBinary::Or: if (lhsIsAny) return lhsType; - return unionOfTypes(lhsType, rhsType, expr.location); + return unionOfTypes(lhsType, rhsType, scope, expr.location); default: LUAU_ASSERT(0); ice(format("checkRelationalOperation called with incorrect binary expression '%s'", toString(expr.op).c_str()), expr.location); @@ -2816,7 +2809,7 @@ TypeId TypeChecker::checkBinaryOperation( } if (get(rhsType)) - unify(rhsType, lhsType, expr.location); + unify(rhsType, lhsType, scope, expr.location); if (typeCouldHaveMetatable(lhsType) || typeCouldHaveMetatable(rhsType)) { @@ -2826,7 +2819,7 @@ TypeId TypeChecker::checkBinaryOperation( TypePackId retTypePack = freshTypePack(scope); TypeId expectedFunctionType = addType(FunctionTypeVar(scope->level, arguments, retTypePack)); - Unifier state = mkUnifier(expr.location); + Unifier state = mkUnifier(scope, expr.location); state.tryUnify(actualFunctionType, expectedFunctionType, /*isFunctionCall*/ true); reportErrors(state.errors); @@ -2876,8 +2869,8 @@ TypeId TypeChecker::checkBinaryOperation( switch (expr.op) { case AstExprBinary::Concat: - reportErrors(tryUnify(lhsType, addType(UnionTypeVar{{stringType, numberType}}), expr.left->location)); - reportErrors(tryUnify(rhsType, addType(UnionTypeVar{{stringType, numberType}}), expr.right->location)); + reportErrors(tryUnify(lhsType, addType(UnionTypeVar{{stringType, numberType}}), scope, expr.left->location)); + reportErrors(tryUnify(rhsType, addType(UnionTypeVar{{stringType, numberType}}), scope, expr.right->location)); return stringType; case AstExprBinary::Add: case AstExprBinary::Sub: @@ -2885,8 +2878,8 @@ TypeId TypeChecker::checkBinaryOperation( case AstExprBinary::Div: case AstExprBinary::Mod: case AstExprBinary::Pow: - reportErrors(tryUnify(lhsType, numberType, expr.left->location)); - reportErrors(tryUnify(rhsType, numberType, expr.right->location)); + reportErrors(tryUnify(lhsType, numberType, scope, expr.left->location)); + reportErrors(tryUnify(rhsType, numberType, scope, expr.right->location)); return numberType; default: // These should have been handled with checkRelationalOperation @@ -2961,10 +2954,10 @@ WithPredicate TypeChecker::checkExpr(const ScopePtr& scope, const AstExp WithPredicate result = checkExpr(scope, *expr.expr, annotationType); // Note: As an optimization, we try 'number <: number | string' first, as that is the more likely case. - if (canUnify(annotationType, result.type, expr.location).empty()) + if (canUnify(annotationType, result.type, scope, expr.location).empty()) return {annotationType, std::move(result.predicates)}; - if (canUnify(result.type, annotationType, expr.location).empty()) + if (canUnify(result.type, annotationType, scope, expr.location).empty()) return {annotationType, std::move(result.predicates)}; reportError(expr.location, TypesAreUnrelated{result.type, annotationType}); @@ -3101,7 +3094,7 @@ TypeId TypeChecker::checkLValueBinding(const ScopePtr& scope, const AstExprIndex } else if (auto indexer = lhsTable->indexer) { - Unifier state = mkUnifier(expr.location); + Unifier state = mkUnifier(scope, expr.location); state.tryUnify(stringType, indexer->indexType); TypeId retType = indexer->indexResultType; if (!state.errors.empty()) @@ -3213,7 +3206,7 @@ TypeId TypeChecker::checkLValueBinding(const ScopePtr& scope, const AstExprIndex if (exprTable->indexer) { const TableIndexer& indexer = *exprTable->indexer; - unify(indexType, indexer.indexType, expr.index->location); + unify(indexType, indexer.indexType, scope, expr.index->location); return indexer.indexResultType; } else if (exprTable->state == TableState::Unsealed || exprTable->state == TableState::Free) @@ -3829,7 +3822,7 @@ void TypeChecker::checkArgumentList( { // The use of unify here is deliberate. We don't want this unification // to be undoable. - unify(errorRecoveryType(scope), *argIter, state.location); + unify(errorRecoveryType(scope), *argIter, scope, state.location); ++argIter; } reportCountMismatchError(); @@ -3852,7 +3845,7 @@ void TypeChecker::checkArgumentList( TypeId e = errorRecoveryType(scope); while (argIter != endIter) { - unify(e, *argIter, state.location); + unify(e, *argIter, scope, state.location); ++argIter; } @@ -3869,7 +3862,7 @@ void TypeChecker::checkArgumentList( if (argIndex < argLocations.size()) location = argLocations[argIndex]; - unify(*argIter, vtp->ty, location); + unify(*argIter, vtp->ty, scope, location); ++argIter; ++argIndex; } @@ -3906,7 +3899,7 @@ void TypeChecker::checkArgumentList( } else { - unifyWithInstantiationIfNeeded(scope, *argIter, *paramIter, state); + unifyWithInstantiationIfNeeded(*argIter, *paramIter, scope, state); ++argIter; ++paramIter; } @@ -4114,7 +4107,7 @@ std::optional> TypeChecker::checkCallOverload(const Sc if (get(fn)) { - unify(anyTypePack, argPack, expr.location); + unify(anyTypePack, argPack, scope, expr.location); return {{anyTypePack}}; } @@ -4160,7 +4153,7 @@ std::optional> TypeChecker::checkCallOverload(const Sc UnifierOptions options; options.isFunctionCall = true; - unify(r, fn, expr.location, options); + unify(r, fn, scope, expr.location, options); return {{retPack}}; } @@ -4194,7 +4187,7 @@ std::optional> TypeChecker::checkCallOverload(const Sc if (!ftv) { reportError(TypeError{expr.func->location, CannotCallNonFunction{fn}}); - unify(errorRecoveryTypePack(scope), retPack, expr.func->location); + unify(errorRecoveryTypePack(scope), retPack, scope, expr.func->location); return {{errorRecoveryTypePack(retPack)}}; } @@ -4207,7 +4200,7 @@ std::optional> TypeChecker::checkCallOverload(const Sc return *ret; } - Unifier state = mkUnifier(expr.location); + Unifier state = mkUnifier(scope, expr.location); // Unify return types checkArgumentList(scope, state, retPack, ftv->retTypes, /*argLocations*/ {}); @@ -4269,7 +4262,7 @@ bool TypeChecker::handleSelfCallMismatch(const ScopePtr& scope, const AstExprCal std::vector editedParamList(args->head.begin() + 1, args->head.end()); TypePackId editedArgPack = addTypePack(TypePack{editedParamList}); - Unifier editedState = mkUnifier(expr.location); + Unifier editedState = mkUnifier(scope, expr.location); checkArgumentList(scope, editedState, editedArgPack, ftv->argTypes, editedArgLocations); if (editedState.errors.empty()) @@ -4299,7 +4292,7 @@ bool TypeChecker::handleSelfCallMismatch(const ScopePtr& scope, const AstExprCal editedArgList.insert(editedArgList.begin(), checkExpr(scope, *indexName->expr).type); TypePackId editedArgPack = addTypePack(TypePack{editedArgList}); - Unifier editedState = mkUnifier(expr.location); + Unifier editedState = mkUnifier(scope, expr.location); checkArgumentList(scope, editedState, editedArgPack, ftv->argTypes, editedArgLocations); @@ -4365,7 +4358,7 @@ void TypeChecker::reportOverloadResolutionError(const ScopePtr& scope, const Ast for (size_t i = 0; i < overloadTypes.size(); ++i) { TypeId overload = overloadTypes[i]; - Unifier state = mkUnifier(expr.location); + Unifier state = mkUnifier(scope, expr.location); // Unify return types if (const FunctionTypeVar* ftv = get(overload)) @@ -4415,7 +4408,7 @@ WithPredicate TypeChecker::checkExprList(const ScopePtr& scope, cons size_t lastIndex = exprs.size - 1; tp->head.reserve(lastIndex); - Unifier state = mkUnifier(location); + Unifier state = mkUnifier(scope, location); std::vector inverseLogs; @@ -4580,15 +4573,15 @@ TypeId TypeChecker::anyIfNonstrict(TypeId ty) const return ty; } -bool TypeChecker::unify(TypeId subTy, TypeId superTy, const Location& location) +bool TypeChecker::unify(TypeId subTy, TypeId superTy, const ScopePtr& scope, const Location& location) { UnifierOptions options; - return unify(subTy, superTy, location, options); + return unify(subTy, superTy, scope, location, options); } -bool TypeChecker::unify(TypeId subTy, TypeId superTy, const Location& location, const UnifierOptions& options) +bool TypeChecker::unify(TypeId subTy, TypeId superTy, const ScopePtr& scope, const Location& location, const UnifierOptions& options) { - Unifier state = mkUnifier(location); + Unifier state = mkUnifier(scope, location); state.tryUnify(subTy, superTy, options.isFunctionCall); state.log.commit(); @@ -4598,9 +4591,9 @@ bool TypeChecker::unify(TypeId subTy, TypeId superTy, const Location& location, return state.errors.empty(); } -bool TypeChecker::unify(TypePackId subTy, TypePackId superTy, const Location& location, CountMismatch::Context ctx) +bool TypeChecker::unify(TypePackId subTy, TypePackId superTy, const ScopePtr& scope, const Location& location, CountMismatch::Context ctx) { - Unifier state = mkUnifier(location); + Unifier state = mkUnifier(scope, location); state.ctx = ctx; state.tryUnify(subTy, superTy); @@ -4611,10 +4604,10 @@ bool TypeChecker::unify(TypePackId subTy, TypePackId superTy, const Location& lo return state.errors.empty(); } -bool TypeChecker::unifyWithInstantiationIfNeeded(const ScopePtr& scope, TypeId subTy, TypeId superTy, const Location& location) +bool TypeChecker::unifyWithInstantiationIfNeeded(TypeId subTy, TypeId superTy, const ScopePtr& scope, const Location& location) { - Unifier state = mkUnifier(location); - unifyWithInstantiationIfNeeded(scope, subTy, superTy, state); + Unifier state = mkUnifier(scope, location); + unifyWithInstantiationIfNeeded(subTy, superTy, scope, state); state.log.commit(); @@ -4623,7 +4616,7 @@ bool TypeChecker::unifyWithInstantiationIfNeeded(const ScopePtr& scope, TypeId s return state.errors.empty(); } -void TypeChecker::unifyWithInstantiationIfNeeded(const ScopePtr& scope, TypeId subTy, TypeId superTy, Unifier& state) +void TypeChecker::unifyWithInstantiationIfNeeded(TypeId subTy, TypeId superTy, const ScopePtr& scope, Unifier& state) { if (!maybeGeneric(subTy)) // Quick check to see if we definitely can't instantiate @@ -4662,77 +4655,6 @@ void TypeChecker::unifyWithInstantiationIfNeeded(const ScopePtr& scope, TypeId s } } -bool Anyification::isDirty(TypeId ty) -{ - if (ty->persistent) - return false; - - if (const TableTypeVar* ttv = log->getMutable(ty)) - return (ttv->state == TableState::Free || ttv->state == TableState::Unsealed); - else if (log->getMutable(ty)) - return true; - else if (get(ty)) - return true; - else - return false; -} - -bool Anyification::isDirty(TypePackId tp) -{ - if (tp->persistent) - return false; - - if (log->getMutable(tp)) - return true; - else - return false; -} - -TypeId Anyification::clean(TypeId ty) -{ - LUAU_ASSERT(isDirty(ty)); - if (const TableTypeVar* ttv = log->getMutable(ty)) - { - TableTypeVar clone = TableTypeVar{ttv->props, ttv->indexer, ttv->level, TableState::Sealed}; - clone.definitionModuleName = ttv->definitionModuleName; - clone.name = ttv->name; - clone.syntheticName = ttv->syntheticName; - clone.tags = ttv->tags; - TypeId res = addType(std::move(clone)); - asMutable(res)->normal = ty->normal; - return res; - } - else if (auto ctv = get(ty)) - { - if (FFlag::LuauQuantifyConstrained) - { - std::vector copy = ctv->parts; - for (TypeId& ty : copy) - ty = replace(ty); - TypeId res = copy.size() == 1 ? copy[0] : addType(UnionTypeVar{std::move(copy)}); - auto [t, ok] = normalize(res, *arena, *iceHandler); - if (!ok) - normalizationTooComplex = true; - return t; - } - else - { - auto [t, ok] = normalize(ty, *arena, *iceHandler); - if (!ok) - normalizationTooComplex = true; - return t; - } - } - else - return anyType; -} - -TypePackId Anyification::clean(TypePackId tp) -{ - LUAU_ASSERT(isDirty(tp)); - return anyTypePack; -} - TypeId TypeChecker::quantify(const ScopePtr& scope, TypeId ty, Location location) { ty = follow(ty); @@ -4804,7 +4726,7 @@ TypeId TypeChecker::anyify(const ScopePtr& scope, TypeId ty, Location location) ty = t; } - Anyification anyification{¤tModule->internalTypes, iceHandler, anyType, anyTypePack}; + Anyification anyification{¤tModule->internalTypes, scope, iceHandler, anyType, anyTypePack}; std::optional any = anyification.substitute(ty); if (anyification.normalizationTooComplex) reportError(location, NormalizationTooComplex{}); @@ -4827,7 +4749,7 @@ TypePackId TypeChecker::anyify(const ScopePtr& scope, TypePackId ty, Location lo ty = t; } - Anyification anyification{¤tModule->internalTypes, iceHandler, anyType, anyTypePack}; + Anyification anyification{¤tModule->internalTypes, scope, iceHandler, anyType, anyTypePack}; std::optional any = anyification.substitute(ty); if (any.has_value()) return *any; @@ -4963,9 +4885,9 @@ void TypeChecker::merge(RefinementMap& l, const RefinementMap& r) }); } -Unifier TypeChecker::mkUnifier(const Location& location) +Unifier TypeChecker::mkUnifier(const ScopePtr& scope, const Location& location) { - return Unifier{¤tModule->internalTypes, currentModule->mode, location, Variance::Covariant, unifierState}; + return Unifier{¤tModule->internalTypes, currentModule->mode, NotNull{scope.get()}, location, Variance::Covariant, unifierState}; } TypeId TypeChecker::freshType(const ScopePtr& scope) @@ -5029,10 +4951,7 @@ TypeIdPredicate TypeChecker::mkTruthyPredicate(bool sense) return sense ? std::nullopt : std::optional(ty); // at this point, anything else is kept if sense is true, or replaced by nil - if (FFlag::LuauFalsyPredicateReturnsNilInstead) - return sense ? ty : nilType; - else - return sense ? std::optional(ty) : std::nullopt; + return sense ? ty : nilType; }; } @@ -5875,8 +5794,8 @@ void TypeChecker::resolve(const IsAPredicate& isaP, RefinementMap& refis, const { auto predicate = [&](TypeId option) -> std::optional { // This by itself is not truly enough to determine that A is stronger than B or vice versa. - bool optionIsSubtype = canUnify(option, isaP.ty, isaP.location).empty(); - bool targetIsSubtype = canUnify(isaP.ty, option, isaP.location).empty(); + bool optionIsSubtype = canUnify(option, isaP.ty, scope, isaP.location).empty(); + bool targetIsSubtype = canUnify(isaP.ty, option, scope, isaP.location).empty(); // If A is a superset of B, then if sense is true, we promote A to B, otherwise we keep A. if (!optionIsSubtype && targetIsSubtype) @@ -6019,7 +5938,7 @@ void TypeChecker::resolve(const EqPredicate& eqP, RefinementMap& refis, const Sc if (maybeSingleton(eqP.type)) { // Normally we'd write option <: eqP.type, but singletons are always the subtype, so we flip this. - if (!sense || canUnify(eqP.type, option, eqP.location).empty()) + if (!sense || canUnify(eqP.type, option, scope, eqP.location).empty()) return sense ? eqP.type : option; // local variable works around an odd gcc 9.3 warning: may be used uninitialized @@ -6053,7 +5972,7 @@ std::vector TypeChecker::unTypePack(const ScopePtr& scope, TypePackId tp size_t oldErrorsSize = currentModule->errors.size(); - unify(tp, expectedTypePack, location); + unify(tp, expectedTypePack, scope, location); // HACK: tryUnify would undo the changes to the expectedTypePack if the length mismatches, but // we want to tie up free types to be error types, so we do this instead. diff --git a/Analysis/src/TypeUtils.cpp b/Analysis/src/TypeUtils.cpp index 66b38cf3..60bca0a3 100644 --- a/Analysis/src/TypeUtils.cpp +++ b/Analysis/src/TypeUtils.cpp @@ -1,6 +1,7 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #include "Luau/TypeUtils.h" +#include "Luau/Normalize.h" #include "Luau/Scope.h" #include "Luau/ToString.h" #include "Luau/TypeInfer.h" @@ -8,7 +9,7 @@ namespace Luau { -std::optional findMetatableEntry(ErrorVec& errors, TypeId type, std::string entry, Location location) +std::optional findMetatableEntry(ErrorVec& errors, TypeId type, const std::string& entry, Location location) { type = follow(type); @@ -35,7 +36,7 @@ std::optional findMetatableEntry(ErrorVec& errors, TypeId type, std::str return std::nullopt; } -std::optional findTablePropertyRespectingMeta(ErrorVec& errors, TypeId ty, Name name, Location location) +std::optional findTablePropertyRespectingMeta(ErrorVec& errors, TypeId ty, const std::string& name, Location location) { if (get(ty)) return ty; @@ -83,4 +84,110 @@ std::optional findTablePropertyRespectingMeta(ErrorVec& errors, TypeId t return std::nullopt; } +std::optional getIndexTypeFromType( + const ScopePtr& scope, ErrorVec& errors, TypeArena* arena, TypeId type, const std::string& prop, const Location& location, bool addErrors, + InternalErrorReporter& handle) +{ + type = follow(type); + + if (get(type) || get(type) || get(type)) + return type; + + if (auto f = get(type)) + *asMutable(type) = TableTypeVar{TableState::Free, f->level}; + + if (isString(type)) + { + std::optional mtIndex = Luau::findMetatableEntry(errors, getSingletonTypes().stringType, "__index", location); + LUAU_ASSERT(mtIndex); + type = *mtIndex; + } + + if (getTableType(type)) + { + return findTablePropertyRespectingMeta(errors, type, prop, location); + } + else if (const ClassTypeVar* cls = get(type)) + { + if (const Property* p = lookupClassProp(cls, prop)) + return p->type; + } + else if (const UnionTypeVar* utv = get(type)) + { + std::vector goodOptions; + std::vector badOptions; + + for (TypeId t : utv) + { + // TODO: we should probably limit recursion here? + // RecursionLimiter _rl(&recursionCount, FInt::LuauTypeInferRecursionLimit); + + // Not needed when we normalize types. + if (get(follow(t))) + return t; + + if (std::optional ty = getIndexTypeFromType(scope, errors, arena, t, prop, location, /* addErrors= */ false, handle)) + goodOptions.push_back(*ty); + else + badOptions.push_back(t); + } + + if (!badOptions.empty()) + { + if (addErrors) + { + if (goodOptions.empty()) + errors.push_back(TypeError{location, UnknownProperty{type, prop}}); + else + errors.push_back(TypeError{location, MissingUnionProperty{type, badOptions, prop}}); + } + return std::nullopt; + } + + if (goodOptions.empty()) + return getSingletonTypes().neverType; + + if (goodOptions.size() == 1) + return goodOptions[0]; + + // TODO: inefficient. + TypeId result = arena->addType(UnionTypeVar{std::move(goodOptions)}); + auto [ty, ok] = normalize(result, NotNull{scope.get()}, *arena, handle); + if (!ok && addErrors) + errors.push_back(TypeError{location, NormalizationTooComplex{}}); + return ok ? ty : getSingletonTypes().anyType; + } + else if (const IntersectionTypeVar* itv = get(type)) + { + std::vector parts; + + for (TypeId t : itv->parts) + { + // TODO: we should probably limit recursion here? + // RecursionLimiter _rl(&recursionCount, FInt::LuauTypeInferRecursionLimit); + + if (std::optional ty = getIndexTypeFromType(scope, errors, arena, t, prop, location, /* addErrors= */ false, handle)) + parts.push_back(*ty); + } + + // If no parts of the intersection had the property we looked up for, it never existed at all. + if (parts.empty()) + { + if (addErrors) + errors.push_back(TypeError{location, UnknownProperty{type, prop}}); + return std::nullopt; + } + + if (parts.size() == 1) + return parts[0]; + + return arena->addType(IntersectionTypeVar{std::move(parts)}); // Not at all correct. + } + + if (addErrors) + errors.push_back(TypeError{location, UnknownProperty{type, prop}}); + + return std::nullopt; +} + } // namespace Luau diff --git a/Analysis/src/TypeVar.cpp b/Analysis/src/TypeVar.cpp index ada2b012..9020b1ac 100644 --- a/Analysis/src/TypeVar.cpp +++ b/Analysis/src/TypeVar.cpp @@ -1135,7 +1135,7 @@ std::optional> magicFunctionFormat( { Location location = expr.args.data[std::min(i + dataOffset, expr.args.size - 1)]->location; - typechecker.unify(params[i + paramOffset], expected[i], location); + typechecker.unify(params[i + paramOffset], expected[i], scope, location); } // if we know the argument count or if we have too many arguments for sure, we can issue an error @@ -1234,7 +1234,7 @@ static std::optional> magicFunctionGmatch( if (returnTypes.empty()) return std::nullopt; - typechecker.unify(params[0], typechecker.stringType, expr.args.data[0]->location); + typechecker.unify(params[0], typechecker.stringType, scope, expr.args.data[0]->location); const TypePackId emptyPack = arena.addTypePack({}); const TypePackId returnList = arena.addTypePack(returnTypes); @@ -1269,13 +1269,13 @@ static std::optional> magicFunctionMatch( if (returnTypes.empty()) return std::nullopt; - typechecker.unify(params[0], typechecker.stringType, expr.args.data[0]->location); + typechecker.unify(params[0], typechecker.stringType, scope, expr.args.data[0]->location); const TypeId optionalNumber = arena.addType(UnionTypeVar{{typechecker.nilType, typechecker.numberType}}); size_t initIndex = expr.self ? 1 : 2; if (params.size() == 3 && expr.args.size > initIndex) - typechecker.unify(params[2], optionalNumber, expr.args.data[initIndex]->location); + typechecker.unify(params[2], optionalNumber, scope, expr.args.data[initIndex]->location); const TypePackId returnList = arena.addTypePack(returnTypes); return WithPredicate{returnList}; @@ -1320,17 +1320,17 @@ static std::optional> magicFunctionFind( return std::nullopt; } - typechecker.unify(params[0], typechecker.stringType, expr.args.data[0]->location); + typechecker.unify(params[0], typechecker.stringType, scope, expr.args.data[0]->location); const TypeId optionalNumber = arena.addType(UnionTypeVar{{typechecker.nilType, typechecker.numberType}}); const TypeId optionalBoolean = arena.addType(UnionTypeVar{{typechecker.nilType, typechecker.booleanType}}); size_t initIndex = expr.self ? 1 : 2; if (params.size() >= 3 && expr.args.size > initIndex) - typechecker.unify(params[2], optionalNumber, expr.args.data[initIndex]->location); + typechecker.unify(params[2], optionalNumber, scope, expr.args.data[initIndex]->location); if (params.size() == 4 && expr.args.size > plainIndex) - typechecker.unify(params[3], optionalBoolean, expr.args.data[plainIndex]->location); + typechecker.unify(params[3], optionalBoolean, scope, expr.args.data[plainIndex]->location); returnTypes.insert(returnTypes.begin(), {optionalNumber, optionalNumber}); diff --git a/Analysis/src/TypedAllocator.cpp b/Analysis/src/TypedAllocator.cpp index c7f31822..9ce8c3dc 100644 --- a/Analysis/src/TypedAllocator.cpp +++ b/Analysis/src/TypedAllocator.cpp @@ -27,27 +27,6 @@ LUAU_FASTFLAG(DebugLuauFreezeArena) namespace Luau { -static void* systemAllocateAligned(size_t size, size_t align) -{ -#ifdef _WIN32 - return _aligned_malloc(size, align); -#elif defined(__ANDROID__) // for Android 4.1 - return memalign(align, size); -#else - void* ptr; - return posix_memalign(&ptr, align, size) == 0 ? ptr : 0; -#endif -} - -static void systemDeallocateAligned(void* ptr) -{ -#ifdef _WIN32 - _aligned_free(ptr); -#else - free(ptr); -#endif -} - static size_t pageAlign(size_t size) { return (size + kPageSize - 1) & ~(kPageSize - 1); @@ -55,18 +34,31 @@ static size_t pageAlign(size_t size) void* pagedAllocate(size_t size) { - if (FFlag::DebugLuauFreezeArena) - return systemAllocateAligned(pageAlign(size), kPageSize); - else + // By default we use operator new/delete instead of malloc/free so that they can be overridden externally + if (!FFlag::DebugLuauFreezeArena) return ::operator new(size, std::nothrow); + + // On Windows, VirtualAlloc results in 64K granularity allocations; we allocate in chunks of ~32K so aligned_malloc is a little more efficient + // On Linux, we must use mmap because using regular heap results in mprotect() fragmenting the page table and us bumping into 64K mmap limit. +#ifdef _WIN32 + return _aligned_malloc(size, kPageSize); +#else + return mmap(nullptr, pageAlign(size), PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANON, -1, 0); +#endif } -void pagedDeallocate(void* ptr) +void pagedDeallocate(void* ptr, size_t size) { - if (FFlag::DebugLuauFreezeArena) - systemDeallocateAligned(ptr); - else - ::operator delete(ptr); + // By default we use operator new/delete instead of malloc/free so that they can be overridden externally + if (!FFlag::DebugLuauFreezeArena) + return ::operator delete(ptr); + +#ifdef _WIN32 + _aligned_free(ptr); +#else + int rc = munmap(ptr, size); + LUAU_ASSERT(rc == 0); +#endif } void pagedFreeze(void* ptr, size_t size) diff --git a/Analysis/src/Unifier.cpp b/Analysis/src/Unifier.cpp index a0f4725d..b5f58c83 100644 --- a/Analysis/src/Unifier.cpp +++ b/Analysis/src/Unifier.cpp @@ -20,7 +20,6 @@ LUAU_FASTINTVARIABLE(LuauTypeInferLowerBoundsIterationLimit, 2000); LUAU_FASTFLAG(LuauLowerBoundsCalculation); LUAU_FASTFLAG(LuauErrorRecoveryType); LUAU_FASTFLAG(LuauUnknownAndNeverType) -LUAU_FASTFLAG(LuauQuantifyConstrained) LUAU_FASTFLAGVARIABLE(LuauScalarShapeSubtyping, false) LUAU_FASTFLAG(LuauClassTypeVarsInSubstitution) @@ -318,9 +317,10 @@ static std::optional> getTableMat return std::nullopt; } -Unifier::Unifier(TypeArena* types, Mode mode, const Location& location, Variance variance, UnifierSharedState& sharedState, TxnLog* parentLog) +Unifier::Unifier(TypeArena* types, Mode mode, NotNull scope, const Location& location, Variance variance, UnifierSharedState& sharedState, TxnLog* parentLog) : types(types) , mode(mode) + , scope(scope) , log(parentLog) , location(location) , variance(variance) @@ -2091,13 +2091,11 @@ void Unifier::unifyLowerBound(TypePackId subTy, TypePackId superTy, TypeLevel de if (!freeTailPack) return; - TypeLevel level = FFlag::LuauQuantifyConstrained ? demotedLevel : freeTailPack->level; - TypePack* tp = getMutable(log.replace(tailPack, TypePack{})); for (; subIter != subEndIter; ++subIter) { - tp->head.push_back(types->addType(ConstrainedTypeVar{level, {follow(*subIter)}})); + tp->head.push_back(types->addType(ConstrainedTypeVar{demotedLevel, {follow(*subIter)}})); } tp->tail = subIter.tail(); @@ -2270,7 +2268,7 @@ bool Unifier::occursCheck(DenseHashSet& seen, TypePackId needle, Typ Unifier Unifier::makeChildUnifier() { - Unifier u = Unifier{types, mode, location, variance, sharedState, &log}; + Unifier u = Unifier{types, mode, scope, location, variance, sharedState, &log}; u.anyIsTop = anyIsTop; return u; } diff --git a/Ast/include/Luau/Parser.h b/Ast/include/Luau/Parser.h index 4b5ae315..046706d3 100644 --- a/Ast/include/Luau/Parser.h +++ b/Ast/include/Luau/Parser.h @@ -138,7 +138,7 @@ private: // funcbody ::= `(' [parlist] `)' block end // parlist ::= namelist [`,' `...'] | `...' std::pair parseFunctionBody( - bool hasself, const Lexeme& matchFunction, const AstName& debugname, std::optional localName); + bool hasself, const Lexeme& matchFunction, const AstName& debugname, const Name* localName); // explist ::= {exp `,'} exp void parseExprList(TempVector& result); @@ -217,7 +217,7 @@ private: AstExpr* parseSimpleExpr(); // args ::= `(' [explist] `)' | tableconstructor | String - AstExpr* parseFunctionArgs(AstExpr* func, bool self, const Location& selfLocation); + AstExpr* parseFunctionArgs(AstExpr* func, bool self); // tableconstructor ::= `{' [fieldlist] `}' // fieldlist ::= field {fieldsep field} [fieldsep] @@ -241,6 +241,7 @@ private: std::optional> parseCharArray(); AstExpr* parseString(); + AstExpr* parseNumber(); AstLocal* pushLocal(const Binding& binding); @@ -253,11 +254,24 @@ private: bool expectAndConsume(Lexeme::Type type, const char* context = nullptr); void expectAndConsumeFail(Lexeme::Type type, const char* context); - bool expectMatchAndConsume(char value, const Lexeme& begin, bool searchForMissing = false); - void expectMatchAndConsumeFail(Lexeme::Type type, const Lexeme& begin, const char* extra = nullptr); + struct MatchLexeme + { + MatchLexeme(const Lexeme& l) + : type(l.type) + , position(l.location.begin) + { + } - bool expectMatchEndAndConsume(Lexeme::Type type, const Lexeme& begin); - void expectMatchEndAndConsumeFail(Lexeme::Type type, const Lexeme& begin); + Lexeme::Type type; + Position position; + }; + + bool expectMatchAndConsume(char value, const MatchLexeme& begin, bool searchForMissing = false); + void expectMatchAndConsumeFail(Lexeme::Type type, const MatchLexeme& begin, const char* extra = nullptr); + bool expectMatchAndConsumeRecover(char value, const MatchLexeme& begin, bool searchForMissing); + + bool expectMatchEndAndConsume(Lexeme::Type type, const MatchLexeme& begin); + void expectMatchEndAndConsumeFail(Lexeme::Type type, const MatchLexeme& begin); template AstArray copy(const T* data, std::size_t size); @@ -283,6 +297,9 @@ private: AstTypeError* reportTypeAnnotationError(const Location& location, const AstArray& types, bool isMissing, const char* format, ...) LUAU_PRINTF_ATTR(5, 6); + AstExpr* reportFunctionArgsError(AstExpr* func, bool self); + void reportAmbiguousCallError(); + void nextLexeme(); struct Function @@ -350,7 +367,7 @@ private: AstName nameError; AstName nameNil; - Lexeme endMismatchSuspect; + MatchLexeme endMismatchSuspect; std::vector functionStack; diff --git a/Ast/src/Parser.cpp b/Ast/src/Parser.cpp index 1eb9565f..e46eebf3 100644 --- a/Ast/src/Parser.cpp +++ b/Ast/src/Parser.cpp @@ -14,8 +14,6 @@ LUAU_FASTINTVARIABLE(LuauRecursionLimit, 1000) LUAU_FASTINTVARIABLE(LuauParseErrorLimit, 100) -LUAU_FASTFLAGVARIABLE(LuauParserFunctionKeywordAsTypeHelp, false) - LUAU_FASTFLAGVARIABLE(LuauFixNamedFunctionParse, false) LUAU_DYNAMIC_FASTFLAGVARIABLE(LuaReportParseWrongNamedType, false) @@ -177,7 +175,7 @@ Parser::Parser(const char* buffer, size_t bufferSize, AstNameTable& names, Alloc , lexer(buffer, bufferSize, names) , allocator(allocator) , recursionCounter(0) - , endMismatchSuspect(Location(), Lexeme::Eof) + , endMismatchSuspect(Lexeme(Location(), Lexeme::Eof)) , localMap(AstName()) { Function top; @@ -657,7 +655,7 @@ AstStat* Parser::parseFunctionStat() matchRecoveryStopOnToken[Lexeme::ReservedEnd]++; - AstExprFunction* body = parseFunctionBody(hasself, matchFunction, debugname, {}).first; + AstExprFunction* body = parseFunctionBody(hasself, matchFunction, debugname, nullptr).first; matchRecoveryStopOnToken[Lexeme::ReservedEnd]--; @@ -686,7 +684,7 @@ AstStat* Parser::parseLocal() matchRecoveryStopOnToken[Lexeme::ReservedEnd]++; - auto [body, var] = parseFunctionBody(false, matchFunction, name.name, name); + auto [body, var] = parseFunctionBody(false, matchFunction, name.name, &name); matchRecoveryStopOnToken[Lexeme::ReservedEnd]--; @@ -778,7 +776,7 @@ AstDeclaredClassProp Parser::parseDeclaredClassMethod() genericPacks.size = 0; genericPacks.data = nullptr; - Lexeme matchParen = lexer.current(); + MatchLexeme matchParen = lexer.current(); expectAndConsume('(', "function parameter list start"); TempVector args(scratchBinding); @@ -834,7 +832,7 @@ AstStat* Parser::parseDeclaration(const Location& start) auto [generics, genericPacks] = parseGenericTypeList(/* withDefaultValues= */ false); - Lexeme matchParen = lexer.current(); + MatchLexeme matchParen = lexer.current(); expectAndConsume('(', "global function declaration"); @@ -970,13 +968,13 @@ AstStat* Parser::parseCompoundAssignment(AstExpr* initial, AstExprBinary::Op op) // funcbody ::= `(' [parlist] `)' [`:' ReturnType] block end // parlist ::= bindinglist [`,' `...'] | `...' std::pair Parser::parseFunctionBody( - bool hasself, const Lexeme& matchFunction, const AstName& debugname, std::optional localName) + bool hasself, const Lexeme& matchFunction, const AstName& debugname, const Name* localName) { Location start = matchFunction.location; auto [generics, genericPacks] = parseGenericTypeList(/* withDefaultValues= */ false); - Lexeme matchParen = lexer.current(); + MatchLexeme matchParen = lexer.current(); expectAndConsume('(', "function"); TempVector args(scratchBinding); @@ -988,7 +986,7 @@ std::pair Parser::parseFunctionBody( std::tie(vararg, varargAnnotation) = parseBindingList(args, /* allowDot3= */ true); std::optional argLocation = matchParen.type == Lexeme::Type('(') && lexer.current().type == Lexeme::Type(')') - ? std::make_optional(Location(matchParen.location.begin, lexer.current().location.end)) + ? std::make_optional(Location(matchParen.position, lexer.current().location.end)) : std::nullopt; expectMatchAndConsume(')', matchParen, true); @@ -1255,7 +1253,7 @@ AstType* Parser::parseTableTypeAnnotation() Location start = lexer.current().location; - Lexeme matchBrace = lexer.current(); + MatchLexeme matchBrace = lexer.current(); expectAndConsume('{', "table type"); while (lexer.current().type != '}') @@ -1628,7 +1626,7 @@ AstTypeOrPack Parser::parseSimpleTypeAnnotation(bool allowPack) { return parseFunctionTypeAnnotation(allowPack); } - else if (FFlag::LuauParserFunctionKeywordAsTypeHelp && lexer.current().type == Lexeme::ReservedFunction) + else if (lexer.current().type == Lexeme::ReservedFunction) { Location location = lexer.current().location; @@ -1912,14 +1910,14 @@ AstExpr* Parser::parsePrefixExpr() { if (lexer.current().type == '(') { - Location start = lexer.current().location; + Position start = lexer.current().location.begin; - Lexeme matchParen = lexer.current(); + MatchLexeme matchParen = lexer.current(); nextLexeme(); AstExpr* expr = parseExpr(); - Location end = lexer.current().location; + Position end = lexer.current().location.end; if (lexer.current().type != ')') { @@ -1927,7 +1925,7 @@ AstExpr* Parser::parsePrefixExpr() expectMatchAndConsumeFail(static_cast(')'), matchParen, suggestion); - end = lexer.previousLocation(); + end = lexer.previousLocation().end; } else { @@ -1945,7 +1943,7 @@ AstExpr* Parser::parsePrefixExpr() // primaryexp -> prefixexp { `.' NAME | `[' exp `]' | `:' NAME funcargs | funcargs } AstExpr* Parser::parsePrimaryExpr(bool asStatement) { - Location start = lexer.current().location; + Position start = lexer.current().location.begin; AstExpr* expr = parsePrefixExpr(); @@ -1960,16 +1958,16 @@ AstExpr* Parser::parsePrimaryExpr(bool asStatement) Name index = parseIndexName(nullptr, opPosition); - expr = allocator.alloc(Location(start, index.location), expr, index.name, index.location, opPosition, '.'); + expr = allocator.alloc(Location(start, index.location.end), expr, index.name, index.location, opPosition, '.'); } else if (lexer.current().type == '[') { - Lexeme matchBracket = lexer.current(); + MatchLexeme matchBracket = lexer.current(); nextLexeme(); AstExpr* index = parseExpr(); - Location end = lexer.current().location; + Position end = lexer.current().location.end; expectMatchAndConsume(']', matchBracket); @@ -1981,27 +1979,24 @@ AstExpr* Parser::parsePrimaryExpr(bool asStatement) nextLexeme(); Name index = parseIndexName("method name", opPosition); - AstExpr* func = allocator.alloc(Location(start, index.location), expr, index.name, index.location, opPosition, ':'); + AstExpr* func = allocator.alloc(Location(start, index.location.end), expr, index.name, index.location, opPosition, ':'); - expr = parseFunctionArgs(func, true, index.location); + expr = parseFunctionArgs(func, true); } else if (lexer.current().type == '(') { // This error is handled inside 'parseFunctionArgs' as well, but for better error recovery we need to break out the current loop here if (!asStatement && expr->location.end.line != lexer.current().location.begin.line) { - report(lexer.current().location, - "Ambiguous syntax: this looks like an argument list for a function call, but could also be a start of " - "new statement; use ';' to separate statements"); - + reportAmbiguousCallError(); break; } - expr = parseFunctionArgs(expr, false, Location()); + expr = parseFunctionArgs(expr, false); } else if (lexer.current().type == '{' || lexer.current().type == Lexeme::RawString || lexer.current().type == Lexeme::QuotedString) { - expr = parseFunctionArgs(expr, false, Location()); + expr = parseFunctionArgs(expr, false); } else { @@ -2156,7 +2151,7 @@ static ConstantNumberParseResult parseInteger(double& result, const char* data, return ConstantNumberParseResult::Ok; } -static ConstantNumberParseResult parseNumber(double& result, const char* data) +static ConstantNumberParseResult parseDouble(double& result, const char* data) { LUAU_ASSERT(FFlag::LuauLintParseIntegerIssues); @@ -2214,61 +2209,11 @@ AstExpr* Parser::parseSimpleExpr() Lexeme matchFunction = lexer.current(); nextLexeme(); - return parseFunctionBody(false, matchFunction, AstName(), {}).first; + return parseFunctionBody(false, matchFunction, AstName(), nullptr).first; } else if (lexer.current().type == Lexeme::Number) { - scratchData.assign(lexer.current().data, lexer.current().length); - - // Remove all internal _ - they don't hold any meaning and this allows parsing code to just pass the string pointer to strtod et al - if (scratchData.find('_') != std::string::npos) - { - scratchData.erase(std::remove(scratchData.begin(), scratchData.end(), '_'), scratchData.end()); - } - - if (FFlag::LuauLintParseIntegerIssues) - { - double value = 0; - ConstantNumberParseResult result = parseNumber(value, scratchData.c_str()); - nextLexeme(); - - if (result == ConstantNumberParseResult::Malformed) - return reportExprError(start, {}, "Malformed number"); - - return allocator.alloc(start, value, result); - } - else if (DFFlag::LuaReportParseIntegerIssues) - { - double value = 0; - if (const char* error = parseNumber_DEPRECATED2(value, scratchData.c_str())) - { - nextLexeme(); - - return reportExprError(start, {}, "%s", error); - } - else - { - nextLexeme(); - - return allocator.alloc(start, value); - } - } - else - { - double value = 0; - if (parseNumber_DEPRECATED(value, scratchData.c_str())) - { - nextLexeme(); - - return allocator.alloc(start, value); - } - else - { - nextLexeme(); - - return reportExprError(start, {}, "Malformed number"); - } - } + return parseNumber(); } else if (lexer.current().type == Lexeme::RawString || lexer.current().type == Lexeme::QuotedString) { @@ -2309,18 +2254,15 @@ AstExpr* Parser::parseSimpleExpr() } // args ::= `(' [explist] `)' | tableconstructor | String -AstExpr* Parser::parseFunctionArgs(AstExpr* func, bool self, const Location& selfLocation) +AstExpr* Parser::parseFunctionArgs(AstExpr* func, bool self) { if (lexer.current().type == '(') { Position argStart = lexer.current().location.end; if (func->location.end.line != lexer.current().location.begin.line) - { - report(lexer.current().location, "Ambiguous syntax: this looks like an argument list for a function call, but could also be a start of " - "new statement; use ';' to separate statements"); - } + reportAmbiguousCallError(); - Lexeme matchParen = lexer.current(); + MatchLexeme matchParen = lexer.current(); nextLexeme(); TempVector args(scratchExpr); @@ -2352,18 +2294,29 @@ AstExpr* Parser::parseFunctionArgs(AstExpr* func, bool self, const Location& sel } else { - if (self && lexer.current().location.begin.line != func->location.end.line) - { - return reportExprError(func->location, copy({func}), "Expected function call arguments after '('"); - } - else - { - return reportExprError(Location(func->location.begin, lexer.current().location.begin), copy({func}), - "Expected '(', '{' or when parsing function call, got %s", lexer.current().toString().c_str()); - } + return reportFunctionArgsError(func, self); } } +LUAU_NOINLINE AstExpr* Parser::reportFunctionArgsError(AstExpr* func, bool self) +{ + if (self && lexer.current().location.begin.line != func->location.end.line) + { + return reportExprError(func->location, copy({func}), "Expected function call arguments after '('"); + } + else + { + return reportExprError(Location(func->location.begin, lexer.current().location.begin), copy({func}), + "Expected '(', '{' or when parsing function call, got %s", lexer.current().toString().c_str()); + } +} + +LUAU_NOINLINE void Parser::reportAmbiguousCallError() +{ + report(lexer.current().location, "Ambiguous syntax: this looks like an argument list for a function call, but could also be a start of " + "new statement; use ';' to separate statements"); +} + // tableconstructor ::= `{' [fieldlist] `}' // fieldlist ::= field {fieldsep field} [fieldsep] // field ::= `[' exp `]' `=' exp | Name `=' exp | exp @@ -2374,14 +2327,14 @@ AstExpr* Parser::parseTableConstructor() Location start = lexer.current().location; - Lexeme matchBrace = lexer.current(); + MatchLexeme matchBrace = lexer.current(); expectAndConsume('{', "table literal"); while (lexer.current().type != '}') { if (lexer.current().type == '[') { - Lexeme matchLocationBracket = lexer.current(); + MatchLexeme matchLocationBracket = lexer.current(); nextLexeme(); AstExpr* key = parseExpr(); @@ -2692,6 +2645,63 @@ AstExpr* Parser::parseString() return reportExprError(location, {}, "String literal contains malformed escape sequence"); } +AstExpr* Parser::parseNumber() +{ + Location start = lexer.current().location; + + scratchData.assign(lexer.current().data, lexer.current().length); + + // Remove all internal _ - they don't hold any meaning and this allows parsing code to just pass the string pointer to strtod et al + if (scratchData.find('_') != std::string::npos) + { + scratchData.erase(std::remove(scratchData.begin(), scratchData.end(), '_'), scratchData.end()); + } + + if (FFlag::LuauLintParseIntegerIssues) + { + double value = 0; + ConstantNumberParseResult result = parseDouble(value, scratchData.c_str()); + nextLexeme(); + + if (result == ConstantNumberParseResult::Malformed) + return reportExprError(start, {}, "Malformed number"); + + return allocator.alloc(start, value, result); + } + else if (DFFlag::LuaReportParseIntegerIssues) + { + double value = 0; + if (const char* error = parseNumber_DEPRECATED2(value, scratchData.c_str())) + { + nextLexeme(); + + return reportExprError(start, {}, "%s", error); + } + else + { + nextLexeme(); + + return allocator.alloc(start, value); + } + } + else + { + double value = 0; + if (parseNumber_DEPRECATED(value, scratchData.c_str())) + { + nextLexeme(); + + return allocator.alloc(start, value); + } + else + { + nextLexeme(); + + return reportExprError(start, {}, "Malformed number"); + } + } +} + AstLocal* Parser::pushLocal(const Binding& binding) { const Name& name = binding.name; @@ -2763,7 +2773,7 @@ LUAU_NOINLINE void Parser::expectAndConsumeFail(Lexeme::Type type, const char* c report(lexer.current().location, "Expected %s, got %s", typeString.c_str(), currLexemeString.c_str()); } -bool Parser::expectMatchAndConsume(char value, const Lexeme& begin, bool searchForMissing) +bool Parser::expectMatchAndConsume(char value, const MatchLexeme& begin, bool searchForMissing) { Lexeme::Type type = static_cast(static_cast(value)); @@ -2771,42 +2781,7 @@ bool Parser::expectMatchAndConsume(char value, const Lexeme& begin, bool searchF { expectMatchAndConsumeFail(type, begin); - if (searchForMissing) - { - // previous location is taken because 'current' lexeme is already the next token - unsigned currentLine = lexer.previousLocation().end.line; - - // search to the end of the line for expected token - // we will also stop if we hit a token that can be handled by parsing function above the current one - Lexeme::Type lexemeType = lexer.current().type; - - while (currentLine == lexer.current().location.begin.line && lexemeType != type && matchRecoveryStopOnToken[lexemeType] == 0) - { - nextLexeme(); - lexemeType = lexer.current().type; - } - - if (lexemeType == type) - { - nextLexeme(); - - return true; - } - } - else - { - // check if this is an extra token and the expected token is next - if (lexer.lookahead().type == type) - { - // skip invalid and consume expected - nextLexeme(); - nextLexeme(); - - return true; - } - } - - return false; + return expectMatchAndConsumeRecover(value, begin, searchForMissing); } else { @@ -2816,21 +2791,64 @@ bool Parser::expectMatchAndConsume(char value, const Lexeme& begin, bool searchF } } -// LUAU_NOINLINE is used to limit the stack cost of this function due to std::string objects, and to increase caller performance since this code is -// cold -LUAU_NOINLINE void Parser::expectMatchAndConsumeFail(Lexeme::Type type, const Lexeme& begin, const char* extra) +LUAU_NOINLINE bool Parser::expectMatchAndConsumeRecover(char value, const MatchLexeme& begin, bool searchForMissing) { - std::string typeString = Lexeme(Location(Position(0, 0), 0), type).toString(); + Lexeme::Type type = static_cast(static_cast(value)); - if (lexer.current().location.begin.line == begin.location.begin.line) - report(lexer.current().location, "Expected %s (to close %s at column %d), got %s%s", typeString.c_str(), begin.toString().c_str(), - begin.location.begin.column + 1, lexer.current().toString().c_str(), extra ? extra : ""); + if (searchForMissing) + { + // previous location is taken because 'current' lexeme is already the next token + unsigned currentLine = lexer.previousLocation().end.line; + + // search to the end of the line for expected token + // we will also stop if we hit a token that can be handled by parsing function above the current one + Lexeme::Type lexemeType = lexer.current().type; + + while (currentLine == lexer.current().location.begin.line && lexemeType != type && matchRecoveryStopOnToken[lexemeType] == 0) + { + nextLexeme(); + lexemeType = lexer.current().type; + } + + if (lexemeType == type) + { + nextLexeme(); + + return true; + } + } else - report(lexer.current().location, "Expected %s (to close %s at line %d), got %s%s", typeString.c_str(), begin.toString().c_str(), - begin.location.begin.line + 1, lexer.current().toString().c_str(), extra ? extra : ""); + { + // check if this is an extra token and the expected token is next + if (lexer.lookahead().type == type) + { + // skip invalid and consume expected + nextLexeme(); + nextLexeme(); + + return true; + } + } + + return false; } -bool Parser::expectMatchEndAndConsume(Lexeme::Type type, const Lexeme& begin) +// LUAU_NOINLINE is used to limit the stack cost of this function due to std::string objects, and to increase caller performance since this code is +// cold +LUAU_NOINLINE void Parser::expectMatchAndConsumeFail(Lexeme::Type type, const MatchLexeme& begin, const char* extra) +{ + std::string typeString = Lexeme(Location(Position(0, 0), 0), type).toString(); + std::string matchString = Lexeme(Location(Position(0, 0), 0), begin.type).toString(); + + if (lexer.current().location.begin.line == begin.position.line) + report(lexer.current().location, "Expected %s (to close %s at column %d), got %s%s", typeString.c_str(), matchString.c_str(), + begin.position.column + 1, lexer.current().toString().c_str(), extra ? extra : ""); + else + report(lexer.current().location, "Expected %s (to close %s at line %d), got %s%s", typeString.c_str(), matchString.c_str(), + begin.position.line + 1, lexer.current().toString().c_str(), extra ? extra : ""); +} + +bool Parser::expectMatchEndAndConsume(Lexeme::Type type, const MatchLexeme& begin) { if (lexer.current().type != type) { @@ -2852,9 +2870,9 @@ bool Parser::expectMatchEndAndConsume(Lexeme::Type type, const Lexeme& begin) { // If the token matches on a different line and a different column, it suggests misleading indentation // This can be used to pinpoint the problem location for a possible future *actual* mismatch - if (lexer.current().location.begin.line != begin.location.begin.line && - lexer.current().location.begin.column != begin.location.begin.column && - endMismatchSuspect.location.begin.line < begin.location.begin.line) // Only replace the previous suspect with more recent suspects + if (lexer.current().location.begin.line != begin.position.line && + lexer.current().location.begin.column != begin.position.column && + endMismatchSuspect.position.line < begin.position.line) // Only replace the previous suspect with more recent suspects { endMismatchSuspect = begin; } @@ -2867,12 +2885,12 @@ bool Parser::expectMatchEndAndConsume(Lexeme::Type type, const Lexeme& begin) // LUAU_NOINLINE is used to limit the stack cost of this function due to std::string objects, and to increase caller performance since this code is // cold -LUAU_NOINLINE void Parser::expectMatchEndAndConsumeFail(Lexeme::Type type, const Lexeme& begin) +LUAU_NOINLINE void Parser::expectMatchEndAndConsumeFail(Lexeme::Type type, const MatchLexeme& begin) { - if (endMismatchSuspect.type != Lexeme::Eof && endMismatchSuspect.location.begin.line > begin.location.begin.line) + if (endMismatchSuspect.type != Lexeme::Eof && endMismatchSuspect.position.line > begin.position.line) { - std::string suggestion = - format("; did you forget to close %s at line %d?", endMismatchSuspect.toString().c_str(), endMismatchSuspect.location.begin.line + 1); + std::string matchString = Lexeme(Location(Position(0, 0), 0), endMismatchSuspect.type).toString(); + std::string suggestion = format("; did you forget to close %s at line %d?", matchString.c_str(), endMismatchSuspect.position.line + 1); expectMatchAndConsumeFail(type, begin, suggestion.c_str()); } diff --git a/Common/include/Luau/Bytecode.h b/Common/include/Luau/Bytecode.h index 1d6b18e5..d6660f05 100644 --- a/Common/include/Luau/Bytecode.h +++ b/Common/include/Luau/Bytecode.h @@ -515,6 +515,9 @@ enum LuauBuiltinFunction // rawlen LBF_RAWLEN, + + // bit32.extract(_, k, k) + LBF_BIT32_EXTRACTK, }; // Capture type, used in LOP_CAPTURE diff --git a/Compiler/src/BuiltinFolding.cpp b/Compiler/src/BuiltinFolding.cpp index e76da4e2..03b5918c 100644 --- a/Compiler/src/BuiltinFolding.cpp +++ b/Compiler/src/BuiltinFolding.cpp @@ -319,11 +319,12 @@ Constant foldBuiltin(int bfid, const Constant* args, size_t count) break; case LBF_BIT32_EXTRACT: - if (count == 3 && args[0].type == Constant::Type_Number && args[1].type == Constant::Type_Number && args[2].type == Constant::Type_Number) + if (count >= 2 && args[0].type == Constant::Type_Number && args[1].type == Constant::Type_Number && + (count == 2 || args[2].type == Constant::Type_Number)) { uint32_t u = bit32(args[0].valueNumber); int f = int(args[1].valueNumber); - int w = int(args[2].valueNumber); + int w = count == 2 ? 1 : int(args[2].valueNumber); if (f >= 0 && w > 0 && f + w <= 32) { @@ -356,13 +357,13 @@ Constant foldBuiltin(int bfid, const Constant* args, size_t count) break; case LBF_BIT32_REPLACE: - if (count == 4 && args[0].type == Constant::Type_Number && args[1].type == Constant::Type_Number && args[2].type == Constant::Type_Number && - args[3].type == Constant::Type_Number) + if (count >= 3 && args[0].type == Constant::Type_Number && args[1].type == Constant::Type_Number && args[2].type == Constant::Type_Number && + (count == 3 || args[3].type == Constant::Type_Number)) { uint32_t n = bit32(args[0].valueNumber); uint32_t v = bit32(args[1].valueNumber); int f = int(args[2].valueNumber); - int w = int(args[3].valueNumber); + int w = count == 3 ? 1 : int(args[3].valueNumber); if (f >= 0 && w > 0 && f + w <= 32) { diff --git a/Compiler/src/Compiler.cpp b/Compiler/src/Compiler.cpp index eb815225..bd8744cc 100644 --- a/Compiler/src/Compiler.cpp +++ b/Compiler/src/Compiler.cpp @@ -23,13 +23,12 @@ LUAU_FASTINTVARIABLE(LuauCompileInlineThreshold, 25) LUAU_FASTINTVARIABLE(LuauCompileInlineThresholdMaxBoost, 300) LUAU_FASTINTVARIABLE(LuauCompileInlineDepth, 5) -LUAU_FASTFLAGVARIABLE(LuauCompileNoIpairs, false) - -LUAU_FASTFLAGVARIABLE(LuauCompileFreeReassign, false) LUAU_FASTFLAGVARIABLE(LuauCompileXEQ, false) LUAU_FASTFLAGVARIABLE(LuauCompileOptimalAssignment, false) +LUAU_FASTFLAGVARIABLE(LuauCompileExtractK, false) + namespace Luau { @@ -403,18 +402,37 @@ struct Compiler } } - void compileExprFastcallN(AstExprCall* expr, uint8_t target, uint8_t targetCount, bool targetTop, bool multRet, uint8_t regs, int bfid) + void compileExprFastcallN(AstExprCall* expr, uint8_t target, uint8_t targetCount, bool targetTop, bool multRet, uint8_t regs, int bfid, int bfK = -1) { LUAU_ASSERT(!expr->self); - LUAU_ASSERT(expr->args.size <= 2); + LUAU_ASSERT(expr->args.size >= 1); + LUAU_ASSERT(expr->args.size <= 2 || (bfid == LBF_BIT32_EXTRACTK && expr->args.size == 3)); + LUAU_ASSERT(bfid == LBF_BIT32_EXTRACTK ? bfK >= 0 : bfK < 0); LuauOpcode opc = expr->args.size == 1 ? LOP_FASTCALL1 : LOP_FASTCALL2; - uint32_t args[2] = {}; + if (FFlag::LuauCompileExtractK) + { + opc = expr->args.size == 1 ? LOP_FASTCALL1 : (bfK >= 0 || isConstant(expr->args.data[1])) ? LOP_FASTCALL2K : LOP_FASTCALL2; + } + + uint32_t args[3] = {}; for (size_t i = 0; i < expr->args.size; ++i) { - if (i > 0) + if (FFlag::LuauCompileExtractK) + { + if (i > 0 && opc == LOP_FASTCALL2K) + { + int32_t cid = getConstantIndex(expr->args.data[i]); + if (cid < 0) + CompileError::raise(expr->location, "Exceeded constant limit; simplify the code to compile"); + + args[i] = cid; + continue; // TODO: remove this and change if below to else if + } + } + else if (i > 0) { if (int32_t cid = getConstantIndex(expr->args.data[i]); cid >= 0) { @@ -425,7 +443,9 @@ struct Compiler } if (int reg = getExprLocalReg(expr->args.data[i]); reg >= 0) + { args[i] = uint8_t(reg); + } else { args[i] = uint8_t(regs + 1 + i); @@ -437,21 +457,31 @@ struct Compiler bytecode.emitABC(opc, uint8_t(bfid), uint8_t(args[0]), 0); if (opc != LOP_FASTCALL1) - bytecode.emitAux(args[1]); + bytecode.emitAux(bfK >= 0 ? bfK : args[1]); // Set up a traditional Lua stack for the subsequent LOP_CALL. // Note, as with other instructions that immediately follow FASTCALL, these are normally not executed and are used as a fallback for // these FASTCALL variants. for (size_t i = 0; i < expr->args.size; ++i) { - if (i > 0 && opc == LOP_FASTCALL2K) + if (FFlag::LuauCompileExtractK) { - emitLoadK(uint8_t(regs + 1 + i), args[i]); - break; + if (i > 0 && opc == LOP_FASTCALL2K) + emitLoadK(uint8_t(regs + 1 + i), args[i]); + else if (args[i] != regs + 1 + i) + bytecode.emitABC(LOP_MOVE, uint8_t(regs + 1 + i), uint8_t(args[i]), 0); } + else + { + if (i > 0 && opc == LOP_FASTCALL2K) + { + emitLoadK(uint8_t(regs + 1 + i), args[i]); + break; + } - if (args[i] != regs + 1 + i) - bytecode.emitABC(LOP_MOVE, uint8_t(regs + 1 + i), uint8_t(args[i]), 0); + if (args[i] != regs + 1 + i) + bytecode.emitABC(LOP_MOVE, uint8_t(regs + 1 + i), uint8_t(args[i]), 0); + } } // note, these instructions are normally not executed and are used as a fallback for FASTCALL @@ -600,7 +630,7 @@ struct Compiler } else { - AstExprLocal* le = FFlag::LuauCompileFreeReassign ? getExprLocal(arg) : arg->as(); + AstExprLocal* le = getExprLocal(arg); Variable* lv = le ? variables.find(le->local) : nullptr; // if the argument is a local that isn't mutated, we will simply reuse the existing register @@ -723,6 +753,26 @@ struct Compiler bfid = -1; } + // Optimization: for bit32.extract with constant in-range f/w we compile using FASTCALL2K and a special builtin + if (FFlag::LuauCompileExtractK && bfid == LBF_BIT32_EXTRACT && expr->args.size == 3 && isConstant(expr->args.data[1]) && isConstant(expr->args.data[2])) + { + Constant fc = getConstant(expr->args.data[1]); + Constant wc = getConstant(expr->args.data[2]); + + int fi = fc.type == Constant::Type_Number ? int(fc.valueNumber) : -1; + int wi = wc.type == Constant::Type_Number ? int(wc.valueNumber) : -1; + + if (fi >= 0 && wi > 0 && fi + wi <= 32) + { + int fwp = fi | ((wi - 1) << 5); + int32_t cid = bytecode.addConstantNumber(fwp); + if (cid < 0) + CompileError::raise(expr->location, "Exceeded constant limit; simplify the code to compile"); + + return compileExprFastcallN(expr, target, targetCount, targetTop, multRet, regs, LBF_BIT32_EXTRACTK, cid); + } + } + // Optimization: for 1/2 argument fast calls use specialized opcodes if (bfid >= 0 && expr->args.size >= 1 && expr->args.size <= 2 && !isExprMultRet(expr->args.data[expr->args.size - 1])) return compileExprFastcallN(expr, target, targetCount, targetTop, multRet, regs, bfid); @@ -1218,7 +1268,7 @@ struct Compiler { // disambiguation: there's 4 cases (we only need truthy or falsy results based on onlyTruth) // onlyTruth = 1: a and b transforms to a ? b : dontcare - // onlyTruth = 1: a or b transforms to a ? a : a + // onlyTruth = 1: a or b transforms to a ? a : b // onlyTruth = 0: a and b transforms to !a ? a : b // onlyTruth = 0: a or b transforms to !a ? b : dontcare if (onlyTruth == (expr->op == AstExprBinary::And)) @@ -2576,7 +2626,7 @@ struct Compiler return; // Optimization: for 1-1 local assignments, we can reuse the register *if* neither local is mutated - if (FFlag::LuauCompileFreeReassign && options.optimizationLevel >= 1 && stat->vars.size == 1 && stat->values.size == 1) + if (options.optimizationLevel >= 1 && stat->vars.size == 1 && stat->values.size == 1) { if (AstExprLocal* re = getExprLocal(stat->values.data[0])) { @@ -2790,7 +2840,6 @@ struct Compiler LUAU_ASSERT(vars == regs + 3); LuauOpcode skipOp = LOP_FORGPREP; - LuauOpcode loopOp = LOP_FORGLOOP; // Optimization: when we iterate via pairs/ipairs, we generate special bytecode that optimizes the traversal using internal iteration index // These instructions dynamically check if generator is equal to next/inext and bail out @@ -2802,25 +2851,16 @@ struct Compiler Builtin builtin = getBuiltin(stat->values.data[0]->as()->func, globals, variables); if (builtin.isGlobal("ipairs")) // for .. in ipairs(t) - { skipOp = LOP_FORGPREP_INEXT; - loopOp = FFlag::LuauCompileNoIpairs ? LOP_FORGLOOP : LOP_FORGLOOP_INEXT; - } else if (builtin.isGlobal("pairs")) // for .. in pairs(t) - { skipOp = LOP_FORGPREP_NEXT; - loopOp = LOP_FORGLOOP; - } } else if (stat->values.size == 2) { Builtin builtin = getBuiltin(stat->values.data[0], globals, variables); if (builtin.isGlobal("next")) // for .. in next,t - { skipOp = LOP_FORGPREP_NEXT; - loopOp = LOP_FORGLOOP; - } } } @@ -2846,19 +2886,9 @@ struct Compiler size_t backLabel = bytecode.emitLabel(); - bytecode.emitAD(loopOp, regs, 0); - - if (FFlag::LuauCompileNoIpairs) - { - // TODO: remove loopOp as it's a constant now - LUAU_ASSERT(loopOp == LOP_FORGLOOP); - - // FORGLOOP uses aux to encode variable count and fast path flag for ipairs traversal in the high bit - bytecode.emitAux((skipOp == LOP_FORGPREP_INEXT ? 0x80000000 : 0) | uint32_t(stat->vars.size)); - } - // note: FORGLOOP needs variable count encoded in AUX field, other loop instructions assume a fixed variable count - else if (loopOp == LOP_FORGLOOP) - bytecode.emitAux(uint32_t(stat->vars.size)); + // FORGLOOP uses aux to encode variable count and fast path flag for ipairs traversal in the high bit + bytecode.emitAD(LOP_FORGLOOP, regs, 0); + bytecode.emitAux((skipOp == LOP_FORGPREP_INEXT ? 0x80000000 : 0) | uint32_t(stat->vars.size)); size_t endLabel = bytecode.emitLabel(); diff --git a/Sources.cmake b/Sources.cmake index 7a27684e..50770f92 100644 --- a/Sources.cmake +++ b/Sources.cmake @@ -66,6 +66,7 @@ target_sources(Luau.CodeGen PRIVATE # Luau.Analysis Sources target_sources(Luau.Analysis PRIVATE + Analysis/include/Luau/Anyification.h Analysis/include/Luau/ApplyTypeFunction.h Analysis/include/Luau/AstJsonEncoder.h Analysis/include/Luau/AstQuery.h @@ -115,6 +116,7 @@ target_sources(Luau.Analysis PRIVATE Analysis/include/Luau/Variant.h Analysis/include/Luau/VisitTypeVar.h + Analysis/src/Anyification.cpp Analysis/src/ApplyTypeFunction.cpp Analysis/src/AstJsonEncoder.cpp Analysis/src/AstQuery.cpp @@ -126,6 +128,7 @@ target_sources(Luau.Analysis PRIVATE Analysis/src/ConstraintGraphBuilder.cpp Analysis/src/ConstraintSolver.cpp Analysis/src/ConstraintSolverLogger.cpp + Analysis/src/EmbeddedBuiltinDefinitions.cpp Analysis/src/Error.cpp Analysis/src/Frontend.cpp Analysis/src/Instantiation.cpp @@ -155,7 +158,6 @@ target_sources(Luau.Analysis PRIVATE Analysis/src/TypeVar.cpp Analysis/src/Unifiable.cpp Analysis/src/Unifier.cpp - Analysis/src/EmbeddedBuiltinDefinitions.cpp ) # Luau.VM Sources diff --git a/VM/src/lapi.cpp b/VM/src/lapi.cpp index bb994fb4..af97dc4a 100644 --- a/VM/src/lapi.cpp +++ b/VM/src/lapi.cpp @@ -1209,7 +1209,9 @@ void* lua_newuserdatadtor(lua_State* L, size_t sz, void (*dtor)(void*)) { luaC_checkGC(L); luaC_checkthreadsleep(L); - Udata* u = luaU_newudata(L, sz + sizeof(dtor), UTAG_IDTOR); + // make sure sz + sizeof(dtor) doesn't overflow; luaU_newdata will reject SIZE_MAX correctly + size_t as = sz < SIZE_MAX - sizeof(dtor) ? sz + sizeof(dtor) : SIZE_MAX; + Udata* u = luaU_newudata(L, as, UTAG_IDTOR); memcpy(&u->data + sz, &dtor, sizeof(dtor)); setuvalue(L, L->top, u); api_incr_top(L); diff --git a/VM/src/lbuiltins.cpp b/VM/src/lbuiltins.cpp index e98660a7..422c82b1 100644 --- a/VM/src/lbuiltins.cpp +++ b/VM/src/lbuiltins.cpp @@ -15,6 +15,8 @@ #include #endif +LUAU_FASTFLAGVARIABLE(LuauFasterBit32NoWidth, false) + // luauF functions implement FASTCALL instruction that performs a direct execution of some builtin functions from the VM // The rule of thumb is that FASTCALL functions can not call user code, yield, fail, or reallocate stack. // If types of the arguments mismatch, luauF_* needs to return -1 and the execution will fall back to the usual call path @@ -600,24 +602,39 @@ static int luauF_btest(lua_State* L, StkId res, TValue* arg0, int nresults, StkI static int luauF_extract(lua_State* L, StkId res, TValue* arg0, int nresults, StkId args, int nparams) { - if (nparams >= 3 && nresults <= 1 && ttisnumber(arg0) && ttisnumber(args) && ttisnumber(args + 1)) + if (nparams >= (3 - FFlag::LuauFasterBit32NoWidth) && nresults <= 1 && ttisnumber(arg0) && ttisnumber(args)) { double a1 = nvalue(arg0); double a2 = nvalue(args); - double a3 = nvalue(args + 1); unsigned n; luai_num2unsigned(n, a1); int f = int(a2); - int w = int(a3); - if (f >= 0 && w > 0 && f + w <= 32) + if (nparams == 2) { - uint32_t m = ~(0xfffffffeu << (w - 1)); - uint32_t r = (n >> f) & m; + if (unsigned(f) < 32) + { + uint32_t m = 1; + uint32_t r = (n >> f) & m; - setnvalue(res, double(r)); - return 1; + setnvalue(res, double(r)); + return 1; + } + } + else if (ttisnumber(args + 1)) + { + double a3 = nvalue(args + 1); + int w = int(a3); + + if (f >= 0 && w > 0 && f + w <= 32) + { + uint32_t m = ~(0xfffffffeu << (w - 1)); + uint32_t r = (n >> f) & m; + + setnvalue(res, double(r)); + return 1; + } } } @@ -676,26 +693,41 @@ static int luauF_lshift(lua_State* L, StkId res, TValue* arg0, int nresults, Stk static int luauF_replace(lua_State* L, StkId res, TValue* arg0, int nresults, StkId args, int nparams) { - if (nparams >= 4 && nresults <= 1 && ttisnumber(arg0) && ttisnumber(args) && ttisnumber(args + 1) && ttisnumber(args + 2)) + if (nparams >= (4 - FFlag::LuauFasterBit32NoWidth) && nresults <= 1 && ttisnumber(arg0) && ttisnumber(args) && ttisnumber(args + 1)) { double a1 = nvalue(arg0); double a2 = nvalue(args); double a3 = nvalue(args + 1); - double a4 = nvalue(args + 2); unsigned n, v; luai_num2unsigned(n, a1); luai_num2unsigned(v, a2); int f = int(a3); - int w = int(a4); - if (f >= 0 && w > 0 && f + w <= 32) + if (nparams == 3) { - uint32_t m = ~(0xfffffffeu << (w - 1)); - uint32_t r = (n & ~(m << f)) | ((v & m) << f); + if (unsigned(f) < 32) + { + uint32_t m = 1; + uint32_t r = (n & ~(m << f)) | ((v & m) << f); - setnvalue(res, double(r)); - return 1; + setnvalue(res, double(r)); + return 1; + } + } + else if (ttisnumber(args + 2)) + { + double a4 = nvalue(args + 2); + int w = int(a4); + + if (f >= 0 && w > 0 && f + w <= 32) + { + uint32_t m = ~(0xfffffffeu << (w - 1)); + uint32_t r = (n & ~(m << f)) | ((v & m) << f); + + setnvalue(res, double(r)); + return 1; + } } } @@ -1138,6 +1170,31 @@ static int luauF_rawlen(lua_State* L, StkId res, TValue* arg0, int nresults, Stk return -1; } +static int luauF_extractk(lua_State* L, StkId res, TValue* arg0, int nresults, StkId args, int nparams) +{ + // args is known to contain a number constant with packed in-range f/w + if (nparams >= 2 && nresults <= 1 && ttisnumber(arg0)) + { + double a1 = nvalue(arg0); + double a2 = nvalue(args); + + unsigned n; + luai_num2unsigned(n, a1); + int fw = int(a2); + + int f = fw & 31; + int w1 = fw >> 5; + + uint32_t m = ~(0xfffffffeu << w1); + uint32_t r = (n >> f) & m; + + setnvalue(res, double(r)); + return 1; + } + + return -1; +} + luau_FastFunction luauF_table[256] = { NULL, luauF_assert, @@ -1211,4 +1268,6 @@ luau_FastFunction luauF_table[256] = { luauF_select, luauF_rawlen, + + luauF_extractk, }; diff --git a/VM/src/lfunc.cpp b/VM/src/lfunc.cpp index dfde6dcb..bd0f8264 100644 --- a/VM/src/lfunc.cpp +++ b/VM/src/lfunc.cpp @@ -175,8 +175,7 @@ void luaF_freeclosure(lua_State* L, Closure* c, lua_Page* page) const LocVar* luaF_getlocal(const Proto* f, int local_number, int pc) { - int i; - for (i = 0; i < f->sizelocvars; i++) + for (int i = 0; i < f->sizelocvars; i++) { if (pc >= f->locvars[i].startpc && pc < f->locvars[i].endpc) { // is variable active? @@ -185,5 +184,15 @@ const LocVar* luaF_getlocal(const Proto* f, int local_number, int pc) return &f->locvars[i]; } } + + return NULL; // not found +} + +const LocVar* luaF_findlocal(const Proto* f, int local_reg, int pc) +{ + for (int i = 0; i < f->sizelocvars; i++) + if (local_reg == f->locvars[i].reg && pc >= f->locvars[i].startpc && pc < f->locvars[i].endpc) + return &f->locvars[i]; + return NULL; // not found } diff --git a/VM/src/lfunc.h b/VM/src/lfunc.h index a260d00a..59ab5726 100644 --- a/VM/src/lfunc.h +++ b/VM/src/lfunc.h @@ -17,3 +17,4 @@ LUAI_FUNC void luaF_freeclosure(lua_State* L, Closure* c, struct lua_Page* page) LUAI_FUNC void luaF_unlinkupval(UpVal* uv); LUAI_FUNC void luaF_freeupval(lua_State* L, UpVal* uv, struct lua_Page* page); LUAI_FUNC const LocVar* luaF_getlocal(const Proto* func, int local_number, int pc); +LUAI_FUNC const LocVar* luaF_findlocal(const Proto* func, int local_reg, int pc); diff --git a/VM/src/lgcdebug.cpp b/VM/src/lgcdebug.cpp index bc997d44..b6204b1a 100644 --- a/VM/src/lgcdebug.cpp +++ b/VM/src/lgcdebug.cpp @@ -345,6 +345,9 @@ static void dumpclosure(FILE* f, Closure* cl) if (cl->isC) { + if (cl->c.debugname) + fprintf(f, ",\"name\":\"%s\"", cl->c.debugname + 0); + if (cl->nupvalues) { fprintf(f, ",\"upvalues\":["); @@ -354,6 +357,9 @@ static void dumpclosure(FILE* f, Closure* cl) } else { + if (cl->l.p->debugname) + fprintf(f, ",\"name\":\"%s\"", getstr(cl->l.p->debugname)); + fprintf(f, ",\"proto\":"); dumpref(f, obj2gco(cl->l.p)); if (cl->nupvalues) @@ -403,7 +409,7 @@ static void dumpthread(FILE* f, lua_State* th) fprintf(f, ",\"source\":\""); dumpstringdata(f, p->source->data, p->source->len); - fprintf(f, "\",\"line\":%d", p->abslineinfo ? p->abslineinfo[0] : 0); + fprintf(f, "\",\"line\":%d", p->linedefined); } if (th->top > th->stack) @@ -411,6 +417,55 @@ static void dumpthread(FILE* f, lua_State* th) fprintf(f, ",\"stack\":["); dumprefs(f, th->stack, th->top - th->stack); fprintf(f, "]"); + + CallInfo* ci = th->base_ci; + bool first = true; + + fprintf(f, ",\"stacknames\":["); + for (StkId v = th->stack; v < th->top; ++v) + { + if (!iscollectable(v)) + continue; + + while (ci < th->ci && v >= (ci + 1)->func) + ci++; + + if (!first) + fputc(',', f); + first = false; + + if (v == ci->func) + { + Closure* cl = ci_func(ci); + + if (cl->isC) + { + fprintf(f, "\"frame:%s\"", cl->c.debugname ? cl->c.debugname : "[C]"); + } + else + { + Proto* p = cl->l.p; + fprintf(f, "\"frame:"); + if (p->source) + dumpstringdata(f, p->source->data, p->source->len); + fprintf(f, ":%d:%s\"", p->linedefined, p->debugname ? getstr(p->debugname) : ""); + } + } + else if (isLua(ci)) + { + Proto* p = ci_func(ci)->l.p; + int pc = pcRel(ci->savedpc, p); + const LocVar* var = luaF_findlocal(p, int(v - ci->base), pc); + + if (var && var->varname) + fprintf(f, "\"%s\"", getstr(var->varname)); + else + fprintf(f, "null"); + } + else + fprintf(f, "null"); + } + fprintf(f, "]"); } fprintf(f, "}"); } diff --git a/tests/Autocomplete.test.cpp b/tests/Autocomplete.test.cpp index 0f17531b..25447dd8 100644 --- a/tests/Autocomplete.test.cpp +++ b/tests/Autocomplete.test.cpp @@ -3189,4 +3189,27 @@ a.@1 CHECK(ac.entryMap.count("y")); } +TEST_CASE_FIXTURE(ACFixture, "globals_are_order_independent") +{ + ScopedFastFlag sff("LuauAutocompleteFixGlobalOrder", true); + + check(R"( + local myLocal = 4 + function abc0() + local myInnerLocal = 1 +@1 + end + + function abc1() + local myInnerLocal = 1 + end + )"); + + auto ac = autocomplete('1'); + CHECK(ac.entryMap.count("myLocal")); + CHECK(ac.entryMap.count("myInnerLocal")); + CHECK(ac.entryMap.count("abc0")); + CHECK(ac.entryMap.count("abc1")); +} + TEST_SUITE_END(); diff --git a/tests/Compiler.test.cpp b/tests/Compiler.test.cpp index a4ce88b1..a2e748a8 100644 --- a/tests/Compiler.test.cpp +++ b/tests/Compiler.test.cpp @@ -261,8 +261,6 @@ L1: RETURN R0 0 TEST_CASE("ForBytecode") { - ScopedFastFlag sff("LuauCompileNoIpairs", true); - // basic for loop: variable directly refers to internal iteration index (R2) CHECK_EQ("\n" + compileFunction0("for i=1,5 do print(i) end"), R"( LOADN R2 1 @@ -349,8 +347,6 @@ RETURN R0 0 TEST_CASE("ForBytecodeBuiltin") { - ScopedFastFlag sff("LuauCompileNoIpairs", true); - // we generally recognize builtins like pairs/ipairs and emit special opcodes CHECK_EQ("\n" + compileFunction0("for k,v in ipairs({}) do end"), R"( GETIMPORT R0 1 @@ -2065,6 +2061,69 @@ TEST_CASE("RecursionParse") { CHECK_EQ(std::string(e.what()), "Exceeded allowed recursion depth; simplify your block to make the code compile"); } + + try + { + Luau::compileOrThrow(bcb, rep("a(", 1500) + "42" + rep(")", 1500)); + CHECK(!"Expected exception"); + } + catch (std::exception& e) + { + CHECK_EQ(std::string(e.what()), "Exceeded allowed recursion depth; simplify your expression to make the code compile"); + } + + try + { + Luau::compileOrThrow(bcb, "return " + rep("{", 1500) + "42" + rep("}", 1500)); + CHECK(!"Expected exception"); + } + catch (std::exception& e) + { + CHECK_EQ(std::string(e.what()), "Exceeded allowed recursion depth; simplify your expression to make the code compile"); + } + + try + { + Luau::compileOrThrow(bcb, rep("while true do ", 1500) + "print()" + rep(" end", 1500)); + CHECK(!"Expected exception"); + } + catch (std::exception& e) + { + CHECK_EQ(std::string(e.what()), "Exceeded allowed recursion depth; simplify your expression to make the code compile"); + } + + try + { + Luau::compileOrThrow(bcb, rep("for i=1,1 do ", 1500) + "print()" + rep(" end", 1500)); + CHECK(!"Expected exception"); + } + catch (std::exception& e) + { + CHECK_EQ(std::string(e.what()), "Exceeded allowed recursion depth; simplify your expression to make the code compile"); + } + +#if 0 + // This currently requires too much stack space on MSVC/x64 and crashes with stack overflow at recursion depth 935 + try + { + Luau::compileOrThrow(bcb, rep("function a() ", 1500) + "print()" + rep(" end", 1500)); + CHECK(!"Expected exception"); + } + catch (std::exception& e) + { + CHECK_EQ(std::string(e.what()), "Exceeded allowed recursion depth; simplify your block to make the code compile"); + } + + try + { + Luau::compileOrThrow(bcb, "return " + rep("function() return ", 1500) + "42" + rep(" end", 1500)); + CHECK(!"Expected exception"); + } + catch (std::exception& e) + { + CHECK_EQ(std::string(e.what()), "Exceeded allowed recursion depth; simplify your block to make the code compile"); + } +#endif } TEST_CASE("ArrayIndexLiteral") @@ -2111,8 +2170,6 @@ L1: RETURN R3 -1 TEST_CASE("UpvaluesLoopsBytecode") { - ScopedFastFlag sff("LuauCompileNoIpairs", true); - CHECK_EQ("\n" + compileFunction(R"( function test() for i=1,10 do @@ -3790,8 +3847,6 @@ RETURN R0 1 TEST_CASE("SharedClosure") { - ScopedFastFlag sff("LuauCompileFreeReassign", true); - // closures can be shared even if functions refer to upvalues, as long as upvalues are top-level CHECK_EQ("\n" + compileFunction(R"( local val = ... @@ -6004,6 +6059,8 @@ return math.clamp(-1, 0, 1), math.sign(77), math.round(7.6), + bit32.extract(-1, 31), + bit32.replace(100, 1, 0), (type("fin")) )", 0, 2), @@ -6055,8 +6112,10 @@ LOADK R43 K2 LOADN R44 0 LOADN R45 1 LOADN R46 8 -LOADK R47 K3 -RETURN R0 48 +LOADN R47 1 +LOADN R48 101 +LOADK R49 K3 +RETURN R0 50 )"); } @@ -6067,7 +6126,8 @@ return math.abs(), math.max(1, true), string.byte("abc", 42), - bit32.rshift(10, 42) + bit32.rshift(10, 42), + bit32.extract(1, 2, "3") )", 0, 2), R"( @@ -6088,8 +6148,14 @@ L2: LOADN R4 10 FASTCALL2K 39 R4 K7 L3 LOADK R5 K7 GETIMPORT R3 13 -CALL R3 2 -1 -L3: RETURN R0 -1 +CALL R3 2 1 +L3: LOADN R5 1 +LOADN R6 2 +LOADK R7 K14 +FASTCALL 34 L4 +GETIMPORT R4 16 +CALL R4 3 -1 +L4: RETURN R0 -1 )"); } @@ -6146,8 +6212,6 @@ RETURN R0 1 TEST_CASE("LocalReassign") { - ScopedFastFlag sff("LuauCompileFreeReassign", true); - // locals can be re-assigned and the register gets reused CHECK_EQ("\n" + compileFunction0(R"( local function test(a, b) @@ -6459,4 +6523,26 @@ RETURN R0 0 )"); } +TEST_CASE("BuiltinExtractK") +{ + ScopedFastFlag sff("LuauCompileExtractK", true); + + // below, K0 refers to a packed f+w constant for bit32.extractk builtin + // K1 and K2 refer to 1 and 3 and are only used during fallback path + CHECK_EQ("\n" + compileFunction0(R"( +local v = ... + +return bit32.extract(v, 1, 3) +)"), R"( +GETVARARGS R0 1 +FASTCALL2K 59 R0 K0 L0 +MOVE R2 R0 +LOADK R3 K1 +LOADK R4 K2 +GETIMPORT R1 5 +CALL R1 3 -1 +L0: RETURN R1 -1 +)"); +} + TEST_SUITE_END(); diff --git a/tests/Conformance.test.cpp b/tests/Conformance.test.cpp index e07ba12a..be2feacb 100644 --- a/tests/Conformance.test.cpp +++ b/tests/Conformance.test.cpp @@ -716,6 +716,23 @@ TEST_CASE("Reference") CHECK(dtorhits == 2); } +TEST_CASE("NewUserdataOverflow") +{ + StateRef globalState(luaL_newstate(), lua_close); + lua_State* L = globalState.get(); + + lua_pushcfunction(L, [](lua_State* L1) { + // The following userdata request might cause an overflow. + lua_newuserdatadtor(L1, SIZE_MAX, [](void* d){}); + // The overflow might segfault in the following call. + lua_getmetatable(L1, -1); + return 0; + }, nullptr); + + CHECK(lua_pcall(L, 0, 0, 0) == LUA_ERRRUN); + CHECK(strcmp(lua_tostring(L, -1), "memory allocation error: block too big") == 0); +} + TEST_CASE("ApiTables") { StateRef globalState(luaL_newstate(), lua_close); diff --git a/tests/Fixture.cpp b/tests/Fixture.cpp index f0146602..40be39a0 100644 --- a/tests/Fixture.cpp +++ b/tests/Fixture.cpp @@ -443,7 +443,8 @@ BuiltinsFixture::BuiltinsFixture(bool freeze, bool prepareAutocomplete) ConstraintGraphBuilderFixture::ConstraintGraphBuilderFixture() : Fixture() - , cgb(mainModuleName, getMainModule(), &arena, NotNull(&ice), frontend.getGlobalScope()) + , mainModule(new Module) + , cgb(mainModuleName, mainModule, &arena, NotNull(&ice), frontend.getGlobalScope()) , forceTheFlag{"DebugLuauDeferredConstraintResolution", true} { BlockedTypeVar::nextIndex = 0; diff --git a/tests/Fixture.h b/tests/Fixture.h index a716fe9b..8dc3dd24 100644 --- a/tests/Fixture.h +++ b/tests/Fixture.h @@ -162,6 +162,7 @@ struct BuiltinsFixture : Fixture struct ConstraintGraphBuilderFixture : Fixture { TypeArena arena; + ModulePtr mainModule; ConstraintGraphBuilder cgb; ScopedFastFlag forceTheFlag; diff --git a/tests/Normalize.test.cpp b/tests/Normalize.test.cpp index c64c41c5..b017d8dd 100644 --- a/tests/Normalize.test.cpp +++ b/tests/Normalize.test.cpp @@ -12,6 +12,11 @@ using namespace Luau; struct NormalizeFixture : Fixture { ScopedFastFlag sff1{"LuauLowerBoundsCalculation", true}; + + bool isSubtype(TypeId a, TypeId b) + { + return ::Luau::isSubtype(a, b, NotNull{getMainModule()->getModuleScope().get()}, ice); + } }; void createSomeClasses(TypeChecker& typeChecker) @@ -49,12 +54,6 @@ void createSomeClasses(TypeChecker& typeChecker) freeze(arena); } -static bool isSubtype(TypeId a, TypeId b) -{ - InternalErrorReporter ice; - return isSubtype(a, b, ice); -} - TEST_SUITE_BEGIN("isSubtype"); TEST_CASE_FIXTURE(NormalizeFixture, "primitives") @@ -511,6 +510,8 @@ TEST_CASE_FIXTURE(NormalizeFixture, "classes") { createSomeClasses(typeChecker); + check(""); // Ensure that we have a main Module. + TypeId p = typeChecker.globalScope->lookupType("Parent")->type; TypeId c = typeChecker.globalScope->lookupType("Child")->type; TypeId u = typeChecker.globalScope->lookupType("Unrelated")->type; @@ -595,6 +596,7 @@ TEST_CASE_FIXTURE(NormalizeFixture, "union_with_overlapping_field_that_has_a_sub )"); ModulePtr tempModule{new Module}; + tempModule->scopes.emplace_back(Location(), std::make_shared(getSingletonTypes().anyTypePack)); // HACK: Normalization is an in-place operation. We need to cheat a little here and unfreeze // the arena that the type lives in. @@ -880,7 +882,6 @@ TEST_CASE_FIXTURE(Fixture, "intersection_inside_a_table_inside_another_intersect { ScopedFastFlag flags[] = { {"LuauLowerBoundsCalculation", true}, - {"LuauQuantifyConstrained", true}, }; // We use a function and inferred parameter types to prevent intermediate normalizations from being performed. @@ -921,7 +922,6 @@ TEST_CASE_FIXTURE(Fixture, "intersection_inside_a_table_inside_another_intersect { ScopedFastFlag flags[] = { {"LuauLowerBoundsCalculation", true}, - {"LuauQuantifyConstrained", true}, }; // We use a function and inferred parameter types to prevent intermediate normalizations from being performed. @@ -961,7 +961,6 @@ TEST_CASE_FIXTURE(Fixture, "intersection_inside_a_table_inside_another_intersect { ScopedFastFlag flags[] = { {"LuauLowerBoundsCalculation", true}, - {"LuauQuantifyConstrained", true}, }; // We use a function and inferred parameter types to prevent intermediate normalizations from being performed. @@ -1149,7 +1148,6 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "normalization_does_not_convert_ever") { ScopedFastFlag sff[]{ {"LuauLowerBoundsCalculation", true}, - {"LuauQuantifyConstrained", true}, }; CheckResult result = check(R"( diff --git a/tests/Parser.test.cpp b/tests/Parser.test.cpp index 5b86807c..c55ec181 100644 --- a/tests/Parser.test.cpp +++ b/tests/Parser.test.cpp @@ -2535,7 +2535,6 @@ end TEST_CASE_FIXTURE(Fixture, "error_message_for_using_function_as_type_annotation") { - ScopedFastFlag sff{"LuauParserFunctionKeywordAsTypeHelp", true}; ParseResult result = tryParse(R"( type Foo = function )"); diff --git a/tests/TypeInfer.functions.test.cpp b/tests/TypeInfer.functions.test.cpp index 074c86c3..5cc759d6 100644 --- a/tests/TypeInfer.functions.test.cpp +++ b/tests/TypeInfer.functions.test.cpp @@ -1637,7 +1637,6 @@ TEST_CASE_FIXTURE(Fixture, "quantify_constrained_types") { ScopedFastFlag sff[]{ {"LuauLowerBoundsCalculation", true}, - {"LuauQuantifyConstrained", true}, }; CheckResult result = check(R"( @@ -1662,7 +1661,6 @@ TEST_CASE_FIXTURE(Fixture, "call_o_with_another_argument_after_foo_was_quantifie { ScopedFastFlag sff[]{ {"LuauLowerBoundsCalculation", true}, - {"LuauQuantifyConstrained", true}, }; CheckResult result = check(R"( diff --git a/tests/TypeInfer.modules.test.cpp b/tests/TypeInfer.modules.test.cpp index 754fb193..b5f2296c 100644 --- a/tests/TypeInfer.modules.test.cpp +++ b/tests/TypeInfer.modules.test.cpp @@ -329,6 +329,35 @@ function tbl:foo(b: number, c: number) -- introduce BoundTypeVar to imported type arrayops.foo(self._regions) end +-- this alias decreases function type level and causes a demotion of its type +type Table = typeof(tbl) +)"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(BuiltinsFixture, "do_not_modify_imported_types_5") +{ + ScopedFastFlag luauInplaceDemoteSkipAllBound{"LuauInplaceDemoteSkipAllBound", true}; + + fileResolver.source["game/A"] = R"( +export type Type = {x: number, y: number} +local arrayops = {} +function arrayops.foo(x: Type) end +return arrayops + )"; + + CheckResult result = check(R"( +local arrayops = require(game.A) + +local tbl = {} +tbl.a = 2 +function tbl:foo(b: number, c: number) + -- introduce boundTo TableTypeVar to imported type + self.x.a = 2 + arrayops.foo(self.x) +end +-- this alias decreases function type level and causes a demotion of its type type Table = typeof(tbl) )"); diff --git a/tests/TypeInfer.provisional.test.cpp b/tests/TypeInfer.provisional.test.cpp index 01923f38..9a917a6f 100644 --- a/tests/TypeInfer.provisional.test.cpp +++ b/tests/TypeInfer.provisional.test.cpp @@ -485,7 +485,6 @@ TEST_CASE_FIXTURE(Fixture, "constrained_is_level_dependent") { ScopedFastFlag sff[]{ {"LuauLowerBoundsCalculation", true}, - {"LuauQuantifyConstrained", true}, }; CheckResult result = check(R"( diff --git a/tests/TypeInfer.refinements.test.cpp b/tests/TypeInfer.refinements.test.cpp index 8a1cadcf..e61e6e45 100644 --- a/tests/TypeInfer.refinements.test.cpp +++ b/tests/TypeInfer.refinements.test.cpp @@ -41,6 +41,7 @@ struct RefinementClassFixture : Fixture RefinementClassFixture() { TypeArena& arena = typeChecker.globalTypes; + NotNull scope{typeChecker.globalScope.get()}; unfreeze(arena); TypeId vec3 = arena.addType(ClassTypeVar{"Vector3", {}, std::nullopt, std::nullopt, {}, nullptr, "Test"}); @@ -49,7 +50,7 @@ struct RefinementClassFixture : Fixture {"Y", Property{typeChecker.numberType}}, {"Z", Property{typeChecker.numberType}}, }; - normalize(vec3, arena, *typeChecker.iceHandler); + normalize(vec3, scope, arena, *typeChecker.iceHandler); TypeId inst = arena.addType(ClassTypeVar{"Instance", {}, std::nullopt, std::nullopt, {}, nullptr, "Test"}); @@ -57,21 +58,21 @@ struct RefinementClassFixture : Fixture TypePackId isARets = arena.addTypePack({typeChecker.booleanType}); TypeId isA = arena.addType(FunctionTypeVar{isAParams, isARets}); getMutable(isA)->magicFunction = magicFunctionInstanceIsA; - normalize(isA, arena, *typeChecker.iceHandler); + normalize(isA, scope, arena, *typeChecker.iceHandler); getMutable(inst)->props = { {"Name", Property{typeChecker.stringType}}, {"IsA", Property{isA}}, }; - normalize(inst, arena, *typeChecker.iceHandler); + normalize(inst, scope, arena, *typeChecker.iceHandler); TypeId folder = typeChecker.globalTypes.addType(ClassTypeVar{"Folder", {}, inst, std::nullopt, {}, nullptr, "Test"}); - normalize(folder, arena, *typeChecker.iceHandler); + normalize(folder, scope, arena, *typeChecker.iceHandler); TypeId part = typeChecker.globalTypes.addType(ClassTypeVar{"Part", {}, inst, std::nullopt, {}, nullptr, "Test"}); getMutable(part)->props = { {"Position", Property{vec3}}, }; - normalize(part, arena, *typeChecker.iceHandler); + normalize(part, scope, arena, *typeChecker.iceHandler); typeChecker.globalScope->exportedTypeBindings["Vector3"] = TypeFun{{}, vec3}; typeChecker.globalScope->exportedTypeBindings["Instance"] = TypeFun{{}, inst}; @@ -934,8 +935,6 @@ TEST_CASE_FIXTURE(Fixture, "apply_refinements_on_astexprindexexpr_whose_subscrip TEST_CASE_FIXTURE(Fixture, "discriminate_from_truthiness_of_x") { - ScopedFastFlag sff{"LuauFalsyPredicateReturnsNilInstead", true}; - CheckResult result = check(R"( type T = {tag: "missing", x: nil} | {tag: "exists", x: string} @@ -1230,8 +1229,6 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "refine_unknowns") TEST_CASE_FIXTURE(BuiltinsFixture, "falsiness_of_TruthyPredicate_narrows_into_nil") { - ScopedFastFlag sff{"LuauFalsyPredicateReturnsNilInstead", true}; - CheckResult result = check(R"( local function f(t: {number}) local x = t[1] diff --git a/tests/TypeInfer.tables.test.cpp b/tests/TypeInfer.tables.test.cpp index 8c7d8a53..95b85c48 100644 --- a/tests/TypeInfer.tables.test.cpp +++ b/tests/TypeInfer.tables.test.cpp @@ -3003,8 +3003,6 @@ TEST_CASE_FIXTURE(Fixture, "expected_indexer_from_table_union") TEST_CASE_FIXTURE(Fixture, "prop_access_on_key_whose_types_mismatches") { - ScopedFastFlag sff{"LuauReportErrorsOnIndexerKeyMismatch", true}; - CheckResult result = check(R"( local t: {number} = {} local x = t.x @@ -3016,8 +3014,6 @@ TEST_CASE_FIXTURE(Fixture, "prop_access_on_key_whose_types_mismatches") TEST_CASE_FIXTURE(Fixture, "prop_access_on_unions_of_indexers_where_key_whose_types_mismatches") { - ScopedFastFlag sff{"LuauReportErrorsOnIndexerKeyMismatch", true}; - CheckResult result = check(R"( local t: { [number]: number } | { [boolean]: number } = {} local u = t.x diff --git a/tests/TypeInfer.tryUnify.test.cpp b/tests/TypeInfer.tryUnify.test.cpp index e0a0e5b5..02fdfd73 100644 --- a/tests/TypeInfer.tryUnify.test.cpp +++ b/tests/TypeInfer.tryUnify.test.cpp @@ -17,7 +17,8 @@ struct TryUnifyFixture : Fixture ScopePtr globalScope{new Scope{arena.addTypePack({TypeId{}})}}; InternalErrorReporter iceHandler; UnifierSharedState unifierState{&iceHandler}; - Unifier state{&arena, Mode::Strict, Location{}, Variance::Covariant, unifierState}; + + Unifier state{&arena, Mode::Strict, NotNull{globalScope.get()}, Location{}, Variance::Covariant, unifierState}; }; TEST_SUITE_BEGIN("TryUnifyTests"); diff --git a/tests/TypeVar.test.cpp b/tests/TypeVar.test.cpp index edec8442..32be8215 100644 --- a/tests/TypeVar.test.cpp +++ b/tests/TypeVar.test.cpp @@ -193,6 +193,11 @@ TEST_CASE_FIXTURE(Fixture, "UnionTypeVarIterator_with_only_cyclic_union") CHECK(actual.empty()); } + +/* FIXME: This test is pretty weird. It would be much nicer if we could + * perform this operation without a TypeChecker so that we don't have to jam + * all this state into it to make stuff work. + */ TEST_CASE_FIXTURE(Fixture, "substitution_skip_failure") { TypeVar ftv11{FreeTypeVar{TypeLevel{}}}; @@ -268,6 +273,7 @@ TEST_CASE_FIXTURE(Fixture, "substitution_skip_failure") TypeId root = &ttvTweenResult; typeChecker.currentModule = std::make_shared(); + typeChecker.currentModule->scopes.emplace_back(Location{}, std::make_shared(getSingletonTypes().anyTypePack)); TypeId result = typeChecker.anyify(typeChecker.globalScope, root, Location{}); diff --git a/tests/conformance/bitwise.lua b/tests/conformance/bitwise.lua index 13be3f94..f0c5698d 100644 --- a/tests/conformance/bitwise.lua +++ b/tests/conformance/bitwise.lua @@ -100,6 +100,9 @@ assert(bit32.extract(0xa0001111, 28, 4) == 0xa) assert(bit32.extract(0xa0001111, 31, 1) == 1) assert(bit32.extract(0x50000111, 31, 1) == 0) assert(bit32.extract(0xf2345679, 0, 32) == 0xf2345679) +assert(bit32.extract(0xa0001111, 16) == 0) +assert(bit32.extract(0xa0001111, 31) == 1) +assert(bit32.extract(42, 1, 3) == 5) assert(not pcall(bit32.extract, 0, -1)) assert(not pcall(bit32.extract, 0, 32)) @@ -152,5 +155,6 @@ assert(bit32.btest(1, "3") == true) assert(bit32.btest("1", 3) == true) assert(bit32.countlz("42") == 26) assert(bit32.countrz("42") == 1) +assert(bit32.extract("42", 1, 3) == 5) return('OK') diff --git a/tools/faillist.txt b/tools/faillist.txt index d796236d..630bf9f7 100644 --- a/tools/faillist.txt +++ b/tools/faillist.txt @@ -10,12 +10,10 @@ AnnotationTests.luau_ice_triggers_an_ice_exception_with_flag AnnotationTests.luau_ice_triggers_an_ice_exception_with_flag_handler AnnotationTests.luau_ice_triggers_an_ice_handler AnnotationTests.luau_print_is_magic_if_the_flag_is_set -AnnotationTests.luau_print_is_not_special_without_the_flag AnnotationTests.occurs_check_on_cyclic_intersection_typevar AnnotationTests.occurs_check_on_cyclic_union_typevar AnnotationTests.too_many_type_params AnnotationTests.two_type_params -AnnotationTests.unknown_type_reference_generates_error AnnotationTests.use_type_required_from_another_file AstQuery.last_argument_function_call_type AstQuery::getDocumentationSymbolAtPosition.overloaded_fn @@ -27,17 +25,13 @@ AutocompleteTest.autocomplete_end_with_lambda AutocompleteTest.autocomplete_first_function_arg_expected_type AutocompleteTest.autocomplete_for_in_middle_keywords AutocompleteTest.autocomplete_for_middle_keywords -AutocompleteTest.autocomplete_if_else_regression AutocompleteTest.autocomplete_if_middle_keywords -AutocompleteTest.autocomplete_ifelse_expressions AutocompleteTest.autocomplete_on_string_singletons AutocompleteTest.autocomplete_oop_implicit_self AutocompleteTest.autocomplete_repeat_middle_keyword AutocompleteTest.autocomplete_string_singleton_equality AutocompleteTest.autocomplete_string_singleton_escape AutocompleteTest.autocomplete_string_singletons -AutocompleteTest.autocomplete_until_expression -AutocompleteTest.autocomplete_until_in_repeat AutocompleteTest.autocomplete_while_middle_keywords AutocompleteTest.autocompleteProp_index_function_metamethod_is_variadic AutocompleteTest.bias_toward_inner_scope @@ -60,7 +54,6 @@ AutocompleteTest.get_suggestions_for_the_very_start_of_the_script AutocompleteTest.global_function_params AutocompleteTest.global_functions_are_not_scoped_lexically AutocompleteTest.if_then_else_elseif_completions -AutocompleteTest.if_then_else_full_keywords AutocompleteTest.keyword_methods AutocompleteTest.keyword_types AutocompleteTest.library_non_self_calls_are_fine @@ -181,7 +174,6 @@ DefinitionTests.single_class_type_identity_in_global_types FrontendTest.ast_node_at_position FrontendTest.automatically_check_dependent_scripts FrontendTest.check_without_builtin_next -FrontendTest.clearStats FrontendTest.dont_reparse_clean_file_when_linting FrontendTest.environments FrontendTest.imported_table_modification_2 @@ -195,7 +187,6 @@ FrontendTest.reexport_cyclic_type FrontendTest.reexport_type_alias FrontendTest.report_require_to_nonexistent_file FrontendTest.report_syntax_error_in_required_file -FrontendTest.stats_are_not_reset_between_checks FrontendTest.trace_requires_in_nonstrict_mode GenericsTests.apply_type_function_nested_generics1 GenericsTests.apply_type_function_nested_generics2 @@ -213,8 +204,6 @@ GenericsTests.duplicate_generic_types GenericsTests.error_detailed_function_mismatch_generic_pack GenericsTests.error_detailed_function_mismatch_generic_types GenericsTests.factories_of_generics -GenericsTests.function_arguments_can_be_polytypes -GenericsTests.function_results_can_be_polytypes GenericsTests.generic_argument_count_too_few GenericsTests.generic_argument_count_too_many GenericsTests.generic_factories @@ -243,7 +232,6 @@ GenericsTests.properties_can_be_instantiated_polytypes GenericsTests.rank_N_types_via_typeof GenericsTests.reject_clashing_generic_and_pack_names GenericsTests.self_recursive_instantiated_param -GenericsTests.variadic_generics IntersectionTypes.argument_is_intersection IntersectionTypes.error_detailed_intersection_all IntersectionTypes.error_detailed_intersection_part @@ -289,7 +277,6 @@ Normalize.cyclic_intersection Normalize.cyclic_table_normalizes_sensibly Normalize.cyclic_union Normalize.fuzz_failure_bound_type_is_normal_but_not_its_bounded_to -Normalize.higher_order_function Normalize.intersection_combine_on_bound_self Normalize.intersection_inside_a_table_inside_another_intersection Normalize.intersection_inside_a_table_inside_another_intersection_2 @@ -304,7 +291,6 @@ Normalize.normalization_does_not_convert_ever Normalize.normalize_module_return_type Normalize.normalize_unions_containing_never Normalize.normalize_unions_containing_unknown -Normalize.return_type_is_not_a_constrained_intersection Normalize.union_of_distinct_free_types Normalize.variadic_tail_is_marked_normal Normalize.visiting_a_type_twice_is_not_considered_normal @@ -456,7 +442,6 @@ TableTests.inferred_return_type_of_free_table TableTests.inferring_crazy_table_should_also_be_quick TableTests.instantiate_table_cloning_3 TableTests.instantiate_tables_at_scope_level -TableTests.invariant_table_properties_means_instantiating_tables_in_assignment_is_unsound TableTests.leaking_bad_metatable_errors TableTests.length_operator_intersection TableTests.length_operator_non_table_union @@ -521,7 +506,6 @@ ToDot.function ToDot.metatable ToDot.table ToString.exhaustive_toString_of_cyclic_table -ToString.function_type_with_argument_names_and_self ToString.function_type_with_argument_names_generic ToString.no_parentheses_around_cyclic_function_type_in_union ToString.toStringDetailed2 @@ -565,10 +549,7 @@ TypeInfer.do_not_bind_a_free_table_to_a_union_containing_that_table TypeInfer.dont_report_type_errors_within_an_AstStatError TypeInfer.globals TypeInfer.globals2 -TypeInfer.infer_assignment_value_types TypeInfer.infer_assignment_value_types_mutable_lval -TypeInfer.infer_through_group_expr -TypeInfer.no_heap_use_after_free_error TypeInfer.no_stack_overflow_from_isoptional TypeInfer.tc_after_error_recovery_no_replacement_name_in_error TypeInfer.tc_if_else_expressions1 @@ -589,11 +570,8 @@ TypeInferAnyError.for_in_loop_iterator_returns_any TypeInferAnyError.for_in_loop_iterator_returns_any2 TypeInferAnyError.length_of_error_type_does_not_produce_an_error TypeInferAnyError.replace_every_free_type_when_unifying_a_complex_function_with_any -TypeInferAnyError.type_error_addition TypeInferClasses.call_base_method TypeInferClasses.call_instance_method -TypeInferClasses.can_read_prop_of_base_class -TypeInferClasses.can_read_prop_of_base_class_using_string TypeInferClasses.class_type_mismatch_with_name_conflict TypeInferClasses.classes_can_have_overloaded_operators TypeInferClasses.classes_without_overloaded_operators_cannot_be_added @@ -609,8 +587,6 @@ TypeInferFunctions.another_indirect_function_case_where_it_is_ok_to_provide_too_ TypeInferFunctions.another_recursive_local_function TypeInferFunctions.calling_function_with_anytypepack_doesnt_leak_free_types TypeInferFunctions.calling_function_with_incorrect_argument_type_yields_errors_spanning_argument -TypeInferFunctions.cannot_hoist_interior_defns_into_signature -TypeInferFunctions.check_function_before_lambda_that_uses_it TypeInferFunctions.complicated_return_types_require_an_explicit_annotation TypeInferFunctions.cyclic_function_type_in_args TypeInferFunctions.dont_give_other_overloads_message_if_only_one_argument_matching_overload_exists @@ -635,7 +611,6 @@ TypeInferFunctions.ignored_return_values TypeInferFunctions.inconsistent_higher_order_function TypeInferFunctions.inconsistent_return_types TypeInferFunctions.infer_anonymous_function_arguments -TypeInferFunctions.infer_anonymous_function_arguments_outside_call TypeInferFunctions.infer_return_type_from_selected_overload TypeInferFunctions.infer_that_function_does_not_return_a_table TypeInferFunctions.it_is_ok_not_to_supply_enough_retvals @@ -680,9 +655,6 @@ TypeInferLoops.loop_iter_no_indexer_strict TypeInferLoops.loop_iter_trailing_nil TypeInferLoops.loop_typecheck_crash_on_empty_optional TypeInferLoops.properly_infer_iteratee_is_a_free_table -TypeInferLoops.repeat_loop -TypeInferLoops.repeat_loop_condition_binds_to_its_block -TypeInferLoops.symbols_in_repeat_block_should_not_be_visible_beyond_until_condition TypeInferLoops.unreachable_code_after_infinite_loop TypeInferLoops.varlist_declared_by_for_in_loop_should_be_free TypeInferModules.do_not_modify_imported_types @@ -718,8 +690,6 @@ TypeInferOperators.cannot_indirectly_compare_types_that_do_not_offer_overloaded_ TypeInferOperators.cli_38355_recursive_union TypeInferOperators.compare_numbers TypeInferOperators.compare_strings -TypeInferOperators.compound_assign_basic -TypeInferOperators.compound_assign_metatable TypeInferOperators.compound_assign_mismatch_metatable TypeInferOperators.compound_assign_mismatch_op TypeInferOperators.compound_assign_mismatch_result @@ -775,7 +745,6 @@ TypeInferUnknownNever.type_packs_containing_never_is_itself_uninhabitable TypeInferUnknownNever.type_packs_containing_never_is_itself_uninhabitable2 TypeInferUnknownNever.unary_minus_of_never TypeInferUnknownNever.unknown_is_reflexive -TypePackTests.cyclic_type_packs TypePackTests.higher_order_function TypePackTests.multiple_varargs_inference_are_not_confused TypePackTests.no_return_size_should_be_zero diff --git a/tools/heapgraph.py b/tools/heapgraph.py index 2817c38f..b8dc207f 100644 --- a/tools/heapgraph.py +++ b/tools/heapgraph.py @@ -132,8 +132,16 @@ while offset < len(queue): queue.append((obj["metatable"], node.child("__meta"))) elif obj["type"] == "thread": queue.append((obj["env"], node.child("__env"))) - for a in obj.get("stack", []): - queue.append((a, node.child("__stack"))) + stack = obj.get("stack") + stacknames = obj.get("stacknames", []) + stacknode = node.child("__stack") + framenode = None + for i in range(len(stack)): + name = stacknames[i] if stacknames else None + if name and name.startswith("frame:"): + framenode = stacknode.child(name[6:]) + name = None + queue.append((stack[i], framenode.child(name) if framenode and name else framenode or stacknode)) elif obj["type"] == "proto": for a in obj.get("constants", []): queue.append((a, node)) diff --git a/tools/heapstat.py b/tools/heapstat.py index 4c0cb407..7337aa44 100644 --- a/tools/heapstat.py +++ b/tools/heapstat.py @@ -59,5 +59,6 @@ if len(size_category) != 0: print("objects by category:") for type, (count, size) in sortedsize(size_category.items()): - name = dump["stats"]["categories"][type]["name"] + cat = dump["stats"]["categories"][type] + name = cat["name"] if "name" in cat else str(type) print(name.ljust(30), str(size).rjust(8), "bytes", str(count).rjust(5), "objects") diff --git a/tools/perfgraph.py b/tools/perfgraph.py index ae74b7d6..25bb0dd9 100644 --- a/tools/perfgraph.py +++ b/tools/perfgraph.py @@ -56,18 +56,28 @@ def nodeFromCallstackListFile(source_file): return root +def getDuration(obj): + total = obj['TotalDuration'] -def nodeFromJSONbject(node, key, obj): + if 'Children' in obj: + for key, obj in obj['Children'].items(): + total -= obj['TotalDuration'] + + return total + + +def nodeFromJSONObject(node, key, obj): source, function, line = key.split(",") node.function = function node.source = source node.line = int(line) if len(line) > 0 else 0 - node.ticks = obj['Duration'] + node.ticks = getDuration(obj) - for key, obj in obj['Children'].items(): - nodeFromJSONbject(node.child(key), key, obj) + if 'Children' in obj: + for key, obj in obj['Children'].items(): + nodeFromJSONObject(node.child(key), key, obj) return node @@ -77,8 +87,9 @@ def nodeFromJSONFile(source_file): root = Node() - for key, obj in dump['Children'].items(): - nodeFromJSONbject(root.child(key), key, obj) + if 'Children' in dump: + for key, obj in dump['Children'].items(): + nodeFromJSONObject(root.child(key), key, obj) return root From be2769ad146a0b1c2d3588802196f49e4c444c8d Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Thu, 18 Aug 2022 14:32:08 -0700 Subject: [PATCH 345/399] Sync to upstream/release/541 (#644) - Fix autocomplete not suggesting globals defined after the cursor (fixes #622) - Improve type checker stability - Reduce parser C stack consumption which fixes some stack overflow crashes on deeply nested sources - Improve performance of bit32.extract/replace when width is implied (~3% faster chess) - Improve performance of bit32.extract when field/width are constants (~10% faster base64) - Heap dump now annotates thread stacks with local variable/function names --- Analysis/include/Luau/Anyification.h | 38 + Analysis/include/Luau/Constraint.h | 4 +- .../include/Luau/ConstraintGraphBuilder.h | 7 +- Analysis/include/Luau/ConstraintSolver.h | 11 +- Analysis/include/Luau/Module.h | 2 + Analysis/include/Luau/Normalize.h | 20 +- Analysis/include/Luau/TypeInfer.h | 63 +- Analysis/include/Luau/TypeUtils.h | 7 +- Analysis/include/Luau/TypedAllocator.h | 4 +- Analysis/include/Luau/Unifier.h | 6 +- Analysis/include/Luau/VisitTypeVar.h | 22 +- Analysis/src/Anyification.cpp | 96 +++ Analysis/src/Autocomplete.cpp | 45 +- Analysis/src/Constraint.cpp | 3 +- Analysis/src/ConstraintGraphBuilder.cpp | 98 ++- Analysis/src/ConstraintSolver.cpp | 42 +- Analysis/src/Frontend.cpp | 3 + Analysis/src/Module.cpp | 10 +- Analysis/src/Normalize.cpp | 67 +- Analysis/src/Quantify.cpp | 42 +- Analysis/src/TypeChecker2.cpp | 741 +++++++++++++----- Analysis/src/TypeInfer.cpp | 297 +++---- Analysis/src/TypeUtils.cpp | 111 ++- Analysis/src/TypeVar.cpp | 14 +- Analysis/src/TypedAllocator.cpp | 50 +- Analysis/src/Unifier.cpp | 10 +- Ast/include/Luau/Parser.h | 31 +- Ast/src/Parser.cpp | 322 ++++---- Common/include/Luau/Bytecode.h | 3 + Compiler/src/BuiltinFolding.cpp | 11 +- Compiler/src/Compiler.cpp | 108 ++- Sources.cmake | 4 +- VM/src/lapi.cpp | 5 +- VM/src/lbuiltins.cpp | 91 ++- VM/src/lfunc.cpp | 13 +- VM/src/lfunc.h | 1 + VM/src/lgcdebug.cpp | 57 +- tests/Autocomplete.test.cpp | 23 + tests/Compiler.test.cpp | 116 ++- tests/Conformance.test.cpp | 2 +- tests/Fixture.cpp | 3 +- tests/Fixture.h | 1 + tests/Normalize.test.cpp | 18 +- tests/Parser.test.cpp | 1 - tests/TypeInfer.functions.test.cpp | 2 - tests/TypeInfer.modules.test.cpp | 29 + tests/TypeInfer.provisional.test.cpp | 1 - tests/TypeInfer.refinements.test.cpp | 15 +- tests/TypeInfer.tables.test.cpp | 4 - tests/TypeInfer.tryUnify.test.cpp | 3 +- tests/TypeVar.test.cpp | 6 + tests/conformance/bitwise.lua | 4 + tools/faillist.txt | 31 - tools/heapgraph.py | 12 +- tools/heapstat.py | 3 +- tools/perfgraph.py | 23 +- 56 files changed, 1803 insertions(+), 953 deletions(-) create mode 100644 Analysis/include/Luau/Anyification.h create mode 100644 Analysis/src/Anyification.cpp diff --git a/Analysis/include/Luau/Anyification.h b/Analysis/include/Luau/Anyification.h new file mode 100644 index 00000000..ee8d6689 --- /dev/null +++ b/Analysis/include/Luau/Anyification.h @@ -0,0 +1,38 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details + +#pragma once + +#include "Luau/NotNull.h" +#include "Luau/Substitution.h" +#include "Luau/TypeVar.h" + +#include + +namespace Luau +{ + +struct TypeArena; +struct Scope; +struct InternalErrorReporter; +using ScopePtr = std::shared_ptr; + +// A substitution which replaces free types by any +struct Anyification : Substitution +{ + Anyification(TypeArena* arena, const ScopePtr& scope, InternalErrorReporter* iceHandler, TypeId anyType, TypePackId anyTypePack); + NotNull scope; + InternalErrorReporter* iceHandler; + + TypeId anyType; + TypePackId anyTypePack; + bool normalizationTooComplex = false; + bool isDirty(TypeId ty) override; + bool isDirty(TypePackId tp) override; + TypeId clean(TypeId ty) override; + TypePackId clean(TypePackId tp) override; + + bool ignoreChildren(TypeId ty) override; + bool ignoreChildren(TypePackId ty) override; +}; + +} \ No newline at end of file diff --git a/Analysis/include/Luau/Constraint.h b/Analysis/include/Luau/Constraint.h index 5b737146..ce90f2c5 100644 --- a/Analysis/include/Luau/Constraint.h +++ b/Analysis/include/Luau/Constraint.h @@ -40,7 +40,6 @@ struct GeneralizationConstraint { TypeId generalizedType; TypeId sourceType; - Scope* scope; }; // subType ~ inst superType @@ -85,13 +84,14 @@ using ConstraintPtr = std::unique_ptr; struct Constraint { - explicit Constraint(ConstraintV&& c); + Constraint(ConstraintV&& c, NotNull scope); Constraint(const Constraint&) = delete; Constraint& operator=(const Constraint&) = delete; ConstraintV c; std::vector> dependencies; + NotNull scope; }; inline Constraint& asMutable(const Constraint& c) diff --git a/Analysis/include/Luau/ConstraintGraphBuilder.h b/Analysis/include/Luau/ConstraintGraphBuilder.h index 41d14326..0e41e1ee 100644 --- a/Analysis/include/Luau/ConstraintGraphBuilder.h +++ b/Analysis/include/Luau/ConstraintGraphBuilder.h @@ -72,10 +72,10 @@ struct ConstraintGraphBuilder /** * Fabricates a scope that is a child of another scope. - * @param location the lexical extent of the scope in the source code. + * @param node the lexical node that the scope belongs to. * @param parent the parent scope of the new scope. Must not be null. */ - ScopePtr childScope(Location location, const ScopePtr& parent); + ScopePtr childScope(AstNode* node, const ScopePtr& parent); /** * Adds a new constraint with no dependencies to a given scope. @@ -105,10 +105,12 @@ struct ConstraintGraphBuilder void visit(const ScopePtr& scope, AstStatLocal* local); void visit(const ScopePtr& scope, AstStatFor* for_); void visit(const ScopePtr& scope, AstStatWhile* while_); + void visit(const ScopePtr& scope, AstStatRepeat* repeat); void visit(const ScopePtr& scope, AstStatLocalFunction* function); void visit(const ScopePtr& scope, AstStatFunction* function); void visit(const ScopePtr& scope, AstStatReturn* ret); void visit(const ScopePtr& scope, AstStatAssign* assign); + void visit(const ScopePtr& scope, AstStatCompoundAssign* assign); void visit(const ScopePtr& scope, AstStatIf* ifStatement); void visit(const ScopePtr& scope, AstStatTypeAlias* alias); void visit(const ScopePtr& scope, AstStatDeclareGlobal* declareGlobal); @@ -133,6 +135,7 @@ struct ConstraintGraphBuilder TypeId check(const ScopePtr& scope, AstExprIndexExpr* indexExpr); TypeId check(const ScopePtr& scope, AstExprUnary* unary); TypeId check(const ScopePtr& scope, AstExprBinary* binary); + TypeId check(const ScopePtr& scope, AstExprIfElse* ifElse); TypeId check(const ScopePtr& scope, AstExprTypeAssertion* typeAssert); struct FunctionSignature diff --git a/Analysis/include/Luau/ConstraintSolver.h b/Analysis/include/Luau/ConstraintSolver.h index 9cc0e4cb..661d120e 100644 --- a/Analysis/include/Luau/ConstraintSolver.h +++ b/Analysis/include/Luau/ConstraintSolver.h @@ -60,6 +60,9 @@ struct ConstraintSolver // Memoized instantiations of type aliases. DenseHashMap instantiatedAliases{{}}; + // Recorded errors that take place within the solver. + ErrorVec errors; + ConstraintSolverLogger logger; explicit ConstraintSolver(TypeArena* arena, NotNull rootScope); @@ -115,7 +118,7 @@ struct ConstraintSolver * @param subType the sub-type to unify. * @param superType the super-type to unify. */ - void unify(TypeId subType, TypeId superType); + void unify(TypeId subType, TypeId superType, NotNull scope); /** * Creates a new Unifier and performs a single unification operation. Commits @@ -123,13 +126,15 @@ struct ConstraintSolver * @param subPack the sub-type pack to unify. * @param superPack the super-type pack to unify. */ - void unify(TypePackId subPack, TypePackId superPack); + void unify(TypePackId subPack, TypePackId superPack, NotNull scope); /** Pushes a new solver constraint to the solver. * @param cv the body of the constraint. **/ - void pushConstraint(ConstraintV cv); + void pushConstraint(ConstraintV cv, NotNull scope); + void reportError(TypeErrorData&& data, const Location& location); + void reportError(TypeError e); private: /** * Marks a constraint as being blocked on a type or type pack. The constraint diff --git a/Analysis/include/Luau/Module.h b/Analysis/include/Luau/Module.h index 6f4c6098..bec51b81 100644 --- a/Analysis/include/Luau/Module.h +++ b/Analysis/include/Luau/Module.h @@ -77,6 +77,8 @@ struct Module DenseHashMap astOverloadResolvedTypes{nullptr}; DenseHashMap astResolvedTypes{nullptr}; DenseHashMap astResolvedTypePacks{nullptr}; + // Map AST nodes to the scope they create. Cannot be NotNull because we need a sentinel value for the map. + DenseHashMap astScopes{nullptr}; std::unordered_map declaredGlobals; ErrorVec errors; diff --git a/Analysis/include/Luau/Normalize.h b/Analysis/include/Luau/Normalize.h index f5fd9886..78b241e4 100644 --- a/Analysis/include/Luau/Normalize.h +++ b/Analysis/include/Luau/Normalize.h @@ -1,20 +1,28 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details -#include "Luau/Substitution.h" -#include "Luau/TypeVar.h" #include "Luau/Module.h" +#include "Luau/NotNull.h" +#include "Luau/TypeVar.h" + +#include namespace Luau { struct InternalErrorReporter; +struct Module; +struct Scope; -bool isSubtype(TypeId subTy, TypeId superTy, InternalErrorReporter& ice); -bool isSubtype(TypePackId subTy, TypePackId superTy, InternalErrorReporter& ice); +using ModulePtr = std::shared_ptr; -std::pair normalize(TypeId ty, TypeArena& arena, InternalErrorReporter& ice); +bool isSubtype(TypeId subTy, TypeId superTy, NotNull scope, InternalErrorReporter& ice); +bool isSubtype(TypePackId subTy, TypePackId superTy, NotNull scope, InternalErrorReporter& ice); + +std::pair normalize(TypeId ty, NotNull scope, TypeArena& arena, InternalErrorReporter& ice); +std::pair normalize(TypeId ty, NotNull module, InternalErrorReporter& ice); std::pair normalize(TypeId ty, const ModulePtr& module, InternalErrorReporter& ice); -std::pair normalize(TypePackId ty, TypeArena& arena, InternalErrorReporter& ice); +std::pair normalize(TypePackId ty, NotNull scope, TypeArena& arena, InternalErrorReporter& ice); +std::pair normalize(TypePackId ty, NotNull module, InternalErrorReporter& ice); std::pair normalize(TypePackId ty, const ModulePtr& module, InternalErrorReporter& ice); } // namespace Luau diff --git a/Analysis/include/Luau/TypeInfer.h b/Analysis/include/Luau/TypeInfer.h index 5c55ddb0..80f9085f 100644 --- a/Analysis/include/Luau/TypeInfer.h +++ b/Analysis/include/Luau/TypeInfer.h @@ -1,6 +1,7 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #pragma once +#include "Luau/Anyification.h" #include "Luau/Predicate.h" #include "Luau/Error.h" #include "Luau/Module.h" @@ -36,40 +37,6 @@ const AstStat* getFallthrough(const AstStat* node); struct UnifierOptions; struct Unifier; -// A substitution which replaces free types by any -struct Anyification : Substitution -{ - Anyification(TypeArena* arena, InternalErrorReporter* iceHandler, TypeId anyType, TypePackId anyTypePack) - : Substitution(TxnLog::empty(), arena) - , iceHandler(iceHandler) - , anyType(anyType) - , anyTypePack(anyTypePack) - { - } - - InternalErrorReporter* iceHandler; - - TypeId anyType; - TypePackId anyTypePack; - bool normalizationTooComplex = false; - bool isDirty(TypeId ty) override; - bool isDirty(TypePackId tp) override; - TypeId clean(TypeId ty) override; - TypePackId clean(TypePackId tp) override; - - bool ignoreChildren(TypeId ty) override - { - if (FFlag::LuauClassTypeVarsInSubstitution && get(ty)) - return true; - - return ty->persistent; - } - bool ignoreChildren(TypePackId ty) override - { - return ty->persistent; - } -}; - struct GenericTypeDefinitions { std::vector genericTypes; @@ -196,32 +163,32 @@ struct TypeChecker /** Attempt to unify the types. * Treat any failures as type errors in the final typecheck report. */ - bool unify(TypeId subTy, TypeId superTy, const Location& location); - bool unify(TypeId subTy, TypeId superTy, const Location& location, const UnifierOptions& options); - bool unify(TypePackId subTy, TypePackId superTy, const Location& location, CountMismatch::Context ctx = CountMismatch::Context::Arg); + bool unify(TypeId subTy, TypeId superTy, const ScopePtr& scope, const Location& location); + bool unify(TypeId subTy, TypeId superTy, const ScopePtr& scope, const Location& location, const UnifierOptions& options); + bool unify(TypePackId subTy, TypePackId superTy, const ScopePtr& scope, const Location& location, CountMismatch::Context ctx = CountMismatch::Context::Arg); /** Attempt to unify the types. * If this fails, and the subTy type can be instantiated, do so and try unification again. */ - bool unifyWithInstantiationIfNeeded(const ScopePtr& scope, TypeId subTy, TypeId superTy, const Location& location); - void unifyWithInstantiationIfNeeded(const ScopePtr& scope, TypeId subTy, TypeId superTy, Unifier& state); + bool unifyWithInstantiationIfNeeded(TypeId subTy, TypeId superTy, const ScopePtr& scope, const Location& location); + void unifyWithInstantiationIfNeeded(TypeId subTy, TypeId superTy, const ScopePtr& scope, Unifier& state); /** Attempt to unify. * If there are errors, undo everything and return the errors. * If there are no errors, commit and return an empty error vector. */ template - ErrorVec tryUnify_(Id subTy, Id superTy, const Location& location); - ErrorVec tryUnify(TypeId subTy, TypeId superTy, const Location& location); - ErrorVec tryUnify(TypePackId subTy, TypePackId superTy, const Location& location); + ErrorVec tryUnify_(Id subTy, Id superTy, const ScopePtr& scope, const Location& location); + ErrorVec tryUnify(TypeId subTy, TypeId superTy, const ScopePtr& scope, const Location& location); + ErrorVec tryUnify(TypePackId subTy, TypePackId superTy, const ScopePtr& scope, const Location& location); // Test whether the two type vars unify. Never commits the result. template - ErrorVec canUnify_(Id subTy, Id superTy, const Location& location); - ErrorVec canUnify(TypeId subTy, TypeId superTy, const Location& location); - ErrorVec canUnify(TypePackId subTy, TypePackId superTy, const Location& location); + ErrorVec canUnify_(Id subTy, Id superTy, const ScopePtr& scope, const Location& location); + ErrorVec canUnify(TypeId subTy, TypeId superTy, const ScopePtr& scope, const Location& location); + ErrorVec canUnify(TypePackId subTy, TypePackId superTy, const ScopePtr& scope, const Location& location); - void unifyLowerBound(TypePackId subTy, TypePackId superTy, TypeLevel demotedLevel, const Location& location); + void unifyLowerBound(TypePackId subTy, TypePackId superTy, TypeLevel demotedLevel, const ScopePtr& scope, const Location& location); std::optional findMetatableEntry(TypeId type, std::string entry, const Location& location, bool addErrors); std::optional findTablePropertyRespectingMeta(TypeId lhsType, Name name, const Location& location, bool addErrors); @@ -290,7 +257,7 @@ private: void reportErrorCodeTooComplex(const Location& location); private: - Unifier mkUnifier(const Location& location); + Unifier mkUnifier(const ScopePtr& scope, const Location& location); // These functions are only safe to call when we are in the process of typechecking a module. @@ -312,7 +279,7 @@ public: std::pair, bool> pickTypesFromSense(TypeId type, bool sense); private: - TypeId unionOfTypes(TypeId a, TypeId b, const Location& location, bool unifyFreeTypes = true); + TypeId unionOfTypes(TypeId a, TypeId b, const ScopePtr& scope, const Location& location, bool unifyFreeTypes = true); // ex // TypeId id = addType(FreeTypeVar()); diff --git a/Analysis/include/Luau/TypeUtils.h b/Analysis/include/Luau/TypeUtils.h index 42c1bc0b..0aff5a7d 100644 --- a/Analysis/include/Luau/TypeUtils.h +++ b/Analysis/include/Luau/TypeUtils.h @@ -13,7 +13,10 @@ namespace Luau using ScopePtr = std::shared_ptr; -std::optional findMetatableEntry(ErrorVec& errors, TypeId type, std::string entry, Location location); -std::optional findTablePropertyRespectingMeta(ErrorVec& errors, TypeId ty, Name name, Location location); +std::optional findMetatableEntry(ErrorVec& errors, TypeId type, const std::string& entry, Location location); +std::optional findTablePropertyRespectingMeta(ErrorVec& errors, TypeId ty, const std::string& name, Location location); +std::optional getIndexTypeFromType( + const ScopePtr& scope, ErrorVec& errors, TypeArena* arena, TypeId type, const std::string& prop, const Location& location, bool addErrors, + InternalErrorReporter& handle); } // namespace Luau diff --git a/Analysis/include/Luau/TypedAllocator.h b/Analysis/include/Luau/TypedAllocator.h index f67e3d8e..a5dd17b6 100644 --- a/Analysis/include/Luau/TypedAllocator.h +++ b/Analysis/include/Luau/TypedAllocator.h @@ -10,7 +10,7 @@ namespace Luau { void* pagedAllocate(size_t size); -void pagedDeallocate(void* ptr); +void pagedDeallocate(void* ptr, size_t size); void pagedFreeze(void* ptr, size_t size); void pagedUnfreeze(void* ptr, size_t size); @@ -113,7 +113,7 @@ private: for (size_t i = 0; i < blockSize; ++i) block[i].~T(); - pagedDeallocate(block); + pagedDeallocate(block, kBlockSizeBytes); } stuff.clear(); diff --git a/Analysis/include/Luau/Unifier.h b/Analysis/include/Luau/Unifier.h index 9fa29076..312b0584 100644 --- a/Analysis/include/Luau/Unifier.h +++ b/Analysis/include/Luau/Unifier.h @@ -3,9 +3,10 @@ #include "Luau/Error.h" #include "Luau/Location.h" +#include "Luau/Scope.h" #include "Luau/TxnLog.h" -#include "Luau/TypeInfer.h" #include "Luau/TypeArena.h" +#include "Luau/TypeInfer.h" #include "Luau/UnifierSharedState.h" #include @@ -48,6 +49,7 @@ struct Unifier TypeArena* const types; Mode mode; + NotNull scope; // const Scope maybe TxnLog log; ErrorVec errors; Location location; @@ -57,7 +59,7 @@ struct Unifier UnifierSharedState& sharedState; - Unifier(TypeArena* types, Mode mode, const Location& location, Variance variance, UnifierSharedState& sharedState, TxnLog* parentLog = nullptr); + Unifier(TypeArena* types, Mode mode, NotNull scope, const Location& location, Variance variance, UnifierSharedState& sharedState, TxnLog* parentLog = nullptr); // Test whether the two type vars unify. Never commits the result. ErrorVec canUnify(TypeId subTy, TypeId superTy); diff --git a/Analysis/include/Luau/VisitTypeVar.h b/Analysis/include/Luau/VisitTypeVar.h index 7e5d71d6..9d7fa9fe 100644 --- a/Analysis/include/Luau/VisitTypeVar.h +++ b/Analysis/include/Luau/VisitTypeVar.h @@ -69,12 +69,14 @@ struct GenericTypeVarVisitor using Set = S; Set seen; + bool skipBoundTypes = false; int recursionCounter = 0; GenericTypeVarVisitor() = default; - explicit GenericTypeVarVisitor(Set seen) + explicit GenericTypeVarVisitor(Set seen, bool skipBoundTypes = false) : seen(std::move(seen)) + , skipBoundTypes(skipBoundTypes) { } @@ -199,7 +201,9 @@ struct GenericTypeVarVisitor if (auto btv = get(ty)) { - if (visit(ty, *btv)) + if (skipBoundTypes) + traverse(btv->boundTo); + else if (visit(ty, *btv)) traverse(btv->boundTo); } else if (auto ftv = get(ty)) @@ -229,7 +233,11 @@ struct GenericTypeVarVisitor else if (auto ttv = get(ty)) { // Some visitors want to see bound tables, that's why we traverse the original type - if (visit(ty, *ttv)) + if (skipBoundTypes && ttv->boundTo) + { + traverse(*ttv->boundTo); + } + else if (visit(ty, *ttv)) { if (ttv->boundTo) { @@ -394,13 +402,17 @@ struct GenericTypeVarVisitor */ struct TypeVarVisitor : GenericTypeVarVisitor> { + explicit TypeVarVisitor(bool skipBoundTypes = false) + : GenericTypeVarVisitor{{}, skipBoundTypes} + { + } }; /// Visit each type under a given type. Each type will only be checked once even if there are multiple paths to it. struct TypeVarOnceVisitor : GenericTypeVarVisitor> { - TypeVarOnceVisitor() - : GenericTypeVarVisitor{DenseHashSet{nullptr}} + explicit TypeVarOnceVisitor(bool skipBoundTypes = false) + : GenericTypeVarVisitor{DenseHashSet{nullptr}, skipBoundTypes} { } }; diff --git a/Analysis/src/Anyification.cpp b/Analysis/src/Anyification.cpp new file mode 100644 index 00000000..b6e58009 --- /dev/null +++ b/Analysis/src/Anyification.cpp @@ -0,0 +1,96 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details + +#include "Luau/Anyification.h" + +#include "Luau/Common.h" +#include "Luau/Normalize.h" +#include "Luau/TxnLog.h" + +LUAU_FASTFLAG(LuauClassTypeVarsInSubstitution) + +namespace Luau +{ + +Anyification::Anyification(TypeArena* arena, const ScopePtr& scope, InternalErrorReporter* iceHandler, TypeId anyType, TypePackId anyTypePack) + : Substitution(TxnLog::empty(), arena) + , scope(NotNull{scope.get()}) + , iceHandler(iceHandler) + , anyType(anyType) + , anyTypePack(anyTypePack) +{ +} + +bool Anyification::isDirty(TypeId ty) +{ + if (ty->persistent) + return false; + + if (const TableTypeVar* ttv = log->getMutable(ty)) + return (ttv->state == TableState::Free || ttv->state == TableState::Unsealed); + else if (log->getMutable(ty)) + return true; + else if (get(ty)) + return true; + else + return false; +} + +bool Anyification::isDirty(TypePackId tp) +{ + if (tp->persistent) + return false; + + if (log->getMutable(tp)) + return true; + else + return false; +} + +TypeId Anyification::clean(TypeId ty) +{ + LUAU_ASSERT(isDirty(ty)); + if (const TableTypeVar* ttv = log->getMutable(ty)) + { + TableTypeVar clone = TableTypeVar{ttv->props, ttv->indexer, ttv->level, TableState::Sealed}; + clone.definitionModuleName = ttv->definitionModuleName; + clone.name = ttv->name; + clone.syntheticName = ttv->syntheticName; + clone.tags = ttv->tags; + TypeId res = addType(std::move(clone)); + asMutable(res)->normal = ty->normal; + return res; + } + else if (auto ctv = get(ty)) + { + std::vector copy = ctv->parts; + for (TypeId& ty : copy) + ty = replace(ty); + TypeId res = copy.size() == 1 ? copy[0] : addType(UnionTypeVar{std::move(copy)}); + auto [t, ok] = normalize(res, scope, *arena, *iceHandler); + if (!ok) + normalizationTooComplex = true; + return t; + } + else + return anyType; +} + +TypePackId Anyification::clean(TypePackId tp) +{ + LUAU_ASSERT(isDirty(tp)); + return anyTypePack; +} + +bool Anyification::ignoreChildren(TypeId ty) +{ + if (FFlag::LuauClassTypeVarsInSubstitution && get(ty)) + return true; + + return ty->persistent; +} +bool Anyification::ignoreChildren(TypePackId ty) +{ + return ty->persistent; +} + +} diff --git a/Analysis/src/Autocomplete.cpp b/Analysis/src/Autocomplete.cpp index 8d5cc723..5c484899 100644 --- a/Analysis/src/Autocomplete.cpp +++ b/Analysis/src/Autocomplete.cpp @@ -14,6 +14,8 @@ LUAU_FASTFLAG(LuauSelfCallAutocompleteFix3) +LUAU_FASTFLAGVARIABLE(LuauAutocompleteFixGlobalOrder, false) + static const std::unordered_set kStatementStartingKeywords = { "while", "if", "local", "repeat", "function", "do", "for", "return", "break", "continue", "type", "export"}; @@ -135,11 +137,11 @@ static std::optional findExpectedTypeAt(const Module& module, AstNode* n return *it; } -static bool checkTypeMatch(TypeArena* typeArena, TypeId subTy, TypeId superTy) +static bool checkTypeMatch(TypeId subTy, TypeId superTy, NotNull scope, TypeArena* typeArena) { InternalErrorReporter iceReporter; UnifierSharedState unifierState(&iceReporter); - Unifier unifier(typeArena, Mode::Strict, Location(), Variance::Covariant, unifierState); + Unifier unifier(typeArena, Mode::Strict, scope, Location(), Variance::Covariant, unifierState); return unifier.canUnify(subTy, superTy).empty(); } @@ -148,12 +150,14 @@ static TypeCorrectKind checkTypeCorrectKind(const Module& module, TypeArena* typ { ty = follow(ty); - auto canUnify = [&typeArena](TypeId subTy, TypeId superTy) { + NotNull moduleScope{module.getModuleScope().get()}; + + auto canUnify = [&typeArena, moduleScope](TypeId subTy, TypeId superTy) { LUAU_ASSERT(!FFlag::LuauSelfCallAutocompleteFix3); InternalErrorReporter iceReporter; UnifierSharedState unifierState(&iceReporter); - Unifier unifier(typeArena, Mode::Strict, Location(), Variance::Covariant, unifierState); + Unifier unifier(typeArena, Mode::Strict, moduleScope, Location(), Variance::Covariant, unifierState); unifier.tryUnify(subTy, superTy); bool ok = unifier.errors.empty(); @@ -167,11 +171,11 @@ static TypeCorrectKind checkTypeCorrectKind(const Module& module, TypeArena* typ TypeId expectedType = follow(*typeAtPosition); - auto checkFunctionType = [typeArena, &canUnify, &expectedType](const FunctionTypeVar* ftv) { + auto checkFunctionType = [typeArena, moduleScope, &canUnify, &expectedType](const FunctionTypeVar* ftv) { if (FFlag::LuauSelfCallAutocompleteFix3) { if (std::optional firstRetTy = first(ftv->retTypes)) - return checkTypeMatch(typeArena, *firstRetTy, expectedType); + return checkTypeMatch(*firstRetTy, expectedType, moduleScope, typeArena); return false; } @@ -210,7 +214,7 @@ static TypeCorrectKind checkTypeCorrectKind(const Module& module, TypeArena* typ } if (FFlag::LuauSelfCallAutocompleteFix3) - return checkTypeMatch(typeArena, ty, expectedType) ? TypeCorrectKind::Correct : TypeCorrectKind::None; + return checkTypeMatch(ty, expectedType, NotNull{module.getModuleScope().get()}, typeArena) ? TypeCorrectKind::Correct : TypeCorrectKind::None; else return canUnify(ty, expectedType) ? TypeCorrectKind::Correct : TypeCorrectKind::None; } @@ -268,7 +272,7 @@ static void autocompleteProps(const Module& module, TypeArena* typeArena, TypeId return colonIndex; } }; - auto isWrongIndexer = [typeArena, rootTy, indexType](Luau::TypeId type) { + auto isWrongIndexer = [typeArena, &module, rootTy, indexType](Luau::TypeId type) { LUAU_ASSERT(FFlag::LuauSelfCallAutocompleteFix3); if (indexType == PropIndexType::Key) @@ -276,7 +280,7 @@ static void autocompleteProps(const Module& module, TypeArena* typeArena, TypeId bool calledWithSelf = indexType == PropIndexType::Colon; - auto isCompatibleCall = [typeArena, rootTy, calledWithSelf](const FunctionTypeVar* ftv) { + auto isCompatibleCall = [typeArena, &module, rootTy, calledWithSelf](const FunctionTypeVar* ftv) { // Strong match with definition is a success if (calledWithSelf == ftv->hasSelf) return true; @@ -289,7 +293,7 @@ static void autocompleteProps(const Module& module, TypeArena* typeArena, TypeId // When called with '.', but declared with 'self', it is considered invalid if first argument is compatible if (std::optional firstArgTy = first(ftv->argTypes)) { - if (checkTypeMatch(typeArena, rootTy, *firstArgTy)) + if (checkTypeMatch(rootTy, *firstArgTy, NotNull{module.getModuleScope().get()}, typeArena)) return calledWithSelf; } @@ -1073,10 +1077,21 @@ T* extractStat(const std::vector& ancestry) return nullptr; } -static bool isBindingLegalAtCurrentPosition(const Binding& binding, Position pos) +static bool isBindingLegalAtCurrentPosition(const Symbol& symbol, const Binding& binding, Position pos) { - // Default Location used for global bindings, which are always legal. - return binding.location == Location() || binding.location.end < pos; + if (FFlag::LuauAutocompleteFixGlobalOrder) + { + if (symbol.local) + return binding.location.end < pos; + + // Builtin globals have an empty location; for defined globals, we want pos to be outside of the definition range to suggest it + return binding.location == Location() || !binding.location.containsClosed(pos); + } + else + { + // Default Location used for global bindings, which are always legal. + return binding.location == Location() || binding.location.end < pos; + } } static AutocompleteEntryMap autocompleteStatement( @@ -1097,7 +1112,7 @@ static AutocompleteEntryMap autocompleteStatement( { for (const auto& [name, binding] : scope->bindings) { - if (!isBindingLegalAtCurrentPosition(binding, position)) + if (!isBindingLegalAtCurrentPosition(name, binding, position)) continue; std::string n = toString(name); @@ -1225,7 +1240,7 @@ static AutocompleteContext autocompleteExpression(const SourceModule& sourceModu { for (const auto& [name, binding] : scope->bindings) { - if (!isBindingLegalAtCurrentPosition(binding, position)) + if (!isBindingLegalAtCurrentPosition(name, binding, position)) continue; if (isBeingDefined(ancestry, name)) diff --git a/Analysis/src/Constraint.cpp b/Analysis/src/Constraint.cpp index 64e3a666..d272c027 100644 --- a/Analysis/src/Constraint.cpp +++ b/Analysis/src/Constraint.cpp @@ -5,8 +5,9 @@ namespace Luau { -Constraint::Constraint(ConstraintV&& c) +Constraint::Constraint(ConstraintV&& c, NotNull scope) : c(std::move(c)) + , scope(scope) { } diff --git a/Analysis/src/ConstraintGraphBuilder.cpp b/Analysis/src/ConstraintGraphBuilder.cpp index 2c36d422..c1e54df2 100644 --- a/Analysis/src/ConstraintGraphBuilder.cpp +++ b/Analysis/src/ConstraintGraphBuilder.cpp @@ -2,6 +2,7 @@ #include "Luau/ConstraintGraphBuilder.h" #include "Luau/Ast.h" +#include "Luau/Common.h" #include "Luau/Constraint.h" #include "Luau/RecursionCounter.h" #include "Luau/ToString.h" @@ -26,6 +27,7 @@ ConstraintGraphBuilder::ConstraintGraphBuilder( , globalScope(globalScope) { LUAU_ASSERT(arena); + LUAU_ASSERT(module); } TypeId ConstraintGraphBuilder::freshType(const ScopePtr& scope) @@ -39,20 +41,22 @@ TypePackId ConstraintGraphBuilder::freshTypePack(const ScopePtr& scope) return arena->addTypePack(TypePackVar{std::move(f)}); } -ScopePtr ConstraintGraphBuilder::childScope(Location location, const ScopePtr& parent) +ScopePtr ConstraintGraphBuilder::childScope(AstNode* node, const ScopePtr& parent) { auto scope = std::make_shared(parent); - scopes.emplace_back(location, scope); + scopes.emplace_back(node->location, scope); scope->returnType = parent->returnType; - parent->children.push_back(NotNull(scope.get())); + + parent->children.push_back(NotNull{scope.get()}); + module->astScopes[node] = scope.get(); return scope; } void ConstraintGraphBuilder::addConstraint(const ScopePtr& scope, ConstraintV cv) { - scope->constraints.emplace_back(new Constraint{std::move(cv)}); + scope->constraints.emplace_back(new Constraint{std::move(cv), NotNull{scope.get()}}); } void ConstraintGraphBuilder::addConstraint(const ScopePtr& scope, std::unique_ptr c) @@ -67,6 +71,7 @@ void ConstraintGraphBuilder::visit(AstStatBlock* block) ScopePtr scope = std::make_shared(globalScope); rootScope = scope.get(); scopes.emplace_back(block->location, scope); + module->astScopes[block] = NotNull{scope.get()}; rootScope->returnType = freshTypePack(scope); @@ -115,7 +120,7 @@ void ConstraintGraphBuilder::visitBlockWithoutChildScope(const ScopePtr& scope, ScopePtr defnScope = scope; if (hasGenerics) { - defnScope = childScope(alias->location, scope); + defnScope = childScope(alias, scope); } TypeId initialType = freshType(scope); @@ -155,6 +160,8 @@ void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStat* stat) visit(scope, s); else if (auto s = stat->as()) visit(scope, s); + else if (auto s = stat->as()) + visit(scope, s); else if (auto f = stat->as()) visit(scope, f); else if (auto f = stat->as()) @@ -163,6 +170,8 @@ void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStat* stat) visit(scope, r); else if (auto a = stat->as()) visit(scope, a); + else if (auto a = stat->as()) + visit(scope, a); else if (auto e = stat->as()) checkPack(scope, e->expr); else if (auto i = stat->as()) @@ -241,7 +250,7 @@ void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatFor* for_) checkNumber(for_->to); checkNumber(for_->step); - ScopePtr forScope = childScope(for_->location, scope); + ScopePtr forScope = childScope(for_, scope); forScope->bindings[for_->var] = Binding{singletonTypes.numberType, for_->var->location}; visit(forScope, for_->body); @@ -251,11 +260,22 @@ void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatWhile* while_) { check(scope, while_->condition); - ScopePtr whileScope = childScope(while_->location, scope); + ScopePtr whileScope = childScope(while_, scope); visit(whileScope, while_->body); } +void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatRepeat* repeat) +{ + ScopePtr repeatScope = childScope(repeat, scope); + + visit(repeatScope, repeat->body); + + // The condition does indeed have access to bindings from within the body of + // the loop. + check(repeatScope, repeat->condition); +} + void addConstraints(Constraint* constraint, NotNull scope) { scope->constraints.reserve(scope->constraints.size() + scope->constraints.size()); @@ -286,8 +306,8 @@ void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatLocalFunction* checkFunctionBody(sig.bodyScope, function->func); - std::unique_ptr c{ - new Constraint{GeneralizationConstraint{functionType, sig.signature, sig.signatureScope ? sig.signatureScope.get() : sig.bodyScope.get()}}}; + NotNull constraintScope{sig.signatureScope ? sig.signatureScope.get() : sig.bodyScope.get()}; + std::unique_ptr c = std::make_unique(GeneralizationConstraint{functionType, sig.signature}, constraintScope); addConstraints(c.get(), NotNull(sig.bodyScope.get())); addConstraint(scope, std::move(c)); @@ -356,8 +376,8 @@ void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatFunction* funct checkFunctionBody(sig.bodyScope, function->func); - std::unique_ptr c{ - new Constraint{GeneralizationConstraint{functionType, sig.signature, sig.signatureScope ? sig.signatureScope.get() : sig.bodyScope.get()}}}; + NotNull constraintScope{sig.signatureScope ? sig.signatureScope.get() : sig.bodyScope.get()}; + std::unique_ptr c = std::make_unique(GeneralizationConstraint{functionType, sig.signature}, constraintScope); addConstraints(c.get(), NotNull(sig.bodyScope.get())); addConstraint(scope, std::move(c)); @@ -371,7 +391,7 @@ void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatReturn* ret) void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatBlock* block) { - ScopePtr innerScope = childScope(block->location, scope); + ScopePtr innerScope = childScope(block, scope); visitBlockWithoutChildScope(innerScope, block); } @@ -384,16 +404,30 @@ void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatAssign* assign) addConstraint(scope, PackSubtypeConstraint{valuePack, varPackId}); } +void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatCompoundAssign* assign) +{ + // Synthesize A = A op B from A op= B and then build constraints for that instead. + + AstExprBinary exprBinary{assign->location, assign->op, assign->var, assign->value}; + AstExpr* exprBinaryPtr = &exprBinary; + + AstArray vars{&assign->var, 1}; + AstArray values{&exprBinaryPtr, 1}; + AstStatAssign syntheticAssign{assign->location, vars, values}; + + visit(scope, &syntheticAssign); +} + void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatIf* ifStatement) { check(scope, ifStatement->condition); - ScopePtr thenScope = childScope(ifStatement->thenbody->location, scope); + ScopePtr thenScope = childScope(ifStatement->thenbody, scope); visit(thenScope, ifStatement->thenbody); if (ifStatement->elsebody) { - ScopePtr elseScope = childScope(ifStatement->elsebody->location, scope); + ScopePtr elseScope = childScope(ifStatement->elsebody, scope); visit(elseScope, ifStatement->elsebody); } } @@ -561,7 +595,7 @@ void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatDeclareFunction ScopePtr funScope = scope; if (!generics.empty() || !genericPacks.empty()) - funScope = childScope(global->location, scope); + funScope = childScope(global, scope); TypePackId paramPack = resolveTypePack(funScope, global->params); TypePackId retPack = resolveTypePack(funScope, global->retTypes); @@ -739,6 +773,8 @@ TypeId ConstraintGraphBuilder::check(const ScopePtr& scope, AstExpr* expr) result = check(scope, unary); else if (auto binary = expr->as()) result = check(scope, binary); + else if (auto ifElse = expr->as()) + result = check(scope, ifElse); else if (auto typeAssert = expr->as()) result = check(scope, typeAssert); else if (auto err = expr->as()) @@ -819,6 +855,12 @@ TypeId ConstraintGraphBuilder::check(const ScopePtr& scope, AstExprBinary* binar addConstraint(scope, SubtypeConstraint{leftType, rightType}); return leftType; } + case AstExprBinary::Add: + { + TypeId resultType = arena->addType(BlockedTypeVar{}); + addConstraint(scope, BinaryConstraint{AstExprBinary::Add, leftType, rightType, resultType}); + return resultType; + } case AstExprBinary::Sub: { TypeId resultType = arena->addType(BlockedTypeVar{}); @@ -833,6 +875,24 @@ TypeId ConstraintGraphBuilder::check(const ScopePtr& scope, AstExprBinary* binar return nullptr; } +TypeId ConstraintGraphBuilder::check(const ScopePtr& scope, AstExprIfElse* ifElse) +{ + check(scope, ifElse->condition); + + TypeId thenType = check(scope, ifElse->trueExpr); + TypeId elseType = check(scope, ifElse->falseExpr); + + if (ifElse->hasElse) + { + TypeId resultType = arena->addType(BlockedTypeVar{}); + addConstraint(scope, SubtypeConstraint{thenType, resultType}); + addConstraint(scope, SubtypeConstraint{elseType, resultType}); + return resultType; + } + + return thenType; +} + TypeId ConstraintGraphBuilder::check(const ScopePtr& scope, AstExprTypeAssertion* typeAssert) { check(scope, typeAssert->expr); @@ -905,14 +965,14 @@ ConstraintGraphBuilder::FunctionSignature ConstraintGraphBuilder::checkFunctionS // generics properly. if (hasGenerics) { - signatureScope = childScope(fn->location, parent); + signatureScope = childScope(fn, parent); // We need to assign returnType before creating bodyScope so that the // return type gets propogated to bodyScope. returnType = freshTypePack(signatureScope); signatureScope->returnType = returnType; - bodyScope = childScope(fn->body->location, signatureScope); + bodyScope = childScope(fn->body, signatureScope); std::vector> genericDefinitions = createGenerics(signatureScope, fn->generics); std::vector> genericPackDefinitions = createGenericPacks(signatureScope, fn->genericPacks); @@ -933,7 +993,7 @@ ConstraintGraphBuilder::FunctionSignature ConstraintGraphBuilder::checkFunctionS } else { - bodyScope = childScope(fn->body->location, parent); + bodyScope = childScope(fn->body, parent); returnType = freshTypePack(bodyScope); bodyScope->returnType = returnType; @@ -1098,7 +1158,7 @@ TypeId ConstraintGraphBuilder::resolveType(const ScopePtr& scope, AstType* ty, b // for the generic bindings to live on. if (hasGenerics) { - signatureScope = childScope(fn->location, scope); + signatureScope = childScope(fn, scope); std::vector> genericDefinitions = createGenerics(signatureScope, fn->generics); std::vector> genericPackDefinitions = createGenericPacks(signatureScope, fn->genericPacks); diff --git a/Analysis/src/ConstraintSolver.cpp b/Analysis/src/ConstraintSolver.cpp index d8105ce3..6c6d2722 100644 --- a/Analysis/src/ConstraintSolver.cpp +++ b/Analysis/src/ConstraintSolver.cpp @@ -373,7 +373,7 @@ bool ConstraintSolver::tryDispatch(const SubtypeConstraint& c, NotNullscope); unblock(c.subType); unblock(c.superType); @@ -383,7 +383,7 @@ bool ConstraintSolver::tryDispatch(const SubtypeConstraint& c, NotNull constraint, bool force) { - unify(c.subPack, c.superPack); + unify(c.subPack, c.superPack, constraint->scope); unblock(c.subPack); unblock(c.superPack); @@ -398,9 +398,9 @@ bool ConstraintSolver::tryDispatch(const GeneralizationConstraint& c, NotNullty.emplace(c.sourceType); else - unify(c.generalizedType, c.sourceType); + unify(c.generalizedType, c.sourceType, constraint->scope); - TypeId generalized = quantify(arena, c.sourceType, c.scope); + TypeId generalized = quantify(arena, c.sourceType, constraint->scope); *asMutable(c.sourceType) = *generalized; unblock(c.generalizedType); @@ -422,7 +422,7 @@ bool ConstraintSolver::tryDispatch(const InstantiationConstraint& c, NotNullty.emplace(*instantiated); else - unify(c.subType, *instantiated); + unify(c.subType, *instantiated, constraint->scope); unblock(c.subType); @@ -465,7 +465,7 @@ bool ConstraintSolver::tryDispatch(const BinaryConstraint& c, NotNullscope); asMutable(c.resultType)->ty.emplace(leftType); return true; } @@ -528,16 +528,18 @@ struct InstantiationQueuer : TypeVarOnceVisitor { ConstraintSolver* solver; const InstantiationSignature& signature; + NotNull scope; - explicit InstantiationQueuer(ConstraintSolver* solver, const InstantiationSignature& signature) + explicit InstantiationQueuer(ConstraintSolver* solver, const InstantiationSignature& signature, NotNull scope) : solver(solver) , signature(signature) + , scope(scope) { } bool visit(TypeId ty, const PendingExpansionTypeVar& petv) override { - solver->pushConstraint(TypeAliasExpansionConstraint{ty}); + solver->pushConstraint(TypeAliasExpansionConstraint{ty}, scope); return false; } }; @@ -686,7 +688,7 @@ bool ConstraintSolver::tryDispatch(const TypeAliasExpansionConstraint& c, NotNul // The application is not recursive, so we need to queue up application of // any child type function instantiations within the result in order for it // to be complete. - InstantiationQueuer queuer{this, signature}; + InstantiationQueuer queuer{this, signature, constraint->scope}; queuer.traverse(target); instantiatedAliases[signature] = target; @@ -766,30 +768,40 @@ bool ConstraintSolver::isBlocked(NotNull constraint) return blockedIt != blockedConstraints.end() && blockedIt->second > 0; } -void ConstraintSolver::unify(TypeId subType, TypeId superType) +void ConstraintSolver::unify(TypeId subType, TypeId superType, NotNull scope) { UnifierSharedState sharedState{&iceReporter}; - Unifier u{arena, Mode::Strict, Location{}, Covariant, sharedState}; + Unifier u{arena, Mode::Strict, scope, Location{}, Covariant, sharedState}; u.tryUnify(subType, superType); u.log.commit(); } -void ConstraintSolver::unify(TypePackId subPack, TypePackId superPack) +void ConstraintSolver::unify(TypePackId subPack, TypePackId superPack, NotNull scope) { UnifierSharedState sharedState{&iceReporter}; - Unifier u{arena, Mode::Strict, Location{}, Covariant, sharedState}; + Unifier u{arena, Mode::Strict, scope, Location{}, Covariant, sharedState}; u.tryUnify(subPack, superPack); u.log.commit(); } -void ConstraintSolver::pushConstraint(ConstraintV cv) +void ConstraintSolver::pushConstraint(ConstraintV cv, NotNull scope) { - std::unique_ptr c = std::make_unique(std::move(cv)); + std::unique_ptr c = std::make_unique(std::move(cv), scope); NotNull borrow = NotNull(c.get()); solverConstraints.push_back(std::move(c)); unsolvedConstraints.push_back(borrow); } +void ConstraintSolver::reportError(TypeErrorData&& data, const Location& location) +{ + errors.emplace_back(location, std::move(data)); +} + +void ConstraintSolver::reportError(TypeError e) +{ + errors.emplace_back(std::move(e)); +} + } // namespace Luau diff --git a/Analysis/src/Frontend.cpp b/Analysis/src/Frontend.cpp index c450297f..8ab4e865 100644 --- a/Analysis/src/Frontend.cpp +++ b/Analysis/src/Frontend.cpp @@ -839,6 +839,9 @@ ModulePtr Frontend::check(const SourceModule& sourceModule, Mode mode, const Sco ConstraintSolver cs{&result->internalTypes, NotNull(cgb.rootScope)}; cs.run(); + for (TypeError& e : cs.errors) + result->errors.emplace_back(std::move(e)); + result->scopes = std::move(cgb.scopes); result->astTypes = std::move(cgb.astTypes); result->astTypePacks = std::move(cgb.astTypePacks); diff --git a/Analysis/src/Module.cpp b/Analysis/src/Module.cpp index 4e6b2585..de796c7a 100644 --- a/Analysis/src/Module.cpp +++ b/Analysis/src/Module.cpp @@ -244,12 +244,12 @@ void Module::clonePublicInterface(InternalErrorReporter& ice) if (FFlag::LuauLowerBoundsCalculation) { - normalize(returnType, interfaceTypes, ice); + normalize(returnType, NotNull{this}, ice); if (FFlag::LuauForceExportSurfacesToBeNormal) forceNormal.traverse(returnType); if (varargPack) { - normalize(*varargPack, interfaceTypes, ice); + normalize(*varargPack, NotNull{this}, ice); if (FFlag::LuauForceExportSurfacesToBeNormal) forceNormal.traverse(*varargPack); } @@ -265,7 +265,7 @@ void Module::clonePublicInterface(InternalErrorReporter& ice) tf = clone(tf, interfaceTypes, cloneState); if (FFlag::LuauLowerBoundsCalculation) { - normalize(tf.type, interfaceTypes, ice); + normalize(tf.type, NotNull{this}, ice); // We're about to freeze the memory. We know that the flag is conservative by design. Cyclic tables // won't be marked normal. If the types aren't normal by now, they never will be. @@ -276,7 +276,7 @@ void Module::clonePublicInterface(InternalErrorReporter& ice) if (param.defaultValue) { - normalize(*param.defaultValue, interfaceTypes, ice); + normalize(*param.defaultValue, NotNull{this}, ice); forceNormal.traverse(*param.defaultValue); } } @@ -302,7 +302,7 @@ void Module::clonePublicInterface(InternalErrorReporter& ice) ty = clone(ty, interfaceTypes, cloneState); if (FFlag::LuauLowerBoundsCalculation) { - normalize(ty, interfaceTypes, ice); + normalize(ty, NotNull{this}, ice); if (FFlag::LuauForceExportSurfacesToBeNormal) forceNormal.traverse(ty); diff --git a/Analysis/src/Normalize.cpp b/Analysis/src/Normalize.cpp index 33f369a7..94adaf5c 100644 --- a/Analysis/src/Normalize.cpp +++ b/Analysis/src/Normalize.cpp @@ -15,7 +15,6 @@ LUAU_FASTINTVARIABLE(LuauNormalizeIterationLimit, 1200); LUAU_FASTFLAGVARIABLE(LuauNormalizeCombineTableFix, false); LUAU_FASTFLAGVARIABLE(LuauFixNormalizationOfCyclicUnions, false); LUAU_FASTFLAG(LuauUnknownAndNeverType) -LUAU_FASTFLAG(LuauQuantifyConstrained) namespace Luau { @@ -55,11 +54,11 @@ struct Replacer } // anonymous namespace -bool isSubtype(TypeId subTy, TypeId superTy, InternalErrorReporter& ice) +bool isSubtype(TypeId subTy, TypeId superTy, NotNull scope, InternalErrorReporter& ice) { UnifierSharedState sharedState{&ice}; TypeArena arena; - Unifier u{&arena, Mode::Strict, Location{}, Covariant, sharedState}; + Unifier u{&arena, Mode::Strict, scope, Location{}, Covariant, sharedState}; u.anyIsTop = true; u.tryUnify(subTy, superTy); @@ -67,11 +66,11 @@ bool isSubtype(TypeId subTy, TypeId superTy, InternalErrorReporter& ice) return ok; } -bool isSubtype(TypePackId subPack, TypePackId superPack, InternalErrorReporter& ice) +bool isSubtype(TypePackId subPack, TypePackId superPack, NotNull scope, InternalErrorReporter& ice) { UnifierSharedState sharedState{&ice}; TypeArena arena; - Unifier u{&arena, Mode::Strict, Location{}, Covariant, sharedState}; + Unifier u{&arena, Mode::Strict, scope, Location{}, Covariant, sharedState}; u.anyIsTop = true; u.tryUnify(subPack, superPack); @@ -134,13 +133,15 @@ struct Normalize final : TypeVarVisitor { using TypeVarVisitor::Set; - Normalize(TypeArena& arena, InternalErrorReporter& ice) + Normalize(TypeArena& arena, NotNull scope, InternalErrorReporter& ice) : arena(arena) + , scope(scope) , ice(ice) { } TypeArena& arena; + NotNull scope; InternalErrorReporter& ice; int iterationLimit = 0; @@ -215,22 +216,7 @@ struct Normalize final : TypeVarVisitor traverse(part); std::vector newParts = normalizeUnion(parts); - - if (FFlag::LuauQuantifyConstrained) - { - ctv->parts = std::move(newParts); - } - else - { - const bool normal = areNormal(newParts, seen, ice); - - if (newParts.size() == 1) - *asMutable(ty) = BoundTypeVar{newParts[0]}; - else - *asMutable(ty) = UnionTypeVar{std::move(newParts)}; - - asMutable(ty)->normal = normal; - } + ctv->parts = std::move(newParts); return false; } @@ -288,12 +274,7 @@ struct Normalize final : TypeVarVisitor } // An unsealed table can never be normal, ditto for free tables iff the type it is bound to is also not normal. - if (FFlag::LuauQuantifyConstrained) - { - if (ttv.state == TableState::Generic || ttv.state == TableState::Sealed || (ttv.state == TableState::Free && follow(ty)->normal)) - asMutable(ty)->normal = normal; - } - else + if (ttv.state == TableState::Generic || ttv.state == TableState::Sealed || (ttv.state == TableState::Free && follow(ty)->normal)) asMutable(ty)->normal = normal; return false; @@ -518,9 +499,9 @@ struct Normalize final : TypeVarVisitor for (TypeId& part : result) { - if (isSubtype(ty, part, ice)) + if (isSubtype(ty, part, scope, ice)) return; // no need to do anything - else if (isSubtype(part, ty, ice)) + else if (isSubtype(part, ty, scope, ice)) { part = ty; // replace the less general type by the more general one return; @@ -572,12 +553,12 @@ struct Normalize final : TypeVarVisitor bool merged = false; for (TypeId& part : result->parts) { - if (isSubtype(part, ty, ice)) + if (isSubtype(part, ty, scope, ice)) { merged = true; break; // no need to do anything } - else if (isSubtype(ty, part, ice)) + else if (isSubtype(ty, part, scope, ice)) { merged = true; part = ty; // replace the less general type by the more general one @@ -710,13 +691,13 @@ struct Normalize final : TypeVarVisitor /** * @returns A tuple of TypeId and a success indicator. (true indicates that the normalization completed successfully) */ -std::pair normalize(TypeId ty, TypeArena& arena, InternalErrorReporter& ice) +std::pair normalize(TypeId ty, NotNull scope, TypeArena& arena, InternalErrorReporter& ice) { CloneState state; if (FFlag::DebugLuauCopyBeforeNormalizing) (void)clone(ty, arena, state); - Normalize n{arena, ice}; + Normalize n{arena, scope, ice}; n.traverse(ty); return {ty, !n.limitExceeded}; @@ -726,29 +707,39 @@ std::pair normalize(TypeId ty, TypeArena& arena, InternalErrorRepo // reclaim memory used by wantonly allocated intermediate types here. // The main wrinkle here is that we don't want clone() to copy a type if the source and dest // arena are the same. +std::pair normalize(TypeId ty, NotNull module, InternalErrorReporter& ice) +{ + return normalize(ty, NotNull{module->getModuleScope().get()}, module->internalTypes, ice); +} + std::pair normalize(TypeId ty, const ModulePtr& module, InternalErrorReporter& ice) { - return normalize(ty, module->internalTypes, ice); + return normalize(ty, NotNull{module.get()}, ice); } /** * @returns A tuple of TypeId and a success indicator. (true indicates that the normalization completed successfully) */ -std::pair normalize(TypePackId tp, TypeArena& arena, InternalErrorReporter& ice) +std::pair normalize(TypePackId tp, NotNull scope, TypeArena& arena, InternalErrorReporter& ice) { CloneState state; if (FFlag::DebugLuauCopyBeforeNormalizing) (void)clone(tp, arena, state); - Normalize n{arena, ice}; + Normalize n{arena, scope, ice}; n.traverse(tp); return {tp, !n.limitExceeded}; } +std::pair normalize(TypePackId tp, NotNull module, InternalErrorReporter& ice) +{ + return normalize(tp, NotNull{module->getModuleScope().get()}, module->internalTypes, ice); +} + std::pair normalize(TypePackId tp, const ModulePtr& module, InternalErrorReporter& ice) { - return normalize(tp, module->internalTypes, ice); + return normalize(tp, NotNull{module.get()}, ice); } } // namespace Luau diff --git a/Analysis/src/Quantify.cpp b/Analysis/src/Quantify.cpp index ce4afe22..7e6ff2f9 100644 --- a/Analysis/src/Quantify.cpp +++ b/Analysis/src/Quantify.cpp @@ -10,7 +10,6 @@ LUAU_FASTFLAG(DebugLuauSharedSelf) LUAU_FASTFLAG(DebugLuauDeferredConstraintResolution); -LUAU_FASTFLAGVARIABLE(LuauQuantifyConstrained, false) LUAU_FASTFLAG(LuauClassTypeVarsInSubstitution) namespace Luau @@ -82,30 +81,25 @@ struct Quantifier final : TypeVarOnceVisitor bool visit(TypeId ty, const ConstrainedTypeVar&) override { - if (FFlag::LuauQuantifyConstrained) - { - ConstrainedTypeVar* ctv = getMutable(ty); + ConstrainedTypeVar* ctv = getMutable(ty); - seenMutableType = true; - - if (FFlag::DebugLuauDeferredConstraintResolution ? !subsumes(scope, ctv->scope) : !level.subsumes(ctv->level)) - return false; - - std::vector opts = std::move(ctv->parts); - - // We might transmute, so it's not safe to rely on the builtin traversal logic - for (TypeId opt : opts) - traverse(opt); - - if (opts.size() == 1) - *asMutable(ty) = BoundTypeVar{opts[0]}; - else - *asMutable(ty) = UnionTypeVar{std::move(opts)}; + seenMutableType = true; + if (FFlag::DebugLuauDeferredConstraintResolution ? !subsumes(scope, ctv->scope) : !level.subsumes(ctv->level)) return false; - } + + std::vector opts = std::move(ctv->parts); + + // We might transmute, so it's not safe to rely on the builtin traversal logic + for (TypeId opt : opts) + traverse(opt); + + if (opts.size() == 1) + *asMutable(ty) = BoundTypeVar{opts[0]}; else - return true; + *asMutable(ty) = UnionTypeVar{std::move(opts)}; + + return false; } bool visit(TypeId ty, const TableTypeVar&) override @@ -119,12 +113,6 @@ struct Quantifier final : TypeVarOnceVisitor if (ttv.state == TableState::Free) seenMutableType = true; - if (!FFlag::LuauQuantifyConstrained) - { - if (ttv.state == TableState::Sealed || ttv.state == TableState::Generic) - return false; - } - if (FFlag::DebugLuauDeferredConstraintResolution ? !subsumes(scope, ttv.scope) : !level.subsumes(ttv.level)) { if (ttv.state == TableState::Unsealed) diff --git a/Analysis/src/TypeChecker2.cpp b/Analysis/src/TypeChecker2.cpp index 9d97e8d9..e5813cd2 100644 --- a/Analysis/src/TypeChecker2.cpp +++ b/Analysis/src/TypeChecker2.cpp @@ -8,22 +8,59 @@ #include "Luau/Clone.h" #include "Luau/Instantiation.h" #include "Luau/Normalize.h" +#include "Luau/ToString.h" #include "Luau/TxnLog.h" #include "Luau/TypeUtils.h" #include "Luau/TypeVar.h" #include "Luau/Unifier.h" -#include "Luau/ToString.h" namespace Luau { -struct TypeChecker2 : public AstVisitor +/* Push a scope onto the end of a stack for the lifetime of the StackPusher instance. + * TypeChecker2 uses this to maintain knowledge about which scope encloses every + * given AstNode. + */ +struct StackPusher +{ + std::vector>* stack; + NotNull scope; + + explicit StackPusher(std::vector>& stack, Scope* scope) + : stack(&stack) + , scope(scope) + { + stack.push_back(NotNull{scope}); + } + + ~StackPusher() + { + if (stack) + { + LUAU_ASSERT(stack->back() == scope); + stack->pop_back(); + } + } + + StackPusher(const StackPusher&) = delete; + StackPusher&& operator=(const StackPusher&) = delete; + + StackPusher(StackPusher&& other) + : stack(std::exchange(other.stack, nullptr)) + , scope(other.scope) + { + } +}; + +struct TypeChecker2 { const SourceModule* sourceModule; Module* module; InternalErrorReporter ice; // FIXME accept a pointer from Frontend SingletonTypes& singletonTypes; + std::vector> stack; + TypeChecker2(const SourceModule* sourceModule, Module* module) : sourceModule(sourceModule) , module(module) @@ -31,7 +68,13 @@ struct TypeChecker2 : public AstVisitor { } - using AstVisitor::visit; + std::optional pushStack(AstNode* node) + { + if (Scope** scope = module->astScopes.find(node)) + return StackPusher{stack, *scope}; + else + return std::nullopt; + } TypePackId lookupPack(AstExpr* expr) { @@ -118,11 +161,128 @@ struct TypeChecker2 : public AstVisitor return bestScope; } - bool visit(AstStatLocal* local) override + void visit(AstStat* stat) + { + auto pusher = pushStack(stat); + + if (0) + {} + else if (auto s = stat->as()) + return visit(s); + else if (auto s = stat->as()) + return visit(s); + else if (auto s = stat->as()) + return visit(s); + else if (auto s = stat->as()) + return visit(s); + else if (auto s = stat->as()) + return visit(s); + else if (auto s = stat->as()) + return visit(s); + else if (auto s = stat->as()) + return visit(s); + else if (auto s = stat->as()) + return visit(s); + else if (auto s = stat->as()) + return visit(s); + else if (auto s = stat->as()) + return visit(s); + else if (auto s = stat->as()) + return visit(s); + else if (auto s = stat->as()) + return visit(s); + else if (auto s = stat->as()) + return visit(s); + else if (auto s = stat->as()) + return visit(s); + else if (auto s = stat->as()) + return visit(s); + else if (auto s = stat->as()) + return visit(s); + else if (auto s = stat->as()) + return visit(s); + else if (auto s = stat->as()) + return visit(s); + else if (auto s = stat->as()) + return visit(s); + else if (auto s = stat->as()) + return visit(s); + else + LUAU_ASSERT(!"TypeChecker2 encountered an unknown node type"); + } + + void visit(AstStatBlock* block) + { + auto StackPusher = pushStack(block); + + for (AstStat* statement : block->body) + visit(statement); + } + + void visit(AstStatIf* ifStatement) + { + visit(ifStatement->condition); + visit(ifStatement->thenbody); + if (ifStatement->elsebody) + visit(ifStatement->elsebody); + } + + void visit(AstStatWhile* whileStatement) + { + visit(whileStatement->condition); + visit(whileStatement->body); + } + + void visit(AstStatRepeat* repeatStatement) + { + visit(repeatStatement->body); + visit(repeatStatement->condition); + } + + void visit(AstStatBreak*) + {} + + void visit(AstStatContinue*) + {} + + void visit(AstStatReturn* ret) + { + Scope* scope = findInnermostScope(ret->location); + TypePackId expectedRetType = scope->returnType; + + TypeArena arena; + TypePackId actualRetType = reconstructPack(ret->list, arena); + + UnifierSharedState sharedState{&ice}; + Unifier u{&arena, Mode::Strict, stack.back(), ret->location, Covariant, sharedState}; + u.anyIsTop = true; + + u.tryUnify(actualRetType, expectedRetType); + const bool ok = u.errors.empty() && u.log.empty(); + + if (!ok) + { + for (const TypeError& e : u.errors) + reportError(e); + } + + for (AstExpr* expr : ret->list) + visit(expr); + } + + void visit(AstStatExpr* expr) + { + visit(expr->expr); + } + + void visit(AstStatLocal* local) { for (size_t i = 0; i < local->values.size; ++i) { AstExpr* value = local->values.data[i]; + + visit(value); + if (i == local->values.size - 1) { if (i < local->values.size) @@ -140,7 +300,7 @@ struct TypeChecker2 : public AstVisitor if (var->annotation) { TypeId varType = lookupAnnotation(var->annotation); - if (!isSubtype(*it, varType, ice)) + if (!isSubtype(*it, varType, stack.back(), ice)) { reportError(TypeMismatch{varType, *it}, value->location); } @@ -158,64 +318,244 @@ struct TypeChecker2 : public AstVisitor if (var->annotation) { TypeId varType = lookupAnnotation(var->annotation); - if (!isSubtype(varType, valueType, ice)) + if (!isSubtype(varType, valueType, stack.back(), ice)) { reportError(TypeMismatch{varType, valueType}, value->location); } } } } - - return true; } - bool visit(AstStatAssign* assign) override + void visit(AstStatFor* forStatement) + { + if (forStatement->var->annotation) + visit(forStatement->var->annotation); + + visit(forStatement->from); + visit(forStatement->to); + if (forStatement->step) + visit(forStatement->step); + visit(forStatement->body); + } + + void visit(AstStatForIn* forInStatement) + { + for (AstLocal* local : forInStatement->vars) + { + if (local->annotation) + visit(local->annotation); + } + + for (AstExpr* expr : forInStatement->values) + visit(expr); + + visit(forInStatement->body); + } + + void visit(AstStatAssign* assign) { size_t count = std::min(assign->vars.size, assign->values.size); for (size_t i = 0; i < count; ++i) { AstExpr* lhs = assign->vars.data[i]; + visit(lhs); TypeId lhsType = lookupType(lhs); AstExpr* rhs = assign->values.data[i]; + visit(rhs); TypeId rhsType = lookupType(rhs); - if (!isSubtype(rhsType, lhsType, ice)) + if (!isSubtype(rhsType, lhsType, stack.back(), ice)) { reportError(TypeMismatch{lhsType, rhsType}, rhs->location); } } - - return true; } - bool visit(AstStatReturn* ret) override + void visit(AstStatCompoundAssign* stat) { - Scope* scope = findInnermostScope(ret->location); - TypePackId expectedRetType = scope->returnType; + visit(stat->var); + visit(stat->value); + } - TypeArena arena; - TypePackId actualRetType = reconstructPack(ret->list, arena); + void visit(AstStatFunction* stat) + { + visit(stat->name); + visit(stat->func); + } - UnifierSharedState sharedState{&ice}; - Unifier u{&arena, Mode::Strict, ret->location, Covariant, sharedState}; - u.anyIsTop = true; + void visit(AstStatLocalFunction* stat) + { + visit(stat->func); + } - u.tryUnify(actualRetType, expectedRetType); - const bool ok = u.errors.empty() && u.log.empty(); + void visit(const AstTypeList* typeList) + { + for (AstType* ty : typeList->types) + visit(ty); - if (!ok) + if (typeList->tailType) + visit(typeList->tailType); + } + + void visit(AstStatTypeAlias* stat) + { + for (const AstGenericType& el : stat->generics) { - for (const TypeError& e : u.errors) - reportError(e); + if (el.defaultValue) + visit(el.defaultValue); } - return true; + for (const AstGenericTypePack& el : stat->genericPacks) + { + if (el.defaultValue) + visit(el.defaultValue); + } + + visit(stat->type); } - bool visit(AstExprCall* call) override + void visit(AstTypeList types) { + for (AstType* type : types.types) + visit(type); + if (types.tailType) + visit(types.tailType); + } + + void visit(AstStatDeclareFunction* stat) + { + visit(stat->params); + visit(stat->retTypes); + } + + void visit(AstStatDeclareGlobal* stat) + { + visit(stat->type); + } + + void visit(AstStatDeclareClass* stat) + { + for (const AstDeclaredClassProp& prop : stat->props) + visit(prop.ty); + } + + void visit(AstStatError* stat) + { + for (AstExpr* expr : stat->expressions) + visit(expr); + + for (AstStat* s : stat->statements) + visit(s); + } + + void visit(AstExpr* expr) + { + auto StackPusher = pushStack(expr); + + if (0) + {} + else if (auto e = expr->as()) + return visit(e); + else if (auto e = expr->as()) + return visit(e); + else if (auto e = expr->as()) + return visit(e); + else if (auto e = expr->as()) + return visit(e); + else if (auto e = expr->as()) + return visit(e); + else if (auto e = expr->as()) + return visit(e); + else if (auto e = expr->as()) + return visit(e); + else if (auto e = expr->as()) + return visit(e); + else if (auto e = expr->as()) + return visit(e); + else if (auto e = expr->as()) + return visit(e); + else if (auto e = expr->as()) + return visit(e); + else if (auto e = expr->as()) + return visit(e); + else if (auto e = expr->as()) + return visit(e); + else if (auto e = expr->as()) + return visit(e); + else if (auto e = expr->as()) + return visit(e); + else if (auto e = expr->as()) + return visit(e); + else if (auto e = expr->as()) + return visit(e); + else if (auto e = expr->as()) + return visit(e); + else + LUAU_ASSERT(!"TypeChecker2 encountered an unknown expression type"); + } + + void visit(AstExprGroup* expr) + { + visit(expr->expr); + } + + void visit(AstExprConstantNil* expr) + { + // TODO! + } + + void visit(AstExprConstantBool* expr) + { + // TODO! + } + + void visit(AstExprConstantNumber* number) + { + TypeId actualType = lookupType(number); + TypeId numberType = getSingletonTypes().numberType; + + if (!isSubtype(numberType, actualType, stack.back(), ice)) + { + reportError(TypeMismatch{actualType, numberType}, number->location); + } + } + + void visit(AstExprConstantString* string) + { + TypeId actualType = lookupType(string); + TypeId stringType = getSingletonTypes().stringType; + + if (!isSubtype(stringType, actualType, stack.back(), ice)) + { + reportError(TypeMismatch{actualType, stringType}, string->location); + } + } + + void visit(AstExprLocal* expr) + { + // TODO! + } + + void visit(AstExprGlobal* expr) + { + // TODO! + } + + void visit(AstExprVarargs* expr) + { + // TODO! + } + + void visit(AstExprCall* call) + { + visit(call->func); + + for (AstExpr* arg : call->args) + visit(arg); + TypeArena arena; Instantiation instantiation{TxnLog::empty(), &arena, TypeLevel{}}; @@ -225,7 +565,7 @@ struct TypeChecker2 : public AstVisitor LUAU_ASSERT(functionType); TypePack args; - for (const auto& arg : call->args) + for (AstExpr* arg : call->args) { TypeId argTy = module->astTypes[arg]; LUAU_ASSERT(argTy); @@ -235,7 +575,7 @@ struct TypeChecker2 : public AstVisitor TypePackId argsTp = arena.addTypePack(args); FunctionTypeVar ftv{argsTp, expectedRetType}; TypeId expectedType = arena.addType(ftv); - if (!isSubtype(expectedType, instantiatedFunctionType, ice)) + if (!isSubtype(expectedType, instantiatedFunctionType, stack.back(), ice)) { unfreeze(module->interfaceTypes); CloneState cloneState; @@ -243,12 +583,36 @@ struct TypeChecker2 : public AstVisitor freeze(module->interfaceTypes); reportError(TypeMismatch{expectedType, functionType}, call->location); } - - return true; } - bool visit(AstExprFunction* fn) override + void visit(AstExprIndexName* indexName) { + TypeId leftType = lookupType(indexName->expr); + TypeId resultType = lookupType(indexName); + + // leftType must have a property called indexName->index + + std::optional ty = getIndexTypeFromType(module->getModuleScope(), leftType, indexName->index.value, indexName->location, /* addErrors */ true); + if (ty) + { + if (!isSubtype(resultType, *ty, stack.back(), ice)) + { + reportError(TypeMismatch{resultType, *ty}, indexName->location); + } + } + } + + void visit(AstExprIndexExpr* indexExpr) + { + // TODO! + visit(indexExpr->expr); + visit(indexExpr->index); + } + + void visit(AstExprFunction* fn) + { + auto StackPusher = pushStack(fn); + TypeId inferredFnTy = lookupType(fn); const FunctionTypeVar* inferredFtv = get(inferredFnTy); LUAU_ASSERT(inferredFtv); @@ -264,7 +628,7 @@ struct TypeChecker2 : public AstVisitor TypeId inferredArgTy = *argIt; TypeId annotatedArgTy = lookupAnnotation(arg->annotation); - if (!isSubtype(annotatedArgTy, inferredArgTy, ice)) + if (!isSubtype(annotatedArgTy, inferredArgTy, stack.back(), ice)) { reportError(TypeMismatch{annotatedArgTy, inferredArgTy}, arg->location); } @@ -273,68 +637,64 @@ struct TypeChecker2 : public AstVisitor ++argIt; } - return true; + visit(fn->body); } - bool visit(AstExprIndexName* indexName) override + void visit(AstExprTable* expr) { - TypeId leftType = lookupType(indexName->expr); - TypeId resultType = lookupType(indexName); - - // leftType must have a property called indexName->index - - std::optional ty = getIndexTypeFromType(module->getModuleScope(), leftType, indexName->index.value, indexName->location, /* addErrors */ true); - if (ty) + // TODO! + for (const AstExprTable::Item& item : expr->items) { - if (!isSubtype(resultType, *ty, ice)) - { - reportError(TypeMismatch{resultType, *ty}, indexName->location); - } + if (item.key) + visit(item.key); + visit(item.value); } - - return true; } - bool visit(AstExprConstantNumber* number) override + void visit(AstExprUnary* expr) { - TypeId actualType = lookupType(number); - TypeId numberType = getSingletonTypes().numberType; - - if (!isSubtype(numberType, actualType, ice)) - { - reportError(TypeMismatch{actualType, numberType}, number->location); - } - - return true; + // TODO! + visit(expr->expr); } - bool visit(AstExprConstantString* string) override + void visit(AstExprBinary* expr) { - TypeId actualType = lookupType(string); - TypeId stringType = getSingletonTypes().stringType; - - if (!isSubtype(stringType, actualType, ice)) - { - reportError(TypeMismatch{actualType, stringType}, string->location); - } - - return true; + // TODO! + visit(expr->left); + visit(expr->right); } - bool visit(AstExprTypeAssertion* expr) override + void visit(AstExprTypeAssertion* expr) { + visit(expr->expr); + visit(expr->annotation); + TypeId annotationType = lookupAnnotation(expr->annotation); TypeId computedType = lookupType(expr->expr); // Note: As an optimization, we try 'number <: number | string' first, as that is the more likely case. - if (isSubtype(annotationType, computedType, ice)) - return true; + if (isSubtype(annotationType, computedType, stack.back(), ice)) + return; - if (isSubtype(computedType, annotationType, ice)) - return true; + if (isSubtype(computedType, annotationType, stack.back(), ice)) + return; reportError(TypesAreUnrelated{computedType, annotationType}, expr->location); - return true; + } + + void visit(AstExprIfElse* expr) + { + // TODO! + visit(expr->condition); + visit(expr->trueExpr); + visit(expr->falseExpr); + } + + void visit(AstExprError* expr) + { + // TODO! + for (AstExpr* e : expr->expressions) + visit(e); } /** Extract a TypeId for the first type of the provided pack. @@ -375,13 +735,32 @@ struct TypeChecker2 : public AstVisitor ice.ice("flattenPack got a weird pack!"); } - bool visit(AstType* ty) override + void visit(AstType* ty) { - return true; + if (auto t = ty->as()) + return visit(t); + else if (auto t = ty->as()) + return visit(t); + else if (auto t = ty->as()) + return visit(t); + else if (auto t = ty->as()) + return visit(t); + else if (auto t = ty->as()) + return visit(t); + else if (auto t = ty->as()) + return visit(t); } - bool visit(AstTypeReference* ty) override + void visit(AstTypeReference* ty) { + for (const AstTypeOrPack& param : ty->parameters) + { + if (param.type) + visit(param.type); + else + visit(param.typePack); + } + Scope* scope = findInnermostScope(ty->location); LUAU_ASSERT(scope); @@ -500,16 +879,76 @@ struct TypeChecker2 : public AstVisitor reportError(UnknownSymbol{ty->name.value, UnknownSymbol::Context::Type}, ty->location); } } - - return true; } - bool visit(AstTypePack*) override + void visit(AstTypeTable* table) { - return true; + // TODO! + + for (const AstTableProp& prop : table->props) + visit(prop.type); + + if (table->indexer) + { + visit(table->indexer->indexType); + visit(table->indexer->resultType); + } } - bool visit(AstTypePackGeneric* tp) override + void visit(AstTypeFunction* ty) + { + // TODO! + + visit(ty->argTypes); + visit(ty->returnTypes); + } + + void visit(AstTypeTypeof* ty) + { + visit(ty->expr); + } + + void visit(AstTypeUnion* ty) + { + // TODO! + for (AstType* type : ty->types) + visit(type); + } + + void visit(AstTypeIntersection* ty) + { + // TODO! + for (AstType* type : ty->types) + visit(type); + } + + void visit(AstTypePack* pack) + { + if (auto p = pack->as()) + return visit(p); + else if (auto p = pack->as()) + return visit(p); + else if (auto p = pack->as()) + return visit(p); + } + + void visit(AstTypePackExplicit* tp) + { + // TODO! + for (AstType* type : tp->typeList.types) + visit(type); + + if (tp->typeList.tailType) + visit(tp->typeList.tailType); + } + + void visit(AstTypePackVariadic* tp) + { + // TODO! + visit(tp->variadicType); + } + + void visit(AstTypePackGeneric* tp) { Scope* scope = findInnermostScope(tp->location); LUAU_ASSERT(scope); @@ -531,8 +970,6 @@ struct TypeChecker2 : public AstVisitor reportError(UnknownSymbol{tp->genericName.value, UnknownSymbol::Context::Type}, tp->location); } } - - return true; } void reportError(TypeErrorData&& data, const Location& location) @@ -546,139 +983,9 @@ struct TypeChecker2 : public AstVisitor } std::optional getIndexTypeFromType( - const ScopePtr& scope, TypeId type, const Name& name, const Location& location, bool addErrors) + const ScopePtr& scope, TypeId type, const std::string& prop, const Location& location, bool addErrors) { - type = follow(type); - - if (get(type) || get(type) || get(type)) - return type; - - if (auto f = get(type)) - *asMutable(type) = TableTypeVar{TableState::Free, f->level}; - - if (isString(type)) - { - std::optional mtIndex = Luau::findMetatableEntry(module->errors, singletonTypes.stringType, "__index", location); - LUAU_ASSERT(mtIndex); - type = *mtIndex; - } - - if (TableTypeVar* tableType = getMutableTableType(type)) - { - - return findTablePropertyRespectingMeta(module->errors, type, name, location); - } - else if (const ClassTypeVar* cls = get(type)) - { - const Property* prop = lookupClassProp(cls, name); - if (prop) - return prop->type; - } - else if (const UnionTypeVar* utv = get(type)) - { - std::vector goodOptions; - std::vector badOptions; - - for (TypeId t : utv) - { - // TODO: we should probably limit recursion here? - // RecursionLimiter _rl(&recursionCount, FInt::LuauTypeInferRecursionLimit); - - // Not needed when we normalize types. - if (get(follow(t))) - return t; - - if (std::optional ty = getIndexTypeFromType(scope, t, name, location, /* addErrors= */ false)) - goodOptions.push_back(*ty); - else - badOptions.push_back(t); - } - - if (!badOptions.empty()) - { - if (addErrors) - { - if (goodOptions.empty()) - reportError(UnknownProperty{type, name}, location); - else - reportError(MissingUnionProperty{type, badOptions, name}, location); - } - return std::nullopt; - } - - std::vector result = reduceUnion(goodOptions); - if (result.empty()) - return singletonTypes.neverType; - - if (result.size() == 1) - return result[0]; - - return module->internalTypes.addType(UnionTypeVar{std::move(result)}); - } - else if (const IntersectionTypeVar* itv = get(type)) - { - std::vector parts; - - for (TypeId t : itv->parts) - { - // TODO: we should probably limit recursion here? - // RecursionLimiter _rl(&recursionCount, FInt::LuauTypeInferRecursionLimit); - - if (std::optional ty = getIndexTypeFromType(scope, t, name, location, /* addErrors= */ false)) - parts.push_back(*ty); - } - - // If no parts of the intersection had the property we looked up for, it never existed at all. - if (parts.empty()) - { - if (addErrors) - reportError(UnknownProperty{type, name}, location); - return std::nullopt; - } - - if (parts.size() == 1) - return parts[0]; - - return module->internalTypes.addType(IntersectionTypeVar{std::move(parts)}); // Not at all correct. - } - - if (addErrors) - reportError(UnknownProperty{type, name}, location); - - return std::nullopt; - } - - std::vector reduceUnion(const std::vector& types) - { - std::vector result; - for (TypeId t : types) - { - t = follow(t); - if (get(t)) - continue; - - if (get(t) || get(t)) - return {t}; - - if (const UnionTypeVar* utv = get(t)) - { - for (TypeId ty : utv) - { - ty = follow(ty); - if (get(ty)) - continue; - if (get(ty) || get(ty)) - return {ty}; - - if (result.end() == std::find(result.begin(), result.end(), ty)) - result.push_back(ty); - } - } - else if (std::find(result.begin(), result.end(), t) == result.end()) - result.push_back(t); - } - - return result; + return Luau::getIndexTypeFromType(scope, module->errors, &module->internalTypes, type, prop, location, addErrors, ice); } }; @@ -686,7 +993,7 @@ void check(const SourceModule& sourceModule, Module* module) { TypeChecker2 typeChecker{&sourceModule, module}; - sourceModule.root->visit(&typeChecker); + typeChecker.visit(sourceModule.root); } } // namespace Luau diff --git a/Analysis/src/TypeInfer.cpp b/Analysis/src/TypeInfer.cpp index 7ab23360..9886fb19 100644 --- a/Analysis/src/TypeInfer.cpp +++ b/Analysis/src/TypeInfer.cpp @@ -33,15 +33,13 @@ LUAU_FASTINTVARIABLE(LuauVisitRecursionLimit, 500) LUAU_FASTFLAG(LuauKnowsTheDataModel3) LUAU_FASTFLAG(LuauAutocompleteDynamicLimits) LUAU_FASTFLAGVARIABLE(LuauExpectedTableUnionIndexerType, false) +LUAU_FASTFLAGVARIABLE(LuauInplaceDemoteSkipAllBound, false) LUAU_FASTFLAGVARIABLE(LuauLowerBoundsCalculation, false) LUAU_FASTFLAGVARIABLE(DebugLuauFreezeDuringUnification, false) LUAU_FASTFLAGVARIABLE(LuauSelfCallAutocompleteFix3, false) LUAU_FASTFLAGVARIABLE(LuauReturnAnyInsteadOfICE, false) // Eventually removed as false. LUAU_FASTFLAGVARIABLE(DebugLuauSharedSelf, false); -LUAU_FASTFLAGVARIABLE(LuauReportErrorsOnIndexerKeyMismatch, false) LUAU_FASTFLAGVARIABLE(LuauUnknownAndNeverType, false) -LUAU_FASTFLAG(LuauQuantifyConstrained) -LUAU_FASTFLAGVARIABLE(LuauFalsyPredicateReturnsNilInstead, false) LUAU_FASTFLAGVARIABLE(LuauCheckGenericHOFTypes, false) LUAU_FASTFLAGVARIABLE(LuauBinaryNeedsExpectedTypesToo, false) LUAU_FASTFLAGVARIABLE(LuauNeverTypesAndOperatorsInference, false) @@ -473,7 +471,8 @@ struct InplaceDemoter : TypeVarOnceVisitor TypeArena* arena; InplaceDemoter(TypeLevel level, TypeArena* arena) - : newLevel(level) + : TypeVarOnceVisitor(/* skipBoundTypes= */ FFlag::LuauInplaceDemoteSkipAllBound) + , newLevel(level) , arena(arena) { } @@ -494,6 +493,7 @@ struct InplaceDemoter : TypeVarOnceVisitor bool visit(TypeId ty, const BoundTypeVar& btyRef) override { + LUAU_ASSERT(!FFlag::LuauInplaceDemoteSkipAllBound); return true; } @@ -656,7 +656,7 @@ void TypeChecker::checkBlockWithoutRecursionCheck(const ScopePtr& scope, const A TypeId leftType = follow(checkFunctionName(scope, *fun->name, funScope->level)); - unify(funTy, leftType, fun->location); + unify(funTy, leftType, scope, fun->location); } else if (auto fun = (*protoIter)->as()) { @@ -768,20 +768,20 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatIf& statement) } template -ErrorVec TypeChecker::canUnify_(Id subTy, Id superTy, const Location& location) +ErrorVec TypeChecker::canUnify_(Id subTy, Id superTy, const ScopePtr& scope, const Location& location) { - Unifier state = mkUnifier(location); + Unifier state = mkUnifier(scope, location); return state.canUnify(subTy, superTy); } -ErrorVec TypeChecker::canUnify(TypeId subTy, TypeId superTy, const Location& location) +ErrorVec TypeChecker::canUnify(TypeId subTy, TypeId superTy, const ScopePtr& scope, const Location& location) { - return canUnify_(subTy, superTy, location); + return canUnify_(subTy, superTy, scope, location); } -ErrorVec TypeChecker::canUnify(TypePackId subTy, TypePackId superTy, const Location& location) +ErrorVec TypeChecker::canUnify(TypePackId subTy, TypePackId superTy, const ScopePtr& scope, const Location& location) { - return canUnify_(subTy, superTy, location); + return canUnify_(subTy, superTy, scope, location); } void TypeChecker::check(const ScopePtr& scope, const AstStatWhile& statement) @@ -802,9 +802,9 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatRepeat& statement) checkExpr(repScope, *statement.condition); } -void TypeChecker::unifyLowerBound(TypePackId subTy, TypePackId superTy, TypeLevel demotedLevel, const Location& location) +void TypeChecker::unifyLowerBound(TypePackId subTy, TypePackId superTy, TypeLevel demotedLevel, const ScopePtr& scope, const Location& location) { - Unifier state = mkUnifier(location); + Unifier state = mkUnifier(scope, location); state.unifyLowerBound(subTy, superTy, demotedLevel); state.log.commit(); @@ -858,8 +858,6 @@ struct Demoter : Substitution void demote(std::vector>& expectedTypes) { - if (!FFlag::LuauQuantifyConstrained) - return; for (std::optional& ty : expectedTypes) { if (ty) @@ -897,7 +895,7 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatReturn& return_) if (useConstrainedIntersections()) { - unifyLowerBound(retPack, scope->returnType, demoter.demotedLevel(scope->level), return_.location); + unifyLowerBound(retPack, scope->returnType, demoter.demotedLevel(scope->level), scope, return_.location); return; } @@ -905,7 +903,7 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatReturn& return_) // start typechecking everything across module boundaries. if (isNonstrictMode() && follow(scope->returnType) == follow(currentModule->getModuleScope()->returnType)) { - ErrorVec errors = tryUnify(retPack, scope->returnType, return_.location); + ErrorVec errors = tryUnify(retPack, scope->returnType, scope, return_.location); if (!errors.empty()) currentModule->getModuleScope()->returnType = addTypePack({anyType}); @@ -913,13 +911,13 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatReturn& return_) return; } - unify(retPack, scope->returnType, return_.location, CountMismatch::Context::Return); + unify(retPack, scope->returnType, scope, return_.location, CountMismatch::Context::Return); } template -ErrorVec TypeChecker::tryUnify_(Id subTy, Id superTy, const Location& location) +ErrorVec TypeChecker::tryUnify_(Id subTy, Id superTy, const ScopePtr& scope, const Location& location) { - Unifier state = mkUnifier(location); + Unifier state = mkUnifier(scope, location); if (FFlag::DebugLuauFreezeDuringUnification) freeze(currentModule->internalTypes); @@ -935,14 +933,14 @@ ErrorVec TypeChecker::tryUnify_(Id subTy, Id superTy, const Location& location) return state.errors; } -ErrorVec TypeChecker::tryUnify(TypeId subTy, TypeId superTy, const Location& location) +ErrorVec TypeChecker::tryUnify(TypeId subTy, TypeId superTy, const ScopePtr& scope, const Location& location) { - return tryUnify_(subTy, superTy, location); + return tryUnify_(subTy, superTy, scope, location); } -ErrorVec TypeChecker::tryUnify(TypePackId subTy, TypePackId superTy, const Location& location) +ErrorVec TypeChecker::tryUnify(TypePackId subTy, TypePackId superTy, const ScopePtr& scope, const Location& location) { - return tryUnify_(subTy, superTy, location); + return tryUnify_(subTy, superTy, scope, location); } void TypeChecker::check(const ScopePtr& scope, const AstStatAssign& assign) @@ -1036,9 +1034,9 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatAssign& assign) { // In nonstrict mode, any assignments where the lhs is free and rhs isn't a function, we give it any typevar. if (isNonstrictMode() && get(follow(left)) && !get(follow(right))) - unify(anyType, left, loc); + unify(anyType, left, scope, loc); else - unify(right, left, loc); + unify(right, left, scope, loc); } } } @@ -1053,7 +1051,7 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatCompoundAssign& assi TypeId result = checkBinaryOperation(scope, expr, left, right); - unify(result, left, assign.location); + unify(result, left, scope, assign.location); } void TypeChecker::check(const ScopePtr& scope, const AstStatLocal& local) @@ -1108,7 +1106,7 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatLocal& local) TypePackId valuePack = checkExprList(scope, local.location, local.values, /* substituteFreeForNil= */ true, instantiateGenerics, expectedTypes).type; - Unifier state = mkUnifier(local.location); + Unifier state = mkUnifier(scope, local.location); state.ctx = CountMismatch::Result; state.tryUnify(valuePack, variablePack); reportErrors(state.errors); @@ -1184,7 +1182,7 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatFor& expr) TypeId loopVarType = numberType; if (expr.var->annotation) - unify(loopVarType, resolveType(scope, *expr.var->annotation), expr.location); + unify(loopVarType, resolveType(scope, *expr.var->annotation), scope, expr.location); loopScope->bindings[expr.var] = {loopVarType, expr.var->location}; @@ -1194,11 +1192,11 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatFor& expr) if (!expr.to) ice("Bad AstStatFor has no to expr"); - unify(checkExpr(loopScope, *expr.from).type, loopVarType, expr.from->location); - unify(checkExpr(loopScope, *expr.to).type, loopVarType, expr.to->location); + unify(checkExpr(loopScope, *expr.from).type, loopVarType, scope, expr.from->location); + unify(checkExpr(loopScope, *expr.to).type, loopVarType, scope, expr.to->location); if (expr.step) - unify(checkExpr(loopScope, *expr.step).type, loopVarType, expr.step->location); + unify(checkExpr(loopScope, *expr.step).type, loopVarType, scope, expr.step->location); check(loopScope, *expr.body); } @@ -1251,12 +1249,12 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatForIn& forin) if (get(callRetPack)) { iterTy = freshType(scope); - unify(callRetPack, addTypePack({{iterTy}, freshTypePack(scope)}), forin.location); + unify(callRetPack, addTypePack({{iterTy}, freshTypePack(scope)}), scope, forin.location); } else if (get(callRetPack) || !first(callRetPack)) { for (TypeId var : varTypes) - unify(errorRecoveryType(scope), var, forin.location); + unify(errorRecoveryType(scope), var, scope, forin.location); return check(loopScope, *forin.body); } @@ -1277,7 +1275,7 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatForIn& forin) // TODO: this needs to typecheck all returned values by __iter as if they were for loop arguments // the structure of the function makes it difficult to do this especially since we don't have actual expressions, only types for (TypeId var : varTypes) - unify(anyType, var, forin.location); + unify(anyType, var, scope, forin.location); return check(loopScope, *forin.body); } @@ -1289,25 +1287,25 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatForIn& forin) if (iterTable->indexer) { if (varTypes.size() > 0) - unify(iterTable->indexer->indexType, varTypes[0], forin.location); + unify(iterTable->indexer->indexType, varTypes[0], scope, forin.location); if (varTypes.size() > 1) - unify(iterTable->indexer->indexResultType, varTypes[1], forin.location); + unify(iterTable->indexer->indexResultType, varTypes[1], scope, forin.location); for (size_t i = 2; i < varTypes.size(); ++i) - unify(nilType, varTypes[i], forin.location); + unify(nilType, varTypes[i], scope, forin.location); } else if (isNonstrictMode()) { for (TypeId var : varTypes) - unify(anyType, var, forin.location); + unify(anyType, var, scope, forin.location); } else { TypeId varTy = errorRecoveryType(loopScope); for (TypeId var : varTypes) - unify(varTy, var, forin.location); + unify(varTy, var, scope, forin.location); reportError(firstValue->location, GenericError{"Cannot iterate over a table without indexer"}); } @@ -1321,7 +1319,7 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatForIn& forin) TypeId varTy = get(iterTy) ? anyType : errorRecoveryType(loopScope); for (TypeId var : varTypes) - unify(varTy, var, forin.location); + unify(varTy, var, scope, forin.location); if (!get(iterTy) && !get(iterTy) && !get(iterTy) && !get(iterTy)) reportError(firstValue->location, CannotCallNonFunction{iterTy}); @@ -1346,7 +1344,7 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatForIn& forin) argPack = addTypePack(TypePack{}); } - Unifier state = mkUnifier(firstValue->location); + Unifier state = mkUnifier(loopScope, firstValue->location); checkArgumentList(loopScope, state, argPack, iterFunc->argTypes, /*argLocations*/ {}); state.log.commit(); @@ -1365,10 +1363,10 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatForIn& forin) AstExprCall exprCall{Location(start, end), firstValue, arguments, /* self= */ false, Location()}; TypePackId retPack = checkExprPack(scope, exprCall).type; - unify(retPack, varPack, forin.location); + unify(retPack, varPack, scope, forin.location); } else - unify(iterFunc->retTypes, varPack, forin.location); + unify(iterFunc->retTypes, varPack, scope, forin.location); check(loopScope, *forin.body); } @@ -1603,7 +1601,7 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatTypeAlias& typealias TypeId& bindingType = bindingsMap[name].type; - if (unify(ty, bindingType, typealias.location)) + if (unify(ty, bindingType, aliasScope, typealias.location)) bindingType = ty; if (FFlag::LuauLowerBoundsCalculation) @@ -1891,7 +1889,7 @@ WithPredicate TypeChecker::checkExpr(const ScopePtr& scope, const AstExp TypeLevel level = FFlag::LuauLowerBoundsCalculation ? ftp->level : scope->level; TypeId head = freshType(level); TypePackId pack = addTypePack(TypePackVar{TypePack{{head}, freshTypePack(level)}}); - unify(pack, retPack, expr.location); + unify(pack, retPack, scope, expr.location); return {head, std::move(result.predicates)}; } if (get(retPack)) @@ -1983,20 +1981,15 @@ std::optional TypeChecker::getIndexTypeFromTypeImpl( else if (auto indexer = tableType->indexer) { // TODO: Property lookup should work with string singletons or unions thereof as the indexer key type. - ErrorVec errors = tryUnify(stringType, indexer->indexType, location); + ErrorVec errors = tryUnify(stringType, indexer->indexType, scope, location); - if (FFlag::LuauReportErrorsOnIndexerKeyMismatch) - { - if (errors.empty()) - return indexer->indexResultType; - - if (addErrors) - reportError(location, UnknownProperty{type, name}); - - return std::nullopt; - } - else + if (errors.empty()) return indexer->indexResultType; + + if (addErrors) + reportError(location, UnknownProperty{type, name}); + + return std::nullopt; } else if (tableType->state == TableState::Free) { @@ -2228,8 +2221,8 @@ TypeId TypeChecker::checkExprTable( if (indexer) { - unify(numberType, indexer->indexType, value->location); - unify(valueType, indexer->indexResultType, value->location); + unify(numberType, indexer->indexType, scope, value->location); + unify(valueType, indexer->indexResultType, scope, value->location); } else indexer = TableIndexer{numberType, anyIfNonstrict(valueType)}; @@ -2248,13 +2241,13 @@ TypeId TypeChecker::checkExprTable( if (it != expectedTable->props.end()) { Property expectedProp = it->second; - ErrorVec errors = tryUnify(exprType, expectedProp.type, k->location); + ErrorVec errors = tryUnify(exprType, expectedProp.type, scope, k->location); if (errors.empty()) exprType = expectedProp.type; } else if (expectedTable->indexer && maybeString(expectedTable->indexer->indexType)) { - ErrorVec errors = tryUnify(exprType, expectedTable->indexer->indexResultType, k->location); + ErrorVec errors = tryUnify(exprType, expectedTable->indexer->indexResultType, scope, k->location); if (errors.empty()) exprType = expectedTable->indexer->indexResultType; } @@ -2269,8 +2262,8 @@ TypeId TypeChecker::checkExprTable( if (indexer) { - unify(keyType, indexer->indexType, k->location); - unify(valueType, indexer->indexResultType, value->location); + unify(keyType, indexer->indexType, scope, k->location); + unify(valueType, indexer->indexResultType, scope, value->location); } else if (isNonstrictMode()) { @@ -2411,7 +2404,7 @@ WithPredicate TypeChecker::checkExpr(const ScopePtr& scope, const AstExp TypePackId retTypePack = freshTypePack(scope); TypeId expectedFunctionType = addType(FunctionTypeVar(scope->level, arguments, retTypePack)); - Unifier state = mkUnifier(expr.location); + Unifier state = mkUnifier(scope, expr.location); state.tryUnify(actualFunctionType, expectedFunctionType, /*isFunctionCall*/ true); state.log.commit(); @@ -2429,7 +2422,7 @@ WithPredicate TypeChecker::checkExpr(const ScopePtr& scope, const AstExp return {errorRecoveryType(scope)}; } - reportErrors(tryUnify(operandType, numberType, expr.location)); + reportErrors(tryUnify(operandType, numberType, scope, expr.location)); return {numberType}; } case AstExprUnary::Len: @@ -2459,7 +2452,7 @@ WithPredicate TypeChecker::checkExpr(const ScopePtr& scope, const AstExp TypePackId retTypePack = addTypePack({numberType}); TypeId expectedFunctionType = addType(FunctionTypeVar(scope->level, arguments, retTypePack)); - Unifier state = mkUnifier(expr.location); + Unifier state = mkUnifier(scope, expr.location); state.tryUnify(actualFunctionType, expectedFunctionType, /*isFunctionCall*/ true); state.log.commit(); @@ -2509,11 +2502,11 @@ std::string opToMetaTableEntry(const AstExprBinary::Op& op) } } -TypeId TypeChecker::unionOfTypes(TypeId a, TypeId b, const Location& location, bool unifyFreeTypes) +TypeId TypeChecker::unionOfTypes(TypeId a, TypeId b, const ScopePtr& scope, const Location& location, bool unifyFreeTypes) { if (unifyFreeTypes && (get(a) || get(b))) { - if (unify(b, a, location)) + if (unify(b, a, scope, location)) return a; return errorRecoveryType(anyType); @@ -2588,7 +2581,7 @@ TypeId TypeChecker::checkRelationalOperation( { ScopePtr subScope = childScope(scope, subexp->location); resolve(predicates, subScope, true); - return unionOfTypes(rhsType, stripNil(checkExpr(subScope, *subexp->right).type, true), expr.location); + return unionOfTypes(rhsType, stripNil(checkExpr(subScope, *subexp->right).type, true), subScope, expr.location); } } @@ -2624,7 +2617,7 @@ TypeId TypeChecker::checkRelationalOperation( * report any problems that might have been surfaced as a result of this step because we might already * have a better, more descriptive error teed up. */ - Unifier state = mkUnifier(expr.location); + Unifier state = mkUnifier(scope, expr.location); if (!isEquality) { state.tryUnify(rhsType, lhsType); @@ -2703,7 +2696,7 @@ TypeId TypeChecker::checkRelationalOperation( { if (isEquality) { - Unifier state = mkUnifier(expr.location); + Unifier state = mkUnifier(scope, expr.location); state.tryUnify(addTypePack({booleanType}), ftv->retTypes); if (!state.errors.empty()) @@ -2755,11 +2748,11 @@ TypeId TypeChecker::checkRelationalOperation( case AstExprBinary::And: if (lhsIsAny) return lhsType; - return unionOfTypes(rhsType, booleanType, expr.location, false); + return unionOfTypes(rhsType, booleanType, scope, expr.location, false); case AstExprBinary::Or: if (lhsIsAny) return lhsType; - return unionOfTypes(lhsType, rhsType, expr.location); + return unionOfTypes(lhsType, rhsType, scope, expr.location); default: LUAU_ASSERT(0); ice(format("checkRelationalOperation called with incorrect binary expression '%s'", toString(expr.op).c_str()), expr.location); @@ -2816,7 +2809,7 @@ TypeId TypeChecker::checkBinaryOperation( } if (get(rhsType)) - unify(rhsType, lhsType, expr.location); + unify(rhsType, lhsType, scope, expr.location); if (typeCouldHaveMetatable(lhsType) || typeCouldHaveMetatable(rhsType)) { @@ -2826,7 +2819,7 @@ TypeId TypeChecker::checkBinaryOperation( TypePackId retTypePack = freshTypePack(scope); TypeId expectedFunctionType = addType(FunctionTypeVar(scope->level, arguments, retTypePack)); - Unifier state = mkUnifier(expr.location); + Unifier state = mkUnifier(scope, expr.location); state.tryUnify(actualFunctionType, expectedFunctionType, /*isFunctionCall*/ true); reportErrors(state.errors); @@ -2876,8 +2869,8 @@ TypeId TypeChecker::checkBinaryOperation( switch (expr.op) { case AstExprBinary::Concat: - reportErrors(tryUnify(lhsType, addType(UnionTypeVar{{stringType, numberType}}), expr.left->location)); - reportErrors(tryUnify(rhsType, addType(UnionTypeVar{{stringType, numberType}}), expr.right->location)); + reportErrors(tryUnify(lhsType, addType(UnionTypeVar{{stringType, numberType}}), scope, expr.left->location)); + reportErrors(tryUnify(rhsType, addType(UnionTypeVar{{stringType, numberType}}), scope, expr.right->location)); return stringType; case AstExprBinary::Add: case AstExprBinary::Sub: @@ -2885,8 +2878,8 @@ TypeId TypeChecker::checkBinaryOperation( case AstExprBinary::Div: case AstExprBinary::Mod: case AstExprBinary::Pow: - reportErrors(tryUnify(lhsType, numberType, expr.left->location)); - reportErrors(tryUnify(rhsType, numberType, expr.right->location)); + reportErrors(tryUnify(lhsType, numberType, scope, expr.left->location)); + reportErrors(tryUnify(rhsType, numberType, scope, expr.right->location)); return numberType; default: // These should have been handled with checkRelationalOperation @@ -2961,10 +2954,10 @@ WithPredicate TypeChecker::checkExpr(const ScopePtr& scope, const AstExp WithPredicate result = checkExpr(scope, *expr.expr, annotationType); // Note: As an optimization, we try 'number <: number | string' first, as that is the more likely case. - if (canUnify(annotationType, result.type, expr.location).empty()) + if (canUnify(annotationType, result.type, scope, expr.location).empty()) return {annotationType, std::move(result.predicates)}; - if (canUnify(result.type, annotationType, expr.location).empty()) + if (canUnify(result.type, annotationType, scope, expr.location).empty()) return {annotationType, std::move(result.predicates)}; reportError(expr.location, TypesAreUnrelated{result.type, annotationType}); @@ -3101,7 +3094,7 @@ TypeId TypeChecker::checkLValueBinding(const ScopePtr& scope, const AstExprIndex } else if (auto indexer = lhsTable->indexer) { - Unifier state = mkUnifier(expr.location); + Unifier state = mkUnifier(scope, expr.location); state.tryUnify(stringType, indexer->indexType); TypeId retType = indexer->indexResultType; if (!state.errors.empty()) @@ -3213,7 +3206,7 @@ TypeId TypeChecker::checkLValueBinding(const ScopePtr& scope, const AstExprIndex if (exprTable->indexer) { const TableIndexer& indexer = *exprTable->indexer; - unify(indexType, indexer.indexType, expr.index->location); + unify(indexType, indexer.indexType, scope, expr.index->location); return indexer.indexResultType; } else if (exprTable->state == TableState::Unsealed || exprTable->state == TableState::Free) @@ -3829,7 +3822,7 @@ void TypeChecker::checkArgumentList( { // The use of unify here is deliberate. We don't want this unification // to be undoable. - unify(errorRecoveryType(scope), *argIter, state.location); + unify(errorRecoveryType(scope), *argIter, scope, state.location); ++argIter; } reportCountMismatchError(); @@ -3852,7 +3845,7 @@ void TypeChecker::checkArgumentList( TypeId e = errorRecoveryType(scope); while (argIter != endIter) { - unify(e, *argIter, state.location); + unify(e, *argIter, scope, state.location); ++argIter; } @@ -3869,7 +3862,7 @@ void TypeChecker::checkArgumentList( if (argIndex < argLocations.size()) location = argLocations[argIndex]; - unify(*argIter, vtp->ty, location); + unify(*argIter, vtp->ty, scope, location); ++argIter; ++argIndex; } @@ -3906,7 +3899,7 @@ void TypeChecker::checkArgumentList( } else { - unifyWithInstantiationIfNeeded(scope, *argIter, *paramIter, state); + unifyWithInstantiationIfNeeded(*argIter, *paramIter, scope, state); ++argIter; ++paramIter; } @@ -4114,7 +4107,7 @@ std::optional> TypeChecker::checkCallOverload(const Sc if (get(fn)) { - unify(anyTypePack, argPack, expr.location); + unify(anyTypePack, argPack, scope, expr.location); return {{anyTypePack}}; } @@ -4160,7 +4153,7 @@ std::optional> TypeChecker::checkCallOverload(const Sc UnifierOptions options; options.isFunctionCall = true; - unify(r, fn, expr.location, options); + unify(r, fn, scope, expr.location, options); return {{retPack}}; } @@ -4194,7 +4187,7 @@ std::optional> TypeChecker::checkCallOverload(const Sc if (!ftv) { reportError(TypeError{expr.func->location, CannotCallNonFunction{fn}}); - unify(errorRecoveryTypePack(scope), retPack, expr.func->location); + unify(errorRecoveryTypePack(scope), retPack, scope, expr.func->location); return {{errorRecoveryTypePack(retPack)}}; } @@ -4207,7 +4200,7 @@ std::optional> TypeChecker::checkCallOverload(const Sc return *ret; } - Unifier state = mkUnifier(expr.location); + Unifier state = mkUnifier(scope, expr.location); // Unify return types checkArgumentList(scope, state, retPack, ftv->retTypes, /*argLocations*/ {}); @@ -4269,7 +4262,7 @@ bool TypeChecker::handleSelfCallMismatch(const ScopePtr& scope, const AstExprCal std::vector editedParamList(args->head.begin() + 1, args->head.end()); TypePackId editedArgPack = addTypePack(TypePack{editedParamList}); - Unifier editedState = mkUnifier(expr.location); + Unifier editedState = mkUnifier(scope, expr.location); checkArgumentList(scope, editedState, editedArgPack, ftv->argTypes, editedArgLocations); if (editedState.errors.empty()) @@ -4299,7 +4292,7 @@ bool TypeChecker::handleSelfCallMismatch(const ScopePtr& scope, const AstExprCal editedArgList.insert(editedArgList.begin(), checkExpr(scope, *indexName->expr).type); TypePackId editedArgPack = addTypePack(TypePack{editedArgList}); - Unifier editedState = mkUnifier(expr.location); + Unifier editedState = mkUnifier(scope, expr.location); checkArgumentList(scope, editedState, editedArgPack, ftv->argTypes, editedArgLocations); @@ -4365,7 +4358,7 @@ void TypeChecker::reportOverloadResolutionError(const ScopePtr& scope, const Ast for (size_t i = 0; i < overloadTypes.size(); ++i) { TypeId overload = overloadTypes[i]; - Unifier state = mkUnifier(expr.location); + Unifier state = mkUnifier(scope, expr.location); // Unify return types if (const FunctionTypeVar* ftv = get(overload)) @@ -4415,7 +4408,7 @@ WithPredicate TypeChecker::checkExprList(const ScopePtr& scope, cons size_t lastIndex = exprs.size - 1; tp->head.reserve(lastIndex); - Unifier state = mkUnifier(location); + Unifier state = mkUnifier(scope, location); std::vector inverseLogs; @@ -4580,15 +4573,15 @@ TypeId TypeChecker::anyIfNonstrict(TypeId ty) const return ty; } -bool TypeChecker::unify(TypeId subTy, TypeId superTy, const Location& location) +bool TypeChecker::unify(TypeId subTy, TypeId superTy, const ScopePtr& scope, const Location& location) { UnifierOptions options; - return unify(subTy, superTy, location, options); + return unify(subTy, superTy, scope, location, options); } -bool TypeChecker::unify(TypeId subTy, TypeId superTy, const Location& location, const UnifierOptions& options) +bool TypeChecker::unify(TypeId subTy, TypeId superTy, const ScopePtr& scope, const Location& location, const UnifierOptions& options) { - Unifier state = mkUnifier(location); + Unifier state = mkUnifier(scope, location); state.tryUnify(subTy, superTy, options.isFunctionCall); state.log.commit(); @@ -4598,9 +4591,9 @@ bool TypeChecker::unify(TypeId subTy, TypeId superTy, const Location& location, return state.errors.empty(); } -bool TypeChecker::unify(TypePackId subTy, TypePackId superTy, const Location& location, CountMismatch::Context ctx) +bool TypeChecker::unify(TypePackId subTy, TypePackId superTy, const ScopePtr& scope, const Location& location, CountMismatch::Context ctx) { - Unifier state = mkUnifier(location); + Unifier state = mkUnifier(scope, location); state.ctx = ctx; state.tryUnify(subTy, superTy); @@ -4611,10 +4604,10 @@ bool TypeChecker::unify(TypePackId subTy, TypePackId superTy, const Location& lo return state.errors.empty(); } -bool TypeChecker::unifyWithInstantiationIfNeeded(const ScopePtr& scope, TypeId subTy, TypeId superTy, const Location& location) +bool TypeChecker::unifyWithInstantiationIfNeeded(TypeId subTy, TypeId superTy, const ScopePtr& scope, const Location& location) { - Unifier state = mkUnifier(location); - unifyWithInstantiationIfNeeded(scope, subTy, superTy, state); + Unifier state = mkUnifier(scope, location); + unifyWithInstantiationIfNeeded(subTy, superTy, scope, state); state.log.commit(); @@ -4623,7 +4616,7 @@ bool TypeChecker::unifyWithInstantiationIfNeeded(const ScopePtr& scope, TypeId s return state.errors.empty(); } -void TypeChecker::unifyWithInstantiationIfNeeded(const ScopePtr& scope, TypeId subTy, TypeId superTy, Unifier& state) +void TypeChecker::unifyWithInstantiationIfNeeded(TypeId subTy, TypeId superTy, const ScopePtr& scope, Unifier& state) { if (!maybeGeneric(subTy)) // Quick check to see if we definitely can't instantiate @@ -4662,77 +4655,6 @@ void TypeChecker::unifyWithInstantiationIfNeeded(const ScopePtr& scope, TypeId s } } -bool Anyification::isDirty(TypeId ty) -{ - if (ty->persistent) - return false; - - if (const TableTypeVar* ttv = log->getMutable(ty)) - return (ttv->state == TableState::Free || ttv->state == TableState::Unsealed); - else if (log->getMutable(ty)) - return true; - else if (get(ty)) - return true; - else - return false; -} - -bool Anyification::isDirty(TypePackId tp) -{ - if (tp->persistent) - return false; - - if (log->getMutable(tp)) - return true; - else - return false; -} - -TypeId Anyification::clean(TypeId ty) -{ - LUAU_ASSERT(isDirty(ty)); - if (const TableTypeVar* ttv = log->getMutable(ty)) - { - TableTypeVar clone = TableTypeVar{ttv->props, ttv->indexer, ttv->level, TableState::Sealed}; - clone.definitionModuleName = ttv->definitionModuleName; - clone.name = ttv->name; - clone.syntheticName = ttv->syntheticName; - clone.tags = ttv->tags; - TypeId res = addType(std::move(clone)); - asMutable(res)->normal = ty->normal; - return res; - } - else if (auto ctv = get(ty)) - { - if (FFlag::LuauQuantifyConstrained) - { - std::vector copy = ctv->parts; - for (TypeId& ty : copy) - ty = replace(ty); - TypeId res = copy.size() == 1 ? copy[0] : addType(UnionTypeVar{std::move(copy)}); - auto [t, ok] = normalize(res, *arena, *iceHandler); - if (!ok) - normalizationTooComplex = true; - return t; - } - else - { - auto [t, ok] = normalize(ty, *arena, *iceHandler); - if (!ok) - normalizationTooComplex = true; - return t; - } - } - else - return anyType; -} - -TypePackId Anyification::clean(TypePackId tp) -{ - LUAU_ASSERT(isDirty(tp)); - return anyTypePack; -} - TypeId TypeChecker::quantify(const ScopePtr& scope, TypeId ty, Location location) { ty = follow(ty); @@ -4804,7 +4726,7 @@ TypeId TypeChecker::anyify(const ScopePtr& scope, TypeId ty, Location location) ty = t; } - Anyification anyification{¤tModule->internalTypes, iceHandler, anyType, anyTypePack}; + Anyification anyification{¤tModule->internalTypes, scope, iceHandler, anyType, anyTypePack}; std::optional any = anyification.substitute(ty); if (anyification.normalizationTooComplex) reportError(location, NormalizationTooComplex{}); @@ -4827,7 +4749,7 @@ TypePackId TypeChecker::anyify(const ScopePtr& scope, TypePackId ty, Location lo ty = t; } - Anyification anyification{¤tModule->internalTypes, iceHandler, anyType, anyTypePack}; + Anyification anyification{¤tModule->internalTypes, scope, iceHandler, anyType, anyTypePack}; std::optional any = anyification.substitute(ty); if (any.has_value()) return *any; @@ -4963,9 +4885,9 @@ void TypeChecker::merge(RefinementMap& l, const RefinementMap& r) }); } -Unifier TypeChecker::mkUnifier(const Location& location) +Unifier TypeChecker::mkUnifier(const ScopePtr& scope, const Location& location) { - return Unifier{¤tModule->internalTypes, currentModule->mode, location, Variance::Covariant, unifierState}; + return Unifier{¤tModule->internalTypes, currentModule->mode, NotNull{scope.get()}, location, Variance::Covariant, unifierState}; } TypeId TypeChecker::freshType(const ScopePtr& scope) @@ -5029,10 +4951,7 @@ TypeIdPredicate TypeChecker::mkTruthyPredicate(bool sense) return sense ? std::nullopt : std::optional(ty); // at this point, anything else is kept if sense is true, or replaced by nil - if (FFlag::LuauFalsyPredicateReturnsNilInstead) - return sense ? ty : nilType; - else - return sense ? std::optional(ty) : std::nullopt; + return sense ? ty : nilType; }; } @@ -5875,8 +5794,8 @@ void TypeChecker::resolve(const IsAPredicate& isaP, RefinementMap& refis, const { auto predicate = [&](TypeId option) -> std::optional { // This by itself is not truly enough to determine that A is stronger than B or vice versa. - bool optionIsSubtype = canUnify(option, isaP.ty, isaP.location).empty(); - bool targetIsSubtype = canUnify(isaP.ty, option, isaP.location).empty(); + bool optionIsSubtype = canUnify(option, isaP.ty, scope, isaP.location).empty(); + bool targetIsSubtype = canUnify(isaP.ty, option, scope, isaP.location).empty(); // If A is a superset of B, then if sense is true, we promote A to B, otherwise we keep A. if (!optionIsSubtype && targetIsSubtype) @@ -6019,7 +5938,7 @@ void TypeChecker::resolve(const EqPredicate& eqP, RefinementMap& refis, const Sc if (maybeSingleton(eqP.type)) { // Normally we'd write option <: eqP.type, but singletons are always the subtype, so we flip this. - if (!sense || canUnify(eqP.type, option, eqP.location).empty()) + if (!sense || canUnify(eqP.type, option, scope, eqP.location).empty()) return sense ? eqP.type : option; // local variable works around an odd gcc 9.3 warning: may be used uninitialized @@ -6053,7 +5972,7 @@ std::vector TypeChecker::unTypePack(const ScopePtr& scope, TypePackId tp size_t oldErrorsSize = currentModule->errors.size(); - unify(tp, expectedTypePack, location); + unify(tp, expectedTypePack, scope, location); // HACK: tryUnify would undo the changes to the expectedTypePack if the length mismatches, but // we want to tie up free types to be error types, so we do this instead. diff --git a/Analysis/src/TypeUtils.cpp b/Analysis/src/TypeUtils.cpp index 66b38cf3..60bca0a3 100644 --- a/Analysis/src/TypeUtils.cpp +++ b/Analysis/src/TypeUtils.cpp @@ -1,6 +1,7 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #include "Luau/TypeUtils.h" +#include "Luau/Normalize.h" #include "Luau/Scope.h" #include "Luau/ToString.h" #include "Luau/TypeInfer.h" @@ -8,7 +9,7 @@ namespace Luau { -std::optional findMetatableEntry(ErrorVec& errors, TypeId type, std::string entry, Location location) +std::optional findMetatableEntry(ErrorVec& errors, TypeId type, const std::string& entry, Location location) { type = follow(type); @@ -35,7 +36,7 @@ std::optional findMetatableEntry(ErrorVec& errors, TypeId type, std::str return std::nullopt; } -std::optional findTablePropertyRespectingMeta(ErrorVec& errors, TypeId ty, Name name, Location location) +std::optional findTablePropertyRespectingMeta(ErrorVec& errors, TypeId ty, const std::string& name, Location location) { if (get(ty)) return ty; @@ -83,4 +84,110 @@ std::optional findTablePropertyRespectingMeta(ErrorVec& errors, TypeId t return std::nullopt; } +std::optional getIndexTypeFromType( + const ScopePtr& scope, ErrorVec& errors, TypeArena* arena, TypeId type, const std::string& prop, const Location& location, bool addErrors, + InternalErrorReporter& handle) +{ + type = follow(type); + + if (get(type) || get(type) || get(type)) + return type; + + if (auto f = get(type)) + *asMutable(type) = TableTypeVar{TableState::Free, f->level}; + + if (isString(type)) + { + std::optional mtIndex = Luau::findMetatableEntry(errors, getSingletonTypes().stringType, "__index", location); + LUAU_ASSERT(mtIndex); + type = *mtIndex; + } + + if (getTableType(type)) + { + return findTablePropertyRespectingMeta(errors, type, prop, location); + } + else if (const ClassTypeVar* cls = get(type)) + { + if (const Property* p = lookupClassProp(cls, prop)) + return p->type; + } + else if (const UnionTypeVar* utv = get(type)) + { + std::vector goodOptions; + std::vector badOptions; + + for (TypeId t : utv) + { + // TODO: we should probably limit recursion here? + // RecursionLimiter _rl(&recursionCount, FInt::LuauTypeInferRecursionLimit); + + // Not needed when we normalize types. + if (get(follow(t))) + return t; + + if (std::optional ty = getIndexTypeFromType(scope, errors, arena, t, prop, location, /* addErrors= */ false, handle)) + goodOptions.push_back(*ty); + else + badOptions.push_back(t); + } + + if (!badOptions.empty()) + { + if (addErrors) + { + if (goodOptions.empty()) + errors.push_back(TypeError{location, UnknownProperty{type, prop}}); + else + errors.push_back(TypeError{location, MissingUnionProperty{type, badOptions, prop}}); + } + return std::nullopt; + } + + if (goodOptions.empty()) + return getSingletonTypes().neverType; + + if (goodOptions.size() == 1) + return goodOptions[0]; + + // TODO: inefficient. + TypeId result = arena->addType(UnionTypeVar{std::move(goodOptions)}); + auto [ty, ok] = normalize(result, NotNull{scope.get()}, *arena, handle); + if (!ok && addErrors) + errors.push_back(TypeError{location, NormalizationTooComplex{}}); + return ok ? ty : getSingletonTypes().anyType; + } + else if (const IntersectionTypeVar* itv = get(type)) + { + std::vector parts; + + for (TypeId t : itv->parts) + { + // TODO: we should probably limit recursion here? + // RecursionLimiter _rl(&recursionCount, FInt::LuauTypeInferRecursionLimit); + + if (std::optional ty = getIndexTypeFromType(scope, errors, arena, t, prop, location, /* addErrors= */ false, handle)) + parts.push_back(*ty); + } + + // If no parts of the intersection had the property we looked up for, it never existed at all. + if (parts.empty()) + { + if (addErrors) + errors.push_back(TypeError{location, UnknownProperty{type, prop}}); + return std::nullopt; + } + + if (parts.size() == 1) + return parts[0]; + + return arena->addType(IntersectionTypeVar{std::move(parts)}); // Not at all correct. + } + + if (addErrors) + errors.push_back(TypeError{location, UnknownProperty{type, prop}}); + + return std::nullopt; +} + } // namespace Luau diff --git a/Analysis/src/TypeVar.cpp b/Analysis/src/TypeVar.cpp index ada2b012..9020b1ac 100644 --- a/Analysis/src/TypeVar.cpp +++ b/Analysis/src/TypeVar.cpp @@ -1135,7 +1135,7 @@ std::optional> magicFunctionFormat( { Location location = expr.args.data[std::min(i + dataOffset, expr.args.size - 1)]->location; - typechecker.unify(params[i + paramOffset], expected[i], location); + typechecker.unify(params[i + paramOffset], expected[i], scope, location); } // if we know the argument count or if we have too many arguments for sure, we can issue an error @@ -1234,7 +1234,7 @@ static std::optional> magicFunctionGmatch( if (returnTypes.empty()) return std::nullopt; - typechecker.unify(params[0], typechecker.stringType, expr.args.data[0]->location); + typechecker.unify(params[0], typechecker.stringType, scope, expr.args.data[0]->location); const TypePackId emptyPack = arena.addTypePack({}); const TypePackId returnList = arena.addTypePack(returnTypes); @@ -1269,13 +1269,13 @@ static std::optional> magicFunctionMatch( if (returnTypes.empty()) return std::nullopt; - typechecker.unify(params[0], typechecker.stringType, expr.args.data[0]->location); + typechecker.unify(params[0], typechecker.stringType, scope, expr.args.data[0]->location); const TypeId optionalNumber = arena.addType(UnionTypeVar{{typechecker.nilType, typechecker.numberType}}); size_t initIndex = expr.self ? 1 : 2; if (params.size() == 3 && expr.args.size > initIndex) - typechecker.unify(params[2], optionalNumber, expr.args.data[initIndex]->location); + typechecker.unify(params[2], optionalNumber, scope, expr.args.data[initIndex]->location); const TypePackId returnList = arena.addTypePack(returnTypes); return WithPredicate{returnList}; @@ -1320,17 +1320,17 @@ static std::optional> magicFunctionFind( return std::nullopt; } - typechecker.unify(params[0], typechecker.stringType, expr.args.data[0]->location); + typechecker.unify(params[0], typechecker.stringType, scope, expr.args.data[0]->location); const TypeId optionalNumber = arena.addType(UnionTypeVar{{typechecker.nilType, typechecker.numberType}}); const TypeId optionalBoolean = arena.addType(UnionTypeVar{{typechecker.nilType, typechecker.booleanType}}); size_t initIndex = expr.self ? 1 : 2; if (params.size() >= 3 && expr.args.size > initIndex) - typechecker.unify(params[2], optionalNumber, expr.args.data[initIndex]->location); + typechecker.unify(params[2], optionalNumber, scope, expr.args.data[initIndex]->location); if (params.size() == 4 && expr.args.size > plainIndex) - typechecker.unify(params[3], optionalBoolean, expr.args.data[plainIndex]->location); + typechecker.unify(params[3], optionalBoolean, scope, expr.args.data[plainIndex]->location); returnTypes.insert(returnTypes.begin(), {optionalNumber, optionalNumber}); diff --git a/Analysis/src/TypedAllocator.cpp b/Analysis/src/TypedAllocator.cpp index c7f31822..9ce8c3dc 100644 --- a/Analysis/src/TypedAllocator.cpp +++ b/Analysis/src/TypedAllocator.cpp @@ -27,27 +27,6 @@ LUAU_FASTFLAG(DebugLuauFreezeArena) namespace Luau { -static void* systemAllocateAligned(size_t size, size_t align) -{ -#ifdef _WIN32 - return _aligned_malloc(size, align); -#elif defined(__ANDROID__) // for Android 4.1 - return memalign(align, size); -#else - void* ptr; - return posix_memalign(&ptr, align, size) == 0 ? ptr : 0; -#endif -} - -static void systemDeallocateAligned(void* ptr) -{ -#ifdef _WIN32 - _aligned_free(ptr); -#else - free(ptr); -#endif -} - static size_t pageAlign(size_t size) { return (size + kPageSize - 1) & ~(kPageSize - 1); @@ -55,18 +34,31 @@ static size_t pageAlign(size_t size) void* pagedAllocate(size_t size) { - if (FFlag::DebugLuauFreezeArena) - return systemAllocateAligned(pageAlign(size), kPageSize); - else + // By default we use operator new/delete instead of malloc/free so that they can be overridden externally + if (!FFlag::DebugLuauFreezeArena) return ::operator new(size, std::nothrow); + + // On Windows, VirtualAlloc results in 64K granularity allocations; we allocate in chunks of ~32K so aligned_malloc is a little more efficient + // On Linux, we must use mmap because using regular heap results in mprotect() fragmenting the page table and us bumping into 64K mmap limit. +#ifdef _WIN32 + return _aligned_malloc(size, kPageSize); +#else + return mmap(nullptr, pageAlign(size), PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANON, -1, 0); +#endif } -void pagedDeallocate(void* ptr) +void pagedDeallocate(void* ptr, size_t size) { - if (FFlag::DebugLuauFreezeArena) - systemDeallocateAligned(ptr); - else - ::operator delete(ptr); + // By default we use operator new/delete instead of malloc/free so that they can be overridden externally + if (!FFlag::DebugLuauFreezeArena) + return ::operator delete(ptr); + +#ifdef _WIN32 + _aligned_free(ptr); +#else + int rc = munmap(ptr, size); + LUAU_ASSERT(rc == 0); +#endif } void pagedFreeze(void* ptr, size_t size) diff --git a/Analysis/src/Unifier.cpp b/Analysis/src/Unifier.cpp index a0f4725d..b5f58c83 100644 --- a/Analysis/src/Unifier.cpp +++ b/Analysis/src/Unifier.cpp @@ -20,7 +20,6 @@ LUAU_FASTINTVARIABLE(LuauTypeInferLowerBoundsIterationLimit, 2000); LUAU_FASTFLAG(LuauLowerBoundsCalculation); LUAU_FASTFLAG(LuauErrorRecoveryType); LUAU_FASTFLAG(LuauUnknownAndNeverType) -LUAU_FASTFLAG(LuauQuantifyConstrained) LUAU_FASTFLAGVARIABLE(LuauScalarShapeSubtyping, false) LUAU_FASTFLAG(LuauClassTypeVarsInSubstitution) @@ -318,9 +317,10 @@ static std::optional> getTableMat return std::nullopt; } -Unifier::Unifier(TypeArena* types, Mode mode, const Location& location, Variance variance, UnifierSharedState& sharedState, TxnLog* parentLog) +Unifier::Unifier(TypeArena* types, Mode mode, NotNull scope, const Location& location, Variance variance, UnifierSharedState& sharedState, TxnLog* parentLog) : types(types) , mode(mode) + , scope(scope) , log(parentLog) , location(location) , variance(variance) @@ -2091,13 +2091,11 @@ void Unifier::unifyLowerBound(TypePackId subTy, TypePackId superTy, TypeLevel de if (!freeTailPack) return; - TypeLevel level = FFlag::LuauQuantifyConstrained ? demotedLevel : freeTailPack->level; - TypePack* tp = getMutable(log.replace(tailPack, TypePack{})); for (; subIter != subEndIter; ++subIter) { - tp->head.push_back(types->addType(ConstrainedTypeVar{level, {follow(*subIter)}})); + tp->head.push_back(types->addType(ConstrainedTypeVar{demotedLevel, {follow(*subIter)}})); } tp->tail = subIter.tail(); @@ -2270,7 +2268,7 @@ bool Unifier::occursCheck(DenseHashSet& seen, TypePackId needle, Typ Unifier Unifier::makeChildUnifier() { - Unifier u = Unifier{types, mode, location, variance, sharedState, &log}; + Unifier u = Unifier{types, mode, scope, location, variance, sharedState, &log}; u.anyIsTop = anyIsTop; return u; } diff --git a/Ast/include/Luau/Parser.h b/Ast/include/Luau/Parser.h index 4b5ae315..046706d3 100644 --- a/Ast/include/Luau/Parser.h +++ b/Ast/include/Luau/Parser.h @@ -138,7 +138,7 @@ private: // funcbody ::= `(' [parlist] `)' block end // parlist ::= namelist [`,' `...'] | `...' std::pair parseFunctionBody( - bool hasself, const Lexeme& matchFunction, const AstName& debugname, std::optional localName); + bool hasself, const Lexeme& matchFunction, const AstName& debugname, const Name* localName); // explist ::= {exp `,'} exp void parseExprList(TempVector& result); @@ -217,7 +217,7 @@ private: AstExpr* parseSimpleExpr(); // args ::= `(' [explist] `)' | tableconstructor | String - AstExpr* parseFunctionArgs(AstExpr* func, bool self, const Location& selfLocation); + AstExpr* parseFunctionArgs(AstExpr* func, bool self); // tableconstructor ::= `{' [fieldlist] `}' // fieldlist ::= field {fieldsep field} [fieldsep] @@ -241,6 +241,7 @@ private: std::optional> parseCharArray(); AstExpr* parseString(); + AstExpr* parseNumber(); AstLocal* pushLocal(const Binding& binding); @@ -253,11 +254,24 @@ private: bool expectAndConsume(Lexeme::Type type, const char* context = nullptr); void expectAndConsumeFail(Lexeme::Type type, const char* context); - bool expectMatchAndConsume(char value, const Lexeme& begin, bool searchForMissing = false); - void expectMatchAndConsumeFail(Lexeme::Type type, const Lexeme& begin, const char* extra = nullptr); + struct MatchLexeme + { + MatchLexeme(const Lexeme& l) + : type(l.type) + , position(l.location.begin) + { + } - bool expectMatchEndAndConsume(Lexeme::Type type, const Lexeme& begin); - void expectMatchEndAndConsumeFail(Lexeme::Type type, const Lexeme& begin); + Lexeme::Type type; + Position position; + }; + + bool expectMatchAndConsume(char value, const MatchLexeme& begin, bool searchForMissing = false); + void expectMatchAndConsumeFail(Lexeme::Type type, const MatchLexeme& begin, const char* extra = nullptr); + bool expectMatchAndConsumeRecover(char value, const MatchLexeme& begin, bool searchForMissing); + + bool expectMatchEndAndConsume(Lexeme::Type type, const MatchLexeme& begin); + void expectMatchEndAndConsumeFail(Lexeme::Type type, const MatchLexeme& begin); template AstArray copy(const T* data, std::size_t size); @@ -283,6 +297,9 @@ private: AstTypeError* reportTypeAnnotationError(const Location& location, const AstArray& types, bool isMissing, const char* format, ...) LUAU_PRINTF_ATTR(5, 6); + AstExpr* reportFunctionArgsError(AstExpr* func, bool self); + void reportAmbiguousCallError(); + void nextLexeme(); struct Function @@ -350,7 +367,7 @@ private: AstName nameError; AstName nameNil; - Lexeme endMismatchSuspect; + MatchLexeme endMismatchSuspect; std::vector functionStack; diff --git a/Ast/src/Parser.cpp b/Ast/src/Parser.cpp index 1eb9565f..e46eebf3 100644 --- a/Ast/src/Parser.cpp +++ b/Ast/src/Parser.cpp @@ -14,8 +14,6 @@ LUAU_FASTINTVARIABLE(LuauRecursionLimit, 1000) LUAU_FASTINTVARIABLE(LuauParseErrorLimit, 100) -LUAU_FASTFLAGVARIABLE(LuauParserFunctionKeywordAsTypeHelp, false) - LUAU_FASTFLAGVARIABLE(LuauFixNamedFunctionParse, false) LUAU_DYNAMIC_FASTFLAGVARIABLE(LuaReportParseWrongNamedType, false) @@ -177,7 +175,7 @@ Parser::Parser(const char* buffer, size_t bufferSize, AstNameTable& names, Alloc , lexer(buffer, bufferSize, names) , allocator(allocator) , recursionCounter(0) - , endMismatchSuspect(Location(), Lexeme::Eof) + , endMismatchSuspect(Lexeme(Location(), Lexeme::Eof)) , localMap(AstName()) { Function top; @@ -657,7 +655,7 @@ AstStat* Parser::parseFunctionStat() matchRecoveryStopOnToken[Lexeme::ReservedEnd]++; - AstExprFunction* body = parseFunctionBody(hasself, matchFunction, debugname, {}).first; + AstExprFunction* body = parseFunctionBody(hasself, matchFunction, debugname, nullptr).first; matchRecoveryStopOnToken[Lexeme::ReservedEnd]--; @@ -686,7 +684,7 @@ AstStat* Parser::parseLocal() matchRecoveryStopOnToken[Lexeme::ReservedEnd]++; - auto [body, var] = parseFunctionBody(false, matchFunction, name.name, name); + auto [body, var] = parseFunctionBody(false, matchFunction, name.name, &name); matchRecoveryStopOnToken[Lexeme::ReservedEnd]--; @@ -778,7 +776,7 @@ AstDeclaredClassProp Parser::parseDeclaredClassMethod() genericPacks.size = 0; genericPacks.data = nullptr; - Lexeme matchParen = lexer.current(); + MatchLexeme matchParen = lexer.current(); expectAndConsume('(', "function parameter list start"); TempVector args(scratchBinding); @@ -834,7 +832,7 @@ AstStat* Parser::parseDeclaration(const Location& start) auto [generics, genericPacks] = parseGenericTypeList(/* withDefaultValues= */ false); - Lexeme matchParen = lexer.current(); + MatchLexeme matchParen = lexer.current(); expectAndConsume('(', "global function declaration"); @@ -970,13 +968,13 @@ AstStat* Parser::parseCompoundAssignment(AstExpr* initial, AstExprBinary::Op op) // funcbody ::= `(' [parlist] `)' [`:' ReturnType] block end // parlist ::= bindinglist [`,' `...'] | `...' std::pair Parser::parseFunctionBody( - bool hasself, const Lexeme& matchFunction, const AstName& debugname, std::optional localName) + bool hasself, const Lexeme& matchFunction, const AstName& debugname, const Name* localName) { Location start = matchFunction.location; auto [generics, genericPacks] = parseGenericTypeList(/* withDefaultValues= */ false); - Lexeme matchParen = lexer.current(); + MatchLexeme matchParen = lexer.current(); expectAndConsume('(', "function"); TempVector args(scratchBinding); @@ -988,7 +986,7 @@ std::pair Parser::parseFunctionBody( std::tie(vararg, varargAnnotation) = parseBindingList(args, /* allowDot3= */ true); std::optional argLocation = matchParen.type == Lexeme::Type('(') && lexer.current().type == Lexeme::Type(')') - ? std::make_optional(Location(matchParen.location.begin, lexer.current().location.end)) + ? std::make_optional(Location(matchParen.position, lexer.current().location.end)) : std::nullopt; expectMatchAndConsume(')', matchParen, true); @@ -1255,7 +1253,7 @@ AstType* Parser::parseTableTypeAnnotation() Location start = lexer.current().location; - Lexeme matchBrace = lexer.current(); + MatchLexeme matchBrace = lexer.current(); expectAndConsume('{', "table type"); while (lexer.current().type != '}') @@ -1628,7 +1626,7 @@ AstTypeOrPack Parser::parseSimpleTypeAnnotation(bool allowPack) { return parseFunctionTypeAnnotation(allowPack); } - else if (FFlag::LuauParserFunctionKeywordAsTypeHelp && lexer.current().type == Lexeme::ReservedFunction) + else if (lexer.current().type == Lexeme::ReservedFunction) { Location location = lexer.current().location; @@ -1912,14 +1910,14 @@ AstExpr* Parser::parsePrefixExpr() { if (lexer.current().type == '(') { - Location start = lexer.current().location; + Position start = lexer.current().location.begin; - Lexeme matchParen = lexer.current(); + MatchLexeme matchParen = lexer.current(); nextLexeme(); AstExpr* expr = parseExpr(); - Location end = lexer.current().location; + Position end = lexer.current().location.end; if (lexer.current().type != ')') { @@ -1927,7 +1925,7 @@ AstExpr* Parser::parsePrefixExpr() expectMatchAndConsumeFail(static_cast(')'), matchParen, suggestion); - end = lexer.previousLocation(); + end = lexer.previousLocation().end; } else { @@ -1945,7 +1943,7 @@ AstExpr* Parser::parsePrefixExpr() // primaryexp -> prefixexp { `.' NAME | `[' exp `]' | `:' NAME funcargs | funcargs } AstExpr* Parser::parsePrimaryExpr(bool asStatement) { - Location start = lexer.current().location; + Position start = lexer.current().location.begin; AstExpr* expr = parsePrefixExpr(); @@ -1960,16 +1958,16 @@ AstExpr* Parser::parsePrimaryExpr(bool asStatement) Name index = parseIndexName(nullptr, opPosition); - expr = allocator.alloc(Location(start, index.location), expr, index.name, index.location, opPosition, '.'); + expr = allocator.alloc(Location(start, index.location.end), expr, index.name, index.location, opPosition, '.'); } else if (lexer.current().type == '[') { - Lexeme matchBracket = lexer.current(); + MatchLexeme matchBracket = lexer.current(); nextLexeme(); AstExpr* index = parseExpr(); - Location end = lexer.current().location; + Position end = lexer.current().location.end; expectMatchAndConsume(']', matchBracket); @@ -1981,27 +1979,24 @@ AstExpr* Parser::parsePrimaryExpr(bool asStatement) nextLexeme(); Name index = parseIndexName("method name", opPosition); - AstExpr* func = allocator.alloc(Location(start, index.location), expr, index.name, index.location, opPosition, ':'); + AstExpr* func = allocator.alloc(Location(start, index.location.end), expr, index.name, index.location, opPosition, ':'); - expr = parseFunctionArgs(func, true, index.location); + expr = parseFunctionArgs(func, true); } else if (lexer.current().type == '(') { // This error is handled inside 'parseFunctionArgs' as well, but for better error recovery we need to break out the current loop here if (!asStatement && expr->location.end.line != lexer.current().location.begin.line) { - report(lexer.current().location, - "Ambiguous syntax: this looks like an argument list for a function call, but could also be a start of " - "new statement; use ';' to separate statements"); - + reportAmbiguousCallError(); break; } - expr = parseFunctionArgs(expr, false, Location()); + expr = parseFunctionArgs(expr, false); } else if (lexer.current().type == '{' || lexer.current().type == Lexeme::RawString || lexer.current().type == Lexeme::QuotedString) { - expr = parseFunctionArgs(expr, false, Location()); + expr = parseFunctionArgs(expr, false); } else { @@ -2156,7 +2151,7 @@ static ConstantNumberParseResult parseInteger(double& result, const char* data, return ConstantNumberParseResult::Ok; } -static ConstantNumberParseResult parseNumber(double& result, const char* data) +static ConstantNumberParseResult parseDouble(double& result, const char* data) { LUAU_ASSERT(FFlag::LuauLintParseIntegerIssues); @@ -2214,61 +2209,11 @@ AstExpr* Parser::parseSimpleExpr() Lexeme matchFunction = lexer.current(); nextLexeme(); - return parseFunctionBody(false, matchFunction, AstName(), {}).first; + return parseFunctionBody(false, matchFunction, AstName(), nullptr).first; } else if (lexer.current().type == Lexeme::Number) { - scratchData.assign(lexer.current().data, lexer.current().length); - - // Remove all internal _ - they don't hold any meaning and this allows parsing code to just pass the string pointer to strtod et al - if (scratchData.find('_') != std::string::npos) - { - scratchData.erase(std::remove(scratchData.begin(), scratchData.end(), '_'), scratchData.end()); - } - - if (FFlag::LuauLintParseIntegerIssues) - { - double value = 0; - ConstantNumberParseResult result = parseNumber(value, scratchData.c_str()); - nextLexeme(); - - if (result == ConstantNumberParseResult::Malformed) - return reportExprError(start, {}, "Malformed number"); - - return allocator.alloc(start, value, result); - } - else if (DFFlag::LuaReportParseIntegerIssues) - { - double value = 0; - if (const char* error = parseNumber_DEPRECATED2(value, scratchData.c_str())) - { - nextLexeme(); - - return reportExprError(start, {}, "%s", error); - } - else - { - nextLexeme(); - - return allocator.alloc(start, value); - } - } - else - { - double value = 0; - if (parseNumber_DEPRECATED(value, scratchData.c_str())) - { - nextLexeme(); - - return allocator.alloc(start, value); - } - else - { - nextLexeme(); - - return reportExprError(start, {}, "Malformed number"); - } - } + return parseNumber(); } else if (lexer.current().type == Lexeme::RawString || lexer.current().type == Lexeme::QuotedString) { @@ -2309,18 +2254,15 @@ AstExpr* Parser::parseSimpleExpr() } // args ::= `(' [explist] `)' | tableconstructor | String -AstExpr* Parser::parseFunctionArgs(AstExpr* func, bool self, const Location& selfLocation) +AstExpr* Parser::parseFunctionArgs(AstExpr* func, bool self) { if (lexer.current().type == '(') { Position argStart = lexer.current().location.end; if (func->location.end.line != lexer.current().location.begin.line) - { - report(lexer.current().location, "Ambiguous syntax: this looks like an argument list for a function call, but could also be a start of " - "new statement; use ';' to separate statements"); - } + reportAmbiguousCallError(); - Lexeme matchParen = lexer.current(); + MatchLexeme matchParen = lexer.current(); nextLexeme(); TempVector args(scratchExpr); @@ -2352,18 +2294,29 @@ AstExpr* Parser::parseFunctionArgs(AstExpr* func, bool self, const Location& sel } else { - if (self && lexer.current().location.begin.line != func->location.end.line) - { - return reportExprError(func->location, copy({func}), "Expected function call arguments after '('"); - } - else - { - return reportExprError(Location(func->location.begin, lexer.current().location.begin), copy({func}), - "Expected '(', '{' or when parsing function call, got %s", lexer.current().toString().c_str()); - } + return reportFunctionArgsError(func, self); } } +LUAU_NOINLINE AstExpr* Parser::reportFunctionArgsError(AstExpr* func, bool self) +{ + if (self && lexer.current().location.begin.line != func->location.end.line) + { + return reportExprError(func->location, copy({func}), "Expected function call arguments after '('"); + } + else + { + return reportExprError(Location(func->location.begin, lexer.current().location.begin), copy({func}), + "Expected '(', '{' or when parsing function call, got %s", lexer.current().toString().c_str()); + } +} + +LUAU_NOINLINE void Parser::reportAmbiguousCallError() +{ + report(lexer.current().location, "Ambiguous syntax: this looks like an argument list for a function call, but could also be a start of " + "new statement; use ';' to separate statements"); +} + // tableconstructor ::= `{' [fieldlist] `}' // fieldlist ::= field {fieldsep field} [fieldsep] // field ::= `[' exp `]' `=' exp | Name `=' exp | exp @@ -2374,14 +2327,14 @@ AstExpr* Parser::parseTableConstructor() Location start = lexer.current().location; - Lexeme matchBrace = lexer.current(); + MatchLexeme matchBrace = lexer.current(); expectAndConsume('{', "table literal"); while (lexer.current().type != '}') { if (lexer.current().type == '[') { - Lexeme matchLocationBracket = lexer.current(); + MatchLexeme matchLocationBracket = lexer.current(); nextLexeme(); AstExpr* key = parseExpr(); @@ -2692,6 +2645,63 @@ AstExpr* Parser::parseString() return reportExprError(location, {}, "String literal contains malformed escape sequence"); } +AstExpr* Parser::parseNumber() +{ + Location start = lexer.current().location; + + scratchData.assign(lexer.current().data, lexer.current().length); + + // Remove all internal _ - they don't hold any meaning and this allows parsing code to just pass the string pointer to strtod et al + if (scratchData.find('_') != std::string::npos) + { + scratchData.erase(std::remove(scratchData.begin(), scratchData.end(), '_'), scratchData.end()); + } + + if (FFlag::LuauLintParseIntegerIssues) + { + double value = 0; + ConstantNumberParseResult result = parseDouble(value, scratchData.c_str()); + nextLexeme(); + + if (result == ConstantNumberParseResult::Malformed) + return reportExprError(start, {}, "Malformed number"); + + return allocator.alloc(start, value, result); + } + else if (DFFlag::LuaReportParseIntegerIssues) + { + double value = 0; + if (const char* error = parseNumber_DEPRECATED2(value, scratchData.c_str())) + { + nextLexeme(); + + return reportExprError(start, {}, "%s", error); + } + else + { + nextLexeme(); + + return allocator.alloc(start, value); + } + } + else + { + double value = 0; + if (parseNumber_DEPRECATED(value, scratchData.c_str())) + { + nextLexeme(); + + return allocator.alloc(start, value); + } + else + { + nextLexeme(); + + return reportExprError(start, {}, "Malformed number"); + } + } +} + AstLocal* Parser::pushLocal(const Binding& binding) { const Name& name = binding.name; @@ -2763,7 +2773,7 @@ LUAU_NOINLINE void Parser::expectAndConsumeFail(Lexeme::Type type, const char* c report(lexer.current().location, "Expected %s, got %s", typeString.c_str(), currLexemeString.c_str()); } -bool Parser::expectMatchAndConsume(char value, const Lexeme& begin, bool searchForMissing) +bool Parser::expectMatchAndConsume(char value, const MatchLexeme& begin, bool searchForMissing) { Lexeme::Type type = static_cast(static_cast(value)); @@ -2771,42 +2781,7 @@ bool Parser::expectMatchAndConsume(char value, const Lexeme& begin, bool searchF { expectMatchAndConsumeFail(type, begin); - if (searchForMissing) - { - // previous location is taken because 'current' lexeme is already the next token - unsigned currentLine = lexer.previousLocation().end.line; - - // search to the end of the line for expected token - // we will also stop if we hit a token that can be handled by parsing function above the current one - Lexeme::Type lexemeType = lexer.current().type; - - while (currentLine == lexer.current().location.begin.line && lexemeType != type && matchRecoveryStopOnToken[lexemeType] == 0) - { - nextLexeme(); - lexemeType = lexer.current().type; - } - - if (lexemeType == type) - { - nextLexeme(); - - return true; - } - } - else - { - // check if this is an extra token and the expected token is next - if (lexer.lookahead().type == type) - { - // skip invalid and consume expected - nextLexeme(); - nextLexeme(); - - return true; - } - } - - return false; + return expectMatchAndConsumeRecover(value, begin, searchForMissing); } else { @@ -2816,21 +2791,64 @@ bool Parser::expectMatchAndConsume(char value, const Lexeme& begin, bool searchF } } -// LUAU_NOINLINE is used to limit the stack cost of this function due to std::string objects, and to increase caller performance since this code is -// cold -LUAU_NOINLINE void Parser::expectMatchAndConsumeFail(Lexeme::Type type, const Lexeme& begin, const char* extra) +LUAU_NOINLINE bool Parser::expectMatchAndConsumeRecover(char value, const MatchLexeme& begin, bool searchForMissing) { - std::string typeString = Lexeme(Location(Position(0, 0), 0), type).toString(); + Lexeme::Type type = static_cast(static_cast(value)); - if (lexer.current().location.begin.line == begin.location.begin.line) - report(lexer.current().location, "Expected %s (to close %s at column %d), got %s%s", typeString.c_str(), begin.toString().c_str(), - begin.location.begin.column + 1, lexer.current().toString().c_str(), extra ? extra : ""); + if (searchForMissing) + { + // previous location is taken because 'current' lexeme is already the next token + unsigned currentLine = lexer.previousLocation().end.line; + + // search to the end of the line for expected token + // we will also stop if we hit a token that can be handled by parsing function above the current one + Lexeme::Type lexemeType = lexer.current().type; + + while (currentLine == lexer.current().location.begin.line && lexemeType != type && matchRecoveryStopOnToken[lexemeType] == 0) + { + nextLexeme(); + lexemeType = lexer.current().type; + } + + if (lexemeType == type) + { + nextLexeme(); + + return true; + } + } else - report(lexer.current().location, "Expected %s (to close %s at line %d), got %s%s", typeString.c_str(), begin.toString().c_str(), - begin.location.begin.line + 1, lexer.current().toString().c_str(), extra ? extra : ""); + { + // check if this is an extra token and the expected token is next + if (lexer.lookahead().type == type) + { + // skip invalid and consume expected + nextLexeme(); + nextLexeme(); + + return true; + } + } + + return false; } -bool Parser::expectMatchEndAndConsume(Lexeme::Type type, const Lexeme& begin) +// LUAU_NOINLINE is used to limit the stack cost of this function due to std::string objects, and to increase caller performance since this code is +// cold +LUAU_NOINLINE void Parser::expectMatchAndConsumeFail(Lexeme::Type type, const MatchLexeme& begin, const char* extra) +{ + std::string typeString = Lexeme(Location(Position(0, 0), 0), type).toString(); + std::string matchString = Lexeme(Location(Position(0, 0), 0), begin.type).toString(); + + if (lexer.current().location.begin.line == begin.position.line) + report(lexer.current().location, "Expected %s (to close %s at column %d), got %s%s", typeString.c_str(), matchString.c_str(), + begin.position.column + 1, lexer.current().toString().c_str(), extra ? extra : ""); + else + report(lexer.current().location, "Expected %s (to close %s at line %d), got %s%s", typeString.c_str(), matchString.c_str(), + begin.position.line + 1, lexer.current().toString().c_str(), extra ? extra : ""); +} + +bool Parser::expectMatchEndAndConsume(Lexeme::Type type, const MatchLexeme& begin) { if (lexer.current().type != type) { @@ -2852,9 +2870,9 @@ bool Parser::expectMatchEndAndConsume(Lexeme::Type type, const Lexeme& begin) { // If the token matches on a different line and a different column, it suggests misleading indentation // This can be used to pinpoint the problem location for a possible future *actual* mismatch - if (lexer.current().location.begin.line != begin.location.begin.line && - lexer.current().location.begin.column != begin.location.begin.column && - endMismatchSuspect.location.begin.line < begin.location.begin.line) // Only replace the previous suspect with more recent suspects + if (lexer.current().location.begin.line != begin.position.line && + lexer.current().location.begin.column != begin.position.column && + endMismatchSuspect.position.line < begin.position.line) // Only replace the previous suspect with more recent suspects { endMismatchSuspect = begin; } @@ -2867,12 +2885,12 @@ bool Parser::expectMatchEndAndConsume(Lexeme::Type type, const Lexeme& begin) // LUAU_NOINLINE is used to limit the stack cost of this function due to std::string objects, and to increase caller performance since this code is // cold -LUAU_NOINLINE void Parser::expectMatchEndAndConsumeFail(Lexeme::Type type, const Lexeme& begin) +LUAU_NOINLINE void Parser::expectMatchEndAndConsumeFail(Lexeme::Type type, const MatchLexeme& begin) { - if (endMismatchSuspect.type != Lexeme::Eof && endMismatchSuspect.location.begin.line > begin.location.begin.line) + if (endMismatchSuspect.type != Lexeme::Eof && endMismatchSuspect.position.line > begin.position.line) { - std::string suggestion = - format("; did you forget to close %s at line %d?", endMismatchSuspect.toString().c_str(), endMismatchSuspect.location.begin.line + 1); + std::string matchString = Lexeme(Location(Position(0, 0), 0), endMismatchSuspect.type).toString(); + std::string suggestion = format("; did you forget to close %s at line %d?", matchString.c_str(), endMismatchSuspect.position.line + 1); expectMatchAndConsumeFail(type, begin, suggestion.c_str()); } diff --git a/Common/include/Luau/Bytecode.h b/Common/include/Luau/Bytecode.h index 1d6b18e5..d6660f05 100644 --- a/Common/include/Luau/Bytecode.h +++ b/Common/include/Luau/Bytecode.h @@ -515,6 +515,9 @@ enum LuauBuiltinFunction // rawlen LBF_RAWLEN, + + // bit32.extract(_, k, k) + LBF_BIT32_EXTRACTK, }; // Capture type, used in LOP_CAPTURE diff --git a/Compiler/src/BuiltinFolding.cpp b/Compiler/src/BuiltinFolding.cpp index e76da4e2..03b5918c 100644 --- a/Compiler/src/BuiltinFolding.cpp +++ b/Compiler/src/BuiltinFolding.cpp @@ -319,11 +319,12 @@ Constant foldBuiltin(int bfid, const Constant* args, size_t count) break; case LBF_BIT32_EXTRACT: - if (count == 3 && args[0].type == Constant::Type_Number && args[1].type == Constant::Type_Number && args[2].type == Constant::Type_Number) + if (count >= 2 && args[0].type == Constant::Type_Number && args[1].type == Constant::Type_Number && + (count == 2 || args[2].type == Constant::Type_Number)) { uint32_t u = bit32(args[0].valueNumber); int f = int(args[1].valueNumber); - int w = int(args[2].valueNumber); + int w = count == 2 ? 1 : int(args[2].valueNumber); if (f >= 0 && w > 0 && f + w <= 32) { @@ -356,13 +357,13 @@ Constant foldBuiltin(int bfid, const Constant* args, size_t count) break; case LBF_BIT32_REPLACE: - if (count == 4 && args[0].type == Constant::Type_Number && args[1].type == Constant::Type_Number && args[2].type == Constant::Type_Number && - args[3].type == Constant::Type_Number) + if (count >= 3 && args[0].type == Constant::Type_Number && args[1].type == Constant::Type_Number && args[2].type == Constant::Type_Number && + (count == 3 || args[3].type == Constant::Type_Number)) { uint32_t n = bit32(args[0].valueNumber); uint32_t v = bit32(args[1].valueNumber); int f = int(args[2].valueNumber); - int w = int(args[3].valueNumber); + int w = count == 3 ? 1 : int(args[3].valueNumber); if (f >= 0 && w > 0 && f + w <= 32) { diff --git a/Compiler/src/Compiler.cpp b/Compiler/src/Compiler.cpp index eb815225..bd8744cc 100644 --- a/Compiler/src/Compiler.cpp +++ b/Compiler/src/Compiler.cpp @@ -23,13 +23,12 @@ LUAU_FASTINTVARIABLE(LuauCompileInlineThreshold, 25) LUAU_FASTINTVARIABLE(LuauCompileInlineThresholdMaxBoost, 300) LUAU_FASTINTVARIABLE(LuauCompileInlineDepth, 5) -LUAU_FASTFLAGVARIABLE(LuauCompileNoIpairs, false) - -LUAU_FASTFLAGVARIABLE(LuauCompileFreeReassign, false) LUAU_FASTFLAGVARIABLE(LuauCompileXEQ, false) LUAU_FASTFLAGVARIABLE(LuauCompileOptimalAssignment, false) +LUAU_FASTFLAGVARIABLE(LuauCompileExtractK, false) + namespace Luau { @@ -403,18 +402,37 @@ struct Compiler } } - void compileExprFastcallN(AstExprCall* expr, uint8_t target, uint8_t targetCount, bool targetTop, bool multRet, uint8_t regs, int bfid) + void compileExprFastcallN(AstExprCall* expr, uint8_t target, uint8_t targetCount, bool targetTop, bool multRet, uint8_t regs, int bfid, int bfK = -1) { LUAU_ASSERT(!expr->self); - LUAU_ASSERT(expr->args.size <= 2); + LUAU_ASSERT(expr->args.size >= 1); + LUAU_ASSERT(expr->args.size <= 2 || (bfid == LBF_BIT32_EXTRACTK && expr->args.size == 3)); + LUAU_ASSERT(bfid == LBF_BIT32_EXTRACTK ? bfK >= 0 : bfK < 0); LuauOpcode opc = expr->args.size == 1 ? LOP_FASTCALL1 : LOP_FASTCALL2; - uint32_t args[2] = {}; + if (FFlag::LuauCompileExtractK) + { + opc = expr->args.size == 1 ? LOP_FASTCALL1 : (bfK >= 0 || isConstant(expr->args.data[1])) ? LOP_FASTCALL2K : LOP_FASTCALL2; + } + + uint32_t args[3] = {}; for (size_t i = 0; i < expr->args.size; ++i) { - if (i > 0) + if (FFlag::LuauCompileExtractK) + { + if (i > 0 && opc == LOP_FASTCALL2K) + { + int32_t cid = getConstantIndex(expr->args.data[i]); + if (cid < 0) + CompileError::raise(expr->location, "Exceeded constant limit; simplify the code to compile"); + + args[i] = cid; + continue; // TODO: remove this and change if below to else if + } + } + else if (i > 0) { if (int32_t cid = getConstantIndex(expr->args.data[i]); cid >= 0) { @@ -425,7 +443,9 @@ struct Compiler } if (int reg = getExprLocalReg(expr->args.data[i]); reg >= 0) + { args[i] = uint8_t(reg); + } else { args[i] = uint8_t(regs + 1 + i); @@ -437,21 +457,31 @@ struct Compiler bytecode.emitABC(opc, uint8_t(bfid), uint8_t(args[0]), 0); if (opc != LOP_FASTCALL1) - bytecode.emitAux(args[1]); + bytecode.emitAux(bfK >= 0 ? bfK : args[1]); // Set up a traditional Lua stack for the subsequent LOP_CALL. // Note, as with other instructions that immediately follow FASTCALL, these are normally not executed and are used as a fallback for // these FASTCALL variants. for (size_t i = 0; i < expr->args.size; ++i) { - if (i > 0 && opc == LOP_FASTCALL2K) + if (FFlag::LuauCompileExtractK) { - emitLoadK(uint8_t(regs + 1 + i), args[i]); - break; + if (i > 0 && opc == LOP_FASTCALL2K) + emitLoadK(uint8_t(regs + 1 + i), args[i]); + else if (args[i] != regs + 1 + i) + bytecode.emitABC(LOP_MOVE, uint8_t(regs + 1 + i), uint8_t(args[i]), 0); } + else + { + if (i > 0 && opc == LOP_FASTCALL2K) + { + emitLoadK(uint8_t(regs + 1 + i), args[i]); + break; + } - if (args[i] != regs + 1 + i) - bytecode.emitABC(LOP_MOVE, uint8_t(regs + 1 + i), uint8_t(args[i]), 0); + if (args[i] != regs + 1 + i) + bytecode.emitABC(LOP_MOVE, uint8_t(regs + 1 + i), uint8_t(args[i]), 0); + } } // note, these instructions are normally not executed and are used as a fallback for FASTCALL @@ -600,7 +630,7 @@ struct Compiler } else { - AstExprLocal* le = FFlag::LuauCompileFreeReassign ? getExprLocal(arg) : arg->as(); + AstExprLocal* le = getExprLocal(arg); Variable* lv = le ? variables.find(le->local) : nullptr; // if the argument is a local that isn't mutated, we will simply reuse the existing register @@ -723,6 +753,26 @@ struct Compiler bfid = -1; } + // Optimization: for bit32.extract with constant in-range f/w we compile using FASTCALL2K and a special builtin + if (FFlag::LuauCompileExtractK && bfid == LBF_BIT32_EXTRACT && expr->args.size == 3 && isConstant(expr->args.data[1]) && isConstant(expr->args.data[2])) + { + Constant fc = getConstant(expr->args.data[1]); + Constant wc = getConstant(expr->args.data[2]); + + int fi = fc.type == Constant::Type_Number ? int(fc.valueNumber) : -1; + int wi = wc.type == Constant::Type_Number ? int(wc.valueNumber) : -1; + + if (fi >= 0 && wi > 0 && fi + wi <= 32) + { + int fwp = fi | ((wi - 1) << 5); + int32_t cid = bytecode.addConstantNumber(fwp); + if (cid < 0) + CompileError::raise(expr->location, "Exceeded constant limit; simplify the code to compile"); + + return compileExprFastcallN(expr, target, targetCount, targetTop, multRet, regs, LBF_BIT32_EXTRACTK, cid); + } + } + // Optimization: for 1/2 argument fast calls use specialized opcodes if (bfid >= 0 && expr->args.size >= 1 && expr->args.size <= 2 && !isExprMultRet(expr->args.data[expr->args.size - 1])) return compileExprFastcallN(expr, target, targetCount, targetTop, multRet, regs, bfid); @@ -1218,7 +1268,7 @@ struct Compiler { // disambiguation: there's 4 cases (we only need truthy or falsy results based on onlyTruth) // onlyTruth = 1: a and b transforms to a ? b : dontcare - // onlyTruth = 1: a or b transforms to a ? a : a + // onlyTruth = 1: a or b transforms to a ? a : b // onlyTruth = 0: a and b transforms to !a ? a : b // onlyTruth = 0: a or b transforms to !a ? b : dontcare if (onlyTruth == (expr->op == AstExprBinary::And)) @@ -2576,7 +2626,7 @@ struct Compiler return; // Optimization: for 1-1 local assignments, we can reuse the register *if* neither local is mutated - if (FFlag::LuauCompileFreeReassign && options.optimizationLevel >= 1 && stat->vars.size == 1 && stat->values.size == 1) + if (options.optimizationLevel >= 1 && stat->vars.size == 1 && stat->values.size == 1) { if (AstExprLocal* re = getExprLocal(stat->values.data[0])) { @@ -2790,7 +2840,6 @@ struct Compiler LUAU_ASSERT(vars == regs + 3); LuauOpcode skipOp = LOP_FORGPREP; - LuauOpcode loopOp = LOP_FORGLOOP; // Optimization: when we iterate via pairs/ipairs, we generate special bytecode that optimizes the traversal using internal iteration index // These instructions dynamically check if generator is equal to next/inext and bail out @@ -2802,25 +2851,16 @@ struct Compiler Builtin builtin = getBuiltin(stat->values.data[0]->as()->func, globals, variables); if (builtin.isGlobal("ipairs")) // for .. in ipairs(t) - { skipOp = LOP_FORGPREP_INEXT; - loopOp = FFlag::LuauCompileNoIpairs ? LOP_FORGLOOP : LOP_FORGLOOP_INEXT; - } else if (builtin.isGlobal("pairs")) // for .. in pairs(t) - { skipOp = LOP_FORGPREP_NEXT; - loopOp = LOP_FORGLOOP; - } } else if (stat->values.size == 2) { Builtin builtin = getBuiltin(stat->values.data[0], globals, variables); if (builtin.isGlobal("next")) // for .. in next,t - { skipOp = LOP_FORGPREP_NEXT; - loopOp = LOP_FORGLOOP; - } } } @@ -2846,19 +2886,9 @@ struct Compiler size_t backLabel = bytecode.emitLabel(); - bytecode.emitAD(loopOp, regs, 0); - - if (FFlag::LuauCompileNoIpairs) - { - // TODO: remove loopOp as it's a constant now - LUAU_ASSERT(loopOp == LOP_FORGLOOP); - - // FORGLOOP uses aux to encode variable count and fast path flag for ipairs traversal in the high bit - bytecode.emitAux((skipOp == LOP_FORGPREP_INEXT ? 0x80000000 : 0) | uint32_t(stat->vars.size)); - } - // note: FORGLOOP needs variable count encoded in AUX field, other loop instructions assume a fixed variable count - else if (loopOp == LOP_FORGLOOP) - bytecode.emitAux(uint32_t(stat->vars.size)); + // FORGLOOP uses aux to encode variable count and fast path flag for ipairs traversal in the high bit + bytecode.emitAD(LOP_FORGLOOP, regs, 0); + bytecode.emitAux((skipOp == LOP_FORGPREP_INEXT ? 0x80000000 : 0) | uint32_t(stat->vars.size)); size_t endLabel = bytecode.emitLabel(); diff --git a/Sources.cmake b/Sources.cmake index 7a27684e..50770f92 100644 --- a/Sources.cmake +++ b/Sources.cmake @@ -66,6 +66,7 @@ target_sources(Luau.CodeGen PRIVATE # Luau.Analysis Sources target_sources(Luau.Analysis PRIVATE + Analysis/include/Luau/Anyification.h Analysis/include/Luau/ApplyTypeFunction.h Analysis/include/Luau/AstJsonEncoder.h Analysis/include/Luau/AstQuery.h @@ -115,6 +116,7 @@ target_sources(Luau.Analysis PRIVATE Analysis/include/Luau/Variant.h Analysis/include/Luau/VisitTypeVar.h + Analysis/src/Anyification.cpp Analysis/src/ApplyTypeFunction.cpp Analysis/src/AstJsonEncoder.cpp Analysis/src/AstQuery.cpp @@ -126,6 +128,7 @@ target_sources(Luau.Analysis PRIVATE Analysis/src/ConstraintGraphBuilder.cpp Analysis/src/ConstraintSolver.cpp Analysis/src/ConstraintSolverLogger.cpp + Analysis/src/EmbeddedBuiltinDefinitions.cpp Analysis/src/Error.cpp Analysis/src/Frontend.cpp Analysis/src/Instantiation.cpp @@ -155,7 +158,6 @@ target_sources(Luau.Analysis PRIVATE Analysis/src/TypeVar.cpp Analysis/src/Unifiable.cpp Analysis/src/Unifier.cpp - Analysis/src/EmbeddedBuiltinDefinitions.cpp ) # Luau.VM Sources diff --git a/VM/src/lapi.cpp b/VM/src/lapi.cpp index 77788e40..af97dc4a 100644 --- a/VM/src/lapi.cpp +++ b/VM/src/lapi.cpp @@ -1209,9 +1209,8 @@ void* lua_newuserdatadtor(lua_State* L, size_t sz, void (*dtor)(void*)) { luaC_checkGC(L); luaC_checkthreadsleep(L); - size_t as = sz + sizeof(dtor); - if (as < sizeof(dtor)) - as = SIZE_MAX; // Will cause a memory error in luaU_newudata. + // make sure sz + sizeof(dtor) doesn't overflow; luaU_newdata will reject SIZE_MAX correctly + size_t as = sz < SIZE_MAX - sizeof(dtor) ? sz + sizeof(dtor) : SIZE_MAX; Udata* u = luaU_newudata(L, as, UTAG_IDTOR); memcpy(&u->data + sz, &dtor, sizeof(dtor)); setuvalue(L, L->top, u); diff --git a/VM/src/lbuiltins.cpp b/VM/src/lbuiltins.cpp index e98660a7..422c82b1 100644 --- a/VM/src/lbuiltins.cpp +++ b/VM/src/lbuiltins.cpp @@ -15,6 +15,8 @@ #include #endif +LUAU_FASTFLAGVARIABLE(LuauFasterBit32NoWidth, false) + // luauF functions implement FASTCALL instruction that performs a direct execution of some builtin functions from the VM // The rule of thumb is that FASTCALL functions can not call user code, yield, fail, or reallocate stack. // If types of the arguments mismatch, luauF_* needs to return -1 and the execution will fall back to the usual call path @@ -600,24 +602,39 @@ static int luauF_btest(lua_State* L, StkId res, TValue* arg0, int nresults, StkI static int luauF_extract(lua_State* L, StkId res, TValue* arg0, int nresults, StkId args, int nparams) { - if (nparams >= 3 && nresults <= 1 && ttisnumber(arg0) && ttisnumber(args) && ttisnumber(args + 1)) + if (nparams >= (3 - FFlag::LuauFasterBit32NoWidth) && nresults <= 1 && ttisnumber(arg0) && ttisnumber(args)) { double a1 = nvalue(arg0); double a2 = nvalue(args); - double a3 = nvalue(args + 1); unsigned n; luai_num2unsigned(n, a1); int f = int(a2); - int w = int(a3); - if (f >= 0 && w > 0 && f + w <= 32) + if (nparams == 2) { - uint32_t m = ~(0xfffffffeu << (w - 1)); - uint32_t r = (n >> f) & m; + if (unsigned(f) < 32) + { + uint32_t m = 1; + uint32_t r = (n >> f) & m; - setnvalue(res, double(r)); - return 1; + setnvalue(res, double(r)); + return 1; + } + } + else if (ttisnumber(args + 1)) + { + double a3 = nvalue(args + 1); + int w = int(a3); + + if (f >= 0 && w > 0 && f + w <= 32) + { + uint32_t m = ~(0xfffffffeu << (w - 1)); + uint32_t r = (n >> f) & m; + + setnvalue(res, double(r)); + return 1; + } } } @@ -676,26 +693,41 @@ static int luauF_lshift(lua_State* L, StkId res, TValue* arg0, int nresults, Stk static int luauF_replace(lua_State* L, StkId res, TValue* arg0, int nresults, StkId args, int nparams) { - if (nparams >= 4 && nresults <= 1 && ttisnumber(arg0) && ttisnumber(args) && ttisnumber(args + 1) && ttisnumber(args + 2)) + if (nparams >= (4 - FFlag::LuauFasterBit32NoWidth) && nresults <= 1 && ttisnumber(arg0) && ttisnumber(args) && ttisnumber(args + 1)) { double a1 = nvalue(arg0); double a2 = nvalue(args); double a3 = nvalue(args + 1); - double a4 = nvalue(args + 2); unsigned n, v; luai_num2unsigned(n, a1); luai_num2unsigned(v, a2); int f = int(a3); - int w = int(a4); - if (f >= 0 && w > 0 && f + w <= 32) + if (nparams == 3) { - uint32_t m = ~(0xfffffffeu << (w - 1)); - uint32_t r = (n & ~(m << f)) | ((v & m) << f); + if (unsigned(f) < 32) + { + uint32_t m = 1; + uint32_t r = (n & ~(m << f)) | ((v & m) << f); - setnvalue(res, double(r)); - return 1; + setnvalue(res, double(r)); + return 1; + } + } + else if (ttisnumber(args + 2)) + { + double a4 = nvalue(args + 2); + int w = int(a4); + + if (f >= 0 && w > 0 && f + w <= 32) + { + uint32_t m = ~(0xfffffffeu << (w - 1)); + uint32_t r = (n & ~(m << f)) | ((v & m) << f); + + setnvalue(res, double(r)); + return 1; + } } } @@ -1138,6 +1170,31 @@ static int luauF_rawlen(lua_State* L, StkId res, TValue* arg0, int nresults, Stk return -1; } +static int luauF_extractk(lua_State* L, StkId res, TValue* arg0, int nresults, StkId args, int nparams) +{ + // args is known to contain a number constant with packed in-range f/w + if (nparams >= 2 && nresults <= 1 && ttisnumber(arg0)) + { + double a1 = nvalue(arg0); + double a2 = nvalue(args); + + unsigned n; + luai_num2unsigned(n, a1); + int fw = int(a2); + + int f = fw & 31; + int w1 = fw >> 5; + + uint32_t m = ~(0xfffffffeu << w1); + uint32_t r = (n >> f) & m; + + setnvalue(res, double(r)); + return 1; + } + + return -1; +} + luau_FastFunction luauF_table[256] = { NULL, luauF_assert, @@ -1211,4 +1268,6 @@ luau_FastFunction luauF_table[256] = { luauF_select, luauF_rawlen, + + luauF_extractk, }; diff --git a/VM/src/lfunc.cpp b/VM/src/lfunc.cpp index dfde6dcb..bd0f8264 100644 --- a/VM/src/lfunc.cpp +++ b/VM/src/lfunc.cpp @@ -175,8 +175,7 @@ void luaF_freeclosure(lua_State* L, Closure* c, lua_Page* page) const LocVar* luaF_getlocal(const Proto* f, int local_number, int pc) { - int i; - for (i = 0; i < f->sizelocvars; i++) + for (int i = 0; i < f->sizelocvars; i++) { if (pc >= f->locvars[i].startpc && pc < f->locvars[i].endpc) { // is variable active? @@ -185,5 +184,15 @@ const LocVar* luaF_getlocal(const Proto* f, int local_number, int pc) return &f->locvars[i]; } } + + return NULL; // not found +} + +const LocVar* luaF_findlocal(const Proto* f, int local_reg, int pc) +{ + for (int i = 0; i < f->sizelocvars; i++) + if (local_reg == f->locvars[i].reg && pc >= f->locvars[i].startpc && pc < f->locvars[i].endpc) + return &f->locvars[i]; + return NULL; // not found } diff --git a/VM/src/lfunc.h b/VM/src/lfunc.h index a260d00a..59ab5726 100644 --- a/VM/src/lfunc.h +++ b/VM/src/lfunc.h @@ -17,3 +17,4 @@ LUAI_FUNC void luaF_freeclosure(lua_State* L, Closure* c, struct lua_Page* page) LUAI_FUNC void luaF_unlinkupval(UpVal* uv); LUAI_FUNC void luaF_freeupval(lua_State* L, UpVal* uv, struct lua_Page* page); LUAI_FUNC const LocVar* luaF_getlocal(const Proto* func, int local_number, int pc); +LUAI_FUNC const LocVar* luaF_findlocal(const Proto* func, int local_reg, int pc); diff --git a/VM/src/lgcdebug.cpp b/VM/src/lgcdebug.cpp index bc997d44..b6204b1a 100644 --- a/VM/src/lgcdebug.cpp +++ b/VM/src/lgcdebug.cpp @@ -345,6 +345,9 @@ static void dumpclosure(FILE* f, Closure* cl) if (cl->isC) { + if (cl->c.debugname) + fprintf(f, ",\"name\":\"%s\"", cl->c.debugname + 0); + if (cl->nupvalues) { fprintf(f, ",\"upvalues\":["); @@ -354,6 +357,9 @@ static void dumpclosure(FILE* f, Closure* cl) } else { + if (cl->l.p->debugname) + fprintf(f, ",\"name\":\"%s\"", getstr(cl->l.p->debugname)); + fprintf(f, ",\"proto\":"); dumpref(f, obj2gco(cl->l.p)); if (cl->nupvalues) @@ -403,7 +409,7 @@ static void dumpthread(FILE* f, lua_State* th) fprintf(f, ",\"source\":\""); dumpstringdata(f, p->source->data, p->source->len); - fprintf(f, "\",\"line\":%d", p->abslineinfo ? p->abslineinfo[0] : 0); + fprintf(f, "\",\"line\":%d", p->linedefined); } if (th->top > th->stack) @@ -411,6 +417,55 @@ static void dumpthread(FILE* f, lua_State* th) fprintf(f, ",\"stack\":["); dumprefs(f, th->stack, th->top - th->stack); fprintf(f, "]"); + + CallInfo* ci = th->base_ci; + bool first = true; + + fprintf(f, ",\"stacknames\":["); + for (StkId v = th->stack; v < th->top; ++v) + { + if (!iscollectable(v)) + continue; + + while (ci < th->ci && v >= (ci + 1)->func) + ci++; + + if (!first) + fputc(',', f); + first = false; + + if (v == ci->func) + { + Closure* cl = ci_func(ci); + + if (cl->isC) + { + fprintf(f, "\"frame:%s\"", cl->c.debugname ? cl->c.debugname : "[C]"); + } + else + { + Proto* p = cl->l.p; + fprintf(f, "\"frame:"); + if (p->source) + dumpstringdata(f, p->source->data, p->source->len); + fprintf(f, ":%d:%s\"", p->linedefined, p->debugname ? getstr(p->debugname) : ""); + } + } + else if (isLua(ci)) + { + Proto* p = ci_func(ci)->l.p; + int pc = pcRel(ci->savedpc, p); + const LocVar* var = luaF_findlocal(p, int(v - ci->base), pc); + + if (var && var->varname) + fprintf(f, "\"%s\"", getstr(var->varname)); + else + fprintf(f, "null"); + } + else + fprintf(f, "null"); + } + fprintf(f, "]"); } fprintf(f, "}"); } diff --git a/tests/Autocomplete.test.cpp b/tests/Autocomplete.test.cpp index 0f17531b..25447dd8 100644 --- a/tests/Autocomplete.test.cpp +++ b/tests/Autocomplete.test.cpp @@ -3189,4 +3189,27 @@ a.@1 CHECK(ac.entryMap.count("y")); } +TEST_CASE_FIXTURE(ACFixture, "globals_are_order_independent") +{ + ScopedFastFlag sff("LuauAutocompleteFixGlobalOrder", true); + + check(R"( + local myLocal = 4 + function abc0() + local myInnerLocal = 1 +@1 + end + + function abc1() + local myInnerLocal = 1 + end + )"); + + auto ac = autocomplete('1'); + CHECK(ac.entryMap.count("myLocal")); + CHECK(ac.entryMap.count("myInnerLocal")); + CHECK(ac.entryMap.count("abc0")); + CHECK(ac.entryMap.count("abc1")); +} + TEST_SUITE_END(); diff --git a/tests/Compiler.test.cpp b/tests/Compiler.test.cpp index a4ce88b1..a2e748a8 100644 --- a/tests/Compiler.test.cpp +++ b/tests/Compiler.test.cpp @@ -261,8 +261,6 @@ L1: RETURN R0 0 TEST_CASE("ForBytecode") { - ScopedFastFlag sff("LuauCompileNoIpairs", true); - // basic for loop: variable directly refers to internal iteration index (R2) CHECK_EQ("\n" + compileFunction0("for i=1,5 do print(i) end"), R"( LOADN R2 1 @@ -349,8 +347,6 @@ RETURN R0 0 TEST_CASE("ForBytecodeBuiltin") { - ScopedFastFlag sff("LuauCompileNoIpairs", true); - // we generally recognize builtins like pairs/ipairs and emit special opcodes CHECK_EQ("\n" + compileFunction0("for k,v in ipairs({}) do end"), R"( GETIMPORT R0 1 @@ -2065,6 +2061,69 @@ TEST_CASE("RecursionParse") { CHECK_EQ(std::string(e.what()), "Exceeded allowed recursion depth; simplify your block to make the code compile"); } + + try + { + Luau::compileOrThrow(bcb, rep("a(", 1500) + "42" + rep(")", 1500)); + CHECK(!"Expected exception"); + } + catch (std::exception& e) + { + CHECK_EQ(std::string(e.what()), "Exceeded allowed recursion depth; simplify your expression to make the code compile"); + } + + try + { + Luau::compileOrThrow(bcb, "return " + rep("{", 1500) + "42" + rep("}", 1500)); + CHECK(!"Expected exception"); + } + catch (std::exception& e) + { + CHECK_EQ(std::string(e.what()), "Exceeded allowed recursion depth; simplify your expression to make the code compile"); + } + + try + { + Luau::compileOrThrow(bcb, rep("while true do ", 1500) + "print()" + rep(" end", 1500)); + CHECK(!"Expected exception"); + } + catch (std::exception& e) + { + CHECK_EQ(std::string(e.what()), "Exceeded allowed recursion depth; simplify your expression to make the code compile"); + } + + try + { + Luau::compileOrThrow(bcb, rep("for i=1,1 do ", 1500) + "print()" + rep(" end", 1500)); + CHECK(!"Expected exception"); + } + catch (std::exception& e) + { + CHECK_EQ(std::string(e.what()), "Exceeded allowed recursion depth; simplify your expression to make the code compile"); + } + +#if 0 + // This currently requires too much stack space on MSVC/x64 and crashes with stack overflow at recursion depth 935 + try + { + Luau::compileOrThrow(bcb, rep("function a() ", 1500) + "print()" + rep(" end", 1500)); + CHECK(!"Expected exception"); + } + catch (std::exception& e) + { + CHECK_EQ(std::string(e.what()), "Exceeded allowed recursion depth; simplify your block to make the code compile"); + } + + try + { + Luau::compileOrThrow(bcb, "return " + rep("function() return ", 1500) + "42" + rep(" end", 1500)); + CHECK(!"Expected exception"); + } + catch (std::exception& e) + { + CHECK_EQ(std::string(e.what()), "Exceeded allowed recursion depth; simplify your block to make the code compile"); + } +#endif } TEST_CASE("ArrayIndexLiteral") @@ -2111,8 +2170,6 @@ L1: RETURN R3 -1 TEST_CASE("UpvaluesLoopsBytecode") { - ScopedFastFlag sff("LuauCompileNoIpairs", true); - CHECK_EQ("\n" + compileFunction(R"( function test() for i=1,10 do @@ -3790,8 +3847,6 @@ RETURN R0 1 TEST_CASE("SharedClosure") { - ScopedFastFlag sff("LuauCompileFreeReassign", true); - // closures can be shared even if functions refer to upvalues, as long as upvalues are top-level CHECK_EQ("\n" + compileFunction(R"( local val = ... @@ -6004,6 +6059,8 @@ return math.clamp(-1, 0, 1), math.sign(77), math.round(7.6), + bit32.extract(-1, 31), + bit32.replace(100, 1, 0), (type("fin")) )", 0, 2), @@ -6055,8 +6112,10 @@ LOADK R43 K2 LOADN R44 0 LOADN R45 1 LOADN R46 8 -LOADK R47 K3 -RETURN R0 48 +LOADN R47 1 +LOADN R48 101 +LOADK R49 K3 +RETURN R0 50 )"); } @@ -6067,7 +6126,8 @@ return math.abs(), math.max(1, true), string.byte("abc", 42), - bit32.rshift(10, 42) + bit32.rshift(10, 42), + bit32.extract(1, 2, "3") )", 0, 2), R"( @@ -6088,8 +6148,14 @@ L2: LOADN R4 10 FASTCALL2K 39 R4 K7 L3 LOADK R5 K7 GETIMPORT R3 13 -CALL R3 2 -1 -L3: RETURN R0 -1 +CALL R3 2 1 +L3: LOADN R5 1 +LOADN R6 2 +LOADK R7 K14 +FASTCALL 34 L4 +GETIMPORT R4 16 +CALL R4 3 -1 +L4: RETURN R0 -1 )"); } @@ -6146,8 +6212,6 @@ RETURN R0 1 TEST_CASE("LocalReassign") { - ScopedFastFlag sff("LuauCompileFreeReassign", true); - // locals can be re-assigned and the register gets reused CHECK_EQ("\n" + compileFunction0(R"( local function test(a, b) @@ -6459,4 +6523,26 @@ RETURN R0 0 )"); } +TEST_CASE("BuiltinExtractK") +{ + ScopedFastFlag sff("LuauCompileExtractK", true); + + // below, K0 refers to a packed f+w constant for bit32.extractk builtin + // K1 and K2 refer to 1 and 3 and are only used during fallback path + CHECK_EQ("\n" + compileFunction0(R"( +local v = ... + +return bit32.extract(v, 1, 3) +)"), R"( +GETVARARGS R0 1 +FASTCALL2K 59 R0 K0 L0 +MOVE R2 R0 +LOADK R3 K1 +LOADK R4 K2 +GETIMPORT R1 5 +CALL R1 3 -1 +L0: RETURN R1 -1 +)"); +} + TEST_SUITE_END(); diff --git a/tests/Conformance.test.cpp b/tests/Conformance.test.cpp index 17c7ac58..be2feacb 100644 --- a/tests/Conformance.test.cpp +++ b/tests/Conformance.test.cpp @@ -727,7 +727,7 @@ TEST_CASE("NewUserdataOverflow") // The overflow might segfault in the following call. lua_getmetatable(L1, -1); return 0; - }, "PCall"); + }, nullptr); CHECK(lua_pcall(L, 0, 0, 0) == LUA_ERRRUN); CHECK(strcmp(lua_tostring(L, -1), "memory allocation error: block too big") == 0); diff --git a/tests/Fixture.cpp b/tests/Fixture.cpp index f0146602..40be39a0 100644 --- a/tests/Fixture.cpp +++ b/tests/Fixture.cpp @@ -443,7 +443,8 @@ BuiltinsFixture::BuiltinsFixture(bool freeze, bool prepareAutocomplete) ConstraintGraphBuilderFixture::ConstraintGraphBuilderFixture() : Fixture() - , cgb(mainModuleName, getMainModule(), &arena, NotNull(&ice), frontend.getGlobalScope()) + , mainModule(new Module) + , cgb(mainModuleName, mainModule, &arena, NotNull(&ice), frontend.getGlobalScope()) , forceTheFlag{"DebugLuauDeferredConstraintResolution", true} { BlockedTypeVar::nextIndex = 0; diff --git a/tests/Fixture.h b/tests/Fixture.h index a716fe9b..8dc3dd24 100644 --- a/tests/Fixture.h +++ b/tests/Fixture.h @@ -162,6 +162,7 @@ struct BuiltinsFixture : Fixture struct ConstraintGraphBuilderFixture : Fixture { TypeArena arena; + ModulePtr mainModule; ConstraintGraphBuilder cgb; ScopedFastFlag forceTheFlag; diff --git a/tests/Normalize.test.cpp b/tests/Normalize.test.cpp index c64c41c5..b017d8dd 100644 --- a/tests/Normalize.test.cpp +++ b/tests/Normalize.test.cpp @@ -12,6 +12,11 @@ using namespace Luau; struct NormalizeFixture : Fixture { ScopedFastFlag sff1{"LuauLowerBoundsCalculation", true}; + + bool isSubtype(TypeId a, TypeId b) + { + return ::Luau::isSubtype(a, b, NotNull{getMainModule()->getModuleScope().get()}, ice); + } }; void createSomeClasses(TypeChecker& typeChecker) @@ -49,12 +54,6 @@ void createSomeClasses(TypeChecker& typeChecker) freeze(arena); } -static bool isSubtype(TypeId a, TypeId b) -{ - InternalErrorReporter ice; - return isSubtype(a, b, ice); -} - TEST_SUITE_BEGIN("isSubtype"); TEST_CASE_FIXTURE(NormalizeFixture, "primitives") @@ -511,6 +510,8 @@ TEST_CASE_FIXTURE(NormalizeFixture, "classes") { createSomeClasses(typeChecker); + check(""); // Ensure that we have a main Module. + TypeId p = typeChecker.globalScope->lookupType("Parent")->type; TypeId c = typeChecker.globalScope->lookupType("Child")->type; TypeId u = typeChecker.globalScope->lookupType("Unrelated")->type; @@ -595,6 +596,7 @@ TEST_CASE_FIXTURE(NormalizeFixture, "union_with_overlapping_field_that_has_a_sub )"); ModulePtr tempModule{new Module}; + tempModule->scopes.emplace_back(Location(), std::make_shared(getSingletonTypes().anyTypePack)); // HACK: Normalization is an in-place operation. We need to cheat a little here and unfreeze // the arena that the type lives in. @@ -880,7 +882,6 @@ TEST_CASE_FIXTURE(Fixture, "intersection_inside_a_table_inside_another_intersect { ScopedFastFlag flags[] = { {"LuauLowerBoundsCalculation", true}, - {"LuauQuantifyConstrained", true}, }; // We use a function and inferred parameter types to prevent intermediate normalizations from being performed. @@ -921,7 +922,6 @@ TEST_CASE_FIXTURE(Fixture, "intersection_inside_a_table_inside_another_intersect { ScopedFastFlag flags[] = { {"LuauLowerBoundsCalculation", true}, - {"LuauQuantifyConstrained", true}, }; // We use a function and inferred parameter types to prevent intermediate normalizations from being performed. @@ -961,7 +961,6 @@ TEST_CASE_FIXTURE(Fixture, "intersection_inside_a_table_inside_another_intersect { ScopedFastFlag flags[] = { {"LuauLowerBoundsCalculation", true}, - {"LuauQuantifyConstrained", true}, }; // We use a function and inferred parameter types to prevent intermediate normalizations from being performed. @@ -1149,7 +1148,6 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "normalization_does_not_convert_ever") { ScopedFastFlag sff[]{ {"LuauLowerBoundsCalculation", true}, - {"LuauQuantifyConstrained", true}, }; CheckResult result = check(R"( diff --git a/tests/Parser.test.cpp b/tests/Parser.test.cpp index 5b86807c..c55ec181 100644 --- a/tests/Parser.test.cpp +++ b/tests/Parser.test.cpp @@ -2535,7 +2535,6 @@ end TEST_CASE_FIXTURE(Fixture, "error_message_for_using_function_as_type_annotation") { - ScopedFastFlag sff{"LuauParserFunctionKeywordAsTypeHelp", true}; ParseResult result = tryParse(R"( type Foo = function )"); diff --git a/tests/TypeInfer.functions.test.cpp b/tests/TypeInfer.functions.test.cpp index 074c86c3..5cc759d6 100644 --- a/tests/TypeInfer.functions.test.cpp +++ b/tests/TypeInfer.functions.test.cpp @@ -1637,7 +1637,6 @@ TEST_CASE_FIXTURE(Fixture, "quantify_constrained_types") { ScopedFastFlag sff[]{ {"LuauLowerBoundsCalculation", true}, - {"LuauQuantifyConstrained", true}, }; CheckResult result = check(R"( @@ -1662,7 +1661,6 @@ TEST_CASE_FIXTURE(Fixture, "call_o_with_another_argument_after_foo_was_quantifie { ScopedFastFlag sff[]{ {"LuauLowerBoundsCalculation", true}, - {"LuauQuantifyConstrained", true}, }; CheckResult result = check(R"( diff --git a/tests/TypeInfer.modules.test.cpp b/tests/TypeInfer.modules.test.cpp index 754fb193..b5f2296c 100644 --- a/tests/TypeInfer.modules.test.cpp +++ b/tests/TypeInfer.modules.test.cpp @@ -329,6 +329,35 @@ function tbl:foo(b: number, c: number) -- introduce BoundTypeVar to imported type arrayops.foo(self._regions) end +-- this alias decreases function type level and causes a demotion of its type +type Table = typeof(tbl) +)"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(BuiltinsFixture, "do_not_modify_imported_types_5") +{ + ScopedFastFlag luauInplaceDemoteSkipAllBound{"LuauInplaceDemoteSkipAllBound", true}; + + fileResolver.source["game/A"] = R"( +export type Type = {x: number, y: number} +local arrayops = {} +function arrayops.foo(x: Type) end +return arrayops + )"; + + CheckResult result = check(R"( +local arrayops = require(game.A) + +local tbl = {} +tbl.a = 2 +function tbl:foo(b: number, c: number) + -- introduce boundTo TableTypeVar to imported type + self.x.a = 2 + arrayops.foo(self.x) +end +-- this alias decreases function type level and causes a demotion of its type type Table = typeof(tbl) )"); diff --git a/tests/TypeInfer.provisional.test.cpp b/tests/TypeInfer.provisional.test.cpp index 01923f38..9a917a6f 100644 --- a/tests/TypeInfer.provisional.test.cpp +++ b/tests/TypeInfer.provisional.test.cpp @@ -485,7 +485,6 @@ TEST_CASE_FIXTURE(Fixture, "constrained_is_level_dependent") { ScopedFastFlag sff[]{ {"LuauLowerBoundsCalculation", true}, - {"LuauQuantifyConstrained", true}, }; CheckResult result = check(R"( diff --git a/tests/TypeInfer.refinements.test.cpp b/tests/TypeInfer.refinements.test.cpp index 8a1cadcf..e61e6e45 100644 --- a/tests/TypeInfer.refinements.test.cpp +++ b/tests/TypeInfer.refinements.test.cpp @@ -41,6 +41,7 @@ struct RefinementClassFixture : Fixture RefinementClassFixture() { TypeArena& arena = typeChecker.globalTypes; + NotNull scope{typeChecker.globalScope.get()}; unfreeze(arena); TypeId vec3 = arena.addType(ClassTypeVar{"Vector3", {}, std::nullopt, std::nullopt, {}, nullptr, "Test"}); @@ -49,7 +50,7 @@ struct RefinementClassFixture : Fixture {"Y", Property{typeChecker.numberType}}, {"Z", Property{typeChecker.numberType}}, }; - normalize(vec3, arena, *typeChecker.iceHandler); + normalize(vec3, scope, arena, *typeChecker.iceHandler); TypeId inst = arena.addType(ClassTypeVar{"Instance", {}, std::nullopt, std::nullopt, {}, nullptr, "Test"}); @@ -57,21 +58,21 @@ struct RefinementClassFixture : Fixture TypePackId isARets = arena.addTypePack({typeChecker.booleanType}); TypeId isA = arena.addType(FunctionTypeVar{isAParams, isARets}); getMutable(isA)->magicFunction = magicFunctionInstanceIsA; - normalize(isA, arena, *typeChecker.iceHandler); + normalize(isA, scope, arena, *typeChecker.iceHandler); getMutable(inst)->props = { {"Name", Property{typeChecker.stringType}}, {"IsA", Property{isA}}, }; - normalize(inst, arena, *typeChecker.iceHandler); + normalize(inst, scope, arena, *typeChecker.iceHandler); TypeId folder = typeChecker.globalTypes.addType(ClassTypeVar{"Folder", {}, inst, std::nullopt, {}, nullptr, "Test"}); - normalize(folder, arena, *typeChecker.iceHandler); + normalize(folder, scope, arena, *typeChecker.iceHandler); TypeId part = typeChecker.globalTypes.addType(ClassTypeVar{"Part", {}, inst, std::nullopt, {}, nullptr, "Test"}); getMutable(part)->props = { {"Position", Property{vec3}}, }; - normalize(part, arena, *typeChecker.iceHandler); + normalize(part, scope, arena, *typeChecker.iceHandler); typeChecker.globalScope->exportedTypeBindings["Vector3"] = TypeFun{{}, vec3}; typeChecker.globalScope->exportedTypeBindings["Instance"] = TypeFun{{}, inst}; @@ -934,8 +935,6 @@ TEST_CASE_FIXTURE(Fixture, "apply_refinements_on_astexprindexexpr_whose_subscrip TEST_CASE_FIXTURE(Fixture, "discriminate_from_truthiness_of_x") { - ScopedFastFlag sff{"LuauFalsyPredicateReturnsNilInstead", true}; - CheckResult result = check(R"( type T = {tag: "missing", x: nil} | {tag: "exists", x: string} @@ -1230,8 +1229,6 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "refine_unknowns") TEST_CASE_FIXTURE(BuiltinsFixture, "falsiness_of_TruthyPredicate_narrows_into_nil") { - ScopedFastFlag sff{"LuauFalsyPredicateReturnsNilInstead", true}; - CheckResult result = check(R"( local function f(t: {number}) local x = t[1] diff --git a/tests/TypeInfer.tables.test.cpp b/tests/TypeInfer.tables.test.cpp index 8c7d8a53..95b85c48 100644 --- a/tests/TypeInfer.tables.test.cpp +++ b/tests/TypeInfer.tables.test.cpp @@ -3003,8 +3003,6 @@ TEST_CASE_FIXTURE(Fixture, "expected_indexer_from_table_union") TEST_CASE_FIXTURE(Fixture, "prop_access_on_key_whose_types_mismatches") { - ScopedFastFlag sff{"LuauReportErrorsOnIndexerKeyMismatch", true}; - CheckResult result = check(R"( local t: {number} = {} local x = t.x @@ -3016,8 +3014,6 @@ TEST_CASE_FIXTURE(Fixture, "prop_access_on_key_whose_types_mismatches") TEST_CASE_FIXTURE(Fixture, "prop_access_on_unions_of_indexers_where_key_whose_types_mismatches") { - ScopedFastFlag sff{"LuauReportErrorsOnIndexerKeyMismatch", true}; - CheckResult result = check(R"( local t: { [number]: number } | { [boolean]: number } = {} local u = t.x diff --git a/tests/TypeInfer.tryUnify.test.cpp b/tests/TypeInfer.tryUnify.test.cpp index e0a0e5b5..02fdfd73 100644 --- a/tests/TypeInfer.tryUnify.test.cpp +++ b/tests/TypeInfer.tryUnify.test.cpp @@ -17,7 +17,8 @@ struct TryUnifyFixture : Fixture ScopePtr globalScope{new Scope{arena.addTypePack({TypeId{}})}}; InternalErrorReporter iceHandler; UnifierSharedState unifierState{&iceHandler}; - Unifier state{&arena, Mode::Strict, Location{}, Variance::Covariant, unifierState}; + + Unifier state{&arena, Mode::Strict, NotNull{globalScope.get()}, Location{}, Variance::Covariant, unifierState}; }; TEST_SUITE_BEGIN("TryUnifyTests"); diff --git a/tests/TypeVar.test.cpp b/tests/TypeVar.test.cpp index edec8442..32be8215 100644 --- a/tests/TypeVar.test.cpp +++ b/tests/TypeVar.test.cpp @@ -193,6 +193,11 @@ TEST_CASE_FIXTURE(Fixture, "UnionTypeVarIterator_with_only_cyclic_union") CHECK(actual.empty()); } + +/* FIXME: This test is pretty weird. It would be much nicer if we could + * perform this operation without a TypeChecker so that we don't have to jam + * all this state into it to make stuff work. + */ TEST_CASE_FIXTURE(Fixture, "substitution_skip_failure") { TypeVar ftv11{FreeTypeVar{TypeLevel{}}}; @@ -268,6 +273,7 @@ TEST_CASE_FIXTURE(Fixture, "substitution_skip_failure") TypeId root = &ttvTweenResult; typeChecker.currentModule = std::make_shared(); + typeChecker.currentModule->scopes.emplace_back(Location{}, std::make_shared(getSingletonTypes().anyTypePack)); TypeId result = typeChecker.anyify(typeChecker.globalScope, root, Location{}); diff --git a/tests/conformance/bitwise.lua b/tests/conformance/bitwise.lua index 13be3f94..f0c5698d 100644 --- a/tests/conformance/bitwise.lua +++ b/tests/conformance/bitwise.lua @@ -100,6 +100,9 @@ assert(bit32.extract(0xa0001111, 28, 4) == 0xa) assert(bit32.extract(0xa0001111, 31, 1) == 1) assert(bit32.extract(0x50000111, 31, 1) == 0) assert(bit32.extract(0xf2345679, 0, 32) == 0xf2345679) +assert(bit32.extract(0xa0001111, 16) == 0) +assert(bit32.extract(0xa0001111, 31) == 1) +assert(bit32.extract(42, 1, 3) == 5) assert(not pcall(bit32.extract, 0, -1)) assert(not pcall(bit32.extract, 0, 32)) @@ -152,5 +155,6 @@ assert(bit32.btest(1, "3") == true) assert(bit32.btest("1", 3) == true) assert(bit32.countlz("42") == 26) assert(bit32.countrz("42") == 1) +assert(bit32.extract("42", 1, 3) == 5) return('OK') diff --git a/tools/faillist.txt b/tools/faillist.txt index d796236d..630bf9f7 100644 --- a/tools/faillist.txt +++ b/tools/faillist.txt @@ -10,12 +10,10 @@ AnnotationTests.luau_ice_triggers_an_ice_exception_with_flag AnnotationTests.luau_ice_triggers_an_ice_exception_with_flag_handler AnnotationTests.luau_ice_triggers_an_ice_handler AnnotationTests.luau_print_is_magic_if_the_flag_is_set -AnnotationTests.luau_print_is_not_special_without_the_flag AnnotationTests.occurs_check_on_cyclic_intersection_typevar AnnotationTests.occurs_check_on_cyclic_union_typevar AnnotationTests.too_many_type_params AnnotationTests.two_type_params -AnnotationTests.unknown_type_reference_generates_error AnnotationTests.use_type_required_from_another_file AstQuery.last_argument_function_call_type AstQuery::getDocumentationSymbolAtPosition.overloaded_fn @@ -27,17 +25,13 @@ AutocompleteTest.autocomplete_end_with_lambda AutocompleteTest.autocomplete_first_function_arg_expected_type AutocompleteTest.autocomplete_for_in_middle_keywords AutocompleteTest.autocomplete_for_middle_keywords -AutocompleteTest.autocomplete_if_else_regression AutocompleteTest.autocomplete_if_middle_keywords -AutocompleteTest.autocomplete_ifelse_expressions AutocompleteTest.autocomplete_on_string_singletons AutocompleteTest.autocomplete_oop_implicit_self AutocompleteTest.autocomplete_repeat_middle_keyword AutocompleteTest.autocomplete_string_singleton_equality AutocompleteTest.autocomplete_string_singleton_escape AutocompleteTest.autocomplete_string_singletons -AutocompleteTest.autocomplete_until_expression -AutocompleteTest.autocomplete_until_in_repeat AutocompleteTest.autocomplete_while_middle_keywords AutocompleteTest.autocompleteProp_index_function_metamethod_is_variadic AutocompleteTest.bias_toward_inner_scope @@ -60,7 +54,6 @@ AutocompleteTest.get_suggestions_for_the_very_start_of_the_script AutocompleteTest.global_function_params AutocompleteTest.global_functions_are_not_scoped_lexically AutocompleteTest.if_then_else_elseif_completions -AutocompleteTest.if_then_else_full_keywords AutocompleteTest.keyword_methods AutocompleteTest.keyword_types AutocompleteTest.library_non_self_calls_are_fine @@ -181,7 +174,6 @@ DefinitionTests.single_class_type_identity_in_global_types FrontendTest.ast_node_at_position FrontendTest.automatically_check_dependent_scripts FrontendTest.check_without_builtin_next -FrontendTest.clearStats FrontendTest.dont_reparse_clean_file_when_linting FrontendTest.environments FrontendTest.imported_table_modification_2 @@ -195,7 +187,6 @@ FrontendTest.reexport_cyclic_type FrontendTest.reexport_type_alias FrontendTest.report_require_to_nonexistent_file FrontendTest.report_syntax_error_in_required_file -FrontendTest.stats_are_not_reset_between_checks FrontendTest.trace_requires_in_nonstrict_mode GenericsTests.apply_type_function_nested_generics1 GenericsTests.apply_type_function_nested_generics2 @@ -213,8 +204,6 @@ GenericsTests.duplicate_generic_types GenericsTests.error_detailed_function_mismatch_generic_pack GenericsTests.error_detailed_function_mismatch_generic_types GenericsTests.factories_of_generics -GenericsTests.function_arguments_can_be_polytypes -GenericsTests.function_results_can_be_polytypes GenericsTests.generic_argument_count_too_few GenericsTests.generic_argument_count_too_many GenericsTests.generic_factories @@ -243,7 +232,6 @@ GenericsTests.properties_can_be_instantiated_polytypes GenericsTests.rank_N_types_via_typeof GenericsTests.reject_clashing_generic_and_pack_names GenericsTests.self_recursive_instantiated_param -GenericsTests.variadic_generics IntersectionTypes.argument_is_intersection IntersectionTypes.error_detailed_intersection_all IntersectionTypes.error_detailed_intersection_part @@ -289,7 +277,6 @@ Normalize.cyclic_intersection Normalize.cyclic_table_normalizes_sensibly Normalize.cyclic_union Normalize.fuzz_failure_bound_type_is_normal_but_not_its_bounded_to -Normalize.higher_order_function Normalize.intersection_combine_on_bound_self Normalize.intersection_inside_a_table_inside_another_intersection Normalize.intersection_inside_a_table_inside_another_intersection_2 @@ -304,7 +291,6 @@ Normalize.normalization_does_not_convert_ever Normalize.normalize_module_return_type Normalize.normalize_unions_containing_never Normalize.normalize_unions_containing_unknown -Normalize.return_type_is_not_a_constrained_intersection Normalize.union_of_distinct_free_types Normalize.variadic_tail_is_marked_normal Normalize.visiting_a_type_twice_is_not_considered_normal @@ -456,7 +442,6 @@ TableTests.inferred_return_type_of_free_table TableTests.inferring_crazy_table_should_also_be_quick TableTests.instantiate_table_cloning_3 TableTests.instantiate_tables_at_scope_level -TableTests.invariant_table_properties_means_instantiating_tables_in_assignment_is_unsound TableTests.leaking_bad_metatable_errors TableTests.length_operator_intersection TableTests.length_operator_non_table_union @@ -521,7 +506,6 @@ ToDot.function ToDot.metatable ToDot.table ToString.exhaustive_toString_of_cyclic_table -ToString.function_type_with_argument_names_and_self ToString.function_type_with_argument_names_generic ToString.no_parentheses_around_cyclic_function_type_in_union ToString.toStringDetailed2 @@ -565,10 +549,7 @@ TypeInfer.do_not_bind_a_free_table_to_a_union_containing_that_table TypeInfer.dont_report_type_errors_within_an_AstStatError TypeInfer.globals TypeInfer.globals2 -TypeInfer.infer_assignment_value_types TypeInfer.infer_assignment_value_types_mutable_lval -TypeInfer.infer_through_group_expr -TypeInfer.no_heap_use_after_free_error TypeInfer.no_stack_overflow_from_isoptional TypeInfer.tc_after_error_recovery_no_replacement_name_in_error TypeInfer.tc_if_else_expressions1 @@ -589,11 +570,8 @@ TypeInferAnyError.for_in_loop_iterator_returns_any TypeInferAnyError.for_in_loop_iterator_returns_any2 TypeInferAnyError.length_of_error_type_does_not_produce_an_error TypeInferAnyError.replace_every_free_type_when_unifying_a_complex_function_with_any -TypeInferAnyError.type_error_addition TypeInferClasses.call_base_method TypeInferClasses.call_instance_method -TypeInferClasses.can_read_prop_of_base_class -TypeInferClasses.can_read_prop_of_base_class_using_string TypeInferClasses.class_type_mismatch_with_name_conflict TypeInferClasses.classes_can_have_overloaded_operators TypeInferClasses.classes_without_overloaded_operators_cannot_be_added @@ -609,8 +587,6 @@ TypeInferFunctions.another_indirect_function_case_where_it_is_ok_to_provide_too_ TypeInferFunctions.another_recursive_local_function TypeInferFunctions.calling_function_with_anytypepack_doesnt_leak_free_types TypeInferFunctions.calling_function_with_incorrect_argument_type_yields_errors_spanning_argument -TypeInferFunctions.cannot_hoist_interior_defns_into_signature -TypeInferFunctions.check_function_before_lambda_that_uses_it TypeInferFunctions.complicated_return_types_require_an_explicit_annotation TypeInferFunctions.cyclic_function_type_in_args TypeInferFunctions.dont_give_other_overloads_message_if_only_one_argument_matching_overload_exists @@ -635,7 +611,6 @@ TypeInferFunctions.ignored_return_values TypeInferFunctions.inconsistent_higher_order_function TypeInferFunctions.inconsistent_return_types TypeInferFunctions.infer_anonymous_function_arguments -TypeInferFunctions.infer_anonymous_function_arguments_outside_call TypeInferFunctions.infer_return_type_from_selected_overload TypeInferFunctions.infer_that_function_does_not_return_a_table TypeInferFunctions.it_is_ok_not_to_supply_enough_retvals @@ -680,9 +655,6 @@ TypeInferLoops.loop_iter_no_indexer_strict TypeInferLoops.loop_iter_trailing_nil TypeInferLoops.loop_typecheck_crash_on_empty_optional TypeInferLoops.properly_infer_iteratee_is_a_free_table -TypeInferLoops.repeat_loop -TypeInferLoops.repeat_loop_condition_binds_to_its_block -TypeInferLoops.symbols_in_repeat_block_should_not_be_visible_beyond_until_condition TypeInferLoops.unreachable_code_after_infinite_loop TypeInferLoops.varlist_declared_by_for_in_loop_should_be_free TypeInferModules.do_not_modify_imported_types @@ -718,8 +690,6 @@ TypeInferOperators.cannot_indirectly_compare_types_that_do_not_offer_overloaded_ TypeInferOperators.cli_38355_recursive_union TypeInferOperators.compare_numbers TypeInferOperators.compare_strings -TypeInferOperators.compound_assign_basic -TypeInferOperators.compound_assign_metatable TypeInferOperators.compound_assign_mismatch_metatable TypeInferOperators.compound_assign_mismatch_op TypeInferOperators.compound_assign_mismatch_result @@ -775,7 +745,6 @@ TypeInferUnknownNever.type_packs_containing_never_is_itself_uninhabitable TypeInferUnknownNever.type_packs_containing_never_is_itself_uninhabitable2 TypeInferUnknownNever.unary_minus_of_never TypeInferUnknownNever.unknown_is_reflexive -TypePackTests.cyclic_type_packs TypePackTests.higher_order_function TypePackTests.multiple_varargs_inference_are_not_confused TypePackTests.no_return_size_should_be_zero diff --git a/tools/heapgraph.py b/tools/heapgraph.py index 2817c38f..b8dc207f 100644 --- a/tools/heapgraph.py +++ b/tools/heapgraph.py @@ -132,8 +132,16 @@ while offset < len(queue): queue.append((obj["metatable"], node.child("__meta"))) elif obj["type"] == "thread": queue.append((obj["env"], node.child("__env"))) - for a in obj.get("stack", []): - queue.append((a, node.child("__stack"))) + stack = obj.get("stack") + stacknames = obj.get("stacknames", []) + stacknode = node.child("__stack") + framenode = None + for i in range(len(stack)): + name = stacknames[i] if stacknames else None + if name and name.startswith("frame:"): + framenode = stacknode.child(name[6:]) + name = None + queue.append((stack[i], framenode.child(name) if framenode and name else framenode or stacknode)) elif obj["type"] == "proto": for a in obj.get("constants", []): queue.append((a, node)) diff --git a/tools/heapstat.py b/tools/heapstat.py index 4c0cb407..7337aa44 100644 --- a/tools/heapstat.py +++ b/tools/heapstat.py @@ -59,5 +59,6 @@ if len(size_category) != 0: print("objects by category:") for type, (count, size) in sortedsize(size_category.items()): - name = dump["stats"]["categories"][type]["name"] + cat = dump["stats"]["categories"][type] + name = cat["name"] if "name" in cat else str(type) print(name.ljust(30), str(size).rjust(8), "bytes", str(count).rjust(5), "objects") diff --git a/tools/perfgraph.py b/tools/perfgraph.py index ae74b7d6..25bb0dd9 100644 --- a/tools/perfgraph.py +++ b/tools/perfgraph.py @@ -56,18 +56,28 @@ def nodeFromCallstackListFile(source_file): return root +def getDuration(obj): + total = obj['TotalDuration'] -def nodeFromJSONbject(node, key, obj): + if 'Children' in obj: + for key, obj in obj['Children'].items(): + total -= obj['TotalDuration'] + + return total + + +def nodeFromJSONObject(node, key, obj): source, function, line = key.split(",") node.function = function node.source = source node.line = int(line) if len(line) > 0 else 0 - node.ticks = obj['Duration'] + node.ticks = getDuration(obj) - for key, obj in obj['Children'].items(): - nodeFromJSONbject(node.child(key), key, obj) + if 'Children' in obj: + for key, obj in obj['Children'].items(): + nodeFromJSONObject(node.child(key), key, obj) return node @@ -77,8 +87,9 @@ def nodeFromJSONFile(source_file): root = Node() - for key, obj in dump['Children'].items(): - nodeFromJSONbject(root.child(key), key, obj) + if 'Children' in dump: + for key, obj in dump['Children'].items(): + nodeFromJSONObject(root.child(key), key, obj) return root From a95dff3223cc10e8b7826aab99de02ca32ef7ab7 Mon Sep 17 00:00:00 2001 From: JohnnyMorganz Date: Tue, 23 Aug 2022 19:53:02 +0100 Subject: [PATCH 346/399] Pass environment scope to autocomplete typechecker (#610) Fixes a bug where the environment scope is not passed to the autocomplete typechecker, so environment-related types are not provided. This caused an issue where `getModuleEnvironment` returns `typeChecker.globalScope` when there is no environment. We pass this through to `typeCheckerForAutocomplete.check()` then environmentscope is no longer nullopt and it will use `typeChecker.globalScope` instead of falling back to `typeCheckerForAutocomplete.globalScope`. This is solved by passing `forAutocomplete` to `getModuleEnvironment` so it falls back to the autocomplete global scope if none found. --- Analysis/include/Luau/Frontend.h | 2 +- Analysis/src/Frontend.cpp | 12 ++++++++---- 2 files changed, 9 insertions(+), 5 deletions(-) diff --git a/Analysis/include/Luau/Frontend.h b/Analysis/include/Luau/Frontend.h index 82df493a..f8da3273 100644 --- a/Analysis/include/Luau/Frontend.h +++ b/Analysis/include/Luau/Frontend.h @@ -166,7 +166,7 @@ private: static LintResult classifyLints(const std::vector& warnings, const Config& config); - ScopePtr getModuleEnvironment(const SourceModule& module, const Config& config); + ScopePtr getModuleEnvironment(const SourceModule& module, const Config& config, bool forAutocomplete = false); std::unordered_map environments; std::unordered_map> builtinDefinitions; diff --git a/Analysis/src/Frontend.cpp b/Analysis/src/Frontend.cpp index 8ab4e865..c8c5d4b5 100644 --- a/Analysis/src/Frontend.cpp +++ b/Analysis/src/Frontend.cpp @@ -455,7 +455,7 @@ CheckResult Frontend::check(const ModuleName& name, std::optional& buildQueue, CheckResult& chec return cyclic; } -ScopePtr Frontend::getModuleEnvironment(const SourceModule& module, const Config& config) +ScopePtr Frontend::getModuleEnvironment(const SourceModule& module, const Config& config, bool forAutocomplete) { - ScopePtr result = typeChecker.globalScope; + ScopePtr result; + if (forAutocomplete) + result = typeCheckerForAutocomplete.globalScope; + else + result = typeChecker.globalScope; if (module.environmentName) result = getEnvironmentScope(*module.environmentName); From 0ce4c454360b8976c8814f0ad0933683066292cb Mon Sep 17 00:00:00 2001 From: Alexander McCord <11488393+alexmccord@users.noreply.github.com> Date: Tue, 23 Aug 2022 12:03:53 -0700 Subject: [PATCH 347/399] RFC: Explicitly support escape sequence in string interpolation literals (#642) This was implicitly assumed to be supported, but we should really just say so explicitly. If we didn't, there'd be two ways to interpret it: string interpolations are raw strings and escape sequences don't exist for the most part (false), or string interpolations are like strings and escape sequences do exist (true). Also talks about the ambiguity with `\u{` and that it is defined to take priority and does not terminate the interpolation chunk and starts a new expression, instead it must be a well-formed escape sequence `\u{...}`. It is very unlikely there'd be another escape sequence that also has this same ambiguity problem, so we don't need to worry about this. --- rfcs/syntax-string-interpolation.md | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/rfcs/syntax-string-interpolation.md b/rfcs/syntax-string-interpolation.md index ad182620..9f7ae085 100644 --- a/rfcs/syntax-string-interpolation.md +++ b/rfcs/syntax-string-interpolation.md @@ -139,6 +139,13 @@ local foo: `foo` local bar: `bar{baz}` ``` +String interpolation syntax will also support escape sequences. Except `\u{...}`, there is no ambiguity with other escape sequences. If `\u{...}` occurs within a string interpolation literal, it takes priority. + +```lua +local foo = `foo\tbar` -- "foo bar" +local bar = `\u{0041} \u{42}` -- "A B" +``` + ## Drawbacks If we want to use backticks for other purposes, it may introduce some potential ambiguity. One option to solve that is to only ever produce string interpolation tokens from the context of an expression. This is messy but doable because the parser and the lexer are already implemented to work in tandem. The other option is to pick a different delimiter syntax to keep backticks available for use in the future. From da9d8e8c601eb30ddf37e9ac5d31601831cca4bf Mon Sep 17 00:00:00 2001 From: boyned//Kampfkarren Date: Wed, 24 Aug 2022 12:01:00 -0700 Subject: [PATCH 348/399] String interpolation (#614) Implements the string interpolation RFC (#165). Adds the string interpolation as per the RFC. ```lua local name = "world" print(`Hello {name}!`) -- Hello world! ``` Co-authored-by: Arseny Kapoulkine Co-authored-by: Alexander McCord <11488393+alexmccord@users.noreply.github.com> --- Analysis/include/Luau/TypeInfer.h | 1 + Analysis/src/AstJsonEncoder.cpp | 14 +++ Analysis/src/Linter.cpp | 18 +++ Analysis/src/Transpiler.cpp | 22 ++++ Analysis/src/TypeInfer.cpp | 10 ++ Ast/include/Luau/Ast.h | 20 ++++ Ast/include/Luau/Lexer.h | 21 ++++ Ast/include/Luau/Parser.h | 4 + Ast/include/Luau/StringUtils.h | 2 +- Ast/src/Ast.cpp | 16 +++ Ast/src/Lexer.cpp | 182 +++++++++++++++++++++++++---- Ast/src/Parser.cpp | 89 +++++++++++++- Ast/src/StringUtils.cpp | 10 +- Compiler/src/Compiler.cpp | 79 +++++++++++++ Compiler/src/ConstantFolding.cpp | 5 + Compiler/src/CostModel.cpp | 10 ++ docs/_pages/grammar.md | 3 +- tests/AstJsonEncoder.test.cpp | 12 ++ tests/Autocomplete.test.cpp | 9 ++ tests/Compiler.test.cpp | 52 +++++++++ tests/Conformance.test.cpp | 8 ++ tests/Lexer.test.cpp | 86 ++++++++++++++ tests/Linter.test.cpp | 24 +++- tests/Parser.test.cpp | 140 ++++++++++++++++++++++ tests/Transpiler.test.cpp | 19 +++ tests/TypeInfer.test.cpp | 36 ++++++ tests/conformance/stringinterp.lua | 59 ++++++++++ 27 files changed, 915 insertions(+), 36 deletions(-) create mode 100644 tests/conformance/stringinterp.lua diff --git a/Analysis/include/Luau/TypeInfer.h b/Analysis/include/Luau/TypeInfer.h index 80f9085f..e253eddf 100644 --- a/Analysis/include/Luau/TypeInfer.h +++ b/Analysis/include/Luau/TypeInfer.h @@ -107,6 +107,7 @@ struct TypeChecker WithPredicate checkExpr(const ScopePtr& scope, const AstExprTypeAssertion& expr); WithPredicate checkExpr(const ScopePtr& scope, const AstExprError& expr); WithPredicate checkExpr(const ScopePtr& scope, const AstExprIfElse& expr, std::optional expectedType = std::nullopt); + WithPredicate checkExpr(const ScopePtr& scope, const AstExprInterpString& expr); TypeId checkExprTable(const ScopePtr& scope, const AstExprTable& expr, const std::vector>& fieldTypes, std::optional expectedType); diff --git a/Analysis/src/AstJsonEncoder.cpp b/Analysis/src/AstJsonEncoder.cpp index 2897875d..8d589037 100644 --- a/Analysis/src/AstJsonEncoder.cpp +++ b/Analysis/src/AstJsonEncoder.cpp @@ -445,6 +445,14 @@ struct AstJsonEncoder : public AstVisitor }); } + void write(class AstExprInterpString* node) + { + writeNode(node, "AstExprInterpString", [&]() { + PROP(strings); + PROP(expressions); + }); + } + void write(class AstExprTable* node) { writeNode(node, "AstExprTable", [&]() { @@ -888,6 +896,12 @@ struct AstJsonEncoder : public AstVisitor return false; } + bool visit(class AstExprInterpString* node) override + { + write(node); + return false; + } + bool visit(class AstExprLocal* node) override { write(node); diff --git a/Analysis/src/Linter.cpp b/Analysis/src/Linter.cpp index 2d058376..9f89efe9 100644 --- a/Analysis/src/Linter.cpp +++ b/Analysis/src/Linter.cpp @@ -206,6 +206,24 @@ static bool similar(AstExpr* lhs, AstExpr* rhs) return true; } CASE(AstExprIfElse) return similar(le->condition, re->condition) && similar(le->trueExpr, re->trueExpr) && similar(le->falseExpr, re->falseExpr); + CASE(AstExprInterpString) + { + if (le->strings.size != re->strings.size) + return false; + + if (le->expressions.size != re->expressions.size) + return false; + + for (size_t i = 0; i < le->strings.size; ++i) + if (le->strings.data[i].size != re->strings.data[i].size || memcmp(le->strings.data[i].data, re->strings.data[i].data, le->strings.data[i].size) != 0) + return false; + + for (size_t i = 0; i < le->expressions.size; ++i) + if (!similar(le->expressions.data[i], re->expressions.data[i])) + return false; + + return true; + } else { LUAU_ASSERT(!"Unknown expression type"); diff --git a/Analysis/src/Transpiler.cpp b/Analysis/src/Transpiler.cpp index 9feff1c0..cdfe6549 100644 --- a/Analysis/src/Transpiler.cpp +++ b/Analysis/src/Transpiler.cpp @@ -511,6 +511,28 @@ struct Printer writer.keyword("else"); visualize(*a->falseExpr); } + else if (const auto& a = expr.as()) + { + writer.symbol("`"); + + size_t index = 0; + + for (const auto& string : a->strings) + { + writer.write(escape(std::string_view(string.data, string.size), /* escapeForInterpString = */ true)); + + if (index < a->expressions.size) + { + writer.symbol("{"); + visualize(*a->expressions.data[index]); + writer.symbol("}"); + } + + index++; + } + + writer.symbol("`"); + } else if (const auto& a = expr.as()) { writer.symbol("(error-expr"); diff --git a/Analysis/src/TypeInfer.cpp b/Analysis/src/TypeInfer.cpp index 9886fb19..77168053 100644 --- a/Analysis/src/TypeInfer.cpp +++ b/Analysis/src/TypeInfer.cpp @@ -1805,6 +1805,8 @@ WithPredicate TypeChecker::checkExpr(const ScopePtr& scope, const AstExp result = checkExpr(scope, *a); else if (auto a = expr.as()) result = checkExpr(scope, *a, expectedType); + else if (auto a = expr.as()) + result = checkExpr(scope, *a); else ice("Unhandled AstExpr?"); @@ -2999,6 +3001,14 @@ WithPredicate TypeChecker::checkExpr(const ScopePtr& scope, const AstExp return {types.size() == 1 ? types[0] : addType(UnionTypeVar{std::move(types)})}; } +WithPredicate TypeChecker::checkExpr(const ScopePtr& scope, const AstExprInterpString& expr) +{ + for (AstExpr* expr : expr.expressions) + checkExpr(scope, *expr); + + return {stringType}; +} + TypeId TypeChecker::checkLValue(const ScopePtr& scope, const AstExpr& expr) { return checkLValueBinding(scope, expr); diff --git a/Ast/include/Luau/Ast.h b/Ast/include/Luau/Ast.h index 1e164d04..5c040007 100644 --- a/Ast/include/Luau/Ast.h +++ b/Ast/include/Luau/Ast.h @@ -134,6 +134,10 @@ public: { return visit((class AstExpr*)node); } + virtual bool visit(class AstExprInterpString* node) + { + return visit((class AstExpr*)node); + } virtual bool visit(class AstExprError* node) { return visit((class AstExpr*)node); @@ -732,6 +736,22 @@ public: AstExpr* falseExpr; }; +class AstExprInterpString : public AstExpr +{ +public: + LUAU_RTTI(AstExprInterpString) + + AstExprInterpString(const Location& location, const AstArray>& strings, const AstArray& expressions); + + void visit(AstVisitor* visitor) override; + + /// An interpolated string such as `foo{bar}baz` is represented as + /// an array of strings for "foo" and "bar", and an array of expressions for "baz". + /// `strings` will always have one more element than `expressions`. + AstArray> strings; + AstArray expressions; +}; + class AstStatBlock : public AstStat { public: diff --git a/Ast/include/Luau/Lexer.h b/Ast/include/Luau/Lexer.h index 4f3dbbd5..7e7fe76b 100644 --- a/Ast/include/Luau/Lexer.h +++ b/Ast/include/Luau/Lexer.h @@ -61,6 +61,12 @@ struct Lexeme SkinnyArrow, DoubleColon, + InterpStringBegin, + InterpStringMid, + InterpStringEnd, + // An interpolated string with no expressions (like `x`) + InterpStringSimple, + AddAssign, SubAssign, MulAssign, @@ -80,6 +86,8 @@ struct Lexeme BrokenString, BrokenComment, BrokenUnicode, + BrokenInterpDoubleBrace, + Error, Reserved_BEGIN, @@ -208,6 +216,11 @@ private: Lexeme readLongString(const Position& start, int sep, Lexeme::Type ok, Lexeme::Type broken); Lexeme readQuotedString(); + Lexeme readInterpolatedStringBegin(); + Lexeme readInterpolatedStringSection(Position start, Lexeme::Type formatType, Lexeme::Type endType); + + void readBackslashInString(); + std::pair readName(); Lexeme readNumber(const Position& start, unsigned int startOffset); @@ -231,6 +244,14 @@ private: bool skipComments; bool readNames; + + enum class BraceType + { + InterpolatedString, + Normal + }; + + std::vector braceStack; }; inline bool isSpace(char ch) diff --git a/Ast/include/Luau/Parser.h b/Ast/include/Luau/Parser.h index 046706d3..6f56f90d 100644 --- a/Ast/include/Luau/Parser.h +++ b/Ast/include/Luau/Parser.h @@ -228,6 +228,9 @@ private: // TODO: Add grammar rules here? AstExpr* parseIfElseExpr(); + // stringinterp ::= exp { exp} + AstExpr* parseInterpString(); + // Name std::optional parseNameOpt(const char* context = nullptr); Name parseName(const char* context = nullptr); @@ -379,6 +382,7 @@ private: std::vector matchRecoveryStopOnToken; std::vector scratchStat; + std::vector> scratchString; std::vector scratchExpr; std::vector scratchExprAux; std::vector scratchName; diff --git a/Ast/include/Luau/StringUtils.h b/Ast/include/Luau/StringUtils.h index 6ae9e977..dab76106 100644 --- a/Ast/include/Luau/StringUtils.h +++ b/Ast/include/Luau/StringUtils.h @@ -35,6 +35,6 @@ bool equalsLower(std::string_view lhs, std::string_view rhs); size_t hashRange(const char* data, size_t size); -std::string escape(std::string_view s); +std::string escape(std::string_view s, bool escapeForInterpString = false); bool isIdentifier(std::string_view s); } // namespace Luau diff --git a/Ast/src/Ast.cpp b/Ast/src/Ast.cpp index 3066b756..5ede3893 100644 --- a/Ast/src/Ast.cpp +++ b/Ast/src/Ast.cpp @@ -349,6 +349,22 @@ AstExprError::AstExprError(const Location& location, const AstArray& e { } +AstExprInterpString::AstExprInterpString(const Location& location, const AstArray>& strings, const AstArray& expressions) + : AstExpr(ClassIndex(), location) + , strings(strings) + , expressions(expressions) +{ +} + +void AstExprInterpString::visit(AstVisitor* visitor) +{ + if (visitor->visit(this)) + { + for (AstExpr* expr : expressions) + expr->visit(visitor); + } +} + void AstExprError::visit(AstVisitor* visitor) { if (visitor->visit(this)) diff --git a/Ast/src/Lexer.cpp b/Ast/src/Lexer.cpp index a1f1d469..b4db8bdf 100644 --- a/Ast/src/Lexer.cpp +++ b/Ast/src/Lexer.cpp @@ -6,6 +6,8 @@ #include +LUAU_FASTFLAG(LuauInterpolatedStringBaseSupport) + namespace Luau { @@ -89,7 +91,18 @@ Lexeme::Lexeme(const Location& location, Type type, const char* data, size_t siz , length(unsigned(size)) , data(data) { - LUAU_ASSERT(type == RawString || type == QuotedString || type == Number || type == Comment || type == BlockComment); + LUAU_ASSERT( + type == RawString + || type == QuotedString + || type == InterpStringBegin + || type == InterpStringMid + || type == InterpStringEnd + || type == InterpStringSimple + || type == BrokenInterpDoubleBrace + || type == Number + || type == Comment + || type == BlockComment + ); } Lexeme::Lexeme(const Location& location, Type type, const char* name) @@ -160,6 +173,18 @@ std::string Lexeme::toString() const case QuotedString: return data ? format("\"%.*s\"", length, data) : "string"; + case InterpStringBegin: + return data ? format("`%.*s{", length, data) : "the beginning of an interpolated string"; + + case InterpStringMid: + return data ? format("}%.*s{", length, data) : "the middle of an interpolated string"; + + case InterpStringEnd: + return data ? format("}%.*s`", length, data) : "the end of an interpolated string"; + + case InterpStringSimple: + return data ? format("`%.*s`", length, data) : "interpolated string"; + case Number: return data ? format("'%.*s'", length, data) : "number"; @@ -175,6 +200,9 @@ std::string Lexeme::toString() const case BrokenComment: return "unfinished comment"; + case BrokenInterpDoubleBrace: + return "'{{', which is invalid (did you mean '\\{'?)"; + case BrokenUnicode: if (codepoint) { @@ -515,6 +543,32 @@ Lexeme Lexer::readLongString(const Position& start, int sep, Lexeme::Type ok, Le return Lexeme(Location(start, position()), broken); } +void Lexer::readBackslashInString() +{ + LUAU_ASSERT(peekch() == '\\'); + consume(); + switch (peekch()) + { + case '\r': + consume(); + if (peekch() == '\n') + consume(); + break; + + case 0: + break; + + case 'z': + consume(); + while (isSpace(peekch())) + consume(); + break; + + default: + consume(); + } +} + Lexeme Lexer::readQuotedString() { Position start = position(); @@ -535,27 +589,7 @@ Lexeme Lexer::readQuotedString() return Lexeme(Location(start, position()), Lexeme::BrokenString); case '\\': - consume(); - switch (peekch()) - { - case '\r': - consume(); - if (peekch() == '\n') - consume(); - break; - - case 0: - break; - - case 'z': - consume(); - while (isSpace(peekch())) - consume(); - break; - - default: - consume(); - } + readBackslashInString(); break; default: @@ -568,6 +602,69 @@ Lexeme Lexer::readQuotedString() return Lexeme(Location(start, position()), Lexeme::QuotedString, &buffer[startOffset], offset - startOffset - 1); } +Lexeme Lexer::readInterpolatedStringBegin() +{ + LUAU_ASSERT(peekch() == '`'); + + Position start = position(); + consume(); + + return readInterpolatedStringSection(start, Lexeme::InterpStringBegin, Lexeme::InterpStringSimple); +} + +Lexeme Lexer::readInterpolatedStringSection(Position start, Lexeme::Type formatType, Lexeme::Type endType) +{ + unsigned int startOffset = offset; + + while (peekch() != '`') + { + switch (peekch()) + { + case 0: + case '\r': + case '\n': + return Lexeme(Location(start, position()), Lexeme::BrokenString); + + case '\\': + // Allow for \u{}, which would otherwise be consumed by looking for { + if (peekch(1) == 'u' && peekch(2) == '{') + { + consume(); // backslash + consume(); // u + consume(); // { + break; + } + + readBackslashInString(); + break; + + case '{': + { + braceStack.push_back(BraceType::InterpolatedString); + + if (peekch(1) == '{') + { + Lexeme brokenDoubleBrace = Lexeme(Location(start, position()), Lexeme::BrokenInterpDoubleBrace, &buffer[startOffset], offset - startOffset); + consume(); + consume(); + return brokenDoubleBrace; + } + + Lexeme lexemeOutput(Location(start, position()), Lexeme::InterpStringBegin, &buffer[startOffset], offset - startOffset); + consume(); + return lexemeOutput; + } + + default: + consume(); + } + } + + consume(); + + return Lexeme(Location(start, position()), endType, &buffer[startOffset], offset - startOffset - 1); +} + Lexeme Lexer::readNumber(const Position& start, unsigned int startOffset) { LUAU_ASSERT(isDigit(peekch())); @@ -660,6 +757,36 @@ Lexeme Lexer::readNext() } } + case '{': + { + consume(); + + if (!braceStack.empty()) + braceStack.push_back(BraceType::Normal); + + return Lexeme(Location(start, 1), '{'); + } + + case '}': + { + consume(); + + if (braceStack.empty()) + { + return Lexeme(Location(start, 1), '}'); + } + + const BraceType braceStackTop = braceStack.back(); + braceStack.pop_back(); + + if (braceStackTop != BraceType::InterpolatedString) + { + return Lexeme(Location(start, 1), '}'); + } + + return readInterpolatedStringSection(position(), Lexeme::InterpStringMid, Lexeme::InterpStringEnd); + } + case '=': { consume(); @@ -716,6 +843,15 @@ Lexeme Lexer::readNext() case '\'': return readQuotedString(); + case '`': + if (FFlag::LuauInterpolatedStringBaseSupport) + return readInterpolatedStringBegin(); + else + { + consume(); + return Lexeme(Location(start, 1), '`'); + } + case '.': consume(); @@ -817,8 +953,6 @@ Lexeme Lexer::readNext() case '(': case ')': - case '{': - case '}': case ']': case ';': case ',': diff --git a/Ast/src/Parser.cpp b/Ast/src/Parser.cpp index e46eebf3..cc624c10 100644 --- a/Ast/src/Parser.cpp +++ b/Ast/src/Parser.cpp @@ -23,10 +23,14 @@ LUAU_FASTFLAGVARIABLE(LuauErrorDoubleHexPrefix, false) LUAU_FASTFLAGVARIABLE(LuauLintParseIntegerIssues, false) LUAU_DYNAMIC_FASTFLAGVARIABLE(LuaReportParseIntegerIssues, false) +LUAU_FASTFLAGVARIABLE(LuauInterpolatedStringBaseSupport, false) + bool lua_telemetry_parsed_out_of_range_bin_integer = false; bool lua_telemetry_parsed_out_of_range_hex_integer = false; bool lua_telemetry_parsed_double_prefix_hex_integer = false; +#define ERROR_INVALID_INTERP_DOUBLE_BRACE "Double braces are not permitted within interpolated strings. Did you mean '\\{'?" + namespace Luau { @@ -1567,6 +1571,12 @@ AstTypeOrPack Parser::parseSimpleTypeAnnotation(bool allowPack) else return {reportTypeAnnotationError(begin, {}, /*isMissing*/ false, "String literal contains malformed escape sequence")}; } + else if (lexer.current().type == Lexeme::InterpStringBegin || lexer.current().type == Lexeme::InterpStringSimple) + { + parseInterpString(); + + return {reportTypeAnnotationError(begin, {}, /*isMissing*/ false, "Interpolated string literals cannot be used as types")}; + } else if (lexer.current().type == Lexeme::BrokenString) { Location location = lexer.current().location; @@ -2215,15 +2225,24 @@ AstExpr* Parser::parseSimpleExpr() { return parseNumber(); } - else if (lexer.current().type == Lexeme::RawString || lexer.current().type == Lexeme::QuotedString) + else if (lexer.current().type == Lexeme::RawString || lexer.current().type == Lexeme::QuotedString || (FFlag::LuauInterpolatedStringBaseSupport && lexer.current().type == Lexeme::InterpStringSimple)) { return parseString(); } + else if (FFlag::LuauInterpolatedStringBaseSupport && lexer.current().type == Lexeme::InterpStringBegin) + { + return parseInterpString(); + } else if (lexer.current().type == Lexeme::BrokenString) { nextLexeme(); return reportExprError(start, {}, "Malformed string"); } + else if (lexer.current().type == Lexeme::BrokenInterpDoubleBrace) + { + nextLexeme(); + return reportExprError(start, {}, ERROR_INVALID_INTERP_DOUBLE_BRACE); + } else if (lexer.current().type == Lexeme::Dot3) { if (functionStack.back().vararg) @@ -2614,11 +2633,11 @@ AstArray Parser::parseTypeParams() std::optional> Parser::parseCharArray() { - LUAU_ASSERT(lexer.current().type == Lexeme::QuotedString || lexer.current().type == Lexeme::RawString); + LUAU_ASSERT(lexer.current().type == Lexeme::QuotedString || lexer.current().type == Lexeme::RawString || lexer.current().type == Lexeme::InterpStringSimple); scratchData.assign(lexer.current().data, lexer.current().length); - if (lexer.current().type == Lexeme::QuotedString) + if (lexer.current().type == Lexeme::QuotedString || lexer.current().type == Lexeme::InterpStringSimple) { if (!Lexer::fixupQuotedString(scratchData)) { @@ -2645,6 +2664,70 @@ AstExpr* Parser::parseString() return reportExprError(location, {}, "String literal contains malformed escape sequence"); } +AstExpr* Parser::parseInterpString() +{ + TempVector> strings(scratchString); + TempVector expressions(scratchExpr); + + Location startLocation = lexer.current().location; + + do { + Lexeme currentLexeme = lexer.current(); + LUAU_ASSERT( + currentLexeme.type == Lexeme::InterpStringBegin + || currentLexeme.type == Lexeme::InterpStringMid + || currentLexeme.type == Lexeme::InterpStringEnd + || currentLexeme.type == Lexeme::InterpStringSimple + ); + + Location location = currentLexeme.location; + + Location startOfBrace = Location(location.end, 1); + + scratchData.assign(currentLexeme.data, currentLexeme.length); + + if (!Lexer::fixupQuotedString(scratchData)) + { + nextLexeme(); + return reportExprError(startLocation, {}, "Interpolated string literal contains malformed escape sequence"); + } + + AstArray chars = copy(scratchData); + + nextLexeme(); + + strings.push_back(chars); + + if (currentLexeme.type == Lexeme::InterpStringEnd || currentLexeme.type == Lexeme::InterpStringSimple) + { + AstArray> stringsArray = copy(strings); + AstArray expressionsArray = copy(expressions); + + return allocator.alloc(startLocation, stringsArray, expressionsArray); + } + + AstExpr* expression = parseExpr(); + + expressions.push_back(expression); + + switch (lexer.current().type) + { + case Lexeme::InterpStringBegin: + case Lexeme::InterpStringMid: + case Lexeme::InterpStringEnd: + break; + case Lexeme::BrokenInterpDoubleBrace: + nextLexeme(); + return reportExprError(location, {}, ERROR_INVALID_INTERP_DOUBLE_BRACE); + case Lexeme::BrokenString: + nextLexeme(); + return reportExprError(location, {}, "Malformed interpolated string, did you forget to add a '}'?"); + default: + return reportExprError(location, {}, "Malformed interpolated string, got %s", lexer.current().toString().c_str()); + } + } while (true); +} + AstExpr* Parser::parseNumber() { Location start = lexer.current().location; diff --git a/Ast/src/StringUtils.cpp b/Ast/src/StringUtils.cpp index 0dc3f3f5..11e0076a 100644 --- a/Ast/src/StringUtils.cpp +++ b/Ast/src/StringUtils.cpp @@ -230,19 +230,25 @@ bool isIdentifier(std::string_view s) return (s.find_first_not_of("abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890_") == std::string::npos); } -std::string escape(std::string_view s) +std::string escape(std::string_view s, bool escapeForInterpString) { std::string r; r.reserve(s.size() + 50); // arbitrary number to guess how many characters we'll be inserting for (uint8_t c : s) { - if (c >= ' ' && c != '\\' && c != '\'' && c != '\"') + if (c >= ' ' && c != '\\' && c != '\'' && c != '\"' && c != '`' && c != '{') r += c; else { r += '\\'; + if (escapeForInterpString && (c == '`' || c == '{')) + { + r += c; + continue; + } + switch (c) { case '\a': diff --git a/Compiler/src/Compiler.cpp b/Compiler/src/Compiler.cpp index bd8744cc..a9394d74 100644 --- a/Compiler/src/Compiler.cpp +++ b/Compiler/src/Compiler.cpp @@ -14,6 +14,8 @@ #include #include +#include + #include LUAU_FASTINTVARIABLE(LuauCompileLoopUnrollThreshold, 25) @@ -25,6 +27,8 @@ LUAU_FASTINTVARIABLE(LuauCompileInlineDepth, 5) LUAU_FASTFLAGVARIABLE(LuauCompileXEQ, false) +LUAU_FASTFLAG(LuauInterpolatedStringBaseSupport) + LUAU_FASTFLAGVARIABLE(LuauCompileOptimalAssignment, false) LUAU_FASTFLAGVARIABLE(LuauCompileExtractK, false) @@ -1585,6 +1589,76 @@ struct Compiler } } + void compileExprInterpString(AstExprInterpString* expr, uint8_t target, bool targetTemp) + { + size_t formatCapacity = 0; + for (AstArray string : expr->strings) + { + formatCapacity += string.size + std::count(string.data, string.data + string.size, '%'); + } + + std::string formatString; + formatString.reserve(formatCapacity); + + size_t stringsLeft = expr->strings.size; + + for (AstArray string : expr->strings) + { + if (memchr(string.data, '%', string.size)) + { + for (size_t characterIndex = 0; characterIndex < string.size; ++characterIndex) + { + char character = string.data[characterIndex]; + formatString.push_back(character); + + if (character == '%') + formatString.push_back('%'); + } + } + else + formatString.append(string.data, string.size); + + stringsLeft--; + + if (stringsLeft > 0) + formatString += "%*"; + } + + size_t formatStringSize = formatString.size(); + + // We can't use formatStringRef.data() directly, because short strings don't have their data + // pinned in memory, so when interpFormatStrings grows, these pointers will move and become invalid. + std::unique_ptr formatStringPtr(new char[formatStringSize]); + memcpy(formatStringPtr.get(), formatString.data(), formatStringSize); + + AstArray formatStringArray{formatStringPtr.get(), formatStringSize}; + interpStrings.emplace_back(std::move(formatStringPtr)); // invalidates formatStringPtr, but keeps formatStringArray intact + + int32_t formatStringIndex = bytecode.addConstantString(sref(formatStringArray)); + if (formatStringIndex < 0) + CompileError::raise(expr->location, "Exceeded constant limit; simplify the code to compile"); + + RegScope rs(this); + + uint8_t baseReg = allocReg(expr, uint8_t(2 + expr->expressions.size)); + + emitLoadK(baseReg, formatStringIndex); + + for (size_t index = 0; index < expr->expressions.size; ++index) + compileExprTempTop(expr->expressions.data[index], uint8_t(baseReg + 2 + index)); + + BytecodeBuilder::StringRef formatMethod = sref(AstName("format")); + + int32_t formatMethodIndex = bytecode.addConstantString(formatMethod); + if (formatMethodIndex < 0) + CompileError::raise(expr->location, "Exceeded constant limit; simplify the code to compile"); + + bytecode.emitABC(LOP_NAMECALL, baseReg, baseReg, uint8_t(BytecodeBuilder::getStringHash(formatMethod))); + bytecode.emitAux(formatMethodIndex); + bytecode.emitABC(LOP_CALL, baseReg, uint8_t(expr->expressions.size + 2), 2); + bytecode.emitABC(LOP_MOVE, target, baseReg, 0); + } + static uint8_t encodeHashSize(unsigned int hashSize) { size_t hashSizeLog2 = 0; @@ -2059,6 +2133,10 @@ struct Compiler { compileExprIfElse(expr, target, targetTemp); } + else if (AstExprInterpString* interpString = node->as(); FFlag::LuauInterpolatedStringBaseSupport && interpString) + { + compileExprInterpString(interpString, target, targetTemp); + } else { LUAU_ASSERT(!"Unknown expression type"); @@ -3808,6 +3886,7 @@ struct Compiler std::vector loops; std::vector inlineFrames; std::vector captures; + std::vector> interpStrings; }; void compileOrThrow(BytecodeBuilder& bytecode, const ParseResult& parseResult, const AstNameTable& names, const CompileOptions& inputOptions) diff --git a/Compiler/src/ConstantFolding.cpp b/Compiler/src/ConstantFolding.cpp index 34f79544..e35c883a 100644 --- a/Compiler/src/ConstantFolding.cpp +++ b/Compiler/src/ConstantFolding.cpp @@ -349,6 +349,11 @@ struct ConstantVisitor : AstVisitor if (cond.type != Constant::Type_Unknown) result = cond.isTruthful() ? trueExpr : falseExpr; } + else if (AstExprInterpString* expr = node->as()) + { + for (AstExpr* expression : expr->expressions) + analyze(expression); + } else { LUAU_ASSERT(!"Unknown expression type"); diff --git a/Compiler/src/CostModel.cpp b/Compiler/src/CostModel.cpp index 81cbfd7a..ffc1cb1f 100644 --- a/Compiler/src/CostModel.cpp +++ b/Compiler/src/CostModel.cpp @@ -215,6 +215,16 @@ struct CostVisitor : AstVisitor { return model(expr->condition) + model(expr->trueExpr) + model(expr->falseExpr) + 2; } + else if (AstExprInterpString* expr = node->as()) + { + // Baseline cost of string.format + Cost cost = 3; + + for (AstExpr* innerExpression : expr->expressions) + cost += model(innerExpression); + + return cost; + } else { LUAU_ASSERT(!"Unknown expression type"); diff --git a/docs/_pages/grammar.md b/docs/_pages/grammar.md index ffb1dbf6..90d918e7 100644 --- a/docs/_pages/grammar.md +++ b/docs/_pages/grammar.md @@ -44,7 +44,8 @@ functioncall = prefixexp funcargs | prefixexp ':' NAME funcargs exp = (asexp | unop exp) { binop exp } ifelseexp = 'if' exp 'then' exp {'elseif' exp 'then' exp} 'else' exp asexp = simpleexp ['::' Type] -simpleexp = NUMBER | STRING | 'nil' | 'true' | 'false' | '...' | tableconstructor | 'function' body | prefixexp | ifelseexp +stringinterp = INTERP_BEGIN exp { INTERP_MID exp } INTERP_END +simpleexp = NUMBER | STRING | 'nil' | 'true' | 'false' | '...' | tableconstructor | 'function' body | prefixexp | ifelseexp | stringinterp funcargs = '(' [explist] ')' | tableconstructor | STRING tableconstructor = '{' [fieldlist] '}' diff --git a/tests/AstJsonEncoder.test.cpp b/tests/AstJsonEncoder.test.cpp index 3ff36741..a23f6f47 100644 --- a/tests/AstJsonEncoder.test.cpp +++ b/tests/AstJsonEncoder.test.cpp @@ -2,6 +2,7 @@ #include "Luau/Ast.h" #include "Luau/AstJsonEncoder.h" #include "Luau/Parser.h" +#include "ScopedFlags.h" #include "doctest.h" @@ -175,6 +176,17 @@ TEST_CASE_FIXTURE(JsonEncoderFixture, "encode_AstExprIfThen") CHECK(toJson(statement) == expected); } +TEST_CASE_FIXTURE(JsonEncoderFixture, "encode_AstExprInterpString") +{ + ScopedFastFlag sff{"LuauInterpolatedStringBaseSupport", true}; + + AstStat* statement = expectParseStatement("local a = `var = {x}`"); + + std::string_view expected = + R"({"type":"AstStatLocal","location":"0,0 - 0,17","vars":[{"luauType":null,"name":"a","type":"AstLocal","location":"0,6 - 0,7"}],"values":[{"type":"AstExprInterpString","location":"0,10 - 0,17","strings":["var = ",""],"expressions":[{"type":"AstExprGlobal","location":"0,18 - 0,19","global":"x"}]}]})"; + + CHECK(toJson(statement) == expected); +} TEST_CASE("encode_AstExprLocal") { diff --git a/tests/Autocomplete.test.cpp b/tests/Autocomplete.test.cpp index 25447dd8..988cbe80 100644 --- a/tests/Autocomplete.test.cpp +++ b/tests/Autocomplete.test.cpp @@ -2708,6 +2708,15 @@ a = if temp then even else abc@3 CHECK(ac.entryMap.count("abcdef")); } +TEST_CASE_FIXTURE(ACFixture, "autocomplete_interpolated_string") +{ + check(R"(f(`expression = {@1}`))"); + + auto ac = autocomplete('1'); + CHECK(ac.entryMap.count("table")); + CHECK_EQ(ac.context, AutocompleteContext::Expression); +} + TEST_CASE_FIXTURE(ACFixture, "autocomplete_explicit_type_pack") { check(R"( diff --git a/tests/Compiler.test.cpp b/tests/Compiler.test.cpp index a2e748a8..1aa69112 100644 --- a/tests/Compiler.test.cpp +++ b/tests/Compiler.test.cpp @@ -1230,6 +1230,58 @@ RETURN R0 0 )"); } +TEST_CASE("InterpStringWithNoExpressions") +{ + ScopedFastFlag sff{"LuauInterpolatedStringBaseSupport", true}; + + CHECK_EQ(compileFunction0(R"(return "hello")"), compileFunction0("return `hello`")); +} + +TEST_CASE("InterpStringZeroCost") +{ + ScopedFastFlag sff{"LuauInterpolatedStringBaseSupport", true}; + + CHECK_EQ( + "\n" + compileFunction0(R"(local _ = `hello, {"world"}!`)"), + R"( +LOADK R1 K0 +LOADK R3 K1 +NAMECALL R1 R1 K2 +CALL R1 2 1 +MOVE R0 R1 +RETURN R0 0 +)" + ); +} + +TEST_CASE("InterpStringRegisterCleanup") +{ + ScopedFastFlag sff{"LuauInterpolatedStringBaseSupport", true}; + + CHECK_EQ( + "\n" + compileFunction0(R"( + local a, b, c = nil, "um", "uh oh" + a = `foo{"bar"}` + print(a) + )"), + + R"( +LOADNIL R0 +LOADK R1 K0 +LOADK R2 K1 +LOADK R3 K2 +LOADK R5 K3 +NAMECALL R3 R3 K4 +CALL R3 2 1 +MOVE R0 R3 +GETIMPORT R3 6 +MOVE R4 R0 +CALL R3 1 0 +RETURN R0 0 +)" + ); +} + TEST_CASE("ConstantFoldArith") { CHECK_EQ("\n" + compileFunction0("return 10 + 2"), R"( diff --git a/tests/Conformance.test.cpp b/tests/Conformance.test.cpp index be2feacb..89cb0751 100644 --- a/tests/Conformance.test.cpp +++ b/tests/Conformance.test.cpp @@ -294,6 +294,14 @@ TEST_CASE("Strings") runConformance("strings.lua"); } +TEST_CASE("StringInterp") +{ + ScopedFastFlag sffInterpStrings{"LuauInterpolatedStringBaseSupport", true}; + ScopedFastFlag sffTostringFormat{"LuauTostringFormatSpecifier", true}; + + runConformance("stringinterp.lua"); +} + TEST_CASE("VarArg") { runConformance("vararg.lua"); diff --git a/tests/Lexer.test.cpp b/tests/Lexer.test.cpp index 20d8d0d5..890d100b 100644 --- a/tests/Lexer.test.cpp +++ b/tests/Lexer.test.cpp @@ -138,4 +138,90 @@ TEST_CASE("lookahead") CHECK_EQ(lexer.lookahead().type, Lexeme::Eof); } +TEST_CASE("string_interpolation_basic") +{ + ScopedFastFlag sff{"LuauInterpolatedStringBaseSupport", true}; + + const std::string testInput = R"(`foo {"bar"}`)"; + Luau::Allocator alloc; + AstNameTable table(alloc); + Lexer lexer(testInput.c_str(), testInput.size(), table); + + Lexeme interpBegin = lexer.next(); + CHECK_EQ(interpBegin.type, Lexeme::InterpStringBegin); + + Lexeme quote = lexer.next(); + CHECK_EQ(quote.type, Lexeme::QuotedString); + + Lexeme interpEnd = lexer.next(); + CHECK_EQ(interpEnd.type, Lexeme::InterpStringEnd); +} + +TEST_CASE("string_interpolation_double_brace") +{ + ScopedFastFlag sff{"LuauInterpolatedStringBaseSupport", true}; + + const std::string testInput = R"(`foo{{bad}}bar`)"; + Luau::Allocator alloc; + AstNameTable table(alloc); + Lexer lexer(testInput.c_str(), testInput.size(), table); + + auto brokenInterpBegin = lexer.next(); + CHECK_EQ(brokenInterpBegin.type, Lexeme::BrokenInterpDoubleBrace); + CHECK_EQ(std::string(brokenInterpBegin.data, brokenInterpBegin.length), std::string("foo")); + + CHECK_EQ(lexer.next().type, Lexeme::Name); + + auto interpEnd = lexer.next(); + CHECK_EQ(interpEnd.type, Lexeme::InterpStringEnd); + CHECK_EQ(std::string(interpEnd.data, interpEnd.length), std::string("}bar")); +} + +TEST_CASE("string_interpolation_double_but_unmatched_brace") +{ + ScopedFastFlag sff{"LuauInterpolatedStringBaseSupport", true}; + + const std::string testInput = R"(`{{oops}`, 1)"; + Luau::Allocator alloc; + AstNameTable table(alloc); + Lexer lexer(testInput.c_str(), testInput.size(), table); + + CHECK_EQ(lexer.next().type, Lexeme::BrokenInterpDoubleBrace); + CHECK_EQ(lexer.next().type, Lexeme::Name); + CHECK_EQ(lexer.next().type, Lexeme::InterpStringEnd); + CHECK_EQ(lexer.next().type, ','); + CHECK_EQ(lexer.next().type, Lexeme::Number); +} + +TEST_CASE("string_interpolation_unmatched_brace") +{ + ScopedFastFlag sff{"LuauInterpolatedStringBaseSupport", true}; + + const std::string testInput = R"({ + `hello {"world"} + } -- this might be incorrectly parsed as a string)"; + Luau::Allocator alloc; + AstNameTable table(alloc); + Lexer lexer(testInput.c_str(), testInput.size(), table); + + CHECK_EQ(lexer.next().type, '{'); + CHECK_EQ(lexer.next().type, Lexeme::InterpStringBegin); + CHECK_EQ(lexer.next().type, Lexeme::QuotedString); + CHECK_EQ(lexer.next().type, Lexeme::BrokenString); + CHECK_EQ(lexer.next().type, '}'); +} + +TEST_CASE("string_interpolation_with_unicode_escape") +{ + ScopedFastFlag sff{"LuauInterpolatedStringBaseSupport", true}; + + const std::string testInput = R"(`\u{1F41B}`)"; + Luau::Allocator alloc; + AstNameTable table(alloc); + Lexer lexer(testInput.c_str(), testInput.size(), table); + + CHECK_EQ(lexer.next().type, Lexeme::InterpStringSimple); + CHECK_EQ(lexer.next().type, Lexeme::Eof); +} + TEST_SUITE_END(); diff --git a/tests/Linter.test.cpp b/tests/Linter.test.cpp index 64c6d3e4..5a4ab33e 100644 --- a/tests/Linter.test.cpp +++ b/tests/Linter.test.cpp @@ -1662,17 +1662,31 @@ TEST_CASE_FIXTURE(Fixture, "WrongCommentOptimize") { LintResult result = lint(R"( --!optimize ---!optimize --!optimize me --!optimize 100500 --!optimize 2 )"); - REQUIRE_EQ(result.warnings.size(), 4); + REQUIRE_EQ(result.warnings.size(), 3); CHECK_EQ(result.warnings[0].text, "optimize directive requires an optimization level"); - CHECK_EQ(result.warnings[1].text, "optimize directive requires an optimization level"); - CHECK_EQ(result.warnings[2].text, "optimize directive uses unknown optimization level 'me', 0..2 expected"); - CHECK_EQ(result.warnings[3].text, "optimize directive uses unknown optimization level '100500', 0..2 expected"); + CHECK_EQ(result.warnings[1].text, "optimize directive uses unknown optimization level 'me', 0..2 expected"); + CHECK_EQ(result.warnings[2].text, "optimize directive uses unknown optimization level '100500', 0..2 expected"); + + result = lint("--!optimize "); + REQUIRE_EQ(result.warnings.size(), 1); + CHECK_EQ(result.warnings[0].text, "optimize directive requires an optimization level"); +} + +TEST_CASE_FIXTURE(Fixture, "TestStringInterpolation") +{ + ScopedFastFlag sff{"LuauInterpolatedStringBaseSupport", true}; + + LintResult result = lint(R"( + --!nocheck + local _ = `unknown {foo}` + )"); + + REQUIRE_EQ(result.warnings.size(), 1); } TEST_CASE_FIXTURE(Fixture, "IntegerParsing") diff --git a/tests/Parser.test.cpp b/tests/Parser.test.cpp index c55ec181..2dd47708 100644 --- a/tests/Parser.test.cpp +++ b/tests/Parser.test.cpp @@ -905,6 +905,146 @@ TEST_CASE_FIXTURE(Fixture, "parse_compound_assignment_error_multiple") } } +TEST_CASE_FIXTURE(Fixture, "parse_interpolated_string_double_brace_begin") +{ + ScopedFastFlag sff{"LuauInterpolatedStringBaseSupport", true}; + + try + { + parse(R"( + _ = `{{oops}}` + )"); + FAIL("Expected ParseErrors to be thrown"); + } + catch (const ParseErrors& e) + { + CHECK_EQ("Double braces are not permitted within interpolated strings. Did you mean '\\{'?", e.getErrors().front().getMessage()); + } +} + +TEST_CASE_FIXTURE(Fixture, "parse_interpolated_string_double_brace_mid") +{ + ScopedFastFlag sff{"LuauInterpolatedStringBaseSupport", true}; + + try + { + parse(R"( + _ = `{nice} {{oops}}` + )"); + FAIL("Expected ParseErrors to be thrown"); + } + catch (const ParseErrors& e) + { + CHECK_EQ("Double braces are not permitted within interpolated strings. Did you mean '\\{'?", e.getErrors().front().getMessage()); + } +} + +TEST_CASE_FIXTURE(Fixture, "parse_interpolated_string_without_end_brace") +{ + ScopedFastFlag sff{"LuauInterpolatedStringBaseSupport", true}; + + auto columnOfEndBraceError = [this](const char* code) + { + try + { + parse(code); + FAIL("Expected ParseErrors to be thrown"); + return UINT_MAX; + } + catch (const ParseErrors& e) + { + CHECK_EQ(e.getErrors().size(), 1); + + auto error = e.getErrors().front(); + CHECK_EQ("Malformed interpolated string, did you forget to add a '}'?", error.getMessage()); + return error.getLocation().begin.column; + } + }; + + // This makes sure that the error is coming from the brace itself + CHECK_EQ(columnOfEndBraceError("_ = `{a`"), columnOfEndBraceError("_ = `{abcdefg`")); + CHECK_NE(columnOfEndBraceError("_ = `{a`"), columnOfEndBraceError("_ = `{a`")); +} + +TEST_CASE_FIXTURE(Fixture, "parse_interpolated_string_without_end_brace_in_table") +{ + ScopedFastFlag sff{"LuauInterpolatedStringBaseSupport", true}; + + try + { + parse(R"( + _ = { `{a` } + )"); + FAIL("Expected ParseErrors to be thrown"); + } + catch (const ParseErrors& e) + { + CHECK_EQ(e.getErrors().size(), 2); + + CHECK_EQ("Malformed interpolated string, did you forget to add a '}'?", e.getErrors().front().getMessage()); + CHECK_EQ("Expected '}' (to close '{' at line 2), got ", e.getErrors().back().getMessage()); + } +} + +TEST_CASE_FIXTURE(Fixture, "parse_interpolated_string_mid_without_end_brace_in_table") +{ + ScopedFastFlag sff{"LuauInterpolatedStringBaseSupport", true}; + + try + { + parse(R"( + _ = { `x {"y"} {z` } + )"); + FAIL("Expected ParseErrors to be thrown"); + } + catch (const ParseErrors& e) + { + CHECK_EQ(e.getErrors().size(), 2); + + CHECK_EQ("Malformed interpolated string, did you forget to add a '}'?", e.getErrors().front().getMessage()); + CHECK_EQ("Expected '}' (to close '{' at line 2), got ", e.getErrors().back().getMessage()); + } +} + +TEST_CASE_FIXTURE(Fixture, "parse_interpolated_string_as_type_fail") +{ + ScopedFastFlag sff{"LuauInterpolatedStringBaseSupport", true}; + + try + { + parse(R"( + local a: `what` = `???` + local b: `what {"the"}` = `???` + local c: `what {"the"} heck` = `???` + )"); + FAIL("Expected ParseErrors to be thrown"); + } + catch (const ParseErrors& parseErrors) + { + CHECK_EQ(parseErrors.getErrors().size(), 3); + + for (ParseError error : parseErrors.getErrors()) + CHECK_EQ(error.getMessage(), "Interpolated string literals cannot be used as types"); + } +} + +TEST_CASE_FIXTURE(Fixture, "parse_interpolated_string_call_without_parens") +{ + ScopedFastFlag sff{"LuauInterpolatedStringBaseSupport", true}; + + try + { + parse(R"( + _ = print `{42}` + )"); + FAIL("Expected ParseErrors to be thrown"); + } + catch (const ParseErrors& e) + { + CHECK_EQ("Expected identifier when parsing expression, got `{", e.getErrors().front().getMessage()); + } +} + TEST_CASE_FIXTURE(Fixture, "parse_nesting_based_end_detection") { try diff --git a/tests/Transpiler.test.cpp b/tests/Transpiler.test.cpp index d2ed9aef..e79bc9b7 100644 --- a/tests/Transpiler.test.cpp +++ b/tests/Transpiler.test.cpp @@ -6,6 +6,7 @@ #include "Luau/Transpiler.h" #include "Fixture.h" +#include "ScopedFlags.h" #include "doctest.h" @@ -678,4 +679,22 @@ TEST_CASE_FIXTURE(Fixture, "transpile_for_in_multiple_types") CHECK_EQ(code, transpile(code, {}, true).code); } +TEST_CASE_FIXTURE(Fixture, "transpile_string_interp") +{ + ScopedFastFlag sff{"LuauInterpolatedStringBaseSupport", true}; + + std::string code = R"( local _ = `hello {name}` )"; + + CHECK_EQ(code, transpile(code, {}, true).code); +} + +TEST_CASE_FIXTURE(Fixture, "transpile_string_literal_escape") +{ + ScopedFastFlag sff{"LuauInterpolatedStringBaseSupport", true}; + + std::string code = R"( local _ = ` bracket = \{, backtick = \` = {'ok'} ` )"; + + CHECK_EQ(code, transpile(code, {}, true).code); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.test.cpp b/tests/TypeInfer.test.cpp index 80889363..e1dc5023 100644 --- a/tests/TypeInfer.test.cpp +++ b/tests/TypeInfer.test.cpp @@ -8,6 +8,7 @@ #include "Luau/VisitTypeVar.h" #include "Fixture.h" +#include "ScopedFlags.h" #include "doctest.h" @@ -828,6 +829,41 @@ end LUAU_REQUIRE_NO_ERRORS(result); } +TEST_CASE_FIXTURE(Fixture, "tc_interpolated_string_basic") +{ + ScopedFastFlag sff{"LuauInterpolatedStringBaseSupport", true}; + + CheckResult result = check(R"( + local foo: string = `hello {"world"}` + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "tc_interpolated_string_with_invalid_expression") +{ + ScopedFastFlag sff{"LuauInterpolatedStringBaseSupport", true}; + + CheckResult result = check(R"( + local function f(x: number) end + + local foo: string = `hello {f("uh oh")}` + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); +} + +TEST_CASE_FIXTURE(Fixture, "tc_interpolated_string_constant_type") +{ + ScopedFastFlag sff{"LuauInterpolatedStringBaseSupport", true}; + + CheckResult result = check(R"( + local foo: "hello" = `hello` + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + /* * If it wasn't instantly obvious, we have the fuzzer to thank for this gem of a test. * diff --git a/tests/conformance/stringinterp.lua b/tests/conformance/stringinterp.lua new file mode 100644 index 00000000..efb25bae --- /dev/null +++ b/tests/conformance/stringinterp.lua @@ -0,0 +1,59 @@ +local function assertEq(left, right) + assert(typeof(left) == "string", "left is a " .. typeof(left)) + assert(typeof(right) == "string", "right is a " .. typeof(right)) + + if left ~= right then + error(string.format("%q ~= %q", left, right)) + end +end + +assertEq(`hello {"world"}`, "hello world") +assertEq(`Welcome {"to"} {"Luau"}!`, "Welcome to Luau!") + +assertEq(`2 + 2 = {2 + 2}`, "2 + 2 = 4") + +assertEq(`{1} {2} {3} {4} {5} {6} {7}`, "1 2 3 4 5 6 7") + +local combo = {5, 2, 8, 9} +assertEq(`The lock combinations are: {table.concat(combo, ", ")}`, "The lock combinations are: 5, 2, 8, 9") + +assertEq(`true = {true}`, "true = true") + +local name = "Luau" +assertEq(`Welcome to { + name +}!`, "Welcome to Luau!") + +local nameNotConstantEvaluated = (function() return "Luau" end)() +assertEq(`Welcome to {nameNotConstantEvaluated}!`, "Welcome to Luau!") + +assertEq(`This {localName} does not exist`, "This nil does not exist") + +assertEq(`Welcome to \ +{name}!`, "Welcome to \nLuau!") + +assertEq(`empty`, "empty") + +assertEq(`Escaped brace: \{}`, "Escaped brace: {}") +assertEq(`Escaped brace \{} with {"expression"}`, "Escaped brace {} with expression") +assertEq(`Backslash \ that escapes the space is not a part of the string...`, "Backslash that escapes the space is not a part of the string...") +assertEq(`Escaped backslash \\`, "Escaped backslash \\") +assertEq(`Escaped backtick: \``, "Escaped backtick: `") + +assertEq(`Hello {`from inside {"a nested string"}`}`, "Hello from inside a nested string") + +assertEq(`1 {`2 {`3 {4}`}`}`, "1 2 3 4") + +local health = 50 +assert(`You have {health}% health` == "You have 50% health") + +local function shadowsString(string) + return `Value is {string}` +end + +assertEq(shadowsString("hello"), "Value is hello") +assertEq(shadowsString(1), "Value is 1") + +assertEq(`\u{0041}\t`, "A\t") + +return "OK" From 3008da98df9bcd7ea8da70db007c0c0aa6359ba4 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Thu, 25 Aug 2022 13:55:08 -0700 Subject: [PATCH 349/399] Sync to upstream/release/542 --- Analysis/include/Luau/Constraint.h | 2 +- Analysis/include/Luau/ConstraintSolver.h | 2 + .../include/Luau/ConstraintSolverLogger.h | 2 +- Analysis/include/Luau/Frontend.h | 2 +- Analysis/include/Luau/ToString.h | 63 ++++- Analysis/include/Luau/TxnLog.h | 2 + Analysis/include/Luau/TypeInfer.h | 1 + Analysis/src/AstJsonEncoder.cpp | 14 + Analysis/src/ConstraintGraphBuilder.cpp | 11 +- Analysis/src/ConstraintSolver.cpp | 119 ++++++-- Analysis/src/ConstraintSolverLogger.cpp | 36 ++- Analysis/src/Frontend.cpp | 12 +- Analysis/src/Linter.cpp | 38 ++- Analysis/src/ToDot.cpp | 4 +- Analysis/src/ToString.cpp | 149 +++++----- Analysis/src/Transpiler.cpp | 22 ++ Analysis/src/TxnLog.cpp | 12 + Analysis/src/TypeInfer.cpp | 10 + Analysis/src/TypeVar.cpp | 17 +- Ast/include/Luau/Ast.h | 26 +- Ast/include/Luau/Lexer.h | 21 ++ Ast/include/Luau/Parser.h | 17 +- Ast/include/Luau/StringUtils.h | 2 +- Ast/src/Ast.cpp | 26 +- Ast/src/Lexer.cpp | 182 ++++++++++-- Ast/src/Parser.cpp | 179 +++++++++--- Ast/src/StringUtils.cpp | 10 +- Common/include/Luau/ExperimentalFlags.h | 1 + Compiler/src/Compiler.cpp | 94 ++++++- Compiler/src/ConstantFolding.cpp | 5 + Compiler/src/CostModel.cpp | 10 + Makefile | 8 +- VM/src/lapi.cpp | 69 ++--- VM/src/ldebug.cpp | 40 +-- VM/src/ldo.cpp | 8 +- VM/src/lfunc.cpp | 127 ++++++--- VM/src/lfunc.h | 1 + VM/src/lgc.cpp | 259 +++++++++++++++--- VM/src/lgc.h | 31 ++- VM/src/lgcdebug.cpp | 16 +- VM/src/lobject.h | 15 +- VM/src/lstate.cpp | 13 +- VM/src/lstate.h | 2 +- VM/src/lstring.cpp | 74 +++-- VM/src/lvmexecute.cpp | 15 +- VM/src/lvmload.cpp | 2 +- VM/src/lvmutils.cpp | 46 +--- bench/bench.py | 14 +- tests/AstJsonEncoder.test.cpp | 12 + tests/AstQuery.test.cpp | 76 +++++ tests/Autocomplete.test.cpp | 9 + tests/Compiler.test.cpp | 55 +++- tests/Conformance.test.cpp | 12 +- tests/Fixture.cpp | 37 +++ tests/Fixture.h | 70 +++++ tests/Lexer.test.cpp | 86 ++++++ tests/Linter.test.cpp | 38 ++- tests/Parser.test.cpp | 140 ++++++++++ tests/ToString.test.cpp | 52 ++-- tests/Transpiler.test.cpp | 19 ++ tests/TypeInfer.builtins.test.cpp | 26 +- tests/TypeInfer.provisional.test.cpp | 65 +++++ tests/TypeInfer.test.cpp | 36 +++ tests/conformance/gc.lua | 60 ++-- tests/conformance/stringinterp.lua | 59 ++++ tools/faillist.txt | 3 +- tools/natvis/VM.natvis | 2 +- tools/perfgraph.py | 76 ++++- tools/test_dcr.py | 31 ++- 69 files changed, 2284 insertions(+), 511 deletions(-) create mode 100644 tests/conformance/stringinterp.lua diff --git a/Analysis/include/Luau/Constraint.h b/Analysis/include/Luau/Constraint.h index ce90f2c5..e9f04e79 100644 --- a/Analysis/include/Luau/Constraint.h +++ b/Analysis/include/Luau/Constraint.h @@ -35,7 +35,7 @@ struct PackSubtypeConstraint TypePackId superPack; }; -// subType ~ gen superType +// generalizedType ~ gen sourceType struct GeneralizationConstraint { TypeId generalizedType; diff --git a/Analysis/include/Luau/ConstraintSolver.h b/Analysis/include/Luau/ConstraintSolver.h index 661d120e..a270ec99 100644 --- a/Analysis/include/Luau/ConstraintSolver.h +++ b/Analysis/include/Luau/ConstraintSolver.h @@ -100,6 +100,8 @@ struct ConstraintSolver void unblock(NotNull progressed); void unblock(TypeId progressed); void unblock(TypePackId progressed); + void unblock(const std::vector& types); + void unblock(const std::vector& packs); /** * @returns true if the TypeId is in a blocked state. diff --git a/Analysis/include/Luau/ConstraintSolverLogger.h b/Analysis/include/Luau/ConstraintSolverLogger.h index fe2177c4..55170c42 100644 --- a/Analysis/include/Luau/ConstraintSolverLogger.h +++ b/Analysis/include/Luau/ConstraintSolverLogger.h @@ -16,7 +16,7 @@ struct ConstraintSolverLogger { std::string compileOutput(); void captureBoundarySnapshot(const Scope* rootScope, std::vector>& unsolvedConstraints); - void prepareStepSnapshot(const Scope* rootScope, NotNull current, std::vector>& unsolvedConstraints); + void prepareStepSnapshot(const Scope* rootScope, NotNull current, std::vector>& unsolvedConstraints, bool force); void commitPreparedStepSnapshot(); private: diff --git a/Analysis/include/Luau/Frontend.h b/Analysis/include/Luau/Frontend.h index 82df493a..f8da3273 100644 --- a/Analysis/include/Luau/Frontend.h +++ b/Analysis/include/Luau/Frontend.h @@ -166,7 +166,7 @@ private: static LintResult classifyLints(const std::vector& warnings, const Config& config); - ScopePtr getModuleEnvironment(const SourceModule& module, const Config& config); + ScopePtr getModuleEnvironment(const SourceModule& module, const Config& config, bool forAutocomplete = false); std::unordered_map environments; std::unordered_map> builtinDefinitions; diff --git a/Analysis/include/Luau/ToString.h b/Analysis/include/Luau/ToString.h index a50fef78..eabbc2be 100644 --- a/Analysis/include/Luau/ToString.h +++ b/Analysis/include/Luau/ToString.h @@ -33,7 +33,8 @@ struct ToStringOptions bool indent = false; size_t maxTableLength = size_t(FInt::LuauTableTypeMaximumStringifierLength); // Only applied to TableTypeVars size_t maxTypeLength = size_t(FInt::LuauTypeMaximumStringifierLength); - std::optional nameMap; + ToStringNameMap nameMap; + std::optional DEPRECATED_nameMap; std::shared_ptr scope; // If present, module names will be added and types that are not available in scope will be marked as 'invalid' std::vector namedFunctionOverrideArgNames; // If present, named function argument names will be overridden }; @@ -41,7 +42,7 @@ struct ToStringOptions struct ToStringResult { std::string name; - ToStringNameMap nameMap; + ToStringNameMap DEPRECATED_nameMap; bool invalid = false; bool error = false; @@ -49,12 +50,24 @@ struct ToStringResult bool truncated = false; }; -ToStringResult toStringDetailed(TypeId ty, const ToStringOptions& opts = {}); -ToStringResult toStringDetailed(TypePackId ty, const ToStringOptions& opts = {}); +ToStringResult toStringDetailed(TypeId ty, ToStringOptions& opts); +ToStringResult toStringDetailed(TypePackId ty, ToStringOptions& opts); -std::string toString(TypeId ty, const ToStringOptions& opts); -std::string toString(TypePackId ty, const ToStringOptions& opts); -std::string toString(const Constraint& c, ToStringOptions& opts); +std::string toString(TypeId ty, ToStringOptions& opts); +std::string toString(TypePackId ty, ToStringOptions& opts); + +// These overloads are selected when a temporary ToStringOptions is passed. (eg +// via an initializer list) +inline std::string toString(TypePackId ty, ToStringOptions&& opts) +{ + // Delegate to the overload (TypePackId, ToStringOptions&) + return toString(ty, opts); +} +inline std::string toString(TypeId ty, ToStringOptions&& opts) +{ + // Delegate to the overload (TypeId, ToStringOptions&) + return toString(ty, opts); +} // These are offered as overloads rather than a default parameter so that they can be easily invoked from within the MSVC debugger. // You can use them in watch expressions! @@ -66,16 +79,42 @@ inline std::string toString(TypePackId ty) { return toString(ty, ToStringOptions{}); } -inline std::string toString(const Constraint& c) + +std::string toString(const Constraint& c, ToStringOptions& opts); + +inline std::string toString(const Constraint& c, ToStringOptions&& opts) { - ToStringOptions opts; return toString(c, opts); } -std::string toString(const TypeVar& tv, const ToStringOptions& opts = {}); -std::string toString(const TypePackVar& tp, const ToStringOptions& opts = {}); +inline std::string toString(const Constraint& c) +{ + return toString(c, ToStringOptions{}); +} -std::string toStringNamedFunction(const std::string& funcName, const FunctionTypeVar& ftv, const ToStringOptions& opts = {}); + +std::string toString(const TypeVar& tv, ToStringOptions& opts); +std::string toString(const TypePackVar& tp, ToStringOptions& opts); + +inline std::string toString(const TypeVar& tv) +{ + ToStringOptions opts; + return toString(tv, opts); +} + +inline std::string toString(const TypePackVar& tp) +{ + ToStringOptions opts; + return toString(tp, opts); +} + +std::string toStringNamedFunction(const std::string& funcName, const FunctionTypeVar& ftv, ToStringOptions& opts); + +inline std::string toStringNamedFunction(const std::string& funcName, const FunctionTypeVar& ftv) +{ + ToStringOptions opts; + return toStringNamedFunction(funcName, ftv, opts); +} // It could be useful to see the text representation of a type during a debugging session instead of exploring the content of the class // These functions will dump the type to stdout and can be evaluated in Watch/Immediate windows or as gdb/lldb expression diff --git a/Analysis/include/Luau/TxnLog.h b/Analysis/include/Luau/TxnLog.h index cd115e3b..016cc927 100644 --- a/Analysis/include/Luau/TxnLog.h +++ b/Analysis/include/Luau/TxnLog.h @@ -263,6 +263,8 @@ struct TxnLog return Luau::get_if(&ty->ty) != nullptr; } + std::pair, std::vector> getChanges() const; + private: // unique_ptr is used to give us stable pointers across insertions into the // map. Otherwise, it would be really easy to accidentally invalidate the diff --git a/Analysis/include/Luau/TypeInfer.h b/Analysis/include/Luau/TypeInfer.h index 80f9085f..e253eddf 100644 --- a/Analysis/include/Luau/TypeInfer.h +++ b/Analysis/include/Luau/TypeInfer.h @@ -107,6 +107,7 @@ struct TypeChecker WithPredicate checkExpr(const ScopePtr& scope, const AstExprTypeAssertion& expr); WithPredicate checkExpr(const ScopePtr& scope, const AstExprError& expr); WithPredicate checkExpr(const ScopePtr& scope, const AstExprIfElse& expr, std::optional expectedType = std::nullopt); + WithPredicate checkExpr(const ScopePtr& scope, const AstExprInterpString& expr); TypeId checkExprTable(const ScopePtr& scope, const AstExprTable& expr, const std::vector>& fieldTypes, std::optional expectedType); diff --git a/Analysis/src/AstJsonEncoder.cpp b/Analysis/src/AstJsonEncoder.cpp index 2897875d..8d589037 100644 --- a/Analysis/src/AstJsonEncoder.cpp +++ b/Analysis/src/AstJsonEncoder.cpp @@ -445,6 +445,14 @@ struct AstJsonEncoder : public AstVisitor }); } + void write(class AstExprInterpString* node) + { + writeNode(node, "AstExprInterpString", [&]() { + PROP(strings); + PROP(expressions); + }); + } + void write(class AstExprTable* node) { writeNode(node, "AstExprTable", [&]() { @@ -888,6 +896,12 @@ struct AstJsonEncoder : public AstVisitor return false; } + bool visit(class AstExprInterpString* node) override + { + write(node); + return false; + } + bool visit(class AstExprLocal* node) override { write(node); diff --git a/Analysis/src/ConstraintGraphBuilder.cpp b/Analysis/src/ConstraintGraphBuilder.cpp index c1e54df2..8f994740 100644 --- a/Analysis/src/ConstraintGraphBuilder.cpp +++ b/Analysis/src/ConstraintGraphBuilder.cpp @@ -210,7 +210,8 @@ void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatLocal* local) for (size_t i = 0; i < local->values.size; ++i) { - if (local->values.data[i]->is()) + AstExpr* value = local->values.data[i]; + if (value->is()) { // HACK: we leave nil-initialized things floating under the assumption that they will later be populated. // See the test TypeInfer/infer_locals_with_nil_value. @@ -218,7 +219,7 @@ void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatLocal* local) } else if (i == local->values.size - 1) { - TypePackId exprPack = checkPack(scope, local->values.data[i]); + TypePackId exprPack = checkPack(scope, value); if (i < local->vars.size) { @@ -229,7 +230,7 @@ void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatLocal* local) } else { - TypeId exprType = check(scope, local->values.data[i]); + TypeId exprType = check(scope, value); if (i < varTypes.size()) addConstraint(scope, SubtypeConstraint{varTypes[i], exprType}); } @@ -1107,9 +1108,7 @@ TypeId ConstraintGraphBuilder::resolveType(const ScopePtr& scope, AstType* ty, b if (topLevel) { - addConstraint(scope, TypeAliasExpansionConstraint{ - /* target */ result, - }); + addConstraint(scope, TypeAliasExpansionConstraint{ /* target */ result }); } } } diff --git a/Analysis/src/ConstraintSolver.cpp b/Analysis/src/ConstraintSolver.cpp index 6c6d2722..b2b1d472 100644 --- a/Analysis/src/ConstraintSolver.cpp +++ b/Analysis/src/ConstraintSolver.cpp @@ -11,6 +11,7 @@ LUAU_FASTFLAGVARIABLE(DebugLuauLogSolver, false); LUAU_FASTFLAGVARIABLE(DebugLuauLogSolverToJson, false); +LUAU_FASTFLAG(LuauFixNameMaps) namespace Luau { @@ -19,9 +20,17 @@ namespace Luau { for (const auto& [k, v] : scope->bindings) { - auto d = toStringDetailed(v.typeId, opts); - opts.nameMap = d.nameMap; - printf("\t%s : %s\n", k.c_str(), d.name.c_str()); + if (FFlag::LuauFixNameMaps) + { + auto d = toString(v.typeId, opts); + printf("\t%s : %s\n", k.c_str(), d.c_str()); + } + else + { + auto d = toStringDetailed(v.typeId, opts); + opts.DEPRECATED_nameMap = d.DEPRECATED_nameMap; + printf("\t%s : %s\n", k.c_str(), d.name.c_str()); + } } for (NotNull child : scope->children) @@ -212,12 +221,22 @@ void dump(NotNull rootScope, ToStringOptions& opts) void dump(ConstraintSolver* cs, ToStringOptions& opts) { printf("constraints:\n"); - for (const Constraint* c : cs->unsolvedConstraints) + for (NotNull c : cs->unsolvedConstraints) { - printf("\t%s\n", toString(*c, opts).c_str()); + auto it = cs->blockedConstraints.find(c); + int blockCount = it == cs->blockedConstraints.end() ? 0 : int(it->second); + printf("\t%d\t%s\n", blockCount, toString(*c, opts).c_str()); - for (const Constraint* dep : c->dependencies) - printf("\t\t%s\n", toString(*dep, opts).c_str()); + for (NotNull dep : c->dependencies) + { + auto unsolvedIter = std::find(begin(cs->unsolvedConstraints), end(cs->unsolvedConstraints), dep); + if (unsolvedIter == cs->unsolvedConstraints.end()) + continue; + + auto it = cs->blockedConstraints.find(dep); + int blockCount = it == cs->blockedConstraints.end() ? 0 : int(it->second); + printf("\t%d\t\t%s\n", blockCount, toString(*dep, opts).c_str()); + } } } @@ -273,7 +292,7 @@ void ConstraintSolver::run() if (FFlag::DebugLuauLogSolverToJson) { - logger.prepareStepSnapshot(rootScope, c, unsolvedConstraints); + logger.prepareStepSnapshot(rootScope, c, unsolvedConstraints, force); } bool success = tryDispatch(c, force); @@ -282,6 +301,7 @@ void ConstraintSolver::run() if (success) { + unblock(c); unsolvedConstraints.erase(unsolvedConstraints.begin() + i); if (FFlag::DebugLuauLogSolverToJson) @@ -375,18 +395,12 @@ bool ConstraintSolver::tryDispatch(const SubtypeConstraint& c, NotNullscope); - unblock(c.subType); - unblock(c.superType); - return true; } bool ConstraintSolver::tryDispatch(const PackSubtypeConstraint& c, NotNull constraint, bool force) { unify(c.subPack, c.superPack, constraint->scope); - unblock(c.subPack); - unblock(c.superPack); - return true; } @@ -395,13 +409,12 @@ bool ConstraintSolver::tryDispatch(const GeneralizationConstraint& c, NotNullty.emplace(c.sourceType); - else - unify(c.generalizedType, c.sourceType, constraint->scope); - TypeId generalized = quantify(arena, c.sourceType, constraint->scope); - *asMutable(c.sourceType) = *generalized; + + if (isBlocked(c.generalizedType)) + asMutable(c.generalizedType)->ty.emplace(generalized); + else + unify(c.generalizedType, generalized, constraint->scope); unblock(c.generalizedType); unblock(c.sourceType); @@ -455,23 +468,44 @@ bool ConstraintSolver::tryDispatch(const BinaryConstraint& c, NotNull + * + * This constraint is the one that is meant to unblock A, so it doesn't + * make any sense to stop and wait for someone else to do it. + */ + if (leftType != resultType && rightType != resultType) + { + block(c.leftType, constraint); + block(c.rightType, constraint); + return false; + } } if (isNumber(leftType)) { unify(leftType, rightType, constraint->scope); - asMutable(c.resultType)->ty.emplace(leftType); + asMutable(resultType)->ty.emplace(leftType); return true; } - if (get(leftType) && !force) - return block(leftType, constraint); + if (!force) + { + if (get(leftType)) + return block(leftType, constraint); + } + + if (isBlocked(leftType)) + { + asMutable(resultType)->ty.emplace(getSingletonTypes().errorRecoveryType()); + // reportError(constraint->location, CannotInferBinaryOperation{c.op, std::nullopt, CannotInferBinaryOperation::Operation}); + return true; + } // TODO metatables, classes @@ -706,17 +740,23 @@ void ConstraintSolver::block_(BlockedConstraintId target, NotNull target, NotNull constraint) { + if (FFlag::DebugLuauLogSolver) + printf("block Constraint %s on\t%s\n", toString(*target).c_str(), toString(*constraint).c_str()); block_(target, constraint); } bool ConstraintSolver::block(TypeId target, NotNull constraint) { + if (FFlag::DebugLuauLogSolver) + printf("block TypeId %s on\t%s\n", toString(target).c_str(), toString(*constraint).c_str()); block_(target, constraint); return false; } bool ConstraintSolver::block(TypePackId target, NotNull constraint) { + if (FFlag::DebugLuauLogSolver) + printf("block TypeId %s on\t%s\n", toString(target).c_str(), toString(*constraint).c_str()); block_(target, constraint); return false; } @@ -731,6 +771,9 @@ void ConstraintSolver::unblock_(BlockedConstraintId progressed) for (NotNull unblockedConstraint : it->second) { auto& count = blockedConstraints[unblockedConstraint]; + if (FFlag::DebugLuauLogSolver) + printf("Unblocking count=%d\t%s\n", int(count), toString(*unblockedConstraint).c_str()); + // This assertion being hit indicates that `blocked` and // `blockedConstraints` desynchronized at some point. This is problematic // because we rely on this count being correct to skip over blocked @@ -757,6 +800,18 @@ void ConstraintSolver::unblock(TypePackId progressed) return unblock_(progressed); } +void ConstraintSolver::unblock(const std::vector& types) +{ + for (TypeId t : types) + unblock(t); +} + +void ConstraintSolver::unblock(const std::vector& packs) +{ + for (TypePackId t : packs) + unblock(t); +} + bool ConstraintSolver::isBlocked(TypeId ty) { return nullptr != get(follow(ty)) || nullptr != get(follow(ty)); @@ -774,7 +829,13 @@ void ConstraintSolver::unify(TypeId subType, TypeId superType, NotNull sc Unifier u{arena, Mode::Strict, scope, Location{}, Covariant, sharedState}; u.tryUnify(subType, superType); + + const auto [changedTypes, changedPacks] = u.log.getChanges(); + u.log.commit(); + + unblock(changedTypes); + unblock(changedPacks); } void ConstraintSolver::unify(TypePackId subPack, TypePackId superPack, NotNull scope) @@ -783,7 +844,13 @@ void ConstraintSolver::unify(TypePackId subPack, TypePackId superPack, NotNull scope) diff --git a/Analysis/src/ConstraintSolverLogger.cpp b/Analysis/src/ConstraintSolverLogger.cpp index adb9c54e..097ceeec 100644 --- a/Analysis/src/ConstraintSolverLogger.cpp +++ b/Analysis/src/ConstraintSolverLogger.cpp @@ -4,6 +4,8 @@ #include "Luau/JsonEmitter.h" +LUAU_FASTFLAG(LuauFixNameMaps); + namespace Luau { @@ -17,9 +19,14 @@ static void dumpScopeAndChildren(const Scope* scope, Json::JsonEmitter& emitter, for (const auto& [name, binding] : scope->bindings) { - ToStringResult result = toStringDetailed(binding.typeId, opts); - opts.nameMap = std::move(result.nameMap); - o.writePair(name.c_str(), result.name); + if (FFlag::LuauFixNameMaps) + o.writePair(name.c_str(), toString(binding.typeId, opts)); + else + { + ToStringResult result = toStringDetailed(binding.typeId, opts); + opts.DEPRECATED_nameMap = std::move(result.DEPRECATED_nameMap); + o.writePair(name.c_str(), result.name); + } } o.finish(); @@ -30,6 +37,7 @@ static void dumpScopeAndChildren(const Scope* scope, Json::JsonEmitter& emitter, Json::ArrayEmitter a = emitter.writeArray(); for (const Scope* child : scope->children) { + emitter.writeComma(); dumpScopeAndChildren(child, emitter, opts); } @@ -39,7 +47,8 @@ static void dumpScopeAndChildren(const Scope* scope, Json::JsonEmitter& emitter, static std::string dumpConstraintsToDot(std::vector>& constraints, ToStringOptions& opts) { - std::string result = "digraph Constraints {\\n"; + std::string result = "digraph Constraints {\n"; + result += "rankdir=LR\n"; std::unordered_set> contained; for (NotNull c : constraints) @@ -49,11 +58,19 @@ static std::string dumpConstraintsToDot(std::vector>& for (NotNull c : constraints) { + std::string shape; + if (get(*c)) + shape = "box"; + else if (get(*c)) + shape = "box3d"; + else + shape = "oval"; + std::string id = std::to_string(reinterpret_cast(c.get())); result += id; - result += " [label=\\\""; - result += toString(*c, opts).c_str(); - result += "\\\"];\\n"; + result += " [label=\""; + result += toString(*c, opts); + result += "\" shape=" + shape + "];\n"; for (NotNull dep : c->dependencies) { @@ -63,7 +80,7 @@ static std::string dumpConstraintsToDot(std::vector>& result += std::to_string(reinterpret_cast(dep.get())); result += " -> "; result += id; - result += ";\\n"; + result += ";\n"; } } @@ -102,7 +119,7 @@ void ConstraintSolverLogger::captureBoundarySnapshot(const Scope* rootScope, std } void ConstraintSolverLogger::prepareStepSnapshot( - const Scope* rootScope, NotNull current, std::vector>& unsolvedConstraints) + const Scope* rootScope, NotNull current, std::vector>& unsolvedConstraints, bool force) { Json::JsonEmitter emitter; Json::ObjectEmitter o = emitter.writeObject(); @@ -110,6 +127,7 @@ void ConstraintSolverLogger::prepareStepSnapshot( o.writePair("constraintGraph", dumpConstraintsToDot(unsolvedConstraints, opts)); o.writePair("currentId", std::to_string(reinterpret_cast(current.get()))); o.writePair("current", toString(*current, opts)); + o.writePair("force", force); emitter.writeComma(); Json::write(emitter, "rootScope"); emitter.writeRaw(":"); diff --git a/Analysis/src/Frontend.cpp b/Analysis/src/Frontend.cpp index 8ab4e865..c8c5d4b5 100644 --- a/Analysis/src/Frontend.cpp +++ b/Analysis/src/Frontend.cpp @@ -455,7 +455,7 @@ CheckResult Frontend::check(const ModuleName& name, std::optional& buildQueue, CheckResult& chec return cyclic; } -ScopePtr Frontend::getModuleEnvironment(const SourceModule& module, const Config& config) +ScopePtr Frontend::getModuleEnvironment(const SourceModule& module, const Config& config, bool forAutocomplete) { - ScopePtr result = typeChecker.globalScope; + ScopePtr result; + if (forAutocomplete) + result = typeCheckerForAutocomplete.globalScope; + else + result = typeChecker.globalScope; if (module.environmentName) result = getEnvironmentScope(*module.environmentName); diff --git a/Analysis/src/Linter.cpp b/Analysis/src/Linter.cpp index 2d058376..426ff9d6 100644 --- a/Analysis/src/Linter.cpp +++ b/Analysis/src/Linter.cpp @@ -15,6 +15,7 @@ LUAU_FASTINTVARIABLE(LuauSuggestionDistance, 4) LUAU_FASTFLAGVARIABLE(LuauLintGlobalNeverReadBeforeWritten, false) LUAU_FASTFLAGVARIABLE(LuauLintComparisonPrecedence, false) +LUAU_FASTFLAGVARIABLE(LuauLintFixDeprecationMessage, false) namespace Luau { @@ -206,6 +207,24 @@ static bool similar(AstExpr* lhs, AstExpr* rhs) return true; } CASE(AstExprIfElse) return similar(le->condition, re->condition) && similar(le->trueExpr, re->trueExpr) && similar(le->falseExpr, re->falseExpr); + CASE(AstExprInterpString) + { + if (le->strings.size != re->strings.size) + return false; + + if (le->expressions.size != re->expressions.size) + return false; + + for (size_t i = 0; i < le->strings.size; ++i) + if (le->strings.data[i].size != re->strings.data[i].size || memcmp(le->strings.data[i].data, re->strings.data[i].data, le->strings.data[i].size) != 0) + return false; + + for (size_t i = 0; i < le->expressions.size; ++i) + if (!similar(le->expressions.data[i], re->expressions.data[i])) + return false; + + return true; + } else { LUAU_ASSERT(!"Unknown expression type"); @@ -288,11 +307,22 @@ private: emitWarning(*context, LintWarning::Code_UnknownGlobal, gv->location, "Unknown global '%s'", gv->name.value); else if (g->deprecated) { - if (*g->deprecated) - emitWarning(*context, LintWarning::Code_DeprecatedGlobal, gv->location, "Global '%s' is deprecated, use '%s' instead", - gv->name.value, *g->deprecated); + if (FFlag::LuauLintFixDeprecationMessage) + { + if (const char* replacement = *g->deprecated; replacement && strlen(replacement)) + emitWarning(*context, LintWarning::Code_DeprecatedGlobal, gv->location, "Global '%s' is deprecated, use '%s' instead", + gv->name.value, replacement); + else + emitWarning(*context, LintWarning::Code_DeprecatedGlobal, gv->location, "Global '%s' is deprecated", gv->name.value); + } else - emitWarning(*context, LintWarning::Code_DeprecatedGlobal, gv->location, "Global '%s' is deprecated", gv->name.value); + { + if (*g->deprecated) + emitWarning(*context, LintWarning::Code_DeprecatedGlobal, gv->location, "Global '%s' is deprecated, use '%s' instead", + gv->name.value, *g->deprecated); + else + emitWarning(*context, LintWarning::Code_DeprecatedGlobal, gv->location, "Global '%s' is deprecated", gv->name.value); + } } } diff --git a/Analysis/src/ToDot.cpp b/Analysis/src/ToDot.cpp index 6b677bb8..0d989ca0 100644 --- a/Analysis/src/ToDot.cpp +++ b/Analysis/src/ToDot.cpp @@ -73,7 +73,7 @@ void StateDot::visitChild(TypeId ty, int parentIndex, const char* linkName) if (opts.duplicatePrimitives && canDuplicatePrimitive(ty)) { if (get(ty)) - formatAppend(result, "n%d [label=\"%s\"];\n", index, toStringDetailed(ty, {}).name.c_str()); + formatAppend(result, "n%d [label=\"%s\"];\n", index, toString(ty).c_str()); else if (get(ty)) formatAppend(result, "n%d [label=\"any\"];\n", index); } @@ -233,7 +233,7 @@ void StateDot::visitChildren(TypeId ty, int index) } else if (get(ty)) { - formatAppend(result, "PrimitiveTypeVar %s", toStringDetailed(ty, {}).name.c_str()); + formatAppend(result, "PrimitiveTypeVar %s", toString(ty).c_str()); finishNodeLabel(ty); finishNode(); } diff --git a/Analysis/src/ToString.cpp b/Analysis/src/ToString.cpp index e31e690a..ace44cda 100644 --- a/Analysis/src/ToString.cpp +++ b/Analysis/src/ToString.cpp @@ -13,6 +13,7 @@ LUAU_FASTFLAG(LuauLowerBoundsCalculation) LUAU_FASTFLAG(LuauUnknownAndNeverType) LUAU_FASTFLAGVARIABLE(LuauSpecialTypesAsterisked, false) +LUAU_FASTFLAGVARIABLE(LuauFixNameMaps, false) /* * Prefix generic typenames with gen- @@ -116,7 +117,7 @@ static std::pair> canUseTypeNameInScope(ScopePtr struct StringifierState { - const ToStringOptions& opts; + ToStringOptions& opts; ToStringResult& result; std::unordered_map cycleNames; @@ -127,18 +128,28 @@ struct StringifierState bool exhaustive; - StringifierState(const ToStringOptions& opts, ToStringResult& result, const std::optional& nameMap) + StringifierState(ToStringOptions& opts, ToStringResult& result, const std::optional& DEPRECATED_nameMap) : opts(opts) , result(result) , exhaustive(opts.exhaustive) { - if (nameMap) - result.nameMap = *nameMap; + if (!FFlag::LuauFixNameMaps && DEPRECATED_nameMap) + result.DEPRECATED_nameMap = *DEPRECATED_nameMap; - for (const auto& [_, v] : result.nameMap.typeVars) - usedNames.insert(v); - for (const auto& [_, v] : result.nameMap.typePacks) - usedNames.insert(v); + if (!FFlag::LuauFixNameMaps) + { + for (const auto& [_, v] : result.DEPRECATED_nameMap.typeVars) + usedNames.insert(v); + for (const auto& [_, v] : result.DEPRECATED_nameMap.typePacks) + usedNames.insert(v); + } + else + { + for (const auto& [_, v] : opts.nameMap.typeVars) + usedNames.insert(v); + for (const auto& [_, v] : opts.nameMap.typePacks) + usedNames.insert(v); + } } bool hasSeen(const void* tv) @@ -161,8 +172,8 @@ struct StringifierState std::string getName(TypeId ty) { - const size_t s = result.nameMap.typeVars.size(); - std::string& n = result.nameMap.typeVars[ty]; + const size_t s = FFlag::LuauFixNameMaps ? opts.nameMap.typeVars.size() : result.DEPRECATED_nameMap.typeVars.size(); + std::string& n = FFlag::LuauFixNameMaps ? opts.nameMap.typeVars[ty] : result.DEPRECATED_nameMap.typeVars[ty]; if (!n.empty()) return n; @@ -184,8 +195,8 @@ struct StringifierState std::string getName(TypePackId ty) { - const size_t s = result.nameMap.typePacks.size(); - std::string& n = result.nameMap.typePacks[ty]; + const size_t s = FFlag::LuauFixNameMaps ? opts.nameMap.typePacks.size() : result.DEPRECATED_nameMap.typePacks.size(); + std::string& n = FFlag::LuauFixNameMaps ? opts.nameMap.typePacks[ty] : result.DEPRECATED_nameMap.typePacks[ty]; if (!n.empty()) return n; @@ -377,7 +388,10 @@ struct TypeVarStringifier if (gtv.explicitName) { state.usedNames.insert(gtv.name); - state.result.nameMap.typeVars[ty] = gtv.name; + if (FFlag::LuauFixNameMaps) + state.opts.nameMap.typeVars[ty] = gtv.name; + else + state.result.DEPRECATED_nameMap.typeVars[ty] = gtv.name; state.emit(gtv.name); } else @@ -987,7 +1001,10 @@ struct TypePackStringifier if (pack.explicitName) { state.usedNames.insert(pack.name); - state.result.nameMap.typePacks[tp] = pack.name; + if (FFlag::LuauFixNameMaps) + state.opts.nameMap.typePacks[tp] = pack.name; + else + state.result.DEPRECATED_nameMap.typePacks[tp] = pack.name; state.emit(pack.name); } else @@ -1066,7 +1083,7 @@ static void assignCycleNames(const std::set& cycles, const std::set cycles; std::set cycleTPs; @@ -1176,7 +1195,7 @@ ToStringResult toStringDetailed(TypeId ty, const ToStringOptions& opts) return result; } -ToStringResult toStringDetailed(TypePackId tp, const ToStringOptions& opts) +ToStringResult toStringDetailed(TypePackId tp, ToStringOptions& opts) { /* * 1. Walk the TypeVar and track seen TypeIds. When you reencounter a TypeId, add it to a set of seen cycles. @@ -1185,7 +1204,9 @@ ToStringResult toStringDetailed(TypePackId tp, const ToStringOptions& opts) * 4. Print out the root of the type using the same algorithm as step 3. */ ToStringResult result; - StringifierState state{opts, result, opts.nameMap}; + StringifierState state = FFlag::LuauFixNameMaps + ? StringifierState{opts, result, opts.nameMap} + : StringifierState{opts, result, opts.DEPRECATED_nameMap}; std::set cycles; std::set cycleTPs; @@ -1248,30 +1269,32 @@ ToStringResult toStringDetailed(TypePackId tp, const ToStringOptions& opts) return result; } -std::string toString(TypeId ty, const ToStringOptions& opts) +std::string toString(TypeId ty, ToStringOptions& opts) { return toStringDetailed(ty, opts).name; } -std::string toString(TypePackId tp, const ToStringOptions& opts) +std::string toString(TypePackId tp, ToStringOptions& opts) { return toStringDetailed(tp, opts).name; } -std::string toString(const TypeVar& tv, const ToStringOptions& opts) +std::string toString(const TypeVar& tv, ToStringOptions& opts) { - return toString(const_cast(&tv), std::move(opts)); + return toString(const_cast(&tv), opts); } -std::string toString(const TypePackVar& tp, const ToStringOptions& opts) +std::string toString(const TypePackVar& tp, ToStringOptions& opts) { - return toString(const_cast(&tp), std::move(opts)); + return toString(const_cast(&tp), opts); } -std::string toStringNamedFunction(const std::string& funcName, const FunctionTypeVar& ftv, const ToStringOptions& opts) +std::string toStringNamedFunction(const std::string& funcName, const FunctionTypeVar& ftv, ToStringOptions& opts) { ToStringResult result; - StringifierState state(opts, result, opts.nameMap); + StringifierState state = FFlag::LuauFixNameMaps + ? StringifierState{opts, result, opts.nameMap} + : StringifierState{opts, result, opts.DEPRECATED_nameMap}; TypeVarStringifier tvs{state}; state.emit(funcName); @@ -1403,69 +1426,67 @@ std::string toString(const Constraint& constraint, ToStringOptions& opts) auto go = [&opts](auto&& c) { using T = std::decay_t; + // TODO: Inline and delete this function when clipping FFlag::LuauFixNameMaps + auto tos = [](auto&& a, ToStringOptions& opts) + { + if (FFlag::LuauFixNameMaps) + return toString(a, opts); + else + { + ToStringResult tsr = toStringDetailed(a, opts); + opts.DEPRECATED_nameMap = std::move(tsr.DEPRECATED_nameMap); + return tsr.name; + } + }; + if constexpr (std::is_same_v) { - ToStringResult subStr = toStringDetailed(c.subType, opts); - opts.nameMap = std::move(subStr.nameMap); - ToStringResult superStr = toStringDetailed(c.superType, opts); - opts.nameMap = std::move(superStr.nameMap); - return subStr.name + " <: " + superStr.name; + std::string subStr = tos(c.subType, opts); + std::string superStr = tos(c.superType, opts); + return subStr + " <: " + superStr; } else if constexpr (std::is_same_v) { - ToStringResult subStr = toStringDetailed(c.subPack, opts); - opts.nameMap = std::move(subStr.nameMap); - ToStringResult superStr = toStringDetailed(c.superPack, opts); - opts.nameMap = std::move(superStr.nameMap); - return subStr.name + " <: " + superStr.name; + std::string subStr = tos(c.subPack, opts); + std::string superStr = tos(c.superPack, opts); + return subStr + " <: " + superStr; } else if constexpr (std::is_same_v) { - ToStringResult subStr = toStringDetailed(c.generalizedType, opts); - opts.nameMap = std::move(subStr.nameMap); - ToStringResult superStr = toStringDetailed(c.sourceType, opts); - opts.nameMap = std::move(superStr.nameMap); - return subStr.name + " ~ gen " + superStr.name; + std::string subStr = tos(c.generalizedType, opts); + std::string superStr = tos(c.sourceType, opts); + return subStr + " ~ gen " + superStr; } else if constexpr (std::is_same_v) { - ToStringResult subStr = toStringDetailed(c.subType, opts); - opts.nameMap = std::move(subStr.nameMap); - ToStringResult superStr = toStringDetailed(c.superType, opts); - opts.nameMap = std::move(superStr.nameMap); - return subStr.name + " ~ inst " + superStr.name; + std::string subStr = tos(c.subType, opts); + std::string superStr = tos(c.superType, opts); + return subStr + " ~ inst " + superStr; } else if constexpr (std::is_same_v) { - ToStringResult resultStr = toStringDetailed(c.resultType, opts); - opts.nameMap = std::move(resultStr.nameMap); - ToStringResult operandStr = toStringDetailed(c.operandType, opts); - opts.nameMap = std::move(operandStr.nameMap); + std::string resultStr = tos(c.resultType, opts); + std::string operandStr = tos(c.operandType, opts); - return resultStr.name + " ~ Unary<" + toString(c.op) + ", " + operandStr.name + ">"; + return resultStr + " ~ Unary<" + toString(c.op) + ", " + operandStr + ">"; } else if constexpr (std::is_same_v) { - ToStringResult resultStr = toStringDetailed(c.resultType); - opts.nameMap = std::move(resultStr.nameMap); - ToStringResult leftStr = toStringDetailed(c.leftType); - opts.nameMap = std::move(leftStr.nameMap); - ToStringResult rightStr = toStringDetailed(c.rightType); - opts.nameMap = std::move(rightStr.nameMap); + std::string resultStr = tos(c.resultType, opts); + std::string leftStr = tos(c.leftType, opts); + std::string rightStr = tos(c.rightType, opts); - return resultStr.name + " ~ Binary<" + toString(c.op) + ", " + leftStr.name + ", " + rightStr.name + ">"; + return resultStr + " ~ Binary<" + toString(c.op) + ", " + leftStr + ", " + rightStr + ">"; } else if constexpr (std::is_same_v) { - ToStringResult namedStr = toStringDetailed(c.namedType, opts); - opts.nameMap = std::move(namedStr.nameMap); - return "@name(" + namedStr.name + ") = " + c.name; + std::string namedStr = tos(c.namedType, opts); + return "@name(" + namedStr + ") = " + c.name; } else if constexpr (std::is_same_v) { - ToStringResult targetStr = toStringDetailed(c.target, opts); - opts.nameMap = std::move(targetStr.nameMap); - return "expand " + targetStr.name; + std::string targetStr = tos(c.target, opts); + return "expand " + targetStr; } else static_assert(always_false_v, "Non-exhaustive constraint switch"); diff --git a/Analysis/src/Transpiler.cpp b/Analysis/src/Transpiler.cpp index 9feff1c0..cdfe6549 100644 --- a/Analysis/src/Transpiler.cpp +++ b/Analysis/src/Transpiler.cpp @@ -511,6 +511,28 @@ struct Printer writer.keyword("else"); visualize(*a->falseExpr); } + else if (const auto& a = expr.as()) + { + writer.symbol("`"); + + size_t index = 0; + + for (const auto& string : a->strings) + { + writer.write(escape(std::string_view(string.data, string.size), /* escapeForInterpString = */ true)); + + if (index < a->expressions.size) + { + writer.symbol("{"); + visualize(*a->expressions.data[index]); + writer.symbol("}"); + } + + index++; + } + + writer.symbol("`"); + } else if (const auto& a = expr.as()) { writer.symbol("(error-expr"); diff --git a/Analysis/src/TxnLog.cpp b/Analysis/src/TxnLog.cpp index b3f60d30..74d77307 100644 --- a/Analysis/src/TxnLog.cpp +++ b/Analysis/src/TxnLog.cpp @@ -344,4 +344,16 @@ TypePackId TxnLog::follow(TypePackId tp) const }); } +std::pair, std::vector> TxnLog::getChanges() const +{ + std::pair, std::vector> result; + + for (const auto& [typeId, _newState] : typeVarChanges) + result.first.push_back(typeId); + for (const auto& [typePackId, _newState] : typePackChanges) + result.second.push_back(typePackId); + + return result; +} + } // namespace Luau diff --git a/Analysis/src/TypeInfer.cpp b/Analysis/src/TypeInfer.cpp index 9886fb19..77168053 100644 --- a/Analysis/src/TypeInfer.cpp +++ b/Analysis/src/TypeInfer.cpp @@ -1805,6 +1805,8 @@ WithPredicate TypeChecker::checkExpr(const ScopePtr& scope, const AstExp result = checkExpr(scope, *a); else if (auto a = expr.as()) result = checkExpr(scope, *a, expectedType); + else if (auto a = expr.as()) + result = checkExpr(scope, *a); else ice("Unhandled AstExpr?"); @@ -2999,6 +3001,14 @@ WithPredicate TypeChecker::checkExpr(const ScopePtr& scope, const AstExp return {types.size() == 1 ? types[0] : addType(UnionTypeVar{std::move(types)})}; } +WithPredicate TypeChecker::checkExpr(const ScopePtr& scope, const AstExprInterpString& expr) +{ + for (AstExpr* expr : expr.expressions) + checkExpr(scope, *expr); + + return {stringType}; +} + TypeId TypeChecker::checkLValue(const ScopePtr& scope, const AstExpr& expr) { return checkLValueBinding(scope, expr); diff --git a/Analysis/src/TypeVar.cpp b/Analysis/src/TypeVar.cpp index 9020b1ac..8974f8c7 100644 --- a/Analysis/src/TypeVar.cpp +++ b/Analysis/src/TypeVar.cpp @@ -27,6 +27,7 @@ LUAU_FASTFLAG(LuauUnknownAndNeverType) LUAU_FASTFLAGVARIABLE(LuauDeduceGmatchReturnTypes, false) LUAU_FASTFLAGVARIABLE(LuauMaybeGenericIntersectionTypes, false) LUAU_FASTFLAGVARIABLE(LuauDeduceFindMatchReturnTypes, false) +LUAU_FASTFLAGVARIABLE(LuauStringFormatArgumentErrorFix, false) namespace Luau { @@ -1139,11 +1140,21 @@ std::optional> magicFunctionFormat( } // if we know the argument count or if we have too many arguments for sure, we can issue an error - size_t actualParamSize = params.size() - paramOffset; + if (FFlag::LuauStringFormatArgumentErrorFix) + { + size_t numActualParams = params.size(); + size_t numExpectedParams = expected.size() + 1; // + 1 for the format string - if (expected.size() != actualParamSize && (!tail || expected.size() < actualParamSize)) - typechecker.reportError(TypeError{expr.location, CountMismatch{expected.size(), actualParamSize}}); + if (numExpectedParams != numActualParams && (!tail || numExpectedParams < numActualParams)) + typechecker.reportError(TypeError{expr.location, CountMismatch{numExpectedParams, numActualParams}}); + } + else + { + size_t actualParamSize = params.size() - paramOffset; + if (expected.size() != actualParamSize && (!tail || expected.size() < actualParamSize)) + typechecker.reportError(TypeError{expr.location, CountMismatch{expected.size(), actualParamSize}}); + } return WithPredicate{arena.addTypePack({typechecker.stringType})}; } diff --git a/Ast/include/Luau/Ast.h b/Ast/include/Luau/Ast.h index 1e164d04..612283fb 100644 --- a/Ast/include/Luau/Ast.h +++ b/Ast/include/Luau/Ast.h @@ -134,6 +134,10 @@ public: { return visit((class AstExpr*)node); } + virtual bool visit(class AstExprInterpString* node) + { + return visit((class AstExpr*)node); + } virtual bool visit(class AstExprError* node) { return visit((class AstExpr*)node); @@ -594,9 +598,9 @@ public: LUAU_RTTI(AstExprFunction) AstExprFunction(const Location& location, const AstArray& generics, const AstArray& genericPacks, - AstLocal* self, const AstArray& args, std::optional vararg, AstStatBlock* body, size_t functionDepth, - const AstName& debugname, std::optional returnAnnotation = {}, AstTypePack* varargAnnotation = nullptr, bool hasEnd = false, - std::optional argLocation = std::nullopt); + AstLocal* self, const AstArray& args, bool vararg, const Location& varargLocation, AstStatBlock* body, size_t functionDepth, + const AstName& debugname, const std::optional& returnAnnotation = {}, AstTypePack* varargAnnotation = nullptr, + bool hasEnd = false, const std::optional& argLocation = std::nullopt); void visit(AstVisitor* visitor) override; @@ -732,6 +736,22 @@ public: AstExpr* falseExpr; }; +class AstExprInterpString : public AstExpr +{ +public: + LUAU_RTTI(AstExprInterpString) + + AstExprInterpString(const Location& location, const AstArray>& strings, const AstArray& expressions); + + void visit(AstVisitor* visitor) override; + + /// An interpolated string such as `foo{bar}baz` is represented as + /// an array of strings for "foo" and "bar", and an array of expressions for "baz". + /// `strings` will always have one more element than `expressions`. + AstArray> strings; + AstArray expressions; +}; + class AstStatBlock : public AstStat { public: diff --git a/Ast/include/Luau/Lexer.h b/Ast/include/Luau/Lexer.h index 4f3dbbd5..7e7fe76b 100644 --- a/Ast/include/Luau/Lexer.h +++ b/Ast/include/Luau/Lexer.h @@ -61,6 +61,12 @@ struct Lexeme SkinnyArrow, DoubleColon, + InterpStringBegin, + InterpStringMid, + InterpStringEnd, + // An interpolated string with no expressions (like `x`) + InterpStringSimple, + AddAssign, SubAssign, MulAssign, @@ -80,6 +86,8 @@ struct Lexeme BrokenString, BrokenComment, BrokenUnicode, + BrokenInterpDoubleBrace, + Error, Reserved_BEGIN, @@ -208,6 +216,11 @@ private: Lexeme readLongString(const Position& start, int sep, Lexeme::Type ok, Lexeme::Type broken); Lexeme readQuotedString(); + Lexeme readInterpolatedStringBegin(); + Lexeme readInterpolatedStringSection(Position start, Lexeme::Type formatType, Lexeme::Type endType); + + void readBackslashInString(); + std::pair readName(); Lexeme readNumber(const Position& start, unsigned int startOffset); @@ -231,6 +244,14 @@ private: bool skipComments; bool readNames; + + enum class BraceType + { + InterpolatedString, + Normal + }; + + std::vector braceStack; }; inline bool isSpace(char ch) diff --git a/Ast/include/Luau/Parser.h b/Ast/include/Luau/Parser.h index 046706d3..956fcf64 100644 --- a/Ast/include/Luau/Parser.h +++ b/Ast/include/Luau/Parser.h @@ -11,6 +11,7 @@ #include #include +#include namespace Luau { @@ -109,8 +110,10 @@ private: // for namelist in explist do block end | AstStat* parseFor(); - // function funcname funcbody | // funcname ::= Name {`.' Name} [`:' Name] + AstExpr* parseFunctionName(Location start, bool& hasself, AstName& debugname); + + // function funcname funcbody AstStat* parseFunctionStat(); // local function Name funcbody | @@ -135,8 +138,10 @@ private: // var [`+=' | `-=' | `*=' | `/=' | `%=' | `^=' | `..='] exp AstStat* parseCompoundAssignment(AstExpr* initial, AstExprBinary::Op op); - // funcbody ::= `(' [parlist] `)' block end - // parlist ::= namelist [`,' `...'] | `...' + std::pair> prepareFunctionArguments(const Location& start, bool hasself, const TempVector& args); + + // funcbodyhead ::= `(' [namelist [`,' `...'] | `...'] `)' [`:` TypeAnnotation] + // funcbody ::= funcbodyhead block end std::pair parseFunctionBody( bool hasself, const Lexeme& matchFunction, const AstName& debugname, const Name* localName); @@ -148,7 +153,7 @@ private: // bindinglist ::= (binding | `...') {`,' bindinglist} // Returns the location of the vararg ..., or std::nullopt if the function is not vararg. - std::pair, AstTypePack*> parseBindingList(TempVector& result, bool allowDot3 = false); + std::tuple parseBindingList(TempVector& result, bool allowDot3 = false); AstType* parseOptionalTypeAnnotation(); @@ -228,6 +233,9 @@ private: // TODO: Add grammar rules here? AstExpr* parseIfElseExpr(); + // stringinterp ::= exp { exp} + AstExpr* parseInterpString(); + // Name std::optional parseNameOpt(const char* context = nullptr); Name parseName(const char* context = nullptr); @@ -379,6 +387,7 @@ private: std::vector matchRecoveryStopOnToken; std::vector scratchStat; + std::vector> scratchString; std::vector scratchExpr; std::vector scratchExprAux; std::vector scratchName; diff --git a/Ast/include/Luau/StringUtils.h b/Ast/include/Luau/StringUtils.h index 6ae9e977..dab76106 100644 --- a/Ast/include/Luau/StringUtils.h +++ b/Ast/include/Luau/StringUtils.h @@ -35,6 +35,6 @@ bool equalsLower(std::string_view lhs, std::string_view rhs); size_t hashRange(const char* data, size_t size); -std::string escape(std::string_view s); +std::string escape(std::string_view s, bool escapeForInterpString = false); bool isIdentifier(std::string_view s); } // namespace Luau diff --git a/Ast/src/Ast.cpp b/Ast/src/Ast.cpp index 3066b756..8291a5b1 100644 --- a/Ast/src/Ast.cpp +++ b/Ast/src/Ast.cpp @@ -160,17 +160,17 @@ void AstExprIndexExpr::visit(AstVisitor* visitor) } AstExprFunction::AstExprFunction(const Location& location, const AstArray& generics, const AstArray& genericPacks, - AstLocal* self, const AstArray& args, std::optional vararg, AstStatBlock* body, size_t functionDepth, - const AstName& debugname, std::optional returnAnnotation, AstTypePack* varargAnnotation, bool hasEnd, - std::optional argLocation) + AstLocal* self, const AstArray& args, bool vararg, const Location& varargLocation, AstStatBlock* body, size_t functionDepth, + const AstName& debugname, const std::optional& returnAnnotation, AstTypePack* varargAnnotation, bool hasEnd, + const std::optional& argLocation) : AstExpr(ClassIndex(), location) , generics(generics) , genericPacks(genericPacks) , self(self) , args(args) , returnAnnotation(returnAnnotation) - , vararg(vararg.has_value()) - , varargLocation(vararg.value_or(Location())) + , vararg(vararg) + , varargLocation(varargLocation) , varargAnnotation(varargAnnotation) , body(body) , functionDepth(functionDepth) @@ -349,6 +349,22 @@ AstExprError::AstExprError(const Location& location, const AstArray& e { } +AstExprInterpString::AstExprInterpString(const Location& location, const AstArray>& strings, const AstArray& expressions) + : AstExpr(ClassIndex(), location) + , strings(strings) + , expressions(expressions) +{ +} + +void AstExprInterpString::visit(AstVisitor* visitor) +{ + if (visitor->visit(this)) + { + for (AstExpr* expr : expressions) + expr->visit(visitor); + } +} + void AstExprError::visit(AstVisitor* visitor) { if (visitor->visit(this)) diff --git a/Ast/src/Lexer.cpp b/Ast/src/Lexer.cpp index a1f1d469..b4db8bdf 100644 --- a/Ast/src/Lexer.cpp +++ b/Ast/src/Lexer.cpp @@ -6,6 +6,8 @@ #include +LUAU_FASTFLAG(LuauInterpolatedStringBaseSupport) + namespace Luau { @@ -89,7 +91,18 @@ Lexeme::Lexeme(const Location& location, Type type, const char* data, size_t siz , length(unsigned(size)) , data(data) { - LUAU_ASSERT(type == RawString || type == QuotedString || type == Number || type == Comment || type == BlockComment); + LUAU_ASSERT( + type == RawString + || type == QuotedString + || type == InterpStringBegin + || type == InterpStringMid + || type == InterpStringEnd + || type == InterpStringSimple + || type == BrokenInterpDoubleBrace + || type == Number + || type == Comment + || type == BlockComment + ); } Lexeme::Lexeme(const Location& location, Type type, const char* name) @@ -160,6 +173,18 @@ std::string Lexeme::toString() const case QuotedString: return data ? format("\"%.*s\"", length, data) : "string"; + case InterpStringBegin: + return data ? format("`%.*s{", length, data) : "the beginning of an interpolated string"; + + case InterpStringMid: + return data ? format("}%.*s{", length, data) : "the middle of an interpolated string"; + + case InterpStringEnd: + return data ? format("}%.*s`", length, data) : "the end of an interpolated string"; + + case InterpStringSimple: + return data ? format("`%.*s`", length, data) : "interpolated string"; + case Number: return data ? format("'%.*s'", length, data) : "number"; @@ -175,6 +200,9 @@ std::string Lexeme::toString() const case BrokenComment: return "unfinished comment"; + case BrokenInterpDoubleBrace: + return "'{{', which is invalid (did you mean '\\{'?)"; + case BrokenUnicode: if (codepoint) { @@ -515,6 +543,32 @@ Lexeme Lexer::readLongString(const Position& start, int sep, Lexeme::Type ok, Le return Lexeme(Location(start, position()), broken); } +void Lexer::readBackslashInString() +{ + LUAU_ASSERT(peekch() == '\\'); + consume(); + switch (peekch()) + { + case '\r': + consume(); + if (peekch() == '\n') + consume(); + break; + + case 0: + break; + + case 'z': + consume(); + while (isSpace(peekch())) + consume(); + break; + + default: + consume(); + } +} + Lexeme Lexer::readQuotedString() { Position start = position(); @@ -535,27 +589,7 @@ Lexeme Lexer::readQuotedString() return Lexeme(Location(start, position()), Lexeme::BrokenString); case '\\': - consume(); - switch (peekch()) - { - case '\r': - consume(); - if (peekch() == '\n') - consume(); - break; - - case 0: - break; - - case 'z': - consume(); - while (isSpace(peekch())) - consume(); - break; - - default: - consume(); - } + readBackslashInString(); break; default: @@ -568,6 +602,69 @@ Lexeme Lexer::readQuotedString() return Lexeme(Location(start, position()), Lexeme::QuotedString, &buffer[startOffset], offset - startOffset - 1); } +Lexeme Lexer::readInterpolatedStringBegin() +{ + LUAU_ASSERT(peekch() == '`'); + + Position start = position(); + consume(); + + return readInterpolatedStringSection(start, Lexeme::InterpStringBegin, Lexeme::InterpStringSimple); +} + +Lexeme Lexer::readInterpolatedStringSection(Position start, Lexeme::Type formatType, Lexeme::Type endType) +{ + unsigned int startOffset = offset; + + while (peekch() != '`') + { + switch (peekch()) + { + case 0: + case '\r': + case '\n': + return Lexeme(Location(start, position()), Lexeme::BrokenString); + + case '\\': + // Allow for \u{}, which would otherwise be consumed by looking for { + if (peekch(1) == 'u' && peekch(2) == '{') + { + consume(); // backslash + consume(); // u + consume(); // { + break; + } + + readBackslashInString(); + break; + + case '{': + { + braceStack.push_back(BraceType::InterpolatedString); + + if (peekch(1) == '{') + { + Lexeme brokenDoubleBrace = Lexeme(Location(start, position()), Lexeme::BrokenInterpDoubleBrace, &buffer[startOffset], offset - startOffset); + consume(); + consume(); + return brokenDoubleBrace; + } + + Lexeme lexemeOutput(Location(start, position()), Lexeme::InterpStringBegin, &buffer[startOffset], offset - startOffset); + consume(); + return lexemeOutput; + } + + default: + consume(); + } + } + + consume(); + + return Lexeme(Location(start, position()), endType, &buffer[startOffset], offset - startOffset - 1); +} + Lexeme Lexer::readNumber(const Position& start, unsigned int startOffset) { LUAU_ASSERT(isDigit(peekch())); @@ -660,6 +757,36 @@ Lexeme Lexer::readNext() } } + case '{': + { + consume(); + + if (!braceStack.empty()) + braceStack.push_back(BraceType::Normal); + + return Lexeme(Location(start, 1), '{'); + } + + case '}': + { + consume(); + + if (braceStack.empty()) + { + return Lexeme(Location(start, 1), '}'); + } + + const BraceType braceStackTop = braceStack.back(); + braceStack.pop_back(); + + if (braceStackTop != BraceType::InterpolatedString) + { + return Lexeme(Location(start, 1), '}'); + } + + return readInterpolatedStringSection(position(), Lexeme::InterpStringMid, Lexeme::InterpStringEnd); + } + case '=': { consume(); @@ -716,6 +843,15 @@ Lexeme Lexer::readNext() case '\'': return readQuotedString(); + case '`': + if (FFlag::LuauInterpolatedStringBaseSupport) + return readInterpolatedStringBegin(); + else + { + consume(); + return Lexeme(Location(start, 1), '`'); + } + case '.': consume(); @@ -817,8 +953,6 @@ Lexeme Lexer::readNext() case '(': case ')': - case '{': - case '}': case ']': case ';': case ',': diff --git a/Ast/src/Parser.cpp b/Ast/src/Parser.cpp index e46eebf3..b6de27da 100644 --- a/Ast/src/Parser.cpp +++ b/Ast/src/Parser.cpp @@ -23,10 +23,14 @@ LUAU_FASTFLAGVARIABLE(LuauErrorDoubleHexPrefix, false) LUAU_FASTFLAGVARIABLE(LuauLintParseIntegerIssues, false) LUAU_DYNAMIC_FASTFLAGVARIABLE(LuaReportParseIntegerIssues, false) +LUAU_FASTFLAGVARIABLE(LuauInterpolatedStringBaseSupport, false) + bool lua_telemetry_parsed_out_of_range_bin_integer = false; bool lua_telemetry_parsed_out_of_range_hex_integer = false; bool lua_telemetry_parsed_double_prefix_hex_integer = false; +#define ERROR_INVALID_INTERP_DOUBLE_BRACE "Double braces are not permitted within interpolated strings. Did you mean '\\{'?" + namespace Luau { @@ -601,16 +605,11 @@ AstStat* Parser::parseFor() } } -// function funcname funcbody | // funcname ::= Name {`.' Name} [`:' Name] -AstStat* Parser::parseFunctionStat() +AstExpr* Parser::parseFunctionName(Location start, bool& hasself, AstName& debugname) { - Location start = lexer.current().location; - - Lexeme matchFunction = lexer.current(); - nextLexeme(); - - AstName debugname = (lexer.current().type == Lexeme::Name) ? AstName(lexer.current().name) : AstName(); + if (lexer.current().type == Lexeme::Name) + debugname = AstName(lexer.current().name); // parse funcname into a chain of indexing operators AstExpr* expr = parseNameExpr("function name"); @@ -636,8 +635,6 @@ AstStat* Parser::parseFunctionStat() recursionCounter = recursionCounterOld; // finish with : - bool hasself = false; - if (lexer.current().type == ':') { Position opPosition = lexer.current().location.begin; @@ -653,6 +650,21 @@ AstStat* Parser::parseFunctionStat() hasself = true; } + return expr; +} + +// function funcname funcbody +AstStat* Parser::parseFunctionStat() +{ + Location start = lexer.current().location; + + Lexeme matchFunction = lexer.current(); + nextLexeme(); + + bool hasself = false; + AstName debugname; + AstExpr* expr = parseFunctionName(start, hasself, debugname); + matchRecoveryStopOnToken[Lexeme::ReservedEnd]++; AstExprFunction* body = parseFunctionBody(hasself, matchFunction, debugname, nullptr).first; @@ -781,10 +793,11 @@ AstDeclaredClassProp Parser::parseDeclaredClassMethod() TempVector args(scratchBinding); - std::optional vararg = std::nullopt; + bool vararg = false; + Location varargLocation; AstTypePack* varargAnnotation = nullptr; if (lexer.current().type != ')') - std::tie(vararg, varargAnnotation) = parseBindingList(args, /* allowDot3 */ true); + std::tie(vararg, varargLocation, varargAnnotation) = parseBindingList(args, /* allowDot3 */ true); expectMatchAndConsume(')', matchParen); @@ -838,11 +851,12 @@ AstStat* Parser::parseDeclaration(const Location& start) TempVector args(scratchBinding); - std::optional vararg; + bool vararg = false; + Location varargLocation; AstTypePack* varargAnnotation = nullptr; if (lexer.current().type != ')') - std::tie(vararg, varargAnnotation) = parseBindingList(args, /* allowDot3= */ true); + std::tie(vararg, varargLocation, varargAnnotation) = parseBindingList(args, /* allowDot3= */ true); expectMatchAndConsume(')', matchParen); @@ -965,6 +979,21 @@ AstStat* Parser::parseCompoundAssignment(AstExpr* initial, AstExprBinary::Op op) return allocator.alloc(Location(initial->location, value->location), op, initial, value); } +std::pair> Parser::prepareFunctionArguments(const Location& start, bool hasself, const TempVector& args) +{ + AstLocal* self = nullptr; + + if (hasself) + self = pushLocal(Binding(Name(nameSelf, start), nullptr)); + + TempVector vars(scratchLocal); + + for (size_t i = 0; i < args.size(); ++i) + vars.push_back(pushLocal(args[i])); + + return {self, copy(vars)}; +} + // funcbody ::= `(' [parlist] `)' [`:' ReturnType] block end // parlist ::= bindinglist [`,' `...'] | `...' std::pair Parser::parseFunctionBody( @@ -979,15 +1008,18 @@ std::pair Parser::parseFunctionBody( TempVector args(scratchBinding); - std::optional vararg; + bool vararg = false; + Location varargLocation; AstTypePack* varargAnnotation = nullptr; if (lexer.current().type != ')') - std::tie(vararg, varargAnnotation) = parseBindingList(args, /* allowDot3= */ true); + std::tie(vararg, varargLocation, varargAnnotation) = parseBindingList(args, /* allowDot3= */ true); + + std::optional argLocation; + + if (matchParen.type == Lexeme::Type('(') && lexer.current().type == Lexeme::Type(')')) + argLocation = Location(matchParen.position, lexer.current().location.end); - std::optional argLocation = matchParen.type == Lexeme::Type('(') && lexer.current().type == Lexeme::Type(')') - ? std::make_optional(Location(matchParen.position, lexer.current().location.end)) - : std::nullopt; expectMatchAndConsume(')', matchParen, true); std::optional typelist = parseOptionalReturnTypeAnnotation(); @@ -1000,19 +1032,11 @@ std::pair Parser::parseFunctionBody( unsigned int localsBegin = saveLocals(); Function fun; - fun.vararg = vararg.has_value(); + fun.vararg = vararg; - functionStack.push_back(fun); + functionStack.emplace_back(fun); - AstLocal* self = nullptr; - - if (hasself) - self = pushLocal(Binding(Name(nameSelf, start), nullptr)); - - TempVector vars(scratchLocal); - - for (size_t i = 0; i < args.size(); ++i) - vars.push_back(pushLocal(args[i])); + auto [self, vars] = prepareFunctionArguments(start, hasself, args); AstStatBlock* body = parseBlock(); @@ -1024,8 +1048,8 @@ std::pair Parser::parseFunctionBody( bool hasEnd = expectMatchEndAndConsume(Lexeme::ReservedEnd, matchFunction); - return {allocator.alloc(Location(start, end), generics, genericPacks, self, copy(vars), vararg, body, functionStack.size(), - debugname, typelist, varargAnnotation, hasEnd, argLocation), + return {allocator.alloc(Location(start, end), generics, genericPacks, self, vars, vararg, varargLocation, body, + functionStack.size(), debugname, typelist, varargAnnotation, hasEnd, argLocation), funLocal}; } @@ -1056,7 +1080,7 @@ Parser::Binding Parser::parseBinding() } // bindinglist ::= (binding | `...') [`,' bindinglist] -std::pair, AstTypePack*> Parser::parseBindingList(TempVector& result, bool allowDot3) +std::tuple Parser::parseBindingList(TempVector& result, bool allowDot3) { while (true) { @@ -1072,7 +1096,7 @@ std::pair, AstTypePack*> Parser::parseBindingList(TempVe tailAnnotation = parseVariadicArgumentAnnotation(); } - return {varargLocation, tailAnnotation}; + return {true, varargLocation, tailAnnotation}; } result.push_back(parseBinding()); @@ -1082,7 +1106,7 @@ std::pair, AstTypePack*> Parser::parseBindingList(TempVe nextLexeme(); } - return {std::nullopt, nullptr}; + return {false, Location(), nullptr}; } AstType* Parser::parseOptionalTypeAnnotation() @@ -1567,6 +1591,12 @@ AstTypeOrPack Parser::parseSimpleTypeAnnotation(bool allowPack) else return {reportTypeAnnotationError(begin, {}, /*isMissing*/ false, "String literal contains malformed escape sequence")}; } + else if (lexer.current().type == Lexeme::InterpStringBegin || lexer.current().type == Lexeme::InterpStringSimple) + { + parseInterpString(); + + return {reportTypeAnnotationError(begin, {}, /*isMissing*/ false, "Interpolated string literals cannot be used as types")}; + } else if (lexer.current().type == Lexeme::BrokenString) { Location location = lexer.current().location; @@ -2215,15 +2245,24 @@ AstExpr* Parser::parseSimpleExpr() { return parseNumber(); } - else if (lexer.current().type == Lexeme::RawString || lexer.current().type == Lexeme::QuotedString) + else if (lexer.current().type == Lexeme::RawString || lexer.current().type == Lexeme::QuotedString || (FFlag::LuauInterpolatedStringBaseSupport && lexer.current().type == Lexeme::InterpStringSimple)) { return parseString(); } + else if (FFlag::LuauInterpolatedStringBaseSupport && lexer.current().type == Lexeme::InterpStringBegin) + { + return parseInterpString(); + } else if (lexer.current().type == Lexeme::BrokenString) { nextLexeme(); return reportExprError(start, {}, "Malformed string"); } + else if (lexer.current().type == Lexeme::BrokenInterpDoubleBrace) + { + nextLexeme(); + return reportExprError(start, {}, ERROR_INVALID_INTERP_DOUBLE_BRACE); + } else if (lexer.current().type == Lexeme::Dot3) { if (functionStack.back().vararg) @@ -2614,11 +2653,11 @@ AstArray Parser::parseTypeParams() std::optional> Parser::parseCharArray() { - LUAU_ASSERT(lexer.current().type == Lexeme::QuotedString || lexer.current().type == Lexeme::RawString); + LUAU_ASSERT(lexer.current().type == Lexeme::QuotedString || lexer.current().type == Lexeme::RawString || lexer.current().type == Lexeme::InterpStringSimple); scratchData.assign(lexer.current().data, lexer.current().length); - if (lexer.current().type == Lexeme::QuotedString) + if (lexer.current().type == Lexeme::QuotedString || lexer.current().type == Lexeme::InterpStringSimple) { if (!Lexer::fixupQuotedString(scratchData)) { @@ -2645,6 +2684,70 @@ AstExpr* Parser::parseString() return reportExprError(location, {}, "String literal contains malformed escape sequence"); } +AstExpr* Parser::parseInterpString() +{ + TempVector> strings(scratchString); + TempVector expressions(scratchExpr); + + Location startLocation = lexer.current().location; + + do { + Lexeme currentLexeme = lexer.current(); + LUAU_ASSERT( + currentLexeme.type == Lexeme::InterpStringBegin + || currentLexeme.type == Lexeme::InterpStringMid + || currentLexeme.type == Lexeme::InterpStringEnd + || currentLexeme.type == Lexeme::InterpStringSimple + ); + + Location location = currentLexeme.location; + + Location startOfBrace = Location(location.end, 1); + + scratchData.assign(currentLexeme.data, currentLexeme.length); + + if (!Lexer::fixupQuotedString(scratchData)) + { + nextLexeme(); + return reportExprError(startLocation, {}, "Interpolated string literal contains malformed escape sequence"); + } + + AstArray chars = copy(scratchData); + + nextLexeme(); + + strings.push_back(chars); + + if (currentLexeme.type == Lexeme::InterpStringEnd || currentLexeme.type == Lexeme::InterpStringSimple) + { + AstArray> stringsArray = copy(strings); + AstArray expressionsArray = copy(expressions); + + return allocator.alloc(startLocation, stringsArray, expressionsArray); + } + + AstExpr* expression = parseExpr(); + + expressions.push_back(expression); + + switch (lexer.current().type) + { + case Lexeme::InterpStringBegin: + case Lexeme::InterpStringMid: + case Lexeme::InterpStringEnd: + break; + case Lexeme::BrokenInterpDoubleBrace: + nextLexeme(); + return reportExprError(location, {}, ERROR_INVALID_INTERP_DOUBLE_BRACE); + case Lexeme::BrokenString: + nextLexeme(); + return reportExprError(location, {}, "Malformed interpolated string, did you forget to add a '}'?"); + default: + return reportExprError(location, {}, "Malformed interpolated string, got %s", lexer.current().toString().c_str()); + } + } while (true); +} + AstExpr* Parser::parseNumber() { Location start = lexer.current().location; diff --git a/Ast/src/StringUtils.cpp b/Ast/src/StringUtils.cpp index 0dc3f3f5..11e0076a 100644 --- a/Ast/src/StringUtils.cpp +++ b/Ast/src/StringUtils.cpp @@ -230,19 +230,25 @@ bool isIdentifier(std::string_view s) return (s.find_first_not_of("abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890_") == std::string::npos); } -std::string escape(std::string_view s) +std::string escape(std::string_view s, bool escapeForInterpString) { std::string r; r.reserve(s.size() + 50); // arbitrary number to guess how many characters we'll be inserting for (uint8_t c : s) { - if (c >= ' ' && c != '\\' && c != '\'' && c != '\"') + if (c >= ' ' && c != '\\' && c != '\'' && c != '\"' && c != '`' && c != '{') r += c; else { r += '\\'; + if (escapeForInterpString && (c == '`' || c == '{')) + { + r += c; + continue; + } + switch (c) { case '\a': diff --git a/Common/include/Luau/ExperimentalFlags.h b/Common/include/Luau/ExperimentalFlags.h index 71e76ffe..809c78da 100644 --- a/Common/include/Luau/ExperimentalFlags.h +++ b/Common/include/Luau/ExperimentalFlags.h @@ -12,6 +12,7 @@ inline bool isFlagExperimental(const char* flag) // or critical bugs that are found after the code has been submitted. static const char* kList[] = { "LuauLowerBoundsCalculation", + "LuauInterpolatedStringBaseSupport", nullptr, // makes sure we always have at least one entry }; diff --git a/Compiler/src/Compiler.cpp b/Compiler/src/Compiler.cpp index bd8744cc..4429e4cc 100644 --- a/Compiler/src/Compiler.cpp +++ b/Compiler/src/Compiler.cpp @@ -14,6 +14,8 @@ #include #include +#include + #include LUAU_FASTINTVARIABLE(LuauCompileLoopUnrollThreshold, 25) @@ -25,6 +27,8 @@ LUAU_FASTINTVARIABLE(LuauCompileInlineDepth, 5) LUAU_FASTFLAGVARIABLE(LuauCompileXEQ, false) +LUAU_FASTFLAG(LuauInterpolatedStringBaseSupport) + LUAU_FASTFLAGVARIABLE(LuauCompileOptimalAssignment, false) LUAU_FASTFLAGVARIABLE(LuauCompileExtractK, false) @@ -1585,6 +1589,76 @@ struct Compiler } } + void compileExprInterpString(AstExprInterpString* expr, uint8_t target, bool targetTemp) + { + size_t formatCapacity = 0; + for (AstArray string : expr->strings) + { + formatCapacity += string.size + std::count(string.data, string.data + string.size, '%'); + } + + std::string formatString; + formatString.reserve(formatCapacity); + + size_t stringsLeft = expr->strings.size; + + for (AstArray string : expr->strings) + { + if (memchr(string.data, '%', string.size)) + { + for (size_t characterIndex = 0; characterIndex < string.size; ++characterIndex) + { + char character = string.data[characterIndex]; + formatString.push_back(character); + + if (character == '%') + formatString.push_back('%'); + } + } + else + formatString.append(string.data, string.size); + + stringsLeft--; + + if (stringsLeft > 0) + formatString += "%*"; + } + + size_t formatStringSize = formatString.size(); + + // We can't use formatStringRef.data() directly, because short strings don't have their data + // pinned in memory, so when interpFormatStrings grows, these pointers will move and become invalid. + std::unique_ptr formatStringPtr(new char[formatStringSize]); + memcpy(formatStringPtr.get(), formatString.data(), formatStringSize); + + AstArray formatStringArray{formatStringPtr.get(), formatStringSize}; + interpStrings.emplace_back(std::move(formatStringPtr)); // invalidates formatStringPtr, but keeps formatStringArray intact + + int32_t formatStringIndex = bytecode.addConstantString(sref(formatStringArray)); + if (formatStringIndex < 0) + CompileError::raise(expr->location, "Exceeded constant limit; simplify the code to compile"); + + RegScope rs(this); + + uint8_t baseReg = allocReg(expr, uint8_t(2 + expr->expressions.size)); + + emitLoadK(baseReg, formatStringIndex); + + for (size_t index = 0; index < expr->expressions.size; ++index) + compileExprTempTop(expr->expressions.data[index], uint8_t(baseReg + 2 + index)); + + BytecodeBuilder::StringRef formatMethod = sref(AstName("format")); + + int32_t formatMethodIndex = bytecode.addConstantString(formatMethod); + if (formatMethodIndex < 0) + CompileError::raise(expr->location, "Exceeded constant limit; simplify the code to compile"); + + bytecode.emitABC(LOP_NAMECALL, baseReg, baseReg, uint8_t(BytecodeBuilder::getStringHash(formatMethod))); + bytecode.emitAux(formatMethodIndex); + bytecode.emitABC(LOP_CALL, baseReg, uint8_t(expr->expressions.size + 2), 2); + bytecode.emitABC(LOP_MOVE, target, baseReg, 0); + } + static uint8_t encodeHashSize(unsigned int hashSize) { size_t hashSizeLog2 = 0; @@ -2059,6 +2133,10 @@ struct Compiler { compileExprIfElse(expr, target, targetTemp); } + else if (AstExprInterpString* interpString = node->as(); FFlag::LuauInterpolatedStringBaseSupport && interpString) + { + compileExprInterpString(interpString, target, targetTemp); + } else { LUAU_ASSERT(!"Unknown expression type"); @@ -2965,6 +3043,18 @@ struct Compiler uint8_t valueReg = kInvalidReg; }; + // This function analyzes assignments and marks assignment conflicts: cases when a variable is assigned on lhs + // but subsequently used on the rhs, assuming assignments are performed in order. Note that it's also possible + // for a variable to conflict on the lhs, if it's used in an lvalue expression after it's assigned. + // When conflicts are found, Assignment::conflictReg is allocated and that's where assignment is performed instead, + // until the final fixup in compileStatAssign. Assignment::valueReg is allocated by compileStatAssign as well. + // + // Per Lua manual, section 3.3.3 (Assignments), the proper assignment order is only guaranteed to hold for syntactic access: + // + // Note that this guarantee covers only accesses syntactically inside the assignment statement. If a function or a metamethod called + // during the assignment changes the value of a variable, Lua gives no guarantees about the order of that access. + // + // As such, we currently don't check if an assigned local is captured, which may mean it gets reassigned during a function call. void resolveAssignConflicts(AstStat* stat, std::vector& vars, const AstArray& values) { struct Visitor : AstVisitor @@ -3808,6 +3898,7 @@ struct Compiler std::vector loops; std::vector inlineFrames; std::vector captures; + std::vector> interpStrings; }; void compileOrThrow(BytecodeBuilder& bytecode, const ParseResult& parseResult, const AstNameTable& names, const CompileOptions& inputOptions) @@ -3866,7 +3957,8 @@ void compileOrThrow(BytecodeBuilder& bytecode, const ParseResult& parseResult, c compiler.compileFunction(expr); AstExprFunction main(root->location, /*generics= */ AstArray(), /*genericPacks= */ AstArray(), - /* self= */ nullptr, AstArray(), /* vararg= */ Luau::Location(), root, /* functionDepth= */ 0, /* debugname= */ AstName()); + /* self= */ nullptr, AstArray(), /* vararg= */ true, /* varargLocation= */ Luau::Location(), root, /* functionDepth= */ 0, + /* debugname= */ AstName()); uint32_t mainid = compiler.compileFunction(&main); const Compiler::Function* mainf = compiler.functions.find(&main); diff --git a/Compiler/src/ConstantFolding.cpp b/Compiler/src/ConstantFolding.cpp index 34f79544..e35c883a 100644 --- a/Compiler/src/ConstantFolding.cpp +++ b/Compiler/src/ConstantFolding.cpp @@ -349,6 +349,11 @@ struct ConstantVisitor : AstVisitor if (cond.type != Constant::Type_Unknown) result = cond.isTruthful() ? trueExpr : falseExpr; } + else if (AstExprInterpString* expr = node->as()) + { + for (AstExpr* expression : expr->expressions) + analyze(expression); + } else { LUAU_ASSERT(!"Unknown expression type"); diff --git a/Compiler/src/CostModel.cpp b/Compiler/src/CostModel.cpp index 81cbfd7a..ffc1cb1f 100644 --- a/Compiler/src/CostModel.cpp +++ b/Compiler/src/CostModel.cpp @@ -215,6 +215,16 @@ struct CostVisitor : AstVisitor { return model(expr->condition) + model(expr->trueExpr) + model(expr->falseExpr) + 2; } + else if (AstExprInterpString* expr = node->as()) + { + // Baseline cost of string.format + Cost cost = 3; + + for (AstExpr* innerExpression : expr->expressions) + cost += model(innerExpression); + + return cost; + } else { LUAU_ASSERT(!"Unknown expression type"); diff --git a/Makefile b/Makefile index 8d95aac3..0db7b281 100644 --- a/Makefile +++ b/Makefile @@ -93,7 +93,7 @@ endif ifeq ($(config),fuzz) CXX=clang++ # our fuzzing infra relies on llvm fuzzer - CXXFLAGS+=-fsanitize=address,fuzzer -Ibuild/libprotobuf-mutator -Ibuild/libprotobuf-mutator/external.protobuf/include -O2 + CXXFLAGS+=-fsanitize=address,fuzzer -Ibuild/libprotobuf-mutator -O2 LDFLAGS+=-fsanitize=address,fuzzer endif @@ -115,7 +115,7 @@ $(FUZZ_OBJECTS): CXXFLAGS+=-std=c++17 -ICommon/include -IAst/include -ICompiler/ $(TESTS_TARGET): LDFLAGS+=-lpthread $(REPL_CLI_TARGET): LDFLAGS+=-lpthread -fuzz-proto fuzz-prototest: LDFLAGS+=build/libprotobuf-mutator/src/libfuzzer/libprotobuf-mutator-libfuzzer.a build/libprotobuf-mutator/src/libprotobuf-mutator.a build/libprotobuf-mutator/external.protobuf/lib/libprotobuf.a +fuzz-proto fuzz-prototest: LDFLAGS+=build/libprotobuf-mutator/src/libfuzzer/libprotobuf-mutator-libfuzzer.a build/libprotobuf-mutator/src/libprotobuf-mutator.a -lprotobuf # pseudo targets .PHONY: all test clean coverage format luau-size aliases @@ -195,7 +195,7 @@ $(BUILD)/%.c.o: %.c # protobuf fuzzer setup fuzz/luau.pb.cpp: fuzz/luau.proto build/libprotobuf-mutator - cd fuzz && ../build/libprotobuf-mutator/external.protobuf/bin/protoc luau.proto --cpp_out=. + cd fuzz && protoc luau.proto --cpp_out=. mv fuzz/luau.pb.cc fuzz/luau.pb.cpp $(BUILD)/fuzz/proto.cpp.o: fuzz/luau.pb.cpp @@ -203,7 +203,7 @@ $(BUILD)/fuzz/protoprint.cpp.o: fuzz/luau.pb.cpp build/libprotobuf-mutator: git clone https://github.com/google/libprotobuf-mutator build/libprotobuf-mutator - CXX= cmake -S build/libprotobuf-mutator -B build/libprotobuf-mutator -D CMAKE_BUILD_TYPE=Release -D LIB_PROTO_MUTATOR_DOWNLOAD_PROTOBUF=ON -D LIB_PROTO_MUTATOR_TESTING=OFF + CXX= cmake -S build/libprotobuf-mutator -B build/libprotobuf-mutator -D CMAKE_BUILD_TYPE=Release -D LIB_PROTO_MUTATOR_TESTING=OFF make -C build/libprotobuf-mutator -j8 # picks up include dependencies for all object files diff --git a/VM/src/lapi.cpp b/VM/src/lapi.cpp index af97dc4a..4396e5d1 100644 --- a/VM/src/lapi.cpp +++ b/VM/src/lapi.cpp @@ -24,16 +24,18 @@ * The caller is expected to handle stack reservation (by using less than LUA_MINSTACK slots or by calling lua_checkstack). * To ensure this is handled correctly, use api_incr_top(L) when pushing values to the stack. * - * Functions that push any collectable objects to the stack *should* call luaC_checkthreadsleep. Failure to do this can result - * in stack references that point to dead objects since sleeping threads don't get rescanned. + * Functions that push any collectable objects to the stack *should* call luaC_threadbarrier. Failure to do this can result + * in stack references that point to dead objects since black threads don't get rescanned. * - * Functions that push newly created objects to the stack *should* call luaC_checkGC in addition to luaC_checkthreadsleep. + * Functions that push newly created objects to the stack *should* call luaC_checkGC in addition to luaC_threadbarrier. * Failure to do this can result in OOM since GC may never run. * - * Note that luaC_checkGC may scan the thread and put it back to sleep; functions that call both before pushing objects must - * therefore call luaC_checkGC before luaC_checkthreadsleep to guarantee the object is pushed to an awake thread. + * Note that luaC_checkGC may mark the thread and paint it black; functions that call both before pushing objects must + * therefore call luaC_checkGC before luaC_threadbarrier to guarantee the object is pushed to a gray thread. */ +LUAU_FASTFLAG(LuauSimplerUpval) + const char* lua_ident = "$Lua: Lua 5.1.4 Copyright (C) 1994-2008 Lua.org, PUC-Rio $\n" "$Authors: R. Ierusalimschy, L. H. de Figueiredo & W. Celes $\n" "$URL: www.lua.org $\n"; @@ -152,7 +154,7 @@ void lua_xmove(lua_State* from, lua_State* to, int n) api_checknelems(from, n); api_check(from, from->global == to->global); api_check(from, to->ci->top - to->top >= n); - luaC_checkthreadsleep(to); + luaC_threadbarrier(to); StkId ttop = to->top; StkId ftop = from->top - n; @@ -168,7 +170,7 @@ void lua_xmove(lua_State* from, lua_State* to, int n) void lua_xpush(lua_State* from, lua_State* to, int idx) { api_check(from, from->global == to->global); - luaC_checkthreadsleep(to); + luaC_threadbarrier(to); setobj2s(to, to->top, index2addr(from, idx)); api_incr_top(to); return; @@ -177,7 +179,7 @@ void lua_xpush(lua_State* from, lua_State* to, int idx) lua_State* lua_newthread(lua_State* L) { luaC_checkGC(L); - luaC_checkthreadsleep(L); + luaC_threadbarrier(L); lua_State* L1 = luaE_newthread(L); setthvalue(L, L->top, L1); api_incr_top(L); @@ -236,7 +238,7 @@ void lua_remove(lua_State* L, int idx) void lua_insert(lua_State* L, int idx) { - luaC_checkthreadsleep(L); + luaC_threadbarrier(L); StkId p = index2addr(L, idx); api_checkvalidindex(L, p); for (StkId q = L->top; q > p; q--) @@ -248,7 +250,7 @@ void lua_insert(lua_State* L, int idx) void lua_replace(lua_State* L, int idx) { api_checknelems(L, 1); - luaC_checkthreadsleep(L); + luaC_threadbarrier(L); StkId o = index2addr(L, idx); api_checkvalidindex(L, o); if (idx == LUA_ENVIRONINDEX) @@ -276,7 +278,7 @@ void lua_replace(lua_State* L, int idx) void lua_pushvalue(lua_State* L, int idx) { - luaC_checkthreadsleep(L); + luaC_threadbarrier(L); StkId o = index2addr(L, idx); setobj2s(L, L->top, o); api_incr_top(L); @@ -427,7 +429,7 @@ const char* lua_tolstring(lua_State* L, int idx, size_t* len) StkId o = index2addr(L, idx); if (!ttisstring(o)) { - luaC_checkthreadsleep(L); + luaC_threadbarrier(L); if (!luaV_tostring(L, o)) { // conversion failed? if (len != NULL) @@ -607,7 +609,7 @@ void lua_pushvector(lua_State* L, float x, float y, float z) void lua_pushlstring(lua_State* L, const char* s, size_t len) { luaC_checkGC(L); - luaC_checkthreadsleep(L); + luaC_threadbarrier(L); setsvalue2s(L, L->top, luaS_newlstr(L, s, len)); api_incr_top(L); return; @@ -624,7 +626,7 @@ void lua_pushstring(lua_State* L, const char* s) const char* lua_pushvfstring(lua_State* L, const char* fmt, va_list argp) { luaC_checkGC(L); - luaC_checkthreadsleep(L); + luaC_threadbarrier(L); const char* ret = luaO_pushvfstring(L, fmt, argp); return ret; } @@ -632,7 +634,7 @@ const char* lua_pushvfstring(lua_State* L, const char* fmt, va_list argp) const char* lua_pushfstringL(lua_State* L, const char* fmt, ...) { luaC_checkGC(L); - luaC_checkthreadsleep(L); + luaC_threadbarrier(L); va_list argp; va_start(argp, fmt); const char* ret = luaO_pushvfstring(L, fmt, argp); @@ -643,7 +645,7 @@ const char* lua_pushfstringL(lua_State* L, const char* fmt, ...) void lua_pushcclosurek(lua_State* L, lua_CFunction fn, const char* debugname, int nup, lua_Continuation cont) { luaC_checkGC(L); - luaC_checkthreadsleep(L); + luaC_threadbarrier(L); api_checknelems(L, nup); Closure* cl = luaF_newCclosure(L, nup, getcurrenv(L)); cl->c.f = fn; @@ -674,7 +676,7 @@ void lua_pushlightuserdata(lua_State* L, void* p) int lua_pushthread(lua_State* L) { - luaC_checkthreadsleep(L); + luaC_threadbarrier(L); setthvalue(L, L->top, L); api_incr_top(L); return L->global->mainthread == L; @@ -686,7 +688,7 @@ int lua_pushthread(lua_State* L) int lua_gettable(lua_State* L, int idx) { - luaC_checkthreadsleep(L); + luaC_threadbarrier(L); StkId t = index2addr(L, idx); api_checkvalidindex(L, t); luaV_gettable(L, t, L->top - 1, L->top - 1); @@ -695,7 +697,7 @@ int lua_gettable(lua_State* L, int idx) int lua_getfield(lua_State* L, int idx, const char* k) { - luaC_checkthreadsleep(L); + luaC_threadbarrier(L); StkId t = index2addr(L, idx); api_checkvalidindex(L, t); TValue key; @@ -707,7 +709,7 @@ int lua_getfield(lua_State* L, int idx, const char* k) int lua_rawgetfield(lua_State* L, int idx, const char* k) { - luaC_checkthreadsleep(L); + luaC_threadbarrier(L); StkId t = index2addr(L, idx); api_check(L, ttistable(t)); TValue key; @@ -719,7 +721,7 @@ int lua_rawgetfield(lua_State* L, int idx, const char* k) int lua_rawget(lua_State* L, int idx) { - luaC_checkthreadsleep(L); + luaC_threadbarrier(L); StkId t = index2addr(L, idx); api_check(L, ttistable(t)); setobj2s(L, L->top - 1, luaH_get(hvalue(t), L->top - 1)); @@ -728,7 +730,7 @@ int lua_rawget(lua_State* L, int idx) int lua_rawgeti(lua_State* L, int idx, int n) { - luaC_checkthreadsleep(L); + luaC_threadbarrier(L); StkId t = index2addr(L, idx); api_check(L, ttistable(t)); setobj2s(L, L->top, luaH_getnum(hvalue(t), n)); @@ -739,7 +741,7 @@ int lua_rawgeti(lua_State* L, int idx, int n) void lua_createtable(lua_State* L, int narray, int nrec) { luaC_checkGC(L); - luaC_checkthreadsleep(L); + luaC_threadbarrier(L); sethvalue(L, L->top, luaH_new(L, narray, nrec)); api_incr_top(L); return; @@ -775,7 +777,7 @@ void lua_setsafeenv(lua_State* L, int objindex, int enabled) int lua_getmetatable(lua_State* L, int objindex) { - luaC_checkthreadsleep(L); + luaC_threadbarrier(L); Table* mt = NULL; const TValue* obj = index2addr(L, objindex); switch (ttype(obj)) @@ -800,7 +802,7 @@ int lua_getmetatable(lua_State* L, int objindex) void lua_getfenv(lua_State* L, int idx) { - luaC_checkthreadsleep(L); + luaC_threadbarrier(L); StkId o = index2addr(L, idx); api_checkvalidindex(L, o); switch (ttype(o)) @@ -1161,7 +1163,7 @@ l_noret lua_error(lua_State* L) int lua_next(lua_State* L, int idx) { - luaC_checkthreadsleep(L); + luaC_threadbarrier(L); StkId t = index2addr(L, idx); api_check(L, ttistable(t)); int more = luaH_next(L, hvalue(t), L->top - 1); @@ -1180,13 +1182,13 @@ void lua_concat(lua_State* L, int n) if (n >= 2) { luaC_checkGC(L); - luaC_checkthreadsleep(L); + luaC_threadbarrier(L); luaV_concat(L, n, cast_int(L->top - L->base) - 1); L->top -= (n - 1); } else if (n == 0) { // push empty string - luaC_checkthreadsleep(L); + luaC_threadbarrier(L); setsvalue2s(L, L->top, luaS_newlstr(L, "", 0)); api_incr_top(L); } @@ -1198,7 +1200,7 @@ void* lua_newuserdatatagged(lua_State* L, size_t sz, int tag) { api_check(L, unsigned(tag) < LUA_UTAG_LIMIT || tag == UTAG_PROXY); luaC_checkGC(L); - luaC_checkthreadsleep(L); + luaC_threadbarrier(L); Udata* u = luaU_newudata(L, sz, tag); setuvalue(L, L->top, u); api_incr_top(L); @@ -1208,7 +1210,7 @@ void* lua_newuserdatatagged(lua_State* L, size_t sz, int tag) void* lua_newuserdatadtor(lua_State* L, size_t sz, void (*dtor)(void*)) { luaC_checkGC(L); - luaC_checkthreadsleep(L); + luaC_threadbarrier(L); // make sure sz + sizeof(dtor) doesn't overflow; luaU_newdata will reject SIZE_MAX correctly size_t as = sz < SIZE_MAX - sizeof(dtor) ? sz + sizeof(dtor) : SIZE_MAX; Udata* u = luaU_newudata(L, as, UTAG_IDTOR); @@ -1244,7 +1246,7 @@ static const char* aux_upvalue(StkId fi, int n, TValue** val) const char* lua_getupvalue(lua_State* L, int funcindex, int n) { - luaC_checkthreadsleep(L); + luaC_threadbarrier(L); TValue* val; const char* name = aux_upvalue(index2addr(L, funcindex), n, &val); if (name) @@ -1266,7 +1268,8 @@ const char* lua_setupvalue(lua_State* L, int funcindex, int n) L->top--; setobj(L, val, L->top); luaC_barrier(L, clvalue(fi), L->top); - luaC_upvalbarrier(L, cast_to(UpVal*, NULL), val); + if (!FFlag::LuauSimplerUpval) + luaC_upvalbarrier(L, cast_to(UpVal*, NULL), val); } return name; } @@ -1336,7 +1339,7 @@ void lua_setuserdatadtor(lua_State* L, int tag, void (*dtor)(lua_State*, void*)) void lua_clonefunction(lua_State* L, int idx) { luaC_checkGC(L); - luaC_checkthreadsleep(L); + luaC_threadbarrier(L); StkId p = index2addr(L, idx); api_check(L, isLfunction(p)); Closure* cl = clvalue(p); diff --git a/VM/src/ldebug.cpp b/VM/src/ldebug.cpp index c44ccbed..fee9aaa9 100644 --- a/VM/src/ldebug.cpp +++ b/VM/src/ldebug.cpp @@ -12,8 +12,6 @@ #include #include -LUAU_FASTFLAGVARIABLE(LuauDebuggerBreakpointHitOnNextBestLine, false); - static const char* getfuncname(Closure* f); static int currentpc(lua_State* L, CallInfo* ci) @@ -44,13 +42,13 @@ int lua_getargument(lua_State* L, int level, int n) { if (n <= fp->numparams) { - luaC_checkthreadsleep(L); + luaC_threadbarrier(L); luaA_pushobject(L, ci->base + (n - 1)); res = 1; } else if (fp->is_vararg && n < ci->base - ci->func) { - luaC_checkthreadsleep(L); + luaC_threadbarrier(L); luaA_pushobject(L, ci->func + n); res = 1; } @@ -69,7 +67,7 @@ const char* lua_getlocal(lua_State* L, int level, int n) const LocVar* var = fp ? luaF_getlocal(fp, n, currentpc(L, ci)) : NULL; if (var) { - luaC_checkthreadsleep(L); + luaC_threadbarrier(L); luaA_pushobject(L, ci->base + var->reg); } const char* name = var ? getstr(var->varname) : NULL; @@ -185,7 +183,7 @@ int lua_getinfo(lua_State* L, int level, const char* what, lua_Debug* ar) status = auxgetinfo(L, what, ar, f, ci); if (strchr(what, 'f')) { - luaC_checkthreadsleep(L); + luaC_threadbarrier(L); setclvalue(L, L->top, f); incr_top(L); } @@ -437,29 +435,17 @@ static int getnextline(Proto* p, int line) int lua_breakpoint(lua_State* L, int funcindex, int line, int enabled) { - int target = -1; + const TValue* func = luaA_toobject(L, funcindex); + api_check(L, ttisfunction(func) && !clvalue(func)->isC); - if (FFlag::LuauDebuggerBreakpointHitOnNextBestLine) + Proto* p = clvalue(func)->l.p; + // Find line number to add the breakpoint to. + int target = getnextline(p, line); + + if (target != -1) { - const TValue* func = luaA_toobject(L, funcindex); - api_check(L, ttisfunction(func) && !clvalue(func)->isC); - - Proto* p = clvalue(func)->l.p; - // Find line number to add the breakpoint to. - target = getnextline(p, line); - - if (target != -1) - { - // Add breakpoint on the exact line - luaG_breakpoint(L, p, target, bool(enabled)); - } - } - else - { - const TValue* func = luaA_toobject(L, funcindex); - api_check(L, ttisfunction(func) && !clvalue(func)->isC); - - luaG_breakpoint(L, clvalue(func)->l.p, line, bool(enabled)); + // Add breakpoint on the exact line + luaG_breakpoint(L, p, target, bool(enabled)); } return target; diff --git a/VM/src/ldo.cpp b/VM/src/ldo.cpp index 6016e41f..51f63d32 100644 --- a/VM/src/ldo.cpp +++ b/VM/src/ldo.cpp @@ -158,7 +158,7 @@ l_noret luaD_throw(lua_State* L, int errcode) static void correctstack(lua_State* L, TValue* oldstack) { L->top = (L->top - oldstack) + L->stack; - for (UpVal* up = L->openupval; up != NULL; up = up->u.l.threadnext) + for (UpVal* up = L->openupval; up != NULL; up = up->u.open.threadnext) up->v = (up->v - oldstack) + L->stack; for (CallInfo* ci = L->base_ci; ci <= L->ci; ci++) { @@ -245,7 +245,7 @@ void luaD_call(lua_State* L, StkId func, int nResults) int oldactive = luaC_threadactive(L); l_setbit(L->stackstate, THREAD_ACTIVEBIT); - luaC_checkthreadsleep(L); + luaC_threadbarrier(L); luau_execute(L); // call it @@ -454,7 +454,7 @@ int lua_resume(lua_State* L, lua_State* from, int nargs) L->baseCcalls = ++L->nCcalls; l_setbit(L->stackstate, THREAD_ACTIVEBIT); - luaC_checkthreadsleep(L); + luaC_threadbarrier(L); status = luaD_rawrunprotected(L, resume, L->top - nargs); @@ -483,7 +483,7 @@ int lua_resumeerror(lua_State* L, lua_State* from) L->baseCcalls = ++L->nCcalls; l_setbit(L->stackstate, THREAD_ACTIVEBIT); - luaC_checkthreadsleep(L); + luaC_threadbarrier(L); status = LUA_ERRRUN; diff --git a/VM/src/lfunc.cpp b/VM/src/lfunc.cpp index bd0f8264..8c78083b 100644 --- a/VM/src/lfunc.cpp +++ b/VM/src/lfunc.cpp @@ -6,6 +6,9 @@ #include "lmem.h" #include "lgc.h" +LUAU_FASTFLAG(LuauSimplerUpval) +LUAU_FASTFLAG(LuauNoSleepBit) + Proto* luaF_newproto(lua_State* L) { Proto* f = luaM_newgco(L, Proto, sizeof(Proto), L->activememcat); @@ -71,59 +74,76 @@ UpVal* luaF_findupval(lua_State* L, StkId level) UpVal* p; while (*pp != NULL && (p = *pp)->v >= level) { - LUAU_ASSERT(p->v != &p->u.value); + LUAU_ASSERT(!FFlag::LuauSimplerUpval || !isdead(g, obj2gco(p))); + LUAU_ASSERT(upisopen(p)); if (p->v == level) - { // found a corresponding upvalue? - if (isdead(g, obj2gco(p))) // is it dead? - changewhite(obj2gco(p)); // resurrect it + { // found a corresponding upvalue? + if (!FFlag::LuauSimplerUpval && isdead(g, obj2gco(p))) // is it dead? + changewhite(obj2gco(p)); // resurrect it return p; } - pp = &p->u.l.threadnext; + pp = &p->u.open.threadnext; } + LUAU_ASSERT(luaC_threadactive(L)); + LUAU_ASSERT(!luaC_threadsleeping(L)); + LUAU_ASSERT(!FFlag::LuauNoSleepBit || !isblack(obj2gco(L))); // we don't use luaC_threadbarrier because active threads never turn black + UpVal* uv = luaM_newgco(L, UpVal, sizeof(UpVal), L->activememcat); // not found: create a new one - uv->tt = LUA_TUPVAL; - uv->marked = luaC_white(g); - uv->memcat = L->activememcat; + luaC_init(L, uv, LUA_TUPVAL); + uv->markedopen = 0; uv->v = level; // current value lives in the stack // chain the upvalue in the threads open upvalue list at the proper position - UpVal* next = *pp; - uv->u.l.threadnext = next; - uv->u.l.threadprev = pp; - if (next) - next->u.l.threadprev = &uv->u.l.threadnext; + if (FFlag::LuauSimplerUpval) + { + uv->u.open.threadnext = *pp; + *pp = uv; + } + else + { + UpVal* next = *pp; + uv->u.open.threadnext = next; - *pp = uv; + uv->u.open.threadprev = pp; + if (next) + next->u.open.threadprev = &uv->u.open.threadnext; + + *pp = uv; + } // double link the upvalue in the global open upvalue list - uv->u.l.prev = &g->uvhead; - uv->u.l.next = g->uvhead.u.l.next; - uv->u.l.next->u.l.prev = uv; - g->uvhead.u.l.next = uv; - LUAU_ASSERT(uv->u.l.next->u.l.prev == uv && uv->u.l.prev->u.l.next == uv); + uv->u.open.prev = &g->uvhead; + uv->u.open.next = g->uvhead.u.open.next; + uv->u.open.next->u.open.prev = uv; + g->uvhead.u.open.next = uv; + LUAU_ASSERT(uv->u.open.next->u.open.prev == uv && uv->u.open.prev->u.open.next == uv); + return uv; } + void luaF_unlinkupval(UpVal* uv) { + LUAU_ASSERT(!FFlag::LuauSimplerUpval); + // unlink upvalue from the global open upvalue list - LUAU_ASSERT(uv->u.l.next->u.l.prev == uv && uv->u.l.prev->u.l.next == uv); - uv->u.l.next->u.l.prev = uv->u.l.prev; - uv->u.l.prev->u.l.next = uv->u.l.next; + LUAU_ASSERT(uv->u.open.next->u.open.prev == uv && uv->u.open.prev->u.open.next == uv); + uv->u.open.next->u.open.prev = uv->u.open.prev; + uv->u.open.prev->u.open.next = uv->u.open.next; // unlink upvalue from the thread open upvalue list - *uv->u.l.threadprev = uv->u.l.threadnext; + *uv->u.open.threadprev = uv->u.open.threadnext; - if (UpVal* next = uv->u.l.threadnext) - next->u.l.threadprev = uv->u.l.threadprev; + if (UpVal* next = uv->u.open.threadnext) + next->u.open.threadprev = uv->u.open.threadprev; } void luaF_freeupval(lua_State* L, UpVal* uv, lua_Page* page) { - if (uv->v != &uv->u.value) // is it open? - luaF_unlinkupval(uv); // remove from open list - luaM_freegco(L, uv, sizeof(UpVal), uv->memcat, page); // free upvalue + if (!FFlag::LuauSimplerUpval && uv->v != &uv->u.value) // is it open? + luaF_unlinkupval(uv); // remove from open list + luaM_freegco(L, uv, sizeof(UpVal), uv->memcat, page); // free upvalue } void luaF_close(lua_State* L, StkId level) @@ -133,26 +153,55 @@ void luaF_close(lua_State* L, StkId level) while (L->openupval != NULL && (uv = L->openupval)->v >= level) { GCObject* o = obj2gco(uv); - LUAU_ASSERT(!isblack(o) && uv->v != &uv->u.value); + LUAU_ASSERT(!isblack(o) && upisopen(uv)); - // by removing the upvalue from global/thread open upvalue lists, L->openupval will be pointing to the next upvalue - luaF_unlinkupval(uv); - - if (isdead(g, o)) + if (FFlag::LuauSimplerUpval) { - // close the upvalue without copying the dead data so that luaF_freeupval will not unlink again - uv->v = &uv->u.value; + LUAU_ASSERT(!isdead(g, o)); + + // unlink value *before* closing it since value storage overlaps + L->openupval = uv->u.open.threadnext; + + luaF_closeupval(L, uv, /* dead= */ false); } else { - setobj(L, &uv->u.value, uv->v); - uv->v = &uv->u.value; - // GC state of a new closed upvalue has to be initialized - luaC_initupval(L, uv); + // by removing the upvalue from global/thread open upvalue lists, L->openupval will be pointing to the next upvalue + luaF_unlinkupval(uv); + + if (isdead(g, o)) + { + // close the upvalue without copying the dead data so that luaF_freeupval will not unlink again + uv->v = &uv->u.value; + } + else + { + setobj(L, &uv->u.value, uv->v); + uv->v = &uv->u.value; + // GC state of a new closed upvalue has to be initialized + luaC_upvalclosed(L, uv); + } } } } +void luaF_closeupval(lua_State* L, UpVal* uv, bool dead) +{ + LUAU_ASSERT(FFlag::LuauSimplerUpval); + + // unlink value from all lists *before* closing it since value storage overlaps + LUAU_ASSERT(uv->u.open.next->u.open.prev == uv && uv->u.open.prev->u.open.next == uv); + uv->u.open.next->u.open.prev = uv->u.open.prev; + uv->u.open.prev->u.open.next = uv->u.open.next; + + if (dead) + return; + + setobj(L, &uv->u.value, uv->v); + uv->v = &uv->u.value; + luaC_upvalclosed(L, uv); +} + void luaF_freeproto(lua_State* L, Proto* f, lua_Page* page) { luaM_freearray(L, f->code, f->sizecode, Instruction, f->memcat); diff --git a/VM/src/lfunc.h b/VM/src/lfunc.h index 59ab5726..899d040b 100644 --- a/VM/src/lfunc.h +++ b/VM/src/lfunc.h @@ -12,6 +12,7 @@ LUAI_FUNC Closure* luaF_newLclosure(lua_State* L, int nelems, Table* e, Proto* p LUAI_FUNC Closure* luaF_newCclosure(lua_State* L, int nelems, Table* e); LUAI_FUNC UpVal* luaF_findupval(lua_State* L, StkId level); LUAI_FUNC void luaF_close(lua_State* L, StkId level); +LUAI_FUNC void luaF_closeupval(lua_State* L, UpVal* uv, bool dead); LUAI_FUNC void luaF_freeproto(lua_State* L, Proto* f, struct lua_Page* page); LUAI_FUNC void luaF_freeclosure(lua_State* L, Closure* c, struct lua_Page* page); LUAI_FUNC void luaF_unlinkupval(UpVal* uv); diff --git a/VM/src/lgc.cpp b/VM/src/lgc.cpp index f7a851f4..b95d6dee 100644 --- a/VM/src/lgc.cpp +++ b/VM/src/lgc.cpp @@ -13,6 +13,117 @@ #include +/* + * Luau uses an incremental non-generational non-moving mark&sweep garbage collector. + * + * The collector runs in three stages: mark, atomic and sweep. Mark and sweep are incremental and try to do a limited amount + * of work every GC step; atomic is ran once per the GC cycle and is indivisible. In either case, the work happens during GC + * steps that are "scheduled" by the GC pacing algorithm - the steps happen either from explicit calls to lua_gc, or after + * the mutator (aka application) allocates some amount of memory, which is known as "GC assist". In either case, GC steps + * can't happen concurrently with other access to VM state. + * + * Current GC stage is stored in global_State::gcstate, and has two additional stages for pause and second-phase mark, explained below. + * + * GC pacer is an algorithm that tries to ensure that GC can always catch up to the application allocating garbage, but do this + * with minimal amount of effort. To configure the pacer Luau provides control over three variables: GC goal, defined as the + * target heap size during atomic phase in relation to live heap size (e.g. 200% goal means the heap's worst case size is double + * the total size of alive objects), step size (how many kilobytes should the application allocate for GC step to trigger), and + * GC multiplier (how much should the GC try to mark relative to how much the application allocated). It's critical that step + * multiplier is significantly above 1, as this is what allows the GC to catch up to the application's allocation rate, and + * GC goal and GC multiplier are linked in subtle ways, described in lua.h comments for LUA_GCSETGOAL. + * + * During mark, GC tries to identify all reachable objects and mark them as reachable, while keeping unreachable objects unmarked. + * During sweep, GC tries to sweep all objects that were not reachable at the end of mark. The atomic phase is needed to ensure + * that all pending marking has completed and all objects that are still marked as unreachable are, in fact, unreachable. + * + * Notably, during mark GC doesn't free any objects, and so the heap size constantly grows; during sweep, GC doesn't do any marking + * work, so it can't immediately free objects that became unreachable after sweeping started. + * + * Every collectable object has one of three colors at any given point in time: white, gray or black. This coloring scheme + * is necessary to implement incremental marking: white objects have not been marked and may be unreachable, black objects + * have been marked and will not be marked again if they stay black, and gray objects have been marked but may contain unmarked + * references. + * + * Objects are allocated as white; however, during sweep, we need to differentiate between objects that remained white in the mark + * phase (these are not reachable and can be freed) and objects that were allocated after the mark phase ended. Because of this, the + * colors are encoded using three bits inside GCheader::marked: white0, white1 and black (so technically we use a four-color scheme: + * any object can be white0, white1, gray or black). All bits are exclusive, and gray objects have all three bits unset. This allows + * us to have the "current" white bit, which is flipped during atomic stage - during sweeping, objects that have the white color from + * the previous mark may be deleted, and all other objects may or may not be reachable, and will be changed to the current white color, + * so that the next mark can start coloring objects from scratch again. + * + * Crucially, the coloring scheme comes with what's known as a tri-color invariant: a black object may never point to a white object. + * + * At the end of atomic stage, the expectation is that there are no gray objects anymore, which means all objects are either black + * (reachable) or white (unreachable = dead). Tri-color invariant is maintained throughout mark and atomic phase. To uphold this + * invariant, every modification of an object needs to check if the object is black and the new referent is white; if so, we + * need to either mark the referent, making it non-white (known as a forward barrier), or mark the object as gray and queue it + * for additional marking (known as a backward barrier). + * + * Luau uses both types of barriers. Forward barriers advance GC progress, since they don't create new outstanding work for GC, + * but they may be expensive when an object is modified many times in succession. Backward barriers are cheaper, as they defer + * most of the work until "later", but they require queueing the object for a rescan which isn't always possible. Table writes usually + * use backward barriers (but switch to forward barriers during second-phase mark), whereas upvalue writes and setmetatable use forward + * barriers. + * + * Since marking is incremental, it needs a way to track progress, which is implemented as a gray set: at any point, objects that + * are gray need to mark their white references, objects that are black have no pending work, and objects that are white have not yet + * been reached. Once the gray set is empty, the work completes; as such, incremental marking is as simple as removing an object from + * the gray set, and turning it to black (which requires turning all its white references to gray). The gray set is implemented as + * an intrusive singly linked list, using `gclist` field in multiple objects (functions, tables, threads and protos). When an object + * doesn't have gclist field, the marking of that object needs to be "immediate", changing the colors of all references in one go. + * + * When a black object is modified, it needs to become gray again. Objects like this are placed on a separate `grayagain` list by a + * barrier - this is important because it allows us to have a mark stage that terminates when the gray set is empty even if the mutator + * is constantly changing existing objects to gray. After mark stage finishes traversing `gray` list, we copy `grayagain` list to `gray` + * once and incrementally mark it again. During this phase of marking, we may get more objects marked as `grayagain`, so after we finish + * emptying out the `gray` list the second time, we finish the mark stage and do final marking of `grayagain` during atomic phase. + * GC works correctly without this second-phase mark (called GCSpropagateagain), but it reduces the time spent during atomic phase. + * + * Sweeping is also incremental, but instead of working at a granularity of an object, it works at a granularity of a page: all GC + * objects are allocated in special pages (see lmem.cpp for details), and sweeper traverses all objects in one page in one incremental + * step, freeing objects that aren't reachable (old white), and recoloring all other objects with the new white to prepare them for next + * mark. During sweeping we don't need to maintain the GC invariant, because our goal is to paint all objects with current white - + * however, some barriers will still trigger (because some reachable objects are still black as sweeping didn't get to them yet), and + * some barriers will proactively mark black objects as white to avoid extra barriers from triggering excessively. + * + * Most references that GC deals with are strong, and as such they fit neatly into the incremental marking scheme. Some, however, are + * weak - notably, tables can be marked as having weak keys/values (using __mode metafield). During incremental marking, we don't know + * for certain if a given object is alive - if it's marked as black, it definitely was reachable during marking, but if it's marked as + * white, we don't know if it's actually unreachable. Because of this, we need to defer weak table handling to the atomic phase; after + * all objects are marked, we traverse all weak tables (that are linked into special weak table lists using `gclist` during marking), + * and remove all entries that have white keys or values. If keys or values are strong, they are marked normally. + * + * The simplified scheme described above isn't fully accurate because of threads, upvalues and strings. + * + * Strings are semantically black (they are initially white, and when the mark stage reaches a string, it changes its color and never + * touches the object again), but they are technically marked as gray - the black bit is never set on a string object. This behavior + * is inherited from Lua 5.1 GC, but doesn't have a clear rationale - effectively, strings are marked as gray but are never part of + * a gray list. + * + * Threads are hard to deal with because for them to fit into the white-gray-black scheme, writes to thread stacks need to have barriers + * that turn the thread from black (already scanned) to gray - but this is very expensive because stack writes are very common. To + * get around this problem, threads have an "active" state which means that a thread is actively executing code. When GC reaches an active + * thread, it keeps it as gray, and rescans it during atomic phase. When a thread is inactive, GC instead paints the thread black. All + * API calls that can write to thread stacks outside of execution (which implies active) uses a thread barrier that checks if the thread is + * black, and if it is it marks it as gray and puts it on a gray list to be rescanned during atomic phase. + * + * NOTE: The above is only true when LuauNoSleepBit is enabled. + * + * Upvalues are special objects that can be closed, in which case they contain the value (acting as a reference cell) and can be dealt + * with using the regular algorithm, or open, in which case they refer to a stack slot in some other thread. These are difficult to deal + * with because the stack writes are not monitored. Because of this open upvalues are treated in a somewhat special way: they are never marked + * as black (doing so would violate the GC invariant), and they are kept in a special global list (global_State::uvhead) which is traversed + * during atomic phase. This is needed because an open upvalue might point to a stack location in a dead thread that never marked the stack + * slot - upvalues like this are identified since they don't have `markedopen` bit set during thread traversal and closed in `clearupvals`. + * + * NOTE: The above is only true when LuauSimplerUpval is enabled. + */ + +LUAU_FASTFLAGVARIABLE(LuauSimplerUpval, false) +LUAU_FASTFLAGVARIABLE(LuauNoSleepBit, false) +LUAU_FASTFLAGVARIABLE(LuauEagerShrink, false) + #define GC_SWEEPPAGESTEPCOST 16 #define GC_INTERRUPT(state) \ @@ -150,8 +261,8 @@ static void reallymarkobject(global_State* g, GCObject* o) { UpVal* uv = gco2uv(o); markvalue(g, uv->v); - if (uv->v == &uv->u.value) // closed? - gray2black(o); // open upvalues are never black + if (!upisopen(uv)) // closed? + gray2black(o); // open upvalues are never black return; } case LUA_TFUNCTION: @@ -289,22 +400,34 @@ static void traverseclosure(global_State* g, Closure* cl) } } -static void traversestack(global_State* g, lua_State* l, bool clearstack) +static void traversestack(global_State* g, lua_State* l) { markobject(g, l->gt); if (l->namecall) stringmark(l->namecall); for (StkId o = l->stack; o < l->top; o++) markvalue(g, o); - // final traversal? - if (g->gcstate == GCSatomic || clearstack) + if (FFlag::LuauSimplerUpval) { - StkId stack_end = l->stack + l->stacksize; - for (StkId o = l->top; o < stack_end; o++) // clear not-marked stack slice - setnilvalue(o); + for (UpVal* uv = l->openupval; uv; uv = uv->u.open.threadnext) + { + LUAU_ASSERT(upisopen(uv)); + uv->markedopen = 1; + markobject(g, uv); + } } } +static void clearstack(lua_State* l) +{ + StkId stack_end = l->stack + l->stacksize; + for (StkId o = l->top; o < stack_end; o++) // clear not-marked stack slice + setnilvalue(o); +} + +// TODO: pull function definition here when FFlag::LuauEagerShrink is removed +static void shrinkstack(lua_State* L); + /* ** traverse one gray object, turning it to black. ** Returns `quantity' traversed. @@ -338,14 +461,17 @@ static size_t propagatemark(global_State* g) LUAU_ASSERT(!luaC_threadsleeping(th)); - // threads that are executing and the main thread are not deactivated + // threads that are executing and the main thread remain gray bool active = luaC_threadactive(th) || th == th->global->mainthread; + // TODO: Refactor this logic after LuauNoSleepBit is removed if (!active && g->gcstate == GCSpropagate) { - traversestack(g, th, /* clearstack= */ true); + traversestack(g, th); + clearstack(th); - l_setbit(th->stackstate, THREAD_SLEEPINGBIT); + if (!FFlag::LuauNoSleepBit) + l_setbit(th->stackstate, THREAD_SLEEPINGBIT); } else { @@ -354,9 +480,17 @@ static size_t propagatemark(global_State* g) black2gray(o); - traversestack(g, th, /* clearstack= */ false); + traversestack(g, th); + + // final traversal? + if (g->gcstate == GCSatomic) + clearstack(th); } + // we could shrink stack at any time but we opt to skip it during atomic since it's redundant to do that more than once per cycle + if (FFlag::LuauEagerShrink && g->gcstate != GCSatomic) + shrinkstack(th); + return sizeof(lua_State) + sizeof(TValue) * th->stacksize + sizeof(CallInfo) * th->size_ci; } case LUA_TPROTO: @@ -537,7 +671,7 @@ static bool deletegco(void* context, lua_Page* page, GCObject* gco) // we are in the process of deleting everything // threads with open upvalues will attempt to close them all on removal // but those upvalues might point to stack values that were already deleted - if (gco->gch.tt == LUA_TTHREAD) + if (!FFlag::LuauSimplerUpval && gco->gch.tt == LUA_TTHREAD) { lua_State* th = gco2th(gco); @@ -595,13 +729,53 @@ static void markroot(lua_State* L) static size_t remarkupvals(global_State* g) { size_t work = 0; - for (UpVal* uv = g->uvhead.u.l.next; uv != &g->uvhead; uv = uv->u.l.next) + + for (UpVal* uv = g->uvhead.u.open.next; uv != &g->uvhead; uv = uv->u.open.next) { work += sizeof(UpVal); - LUAU_ASSERT(uv->u.l.next->u.l.prev == uv && uv->u.l.prev->u.l.next == uv); + + LUAU_ASSERT(upisopen(uv)); + LUAU_ASSERT(uv->u.open.next->u.open.prev == uv && uv->u.open.prev->u.open.next == uv); + LUAU_ASSERT(!isblack(obj2gco(uv))); // open upvalues are never black + if (isgray(obj2gco(uv))) markvalue(g, uv->v); } + + return work; +} + +static size_t clearupvals(lua_State* L) +{ + global_State* g = L->global; + + size_t work = 0; + + for (UpVal* uv = g->uvhead.u.open.next; uv != &g->uvhead;) + { + work += sizeof(UpVal); + + LUAU_ASSERT(upisopen(uv)); + LUAU_ASSERT(uv->u.open.next->u.open.prev == uv && uv->u.open.prev->u.open.next == uv); + LUAU_ASSERT(!isblack(obj2gco(uv))); // open upvalues are never black + LUAU_ASSERT(iswhite(obj2gco(uv)) || !iscollectable(uv->v) || !iswhite(gcvalue(uv->v))); + + if (uv->markedopen) + { + // upvalue is still open (belongs to alive thread) + LUAU_ASSERT(isgray(obj2gco(uv))); + uv->markedopen = 0; // for next cycle + uv = uv->u.open.next; + } + else + { + // upvalue is either dead, or alive but the thread is dead; unlink and close + UpVal* next = uv->u.open.next; + luaF_closeupval(L, uv, /* dead= */ iswhite(obj2gco(uv))); + uv = next; + } + } + return work; } @@ -654,6 +828,16 @@ static size_t atomic(lua_State* L) g->gcmetrics.currcycle.atomictimeclear += recordGcDeltaTime(currts); #endif + if (FFlag::LuauSimplerUpval) + { + // close orphaned live upvalues of dead threads and clear dead upvalues + work += clearupvals(L); + +#ifdef LUAI_GCMETRICS + g->gcmetrics.currcycle.atomictimeupval += recordGcDeltaTime(currts); +#endif + } + // flip current white g->currentwhite = cast_byte(otherwhite(g)); g->sweepgcopage = g->allgcopages; @@ -677,8 +861,11 @@ static bool sweepgco(lua_State* L, lua_Page* page, GCObject* gco) if (alive) { - resetbit(th->stackstate, THREAD_SLEEPINGBIT); - shrinkstack(th); + if (!FFlag::LuauNoSleepBit) + resetbit(th->stackstate, THREAD_SLEEPINGBIT); + + if (!FFlag::LuauEagerShrink) + shrinkstack(th); } } @@ -945,7 +1132,7 @@ void luaC_fullgc(lua_State* L) startGcCycleMetrics(g); #endif - if (g->gcstate <= GCSatomic) + if (FFlag::LuauSimplerUpval ? keepinvariant(g) : g->gcstate <= GCSatomic) { // reset sweep marks to sweep all elements (returning them to white) g->sweepgcopage = g->allgcopages; @@ -955,7 +1142,7 @@ void luaC_fullgc(lua_State* L) g->weak = NULL; g->gcstate = GCSsweep; } - LUAU_ASSERT(g->gcstate == GCSsweep); + LUAU_ASSERT(g->gcstate == GCSpause || g->gcstate == GCSsweep); // finish any pending sweep phase while (g->gcstate != GCSpause) { @@ -963,6 +1150,16 @@ void luaC_fullgc(lua_State* L) gcstep(L, SIZE_MAX); } + if (FFlag::LuauSimplerUpval) + { + // clear markedopen bits for all open upvalues; these might be stuck from half-finished mark prior to full gc + for (UpVal* uv = g->uvhead.u.open.next; uv != &g->uvhead; uv = uv->u.open.next) + { + LUAU_ASSERT(upisopen(uv)); + uv->markedopen = 0; + } + } + #ifdef LUAI_GCMETRICS finishGcCycleMetrics(g); startGcCycleMetrics(g); @@ -999,6 +1196,7 @@ void luaC_fullgc(lua_State* L) void luaC_barrierupval(lua_State* L, GCObject* v) { + LUAU_ASSERT(!FFlag::LuauSimplerUpval); global_State* g = L->global; LUAU_ASSERT(iswhite(v) && !isdead(g, v)); @@ -1038,30 +1236,24 @@ void luaC_barriertable(lua_State* L, Table* t, GCObject* v) g->grayagain = o; } -void luaC_barrierback(lua_State* L, Table* t) +void luaC_barrierback(lua_State* L, GCObject* o, GCObject** gclist) { global_State* g = L->global; - GCObject* o = obj2gco(t); LUAU_ASSERT(isblack(o) && !isdead(g, o)); LUAU_ASSERT(g->gcstate != GCSpause); - black2gray(o); // make table gray (again) - t->gclist = g->grayagain; + + black2gray(o); // make object gray (again) + *gclist = g->grayagain; g->grayagain = o; } -void luaC_initobj(lua_State* L, GCObject* o, uint8_t tt) -{ - global_State* g = L->global; - o->gch.marked = luaC_white(g); - o->gch.tt = tt; - o->gch.memcat = L->activememcat; -} - -void luaC_initupval(lua_State* L, UpVal* uv) +void luaC_upvalclosed(lua_State* L, UpVal* uv) { global_State* g = L->global; GCObject* o = obj2gco(uv); + LUAU_ASSERT(!upisopen(uv)); // upvalue was closed but needs GC state fixup + if (isgray(o)) { if (keepinvariant(g)) @@ -1105,6 +1297,7 @@ int64_t luaC_allocationrate(lua_State* L) void luaC_wakethread(lua_State* L) { + LUAU_ASSERT(!FFlag::LuauNoSleepBit); if (!luaC_threadsleeping(L)) return; @@ -1116,6 +1309,8 @@ void luaC_wakethread(lua_State* L) { GCObject* o = obj2gco(L); + LUAU_ASSERT(isblack(o)); + L->gclist = g->grayagain; g->grayagain = o; diff --git a/VM/src/lgc.h b/VM/src/lgc.h index 7b03a25d..69379c89 100644 --- a/VM/src/lgc.h +++ b/VM/src/lgc.h @@ -6,6 +6,8 @@ #include "lobject.h" #include "lstate.h" +LUAU_FASTFLAG(LuauNoSleepBit) + /* ** Default settings for GC tunables (settable via lua_gc) */ @@ -74,6 +76,7 @@ #define luaC_white(g) cast_to(uint8_t, ((g)->currentwhite) & WHITEBITS) // Thread stack states +// TODO: Remove with FFlag::LuauNoSleepBit and replace with lua_State::threadactive #define THREAD_ACTIVEBIT 0 // thread is currently active #define THREAD_SLEEPINGBIT 1 // thread is not executing and stack should not be modified @@ -109,7 +112,7 @@ #define luaC_barrierfast(L, t) \ { \ if (isblack(obj2gco(t))) \ - luaC_barrierback(L, t); \ + luaC_barrierback(L, obj2gco(t), &t->gclist); \ } #define luaC_objbarrier(L, p, o) \ @@ -118,29 +121,43 @@ luaC_barrierf(L, obj2gco(p), obj2gco(o)); \ } +// TODO: Remove with FFlag::LuauSimplerUpval #define luaC_upvalbarrier(L, uv, tv) \ { \ if (iscollectable(tv) && iswhite(gcvalue(tv)) && (!(uv) || (uv)->v != &(uv)->u.value)) \ luaC_barrierupval(L, gcvalue(tv)); \ } -#define luaC_checkthreadsleep(L) \ +#define luaC_threadbarrier(L) \ { \ - if (luaC_threadsleeping(L)) \ - luaC_wakethread(L); \ + if (FFlag::LuauNoSleepBit) \ + { \ + if (isblack(obj2gco(L))) \ + luaC_barrierback(L, obj2gco(L), &L->gclist); \ + } \ + else \ + { \ + if (luaC_threadsleeping(L)) \ + luaC_wakethread(L); \ + } \ } -#define luaC_init(L, o, tt) luaC_initobj(L, cast_to(GCObject*, (o)), tt) +#define luaC_init(L, o, tt_) \ + { \ + o->marked = luaC_white(L->global); \ + o->tt = tt_; \ + o->memcat = L->activememcat; \ + } LUAI_FUNC void luaC_freeall(lua_State* L); LUAI_FUNC size_t luaC_step(lua_State* L, bool assist); LUAI_FUNC void luaC_fullgc(lua_State* L); LUAI_FUNC void luaC_initobj(lua_State* L, GCObject* o, uint8_t tt); -LUAI_FUNC void luaC_initupval(lua_State* L, UpVal* uv); +LUAI_FUNC void luaC_upvalclosed(lua_State* L, UpVal* uv); LUAI_FUNC void luaC_barrierupval(lua_State* L, GCObject* v); LUAI_FUNC void luaC_barrierf(lua_State* L, GCObject* o, GCObject* v); LUAI_FUNC void luaC_barriertable(lua_State* L, Table* t, GCObject* v); -LUAI_FUNC void luaC_barrierback(lua_State* L, Table* t); +LUAI_FUNC void luaC_barrierback(lua_State* L, GCObject* o, GCObject** gclist); LUAI_FUNC void luaC_validate(lua_State* L); LUAI_FUNC void luaC_dump(lua_State* L, void* file, const char* (*categoryName)(lua_State* L, uint8_t memcat)); LUAI_FUNC int64_t luaC_allocationrate(lua_State* L); diff --git a/VM/src/lgcdebug.cpp b/VM/src/lgcdebug.cpp index b6204b1a..2f9c1756 100644 --- a/VM/src/lgcdebug.cpp +++ b/VM/src/lgcdebug.cpp @@ -102,10 +102,12 @@ static void validatestack(global_State* g, lua_State* l) if (l->namecall) validateobjref(g, obj2gco(l), obj2gco(l->namecall)); - for (UpVal* uv = l->openupval; uv; uv = uv->u.l.threadnext) + for (UpVal* uv = l->openupval; uv; uv = uv->u.open.threadnext) { LUAU_ASSERT(uv->tt == LUA_TUPVAL); - LUAU_ASSERT(uv->v != &uv->u.value); + LUAU_ASSERT(upisopen(uv)); + LUAU_ASSERT(uv->u.open.next->u.open.prev == uv && uv->u.open.prev->u.open.next == uv); + LUAU_ASSERT(!isblack(obj2gco(uv))); // open upvalues are never black } } @@ -235,11 +237,12 @@ void luaC_validate(lua_State* L) luaM_visitgco(L, L, validategco); - for (UpVal* uv = g->uvhead.u.l.next; uv != &g->uvhead; uv = uv->u.l.next) + for (UpVal* uv = g->uvhead.u.open.next; uv != &g->uvhead; uv = uv->u.open.next) { LUAU_ASSERT(uv->tt == LUA_TUPVAL); - LUAU_ASSERT(uv->v != &uv->u.value); - LUAU_ASSERT(uv->u.l.next->u.l.prev == uv && uv->u.l.prev->u.l.next == uv); + LUAU_ASSERT(upisopen(uv)); + LUAU_ASSERT(uv->u.open.next->u.open.prev == uv && uv->u.open.prev->u.open.next == uv); + LUAU_ASSERT(!isblack(obj2gco(uv))); // open upvalues are never black } } @@ -508,13 +511,14 @@ static void dumpproto(FILE* f, Proto* p) static void dumpupval(FILE* f, UpVal* uv) { - fprintf(f, "{\"type\":\"upvalue\",\"cat\":%d,\"size\":%d", uv->memcat, int(sizeof(UpVal))); + fprintf(f, "{\"type\":\"upvalue\",\"cat\":%d,\"size\":%d,\"open\":%s", uv->memcat, int(sizeof(UpVal)), upisopen(uv) ? "true" : "false"); if (iscollectable(uv->v)) { fprintf(f, ",\"object\":"); dumpref(f, gcvalue(uv->v)); } + fprintf(f, "}"); } diff --git a/VM/src/lobject.h b/VM/src/lobject.h index 2097e335..778e22ba 100644 --- a/VM/src/lobject.h +++ b/VM/src/lobject.h @@ -232,7 +232,7 @@ typedef struct TString int16_t atom; // 2 byte padding - TString* next; // next string in the hash table bucket or the string buffer linked list + TString* next; // next string in the hash table bucket unsigned int hash; unsigned int len; @@ -316,7 +316,10 @@ typedef struct LocVar typedef struct UpVal { CommonHeader; - // 1 (x86) or 5 (x64) byte padding + uint8_t markedopen; // set if reachable from an alive thread (only valid during atomic) + + // 4 byte padding (x64) + TValue* v; // points to stack or to its own value union { @@ -327,14 +330,16 @@ typedef struct UpVal struct UpVal* prev; struct UpVal* next; - // thread double linked list (when open) + // thread linked list (when open) struct UpVal* threadnext; // note: this is the location of a pointer to this upvalue in the previous element that can be either an UpVal or a lua_State - struct UpVal** threadprev; - } l; + struct UpVal** threadprev; // TODO: remove with FFlag::LuauSimplerUpval + } open; } u; } UpVal; +#define upisopen(up) ((up)->v != &(up)->u.value) + /* ** Closures */ diff --git a/VM/src/lstate.cpp b/VM/src/lstate.cpp index 4489f840..e1cb2ab7 100644 --- a/VM/src/lstate.cpp +++ b/VM/src/lstate.cpp @@ -10,6 +10,8 @@ #include "ldo.h" #include "ldebug.h" +LUAU_FASTFLAG(LuauSimplerUpval) + /* ** Main thread combines a thread state and the global state */ @@ -119,8 +121,11 @@ lua_State* luaE_newthread(lua_State* L) void luaE_freethread(lua_State* L, lua_State* L1, lua_Page* page) { - luaF_close(L1, L1->stack); // close all upvalues for this thread - LUAU_ASSERT(L1->openupval == NULL); + if (!FFlag::LuauSimplerUpval) + { + luaF_close(L1, L1->stack); // close all upvalues for this thread + LUAU_ASSERT(L1->openupval == NULL); + } global_State* g = L->global; if (g->cb.userthread) g->cb.userthread(NULL, L1); @@ -175,8 +180,8 @@ lua_State* lua_newstate(lua_Alloc f, void* ud) g->frealloc = f; g->ud = ud; g->mainthread = L; - g->uvhead.u.l.prev = &g->uvhead; - g->uvhead.u.l.next = &g->uvhead; + g->uvhead.u.open.prev = &g->uvhead; + g->uvhead.u.open.next = &g->uvhead; g->GCthreshold = 0; // mark it as unfinished state g->registryfree = 0; g->errorjmp = NULL; diff --git a/VM/src/lstate.h b/VM/src/lstate.h index 72a09713..df47ce7e 100644 --- a/VM/src/lstate.h +++ b/VM/src/lstate.h @@ -167,7 +167,7 @@ typedef struct global_State GCObject* grayagain; // list of objects to be traversed atomically GCObject* weak; // list of weak tables (to be cleared) - TString* strbufgc; // list of all string buffer objects + TString* strbufgc; // list of all string buffer objects; TODO: remove with LuauNoStrbufLink size_t GCthreshold; // when totalbytes > GCthreshold, run GC step diff --git a/VM/src/lstring.cpp b/VM/src/lstring.cpp index 9c266031..f43d03b1 100644 --- a/VM/src/lstring.cpp +++ b/VM/src/lstring.cpp @@ -7,6 +7,8 @@ #include +LUAU_FASTFLAGVARIABLE(LuauNoStrbufLink, false) + unsigned int luaS_hash(const char* str, size_t len) { // Note that this hashing algorithm is replicated in BytecodeBuilder.cpp, BytecodeBuilder::getStringHash @@ -70,40 +72,33 @@ void luaS_resize(lua_State* L, int newsize) static TString* newlstr(lua_State* L, const char* str, size_t l, unsigned int h) { - TString* ts; - stringtable* tb; if (l > MAXSSIZE) luaM_toobig(L); - ts = luaM_newgco(L, TString, sizestring(l), L->activememcat); - ts->len = unsigned(l); + + TString* ts = luaM_newgco(L, TString, sizestring(l), L->activememcat); + luaC_init(L, ts, LUA_TSTRING); + ts->atom = ATOM_UNDEF; ts->hash = h; - ts->marked = luaC_white(L->global); - ts->tt = LUA_TSTRING; - ts->memcat = L->activememcat; + ts->len = unsigned(l); + memcpy(ts->data, str, l); ts->data[l] = '\0'; // ending 0 - ts->atom = ATOM_UNDEF; - tb = &L->global->strt; + + stringtable* tb = &L->global->strt; h = lmod(h, tb->size); ts->next = tb->hash[h]; // chain new entry tb->hash[h] = ts; + tb->nuse++; if (tb->nuse > cast_to(uint32_t, tb->size) && tb->size <= INT_MAX / 2) luaS_resize(L, tb->size * 2); // too crowded + return ts; } -static void linkstrbuf(lua_State* L, TString* ts) -{ - global_State* g = L->global; - - ts->next = g->strbufgc; - g->strbufgc = ts; - ts->marked = luaC_white(g); -} - static void unlinkstrbuf(lua_State* L, TString* ts) { + LUAU_ASSERT(!FFlag::LuauNoStrbufLink); global_State* g = L->global; TString** p = &g->strbufgc; @@ -129,14 +124,24 @@ TString* luaS_bufstart(lua_State* L, size_t size) if (size > MAXSSIZE) luaM_toobig(L); + global_State* g = L->global; + TString* ts = luaM_newgco(L, TString, sizestring(size), L->activememcat); - - ts->tt = LUA_TSTRING; - ts->memcat = L->activememcat; - linkstrbuf(L, ts); - + luaC_init(L, ts, LUA_TSTRING); + ts->atom = ATOM_UNDEF; + ts->hash = 0; // computed in luaS_buffinish ts->len = unsigned(size); + if (FFlag::LuauNoStrbufLink) + { + ts->next = NULL; + } + else + { + ts->next = g->strbufgc; + g->strbufgc = ts; + } + return ts; } @@ -159,7 +164,10 @@ TString* luaS_buffinish(lua_State* L, TString* ts) } } - unlinkstrbuf(L, ts); + if (FFlag::LuauNoStrbufLink) + LUAU_ASSERT(ts->next == NULL); + else + unlinkstrbuf(L, ts); ts->hash = h; ts->data[ts->len] = '\0'; // ending 0 @@ -214,11 +222,21 @@ static bool unlinkstr(lua_State* L, TString* ts) void luaS_free(lua_State* L, TString* ts, lua_Page* page) { - // Unchain from the string table - if (!unlinkstr(L, ts)) - unlinkstrbuf(L, ts); // An unlikely scenario when we have a string buffer on our hands + if (FFlag::LuauNoStrbufLink) + { + if (unlinkstr(L, ts)) + L->global->strt.nuse--; + else + LUAU_ASSERT(ts->next == NULL); // orphaned string buffer + } else - L->global->strt.nuse--; + { + // Unchain from the string table + if (!unlinkstr(L, ts)) + unlinkstrbuf(L, ts); // An unlikely scenario when we have a string buffer on our hands + else + L->global->strt.nuse--; + } luaM_freegco(L, ts, sizestring(ts->len), ts->memcat, page); } diff --git a/VM/src/lvmexecute.cpp b/VM/src/lvmexecute.cpp index 376dd400..7306b055 100644 --- a/VM/src/lvmexecute.cpp +++ b/VM/src/lvmexecute.cpp @@ -16,7 +16,8 @@ #include -LUAU_FASTFLAGVARIABLE(LuauNicerMethodErrors, false) +LUAU_FASTFLAG(LuauSimplerUpval) +LUAU_FASTFLAG(LuauNoSleepBit) // Disable c99-designator to avoid the warning in CGOTO dispatch table #ifdef __clang__ @@ -111,7 +112,7 @@ LUAU_FASTFLAGVARIABLE(LuauNicerMethodErrors, false) VM_DISPATCH_OP(LOP_LOADKX), VM_DISPATCH_OP(LOP_JUMPX), VM_DISPATCH_OP(LOP_FASTCALL), VM_DISPATCH_OP(LOP_COVERAGE), \ VM_DISPATCH_OP(LOP_CAPTURE), VM_DISPATCH_OP(LOP_JUMPIFEQK), VM_DISPATCH_OP(LOP_JUMPIFNOTEQK), VM_DISPATCH_OP(LOP_FASTCALL1), \ VM_DISPATCH_OP(LOP_FASTCALL2), VM_DISPATCH_OP(LOP_FASTCALL2K), VM_DISPATCH_OP(LOP_FORGPREP), VM_DISPATCH_OP(LOP_JUMPXEQKNIL), \ - VM_DISPATCH_OP(LOP_JUMPXEQKB), VM_DISPATCH_OP(LOP_JUMPXEQKN), VM_DISPATCH_OP(LOP_JUMPXEQKS), \ + VM_DISPATCH_OP(LOP_JUMPXEQKB), VM_DISPATCH_OP(LOP_JUMPXEQKN), VM_DISPATCH_OP(LOP_JUMPXEQKS), #if defined(__GNUC__) || defined(__clang__) #define VM_USE_CGOTO 1 @@ -317,6 +318,7 @@ static void luau_execute(lua_State* L) LUAU_ASSERT(isLua(L->ci)); LUAU_ASSERT(luaC_threadactive(L)); LUAU_ASSERT(!luaC_threadsleeping(L)); + LUAU_ASSERT(!FFlag::LuauNoSleepBit || !isblack(obj2gco(L))); // we don't use luaC_threadbarrier because active threads never turn black pc = L->ci->savedpc; cl = clvalue(L->ci->func); @@ -496,7 +498,8 @@ static void luau_execute(lua_State* L) setobj(L, uv->v, ra); luaC_barrier(L, uv, ra); - luaC_upvalbarrier(L, uv, uv->v); + if (!FFlag::LuauSimplerUpval) + luaC_upvalbarrier(L, uv, uv->v); VM_NEXT(); } @@ -932,7 +935,7 @@ static void luau_execute(lua_State* L) VM_PATCH_C(pc - 2, L->cachedslot); // recompute ra since stack might have been reallocated ra = VM_REG(LUAU_INSN_A(insn)); - if (FFlag::LuauNicerMethodErrors && ttisnil(ra)) + if (ttisnil(ra)) luaG_methoderror(L, ra + 1, tsvalue(kv)); } } @@ -973,7 +976,7 @@ static void luau_execute(lua_State* L) VM_PATCH_C(pc - 2, L->cachedslot); // recompute ra since stack might have been reallocated ra = VM_REG(LUAU_INSN_A(insn)); - if (FFlag::LuauNicerMethodErrors && ttisnil(ra)) + if (ttisnil(ra)) luaG_methoderror(L, ra + 1, tsvalue(kv)); } } @@ -984,7 +987,7 @@ static void luau_execute(lua_State* L) VM_PROTECT(luaV_gettable(L, rb, kv, ra)); // recompute ra since stack might have been reallocated ra = VM_REG(LUAU_INSN_A(insn)); - if (FFlag::LuauNicerMethodErrors && ttisnil(ra)) + if (ttisnil(ra)) luaG_methoderror(L, ra + 1, tsvalue(kv)); } } diff --git a/VM/src/lvmload.cpp b/VM/src/lvmload.cpp index 86afddd2..0ae85ab6 100644 --- a/VM/src/lvmload.cpp +++ b/VM/src/lvmload.cpp @@ -351,7 +351,7 @@ int luau_load(lua_State* L, const char* chunkname, const char* data, size_t size uint32_t mainid = readVarInt(data, size, offset); Proto* main = protos[mainid]; - luaC_checkthreadsleep(L); + luaC_threadbarrier(L); Closure* cl = luaF_newLclosure(L, 0, envt, main); setclvalue(L, L->top, cl); diff --git a/VM/src/lvmutils.cpp b/VM/src/lvmutils.cpp index 8be241e0..33d47020 100644 --- a/VM/src/lvmutils.cpp +++ b/VM/src/lvmutils.cpp @@ -10,9 +10,6 @@ #include "lnumutils.h" #include -#include - -LUAU_FASTFLAGVARIABLE(LuauBetterNewindex, false) // limit for table tag-method chains (to avoid loops) #define MAXTAGLOOP 100 @@ -142,46 +139,25 @@ void luaV_settable(lua_State* L, const TValue* t, TValue* key, StkId val) { // `t' is a table? Table* h = hvalue(t); - if (FFlag::LuauBetterNewindex) - { - const TValue* oldval = luaH_get(h, key); + const TValue* oldval = luaH_get(h, key); - // should we assign the key? (if key is valid or __newindex is not set) - if (!ttisnil(oldval) || (tm = fasttm(L, h->metatable, TM_NEWINDEX)) == NULL) - { - if (h->readonly) - luaG_readonlyerror(L); - - // luaH_set would work but would repeat the lookup so we use luaH_setslot that can reuse oldval if it's safe - TValue* newval = luaH_setslot(L, h, oldval, key); - - L->cachedslot = gval2slot(h, newval); // remember slot to accelerate future lookups - - setobj2t(L, newval, val); - luaC_barriert(L, h, val); - return; - } - - // fallthrough to metamethod - } - else + // should we assign the key? (if key is valid or __newindex is not set) + if (!ttisnil(oldval) || (tm = fasttm(L, h->metatable, TM_NEWINDEX)) == NULL) { if (h->readonly) luaG_readonlyerror(L); - TValue* oldval = luaH_set(L, h, key); // do a primitive set + // luaH_set would work but would repeat the lookup so we use luaH_setslot that can reuse oldval if it's safe + TValue* newval = luaH_setslot(L, h, oldval, key); - L->cachedslot = gval2slot(h, oldval); // remember slot to accelerate future lookups + L->cachedslot = gval2slot(h, newval); // remember slot to accelerate future lookups - if (!ttisnil(oldval) || // result is no nil? - (tm = fasttm(L, h->metatable, TM_NEWINDEX)) == NULL) - { // or no TM? - setobj2t(L, oldval, val); - luaC_barriert(L, h, val); - return; - } - // else will try the tag method + setobj2t(L, newval, val); + luaC_barriert(L, h, val); + return; } + + // fallthrough to metamethod } else if (ttisnil(tm = luaT_gettmbyobj(L, t, TM_NEWINDEX))) luaG_indexerror(L, t, key); diff --git a/bench/bench.py b/bench/bench.py index 42a0ac97..0db33950 100644 --- a/bench/bench.py +++ b/bench/bench.py @@ -38,6 +38,7 @@ argumentParser.add_argument('--run-test', action='store', default=None, help='Re argumentParser.add_argument('--extra-loops', action='store',type=int,default=0, help='Amount of times to loop over one test (one test already performs multiple runs)') argumentParser.add_argument('--filename', action='store',type=str,default='bench', help='File name for graph and results file') argumentParser.add_argument('--callgrind', dest='callgrind',action='store_const',const=1,default=0,help='Use callgrind to run benchmarks') +argumentParser.add_argument('--show-commands', dest='show_commands',action='store_const',const=1,default=0,help='Show the command line used to launch the VM and tests') if matplotlib != None: argumentParser.add_argument('--absolute', dest='absolute',action='store_const',const=1,default=0,help='Display absolute values instead of relative (enabled by default when benchmarking a single VM)') @@ -87,17 +88,25 @@ def getCallgrindOutput(lines): return "".join(result) +def conditionallyShowCommand(cmd): + if arguments.show_commands: + print(f'{colored(Color.BLUE, "EXECUTING")}: {cmd}') + def getVmOutput(cmd): if os.name == "nt": try: - return subprocess.check_output("start /realtime /affinity 1 /b /wait cmd /C \"" + cmd + "\"", shell=True, cwd=scriptdir).decode() + fullCmd = "start /realtime /affinity 1 /b /wait cmd /C \"" + cmd + "\"" + conditionallyShowCommand(fullCmd) + return subprocess.check_output(fullCmd, shell=True, cwd=scriptdir).decode() except KeyboardInterrupt: exit(1) except: return "" elif arguments.callgrind: try: - subprocess.check_call("valgrind --tool=callgrind --callgrind-out-file=callgrind.out --combine-dumps=yes --dump-line=no " + cmd, shell=True, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, cwd=scriptdir) + fullCmd = "valgrind --tool=callgrind --callgrind-out-file=callgrind.out --combine-dumps=yes --dump-line=no " + cmd + conditionallyShowCommand(fullCmd) + subprocess.check_call(fullCmd, shell=True, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, cwd=scriptdir) path = os.path.join(scriptdir, "callgrind.out") with open(path, "r") as file: lines = file.readlines() @@ -106,6 +115,7 @@ def getVmOutput(cmd): except: return "" else: + conditionallyShowCommand(cmd) with subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True, cwd=scriptdir) as p: # Try to lock to a single processor if sys.platform != "darwin": diff --git a/tests/AstJsonEncoder.test.cpp b/tests/AstJsonEncoder.test.cpp index 3ff36741..a23f6f47 100644 --- a/tests/AstJsonEncoder.test.cpp +++ b/tests/AstJsonEncoder.test.cpp @@ -2,6 +2,7 @@ #include "Luau/Ast.h" #include "Luau/AstJsonEncoder.h" #include "Luau/Parser.h" +#include "ScopedFlags.h" #include "doctest.h" @@ -175,6 +176,17 @@ TEST_CASE_FIXTURE(JsonEncoderFixture, "encode_AstExprIfThen") CHECK(toJson(statement) == expected); } +TEST_CASE_FIXTURE(JsonEncoderFixture, "encode_AstExprInterpString") +{ + ScopedFastFlag sff{"LuauInterpolatedStringBaseSupport", true}; + + AstStat* statement = expectParseStatement("local a = `var = {x}`"); + + std::string_view expected = + R"({"type":"AstStatLocal","location":"0,0 - 0,17","vars":[{"luauType":null,"name":"a","type":"AstLocal","location":"0,6 - 0,7"}],"values":[{"type":"AstExprInterpString","location":"0,10 - 0,17","strings":["var = ",""],"expressions":[{"type":"AstExprGlobal","location":"0,18 - 0,19","global":"x"}]}]})"; + + CHECK(toJson(statement) == expected); +} TEST_CASE("encode_AstExprLocal") { diff --git a/tests/AstQuery.test.cpp b/tests/AstQuery.test.cpp index 6ec1426c..2b650fa4 100644 --- a/tests/AstQuery.test.cpp +++ b/tests/AstQuery.test.cpp @@ -138,4 +138,80 @@ print(workspace:) REQUIRE(ancestry.back()->is()); } +TEST_CASE_FIXTURE(Fixture, "Luau_query") +{ + AstStatBlock* block = parse(R"( + if true then + end + )"); + + AstStatIf* if_ = Luau::query(block); + CHECK(if_); +} + +TEST_CASE_FIXTURE(Fixture, "Luau_query_for_2nd_if_stat_which_doesnt_exist") +{ + AstStatBlock* block = parse(R"( + if true then + end + )"); + + AstStatIf* if_ = Luau::query(block); + CHECK(!if_); +} + +TEST_CASE_FIXTURE(Fixture, "Luau_nested_query") +{ + AstStatBlock* block = parse(R"( + if true then + end + )"); + + AstStatIf* if_ = Luau::query(block); + REQUIRE(if_); + AstExprConstantBool* bool_ = Luau::query(if_); + REQUIRE(bool_); +} + +TEST_CASE_FIXTURE(Fixture, "Luau_nested_query_but_first_query_failed") +{ + AstStatBlock* block = parse(R"( + if true then + end + )"); + + AstStatIf* if_ = Luau::query(block); + REQUIRE(!if_); + AstExprConstantBool* bool_ = Luau::query(if_); // ensure it doesn't crash + REQUIRE(!bool_); +} + +TEST_CASE_FIXTURE(Fixture, "Luau_selectively_query_for_a_different_boolean") +{ + AstStatBlock* block = parse(R"( + local x = false and true + local y = true and false + )"); + + AstExprConstantBool* fst = Luau::query(block, {nth(), nth(2)}); + REQUIRE(fst); + REQUIRE(fst->value == true); + + AstExprConstantBool* snd = Luau::query(block, {nth(2), nth(2)}); + REQUIRE(snd); + REQUIRE(snd->value == false); +} + +TEST_CASE_FIXTURE(Fixture, "Luau_selectively_query_for_a_different_boolean_2") +{ + AstStatBlock* block = parse(R"( + local x = false and true + local y = true and false + )"); + + AstExprConstantBool* snd = Luau::query(block, {nth(2), nth()}); + REQUIRE(snd); + REQUIRE(snd->value == true); +} + TEST_SUITE_END(); diff --git a/tests/Autocomplete.test.cpp b/tests/Autocomplete.test.cpp index 25447dd8..988cbe80 100644 --- a/tests/Autocomplete.test.cpp +++ b/tests/Autocomplete.test.cpp @@ -2708,6 +2708,15 @@ a = if temp then even else abc@3 CHECK(ac.entryMap.count("abcdef")); } +TEST_CASE_FIXTURE(ACFixture, "autocomplete_interpolated_string") +{ + check(R"(f(`expression = {@1}`))"); + + auto ac = autocomplete('1'); + CHECK(ac.entryMap.count("table")); + CHECK_EQ(ac.context, AutocompleteContext::Expression); +} + TEST_CASE_FIXTURE(ACFixture, "autocomplete_explicit_type_pack") { check(R"( diff --git a/tests/Compiler.test.cpp b/tests/Compiler.test.cpp index a2e748a8..0a3c6507 100644 --- a/tests/Compiler.test.cpp +++ b/tests/Compiler.test.cpp @@ -1230,6 +1230,58 @@ RETURN R0 0 )"); } +TEST_CASE("InterpStringWithNoExpressions") +{ + ScopedFastFlag sff{"LuauInterpolatedStringBaseSupport", true}; + + CHECK_EQ(compileFunction0(R"(return "hello")"), compileFunction0("return `hello`")); +} + +TEST_CASE("InterpStringZeroCost") +{ + ScopedFastFlag sff{"LuauInterpolatedStringBaseSupport", true}; + + CHECK_EQ( + "\n" + compileFunction0(R"(local _ = `hello, {"world"}!`)"), + R"( +LOADK R1 K0 +LOADK R3 K1 +NAMECALL R1 R1 K2 +CALL R1 2 1 +MOVE R0 R1 +RETURN R0 0 +)" + ); +} + +TEST_CASE("InterpStringRegisterCleanup") +{ + ScopedFastFlag sff{"LuauInterpolatedStringBaseSupport", true}; + + CHECK_EQ( + "\n" + compileFunction0(R"( + local a, b, c = nil, "um", "uh oh" + a = `foo{"bar"}` + print(a) + )"), + + R"( +LOADNIL R0 +LOADK R1 K0 +LOADK R2 K1 +LOADK R3 K2 +LOADK R5 K3 +NAMECALL R3 R3 K4 +CALL R3 2 1 +MOVE R0 R3 +GETIMPORT R3 6 +MOVE R4 R0 +CALL R3 1 0 +RETURN R0 0 +)" + ); +} + TEST_CASE("ConstantFoldArith") { CHECK_EQ("\n" + compileFunction0("return 10 + 2"), R"( @@ -2102,8 +2154,6 @@ TEST_CASE("RecursionParse") CHECK_EQ(std::string(e.what()), "Exceeded allowed recursion depth; simplify your expression to make the code compile"); } -#if 0 - // This currently requires too much stack space on MSVC/x64 and crashes with stack overflow at recursion depth 935 try { Luau::compileOrThrow(bcb, rep("function a() ", 1500) + "print()" + rep(" end", 1500)); @@ -2123,7 +2173,6 @@ TEST_CASE("RecursionParse") { CHECK_EQ(std::string(e.what()), "Exceeded allowed recursion depth; simplify your block to make the code compile"); } -#endif } TEST_CASE("ArrayIndexLiteral") diff --git a/tests/Conformance.test.cpp b/tests/Conformance.test.cpp index be2feacb..f6f5b41f 100644 --- a/tests/Conformance.test.cpp +++ b/tests/Conformance.test.cpp @@ -294,6 +294,14 @@ TEST_CASE("Strings") runConformance("strings.lua"); } +TEST_CASE("StringInterp") +{ + ScopedFastFlag sffInterpStrings{"LuauInterpolatedStringBaseSupport", true}; + ScopedFastFlag sffTostringFormat{"LuauTostringFormatSpecifier", true}; + + runConformance("stringinterp.lua"); +} + TEST_CASE("VarArg") { runConformance("vararg.lua"); @@ -311,15 +319,11 @@ TEST_CASE("Literals") TEST_CASE("Errors") { - ScopedFastFlag sff("LuauNicerMethodErrors", true); - runConformance("errors.lua"); } TEST_CASE("Events") { - ScopedFastFlag sff("LuauBetterNewindex", true); - runConformance("events.lua"); } diff --git a/tests/Fixture.cpp b/tests/Fixture.cpp index 40be39a0..4051f851 100644 --- a/tests/Fixture.cpp +++ b/tests/Fixture.cpp @@ -512,4 +512,41 @@ void dump(const std::vector& constraints) printf("%s\n", toString(c, opts).c_str()); } +FindNthOccurenceOf::FindNthOccurenceOf(Nth nth) + : requestedNth(nth) +{ +} + +bool FindNthOccurenceOf::checkIt(AstNode* n) +{ + if (theNode) + return false; + + if (n->classIndex == requestedNth.classIndex) + { + // Human factor: the requestedNth starts from 1 because of the term `nth`. + if (currentOccurrence + 1 != requestedNth.nth) + ++currentOccurrence; + else + theNode = n; + } + + return !theNode; // once found, returns false and stops traversal +} + +bool FindNthOccurenceOf::visit(AstNode* n) +{ + return checkIt(n); +} + +bool FindNthOccurenceOf::visit(AstType* t) +{ + return checkIt(t); +} + +bool FindNthOccurenceOf::visit(AstTypePack* t) +{ + return checkIt(t); +} + } // namespace Luau diff --git a/tests/Fixture.h b/tests/Fixture.h index 8dc3dd24..e82ebf00 100644 --- a/tests/Fixture.h +++ b/tests/Fixture.h @@ -195,6 +195,76 @@ std::optional lookupName(ScopePtr scope, const std::string& name); // Wa std::optional linearSearchForBinding(Scope* scope, const char* name); +struct Nth +{ + int classIndex; + int nth; +}; + +template +Nth nth(int nth = 1) +{ + static_assert(std::is_base_of_v, "T must be a derived class of AstNode"); + LUAU_ASSERT(nth > 0); // Did you mean to use `nth(1)`? + + return Nth{T::ClassIndex(), nth}; +} + +struct FindNthOccurenceOf : public AstVisitor +{ + Nth requestedNth; + size_t currentOccurrence = 0; + AstNode* theNode = nullptr; + + FindNthOccurenceOf(Nth nth); + + bool checkIt(AstNode* n); + + bool visit(AstNode* n) override; + bool visit(AstType* n) override; + bool visit(AstTypePack* n) override; +}; + +/** DSL querying of the AST. + * + * Given an AST, one can query for a particular node directly without having to manually unwrap the tree, for example: + * + * ``` + * if a and b then + * print(a + b) + * end + * + * function f(x, y) + * return x + y + * end + * ``` + * + * There are numerous ways to access the second AstExprBinary. + * 1. Luau::query(block, {nth(), nth()}) + * 2. Luau::query(Luau::query(block)) + * 3. Luau::query(block, {nth(2)}) + */ +template +T* query(AstNode* node, const std::vector& nths = {nth(N)}) +{ + static_assert(std::is_base_of_v, "T must be a derived class of AstNode"); + + // If a nested query call fails to find the node in question, subsequent calls can propagate rather than trying to do more. + // This supports `query(query(...))` + + for (Nth nth : nths) + { + if (!node) + return nullptr; + + FindNthOccurenceOf finder{nth}; + node->visit(&finder); + node = finder.theNode; + } + + return node ? node->as() : nullptr; +} + } // namespace Luau #define LUAU_REQUIRE_ERRORS(result) \ diff --git a/tests/Lexer.test.cpp b/tests/Lexer.test.cpp index 20d8d0d5..890d100b 100644 --- a/tests/Lexer.test.cpp +++ b/tests/Lexer.test.cpp @@ -138,4 +138,90 @@ TEST_CASE("lookahead") CHECK_EQ(lexer.lookahead().type, Lexeme::Eof); } +TEST_CASE("string_interpolation_basic") +{ + ScopedFastFlag sff{"LuauInterpolatedStringBaseSupport", true}; + + const std::string testInput = R"(`foo {"bar"}`)"; + Luau::Allocator alloc; + AstNameTable table(alloc); + Lexer lexer(testInput.c_str(), testInput.size(), table); + + Lexeme interpBegin = lexer.next(); + CHECK_EQ(interpBegin.type, Lexeme::InterpStringBegin); + + Lexeme quote = lexer.next(); + CHECK_EQ(quote.type, Lexeme::QuotedString); + + Lexeme interpEnd = lexer.next(); + CHECK_EQ(interpEnd.type, Lexeme::InterpStringEnd); +} + +TEST_CASE("string_interpolation_double_brace") +{ + ScopedFastFlag sff{"LuauInterpolatedStringBaseSupport", true}; + + const std::string testInput = R"(`foo{{bad}}bar`)"; + Luau::Allocator alloc; + AstNameTable table(alloc); + Lexer lexer(testInput.c_str(), testInput.size(), table); + + auto brokenInterpBegin = lexer.next(); + CHECK_EQ(brokenInterpBegin.type, Lexeme::BrokenInterpDoubleBrace); + CHECK_EQ(std::string(brokenInterpBegin.data, brokenInterpBegin.length), std::string("foo")); + + CHECK_EQ(lexer.next().type, Lexeme::Name); + + auto interpEnd = lexer.next(); + CHECK_EQ(interpEnd.type, Lexeme::InterpStringEnd); + CHECK_EQ(std::string(interpEnd.data, interpEnd.length), std::string("}bar")); +} + +TEST_CASE("string_interpolation_double_but_unmatched_brace") +{ + ScopedFastFlag sff{"LuauInterpolatedStringBaseSupport", true}; + + const std::string testInput = R"(`{{oops}`, 1)"; + Luau::Allocator alloc; + AstNameTable table(alloc); + Lexer lexer(testInput.c_str(), testInput.size(), table); + + CHECK_EQ(lexer.next().type, Lexeme::BrokenInterpDoubleBrace); + CHECK_EQ(lexer.next().type, Lexeme::Name); + CHECK_EQ(lexer.next().type, Lexeme::InterpStringEnd); + CHECK_EQ(lexer.next().type, ','); + CHECK_EQ(lexer.next().type, Lexeme::Number); +} + +TEST_CASE("string_interpolation_unmatched_brace") +{ + ScopedFastFlag sff{"LuauInterpolatedStringBaseSupport", true}; + + const std::string testInput = R"({ + `hello {"world"} + } -- this might be incorrectly parsed as a string)"; + Luau::Allocator alloc; + AstNameTable table(alloc); + Lexer lexer(testInput.c_str(), testInput.size(), table); + + CHECK_EQ(lexer.next().type, '{'); + CHECK_EQ(lexer.next().type, Lexeme::InterpStringBegin); + CHECK_EQ(lexer.next().type, Lexeme::QuotedString); + CHECK_EQ(lexer.next().type, Lexeme::BrokenString); + CHECK_EQ(lexer.next().type, '}'); +} + +TEST_CASE("string_interpolation_with_unicode_escape") +{ + ScopedFastFlag sff{"LuauInterpolatedStringBaseSupport", true}; + + const std::string testInput = R"(`\u{1F41B}`)"; + Luau::Allocator alloc; + AstNameTable table(alloc); + Lexer lexer(testInput.c_str(), testInput.size(), table); + + CHECK_EQ(lexer.next().type, Lexeme::InterpStringSimple); + CHECK_EQ(lexer.next().type, Lexeme::Eof); +} + TEST_SUITE_END(); diff --git a/tests/Linter.test.cpp b/tests/Linter.test.cpp index 64c6d3e4..a7d09e8f 100644 --- a/tests/Linter.test.cpp +++ b/tests/Linter.test.cpp @@ -43,6 +43,20 @@ TEST_CASE_FIXTURE(Fixture, "DeprecatedGlobal") CHECK_EQ(result.warnings[0].text, "Global 'Wait' is deprecated, use 'wait' instead"); } +TEST_CASE_FIXTURE(Fixture, "DeprecatedGlobalNoReplacement") +{ + ScopedFastFlag sff{"LuauLintFixDeprecationMessage", true}; + + // Normally this would be defined externally, so hack it in for testing + const char* deprecationReplacementString = ""; + addGlobalBinding(typeChecker, "Version", Binding{typeChecker.anyType, {}, true, deprecationReplacementString}); + + LintResult result = lintTyped("Version()"); + + REQUIRE_EQ(result.warnings.size(), 1); + CHECK_EQ(result.warnings[0].text, "Global 'Version' is deprecated"); +} + TEST_CASE_FIXTURE(Fixture, "PlaceholderRead") { LintResult result = lint(R"( @@ -1662,17 +1676,31 @@ TEST_CASE_FIXTURE(Fixture, "WrongCommentOptimize") { LintResult result = lint(R"( --!optimize ---!optimize --!optimize me --!optimize 100500 --!optimize 2 )"); - REQUIRE_EQ(result.warnings.size(), 4); + REQUIRE_EQ(result.warnings.size(), 3); CHECK_EQ(result.warnings[0].text, "optimize directive requires an optimization level"); - CHECK_EQ(result.warnings[1].text, "optimize directive requires an optimization level"); - CHECK_EQ(result.warnings[2].text, "optimize directive uses unknown optimization level 'me', 0..2 expected"); - CHECK_EQ(result.warnings[3].text, "optimize directive uses unknown optimization level '100500', 0..2 expected"); + CHECK_EQ(result.warnings[1].text, "optimize directive uses unknown optimization level 'me', 0..2 expected"); + CHECK_EQ(result.warnings[2].text, "optimize directive uses unknown optimization level '100500', 0..2 expected"); + + result = lint("--!optimize "); + REQUIRE_EQ(result.warnings.size(), 1); + CHECK_EQ(result.warnings[0].text, "optimize directive requires an optimization level"); +} + +TEST_CASE_FIXTURE(Fixture, "TestStringInterpolation") +{ + ScopedFastFlag sff{"LuauInterpolatedStringBaseSupport", true}; + + LintResult result = lint(R"( + --!nocheck + local _ = `unknown {foo}` + )"); + + REQUIRE_EQ(result.warnings.size(), 1); } TEST_CASE_FIXTURE(Fixture, "IntegerParsing") diff --git a/tests/Parser.test.cpp b/tests/Parser.test.cpp index c55ec181..2dd47708 100644 --- a/tests/Parser.test.cpp +++ b/tests/Parser.test.cpp @@ -905,6 +905,146 @@ TEST_CASE_FIXTURE(Fixture, "parse_compound_assignment_error_multiple") } } +TEST_CASE_FIXTURE(Fixture, "parse_interpolated_string_double_brace_begin") +{ + ScopedFastFlag sff{"LuauInterpolatedStringBaseSupport", true}; + + try + { + parse(R"( + _ = `{{oops}}` + )"); + FAIL("Expected ParseErrors to be thrown"); + } + catch (const ParseErrors& e) + { + CHECK_EQ("Double braces are not permitted within interpolated strings. Did you mean '\\{'?", e.getErrors().front().getMessage()); + } +} + +TEST_CASE_FIXTURE(Fixture, "parse_interpolated_string_double_brace_mid") +{ + ScopedFastFlag sff{"LuauInterpolatedStringBaseSupport", true}; + + try + { + parse(R"( + _ = `{nice} {{oops}}` + )"); + FAIL("Expected ParseErrors to be thrown"); + } + catch (const ParseErrors& e) + { + CHECK_EQ("Double braces are not permitted within interpolated strings. Did you mean '\\{'?", e.getErrors().front().getMessage()); + } +} + +TEST_CASE_FIXTURE(Fixture, "parse_interpolated_string_without_end_brace") +{ + ScopedFastFlag sff{"LuauInterpolatedStringBaseSupport", true}; + + auto columnOfEndBraceError = [this](const char* code) + { + try + { + parse(code); + FAIL("Expected ParseErrors to be thrown"); + return UINT_MAX; + } + catch (const ParseErrors& e) + { + CHECK_EQ(e.getErrors().size(), 1); + + auto error = e.getErrors().front(); + CHECK_EQ("Malformed interpolated string, did you forget to add a '}'?", error.getMessage()); + return error.getLocation().begin.column; + } + }; + + // This makes sure that the error is coming from the brace itself + CHECK_EQ(columnOfEndBraceError("_ = `{a`"), columnOfEndBraceError("_ = `{abcdefg`")); + CHECK_NE(columnOfEndBraceError("_ = `{a`"), columnOfEndBraceError("_ = `{a`")); +} + +TEST_CASE_FIXTURE(Fixture, "parse_interpolated_string_without_end_brace_in_table") +{ + ScopedFastFlag sff{"LuauInterpolatedStringBaseSupport", true}; + + try + { + parse(R"( + _ = { `{a` } + )"); + FAIL("Expected ParseErrors to be thrown"); + } + catch (const ParseErrors& e) + { + CHECK_EQ(e.getErrors().size(), 2); + + CHECK_EQ("Malformed interpolated string, did you forget to add a '}'?", e.getErrors().front().getMessage()); + CHECK_EQ("Expected '}' (to close '{' at line 2), got ", e.getErrors().back().getMessage()); + } +} + +TEST_CASE_FIXTURE(Fixture, "parse_interpolated_string_mid_without_end_brace_in_table") +{ + ScopedFastFlag sff{"LuauInterpolatedStringBaseSupport", true}; + + try + { + parse(R"( + _ = { `x {"y"} {z` } + )"); + FAIL("Expected ParseErrors to be thrown"); + } + catch (const ParseErrors& e) + { + CHECK_EQ(e.getErrors().size(), 2); + + CHECK_EQ("Malformed interpolated string, did you forget to add a '}'?", e.getErrors().front().getMessage()); + CHECK_EQ("Expected '}' (to close '{' at line 2), got ", e.getErrors().back().getMessage()); + } +} + +TEST_CASE_FIXTURE(Fixture, "parse_interpolated_string_as_type_fail") +{ + ScopedFastFlag sff{"LuauInterpolatedStringBaseSupport", true}; + + try + { + parse(R"( + local a: `what` = `???` + local b: `what {"the"}` = `???` + local c: `what {"the"} heck` = `???` + )"); + FAIL("Expected ParseErrors to be thrown"); + } + catch (const ParseErrors& parseErrors) + { + CHECK_EQ(parseErrors.getErrors().size(), 3); + + for (ParseError error : parseErrors.getErrors()) + CHECK_EQ(error.getMessage(), "Interpolated string literals cannot be used as types"); + } +} + +TEST_CASE_FIXTURE(Fixture, "parse_interpolated_string_call_without_parens") +{ + ScopedFastFlag sff{"LuauInterpolatedStringBaseSupport", true}; + + try + { + parse(R"( + _ = print `{42}` + )"); + FAIL("Expected ParseErrors to be thrown"); + } + catch (const ParseErrors& e) + { + CHECK_EQ("Expected identifier when parsing expression, got `{", e.getErrors().front().getMessage()); + } +} + TEST_CASE_FIXTURE(Fixture, "parse_nesting_based_end_detection") { try diff --git a/tests/ToString.test.cpp b/tests/ToString.test.cpp index fe376d86..ab5d8598 100644 --- a/tests/ToString.test.cpp +++ b/tests/ToString.test.cpp @@ -11,6 +11,7 @@ using namespace Luau; LUAU_FASTFLAG(LuauRecursiveTypeParameterRestriction); LUAU_FASTFLAG(LuauSpecialTypesAsterisked); +LUAU_FASTFLAG(LuauFixNameMaps); TEST_SUITE_BEGIN("ToString"); @@ -433,29 +434,40 @@ TEST_CASE_FIXTURE(Fixture, "toStringDetailed") LUAU_REQUIRE_NO_ERRORS(result); - TypeId id3Type = requireType("id3"); - ToStringResult nameData = toStringDetailed(id3Type); + ToStringOptions opts; + + TypeId id3Type = requireType("id3"); + ToStringResult nameData = toStringDetailed(id3Type, opts); + + if (FFlag::LuauFixNameMaps) + REQUIRE(3 == opts.nameMap.typeVars.size()); + else + REQUIRE_EQ(3, nameData.DEPRECATED_nameMap.typeVars.size()); - REQUIRE_EQ(3, nameData.nameMap.typeVars.size()); REQUIRE_EQ("(a, b, c) -> (a, b, c)", nameData.name); - ToStringOptions opts; - opts.nameMap = std::move(nameData.nameMap); + ToStringOptions opts2; // TODO: delete opts2 when clipping FFlag::LuauFixNameMaps + if (FFlag::LuauFixNameMaps) + opts2.nameMap = std::move(opts.nameMap); + else + opts2.DEPRECATED_nameMap = std::move(nameData.DEPRECATED_nameMap); const FunctionTypeVar* ftv = get(follow(id3Type)); REQUIRE(ftv != nullptr); auto params = flatten(ftv->argTypes).first; - REQUIRE_EQ(3, params.size()); + REQUIRE(3 == params.size()); - REQUIRE_EQ("a", toString(params[0], opts)); - REQUIRE_EQ("b", toString(params[1], opts)); - REQUIRE_EQ("c", toString(params[2], opts)); + CHECK("a" == toString(params[0], opts2)); + CHECK("b" == toString(params[1], opts2)); + CHECK("c" == toString(params[2], opts2)); } TEST_CASE_FIXTURE(BuiltinsFixture, "toStringDetailed2") { - ScopedFastFlag sff2{"DebugLuauSharedSelf", true}; + ScopedFastFlag sff[] = { + {"DebugLuauSharedSelf", true}, + }; CheckResult result = check(R"( local base = {} @@ -470,13 +482,18 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "toStringDetailed2") )"); LUAU_REQUIRE_NO_ERRORS(result); - TypeId tType = requireType("inst"); - ToStringResult r = toStringDetailed(tType); - CHECK_EQ("{ @metatable { __index: { @metatable {| __index: base |}, child } }, inst }", r.name); - CHECK_EQ(0, r.nameMap.typeVars.size()); - ToStringOptions opts; - opts.nameMap = r.nameMap; + + TypeId tType = requireType("inst"); + ToStringResult r = toStringDetailed(tType, opts); + CHECK_EQ("{ @metatable { __index: { @metatable {| __index: base |}, child } }, inst }", r.name); + if (FFlag::LuauFixNameMaps) + CHECK(0 == opts.nameMap.typeVars.size()); + else + CHECK_EQ(0, r.DEPRECATED_nameMap.typeVars.size()); + + if (!FFlag::LuauFixNameMaps) + opts.DEPRECATED_nameMap = r.DEPRECATED_nameMap; const MetatableTypeVar* tMeta = get(tType); REQUIRE(tMeta); @@ -499,7 +516,8 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "toStringDetailed2") REQUIRE(tMeta6); ToStringResult oneResult = toStringDetailed(tMeta5->props["one"].type, opts); - opts.nameMap = oneResult.nameMap; + if (!FFlag::LuauFixNameMaps) + opts.DEPRECATED_nameMap = oneResult.DEPRECATED_nameMap; std::string twoResult = toString(tMeta6->props["two"].type, opts); diff --git a/tests/Transpiler.test.cpp b/tests/Transpiler.test.cpp index d2ed9aef..e79bc9b7 100644 --- a/tests/Transpiler.test.cpp +++ b/tests/Transpiler.test.cpp @@ -6,6 +6,7 @@ #include "Luau/Transpiler.h" #include "Fixture.h" +#include "ScopedFlags.h" #include "doctest.h" @@ -678,4 +679,22 @@ TEST_CASE_FIXTURE(Fixture, "transpile_for_in_multiple_types") CHECK_EQ(code, transpile(code, {}, true).code); } +TEST_CASE_FIXTURE(Fixture, "transpile_string_interp") +{ + ScopedFastFlag sff{"LuauInterpolatedStringBaseSupport", true}; + + std::string code = R"( local _ = `hello {name}` )"; + + CHECK_EQ(code, transpile(code, {}, true).code); +} + +TEST_CASE_FIXTURE(Fixture, "transpile_string_literal_escape") +{ + ScopedFastFlag sff{"LuauInterpolatedStringBaseSupport", true}; + + std::string code = R"( local _ = ` bracket = \{, backtick = \` = {'ok'} ` )"; + + CHECK_EQ(code, transpile(code, {}, true).code); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.builtins.test.cpp b/tests/TypeInfer.builtins.test.cpp index 10da0efa..12fb4aa2 100644 --- a/tests/TypeInfer.builtins.test.cpp +++ b/tests/TypeInfer.builtins.test.cpp @@ -10,6 +10,7 @@ using namespace Luau; LUAU_FASTFLAG(LuauLowerBoundsCalculation); LUAU_FASTFLAG(LuauSpecialTypesAsterisked); +LUAU_FASTFLAG(LuauStringFormatArgumentErrorFix) TEST_SUITE_BEGIN("BuiltinTests"); @@ -721,7 +722,14 @@ TEST_CASE_FIXTURE(Fixture, "string_format_use_correct_argument") LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK_EQ("Argument count mismatch. Function expects 1 argument, but 2 are specified", toString(result.errors[0])); + if (FFlag::LuauStringFormatArgumentErrorFix) + { + CHECK_EQ("Argument count mismatch. Function expects 2 arguments, but 3 are specified", toString(result.errors[0])); + } + else + { + CHECK_EQ("Argument count mismatch. Function expects 1 argument, but 2 are specified", toString(result.errors[0])); + } } TEST_CASE_FIXTURE(Fixture, "string_format_use_correct_argument2") @@ -736,6 +744,22 @@ TEST_CASE_FIXTURE(Fixture, "string_format_use_correct_argument2") CHECK_EQ("Type 'number' could not be converted into 'string'", toString(result.errors[1])); } +TEST_CASE_FIXTURE(BuiltinsFixture, "string_format_use_correct_argument3") +{ + ScopedFastFlag LuauStringFormatArgumentErrorFix{"LuauStringFormatArgumentErrorFix", true}; + + CheckResult result = check(R"( + local s1 = string.format("%d") + local s2 = string.format("%d", 1) + local s3 = string.format("%d", 1, 2) + )"); + + LUAU_REQUIRE_ERROR_COUNT(2, result); + + CHECK_EQ("Argument count mismatch. Function expects 2 arguments, but only 1 is specified", toString(result.errors[0])); + CHECK_EQ("Argument count mismatch. Function expects 2 arguments, but 3 are specified", toString(result.errors[1])); +} + TEST_CASE_FIXTURE(BuiltinsFixture, "debug_traceback_is_crazy") { CheckResult result = check(R"( diff --git a/tests/TypeInfer.provisional.test.cpp b/tests/TypeInfer.provisional.test.cpp index 9a917a6f..c8fc7f2d 100644 --- a/tests/TypeInfer.provisional.test.cpp +++ b/tests/TypeInfer.provisional.test.cpp @@ -544,4 +544,69 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "greedy_inference_with_shared_self_triggers_f CHECK_EQ("Not all codepaths in this function return 'self, a...'.", toString(result.errors[0])); } +TEST_CASE_FIXTURE(Fixture, "dcr_cant_partially_dispatch_a_constraint") +{ + ScopedFastFlag sff[] = { + {"DebugLuauDeferredConstraintResolution", true}, + {"LuauSpecialTypesAsterisked", true}, + }; + + CheckResult result = check(R"( + local function hasDivisors(value: number) + end + + function prime_iter(state, index) + hasDivisors(index) + index += 1 + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + // We should be able to resolve this to number, but we're not there yet. + // Solving this requires recognizing that we can partially solve the + // following constraint: + // + // (*blocked*) -> () <: (number) -> (b...) + // + // The correct thing for us to do is to consider the constraint dispatched, + // but we need to also record a new constraint number <: *blocked* to finish + // the job later. + CHECK("(a, *error-type*) -> ()" == toString(requireType("prime_iter"))); +} + +TEST_CASE_FIXTURE(Fixture, "free_options_cannot_be_unified_together") +{ + ScopedFastFlag sff[] = { + {"LuauFixNameMaps", true}, + }; + + TypeArena arena; + TypeId nilType = getSingletonTypes().nilType; + + std::unique_ptr scope = std::make_unique(getSingletonTypes().anyTypePack); + + TypeId free1 = arena.addType(FreeTypePack{scope.get()}); + TypeId option1 = arena.addType(UnionTypeVar{{nilType, free1}}); + + TypeId free2 = arena.addType(FreeTypePack{scope.get()}); + TypeId option2 = arena.addType(UnionTypeVar{{nilType, free2}}); + + InternalErrorReporter iceHandler; + UnifierSharedState sharedState{&iceHandler}; + Unifier u{&arena, Mode::Strict, NotNull{scope.get()}, Location{}, Variance::Covariant, sharedState}; + + u.tryUnify(option1, option2); + + CHECK(u.errors.empty()); + + u.log.commit(); + + ToStringOptions opts; + CHECK("a?" == toString(option1, opts)); + + // CHECK("a?" == toString(option2, opts)); // This should hold, but does not. + CHECK("b?" == toString(option2, opts)); // This should not hold. +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.test.cpp b/tests/TypeInfer.test.cpp index 80889363..e1dc5023 100644 --- a/tests/TypeInfer.test.cpp +++ b/tests/TypeInfer.test.cpp @@ -8,6 +8,7 @@ #include "Luau/VisitTypeVar.h" #include "Fixture.h" +#include "ScopedFlags.h" #include "doctest.h" @@ -828,6 +829,41 @@ end LUAU_REQUIRE_NO_ERRORS(result); } +TEST_CASE_FIXTURE(Fixture, "tc_interpolated_string_basic") +{ + ScopedFastFlag sff{"LuauInterpolatedStringBaseSupport", true}; + + CheckResult result = check(R"( + local foo: string = `hello {"world"}` + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "tc_interpolated_string_with_invalid_expression") +{ + ScopedFastFlag sff{"LuauInterpolatedStringBaseSupport", true}; + + CheckResult result = check(R"( + local function f(x: number) end + + local foo: string = `hello {f("uh oh")}` + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); +} + +TEST_CASE_FIXTURE(Fixture, "tc_interpolated_string_constant_type") +{ + ScopedFastFlag sff{"LuauInterpolatedStringBaseSupport", true}; + + CheckResult result = check(R"( + local foo: "hello" = `hello` + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + /* * If it wasn't instantly obvious, we have the fuzzer to thank for this gem of a test. * diff --git a/tests/conformance/gc.lua b/tests/conformance/gc.lua index 5804ea7f..c49dbe77 100644 --- a/tests/conformance/gc.lua +++ b/tests/conformance/gc.lua @@ -252,25 +252,30 @@ if not rawget(_G, "_soft") then end -- create many threads with self-references and open upvalues -local thread_id = 0 -local threads = {} +do + local thread_id = 0 + local threads = {} -function fn(thread) - local x = {} - threads[thread_id] = function() - thread = x - end - coroutine.yield() + function fn(thread) + local x = {} + threads[thread_id] = function() + thread = x + end + coroutine.yield() + end + + while thread_id < 1000 do + local thread = coroutine.create(fn) + coroutine.resume(thread, thread) + thread_id = thread_id + 1 + end + + collectgarbage() + + -- ensure that we no longer have a lot of reachable threads for subsequent tests + threads = {} end -while thread_id < 1000 do - local thread = coroutine.create(fn) - coroutine.resume(thread, thread) - thread_id = thread_id + 1 -end - - - -- create a userdata to be collected when state is closed do local newproxy,assert,type,print,getmetatable = @@ -322,4 +327,27 @@ do collectgarbage() end +-- create a lot of threads with upvalues to force a case where full gc happens after we've marked some upvalues +do + local t = {} + for i = 1,100 do + local c = coroutine.wrap(function() + local uv = {i + 1} + local function f() + return uv[1] * 10 + end + coroutine.yield(uv[1]) + uv = {i + 2} + coroutine.yield(f()) + end) + + assert(c() == i + 1) + table.insert(t, c) + end + + t = {} + + collectgarbage() +end + return('OK') diff --git a/tests/conformance/stringinterp.lua b/tests/conformance/stringinterp.lua new file mode 100644 index 00000000..efb25bae --- /dev/null +++ b/tests/conformance/stringinterp.lua @@ -0,0 +1,59 @@ +local function assertEq(left, right) + assert(typeof(left) == "string", "left is a " .. typeof(left)) + assert(typeof(right) == "string", "right is a " .. typeof(right)) + + if left ~= right then + error(string.format("%q ~= %q", left, right)) + end +end + +assertEq(`hello {"world"}`, "hello world") +assertEq(`Welcome {"to"} {"Luau"}!`, "Welcome to Luau!") + +assertEq(`2 + 2 = {2 + 2}`, "2 + 2 = 4") + +assertEq(`{1} {2} {3} {4} {5} {6} {7}`, "1 2 3 4 5 6 7") + +local combo = {5, 2, 8, 9} +assertEq(`The lock combinations are: {table.concat(combo, ", ")}`, "The lock combinations are: 5, 2, 8, 9") + +assertEq(`true = {true}`, "true = true") + +local name = "Luau" +assertEq(`Welcome to { + name +}!`, "Welcome to Luau!") + +local nameNotConstantEvaluated = (function() return "Luau" end)() +assertEq(`Welcome to {nameNotConstantEvaluated}!`, "Welcome to Luau!") + +assertEq(`This {localName} does not exist`, "This nil does not exist") + +assertEq(`Welcome to \ +{name}!`, "Welcome to \nLuau!") + +assertEq(`empty`, "empty") + +assertEq(`Escaped brace: \{}`, "Escaped brace: {}") +assertEq(`Escaped brace \{} with {"expression"}`, "Escaped brace {} with expression") +assertEq(`Backslash \ that escapes the space is not a part of the string...`, "Backslash that escapes the space is not a part of the string...") +assertEq(`Escaped backslash \\`, "Escaped backslash \\") +assertEq(`Escaped backtick: \``, "Escaped backtick: `") + +assertEq(`Hello {`from inside {"a nested string"}`}`, "Hello from inside a nested string") + +assertEq(`1 {`2 {`3 {4}`}`}`, "1 2 3 4") + +local health = 50 +assert(`You have {health}% health` == "You have 50% health") + +local function shadowsString(string) + return `Value is {string}` +end + +assertEq(shadowsString("hello"), "Value is hello") +assertEq(shadowsString(1), "Value is 1") + +assertEq(`\u{0041}\t`, "A\t") + +return "OK" diff --git a/tools/faillist.txt b/tools/faillist.txt index 630bf9f7..54e7ac05 100644 --- a/tools/faillist.txt +++ b/tools/faillist.txt @@ -53,6 +53,7 @@ AutocompleteTest.generic_types AutocompleteTest.get_suggestions_for_the_very_start_of_the_script AutocompleteTest.global_function_params AutocompleteTest.global_functions_are_not_scoped_lexically +AutocompleteTest.globals_are_order_independent AutocompleteTest.if_then_else_elseif_completions AutocompleteTest.keyword_methods AutocompleteTest.keyword_types @@ -588,7 +589,6 @@ TypeInferFunctions.another_recursive_local_function TypeInferFunctions.calling_function_with_anytypepack_doesnt_leak_free_types TypeInferFunctions.calling_function_with_incorrect_argument_type_yields_errors_spanning_argument TypeInferFunctions.complicated_return_types_require_an_explicit_annotation -TypeInferFunctions.cyclic_function_type_in_args TypeInferFunctions.dont_give_other_overloads_message_if_only_one_argument_matching_overload_exists TypeInferFunctions.dont_infer_parameter_types_for_functions_from_their_call_site TypeInferFunctions.duplicate_functions_with_different_signatures_not_allowed_in_nonstrict @@ -744,7 +744,6 @@ TypeInferUnknownNever.math_operators_and_never TypeInferUnknownNever.type_packs_containing_never_is_itself_uninhabitable TypeInferUnknownNever.type_packs_containing_never_is_itself_uninhabitable2 TypeInferUnknownNever.unary_minus_of_never -TypeInferUnknownNever.unknown_is_reflexive TypePackTests.higher_order_function TypePackTests.multiple_varargs_inference_are_not_confused TypePackTests.no_return_size_should_be_zero diff --git a/tools/natvis/VM.natvis b/tools/natvis/VM.natvis index cb2f355f..e45e4e28 100644 --- a/tools/natvis/VM.natvis +++ b/tools/natvis/VM.natvis @@ -195,7 +195,7 @@ openupval - u.l.threadnext + u.open.threadnext this diff --git a/tools/perfgraph.py b/tools/perfgraph.py index 25bb0dd9..7d2639df 100644 --- a/tools/perfgraph.py +++ b/tools/perfgraph.py @@ -56,7 +56,63 @@ def nodeFromCallstackListFile(source_file): return root -def getDuration(obj): + +def getDuration(nodes, nid): + node = nodes[nid - 1] + total = node['TotalDuration'] + + for cid in node['NodeIds']: + total -= nodes[cid - 1]['TotalDuration'] + + return total + +def getFunctionKey(fn): + return fn['Source'] + "," + fn['Name'] + "," + str(fn['Line']) + +def recursivelyBuildNodeTree(nodes, functions, parent, fid, nid): + ninfo = nodes[nid - 1] + finfo = functions[fid - 1] + + child = parent.child(getFunctionKey(finfo)) + child.source = finfo['Source'] + child.function = finfo['Name'] + child.line = int(finfo['Line']) if finfo['Line'] > 0 else 0 + + child.ticks = getDuration(nodes, nid) + + assert(len(ninfo['FunctionIds']) == len(ninfo['NodeIds'])) + + for i in range(0, len(ninfo['FunctionIds'])): + recursivelyBuildNodeTree(nodes, functions, child, ninfo['FunctionIds'][i], ninfo['NodeIds'][i]) + + return + +def nodeFromJSONV2(dump): + assert(dump['Version'] == 2) + + nodes = dump['Nodes'] + functions = dump['Functions'] + categories = dump['Categories'] + + root = Node() + + for category in categories: + nid = category['NodeId'] + node = nodes[nid - 1] + name = category['Name'] + + child = root.child(name) + child.function = name + child.ticks = getDuration(nodes, nid) + + assert(len(node['FunctionIds']) == len(node['NodeIds'])) + + for i in range(0, len(node['FunctionIds'])): + recursivelyBuildNodeTree(nodes, functions, child, node['FunctionIds'][i], node['NodeIds'][i]) + + return root + +def getDurationV1(obj): total = obj['TotalDuration'] if 'Children' in obj: @@ -73,7 +129,7 @@ def nodeFromJSONObject(node, key, obj): node.source = source node.line = int(line) if len(line) > 0 else 0 - node.ticks = getDuration(obj) + node.ticks = getDurationV1(obj) if 'Children' in obj: for key, obj in obj['Children'].items(): @@ -81,10 +137,8 @@ def nodeFromJSONObject(node, key, obj): return node - -def nodeFromJSONFile(source_file): - dump = json.load(source_file) - +def nodeFromJSONV1(dump): + assert(dump['Version'] == 1) root = Node() if 'Children' in dump: @@ -93,6 +147,16 @@ def nodeFromJSONFile(source_file): return root +def nodeFromJSONFile(source_file): + dump = json.load(source_file) + + if dump['Version'] == 2: + return nodeFromJSONV2(dump) + elif dump['Version'] == 1: + return nodeFromJSONV1(dump) + + return Node() + arguments = argumentParser.parse_args() diff --git a/tools/test_dcr.py b/tools/test_dcr.py index 0efea3c4..da33706c 100644 --- a/tools/test_dcr.py +++ b/tools/test_dcr.py @@ -14,12 +14,14 @@ def loadFailList(): with open(FAIL_LIST_PATH) as f: return set(map(str.strip, f.readlines())) + def safeParseInt(i, default=0): try: return int(i) except ValueError: return default + class Handler(x.ContentHandler): def __init__(self, failList): self.currentTest = [] @@ -47,7 +49,7 @@ class Handler(x.ContentHandler): r = self.results.get(dottedName, True) self.results[dottedName] = r and passed - elif name == 'OverallResultsTestCases': + elif name == "OverallResultsTestCases": self.numSkippedTests = safeParseInt(attrs.get("skipped", 0)) def endElement(self, name): @@ -104,9 +106,9 @@ def main(): for testName, passed in handler.results.items(): if passed and testName in failList: - print('UNEXPECTED: {} should have failed'.format(testName)) + print("UNEXPECTED: {} should have failed".format(testName)) elif not passed and testName not in failList: - print('UNEXPECTED: {} should have passed'.format(testName)) + print("UNEXPECTED: {} should have passed".format(testName)) if args.write: newFailList = sorted( @@ -123,17 +125,24 @@ def main(): print("Updated faillist.txt") if handler.numSkippedTests > 0: - print('{} test(s) were skipped! That probably means that a test segfaulted!'.format(handler.numSkippedTests), file=sys.stderr) + print( + "{} test(s) were skipped! That probably means that a test segfaulted!".format( + handler.numSkippedTests + ), + file=sys.stderr, + ) sys.exit(1) - sys.exit( - 0 - if all( - not passed == (dottedName in failList) - for dottedName, passed in handler.results.items() - ) - else 1 + ok = all( + not passed == (dottedName in failList) + for dottedName, passed in handler.results.items() ) + if ok: + print("Everything in order!", file=sys.stderr) + + sys.exit(0 if ok else 1) + + if __name__ == "__main__": main() From 5eb4fb6089897f0b8effb2acfe413b2f0d4bf653 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Thu, 25 Aug 2022 14:10:12 -0700 Subject: [PATCH 350/399] Fix gcc warning --- tests/Fixture.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/Fixture.h b/tests/Fixture.h index e82ebf00..956f95d0 100644 --- a/tests/Fixture.h +++ b/tests/Fixture.h @@ -213,7 +213,7 @@ Nth nth(int nth = 1) struct FindNthOccurenceOf : public AstVisitor { Nth requestedNth; - size_t currentOccurrence = 0; + int currentOccurrence = 0; AstNode* theNode = nullptr; FindNthOccurenceOf(Nth nth); From b2f9f53ae329a566bca84c4ba1b166cdaed1365f Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Thu, 25 Aug 2022 14:53:50 -0700 Subject: [PATCH 351/399] Sync to upstream/release/542 (#649) - Fix DeprecatedGlobal warning text in cases when the global is deprecated without a suggested alternative - Fix an off-by-one error in type error text for incorrect use of string.format - Reduce stack consumption further during parsing, hopefully eliminating stack overflows during parsing/compilation for good - Mark interpolated string support as experimental (requires --fflags=LuauInterpolatedStringBaseSupport to enable) - Simplify garbage collection treatment of upvalues, reducing cache misses during sweeping stage and reducing the cost of upvalue assignment (SETUPVAL); supersedes #643 - Simplify garbage collection treatment of sleeping threads - Simplify sweeping of alive threads, reducing cache misses during sweeping stage - Simplify management of string buffers, removing redundant linked list operations --- Analysis/include/Luau/Constraint.h | 2 +- Analysis/include/Luau/ConstraintSolver.h | 2 + .../include/Luau/ConstraintSolverLogger.h | 2 +- Analysis/include/Luau/ToString.h | 63 ++++- Analysis/include/Luau/TxnLog.h | 2 + Analysis/src/ConstraintGraphBuilder.cpp | 11 +- Analysis/src/ConstraintSolver.cpp | 119 ++++++-- Analysis/src/ConstraintSolverLogger.cpp | 36 ++- Analysis/src/Linter.cpp | 20 +- Analysis/src/ToDot.cpp | 4 +- Analysis/src/ToString.cpp | 149 +++++----- Analysis/src/TxnLog.cpp | 12 + Analysis/src/TypeVar.cpp | 17 +- Ast/include/Luau/Ast.h | 6 +- Ast/include/Luau/Parser.h | 13 +- Ast/src/Ast.cpp | 10 +- Ast/src/Parser.cpp | 90 +++--- Common/include/Luau/ExperimentalFlags.h | 1 + Compiler/src/Compiler.cpp | 15 +- Makefile | 8 +- VM/src/lapi.cpp | 69 ++--- VM/src/ldebug.cpp | 40 +-- VM/src/ldo.cpp | 8 +- VM/src/lfunc.cpp | 127 ++++++--- VM/src/lfunc.h | 1 + VM/src/lgc.cpp | 259 +++++++++++++++--- VM/src/lgc.h | 31 ++- VM/src/lgcdebug.cpp | 16 +- VM/src/lobject.h | 15 +- VM/src/lstate.cpp | 13 +- VM/src/lstate.h | 2 +- VM/src/lstring.cpp | 74 +++-- VM/src/lvmexecute.cpp | 15 +- VM/src/lvmload.cpp | 2 +- VM/src/lvmutils.cpp | 46 +--- bench/bench.py | 14 +- tests/AstQuery.test.cpp | 76 +++++ tests/Compiler.test.cpp | 3 - tests/Conformance.test.cpp | 4 - tests/Fixture.cpp | 37 +++ tests/Fixture.h | 70 +++++ tests/Linter.test.cpp | 14 + tests/ToString.test.cpp | 52 ++-- tests/TypeInfer.builtins.test.cpp | 26 +- tests/TypeInfer.provisional.test.cpp | 65 +++++ tests/conformance/gc.lua | 60 ++-- tools/faillist.txt | 3 +- tools/natvis/VM.natvis | 2 +- tools/perfgraph.py | 76 ++++- tools/test_dcr.py | 31 ++- 50 files changed, 1362 insertions(+), 471 deletions(-) diff --git a/Analysis/include/Luau/Constraint.h b/Analysis/include/Luau/Constraint.h index ce90f2c5..e9f04e79 100644 --- a/Analysis/include/Luau/Constraint.h +++ b/Analysis/include/Luau/Constraint.h @@ -35,7 +35,7 @@ struct PackSubtypeConstraint TypePackId superPack; }; -// subType ~ gen superType +// generalizedType ~ gen sourceType struct GeneralizationConstraint { TypeId generalizedType; diff --git a/Analysis/include/Luau/ConstraintSolver.h b/Analysis/include/Luau/ConstraintSolver.h index 661d120e..a270ec99 100644 --- a/Analysis/include/Luau/ConstraintSolver.h +++ b/Analysis/include/Luau/ConstraintSolver.h @@ -100,6 +100,8 @@ struct ConstraintSolver void unblock(NotNull progressed); void unblock(TypeId progressed); void unblock(TypePackId progressed); + void unblock(const std::vector& types); + void unblock(const std::vector& packs); /** * @returns true if the TypeId is in a blocked state. diff --git a/Analysis/include/Luau/ConstraintSolverLogger.h b/Analysis/include/Luau/ConstraintSolverLogger.h index fe2177c4..55170c42 100644 --- a/Analysis/include/Luau/ConstraintSolverLogger.h +++ b/Analysis/include/Luau/ConstraintSolverLogger.h @@ -16,7 +16,7 @@ struct ConstraintSolverLogger { std::string compileOutput(); void captureBoundarySnapshot(const Scope* rootScope, std::vector>& unsolvedConstraints); - void prepareStepSnapshot(const Scope* rootScope, NotNull current, std::vector>& unsolvedConstraints); + void prepareStepSnapshot(const Scope* rootScope, NotNull current, std::vector>& unsolvedConstraints, bool force); void commitPreparedStepSnapshot(); private: diff --git a/Analysis/include/Luau/ToString.h b/Analysis/include/Luau/ToString.h index a50fef78..eabbc2be 100644 --- a/Analysis/include/Luau/ToString.h +++ b/Analysis/include/Luau/ToString.h @@ -33,7 +33,8 @@ struct ToStringOptions bool indent = false; size_t maxTableLength = size_t(FInt::LuauTableTypeMaximumStringifierLength); // Only applied to TableTypeVars size_t maxTypeLength = size_t(FInt::LuauTypeMaximumStringifierLength); - std::optional nameMap; + ToStringNameMap nameMap; + std::optional DEPRECATED_nameMap; std::shared_ptr scope; // If present, module names will be added and types that are not available in scope will be marked as 'invalid' std::vector namedFunctionOverrideArgNames; // If present, named function argument names will be overridden }; @@ -41,7 +42,7 @@ struct ToStringOptions struct ToStringResult { std::string name; - ToStringNameMap nameMap; + ToStringNameMap DEPRECATED_nameMap; bool invalid = false; bool error = false; @@ -49,12 +50,24 @@ struct ToStringResult bool truncated = false; }; -ToStringResult toStringDetailed(TypeId ty, const ToStringOptions& opts = {}); -ToStringResult toStringDetailed(TypePackId ty, const ToStringOptions& opts = {}); +ToStringResult toStringDetailed(TypeId ty, ToStringOptions& opts); +ToStringResult toStringDetailed(TypePackId ty, ToStringOptions& opts); -std::string toString(TypeId ty, const ToStringOptions& opts); -std::string toString(TypePackId ty, const ToStringOptions& opts); -std::string toString(const Constraint& c, ToStringOptions& opts); +std::string toString(TypeId ty, ToStringOptions& opts); +std::string toString(TypePackId ty, ToStringOptions& opts); + +// These overloads are selected when a temporary ToStringOptions is passed. (eg +// via an initializer list) +inline std::string toString(TypePackId ty, ToStringOptions&& opts) +{ + // Delegate to the overload (TypePackId, ToStringOptions&) + return toString(ty, opts); +} +inline std::string toString(TypeId ty, ToStringOptions&& opts) +{ + // Delegate to the overload (TypeId, ToStringOptions&) + return toString(ty, opts); +} // These are offered as overloads rather than a default parameter so that they can be easily invoked from within the MSVC debugger. // You can use them in watch expressions! @@ -66,16 +79,42 @@ inline std::string toString(TypePackId ty) { return toString(ty, ToStringOptions{}); } -inline std::string toString(const Constraint& c) + +std::string toString(const Constraint& c, ToStringOptions& opts); + +inline std::string toString(const Constraint& c, ToStringOptions&& opts) { - ToStringOptions opts; return toString(c, opts); } -std::string toString(const TypeVar& tv, const ToStringOptions& opts = {}); -std::string toString(const TypePackVar& tp, const ToStringOptions& opts = {}); +inline std::string toString(const Constraint& c) +{ + return toString(c, ToStringOptions{}); +} -std::string toStringNamedFunction(const std::string& funcName, const FunctionTypeVar& ftv, const ToStringOptions& opts = {}); + +std::string toString(const TypeVar& tv, ToStringOptions& opts); +std::string toString(const TypePackVar& tp, ToStringOptions& opts); + +inline std::string toString(const TypeVar& tv) +{ + ToStringOptions opts; + return toString(tv, opts); +} + +inline std::string toString(const TypePackVar& tp) +{ + ToStringOptions opts; + return toString(tp, opts); +} + +std::string toStringNamedFunction(const std::string& funcName, const FunctionTypeVar& ftv, ToStringOptions& opts); + +inline std::string toStringNamedFunction(const std::string& funcName, const FunctionTypeVar& ftv) +{ + ToStringOptions opts; + return toStringNamedFunction(funcName, ftv, opts); +} // It could be useful to see the text representation of a type during a debugging session instead of exploring the content of the class // These functions will dump the type to stdout and can be evaluated in Watch/Immediate windows or as gdb/lldb expression diff --git a/Analysis/include/Luau/TxnLog.h b/Analysis/include/Luau/TxnLog.h index cd115e3b..016cc927 100644 --- a/Analysis/include/Luau/TxnLog.h +++ b/Analysis/include/Luau/TxnLog.h @@ -263,6 +263,8 @@ struct TxnLog return Luau::get_if(&ty->ty) != nullptr; } + std::pair, std::vector> getChanges() const; + private: // unique_ptr is used to give us stable pointers across insertions into the // map. Otherwise, it would be really easy to accidentally invalidate the diff --git a/Analysis/src/ConstraintGraphBuilder.cpp b/Analysis/src/ConstraintGraphBuilder.cpp index c1e54df2..8f994740 100644 --- a/Analysis/src/ConstraintGraphBuilder.cpp +++ b/Analysis/src/ConstraintGraphBuilder.cpp @@ -210,7 +210,8 @@ void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatLocal* local) for (size_t i = 0; i < local->values.size; ++i) { - if (local->values.data[i]->is()) + AstExpr* value = local->values.data[i]; + if (value->is()) { // HACK: we leave nil-initialized things floating under the assumption that they will later be populated. // See the test TypeInfer/infer_locals_with_nil_value. @@ -218,7 +219,7 @@ void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatLocal* local) } else if (i == local->values.size - 1) { - TypePackId exprPack = checkPack(scope, local->values.data[i]); + TypePackId exprPack = checkPack(scope, value); if (i < local->vars.size) { @@ -229,7 +230,7 @@ void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatLocal* local) } else { - TypeId exprType = check(scope, local->values.data[i]); + TypeId exprType = check(scope, value); if (i < varTypes.size()) addConstraint(scope, SubtypeConstraint{varTypes[i], exprType}); } @@ -1107,9 +1108,7 @@ TypeId ConstraintGraphBuilder::resolveType(const ScopePtr& scope, AstType* ty, b if (topLevel) { - addConstraint(scope, TypeAliasExpansionConstraint{ - /* target */ result, - }); + addConstraint(scope, TypeAliasExpansionConstraint{ /* target */ result }); } } } diff --git a/Analysis/src/ConstraintSolver.cpp b/Analysis/src/ConstraintSolver.cpp index 6c6d2722..b2b1d472 100644 --- a/Analysis/src/ConstraintSolver.cpp +++ b/Analysis/src/ConstraintSolver.cpp @@ -11,6 +11,7 @@ LUAU_FASTFLAGVARIABLE(DebugLuauLogSolver, false); LUAU_FASTFLAGVARIABLE(DebugLuauLogSolverToJson, false); +LUAU_FASTFLAG(LuauFixNameMaps) namespace Luau { @@ -19,9 +20,17 @@ namespace Luau { for (const auto& [k, v] : scope->bindings) { - auto d = toStringDetailed(v.typeId, opts); - opts.nameMap = d.nameMap; - printf("\t%s : %s\n", k.c_str(), d.name.c_str()); + if (FFlag::LuauFixNameMaps) + { + auto d = toString(v.typeId, opts); + printf("\t%s : %s\n", k.c_str(), d.c_str()); + } + else + { + auto d = toStringDetailed(v.typeId, opts); + opts.DEPRECATED_nameMap = d.DEPRECATED_nameMap; + printf("\t%s : %s\n", k.c_str(), d.name.c_str()); + } } for (NotNull child : scope->children) @@ -212,12 +221,22 @@ void dump(NotNull rootScope, ToStringOptions& opts) void dump(ConstraintSolver* cs, ToStringOptions& opts) { printf("constraints:\n"); - for (const Constraint* c : cs->unsolvedConstraints) + for (NotNull c : cs->unsolvedConstraints) { - printf("\t%s\n", toString(*c, opts).c_str()); + auto it = cs->blockedConstraints.find(c); + int blockCount = it == cs->blockedConstraints.end() ? 0 : int(it->second); + printf("\t%d\t%s\n", blockCount, toString(*c, opts).c_str()); - for (const Constraint* dep : c->dependencies) - printf("\t\t%s\n", toString(*dep, opts).c_str()); + for (NotNull dep : c->dependencies) + { + auto unsolvedIter = std::find(begin(cs->unsolvedConstraints), end(cs->unsolvedConstraints), dep); + if (unsolvedIter == cs->unsolvedConstraints.end()) + continue; + + auto it = cs->blockedConstraints.find(dep); + int blockCount = it == cs->blockedConstraints.end() ? 0 : int(it->second); + printf("\t%d\t\t%s\n", blockCount, toString(*dep, opts).c_str()); + } } } @@ -273,7 +292,7 @@ void ConstraintSolver::run() if (FFlag::DebugLuauLogSolverToJson) { - logger.prepareStepSnapshot(rootScope, c, unsolvedConstraints); + logger.prepareStepSnapshot(rootScope, c, unsolvedConstraints, force); } bool success = tryDispatch(c, force); @@ -282,6 +301,7 @@ void ConstraintSolver::run() if (success) { + unblock(c); unsolvedConstraints.erase(unsolvedConstraints.begin() + i); if (FFlag::DebugLuauLogSolverToJson) @@ -375,18 +395,12 @@ bool ConstraintSolver::tryDispatch(const SubtypeConstraint& c, NotNullscope); - unblock(c.subType); - unblock(c.superType); - return true; } bool ConstraintSolver::tryDispatch(const PackSubtypeConstraint& c, NotNull constraint, bool force) { unify(c.subPack, c.superPack, constraint->scope); - unblock(c.subPack); - unblock(c.superPack); - return true; } @@ -395,13 +409,12 @@ bool ConstraintSolver::tryDispatch(const GeneralizationConstraint& c, NotNullty.emplace(c.sourceType); - else - unify(c.generalizedType, c.sourceType, constraint->scope); - TypeId generalized = quantify(arena, c.sourceType, constraint->scope); - *asMutable(c.sourceType) = *generalized; + + if (isBlocked(c.generalizedType)) + asMutable(c.generalizedType)->ty.emplace(generalized); + else + unify(c.generalizedType, generalized, constraint->scope); unblock(c.generalizedType); unblock(c.sourceType); @@ -455,23 +468,44 @@ bool ConstraintSolver::tryDispatch(const BinaryConstraint& c, NotNull + * + * This constraint is the one that is meant to unblock A, so it doesn't + * make any sense to stop and wait for someone else to do it. + */ + if (leftType != resultType && rightType != resultType) + { + block(c.leftType, constraint); + block(c.rightType, constraint); + return false; + } } if (isNumber(leftType)) { unify(leftType, rightType, constraint->scope); - asMutable(c.resultType)->ty.emplace(leftType); + asMutable(resultType)->ty.emplace(leftType); return true; } - if (get(leftType) && !force) - return block(leftType, constraint); + if (!force) + { + if (get(leftType)) + return block(leftType, constraint); + } + + if (isBlocked(leftType)) + { + asMutable(resultType)->ty.emplace(getSingletonTypes().errorRecoveryType()); + // reportError(constraint->location, CannotInferBinaryOperation{c.op, std::nullopt, CannotInferBinaryOperation::Operation}); + return true; + } // TODO metatables, classes @@ -706,17 +740,23 @@ void ConstraintSolver::block_(BlockedConstraintId target, NotNull target, NotNull constraint) { + if (FFlag::DebugLuauLogSolver) + printf("block Constraint %s on\t%s\n", toString(*target).c_str(), toString(*constraint).c_str()); block_(target, constraint); } bool ConstraintSolver::block(TypeId target, NotNull constraint) { + if (FFlag::DebugLuauLogSolver) + printf("block TypeId %s on\t%s\n", toString(target).c_str(), toString(*constraint).c_str()); block_(target, constraint); return false; } bool ConstraintSolver::block(TypePackId target, NotNull constraint) { + if (FFlag::DebugLuauLogSolver) + printf("block TypeId %s on\t%s\n", toString(target).c_str(), toString(*constraint).c_str()); block_(target, constraint); return false; } @@ -731,6 +771,9 @@ void ConstraintSolver::unblock_(BlockedConstraintId progressed) for (NotNull unblockedConstraint : it->second) { auto& count = blockedConstraints[unblockedConstraint]; + if (FFlag::DebugLuauLogSolver) + printf("Unblocking count=%d\t%s\n", int(count), toString(*unblockedConstraint).c_str()); + // This assertion being hit indicates that `blocked` and // `blockedConstraints` desynchronized at some point. This is problematic // because we rely on this count being correct to skip over blocked @@ -757,6 +800,18 @@ void ConstraintSolver::unblock(TypePackId progressed) return unblock_(progressed); } +void ConstraintSolver::unblock(const std::vector& types) +{ + for (TypeId t : types) + unblock(t); +} + +void ConstraintSolver::unblock(const std::vector& packs) +{ + for (TypePackId t : packs) + unblock(t); +} + bool ConstraintSolver::isBlocked(TypeId ty) { return nullptr != get(follow(ty)) || nullptr != get(follow(ty)); @@ -774,7 +829,13 @@ void ConstraintSolver::unify(TypeId subType, TypeId superType, NotNull sc Unifier u{arena, Mode::Strict, scope, Location{}, Covariant, sharedState}; u.tryUnify(subType, superType); + + const auto [changedTypes, changedPacks] = u.log.getChanges(); + u.log.commit(); + + unblock(changedTypes); + unblock(changedPacks); } void ConstraintSolver::unify(TypePackId subPack, TypePackId superPack, NotNull scope) @@ -783,7 +844,13 @@ void ConstraintSolver::unify(TypePackId subPack, TypePackId superPack, NotNull scope) diff --git a/Analysis/src/ConstraintSolverLogger.cpp b/Analysis/src/ConstraintSolverLogger.cpp index adb9c54e..097ceeec 100644 --- a/Analysis/src/ConstraintSolverLogger.cpp +++ b/Analysis/src/ConstraintSolverLogger.cpp @@ -4,6 +4,8 @@ #include "Luau/JsonEmitter.h" +LUAU_FASTFLAG(LuauFixNameMaps); + namespace Luau { @@ -17,9 +19,14 @@ static void dumpScopeAndChildren(const Scope* scope, Json::JsonEmitter& emitter, for (const auto& [name, binding] : scope->bindings) { - ToStringResult result = toStringDetailed(binding.typeId, opts); - opts.nameMap = std::move(result.nameMap); - o.writePair(name.c_str(), result.name); + if (FFlag::LuauFixNameMaps) + o.writePair(name.c_str(), toString(binding.typeId, opts)); + else + { + ToStringResult result = toStringDetailed(binding.typeId, opts); + opts.DEPRECATED_nameMap = std::move(result.DEPRECATED_nameMap); + o.writePair(name.c_str(), result.name); + } } o.finish(); @@ -30,6 +37,7 @@ static void dumpScopeAndChildren(const Scope* scope, Json::JsonEmitter& emitter, Json::ArrayEmitter a = emitter.writeArray(); for (const Scope* child : scope->children) { + emitter.writeComma(); dumpScopeAndChildren(child, emitter, opts); } @@ -39,7 +47,8 @@ static void dumpScopeAndChildren(const Scope* scope, Json::JsonEmitter& emitter, static std::string dumpConstraintsToDot(std::vector>& constraints, ToStringOptions& opts) { - std::string result = "digraph Constraints {\\n"; + std::string result = "digraph Constraints {\n"; + result += "rankdir=LR\n"; std::unordered_set> contained; for (NotNull c : constraints) @@ -49,11 +58,19 @@ static std::string dumpConstraintsToDot(std::vector>& for (NotNull c : constraints) { + std::string shape; + if (get(*c)) + shape = "box"; + else if (get(*c)) + shape = "box3d"; + else + shape = "oval"; + std::string id = std::to_string(reinterpret_cast(c.get())); result += id; - result += " [label=\\\""; - result += toString(*c, opts).c_str(); - result += "\\\"];\\n"; + result += " [label=\""; + result += toString(*c, opts); + result += "\" shape=" + shape + "];\n"; for (NotNull dep : c->dependencies) { @@ -63,7 +80,7 @@ static std::string dumpConstraintsToDot(std::vector>& result += std::to_string(reinterpret_cast(dep.get())); result += " -> "; result += id; - result += ";\\n"; + result += ";\n"; } } @@ -102,7 +119,7 @@ void ConstraintSolverLogger::captureBoundarySnapshot(const Scope* rootScope, std } void ConstraintSolverLogger::prepareStepSnapshot( - const Scope* rootScope, NotNull current, std::vector>& unsolvedConstraints) + const Scope* rootScope, NotNull current, std::vector>& unsolvedConstraints, bool force) { Json::JsonEmitter emitter; Json::ObjectEmitter o = emitter.writeObject(); @@ -110,6 +127,7 @@ void ConstraintSolverLogger::prepareStepSnapshot( o.writePair("constraintGraph", dumpConstraintsToDot(unsolvedConstraints, opts)); o.writePair("currentId", std::to_string(reinterpret_cast(current.get()))); o.writePair("current", toString(*current, opts)); + o.writePair("force", force); emitter.writeComma(); Json::write(emitter, "rootScope"); emitter.writeRaw(":"); diff --git a/Analysis/src/Linter.cpp b/Analysis/src/Linter.cpp index 9f89efe9..426ff9d6 100644 --- a/Analysis/src/Linter.cpp +++ b/Analysis/src/Linter.cpp @@ -15,6 +15,7 @@ LUAU_FASTINTVARIABLE(LuauSuggestionDistance, 4) LUAU_FASTFLAGVARIABLE(LuauLintGlobalNeverReadBeforeWritten, false) LUAU_FASTFLAGVARIABLE(LuauLintComparisonPrecedence, false) +LUAU_FASTFLAGVARIABLE(LuauLintFixDeprecationMessage, false) namespace Luau { @@ -306,11 +307,22 @@ private: emitWarning(*context, LintWarning::Code_UnknownGlobal, gv->location, "Unknown global '%s'", gv->name.value); else if (g->deprecated) { - if (*g->deprecated) - emitWarning(*context, LintWarning::Code_DeprecatedGlobal, gv->location, "Global '%s' is deprecated, use '%s' instead", - gv->name.value, *g->deprecated); + if (FFlag::LuauLintFixDeprecationMessage) + { + if (const char* replacement = *g->deprecated; replacement && strlen(replacement)) + emitWarning(*context, LintWarning::Code_DeprecatedGlobal, gv->location, "Global '%s' is deprecated, use '%s' instead", + gv->name.value, replacement); + else + emitWarning(*context, LintWarning::Code_DeprecatedGlobal, gv->location, "Global '%s' is deprecated", gv->name.value); + } else - emitWarning(*context, LintWarning::Code_DeprecatedGlobal, gv->location, "Global '%s' is deprecated", gv->name.value); + { + if (*g->deprecated) + emitWarning(*context, LintWarning::Code_DeprecatedGlobal, gv->location, "Global '%s' is deprecated, use '%s' instead", + gv->name.value, *g->deprecated); + else + emitWarning(*context, LintWarning::Code_DeprecatedGlobal, gv->location, "Global '%s' is deprecated", gv->name.value); + } } } diff --git a/Analysis/src/ToDot.cpp b/Analysis/src/ToDot.cpp index 6b677bb8..0d989ca0 100644 --- a/Analysis/src/ToDot.cpp +++ b/Analysis/src/ToDot.cpp @@ -73,7 +73,7 @@ void StateDot::visitChild(TypeId ty, int parentIndex, const char* linkName) if (opts.duplicatePrimitives && canDuplicatePrimitive(ty)) { if (get(ty)) - formatAppend(result, "n%d [label=\"%s\"];\n", index, toStringDetailed(ty, {}).name.c_str()); + formatAppend(result, "n%d [label=\"%s\"];\n", index, toString(ty).c_str()); else if (get(ty)) formatAppend(result, "n%d [label=\"any\"];\n", index); } @@ -233,7 +233,7 @@ void StateDot::visitChildren(TypeId ty, int index) } else if (get(ty)) { - formatAppend(result, "PrimitiveTypeVar %s", toStringDetailed(ty, {}).name.c_str()); + formatAppend(result, "PrimitiveTypeVar %s", toString(ty).c_str()); finishNodeLabel(ty); finishNode(); } diff --git a/Analysis/src/ToString.cpp b/Analysis/src/ToString.cpp index 79d2e125..b4ef4384 100644 --- a/Analysis/src/ToString.cpp +++ b/Analysis/src/ToString.cpp @@ -13,6 +13,7 @@ LUAU_FASTFLAG(LuauLowerBoundsCalculation) LUAU_FASTFLAG(LuauUnknownAndNeverType) LUAU_FASTFLAGVARIABLE(LuauSpecialTypesAsterisked, false) +LUAU_FASTFLAGVARIABLE(LuauFixNameMaps, false) /* * Prefix generic typenames with gen- @@ -116,7 +117,7 @@ static std::pair> canUseTypeNameInScope(ScopePtr struct StringifierState { - const ToStringOptions& opts; + ToStringOptions& opts; ToStringResult& result; std::unordered_map cycleNames; @@ -127,18 +128,28 @@ struct StringifierState bool exhaustive; - StringifierState(const ToStringOptions& opts, ToStringResult& result, const std::optional& nameMap) + StringifierState(ToStringOptions& opts, ToStringResult& result, const std::optional& DEPRECATED_nameMap) : opts(opts) , result(result) , exhaustive(opts.exhaustive) { - if (nameMap) - result.nameMap = *nameMap; + if (!FFlag::LuauFixNameMaps && DEPRECATED_nameMap) + result.DEPRECATED_nameMap = *DEPRECATED_nameMap; - for (const auto& [_, v] : result.nameMap.typeVars) - usedNames.insert(v); - for (const auto& [_, v] : result.nameMap.typePacks) - usedNames.insert(v); + if (!FFlag::LuauFixNameMaps) + { + for (const auto& [_, v] : result.DEPRECATED_nameMap.typeVars) + usedNames.insert(v); + for (const auto& [_, v] : result.DEPRECATED_nameMap.typePacks) + usedNames.insert(v); + } + else + { + for (const auto& [_, v] : opts.nameMap.typeVars) + usedNames.insert(v); + for (const auto& [_, v] : opts.nameMap.typePacks) + usedNames.insert(v); + } } bool hasSeen(const void* tv) @@ -161,8 +172,8 @@ struct StringifierState std::string getName(TypeId ty) { - const size_t s = result.nameMap.typeVars.size(); - std::string& n = result.nameMap.typeVars[ty]; + const size_t s = FFlag::LuauFixNameMaps ? opts.nameMap.typeVars.size() : result.DEPRECATED_nameMap.typeVars.size(); + std::string& n = FFlag::LuauFixNameMaps ? opts.nameMap.typeVars[ty] : result.DEPRECATED_nameMap.typeVars[ty]; if (!n.empty()) return n; @@ -184,8 +195,8 @@ struct StringifierState std::string getName(TypePackId ty) { - const size_t s = result.nameMap.typePacks.size(); - std::string& n = result.nameMap.typePacks[ty]; + const size_t s = FFlag::LuauFixNameMaps ? opts.nameMap.typePacks.size() : result.DEPRECATED_nameMap.typePacks.size(); + std::string& n = FFlag::LuauFixNameMaps ? opts.nameMap.typePacks[ty] : result.DEPRECATED_nameMap.typePacks[ty]; if (!n.empty()) return n; @@ -378,7 +389,10 @@ struct TypeVarStringifier if (gtv.explicitName) { state.usedNames.insert(gtv.name); - state.result.nameMap.typeVars[ty] = gtv.name; + if (FFlag::LuauFixNameMaps) + state.opts.nameMap.typeVars[ty] = gtv.name; + else + state.result.DEPRECATED_nameMap.typeVars[ty] = gtv.name; state.emit(gtv.name); } else @@ -989,7 +1003,10 @@ struct TypePackStringifier if (pack.explicitName) { state.usedNames.insert(pack.name); - state.result.nameMap.typePacks[tp] = pack.name; + if (FFlag::LuauFixNameMaps) + state.opts.nameMap.typePacks[tp] = pack.name; + else + state.result.DEPRECATED_nameMap.typePacks[tp] = pack.name; state.emit(pack.name); } else @@ -1070,7 +1087,7 @@ static void assignCycleNames(const std::set& cycles, const std::set cycles; std::set cycleTPs; @@ -1183,7 +1202,7 @@ ToStringResult toStringDetailed(TypeId ty, const ToStringOptions& opts) return result; } -ToStringResult toStringDetailed(TypePackId tp, const ToStringOptions& opts) +ToStringResult toStringDetailed(TypePackId tp, ToStringOptions& opts) { /* * 1. Walk the TypeVar and track seen TypeIds. When you reencounter a TypeId, add it to a set of seen cycles. @@ -1192,7 +1211,9 @@ ToStringResult toStringDetailed(TypePackId tp, const ToStringOptions& opts) * 4. Print out the root of the type using the same algorithm as step 3. */ ToStringResult result; - StringifierState state{opts, result, opts.nameMap}; + StringifierState state = FFlag::LuauFixNameMaps + ? StringifierState{opts, result, opts.nameMap} + : StringifierState{opts, result, opts.DEPRECATED_nameMap}; std::set cycles; std::set cycleTPs; @@ -1258,30 +1279,32 @@ ToStringResult toStringDetailed(TypePackId tp, const ToStringOptions& opts) return result; } -std::string toString(TypeId ty, const ToStringOptions& opts) +std::string toString(TypeId ty, ToStringOptions& opts) { return toStringDetailed(ty, opts).name; } -std::string toString(TypePackId tp, const ToStringOptions& opts) +std::string toString(TypePackId tp, ToStringOptions& opts) { return toStringDetailed(tp, opts).name; } -std::string toString(const TypeVar& tv, const ToStringOptions& opts) +std::string toString(const TypeVar& tv, ToStringOptions& opts) { - return toString(const_cast(&tv), std::move(opts)); + return toString(const_cast(&tv), opts); } -std::string toString(const TypePackVar& tp, const ToStringOptions& opts) +std::string toString(const TypePackVar& tp, ToStringOptions& opts) { - return toString(const_cast(&tp), std::move(opts)); + return toString(const_cast(&tp), opts); } -std::string toStringNamedFunction(const std::string& funcName, const FunctionTypeVar& ftv, const ToStringOptions& opts) +std::string toStringNamedFunction(const std::string& funcName, const FunctionTypeVar& ftv, ToStringOptions& opts) { ToStringResult result; - StringifierState state(opts, result, opts.nameMap); + StringifierState state = FFlag::LuauFixNameMaps + ? StringifierState{opts, result, opts.nameMap} + : StringifierState{opts, result, opts.DEPRECATED_nameMap}; TypeVarStringifier tvs{state}; state.emit(funcName); @@ -1413,69 +1436,67 @@ std::string toString(const Constraint& constraint, ToStringOptions& opts) auto go = [&opts](auto&& c) { using T = std::decay_t; + // TODO: Inline and delete this function when clipping FFlag::LuauFixNameMaps + auto tos = [](auto&& a, ToStringOptions& opts) + { + if (FFlag::LuauFixNameMaps) + return toString(a, opts); + else + { + ToStringResult tsr = toStringDetailed(a, opts); + opts.DEPRECATED_nameMap = std::move(tsr.DEPRECATED_nameMap); + return tsr.name; + } + }; + if constexpr (std::is_same_v) { - ToStringResult subStr = toStringDetailed(c.subType, opts); - opts.nameMap = std::move(subStr.nameMap); - ToStringResult superStr = toStringDetailed(c.superType, opts); - opts.nameMap = std::move(superStr.nameMap); - return subStr.name + " <: " + superStr.name; + std::string subStr = tos(c.subType, opts); + std::string superStr = tos(c.superType, opts); + return subStr + " <: " + superStr; } else if constexpr (std::is_same_v) { - ToStringResult subStr = toStringDetailed(c.subPack, opts); - opts.nameMap = std::move(subStr.nameMap); - ToStringResult superStr = toStringDetailed(c.superPack, opts); - opts.nameMap = std::move(superStr.nameMap); - return subStr.name + " <: " + superStr.name; + std::string subStr = tos(c.subPack, opts); + std::string superStr = tos(c.superPack, opts); + return subStr + " <: " + superStr; } else if constexpr (std::is_same_v) { - ToStringResult subStr = toStringDetailed(c.generalizedType, opts); - opts.nameMap = std::move(subStr.nameMap); - ToStringResult superStr = toStringDetailed(c.sourceType, opts); - opts.nameMap = std::move(superStr.nameMap); - return subStr.name + " ~ gen " + superStr.name; + std::string subStr = tos(c.generalizedType, opts); + std::string superStr = tos(c.sourceType, opts); + return subStr + " ~ gen " + superStr; } else if constexpr (std::is_same_v) { - ToStringResult subStr = toStringDetailed(c.subType, opts); - opts.nameMap = std::move(subStr.nameMap); - ToStringResult superStr = toStringDetailed(c.superType, opts); - opts.nameMap = std::move(superStr.nameMap); - return subStr.name + " ~ inst " + superStr.name; + std::string subStr = tos(c.subType, opts); + std::string superStr = tos(c.superType, opts); + return subStr + " ~ inst " + superStr; } else if constexpr (std::is_same_v) { - ToStringResult resultStr = toStringDetailed(c.resultType, opts); - opts.nameMap = std::move(resultStr.nameMap); - ToStringResult operandStr = toStringDetailed(c.operandType, opts); - opts.nameMap = std::move(operandStr.nameMap); + std::string resultStr = tos(c.resultType, opts); + std::string operandStr = tos(c.operandType, opts); - return resultStr.name + " ~ Unary<" + toString(c.op) + ", " + operandStr.name + ">"; + return resultStr + " ~ Unary<" + toString(c.op) + ", " + operandStr + ">"; } else if constexpr (std::is_same_v) { - ToStringResult resultStr = toStringDetailed(c.resultType); - opts.nameMap = std::move(resultStr.nameMap); - ToStringResult leftStr = toStringDetailed(c.leftType); - opts.nameMap = std::move(leftStr.nameMap); - ToStringResult rightStr = toStringDetailed(c.rightType); - opts.nameMap = std::move(rightStr.nameMap); + std::string resultStr = tos(c.resultType, opts); + std::string leftStr = tos(c.leftType, opts); + std::string rightStr = tos(c.rightType, opts); - return resultStr.name + " ~ Binary<" + toString(c.op) + ", " + leftStr.name + ", " + rightStr.name + ">"; + return resultStr + " ~ Binary<" + toString(c.op) + ", " + leftStr + ", " + rightStr + ">"; } else if constexpr (std::is_same_v) { - ToStringResult namedStr = toStringDetailed(c.namedType, opts); - opts.nameMap = std::move(namedStr.nameMap); - return "@name(" + namedStr.name + ") = " + c.name; + std::string namedStr = tos(c.namedType, opts); + return "@name(" + namedStr + ") = " + c.name; } else if constexpr (std::is_same_v) { - ToStringResult targetStr = toStringDetailed(c.target, opts); - opts.nameMap = std::move(targetStr.nameMap); - return "expand " + targetStr.name; + std::string targetStr = tos(c.target, opts); + return "expand " + targetStr; } else static_assert(always_false_v, "Non-exhaustive constraint switch"); diff --git a/Analysis/src/TxnLog.cpp b/Analysis/src/TxnLog.cpp index b3f60d30..74d77307 100644 --- a/Analysis/src/TxnLog.cpp +++ b/Analysis/src/TxnLog.cpp @@ -344,4 +344,16 @@ TypePackId TxnLog::follow(TypePackId tp) const }); } +std::pair, std::vector> TxnLog::getChanges() const +{ + std::pair, std::vector> result; + + for (const auto& [typeId, _newState] : typeVarChanges) + result.first.push_back(typeId); + for (const auto& [typePackId, _newState] : typePackChanges) + result.second.push_back(typePackId); + + return result; +} + } // namespace Luau diff --git a/Analysis/src/TypeVar.cpp b/Analysis/src/TypeVar.cpp index 9020b1ac..8974f8c7 100644 --- a/Analysis/src/TypeVar.cpp +++ b/Analysis/src/TypeVar.cpp @@ -27,6 +27,7 @@ LUAU_FASTFLAG(LuauUnknownAndNeverType) LUAU_FASTFLAGVARIABLE(LuauDeduceGmatchReturnTypes, false) LUAU_FASTFLAGVARIABLE(LuauMaybeGenericIntersectionTypes, false) LUAU_FASTFLAGVARIABLE(LuauDeduceFindMatchReturnTypes, false) +LUAU_FASTFLAGVARIABLE(LuauStringFormatArgumentErrorFix, false) namespace Luau { @@ -1139,11 +1140,21 @@ std::optional> magicFunctionFormat( } // if we know the argument count or if we have too many arguments for sure, we can issue an error - size_t actualParamSize = params.size() - paramOffset; + if (FFlag::LuauStringFormatArgumentErrorFix) + { + size_t numActualParams = params.size(); + size_t numExpectedParams = expected.size() + 1; // + 1 for the format string - if (expected.size() != actualParamSize && (!tail || expected.size() < actualParamSize)) - typechecker.reportError(TypeError{expr.location, CountMismatch{expected.size(), actualParamSize}}); + if (numExpectedParams != numActualParams && (!tail || numExpectedParams < numActualParams)) + typechecker.reportError(TypeError{expr.location, CountMismatch{numExpectedParams, numActualParams}}); + } + else + { + size_t actualParamSize = params.size() - paramOffset; + if (expected.size() != actualParamSize && (!tail || expected.size() < actualParamSize)) + typechecker.reportError(TypeError{expr.location, CountMismatch{expected.size(), actualParamSize}}); + } return WithPredicate{arena.addTypePack({typechecker.stringType})}; } diff --git a/Ast/include/Luau/Ast.h b/Ast/include/Luau/Ast.h index 5c040007..612283fb 100644 --- a/Ast/include/Luau/Ast.h +++ b/Ast/include/Luau/Ast.h @@ -598,9 +598,9 @@ public: LUAU_RTTI(AstExprFunction) AstExprFunction(const Location& location, const AstArray& generics, const AstArray& genericPacks, - AstLocal* self, const AstArray& args, std::optional vararg, AstStatBlock* body, size_t functionDepth, - const AstName& debugname, std::optional returnAnnotation = {}, AstTypePack* varargAnnotation = nullptr, bool hasEnd = false, - std::optional argLocation = std::nullopt); + AstLocal* self, const AstArray& args, bool vararg, const Location& varargLocation, AstStatBlock* body, size_t functionDepth, + const AstName& debugname, const std::optional& returnAnnotation = {}, AstTypePack* varargAnnotation = nullptr, + bool hasEnd = false, const std::optional& argLocation = std::nullopt); void visit(AstVisitor* visitor) override; diff --git a/Ast/include/Luau/Parser.h b/Ast/include/Luau/Parser.h index 6f56f90d..956fcf64 100644 --- a/Ast/include/Luau/Parser.h +++ b/Ast/include/Luau/Parser.h @@ -11,6 +11,7 @@ #include #include +#include namespace Luau { @@ -109,8 +110,10 @@ private: // for namelist in explist do block end | AstStat* parseFor(); - // function funcname funcbody | // funcname ::= Name {`.' Name} [`:' Name] + AstExpr* parseFunctionName(Location start, bool& hasself, AstName& debugname); + + // function funcname funcbody AstStat* parseFunctionStat(); // local function Name funcbody | @@ -135,8 +138,10 @@ private: // var [`+=' | `-=' | `*=' | `/=' | `%=' | `^=' | `..='] exp AstStat* parseCompoundAssignment(AstExpr* initial, AstExprBinary::Op op); - // funcbody ::= `(' [parlist] `)' block end - // parlist ::= namelist [`,' `...'] | `...' + std::pair> prepareFunctionArguments(const Location& start, bool hasself, const TempVector& args); + + // funcbodyhead ::= `(' [namelist [`,' `...'] | `...'] `)' [`:` TypeAnnotation] + // funcbody ::= funcbodyhead block end std::pair parseFunctionBody( bool hasself, const Lexeme& matchFunction, const AstName& debugname, const Name* localName); @@ -148,7 +153,7 @@ private: // bindinglist ::= (binding | `...') {`,' bindinglist} // Returns the location of the vararg ..., or std::nullopt if the function is not vararg. - std::pair, AstTypePack*> parseBindingList(TempVector& result, bool allowDot3 = false); + std::tuple parseBindingList(TempVector& result, bool allowDot3 = false); AstType* parseOptionalTypeAnnotation(); diff --git a/Ast/src/Ast.cpp b/Ast/src/Ast.cpp index 5ede3893..8291a5b1 100644 --- a/Ast/src/Ast.cpp +++ b/Ast/src/Ast.cpp @@ -160,17 +160,17 @@ void AstExprIndexExpr::visit(AstVisitor* visitor) } AstExprFunction::AstExprFunction(const Location& location, const AstArray& generics, const AstArray& genericPacks, - AstLocal* self, const AstArray& args, std::optional vararg, AstStatBlock* body, size_t functionDepth, - const AstName& debugname, std::optional returnAnnotation, AstTypePack* varargAnnotation, bool hasEnd, - std::optional argLocation) + AstLocal* self, const AstArray& args, bool vararg, const Location& varargLocation, AstStatBlock* body, size_t functionDepth, + const AstName& debugname, const std::optional& returnAnnotation, AstTypePack* varargAnnotation, bool hasEnd, + const std::optional& argLocation) : AstExpr(ClassIndex(), location) , generics(generics) , genericPacks(genericPacks) , self(self) , args(args) , returnAnnotation(returnAnnotation) - , vararg(vararg.has_value()) - , varargLocation(vararg.value_or(Location())) + , vararg(vararg) + , varargLocation(varargLocation) , varargAnnotation(varargAnnotation) , body(body) , functionDepth(functionDepth) diff --git a/Ast/src/Parser.cpp b/Ast/src/Parser.cpp index cc624c10..b6de27da 100644 --- a/Ast/src/Parser.cpp +++ b/Ast/src/Parser.cpp @@ -605,16 +605,11 @@ AstStat* Parser::parseFor() } } -// function funcname funcbody | // funcname ::= Name {`.' Name} [`:' Name] -AstStat* Parser::parseFunctionStat() +AstExpr* Parser::parseFunctionName(Location start, bool& hasself, AstName& debugname) { - Location start = lexer.current().location; - - Lexeme matchFunction = lexer.current(); - nextLexeme(); - - AstName debugname = (lexer.current().type == Lexeme::Name) ? AstName(lexer.current().name) : AstName(); + if (lexer.current().type == Lexeme::Name) + debugname = AstName(lexer.current().name); // parse funcname into a chain of indexing operators AstExpr* expr = parseNameExpr("function name"); @@ -640,8 +635,6 @@ AstStat* Parser::parseFunctionStat() recursionCounter = recursionCounterOld; // finish with : - bool hasself = false; - if (lexer.current().type == ':') { Position opPosition = lexer.current().location.begin; @@ -657,6 +650,21 @@ AstStat* Parser::parseFunctionStat() hasself = true; } + return expr; +} + +// function funcname funcbody +AstStat* Parser::parseFunctionStat() +{ + Location start = lexer.current().location; + + Lexeme matchFunction = lexer.current(); + nextLexeme(); + + bool hasself = false; + AstName debugname; + AstExpr* expr = parseFunctionName(start, hasself, debugname); + matchRecoveryStopOnToken[Lexeme::ReservedEnd]++; AstExprFunction* body = parseFunctionBody(hasself, matchFunction, debugname, nullptr).first; @@ -785,10 +793,11 @@ AstDeclaredClassProp Parser::parseDeclaredClassMethod() TempVector args(scratchBinding); - std::optional vararg = std::nullopt; + bool vararg = false; + Location varargLocation; AstTypePack* varargAnnotation = nullptr; if (lexer.current().type != ')') - std::tie(vararg, varargAnnotation) = parseBindingList(args, /* allowDot3 */ true); + std::tie(vararg, varargLocation, varargAnnotation) = parseBindingList(args, /* allowDot3 */ true); expectMatchAndConsume(')', matchParen); @@ -842,11 +851,12 @@ AstStat* Parser::parseDeclaration(const Location& start) TempVector args(scratchBinding); - std::optional vararg; + bool vararg = false; + Location varargLocation; AstTypePack* varargAnnotation = nullptr; if (lexer.current().type != ')') - std::tie(vararg, varargAnnotation) = parseBindingList(args, /* allowDot3= */ true); + std::tie(vararg, varargLocation, varargAnnotation) = parseBindingList(args, /* allowDot3= */ true); expectMatchAndConsume(')', matchParen); @@ -969,6 +979,21 @@ AstStat* Parser::parseCompoundAssignment(AstExpr* initial, AstExprBinary::Op op) return allocator.alloc(Location(initial->location, value->location), op, initial, value); } +std::pair> Parser::prepareFunctionArguments(const Location& start, bool hasself, const TempVector& args) +{ + AstLocal* self = nullptr; + + if (hasself) + self = pushLocal(Binding(Name(nameSelf, start), nullptr)); + + TempVector vars(scratchLocal); + + for (size_t i = 0; i < args.size(); ++i) + vars.push_back(pushLocal(args[i])); + + return {self, copy(vars)}; +} + // funcbody ::= `(' [parlist] `)' [`:' ReturnType] block end // parlist ::= bindinglist [`,' `...'] | `...' std::pair Parser::parseFunctionBody( @@ -983,15 +1008,18 @@ std::pair Parser::parseFunctionBody( TempVector args(scratchBinding); - std::optional vararg; + bool vararg = false; + Location varargLocation; AstTypePack* varargAnnotation = nullptr; if (lexer.current().type != ')') - std::tie(vararg, varargAnnotation) = parseBindingList(args, /* allowDot3= */ true); + std::tie(vararg, varargLocation, varargAnnotation) = parseBindingList(args, /* allowDot3= */ true); + + std::optional argLocation; + + if (matchParen.type == Lexeme::Type('(') && lexer.current().type == Lexeme::Type(')')) + argLocation = Location(matchParen.position, lexer.current().location.end); - std::optional argLocation = matchParen.type == Lexeme::Type('(') && lexer.current().type == Lexeme::Type(')') - ? std::make_optional(Location(matchParen.position, lexer.current().location.end)) - : std::nullopt; expectMatchAndConsume(')', matchParen, true); std::optional typelist = parseOptionalReturnTypeAnnotation(); @@ -1004,19 +1032,11 @@ std::pair Parser::parseFunctionBody( unsigned int localsBegin = saveLocals(); Function fun; - fun.vararg = vararg.has_value(); + fun.vararg = vararg; - functionStack.push_back(fun); + functionStack.emplace_back(fun); - AstLocal* self = nullptr; - - if (hasself) - self = pushLocal(Binding(Name(nameSelf, start), nullptr)); - - TempVector vars(scratchLocal); - - for (size_t i = 0; i < args.size(); ++i) - vars.push_back(pushLocal(args[i])); + auto [self, vars] = prepareFunctionArguments(start, hasself, args); AstStatBlock* body = parseBlock(); @@ -1028,8 +1048,8 @@ std::pair Parser::parseFunctionBody( bool hasEnd = expectMatchEndAndConsume(Lexeme::ReservedEnd, matchFunction); - return {allocator.alloc(Location(start, end), generics, genericPacks, self, copy(vars), vararg, body, functionStack.size(), - debugname, typelist, varargAnnotation, hasEnd, argLocation), + return {allocator.alloc(Location(start, end), generics, genericPacks, self, vars, vararg, varargLocation, body, + functionStack.size(), debugname, typelist, varargAnnotation, hasEnd, argLocation), funLocal}; } @@ -1060,7 +1080,7 @@ Parser::Binding Parser::parseBinding() } // bindinglist ::= (binding | `...') [`,' bindinglist] -std::pair, AstTypePack*> Parser::parseBindingList(TempVector& result, bool allowDot3) +std::tuple Parser::parseBindingList(TempVector& result, bool allowDot3) { while (true) { @@ -1076,7 +1096,7 @@ std::pair, AstTypePack*> Parser::parseBindingList(TempVe tailAnnotation = parseVariadicArgumentAnnotation(); } - return {varargLocation, tailAnnotation}; + return {true, varargLocation, tailAnnotation}; } result.push_back(parseBinding()); @@ -1086,7 +1106,7 @@ std::pair, AstTypePack*> Parser::parseBindingList(TempVe nextLexeme(); } - return {std::nullopt, nullptr}; + return {false, Location(), nullptr}; } AstType* Parser::parseOptionalTypeAnnotation() diff --git a/Common/include/Luau/ExperimentalFlags.h b/Common/include/Luau/ExperimentalFlags.h index 71e76ffe..809c78da 100644 --- a/Common/include/Luau/ExperimentalFlags.h +++ b/Common/include/Luau/ExperimentalFlags.h @@ -12,6 +12,7 @@ inline bool isFlagExperimental(const char* flag) // or critical bugs that are found after the code has been submitted. static const char* kList[] = { "LuauLowerBoundsCalculation", + "LuauInterpolatedStringBaseSupport", nullptr, // makes sure we always have at least one entry }; diff --git a/Compiler/src/Compiler.cpp b/Compiler/src/Compiler.cpp index a9394d74..4429e4cc 100644 --- a/Compiler/src/Compiler.cpp +++ b/Compiler/src/Compiler.cpp @@ -3043,6 +3043,18 @@ struct Compiler uint8_t valueReg = kInvalidReg; }; + // This function analyzes assignments and marks assignment conflicts: cases when a variable is assigned on lhs + // but subsequently used on the rhs, assuming assignments are performed in order. Note that it's also possible + // for a variable to conflict on the lhs, if it's used in an lvalue expression after it's assigned. + // When conflicts are found, Assignment::conflictReg is allocated and that's where assignment is performed instead, + // until the final fixup in compileStatAssign. Assignment::valueReg is allocated by compileStatAssign as well. + // + // Per Lua manual, section 3.3.3 (Assignments), the proper assignment order is only guaranteed to hold for syntactic access: + // + // Note that this guarantee covers only accesses syntactically inside the assignment statement. If a function or a metamethod called + // during the assignment changes the value of a variable, Lua gives no guarantees about the order of that access. + // + // As such, we currently don't check if an assigned local is captured, which may mean it gets reassigned during a function call. void resolveAssignConflicts(AstStat* stat, std::vector& vars, const AstArray& values) { struct Visitor : AstVisitor @@ -3945,7 +3957,8 @@ void compileOrThrow(BytecodeBuilder& bytecode, const ParseResult& parseResult, c compiler.compileFunction(expr); AstExprFunction main(root->location, /*generics= */ AstArray(), /*genericPacks= */ AstArray(), - /* self= */ nullptr, AstArray(), /* vararg= */ Luau::Location(), root, /* functionDepth= */ 0, /* debugname= */ AstName()); + /* self= */ nullptr, AstArray(), /* vararg= */ true, /* varargLocation= */ Luau::Location(), root, /* functionDepth= */ 0, + /* debugname= */ AstName()); uint32_t mainid = compiler.compileFunction(&main); const Compiler::Function* mainf = compiler.functions.find(&main); diff --git a/Makefile b/Makefile index 4333a973..46dfb868 100644 --- a/Makefile +++ b/Makefile @@ -93,7 +93,7 @@ endif ifeq ($(config),fuzz) CXX=clang++ # our fuzzing infra relies on llvm fuzzer - CXXFLAGS+=-fsanitize=address,fuzzer -Ibuild/libprotobuf-mutator -Ibuild/libprotobuf-mutator/external.protobuf/include -O2 + CXXFLAGS+=-fsanitize=address,fuzzer -Ibuild/libprotobuf-mutator -O2 LDFLAGS+=-fsanitize=address,fuzzer endif @@ -115,7 +115,7 @@ $(FUZZ_OBJECTS): CXXFLAGS+=-std=c++17 -ICommon/include -IAst/include -ICompiler/ $(TESTS_TARGET): LDFLAGS+=-lpthread $(REPL_CLI_TARGET): LDFLAGS+=-lpthread -fuzz-proto fuzz-prototest: LDFLAGS+=build/libprotobuf-mutator/src/libfuzzer/libprotobuf-mutator-libfuzzer.a build/libprotobuf-mutator/src/libprotobuf-mutator.a build/libprotobuf-mutator/external.protobuf/lib/libprotobuf.a +fuzz-proto fuzz-prototest: LDFLAGS+=build/libprotobuf-mutator/src/libfuzzer/libprotobuf-mutator-libfuzzer.a build/libprotobuf-mutator/src/libprotobuf-mutator.a -lprotobuf # pseudo targets .PHONY: all test clean coverage format luau-size aliases @@ -195,7 +195,7 @@ $(BUILD)/%.c.o: %.c # protobuf fuzzer setup fuzz/luau.pb.cpp: fuzz/luau.proto build/libprotobuf-mutator - cd fuzz && ../build/libprotobuf-mutator/external.protobuf/bin/protoc luau.proto --cpp_out=. + cd fuzz && protoc luau.proto --cpp_out=. mv fuzz/luau.pb.cc fuzz/luau.pb.cpp $(BUILD)/fuzz/proto.cpp.o: fuzz/luau.pb.cpp @@ -203,7 +203,7 @@ $(BUILD)/fuzz/protoprint.cpp.o: fuzz/luau.pb.cpp build/libprotobuf-mutator: git clone https://github.com/google/libprotobuf-mutator build/libprotobuf-mutator - CXX= cmake -S build/libprotobuf-mutator -B build/libprotobuf-mutator -D CMAKE_BUILD_TYPE=Release -D LIB_PROTO_MUTATOR_DOWNLOAD_PROTOBUF=ON -D LIB_PROTO_MUTATOR_TESTING=OFF + CXX= cmake -S build/libprotobuf-mutator -B build/libprotobuf-mutator -D CMAKE_BUILD_TYPE=Release -D LIB_PROTO_MUTATOR_TESTING=OFF make -C build/libprotobuf-mutator -j8 # picks up include dependencies for all object files diff --git a/VM/src/lapi.cpp b/VM/src/lapi.cpp index af97dc4a..4396e5d1 100644 --- a/VM/src/lapi.cpp +++ b/VM/src/lapi.cpp @@ -24,16 +24,18 @@ * The caller is expected to handle stack reservation (by using less than LUA_MINSTACK slots or by calling lua_checkstack). * To ensure this is handled correctly, use api_incr_top(L) when pushing values to the stack. * - * Functions that push any collectable objects to the stack *should* call luaC_checkthreadsleep. Failure to do this can result - * in stack references that point to dead objects since sleeping threads don't get rescanned. + * Functions that push any collectable objects to the stack *should* call luaC_threadbarrier. Failure to do this can result + * in stack references that point to dead objects since black threads don't get rescanned. * - * Functions that push newly created objects to the stack *should* call luaC_checkGC in addition to luaC_checkthreadsleep. + * Functions that push newly created objects to the stack *should* call luaC_checkGC in addition to luaC_threadbarrier. * Failure to do this can result in OOM since GC may never run. * - * Note that luaC_checkGC may scan the thread and put it back to sleep; functions that call both before pushing objects must - * therefore call luaC_checkGC before luaC_checkthreadsleep to guarantee the object is pushed to an awake thread. + * Note that luaC_checkGC may mark the thread and paint it black; functions that call both before pushing objects must + * therefore call luaC_checkGC before luaC_threadbarrier to guarantee the object is pushed to a gray thread. */ +LUAU_FASTFLAG(LuauSimplerUpval) + const char* lua_ident = "$Lua: Lua 5.1.4 Copyright (C) 1994-2008 Lua.org, PUC-Rio $\n" "$Authors: R. Ierusalimschy, L. H. de Figueiredo & W. Celes $\n" "$URL: www.lua.org $\n"; @@ -152,7 +154,7 @@ void lua_xmove(lua_State* from, lua_State* to, int n) api_checknelems(from, n); api_check(from, from->global == to->global); api_check(from, to->ci->top - to->top >= n); - luaC_checkthreadsleep(to); + luaC_threadbarrier(to); StkId ttop = to->top; StkId ftop = from->top - n; @@ -168,7 +170,7 @@ void lua_xmove(lua_State* from, lua_State* to, int n) void lua_xpush(lua_State* from, lua_State* to, int idx) { api_check(from, from->global == to->global); - luaC_checkthreadsleep(to); + luaC_threadbarrier(to); setobj2s(to, to->top, index2addr(from, idx)); api_incr_top(to); return; @@ -177,7 +179,7 @@ void lua_xpush(lua_State* from, lua_State* to, int idx) lua_State* lua_newthread(lua_State* L) { luaC_checkGC(L); - luaC_checkthreadsleep(L); + luaC_threadbarrier(L); lua_State* L1 = luaE_newthread(L); setthvalue(L, L->top, L1); api_incr_top(L); @@ -236,7 +238,7 @@ void lua_remove(lua_State* L, int idx) void lua_insert(lua_State* L, int idx) { - luaC_checkthreadsleep(L); + luaC_threadbarrier(L); StkId p = index2addr(L, idx); api_checkvalidindex(L, p); for (StkId q = L->top; q > p; q--) @@ -248,7 +250,7 @@ void lua_insert(lua_State* L, int idx) void lua_replace(lua_State* L, int idx) { api_checknelems(L, 1); - luaC_checkthreadsleep(L); + luaC_threadbarrier(L); StkId o = index2addr(L, idx); api_checkvalidindex(L, o); if (idx == LUA_ENVIRONINDEX) @@ -276,7 +278,7 @@ void lua_replace(lua_State* L, int idx) void lua_pushvalue(lua_State* L, int idx) { - luaC_checkthreadsleep(L); + luaC_threadbarrier(L); StkId o = index2addr(L, idx); setobj2s(L, L->top, o); api_incr_top(L); @@ -427,7 +429,7 @@ const char* lua_tolstring(lua_State* L, int idx, size_t* len) StkId o = index2addr(L, idx); if (!ttisstring(o)) { - luaC_checkthreadsleep(L); + luaC_threadbarrier(L); if (!luaV_tostring(L, o)) { // conversion failed? if (len != NULL) @@ -607,7 +609,7 @@ void lua_pushvector(lua_State* L, float x, float y, float z) void lua_pushlstring(lua_State* L, const char* s, size_t len) { luaC_checkGC(L); - luaC_checkthreadsleep(L); + luaC_threadbarrier(L); setsvalue2s(L, L->top, luaS_newlstr(L, s, len)); api_incr_top(L); return; @@ -624,7 +626,7 @@ void lua_pushstring(lua_State* L, const char* s) const char* lua_pushvfstring(lua_State* L, const char* fmt, va_list argp) { luaC_checkGC(L); - luaC_checkthreadsleep(L); + luaC_threadbarrier(L); const char* ret = luaO_pushvfstring(L, fmt, argp); return ret; } @@ -632,7 +634,7 @@ const char* lua_pushvfstring(lua_State* L, const char* fmt, va_list argp) const char* lua_pushfstringL(lua_State* L, const char* fmt, ...) { luaC_checkGC(L); - luaC_checkthreadsleep(L); + luaC_threadbarrier(L); va_list argp; va_start(argp, fmt); const char* ret = luaO_pushvfstring(L, fmt, argp); @@ -643,7 +645,7 @@ const char* lua_pushfstringL(lua_State* L, const char* fmt, ...) void lua_pushcclosurek(lua_State* L, lua_CFunction fn, const char* debugname, int nup, lua_Continuation cont) { luaC_checkGC(L); - luaC_checkthreadsleep(L); + luaC_threadbarrier(L); api_checknelems(L, nup); Closure* cl = luaF_newCclosure(L, nup, getcurrenv(L)); cl->c.f = fn; @@ -674,7 +676,7 @@ void lua_pushlightuserdata(lua_State* L, void* p) int lua_pushthread(lua_State* L) { - luaC_checkthreadsleep(L); + luaC_threadbarrier(L); setthvalue(L, L->top, L); api_incr_top(L); return L->global->mainthread == L; @@ -686,7 +688,7 @@ int lua_pushthread(lua_State* L) int lua_gettable(lua_State* L, int idx) { - luaC_checkthreadsleep(L); + luaC_threadbarrier(L); StkId t = index2addr(L, idx); api_checkvalidindex(L, t); luaV_gettable(L, t, L->top - 1, L->top - 1); @@ -695,7 +697,7 @@ int lua_gettable(lua_State* L, int idx) int lua_getfield(lua_State* L, int idx, const char* k) { - luaC_checkthreadsleep(L); + luaC_threadbarrier(L); StkId t = index2addr(L, idx); api_checkvalidindex(L, t); TValue key; @@ -707,7 +709,7 @@ int lua_getfield(lua_State* L, int idx, const char* k) int lua_rawgetfield(lua_State* L, int idx, const char* k) { - luaC_checkthreadsleep(L); + luaC_threadbarrier(L); StkId t = index2addr(L, idx); api_check(L, ttistable(t)); TValue key; @@ -719,7 +721,7 @@ int lua_rawgetfield(lua_State* L, int idx, const char* k) int lua_rawget(lua_State* L, int idx) { - luaC_checkthreadsleep(L); + luaC_threadbarrier(L); StkId t = index2addr(L, idx); api_check(L, ttistable(t)); setobj2s(L, L->top - 1, luaH_get(hvalue(t), L->top - 1)); @@ -728,7 +730,7 @@ int lua_rawget(lua_State* L, int idx) int lua_rawgeti(lua_State* L, int idx, int n) { - luaC_checkthreadsleep(L); + luaC_threadbarrier(L); StkId t = index2addr(L, idx); api_check(L, ttistable(t)); setobj2s(L, L->top, luaH_getnum(hvalue(t), n)); @@ -739,7 +741,7 @@ int lua_rawgeti(lua_State* L, int idx, int n) void lua_createtable(lua_State* L, int narray, int nrec) { luaC_checkGC(L); - luaC_checkthreadsleep(L); + luaC_threadbarrier(L); sethvalue(L, L->top, luaH_new(L, narray, nrec)); api_incr_top(L); return; @@ -775,7 +777,7 @@ void lua_setsafeenv(lua_State* L, int objindex, int enabled) int lua_getmetatable(lua_State* L, int objindex) { - luaC_checkthreadsleep(L); + luaC_threadbarrier(L); Table* mt = NULL; const TValue* obj = index2addr(L, objindex); switch (ttype(obj)) @@ -800,7 +802,7 @@ int lua_getmetatable(lua_State* L, int objindex) void lua_getfenv(lua_State* L, int idx) { - luaC_checkthreadsleep(L); + luaC_threadbarrier(L); StkId o = index2addr(L, idx); api_checkvalidindex(L, o); switch (ttype(o)) @@ -1161,7 +1163,7 @@ l_noret lua_error(lua_State* L) int lua_next(lua_State* L, int idx) { - luaC_checkthreadsleep(L); + luaC_threadbarrier(L); StkId t = index2addr(L, idx); api_check(L, ttistable(t)); int more = luaH_next(L, hvalue(t), L->top - 1); @@ -1180,13 +1182,13 @@ void lua_concat(lua_State* L, int n) if (n >= 2) { luaC_checkGC(L); - luaC_checkthreadsleep(L); + luaC_threadbarrier(L); luaV_concat(L, n, cast_int(L->top - L->base) - 1); L->top -= (n - 1); } else if (n == 0) { // push empty string - luaC_checkthreadsleep(L); + luaC_threadbarrier(L); setsvalue2s(L, L->top, luaS_newlstr(L, "", 0)); api_incr_top(L); } @@ -1198,7 +1200,7 @@ void* lua_newuserdatatagged(lua_State* L, size_t sz, int tag) { api_check(L, unsigned(tag) < LUA_UTAG_LIMIT || tag == UTAG_PROXY); luaC_checkGC(L); - luaC_checkthreadsleep(L); + luaC_threadbarrier(L); Udata* u = luaU_newudata(L, sz, tag); setuvalue(L, L->top, u); api_incr_top(L); @@ -1208,7 +1210,7 @@ void* lua_newuserdatatagged(lua_State* L, size_t sz, int tag) void* lua_newuserdatadtor(lua_State* L, size_t sz, void (*dtor)(void*)) { luaC_checkGC(L); - luaC_checkthreadsleep(L); + luaC_threadbarrier(L); // make sure sz + sizeof(dtor) doesn't overflow; luaU_newdata will reject SIZE_MAX correctly size_t as = sz < SIZE_MAX - sizeof(dtor) ? sz + sizeof(dtor) : SIZE_MAX; Udata* u = luaU_newudata(L, as, UTAG_IDTOR); @@ -1244,7 +1246,7 @@ static const char* aux_upvalue(StkId fi, int n, TValue** val) const char* lua_getupvalue(lua_State* L, int funcindex, int n) { - luaC_checkthreadsleep(L); + luaC_threadbarrier(L); TValue* val; const char* name = aux_upvalue(index2addr(L, funcindex), n, &val); if (name) @@ -1266,7 +1268,8 @@ const char* lua_setupvalue(lua_State* L, int funcindex, int n) L->top--; setobj(L, val, L->top); luaC_barrier(L, clvalue(fi), L->top); - luaC_upvalbarrier(L, cast_to(UpVal*, NULL), val); + if (!FFlag::LuauSimplerUpval) + luaC_upvalbarrier(L, cast_to(UpVal*, NULL), val); } return name; } @@ -1336,7 +1339,7 @@ void lua_setuserdatadtor(lua_State* L, int tag, void (*dtor)(lua_State*, void*)) void lua_clonefunction(lua_State* L, int idx) { luaC_checkGC(L); - luaC_checkthreadsleep(L); + luaC_threadbarrier(L); StkId p = index2addr(L, idx); api_check(L, isLfunction(p)); Closure* cl = clvalue(p); diff --git a/VM/src/ldebug.cpp b/VM/src/ldebug.cpp index c44ccbed..fee9aaa9 100644 --- a/VM/src/ldebug.cpp +++ b/VM/src/ldebug.cpp @@ -12,8 +12,6 @@ #include #include -LUAU_FASTFLAGVARIABLE(LuauDebuggerBreakpointHitOnNextBestLine, false); - static const char* getfuncname(Closure* f); static int currentpc(lua_State* L, CallInfo* ci) @@ -44,13 +42,13 @@ int lua_getargument(lua_State* L, int level, int n) { if (n <= fp->numparams) { - luaC_checkthreadsleep(L); + luaC_threadbarrier(L); luaA_pushobject(L, ci->base + (n - 1)); res = 1; } else if (fp->is_vararg && n < ci->base - ci->func) { - luaC_checkthreadsleep(L); + luaC_threadbarrier(L); luaA_pushobject(L, ci->func + n); res = 1; } @@ -69,7 +67,7 @@ const char* lua_getlocal(lua_State* L, int level, int n) const LocVar* var = fp ? luaF_getlocal(fp, n, currentpc(L, ci)) : NULL; if (var) { - luaC_checkthreadsleep(L); + luaC_threadbarrier(L); luaA_pushobject(L, ci->base + var->reg); } const char* name = var ? getstr(var->varname) : NULL; @@ -185,7 +183,7 @@ int lua_getinfo(lua_State* L, int level, const char* what, lua_Debug* ar) status = auxgetinfo(L, what, ar, f, ci); if (strchr(what, 'f')) { - luaC_checkthreadsleep(L); + luaC_threadbarrier(L); setclvalue(L, L->top, f); incr_top(L); } @@ -437,29 +435,17 @@ static int getnextline(Proto* p, int line) int lua_breakpoint(lua_State* L, int funcindex, int line, int enabled) { - int target = -1; + const TValue* func = luaA_toobject(L, funcindex); + api_check(L, ttisfunction(func) && !clvalue(func)->isC); - if (FFlag::LuauDebuggerBreakpointHitOnNextBestLine) + Proto* p = clvalue(func)->l.p; + // Find line number to add the breakpoint to. + int target = getnextline(p, line); + + if (target != -1) { - const TValue* func = luaA_toobject(L, funcindex); - api_check(L, ttisfunction(func) && !clvalue(func)->isC); - - Proto* p = clvalue(func)->l.p; - // Find line number to add the breakpoint to. - target = getnextline(p, line); - - if (target != -1) - { - // Add breakpoint on the exact line - luaG_breakpoint(L, p, target, bool(enabled)); - } - } - else - { - const TValue* func = luaA_toobject(L, funcindex); - api_check(L, ttisfunction(func) && !clvalue(func)->isC); - - luaG_breakpoint(L, clvalue(func)->l.p, line, bool(enabled)); + // Add breakpoint on the exact line + luaG_breakpoint(L, p, target, bool(enabled)); } return target; diff --git a/VM/src/ldo.cpp b/VM/src/ldo.cpp index 6016e41f..51f63d32 100644 --- a/VM/src/ldo.cpp +++ b/VM/src/ldo.cpp @@ -158,7 +158,7 @@ l_noret luaD_throw(lua_State* L, int errcode) static void correctstack(lua_State* L, TValue* oldstack) { L->top = (L->top - oldstack) + L->stack; - for (UpVal* up = L->openupval; up != NULL; up = up->u.l.threadnext) + for (UpVal* up = L->openupval; up != NULL; up = up->u.open.threadnext) up->v = (up->v - oldstack) + L->stack; for (CallInfo* ci = L->base_ci; ci <= L->ci; ci++) { @@ -245,7 +245,7 @@ void luaD_call(lua_State* L, StkId func, int nResults) int oldactive = luaC_threadactive(L); l_setbit(L->stackstate, THREAD_ACTIVEBIT); - luaC_checkthreadsleep(L); + luaC_threadbarrier(L); luau_execute(L); // call it @@ -454,7 +454,7 @@ int lua_resume(lua_State* L, lua_State* from, int nargs) L->baseCcalls = ++L->nCcalls; l_setbit(L->stackstate, THREAD_ACTIVEBIT); - luaC_checkthreadsleep(L); + luaC_threadbarrier(L); status = luaD_rawrunprotected(L, resume, L->top - nargs); @@ -483,7 +483,7 @@ int lua_resumeerror(lua_State* L, lua_State* from) L->baseCcalls = ++L->nCcalls; l_setbit(L->stackstate, THREAD_ACTIVEBIT); - luaC_checkthreadsleep(L); + luaC_threadbarrier(L); status = LUA_ERRRUN; diff --git a/VM/src/lfunc.cpp b/VM/src/lfunc.cpp index bd0f8264..8c78083b 100644 --- a/VM/src/lfunc.cpp +++ b/VM/src/lfunc.cpp @@ -6,6 +6,9 @@ #include "lmem.h" #include "lgc.h" +LUAU_FASTFLAG(LuauSimplerUpval) +LUAU_FASTFLAG(LuauNoSleepBit) + Proto* luaF_newproto(lua_State* L) { Proto* f = luaM_newgco(L, Proto, sizeof(Proto), L->activememcat); @@ -71,59 +74,76 @@ UpVal* luaF_findupval(lua_State* L, StkId level) UpVal* p; while (*pp != NULL && (p = *pp)->v >= level) { - LUAU_ASSERT(p->v != &p->u.value); + LUAU_ASSERT(!FFlag::LuauSimplerUpval || !isdead(g, obj2gco(p))); + LUAU_ASSERT(upisopen(p)); if (p->v == level) - { // found a corresponding upvalue? - if (isdead(g, obj2gco(p))) // is it dead? - changewhite(obj2gco(p)); // resurrect it + { // found a corresponding upvalue? + if (!FFlag::LuauSimplerUpval && isdead(g, obj2gco(p))) // is it dead? + changewhite(obj2gco(p)); // resurrect it return p; } - pp = &p->u.l.threadnext; + pp = &p->u.open.threadnext; } + LUAU_ASSERT(luaC_threadactive(L)); + LUAU_ASSERT(!luaC_threadsleeping(L)); + LUAU_ASSERT(!FFlag::LuauNoSleepBit || !isblack(obj2gco(L))); // we don't use luaC_threadbarrier because active threads never turn black + UpVal* uv = luaM_newgco(L, UpVal, sizeof(UpVal), L->activememcat); // not found: create a new one - uv->tt = LUA_TUPVAL; - uv->marked = luaC_white(g); - uv->memcat = L->activememcat; + luaC_init(L, uv, LUA_TUPVAL); + uv->markedopen = 0; uv->v = level; // current value lives in the stack // chain the upvalue in the threads open upvalue list at the proper position - UpVal* next = *pp; - uv->u.l.threadnext = next; - uv->u.l.threadprev = pp; - if (next) - next->u.l.threadprev = &uv->u.l.threadnext; + if (FFlag::LuauSimplerUpval) + { + uv->u.open.threadnext = *pp; + *pp = uv; + } + else + { + UpVal* next = *pp; + uv->u.open.threadnext = next; - *pp = uv; + uv->u.open.threadprev = pp; + if (next) + next->u.open.threadprev = &uv->u.open.threadnext; + + *pp = uv; + } // double link the upvalue in the global open upvalue list - uv->u.l.prev = &g->uvhead; - uv->u.l.next = g->uvhead.u.l.next; - uv->u.l.next->u.l.prev = uv; - g->uvhead.u.l.next = uv; - LUAU_ASSERT(uv->u.l.next->u.l.prev == uv && uv->u.l.prev->u.l.next == uv); + uv->u.open.prev = &g->uvhead; + uv->u.open.next = g->uvhead.u.open.next; + uv->u.open.next->u.open.prev = uv; + g->uvhead.u.open.next = uv; + LUAU_ASSERT(uv->u.open.next->u.open.prev == uv && uv->u.open.prev->u.open.next == uv); + return uv; } + void luaF_unlinkupval(UpVal* uv) { + LUAU_ASSERT(!FFlag::LuauSimplerUpval); + // unlink upvalue from the global open upvalue list - LUAU_ASSERT(uv->u.l.next->u.l.prev == uv && uv->u.l.prev->u.l.next == uv); - uv->u.l.next->u.l.prev = uv->u.l.prev; - uv->u.l.prev->u.l.next = uv->u.l.next; + LUAU_ASSERT(uv->u.open.next->u.open.prev == uv && uv->u.open.prev->u.open.next == uv); + uv->u.open.next->u.open.prev = uv->u.open.prev; + uv->u.open.prev->u.open.next = uv->u.open.next; // unlink upvalue from the thread open upvalue list - *uv->u.l.threadprev = uv->u.l.threadnext; + *uv->u.open.threadprev = uv->u.open.threadnext; - if (UpVal* next = uv->u.l.threadnext) - next->u.l.threadprev = uv->u.l.threadprev; + if (UpVal* next = uv->u.open.threadnext) + next->u.open.threadprev = uv->u.open.threadprev; } void luaF_freeupval(lua_State* L, UpVal* uv, lua_Page* page) { - if (uv->v != &uv->u.value) // is it open? - luaF_unlinkupval(uv); // remove from open list - luaM_freegco(L, uv, sizeof(UpVal), uv->memcat, page); // free upvalue + if (!FFlag::LuauSimplerUpval && uv->v != &uv->u.value) // is it open? + luaF_unlinkupval(uv); // remove from open list + luaM_freegco(L, uv, sizeof(UpVal), uv->memcat, page); // free upvalue } void luaF_close(lua_State* L, StkId level) @@ -133,26 +153,55 @@ void luaF_close(lua_State* L, StkId level) while (L->openupval != NULL && (uv = L->openupval)->v >= level) { GCObject* o = obj2gco(uv); - LUAU_ASSERT(!isblack(o) && uv->v != &uv->u.value); + LUAU_ASSERT(!isblack(o) && upisopen(uv)); - // by removing the upvalue from global/thread open upvalue lists, L->openupval will be pointing to the next upvalue - luaF_unlinkupval(uv); - - if (isdead(g, o)) + if (FFlag::LuauSimplerUpval) { - // close the upvalue without copying the dead data so that luaF_freeupval will not unlink again - uv->v = &uv->u.value; + LUAU_ASSERT(!isdead(g, o)); + + // unlink value *before* closing it since value storage overlaps + L->openupval = uv->u.open.threadnext; + + luaF_closeupval(L, uv, /* dead= */ false); } else { - setobj(L, &uv->u.value, uv->v); - uv->v = &uv->u.value; - // GC state of a new closed upvalue has to be initialized - luaC_initupval(L, uv); + // by removing the upvalue from global/thread open upvalue lists, L->openupval will be pointing to the next upvalue + luaF_unlinkupval(uv); + + if (isdead(g, o)) + { + // close the upvalue without copying the dead data so that luaF_freeupval will not unlink again + uv->v = &uv->u.value; + } + else + { + setobj(L, &uv->u.value, uv->v); + uv->v = &uv->u.value; + // GC state of a new closed upvalue has to be initialized + luaC_upvalclosed(L, uv); + } } } } +void luaF_closeupval(lua_State* L, UpVal* uv, bool dead) +{ + LUAU_ASSERT(FFlag::LuauSimplerUpval); + + // unlink value from all lists *before* closing it since value storage overlaps + LUAU_ASSERT(uv->u.open.next->u.open.prev == uv && uv->u.open.prev->u.open.next == uv); + uv->u.open.next->u.open.prev = uv->u.open.prev; + uv->u.open.prev->u.open.next = uv->u.open.next; + + if (dead) + return; + + setobj(L, &uv->u.value, uv->v); + uv->v = &uv->u.value; + luaC_upvalclosed(L, uv); +} + void luaF_freeproto(lua_State* L, Proto* f, lua_Page* page) { luaM_freearray(L, f->code, f->sizecode, Instruction, f->memcat); diff --git a/VM/src/lfunc.h b/VM/src/lfunc.h index 59ab5726..899d040b 100644 --- a/VM/src/lfunc.h +++ b/VM/src/lfunc.h @@ -12,6 +12,7 @@ LUAI_FUNC Closure* luaF_newLclosure(lua_State* L, int nelems, Table* e, Proto* p LUAI_FUNC Closure* luaF_newCclosure(lua_State* L, int nelems, Table* e); LUAI_FUNC UpVal* luaF_findupval(lua_State* L, StkId level); LUAI_FUNC void luaF_close(lua_State* L, StkId level); +LUAI_FUNC void luaF_closeupval(lua_State* L, UpVal* uv, bool dead); LUAI_FUNC void luaF_freeproto(lua_State* L, Proto* f, struct lua_Page* page); LUAI_FUNC void luaF_freeclosure(lua_State* L, Closure* c, struct lua_Page* page); LUAI_FUNC void luaF_unlinkupval(UpVal* uv); diff --git a/VM/src/lgc.cpp b/VM/src/lgc.cpp index f7a851f4..b95d6dee 100644 --- a/VM/src/lgc.cpp +++ b/VM/src/lgc.cpp @@ -13,6 +13,117 @@ #include +/* + * Luau uses an incremental non-generational non-moving mark&sweep garbage collector. + * + * The collector runs in three stages: mark, atomic and sweep. Mark and sweep are incremental and try to do a limited amount + * of work every GC step; atomic is ran once per the GC cycle and is indivisible. In either case, the work happens during GC + * steps that are "scheduled" by the GC pacing algorithm - the steps happen either from explicit calls to lua_gc, or after + * the mutator (aka application) allocates some amount of memory, which is known as "GC assist". In either case, GC steps + * can't happen concurrently with other access to VM state. + * + * Current GC stage is stored in global_State::gcstate, and has two additional stages for pause and second-phase mark, explained below. + * + * GC pacer is an algorithm that tries to ensure that GC can always catch up to the application allocating garbage, but do this + * with minimal amount of effort. To configure the pacer Luau provides control over three variables: GC goal, defined as the + * target heap size during atomic phase in relation to live heap size (e.g. 200% goal means the heap's worst case size is double + * the total size of alive objects), step size (how many kilobytes should the application allocate for GC step to trigger), and + * GC multiplier (how much should the GC try to mark relative to how much the application allocated). It's critical that step + * multiplier is significantly above 1, as this is what allows the GC to catch up to the application's allocation rate, and + * GC goal and GC multiplier are linked in subtle ways, described in lua.h comments for LUA_GCSETGOAL. + * + * During mark, GC tries to identify all reachable objects and mark them as reachable, while keeping unreachable objects unmarked. + * During sweep, GC tries to sweep all objects that were not reachable at the end of mark. The atomic phase is needed to ensure + * that all pending marking has completed and all objects that are still marked as unreachable are, in fact, unreachable. + * + * Notably, during mark GC doesn't free any objects, and so the heap size constantly grows; during sweep, GC doesn't do any marking + * work, so it can't immediately free objects that became unreachable after sweeping started. + * + * Every collectable object has one of three colors at any given point in time: white, gray or black. This coloring scheme + * is necessary to implement incremental marking: white objects have not been marked and may be unreachable, black objects + * have been marked and will not be marked again if they stay black, and gray objects have been marked but may contain unmarked + * references. + * + * Objects are allocated as white; however, during sweep, we need to differentiate between objects that remained white in the mark + * phase (these are not reachable and can be freed) and objects that were allocated after the mark phase ended. Because of this, the + * colors are encoded using three bits inside GCheader::marked: white0, white1 and black (so technically we use a four-color scheme: + * any object can be white0, white1, gray or black). All bits are exclusive, and gray objects have all three bits unset. This allows + * us to have the "current" white bit, which is flipped during atomic stage - during sweeping, objects that have the white color from + * the previous mark may be deleted, and all other objects may or may not be reachable, and will be changed to the current white color, + * so that the next mark can start coloring objects from scratch again. + * + * Crucially, the coloring scheme comes with what's known as a tri-color invariant: a black object may never point to a white object. + * + * At the end of atomic stage, the expectation is that there are no gray objects anymore, which means all objects are either black + * (reachable) or white (unreachable = dead). Tri-color invariant is maintained throughout mark and atomic phase. To uphold this + * invariant, every modification of an object needs to check if the object is black and the new referent is white; if so, we + * need to either mark the referent, making it non-white (known as a forward barrier), or mark the object as gray and queue it + * for additional marking (known as a backward barrier). + * + * Luau uses both types of barriers. Forward barriers advance GC progress, since they don't create new outstanding work for GC, + * but they may be expensive when an object is modified many times in succession. Backward barriers are cheaper, as they defer + * most of the work until "later", but they require queueing the object for a rescan which isn't always possible. Table writes usually + * use backward barriers (but switch to forward barriers during second-phase mark), whereas upvalue writes and setmetatable use forward + * barriers. + * + * Since marking is incremental, it needs a way to track progress, which is implemented as a gray set: at any point, objects that + * are gray need to mark their white references, objects that are black have no pending work, and objects that are white have not yet + * been reached. Once the gray set is empty, the work completes; as such, incremental marking is as simple as removing an object from + * the gray set, and turning it to black (which requires turning all its white references to gray). The gray set is implemented as + * an intrusive singly linked list, using `gclist` field in multiple objects (functions, tables, threads and protos). When an object + * doesn't have gclist field, the marking of that object needs to be "immediate", changing the colors of all references in one go. + * + * When a black object is modified, it needs to become gray again. Objects like this are placed on a separate `grayagain` list by a + * barrier - this is important because it allows us to have a mark stage that terminates when the gray set is empty even if the mutator + * is constantly changing existing objects to gray. After mark stage finishes traversing `gray` list, we copy `grayagain` list to `gray` + * once and incrementally mark it again. During this phase of marking, we may get more objects marked as `grayagain`, so after we finish + * emptying out the `gray` list the second time, we finish the mark stage and do final marking of `grayagain` during atomic phase. + * GC works correctly without this second-phase mark (called GCSpropagateagain), but it reduces the time spent during atomic phase. + * + * Sweeping is also incremental, but instead of working at a granularity of an object, it works at a granularity of a page: all GC + * objects are allocated in special pages (see lmem.cpp for details), and sweeper traverses all objects in one page in one incremental + * step, freeing objects that aren't reachable (old white), and recoloring all other objects with the new white to prepare them for next + * mark. During sweeping we don't need to maintain the GC invariant, because our goal is to paint all objects with current white - + * however, some barriers will still trigger (because some reachable objects are still black as sweeping didn't get to them yet), and + * some barriers will proactively mark black objects as white to avoid extra barriers from triggering excessively. + * + * Most references that GC deals with are strong, and as such they fit neatly into the incremental marking scheme. Some, however, are + * weak - notably, tables can be marked as having weak keys/values (using __mode metafield). During incremental marking, we don't know + * for certain if a given object is alive - if it's marked as black, it definitely was reachable during marking, but if it's marked as + * white, we don't know if it's actually unreachable. Because of this, we need to defer weak table handling to the atomic phase; after + * all objects are marked, we traverse all weak tables (that are linked into special weak table lists using `gclist` during marking), + * and remove all entries that have white keys or values. If keys or values are strong, they are marked normally. + * + * The simplified scheme described above isn't fully accurate because of threads, upvalues and strings. + * + * Strings are semantically black (they are initially white, and when the mark stage reaches a string, it changes its color and never + * touches the object again), but they are technically marked as gray - the black bit is never set on a string object. This behavior + * is inherited from Lua 5.1 GC, but doesn't have a clear rationale - effectively, strings are marked as gray but are never part of + * a gray list. + * + * Threads are hard to deal with because for them to fit into the white-gray-black scheme, writes to thread stacks need to have barriers + * that turn the thread from black (already scanned) to gray - but this is very expensive because stack writes are very common. To + * get around this problem, threads have an "active" state which means that a thread is actively executing code. When GC reaches an active + * thread, it keeps it as gray, and rescans it during atomic phase. When a thread is inactive, GC instead paints the thread black. All + * API calls that can write to thread stacks outside of execution (which implies active) uses a thread barrier that checks if the thread is + * black, and if it is it marks it as gray and puts it on a gray list to be rescanned during atomic phase. + * + * NOTE: The above is only true when LuauNoSleepBit is enabled. + * + * Upvalues are special objects that can be closed, in which case they contain the value (acting as a reference cell) and can be dealt + * with using the regular algorithm, or open, in which case they refer to a stack slot in some other thread. These are difficult to deal + * with because the stack writes are not monitored. Because of this open upvalues are treated in a somewhat special way: they are never marked + * as black (doing so would violate the GC invariant), and they are kept in a special global list (global_State::uvhead) which is traversed + * during atomic phase. This is needed because an open upvalue might point to a stack location in a dead thread that never marked the stack + * slot - upvalues like this are identified since they don't have `markedopen` bit set during thread traversal and closed in `clearupvals`. + * + * NOTE: The above is only true when LuauSimplerUpval is enabled. + */ + +LUAU_FASTFLAGVARIABLE(LuauSimplerUpval, false) +LUAU_FASTFLAGVARIABLE(LuauNoSleepBit, false) +LUAU_FASTFLAGVARIABLE(LuauEagerShrink, false) + #define GC_SWEEPPAGESTEPCOST 16 #define GC_INTERRUPT(state) \ @@ -150,8 +261,8 @@ static void reallymarkobject(global_State* g, GCObject* o) { UpVal* uv = gco2uv(o); markvalue(g, uv->v); - if (uv->v == &uv->u.value) // closed? - gray2black(o); // open upvalues are never black + if (!upisopen(uv)) // closed? + gray2black(o); // open upvalues are never black return; } case LUA_TFUNCTION: @@ -289,22 +400,34 @@ static void traverseclosure(global_State* g, Closure* cl) } } -static void traversestack(global_State* g, lua_State* l, bool clearstack) +static void traversestack(global_State* g, lua_State* l) { markobject(g, l->gt); if (l->namecall) stringmark(l->namecall); for (StkId o = l->stack; o < l->top; o++) markvalue(g, o); - // final traversal? - if (g->gcstate == GCSatomic || clearstack) + if (FFlag::LuauSimplerUpval) { - StkId stack_end = l->stack + l->stacksize; - for (StkId o = l->top; o < stack_end; o++) // clear not-marked stack slice - setnilvalue(o); + for (UpVal* uv = l->openupval; uv; uv = uv->u.open.threadnext) + { + LUAU_ASSERT(upisopen(uv)); + uv->markedopen = 1; + markobject(g, uv); + } } } +static void clearstack(lua_State* l) +{ + StkId stack_end = l->stack + l->stacksize; + for (StkId o = l->top; o < stack_end; o++) // clear not-marked stack slice + setnilvalue(o); +} + +// TODO: pull function definition here when FFlag::LuauEagerShrink is removed +static void shrinkstack(lua_State* L); + /* ** traverse one gray object, turning it to black. ** Returns `quantity' traversed. @@ -338,14 +461,17 @@ static size_t propagatemark(global_State* g) LUAU_ASSERT(!luaC_threadsleeping(th)); - // threads that are executing and the main thread are not deactivated + // threads that are executing and the main thread remain gray bool active = luaC_threadactive(th) || th == th->global->mainthread; + // TODO: Refactor this logic after LuauNoSleepBit is removed if (!active && g->gcstate == GCSpropagate) { - traversestack(g, th, /* clearstack= */ true); + traversestack(g, th); + clearstack(th); - l_setbit(th->stackstate, THREAD_SLEEPINGBIT); + if (!FFlag::LuauNoSleepBit) + l_setbit(th->stackstate, THREAD_SLEEPINGBIT); } else { @@ -354,9 +480,17 @@ static size_t propagatemark(global_State* g) black2gray(o); - traversestack(g, th, /* clearstack= */ false); + traversestack(g, th); + + // final traversal? + if (g->gcstate == GCSatomic) + clearstack(th); } + // we could shrink stack at any time but we opt to skip it during atomic since it's redundant to do that more than once per cycle + if (FFlag::LuauEagerShrink && g->gcstate != GCSatomic) + shrinkstack(th); + return sizeof(lua_State) + sizeof(TValue) * th->stacksize + sizeof(CallInfo) * th->size_ci; } case LUA_TPROTO: @@ -537,7 +671,7 @@ static bool deletegco(void* context, lua_Page* page, GCObject* gco) // we are in the process of deleting everything // threads with open upvalues will attempt to close them all on removal // but those upvalues might point to stack values that were already deleted - if (gco->gch.tt == LUA_TTHREAD) + if (!FFlag::LuauSimplerUpval && gco->gch.tt == LUA_TTHREAD) { lua_State* th = gco2th(gco); @@ -595,13 +729,53 @@ static void markroot(lua_State* L) static size_t remarkupvals(global_State* g) { size_t work = 0; - for (UpVal* uv = g->uvhead.u.l.next; uv != &g->uvhead; uv = uv->u.l.next) + + for (UpVal* uv = g->uvhead.u.open.next; uv != &g->uvhead; uv = uv->u.open.next) { work += sizeof(UpVal); - LUAU_ASSERT(uv->u.l.next->u.l.prev == uv && uv->u.l.prev->u.l.next == uv); + + LUAU_ASSERT(upisopen(uv)); + LUAU_ASSERT(uv->u.open.next->u.open.prev == uv && uv->u.open.prev->u.open.next == uv); + LUAU_ASSERT(!isblack(obj2gco(uv))); // open upvalues are never black + if (isgray(obj2gco(uv))) markvalue(g, uv->v); } + + return work; +} + +static size_t clearupvals(lua_State* L) +{ + global_State* g = L->global; + + size_t work = 0; + + for (UpVal* uv = g->uvhead.u.open.next; uv != &g->uvhead;) + { + work += sizeof(UpVal); + + LUAU_ASSERT(upisopen(uv)); + LUAU_ASSERT(uv->u.open.next->u.open.prev == uv && uv->u.open.prev->u.open.next == uv); + LUAU_ASSERT(!isblack(obj2gco(uv))); // open upvalues are never black + LUAU_ASSERT(iswhite(obj2gco(uv)) || !iscollectable(uv->v) || !iswhite(gcvalue(uv->v))); + + if (uv->markedopen) + { + // upvalue is still open (belongs to alive thread) + LUAU_ASSERT(isgray(obj2gco(uv))); + uv->markedopen = 0; // for next cycle + uv = uv->u.open.next; + } + else + { + // upvalue is either dead, or alive but the thread is dead; unlink and close + UpVal* next = uv->u.open.next; + luaF_closeupval(L, uv, /* dead= */ iswhite(obj2gco(uv))); + uv = next; + } + } + return work; } @@ -654,6 +828,16 @@ static size_t atomic(lua_State* L) g->gcmetrics.currcycle.atomictimeclear += recordGcDeltaTime(currts); #endif + if (FFlag::LuauSimplerUpval) + { + // close orphaned live upvalues of dead threads and clear dead upvalues + work += clearupvals(L); + +#ifdef LUAI_GCMETRICS + g->gcmetrics.currcycle.atomictimeupval += recordGcDeltaTime(currts); +#endif + } + // flip current white g->currentwhite = cast_byte(otherwhite(g)); g->sweepgcopage = g->allgcopages; @@ -677,8 +861,11 @@ static bool sweepgco(lua_State* L, lua_Page* page, GCObject* gco) if (alive) { - resetbit(th->stackstate, THREAD_SLEEPINGBIT); - shrinkstack(th); + if (!FFlag::LuauNoSleepBit) + resetbit(th->stackstate, THREAD_SLEEPINGBIT); + + if (!FFlag::LuauEagerShrink) + shrinkstack(th); } } @@ -945,7 +1132,7 @@ void luaC_fullgc(lua_State* L) startGcCycleMetrics(g); #endif - if (g->gcstate <= GCSatomic) + if (FFlag::LuauSimplerUpval ? keepinvariant(g) : g->gcstate <= GCSatomic) { // reset sweep marks to sweep all elements (returning them to white) g->sweepgcopage = g->allgcopages; @@ -955,7 +1142,7 @@ void luaC_fullgc(lua_State* L) g->weak = NULL; g->gcstate = GCSsweep; } - LUAU_ASSERT(g->gcstate == GCSsweep); + LUAU_ASSERT(g->gcstate == GCSpause || g->gcstate == GCSsweep); // finish any pending sweep phase while (g->gcstate != GCSpause) { @@ -963,6 +1150,16 @@ void luaC_fullgc(lua_State* L) gcstep(L, SIZE_MAX); } + if (FFlag::LuauSimplerUpval) + { + // clear markedopen bits for all open upvalues; these might be stuck from half-finished mark prior to full gc + for (UpVal* uv = g->uvhead.u.open.next; uv != &g->uvhead; uv = uv->u.open.next) + { + LUAU_ASSERT(upisopen(uv)); + uv->markedopen = 0; + } + } + #ifdef LUAI_GCMETRICS finishGcCycleMetrics(g); startGcCycleMetrics(g); @@ -999,6 +1196,7 @@ void luaC_fullgc(lua_State* L) void luaC_barrierupval(lua_State* L, GCObject* v) { + LUAU_ASSERT(!FFlag::LuauSimplerUpval); global_State* g = L->global; LUAU_ASSERT(iswhite(v) && !isdead(g, v)); @@ -1038,30 +1236,24 @@ void luaC_barriertable(lua_State* L, Table* t, GCObject* v) g->grayagain = o; } -void luaC_barrierback(lua_State* L, Table* t) +void luaC_barrierback(lua_State* L, GCObject* o, GCObject** gclist) { global_State* g = L->global; - GCObject* o = obj2gco(t); LUAU_ASSERT(isblack(o) && !isdead(g, o)); LUAU_ASSERT(g->gcstate != GCSpause); - black2gray(o); // make table gray (again) - t->gclist = g->grayagain; + + black2gray(o); // make object gray (again) + *gclist = g->grayagain; g->grayagain = o; } -void luaC_initobj(lua_State* L, GCObject* o, uint8_t tt) -{ - global_State* g = L->global; - o->gch.marked = luaC_white(g); - o->gch.tt = tt; - o->gch.memcat = L->activememcat; -} - -void luaC_initupval(lua_State* L, UpVal* uv) +void luaC_upvalclosed(lua_State* L, UpVal* uv) { global_State* g = L->global; GCObject* o = obj2gco(uv); + LUAU_ASSERT(!upisopen(uv)); // upvalue was closed but needs GC state fixup + if (isgray(o)) { if (keepinvariant(g)) @@ -1105,6 +1297,7 @@ int64_t luaC_allocationrate(lua_State* L) void luaC_wakethread(lua_State* L) { + LUAU_ASSERT(!FFlag::LuauNoSleepBit); if (!luaC_threadsleeping(L)) return; @@ -1116,6 +1309,8 @@ void luaC_wakethread(lua_State* L) { GCObject* o = obj2gco(L); + LUAU_ASSERT(isblack(o)); + L->gclist = g->grayagain; g->grayagain = o; diff --git a/VM/src/lgc.h b/VM/src/lgc.h index 7b03a25d..69379c89 100644 --- a/VM/src/lgc.h +++ b/VM/src/lgc.h @@ -6,6 +6,8 @@ #include "lobject.h" #include "lstate.h" +LUAU_FASTFLAG(LuauNoSleepBit) + /* ** Default settings for GC tunables (settable via lua_gc) */ @@ -74,6 +76,7 @@ #define luaC_white(g) cast_to(uint8_t, ((g)->currentwhite) & WHITEBITS) // Thread stack states +// TODO: Remove with FFlag::LuauNoSleepBit and replace with lua_State::threadactive #define THREAD_ACTIVEBIT 0 // thread is currently active #define THREAD_SLEEPINGBIT 1 // thread is not executing and stack should not be modified @@ -109,7 +112,7 @@ #define luaC_barrierfast(L, t) \ { \ if (isblack(obj2gco(t))) \ - luaC_barrierback(L, t); \ + luaC_barrierback(L, obj2gco(t), &t->gclist); \ } #define luaC_objbarrier(L, p, o) \ @@ -118,29 +121,43 @@ luaC_barrierf(L, obj2gco(p), obj2gco(o)); \ } +// TODO: Remove with FFlag::LuauSimplerUpval #define luaC_upvalbarrier(L, uv, tv) \ { \ if (iscollectable(tv) && iswhite(gcvalue(tv)) && (!(uv) || (uv)->v != &(uv)->u.value)) \ luaC_barrierupval(L, gcvalue(tv)); \ } -#define luaC_checkthreadsleep(L) \ +#define luaC_threadbarrier(L) \ { \ - if (luaC_threadsleeping(L)) \ - luaC_wakethread(L); \ + if (FFlag::LuauNoSleepBit) \ + { \ + if (isblack(obj2gco(L))) \ + luaC_barrierback(L, obj2gco(L), &L->gclist); \ + } \ + else \ + { \ + if (luaC_threadsleeping(L)) \ + luaC_wakethread(L); \ + } \ } -#define luaC_init(L, o, tt) luaC_initobj(L, cast_to(GCObject*, (o)), tt) +#define luaC_init(L, o, tt_) \ + { \ + o->marked = luaC_white(L->global); \ + o->tt = tt_; \ + o->memcat = L->activememcat; \ + } LUAI_FUNC void luaC_freeall(lua_State* L); LUAI_FUNC size_t luaC_step(lua_State* L, bool assist); LUAI_FUNC void luaC_fullgc(lua_State* L); LUAI_FUNC void luaC_initobj(lua_State* L, GCObject* o, uint8_t tt); -LUAI_FUNC void luaC_initupval(lua_State* L, UpVal* uv); +LUAI_FUNC void luaC_upvalclosed(lua_State* L, UpVal* uv); LUAI_FUNC void luaC_barrierupval(lua_State* L, GCObject* v); LUAI_FUNC void luaC_barrierf(lua_State* L, GCObject* o, GCObject* v); LUAI_FUNC void luaC_barriertable(lua_State* L, Table* t, GCObject* v); -LUAI_FUNC void luaC_barrierback(lua_State* L, Table* t); +LUAI_FUNC void luaC_barrierback(lua_State* L, GCObject* o, GCObject** gclist); LUAI_FUNC void luaC_validate(lua_State* L); LUAI_FUNC void luaC_dump(lua_State* L, void* file, const char* (*categoryName)(lua_State* L, uint8_t memcat)); LUAI_FUNC int64_t luaC_allocationrate(lua_State* L); diff --git a/VM/src/lgcdebug.cpp b/VM/src/lgcdebug.cpp index b6204b1a..2f9c1756 100644 --- a/VM/src/lgcdebug.cpp +++ b/VM/src/lgcdebug.cpp @@ -102,10 +102,12 @@ static void validatestack(global_State* g, lua_State* l) if (l->namecall) validateobjref(g, obj2gco(l), obj2gco(l->namecall)); - for (UpVal* uv = l->openupval; uv; uv = uv->u.l.threadnext) + for (UpVal* uv = l->openupval; uv; uv = uv->u.open.threadnext) { LUAU_ASSERT(uv->tt == LUA_TUPVAL); - LUAU_ASSERT(uv->v != &uv->u.value); + LUAU_ASSERT(upisopen(uv)); + LUAU_ASSERT(uv->u.open.next->u.open.prev == uv && uv->u.open.prev->u.open.next == uv); + LUAU_ASSERT(!isblack(obj2gco(uv))); // open upvalues are never black } } @@ -235,11 +237,12 @@ void luaC_validate(lua_State* L) luaM_visitgco(L, L, validategco); - for (UpVal* uv = g->uvhead.u.l.next; uv != &g->uvhead; uv = uv->u.l.next) + for (UpVal* uv = g->uvhead.u.open.next; uv != &g->uvhead; uv = uv->u.open.next) { LUAU_ASSERT(uv->tt == LUA_TUPVAL); - LUAU_ASSERT(uv->v != &uv->u.value); - LUAU_ASSERT(uv->u.l.next->u.l.prev == uv && uv->u.l.prev->u.l.next == uv); + LUAU_ASSERT(upisopen(uv)); + LUAU_ASSERT(uv->u.open.next->u.open.prev == uv && uv->u.open.prev->u.open.next == uv); + LUAU_ASSERT(!isblack(obj2gco(uv))); // open upvalues are never black } } @@ -508,13 +511,14 @@ static void dumpproto(FILE* f, Proto* p) static void dumpupval(FILE* f, UpVal* uv) { - fprintf(f, "{\"type\":\"upvalue\",\"cat\":%d,\"size\":%d", uv->memcat, int(sizeof(UpVal))); + fprintf(f, "{\"type\":\"upvalue\",\"cat\":%d,\"size\":%d,\"open\":%s", uv->memcat, int(sizeof(UpVal)), upisopen(uv) ? "true" : "false"); if (iscollectable(uv->v)) { fprintf(f, ",\"object\":"); dumpref(f, gcvalue(uv->v)); } + fprintf(f, "}"); } diff --git a/VM/src/lobject.h b/VM/src/lobject.h index 2097e335..778e22ba 100644 --- a/VM/src/lobject.h +++ b/VM/src/lobject.h @@ -232,7 +232,7 @@ typedef struct TString int16_t atom; // 2 byte padding - TString* next; // next string in the hash table bucket or the string buffer linked list + TString* next; // next string in the hash table bucket unsigned int hash; unsigned int len; @@ -316,7 +316,10 @@ typedef struct LocVar typedef struct UpVal { CommonHeader; - // 1 (x86) or 5 (x64) byte padding + uint8_t markedopen; // set if reachable from an alive thread (only valid during atomic) + + // 4 byte padding (x64) + TValue* v; // points to stack or to its own value union { @@ -327,14 +330,16 @@ typedef struct UpVal struct UpVal* prev; struct UpVal* next; - // thread double linked list (when open) + // thread linked list (when open) struct UpVal* threadnext; // note: this is the location of a pointer to this upvalue in the previous element that can be either an UpVal or a lua_State - struct UpVal** threadprev; - } l; + struct UpVal** threadprev; // TODO: remove with FFlag::LuauSimplerUpval + } open; } u; } UpVal; +#define upisopen(up) ((up)->v != &(up)->u.value) + /* ** Closures */ diff --git a/VM/src/lstate.cpp b/VM/src/lstate.cpp index 4489f840..e1cb2ab7 100644 --- a/VM/src/lstate.cpp +++ b/VM/src/lstate.cpp @@ -10,6 +10,8 @@ #include "ldo.h" #include "ldebug.h" +LUAU_FASTFLAG(LuauSimplerUpval) + /* ** Main thread combines a thread state and the global state */ @@ -119,8 +121,11 @@ lua_State* luaE_newthread(lua_State* L) void luaE_freethread(lua_State* L, lua_State* L1, lua_Page* page) { - luaF_close(L1, L1->stack); // close all upvalues for this thread - LUAU_ASSERT(L1->openupval == NULL); + if (!FFlag::LuauSimplerUpval) + { + luaF_close(L1, L1->stack); // close all upvalues for this thread + LUAU_ASSERT(L1->openupval == NULL); + } global_State* g = L->global; if (g->cb.userthread) g->cb.userthread(NULL, L1); @@ -175,8 +180,8 @@ lua_State* lua_newstate(lua_Alloc f, void* ud) g->frealloc = f; g->ud = ud; g->mainthread = L; - g->uvhead.u.l.prev = &g->uvhead; - g->uvhead.u.l.next = &g->uvhead; + g->uvhead.u.open.prev = &g->uvhead; + g->uvhead.u.open.next = &g->uvhead; g->GCthreshold = 0; // mark it as unfinished state g->registryfree = 0; g->errorjmp = NULL; diff --git a/VM/src/lstate.h b/VM/src/lstate.h index 72a09713..df47ce7e 100644 --- a/VM/src/lstate.h +++ b/VM/src/lstate.h @@ -167,7 +167,7 @@ typedef struct global_State GCObject* grayagain; // list of objects to be traversed atomically GCObject* weak; // list of weak tables (to be cleared) - TString* strbufgc; // list of all string buffer objects + TString* strbufgc; // list of all string buffer objects; TODO: remove with LuauNoStrbufLink size_t GCthreshold; // when totalbytes > GCthreshold, run GC step diff --git a/VM/src/lstring.cpp b/VM/src/lstring.cpp index 9c266031..f43d03b1 100644 --- a/VM/src/lstring.cpp +++ b/VM/src/lstring.cpp @@ -7,6 +7,8 @@ #include +LUAU_FASTFLAGVARIABLE(LuauNoStrbufLink, false) + unsigned int luaS_hash(const char* str, size_t len) { // Note that this hashing algorithm is replicated in BytecodeBuilder.cpp, BytecodeBuilder::getStringHash @@ -70,40 +72,33 @@ void luaS_resize(lua_State* L, int newsize) static TString* newlstr(lua_State* L, const char* str, size_t l, unsigned int h) { - TString* ts; - stringtable* tb; if (l > MAXSSIZE) luaM_toobig(L); - ts = luaM_newgco(L, TString, sizestring(l), L->activememcat); - ts->len = unsigned(l); + + TString* ts = luaM_newgco(L, TString, sizestring(l), L->activememcat); + luaC_init(L, ts, LUA_TSTRING); + ts->atom = ATOM_UNDEF; ts->hash = h; - ts->marked = luaC_white(L->global); - ts->tt = LUA_TSTRING; - ts->memcat = L->activememcat; + ts->len = unsigned(l); + memcpy(ts->data, str, l); ts->data[l] = '\0'; // ending 0 - ts->atom = ATOM_UNDEF; - tb = &L->global->strt; + + stringtable* tb = &L->global->strt; h = lmod(h, tb->size); ts->next = tb->hash[h]; // chain new entry tb->hash[h] = ts; + tb->nuse++; if (tb->nuse > cast_to(uint32_t, tb->size) && tb->size <= INT_MAX / 2) luaS_resize(L, tb->size * 2); // too crowded + return ts; } -static void linkstrbuf(lua_State* L, TString* ts) -{ - global_State* g = L->global; - - ts->next = g->strbufgc; - g->strbufgc = ts; - ts->marked = luaC_white(g); -} - static void unlinkstrbuf(lua_State* L, TString* ts) { + LUAU_ASSERT(!FFlag::LuauNoStrbufLink); global_State* g = L->global; TString** p = &g->strbufgc; @@ -129,14 +124,24 @@ TString* luaS_bufstart(lua_State* L, size_t size) if (size > MAXSSIZE) luaM_toobig(L); + global_State* g = L->global; + TString* ts = luaM_newgco(L, TString, sizestring(size), L->activememcat); - - ts->tt = LUA_TSTRING; - ts->memcat = L->activememcat; - linkstrbuf(L, ts); - + luaC_init(L, ts, LUA_TSTRING); + ts->atom = ATOM_UNDEF; + ts->hash = 0; // computed in luaS_buffinish ts->len = unsigned(size); + if (FFlag::LuauNoStrbufLink) + { + ts->next = NULL; + } + else + { + ts->next = g->strbufgc; + g->strbufgc = ts; + } + return ts; } @@ -159,7 +164,10 @@ TString* luaS_buffinish(lua_State* L, TString* ts) } } - unlinkstrbuf(L, ts); + if (FFlag::LuauNoStrbufLink) + LUAU_ASSERT(ts->next == NULL); + else + unlinkstrbuf(L, ts); ts->hash = h; ts->data[ts->len] = '\0'; // ending 0 @@ -214,11 +222,21 @@ static bool unlinkstr(lua_State* L, TString* ts) void luaS_free(lua_State* L, TString* ts, lua_Page* page) { - // Unchain from the string table - if (!unlinkstr(L, ts)) - unlinkstrbuf(L, ts); // An unlikely scenario when we have a string buffer on our hands + if (FFlag::LuauNoStrbufLink) + { + if (unlinkstr(L, ts)) + L->global->strt.nuse--; + else + LUAU_ASSERT(ts->next == NULL); // orphaned string buffer + } else - L->global->strt.nuse--; + { + // Unchain from the string table + if (!unlinkstr(L, ts)) + unlinkstrbuf(L, ts); // An unlikely scenario when we have a string buffer on our hands + else + L->global->strt.nuse--; + } luaM_freegco(L, ts, sizestring(ts->len), ts->memcat, page); } diff --git a/VM/src/lvmexecute.cpp b/VM/src/lvmexecute.cpp index 376dd400..7306b055 100644 --- a/VM/src/lvmexecute.cpp +++ b/VM/src/lvmexecute.cpp @@ -16,7 +16,8 @@ #include -LUAU_FASTFLAGVARIABLE(LuauNicerMethodErrors, false) +LUAU_FASTFLAG(LuauSimplerUpval) +LUAU_FASTFLAG(LuauNoSleepBit) // Disable c99-designator to avoid the warning in CGOTO dispatch table #ifdef __clang__ @@ -111,7 +112,7 @@ LUAU_FASTFLAGVARIABLE(LuauNicerMethodErrors, false) VM_DISPATCH_OP(LOP_LOADKX), VM_DISPATCH_OP(LOP_JUMPX), VM_DISPATCH_OP(LOP_FASTCALL), VM_DISPATCH_OP(LOP_COVERAGE), \ VM_DISPATCH_OP(LOP_CAPTURE), VM_DISPATCH_OP(LOP_JUMPIFEQK), VM_DISPATCH_OP(LOP_JUMPIFNOTEQK), VM_DISPATCH_OP(LOP_FASTCALL1), \ VM_DISPATCH_OP(LOP_FASTCALL2), VM_DISPATCH_OP(LOP_FASTCALL2K), VM_DISPATCH_OP(LOP_FORGPREP), VM_DISPATCH_OP(LOP_JUMPXEQKNIL), \ - VM_DISPATCH_OP(LOP_JUMPXEQKB), VM_DISPATCH_OP(LOP_JUMPXEQKN), VM_DISPATCH_OP(LOP_JUMPXEQKS), \ + VM_DISPATCH_OP(LOP_JUMPXEQKB), VM_DISPATCH_OP(LOP_JUMPXEQKN), VM_DISPATCH_OP(LOP_JUMPXEQKS), #if defined(__GNUC__) || defined(__clang__) #define VM_USE_CGOTO 1 @@ -317,6 +318,7 @@ static void luau_execute(lua_State* L) LUAU_ASSERT(isLua(L->ci)); LUAU_ASSERT(luaC_threadactive(L)); LUAU_ASSERT(!luaC_threadsleeping(L)); + LUAU_ASSERT(!FFlag::LuauNoSleepBit || !isblack(obj2gco(L))); // we don't use luaC_threadbarrier because active threads never turn black pc = L->ci->savedpc; cl = clvalue(L->ci->func); @@ -496,7 +498,8 @@ static void luau_execute(lua_State* L) setobj(L, uv->v, ra); luaC_barrier(L, uv, ra); - luaC_upvalbarrier(L, uv, uv->v); + if (!FFlag::LuauSimplerUpval) + luaC_upvalbarrier(L, uv, uv->v); VM_NEXT(); } @@ -932,7 +935,7 @@ static void luau_execute(lua_State* L) VM_PATCH_C(pc - 2, L->cachedslot); // recompute ra since stack might have been reallocated ra = VM_REG(LUAU_INSN_A(insn)); - if (FFlag::LuauNicerMethodErrors && ttisnil(ra)) + if (ttisnil(ra)) luaG_methoderror(L, ra + 1, tsvalue(kv)); } } @@ -973,7 +976,7 @@ static void luau_execute(lua_State* L) VM_PATCH_C(pc - 2, L->cachedslot); // recompute ra since stack might have been reallocated ra = VM_REG(LUAU_INSN_A(insn)); - if (FFlag::LuauNicerMethodErrors && ttisnil(ra)) + if (ttisnil(ra)) luaG_methoderror(L, ra + 1, tsvalue(kv)); } } @@ -984,7 +987,7 @@ static void luau_execute(lua_State* L) VM_PROTECT(luaV_gettable(L, rb, kv, ra)); // recompute ra since stack might have been reallocated ra = VM_REG(LUAU_INSN_A(insn)); - if (FFlag::LuauNicerMethodErrors && ttisnil(ra)) + if (ttisnil(ra)) luaG_methoderror(L, ra + 1, tsvalue(kv)); } } diff --git a/VM/src/lvmload.cpp b/VM/src/lvmload.cpp index 86afddd2..0ae85ab6 100644 --- a/VM/src/lvmload.cpp +++ b/VM/src/lvmload.cpp @@ -351,7 +351,7 @@ int luau_load(lua_State* L, const char* chunkname, const char* data, size_t size uint32_t mainid = readVarInt(data, size, offset); Proto* main = protos[mainid]; - luaC_checkthreadsleep(L); + luaC_threadbarrier(L); Closure* cl = luaF_newLclosure(L, 0, envt, main); setclvalue(L, L->top, cl); diff --git a/VM/src/lvmutils.cpp b/VM/src/lvmutils.cpp index 8be241e0..33d47020 100644 --- a/VM/src/lvmutils.cpp +++ b/VM/src/lvmutils.cpp @@ -10,9 +10,6 @@ #include "lnumutils.h" #include -#include - -LUAU_FASTFLAGVARIABLE(LuauBetterNewindex, false) // limit for table tag-method chains (to avoid loops) #define MAXTAGLOOP 100 @@ -142,46 +139,25 @@ void luaV_settable(lua_State* L, const TValue* t, TValue* key, StkId val) { // `t' is a table? Table* h = hvalue(t); - if (FFlag::LuauBetterNewindex) - { - const TValue* oldval = luaH_get(h, key); + const TValue* oldval = luaH_get(h, key); - // should we assign the key? (if key is valid or __newindex is not set) - if (!ttisnil(oldval) || (tm = fasttm(L, h->metatable, TM_NEWINDEX)) == NULL) - { - if (h->readonly) - luaG_readonlyerror(L); - - // luaH_set would work but would repeat the lookup so we use luaH_setslot that can reuse oldval if it's safe - TValue* newval = luaH_setslot(L, h, oldval, key); - - L->cachedslot = gval2slot(h, newval); // remember slot to accelerate future lookups - - setobj2t(L, newval, val); - luaC_barriert(L, h, val); - return; - } - - // fallthrough to metamethod - } - else + // should we assign the key? (if key is valid or __newindex is not set) + if (!ttisnil(oldval) || (tm = fasttm(L, h->metatable, TM_NEWINDEX)) == NULL) { if (h->readonly) luaG_readonlyerror(L); - TValue* oldval = luaH_set(L, h, key); // do a primitive set + // luaH_set would work but would repeat the lookup so we use luaH_setslot that can reuse oldval if it's safe + TValue* newval = luaH_setslot(L, h, oldval, key); - L->cachedslot = gval2slot(h, oldval); // remember slot to accelerate future lookups + L->cachedslot = gval2slot(h, newval); // remember slot to accelerate future lookups - if (!ttisnil(oldval) || // result is no nil? - (tm = fasttm(L, h->metatable, TM_NEWINDEX)) == NULL) - { // or no TM? - setobj2t(L, oldval, val); - luaC_barriert(L, h, val); - return; - } - // else will try the tag method + setobj2t(L, newval, val); + luaC_barriert(L, h, val); + return; } + + // fallthrough to metamethod } else if (ttisnil(tm = luaT_gettmbyobj(L, t, TM_NEWINDEX))) luaG_indexerror(L, t, key); diff --git a/bench/bench.py b/bench/bench.py index 42a0ac97..0db33950 100644 --- a/bench/bench.py +++ b/bench/bench.py @@ -38,6 +38,7 @@ argumentParser.add_argument('--run-test', action='store', default=None, help='Re argumentParser.add_argument('--extra-loops', action='store',type=int,default=0, help='Amount of times to loop over one test (one test already performs multiple runs)') argumentParser.add_argument('--filename', action='store',type=str,default='bench', help='File name for graph and results file') argumentParser.add_argument('--callgrind', dest='callgrind',action='store_const',const=1,default=0,help='Use callgrind to run benchmarks') +argumentParser.add_argument('--show-commands', dest='show_commands',action='store_const',const=1,default=0,help='Show the command line used to launch the VM and tests') if matplotlib != None: argumentParser.add_argument('--absolute', dest='absolute',action='store_const',const=1,default=0,help='Display absolute values instead of relative (enabled by default when benchmarking a single VM)') @@ -87,17 +88,25 @@ def getCallgrindOutput(lines): return "".join(result) +def conditionallyShowCommand(cmd): + if arguments.show_commands: + print(f'{colored(Color.BLUE, "EXECUTING")}: {cmd}') + def getVmOutput(cmd): if os.name == "nt": try: - return subprocess.check_output("start /realtime /affinity 1 /b /wait cmd /C \"" + cmd + "\"", shell=True, cwd=scriptdir).decode() + fullCmd = "start /realtime /affinity 1 /b /wait cmd /C \"" + cmd + "\"" + conditionallyShowCommand(fullCmd) + return subprocess.check_output(fullCmd, shell=True, cwd=scriptdir).decode() except KeyboardInterrupt: exit(1) except: return "" elif arguments.callgrind: try: - subprocess.check_call("valgrind --tool=callgrind --callgrind-out-file=callgrind.out --combine-dumps=yes --dump-line=no " + cmd, shell=True, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, cwd=scriptdir) + fullCmd = "valgrind --tool=callgrind --callgrind-out-file=callgrind.out --combine-dumps=yes --dump-line=no " + cmd + conditionallyShowCommand(fullCmd) + subprocess.check_call(fullCmd, shell=True, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, cwd=scriptdir) path = os.path.join(scriptdir, "callgrind.out") with open(path, "r") as file: lines = file.readlines() @@ -106,6 +115,7 @@ def getVmOutput(cmd): except: return "" else: + conditionallyShowCommand(cmd) with subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True, cwd=scriptdir) as p: # Try to lock to a single processor if sys.platform != "darwin": diff --git a/tests/AstQuery.test.cpp b/tests/AstQuery.test.cpp index 6ec1426c..2b650fa4 100644 --- a/tests/AstQuery.test.cpp +++ b/tests/AstQuery.test.cpp @@ -138,4 +138,80 @@ print(workspace:) REQUIRE(ancestry.back()->is()); } +TEST_CASE_FIXTURE(Fixture, "Luau_query") +{ + AstStatBlock* block = parse(R"( + if true then + end + )"); + + AstStatIf* if_ = Luau::query(block); + CHECK(if_); +} + +TEST_CASE_FIXTURE(Fixture, "Luau_query_for_2nd_if_stat_which_doesnt_exist") +{ + AstStatBlock* block = parse(R"( + if true then + end + )"); + + AstStatIf* if_ = Luau::query(block); + CHECK(!if_); +} + +TEST_CASE_FIXTURE(Fixture, "Luau_nested_query") +{ + AstStatBlock* block = parse(R"( + if true then + end + )"); + + AstStatIf* if_ = Luau::query(block); + REQUIRE(if_); + AstExprConstantBool* bool_ = Luau::query(if_); + REQUIRE(bool_); +} + +TEST_CASE_FIXTURE(Fixture, "Luau_nested_query_but_first_query_failed") +{ + AstStatBlock* block = parse(R"( + if true then + end + )"); + + AstStatIf* if_ = Luau::query(block); + REQUIRE(!if_); + AstExprConstantBool* bool_ = Luau::query(if_); // ensure it doesn't crash + REQUIRE(!bool_); +} + +TEST_CASE_FIXTURE(Fixture, "Luau_selectively_query_for_a_different_boolean") +{ + AstStatBlock* block = parse(R"( + local x = false and true + local y = true and false + )"); + + AstExprConstantBool* fst = Luau::query(block, {nth(), nth(2)}); + REQUIRE(fst); + REQUIRE(fst->value == true); + + AstExprConstantBool* snd = Luau::query(block, {nth(2), nth(2)}); + REQUIRE(snd); + REQUIRE(snd->value == false); +} + +TEST_CASE_FIXTURE(Fixture, "Luau_selectively_query_for_a_different_boolean_2") +{ + AstStatBlock* block = parse(R"( + local x = false and true + local y = true and false + )"); + + AstExprConstantBool* snd = Luau::query(block, {nth(2), nth()}); + REQUIRE(snd); + REQUIRE(snd->value == true); +} + TEST_SUITE_END(); diff --git a/tests/Compiler.test.cpp b/tests/Compiler.test.cpp index 1aa69112..0a3c6507 100644 --- a/tests/Compiler.test.cpp +++ b/tests/Compiler.test.cpp @@ -2154,8 +2154,6 @@ TEST_CASE("RecursionParse") CHECK_EQ(std::string(e.what()), "Exceeded allowed recursion depth; simplify your expression to make the code compile"); } -#if 0 - // This currently requires too much stack space on MSVC/x64 and crashes with stack overflow at recursion depth 935 try { Luau::compileOrThrow(bcb, rep("function a() ", 1500) + "print()" + rep(" end", 1500)); @@ -2175,7 +2173,6 @@ TEST_CASE("RecursionParse") { CHECK_EQ(std::string(e.what()), "Exceeded allowed recursion depth; simplify your block to make the code compile"); } -#endif } TEST_CASE("ArrayIndexLiteral") diff --git a/tests/Conformance.test.cpp b/tests/Conformance.test.cpp index 89cb0751..f6f5b41f 100644 --- a/tests/Conformance.test.cpp +++ b/tests/Conformance.test.cpp @@ -319,15 +319,11 @@ TEST_CASE("Literals") TEST_CASE("Errors") { - ScopedFastFlag sff("LuauNicerMethodErrors", true); - runConformance("errors.lua"); } TEST_CASE("Events") { - ScopedFastFlag sff("LuauBetterNewindex", true); - runConformance("events.lua"); } diff --git a/tests/Fixture.cpp b/tests/Fixture.cpp index 40be39a0..4051f851 100644 --- a/tests/Fixture.cpp +++ b/tests/Fixture.cpp @@ -512,4 +512,41 @@ void dump(const std::vector& constraints) printf("%s\n", toString(c, opts).c_str()); } +FindNthOccurenceOf::FindNthOccurenceOf(Nth nth) + : requestedNth(nth) +{ +} + +bool FindNthOccurenceOf::checkIt(AstNode* n) +{ + if (theNode) + return false; + + if (n->classIndex == requestedNth.classIndex) + { + // Human factor: the requestedNth starts from 1 because of the term `nth`. + if (currentOccurrence + 1 != requestedNth.nth) + ++currentOccurrence; + else + theNode = n; + } + + return !theNode; // once found, returns false and stops traversal +} + +bool FindNthOccurenceOf::visit(AstNode* n) +{ + return checkIt(n); +} + +bool FindNthOccurenceOf::visit(AstType* t) +{ + return checkIt(t); +} + +bool FindNthOccurenceOf::visit(AstTypePack* t) +{ + return checkIt(t); +} + } // namespace Luau diff --git a/tests/Fixture.h b/tests/Fixture.h index 8dc3dd24..956f95d0 100644 --- a/tests/Fixture.h +++ b/tests/Fixture.h @@ -195,6 +195,76 @@ std::optional lookupName(ScopePtr scope, const std::string& name); // Wa std::optional linearSearchForBinding(Scope* scope, const char* name); +struct Nth +{ + int classIndex; + int nth; +}; + +template +Nth nth(int nth = 1) +{ + static_assert(std::is_base_of_v, "T must be a derived class of AstNode"); + LUAU_ASSERT(nth > 0); // Did you mean to use `nth(1)`? + + return Nth{T::ClassIndex(), nth}; +} + +struct FindNthOccurenceOf : public AstVisitor +{ + Nth requestedNth; + int currentOccurrence = 0; + AstNode* theNode = nullptr; + + FindNthOccurenceOf(Nth nth); + + bool checkIt(AstNode* n); + + bool visit(AstNode* n) override; + bool visit(AstType* n) override; + bool visit(AstTypePack* n) override; +}; + +/** DSL querying of the AST. + * + * Given an AST, one can query for a particular node directly without having to manually unwrap the tree, for example: + * + * ``` + * if a and b then + * print(a + b) + * end + * + * function f(x, y) + * return x + y + * end + * ``` + * + * There are numerous ways to access the second AstExprBinary. + * 1. Luau::query(block, {nth(), nth()}) + * 2. Luau::query(Luau::query(block)) + * 3. Luau::query(block, {nth(2)}) + */ +template +T* query(AstNode* node, const std::vector& nths = {nth(N)}) +{ + static_assert(std::is_base_of_v, "T must be a derived class of AstNode"); + + // If a nested query call fails to find the node in question, subsequent calls can propagate rather than trying to do more. + // This supports `query(query(...))` + + for (Nth nth : nths) + { + if (!node) + return nullptr; + + FindNthOccurenceOf finder{nth}; + node->visit(&finder); + node = finder.theNode; + } + + return node ? node->as() : nullptr; +} + } // namespace Luau #define LUAU_REQUIRE_ERRORS(result) \ diff --git a/tests/Linter.test.cpp b/tests/Linter.test.cpp index 5a4ab33e..a7d09e8f 100644 --- a/tests/Linter.test.cpp +++ b/tests/Linter.test.cpp @@ -43,6 +43,20 @@ TEST_CASE_FIXTURE(Fixture, "DeprecatedGlobal") CHECK_EQ(result.warnings[0].text, "Global 'Wait' is deprecated, use 'wait' instead"); } +TEST_CASE_FIXTURE(Fixture, "DeprecatedGlobalNoReplacement") +{ + ScopedFastFlag sff{"LuauLintFixDeprecationMessage", true}; + + // Normally this would be defined externally, so hack it in for testing + const char* deprecationReplacementString = ""; + addGlobalBinding(typeChecker, "Version", Binding{typeChecker.anyType, {}, true, deprecationReplacementString}); + + LintResult result = lintTyped("Version()"); + + REQUIRE_EQ(result.warnings.size(), 1); + CHECK_EQ(result.warnings[0].text, "Global 'Version' is deprecated"); +} + TEST_CASE_FIXTURE(Fixture, "PlaceholderRead") { LintResult result = lint(R"( diff --git a/tests/ToString.test.cpp b/tests/ToString.test.cpp index fe376d86..ab5d8598 100644 --- a/tests/ToString.test.cpp +++ b/tests/ToString.test.cpp @@ -11,6 +11,7 @@ using namespace Luau; LUAU_FASTFLAG(LuauRecursiveTypeParameterRestriction); LUAU_FASTFLAG(LuauSpecialTypesAsterisked); +LUAU_FASTFLAG(LuauFixNameMaps); TEST_SUITE_BEGIN("ToString"); @@ -433,29 +434,40 @@ TEST_CASE_FIXTURE(Fixture, "toStringDetailed") LUAU_REQUIRE_NO_ERRORS(result); - TypeId id3Type = requireType("id3"); - ToStringResult nameData = toStringDetailed(id3Type); + ToStringOptions opts; + + TypeId id3Type = requireType("id3"); + ToStringResult nameData = toStringDetailed(id3Type, opts); + + if (FFlag::LuauFixNameMaps) + REQUIRE(3 == opts.nameMap.typeVars.size()); + else + REQUIRE_EQ(3, nameData.DEPRECATED_nameMap.typeVars.size()); - REQUIRE_EQ(3, nameData.nameMap.typeVars.size()); REQUIRE_EQ("(a, b, c) -> (a, b, c)", nameData.name); - ToStringOptions opts; - opts.nameMap = std::move(nameData.nameMap); + ToStringOptions opts2; // TODO: delete opts2 when clipping FFlag::LuauFixNameMaps + if (FFlag::LuauFixNameMaps) + opts2.nameMap = std::move(opts.nameMap); + else + opts2.DEPRECATED_nameMap = std::move(nameData.DEPRECATED_nameMap); const FunctionTypeVar* ftv = get(follow(id3Type)); REQUIRE(ftv != nullptr); auto params = flatten(ftv->argTypes).first; - REQUIRE_EQ(3, params.size()); + REQUIRE(3 == params.size()); - REQUIRE_EQ("a", toString(params[0], opts)); - REQUIRE_EQ("b", toString(params[1], opts)); - REQUIRE_EQ("c", toString(params[2], opts)); + CHECK("a" == toString(params[0], opts2)); + CHECK("b" == toString(params[1], opts2)); + CHECK("c" == toString(params[2], opts2)); } TEST_CASE_FIXTURE(BuiltinsFixture, "toStringDetailed2") { - ScopedFastFlag sff2{"DebugLuauSharedSelf", true}; + ScopedFastFlag sff[] = { + {"DebugLuauSharedSelf", true}, + }; CheckResult result = check(R"( local base = {} @@ -470,13 +482,18 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "toStringDetailed2") )"); LUAU_REQUIRE_NO_ERRORS(result); - TypeId tType = requireType("inst"); - ToStringResult r = toStringDetailed(tType); - CHECK_EQ("{ @metatable { __index: { @metatable {| __index: base |}, child } }, inst }", r.name); - CHECK_EQ(0, r.nameMap.typeVars.size()); - ToStringOptions opts; - opts.nameMap = r.nameMap; + + TypeId tType = requireType("inst"); + ToStringResult r = toStringDetailed(tType, opts); + CHECK_EQ("{ @metatable { __index: { @metatable {| __index: base |}, child } }, inst }", r.name); + if (FFlag::LuauFixNameMaps) + CHECK(0 == opts.nameMap.typeVars.size()); + else + CHECK_EQ(0, r.DEPRECATED_nameMap.typeVars.size()); + + if (!FFlag::LuauFixNameMaps) + opts.DEPRECATED_nameMap = r.DEPRECATED_nameMap; const MetatableTypeVar* tMeta = get(tType); REQUIRE(tMeta); @@ -499,7 +516,8 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "toStringDetailed2") REQUIRE(tMeta6); ToStringResult oneResult = toStringDetailed(tMeta5->props["one"].type, opts); - opts.nameMap = oneResult.nameMap; + if (!FFlag::LuauFixNameMaps) + opts.DEPRECATED_nameMap = oneResult.DEPRECATED_nameMap; std::string twoResult = toString(tMeta6->props["two"].type, opts); diff --git a/tests/TypeInfer.builtins.test.cpp b/tests/TypeInfer.builtins.test.cpp index 10da0efa..12fb4aa2 100644 --- a/tests/TypeInfer.builtins.test.cpp +++ b/tests/TypeInfer.builtins.test.cpp @@ -10,6 +10,7 @@ using namespace Luau; LUAU_FASTFLAG(LuauLowerBoundsCalculation); LUAU_FASTFLAG(LuauSpecialTypesAsterisked); +LUAU_FASTFLAG(LuauStringFormatArgumentErrorFix) TEST_SUITE_BEGIN("BuiltinTests"); @@ -721,7 +722,14 @@ TEST_CASE_FIXTURE(Fixture, "string_format_use_correct_argument") LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK_EQ("Argument count mismatch. Function expects 1 argument, but 2 are specified", toString(result.errors[0])); + if (FFlag::LuauStringFormatArgumentErrorFix) + { + CHECK_EQ("Argument count mismatch. Function expects 2 arguments, but 3 are specified", toString(result.errors[0])); + } + else + { + CHECK_EQ("Argument count mismatch. Function expects 1 argument, but 2 are specified", toString(result.errors[0])); + } } TEST_CASE_FIXTURE(Fixture, "string_format_use_correct_argument2") @@ -736,6 +744,22 @@ TEST_CASE_FIXTURE(Fixture, "string_format_use_correct_argument2") CHECK_EQ("Type 'number' could not be converted into 'string'", toString(result.errors[1])); } +TEST_CASE_FIXTURE(BuiltinsFixture, "string_format_use_correct_argument3") +{ + ScopedFastFlag LuauStringFormatArgumentErrorFix{"LuauStringFormatArgumentErrorFix", true}; + + CheckResult result = check(R"( + local s1 = string.format("%d") + local s2 = string.format("%d", 1) + local s3 = string.format("%d", 1, 2) + )"); + + LUAU_REQUIRE_ERROR_COUNT(2, result); + + CHECK_EQ("Argument count mismatch. Function expects 2 arguments, but only 1 is specified", toString(result.errors[0])); + CHECK_EQ("Argument count mismatch. Function expects 2 arguments, but 3 are specified", toString(result.errors[1])); +} + TEST_CASE_FIXTURE(BuiltinsFixture, "debug_traceback_is_crazy") { CheckResult result = check(R"( diff --git a/tests/TypeInfer.provisional.test.cpp b/tests/TypeInfer.provisional.test.cpp index 9a917a6f..c8fc7f2d 100644 --- a/tests/TypeInfer.provisional.test.cpp +++ b/tests/TypeInfer.provisional.test.cpp @@ -544,4 +544,69 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "greedy_inference_with_shared_self_triggers_f CHECK_EQ("Not all codepaths in this function return 'self, a...'.", toString(result.errors[0])); } +TEST_CASE_FIXTURE(Fixture, "dcr_cant_partially_dispatch_a_constraint") +{ + ScopedFastFlag sff[] = { + {"DebugLuauDeferredConstraintResolution", true}, + {"LuauSpecialTypesAsterisked", true}, + }; + + CheckResult result = check(R"( + local function hasDivisors(value: number) + end + + function prime_iter(state, index) + hasDivisors(index) + index += 1 + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + // We should be able to resolve this to number, but we're not there yet. + // Solving this requires recognizing that we can partially solve the + // following constraint: + // + // (*blocked*) -> () <: (number) -> (b...) + // + // The correct thing for us to do is to consider the constraint dispatched, + // but we need to also record a new constraint number <: *blocked* to finish + // the job later. + CHECK("(a, *error-type*) -> ()" == toString(requireType("prime_iter"))); +} + +TEST_CASE_FIXTURE(Fixture, "free_options_cannot_be_unified_together") +{ + ScopedFastFlag sff[] = { + {"LuauFixNameMaps", true}, + }; + + TypeArena arena; + TypeId nilType = getSingletonTypes().nilType; + + std::unique_ptr scope = std::make_unique(getSingletonTypes().anyTypePack); + + TypeId free1 = arena.addType(FreeTypePack{scope.get()}); + TypeId option1 = arena.addType(UnionTypeVar{{nilType, free1}}); + + TypeId free2 = arena.addType(FreeTypePack{scope.get()}); + TypeId option2 = arena.addType(UnionTypeVar{{nilType, free2}}); + + InternalErrorReporter iceHandler; + UnifierSharedState sharedState{&iceHandler}; + Unifier u{&arena, Mode::Strict, NotNull{scope.get()}, Location{}, Variance::Covariant, sharedState}; + + u.tryUnify(option1, option2); + + CHECK(u.errors.empty()); + + u.log.commit(); + + ToStringOptions opts; + CHECK("a?" == toString(option1, opts)); + + // CHECK("a?" == toString(option2, opts)); // This should hold, but does not. + CHECK("b?" == toString(option2, opts)); // This should not hold. +} + TEST_SUITE_END(); diff --git a/tests/conformance/gc.lua b/tests/conformance/gc.lua index 5804ea7f..c49dbe77 100644 --- a/tests/conformance/gc.lua +++ b/tests/conformance/gc.lua @@ -252,25 +252,30 @@ if not rawget(_G, "_soft") then end -- create many threads with self-references and open upvalues -local thread_id = 0 -local threads = {} +do + local thread_id = 0 + local threads = {} -function fn(thread) - local x = {} - threads[thread_id] = function() - thread = x - end - coroutine.yield() + function fn(thread) + local x = {} + threads[thread_id] = function() + thread = x + end + coroutine.yield() + end + + while thread_id < 1000 do + local thread = coroutine.create(fn) + coroutine.resume(thread, thread) + thread_id = thread_id + 1 + end + + collectgarbage() + + -- ensure that we no longer have a lot of reachable threads for subsequent tests + threads = {} end -while thread_id < 1000 do - local thread = coroutine.create(fn) - coroutine.resume(thread, thread) - thread_id = thread_id + 1 -end - - - -- create a userdata to be collected when state is closed do local newproxy,assert,type,print,getmetatable = @@ -322,4 +327,27 @@ do collectgarbage() end +-- create a lot of threads with upvalues to force a case where full gc happens after we've marked some upvalues +do + local t = {} + for i = 1,100 do + local c = coroutine.wrap(function() + local uv = {i + 1} + local function f() + return uv[1] * 10 + end + coroutine.yield(uv[1]) + uv = {i + 2} + coroutine.yield(f()) + end) + + assert(c() == i + 1) + table.insert(t, c) + end + + t = {} + + collectgarbage() +end + return('OK') diff --git a/tools/faillist.txt b/tools/faillist.txt index 630bf9f7..54e7ac05 100644 --- a/tools/faillist.txt +++ b/tools/faillist.txt @@ -53,6 +53,7 @@ AutocompleteTest.generic_types AutocompleteTest.get_suggestions_for_the_very_start_of_the_script AutocompleteTest.global_function_params AutocompleteTest.global_functions_are_not_scoped_lexically +AutocompleteTest.globals_are_order_independent AutocompleteTest.if_then_else_elseif_completions AutocompleteTest.keyword_methods AutocompleteTest.keyword_types @@ -588,7 +589,6 @@ TypeInferFunctions.another_recursive_local_function TypeInferFunctions.calling_function_with_anytypepack_doesnt_leak_free_types TypeInferFunctions.calling_function_with_incorrect_argument_type_yields_errors_spanning_argument TypeInferFunctions.complicated_return_types_require_an_explicit_annotation -TypeInferFunctions.cyclic_function_type_in_args TypeInferFunctions.dont_give_other_overloads_message_if_only_one_argument_matching_overload_exists TypeInferFunctions.dont_infer_parameter_types_for_functions_from_their_call_site TypeInferFunctions.duplicate_functions_with_different_signatures_not_allowed_in_nonstrict @@ -744,7 +744,6 @@ TypeInferUnknownNever.math_operators_and_never TypeInferUnknownNever.type_packs_containing_never_is_itself_uninhabitable TypeInferUnknownNever.type_packs_containing_never_is_itself_uninhabitable2 TypeInferUnknownNever.unary_minus_of_never -TypeInferUnknownNever.unknown_is_reflexive TypePackTests.higher_order_function TypePackTests.multiple_varargs_inference_are_not_confused TypePackTests.no_return_size_should_be_zero diff --git a/tools/natvis/VM.natvis b/tools/natvis/VM.natvis index cb2f355f..e45e4e28 100644 --- a/tools/natvis/VM.natvis +++ b/tools/natvis/VM.natvis @@ -195,7 +195,7 @@ openupval - u.l.threadnext + u.open.threadnext this diff --git a/tools/perfgraph.py b/tools/perfgraph.py index 25bb0dd9..7d2639df 100644 --- a/tools/perfgraph.py +++ b/tools/perfgraph.py @@ -56,7 +56,63 @@ def nodeFromCallstackListFile(source_file): return root -def getDuration(obj): + +def getDuration(nodes, nid): + node = nodes[nid - 1] + total = node['TotalDuration'] + + for cid in node['NodeIds']: + total -= nodes[cid - 1]['TotalDuration'] + + return total + +def getFunctionKey(fn): + return fn['Source'] + "," + fn['Name'] + "," + str(fn['Line']) + +def recursivelyBuildNodeTree(nodes, functions, parent, fid, nid): + ninfo = nodes[nid - 1] + finfo = functions[fid - 1] + + child = parent.child(getFunctionKey(finfo)) + child.source = finfo['Source'] + child.function = finfo['Name'] + child.line = int(finfo['Line']) if finfo['Line'] > 0 else 0 + + child.ticks = getDuration(nodes, nid) + + assert(len(ninfo['FunctionIds']) == len(ninfo['NodeIds'])) + + for i in range(0, len(ninfo['FunctionIds'])): + recursivelyBuildNodeTree(nodes, functions, child, ninfo['FunctionIds'][i], ninfo['NodeIds'][i]) + + return + +def nodeFromJSONV2(dump): + assert(dump['Version'] == 2) + + nodes = dump['Nodes'] + functions = dump['Functions'] + categories = dump['Categories'] + + root = Node() + + for category in categories: + nid = category['NodeId'] + node = nodes[nid - 1] + name = category['Name'] + + child = root.child(name) + child.function = name + child.ticks = getDuration(nodes, nid) + + assert(len(node['FunctionIds']) == len(node['NodeIds'])) + + for i in range(0, len(node['FunctionIds'])): + recursivelyBuildNodeTree(nodes, functions, child, node['FunctionIds'][i], node['NodeIds'][i]) + + return root + +def getDurationV1(obj): total = obj['TotalDuration'] if 'Children' in obj: @@ -73,7 +129,7 @@ def nodeFromJSONObject(node, key, obj): node.source = source node.line = int(line) if len(line) > 0 else 0 - node.ticks = getDuration(obj) + node.ticks = getDurationV1(obj) if 'Children' in obj: for key, obj in obj['Children'].items(): @@ -81,10 +137,8 @@ def nodeFromJSONObject(node, key, obj): return node - -def nodeFromJSONFile(source_file): - dump = json.load(source_file) - +def nodeFromJSONV1(dump): + assert(dump['Version'] == 1) root = Node() if 'Children' in dump: @@ -93,6 +147,16 @@ def nodeFromJSONFile(source_file): return root +def nodeFromJSONFile(source_file): + dump = json.load(source_file) + + if dump['Version'] == 2: + return nodeFromJSONV2(dump) + elif dump['Version'] == 1: + return nodeFromJSONV1(dump) + + return Node() + arguments = argumentParser.parse_args() diff --git a/tools/test_dcr.py b/tools/test_dcr.py index 0efea3c4..da33706c 100644 --- a/tools/test_dcr.py +++ b/tools/test_dcr.py @@ -14,12 +14,14 @@ def loadFailList(): with open(FAIL_LIST_PATH) as f: return set(map(str.strip, f.readlines())) + def safeParseInt(i, default=0): try: return int(i) except ValueError: return default + class Handler(x.ContentHandler): def __init__(self, failList): self.currentTest = [] @@ -47,7 +49,7 @@ class Handler(x.ContentHandler): r = self.results.get(dottedName, True) self.results[dottedName] = r and passed - elif name == 'OverallResultsTestCases': + elif name == "OverallResultsTestCases": self.numSkippedTests = safeParseInt(attrs.get("skipped", 0)) def endElement(self, name): @@ -104,9 +106,9 @@ def main(): for testName, passed in handler.results.items(): if passed and testName in failList: - print('UNEXPECTED: {} should have failed'.format(testName)) + print("UNEXPECTED: {} should have failed".format(testName)) elif not passed and testName not in failList: - print('UNEXPECTED: {} should have passed'.format(testName)) + print("UNEXPECTED: {} should have passed".format(testName)) if args.write: newFailList = sorted( @@ -123,17 +125,24 @@ def main(): print("Updated faillist.txt") if handler.numSkippedTests > 0: - print('{} test(s) were skipped! That probably means that a test segfaulted!'.format(handler.numSkippedTests), file=sys.stderr) + print( + "{} test(s) were skipped! That probably means that a test segfaulted!".format( + handler.numSkippedTests + ), + file=sys.stderr, + ) sys.exit(1) - sys.exit( - 0 - if all( - not passed == (dottedName in failList) - for dottedName, passed in handler.results.items() - ) - else 1 + ok = all( + not passed == (dottedName in failList) + for dottedName, passed in handler.results.items() ) + if ok: + print("Everything in order!", file=sys.stderr) + + sys.exit(0 if ok else 1) + + if __name__ == "__main__": main() From e84bccc5c012885276f7fef57ab7df7b185b605c Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Thu, 25 Aug 2022 14:54:02 -0700 Subject: [PATCH 352/399] RFC: Prohibit use of interpolated strings in function calls without parentheses (#648) We've had this restriction in the original RFC, but decided to remove it afterwards. This change puts the restriction back - we need to work through some implications of future support for string-based DSLs together with interpolated strings which may or may not change the behavior here, for example to allow something like ``` local fragment = xml ` ` ``` --- rfcs/syntax-string-interpolation.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/rfcs/syntax-string-interpolation.md b/rfcs/syntax-string-interpolation.md index 9f7ae085..2fbb04b0 100644 --- a/rfcs/syntax-string-interpolation.md +++ b/rfcs/syntax-string-interpolation.md @@ -87,13 +87,15 @@ print(`Welcome to \ -- Luau! ``` -This expression can also come after a `prefixexp`: +We currently *prohibit* using interpolated strings in function calls without parentheses, this is illegal: ``` local name = "world" print`Hello {name}` ``` +> Note: This restriction is likely temporary while we work through string interpolation DSLs, an ability to pass individual components of interpolated strings to a function. + The restriction on `{{` exists solely for the people coming from languages e.g. C#, Rust, or Python which uses `{{` to escape and get the character `{` at runtime. We're also rejecting this at parse time too, since the proper way to escape it is `\{`, so: ```lua From e9e2cba77a649d082534427754bd416e9b9e0ce3 Mon Sep 17 00:00:00 2001 From: JohnnyMorganz Date: Mon, 29 Aug 2022 16:28:04 +0100 Subject: [PATCH 353/399] Fix overloaded metamethod in class definitions (#653) Fixes #652 --- Analysis/src/TypeInfer.cpp | 20 ++++++-------------- tests/TypeInfer.definitions.test.cpp | 26 ++++++++++++++++++++++++++ 2 files changed, 32 insertions(+), 14 deletions(-) diff --git a/Analysis/src/TypeInfer.cpp b/Analysis/src/TypeInfer.cpp index 77168053..c9340945 100644 --- a/Analysis/src/TypeInfer.cpp +++ b/Analysis/src/TypeInfer.cpp @@ -1659,6 +1659,7 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatDeclareClass& declar TypeId propTy = resolveType(scope, *prop.ty); bool assignToMetatable = isMetamethod(propName); + Luau::ClassTypeVar::Props& assignTo = assignToMetatable ? metatable->props : ctv->props; // Function types always take 'self', but this isn't reflected in the // parsed annotation. Add it here. @@ -1674,16 +1675,13 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatDeclareClass& declar } } - if (ctv->props.count(propName) == 0) + if (assignTo.count(propName) == 0) { - if (assignToMetatable) - metatable->props[propName] = {propTy}; - else - ctv->props[propName] = {propTy}; + assignTo[propName] = {propTy}; } else { - TypeId currentTy = assignToMetatable ? metatable->props[propName].type : ctv->props[propName].type; + TypeId currentTy = assignTo[propName].type; // We special-case this logic to keep the intersection flat; otherwise we // would create a ton of nested intersection types. @@ -1693,19 +1691,13 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatDeclareClass& declar options.push_back(propTy); TypeId newItv = addType(IntersectionTypeVar{std::move(options)}); - if (assignToMetatable) - metatable->props[propName] = {newItv}; - else - ctv->props[propName] = {newItv}; + assignTo[propName] = {newItv}; } else if (get(currentTy)) { TypeId intersection = addType(IntersectionTypeVar{{currentTy, propTy}}); - if (assignToMetatable) - metatable->props[propName] = {intersection}; - else - ctv->props[propName] = {intersection}; + assignTo[propName] = {intersection}; } else { diff --git a/tests/TypeInfer.definitions.test.cpp b/tests/TypeInfer.definitions.test.cpp index fd6fb83f..9fe0c6aa 100644 --- a/tests/TypeInfer.definitions.test.cpp +++ b/tests/TypeInfer.definitions.test.cpp @@ -336,4 +336,30 @@ local s : Cls = GetCls() LUAU_REQUIRE_NO_ERRORS(result); } +TEST_CASE_FIXTURE(Fixture, "class_definition_overload_metamethods") +{ + loadDefinition(R"( + declare class Vector3 + end + + declare class CFrame + function __mul(self, other: CFrame): CFrame + function __mul(self, other: Vector3): Vector3 + end + + declare function newVector3(): Vector3 + declare function newCFrame(): CFrame + )"); + + CheckResult result = check(R"( + local base = newCFrame() + local shouldBeCFrame = base * newCFrame() + local shouldBeVector = base * newVector3() + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + CHECK_EQ(toString(requireType("shouldBeCFrame")), "CFrame"); + CHECK_EQ(toString(requireType("shouldBeVector")), "Vector3"); +} + TEST_SUITE_END(); From 42c24f98d91ce326eb875ecfa6179c79bfd890c1 Mon Sep 17 00:00:00 2001 From: Alexander McCord <11488393+alexmccord@users.noreply.github.com> Date: Wed, 31 Aug 2022 10:46:06 -0700 Subject: [PATCH 354/399] July and August recap (#654) Co-authored-by: vegorov-rbx <75688451+vegorov-rbx@users.noreply.github.com> Co-authored-by: Arseny Kapoulkine --- .../2022-08-29-luau-recap-august-2022.md | 187 ++++++++++++++++++ 1 file changed, 187 insertions(+) create mode 100644 docs/_posts/2022-08-29-luau-recap-august-2022.md diff --git a/docs/_posts/2022-08-29-luau-recap-august-2022.md b/docs/_posts/2022-08-29-luau-recap-august-2022.md new file mode 100644 index 00000000..a0e88536 --- /dev/null +++ b/docs/_posts/2022-08-29-luau-recap-august-2022.md @@ -0,0 +1,187 @@ +--- +layout: single +title: "Luau Recap: August 2022" +--- + +Luau is our new language that you can read more about at [https://luau-lang.org](https://luau-lang.org). + +[Cross-posted to the [Roblox Developer Forum](https://devforum.roblox.com/t/luau-recap-august-2022/).] + +## Tables now support `__len` metamethod + +See the RFC [Support `__len` metamethod for tables and `rawlen` function](https://github.com/Roblox/luau/blob/master/rfcs/len-metamethod-rawlen.md) for more details. + +With generalized iteration released in May, custom containers are easier than ever to use. The only thing missing was the fact that tables didn't respect `__len`. + +Simply, tables now honor the `__len` metamethod, and `rawlen` is also added with similar semantics as `rawget` and `rawset`: + +```lua +local my_cool_container = setmetatable({ items = { 1, 2 } }, { + __len = function(self) return #self.items end +}) + +print(#my_cool_container) --> 2 +print(rawlen(my_cool_container)) --> 0 +``` + +## `never` and `unknown` types + +See the RFC [`never` and `unknown` types](https://github.com/Roblox/luau/blob/master/rfcs/never-and-unknown-types.md) for more details. + +We've added two new types, `never` and `unknown`. These two types are the opposites of each other by the fact that there's no value that inhabits the type `never`, and the dual of that is every value inhabits the type `unknown`. + +Type inference may infer a variable to have the type `never` if and only if the set of possible types becomes empty, for example through type refinements. + +```lua +function f(x: string | number) + if typeof(x) == "string" and typeof(x) == "number" then + -- x: never + end +end +``` + +This is useful because we still needed to ascribe a type to `x` here, but the type we used previously had unsound semantics. For example, it was possible to be able to _expand_ the domain of a variable once the user had proved it impossible. With `never`, narrowing a type from `never` yields `never`. + +Conversely, `unknown` can be used to enforce a stronger contract than `any`. That is, `unknown` and `any` are similar in terms of allowing every type to inhabit them, and other than `unknown` or `any`, `any` allows itself to inhabit into a different type, whereas `unknown` does not. + +```lua +function any(): any return 5 end +function unknown(): unknown return 5 end + +-- no type error, but assigns a number to x which expects string +local x: string = any() + +-- has type error, unknown cannot be converted into string +local y: string = unknown() +``` + +To be able to do this soundly, you must apply type refinements on a variable of type `unknown`. + +```lua +local u = unknown() + +if typeof(u) == "string" then + local y: string = u -- no type error +end +``` + +A use case of `unknown` is to enforce type safety at implementation sites for data that do not originate in code, but from over the wire. + +## Argument names in type packs when instantiating a type + +We had a bug in the parser which erroneously allowed argument names in type packs that didn't fold into a function type. That is, the below syntax did not generate a parse error when it should have. + +```lua +Foo<(a: number, b: string)> +``` + +## New IntegerParsing lint + +See [the announcement](https://devforum.roblox.com/t/improving-binary-and-hexadecimal-integer-literal-parsing-rules-in-luau/) for more details. We include this here for posterity. + +We've introduced a new lint called IntegerParsing. Right now, it lints three classes of errors: + +1. Truncation of binary literals that resolves to a value over 64 bits, +2. Truncation of hexadecimal literals that resolves to a value over 64 bits, and +3. Double hexadecimal prefix. + +For 1.) and 2.), they are currently not planned to become a parse error, so action is not strictly required here. + +For 3.), this will be a breaking change! See [the rollout plan](https://devforum.roblox.com/t/improving-binary-and-hexadecimal-integer-literal-parsing-rules-in-luau/#rollout-5) for details. + +## New ComparisonPrecedence lint + +We've also introduced a new lint called `ComparisonPrecedence`. It fires in two particular cases: + +1. `not X op Y` where `op` is `==` or `~=`, or +2. `X op Y op Z` where `op` is any of the comparison or equality operators. + +In languages that uses `!` to negate the boolean i.e. `!x == y` looks fine because `!x` _visually_ binds more tightly than Lua's equivalent, `not x`. Unfortunately, the precedences here are identical, that is `!x == y` is `(!x) == y` in the same way that `not x == y` is `(not x) == y`. We also apply this on other operators e.g. `x <= y == y`. + +```lua +-- not X == Y is equivalent to (not X) == Y; consider using X ~= Y, or wrap one of the expressions in parentheses to silence +if not x == y then end + +-- not X ~= Y is equivalent to (not X) ~= Y; consider using X == Y, or wrap one of the expressions in parentheses to silence +if not x ~= y then end + +-- not X <= Y is equivalent to (not X) <= Y; wrap one of the expressions in parentheses to silence +if not x <= y then end + +-- X <= Y == Z is equivalent to (X <= Y) == Z; wrap one of the expressions in parentheses to silence +if x <= y == 0 then end +``` + +As a special exception, this lint pass will not warn for cases like `x == not y` or `not x == not y`, which both looks intentional as it is written and interpreted. + +## Function calls returning singleton types incorrectly widened + +Fix a bug where widening was a little too happy to fire in the case of function calls returning singleton types or union thereof. This was an artifact of the logic that knows not to infer singleton types in cases that makes no sense to. + +```lua +function f(): "abc" | "def" + return if math.random() > 0.5 then "abc" else "def" +end + +-- previously reported that 'string' could not be converted into '"abc" | "def"' +local x: "abc" | "def" = f() +``` + +## `string` can be a subtype of a table with a shape similar to `string` + +The function `my_cool_lower` is a function `(t: t1) -> a... where t1 = {+ lower: (t1) -> a... +}`. + +```lua +function my_cool_lower(t) + return t:lower() +end +``` + +Even though `t1` is a table type, we know `string` is a subtype of `t1` because `string` also has `lower` which is a subtype of `t1`'s `lower`, so this call site now type checks. + +```lua +local s: string = my_cool_lower("HI") +``` + +## Other analysis improvements + +* `string.gmatch`/`string.match`/`string.find` may now return more precise type depending on the patterns used +* Fix a bug where type arena ownership invariant could be violated, causing stability issues +* Fix a bug where internal type error could be presented to the user +* Fix a false positive with optionals & nested tables +* Fix a false positive in non-strict mode when using generalized iteration +* Improve autocomplete behavior in certain cases for `:` calls +* Fix minor inconsistencies in synthesized names for types with metatables +* Fix autocomplete not suggesting globals defined after the cursor +* Fix DeprecatedGlobal warning text in cases when the global is deprecated without a suggested alternative +* Fix an off-by-one error in type error text for incorrect use of `string.format` + +## Other runtime improvements + +* Comparisons with constants are now significantly faster when using clang as a compiler (10-50% gains on internal benchmarks) +* When calling non-existent methods on tables or strings, `foo:bar` now produces a more precise error message +* Improve performance for iteration of tables +* Fix a bug with negative zero in vector components when using vectors as table keys +* Compiler can now constant fold builtins under -O2, for example `string.byte("A")` is compiled to a constant +* Compiler can model the cost of builtins for the purpose of inlining/unrolling +* Local reassignment i.e. `local x = y :: T` is free iff neither `x` nor `y` is mutated/captured +* Improve `debug.traceback` performance by 1.15-1.75x depending on the platform +* Fix a corner case with table assignment semantics when key didn't exist in the table and `__newindex` was defined: we now use Lua 5.2 semantics and call `__newindex`, which results in less wasted space, support for NaN keys in `__newindex` path and correct support for frozen tables +* Reduce parser C stack consumption which fixes some stack overflow crashes on deeply nested sources +* Improve performance of `bit32.extract`/`replace` when width is implied (~3% faster chess) +* Improve performance of `bit32.extract` when field/width are constants (~10% faster base64) +* `string.format` now supports a new format specifier, `%*`, that accepts any value type and formats it using `tostring` rules + +## Thanks + +Thanks for all the contributions! + +* [natteko](https://github.com/natteko) +* [JohnnyMorganz](https://github.com/JohnnyMorganz) +* [khvzak](https://github.com/khvzak) +* [Anaminus](https://github.com/Anaminus) +* [memery-rbx](https://github.com/memery-rbx) +* [jaykru](https://github.com/jaykru) +* [Kampfkarren](https://github.com/Kampfkarren) +* [XmiliaH](https://github.com/XmiliaH) +* [Mactavsin](https://github.com/Mactavsin) From ae35ada579f27df392b711e274cf031f4de496b5 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Thu, 1 Sep 2022 16:14:03 -0700 Subject: [PATCH 355/399] Sync to upstream/release/543 (#657) - Improve ComparisonPrecedence lint suggestions for three-way comparisons (X < Y < Z) - Improve type checking stability - Improve location information for errors when parsing invalid type annotations - Compiler now generates bytecode version 3 in all configurations - Improve performance of comparisons against numeric constants on AArch64 --- Analysis/include/Luau/Anyification.h | 3 +- Analysis/include/Luau/BuiltinDefinitions.h | 1 + Analysis/include/Luau/Constraint.h | 33 +- .../include/Luau/ConstraintGraphBuilder.h | 11 +- Analysis/include/Luau/ConstraintSolver.h | 39 +- .../include/Luau/ConstraintSolverLogger.h | 3 +- Analysis/include/Luau/Frontend.h | 2 +- Analysis/include/Luau/Scope.h | 3 + Analysis/include/Luau/ToString.h | 1 - Analysis/include/Luau/TypeArena.h | 3 +- Analysis/include/Luau/TypeInfer.h | 3 +- Analysis/include/Luau/TypePack.h | 14 +- Analysis/include/Luau/TypeUtils.h | 10 +- Analysis/include/Luau/TypeVar.h | 12 +- Analysis/include/Luau/Unifier.h | 3 +- Analysis/include/Luau/VisitTypeVar.h | 25 +- Analysis/src/Anyification.cpp | 11 +- Analysis/src/Autocomplete.cpp | 12 +- Analysis/src/BuiltinDefinitions.cpp | 57 +++ Analysis/src/Clone.cpp | 36 +- Analysis/src/Constraint.cpp | 7 +- Analysis/src/ConstraintGraphBuilder.cpp | 225 ++++++---- Analysis/src/ConstraintSolver.cpp | 386 ++++++++++++++++-- Analysis/src/ConstraintSolverLogger.cpp | 1 + Analysis/src/Frontend.cpp | 16 +- Analysis/src/Instantiation.cpp | 1 + Analysis/src/JsonEmitter.cpp | 6 +- Analysis/src/Linter.cpp | 30 +- Analysis/src/Module.cpp | 3 +- Analysis/src/Substitution.cpp | 4 +- Analysis/src/ToString.cpp | 36 +- Analysis/src/TypeArena.cpp | 13 +- Analysis/src/TypeAttach.cpp | 5 + Analysis/src/TypeChecker2.cpp | 224 +++++++++- Analysis/src/TypeInfer.cpp | 41 +- Analysis/src/TypePack.cpp | 7 + Analysis/src/TypeUtils.cpp | 34 +- Analysis/src/TypeVar.cpp | 21 +- Analysis/src/TypedAllocator.cpp | 2 + Analysis/src/Unifiable.cpp | 3 +- Analysis/src/Unifier.cpp | 15 +- Ast/include/Luau/Ast.h | 10 + Ast/include/Luau/Parser.h | 6 + Ast/src/Ast.cpp | 13 + Ast/src/Lexer.cpp | 17 +- Ast/src/Parser.cpp | 87 ++-- Common/include/Luau/Bytecode.h | 2 +- Common/include/Luau/ExperimentalFlags.h | 3 +- Compiler/src/BytecodeBuilder.cpp | 5 - Compiler/src/Compiler.cpp | 377 +++++------------ VM/src/lbuiltins.cpp | 6 +- VM/src/lstrlib.cpp | 5 - VM/src/lvmexecute.cpp | 9 + tests/AssemblyBuilderX64.test.cpp | 3 +- tests/Compiler.test.cpp | 27 +- tests/Conformance.test.cpp | 20 +- tests/ConstraintGraphBuilder.test.cpp | 23 +- tests/ConstraintSolver.test.cpp | 11 +- tests/Fixture.cpp | 5 +- tests/Fixture.h | 3 +- tests/JsonEmitter.test.cpp | 2 +- tests/Linter.test.cpp | 14 +- tests/Parser.test.cpp | 45 +- tests/TypeInfer.anyerror.test.cpp | 13 + tests/TypeInfer.builtins.test.cpp | 12 - tests/TypeInfer.functions.test.cpp | 1 + tests/TypeInfer.loops.test.cpp | 41 +- tests/TypeInfer.modules.test.cpp | 29 ++ tests/TypeInfer.primitives.test.cpp | 12 +- tests/TypeInfer.provisional.test.cpp | 20 +- tests/TypeInfer.tables.test.cpp | 2 - tests/conformance/basic.lua | 5 + tests/conformance/errors.lua | 14 +- tools/faillist.txt | 72 +--- tools/test_dcr.py | 16 +- 75 files changed, 1504 insertions(+), 788 deletions(-) diff --git a/Analysis/include/Luau/Anyification.h b/Analysis/include/Luau/Anyification.h index ee8d6689..9dd7d8e0 100644 --- a/Analysis/include/Luau/Anyification.h +++ b/Analysis/include/Luau/Anyification.h @@ -19,6 +19,7 @@ using ScopePtr = std::shared_ptr; // A substitution which replaces free types by any struct Anyification : Substitution { + Anyification(TypeArena* arena, NotNull scope, InternalErrorReporter* iceHandler, TypeId anyType, TypePackId anyTypePack); Anyification(TypeArena* arena, const ScopePtr& scope, InternalErrorReporter* iceHandler, TypeId anyType, TypePackId anyTypePack); NotNull scope; InternalErrorReporter* iceHandler; @@ -35,4 +36,4 @@ struct Anyification : Substitution bool ignoreChildren(TypePackId ty) override; }; -} \ No newline at end of file +} // namespace Luau \ No newline at end of file diff --git a/Analysis/include/Luau/BuiltinDefinitions.h b/Analysis/include/Luau/BuiltinDefinitions.h index 07d897b2..28a4368e 100644 --- a/Analysis/include/Luau/BuiltinDefinitions.h +++ b/Analysis/include/Luau/BuiltinDefinitions.h @@ -34,6 +34,7 @@ TypeId makeFunction( // Polymorphic std::initializer_list paramTypes, std::initializer_list paramNames, std::initializer_list retTypes); void attachMagicFunction(TypeId ty, MagicFunction fn); +void attachDcrMagicFunction(TypeId ty, DcrMagicFunction fn); Property makeProperty(TypeId ty, std::optional documentationSymbol = std::nullopt); void assignPropDocumentationSymbols(TableTypeVar::Props& props, const std::string& baseName); diff --git a/Analysis/include/Luau/Constraint.h b/Analysis/include/Luau/Constraint.h index e9f04e79..d5cfcf3f 100644 --- a/Analysis/include/Luau/Constraint.h +++ b/Analysis/include/Luau/Constraint.h @@ -56,6 +56,10 @@ struct UnaryConstraint TypeId resultType; }; +// let L : leftType +// let R : rightType +// in +// L op R : resultType struct BinaryConstraint { AstExprBinary::Op op; @@ -64,6 +68,14 @@ struct BinaryConstraint TypeId resultType; }; +// iteratee is iterable +// iterators is the iteration types. +struct IterableConstraint +{ + TypePackId iterator; + TypePackId variables; +}; + // name(namedType) = name struct NameConstraint { @@ -78,20 +90,31 @@ struct TypeAliasExpansionConstraint TypeId target; }; -using ConstraintV = Variant; using ConstraintPtr = std::unique_ptr; +struct FunctionCallConstraint +{ + std::vector> innerConstraints; + TypeId fn; + TypePackId result; + class AstExprCall* astFragment; +}; + +using ConstraintV = Variant; + struct Constraint { - Constraint(ConstraintV&& c, NotNull scope); + Constraint(NotNull scope, const Location& location, ConstraintV&& c); Constraint(const Constraint&) = delete; Constraint& operator=(const Constraint&) = delete; - ConstraintV c; - std::vector> dependencies; NotNull scope; + Location location; + ConstraintV c; + + std::vector> dependencies; }; inline Constraint& asMutable(const Constraint& c) diff --git a/Analysis/include/Luau/ConstraintGraphBuilder.h b/Analysis/include/Luau/ConstraintGraphBuilder.h index 0e41e1ee..1cba0d33 100644 --- a/Analysis/include/Luau/ConstraintGraphBuilder.h +++ b/Analysis/include/Luau/ConstraintGraphBuilder.h @@ -9,6 +9,7 @@ #include "Luau/Ast.h" #include "Luau/Constraint.h" #include "Luau/Module.h" +#include "Luau/ModuleResolver.h" #include "Luau/NotNull.h" #include "Luau/Symbol.h" #include "Luau/TypeVar.h" @@ -51,12 +52,15 @@ struct ConstraintGraphBuilder // It is pretty uncommon for constraint generation to itself produce errors, but it can happen. std::vector errors; + // Needed to resolve modules to make 'require' import types properly. + NotNull moduleResolver; // Occasionally constraint generation needs to produce an ICE. const NotNull ice; ScopePtr globalScope; - ConstraintGraphBuilder(const ModuleName& moduleName, ModulePtr module, TypeArena* arena, NotNull ice, const ScopePtr& globalScope); + ConstraintGraphBuilder(const ModuleName& moduleName, ModulePtr module, TypeArena* arena, NotNull moduleResolver, + NotNull ice, const ScopePtr& globalScope); /** * Fabricates a new free type belonging to a given scope. @@ -82,7 +86,7 @@ struct ConstraintGraphBuilder * @param scope the scope to add the constraint to. * @param cv the constraint variant to add. */ - void addConstraint(const ScopePtr& scope, ConstraintV cv); + void addConstraint(const ScopePtr& scope, const Location& location, ConstraintV cv); /** * Adds a constraint to a given scope. @@ -104,6 +108,7 @@ struct ConstraintGraphBuilder void visit(const ScopePtr& scope, AstStatBlock* block); void visit(const ScopePtr& scope, AstStatLocal* local); void visit(const ScopePtr& scope, AstStatFor* for_); + void visit(const ScopePtr& scope, AstStatForIn* forIn); void visit(const ScopePtr& scope, AstStatWhile* while_); void visit(const ScopePtr& scope, AstStatRepeat* repeat); void visit(const ScopePtr& scope, AstStatLocalFunction* function); @@ -117,8 +122,6 @@ struct ConstraintGraphBuilder void visit(const ScopePtr& scope, AstStatDeclareClass* declareClass); void visit(const ScopePtr& scope, AstStatDeclareFunction* declareFunction); - TypePackId checkExprList(const ScopePtr& scope, const AstArray& exprs); - TypePackId checkPack(const ScopePtr& scope, AstArray exprs); TypePackId checkPack(const ScopePtr& scope, AstExpr* expr); diff --git a/Analysis/include/Luau/ConstraintSolver.h b/Analysis/include/Luau/ConstraintSolver.h index a270ec99..002aa947 100644 --- a/Analysis/include/Luau/ConstraintSolver.h +++ b/Analysis/include/Luau/ConstraintSolver.h @@ -17,6 +17,8 @@ namespace Luau // never dereference this pointer. using BlockedConstraintId = const void*; +struct ModuleResolver; + struct InstantiationSignature { TypeFun fn; @@ -42,6 +44,7 @@ struct ConstraintSolver // The entire set of constraints that the solver is trying to resolve. std::vector> constraints; NotNull rootScope; + ModuleName currentModuleName; // Constraints that the solver has generated, rather than sourcing from the // scope tree. @@ -63,9 +66,13 @@ struct ConstraintSolver // Recorded errors that take place within the solver. ErrorVec errors; + NotNull moduleResolver; + std::vector requireCycles; + ConstraintSolverLogger logger; - explicit ConstraintSolver(TypeArena* arena, NotNull rootScope); + explicit ConstraintSolver(TypeArena* arena, NotNull rootScope, ModuleName moduleName, NotNull moduleResolver, + std::vector requireCycles); /** * Attempts to dispatch all pending constraints and reach a type solution @@ -86,8 +93,17 @@ struct ConstraintSolver bool tryDispatch(const InstantiationConstraint& c, NotNull constraint, bool force); bool tryDispatch(const UnaryConstraint& c, NotNull constraint, bool force); bool tryDispatch(const BinaryConstraint& c, NotNull constraint, bool force); + bool tryDispatch(const IterableConstraint& c, NotNull constraint, bool force); bool tryDispatch(const NameConstraint& c, NotNull constraint); bool tryDispatch(const TypeAliasExpansionConstraint& c, NotNull constraint); + bool tryDispatch(const FunctionCallConstraint& c, NotNull constraint); + + // for a, ... in some_table do + bool tryDispatchIterableTable(TypeId iteratorTy, const IterableConstraint& c, NotNull constraint, bool force); + + // for a, ... in next_function, t, ... do + bool tryDispatchIterableFunction( + TypeId nextTy, TypeId tableTy, TypeId firstIndexTy, const IterableConstraint& c, NotNull constraint, bool force); void block(NotNull target, NotNull constraint); /** @@ -108,6 +124,11 @@ struct ConstraintSolver */ bool isBlocked(TypeId ty); + /** + * @returns true if the TypePackId is in a blocked state. + */ + bool isBlocked(TypePackId tp); + /** * Returns whether the constraint is blocked on anything. * @param constraint the constraint to check. @@ -133,10 +154,22 @@ struct ConstraintSolver /** Pushes a new solver constraint to the solver. * @param cv the body of the constraint. **/ - void pushConstraint(ConstraintV cv, NotNull scope); + void pushConstraint(NotNull scope, const Location& location, ConstraintV cv); + + /** + * Attempts to resolve a module from its module information. Returns the + * module-level return type of the module, or the error type if one cannot + * be found. Reports errors to the solver if the module cannot be found or + * the require is illegal. + * @param module the module information to look up. + * @param location the location where the require is taking place; used for + * error locations. + **/ + TypeId resolveModule(const ModuleInfo& module, const Location& location); void reportError(TypeErrorData&& data, const Location& location); void reportError(TypeError e); + private: /** * Marks a constraint as being blocked on a type or type pack. The constraint @@ -154,6 +187,8 @@ private: * @param progressed the type or type pack pointer that has progressed. **/ void unblock_(BlockedConstraintId progressed); + + ToStringOptions opts; }; void dump(NotNull rootScope, struct ToStringOptions& opts); diff --git a/Analysis/include/Luau/ConstraintSolverLogger.h b/Analysis/include/Luau/ConstraintSolverLogger.h index 55170c42..65aa9a7e 100644 --- a/Analysis/include/Luau/ConstraintSolverLogger.h +++ b/Analysis/include/Luau/ConstraintSolverLogger.h @@ -16,7 +16,8 @@ struct ConstraintSolverLogger { std::string compileOutput(); void captureBoundarySnapshot(const Scope* rootScope, std::vector>& unsolvedConstraints); - void prepareStepSnapshot(const Scope* rootScope, NotNull current, std::vector>& unsolvedConstraints, bool force); + void prepareStepSnapshot( + const Scope* rootScope, NotNull current, std::vector>& unsolvedConstraints, bool force); void commitPreparedStepSnapshot(); private: diff --git a/Analysis/include/Luau/Frontend.h b/Analysis/include/Luau/Frontend.h index f8da3273..55612689 100644 --- a/Analysis/include/Luau/Frontend.h +++ b/Analysis/include/Luau/Frontend.h @@ -157,7 +157,7 @@ struct Frontend ScopePtr getGlobalScope(); private: - ModulePtr check(const SourceModule& sourceModule, Mode mode, const ScopePtr& environmentScope); + ModulePtr check(const SourceModule& sourceModule, Mode mode, const ScopePtr& environmentScope, std::vector requireCycles); std::pair getSourceNode(CheckResult& checkResult, const ModuleName& name); SourceModule parse(const ModuleName& name, std::string_view src, const ParseOptions& parseOptions); diff --git a/Analysis/include/Luau/Scope.h b/Analysis/include/Luau/Scope.h index dc123335..b7569d8e 100644 --- a/Analysis/include/Luau/Scope.h +++ b/Analysis/include/Luau/Scope.h @@ -40,6 +40,9 @@ struct Scope std::optional varargPack; // All constraints belonging to this scope. std::vector constraints; + // Constraints belonging to this scope that are queued manually by other + // constraints. + std::vector unqueuedConstraints; TypeLevel level; diff --git a/Analysis/include/Luau/ToString.h b/Analysis/include/Luau/ToString.h index eabbc2be..61e07e9f 100644 --- a/Analysis/include/Luau/ToString.h +++ b/Analysis/include/Luau/ToString.h @@ -92,7 +92,6 @@ inline std::string toString(const Constraint& c) return toString(c, ToStringOptions{}); } - std::string toString(const TypeVar& tv, ToStringOptions& opts); std::string toString(const TypePackVar& tp, ToStringOptions& opts); diff --git a/Analysis/include/Luau/TypeArena.h b/Analysis/include/Luau/TypeArena.h index be36f19c..decc8c59 100644 --- a/Analysis/include/Luau/TypeArena.h +++ b/Analysis/include/Luau/TypeArena.h @@ -29,9 +29,10 @@ struct TypeArena TypeId addTV(TypeVar&& tv); TypeId freshType(TypeLevel level); + TypeId freshType(Scope* scope); TypePackId addTypePack(std::initializer_list types); - TypePackId addTypePack(std::vector types); + TypePackId addTypePack(std::vector types, std::optional tail = {}); TypePackId addTypePack(TypePack pack); TypePackId addTypePack(TypePackVar pack); diff --git a/Analysis/include/Luau/TypeInfer.h b/Analysis/include/Luau/TypeInfer.h index e253eddf..0b427946 100644 --- a/Analysis/include/Luau/TypeInfer.h +++ b/Analysis/include/Luau/TypeInfer.h @@ -166,7 +166,8 @@ struct TypeChecker */ bool unify(TypeId subTy, TypeId superTy, const ScopePtr& scope, const Location& location); bool unify(TypeId subTy, TypeId superTy, const ScopePtr& scope, const Location& location, const UnifierOptions& options); - bool unify(TypePackId subTy, TypePackId superTy, const ScopePtr& scope, const Location& location, CountMismatch::Context ctx = CountMismatch::Context::Arg); + bool unify(TypePackId subTy, TypePackId superTy, const ScopePtr& scope, const Location& location, + CountMismatch::Context ctx = CountMismatch::Context::Arg); /** Attempt to unify the types. * If this fails, and the subTy type can be instantiated, do so and try unification again. diff --git a/Analysis/include/Luau/TypePack.h b/Analysis/include/Luau/TypePack.h index b17003b1..8269230b 100644 --- a/Analysis/include/Luau/TypePack.h +++ b/Analysis/include/Luau/TypePack.h @@ -15,6 +15,7 @@ struct TypeArena; struct TypePack; struct VariadicTypePack; +struct BlockedTypePack; struct TypePackVar; @@ -24,7 +25,7 @@ using TypePackId = const TypePackVar*; using FreeTypePack = Unifiable::Free; using BoundTypePack = Unifiable::Bound; using GenericTypePack = Unifiable::Generic; -using TypePackVariant = Unifiable::Variant; +using TypePackVariant = Unifiable::Variant; /* A TypePack is a rope-like string of TypeIds. We use this structure to encode * notions like packs of unknown length and packs of any length, as well as more @@ -43,6 +44,17 @@ struct VariadicTypePack bool hidden = false; // if true, we don't display this when toString()ing a pack with this variadic as its tail. }; +/** + * Analogous to a BlockedTypeVar. + */ +struct BlockedTypePack +{ + BlockedTypePack(); + size_t index; + + static size_t nextIndex; +}; + struct TypePackVar { explicit TypePackVar(const TypePackVariant& ty); diff --git a/Analysis/include/Luau/TypeUtils.h b/Analysis/include/Luau/TypeUtils.h index 0aff5a7d..6c611fb2 100644 --- a/Analysis/include/Luau/TypeUtils.h +++ b/Analysis/include/Luau/TypeUtils.h @@ -11,12 +11,16 @@ namespace Luau { +struct TxnLog; + using ScopePtr = std::shared_ptr; std::optional findMetatableEntry(ErrorVec& errors, TypeId type, const std::string& entry, Location location); std::optional findTablePropertyRespectingMeta(ErrorVec& errors, TypeId ty, const std::string& name, Location location); -std::optional getIndexTypeFromType( - const ScopePtr& scope, ErrorVec& errors, TypeArena* arena, TypeId type, const std::string& prop, const Location& location, bool addErrors, - InternalErrorReporter& handle); +std::optional getIndexTypeFromType(const ScopePtr& scope, ErrorVec& errors, TypeArena* arena, TypeId type, const std::string& prop, + const Location& location, bool addErrors, InternalErrorReporter& handle); + +// Returns the minimum and maximum number of types the argument list can accept. +std::pair> getParameterExtents(const TxnLog* log, TypePackId tp); } // namespace Luau diff --git a/Analysis/include/Luau/TypeVar.h b/Analysis/include/Luau/TypeVar.h index 35b37394..e67b3601 100644 --- a/Analysis/include/Luau/TypeVar.h +++ b/Analysis/include/Luau/TypeVar.h @@ -1,11 +1,13 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #pragma once +#include "Luau/Ast.h" #include "Luau/DenseHash.h" #include "Luau/Predicate.h" #include "Luau/Unifiable.h" #include "Luau/Variant.h" #include "Luau/Common.h" +#include "Luau/NotNull.h" #include #include @@ -262,6 +264,8 @@ struct WithPredicate using MagicFunction = std::function>( struct TypeChecker&, const std::shared_ptr&, const class AstExprCall&, WithPredicate)>; +using DcrMagicFunction = std::function, TypePackId, const class AstExprCall*)>; + struct FunctionTypeVar { // Global monomorphic function @@ -287,7 +291,8 @@ struct FunctionTypeVar std::vector> argNames; TypePackId retTypes; std::optional definition; - MagicFunction magicFunction = nullptr; // Function pointer, can be nullptr. + MagicFunction magicFunction = nullptr; // Function pointer, can be nullptr. + DcrMagicFunction dcrMagicFunction = nullptr; // can be nullptr bool hasSelf; Tags tags; bool hasNoGenerics = false; @@ -462,8 +467,9 @@ struct TypeFun */ struct PendingExpansionTypeVar { - PendingExpansionTypeVar(TypeFun fn, std::vector typeArguments, std::vector packArguments); - TypeFun fn; + PendingExpansionTypeVar(std::optional prefix, AstName name, std::vector typeArguments, std::vector packArguments); + std::optional prefix; + AstName name; std::vector typeArguments; std::vector packArguments; size_t index; diff --git a/Analysis/include/Luau/Unifier.h b/Analysis/include/Luau/Unifier.h index 312b0584..c7eb51a6 100644 --- a/Analysis/include/Luau/Unifier.h +++ b/Analysis/include/Luau/Unifier.h @@ -59,7 +59,8 @@ struct Unifier UnifierSharedState& sharedState; - Unifier(TypeArena* types, Mode mode, NotNull scope, const Location& location, Variance variance, UnifierSharedState& sharedState, TxnLog* parentLog = nullptr); + Unifier(TypeArena* types, Mode mode, NotNull scope, const Location& location, Variance variance, UnifierSharedState& sharedState, + TxnLog* parentLog = nullptr); // Test whether the two type vars unify. Never commits the result. ErrorVec canUnify(TypeId subTy, TypeId superTy); diff --git a/Analysis/include/Luau/VisitTypeVar.h b/Analysis/include/Luau/VisitTypeVar.h index 9d7fa9fe..315e5992 100644 --- a/Analysis/include/Luau/VisitTypeVar.h +++ b/Analysis/include/Luau/VisitTypeVar.h @@ -188,6 +188,10 @@ struct GenericTypeVarVisitor { return visit(tp); } + virtual bool visit(TypePackId tp, const BlockedTypePack& btp) + { + return visit(tp); + } void traverse(TypeId ty) { @@ -314,24 +318,6 @@ struct GenericTypeVarVisitor { if (visit(ty, *petv)) { - traverse(petv->fn.type); - - for (const GenericTypeDefinition& p : petv->fn.typeParams) - { - traverse(p.ty); - - if (p.defaultValue) - traverse(*p.defaultValue); - } - - for (const GenericTypePackDefinition& p : petv->fn.typePackParams) - { - traverse(p.tp); - - if (p.defaultValue) - traverse(*p.defaultValue); - } - for (TypeId a : petv->typeArguments) traverse(a); @@ -388,6 +374,9 @@ struct GenericTypeVarVisitor if (res) traverse(pack->ty); } + else if (auto btp = get(tp)) + visit(tp, *btp); + else LUAU_ASSERT(!"GenericTypeVarVisitor::traverse(TypePackId) is not exhaustive!"); diff --git a/Analysis/src/Anyification.cpp b/Analysis/src/Anyification.cpp index b6e58009..abcaba02 100644 --- a/Analysis/src/Anyification.cpp +++ b/Analysis/src/Anyification.cpp @@ -11,15 +11,20 @@ LUAU_FASTFLAG(LuauClassTypeVarsInSubstitution) namespace Luau { -Anyification::Anyification(TypeArena* arena, const ScopePtr& scope, InternalErrorReporter* iceHandler, TypeId anyType, TypePackId anyTypePack) +Anyification::Anyification(TypeArena* arena, NotNull scope, InternalErrorReporter* iceHandler, TypeId anyType, TypePackId anyTypePack) : Substitution(TxnLog::empty(), arena) - , scope(NotNull{scope.get()}) + , scope(scope) , iceHandler(iceHandler) , anyType(anyType) , anyTypePack(anyTypePack) { } +Anyification::Anyification(TypeArena* arena, const ScopePtr& scope, InternalErrorReporter* iceHandler, TypeId anyType, TypePackId anyTypePack) + : Anyification(arena, NotNull{scope.get()}, iceHandler, anyType, anyTypePack) +{ +} + bool Anyification::isDirty(TypeId ty) { if (ty->persistent) @@ -93,4 +98,4 @@ bool Anyification::ignoreChildren(TypePackId ty) return ty->persistent; } -} +} // namespace Luau diff --git a/Analysis/src/Autocomplete.cpp b/Analysis/src/Autocomplete.cpp index 5c484899..378a1cb7 100644 --- a/Analysis/src/Autocomplete.cpp +++ b/Analysis/src/Autocomplete.cpp @@ -1215,8 +1215,8 @@ static bool autocompleteIfElseExpression( } } -static AutocompleteContext autocompleteExpression(const SourceModule& sourceModule, const Module& module, const TypeChecker& typeChecker, TypeArena* typeArena, - const std::vector& ancestry, Position position, AutocompleteEntryMap& result) +static AutocompleteContext autocompleteExpression(const SourceModule& sourceModule, const Module& module, const TypeChecker& typeChecker, + TypeArena* typeArena, const std::vector& ancestry, Position position, AutocompleteEntryMap& result) { LUAU_ASSERT(!ancestry.empty()); @@ -1422,8 +1422,8 @@ static AutocompleteResult autocomplete(const SourceModule& sourceModule, const M PropIndexType indexType = indexName->op == ':' ? PropIndexType::Colon : PropIndexType::Point; if (!FFlag::LuauSelfCallAutocompleteFix3 && isString(ty)) - return { - autocompleteProps(*module, typeArena, typeChecker.globalScope->bindings[AstName{"string"}].typeId, indexType, ancestry), ancestry, AutocompleteContext::Property}; + return {autocompleteProps(*module, typeArena, typeChecker.globalScope->bindings[AstName{"string"}].typeId, indexType, ancestry), ancestry, + AutocompleteContext::Property}; else return {autocompleteProps(*module, typeArena, ty, indexType, ancestry), ancestry, AutocompleteContext::Property}; } @@ -1522,8 +1522,8 @@ static AutocompleteResult autocomplete(const SourceModule& sourceModule, const M else if (AstStatIf* statIf = node->as(); statIf && !statIf->elseLocation.has_value()) { - return { - {{"else", AutocompleteEntry{AutocompleteEntryKind::Keyword}}, {"elseif", AutocompleteEntry{AutocompleteEntryKind::Keyword}}}, ancestry, AutocompleteContext::Keyword}; + return {{{"else", AutocompleteEntry{AutocompleteEntryKind::Keyword}}, {"elseif", AutocompleteEntry{AutocompleteEntryKind::Keyword}}}, + ancestry, AutocompleteContext::Keyword}; } else if (AstStatIf* statIf = parent->as(); statIf && node->is()) { diff --git a/Analysis/src/BuiltinDefinitions.cpp b/Analysis/src/BuiltinDefinitions.cpp index 826179b3..e011eaa5 100644 --- a/Analysis/src/BuiltinDefinitions.cpp +++ b/Analysis/src/BuiltinDefinitions.cpp @@ -5,6 +5,7 @@ #include "Luau/Symbol.h" #include "Luau/Common.h" #include "Luau/ToString.h" +#include "Luau/ConstraintSolver.h" #include @@ -32,6 +33,8 @@ static std::optional> magicFunctionPack( static std::optional> magicFunctionRequire( TypeChecker& typechecker, const ScopePtr& scope, const AstExprCall& expr, WithPredicate withPredicate); +static bool dcrMagicFunctionRequire(NotNull solver, TypePackId result, const AstExprCall* expr); + TypeId makeUnion(TypeArena& arena, std::vector&& types) { return arena.addType(UnionTypeVar{std::move(types)}); @@ -105,6 +108,14 @@ void attachMagicFunction(TypeId ty, MagicFunction fn) LUAU_ASSERT(!"Got a non functional type"); } +void attachDcrMagicFunction(TypeId ty, DcrMagicFunction fn) +{ + if (auto ftv = getMutable(ty)) + ftv->dcrMagicFunction = fn; + else + LUAU_ASSERT(!"Got a non functional type"); +} + Property makeProperty(TypeId ty, std::optional documentationSymbol) { return { @@ -263,6 +274,7 @@ void registerBuiltinTypes(TypeChecker& typeChecker) } attachMagicFunction(getGlobalBinding(typeChecker, "require"), magicFunctionRequire); + attachDcrMagicFunction(getGlobalBinding(typeChecker, "require"), dcrMagicFunctionRequire); } static std::optional> magicFunctionSelect( @@ -509,4 +521,49 @@ static std::optional> magicFunctionRequire( return std::nullopt; } +static bool checkRequirePathDcr(NotNull solver, AstExpr* expr) +{ + // require(foo.parent.bar) will technically work, but it depends on legacy goop that + // Luau does not and could not support without a bunch of work. It's deprecated anyway, so + // we'll warn here if we see it. + bool good = true; + AstExprIndexName* indexExpr = expr->as(); + + while (indexExpr) + { + if (indexExpr->index == "parent") + { + solver->reportError(DeprecatedApiUsed{"parent", "Parent"}, indexExpr->indexLocation); + good = false; + } + + indexExpr = indexExpr->expr->as(); + } + + return good; +} + +static bool dcrMagicFunctionRequire(NotNull solver, TypePackId result, const AstExprCall* expr) +{ + if (expr->args.size != 1) + { + solver->reportError(GenericError{"require takes 1 argument"}, expr->location); + return false; + } + + if (!checkRequirePathDcr(solver, expr->args.data[0])) + return false; + + if (auto moduleInfo = solver->moduleResolver->resolveModuleInfo(solver->currentModuleName, *expr)) + { + TypeId moduleType = solver->resolveModule(*moduleInfo, expr->location); + TypePackId moduleResult = solver->arena->addTypePack({moduleType}); + asMutable(result)->ty.emplace(moduleResult); + + return true; + } + + return false; +} + } // namespace Luau diff --git a/Analysis/src/Clone.cpp b/Analysis/src/Clone.cpp index 2e04b527..7048d201 100644 --- a/Analysis/src/Clone.cpp +++ b/Analysis/src/Clone.cpp @@ -102,6 +102,11 @@ struct TypePackCloner defaultClone(t); } + void operator()(const BlockedTypePack& t) + { + defaultClone(t); + } + // While we are a-cloning, we can flatten out bound TypeVars and make things a bit tighter. // We just need to be sure that we rewrite pointers both to the binder and the bindee to the same pointer. void operator()(const Unifiable::Bound& t) @@ -170,7 +175,7 @@ void TypeCloner::operator()(const BlockedTypeVar& t) void TypeCloner::operator()(const PendingExpansionTypeVar& t) { - TypeId res = dest.addType(PendingExpansionTypeVar{t.fn, t.typeArguments, t.packArguments}); + TypeId res = dest.addType(PendingExpansionTypeVar{t.prefix, t.name, t.typeArguments, t.packArguments}); PendingExpansionTypeVar* petv = getMutable(res); LUAU_ASSERT(petv); @@ -184,32 +189,6 @@ void TypeCloner::operator()(const PendingExpansionTypeVar& t) for (TypePackId arg : t.packArguments) packArguments.push_back(clone(arg, dest, cloneState)); - TypeFun fn; - fn.type = clone(t.fn.type, dest, cloneState); - - for (const GenericTypeDefinition& param : t.fn.typeParams) - { - TypeId ty = clone(param.ty, dest, cloneState); - std::optional defaultValue = param.defaultValue; - - if (defaultValue) - defaultValue = clone(*defaultValue, dest, cloneState); - - fn.typeParams.push_back(GenericTypeDefinition{ty, defaultValue}); - } - - for (const GenericTypePackDefinition& param : t.fn.typePackParams) - { - TypePackId tp = clone(param.tp, dest, cloneState); - std::optional defaultValue = param.defaultValue; - - if (defaultValue) - defaultValue = clone(*defaultValue, dest, cloneState); - - fn.typePackParams.push_back(GenericTypePackDefinition{tp, defaultValue}); - } - - petv->fn = std::move(fn); petv->typeArguments = std::move(typeArguments); petv->packArguments = std::move(packArguments); } @@ -461,6 +440,7 @@ TypeId shallowClone(TypeId ty, TypeArena& dest, const TxnLog* log, bool alwaysCl clone.generics = ftv->generics; clone.genericPacks = ftv->genericPacks; clone.magicFunction = ftv->magicFunction; + clone.dcrMagicFunction = ftv->dcrMagicFunction; clone.tags = ftv->tags; clone.argNames = ftv->argNames; result = dest.addType(std::move(clone)); @@ -502,7 +482,7 @@ TypeId shallowClone(TypeId ty, TypeArena& dest, const TxnLog* log, bool alwaysCl } else if (const PendingExpansionTypeVar* petv = get(ty)) { - PendingExpansionTypeVar clone{petv->fn, petv->typeArguments, petv->packArguments}; + PendingExpansionTypeVar clone{petv->prefix, petv->name, petv->typeArguments, petv->packArguments}; result = dest.addType(std::move(clone)); } else if (const ClassTypeVar* ctv = get(ty); FFlag::LuauClonePublicInterfaceLess && ctv && alwaysClone) diff --git a/Analysis/src/Constraint.cpp b/Analysis/src/Constraint.cpp index d272c027..3a6417dc 100644 --- a/Analysis/src/Constraint.cpp +++ b/Analysis/src/Constraint.cpp @@ -5,9 +5,10 @@ namespace Luau { -Constraint::Constraint(ConstraintV&& c, NotNull scope) - : c(std::move(c)) - , scope(scope) +Constraint::Constraint(NotNull scope, const Location& location, ConstraintV&& c) + : scope(scope) + , location(location) + , c(std::move(c)) { } diff --git a/Analysis/src/ConstraintGraphBuilder.cpp b/Analysis/src/ConstraintGraphBuilder.cpp index 8f994740..9fabc528 100644 --- a/Analysis/src/ConstraintGraphBuilder.cpp +++ b/Analysis/src/ConstraintGraphBuilder.cpp @@ -4,6 +4,7 @@ #include "Luau/Ast.h" #include "Luau/Common.h" #include "Luau/Constraint.h" +#include "Luau/ModuleResolver.h" #include "Luau/RecursionCounter.h" #include "Luau/ToString.h" @@ -16,13 +17,31 @@ namespace Luau const AstStat* getFallthrough(const AstStat* node); // TypeInfer.cpp -ConstraintGraphBuilder::ConstraintGraphBuilder( - const ModuleName& moduleName, ModulePtr module, TypeArena* arena, NotNull ice, const ScopePtr& globalScope) +static std::optional matchRequire(const AstExprCall& call) +{ + const char* require = "require"; + + if (call.args.size != 1) + return std::nullopt; + + const AstExprGlobal* funcAsGlobal = call.func->as(); + if (!funcAsGlobal || funcAsGlobal->name != require) + return std::nullopt; + + if (call.args.size != 1) + return std::nullopt; + + return call.args.data[0]; +} + +ConstraintGraphBuilder::ConstraintGraphBuilder(const ModuleName& moduleName, ModulePtr module, TypeArena* arena, + NotNull moduleResolver, NotNull ice, const ScopePtr& globalScope) : moduleName(moduleName) , module(module) , singletonTypes(getSingletonTypes()) , arena(arena) , rootScope(nullptr) + , moduleResolver(moduleResolver) , ice(ice) , globalScope(globalScope) { @@ -54,9 +73,9 @@ ScopePtr ConstraintGraphBuilder::childScope(AstNode* node, const ScopePtr& paren return scope; } -void ConstraintGraphBuilder::addConstraint(const ScopePtr& scope, ConstraintV cv) +void ConstraintGraphBuilder::addConstraint(const ScopePtr& scope, const Location& location, ConstraintV cv) { - scope->constraints.emplace_back(new Constraint{std::move(cv), NotNull{scope.get()}}); + scope->constraints.emplace_back(new Constraint{NotNull{scope.get()}, location, std::move(cv)}); } void ConstraintGraphBuilder::addConstraint(const ScopePtr& scope, std::unique_ptr c) @@ -77,13 +96,6 @@ void ConstraintGraphBuilder::visit(AstStatBlock* block) prepopulateGlobalScope(scope, block); - // TODO: We should share the global scope. - rootScope->privateTypeBindings["nil"] = TypeFun{singletonTypes.nilType}; - rootScope->privateTypeBindings["number"] = TypeFun{singletonTypes.numberType}; - rootScope->privateTypeBindings["string"] = TypeFun{singletonTypes.stringType}; - rootScope->privateTypeBindings["boolean"] = TypeFun{singletonTypes.booleanType}; - rootScope->privateTypeBindings["thread"] = TypeFun{singletonTypes.threadType}; - visitBlockWithoutChildScope(scope, block); } @@ -158,6 +170,8 @@ void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStat* stat) visit(scope, s); else if (auto s = stat->as()) visit(scope, s); + else if (auto s = stat->as()) + visit(scope, s); else if (auto s = stat->as()) visit(scope, s); else if (auto s = stat->as()) @@ -201,7 +215,7 @@ void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatLocal* local) { location = local->annotation->location; TypeId annotation = resolveType(scope, local->annotation, /* topLevel */ true); - addConstraint(scope, SubtypeConstraint{ty, annotation}); + addConstraint(scope, location, SubtypeConstraint{ty, annotation}); } varTypes.push_back(ty); @@ -225,14 +239,38 @@ void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatLocal* local) { std::vector tailValues{varTypes.begin() + i, varTypes.end()}; TypePackId tailPack = arena->addTypePack(std::move(tailValues)); - addConstraint(scope, PackSubtypeConstraint{exprPack, tailPack}); + addConstraint(scope, local->location, PackSubtypeConstraint{exprPack, tailPack}); } } else { TypeId exprType = check(scope, value); if (i < varTypes.size()) - addConstraint(scope, SubtypeConstraint{varTypes[i], exprType}); + addConstraint(scope, local->location, SubtypeConstraint{varTypes[i], exprType}); + } + } + + if (local->values.size > 0) + { + // To correctly handle 'require', we need to import the exported type bindings into the variable 'namespace'. + for (size_t i = 0; i < local->values.size && i < local->vars.size; ++i) + { + const AstExprCall* call = local->values.data[i]->as(); + if (!call) + continue; + + if (auto maybeRequire = matchRequire(*call)) + { + AstExpr* require = *maybeRequire; + + if (auto moduleInfo = moduleResolver->resolveModuleInfo(moduleName, *require)) + { + const Name name{local->vars.data[i]->name.value}; + + if (ModulePtr module = moduleResolver->getModule(moduleInfo->name)) + scope->importedTypeBindings[name] = module->getModuleScope()->exportedTypeBindings; + } + } } } } @@ -244,7 +282,7 @@ void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatFor* for_) return; TypeId t = check(scope, expr); - addConstraint(scope, SubtypeConstraint{t, singletonTypes.numberType}); + addConstraint(scope, expr->location, SubtypeConstraint{t, singletonTypes.numberType}); }; checkNumber(for_->from); @@ -257,6 +295,29 @@ void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatFor* for_) visit(forScope, for_->body); } +void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatForIn* forIn) +{ + ScopePtr loopScope = childScope(forIn, scope); + + TypePackId iterator = checkPack(scope, forIn->values); + + std::vector variableTypes; + variableTypes.reserve(forIn->vars.size); + for (AstLocal* var : forIn->vars) + { + TypeId ty = freshType(loopScope); + loopScope->bindings[var] = Binding{ty, var->location}; + variableTypes.push_back(ty); + } + + // It is always ok to provide too few variables, so we give this pack a free tail. + TypePackId variablePack = arena->addTypePack(std::move(variableTypes), arena->addTypePack(FreeTypePack{loopScope.get()})); + + addConstraint(loopScope, getLocation(forIn->values), IterableConstraint{iterator, variablePack}); + + visit(loopScope, forIn->body); +} + void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatWhile* while_) { check(scope, while_->condition); @@ -284,6 +345,9 @@ void addConstraints(Constraint* constraint, NotNull scope) for (const auto& c : scope->constraints) constraint->dependencies.push_back(NotNull{c.get()}); + for (const auto& c : scope->unqueuedConstraints) + constraint->dependencies.push_back(NotNull{c.get()}); + for (NotNull childScope : scope->children) addConstraints(constraint, childScope); } @@ -308,7 +372,8 @@ void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatLocalFunction* checkFunctionBody(sig.bodyScope, function->func); NotNull constraintScope{sig.signatureScope ? sig.signatureScope.get() : sig.bodyScope.get()}; - std::unique_ptr c = std::make_unique(GeneralizationConstraint{functionType, sig.signature}, constraintScope); + std::unique_ptr c = + std::make_unique(constraintScope, function->name->location, GeneralizationConstraint{functionType, sig.signature}); addConstraints(c.get(), NotNull(sig.bodyScope.get())); addConstraint(scope, std::move(c)); @@ -366,7 +431,7 @@ void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatFunction* funct prop.type = functionType; prop.location = function->name->location; - addConstraint(scope, SubtypeConstraint{containingTableType, prospectiveTableType}); + addConstraint(scope, indexName->location, SubtypeConstraint{containingTableType, prospectiveTableType}); } else if (AstExprError* err = function->name->as()) { @@ -378,7 +443,8 @@ void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatFunction* funct checkFunctionBody(sig.bodyScope, function->func); NotNull constraintScope{sig.signatureScope ? sig.signatureScope.get() : sig.bodyScope.get()}; - std::unique_ptr c = std::make_unique(GeneralizationConstraint{functionType, sig.signature}, constraintScope); + std::unique_ptr c = + std::make_unique(constraintScope, function->name->location, GeneralizationConstraint{functionType, sig.signature}); addConstraints(c.get(), NotNull(sig.bodyScope.get())); addConstraint(scope, std::move(c)); @@ -387,7 +453,7 @@ void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatFunction* funct void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatReturn* ret) { TypePackId exprTypes = checkPack(scope, ret->list); - addConstraint(scope, PackSubtypeConstraint{exprTypes, scope->returnType}); + addConstraint(scope, ret->location, PackSubtypeConstraint{exprTypes, scope->returnType}); } void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatBlock* block) @@ -399,10 +465,10 @@ void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatBlock* block) void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatAssign* assign) { - TypePackId varPackId = checkExprList(scope, assign->vars); + TypePackId varPackId = checkPack(scope, assign->vars); TypePackId valuePack = checkPack(scope, assign->values); - addConstraint(scope, PackSubtypeConstraint{valuePack, varPackId}); + addConstraint(scope, assign->location, PackSubtypeConstraint{valuePack, varPackId}); } void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatCompoundAssign* assign) @@ -435,8 +501,6 @@ void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatIf* ifStatement void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatTypeAlias* alias) { - // TODO: Exported type aliases - auto bindingIt = scope->privateTypeBindings.find(alias->name.value); ScopePtr* defnIt = astTypeAliasDefiningScopes.find(alias); // These will be undefined if the alias was a duplicate definition, in which @@ -449,6 +513,12 @@ void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatTypeAlias* alia ScopePtr resolvingScope = *defnIt; TypeId ty = resolveType(resolvingScope, alias->type, /* topLevel */ true); + if (alias->exported) + { + Name typeName(alias->name.value); + scope->exportedTypeBindings[typeName] = TypeFun{ty}; + } + LUAU_ASSERT(get(bindingIt->second.type)); // Rather than using a subtype constraint, we instead directly bind @@ -457,7 +527,7 @@ void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatTypeAlias* alia // bind the free alias type to an unrelated type, causing havoc. asMutable(bindingIt->second.type)->ty.emplace(ty); - addConstraint(scope, NameConstraint{ty, alias->name.value}); + addConstraint(scope, alias->location, NameConstraint{ty, alias->name.value}); } void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatDeclareGlobal* global) @@ -615,44 +685,22 @@ void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatDeclareFunction TypePackId ConstraintGraphBuilder::checkPack(const ScopePtr& scope, AstArray exprs) { - if (exprs.size == 0) - return arena->addTypePack({}); - - std::vector types; - TypePackId last = nullptr; - - for (size_t i = 0; i < exprs.size; ++i) - { - if (i < exprs.size - 1) - types.push_back(check(scope, exprs.data[i])); - else - last = checkPack(scope, exprs.data[i]); - } - - LUAU_ASSERT(last != nullptr); - - return arena->addTypePack(TypePack{std::move(types), last}); -} - -TypePackId ConstraintGraphBuilder::checkExprList(const ScopePtr& scope, const AstArray& exprs) -{ - TypePackId result = arena->addTypePack({}); - TypePack* resultPack = getMutable(result); - LUAU_ASSERT(resultPack); + std::vector head; + std::optional tail; for (size_t i = 0; i < exprs.size; ++i) { AstExpr* expr = exprs.data[i]; if (i < exprs.size - 1) - resultPack->head.push_back(check(scope, expr)); + head.push_back(check(scope, expr)); else - resultPack->tail = checkPack(scope, expr); + tail = checkPack(scope, expr); } - if (resultPack->head.empty() && resultPack->tail) - return *resultPack->tail; + if (head.empty() && tail) + return *tail; else - return result; + return arena->addTypePack(TypePack{std::move(head), tail}); } TypePackId ConstraintGraphBuilder::checkPack(const ScopePtr& scope, AstExpr* expr) @@ -683,13 +731,26 @@ TypePackId ConstraintGraphBuilder::checkPack(const ScopePtr& scope, AstExpr* exp astOriginalCallTypes[call->func] = fnType; TypeId instantiatedType = arena->addType(BlockedTypeVar{}); - addConstraint(scope, InstantiationConstraint{instantiatedType, fnType}); - - TypePackId rets = freshTypePack(scope); + TypePackId rets = arena->addTypePack(BlockedTypePack{}); FunctionTypeVar ftv(arena->addTypePack(TypePack{args, {}}), rets); TypeId inferredFnType = arena->addType(ftv); - addConstraint(scope, SubtypeConstraint{inferredFnType, instantiatedType}); + scope->unqueuedConstraints.push_back( + std::make_unique(NotNull{scope.get()}, call->func->location, InstantiationConstraint{instantiatedType, fnType})); + NotNull ic(scope->unqueuedConstraints.back().get()); + + scope->unqueuedConstraints.push_back( + std::make_unique(NotNull{scope.get()}, call->func->location, SubtypeConstraint{inferredFnType, instantiatedType})); + NotNull sc(scope->unqueuedConstraints.back().get()); + + addConstraint(scope, call->func->location, + FunctionCallConstraint{ + {ic, sc}, + fnType, + rets, + call, + }); + result = rets; } else if (AstExprVarargs* varargs = expr->as()) @@ -805,7 +866,7 @@ TypeId ConstraintGraphBuilder::check(const ScopePtr& scope, AstExprIndexName* in TypeId expectedTableType = arena->addType(std::move(ttv)); - addConstraint(scope, SubtypeConstraint{obj, expectedTableType}); + addConstraint(scope, indexName->expr->location, SubtypeConstraint{obj, expectedTableType}); return result; } @@ -820,7 +881,7 @@ TypeId ConstraintGraphBuilder::check(const ScopePtr& scope, AstExprIndexExpr* in TableIndexer indexer{indexType, result}; TypeId tableType = arena->addType(TableTypeVar{TableTypeVar::Props{}, TableIndexer{indexType, result}, TypeLevel{}, TableState::Free}); - addConstraint(scope, SubtypeConstraint{obj, tableType}); + addConstraint(scope, indexExpr->expr->location, SubtypeConstraint{obj, tableType}); return result; } @@ -834,7 +895,7 @@ TypeId ConstraintGraphBuilder::check(const ScopePtr& scope, AstExprUnary* unary) case AstExprUnary::Minus: { TypeId resultType = arena->addType(BlockedTypeVar{}); - addConstraint(scope, UnaryConstraint{AstExprUnary::Minus, operandType, resultType}); + addConstraint(scope, unary->location, UnaryConstraint{AstExprUnary::Minus, operandType, resultType}); return resultType; } default: @@ -853,19 +914,19 @@ TypeId ConstraintGraphBuilder::check(const ScopePtr& scope, AstExprBinary* binar { case AstExprBinary::Or: { - addConstraint(scope, SubtypeConstraint{leftType, rightType}); + addConstraint(scope, binary->location, SubtypeConstraint{leftType, rightType}); return leftType; } case AstExprBinary::Add: { TypeId resultType = arena->addType(BlockedTypeVar{}); - addConstraint(scope, BinaryConstraint{AstExprBinary::Add, leftType, rightType, resultType}); + addConstraint(scope, binary->location, BinaryConstraint{AstExprBinary::Add, leftType, rightType, resultType}); return resultType; } case AstExprBinary::Sub: { TypeId resultType = arena->addType(BlockedTypeVar{}); - addConstraint(scope, BinaryConstraint{AstExprBinary::Sub, leftType, rightType, resultType}); + addConstraint(scope, binary->location, BinaryConstraint{AstExprBinary::Sub, leftType, rightType, resultType}); return resultType; } default: @@ -886,8 +947,8 @@ TypeId ConstraintGraphBuilder::check(const ScopePtr& scope, AstExprIfElse* ifEls if (ifElse->hasElse) { TypeId resultType = arena->addType(BlockedTypeVar{}); - addConstraint(scope, SubtypeConstraint{thenType, resultType}); - addConstraint(scope, SubtypeConstraint{elseType, resultType}); + addConstraint(scope, ifElse->trueExpr->location, SubtypeConstraint{thenType, resultType}); + addConstraint(scope, ifElse->falseExpr->location, SubtypeConstraint{elseType, resultType}); return resultType; } @@ -906,7 +967,7 @@ TypeId ConstraintGraphBuilder::checkExprTable(const ScopePtr& scope, AstExprTabl TableTypeVar* ttv = getMutable(ty); LUAU_ASSERT(ttv); - auto createIndexer = [this, scope, ttv](TypeId currentIndexType, TypeId currentResultType) { + auto createIndexer = [this, scope, ttv](const Location& location, TypeId currentIndexType, TypeId currentResultType) { if (!ttv->indexer) { TypeId indexType = this->freshType(scope); @@ -914,8 +975,8 @@ TypeId ConstraintGraphBuilder::checkExprTable(const ScopePtr& scope, AstExprTabl ttv->indexer = TableIndexer{indexType, resultType}; } - addConstraint(scope, SubtypeConstraint{ttv->indexer->indexType, currentIndexType}); - addConstraint(scope, SubtypeConstraint{ttv->indexer->indexResultType, currentResultType}); + addConstraint(scope, location, SubtypeConstraint{ttv->indexer->indexType, currentIndexType}); + addConstraint(scope, location, SubtypeConstraint{ttv->indexer->indexResultType, currentResultType}); }; for (const AstExprTable::Item& item : expr->items) @@ -937,13 +998,15 @@ TypeId ConstraintGraphBuilder::checkExprTable(const ScopePtr& scope, AstExprTabl } else { - createIndexer(keyTy, itemTy); + createIndexer(item.key->location, keyTy, itemTy); } } else { TypeId numberType = singletonTypes.numberType; - createIndexer(numberType, itemTy); + // FIXME? The location isn't quite right here. Not sure what is + // right. + createIndexer(item.value->location, numberType, itemTy); } } @@ -1008,7 +1071,7 @@ ConstraintGraphBuilder::FunctionSignature ConstraintGraphBuilder::checkFunctionS if (fn->returnAnnotation) { TypePackId annotatedRetType = resolveTypePack(signatureScope, *fn->returnAnnotation); - addConstraint(signatureScope, PackSubtypeConstraint{returnType, annotatedRetType}); + addConstraint(signatureScope, getLocation(*fn->returnAnnotation), PackSubtypeConstraint{returnType, annotatedRetType}); } std::vector argTypes; @@ -1022,7 +1085,7 @@ ConstraintGraphBuilder::FunctionSignature ConstraintGraphBuilder::checkFunctionS if (local->annotation) { TypeId argAnnotation = resolveType(signatureScope, local->annotation, /* topLevel */ true); - addConstraint(signatureScope, SubtypeConstraint{t, argAnnotation}); + addConstraint(signatureScope, local->annotation->location, SubtypeConstraint{t, argAnnotation}); } } @@ -1056,7 +1119,7 @@ void ConstraintGraphBuilder::checkFunctionBody(const ScopePtr& scope, AstExprFun if (nullptr != getFallthrough(fn->body)) { TypePackId empty = arena->addTypePack({}); // TODO we could have CSG retain one of these forever - addConstraint(scope, PackSubtypeConstraint{scope->returnType, empty}); + addConstraint(scope, fn->location, PackSubtypeConstraint{scope->returnType, empty}); } } @@ -1066,16 +1129,13 @@ TypeId ConstraintGraphBuilder::resolveType(const ScopePtr& scope, AstType* ty, b if (auto ref = ty->as()) { - // TODO: Support imported types w/ require tracing. - LUAU_ASSERT(!ref->prefix); - std::optional alias = scope->lookupType(ref->name.value); - if (alias.has_value()) + if (alias.has_value() || ref->prefix.has_value()) { // If the alias is not generic, we don't need to set up a blocked // type and an instantiation constraint. - if (alias->typeParams.empty() && alias->typePackParams.empty()) + if (alias.has_value() && alias->typeParams.empty() && alias->typePackParams.empty()) { result = alias->type; } @@ -1104,11 +1164,11 @@ TypeId ConstraintGraphBuilder::resolveType(const ScopePtr& scope, AstType* ty, b } } - result = arena->addType(PendingExpansionTypeVar{*alias, parameters, packParameters}); + result = arena->addType(PendingExpansionTypeVar{ref->prefix, ref->name, parameters, packParameters}); if (topLevel) { - addConstraint(scope, TypeAliasExpansionConstraint{ /* target */ result }); + addConstraint(scope, ty->location, TypeAliasExpansionConstraint{/* target */ result}); } } } @@ -1141,8 +1201,7 @@ TypeId ConstraintGraphBuilder::resolveType(const ScopePtr& scope, AstType* ty, b }; } - // TODO: Remove TypeLevel{} here, we don't need it. - result = arena->addType(TableTypeVar{props, indexer, TypeLevel{}, TableState::Sealed}); + result = arena->addType(TableTypeVar{props, indexer, scope->level, TableState::Sealed}); } else if (auto fn = ty->as()) { @@ -1363,7 +1422,7 @@ TypeId ConstraintGraphBuilder::flattenPack(const ScopePtr& scope, Location locat TypePack onePack{{typeResult}, freshTypePack(scope)}; TypePackId oneTypePack = arena->addTypePack(std::move(onePack)); - addConstraint(scope, PackSubtypeConstraint{tp, oneTypePack}); + addConstraint(scope, location, PackSubtypeConstraint{tp, oneTypePack}); return typeResult; } diff --git a/Analysis/src/ConstraintSolver.cpp b/Analysis/src/ConstraintSolver.cpp index b2b1d472..1088d982 100644 --- a/Analysis/src/ConstraintSolver.cpp +++ b/Analysis/src/ConstraintSolver.cpp @@ -1,9 +1,11 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +#include "Luau/Anyification.h" #include "Luau/ApplyTypeFunction.h" #include "Luau/ConstraintSolver.h" #include "Luau/Instantiation.h" #include "Luau/Location.h" +#include "Luau/ModuleResolver.h" #include "Luau/Quantify.h" #include "Luau/ToString.h" #include "Luau/Unifier.h" @@ -240,11 +242,17 @@ void dump(ConstraintSolver* cs, ToStringOptions& opts) } } -ConstraintSolver::ConstraintSolver(TypeArena* arena, NotNull rootScope) +ConstraintSolver::ConstraintSolver(TypeArena* arena, NotNull rootScope, ModuleName moduleName, NotNull moduleResolver, + std::vector requireCycles) : arena(arena) , constraints(collectConstraints(rootScope)) , rootScope(rootScope) + , currentModuleName(std::move(moduleName)) + , moduleResolver(moduleResolver) + , requireCycles(requireCycles) { + opts.exhaustive = true; + for (NotNull c : constraints) { unsolvedConstraints.push_back(c); @@ -261,9 +269,6 @@ void ConstraintSolver::run() if (done()) return; - ToStringOptions opts; - opts.exhaustive = true; - if (FFlag::DebugLuauLogSolver) { printf("Starting solver\n"); @@ -371,10 +376,14 @@ bool ConstraintSolver::tryDispatch(NotNull constraint, bool fo success = tryDispatch(*uc, constraint, force); else if (auto bc = get(*constraint)) success = tryDispatch(*bc, constraint, force); + else if (auto ic = get(*constraint)) + success = tryDispatch(*ic, constraint, force); else if (auto nc = get(*constraint)) success = tryDispatch(*nc, constraint); else if (auto taec = get(*constraint)) success = tryDispatch(*taec, constraint); + else if (auto fcc = get(*constraint)) + success = tryDispatch(*fcc, constraint); else LUAU_ASSERT(0); @@ -400,6 +409,11 @@ bool ConstraintSolver::tryDispatch(const SubtypeConstraint& c, NotNull constraint, bool force) { + if (isBlocked(c.subPack)) + return block(c.subPack, constraint); + else if (isBlocked(c.superPack)) + return block(c.superPack, constraint); + unify(c.subPack, c.superPack, constraint->scope); return true; } @@ -512,6 +526,82 @@ bool ConstraintSolver::tryDispatch(const BinaryConstraint& c, NotNull constraint, bool force) +{ + /* + * for .. in loops can play out in a bunch of different ways depending on + * the shape of iteratee. + * + * iteratee might be: + * * (nextFn) + * * (nextFn, table) + * * (nextFn, table, firstIndex) + * * table with a metatable and __index + * * table with a metatable and __call but no __index (if the metatable has + * both, __index takes precedence) + * * table with an indexer but no __index or __call (or no metatable) + * + * To dispatch this constraint, we need first to know enough about iteratee + * to figure out which of the above shapes we are actually working with. + * + * If `force` is true and we still do not know, we must flag a warning. Type + * families are the fix for this. + * + * Since we need to know all of this stuff about the types of the iteratee, + * we have no choice but for ConstraintSolver to also be the thing that + * applies constraints to the types of the iterators. + */ + + auto block_ = [&](auto&& t) { + if (force) + { + // If we haven't figured out the type of the iteratee by now, + // there's nothing we can do. + return true; + } + + block(t, constraint); + return false; + }; + + auto [iteratorTypes, iteratorTail] = flatten(c.iterator); + if (iteratorTail) + return block_(*iteratorTail); + + if (0 == iteratorTypes.size()) + { + Anyification anyify{ + arena, constraint->scope, &iceReporter, getSingletonTypes().errorRecoveryType(), getSingletonTypes().errorRecoveryTypePack()}; + std::optional anyified = anyify.substitute(c.variables); + LUAU_ASSERT(anyified); + unify(*anyified, c.variables, constraint->scope); + + return true; + } + + TypeId nextTy = follow(iteratorTypes[0]); + if (get(nextTy)) + return block_(nextTy); + + if (get(nextTy)) + { + TypeId tableTy = getSingletonTypes().nilType; + if (iteratorTypes.size() >= 2) + tableTy = iteratorTypes[1]; + + TypeId firstIndexTy = getSingletonTypes().nilType; + if (iteratorTypes.size() >= 3) + firstIndexTy = iteratorTypes[2]; + + return tryDispatchIterableFunction(nextTy, tableTy, firstIndexTy, c, constraint, force); + } + + else + return tryDispatchIterableTable(iteratorTypes[0], c, constraint, force); + + return true; +} + bool ConstraintSolver::tryDispatch(const NameConstraint& c, NotNull constraint) { if (isBlocked(c.namedType)) @@ -519,7 +609,7 @@ bool ConstraintSolver::tryDispatch(const NameConstraint& c, NotNullpersistent) + if (target->persistent || target->owningArena != arena) return true; if (TableTypeVar* ttv = getMutable(target)) @@ -536,19 +626,27 @@ struct InfiniteTypeFinder : TypeVarOnceVisitor { ConstraintSolver* solver; const InstantiationSignature& signature; + NotNull scope; bool foundInfiniteType = false; - explicit InfiniteTypeFinder(ConstraintSolver* solver, const InstantiationSignature& signature) + explicit InfiniteTypeFinder(ConstraintSolver* solver, const InstantiationSignature& signature, NotNull scope) : solver(solver) , signature(signature) + , scope(scope) { } bool visit(TypeId ty, const PendingExpansionTypeVar& petv) override { - auto [typeArguments, packArguments] = saturateArguments(petv.fn, petv.typeArguments, petv.packArguments, solver->arena); + std::optional tf = + (petv.prefix) ? scope->lookupImportedType(petv.prefix->value, petv.name.value) : scope->lookupType(petv.name.value); - if (follow(petv.fn.type) == follow(signature.fn.type) && (signature.arguments != typeArguments || signature.packArguments != packArguments)) + if (!tf.has_value()) + return true; + + auto [typeArguments, packArguments] = saturateArguments(*tf, petv.typeArguments, petv.packArguments, solver->arena); + + if (follow(tf->type) == follow(signature.fn.type) && (signature.arguments != typeArguments || signature.packArguments != packArguments)) { foundInfiniteType = true; return false; @@ -563,17 +661,19 @@ struct InstantiationQueuer : TypeVarOnceVisitor ConstraintSolver* solver; const InstantiationSignature& signature; NotNull scope; + Location location; - explicit InstantiationQueuer(ConstraintSolver* solver, const InstantiationSignature& signature, NotNull scope) + explicit InstantiationQueuer(NotNull scope, const Location& location, ConstraintSolver* solver, const InstantiationSignature& signature) : solver(solver) , signature(signature) , scope(scope) + , location(location) { } bool visit(TypeId ty, const PendingExpansionTypeVar& petv) override { - solver->pushConstraint(TypeAliasExpansionConstraint{ty}, scope); + solver->pushConstraint(scope, location, TypeAliasExpansionConstraint{ty}); return false; } }; @@ -592,23 +692,32 @@ bool ConstraintSolver::tryDispatch(const TypeAliasExpansionConstraint& c, NotNul unblock(c.target); }; - // If there are no parameters to the type function we can just use the type - // directly. - if (petv->fn.typeParams.empty() && petv->fn.typePackParams.empty()) + std::optional tf = (petv->prefix) ? constraint->scope->lookupImportedType(petv->prefix->value, petv->name.value) + : constraint->scope->lookupType(petv->name.value); + + if (!tf.has_value()) { - bindResult(petv->fn.type); + reportError(UnknownSymbol{petv->name.value, UnknownSymbol::Context::Type}, constraint->location); + bindResult(getSingletonTypes().errorRecoveryType()); return true; } - auto [typeArguments, packArguments] = saturateArguments(petv->fn, petv->typeArguments, petv->packArguments, arena); + // If there are no parameters to the type function we can just use the type + // directly. + if (tf->typeParams.empty() && tf->typePackParams.empty()) + { + bindResult(tf->type); + return true; + } - bool sameTypes = - std::equal(typeArguments.begin(), typeArguments.end(), petv->fn.typeParams.begin(), petv->fn.typeParams.end(), [](auto&& itp, auto&& p) { - return itp == p.ty; - }); + auto [typeArguments, packArguments] = saturateArguments(*tf, petv->typeArguments, petv->packArguments, arena); - bool samePacks = std::equal( - packArguments.begin(), packArguments.end(), petv->fn.typePackParams.begin(), petv->fn.typePackParams.end(), [](auto&& itp, auto&& p) { + bool sameTypes = std::equal(typeArguments.begin(), typeArguments.end(), tf->typeParams.begin(), tf->typeParams.end(), [](auto&& itp, auto&& p) { + return itp == p.ty; + }); + + bool samePacks = + std::equal(packArguments.begin(), packArguments.end(), tf->typePackParams.begin(), tf->typePackParams.end(), [](auto&& itp, auto&& p) { return itp == p.tp; }); @@ -617,12 +726,12 @@ bool ConstraintSolver::tryDispatch(const TypeAliasExpansionConstraint& c, NotNul // to the TypeFun's type. if (sameTypes && samePacks) { - bindResult(petv->fn.type); + bindResult(tf->type); return true; } InstantiationSignature signature{ - petv->fn, + *tf, typeArguments, packArguments, }; @@ -642,8 +751,8 @@ bool ConstraintSolver::tryDispatch(const TypeAliasExpansionConstraint& c, NotNul // https://github.com/Roblox/luau/pull/68 for the RFC responsible for this. // This is a little nicer than using a recursion limit because we can catch // the infinite expansion before actually trying to expand it. - InfiniteTypeFinder itf{this, signature}; - itf.traverse(petv->fn.type); + InfiniteTypeFinder itf{this, signature, constraint->scope}; + itf.traverse(tf->type); if (itf.foundInfiniteType) { @@ -655,15 +764,15 @@ bool ConstraintSolver::tryDispatch(const TypeAliasExpansionConstraint& c, NotNul ApplyTypeFunction applyTypeFunction{arena}; for (size_t i = 0; i < typeArguments.size(); ++i) { - applyTypeFunction.typeArguments[petv->fn.typeParams[i].ty] = typeArguments[i]; + applyTypeFunction.typeArguments[tf->typeParams[i].ty] = typeArguments[i]; } for (size_t i = 0; i < packArguments.size(); ++i) { - applyTypeFunction.typePackArguments[petv->fn.typePackParams[i].tp] = packArguments[i]; + applyTypeFunction.typePackArguments[tf->typePackParams[i].tp] = packArguments[i]; } - std::optional maybeInstantiated = applyTypeFunction.substitute(petv->fn.type); + std::optional maybeInstantiated = applyTypeFunction.substitute(tf->type); // Note that ApplyTypeFunction::encounteredForwardedType is never set in // DCR, because we do not use free types for forward-declared generic // aliases. @@ -683,7 +792,7 @@ bool ConstraintSolver::tryDispatch(const TypeAliasExpansionConstraint& c, NotNul // Type function application will happily give us the exact same type if // there are e.g. generic saturatedTypeArguments that go unused. - bool needsClone = follow(petv->fn.type) == target; + bool needsClone = follow(tf->type) == target; // Only tables have the properties we're trying to set. TableTypeVar* ttv = getMutableTableType(target); @@ -722,7 +831,7 @@ bool ConstraintSolver::tryDispatch(const TypeAliasExpansionConstraint& c, NotNul // The application is not recursive, so we need to queue up application of // any child type function instantiations within the result in order for it // to be complete. - InstantiationQueuer queuer{this, signature, constraint->scope}; + InstantiationQueuer queuer{constraint->scope, constraint->location, this, signature}; queuer.traverse(target); instantiatedAliases[signature] = target; @@ -730,6 +839,152 @@ bool ConstraintSolver::tryDispatch(const TypeAliasExpansionConstraint& c, NotNul return true; } +bool ConstraintSolver::tryDispatch(const FunctionCallConstraint& c, NotNull constraint) +{ + TypeId fn = follow(c.fn); + TypePackId result = follow(c.result); + + if (isBlocked(c.fn)) + { + return block(c.fn, constraint); + } + + const FunctionTypeVar* ftv = get(fn); + bool usedMagic = false; + + if (ftv && ftv->dcrMagicFunction != nullptr) + { + usedMagic = ftv->dcrMagicFunction(NotNull(this), result, c.astFragment); + } + + if (!usedMagic) + { + for (const auto& inner : c.innerConstraints) + { + unsolvedConstraints.push_back(inner); + } + + asMutable(c.result)->ty.emplace(constraint->scope); + } + + unblock(c.result); + + return true; +} + +bool ConstraintSolver::tryDispatchIterableTable(TypeId iteratorTy, const IterableConstraint& c, NotNull constraint, bool force) +{ + auto block_ = [&](auto&& t) { + if (force) + { + // TODO: I believe it is the case that, if we are asked to force + // this constraint, then we can do nothing but fail. I'd like to + // find a code sample that gets here. + LUAU_ASSERT(0); + } + else + block(t, constraint); + return false; + }; + + // We may have to block here if we don't know what the iteratee type is, + // if it's a free table, if we don't know it has a metatable, and so on. + iteratorTy = follow(iteratorTy); + if (get(iteratorTy)) + return block_(iteratorTy); + + auto anyify = [&](auto ty) { + Anyification anyify{arena, constraint->scope, &iceReporter, getSingletonTypes().anyType, getSingletonTypes().anyTypePack}; + std::optional anyified = anyify.substitute(ty); + if (!anyified) + reportError(CodeTooComplex{}, constraint->location); + else + unify(*anyified, ty, constraint->scope); + }; + + auto errorify = [&](auto ty) { + Anyification anyify{ + arena, constraint->scope, &iceReporter, getSingletonTypes().errorRecoveryType(), getSingletonTypes().errorRecoveryTypePack()}; + std::optional errorified = anyify.substitute(ty); + if (!errorified) + reportError(CodeTooComplex{}, constraint->location); + else + unify(*errorified, ty, constraint->scope); + }; + + if (get(iteratorTy)) + { + anyify(c.variables); + return true; + } + + if (get(iteratorTy)) + { + errorify(c.variables); + return true; + } + + // Irksome: I don't think we have any way to guarantee that this table + // type never has a metatable. + + if (auto iteratorTable = get(iteratorTy)) + { + if (iteratorTable->state == TableState::Free) + return block_(iteratorTy); + + if (iteratorTable->indexer) + { + TypePackId expectedVariablePack = arena->addTypePack({iteratorTable->indexer->indexType, iteratorTable->indexer->indexResultType}); + unify(c.variables, expectedVariablePack, constraint->scope); + } + else + errorify(c.variables); + } + else if (auto iteratorMetatable = get(iteratorTy)) + { + TypeId metaTy = follow(iteratorMetatable->metatable); + if (get(metaTy)) + return block_(metaTy); + + LUAU_ASSERT(0); + } + else + errorify(c.variables); + + return true; +} + +bool ConstraintSolver::tryDispatchIterableFunction( + TypeId nextTy, TypeId tableTy, TypeId firstIndexTy, const IterableConstraint& c, NotNull constraint, bool force) +{ + // We need to know whether or not this type is nil or not. + // If we don't know, block and reschedule ourselves. + firstIndexTy = follow(firstIndexTy); + if (get(firstIndexTy)) + { + if (force) + LUAU_ASSERT(0); + else + block(firstIndexTy, constraint); + return false; + } + + const TypeId firstIndex = isNil(firstIndexTy) ? arena->freshType(constraint->scope) // FIXME: Surely this should be a union (free | nil) + : firstIndexTy; + + // nextTy : (tableTy, indexTy?) -> (indexTy, valueTailTy...) + const TypePackId nextArgPack = arena->addTypePack({tableTy, arena->addType(UnionTypeVar{{firstIndex, getSingletonTypes().nilType}})}); + const TypePackId valueTailTy = arena->addTypePack(FreeTypePack{constraint->scope}); + const TypePackId nextRetPack = arena->addTypePack(TypePack{{firstIndex}, valueTailTy}); + + const TypeId expectedNextTy = arena->addType(FunctionTypeVar{nextArgPack, nextRetPack}); + unify(nextTy, expectedNextTy, constraint->scope); + + pushConstraint(constraint->scope, constraint->location, PackSubtypeConstraint{c.variables, nextRetPack}); + + return true; +} + void ConstraintSolver::block_(BlockedConstraintId target, NotNull constraint) { blocked[target].push_back(constraint); @@ -741,14 +996,14 @@ void ConstraintSolver::block_(BlockedConstraintId target, NotNull target, NotNull constraint) { if (FFlag::DebugLuauLogSolver) - printf("block Constraint %s on\t%s\n", toString(*target).c_str(), toString(*constraint).c_str()); + printf("block Constraint %s on\t%s\n", toString(*target, opts).c_str(), toString(*constraint, opts).c_str()); block_(target, constraint); } bool ConstraintSolver::block(TypeId target, NotNull constraint) { if (FFlag::DebugLuauLogSolver) - printf("block TypeId %s on\t%s\n", toString(target).c_str(), toString(*constraint).c_str()); + printf("block TypeId %s on\t%s\n", toString(target, opts).c_str(), toString(*constraint, opts).c_str()); block_(target, constraint); return false; } @@ -756,7 +1011,7 @@ bool ConstraintSolver::block(TypeId target, NotNull constraint bool ConstraintSolver::block(TypePackId target, NotNull constraint) { if (FFlag::DebugLuauLogSolver) - printf("block TypeId %s on\t%s\n", toString(target).c_str(), toString(*constraint).c_str()); + printf("block TypeId %s on\t%s\n", toString(target, opts).c_str(), toString(*constraint, opts).c_str()); block_(target, constraint); return false; } @@ -772,7 +1027,7 @@ void ConstraintSolver::unblock_(BlockedConstraintId progressed) { auto& count = blockedConstraints[unblockedConstraint]; if (FFlag::DebugLuauLogSolver) - printf("Unblocking count=%d\t%s\n", int(count), toString(*unblockedConstraint).c_str()); + printf("Unblocking count=%d\t%s\n", int(count), toString(*unblockedConstraint, opts).c_str()); // This assertion being hit indicates that `blocked` and // `blockedConstraints` desynchronized at some point. This is problematic @@ -817,6 +1072,11 @@ bool ConstraintSolver::isBlocked(TypeId ty) return nullptr != get(follow(ty)) || nullptr != get(follow(ty)); } +bool ConstraintSolver::isBlocked(TypePackId tp) +{ + return nullptr != get(follow(tp)); +} + bool ConstraintSolver::isBlocked(NotNull constraint) { auto blockedIt = blockedConstraints.find(constraint); @@ -830,6 +1090,13 @@ void ConstraintSolver::unify(TypeId subType, TypeId superType, NotNull sc u.tryUnify(subType, superType); + if (!u.errors.empty()) + { + TypeId errorType = getSingletonTypes().errorRecoveryType(); + u.tryUnify(subType, errorType); + u.tryUnify(superType, errorType); + } + const auto [changedTypes, changedPacks] = u.log.getChanges(); u.log.commit(); @@ -853,22 +1120,69 @@ void ConstraintSolver::unify(TypePackId subPack, TypePackId superPack, NotNull scope) +void ConstraintSolver::pushConstraint(NotNull scope, const Location& location, ConstraintV cv) { - std::unique_ptr c = std::make_unique(std::move(cv), scope); + std::unique_ptr c = std::make_unique(scope, location, std::move(cv)); NotNull borrow = NotNull(c.get()); solverConstraints.push_back(std::move(c)); unsolvedConstraints.push_back(borrow); } +TypeId ConstraintSolver::resolveModule(const ModuleInfo& info, const Location& location) +{ + if (info.name.empty()) + { + reportError(UnknownRequire{}, location); + return getSingletonTypes().errorRecoveryType(); + } + + std::string humanReadableName = moduleResolver->getHumanReadableModuleName(info.name); + + for (const auto& [location, path] : requireCycles) + { + if (!path.empty() && path.front() == humanReadableName) + return getSingletonTypes().anyType; + } + + ModulePtr module = moduleResolver->getModule(info.name); + if (!module) + { + if (!moduleResolver->moduleExists(info.name) && !info.optional) + reportError(UnknownRequire{humanReadableName}, location); + + return getSingletonTypes().errorRecoveryType(); + } + + if (module->type != SourceCode::Type::Module) + { + reportError(IllegalRequire{humanReadableName, "Module is not a ModuleScript. It cannot be required."}, location); + return getSingletonTypes().errorRecoveryType(); + } + + TypePackId modulePack = module->getModuleScope()->returnType; + if (get(modulePack)) + return getSingletonTypes().errorRecoveryType(); + + std::optional moduleType = first(modulePack); + if (!moduleType) + { + reportError(IllegalRequire{humanReadableName, "Module does not return exactly 1 value. It cannot be required."}, location); + return getSingletonTypes().errorRecoveryType(); + } + + return *moduleType; +} + void ConstraintSolver::reportError(TypeErrorData&& data, const Location& location) { errors.emplace_back(location, std::move(data)); + errors.back().moduleName = currentModuleName; } void ConstraintSolver::reportError(TypeError e) { errors.emplace_back(std::move(e)); + errors.back().moduleName = currentModuleName; } } // namespace Luau diff --git a/Analysis/src/ConstraintSolverLogger.cpp b/Analysis/src/ConstraintSolverLogger.cpp index 097ceeec..5ba40521 100644 --- a/Analysis/src/ConstraintSolverLogger.cpp +++ b/Analysis/src/ConstraintSolverLogger.cpp @@ -3,6 +3,7 @@ #include "Luau/ConstraintSolverLogger.h" #include "Luau/JsonEmitter.h" +#include "Luau/ToString.h" LUAU_FASTFLAG(LuauFixNameMaps); diff --git a/Analysis/src/Frontend.cpp b/Analysis/src/Frontend.cpp index c8c5d4b5..d8839f2f 100644 --- a/Analysis/src/Frontend.cpp +++ b/Analysis/src/Frontend.cpp @@ -14,6 +14,7 @@ #include "Luau/TypeChecker2.h" #include "Luau/TypeInfer.h" #include "Luau/Variant.h" +#include "Luau/BuiltinDefinitions.h" #include #include @@ -99,7 +100,7 @@ LoadDefinitionFileResult Frontend::loadDefinitionFile(std::string_view source, c module.root = parseResult.root; module.mode = Mode::Definition; - ModulePtr checkedModule = check(module, Mode::Definition, globalScope); + ModulePtr checkedModule = check(module, Mode::Definition, globalScope, {}); if (checkedModule->errors.size() > 0) return LoadDefinitionFileResult{false, parseResult, checkedModule}; @@ -526,7 +527,7 @@ CheckResult Frontend::check(const ModuleName& name, std::optional requireCycles) { ModulePtr result = std::make_shared(); - ConstraintGraphBuilder cgb{sourceModule.name, result, &result->internalTypes, NotNull(&iceHandler), getGlobalScope()}; + ConstraintGraphBuilder cgb{sourceModule.name, result, &result->internalTypes, NotNull(&moduleResolver), NotNull(&iceHandler), getGlobalScope()}; cgb.visit(sourceModule.root); result->errors = std::move(cgb.errors); - ConstraintSolver cs{&result->internalTypes, NotNull(cgb.rootScope)}; + ConstraintSolver cs{&result->internalTypes, NotNull(cgb.rootScope), sourceModule.name, NotNull(&moduleResolver), requireCycles}; cs.run(); for (TypeError& e : cs.errors) @@ -852,11 +853,12 @@ ModulePtr Frontend::check(const SourceModule& sourceModule, Mode mode, const Sco result->astOriginalCallTypes = std::move(cgb.astOriginalCallTypes); result->astResolvedTypes = std::move(cgb.astResolvedTypes); result->astResolvedTypePacks = std::move(cgb.astResolvedTypePacks); - - result->clonePublicInterface(iceHandler); + result->type = sourceModule.type; Luau::check(sourceModule, result.get()); + result->clonePublicInterface(iceHandler); + return result; } diff --git a/Analysis/src/Instantiation.cpp b/Analysis/src/Instantiation.cpp index e98ab185..2d1d62f3 100644 --- a/Analysis/src/Instantiation.cpp +++ b/Analysis/src/Instantiation.cpp @@ -46,6 +46,7 @@ TypeId Instantiation::clean(TypeId ty) FunctionTypeVar clone = FunctionTypeVar{level, ftv->argTypes, ftv->retTypes, ftv->definition, ftv->hasSelf}; clone.magicFunction = ftv->magicFunction; + clone.dcrMagicFunction = ftv->dcrMagicFunction; clone.tags = ftv->tags; clone.argNames = ftv->argNames; TypeId result = addType(std::move(clone)); diff --git a/Analysis/src/JsonEmitter.cpp b/Analysis/src/JsonEmitter.cpp index e99619ba..9c8a7af9 100644 --- a/Analysis/src/JsonEmitter.cpp +++ b/Analysis/src/JsonEmitter.cpp @@ -11,7 +11,8 @@ namespace Luau::Json static constexpr int CHUNK_SIZE = 1024; ObjectEmitter::ObjectEmitter(NotNull emitter) - : emitter(emitter), finished(false) + : emitter(emitter) + , finished(false) { comma = emitter->pushComma(); emitter->writeRaw('{'); @@ -33,7 +34,8 @@ void ObjectEmitter::finish() } ArrayEmitter::ArrayEmitter(NotNull emitter) - : emitter(emitter), finished(false) + : emitter(emitter) + , finished(false) { comma = emitter->pushComma(); emitter->writeRaw('['); diff --git a/Analysis/src/Linter.cpp b/Analysis/src/Linter.cpp index 426ff9d6..669739a0 100644 --- a/Analysis/src/Linter.cpp +++ b/Analysis/src/Linter.cpp @@ -216,7 +216,8 @@ static bool similar(AstExpr* lhs, AstExpr* rhs) return false; for (size_t i = 0; i < le->strings.size; ++i) - if (le->strings.data[i].size != re->strings.data[i].size || memcmp(le->strings.data[i].data, re->strings.data[i].data, le->strings.data[i].size) != 0) + if (le->strings.data[i].size != re->strings.data[i].size || + memcmp(le->strings.data[i].data, re->strings.data[i].data, le->strings.data[i].size) != 0) return false; for (size_t i = 0; i < le->expressions.size; ++i) @@ -2675,13 +2676,18 @@ public: private: LintContext* context; - bool isComparison(AstExprBinary::Op op) + static bool isEquality(AstExprBinary::Op op) + { + return op == AstExprBinary::CompareNe || op == AstExprBinary::CompareEq; + } + + static bool isComparison(AstExprBinary::Op op) { return op == AstExprBinary::CompareNe || op == AstExprBinary::CompareEq || op == AstExprBinary::CompareLt || op == AstExprBinary::CompareLe || op == AstExprBinary::CompareGt || op == AstExprBinary::CompareGe; } - bool isNot(AstExpr* node) + static bool isNot(AstExpr* node) { AstExprUnary* expr = node->as(); @@ -2698,22 +2704,26 @@ private: { std::string op = toString(node->op); - if (node->op == AstExprBinary::CompareEq || node->op == AstExprBinary::CompareNe) + if (isEquality(node->op)) emitWarning(*context, LintWarning::Code_ComparisonPrecedence, node->location, - "not X %s Y is equivalent to (not X) %s Y; consider using X %s Y, or wrap one of the expressions in parentheses to silence", - op.c_str(), op.c_str(), node->op == AstExprBinary::CompareEq ? "~=" : "=="); + "not X %s Y is equivalent to (not X) %s Y; consider using X %s Y, or add parentheses to silence", op.c_str(), op.c_str(), + node->op == AstExprBinary::CompareEq ? "~=" : "=="); else emitWarning(*context, LintWarning::Code_ComparisonPrecedence, node->location, - "not X %s Y is equivalent to (not X) %s Y; wrap one of the expressions in parentheses to silence", op.c_str(), op.c_str()); + "not X %s Y is equivalent to (not X) %s Y; add parentheses to silence", op.c_str(), op.c_str()); } else if (AstExprBinary* left = node->left->as(); left && isComparison(left->op)) { std::string lop = toString(left->op); std::string rop = toString(node->op); - emitWarning(*context, LintWarning::Code_ComparisonPrecedence, node->location, - "X %s Y %s Z is equivalent to (X %s Y) %s Z; wrap one of the expressions in parentheses to silence", lop.c_str(), rop.c_str(), - lop.c_str(), rop.c_str()); + if (isEquality(left->op) || isEquality(node->op)) + emitWarning(*context, LintWarning::Code_ComparisonPrecedence, node->location, + "X %s Y %s Z is equivalent to (X %s Y) %s Z; add parentheses to silence", lop.c_str(), rop.c_str(), lop.c_str(), rop.c_str()); + else + emitWarning(*context, LintWarning::Code_ComparisonPrecedence, node->location, + "X %s Y %s Z is equivalent to (X %s Y) %s Z; did you mean X %s Y and Y %s Z?", lop.c_str(), rop.c_str(), lop.c_str(), rop.c_str(), + lop.c_str(), rop.c_str()); } return true; diff --git a/Analysis/src/Module.cpp b/Analysis/src/Module.cpp index de796c7a..4c9e9537 100644 --- a/Analysis/src/Module.cpp +++ b/Analysis/src/Module.cpp @@ -219,8 +219,7 @@ void Module::clonePublicInterface(InternalErrorReporter& ice) TypePackId returnType = moduleScope->returnType; std::optional varargPack = FFlag::DebugLuauDeferredConstraintResolution ? std::nullopt : moduleScope->varargPack; - std::unordered_map* exportedTypeBindings = - FFlag::DebugLuauDeferredConstraintResolution ? nullptr : &moduleScope->exportedTypeBindings; + std::unordered_map* exportedTypeBindings = &moduleScope->exportedTypeBindings; TxnLog log; ClonePublicInterface clonePublicInterface{&log, this}; diff --git a/Analysis/src/Substitution.cpp b/Analysis/src/Substitution.cpp index 0beeb58c..fa12f306 100644 --- a/Analysis/src/Substitution.cpp +++ b/Analysis/src/Substitution.cpp @@ -510,7 +510,7 @@ void Substitution::foundDirty(TypeId ty) ty = log->follow(ty); if (FFlag::LuauSubstitutionReentrant && newTypes.contains(ty)) - return; + return; if (isDirty(ty)) newTypes[ty] = follow(clean(ty)); @@ -523,7 +523,7 @@ void Substitution::foundDirty(TypePackId tp) tp = log->follow(tp); if (FFlag::LuauSubstitutionReentrant && newPacks.contains(tp)) - return; + return; if (isDirty(tp)) newPacks[tp] = follow(clean(tp)); diff --git a/Analysis/src/ToString.cpp b/Analysis/src/ToString.cpp index b4ef4384..62b7afd5 100644 --- a/Analysis/src/ToString.cpp +++ b/Analysis/src/ToString.cpp @@ -1036,6 +1036,13 @@ struct TypePackStringifier { stringify(btv.boundTo); } + + void operator()(TypePackId, const BlockedTypePack& btp) + { + state.emit("*blocked-tp-"); + state.emit(btp.index); + state.emit("*"); + } }; void TypeVarStringifier::stringify(TypePackId tp) @@ -1099,9 +1106,8 @@ ToStringResult toStringDetailed(TypeId ty, ToStringOptions& opts) ToStringResult result; - StringifierState state = FFlag::LuauFixNameMaps - ? StringifierState{opts, result, opts.nameMap} - : StringifierState{opts, result, opts.DEPRECATED_nameMap}; + StringifierState state = + FFlag::LuauFixNameMaps ? StringifierState{opts, result, opts.nameMap} : StringifierState{opts, result, opts.DEPRECATED_nameMap}; std::set cycles; std::set cycleTPs; @@ -1211,9 +1217,8 @@ ToStringResult toStringDetailed(TypePackId tp, ToStringOptions& opts) * 4. Print out the root of the type using the same algorithm as step 3. */ ToStringResult result; - StringifierState state = FFlag::LuauFixNameMaps - ? StringifierState{opts, result, opts.nameMap} - : StringifierState{opts, result, opts.DEPRECATED_nameMap}; + StringifierState state = + FFlag::LuauFixNameMaps ? StringifierState{opts, result, opts.nameMap} : StringifierState{opts, result, opts.DEPRECATED_nameMap}; std::set cycles; std::set cycleTPs; @@ -1302,9 +1307,8 @@ std::string toString(const TypePackVar& tp, ToStringOptions& opts) std::string toStringNamedFunction(const std::string& funcName, const FunctionTypeVar& ftv, ToStringOptions& opts) { ToStringResult result; - StringifierState state = FFlag::LuauFixNameMaps - ? StringifierState{opts, result, opts.nameMap} - : StringifierState{opts, result, opts.DEPRECATED_nameMap}; + StringifierState state = + FFlag::LuauFixNameMaps ? StringifierState{opts, result, opts.nameMap} : StringifierState{opts, result, opts.DEPRECATED_nameMap}; TypeVarStringifier tvs{state}; state.emit(funcName); @@ -1437,8 +1441,7 @@ std::string toString(const Constraint& constraint, ToStringOptions& opts) using T = std::decay_t; // TODO: Inline and delete this function when clipping FFlag::LuauFixNameMaps - auto tos = [](auto&& a, ToStringOptions& opts) - { + auto tos = [](auto&& a, ToStringOptions& opts) { if (FFlag::LuauFixNameMaps) return toString(a, opts); else @@ -1488,6 +1491,13 @@ std::string toString(const Constraint& constraint, ToStringOptions& opts) return resultStr + " ~ Binary<" + toString(c.op) + ", " + leftStr + ", " + rightStr + ">"; } + else if constexpr (std::is_same_v) + { + std::string iteratorStr = tos(c.iterator, opts); + std::string variableStr = tos(c.variables, opts); + + return variableStr + " ~ Iterate<" + iteratorStr + ">"; + } else if constexpr (std::is_same_v) { std::string namedStr = tos(c.namedType, opts); @@ -1498,6 +1508,10 @@ std::string toString(const Constraint& constraint, ToStringOptions& opts) std::string targetStr = tos(c.target, opts); return "expand " + targetStr; } + else if constexpr (std::is_same_v) + { + return "call " + tos(c.fn, opts) + " with { result = " + tos(c.result, opts) + " }"; + } else static_assert(always_false_v, "Non-exhaustive constraint switch"); }; diff --git a/Analysis/src/TypeArena.cpp b/Analysis/src/TypeArena.cpp index 0c89d130..c7980ab0 100644 --- a/Analysis/src/TypeArena.cpp +++ b/Analysis/src/TypeArena.cpp @@ -31,6 +31,15 @@ TypeId TypeArena::freshType(TypeLevel level) return allocated; } +TypeId TypeArena::freshType(Scope* scope) +{ + TypeId allocated = typeVars.allocate(FreeTypeVar{scope}); + + asMutable(allocated)->owningArena = this; + + return allocated; +} + TypePackId TypeArena::addTypePack(std::initializer_list types) { TypePackId allocated = typePacks.allocate(TypePack{std::move(types)}); @@ -40,9 +49,9 @@ TypePackId TypeArena::addTypePack(std::initializer_list types) return allocated; } -TypePackId TypeArena::addTypePack(std::vector types) +TypePackId TypeArena::addTypePack(std::vector types, std::optional tail) { - TypePackId allocated = typePacks.allocate(TypePack{std::move(types)}); + TypePackId allocated = typePacks.allocate(TypePack{std::move(types), tail}); asMutable(allocated)->owningArena = this; diff --git a/Analysis/src/TypeAttach.cpp b/Analysis/src/TypeAttach.cpp index f21a4fa9..84494083 100644 --- a/Analysis/src/TypeAttach.cpp +++ b/Analysis/src/TypeAttach.cpp @@ -373,6 +373,11 @@ public: return Luau::visit(*this, btp.boundTo->ty); } + AstTypePack* operator()(const BlockedTypePack& btp) const + { + return allocator->alloc(Location(), AstName("*blocked*")); + } + AstTypePack* operator()(const TypePack& tp) const { AstArray head; diff --git a/Analysis/src/TypeChecker2.cpp b/Analysis/src/TypeChecker2.cpp index e5813cd2..480bdf40 100644 --- a/Analysis/src/TypeChecker2.cpp +++ b/Analysis/src/TypeChecker2.cpp @@ -166,7 +166,8 @@ struct TypeChecker2 auto pusher = pushStack(stat); if (0) - {} + { + } else if (auto s = stat->as()) return visit(s); else if (auto s = stat->as()) @@ -239,11 +240,9 @@ struct TypeChecker2 visit(repeatStatement->condition); } - void visit(AstStatBreak*) - {} + void visit(AstStatBreak*) {} - void visit(AstStatContinue*) - {} + void visit(AstStatContinue*) {} void visit(AstStatReturn* ret) { @@ -339,6 +338,50 @@ struct TypeChecker2 visit(forStatement->body); } + // "Render" a type pack out to an array of a given length. Expands + // variadics and various other things to get there. + static std::vector flatten(TypeArena& arena, TypePackId pack, size_t length) + { + std::vector result; + + auto it = begin(pack); + auto endIt = end(pack); + + while (it != endIt) + { + result.push_back(*it); + + if (result.size() >= length) + return result; + + ++it; + } + + if (!it.tail()) + return result; + + TypePackId tail = *it.tail(); + if (get(tail)) + LUAU_ASSERT(0); + else if (auto vtp = get(tail)) + { + while (result.size() < length) + result.push_back(vtp->ty); + } + else if (get(tail) || get(tail)) + { + while (result.size() < length) + result.push_back(arena.addType(FreeTypeVar{nullptr})); + } + else if (auto etp = get(tail)) + { + while (result.size() < length) + result.push_back(getSingletonTypes().errorRecoveryType()); + } + + return result; + } + void visit(AstStatForIn* forInStatement) { for (AstLocal* local : forInStatement->vars) @@ -351,6 +394,128 @@ struct TypeChecker2 visit(expr); visit(forInStatement->body); + + // Rule out crazy stuff. Maybe possible if the file is not syntactically valid. + if (!forInStatement->vars.size || !forInStatement->values.size) + return; + + NotNull scope = stack.back(); + TypeArena tempArena; + + std::vector variableTypes; + for (AstLocal* var : forInStatement->vars) + { + std::optional ty = scope->lookup(var); + LUAU_ASSERT(ty); + variableTypes.emplace_back(*ty); + } + + // ugh. There's nothing in the AST to hang a whole type pack on for the + // set of iteratees, so we have to piece it back together by hand. + std::vector valueTypes; + for (size_t i = 0; i < forInStatement->values.size - 1; ++i) + valueTypes.emplace_back(lookupType(forInStatement->values.data[i])); + TypePackId iteratorTail = lookupPack(forInStatement->values.data[forInStatement->values.size - 1]); + TypePackId iteratorPack = tempArena.addTypePack(valueTypes, iteratorTail); + + // ... and then expand it out to 3 values (if possible) + const std::vector iteratorTypes = flatten(tempArena, iteratorPack, 3); + if (iteratorTypes.empty()) + { + reportError(GenericError{"for..in loops require at least one value to iterate over. Got zero"}, getLocation(forInStatement->values)); + return; + } + TypeId iteratorTy = follow(iteratorTypes[0]); + + /* + * If the first iterator argument is a function + * * There must be 1 to 3 iterator arguments. Name them (nextTy, + * arrayTy, startIndexTy) + * * The return type of nextTy() must correspond to the variables' + * types and counts. HOWEVER the first iterator will never be nil. + * * The first return value of nextTy must be compatible with + * startIndexTy. + * * The first argument to nextTy() must be compatible with arrayTy if + * present. nil if not. + * * The second argument to nextTy() must be compatible with + * startIndexTy if it is present. Else, it must be compatible with + * nil. + * * nextTy() must be callable with only 2 arguments. + */ + if (const FunctionTypeVar* nextFn = get(iteratorTy)) + { + if (iteratorTypes.size() < 1 || iteratorTypes.size() > 3) + reportError(GenericError{"for..in loops must be passed (next, [table[, state]])"}, getLocation(forInStatement->values)); + + // It is okay if there aren't enough iterators, but the iteratee must provide enough. + std::vector expectedVariableTypes = flatten(tempArena, nextFn->retTypes, variableTypes.size()); + if (expectedVariableTypes.size() < variableTypes.size()) + reportError(GenericError{"next() does not return enough values"}, forInStatement->vars.data[0]->location); + + for (size_t i = 0; i < std::min(expectedVariableTypes.size(), variableTypes.size()); ++i) + reportErrors(tryUnify(scope, forInStatement->vars.data[i]->location, variableTypes[i], expectedVariableTypes[i])); + + // nextFn is going to be invoked with (arrayTy, startIndexTy) + + // It will be passed two arguments on every iteration save the + // first. + + // It may be invoked with 0 or 1 argument on the first iteration. + // This depends on the types in iterateePack and therefore + // iteratorTypes. + + // If iteratorTypes is too short to be a valid call to nextFn, we have to report a count mismatch error. + // If 2 is too short to be a valid call to nextFn, we have to report a count mismatch error. + // If 2 is too long to be a valid call to nextFn, we have to report a count mismatch error. + auto [minCount, maxCount] = getParameterExtents(TxnLog::empty(), nextFn->argTypes); + + if (minCount > 2) + reportError(CountMismatch{2, minCount, CountMismatch::Arg}, forInStatement->vars.data[0]->location); + if (maxCount && *maxCount < 2) + reportError(CountMismatch{2, *maxCount, CountMismatch::Arg}, forInStatement->vars.data[0]->location); + + const std::vector flattenedArgTypes = flatten(tempArena, nextFn->argTypes, 2); + const auto [argTypes, argsTail] = Luau::flatten(nextFn->argTypes); + + size_t firstIterationArgCount = iteratorTypes.empty() ? 0 : iteratorTypes.size() - 1; + size_t actualArgCount = expectedVariableTypes.size(); + + if (firstIterationArgCount < minCount) + reportError(CountMismatch{2, firstIterationArgCount, CountMismatch::Arg}, forInStatement->vars.data[0]->location); + else if (actualArgCount < minCount) + reportError(CountMismatch{2, actualArgCount, CountMismatch::Arg}, forInStatement->vars.data[0]->location); + + if (iteratorTypes.size() >= 2 && flattenedArgTypes.size() > 0) + { + size_t valueIndex = forInStatement->values.size > 1 ? 1 : 0; + reportErrors(tryUnify(scope, forInStatement->values.data[valueIndex]->location, iteratorTypes[1], flattenedArgTypes[0])); + } + + if (iteratorTypes.size() == 3 && flattenedArgTypes.size() > 1) + { + size_t valueIndex = forInStatement->values.size > 2 ? 2 : 0; + reportErrors(tryUnify(scope, forInStatement->values.data[valueIndex]->location, iteratorTypes[2], flattenedArgTypes[1])); + } + } + else if (const TableTypeVar* ttv = get(iteratorTy)) + { + if ((forInStatement->vars.size == 1 || forInStatement->vars.size == 2) && ttv->indexer) + { + reportErrors(tryUnify(scope, forInStatement->vars.data[0]->location, variableTypes[0], ttv->indexer->indexType)); + if (variableTypes.size() == 2) + reportErrors(tryUnify(scope, forInStatement->vars.data[1]->location, variableTypes[1], ttv->indexer->indexResultType)); + } + else + reportError(GenericError{"Cannot iterate over a table without indexer"}, forInStatement->values.data[0]->location); + } + else if (get(iteratorTy) || get(iteratorTy)) + { + // nothing + } + else + { + reportError(CannotCallNonFunction{iteratorTy}, forInStatement->values.data[0]->location); + } } void visit(AstStatAssign* assign) @@ -456,7 +621,8 @@ struct TypeChecker2 auto StackPusher = pushStack(expr); if (0) - {} + { + } else if (auto e = expr->as()) return visit(e); else if (auto e = expr->as()) @@ -561,9 +727,21 @@ struct TypeChecker2 TypePackId expectedRetType = lookupPack(call); TypeId functionType = lookupType(call->func); - TypeId instantiatedFunctionType = instantiation.substitute(functionType).value_or(nullptr); LUAU_ASSERT(functionType); + if (get(functionType) || get(functionType)) + return; + + // TODO: Lots of other types are callable: intersections of functions + // and things with the __call metamethod. + if (!get(functionType)) + { + reportError(CannotCallNonFunction{functionType}, call->func->location); + return; + } + + TypeId instantiatedFunctionType = follow(instantiation.substitute(functionType).value_or(nullptr)); + TypePack args; for (AstExpr* arg : call->args) { @@ -575,12 +753,11 @@ struct TypeChecker2 TypePackId argsTp = arena.addTypePack(args); FunctionTypeVar ftv{argsTp, expectedRetType}; TypeId expectedType = arena.addType(ftv); + if (!isSubtype(expectedType, instantiatedFunctionType, stack.back(), ice)) { - unfreeze(module->interfaceTypes); CloneState cloneState; - expectedType = clone(expectedType, module->interfaceTypes, cloneState); - freeze(module->interfaceTypes); + expectedType = clone(expectedType, module->internalTypes, cloneState); reportError(TypeMismatch{expectedType, functionType}, call->location); } } @@ -592,7 +769,8 @@ struct TypeChecker2 // leftType must have a property called indexName->index - std::optional ty = getIndexTypeFromType(module->getModuleScope(), leftType, indexName->index.value, indexName->location, /* addErrors */ true); + std::optional ty = + getIndexTypeFromType(module->getModuleScope(), leftType, indexName->index.value, indexName->location, /* addErrors */ true); if (ty) { if (!isSubtype(resultType, *ty, stack.back(), ice)) @@ -972,18 +1150,34 @@ struct TypeChecker2 } } - void reportError(TypeErrorData&& data, const Location& location) + template + ErrorVec tryUnify(NotNull scope, const Location& location, TID subTy, TID superTy) + { + UnifierSharedState sharedState{&ice}; + Unifier u{&module->internalTypes, Mode::Strict, scope, location, Covariant, sharedState}; + u.anyIsTop = true; + u.tryUnify(subTy, superTy); + + return std::move(u.errors); + } + + void reportError(TypeErrorData data, const Location& location) { module->errors.emplace_back(location, sourceModule->name, std::move(data)); } void reportError(TypeError e) { - module->errors.emplace_back(std::move(e)); + reportError(std::move(e.data), e.location); } - std::optional getIndexTypeFromType( - const ScopePtr& scope, TypeId type, const std::string& prop, const Location& location, bool addErrors) + void reportErrors(ErrorVec errors) + { + for (TypeError e : errors) + reportError(std::move(e)); + } + + std::optional getIndexTypeFromType(const ScopePtr& scope, TypeId type, const std::string& prop, const Location& location, bool addErrors) { return Luau::getIndexTypeFromType(scope, module->errors, &module->internalTypes, type, prop, location, addErrors, ice); } diff --git a/Analysis/src/TypeInfer.cpp b/Analysis/src/TypeInfer.cpp index c9340945..2bda2804 100644 --- a/Analysis/src/TypeInfer.cpp +++ b/Analysis/src/TypeInfer.cpp @@ -32,7 +32,6 @@ LUAU_FASTINTVARIABLE(LuauCheckRecursionLimit, 300) LUAU_FASTINTVARIABLE(LuauVisitRecursionLimit, 500) LUAU_FASTFLAG(LuauKnowsTheDataModel3) LUAU_FASTFLAG(LuauAutocompleteDynamicLimits) -LUAU_FASTFLAGVARIABLE(LuauExpectedTableUnionIndexerType, false) LUAU_FASTFLAGVARIABLE(LuauInplaceDemoteSkipAllBound, false) LUAU_FASTFLAGVARIABLE(LuauLowerBoundsCalculation, false) LUAU_FASTFLAGVARIABLE(DebugLuauFreezeDuringUnification, false) @@ -45,6 +44,7 @@ LUAU_FASTFLAGVARIABLE(LuauBinaryNeedsExpectedTypesToo, false) LUAU_FASTFLAGVARIABLE(LuauNeverTypesAndOperatorsInference, false) LUAU_FASTFLAGVARIABLE(LuauReturnsFromCallsitesAreNotWidened, false) LUAU_FASTFLAGVARIABLE(LuauCompleteVisitor, false) +LUAU_FASTFLAGVARIABLE(LuauUnionOfTypesFollow, false) namespace Luau { @@ -2343,7 +2343,7 @@ WithPredicate TypeChecker::checkExpr(const ScopePtr& scope, const AstExp { if (auto prop = ttv->props.find(key->value.data); prop != ttv->props.end()) expectedResultTypes.push_back(prop->second.type); - else if (FFlag::LuauExpectedTableUnionIndexerType && ttv->indexer && maybeString(ttv->indexer->indexType)) + else if (ttv->indexer && maybeString(ttv->indexer->indexType)) expectedResultTypes.push_back(ttv->indexer->indexResultType); } } @@ -2498,6 +2498,12 @@ std::string opToMetaTableEntry(const AstExprBinary::Op& op) TypeId TypeChecker::unionOfTypes(TypeId a, TypeId b, const ScopePtr& scope, const Location& location, bool unifyFreeTypes) { + if (FFlag::LuauUnionOfTypesFollow) + { + a = follow(a); + b = follow(b); + } + if (unifyFreeTypes && (get(a) || get(b))) { if (unify(b, a, scope, location)) @@ -3659,33 +3665,6 @@ WithPredicate TypeChecker::checkExprPackHelper(const ScopePtr& scope } } -// Returns the minimum number of arguments the argument list can accept. -static size_t getMinParameterCount(TxnLog* log, TypePackId tp) -{ - size_t minCount = 0; - size_t optionalCount = 0; - - auto it = begin(tp, log); - auto endIter = end(tp); - - while (it != endIter) - { - TypeId ty = *it; - if (isOptional(ty)) - ++optionalCount; - else - { - minCount += optionalCount; - optionalCount = 0; - minCount++; - } - - ++it; - } - - return minCount; -} - void TypeChecker::checkArgumentList( const ScopePtr& scope, Unifier& state, TypePackId argPack, TypePackId paramPack, const std::vector& argLocations) { @@ -3705,7 +3684,7 @@ void TypeChecker::checkArgumentList( if (!argLocations.empty()) location = {state.location.begin, argLocations.back().end}; - size_t minParams = getMinParameterCount(&state.log, paramPack); + size_t minParams = getParameterExtents(&state.log, paramPack).first; state.reportError(TypeError{location, CountMismatch{minParams, std::distance(begin(argPack), end(argPack))}}); }; @@ -3804,7 +3783,7 @@ void TypeChecker::checkArgumentList( } // ok else { - size_t minParams = getMinParameterCount(&state.log, paramPack); + size_t minParams = getParameterExtents(&state.log, paramPack).first; std::optional tail = flatten(paramPack, state.log).second; bool isVariadic = tail && Luau::isVariadic(*tail); diff --git a/Analysis/src/TypePack.cpp b/Analysis/src/TypePack.cpp index d4544483..2fa9413a 100644 --- a/Analysis/src/TypePack.cpp +++ b/Analysis/src/TypePack.cpp @@ -8,6 +8,13 @@ namespace Luau { +BlockedTypePack::BlockedTypePack() + : index(++nextIndex) +{ +} + +size_t BlockedTypePack::nextIndex = 0; + TypePackVar::TypePackVar(const TypePackVariant& tp) : ty(tp) { diff --git a/Analysis/src/TypeUtils.cpp b/Analysis/src/TypeUtils.cpp index 60bca0a3..56fccecc 100644 --- a/Analysis/src/TypeUtils.cpp +++ b/Analysis/src/TypeUtils.cpp @@ -84,9 +84,8 @@ std::optional findTablePropertyRespectingMeta(ErrorVec& errors, TypeId t return std::nullopt; } -std::optional getIndexTypeFromType( - const ScopePtr& scope, ErrorVec& errors, TypeArena* arena, TypeId type, const std::string& prop, const Location& location, bool addErrors, - InternalErrorReporter& handle) +std::optional getIndexTypeFromType(const ScopePtr& scope, ErrorVec& errors, TypeArena* arena, TypeId type, const std::string& prop, + const Location& location, bool addErrors, InternalErrorReporter& handle) { type = follow(type); @@ -190,4 +189,33 @@ std::optional getIndexTypeFromType( return std::nullopt; } +std::pair> getParameterExtents(const TxnLog* log, TypePackId tp) +{ + size_t minCount = 0; + size_t optionalCount = 0; + + auto it = begin(tp, log); + auto endIter = end(tp); + + while (it != endIter) + { + TypeId ty = *it; + if (isOptional(ty)) + ++optionalCount; + else + { + minCount += optionalCount; + optionalCount = 0; + minCount++; + } + + ++it; + } + + if (it.tail()) + return {minCount, std::nullopt}; + else + return {minCount, minCount + optionalCount}; +} + } // namespace Luau diff --git a/Analysis/src/TypeVar.cpp b/Analysis/src/TypeVar.cpp index 8974f8c7..4abee0f6 100644 --- a/Analysis/src/TypeVar.cpp +++ b/Analysis/src/TypeVar.cpp @@ -24,9 +24,7 @@ LUAU_FASTINTVARIABLE(LuauTypeMaximumStringifierLength, 500) LUAU_FASTINTVARIABLE(LuauTableTypeMaximumStringifierLength, 0) LUAU_FASTINT(LuauTypeInferRecursionLimit) LUAU_FASTFLAG(LuauUnknownAndNeverType) -LUAU_FASTFLAGVARIABLE(LuauDeduceGmatchReturnTypes, false) LUAU_FASTFLAGVARIABLE(LuauMaybeGenericIntersectionTypes, false) -LUAU_FASTFLAGVARIABLE(LuauDeduceFindMatchReturnTypes, false) LUAU_FASTFLAGVARIABLE(LuauStringFormatArgumentErrorFix, false) namespace Luau @@ -446,8 +444,10 @@ BlockedTypeVar::BlockedTypeVar() int BlockedTypeVar::nextIndex = 0; -PendingExpansionTypeVar::PendingExpansionTypeVar(TypeFun fn, std::vector typeArguments, std::vector packArguments) - : fn(fn) +PendingExpansionTypeVar::PendingExpansionTypeVar( + std::optional prefix, AstName name, std::vector typeArguments, std::vector packArguments) + : prefix(prefix) + , name(name) , typeArguments(typeArguments) , packArguments(packArguments) , index(++nextIndex) @@ -787,8 +787,8 @@ TypeId SingletonTypes::makeStringMetatable() makeFunction(*arena, stringType, {}, {}, {stringType}, {}, {arena->addType(FunctionTypeVar{emptyPack, stringVariadicList})}); attachMagicFunction(gmatchFunc, magicFunctionGmatch); - const TypeId matchFunc = arena->addType(FunctionTypeVar{arena->addTypePack({stringType, stringType, optionalNumber}), - arena->addTypePack(TypePackVar{VariadicTypePack{FFlag::LuauDeduceFindMatchReturnTypes ? stringType : optionalString}})}); + const TypeId matchFunc = arena->addType( + FunctionTypeVar{arena->addTypePack({stringType, stringType, optionalNumber}), arena->addTypePack(TypePackVar{VariadicTypePack{stringType}})}); attachMagicFunction(matchFunc, magicFunctionMatch); const TypeId findFunc = arena->addType(FunctionTypeVar{arena->addTypePack({stringType, stringType, optionalNumber, optionalBoolean}), @@ -1221,9 +1221,6 @@ static std::vector parsePatternString(TypeChecker& typechecker, const ch static std::optional> magicFunctionGmatch( TypeChecker& typechecker, const ScopePtr& scope, const AstExprCall& expr, WithPredicate withPredicate) { - if (!FFlag::LuauDeduceGmatchReturnTypes) - return std::nullopt; - auto [paramPack, _predicates] = withPredicate; const auto& [params, tail] = flatten(paramPack); @@ -1256,9 +1253,6 @@ static std::optional> magicFunctionGmatch( static std::optional> magicFunctionMatch( TypeChecker& typechecker, const ScopePtr& scope, const AstExprCall& expr, WithPredicate withPredicate) { - if (!FFlag::LuauDeduceFindMatchReturnTypes) - return std::nullopt; - auto [paramPack, _predicates] = withPredicate; const auto& [params, tail] = flatten(paramPack); @@ -1295,9 +1289,6 @@ static std::optional> magicFunctionMatch( static std::optional> magicFunctionFind( TypeChecker& typechecker, const ScopePtr& scope, const AstExprCall& expr, WithPredicate withPredicate) { - if (!FFlag::LuauDeduceFindMatchReturnTypes) - return std::nullopt; - auto [paramPack, _predicates] = withPredicate; const auto& [params, tail] = flatten(paramPack); diff --git a/Analysis/src/TypedAllocator.cpp b/Analysis/src/TypedAllocator.cpp index 9ce8c3dc..c95c8eae 100644 --- a/Analysis/src/TypedAllocator.cpp +++ b/Analysis/src/TypedAllocator.cpp @@ -36,7 +36,9 @@ void* pagedAllocate(size_t size) { // By default we use operator new/delete instead of malloc/free so that they can be overridden externally if (!FFlag::DebugLuauFreezeArena) + { return ::operator new(size, std::nothrow); + } // On Windows, VirtualAlloc results in 64K granularity allocations; we allocate in chunks of ~32K so aligned_malloc is a little more efficient // On Linux, we must use mmap because using regular heap results in mprotect() fragmenting the page table and us bumping into 64K mmap limit. diff --git a/Analysis/src/Unifiable.cpp b/Analysis/src/Unifiable.cpp index 63d8647d..fa76e820 100644 --- a/Analysis/src/Unifiable.cpp +++ b/Analysis/src/Unifiable.cpp @@ -13,7 +13,8 @@ Free::Free(TypeLevel level) } Free::Free(Scope* scope) - : scope(scope) + : index(++nextIndex) + , scope(scope) { } diff --git a/Analysis/src/Unifier.cpp b/Analysis/src/Unifier.cpp index b5f58c83..b135cd0c 100644 --- a/Analysis/src/Unifier.cpp +++ b/Analysis/src/Unifier.cpp @@ -317,7 +317,8 @@ static std::optional> getTableMat return std::nullopt; } -Unifier::Unifier(TypeArena* types, Mode mode, NotNull scope, const Location& location, Variance variance, UnifierSharedState& sharedState, TxnLog* parentLog) +Unifier::Unifier(TypeArena* types, Mode mode, NotNull scope, const Location& location, Variance variance, UnifierSharedState& sharedState, + TxnLog* parentLog) : types(types) , mode(mode) , scope(scope) @@ -492,9 +493,9 @@ void Unifier::tryUnify_(TypeId subTy, TypeId superTy, bool isFunctionCall, bool if (log.get(subTy)) tryUnifyWithConstrainedSubTypeVar(subTy, superTy); - else if (const UnionTypeVar* uv = log.getMutable(subTy)) + else if (const UnionTypeVar* subUnion = log.getMutable(subTy)) { - tryUnifyUnionWithType(subTy, uv, superTy); + tryUnifyUnionWithType(subTy, subUnion, superTy); } else if (const UnionTypeVar* uv = log.getMutable(superTy)) { @@ -555,14 +556,14 @@ void Unifier::tryUnify_(TypeId subTy, TypeId superTy, bool isFunctionCall, bool log.popSeen(superTy, subTy); } -void Unifier::tryUnifyUnionWithType(TypeId subTy, const UnionTypeVar* uv, TypeId superTy) +void Unifier::tryUnifyUnionWithType(TypeId subTy, const UnionTypeVar* subUnion, TypeId superTy) { // A | B <: T if A <: T and B <: T bool failed = false; std::optional unificationTooComplex; std::optional firstFailedOption; - for (TypeId type : uv->options) + for (TypeId type : subUnion->options) { Unifier innerState = makeChildUnifier(); innerState.tryUnify_(type, superTy); @@ -608,9 +609,9 @@ void Unifier::tryUnifyUnionWithType(TypeId subTy, const UnionTypeVar* uv, TypeId } }; - if (auto utv = log.getMutable(superTy)) + if (auto superUnion = log.getMutable(superTy)) { - for (TypeId ty : utv) + for (TypeId ty : superUnion) tryBind(ty); } else diff --git a/Ast/include/Luau/Ast.h b/Ast/include/Luau/Ast.h index 612283fb..07051163 100644 --- a/Ast/include/Luau/Ast.h +++ b/Ast/include/Luau/Ast.h @@ -1276,6 +1276,16 @@ public: }; AstName getIdentifier(AstExpr*); +Location getLocation(const AstTypeList& typeList); + +template // AstNode, AstExpr, AstLocal, etc +Location getLocation(AstArray array) +{ + if (0 == array.size) + return {}; + + return Location{array.data[0]->location.begin, array.data[array.size - 1]->location.end}; +} #undef LUAU_RTTI diff --git a/Ast/include/Luau/Parser.h b/Ast/include/Luau/Parser.h index 956fcf64..848d7117 100644 --- a/Ast/include/Luau/Parser.h +++ b/Ast/include/Luau/Parser.h @@ -304,6 +304,12 @@ private: AstExprError* reportExprError(const Location& location, const AstArray& expressions, const char* format, ...) LUAU_PRINTF_ATTR(4, 5); AstTypeError* reportTypeAnnotationError(const Location& location, const AstArray& types, bool isMissing, const char* format, ...) LUAU_PRINTF_ATTR(5, 6); + // `parseErrorLocation` is associated with the parser error + // `astErrorLocation` is associated with the AstTypeError created + // It can be useful to have different error locations so that the parse error can include the next lexeme, while the AstTypeError can precisely + // define the location (possibly of zero size) where a type annotation is expected. + AstTypeError* reportMissingTypeAnnotationError(const Location& parseErrorLocation, const Location& astErrorLocation, const char* format, ...) + LUAU_PRINTF_ATTR(4, 5); AstExpr* reportFunctionArgsError(AstExpr* func, bool self); void reportAmbiguousCallError(); diff --git a/Ast/src/Ast.cpp b/Ast/src/Ast.cpp index 8291a5b1..cbed8bae 100644 --- a/Ast/src/Ast.cpp +++ b/Ast/src/Ast.cpp @@ -952,4 +952,17 @@ AstName getIdentifier(AstExpr* node) return AstName(); } +Location getLocation(const AstTypeList& typeList) +{ + Location result; + if (typeList.types.size) + { + result = Location{typeList.types.data[0]->location, typeList.types.data[typeList.types.size - 1]->location}; + } + if (typeList.tailType) + result.end = typeList.tailType->location.end; + + return result; +} + } // namespace Luau diff --git a/Ast/src/Lexer.cpp b/Ast/src/Lexer.cpp index b4db8bdf..d93f2ccb 100644 --- a/Ast/src/Lexer.cpp +++ b/Ast/src/Lexer.cpp @@ -91,18 +91,8 @@ Lexeme::Lexeme(const Location& location, Type type, const char* data, size_t siz , length(unsigned(size)) , data(data) { - LUAU_ASSERT( - type == RawString - || type == QuotedString - || type == InterpStringBegin - || type == InterpStringMid - || type == InterpStringEnd - || type == InterpStringSimple - || type == BrokenInterpDoubleBrace - || type == Number - || type == Comment - || type == BlockComment - ); + LUAU_ASSERT(type == RawString || type == QuotedString || type == InterpStringBegin || type == InterpStringMid || type == InterpStringEnd || + type == InterpStringSimple || type == BrokenInterpDoubleBrace || type == Number || type == Comment || type == BlockComment); } Lexeme::Lexeme(const Location& location, Type type, const char* name) @@ -644,7 +634,8 @@ Lexeme Lexer::readInterpolatedStringSection(Position start, Lexeme::Type formatT if (peekch(1) == '{') { - Lexeme brokenDoubleBrace = Lexeme(Location(start, position()), Lexeme::BrokenInterpDoubleBrace, &buffer[startOffset], offset - startOffset); + Lexeme brokenDoubleBrace = + Lexeme(Location(start, position()), Lexeme::BrokenInterpDoubleBrace, &buffer[startOffset], offset - startOffset); consume(); consume(); return brokenDoubleBrace; diff --git a/Ast/src/Parser.cpp b/Ast/src/Parser.cpp index b6de27da..0914054f 100644 --- a/Ast/src/Parser.cpp +++ b/Ast/src/Parser.cpp @@ -24,6 +24,7 @@ LUAU_FASTFLAGVARIABLE(LuauLintParseIntegerIssues, false) LUAU_DYNAMIC_FASTFLAGVARIABLE(LuaReportParseIntegerIssues, false) LUAU_FASTFLAGVARIABLE(LuauInterpolatedStringBaseSupport, false) +LUAU_FASTFLAGVARIABLE(LuauTypeAnnotationLocationChange, false) bool lua_telemetry_parsed_out_of_range_bin_integer = false; bool lua_telemetry_parsed_out_of_range_hex_integer = false; @@ -1564,44 +1565,43 @@ AstTypeOrPack Parser::parseSimpleTypeAnnotation(bool allowPack) { incrementRecursionCounter("type annotation"); - Location begin = lexer.current().location; + Location start = lexer.current().location; if (lexer.current().type == Lexeme::ReservedNil) { nextLexeme(); - return {allocator.alloc(begin, std::nullopt, nameNil), {}}; + return {allocator.alloc(start, std::nullopt, nameNil), {}}; } else if (lexer.current().type == Lexeme::ReservedTrue) { nextLexeme(); - return {allocator.alloc(begin, true)}; + return {allocator.alloc(start, true)}; } else if (lexer.current().type == Lexeme::ReservedFalse) { nextLexeme(); - return {allocator.alloc(begin, false)}; + return {allocator.alloc(start, false)}; } else if (lexer.current().type == Lexeme::RawString || lexer.current().type == Lexeme::QuotedString) { if (std::optional> value = parseCharArray()) { AstArray svalue = *value; - return {allocator.alloc(begin, svalue)}; + return {allocator.alloc(start, svalue)}; } else - return {reportTypeAnnotationError(begin, {}, /*isMissing*/ false, "String literal contains malformed escape sequence")}; + return {reportTypeAnnotationError(start, {}, /*isMissing*/ false, "String literal contains malformed escape sequence")}; } else if (lexer.current().type == Lexeme::InterpStringBegin || lexer.current().type == Lexeme::InterpStringSimple) { parseInterpString(); - return {reportTypeAnnotationError(begin, {}, /*isMissing*/ false, "Interpolated string literals cannot be used as types")}; + return {reportTypeAnnotationError(start, {}, /*isMissing*/ false, "Interpolated string literals cannot be used as types")}; } else if (lexer.current().type == Lexeme::BrokenString) { - Location location = lexer.current().location; nextLexeme(); - return {reportTypeAnnotationError(location, {}, /*isMissing*/ false, "Malformed string")}; + return {reportTypeAnnotationError(start, {}, /*isMissing*/ false, "Malformed string")}; } else if (lexer.current().type == Lexeme::Name) { @@ -1632,7 +1632,7 @@ AstTypeOrPack Parser::parseSimpleTypeAnnotation(bool allowPack) expectMatchAndConsume(')', typeofBegin); - return {allocator.alloc(Location(begin, end), expr), {}}; + return {allocator.alloc(Location(start, end), expr), {}}; } bool hasParameters = false; @@ -1646,7 +1646,7 @@ AstTypeOrPack Parser::parseSimpleTypeAnnotation(bool allowPack) Location end = lexer.previousLocation(); - return {allocator.alloc(Location(begin, end), prefix, name.name, hasParameters, parameters), {}}; + return {allocator.alloc(Location(start, end), prefix, name.name, hasParameters, parameters), {}}; } else if (lexer.current().type == '{') { @@ -1658,23 +1658,35 @@ AstTypeOrPack Parser::parseSimpleTypeAnnotation(bool allowPack) } else if (lexer.current().type == Lexeme::ReservedFunction) { - Location location = lexer.current().location; - nextLexeme(); - return {reportTypeAnnotationError(location, {}, /*isMissing*/ false, + return {reportTypeAnnotationError(start, {}, /*isMissing*/ false, "Using 'function' as a type annotation is not supported, consider replacing with a function type annotation e.g. '(...any) -> " "...any'"), {}}; } else { - Location location = lexer.current().location; + if (FFlag::LuauTypeAnnotationLocationChange) + { + // For a missing type annotation, capture 'space' between last token and the next one + Location astErrorlocation(lexer.previousLocation().end, start.begin); + // The parse error includes the next lexeme to make it easier to display where the error is (e.g. in an IDE or a CLI error message). + // Including the current lexeme also makes the parse error consistent with other parse errors returned by Luau. + Location parseErrorLocation(lexer.previousLocation().end, start.end); + return { + reportMissingTypeAnnotationError(parseErrorLocation, astErrorlocation, "Expected type, got %s", lexer.current().toString().c_str()), + {}}; + } + else + { + Location location = lexer.current().location; - // For a missing type annotation, capture 'space' between last token and the next one - location = Location(lexer.previousLocation().end, lexer.current().location.begin); + // For a missing type annotation, capture 'space' between last token and the next one + location = Location(lexer.previousLocation().end, lexer.current().location.begin); - return {reportTypeAnnotationError(location, {}, /*isMissing*/ true, "Expected type, got %s", lexer.current().toString().c_str()), {}}; + return {reportTypeAnnotationError(location, {}, /*isMissing*/ true, "Expected type, got %s", lexer.current().toString().c_str()), {}}; + } } } @@ -2245,7 +2257,8 @@ AstExpr* Parser::parseSimpleExpr() { return parseNumber(); } - else if (lexer.current().type == Lexeme::RawString || lexer.current().type == Lexeme::QuotedString || (FFlag::LuauInterpolatedStringBaseSupport && lexer.current().type == Lexeme::InterpStringSimple)) + else if (lexer.current().type == Lexeme::RawString || lexer.current().type == Lexeme::QuotedString || + (FFlag::LuauInterpolatedStringBaseSupport && lexer.current().type == Lexeme::InterpStringSimple)) { return parseString(); } @@ -2653,7 +2666,8 @@ AstArray Parser::parseTypeParams() std::optional> Parser::parseCharArray() { - LUAU_ASSERT(lexer.current().type == Lexeme::QuotedString || lexer.current().type == Lexeme::RawString || lexer.current().type == Lexeme::InterpStringSimple); + LUAU_ASSERT(lexer.current().type == Lexeme::QuotedString || lexer.current().type == Lexeme::RawString || + lexer.current().type == Lexeme::InterpStringSimple); scratchData.assign(lexer.current().data, lexer.current().length); @@ -2691,14 +2705,11 @@ AstExpr* Parser::parseInterpString() Location startLocation = lexer.current().location; - do { + do + { Lexeme currentLexeme = lexer.current(); - LUAU_ASSERT( - currentLexeme.type == Lexeme::InterpStringBegin - || currentLexeme.type == Lexeme::InterpStringMid - || currentLexeme.type == Lexeme::InterpStringEnd - || currentLexeme.type == Lexeme::InterpStringSimple - ); + LUAU_ASSERT(currentLexeme.type == Lexeme::InterpStringBegin || currentLexeme.type == Lexeme::InterpStringMid || + currentLexeme.type == Lexeme::InterpStringEnd || currentLexeme.type == Lexeme::InterpStringSimple); Location location = currentLexeme.location; @@ -2973,8 +2984,7 @@ bool Parser::expectMatchEndAndConsume(Lexeme::Type type, const MatchLexeme& begi { // If the token matches on a different line and a different column, it suggests misleading indentation // This can be used to pinpoint the problem location for a possible future *actual* mismatch - if (lexer.current().location.begin.line != begin.position.line && - lexer.current().location.begin.column != begin.position.column && + if (lexer.current().location.begin.line != begin.position.line && lexer.current().location.begin.column != begin.position.column && endMismatchSuspect.position.line < begin.position.line) // Only replace the previous suspect with more recent suspects { endMismatchSuspect = begin; @@ -3108,6 +3118,13 @@ AstExprError* Parser::reportExprError(const Location& location, const AstArray& types, bool isMissing, const char* format, ...) { + if (FFlag::LuauTypeAnnotationLocationChange) + { + // Missing type annotations should be using `reportMissingTypeAnnotationError` when LuauTypeAnnotationLocationChange is enabled + // Note: `isMissing` can be removed once FFlag::LuauTypeAnnotationLocationChange is removed since it will always be true. + LUAU_ASSERT(!isMissing); + } + va_list args; va_start(args, format); report(location, format, args); @@ -3116,6 +3133,18 @@ AstTypeError* Parser::reportTypeAnnotationError(const Location& location, const return allocator.alloc(location, types, isMissing, unsigned(parseErrors.size() - 1)); } +AstTypeError* Parser::reportMissingTypeAnnotationError(const Location& parseErrorLocation, const Location& astErrorLocation, const char* format, ...) +{ + LUAU_ASSERT(FFlag::LuauTypeAnnotationLocationChange); + + va_list args; + va_start(args, format); + report(parseErrorLocation, format, args); + va_end(args); + + return allocator.alloc(astErrorLocation, AstArray{}, true, unsigned(parseErrors.size() - 1)); +} + void Parser::nextLexeme() { Lexeme::Type type = lexer.next(/* skipComments= */ false, true).type; diff --git a/Common/include/Luau/Bytecode.h b/Common/include/Luau/Bytecode.h index d6660f05..decde93f 100644 --- a/Common/include/Luau/Bytecode.h +++ b/Common/include/Luau/Bytecode.h @@ -414,7 +414,7 @@ enum LuauBytecodeTag // Bytecode version; runtime supports [MIN, MAX], compiler emits TARGET by default but may emit a higher version when flags are enabled LBC_VERSION_MIN = 2, LBC_VERSION_MAX = 3, - LBC_VERSION_TARGET = 2, + LBC_VERSION_TARGET = 3, // Types of constant table entries LBC_CONSTANT_NIL = 0, LBC_CONSTANT_BOOLEAN, diff --git a/Common/include/Luau/ExperimentalFlags.h b/Common/include/Luau/ExperimentalFlags.h index 809c78da..ce47cd9a 100644 --- a/Common/include/Luau/ExperimentalFlags.h +++ b/Common/include/Luau/ExperimentalFlags.h @@ -13,7 +13,8 @@ inline bool isFlagExperimental(const char* flag) static const char* kList[] = { "LuauLowerBoundsCalculation", "LuauInterpolatedStringBaseSupport", - nullptr, // makes sure we always have at least one entry + // makes sure we always have at least one entry + nullptr, }; for (const char* item : kList) diff --git a/Compiler/src/BytecodeBuilder.cpp b/Compiler/src/BytecodeBuilder.cpp index 46ab2648..713d08cd 100644 --- a/Compiler/src/BytecodeBuilder.cpp +++ b/Compiler/src/BytecodeBuilder.cpp @@ -6,8 +6,6 @@ #include #include -LUAU_FASTFLAGVARIABLE(LuauCompileBytecodeV3, false) - namespace Luau { @@ -1079,9 +1077,6 @@ std::string BytecodeBuilder::getError(const std::string& message) uint8_t BytecodeBuilder::getVersion() { - if (FFlag::LuauCompileBytecodeV3) - return 3; - // This function usually returns LBC_VERSION_TARGET but may sometimes return a higher number (within LBC_VERSION_MIN/MAX) under fast flags return LBC_VERSION_TARGET; } diff --git a/Compiler/src/Compiler.cpp b/Compiler/src/Compiler.cpp index 4429e4cc..d44daf0c 100644 --- a/Compiler/src/Compiler.cpp +++ b/Compiler/src/Compiler.cpp @@ -25,14 +25,8 @@ LUAU_FASTINTVARIABLE(LuauCompileInlineThreshold, 25) LUAU_FASTINTVARIABLE(LuauCompileInlineThresholdMaxBoost, 300) LUAU_FASTINTVARIABLE(LuauCompileInlineDepth, 5) -LUAU_FASTFLAGVARIABLE(LuauCompileXEQ, false) - LUAU_FASTFLAG(LuauInterpolatedStringBaseSupport) -LUAU_FASTFLAGVARIABLE(LuauCompileOptimalAssignment, false) - -LUAU_FASTFLAGVARIABLE(LuauCompileExtractK, false) - namespace Luau { @@ -406,47 +400,29 @@ struct Compiler } } - void compileExprFastcallN(AstExprCall* expr, uint8_t target, uint8_t targetCount, bool targetTop, bool multRet, uint8_t regs, int bfid, int bfK = -1) + void compileExprFastcallN( + AstExprCall* expr, uint8_t target, uint8_t targetCount, bool targetTop, bool multRet, uint8_t regs, int bfid, int bfK = -1) { LUAU_ASSERT(!expr->self); LUAU_ASSERT(expr->args.size >= 1); LUAU_ASSERT(expr->args.size <= 2 || (bfid == LBF_BIT32_EXTRACTK && expr->args.size == 3)); LUAU_ASSERT(bfid == LBF_BIT32_EXTRACTK ? bfK >= 0 : bfK < 0); - LuauOpcode opc = expr->args.size == 1 ? LOP_FASTCALL1 : LOP_FASTCALL2; - - if (FFlag::LuauCompileExtractK) - { - opc = expr->args.size == 1 ? LOP_FASTCALL1 : (bfK >= 0 || isConstant(expr->args.data[1])) ? LOP_FASTCALL2K : LOP_FASTCALL2; - } + LuauOpcode opc = expr->args.size == 1 ? LOP_FASTCALL1 : (bfK >= 0 || isConstant(expr->args.data[1])) ? LOP_FASTCALL2K : LOP_FASTCALL2; uint32_t args[3] = {}; for (size_t i = 0; i < expr->args.size; ++i) { - if (FFlag::LuauCompileExtractK) + if (i > 0 && opc == LOP_FASTCALL2K) { - if (i > 0 && opc == LOP_FASTCALL2K) - { - int32_t cid = getConstantIndex(expr->args.data[i]); - if (cid < 0) - CompileError::raise(expr->location, "Exceeded constant limit; simplify the code to compile"); + int32_t cid = getConstantIndex(expr->args.data[i]); + if (cid < 0) + CompileError::raise(expr->location, "Exceeded constant limit; simplify the code to compile"); - args[i] = cid; - continue; // TODO: remove this and change if below to else if - } + args[i] = cid; } - else if (i > 0) - { - if (int32_t cid = getConstantIndex(expr->args.data[i]); cid >= 0) - { - opc = LOP_FASTCALL2K; - args[i] = cid; - break; - } - } - - if (int reg = getExprLocalReg(expr->args.data[i]); reg >= 0) + else if (int reg = getExprLocalReg(expr->args.data[i]); reg >= 0) { args[i] = uint8_t(reg); } @@ -468,24 +444,10 @@ struct Compiler // these FASTCALL variants. for (size_t i = 0; i < expr->args.size; ++i) { - if (FFlag::LuauCompileExtractK) - { - if (i > 0 && opc == LOP_FASTCALL2K) - emitLoadK(uint8_t(regs + 1 + i), args[i]); - else if (args[i] != regs + 1 + i) - bytecode.emitABC(LOP_MOVE, uint8_t(regs + 1 + i), uint8_t(args[i]), 0); - } - else - { - if (i > 0 && opc == LOP_FASTCALL2K) - { - emitLoadK(uint8_t(regs + 1 + i), args[i]); - break; - } - - if (args[i] != regs + 1 + i) - bytecode.emitABC(LOP_MOVE, uint8_t(regs + 1 + i), uint8_t(args[i]), 0); - } + if (i > 0 && opc == LOP_FASTCALL2K) + emitLoadK(uint8_t(regs + 1 + i), args[i]); + else if (args[i] != regs + 1 + i) + bytecode.emitABC(LOP_MOVE, uint8_t(regs + 1 + i), uint8_t(args[i]), 0); } // note, these instructions are normally not executed and are used as a fallback for FASTCALL @@ -758,7 +720,7 @@ struct Compiler } // Optimization: for bit32.extract with constant in-range f/w we compile using FASTCALL2K and a special builtin - if (FFlag::LuauCompileExtractK && bfid == LBF_BIT32_EXTRACT && expr->args.size == 3 && isConstant(expr->args.data[1]) && isConstant(expr->args.data[2])) + if (bfid == LBF_BIT32_EXTRACT && expr->args.size == 3 && isConstant(expr->args.data[1]) && isConstant(expr->args.data[2])) { Constant fc = getConstant(expr->args.data[1]); Constant wc = getConstant(expr->args.data[2]); @@ -1080,102 +1042,64 @@ struct Compiler std::swap(left, right); } - if (FFlag::LuauCompileXEQ) + uint8_t rl = compileExprAuto(left, rs); + + if (isEq && operandIsConstant) { - uint8_t rl = compileExprAuto(left, rs); + const Constant* cv = constants.find(right); + LUAU_ASSERT(cv && cv->type != Constant::Type_Unknown); - if (isEq && operandIsConstant) + LuauOpcode opc = LOP_NOP; + int32_t cid = -1; + uint32_t flip = (expr->op == AstExprBinary::CompareEq) == not_ ? 0x80000000 : 0; + + switch (cv->type) { - const Constant* cv = constants.find(right); - LUAU_ASSERT(cv && cv->type != Constant::Type_Unknown); + case Constant::Type_Nil: + opc = LOP_JUMPXEQKNIL; + cid = 0; + break; - LuauOpcode opc = LOP_NOP; - int32_t cid = -1; - uint32_t flip = (expr->op == AstExprBinary::CompareEq) == not_ ? 0x80000000 : 0; + case Constant::Type_Boolean: + opc = LOP_JUMPXEQKB; + cid = cv->valueBoolean; + break; - switch (cv->type) - { - case Constant::Type_Nil: - opc = LOP_JUMPXEQKNIL; - cid = 0; - break; + case Constant::Type_Number: + opc = LOP_JUMPXEQKN; + cid = getConstantIndex(right); + break; - case Constant::Type_Boolean: - opc = LOP_JUMPXEQKB; - cid = cv->valueBoolean; - break; + case Constant::Type_String: + opc = LOP_JUMPXEQKS; + cid = getConstantIndex(right); + break; - case Constant::Type_Number: - opc = LOP_JUMPXEQKN; - cid = getConstantIndex(right); - break; - - case Constant::Type_String: - opc = LOP_JUMPXEQKS; - cid = getConstantIndex(right); - break; - - default: - LUAU_ASSERT(!"Unexpected constant type"); - } - - if (cid < 0) - CompileError::raise(expr->location, "Exceeded constant limit; simplify the code to compile"); - - size_t jumpLabel = bytecode.emitLabel(); - - bytecode.emitAD(opc, rl, 0); - bytecode.emitAux(cid | flip); - - return jumpLabel; + default: + LUAU_ASSERT(!"Unexpected constant type"); } - else - { - LuauOpcode opc = getJumpOpCompare(expr->op, not_); - uint8_t rr = compileExprAuto(right, rs); + if (cid < 0) + CompileError::raise(expr->location, "Exceeded constant limit; simplify the code to compile"); - size_t jumpLabel = bytecode.emitLabel(); + size_t jumpLabel = bytecode.emitLabel(); - if (expr->op == AstExprBinary::CompareGt || expr->op == AstExprBinary::CompareGe) - { - bytecode.emitAD(opc, rr, 0); - bytecode.emitAux(rl); - } - else - { - bytecode.emitAD(opc, rl, 0); - bytecode.emitAux(rr); - } + bytecode.emitAD(opc, rl, 0); + bytecode.emitAux(cid | flip); - return jumpLabel; - } + return jumpLabel; } else { LuauOpcode opc = getJumpOpCompare(expr->op, not_); - uint8_t rl = compileExprAuto(left, rs); - int32_t rr = -1; - - if (isEq && operandIsConstant) - { - if (opc == LOP_JUMPIFEQ) - opc = LOP_JUMPIFEQK; - else if (opc == LOP_JUMPIFNOTEQ) - opc = LOP_JUMPIFNOTEQK; - - rr = getConstantIndex(right); - LUAU_ASSERT(rr >= 0); - } - else - rr = compileExprAuto(right, rs); + uint8_t rr = compileExprAuto(right, rs); size_t jumpLabel = bytecode.emitLabel(); if (expr->op == AstExprBinary::CompareGt || expr->op == AstExprBinary::CompareGe) { - bytecode.emitAD(opc, uint8_t(rr), 0); + bytecode.emitAD(opc, rr, 0); bytecode.emitAux(rl); } else @@ -2979,62 +2903,6 @@ struct Compiler loops.pop_back(); } - void resolveAssignConflicts(AstStat* stat, std::vector& vars) - { - LUAU_ASSERT(!FFlag::LuauCompileOptimalAssignment); - - // regsUsed[i] is true if we have assigned the register during earlier assignments - // regsRemap[i] is set to the register where the original (pre-assignment) copy was made - // note: regsRemap is uninitialized intentionally to speed small assignments up; regsRemap[i] is valid iff regsUsed[i] - std::bitset<256> regsUsed; - uint8_t regsRemap[256]; - - for (size_t i = 0; i < vars.size(); ++i) - { - LValue& li = vars[i]; - - if (li.kind == LValue::Kind_Local) - { - if (!regsUsed[li.reg]) - { - regsUsed[li.reg] = true; - regsRemap[li.reg] = li.reg; - } - } - else if (li.kind == LValue::Kind_IndexName || li.kind == LValue::Kind_IndexNumber || li.kind == LValue::Kind_IndexExpr) - { - // we're looking for assignments before this one that invalidate any of the registers involved - if (regsUsed[li.reg]) - { - // the register may have been evacuated previously, but if it wasn't - move it now - if (regsRemap[li.reg] == li.reg) - { - uint8_t reg = allocReg(stat, 1); - bytecode.emitABC(LOP_MOVE, reg, li.reg, 0); - - regsRemap[li.reg] = reg; - } - - li.reg = regsRemap[li.reg]; - } - - if (li.kind == LValue::Kind_IndexExpr && regsUsed[li.index]) - { - // the register may have been evacuated previously, but if it wasn't - move it now - if (regsRemap[li.index] == li.index) - { - uint8_t reg = allocReg(stat, 1); - bytecode.emitABC(LOP_MOVE, reg, li.index, 0); - - regsRemap[li.index] = reg; - } - - li.index = regsRemap[li.index]; - } - } - } - } - struct Assignment { LValue lvalue; @@ -3146,111 +3014,82 @@ struct Compiler return; } - if (FFlag::LuauCompileOptimalAssignment) + // compute all l-values: note that this doesn't assign anything yet but it allocates registers and computes complex expressions on the + // left hand side - for example, in "a[expr] = foo" expr will get evaluated here + std::vector vars(stat->vars.size); + + for (size_t i = 0; i < stat->vars.size; ++i) + vars[i].lvalue = compileLValue(stat->vars.data[i], rs); + + // perform conflict resolution: if any expression refers to a local that is assigned before evaluating it, we assign to a temporary + // register after this, vars[i].conflictReg is set for locals that need to be assigned in the second pass + resolveAssignConflicts(stat, vars, stat->values); + + // compute rhs into (mostly) fresh registers + // note that when the lhs assigment is a local, we evaluate directly into that register + // this is possible because resolveAssignConflicts renamed conflicting locals into temporaries + // after this, vars[i].valueReg is set to a register with the value for *all* vars, but some have already been assigned + for (size_t i = 0; i < stat->vars.size && i < stat->values.size; ++i) { - // compute all l-values: note that this doesn't assign anything yet but it allocates registers and computes complex expressions on the - // left hand side - for example, in "a[expr] = foo" expr will get evaluated here - std::vector vars(stat->vars.size); + AstExpr* value = stat->values.data[i]; - for (size_t i = 0; i < stat->vars.size; ++i) - vars[i].lvalue = compileLValue(stat->vars.data[i], rs); - - // perform conflict resolution: if any expression refers to a local that is assigned before evaluating it, we assign to a temporary - // register after this, vars[i].conflictReg is set for locals that need to be assigned in the second pass - resolveAssignConflicts(stat, vars, stat->values); - - // compute rhs into (mostly) fresh registers - // note that when the lhs assigment is a local, we evaluate directly into that register - // this is possible because resolveAssignConflicts renamed conflicting locals into temporaries - // after this, vars[i].valueReg is set to a register with the value for *all* vars, but some have already been assigned - for (size_t i = 0; i < stat->vars.size && i < stat->values.size; ++i) + if (i + 1 == stat->values.size && stat->vars.size > stat->values.size) { - AstExpr* value = stat->values.data[i]; + // allocate a consecutive range of regs for all remaining vars and compute everything into temps + // note, this also handles trailing nils + uint8_t rest = uint8_t(stat->vars.size - stat->values.size + 1); + uint8_t temp = allocReg(stat, rest); - if (i + 1 == stat->values.size && stat->vars.size > stat->values.size) + compileExprTempN(value, temp, rest, /* targetTop= */ true); + + for (size_t j = i; j < stat->vars.size; ++j) + vars[j].valueReg = uint8_t(temp + (j - i)); + } + else + { + Assignment& var = vars[i]; + + // if target is a local, use compileExpr directly to target + if (var.lvalue.kind == LValue::Kind_Local) { - // allocate a consecutive range of regs for all remaining vars and compute everything into temps - // note, this also handles trailing nils - uint8_t rest = uint8_t(stat->vars.size - stat->values.size + 1); - uint8_t temp = allocReg(stat, rest); + var.valueReg = (var.conflictReg == kInvalidReg) ? var.lvalue.reg : var.conflictReg; - compileExprTempN(value, temp, rest, /* targetTop= */ true); - - for (size_t j = i; j < stat->vars.size; ++j) - vars[j].valueReg = uint8_t(temp + (j - i)); + compileExpr(stat->values.data[i], var.valueReg); } else { - Assignment& var = vars[i]; - - // if target is a local, use compileExpr directly to target - if (var.lvalue.kind == LValue::Kind_Local) - { - var.valueReg = (var.conflictReg == kInvalidReg) ? var.lvalue.reg : var.conflictReg; - - compileExpr(stat->values.data[i], var.valueReg); - } - else - { - var.valueReg = compileExprAuto(stat->values.data[i], rs); - } + var.valueReg = compileExprAuto(stat->values.data[i], rs); } } - - // compute expressions with side effects for lulz - for (size_t i = stat->vars.size; i < stat->values.size; ++i) - { - RegScope rsi(this); - compileExprAuto(stat->values.data[i], rsi); - } - - // almost done... let's assign everything left to right, noting that locals were either written-to directly, or will be written-to in a - // separate pass to avoid conflicts - for (const Assignment& var : vars) - { - LUAU_ASSERT(var.valueReg != kInvalidReg); - - if (var.lvalue.kind != LValue::Kind_Local) - { - setDebugLine(var.lvalue.location); - compileAssign(var.lvalue, var.valueReg); - } - } - - // all regular local writes are done by the prior loops by computing result directly into target, so this just handles conflicts OR - // local copies from temporary registers in multret context, since in that case we have to allocate consecutive temporaries - for (const Assignment& var : vars) - { - if (var.lvalue.kind == LValue::Kind_Local && var.valueReg != var.lvalue.reg) - bytecode.emitABC(LOP_MOVE, var.lvalue.reg, var.valueReg, 0); - } } - else + + // compute expressions with side effects for lulz + for (size_t i = stat->vars.size; i < stat->values.size; ++i) { - // compute all l-values: note that this doesn't assign anything yet but it allocates registers and computes complex expressions on the - // left hand side for example, in "a[expr] = foo" expr will get evaluated here - std::vector vars(stat->vars.size); + RegScope rsi(this); + compileExprAuto(stat->values.data[i], rsi); + } - for (size_t i = 0; i < stat->vars.size; ++i) - vars[i] = compileLValue(stat->vars.data[i], rs); + // almost done... let's assign everything left to right, noting that locals were either written-to directly, or will be written-to in a + // separate pass to avoid conflicts + for (const Assignment& var : vars) + { + LUAU_ASSERT(var.valueReg != kInvalidReg); - // perform conflict resolution: if any lvalue refers to a local reg that will be reassigned before that, we save the local variable in a - // temporary reg - resolveAssignConflicts(stat, vars); - - // compute values into temporaries - uint8_t regs = allocReg(stat, unsigned(stat->vars.size)); - - compileExprListTemp(stat->values, regs, uint8_t(stat->vars.size), /* targetTop= */ true); - - // assign variables that have associated values; note that if we have fewer values than variables, we'll assign nil because - // compileExprListTemp will generate nils - for (size_t i = 0; i < stat->vars.size; ++i) + if (var.lvalue.kind != LValue::Kind_Local) { - setDebugLine(stat->vars.data[i]); - compileAssign(vars[i], uint8_t(regs + i)); + setDebugLine(var.lvalue.location); + compileAssign(var.lvalue, var.valueReg); } } + + // all regular local writes are done by the prior loops by computing result directly into target, so this just handles conflicts OR + // local copies from temporary registers in multret context, since in that case we have to allocate consecutive temporaries + for (const Assignment& var : vars) + { + if (var.lvalue.kind == LValue::Kind_Local && var.valueReg != var.lvalue.reg) + bytecode.emitABC(LOP_MOVE, var.lvalue.reg, var.valueReg, 0); + } } void compileStatCompoundAssign(AstStatCompoundAssign* stat) diff --git a/VM/src/lbuiltins.cpp b/VM/src/lbuiltins.cpp index 422c82b1..c16e5aa7 100644 --- a/VM/src/lbuiltins.cpp +++ b/VM/src/lbuiltins.cpp @@ -15,8 +15,6 @@ #include #endif -LUAU_FASTFLAGVARIABLE(LuauFasterBit32NoWidth, false) - // luauF functions implement FASTCALL instruction that performs a direct execution of some builtin functions from the VM // The rule of thumb is that FASTCALL functions can not call user code, yield, fail, or reallocate stack. // If types of the arguments mismatch, luauF_* needs to return -1 and the execution will fall back to the usual call path @@ -602,7 +600,7 @@ static int luauF_btest(lua_State* L, StkId res, TValue* arg0, int nresults, StkI static int luauF_extract(lua_State* L, StkId res, TValue* arg0, int nresults, StkId args, int nparams) { - if (nparams >= (3 - FFlag::LuauFasterBit32NoWidth) && nresults <= 1 && ttisnumber(arg0) && ttisnumber(args)) + if (nparams >= 2 && nresults <= 1 && ttisnumber(arg0) && ttisnumber(args)) { double a1 = nvalue(arg0); double a2 = nvalue(args); @@ -693,7 +691,7 @@ static int luauF_lshift(lua_State* L, StkId res, TValue* arg0, int nresults, Stk static int luauF_replace(lua_State* L, StkId res, TValue* arg0, int nresults, StkId args, int nparams) { - if (nparams >= (4 - FFlag::LuauFasterBit32NoWidth) && nresults <= 1 && ttisnumber(arg0) && ttisnumber(args) && ttisnumber(args + 1)) + if (nparams >= 3 && nresults <= 1 && ttisnumber(arg0) && ttisnumber(args) && ttisnumber(args + 1)) { double a1 = nvalue(arg0); double a2 = nvalue(args); diff --git a/VM/src/lstrlib.cpp b/VM/src/lstrlib.cpp index b3ea1094..192ea0b5 100644 --- a/VM/src/lstrlib.cpp +++ b/VM/src/lstrlib.cpp @@ -8,8 +8,6 @@ #include #include -LUAU_FASTFLAGVARIABLE(LuauTostringFormatSpecifier, false); - // macro to `unsign' a character #define uchar(c) ((unsigned char)(c)) @@ -1036,9 +1034,6 @@ static int str_format(lua_State* L) } case '*': { - if (!FFlag::LuauTostringFormatSpecifier) - luaL_error(L, "invalid option '%%*' to 'format'"); - if (formatItemSize != 1) luaL_error(L, "'%%*' does not take a form"); diff --git a/VM/src/lvmexecute.cpp b/VM/src/lvmexecute.cpp index 7306b055..aa1da8ae 100644 --- a/VM/src/lvmexecute.cpp +++ b/VM/src/lvmexecute.cpp @@ -3031,7 +3031,16 @@ static void luau_execute(lua_State* L) TValue* kv = VM_KV(aux & 0xffffff); LUAU_ASSERT(ttisnumber(kv)); +#if defined(__aarch64__) + // On several ARM chips (Apple M1/M2, Neoverse N1), comparing the result of a floating-point comparison is expensive, and a branch + // is much cheaper; on some 32-bit ARM chips (Cortex A53) the performance is about the same so we prefer less branchy variant there + if (aux >> 31) + pc += !(ttisnumber(ra) && nvalue(ra) == nvalue(kv)) ? LUAU_INSN_D(insn) : 1; + else + pc += (ttisnumber(ra) && nvalue(ra) == nvalue(kv)) ? LUAU_INSN_D(insn) : 1; +#else pc += int(ttisnumber(ra) && nvalue(ra) == nvalue(kv)) != (aux >> 31) ? LUAU_INSN_D(insn) : 1; +#endif LUAU_ASSERT(unsigned(pc - cl->l.p->code) < unsigned(cl->l.p->sizecode)); VM_NEXT(); } diff --git a/tests/AssemblyBuilderX64.test.cpp b/tests/AssemblyBuilderX64.test.cpp index 3f75ebac..08f241ed 100644 --- a/tests/AssemblyBuilderX64.test.cpp +++ b/tests/AssemblyBuilderX64.test.cpp @@ -4,7 +4,6 @@ #include "doctest.h" -#include #include using namespace Luau::CodeGen; @@ -22,7 +21,7 @@ std::string bytecodeAsArray(const std::vector& bytecode) class AssemblyBuilderX64Fixture { public: - void check(std::function f, std::vector code, std::vector data = {}) + void check(void (*f)(AssemblyBuilderX64& build), std::vector code, std::vector data = {}) { AssemblyBuilderX64 build(/* logText= */ false); diff --git a/tests/Compiler.test.cpp b/tests/Compiler.test.cpp index 0a3c6507..d8520a6e 100644 --- a/tests/Compiler.test.cpp +++ b/tests/Compiler.test.cpp @@ -1241,8 +1241,7 @@ TEST_CASE("InterpStringZeroCost") { ScopedFastFlag sff{"LuauInterpolatedStringBaseSupport", true}; - CHECK_EQ( - "\n" + compileFunction0(R"(local _ = `hello, {"world"}!`)"), + CHECK_EQ("\n" + compileFunction0(R"(local _ = `hello, {"world"}!`)"), R"( LOADK R1 K0 LOADK R3 K1 @@ -1250,16 +1249,14 @@ NAMECALL R1 R1 K2 CALL R1 2 1 MOVE R0 R1 RETURN R0 0 -)" - ); +)"); } TEST_CASE("InterpStringRegisterCleanup") { ScopedFastFlag sff{"LuauInterpolatedStringBaseSupport", true}; - CHECK_EQ( - "\n" + compileFunction0(R"( + CHECK_EQ("\n" + compileFunction0(R"( local a, b, c = nil, "um", "uh oh" a = `foo{"bar"}` print(a) @@ -1278,8 +1275,7 @@ GETIMPORT R3 6 MOVE R4 R0 CALL R3 1 0 RETURN R0 0 -)" - ); +)"); } TEST_CASE("ConstantFoldArith") @@ -2488,8 +2484,6 @@ end TEST_CASE("DebugLineInfoRepeatUntil") { - ScopedFastFlag sff("LuauCompileXEQ", true); - CHECK_EQ("\n" + compileFunction0Coverage(R"( local f = 0 repeat @@ -2834,8 +2828,6 @@ RETURN R0 0 TEST_CASE("AssignmentConflict") { - ScopedFastFlag sff("LuauCompileOptimalAssignment", true); - // assignments are left to right CHECK_EQ("\n" + compileFunction0("local a, b a, b = 1, 2"), R"( LOADNIL R0 @@ -3610,8 +3602,6 @@ RETURN R0 1 TEST_CASE("ConstantJumpCompare") { - ScopedFastFlag sff("LuauCompileXEQ", true); - CHECK_EQ("\n" + compileFunction0(R"( local obj = ... local b = obj == 1 @@ -6210,8 +6200,6 @@ L4: RETURN R0 -1 TEST_CASE("BuiltinFoldingMultret") { - ScopedFastFlag sff("LuauCompileXEQ", true); - CHECK_EQ("\n" + compileFunction(R"( local NoLanes: Lanes = --[[ ]] 0b0000000000000000000000000000000 local OffscreenLane: Lane = --[[ ]] 0b1000000000000000000000000000000 @@ -6350,8 +6338,6 @@ RETURN R2 1 TEST_CASE("MultipleAssignments") { - ScopedFastFlag sff("LuauCompileOptimalAssignment", true); - // order of assignments is left to right CHECK_EQ("\n" + compileFunction0(R"( local a, b @@ -6574,15 +6560,14 @@ RETURN R0 0 TEST_CASE("BuiltinExtractK") { - ScopedFastFlag sff("LuauCompileExtractK", true); - // below, K0 refers to a packed f+w constant for bit32.extractk builtin // K1 and K2 refer to 1 and 3 and are only used during fallback path CHECK_EQ("\n" + compileFunction0(R"( local v = ... return bit32.extract(v, 1, 3) -)"), R"( +)"), + R"( GETVARARGS R0 1 FASTCALL2K 59 R0 K0 L0 MOVE R2 R0 diff --git a/tests/Conformance.test.cpp b/tests/Conformance.test.cpp index f6f5b41f..a7ffb493 100644 --- a/tests/Conformance.test.cpp +++ b/tests/Conformance.test.cpp @@ -289,15 +289,12 @@ TEST_CASE("Clear") TEST_CASE("Strings") { - ScopedFastFlag sff{"LuauTostringFormatSpecifier", true}; - runConformance("strings.lua"); } TEST_CASE("StringInterp") { ScopedFastFlag sffInterpStrings{"LuauInterpolatedStringBaseSupport", true}; - ScopedFastFlag sffTostringFormat{"LuauTostringFormatSpecifier", true}; runConformance("stringinterp.lua"); } @@ -725,13 +722,16 @@ TEST_CASE("NewUserdataOverflow") StateRef globalState(luaL_newstate(), lua_close); lua_State* L = globalState.get(); - lua_pushcfunction(L, [](lua_State* L1) { - // The following userdata request might cause an overflow. - lua_newuserdatadtor(L1, SIZE_MAX, [](void* d){}); - // The overflow might segfault in the following call. - lua_getmetatable(L1, -1); - return 0; - }, nullptr); + lua_pushcfunction( + L, + [](lua_State* L1) { + // The following userdata request might cause an overflow. + lua_newuserdatadtor(L1, SIZE_MAX, [](void* d) {}); + // The overflow might segfault in the following call. + lua_getmetatable(L1, -1); + return 0; + }, + nullptr); CHECK(lua_pcall(L, 0, 0, 0) == LUA_ERRRUN); CHECK(strcmp(lua_tostring(L, -1), "memory allocation error: block too big") == 0); diff --git a/tests/ConstraintGraphBuilder.test.cpp b/tests/ConstraintGraphBuilder.test.cpp index 00c3309c..bbe29429 100644 --- a/tests/ConstraintGraphBuilder.test.cpp +++ b/tests/ConstraintGraphBuilder.test.cpp @@ -57,13 +57,12 @@ TEST_CASE_FIXTURE(ConstraintGraphBuilderFixture, "nil_primitive") auto constraints = collectConstraints(NotNull(cgb.rootScope)); ToStringOptions opts; - REQUIRE(5 <= constraints.size()); + REQUIRE(4 <= constraints.size()); CHECK("*blocked-1* ~ gen () -> (a...)" == toString(*constraints[0], opts)); - CHECK("*blocked-2* ~ inst *blocked-1*" == toString(*constraints[1], opts)); - CHECK("() -> (b...) <: *blocked-2*" == toString(*constraints[2], opts)); - CHECK("b... <: c" == toString(*constraints[3], opts)); - CHECK("nil <: a..." == toString(*constraints[4], opts)); + CHECK("call *blocked-1* with { result = *blocked-tp-1* }" == toString(*constraints[1], opts)); + CHECK("*blocked-tp-1* <: b" == toString(*constraints[2], opts)); + CHECK("nil <: a..." == toString(*constraints[3], opts)); } TEST_CASE_FIXTURE(ConstraintGraphBuilderFixture, "function_application") @@ -76,13 +75,12 @@ TEST_CASE_FIXTURE(ConstraintGraphBuilderFixture, "function_application") cgb.visit(block); auto constraints = collectConstraints(NotNull(cgb.rootScope)); - REQUIRE(4 == constraints.size()); + REQUIRE(3 == constraints.size()); ToStringOptions opts; CHECK("string <: a" == toString(*constraints[0], opts)); - CHECK("*blocked-1* ~ inst a" == toString(*constraints[1], opts)); - CHECK("(string) -> (b...) <: *blocked-1*" == toString(*constraints[2], opts)); - CHECK("b... <: c" == toString(*constraints[3], opts)); + CHECK("call a with { result = *blocked-tp-1* }" == toString(*constraints[1], opts)); + CHECK("*blocked-tp-1* <: b" == toString(*constraints[2], opts)); } TEST_CASE_FIXTURE(ConstraintGraphBuilderFixture, "local_function_definition") @@ -114,13 +112,12 @@ TEST_CASE_FIXTURE(ConstraintGraphBuilderFixture, "recursive_function") cgb.visit(block); auto constraints = collectConstraints(NotNull(cgb.rootScope)); - REQUIRE(4 == constraints.size()); + REQUIRE(3 == constraints.size()); ToStringOptions opts; CHECK("*blocked-1* ~ gen (a) -> (b...)" == toString(*constraints[0], opts)); - CHECK("*blocked-2* ~ inst (a) -> (b...)" == toString(*constraints[1], opts)); - CHECK("(a) -> (c...) <: *blocked-2*" == toString(*constraints[2], opts)); - CHECK("c... <: b..." == toString(*constraints[3], opts)); + CHECK("call (a) -> (b...) with { result = *blocked-tp-1* }" == toString(*constraints[1], opts)); + CHECK("*blocked-tp-1* <: b..." == toString(*constraints[2], opts)); } TEST_SUITE_END(); diff --git a/tests/ConstraintSolver.test.cpp b/tests/ConstraintSolver.test.cpp index e33f6570..2c489733 100644 --- a/tests/ConstraintSolver.test.cpp +++ b/tests/ConstraintSolver.test.cpp @@ -28,7 +28,8 @@ TEST_CASE_FIXTURE(ConstraintGraphBuilderFixture, "hello") cgb.visit(block); NotNull rootScope = NotNull(cgb.rootScope); - ConstraintSolver cs{&arena, rootScope}; + NullModuleResolver resolver; + ConstraintSolver cs{&arena, rootScope, "MainModule", NotNull(&resolver), {}}; cs.run(); @@ -48,7 +49,8 @@ TEST_CASE_FIXTURE(ConstraintGraphBuilderFixture, "generic_function") cgb.visit(block); NotNull rootScope = NotNull(cgb.rootScope); - ConstraintSolver cs{&arena, rootScope}; + NullModuleResolver resolver; + ConstraintSolver cs{&arena, rootScope, "MainModule", NotNull(&resolver), {}}; cs.run(); @@ -57,7 +59,6 @@ TEST_CASE_FIXTURE(ConstraintGraphBuilderFixture, "generic_function") CHECK("(a) -> a" == toString(idType)); } -#if 1 TEST_CASE_FIXTURE(ConstraintGraphBuilderFixture, "proper_let_generalization") { AstStatBlock* block = parse(R"( @@ -77,7 +78,8 @@ TEST_CASE_FIXTURE(ConstraintGraphBuilderFixture, "proper_let_generalization") ToStringOptions opts; - ConstraintSolver cs{&arena, rootScope}; + NullModuleResolver resolver; + ConstraintSolver cs{&arena, rootScope, "MainModule", NotNull(&resolver), {}}; cs.run(); @@ -85,6 +87,5 @@ TEST_CASE_FIXTURE(ConstraintGraphBuilderFixture, "proper_let_generalization") CHECK("(a) -> number" == toString(idType, opts)); } -#endif TEST_SUITE_END(); diff --git a/tests/Fixture.cpp b/tests/Fixture.cpp index 4051f851..476b7a2a 100644 --- a/tests/Fixture.cpp +++ b/tests/Fixture.cpp @@ -2,6 +2,8 @@ #include "Fixture.h" #include "Luau/AstQuery.h" +#include "Luau/ModuleResolver.h" +#include "Luau/NotNull.h" #include "Luau/Parser.h" #include "Luau/TypeVar.h" #include "Luau/TypeAttach.h" @@ -444,10 +446,11 @@ BuiltinsFixture::BuiltinsFixture(bool freeze, bool prepareAutocomplete) ConstraintGraphBuilderFixture::ConstraintGraphBuilderFixture() : Fixture() , mainModule(new Module) - , cgb(mainModuleName, mainModule, &arena, NotNull(&ice), frontend.getGlobalScope()) + , cgb(mainModuleName, mainModule, &arena, NotNull(&moduleResolver), NotNull(&ice), frontend.getGlobalScope()) , forceTheFlag{"DebugLuauDeferredConstraintResolution", true} { BlockedTypeVar::nextIndex = 0; + BlockedTypePack::nextIndex = 0; } ModuleName fromString(std::string_view name) diff --git a/tests/Fixture.h b/tests/Fixture.h index 956f95d0..8923b208 100644 --- a/tests/Fixture.h +++ b/tests/Fixture.h @@ -132,6 +132,7 @@ struct Fixture TestFileResolver fileResolver; TestConfigResolver configResolver; + NullModuleResolver moduleResolver; std::unique_ptr sourceModule; Frontend frontend; InternalErrorReporter ice; @@ -244,7 +245,7 @@ struct FindNthOccurenceOf : public AstVisitor * 2. Luau::query(Luau::query(block)) * 3. Luau::query(block, {nth(2)}) */ -template +template T* query(AstNode* node, const std::vector& nths = {nth(N)}) { static_assert(std::is_base_of_v, "T must be a derived class of AstNode"); diff --git a/tests/JsonEmitter.test.cpp b/tests/JsonEmitter.test.cpp index ebe83209..ff9a5955 100644 --- a/tests/JsonEmitter.test.cpp +++ b/tests/JsonEmitter.test.cpp @@ -94,7 +94,7 @@ TEST_CASE("write_optional") write(emitter, std::optional{true}); emitter.writeComma(); write(emitter, std::nullopt); - + CHECK(emitter.str() == "true,null"); } diff --git a/tests/Linter.test.cpp b/tests/Linter.test.cpp index a7d09e8f..b560c89e 100644 --- a/tests/Linter.test.cpp +++ b/tests/Linter.test.cpp @@ -1181,7 +1181,7 @@ s:match("[]") nons:match("[]") )~"); - CHECK_EQ(result.warnings.size(), 2); + REQUIRE_EQ(result.warnings.size(), 2); CHECK_EQ(result.warnings[0].text, "Invalid match pattern: expected ] at the end of the string to close a set"); CHECK_EQ(result.warnings[0].location.begin.line, 3); CHECK_EQ(result.warnings[1].text, "Invalid match pattern: expected ] at the end of the string to close a set"); @@ -1746,6 +1746,7 @@ local _ = not a == b local _ = not a ~= b local _ = not a <= b local _ = a <= b == 0 +local _ = a <= b <= 0 local _ = not a == not b -- weird but ok @@ -1760,11 +1761,12 @@ local _ = (a <= b) == 0 local _ = a <= (b == 0) )"); - REQUIRE_EQ(result.warnings.size(), 4); - CHECK_EQ(result.warnings[0].text, "not X == Y is equivalent to (not X) == Y; consider using X ~= Y, or wrap one of the expressions in parentheses to silence"); - CHECK_EQ(result.warnings[1].text, "not X ~= Y is equivalent to (not X) ~= Y; consider using X == Y, or wrap one of the expressions in parentheses to silence"); - CHECK_EQ(result.warnings[2].text, "not X <= Y is equivalent to (not X) <= Y; wrap one of the expressions in parentheses to silence"); - CHECK_EQ(result.warnings[3].text, "X <= Y == Z is equivalent to (X <= Y) == Z; wrap one of the expressions in parentheses to silence"); + REQUIRE_EQ(result.warnings.size(), 5); + CHECK_EQ(result.warnings[0].text, "not X == Y is equivalent to (not X) == Y; consider using X ~= Y, or add parentheses to silence"); + CHECK_EQ(result.warnings[1].text, "not X ~= Y is equivalent to (not X) ~= Y; consider using X == Y, or add parentheses to silence"); + CHECK_EQ(result.warnings[2].text, "not X <= Y is equivalent to (not X) <= Y; add parentheses to silence"); + CHECK_EQ(result.warnings[3].text, "X <= Y == Z is equivalent to (X <= Y) == Z; add parentheses to silence"); + CHECK_EQ(result.warnings[4].text, "X <= Y <= Z is equivalent to (X <= Y) <= Z; did you mean X <= Y and Y <= Z?"); } TEST_SUITE_END(); diff --git a/tests/Parser.test.cpp b/tests/Parser.test.cpp index 2dd47708..44a6b4ac 100644 --- a/tests/Parser.test.cpp +++ b/tests/Parser.test.cpp @@ -943,8 +943,7 @@ TEST_CASE_FIXTURE(Fixture, "parse_interpolated_string_without_end_brace") { ScopedFastFlag sff{"LuauInterpolatedStringBaseSupport", true}; - auto columnOfEndBraceError = [this](const char* code) - { + auto columnOfEndBraceError = [this](const char* code) { try { parse(code); @@ -1737,6 +1736,48 @@ TEST_CASE_FIXTURE(Fixture, "parse_error_type_annotation") matchParseError("local a : 2 = 2", "Expected type, got '2'"); } +TEST_CASE_FIXTURE(Fixture, "parse_error_missing_type_annotation") +{ + ScopedFastFlag LuauTypeAnnotationLocationChange{"LuauTypeAnnotationLocationChange", true}; + + { + ParseResult result = tryParse("local x:"); + CHECK(result.errors.size() == 1); + Position begin = result.errors[0].getLocation().begin; + Position end = result.errors[0].getLocation().end; + CHECK(begin.line == end.line); + int width = end.column - begin.column; + CHECK(width == 0); + CHECK(result.errors[0].getMessage() == "Expected type, got "); + } + + { + ParseResult result = tryParse(R"( +local x:=42 + )"); + CHECK(result.errors.size() == 1); + Position begin = result.errors[0].getLocation().begin; + Position end = result.errors[0].getLocation().end; + CHECK(begin.line == end.line); + int width = end.column - begin.column; + CHECK(width == 1); // Length of `=` + CHECK(result.errors[0].getMessage() == "Expected type, got '='"); + } + + { + ParseResult result = tryParse(R"( +function func():end + )"); + CHECK(result.errors.size() == 1); + Position begin = result.errors[0].getLocation().begin; + Position end = result.errors[0].getLocation().end; + CHECK(begin.line == end.line); + int width = end.column - begin.column; + CHECK(width == 3); // Length of `end` + CHECK(result.errors[0].getMessage() == "Expected type, got 'end'"); + } +} + TEST_CASE_FIXTURE(Fixture, "parse_declarations") { AstStatBlock* stat = parseEx(R"( diff --git a/tests/TypeInfer.anyerror.test.cpp b/tests/TypeInfer.anyerror.test.cpp index f4766104..8c6f2e4f 100644 --- a/tests/TypeInfer.anyerror.test.cpp +++ b/tests/TypeInfer.anyerror.test.cpp @@ -346,4 +346,17 @@ TEST_CASE_FIXTURE(Fixture, "prop_access_on_any_with_other_options") LUAU_REQUIRE_NO_ERRORS(result); } +TEST_CASE_FIXTURE(BuiltinsFixture, "union_of_types_regression_test") +{ + ScopedFastFlag LuauUnionOfTypesFollow{"LuauUnionOfTypesFollow", true}; + + CheckResult result = check(R"( +--!strict +local stat +stat = stat and tonumber(stat) or stat + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.builtins.test.cpp b/tests/TypeInfer.builtins.test.cpp index 12fb4aa2..07a04363 100644 --- a/tests/TypeInfer.builtins.test.cpp +++ b/tests/TypeInfer.builtins.test.cpp @@ -1061,7 +1061,6 @@ end TEST_CASE_FIXTURE(BuiltinsFixture, "gmatch_capture_types") { - ScopedFastFlag sffs{"LuauDeduceGmatchReturnTypes", true}; CheckResult result = check(R"END( local a, b, c = string.gmatch("This is a string", "(.()(%a+))")() )END"); @@ -1075,7 +1074,6 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "gmatch_capture_types") TEST_CASE_FIXTURE(BuiltinsFixture, "gmatch_capture_types2") { - ScopedFastFlag sffs{"LuauDeduceGmatchReturnTypes", true}; CheckResult result = check(R"END( local a, b, c = ("This is a string"):gmatch("(.()(%a+))")() )END"); @@ -1089,7 +1087,6 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "gmatch_capture_types2") TEST_CASE_FIXTURE(BuiltinsFixture, "gmatch_capture_types_default_capture") { - ScopedFastFlag sffs{"LuauDeduceGmatchReturnTypes", true}; CheckResult result = check(R"END( local a, b, c, d = string.gmatch("T(his)() is a string", ".")() )END"); @@ -1107,7 +1104,6 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "gmatch_capture_types_default_capture") TEST_CASE_FIXTURE(BuiltinsFixture, "gmatch_capture_types_balanced_escaped_parens") { - ScopedFastFlag sffs{"LuauDeduceGmatchReturnTypes", true}; CheckResult result = check(R"END( local a, b, c, d = string.gmatch("T(his) is a string", "((.)%b()())")() )END"); @@ -1127,7 +1123,6 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "gmatch_capture_types_balanced_escaped_parens TEST_CASE_FIXTURE(BuiltinsFixture, "gmatch_capture_types_parens_in_sets_are_ignored") { - ScopedFastFlag sffs{"LuauDeduceGmatchReturnTypes", true}; CheckResult result = check(R"END( local a, b, c = string.gmatch("T(his)() is a string", "(T[()])()")() )END"); @@ -1146,7 +1141,6 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "gmatch_capture_types_parens_in_sets_are_igno TEST_CASE_FIXTURE(BuiltinsFixture, "gmatch_capture_types_set_containing_lbracket") { - ScopedFastFlag sffs{"LuauDeduceGmatchReturnTypes", true}; CheckResult result = check(R"END( local a, b = string.gmatch("[[[", "()([[])")() )END"); @@ -1196,7 +1190,6 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "gmatch_capture_types_invalid_pattern_fallbac TEST_CASE_FIXTURE(BuiltinsFixture, "match_capture_types") { - ScopedFastFlag sffs{"LuauDeduceFindMatchReturnTypes", true}; CheckResult result = check(R"END( local a, b, c = string.match("This is a string", "(.()(%a+))") )END"); @@ -1210,7 +1203,6 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "match_capture_types") TEST_CASE_FIXTURE(BuiltinsFixture, "match_capture_types2") { - ScopedFastFlag sffs{"LuauDeduceFindMatchReturnTypes", true}; CheckResult result = check(R"END( local a, b, c = string.match("This is a string", "(.()(%a+))", "this should be a number") )END"); @@ -1229,7 +1221,6 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "match_capture_types2") TEST_CASE_FIXTURE(BuiltinsFixture, "find_capture_types") { - ScopedFastFlag sffs{"LuauDeduceFindMatchReturnTypes", true}; CheckResult result = check(R"END( local d, e, a, b, c = string.find("This is a string", "(.()(%a+))") )END"); @@ -1245,7 +1236,6 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "find_capture_types") TEST_CASE_FIXTURE(BuiltinsFixture, "find_capture_types2") { - ScopedFastFlag sffs{"LuauDeduceFindMatchReturnTypes", true}; CheckResult result = check(R"END( local d, e, a, b, c = string.find("This is a string", "(.()(%a+))", "this should be a number") )END"); @@ -1266,7 +1256,6 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "find_capture_types2") TEST_CASE_FIXTURE(BuiltinsFixture, "find_capture_types3") { - ScopedFastFlag sffs{"LuauDeduceFindMatchReturnTypes", true}; CheckResult result = check(R"END( local d, e, a, b, c = string.find("This is a string", "(.()(%a+))", 1, "this should be a bool") )END"); @@ -1287,7 +1276,6 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "find_capture_types3") TEST_CASE_FIXTURE(BuiltinsFixture, "find_capture_types3") { - ScopedFastFlag sffs{"LuauDeduceFindMatchReturnTypes", true}; CheckResult result = check(R"END( local d, e, a, b = string.find("This is a string", "(.()(%a+))", 1, true) )END"); diff --git a/tests/TypeInfer.functions.test.cpp b/tests/TypeInfer.functions.test.cpp index 5cc759d6..3a6f4491 100644 --- a/tests/TypeInfer.functions.test.cpp +++ b/tests/TypeInfer.functions.test.cpp @@ -133,6 +133,7 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "vararg_function_is_quantified") TableTypeVar* ttv = getMutable(*r); REQUIRE(ttv); + REQUIRE(ttv->props.count("f")); TypeId k = ttv->props["f"].type; REQUIRE(k); diff --git a/tests/TypeInfer.loops.test.cpp b/tests/TypeInfer.loops.test.cpp index 9b10092c..85249ecd 100644 --- a/tests/TypeInfer.loops.test.cpp +++ b/tests/TypeInfer.loops.test.cpp @@ -14,6 +14,7 @@ using namespace Luau; LUAU_FASTFLAG(LuauSpecialTypesAsterisked) +LUAU_FASTFLAG(DebugLuauDeferredConstraintResolution) TEST_SUITE_BEGIN("TypeInferLoops"); @@ -109,6 +110,18 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "for_in_with_just_one_iterator_is_ok") LUAU_REQUIRE_NO_ERRORS(result); } +TEST_CASE_FIXTURE(BuiltinsFixture, "for_in_loop_with_zero_iterators_dcr") +{ + ScopedFastFlag sff{"DebugLuauDeferredConstraintResolution", true}; + + CheckResult result = check(R"( + function no_iter() end + for key in no_iter() do end + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); +} + TEST_CASE_FIXTURE(BuiltinsFixture, "for_in_with_a_custom_iterator_should_type_check") { CheckResult result = check(R"( @@ -141,7 +154,7 @@ TEST_CASE_FIXTURE(Fixture, "for_in_loop_on_error") end )"); - CHECK_EQ(2, result.errors.size()); + LUAU_REQUIRE_ERROR_COUNT(2, result); TypeId p = requireType("p"); if (FFlag::LuauSpecialTypesAsterisked) @@ -232,6 +245,30 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "for_in_loop_error_on_iterator_requiring_args CHECK_EQ(0, acm->actual); } +TEST_CASE_FIXTURE(Fixture, "for_in_loop_with_incompatible_args_to_iterator") +{ + CheckResult result = check(R"( + function my_iter(state: string, index: number) + return state, index + end + + local my_state = {} + local first_index = "first" + + -- Type errors here. my_state and first_index cannot be passed to my_iter + for a, b in my_iter, my_state, first_index do + end + )"); + + LUAU_REQUIRE_ERROR_COUNT(2, result); + + CHECK(get(result.errors[1])); + CHECK(Location{{9, 29}, {9, 37}} == result.errors[0].location); + + CHECK(get(result.errors[1])); + CHECK(Location{{9, 39}, {9, 50}} == result.errors[1].location); +} + TEST_CASE_FIXTURE(Fixture, "for_in_loop_with_custom_iterator") { CheckResult result = check(R"( @@ -503,7 +540,7 @@ TEST_CASE_FIXTURE(Fixture, "loop_iter_basic") end )"); - LUAU_REQUIRE_ERROR_COUNT(0, result); + LUAU_REQUIRE_NO_ERRORS(result); CHECK_EQ(*typeChecker.numberType, *requireType("key")); } diff --git a/tests/TypeInfer.modules.test.cpp b/tests/TypeInfer.modules.test.cpp index b5f2296c..ede84f4a 100644 --- a/tests/TypeInfer.modules.test.cpp +++ b/tests/TypeInfer.modules.test.cpp @@ -16,6 +16,35 @@ LUAU_FASTFLAG(LuauSpecialTypesAsterisked) TEST_SUITE_BEGIN("TypeInferModules"); +TEST_CASE_FIXTURE(BuiltinsFixture, "dcr_require_basic") +{ + fileResolver.source["game/A"] = R"( + --!strict + return { + a = 1, + } + )"; + + fileResolver.source["game/B"] = R"( + --!strict + local A = require(game.A) + + local b = A.a + )"; + + CheckResult aResult = frontend.check("game/A"); + LUAU_REQUIRE_NO_ERRORS(aResult); + + CheckResult bResult = frontend.check("game/B"); + LUAU_REQUIRE_NO_ERRORS(bResult); + + ModulePtr b = frontend.moduleResolver.modules["game/B"]; + REQUIRE(b != nullptr); + std::optional bType = requireType(b, "b"); + REQUIRE(bType); + CHECK(toString(*bType) == "number"); +} + TEST_CASE_FIXTURE(BuiltinsFixture, "require") { fileResolver.source["game/A"] = R"( diff --git a/tests/TypeInfer.primitives.test.cpp b/tests/TypeInfer.primitives.test.cpp index a06fd749..5a6fb0e4 100644 --- a/tests/TypeInfer.primitives.test.cpp +++ b/tests/TypeInfer.primitives.test.cpp @@ -11,7 +11,6 @@ #include "doctest.h" -LUAU_FASTFLAG(LuauDeduceFindMatchReturnTypes) LUAU_FASTFLAG(LuauSpecialTypesAsterisked) using namespace Luau; @@ -61,8 +60,8 @@ TEST_CASE_FIXTURE(Fixture, "string_method") CheckResult result = check(R"( local p = ("tacos"):len() )"); - CHECK_EQ(0, result.errors.size()); + LUAU_REQUIRE_NO_ERRORS(result); CHECK_EQ(*requireType("p"), *typeChecker.numberType); } @@ -73,8 +72,8 @@ TEST_CASE_FIXTURE(Fixture, "string_function_indirect") local l = s.lower local p = l(s) )"); - CHECK_EQ(0, result.errors.size()); + LUAU_REQUIRE_NO_ERRORS(result); CHECK_EQ(*requireType("p"), *typeChecker.stringType); } @@ -84,12 +83,9 @@ TEST_CASE_FIXTURE(Fixture, "string_function_other") local s:string local p = s:match("foo") )"); - CHECK_EQ(0, result.errors.size()); - if (FFlag::LuauDeduceFindMatchReturnTypes) - CHECK_EQ(toString(requireType("p")), "string"); - else - CHECK_EQ(toString(requireType("p")), "string?"); + LUAU_REQUIRE_NO_ERRORS(result); + CHECK_EQ(toString(requireType("p")), "string"); } TEST_CASE_FIXTURE(Fixture, "CheckMethodsOfNumber") diff --git a/tests/TypeInfer.provisional.test.cpp b/tests/TypeInfer.provisional.test.cpp index c8fc7f2d..472d0ed5 100644 --- a/tests/TypeInfer.provisional.test.cpp +++ b/tests/TypeInfer.provisional.test.cpp @@ -62,7 +62,13 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "xpcall_returns_what_f_returns") local a:boolean,b:number,c:string=xpcall(function(): (number,string)return 1,'foo'end,function(): (string,number)return'foo',1 end) )"; - CHECK_EQ(expected, decorateWithTypes(code)); + CheckResult result = check(code); + + CHECK("boolean" == toString(requireType("a"))); + CHECK("number" == toString(requireType("b"))); + CHECK("string" == toString(requireType("c"))); + + CHECK(expected == decorateWithTypes(code)); } // We had a bug where if you have two type packs that looks like: @@ -609,4 +615,16 @@ TEST_CASE_FIXTURE(Fixture, "free_options_cannot_be_unified_together") CHECK("b?" == toString(option2, opts)); // This should not hold. } +TEST_CASE_FIXTURE(BuiltinsFixture, "for_in_loop_with_zero_iterators") +{ + ScopedFastFlag sff{"DebugLuauDeferredConstraintResolution", false}; + + CheckResult result = check(R"( + function no_iter() end + for key in no_iter() do end -- This should not be ok + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.tables.test.cpp b/tests/TypeInfer.tables.test.cpp index 95b85c48..97c3da4f 100644 --- a/tests/TypeInfer.tables.test.cpp +++ b/tests/TypeInfer.tables.test.cpp @@ -2994,8 +2994,6 @@ TEST_CASE_FIXTURE(Fixture, "expected_indexer_value_type_extra_2") TEST_CASE_FIXTURE(Fixture, "expected_indexer_from_table_union") { - ScopedFastFlag luauExpectedTableUnionIndexerType{"LuauExpectedTableUnionIndexerType", true}; - LUAU_REQUIRE_NO_ERRORS(check(R"(local a: {[string]: {number | string}} = {a = {2, 's'}})")); LUAU_REQUIRE_NO_ERRORS(check(R"(local a: {[string]: {number | string}}? = {a = {2, 's'}})")); LUAU_REQUIRE_NO_ERRORS(check(R"(local a: {[string]: {[string]: {string?}}?} = {["a"] = {["b"] = {"a", "b"}}})")); diff --git a/tests/conformance/basic.lua b/tests/conformance/basic.lua index d9274751..9c19da59 100644 --- a/tests/conformance/basic.lua +++ b/tests/conformance/basic.lua @@ -111,6 +111,11 @@ assert((function() local a = nil a = a and 2 return a end)() == nil) assert((function() local a = 1 a = a or 2 return a end)() == 1) assert((function() local a = nil a = a or 2 return a end)() == 2) +assert((function() local a a = 1 local b = 2 b = a and b return b end)() == 2) +assert((function() local a a = nil local b = 2 b = a and b return b end)() == nil) +assert((function() local a a = 1 local b = 2 b = a or b return b end)() == 1) +assert((function() local a a = nil local b = 2 b = a or b return b end)() == 2) + -- binary arithmetics coerces strings to numbers (sadly) assert(1 + "2" == 3) assert(2 * "0xa" == 20) diff --git a/tests/conformance/errors.lua b/tests/conformance/errors.lua index b69d437b..529e9b0c 100644 --- a/tests/conformance/errors.lua +++ b/tests/conformance/errors.lua @@ -369,21 +369,31 @@ assert(not a and b:match('[^ ]+') == "short:1:") local a,b = loadstring("nope", "=" .. string.rep("thisisaverylongstringitssolongthatitwontfitintotheinternalbufferprovidedtovariousdebugfacilities", 10)) assert(not a and b:match('[^ ]+') == "thisisaverylongstringitssolongthatitwontfitintotheinternalbufferprovidedtovariousdebugfacilitiesthisisaverylongstringitssolongthatitwontfitintotheinternalbufferprovidedtovariousdebugfacilitiesthisisaverylongstringitssolongthatitwontfitintotheinternalbuffe:1:") --- arith errors function ecall(fn, ...) local ok, err = pcall(fn, ...) assert(not ok) - return err:sub(err:find(": ") + 2, #err) + return err:sub((err:find(": ") or -1) + 2, #err) end +-- arith errors assert(ecall(function() return nil + 5 end) == "attempt to perform arithmetic (add) on nil and number") assert(ecall(function() return "a" + "b" end) == "attempt to perform arithmetic (add) on string") assert(ecall(function() return 1 > nil end) == "attempt to compare nil < number") -- note reversed order (by design) assert(ecall(function() return "a" <= 5 end) == "attempt to compare string <= number") +-- table errors assert(ecall(function() local t = {} t[nil] = 2 end) == "table index is nil") assert(ecall(function() local t = {} t[0/0] = 2 end) == "table index is NaN") +assert(ecall(function() local t = {} rawset(t, nil, 2) end) == "table index is nil") +assert(ecall(function() local t = {} rawset(t, 0/0, 2) end) == "table index is NaN") + +assert(ecall(function() local t = {} t[nil] = nil end) == "table index is nil") +assert(ecall(function() local t = {} t[0/0] = nil end) == "table index is NaN") + +assert(ecall(function() local t = {} rawset(t, nil, nil) end) == "table index is nil") +assert(ecall(function() local t = {} rawset(t, 0/0, nil) end) == "table index is NaN") + -- for loop type errors assert(ecall(function() for i='a',2 do end end) == "invalid 'for' initial value (number expected, got string)") assert(ecall(function() for i=1,'a' do end end) == "invalid 'for' limit (number expected, got string)") diff --git a/tools/faillist.txt b/tools/faillist.txt index 54e7ac05..3de9db76 100644 --- a/tools/faillist.txt +++ b/tools/faillist.txt @@ -1,10 +1,8 @@ AnnotationTests.builtin_types_are_not_exported -AnnotationTests.cannot_use_nonexported_type -AnnotationTests.cloned_interface_maintains_pointers_between_definitions AnnotationTests.duplicate_type_param_name AnnotationTests.for_loop_counter_annotation_is_checked AnnotationTests.generic_aliases_are_cloned_properly -AnnotationTests.interface_types_belong_to_interface_arena +AnnotationTests.instantiation_clone_has_to_follow AnnotationTests.luau_ice_triggers_an_ice AnnotationTests.luau_ice_triggers_an_ice_exception_with_flag AnnotationTests.luau_ice_triggers_an_ice_exception_with_flag_handler @@ -26,6 +24,7 @@ AutocompleteTest.autocomplete_first_function_arg_expected_type AutocompleteTest.autocomplete_for_in_middle_keywords AutocompleteTest.autocomplete_for_middle_keywords AutocompleteTest.autocomplete_if_middle_keywords +AutocompleteTest.autocomplete_interpolated_string AutocompleteTest.autocomplete_on_string_singletons AutocompleteTest.autocomplete_oop_implicit_self AutocompleteTest.autocomplete_repeat_middle_keyword @@ -56,14 +55,12 @@ AutocompleteTest.global_functions_are_not_scoped_lexically AutocompleteTest.globals_are_order_independent AutocompleteTest.if_then_else_elseif_completions AutocompleteTest.keyword_methods -AutocompleteTest.keyword_types AutocompleteTest.library_non_self_calls_are_fine AutocompleteTest.library_self_calls_are_invalid AutocompleteTest.local_function AutocompleteTest.local_function_params AutocompleteTest.local_functions_fall_out_of_scope AutocompleteTest.method_call_inside_function_body -AutocompleteTest.module_type_members AutocompleteTest.nested_member_completions AutocompleteTest.nested_recursive_function AutocompleteTest.no_function_name_suggestions @@ -78,7 +75,6 @@ AutocompleteTest.return_types AutocompleteTest.sometimes_the_metatable_is_an_error AutocompleteTest.source_module_preservation_and_invalidation AutocompleteTest.statement_between_two_statements -AutocompleteTest.stop_at_first_stat_when_recommending_keywords AutocompleteTest.string_prim_non_self_calls_are_avoided AutocompleteTest.string_prim_self_calls_are_fine AutocompleteTest.suggest_external_module_type @@ -155,6 +151,7 @@ BuiltinTests.string_format_tostring_specifier BuiltinTests.string_format_tostring_specifier_type_constraint BuiltinTests.string_format_use_correct_argument BuiltinTests.string_format_use_correct_argument2 +BuiltinTests.string_format_use_correct_argument3 BuiltinTests.string_lib_self_noself BuiltinTests.table_concat_returns_string BuiltinTests.table_dot_remove_optionally_returns_generic @@ -168,31 +165,21 @@ BuiltinTests.tonumber_returns_optional_number_type BuiltinTests.tonumber_returns_optional_number_type2 DefinitionTests.declaring_generic_functions DefinitionTests.definition_file_classes -DefinitionTests.definition_file_loading -DefinitionTests.definitions_documentation_symbols -DefinitionTests.documentation_symbols_dont_attach_to_persistent_types -DefinitionTests.single_class_type_identity_in_global_types FrontendTest.ast_node_at_position FrontendTest.automatically_check_dependent_scripts -FrontendTest.check_without_builtin_next FrontendTest.dont_reparse_clean_file_when_linting FrontendTest.environments FrontendTest.imported_table_modification_2 FrontendTest.it_should_be_safe_to_stringify_errors_when_full_type_graph_is_discarded -FrontendTest.no_use_after_free_with_type_fun_instantiation FrontendTest.nocheck_cycle_used_by_checked -FrontendTest.nocheck_modules_are_typed FrontendTest.produce_errors_for_unchanged_file_with_a_syntax_error FrontendTest.recheck_if_dependent_script_is_dirty FrontendTest.reexport_cyclic_type -FrontendTest.reexport_type_alias -FrontendTest.report_require_to_nonexistent_file FrontendTest.report_syntax_error_in_required_file FrontendTest.trace_requires_in_nonstrict_mode GenericsTests.apply_type_function_nested_generics1 GenericsTests.apply_type_function_nested_generics2 GenericsTests.better_mismatch_error_messages -GenericsTests.bound_tables_do_not_clone_original_fields GenericsTests.calling_self_generic_methods GenericsTests.check_generic_typepack_function GenericsTests.check_mutual_generic_functions @@ -208,7 +195,6 @@ GenericsTests.factories_of_generics GenericsTests.generic_argument_count_too_few GenericsTests.generic_argument_count_too_many GenericsTests.generic_factories -GenericsTests.generic_functions_dont_cache_type_parameters GenericsTests.generic_functions_in_types GenericsTests.generic_functions_should_be_memory_safe GenericsTests.generic_table_method @@ -245,7 +231,6 @@ IntersectionTypes.no_stack_overflow_from_flattenintersection IntersectionTypes.overload_is_not_a_function IntersectionTypes.select_correct_union_fn IntersectionTypes.should_still_pick_an_overload_whose_arguments_are_unions -IntersectionTypes.table_intersection_write IntersectionTypes.table_intersection_write_sealed IntersectionTypes.table_intersection_write_sealed_indirect IntersectionTypes.table_write_sealed_indirect @@ -255,7 +240,6 @@ Linter.TableOperations ModuleTests.clone_self_property ModuleTests.deepClone_cyclic_table ModuleTests.do_not_clone_reexports -ModuleTests.do_not_clone_types_of_reexported_values NonstrictModeTests.delay_function_does_not_require_its_argument_to_return_anything NonstrictModeTests.for_in_iterator_variables_are_any NonstrictModeTests.function_parameters_are_any @@ -319,6 +303,7 @@ ProvisionalTests.typeguard_inference_incomplete ProvisionalTests.weird_fail_to_unify_type_pack ProvisionalTests.weirditer_should_not_loop_forever ProvisionalTests.while_body_are_also_refined +ProvisionalTests.xpcall_returns_what_f_returns RefinementTest.and_constraint RefinementTest.and_or_peephole_refinement RefinementTest.apply_refinements_on_astexprindexexpr_whose_subscript_expr_is_constant_string @@ -358,6 +343,7 @@ RefinementTest.not_and_constraint RefinementTest.not_t_or_some_prop_of_t RefinementTest.or_predicate_with_truthy_predicates RefinementTest.parenthesized_expressions_are_followed_through +RefinementTest.refine_a_property_not_to_be_nil_through_an_intersection_table RefinementTest.refine_the_correct_types_opposite_of_when_a_is_not_number_or_string RefinementTest.refine_unknowns RefinementTest.string_not_equal_to_string_or_nil @@ -394,7 +380,6 @@ TableTests.augment_table TableTests.builtin_table_names TableTests.call_method TableTests.cannot_augment_sealed_table -TableTests.cannot_call_tables TableTests.cannot_change_type_of_unsealed_table_prop TableTests.casting_sealed_tables_with_props_into_table_with_indexer TableTests.casting_tables_with_props_into_table_with_indexer3 @@ -409,7 +394,6 @@ TableTests.defining_a_self_method_for_a_builtin_sealed_table_must_fail TableTests.defining_a_self_method_for_a_local_sealed_table_must_fail TableTests.dont_crash_when_setmetatable_does_not_produce_a_metatabletypevar TableTests.dont_hang_when_trying_to_look_up_in_cyclic_metatable_index -TableTests.dont_invalidate_the_properties_iterator_of_free_table_when_rolled_back TableTests.dont_leak_free_table_props TableTests.dont_quantify_table_that_belongs_to_outer_scope TableTests.dont_suggest_exact_match_keys @@ -448,6 +432,7 @@ TableTests.length_operator_intersection TableTests.length_operator_non_table_union TableTests.length_operator_union TableTests.length_operator_union_errors +TableTests.less_exponential_blowup_please TableTests.meta_add TableTests.meta_add_both_ways TableTests.meta_add_inferred @@ -470,7 +455,6 @@ TableTests.quantify_even_that_table_was_never_exported_at_all TableTests.quantify_metatables_of_metatables_of_table TableTests.quantifying_a_bound_var_works TableTests.reasonable_error_when_adding_a_nonexistent_property_to_an_array_like_table -TableTests.recursive_metatable_type_call TableTests.result_is_always_any_if_lhs_is_any TableTests.result_is_bool_for_equality_operators_if_lhs_is_any TableTests.right_table_missing_key @@ -480,7 +464,6 @@ TableTests.scalar_is_not_a_subtype_of_a_compatible_polymorphic_shape_type TableTests.shared_selfs TableTests.shared_selfs_from_free_param TableTests.shared_selfs_through_metatables -TableTests.table_function_check_use_after_free TableTests.table_indexing_error_location TableTests.table_insert_should_cope_with_optional_properties_in_nonstrict TableTests.table_insert_should_cope_with_optional_properties_in_strict @@ -525,13 +508,10 @@ TryUnifyTests.result_of_failed_typepack_unification_is_constrained TryUnifyTests.typepack_unification_should_trim_free_tails TryUnifyTests.variadics_should_use_reversed_properly TypeAliases.cli_38393_recursive_intersection_oom -TypeAliases.corecursive_types_generic TypeAliases.forward_declared_alias_is_not_clobbered_by_prior_unification_with_any -TypeAliases.general_require_multi_assign TypeAliases.generic_param_remap TypeAliases.mismatched_generic_pack_type_param TypeAliases.mismatched_generic_type_param -TypeAliases.mutually_recursive_types_errors TypeAliases.mutually_recursive_types_restriction_not_ok_1 TypeAliases.mutually_recursive_types_restriction_not_ok_2 TypeAliases.mutually_recursive_types_swapsies_not_ok @@ -543,7 +523,6 @@ TypeAliases.type_alias_fwd_declaration_is_precise TypeAliases.type_alias_local_mutation TypeAliases.type_alias_local_rename TypeAliases.type_alias_of_an_imported_recursive_generic_type -TypeAliases.type_alias_of_an_imported_recursive_type TypeInfer.checking_should_not_ice TypeInfer.cyclic_follow TypeInfer.do_not_bind_a_free_table_to_a_union_containing_that_table @@ -559,20 +538,21 @@ TypeInfer.tc_if_else_expressions_expected_type_1 TypeInfer.tc_if_else_expressions_expected_type_2 TypeInfer.tc_if_else_expressions_expected_type_3 TypeInfer.tc_if_else_expressions_type_union +TypeInfer.tc_interpolated_string_basic +TypeInfer.tc_interpolated_string_constant_type +TypeInfer.tc_interpolated_string_with_invalid_expression TypeInfer.type_infer_recursion_limit_no_ice -TypeInfer.warn_on_lowercase_parent_property TypeInferAnyError.assign_prop_to_table_by_calling_any_yields_any TypeInferAnyError.can_get_length_of_any -TypeInferAnyError.for_in_loop_iterator_is_any TypeInferAnyError.for_in_loop_iterator_is_any2 -TypeInferAnyError.for_in_loop_iterator_is_error -TypeInferAnyError.for_in_loop_iterator_is_error2 TypeInferAnyError.for_in_loop_iterator_returns_any -TypeInferAnyError.for_in_loop_iterator_returns_any2 TypeInferAnyError.length_of_error_type_does_not_produce_an_error TypeInferAnyError.replace_every_free_type_when_unifying_a_complex_function_with_any +TypeInferAnyError.union_of_types_regression_test TypeInferClasses.call_base_method TypeInferClasses.call_instance_method +TypeInferClasses.can_assign_to_prop_of_base_class_using_string +TypeInferClasses.can_read_prop_of_base_class_using_string TypeInferClasses.class_type_mismatch_with_name_conflict TypeInferClasses.classes_can_have_overloaded_operators TypeInferClasses.classes_without_overloaded_operators_cannot_be_added @@ -582,10 +562,13 @@ TypeInferClasses.higher_order_function_return_type_is_not_contravariant TypeInferClasses.higher_order_function_return_values_are_covariant TypeInferClasses.optional_class_field_access_error TypeInferClasses.table_class_unification_reports_sane_errors_for_missing_properties +TypeInferClasses.table_indexers_are_invariant +TypeInferClasses.table_properties_are_invariant TypeInferClasses.warn_when_prop_almost_matches TypeInferClasses.we_can_report_when_someone_is_trying_to_use_a_table_rather_than_a_class TypeInferFunctions.another_indirect_function_case_where_it_is_ok_to_provide_too_many_arguments TypeInferFunctions.another_recursive_local_function +TypeInferFunctions.call_o_with_another_argument_after_foo_was_quantified TypeInferFunctions.calling_function_with_anytypepack_doesnt_leak_free_types TypeInferFunctions.calling_function_with_incorrect_argument_type_yields_errors_spanning_argument TypeInferFunctions.complicated_return_types_require_an_explicit_annotation @@ -634,43 +617,23 @@ TypeInferFunctions.too_many_arguments TypeInferFunctions.too_many_return_values TypeInferFunctions.vararg_function_is_quantified TypeInferFunctions.vararg_functions_should_allow_calls_of_any_types_and_size -TypeInferLoops.for_in_loop TypeInferLoops.for_in_loop_error_on_factory_not_returning_the_right_amount_of_values -TypeInferLoops.for_in_loop_error_on_iterator_requiring_args_but_none_given -TypeInferLoops.for_in_loop_on_error -TypeInferLoops.for_in_loop_on_non_function -TypeInferLoops.for_in_loop_should_fail_with_non_function_iterator -TypeInferLoops.for_in_loop_where_iteratee_is_free TypeInferLoops.for_in_loop_with_custom_iterator TypeInferLoops.for_in_loop_with_next -TypeInferLoops.for_in_with_a_custom_iterator_should_type_check -TypeInferLoops.for_in_with_an_iterator_of_type_any TypeInferLoops.for_in_with_just_one_iterator_is_ok -TypeInferLoops.fuzz_fail_missing_instantitation_follow -TypeInferLoops.ipairs_produces_integral_indices -TypeInferLoops.loop_iter_basic TypeInferLoops.loop_iter_iter_metamethod TypeInferLoops.loop_iter_no_indexer_nonstrict -TypeInferLoops.loop_iter_no_indexer_strict TypeInferLoops.loop_iter_trailing_nil TypeInferLoops.loop_typecheck_crash_on_empty_optional -TypeInferLoops.properly_infer_iteratee_is_a_free_table TypeInferLoops.unreachable_code_after_infinite_loop TypeInferLoops.varlist_declared_by_for_in_loop_should_be_free -TypeInferModules.do_not_modify_imported_types -TypeInferModules.do_not_modify_imported_types_2 -TypeInferModules.do_not_modify_imported_types_3 -TypeInferModules.general_require_call_expression +TypeInferModules.custom_require_global TypeInferModules.general_require_type_mismatch TypeInferModules.module_type_conflict TypeInferModules.module_type_conflict_instantiated -TypeInferModules.require TypeInferModules.require_a_variadic_function -TypeInferModules.require_failed_module -TypeInferModules.require_module_that_does_not_export TypeInferModules.require_types TypeInferModules.type_error_of_unknown_qualified_type -TypeInferModules.warn_if_you_try_to_require_a_non_modulescript TypeInferOOP.CheckMethodsOfSealed TypeInferOOP.dont_suggest_using_colon_rather_than_dot_if_another_overload_works TypeInferOOP.dont_suggest_using_colon_rather_than_dot_if_it_wont_help_2 @@ -725,7 +688,6 @@ TypeInferOperators.typecheck_unary_minus_error TypeInferOperators.unary_not_is_boolean TypeInferOperators.unknown_type_in_comparison TypeInferOperators.UnknownGlobalCompoundAssign -TypeInferPrimitives.cannot_call_primitives TypeInferPrimitives.CheckMethodsOfNumber TypeInferPrimitives.string_function_other TypeInferPrimitives.string_index @@ -768,7 +730,6 @@ TypePackTests.type_alias_type_packs_nested TypePackTests.type_pack_hidden_free_tail_infinite_growth TypePackTests.type_pack_type_parameters TypePackTests.varargs_inference_through_multiple_scopes -TypePackTests.variadic_argument_tail TypePackTests.variadic_pack_syntax TypePackTests.variadic_packs TypeSingletons.bool_singleton_subtype @@ -793,7 +754,6 @@ TypeSingletons.string_singletons_escape_chars TypeSingletons.string_singletons_mismatch TypeSingletons.table_insert_with_a_singleton_argument TypeSingletons.table_properties_type_error_escapes -TypeSingletons.tagged_unions_immutable_tag TypeSingletons.tagged_unions_using_singletons TypeSingletons.taking_the_length_of_string_singleton TypeSingletons.taking_the_length_of_union_of_string_singleton diff --git a/tools/test_dcr.py b/tools/test_dcr.py index da33706c..db932253 100644 --- a/tools/test_dcr.py +++ b/tools/test_dcr.py @@ -84,12 +84,16 @@ def main(): failList = loadFailList() + commandLine = [ + args.path, + "--reporters=xml", + "--fflags=true,DebugLuauDeferredConstraintResolution=true", + ] + + print('>', ' '.join(commandLine), file=sys.stderr) + p = sp.Popen( - [ - args.path, - "--reporters=xml", - "--fflags=true,DebugLuauDeferredConstraintResolution=true", - ], + commandLine, stdout=sp.PIPE, ) @@ -122,7 +126,7 @@ def main(): with open(FAIL_LIST_PATH, "w", newline="\n") as f: for name in newFailList: print(name, file=f) - print("Updated faillist.txt") + print("Updated faillist.txt", file=sys.stderr) if handler.numSkippedTests > 0: print( From b2e357da30bc59c427fda2477bdc92d580b0138f Mon Sep 17 00:00:00 2001 From: Alexander McCord <11488393+alexmccord@users.noreply.github.com> Date: Sun, 4 Sep 2022 13:05:25 -0700 Subject: [PATCH 356/399] Update title and fix a dead link. (#659) Originally it was titled "Luau Recap: August 2022" but it got renamed to "Luau Recap: July & August 2022" and we just didn't fix the link here too. Also backports the title change to here too for consistency. --- docs/_posts/2022-08-29-luau-recap-august-2022.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/_posts/2022-08-29-luau-recap-august-2022.md b/docs/_posts/2022-08-29-luau-recap-august-2022.md index a0e88536..e43f0f34 100644 --- a/docs/_posts/2022-08-29-luau-recap-august-2022.md +++ b/docs/_posts/2022-08-29-luau-recap-august-2022.md @@ -1,11 +1,11 @@ --- layout: single -title: "Luau Recap: August 2022" +title: "Luau Recap: July & August 2022" --- Luau is our new language that you can read more about at [https://luau-lang.org](https://luau-lang.org). -[Cross-posted to the [Roblox Developer Forum](https://devforum.roblox.com/t/luau-recap-august-2022/).] +[Cross-posted to the [Roblox Developer Forum](https://devforum.roblox.com/t/luau-recap-july-august-2022/).] ## Tables now support `__len` metamethod From ce2c3b3a4e59e8bfc9ed94ee380a5b1627092d8c Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Thu, 8 Sep 2022 15:14:25 -0700 Subject: [PATCH 357/399] Sync to upstream/release/544 (#669) - Remove type definitions of `utf8.nfcnormalize`/`nfdnormalize`/`graphemes` that aren't supported by standalone Luau library - Add `lua_costatus` to retrieve extended thread status (similar to `coroutine.status`) - Improve GC sweeping performance (2-10% improvement on allocation-heavy benchmarks) --- Analysis/include/Luau/Anyification.h | 7 +- Analysis/include/Luau/BuiltinDefinitions.h | 12 +- .../include/Luau/ConstraintGraphBuilder.h | 7 +- Analysis/include/Luau/ConstraintSolver.h | 11 +- .../include/Luau/ConstraintSolverLogger.h | 29 -- Analysis/include/Luau/DcrLogger.h | 131 ++++++ Analysis/include/Luau/Documentation.h | 1 + Analysis/include/Luau/Frontend.h | 3 + Analysis/include/Luau/JsonEmitter.h | 12 + Analysis/include/Luau/Module.h | 2 +- Analysis/include/Luau/Normalize.h | 20 +- Analysis/include/Luau/TypeArena.h | 2 + Analysis/include/Luau/TypeChecker2.h | 6 +- Analysis/include/Luau/TypeInfer.h | 3 +- Analysis/include/Luau/TypeUtils.h | 10 +- Analysis/include/Luau/TypeVar.h | 38 +- Analysis/include/Luau/Unifier.h | 13 +- Analysis/src/Anyification.cpp | 11 +- Analysis/src/Autocomplete.cpp | 108 +++-- Analysis/src/BuiltinDefinitions.cpp | 148 ++++++- Analysis/src/ConstraintGraphBuilder.cpp | 83 ++-- Analysis/src/ConstraintSolver.cpp | 95 +++-- Analysis/src/ConstraintSolverLogger.cpp | 150 ------- Analysis/src/DcrLogger.cpp | 395 ++++++++++++++++++ Analysis/src/EmbeddedBuiltinDefinitions.cpp | 6 +- Analysis/src/Frontend.cpp | 35 +- Analysis/src/Linter.cpp | 3 +- Analysis/src/Module.cpp | 22 +- Analysis/src/Normalize.cpp | 46 +- Analysis/src/TypeArena.cpp | 9 + Analysis/src/TypeChecker2.cpp | 71 ++-- Analysis/src/TypeInfer.cpp | 144 ++++--- Analysis/src/TypeUtils.cpp | 38 +- Analysis/src/TypeVar.cpp | 63 ++- Analysis/src/Unifier.cpp | 39 +- CodeGen/include/Luau/CodeAllocator.h | 50 +++ CodeGen/include/Luau/OperandX64.h | 1 + CodeGen/include/Luau/RegisterX64.h | 1 + CodeGen/src/CodeAllocator.cpp | 188 +++++++++ Common/include/Luau/Bytecode.h | 27 +- Compiler/src/BytecodeBuilder.cpp | 31 -- Compiler/src/Compiler.cpp | 8 - Makefile | 6 +- Sources.cmake | 7 +- VM/include/lua.h | 10 + VM/src/lapi.cpp | 17 + VM/src/lcorolib.cpp | 34 +- VM/src/lgc.cpp | 77 +++- VM/src/lvmexecute.cpp | 14 +- tests/Autocomplete.test.cpp | 6 +- tests/CodeAllocator.test.cpp | 162 +++++++ tests/Compiler.test.cpp | 102 ++++- tests/Conformance.test.cpp | 3 +- tests/ConstraintSolver.test.cpp | 12 +- tests/CostModel.test.cpp | 19 + tests/Fixture.cpp | 11 +- tests/Fixture.h | 3 + tests/Frontend.test.cpp | 4 +- tests/LValue.test.cpp | 55 +-- tests/Linter.test.cpp | 10 +- tests/Module.test.cpp | 12 +- tests/Normalize.test.cpp | 28 +- tests/NotNull.test.cpp | 1 + tests/TypeInfer.aliases.test.cpp | 10 +- tests/TypeInfer.annotations.test.cpp | 6 +- tests/TypeInfer.classes.test.cpp | 10 +- tests/TypeInfer.definitions.test.cpp | 20 +- tests/TypeInfer.functions.test.cpp | 4 +- tests/TypeInfer.primitives.test.cpp | 14 + tests/TypeInfer.provisional.test.cpp | 12 +- tests/TypeInfer.refinements.test.cpp | 10 +- tests/TypeInfer.tryUnify.test.cpp | 2 +- tests/TypeInfer.typePacks.cpp | 4 +- tests/TypeVar.test.cpp | 2 +- tests/conformance/types.lua | 3 - tools/faillist.txt | 11 +- 76 files changed, 1987 insertions(+), 793 deletions(-) delete mode 100644 Analysis/include/Luau/ConstraintSolverLogger.h create mode 100644 Analysis/include/Luau/DcrLogger.h delete mode 100644 Analysis/src/ConstraintSolverLogger.cpp create mode 100644 Analysis/src/DcrLogger.cpp create mode 100644 CodeGen/include/Luau/CodeAllocator.h create mode 100644 CodeGen/src/CodeAllocator.cpp create mode 100644 tests/CodeAllocator.test.cpp diff --git a/Analysis/include/Luau/Anyification.h b/Analysis/include/Luau/Anyification.h index 9dd7d8e0..a6f3e2a9 100644 --- a/Analysis/include/Luau/Anyification.h +++ b/Analysis/include/Luau/Anyification.h @@ -19,9 +19,12 @@ using ScopePtr = std::shared_ptr; // A substitution which replaces free types by any struct Anyification : Substitution { - Anyification(TypeArena* arena, NotNull scope, InternalErrorReporter* iceHandler, TypeId anyType, TypePackId anyTypePack); - Anyification(TypeArena* arena, const ScopePtr& scope, InternalErrorReporter* iceHandler, TypeId anyType, TypePackId anyTypePack); + Anyification(TypeArena* arena, NotNull scope, NotNull singletonTypes, InternalErrorReporter* iceHandler, TypeId anyType, + TypePackId anyTypePack); + Anyification(TypeArena* arena, const ScopePtr& scope, NotNull singletonTypes, InternalErrorReporter* iceHandler, TypeId anyType, + TypePackId anyTypePack); NotNull scope; + NotNull singletonTypes; InternalErrorReporter* iceHandler; TypeId anyType; diff --git a/Analysis/include/Luau/BuiltinDefinitions.h b/Analysis/include/Luau/BuiltinDefinitions.h index 28a4368e..0292dff7 100644 --- a/Analysis/include/Luau/BuiltinDefinitions.h +++ b/Analysis/include/Luau/BuiltinDefinitions.h @@ -1,6 +1,7 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #pragma once +#include "Luau/Frontend.h" #include "Luau/Scope.h" #include "Luau/TypeInfer.h" @@ -8,6 +9,7 @@ namespace Luau { void registerBuiltinTypes(TypeChecker& typeChecker); +void registerBuiltinTypes(Frontend& frontend); TypeId makeUnion(TypeArena& arena, std::vector&& types); TypeId makeIntersection(TypeArena& arena, std::vector&& types); @@ -15,6 +17,7 @@ TypeId makeIntersection(TypeArena& arena, std::vector&& types); /** Build an optional 't' */ TypeId makeOption(TypeChecker& typeChecker, TypeArena& arena, TypeId t); +TypeId makeOption(Frontend& frontend, TypeArena& arena, TypeId t); /** Small utility function for building up type definitions from C++. */ @@ -41,12 +44,17 @@ void assignPropDocumentationSymbols(TableTypeVar::Props& props, const std::strin std::string getBuiltinDefinitionSource(); -void addGlobalBinding(TypeChecker& typeChecker, const std::string& name, TypeId ty, const std::string& packageName); void addGlobalBinding(TypeChecker& typeChecker, const std::string& name, Binding binding); +void addGlobalBinding(TypeChecker& typeChecker, const std::string& name, TypeId ty, const std::string& packageName); void addGlobalBinding(TypeChecker& typeChecker, const ScopePtr& scope, const std::string& name, TypeId ty, const std::string& packageName); void addGlobalBinding(TypeChecker& typeChecker, const ScopePtr& scope, const std::string& name, Binding binding); -std::optional tryGetGlobalBinding(TypeChecker& typeChecker, const std::string& name); +void addGlobalBinding(Frontend& frontend, const std::string& name, TypeId ty, const std::string& packageName); +void addGlobalBinding(Frontend& frontend, const std::string& name, Binding binding); +void addGlobalBinding(Frontend& frontend, const ScopePtr& scope, const std::string& name, TypeId ty, const std::string& packageName); +void addGlobalBinding(Frontend& frontend, const ScopePtr& scope, const std::string& name, Binding binding); +std::optional tryGetGlobalBinding(Frontend& frontend, const std::string& name); Binding* tryGetGlobalBindingRef(TypeChecker& typeChecker, const std::string& name); +TypeId getGlobalBinding(Frontend& frontend, const std::string& name); TypeId getGlobalBinding(TypeChecker& typeChecker, const std::string& name); } // namespace Luau diff --git a/Analysis/include/Luau/ConstraintGraphBuilder.h b/Analysis/include/Luau/ConstraintGraphBuilder.h index 1cba0d33..1567e0ad 100644 --- a/Analysis/include/Luau/ConstraintGraphBuilder.h +++ b/Analysis/include/Luau/ConstraintGraphBuilder.h @@ -21,6 +21,8 @@ namespace Luau struct Scope; using ScopePtr = std::shared_ptr; +struct DcrLogger; + struct ConstraintGraphBuilder { // A list of all the scopes in the module. This vector holds ownership of the @@ -30,7 +32,7 @@ struct ConstraintGraphBuilder ModuleName moduleName; ModulePtr module; - SingletonTypes& singletonTypes; + NotNull singletonTypes; const NotNull arena; // The root scope of the module we're generating constraints for. // This is null when the CGB is initially constructed. @@ -58,9 +60,10 @@ struct ConstraintGraphBuilder const NotNull ice; ScopePtr globalScope; + DcrLogger* logger; ConstraintGraphBuilder(const ModuleName& moduleName, ModulePtr module, TypeArena* arena, NotNull moduleResolver, - NotNull ice, const ScopePtr& globalScope); + NotNull singletonTypes, NotNull ice, const ScopePtr& globalScope, DcrLogger* logger); /** * Fabricates a new free type belonging to a given scope. diff --git a/Analysis/include/Luau/ConstraintSolver.h b/Analysis/include/Luau/ConstraintSolver.h index 002aa947..059e97cb 100644 --- a/Analysis/include/Luau/ConstraintSolver.h +++ b/Analysis/include/Luau/ConstraintSolver.h @@ -5,14 +5,16 @@ #include "Luau/Error.h" #include "Luau/Variant.h" #include "Luau/Constraint.h" -#include "Luau/ConstraintSolverLogger.h" #include "Luau/TypeVar.h" +#include "Luau/ToString.h" #include namespace Luau { +struct DcrLogger; + // TypeId, TypePackId, or Constraint*. It is impossible to know which, but we // never dereference this pointer. using BlockedConstraintId = const void*; @@ -40,6 +42,7 @@ struct HashInstantiationSignature struct ConstraintSolver { TypeArena* arena; + NotNull singletonTypes; InternalErrorReporter iceReporter; // The entire set of constraints that the solver is trying to resolve. std::vector> constraints; @@ -69,10 +72,10 @@ struct ConstraintSolver NotNull moduleResolver; std::vector requireCycles; - ConstraintSolverLogger logger; + DcrLogger* logger; - explicit ConstraintSolver(TypeArena* arena, NotNull rootScope, ModuleName moduleName, NotNull moduleResolver, - std::vector requireCycles); + explicit ConstraintSolver(TypeArena* arena, NotNull singletonTypes, NotNull rootScope, ModuleName moduleName, + NotNull moduleResolver, std::vector requireCycles, DcrLogger* logger); /** * Attempts to dispatch all pending constraints and reach a type solution diff --git a/Analysis/include/Luau/ConstraintSolverLogger.h b/Analysis/include/Luau/ConstraintSolverLogger.h deleted file mode 100644 index 65aa9a7e..00000000 --- a/Analysis/include/Luau/ConstraintSolverLogger.h +++ /dev/null @@ -1,29 +0,0 @@ -// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details - -#include "Luau/Constraint.h" -#include "Luau/NotNull.h" -#include "Luau/Scope.h" -#include "Luau/ToString.h" - -#include -#include -#include - -namespace Luau -{ - -struct ConstraintSolverLogger -{ - std::string compileOutput(); - void captureBoundarySnapshot(const Scope* rootScope, std::vector>& unsolvedConstraints); - void prepareStepSnapshot( - const Scope* rootScope, NotNull current, std::vector>& unsolvedConstraints, bool force); - void commitPreparedStepSnapshot(); - -private: - std::vector snapshots; - std::optional preparedSnapshot; - ToStringOptions opts; -}; - -} // namespace Luau diff --git a/Analysis/include/Luau/DcrLogger.h b/Analysis/include/Luau/DcrLogger.h new file mode 100644 index 00000000..bd8672e3 --- /dev/null +++ b/Analysis/include/Luau/DcrLogger.h @@ -0,0 +1,131 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +#pragma once + +#include "Luau/Constraint.h" +#include "Luau/NotNull.h" +#include "Luau/Scope.h" +#include "Luau/ToString.h" +#include "Luau/Error.h" +#include "Luau/Variant.h" + +#include +#include +#include + +namespace Luau +{ + +struct ErrorSnapshot +{ + std::string message; + Location location; +}; + +struct BindingSnapshot +{ + std::string typeId; + std::string typeString; + Location location; +}; + +struct TypeBindingSnapshot +{ + std::string typeId; + std::string typeString; +}; + +struct ConstraintGenerationLog +{ + std::string source; + std::unordered_map constraintLocations; + std::vector errors; +}; + +struct ScopeSnapshot +{ + std::unordered_map bindings; + std::unordered_map typeBindings; + std::unordered_map typePackBindings; + std::vector children; +}; + +enum class ConstraintBlockKind +{ + TypeId, + TypePackId, + ConstraintId, +}; + +struct ConstraintBlock +{ + ConstraintBlockKind kind; + std::string stringification; +}; + +struct ConstraintSnapshot +{ + std::string stringification; + std::vector blocks; +}; + +struct BoundarySnapshot +{ + std::unordered_map constraints; + ScopeSnapshot rootScope; +}; + +struct StepSnapshot +{ + std::string currentConstraint; + bool forced; + std::unordered_map unsolvedConstraints; + ScopeSnapshot rootScope; +}; + +struct TypeSolveLog +{ + BoundarySnapshot initialState; + std::vector stepStates; + BoundarySnapshot finalState; +}; + +struct TypeCheckLog +{ + std::vector errors; +}; + +using ConstraintBlockTarget = Variant>; + +struct DcrLogger +{ + std::string compileOutput(); + + void captureSource(std::string source); + void captureGenerationError(const TypeError& error); + void captureConstraintLocation(NotNull constraint, Location location); + + void pushBlock(NotNull constraint, TypeId block); + void pushBlock(NotNull constraint, TypePackId block); + void pushBlock(NotNull constraint, NotNull block); + void popBlock(TypeId block); + void popBlock(TypePackId block); + void popBlock(NotNull block); + + void captureInitialSolverState(const Scope* rootScope, const std::vector>& unsolvedConstraints); + StepSnapshot prepareStepSnapshot(const Scope* rootScope, NotNull current, bool force, const std::vector>& unsolvedConstraints); + void commitStepSnapshot(StepSnapshot snapshot); + void captureFinalSolverState(const Scope* rootScope, const std::vector>& unsolvedConstraints); + + void captureTypeCheckError(const TypeError& error); +private: + ConstraintGenerationLog generationLog; + std::unordered_map, std::vector> constraintBlocks; + TypeSolveLog solveLog; + TypeCheckLog checkLog; + + ToStringOptions opts; + + std::vector snapshotBlocks(NotNull constraint); +}; + +} // namespace Luau diff --git a/Analysis/include/Luau/Documentation.h b/Analysis/include/Luau/Documentation.h index 7a2b56ff..67a9feb1 100644 --- a/Analysis/include/Luau/Documentation.h +++ b/Analysis/include/Luau/Documentation.h @@ -1,3 +1,4 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #pragma once #include "Luau/DenseHash.h" diff --git a/Analysis/include/Luau/Frontend.h b/Analysis/include/Luau/Frontend.h index 55612689..04c598de 100644 --- a/Analysis/include/Luau/Frontend.h +++ b/Analysis/include/Luau/Frontend.h @@ -174,6 +174,9 @@ private: ScopePtr globalScope; public: + SingletonTypes singletonTypes_; + const NotNull singletonTypes; + FileResolver* fileResolver; FrontendModuleResolver moduleResolver; FrontendModuleResolver moduleResolverForAutocomplete; diff --git a/Analysis/include/Luau/JsonEmitter.h b/Analysis/include/Luau/JsonEmitter.h index 0bf3327a..d8dc96e4 100644 --- a/Analysis/include/Luau/JsonEmitter.h +++ b/Analysis/include/Luau/JsonEmitter.h @@ -4,6 +4,7 @@ #include #include #include +#include #include #include "Luau/NotNull.h" @@ -232,4 +233,15 @@ void write(JsonEmitter& emitter, const std::optional& v) emitter.writeRaw("null"); } +template +void write(JsonEmitter& emitter, const std::unordered_map& map) +{ + ObjectEmitter o = emitter.writeObject(); + + for (const auto& [k, v] : map) + o.writePair(k, v); + + o.finish(); +} + } // namespace Luau::Json diff --git a/Analysis/include/Luau/Module.h b/Analysis/include/Luau/Module.h index bec51b81..d22aad12 100644 --- a/Analysis/include/Luau/Module.h +++ b/Analysis/include/Luau/Module.h @@ -90,7 +90,7 @@ struct Module // Once a module has been typechecked, we clone its public interface into a separate arena. // This helps us to force TypeVar ownership into a DAG rather than a DCG. - void clonePublicInterface(InternalErrorReporter& ice); + void clonePublicInterface(NotNull singletonTypes, InternalErrorReporter& ice); }; } // namespace Luau diff --git a/Analysis/include/Luau/Normalize.h b/Analysis/include/Luau/Normalize.h index 78b241e4..48dbe2be 100644 --- a/Analysis/include/Luau/Normalize.h +++ b/Analysis/include/Luau/Normalize.h @@ -1,4 +1,5 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +#pragma once #include "Luau/Module.h" #include "Luau/NotNull.h" @@ -12,17 +13,20 @@ namespace Luau struct InternalErrorReporter; struct Module; struct Scope; +struct SingletonTypes; using ModulePtr = std::shared_ptr; -bool isSubtype(TypeId subTy, TypeId superTy, NotNull scope, InternalErrorReporter& ice); -bool isSubtype(TypePackId subTy, TypePackId superTy, NotNull scope, InternalErrorReporter& ice); +bool isSubtype(TypeId subTy, TypeId superTy, NotNull scope, NotNull singletonTypes, InternalErrorReporter& ice); +bool isSubtype(TypePackId subTy, TypePackId superTy, NotNull scope, NotNull singletonTypes, InternalErrorReporter& ice); -std::pair normalize(TypeId ty, NotNull scope, TypeArena& arena, InternalErrorReporter& ice); -std::pair normalize(TypeId ty, NotNull module, InternalErrorReporter& ice); -std::pair normalize(TypeId ty, const ModulePtr& module, InternalErrorReporter& ice); -std::pair normalize(TypePackId ty, NotNull scope, TypeArena& arena, InternalErrorReporter& ice); -std::pair normalize(TypePackId ty, NotNull module, InternalErrorReporter& ice); -std::pair normalize(TypePackId ty, const ModulePtr& module, InternalErrorReporter& ice); +std::pair normalize( + TypeId ty, NotNull scope, TypeArena& arena, NotNull singletonTypes, InternalErrorReporter& ice); +std::pair normalize(TypeId ty, NotNull module, NotNull singletonTypes, InternalErrorReporter& ice); +std::pair normalize(TypeId ty, const ModulePtr& module, NotNull singletonTypes, InternalErrorReporter& ice); +std::pair normalize( + TypePackId ty, NotNull scope, TypeArena& arena, NotNull singletonTypes, InternalErrorReporter& ice); +std::pair normalize(TypePackId ty, NotNull module, NotNull singletonTypes, InternalErrorReporter& ice); +std::pair normalize(TypePackId ty, const ModulePtr& module, NotNull singletonTypes, InternalErrorReporter& ice); } // namespace Luau diff --git a/Analysis/include/Luau/TypeArena.h b/Analysis/include/Luau/TypeArena.h index decc8c59..1e029aeb 100644 --- a/Analysis/include/Luau/TypeArena.h +++ b/Analysis/include/Luau/TypeArena.h @@ -31,6 +31,8 @@ struct TypeArena TypeId freshType(TypeLevel level); TypeId freshType(Scope* scope); + TypePackId freshTypePack(Scope* scope); + TypePackId addTypePack(std::initializer_list types); TypePackId addTypePack(std::vector types, std::optional tail = {}); TypePackId addTypePack(TypePack pack); diff --git a/Analysis/include/Luau/TypeChecker2.h b/Analysis/include/Luau/TypeChecker2.h index a6c7a3e3..a9cd6ec8 100644 --- a/Analysis/include/Luau/TypeChecker2.h +++ b/Analysis/include/Luau/TypeChecker2.h @@ -4,10 +4,14 @@ #include "Luau/Ast.h" #include "Luau/Module.h" +#include "Luau/NotNull.h" namespace Luau { -void check(const SourceModule& sourceModule, Module* module); +struct DcrLogger; +struct SingletonTypes; + +void check(NotNull singletonTypes, DcrLogger* logger, const SourceModule& sourceModule, Module* module); } // namespace Luau diff --git a/Analysis/include/Luau/TypeInfer.h b/Analysis/include/Luau/TypeInfer.h index 0b427946..b0b3f3ac 100644 --- a/Analysis/include/Luau/TypeInfer.h +++ b/Analysis/include/Luau/TypeInfer.h @@ -58,7 +58,7 @@ public: // within a program are borrowed pointers into this set. struct TypeChecker { - explicit TypeChecker(ModuleResolver* resolver, InternalErrorReporter* iceHandler); + explicit TypeChecker(ModuleResolver* resolver, NotNull singletonTypes, InternalErrorReporter* iceHandler); TypeChecker(const TypeChecker&) = delete; TypeChecker& operator=(const TypeChecker&) = delete; @@ -353,6 +353,7 @@ public: ModuleName currentModuleName; std::function prepareModuleScope; + NotNull singletonTypes; InternalErrorReporter* iceHandler; UnifierSharedState unifierState; diff --git a/Analysis/include/Luau/TypeUtils.h b/Analysis/include/Luau/TypeUtils.h index 6c611fb2..6890f881 100644 --- a/Analysis/include/Luau/TypeUtils.h +++ b/Analysis/include/Luau/TypeUtils.h @@ -15,10 +15,12 @@ struct TxnLog; using ScopePtr = std::shared_ptr; -std::optional findMetatableEntry(ErrorVec& errors, TypeId type, const std::string& entry, Location location); -std::optional findTablePropertyRespectingMeta(ErrorVec& errors, TypeId ty, const std::string& name, Location location); -std::optional getIndexTypeFromType(const ScopePtr& scope, ErrorVec& errors, TypeArena* arena, TypeId type, const std::string& prop, - const Location& location, bool addErrors, InternalErrorReporter& handle); +std::optional findMetatableEntry( + NotNull singletonTypes, ErrorVec& errors, TypeId type, const std::string& entry, Location location); +std::optional findTablePropertyRespectingMeta( + NotNull singletonTypes, ErrorVec& errors, TypeId ty, const std::string& name, Location location); +std::optional getIndexTypeFromType(const ScopePtr& scope, ErrorVec& errors, TypeArena* arena, NotNull singletonTypes, + TypeId type, const std::string& prop, const Location& location, bool addErrors, InternalErrorReporter& handle); // Returns the minimum and maximum number of types the argument list can accept. std::pair> getParameterExtents(const TxnLog* log, TypePackId tp); diff --git a/Analysis/include/Luau/TypeVar.h b/Analysis/include/Luau/TypeVar.h index e67b3601..2847d0b1 100644 --- a/Analysis/include/Luau/TypeVar.h +++ b/Analysis/include/Luau/TypeVar.h @@ -586,7 +586,7 @@ bool isOverloadedFunction(TypeId ty); // True when string is a subtype of ty bool maybeString(TypeId ty); -std::optional getMetatable(TypeId type); +std::optional getMetatable(TypeId type, NotNull singletonTypes); TableTypeVar* getMutableTableType(TypeId type); const TableTypeVar* getTableType(TypeId type); @@ -614,21 +614,6 @@ bool hasLength(TypeId ty, DenseHashSet& seen, int* recursionCount); struct SingletonTypes { - const TypeId nilType; - const TypeId numberType; - const TypeId stringType; - const TypeId booleanType; - const TypeId threadType; - const TypeId trueType; - const TypeId falseType; - const TypeId anyType; - const TypeId unknownType; - const TypeId neverType; - - const TypePackId anyTypePack; - const TypePackId neverTypePack; - const TypePackId uninhabitableTypePack; - SingletonTypes(); ~SingletonTypes(); SingletonTypes(const SingletonTypes&) = delete; @@ -644,9 +629,28 @@ private: bool debugFreezeArena = false; TypeId makeStringMetatable(); + +public: + const TypeId nilType; + const TypeId numberType; + const TypeId stringType; + const TypeId booleanType; + const TypeId threadType; + const TypeId trueType; + const TypeId falseType; + const TypeId anyType; + const TypeId unknownType; + const TypeId neverType; + const TypeId errorType; + + const TypePackId anyTypePack; + const TypePackId neverTypePack; + const TypePackId uninhabitableTypePack; + const TypePackId errorTypePack; }; -SingletonTypes& getSingletonTypes(); +// Clip with FFlagLuauNoMoreGlobalSingletonTypes +SingletonTypes& DEPRECATED_getSingletonTypes(); void persist(TypeId ty); void persist(TypePackId tp); diff --git a/Analysis/include/Luau/Unifier.h b/Analysis/include/Luau/Unifier.h index c7eb51a6..4d46869d 100644 --- a/Analysis/include/Luau/Unifier.h +++ b/Analysis/include/Luau/Unifier.h @@ -3,10 +3,11 @@ #include "Luau/Error.h" #include "Luau/Location.h" +#include "Luau/ParseOptions.h" #include "Luau/Scope.h" +#include "Luau/Substitution.h" #include "Luau/TxnLog.h" #include "Luau/TypeArena.h" -#include "Luau/TypeInfer.h" #include "Luau/UnifierSharedState.h" #include @@ -23,11 +24,14 @@ enum Variance // A substitution which replaces singleton types by their wider types struct Widen : Substitution { - Widen(TypeArena* arena) + Widen(TypeArena* arena, NotNull singletonTypes) : Substitution(TxnLog::empty(), arena) + , singletonTypes(singletonTypes) { } + NotNull singletonTypes; + bool isDirty(TypeId ty) override; bool isDirty(TypePackId ty) override; TypeId clean(TypeId ty) override; @@ -47,6 +51,7 @@ struct UnifierOptions struct Unifier { TypeArena* const types; + NotNull singletonTypes; Mode mode; NotNull scope; // const Scope maybe @@ -59,8 +64,8 @@ struct Unifier UnifierSharedState& sharedState; - Unifier(TypeArena* types, Mode mode, NotNull scope, const Location& location, Variance variance, UnifierSharedState& sharedState, - TxnLog* parentLog = nullptr); + Unifier(TypeArena* types, NotNull singletonTypes, Mode mode, NotNull scope, const Location& location, Variance variance, + UnifierSharedState& sharedState, TxnLog* parentLog = nullptr); // Test whether the two type vars unify. Never commits the result. ErrorVec canUnify(TypeId subTy, TypeId superTy); diff --git a/Analysis/src/Anyification.cpp b/Analysis/src/Anyification.cpp index abcaba02..cc9796ee 100644 --- a/Analysis/src/Anyification.cpp +++ b/Analysis/src/Anyification.cpp @@ -11,17 +11,20 @@ LUAU_FASTFLAG(LuauClassTypeVarsInSubstitution) namespace Luau { -Anyification::Anyification(TypeArena* arena, NotNull scope, InternalErrorReporter* iceHandler, TypeId anyType, TypePackId anyTypePack) +Anyification::Anyification(TypeArena* arena, NotNull scope, NotNull singletonTypes, InternalErrorReporter* iceHandler, + TypeId anyType, TypePackId anyTypePack) : Substitution(TxnLog::empty(), arena) , scope(scope) + , singletonTypes(singletonTypes) , iceHandler(iceHandler) , anyType(anyType) , anyTypePack(anyTypePack) { } -Anyification::Anyification(TypeArena* arena, const ScopePtr& scope, InternalErrorReporter* iceHandler, TypeId anyType, TypePackId anyTypePack) - : Anyification(arena, NotNull{scope.get()}, iceHandler, anyType, anyTypePack) +Anyification::Anyification(TypeArena* arena, const ScopePtr& scope, NotNull singletonTypes, InternalErrorReporter* iceHandler, + TypeId anyType, TypePackId anyTypePack) + : Anyification(arena, NotNull{scope.get()}, singletonTypes, iceHandler, anyType, anyTypePack) { } @@ -71,7 +74,7 @@ TypeId Anyification::clean(TypeId ty) for (TypeId& ty : copy) ty = replace(ty); TypeId res = copy.size() == 1 ? copy[0] : addType(UnionTypeVar{std::move(copy)}); - auto [t, ok] = normalize(res, scope, *arena, *iceHandler); + auto [t, ok] = normalize(res, scope, *arena, singletonTypes, *iceHandler); if (!ok) normalizationTooComplex = true; return t; diff --git a/Analysis/src/Autocomplete.cpp b/Analysis/src/Autocomplete.cpp index 378a1cb7..2fc145d3 100644 --- a/Analysis/src/Autocomplete.cpp +++ b/Analysis/src/Autocomplete.cpp @@ -14,8 +14,6 @@ LUAU_FASTFLAG(LuauSelfCallAutocompleteFix3) -LUAU_FASTFLAGVARIABLE(LuauAutocompleteFixGlobalOrder, false) - static const std::unordered_set kStatementStartingKeywords = { "while", "if", "local", "repeat", "function", "do", "for", "return", "break", "continue", "type", "export"}; @@ -137,27 +135,28 @@ static std::optional findExpectedTypeAt(const Module& module, AstNode* n return *it; } -static bool checkTypeMatch(TypeId subTy, TypeId superTy, NotNull scope, TypeArena* typeArena) +static bool checkTypeMatch(TypeId subTy, TypeId superTy, NotNull scope, TypeArena* typeArena, NotNull singletonTypes) { InternalErrorReporter iceReporter; UnifierSharedState unifierState(&iceReporter); - Unifier unifier(typeArena, Mode::Strict, scope, Location(), Variance::Covariant, unifierState); + Unifier unifier(typeArena, singletonTypes, Mode::Strict, scope, Location(), Variance::Covariant, unifierState); return unifier.canUnify(subTy, superTy).empty(); } -static TypeCorrectKind checkTypeCorrectKind(const Module& module, TypeArena* typeArena, AstNode* node, Position position, TypeId ty) +static TypeCorrectKind checkTypeCorrectKind( + const Module& module, TypeArena* typeArena, NotNull singletonTypes, AstNode* node, Position position, TypeId ty) { ty = follow(ty); NotNull moduleScope{module.getModuleScope().get()}; - auto canUnify = [&typeArena, moduleScope](TypeId subTy, TypeId superTy) { + auto canUnify = [&typeArena, singletonTypes, moduleScope](TypeId subTy, TypeId superTy) { LUAU_ASSERT(!FFlag::LuauSelfCallAutocompleteFix3); InternalErrorReporter iceReporter; UnifierSharedState unifierState(&iceReporter); - Unifier unifier(typeArena, Mode::Strict, moduleScope, Location(), Variance::Covariant, unifierState); + Unifier unifier(typeArena, singletonTypes, Mode::Strict, moduleScope, Location(), Variance::Covariant, unifierState); unifier.tryUnify(subTy, superTy); bool ok = unifier.errors.empty(); @@ -171,11 +170,11 @@ static TypeCorrectKind checkTypeCorrectKind(const Module& module, TypeArena* typ TypeId expectedType = follow(*typeAtPosition); - auto checkFunctionType = [typeArena, moduleScope, &canUnify, &expectedType](const FunctionTypeVar* ftv) { + auto checkFunctionType = [typeArena, singletonTypes, moduleScope, &canUnify, &expectedType](const FunctionTypeVar* ftv) { if (FFlag::LuauSelfCallAutocompleteFix3) { if (std::optional firstRetTy = first(ftv->retTypes)) - return checkTypeMatch(*firstRetTy, expectedType, moduleScope, typeArena); + return checkTypeMatch(*firstRetTy, expectedType, moduleScope, typeArena, singletonTypes); return false; } @@ -214,7 +213,8 @@ static TypeCorrectKind checkTypeCorrectKind(const Module& module, TypeArena* typ } if (FFlag::LuauSelfCallAutocompleteFix3) - return checkTypeMatch(ty, expectedType, NotNull{module.getModuleScope().get()}, typeArena) ? TypeCorrectKind::Correct : TypeCorrectKind::None; + return checkTypeMatch(ty, expectedType, NotNull{module.getModuleScope().get()}, typeArena, singletonTypes) ? TypeCorrectKind::Correct + : TypeCorrectKind::None; else return canUnify(ty, expectedType) ? TypeCorrectKind::Correct : TypeCorrectKind::None; } @@ -226,8 +226,8 @@ enum class PropIndexType Key, }; -static void autocompleteProps(const Module& module, TypeArena* typeArena, TypeId rootTy, TypeId ty, PropIndexType indexType, - const std::vector& nodes, AutocompleteEntryMap& result, std::unordered_set& seen, +static void autocompleteProps(const Module& module, TypeArena* typeArena, NotNull singletonTypes, TypeId rootTy, TypeId ty, + PropIndexType indexType, const std::vector& nodes, AutocompleteEntryMap& result, std::unordered_set& seen, std::optional containingClass = std::nullopt) { if (FFlag::LuauSelfCallAutocompleteFix3) @@ -272,7 +272,7 @@ static void autocompleteProps(const Module& module, TypeArena* typeArena, TypeId return colonIndex; } }; - auto isWrongIndexer = [typeArena, &module, rootTy, indexType](Luau::TypeId type) { + auto isWrongIndexer = [typeArena, singletonTypes, &module, rootTy, indexType](Luau::TypeId type) { LUAU_ASSERT(FFlag::LuauSelfCallAutocompleteFix3); if (indexType == PropIndexType::Key) @@ -280,7 +280,7 @@ static void autocompleteProps(const Module& module, TypeArena* typeArena, TypeId bool calledWithSelf = indexType == PropIndexType::Colon; - auto isCompatibleCall = [typeArena, &module, rootTy, calledWithSelf](const FunctionTypeVar* ftv) { + auto isCompatibleCall = [typeArena, singletonTypes, &module, rootTy, calledWithSelf](const FunctionTypeVar* ftv) { // Strong match with definition is a success if (calledWithSelf == ftv->hasSelf) return true; @@ -293,7 +293,7 @@ static void autocompleteProps(const Module& module, TypeArena* typeArena, TypeId // When called with '.', but declared with 'self', it is considered invalid if first argument is compatible if (std::optional firstArgTy = first(ftv->argTypes)) { - if (checkTypeMatch(rootTy, *firstArgTy, NotNull{module.getModuleScope().get()}, typeArena)) + if (checkTypeMatch(rootTy, *firstArgTy, NotNull{module.getModuleScope().get()}, typeArena, singletonTypes)) return calledWithSelf; } @@ -327,8 +327,9 @@ static void autocompleteProps(const Module& module, TypeArena* typeArena, TypeId if (result.count(name) == 0 && name != kParseNameError) { Luau::TypeId type = Luau::follow(prop.type); - TypeCorrectKind typeCorrect = indexType == PropIndexType::Key ? TypeCorrectKind::Correct - : checkTypeCorrectKind(module, typeArena, nodes.back(), {{}, {}}, type); + TypeCorrectKind typeCorrect = indexType == PropIndexType::Key + ? TypeCorrectKind::Correct + : checkTypeCorrectKind(module, typeArena, singletonTypes, nodes.back(), {{}, {}}, type); ParenthesesRecommendation parens = indexType == PropIndexType::Key ? ParenthesesRecommendation::None : getParenRecommendation(type, nodes, typeCorrect); @@ -355,13 +356,13 @@ static void autocompleteProps(const Module& module, TypeArena* typeArena, TypeId TypeId followed = follow(indexIt->second.type); if (get(followed) || get(followed)) { - autocompleteProps(module, typeArena, rootTy, followed, indexType, nodes, result, seen); + autocompleteProps(module, typeArena, singletonTypes, rootTy, followed, indexType, nodes, result, seen); } else if (auto indexFunction = get(followed)) { std::optional indexFunctionResult = first(indexFunction->retTypes); if (indexFunctionResult) - autocompleteProps(module, typeArena, rootTy, *indexFunctionResult, indexType, nodes, result, seen); + autocompleteProps(module, typeArena, singletonTypes, rootTy, *indexFunctionResult, indexType, nodes, result, seen); } } }; @@ -371,13 +372,13 @@ static void autocompleteProps(const Module& module, TypeArena* typeArena, TypeId containingClass = containingClass.value_or(cls); fillProps(cls->props); if (cls->parent) - autocompleteProps(module, typeArena, rootTy, *cls->parent, indexType, nodes, result, seen, containingClass); + autocompleteProps(module, typeArena, singletonTypes, rootTy, *cls->parent, indexType, nodes, result, seen, containingClass); } else if (auto tbl = get(ty)) fillProps(tbl->props); else if (auto mt = get(ty)) { - autocompleteProps(module, typeArena, rootTy, mt->table, indexType, nodes, result, seen); + autocompleteProps(module, typeArena, singletonTypes, rootTy, mt->table, indexType, nodes, result, seen); if (FFlag::LuauSelfCallAutocompleteFix3) { @@ -395,12 +396,12 @@ static void autocompleteProps(const Module& module, TypeArena* typeArena, TypeId { TypeId followed = follow(indexIt->second.type); if (get(followed) || get(followed)) - autocompleteProps(module, typeArena, rootTy, followed, indexType, nodes, result, seen); + autocompleteProps(module, typeArena, singletonTypes, rootTy, followed, indexType, nodes, result, seen); else if (auto indexFunction = get(followed)) { std::optional indexFunctionResult = first(indexFunction->retTypes); if (indexFunctionResult) - autocompleteProps(module, typeArena, rootTy, *indexFunctionResult, indexType, nodes, result, seen); + autocompleteProps(module, typeArena, singletonTypes, rootTy, *indexFunctionResult, indexType, nodes, result, seen); } } } @@ -413,7 +414,7 @@ static void autocompleteProps(const Module& module, TypeArena* typeArena, TypeId AutocompleteEntryMap inner; std::unordered_set innerSeen = seen; - autocompleteProps(module, typeArena, rootTy, ty, indexType, nodes, inner, innerSeen); + autocompleteProps(module, typeArena, singletonTypes, rootTy, ty, indexType, nodes, inner, innerSeen); for (auto& pair : inner) result.insert(pair); @@ -436,7 +437,7 @@ static void autocompleteProps(const Module& module, TypeArena* typeArena, TypeId if (iter == endIter) return; - autocompleteProps(module, typeArena, rootTy, *iter, indexType, nodes, result, seen); + autocompleteProps(module, typeArena, singletonTypes, rootTy, *iter, indexType, nodes, result, seen); ++iter; @@ -454,7 +455,7 @@ static void autocompleteProps(const Module& module, TypeArena* typeArena, TypeId continue; } - autocompleteProps(module, typeArena, rootTy, *iter, indexType, nodes, inner, innerSeen); + autocompleteProps(module, typeArena, singletonTypes, rootTy, *iter, indexType, nodes, inner, innerSeen); std::unordered_set toRemove; @@ -481,7 +482,7 @@ static void autocompleteProps(const Module& module, TypeArena* typeArena, TypeId } else if (FFlag::LuauSelfCallAutocompleteFix3 && get(get(ty))) { - autocompleteProps(module, typeArena, rootTy, getSingletonTypes().stringType, indexType, nodes, result, seen); + autocompleteProps(module, typeArena, singletonTypes, rootTy, singletonTypes->stringType, indexType, nodes, result, seen); } } @@ -506,18 +507,18 @@ static void autocompleteKeywords( } } -static void autocompleteProps( - const Module& module, TypeArena* typeArena, TypeId ty, PropIndexType indexType, const std::vector& nodes, AutocompleteEntryMap& result) +static void autocompleteProps(const Module& module, TypeArena* typeArena, NotNull singletonTypes, TypeId ty, PropIndexType indexType, + const std::vector& nodes, AutocompleteEntryMap& result) { std::unordered_set seen; - autocompleteProps(module, typeArena, ty, ty, indexType, nodes, result, seen); + autocompleteProps(module, typeArena, singletonTypes, ty, ty, indexType, nodes, result, seen); } -AutocompleteEntryMap autocompleteProps( - const Module& module, TypeArena* typeArena, TypeId ty, PropIndexType indexType, const std::vector& nodes) +AutocompleteEntryMap autocompleteProps(const Module& module, TypeArena* typeArena, NotNull singletonTypes, TypeId ty, + PropIndexType indexType, const std::vector& nodes) { AutocompleteEntryMap result; - autocompleteProps(module, typeArena, ty, indexType, nodes, result); + autocompleteProps(module, typeArena, singletonTypes, ty, indexType, nodes, result); return result; } @@ -1079,19 +1080,11 @@ T* extractStat(const std::vector& ancestry) static bool isBindingLegalAtCurrentPosition(const Symbol& symbol, const Binding& binding, Position pos) { - if (FFlag::LuauAutocompleteFixGlobalOrder) - { - if (symbol.local) - return binding.location.end < pos; + if (symbol.local) + return binding.location.end < pos; - // Builtin globals have an empty location; for defined globals, we want pos to be outside of the definition range to suggest it - return binding.location == Location() || !binding.location.containsClosed(pos); - } - else - { - // Default Location used for global bindings, which are always legal. - return binding.location == Location() || binding.location.end < pos; - } + // Builtin globals have an empty location; for defined globals, we want pos to be outside of the definition range to suggest it + return binding.location == Location() || !binding.location.containsClosed(pos); } static AutocompleteEntryMap autocompleteStatement( @@ -1220,12 +1213,14 @@ static AutocompleteContext autocompleteExpression(const SourceModule& sourceModu { LUAU_ASSERT(!ancestry.empty()); + NotNull singletonTypes = typeChecker.singletonTypes; + AstNode* node = ancestry.rbegin()[0]; if (node->is()) { if (auto it = module.astTypes.find(node->asExpr())) - autocompleteProps(module, typeArena, *it, PropIndexType::Point, ancestry, result); + autocompleteProps(module, typeArena, singletonTypes, *it, PropIndexType::Point, ancestry, result); } else if (autocompleteIfElseExpression(node, ancestry, position, result)) return AutocompleteContext::Keyword; @@ -1249,7 +1244,7 @@ static AutocompleteContext autocompleteExpression(const SourceModule& sourceModu std::string n = toString(name); if (!result.count(n)) { - TypeCorrectKind typeCorrect = checkTypeCorrectKind(module, typeArena, node, position, binding.typeId); + TypeCorrectKind typeCorrect = checkTypeCorrectKind(module, typeArena, singletonTypes, node, position, binding.typeId); result[n] = {AutocompleteEntryKind::Binding, binding.typeId, binding.deprecated, false, typeCorrect, std::nullopt, std::nullopt, binding.documentationSymbol, {}, getParenRecommendation(binding.typeId, ancestry, typeCorrect)}; @@ -1259,9 +1254,9 @@ static AutocompleteContext autocompleteExpression(const SourceModule& sourceModu scope = scope->parent; } - TypeCorrectKind correctForNil = checkTypeCorrectKind(module, typeArena, node, position, typeChecker.nilType); - TypeCorrectKind correctForTrue = checkTypeCorrectKind(module, typeArena, node, position, getSingletonTypes().trueType); - TypeCorrectKind correctForFalse = checkTypeCorrectKind(module, typeArena, node, position, getSingletonTypes().falseType); + TypeCorrectKind correctForNil = checkTypeCorrectKind(module, typeArena, singletonTypes, node, position, typeChecker.nilType); + TypeCorrectKind correctForTrue = checkTypeCorrectKind(module, typeArena, singletonTypes, node, position, singletonTypes->trueType); + TypeCorrectKind correctForFalse = checkTypeCorrectKind(module, typeArena, singletonTypes, node, position, singletonTypes->falseType); TypeCorrectKind correctForFunction = functionIsExpectedAt(module, node, position).value_or(false) ? TypeCorrectKind::Correct : TypeCorrectKind::None; @@ -1396,6 +1391,8 @@ static AutocompleteResult autocomplete(const SourceModule& sourceModule, const M if (isWithinComment(sourceModule, position)) return {}; + NotNull singletonTypes = typeChecker.singletonTypes; + std::vector ancestry = findAncestryAtPositionForAutocomplete(sourceModule, position); LUAU_ASSERT(!ancestry.empty()); AstNode* node = ancestry.back(); @@ -1422,10 +1419,11 @@ static AutocompleteResult autocomplete(const SourceModule& sourceModule, const M PropIndexType indexType = indexName->op == ':' ? PropIndexType::Colon : PropIndexType::Point; if (!FFlag::LuauSelfCallAutocompleteFix3 && isString(ty)) - return {autocompleteProps(*module, typeArena, typeChecker.globalScope->bindings[AstName{"string"}].typeId, indexType, ancestry), ancestry, - AutocompleteContext::Property}; + return {autocompleteProps( + *module, typeArena, singletonTypes, typeChecker.globalScope->bindings[AstName{"string"}].typeId, indexType, ancestry), + ancestry, AutocompleteContext::Property}; else - return {autocompleteProps(*module, typeArena, ty, indexType, ancestry), ancestry, AutocompleteContext::Property}; + return {autocompleteProps(*module, typeArena, singletonTypes, ty, indexType, ancestry), ancestry, AutocompleteContext::Property}; } else if (auto typeReference = node->as()) { @@ -1548,7 +1546,7 @@ static AutocompleteResult autocomplete(const SourceModule& sourceModule, const M { if (auto it = module->astExpectedTypes.find(exprTable)) { - auto result = autocompleteProps(*module, typeArena, *it, PropIndexType::Key, ancestry); + auto result = autocompleteProps(*module, typeArena, singletonTypes, *it, PropIndexType::Key, ancestry); // Remove keys that are already completed for (const auto& item : exprTable->items) @@ -1590,7 +1588,7 @@ static AutocompleteResult autocomplete(const SourceModule& sourceModule, const M if (auto idxExpr = ancestry.at(ancestry.size() - 2)->as()) { if (auto it = module->astTypes.find(idxExpr->expr)) - autocompleteProps(*module, typeArena, follow(*it), PropIndexType::Point, ancestry, result); + autocompleteProps(*module, typeArena, singletonTypes, follow(*it), PropIndexType::Point, ancestry, result); } else if (auto binExpr = ancestry.at(ancestry.size() - 2)->as()) { diff --git a/Analysis/src/BuiltinDefinitions.cpp b/Analysis/src/BuiltinDefinitions.cpp index e011eaa5..8f4863d0 100644 --- a/Analysis/src/BuiltinDefinitions.cpp +++ b/Analysis/src/BuiltinDefinitions.cpp @@ -6,6 +6,7 @@ #include "Luau/Common.h" #include "Luau/ToString.h" #include "Luau/ConstraintSolver.h" +#include "Luau/TypeInfer.h" #include @@ -45,6 +46,11 @@ TypeId makeIntersection(TypeArena& arena, std::vector&& types) return arena.addType(IntersectionTypeVar{std::move(types)}); } +TypeId makeOption(Frontend& frontend, TypeArena& arena, TypeId t) +{ + return makeUnion(arena, {frontend.typeChecker.nilType, t}); +} + TypeId makeOption(TypeChecker& typeChecker, TypeArena& arena, TypeId t) { return makeUnion(arena, {typeChecker.nilType, t}); @@ -128,34 +134,50 @@ Property makeProperty(TypeId ty, std::optional documentationSymbol) }; } +void addGlobalBinding(Frontend& frontend, const std::string& name, TypeId ty, const std::string& packageName) +{ + addGlobalBinding(frontend, frontend.getGlobalScope(), name, ty, packageName); +} + +void addGlobalBinding(TypeChecker& typeChecker, const ScopePtr& scope, const std::string& name, TypeId ty, const std::string& packageName); + void addGlobalBinding(TypeChecker& typeChecker, const std::string& name, TypeId ty, const std::string& packageName) { addGlobalBinding(typeChecker, typeChecker.globalScope, name, ty, packageName); } +void addGlobalBinding(Frontend& frontend, const std::string& name, Binding binding) +{ + addGlobalBinding(frontend, frontend.getGlobalScope(), name, binding); +} + void addGlobalBinding(TypeChecker& typeChecker, const std::string& name, Binding binding) { addGlobalBinding(typeChecker, typeChecker.globalScope, name, binding); } +void addGlobalBinding(Frontend& frontend, const ScopePtr& scope, const std::string& name, TypeId ty, const std::string& packageName) +{ + std::string documentationSymbol = packageName + "/global/" + name; + addGlobalBinding(frontend, scope, name, Binding{ty, Location{}, {}, {}, documentationSymbol}); +} + void addGlobalBinding(TypeChecker& typeChecker, const ScopePtr& scope, const std::string& name, TypeId ty, const std::string& packageName) { std::string documentationSymbol = packageName + "/global/" + name; addGlobalBinding(typeChecker, scope, name, Binding{ty, Location{}, {}, {}, documentationSymbol}); } +void addGlobalBinding(Frontend& frontend, const ScopePtr& scope, const std::string& name, Binding binding) +{ + addGlobalBinding(frontend.typeChecker, scope, name, binding); +} + void addGlobalBinding(TypeChecker& typeChecker, const ScopePtr& scope, const std::string& name, Binding binding) { scope->bindings[typeChecker.globalNames.names->getOrAdd(name.c_str())] = binding; } -TypeId getGlobalBinding(TypeChecker& typeChecker, const std::string& name) -{ - auto t = tryGetGlobalBinding(typeChecker, name); - LUAU_ASSERT(t.has_value()); - return t->typeId; -} - std::optional tryGetGlobalBinding(TypeChecker& typeChecker, const std::string& name) { AstName astName = typeChecker.globalNames.names->getOrAdd(name.c_str()); @@ -166,6 +188,23 @@ std::optional tryGetGlobalBinding(TypeChecker& typeChecker, const std:: return std::nullopt; } +TypeId getGlobalBinding(TypeChecker& typeChecker, const std::string& name) +{ + auto t = tryGetGlobalBinding(typeChecker, name); + LUAU_ASSERT(t.has_value()); + return t->typeId; +} + +TypeId getGlobalBinding(Frontend& frontend, const std::string& name) +{ + return getGlobalBinding(frontend.typeChecker, name); +} + +std::optional tryGetGlobalBinding(Frontend& frontend, const std::string& name) +{ + return tryGetGlobalBinding(frontend.typeChecker, name); +} + Binding* tryGetGlobalBindingRef(TypeChecker& typeChecker, const std::string& name) { AstName astName = typeChecker.globalNames.names->get(name.c_str()); @@ -195,6 +234,7 @@ void registerBuiltinTypes(TypeChecker& typeChecker) TypeId nilType = typeChecker.nilType; TypeArena& arena = typeChecker.globalTypes; + NotNull singletonTypes = typeChecker.singletonTypes; LoadDefinitionFileResult loadResult = Luau::loadDefinitionFile(typeChecker, typeChecker.globalScope, getBuiltinDefinitionSource(), "@luau"); LUAU_ASSERT(loadResult.success); @@ -203,7 +243,7 @@ void registerBuiltinTypes(TypeChecker& typeChecker) TypeId genericV = arena.addType(GenericTypeVar{"V"}); TypeId mapOfKtoV = arena.addType(TableTypeVar{{}, TableIndexer(genericK, genericV), typeChecker.globalScope->level, TableState::Generic}); - std::optional stringMetatableTy = getMetatable(getSingletonTypes().stringType); + std::optional stringMetatableTy = getMetatable(singletonTypes->stringType, singletonTypes); LUAU_ASSERT(stringMetatableTy); const TableTypeVar* stringMetatableTable = get(follow(*stringMetatableTy)); LUAU_ASSERT(stringMetatableTable); @@ -277,6 +317,98 @@ void registerBuiltinTypes(TypeChecker& typeChecker) attachDcrMagicFunction(getGlobalBinding(typeChecker, "require"), dcrMagicFunctionRequire); } +void registerBuiltinTypes(Frontend& frontend) +{ + LUAU_ASSERT(!frontend.globalTypes.typeVars.isFrozen()); + LUAU_ASSERT(!frontend.globalTypes.typePacks.isFrozen()); + + TypeId nilType = frontend.typeChecker.nilType; + + TypeArena& arena = frontend.globalTypes; + NotNull singletonTypes = frontend.singletonTypes; + + LoadDefinitionFileResult loadResult = frontend.loadDefinitionFile(getBuiltinDefinitionSource(), "@luau"); + LUAU_ASSERT(loadResult.success); + + TypeId genericK = arena.addType(GenericTypeVar{"K"}); + TypeId genericV = arena.addType(GenericTypeVar{"V"}); + TypeId mapOfKtoV = arena.addType(TableTypeVar{{}, TableIndexer(genericK, genericV), frontend.getGlobalScope()->level, TableState::Generic}); + + std::optional stringMetatableTy = getMetatable(singletonTypes->stringType, singletonTypes); + LUAU_ASSERT(stringMetatableTy); + const TableTypeVar* stringMetatableTable = get(follow(*stringMetatableTy)); + LUAU_ASSERT(stringMetatableTable); + + auto it = stringMetatableTable->props.find("__index"); + LUAU_ASSERT(it != stringMetatableTable->props.end()); + + addGlobalBinding(frontend, "string", it->second.type, "@luau"); + + // next(t: Table, i: K?) -> (K, V) + TypePackId nextArgsTypePack = arena.addTypePack(TypePack{{mapOfKtoV, makeOption(frontend, arena, genericK)}}); + addGlobalBinding(frontend, "next", + arena.addType(FunctionTypeVar{{genericK, genericV}, {}, nextArgsTypePack, arena.addTypePack(TypePack{{genericK, genericV}})}), "@luau"); + + TypePackId pairsArgsTypePack = arena.addTypePack({mapOfKtoV}); + + TypeId pairsNext = arena.addType(FunctionTypeVar{nextArgsTypePack, arena.addTypePack(TypePack{{genericK, genericV}})}); + TypePackId pairsReturnTypePack = arena.addTypePack(TypePack{{pairsNext, mapOfKtoV, nilType}}); + + // pairs(t: Table) -> ((Table, K?) -> (K, V), Table, nil) + addGlobalBinding(frontend, "pairs", arena.addType(FunctionTypeVar{{genericK, genericV}, {}, pairsArgsTypePack, pairsReturnTypePack}), "@luau"); + + TypeId genericMT = arena.addType(GenericTypeVar{"MT"}); + + TableTypeVar tab{TableState::Generic, frontend.getGlobalScope()->level}; + TypeId tabTy = arena.addType(tab); + + TypeId tableMetaMT = arena.addType(MetatableTypeVar{tabTy, genericMT}); + + addGlobalBinding(frontend, "getmetatable", makeFunction(arena, std::nullopt, {genericMT}, {}, {tableMetaMT}, {genericMT}), "@luau"); + + // clang-format off + // setmetatable(T, MT) -> { @metatable MT, T } + addGlobalBinding(frontend, "setmetatable", + arena.addType( + FunctionTypeVar{ + {genericMT}, + {}, + arena.addTypePack(TypePack{{FFlag::LuauUnknownAndNeverType ? tabTy : tableMetaMT, genericMT}}), + arena.addTypePack(TypePack{{tableMetaMT}}) + } + ), "@luau" + ); + // clang-format on + + for (const auto& pair : frontend.getGlobalScope()->bindings) + { + persist(pair.second.typeId); + + if (TableTypeVar* ttv = getMutable(pair.second.typeId)) + { + if (!ttv->name) + ttv->name = toString(pair.first); + } + } + + attachMagicFunction(getGlobalBinding(frontend, "assert"), magicFunctionAssert); + attachMagicFunction(getGlobalBinding(frontend, "setmetatable"), magicFunctionSetMetaTable); + attachMagicFunction(getGlobalBinding(frontend, "select"), magicFunctionSelect); + + if (TableTypeVar* ttv = getMutable(getGlobalBinding(frontend, "table"))) + { + // tabTy is a generic table type which we can't express via declaration syntax yet + ttv->props["freeze"] = makeProperty(makeFunction(arena, std::nullopt, {tabTy}, {tabTy}), "@luau/global/table.freeze"); + ttv->props["clone"] = makeProperty(makeFunction(arena, std::nullopt, {tabTy}, {tabTy}), "@luau/global/table.clone"); + + attachMagicFunction(ttv->props["pack"].type, magicFunctionPack); + } + + attachMagicFunction(getGlobalBinding(frontend, "require"), magicFunctionRequire); + attachDcrMagicFunction(getGlobalBinding(frontend, "require"), dcrMagicFunctionRequire); +} + + static std::optional> magicFunctionSelect( TypeChecker& typechecker, const ScopePtr& scope, const AstExprCall& expr, WithPredicate withPredicate) { diff --git a/Analysis/src/ConstraintGraphBuilder.cpp b/Analysis/src/ConstraintGraphBuilder.cpp index 9fabc528..e9c61e41 100644 --- a/Analysis/src/ConstraintGraphBuilder.cpp +++ b/Analysis/src/ConstraintGraphBuilder.cpp @@ -7,8 +7,10 @@ #include "Luau/ModuleResolver.h" #include "Luau/RecursionCounter.h" #include "Luau/ToString.h" +#include "Luau/DcrLogger.h" LUAU_FASTINT(LuauCheckRecursionLimit); +LUAU_FASTFLAG(DebugLuauLogSolverToJson); #include "Luau/Scope.h" @@ -35,17 +37,20 @@ static std::optional matchRequire(const AstExprCall& call) } ConstraintGraphBuilder::ConstraintGraphBuilder(const ModuleName& moduleName, ModulePtr module, TypeArena* arena, - NotNull moduleResolver, NotNull ice, const ScopePtr& globalScope) + NotNull moduleResolver, NotNull singletonTypes, NotNull ice, const ScopePtr& globalScope, DcrLogger* logger) : moduleName(moduleName) , module(module) - , singletonTypes(getSingletonTypes()) + , singletonTypes(singletonTypes) , arena(arena) , rootScope(nullptr) , moduleResolver(moduleResolver) , ice(ice) , globalScope(globalScope) + , logger(logger) { - LUAU_ASSERT(arena); + if (FFlag::DebugLuauLogSolverToJson) + LUAU_ASSERT(logger); + LUAU_ASSERT(module); } @@ -66,6 +71,7 @@ ScopePtr ConstraintGraphBuilder::childScope(AstNode* node, const ScopePtr& paren scopes.emplace_back(node->location, scope); scope->returnType = parent->returnType; + scope->varargPack = parent->varargPack; parent->children.push_back(NotNull{scope.get()}); module->astScopes[node] = scope.get(); @@ -282,7 +288,7 @@ void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatFor* for_) return; TypeId t = check(scope, expr); - addConstraint(scope, expr->location, SubtypeConstraint{t, singletonTypes.numberType}); + addConstraint(scope, expr->location, SubtypeConstraint{t, singletonTypes->numberType}); }; checkNumber(for_->from); @@ -290,7 +296,7 @@ void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatFor* for_) checkNumber(for_->step); ScopePtr forScope = childScope(for_, scope); - forScope->bindings[for_->var] = Binding{singletonTypes.numberType, for_->var->location}; + forScope->bindings[for_->var] = Binding{singletonTypes->numberType, for_->var->location}; visit(forScope, for_->body); } @@ -435,7 +441,7 @@ void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatFunction* funct } else if (AstExprError* err = function->name->as()) { - functionType = singletonTypes.errorRecoveryType(); + functionType = singletonTypes->errorRecoveryType(); } LUAU_ASSERT(functionType != nullptr); @@ -657,12 +663,18 @@ void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatDeclareFunction std::vector genericTys; genericTys.reserve(generics.size()); for (auto& [name, generic] : generics) + { genericTys.push_back(generic.ty); + scope->privateTypeBindings[name] = TypeFun{generic.ty}; + } std::vector genericTps; genericTps.reserve(genericPacks.size()); for (auto& [name, generic] : genericPacks) + { genericTps.push_back(generic.tp); + scope->privateTypePackBindings[name] = generic.tp; + } ScopePtr funScope = scope; if (!generics.empty() || !genericPacks.empty()) @@ -710,7 +722,7 @@ TypePackId ConstraintGraphBuilder::checkPack(const ScopePtr& scope, AstExpr* exp if (recursionCount >= FInt::LuauCheckRecursionLimit) { reportCodeTooComplex(expr->location); - return singletonTypes.errorRecoveryTypePack(); + return singletonTypes->errorRecoveryTypePack(); } TypePackId result = nullptr; @@ -758,7 +770,7 @@ TypePackId ConstraintGraphBuilder::checkPack(const ScopePtr& scope, AstExpr* exp if (scope->varargPack) result = *scope->varargPack; else - result = singletonTypes.errorRecoveryTypePack(); + result = singletonTypes->errorRecoveryTypePack(); } else { @@ -778,7 +790,7 @@ TypeId ConstraintGraphBuilder::check(const ScopePtr& scope, AstExpr* expr) if (recursionCount >= FInt::LuauCheckRecursionLimit) { reportCodeTooComplex(expr->location); - return singletonTypes.errorRecoveryType(); + return singletonTypes->errorRecoveryType(); } TypeId result = nullptr; @@ -786,20 +798,20 @@ TypeId ConstraintGraphBuilder::check(const ScopePtr& scope, AstExpr* expr) if (auto group = expr->as()) result = check(scope, group->expr); else if (expr->is()) - result = singletonTypes.stringType; + result = singletonTypes->stringType; else if (expr->is()) - result = singletonTypes.numberType; + result = singletonTypes->numberType; else if (expr->is()) - result = singletonTypes.booleanType; + result = singletonTypes->booleanType; else if (expr->is()) - result = singletonTypes.nilType; + result = singletonTypes->nilType; else if (auto a = expr->as()) { std::optional ty = scope->lookup(a->local); if (ty) result = *ty; else - result = singletonTypes.errorRecoveryType(); // FIXME? Record an error at this point? + result = singletonTypes->errorRecoveryType(); // FIXME? Record an error at this point? } else if (auto g = expr->as()) { @@ -812,7 +824,7 @@ TypeId ConstraintGraphBuilder::check(const ScopePtr& scope, AstExpr* expr) * global that is not already in-scope is definitely an unknown symbol. */ reportError(g->location, UnknownSymbol{g->name.value}); - result = singletonTypes.errorRecoveryType(); // FIXME? Record an error at this point? + result = singletonTypes->errorRecoveryType(); // FIXME? Record an error at this point? } } else if (expr->is()) @@ -842,7 +854,7 @@ TypeId ConstraintGraphBuilder::check(const ScopePtr& scope, AstExpr* expr) else if (auto err = expr->as()) { // Open question: Should we traverse into this? - result = singletonTypes.errorRecoveryType(); + result = singletonTypes->errorRecoveryType(); } else { @@ -903,7 +915,7 @@ TypeId ConstraintGraphBuilder::check(const ScopePtr& scope, AstExprUnary* unary) } LUAU_UNREACHABLE(); - return singletonTypes.errorRecoveryType(); + return singletonTypes->errorRecoveryType(); } TypeId ConstraintGraphBuilder::check(const ScopePtr& scope, AstExprBinary* binary) @@ -1003,7 +1015,7 @@ TypeId ConstraintGraphBuilder::checkExprTable(const ScopePtr& scope, AstExprTabl } else { - TypeId numberType = singletonTypes.numberType; + TypeId numberType = singletonTypes->numberType; // FIXME? The location isn't quite right here. Not sure what is // right. createIndexer(item.value->location, numberType, itemTy); @@ -1068,6 +1080,23 @@ ConstraintGraphBuilder::FunctionSignature ConstraintGraphBuilder::checkFunctionS signatureScope = bodyScope; } + std::optional varargPack; + + if (fn->vararg) + { + if (fn->varargAnnotation) + { + TypePackId annotationType = resolveTypePack(signatureScope, fn->varargAnnotation); + varargPack = annotationType; + } + else + { + varargPack = arena->freshTypePack(signatureScope.get()); + } + + signatureScope->varargPack = varargPack; + } + if (fn->returnAnnotation) { TypePackId annotatedRetType = resolveTypePack(signatureScope, *fn->returnAnnotation); @@ -1092,7 +1121,7 @@ ConstraintGraphBuilder::FunctionSignature ConstraintGraphBuilder::checkFunctionS // TODO: Vararg annotation. // TODO: Preserve argument names in the function's type. - FunctionTypeVar actualFunction{arena->addTypePack(argTypes), returnType}; + FunctionTypeVar actualFunction{arena->addTypePack(argTypes, varargPack), returnType}; actualFunction.hasNoGenerics = !hasGenerics; actualFunction.generics = std::move(genericTypes); actualFunction.genericPacks = std::move(genericTypePacks); @@ -1175,7 +1204,7 @@ TypeId ConstraintGraphBuilder::resolveType(const ScopePtr& scope, AstType* ty, b else { reportError(ty->location, UnknownSymbol{ref->name.value, UnknownSymbol::Context::Type}); - result = singletonTypes.errorRecoveryType(); + result = singletonTypes->errorRecoveryType(); } } else if (auto tab = ty->as()) @@ -1308,12 +1337,12 @@ TypeId ConstraintGraphBuilder::resolveType(const ScopePtr& scope, AstType* ty, b } else if (ty->is()) { - result = singletonTypes.errorRecoveryType(); + result = singletonTypes->errorRecoveryType(); } else { LUAU_ASSERT(0); - result = singletonTypes.errorRecoveryType(); + result = singletonTypes->errorRecoveryType(); } astResolvedTypes[ty] = result; @@ -1341,13 +1370,13 @@ TypePackId ConstraintGraphBuilder::resolveTypePack(const ScopePtr& scope, AstTyp else { reportError(tp->location, UnknownSymbol{gen->genericName.value, UnknownSymbol::Context::Type}); - result = singletonTypes.errorRecoveryTypePack(); + result = singletonTypes->errorRecoveryTypePack(); } } else { LUAU_ASSERT(0); - result = singletonTypes.errorRecoveryTypePack(); + result = singletonTypes->errorRecoveryTypePack(); } astResolvedTypePacks[tp] = result; @@ -1430,11 +1459,17 @@ TypeId ConstraintGraphBuilder::flattenPack(const ScopePtr& scope, Location locat void ConstraintGraphBuilder::reportError(Location location, TypeErrorData err) { errors.push_back(TypeError{location, moduleName, std::move(err)}); + + if (FFlag::DebugLuauLogSolverToJson) + logger->captureGenerationError(errors.back()); } void ConstraintGraphBuilder::reportCodeTooComplex(Location location) { errors.push_back(TypeError{location, moduleName, CodeTooComplex{}}); + + if (FFlag::DebugLuauLogSolverToJson) + logger->captureGenerationError(errors.back()); } struct GlobalPrepopulator : AstVisitor diff --git a/Analysis/src/ConstraintSolver.cpp b/Analysis/src/ConstraintSolver.cpp index 1088d982..f964a855 100644 --- a/Analysis/src/ConstraintSolver.cpp +++ b/Analysis/src/ConstraintSolver.cpp @@ -9,6 +9,7 @@ #include "Luau/Quantify.h" #include "Luau/ToString.h" #include "Luau/Unifier.h" +#include "Luau/DcrLogger.h" #include "Luau/VisitTypeVar.h" LUAU_FASTFLAGVARIABLE(DebugLuauLogSolver, false); @@ -50,8 +51,8 @@ static void dumpConstraints(NotNull scope, ToStringOptions& opts) dumpConstraints(child, opts); } -static std::pair, std::vector> saturateArguments( - const TypeFun& fn, const std::vector& rawTypeArguments, const std::vector& rawPackArguments, TypeArena* arena) +static std::pair, std::vector> saturateArguments(TypeArena* arena, NotNull singletonTypes, + const TypeFun& fn, const std::vector& rawTypeArguments, const std::vector& rawPackArguments) { std::vector saturatedTypeArguments; std::vector extraTypes; @@ -131,7 +132,7 @@ static std::pair, std::vector> saturateArguments if (!defaultTy) break; - TypeId instantiatedDefault = atf.substitute(defaultTy).value_or(getSingletonTypes().errorRecoveryType()); + TypeId instantiatedDefault = atf.substitute(defaultTy).value_or(singletonTypes->errorRecoveryType()); atf.typeArguments[fn.typeParams[i].ty] = instantiatedDefault; saturatedTypeArguments.push_back(instantiatedDefault); } @@ -149,7 +150,7 @@ static std::pair, std::vector> saturateArguments if (!defaultTp) break; - TypePackId instantiatedDefault = atf.substitute(defaultTp).value_or(getSingletonTypes().errorRecoveryTypePack()); + TypePackId instantiatedDefault = atf.substitute(defaultTp).value_or(singletonTypes->errorRecoveryTypePack()); atf.typePackArguments[fn.typePackParams[i].tp] = instantiatedDefault; saturatedPackArguments.push_back(instantiatedDefault); } @@ -167,12 +168,12 @@ static std::pair, std::vector> saturateArguments // even if they're missing, so we use the error type as a filler. for (size_t i = saturatedTypeArguments.size(); i < typesRequired; ++i) { - saturatedTypeArguments.push_back(getSingletonTypes().errorRecoveryType()); + saturatedTypeArguments.push_back(singletonTypes->errorRecoveryType()); } for (size_t i = saturatedPackArguments.size(); i < packsRequired; ++i) { - saturatedPackArguments.push_back(getSingletonTypes().errorRecoveryTypePack()); + saturatedPackArguments.push_back(singletonTypes->errorRecoveryTypePack()); } // At this point, these two conditions should be true. If they aren't we @@ -242,14 +243,16 @@ void dump(ConstraintSolver* cs, ToStringOptions& opts) } } -ConstraintSolver::ConstraintSolver(TypeArena* arena, NotNull rootScope, ModuleName moduleName, NotNull moduleResolver, - std::vector requireCycles) +ConstraintSolver::ConstraintSolver(TypeArena* arena, NotNull singletonTypes, NotNull rootScope, ModuleName moduleName, + NotNull moduleResolver, std::vector requireCycles, DcrLogger* logger) : arena(arena) + , singletonTypes(singletonTypes) , constraints(collectConstraints(rootScope)) , rootScope(rootScope) , currentModuleName(std::move(moduleName)) , moduleResolver(moduleResolver) , requireCycles(requireCycles) + , logger(logger) { opts.exhaustive = true; @@ -262,6 +265,9 @@ ConstraintSolver::ConstraintSolver(TypeArena* arena, NotNull rootScope, M block(dep, c); } } + + if (FFlag::DebugLuauLogSolverToJson) + LUAU_ASSERT(logger); } void ConstraintSolver::run() @@ -277,7 +283,7 @@ void ConstraintSolver::run() if (FFlag::DebugLuauLogSolverToJson) { - logger.captureBoundarySnapshot(rootScope, unsolvedConstraints); + logger->captureInitialSolverState(rootScope, unsolvedConstraints); } auto runSolverPass = [&](bool force) { @@ -294,10 +300,11 @@ void ConstraintSolver::run() } std::string saveMe = FFlag::DebugLuauLogSolver ? toString(*c, opts) : std::string{}; + StepSnapshot snapshot; if (FFlag::DebugLuauLogSolverToJson) { - logger.prepareStepSnapshot(rootScope, c, unsolvedConstraints, force); + snapshot = logger->prepareStepSnapshot(rootScope, c, force, unsolvedConstraints); } bool success = tryDispatch(c, force); @@ -311,7 +318,7 @@ void ConstraintSolver::run() if (FFlag::DebugLuauLogSolverToJson) { - logger.commitPreparedStepSnapshot(); + logger->commitStepSnapshot(snapshot); } if (FFlag::DebugLuauLogSolver) @@ -347,8 +354,7 @@ void ConstraintSolver::run() if (FFlag::DebugLuauLogSolverToJson) { - logger.captureBoundarySnapshot(rootScope, unsolvedConstraints); - printf("Logger output:\n%s\n", logger.compileOutput().c_str()); + logger->captureFinalSolverState(rootScope, unsolvedConstraints); } } @@ -516,7 +522,7 @@ bool ConstraintSolver::tryDispatch(const BinaryConstraint& c, NotNullty.emplace(getSingletonTypes().errorRecoveryType()); + asMutable(resultType)->ty.emplace(singletonTypes->errorRecoveryType()); // reportError(constraint->location, CannotInferBinaryOperation{c.op, std::nullopt, CannotInferBinaryOperation::Operation}); return true; } @@ -571,7 +577,7 @@ bool ConstraintSolver::tryDispatch(const IterableConstraint& c, NotNullscope, &iceReporter, getSingletonTypes().errorRecoveryType(), getSingletonTypes().errorRecoveryTypePack()}; + arena, constraint->scope, singletonTypes, &iceReporter, singletonTypes->errorRecoveryType(), singletonTypes->errorRecoveryTypePack()}; std::optional anyified = anyify.substitute(c.variables); LUAU_ASSERT(anyified); unify(*anyified, c.variables, constraint->scope); @@ -585,11 +591,11 @@ bool ConstraintSolver::tryDispatch(const IterableConstraint& c, NotNull(nextTy)) { - TypeId tableTy = getSingletonTypes().nilType; + TypeId tableTy = singletonTypes->nilType; if (iteratorTypes.size() >= 2) tableTy = iteratorTypes[1]; - TypeId firstIndexTy = getSingletonTypes().nilType; + TypeId firstIndexTy = singletonTypes->nilType; if (iteratorTypes.size() >= 3) firstIndexTy = iteratorTypes[2]; @@ -644,7 +650,7 @@ struct InfiniteTypeFinder : TypeVarOnceVisitor if (!tf.has_value()) return true; - auto [typeArguments, packArguments] = saturateArguments(*tf, petv.typeArguments, petv.packArguments, solver->arena); + auto [typeArguments, packArguments] = saturateArguments(solver->arena, solver->singletonTypes, *tf, petv.typeArguments, petv.packArguments); if (follow(tf->type) == follow(signature.fn.type) && (signature.arguments != typeArguments || signature.packArguments != packArguments)) { @@ -698,7 +704,7 @@ bool ConstraintSolver::tryDispatch(const TypeAliasExpansionConstraint& c, NotNul if (!tf.has_value()) { reportError(UnknownSymbol{petv->name.value, UnknownSymbol::Context::Type}, constraint->location); - bindResult(getSingletonTypes().errorRecoveryType()); + bindResult(singletonTypes->errorRecoveryType()); return true; } @@ -710,7 +716,7 @@ bool ConstraintSolver::tryDispatch(const TypeAliasExpansionConstraint& c, NotNul return true; } - auto [typeArguments, packArguments] = saturateArguments(*tf, petv->typeArguments, petv->packArguments, arena); + auto [typeArguments, packArguments] = saturateArguments(arena, singletonTypes, *tf, petv->typeArguments, petv->packArguments); bool sameTypes = std::equal(typeArguments.begin(), typeArguments.end(), tf->typeParams.begin(), tf->typeParams.end(), [](auto&& itp, auto&& p) { return itp == p.ty; @@ -757,7 +763,7 @@ bool ConstraintSolver::tryDispatch(const TypeAliasExpansionConstraint& c, NotNul if (itf.foundInfiniteType) { // TODO (CLI-56761): Report an error. - bindResult(getSingletonTypes().errorRecoveryType()); + bindResult(singletonTypes->errorRecoveryType()); return true; } @@ -780,7 +786,7 @@ bool ConstraintSolver::tryDispatch(const TypeAliasExpansionConstraint& c, NotNul if (!maybeInstantiated.has_value()) { // TODO (CLI-56761): Report an error. - bindResult(getSingletonTypes().errorRecoveryType()); + bindResult(singletonTypes->errorRecoveryType()); return true; } @@ -894,7 +900,7 @@ bool ConstraintSolver::tryDispatchIterableTable(TypeId iteratorTy, const Iterabl return block_(iteratorTy); auto anyify = [&](auto ty) { - Anyification anyify{arena, constraint->scope, &iceReporter, getSingletonTypes().anyType, getSingletonTypes().anyTypePack}; + Anyification anyify{arena, constraint->scope, singletonTypes, &iceReporter, singletonTypes->anyType, singletonTypes->anyTypePack}; std::optional anyified = anyify.substitute(ty); if (!anyified) reportError(CodeTooComplex{}, constraint->location); @@ -904,7 +910,7 @@ bool ConstraintSolver::tryDispatchIterableTable(TypeId iteratorTy, const Iterabl auto errorify = [&](auto ty) { Anyification anyify{ - arena, constraint->scope, &iceReporter, getSingletonTypes().errorRecoveryType(), getSingletonTypes().errorRecoveryTypePack()}; + arena, constraint->scope, singletonTypes, &iceReporter, singletonTypes->errorRecoveryType(), singletonTypes->errorRecoveryTypePack()}; std::optional errorified = anyify.substitute(ty); if (!errorified) reportError(CodeTooComplex{}, constraint->location); @@ -973,7 +979,7 @@ bool ConstraintSolver::tryDispatchIterableFunction( : firstIndexTy; // nextTy : (tableTy, indexTy?) -> (indexTy, valueTailTy...) - const TypePackId nextArgPack = arena->addTypePack({tableTy, arena->addType(UnionTypeVar{{firstIndex, getSingletonTypes().nilType}})}); + const TypePackId nextArgPack = arena->addTypePack({tableTy, arena->addType(UnionTypeVar{{firstIndex, singletonTypes->nilType}})}); const TypePackId valueTailTy = arena->addTypePack(FreeTypePack{constraint->scope}); const TypePackId nextRetPack = arena->addTypePack(TypePack{{firstIndex}, valueTailTy}); @@ -995,23 +1001,35 @@ void ConstraintSolver::block_(BlockedConstraintId target, NotNull target, NotNull constraint) { + if (FFlag::DebugLuauLogSolverToJson) + logger->pushBlock(constraint, target); + if (FFlag::DebugLuauLogSolver) printf("block Constraint %s on\t%s\n", toString(*target, opts).c_str(), toString(*constraint, opts).c_str()); + block_(target, constraint); } bool ConstraintSolver::block(TypeId target, NotNull constraint) { + if (FFlag::DebugLuauLogSolverToJson) + logger->pushBlock(constraint, target); + if (FFlag::DebugLuauLogSolver) printf("block TypeId %s on\t%s\n", toString(target, opts).c_str(), toString(*constraint, opts).c_str()); + block_(target, constraint); return false; } bool ConstraintSolver::block(TypePackId target, NotNull constraint) { + if (FFlag::DebugLuauLogSolverToJson) + logger->pushBlock(constraint, target); + if (FFlag::DebugLuauLogSolver) printf("block TypeId %s on\t%s\n", toString(target, opts).c_str(), toString(*constraint, opts).c_str()); + block_(target, constraint); return false; } @@ -1042,16 +1060,25 @@ void ConstraintSolver::unblock_(BlockedConstraintId progressed) void ConstraintSolver::unblock(NotNull progressed) { + if (FFlag::DebugLuauLogSolverToJson) + logger->popBlock(progressed); + return unblock_(progressed); } void ConstraintSolver::unblock(TypeId progressed) { + if (FFlag::DebugLuauLogSolverToJson) + logger->popBlock(progressed); + return unblock_(progressed); } void ConstraintSolver::unblock(TypePackId progressed) { + if (FFlag::DebugLuauLogSolverToJson) + logger->popBlock(progressed); + return unblock_(progressed); } @@ -1086,13 +1113,13 @@ bool ConstraintSolver::isBlocked(NotNull constraint) void ConstraintSolver::unify(TypeId subType, TypeId superType, NotNull scope) { UnifierSharedState sharedState{&iceReporter}; - Unifier u{arena, Mode::Strict, scope, Location{}, Covariant, sharedState}; + Unifier u{arena, singletonTypes, Mode::Strict, scope, Location{}, Covariant, sharedState}; u.tryUnify(subType, superType); if (!u.errors.empty()) { - TypeId errorType = getSingletonTypes().errorRecoveryType(); + TypeId errorType = singletonTypes->errorRecoveryType(); u.tryUnify(subType, errorType); u.tryUnify(superType, errorType); } @@ -1108,7 +1135,7 @@ void ConstraintSolver::unify(TypeId subType, TypeId superType, NotNull sc void ConstraintSolver::unify(TypePackId subPack, TypePackId superPack, NotNull scope) { UnifierSharedState sharedState{&iceReporter}; - Unifier u{arena, Mode::Strict, scope, Location{}, Covariant, sharedState}; + Unifier u{arena, singletonTypes, Mode::Strict, scope, Location{}, Covariant, sharedState}; u.tryUnify(subPack, superPack); @@ -1133,7 +1160,7 @@ TypeId ConstraintSolver::resolveModule(const ModuleInfo& info, const Location& l if (info.name.empty()) { reportError(UnknownRequire{}, location); - return getSingletonTypes().errorRecoveryType(); + return singletonTypes->errorRecoveryType(); } std::string humanReadableName = moduleResolver->getHumanReadableModuleName(info.name); @@ -1141,7 +1168,7 @@ TypeId ConstraintSolver::resolveModule(const ModuleInfo& info, const Location& l for (const auto& [location, path] : requireCycles) { if (!path.empty() && path.front() == humanReadableName) - return getSingletonTypes().anyType; + return singletonTypes->anyType; } ModulePtr module = moduleResolver->getModule(info.name); @@ -1150,24 +1177,24 @@ TypeId ConstraintSolver::resolveModule(const ModuleInfo& info, const Location& l if (!moduleResolver->moduleExists(info.name) && !info.optional) reportError(UnknownRequire{humanReadableName}, location); - return getSingletonTypes().errorRecoveryType(); + return singletonTypes->errorRecoveryType(); } if (module->type != SourceCode::Type::Module) { reportError(IllegalRequire{humanReadableName, "Module is not a ModuleScript. It cannot be required."}, location); - return getSingletonTypes().errorRecoveryType(); + return singletonTypes->errorRecoveryType(); } TypePackId modulePack = module->getModuleScope()->returnType; if (get(modulePack)) - return getSingletonTypes().errorRecoveryType(); + return singletonTypes->errorRecoveryType(); std::optional moduleType = first(modulePack); if (!moduleType) { reportError(IllegalRequire{humanReadableName, "Module does not return exactly 1 value. It cannot be required."}, location); - return getSingletonTypes().errorRecoveryType(); + return singletonTypes->errorRecoveryType(); } return *moduleType; diff --git a/Analysis/src/ConstraintSolverLogger.cpp b/Analysis/src/ConstraintSolverLogger.cpp deleted file mode 100644 index 5ba40521..00000000 --- a/Analysis/src/ConstraintSolverLogger.cpp +++ /dev/null @@ -1,150 +0,0 @@ -// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details - -#include "Luau/ConstraintSolverLogger.h" - -#include "Luau/JsonEmitter.h" -#include "Luau/ToString.h" - -LUAU_FASTFLAG(LuauFixNameMaps); - -namespace Luau -{ - -static void dumpScopeAndChildren(const Scope* scope, Json::JsonEmitter& emitter, ToStringOptions& opts) -{ - emitter.writeRaw("{"); - Json::write(emitter, "bindings"); - emitter.writeRaw(":"); - - Json::ObjectEmitter o = emitter.writeObject(); - - for (const auto& [name, binding] : scope->bindings) - { - if (FFlag::LuauFixNameMaps) - o.writePair(name.c_str(), toString(binding.typeId, opts)); - else - { - ToStringResult result = toStringDetailed(binding.typeId, opts); - opts.DEPRECATED_nameMap = std::move(result.DEPRECATED_nameMap); - o.writePair(name.c_str(), result.name); - } - } - - o.finish(); - emitter.writeRaw(","); - Json::write(emitter, "children"); - emitter.writeRaw(":"); - - Json::ArrayEmitter a = emitter.writeArray(); - for (const Scope* child : scope->children) - { - emitter.writeComma(); - dumpScopeAndChildren(child, emitter, opts); - } - - a.finish(); - emitter.writeRaw("}"); -} - -static std::string dumpConstraintsToDot(std::vector>& constraints, ToStringOptions& opts) -{ - std::string result = "digraph Constraints {\n"; - result += "rankdir=LR\n"; - - std::unordered_set> contained; - for (NotNull c : constraints) - { - contained.insert(c); - } - - for (NotNull c : constraints) - { - std::string shape; - if (get(*c)) - shape = "box"; - else if (get(*c)) - shape = "box3d"; - else - shape = "oval"; - - std::string id = std::to_string(reinterpret_cast(c.get())); - result += id; - result += " [label=\""; - result += toString(*c, opts); - result += "\" shape=" + shape + "];\n"; - - for (NotNull dep : c->dependencies) - { - if (contained.count(dep) == 0) - continue; - - result += std::to_string(reinterpret_cast(dep.get())); - result += " -> "; - result += id; - result += ";\n"; - } - } - - result += "}"; - - return result; -} - -std::string ConstraintSolverLogger::compileOutput() -{ - Json::JsonEmitter emitter; - emitter.writeRaw("["); - for (const std::string& snapshot : snapshots) - { - emitter.writeComma(); - emitter.writeRaw(snapshot); - } - - emitter.writeRaw("]"); - return emitter.str(); -} - -void ConstraintSolverLogger::captureBoundarySnapshot(const Scope* rootScope, std::vector>& unsolvedConstraints) -{ - Json::JsonEmitter emitter; - Json::ObjectEmitter o = emitter.writeObject(); - o.writePair("type", "boundary"); - o.writePair("constraintGraph", dumpConstraintsToDot(unsolvedConstraints, opts)); - emitter.writeComma(); - Json::write(emitter, "rootScope"); - emitter.writeRaw(":"); - dumpScopeAndChildren(rootScope, emitter, opts); - o.finish(); - - snapshots.push_back(emitter.str()); -} - -void ConstraintSolverLogger::prepareStepSnapshot( - const Scope* rootScope, NotNull current, std::vector>& unsolvedConstraints, bool force) -{ - Json::JsonEmitter emitter; - Json::ObjectEmitter o = emitter.writeObject(); - o.writePair("type", "step"); - o.writePair("constraintGraph", dumpConstraintsToDot(unsolvedConstraints, opts)); - o.writePair("currentId", std::to_string(reinterpret_cast(current.get()))); - o.writePair("current", toString(*current, opts)); - o.writePair("force", force); - emitter.writeComma(); - Json::write(emitter, "rootScope"); - emitter.writeRaw(":"); - dumpScopeAndChildren(rootScope, emitter, opts); - o.finish(); - - preparedSnapshot = emitter.str(); -} - -void ConstraintSolverLogger::commitPreparedStepSnapshot() -{ - if (preparedSnapshot) - { - snapshots.push_back(std::move(*preparedSnapshot)); - preparedSnapshot = std::nullopt; - } -} - -} // namespace Luau diff --git a/Analysis/src/DcrLogger.cpp b/Analysis/src/DcrLogger.cpp new file mode 100644 index 00000000..a2eb96e5 --- /dev/null +++ b/Analysis/src/DcrLogger.cpp @@ -0,0 +1,395 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details + +#include "Luau/DcrLogger.h" + +#include + +#include "Luau/JsonEmitter.h" + +namespace Luau +{ + +namespace Json +{ + +void write(JsonEmitter& emitter, const Location& location) +{ + ObjectEmitter o = emitter.writeObject(); + o.writePair("beginLine", location.begin.line); + o.writePair("beginColumn", location.begin.column); + o.writePair("endLine", location.end.line); + o.writePair("endColumn", location.end.column); + o.finish(); +} + +void write(JsonEmitter& emitter, const ErrorSnapshot& snapshot) +{ + ObjectEmitter o = emitter.writeObject(); + o.writePair("message", snapshot.message); + o.writePair("location", snapshot.location); + o.finish(); +} + +void write(JsonEmitter& emitter, const BindingSnapshot& snapshot) +{ + ObjectEmitter o = emitter.writeObject(); + o.writePair("typeId", snapshot.typeId); + o.writePair("typeString", snapshot.typeString); + o.writePair("location", snapshot.location); + o.finish(); +} + +void write(JsonEmitter& emitter, const TypeBindingSnapshot& snapshot) +{ + ObjectEmitter o = emitter.writeObject(); + o.writePair("typeId", snapshot.typeId); + o.writePair("typeString", snapshot.typeString); + o.finish(); +} + +void write(JsonEmitter& emitter, const ConstraintGenerationLog& log) +{ + ObjectEmitter o = emitter.writeObject(); + o.writePair("source", log.source); + + emitter.writeComma(); + write(emitter, "constraintLocations"); + emitter.writeRaw(":"); + + ObjectEmitter locationEmitter = emitter.writeObject(); + + for (const auto& [id, location] : log.constraintLocations) + { + locationEmitter.writePair(id, location); + } + + locationEmitter.finish(); + o.writePair("errors", log.errors); + o.finish(); +} + +void write(JsonEmitter& emitter, const ScopeSnapshot& snapshot) +{ + ObjectEmitter o = emitter.writeObject(); + o.writePair("bindings", snapshot.bindings); + o.writePair("typeBindings", snapshot.typeBindings); + o.writePair("typePackBindings", snapshot.typePackBindings); + o.writePair("children", snapshot.children); + o.finish(); +} + +void write(JsonEmitter& emitter, const ConstraintBlockKind& kind) +{ + switch (kind) + { + case ConstraintBlockKind::TypeId: + return write(emitter, "type"); + case ConstraintBlockKind::TypePackId: + return write(emitter, "typePack"); + case ConstraintBlockKind::ConstraintId: + return write(emitter, "constraint"); + default: + LUAU_ASSERT(0); + } +} + +void write(JsonEmitter& emitter, const ConstraintBlock& block) +{ + ObjectEmitter o = emitter.writeObject(); + o.writePair("kind", block.kind); + o.writePair("stringification", block.stringification); + o.finish(); +} + +void write(JsonEmitter& emitter, const ConstraintSnapshot& snapshot) +{ + ObjectEmitter o = emitter.writeObject(); + o.writePair("stringification", snapshot.stringification); + o.writePair("blocks", snapshot.blocks); + o.finish(); +} + +void write(JsonEmitter& emitter, const BoundarySnapshot& snapshot) +{ + ObjectEmitter o = emitter.writeObject(); + o.writePair("rootScope", snapshot.rootScope); + o.writePair("constraints", snapshot.constraints); + o.finish(); +} + +void write(JsonEmitter& emitter, const StepSnapshot& snapshot) +{ + ObjectEmitter o = emitter.writeObject(); + o.writePair("currentConstraint", snapshot.currentConstraint); + o.writePair("forced", snapshot.forced); + o.writePair("unsolvedConstraints", snapshot.unsolvedConstraints); + o.writePair("rootScope", snapshot.rootScope); + o.finish(); +} + +void write(JsonEmitter& emitter, const TypeSolveLog& log) +{ + ObjectEmitter o = emitter.writeObject(); + o.writePair("initialState", log.initialState); + o.writePair("stepStates", log.stepStates); + o.writePair("finalState", log.finalState); + o.finish(); +} + +void write(JsonEmitter& emitter, const TypeCheckLog& log) +{ + ObjectEmitter o = emitter.writeObject(); + o.writePair("errors", log.errors); + o.finish(); +} + +} // namespace Json + +static std::string toPointerId(NotNull ptr) +{ + return std::to_string(reinterpret_cast(ptr.get())); +} + +static ScopeSnapshot snapshotScope(const Scope* scope, ToStringOptions& opts) +{ + std::unordered_map bindings; + std::unordered_map typeBindings; + std::unordered_map typePackBindings; + std::vector children; + + for (const auto& [name, binding] : scope->bindings) + { + std::string id = std::to_string(reinterpret_cast(binding.typeId)); + ToStringResult result = toStringDetailed(binding.typeId, opts); + + bindings[name.c_str()] = BindingSnapshot{ + id, + result.name, + binding.location, + }; + } + + for (const auto& [name, tf] : scope->exportedTypeBindings) + { + std::string id = std::to_string(reinterpret_cast(tf.type)); + + typeBindings[name] = TypeBindingSnapshot{ + id, + toString(tf.type, opts), + }; + } + + for (const auto& [name, tf] : scope->privateTypeBindings) + { + std::string id = std::to_string(reinterpret_cast(tf.type)); + + typeBindings[name] = TypeBindingSnapshot{ + id, + toString(tf.type, opts), + }; + } + + for (const auto& [name, tp] : scope->privateTypePackBindings) + { + std::string id = std::to_string(reinterpret_cast(tp)); + + typePackBindings[name] = TypeBindingSnapshot{ + id, + toString(tp, opts), + }; + } + + for (const auto& child : scope->children) + { + children.push_back(snapshotScope(child.get(), opts)); + } + + return ScopeSnapshot{ + bindings, + typeBindings, + typePackBindings, + children, + }; +} + +std::string DcrLogger::compileOutput() +{ + Json::JsonEmitter emitter; + Json::ObjectEmitter o = emitter.writeObject(); + o.writePair("generation", generationLog); + o.writePair("solve", solveLog); + o.writePair("check", checkLog); + o.finish(); + + return emitter.str(); +} + +void DcrLogger::captureSource(std::string source) +{ + generationLog.source = std::move(source); +} + +void DcrLogger::captureGenerationError(const TypeError& error) +{ + std::string stringifiedError = toString(error); + generationLog.errors.push_back(ErrorSnapshot { + /* message */ stringifiedError, + /* location */ error.location, + }); +} + +void DcrLogger::captureConstraintLocation(NotNull constraint, Location location) +{ + std::string id = toPointerId(constraint); + generationLog.constraintLocations[id] = location; +} + +void DcrLogger::pushBlock(NotNull constraint, TypeId block) +{ + constraintBlocks[constraint].push_back(block); +} + +void DcrLogger::pushBlock(NotNull constraint, TypePackId block) +{ + constraintBlocks[constraint].push_back(block); +} + +void DcrLogger::pushBlock(NotNull constraint, NotNull block) +{ + constraintBlocks[constraint].push_back(block); +} + +void DcrLogger::popBlock(TypeId block) +{ + for (auto& [_, list] : constraintBlocks) + { + list.erase(std::remove(list.begin(), list.end(), block), list.end()); + } +} + +void DcrLogger::popBlock(TypePackId block) +{ + for (auto& [_, list] : constraintBlocks) + { + list.erase(std::remove(list.begin(), list.end(), block), list.end()); + } +} + +void DcrLogger::popBlock(NotNull block) +{ + for (auto& [_, list] : constraintBlocks) + { + list.erase(std::remove(list.begin(), list.end(), block), list.end()); + } +} + +void DcrLogger::captureInitialSolverState(const Scope* rootScope, const std::vector>& unsolvedConstraints) +{ + solveLog.initialState.rootScope = snapshotScope(rootScope, opts); + solveLog.initialState.constraints.clear(); + + for (NotNull c : unsolvedConstraints) + { + std::string id = toPointerId(c); + solveLog.initialState.constraints[id] = { + toString(*c.get(), opts), + snapshotBlocks(c), + }; + } +} + +StepSnapshot DcrLogger::prepareStepSnapshot(const Scope* rootScope, NotNull current, bool force, const std::vector>& unsolvedConstraints) +{ + ScopeSnapshot scopeSnapshot = snapshotScope(rootScope, opts); + std::string currentId = toPointerId(current); + std::unordered_map constraints; + + for (NotNull c : unsolvedConstraints) + { + std::string id = toPointerId(c); + constraints[id] = { + toString(*c.get(), opts), + snapshotBlocks(c), + }; + } + + return StepSnapshot{ + currentId, + force, + constraints, + scopeSnapshot, + }; +} + +void DcrLogger::commitStepSnapshot(StepSnapshot snapshot) +{ + solveLog.stepStates.push_back(std::move(snapshot)); +} + +void DcrLogger::captureFinalSolverState(const Scope* rootScope, const std::vector>& unsolvedConstraints) +{ + solveLog.finalState.rootScope = snapshotScope(rootScope, opts); + solveLog.finalState.constraints.clear(); + + for (NotNull c : unsolvedConstraints) + { + std::string id = toPointerId(c); + solveLog.finalState.constraints[id] = { + toString(*c.get(), opts), + snapshotBlocks(c), + }; + } +} + +void DcrLogger::captureTypeCheckError(const TypeError& error) +{ + std::string stringifiedError = toString(error); + checkLog.errors.push_back(ErrorSnapshot { + /* message */ stringifiedError, + /* location */ error.location, + }); +} + +std::vector DcrLogger::snapshotBlocks(NotNull c) +{ + auto it = constraintBlocks.find(c); + if (it == constraintBlocks.end()) + { + return {}; + } + + std::vector snapshot; + + for (const ConstraintBlockTarget& target : it->second) + { + if (const TypeId* ty = get_if(&target)) + { + snapshot.push_back({ + ConstraintBlockKind::TypeId, + toString(*ty, opts), + }); + } + else if (const TypePackId* tp = get_if(&target)) + { + snapshot.push_back({ + ConstraintBlockKind::TypePackId, + toString(*tp, opts), + }); + } + else if (const NotNull* c = get_if>(&target)) + { + snapshot.push_back({ + ConstraintBlockKind::ConstraintId, + toString(*(c->get()), opts), + }); + } + else + { + LUAU_ASSERT(0); + } + } + + return snapshot; +} + +} // namespace Luau diff --git a/Analysis/src/EmbeddedBuiltinDefinitions.cpp b/Analysis/src/EmbeddedBuiltinDefinitions.cpp index 45663531..0f04ace0 100644 --- a/Analysis/src/EmbeddedBuiltinDefinitions.cpp +++ b/Analysis/src/EmbeddedBuiltinDefinitions.cpp @@ -187,13 +187,9 @@ declare utf8: { char: (...number) -> string, charpattern: string, codes: (string) -> ((string, number) -> (number, number), string, number), - -- FIXME - codepoint: (string, number?, number?) -> (number, ...number), + codepoint: (string, number?, number?) -> ...number, len: (string, number?, number?) -> (number?, number?), offset: (string, number?, number?) -> number, - nfdnormalize: (string) -> string, - nfcnormalize: (string) -> string, - graphemes: (string, number?, number?) -> (() -> (number, number)), } -- Cannot use `typeof` here because it will produce a polytype when we expect a monotype. diff --git a/Analysis/src/Frontend.cpp b/Analysis/src/Frontend.cpp index d8839f2f..01e82baa 100644 --- a/Analysis/src/Frontend.cpp +++ b/Analysis/src/Frontend.cpp @@ -6,6 +6,7 @@ #include "Luau/Config.h" #include "Luau/ConstraintGraphBuilder.h" #include "Luau/ConstraintSolver.h" +#include "Luau/DcrLogger.h" #include "Luau/FileResolver.h" #include "Luau/Parser.h" #include "Luau/Scope.h" @@ -23,10 +24,12 @@ LUAU_FASTINT(LuauTypeInferIterationLimit) LUAU_FASTINT(LuauTarjanChildLimit) LUAU_FASTFLAG(LuauInferInNoCheckMode) +LUAU_FASTFLAG(LuauNoMoreGlobalSingletonTypes) LUAU_FASTFLAGVARIABLE(LuauKnowsTheDataModel3, false) LUAU_FASTFLAGVARIABLE(LuauAutocompleteDynamicLimits, false) LUAU_FASTINTVARIABLE(LuauAutocompleteCheckTimeoutMs, 100) LUAU_FASTFLAGVARIABLE(DebugLuauDeferredConstraintResolution, false) +LUAU_FASTFLAG(DebugLuauLogSolverToJson); namespace Luau { @@ -389,11 +392,12 @@ double getTimestamp() } // namespace Frontend::Frontend(FileResolver* fileResolver, ConfigResolver* configResolver, const FrontendOptions& options) - : fileResolver(fileResolver) + : singletonTypes(NotNull{FFlag::LuauNoMoreGlobalSingletonTypes ? &singletonTypes_ : &DEPRECATED_getSingletonTypes()}) + , fileResolver(fileResolver) , moduleResolver(this) , moduleResolverForAutocomplete(this) - , typeChecker(&moduleResolver, &iceHandler) - , typeCheckerForAutocomplete(&moduleResolverForAutocomplete, &iceHandler) + , typeChecker(&moduleResolver, singletonTypes, &iceHandler) + , typeCheckerForAutocomplete(&moduleResolverForAutocomplete, singletonTypes, &iceHandler) , configResolver(configResolver) , options(options) { @@ -837,11 +841,22 @@ ModulePtr Frontend::check(const SourceModule& sourceModule, Mode mode, const Sco { ModulePtr result = std::make_shared(); - ConstraintGraphBuilder cgb{sourceModule.name, result, &result->internalTypes, NotNull(&moduleResolver), NotNull(&iceHandler), getGlobalScope()}; + std::unique_ptr logger; + if (FFlag::DebugLuauLogSolverToJson) + { + logger = std::make_unique(); + std::optional source = fileResolver->readSource(sourceModule.name); + if (source) + { + logger->captureSource(source->source); + } + } + + ConstraintGraphBuilder cgb{sourceModule.name, result, &result->internalTypes, NotNull(&moduleResolver), singletonTypes, NotNull(&iceHandler), getGlobalScope(), logger.get()}; cgb.visit(sourceModule.root); result->errors = std::move(cgb.errors); - ConstraintSolver cs{&result->internalTypes, NotNull(cgb.rootScope), sourceModule.name, NotNull(&moduleResolver), requireCycles}; + ConstraintSolver cs{&result->internalTypes, singletonTypes, NotNull(cgb.rootScope), sourceModule.name, NotNull(&moduleResolver), requireCycles, logger.get()}; cs.run(); for (TypeError& e : cs.errors) @@ -855,9 +870,15 @@ ModulePtr Frontend::check(const SourceModule& sourceModule, Mode mode, const Sco result->astResolvedTypePacks = std::move(cgb.astResolvedTypePacks); result->type = sourceModule.type; - Luau::check(sourceModule, result.get()); + Luau::check(singletonTypes, logger.get(), sourceModule, result.get()); - result->clonePublicInterface(iceHandler); + if (FFlag::DebugLuauLogSolverToJson) + { + std::string output = logger->compileOutput(); + printf("%s\n", output.c_str()); + } + + result->clonePublicInterface(singletonTypes, iceHandler); return result; } diff --git a/Analysis/src/Linter.cpp b/Analysis/src/Linter.cpp index 669739a0..7f67a7db 100644 --- a/Analysis/src/Linter.cpp +++ b/Analysis/src/Linter.cpp @@ -14,7 +14,6 @@ LUAU_FASTINTVARIABLE(LuauSuggestionDistance, 4) LUAU_FASTFLAGVARIABLE(LuauLintGlobalNeverReadBeforeWritten, false) -LUAU_FASTFLAGVARIABLE(LuauLintComparisonPrecedence, false) LUAU_FASTFLAGVARIABLE(LuauLintFixDeprecationMessage, false) namespace Luau @@ -2954,7 +2953,7 @@ std::vector lint(AstStat* root, const AstNameTable& names, const Sc if (context.warningEnabled(LintWarning::Code_IntegerParsing)) LintIntegerParsing::process(context); - if (context.warningEnabled(LintWarning::Code_ComparisonPrecedence) && FFlag::LuauLintComparisonPrecedence) + if (context.warningEnabled(LintWarning::Code_ComparisonPrecedence)) LintComparisonPrecedence::process(context); std::sort(context.result.begin(), context.result.end(), WarningComparator()); diff --git a/Analysis/src/Module.cpp b/Analysis/src/Module.cpp index 4c9e9537..b9deac76 100644 --- a/Analysis/src/Module.cpp +++ b/Analysis/src/Module.cpp @@ -92,10 +92,12 @@ struct ForceNormal : TypeVarOnceVisitor struct ClonePublicInterface : Substitution { + NotNull singletonTypes; NotNull module; - ClonePublicInterface(const TxnLog* log, Module* module) + ClonePublicInterface(const TxnLog* log, NotNull singletonTypes, Module* module) : Substitution(log, &module->interfaceTypes) + , singletonTypes(singletonTypes) , module(module) { LUAU_ASSERT(module); @@ -147,7 +149,7 @@ struct ClonePublicInterface : Substitution else { module->errors.push_back(TypeError{module->scopes[0].first, UnificationTooComplex{}}); - return getSingletonTypes().errorRecoveryType(); + return singletonTypes->errorRecoveryType(); } } @@ -163,7 +165,7 @@ struct ClonePublicInterface : Substitution else { module->errors.push_back(TypeError{module->scopes[0].first, UnificationTooComplex{}}); - return getSingletonTypes().errorRecoveryTypePack(); + return singletonTypes->errorRecoveryTypePack(); } } @@ -208,7 +210,7 @@ Module::~Module() unfreeze(internalTypes); } -void Module::clonePublicInterface(InternalErrorReporter& ice) +void Module::clonePublicInterface(NotNull singletonTypes, InternalErrorReporter& ice) { LUAU_ASSERT(interfaceTypes.typeVars.empty()); LUAU_ASSERT(interfaceTypes.typePacks.empty()); @@ -222,7 +224,7 @@ void Module::clonePublicInterface(InternalErrorReporter& ice) std::unordered_map* exportedTypeBindings = &moduleScope->exportedTypeBindings; TxnLog log; - ClonePublicInterface clonePublicInterface{&log, this}; + ClonePublicInterface clonePublicInterface{&log, singletonTypes, this}; if (FFlag::LuauClonePublicInterfaceLess) returnType = clonePublicInterface.cloneTypePack(returnType); @@ -243,12 +245,12 @@ void Module::clonePublicInterface(InternalErrorReporter& ice) if (FFlag::LuauLowerBoundsCalculation) { - normalize(returnType, NotNull{this}, ice); + normalize(returnType, NotNull{this}, singletonTypes, ice); if (FFlag::LuauForceExportSurfacesToBeNormal) forceNormal.traverse(returnType); if (varargPack) { - normalize(*varargPack, NotNull{this}, ice); + normalize(*varargPack, NotNull{this}, singletonTypes, ice); if (FFlag::LuauForceExportSurfacesToBeNormal) forceNormal.traverse(*varargPack); } @@ -264,7 +266,7 @@ void Module::clonePublicInterface(InternalErrorReporter& ice) tf = clone(tf, interfaceTypes, cloneState); if (FFlag::LuauLowerBoundsCalculation) { - normalize(tf.type, NotNull{this}, ice); + normalize(tf.type, NotNull{this}, singletonTypes, ice); // We're about to freeze the memory. We know that the flag is conservative by design. Cyclic tables // won't be marked normal. If the types aren't normal by now, they never will be. @@ -275,7 +277,7 @@ void Module::clonePublicInterface(InternalErrorReporter& ice) if (param.defaultValue) { - normalize(*param.defaultValue, NotNull{this}, ice); + normalize(*param.defaultValue, NotNull{this}, singletonTypes, ice); forceNormal.traverse(*param.defaultValue); } } @@ -301,7 +303,7 @@ void Module::clonePublicInterface(InternalErrorReporter& ice) ty = clone(ty, interfaceTypes, cloneState); if (FFlag::LuauLowerBoundsCalculation) { - normalize(ty, NotNull{this}, ice); + normalize(ty, NotNull{this}, singletonTypes, ice); if (FFlag::LuauForceExportSurfacesToBeNormal) forceNormal.traverse(ty); diff --git a/Analysis/src/Normalize.cpp b/Analysis/src/Normalize.cpp index 94adaf5c..c3f0bb9d 100644 --- a/Analysis/src/Normalize.cpp +++ b/Analysis/src/Normalize.cpp @@ -54,11 +54,11 @@ struct Replacer } // anonymous namespace -bool isSubtype(TypeId subTy, TypeId superTy, NotNull scope, InternalErrorReporter& ice) +bool isSubtype(TypeId subTy, TypeId superTy, NotNull scope, NotNull singletonTypes, InternalErrorReporter& ice) { UnifierSharedState sharedState{&ice}; TypeArena arena; - Unifier u{&arena, Mode::Strict, scope, Location{}, Covariant, sharedState}; + Unifier u{&arena, singletonTypes, Mode::Strict, scope, Location{}, Covariant, sharedState}; u.anyIsTop = true; u.tryUnify(subTy, superTy); @@ -66,11 +66,11 @@ bool isSubtype(TypeId subTy, TypeId superTy, NotNull scope, InternalError return ok; } -bool isSubtype(TypePackId subPack, TypePackId superPack, NotNull scope, InternalErrorReporter& ice) +bool isSubtype(TypePackId subPack, TypePackId superPack, NotNull scope, NotNull singletonTypes, InternalErrorReporter& ice) { UnifierSharedState sharedState{&ice}; TypeArena arena; - Unifier u{&arena, Mode::Strict, scope, Location{}, Covariant, sharedState}; + Unifier u{&arena, singletonTypes, Mode::Strict, scope, Location{}, Covariant, sharedState}; u.anyIsTop = true; u.tryUnify(subPack, superPack); @@ -133,15 +133,17 @@ struct Normalize final : TypeVarVisitor { using TypeVarVisitor::Set; - Normalize(TypeArena& arena, NotNull scope, InternalErrorReporter& ice) + Normalize(TypeArena& arena, NotNull scope, NotNull singletonTypes, InternalErrorReporter& ice) : arena(arena) , scope(scope) + , singletonTypes(singletonTypes) , ice(ice) { } TypeArena& arena; NotNull scope; + NotNull singletonTypes; InternalErrorReporter& ice; int iterationLimit = 0; @@ -499,9 +501,9 @@ struct Normalize final : TypeVarVisitor for (TypeId& part : result) { - if (isSubtype(ty, part, scope, ice)) + if (isSubtype(ty, part, scope, singletonTypes, ice)) return; // no need to do anything - else if (isSubtype(part, ty, scope, ice)) + else if (isSubtype(part, ty, scope, singletonTypes, ice)) { part = ty; // replace the less general type by the more general one return; @@ -553,12 +555,12 @@ struct Normalize final : TypeVarVisitor bool merged = false; for (TypeId& part : result->parts) { - if (isSubtype(part, ty, scope, ice)) + if (isSubtype(part, ty, scope, singletonTypes, ice)) { merged = true; break; // no need to do anything } - else if (isSubtype(ty, part, scope, ice)) + else if (isSubtype(ty, part, scope, singletonTypes, ice)) { merged = true; part = ty; // replace the less general type by the more general one @@ -691,13 +693,14 @@ struct Normalize final : TypeVarVisitor /** * @returns A tuple of TypeId and a success indicator. (true indicates that the normalization completed successfully) */ -std::pair normalize(TypeId ty, NotNull scope, TypeArena& arena, InternalErrorReporter& ice) +std::pair normalize( + TypeId ty, NotNull scope, TypeArena& arena, NotNull singletonTypes, InternalErrorReporter& ice) { CloneState state; if (FFlag::DebugLuauCopyBeforeNormalizing) (void)clone(ty, arena, state); - Normalize n{arena, scope, ice}; + Normalize n{arena, scope, singletonTypes, ice}; n.traverse(ty); return {ty, !n.limitExceeded}; @@ -707,39 +710,40 @@ std::pair normalize(TypeId ty, NotNull scope, TypeArena& ar // reclaim memory used by wantonly allocated intermediate types here. // The main wrinkle here is that we don't want clone() to copy a type if the source and dest // arena are the same. -std::pair normalize(TypeId ty, NotNull module, InternalErrorReporter& ice) +std::pair normalize(TypeId ty, NotNull module, NotNull singletonTypes, InternalErrorReporter& ice) { - return normalize(ty, NotNull{module->getModuleScope().get()}, module->internalTypes, ice); + return normalize(ty, NotNull{module->getModuleScope().get()}, module->internalTypes, singletonTypes, ice); } -std::pair normalize(TypeId ty, const ModulePtr& module, InternalErrorReporter& ice) +std::pair normalize(TypeId ty, const ModulePtr& module, NotNull singletonTypes, InternalErrorReporter& ice) { - return normalize(ty, NotNull{module.get()}, ice); + return normalize(ty, NotNull{module.get()}, singletonTypes, ice); } /** * @returns A tuple of TypeId and a success indicator. (true indicates that the normalization completed successfully) */ -std::pair normalize(TypePackId tp, NotNull scope, TypeArena& arena, InternalErrorReporter& ice) +std::pair normalize( + TypePackId tp, NotNull scope, TypeArena& arena, NotNull singletonTypes, InternalErrorReporter& ice) { CloneState state; if (FFlag::DebugLuauCopyBeforeNormalizing) (void)clone(tp, arena, state); - Normalize n{arena, scope, ice}; + Normalize n{arena, scope, singletonTypes, ice}; n.traverse(tp); return {tp, !n.limitExceeded}; } -std::pair normalize(TypePackId tp, NotNull module, InternalErrorReporter& ice) +std::pair normalize(TypePackId tp, NotNull module, NotNull singletonTypes, InternalErrorReporter& ice) { - return normalize(tp, NotNull{module->getModuleScope().get()}, module->internalTypes, ice); + return normalize(tp, NotNull{module->getModuleScope().get()}, module->internalTypes, singletonTypes, ice); } -std::pair normalize(TypePackId tp, const ModulePtr& module, InternalErrorReporter& ice) +std::pair normalize(TypePackId tp, const ModulePtr& module, NotNull singletonTypes, InternalErrorReporter& ice) { - return normalize(tp, NotNull{module.get()}, ice); + return normalize(tp, NotNull{module.get()}, singletonTypes, ice); } } // namespace Luau diff --git a/Analysis/src/TypeArena.cpp b/Analysis/src/TypeArena.cpp index c7980ab0..abf31aee 100644 --- a/Analysis/src/TypeArena.cpp +++ b/Analysis/src/TypeArena.cpp @@ -40,6 +40,15 @@ TypeId TypeArena::freshType(Scope* scope) return allocated; } +TypePackId TypeArena::freshTypePack(Scope* scope) +{ + TypePackId allocated = typePacks.allocate(FreeTypePack{scope}); + + asMutable(allocated)->owningArena = this; + + return allocated; +} + TypePackId TypeArena::addTypePack(std::initializer_list types) { TypePackId allocated = typePacks.allocate(TypePack{std::move(types)}); diff --git a/Analysis/src/TypeChecker2.cpp b/Analysis/src/TypeChecker2.cpp index 480bdf40..88363b43 100644 --- a/Analysis/src/TypeChecker2.cpp +++ b/Analysis/src/TypeChecker2.cpp @@ -1,8 +1,6 @@ - +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #include "Luau/TypeChecker2.h" -#include - #include "Luau/Ast.h" #include "Luau/AstQuery.h" #include "Luau/Clone.h" @@ -13,6 +11,12 @@ #include "Luau/TypeUtils.h" #include "Luau/TypeVar.h" #include "Luau/Unifier.h" +#include "Luau/ToString.h" +#include "Luau/DcrLogger.h" + +#include + +LUAU_FASTFLAG(DebugLuauLogSolverToJson); namespace Luau { @@ -54,18 +58,22 @@ struct StackPusher struct TypeChecker2 { + NotNull singletonTypes; + DcrLogger* logger; + InternalErrorReporter ice; // FIXME accept a pointer from Frontend const SourceModule* sourceModule; Module* module; - InternalErrorReporter ice; // FIXME accept a pointer from Frontend - SingletonTypes& singletonTypes; std::vector> stack; - TypeChecker2(const SourceModule* sourceModule, Module* module) - : sourceModule(sourceModule) + TypeChecker2(NotNull singletonTypes, DcrLogger* logger, const SourceModule* sourceModule, Module* module) + : singletonTypes(singletonTypes) + , logger(logger) + , sourceModule(sourceModule) , module(module) - , singletonTypes(getSingletonTypes()) { + if (FFlag::DebugLuauLogSolverToJson) + LUAU_ASSERT(logger); } std::optional pushStack(AstNode* node) @@ -85,7 +93,7 @@ struct TypeChecker2 if (tp) return follow(*tp); else - return singletonTypes.anyTypePack; + return singletonTypes->anyTypePack; } TypeId lookupType(AstExpr* expr) @@ -101,7 +109,7 @@ struct TypeChecker2 if (tp) return flattenPack(*tp); - return singletonTypes.anyType; + return singletonTypes->anyType; } TypeId lookupAnnotation(AstType* annotation) @@ -253,7 +261,7 @@ struct TypeChecker2 TypePackId actualRetType = reconstructPack(ret->list, arena); UnifierSharedState sharedState{&ice}; - Unifier u{&arena, Mode::Strict, stack.back(), ret->location, Covariant, sharedState}; + Unifier u{&arena, singletonTypes, Mode::Strict, stack.back(), ret->location, Covariant, sharedState}; u.anyIsTop = true; u.tryUnify(actualRetType, expectedRetType); @@ -299,7 +307,7 @@ struct TypeChecker2 if (var->annotation) { TypeId varType = lookupAnnotation(var->annotation); - if (!isSubtype(*it, varType, stack.back(), ice)) + if (!isSubtype(*it, varType, stack.back(), singletonTypes, ice)) { reportError(TypeMismatch{varType, *it}, value->location); } @@ -317,7 +325,7 @@ struct TypeChecker2 if (var->annotation) { TypeId varType = lookupAnnotation(var->annotation); - if (!isSubtype(varType, valueType, stack.back(), ice)) + if (!isSubtype(varType, valueType, stack.back(), singletonTypes, ice)) { reportError(TypeMismatch{varType, valueType}, value->location); } @@ -340,7 +348,7 @@ struct TypeChecker2 // "Render" a type pack out to an array of a given length. Expands // variadics and various other things to get there. - static std::vector flatten(TypeArena& arena, TypePackId pack, size_t length) + std::vector flatten(TypeArena& arena, TypePackId pack, size_t length) { std::vector result; @@ -376,7 +384,7 @@ struct TypeChecker2 else if (auto etp = get(tail)) { while (result.size() < length) - result.push_back(getSingletonTypes().errorRecoveryType()); + result.push_back(singletonTypes->errorRecoveryType()); } return result; @@ -532,7 +540,7 @@ struct TypeChecker2 visit(rhs); TypeId rhsType = lookupType(rhs); - if (!isSubtype(rhsType, lhsType, stack.back(), ice)) + if (!isSubtype(rhsType, lhsType, stack.back(), singletonTypes, ice)) { reportError(TypeMismatch{lhsType, rhsType}, rhs->location); } @@ -681,9 +689,9 @@ struct TypeChecker2 void visit(AstExprConstantNumber* number) { TypeId actualType = lookupType(number); - TypeId numberType = getSingletonTypes().numberType; + TypeId numberType = singletonTypes->numberType; - if (!isSubtype(numberType, actualType, stack.back(), ice)) + if (!isSubtype(numberType, actualType, stack.back(), singletonTypes, ice)) { reportError(TypeMismatch{actualType, numberType}, number->location); } @@ -692,9 +700,9 @@ struct TypeChecker2 void visit(AstExprConstantString* string) { TypeId actualType = lookupType(string); - TypeId stringType = getSingletonTypes().stringType; + TypeId stringType = singletonTypes->stringType; - if (!isSubtype(stringType, actualType, stack.back(), ice)) + if (!isSubtype(stringType, actualType, stack.back(), singletonTypes, ice)) { reportError(TypeMismatch{actualType, stringType}, string->location); } @@ -754,7 +762,7 @@ struct TypeChecker2 FunctionTypeVar ftv{argsTp, expectedRetType}; TypeId expectedType = arena.addType(ftv); - if (!isSubtype(expectedType, instantiatedFunctionType, stack.back(), ice)) + if (!isSubtype(expectedType, instantiatedFunctionType, stack.back(), singletonTypes, ice)) { CloneState cloneState; expectedType = clone(expectedType, module->internalTypes, cloneState); @@ -773,7 +781,7 @@ struct TypeChecker2 getIndexTypeFromType(module->getModuleScope(), leftType, indexName->index.value, indexName->location, /* addErrors */ true); if (ty) { - if (!isSubtype(resultType, *ty, stack.back(), ice)) + if (!isSubtype(resultType, *ty, stack.back(), singletonTypes, ice)) { reportError(TypeMismatch{resultType, *ty}, indexName->location); } @@ -806,7 +814,7 @@ struct TypeChecker2 TypeId inferredArgTy = *argIt; TypeId annotatedArgTy = lookupAnnotation(arg->annotation); - if (!isSubtype(annotatedArgTy, inferredArgTy, stack.back(), ice)) + if (!isSubtype(annotatedArgTy, inferredArgTy, stack.back(), singletonTypes, ice)) { reportError(TypeMismatch{annotatedArgTy, inferredArgTy}, arg->location); } @@ -851,10 +859,10 @@ struct TypeChecker2 TypeId computedType = lookupType(expr->expr); // Note: As an optimization, we try 'number <: number | string' first, as that is the more likely case. - if (isSubtype(annotationType, computedType, stack.back(), ice)) + if (isSubtype(annotationType, computedType, stack.back(), singletonTypes, ice)) return; - if (isSubtype(computedType, annotationType, stack.back(), ice)) + if (isSubtype(computedType, annotationType, stack.back(), singletonTypes, ice)) return; reportError(TypesAreUnrelated{computedType, annotationType}, expr->location); @@ -908,7 +916,7 @@ struct TypeChecker2 return result; } else if (get(pack)) - return singletonTypes.errorRecoveryType(); + return singletonTypes->errorRecoveryType(); else ice.ice("flattenPack got a weird pack!"); } @@ -1154,7 +1162,7 @@ struct TypeChecker2 ErrorVec tryUnify(NotNull scope, const Location& location, TID subTy, TID superTy) { UnifierSharedState sharedState{&ice}; - Unifier u{&module->internalTypes, Mode::Strict, scope, location, Covariant, sharedState}; + Unifier u{&module->internalTypes, singletonTypes, Mode::Strict, scope, location, Covariant, sharedState}; u.anyIsTop = true; u.tryUnify(subTy, superTy); @@ -1164,6 +1172,9 @@ struct TypeChecker2 void reportError(TypeErrorData data, const Location& location) { module->errors.emplace_back(location, sourceModule->name, std::move(data)); + + if (FFlag::DebugLuauLogSolverToJson) + logger->captureTypeCheckError(module->errors.back()); } void reportError(TypeError e) @@ -1179,13 +1190,13 @@ struct TypeChecker2 std::optional getIndexTypeFromType(const ScopePtr& scope, TypeId type, const std::string& prop, const Location& location, bool addErrors) { - return Luau::getIndexTypeFromType(scope, module->errors, &module->internalTypes, type, prop, location, addErrors, ice); + return Luau::getIndexTypeFromType(scope, module->errors, &module->internalTypes, singletonTypes, type, prop, location, addErrors, ice); } }; -void check(const SourceModule& sourceModule, Module* module) +void check(NotNull singletonTypes, DcrLogger* logger, const SourceModule& sourceModule, Module* module) { - TypeChecker2 typeChecker{&sourceModule, module}; + TypeChecker2 typeChecker{singletonTypes, logger, &sourceModule, module}; typeChecker.visit(sourceModule.root); } diff --git a/Analysis/src/TypeInfer.cpp b/Analysis/src/TypeInfer.cpp index 2bda2804..c1408196 100644 --- a/Analysis/src/TypeInfer.cpp +++ b/Analysis/src/TypeInfer.cpp @@ -248,21 +248,22 @@ size_t HashBoolNamePair::operator()(const std::pair& pair) const return std::hash()(pair.first) ^ std::hash()(pair.second); } -TypeChecker::TypeChecker(ModuleResolver* resolver, InternalErrorReporter* iceHandler) +TypeChecker::TypeChecker(ModuleResolver* resolver, NotNull singletonTypes, InternalErrorReporter* iceHandler) : resolver(resolver) + , singletonTypes(singletonTypes) , iceHandler(iceHandler) , unifierState(iceHandler) - , nilType(getSingletonTypes().nilType) - , numberType(getSingletonTypes().numberType) - , stringType(getSingletonTypes().stringType) - , booleanType(getSingletonTypes().booleanType) - , threadType(getSingletonTypes().threadType) - , anyType(getSingletonTypes().anyType) - , unknownType(getSingletonTypes().unknownType) - , neverType(getSingletonTypes().neverType) - , anyTypePack(getSingletonTypes().anyTypePack) - , neverTypePack(getSingletonTypes().neverTypePack) - , uninhabitableTypePack(getSingletonTypes().uninhabitableTypePack) + , nilType(singletonTypes->nilType) + , numberType(singletonTypes->numberType) + , stringType(singletonTypes->stringType) + , booleanType(singletonTypes->booleanType) + , threadType(singletonTypes->threadType) + , anyType(singletonTypes->anyType) + , unknownType(singletonTypes->unknownType) + , neverType(singletonTypes->neverType) + , anyTypePack(singletonTypes->anyTypePack) + , neverTypePack(singletonTypes->neverTypePack) + , uninhabitableTypePack(singletonTypes->uninhabitableTypePack) , duplicateTypeAliases{{false, {}}} { globalScope = std::make_shared(globalTypes.addTypePack(TypePackVar{FreeTypePack{TypeLevel{}}})); @@ -357,7 +358,7 @@ ModulePtr TypeChecker::checkWithoutRecursionCheck(const SourceModule& module, Mo prepareErrorsForDisplay(currentModule->errors); - currentModule->clonePublicInterface(*iceHandler); + currentModule->clonePublicInterface(singletonTypes, *iceHandler); // Clear unifier cache since it's keyed off internal types that get deallocated // This avoids fake cross-module cache hits and keeps cache size at bay when typechecking large module graphs. @@ -1606,7 +1607,7 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatTypeAlias& typealias if (FFlag::LuauLowerBoundsCalculation) { - auto [t, ok] = normalize(bindingType, currentModule, *iceHandler); + auto [t, ok] = normalize(bindingType, currentModule, singletonTypes, *iceHandler); bindingType = t; if (!ok) reportError(typealias.location, NormalizationTooComplex{}); @@ -1923,7 +1924,7 @@ WithPredicate TypeChecker::checkExpr(const ScopePtr& scope, const AstExp std::optional TypeChecker::findTablePropertyRespectingMeta(TypeId lhsType, Name name, const Location& location, bool addErrors) { ErrorVec errors; - auto result = Luau::findTablePropertyRespectingMeta(errors, lhsType, name, location); + auto result = Luau::findTablePropertyRespectingMeta(singletonTypes, errors, lhsType, name, location); if (addErrors) reportErrors(errors); return result; @@ -1932,7 +1933,7 @@ std::optional TypeChecker::findTablePropertyRespectingMeta(TypeId lhsTyp std::optional TypeChecker::findMetatableEntry(TypeId type, std::string entry, const Location& location, bool addErrors) { ErrorVec errors; - auto result = Luau::findMetatableEntry(errors, type, entry, location); + auto result = Luau::findMetatableEntry(singletonTypes, errors, type, entry, location); if (addErrors) reportErrors(errors); return result; @@ -2034,8 +2035,8 @@ std::optional TypeChecker::getIndexTypeFromTypeImpl( if (FFlag::LuauLowerBoundsCalculation) { - auto [t, ok] = normalize(addType(UnionTypeVar{std::move(goodOptions)}), currentModule, - *iceHandler); // FIXME Inefficient. We craft a UnionTypeVar and immediately throw it away. + // FIXME Inefficient. We craft a UnionTypeVar and immediately throw it away. + auto [t, ok] = normalize(addType(UnionTypeVar{std::move(goodOptions)}), currentModule, singletonTypes, *iceHandler); if (!ok) reportError(location, NormalizationTooComplex{}); @@ -2642,8 +2643,8 @@ TypeId TypeChecker::checkRelationalOperation( std::string metamethodName = opToMetaTableEntry(expr.op); - std::optional leftMetatable = isString(lhsType) ? std::nullopt : getMetatable(follow(lhsType)); - std::optional rightMetatable = isString(rhsType) ? std::nullopt : getMetatable(follow(rhsType)); + std::optional leftMetatable = isString(lhsType) ? std::nullopt : getMetatable(follow(lhsType), singletonTypes); + std::optional rightMetatable = isString(rhsType) ? std::nullopt : getMetatable(follow(rhsType), singletonTypes); if (leftMetatable != rightMetatable) { @@ -2654,7 +2655,7 @@ TypeId TypeChecker::checkRelationalOperation( { for (TypeId leftOption : utv) { - if (getMetatable(follow(leftOption)) == rightMetatable) + if (getMetatable(follow(leftOption), singletonTypes) == rightMetatable) { matches = true; break; @@ -2668,7 +2669,7 @@ TypeId TypeChecker::checkRelationalOperation( { for (TypeId rightOption : utv) { - if (getMetatable(follow(rightOption)) == leftMetatable) + if (getMetatable(follow(rightOption), singletonTypes) == leftMetatable) { matches = true; break; @@ -4113,7 +4114,7 @@ std::optional> TypeChecker::checkCallOverload(const Sc std::vector adjustedArgTypes; auto it = begin(argPack); auto endIt = end(argPack); - Widen widen{¤tModule->internalTypes}; + Widen widen{¤tModule->internalTypes, singletonTypes}; for (; it != endIt; ++it) { adjustedArgTypes.push_back(addType(ConstrainedTypeVar{level, {widen(*it)}})); @@ -4649,7 +4650,7 @@ TypeId TypeChecker::quantify(const ScopePtr& scope, TypeId ty, Location location if (FFlag::LuauLowerBoundsCalculation) { - auto [t, ok] = Luau::normalize(ty, currentModule, *iceHandler); + auto [t, ok] = Luau::normalize(ty, currentModule, singletonTypes, *iceHandler); if (!ok) reportError(location, NormalizationTooComplex{}); return t; @@ -4664,7 +4665,7 @@ TypeId TypeChecker::quantify(const ScopePtr& scope, TypeId ty, Location location if (FFlag::LuauLowerBoundsCalculation && ftv) { - auto [t, ok] = Luau::normalize(ty, currentModule, *iceHandler); + auto [t, ok] = Luau::normalize(ty, currentModule, singletonTypes, *iceHandler); if (!ok) reportError(location, NormalizationTooComplex{}); return t; @@ -4701,13 +4702,13 @@ TypeId TypeChecker::anyify(const ScopePtr& scope, TypeId ty, Location location) { if (FFlag::LuauLowerBoundsCalculation) { - auto [t, ok] = normalize(ty, currentModule, *iceHandler); + auto [t, ok] = normalize(ty, currentModule, singletonTypes, *iceHandler); if (!ok) reportError(location, NormalizationTooComplex{}); ty = t; } - Anyification anyification{¤tModule->internalTypes, scope, iceHandler, anyType, anyTypePack}; + Anyification anyification{¤tModule->internalTypes, scope, singletonTypes, iceHandler, anyType, anyTypePack}; std::optional any = anyification.substitute(ty); if (anyification.normalizationTooComplex) reportError(location, NormalizationTooComplex{}); @@ -4724,13 +4725,13 @@ TypePackId TypeChecker::anyify(const ScopePtr& scope, TypePackId ty, Location lo { if (FFlag::LuauLowerBoundsCalculation) { - auto [t, ok] = normalize(ty, currentModule, *iceHandler); + auto [t, ok] = normalize(ty, currentModule, singletonTypes, *iceHandler); if (!ok) reportError(location, NormalizationTooComplex{}); ty = t; } - Anyification anyification{¤tModule->internalTypes, scope, iceHandler, anyType, anyTypePack}; + Anyification anyification{¤tModule->internalTypes, scope, singletonTypes, iceHandler, anyType, anyTypePack}; std::optional any = anyification.substitute(ty); if (any.has_value()) return *any; @@ -4868,7 +4869,8 @@ void TypeChecker::merge(RefinementMap& l, const RefinementMap& r) Unifier TypeChecker::mkUnifier(const ScopePtr& scope, const Location& location) { - return Unifier{¤tModule->internalTypes, currentModule->mode, NotNull{scope.get()}, location, Variance::Covariant, unifierState}; + return Unifier{ + ¤tModule->internalTypes, singletonTypes, currentModule->mode, NotNull{scope.get()}, location, Variance::Covariant, unifierState}; } TypeId TypeChecker::freshType(const ScopePtr& scope) @@ -4883,7 +4885,7 @@ TypeId TypeChecker::freshType(TypeLevel level) TypeId TypeChecker::singletonType(bool value) { - return value ? getSingletonTypes().trueType : getSingletonTypes().falseType; + return value ? singletonTypes->trueType : singletonTypes->falseType; } TypeId TypeChecker::singletonType(std::string value) @@ -4894,22 +4896,22 @@ TypeId TypeChecker::singletonType(std::string value) TypeId TypeChecker::errorRecoveryType(const ScopePtr& scope) { - return getSingletonTypes().errorRecoveryType(); + return singletonTypes->errorRecoveryType(); } TypeId TypeChecker::errorRecoveryType(TypeId guess) { - return getSingletonTypes().errorRecoveryType(guess); + return singletonTypes->errorRecoveryType(guess); } TypePackId TypeChecker::errorRecoveryTypePack(const ScopePtr& scope) { - return getSingletonTypes().errorRecoveryTypePack(); + return singletonTypes->errorRecoveryTypePack(); } TypePackId TypeChecker::errorRecoveryTypePack(TypePackId guess) { - return getSingletonTypes().errorRecoveryTypePack(guess); + return singletonTypes->errorRecoveryTypePack(guess); } TypeIdPredicate TypeChecker::mkTruthyPredicate(bool sense) @@ -5836,48 +5838,52 @@ void TypeChecker::resolve(const TypeGuardPredicate& typeguardP, RefinementMap& r return; } - using ConditionFunc = bool(TypeId); - using SenseToTypeIdPredicate = std::function; - auto mkFilter = [](ConditionFunc f, std::optional other = std::nullopt) -> SenseToTypeIdPredicate { - return [f, other](bool sense) -> TypeIdPredicate { - return [f, other, sense](TypeId ty) -> std::optional { - if (FFlag::LuauUnknownAndNeverType && sense && get(ty)) - return other.value_or(ty); + auto refine = [this, &lvalue = typeguardP.lvalue, &refis, &scope, sense](bool(f)(TypeId), std::optional mapsTo = std::nullopt) { + TypeIdPredicate predicate = [f, mapsTo, sense](TypeId ty) -> std::optional { + if (FFlag::LuauUnknownAndNeverType && sense && get(ty)) + return mapsTo.value_or(ty); - if (f(ty) == sense) - return ty; + if (f(ty) == sense) + return ty; - if (isUndecidable(ty)) - return other.value_or(ty); + if (isUndecidable(ty)) + return mapsTo.value_or(ty); - return std::nullopt; - }; + return std::nullopt; }; + + refineLValue(lvalue, refis, scope, predicate); }; // Note: "vector" never happens here at this point, so we don't have to write something for it. - // clang-format off - static const std::unordered_map primitives{ - // Trivial primitives. - {"nil", mkFilter(isNil, nilType)}, // This can still happen when sense is false! - {"string", mkFilter(isString, stringType)}, - {"number", mkFilter(isNumber, numberType)}, - {"boolean", mkFilter(isBoolean, booleanType)}, - {"thread", mkFilter(isThread, threadType)}, - - // Non-trivial primitives. - {"table", mkFilter([](TypeId ty) -> bool { return isTableIntersection(ty) || get(ty) || get(ty); })}, - {"function", mkFilter([](TypeId ty) -> bool { return isOverloadedFunction(ty) || get(ty); })}, - - // For now, we don't really care about being accurate with userdata if the typeguard was using typeof. - {"userdata", mkFilter([](TypeId ty) -> bool { return get(ty); })}, - }; - // clang-format on - - if (auto it = primitives.find(typeguardP.kind); it != primitives.end()) + if (typeguardP.kind == "nil") + return refine(isNil, nilType); // This can still happen when sense is false! + else if (typeguardP.kind == "string") + return refine(isString, stringType); + else if (typeguardP.kind == "number") + return refine(isNumber, numberType); + else if (typeguardP.kind == "boolean") + return refine(isBoolean, booleanType); + else if (typeguardP.kind == "thread") + return refine(isThread, threadType); + else if (typeguardP.kind == "table") { - refineLValue(typeguardP.lvalue, refis, scope, it->second(sense)); - return; + return refine([](TypeId ty) -> bool { + return isTableIntersection(ty) || get(ty) || get(ty); + }); + } + else if (typeguardP.kind == "function") + { + return refine([](TypeId ty) -> bool { + return isOverloadedFunction(ty) || get(ty); + }); + } + else if (typeguardP.kind == "userdata") + { + // For now, we don't really care about being accurate with userdata if the typeguard was using typeof. + return refine([](TypeId ty) -> bool { + return get(ty); + }); } if (!typeguardP.isTypeof) diff --git a/Analysis/src/TypeUtils.cpp b/Analysis/src/TypeUtils.cpp index 56fccecc..a96820d6 100644 --- a/Analysis/src/TypeUtils.cpp +++ b/Analysis/src/TypeUtils.cpp @@ -9,18 +9,19 @@ namespace Luau { -std::optional findMetatableEntry(ErrorVec& errors, TypeId type, const std::string& entry, Location location) +std::optional findMetatableEntry( + NotNull singletonTypes, ErrorVec& errors, TypeId type, const std::string& entry, Location location) { type = follow(type); - std::optional metatable = getMetatable(type); + std::optional metatable = getMetatable(type, singletonTypes); if (!metatable) return std::nullopt; TypeId unwrapped = follow(*metatable); if (get(unwrapped)) - return getSingletonTypes().anyType; + return singletonTypes->anyType; const TableTypeVar* mtt = getTableType(unwrapped); if (!mtt) @@ -36,7 +37,8 @@ std::optional findMetatableEntry(ErrorVec& errors, TypeId type, const st return std::nullopt; } -std::optional findTablePropertyRespectingMeta(ErrorVec& errors, TypeId ty, const std::string& name, Location location) +std::optional findTablePropertyRespectingMeta( + NotNull singletonTypes, ErrorVec& errors, TypeId ty, const std::string& name, Location location) { if (get(ty)) return ty; @@ -48,7 +50,7 @@ std::optional findTablePropertyRespectingMeta(ErrorVec& errors, TypeId t return it->second.type; } - std::optional mtIndex = findMetatableEntry(errors, ty, "__index", location); + std::optional mtIndex = findMetatableEntry(singletonTypes, errors, ty, "__index", location); int count = 0; while (mtIndex) { @@ -69,23 +71,23 @@ std::optional findTablePropertyRespectingMeta(ErrorVec& errors, TypeId t { std::optional r = first(follow(itf->retTypes)); if (!r) - return getSingletonTypes().nilType; + return singletonTypes->nilType; else return *r; } else if (get(index)) - return getSingletonTypes().anyType; + return singletonTypes->anyType; else errors.push_back(TypeError{location, GenericError{"__index should either be a function or table. Got " + toString(index)}}); - mtIndex = findMetatableEntry(errors, *mtIndex, "__index", location); + mtIndex = findMetatableEntry(singletonTypes, errors, *mtIndex, "__index", location); } return std::nullopt; } -std::optional getIndexTypeFromType(const ScopePtr& scope, ErrorVec& errors, TypeArena* arena, TypeId type, const std::string& prop, - const Location& location, bool addErrors, InternalErrorReporter& handle) +std::optional getIndexTypeFromType(const ScopePtr& scope, ErrorVec& errors, TypeArena* arena, NotNull singletonTypes, + TypeId type, const std::string& prop, const Location& location, bool addErrors, InternalErrorReporter& handle) { type = follow(type); @@ -97,14 +99,14 @@ std::optional getIndexTypeFromType(const ScopePtr& scope, ErrorVec& erro if (isString(type)) { - std::optional mtIndex = Luau::findMetatableEntry(errors, getSingletonTypes().stringType, "__index", location); + std::optional mtIndex = Luau::findMetatableEntry(singletonTypes, errors, singletonTypes->stringType, "__index", location); LUAU_ASSERT(mtIndex); type = *mtIndex; } if (getTableType(type)) { - return findTablePropertyRespectingMeta(errors, type, prop, location); + return findTablePropertyRespectingMeta(singletonTypes, errors, type, prop, location); } else if (const ClassTypeVar* cls = get(type)) { @@ -125,7 +127,8 @@ std::optional getIndexTypeFromType(const ScopePtr& scope, ErrorVec& erro if (get(follow(t))) return t; - if (std::optional ty = getIndexTypeFromType(scope, errors, arena, t, prop, location, /* addErrors= */ false, handle)) + if (std::optional ty = + getIndexTypeFromType(scope, errors, arena, singletonTypes, t, prop, location, /* addErrors= */ false, handle)) goodOptions.push_back(*ty); else badOptions.push_back(t); @@ -144,17 +147,17 @@ std::optional getIndexTypeFromType(const ScopePtr& scope, ErrorVec& erro } if (goodOptions.empty()) - return getSingletonTypes().neverType; + return singletonTypes->neverType; if (goodOptions.size() == 1) return goodOptions[0]; // TODO: inefficient. TypeId result = arena->addType(UnionTypeVar{std::move(goodOptions)}); - auto [ty, ok] = normalize(result, NotNull{scope.get()}, *arena, handle); + auto [ty, ok] = normalize(result, NotNull{scope.get()}, *arena, singletonTypes, handle); if (!ok && addErrors) errors.push_back(TypeError{location, NormalizationTooComplex{}}); - return ok ? ty : getSingletonTypes().anyType; + return ok ? ty : singletonTypes->anyType; } else if (const IntersectionTypeVar* itv = get(type)) { @@ -165,7 +168,8 @@ std::optional getIndexTypeFromType(const ScopePtr& scope, ErrorVec& erro // TODO: we should probably limit recursion here? // RecursionLimiter _rl(&recursionCount, FInt::LuauTypeInferRecursionLimit); - if (std::optional ty = getIndexTypeFromType(scope, errors, arena, t, prop, location, /* addErrors= */ false, handle)) + if (std::optional ty = + getIndexTypeFromType(scope, errors, arena, singletonTypes, t, prop, location, /* addErrors= */ false, handle)) parts.push_back(*ty); } diff --git a/Analysis/src/TypeVar.cpp b/Analysis/src/TypeVar.cpp index 4abee0f6..4f6603fb 100644 --- a/Analysis/src/TypeVar.cpp +++ b/Analysis/src/TypeVar.cpp @@ -26,6 +26,7 @@ LUAU_FASTINT(LuauTypeInferRecursionLimit) LUAU_FASTFLAG(LuauUnknownAndNeverType) LUAU_FASTFLAGVARIABLE(LuauMaybeGenericIntersectionTypes, false) LUAU_FASTFLAGVARIABLE(LuauStringFormatArgumentErrorFix, false) +LUAU_FASTFLAGVARIABLE(LuauNoMoreGlobalSingletonTypes, false) namespace Luau { @@ -239,7 +240,7 @@ bool isOverloadedFunction(TypeId ty) return std::all_of(parts.begin(), parts.end(), isFunction); } -std::optional getMetatable(TypeId type) +std::optional getMetatable(TypeId type, NotNull singletonTypes) { type = follow(type); @@ -249,7 +250,7 @@ std::optional getMetatable(TypeId type) return classType->metatable; else if (isString(type)) { - auto ptv = get(getSingletonTypes().stringType); + auto ptv = get(singletonTypes->stringType); LUAU_ASSERT(ptv && ptv->metatable); return ptv->metatable; } @@ -707,44 +708,30 @@ TypeId makeFunction(TypeArena& arena, std::optional selfType, std::initi std::initializer_list genericPacks, std::initializer_list paramTypes, std::initializer_list paramNames, std::initializer_list retTypes); -static TypeVar nilType_{PrimitiveTypeVar{PrimitiveTypeVar::NilType}, /*persistent*/ true}; -static TypeVar numberType_{PrimitiveTypeVar{PrimitiveTypeVar::Number}, /*persistent*/ true}; -static TypeVar stringType_{PrimitiveTypeVar{PrimitiveTypeVar::String}, /*persistent*/ true}; -static TypeVar booleanType_{PrimitiveTypeVar{PrimitiveTypeVar::Boolean}, /*persistent*/ true}; -static TypeVar threadType_{PrimitiveTypeVar{PrimitiveTypeVar::Thread}, /*persistent*/ true}; -static TypeVar trueType_{SingletonTypeVar{BooleanSingleton{true}}, /*persistent*/ true}; -static TypeVar falseType_{SingletonTypeVar{BooleanSingleton{false}}, /*persistent*/ true}; -static TypeVar anyType_{AnyTypeVar{}, /*persistent*/ true}; -static TypeVar unknownType_{UnknownTypeVar{}, /*persistent*/ true}; -static TypeVar neverType_{NeverTypeVar{}, /*persistent*/ true}; -static TypeVar errorType_{ErrorTypeVar{}, /*persistent*/ true}; - -static TypePackVar anyTypePack_{VariadicTypePack{&anyType_}, /*persistent*/ true}; -static TypePackVar errorTypePack_{Unifiable::Error{}, /*persistent*/ true}; -static TypePackVar neverTypePack_{VariadicTypePack{&neverType_}, /*persistent*/ true}; -static TypePackVar uninhabitableTypePack_{TypePack{{&neverType_}, &neverTypePack_}, /*persistent*/ true}; - SingletonTypes::SingletonTypes() - : nilType(&nilType_) - , numberType(&numberType_) - , stringType(&stringType_) - , booleanType(&booleanType_) - , threadType(&threadType_) - , trueType(&trueType_) - , falseType(&falseType_) - , anyType(&anyType_) - , unknownType(&unknownType_) - , neverType(&neverType_) - , anyTypePack(&anyTypePack_) - , neverTypePack(&neverTypePack_) - , uninhabitableTypePack(&uninhabitableTypePack_) - , arena(new TypeArena) + : arena(new TypeArena) + , debugFreezeArena(FFlag::DebugLuauFreezeArena) + , nilType(arena->addType(TypeVar{PrimitiveTypeVar{PrimitiveTypeVar::NilType}, /*persistent*/ true})) + , numberType(arena->addType(TypeVar{PrimitiveTypeVar{PrimitiveTypeVar::Number}, /*persistent*/ true})) + , stringType(arena->addType(TypeVar{PrimitiveTypeVar{PrimitiveTypeVar::String}, /*persistent*/ true})) + , booleanType(arena->addType(TypeVar{PrimitiveTypeVar{PrimitiveTypeVar::Boolean}, /*persistent*/ true})) + , threadType(arena->addType(TypeVar{PrimitiveTypeVar{PrimitiveTypeVar::Thread}, /*persistent*/ true})) + , trueType(arena->addType(TypeVar{SingletonTypeVar{BooleanSingleton{true}}, /*persistent*/ true})) + , falseType(arena->addType(TypeVar{SingletonTypeVar{BooleanSingleton{false}}, /*persistent*/ true})) + , anyType(arena->addType(TypeVar{AnyTypeVar{}, /*persistent*/ true})) + , unknownType(arena->addType(TypeVar{UnknownTypeVar{}, /*persistent*/ true})) + , neverType(arena->addType(TypeVar{NeverTypeVar{}, /*persistent*/ true})) + , errorType(arena->addType(TypeVar{ErrorTypeVar{}, /*persistent*/ true})) + , anyTypePack(arena->addTypePack(TypePackVar{VariadicTypePack{anyType}, /*persistent*/ true})) + , neverTypePack(arena->addTypePack(TypePackVar{VariadicTypePack{neverType}, /*persistent*/ true})) + , uninhabitableTypePack(arena->addTypePack({neverType}, neverTypePack)) + , errorTypePack(arena->addTypePack(TypePackVar{Unifiable::Error{}, /*persistent*/ true})) { TypeId stringMetatable = makeStringMetatable(); - stringType_.ty = PrimitiveTypeVar{PrimitiveTypeVar::String, stringMetatable}; + asMutable(stringType)->ty = PrimitiveTypeVar{PrimitiveTypeVar::String, stringMetatable}; persist(stringMetatable); + persist(uninhabitableTypePack); - debugFreezeArena = FFlag::DebugLuauFreezeArena; freeze(*arena); } @@ -834,12 +821,12 @@ TypeId SingletonTypes::makeStringMetatable() TypeId SingletonTypes::errorRecoveryType() { - return &errorType_; + return errorType; } TypePackId SingletonTypes::errorRecoveryTypePack() { - return &errorTypePack_; + return errorTypePack; } TypeId SingletonTypes::errorRecoveryType(TypeId guess) @@ -852,7 +839,7 @@ TypePackId SingletonTypes::errorRecoveryTypePack(TypePackId guess) return guess; } -SingletonTypes& getSingletonTypes() +SingletonTypes& DEPRECATED_getSingletonTypes() { static SingletonTypes singletonTypes; return singletonTypes; diff --git a/Analysis/src/Unifier.cpp b/Analysis/src/Unifier.cpp index b135cd0c..fd678432 100644 --- a/Analysis/src/Unifier.cpp +++ b/Analysis/src/Unifier.cpp @@ -257,12 +257,12 @@ TypeId Widen::clean(TypeId ty) LUAU_ASSERT(stv); if (get(stv)) - return getSingletonTypes().stringType; + return singletonTypes->stringType; else { // If this assert trips, it's likely we now have number singletons. LUAU_ASSERT(get(stv)); - return getSingletonTypes().booleanType; + return singletonTypes->booleanType; } } @@ -317,9 +317,10 @@ static std::optional> getTableMat return std::nullopt; } -Unifier::Unifier(TypeArena* types, Mode mode, NotNull scope, const Location& location, Variance variance, UnifierSharedState& sharedState, - TxnLog* parentLog) +Unifier::Unifier(TypeArena* types, NotNull singletonTypes, Mode mode, NotNull scope, const Location& location, + Variance variance, UnifierSharedState& sharedState, TxnLog* parentLog) : types(types) + , singletonTypes(singletonTypes) , mode(mode) , scope(scope) , log(parentLog) @@ -409,7 +410,7 @@ void Unifier::tryUnify_(TypeId subTy, TypeId superTy, bool isFunctionCall, bool { promoteTypeLevels(log, types, superFree->level, subTy); - Widen widen{types}; + Widen widen{types, singletonTypes}; log.replace(superTy, BoundTypeVar(widen(subTy))); } @@ -1018,7 +1019,7 @@ void Unifier::tryUnify_(TypePackId subTp, TypePackId superTp, bool isFunctionCal { if (!occursCheck(superTp, subTp)) { - Widen widen{types}; + Widen widen{types, singletonTypes}; log.replace(superTp, Unifiable::Bound(widen(subTp))); } } @@ -1162,13 +1163,13 @@ void Unifier::tryUnify_(TypePackId subTp, TypePackId superTp, bool isFunctionCal while (superIter.good()) { - tryUnify_(*superIter, getSingletonTypes().errorRecoveryType()); + tryUnify_(*superIter, singletonTypes->errorRecoveryType()); superIter.advance(); } while (subIter.good()) { - tryUnify_(*subIter, getSingletonTypes().errorRecoveryType()); + tryUnify_(*subIter, singletonTypes->errorRecoveryType()); subIter.advance(); } @@ -1613,7 +1614,7 @@ void Unifier::tryUnifyScalarShape(TypeId subTy, TypeId superTy, bool reversed) // Given t1 where t1 = { lower: (t1) -> (a, b...) } // It should be the case that `string <: t1` iff `(subtype's metatable).__index <: t1` - if (auto metatable = getMetatable(subTy)) + if (auto metatable = getMetatable(subTy, singletonTypes)) { auto mttv = log.get(*metatable); if (!mttv) @@ -1658,10 +1659,10 @@ TypeId Unifier::deeplyOptional(TypeId ty, std::unordered_map see TableTypeVar* resultTtv = getMutable(result); for (auto& [name, prop] : resultTtv->props) prop.type = deeplyOptional(prop.type, seen); - return types->addType(UnionTypeVar{{getSingletonTypes().nilType, result}}); + return types->addType(UnionTypeVar{{singletonTypes->nilType, result}}); } else - return types->addType(UnionTypeVar{{getSingletonTypes().nilType, ty}}); + return types->addType(UnionTypeVar{{singletonTypes->nilType, ty}}); } void Unifier::tryUnifyWithMetatable(TypeId subTy, TypeId superTy, bool reversed) @@ -1951,7 +1952,7 @@ void Unifier::tryUnifyWithAny(TypeId subTy, TypeId anyTy) anyTp = types->addTypePack(TypePackVar{VariadicTypePack{anyTy}}); else { - const TypePackId anyTypePack = types->addTypePack(TypePackVar{VariadicTypePack{getSingletonTypes().anyType}}); + const TypePackId anyTypePack = types->addTypePack(TypePackVar{VariadicTypePack{singletonTypes->anyType}}); anyTp = get(anyTy) ? anyTypePack : types->addTypePack(TypePackVar{Unifiable::Error{}}); } @@ -1960,15 +1961,15 @@ void Unifier::tryUnifyWithAny(TypeId subTy, TypeId anyTy) sharedState.tempSeenTy.clear(); sharedState.tempSeenTp.clear(); - Luau::tryUnifyWithAny(queue, *this, sharedState.tempSeenTy, sharedState.tempSeenTp, types, - FFlag::LuauUnknownAndNeverType ? anyTy : getSingletonTypes().anyType, anyTp); + Luau::tryUnifyWithAny( + queue, *this, sharedState.tempSeenTy, sharedState.tempSeenTp, types, FFlag::LuauUnknownAndNeverType ? anyTy : singletonTypes->anyType, anyTp); } void Unifier::tryUnifyWithAny(TypePackId subTy, TypePackId anyTp) { LUAU_ASSERT(get(anyTp)); - const TypeId anyTy = getSingletonTypes().errorRecoveryType(); + const TypeId anyTy = singletonTypes->errorRecoveryType(); std::vector queue; @@ -1982,7 +1983,7 @@ void Unifier::tryUnifyWithAny(TypePackId subTy, TypePackId anyTp) std::optional Unifier::findTablePropertyRespectingMeta(TypeId lhsType, Name name) { - return Luau::findTablePropertyRespectingMeta(errors, lhsType, name, location); + return Luau::findTablePropertyRespectingMeta(singletonTypes, errors, lhsType, name, location); } void Unifier::tryUnifyWithConstrainedSubTypeVar(TypeId subTy, TypeId superTy) @@ -2193,7 +2194,7 @@ bool Unifier::occursCheck(DenseHashSet& seen, TypeId needle, TypeId hays if (needle == haystack) { reportError(TypeError{location, OccursCheckFailed{}}); - log.replace(needle, *getSingletonTypes().errorRecoveryType()); + log.replace(needle, *singletonTypes->errorRecoveryType()); return true; } @@ -2250,7 +2251,7 @@ bool Unifier::occursCheck(DenseHashSet& seen, TypePackId needle, Typ if (needle == haystack) { reportError(TypeError{location, OccursCheckFailed{}}); - log.replace(needle, *getSingletonTypes().errorRecoveryTypePack()); + log.replace(needle, *singletonTypes->errorRecoveryTypePack()); return true; } @@ -2269,7 +2270,7 @@ bool Unifier::occursCheck(DenseHashSet& seen, TypePackId needle, Typ Unifier Unifier::makeChildUnifier() { - Unifier u = Unifier{types, mode, scope, location, variance, sharedState, &log}; + Unifier u = Unifier{types, singletonTypes, mode, scope, location, variance, sharedState, &log}; u.anyIsTop = anyIsTop; return u; } diff --git a/CodeGen/include/Luau/CodeAllocator.h b/CodeGen/include/Luau/CodeAllocator.h new file mode 100644 index 00000000..c80b5c38 --- /dev/null +++ b/CodeGen/include/Luau/CodeAllocator.h @@ -0,0 +1,50 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +#pragma once + +#include +#include +#include + +namespace Luau +{ +namespace CodeGen +{ + +struct CodeAllocator +{ + CodeAllocator(size_t blockSize, size_t maxTotalSize); + ~CodeAllocator(); + + // Places data and code into the executable page area + // To allow allocation while previously allocated code is already running, allocation has page granularity + // It's important to group functions together so that page alignment won't result in a lot of wasted space + bool allocate(uint8_t* data, size_t dataSize, uint8_t* code, size_t codeSize, uint8_t*& result, size_t& resultSize, uint8_t*& resultCodeStart); + + // Provided to callbacks + void* context = nullptr; + + // Called when new block is created to create and setup the unwinding information for all the code in the block + // Some platforms require this data to be placed inside the block itself, so we also return 'unwindDataSizeInBlock' + void* (*createBlockUnwindInfo)(void* context, uint8_t* block, size_t blockSize, size_t& unwindDataSizeInBlock) = nullptr; + + // Called to destroy unwinding information returned by 'createBlockUnwindInfo' + void (*destroyBlockUnwindInfo)(void* context, void* unwindData) = nullptr; + + static const size_t kMaxUnwindDataSize = 128; + + bool allocateNewBlock(size_t& unwindInfoSize); + + // Current block we use for allocations + uint8_t* blockPos = nullptr; + uint8_t* blockEnd = nullptr; + + // All allocated blocks + std::vector blocks; + std::vector unwindInfos; + + size_t blockSize = 0; + size_t maxTotalSize = 0; +}; + +} // namespace CodeGen +} // namespace Luau diff --git a/CodeGen/include/Luau/OperandX64.h b/CodeGen/include/Luau/OperandX64.h index 146beafb..432b5874 100644 --- a/CodeGen/include/Luau/OperandX64.h +++ b/CodeGen/include/Luau/OperandX64.h @@ -1,3 +1,4 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #pragma once #include "Luau/Common.h" diff --git a/CodeGen/include/Luau/RegisterX64.h b/CodeGen/include/Luau/RegisterX64.h index ae89f600..3b6e1a48 100644 --- a/CodeGen/include/Luau/RegisterX64.h +++ b/CodeGen/include/Luau/RegisterX64.h @@ -1,3 +1,4 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #pragma once #include "Luau/Common.h" diff --git a/CodeGen/src/CodeAllocator.cpp b/CodeGen/src/CodeAllocator.cpp new file mode 100644 index 00000000..f7432064 --- /dev/null +++ b/CodeGen/src/CodeAllocator.cpp @@ -0,0 +1,188 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +#include "Luau/CodeAllocator.h" + +#include "Luau/Common.h" + +#include + +#if defined(_WIN32) +#define WIN32_LEAN_AND_MEAN +#define NOMINMAX +#include + +const size_t kPageSize = 4096; +#else +#include +#include + +const size_t kPageSize = sysconf(_SC_PAGESIZE); +#endif + +static size_t alignToPageSize(size_t size) +{ + return (size + kPageSize - 1) & ~(kPageSize - 1); +} + +#if defined(_WIN32) +static uint8_t* allocatePages(size_t size) +{ + return (uint8_t*)VirtualAlloc(nullptr, alignToPageSize(size), MEM_RESERVE | MEM_COMMIT, PAGE_READWRITE); +} + +static void freePages(uint8_t* mem, size_t size) +{ + if (VirtualFree(mem, 0, MEM_RELEASE) == 0) + LUAU_ASSERT(!"failed to deallocate block memory"); +} + +static void makePagesExecutable(uint8_t* mem, size_t size) +{ + LUAU_ASSERT((uintptr_t(mem) & (kPageSize - 1)) == 0); + LUAU_ASSERT(size == alignToPageSize(size)); + + DWORD oldProtect; + if (VirtualProtect(mem, size, PAGE_EXECUTE_READ, &oldProtect) == 0) + LUAU_ASSERT(!"failed to change page protection"); +} + +static void flushInstructionCache(uint8_t* mem, size_t size) +{ + if (FlushInstructionCache(GetCurrentProcess(), mem, size) == 0) + LUAU_ASSERT(!"failed to flush instruction cache"); +} +#else +static uint8_t* allocatePages(size_t size) +{ + return (uint8_t*)mmap(nullptr, alignToPageSize(size), PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANON, -1, 0); +} + +static void freePages(uint8_t* mem, size_t size) +{ + if (munmap(mem, alignToPageSize(size)) != 0) + LUAU_ASSERT(!"failed to deallocate block memory"); +} + +static void makePagesExecutable(uint8_t* mem, size_t size) +{ + LUAU_ASSERT((uintptr_t(mem) & (kPageSize - 1)) == 0); + LUAU_ASSERT(size == alignToPageSize(size)); + + if (mprotect(mem, size, PROT_READ | PROT_EXEC) != 0) + LUAU_ASSERT(!"failed to change page protection"); +} + +static void flushInstructionCache(uint8_t* mem, size_t size) +{ + __builtin___clear_cache((char*)mem, (char*)mem + size); +} +#endif + +namespace Luau +{ +namespace CodeGen +{ + +CodeAllocator::CodeAllocator(size_t blockSize, size_t maxTotalSize) + : blockSize(blockSize) + , maxTotalSize(maxTotalSize) +{ + LUAU_ASSERT(blockSize > kMaxUnwindDataSize); + LUAU_ASSERT(maxTotalSize >= blockSize); +} + +CodeAllocator::~CodeAllocator() +{ + if (destroyBlockUnwindInfo) + { + for (void* unwindInfo : unwindInfos) + destroyBlockUnwindInfo(context, unwindInfo); + } + + for (uint8_t* block : blocks) + freePages(block, blockSize); +} + +bool CodeAllocator::allocate( + uint8_t* data, size_t dataSize, uint8_t* code, size_t codeSize, uint8_t*& result, size_t& resultSize, uint8_t*& resultCodeStart) +{ + // 'Round up' to preserve 16 byte alignment + size_t alignedDataSize = (dataSize + 15) & ~15; + + size_t totalSize = alignedDataSize + codeSize; + + // Function has to fit into a single block with unwinding information + if (totalSize > blockSize - kMaxUnwindDataSize) + return false; + + size_t unwindInfoSize = 0; + + // We might need a new block + if (totalSize > size_t(blockEnd - blockPos)) + { + if (!allocateNewBlock(unwindInfoSize)) + return false; + + LUAU_ASSERT(totalSize <= size_t(blockEnd - blockPos)); + } + + LUAU_ASSERT((uintptr_t(blockPos) & (kPageSize - 1)) == 0); // Allocation starts on page boundary + + size_t dataOffset = unwindInfoSize + alignedDataSize - dataSize; + size_t codeOffset = unwindInfoSize + alignedDataSize; + + if (dataSize) + memcpy(blockPos + dataOffset, data, dataSize); + if (codeSize) + memcpy(blockPos + codeOffset, code, codeSize); + + size_t pageSize = alignToPageSize(unwindInfoSize + totalSize); + + makePagesExecutable(blockPos, pageSize); + flushInstructionCache(blockPos + codeOffset, codeSize); + + result = blockPos + unwindInfoSize; + resultSize = totalSize; + resultCodeStart = blockPos + codeOffset; + + blockPos += pageSize; + LUAU_ASSERT((uintptr_t(blockPos) & (kPageSize - 1)) == 0); // Allocation ends on page boundary + + return true; +} + +bool CodeAllocator::allocateNewBlock(size_t& unwindInfoSize) +{ + // Stop allocating once we reach a global limit + if ((blocks.size() + 1) * blockSize > maxTotalSize) + return false; + + uint8_t* block = allocatePages(blockSize); + + if (!block) + return false; + + blockPos = block; + blockEnd = block + blockSize; + + blocks.push_back(block); + + if (createBlockUnwindInfo) + { + void* unwindInfo = createBlockUnwindInfo(context, block, blockSize, unwindInfoSize); + + // 'Round up' to preserve 16 byte alignment of the following data and code + unwindInfoSize = (unwindInfoSize + 15) & ~15; + + LUAU_ASSERT(unwindInfoSize <= kMaxUnwindDataSize); + + if (!unwindInfo) + return false; + + unwindInfos.push_back(unwindInfo); + } + + return true; +} + +} // namespace CodeGen +} // namespace Luau diff --git a/Common/include/Luau/Bytecode.h b/Common/include/Luau/Bytecode.h index decde93f..8b6ccddf 100644 --- a/Common/include/Luau/Bytecode.h +++ b/Common/include/Luau/Bytecode.h @@ -289,17 +289,19 @@ enum LuauOpcode // the first variable is then copied into index; generator/state are immutable, index isn't visible to user code LOP_FORGLOOP, - // FORGPREP_INEXT/FORGLOOP_INEXT: FORGLOOP with 2 output variables (no AUX encoding), assuming generator is luaB_inext - // FORGPREP_INEXT prepares the index variable and jumps to FORGLOOP_INEXT - // FORGLOOP_INEXT has identical encoding and semantics to FORGLOOP (except for AUX encoding) + // FORGPREP_INEXT: prepare FORGLOOP with 2 output variables (no AUX encoding), assuming generator is luaB_inext, and jump to FORGLOOP + // A: target register (see FORGLOOP for register layout) LOP_FORGPREP_INEXT, - LOP_FORGLOOP_INEXT, - // FORGPREP_NEXT/FORGLOOP_NEXT: FORGLOOP with 2 output variables (no AUX encoding), assuming generator is luaB_next - // FORGPREP_NEXT prepares the index variable and jumps to FORGLOOP_NEXT - // FORGLOOP_NEXT has identical encoding and semantics to FORGLOOP (except for AUX encoding) + // removed in v3 + LOP_DEP_FORGLOOP_INEXT, + + // FORGPREP_NEXT: prepare FORGLOOP with 2 output variables (no AUX encoding), assuming generator is luaB_next, and jump to FORGLOOP + // A: target register (see FORGLOOP for register layout) LOP_FORGPREP_NEXT, - LOP_FORGLOOP_NEXT, + + // removed in v3 + LOP_DEP_FORGLOOP_NEXT, // GETVARARGS: copy variables into the target register from vararg storage for current function // A: target register @@ -343,12 +345,9 @@ enum LuauOpcode // B: source register (for VAL/REF) or upvalue index (for UPVAL/UPREF) LOP_CAPTURE, - // JUMPIFEQK, JUMPIFNOTEQK: jumps to target offset if the comparison with constant is true (or false, for NOT variants) - // A: source register 1 - // D: jump offset (-32768..32767; 0 means "next instruction" aka "don't jump") - // AUX: constant table index - LOP_JUMPIFEQK, - LOP_JUMPIFNOTEQK, + // removed in v3 + LOP_DEP_JUMPIFEQK, + LOP_DEP_JUMPIFNOTEQK, // FASTCALL1: perform a fast call of a built-in function using 1 register argument // A: builtin function id (see LuauBuiltinFunction) diff --git a/Compiler/src/BytecodeBuilder.cpp b/Compiler/src/BytecodeBuilder.cpp index 713d08cd..2848447f 100644 --- a/Compiler/src/BytecodeBuilder.cpp +++ b/Compiler/src/BytecodeBuilder.cpp @@ -73,8 +73,6 @@ static int getOpLength(LuauOpcode op) case LOP_SETLIST: case LOP_FORGLOOP: case LOP_LOADKX: - case LOP_JUMPIFEQK: - case LOP_JUMPIFNOTEQK: case LOP_FASTCALL2: case LOP_FASTCALL2K: case LOP_JUMPXEQKNIL: @@ -106,12 +104,8 @@ inline bool isJumpD(LuauOpcode op) case LOP_FORGPREP: case LOP_FORGLOOP: case LOP_FORGPREP_INEXT: - case LOP_FORGLOOP_INEXT: case LOP_FORGPREP_NEXT: - case LOP_FORGLOOP_NEXT: case LOP_JUMPBACK: - case LOP_JUMPIFEQK: - case LOP_JUMPIFNOTEQK: case LOP_JUMPXEQKNIL: case LOP_JUMPXEQKB: case LOP_JUMPXEQKN: @@ -1247,13 +1241,6 @@ void BytecodeBuilder::validate() const VJUMP(LUAU_INSN_D(insn)); break; - case LOP_JUMPIFEQK: - case LOP_JUMPIFNOTEQK: - VREG(LUAU_INSN_A(insn)); - VCONSTANY(insns[i + 1]); - VJUMP(LUAU_INSN_D(insn)); - break; - case LOP_JUMPXEQKNIL: case LOP_JUMPXEQKB: VREG(LUAU_INSN_A(insn)); @@ -1360,9 +1347,7 @@ void BytecodeBuilder::validate() const break; case LOP_FORGPREP_INEXT: - case LOP_FORGLOOP_INEXT: case LOP_FORGPREP_NEXT: - case LOP_FORGLOOP_NEXT: VREG(LUAU_INSN_A(insn) + 4); // forg loop protocol: A, A+1, A+2 are used for iteration protocol; A+3, A+4 are loop variables VJUMP(LUAU_INSN_D(insn)); break; @@ -1728,18 +1713,10 @@ void BytecodeBuilder::dumpInstruction(const uint32_t* code, std::string& result, formatAppend(result, "FORGPREP_INEXT R%d L%d\n", LUAU_INSN_A(insn), targetLabel); break; - case LOP_FORGLOOP_INEXT: - formatAppend(result, "FORGLOOP_INEXT R%d L%d\n", LUAU_INSN_A(insn), targetLabel); - break; - case LOP_FORGPREP_NEXT: formatAppend(result, "FORGPREP_NEXT R%d L%d\n", LUAU_INSN_A(insn), targetLabel); break; - case LOP_FORGLOOP_NEXT: - formatAppend(result, "FORGLOOP_NEXT R%d L%d\n", LUAU_INSN_A(insn), targetLabel); - break; - case LOP_GETVARARGS: formatAppend(result, "GETVARARGS R%d %d\n", LUAU_INSN_A(insn), LUAU_INSN_B(insn) - 1); break; @@ -1797,14 +1774,6 @@ void BytecodeBuilder::dumpInstruction(const uint32_t* code, std::string& result, LUAU_INSN_A(insn) == LCT_UPVAL ? 'U' : 'R', LUAU_INSN_B(insn)); break; - case LOP_JUMPIFEQK: - formatAppend(result, "JUMPIFEQK R%d K%d L%d\n", LUAU_INSN_A(insn), *code++, targetLabel); - break; - - case LOP_JUMPIFNOTEQK: - formatAppend(result, "JUMPIFNOTEQK R%d K%d L%d\n", LUAU_INSN_A(insn), *code++, targetLabel); - break; - case LOP_JUMPXEQKNIL: formatAppend(result, "JUMPXEQKNIL R%d L%d%s\n", LUAU_INSN_A(insn), targetLabel, *code >> 31 ? " NOT" : ""); code++; diff --git a/Compiler/src/Compiler.cpp b/Compiler/src/Compiler.cpp index d44daf0c..ce2c5a92 100644 --- a/Compiler/src/Compiler.cpp +++ b/Compiler/src/Compiler.cpp @@ -3457,14 +3457,6 @@ struct Compiler return uint8_t(top); } - void reserveReg(AstNode* node, unsigned int count) - { - if (regTop + count > kMaxRegisterCount) - CompileError::raise(node->location, "Out of registers when trying to allocate %d registers: exceeded limit %d", count, kMaxRegisterCount); - - stackSize = std::max(stackSize, regTop + count); - } - void setDebugLine(AstNode* node) { if (options.debugLevel >= 1) diff --git a/Makefile b/Makefile index 46dfb868..f51f8ba4 100644 --- a/Makefile +++ b/Makefile @@ -142,12 +142,16 @@ coverage: $(TESTS_TARGET) llvm-cov export -ignore-filename-regex=\(tests\|extern\|CLI\)/.* -format lcov --instr-profile default.profdata build/coverage/luau-tests >coverage.info format: - find . -name '*.h' -or -name '*.cpp' | xargs clang-format-11 -i + git ls-files '*.h' '*.cpp' | xargs clang-format-11 -i luau-size: luau nm --print-size --demangle luau | grep ' t void luau_execute' | awk -F ' ' '{sum += strtonum("0x" $$2)} END {print sum " interpreter" }' nm --print-size --demangle luau | grep ' t luauF_' | awk -F ' ' '{sum += strtonum("0x" $$2)} END {print sum " builtins" }' +check-source: + git ls-files '*.h' '*.cpp' | xargs -I+ sh -c 'grep -L LICENSE +' + git ls-files '*.h' ':!:extern' | xargs -I+ sh -c 'grep -L "#pragma once" +' + # executable target aliases luau: $(REPL_CLI_TARGET) ln -fs $^ $@ diff --git a/Sources.cmake b/Sources.cmake index 50770f92..ff0b5a6e 100644 --- a/Sources.cmake +++ b/Sources.cmake @@ -56,12 +56,14 @@ target_sources(Luau.Compiler PRIVATE # Luau.CodeGen Sources target_sources(Luau.CodeGen PRIVATE CodeGen/include/Luau/AssemblyBuilderX64.h + CodeGen/include/Luau/CodeAllocator.h CodeGen/include/Luau/Condition.h CodeGen/include/Luau/Label.h CodeGen/include/Luau/OperandX64.h CodeGen/include/Luau/RegisterX64.h CodeGen/src/AssemblyBuilderX64.cpp + CodeGen/src/CodeAllocator.cpp ) # Luau.Analysis Sources @@ -77,7 +79,7 @@ target_sources(Luau.Analysis PRIVATE Analysis/include/Luau/Constraint.h Analysis/include/Luau/ConstraintGraphBuilder.h Analysis/include/Luau/ConstraintSolver.h - Analysis/include/Luau/ConstraintSolverLogger.h + Analysis/include/Luau/DcrLogger.h Analysis/include/Luau/Documentation.h Analysis/include/Luau/Error.h Analysis/include/Luau/FileResolver.h @@ -127,7 +129,7 @@ target_sources(Luau.Analysis PRIVATE Analysis/src/Constraint.cpp Analysis/src/ConstraintGraphBuilder.cpp Analysis/src/ConstraintSolver.cpp - Analysis/src/ConstraintSolverLogger.cpp + Analysis/src/DcrLogger.cpp Analysis/src/EmbeddedBuiltinDefinitions.cpp Analysis/src/Error.cpp Analysis/src/Frontend.cpp @@ -266,6 +268,7 @@ if(TARGET Luau.UnitTest) tests/AstVisitor.test.cpp tests/Autocomplete.test.cpp tests/BuiltinDefinitions.test.cpp + tests/CodeAllocator.test.cpp tests/Compiler.test.cpp tests/Config.test.cpp tests/ConstraintGraphBuilder.test.cpp diff --git a/VM/include/lua.h b/VM/include/lua.h index f986c2b3..1f386355 100644 --- a/VM/include/lua.h +++ b/VM/include/lua.h @@ -35,6 +35,15 @@ enum lua_Status LUA_BREAK, // yielded for a debug breakpoint }; +enum lua_CoStatus +{ + LUA_CORUN = 0, // running + LUA_COSUS, // suspended + LUA_CONOR, // 'normal' (it resumed another coroutine) + LUA_COFIN, // finished + LUA_COERR, // finished with error +}; + typedef struct lua_State lua_State; typedef int (*lua_CFunction)(lua_State* L); @@ -224,6 +233,7 @@ LUA_API int lua_status(lua_State* L); LUA_API int lua_isyieldable(lua_State* L); LUA_API void* lua_getthreaddata(lua_State* L); LUA_API void lua_setthreaddata(lua_State* L, void* data); +LUA_API int lua_costatus(lua_State* L, lua_State* co); /* ** garbage-collection function and options diff --git a/VM/src/lapi.cpp b/VM/src/lapi.cpp index 4396e5d1..6a9c46da 100644 --- a/VM/src/lapi.cpp +++ b/VM/src/lapi.cpp @@ -1008,6 +1008,23 @@ int lua_status(lua_State* L) return L->status; } +int lua_costatus(lua_State* L, lua_State* co) +{ + if (co == L) + return LUA_CORUN; + if (co->status == LUA_YIELD) + return LUA_COSUS; + if (co->status == LUA_BREAK) + return LUA_CONOR; + if (co->status != 0) // some error occurred + return LUA_COERR; + if (co->ci != co->base_ci) // does it have frames? + return LUA_CONOR; + if (co->top == co->base) + return LUA_COFIN; + return LUA_COSUS; // initial state +} + void* lua_getthreaddata(lua_State* L) { return L->userdata; diff --git a/VM/src/lcorolib.cpp b/VM/src/lcorolib.cpp index 7b967e34..3d39a2de 100644 --- a/VM/src/lcorolib.cpp +++ b/VM/src/lcorolib.cpp @@ -5,38 +5,16 @@ #include "lstate.h" #include "lvm.h" -#define CO_RUN 0 // running -#define CO_SUS 1 // suspended -#define CO_NOR 2 // 'normal' (it resumed another coroutine) -#define CO_DEAD 3 - #define CO_STATUS_ERROR -1 #define CO_STATUS_BREAK -2 -static const char* const statnames[] = {"running", "suspended", "normal", "dead"}; - -static int auxstatus(lua_State* L, lua_State* co) -{ - if (co == L) - return CO_RUN; - if (co->status == LUA_YIELD) - return CO_SUS; - if (co->status == LUA_BREAK) - return CO_NOR; - if (co->status != 0) // some error occurred - return CO_DEAD; - if (co->ci != co->base_ci) // does it have frames? - return CO_NOR; - if (co->top == co->base) - return CO_DEAD; - return CO_SUS; // initial state -} +static const char* const statnames[] = {"running", "suspended", "normal", "dead", "dead"}; // dead appears twice for LUA_COERR and LUA_COFIN static int costatus(lua_State* L) { lua_State* co = lua_tothread(L, 1); luaL_argexpected(L, co, 1, "thread"); - lua_pushstring(L, statnames[auxstatus(L, co)]); + lua_pushstring(L, statnames[lua_costatus(L, co)]); return 1; } @@ -45,8 +23,8 @@ static int auxresume(lua_State* L, lua_State* co, int narg) // error handling for edge cases if (co->status != LUA_YIELD) { - int status = auxstatus(L, co); - if (status != CO_SUS) + int status = lua_costatus(L, co); + if (status != LUA_COSUS) { lua_pushfstring(L, "cannot resume %s coroutine", statnames[status]); return CO_STATUS_ERROR; @@ -236,8 +214,8 @@ static int coclose(lua_State* L) lua_State* co = lua_tothread(L, 1); luaL_argexpected(L, co, 1, "thread"); - int status = auxstatus(L, co); - if (status != CO_DEAD && status != CO_SUS) + int status = lua_costatus(L, co); + if (status != LUA_COFIN && status != LUA_COERR && status != LUA_COSUS) luaL_error(L, "cannot close %s coroutine", statnames[status]); if (co->status == LUA_OK || co->status == LUA_YIELD) diff --git a/VM/src/lgc.cpp b/VM/src/lgc.cpp index b95d6dee..fb610e13 100644 --- a/VM/src/lgc.cpp +++ b/VM/src/lgc.cpp @@ -123,6 +123,7 @@ LUAU_FASTFLAGVARIABLE(LuauSimplerUpval, false) LUAU_FASTFLAGVARIABLE(LuauNoSleepBit, false) LUAU_FASTFLAGVARIABLE(LuauEagerShrink, false) +LUAU_FASTFLAGVARIABLE(LuauFasterSweep, false) #define GC_SWEEPPAGESTEPCOST 16 @@ -848,6 +849,7 @@ static size_t atomic(lua_State* L) static bool sweepgco(lua_State* L, lua_Page* page, GCObject* gco) { + LUAU_ASSERT(!FFlag::LuauFasterSweep); global_State* g = L->global; int deadmask = otherwhite(g); @@ -890,22 +892,62 @@ static int sweepgcopage(lua_State* L, lua_Page* page) int blockSize; luaM_getpagewalkinfo(page, &start, &end, &busyBlocks, &blockSize); - for (char* pos = start; pos != end; pos += blockSize) + LUAU_ASSERT(busyBlocks > 0); + + if (FFlag::LuauFasterSweep) { - GCObject* gco = (GCObject*)pos; + LUAU_ASSERT(FFlag::LuauNoSleepBit && FFlag::LuauEagerShrink); - // skip memory blocks that are already freed - if (gco->gch.tt == LUA_TNIL) - continue; + global_State* g = L->global; - // when true is returned it means that the element was deleted - if (sweepgco(L, page, gco)) + int deadmask = otherwhite(g); + LUAU_ASSERT(testbit(deadmask, FIXEDBIT)); // make sure we never sweep fixed objects + + int newwhite = luaC_white(g); + + for (char* pos = start; pos != end; pos += blockSize) { - LUAU_ASSERT(busyBlocks > 0); + GCObject* gco = (GCObject*)pos; - // if the last block was removed, page would be removed as well - if (--busyBlocks == 0) - return int(pos - start) / blockSize + 1; + // skip memory blocks that are already freed + if (gco->gch.tt == LUA_TNIL) + continue; + + // is the object alive? + if ((gco->gch.marked ^ WHITEBITS) & deadmask) + { + LUAU_ASSERT(!isdead(g, gco)); + // make it white (for next cycle) + gco->gch.marked = cast_byte((gco->gch.marked & maskmarks) | newwhite); + } + else + { + LUAU_ASSERT(isdead(g, gco)); + freeobj(L, gco, page); + + // if the last block was removed, page would be removed as well + if (--busyBlocks == 0) + return int(pos - start) / blockSize + 1; + } + } + } + else + { + for (char* pos = start; pos != end; pos += blockSize) + { + GCObject* gco = (GCObject*)pos; + + // skip memory blocks that are already freed + if (gco->gch.tt == LUA_TNIL) + continue; + + // when true is returned it means that the element was deleted + if (sweepgco(L, page, gco)) + { + // if the last block was removed, page would be removed as well + if (--busyBlocks == 0) + return int(pos - start) / blockSize + 1; + } } } @@ -993,10 +1035,19 @@ static size_t gcstep(lua_State* L, size_t limit) // nothing more to sweep? if (g->sweepgcopage == NULL) { - // don't forget to visit main thread - sweepgco(L, NULL, obj2gco(g->mainthread)); + // don't forget to visit main thread, it's the only object not allocated in GCO pages + if (FFlag::LuauFasterSweep) + { + LUAU_ASSERT(!isdead(g, obj2gco(g->mainthread))); + makewhite(g, obj2gco(g->mainthread)); // make it white (for next cycle) + } + else + { + sweepgco(L, NULL, obj2gco(g->mainthread)); + } shrinkbuffers(L); + g->gcstate = GCSpause; // end collection } break; diff --git a/VM/src/lvmexecute.cpp b/VM/src/lvmexecute.cpp index aa1da8ae..e3488916 100644 --- a/VM/src/lvmexecute.cpp +++ b/VM/src/lvmexecute.cpp @@ -107,10 +107,10 @@ LUAU_FASTFLAG(LuauNoSleepBit) VM_DISPATCH_OP(LOP_POWK), VM_DISPATCH_OP(LOP_AND), VM_DISPATCH_OP(LOP_OR), VM_DISPATCH_OP(LOP_ANDK), VM_DISPATCH_OP(LOP_ORK), \ VM_DISPATCH_OP(LOP_CONCAT), VM_DISPATCH_OP(LOP_NOT), VM_DISPATCH_OP(LOP_MINUS), VM_DISPATCH_OP(LOP_LENGTH), VM_DISPATCH_OP(LOP_NEWTABLE), \ VM_DISPATCH_OP(LOP_DUPTABLE), VM_DISPATCH_OP(LOP_SETLIST), VM_DISPATCH_OP(LOP_FORNPREP), VM_DISPATCH_OP(LOP_FORNLOOP), \ - VM_DISPATCH_OP(LOP_FORGLOOP), VM_DISPATCH_OP(LOP_FORGPREP_INEXT), VM_DISPATCH_OP(LOP_FORGLOOP_INEXT), VM_DISPATCH_OP(LOP_FORGPREP_NEXT), \ - VM_DISPATCH_OP(LOP_FORGLOOP_NEXT), VM_DISPATCH_OP(LOP_GETVARARGS), VM_DISPATCH_OP(LOP_DUPCLOSURE), VM_DISPATCH_OP(LOP_PREPVARARGS), \ + VM_DISPATCH_OP(LOP_FORGLOOP), VM_DISPATCH_OP(LOP_FORGPREP_INEXT), VM_DISPATCH_OP(LOP_DEP_FORGLOOP_INEXT), VM_DISPATCH_OP(LOP_FORGPREP_NEXT), \ + VM_DISPATCH_OP(LOP_DEP_FORGLOOP_NEXT), VM_DISPATCH_OP(LOP_GETVARARGS), VM_DISPATCH_OP(LOP_DUPCLOSURE), VM_DISPATCH_OP(LOP_PREPVARARGS), \ VM_DISPATCH_OP(LOP_LOADKX), VM_DISPATCH_OP(LOP_JUMPX), VM_DISPATCH_OP(LOP_FASTCALL), VM_DISPATCH_OP(LOP_COVERAGE), \ - VM_DISPATCH_OP(LOP_CAPTURE), VM_DISPATCH_OP(LOP_JUMPIFEQK), VM_DISPATCH_OP(LOP_JUMPIFNOTEQK), VM_DISPATCH_OP(LOP_FASTCALL1), \ + VM_DISPATCH_OP(LOP_CAPTURE), VM_DISPATCH_OP(LOP_DEP_JUMPIFEQK), VM_DISPATCH_OP(LOP_DEP_JUMPIFNOTEQK), VM_DISPATCH_OP(LOP_FASTCALL1), \ VM_DISPATCH_OP(LOP_FASTCALL2), VM_DISPATCH_OP(LOP_FASTCALL2K), VM_DISPATCH_OP(LOP_FORGPREP), VM_DISPATCH_OP(LOP_JUMPXEQKNIL), \ VM_DISPATCH_OP(LOP_JUMPXEQKB), VM_DISPATCH_OP(LOP_JUMPXEQKN), VM_DISPATCH_OP(LOP_JUMPXEQKS), @@ -2401,7 +2401,7 @@ static void luau_execute(lua_State* L) VM_NEXT(); } - VM_CASE(LOP_FORGLOOP_INEXT) + VM_CASE(LOP_DEP_FORGLOOP_INEXT) { VM_INTERRUPT(); Instruction insn = *pc++; @@ -2473,7 +2473,7 @@ static void luau_execute(lua_State* L) VM_NEXT(); } - VM_CASE(LOP_FORGLOOP_NEXT) + VM_CASE(LOP_DEP_FORGLOOP_NEXT) { VM_INTERRUPT(); Instruction insn = *pc++; @@ -2748,7 +2748,7 @@ static void luau_execute(lua_State* L) LUAU_UNREACHABLE(); } - VM_CASE(LOP_JUMPIFEQK) + VM_CASE(LOP_DEP_JUMPIFEQK) { Instruction insn = *pc++; uint32_t aux = *pc; @@ -2793,7 +2793,7 @@ static void luau_execute(lua_State* L) } } - VM_CASE(LOP_JUMPIFNOTEQK) + VM_CASE(LOP_DEP_JUMPIFNOTEQK) { Instruction insn = *pc++; uint32_t aux = *pc; diff --git a/tests/Autocomplete.test.cpp b/tests/Autocomplete.test.cpp index 988cbe80..a64d372f 100644 --- a/tests/Autocomplete.test.cpp +++ b/tests/Autocomplete.test.cpp @@ -107,8 +107,8 @@ struct ACFixture : ACFixtureImpl ACFixture() : ACFixtureImpl() { - addGlobalBinding(frontend.typeChecker, "table", Binding{typeChecker.anyType}); - addGlobalBinding(frontend.typeChecker, "math", Binding{typeChecker.anyType}); + addGlobalBinding(frontend, "table", Binding{typeChecker.anyType}); + addGlobalBinding(frontend, "math", Binding{typeChecker.anyType}); addGlobalBinding(frontend.typeCheckerForAutocomplete, "table", Binding{typeChecker.anyType}); addGlobalBinding(frontend.typeCheckerForAutocomplete, "math", Binding{typeChecker.anyType}); } @@ -3200,8 +3200,6 @@ a.@1 TEST_CASE_FIXTURE(ACFixture, "globals_are_order_independent") { - ScopedFastFlag sff("LuauAutocompleteFixGlobalOrder", true); - check(R"( local myLocal = 4 function abc0() diff --git a/tests/CodeAllocator.test.cpp b/tests/CodeAllocator.test.cpp new file mode 100644 index 00000000..41f3a900 --- /dev/null +++ b/tests/CodeAllocator.test.cpp @@ -0,0 +1,162 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +#include "Luau/AssemblyBuilderX64.h" +#include "Luau/CodeAllocator.h" + +#include "doctest.h" + +#include + +using namespace Luau::CodeGen; + +TEST_SUITE_BEGIN("CodeAllocation"); + +TEST_CASE("CodeAllocation") +{ + size_t blockSize = 1024 * 1024; + size_t maxTotalSize = 1024 * 1024; + CodeAllocator allocator(blockSize, maxTotalSize); + + uint8_t* nativeData = nullptr; + size_t sizeNativeData = 0; + uint8_t* nativeEntry = nullptr; + + std::vector code; + code.resize(128); + + REQUIRE(allocator.allocate(nullptr, 0, code.data(), code.size(), nativeData, sizeNativeData, nativeEntry)); + CHECK(nativeData != nullptr); + CHECK(sizeNativeData == 128); + CHECK(nativeEntry != nullptr); + CHECK(nativeEntry == nativeData); + + std::vector data; + data.resize(8); + + REQUIRE(allocator.allocate(data.data(), data.size(), code.data(), code.size(), nativeData, sizeNativeData, nativeEntry)); + CHECK(nativeData != nullptr); + CHECK(sizeNativeData == 16 + 128); + CHECK(nativeEntry != nullptr); + CHECK(nativeEntry == nativeData + 16); +} + +TEST_CASE("CodeAllocationFailure") +{ + size_t blockSize = 16384; + size_t maxTotalSize = 32768; + CodeAllocator allocator(blockSize, maxTotalSize); + + uint8_t* nativeData; + size_t sizeNativeData; + uint8_t* nativeEntry; + + std::vector code; + code.resize(18000); + + // allocation has to fit in a block + REQUIRE(!allocator.allocate(nullptr, 0, code.data(), code.size(), nativeData, sizeNativeData, nativeEntry)); + + // each allocation exhausts a block, so third allocation fails + code.resize(10000); + REQUIRE(allocator.allocate(nullptr, 0, code.data(), code.size(), nativeData, sizeNativeData, nativeEntry)); + REQUIRE(allocator.allocate(nullptr, 0, code.data(), code.size(), nativeData, sizeNativeData, nativeEntry)); + REQUIRE(!allocator.allocate(nullptr, 0, code.data(), code.size(), nativeData, sizeNativeData, nativeEntry)); +} + +TEST_CASE("CodeAllocationWithUnwindCallbacks") +{ + struct Info + { + std::vector unwind; + uint8_t* block = nullptr; + bool destroyCalled = false; + }; + Info info; + info.unwind.resize(8); + + { + size_t blockSize = 1024 * 1024; + size_t maxTotalSize = 1024 * 1024; + CodeAllocator allocator(blockSize, maxTotalSize); + + uint8_t* nativeData = nullptr; + size_t sizeNativeData = 0; + uint8_t* nativeEntry = nullptr; + + std::vector code; + code.resize(128); + + std::vector data; + data.resize(8); + + allocator.context = &info; + allocator.createBlockUnwindInfo = [](void* context, uint8_t* block, size_t blockSize, size_t& unwindDataSizeInBlock) -> void* { + Info& info = *(Info*)context; + + CHECK(info.unwind.size() == 8); + memcpy(block, info.unwind.data(), info.unwind.size()); + unwindDataSizeInBlock = 8; + + info.block = block; + + return new int(7); + }; + allocator.destroyBlockUnwindInfo = [](void* context, void* unwindData) { + Info& info = *(Info*)context; + + info.destroyCalled = true; + + CHECK(*(int*)unwindData == 7); + delete (int*)unwindData; + }; + + REQUIRE(allocator.allocate(data.data(), data.size(), code.data(), code.size(), nativeData, sizeNativeData, nativeEntry)); + CHECK(nativeData != nullptr); + CHECK(sizeNativeData == 16 + 128); + CHECK(nativeEntry != nullptr); + CHECK(nativeEntry == nativeData + 16); + CHECK(nativeData == info.block + 16); + } + + CHECK(info.destroyCalled); +} + +#if defined(__x86_64__) || defined(_M_X64) +TEST_CASE("GeneratedCodeExecution") +{ +#if defined(_WIN32) + // Windows x64 ABI + constexpr RegisterX64 rArg1 = rcx; + constexpr RegisterX64 rArg2 = rdx; +#else + // System V AMD64 ABI + constexpr RegisterX64 rArg1 = rdi; + constexpr RegisterX64 rArg2 = rsi; +#endif + + AssemblyBuilderX64 build(/* logText= */ false); + + build.mov(rax, rArg1); + build.add(rax, rArg2); + build.imul(rax, rax, 7); + build.ret(); + + build.finalize(); + + size_t blockSize = 1024 * 1024; + size_t maxTotalSize = 1024 * 1024; + CodeAllocator allocator(blockSize, maxTotalSize); + + uint8_t* nativeData; + size_t sizeNativeData; + uint8_t* nativeEntry; + REQUIRE(allocator.allocate(build.data.data(), build.data.size(), build.code.data(), build.code.size(), nativeData, sizeNativeData, nativeEntry)); + REQUIRE(nativeEntry); + + using FunctionType = int64_t(int64_t, int64_t); + FunctionType* f = (FunctionType*)nativeEntry; + int64_t result = f(10, 20); + CHECK(result == 210); +} +#endif + +TEST_SUITE_END(); diff --git a/tests/Compiler.test.cpp b/tests/Compiler.test.cpp index d8520a6e..e6222b02 100644 --- a/tests/Compiler.test.cpp +++ b/tests/Compiler.test.cpp @@ -6100,6 +6100,7 @@ return math.round(7.6), bit32.extract(-1, 31), bit32.replace(100, 1, 0), + math.log(100, 10), (type("fin")) )", 0, 2), @@ -6153,8 +6154,9 @@ LOADN R45 1 LOADN R46 8 LOADN R47 1 LOADN R48 101 -LOADK R49 K3 -RETURN R0 50 +LOADN R49 2 +LOADK R50 K3 +RETURN R0 51 )"); } @@ -6166,7 +6168,12 @@ return math.max(1, true), string.byte("abc", 42), bit32.rshift(10, 42), - bit32.extract(1, 2, "3") + bit32.extract(1, 2, "3"), + bit32.bor(1, true), + bit32.band(1, true), + bit32.bxor(1, true), + bit32.btest(1, true), + math.min(1, true) )", 0, 2), R"( @@ -6193,11 +6200,96 @@ LOADN R6 2 LOADK R7 K14 FASTCALL 34 L4 GETIMPORT R4 16 -CALL R4 3 -1 -L4: RETURN R0 -1 +CALL R4 3 1 +L4: LOADN R6 1 +FASTCALL2K 31 R6 K3 L5 +LOADK R7 K3 +GETIMPORT R5 18 +CALL R5 2 1 +L5: LOADN R7 1 +FASTCALL2K 29 R7 K3 L6 +LOADK R8 K3 +GETIMPORT R6 20 +CALL R6 2 1 +L6: LOADN R8 1 +FASTCALL2K 32 R8 K3 L7 +LOADK R9 K3 +GETIMPORT R7 22 +CALL R7 2 1 +L7: LOADN R9 1 +FASTCALL2K 33 R9 K3 L8 +LOADK R10 K3 +GETIMPORT R8 24 +CALL R8 2 1 +L8: LOADN R10 1 +FASTCALL2K 19 R10 K3 L9 +LOADK R11 K3 +GETIMPORT R9 26 +CALL R9 2 -1 +L9: RETURN R0 -1 )"); } +TEST_CASE("BuiltinFoldingProhibitedCoverage") +{ + const char* builtins[] = { + "math.abs", + "math.acos", + "math.asin", + "math.atan2", + "math.atan", + "math.ceil", + "math.cosh", + "math.cos", + "math.deg", + "math.exp", + "math.floor", + "math.fmod", + "math.ldexp", + "math.log10", + "math.log", + "math.max", + "math.min", + "math.pow", + "math.rad", + "math.sinh", + "math.sin", + "math.sqrt", + "math.tanh", + "math.tan", + "bit32.arshift", + "bit32.band", + "bit32.bnot", + "bit32.bor", + "bit32.bxor", + "bit32.btest", + "bit32.extract", + "bit32.lrotate", + "bit32.lshift", + "bit32.replace", + "bit32.rrotate", + "bit32.rshift", + "type", + "string.byte", + "string.len", + "typeof", + "math.clamp", + "math.sign", + "math.round", + }; + + for (const char* func : builtins) + { + std::string source = "return "; + source += func; + source += "()"; + + std::string bc = compileFunction(source.c_str(), 0, 2); + + CHECK(bc.find("FASTCALL") != std::string::npos); + } +} + TEST_CASE("BuiltinFoldingMultret") { CHECK_EQ("\n" + compileFunction(R"( diff --git a/tests/Conformance.test.cpp b/tests/Conformance.test.cpp index a7ffb493..c6bdb4db 100644 --- a/tests/Conformance.test.cpp +++ b/tests/Conformance.test.cpp @@ -496,7 +496,8 @@ TEST_CASE("Types") runConformance("types.lua", [](lua_State* L) { Luau::NullModuleResolver moduleResolver; Luau::InternalErrorReporter iceHandler; - Luau::TypeChecker env(&moduleResolver, &iceHandler); + Luau::SingletonTypes singletonTypes; + Luau::TypeChecker env(&moduleResolver, Luau::NotNull{&singletonTypes}, &iceHandler); Luau::registerBuiltinTypes(env); Luau::freeze(env.globalTypes); diff --git a/tests/ConstraintSolver.test.cpp b/tests/ConstraintSolver.test.cpp index 2c489733..fba57823 100644 --- a/tests/ConstraintSolver.test.cpp +++ b/tests/ConstraintSolver.test.cpp @@ -26,10 +26,10 @@ TEST_CASE_FIXTURE(ConstraintGraphBuilderFixture, "hello") )"); cgb.visit(block); - NotNull rootScope = NotNull(cgb.rootScope); + NotNull rootScope{cgb.rootScope}; NullModuleResolver resolver; - ConstraintSolver cs{&arena, rootScope, "MainModule", NotNull(&resolver), {}}; + ConstraintSolver cs{&arena, singletonTypes, rootScope, "MainModule", NotNull(&resolver), {}, &logger}; cs.run(); @@ -47,10 +47,10 @@ TEST_CASE_FIXTURE(ConstraintGraphBuilderFixture, "generic_function") )"); cgb.visit(block); - NotNull rootScope = NotNull(cgb.rootScope); + NotNull rootScope{cgb.rootScope}; NullModuleResolver resolver; - ConstraintSolver cs{&arena, rootScope, "MainModule", NotNull(&resolver), {}}; + ConstraintSolver cs{&arena, singletonTypes, rootScope, "MainModule", NotNull(&resolver), {}, &logger}; cs.run(); @@ -74,12 +74,12 @@ TEST_CASE_FIXTURE(ConstraintGraphBuilderFixture, "proper_let_generalization") )"); cgb.visit(block); - NotNull rootScope = NotNull(cgb.rootScope); + NotNull rootScope{cgb.rootScope}; ToStringOptions opts; NullModuleResolver resolver; - ConstraintSolver cs{&arena, rootScope, "MainModule", NotNull(&resolver), {}}; + ConstraintSolver cs{&arena, singletonTypes, rootScope, "MainModule", NotNull(&resolver), {}, &logger}; cs.run(); diff --git a/tests/CostModel.test.cpp b/tests/CostModel.test.cpp index eacc718b..d82d5d83 100644 --- a/tests/CostModel.test.cpp +++ b/tests/CostModel.test.cpp @@ -1,6 +1,8 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #include "Luau/Parser.h" +#include "ScopedFlags.h" + #include "doctest.h" using namespace Luau; @@ -223,4 +225,21 @@ end CHECK_EQ(6, Luau::Compile::computeCost(model, args2, 1)); } +TEST_CASE("InterpString") +{ + ScopedFastFlag sff("LuauInterpolatedStringBaseSupport", true); + + uint64_t model = modelFunction(R"( +function test(a) + return `hello, {a}!` +end +)"); + + const bool args1[] = {false}; + const bool args2[] = {true}; + + CHECK_EQ(3, Luau::Compile::computeCost(model, args1, 1)); + CHECK_EQ(3, Luau::Compile::computeCost(model, args2, 1)); +} + TEST_SUITE_END(); diff --git a/tests/Fixture.cpp b/tests/Fixture.cpp index 476b7a2a..6c4594f4 100644 --- a/tests/Fixture.cpp +++ b/tests/Fixture.cpp @@ -91,6 +91,7 @@ Fixture::Fixture(bool freeze, bool prepareAutocomplete) : sff_DebugLuauFreezeArena("DebugLuauFreezeArena", freeze) , frontend(&fileResolver, &configResolver, {/* retainFullTypeGraphs= */ true}) , typeChecker(frontend.typeChecker) + , singletonTypes(frontend.singletonTypes) { configResolver.defaultConfig.mode = Mode::Strict; configResolver.defaultConfig.enabledLint.warningMask = ~0ull; @@ -367,9 +368,9 @@ void Fixture::dumpErrors(std::ostream& os, const std::vector& errors) void Fixture::registerTestTypes() { - addGlobalBinding(typeChecker, "game", typeChecker.anyType, "@luau"); - addGlobalBinding(typeChecker, "workspace", typeChecker.anyType, "@luau"); - addGlobalBinding(typeChecker, "script", typeChecker.anyType, "@luau"); + addGlobalBinding(frontend, "game", typeChecker.anyType, "@luau"); + addGlobalBinding(frontend, "workspace", typeChecker.anyType, "@luau"); + addGlobalBinding(frontend, "script", typeChecker.anyType, "@luau"); } void Fixture::dumpErrors(const CheckResult& cr) @@ -434,7 +435,7 @@ BuiltinsFixture::BuiltinsFixture(bool freeze, bool prepareAutocomplete) Luau::unfreeze(frontend.typeChecker.globalTypes); Luau::unfreeze(frontend.typeCheckerForAutocomplete.globalTypes); - registerBuiltinTypes(frontend.typeChecker); + registerBuiltinTypes(frontend); if (prepareAutocomplete) registerBuiltinTypes(frontend.typeCheckerForAutocomplete); registerTestTypes(); @@ -446,7 +447,7 @@ BuiltinsFixture::BuiltinsFixture(bool freeze, bool prepareAutocomplete) ConstraintGraphBuilderFixture::ConstraintGraphBuilderFixture() : Fixture() , mainModule(new Module) - , cgb(mainModuleName, mainModule, &arena, NotNull(&moduleResolver), NotNull(&ice), frontend.getGlobalScope()) + , cgb(mainModuleName, mainModule, &arena, NotNull(&moduleResolver), singletonTypes, NotNull(&ice), frontend.getGlobalScope(), &logger) , forceTheFlag{"DebugLuauDeferredConstraintResolution", true} { BlockedTypeVar::nextIndex = 0; diff --git a/tests/Fixture.h b/tests/Fixture.h index 8923b208..03101bbf 100644 --- a/tests/Fixture.h +++ b/tests/Fixture.h @@ -12,6 +12,7 @@ #include "Luau/ToString.h" #include "Luau/TypeInfer.h" #include "Luau/TypeVar.h" +#include "Luau/DcrLogger.h" #include "IostreamOptional.h" #include "ScopedFlags.h" @@ -137,6 +138,7 @@ struct Fixture Frontend frontend; InternalErrorReporter ice; TypeChecker& typeChecker; + NotNull singletonTypes; std::string decorateWithTypes(const std::string& code); @@ -165,6 +167,7 @@ struct ConstraintGraphBuilderFixture : Fixture TypeArena arena; ModulePtr mainModule; ConstraintGraphBuilder cgb; + DcrLogger logger; ScopedFastFlag forceTheFlag; diff --git a/tests/Frontend.test.cpp b/tests/Frontend.test.cpp index 3c27ad54..a8a9e044 100644 --- a/tests/Frontend.test.cpp +++ b/tests/Frontend.test.cpp @@ -81,8 +81,8 @@ struct FrontendFixture : BuiltinsFixture { FrontendFixture() { - addGlobalBinding(typeChecker, "game", frontend.typeChecker.anyType, "@test"); - addGlobalBinding(typeChecker, "script", frontend.typeChecker.anyType, "@test"); + addGlobalBinding(frontend, "game", frontend.typeChecker.anyType, "@test"); + addGlobalBinding(frontend, "script", frontend.typeChecker.anyType, "@test"); } }; diff --git a/tests/LValue.test.cpp b/tests/LValue.test.cpp index 606f6de3..0bb91ece 100644 --- a/tests/LValue.test.cpp +++ b/tests/LValue.test.cpp @@ -34,23 +34,28 @@ static LValue mkSymbol(const std::string& s) return Symbol{AstName{s.data()}}; } +struct LValueFixture +{ + SingletonTypes singletonTypes; +}; + TEST_SUITE_BEGIN("LValue"); -TEST_CASE("Luau_merge_hashmap_order") +TEST_CASE_FIXTURE(LValueFixture, "Luau_merge_hashmap_order") { std::string a = "a"; std::string b = "b"; std::string c = "c"; RefinementMap m{{ - {mkSymbol(b), getSingletonTypes().stringType}, - {mkSymbol(c), getSingletonTypes().numberType}, + {mkSymbol(b), singletonTypes.stringType}, + {mkSymbol(c), singletonTypes.numberType}, }}; RefinementMap other{{ - {mkSymbol(a), getSingletonTypes().stringType}, - {mkSymbol(b), getSingletonTypes().stringType}, - {mkSymbol(c), getSingletonTypes().booleanType}, + {mkSymbol(a), singletonTypes.stringType}, + {mkSymbol(b), singletonTypes.stringType}, + {mkSymbol(c), singletonTypes.booleanType}, }}; TypeArena arena; @@ -66,21 +71,21 @@ TEST_CASE("Luau_merge_hashmap_order") CHECK_EQ("boolean | number", toString(m[mkSymbol(c)])); } -TEST_CASE("Luau_merge_hashmap_order2") +TEST_CASE_FIXTURE(LValueFixture, "Luau_merge_hashmap_order2") { std::string a = "a"; std::string b = "b"; std::string c = "c"; RefinementMap m{{ - {mkSymbol(a), getSingletonTypes().stringType}, - {mkSymbol(b), getSingletonTypes().stringType}, - {mkSymbol(c), getSingletonTypes().numberType}, + {mkSymbol(a), singletonTypes.stringType}, + {mkSymbol(b), singletonTypes.stringType}, + {mkSymbol(c), singletonTypes.numberType}, }}; RefinementMap other{{ - {mkSymbol(b), getSingletonTypes().stringType}, - {mkSymbol(c), getSingletonTypes().booleanType}, + {mkSymbol(b), singletonTypes.stringType}, + {mkSymbol(c), singletonTypes.booleanType}, }}; TypeArena arena; @@ -96,7 +101,7 @@ TEST_CASE("Luau_merge_hashmap_order2") CHECK_EQ("boolean | number", toString(m[mkSymbol(c)])); } -TEST_CASE("one_map_has_overlap_at_end_whereas_other_has_it_in_start") +TEST_CASE_FIXTURE(LValueFixture, "one_map_has_overlap_at_end_whereas_other_has_it_in_start") { std::string a = "a"; std::string b = "b"; @@ -105,15 +110,15 @@ TEST_CASE("one_map_has_overlap_at_end_whereas_other_has_it_in_start") std::string e = "e"; RefinementMap m{{ - {mkSymbol(a), getSingletonTypes().stringType}, - {mkSymbol(b), getSingletonTypes().numberType}, - {mkSymbol(c), getSingletonTypes().booleanType}, + {mkSymbol(a), singletonTypes.stringType}, + {mkSymbol(b), singletonTypes.numberType}, + {mkSymbol(c), singletonTypes.booleanType}, }}; RefinementMap other{{ - {mkSymbol(c), getSingletonTypes().stringType}, - {mkSymbol(d), getSingletonTypes().numberType}, - {mkSymbol(e), getSingletonTypes().booleanType}, + {mkSymbol(c), singletonTypes.stringType}, + {mkSymbol(d), singletonTypes.numberType}, + {mkSymbol(e), singletonTypes.booleanType}, }}; TypeArena arena; @@ -133,7 +138,7 @@ TEST_CASE("one_map_has_overlap_at_end_whereas_other_has_it_in_start") CHECK_EQ("boolean", toString(m[mkSymbol(e)])); } -TEST_CASE("hashing_lvalue_global_prop_access") +TEST_CASE_FIXTURE(LValueFixture, "hashing_lvalue_global_prop_access") { std::string t1 = "t"; std::string x1 = "x"; @@ -154,13 +159,13 @@ TEST_CASE("hashing_lvalue_global_prop_access") CHECK_EQ(LValueHasher{}(t_x2), LValueHasher{}(t_x2)); RefinementMap m; - m[t_x1] = getSingletonTypes().stringType; - m[t_x2] = getSingletonTypes().numberType; + m[t_x1] = singletonTypes.stringType; + m[t_x2] = singletonTypes.numberType; CHECK_EQ(1, m.size()); } -TEST_CASE("hashing_lvalue_local_prop_access") +TEST_CASE_FIXTURE(LValueFixture, "hashing_lvalue_local_prop_access") { std::string t1 = "t"; std::string x1 = "x"; @@ -183,8 +188,8 @@ TEST_CASE("hashing_lvalue_local_prop_access") CHECK_EQ(LValueHasher{}(t_x2), LValueHasher{}(t_x2)); RefinementMap m; - m[t_x1] = getSingletonTypes().stringType; - m[t_x2] = getSingletonTypes().numberType; + m[t_x1] = singletonTypes.stringType; + m[t_x2] = singletonTypes.numberType; CHECK_EQ(2, m.size()); } diff --git a/tests/Linter.test.cpp b/tests/Linter.test.cpp index b560c89e..8c7d762e 100644 --- a/tests/Linter.test.cpp +++ b/tests/Linter.test.cpp @@ -35,7 +35,7 @@ TEST_CASE_FIXTURE(Fixture, "UnknownGlobal") TEST_CASE_FIXTURE(Fixture, "DeprecatedGlobal") { // Normally this would be defined externally, so hack it in for testing - addGlobalBinding(typeChecker, "Wait", Binding{typeChecker.anyType, {}, true, "wait", "@test/global/Wait"}); + addGlobalBinding(frontend, "Wait", Binding{typeChecker.anyType, {}, true, "wait", "@test/global/Wait"}); LintResult result = lintTyped("Wait(5)"); @@ -49,7 +49,7 @@ TEST_CASE_FIXTURE(Fixture, "DeprecatedGlobalNoReplacement") // Normally this would be defined externally, so hack it in for testing const char* deprecationReplacementString = ""; - addGlobalBinding(typeChecker, "Version", Binding{typeChecker.anyType, {}, true, deprecationReplacementString}); + addGlobalBinding(frontend, "Version", Binding{typeChecker.anyType, {}, true, deprecationReplacementString}); LintResult result = lintTyped("Version()"); @@ -380,7 +380,7 @@ return bar() TEST_CASE_FIXTURE(Fixture, "ImportUnused") { // Normally this would be defined externally, so hack it in for testing - addGlobalBinding(typeChecker, "game", typeChecker.anyType, "@test"); + addGlobalBinding(frontend, "game", typeChecker.anyType, "@test"); LintResult result = lint(R"( local Roact = require(game.Packages.Roact) @@ -1464,7 +1464,7 @@ TEST_CASE_FIXTURE(Fixture, "DeprecatedApi") getMutable(colorType)->props = {{"toHSV", {typeChecker.anyType, /* deprecated= */ true, "Color3:ToHSV"}}}; - addGlobalBinding(typeChecker, "Color3", Binding{colorType, {}}); + addGlobalBinding(frontend, "Color3", Binding{colorType, {}}); freeze(typeChecker.globalTypes); @@ -1737,8 +1737,6 @@ local _ = 0x0xffffffffffffffffffffffffffffffffff TEST_CASE_FIXTURE(Fixture, "ComparisonPrecedence") { - ScopedFastFlag sff("LuauLintComparisonPrecedence", true); - LintResult result = lint(R"( local a, b = ... diff --git a/tests/Module.test.cpp b/tests/Module.test.cpp index 5ec375c1..58d38994 100644 --- a/tests/Module.test.cpp +++ b/tests/Module.test.cpp @@ -10,6 +10,7 @@ using namespace Luau; +LUAU_FASTFLAG(DebugLuauDeferredConstraintResolution); LUAU_FASTFLAG(LuauLowerBoundsCalculation); TEST_SUITE_BEGIN("ModuleTests"); @@ -134,7 +135,10 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "builtin_types_point_into_globalTypes_arena") REQUIRE(signType != nullptr); CHECK(!isInArena(signType, module->interfaceTypes)); - CHECK(isInArena(signType, typeChecker.globalTypes)); + if (FFlag::DebugLuauDeferredConstraintResolution) + CHECK(isInArena(signType, frontend.globalTypes)); + else + CHECK(isInArena(signType, typeChecker.globalTypes)); } TEST_CASE_FIXTURE(Fixture, "deepClone_union") @@ -230,7 +234,7 @@ TEST_CASE_FIXTURE(Fixture, "clone_constrained_intersection") { TypeArena src; - TypeId constrained = src.addType(ConstrainedTypeVar{TypeLevel{}, {getSingletonTypes().numberType, getSingletonTypes().stringType}}); + TypeId constrained = src.addType(ConstrainedTypeVar{TypeLevel{}, {singletonTypes->numberType, singletonTypes->stringType}}); TypeArena dest; CloneState cloneState; @@ -240,8 +244,8 @@ TEST_CASE_FIXTURE(Fixture, "clone_constrained_intersection") const ConstrainedTypeVar* ctv = get(cloned); REQUIRE_EQ(2, ctv->parts.size()); - CHECK_EQ(getSingletonTypes().numberType, ctv->parts[0]); - CHECK_EQ(getSingletonTypes().stringType, ctv->parts[1]); + CHECK_EQ(singletonTypes->numberType, ctv->parts[0]); + CHECK_EQ(singletonTypes->stringType, ctv->parts[1]); } TEST_CASE_FIXTURE(BuiltinsFixture, "clone_self_property") diff --git a/tests/Normalize.test.cpp b/tests/Normalize.test.cpp index b017d8dd..42e9a933 100644 --- a/tests/Normalize.test.cpp +++ b/tests/Normalize.test.cpp @@ -15,13 +15,13 @@ struct NormalizeFixture : Fixture bool isSubtype(TypeId a, TypeId b) { - return ::Luau::isSubtype(a, b, NotNull{getMainModule()->getModuleScope().get()}, ice); + return ::Luau::isSubtype(a, b, NotNull{getMainModule()->getModuleScope().get()}, singletonTypes, ice); } }; -void createSomeClasses(TypeChecker& typeChecker) +void createSomeClasses(Frontend& frontend) { - auto& arena = typeChecker.globalTypes; + auto& arena = frontend.globalTypes; unfreeze(arena); @@ -32,23 +32,23 @@ void createSomeClasses(TypeChecker& typeChecker) parentClass->props["virtual_method"] = {makeFunction(arena, parentType, {}, {})}; - addGlobalBinding(typeChecker, "Parent", {parentType}); - typeChecker.globalScope->exportedTypeBindings["Parent"] = TypeFun{{}, parentType}; + addGlobalBinding(frontend, "Parent", {parentType}); + frontend.getGlobalScope()->exportedTypeBindings["Parent"] = TypeFun{{}, parentType}; TypeId childType = arena.addType(ClassTypeVar{"Child", {}, parentType, std::nullopt, {}, nullptr, "Test"}); ClassTypeVar* childClass = getMutable(childType); childClass->props["virtual_method"] = {makeFunction(arena, childType, {}, {})}; - addGlobalBinding(typeChecker, "Child", {childType}); - typeChecker.globalScope->exportedTypeBindings["Child"] = TypeFun{{}, childType}; + addGlobalBinding(frontend, "Child", {childType}); + frontend.getGlobalScope()->exportedTypeBindings["Child"] = TypeFun{{}, childType}; TypeId unrelatedType = arena.addType(ClassTypeVar{"Unrelated", {}, std::nullopt, std::nullopt, {}, nullptr, "Test"}); - addGlobalBinding(typeChecker, "Unrelated", {unrelatedType}); - typeChecker.globalScope->exportedTypeBindings["Unrelated"] = TypeFun{{}, unrelatedType}; + addGlobalBinding(frontend, "Unrelated", {unrelatedType}); + frontend.getGlobalScope()->exportedTypeBindings["Unrelated"] = TypeFun{{}, unrelatedType}; - for (const auto& [name, ty] : typeChecker.globalScope->exportedTypeBindings) + for (const auto& [name, ty] : frontend.getGlobalScope()->exportedTypeBindings) persist(ty.type); freeze(arena); @@ -508,7 +508,7 @@ TEST_CASE_FIXTURE(NormalizeFixture, "cyclic_table") TEST_CASE_FIXTURE(NormalizeFixture, "classes") { - createSomeClasses(typeChecker); + createSomeClasses(frontend); check(""); // Ensure that we have a main Module. @@ -596,7 +596,7 @@ TEST_CASE_FIXTURE(NormalizeFixture, "union_with_overlapping_field_that_has_a_sub )"); ModulePtr tempModule{new Module}; - tempModule->scopes.emplace_back(Location(), std::make_shared(getSingletonTypes().anyTypePack)); + tempModule->scopes.emplace_back(Location(), std::make_shared(singletonTypes->anyTypePack)); // HACK: Normalization is an in-place operation. We need to cheat a little here and unfreeze // the arena that the type lives in. @@ -604,7 +604,7 @@ TEST_CASE_FIXTURE(NormalizeFixture, "union_with_overlapping_field_that_has_a_sub unfreeze(mainModule->internalTypes); TypeId tType = requireType("t"); - normalize(tType, tempModule, *typeChecker.iceHandler); + normalize(tType, tempModule, singletonTypes, *typeChecker.iceHandler); CHECK_EQ("{| x: number? |}", toString(tType, {true})); } @@ -1085,7 +1085,7 @@ TEST_CASE_FIXTURE(Fixture, "bound_typevars_should_only_be_marked_normal_if_their TEST_CASE_FIXTURE(BuiltinsFixture, "skip_force_normal_on_external_types") { - createSomeClasses(typeChecker); + createSomeClasses(frontend); CheckResult result = check(R"( export type t0 = { a: Child } diff --git a/tests/NotNull.test.cpp b/tests/NotNull.test.cpp index e77ba78a..dfa06aa1 100644 --- a/tests/NotNull.test.cpp +++ b/tests/NotNull.test.cpp @@ -1,3 +1,4 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #include "Luau/NotNull.h" #include "doctest.h" diff --git a/tests/TypeInfer.aliases.test.cpp b/tests/TypeInfer.aliases.test.cpp index e487fd48..dd91467d 100644 --- a/tests/TypeInfer.aliases.test.cpp +++ b/tests/TypeInfer.aliases.test.cpp @@ -76,8 +76,8 @@ TEST_CASE_FIXTURE(Fixture, "cannot_steal_hoisted_type_alias") Location{{1, 21}, {1, 26}}, getMainSourceModule()->name, TypeMismatch{ - getSingletonTypes().numberType, - getSingletonTypes().stringType, + singletonTypes->numberType, + singletonTypes->stringType, }, }); } @@ -87,8 +87,8 @@ TEST_CASE_FIXTURE(Fixture, "cannot_steal_hoisted_type_alias") Location{{1, 8}, {1, 26}}, getMainSourceModule()->name, TypeMismatch{ - getSingletonTypes().numberType, - getSingletonTypes().stringType, + singletonTypes->numberType, + singletonTypes->stringType, }, }); } @@ -501,7 +501,7 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "type_alias_import_mutation") CheckResult result = check("type t10 = typeof(table)"); LUAU_REQUIRE_NO_ERRORS(result); - TypeId ty = getGlobalBinding(frontend.typeChecker, "table"); + TypeId ty = getGlobalBinding(frontend, "table"); CHECK_EQ(toString(ty), "table"); const TableTypeVar* ttv = get(ty); diff --git a/tests/TypeInfer.annotations.test.cpp b/tests/TypeInfer.annotations.test.cpp index 8a86ee5f..5d18b335 100644 --- a/tests/TypeInfer.annotations.test.cpp +++ b/tests/TypeInfer.annotations.test.cpp @@ -557,7 +557,7 @@ TEST_CASE_FIXTURE(Fixture, "cloned_interface_maintains_pointers_between_definiti TEST_CASE_FIXTURE(BuiltinsFixture, "use_type_required_from_another_file") { - addGlobalBinding(frontend.typeChecker, "script", frontend.typeChecker.anyType, "@test"); + addGlobalBinding(frontend, "script", frontend.typeChecker.anyType, "@test"); fileResolver.source["Modules/Main"] = R"( --!strict @@ -583,7 +583,7 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "use_type_required_from_another_file") TEST_CASE_FIXTURE(BuiltinsFixture, "cannot_use_nonexported_type") { - addGlobalBinding(frontend.typeChecker, "script", frontend.typeChecker.anyType, "@test"); + addGlobalBinding(frontend, "script", frontend.typeChecker.anyType, "@test"); fileResolver.source["Modules/Main"] = R"( --!strict @@ -609,7 +609,7 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "cannot_use_nonexported_type") TEST_CASE_FIXTURE(BuiltinsFixture, "builtin_types_are_not_exported") { - addGlobalBinding(frontend.typeChecker, "script", frontend.typeChecker.anyType, "@test"); + addGlobalBinding(frontend, "script", frontend.typeChecker.anyType, "@test"); fileResolver.source["Modules/Main"] = R"( --!strict diff --git a/tests/TypeInfer.classes.test.cpp b/tests/TypeInfer.classes.test.cpp index 6f4191e3..98883dfa 100644 --- a/tests/TypeInfer.classes.test.cpp +++ b/tests/TypeInfer.classes.test.cpp @@ -32,7 +32,7 @@ struct ClassFixture : BuiltinsFixture {"New", {makeFunction(arena, nullopt, {}, {baseClassInstanceType})}}, }; typeChecker.globalScope->exportedTypeBindings["BaseClass"] = TypeFun{{}, baseClassInstanceType}; - addGlobalBinding(typeChecker, "BaseClass", baseClassType, "@test"); + addGlobalBinding(frontend, "BaseClass", baseClassType, "@test"); TypeId childClassInstanceType = arena.addType(ClassTypeVar{"ChildClass", {}, baseClassInstanceType, nullopt, {}, {}, "Test"}); @@ -45,7 +45,7 @@ struct ClassFixture : BuiltinsFixture {"New", {makeFunction(arena, nullopt, {}, {childClassInstanceType})}}, }; typeChecker.globalScope->exportedTypeBindings["ChildClass"] = TypeFun{{}, childClassInstanceType}; - addGlobalBinding(typeChecker, "ChildClass", childClassType, "@test"); + addGlobalBinding(frontend, "ChildClass", childClassType, "@test"); TypeId grandChildInstanceType = arena.addType(ClassTypeVar{"GrandChild", {}, childClassInstanceType, nullopt, {}, {}, "Test"}); @@ -58,7 +58,7 @@ struct ClassFixture : BuiltinsFixture {"New", {makeFunction(arena, nullopt, {}, {grandChildInstanceType})}}, }; typeChecker.globalScope->exportedTypeBindings["GrandChild"] = TypeFun{{}, grandChildInstanceType}; - addGlobalBinding(typeChecker, "GrandChild", childClassType, "@test"); + addGlobalBinding(frontend, "GrandChild", childClassType, "@test"); TypeId anotherChildInstanceType = arena.addType(ClassTypeVar{"AnotherChild", {}, baseClassInstanceType, nullopt, {}, {}, "Test"}); @@ -71,7 +71,7 @@ struct ClassFixture : BuiltinsFixture {"New", {makeFunction(arena, nullopt, {}, {anotherChildInstanceType})}}, }; typeChecker.globalScope->exportedTypeBindings["AnotherChild"] = TypeFun{{}, anotherChildInstanceType}; - addGlobalBinding(typeChecker, "AnotherChild", childClassType, "@test"); + addGlobalBinding(frontend, "AnotherChild", childClassType, "@test"); TypeId vector2MetaType = arena.addType(TableTypeVar{}); @@ -89,7 +89,7 @@ struct ClassFixture : BuiltinsFixture {"__add", {makeFunction(arena, nullopt, {vector2InstanceType, vector2InstanceType}, {vector2InstanceType})}}, }; typeChecker.globalScope->exportedTypeBindings["Vector2"] = TypeFun{{}, vector2InstanceType}; - addGlobalBinding(typeChecker, "Vector2", vector2Type, "@test"); + addGlobalBinding(frontend, "Vector2", vector2Type, "@test"); for (const auto& [name, tf] : typeChecker.globalScope->exportedTypeBindings) persist(tf.type); diff --git a/tests/TypeInfer.definitions.test.cpp b/tests/TypeInfer.definitions.test.cpp index 9fe0c6aa..26280c13 100644 --- a/tests/TypeInfer.definitions.test.cpp +++ b/tests/TypeInfer.definitions.test.cpp @@ -19,13 +19,13 @@ TEST_CASE_FIXTURE(Fixture, "definition_file_simple") declare foo2: typeof(foo) )"); - TypeId globalFooTy = getGlobalBinding(frontend.typeChecker, "foo"); + TypeId globalFooTy = getGlobalBinding(frontend, "foo"); CHECK_EQ(toString(globalFooTy), "number"); - TypeId globalBarTy = getGlobalBinding(frontend.typeChecker, "bar"); + TypeId globalBarTy = getGlobalBinding(frontend, "bar"); CHECK_EQ(toString(globalBarTy), "(number) -> string"); - TypeId globalFoo2Ty = getGlobalBinding(frontend.typeChecker, "foo2"); + TypeId globalFoo2Ty = getGlobalBinding(frontend, "foo2"); CHECK_EQ(toString(globalFoo2Ty), "number"); CheckResult result = check(R"( @@ -48,20 +48,20 @@ TEST_CASE_FIXTURE(Fixture, "definition_file_loading") declare function var(...: any): string )"); - TypeId globalFooTy = getGlobalBinding(frontend.typeChecker, "foo"); + TypeId globalFooTy = getGlobalBinding(frontend, "foo"); CHECK_EQ(toString(globalFooTy), "number"); - std::optional globalAsdfTy = frontend.typeChecker.globalScope->lookupType("Asdf"); + std::optional globalAsdfTy = frontend.getGlobalScope()->lookupType("Asdf"); REQUIRE(bool(globalAsdfTy)); CHECK_EQ(toString(globalAsdfTy->type), "number | string"); - TypeId globalBarTy = getGlobalBinding(frontend.typeChecker, "bar"); + TypeId globalBarTy = getGlobalBinding(frontend, "bar"); CHECK_EQ(toString(globalBarTy), "(number) -> string"); - TypeId globalFoo2Ty = getGlobalBinding(frontend.typeChecker, "foo2"); + TypeId globalFoo2Ty = getGlobalBinding(frontend, "foo2"); CHECK_EQ(toString(globalFoo2Ty), "number"); - TypeId globalVarTy = getGlobalBinding(frontend.typeChecker, "var"); + TypeId globalVarTy = getGlobalBinding(frontend, "var"); CHECK_EQ(toString(globalVarTy), "(...any) -> string"); @@ -85,7 +85,7 @@ TEST_CASE_FIXTURE(Fixture, "load_definition_file_errors_do_not_pollute_global_sc freeze(typeChecker.globalTypes); REQUIRE(!parseFailResult.success); - std::optional fooTy = tryGetGlobalBinding(typeChecker, "foo"); + std::optional fooTy = tryGetGlobalBinding(frontend, "foo"); CHECK(!fooTy.has_value()); LoadDefinitionFileResult checkFailResult = loadDefinitionFile(typeChecker, typeChecker.globalScope, R"( @@ -95,7 +95,7 @@ TEST_CASE_FIXTURE(Fixture, "load_definition_file_errors_do_not_pollute_global_sc "@test"); REQUIRE(!checkFailResult.success); - std::optional barTy = tryGetGlobalBinding(typeChecker, "bar"); + std::optional barTy = tryGetGlobalBinding(frontend, "bar"); CHECK(!barTy.has_value()); } diff --git a/tests/TypeInfer.functions.test.cpp b/tests/TypeInfer.functions.test.cpp index 3a6f4491..35e67ec5 100644 --- a/tests/TypeInfer.functions.test.cpp +++ b/tests/TypeInfer.functions.test.cpp @@ -127,6 +127,8 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "vararg_function_is_quantified") return T )"); + LUAU_REQUIRE_NO_ERRORS(result); + auto r = first(getMainModule()->getModuleScope()->returnType); REQUIRE(r); @@ -136,8 +138,6 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "vararg_function_is_quantified") REQUIRE(ttv->props.count("f")); TypeId k = ttv->props["f"].type; REQUIRE(k); - - LUAU_REQUIRE_NO_ERRORS(result); } TEST_CASE_FIXTURE(Fixture, "list_only_alternative_overloads_that_match_argument_count") diff --git a/tests/TypeInfer.primitives.test.cpp b/tests/TypeInfer.primitives.test.cpp index 5a6fb0e4..a31c9c50 100644 --- a/tests/TypeInfer.primitives.test.cpp +++ b/tests/TypeInfer.primitives.test.cpp @@ -102,4 +102,18 @@ end CHECK_EQ(toString(result.errors[1]), "Type 'number' could not be converted into 'string'"); } +TEST_CASE("singleton_types") +{ + BuiltinsFixture a; + + { + BuiltinsFixture b; + } + + // Check that Frontend 'a' environment wasn't modified by 'b' + CheckResult result = a.check("local s: string = 'hello' local t = s:lower()"); + + CHECK(result.errors.empty()); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.provisional.test.cpp b/tests/TypeInfer.provisional.test.cpp index 472d0ed5..3482b75c 100644 --- a/tests/TypeInfer.provisional.test.cpp +++ b/tests/TypeInfer.provisional.test.cpp @@ -353,6 +353,9 @@ TEST_CASE_FIXTURE(Fixture, "weird_fail_to_unify_type_pack") { ScopedFastFlag sff[] = { {"LuauLowerBoundsCalculation", false}, + // I'm not sure why this is broken without DCR, but it seems to be fixed + // when DCR is enabled. + {"DebugLuauDeferredConstraintResolution", false}, }; CheckResult result = check(R"( @@ -367,6 +370,9 @@ TEST_CASE_FIXTURE(Fixture, "weird_fail_to_unify_variadic_pack") { ScopedFastFlag sff[] = { {"LuauLowerBoundsCalculation", false}, + // I'm not sure why this is broken without DCR, but it seems to be fixed + // when DCR is enabled. + {"DebugLuauDeferredConstraintResolution", false}, }; CheckResult result = check(R"( @@ -588,9 +594,9 @@ TEST_CASE_FIXTURE(Fixture, "free_options_cannot_be_unified_together") }; TypeArena arena; - TypeId nilType = getSingletonTypes().nilType; + TypeId nilType = singletonTypes->nilType; - std::unique_ptr scope = std::make_unique(getSingletonTypes().anyTypePack); + std::unique_ptr scope = std::make_unique(singletonTypes->anyTypePack); TypeId free1 = arena.addType(FreeTypePack{scope.get()}); TypeId option1 = arena.addType(UnionTypeVar{{nilType, free1}}); @@ -600,7 +606,7 @@ TEST_CASE_FIXTURE(Fixture, "free_options_cannot_be_unified_together") InternalErrorReporter iceHandler; UnifierSharedState sharedState{&iceHandler}; - Unifier u{&arena, Mode::Strict, NotNull{scope.get()}, Location{}, Variance::Covariant, sharedState}; + Unifier u{&arena, singletonTypes, Mode::Strict, NotNull{scope.get()}, Location{}, Variance::Covariant, sharedState}; u.tryUnify(option1, option2); diff --git a/tests/TypeInfer.refinements.test.cpp b/tests/TypeInfer.refinements.test.cpp index e61e6e45..d2bff9c3 100644 --- a/tests/TypeInfer.refinements.test.cpp +++ b/tests/TypeInfer.refinements.test.cpp @@ -50,7 +50,7 @@ struct RefinementClassFixture : Fixture {"Y", Property{typeChecker.numberType}}, {"Z", Property{typeChecker.numberType}}, }; - normalize(vec3, scope, arena, *typeChecker.iceHandler); + normalize(vec3, scope, arena, singletonTypes, *typeChecker.iceHandler); TypeId inst = arena.addType(ClassTypeVar{"Instance", {}, std::nullopt, std::nullopt, {}, nullptr, "Test"}); @@ -58,21 +58,21 @@ struct RefinementClassFixture : Fixture TypePackId isARets = arena.addTypePack({typeChecker.booleanType}); TypeId isA = arena.addType(FunctionTypeVar{isAParams, isARets}); getMutable(isA)->magicFunction = magicFunctionInstanceIsA; - normalize(isA, scope, arena, *typeChecker.iceHandler); + normalize(isA, scope, arena, singletonTypes, *typeChecker.iceHandler); getMutable(inst)->props = { {"Name", Property{typeChecker.stringType}}, {"IsA", Property{isA}}, }; - normalize(inst, scope, arena, *typeChecker.iceHandler); + normalize(inst, scope, arena, singletonTypes, *typeChecker.iceHandler); TypeId folder = typeChecker.globalTypes.addType(ClassTypeVar{"Folder", {}, inst, std::nullopt, {}, nullptr, "Test"}); - normalize(folder, scope, arena, *typeChecker.iceHandler); + normalize(folder, scope, arena, singletonTypes, *typeChecker.iceHandler); TypeId part = typeChecker.globalTypes.addType(ClassTypeVar{"Part", {}, inst, std::nullopt, {}, nullptr, "Test"}); getMutable(part)->props = { {"Position", Property{vec3}}, }; - normalize(part, scope, arena, *typeChecker.iceHandler); + normalize(part, scope, arena, singletonTypes, *typeChecker.iceHandler); typeChecker.globalScope->exportedTypeBindings["Vector3"] = TypeFun{{}, vec3}; typeChecker.globalScope->exportedTypeBindings["Instance"] = TypeFun{{}, inst}; diff --git a/tests/TypeInfer.tryUnify.test.cpp b/tests/TypeInfer.tryUnify.test.cpp index 02fdfd73..7fa0fac0 100644 --- a/tests/TypeInfer.tryUnify.test.cpp +++ b/tests/TypeInfer.tryUnify.test.cpp @@ -18,7 +18,7 @@ struct TryUnifyFixture : Fixture InternalErrorReporter iceHandler; UnifierSharedState unifierState{&iceHandler}; - Unifier state{&arena, Mode::Strict, NotNull{globalScope.get()}, Location{}, Variance::Covariant, unifierState}; + Unifier state{&arena, singletonTypes, Mode::Strict, NotNull{globalScope.get()}, Location{}, Variance::Covariant, unifierState}; }; TEST_SUITE_BEGIN("TryUnifyTests"); diff --git a/tests/TypeInfer.typePacks.cpp b/tests/TypeInfer.typePacks.cpp index 7aefa00d..eaa8b053 100644 --- a/tests/TypeInfer.typePacks.cpp +++ b/tests/TypeInfer.typePacks.cpp @@ -194,7 +194,7 @@ TEST_CASE_FIXTURE(Fixture, "variadic_packs") TypePackId listOfStrings = arena.addTypePack(TypePackVar{VariadicTypePack{typeChecker.stringType}}); // clang-format off - addGlobalBinding(typeChecker, "foo", + addGlobalBinding(frontend, "foo", arena.addType( FunctionTypeVar{ listOfNumbers, @@ -203,7 +203,7 @@ TEST_CASE_FIXTURE(Fixture, "variadic_packs") ), "@test" ); - addGlobalBinding(typeChecker, "bar", + addGlobalBinding(frontend, "bar", arena.addType( FunctionTypeVar{ arena.addTypePack({{typeChecker.numberType}, listOfStrings}), diff --git a/tests/TypeVar.test.cpp b/tests/TypeVar.test.cpp index 32be8215..b81c80ce 100644 --- a/tests/TypeVar.test.cpp +++ b/tests/TypeVar.test.cpp @@ -273,7 +273,7 @@ TEST_CASE_FIXTURE(Fixture, "substitution_skip_failure") TypeId root = &ttvTweenResult; typeChecker.currentModule = std::make_shared(); - typeChecker.currentModule->scopes.emplace_back(Location{}, std::make_shared(getSingletonTypes().anyTypePack)); + typeChecker.currentModule->scopes.emplace_back(Location{}, std::make_shared(singletonTypes->anyTypePack)); TypeId result = typeChecker.anyify(typeChecker.globalScope, root, Location{}); diff --git a/tests/conformance/types.lua b/tests/conformance/types.lua index cdddceef..3539b34d 100644 --- a/tests/conformance/types.lua +++ b/tests/conformance/types.lua @@ -10,9 +10,6 @@ local ignore = -- what follows is a set of mismatches that hopefully eventually will go down to 0 "_G.require", -- need to move to Roblox type defs - "_G.utf8.nfcnormalize", -- need to move to Roblox type defs - "_G.utf8.nfdnormalize", -- need to move to Roblox type defs - "_G.utf8.graphemes", -- need to move to Roblox type defs } function verify(real, rtti, path) diff --git a/tools/faillist.txt b/tools/faillist.txt index 3de9db76..ef995aa6 100644 --- a/tools/faillist.txt +++ b/tools/faillist.txt @@ -163,6 +163,7 @@ BuiltinTests.table_pack_reduce BuiltinTests.table_pack_variadic BuiltinTests.tonumber_returns_optional_number_type BuiltinTests.tonumber_returns_optional_number_type2 +DefinitionTests.class_definition_overload_metamethods DefinitionTests.declaring_generic_functions DefinitionTests.definition_file_classes FrontendTest.ast_node_at_position @@ -199,7 +200,6 @@ GenericsTests.generic_functions_in_types GenericsTests.generic_functions_should_be_memory_safe GenericsTests.generic_table_method GenericsTests.generic_type_pack_parentheses -GenericsTests.generic_type_pack_syntax GenericsTests.generic_type_pack_unification1 GenericsTests.generic_type_pack_unification2 GenericsTests.generic_type_pack_unification3 @@ -300,10 +300,8 @@ ProvisionalTests.operator_eq_completely_incompatible ProvisionalTests.pcall_returns_at_least_two_value_but_function_returns_nothing ProvisionalTests.setmetatable_constrains_free_type_into_free_table ProvisionalTests.typeguard_inference_incomplete -ProvisionalTests.weird_fail_to_unify_type_pack ProvisionalTests.weirditer_should_not_loop_forever ProvisionalTests.while_body_are_also_refined -ProvisionalTests.xpcall_returns_what_f_returns RefinementTest.and_constraint RefinementTest.and_or_peephole_refinement RefinementTest.apply_refinements_on_astexprindexexpr_whose_subscript_expr_is_constant_string @@ -494,13 +492,10 @@ ToString.function_type_with_argument_names_generic ToString.no_parentheses_around_cyclic_function_type_in_union ToString.toStringDetailed2 ToString.toStringErrorPack -ToString.toStringNamedFunction_generic_pack ToString.toStringNamedFunction_hide_type_params ToString.toStringNamedFunction_id ToString.toStringNamedFunction_map -ToString.toStringNamedFunction_overrides_param_names ToString.toStringNamedFunction_variadics -TranspilerTests.type_lists_should_be_emitted_correctly TranspilerTests.types_should_not_be_considered_cyclic_if_they_are_not_recursive TryUnifyTests.cli_41095_concat_log_in_sealed_table_unification TryUnifyTests.members_of_failed_typepack_unification_are_unified_with_errorType @@ -616,7 +611,6 @@ TypeInferFunctions.too_few_arguments_variadic_generic2 TypeInferFunctions.too_many_arguments TypeInferFunctions.too_many_return_values TypeInferFunctions.vararg_function_is_quantified -TypeInferFunctions.vararg_functions_should_allow_calls_of_any_types_and_size TypeInferLoops.for_in_loop_error_on_factory_not_returning_the_right_amount_of_values TypeInferLoops.for_in_loop_with_custom_iterator TypeInferLoops.for_in_loop_with_next @@ -641,7 +635,6 @@ TypeInferOOP.dont_suggest_using_colon_rather_than_dot_if_not_defined_with_colon TypeInferOOP.inferred_methods_of_free_tables_have_the_same_level_as_the_enclosing_table TypeInferOOP.inferring_hundreds_of_self_calls_should_not_suffocate_memory TypeInferOOP.methods_are_topologically_sorted -TypeInferOOP.nonstrict_self_mismatch_tail TypeInferOperators.and_adds_boolean TypeInferOperators.and_adds_boolean_no_superfluous_union TypeInferOperators.and_binexps_dont_unify @@ -689,6 +682,7 @@ TypeInferOperators.unary_not_is_boolean TypeInferOperators.unknown_type_in_comparison TypeInferOperators.UnknownGlobalCompoundAssign TypeInferPrimitives.CheckMethodsOfNumber +TypeInferPrimitives.singleton_types TypeInferPrimitives.string_function_other TypeInferPrimitives.string_index TypeInferPrimitives.string_length @@ -730,7 +724,6 @@ TypePackTests.type_alias_type_packs_nested TypePackTests.type_pack_hidden_free_tail_infinite_growth TypePackTests.type_pack_type_parameters TypePackTests.varargs_inference_through_multiple_scopes -TypePackTests.variadic_pack_syntax TypePackTests.variadic_packs TypeSingletons.bool_singleton_subtype TypeSingletons.bool_singletons From 366df9639384e8d7ab92b29744303adec6770d77 Mon Sep 17 00:00:00 2001 From: Allan N Jeremy Date: Mon, 12 Sep 2022 18:45:55 +0300 Subject: [PATCH 358/399] Feat: Added retry functionality to benchmarks (#667) Resolves #668 ## The problem Benchmarks jobs run concurrently for the different operating systems. This means that when it comes time to push the benchmark results to [the assigned benchmark results repo](https://github.com/luau-lang/benchmark-data), there can be two different jobs trying to push changes at the same time. In such a case, one of the pushes will fail and we end up missing some benchmark results data from the workflow run. ## The solution Whenever a push fails, we need to retry the steps leading up to the push (checking out the benchmark results repo, storing benchmark results, pushing the results to [a specific repo](https://github.com/luau-lang/benchmark-data)). ### Note There are 3 push attempts before submitting to failure. ## TL;DR This PR retries pushing benchmark results when they fail to get pushed (often due to pushing from multiple jobs concurrently) Co-authored-by: Jamie Kuppens Co-authored-by: Ignacio Falk --- .github/workflows/benchmark-dev.yml | 275 +++++++++++++++------- .github/workflows/push-results/action.yml | 63 +++++ 2 files changed, 258 insertions(+), 80 deletions(-) create mode 100644 .github/workflows/push-results/action.yml diff --git a/.github/workflows/benchmark-dev.yml b/.github/workflows/benchmark-dev.yml index 2c6eae4b..210a38e5 100644 --- a/.github/workflows/benchmark-dev.yml +++ b/.github/workflows/benchmark-dev.yml @@ -4,6 +4,7 @@ on: push: branches: - master + paths-ignore: - "docs/**" - "papers/**" @@ -61,34 +62,49 @@ jobs: run: | python bench/bench.py | tee ${{ matrix.bench.script }}-output.txt - - name: Checkout Benchmark Results repository - uses: actions/checkout@v3 + - name: Push benchmark results + id: pushBenchmarkAttempt1 + continue-on-error: true + uses: ./.github/workflows/push-results with: repository: ${{ matrix.benchResultsRepo.name }} - ref: ${{ matrix.benchResultsRepo.branch }} + branch: ${{ matrix.benchResultsRepo.branch }} token: ${{ secrets.BENCH_GITHUB_TOKEN }} path: "./gh-pages" + bench_name: "${{ matrix.bench.title }} (Windows ${{matrix.arch}})" + bench_tool: "benchmarkluau" + bench_output_file_path: "./${{ matrix.bench.script }}-output.txt" + bench_external_data_json_path: "./gh-pages/dev/bench/data-${{ matrix.os }}.json" - - name: Store ${{ matrix.bench.title }} result - uses: Roblox/rhysd-github-action-benchmark@v-luau + - name: Push benchmark results (Attempt 2) + id: pushBenchmarkAttempt2 + continue-on-error: true + if: steps.pushBenchmarkAttempt1.outcome == 'failure' + uses: ./.github/workflows/push-results with: - name: ${{ matrix.bench.title }} (Windows ${{matrix.arch}}) - tool: "benchmarkluau" - output-file-path: ./${{ matrix.bench.script }}-output.txt - external-data-json-path: ./gh-pages/dev/bench/data-${{ matrix.os }}.json - github-token: ${{ secrets.GITHUB_TOKEN }} + repository: ${{ matrix.benchResultsRepo.name }} + branch: ${{ matrix.benchResultsRepo.branch }} + token: ${{ secrets.BENCH_GITHUB_TOKEN }} + path: "./gh-pages" + bench_name: "${{ matrix.bench.title }} (Windows ${{matrix.arch}})" + bench_tool: "benchmarkluau" + bench_output_file_path: "./${{ matrix.bench.script }}-output.txt" + bench_external_data_json_path: "./gh-pages/dev/bench/data-${{ matrix.os }}.json" - - name: Push benchmark results - if: github.event_name == 'push' - run: | - echo "Pushing benchmark results..." - cd gh-pages - git config user.name github-actions - git config user.email github@users.noreply.github.com - git add ./dev/bench/data-${{ matrix.os }}.json - git commit -m "Add benchmarks results for ${{ github.sha }}" - git push - cd .. + - name: Push benchmark results (Attempt 3) + id: pushBenchmarkAttempt3 + continue-on-error: true + if: steps.pushBenchmarkAttempt2.outcome == 'failure' + uses: ./.github/workflows/push-results + with: + repository: ${{ matrix.benchResultsRepo.name }} + branch: ${{ matrix.benchResultsRepo.branch }} + token: ${{ secrets.BENCH_GITHUB_TOKEN }} + path: "./gh-pages" + bench_name: "${{ matrix.bench.title }} (Windows ${{matrix.arch}})" + bench_tool: "benchmarkluau" + bench_output_file_path: "./${{ matrix.bench.script }}-output.txt" + bench_external_data_json_path: "./gh-pages/dev/bench/data-${{ matrix.os }}.json" unix: name: ${{matrix.os}} @@ -142,44 +158,94 @@ jobs: if: matrix.os == 'ubuntu-latest' run: sudo bash ./scripts/run-with-cachegrind.sh python ./bench/bench.py "${{ matrix.bench.cachegrindTitle }}" ${{ matrix.bench.cachegrindIterCount }} | tee -a ${{ matrix.bench.script }}-output.txt - - name: Checkout Benchmark Results repository - uses: actions/checkout@v3 + - name: Push benchmark results + id: pushBenchmarkAttempt1 + continue-on-error: true + uses: ./.github/workflows/push-results with: repository: ${{ matrix.benchResultsRepo.name }} - ref: ${{ matrix.benchResultsRepo.branch }} + branch: ${{ matrix.benchResultsRepo.branch }} token: ${{ secrets.BENCH_GITHUB_TOKEN }} path: "./gh-pages" + bench_name: ${{ matrix.bench.title }} + bench_tool: "benchmarkluau" + bench_output_file_path: "./${{ matrix.bench.script }}-output.txt" + bench_external_data_json_path: "./gh-pages/dev/bench/data-${{ matrix.os }}.json" - - name: Store ${{ matrix.bench.title }} result - uses: Roblox/rhysd-github-action-benchmark@v-luau + - name: Push benchmark results (Attempt 2) + id: pushBenchmarkAttempt2 + continue-on-error: true + if: steps.pushBenchmarkAttempt1.outcome == 'failure' + uses: ./.github/workflows/push-results with: - name: ${{ matrix.bench.title }} - tool: "benchmarkluau" - output-file-path: ./${{ matrix.bench.script }}-output.txt - external-data-json-path: ./gh-pages/dev/bench/data-${{ matrix.os }}.json - github-token: ${{ secrets.BENCH_GITHUB_TOKEN }} + repository: ${{ matrix.benchResultsRepo.name }} + branch: ${{ matrix.benchResultsRepo.branch }} + token: ${{ secrets.BENCH_GITHUB_TOKEN }} + path: "./gh-pages" + bench_name: ${{ matrix.bench.title }} + bench_tool: "benchmarkluau" + bench_output_file_path: "./${{ matrix.bench.script }}-output.txt" + bench_external_data_json_path: "./gh-pages/dev/bench/data-${{ matrix.os }}.json" - - name: Store ${{ matrix.bench.title }} result (CacheGrind) + - name: Push benchmark results (Attempt 3) + id: pushBenchmarkAttempt3 + continue-on-error: true + if: steps.pushBenchmarkAttempt2.outcome == 'failure' + uses: ./.github/workflows/push-results + with: + repository: ${{ matrix.benchResultsRepo.name }} + branch: ${{ matrix.benchResultsRepo.branch }} + token: ${{ secrets.BENCH_GITHUB_TOKEN }} + path: "./gh-pages" + bench_name: ${{ matrix.bench.title }} + bench_tool: "benchmarkluau" + bench_output_file_path: "./${{ matrix.bench.script }}-output.txt" + bench_external_data_json_path: "./gh-pages/dev/bench/data-${{ matrix.os }}.json" + + - name: Push Cachegrind benchmark results if: matrix.os == 'ubuntu-latest' - uses: Roblox/rhysd-github-action-benchmark@v-luau + id: pushBenchmarkCachegrindAttempt1 + continue-on-error: true + uses: ./.github/workflows/push-results with: - name: ${{ matrix.bench.title }} (CacheGrind) - tool: "roblox" - output-file-path: ./${{ matrix.bench.script }}-output.txt - external-data-json-path: ./gh-pages/dev/bench/data-${{ matrix.os }}.json - github-token: ${{ secrets.BENCH_GITHUB_TOKEN }} + repository: ${{ matrix.benchResultsRepo.name }} + branch: ${{ matrix.benchResultsRepo.branch }} + token: ${{ secrets.BENCH_GITHUB_TOKEN }} + path: "./gh-pages" + bench_name: ${{ matrix.bench.title }} (CacheGrind) + bench_tool: "roblox" + bench_output_file_path: "./${{ matrix.bench.script }}-output.txt" + bench_external_data_json_path: "./gh-pages/dev/bench/data-${{ matrix.os }}.json" - - name: Push benchmark results - if: github.event_name == 'push' - run: | - echo "Pushing benchmark results..." - cd gh-pages - git config user.name github-actions - git config user.email github@users.noreply.github.com - git add ./dev/bench/data-${{ matrix.os }}.json - git commit -m "Add benchmarks results for ${{ github.sha }}" - git push - cd .. + - name: Push Cachegrind benchmark results (Attempt 2) + if: matrix.os == 'ubuntu-latest' && steps.pushBenchmarkCachegrindAttempt1.outcome == 'failure' + id: pushBenchmarkCachegrindAttempt2 + continue-on-error: true + uses: ./.github/workflows/push-results + with: + repository: ${{ matrix.benchResultsRepo.name }} + branch: ${{ matrix.benchResultsRepo.branch }} + token: ${{ secrets.BENCH_GITHUB_TOKEN }} + path: "./gh-pages" + bench_name: ${{ matrix.bench.title }} (CacheGrind) + bench_tool: "roblox" + bench_output_file_path: "./${{ matrix.bench.script }}-output.txt" + bench_external_data_json_path: "./gh-pages/dev/bench/data-${{ matrix.os }}.json" + + - name: Push Cachegrind benchmark results (Attempt 3) + if: matrix.os == 'ubuntu-latest' && steps.pushBenchmarkCachegrindAttempt2.outcome == 'failure' + id: pushBenchmarkCachegrindAttempt3 + continue-on-error: true + uses: ./.github/workflows/push-results + with: + repository: ${{ matrix.benchResultsRepo.name }} + branch: ${{ matrix.benchResultsRepo.branch }} + token: ${{ secrets.BENCH_GITHUB_TOKEN }} + path: "./gh-pages" + bench_name: ${{ matrix.bench.title }} (CacheGrind) + bench_tool: "roblox" + bench_output_file_path: "./${{ matrix.bench.script }}-output.txt" + bench_external_data_json_path: "./gh-pages/dev/bench/data-${{ matrix.os }}.json" static-analysis: name: luau-analyze @@ -197,6 +263,7 @@ jobs: } benchResultsRepo: - { name: "luau-lang/benchmark-data", branch: "main" } + runs-on: ${{ matrix.os }} steps: - uses: actions/checkout@v3 @@ -228,43 +295,91 @@ jobs: - name: Run ${{ matrix.bench.title }} (Warm Cachegrind) run: sudo bash ./scripts/run-with-cachegrind.sh python ./bench/measure_time.py "${{ matrix.bench.cachegrindTitle}}" 1 ./build/release/luau-analyze bench/other/LuauPolyfillMap.lua | tee -a ${{ matrix.bench.script }}-output.txt - - name: Checkout Benchmark Results repository - uses: actions/checkout@v3 + - name: Push static analysis results + id: pushStaticAnalysisAttempt1 + continue-on-error: true + uses: ./.github/workflows/push-results with: repository: ${{ matrix.benchResultsRepo.name }} - ref: ${{ matrix.benchResultsRepo.branch }} + branch: ${{ matrix.benchResultsRepo.branch }} token: ${{ secrets.BENCH_GITHUB_TOKEN }} path: "./gh-pages" + bench_name: ${{ matrix.bench.title }} + bench_tool: "roblox" + bench_output_file_path: "./${{ matrix.bench.script }}-output.txt" + bench_external_data_json_path: "./gh-pages/dev/bench/data-${{ matrix.os }}.json" - - name: Store ${{ matrix.bench.title }} result - uses: Roblox/rhysd-github-action-benchmark@v-luau + - name: Push static analysis results (Attempt 2) + if: steps.pushStaticAnalysisAttempt1.outcome == 'failure' + id: pushStaticAnalysisAttempt2 + continue-on-error: true + uses: ./.github/workflows/push-results with: - name: ${{ matrix.bench.title }} - tool: "benchmarkluau" + repository: ${{ matrix.benchResultsRepo.name }} + branch: ${{ matrix.benchResultsRepo.branch }} + token: ${{ secrets.BENCH_GITHUB_TOKEN }} + path: "./gh-pages" + bench_name: ${{ matrix.bench.title }} + bench_tool: "roblox" + bench_output_file_path: "./${{ matrix.bench.script }}-output.txt" + bench_external_data_json_path: "./gh-pages/dev/bench/data-${{ matrix.os }}.json" - gh-pages-branch: "main" - output-file-path: ./${{ matrix.bench.script }}-output.txt - external-data-json-path: ./gh-pages/dev/bench/data-${{ matrix.os }}.json - github-token: ${{ secrets.BENCH_GITHUB_TOKEN }} - - - name: Store ${{ matrix.bench.title }} result (CacheGrind) - uses: Roblox/rhysd-github-action-benchmark@v-luau + - name: Push static analysis results (Attempt 3) + if: steps.pushStaticAnalysisAttempt2.outcome == 'failure' + id: pushStaticAnalysisAttempt3 + continue-on-error: true + uses: ./.github/workflows/push-results with: - name: ${{ matrix.bench.title }} - tool: "roblox" - gh-pages-branch: "main" - output-file-path: ./${{ matrix.bench.script }}-output.txt - external-data-json-path: ./gh-pages/dev/bench/data-${{ matrix.os }}.json - github-token: ${{ secrets.BENCH_GITHUB_TOKEN }} + repository: ${{ matrix.benchResultsRepo.name }} + branch: ${{ matrix.benchResultsRepo.branch }} + token: ${{ secrets.BENCH_GITHUB_TOKEN }} + path: "./gh-pages" + bench_name: ${{ matrix.bench.title }} + bench_tool: "roblox" + bench_output_file_path: "./${{ matrix.bench.script }}-output.txt" + bench_external_data_json_path: "./gh-pages/dev/bench/data-${{ matrix.os }}.json" - - name: Push benchmark results - if: github.event_name == 'push' - run: | - echo "Pushing benchmark results..." - cd gh-pages - git config user.name github-actions - git config user.email github@users.noreply.github.com - git add ./dev/bench/data-${{ matrix.os }}.json - git commit -m "Add benchmarks results for ${{ github.sha }}" - git push - cd .. + - name: Push static analysis Cachegrind results + if: matrix.os == 'ubuntu-latest' + id: pushStaticAnalysisCachegrindAttempt1 + continue-on-error: true + uses: ./.github/workflows/push-results + with: + repository: ${{ matrix.benchResultsRepo.name }} + branch: ${{ matrix.benchResultsRepo.branch }} + token: ${{ secrets.BENCH_GITHUB_TOKEN }} + path: "./gh-pages" + bench_name: ${{ matrix.bench.title }} + bench_tool: "roblox" + bench_output_file_path: "./${{ matrix.bench.script }}-output.txt" + bench_external_data_json_path: "./gh-pages/dev/bench/data-${{ matrix.os }}.json" + + - name: Push static analysis Cachegrind results (Attempt 2) + if: matrix.os == 'ubuntu-latest' && steps.pushStaticAnalysisCachegrindAttempt1.outcome == 'failure' + id: pushStaticAnalysisCachegrindAttempt2 + continue-on-error: true + uses: ./.github/workflows/push-results + with: + repository: ${{ matrix.benchResultsRepo.name }} + branch: ${{ matrix.benchResultsRepo.branch }} + token: ${{ secrets.BENCH_GITHUB_TOKEN }} + path: "./gh-pages" + bench_name: ${{ matrix.bench.title }} + bench_tool: "roblox" + bench_output_file_path: "./${{ matrix.bench.script }}-output.txt" + bench_external_data_json_path: "./gh-pages/dev/bench/data-${{ matrix.os }}.json" + + - name: Push static analysis Cachegrind results (Attempt 2) + if: matrix.os == 'ubuntu-latest' && steps.pushStaticAnalysisCachegrindAttempt2.outcome == 'failure' + id: pushStaticAnalysisCachegrindAttempt3 + continue-on-error: true + uses: ./.github/workflows/push-results + with: + repository: ${{ matrix.benchResultsRepo.name }} + branch: ${{ matrix.benchResultsRepo.branch }} + token: ${{ secrets.BENCH_GITHUB_TOKEN }} + path: "./gh-pages" + bench_name: ${{ matrix.bench.title }} + bench_tool: "roblox" + bench_output_file_path: "./${{ matrix.bench.script }}-output.txt" + bench_external_data_json_path: "./gh-pages/dev/bench/data-${{ matrix.os }}.json" diff --git a/.github/workflows/push-results/action.yml b/.github/workflows/push-results/action.yml new file mode 100644 index 00000000..b5ffebee --- /dev/null +++ b/.github/workflows/push-results/action.yml @@ -0,0 +1,63 @@ +name: Checkout & push results +description: Checkout a given repo and push results to GitHub +inputs: + repository: + required: true + type: string + description: The benchmark results repository to check out + branch: + required: true + type: string + description: The benchmark results repository's branch to check out + token: + required: true + type: string + description: The GitHub token to use for pushing results + path: + required: true + type: string + description: The path to check out the results repository to + bench_name: + required: true + type: string + bench_tool: + required: true + type: string + bench_output_file_path: + required: true + type: string + bench_external_data_json_path: + required: true + type: string + +runs: + using: "composite" + steps: + - name: Checkout repository + uses: actions/checkout@v3 + with: + repository: ${{ inputs.repository }} + ref: ${{ inputs.branch }} + token: ${{ inputs.token }} + path: ${{ inputs.path }} + + - name: Store results + uses: Roblox/rhysd-github-action-benchmark@v-luau + with: + name: ${{ inputs.bench_name }} + tool: ${{ inputs.bench_tool }} + gh-pages-branch: ${{ inputs.branch }} + output-file-path: ${{ inputs.bench_output_file_path }} + external-data-json-path: ${{ inputs.bench_external_data_json_path }} + + - name: Push benchmark results + shell: bash + run: | + echo "Pushing benchmark results..." + cd gh-pages + git config user.name github-actions + git config user.email github@users.noreply.github.com + git add *.json + git commit -m "Add benchmarks results for ${{ github.sha }}" + git push + cd .. From 6fbea7cc848a18cfaf0022dc661040521c590789 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Petri=20H=C3=A4kkinen?= Date: Thu, 15 Sep 2022 18:26:54 +0300 Subject: [PATCH 359/399] Add lua_rawsetfield (#671) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Luau currently has the following functions in the C API for dealing with tables without invoking metamethods: lua_rawgetfield lua_rawget lua_rawgeti lua_rawset lua_rawseti This change adds the missing function lua_rawsetfield for consistency and because it's more efficient to use it in place of plain lua_rawset which requires pushing the key and value separately. Co-authored-by: Petri Häkkinen --- VM/include/lua.h | 1 + VM/src/lapi.cpp | 13 +++++++++++++ tests/Conformance.test.cpp | 6 ++++-- 3 files changed, 18 insertions(+), 2 deletions(-) diff --git a/VM/include/lua.h b/VM/include/lua.h index 1f386355..0a3acb4f 100644 --- a/VM/include/lua.h +++ b/VM/include/lua.h @@ -210,6 +210,7 @@ LUA_API void lua_getfenv(lua_State* L, int idx); */ LUA_API void lua_settable(lua_State* L, int idx); LUA_API void lua_setfield(lua_State* L, int idx, const char* k); +LUA_API void lua_rawsetfield(lua_State* L, int idx, const char* k); LUA_API void lua_rawset(lua_State* L, int idx); LUA_API void lua_rawseti(lua_State* L, int idx, int n); LUA_API int lua_setmetatable(lua_State* L, int objindex); diff --git a/VM/src/lapi.cpp b/VM/src/lapi.cpp index 6a9c46da..7603a79e 100644 --- a/VM/src/lapi.cpp +++ b/VM/src/lapi.cpp @@ -847,6 +847,19 @@ void lua_setfield(lua_State* L, int idx, const char* k) return; } +void lua_rawsetfield(lua_State* L, int idx, const char* k) +{ + api_checknelems(L, 1); + StkId t = index2addr(L, idx); + api_check(L, ttistable(t)); + if (hvalue(t)->readonly) + luaG_runerror(L, "Attempt to modify a readonly table"); + setobj2t(L, luaH_setstr(L, hvalue(t), luaS_new(L, k)), L->top - 1); + luaC_barriert(L, hvalue(t), L->top - 1); + L->top--; + return; +} + void lua_rawset(lua_State* L, int idx) { api_checknelems(L, 2); diff --git a/tests/Conformance.test.cpp b/tests/Conformance.test.cpp index c6bdb4db..25129bff 100644 --- a/tests/Conformance.test.cpp +++ b/tests/Conformance.test.cpp @@ -746,6 +746,8 @@ TEST_CASE("ApiTables") lua_newtable(L); lua_pushnumber(L, 123.0); lua_setfield(L, -2, "key"); + lua_pushnumber(L, 456.0); + lua_rawsetfield(L, -2, "key2"); lua_pushstring(L, "test"); lua_rawseti(L, -2, 5); @@ -761,8 +763,8 @@ TEST_CASE("ApiTables") lua_pop(L, 1); // lua_rawgetfield - CHECK(lua_rawgetfield(L, -1, "key") == LUA_TNUMBER); - CHECK(lua_tonumber(L, -1) == 123.0); + CHECK(lua_rawgetfield(L, -1, "key2") == LUA_TNUMBER); + CHECK(lua_tonumber(L, -1) == 456.0); lua_pop(L, 1); // lua_rawget From 3d74a8f4d4fa9de03f1952c5a6e014a6dedbb4fc Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Thu, 15 Sep 2022 15:38:17 -0700 Subject: [PATCH 360/399] Sync to upstream/release/545 (#674) - Improve type error messages for argument count mismatch in certain cases - Fix type checking variadic returns when the type is incompatible which type checked in certain cases - Reduce size of upvalue objects by 8 bytes on 64-bit platforms - Reduce I$ footprint of interpreter by 1.5KB - Reduce GC pause during atomic stage for programs with a lot of threads - Remove support for bytecode v2 --- Analysis/include/Luau/ConstraintSolver.h | 3 + Analysis/include/Luau/Error.h | 2 + Analysis/include/Luau/ToString.h | 2 + Analysis/include/Luau/TypeInfer.h | 4 +- Analysis/include/Luau/TypePack.h | 3 + Analysis/include/Luau/TypeUtils.h | 2 +- Analysis/src/ConstraintGraphBuilder.cpp | 36 ++- Analysis/src/ConstraintSolver.cpp | 77 +++++-- Analysis/src/Error.cpp | 21 +- Analysis/src/ToString.cpp | 16 ++ Analysis/src/TypeChecker2.cpp | 20 +- Analysis/src/TypeInfer.cpp | 69 ++++-- Analysis/src/TypePack.cpp | 9 +- Analysis/src/TypeUtils.cpp | 6 +- Analysis/src/TypeVar.cpp | 4 +- Analysis/src/Unifier.cpp | 14 +- CodeGen/include/Luau/CodeAllocator.h | 2 +- CodeGen/include/Luau/CodeBlockUnwind.h | 17 ++ CodeGen/include/Luau/UnwindBuilder.h | 35 +++ CodeGen/include/Luau/UnwindBuilderDwarf2.h | 40 ++++ CodeGen/include/Luau/UnwindBuilderWin.h | 51 +++++ CodeGen/src/CodeAllocator.cpp | 25 +- CodeGen/src/CodeBlockUnwind.cpp | 123 ++++++++++ CodeGen/src/UnwindBuilderDwarf2.cpp | 253 +++++++++++++++++++++ CodeGen/src/UnwindBuilderWin.cpp | 120 ++++++++++ Common/include/Luau/Bytecode.h | 2 +- Sources.cmake | 11 + VM/src/lapi.cpp | 4 - VM/src/ldo.cpp | 16 +- VM/src/lfunc.cpp | 82 +------ VM/src/lfunc.h | 1 - VM/src/lgc.cpp | 219 +++++++----------- VM/src/lgc.h | 31 +-- VM/src/lobject.h | 2 - VM/src/lstate.cpp | 11 +- VM/src/lstate.h | 4 +- VM/src/lstring.cpp | 59 +---- VM/src/lvmexecute.cpp | 240 +------------------ tests/AstQuery.test.cpp | 3 +- tests/AstQueryDsl.cpp | 45 ++++ tests/AstQueryDsl.h | 83 +++++++ tests/CodeAllocator.test.cpp | 180 ++++++++++++++- tests/ConstraintGraphBuilder.test.cpp | 5 +- tests/ConstraintGraphBuilderFixture.cpp | 17 ++ tests/ConstraintGraphBuilderFixture.h | 27 +++ tests/ConstraintSolver.test.cpp | 8 +- tests/Fixture.cpp | 47 ---- tests/Fixture.h | 83 ------- tests/TypeInfer.builtins.test.cpp | 24 +- tests/TypeInfer.functions.test.cpp | 52 +++++ tests/TypeInfer.generics.test.cpp | 8 +- tests/TypeInfer.provisional.test.cpp | 2 + tests/TypeInfer.singletons.test.cpp | 9 + tests/TypeInfer.tables.test.cpp | 8 +- tests/TypeInfer.test.cpp | 17 ++ tests/TypeInfer.typePacks.cpp | 36 +++ tools/faillist.txt | 7 +- 57 files changed, 1499 insertions(+), 798 deletions(-) create mode 100644 CodeGen/include/Luau/CodeBlockUnwind.h create mode 100644 CodeGen/include/Luau/UnwindBuilder.h create mode 100644 CodeGen/include/Luau/UnwindBuilderDwarf2.h create mode 100644 CodeGen/include/Luau/UnwindBuilderWin.h create mode 100644 CodeGen/src/CodeBlockUnwind.cpp create mode 100644 CodeGen/src/UnwindBuilderDwarf2.cpp create mode 100644 CodeGen/src/UnwindBuilderWin.cpp create mode 100644 tests/AstQueryDsl.cpp create mode 100644 tests/AstQueryDsl.h create mode 100644 tests/ConstraintGraphBuilderFixture.cpp create mode 100644 tests/ConstraintGraphBuilderFixture.h diff --git a/Analysis/include/Luau/ConstraintSolver.h b/Analysis/include/Luau/ConstraintSolver.h index 059e97cb..fe6a025b 100644 --- a/Analysis/include/Luau/ConstraintSolver.h +++ b/Analysis/include/Luau/ConstraintSolver.h @@ -191,6 +191,9 @@ private: **/ void unblock_(BlockedConstraintId progressed); + TypeId errorRecoveryType() const; + TypePackId errorRecoveryTypePack() const; + ToStringOptions opts; }; diff --git a/Analysis/include/Luau/Error.h b/Analysis/include/Luau/Error.h index 4c81d33d..eab6a21d 100644 --- a/Analysis/include/Luau/Error.h +++ b/Analysis/include/Luau/Error.h @@ -95,9 +95,11 @@ struct CountMismatch Return, }; size_t expected; + std::optional maximum; size_t actual; Context context = Arg; bool isVariadic = false; + std::string function; bool operator==(const CountMismatch& rhs) const; }; diff --git a/Analysis/include/Luau/ToString.h b/Analysis/include/Luau/ToString.h index 61e07e9f..dd2d709b 100644 --- a/Analysis/include/Luau/ToString.h +++ b/Analysis/include/Luau/ToString.h @@ -92,6 +92,8 @@ inline std::string toString(const Constraint& c) return toString(c, ToStringOptions{}); } +std::string toString(const LValue& lvalue); + std::string toString(const TypeVar& tv, ToStringOptions& opts); std::string toString(const TypePackVar& tp, ToStringOptions& opts); diff --git a/Analysis/include/Luau/TypeInfer.h b/Analysis/include/Luau/TypeInfer.h index b0b3f3ac..bbb9bd6d 100644 --- a/Analysis/include/Luau/TypeInfer.h +++ b/Analysis/include/Luau/TypeInfer.h @@ -127,8 +127,8 @@ struct TypeChecker std::optional originalNameLoc, std::optional selfType, std::optional expectedType); void checkFunctionBody(const ScopePtr& scope, TypeId type, const AstExprFunction& function); - void checkArgumentList( - const ScopePtr& scope, Unifier& state, TypePackId paramPack, TypePackId argPack, const std::vector& argLocations); + void checkArgumentList(const ScopePtr& scope, const AstExpr& funName, Unifier& state, TypePackId paramPack, TypePackId argPack, + const std::vector& argLocations); WithPredicate checkExprPack(const ScopePtr& scope, const AstExpr& expr); diff --git a/Analysis/include/Luau/TypePack.h b/Analysis/include/Luau/TypePack.h index 8269230b..29688094 100644 --- a/Analysis/include/Luau/TypePack.h +++ b/Analysis/include/Luau/TypePack.h @@ -185,6 +185,9 @@ std::pair, std::optional> flatten(TypePackId tp, bool isVariadic(TypePackId tp); bool isVariadic(TypePackId tp, const TxnLog& log); +// Returns true if the TypePack is Generic or Variadic. Does not walk TypePacks!! +bool isVariadicTail(TypePackId tp, const TxnLog& log, bool includeHiddenVariadics = false); + bool containsNever(TypePackId tp); } // namespace Luau diff --git a/Analysis/include/Luau/TypeUtils.h b/Analysis/include/Luau/TypeUtils.h index 6890f881..efc1d881 100644 --- a/Analysis/include/Luau/TypeUtils.h +++ b/Analysis/include/Luau/TypeUtils.h @@ -23,6 +23,6 @@ std::optional getIndexTypeFromType(const ScopePtr& scope, ErrorVec& erro TypeId type, const std::string& prop, const Location& location, bool addErrors, InternalErrorReporter& handle); // Returns the minimum and maximum number of types the argument list can accept. -std::pair> getParameterExtents(const TxnLog* log, TypePackId tp); +std::pair> getParameterExtents(const TxnLog* log, TypePackId tp, bool includeHiddenVariadics = false); } // namespace Luau diff --git a/Analysis/src/ConstraintGraphBuilder.cpp b/Analysis/src/ConstraintGraphBuilder.cpp index e9c61e41..6a65ab92 100644 --- a/Analysis/src/ConstraintGraphBuilder.cpp +++ b/Analysis/src/ConstraintGraphBuilder.cpp @@ -729,6 +729,11 @@ TypePackId ConstraintGraphBuilder::checkPack(const ScopePtr& scope, AstExpr* exp if (AstExprCall* call = expr->as()) { + TypeId fnType = check(scope, call->func); + + const size_t constraintIndex = scope->constraints.size(); + const size_t scopeIndex = scopes.size(); + std::vector args; for (AstExpr* arg : call->args) @@ -738,7 +743,8 @@ TypePackId ConstraintGraphBuilder::checkPack(const ScopePtr& scope, AstExpr* exp // TODO self - TypeId fnType = check(scope, call->func); + const size_t constraintEndIndex = scope->constraints.size(); + const size_t scopeEndIndex = scopes.size(); astOriginalCallTypes[call->func] = fnType; @@ -753,7 +759,23 @@ TypePackId ConstraintGraphBuilder::checkPack(const ScopePtr& scope, AstExpr* exp scope->unqueuedConstraints.push_back( std::make_unique(NotNull{scope.get()}, call->func->location, SubtypeConstraint{inferredFnType, instantiatedType})); - NotNull sc(scope->unqueuedConstraints.back().get()); + NotNull sc(scope->unqueuedConstraints.back().get()); + + // We force constraints produced by checking function arguments to wait + // until after we have resolved the constraint on the function itself. + // This ensures, for instance, that we start inferring the contents of + // lambdas under the assumption that their arguments and return types + // will be compatible with the enclosing function call. + for (size_t ci = constraintIndex; ci < constraintEndIndex; ++ci) + scope->constraints[ci]->dependencies.push_back(sc); + + for (size_t si = scopeIndex; si < scopeEndIndex; ++si) + { + for (auto& c : scopes[si].second->constraints) + { + c->dependencies.push_back(sc); + } + } addConstraint(scope, call->func->location, FunctionCallConstraint{ @@ -1080,7 +1102,7 @@ ConstraintGraphBuilder::FunctionSignature ConstraintGraphBuilder::checkFunctionS signatureScope = bodyScope; } - std::optional varargPack; + TypePackId varargPack = nullptr; if (fn->vararg) { @@ -1096,6 +1118,14 @@ ConstraintGraphBuilder::FunctionSignature ConstraintGraphBuilder::checkFunctionS signatureScope->varargPack = varargPack; } + else + { + varargPack = arena->addTypePack(VariadicTypePack{singletonTypes->anyType, /*hidden*/ true}); + // We do not add to signatureScope->varargPack because ... is not valid + // in functions without an explicit ellipsis. + } + + LUAU_ASSERT(nullptr != varargPack); if (fn->returnAnnotation) { diff --git a/Analysis/src/ConstraintSolver.cpp b/Analysis/src/ConstraintSolver.cpp index f964a855..6fb57b15 100644 --- a/Analysis/src/ConstraintSolver.cpp +++ b/Analysis/src/ConstraintSolver.cpp @@ -240,6 +240,12 @@ void dump(ConstraintSolver* cs, ToStringOptions& opts) int blockCount = it == cs->blockedConstraints.end() ? 0 : int(it->second); printf("\t%d\t\t%s\n", blockCount, toString(*dep, opts).c_str()); } + + if (auto fcc = get(*c)) + { + for (NotNull inner : fcc->innerConstraints) + printf("\t\t\t%s\n", toString(*inner, opts).c_str()); + } } } @@ -522,7 +528,7 @@ bool ConstraintSolver::tryDispatch(const BinaryConstraint& c, NotNullty.emplace(singletonTypes->errorRecoveryType()); + asMutable(resultType)->ty.emplace(errorRecoveryType()); // reportError(constraint->location, CannotInferBinaryOperation{c.op, std::nullopt, CannotInferBinaryOperation::Operation}); return true; } @@ -574,10 +580,24 @@ bool ConstraintSolver::tryDispatch(const IterableConstraint& c, NotNullscope, singletonTypes, &iceReporter, singletonTypes->errorRecoveryType(), singletonTypes->errorRecoveryTypePack()}; + Anyification anyify{arena, constraint->scope, singletonTypes, &iceReporter, errorRecoveryType(), errorRecoveryTypePack()}; std::optional anyified = anyify.substitute(c.variables); LUAU_ASSERT(anyified); unify(*anyified, c.variables, constraint->scope); @@ -704,7 +724,7 @@ bool ConstraintSolver::tryDispatch(const TypeAliasExpansionConstraint& c, NotNul if (!tf.has_value()) { reportError(UnknownSymbol{petv->name.value, UnknownSymbol::Context::Type}, constraint->location); - bindResult(singletonTypes->errorRecoveryType()); + bindResult(errorRecoveryType()); return true; } @@ -763,7 +783,7 @@ bool ConstraintSolver::tryDispatch(const TypeAliasExpansionConstraint& c, NotNul if (itf.foundInfiniteType) { // TODO (CLI-56761): Report an error. - bindResult(singletonTypes->errorRecoveryType()); + bindResult(errorRecoveryType()); return true; } @@ -786,7 +806,7 @@ bool ConstraintSolver::tryDispatch(const TypeAliasExpansionConstraint& c, NotNul if (!maybeInstantiated.has_value()) { // TODO (CLI-56761): Report an error. - bindResult(singletonTypes->errorRecoveryType()); + bindResult(errorRecoveryType()); return true; } @@ -863,13 +883,21 @@ bool ConstraintSolver::tryDispatch(const FunctionCallConstraint& c, NotNulldcrMagicFunction(NotNull(this), result, c.astFragment); } - if (!usedMagic) + if (usedMagic) { - for (const auto& inner : c.innerConstraints) - { - unsolvedConstraints.push_back(inner); - } - + // There are constraints that are blocked on these constraints. If we + // are never going to even examine them, then we should not block + // anything else on them. + // + // TODO CLI-58842 +#if 0 + for (auto& c: c.innerConstraints) + unblock(c); +#endif + } + else + { + unsolvedConstraints.insert(end(unsolvedConstraints), begin(c.innerConstraints), end(c.innerConstraints)); asMutable(c.result)->ty.emplace(constraint->scope); } @@ -909,8 +937,7 @@ bool ConstraintSolver::tryDispatchIterableTable(TypeId iteratorTy, const Iterabl }; auto errorify = [&](auto ty) { - Anyification anyify{ - arena, constraint->scope, singletonTypes, &iceReporter, singletonTypes->errorRecoveryType(), singletonTypes->errorRecoveryTypePack()}; + Anyification anyify{arena, constraint->scope, singletonTypes, &iceReporter, errorRecoveryType(), errorRecoveryTypePack()}; std::optional errorified = anyify.substitute(ty); if (!errorified) reportError(CodeTooComplex{}, constraint->location); @@ -1119,7 +1146,7 @@ void ConstraintSolver::unify(TypeId subType, TypeId superType, NotNull sc if (!u.errors.empty()) { - TypeId errorType = singletonTypes->errorRecoveryType(); + TypeId errorType = errorRecoveryType(); u.tryUnify(subType, errorType); u.tryUnify(superType, errorType); } @@ -1160,7 +1187,7 @@ TypeId ConstraintSolver::resolveModule(const ModuleInfo& info, const Location& l if (info.name.empty()) { reportError(UnknownRequire{}, location); - return singletonTypes->errorRecoveryType(); + return errorRecoveryType(); } std::string humanReadableName = moduleResolver->getHumanReadableModuleName(info.name); @@ -1177,24 +1204,24 @@ TypeId ConstraintSolver::resolveModule(const ModuleInfo& info, const Location& l if (!moduleResolver->moduleExists(info.name) && !info.optional) reportError(UnknownRequire{humanReadableName}, location); - return singletonTypes->errorRecoveryType(); + return errorRecoveryType(); } if (module->type != SourceCode::Type::Module) { reportError(IllegalRequire{humanReadableName, "Module is not a ModuleScript. It cannot be required."}, location); - return singletonTypes->errorRecoveryType(); + return errorRecoveryType(); } TypePackId modulePack = module->getModuleScope()->returnType; if (get(modulePack)) - return singletonTypes->errorRecoveryType(); + return errorRecoveryType(); std::optional moduleType = first(modulePack); if (!moduleType) { reportError(IllegalRequire{humanReadableName, "Module does not return exactly 1 value. It cannot be required."}, location); - return singletonTypes->errorRecoveryType(); + return errorRecoveryType(); } return *moduleType; @@ -1212,4 +1239,14 @@ void ConstraintSolver::reportError(TypeError e) errors.back().moduleName = currentModuleName; } +TypeId ConstraintSolver::errorRecoveryType() const +{ + return singletonTypes->errorRecoveryType(); +} + +TypePackId ConstraintSolver::errorRecoveryTypePack() const +{ + return singletonTypes->errorRecoveryTypePack(); +} + } // namespace Luau diff --git a/Analysis/src/Error.cpp b/Analysis/src/Error.cpp index 93cb65b9..9ecdc82b 100644 --- a/Analysis/src/Error.cpp +++ b/Analysis/src/Error.cpp @@ -10,7 +10,8 @@ LUAU_FASTFLAGVARIABLE(LuauTypeMismatchModuleNameResolution, false) LUAU_FASTFLAGVARIABLE(LuauUseInternalCompilerErrorException, false) -static std::string wrongNumberOfArgsString(size_t expectedCount, size_t actualCount, const char* argPrefix = nullptr, bool isVariadic = false) +static std::string wrongNumberOfArgsString( + size_t expectedCount, std::optional maximumCount, size_t actualCount, const char* argPrefix = nullptr, bool isVariadic = false) { std::string s = "expects "; @@ -19,11 +20,14 @@ static std::string wrongNumberOfArgsString(size_t expectedCount, size_t actualCo s += std::to_string(expectedCount) + " "; + if (maximumCount && expectedCount != *maximumCount) + s += "to " + std::to_string(*maximumCount) + " "; + if (argPrefix) s += std::string(argPrefix) + " "; s += "argument"; - if (expectedCount != 1) + if ((maximumCount ? *maximumCount : expectedCount) != 1) s += "s"; s += ", but "; @@ -185,7 +189,12 @@ struct ErrorConverter return "Function only returns " + std::to_string(e.expected) + " value" + expectedS + ". " + std::to_string(e.actual) + " are required here"; case CountMismatch::Arg: - return "Argument count mismatch. Function " + wrongNumberOfArgsString(e.expected, e.actual, /*argPrefix*/ nullptr, e.isVariadic); + if (!e.function.empty()) + return "Argument count mismatch. Function '" + e.function + "' " + + wrongNumberOfArgsString(e.expected, e.maximum, e.actual, /*argPrefix*/ nullptr, e.isVariadic); + else + return "Argument count mismatch. Function " + + wrongNumberOfArgsString(e.expected, e.maximum, e.actual, /*argPrefix*/ nullptr, e.isVariadic); } LUAU_ASSERT(!"Unknown context"); @@ -247,10 +256,10 @@ struct ErrorConverter if (e.typeFun.typeParams.size() != e.actualParameters) return "Generic type '" + name + "' " + - wrongNumberOfArgsString(e.typeFun.typeParams.size(), e.actualParameters, "type", !e.typeFun.typePackParams.empty()); + wrongNumberOfArgsString(e.typeFun.typeParams.size(), std::nullopt, e.actualParameters, "type", !e.typeFun.typePackParams.empty()); return "Generic type '" + name + "' " + - wrongNumberOfArgsString(e.typeFun.typePackParams.size(), e.actualPackParameters, "type pack", /*isVariadic*/ false); + wrongNumberOfArgsString(e.typeFun.typePackParams.size(), std::nullopt, e.actualPackParameters, "type pack", /*isVariadic*/ false); } std::string operator()(const Luau::SyntaxError& e) const @@ -547,7 +556,7 @@ bool DuplicateTypeDefinition::operator==(const DuplicateTypeDefinition& rhs) con bool CountMismatch::operator==(const CountMismatch& rhs) const { - return expected == rhs.expected && actual == rhs.actual && context == rhs.context; + return expected == rhs.expected && maximum == rhs.maximum && actual == rhs.actual && context == rhs.context && function == rhs.function; } bool FunctionDoesNotTakeSelf::operator==(const FunctionDoesNotTakeSelf&) const diff --git a/Analysis/src/ToString.cpp b/Analysis/src/ToString.cpp index 62b7afd5..4ec2371f 100644 --- a/Analysis/src/ToString.cpp +++ b/Analysis/src/ToString.cpp @@ -1529,4 +1529,20 @@ std::string dump(const Constraint& c) return s; } +std::string toString(const LValue& lvalue) +{ + std::string s; + for (const LValue* current = &lvalue; current; current = baseof(*current)) + { + if (auto field = get(*current)) + s = "." + field->key + s; + else if (auto symbol = get(*current)) + s = toString(*symbol) + s; + else + LUAU_ASSERT(!"Unknown LValue"); + } + + return s; +} + } // namespace Luau diff --git a/Analysis/src/TypeChecker2.cpp b/Analysis/src/TypeChecker2.cpp index 88363b43..76b27acd 100644 --- a/Analysis/src/TypeChecker2.cpp +++ b/Analysis/src/TypeChecker2.cpp @@ -408,7 +408,7 @@ struct TypeChecker2 return; NotNull scope = stack.back(); - TypeArena tempArena; + TypeArena& arena = module->internalTypes; std::vector variableTypes; for (AstLocal* var : forInStatement->vars) @@ -424,10 +424,10 @@ struct TypeChecker2 for (size_t i = 0; i < forInStatement->values.size - 1; ++i) valueTypes.emplace_back(lookupType(forInStatement->values.data[i])); TypePackId iteratorTail = lookupPack(forInStatement->values.data[forInStatement->values.size - 1]); - TypePackId iteratorPack = tempArena.addTypePack(valueTypes, iteratorTail); + TypePackId iteratorPack = arena.addTypePack(valueTypes, iteratorTail); // ... and then expand it out to 3 values (if possible) - const std::vector iteratorTypes = flatten(tempArena, iteratorPack, 3); + const std::vector iteratorTypes = flatten(arena, iteratorPack, 3); if (iteratorTypes.empty()) { reportError(GenericError{"for..in loops require at least one value to iterate over. Got zero"}, getLocation(forInStatement->values)); @@ -456,7 +456,7 @@ struct TypeChecker2 reportError(GenericError{"for..in loops must be passed (next, [table[, state]])"}, getLocation(forInStatement->values)); // It is okay if there aren't enough iterators, but the iteratee must provide enough. - std::vector expectedVariableTypes = flatten(tempArena, nextFn->retTypes, variableTypes.size()); + std::vector expectedVariableTypes = flatten(arena, nextFn->retTypes, variableTypes.size()); if (expectedVariableTypes.size() < variableTypes.size()) reportError(GenericError{"next() does not return enough values"}, forInStatement->vars.data[0]->location); @@ -475,23 +475,23 @@ struct TypeChecker2 // If iteratorTypes is too short to be a valid call to nextFn, we have to report a count mismatch error. // If 2 is too short to be a valid call to nextFn, we have to report a count mismatch error. // If 2 is too long to be a valid call to nextFn, we have to report a count mismatch error. - auto [minCount, maxCount] = getParameterExtents(TxnLog::empty(), nextFn->argTypes); + auto [minCount, maxCount] = getParameterExtents(TxnLog::empty(), nextFn->argTypes, /*includeHiddenVariadics*/ true); if (minCount > 2) - reportError(CountMismatch{2, minCount, CountMismatch::Arg}, forInStatement->vars.data[0]->location); + reportError(CountMismatch{2, std::nullopt, minCount, CountMismatch::Arg}, forInStatement->vars.data[0]->location); if (maxCount && *maxCount < 2) - reportError(CountMismatch{2, *maxCount, CountMismatch::Arg}, forInStatement->vars.data[0]->location); + reportError(CountMismatch{2, std::nullopt, *maxCount, CountMismatch::Arg}, forInStatement->vars.data[0]->location); - const std::vector flattenedArgTypes = flatten(tempArena, nextFn->argTypes, 2); + const std::vector flattenedArgTypes = flatten(arena, nextFn->argTypes, 2); const auto [argTypes, argsTail] = Luau::flatten(nextFn->argTypes); size_t firstIterationArgCount = iteratorTypes.empty() ? 0 : iteratorTypes.size() - 1; size_t actualArgCount = expectedVariableTypes.size(); if (firstIterationArgCount < minCount) - reportError(CountMismatch{2, firstIterationArgCount, CountMismatch::Arg}, forInStatement->vars.data[0]->location); + reportError(CountMismatch{2, std::nullopt, firstIterationArgCount, CountMismatch::Arg}, forInStatement->vars.data[0]->location); else if (actualArgCount < minCount) - reportError(CountMismatch{2, actualArgCount, CountMismatch::Arg}, forInStatement->vars.data[0]->location); + reportError(CountMismatch{2, std::nullopt, actualArgCount, CountMismatch::Arg}, forInStatement->vars.data[0]->location); if (iteratorTypes.size() >= 2 && flattenedArgTypes.size() > 0) { diff --git a/Analysis/src/TypeInfer.cpp b/Analysis/src/TypeInfer.cpp index c1408196..b6f20ac8 100644 --- a/Analysis/src/TypeInfer.cpp +++ b/Analysis/src/TypeInfer.cpp @@ -32,13 +32,15 @@ LUAU_FASTINTVARIABLE(LuauCheckRecursionLimit, 300) LUAU_FASTINTVARIABLE(LuauVisitRecursionLimit, 500) LUAU_FASTFLAG(LuauKnowsTheDataModel3) LUAU_FASTFLAG(LuauAutocompleteDynamicLimits) +LUAU_FASTFLAGVARIABLE(LuauFunctionArgMismatchDetails, false) LUAU_FASTFLAGVARIABLE(LuauInplaceDemoteSkipAllBound, false) LUAU_FASTFLAGVARIABLE(LuauLowerBoundsCalculation, false) LUAU_FASTFLAGVARIABLE(DebugLuauFreezeDuringUnification, false) LUAU_FASTFLAGVARIABLE(LuauSelfCallAutocompleteFix3, false) LUAU_FASTFLAGVARIABLE(LuauReturnAnyInsteadOfICE, false) // Eventually removed as false. -LUAU_FASTFLAGVARIABLE(DebugLuauSharedSelf, false); +LUAU_FASTFLAGVARIABLE(DebugLuauSharedSelf, false) LUAU_FASTFLAGVARIABLE(LuauUnknownAndNeverType, false) +LUAU_FASTFLAGVARIABLE(LuauCallUnifyPackTails, false) LUAU_FASTFLAGVARIABLE(LuauCheckGenericHOFTypes, false) LUAU_FASTFLAGVARIABLE(LuauBinaryNeedsExpectedTypesToo, false) LUAU_FASTFLAGVARIABLE(LuauNeverTypesAndOperatorsInference, false) @@ -1346,7 +1348,7 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatForIn& forin) } Unifier state = mkUnifier(loopScope, firstValue->location); - checkArgumentList(loopScope, state, argPack, iterFunc->argTypes, /*argLocations*/ {}); + checkArgumentList(loopScope, *firstValue, state, argPack, iterFunc->argTypes, /*argLocations*/ {}); state.log.commit(); @@ -3666,8 +3668,8 @@ WithPredicate TypeChecker::checkExprPackHelper(const ScopePtr& scope } } -void TypeChecker::checkArgumentList( - const ScopePtr& scope, Unifier& state, TypePackId argPack, TypePackId paramPack, const std::vector& argLocations) +void TypeChecker::checkArgumentList(const ScopePtr& scope, const AstExpr& funName, Unifier& state, TypePackId argPack, TypePackId paramPack, + const std::vector& argLocations) { /* Important terminology refresher: * A function requires parameters. @@ -3679,14 +3681,27 @@ void TypeChecker::checkArgumentList( size_t paramIndex = 0; - auto reportCountMismatchError = [&state, &argLocations, paramPack, argPack]() { + auto reportCountMismatchError = [&state, &argLocations, paramPack, argPack, &funName]() { // For this case, we want the error span to cover every errant extra parameter Location location = state.location; if (!argLocations.empty()) location = {state.location.begin, argLocations.back().end}; - size_t minParams = getParameterExtents(&state.log, paramPack).first; - state.reportError(TypeError{location, CountMismatch{minParams, std::distance(begin(argPack), end(argPack))}}); + if (FFlag::LuauFunctionArgMismatchDetails) + { + std::string namePath; + if (std::optional lValue = tryGetLValue(funName)) + namePath = toString(*lValue); + + auto [minParams, optMaxParams] = getParameterExtents(&state.log, paramPack); + state.reportError(TypeError{location, + CountMismatch{minParams, optMaxParams, std::distance(begin(argPack), end(argPack)), CountMismatch::Context::Arg, false, namePath}}); + } + else + { + size_t minParams = getParameterExtents(&state.log, paramPack).first; + state.reportError(TypeError{location, CountMismatch{minParams, std::nullopt, std::distance(begin(argPack), end(argPack))}}); + } }; while (true) @@ -3698,11 +3713,8 @@ void TypeChecker::checkArgumentList( std::optional argTail = argIter.tail(); std::optional paramTail = paramIter.tail(); - // If we hit the end of both type packs simultaneously, then there are definitely no further type - // errors to report. All we need to do is tie up any free tails. - // - // If one side has a free tail and the other has none at all, we create an empty pack and bind the - // free tail to that. + // If we hit the end of both type packs simultaneously, we have to unify them. + // But if one side has a free tail and the other has none at all, we create an empty pack and bind the free tail to that. if (argTail) { @@ -3713,6 +3725,10 @@ void TypeChecker::checkArgumentList( else state.log.replace(*argTail, TypePackVar(TypePack{{}})); } + else if (FFlag::LuauCallUnifyPackTails && paramTail) + { + state.tryUnify(*argTail, *paramTail); + } } else if (paramTail) { @@ -3784,12 +3800,25 @@ void TypeChecker::checkArgumentList( } // ok else { - size_t minParams = getParameterExtents(&state.log, paramPack).first; + auto [minParams, optMaxParams] = getParameterExtents(&state.log, paramPack); std::optional tail = flatten(paramPack, state.log).second; bool isVariadic = tail && Luau::isVariadic(*tail); - state.reportError(TypeError{state.location, CountMismatch{minParams, paramIndex, CountMismatch::Context::Arg, isVariadic}}); + if (FFlag::LuauFunctionArgMismatchDetails) + { + std::string namePath; + if (std::optional lValue = tryGetLValue(funName)) + namePath = toString(*lValue); + + state.reportError(TypeError{ + state.location, CountMismatch{minParams, optMaxParams, paramIndex, CountMismatch::Context::Arg, isVariadic, namePath}}); + } + else + { + state.reportError( + TypeError{state.location, CountMismatch{minParams, std::nullopt, paramIndex, CountMismatch::Context::Arg, isVariadic}}); + } return; } ++paramIter; @@ -4185,13 +4214,13 @@ std::optional> TypeChecker::checkCallOverload(const Sc Unifier state = mkUnifier(scope, expr.location); // Unify return types - checkArgumentList(scope, state, retPack, ftv->retTypes, /*argLocations*/ {}); + checkArgumentList(scope, *expr.func, state, retPack, ftv->retTypes, /*argLocations*/ {}); if (!state.errors.empty()) { return {}; } - checkArgumentList(scope, state, argPack, ftv->argTypes, *argLocations); + checkArgumentList(scope, *expr.func, state, argPack, ftv->argTypes, *argLocations); if (!state.errors.empty()) { @@ -4245,7 +4274,7 @@ bool TypeChecker::handleSelfCallMismatch(const ScopePtr& scope, const AstExprCal TypePackId editedArgPack = addTypePack(TypePack{editedParamList}); Unifier editedState = mkUnifier(scope, expr.location); - checkArgumentList(scope, editedState, editedArgPack, ftv->argTypes, editedArgLocations); + checkArgumentList(scope, *expr.func, editedState, editedArgPack, ftv->argTypes, editedArgLocations); if (editedState.errors.empty()) { @@ -4276,7 +4305,7 @@ bool TypeChecker::handleSelfCallMismatch(const ScopePtr& scope, const AstExprCal Unifier editedState = mkUnifier(scope, expr.location); - checkArgumentList(scope, editedState, editedArgPack, ftv->argTypes, editedArgLocations); + checkArgumentList(scope, *expr.func, editedState, editedArgPack, ftv->argTypes, editedArgLocations); if (editedState.errors.empty()) { @@ -4345,8 +4374,8 @@ void TypeChecker::reportOverloadResolutionError(const ScopePtr& scope, const Ast // Unify return types if (const FunctionTypeVar* ftv = get(overload)) { - checkArgumentList(scope, state, retPack, ftv->retTypes, {}); - checkArgumentList(scope, state, argPack, ftv->argTypes, argLocations); + checkArgumentList(scope, *expr.func, state, retPack, ftv->retTypes, {}); + checkArgumentList(scope, *expr.func, state, argPack, ftv->argTypes, argLocations); } if (state.errors.empty()) diff --git a/Analysis/src/TypePack.cpp b/Analysis/src/TypePack.cpp index 2fa9413a..0fa4df60 100644 --- a/Analysis/src/TypePack.cpp +++ b/Analysis/src/TypePack.cpp @@ -357,10 +357,15 @@ bool isVariadic(TypePackId tp, const TxnLog& log) if (!tail) return false; - if (log.get(*tail)) + return isVariadicTail(*tail, log); +} + +bool isVariadicTail(TypePackId tp, const TxnLog& log, bool includeHiddenVariadics) +{ + if (log.get(tp)) return true; - if (auto vtp = log.get(*tail); vtp && !vtp->hidden) + if (auto vtp = log.get(tp); vtp && (includeHiddenVariadics || !vtp->hidden)) return true; return false; diff --git a/Analysis/src/TypeUtils.cpp b/Analysis/src/TypeUtils.cpp index a96820d6..6ea04ea9 100644 --- a/Analysis/src/TypeUtils.cpp +++ b/Analysis/src/TypeUtils.cpp @@ -6,6 +6,8 @@ #include "Luau/ToString.h" #include "Luau/TypeInfer.h" +LUAU_FASTFLAG(LuauFunctionArgMismatchDetails) + namespace Luau { @@ -193,7 +195,7 @@ std::optional getIndexTypeFromType(const ScopePtr& scope, ErrorVec& erro return std::nullopt; } -std::pair> getParameterExtents(const TxnLog* log, TypePackId tp) +std::pair> getParameterExtents(const TxnLog* log, TypePackId tp, bool includeHiddenVariadics) { size_t minCount = 0; size_t optionalCount = 0; @@ -216,7 +218,7 @@ std::pair> getParameterExtents(const TxnLog* log, ++it; } - if (it.tail()) + if (it.tail() && (!FFlag::LuauFunctionArgMismatchDetails || isVariadicTail(*it.tail(), *log, includeHiddenVariadics))) return {minCount, std::nullopt}; else return {minCount, minCount + optionalCount}; diff --git a/Analysis/src/TypeVar.cpp b/Analysis/src/TypeVar.cpp index 4f6603fb..3a820ea6 100644 --- a/Analysis/src/TypeVar.cpp +++ b/Analysis/src/TypeVar.cpp @@ -1133,14 +1133,14 @@ std::optional> magicFunctionFormat( size_t numExpectedParams = expected.size() + 1; // + 1 for the format string if (numExpectedParams != numActualParams && (!tail || numExpectedParams < numActualParams)) - typechecker.reportError(TypeError{expr.location, CountMismatch{numExpectedParams, numActualParams}}); + typechecker.reportError(TypeError{expr.location, CountMismatch{numExpectedParams, std::nullopt, numActualParams}}); } else { size_t actualParamSize = params.size() - paramOffset; if (expected.size() != actualParamSize && (!tail || expected.size() < actualParamSize)) - typechecker.reportError(TypeError{expr.location, CountMismatch{expected.size(), actualParamSize}}); + typechecker.reportError(TypeError{expr.location, CountMismatch{expected.size(), std::nullopt, actualParamSize}}); } return WithPredicate{arena.addTypePack({typechecker.stringType})}; } diff --git a/Analysis/src/Unifier.cpp b/Analysis/src/Unifier.cpp index fd678432..505e9e43 100644 --- a/Analysis/src/Unifier.cpp +++ b/Analysis/src/Unifier.cpp @@ -22,6 +22,7 @@ LUAU_FASTFLAG(LuauErrorRecoveryType); LUAU_FASTFLAG(LuauUnknownAndNeverType) LUAU_FASTFLAGVARIABLE(LuauScalarShapeSubtyping, false) LUAU_FASTFLAG(LuauClassTypeVarsInSubstitution) +LUAU_FASTFLAG(LuauCallUnifyPackTails) namespace Luau { @@ -1159,7 +1160,7 @@ void Unifier::tryUnify_(TypePackId subTp, TypePackId superTp, bool isFunctionCal size_t actualSize = size(subTp); if (ctx == CountMismatch::Result) std::swap(expectedSize, actualSize); - reportError(TypeError{location, CountMismatch{expectedSize, actualSize, ctx}}); + reportError(TypeError{location, CountMismatch{expectedSize, std::nullopt, actualSize, ctx}}); while (superIter.good()) { @@ -2118,6 +2119,15 @@ void Unifier::unifyLowerBound(TypePackId subTy, TypePackId superTy, TypeLevel de for (; superIter != superEndIter; ++superIter) tp->head.push_back(*superIter); } + else if (const VariadicTypePack* subVariadic = log.getMutable(subTailPack); + subVariadic && FFlag::LuauCallUnifyPackTails) + { + while (superIter != superEndIter) + { + tryUnify_(subVariadic->ty, *superIter); + ++superIter; + } + } } else { @@ -2125,7 +2135,7 @@ void Unifier::unifyLowerBound(TypePackId subTy, TypePackId superTy, TypeLevel de { if (!isOptional(*superIter)) { - errors.push_back(TypeError{location, CountMismatch{size(superTy), size(subTy), CountMismatch::Return}}); + errors.push_back(TypeError{location, CountMismatch{size(superTy), std::nullopt, size(subTy), CountMismatch::Return}}); return; } ++superIter; diff --git a/CodeGen/include/Luau/CodeAllocator.h b/CodeGen/include/Luau/CodeAllocator.h index c80b5c38..01e13121 100644 --- a/CodeGen/include/Luau/CodeAllocator.h +++ b/CodeGen/include/Luau/CodeAllocator.h @@ -24,7 +24,7 @@ struct CodeAllocator void* context = nullptr; // Called when new block is created to create and setup the unwinding information for all the code in the block - // Some platforms require this data to be placed inside the block itself, so we also return 'unwindDataSizeInBlock' + // If data is placed inside the block itself (some platforms require this), we also return 'unwindDataSizeInBlock' void* (*createBlockUnwindInfo)(void* context, uint8_t* block, size_t blockSize, size_t& unwindDataSizeInBlock) = nullptr; // Called to destroy unwinding information returned by 'createBlockUnwindInfo' diff --git a/CodeGen/include/Luau/CodeBlockUnwind.h b/CodeGen/include/Luau/CodeBlockUnwind.h new file mode 100644 index 00000000..ddae33a6 --- /dev/null +++ b/CodeGen/include/Luau/CodeBlockUnwind.h @@ -0,0 +1,17 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +#pragma once + +#include +#include + +namespace Luau +{ +namespace CodeGen +{ + +// context must be an UnwindBuilder +void* createBlockUnwindInfo(void* context, uint8_t* block, size_t blockSize, size_t& unwindDataSizeInBlock); +void destroyBlockUnwindInfo(void* context, void* unwindData); + +} // namespace CodeGen +} // namespace Luau diff --git a/CodeGen/include/Luau/UnwindBuilder.h b/CodeGen/include/Luau/UnwindBuilder.h new file mode 100644 index 00000000..c6f611b0 --- /dev/null +++ b/CodeGen/include/Luau/UnwindBuilder.h @@ -0,0 +1,35 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +#pragma once + +#include "Luau/RegisterX64.h" + +#include +#include + +namespace Luau +{ +namespace CodeGen +{ + +class UnwindBuilder +{ +public: + virtual ~UnwindBuilder() {} + + virtual void start() = 0; + + virtual void spill(int espOffset, RegisterX64 reg) = 0; + virtual void save(RegisterX64 reg) = 0; + virtual void allocStack(int size) = 0; + virtual void setupFrameReg(RegisterX64 reg, int espOffset) = 0; + + virtual void finish() = 0; + + virtual size_t getSize() const = 0; + + // This will place the unwinding data at the target address and might update values of some fields + virtual void finalize(char* target, void* funcAddress, size_t funcSize) const = 0; +}; + +} // namespace CodeGen +} // namespace Luau diff --git a/CodeGen/include/Luau/UnwindBuilderDwarf2.h b/CodeGen/include/Luau/UnwindBuilderDwarf2.h new file mode 100644 index 00000000..09c91d43 --- /dev/null +++ b/CodeGen/include/Luau/UnwindBuilderDwarf2.h @@ -0,0 +1,40 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +#pragma once + +#include "Luau/RegisterX64.h" +#include "UnwindBuilder.h" + +namespace Luau +{ +namespace CodeGen +{ + +class UnwindBuilderDwarf2 : public UnwindBuilder +{ +public: + void start() override; + + void spill(int espOffset, RegisterX64 reg) override; + void save(RegisterX64 reg) override; + void allocStack(int size) override; + void setupFrameReg(RegisterX64 reg, int espOffset) override; + + void finish() override; + + size_t getSize() const override; + + void finalize(char* target, void* funcAddress, size_t funcSize) const override; + +private: + static const unsigned kRawDataLimit = 128; + char rawData[kRawDataLimit]; + char* pos = rawData; + + uint32_t stackOffset = 0; + + // We will remember the FDE location to write some of the fields like entry length, function start and size later + char* fdeEntryStart = nullptr; +}; + +} // namespace CodeGen +} // namespace Luau diff --git a/CodeGen/include/Luau/UnwindBuilderWin.h b/CodeGen/include/Luau/UnwindBuilderWin.h new file mode 100644 index 00000000..801eb6e4 --- /dev/null +++ b/CodeGen/include/Luau/UnwindBuilderWin.h @@ -0,0 +1,51 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +#pragma once + +#include "Luau/RegisterX64.h" +#include "UnwindBuilder.h" + +#include + +namespace Luau +{ +namespace CodeGen +{ + +// This struct matches the layout of UNWIND_CODE from ehdata.h +struct UnwindCodeWin +{ + uint8_t offset; + uint8_t opcode : 4; + uint8_t opinfo : 4; +}; + +class UnwindBuilderWin : public UnwindBuilder +{ +public: + void start() override; + + void spill(int espOffset, RegisterX64 reg) override; + void save(RegisterX64 reg) override; + void allocStack(int size) override; + void setupFrameReg(RegisterX64 reg, int espOffset) override; + + void finish() override; + + size_t getSize() const override; + + void finalize(char* target, void* funcAddress, size_t funcSize) const override; + +private: + // Windows unwind codes are written in reverse, so we have to collect them all first + std::vector unwindCodes; + + uint8_t prologSize = 0; + RegisterX64 frameReg = rax; // rax means that frame register is not used + uint8_t frameRegOffset = 0; + uint32_t stackOffset = 0; + + size_t infoSize = 0; +}; + +} // namespace CodeGen +} // namespace Luau diff --git a/CodeGen/src/CodeAllocator.cpp b/CodeGen/src/CodeAllocator.cpp index f7432064..aacf40a3 100644 --- a/CodeGen/src/CodeAllocator.cpp +++ b/CodeGen/src/CodeAllocator.cpp @@ -6,8 +6,13 @@ #include #if defined(_WIN32) + +#ifndef WIN32_LEAN_AND_MEAN #define WIN32_LEAN_AND_MEAN +#endif +#ifndef NOMINMAX #define NOMINMAX +#endif #include const size_t kPageSize = 4096; @@ -135,17 +140,29 @@ bool CodeAllocator::allocate( if (codeSize) memcpy(blockPos + codeOffset, code, codeSize); - size_t pageSize = alignToPageSize(unwindInfoSize + totalSize); + size_t pageAlignedSize = alignToPageSize(unwindInfoSize + totalSize); - makePagesExecutable(blockPos, pageSize); + makePagesExecutable(blockPos, pageAlignedSize); flushInstructionCache(blockPos + codeOffset, codeSize); result = blockPos + unwindInfoSize; resultSize = totalSize; resultCodeStart = blockPos + codeOffset; - blockPos += pageSize; - LUAU_ASSERT((uintptr_t(blockPos) & (kPageSize - 1)) == 0); // Allocation ends on page boundary + // Ensure that future allocations from the block start from a page boundary. + // This is important since we use W^X, and writing to the previous page would require briefly removing + // executable bit from it, which may result in access violations if that code is being executed concurrently. + if (pageAlignedSize <= size_t(blockEnd - blockPos)) + { + blockPos += pageAlignedSize; + LUAU_ASSERT((uintptr_t(blockPos) & (kPageSize - 1)) == 0); + LUAU_ASSERT(blockPos <= blockEnd); + } + else + { + // Future allocations will need to allocate fresh blocks + blockPos = blockEnd; + } return true; } diff --git a/CodeGen/src/CodeBlockUnwind.cpp b/CodeGen/src/CodeBlockUnwind.cpp new file mode 100644 index 00000000..6191cee4 --- /dev/null +++ b/CodeGen/src/CodeBlockUnwind.cpp @@ -0,0 +1,123 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +#include "Luau/CodeBlockUnwind.h" + +#include "Luau/UnwindBuilder.h" + +#include + +#if defined(_WIN32) && defined(_M_X64) + +#ifndef WIN32_LEAN_AND_MEAN +#define WIN32_LEAN_AND_MEAN +#endif +#ifndef NOMINMAX +#define NOMINMAX +#endif +#include + +#elif !defined(_WIN32) + +// Defined in unwind.h which may not be easily discoverable on various platforms +extern "C" void __register_frame(const void*); +extern "C" void __deregister_frame(const void*); + +#endif + +#if defined(__APPLE__) +// On Mac, each FDE inside eh_frame section has to be handled separately +static void visitFdeEntries(char* pos, void (*cb)(const void*)) +{ + for (;;) + { + unsigned partLength; + memcpy(&partLength, pos, sizeof(partLength)); + + if (partLength == 0) // Zero-length section signals completion + break; + + unsigned partId; + memcpy(&partId, pos + 4, sizeof(partId)); + + if (partId != 0) // Skip CIE part + cb(pos); // CIE is found using an offset in FDE + + pos += partLength + 4; + } +} +#endif + +namespace Luau +{ +namespace CodeGen +{ + +void* createBlockUnwindInfo(void* context, uint8_t* block, size_t blockSize, size_t& unwindDataSizeInBlock) +{ +#if defined(_WIN32) && defined(_M_X64) + UnwindBuilder* unwind = (UnwindBuilder*)context; + + // All unwinding related data is placed together at the start of the block + size_t unwindSize = sizeof(RUNTIME_FUNCTION) + unwind->getSize(); + unwindSize = (unwindSize + 15) & ~15; // Align to 16 bytes + LUAU_ASSERT(blockSize >= unwindSize); + + RUNTIME_FUNCTION* runtimeFunc = (RUNTIME_FUNCTION*)block; + runtimeFunc->BeginAddress = DWORD(unwindSize); // Code will start after the unwind info + runtimeFunc->EndAddress = DWORD(blockSize); // Whole block is a part of a 'single function' + runtimeFunc->UnwindInfoAddress = DWORD(sizeof(RUNTIME_FUNCTION)); // Unwind info is placed at the start of the block + + char* unwindData = (char*)block + runtimeFunc->UnwindInfoAddress; + unwind->finalize(unwindData, block + unwindSize, blockSize - unwindSize); + + if (!RtlAddFunctionTable(runtimeFunc, 1, uintptr_t(block))) + { + LUAU_ASSERT(!"failed to allocate function table"); + return nullptr; + } + + unwindDataSizeInBlock = unwindSize; + return block; +#elif !defined(_WIN32) + UnwindBuilder* unwind = (UnwindBuilder*)context; + + // All unwinding related data is placed together at the start of the block + size_t unwindSize = unwind->getSize(); + unwindSize = (unwindSize + 15) & ~15; // Align to 16 bytes + LUAU_ASSERT(blockSize >= unwindSize); + + char* unwindData = (char*)block; + unwind->finalize(unwindData, block, blockSize); + +#if defined(__APPLE__) + visitFdeEntries(unwindData, __register_frame); +#else + __register_frame(unwindData); +#endif + + unwindDataSizeInBlock = unwindSize; + return block; +#endif + + return nullptr; +} + +void destroyBlockUnwindInfo(void* context, void* unwindData) +{ +#if defined(_WIN32) && defined(_M_X64) + RUNTIME_FUNCTION* runtimeFunc = (RUNTIME_FUNCTION*)unwindData; + + if (!RtlDeleteFunctionTable(runtimeFunc)) + LUAU_ASSERT(!"failed to deallocate function table"); +#elif !defined(_WIN32) + +#if defined(__APPLE__) + visitFdeEntries((char*)unwindData, __deregister_frame); +#else + __deregister_frame(unwindData); +#endif + +#endif +} + +} // namespace CodeGen +} // namespace Luau diff --git a/CodeGen/src/UnwindBuilderDwarf2.cpp b/CodeGen/src/UnwindBuilderDwarf2.cpp new file mode 100644 index 00000000..38e3e712 --- /dev/null +++ b/CodeGen/src/UnwindBuilderDwarf2.cpp @@ -0,0 +1,253 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +#include "Luau/UnwindBuilderDwarf2.h" + +#include + +// General information about Dwarf2 format can be found at: +// https://dwarfstd.org/doc/dwarf-2.0.0.pdf [DWARF Debugging Information Format] +// Main part for async exception unwinding is in section '6.4 Call Frame Information' + +// Information about System V ABI (AMD64) can be found at: +// https://refspecs.linuxbase.org/elf/x86_64-abi-0.99.pdf [System V Application Binary Interface (AMD64 Architecture Processor Supplement)] +// Interaction between Dwarf2 and System V ABI can be found in sections '3.6.2 DWARF Register Number Mapping' and '4.2.4 EH_FRAME sections' + +static char* writeu8(char* target, uint8_t value) +{ + memcpy(target, &value, sizeof(value)); + return target + sizeof(value); +} + +static char* writeu32(char* target, uint32_t value) +{ + memcpy(target, &value, sizeof(value)); + return target + sizeof(value); +} + +static char* writeu64(char* target, uint64_t value) +{ + memcpy(target, &value, sizeof(value)); + return target + sizeof(value); +} + +static char* writeuleb128(char* target, uint64_t value) +{ + do + { + char byte = value & 0x7f; + value >>= 7; + + if (value) + byte |= 0x80; + + *target++ = byte; + } while (value); + + return target; +} + +// Call frame instruction opcodes +#define DW_CFA_advance_loc 0x40 +#define DW_CFA_offset 0x80 +#define DW_CFA_restore 0xc0 +#define DW_CFA_nop 0x00 +#define DW_CFA_set_loc 0x01 +#define DW_CFA_advance_loc1 0x02 +#define DW_CFA_advance_loc2 0x03 +#define DW_CFA_advance_loc4 0x04 +#define DW_CFA_offset_extended 0x05 +#define DW_CFA_restore_extended 0x06 +#define DW_CFA_undefined 0x07 +#define DW_CFA_same_value 0x08 +#define DW_CFA_register 0x09 +#define DW_CFA_remember_state 0x0a +#define DW_CFA_restore_state 0x0b +#define DW_CFA_def_cfa 0x0c +#define DW_CFA_def_cfa_register 0x0d +#define DW_CFA_def_cfa_offset 0x0e +#define DW_CFA_def_cfa_expression 0x0f +#define DW_CFA_expression 0x10 +#define DW_CFA_offset_extended_sf 0x11 +#define DW_CFA_def_cfa_sf 0x12 +#define DW_CFA_def_cfa_offset_sf 0x13 +#define DW_CFA_val_offset 0x14 +#define DW_CFA_val_offset_sf 0x15 +#define DW_CFA_val_expression 0x16 +#define DW_CFA_lo_user 0x1c +#define DW_CFA_hi_user 0x3f + +// Register numbers for x64 +#define DW_REG_RAX 0 +#define DW_REG_RDX 1 +#define DW_REG_RCX 2 +#define DW_REG_RBX 3 +#define DW_REG_RSI 4 +#define DW_REG_RDI 5 +#define DW_REG_RBP 6 +#define DW_REG_RSP 7 +#define DW_REG_R8 8 +#define DW_REG_R9 9 +#define DW_REG_R10 10 +#define DW_REG_R11 11 +#define DW_REG_R12 12 +#define DW_REG_R13 13 +#define DW_REG_R14 14 +#define DW_REG_R15 15 +#define DW_REG_RA 16 + +const int regIndexToDwRegX64[16] = {DW_REG_RAX, DW_REG_RCX, DW_REG_RDX, DW_REG_RBX, DW_REG_RSP, DW_REG_RBP, DW_REG_RSI, DW_REG_RDI, DW_REG_R8, + DW_REG_R9, DW_REG_R10, DW_REG_R11, DW_REG_R12, DW_REG_R13, DW_REG_R14, DW_REG_R15}; + +const int kCodeAlignFactor = 1; +const int kDataAlignFactor = 8; +const int kDwarfAlign = 8; +const int kFdeInitialLocationOffset = 8; +const int kFdeAddressRangeOffset = 16; + +// Define canonical frame address expression as [reg + offset] +static char* defineCfaExpression(char* pos, int dwReg, uint32_t stackOffset) +{ + pos = writeu8(pos, DW_CFA_def_cfa); + pos = writeuleb128(pos, dwReg); + pos = writeuleb128(pos, stackOffset); + return pos; +} + +// Update offset value in canonical frame address expression +static char* defineCfaExpressionOffset(char* pos, uint32_t stackOffset) +{ + pos = writeu8(pos, DW_CFA_def_cfa_offset); + pos = writeuleb128(pos, stackOffset); + return pos; +} + +static char* defineSavedRegisterLocation(char* pos, int dwReg, uint32_t stackOffset) +{ + LUAU_ASSERT(stackOffset % kDataAlignFactor == 0 && "stack offsets have to be measured in kDataAlignFactor units"); + + if (dwReg <= 15) + { + pos = writeu8(pos, DW_CFA_offset + dwReg); + } + else + { + pos = writeu8(pos, DW_CFA_offset_extended); + pos = writeuleb128(pos, dwReg); + } + + pos = writeuleb128(pos, stackOffset / kDataAlignFactor); + return pos; +} + +static char* advanceLocation(char* pos, uint8_t offset) +{ + pos = writeu8(pos, DW_CFA_advance_loc1); + pos = writeu8(pos, offset); + return pos; +} + +static char* alignPosition(char* start, char* pos) +{ + size_t size = pos - start; + size_t pad = ((size + kDwarfAlign - 1) & ~(kDwarfAlign - 1)) - size; + + for (size_t i = 0; i < pad; i++) + pos = writeu8(pos, DW_CFA_nop); + + return pos; +} + +namespace Luau +{ +namespace CodeGen +{ + +void UnwindBuilderDwarf2::start() +{ + char* cieLength = pos; + pos = writeu32(pos, 0); // Length (to be filled later) + + pos = writeu32(pos, 0); // CIE id. 0 -- .eh_frame + pos = writeu8(pos, 1); // Version + + pos = writeu8(pos, 0); // CIE augmentation String "" + + pos = writeuleb128(pos, kCodeAlignFactor); // Code align factor + pos = writeuleb128(pos, -kDataAlignFactor & 0x7f); // Data align factor of (as signed LEB128) + pos = writeu8(pos, DW_REG_RA); // Return address register + + // Optional CIE augmentation section (not present) + + // Call frame instructions (common for all FDEs, of which we have 1) + stackOffset = 8; // Return address was pushed by calling the function + + pos = defineCfaExpression(pos, DW_REG_RSP, stackOffset); // Define CFA to be the rsp + 8 + pos = defineSavedRegisterLocation(pos, DW_REG_RA, 8); // Define return address register (RA) to be located at CFA - 8 + + pos = alignPosition(cieLength, pos); + writeu32(cieLength, unsigned(pos - cieLength - 4)); // Length field itself is excluded from length + + fdeEntryStart = pos; // Will be written at the end + pos = writeu32(pos, 0); // Length (to be filled later) + pos = writeu32(pos, unsigned(pos - rawData)); // CIE pointer + pos = writeu64(pos, 0); // Initial location (to be filled later) + pos = writeu64(pos, 0); // Address range (to be filled later) + + // Optional CIE augmentation section (not present) + + // Function call frame instructions to follow +} + +void UnwindBuilderDwarf2::spill(int espOffset, RegisterX64 reg) +{ + pos = advanceLocation(pos, 5); // REX.W mov [rsp + imm8], reg +} + +void UnwindBuilderDwarf2::save(RegisterX64 reg) +{ + stackOffset += 8; + pos = advanceLocation(pos, 2); // REX.W push reg + pos = defineCfaExpressionOffset(pos, stackOffset); + pos = defineSavedRegisterLocation(pos, regIndexToDwRegX64[reg.index], stackOffset); +} + +void UnwindBuilderDwarf2::allocStack(int size) +{ + stackOffset += size; + pos = advanceLocation(pos, 4); // REX.W sub rsp, imm8 + pos = defineCfaExpressionOffset(pos, stackOffset); +} + +void UnwindBuilderDwarf2::setupFrameReg(RegisterX64 reg, int espOffset) +{ + // Not required for unwinding +} + +void UnwindBuilderDwarf2::finish() +{ + LUAU_ASSERT(stackOffset % 16 == 0 && "stack has to be aligned to 16 bytes after prologue"); + + pos = alignPosition(fdeEntryStart, pos); + writeu32(fdeEntryStart, unsigned(pos - fdeEntryStart - 4)); // Length field itself is excluded from length + + // Terminate section + pos = writeu32(pos, 0); + + LUAU_ASSERT(getSize() <= kRawDataLimit); +} + +size_t UnwindBuilderDwarf2::getSize() const +{ + return size_t(pos - rawData); +} + +void UnwindBuilderDwarf2::finalize(char* target, void* funcAddress, size_t funcSize) const +{ + memcpy(target, rawData, getSize()); + + unsigned fdeEntryStartPos = unsigned(fdeEntryStart - rawData); + writeu64(target + fdeEntryStartPos + kFdeInitialLocationOffset, uintptr_t(funcAddress)); + writeu64(target + fdeEntryStartPos + kFdeAddressRangeOffset, funcSize); +} + +} // namespace CodeGen +} // namespace Luau diff --git a/CodeGen/src/UnwindBuilderWin.cpp b/CodeGen/src/UnwindBuilderWin.cpp new file mode 100644 index 00000000..5405fcf2 --- /dev/null +++ b/CodeGen/src/UnwindBuilderWin.cpp @@ -0,0 +1,120 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +#include "Luau/UnwindBuilderWin.h" + +#include + +// Information about the Windows x64 unwinding data setup can be found at: +// https://docs.microsoft.com/en-us/cpp/build/exception-handling-x64 [x64 exception handling] + +#define UWOP_PUSH_NONVOL 0 +#define UWOP_ALLOC_LARGE 1 +#define UWOP_ALLOC_SMALL 2 +#define UWOP_SET_FPREG 3 +#define UWOP_SAVE_NONVOL 4 +#define UWOP_SAVE_NONVOL_FAR 5 +#define UWOP_SAVE_XMM128 8 +#define UWOP_SAVE_XMM128_FAR 9 +#define UWOP_PUSH_MACHFRAME 10 + +namespace Luau +{ +namespace CodeGen +{ + +// This struct matches the layout of UNWIND_INFO from ehdata.h +struct UnwindInfoWin +{ + uint8_t version : 3; + uint8_t flags : 5; + uint8_t prologsize; + uint8_t unwindcodecount; + uint8_t framereg : 4; + uint8_t frameregoff : 4; +}; + +void UnwindBuilderWin::start() +{ + stackOffset = 8; // Return address was pushed by calling the function + + unwindCodes.reserve(16); +} + +void UnwindBuilderWin::spill(int espOffset, RegisterX64 reg) +{ + prologSize += 5; // REX.W mov [rsp + imm8], reg +} + +void UnwindBuilderWin::save(RegisterX64 reg) +{ + prologSize += 2; // REX.W push reg + stackOffset += 8; + unwindCodes.push_back({prologSize, UWOP_PUSH_NONVOL, reg.index}); +} + +void UnwindBuilderWin::allocStack(int size) +{ + LUAU_ASSERT(size >= 8 && size <= 128 && size % 8 == 0); + + prologSize += 4; // REX.W sub rsp, imm8 + stackOffset += size; + unwindCodes.push_back({prologSize, UWOP_ALLOC_SMALL, uint8_t((size - 8) / 8)}); +} + +void UnwindBuilderWin::setupFrameReg(RegisterX64 reg, int espOffset) +{ + LUAU_ASSERT(espOffset < 256 && espOffset % 16 == 0); + + frameReg = reg; + frameRegOffset = uint8_t(espOffset / 16); + + prologSize += 5; // REX.W lea rbp, [rsp + imm8] + unwindCodes.push_back({prologSize, UWOP_SET_FPREG, frameRegOffset}); +} + +void UnwindBuilderWin::finish() +{ + // Windows unwind code count is stored in uint8_t, so we can't have more + LUAU_ASSERT(unwindCodes.size() < 256); + + LUAU_ASSERT(stackOffset % 16 == 0 && "stack has to be aligned to 16 bytes after prologue"); + + size_t codeArraySize = unwindCodes.size(); + codeArraySize = (codeArraySize + 1) & ~1; // Size has to be even, but unwind code count doesn't have to + + infoSize = sizeof(UnwindInfoWin) + sizeof(UnwindCodeWin) * codeArraySize; +} + +size_t UnwindBuilderWin::getSize() const +{ + return infoSize; +} + +void UnwindBuilderWin::finalize(char* target, void* funcAddress, size_t funcSize) const +{ + UnwindInfoWin info; + info.version = 1; + info.flags = 0; // No EH + info.prologsize = prologSize; + info.unwindcodecount = uint8_t(unwindCodes.size()); + info.framereg = frameReg.index; + info.frameregoff = frameRegOffset; + + memcpy(target, &info, sizeof(info)); + target += sizeof(UnwindInfoWin); + + if (!unwindCodes.empty()) + { + // Copy unwind codes in reverse order + // Some unwind codes take up two array slots, but we don't use those atm + char* pos = target + sizeof(UnwindCodeWin) * (unwindCodes.size() - 1); + + for (size_t i = 0; i < unwindCodes.size(); i++) + { + memcpy(pos, &unwindCodes[i], sizeof(UnwindCodeWin)); + pos -= sizeof(UnwindCodeWin); + } + } +} + +} // namespace CodeGen +} // namespace Luau diff --git a/Common/include/Luau/Bytecode.h b/Common/include/Luau/Bytecode.h index 8b6ccddf..f8652434 100644 --- a/Common/include/Luau/Bytecode.h +++ b/Common/include/Luau/Bytecode.h @@ -411,7 +411,7 @@ enum LuauOpcode enum LuauBytecodeTag { // Bytecode version; runtime supports [MIN, MAX], compiler emits TARGET by default but may emit a higher version when flags are enabled - LBC_VERSION_MIN = 2, + LBC_VERSION_MIN = 3, LBC_VERSION_MAX = 3, LBC_VERSION_TARGET = 3, // Types of constant table entries diff --git a/Sources.cmake b/Sources.cmake index ff0b5a6e..580c8b3c 100644 --- a/Sources.cmake +++ b/Sources.cmake @@ -57,13 +57,20 @@ target_sources(Luau.Compiler PRIVATE target_sources(Luau.CodeGen PRIVATE CodeGen/include/Luau/AssemblyBuilderX64.h CodeGen/include/Luau/CodeAllocator.h + CodeGen/include/Luau/CodeBlockUnwind.h CodeGen/include/Luau/Condition.h CodeGen/include/Luau/Label.h CodeGen/include/Luau/OperandX64.h CodeGen/include/Luau/RegisterX64.h + CodeGen/include/Luau/UnwindBuilder.h + CodeGen/include/Luau/UnwindBuilderDwarf2.h + CodeGen/include/Luau/UnwindBuilderWin.h CodeGen/src/AssemblyBuilderX64.cpp CodeGen/src/CodeAllocator.cpp + CodeGen/src/CodeBlockUnwind.cpp + CodeGen/src/UnwindBuilderDwarf2.cpp + CodeGen/src/UnwindBuilderWin.cpp ) # Luau.Analysis Sources @@ -258,9 +265,13 @@ endif() if(TARGET Luau.UnitTest) # Luau.UnitTest Sources target_sources(Luau.UnitTest PRIVATE + tests/AstQueryDsl.h + tests/ConstraintGraphBuilderFixture.h tests/Fixture.h tests/IostreamOptional.h tests/ScopedFlags.h + tests/AstQueryDsl.cpp + tests/ConstraintGraphBuilderFixture.cpp tests/Fixture.cpp tests/AssemblyBuilderX64.test.cpp tests/AstJsonEncoder.test.cpp diff --git a/VM/src/lapi.cpp b/VM/src/lapi.cpp index 7603a79e..cbcaa3cc 100644 --- a/VM/src/lapi.cpp +++ b/VM/src/lapi.cpp @@ -34,8 +34,6 @@ * therefore call luaC_checkGC before luaC_threadbarrier to guarantee the object is pushed to a gray thread. */ -LUAU_FASTFLAG(LuauSimplerUpval) - const char* lua_ident = "$Lua: Lua 5.1.4 Copyright (C) 1994-2008 Lua.org, PUC-Rio $\n" "$Authors: R. Ierusalimschy, L. H. de Figueiredo & W. Celes $\n" "$URL: www.lua.org $\n"; @@ -1298,8 +1296,6 @@ const char* lua_setupvalue(lua_State* L, int funcindex, int n) L->top--; setobj(L, val, L->top); luaC_barrier(L, clvalue(fi), L->top); - if (!FFlag::LuauSimplerUpval) - luaC_upvalbarrier(L, cast_to(UpVal*, NULL), val); } return name; } diff --git a/VM/src/ldo.cpp b/VM/src/ldo.cpp index 51f63d32..ecd2fcbb 100644 --- a/VM/src/ldo.cpp +++ b/VM/src/ldo.cpp @@ -243,14 +243,14 @@ void luaD_call(lua_State* L, StkId func, int nResults) { // is a Lua function? L->ci->flags |= LUA_CALLINFO_RETURN; // luau_execute will stop after returning from the stack frame - int oldactive = luaC_threadactive(L); - l_setbit(L->stackstate, THREAD_ACTIVEBIT); + bool oldactive = L->isactive; + L->isactive = true; luaC_threadbarrier(L); luau_execute(L); // call it if (!oldactive) - resetbit(L->stackstate, THREAD_ACTIVEBIT); + L->isactive = false; } L->nCcalls--; @@ -427,7 +427,7 @@ static int resume_error(lua_State* L, const char* msg) static void resume_finish(lua_State* L, int status) { L->nCcalls = L->baseCcalls; - resetbit(L->stackstate, THREAD_ACTIVEBIT); + L->isactive = false; if (status != 0) { // error? @@ -452,7 +452,7 @@ int lua_resume(lua_State* L, lua_State* from, int nargs) return resume_error(L, "C stack overflow"); L->baseCcalls = ++L->nCcalls; - l_setbit(L->stackstate, THREAD_ACTIVEBIT); + L->isactive = true; luaC_threadbarrier(L); @@ -481,7 +481,7 @@ int lua_resumeerror(lua_State* L, lua_State* from) return resume_error(L, "C stack overflow"); L->baseCcalls = ++L->nCcalls; - l_setbit(L->stackstate, THREAD_ACTIVEBIT); + L->isactive = true; luaC_threadbarrier(L); @@ -546,7 +546,7 @@ int luaD_pcall(lua_State* L, Pfunc func, void* u, ptrdiff_t old_top, ptrdiff_t e { unsigned short oldnCcalls = L->nCcalls; ptrdiff_t old_ci = saveci(L, L->ci); - int oldactive = luaC_threadactive(L); + bool oldactive = L->isactive; int status = luaD_rawrunprotected(L, func, u); if (status != 0) { @@ -560,7 +560,7 @@ int luaD_pcall(lua_State* L, Pfunc func, void* u, ptrdiff_t old_top, ptrdiff_t e // since the call failed with an error, we might have to reset the 'active' thread state if (!oldactive) - resetbit(L->stackstate, THREAD_ACTIVEBIT); + L->isactive = false; // Restore nCcalls before calling the debugprotectederror callback which may rely on the proper value to have been restored. L->nCcalls = oldnCcalls; diff --git a/VM/src/lfunc.cpp b/VM/src/lfunc.cpp index 8c78083b..3c1869b5 100644 --- a/VM/src/lfunc.cpp +++ b/VM/src/lfunc.cpp @@ -6,9 +6,6 @@ #include "lmem.h" #include "lgc.h" -LUAU_FASTFLAG(LuauSimplerUpval) -LUAU_FASTFLAG(LuauNoSleepBit) - Proto* luaF_newproto(lua_State* L) { Proto* f = luaM_newgco(L, Proto, sizeof(Proto), L->activememcat); @@ -74,21 +71,16 @@ UpVal* luaF_findupval(lua_State* L, StkId level) UpVal* p; while (*pp != NULL && (p = *pp)->v >= level) { - LUAU_ASSERT(!FFlag::LuauSimplerUpval || !isdead(g, obj2gco(p))); + LUAU_ASSERT(!isdead(g, obj2gco(p))); LUAU_ASSERT(upisopen(p)); if (p->v == level) - { // found a corresponding upvalue? - if (!FFlag::LuauSimplerUpval && isdead(g, obj2gco(p))) // is it dead? - changewhite(obj2gco(p)); // resurrect it return p; - } pp = &p->u.open.threadnext; } - LUAU_ASSERT(luaC_threadactive(L)); - LUAU_ASSERT(!luaC_threadsleeping(L)); - LUAU_ASSERT(!FFlag::LuauNoSleepBit || !isblack(obj2gco(L))); // we don't use luaC_threadbarrier because active threads never turn black + LUAU_ASSERT(L->isactive); + LUAU_ASSERT(!isblack(obj2gco(L))); // we don't use luaC_threadbarrier because active threads never turn black UpVal* uv = luaM_newgco(L, UpVal, sizeof(UpVal), L->activememcat); // not found: create a new one luaC_init(L, uv, LUA_TUPVAL); @@ -96,22 +88,8 @@ UpVal* luaF_findupval(lua_State* L, StkId level) uv->v = level; // current value lives in the stack // chain the upvalue in the threads open upvalue list at the proper position - if (FFlag::LuauSimplerUpval) - { - uv->u.open.threadnext = *pp; - *pp = uv; - } - else - { - UpVal* next = *pp; - uv->u.open.threadnext = next; - - uv->u.open.threadprev = pp; - if (next) - next->u.open.threadprev = &uv->u.open.threadnext; - - *pp = uv; - } + uv->u.open.threadnext = *pp; + *pp = uv; // double link the upvalue in the global open upvalue list uv->u.open.prev = &g->uvhead; @@ -123,26 +101,8 @@ UpVal* luaF_findupval(lua_State* L, StkId level) return uv; } -void luaF_unlinkupval(UpVal* uv) -{ - LUAU_ASSERT(!FFlag::LuauSimplerUpval); - - // unlink upvalue from the global open upvalue list - LUAU_ASSERT(uv->u.open.next->u.open.prev == uv && uv->u.open.prev->u.open.next == uv); - uv->u.open.next->u.open.prev = uv->u.open.prev; - uv->u.open.prev->u.open.next = uv->u.open.next; - - // unlink upvalue from the thread open upvalue list - *uv->u.open.threadprev = uv->u.open.threadnext; - - if (UpVal* next = uv->u.open.threadnext) - next->u.open.threadprev = uv->u.open.threadprev; -} - void luaF_freeupval(lua_State* L, UpVal* uv, lua_Page* page) { - if (!FFlag::LuauSimplerUpval && uv->v != &uv->u.value) // is it open? - luaF_unlinkupval(uv); // remove from open list luaM_freegco(L, uv, sizeof(UpVal), uv->memcat, page); // free upvalue } @@ -154,41 +114,17 @@ void luaF_close(lua_State* L, StkId level) { GCObject* o = obj2gco(uv); LUAU_ASSERT(!isblack(o) && upisopen(uv)); + LUAU_ASSERT(!isdead(g, o)); - if (FFlag::LuauSimplerUpval) - { - LUAU_ASSERT(!isdead(g, o)); + // unlink value *before* closing it since value storage overlaps + L->openupval = uv->u.open.threadnext; - // unlink value *before* closing it since value storage overlaps - L->openupval = uv->u.open.threadnext; - - luaF_closeupval(L, uv, /* dead= */ false); - } - else - { - // by removing the upvalue from global/thread open upvalue lists, L->openupval will be pointing to the next upvalue - luaF_unlinkupval(uv); - - if (isdead(g, o)) - { - // close the upvalue without copying the dead data so that luaF_freeupval will not unlink again - uv->v = &uv->u.value; - } - else - { - setobj(L, &uv->u.value, uv->v); - uv->v = &uv->u.value; - // GC state of a new closed upvalue has to be initialized - luaC_upvalclosed(L, uv); - } - } + luaF_closeupval(L, uv, /* dead= */ false); } } void luaF_closeupval(lua_State* L, UpVal* uv, bool dead) { - LUAU_ASSERT(FFlag::LuauSimplerUpval); - // unlink value from all lists *before* closing it since value storage overlaps LUAU_ASSERT(uv->u.open.next->u.open.prev == uv && uv->u.open.prev->u.open.next == uv); uv->u.open.next->u.open.prev = uv->u.open.prev; diff --git a/VM/src/lfunc.h b/VM/src/lfunc.h index 899d040b..679836e7 100644 --- a/VM/src/lfunc.h +++ b/VM/src/lfunc.h @@ -15,7 +15,6 @@ LUAI_FUNC void luaF_close(lua_State* L, StkId level); LUAI_FUNC void luaF_closeupval(lua_State* L, UpVal* uv, bool dead); LUAI_FUNC void luaF_freeproto(lua_State* L, Proto* f, struct lua_Page* page); LUAI_FUNC void luaF_freeclosure(lua_State* L, Closure* c, struct lua_Page* page); -LUAI_FUNC void luaF_unlinkupval(UpVal* uv); LUAI_FUNC void luaF_freeupval(lua_State* L, UpVal* uv, struct lua_Page* page); LUAI_FUNC const LocVar* luaF_getlocal(const Proto* func, int local_number, int pc); LUAI_FUNC const LocVar* luaF_findlocal(const Proto* func, int local_reg, int pc); diff --git a/VM/src/lgc.cpp b/VM/src/lgc.cpp index fb610e13..c2a672e6 100644 --- a/VM/src/lgc.cpp +++ b/VM/src/lgc.cpp @@ -13,6 +13,8 @@ #include +LUAU_FASTFLAGVARIABLE(LuauBetterThreadMark, false) + /* * Luau uses an incremental non-generational non-moving mark&sweep garbage collector. * @@ -108,21 +110,14 @@ * API calls that can write to thread stacks outside of execution (which implies active) uses a thread barrier that checks if the thread is * black, and if it is it marks it as gray and puts it on a gray list to be rescanned during atomic phase. * - * NOTE: The above is only true when LuauNoSleepBit is enabled. - * * Upvalues are special objects that can be closed, in which case they contain the value (acting as a reference cell) and can be dealt * with using the regular algorithm, or open, in which case they refer to a stack slot in some other thread. These are difficult to deal * with because the stack writes are not monitored. Because of this open upvalues are treated in a somewhat special way: they are never marked * as black (doing so would violate the GC invariant), and they are kept in a special global list (global_State::uvhead) which is traversed * during atomic phase. This is needed because an open upvalue might point to a stack location in a dead thread that never marked the stack * slot - upvalues like this are identified since they don't have `markedopen` bit set during thread traversal and closed in `clearupvals`. - * - * NOTE: The above is only true when LuauSimplerUpval is enabled. */ -LUAU_FASTFLAGVARIABLE(LuauSimplerUpval, false) -LUAU_FASTFLAGVARIABLE(LuauNoSleepBit, false) -LUAU_FASTFLAGVARIABLE(LuauEagerShrink, false) LUAU_FASTFLAGVARIABLE(LuauFasterSweep, false) #define GC_SWEEPPAGESTEPCOST 16 @@ -408,14 +403,11 @@ static void traversestack(global_State* g, lua_State* l) stringmark(l->namecall); for (StkId o = l->stack; o < l->top; o++) markvalue(g, o); - if (FFlag::LuauSimplerUpval) + for (UpVal* uv = l->openupval; uv; uv = uv->u.open.threadnext) { - for (UpVal* uv = l->openupval; uv; uv = uv->u.open.threadnext) - { - LUAU_ASSERT(upisopen(uv)); - uv->markedopen = 1; - markobject(g, uv); - } + LUAU_ASSERT(upisopen(uv)); + uv->markedopen = 1; + markobject(g, uv); } } @@ -426,8 +418,29 @@ static void clearstack(lua_State* l) setnilvalue(o); } -// TODO: pull function definition here when FFlag::LuauEagerShrink is removed -static void shrinkstack(lua_State* L); +static void shrinkstack(lua_State* L) +{ + // compute used stack - note that we can't use th->top if we're in the middle of vararg call + StkId lim = L->top; + for (CallInfo* ci = L->base_ci; ci <= L->ci; ci++) + { + LUAU_ASSERT(ci->top <= L->stack_last); + if (lim < ci->top) + lim = ci->top; + } + + // shrink stack and callinfo arrays if we aren't using most of the space + int ci_used = cast_int(L->ci - L->base_ci); // number of `ci' in use + int s_used = cast_int(lim - L->stack); // part of stack in use + if (L->size_ci > LUAI_MAXCALLS) // handling overflow? + return; // do not touch the stacks + if (3 * ci_used < L->size_ci && 2 * BASIC_CI_SIZE < L->size_ci) + luaD_reallocCI(L, L->size_ci / 2); // still big enough... + condhardstacktests(luaD_reallocCI(L, ci_used + 1)); + if (3 * s_used < L->stacksize && 2 * (BASIC_STACK_SIZE + EXTRA_STACK) < L->stacksize) + luaD_reallocstack(L, L->stacksize / 2); // still big enough... + condhardstacktests(luaD_reallocstack(L, s_used)); +} /* ** traverse one gray object, turning it to black. @@ -460,37 +473,56 @@ static size_t propagatemark(global_State* g) lua_State* th = gco2th(o); g->gray = th->gclist; - LUAU_ASSERT(!luaC_threadsleeping(th)); + bool active = th->isactive || th == th->global->mainthread; - // threads that are executing and the main thread remain gray - bool active = luaC_threadactive(th) || th == th->global->mainthread; - - // TODO: Refactor this logic after LuauNoSleepBit is removed - if (!active && g->gcstate == GCSpropagate) + if (FFlag::LuauBetterThreadMark) { traversestack(g, th); - clearstack(th); - if (!FFlag::LuauNoSleepBit) - l_setbit(th->stackstate, THREAD_SLEEPINGBIT); + // active threads will need to be rescanned later to mark new stack writes so we mark them gray again + if (active) + { + th->gclist = g->grayagain; + g->grayagain = o; + + black2gray(o); + } + + // the stack needs to be cleared after the last modification of the thread state before sweep begins + // if the thread is inactive, we might not see the thread in this cycle so we must clear it now + if (!active || g->gcstate == GCSatomic) + clearstack(th); + + // we could shrink stack at any time but we opt to do it during initial mark to do that just once per cycle + if (g->gcstate == GCSpropagate) + shrinkstack(th); } else { - th->gclist = g->grayagain; - g->grayagain = o; - - black2gray(o); - - traversestack(g, th); - - // final traversal? - if (g->gcstate == GCSatomic) + // TODO: Refactor this logic! + if (!active && g->gcstate == GCSpropagate) + { + traversestack(g, th); clearstack(th); - } + } + else + { + th->gclist = g->grayagain; + g->grayagain = o; - // we could shrink stack at any time but we opt to skip it during atomic since it's redundant to do that more than once per cycle - if (FFlag::LuauEagerShrink && g->gcstate != GCSatomic) - shrinkstack(th); + black2gray(o); + + traversestack(g, th); + + // final traversal? + if (g->gcstate == GCSatomic) + clearstack(th); + } + + // we could shrink stack at any time but we opt to skip it during atomic since it's redundant to do that more than once per cycle + if (g->gcstate != GCSatomic) + shrinkstack(th); + } return sizeof(lua_State) + sizeof(TValue) * th->stacksize + sizeof(CallInfo) * th->size_ci; } @@ -593,30 +625,6 @@ static size_t cleartable(lua_State* L, GCObject* l) return work; } -static void shrinkstack(lua_State* L) -{ - // compute used stack - note that we can't use th->top if we're in the middle of vararg call - StkId lim = L->top; - for (CallInfo* ci = L->base_ci; ci <= L->ci; ci++) - { - LUAU_ASSERT(ci->top <= L->stack_last); - if (lim < ci->top) - lim = ci->top; - } - - // shrink stack and callinfo arrays if we aren't using most of the space - int ci_used = cast_int(L->ci - L->base_ci); // number of `ci' in use - int s_used = cast_int(lim - L->stack); // part of stack in use - if (L->size_ci > LUAI_MAXCALLS) // handling overflow? - return; // do not touch the stacks - if (3 * ci_used < L->size_ci && 2 * BASIC_CI_SIZE < L->size_ci) - luaD_reallocCI(L, L->size_ci / 2); // still big enough... - condhardstacktests(luaD_reallocCI(L, ci_used + 1)); - if (3 * s_used < L->stacksize && 2 * (BASIC_STACK_SIZE + EXTRA_STACK) < L->stacksize) - luaD_reallocstack(L, L->stacksize / 2); // still big enough... - condhardstacktests(luaD_reallocstack(L, s_used)); -} - static void freeobj(lua_State* L, GCObject* o, lua_Page* page) { switch (o->gch.tt) @@ -669,21 +677,6 @@ static void shrinkbuffersfull(lua_State* L) static bool deletegco(void* context, lua_Page* page, GCObject* gco) { - // we are in the process of deleting everything - // threads with open upvalues will attempt to close them all on removal - // but those upvalues might point to stack values that were already deleted - if (!FFlag::LuauSimplerUpval && gco->gch.tt == LUA_TTHREAD) - { - lua_State* th = gco2th(gco); - - while (UpVal* uv = th->openupval) - { - luaF_unlinkupval(uv); - // close the upvalue without copying the dead data so that luaF_freeupval will not unlink again - uv->v = &uv->u.value; - } - } - lua_State* L = (lua_State*)context; freeobj(L, gco, page); return true; @@ -701,7 +694,6 @@ void luaC_freeall(lua_State* L) LUAU_ASSERT(g->strt.hash[i] == NULL); LUAU_ASSERT(L->global->strt.nuse == 0); - LUAU_ASSERT(g->strbufgc == NULL); } static void markmt(global_State* g) @@ -829,15 +821,12 @@ static size_t atomic(lua_State* L) g->gcmetrics.currcycle.atomictimeclear += recordGcDeltaTime(currts); #endif - if (FFlag::LuauSimplerUpval) - { - // close orphaned live upvalues of dead threads and clear dead upvalues - work += clearupvals(L); + // close orphaned live upvalues of dead threads and clear dead upvalues + work += clearupvals(L); #ifdef LUAI_GCMETRICS - g->gcmetrics.currcycle.atomictimeupval += recordGcDeltaTime(currts); + g->gcmetrics.currcycle.atomictimeupval += recordGcDeltaTime(currts); #endif - } // flip current white g->currentwhite = cast_byte(otherwhite(g)); @@ -857,20 +846,6 @@ static bool sweepgco(lua_State* L, lua_Page* page, GCObject* gco) int alive = (gco->gch.marked ^ WHITEBITS) & deadmask; - if (gco->gch.tt == LUA_TTHREAD) - { - lua_State* th = gco2th(gco); - - if (alive) - { - if (!FFlag::LuauNoSleepBit) - resetbit(th->stackstate, THREAD_SLEEPINGBIT); - - if (!FFlag::LuauEagerShrink) - shrinkstack(th); - } - } - if (alive) { LUAU_ASSERT(!isdead(g, gco)); @@ -896,8 +871,6 @@ static int sweepgcopage(lua_State* L, lua_Page* page) if (FFlag::LuauFasterSweep) { - LUAU_ASSERT(FFlag::LuauNoSleepBit && FFlag::LuauEagerShrink); - global_State* g = L->global; int deadmask = otherwhite(g); @@ -1183,7 +1156,7 @@ void luaC_fullgc(lua_State* L) startGcCycleMetrics(g); #endif - if (FFlag::LuauSimplerUpval ? keepinvariant(g) : g->gcstate <= GCSatomic) + if (keepinvariant(g)) { // reset sweep marks to sweep all elements (returning them to white) g->sweepgcopage = g->allgcopages; @@ -1201,14 +1174,11 @@ void luaC_fullgc(lua_State* L) gcstep(L, SIZE_MAX); } - if (FFlag::LuauSimplerUpval) + // clear markedopen bits for all open upvalues; these might be stuck from half-finished mark prior to full gc + for (UpVal* uv = g->uvhead.u.open.next; uv != &g->uvhead; uv = uv->u.open.next) { - // clear markedopen bits for all open upvalues; these might be stuck from half-finished mark prior to full gc - for (UpVal* uv = g->uvhead.u.open.next; uv != &g->uvhead; uv = uv->u.open.next) - { - LUAU_ASSERT(upisopen(uv)); - uv->markedopen = 0; - } + LUAU_ASSERT(upisopen(uv)); + uv->markedopen = 0; } #ifdef LUAI_GCMETRICS @@ -1245,16 +1215,6 @@ void luaC_fullgc(lua_State* L) #endif } -void luaC_barrierupval(lua_State* L, GCObject* v) -{ - LUAU_ASSERT(!FFlag::LuauSimplerUpval); - global_State* g = L->global; - LUAU_ASSERT(iswhite(v) && !isdead(g, v)); - - if (keepinvariant(g)) - reallymarkobject(g, v); -} - void luaC_barrierf(lua_State* L, GCObject* o, GCObject* v) { global_State* g = L->global; @@ -1346,29 +1306,6 @@ int64_t luaC_allocationrate(lua_State* L) return int64_t((g->gcstats.atomicstarttotalsizebytes - g->gcstats.endtotalsizebytes) / duration); } -void luaC_wakethread(lua_State* L) -{ - LUAU_ASSERT(!FFlag::LuauNoSleepBit); - if (!luaC_threadsleeping(L)) - return; - - global_State* g = L->global; - - resetbit(L->stackstate, THREAD_SLEEPINGBIT); - - if (keepinvariant(g)) - { - GCObject* o = obj2gco(L); - - LUAU_ASSERT(isblack(o)); - - L->gclist = g->grayagain; - g->grayagain = o; - - black2gray(o); - } -} - const char* luaC_statename(int state) { switch (state) diff --git a/VM/src/lgc.h b/VM/src/lgc.h index 69379c89..51216bd8 100644 --- a/VM/src/lgc.h +++ b/VM/src/lgc.h @@ -6,8 +6,6 @@ #include "lobject.h" #include "lstate.h" -LUAU_FASTFLAG(LuauNoSleepBit) - /* ** Default settings for GC tunables (settable via lua_gc) */ @@ -75,14 +73,6 @@ LUAU_FASTFLAG(LuauNoSleepBit) #define luaC_white(g) cast_to(uint8_t, ((g)->currentwhite) & WHITEBITS) -// Thread stack states -// TODO: Remove with FFlag::LuauNoSleepBit and replace with lua_State::threadactive -#define THREAD_ACTIVEBIT 0 // thread is currently active -#define THREAD_SLEEPINGBIT 1 // thread is not executing and stack should not be modified - -#define luaC_threadactive(L) (testbit((L)->stackstate, THREAD_ACTIVEBIT)) -#define luaC_threadsleeping(L) (testbit((L)->stackstate, THREAD_SLEEPINGBIT)) - #define luaC_checkGC(L) \ { \ condhardstacktests(luaD_reallocstack(L, L->stacksize - EXTRA_STACK)); \ @@ -121,25 +111,10 @@ LUAU_FASTFLAG(LuauNoSleepBit) luaC_barrierf(L, obj2gco(p), obj2gco(o)); \ } -// TODO: Remove with FFlag::LuauSimplerUpval -#define luaC_upvalbarrier(L, uv, tv) \ - { \ - if (iscollectable(tv) && iswhite(gcvalue(tv)) && (!(uv) || (uv)->v != &(uv)->u.value)) \ - luaC_barrierupval(L, gcvalue(tv)); \ - } - #define luaC_threadbarrier(L) \ { \ - if (FFlag::LuauNoSleepBit) \ - { \ - if (isblack(obj2gco(L))) \ - luaC_barrierback(L, obj2gco(L), &L->gclist); \ - } \ - else \ - { \ - if (luaC_threadsleeping(L)) \ - luaC_wakethread(L); \ - } \ + if (isblack(obj2gco(L))) \ + luaC_barrierback(L, obj2gco(L), &L->gclist); \ } #define luaC_init(L, o, tt_) \ @@ -154,12 +129,10 @@ LUAI_FUNC size_t luaC_step(lua_State* L, bool assist); LUAI_FUNC void luaC_fullgc(lua_State* L); LUAI_FUNC void luaC_initobj(lua_State* L, GCObject* o, uint8_t tt); LUAI_FUNC void luaC_upvalclosed(lua_State* L, UpVal* uv); -LUAI_FUNC void luaC_barrierupval(lua_State* L, GCObject* v); LUAI_FUNC void luaC_barrierf(lua_State* L, GCObject* o, GCObject* v); LUAI_FUNC void luaC_barriertable(lua_State* L, Table* t, GCObject* v); LUAI_FUNC void luaC_barrierback(lua_State* L, GCObject* o, GCObject** gclist); LUAI_FUNC void luaC_validate(lua_State* L); LUAI_FUNC void luaC_dump(lua_State* L, void* file, const char* (*categoryName)(lua_State* L, uint8_t memcat)); LUAI_FUNC int64_t luaC_allocationrate(lua_State* L); -LUAI_FUNC void luaC_wakethread(lua_State* L); LUAI_FUNC const char* luaC_statename(int state); diff --git a/VM/src/lobject.h b/VM/src/lobject.h index 778e22ba..48aaf94b 100644 --- a/VM/src/lobject.h +++ b/VM/src/lobject.h @@ -332,8 +332,6 @@ typedef struct UpVal // thread linked list (when open) struct UpVal* threadnext; - // note: this is the location of a pointer to this upvalue in the previous element that can be either an UpVal or a lua_State - struct UpVal** threadprev; // TODO: remove with FFlag::LuauSimplerUpval } open; } u; } UpVal; diff --git a/VM/src/lstate.cpp b/VM/src/lstate.cpp index e1cb2ab7..fdd7fc2b 100644 --- a/VM/src/lstate.cpp +++ b/VM/src/lstate.cpp @@ -10,8 +10,6 @@ #include "ldo.h" #include "ldebug.h" -LUAU_FASTFLAG(LuauSimplerUpval) - /* ** Main thread combines a thread state and the global state */ @@ -79,7 +77,7 @@ static void preinit_state(lua_State* L, global_State* g) L->namecall = NULL; L->cachedslot = 0; L->singlestep = false; - L->stackstate = 0; + L->isactive = false; L->activememcat = 0; L->userdata = NULL; } @@ -89,7 +87,6 @@ static void close_state(lua_State* L) global_State* g = L->global; luaF_close(L, L->stack); // close all upvalues for this thread luaC_freeall(L); // collect all objects - LUAU_ASSERT(g->strbufgc == NULL); LUAU_ASSERT(g->strt.nuse == 0); luaM_freearray(L, L->global->strt.hash, L->global->strt.size, TString*, 0); freestack(L, L); @@ -121,11 +118,6 @@ lua_State* luaE_newthread(lua_State* L) void luaE_freethread(lua_State* L, lua_State* L1, lua_Page* page) { - if (!FFlag::LuauSimplerUpval) - { - luaF_close(L1, L1->stack); // close all upvalues for this thread - LUAU_ASSERT(L1->openupval == NULL); - } global_State* g = L->global; if (g->cb.userthread) g->cb.userthread(NULL, L1); @@ -199,7 +191,6 @@ lua_State* lua_newstate(lua_Alloc f, void* ud) g->gray = NULL; g->grayagain = NULL; g->weak = NULL; - g->strbufgc = NULL; g->totalbytes = sizeof(LG); g->gcgoal = LUAI_GCGOAL; g->gcstepmul = LUAI_GCSTEPMUL; diff --git a/VM/src/lstate.h b/VM/src/lstate.h index df47ce7e..06544463 100644 --- a/VM/src/lstate.h +++ b/VM/src/lstate.h @@ -167,8 +167,6 @@ typedef struct global_State GCObject* grayagain; // list of objects to be traversed atomically GCObject* weak; // list of weak tables (to be cleared) - TString* strbufgc; // list of all string buffer objects; TODO: remove with LuauNoStrbufLink - size_t GCthreshold; // when totalbytes > GCthreshold, run GC step size_t totalbytes; // number of bytes currently allocated @@ -222,8 +220,8 @@ struct lua_State uint8_t status; uint8_t activememcat; // memory category that is used for new GC object allocations - uint8_t stackstate; + bool isactive; // thread is currently executing, stack may be mutated without barriers bool singlestep; // call debugstep hook after each instruction diff --git a/VM/src/lstring.cpp b/VM/src/lstring.cpp index f43d03b1..e57f6c29 100644 --- a/VM/src/lstring.cpp +++ b/VM/src/lstring.cpp @@ -7,8 +7,6 @@ #include -LUAU_FASTFLAGVARIABLE(LuauNoStrbufLink, false) - unsigned int luaS_hash(const char* str, size_t len) { // Note that this hashing algorithm is replicated in BytecodeBuilder.cpp, BytecodeBuilder::getStringHash @@ -96,51 +94,18 @@ static TString* newlstr(lua_State* L, const char* str, size_t l, unsigned int h) return ts; } -static void unlinkstrbuf(lua_State* L, TString* ts) -{ - LUAU_ASSERT(!FFlag::LuauNoStrbufLink); - global_State* g = L->global; - - TString** p = &g->strbufgc; - - while (TString* curr = *p) - { - if (curr == ts) - { - *p = curr->next; - return; - } - else - { - p = &curr->next; - } - } - - LUAU_ASSERT(!"failed to find string buffer"); -} - TString* luaS_bufstart(lua_State* L, size_t size) { if (size > MAXSSIZE) luaM_toobig(L); - global_State* g = L->global; - TString* ts = luaM_newgco(L, TString, sizestring(size), L->activememcat); luaC_init(L, ts, LUA_TSTRING); ts->atom = ATOM_UNDEF; ts->hash = 0; // computed in luaS_buffinish ts->len = unsigned(size); - if (FFlag::LuauNoStrbufLink) - { - ts->next = NULL; - } - else - { - ts->next = g->strbufgc; - g->strbufgc = ts; - } + ts->next = NULL; return ts; } @@ -164,10 +129,7 @@ TString* luaS_buffinish(lua_State* L, TString* ts) } } - if (FFlag::LuauNoStrbufLink) - LUAU_ASSERT(ts->next == NULL); - else - unlinkstrbuf(L, ts); + LUAU_ASSERT(ts->next == NULL); ts->hash = h; ts->data[ts->len] = '\0'; // ending 0 @@ -222,21 +184,10 @@ static bool unlinkstr(lua_State* L, TString* ts) void luaS_free(lua_State* L, TString* ts, lua_Page* page) { - if (FFlag::LuauNoStrbufLink) - { - if (unlinkstr(L, ts)) - L->global->strt.nuse--; - else - LUAU_ASSERT(ts->next == NULL); // orphaned string buffer - } + if (unlinkstr(L, ts)) + L->global->strt.nuse--; else - { - // Unchain from the string table - if (!unlinkstr(L, ts)) - unlinkstrbuf(L, ts); // An unlikely scenario when we have a string buffer on our hands - else - L->global->strt.nuse--; - } + LUAU_ASSERT(ts->next == NULL); // orphaned string buffer luaM_freegco(L, ts, sizestring(ts->len), ts->memcat, page); } diff --git a/VM/src/lvmexecute.cpp b/VM/src/lvmexecute.cpp index e3488916..c3c744b2 100644 --- a/VM/src/lvmexecute.cpp +++ b/VM/src/lvmexecute.cpp @@ -16,9 +16,6 @@ #include -LUAU_FASTFLAG(LuauSimplerUpval) -LUAU_FASTFLAG(LuauNoSleepBit) - // Disable c99-designator to avoid the warning in CGOTO dispatch table #ifdef __clang__ #if __has_warning("-Wc99-designator") @@ -69,11 +66,6 @@ LUAU_FASTFLAG(LuauNoSleepBit) #define VM_PATCH_C(pc, slot) *const_cast(pc) = ((uint8_t(slot) << 24) | (0x00ffffffu & *(pc))) #define VM_PATCH_E(pc, slot) *const_cast(pc) = ((uint32_t(slot) << 8) | (0x000000ffu & *(pc))) -// NOTE: If debugging the Luau code, disable this macro to prevent timeouts from -// occurring when tracing code in Visual Studio / XCode -#if 0 -#define VM_INTERRUPT() -#else #define VM_INTERRUPT() \ { \ void (*interrupt)(lua_State*, int) = L->global->cb.interrupt; \ @@ -87,7 +79,6 @@ LUAU_FASTFLAG(LuauNoSleepBit) } \ } \ } -#endif #define VM_DISPATCH_OP(op) &&CASE_##op @@ -150,32 +141,6 @@ LUAU_NOINLINE static void luau_prepareFORN(lua_State* L, StkId plimit, StkId pst luaG_forerror(L, pstep, "step"); } -LUAU_NOINLINE static bool luau_loopFORG(lua_State* L, int a, int c) -{ - // note: it's safe to push arguments past top for complicated reasons (see top of the file) - StkId ra = &L->base[a]; - LUAU_ASSERT(ra + 3 <= L->top); - - setobjs2s(L, ra + 3 + 2, ra + 2); - setobjs2s(L, ra + 3 + 1, ra + 1); - setobjs2s(L, ra + 3, ra); - - L->top = ra + 3 + 3; // func. + 2 args (state and index) - LUAU_ASSERT(L->top <= L->stack_last); - - luaD_call(L, ra + 3, c); - L->top = L->ci->top; - - // recompute ra since stack might have been reallocated - ra = &L->base[a]; - LUAU_ASSERT(ra < L->top); - - // copy first variable back into the iteration index - setobjs2s(L, ra + 2, ra + 3); - - return ttisnil(ra + 2); -} - // calls a C function f with no yielding support; optionally save one resulting value to the res register // the function and arguments have to already be pushed to L->top LUAU_NOINLINE static void luau_callTM(lua_State* L, int nparams, int res) @@ -316,9 +281,8 @@ static void luau_execute(lua_State* L) const Instruction* pc; LUAU_ASSERT(isLua(L->ci)); - LUAU_ASSERT(luaC_threadactive(L)); - LUAU_ASSERT(!luaC_threadsleeping(L)); - LUAU_ASSERT(!FFlag::LuauNoSleepBit || !isblack(obj2gco(L))); // we don't use luaC_threadbarrier because active threads never turn black + LUAU_ASSERT(L->isactive); + LUAU_ASSERT(!isblack(obj2gco(L))); // we don't use luaC_threadbarrier because active threads never turn black pc = L->ci->savedpc; cl = clvalue(L->ci->func); @@ -498,8 +462,6 @@ static void luau_execute(lua_State* L) setobj(L, uv->v, ra); luaC_barrier(L, uv, ra); - if (!FFlag::LuauSimplerUpval) - luaC_upvalbarrier(L, uv, uv->v); VM_NEXT(); } @@ -2403,52 +2365,8 @@ static void luau_execute(lua_State* L) VM_CASE(LOP_DEP_FORGLOOP_INEXT) { - VM_INTERRUPT(); - Instruction insn = *pc++; - StkId ra = VM_REG(LUAU_INSN_A(insn)); - - // fast-path: ipairs/inext - if (ttisnil(ra) && ttistable(ra + 1) && ttislightuserdata(ra + 2)) - { - Table* h = hvalue(ra + 1); - int index = int(reinterpret_cast(pvalue(ra + 2))); - - // if 1-based index of the last iteration is in bounds, this means 0-based index of the current iteration is in bounds - if (unsigned(index) < unsigned(h->sizearray)) - { - // note that nil elements inside the array terminate the traversal - if (!ttisnil(&h->array[index])) - { - setpvalue(ra + 2, reinterpret_cast(uintptr_t(index + 1))); - setnvalue(ra + 3, double(index + 1)); - setobj2s(L, ra + 4, &h->array[index]); - - pc += LUAU_INSN_D(insn); - LUAU_ASSERT(unsigned(pc - cl->l.p->code) < unsigned(cl->l.p->sizecode)); - VM_NEXT(); - } - else - { - // fallthrough to exit - VM_NEXT(); - } - } - else - { - // fallthrough to exit - VM_NEXT(); - } - } - else - { - // slow-path; can call Lua/C generators - bool stop; - VM_PROTECT(stop = luau_loopFORG(L, LUAU_INSN_A(insn), 2)); - - pc += stop ? 0 : LUAU_INSN_D(insn); - LUAU_ASSERT(unsigned(pc - cl->l.p->code) < unsigned(cl->l.p->sizecode)); - VM_NEXT(); - } + LUAU_ASSERT(!"Unsupported deprecated opcode"); + LUAU_UNREACHABLE(); } VM_CASE(LOP_FORGPREP_NEXT) @@ -2475,68 +2393,8 @@ static void luau_execute(lua_State* L) VM_CASE(LOP_DEP_FORGLOOP_NEXT) { - VM_INTERRUPT(); - Instruction insn = *pc++; - StkId ra = VM_REG(LUAU_INSN_A(insn)); - - // fast-path: pairs/next - if (ttisnil(ra) && ttistable(ra + 1) && ttislightuserdata(ra + 2)) - { - Table* h = hvalue(ra + 1); - int index = int(reinterpret_cast(pvalue(ra + 2))); - - int sizearray = h->sizearray; - int sizenode = 1 << h->lsizenode; - - // first we advance index through the array portion - while (unsigned(index) < unsigned(sizearray)) - { - if (!ttisnil(&h->array[index])) - { - setpvalue(ra + 2, reinterpret_cast(uintptr_t(index + 1))); - setnvalue(ra + 3, double(index + 1)); - setobj2s(L, ra + 4, &h->array[index]); - - pc += LUAU_INSN_D(insn); - LUAU_ASSERT(unsigned(pc - cl->l.p->code) < unsigned(cl->l.p->sizecode)); - VM_NEXT(); - } - - index++; - } - - // then we advance index through the hash portion - while (unsigned(index - sizearray) < unsigned(sizenode)) - { - LuaNode* n = &h->node[index - sizearray]; - - if (!ttisnil(gval(n))) - { - setpvalue(ra + 2, reinterpret_cast(uintptr_t(index + 1))); - getnodekey(L, ra + 3, n); - setobj2s(L, ra + 4, gval(n)); - - pc += LUAU_INSN_D(insn); - LUAU_ASSERT(unsigned(pc - cl->l.p->code) < unsigned(cl->l.p->sizecode)); - VM_NEXT(); - } - - index++; - } - - // fallthrough to exit - VM_NEXT(); - } - else - { - // slow-path; can call Lua/C generators - bool stop; - VM_PROTECT(stop = luau_loopFORG(L, LUAU_INSN_A(insn), 2)); - - pc += stop ? 0 : LUAU_INSN_D(insn); - LUAU_ASSERT(unsigned(pc - cl->l.p->code) < unsigned(cl->l.p->sizecode)); - VM_NEXT(); - } + LUAU_ASSERT(!"Unsupported deprecated opcode"); + LUAU_UNREACHABLE(); } VM_CASE(LOP_GETVARARGS) @@ -2750,92 +2608,14 @@ static void luau_execute(lua_State* L) VM_CASE(LOP_DEP_JUMPIFEQK) { - Instruction insn = *pc++; - uint32_t aux = *pc; - StkId ra = VM_REG(LUAU_INSN_A(insn)); - TValue* rb = VM_KV(aux); - - // Note that all jumps below jump by 1 in the "false" case to skip over aux - if (ttype(ra) == ttype(rb)) - { - switch (ttype(ra)) - { - case LUA_TNIL: - pc += LUAU_INSN_D(insn); - LUAU_ASSERT(unsigned(pc - cl->l.p->code) < unsigned(cl->l.p->sizecode)); - VM_NEXT(); - - case LUA_TBOOLEAN: - pc += bvalue(ra) == bvalue(rb) ? LUAU_INSN_D(insn) : 1; - LUAU_ASSERT(unsigned(pc - cl->l.p->code) < unsigned(cl->l.p->sizecode)); - VM_NEXT(); - - case LUA_TNUMBER: - pc += nvalue(ra) == nvalue(rb) ? LUAU_INSN_D(insn) : 1; - LUAU_ASSERT(unsigned(pc - cl->l.p->code) < unsigned(cl->l.p->sizecode)); - VM_NEXT(); - - case LUA_TSTRING: - pc += gcvalue(ra) == gcvalue(rb) ? LUAU_INSN_D(insn) : 1; - LUAU_ASSERT(unsigned(pc - cl->l.p->code) < unsigned(cl->l.p->sizecode)); - VM_NEXT(); - - default:; - } - - LUAU_ASSERT(!"Constant is expected to be of primitive type"); - } - else - { - pc += 1; - LUAU_ASSERT(unsigned(pc - cl->l.p->code) < unsigned(cl->l.p->sizecode)); - VM_NEXT(); - } + LUAU_ASSERT(!"Unsupported deprecated opcode"); + LUAU_UNREACHABLE(); } VM_CASE(LOP_DEP_JUMPIFNOTEQK) { - Instruction insn = *pc++; - uint32_t aux = *pc; - StkId ra = VM_REG(LUAU_INSN_A(insn)); - TValue* rb = VM_KV(aux); - - // Note that all jumps below jump by 1 in the "true" case to skip over aux - if (ttype(ra) == ttype(rb)) - { - switch (ttype(ra)) - { - case LUA_TNIL: - pc += 1; - LUAU_ASSERT(unsigned(pc - cl->l.p->code) < unsigned(cl->l.p->sizecode)); - VM_NEXT(); - - case LUA_TBOOLEAN: - pc += bvalue(ra) != bvalue(rb) ? LUAU_INSN_D(insn) : 1; - LUAU_ASSERT(unsigned(pc - cl->l.p->code) < unsigned(cl->l.p->sizecode)); - VM_NEXT(); - - case LUA_TNUMBER: - pc += nvalue(ra) != nvalue(rb) ? LUAU_INSN_D(insn) : 1; - LUAU_ASSERT(unsigned(pc - cl->l.p->code) < unsigned(cl->l.p->sizecode)); - VM_NEXT(); - - case LUA_TSTRING: - pc += gcvalue(ra) != gcvalue(rb) ? LUAU_INSN_D(insn) : 1; - LUAU_ASSERT(unsigned(pc - cl->l.p->code) < unsigned(cl->l.p->sizecode)); - VM_NEXT(); - - default:; - } - - LUAU_ASSERT(!"Constant is expected to be of primitive type"); - } - else - { - pc += LUAU_INSN_D(insn); - LUAU_ASSERT(unsigned(pc - cl->l.p->code) < unsigned(cl->l.p->sizecode)); - VM_NEXT(); - } + LUAU_ASSERT(!"Unsupported deprecated opcode"); + LUAU_UNREACHABLE(); } VM_CASE(LOP_FASTCALL1) diff --git a/tests/AstQuery.test.cpp b/tests/AstQuery.test.cpp index 2b650fa4..4b21c443 100644 --- a/tests/AstQuery.test.cpp +++ b/tests/AstQuery.test.cpp @@ -1,9 +1,10 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details -#include "Fixture.h" #include "Luau/AstQuery.h" +#include "AstQueryDsl.h" #include "doctest.h" +#include "Fixture.h" using namespace Luau; diff --git a/tests/AstQueryDsl.cpp b/tests/AstQueryDsl.cpp new file mode 100644 index 00000000..0cf28f3b --- /dev/null +++ b/tests/AstQueryDsl.cpp @@ -0,0 +1,45 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details + +#include "AstQueryDsl.h" + +namespace Luau +{ + +FindNthOccurenceOf::FindNthOccurenceOf(Nth nth) + : requestedNth(nth) +{ +} + +bool FindNthOccurenceOf::checkIt(AstNode* n) +{ + if (theNode) + return false; + + if (n->classIndex == requestedNth.classIndex) + { + // Human factor: the requestedNth starts from 1 because of the term `nth`. + if (currentOccurrence + 1 != requestedNth.nth) + ++currentOccurrence; + else + theNode = n; + } + + return !theNode; // once found, returns false and stops traversal +} + +bool FindNthOccurenceOf::visit(AstNode* n) +{ + return checkIt(n); +} + +bool FindNthOccurenceOf::visit(AstType* t) +{ + return checkIt(t); +} + +bool FindNthOccurenceOf::visit(AstTypePack* t) +{ + return checkIt(t); +} + +} diff --git a/tests/AstQueryDsl.h b/tests/AstQueryDsl.h new file mode 100644 index 00000000..6bf3bd30 --- /dev/null +++ b/tests/AstQueryDsl.h @@ -0,0 +1,83 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +#pragma once + +#include "Luau/Ast.h" +#include "Luau/Common.h" + +#include +#include + +namespace Luau +{ + +struct Nth +{ + int classIndex; + int nth; +}; + +template +Nth nth(int nth = 1) +{ + static_assert(std::is_base_of_v, "T must be a derived class of AstNode"); + LUAU_ASSERT(nth > 0); // Did you mean to use `nth(1)`? + + return Nth{T::ClassIndex(), nth}; +} + +struct FindNthOccurenceOf : public AstVisitor +{ + Nth requestedNth; + int currentOccurrence = 0; + AstNode* theNode = nullptr; + + FindNthOccurenceOf(Nth nth); + + bool checkIt(AstNode* n); + + bool visit(AstNode* n) override; + bool visit(AstType* n) override; + bool visit(AstTypePack* n) override; +}; + +/** DSL querying of the AST. + * + * Given an AST, one can query for a particular node directly without having to manually unwrap the tree, for example: + * + * ``` + * if a and b then + * print(a + b) + * end + * + * function f(x, y) + * return x + y + * end + * ``` + * + * There are numerous ways to access the second AstExprBinary. + * 1. Luau::query(block, {nth(), nth()}) + * 2. Luau::query(Luau::query(block)) + * 3. Luau::query(block, {nth(2)}) + */ +template +T* query(AstNode* node, const std::vector& nths = {nth(N)}) +{ + static_assert(std::is_base_of_v, "T must be a derived class of AstNode"); + + // If a nested query call fails to find the node in question, subsequent calls can propagate rather than trying to do more. + // This supports `query(query(...))` + + for (Nth nth : nths) + { + if (!node) + return nullptr; + + FindNthOccurenceOf finder{nth}; + node->visit(&finder); + node = finder.theNode; + } + + return node ? node->as() : nullptr; +} + +} diff --git a/tests/CodeAllocator.test.cpp b/tests/CodeAllocator.test.cpp index 41f3a900..758fb44c 100644 --- a/tests/CodeAllocator.test.cpp +++ b/tests/CodeAllocator.test.cpp @@ -1,9 +1,16 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #include "Luau/AssemblyBuilderX64.h" #include "Luau/CodeAllocator.h" +#include "Luau/CodeBlockUnwind.h" +#include "Luau/UnwindBuilder.h" +#include "Luau/UnwindBuilderDwarf2.h" +#include "Luau/UnwindBuilderWin.h" #include "doctest.h" +#include +#include + #include using namespace Luau::CodeGen; @@ -41,8 +48,8 @@ TEST_CASE("CodeAllocation") TEST_CASE("CodeAllocationFailure") { - size_t blockSize = 16384; - size_t maxTotalSize = 32768; + size_t blockSize = 3000; + size_t maxTotalSize = 7000; CodeAllocator allocator(blockSize, maxTotalSize); uint8_t* nativeData; @@ -50,13 +57,13 @@ TEST_CASE("CodeAllocationFailure") uint8_t* nativeEntry; std::vector code; - code.resize(18000); + code.resize(4000); // allocation has to fit in a block REQUIRE(!allocator.allocate(nullptr, 0, code.data(), code.size(), nativeData, sizeNativeData, nativeEntry)); // each allocation exhausts a block, so third allocation fails - code.resize(10000); + code.resize(2000); REQUIRE(allocator.allocate(nullptr, 0, code.data(), code.size(), nativeData, sizeNativeData, nativeEntry)); REQUIRE(allocator.allocate(nullptr, 0, code.data(), code.size(), nativeData, sizeNativeData, nativeEntry)); REQUIRE(!allocator.allocate(nullptr, 0, code.data(), code.size(), nativeData, sizeNativeData, nativeEntry)); @@ -120,19 +127,84 @@ TEST_CASE("CodeAllocationWithUnwindCallbacks") CHECK(info.destroyCalled); } -#if defined(__x86_64__) || defined(_M_X64) -TEST_CASE("GeneratedCodeExecution") +TEST_CASE("WindowsUnwindCodesX64") { + UnwindBuilderWin unwind; + + unwind.start(); + unwind.spill(16, rdx); + unwind.spill(8, rcx); + unwind.save(rdi); + unwind.save(rsi); + unwind.save(rbx); + unwind.save(rbp); + unwind.save(r12); + unwind.save(r13); + unwind.save(r14); + unwind.save(r15); + unwind.allocStack(72); + unwind.setupFrameReg(rbp, 48); + unwind.finish(); + + std::vector data; + data.resize(unwind.getSize()); + unwind.finalize(data.data(), nullptr, 0); + + std::vector expected{0x01, 0x23, 0x0a, 0x35, 0x23, 0x33, 0x1e, 0x82, 0x1a, 0xf0, 0x18, 0xe0, 0x16, 0xd0, 0x14, 0xc0, 0x12, 0x50, 0x10, + 0x30, 0x0e, 0x60, 0x0c, 0x70}; + + REQUIRE(data.size() == expected.size()); + CHECK(memcmp(data.data(), expected.data(), expected.size()) == 0); +} + +TEST_CASE("Dwarf2UnwindCodesX64") +{ + UnwindBuilderDwarf2 unwind; + + unwind.start(); + unwind.save(rdi); + unwind.save(rsi); + unwind.save(rbx); + unwind.save(rbp); + unwind.save(r12); + unwind.save(r13); + unwind.save(r14); + unwind.save(r15); + unwind.allocStack(72); + unwind.setupFrameReg(rbp, 48); + unwind.finish(); + + std::vector data; + data.resize(unwind.getSize()); + unwind.finalize(data.data(), nullptr, 0); + + std::vector expected{0x14, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01, 0x00, 0x01, 0x78, 0x10, 0x0c, 0x07, 0x08, 0x05, 0x10, 0x01, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x4c, 0x00, 0x00, 0x00, 0x1c, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x02, 0x02, 0x0e, 0x10, 0x85, 0x02, 0x02, 0x02, 0x0e, 0x18, 0x84, 0x03, 0x02, 0x02, 0x0e, 0x20, 0x83, + 0x04, 0x02, 0x02, 0x0e, 0x28, 0x86, 0x05, 0x02, 0x02, 0x0e, 0x30, 0x8c, 0x06, 0x02, 0x02, 0x0e, 0x38, 0x8d, 0x07, 0x02, 0x02, 0x0e, 0x40, + 0x8e, 0x08, 0x02, 0x02, 0x0e, 0x48, 0x8f, 0x09, 0x02, 0x04, 0x0e, 0x90, 0x01, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00}; + + REQUIRE(data.size() == expected.size()); + CHECK(memcmp(data.data(), expected.data(), expected.size()) == 0); +} + +#if defined(__x86_64__) || defined(_M_X64) + #if defined(_WIN32) - // Windows x64 ABI - constexpr RegisterX64 rArg1 = rcx; - constexpr RegisterX64 rArg2 = rdx; +// Windows x64 ABI +constexpr RegisterX64 rArg1 = rcx; +constexpr RegisterX64 rArg2 = rdx; #else - // System V AMD64 ABI - constexpr RegisterX64 rArg1 = rdi; - constexpr RegisterX64 rArg2 = rsi; +// System V AMD64 ABI +constexpr RegisterX64 rArg1 = rdi; +constexpr RegisterX64 rArg2 = rsi; #endif +constexpr RegisterX64 rNonVol1 = r12; +constexpr RegisterX64 rNonVol2 = rbx; + +TEST_CASE("GeneratedCodeExecution") +{ AssemblyBuilderX64 build(/* logText= */ false); build.mov(rax, rArg1); @@ -157,6 +229,90 @@ TEST_CASE("GeneratedCodeExecution") int64_t result = f(10, 20); CHECK(result == 210); } + +void throwing(int64_t arg) +{ + CHECK(arg == 25); + + throw std::runtime_error("testing"); +} + +TEST_CASE("GeneratedCodeExecutionWithThrow") +{ + AssemblyBuilderX64 build(/* logText= */ false); + +#if defined(_WIN32) + std::unique_ptr unwind = std::make_unique(); +#else + std::unique_ptr unwind = std::make_unique(); +#endif + + unwind->start(); + + // Prologue + build.push(rNonVol1); + unwind->save(rNonVol1); + build.push(rNonVol2); + unwind->save(rNonVol2); + build.push(rbp); + unwind->save(rbp); + + int stackSize = 32; + int localsSize = 16; + + build.sub(rsp, stackSize + localsSize); + unwind->allocStack(stackSize + localsSize); + + build.lea(rbp, qword[rsp + stackSize]); + unwind->setupFrameReg(rbp, stackSize); + + unwind->finish(); + + // Body + build.mov(rNonVol1, rArg1); + build.mov(rNonVol2, rArg2); + + build.add(rNonVol1, 15); + build.mov(rArg1, rNonVol1); + build.call(rNonVol2); + + // Epilogue + build.lea(rsp, qword[rbp + localsSize]); + build.pop(rbp); + build.pop(rNonVol2); + build.pop(rNonVol1); + build.ret(); + + build.finalize(); + + size_t blockSize = 1024 * 1024; + size_t maxTotalSize = 1024 * 1024; + CodeAllocator allocator(blockSize, maxTotalSize); + + allocator.context = unwind.get(); + allocator.createBlockUnwindInfo = createBlockUnwindInfo; + allocator.destroyBlockUnwindInfo = destroyBlockUnwindInfo; + + uint8_t* nativeData; + size_t sizeNativeData; + uint8_t* nativeEntry; + REQUIRE(allocator.allocate(build.data.data(), build.data.size(), build.code.data(), build.code.size(), nativeData, sizeNativeData, nativeEntry)); + REQUIRE(nativeEntry); + + using FunctionType = int64_t(int64_t, void (*)(int64_t)); + FunctionType* f = (FunctionType*)nativeEntry; + + // To simplify debugging, CHECK_THROWS_WITH_AS is not used here + try + { + f(10, throwing); + } + catch (const std::runtime_error& error) + { + CHECK(strcmp(error.what(), "testing") == 0); + } +} + #endif TEST_SUITE_END(); diff --git a/tests/ConstraintGraphBuilder.test.cpp b/tests/ConstraintGraphBuilder.test.cpp index bbe29429..5c34e3d6 100644 --- a/tests/ConstraintGraphBuilder.test.cpp +++ b/tests/ConstraintGraphBuilder.test.cpp @@ -1,8 +1,11 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details -#include "Fixture.h" #include "Luau/ConstraintGraphBuilder.h" +#include "Luau/NotNull.h" +#include "Luau/ToString.h" +#include "ConstraintGraphBuilderFixture.h" +#include "Fixture.h" #include "doctest.h" using namespace Luau; diff --git a/tests/ConstraintGraphBuilderFixture.cpp b/tests/ConstraintGraphBuilderFixture.cpp new file mode 100644 index 00000000..1958d1dc --- /dev/null +++ b/tests/ConstraintGraphBuilderFixture.cpp @@ -0,0 +1,17 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +#include "ConstraintGraphBuilderFixture.h" + +namespace Luau +{ + +ConstraintGraphBuilderFixture::ConstraintGraphBuilderFixture() + : Fixture() + , mainModule(new Module) + , cgb("MainModule", mainModule, &arena, NotNull(&moduleResolver), singletonTypes, NotNull(&ice), frontend.getGlobalScope(), &logger) + , forceTheFlag{"DebugLuauDeferredConstraintResolution", true} +{ + BlockedTypeVar::nextIndex = 0; + BlockedTypePack::nextIndex = 0; +} + +} diff --git a/tests/ConstraintGraphBuilderFixture.h b/tests/ConstraintGraphBuilderFixture.h new file mode 100644 index 00000000..262e3901 --- /dev/null +++ b/tests/ConstraintGraphBuilderFixture.h @@ -0,0 +1,27 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +#pragma once + +#include "Luau/ConstraintGraphBuilder.h" +#include "Luau/DcrLogger.h" +#include "Luau/TypeArena.h" +#include "Luau/Module.h" + +#include "Fixture.h" +#include "ScopedFlags.h" + +namespace Luau +{ + +struct ConstraintGraphBuilderFixture : Fixture +{ + TypeArena arena; + ModulePtr mainModule; + ConstraintGraphBuilder cgb; + DcrLogger logger; + + ScopedFastFlag forceTheFlag; + + ConstraintGraphBuilderFixture(); +}; + +} diff --git a/tests/ConstraintSolver.test.cpp b/tests/ConstraintSolver.test.cpp index fba57823..9976bd2c 100644 --- a/tests/ConstraintSolver.test.cpp +++ b/tests/ConstraintSolver.test.cpp @@ -1,12 +1,12 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details -#include "Fixture.h" - -#include "doctest.h" - #include "Luau/ConstraintGraphBuilder.h" #include "Luau/ConstraintSolver.h" +#include "ConstraintGraphBuilderFixture.h" +#include "Fixture.h" +#include "doctest.h" + using namespace Luau; static TypeId requireBinding(NotNull scope, const char* name) diff --git a/tests/Fixture.cpp b/tests/Fixture.cpp index 6c4594f4..3f77978c 100644 --- a/tests/Fixture.cpp +++ b/tests/Fixture.cpp @@ -444,16 +444,6 @@ BuiltinsFixture::BuiltinsFixture(bool freeze, bool prepareAutocomplete) Luau::freeze(frontend.typeCheckerForAutocomplete.globalTypes); } -ConstraintGraphBuilderFixture::ConstraintGraphBuilderFixture() - : Fixture() - , mainModule(new Module) - , cgb(mainModuleName, mainModule, &arena, NotNull(&moduleResolver), singletonTypes, NotNull(&ice), frontend.getGlobalScope(), &logger) - , forceTheFlag{"DebugLuauDeferredConstraintResolution", true} -{ - BlockedTypeVar::nextIndex = 0; - BlockedTypePack::nextIndex = 0; -} - ModuleName fromString(std::string_view name) { return ModuleName(name); @@ -516,41 +506,4 @@ void dump(const std::vector& constraints) printf("%s\n", toString(c, opts).c_str()); } -FindNthOccurenceOf::FindNthOccurenceOf(Nth nth) - : requestedNth(nth) -{ -} - -bool FindNthOccurenceOf::checkIt(AstNode* n) -{ - if (theNode) - return false; - - if (n->classIndex == requestedNth.classIndex) - { - // Human factor: the requestedNth starts from 1 because of the term `nth`. - if (currentOccurrence + 1 != requestedNth.nth) - ++currentOccurrence; - else - theNode = n; - } - - return !theNode; // once found, returns false and stops traversal -} - -bool FindNthOccurenceOf::visit(AstNode* n) -{ - return checkIt(n); -} - -bool FindNthOccurenceOf::visit(AstType* t) -{ - return checkIt(t); -} - -bool FindNthOccurenceOf::visit(AstTypePack* t) -{ - return checkIt(t); -} - } // namespace Luau diff --git a/tests/Fixture.h b/tests/Fixture.h index 03101bbf..2fb48468 100644 --- a/tests/Fixture.h +++ b/tests/Fixture.h @@ -12,7 +12,6 @@ #include "Luau/ToString.h" #include "Luau/TypeInfer.h" #include "Luau/TypeVar.h" -#include "Luau/DcrLogger.h" #include "IostreamOptional.h" #include "ScopedFlags.h" @@ -162,18 +161,6 @@ struct BuiltinsFixture : Fixture BuiltinsFixture(bool freeze = true, bool prepareAutocomplete = false); }; -struct ConstraintGraphBuilderFixture : Fixture -{ - TypeArena arena; - ModulePtr mainModule; - ConstraintGraphBuilder cgb; - DcrLogger logger; - - ScopedFastFlag forceTheFlag; - - ConstraintGraphBuilderFixture(); -}; - ModuleName fromString(std::string_view name); template @@ -199,76 +186,6 @@ std::optional lookupName(ScopePtr scope, const std::string& name); // Wa std::optional linearSearchForBinding(Scope* scope, const char* name); -struct Nth -{ - int classIndex; - int nth; -}; - -template -Nth nth(int nth = 1) -{ - static_assert(std::is_base_of_v, "T must be a derived class of AstNode"); - LUAU_ASSERT(nth > 0); // Did you mean to use `nth(1)`? - - return Nth{T::ClassIndex(), nth}; -} - -struct FindNthOccurenceOf : public AstVisitor -{ - Nth requestedNth; - int currentOccurrence = 0; - AstNode* theNode = nullptr; - - FindNthOccurenceOf(Nth nth); - - bool checkIt(AstNode* n); - - bool visit(AstNode* n) override; - bool visit(AstType* n) override; - bool visit(AstTypePack* n) override; -}; - -/** DSL querying of the AST. - * - * Given an AST, one can query for a particular node directly without having to manually unwrap the tree, for example: - * - * ``` - * if a and b then - * print(a + b) - * end - * - * function f(x, y) - * return x + y - * end - * ``` - * - * There are numerous ways to access the second AstExprBinary. - * 1. Luau::query(block, {nth(), nth()}) - * 2. Luau::query(Luau::query(block)) - * 3. Luau::query(block, {nth(2)}) - */ -template -T* query(AstNode* node, const std::vector& nths = {nth(N)}) -{ - static_assert(std::is_base_of_v, "T must be a derived class of AstNode"); - - // If a nested query call fails to find the node in question, subsequent calls can propagate rather than trying to do more. - // This supports `query(query(...))` - - for (Nth nth : nths) - { - if (!node) - return nullptr; - - FindNthOccurenceOf finder{nth}; - node->visit(&finder); - node = finder.theNode; - } - - return node ? node->as() : nullptr; -} - } // namespace Luau #define LUAU_REQUIRE_ERRORS(result) \ diff --git a/tests/TypeInfer.builtins.test.cpp b/tests/TypeInfer.builtins.test.cpp index 07a04363..6da3f569 100644 --- a/tests/TypeInfer.builtins.test.cpp +++ b/tests/TypeInfer.builtins.test.cpp @@ -459,7 +459,7 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "thread_is_a_type") )"); LUAU_REQUIRE_NO_ERRORS(result); - CHECK_EQ(*typeChecker.threadType, *requireType("co")); + CHECK("thread" == toString(requireType("co"))); } TEST_CASE_FIXTURE(BuiltinsFixture, "coroutine_resume_anything_goes") @@ -627,6 +627,8 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "select_with_decimal_argument_is_rounded_down // Could be flaky if the fix has regressed. TEST_CASE_FIXTURE(BuiltinsFixture, "bad_select_should_not_crash") { + ScopedFastFlag luauFunctionArgMismatchDetails{"LuauFunctionArgMismatchDetails", true}; + CheckResult result = check(R"( do end local _ = function(l0,...) @@ -638,8 +640,8 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "bad_select_should_not_crash") )"); LUAU_REQUIRE_ERROR_COUNT(2, result); - CHECK_EQ("Argument count mismatch. Function expects at least 1 argument, but none are specified", toString(result.errors[0])); - CHECK_EQ("Argument count mismatch. Function expects 1 argument, but none are specified", toString(result.errors[1])); + CHECK_EQ("Argument count mismatch. Function '_' expects at least 1 argument, but none are specified", toString(result.errors[0])); + CHECK_EQ("Argument count mismatch. Function 'select' expects 1 argument, but none are specified", toString(result.errors[1])); } TEST_CASE_FIXTURE(BuiltinsFixture, "select_way_out_of_range") @@ -824,12 +826,12 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "string_lib_self_noself") TEST_CASE_FIXTURE(BuiltinsFixture, "gmatch_definition") { CheckResult result = check(R"_( -local a, b, c = ("hey"):gmatch("(.)(.)(.)")() + local a, b, c = ("hey"):gmatch("(.)(.)(.)")() -for c in ("hey"):gmatch("(.)") do - print(c:upper()) -end -)_"); + for c in ("hey"):gmatch("(.)") do + print(c:upper()) + end + )_"); LUAU_REQUIRE_NO_ERRORS(result); } @@ -1008,6 +1010,8 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "table_freeze_is_generic") TEST_CASE_FIXTURE(BuiltinsFixture, "set_metatable_needs_arguments") { + ScopedFastFlag luauFunctionArgMismatchDetails{"LuauFunctionArgMismatchDetails", true}; + ScopedFastFlag sff{"LuauSetMetaTableArgsCheck", true}; CheckResult result = check(R"( local a = {b=setmetatable} @@ -1016,8 +1020,8 @@ a:b() a:b({}) )"); LUAU_REQUIRE_ERROR_COUNT(2, result); - CHECK_EQ(toString(result.errors[0]), "Argument count mismatch. Function expects 2 arguments, but none are specified"); - CHECK_EQ(toString(result.errors[1]), "Argument count mismatch. Function expects 2 arguments, but only 1 is specified"); + CHECK_EQ(toString(result.errors[0]), "Argument count mismatch. Function 'a.b' expects 2 arguments, but none are specified"); + CHECK_EQ(toString(result.errors[1]), "Argument count mismatch. Function 'a.b' expects 2 arguments, but only 1 is specified"); } TEST_CASE_FIXTURE(Fixture, "typeof_unresolved_function") diff --git a/tests/TypeInfer.functions.test.cpp b/tests/TypeInfer.functions.test.cpp index 35e67ec5..bde28dcc 100644 --- a/tests/TypeInfer.functions.test.cpp +++ b/tests/TypeInfer.functions.test.cpp @@ -1732,4 +1732,56 @@ TEST_CASE_FIXTURE(Fixture, "dont_mutate_the_underlying_head_of_typepack_when_cal LUAU_REQUIRE_NO_ERRORS(result); } +TEST_CASE_FIXTURE(BuiltinsFixture, "improved_function_arg_mismatch_errors") +{ + ScopedFastFlag luauFunctionArgMismatchDetails{"LuauFunctionArgMismatchDetails", true}; + + CheckResult result = check(R"( +local function foo1(a: number) end +foo1() + +local function foo2(a: number, b: string?) end +foo2() + +local function foo3(a: number, b: string?, c: any) end -- any is optional +foo3() + +string.find() + +local t = {} +function t.foo(x: number, y: string?, ...: any) end +function t:bar(x: number, y: string?) end +t.foo() + +t:bar() + +local u = { a = t } +u.a.foo() + )"); + + LUAU_REQUIRE_ERROR_COUNT(7, result); + CHECK_EQ(toString(result.errors[0]), "Argument count mismatch. Function 'foo1' expects 1 argument, but none are specified"); + CHECK_EQ(toString(result.errors[1]), "Argument count mismatch. Function 'foo2' expects 1 to 2 arguments, but none are specified"); + CHECK_EQ(toString(result.errors[2]), "Argument count mismatch. Function 'foo3' expects 1 to 3 arguments, but none are specified"); + CHECK_EQ(toString(result.errors[3]), "Argument count mismatch. Function 'string.find' expects 2 to 4 arguments, but none are specified"); + CHECK_EQ(toString(result.errors[4]), "Argument count mismatch. Function 't.foo' expects at least 1 argument, but none are specified"); + CHECK_EQ(toString(result.errors[5]), "Argument count mismatch. Function 't.bar' expects 2 to 3 arguments, but only 1 is specified"); + CHECK_EQ(toString(result.errors[6]), "Argument count mismatch. Function 'u.a.foo' expects at least 1 argument, but none are specified"); +} + +// This might be surprising, but since 'any' became optional, unannotated functions in non-strict 'expect' 0 arguments +TEST_CASE_FIXTURE(BuiltinsFixture, "improved_function_arg_mismatch_error_nonstrict") +{ + ScopedFastFlag luauFunctionArgMismatchDetails{"LuauFunctionArgMismatchDetails", true}; + + CheckResult result = check(R"( +--!nonstrict +local function foo(a, b) end +foo(string.find("hello", "e")) + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ(toString(result.errors[0]), "Argument count mismatch. Function 'foo' expects 0 to 2 arguments, but 3 are specified"); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.generics.test.cpp b/tests/TypeInfer.generics.test.cpp index 9ac259cf..3c867770 100644 --- a/tests/TypeInfer.generics.test.cpp +++ b/tests/TypeInfer.generics.test.cpp @@ -782,6 +782,8 @@ local TheDispatcher: Dispatcher = { TEST_CASE_FIXTURE(Fixture, "generic_argument_count_too_few") { + ScopedFastFlag luauFunctionArgMismatchDetails{"LuauFunctionArgMismatchDetails", true}; + CheckResult result = check(R"( function test(a: number) return 1 @@ -794,11 +796,13 @@ wrapper(test) )"); LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK_EQ(toString(result.errors[0]), R"(Argument count mismatch. Function expects 2 arguments, but only 1 is specified)"); + CHECK_EQ(toString(result.errors[0]), R"(Argument count mismatch. Function 'wrapper' expects 2 arguments, but only 1 is specified)"); } TEST_CASE_FIXTURE(Fixture, "generic_argument_count_too_many") { + ScopedFastFlag luauFunctionArgMismatchDetails{"LuauFunctionArgMismatchDetails", true}; + CheckResult result = check(R"( function test2(a: number, b: string) return 1 @@ -811,7 +815,7 @@ wrapper(test2, 1, "", 3) )"); LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK_EQ(toString(result.errors[0]), R"(Argument count mismatch. Function expects 3 arguments, but 4 are specified)"); + CHECK_EQ(toString(result.errors[0]), R"(Argument count mismatch. Function 'wrapper' expects 3 arguments, but 4 are specified)"); } TEST_CASE_FIXTURE(Fixture, "generic_function") diff --git a/tests/TypeInfer.provisional.test.cpp b/tests/TypeInfer.provisional.test.cpp index 3482b75c..40ea0ca1 100644 --- a/tests/TypeInfer.provisional.test.cpp +++ b/tests/TypeInfer.provisional.test.cpp @@ -69,6 +69,8 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "xpcall_returns_what_f_returns") CHECK("string" == toString(requireType("c"))); CHECK(expected == decorateWithTypes(code)); + + LUAU_REQUIRE_NO_ERRORS(result); } // We had a bug where if you have two type packs that looks like: diff --git a/tests/TypeInfer.singletons.test.cpp b/tests/TypeInfer.singletons.test.cpp index 0a130d49..7d98b5db 100644 --- a/tests/TypeInfer.singletons.test.cpp +++ b/tests/TypeInfer.singletons.test.cpp @@ -202,6 +202,15 @@ TEST_CASE_FIXTURE(Fixture, "tagged_unions_immutable_tag") LUAU_REQUIRE_ERRORS(result); } +TEST_CASE_FIXTURE(Fixture, "table_has_a_boolean") +{ + CheckResult result = check(R"( + local t={a=1,b=false} + )"); + + CHECK("{ a: number, b: boolean }" == toString(requireType("t"), {true})); +} + TEST_CASE_FIXTURE(Fixture, "table_properties_singleton_strings") { CheckResult result = check(R"( diff --git a/tests/TypeInfer.tables.test.cpp b/tests/TypeInfer.tables.test.cpp index 97c3da4f..d183f650 100644 --- a/tests/TypeInfer.tables.test.cpp +++ b/tests/TypeInfer.tables.test.cpp @@ -2615,9 +2615,11 @@ do end TEST_CASE_FIXTURE(BuiltinsFixture, "dont_crash_when_setmetatable_does_not_produce_a_metatabletypevar") { + ScopedFastFlag luauFunctionArgMismatchDetails{"LuauFunctionArgMismatchDetails", true}; + CheckResult result = check("local x = setmetatable({})"); LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK_EQ("Argument count mismatch. Function expects 2 arguments, but only 1 is specified", toString(result.errors[0])); + CHECK_EQ("Argument count mismatch. Function 'setmetatable' expects 2 arguments, but only 1 is specified", toString(result.errors[0])); } TEST_CASE_FIXTURE(BuiltinsFixture, "instantiate_table_cloning") @@ -2695,6 +2697,8 @@ local baz = foo[bar] TEST_CASE_FIXTURE(BuiltinsFixture, "table_simple_call") { + ScopedFastFlag luauFunctionArgMismatchDetails{"LuauFunctionArgMismatchDetails", true}; + CheckResult result = check(R"( local a = setmetatable({ x = 2 }, { __call = function(self) @@ -2706,7 +2710,7 @@ local c = a(2) -- too many arguments )"); LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK_EQ("Argument count mismatch. Function expects 1 argument, but 2 are specified", toString(result.errors[0])); + CHECK_EQ("Argument count mismatch. Function 'a' expects 1 argument, but 2 are specified", toString(result.errors[0])); } TEST_CASE_FIXTURE(BuiltinsFixture, "access_index_metamethod_that_returns_variadic") diff --git a/tests/TypeInfer.test.cpp b/tests/TypeInfer.test.cpp index e1dc5023..e8bfb67f 100644 --- a/tests/TypeInfer.test.cpp +++ b/tests/TypeInfer.test.cpp @@ -1105,4 +1105,21 @@ end CHECK_EQ(*getMainModule()->astResolvedTypes.find(annotation), *ty); } +TEST_CASE_FIXTURE(Fixture, "bidirectional_checking_of_higher_order_function") +{ + CheckResult result = check(R"( + function higher(cb: (number) -> ()) end + + higher(function(n) -- no error here. n : number + local e: string = n -- error here. n /: string + end) + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + Location location = result.errors[0].location; + CHECK(location.begin.line == 4); + CHECK(location.end.line == 4); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.typePacks.cpp b/tests/TypeInfer.typePacks.cpp index eaa8b053..7d33809f 100644 --- a/tests/TypeInfer.typePacks.cpp +++ b/tests/TypeInfer.typePacks.cpp @@ -964,4 +964,40 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "detect_cyclic_typepacks2") LUAU_REQUIRE_ERRORS(result); } +TEST_CASE_FIXTURE(Fixture, "unify_variadic_tails_in_arguments") +{ + ScopedFastFlag luauCallUnifyPackTails{"LuauCallUnifyPackTails", true}; + + CheckResult result = check(R"( + function foo(...: string): number + return 1 + end + + function bar(...: number): number + return foo(...) + end + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ(toString(result.errors[0]), "Type 'number' could not be converted into 'string'"); +} + +TEST_CASE_FIXTURE(Fixture, "unify_variadic_tails_in_arguments_free") +{ + ScopedFastFlag luauCallUnifyPackTails{"LuauCallUnifyPackTails", true}; + + CheckResult result = check(R"( + function foo(...: T...): T... + return ... + end + + function bar(...: number): boolean + return foo(...) + end + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ(toString(result.errors[0]), "Type 'number' could not be converted into 'boolean'"); +} + TEST_SUITE_END(); diff --git a/tools/faillist.txt b/tools/faillist.txt index ef995aa6..9c2df805 100644 --- a/tools/faillist.txt +++ b/tools/faillist.txt @@ -383,8 +383,6 @@ TableTests.casting_sealed_tables_with_props_into_table_with_indexer TableTests.casting_tables_with_props_into_table_with_indexer3 TableTests.casting_tables_with_props_into_table_with_indexer4 TableTests.checked_prop_too_early -TableTests.common_table_element_union_in_call -TableTests.common_table_element_union_in_call_tail TableTests.confusing_indexing TableTests.defining_a_method_for_a_builtin_sealed_table_must_fail TableTests.defining_a_method_for_a_local_sealed_table_must_fail @@ -540,7 +538,6 @@ TypeInfer.type_infer_recursion_limit_no_ice TypeInferAnyError.assign_prop_to_table_by_calling_any_yields_any TypeInferAnyError.can_get_length_of_any TypeInferAnyError.for_in_loop_iterator_is_any2 -TypeInferAnyError.for_in_loop_iterator_returns_any TypeInferAnyError.length_of_error_type_does_not_produce_an_error TypeInferAnyError.replace_every_free_type_when_unifying_a_complex_function_with_any TypeInferAnyError.union_of_types_regression_test @@ -561,7 +558,6 @@ TypeInferClasses.table_indexers_are_invariant TypeInferClasses.table_properties_are_invariant TypeInferClasses.warn_when_prop_almost_matches TypeInferClasses.we_can_report_when_someone_is_trying_to_use_a_table_rather_than_a_class -TypeInferFunctions.another_indirect_function_case_where_it_is_ok_to_provide_too_many_arguments TypeInferFunctions.another_recursive_local_function TypeInferFunctions.call_o_with_another_argument_after_foo_was_quantified TypeInferFunctions.calling_function_with_anytypepack_doesnt_leak_free_types @@ -586,13 +582,14 @@ TypeInferFunctions.function_statement_sealed_table_assignment_through_indexer TypeInferFunctions.higher_order_function_2 TypeInferFunctions.higher_order_function_4 TypeInferFunctions.ignored_return_values +TypeInferFunctions.improved_function_arg_mismatch_error_nonstrict +TypeInferFunctions.improved_function_arg_mismatch_errors TypeInferFunctions.inconsistent_higher_order_function TypeInferFunctions.inconsistent_return_types TypeInferFunctions.infer_anonymous_function_arguments TypeInferFunctions.infer_return_type_from_selected_overload TypeInferFunctions.infer_that_function_does_not_return_a_table TypeInferFunctions.it_is_ok_not_to_supply_enough_retvals -TypeInferFunctions.it_is_ok_to_oversaturate_a_higher_order_function_argument TypeInferFunctions.list_all_overloads_if_no_overload_takes_given_argument_count TypeInferFunctions.list_only_alternative_overloads_that_match_argument_count TypeInferFunctions.no_lossy_function_type From 8b36409c2c9f1b617f1bb66bbc2373edeadd09f3 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Fri, 16 Sep 2022 11:39:30 -0700 Subject: [PATCH 361/399] Update build.yml (#675) Pass codecov token explicitly, for some reason automatic configuration stopped working --- .github/workflows/build.yml | 1 + 1 file changed, 1 insertion(+) diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml index da525ad8..0958747d 100644 --- a/.github/workflows/build.yml +++ b/.github/workflows/build.yml @@ -80,6 +80,7 @@ jobs: uses: codecov/codecov-action@v3 with: files: ./coverage.info + token: ${{ secrets.CODECOV_TOKEN }} web: runs-on: ubuntu-latest From bdf7d8c185fb03afc5d3612ab55acb6d5e6c63ea Mon Sep 17 00:00:00 2001 From: Aaron Weiss Date: Mon, 19 Sep 2022 16:42:20 -0400 Subject: [PATCH 362/399] RFC: Expanded Subtyping for Generic Function Types (#641) [Rendered](https://github.com/aatxe/luau/blob/generic-function-subtyping/rfcs/generic-function-subtyping.md) --- rfcs/generic-function-subtyping.md | 211 +++++++++++++++++++++++++++++ 1 file changed, 211 insertions(+) create mode 100644 rfcs/generic-function-subtyping.md diff --git a/rfcs/generic-function-subtyping.md b/rfcs/generic-function-subtyping.md new file mode 100644 index 00000000..b9c0c430 --- /dev/null +++ b/rfcs/generic-function-subtyping.md @@ -0,0 +1,211 @@ +# Expanded Subtyping for Generic Function Types + +## Summary + +Extend the subtyping relation for function types to relate generic function +types with compatible instantiated function types. + +## Motivation + +As Luau does not have an explicit syntax for instantiation, there are a number +of places where the typechecker will automatically perform instantiation with +the goal of permitting more programs. These instances of instantiation are +ad-hoc and strategic, but useful in practice for permitting programs such as: + +```lua +function id(x: T): T + return x +end + +local idNum : (number) -> number +idNum = id -- ok +``` + +However, they have also been a source of some typechecking bugs because of how +they actually make a determination as to whether the instantation should happen, +and they currently open up some potential soundness holes when instantiating +functions in table types since properties of tables are mutable and thus need to +be invariant (which the automatic-instantiation potentially masks). + +## Design + +The goal then is to rework subtyping to support the relationship we want in the +first place: allowing polymorphic functions to be used where instantiated +functions are expected. In particular, this means adding instantiation itself to +the subtyping relation. Formally, that'd look something like: + +``` +instantiate((T1) -> T2) = (T1') -> T2' +(T1') -> T2' <: (T3) -> T4 +-------------------------------------------- +(T1) -> T2 <: (T3) -> T4) +``` + +Or informally, we'd say that a generic function type is a subtype of another +function type if we can instantiate it and show that instantiated function type +to be a subtype of the original function type. Implementation-wise, this loose +formal rule suggests a strategy of when we'll want to apply instantiation. +Namely, whenever the subtype and supertype are both functions with the potential +subtype having some generic parameters and the supertype having none. So, if we +look once again at our simple example from motivation, we can walk through how +we expect it to type check: + +```lua +function id(x: T): T + return x +end + +local idNum : (number) -> number +idNum = id -- ok +``` + +First, `id` is given the type `(T) -> T` and `idNum` is given the type +`(number) -> number`. When we actually perform the assignment, we must show that +the type of the right-hand side is compatible with the type of the left-hand +side according to subtyping. That is, we'll ask if `(T) -> T` is a subtype of +`(number) -> number` which matches the rule to apply instantiation since the +would-be subtype has a generic parameter while the would-be supertype has no +generic parameters. This contrasts with the current implementation which, before +asking the subtyping question, checks if the type of the right-hand side +contains any generics at any point and if the type of the left-hand side cannot +_possibly_ contain generics and instantiates the right-hand side if so. + +Adding instantiation to subtyping does pose some additional questions still +about when exactly to instantiate. Namely, we need to consider cases like +function application. We can see why by looking at some examples: + +```lua +function rank2(f: (a) -> a): (number) -> number + return f +end +``` + +In this case, we expect to allow the instantiation of `f` from `(a) -> a` to +`(number) -> number`. After all, we can consider other cases like where the body +instead applies `f` to some particular value, e.g. `f(42)`, and we'd want the +instantiation to be allowed there. However, this means we'd potentially run into +issues if we allowed call sites to `rank2` to pass in non-polymorphic functions. +A naive approach to implementing this proposal would do exactly that because we +currently treat contravariant subtyping positions (i.e. for the arguments of +functions) as being the same as our normal (i.e. covariant) subtyping relation +but with the arguments reversed. So, to type check an application like +`rank2(function(str: string) return str + "s" end)` (where the function argument +is of type `(string) -> string`), we would ask if `(a) -> a` is a subtype of +`(string) -> string`. This is precisely the question we asked in the original +example, but in the contravariant context, this is actually unsound since +`rank2` would then function as a general coercion from, e.g., +`(string) -> string` to `(number) -> number`. + +This sort of behavior does come up in other languages that mix polymorphism and +subtyping. If we consider the same example in F#, we can compare its behavior: + +```fsharp +let ranktwo (f : 'a -> 'a) : int -> int = f +let pluralize (s : string) : string = s + "s" +let x = ranktwo pluralize +``` + +For this example, F# produces one warning and one error. The warning is applied +to the function definition of `ranktwo` itself (coded `FS0064`), and says "This +construct causes code to be less generic than indicated by the type annotations. +The type variable 'a has been constrained to be type 'int'." This warning +highlights the actual difference between our example in Luau and the F# +translation. In F#, `'a` is really a free type variable, rather than a generic +type parameter of the function `ranktwo`, as such, this code actually +constrains the type of `ranktwo` to be `(int -> int) -> (int -> int)`. As such, +the application on line 3 errors because our `(string -> string)` function is +simply not compatible with that type. With higher-rank polymorphic function +parameters, it doesn't make sense to warn on their instantiation (as illustrated +by the example of actually applying `f` to some particular data in the +definition of `rank2`), but it's still just as problematic if we were to accept +instantiated functions at polymorphic types. Thus, it's important that we +actually ensure that we only instantiate in covariant contexts. So, we must +ensure that subtyping only instantiates in covariant contexts. + +It may also be helpful to consider an example of rank-1 polymorphism to +understand the full scope of the behavior. So, we can look at what happens if we +simply move the type parameter out in our working example: + +```lua +function rank1(f: (a) -> a): (number) -> number + return f +end +``` + +In this case, we expect an error to occur because the type of `f` depends on +what we instantiate `rank1` with. If we allowed this, it would naturally be +unsound because we could again provide a `(string) -> string` argument (by +instantiating `a` with `string`). This reinforces the idea that the presence of +the generic type parameter is likely to be a good option for determining +instantiation (at least when compared to the presence of free type variables). + +## Drawbacks + +One of the aims of this proposal is to provide a clear and predictable mental +model of when instantiation will take place in Luau. The author feels this +proposal is step forward compared to the existing ad-hoc usage of instantiation +in the typechecker, but it's possible that programmers are already comfortable +with the mental model they have built for the existing implementation. +Hopefully, this is mitigated by the fact that the new setup should allow all of +the _sound_ uses of instantiation permitted by the existing system. Notably, +however, programmers may be surprised by the added restriction when it comes to +properties in tables. In particular, we can consider a small variation of our +original example with identity functions: + +```lua +function id(x: T): T + return x +end + +local poly : { id : (a) -> a } = { id = id } + +local mono : { id : (number) -> number } +mono = poly -- error! +mono.id = id -- also an error! +``` + +In this case, the fact that we're dealing with a _property_ of a table type +means that we're in a context that needs to be invariant (i.e. not allow +subtyping) to avoid unsoundness caused by interactions between mutable +references and polymorphism (see things like the [value +restriction in OCaml][value-restriction] to understand why). In most cases, we +believe programmers will be using functions in tables as an implementation of +methods for objects, so we don't anticipate that they'll actually _want_ to do +the unsound thing here. The accepted RFC for [read-only +properties][read-only-props] gives us a technically-precise solution since +read-only properties would be free to be typechecked as a covariant context +(since they disallow mutation), and thus if the property `id` was marked +read-only, we'd be able to do both of the assignments in the above example. + +## Alternatives + +The main alternatives would likely be keeping the existing solution (and +likely having to tactically fix future bugs where instantiation either happens +too much or not enough), or removing automatic instantiation altogether in favor +of manual instantiation syntax. The former solution (changing nothing) is cheap +now (both in terms of runtime performance and also development cost), but the +existing implementation involves extra walks of both types to make a decision +about whether or not to perform instantiation. To minimize the performance +impact, the functions that perform these questions (`isGeneric` and +`maybeGeneric`) actually do not perform a full walk, and instead try to +strategically look at only enough to make the decision. We already found and +fixed one bug that was caused by these functions being too imprecise against +their spec, but fleshing them out entirely could potentially be a noticeable +performance regression since the decision to potentially instantiate is one that +comes up often. + +Removing automatic instantiation altogether, by contrast, will definitely be +"correct" in that we'll never instantiate in the wrong spot and programmers will +always have the ability to instantiate, but it would be a marked regression on +developer experience since it would increase the annotation burden considerably +and generally runs counter to the overall design strategy of Luau (which focuses +heavily on type inference). It would also require us to actually pick a syntax +for manual instantiation (which we are still open to do in the future if we +maintain an automatic instantiation solution) which is frought with parser +ambiguity issues or requires the introduction of a sigil like Rust's turbofish +for instantiation. Discussion of that syntax is present in the [generic +functions][generic-functions] RFC. + +[value-restriction]: https://stackoverflow.com/questions/22507448/the-value-restriction#22507665 +[read-only-props]: https://github.com/Roblox/luau/blob/master/rfcs/property-readonly.md +[generic-functions]: https://github.com/Roblox/luau/blob/master/rfcs/generic-functions.md From fc4871989a147be7bc817573f09e49361853f999 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Petri=20H=C3=A4kkinen?= Date: Thu, 22 Sep 2022 19:54:03 +0300 Subject: [PATCH 363/399] Add lua_cleartable (#678) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit To my understanding lua_cleartable does not need GC barriers because it's only removing elements and not modifying the stack. But I'm not a GC expert so please correct if I'm wrong. resolves #672 Co-authored-by: Petri Häkkinen --- VM/include/lua.h | 2 ++ VM/src/lapi.cpp | 10 ++++++++++ tests/Conformance.test.cpp | 5 +++++ 3 files changed, 17 insertions(+) diff --git a/VM/include/lua.h b/VM/include/lua.h index 0a3acb4f..5c61f136 100644 --- a/VM/include/lua.h +++ b/VM/include/lua.h @@ -316,6 +316,8 @@ LUA_API void lua_setuserdatadtor(lua_State* L, int tag, void (*dtor)(lua_State*, LUA_API void lua_clonefunction(lua_State* L, int idx); +LUA_API void lua_cleartable(lua_State* L, int idx); + /* ** reference system, can be used to pin objects */ diff --git a/VM/src/lapi.cpp b/VM/src/lapi.cpp index cbcaa3cc..968b8d00 100644 --- a/VM/src/lapi.cpp +++ b/VM/src/lapi.cpp @@ -1376,6 +1376,16 @@ void lua_clonefunction(lua_State* L, int idx) api_incr_top(L); } +void lua_cleartable(lua_State* L, int idx) +{ + StkId t = index2addr(L, idx); + api_check(L, ttistable(t)); + Table* tt = hvalue(t); + if (tt->readonly) + luaG_runerror(L, "Attempt to modify a readonly table"); + luaH_clear(tt); +} + lua_Callbacks* lua_callbacks(lua_State* L) { return &L->global->cb; diff --git a/tests/Conformance.test.cpp b/tests/Conformance.test.cpp index 25129bff..cf4a14c7 100644 --- a/tests/Conformance.test.cpp +++ b/tests/Conformance.test.cpp @@ -778,6 +778,11 @@ TEST_CASE("ApiTables") CHECK(strcmp(lua_tostring(L, -1), "test") == 0); lua_pop(L, 1); + // lua_cleartable + lua_cleartable(L, -1); + lua_pushnil(L); + CHECK(lua_next(L, -2) == 0); + lua_pop(L, 1); } From 59fd9de48598724c7e44bb455bff9e5f5a7df0f0 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Fri, 23 Sep 2022 12:17:25 -0700 Subject: [PATCH 364/399] Sync to upstream/release/546 (#681) --- Analysis/include/Luau/Constraint.h | 43 +- .../include/Luau/ConstraintGraphBuilder.h | 12 +- Analysis/include/Luau/ConstraintSolver.h | 12 + Analysis/include/Luau/Normalize.h | 4 +- Analysis/include/Luau/RequireTracer.h | 1 + Analysis/src/ConstraintGraphBuilder.cpp | 275 +- Analysis/src/ConstraintSolver.cpp | 172 +- Analysis/src/Normalize.cpp | 169 +- Analysis/src/ToString.cpp | 9 + Analysis/src/TypeChecker2.cpp | 25 +- Ast/include/Luau/DenseHash.h | 188 +- Ast/include/Luau/Lexer.h | 2 + Ast/src/Parser.cpp | 137 +- CMakeLists.txt | 5 + CodeGen/include/Luau/UnwindBuilderDwarf2.h | 6 +- CodeGen/src/AssemblyBuilderX64.cpp | 46 +- CodeGen/src/ByteUtils.h | 78 + CodeGen/src/Fallbacks.cpp | 2511 +++++++++++++++++ CodeGen/src/Fallbacks.h | 93 + CodeGen/src/FallbacksProlog.h | 56 + CodeGen/src/UnwindBuilderDwarf2.cpp | 52 +- Compiler/src/ConstantFolding.cpp | 1 + Makefile | 2 +- Sources.cmake | 5 + VM/include/lua.h | 1 + VM/include/luaconf.h | 5 + VM/src/lapi.cpp | 56 +- VM/src/ldebug.cpp | 5 + VM/src/lfunc.cpp | 14 + VM/src/lobject.h | 4 + VM/src/lstate.cpp | 5 + VM/src/lstate.h | 17 + VM/src/lvm.h | 3 + VM/src/lvmexecute.cpp | 153 +- VM/src/lvmutils.cpp | 78 + tests/Compiler.test.cpp | 69 +- tests/Conformance.test.cpp | 38 + tests/Linter.test.cpp | 3 - tests/Normalize.test.cpp | 4 +- tests/Parser.test.cpp | 2 - tests/TypeInfer.aliases.test.cpp | 14 +- tests/TypeInfer.provisional.test.cpp | 6 +- tests/TypeInfer.singletons.test.cpp | 25 + tests/TypeInfer.test.cpp | 39 +- tools/faillist.txt | 122 +- tools/lvmexecute_split.py | 100 + 46 files changed, 3942 insertions(+), 725 deletions(-) create mode 100644 CodeGen/src/ByteUtils.h create mode 100644 CodeGen/src/Fallbacks.cpp create mode 100644 CodeGen/src/Fallbacks.h create mode 100644 CodeGen/src/FallbacksProlog.h create mode 100644 tools/lvmexecute_split.py diff --git a/Analysis/include/Luau/Constraint.h b/Analysis/include/Luau/Constraint.h index d5cfcf3f..3ffb3fb8 100644 --- a/Analysis/include/Luau/Constraint.h +++ b/Analysis/include/Luau/Constraint.h @@ -90,18 +90,49 @@ struct TypeAliasExpansionConstraint TypeId target; }; -using ConstraintPtr = std::unique_ptr; - struct FunctionCallConstraint { - std::vector> innerConstraints; + std::vector> innerConstraints; TypeId fn; TypePackId result; class AstExprCall* astFragment; }; -using ConstraintV = Variant; +// result ~ prim ExpectedType SomeSingletonType MultitonType +// +// If ExpectedType is potentially a singleton (an actual singleton or a union +// that contains a singleton), then result ~ SomeSingletonType +// +// else result ~ MultitonType +struct PrimitiveTypeConstraint +{ + TypeId resultType; + TypeId expectedType; + TypeId singletonType; + TypeId multitonType; +}; + +// result ~ hasProp type "prop_name" +// +// If the subject is a table, bind the result to the named prop. If the table +// has an indexer, bind it to the index result type. If the subject is a union, +// bind the result to the union of its constituents' properties. +// +// It would be nice to get rid of this constraint and someday replace it with +// +// T <: {p: X} +// +// Where {} describes an inexact shape type. +struct HasPropConstraint +{ + TypeId resultType; + TypeId subjectType; + std::string prop; +}; + +using ConstraintV = + Variant; struct Constraint { @@ -117,6 +148,8 @@ struct Constraint std::vector> dependencies; }; +using ConstraintPtr = std::unique_ptr; + inline Constraint& asMutable(const Constraint& c) { return const_cast(c); diff --git a/Analysis/include/Luau/ConstraintGraphBuilder.h b/Analysis/include/Luau/ConstraintGraphBuilder.h index 1567e0ad..e7d8ad45 100644 --- a/Analysis/include/Luau/ConstraintGraphBuilder.h +++ b/Analysis/include/Luau/ConstraintGraphBuilder.h @@ -125,23 +125,25 @@ struct ConstraintGraphBuilder void visit(const ScopePtr& scope, AstStatDeclareClass* declareClass); void visit(const ScopePtr& scope, AstStatDeclareFunction* declareFunction); - TypePackId checkPack(const ScopePtr& scope, AstArray exprs); - TypePackId checkPack(const ScopePtr& scope, AstExpr* expr); + TypePackId checkPack(const ScopePtr& scope, AstArray exprs, const std::vector& expectedTypes = {}); + TypePackId checkPack(const ScopePtr& scope, AstExpr* expr, const std::vector& expectedTypes = {}); /** * Checks an expression that is expected to evaluate to one type. * @param scope the scope the expression is contained within. * @param expr the expression to check. + * @param expectedType the type of the expression that is expected from its + * surrounding context. Used to implement bidirectional type checking. * @return the type of the expression. */ - TypeId check(const ScopePtr& scope, AstExpr* expr); + TypeId check(const ScopePtr& scope, AstExpr* expr, std::optional expectedType = {}); - TypeId checkExprTable(const ScopePtr& scope, AstExprTable* expr); + TypeId check(const ScopePtr& scope, AstExprTable* expr, std::optional expectedType); TypeId check(const ScopePtr& scope, AstExprIndexName* indexName); TypeId check(const ScopePtr& scope, AstExprIndexExpr* indexExpr); TypeId check(const ScopePtr& scope, AstExprUnary* unary); TypeId check(const ScopePtr& scope, AstExprBinary* binary); - TypeId check(const ScopePtr& scope, AstExprIfElse* ifElse); + TypeId check(const ScopePtr& scope, AstExprIfElse* ifElse, std::optional expectedType); TypeId check(const ScopePtr& scope, AstExprTypeAssertion* typeAssert); struct FunctionSignature diff --git a/Analysis/include/Luau/ConstraintSolver.h b/Analysis/include/Luau/ConstraintSolver.h index fe6a025b..abea51b8 100644 --- a/Analysis/include/Luau/ConstraintSolver.h +++ b/Analysis/include/Luau/ConstraintSolver.h @@ -100,6 +100,8 @@ struct ConstraintSolver bool tryDispatch(const NameConstraint& c, NotNull constraint); bool tryDispatch(const TypeAliasExpansionConstraint& c, NotNull constraint); bool tryDispatch(const FunctionCallConstraint& c, NotNull constraint); + bool tryDispatch(const PrimitiveTypeConstraint& c, NotNull constraint); + bool tryDispatch(const HasPropConstraint& c, NotNull constraint); // for a, ... in some_table do bool tryDispatchIterableTable(TypeId iteratorTy, const IterableConstraint& c, NotNull constraint, bool force); @@ -116,6 +118,16 @@ struct ConstraintSolver bool block(TypeId target, NotNull constraint); bool block(TypePackId target, NotNull constraint); + // Traverse the type. If any blocked or pending typevars are found, block + // the constraint on them. + // + // Returns false if a type blocks the constraint. + // + // FIXME: This use of a boolean for the return result is an appalling + // interface. + bool recursiveBlock(TypeId target, NotNull constraint); + bool recursiveBlock(TypePackId target, NotNull constraint); + void unblock(NotNull progressed); void unblock(TypeId progressed); void unblock(TypePackId progressed); diff --git a/Analysis/include/Luau/Normalize.h b/Analysis/include/Luau/Normalize.h index 48dbe2be..8e8b889b 100644 --- a/Analysis/include/Luau/Normalize.h +++ b/Analysis/include/Luau/Normalize.h @@ -17,8 +17,8 @@ struct SingletonTypes; using ModulePtr = std::shared_ptr; -bool isSubtype(TypeId subTy, TypeId superTy, NotNull scope, NotNull singletonTypes, InternalErrorReporter& ice); -bool isSubtype(TypePackId subTy, TypePackId superTy, NotNull scope, NotNull singletonTypes, InternalErrorReporter& ice); +bool isSubtype(TypeId subTy, TypeId superTy, NotNull scope, NotNull singletonTypes, InternalErrorReporter& ice, bool anyIsTop = true); +bool isSubtype(TypePackId subTy, TypePackId superTy, NotNull scope, NotNull singletonTypes, InternalErrorReporter& ice, bool anyIsTop = true); std::pair normalize( TypeId ty, NotNull scope, TypeArena& arena, NotNull singletonTypes, InternalErrorReporter& ice); diff --git a/Analysis/include/Luau/RequireTracer.h b/Analysis/include/Luau/RequireTracer.h index f69d133e..718a6cc1 100644 --- a/Analysis/include/Luau/RequireTracer.h +++ b/Analysis/include/Luau/RequireTracer.h @@ -6,6 +6,7 @@ #include "Luau/Location.h" #include +#include namespace Luau { diff --git a/Analysis/src/ConstraintGraphBuilder.cpp b/Analysis/src/ConstraintGraphBuilder.cpp index 6a65ab92..aa1e9547 100644 --- a/Analysis/src/ConstraintGraphBuilder.cpp +++ b/Analysis/src/ConstraintGraphBuilder.cpp @@ -36,6 +36,20 @@ static std::optional matchRequire(const AstExprCall& call) return call.args.data[0]; } +static bool matchSetmetatable(const AstExprCall& call) +{ + const char* smt = "setmetatable"; + + if (call.args.size != 2) + return false; + + const AstExprGlobal* funcAsGlobal = call.func->as(); + if (!funcAsGlobal || funcAsGlobal->name != smt) + return false; + + return true; +} + ConstraintGraphBuilder::ConstraintGraphBuilder(const ModuleName& moduleName, ModulePtr module, TypeArena* arena, NotNull moduleResolver, NotNull singletonTypes, NotNull ice, const ScopePtr& globalScope, DcrLogger* logger) : moduleName(moduleName) @@ -214,15 +228,16 @@ void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatLocal* local) for (AstLocal* local : local->vars) { - TypeId ty = freshType(scope); + TypeId ty = nullptr; Location location = local->location; if (local->annotation) { location = local->annotation->location; - TypeId annotation = resolveType(scope, local->annotation, /* topLevel */ true); - addConstraint(scope, location, SubtypeConstraint{ty, annotation}); + ty = resolveType(scope, local->annotation, /* topLevel */ true); } + else + ty = freshType(scope); varTypes.push_back(ty); scope->bindings[local] = Binding{ty, location}; @@ -231,6 +246,8 @@ void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatLocal* local) for (size_t i = 0; i < local->values.size; ++i) { AstExpr* value = local->values.data[i]; + const bool hasAnnotation = i < local->vars.size && nullptr != local->vars.data[i]->annotation; + if (value->is()) { // HACK: we leave nil-initialized things floating under the assumption that they will later be populated. @@ -239,7 +256,11 @@ void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatLocal* local) } else if (i == local->values.size - 1) { - TypePackId exprPack = checkPack(scope, value); + std::vector expectedTypes; + if (hasAnnotation) + expectedTypes.insert(begin(expectedTypes), begin(varTypes) + i, end(varTypes)); + + TypePackId exprPack = checkPack(scope, value, expectedTypes); if (i < local->vars.size) { @@ -250,7 +271,11 @@ void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatLocal* local) } else { - TypeId exprType = check(scope, value); + std::optional expectedType; + if (hasAnnotation) + expectedType = varTypes.at(i); + + TypeId exprType = check(scope, value, expectedType); if (i < varTypes.size()) addConstraint(scope, local->location, SubtypeConstraint{varTypes[i], exprType}); } @@ -458,7 +483,15 @@ void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatFunction* funct void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatReturn* ret) { - TypePackId exprTypes = checkPack(scope, ret->list); + // At this point, the only way scope->returnType should have anything + // interesting in it is if the function has an explicit return annotation. + // If this is the case, then we can expect that the return expression + // conforms to that. + std::vector expectedTypes; + for (TypeId ty : scope->returnType) + expectedTypes.push_back(ty); + + TypePackId exprTypes = checkPack(scope, ret->list, expectedTypes); addConstraint(scope, ret->location, PackSubtypeConstraint{exprTypes, scope->returnType}); } @@ -695,7 +728,7 @@ void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatDeclareFunction scope->bindings[global->name] = Binding{fnType, global->location}; } -TypePackId ConstraintGraphBuilder::checkPack(const ScopePtr& scope, AstArray exprs) +TypePackId ConstraintGraphBuilder::checkPack(const ScopePtr& scope, AstArray exprs, const std::vector& expectedTypes) { std::vector head; std::optional tail; @@ -704,9 +737,17 @@ TypePackId ConstraintGraphBuilder::checkPack(const ScopePtr& scope, AstArray expectedType; + if (i < expectedTypes.size()) + expectedType = expectedTypes[i]; head.push_back(check(scope, expr)); + } else - tail = checkPack(scope, expr); + { + std::vector expectedTailTypes{begin(expectedTypes) + i, end(expectedTypes)}; + tail = checkPack(scope, expr, expectedTailTypes); + } } if (head.empty() && tail) @@ -715,7 +756,7 @@ TypePackId ConstraintGraphBuilder::checkPack(const ScopePtr& scope, AstArrayaddTypePack(TypePack{std::move(head), tail}); } -TypePackId ConstraintGraphBuilder::checkPack(const ScopePtr& scope, AstExpr* expr) +TypePackId ConstraintGraphBuilder::checkPack(const ScopePtr& scope, AstExpr* expr, const std::vector& expectedTypes) { RecursionCounter counter{&recursionCount}; @@ -730,7 +771,6 @@ TypePackId ConstraintGraphBuilder::checkPack(const ScopePtr& scope, AstExpr* exp if (AstExprCall* call = expr->as()) { TypeId fnType = check(scope, call->func); - const size_t constraintIndex = scope->constraints.size(); const size_t scopeIndex = scopes.size(); @@ -743,49 +783,63 @@ TypePackId ConstraintGraphBuilder::checkPack(const ScopePtr& scope, AstExpr* exp // TODO self - const size_t constraintEndIndex = scope->constraints.size(); - const size_t scopeEndIndex = scopes.size(); - - astOriginalCallTypes[call->func] = fnType; - - TypeId instantiatedType = arena->addType(BlockedTypeVar{}); - TypePackId rets = arena->addTypePack(BlockedTypePack{}); - FunctionTypeVar ftv(arena->addTypePack(TypePack{args, {}}), rets); - TypeId inferredFnType = arena->addType(ftv); - - scope->unqueuedConstraints.push_back( - std::make_unique(NotNull{scope.get()}, call->func->location, InstantiationConstraint{instantiatedType, fnType})); - NotNull ic(scope->unqueuedConstraints.back().get()); - - scope->unqueuedConstraints.push_back( - std::make_unique(NotNull{scope.get()}, call->func->location, SubtypeConstraint{inferredFnType, instantiatedType})); - NotNull sc(scope->unqueuedConstraints.back().get()); - - // We force constraints produced by checking function arguments to wait - // until after we have resolved the constraint on the function itself. - // This ensures, for instance, that we start inferring the contents of - // lambdas under the assumption that their arguments and return types - // will be compatible with the enclosing function call. - for (size_t ci = constraintIndex; ci < constraintEndIndex; ++ci) - scope->constraints[ci]->dependencies.push_back(sc); - - for (size_t si = scopeIndex; si < scopeEndIndex; ++si) + if (matchSetmetatable(*call)) { - for (auto& c : scopes[si].second->constraints) - { - c->dependencies.push_back(sc); - } + LUAU_ASSERT(args.size() == 2); + TypeId target = args[0]; + TypeId mt = args[1]; + + MetatableTypeVar mtv{target, mt}; + TypeId resultTy = arena->addType(mtv); + result = arena->addTypePack({resultTy}); } + else + { + const size_t constraintEndIndex = scope->constraints.size(); + const size_t scopeEndIndex = scopes.size(); - addConstraint(scope, call->func->location, - FunctionCallConstraint{ - {ic, sc}, - fnType, - rets, - call, - }); + astOriginalCallTypes[call->func] = fnType; - result = rets; + TypeId instantiatedType = arena->addType(BlockedTypeVar{}); + // TODO: How do expectedTypes play into this? Do they? + TypePackId rets = arena->addTypePack(BlockedTypePack{}); + FunctionTypeVar ftv(arena->addTypePack(TypePack{args, {}}), rets); + TypeId inferredFnType = arena->addType(ftv); + + scope->unqueuedConstraints.push_back( + std::make_unique(NotNull{scope.get()}, call->func->location, InstantiationConstraint{instantiatedType, fnType})); + NotNull ic(scope->unqueuedConstraints.back().get()); + + scope->unqueuedConstraints.push_back( + std::make_unique(NotNull{scope.get()}, call->func->location, SubtypeConstraint{inferredFnType, instantiatedType})); + NotNull sc(scope->unqueuedConstraints.back().get()); + + // We force constraints produced by checking function arguments to wait + // until after we have resolved the constraint on the function itself. + // This ensures, for instance, that we start inferring the contents of + // lambdas under the assumption that their arguments and return types + // will be compatible with the enclosing function call. + for (size_t ci = constraintIndex; ci < constraintEndIndex; ++ci) + scope->constraints[ci]->dependencies.push_back(sc); + + for (size_t si = scopeIndex; si < scopeEndIndex; ++si) + { + for (auto& c : scopes[si].second->constraints) + { + c->dependencies.push_back(sc); + } + } + + addConstraint(scope, call->func->location, + FunctionCallConstraint{ + {ic, sc}, + fnType, + rets, + call, + }); + + result = rets; + } } else if (AstExprVarargs* varargs = expr->as()) { @@ -796,7 +850,10 @@ TypePackId ConstraintGraphBuilder::checkPack(const ScopePtr& scope, AstExpr* exp } else { - TypeId t = check(scope, expr); + std::optional expectedType; + if (!expectedTypes.empty()) + expectedType = expectedTypes[0]; + TypeId t = check(scope, expr, expectedType); result = arena->addTypePack({t}); } @@ -805,7 +862,7 @@ TypePackId ConstraintGraphBuilder::checkPack(const ScopePtr& scope, AstExpr* exp return result; } -TypeId ConstraintGraphBuilder::check(const ScopePtr& scope, AstExpr* expr) +TypeId ConstraintGraphBuilder::check(const ScopePtr& scope, AstExpr* expr, std::optional expectedType) { RecursionCounter counter{&recursionCount}; @@ -819,12 +876,47 @@ TypeId ConstraintGraphBuilder::check(const ScopePtr& scope, AstExpr* expr) if (auto group = expr->as()) result = check(scope, group->expr); - else if (expr->is()) - result = singletonTypes->stringType; + else if (auto stringExpr = expr->as()) + { + if (expectedType) + { + const TypeId expectedTy = follow(*expectedType); + if (get(expectedTy) || get(expectedTy)) + { + result = arena->addType(BlockedTypeVar{}); + TypeId singletonType = arena->addType(SingletonTypeVar(StringSingleton{std::string(stringExpr->value.data, stringExpr->value.size)})); + addConstraint(scope, expr->location, PrimitiveTypeConstraint{result, expectedTy, singletonType, singletonTypes->stringType}); + } + else if (maybeSingleton(expectedTy)) + result = arena->addType(SingletonTypeVar{StringSingleton{std::string{stringExpr->value.data, stringExpr->value.size}}}); + else + result = singletonTypes->stringType; + } + else + result = singletonTypes->stringType; + } else if (expr->is()) result = singletonTypes->numberType; - else if (expr->is()) - result = singletonTypes->booleanType; + else if (auto boolExpr = expr->as()) + { + if (expectedType) + { + const TypeId expectedTy = follow(*expectedType); + const TypeId singletonType = boolExpr->value ? singletonTypes->trueType : singletonTypes->falseType; + + if (get(expectedTy) || get(expectedTy)) + { + result = arena->addType(BlockedTypeVar{}); + addConstraint(scope, expr->location, PrimitiveTypeConstraint{result, expectedTy, singletonType, singletonTypes->booleanType}); + } + else if (maybeSingleton(expectedTy)) + result = singletonType; + else + result = singletonTypes->booleanType; + } + else + result = singletonTypes->booleanType; + } else if (expr->is()) result = singletonTypes->nilType; else if (auto a = expr->as()) @@ -864,13 +956,13 @@ TypeId ConstraintGraphBuilder::check(const ScopePtr& scope, AstExpr* expr) else if (auto indexExpr = expr->as()) result = check(scope, indexExpr); else if (auto table = expr->as()) - result = checkExprTable(scope, table); + result = check(scope, table, expectedType); else if (auto unary = expr->as()) result = check(scope, unary); else if (auto binary = expr->as()) result = check(scope, binary); else if (auto ifElse = expr->as()) - result = check(scope, ifElse); + result = check(scope, ifElse, expectedType); else if (auto typeAssert = expr->as()) result = check(scope, typeAssert); else if (auto err = expr->as()) @@ -924,20 +1016,9 @@ TypeId ConstraintGraphBuilder::check(const ScopePtr& scope, AstExprUnary* unary) { TypeId operandType = check(scope, unary->expr); - switch (unary->op) - { - case AstExprUnary::Minus: - { - TypeId resultType = arena->addType(BlockedTypeVar{}); - addConstraint(scope, unary->location, UnaryConstraint{AstExprUnary::Minus, operandType, resultType}); - return resultType; - } - default: - LUAU_ASSERT(0); - } - - LUAU_UNREACHABLE(); - return singletonTypes->errorRecoveryType(); + TypeId resultType = arena->addType(BlockedTypeVar{}); + addConstraint(scope, unary->location, UnaryConstraint{unary->op, operandType, resultType}); + return resultType; } TypeId ConstraintGraphBuilder::check(const ScopePtr& scope, AstExprBinary* binary) @@ -946,22 +1027,34 @@ TypeId ConstraintGraphBuilder::check(const ScopePtr& scope, AstExprBinary* binar TypeId rightType = check(scope, binary->right); switch (binary->op) { + case AstExprBinary::And: case AstExprBinary::Or: { addConstraint(scope, binary->location, SubtypeConstraint{leftType, rightType}); return leftType; } case AstExprBinary::Add: + case AstExprBinary::Sub: + case AstExprBinary::Mul: + case AstExprBinary::Div: + case AstExprBinary::Mod: + case AstExprBinary::Pow: + case AstExprBinary::CompareNe: + case AstExprBinary::CompareEq: + case AstExprBinary::CompareLt: + case AstExprBinary::CompareLe: + case AstExprBinary::CompareGt: + case AstExprBinary::CompareGe: { TypeId resultType = arena->addType(BlockedTypeVar{}); - addConstraint(scope, binary->location, BinaryConstraint{AstExprBinary::Add, leftType, rightType, resultType}); + addConstraint(scope, binary->location, BinaryConstraint{binary->op, leftType, rightType, resultType}); return resultType; } - case AstExprBinary::Sub: + case AstExprBinary::Concat: { - TypeId resultType = arena->addType(BlockedTypeVar{}); - addConstraint(scope, binary->location, BinaryConstraint{AstExprBinary::Sub, leftType, rightType, resultType}); - return resultType; + addConstraint(scope, binary->left->location, SubtypeConstraint{leftType, singletonTypes->stringType}); + addConstraint(scope, binary->right->location, SubtypeConstraint{rightType, singletonTypes->stringType}); + return singletonTypes->stringType; } default: LUAU_ASSERT(0); @@ -971,16 +1064,16 @@ TypeId ConstraintGraphBuilder::check(const ScopePtr& scope, AstExprBinary* binar return nullptr; } -TypeId ConstraintGraphBuilder::check(const ScopePtr& scope, AstExprIfElse* ifElse) +TypeId ConstraintGraphBuilder::check(const ScopePtr& scope, AstExprIfElse* ifElse, std::optional expectedType) { check(scope, ifElse->condition); - TypeId thenType = check(scope, ifElse->trueExpr); - TypeId elseType = check(scope, ifElse->falseExpr); + TypeId thenType = check(scope, ifElse->trueExpr, expectedType); + TypeId elseType = check(scope, ifElse->falseExpr, expectedType); if (ifElse->hasElse) { - TypeId resultType = arena->addType(BlockedTypeVar{}); + TypeId resultType = expectedType ? *expectedType : freshType(scope); addConstraint(scope, ifElse->trueExpr->location, SubtypeConstraint{thenType, resultType}); addConstraint(scope, ifElse->falseExpr->location, SubtypeConstraint{elseType, resultType}); return resultType; @@ -995,7 +1088,7 @@ TypeId ConstraintGraphBuilder::check(const ScopePtr& scope, AstExprTypeAssertion return resolveType(scope, typeAssert->annotation); } -TypeId ConstraintGraphBuilder::checkExprTable(const ScopePtr& scope, AstExprTable* expr) +TypeId ConstraintGraphBuilder::check(const ScopePtr& scope, AstExprTable* expr, std::optional expectedType) { TypeId ty = arena->addType(TableTypeVar{}); TableTypeVar* ttv = getMutable(ty); @@ -1015,7 +1108,18 @@ TypeId ConstraintGraphBuilder::checkExprTable(const ScopePtr& scope, AstExprTabl for (const AstExprTable::Item& item : expr->items) { - TypeId itemTy = check(scope, item.value); + std::optional expectedValueType; + + if (item.key && expectedType) + { + if (auto stringKey = item.key->as()) + { + expectedValueType = arena->addType(BlockedTypeVar{}); + addConstraint(scope, item.value->location, HasPropConstraint{*expectedValueType, *expectedType, stringKey->value.data}); + } + } + + TypeId itemTy = check(scope, item.value, expectedValueType); if (get(follow(itemTy))) return ty; @@ -1130,7 +1234,12 @@ ConstraintGraphBuilder::FunctionSignature ConstraintGraphBuilder::checkFunctionS if (fn->returnAnnotation) { TypePackId annotatedRetType = resolveTypePack(signatureScope, *fn->returnAnnotation); - addConstraint(signatureScope, getLocation(*fn->returnAnnotation), PackSubtypeConstraint{returnType, annotatedRetType}); + + // We bind the annotated type directly here so that, when we need to + // generate constraints for return types, we have a guarantee that we + // know the annotated return type already, if one was provided. + LUAU_ASSERT(get(returnType)); + asMutable(returnType)->ty.emplace(annotatedRetType); } std::vector argTypes; diff --git a/Analysis/src/ConstraintSolver.cpp b/Analysis/src/ConstraintSolver.cpp index 6fb57b15..b2bf773f 100644 --- a/Analysis/src/ConstraintSolver.cpp +++ b/Analysis/src/ConstraintSolver.cpp @@ -396,8 +396,12 @@ bool ConstraintSolver::tryDispatch(NotNull constraint, bool fo success = tryDispatch(*taec, constraint); else if (auto fcc = get(*constraint)) success = tryDispatch(*fcc, constraint); + else if (auto fcc = get(*constraint)) + success = tryDispatch(*fcc, constraint); + else if (auto hpc = get(*constraint)) + success = tryDispatch(*hpc, constraint); else - LUAU_ASSERT(0); + LUAU_ASSERT(false); if (success) { @@ -409,6 +413,11 @@ bool ConstraintSolver::tryDispatch(NotNull constraint, bool fo bool ConstraintSolver::tryDispatch(const SubtypeConstraint& c, NotNull constraint, bool force) { + if (!recursiveBlock(c.subType, constraint)) + return false; + if (!recursiveBlock(c.superType, constraint)) + return false; + if (isBlocked(c.subType)) return block(c.subType, constraint); else if (isBlocked(c.superType)) @@ -421,6 +430,9 @@ bool ConstraintSolver::tryDispatch(const SubtypeConstraint& c, NotNull constraint, bool force) { + if (!recursiveBlock(c.subPack, constraint) || !recursiveBlock(c.superPack, constraint)) + return false; + if (isBlocked(c.subPack)) return block(c.subPack, constraint); else if (isBlocked(c.superPack)) @@ -480,13 +492,30 @@ bool ConstraintSolver::tryDispatch(const UnaryConstraint& c, NotNull(c.resultType)); - if (isNumber(operandType) || get(operandType) || get(operandType)) + switch (c.op) { - asMutable(c.resultType)->ty.emplace(c.operandType); - return true; + case AstExprUnary::Not: + { + asMutable(c.resultType)->ty.emplace(singletonTypes->booleanType); + return true; + } + case AstExprUnary::Len: + { + asMutable(c.resultType)->ty.emplace(singletonTypes->numberType); + return true; + } + case AstExprUnary::Minus: + { + if (isNumber(operandType) || get(operandType) || get(operandType)) + { + asMutable(c.resultType)->ty.emplace(c.operandType); + return true; + } + break; + } } - LUAU_ASSERT(0); // TODO metatable handling + LUAU_ASSERT(false); // TODO metatable handling return false; } @@ -906,6 +935,91 @@ bool ConstraintSolver::tryDispatch(const FunctionCallConstraint& c, NotNull constraint) +{ + TypeId expectedType = follow(c.expectedType); + if (isBlocked(expectedType) || get(expectedType)) + return block(expectedType, constraint); + + TypeId bindTo = maybeSingleton(expectedType) ? c.singletonType : c.multitonType; + asMutable(c.resultType)->ty.emplace(bindTo); + + return true; +} + +bool ConstraintSolver::tryDispatch(const HasPropConstraint& c, NotNull constraint) +{ + TypeId subjectType = follow(c.subjectType); + + if (isBlocked(subjectType) || get(subjectType)) + return block(subjectType, constraint); + + TypeId resultType = nullptr; + + auto collectParts = [&](auto&& unionOrIntersection) -> std::pair> { + bool blocked = false; + + std::vector parts; + for (TypeId expectedPart : unionOrIntersection) + { + expectedPart = follow(expectedPart); + if (isBlocked(expectedPart) || get(expectedPart)) + { + blocked = true; + block(expectedPart, constraint); + } + else if (const TableTypeVar* ttv = get(follow(expectedPart))) + { + if (auto prop = ttv->props.find(c.prop); prop != ttv->props.end()) + parts.push_back(prop->second.type); + else if (ttv->indexer && maybeString(ttv->indexer->indexType)) + parts.push_back(ttv->indexer->indexResultType); + } + } + + return {blocked, parts}; + }; + + if (auto ttv = get(subjectType)) + { + if (auto prop = ttv->props.find(c.prop); prop != ttv->props.end()) + resultType = prop->second.type; + else if (ttv->indexer && maybeString(ttv->indexer->indexType)) + resultType = ttv->indexer->indexResultType; + } + else if (auto utv = get(subjectType)) + { + auto [blocked, parts] = collectParts(utv); + + if (blocked) + return false; + else if (parts.size() == 1) + resultType = parts[0]; + else if (parts.size() > 1) + resultType = arena->addType(UnionTypeVar{std::move(parts)}); + else + LUAU_ASSERT(false); // parts.size() == 0 + } + else if (auto itv = get(subjectType)) + { + auto [blocked, parts] = collectParts(itv); + + if (blocked) + return false; + else if (parts.size() == 1) + resultType = parts[0]; + else if (parts.size() > 1) + resultType = arena->addType(IntersectionTypeVar{std::move(parts)}); + else + LUAU_ASSERT(false); // parts.size() == 0 + } + + if (resultType) + asMutable(c.resultType)->ty.emplace(resultType); + + return true; +} + bool ConstraintSolver::tryDispatchIterableTable(TypeId iteratorTy, const IterableConstraint& c, NotNull constraint, bool force) { auto block_ = [&](auto&& t) { @@ -914,7 +1028,7 @@ bool ConstraintSolver::tryDispatchIterableTable(TypeId iteratorTy, const Iterabl // TODO: I believe it is the case that, if we are asked to force // this constraint, then we can do nothing but fail. I'd like to // find a code sample that gets here. - LUAU_ASSERT(0); + LUAU_ASSERT(false); } else block(t, constraint); @@ -979,7 +1093,7 @@ bool ConstraintSolver::tryDispatchIterableTable(TypeId iteratorTy, const Iterabl if (get(metaTy)) return block_(metaTy); - LUAU_ASSERT(0); + LUAU_ASSERT(false); } else errorify(c.variables); @@ -996,7 +1110,7 @@ bool ConstraintSolver::tryDispatchIterableFunction( if (get(firstIndexTy)) { if (force) - LUAU_ASSERT(0); + LUAU_ASSERT(false); else block(firstIndexTy, constraint); return false; @@ -1061,6 +1175,48 @@ bool ConstraintSolver::block(TypePackId target, NotNull constr return false; } +struct Blocker : TypeVarOnceVisitor +{ + NotNull solver; + NotNull constraint; + + bool blocked = false; + + explicit Blocker(NotNull solver, NotNull constraint) + : solver(solver) + , constraint(constraint) + { + } + + bool visit(TypeId ty, const BlockedTypeVar&) + { + blocked = true; + solver->block(ty, constraint); + return false; + } + + bool visit(TypeId ty, const PendingExpansionTypeVar&) + { + blocked = true; + solver->block(ty, constraint); + return false; + } +}; + +bool ConstraintSolver::recursiveBlock(TypeId target, NotNull constraint) +{ + Blocker blocker{NotNull{this}, constraint}; + blocker.traverse(target); + return !blocker.blocked; +} + +bool ConstraintSolver::recursiveBlock(TypePackId pack, NotNull constraint) +{ + Blocker blocker{NotNull{this}, constraint}; + blocker.traverse(pack); + return !blocker.blocked; +} + void ConstraintSolver::unblock_(BlockedConstraintId progressed) { auto it = blocked.find(progressed); diff --git a/Analysis/src/Normalize.cpp b/Analysis/src/Normalize.cpp index c3f0bb9d..42f61517 100644 --- a/Analysis/src/Normalize.cpp +++ b/Analysis/src/Normalize.cpp @@ -5,6 +5,7 @@ #include #include "Luau/Clone.h" +#include "Luau/Common.h" #include "Luau/Unifier.h" #include "Luau/VisitTypeVar.h" @@ -13,8 +14,8 @@ LUAU_FASTFLAGVARIABLE(DebugLuauCopyBeforeNormalizing, false) // This could theoretically be 2000 on amd64, but x86 requires this. LUAU_FASTINTVARIABLE(LuauNormalizeIterationLimit, 1200); LUAU_FASTFLAGVARIABLE(LuauNormalizeCombineTableFix, false); -LUAU_FASTFLAGVARIABLE(LuauFixNormalizationOfCyclicUnions, false); LUAU_FASTFLAG(LuauUnknownAndNeverType) +LUAU_FASTFLAG(DebugLuauDeferredConstraintResolution) namespace Luau { @@ -54,24 +55,24 @@ struct Replacer } // anonymous namespace -bool isSubtype(TypeId subTy, TypeId superTy, NotNull scope, NotNull singletonTypes, InternalErrorReporter& ice) +bool isSubtype(TypeId subTy, TypeId superTy, NotNull scope, NotNull singletonTypes, InternalErrorReporter& ice, bool anyIsTop) { UnifierSharedState sharedState{&ice}; TypeArena arena; Unifier u{&arena, singletonTypes, Mode::Strict, scope, Location{}, Covariant, sharedState}; - u.anyIsTop = true; + u.anyIsTop = anyIsTop; u.tryUnify(subTy, superTy); const bool ok = u.errors.empty() && u.log.empty(); return ok; } -bool isSubtype(TypePackId subPack, TypePackId superPack, NotNull scope, NotNull singletonTypes, InternalErrorReporter& ice) +bool isSubtype(TypePackId subPack, TypePackId superPack, NotNull scope, NotNull singletonTypes, InternalErrorReporter& ice, bool anyIsTop) { UnifierSharedState sharedState{&ice}; TypeArena arena; Unifier u{&arena, singletonTypes, Mode::Strict, scope, Location{}, Covariant, sharedState}; - u.anyIsTop = true; + u.anyIsTop = anyIsTop; u.tryUnify(subPack, superPack); const bool ok = u.errors.empty() && u.log.empty(); @@ -319,18 +320,11 @@ struct Normalize final : TypeVarVisitor UnionTypeVar* utv = &const_cast(utvRef); - // TODO: Clip tempOptions and optionsRef when clipping FFlag::LuauFixNormalizationOfCyclicUnions - std::vector tempOptions; - if (!FFlag::LuauFixNormalizationOfCyclicUnions) - tempOptions = std::move(utv->options); - - std::vector& optionsRef = FFlag::LuauFixNormalizationOfCyclicUnions ? utv->options : tempOptions; - // We might transmute, so it's not safe to rely on the builtin traversal logic of visitTypeVar - for (TypeId option : optionsRef) + for (TypeId option : utv->options) traverse(option); - std::vector newOptions = normalizeUnion(optionsRef); + std::vector newOptions = normalizeUnion(utv->options); const bool normal = areNormal(newOptions, seen, ice); @@ -355,106 +349,54 @@ struct Normalize final : TypeVarVisitor IntersectionTypeVar* itv = &const_cast(itvRef); - if (FFlag::LuauFixNormalizationOfCyclicUnions) + std::vector oldParts = itv->parts; + IntersectionTypeVar newIntersection; + + for (TypeId part : oldParts) + traverse(part); + + std::vector tables; + for (TypeId part : oldParts) { - std::vector oldParts = itv->parts; - IntersectionTypeVar newIntersection; - - for (TypeId part : oldParts) - traverse(part); - - std::vector tables; - for (TypeId part : oldParts) + part = follow(part); + if (get(part)) + tables.push_back(part); + else { - part = follow(part); - if (get(part)) - tables.push_back(part); - else - { - Replacer replacer{&arena, nullptr, nullptr}; // FIXME this is super super WEIRD - combineIntoIntersection(replacer, &newIntersection, part); - } - } - - // Don't allocate a new table if there's just one in the intersection. - if (tables.size() == 1) - newIntersection.parts.push_back(tables[0]); - else if (!tables.empty()) - { - const TableTypeVar* first = get(tables[0]); - LUAU_ASSERT(first); - - TypeId newTable = arena.addType(TableTypeVar{first->state, first->level}); - TableTypeVar* ttv = getMutable(newTable); - for (TypeId part : tables) - { - // Intuition: If combineIntoTable() needs to clone a table, any references to 'part' are cyclic and need - // to be rewritten to point at 'newTable' in the clone. - Replacer replacer{&arena, part, newTable}; - combineIntoTable(replacer, ttv, part); - } - - newIntersection.parts.push_back(newTable); - } - - itv->parts = std::move(newIntersection.parts); - - asMutable(ty)->normal = areNormal(itv->parts, seen, ice); - - if (itv->parts.size() == 1) - { - TypeId part = itv->parts[0]; - *asMutable(ty) = BoundTypeVar{part}; + Replacer replacer{&arena, nullptr, nullptr}; // FIXME this is super super WEIRD + combineIntoIntersection(replacer, &newIntersection, part); } } - else + + // Don't allocate a new table if there's just one in the intersection. + if (tables.size() == 1) + newIntersection.parts.push_back(tables[0]); + else if (!tables.empty()) { - std::vector oldParts = std::move(itv->parts); + const TableTypeVar* first = get(tables[0]); + LUAU_ASSERT(first); - for (TypeId part : oldParts) - traverse(part); - - std::vector tables; - for (TypeId part : oldParts) + TypeId newTable = arena.addType(TableTypeVar{first->state, first->level}); + TableTypeVar* ttv = getMutable(newTable); + for (TypeId part : tables) { - part = follow(part); - if (get(part)) - tables.push_back(part); - else - { - Replacer replacer{&arena, nullptr, nullptr}; // FIXME this is super super WEIRD - combineIntoIntersection(replacer, itv, part); - } + // Intuition: If combineIntoTable() needs to clone a table, any references to 'part' are cyclic and need + // to be rewritten to point at 'newTable' in the clone. + Replacer replacer{&arena, part, newTable}; + combineIntoTable(replacer, ttv, part); } - // Don't allocate a new table if there's just one in the intersection. - if (tables.size() == 1) - itv->parts.push_back(tables[0]); - else if (!tables.empty()) - { - const TableTypeVar* first = get(tables[0]); - LUAU_ASSERT(first); + newIntersection.parts.push_back(newTable); + } - TypeId newTable = arena.addType(TableTypeVar{first->state, first->level}); - TableTypeVar* ttv = getMutable(newTable); - for (TypeId part : tables) - { - // Intuition: If combineIntoTable() needs to clone a table, any references to 'part' are cyclic and need - // to be rewritten to point at 'newTable' in the clone. - Replacer replacer{&arena, part, newTable}; - combineIntoTable(replacer, ttv, part); - } + itv->parts = std::move(newIntersection.parts); - itv->parts.push_back(newTable); - } + asMutable(ty)->normal = areNormal(itv->parts, seen, ice); - asMutable(ty)->normal = areNormal(itv->parts, seen, ice); - - if (itv->parts.size() == 1) - { - TypeId part = itv->parts[0]; - *asMutable(ty) = BoundTypeVar{part}; - } + if (itv->parts.size() == 1) + { + TypeId part = itv->parts[0]; + *asMutable(ty) = BoundTypeVar{part}; } return false; @@ -629,21 +571,18 @@ struct Normalize final : TypeVarVisitor table->props.insert({propName, prop}); } - if (FFlag::LuauFixNormalizationOfCyclicUnions) + if (tyTable->indexer) { - if (tyTable->indexer) + if (table->indexer) { - if (table->indexer) - { - table->indexer->indexType = combine(replacer, replacer.smartClone(tyTable->indexer->indexType), table->indexer->indexType); - table->indexer->indexResultType = - combine(replacer, replacer.smartClone(tyTable->indexer->indexResultType), table->indexer->indexResultType); - } - else - { - table->indexer = - TableIndexer{replacer.smartClone(tyTable->indexer->indexType), replacer.smartClone(tyTable->indexer->indexResultType)}; - } + table->indexer->indexType = combine(replacer, replacer.smartClone(tyTable->indexer->indexType), table->indexer->indexType); + table->indexer->indexResultType = + combine(replacer, replacer.smartClone(tyTable->indexer->indexResultType), table->indexer->indexResultType); + } + else + { + table->indexer = + TableIndexer{replacer.smartClone(tyTable->indexer->indexType), replacer.smartClone(tyTable->indexer->indexResultType)}; } } diff --git a/Analysis/src/ToString.cpp b/Analysis/src/ToString.cpp index 4ec2371f..8ab124ea 100644 --- a/Analysis/src/ToString.cpp +++ b/Analysis/src/ToString.cpp @@ -1512,6 +1512,15 @@ std::string toString(const Constraint& constraint, ToStringOptions& opts) { return "call " + tos(c.fn, opts) + " with { result = " + tos(c.result, opts) + " }"; } + else if constexpr (std::is_same_v) + { + return tos(c.resultType, opts) + " ~ prim " + tos(c.expectedType, opts) + ", " + tos(c.singletonType, opts) + ", " + + tos(c.multitonType, opts); + } + else if constexpr (std::is_same_v) + { + return tos(c.resultType, opts) + " ~ hasProp " + tos(c.subjectType, opts) + ", \"" + c.prop + "\""; + } else static_assert(always_false_v, "Non-exhaustive constraint switch"); }; diff --git a/Analysis/src/TypeChecker2.cpp b/Analysis/src/TypeChecker2.cpp index 76b27acd..ea06882a 100644 --- a/Analysis/src/TypeChecker2.cpp +++ b/Analysis/src/TypeChecker2.cpp @@ -307,10 +307,9 @@ struct TypeChecker2 if (var->annotation) { TypeId varType = lookupAnnotation(var->annotation); - if (!isSubtype(*it, varType, stack.back(), singletonTypes, ice)) - { - reportError(TypeMismatch{varType, *it}, value->location); - } + ErrorVec errors = tryUnify(stack.back(), value->location, *it, varType); + if (!errors.empty()) + reportErrors(std::move(errors)); } ++it; @@ -325,7 +324,7 @@ struct TypeChecker2 if (var->annotation) { TypeId varType = lookupAnnotation(var->annotation); - if (!isSubtype(varType, valueType, stack.back(), singletonTypes, ice)) + if (!isSubtype(varType, valueType, stack.back(), singletonTypes, ice, /* anyIsTop */ false)) { reportError(TypeMismatch{varType, valueType}, value->location); } @@ -540,7 +539,7 @@ struct TypeChecker2 visit(rhs); TypeId rhsType = lookupType(rhs); - if (!isSubtype(rhsType, lhsType, stack.back(), singletonTypes, ice)) + if (!isSubtype(rhsType, lhsType, stack.back(), singletonTypes, ice, /* anyIsTop */ false)) { reportError(TypeMismatch{lhsType, rhsType}, rhs->location); } @@ -691,7 +690,7 @@ struct TypeChecker2 TypeId actualType = lookupType(number); TypeId numberType = singletonTypes->numberType; - if (!isSubtype(numberType, actualType, stack.back(), singletonTypes, ice)) + if (!isSubtype(numberType, actualType, stack.back(), singletonTypes, ice, /* anyIsTop */ false)) { reportError(TypeMismatch{actualType, numberType}, number->location); } @@ -702,7 +701,7 @@ struct TypeChecker2 TypeId actualType = lookupType(string); TypeId stringType = singletonTypes->stringType; - if (!isSubtype(stringType, actualType, stack.back(), singletonTypes, ice)) + if (!isSubtype(stringType, actualType, stack.back(), singletonTypes, ice, /* anyIsTop */ false)) { reportError(TypeMismatch{actualType, stringType}, string->location); } @@ -762,7 +761,7 @@ struct TypeChecker2 FunctionTypeVar ftv{argsTp, expectedRetType}; TypeId expectedType = arena.addType(ftv); - if (!isSubtype(expectedType, instantiatedFunctionType, stack.back(), singletonTypes, ice)) + if (!isSubtype(instantiatedFunctionType, expectedType, stack.back(), singletonTypes, ice, /* anyIsTop */ false)) { CloneState cloneState; expectedType = clone(expectedType, module->internalTypes, cloneState); @@ -781,7 +780,7 @@ struct TypeChecker2 getIndexTypeFromType(module->getModuleScope(), leftType, indexName->index.value, indexName->location, /* addErrors */ true); if (ty) { - if (!isSubtype(resultType, *ty, stack.back(), singletonTypes, ice)) + if (!isSubtype(resultType, *ty, stack.back(), singletonTypes, ice, /* anyIsTop */ false)) { reportError(TypeMismatch{resultType, *ty}, indexName->location); } @@ -814,7 +813,7 @@ struct TypeChecker2 TypeId inferredArgTy = *argIt; TypeId annotatedArgTy = lookupAnnotation(arg->annotation); - if (!isSubtype(annotatedArgTy, inferredArgTy, stack.back(), singletonTypes, ice)) + if (!isSubtype(annotatedArgTy, inferredArgTy, stack.back(), singletonTypes, ice, /* anyIsTop */ false)) { reportError(TypeMismatch{annotatedArgTy, inferredArgTy}, arg->location); } @@ -859,10 +858,10 @@ struct TypeChecker2 TypeId computedType = lookupType(expr->expr); // Note: As an optimization, we try 'number <: number | string' first, as that is the more likely case. - if (isSubtype(annotationType, computedType, stack.back(), singletonTypes, ice)) + if (isSubtype(annotationType, computedType, stack.back(), singletonTypes, ice, /* anyIsTop */ false)) return; - if (isSubtype(computedType, annotationType, stack.back(), singletonTypes, ice)) + if (isSubtype(computedType, annotationType, stack.back(), singletonTypes, ice, /* anyIsTop */ false)) return; reportError(TypesAreUnrelated{computedType, annotationType}, expr->location); diff --git a/Ast/include/Luau/DenseHash.h b/Ast/include/Luau/DenseHash.h index f8543111..b222feb0 100644 --- a/Ast/include/Luau/DenseHash.h +++ b/Ast/include/Luau/DenseHash.h @@ -5,7 +5,6 @@ #include #include -#include #include #include @@ -35,30 +34,125 @@ public: class iterator; DenseHashTable(const Key& empty_key, size_t buckets = 0) - : count(0) + : data(nullptr) + , capacity(0) + , count(0) , empty_key(empty_key) { + // validate that equality operator is at least somewhat functional + LUAU_ASSERT(eq(empty_key, empty_key)); // buckets has to be power-of-two or zero LUAU_ASSERT((buckets & (buckets - 1)) == 0); - // don't move this to initializer list! this works around an MSVC codegen issue on AMD CPUs: - // https://developercommunity.visualstudio.com/t/stdvector-constructor-from-size-t-is-25-times-slow/1546547 if (buckets) - resize_data(buckets); + { + data = static_cast(::operator new(sizeof(Item) * buckets)); + capacity = buckets; + + ItemInterface::fill(data, buckets, empty_key); + } + } + + ~DenseHashTable() + { + if (data) + destroy(); + } + + DenseHashTable(const DenseHashTable& other) + : data(nullptr) + , capacity(0) + , count(other.count) + , empty_key(other.empty_key) + { + if (other.capacity) + { + data = static_cast(::operator new(sizeof(Item) * other.capacity)); + + for (size_t i = 0; i < other.capacity; ++i) + { + new (&data[i]) Item(other.data[i]); + capacity = i + 1; // if Item copy throws, capacity will note the number of initialized objects for destroy() to clean up + } + } + } + + DenseHashTable(DenseHashTable&& other) + : data(other.data) + , capacity(other.capacity) + , count(other.count) + , empty_key(other.empty_key) + { + other.data = nullptr; + other.capacity = 0; + other.count = 0; + } + + DenseHashTable& operator=(DenseHashTable&& other) + { + if (this != &other) + { + if (data) + destroy(); + + data = other.data; + capacity = other.capacity; + count = other.count; + empty_key = other.empty_key; + + other.data = nullptr; + other.capacity = 0; + other.count = 0; + } + + return *this; + } + + DenseHashTable& operator=(const DenseHashTable& other) + { + if (this != &other) + { + DenseHashTable copy(other); + *this = std::move(copy); + } + + return *this; } void clear() { - data.clear(); + if (count == 0) + return; + + if (capacity > 32) + { + destroy(); + } + else + { + ItemInterface::destroy(data, capacity); + ItemInterface::fill(data, capacity, empty_key); + } + count = 0; } + void destroy() + { + ItemInterface::destroy(data, capacity); + + ::operator delete(data); + data = nullptr; + + capacity = 0; + } + Item* insert_unsafe(const Key& key) { // It is invalid to insert empty_key into the table since it acts as a "entry does not exist" marker LUAU_ASSERT(!eq(key, empty_key)); - size_t hashmod = data.size() - 1; + size_t hashmod = capacity - 1; size_t bucket = hasher(key) & hashmod; for (size_t probe = 0; probe <= hashmod; ++probe) @@ -90,12 +184,12 @@ public: const Item* find(const Key& key) const { - if (data.empty()) + if (count == 0) return 0; if (eq(key, empty_key)) return 0; - size_t hashmod = data.size() - 1; + size_t hashmod = capacity - 1; size_t bucket = hasher(key) & hashmod; for (size_t probe = 0; probe <= hashmod; ++probe) @@ -121,18 +215,11 @@ public: void rehash() { - size_t newsize = data.empty() ? 16 : data.size() * 2; - - if (data.empty() && data.capacity() >= newsize) - { - LUAU_ASSERT(count == 0); - resize_data(newsize); - return; - } + size_t newsize = capacity == 0 ? 16 : capacity * 2; DenseHashTable newtable(empty_key, newsize); - for (size_t i = 0; i < data.size(); ++i) + for (size_t i = 0; i < capacity; ++i) { const Key& key = ItemInterface::getKey(data[i]); @@ -144,12 +231,14 @@ public: } LUAU_ASSERT(count == newtable.count); - data.swap(newtable.data); + + std::swap(data, newtable.data); + std::swap(capacity, newtable.capacity); } void rehash_if_full() { - if (count >= data.size() * 3 / 4) + if (count >= capacity * 3 / 4) { rehash(); } @@ -159,7 +248,7 @@ public: { size_t start = 0; - while (start < data.size() && eq(ItemInterface::getKey(data[start]), empty_key)) + while (start < capacity && eq(ItemInterface::getKey(data[start]), empty_key)) start++; return const_iterator(this, start); @@ -167,14 +256,14 @@ public: const_iterator end() const { - return const_iterator(this, data.size()); + return const_iterator(this, capacity); } iterator begin() { size_t start = 0; - while (start < data.size() && eq(ItemInterface::getKey(data[start]), empty_key)) + while (start < capacity && eq(ItemInterface::getKey(data[start]), empty_key)) start++; return iterator(this, start); @@ -182,7 +271,7 @@ public: iterator end() { - return iterator(this, data.size()); + return iterator(this, capacity); } size_t size() const @@ -227,7 +316,7 @@ public: const_iterator& operator++() { - size_t size = set->data.size(); + size_t size = set->capacity; do { @@ -286,7 +375,7 @@ public: iterator& operator++() { - size_t size = set->data.size(); + size_t size = set->capacity; do { @@ -309,23 +398,8 @@ public: }; private: - template - void resize_data(size_t count, typename std::enable_if_t>* dummy = nullptr) - { - data.resize(count, ItemInterface::create(empty_key)); - } - - template - void resize_data(size_t count, typename std::enable_if_t>* dummy = nullptr) - { - size_t size = data.size(); - data.resize(count); - - for (size_t i = size; i < count; i++) - data[i].first = empty_key; - } - - std::vector data; + Item* data; + size_t capacity; size_t count; Key empty_key; Hash hasher; @@ -345,9 +419,16 @@ struct ItemInterfaceSet item = key; } - static Key create(const Key& key) + static void fill(Key* data, size_t count, const Key& key) { - return key; + for (size_t i = 0; i < count; ++i) + new (&data[i]) Key(key); + } + + static void destroy(Key* data, size_t count) + { + for (size_t i = 0; i < count; ++i) + data[i].~Key(); } }; @@ -364,9 +445,22 @@ struct ItemInterfaceMap item.first = key; } - static std::pair create(const Key& key) + static void fill(std::pair* data, size_t count, const Key& key) { - return std::pair(key, Value()); + for (size_t i = 0; i < count; ++i) + { + new (&data[i].first) Key(key); + new (&data[i].second) Value(); + } + } + + static void destroy(std::pair* data, size_t count) + { + for (size_t i = 0; i < count; ++i) + { + data[i].first.~Key(); + data[i].second.~Value(); + } } }; diff --git a/Ast/include/Luau/Lexer.h b/Ast/include/Luau/Lexer.h index 7e7fe76b..929402b3 100644 --- a/Ast/include/Luau/Lexer.h +++ b/Ast/include/Luau/Lexer.h @@ -6,6 +6,8 @@ #include "Luau/DenseHash.h" #include "Luau/Common.h" +#include + namespace Luau { diff --git a/Ast/src/Parser.cpp b/Ast/src/Parser.cpp index 0914054f..cf3eaaae 100644 --- a/Ast/src/Parser.cpp +++ b/Ast/src/Parser.cpp @@ -20,7 +20,6 @@ LUAU_DYNAMIC_FASTFLAGVARIABLE(LuaReportParseWrongNamedType, false) bool lua_telemetry_parsed_named_non_function_type = false; LUAU_FASTFLAGVARIABLE(LuauErrorDoubleHexPrefix, false) -LUAU_FASTFLAGVARIABLE(LuauLintParseIntegerIssues, false) LUAU_DYNAMIC_FASTFLAGVARIABLE(LuaReportParseIntegerIssues, false) LUAU_FASTFLAGVARIABLE(LuauInterpolatedStringBaseSupport, false) @@ -2070,95 +2069,8 @@ AstExpr* Parser::parseAssertionExpr() return expr; } -static const char* parseInteger_DEPRECATED(double& result, const char* data, int base) -{ - LUAU_ASSERT(!FFlag::LuauLintParseIntegerIssues); - - char* end = nullptr; - unsigned long long value = strtoull(data, &end, base); - - if (value == ULLONG_MAX && errno == ERANGE) - { - // 'errno' might have been set before we called 'strtoull', but we don't want the overhead of resetting a TLS variable on each call - // so we only reset it when we get a result that might be an out-of-range error and parse again to make sure - errno = 0; - value = strtoull(data, &end, base); - - if (errno == ERANGE) - { - if (DFFlag::LuaReportParseIntegerIssues) - { - if (base == 2) - lua_telemetry_parsed_out_of_range_bin_integer = true; - else - lua_telemetry_parsed_out_of_range_hex_integer = true; - } - } - } - - result = double(value); - return *end == 0 ? nullptr : "Malformed number"; -} - -static const char* parseNumber_DEPRECATED2(double& result, const char* data) -{ - LUAU_ASSERT(!FFlag::LuauLintParseIntegerIssues); - - // binary literal - if (data[0] == '0' && (data[1] == 'b' || data[1] == 'B') && data[2]) - return parseInteger_DEPRECATED(result, data + 2, 2); - - // hexadecimal literal - if (data[0] == '0' && (data[1] == 'x' || data[1] == 'X') && data[2]) - { - if (DFFlag::LuaReportParseIntegerIssues && data[2] == '0' && (data[3] == 'x' || data[3] == 'X')) - lua_telemetry_parsed_double_prefix_hex_integer = true; - - return parseInteger_DEPRECATED(result, data + 2, 16); - } - - char* end = nullptr; - double value = strtod(data, &end); - - result = value; - return *end == 0 ? nullptr : "Malformed number"; -} - -static bool parseNumber_DEPRECATED(double& result, const char* data) -{ - LUAU_ASSERT(!FFlag::LuauLintParseIntegerIssues); - - // binary literal - if (data[0] == '0' && (data[1] == 'b' || data[1] == 'B') && data[2]) - { - char* end = nullptr; - unsigned long long value = strtoull(data + 2, &end, 2); - - result = double(value); - return *end == 0; - } - // hexadecimal literal - else if (data[0] == '0' && (data[1] == 'x' || data[1] == 'X') && data[2]) - { - char* end = nullptr; - unsigned long long value = strtoull(data + 2, &end, 16); - - result = double(value); - return *end == 0; - } - else - { - char* end = nullptr; - double value = strtod(data, &end); - - result = value; - return *end == 0; - } -} - static ConstantNumberParseResult parseInteger(double& result, const char* data, int base) { - LUAU_ASSERT(FFlag::LuauLintParseIntegerIssues); LUAU_ASSERT(base == 2 || base == 16); char* end = nullptr; @@ -2195,8 +2107,6 @@ static ConstantNumberParseResult parseInteger(double& result, const char* data, static ConstantNumberParseResult parseDouble(double& result, const char* data) { - LUAU_ASSERT(FFlag::LuauLintParseIntegerIssues); - // binary literal if (data[0] == '0' && (data[1] == 'b' || data[1] == 'B') && data[2]) return parseInteger(result, data + 2, 2); @@ -2771,49 +2681,14 @@ AstExpr* Parser::parseNumber() scratchData.erase(std::remove(scratchData.begin(), scratchData.end(), '_'), scratchData.end()); } - if (FFlag::LuauLintParseIntegerIssues) - { - double value = 0; - ConstantNumberParseResult result = parseDouble(value, scratchData.c_str()); - nextLexeme(); + double value = 0; + ConstantNumberParseResult result = parseDouble(value, scratchData.c_str()); + nextLexeme(); - if (result == ConstantNumberParseResult::Malformed) - return reportExprError(start, {}, "Malformed number"); + if (result == ConstantNumberParseResult::Malformed) + return reportExprError(start, {}, "Malformed number"); - return allocator.alloc(start, value, result); - } - else if (DFFlag::LuaReportParseIntegerIssues) - { - double value = 0; - if (const char* error = parseNumber_DEPRECATED2(value, scratchData.c_str())) - { - nextLexeme(); - - return reportExprError(start, {}, "%s", error); - } - else - { - nextLexeme(); - - return allocator.alloc(start, value); - } - } - else - { - double value = 0; - if (parseNumber_DEPRECATED(value, scratchData.c_str())) - { - nextLexeme(); - - return allocator.alloc(start, value); - } - else - { - nextLexeme(); - - return reportExprError(start, {}, "Malformed number"); - } - } + return allocator.alloc(start, value, result); } AstLocal* Parser::pushLocal(const Binding& binding) diff --git a/CMakeLists.txt b/CMakeLists.txt index 3dafe5fe..43289f41 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -51,6 +51,8 @@ if(LUAU_BUILD_WEB) add_executable(Luau.Web) endif() +# Proxy target to make it possible to depend on private VM headers +add_library(Luau.VM.Internals INTERFACE) include(Sources.cmake) @@ -70,6 +72,7 @@ target_link_libraries(Luau.Analysis PUBLIC Luau.Ast) target_compile_features(Luau.CodeGen PRIVATE cxx_std_17) target_include_directories(Luau.CodeGen PUBLIC CodeGen/include) +target_link_libraries(Luau.CodeGen PRIVATE Luau.VM Luau.VM.Internals) # Code generation needs VM internals target_link_libraries(Luau.CodeGen PUBLIC Luau.Common) target_compile_features(Luau.VM PRIVATE cxx_std_11) @@ -78,6 +81,8 @@ target_link_libraries(Luau.VM PUBLIC Luau.Common) target_include_directories(isocline PUBLIC extern/isocline/include) +target_include_directories(Luau.VM.Internals INTERFACE VM/src) + set(LUAU_OPTIONS) if(MSVC) diff --git a/CodeGen/include/Luau/UnwindBuilderDwarf2.h b/CodeGen/include/Luau/UnwindBuilderDwarf2.h index 09c91d43..25dbc55b 100644 --- a/CodeGen/include/Luau/UnwindBuilderDwarf2.h +++ b/CodeGen/include/Luau/UnwindBuilderDwarf2.h @@ -27,13 +27,13 @@ public: private: static const unsigned kRawDataLimit = 128; - char rawData[kRawDataLimit]; - char* pos = rawData; + uint8_t rawData[kRawDataLimit]; + uint8_t* pos = rawData; uint32_t stackOffset = 0; // We will remember the FDE location to write some of the fields like entry length, function start and size later - char* fdeEntryStart = nullptr; + uint8_t* fdeEntryStart = nullptr; }; } // namespace CodeGen diff --git a/CodeGen/src/AssemblyBuilderX64.cpp b/CodeGen/src/AssemblyBuilderX64.cpp index 0fd10328..32325b0d 100644 --- a/CodeGen/src/AssemblyBuilderX64.cpp +++ b/CodeGen/src/AssemblyBuilderX64.cpp @@ -1,6 +1,8 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #include "Luau/AssemblyBuilderX64.h" +#include "ByteUtils.h" + #include #include #include @@ -46,44 +48,6 @@ const unsigned AVX_F2 = 0b11; const unsigned kMaxAlign = 16; -// Utility functions to correctly write data on big endian machines -#if defined(__BYTE_ORDER__) && __BYTE_ORDER__ == __ORDER_BIG_ENDIAN__ -#include - -static void writeu32(uint8_t* target, uint32_t value) -{ - value = htole32(value); - memcpy(target, &value, sizeof(value)); -} - -static void writeu64(uint8_t* target, uint64_t value) -{ - value = htole64(value); - memcpy(target, &value, sizeof(value)); -} - -static void writef32(uint8_t* target, float value) -{ - static_assert(sizeof(float) == sizeof(uint32_t), "type size must match to reinterpret data"); - uint32_t data; - memcpy(&data, &value, sizeof(value)); - writeu32(target, data); -} - -static void writef64(uint8_t* target, double value) -{ - static_assert(sizeof(double) == sizeof(uint64_t), "type size must match to reinterpret data"); - uint64_t data; - memcpy(&data, &value, sizeof(value)); - writeu64(target, data); -} -#else -#define writeu32(target, value) memcpy(target, &value, sizeof(value)) -#define writeu64(target, value) memcpy(target, &value, sizeof(value)) -#define writef32(target, value) memcpy(target, &value, sizeof(value)) -#define writef64(target, value) memcpy(target, &value, sizeof(value)) -#endif - AssemblyBuilderX64::AssemblyBuilderX64(bool logText) : logText(logText) { @@ -1014,16 +978,14 @@ void AssemblyBuilderX64::placeImm32(int32_t imm) { uint8_t* pos = codePos; LUAU_ASSERT(pos + sizeof(imm) < codeEnd); - writeu32(pos, imm); - codePos = pos + sizeof(imm); + codePos = writeu32(pos, imm); } void AssemblyBuilderX64::placeImm64(int64_t imm) { uint8_t* pos = codePos; LUAU_ASSERT(pos + sizeof(imm) < codeEnd); - writeu64(pos, imm); - codePos = pos + sizeof(imm); + codePos = writeu64(pos, imm); } void AssemblyBuilderX64::placeLabel(Label& label) diff --git a/CodeGen/src/ByteUtils.h b/CodeGen/src/ByteUtils.h new file mode 100644 index 00000000..1e1e341d --- /dev/null +++ b/CodeGen/src/ByteUtils.h @@ -0,0 +1,78 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +#pragma once + +#if defined(__BYTE_ORDER__) && __BYTE_ORDER__ == __ORDER_BIG_ENDIAN__ +#include +#endif + +#include + +inline uint8_t* writeu8(uint8_t* target, uint8_t value) +{ + *target = value; + return target + sizeof(value); +} + +inline uint8_t* writeu32(uint8_t* target, uint32_t value) +{ +#if defined(__BYTE_ORDER__) && __BYTE_ORDER__ == __ORDER_BIG_ENDIAN__ + value = htole32(value); +#endif + + memcpy(target, &value, sizeof(value)); + return target + sizeof(value); +} + +inline uint8_t* writeu64(uint8_t* target, uint64_t value) +{ +#if defined(__BYTE_ORDER__) && __BYTE_ORDER__ == __ORDER_BIG_ENDIAN__ + value = htole64(value); +#endif + + memcpy(target, &value, sizeof(value)); + return target + sizeof(value); +} + +inline uint8_t* writeuleb128(uint8_t* target, uint64_t value) +{ + do + { + uint8_t byte = value & 0x7f; + value >>= 7; + + if (value) + byte |= 0x80; + + *target++ = byte; + } while (value); + + return target; +} + +inline uint8_t* writef32(uint8_t* target, float value) +{ +#if defined(__BYTE_ORDER__) && __BYTE_ORDER__ == __ORDER_BIG_ENDIAN__ + static_assert(sizeof(float) == sizeof(uint32_t), "type size must match to reinterpret data"); + uint32_t data; + memcpy(&data, &value, sizeof(value)); + writeu32(target, data); +#else + memcpy(target, &value, sizeof(value)); +#endif + + return target + sizeof(value); +} + +inline uint8_t* writef64(uint8_t* target, double value) +{ +#if defined(__BYTE_ORDER__) && __BYTE_ORDER__ == __ORDER_BIG_ENDIAN__ + static_assert(sizeof(double) == sizeof(uint64_t), "type size must match to reinterpret data"); + uint64_t data; + memcpy(&data, &value, sizeof(value)); + writeu64(target, data); +#else + memcpy(target, &value, sizeof(value)); +#endif + + return target + sizeof(value); +} diff --git a/CodeGen/src/Fallbacks.cpp b/CodeGen/src/Fallbacks.cpp new file mode 100644 index 00000000..3893d349 --- /dev/null +++ b/CodeGen/src/Fallbacks.cpp @@ -0,0 +1,2511 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +// This code is based on Lua 5.x implementation licensed under MIT License; see lua_LICENSE.txt for details +// This file was generated by 'tools/lvmexecute_split.py' script, do not modify it by hand +#include "Fallbacks.h" +#include "FallbacksProlog.h" + +const Instruction* execute_LOP_NOP(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k) +{ + Instruction insn = *pc++; + LUAU_ASSERT(insn == 0); + return pc; +} + +const Instruction* execute_LOP_LOADNIL(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k) +{ + Instruction insn = *pc++; + StkId ra = VM_REG(LUAU_INSN_A(insn)); + + setnilvalue(ra); + return pc; +} + +const Instruction* execute_LOP_LOADB(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k) +{ + Instruction insn = *pc++; + StkId ra = VM_REG(LUAU_INSN_A(insn)); + + setbvalue(ra, LUAU_INSN_B(insn)); + + pc += LUAU_INSN_C(insn); + LUAU_ASSERT(unsigned(pc - cl->l.p->code) < unsigned(cl->l.p->sizecode)); + return pc; +} + +const Instruction* execute_LOP_LOADN(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k) +{ + Instruction insn = *pc++; + StkId ra = VM_REG(LUAU_INSN_A(insn)); + + setnvalue(ra, LUAU_INSN_D(insn)); + return pc; +} + +const Instruction* execute_LOP_LOADK(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k) +{ + Instruction insn = *pc++; + StkId ra = VM_REG(LUAU_INSN_A(insn)); + TValue* kv = VM_KV(LUAU_INSN_D(insn)); + + setobj2s(L, ra, kv); + return pc; +} + +const Instruction* execute_LOP_MOVE(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k) +{ + Instruction insn = *pc++; + StkId ra = VM_REG(LUAU_INSN_A(insn)); + StkId rb = VM_REG(LUAU_INSN_B(insn)); + + setobj2s(L, ra, rb); + return pc; +} + +const Instruction* execute_LOP_GETGLOBAL(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k) +{ + Instruction insn = *pc++; + StkId ra = VM_REG(LUAU_INSN_A(insn)); + uint32_t aux = *pc++; + TValue* kv = VM_KV(aux); + LUAU_ASSERT(ttisstring(kv)); + + // fast-path: value is in expected slot + Table* h = cl->env; + int slot = LUAU_INSN_C(insn) & h->nodemask8; + LuaNode* n = &h->node[slot]; + + if (LUAU_LIKELY(ttisstring(gkey(n)) && tsvalue(gkey(n)) == tsvalue(kv)) && !ttisnil(gval(n))) + { + setobj2s(L, ra, gval(n)); + return pc; + } + else + { + // slow-path, may invoke Lua calls via __index metamethod + TValue g; + sethvalue(L, &g, h); + L->cachedslot = slot; + VM_PROTECT(luaV_gettable(L, &g, kv, ra)); + // save cachedslot to accelerate future lookups; patches currently executing instruction since pc-2 rolls back two pc++ + VM_PATCH_C(pc - 2, L->cachedslot); + return pc; + } +} + +const Instruction* execute_LOP_SETGLOBAL(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k) +{ + Instruction insn = *pc++; + StkId ra = VM_REG(LUAU_INSN_A(insn)); + uint32_t aux = *pc++; + TValue* kv = VM_KV(aux); + LUAU_ASSERT(ttisstring(kv)); + + // fast-path: value is in expected slot + Table* h = cl->env; + int slot = LUAU_INSN_C(insn) & h->nodemask8; + LuaNode* n = &h->node[slot]; + + if (LUAU_LIKELY(ttisstring(gkey(n)) && tsvalue(gkey(n)) == tsvalue(kv) && !ttisnil(gval(n)) && !h->readonly)) + { + setobj2t(L, gval(n), ra); + luaC_barriert(L, h, ra); + return pc; + } + else + { + // slow-path, may invoke Lua calls via __newindex metamethod + TValue g; + sethvalue(L, &g, h); + L->cachedslot = slot; + VM_PROTECT(luaV_settable(L, &g, kv, ra)); + // save cachedslot to accelerate future lookups; patches currently executing instruction since pc-2 rolls back two pc++ + VM_PATCH_C(pc - 2, L->cachedslot); + return pc; + } +} + +const Instruction* execute_LOP_GETUPVAL(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k) +{ + Instruction insn = *pc++; + StkId ra = VM_REG(LUAU_INSN_A(insn)); + TValue* ur = VM_UV(LUAU_INSN_B(insn)); + TValue* v = ttisupval(ur) ? upvalue(ur)->v : ur; + + setobj2s(L, ra, v); + return pc; +} + +const Instruction* execute_LOP_SETUPVAL(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k) +{ + Instruction insn = *pc++; + StkId ra = VM_REG(LUAU_INSN_A(insn)); + TValue* ur = VM_UV(LUAU_INSN_B(insn)); + UpVal* uv = upvalue(ur); + + setobj(L, uv->v, ra); + luaC_barrier(L, uv, ra); + return pc; +} + +const Instruction* execute_LOP_CLOSEUPVALS(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k) +{ + Instruction insn = *pc++; + StkId ra = VM_REG(LUAU_INSN_A(insn)); + + if (L->openupval && L->openupval->v >= ra) + luaF_close(L, ra); + return pc; +} + +const Instruction* execute_LOP_GETIMPORT(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k) +{ + Instruction insn = *pc++; + StkId ra = VM_REG(LUAU_INSN_A(insn)); + TValue* kv = VM_KV(LUAU_INSN_D(insn)); + + // fast-path: import resolution was successful and closure environment is "safe" for import + if (!ttisnil(kv) && cl->env->safeenv) + { + setobj2s(L, ra, kv); + pc++; // skip over AUX + return pc; + } + else + { + uint32_t aux = *pc++; + + VM_PROTECT(luaV_getimport(L, cl->env, k, aux, /* propagatenil= */ false)); + ra = VM_REG(LUAU_INSN_A(insn)); // previous call may change the stack + + setobj2s(L, ra, L->top - 1); + L->top--; + return pc; + } +} + +const Instruction* execute_LOP_GETTABLEKS(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k) +{ + Instruction insn = *pc++; + StkId ra = VM_REG(LUAU_INSN_A(insn)); + StkId rb = VM_REG(LUAU_INSN_B(insn)); + uint32_t aux = *pc++; + TValue* kv = VM_KV(aux); + LUAU_ASSERT(ttisstring(kv)); + + // fast-path: built-in table + if (ttistable(rb)) + { + Table* h = hvalue(rb); + + int slot = LUAU_INSN_C(insn) & h->nodemask8; + LuaNode* n = &h->node[slot]; + + // fast-path: value is in expected slot + if (LUAU_LIKELY(ttisstring(gkey(n)) && tsvalue(gkey(n)) == tsvalue(kv) && !ttisnil(gval(n)))) + { + setobj2s(L, ra, gval(n)); + return pc; + } + else if (!h->metatable) + { + // fast-path: value is not in expected slot, but the table lookup doesn't involve metatable + const TValue* res = luaH_getstr(h, tsvalue(kv)); + + if (res != luaO_nilobject) + { + int cachedslot = gval2slot(h, res); + // save cachedslot to accelerate future lookups; patches currently executing instruction since pc-2 rolls back two pc++ + VM_PATCH_C(pc - 2, cachedslot); + } + + setobj2s(L, ra, res); + return pc; + } + else + { + // slow-path, may invoke Lua calls via __index metamethod + L->cachedslot = slot; + VM_PROTECT(luaV_gettable(L, rb, kv, ra)); + // save cachedslot to accelerate future lookups; patches currently executing instruction since pc-2 rolls back two pc++ + VM_PATCH_C(pc - 2, L->cachedslot); + return pc; + } + } + else + { + // fast-path: user data with C __index TM + const TValue* fn = 0; + if (ttisuserdata(rb) && (fn = fasttm(L, uvalue(rb)->metatable, TM_INDEX)) && ttisfunction(fn) && clvalue(fn)->isC) + { + // note: it's safe to push arguments past top for complicated reasons (see top of the file) + LUAU_ASSERT(L->top + 3 < L->stack + L->stacksize); + StkId top = L->top; + setobj2s(L, top + 0, fn); + setobj2s(L, top + 1, rb); + setobj2s(L, top + 2, kv); + L->top = top + 3; + + L->cachedslot = LUAU_INSN_C(insn); + VM_PROTECT(luaV_callTM(L, 2, LUAU_INSN_A(insn))); + // save cachedslot to accelerate future lookups; patches currently executing instruction since pc-2 rolls back two pc++ + VM_PATCH_C(pc - 2, L->cachedslot); + return pc; + } + else if (ttisvector(rb)) + { + // fast-path: quick case-insensitive comparison with "X"/"Y"/"Z" + const char* name = getstr(tsvalue(kv)); + int ic = (name[0] | ' ') - 'x'; + +#if LUA_VECTOR_SIZE == 4 + // 'w' is before 'x' in ascii, so ic is -1 when indexing with 'w' + if (ic == -1) + ic = 3; +#endif + + if (unsigned(ic) < LUA_VECTOR_SIZE && name[1] == '\0') + { + const float* v = rb->value.v; // silences ubsan when indexing v[] + setnvalue(ra, v[ic]); + return pc; + } + + fn = fasttm(L, L->global->mt[LUA_TVECTOR], TM_INDEX); + + if (fn && ttisfunction(fn) && clvalue(fn)->isC) + { + // note: it's safe to push arguments past top for complicated reasons (see top of the file) + LUAU_ASSERT(L->top + 3 < L->stack + L->stacksize); + StkId top = L->top; + setobj2s(L, top + 0, fn); + setobj2s(L, top + 1, rb); + setobj2s(L, top + 2, kv); + L->top = top + 3; + + L->cachedslot = LUAU_INSN_C(insn); + VM_PROTECT(luaV_callTM(L, 2, LUAU_INSN_A(insn))); + // save cachedslot to accelerate future lookups; patches currently executing instruction since pc-2 rolls back two pc++ + VM_PATCH_C(pc - 2, L->cachedslot); + return pc; + } + + // fall through to slow path + } + + // fall through to slow path + } + + // slow-path, may invoke Lua calls via __index metamethod + VM_PROTECT(luaV_gettable(L, rb, kv, ra)); + return pc; +} + +const Instruction* execute_LOP_SETTABLEKS(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k) +{ + Instruction insn = *pc++; + StkId ra = VM_REG(LUAU_INSN_A(insn)); + StkId rb = VM_REG(LUAU_INSN_B(insn)); + uint32_t aux = *pc++; + TValue* kv = VM_KV(aux); + LUAU_ASSERT(ttisstring(kv)); + + // fast-path: built-in table + if (ttistable(rb)) + { + Table* h = hvalue(rb); + + int slot = LUAU_INSN_C(insn) & h->nodemask8; + LuaNode* n = &h->node[slot]; + + // fast-path: value is in expected slot + if (LUAU_LIKELY(ttisstring(gkey(n)) && tsvalue(gkey(n)) == tsvalue(kv) && !ttisnil(gval(n)) && !h->readonly)) + { + setobj2t(L, gval(n), ra); + luaC_barriert(L, h, ra); + return pc; + } + else if (fastnotm(h->metatable, TM_NEWINDEX) && !h->readonly) + { + VM_PROTECT_PC(); // set may fail + + TValue* res = luaH_setstr(L, h, tsvalue(kv)); + int cachedslot = gval2slot(h, res); + // save cachedslot to accelerate future lookups; patches currently executing instruction since pc-2 rolls back two pc++ + VM_PATCH_C(pc - 2, cachedslot); + setobj2t(L, res, ra); + luaC_barriert(L, h, ra); + return pc; + } + else + { + // slow-path, may invoke Lua calls via __newindex metamethod + L->cachedslot = slot; + VM_PROTECT(luaV_settable(L, rb, kv, ra)); + // save cachedslot to accelerate future lookups; patches currently executing instruction since pc-2 rolls back two pc++ + VM_PATCH_C(pc - 2, L->cachedslot); + return pc; + } + } + else + { + // fast-path: user data with C __newindex TM + const TValue* fn = 0; + if (ttisuserdata(rb) && (fn = fasttm(L, uvalue(rb)->metatable, TM_NEWINDEX)) && ttisfunction(fn) && clvalue(fn)->isC) + { + // note: it's safe to push arguments past top for complicated reasons (see top of the file) + LUAU_ASSERT(L->top + 4 < L->stack + L->stacksize); + StkId top = L->top; + setobj2s(L, top + 0, fn); + setobj2s(L, top + 1, rb); + setobj2s(L, top + 2, kv); + setobj2s(L, top + 3, ra); + L->top = top + 4; + + L->cachedslot = LUAU_INSN_C(insn); + VM_PROTECT(luaV_callTM(L, 3, -1)); + // save cachedslot to accelerate future lookups; patches currently executing instruction since pc-2 rolls back two pc++ + VM_PATCH_C(pc - 2, L->cachedslot); + return pc; + } + else + { + // slow-path, may invoke Lua calls via __newindex metamethod + VM_PROTECT(luaV_settable(L, rb, kv, ra)); + return pc; + } + } +} + +const Instruction* execute_LOP_GETTABLE(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k) +{ + Instruction insn = *pc++; + StkId ra = VM_REG(LUAU_INSN_A(insn)); + StkId rb = VM_REG(LUAU_INSN_B(insn)); + StkId rc = VM_REG(LUAU_INSN_C(insn)); + + // fast-path: array lookup + if (ttistable(rb) && ttisnumber(rc)) + { + Table* h = hvalue(rb); + + double indexd = nvalue(rc); + int index = int(indexd); + + // index has to be an exact integer and in-bounds for the array portion + if (LUAU_LIKELY(unsigned(index - 1) < unsigned(h->sizearray) && !h->metatable && double(index) == indexd)) + { + setobj2s(L, ra, &h->array[unsigned(index - 1)]); + return pc; + } + + // fall through to slow path + } + + // slow-path: handles out of bounds array lookups, non-integer numeric keys, non-array table lookup, __index MT calls + VM_PROTECT(luaV_gettable(L, rb, rc, ra)); + return pc; +} + +const Instruction* execute_LOP_SETTABLE(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k) +{ + Instruction insn = *pc++; + StkId ra = VM_REG(LUAU_INSN_A(insn)); + StkId rb = VM_REG(LUAU_INSN_B(insn)); + StkId rc = VM_REG(LUAU_INSN_C(insn)); + + // fast-path: array assign + if (ttistable(rb) && ttisnumber(rc)) + { + Table* h = hvalue(rb); + + double indexd = nvalue(rc); + int index = int(indexd); + + // index has to be an exact integer and in-bounds for the array portion + if (LUAU_LIKELY(unsigned(index - 1) < unsigned(h->sizearray) && !h->metatable && !h->readonly && double(index) == indexd)) + { + setobj2t(L, &h->array[unsigned(index - 1)], ra); + luaC_barriert(L, h, ra); + return pc; + } + + // fall through to slow path + } + + // slow-path: handles out of bounds array assignments, non-integer numeric keys, non-array table access, __newindex MT calls + VM_PROTECT(luaV_settable(L, rb, rc, ra)); + return pc; +} + +const Instruction* execute_LOP_GETTABLEN(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k) +{ + Instruction insn = *pc++; + StkId ra = VM_REG(LUAU_INSN_A(insn)); + StkId rb = VM_REG(LUAU_INSN_B(insn)); + int c = LUAU_INSN_C(insn); + + // fast-path: array lookup + if (ttistable(rb)) + { + Table* h = hvalue(rb); + + if (LUAU_LIKELY(unsigned(c) < unsigned(h->sizearray) && !h->metatable)) + { + setobj2s(L, ra, &h->array[c]); + return pc; + } + + // fall through to slow path + } + + // slow-path: handles out of bounds array lookups + TValue n; + setnvalue(&n, c + 1); + VM_PROTECT(luaV_gettable(L, rb, &n, ra)); + return pc; +} + +const Instruction* execute_LOP_SETTABLEN(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k) +{ + Instruction insn = *pc++; + StkId ra = VM_REG(LUAU_INSN_A(insn)); + StkId rb = VM_REG(LUAU_INSN_B(insn)); + int c = LUAU_INSN_C(insn); + + // fast-path: array assign + if (ttistable(rb)) + { + Table* h = hvalue(rb); + + if (LUAU_LIKELY(unsigned(c) < unsigned(h->sizearray) && !h->metatable && !h->readonly)) + { + setobj2t(L, &h->array[c], ra); + luaC_barriert(L, h, ra); + return pc; + } + + // fall through to slow path + } + + // slow-path: handles out of bounds array lookups + TValue n; + setnvalue(&n, c + 1); + VM_PROTECT(luaV_settable(L, rb, &n, ra)); + return pc; +} + +const Instruction* execute_LOP_NEWCLOSURE(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k) +{ + Instruction insn = *pc++; + StkId ra = VM_REG(LUAU_INSN_A(insn)); + + Proto* pv = cl->l.p->p[LUAU_INSN_D(insn)]; + LUAU_ASSERT(unsigned(LUAU_INSN_D(insn)) < unsigned(cl->l.p->sizep)); + + // note: we save closure to stack early in case the code below wants to capture it by value + Closure* ncl = luaF_newLclosure(L, pv->nups, cl->env, pv); + setclvalue(L, ra, ncl); + + for (int ui = 0; ui < pv->nups; ++ui) + { + Instruction uinsn = *pc++; + LUAU_ASSERT(LUAU_INSN_OP(uinsn) == LOP_CAPTURE); + + switch (LUAU_INSN_A(uinsn)) + { + case LCT_VAL: + setobj(L, &ncl->l.uprefs[ui], VM_REG(LUAU_INSN_B(uinsn))); + break; + + case LCT_REF: + setupvalue(L, &ncl->l.uprefs[ui], luaF_findupval(L, VM_REG(LUAU_INSN_B(uinsn)))); + break; + + case LCT_UPVAL: + setobj(L, &ncl->l.uprefs[ui], VM_UV(LUAU_INSN_B(uinsn))); + break; + + default: + LUAU_ASSERT(!"Unknown upvalue capture type"); + } + } + + VM_PROTECT(luaC_checkGC(L)); + return pc; +} + +const Instruction* execute_LOP_NAMECALL(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k) +{ + Instruction insn = *pc++; + StkId ra = VM_REG(LUAU_INSN_A(insn)); + StkId rb = VM_REG(LUAU_INSN_B(insn)); + uint32_t aux = *pc++; + TValue* kv = VM_KV(aux); + LUAU_ASSERT(ttisstring(kv)); + + if (ttistable(rb)) + { + Table* h = hvalue(rb); + // note: we can't use nodemask8 here because we need to query the main position of the table, and 8-bit nodemask8 only works + // for predictive lookups + LuaNode* n = &h->node[tsvalue(kv)->hash & (sizenode(h) - 1)]; + + const TValue* mt = 0; + const LuaNode* mtn = 0; + + // fast-path: key is in the table in expected slot + if (ttisstring(gkey(n)) && tsvalue(gkey(n)) == tsvalue(kv) && !ttisnil(gval(n))) + { + // note: order of copies allows rb to alias ra+1 or ra + setobj2s(L, ra + 1, rb); + setobj2s(L, ra, gval(n)); + } + // fast-path: key is absent from the base, table has an __index table, and it has the result in the expected slot + else if (gnext(n) == 0 && (mt = fasttm(L, hvalue(rb)->metatable, TM_INDEX)) && ttistable(mt) && + (mtn = &hvalue(mt)->node[LUAU_INSN_C(insn) & hvalue(mt)->nodemask8]) && ttisstring(gkey(mtn)) && tsvalue(gkey(mtn)) == tsvalue(kv) && + !ttisnil(gval(mtn))) + { + // note: order of copies allows rb to alias ra+1 or ra + setobj2s(L, ra + 1, rb); + setobj2s(L, ra, gval(mtn)); + } + else + { + // slow-path: handles full table lookup + setobj2s(L, ra + 1, rb); + L->cachedslot = LUAU_INSN_C(insn); + VM_PROTECT(luaV_gettable(L, rb, kv, ra)); + // save cachedslot to accelerate future lookups; patches currently executing instruction since pc-2 rolls back two pc++ + VM_PATCH_C(pc - 2, L->cachedslot); + // recompute ra since stack might have been reallocated + ra = VM_REG(LUAU_INSN_A(insn)); + if (ttisnil(ra)) + luaG_methoderror(L, ra + 1, tsvalue(kv)); + } + } + else + { + Table* mt = ttisuserdata(rb) ? uvalue(rb)->metatable : L->global->mt[ttype(rb)]; + const TValue* tmi = 0; + + // fast-path: metatable with __namecall + if (const TValue* fn = fasttm(L, mt, TM_NAMECALL)) + { + // note: order of copies allows rb to alias ra+1 or ra + setobj2s(L, ra + 1, rb); + setobj2s(L, ra, fn); + + L->namecall = tsvalue(kv); + } + else if ((tmi = fasttm(L, mt, TM_INDEX)) && ttistable(tmi)) + { + Table* h = hvalue(tmi); + int slot = LUAU_INSN_C(insn) & h->nodemask8; + LuaNode* n = &h->node[slot]; + + // fast-path: metatable with __index that has method in expected slot + if (LUAU_LIKELY(ttisstring(gkey(n)) && tsvalue(gkey(n)) == tsvalue(kv) && !ttisnil(gval(n)))) + { + // note: order of copies allows rb to alias ra+1 or ra + setobj2s(L, ra + 1, rb); + setobj2s(L, ra, gval(n)); + } + else + { + // slow-path: handles slot mismatch + setobj2s(L, ra + 1, rb); + L->cachedslot = slot; + VM_PROTECT(luaV_gettable(L, rb, kv, ra)); + // save cachedslot to accelerate future lookups; patches currently executing instruction since pc-2 rolls back two pc++ + VM_PATCH_C(pc - 2, L->cachedslot); + // recompute ra since stack might have been reallocated + ra = VM_REG(LUAU_INSN_A(insn)); + if (ttisnil(ra)) + luaG_methoderror(L, ra + 1, tsvalue(kv)); + } + } + else + { + // slow-path: handles non-table __index + setobj2s(L, ra + 1, rb); + VM_PROTECT(luaV_gettable(L, rb, kv, ra)); + // recompute ra since stack might have been reallocated + ra = VM_REG(LUAU_INSN_A(insn)); + if (ttisnil(ra)) + luaG_methoderror(L, ra + 1, tsvalue(kv)); + } + } + + // intentional fallthrough to CALL + LUAU_ASSERT(LUAU_INSN_OP(*pc) == LOP_CALL); + return pc; +} + +const Instruction* execute_LOP_CALL(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k) +{ + VM_INTERRUPT(); + Instruction insn = *pc++; + StkId ra = VM_REG(LUAU_INSN_A(insn)); + + int nparams = LUAU_INSN_B(insn) - 1; + int nresults = LUAU_INSN_C(insn) - 1; + + StkId argtop = L->top; + argtop = (nparams == LUA_MULTRET) ? argtop : ra + 1 + nparams; + + // slow-path: not a function call + if (LUAU_UNLIKELY(!ttisfunction(ra))) + { + VM_PROTECT(luaV_tryfuncTM(L, ra)); + argtop++; // __call adds an extra self + } + + Closure* ccl = clvalue(ra); + L->ci->savedpc = pc; + + CallInfo* ci = incr_ci(L); + ci->func = ra; + ci->base = ra + 1; + ci->top = argtop + ccl->stacksize; // note: technically UB since we haven't reallocated the stack yet + ci->savedpc = NULL; + ci->flags = 0; + ci->nresults = nresults; + + L->base = ci->base; + L->top = argtop; + + // note: this reallocs stack, but we don't need to VM_PROTECT this + // this is because we're going to modify base/savedpc manually anyhow + // crucially, we can't use ra/argtop after this line + luaD_checkstack(L, ccl->stacksize); + + LUAU_ASSERT(ci->top <= L->stack_last); + + if (!ccl->isC) + { + Proto* p = ccl->l.p; + + // fill unused parameters with nil + StkId argi = L->top; + StkId argend = L->base + p->numparams; + while (argi < argend) + setnilvalue(argi++); // complete missing arguments + L->top = p->is_vararg ? argi : ci->top; + + // reentry + pc = p->code; + cl = ccl; + base = L->base; + k = p->k; + return pc; + } + else + { + lua_CFunction func = ccl->c.f; + int n = func(L); + + // yield + if (n < 0) + return NULL; + + // ci is our callinfo, cip is our parent + CallInfo* ci = L->ci; + CallInfo* cip = ci - 1; + + // copy return values into parent stack (but only up to nresults!), fill the rest with nil + // note: in MULTRET context nresults starts as -1 so i != 0 condition never activates intentionally + StkId res = ci->func; + StkId vali = L->top - n; + StkId valend = L->top; + + int i; + for (i = nresults; i != 0 && vali < valend; i--) + setobjs2s(L, res++, vali++); + while (i-- > 0) + setnilvalue(res++); + + // pop the stack frame + L->ci = cip; + L->base = cip->base; + L->top = (nresults == LUA_MULTRET) ? res : cip->top; + + base = L->base; // stack may have been reallocated, so we need to refresh base ptr + return pc; + } +} + +const Instruction* execute_LOP_RETURN(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k) +{ + VM_INTERRUPT(); + Instruction insn = *pc++; + StkId ra = &base[LUAU_INSN_A(insn)]; // note: this can point to L->top if b == LUA_MULTRET making VM_REG unsafe to use + int b = LUAU_INSN_B(insn) - 1; + + // ci is our callinfo, cip is our parent + CallInfo* ci = L->ci; + CallInfo* cip = ci - 1; + + StkId res = ci->func; // note: we assume CALL always puts func+args and expects results to start at func + + StkId vali = ra; + StkId valend = (b == LUA_MULTRET) ? L->top : ra + b; // copy as much as possible for MULTRET calls, and only as much as needed otherwise + + int nresults = ci->nresults; + + // copy return values into parent stack (but only up to nresults!), fill the rest with nil + // note: in MULTRET context nresults starts as -1 so i != 0 condition never activates intentionally + int i; + for (i = nresults; i != 0 && vali < valend; i--) + setobjs2s(L, res++, vali++); + while (i-- > 0) + setnilvalue(res++); + + // pop the stack frame + L->ci = cip; + L->base = cip->base; + L->top = (nresults == LUA_MULTRET) ? res : cip->top; + + // we're done! + if (LUAU_UNLIKELY(ci->flags & LUA_CALLINFO_RETURN)) + { + L->top = res; + return NULL; + } + + LUAU_ASSERT(isLua(L->ci)); + + Closure* nextcl = clvalue(cip->func); + Proto* nextproto = nextcl->l.p; + + // reentry + pc = cip->savedpc; + cl = nextcl; + base = L->base; + k = nextproto->k; + return pc; +} + +const Instruction* execute_LOP_JUMP(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k) +{ + Instruction insn = *pc++; + + pc += LUAU_INSN_D(insn); + LUAU_ASSERT(unsigned(pc - cl->l.p->code) < unsigned(cl->l.p->sizecode)); + return pc; +} + +const Instruction* execute_LOP_JUMPIF(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k) +{ + Instruction insn = *pc++; + StkId ra = VM_REG(LUAU_INSN_A(insn)); + + pc += l_isfalse(ra) ? 0 : LUAU_INSN_D(insn); + LUAU_ASSERT(unsigned(pc - cl->l.p->code) < unsigned(cl->l.p->sizecode)); + return pc; +} + +const Instruction* execute_LOP_JUMPIFNOT(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k) +{ + Instruction insn = *pc++; + StkId ra = VM_REG(LUAU_INSN_A(insn)); + + pc += l_isfalse(ra) ? LUAU_INSN_D(insn) : 0; + LUAU_ASSERT(unsigned(pc - cl->l.p->code) < unsigned(cl->l.p->sizecode)); + return pc; +} + +const Instruction* execute_LOP_JUMPIFEQ(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k) +{ + Instruction insn = *pc++; + uint32_t aux = *pc; + StkId ra = VM_REG(LUAU_INSN_A(insn)); + StkId rb = VM_REG(aux); + + // Note that all jumps below jump by 1 in the "false" case to skip over aux + if (ttype(ra) == ttype(rb)) + { + switch (ttype(ra)) + { + case LUA_TNIL: + pc += LUAU_INSN_D(insn); + LUAU_ASSERT(unsigned(pc - cl->l.p->code) < unsigned(cl->l.p->sizecode)); + return pc; + + case LUA_TBOOLEAN: + pc += bvalue(ra) == bvalue(rb) ? LUAU_INSN_D(insn) : 1; + LUAU_ASSERT(unsigned(pc - cl->l.p->code) < unsigned(cl->l.p->sizecode)); + return pc; + + case LUA_TLIGHTUSERDATA: + pc += pvalue(ra) == pvalue(rb) ? LUAU_INSN_D(insn) : 1; + LUAU_ASSERT(unsigned(pc - cl->l.p->code) < unsigned(cl->l.p->sizecode)); + return pc; + + case LUA_TNUMBER: + pc += nvalue(ra) == nvalue(rb) ? LUAU_INSN_D(insn) : 1; + LUAU_ASSERT(unsigned(pc - cl->l.p->code) < unsigned(cl->l.p->sizecode)); + return pc; + + case LUA_TVECTOR: + pc += luai_veceq(vvalue(ra), vvalue(rb)) ? LUAU_INSN_D(insn) : 1; + LUAU_ASSERT(unsigned(pc - cl->l.p->code) < unsigned(cl->l.p->sizecode)); + return pc; + + case LUA_TSTRING: + case LUA_TFUNCTION: + case LUA_TTHREAD: + pc += gcvalue(ra) == gcvalue(rb) ? LUAU_INSN_D(insn) : 1; + LUAU_ASSERT(unsigned(pc - cl->l.p->code) < unsigned(cl->l.p->sizecode)); + return pc; + + case LUA_TTABLE: + // fast-path: same metatable, no EQ metamethod + if (hvalue(ra)->metatable == hvalue(rb)->metatable) + { + const TValue* fn = fasttm(L, hvalue(ra)->metatable, TM_EQ); + + if (!fn) + { + pc += hvalue(ra) == hvalue(rb) ? LUAU_INSN_D(insn) : 1; + LUAU_ASSERT(unsigned(pc - cl->l.p->code) < unsigned(cl->l.p->sizecode)); + return pc; + } + } + // slow path after switch() + break; + + case LUA_TUSERDATA: + // fast-path: same metatable, no EQ metamethod or C metamethod + if (uvalue(ra)->metatable == uvalue(rb)->metatable) + { + const TValue* fn = fasttm(L, uvalue(ra)->metatable, TM_EQ); + + if (!fn) + { + pc += uvalue(ra) == uvalue(rb) ? LUAU_INSN_D(insn) : 1; + LUAU_ASSERT(unsigned(pc - cl->l.p->code) < unsigned(cl->l.p->sizecode)); + return pc; + } + else if (ttisfunction(fn) && clvalue(fn)->isC) + { + // note: it's safe to push arguments past top for complicated reasons (see top of the file) + LUAU_ASSERT(L->top + 3 < L->stack + L->stacksize); + StkId top = L->top; + setobj2s(L, top + 0, fn); + setobj2s(L, top + 1, ra); + setobj2s(L, top + 2, rb); + int res = int(top - base); + L->top = top + 3; + + VM_PROTECT(luaV_callTM(L, 2, res)); + pc += !l_isfalse(&base[res]) ? LUAU_INSN_D(insn) : 1; + LUAU_ASSERT(unsigned(pc - cl->l.p->code) < unsigned(cl->l.p->sizecode)); + return pc; + } + } + // slow path after switch() + break; + + default:; + } + + // slow-path: tables with metatables and userdata values + // note that we don't have a fast path for userdata values without metatables, since that's very rare + int res; + VM_PROTECT(res = luaV_equalval(L, ra, rb)); + + pc += (res == 1) ? LUAU_INSN_D(insn) : 1; + LUAU_ASSERT(unsigned(pc - cl->l.p->code) < unsigned(cl->l.p->sizecode)); + return pc; + } + else + { + pc += 1; + LUAU_ASSERT(unsigned(pc - cl->l.p->code) < unsigned(cl->l.p->sizecode)); + return pc; + } +} + +const Instruction* execute_LOP_JUMPIFNOTEQ(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k) +{ + Instruction insn = *pc++; + uint32_t aux = *pc; + StkId ra = VM_REG(LUAU_INSN_A(insn)); + StkId rb = VM_REG(aux); + + // Note that all jumps below jump by 1 in the "true" case to skip over aux + if (ttype(ra) == ttype(rb)) + { + switch (ttype(ra)) + { + case LUA_TNIL: + pc += 1; + LUAU_ASSERT(unsigned(pc - cl->l.p->code) < unsigned(cl->l.p->sizecode)); + return pc; + + case LUA_TBOOLEAN: + pc += bvalue(ra) != bvalue(rb) ? LUAU_INSN_D(insn) : 1; + LUAU_ASSERT(unsigned(pc - cl->l.p->code) < unsigned(cl->l.p->sizecode)); + return pc; + + case LUA_TLIGHTUSERDATA: + pc += pvalue(ra) != pvalue(rb) ? LUAU_INSN_D(insn) : 1; + LUAU_ASSERT(unsigned(pc - cl->l.p->code) < unsigned(cl->l.p->sizecode)); + return pc; + + case LUA_TNUMBER: + pc += nvalue(ra) != nvalue(rb) ? LUAU_INSN_D(insn) : 1; + LUAU_ASSERT(unsigned(pc - cl->l.p->code) < unsigned(cl->l.p->sizecode)); + return pc; + + case LUA_TVECTOR: + pc += !luai_veceq(vvalue(ra), vvalue(rb)) ? LUAU_INSN_D(insn) : 1; + LUAU_ASSERT(unsigned(pc - cl->l.p->code) < unsigned(cl->l.p->sizecode)); + return pc; + + case LUA_TSTRING: + case LUA_TFUNCTION: + case LUA_TTHREAD: + pc += gcvalue(ra) != gcvalue(rb) ? LUAU_INSN_D(insn) : 1; + LUAU_ASSERT(unsigned(pc - cl->l.p->code) < unsigned(cl->l.p->sizecode)); + return pc; + + case LUA_TTABLE: + // fast-path: same metatable, no EQ metamethod + if (hvalue(ra)->metatable == hvalue(rb)->metatable) + { + const TValue* fn = fasttm(L, hvalue(ra)->metatable, TM_EQ); + + if (!fn) + { + pc += hvalue(ra) != hvalue(rb) ? LUAU_INSN_D(insn) : 1; + LUAU_ASSERT(unsigned(pc - cl->l.p->code) < unsigned(cl->l.p->sizecode)); + return pc; + } + } + // slow path after switch() + break; + + case LUA_TUSERDATA: + // fast-path: same metatable, no EQ metamethod or C metamethod + if (uvalue(ra)->metatable == uvalue(rb)->metatable) + { + const TValue* fn = fasttm(L, uvalue(ra)->metatable, TM_EQ); + + if (!fn) + { + pc += uvalue(ra) != uvalue(rb) ? LUAU_INSN_D(insn) : 1; + LUAU_ASSERT(unsigned(pc - cl->l.p->code) < unsigned(cl->l.p->sizecode)); + return pc; + } + else if (ttisfunction(fn) && clvalue(fn)->isC) + { + // note: it's safe to push arguments past top for complicated reasons (see top of the file) + LUAU_ASSERT(L->top + 3 < L->stack + L->stacksize); + StkId top = L->top; + setobj2s(L, top + 0, fn); + setobj2s(L, top + 1, ra); + setobj2s(L, top + 2, rb); + int res = int(top - base); + L->top = top + 3; + + VM_PROTECT(luaV_callTM(L, 2, res)); + pc += l_isfalse(&base[res]) ? LUAU_INSN_D(insn) : 1; + LUAU_ASSERT(unsigned(pc - cl->l.p->code) < unsigned(cl->l.p->sizecode)); + return pc; + } + } + // slow path after switch() + break; + + default:; + } + + // slow-path: tables with metatables and userdata values + // note that we don't have a fast path for userdata values without metatables, since that's very rare + int res; + VM_PROTECT(res = luaV_equalval(L, ra, rb)); + + pc += (res == 0) ? LUAU_INSN_D(insn) : 1; + LUAU_ASSERT(unsigned(pc - cl->l.p->code) < unsigned(cl->l.p->sizecode)); + return pc; + } + else + { + pc += LUAU_INSN_D(insn); + LUAU_ASSERT(unsigned(pc - cl->l.p->code) < unsigned(cl->l.p->sizecode)); + return pc; + } +} + +const Instruction* execute_LOP_JUMPIFLE(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k) +{ + Instruction insn = *pc++; + uint32_t aux = *pc; + StkId ra = VM_REG(LUAU_INSN_A(insn)); + StkId rb = VM_REG(aux); + + // fast-path: number + // Note that all jumps below jump by 1 in the "false" case to skip over aux + if (ttisnumber(ra) && ttisnumber(rb)) + { + pc += nvalue(ra) <= nvalue(rb) ? LUAU_INSN_D(insn) : 1; + LUAU_ASSERT(unsigned(pc - cl->l.p->code) < unsigned(cl->l.p->sizecode)); + return pc; + } + // fast-path: string + else if (ttisstring(ra) && ttisstring(rb)) + { + pc += luaV_strcmp(tsvalue(ra), tsvalue(rb)) <= 0 ? LUAU_INSN_D(insn) : 1; + LUAU_ASSERT(unsigned(pc - cl->l.p->code) < unsigned(cl->l.p->sizecode)); + return pc; + } + else + { + int res; + VM_PROTECT(res = luaV_lessequal(L, ra, rb)); + + pc += (res == 1) ? LUAU_INSN_D(insn) : 1; + LUAU_ASSERT(unsigned(pc - cl->l.p->code) < unsigned(cl->l.p->sizecode)); + return pc; + } +} + +const Instruction* execute_LOP_JUMPIFNOTLE(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k) +{ + Instruction insn = *pc++; + uint32_t aux = *pc; + StkId ra = VM_REG(LUAU_INSN_A(insn)); + StkId rb = VM_REG(aux); + + // fast-path: number + // Note that all jumps below jump by 1 in the "true" case to skip over aux + if (ttisnumber(ra) && ttisnumber(rb)) + { + pc += !(nvalue(ra) <= nvalue(rb)) ? LUAU_INSN_D(insn) : 1; + LUAU_ASSERT(unsigned(pc - cl->l.p->code) < unsigned(cl->l.p->sizecode)); + return pc; + } + // fast-path: string + else if (ttisstring(ra) && ttisstring(rb)) + { + pc += !(luaV_strcmp(tsvalue(ra), tsvalue(rb)) <= 0) ? LUAU_INSN_D(insn) : 1; + LUAU_ASSERT(unsigned(pc - cl->l.p->code) < unsigned(cl->l.p->sizecode)); + return pc; + } + else + { + int res; + VM_PROTECT(res = luaV_lessequal(L, ra, rb)); + + pc += (res == 0) ? LUAU_INSN_D(insn) : 1; + LUAU_ASSERT(unsigned(pc - cl->l.p->code) < unsigned(cl->l.p->sizecode)); + return pc; + } +} + +const Instruction* execute_LOP_JUMPIFLT(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k) +{ + Instruction insn = *pc++; + uint32_t aux = *pc; + StkId ra = VM_REG(LUAU_INSN_A(insn)); + StkId rb = VM_REG(aux); + + // fast-path: number + // Note that all jumps below jump by 1 in the "false" case to skip over aux + if (ttisnumber(ra) && ttisnumber(rb)) + { + pc += nvalue(ra) < nvalue(rb) ? LUAU_INSN_D(insn) : 1; + LUAU_ASSERT(unsigned(pc - cl->l.p->code) < unsigned(cl->l.p->sizecode)); + return pc; + } + // fast-path: string + else if (ttisstring(ra) && ttisstring(rb)) + { + pc += luaV_strcmp(tsvalue(ra), tsvalue(rb)) < 0 ? LUAU_INSN_D(insn) : 1; + LUAU_ASSERT(unsigned(pc - cl->l.p->code) < unsigned(cl->l.p->sizecode)); + return pc; + } + else + { + int res; + VM_PROTECT(res = luaV_lessthan(L, ra, rb)); + + pc += (res == 1) ? LUAU_INSN_D(insn) : 1; + LUAU_ASSERT(unsigned(pc - cl->l.p->code) < unsigned(cl->l.p->sizecode)); + return pc; + } +} + +const Instruction* execute_LOP_JUMPIFNOTLT(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k) +{ + Instruction insn = *pc++; + uint32_t aux = *pc; + StkId ra = VM_REG(LUAU_INSN_A(insn)); + StkId rb = VM_REG(aux); + + // fast-path: number + // Note that all jumps below jump by 1 in the "true" case to skip over aux + if (ttisnumber(ra) && ttisnumber(rb)) + { + pc += !(nvalue(ra) < nvalue(rb)) ? LUAU_INSN_D(insn) : 1; + LUAU_ASSERT(unsigned(pc - cl->l.p->code) < unsigned(cl->l.p->sizecode)); + return pc; + } + // fast-path: string + else if (ttisstring(ra) && ttisstring(rb)) + { + pc += !(luaV_strcmp(tsvalue(ra), tsvalue(rb)) < 0) ? LUAU_INSN_D(insn) : 1; + LUAU_ASSERT(unsigned(pc - cl->l.p->code) < unsigned(cl->l.p->sizecode)); + return pc; + } + else + { + int res; + VM_PROTECT(res = luaV_lessthan(L, ra, rb)); + + pc += (res == 0) ? LUAU_INSN_D(insn) : 1; + LUAU_ASSERT(unsigned(pc - cl->l.p->code) < unsigned(cl->l.p->sizecode)); + return pc; + } +} + +const Instruction* execute_LOP_ADD(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k) +{ + Instruction insn = *pc++; + StkId ra = VM_REG(LUAU_INSN_A(insn)); + StkId rb = VM_REG(LUAU_INSN_B(insn)); + StkId rc = VM_REG(LUAU_INSN_C(insn)); + + // fast-path + if (ttisnumber(rb) && ttisnumber(rc)) + { + setnvalue(ra, nvalue(rb) + nvalue(rc)); + return pc; + } + else if (ttisvector(rb) && ttisvector(rc)) + { + const float* vb = rb->value.v; + const float* vc = rc->value.v; + setvvalue(ra, vb[0] + vc[0], vb[1] + vc[1], vb[2] + vc[2], vb[3] + vc[3]); + return pc; + } + else + { + // fast-path for userdata with C functions + const TValue* fn = 0; + if (ttisuserdata(rb) && (fn = luaT_gettmbyobj(L, rb, TM_ADD)) && ttisfunction(fn) && clvalue(fn)->isC) + { + // note: it's safe to push arguments past top for complicated reasons (see top of the file) + LUAU_ASSERT(L->top + 3 < L->stack + L->stacksize); + StkId top = L->top; + setobj2s(L, top + 0, fn); + setobj2s(L, top + 1, rb); + setobj2s(L, top + 2, rc); + L->top = top + 3; + + VM_PROTECT(luaV_callTM(L, 2, LUAU_INSN_A(insn))); + return pc; + } + else + { + // slow-path, may invoke C/Lua via metamethods + VM_PROTECT(luaV_doarith(L, ra, rb, rc, TM_ADD)); + return pc; + } + } +} + +const Instruction* execute_LOP_SUB(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k) +{ + Instruction insn = *pc++; + StkId ra = VM_REG(LUAU_INSN_A(insn)); + StkId rb = VM_REG(LUAU_INSN_B(insn)); + StkId rc = VM_REG(LUAU_INSN_C(insn)); + + // fast-path + if (ttisnumber(rb) && ttisnumber(rc)) + { + setnvalue(ra, nvalue(rb) - nvalue(rc)); + return pc; + } + else if (ttisvector(rb) && ttisvector(rc)) + { + const float* vb = rb->value.v; + const float* vc = rc->value.v; + setvvalue(ra, vb[0] - vc[0], vb[1] - vc[1], vb[2] - vc[2], vb[3] - vc[3]); + return pc; + } + else + { + // fast-path for userdata with C functions + const TValue* fn = 0; + if (ttisuserdata(rb) && (fn = luaT_gettmbyobj(L, rb, TM_SUB)) && ttisfunction(fn) && clvalue(fn)->isC) + { + // note: it's safe to push arguments past top for complicated reasons (see top of the file) + LUAU_ASSERT(L->top + 3 < L->stack + L->stacksize); + StkId top = L->top; + setobj2s(L, top + 0, fn); + setobj2s(L, top + 1, rb); + setobj2s(L, top + 2, rc); + L->top = top + 3; + + VM_PROTECT(luaV_callTM(L, 2, LUAU_INSN_A(insn))); + return pc; + } + else + { + // slow-path, may invoke C/Lua via metamethods + VM_PROTECT(luaV_doarith(L, ra, rb, rc, TM_SUB)); + return pc; + } + } +} + +const Instruction* execute_LOP_MUL(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k) +{ + Instruction insn = *pc++; + StkId ra = VM_REG(LUAU_INSN_A(insn)); + StkId rb = VM_REG(LUAU_INSN_B(insn)); + StkId rc = VM_REG(LUAU_INSN_C(insn)); + + // fast-path + if (ttisnumber(rb) && ttisnumber(rc)) + { + setnvalue(ra, nvalue(rb) * nvalue(rc)); + return pc; + } + else if (ttisvector(rb) && ttisnumber(rc)) + { + const float* vb = rb->value.v; + float vc = cast_to(float, nvalue(rc)); + setvvalue(ra, vb[0] * vc, vb[1] * vc, vb[2] * vc, vb[3] * vc); + return pc; + } + else if (ttisvector(rb) && ttisvector(rc)) + { + const float* vb = rb->value.v; + const float* vc = rc->value.v; + setvvalue(ra, vb[0] * vc[0], vb[1] * vc[1], vb[2] * vc[2], vb[3] * vc[3]); + return pc; + } + else if (ttisnumber(rb) && ttisvector(rc)) + { + float vb = cast_to(float, nvalue(rb)); + const float* vc = rc->value.v; + setvvalue(ra, vb * vc[0], vb * vc[1], vb * vc[2], vb * vc[3]); + return pc; + } + else + { + // fast-path for userdata with C functions + StkId rbc = ttisnumber(rb) ? rc : rb; + const TValue* fn = 0; + if (ttisuserdata(rbc) && (fn = luaT_gettmbyobj(L, rbc, TM_MUL)) && ttisfunction(fn) && clvalue(fn)->isC) + { + // note: it's safe to push arguments past top for complicated reasons (see top of the file) + LUAU_ASSERT(L->top + 3 < L->stack + L->stacksize); + StkId top = L->top; + setobj2s(L, top + 0, fn); + setobj2s(L, top + 1, rb); + setobj2s(L, top + 2, rc); + L->top = top + 3; + + VM_PROTECT(luaV_callTM(L, 2, LUAU_INSN_A(insn))); + return pc; + } + else + { + // slow-path, may invoke C/Lua via metamethods + VM_PROTECT(luaV_doarith(L, ra, rb, rc, TM_MUL)); + return pc; + } + } +} + +const Instruction* execute_LOP_DIV(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k) +{ + Instruction insn = *pc++; + StkId ra = VM_REG(LUAU_INSN_A(insn)); + StkId rb = VM_REG(LUAU_INSN_B(insn)); + StkId rc = VM_REG(LUAU_INSN_C(insn)); + + // fast-path + if (ttisnumber(rb) && ttisnumber(rc)) + { + setnvalue(ra, nvalue(rb) / nvalue(rc)); + return pc; + } + else if (ttisvector(rb) && ttisnumber(rc)) + { + const float* vb = rb->value.v; + float vc = cast_to(float, nvalue(rc)); + setvvalue(ra, vb[0] / vc, vb[1] / vc, vb[2] / vc, vb[3] / vc); + return pc; + } + else if (ttisvector(rb) && ttisvector(rc)) + { + const float* vb = rb->value.v; + const float* vc = rc->value.v; + setvvalue(ra, vb[0] / vc[0], vb[1] / vc[1], vb[2] / vc[2], vb[3] / vc[3]); + return pc; + } + else if (ttisnumber(rb) && ttisvector(rc)) + { + float vb = cast_to(float, nvalue(rb)); + const float* vc = rc->value.v; + setvvalue(ra, vb / vc[0], vb / vc[1], vb / vc[2], vb / vc[3]); + return pc; + } + else + { + // fast-path for userdata with C functions + StkId rbc = ttisnumber(rb) ? rc : rb; + const TValue* fn = 0; + if (ttisuserdata(rbc) && (fn = luaT_gettmbyobj(L, rbc, TM_DIV)) && ttisfunction(fn) && clvalue(fn)->isC) + { + // note: it's safe to push arguments past top for complicated reasons (see top of the file) + LUAU_ASSERT(L->top + 3 < L->stack + L->stacksize); + StkId top = L->top; + setobj2s(L, top + 0, fn); + setobj2s(L, top + 1, rb); + setobj2s(L, top + 2, rc); + L->top = top + 3; + + VM_PROTECT(luaV_callTM(L, 2, LUAU_INSN_A(insn))); + return pc; + } + else + { + // slow-path, may invoke C/Lua via metamethods + VM_PROTECT(luaV_doarith(L, ra, rb, rc, TM_DIV)); + return pc; + } + } +} + +const Instruction* execute_LOP_MOD(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k) +{ + Instruction insn = *pc++; + StkId ra = VM_REG(LUAU_INSN_A(insn)); + StkId rb = VM_REG(LUAU_INSN_B(insn)); + StkId rc = VM_REG(LUAU_INSN_C(insn)); + + // fast-path + if (ttisnumber(rb) && ttisnumber(rc)) + { + double nb = nvalue(rb); + double nc = nvalue(rc); + setnvalue(ra, luai_nummod(nb, nc)); + return pc; + } + else + { + // slow-path, may invoke C/Lua via metamethods + VM_PROTECT(luaV_doarith(L, ra, rb, rc, TM_MOD)); + return pc; + } +} + +const Instruction* execute_LOP_POW(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k) +{ + Instruction insn = *pc++; + StkId ra = VM_REG(LUAU_INSN_A(insn)); + StkId rb = VM_REG(LUAU_INSN_B(insn)); + StkId rc = VM_REG(LUAU_INSN_C(insn)); + + // fast-path + if (ttisnumber(rb) && ttisnumber(rc)) + { + setnvalue(ra, pow(nvalue(rb), nvalue(rc))); + return pc; + } + else + { + // slow-path, may invoke C/Lua via metamethods + VM_PROTECT(luaV_doarith(L, ra, rb, rc, TM_POW)); + return pc; + } +} + +const Instruction* execute_LOP_ADDK(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k) +{ + Instruction insn = *pc++; + StkId ra = VM_REG(LUAU_INSN_A(insn)); + StkId rb = VM_REG(LUAU_INSN_B(insn)); + TValue* kv = VM_KV(LUAU_INSN_C(insn)); + + // fast-path + if (ttisnumber(rb)) + { + setnvalue(ra, nvalue(rb) + nvalue(kv)); + return pc; + } + else + { + // slow-path, may invoke C/Lua via metamethods + VM_PROTECT(luaV_doarith(L, ra, rb, kv, TM_ADD)); + return pc; + } +} + +const Instruction* execute_LOP_SUBK(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k) +{ + Instruction insn = *pc++; + StkId ra = VM_REG(LUAU_INSN_A(insn)); + StkId rb = VM_REG(LUAU_INSN_B(insn)); + TValue* kv = VM_KV(LUAU_INSN_C(insn)); + + // fast-path + if (ttisnumber(rb)) + { + setnvalue(ra, nvalue(rb) - nvalue(kv)); + return pc; + } + else + { + // slow-path, may invoke C/Lua via metamethods + VM_PROTECT(luaV_doarith(L, ra, rb, kv, TM_SUB)); + return pc; + } +} + +const Instruction* execute_LOP_MULK(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k) +{ + Instruction insn = *pc++; + StkId ra = VM_REG(LUAU_INSN_A(insn)); + StkId rb = VM_REG(LUAU_INSN_B(insn)); + TValue* kv = VM_KV(LUAU_INSN_C(insn)); + + // fast-path + if (ttisnumber(rb)) + { + setnvalue(ra, nvalue(rb) * nvalue(kv)); + return pc; + } + else if (ttisvector(rb)) + { + const float* vb = rb->value.v; + float vc = cast_to(float, nvalue(kv)); + setvvalue(ra, vb[0] * vc, vb[1] * vc, vb[2] * vc, vb[3] * vc); + return pc; + } + else + { + // fast-path for userdata with C functions + const TValue* fn = 0; + if (ttisuserdata(rb) && (fn = luaT_gettmbyobj(L, rb, TM_MUL)) && ttisfunction(fn) && clvalue(fn)->isC) + { + // note: it's safe to push arguments past top for complicated reasons (see top of the file) + LUAU_ASSERT(L->top + 3 < L->stack + L->stacksize); + StkId top = L->top; + setobj2s(L, top + 0, fn); + setobj2s(L, top + 1, rb); + setobj2s(L, top + 2, kv); + L->top = top + 3; + + VM_PROTECT(luaV_callTM(L, 2, LUAU_INSN_A(insn))); + return pc; + } + else + { + // slow-path, may invoke C/Lua via metamethods + VM_PROTECT(luaV_doarith(L, ra, rb, kv, TM_MUL)); + return pc; + } + } +} + +const Instruction* execute_LOP_DIVK(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k) +{ + Instruction insn = *pc++; + StkId ra = VM_REG(LUAU_INSN_A(insn)); + StkId rb = VM_REG(LUAU_INSN_B(insn)); + TValue* kv = VM_KV(LUAU_INSN_C(insn)); + + // fast-path + if (ttisnumber(rb)) + { + setnvalue(ra, nvalue(rb) / nvalue(kv)); + return pc; + } + else if (ttisvector(rb)) + { + const float* vb = rb->value.v; + float vc = cast_to(float, nvalue(kv)); + setvvalue(ra, vb[0] / vc, vb[1] / vc, vb[2] / vc, vb[3] / vc); + return pc; + } + else + { + // fast-path for userdata with C functions + const TValue* fn = 0; + if (ttisuserdata(rb) && (fn = luaT_gettmbyobj(L, rb, TM_DIV)) && ttisfunction(fn) && clvalue(fn)->isC) + { + // note: it's safe to push arguments past top for complicated reasons (see top of the file) + LUAU_ASSERT(L->top + 3 < L->stack + L->stacksize); + StkId top = L->top; + setobj2s(L, top + 0, fn); + setobj2s(L, top + 1, rb); + setobj2s(L, top + 2, kv); + L->top = top + 3; + + VM_PROTECT(luaV_callTM(L, 2, LUAU_INSN_A(insn))); + return pc; + } + else + { + // slow-path, may invoke C/Lua via metamethods + VM_PROTECT(luaV_doarith(L, ra, rb, kv, TM_DIV)); + return pc; + } + } +} + +const Instruction* execute_LOP_MODK(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k) +{ + Instruction insn = *pc++; + StkId ra = VM_REG(LUAU_INSN_A(insn)); + StkId rb = VM_REG(LUAU_INSN_B(insn)); + TValue* kv = VM_KV(LUAU_INSN_C(insn)); + + // fast-path + if (ttisnumber(rb)) + { + double nb = nvalue(rb); + double nk = nvalue(kv); + setnvalue(ra, luai_nummod(nb, nk)); + return pc; + } + else + { + // slow-path, may invoke C/Lua via metamethods + VM_PROTECT(luaV_doarith(L, ra, rb, kv, TM_MOD)); + return pc; + } +} + +const Instruction* execute_LOP_POWK(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k) +{ + Instruction insn = *pc++; + StkId ra = VM_REG(LUAU_INSN_A(insn)); + StkId rb = VM_REG(LUAU_INSN_B(insn)); + TValue* kv = VM_KV(LUAU_INSN_C(insn)); + + // fast-path + if (ttisnumber(rb)) + { + double nb = nvalue(rb); + double nk = nvalue(kv); + + // pow is very slow so we specialize this for ^2, ^0.5 and ^3 + double r = (nk == 2.0) ? nb * nb : (nk == 0.5) ? sqrt(nb) : (nk == 3.0) ? nb * nb * nb : pow(nb, nk); + + setnvalue(ra, r); + return pc; + } + else + { + // slow-path, may invoke C/Lua via metamethods + VM_PROTECT(luaV_doarith(L, ra, rb, kv, TM_POW)); + return pc; + } +} + +const Instruction* execute_LOP_AND(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k) +{ + Instruction insn = *pc++; + StkId ra = VM_REG(LUAU_INSN_A(insn)); + StkId rb = VM_REG(LUAU_INSN_B(insn)); + StkId rc = VM_REG(LUAU_INSN_C(insn)); + + setobj2s(L, ra, l_isfalse(rb) ? rb : rc); + return pc; +} + +const Instruction* execute_LOP_OR(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k) +{ + Instruction insn = *pc++; + StkId ra = VM_REG(LUAU_INSN_A(insn)); + StkId rb = VM_REG(LUAU_INSN_B(insn)); + StkId rc = VM_REG(LUAU_INSN_C(insn)); + + setobj2s(L, ra, l_isfalse(rb) ? rc : rb); + return pc; +} + +const Instruction* execute_LOP_ANDK(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k) +{ + Instruction insn = *pc++; + StkId ra = VM_REG(LUAU_INSN_A(insn)); + StkId rb = VM_REG(LUAU_INSN_B(insn)); + TValue* kv = VM_KV(LUAU_INSN_C(insn)); + + setobj2s(L, ra, l_isfalse(rb) ? rb : kv); + return pc; +} + +const Instruction* execute_LOP_ORK(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k) +{ + Instruction insn = *pc++; + StkId ra = VM_REG(LUAU_INSN_A(insn)); + StkId rb = VM_REG(LUAU_INSN_B(insn)); + TValue* kv = VM_KV(LUAU_INSN_C(insn)); + + setobj2s(L, ra, l_isfalse(rb) ? kv : rb); + return pc; +} + +const Instruction* execute_LOP_CONCAT(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k) +{ + Instruction insn = *pc++; + int b = LUAU_INSN_B(insn); + int c = LUAU_INSN_C(insn); + + // This call may realloc the stack! So we need to query args further down + VM_PROTECT(luaV_concat(L, c - b + 1, c)); + + StkId ra = VM_REG(LUAU_INSN_A(insn)); + + setobjs2s(L, ra, base + b); + VM_PROTECT(luaC_checkGC(L)); + return pc; +} + +const Instruction* execute_LOP_NOT(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k) +{ + Instruction insn = *pc++; + StkId ra = VM_REG(LUAU_INSN_A(insn)); + StkId rb = VM_REG(LUAU_INSN_B(insn)); + + int res = l_isfalse(rb); + setbvalue(ra, res); + return pc; +} + +const Instruction* execute_LOP_MINUS(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k) +{ + Instruction insn = *pc++; + StkId ra = VM_REG(LUAU_INSN_A(insn)); + StkId rb = VM_REG(LUAU_INSN_B(insn)); + + // fast-path + if (ttisnumber(rb)) + { + setnvalue(ra, -nvalue(rb)); + return pc; + } + else if (ttisvector(rb)) + { + const float* vb = rb->value.v; + setvvalue(ra, -vb[0], -vb[1], -vb[2], -vb[3]); + return pc; + } + else + { + // fast-path for userdata with C functions + const TValue* fn = 0; + if (ttisuserdata(rb) && (fn = luaT_gettmbyobj(L, rb, TM_UNM)) && ttisfunction(fn) && clvalue(fn)->isC) + { + // note: it's safe to push arguments past top for complicated reasons (see top of the file) + LUAU_ASSERT(L->top + 2 < L->stack + L->stacksize); + StkId top = L->top; + setobj2s(L, top + 0, fn); + setobj2s(L, top + 1, rb); + L->top = top + 2; + + VM_PROTECT(luaV_callTM(L, 1, LUAU_INSN_A(insn))); + return pc; + } + else + { + // slow-path, may invoke C/Lua via metamethods + VM_PROTECT(luaV_doarith(L, ra, rb, rb, TM_UNM)); + return pc; + } + } +} + +const Instruction* execute_LOP_LENGTH(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k) +{ + Instruction insn = *pc++; + StkId ra = VM_REG(LUAU_INSN_A(insn)); + StkId rb = VM_REG(LUAU_INSN_B(insn)); + + // fast-path #1: tables + if (ttistable(rb)) + { + Table* h = hvalue(rb); + + if (fastnotm(h->metatable, TM_LEN)) + { + setnvalue(ra, cast_num(luaH_getn(h))); + return pc; + } + else + { + // slow-path, may invoke C/Lua via metamethods + VM_PROTECT(luaV_dolen(L, ra, rb)); + return pc; + } + } + // fast-path #2: strings (not very important but easy to do) + else if (ttisstring(rb)) + { + TString* ts = tsvalue(rb); + setnvalue(ra, cast_num(ts->len)); + return pc; + } + else + { + // slow-path, may invoke C/Lua via metamethods + VM_PROTECT(luaV_dolen(L, ra, rb)); + return pc; + } +} + +const Instruction* execute_LOP_NEWTABLE(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k) +{ + Instruction insn = *pc++; + StkId ra = VM_REG(LUAU_INSN_A(insn)); + int b = LUAU_INSN_B(insn); + uint32_t aux = *pc++; + + sethvalue(L, ra, luaH_new(L, aux, b == 0 ? 0 : (1 << (b - 1)))); + VM_PROTECT(luaC_checkGC(L)); + return pc; +} + +const Instruction* execute_LOP_DUPTABLE(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k) +{ + Instruction insn = *pc++; + StkId ra = VM_REG(LUAU_INSN_A(insn)); + TValue* kv = VM_KV(LUAU_INSN_D(insn)); + + sethvalue(L, ra, luaH_clone(L, hvalue(kv))); + VM_PROTECT(luaC_checkGC(L)); + return pc; +} + +const Instruction* execute_LOP_SETLIST(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k) +{ + Instruction insn = *pc++; + StkId ra = VM_REG(LUAU_INSN_A(insn)); + StkId rb = &base[LUAU_INSN_B(insn)]; // note: this can point to L->top if c == LUA_MULTRET making VM_REG unsafe to use + int c = LUAU_INSN_C(insn) - 1; + uint32_t index = *pc++; + + if (c == LUA_MULTRET) + { + c = int(L->top - rb); + L->top = L->ci->top; + } + + Table* h = hvalue(ra); + + if (!ttistable(ra)) + return NULL; // temporary workaround to weaken a rather powerful exploitation primitive in case of a MITM attack on bytecode + + int last = index + c - 1; + if (last > h->sizearray) + luaH_resizearray(L, h, last); + + TValue* array = h->array; + + for (int i = 0; i < c; ++i) + setobj2t(L, &array[index + i - 1], rb + i); + + luaC_barrierfast(L, h); + return pc; +} + +const Instruction* execute_LOP_FORNPREP(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k) +{ + Instruction insn = *pc++; + StkId ra = VM_REG(LUAU_INSN_A(insn)); + + if (!ttisnumber(ra + 0) || !ttisnumber(ra + 1) || !ttisnumber(ra + 2)) + { + // slow-path: can convert arguments to numbers and trigger Lua errors + // Note: this doesn't reallocate stack so we don't need to recompute ra/base + VM_PROTECT_PC(); + + luaV_prepareFORN(L, ra + 0, ra + 1, ra + 2); + } + + double limit = nvalue(ra + 0); + double step = nvalue(ra + 1); + double idx = nvalue(ra + 2); + + // Note: make sure the loop condition is exactly the same between this and LOP_FORNLOOP so that we handle NaN/etc. consistently + pc += (step > 0 ? idx <= limit : limit <= idx) ? 0 : LUAU_INSN_D(insn); + LUAU_ASSERT(unsigned(pc - cl->l.p->code) < unsigned(cl->l.p->sizecode)); + return pc; +} + +const Instruction* execute_LOP_FORNLOOP(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k) +{ + VM_INTERRUPT(); + Instruction insn = *pc++; + StkId ra = VM_REG(LUAU_INSN_A(insn)); + LUAU_ASSERT(ttisnumber(ra + 0) && ttisnumber(ra + 1) && ttisnumber(ra + 2)); + + double limit = nvalue(ra + 0); + double step = nvalue(ra + 1); + double idx = nvalue(ra + 2) + step; + + setnvalue(ra + 2, idx); + + // Note: make sure the loop condition is exactly the same between this and LOP_FORNPREP so that we handle NaN/etc. consistently + if (step > 0 ? idx <= limit : limit <= idx) + { + pc += LUAU_INSN_D(insn); + LUAU_ASSERT(unsigned(pc - cl->l.p->code) < unsigned(cl->l.p->sizecode)); + return pc; + } + else + { + // fallthrough to exit + return pc; + } +} + +const Instruction* execute_LOP_FORGPREP(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k) +{ + Instruction insn = *pc++; + StkId ra = VM_REG(LUAU_INSN_A(insn)); + + if (ttisfunction(ra)) + { + // will be called during FORGLOOP + } + else + { + Table* mt = ttistable(ra) ? hvalue(ra)->metatable : ttisuserdata(ra) ? uvalue(ra)->metatable : cast_to(Table*, NULL); + + if (const TValue* fn = fasttm(L, mt, TM_ITER)) + { + setobj2s(L, ra + 1, ra); + setobj2s(L, ra, fn); + + L->top = ra + 2; // func + self arg + LUAU_ASSERT(L->top <= L->stack_last); + + VM_PROTECT(luaD_call(L, ra, 3)); + L->top = L->ci->top; + + // recompute ra since stack might have been reallocated + ra = VM_REG(LUAU_INSN_A(insn)); + + // protect against __iter returning nil, since nil is used as a marker for builtin iteration in FORGLOOP + if (ttisnil(ra)) + { + VM_PROTECT(luaG_typeerror(L, ra, "call")); + } + } + else if (fasttm(L, mt, TM_CALL)) + { + // table or userdata with __call, will be called during FORGLOOP + // TODO: we might be able to stop supporting this depending on whether it's used in practice + } + else if (ttistable(ra)) + { + // set up registers for builtin iteration + setobj2s(L, ra + 1, ra); + setpvalue(ra + 2, reinterpret_cast(uintptr_t(0))); + setnilvalue(ra); + } + else + { + VM_PROTECT(luaG_typeerror(L, ra, "iterate over")); + } + } + + pc += LUAU_INSN_D(insn); + LUAU_ASSERT(unsigned(pc - cl->l.p->code) < unsigned(cl->l.p->sizecode)); + return pc; +} + +const Instruction* execute_LOP_FORGLOOP(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k) +{ + VM_INTERRUPT(); + Instruction insn = *pc++; + StkId ra = VM_REG(LUAU_INSN_A(insn)); + uint32_t aux = *pc; + + // fast-path: builtin table iteration + // note: ra=nil guarantees ra+1=table and ra+2=userdata because of the setup by FORGPREP* opcodes + // TODO: remove the table check per guarantee above + if (ttisnil(ra) && ttistable(ra + 1)) + { + Table* h = hvalue(ra + 1); + int index = int(reinterpret_cast(pvalue(ra + 2))); + + int sizearray = h->sizearray; + + // clear extra variables since we might have more than two + // note: while aux encodes ipairs bit, when set we always use 2 variables, so it's safe to check this via a signed comparison + if (LUAU_UNLIKELY(int(aux) > 2)) + for (int i = 2; i < int(aux); ++i) + setnilvalue(ra + 3 + i); + + // terminate ipairs-style traversal early when encountering nil + if (int(aux) < 0 && (unsigned(index) >= unsigned(sizearray) || ttisnil(&h->array[index]))) + { + pc++; + return pc; + } + + // first we advance index through the array portion + while (unsigned(index) < unsigned(sizearray)) + { + TValue* e = &h->array[index]; + + if (!ttisnil(e)) + { + setpvalue(ra + 2, reinterpret_cast(uintptr_t(index + 1))); + setnvalue(ra + 3, double(index + 1)); + setobj2s(L, ra + 4, e); + + pc += LUAU_INSN_D(insn); + LUAU_ASSERT(unsigned(pc - cl->l.p->code) < unsigned(cl->l.p->sizecode)); + return pc; + } + + index++; + } + + int sizenode = 1 << h->lsizenode; + + // then we advance index through the hash portion + while (unsigned(index - sizearray) < unsigned(sizenode)) + { + LuaNode* n = &h->node[index - sizearray]; + + if (!ttisnil(gval(n))) + { + setpvalue(ra + 2, reinterpret_cast(uintptr_t(index + 1))); + getnodekey(L, ra + 3, n); + setobj2s(L, ra + 4, gval(n)); + + pc += LUAU_INSN_D(insn); + LUAU_ASSERT(unsigned(pc - cl->l.p->code) < unsigned(cl->l.p->sizecode)); + return pc; + } + + index++; + } + + // fallthrough to exit + pc++; + return pc; + } + else + { + // note: it's safe to push arguments past top for complicated reasons (see top of the file) + setobjs2s(L, ra + 3 + 2, ra + 2); + setobjs2s(L, ra + 3 + 1, ra + 1); + setobjs2s(L, ra + 3, ra); + + L->top = ra + 3 + 3; // func + 2 args (state and index) + LUAU_ASSERT(L->top <= L->stack_last); + + VM_PROTECT(luaD_call(L, ra + 3, uint8_t(aux))); + L->top = L->ci->top; + + // recompute ra since stack might have been reallocated + ra = VM_REG(LUAU_INSN_A(insn)); + + // copy first variable back into the iteration index + setobjs2s(L, ra + 2, ra + 3); + + // note that we need to increment pc by 1 to exit the loop since we need to skip over aux + pc += ttisnil(ra + 3) ? 1 : LUAU_INSN_D(insn); + LUAU_ASSERT(unsigned(pc - cl->l.p->code) < unsigned(cl->l.p->sizecode)); + return pc; + } +} + +const Instruction* execute_LOP_FORGPREP_INEXT(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k) +{ + Instruction insn = *pc++; + StkId ra = VM_REG(LUAU_INSN_A(insn)); + + // fast-path: ipairs/inext + if (cl->env->safeenv && ttistable(ra + 1) && ttisnumber(ra + 2) && nvalue(ra + 2) == 0.0) + { + setnilvalue(ra); + // ra+1 is already the table + setpvalue(ra + 2, reinterpret_cast(uintptr_t(0))); + } + else if (!ttisfunction(ra)) + { + VM_PROTECT(luaG_typeerror(L, ra, "iterate over")); + } + + pc += LUAU_INSN_D(insn); + LUAU_ASSERT(unsigned(pc - cl->l.p->code) < unsigned(cl->l.p->sizecode)); + return pc; +} + +const Instruction* execute_LOP_DEP_FORGLOOP_INEXT(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k) +{ + LUAU_ASSERT(!"Unsupported deprecated opcode"); + LUAU_UNREACHABLE(); +} + +const Instruction* execute_LOP_FORGPREP_NEXT(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k) +{ + Instruction insn = *pc++; + StkId ra = VM_REG(LUAU_INSN_A(insn)); + + // fast-path: pairs/next + if (cl->env->safeenv && ttistable(ra + 1) && ttisnil(ra + 2)) + { + setnilvalue(ra); + // ra+1 is already the table + setpvalue(ra + 2, reinterpret_cast(uintptr_t(0))); + } + else if (!ttisfunction(ra)) + { + VM_PROTECT(luaG_typeerror(L, ra, "iterate over")); + } + + pc += LUAU_INSN_D(insn); + LUAU_ASSERT(unsigned(pc - cl->l.p->code) < unsigned(cl->l.p->sizecode)); + return pc; +} + +const Instruction* execute_LOP_DEP_FORGLOOP_NEXT(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k) +{ + LUAU_ASSERT(!"Unsupported deprecated opcode"); + LUAU_UNREACHABLE(); +} + +const Instruction* execute_LOP_GETVARARGS(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k) +{ + Instruction insn = *pc++; + int b = LUAU_INSN_B(insn) - 1; + int n = cast_int(base - L->ci->func) - cl->l.p->numparams - 1; + + if (b == LUA_MULTRET) + { + VM_PROTECT(luaD_checkstack(L, n)); + StkId ra = VM_REG(LUAU_INSN_A(insn)); // previous call may change the stack + + for (int j = 0; j < n; j++) + setobjs2s(L, ra + j, base - n + j); + + L->top = ra + n; + return pc; + } + else + { + StkId ra = VM_REG(LUAU_INSN_A(insn)); + + for (int j = 0; j < b && j < n; j++) + setobjs2s(L, ra + j, base - n + j); + for (int j = n; j < b; j++) + setnilvalue(ra + j); + return pc; + } +} + +const Instruction* execute_LOP_DUPCLOSURE(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k) +{ + Instruction insn = *pc++; + StkId ra = VM_REG(LUAU_INSN_A(insn)); + TValue* kv = VM_KV(LUAU_INSN_D(insn)); + + Closure* kcl = clvalue(kv); + + // clone closure if the environment is not shared + // note: we save closure to stack early in case the code below wants to capture it by value + Closure* ncl = (kcl->env == cl->env) ? kcl : luaF_newLclosure(L, kcl->nupvalues, cl->env, kcl->l.p); + setclvalue(L, ra, ncl); + + // this loop does three things: + // - if the closure was created anew, it just fills it with upvalues + // - if the closure from the constant table is used, it fills it with upvalues so that it can be shared in the future + // - if the closure is reused, it checks if the reuse is safe via rawequal, and falls back to duplicating the closure + // normally this would use two separate loops, for reuse check and upvalue setup, but MSVC codegen goes crazy if you do that + for (int ui = 0; ui < kcl->nupvalues; ++ui) + { + Instruction uinsn = pc[ui]; + LUAU_ASSERT(LUAU_INSN_OP(uinsn) == LOP_CAPTURE); + LUAU_ASSERT(LUAU_INSN_A(uinsn) == LCT_VAL || LUAU_INSN_A(uinsn) == LCT_UPVAL); + + TValue* uv = (LUAU_INSN_A(uinsn) == LCT_VAL) ? VM_REG(LUAU_INSN_B(uinsn)) : VM_UV(LUAU_INSN_B(uinsn)); + + // check if the existing closure is safe to reuse + if (ncl == kcl && luaO_rawequalObj(&ncl->l.uprefs[ui], uv)) + continue; + + // lazily clone the closure and update the upvalues + if (ncl == kcl && kcl->preload == 0) + { + ncl = luaF_newLclosure(L, kcl->nupvalues, cl->env, kcl->l.p); + setclvalue(L, ra, ncl); + + ui = -1; // restart the loop to fill all upvalues + continue; + } + + // this updates a newly created closure, or an existing closure created during preload, in which case we need a barrier + setobj(L, &ncl->l.uprefs[ui], uv); + luaC_barrier(L, ncl, uv); + } + + // this is a noop if ncl is newly created or shared successfully, but it has to run after the closure is preloaded for the first time + ncl->preload = 0; + + if (kcl != ncl) + VM_PROTECT(luaC_checkGC(L)); + + pc += kcl->nupvalues; + return pc; +} + +const Instruction* execute_LOP_PREPVARARGS(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k) +{ + Instruction insn = *pc++; + int numparams = LUAU_INSN_A(insn); + + // all fixed parameters are copied after the top so we need more stack space + VM_PROTECT(luaD_checkstack(L, cl->stacksize + numparams)); + + // the caller must have filled extra fixed arguments with nil + LUAU_ASSERT(cast_int(L->top - base) >= numparams); + + // move fixed parameters to final position + StkId fixed = base; // first fixed argument + base = L->top; // final position of first argument + + for (int i = 0; i < numparams; ++i) + { + setobjs2s(L, base + i, fixed + i); + setnilvalue(fixed + i); + } + + // rewire our stack frame to point to the new base + L->ci->base = base; + L->ci->top = base + cl->stacksize; + + L->base = base; + L->top = L->ci->top; + return pc; +} + +const Instruction* execute_LOP_JUMPBACK(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k) +{ + VM_INTERRUPT(); + Instruction insn = *pc++; + + pc += LUAU_INSN_D(insn); + LUAU_ASSERT(unsigned(pc - cl->l.p->code) < unsigned(cl->l.p->sizecode)); + return pc; +} + +const Instruction* execute_LOP_LOADKX(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k) +{ + Instruction insn = *pc++; + StkId ra = VM_REG(LUAU_INSN_A(insn)); + uint32_t aux = *pc++; + TValue* kv = VM_KV(aux); + + setobj2s(L, ra, kv); + return pc; +} + +const Instruction* execute_LOP_JUMPX(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k) +{ + VM_INTERRUPT(); + Instruction insn = *pc++; + + pc += LUAU_INSN_E(insn); + LUAU_ASSERT(unsigned(pc - cl->l.p->code) < unsigned(cl->l.p->sizecode)); + return pc; +} + +const Instruction* execute_LOP_FASTCALL(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k) +{ + Instruction insn = *pc++; + int bfid = LUAU_INSN_A(insn); + int skip = LUAU_INSN_C(insn); + LUAU_ASSERT(unsigned(pc - cl->l.p->code + skip) < unsigned(cl->l.p->sizecode)); + + Instruction call = pc[skip]; + LUAU_ASSERT(LUAU_INSN_OP(call) == LOP_CALL); + + StkId ra = VM_REG(LUAU_INSN_A(call)); + + int nparams = LUAU_INSN_B(call) - 1; + int nresults = LUAU_INSN_C(call) - 1; + + nparams = (nparams == LUA_MULTRET) ? int(L->top - ra - 1) : nparams; + + luau_FastFunction f = luauF_table[bfid]; + + if (cl->env->safeenv && f) + { + VM_PROTECT_PC(); + + int n = f(L, ra, ra + 1, nresults, ra + 2, nparams); + + if (n >= 0) + { + L->top = (nresults == LUA_MULTRET) ? ra + n : L->ci->top; + + pc += skip + 1; // skip instructions that compute function as well as CALL + LUAU_ASSERT(unsigned(pc - cl->l.p->code) < unsigned(cl->l.p->sizecode)); + return pc; + } + else + { + // continue execution through the fallback code + return pc; + } + } + else + { + // continue execution through the fallback code + return pc; + } +} + +const Instruction* execute_LOP_COVERAGE(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k) +{ + Instruction insn = *pc++; + int hits = LUAU_INSN_E(insn); + + // update hits with saturated add and patch the instruction in place + hits = (hits < (1 << 23) - 1) ? hits + 1 : hits; + VM_PATCH_E(pc - 1, hits); + + return pc; +} + +const Instruction* execute_LOP_CAPTURE(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k) +{ + LUAU_ASSERT(!"CAPTURE is a pseudo-opcode and must be executed as part of NEWCLOSURE"); + LUAU_UNREACHABLE(); +} + +const Instruction* execute_LOP_DEP_JUMPIFEQK(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k) +{ + LUAU_ASSERT(!"Unsupported deprecated opcode"); + LUAU_UNREACHABLE(); +} + +const Instruction* execute_LOP_DEP_JUMPIFNOTEQK(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k) +{ + LUAU_ASSERT(!"Unsupported deprecated opcode"); + LUAU_UNREACHABLE(); +} + +const Instruction* execute_LOP_FASTCALL1(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k) +{ + Instruction insn = *pc++; + int bfid = LUAU_INSN_A(insn); + TValue* arg = VM_REG(LUAU_INSN_B(insn)); + int skip = LUAU_INSN_C(insn); + + LUAU_ASSERT(unsigned(pc - cl->l.p->code + skip) < unsigned(cl->l.p->sizecode)); + + Instruction call = pc[skip]; + LUAU_ASSERT(LUAU_INSN_OP(call) == LOP_CALL); + + StkId ra = VM_REG(LUAU_INSN_A(call)); + + int nparams = 1; + int nresults = LUAU_INSN_C(call) - 1; + + luau_FastFunction f = luauF_table[bfid]; + + if (cl->env->safeenv && f) + { + VM_PROTECT_PC(); + + int n = f(L, ra, arg, nresults, NULL, nparams); + + if (n >= 0) + { + L->top = (nresults == LUA_MULTRET) ? ra + n : L->ci->top; + + pc += skip + 1; // skip instructions that compute function as well as CALL + LUAU_ASSERT(unsigned(pc - cl->l.p->code) < unsigned(cl->l.p->sizecode)); + return pc; + } + else + { + // continue execution through the fallback code + return pc; + } + } + else + { + // continue execution through the fallback code + return pc; + } +} + +const Instruction* execute_LOP_FASTCALL2(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k) +{ + Instruction insn = *pc++; + int bfid = LUAU_INSN_A(insn); + int skip = LUAU_INSN_C(insn) - 1; + uint32_t aux = *pc++; + TValue* arg1 = VM_REG(LUAU_INSN_B(insn)); + TValue* arg2 = VM_REG(aux); + + LUAU_ASSERT(unsigned(pc - cl->l.p->code + skip) < unsigned(cl->l.p->sizecode)); + + Instruction call = pc[skip]; + LUAU_ASSERT(LUAU_INSN_OP(call) == LOP_CALL); + + StkId ra = VM_REG(LUAU_INSN_A(call)); + + int nparams = 2; + int nresults = LUAU_INSN_C(call) - 1; + + luau_FastFunction f = luauF_table[bfid]; + + if (cl->env->safeenv && f) + { + VM_PROTECT_PC(); + + int n = f(L, ra, arg1, nresults, arg2, nparams); + + if (n >= 0) + { + L->top = (nresults == LUA_MULTRET) ? ra + n : L->ci->top; + + pc += skip + 1; // skip instructions that compute function as well as CALL + LUAU_ASSERT(unsigned(pc - cl->l.p->code) < unsigned(cl->l.p->sizecode)); + return pc; + } + else + { + // continue execution through the fallback code + return pc; + } + } + else + { + // continue execution through the fallback code + return pc; + } +} + +const Instruction* execute_LOP_FASTCALL2K(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k) +{ + Instruction insn = *pc++; + int bfid = LUAU_INSN_A(insn); + int skip = LUAU_INSN_C(insn) - 1; + uint32_t aux = *pc++; + TValue* arg1 = VM_REG(LUAU_INSN_B(insn)); + TValue* arg2 = VM_KV(aux); + + LUAU_ASSERT(unsigned(pc - cl->l.p->code + skip) < unsigned(cl->l.p->sizecode)); + + Instruction call = pc[skip]; + LUAU_ASSERT(LUAU_INSN_OP(call) == LOP_CALL); + + StkId ra = VM_REG(LUAU_INSN_A(call)); + + int nparams = 2; + int nresults = LUAU_INSN_C(call) - 1; + + luau_FastFunction f = luauF_table[bfid]; + + if (cl->env->safeenv && f) + { + VM_PROTECT_PC(); + + int n = f(L, ra, arg1, nresults, arg2, nparams); + + if (n >= 0) + { + L->top = (nresults == LUA_MULTRET) ? ra + n : L->ci->top; + + pc += skip + 1; // skip instructions that compute function as well as CALL + LUAU_ASSERT(unsigned(pc - cl->l.p->code) < unsigned(cl->l.p->sizecode)); + return pc; + } + else + { + // continue execution through the fallback code + return pc; + } + } + else + { + // continue execution through the fallback code + return pc; + } +} + +const Instruction* execute_LOP_BREAK(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k) +{ + LUAU_ASSERT(!"Unsupported deprecated opcode"); + LUAU_UNREACHABLE(); +} + +const Instruction* execute_LOP_JUMPXEQKNIL(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k) +{ + Instruction insn = *pc++; + uint32_t aux = *pc; + StkId ra = VM_REG(LUAU_INSN_A(insn)); + + static_assert(LUA_TNIL == 0, "we expect type-1 to be negative iff type is nil"); + // condition is equivalent to: int(ttisnil(ra)) != (aux >> 31) + pc += int((ttype(ra) - 1) ^ aux) < 0 ? LUAU_INSN_D(insn) : 1; + LUAU_ASSERT(unsigned(pc - cl->l.p->code) < unsigned(cl->l.p->sizecode)); + return pc; +} + +const Instruction* execute_LOP_JUMPXEQKB(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k) +{ + Instruction insn = *pc++; + uint32_t aux = *pc; + StkId ra = VM_REG(LUAU_INSN_A(insn)); + + pc += int(ttisboolean(ra) && bvalue(ra) == int(aux & 1)) != (aux >> 31) ? LUAU_INSN_D(insn) : 1; + LUAU_ASSERT(unsigned(pc - cl->l.p->code) < unsigned(cl->l.p->sizecode)); + return pc; +} + +const Instruction* execute_LOP_JUMPXEQKN(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k) +{ + Instruction insn = *pc++; + uint32_t aux = *pc; + StkId ra = VM_REG(LUAU_INSN_A(insn)); + TValue* kv = VM_KV(aux & 0xffffff); + LUAU_ASSERT(ttisnumber(kv)); + +#if defined(__aarch64__) + // On several ARM chips (Apple M1/M2, Neoverse N1), comparing the result of a floating-point comparison is expensive, and a branch + // is much cheaper; on some 32-bit ARM chips (Cortex A53) the performance is about the same so we prefer less branchy variant there + if (aux >> 31) + pc += !(ttisnumber(ra) && nvalue(ra) == nvalue(kv)) ? LUAU_INSN_D(insn) : 1; + else + pc += (ttisnumber(ra) && nvalue(ra) == nvalue(kv)) ? LUAU_INSN_D(insn) : 1; +#else + pc += int(ttisnumber(ra) && nvalue(ra) == nvalue(kv)) != (aux >> 31) ? LUAU_INSN_D(insn) : 1; +#endif + LUAU_ASSERT(unsigned(pc - cl->l.p->code) < unsigned(cl->l.p->sizecode)); + return pc; +} + +const Instruction* execute_LOP_JUMPXEQKS(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k) +{ + Instruction insn = *pc++; + uint32_t aux = *pc; + StkId ra = VM_REG(LUAU_INSN_A(insn)); + TValue* kv = VM_KV(aux & 0xffffff); + LUAU_ASSERT(ttisstring(kv)); + + pc += int(ttisstring(ra) && gcvalue(ra) == gcvalue(kv)) != (aux >> 31) ? LUAU_INSN_D(insn) : 1; + LUAU_ASSERT(unsigned(pc - cl->l.p->code) < unsigned(cl->l.p->sizecode)); + return pc; +} diff --git a/CodeGen/src/Fallbacks.h b/CodeGen/src/Fallbacks.h new file mode 100644 index 00000000..3bec8c5b --- /dev/null +++ b/CodeGen/src/Fallbacks.h @@ -0,0 +1,93 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +// This file was generated by 'tools/lvmexecute_split.py' script, do not modify it by hand +#pragma once + +#include + +struct lua_State; +struct Closure; +typedef uint32_t Instruction; +typedef struct lua_TValue TValue; +typedef TValue* StkId; + +const Instruction* execute_LOP_NOP(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k); +const Instruction* execute_LOP_LOADNIL(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k); +const Instruction* execute_LOP_LOADB(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k); +const Instruction* execute_LOP_LOADN(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k); +const Instruction* execute_LOP_LOADK(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k); +const Instruction* execute_LOP_MOVE(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k); +const Instruction* execute_LOP_GETGLOBAL(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k); +const Instruction* execute_LOP_SETGLOBAL(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k); +const Instruction* execute_LOP_GETUPVAL(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k); +const Instruction* execute_LOP_SETUPVAL(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k); +const Instruction* execute_LOP_CLOSEUPVALS(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k); +const Instruction* execute_LOP_GETIMPORT(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k); +const Instruction* execute_LOP_GETTABLEKS(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k); +const Instruction* execute_LOP_SETTABLEKS(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k); +const Instruction* execute_LOP_GETTABLE(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k); +const Instruction* execute_LOP_SETTABLE(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k); +const Instruction* execute_LOP_GETTABLEN(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k); +const Instruction* execute_LOP_SETTABLEN(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k); +const Instruction* execute_LOP_NEWCLOSURE(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k); +const Instruction* execute_LOP_NAMECALL(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k); +const Instruction* execute_LOP_CALL(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k); +const Instruction* execute_LOP_RETURN(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k); +const Instruction* execute_LOP_JUMP(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k); +const Instruction* execute_LOP_JUMPIF(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k); +const Instruction* execute_LOP_JUMPIFNOT(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k); +const Instruction* execute_LOP_JUMPIFEQ(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k); +const Instruction* execute_LOP_JUMPIFNOTEQ(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k); +const Instruction* execute_LOP_JUMPIFLE(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k); +const Instruction* execute_LOP_JUMPIFNOTLE(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k); +const Instruction* execute_LOP_JUMPIFLT(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k); +const Instruction* execute_LOP_JUMPIFNOTLT(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k); +const Instruction* execute_LOP_ADD(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k); +const Instruction* execute_LOP_SUB(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k); +const Instruction* execute_LOP_MUL(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k); +const Instruction* execute_LOP_DIV(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k); +const Instruction* execute_LOP_MOD(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k); +const Instruction* execute_LOP_POW(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k); +const Instruction* execute_LOP_ADDK(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k); +const Instruction* execute_LOP_SUBK(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k); +const Instruction* execute_LOP_MULK(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k); +const Instruction* execute_LOP_DIVK(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k); +const Instruction* execute_LOP_MODK(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k); +const Instruction* execute_LOP_POWK(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k); +const Instruction* execute_LOP_AND(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k); +const Instruction* execute_LOP_OR(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k); +const Instruction* execute_LOP_ANDK(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k); +const Instruction* execute_LOP_ORK(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k); +const Instruction* execute_LOP_CONCAT(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k); +const Instruction* execute_LOP_NOT(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k); +const Instruction* execute_LOP_MINUS(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k); +const Instruction* execute_LOP_LENGTH(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k); +const Instruction* execute_LOP_NEWTABLE(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k); +const Instruction* execute_LOP_DUPTABLE(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k); +const Instruction* execute_LOP_SETLIST(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k); +const Instruction* execute_LOP_FORNPREP(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k); +const Instruction* execute_LOP_FORNLOOP(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k); +const Instruction* execute_LOP_FORGPREP(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k); +const Instruction* execute_LOP_FORGLOOP(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k); +const Instruction* execute_LOP_FORGPREP_INEXT(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k); +const Instruction* execute_LOP_DEP_FORGLOOP_INEXT(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k); +const Instruction* execute_LOP_FORGPREP_NEXT(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k); +const Instruction* execute_LOP_DEP_FORGLOOP_NEXT(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k); +const Instruction* execute_LOP_GETVARARGS(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k); +const Instruction* execute_LOP_DUPCLOSURE(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k); +const Instruction* execute_LOP_PREPVARARGS(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k); +const Instruction* execute_LOP_JUMPBACK(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k); +const Instruction* execute_LOP_LOADKX(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k); +const Instruction* execute_LOP_JUMPX(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k); +const Instruction* execute_LOP_FASTCALL(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k); +const Instruction* execute_LOP_COVERAGE(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k); +const Instruction* execute_LOP_CAPTURE(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k); +const Instruction* execute_LOP_DEP_JUMPIFEQK(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k); +const Instruction* execute_LOP_DEP_JUMPIFNOTEQK(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k); +const Instruction* execute_LOP_FASTCALL1(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k); +const Instruction* execute_LOP_FASTCALL2(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k); +const Instruction* execute_LOP_FASTCALL2K(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k); +const Instruction* execute_LOP_BREAK(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k); +const Instruction* execute_LOP_JUMPXEQKNIL(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k); +const Instruction* execute_LOP_JUMPXEQKB(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k); +const Instruction* execute_LOP_JUMPXEQKN(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k); +const Instruction* execute_LOP_JUMPXEQKS(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k); diff --git a/CodeGen/src/FallbacksProlog.h b/CodeGen/src/FallbacksProlog.h new file mode 100644 index 00000000..bbb06b84 --- /dev/null +++ b/CodeGen/src/FallbacksProlog.h @@ -0,0 +1,56 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +#pragma once + +#include "lvm.h" + +#include "lbuiltins.h" +#include "lbytecode.h" +#include "ldebug.h" +#include "ldo.h" +#include "lfunc.h" +#include "lgc.h" +#include "lmem.h" +#include "lnumutils.h" +#include "lstate.h" +#include "lstring.h" +#include "ltable.h" + +#include + +// All external function calls that can cause stack realloc or Lua calls have to be wrapped in VM_PROTECT +// This makes sure that we save the pc (in case the Lua call needs to generate a backtrace) before the call, +// and restores the stack pointer after in case stack gets reallocated +// Should only be used on the slow paths. +#define VM_PROTECT(x) \ + { \ + L->ci->savedpc = pc; \ + { \ + x; \ + }; \ + base = L->base; \ + } + +// Some external functions can cause an error, but never reallocate the stack; for these, VM_PROTECT_PC() is +// a cheaper version of VM_PROTECT that can be called before the external call. +#define VM_PROTECT_PC() L->ci->savedpc = pc + +#define VM_REG(i) (LUAU_ASSERT(unsigned(i) < unsigned(L->top - base)), &base[i]) +#define VM_KV(i) (LUAU_ASSERT(unsigned(i) < unsigned(cl->l.p->sizek)), &k[i]) +#define VM_UV(i) (LUAU_ASSERT(unsigned(i) < unsigned(cl->nupvalues)), &cl->l.uprefs[i]) + +#define VM_PATCH_C(pc, slot) *const_cast(pc) = ((uint8_t(slot) << 24) | (0x00ffffffu & *(pc))) +#define VM_PATCH_E(pc, slot) *const_cast(pc) = ((uint32_t(slot) << 8) | (0x000000ffu & *(pc))) + +#define VM_INTERRUPT() \ + { \ + void (*interrupt)(lua_State*, int) = L->global->cb.interrupt; \ + if (LUAU_UNLIKELY(!!interrupt)) \ + { /* the interrupt hook is called right before we advance pc */ \ + VM_PROTECT(L->ci->savedpc++; interrupt(L, -1)); \ + if (L->status != 0) \ + { \ + L->ci->savedpc--; \ + return NULL; \ + } \ + } \ + } diff --git a/CodeGen/src/UnwindBuilderDwarf2.cpp b/CodeGen/src/UnwindBuilderDwarf2.cpp index 38e3e712..f3886d9c 100644 --- a/CodeGen/src/UnwindBuilderDwarf2.cpp +++ b/CodeGen/src/UnwindBuilderDwarf2.cpp @@ -1,6 +1,8 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #include "Luau/UnwindBuilderDwarf2.h" +#include "ByteUtils.h" + #include // General information about Dwarf2 format can be found at: @@ -11,40 +13,6 @@ // https://refspecs.linuxbase.org/elf/x86_64-abi-0.99.pdf [System V Application Binary Interface (AMD64 Architecture Processor Supplement)] // Interaction between Dwarf2 and System V ABI can be found in sections '3.6.2 DWARF Register Number Mapping' and '4.2.4 EH_FRAME sections' -static char* writeu8(char* target, uint8_t value) -{ - memcpy(target, &value, sizeof(value)); - return target + sizeof(value); -} - -static char* writeu32(char* target, uint32_t value) -{ - memcpy(target, &value, sizeof(value)); - return target + sizeof(value); -} - -static char* writeu64(char* target, uint64_t value) -{ - memcpy(target, &value, sizeof(value)); - return target + sizeof(value); -} - -static char* writeuleb128(char* target, uint64_t value) -{ - do - { - char byte = value & 0x7f; - value >>= 7; - - if (value) - byte |= 0x80; - - *target++ = byte; - } while (value); - - return target; -} - // Call frame instruction opcodes #define DW_CFA_advance_loc 0x40 #define DW_CFA_offset 0x80 @@ -104,7 +72,7 @@ const int kFdeInitialLocationOffset = 8; const int kFdeAddressRangeOffset = 16; // Define canonical frame address expression as [reg + offset] -static char* defineCfaExpression(char* pos, int dwReg, uint32_t stackOffset) +static uint8_t* defineCfaExpression(uint8_t* pos, int dwReg, uint32_t stackOffset) { pos = writeu8(pos, DW_CFA_def_cfa); pos = writeuleb128(pos, dwReg); @@ -113,14 +81,14 @@ static char* defineCfaExpression(char* pos, int dwReg, uint32_t stackOffset) } // Update offset value in canonical frame address expression -static char* defineCfaExpressionOffset(char* pos, uint32_t stackOffset) +static uint8_t* defineCfaExpressionOffset(uint8_t* pos, uint32_t stackOffset) { pos = writeu8(pos, DW_CFA_def_cfa_offset); pos = writeuleb128(pos, stackOffset); return pos; } -static char* defineSavedRegisterLocation(char* pos, int dwReg, uint32_t stackOffset) +static uint8_t* defineSavedRegisterLocation(uint8_t* pos, int dwReg, uint32_t stackOffset) { LUAU_ASSERT(stackOffset % kDataAlignFactor == 0 && "stack offsets have to be measured in kDataAlignFactor units"); @@ -138,14 +106,14 @@ static char* defineSavedRegisterLocation(char* pos, int dwReg, uint32_t stackOff return pos; } -static char* advanceLocation(char* pos, uint8_t offset) +static uint8_t* advanceLocation(uint8_t* pos, uint8_t offset) { pos = writeu8(pos, DW_CFA_advance_loc1); pos = writeu8(pos, offset); return pos; } -static char* alignPosition(char* start, char* pos) +static uint8_t* alignPosition(uint8_t* start, uint8_t* pos) { size_t size = pos - start; size_t pad = ((size + kDwarfAlign - 1) & ~(kDwarfAlign - 1)) - size; @@ -163,7 +131,7 @@ namespace CodeGen void UnwindBuilderDwarf2::start() { - char* cieLength = pos; + uint8_t* cieLength = pos; pos = writeu32(pos, 0); // Length (to be filled later) pos = writeu32(pos, 0); // CIE id. 0 -- .eh_frame @@ -245,8 +213,8 @@ void UnwindBuilderDwarf2::finalize(char* target, void* funcAddress, size_t funcS memcpy(target, rawData, getSize()); unsigned fdeEntryStartPos = unsigned(fdeEntryStart - rawData); - writeu64(target + fdeEntryStartPos + kFdeInitialLocationOffset, uintptr_t(funcAddress)); - writeu64(target + fdeEntryStartPos + kFdeAddressRangeOffset, funcSize); + writeu64((uint8_t*)target + fdeEntryStartPos + kFdeInitialLocationOffset, uintptr_t(funcAddress)); + writeu64((uint8_t*)target + fdeEntryStartPos + kFdeAddressRangeOffset, funcSize); } } // namespace CodeGen diff --git a/Compiler/src/ConstantFolding.cpp b/Compiler/src/ConstantFolding.cpp index e35c883a..510f2b7b 100644 --- a/Compiler/src/ConstantFolding.cpp +++ b/Compiler/src/ConstantFolding.cpp @@ -3,6 +3,7 @@ #include "BuiltinFolding.h" +#include #include namespace Luau diff --git a/Makefile b/Makefile index f51f8ba4..5b8eb935 100644 --- a/Makefile +++ b/Makefile @@ -105,7 +105,7 @@ endif $(AST_OBJECTS): CXXFLAGS+=-std=c++17 -ICommon/include -IAst/include $(COMPILER_OBJECTS): CXXFLAGS+=-std=c++17 -ICompiler/include -ICommon/include -IAst/include $(ANALYSIS_OBJECTS): CXXFLAGS+=-std=c++17 -ICommon/include -IAst/include -IAnalysis/include -$(CODEGEN_OBJECTS): CXXFLAGS+=-std=c++17 -ICommon/include -ICodeGen/include +$(CODEGEN_OBJECTS): CXXFLAGS+=-std=c++17 -ICommon/include -ICodeGen/include -IVM/include -IVM/src # Code generation needs VM internals $(VM_OBJECTS): CXXFLAGS+=-std=c++11 -ICommon/include -IVM/include $(ISOCLINE_OBJECTS): CXXFLAGS+=-Wno-unused-function -Iextern/isocline/include $(TESTS_OBJECTS): CXXFLAGS+=-std=c++17 -ICommon/include -IAst/include -ICompiler/include -IAnalysis/include -ICodeGen/include -IVM/include -ICLI -Iextern -DDOCTEST_CONFIG_DOUBLE_STRINGIFY diff --git a/Sources.cmake b/Sources.cmake index 580c8b3c..38100483 100644 --- a/Sources.cmake +++ b/Sources.cmake @@ -69,8 +69,13 @@ target_sources(Luau.CodeGen PRIVATE CodeGen/src/AssemblyBuilderX64.cpp CodeGen/src/CodeAllocator.cpp CodeGen/src/CodeBlockUnwind.cpp + CodeGen/src/Fallbacks.cpp CodeGen/src/UnwindBuilderDwarf2.cpp CodeGen/src/UnwindBuilderWin.cpp + + CodeGen/src/ByteUtils.h + CodeGen/src/Fallbacks.h + CodeGen/src/FallbacksProlog.h ) # Luau.Analysis Sources diff --git a/VM/include/lua.h b/VM/include/lua.h index 5c61f136..33e68517 100644 --- a/VM/include/lua.h +++ b/VM/include/lua.h @@ -304,6 +304,7 @@ LUA_API size_t lua_totalbytes(lua_State* L, int category); LUA_API l_noret lua_error(lua_State* L); LUA_API int lua_next(lua_State* L, int idx); +LUA_API int lua_rawiter(lua_State* L, int idx, int iter); LUA_API void lua_concat(lua_State* L, int n); diff --git a/VM/include/luaconf.h b/VM/include/luaconf.h index 7b0f4c30..1d8d7813 100644 --- a/VM/include/luaconf.h +++ b/VM/include/luaconf.h @@ -109,6 +109,11 @@ #define LUA_MAXCAPTURES 32 #endif +// enables callbacks to redirect code execution from Luau VM to a custom implementation +#ifndef LUA_CUSTOM_EXECUTION +#define LUA_CUSTOM_EXECUTION 0 +#endif + // }================================================================== /* diff --git a/VM/src/lapi.cpp b/VM/src/lapi.cpp index 968b8d00..e5ce4d5a 100644 --- a/VM/src/lapi.cpp +++ b/VM/src/lapi.cpp @@ -51,6 +51,12 @@ const char* luau_ident = "$Luau: Copyright (C) 2019-2022 Roblox Corporation $\n" L->top++; \ } +#define api_update_top(L, p) \ + { \ + api_check(L, p >= L->base && p < L->ci->top); \ + L->top = p; \ + } + #define updateatom(L, ts) \ { \ if (ts->atom == ATOM_UNDEF) \ @@ -851,7 +857,7 @@ void lua_rawsetfield(lua_State* L, int idx, const char* k) StkId t = index2addr(L, idx); api_check(L, ttistable(t)); if (hvalue(t)->readonly) - luaG_runerror(L, "Attempt to modify a readonly table"); + luaG_readonlyerror(L); setobj2t(L, luaH_setstr(L, hvalue(t), luaS_new(L, k)), L->top - 1); luaC_barriert(L, hvalue(t), L->top - 1); L->top--; @@ -1204,6 +1210,52 @@ int lua_next(lua_State* L, int idx) return more; } +int lua_rawiter(lua_State* L, int idx, int iter) +{ + luaC_threadbarrier(L); + StkId t = index2addr(L, idx); + api_check(L, ttistable(t)); + api_check(L, iter >= 0); + + Table* h = hvalue(t); + int sizearray = h->sizearray; + + // first we advance iter through the array portion + for (; unsigned(iter) < unsigned(sizearray); ++iter) + { + TValue* e = &h->array[iter]; + + if (!ttisnil(e)) + { + StkId top = L->top; + setnvalue(top + 0, double(iter + 1)); + setobj2s(L, top + 1, e); + api_update_top(L, top + 2); + return iter + 1; + } + } + + int sizenode = 1 << h->lsizenode; + + // then we advance iter through the hash portion + for (; unsigned(iter - sizearray) < unsigned(sizenode); ++iter) + { + LuaNode* n = &h->node[iter - sizearray]; + + if (!ttisnil(gval(n))) + { + StkId top = L->top; + getnodekey(L, top + 0, n); + setobj2s(L, top + 1, gval(n)); + api_update_top(L, top + 2); + return iter + 1; + } + } + + // traversal finished + return -1; +} + void lua_concat(lua_State* L, int n) { api_checknelems(L, n); @@ -1382,7 +1434,7 @@ void lua_cleartable(lua_State* L, int idx) api_check(L, ttistable(t)); Table* tt = hvalue(t); if (tt->readonly) - luaG_runerror(L, "Attempt to modify a readonly table"); + luaG_readonlyerror(L); luaH_clear(tt); } diff --git a/VM/src/ldebug.cpp b/VM/src/ldebug.cpp index fee9aaa9..7c181d4e 100644 --- a/VM/src/ldebug.cpp +++ b/VM/src/ldebug.cpp @@ -340,6 +340,11 @@ void luaG_breakpoint(lua_State* L, Proto* p, int line, bool enable) p->code[i] |= op; LUAU_ASSERT(LUAU_INSN_OP(p->code[i]) == op); +#if LUA_CUSTOM_EXECUTION + if (L->global->ecb.setbreakpoint) + L->global->ecb.setbreakpoint(L, p, i); +#endif + // note: this is important! // we only patch the *first* instruction in each proto that's attributed to a given line // this can be changed, but if requires making patching a bit more nuanced so that we don't patch AUX words diff --git a/VM/src/lfunc.cpp b/VM/src/lfunc.cpp index 3c1869b5..aeb3d710 100644 --- a/VM/src/lfunc.cpp +++ b/VM/src/lfunc.cpp @@ -31,6 +31,11 @@ Proto* luaF_newproto(lua_State* L) f->source = NULL; f->debugname = NULL; f->debuginsn = NULL; + +#if LUA_CUSTOM_EXECUTION + f->execdata = NULL; +#endif + return f; } @@ -149,6 +154,15 @@ void luaF_freeproto(lua_State* L, Proto* f, lua_Page* page) luaM_freearray(L, f->upvalues, f->sizeupvalues, TString*, f->memcat); if (f->debuginsn) luaM_freearray(L, f->debuginsn, f->sizecode, uint8_t, f->memcat); + +#if LUA_CUSTOM_EXECUTION + if (f->execdata) + { + LUAU_ASSERT(L->global->ecb.destroy); + L->global->ecb.destroy(L, f); + } +#endif + luaM_freegco(L, f, sizeof(Proto), f->memcat, page); } diff --git a/VM/src/lobject.h b/VM/src/lobject.h index 48aaf94b..97cbfbb1 100644 --- a/VM/src/lobject.h +++ b/VM/src/lobject.h @@ -281,6 +281,10 @@ typedef struct Proto TString* debugname; uint8_t* debuginsn; // a copy of code[] array with just opcodes +#if LUA_CUSTOM_EXECUTION + void* execdata; +#endif + GCObject* gclist; diff --git a/VM/src/lstate.cpp b/VM/src/lstate.cpp index fdd7fc2b..cfe2cbfb 100644 --- a/VM/src/lstate.cpp +++ b/VM/src/lstate.cpp @@ -212,6 +212,11 @@ lua_State* lua_newstate(lua_Alloc f, void* ud) g->memcatbytes[0] = sizeof(LG); g->cb = lua_Callbacks(); + +#if LUA_CUSTOM_EXECUTION + g->ecb = lua_ExecutionCallbacks(); +#endif + g->gcstats = GCStats(); #ifdef LUAI_GCMETRICS diff --git a/VM/src/lstate.h b/VM/src/lstate.h index 06544463..5b7d0883 100644 --- a/VM/src/lstate.h +++ b/VM/src/lstate.h @@ -146,6 +146,19 @@ struct GCMetrics }; #endif +#if LUA_CUSTOM_EXECUTION + +// Callbacks that can be used to to redirect code execution from Luau bytecode VM to a custom implementation (AoT/JiT/sandboxing/...) +typedef struct lua_ExecutionCallbacks +{ + void* context; + void (*destroy)(lua_State* L, Proto* proto); // called when function is destroyed + int (*enter)(lua_State* L, Proto* proto); // called when function is about to start/resume (when execdata is present), return 0 to exit VM + void (*setbreakpoint)(lua_State* L, Proto* proto, int line); // called when a breakpoint is set in a function +} lua_ExecutionCallbacks; + +#endif + /* ** `global state', shared by all threads of this state */ @@ -202,6 +215,10 @@ typedef struct global_State lua_Callbacks cb; +#if LUA_CUSTOM_EXECUTION + lua_ExecutionCallbacks ecb; +#endif + GCStats gcstats; #ifdef LUAI_GCMETRICS diff --git a/VM/src/lvm.h b/VM/src/lvm.h index 25a27166..c4b1c18b 100644 --- a/VM/src/lvm.h +++ b/VM/src/lvm.h @@ -24,6 +24,9 @@ LUAI_FUNC void luaV_gettable(lua_State* L, const TValue* t, TValue* key, StkId v LUAI_FUNC void luaV_settable(lua_State* L, const TValue* t, TValue* key, StkId val); LUAI_FUNC void luaV_concat(lua_State* L, int total, int last); LUAI_FUNC void luaV_getimport(lua_State* L, Table* env, TValue* k, uint32_t id, bool propagatenil); +LUAI_FUNC void luaV_prepareFORN(lua_State* L, StkId plimit, StkId pstep, StkId pinit); +LUAI_FUNC void luaV_callTM(lua_State* L, int nparams, int res); +LUAI_FUNC void luaV_tryfuncTM(lua_State* L, StkId func); LUAI_FUNC void luau_execute(lua_State* L); LUAI_FUNC int luau_precall(lua_State* L, struct lua_TValue* func, int nresults); diff --git a/VM/src/lvmexecute.cpp b/VM/src/lvmexecute.cpp index c3c744b2..6ceed512 100644 --- a/VM/src/lvmexecute.cpp +++ b/VM/src/lvmexecute.cpp @@ -131,84 +131,6 @@ goto dispatchContinue #endif -LUAU_NOINLINE static void luau_prepareFORN(lua_State* L, StkId plimit, StkId pstep, StkId pinit) -{ - if (!ttisnumber(pinit) && !luaV_tonumber(pinit, pinit)) - luaG_forerror(L, pinit, "initial value"); - if (!ttisnumber(plimit) && !luaV_tonumber(plimit, plimit)) - luaG_forerror(L, plimit, "limit"); - if (!ttisnumber(pstep) && !luaV_tonumber(pstep, pstep)) - luaG_forerror(L, pstep, "step"); -} - -// calls a C function f with no yielding support; optionally save one resulting value to the res register -// the function and arguments have to already be pushed to L->top -LUAU_NOINLINE static void luau_callTM(lua_State* L, int nparams, int res) -{ - ++L->nCcalls; - - if (L->nCcalls >= LUAI_MAXCCALLS) - luaD_checkCstack(L); - - luaD_checkstack(L, LUA_MINSTACK); - - StkId top = L->top; - StkId fun = top - nparams - 1; - - CallInfo* ci = incr_ci(L); - ci->func = fun; - ci->base = fun + 1; - ci->top = top + LUA_MINSTACK; - ci->savedpc = NULL; - ci->flags = 0; - ci->nresults = (res >= 0); - LUAU_ASSERT(ci->top <= L->stack_last); - - LUAU_ASSERT(ttisfunction(ci->func)); - LUAU_ASSERT(clvalue(ci->func)->isC); - - L->base = fun + 1; - LUAU_ASSERT(L->top == L->base + nparams); - - lua_CFunction func = clvalue(fun)->c.f; - int n = func(L); - LUAU_ASSERT(n >= 0); // yields should have been blocked by nCcalls - - // ci is our callinfo, cip is our parent - // note that we read L->ci again since it may have been reallocated by the call - CallInfo* cip = L->ci - 1; - - // copy return value into parent stack - if (res >= 0) - { - if (n > 0) - { - setobj2s(L, &cip->base[res], L->top - n); - } - else - { - setnilvalue(&cip->base[res]); - } - } - - L->ci = cip; - L->base = cip->base; - L->top = cip->top; - - --L->nCcalls; -} - -LUAU_NOINLINE static void luau_tryfuncTM(lua_State* L, StkId func) -{ - const TValue* tm = luaT_gettmbyobj(L, func, TM_CALL); - if (!ttisfunction(tm)) - luaG_typeerror(L, func, "call"); - for (StkId p = L->top; p > func; p--) // open space for metamethod - setobjs2s(L, p, p - 1); - L->top++; // stack space pre-allocated by the caller - setobj2s(L, func, tm); // tag method is the new function to be called -} - LUAU_NOINLINE void luau_callhook(lua_State* L, lua_Hook hook, void* userdata) { ptrdiff_t base = savestack(L, L->base); @@ -284,6 +206,20 @@ static void luau_execute(lua_State* L) LUAU_ASSERT(L->isactive); LUAU_ASSERT(!isblack(obj2gco(L))); // we don't use luaC_threadbarrier because active threads never turn black +#if LUA_CUSTOM_EXECUTION + Proto* p = clvalue(L->ci->func)->l.p; + + if (p->execdata) + { + if (L->global->ecb.enter(L, p) == 0) + return; + } + +reentry: +#endif + + LUAU_ASSERT(isLua(L->ci)); + pc = L->ci->savedpc; cl = clvalue(L->ci->func); base = L->base; @@ -564,7 +500,7 @@ static void luau_execute(lua_State* L) L->top = top + 3; L->cachedslot = LUAU_INSN_C(insn); - VM_PROTECT(luau_callTM(L, 2, LUAU_INSN_A(insn))); + VM_PROTECT(luaV_callTM(L, 2, LUAU_INSN_A(insn))); // save cachedslot to accelerate future lookups; patches currently executing instruction since pc-2 rolls back two pc++ VM_PATCH_C(pc - 2, L->cachedslot); VM_NEXT(); @@ -601,7 +537,7 @@ static void luau_execute(lua_State* L) L->top = top + 3; L->cachedslot = LUAU_INSN_C(insn); - VM_PROTECT(luau_callTM(L, 2, LUAU_INSN_A(insn))); + VM_PROTECT(luaV_callTM(L, 2, LUAU_INSN_A(insn))); // save cachedslot to accelerate future lookups; patches currently executing instruction since pc-2 rolls back two pc++ VM_PATCH_C(pc - 2, L->cachedslot); VM_NEXT(); @@ -680,7 +616,7 @@ static void luau_execute(lua_State* L) L->top = top + 4; L->cachedslot = LUAU_INSN_C(insn); - VM_PROTECT(luau_callTM(L, 3, -1)); + VM_PROTECT(luaV_callTM(L, 3, -1)); // save cachedslot to accelerate future lookups; patches currently executing instruction since pc-2 rolls back two pc++ VM_PATCH_C(pc - 2, L->cachedslot); VM_NEXT(); @@ -973,7 +909,7 @@ static void luau_execute(lua_State* L) // slow-path: not a function call if (LUAU_UNLIKELY(!ttisfunction(ra))) { - VM_PROTECT(luau_tryfuncTM(L, ra)); + VM_PROTECT(luaV_tryfuncTM(L, ra)); argtop++; // __call adds an extra self } @@ -1009,6 +945,18 @@ static void luau_execute(lua_State* L) setnilvalue(argi++); // complete missing arguments L->top = p->is_vararg ? argi : ci->top; +#if LUA_CUSTOM_EXECUTION + if (p->execdata) + { + LUAU_ASSERT(L->global->ecb.enter); + + if (L->global->ecb.enter(L, p) == 1) + goto reentry; + else + goto exit; + } +#endif + // reentry pc = p->code; cl = ccl; @@ -1092,11 +1040,26 @@ static void luau_execute(lua_State* L) LUAU_ASSERT(isLua(L->ci)); + Closure* nextcl = clvalue(cip->func); + Proto* nextproto = nextcl->l.p; + +#if LUA_CUSTOM_EXECUTION + if (nextproto->execdata) + { + LUAU_ASSERT(L->global->ecb.enter); + + if (L->global->ecb.enter(L, nextproto) == 1) + goto reentry; + else + goto exit; + } +#endif + // reentry pc = cip->savedpc; - cl = clvalue(cip->func); + cl = nextcl; base = L->base; - k = cl->l.p->k; + k = nextproto->k; VM_NEXT(); } @@ -1212,7 +1175,7 @@ static void luau_execute(lua_State* L) int res = int(top - base); L->top = top + 3; - VM_PROTECT(luau_callTM(L, 2, res)); + VM_PROTECT(luaV_callTM(L, 2, res)); pc += !l_isfalse(&base[res]) ? LUAU_INSN_D(insn) : 1; LUAU_ASSERT(unsigned(pc - cl->l.p->code) < unsigned(cl->l.p->sizecode)); VM_NEXT(); @@ -1324,7 +1287,7 @@ static void luau_execute(lua_State* L) int res = int(top - base); L->top = top + 3; - VM_PROTECT(luau_callTM(L, 2, res)); + VM_PROTECT(luaV_callTM(L, 2, res)); pc += l_isfalse(&base[res]) ? LUAU_INSN_D(insn) : 1; LUAU_ASSERT(unsigned(pc - cl->l.p->code) < unsigned(cl->l.p->sizecode)); VM_NEXT(); @@ -1519,7 +1482,7 @@ static void luau_execute(lua_State* L) setobj2s(L, top + 2, rc); L->top = top + 3; - VM_PROTECT(luau_callTM(L, 2, LUAU_INSN_A(insn))); + VM_PROTECT(luaV_callTM(L, 2, LUAU_INSN_A(insn))); VM_NEXT(); } else @@ -1565,7 +1528,7 @@ static void luau_execute(lua_State* L) setobj2s(L, top + 2, rc); L->top = top + 3; - VM_PROTECT(luau_callTM(L, 2, LUAU_INSN_A(insn))); + VM_PROTECT(luaV_callTM(L, 2, LUAU_INSN_A(insn))); VM_NEXT(); } else @@ -1626,7 +1589,7 @@ static void luau_execute(lua_State* L) setobj2s(L, top + 2, rc); L->top = top + 3; - VM_PROTECT(luau_callTM(L, 2, LUAU_INSN_A(insn))); + VM_PROTECT(luaV_callTM(L, 2, LUAU_INSN_A(insn))); VM_NEXT(); } else @@ -1687,7 +1650,7 @@ static void luau_execute(lua_State* L) setobj2s(L, top + 2, rc); L->top = top + 3; - VM_PROTECT(luau_callTM(L, 2, LUAU_INSN_A(insn))); + VM_PROTECT(luaV_callTM(L, 2, LUAU_INSN_A(insn))); VM_NEXT(); } else @@ -1819,7 +1782,7 @@ static void luau_execute(lua_State* L) setobj2s(L, top + 2, kv); L->top = top + 3; - VM_PROTECT(luau_callTM(L, 2, LUAU_INSN_A(insn))); + VM_PROTECT(luaV_callTM(L, 2, LUAU_INSN_A(insn))); VM_NEXT(); } else @@ -1865,7 +1828,7 @@ static void luau_execute(lua_State* L) setobj2s(L, top + 2, kv); L->top = top + 3; - VM_PROTECT(luau_callTM(L, 2, LUAU_INSN_A(insn))); + VM_PROTECT(luaV_callTM(L, 2, LUAU_INSN_A(insn))); VM_NEXT(); } else @@ -2029,7 +1992,7 @@ static void luau_execute(lua_State* L) setobj2s(L, top + 1, rb); L->top = top + 2; - VM_PROTECT(luau_callTM(L, 1, LUAU_INSN_A(insn))); + VM_PROTECT(luaV_callTM(L, 1, LUAU_INSN_A(insn))); VM_NEXT(); } else @@ -2145,7 +2108,7 @@ static void luau_execute(lua_State* L) // Note: this doesn't reallocate stack so we don't need to recompute ra/base VM_PROTECT_PC(); - luau_prepareFORN(L, ra + 0, ra + 1, ra + 2); + luaV_prepareFORN(L, ra + 0, ra + 1, ra + 2); } double limit = nvalue(ra + 0); @@ -2861,7 +2824,7 @@ int luau_precall(lua_State* L, StkId func, int nresults) { if (!ttisfunction(func)) { - luau_tryfuncTM(L, func); + luaV_tryfuncTM(L, func); // L->top is incremented by tryfuncTM } diff --git a/VM/src/lvmutils.cpp b/VM/src/lvmutils.cpp index 33d47020..35124e63 100644 --- a/VM/src/lvmutils.cpp +++ b/VM/src/lvmutils.cpp @@ -508,3 +508,81 @@ void luaV_dolen(lua_State* L, StkId ra, const TValue* rb) if (!ttisnumber(res)) luaG_runerror(L, "'__len' must return a number"); // note, we can't access rb since stack may have been reallocated } + +LUAU_NOINLINE void luaV_prepareFORN(lua_State* L, StkId plimit, StkId pstep, StkId pinit) +{ + if (!ttisnumber(pinit) && !luaV_tonumber(pinit, pinit)) + luaG_forerror(L, pinit, "initial value"); + if (!ttisnumber(plimit) && !luaV_tonumber(plimit, plimit)) + luaG_forerror(L, plimit, "limit"); + if (!ttisnumber(pstep) && !luaV_tonumber(pstep, pstep)) + luaG_forerror(L, pstep, "step"); +} + +// calls a C function f with no yielding support; optionally save one resulting value to the res register +// the function and arguments have to already be pushed to L->top +LUAU_NOINLINE void luaV_callTM(lua_State* L, int nparams, int res) +{ + ++L->nCcalls; + + if (L->nCcalls >= LUAI_MAXCCALLS) + luaD_checkCstack(L); + + luaD_checkstack(L, LUA_MINSTACK); + + StkId top = L->top; + StkId fun = top - nparams - 1; + + CallInfo* ci = incr_ci(L); + ci->func = fun; + ci->base = fun + 1; + ci->top = top + LUA_MINSTACK; + ci->savedpc = NULL; + ci->flags = 0; + ci->nresults = (res >= 0); + LUAU_ASSERT(ci->top <= L->stack_last); + + LUAU_ASSERT(ttisfunction(ci->func)); + LUAU_ASSERT(clvalue(ci->func)->isC); + + L->base = fun + 1; + LUAU_ASSERT(L->top == L->base + nparams); + + lua_CFunction func = clvalue(fun)->c.f; + int n = func(L); + LUAU_ASSERT(n >= 0); // yields should have been blocked by nCcalls + + // ci is our callinfo, cip is our parent + // note that we read L->ci again since it may have been reallocated by the call + CallInfo* cip = L->ci - 1; + + // copy return value into parent stack + if (res >= 0) + { + if (n > 0) + { + setobj2s(L, &cip->base[res], L->top - n); + } + else + { + setnilvalue(&cip->base[res]); + } + } + + L->ci = cip; + L->base = cip->base; + L->top = cip->top; + + --L->nCcalls; +} + +LUAU_NOINLINE void luaV_tryfuncTM(lua_State* L, StkId func) +{ + const TValue* tm = luaT_gettmbyobj(L, func, TM_CALL); + if (!ttisfunction(tm)) + luaG_typeerror(L, func, "call"); + for (StkId p = L->top; p > func; p--) // open space for metamethod + setobjs2s(L, p, p - 1); + L->top++; // stack space pre-allocated by the caller + setobj2s(L, func, tm); // tag method is the new function to be called +} diff --git a/tests/Compiler.test.cpp b/tests/Compiler.test.cpp index e6222b02..9409c822 100644 --- a/tests/Compiler.test.cpp +++ b/tests/Compiler.test.cpp @@ -62,6 +62,29 @@ LOADN R0 5 LOADK R1 K0 RETURN R0 2 )"); + + CHECK_EQ("\n" + bcb.dumpEverything(), R"( +Function 0 (??): +LOADN R0 5 +LOADK R1 K0 +RETURN R0 2 + +)"); +} + +TEST_CASE("CompileError") +{ + std::string source = "local " + rep("a,", 300) + "a = ..."; + + // fails to parse + std::string bc1 = Luau::compile(source + " !#*$!#$^&!*#&$^*"); + + // parses, but fails to compile (too many locals) + std::string bc2 = Luau::compile(source); + + // 0 acts as a special marker for error bytecode + CHECK_EQ(bc1[0], 0); + CHECK_EQ(bc2[0], 0); } TEST_CASE("LocalsDirectReference") @@ -1230,6 +1253,27 @@ RETURN R0 0 )"); } +TEST_CASE("UnaryBasic") +{ + CHECK_EQ("\n" + compileFunction0("local a = ... return not a"), R"( +GETVARARGS R0 1 +NOT R1 R0 +RETURN R1 1 +)"); + + CHECK_EQ("\n" + compileFunction0("local a = ... return -a"), R"( +GETVARARGS R0 1 +MINUS R1 R0 +RETURN R1 1 +)"); + + CHECK_EQ("\n" + compileFunction0("local a = ... return #a"), R"( +GETVARARGS R0 1 +LENGTH R1 R0 +RETURN R1 1 +)"); +} + TEST_CASE("InterpStringWithNoExpressions") { ScopedFastFlag sff{"LuauInterpolatedStringBaseSupport", true}; @@ -4975,6 +5019,27 @@ MOVE R1 R0 CALL R1 0 1 RETURN R1 1 )"); + + // we can't inline any functions in modules with getfenv/setfenv + CHECK_EQ("\n" + compileFunction(R"( +local function foo() + return 42 +end + +local x = foo() +getfenv() +return x +)", + 1, 2), + R"( +DUPCLOSURE R0 K0 +MOVE R1 R0 +CALL R1 0 1 +GETIMPORT R2 2 +CALL R2 0 0 +RETURN R1 1 +)"); + } TEST_CASE("InlineNestedLoops") @@ -6101,6 +6166,7 @@ return bit32.extract(-1, 31), bit32.replace(100, 1, 0), math.log(100, 10), + typeof(nil), (type("fin")) )", 0, 2), @@ -6156,7 +6222,8 @@ LOADN R47 1 LOADN R48 101 LOADN R49 2 LOADK R50 K3 -RETURN R0 51 +LOADK R51 K4 +RETURN R0 52 )"); } diff --git a/tests/Conformance.test.cpp b/tests/Conformance.test.cpp index cf4a14c7..77b30487 100644 --- a/tests/Conformance.test.cpp +++ b/tests/Conformance.test.cpp @@ -786,6 +786,44 @@ TEST_CASE("ApiTables") lua_pop(L, 1); } +TEST_CASE("ApiIter") +{ + StateRef globalState(luaL_newstate(), lua_close); + lua_State* L = globalState.get(); + + lua_newtable(L); + lua_pushnumber(L, 123.0); + lua_setfield(L, -2, "key"); + lua_pushnumber(L, 456.0); + lua_rawsetfield(L, -2, "key2"); + lua_pushstring(L, "test"); + lua_rawseti(L, -2, 1); + + // Lua-compatible iteration interface: lua_next + double sum1 = 0; + lua_pushnil(L); + while (lua_next(L, -2)) + { + sum1 += lua_tonumber(L, -2); // key + sum1 += lua_tonumber(L, -1); // value + lua_pop(L, 1); // pop value, key is used by lua_next + } + CHECK(sum1 == 580); + + // Luau iteration interface: lua_rawiter (faster and preferable to lua_next) + double sum2 = 0; + for (int index = 0; index = lua_rawiter(L, -1, index), index >= 0; ) + { + sum2 += lua_tonumber(L, -2); // key + sum2 += lua_tonumber(L, -1); // value + lua_pop(L, 2); // pop both key and value + } + CHECK(sum2 == 580); + + // pop table + lua_pop(L, 1); +} + TEST_CASE("ApiCalls") { StateRef globalState = runConformance("apicalls.lua"); diff --git a/tests/Linter.test.cpp b/tests/Linter.test.cpp index 8c7d762e..921e6691 100644 --- a/tests/Linter.test.cpp +++ b/tests/Linter.test.cpp @@ -1705,8 +1705,6 @@ TEST_CASE_FIXTURE(Fixture, "TestStringInterpolation") TEST_CASE_FIXTURE(Fixture, "IntegerParsing") { - ScopedFastFlag luauLintParseIntegerIssues{"LuauLintParseIntegerIssues", true}; - LintResult result = lint(R"( local _ = 0b10000000000000000000000000000000000000000000000000000000000000000 local _ = 0x10000000000000000 @@ -1720,7 +1718,6 @@ local _ = 0x10000000000000000 // TODO: remove with FFlagLuauErrorDoubleHexPrefix TEST_CASE_FIXTURE(Fixture, "IntegerParsingDoublePrefix") { - ScopedFastFlag luauLintParseIntegerIssues{"LuauLintParseIntegerIssues", true}; ScopedFastFlag luauErrorDoubleHexPrefix{"LuauErrorDoubleHexPrefix", false}; // Lint will be available until we start rejecting code LintResult result = lint(R"( diff --git a/tests/Normalize.test.cpp b/tests/Normalize.test.cpp index 42e9a933..31df707d 100644 --- a/tests/Normalize.test.cpp +++ b/tests/Normalize.test.cpp @@ -2,6 +2,7 @@ #include "Fixture.h" +#include "Luau/Common.h" #include "doctest.h" #include "Luau/Normalize.h" @@ -747,7 +748,6 @@ TEST_CASE_FIXTURE(Fixture, "cyclic_union") { ScopedFastFlag sff[] = { {"LuauLowerBoundsCalculation", true}, - {"LuauFixNormalizationOfCyclicUnions", true}, }; CheckResult result = check(R"( @@ -765,7 +765,6 @@ TEST_CASE_FIXTURE(Fixture, "cyclic_intersection") { ScopedFastFlag sff[] = { {"LuauLowerBoundsCalculation", true}, - {"LuauFixNormalizationOfCyclicUnions", true}, }; CheckResult result = check(R"( @@ -784,7 +783,6 @@ TEST_CASE_FIXTURE(Fixture, "intersection_of_tables_with_indexers") { ScopedFastFlag sff[] = { {"LuauLowerBoundsCalculation", true}, - {"LuauFixNormalizationOfCyclicUnions", true}, }; CheckResult result = check(R"( diff --git a/tests/Parser.test.cpp b/tests/Parser.test.cpp index 44a6b4ac..b4064cfb 100644 --- a/tests/Parser.test.cpp +++ b/tests/Parser.test.cpp @@ -682,7 +682,6 @@ TEST_CASE_FIXTURE(Fixture, "parse_numbers_binary") TEST_CASE_FIXTURE(Fixture, "parse_numbers_error") { - ScopedFastFlag luauLintParseIntegerIssues{"LuauLintParseIntegerIssues", true}; ScopedFastFlag luauErrorDoubleHexPrefix{"LuauErrorDoubleHexPrefix", true}; CHECK_EQ(getParseError("return 0b123"), "Malformed number"); @@ -695,7 +694,6 @@ TEST_CASE_FIXTURE(Fixture, "parse_numbers_error") TEST_CASE_FIXTURE(Fixture, "parse_numbers_error_soft") { - ScopedFastFlag luauLintParseIntegerIssues{"LuauLintParseIntegerIssues", true}; ScopedFastFlag luauErrorDoubleHexPrefix{"LuauErrorDoubleHexPrefix", false}; CHECK_EQ(getParseError("return 0x0x0x0x0x0x0x0"), "Malformed number"); diff --git a/tests/TypeInfer.aliases.test.cpp b/tests/TypeInfer.aliases.test.cpp index dd91467d..834391a7 100644 --- a/tests/TypeInfer.aliases.test.cpp +++ b/tests/TypeInfer.aliases.test.cpp @@ -198,8 +198,12 @@ TEST_CASE_FIXTURE(Fixture, "generic_aliases") LUAU_REQUIRE_ERROR_COUNT(1, result); + const char* expectedError = "Type '{ v: string }' could not be converted into 'T'\n" + "caused by:\n" + " Property 'v' is not compatible. Type 'string' could not be converted into 'number'"; + CHECK(result.errors[0].location == Location{{4, 31}, {4, 44}}); - CHECK(toString(result.errors[0]) == "Type '{ v: string }' could not be converted into 'T'"); + CHECK(toString(result.errors[0]) == expectedError); } TEST_CASE_FIXTURE(Fixture, "dependent_generic_aliases") @@ -215,8 +219,14 @@ TEST_CASE_FIXTURE(Fixture, "dependent_generic_aliases") LUAU_REQUIRE_ERROR_COUNT(1, result); + const char* expectedError = "Type '{ t: { v: string } }' could not be converted into 'U'\n" + "caused by:\n" + " Property 't' is not compatible. Type '{ v: string }' could not be converted into 'T'\n" + "caused by:\n" + " Property 'v' is not compatible. Type 'string' could not be converted into 'number'"; + CHECK(result.errors[0].location == Location{{4, 31}, {4, 52}}); - CHECK(toString(result.errors[0]) == "Type '{ t: { v: string } }' could not be converted into 'U'"); + CHECK(toString(result.errors[0]) == expectedError); } TEST_CASE_FIXTURE(Fixture, "mutually_recursive_generic_aliases") diff --git a/tests/TypeInfer.provisional.test.cpp b/tests/TypeInfer.provisional.test.cpp index 40ea0ca1..2c4c3504 100644 --- a/tests/TypeInfer.provisional.test.cpp +++ b/tests/TypeInfer.provisional.test.cpp @@ -558,11 +558,10 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "greedy_inference_with_shared_self_triggers_f CHECK_EQ("Not all codepaths in this function return 'self, a...'.", toString(result.errors[0])); } -TEST_CASE_FIXTURE(Fixture, "dcr_cant_partially_dispatch_a_constraint") +TEST_CASE_FIXTURE(Fixture, "dcr_can_partially_dispatch_a_constraint") { ScopedFastFlag sff[] = { {"DebugLuauDeferredConstraintResolution", true}, - {"LuauSpecialTypesAsterisked", true}, }; CheckResult result = check(R"( @@ -577,7 +576,6 @@ TEST_CASE_FIXTURE(Fixture, "dcr_cant_partially_dispatch_a_constraint") LUAU_REQUIRE_NO_ERRORS(result); - // We should be able to resolve this to number, but we're not there yet. // Solving this requires recognizing that we can partially solve the // following constraint: // @@ -586,7 +584,7 @@ TEST_CASE_FIXTURE(Fixture, "dcr_cant_partially_dispatch_a_constraint") // The correct thing for us to do is to consider the constraint dispatched, // but we need to also record a new constraint number <: *blocked* to finish // the job later. - CHECK("(a, *error-type*) -> ()" == toString(requireType("prime_iter"))); + CHECK("(a, number) -> ()" == toString(requireType("prime_iter"))); } TEST_CASE_FIXTURE(Fixture, "free_options_cannot_be_unified_together") diff --git a/tests/TypeInfer.singletons.test.cpp b/tests/TypeInfer.singletons.test.cpp index 7d98b5db..5ee956d7 100644 --- a/tests/TypeInfer.singletons.test.cpp +++ b/tests/TypeInfer.singletons.test.cpp @@ -314,6 +314,31 @@ caused by: toString(result.errors[0])); } +TEST_CASE_FIXTURE(Fixture, "parametric_tagged_union_alias") +{ + ScopedFastFlag sff[] = { + {"DebugLuauDeferredConstraintResolution", true}, + }; + + CheckResult result = check(R"( + type Ok = {success: true, result: T} + type Err = {success: false, error: T} + type Result = Ok | Err + + local a : Result = {success = false, result = "hotdogs"} + local b : Result = {success = true, result = "hotdogs"} + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + const std::string expectedError = "Type '{ result: string, success: false }' could not be converted into 'Err | Ok'\n" + "caused by:\n" + " None of the union options are compatible. For example: Table type '{ result: string, success: false }'" + " not compatible with type 'Err' because the former is missing field 'error'"; + + CHECK(toString(result.errors[0]) == expectedError); +} + TEST_CASE_FIXTURE(Fixture, "if_then_else_expression_singleton_options") { CheckResult result = check(R"( diff --git a/tests/TypeInfer.test.cpp b/tests/TypeInfer.test.cpp index e8bfb67f..a96c3676 100644 --- a/tests/TypeInfer.test.cpp +++ b/tests/TypeInfer.test.cpp @@ -1082,7 +1082,7 @@ TEST_CASE_FIXTURE(Fixture, "do_not_bind_a_free_table_to_a_union_containing_that_ )"); } -TEST_CASE_FIXTURE(Fixture, "types stored in astResolvedTypes") +TEST_CASE_FIXTURE(Fixture, "types_stored_in_astResolvedTypes") { CheckResult result = check(R"( type alias = typeof("hello") @@ -1122,4 +1122,41 @@ TEST_CASE_FIXTURE(Fixture, "bidirectional_checking_of_higher_order_function") CHECK(location.end.line == 4); } +TEST_CASE_FIXTURE(Fixture, "dcr_can_partially_dispatch_a_constraint") +{ + ScopedFastFlag sff[] = { + {"DebugLuauDeferredConstraintResolution", true}, + }; + + CheckResult result = check(R"( + local function hasDivisors(value: number) + end + + function prime_iter(state, index) + hasDivisors(index) + index += 1 + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + // Solving this requires recognizing that we can't dispatch a constraint + // like this without doing further work: + // + // (*blocked*) -> () <: (number) -> (b...) + // + // We solve this by searching both types for BlockedTypeVars and block the + // constraint on any we find. It also gets the job done, but I'm worried + // about the efficiency of doing so many deep type traversals and it may + // make us more prone to getting stuck on constraint cycles. + // + // If this doesn't pan out, a possible solution is to go further down the + // path of supporting partial constraint dispatch. The way it would work is + // that we'd dispatch the above constraint by binding b... to (), but we + // would append a new constraint number <: *blocked* to the constraint set + // to be solved later. This should be faster and theoretically less prone + // to cyclic constraint dependencies. + CHECK("(a, number) -> ()" == toString(requireType("prime_iter"))); +} + TEST_SUITE_END(); diff --git a/tools/faillist.txt b/tools/faillist.txt index 9c2df805..825fb2f6 100644 --- a/tools/faillist.txt +++ b/tools/faillist.txt @@ -10,7 +10,6 @@ AnnotationTests.luau_ice_triggers_an_ice_handler AnnotationTests.luau_print_is_magic_if_the_flag_is_set AnnotationTests.occurs_check_on_cyclic_intersection_typevar AnnotationTests.occurs_check_on_cyclic_union_typevar -AnnotationTests.too_many_type_params AnnotationTests.two_type_params AnnotationTests.use_type_required_from_another_file AstQuery.last_argument_function_call_type @@ -28,7 +27,6 @@ AutocompleteTest.autocomplete_interpolated_string AutocompleteTest.autocomplete_on_string_singletons AutocompleteTest.autocomplete_oop_implicit_self AutocompleteTest.autocomplete_repeat_middle_keyword -AutocompleteTest.autocomplete_string_singleton_equality AutocompleteTest.autocomplete_string_singleton_escape AutocompleteTest.autocomplete_string_singletons AutocompleteTest.autocomplete_while_middle_keywords @@ -85,7 +83,6 @@ AutocompleteTest.type_correct_expected_argument_type_pack_suggestion AutocompleteTest.type_correct_expected_argument_type_suggestion AutocompleteTest.type_correct_expected_argument_type_suggestion_optional AutocompleteTest.type_correct_expected_argument_type_suggestion_self -AutocompleteTest.type_correct_expected_return_type_pack_suggestion AutocompleteTest.type_correct_expected_return_type_suggestion AutocompleteTest.type_correct_full_type_suggestion AutocompleteTest.type_correct_function_no_parenthesis @@ -113,7 +110,6 @@ BuiltinTests.dont_add_definitions_to_persistent_types BuiltinTests.find_capture_types BuiltinTests.find_capture_types2 BuiltinTests.find_capture_types3 -BuiltinTests.getfenv BuiltinTests.global_singleton_types_are_sealed BuiltinTests.gmatch_capture_types BuiltinTests.gmatch_capture_types2 @@ -130,13 +126,9 @@ BuiltinTests.next_iterator_should_infer_types_and_type_check BuiltinTests.os_time_takes_optional_date_table BuiltinTests.pairs_iterator_should_infer_types_and_type_check BuiltinTests.see_thru_select -BuiltinTests.see_thru_select_count -BuiltinTests.select_on_variadic BuiltinTests.select_slightly_out_of_range BuiltinTests.select_way_out_of_range BuiltinTests.select_with_decimal_argument_is_rounded_down -BuiltinTests.select_with_variadic_typepack_tail -BuiltinTests.select_with_variadic_typepack_tail_and_string_head BuiltinTests.set_metatable_needs_arguments BuiltinTests.setmetatable_should_not_mutate_persisted_types BuiltinTests.sort @@ -147,20 +139,16 @@ BuiltinTests.string_format_arg_types_inference BuiltinTests.string_format_as_method BuiltinTests.string_format_correctly_ordered_types BuiltinTests.string_format_report_all_type_errors_at_correct_positions -BuiltinTests.string_format_tostring_specifier -BuiltinTests.string_format_tostring_specifier_type_constraint BuiltinTests.string_format_use_correct_argument BuiltinTests.string_format_use_correct_argument2 BuiltinTests.string_format_use_correct_argument3 BuiltinTests.string_lib_self_noself BuiltinTests.table_concat_returns_string -BuiltinTests.table_dot_remove_optionally_returns_generic BuiltinTests.table_freeze_is_generic BuiltinTests.table_insert_correctly_infers_type_of_array_2_args_overload BuiltinTests.table_insert_correctly_infers_type_of_array_3_args_overload BuiltinTests.table_pack BuiltinTests.table_pack_reduce -BuiltinTests.table_pack_variadic BuiltinTests.tonumber_returns_optional_number_type BuiltinTests.tonumber_returns_optional_number_type2 DefinitionTests.class_definition_overload_metamethods @@ -168,7 +156,6 @@ DefinitionTests.declaring_generic_functions DefinitionTests.definition_file_classes FrontendTest.ast_node_at_position FrontendTest.automatically_check_dependent_scripts -FrontendTest.dont_reparse_clean_file_when_linting FrontendTest.environments FrontendTest.imported_table_modification_2 FrontendTest.it_should_be_safe_to_stringify_errors_when_full_type_graph_is_discarded @@ -187,11 +174,8 @@ GenericsTests.check_mutual_generic_functions GenericsTests.correctly_instantiate_polymorphic_member_functions GenericsTests.do_not_always_instantiate_generic_intersection_types GenericsTests.do_not_infer_generic_functions -GenericsTests.dont_unify_bound_types GenericsTests.duplicate_generic_type_packs GenericsTests.duplicate_generic_types -GenericsTests.error_detailed_function_mismatch_generic_pack -GenericsTests.error_detailed_function_mismatch_generic_types GenericsTests.factories_of_generics GenericsTests.generic_argument_count_too_few GenericsTests.generic_argument_count_too_many @@ -205,30 +189,22 @@ GenericsTests.generic_type_pack_unification2 GenericsTests.generic_type_pack_unification3 GenericsTests.infer_generic_function_function_argument GenericsTests.infer_generic_function_function_argument_overloaded -GenericsTests.infer_generic_lib_function_function_argument GenericsTests.infer_generic_methods +GenericsTests.inferred_local_vars_can_be_polytypes GenericsTests.instantiate_cyclic_generic_function GenericsTests.instantiate_generic_function_in_assignments GenericsTests.instantiate_generic_function_in_assignments2 GenericsTests.instantiated_function_argument_names GenericsTests.instantiation_sharing_types GenericsTests.local_vars_can_be_instantiated_polytypes -GenericsTests.mutable_state_polymorphism GenericsTests.no_stack_overflow_from_quantifying GenericsTests.properties_can_be_instantiated_polytypes -GenericsTests.rank_N_types_via_typeof GenericsTests.reject_clashing_generic_and_pack_names GenericsTests.self_recursive_instantiated_param -IntersectionTypes.argument_is_intersection -IntersectionTypes.error_detailed_intersection_all -IntersectionTypes.error_detailed_intersection_part -IntersectionTypes.fx_intersection_as_argument -IntersectionTypes.fx_union_as_argument_fails IntersectionTypes.index_on_an_intersection_type_with_mixed_types IntersectionTypes.index_on_an_intersection_type_with_property_guaranteed_to_exist IntersectionTypes.index_on_an_intersection_type_works_at_arbitrary_depth IntersectionTypes.no_stack_overflow_from_flattenintersection -IntersectionTypes.overload_is_not_a_function IntersectionTypes.select_correct_union_fn IntersectionTypes.should_still_pick_an_overload_whose_arguments_are_unions IntersectionTypes.table_intersection_write_sealed @@ -236,17 +212,13 @@ IntersectionTypes.table_intersection_write_sealed_indirect IntersectionTypes.table_write_sealed_indirect isSubtype.intersection_of_tables isSubtype.table_with_table_prop -Linter.TableOperations ModuleTests.clone_self_property ModuleTests.deepClone_cyclic_table ModuleTests.do_not_clone_reexports -NonstrictModeTests.delay_function_does_not_require_its_argument_to_return_anything NonstrictModeTests.for_in_iterator_variables_are_any NonstrictModeTests.function_parameters_are_any NonstrictModeTests.inconsistent_module_return_types_are_ok -NonstrictModeTests.inconsistent_return_types_are_ok NonstrictModeTests.infer_nullary_function -NonstrictModeTests.infer_the_maximum_number_of_values_the_function_could_return NonstrictModeTests.inline_table_props_are_also_any NonstrictModeTests.local_tables_are_not_any NonstrictModeTests.locals_are_any_by_default @@ -294,20 +266,16 @@ ProvisionalTests.greedy_inference_with_shared_self_triggers_function_with_no_ret ProvisionalTests.invariant_table_properties_means_instantiating_tables_in_call_is_unsound ProvisionalTests.it_should_be_agnostic_of_actual_size ProvisionalTests.lower_bounds_calculation_is_too_permissive_with_overloaded_higher_order_functions -ProvisionalTests.lvalue_equals_another_lvalue_with_no_overlap ProvisionalTests.normalization_fails_on_certain_kinds_of_cyclic_tables -ProvisionalTests.operator_eq_completely_incompatible ProvisionalTests.pcall_returns_at_least_two_value_but_function_returns_nothing ProvisionalTests.setmetatable_constrains_free_type_into_free_table ProvisionalTests.typeguard_inference_incomplete ProvisionalTests.weirditer_should_not_loop_forever ProvisionalTests.while_body_are_also_refined RefinementTest.and_constraint -RefinementTest.and_or_peephole_refinement RefinementTest.apply_refinements_on_astexprindexexpr_whose_subscript_expr_is_constant_string RefinementTest.assert_a_to_be_truthy_then_assert_a_to_be_number RefinementTest.assert_non_binary_expressions_actually_resolve_constraints -RefinementTest.assign_table_with_refined_property_with_a_similar_type_is_illegal RefinementTest.call_a_more_specific_function_using_typeguard RefinementTest.correctly_lookup_a_shadowed_local_that_which_was_previously_refined RefinementTest.correctly_lookup_property_whose_base_was_previously_refined @@ -319,24 +287,19 @@ RefinementTest.discriminate_tag RefinementTest.either_number_or_string RefinementTest.eliminate_subclasses_of_instance RefinementTest.falsiness_of_TruthyPredicate_narrows_into_nil -RefinementTest.free_type_is_equal_to_an_lvalue -RefinementTest.impossible_type_narrow_is_not_an_error RefinementTest.index_on_a_refined_property RefinementTest.invert_is_truthy_constraint RefinementTest.invert_is_truthy_constraint_ifelse_expression RefinementTest.is_truthy_constraint RefinementTest.is_truthy_constraint_ifelse_expression -RefinementTest.lvalue_is_equal_to_a_term -RefinementTest.lvalue_is_equal_to_another_lvalue RefinementTest.lvalue_is_not_nil RefinementTest.merge_should_be_fully_agnostic_of_hashmap_ordering +RefinementTest.narrow_boolean_to_true_or_false RefinementTest.narrow_property_of_a_bounded_variable RefinementTest.narrow_this_large_union RefinementTest.nonoptional_type_can_narrow_to_nil_if_sense_is_true RefinementTest.not_a_and_not_b RefinementTest.not_a_and_not_b2 -RefinementTest.not_a_or_not_b -RefinementTest.not_a_or_not_b2 RefinementTest.not_and_constraint RefinementTest.not_t_or_some_prop_of_t RefinementTest.or_predicate_with_truthy_predicates @@ -344,7 +307,6 @@ RefinementTest.parenthesized_expressions_are_followed_through RefinementTest.refine_a_property_not_to_be_nil_through_an_intersection_table RefinementTest.refine_the_correct_types_opposite_of_when_a_is_not_number_or_string RefinementTest.refine_unknowns -RefinementTest.string_not_equal_to_string_or_nil RefinementTest.term_is_equal_to_an_lvalue RefinementTest.truthy_constraint_on_properties RefinementTest.type_assertion_expr_carry_its_constraints @@ -363,11 +325,9 @@ RefinementTest.typeguard_narrows_for_functions RefinementTest.typeguard_narrows_for_table RefinementTest.typeguard_not_to_be_string RefinementTest.typeguard_only_look_up_types_from_global_scope -RefinementTest.unknown_lvalue_is_not_synonymous_with_other_on_not_equal RefinementTest.what_nonsensical_condition RefinementTest.x_as_any_if_x_is_instance_elseif_x_is_table RefinementTest.x_is_not_instance_or_else_not_part -RuntimeLimits.typescript_port_of_Result_type TableTests.a_free_shape_can_turn_into_a_scalar_if_it_is_compatible TableTests.a_free_shape_cannot_turn_into_a_scalar_if_it_is_not_compatible TableTests.access_index_metamethod_that_returns_variadic @@ -383,7 +343,6 @@ TableTests.casting_sealed_tables_with_props_into_table_with_indexer TableTests.casting_tables_with_props_into_table_with_indexer3 TableTests.casting_tables_with_props_into_table_with_indexer4 TableTests.checked_prop_too_early -TableTests.confusing_indexing TableTests.defining_a_method_for_a_builtin_sealed_table_must_fail TableTests.defining_a_method_for_a_local_sealed_table_must_fail TableTests.defining_a_self_method_for_a_builtin_sealed_table_must_fail @@ -393,11 +352,7 @@ TableTests.dont_hang_when_trying_to_look_up_in_cyclic_metatable_index TableTests.dont_leak_free_table_props TableTests.dont_quantify_table_that_belongs_to_outer_scope TableTests.dont_suggest_exact_match_keys -TableTests.error_detailed_indexer_key -TableTests.error_detailed_indexer_value TableTests.error_detailed_metatable_prop -TableTests.error_detailed_prop -TableTests.error_detailed_prop_nested TableTests.expected_indexer_from_table_union TableTests.expected_indexer_value_type_extra TableTests.expected_indexer_value_type_extra_2 @@ -422,11 +377,7 @@ TableTests.infer_indexer_from_value_property_in_literal TableTests.inferred_return_type_of_free_table TableTests.inferring_crazy_table_should_also_be_quick TableTests.instantiate_table_cloning_3 -TableTests.instantiate_tables_at_scope_level TableTests.leaking_bad_metatable_errors -TableTests.length_operator_intersection -TableTests.length_operator_non_table_union -TableTests.length_operator_union TableTests.length_operator_union_errors TableTests.less_exponential_blowup_please TableTests.meta_add @@ -444,16 +395,15 @@ TableTests.open_table_unification_2 TableTests.pass_a_union_of_tables_to_a_function_that_requires_a_table TableTests.pass_a_union_of_tables_to_a_function_that_requires_a_table_2 TableTests.pass_incompatible_union_to_a_generic_table_without_crashing +TableTests.passing_compatible_unions_to_a_generic_table_without_crashing TableTests.persistent_sealed_table_is_immutable TableTests.prop_access_on_key_whose_types_mismatches TableTests.property_lookup_through_tabletypevar_metatable TableTests.quantify_even_that_table_was_never_exported_at_all -TableTests.quantify_metatables_of_metatables_of_table TableTests.quantifying_a_bound_var_works TableTests.reasonable_error_when_adding_a_nonexistent_property_to_an_array_like_table TableTests.result_is_always_any_if_lhs_is_any TableTests.result_is_bool_for_equality_operators_if_lhs_is_any -TableTests.right_table_missing_key TableTests.right_table_missing_key2 TableTests.scalar_is_a_subtype_of_a_compatible_polymorphic_shape_type TableTests.scalar_is_not_a_subtype_of_a_compatible_polymorphic_shape_type @@ -463,14 +413,11 @@ TableTests.shared_selfs_through_metatables TableTests.table_indexing_error_location TableTests.table_insert_should_cope_with_optional_properties_in_nonstrict TableTests.table_insert_should_cope_with_optional_properties_in_strict -TableTests.table_length TableTests.table_param_row_polymorphism_2 TableTests.table_param_row_polymorphism_3 TableTests.table_simple_call TableTests.table_subtyping_with_extra_props_dont_report_multiple_errors TableTests.table_subtyping_with_missing_props_dont_report_multiple_errors -TableTests.table_subtyping_with_missing_props_dont_report_multiple_errors2 -TableTests.table_unifies_into_map TableTests.tables_get_names_from_their_locals TableTests.tc_member_function TableTests.tc_member_function_2 @@ -480,10 +427,8 @@ TableTests.unification_of_unions_in_a_self_referential_type TableTests.unifying_tables_shouldnt_uaf2 TableTests.used_colon_instead_of_dot TableTests.used_dot_instead_of_colon -TableTests.width_subtyping ToDot.bound_table ToDot.function -ToDot.metatable ToDot.table ToString.exhaustive_toString_of_cyclic_table ToString.function_type_with_argument_names_generic @@ -500,7 +445,6 @@ TryUnifyTests.members_of_failed_typepack_unification_are_unified_with_errorType TryUnifyTests.result_of_failed_typepack_unification_is_constrained TryUnifyTests.typepack_unification_should_trim_free_tails TryUnifyTests.variadics_should_use_reversed_properly -TypeAliases.cli_38393_recursive_intersection_oom TypeAliases.forward_declared_alias_is_not_clobbered_by_prior_unification_with_any TypeAliases.generic_param_remap TypeAliases.mismatched_generic_pack_type_param @@ -517,29 +461,19 @@ TypeAliases.type_alias_local_mutation TypeAliases.type_alias_local_rename TypeAliases.type_alias_of_an_imported_recursive_generic_type TypeInfer.checking_should_not_ice -TypeInfer.cyclic_follow -TypeInfer.do_not_bind_a_free_table_to_a_union_containing_that_table TypeInfer.dont_report_type_errors_within_an_AstStatError TypeInfer.globals TypeInfer.globals2 TypeInfer.infer_assignment_value_types_mutable_lval TypeInfer.no_stack_overflow_from_isoptional TypeInfer.tc_after_error_recovery_no_replacement_name_in_error -TypeInfer.tc_if_else_expressions1 -TypeInfer.tc_if_else_expressions2 -TypeInfer.tc_if_else_expressions_expected_type_1 -TypeInfer.tc_if_else_expressions_expected_type_2 TypeInfer.tc_if_else_expressions_expected_type_3 -TypeInfer.tc_if_else_expressions_type_union TypeInfer.tc_interpolated_string_basic TypeInfer.tc_interpolated_string_constant_type TypeInfer.tc_interpolated_string_with_invalid_expression TypeInfer.type_infer_recursion_limit_no_ice TypeInferAnyError.assign_prop_to_table_by_calling_any_yields_any -TypeInferAnyError.can_get_length_of_any TypeInferAnyError.for_in_loop_iterator_is_any2 -TypeInferAnyError.length_of_error_type_does_not_produce_an_error -TypeInferAnyError.replace_every_free_type_when_unifying_a_complex_function_with_any TypeInferAnyError.union_of_types_regression_test TypeInferClasses.call_base_method TypeInferClasses.call_instance_method @@ -549,39 +483,23 @@ TypeInferClasses.class_type_mismatch_with_name_conflict TypeInferClasses.classes_can_have_overloaded_operators TypeInferClasses.classes_without_overloaded_operators_cannot_be_added TypeInferClasses.detailed_class_unification_error -TypeInferClasses.function_arguments_are_covariant -TypeInferClasses.higher_order_function_return_type_is_not_contravariant -TypeInferClasses.higher_order_function_return_values_are_covariant +TypeInferClasses.higher_order_function_arguments_are_contravariant TypeInferClasses.optional_class_field_access_error TypeInferClasses.table_class_unification_reports_sane_errors_for_missing_properties -TypeInferClasses.table_indexers_are_invariant -TypeInferClasses.table_properties_are_invariant TypeInferClasses.warn_when_prop_almost_matches TypeInferClasses.we_can_report_when_someone_is_trying_to_use_a_table_rather_than_a_class -TypeInferFunctions.another_recursive_local_function TypeInferFunctions.call_o_with_another_argument_after_foo_was_quantified -TypeInferFunctions.calling_function_with_anytypepack_doesnt_leak_free_types TypeInferFunctions.calling_function_with_incorrect_argument_type_yields_errors_spanning_argument -TypeInferFunctions.complicated_return_types_require_an_explicit_annotation TypeInferFunctions.dont_give_other_overloads_message_if_only_one_argument_matching_overload_exists TypeInferFunctions.dont_infer_parameter_types_for_functions_from_their_call_site TypeInferFunctions.duplicate_functions_with_different_signatures_not_allowed_in_nonstrict -TypeInferFunctions.error_detailed_function_mismatch_arg -TypeInferFunctions.error_detailed_function_mismatch_arg_count -TypeInferFunctions.error_detailed_function_mismatch_ret -TypeInferFunctions.error_detailed_function_mismatch_ret_count -TypeInferFunctions.error_detailed_function_mismatch_ret_mult TypeInferFunctions.free_is_not_bound_to_unknown TypeInferFunctions.func_expr_doesnt_leak_free TypeInferFunctions.function_cast_error_uses_correct_language -TypeInferFunctions.function_decl_non_self_sealed_overwrite TypeInferFunctions.function_decl_non_self_sealed_overwrite_2 TypeInferFunctions.function_decl_non_self_unsealed_overwrite TypeInferFunctions.function_does_not_return_enough_values TypeInferFunctions.function_statement_sealed_table_assignment_through_indexer -TypeInferFunctions.higher_order_function_2 -TypeInferFunctions.higher_order_function_4 -TypeInferFunctions.ignored_return_values TypeInferFunctions.improved_function_arg_mismatch_error_nonstrict TypeInferFunctions.improved_function_arg_mismatch_errors TypeInferFunctions.inconsistent_higher_order_function @@ -589,15 +507,11 @@ TypeInferFunctions.inconsistent_return_types TypeInferFunctions.infer_anonymous_function_arguments TypeInferFunctions.infer_return_type_from_selected_overload TypeInferFunctions.infer_that_function_does_not_return_a_table -TypeInferFunctions.it_is_ok_not_to_supply_enough_retvals TypeInferFunctions.list_all_overloads_if_no_overload_takes_given_argument_count TypeInferFunctions.list_only_alternative_overloads_that_match_argument_count TypeInferFunctions.no_lossy_function_type -TypeInferFunctions.occurs_check_failure_in_function_return_type TypeInferFunctions.quantify_constrained_types TypeInferFunctions.record_matching_overload -TypeInferFunctions.recursive_function -TypeInferFunctions.recursive_local_function TypeInferFunctions.report_exiting_without_return_nonstrict TypeInferFunctions.report_exiting_without_return_strict TypeInferFunctions.return_type_by_overload @@ -608,11 +522,7 @@ TypeInferFunctions.too_few_arguments_variadic_generic2 TypeInferFunctions.too_many_arguments TypeInferFunctions.too_many_return_values TypeInferFunctions.vararg_function_is_quantified -TypeInferLoops.for_in_loop_error_on_factory_not_returning_the_right_amount_of_values -TypeInferLoops.for_in_loop_with_custom_iterator -TypeInferLoops.for_in_loop_with_next TypeInferLoops.for_in_with_just_one_iterator_is_ok -TypeInferLoops.loop_iter_iter_metamethod TypeInferLoops.loop_iter_no_indexer_nonstrict TypeInferLoops.loop_iter_trailing_nil TypeInferLoops.loop_typecheck_crash_on_empty_optional @@ -631,7 +541,6 @@ TypeInferOOP.dont_suggest_using_colon_rather_than_dot_if_it_wont_help_2 TypeInferOOP.dont_suggest_using_colon_rather_than_dot_if_not_defined_with_colon TypeInferOOP.inferred_methods_of_free_tables_have_the_same_level_as_the_enclosing_table TypeInferOOP.inferring_hundreds_of_self_calls_should_not_suffocate_memory -TypeInferOOP.methods_are_topologically_sorted TypeInferOperators.and_adds_boolean TypeInferOperators.and_adds_boolean_no_superfluous_union TypeInferOperators.and_binexps_dont_unify @@ -641,13 +550,10 @@ TypeInferOperators.cannot_compare_tables_that_do_not_have_the_same_metatable TypeInferOperators.cannot_indirectly_compare_types_that_do_not_have_a_metatable TypeInferOperators.cannot_indirectly_compare_types_that_do_not_offer_overloaded_ordering_operators TypeInferOperators.cli_38355_recursive_union -TypeInferOperators.compare_numbers -TypeInferOperators.compare_strings TypeInferOperators.compound_assign_mismatch_metatable TypeInferOperators.compound_assign_mismatch_op TypeInferOperators.compound_assign_mismatch_result TypeInferOperators.concat_op_on_free_lhs_and_string_rhs -TypeInferOperators.concat_op_on_string_lhs_and_free_rhs TypeInferOperators.disallow_string_and_types_without_metatables_from_arithmetic_binary_ops TypeInferOperators.dont_strip_nil_from_rhs_or_operator TypeInferOperators.equality_operations_succeed_if_any_union_branch_succeeds @@ -655,18 +561,12 @@ TypeInferOperators.error_on_invalid_operand_types_to_relational_operators TypeInferOperators.error_on_invalid_operand_types_to_relational_operators2 TypeInferOperators.expected_types_through_binary_and TypeInferOperators.expected_types_through_binary_or -TypeInferOperators.in_nonstrict_mode_strip_nil_from_intersections_when_considering_relational_operators TypeInferOperators.infer_any_in_all_modes_when_lhs_is_unknown -TypeInferOperators.operator_eq_operands_are_not_subtypes_of_each_other_but_has_overlap -TypeInferOperators.operator_eq_verifies_types_do_intersect TypeInferOperators.or_joins_types TypeInferOperators.or_joins_types_with_no_extras -TypeInferOperators.primitive_arith_no_metatable -TypeInferOperators.primitive_arith_no_metatable_with_follows TypeInferOperators.primitive_arith_possible_metatable TypeInferOperators.produce_the_correct_error_message_when_comparing_a_table_with_a_metatable_with_one_that_does_not TypeInferOperators.refine_and_or -TypeInferOperators.some_primitive_binary_ops TypeInferOperators.strict_binary_op_where_lhs_unknown TypeInferOperators.strip_nil_from_lhs_or_operator TypeInferOperators.strip_nil_from_lhs_or_operator2 @@ -676,13 +576,11 @@ TypeInferOperators.typecheck_unary_len_error TypeInferOperators.typecheck_unary_minus TypeInferOperators.typecheck_unary_minus_error TypeInferOperators.unary_not_is_boolean -TypeInferOperators.unknown_type_in_comparison TypeInferOperators.UnknownGlobalCompoundAssign TypeInferPrimitives.CheckMethodsOfNumber TypeInferPrimitives.singleton_types TypeInferPrimitives.string_function_other TypeInferPrimitives.string_index -TypeInferPrimitives.string_length TypeInferPrimitives.string_method TypeInferUnknownNever.assign_to_global_which_is_never TypeInferUnknownNever.assign_to_local_which_is_never @@ -692,9 +590,7 @@ TypeInferUnknownNever.call_never TypeInferUnknownNever.dont_unify_operands_if_one_of_the_operand_is_never_in_any_ordering_operators TypeInferUnknownNever.index_on_union_of_tables_for_properties_that_is_never TypeInferUnknownNever.index_on_union_of_tables_for_properties_that_is_sorta_never -TypeInferUnknownNever.length_of_never TypeInferUnknownNever.math_operators_and_never -TypeInferUnknownNever.type_packs_containing_never_is_itself_uninhabitable TypeInferUnknownNever.type_packs_containing_never_is_itself_uninhabitable2 TypeInferUnknownNever.unary_minus_of_never TypePackTests.higher_order_function @@ -718,18 +614,17 @@ TypePackTests.type_alias_type_packs TypePackTests.type_alias_type_packs_errors TypePackTests.type_alias_type_packs_import TypePackTests.type_alias_type_packs_nested -TypePackTests.type_pack_hidden_free_tail_infinite_growth TypePackTests.type_pack_type_parameters +TypePackTests.unify_variadic_tails_in_arguments +TypePackTests.unify_variadic_tails_in_arguments_free TypePackTests.varargs_inference_through_multiple_scopes TypePackTests.variadic_packs -TypeSingletons.bool_singleton_subtype -TypeSingletons.bool_singletons -TypeSingletons.bool_singletons_mismatch TypeSingletons.enums_using_singletons TypeSingletons.enums_using_singletons_mismatch TypeSingletons.enums_using_singletons_subtyping TypeSingletons.error_detailed_tagged_union_mismatch_bool TypeSingletons.error_detailed_tagged_union_mismatch_string +TypeSingletons.function_call_with_singletons TypeSingletons.function_call_with_singletons_mismatch TypeSingletons.if_then_else_expression_singleton_options TypeSingletons.indexing_on_string_singletons @@ -752,7 +647,6 @@ TypeSingletons.widening_happens_almost_everywhere TypeSingletons.widening_happens_almost_everywhere_except_for_tables UnionTypes.error_detailed_optional UnionTypes.error_detailed_union_all -UnionTypes.error_detailed_union_part UnionTypes.error_takes_optional_arguments UnionTypes.index_on_a_union_type_with_missing_property UnionTypes.index_on_a_union_type_with_mixed_types @@ -771,6 +665,4 @@ UnionTypes.optional_union_follow UnionTypes.optional_union_functions UnionTypes.optional_union_members UnionTypes.optional_union_methods -UnionTypes.return_types_can_be_disjoint UnionTypes.table_union_write_indirect -UnionTypes.union_equality_comparisons diff --git a/tools/lvmexecute_split.py b/tools/lvmexecute_split.py new file mode 100644 index 00000000..10e3ccbb --- /dev/null +++ b/tools/lvmexecute_split.py @@ -0,0 +1,100 @@ +#!/usr/bin/python +# This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details + +# This code can be used to split lvmexecute.cpp VM switch into separate functions for use as native code generation fallbacks +import sys +import re + +input = sys.stdin.readlines() + +inst = "" + +header = """// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +// This file was generated by 'tools/lvmexecute_split.py' script, do not modify it by hand +#pragma once + +#include + +struct lua_State; +struct Closure; +typedef uint32_t Instruction; +typedef struct lua_TValue TValue; +typedef TValue* StkId; + +""" + +source = """// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +// This code is based on Lua 5.x implementation licensed under MIT License; see lua_LICENSE.txt for details +// This file was generated by 'tools/lvmexecute_split.py' script, do not modify it by hand +#include "Fallbacks.h" +#include "FallbacksProlog.h" + +""" + +function = "" + +state = 0 + +# parse with the state machine +for line in input: + # find the start of an instruction + if state == 0: + match = re.match("\s+VM_CASE\((LOP_[A-Z_0-9]+)\)", line) + + if match: + inst = match[1] + signature = "const Instruction* execute_" + inst + "(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k)" + header += signature + ";\n" + function = signature + "\n" + state = 1 + + # find the end of an instruction + elif state == 1: + # remove jumps back into the native code + if line == "#if LUA_CUSTOM_EXECUTION\n": + state = 2 + continue + + if line[0] == ' ': + finalline = line[12:-1] + "\n" + else: + finalline = line + + finalline = finalline.replace("VM_NEXT();", "return pc;"); + finalline = finalline.replace("goto exit;", "return NULL;"); + finalline = finalline.replace("return;", "return NULL;"); + + function += finalline + match = re.match(" }", line) + + if match: + # break is not supported + if inst == "LOP_BREAK": + function = "const Instruction* execute_" + inst + "(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k)\n" + function += "{\n LUAU_ASSERT(!\"Unsupported deprecated opcode\");\n LUAU_UNREACHABLE();\n}\n" + # handle fallthrough + elif inst == "LOP_NAMECALL": + function = function[:-len(finalline)] + function += " return pc;\n}\n" + + source += function + "\n" + state = 0 + + # skip LUA_CUSTOM_EXECUTION code blocks + elif state == 2: + if line == "#endif\n": + state = 3 + continue + + # skip extra line + elif state == 3: + state = 1 + +# make sure we found the ending +assert(state == 0) + +with open("Fallbacks.h", "w") as fp: + fp.writelines(header) + +with open("Fallbacks.cpp", "w") as fp: + fp.writelines(source) From a7f8c1045c88883549b7e1420067eef3c2a5bb70 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Sat, 24 Sep 2022 09:09:02 -0700 Subject: [PATCH 365/399] Update library.md Fix Markdown formatting --- docs/_pages/library.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/_pages/library.md b/docs/_pages/library.md index d82b6f28..2939f930 100644 --- a/docs/_pages/library.md +++ b/docs/_pages/library.md @@ -507,7 +507,7 @@ Returns a formatted version of the input arguments using a [printf-style format - `q`: expects a string and produces the same string quoted using double quotation marks, with escaped special characters if necessary - `s`: expects a string and produces the same string verbatim -The formats support modifiers `-`, `+`, ` `, `#` and `0`, as well as field width and precision modifiers - with the exception of `*`. +The formats support modifiers `-`, `+`, space, `#` and `0`, as well as field width and precision modifiers - with the exception of `*`. ``` function string.gmatch(s: string, p: string): From 5414cddb277f69ab1c4a4416b63aa750abccf2ca Mon Sep 17 00:00:00 2001 From: Alexander McCord <11488393+alexmccord@users.noreply.github.com> Date: Mon, 26 Sep 2022 14:18:04 -0700 Subject: [PATCH 366/399] Add `unknown` and `never` to typecheck.md (#682) Co-authored-by: Arseny Kapoulkine --- docs/_pages/typecheck.md | 55 ++++++++++++++++++++++++++++++++++++++-- 1 file changed, 53 insertions(+), 2 deletions(-) diff --git a/docs/_pages/typecheck.md b/docs/_pages/typecheck.md index 8e032da2..1a11406e 100644 --- a/docs/_pages/typecheck.md +++ b/docs/_pages/typecheck.md @@ -46,11 +46,11 @@ local a2: A = b1 -- ok local b2: B = a1 -- not ok ``` -## Primitive types +## Builtin types Lua VM supports 8 primitive types: `nil`, `string`, `number`, `boolean`, `table`, `function`, `thread`, and `userdata`. Of these, `table` and `function` are not represented by name, but have their dedicated syntax as covered in this [syntax document](syntax), and `userdata` is represented by [concrete types](#roblox-types); other types can be specified by their name. -Additionally, we also have `any` which is a special built-in type. It effectively disables all type checking, and thus should be used as last resort. +The type checker also provides the builtin types [`unknown`](#unknown-type), [`never`](#never-type), and [`any`](#any-type). ```lua local s = "foo" @@ -69,6 +69,57 @@ local a local b = nil ``` +### `unknown` type + +`unknown` is also said to be the _top_ type, that is it's a union of all types. + +```lua +local a: unknown = "hello world!" +local b: unknown = 5 +local c: unknown = function() return 5 end +``` + +Unlike `any`, `unknown` will not allow itself to be used as a different type! + +```lua +local function unknown(): unknown + return if math.random() > 0.5 then "hello world!" else 5 +end + +local a: string = unknown() -- not ok +local b: number = unknown() -- not ok +local c: string | number = unknown() -- not ok +``` + +In order to turn a variable of type `unknown` into a different type, you must apply [type refinements](#type-refinements) on that variable. + +```lua +local x = unknown() +if typeof(x) == "number" then + -- x : number +end +``` + +### `never` type + +`never` is also said to be the _bottom_ type, meaning there doesn't exist a value that inhabits the type `never`. In fact, it is the _dual_ of `unknown`. `never` is useful in many scenarios, and one such use case is when type refinements proves it impossible: + +```lua +local x = unknown() +if typeof(x) == "number" and typeof(x) == "string" then + -- x : never +end +``` + +### `any` type + +`any` is just like `unknown`, except that it allows itself to be used as an arbitrary type without further checks or annotations. Essentially, it's an opt-out from the type system entirely. + +```lua +local x: any = 5 +local y: string = x -- no type errors here! +``` + ## Function types Let's start with something simple. From 1acd66c97dfbdc1fc9f01396ff49f81cce34c78e Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Thu, 29 Sep 2022 12:13:00 -0700 Subject: [PATCH 367/399] Remove prototyping/ and prototyping.yml (#688) Superseded by https://github.com/luau-lang/agda-typeck --- .github/workflows/prototyping.yml | 68 --- prototyping/.gitignore | 10 - prototyping/Everything.agda | 8 - prototyping/Examples.agda | 7 - prototyping/Examples/OpSem.agda | 10 - prototyping/Examples/Run.agda | 23 - prototyping/Examples/Syntax.agda | 24 - prototyping/Examples/Type.agda | 28 - prototyping/FFI/Data/Aeson.agda | 77 --- prototyping/FFI/Data/ByteString.agda | 7 - prototyping/FFI/Data/Either.agda | 28 - prototyping/FFI/Data/HaskellInt.agda | 14 - prototyping/FFI/Data/HaskellString.agda | 16 - prototyping/FFI/Data/Maybe.agda | 14 - prototyping/FFI/Data/Scientific.agda | 21 - prototyping/FFI/Data/String.agda | 8 - prototyping/FFI/Data/Text/Encoding.agda | 10 - prototyping/FFI/Data/Vector.agda | 53 -- prototyping/FFI/IO.agda | 34 -- prototyping/FFI/System/Exit.agda | 29 -- prototyping/Interpreter.agda | 50 -- prototyping/Luau/Addr.agda | 18 - prototyping/Luau/Addr/ToString.agda | 8 - prototyping/Luau/Heap.agda | 49 -- prototyping/Luau/OpSem.agda | 143 ------ prototyping/Luau/ResolveOverloads.agda | 98 ---- prototyping/Luau/Run.agda | 29 -- prototyping/Luau/RuntimeError.agda | 41 -- prototyping/Luau/RuntimeError/ToString.agda | 31 -- prototyping/Luau/RuntimeType.agda | 17 - prototyping/Luau/RuntimeType/ToString.agda | 11 - prototyping/Luau/StrictMode.agda | 194 ------- prototyping/Luau/StrictMode/ToString.agda | 61 --- prototyping/Luau/Substitution.agda | 28 - prototyping/Luau/Subtyping.agda | 67 --- prototyping/Luau/Syntax.agda | 110 ---- prototyping/Luau/Syntax/FromJSON.agda | 201 -------- prototyping/Luau/Syntax/ToString.agda | 83 --- prototyping/Luau/Type.agda | 164 ------ prototyping/Luau/Type/FromJSON.agda | 72 --- prototyping/Luau/Type/ToString.agda | 29 -- prototyping/Luau/TypeCheck.agda | 160 ------ prototyping/Luau/TypeNormalization.agda | 65 --- prototyping/Luau/TypeSaturation.agda | 66 --- prototyping/Luau/Var.agda | 16 - prototyping/Luau/Var/ToString.agda | 8 - prototyping/Luau/VarCtxt.agda | 43 -- prototyping/PrettyPrinter.agda | 34 -- prototyping/Properties.agda | 15 - prototyping/Properties/Contradiction.agda | 9 - prototyping/Properties/Dec.agda | 7 - prototyping/Properties/DecSubtyping.agda | 174 ------- prototyping/Properties/Equality.agda | 23 - prototyping/Properties/Functions.agda | 6 - prototyping/Properties/Product.agda | 14 - prototyping/Properties/Remember.agda | 9 - prototyping/Properties/ResolveOverloads.agda | 189 ------- prototyping/Properties/Step.agda | 172 ------- prototyping/Properties/StrictMode.agda | 385 -------------- prototyping/Properties/Subtyping.agda | 481 ------------------ prototyping/Properties/TypeCheck.agda | 100 ---- prototyping/Properties/TypeNormalization.agda | 408 --------------- prototyping/Properties/TypeSaturation.agda | 433 ---------------- prototyping/README.md | 45 -- .../Interpreter/binary_equality_bools/in.lua | 1 - .../Interpreter/binary_equality_bools/out.txt | 4 - .../binary_equality_numbers/in.lua | 1 - .../binary_equality_numbers/out.txt | 4 - .../Tests/Interpreter/binary_exps/in.lua | 1 - .../Tests/Interpreter/binary_exps/out.txt | 4 - .../concat_number_and_string/in.lua | 3 - .../concat_number_and_string/out.txt | 12 - .../Interpreter/concat_two_strings/in.lua | 3 - .../Interpreter/concat_two_strings/out.txt | 6 - .../Tests/Interpreter/return_nil/in.lua | 5 - .../Tests/Interpreter/return_nil/out.txt | 7 - .../Tests/Interpreter/return_string/in.lua | 1 - .../Tests/Interpreter/return_string/out.txt | 4 - .../Tests/PrettyPrinter/smoke_test/in.lua | 19 - .../Tests/PrettyPrinter/smoke_test/out.txt | 19 - prototyping/Utility/Bool.agda | 16 - prototyping/tests.py | 197 ------- 82 files changed, 5162 deletions(-) delete mode 100644 .github/workflows/prototyping.yml delete mode 100644 prototyping/.gitignore delete mode 100644 prototyping/Everything.agda delete mode 100644 prototyping/Examples.agda delete mode 100644 prototyping/Examples/OpSem.agda delete mode 100644 prototyping/Examples/Run.agda delete mode 100644 prototyping/Examples/Syntax.agda delete mode 100644 prototyping/Examples/Type.agda delete mode 100644 prototyping/FFI/Data/Aeson.agda delete mode 100644 prototyping/FFI/Data/ByteString.agda delete mode 100644 prototyping/FFI/Data/Either.agda delete mode 100644 prototyping/FFI/Data/HaskellInt.agda delete mode 100644 prototyping/FFI/Data/HaskellString.agda delete mode 100644 prototyping/FFI/Data/Maybe.agda delete mode 100644 prototyping/FFI/Data/Scientific.agda delete mode 100644 prototyping/FFI/Data/String.agda delete mode 100644 prototyping/FFI/Data/Text/Encoding.agda delete mode 100644 prototyping/FFI/Data/Vector.agda delete mode 100644 prototyping/FFI/IO.agda delete mode 100644 prototyping/FFI/System/Exit.agda delete mode 100644 prototyping/Interpreter.agda delete mode 100644 prototyping/Luau/Addr.agda delete mode 100644 prototyping/Luau/Addr/ToString.agda delete mode 100644 prototyping/Luau/Heap.agda delete mode 100644 prototyping/Luau/OpSem.agda delete mode 100644 prototyping/Luau/ResolveOverloads.agda delete mode 100644 prototyping/Luau/Run.agda delete mode 100644 prototyping/Luau/RuntimeError.agda delete mode 100644 prototyping/Luau/RuntimeError/ToString.agda delete mode 100644 prototyping/Luau/RuntimeType.agda delete mode 100644 prototyping/Luau/RuntimeType/ToString.agda delete mode 100644 prototyping/Luau/StrictMode.agda delete mode 100644 prototyping/Luau/StrictMode/ToString.agda delete mode 100644 prototyping/Luau/Substitution.agda delete mode 100644 prototyping/Luau/Subtyping.agda delete mode 100644 prototyping/Luau/Syntax.agda delete mode 100644 prototyping/Luau/Syntax/FromJSON.agda delete mode 100644 prototyping/Luau/Syntax/ToString.agda delete mode 100644 prototyping/Luau/Type.agda delete mode 100644 prototyping/Luau/Type/FromJSON.agda delete mode 100644 prototyping/Luau/Type/ToString.agda delete mode 100644 prototyping/Luau/TypeCheck.agda delete mode 100644 prototyping/Luau/TypeNormalization.agda delete mode 100644 prototyping/Luau/TypeSaturation.agda delete mode 100644 prototyping/Luau/Var.agda delete mode 100644 prototyping/Luau/Var/ToString.agda delete mode 100644 prototyping/Luau/VarCtxt.agda delete mode 100644 prototyping/PrettyPrinter.agda delete mode 100644 prototyping/Properties.agda delete mode 100644 prototyping/Properties/Contradiction.agda delete mode 100644 prototyping/Properties/Dec.agda delete mode 100644 prototyping/Properties/DecSubtyping.agda delete mode 100644 prototyping/Properties/Equality.agda delete mode 100644 prototyping/Properties/Functions.agda delete mode 100644 prototyping/Properties/Product.agda delete mode 100644 prototyping/Properties/Remember.agda delete mode 100644 prototyping/Properties/ResolveOverloads.agda delete mode 100644 prototyping/Properties/Step.agda delete mode 100644 prototyping/Properties/StrictMode.agda delete mode 100644 prototyping/Properties/Subtyping.agda delete mode 100644 prototyping/Properties/TypeCheck.agda delete mode 100644 prototyping/Properties/TypeNormalization.agda delete mode 100644 prototyping/Properties/TypeSaturation.agda delete mode 100644 prototyping/README.md delete mode 100644 prototyping/Tests/Interpreter/binary_equality_bools/in.lua delete mode 100644 prototyping/Tests/Interpreter/binary_equality_bools/out.txt delete mode 100644 prototyping/Tests/Interpreter/binary_equality_numbers/in.lua delete mode 100644 prototyping/Tests/Interpreter/binary_equality_numbers/out.txt delete mode 100644 prototyping/Tests/Interpreter/binary_exps/in.lua delete mode 100644 prototyping/Tests/Interpreter/binary_exps/out.txt delete mode 100644 prototyping/Tests/Interpreter/concat_number_and_string/in.lua delete mode 100644 prototyping/Tests/Interpreter/concat_number_and_string/out.txt delete mode 100644 prototyping/Tests/Interpreter/concat_two_strings/in.lua delete mode 100644 prototyping/Tests/Interpreter/concat_two_strings/out.txt delete mode 100644 prototyping/Tests/Interpreter/return_nil/in.lua delete mode 100644 prototyping/Tests/Interpreter/return_nil/out.txt delete mode 100644 prototyping/Tests/Interpreter/return_string/in.lua delete mode 100644 prototyping/Tests/Interpreter/return_string/out.txt delete mode 100644 prototyping/Tests/PrettyPrinter/smoke_test/in.lua delete mode 100644 prototyping/Tests/PrettyPrinter/smoke_test/out.txt delete mode 100644 prototyping/Utility/Bool.agda delete mode 100755 prototyping/tests.py diff --git a/.github/workflows/prototyping.yml b/.github/workflows/prototyping.yml deleted file mode 100644 index ff66881d..00000000 --- a/.github/workflows/prototyping.yml +++ /dev/null @@ -1,68 +0,0 @@ -name: prototyping - -on: - pull_request: - paths: - - '.github/workflows/prototyping.yml' - - 'prototyping/**' - -jobs: - linux: - strategy: - matrix: - agda: [2.6.2.2] - hackageDate: ["2022-04-07"] - hackageTime: ["23:06:28"] - name: prototyping - runs-on: ubuntu-latest - steps: - - uses: actions/checkout@v1 - - uses: actions/cache@v2 - with: - path: ~/.cabal/store - key: "prototyping-${{ runner.os }}-${{ matrix.agda }}-${{ matrix.hackageDate }}-${{ matrix.hackageTime }}" - - uses: actions/cache@v2 - id: luau-ast-cache - with: - path: ./build - key: prototyping-${{ runner.os }}-${{ hashFiles('Ast/**', 'Analysis/**', 'CLI/Ast.cpp', 'CLI/FileUtils.*')}} - - name: install cabal - run: sudo apt-get install -y cabal-install - - name: cabal update - working-directory: prototyping - run: cabal v2-update "hackage.haskell.org,${{ matrix.hackageDate }}T${{ matrix.hackageTime }}Z" - - name: cabal install - working-directory: prototyping - run: | - cabal install --lib scientific vector aeson --package-env . - cabal install --allow-newer Agda-${{ matrix.agda }} - - name: check targets - working-directory: prototyping - run: | - ~/.cabal/bin/agda Everything.agda - - name: build executables - working-directory: prototyping - run: | - ~/.cabal/bin/agda --compile PrettyPrinter.agda - ~/.cabal/bin/agda --compile Interpreter.agda - - name: cmake configure - if: steps.luau-ast-cache.outputs.cache-hit != 'true' - run: | - mkdir -p build - cd build - cmake build ../ - - name: cmake build luau-ast - if: steps.luau-ast-cache.outputs.cache-hit != 'true' - run: | - cmake --build ./build --target Luau.Ast.CLI - - name: run tests - working-directory: prototyping - run: | - mkdir test-failures - python tests.py -l ../build/luau-ast --write-diff-failures --diff-failure-location test-failures/ - - uses: actions/upload-artifact@v2 - if: failure() - with: - name: test failures - path: prototyping/test-failures - retention-days: 5 diff --git a/prototyping/.gitignore b/prototyping/.gitignore deleted file mode 100644 index 31e8c073..00000000 --- a/prototyping/.gitignore +++ /dev/null @@ -1,10 +0,0 @@ -*~ -*.agdai -Main -MAlonzo -PrettyPrinter -Interpreter -!Tests/Interpreter -!Tests/PrettyPrinter -.ghc.* -test-failures/ diff --git a/prototyping/Everything.agda b/prototyping/Everything.agda deleted file mode 100644 index d8a1fd5a..00000000 --- a/prototyping/Everything.agda +++ /dev/null @@ -1,8 +0,0 @@ -{-# OPTIONS --rewriting #-} - -module Everything where - -import Examples -import Properties -import PrettyPrinter -import Interpreter diff --git a/prototyping/Examples.agda b/prototyping/Examples.agda deleted file mode 100644 index 212067b7..00000000 --- a/prototyping/Examples.agda +++ /dev/null @@ -1,7 +0,0 @@ -{-# OPTIONS --rewriting #-} -module Examples where - -import Examples.Syntax -import Examples.OpSem -import Examples.Run -import Examples.Type diff --git a/prototyping/Examples/OpSem.agda b/prototyping/Examples/OpSem.agda deleted file mode 100644 index c3f74ec1..00000000 --- a/prototyping/Examples/OpSem.agda +++ /dev/null @@ -1,10 +0,0 @@ -{-# OPTIONS --rewriting #-} - -module Examples.OpSem where - -open import Luau.OpSem using (_⊢_⟶ᴱ_⊣_; _⊢_⟶ᴮ_⊣_; subst) -open import Luau.Syntax using (Block; var; val; nil; local_←_; _∙_; done; return; block_is_end) -open import Luau.Heap using (∅) - -ex1 : ∅ ⊢ (local (var "x") ← val nil ∙ return (var "x") ∙ done) ⟶ᴮ (return (val nil) ∙ done) ⊣ ∅ -ex1 = subst nil diff --git a/prototyping/Examples/Run.agda b/prototyping/Examples/Run.agda deleted file mode 100644 index 84ebf84e..00000000 --- a/prototyping/Examples/Run.agda +++ /dev/null @@ -1,23 +0,0 @@ -{-# OPTIONS --rewriting #-} - -module Examples.Run where - -open import Agda.Builtin.Equality using (_≡_; refl) -open import Agda.Builtin.Bool using (true; false) -open import Luau.Syntax using (nil; var; _$_; function_is_end; return; _∙_; done; _⟨_⟩; number; binexp; +; <; val; bool; ~=; string) -open import Luau.Run using (run; return) - -ex1 : (run (function "id" ⟨ var "x" ⟩ is return (var "x") ∙ done end ∙ return (var "id" $ val nil) ∙ done) ≡ return nil _) -ex1 = refl - -ex2 : (run (function "fn" ⟨ var "x" ⟩ is return (val (number 123.0)) ∙ done end ∙ return (var "fn" $ val nil) ∙ done) ≡ return (number 123.0) _) -ex2 = refl - -ex3 : (run (function "fn" ⟨ var "x" ⟩ is return (binexp (val (number 1.0)) + (val (number 2.0))) ∙ done end ∙ return (var "fn" $ val nil) ∙ done) ≡ return (number 3.0) _) -ex3 = refl - -ex4 : (run (function "fn" ⟨ var "x" ⟩ is return (binexp (val (number 1.0)) < (val (number 2.0))) ∙ done end ∙ return (var "fn" $ val nil) ∙ done) ≡ return (bool true) _) -ex4 = refl - -ex5 : (run (function "fn" ⟨ var "x" ⟩ is return (binexp (val (string "foo")) ~= (val (string "bar"))) ∙ done end ∙ return (var "fn" $ val nil) ∙ done) ≡ return (bool true) _) -ex5 = refl diff --git a/prototyping/Examples/Syntax.agda b/prototyping/Examples/Syntax.agda deleted file mode 100644 index e2793023..00000000 --- a/prototyping/Examples/Syntax.agda +++ /dev/null @@ -1,24 +0,0 @@ -module Examples.Syntax where - -open import Agda.Builtin.Equality using (_≡_; refl) -open import FFI.Data.String using (_++_) -open import Luau.Syntax using (var; _$_; return; val; nil; function_is_end; local_←_; done; _∙_; _⟨_⟩) -open import Luau.Syntax.ToString using (exprToString; blockToString) - -ex1 : exprToString(function "" ⟨ var "x" ⟩ is return (var "f" $ var "x") ∙ done end) ≡ - "function(x)\n" ++ - " return f(x)\n" ++ - "end" -ex1 = refl - -ex2 : blockToString(local var "x" ← (val nil) ∙ return (var "x") ∙ done) ≡ - "local x = nil\n" ++ - "return x" -ex2 = refl - -ex3 : blockToString(function "f" ⟨ var "x" ⟩ is return (var "x") ∙ done end ∙ return (var "f") ∙ done) ≡ - "local function f(x)\n" ++ - " return x\n" ++ - "end\n" ++ - "return f" -ex3 = refl diff --git a/prototyping/Examples/Type.agda b/prototyping/Examples/Type.agda deleted file mode 100644 index 3fdd37d5..00000000 --- a/prototyping/Examples/Type.agda +++ /dev/null @@ -1,28 +0,0 @@ -module Examples.Type where - -open import Agda.Builtin.Equality using (_≡_; refl) -open import FFI.Data.String using (_++_) -open import Luau.Type using (nil; _∪_; _∩_; _⇒_) -open import Luau.Type.ToString using (typeToString) - -ex1 : typeToString(nil) ≡ "nil" -ex1 = refl - -ex2 : typeToString(nil ⇒ nil) ≡ "(nil) -> nil" -ex2 = refl - -ex3 : typeToString(nil ⇒ (nil ⇒ nil)) ≡ "(nil) -> (nil) -> nil" -ex3 = refl - -ex4 : typeToString(nil ∪ (nil ⇒ (nil ⇒ nil))) ≡ "((nil) -> (nil) -> nil)?" -ex4 = refl - -ex5 : typeToString(nil ⇒ ((nil ⇒ nil) ∪ nil)) ≡ "(nil) -> ((nil) -> nil)?" -ex5 = refl - -ex6 : typeToString((nil ⇒ nil) ∪ (nil ⇒ (nil ⇒ nil))) ≡ "((nil) -> nil | (nil) -> (nil) -> nil)" -ex6 = refl - -ex7 : typeToString((nil ⇒ nil) ∪ ((nil ⇒ (nil ⇒ nil)) ∪ nil)) ≡ "((nil) -> nil | (nil) -> (nil) -> nil)?" -ex7 = refl - diff --git a/prototyping/FFI/Data/Aeson.agda b/prototyping/FFI/Data/Aeson.agda deleted file mode 100644 index 43014719..00000000 --- a/prototyping/FFI/Data/Aeson.agda +++ /dev/null @@ -1,77 +0,0 @@ -{-# OPTIONS --rewriting #-} - -module FFI.Data.Aeson where - -open import Agda.Builtin.Equality using (_≡_) -open import Agda.Builtin.Equality.Rewrite using () -open import Agda.Builtin.Bool using (Bool) -open import Agda.Builtin.String using (String) - -open import FFI.Data.ByteString using (ByteString) -open import FFI.Data.HaskellString using (HaskellString; pack) -open import FFI.Data.Maybe using (Maybe; just; nothing) -open import FFI.Data.Either using (Either; mapL) -open import FFI.Data.Scientific using (Scientific) -open import FFI.Data.Vector using (Vector) - -open import Properties.Equality using (_≢_) - -{-# FOREIGN GHC import qualified Data.Aeson #-} -{-# FOREIGN GHC import qualified Data.Aeson.Key #-} -{-# FOREIGN GHC import qualified Data.Aeson.KeyMap #-} - -postulate - KeyMap : Set → Set - Key : Set - fromString : String → Key - toString : Key → String - empty : ∀ {A} → KeyMap A - singleton : ∀ {A} → Key → A → (KeyMap A) - insert : ∀ {A} → Key → A → (KeyMap A) → (KeyMap A) - delete : ∀ {A} → Key → (KeyMap A) → (KeyMap A) - unionWith : ∀ {A} → (A → A → A) → (KeyMap A) → (KeyMap A) → (KeyMap A) - lookup : ∀ {A} → Key -> KeyMap A -> Maybe A -{-# POLARITY KeyMap ++ #-} -{-# COMPILE GHC KeyMap = type Data.Aeson.KeyMap.KeyMap #-} -{-# COMPILE GHC Key = type Data.Aeson.Key.Key #-} -{-# COMPILE GHC fromString = Data.Aeson.Key.fromText #-} -{-# COMPILE GHC toString = Data.Aeson.Key.toText #-} -{-# COMPILE GHC empty = \_ -> Data.Aeson.KeyMap.empty #-} -{-# COMPILE GHC singleton = \_ -> Data.Aeson.KeyMap.singleton #-} -{-# COMPILE GHC insert = \_ -> Data.Aeson.KeyMap.insert #-} -{-# COMPILE GHC delete = \_ -> Data.Aeson.KeyMap.delete #-} -{-# COMPILE GHC unionWith = \_ -> Data.Aeson.KeyMap.unionWith #-} -{-# COMPILE GHC lookup = \_ -> Data.Aeson.KeyMap.lookup #-} - -postulate lookup-insert : ∀ {A} k v (m : KeyMap A) → (lookup k (insert k v m) ≡ just v) -postulate lookup-empty : ∀ {A} k → (lookup {A} k empty ≡ nothing) -postulate lookup-insert-not : ∀ {A} j k v (m : KeyMap A) → (j ≢ k) → (lookup k m ≡ lookup k (insert j v m)) -postulate singleton-insert-empty : ∀ {A} k (v : A) → (singleton k v ≡ insert k v empty) -postulate insert-swap : ∀ {A} j k (v w : A) m → (j ≢ k) → insert j v (insert k w m) ≡ insert k w (insert j v m) -postulate insert-over : ∀ {A} j k (v w : A) m → (j ≡ k) → insert j v (insert k w m) ≡ insert j v m -postulate to-from : ∀ k → toString(fromString k) ≡ k -postulate from-to : ∀ k → fromString(toString k) ≡ k - -{-# REWRITE lookup-insert lookup-empty singleton-insert-empty #-} - -data Value : Set where - object : KeyMap Value → Value - array : Vector Value → Value - string : String → Value - number : Scientific → Value - bool : Bool → Value - null : Value -{-# COMPILE GHC Value = data Data.Aeson.Value (Data.Aeson.Object|Data.Aeson.Array|Data.Aeson.String|Data.Aeson.Number|Data.Aeson.Bool|Data.Aeson.Null) #-} - -Object = KeyMap Value -Array = Vector Value - -postulate - decode : ByteString → Maybe Value - eitherHDecode : ByteString → Either HaskellString Value -{-# COMPILE GHC decode = Data.Aeson.decodeStrict #-} -{-# COMPILE GHC eitherHDecode = Data.Aeson.eitherDecodeStrict #-} - -eitherDecode : ByteString → Either String Value -eitherDecode bytes = mapL pack (eitherHDecode bytes) - diff --git a/prototyping/FFI/Data/ByteString.agda b/prototyping/FFI/Data/ByteString.agda deleted file mode 100644 index 670c5234..00000000 --- a/prototyping/FFI/Data/ByteString.agda +++ /dev/null @@ -1,7 +0,0 @@ -module FFI.Data.ByteString where - -{-# FOREIGN GHC import qualified Data.ByteString #-} - -postulate ByteString : Set -{-# COMPILE GHC ByteString = type Data.ByteString.ByteString #-} - diff --git a/prototyping/FFI/Data/Either.agda b/prototyping/FFI/Data/Either.agda deleted file mode 100644 index 8a5d3e66..00000000 --- a/prototyping/FFI/Data/Either.agda +++ /dev/null @@ -1,28 +0,0 @@ -module FFI.Data.Either where - -{-# FOREIGN GHC import qualified Data.Either #-} - -data Either (A B : Set) : Set where - Left : A → Either A B - Right : B → Either A B -{-# COMPILE GHC Either = data Data.Either.Either (Data.Either.Left|Data.Either.Right) #-} - -swapLR : ∀ {A B} → Either A B → Either B A -swapLR (Left x) = Right x -swapLR (Right x) = Left x - -mapL : ∀ {A B C} → (A → B) → Either A C → Either B C -mapL f (Left x) = Left (f x) -mapL f (Right x) = Right x - -mapR : ∀ {A B C} → (B → C) → Either A B → Either A C -mapR f (Left x) = Left x -mapR f (Right x) = Right (f x) - -mapLR : ∀ {A B C D} → (A → B) → (C → D) → Either A C → Either B D -mapLR f g (Left x) = Left (f x) -mapLR f g (Right x) = Right (g x) - -cond : ∀ {A B C : Set} → (A → C) → (B → C) → (Either A B) → C -cond f g (Left x) = f x -cond f g (Right x) = g x diff --git a/prototyping/FFI/Data/HaskellInt.agda b/prototyping/FFI/Data/HaskellInt.agda deleted file mode 100644 index 9ab0868e..00000000 --- a/prototyping/FFI/Data/HaskellInt.agda +++ /dev/null @@ -1,14 +0,0 @@ -module FFI.Data.HaskellInt where - -open import Agda.Builtin.Int using (Int) - -{-# FOREIGN GHC import qualified Data.Int #-} - -postulate HaskellInt : Set -{-# COMPILE GHC HaskellInt = type Data.Int.Int #-} - -postulate - intToHaskellInt : Int → HaskellInt - haskellIntToInt : HaskellInt → Int -{-# COMPILE GHC intToHaskellInt = fromIntegral #-} -{-# COMPILE GHC haskellIntToInt = fromIntegral #-} diff --git a/prototyping/FFI/Data/HaskellString.agda b/prototyping/FFI/Data/HaskellString.agda deleted file mode 100644 index cb4ace37..00000000 --- a/prototyping/FFI/Data/HaskellString.agda +++ /dev/null @@ -1,16 +0,0 @@ -module FFI.Data.HaskellString where - -open import Agda.Builtin.String using (String) - -{-# FOREIGN GHC import qualified Data.String #-} -{-# FOREIGN GHC import qualified Data.Text #-} - -postulate HaskellString : Set -{-# COMPILE GHC HaskellString = type Data.String.String #-} - -postulate pack : HaskellString → String -{-# COMPILE GHC pack = Data.Text.pack #-} - -postulate unpack : String → HaskellString -{-# COMPILE GHC unpack = Data.Text.unpack #-} - diff --git a/prototyping/FFI/Data/Maybe.agda b/prototyping/FFI/Data/Maybe.agda deleted file mode 100644 index 58fd1486..00000000 --- a/prototyping/FFI/Data/Maybe.agda +++ /dev/null @@ -1,14 +0,0 @@ -module FFI.Data.Maybe where - -open import Agda.Builtin.Equality using (_≡_; refl) - -{-# FOREIGN GHC import qualified Data.Maybe #-} - -data Maybe (A : Set) : Set where - nothing : Maybe A - just : A → Maybe A -{-# COMPILE GHC Maybe = data Data.Maybe.Maybe (Data.Maybe.Nothing|Data.Maybe.Just) #-} - -just-inv : ∀ {A} {x y : A} → (just x ≡ just y) → (x ≡ y) -just-inv refl = refl - diff --git a/prototyping/FFI/Data/Scientific.agda b/prototyping/FFI/Data/Scientific.agda deleted file mode 100644 index 772d3367..00000000 --- a/prototyping/FFI/Data/Scientific.agda +++ /dev/null @@ -1,21 +0,0 @@ -module FFI.Data.Scientific where - -open import Agda.Builtin.Float using (Float) -open import FFI.Data.String using (String) -open import FFI.Data.HaskellString using (HaskellString; pack; unpack) - -{-# FOREIGN GHC import qualified Data.Scientific #-} -{-# FOREIGN GHC import qualified Text.Show #-} - -postulate Scientific : Set -{-# COMPILE GHC Scientific = type Data.Scientific.Scientific #-} - -postulate - showHaskell : Scientific → HaskellString - toFloat : Scientific → Float - -{-# COMPILE GHC showHaskell = \x -> Text.Show.show x #-} -{-# COMPILE GHC toFloat = \x -> Data.Scientific.toRealFloat x #-} - -show : Scientific → String -show x = pack (showHaskell x) diff --git a/prototyping/FFI/Data/String.agda b/prototyping/FFI/Data/String.agda deleted file mode 100644 index 5bb315cf..00000000 --- a/prototyping/FFI/Data/String.agda +++ /dev/null @@ -1,8 +0,0 @@ -module FFI.Data.String where - -import Agda.Builtin.String - -String = Agda.Builtin.String.String - -infixr 5 _++_ -_++_ = Agda.Builtin.String.primStringAppend diff --git a/prototyping/FFI/Data/Text/Encoding.agda b/prototyping/FFI/Data/Text/Encoding.agda deleted file mode 100644 index 54f32486..00000000 --- a/prototyping/FFI/Data/Text/Encoding.agda +++ /dev/null @@ -1,10 +0,0 @@ -module FFI.Data.Text.Encoding where - -open import Agda.Builtin.String using (String) - -open import FFI.Data.ByteString using (ByteString) - -{-# FOREIGN GHC import qualified Data.Text.Encoding #-} - -postulate encodeUtf8 : String → ByteString -{-# COMPILE GHC encodeUtf8 = Data.Text.Encoding.encodeUtf8 #-} diff --git a/prototyping/FFI/Data/Vector.agda b/prototyping/FFI/Data/Vector.agda deleted file mode 100644 index 08761edf..00000000 --- a/prototyping/FFI/Data/Vector.agda +++ /dev/null @@ -1,53 +0,0 @@ -{-# OPTIONS --rewriting #-} - -module FFI.Data.Vector where - -open import Agda.Builtin.Equality using (_≡_) -open import Agda.Builtin.Equality.Rewrite using () -open import Agda.Builtin.Int using (Int; pos; negsuc) -open import Agda.Builtin.Nat using (Nat) -open import Agda.Builtin.Bool using (Bool; false; true) -open import FFI.Data.HaskellInt using (HaskellInt; haskellIntToInt; intToHaskellInt) -open import FFI.Data.Maybe using (Maybe; just; nothing) -open import Properties.Equality using (_≢_) - -{-# FOREIGN GHC import qualified Data.Vector #-} - -postulate Vector : Set → Set -{-# POLARITY Vector ++ #-} -{-# COMPILE GHC Vector = type Data.Vector.Vector #-} - -postulate - empty : ∀ {A} → (Vector A) - null : ∀ {A} → (Vector A) → Bool - unsafeHead : ∀ {A} → (Vector A) → A - unsafeTail : ∀ {A} → (Vector A) → (Vector A) - length : ∀ {A} → (Vector A) → Nat - lookup : ∀ {A} → (Vector A) → Nat → (Maybe A) - snoc : ∀ {A} → (Vector A) → A → (Vector A) -{-# COMPILE GHC empty = \_ -> Data.Vector.empty #-} -{-# COMPILE GHC null = \_ -> Data.Vector.null #-} -{-# COMPILE GHC unsafeHead = \_ -> Data.Vector.unsafeHead #-} -{-# COMPILE GHC unsafeTail = \_ -> Data.Vector.unsafeTail #-} -{-# COMPILE GHC length = \_ -> (fromIntegral . Data.Vector.length) #-} -{-# COMPILE GHC lookup = \_ v -> ((v Data.Vector.!?) . fromIntegral) #-} -{-# COMPILE GHC snoc = \_ -> Data.Vector.snoc #-} - -postulate length-empty : ∀ {A} → (length (empty {A}) ≡ 0) -postulate lookup-empty : ∀ {A} n → (lookup (empty {A}) n ≡ nothing) -postulate lookup-snoc : ∀ {A} (x : A) (v : Vector A) → (lookup (snoc v x) (length v) ≡ just x) -postulate lookup-length : ∀ {A} (v : Vector A) → (lookup v (length v) ≡ nothing) -postulate lookup-snoc-empty : ∀ {A} (x : A) → (lookup (snoc empty x) 0 ≡ just x) -postulate lookup-snoc-not : ∀ {A n} (x : A) (v : Vector A) → (n ≢ length v) → (lookup v n ≡ lookup (snoc v x) n) - -{-# REWRITE length-empty lookup-snoc lookup-length lookup-snoc-empty lookup-empty #-} - -head : ∀ {A} → (Vector A) → (Maybe A) -head vec with null vec -head vec | false = just (unsafeHead vec) -head vec | true = nothing - -tail : ∀ {A} → (Vector A) → Vector A -tail vec with null vec -tail vec | false = unsafeTail vec -tail vec | true = empty diff --git a/prototyping/FFI/IO.agda b/prototyping/FFI/IO.agda deleted file mode 100644 index 825a788f..00000000 --- a/prototyping/FFI/IO.agda +++ /dev/null @@ -1,34 +0,0 @@ -module FFI.IO where - -open import Agda.Builtin.IO using (IO) -open import Agda.Builtin.Unit using (⊤) -open import Agda.Builtin.String using (String) - -open import FFI.Data.HaskellString using (HaskellString; pack ; unpack) - -infixl 1 _>>=_ -infixl 1 _>>_ - -postulate - return : ∀ {a} {A : Set a} → A → IO A - _>>=_ : ∀ {a b} {A : Set a} {B : Set b} → IO A → (A → IO B) → IO B - fmap : ∀ {a b} {A : Set a} {B : Set b} → (A → B) → IO A → IO B - -{-# COMPILE GHC return = \_ _ -> return #-} -{-# COMPILE GHC _>>=_ = \_ _ _ _ -> (>>=) #-} -{-# COMPILE GHC fmap = \_ _ _ _ -> fmap #-} - -postulate getHContents : IO HaskellString -{-# COMPILE GHC getHContents = getContents #-} - -postulate putHStrLn : HaskellString → IO ⊤ -{-# COMPILE GHC putHStrLn = putStrLn #-} - -getContents : IO String -getContents = fmap pack getHContents - -putStrLn : String → IO ⊤ -putStrLn txt = putHStrLn (unpack txt) - -_>>_ : ∀ {a} {A : Set a} → IO ⊤ → IO A → IO A -a >> b = a >>= (λ _ → b ) diff --git a/prototyping/FFI/System/Exit.agda b/prototyping/FFI/System/Exit.agda deleted file mode 100644 index fcf01395..00000000 --- a/prototyping/FFI/System/Exit.agda +++ /dev/null @@ -1,29 +0,0 @@ -module FFI.System.Exit where - -open import Agda.Builtin.Int using (Int) -open import Agda.Builtin.IO using (IO) -open import Agda.Builtin.Unit using (⊤) - -data ExitCode : Set where - ExitSuccess : ExitCode - ExitFailure : Int → ExitCode - -{-# FOREIGN GHC data AgdaExitCode = AgdaExitSuccess | AgdaExitFailure Integer #-} -{-# COMPILE GHC ExitCode = data AgdaExitCode (AgdaExitSuccess | AgdaExitFailure) #-} - -{-# FOREIGN GHC import qualified System.Exit #-} - -{-# FOREIGN GHC -toExitCode :: AgdaExitCode -> System.Exit.ExitCode -toExitCode AgdaExitSuccess = System.Exit.ExitSuccess -toExitCode (AgdaExitFailure n) = System.Exit.ExitFailure (fromIntegral n) - -fromExitCode :: System.Exit.ExitCode -> AgdaExitCode -fromExitCode System.Exit.ExitSuccess = AgdaExitSuccess -fromExitCode (System.Exit.ExitFailure n) = AgdaExitFailure (fromIntegral n) -#-} - -postulate - exitWith : ExitCode → IO ⊤ - -{-# COMPILE GHC exitWith = System.Exit.exitWith . toExitCode #-} diff --git a/prototyping/Interpreter.agda b/prototyping/Interpreter.agda deleted file mode 100644 index d8036740..00000000 --- a/prototyping/Interpreter.agda +++ /dev/null @@ -1,50 +0,0 @@ -{-# OPTIONS --rewriting #-} - -module Interpreter where - -open import Agda.Builtin.IO using (IO) -open import Agda.Builtin.Int using (pos) -open import Agda.Builtin.Unit using (⊤) - -open import FFI.IO using (getContents; putStrLn; _>>=_; _>>_) -open import FFI.Data.Aeson using (Value; eitherDecode) -open import FFI.Data.Either using (Left; Right) -open import FFI.Data.Maybe using (just; nothing) -open import FFI.Data.String using (String; _++_) -open import FFI.Data.Text.Encoding using (encodeUtf8) -open import FFI.System.Exit using (exitWith; ExitFailure) - -open import Luau.StrictMode.ToString using (warningToStringᴮ) -open import Luau.Syntax using (Block; yes; maybe; isAnnotatedᴮ) -open import Luau.Syntax.FromJSON using (blockFromJSON) -open import Luau.Syntax.ToString using (blockToString; valueToString) -open import Luau.Run using (run; return; done; error) -open import Luau.RuntimeError.ToString using (errToStringᴮ) - -open import Properties.StrictMode using (wellTypedProgramsDontGoWrong) - -runBlock′ : ∀ a → Block a → IO ⊤ -runBlock′ a block with run block -runBlock′ a block | return V D = putStrLn ("\nRAN WITH RESULT: " ++ valueToString V) -runBlock′ a block | done D = putStrLn ("\nRAN") -runBlock′ maybe block | error E D = putStrLn ("\nRUNTIME ERROR:\n" ++ errToStringᴮ _ E) -runBlock′ yes block | error E D with wellTypedProgramsDontGoWrong _ block _ D E -runBlock′ yes block | error E D | W = putStrLn ("\nRUNTIME ERROR:\n" ++ errToStringᴮ _ E ++ "\n\nTYPE ERROR:\n" ++ warningToStringᴮ _ W) - -runBlock : Block maybe → IO ⊤ -runBlock B with isAnnotatedᴮ B -runBlock B | nothing = putStrLn ("UNANNOTATED PROGRAM:\n" ++ blockToString B) >> runBlock′ maybe B -runBlock B | just B′ = putStrLn ("ANNOTATED PROGRAM:\n" ++ blockToString B) >> runBlock′ yes B′ - -runJSON : Value → IO ⊤ -runJSON value with blockFromJSON(value) -runJSON value | (Left err) = putStrLn ("LUAU ERROR: " ++ err) >> exitWith (ExitFailure (pos 1)) -runJSON value | (Right block) = runBlock block - -runString : String → IO ⊤ -runString txt with eitherDecode (encodeUtf8 txt) -runString txt | (Left err) = putStrLn ("JSON ERROR: " ++ err) >> exitWith (ExitFailure (pos 1)) -runString txt | (Right value) = runJSON value - -main : IO ⊤ -main = getContents >>= runString diff --git a/prototyping/Luau/Addr.agda b/prototyping/Luau/Addr.agda deleted file mode 100644 index b6f989f5..00000000 --- a/prototyping/Luau/Addr.agda +++ /dev/null @@ -1,18 +0,0 @@ -module Luau.Addr where - -open import Agda.Builtin.Bool using (true; false) -open import Agda.Builtin.Equality using (_≡_) -open import Agda.Builtin.Nat using (Nat; _==_) -open import Agda.Builtin.String using (String) -open import Agda.Builtin.TrustMe using (primTrustMe) -open import Properties.Dec using (Dec; yes; no) -open import Properties.Equality using (_≢_) - -Addr : Set -Addr = Nat - -_≡ᴬ_ : (a b : Addr) → Dec (a ≡ b) -a ≡ᴬ b with a == b -a ≡ᴬ b | false = no p where postulate p : (a ≢ b) -a ≡ᴬ b | true = yes primTrustMe - diff --git a/prototyping/Luau/Addr/ToString.agda b/prototyping/Luau/Addr/ToString.agda deleted file mode 100644 index 2fc38335..00000000 --- a/prototyping/Luau/Addr/ToString.agda +++ /dev/null @@ -1,8 +0,0 @@ -module Luau.Addr.ToString where - -open import Agda.Builtin.String using (String; primStringAppend) -open import Luau.Addr using (Addr) -open import Agda.Builtin.Int using (Int; primShowInteger; pos) - -addrToString : Addr → String -addrToString a = primStringAppend "a" (primShowInteger (pos a)) diff --git a/prototyping/Luau/Heap.agda b/prototyping/Luau/Heap.agda deleted file mode 100644 index 713b0a1a..00000000 --- a/prototyping/Luau/Heap.agda +++ /dev/null @@ -1,49 +0,0 @@ -{-# OPTIONS --rewriting #-} - -module Luau.Heap where - -open import Agda.Builtin.Equality using (_≡_; refl) -open import FFI.Data.Maybe using (Maybe; just; nothing) -open import FFI.Data.Vector using (Vector; length; snoc; empty; lookup-snoc-not) -open import Luau.Addr using (Addr; _≡ᴬ_) -open import Luau.Var using (Var) -open import Luau.Syntax using (Block; Expr; Annotated; FunDec; nil; function_is_end) -open import Properties.Equality using (_≢_; trans) -open import Properties.Remember using (remember; _,_) -open import Properties.Dec using (yes; no) - --- Heap-allocated objects -data Object (a : Annotated) : Set where - - function_is_end : FunDec a → Block a → Object a - -Heap : Annotated → Set -Heap a = Vector (Object a) - -data _≡_⊕_↦_ {a} : Heap a → Heap a → Addr → Object a → Set where - - defn : ∀ {H val} → - - ----------------------------------- - (snoc H val) ≡ H ⊕ (length H) ↦ val - -_[_] : ∀ {a} → Heap a → Addr → Maybe (Object a) -_[_] = FFI.Data.Vector.lookup - -∅ : ∀ {a} → Heap a -∅ = empty - -data AllocResult a (H : Heap a) (V : Object a) : Set where - ok : ∀ b H′ → (H′ ≡ H ⊕ b ↦ V) → AllocResult a H V - -alloc : ∀ {a} H V → AllocResult a H V -alloc H V = ok (length H) (snoc H V) defn - -next : ∀ {a} → Heap a → Addr -next = length - -allocated : ∀ {a} → Heap a → Object a → Heap a -allocated = snoc - -lookup-not-allocated : ∀ {a} {H H′ : Heap a} {b c O} → (H′ ≡ H ⊕ b ↦ O) → (c ≢ b) → (H [ c ] ≡ H′ [ c ]) -lookup-not-allocated {H = H} {O = O} defn p = lookup-snoc-not O H p diff --git a/prototyping/Luau/OpSem.agda b/prototyping/Luau/OpSem.agda deleted file mode 100644 index 1f616c7e..00000000 --- a/prototyping/Luau/OpSem.agda +++ /dev/null @@ -1,143 +0,0 @@ -{-# OPTIONS --rewriting #-} - -module Luau.OpSem where - -open import Agda.Builtin.Equality using (_≡_) -open import Agda.Builtin.Float using (Float; primFloatPlus; primFloatMinus; primFloatTimes; primFloatDiv; primFloatEquality; primFloatLess; primFloatInequality) -open import Agda.Builtin.Bool using (Bool; true; false) -open import Agda.Builtin.String using (primStringEquality; primStringAppend) -open import Utility.Bool using (not; _or_; _and_) -open import Agda.Builtin.Nat using () renaming (_==_ to _==ᴬ_) -open import FFI.Data.Maybe using (Maybe; just; nothing) -open import Luau.Heap using (Heap; _≡_⊕_↦_; _[_]; function_is_end) -open import Luau.Substitution using (_[_/_]ᴮ) -open import Luau.Syntax using (Value; Expr; Stat; Block; nil; addr; val; var; function_is_end; _$_; block_is_end; local_←_; _∙_; done; return; name; fun; arg; binexp; BinaryOperator; +; -; *; /; <; >; ==; ~=; <=; >=; ··; number; bool; string) -open import Luau.RuntimeType using (RuntimeType; valueType) -open import Properties.Product using (_×_; _,_) - -evalEqOp : Value → Value → Bool -evalEqOp Value.nil Value.nil = true -evalEqOp (addr x) (addr y) = (x ==ᴬ y) -evalEqOp (number x) (number y) = primFloatEquality x y -evalEqOp (bool true) (bool y) = y -evalEqOp (bool false) (bool y) = not y -evalEqOp _ _ = false - -evalNeqOp : Value → Value → Bool -evalNeqOp (number x) (number y) = primFloatInequality x y -evalNeqOp x y = not (evalEqOp x y) - -data _⟦_⟧_⟶_ : Value → BinaryOperator → Value → Value → Set where - + : ∀ m n → (number m) ⟦ + ⟧ (number n) ⟶ number (primFloatPlus m n) - - : ∀ m n → (number m) ⟦ - ⟧ (number n) ⟶ number (primFloatMinus m n) - / : ∀ m n → (number m) ⟦ / ⟧ (number n) ⟶ number (primFloatTimes m n) - * : ∀ m n → (number m) ⟦ * ⟧ (number n) ⟶ number (primFloatDiv m n) - < : ∀ m n → (number m) ⟦ < ⟧ (number n) ⟶ bool (primFloatLess m n) - > : ∀ m n → (number m) ⟦ > ⟧ (number n) ⟶ bool (primFloatLess n m) - <= : ∀ m n → (number m) ⟦ <= ⟧ (number n) ⟶ bool ((primFloatLess m n) or (primFloatEquality m n)) - >= : ∀ m n → (number m) ⟦ >= ⟧ (number n) ⟶ bool ((primFloatLess n m) or (primFloatEquality m n)) - == : ∀ v w → v ⟦ == ⟧ w ⟶ bool (evalEqOp v w) - ~= : ∀ v w → v ⟦ ~= ⟧ w ⟶ bool (evalNeqOp v w) - ·· : ∀ x y → (string x) ⟦ ·· ⟧ (string y) ⟶ string (primStringAppend x y) - -data _⊢_⟶ᴮ_⊣_ {a} : Heap a → Block a → Block a → Heap a → Set -data _⊢_⟶ᴱ_⊣_ {a} : Heap a → Expr a → Expr a → Heap a → Set - -data _⊢_⟶ᴱ_⊣_ where - - function : ∀ a {H H′ F B} → - - H′ ≡ H ⊕ a ↦ (function F is B end) → - ------------------------------------------- - H ⊢ (function F is B end) ⟶ᴱ val(addr a) ⊣ H′ - - app₁ : ∀ {H H′ M M′ N} → - - H ⊢ M ⟶ᴱ M′ ⊣ H′ → - ----------------------------- - H ⊢ (M $ N) ⟶ᴱ (M′ $ N) ⊣ H′ - - app₂ : ∀ v {H H′ N N′} → - - H ⊢ N ⟶ᴱ N′ ⊣ H′ → - ----------------------------- - H ⊢ (val v $ N) ⟶ᴱ (val v $ N′) ⊣ H′ - - beta : ∀ O v {H a F B} → - - (O ≡ function F is B end) → - H [ a ] ≡ just(O) → - ----------------------------------------------------------------------------- - H ⊢ (val (addr a) $ val v) ⟶ᴱ (block (fun F) is (B [ v / name(arg F) ]ᴮ) end) ⊣ H - - block : ∀ {H H′ B B′ b} → - - H ⊢ B ⟶ᴮ B′ ⊣ H′ → - ---------------------------------------------------- - H ⊢ (block b is B end) ⟶ᴱ (block b is B′ end) ⊣ H′ - - return : ∀ v {H B b} → - - -------------------------------------------------------- - H ⊢ (block b is return (val v) ∙ B end) ⟶ᴱ val v ⊣ H - - done : ∀ {H b} → - - -------------------------------------------- - H ⊢ (block b is done end) ⟶ᴱ (val nil) ⊣ H - - binOp₀ : ∀ {H op v₁ v₂ w} → - - v₁ ⟦ op ⟧ v₂ ⟶ w → - -------------------------------------------------- - H ⊢ (binexp (val v₁) op (val v₂)) ⟶ᴱ (val w) ⊣ H - - binOp₁ : ∀ {H H′ x x′ op y} → - - H ⊢ x ⟶ᴱ x′ ⊣ H′ → - --------------------------------------------- - H ⊢ (binexp x op y) ⟶ᴱ (binexp x′ op y) ⊣ H′ - - binOp₂ : ∀ {H H′ x op y y′} → - - H ⊢ y ⟶ᴱ y′ ⊣ H′ → - --------------------------------------------- - H ⊢ (binexp x op y) ⟶ᴱ (binexp x op y′) ⊣ H′ - -data _⊢_⟶ᴮ_⊣_ where - - local : ∀ {H H′ x M M′ B} → - - H ⊢ M ⟶ᴱ M′ ⊣ H′ → - ------------------------------------------------- - H ⊢ (local x ← M ∙ B) ⟶ᴮ (local x ← M′ ∙ B) ⊣ H′ - - subst : ∀ v {H x B} → - - ------------------------------------------------------ - H ⊢ (local x ← val v ∙ B) ⟶ᴮ (B [ v / name x ]ᴮ) ⊣ H - - function : ∀ a {H H′ F B C} → - - H′ ≡ H ⊕ a ↦ (function F is C end) → - -------------------------------------------------------------- - H ⊢ (function F is C end ∙ B) ⟶ᴮ (B [ addr a / name(fun F) ]ᴮ) ⊣ H′ - - return : ∀ {H H′ M M′ B} → - - H ⊢ M ⟶ᴱ M′ ⊣ H′ → - -------------------------------------------- - H ⊢ (return M ∙ B) ⟶ᴮ (return M′ ∙ B) ⊣ H′ - -data _⊢_⟶*_⊣_ {a} : Heap a → Block a → Block a → Heap a → Set where - - refl : ∀ {H B} → - - ---------------- - H ⊢ B ⟶* B ⊣ H - - step : ∀ {H H′ H″ B B′ B″} → - H ⊢ B ⟶ᴮ B′ ⊣ H′ → - H′ ⊢ B′ ⟶* B″ ⊣ H″ → - ------------------ - H ⊢ B ⟶* B″ ⊣ H″ diff --git a/prototyping/Luau/ResolveOverloads.agda b/prototyping/Luau/ResolveOverloads.agda deleted file mode 100644 index 67175176..00000000 --- a/prototyping/Luau/ResolveOverloads.agda +++ /dev/null @@ -1,98 +0,0 @@ -{-# OPTIONS --rewriting #-} - -module Luau.ResolveOverloads where - -open import FFI.Data.Either using (Left; Right) -open import Luau.Subtyping using (_<:_; _≮:_; Language; witness; scalar; unknown; never; function-ok) -open import Luau.Type using (Type ; _⇒_; _∩_; _∪_; unknown; never) -open import Luau.TypeSaturation using (saturate) -open import Luau.TypeNormalization using (normalize) -open import Properties.Contradiction using (CONTRADICTION) -open import Properties.DecSubtyping using (dec-subtyping; dec-subtypingⁿ; <:-impl-<:ᵒ) -open import Properties.Functions using (_∘_) -open import Properties.Subtyping using (<:-refl; <:-trans; <:-trans-≮:; ≮:-trans-<:; <:-∩-left; <:-∩-right; <:-∩-glb; <:-impl-¬≮:; <:-unknown; <:-function; function-≮:-never; <:-never; unknown-≮:-function; scalar-≮:-function; ≮:-∪-right; scalar-≮:-never; <:-∪-left; <:-∪-right) -open import Properties.TypeNormalization using (Normal; FunType; normal; _⇒_; _∩_; _∪_; never; unknown; <:-normalize; normalize-<:; fun-≮:-never; unknown-≮:-fun; scalar-≮:-fun) -open import Properties.TypeSaturation using (Overloads; Saturated; _⊆ᵒ_; _<:ᵒ_; normal-saturate; saturated; <:-saturate; saturate-<:; defn; here; left; right) - --- The domain of a normalized type -srcⁿ : Type → Type -srcⁿ (S ⇒ T) = S -srcⁿ (S ∩ T) = srcⁿ S ∪ srcⁿ T -srcⁿ never = unknown -srcⁿ T = never - --- To get the domain of a type, we normalize it first We need to do --- this, since if we try to use it on non-normalized types, we get --- --- src(number ∩ string) = src(number) ∪ src(string) = never ∪ never --- src(never) = unknown --- --- so src doesn't respect type equivalence. -src : Type → Type -src (S ⇒ T) = S -src T = srcⁿ(normalize T) - --- Calculate the result of applying a function type `F` to an argument type `V`. --- We do this by finding an overload of `F` that has the most precise type, --- that is an overload `(Sʳ ⇒ Tʳ)` where `V <: Sʳ` and moreover --- for any other such overload `(S ⇒ T)` we have that `Tʳ <: T`. - --- For example if `F` is `(number -> number) & (nil -> nil) & (number? -> number?)` --- then to resolve `F` with argument type `number`, we pick the `number -> number` --- overload, but if the argument is `number?`, we pick `number? -> number?`./ - --- Not all types have such a most precise overload, but saturated ones do. - -data ResolvedTo F G V : Set where - - yes : ∀ Sʳ Tʳ → - - Overloads F (Sʳ ⇒ Tʳ) → - (V <: Sʳ) → - (∀ {S T} → Overloads G (S ⇒ T) → (V <: S) → (Tʳ <: T)) → - -------------------------------------------- - ResolvedTo F G V - - no : - - (∀ {S T} → Overloads G (S ⇒ T) → (V ≮: S)) → - -------------------------------------------- - ResolvedTo F G V - -Resolved : Type → Type → Set -Resolved F V = ResolvedTo F F V - -target : ∀ {F V} → Resolved F V → Type -target (yes _ T _ _ _) = T -target (no _) = unknown - --- We can resolve any saturated function type -resolveˢ : ∀ {F G V} → FunType G → Saturated F → Normal V → (G ⊆ᵒ F) → ResolvedTo F G V -resolveˢ (Sⁿ ⇒ Tⁿ) (defn sat-∩ sat-∪) Vⁿ G⊆F with dec-subtypingⁿ Vⁿ Sⁿ -resolveˢ (Sⁿ ⇒ Tⁿ) (defn sat-∩ sat-∪) Vⁿ G⊆F | Left V≮:S = no (λ { here → V≮:S }) -resolveˢ (Sⁿ ⇒ Tⁿ) (defn sat-∩ sat-∪) Vⁿ G⊆F | Right V<:S = yes _ _ (G⊆F here) V<:S (λ { here _ → <:-refl }) -resolveˢ (Gᶠ ∩ Hᶠ) (defn sat-∩ sat-∪) Vⁿ G⊆F with resolveˢ Gᶠ (defn sat-∩ sat-∪) Vⁿ (G⊆F ∘ left) | resolveˢ Hᶠ (defn sat-∩ sat-∪) Vⁿ (G⊆F ∘ right) -resolveˢ (Gᶠ ∩ Hᶠ) (defn sat-∩ sat-∪) Vⁿ G⊆F | yes S₁ T₁ o₁ V<:S₁ tgt₁ | yes S₂ T₂ o₂ V<:S₂ tgt₂ with sat-∩ o₁ o₂ -resolveˢ (Gᶠ ∩ Hᶠ) (defn sat-∩ sat-∪) Vⁿ G⊆F | yes S₁ T₁ o₁ V<:S₁ tgt₁ | yes S₂ T₂ o₂ V<:S₂ tgt₂ | defn o p₁ p₂ = - yes _ _ o (<:-trans (<:-∩-glb V<:S₁ V<:S₂) p₁) (λ { (left o) p → <:-trans p₂ (<:-trans <:-∩-left (tgt₁ o p)) ; (right o) p → <:-trans p₂ (<:-trans <:-∩-right (tgt₂ o p)) }) -resolveˢ (Gᶠ ∩ Hᶠ) (defn sat-∩ sat-∪) Vⁿ G⊆F | yes S₁ T₁ o₁ V<:S₁ tgt₁ | no src₂ = - yes _ _ o₁ V<:S₁ (λ { (left o) p → tgt₁ o p ; (right o) p → CONTRADICTION (<:-impl-¬≮: p (src₂ o)) }) -resolveˢ (Gᶠ ∩ Hᶠ) (defn sat-∩ sat-∪) Vⁿ G⊆F | no src₁ | yes S₂ T₂ o₂ V<:S₂ tgt₂ = - yes _ _ o₂ V<:S₂ (λ { (left o) p → CONTRADICTION (<:-impl-¬≮: p (src₁ o)) ; (right o) p → tgt₂ o p }) -resolveˢ (Gᶠ ∩ Hᶠ) (defn sat-∩ sat-∪) Vⁿ G⊆F | no src₁ | no src₂ = - no (λ { (left o) → src₁ o ; (right o) → src₂ o }) - --- Which means we can resolve any normalized type, by saturating it first -resolveᶠ : ∀ {F V} → FunType F → Normal V → Type -resolveᶠ Fᶠ Vⁿ = target (resolveˢ (normal-saturate Fᶠ) (saturated Fᶠ) Vⁿ (λ o → o)) - -resolveⁿ : ∀ {F V} → Normal F → Normal V → Type -resolveⁿ (Sⁿ ⇒ Tⁿ) Vⁿ = resolveᶠ (Sⁿ ⇒ Tⁿ) Vⁿ -resolveⁿ (Fᶠ ∩ Gᶠ) Vⁿ = resolveᶠ (Fᶠ ∩ Gᶠ) Vⁿ -resolveⁿ (Sⁿ ∪ Tˢ) Vⁿ = unknown -resolveⁿ unknown Vⁿ = unknown -resolveⁿ never Vⁿ = never - --- Which means we can resolve any type, by normalizing it first -resolve : Type → Type → Type -resolve F V = resolveⁿ (normal F) (normal V) diff --git a/prototyping/Luau/Run.agda b/prototyping/Luau/Run.agda deleted file mode 100644 index bcd75578..00000000 --- a/prototyping/Luau/Run.agda +++ /dev/null @@ -1,29 +0,0 @@ -{-# OPTIONS --rewriting #-} - -module Luau.Run where - -open import Agda.Builtin.Equality using (_≡_; refl) -open import Luau.Heap using (Heap; ∅) -open import Luau.Syntax using (Block; val; return; _∙_; done) -open import Luau.OpSem using (_⊢_⟶*_⊣_; refl; step) -open import Properties.Step using (stepᴮ; step; return; done; error) -open import Luau.RuntimeError using (RuntimeErrorᴮ) - -data RunResult {a} (H : Heap a) (B : Block a) : Set where - return : ∀ v {B′ H′} → (H ⊢ B ⟶* (return (val v) ∙ B′) ⊣ H′) → RunResult H B - done : ∀ {H′} → (H ⊢ B ⟶* done ⊣ H′) → RunResult H B - error : ∀ {B′ H′} → (RuntimeErrorᴮ H′ B′) → (H ⊢ B ⟶* B′ ⊣ H′) → RunResult H B - -{-# TERMINATING #-} -run′ : ∀ {a} H B → RunResult {a} H B -run′ H B with stepᴮ H B -run′ H B | step H′ B′ D with run′ H′ B′ -run′ H B | step H′ B′ D | return V D′ = return V (step D D′) -run′ H B | step H′ B′ D | done D′ = done (step D D′) -run′ H B | step H′ B′ D | error E D′ = error E (step D D′) -run′ H _ | return V refl = return V refl -run′ H _ | done refl = done refl -run′ H B | error E = error E refl - -run : ∀ {a} B → RunResult {a} ∅ B -run = run′ ∅ diff --git a/prototyping/Luau/RuntimeError.agda b/prototyping/Luau/RuntimeError.agda deleted file mode 100644 index b9b305c9..00000000 --- a/prototyping/Luau/RuntimeError.agda +++ /dev/null @@ -1,41 +0,0 @@ -{-# OPTIONS --rewriting #-} - -module Luau.RuntimeError where - -open import Agda.Builtin.Equality using (_≡_) -open import Luau.Heap using (Heap; _[_]) -open import FFI.Data.Maybe using (just; nothing) -open import FFI.Data.String using (String) -open import Luau.Syntax using (BinaryOperator; Block; Expr; nil; var; val; addr; block_is_end; _$_; local_←_; return; done; _∙_; number; string; binexp; +; -; *; /; <; >; <=; >=; ··) -open import Luau.RuntimeType using (RuntimeType; valueType; function; number; string) -open import Properties.Equality using (_≢_) - -data BinOpError : BinaryOperator → RuntimeType → Set where - + : ∀ {t} → (t ≢ number) → BinOpError + t - - : ∀ {t} → (t ≢ number) → BinOpError - t - * : ∀ {t} → (t ≢ number) → BinOpError * t - / : ∀ {t} → (t ≢ number) → BinOpError / t - < : ∀ {t} → (t ≢ number) → BinOpError < t - > : ∀ {t} → (t ≢ number) → BinOpError > t - <= : ∀ {t} → (t ≢ number) → BinOpError <= t - >= : ∀ {t} → (t ≢ number) → BinOpError >= t - ·· : ∀ {t} → (t ≢ string) → BinOpError ·· t - -data RuntimeErrorᴮ {a} (H : Heap a) : Block a → Set -data RuntimeErrorᴱ {a} (H : Heap a) : Expr a → Set - -data RuntimeErrorᴱ H where - FunctionMismatch : ∀ v w → (valueType v ≢ function) → RuntimeErrorᴱ H (val v $ val w) - BinOpMismatch₁ : ∀ v w {op} → (BinOpError op (valueType v)) → RuntimeErrorᴱ H (binexp (val v) op (val w)) - BinOpMismatch₂ : ∀ v w {op} → (BinOpError op (valueType w)) → RuntimeErrorᴱ H (binexp (val v) op (val w)) - UnboundVariable : ∀ {x} → RuntimeErrorᴱ H (var x) - SEGV : ∀ {a} → (H [ a ] ≡ nothing) → RuntimeErrorᴱ H (val (addr a)) - app₁ : ∀ {M N} → RuntimeErrorᴱ H M → RuntimeErrorᴱ H (M $ N) - app₂ : ∀ {M N} → RuntimeErrorᴱ H N → RuntimeErrorᴱ H (M $ N) - block : ∀ {b B} → RuntimeErrorᴮ H B → RuntimeErrorᴱ H (block b is B end) - bin₁ : ∀ {M N op} → RuntimeErrorᴱ H M → RuntimeErrorᴱ H (binexp M op N) - bin₂ : ∀ {M N op} → RuntimeErrorᴱ H N → RuntimeErrorᴱ H (binexp M op N) - -data RuntimeErrorᴮ H where - local : ∀ {x M B} → RuntimeErrorᴱ H M → RuntimeErrorᴮ H (local x ← M ∙ B) - return : ∀ {M B} → RuntimeErrorᴱ H M → RuntimeErrorᴮ H (return M ∙ B) diff --git a/prototyping/Luau/RuntimeError/ToString.agda b/prototyping/Luau/RuntimeError/ToString.agda deleted file mode 100644 index cd5f18f2..00000000 --- a/prototyping/Luau/RuntimeError/ToString.agda +++ /dev/null @@ -1,31 +0,0 @@ -{-# OPTIONS --rewriting #-} - -module Luau.RuntimeError.ToString where - -open import Agda.Builtin.Float using (primShowFloat) -open import FFI.Data.String using (String; _++_) -open import Luau.RuntimeError using (RuntimeErrorᴮ; RuntimeErrorᴱ; local; return; FunctionMismatch; BinOpMismatch₁; BinOpMismatch₂; UnboundVariable; SEGV; app₁; app₂; block; bin₁; bin₂) -open import Luau.RuntimeType.ToString using (runtimeTypeToString) -open import Luau.Addr.ToString using (addrToString) -open import Luau.Syntax.ToString using (valueToString; exprToString) -open import Luau.Var.ToString using (varToString) -open import Luau.Syntax using (var; val; addr; binexp; block_is_end; local_←_; return; _∙_; name; _$_; ··) - -errToStringᴱ : ∀ {a H} M → RuntimeErrorᴱ {a} H M → String -errToStringᴮ : ∀ {a H} B → RuntimeErrorᴮ {a} H B → String - -errToStringᴱ (var x) (UnboundVariable) = "variable " ++ varToString x ++ " is unbound" -errToStringᴱ (val (addr a)) (SEGV p) = "address " ++ addrToString a ++ " is unallocated" -errToStringᴱ (M $ N) (FunctionMismatch v w p) = "value " ++ (valueToString v) ++ " is not a function" -errToStringᴱ (M $ N) (app₁ E) = errToStringᴱ M E -errToStringᴱ (M $ N) (app₂ E) = errToStringᴱ N E -errToStringᴱ (binexp M ·· N) (BinOpMismatch₁ v w p) = "value " ++ (valueToString v) ++ " is not a string" -errToStringᴱ (binexp M ·· N) (BinOpMismatch₂ v w p) = "value " ++ (valueToString w) ++ " is not a string" -errToStringᴱ (binexp M op N) (BinOpMismatch₁ v w p) = "value " ++ (valueToString v) ++ " is not a number" -errToStringᴱ (binexp M op N) (BinOpMismatch₂ v w p) = "value " ++ (valueToString w) ++ " is not a number" -errToStringᴱ (binexp M op N) (bin₁ E) = errToStringᴱ M E -errToStringᴱ (binexp M op N) (bin₂ E) = errToStringᴱ N E -errToStringᴱ (block b is B end) (block E) = errToStringᴮ B E ++ "\n in call of function " ++ varToString (name b) - -errToStringᴮ (local x ← M ∙ B) (local E) = errToStringᴱ M E ++ "\n in definition of " ++ varToString (name x) -errToStringᴮ (return M ∙ B) (return E) = errToStringᴱ M E ++ "\n in return statement" diff --git a/prototyping/Luau/RuntimeType.agda b/prototyping/Luau/RuntimeType.agda deleted file mode 100644 index d585b51b..00000000 --- a/prototyping/Luau/RuntimeType.agda +++ /dev/null @@ -1,17 +0,0 @@ -module Luau.RuntimeType where - -open import Luau.Syntax using (Value; nil; addr; number; bool; string) - -data RuntimeType : Set where - function : RuntimeType - number : RuntimeType - nil : RuntimeType - boolean : RuntimeType - string : RuntimeType - -valueType : Value → RuntimeType -valueType nil = nil -valueType (addr a) = function -valueType (number n) = number -valueType (bool b) = boolean -valueType (string x) = string diff --git a/prototyping/Luau/RuntimeType/ToString.agda b/prototyping/Luau/RuntimeType/ToString.agda deleted file mode 100644 index f3dd1255..00000000 --- a/prototyping/Luau/RuntimeType/ToString.agda +++ /dev/null @@ -1,11 +0,0 @@ -module Luau.RuntimeType.ToString where - -open import FFI.Data.String using (String) -open import Luau.RuntimeType using (RuntimeType; function; number; nil; boolean; string) - -runtimeTypeToString : RuntimeType → String -runtimeTypeToString function = "function" -runtimeTypeToString number = "number" -runtimeTypeToString nil = "nil" -runtimeTypeToString boolean = "boolean" -runtimeTypeToString string = "string" diff --git a/prototyping/Luau/StrictMode.agda b/prototyping/Luau/StrictMode.agda deleted file mode 100644 index 0628951b..00000000 --- a/prototyping/Luau/StrictMode.agda +++ /dev/null @@ -1,194 +0,0 @@ -{-# OPTIONS --rewriting #-} - -module Luau.StrictMode where - -open import Agda.Builtin.Equality using (_≡_) -open import FFI.Data.Maybe using (just; nothing) -open import Luau.Syntax using (Expr; Stat; Block; BinaryOperator; yes; nil; addr; var; binexp; var_∈_; _⟨_⟩∈_; function_is_end; _$_; block_is_end; local_←_; _∙_; done; return; name; +; -; *; /; <; >; <=; >=; ··) -open import Luau.Type using (Type; nil; number; string; boolean; _⇒_; _∪_; _∩_) -open import Luau.ResolveOverloads using (src; resolve) -open import Luau.Subtyping using (_≮:_) -open import Luau.Heap using (Heap; function_is_end) renaming (_[_] to _[_]ᴴ) -open import Luau.VarCtxt using (VarCtxt; ∅; _⋒_; _↦_; _⊕_↦_; _⊝_) renaming (_[_] to _[_]ⱽ) -open import Luau.TypeCheck using (_⊢ᴮ_∈_; _⊢ᴱ_∈_; ⊢ᴴ_; ⊢ᴼ_; _⊢ᴴᴱ_▷_∈_; _⊢ᴴᴮ_▷_∈_; var; addr; app; binexp; block; return; local; function; srcBinOp) -open import Properties.Contradiction using (¬) -open import Properties.TypeCheck using (typeCheckᴮ) -open import Properties.Product using (_,_) - -data Warningᴱ (H : Heap yes) {Γ} : ∀ {M T} → (Γ ⊢ᴱ M ∈ T) → Set -data Warningᴮ (H : Heap yes) {Γ} : ∀ {B T} → (Γ ⊢ᴮ B ∈ T) → Set - -data Warningᴱ H {Γ} where - - UnallocatedAddress : ∀ {a T} → - - (H [ a ]ᴴ ≡ nothing) → - --------------------- - Warningᴱ H (addr {a} T) - - UnboundVariable : ∀ {x T p} → - - (Γ [ x ]ⱽ ≡ nothing) → - ------------------------ - Warningᴱ H (var {x} {T} p) - - FunctionCallMismatch : ∀ {M N T U} {D₁ : Γ ⊢ᴱ M ∈ T} {D₂ : Γ ⊢ᴱ N ∈ U} → - - (U ≮: src T) → - ----------------- - Warningᴱ H (app D₁ D₂) - - app₁ : ∀ {M N T U} {D₁ : Γ ⊢ᴱ M ∈ T} {D₂ : Γ ⊢ᴱ N ∈ U} → - - Warningᴱ H D₁ → - ----------------- - Warningᴱ H (app D₁ D₂) - - app₂ : ∀ {M N T U} {D₁ : Γ ⊢ᴱ M ∈ T} {D₂ : Γ ⊢ᴱ N ∈ U} → - - Warningᴱ H D₂ → - ----------------- - Warningᴱ H (app D₁ D₂) - - BinOpMismatch₁ : ∀ {op M N T U} {D₁ : Γ ⊢ᴱ M ∈ T} {D₂ : Γ ⊢ᴱ N ∈ U} → - - (T ≮: srcBinOp op) → - ------------------------------ - Warningᴱ H (binexp {op} D₁ D₂) - - BinOpMismatch₂ : ∀ {op M N T U} {D₁ : Γ ⊢ᴱ M ∈ T} {D₂ : Γ ⊢ᴱ N ∈ U} → - - (U ≮: srcBinOp op) → - ------------------------------ - Warningᴱ H (binexp {op} D₁ D₂) - - bin₁ : ∀ {op M N T U} {D₁ : Γ ⊢ᴱ M ∈ T} {D₂ : Γ ⊢ᴱ N ∈ U} → - - Warningᴱ H D₁ → - ------------------------------ - Warningᴱ H (binexp {op} D₁ D₂) - - bin₂ : ∀ {op M N T U} {D₁ : Γ ⊢ᴱ M ∈ T} {D₂ : Γ ⊢ᴱ N ∈ U} → - - Warningᴱ H D₂ → - ------------------------------ - Warningᴱ H (binexp {op} D₁ D₂) - - FunctionDefnMismatch : ∀ {f x B T U V} {D : (Γ ⊕ x ↦ T) ⊢ᴮ B ∈ V} → - - (V ≮: U) → - ------------------------- - Warningᴱ H (function {f} {U = U} D) - - function₁ : ∀ {f x B T U V} {D : (Γ ⊕ x ↦ T) ⊢ᴮ B ∈ V} → - - Warningᴮ H D → - ------------------------- - Warningᴱ H (function {f} {U = U} D) - - BlockMismatch : ∀ {b B T U} {D : Γ ⊢ᴮ B ∈ U} → - - (U ≮: T) → - ------------------------------ - Warningᴱ H (block {b} {T = T} D) - - block₁ : ∀ {b B T U} {D : Γ ⊢ᴮ B ∈ U} → - - Warningᴮ H D → - ------------------------------ - Warningᴱ H (block {b} {T = T} D) - -data Warningᴮ H {Γ} where - - return : ∀ {M B T U} {D₁ : Γ ⊢ᴱ M ∈ T} {D₂ : Γ ⊢ᴮ B ∈ U} → - - Warningᴱ H D₁ → - ------------------ - Warningᴮ H (return D₁ D₂) - - LocalVarMismatch : ∀ {x M B T U V} {D₁ : Γ ⊢ᴱ M ∈ U} {D₂ : (Γ ⊕ x ↦ T) ⊢ᴮ B ∈ V} → - - (U ≮: T) → - -------------------- - Warningᴮ H (local D₁ D₂) - - local₁ : ∀ {x M B T U V} {D₁ : Γ ⊢ᴱ M ∈ U} {D₂ : (Γ ⊕ x ↦ T) ⊢ᴮ B ∈ V} → - - Warningᴱ H D₁ → - -------------------- - Warningᴮ H (local D₁ D₂) - - local₂ : ∀ {x M B T U V} {D₁ : Γ ⊢ᴱ M ∈ U} {D₂ : (Γ ⊕ x ↦ T) ⊢ᴮ B ∈ V} → - - Warningᴮ H D₂ → - -------------------- - Warningᴮ H (local D₁ D₂) - - FunctionDefnMismatch : ∀ {f x B C T U V W} {D₁ : (Γ ⊕ x ↦ T) ⊢ᴮ C ∈ V} {D₂ : (Γ ⊕ f ↦ (T ⇒ U)) ⊢ᴮ B ∈ W} → - - (V ≮: U) → - ------------------------------------- - Warningᴮ H (function D₁ D₂) - - function₁ : ∀ {f x B C T U V W} {D₁ : (Γ ⊕ x ↦ T) ⊢ᴮ C ∈ V} {D₂ : (Γ ⊕ f ↦ (T ⇒ U)) ⊢ᴮ B ∈ W} → - - Warningᴮ H D₁ → - -------------------- - Warningᴮ H (function D₁ D₂) - - function₂ : ∀ {f x B C T U V W} {D₁ : (Γ ⊕ x ↦ T) ⊢ᴮ C ∈ V} {D₂ : (Γ ⊕ f ↦ (T ⇒ U)) ⊢ᴮ B ∈ W} → - - Warningᴮ H D₂ → - -------------------- - Warningᴮ H (function D₁ D₂) - -data Warningᴼ (H : Heap yes) : ∀ {V} → (⊢ᴼ V) → Set where - - FunctionDefnMismatch : ∀ {f x B T U V} {D : (x ↦ T) ⊢ᴮ B ∈ V} → - - (V ≮: U) → - --------------------------------- - Warningᴼ H (function {f} {U = U} D) - - function₁ : ∀ {f x B T U V} {D : (x ↦ T) ⊢ᴮ B ∈ V} → - - Warningᴮ H D → - --------------------------------- - Warningᴼ H (function {f} {U = U} D) - -data Warningᴴ H (D : ⊢ᴴ H) : Set where - - addr : ∀ a {O} → - - (p : H [ a ]ᴴ ≡ O) → - Warningᴼ H (D a p) → - --------------- - Warningᴴ H D - -data Warningᴴᴱ H {Γ M T} : (Γ ⊢ᴴᴱ H ▷ M ∈ T) → Set where - - heap : ∀ {D₁ : ⊢ᴴ H} {D₂ : Γ ⊢ᴱ M ∈ T} → - - Warningᴴ H D₁ → - ----------------- - Warningᴴᴱ H (D₁ , D₂) - - expr : ∀ {D₁ : ⊢ᴴ H} {D₂ : Γ ⊢ᴱ M ∈ T} → - - Warningᴱ H D₂ → - --------------------- - Warningᴴᴱ H (D₁ , D₂) - -data Warningᴴᴮ H {Γ B T} : (Γ ⊢ᴴᴮ H ▷ B ∈ T) → Set where - - heap : ∀ {D₁ : ⊢ᴴ H} {D₂ : Γ ⊢ᴮ B ∈ T} → - - Warningᴴ H D₁ → - ----------------- - Warningᴴᴮ H (D₁ , D₂) - - block : ∀ {D₁ : ⊢ᴴ H} {D₂ : Γ ⊢ᴮ B ∈ T} → - - Warningᴮ H D₂ → - --------------------- - Warningᴴᴮ H (D₁ , D₂) diff --git a/prototyping/Luau/StrictMode/ToString.agda b/prototyping/Luau/StrictMode/ToString.agda deleted file mode 100644 index 7c5f0253..00000000 --- a/prototyping/Luau/StrictMode/ToString.agda +++ /dev/null @@ -1,61 +0,0 @@ -{-# OPTIONS --rewriting #-} - -module Luau.StrictMode.ToString where - -open import Agda.Builtin.Nat using (Nat; suc) -open import FFI.Data.String using (String; _++_) -open import Luau.Subtyping using (_≮:_; Tree; witness; scalar; function; function-ok; function-err; function-tgt) -open import Luau.StrictMode using (Warningᴱ; Warningᴮ; UnallocatedAddress; UnboundVariable; FunctionCallMismatch; FunctionDefnMismatch; BlockMismatch; app₁; app₂; BinOpMismatch₁; BinOpMismatch₂; bin₁; bin₂; block₁; return; LocalVarMismatch; local₁; local₂; function₁; function₂; heap; expr; block; addr) -open import Luau.Syntax using (Expr; val; yes; var; var_∈_; _⟨_⟩∈_; _$_; addr; number; binexp; nil; function_is_end; block_is_end; done; return; local_←_; _∙_; fun; arg; name) -open import Luau.Type using (number; boolean; string; nil) -open import Luau.TypeCheck using (_⊢ᴮ_∈_; _⊢ᴱ_∈_) -open import Luau.Addr.ToString using (addrToString) -open import Luau.Var.ToString using (varToString) -open import Luau.Type.ToString using (typeToString) -open import Luau.Syntax.ToString using (binOpToString) - -tmp : Nat → String -tmp 0 = "w" -tmp 1 = "x" -tmp 2 = "y" -tmp 3 = "z" -tmp (suc (suc (suc n))) = tmp n ++ "'" - -treeToString : Tree → Nat → String → String -treeToString (scalar number) n v = v ++ " is a number" -treeToString (scalar boolean) n v = v ++ " is a boolean" -treeToString (scalar string) n v = v ++ " is a string" -treeToString (scalar nil) n v = v ++ " is nil" -treeToString function n v = v ++ " is a function" -treeToString (function-ok s t) n v = treeToString t (suc n) (v ++ "(" ++ w ++ ")") ++ " when\n " ++ treeToString s (suc n) w where w = tmp n -treeToString (function-err t) n v = v ++ "(" ++ w ++ ") can error when\n " ++ treeToString t (suc n) w where w = tmp n -treeToString (function-tgt t) n v = treeToString t n (v ++ "()") - -subtypeWarningToString : ∀ {T U} → (T ≮: U) → String -subtypeWarningToString (witness t p q) = "\n because provided type contains v, where " ++ treeToString t 0 "v" - -warningToStringᴱ : ∀ {H Γ T} M → {D : Γ ⊢ᴱ M ∈ T} → Warningᴱ H D → String -warningToStringᴮ : ∀ {H Γ T} B → {D : Γ ⊢ᴮ B ∈ T} → Warningᴮ H D → String - -warningToStringᴱ (var x) (UnboundVariable p) = "Unbound variable " ++ varToString x -warningToStringᴱ (val (addr a)) (UnallocatedAddress p) = "Unallocated address " ++ addrToString a -warningToStringᴱ (M $ N) (FunctionCallMismatch {T = T} {U = U} p) = "Function has type " ++ typeToString T ++ " but argument has type " ++ typeToString U ++ subtypeWarningToString p -warningToStringᴱ (M $ N) (app₁ W) = warningToStringᴱ M W -warningToStringᴱ (M $ N) (app₂ W) = warningToStringᴱ N W -warningToStringᴱ (function f ⟨ var x ∈ T ⟩∈ U is B end) (FunctionDefnMismatch {V = V} p) = "Function expresion " ++ varToString f ++ " has return type " ++ typeToString U ++ " but body returns " ++ typeToString V ++ subtypeWarningToString p -warningToStringᴱ (function f ⟨ var x ∈ T ⟩∈ U is B end) (function₁ W) = warningToStringᴮ B W ++ "\n in function expression " ++ varToString f -warningToStringᴱ block var b ∈ T is B end (BlockMismatch {U = U} p) = "Block " ++ varToString b ++ " has type " ++ typeToString T ++ " but body returns " ++ typeToString U ++ subtypeWarningToString p -warningToStringᴱ block var b ∈ T is B end (block₁ W) = warningToStringᴮ B W ++ "\n in block " ++ varToString b -warningToStringᴱ (binexp M op N) (BinOpMismatch₁ {T = T} p) = "Binary operator " ++ binOpToString op ++ " lhs has type " ++ typeToString T ++ subtypeWarningToString p -warningToStringᴱ (binexp M op N) (BinOpMismatch₂ {U = U} p) = "Binary operator " ++ binOpToString op ++ " rhs has type " ++ typeToString U ++ subtypeWarningToString p -warningToStringᴱ (binexp M op N) (bin₁ W) = warningToStringᴱ M W -warningToStringᴱ (binexp M op N) (bin₂ W) = warningToStringᴱ N W - -warningToStringᴮ (function f ⟨ var x ∈ T ⟩∈ U is C end ∙ B) (FunctionDefnMismatch {V = V} p) = "Function declaration " ++ varToString f ++ " has return type " ++ typeToString U ++ " but body returns " ++ typeToString V ++ subtypeWarningToString p -warningToStringᴮ (function f ⟨ var x ∈ T ⟩∈ U is C end ∙ B) (function₁ W) = warningToStringᴮ C W ++ "\n in function declaration " ++ varToString f -warningToStringᴮ (function f ⟨ var x ∈ T ⟩∈ U is C end ∙ B) (function₂ W) = warningToStringᴮ B W -warningToStringᴮ (local var x ∈ T ← M ∙ B) (LocalVarMismatch {U = U} p) = "Local variable " ++ varToString x ++ " has type " ++ typeToString T ++ " but expression has type " ++ typeToString U ++ subtypeWarningToString p -warningToStringᴮ (local var x ∈ T ← M ∙ B) (local₁ W) = warningToStringᴱ M W ++ "\n in local variable declaration " ++ varToString x -warningToStringᴮ (local var x ∈ T ← M ∙ B) (local₂ W) = warningToStringᴮ B W -warningToStringᴮ (return M ∙ B) (return W) = warningToStringᴱ M W ++ "\n in return statement" - diff --git a/prototyping/Luau/Substitution.agda b/prototyping/Luau/Substitution.agda deleted file mode 100644 index 883cc638..00000000 --- a/prototyping/Luau/Substitution.agda +++ /dev/null @@ -1,28 +0,0 @@ -module Luau.Substitution where - -open import Luau.Syntax using (Value; Expr; Stat; Block; val; nil; bool; addr; var; function_is_end; _$_; block_is_end; local_←_; _∙_; done; return; _⟨_⟩ ; name; fun; arg; number; binexp) -open import Luau.Var using (Var; _≡ⱽ_) -open import Properties.Dec using (Dec; yes; no) - -_[_/_]ᴱ : ∀ {a} → Expr a → Value → Var → Expr a -_[_/_]ᴮ : ∀ {a} → Block a → Value → Var → Block a -var_[_/_]ᴱwhenever_ : ∀ {a P} → Var → Value → Var → (Dec P) → Expr a -_[_/_]ᴮunless_ : ∀ {a P} → Block a → Value → Var → (Dec P) → Block a - -val w [ v / x ]ᴱ = val w -var y [ v / x ]ᴱ = var y [ v / x ]ᴱwhenever (x ≡ⱽ y) -(M $ N) [ v / x ]ᴱ = (M [ v / x ]ᴱ) $ (N [ v / x ]ᴱ) -function F is C end [ v / x ]ᴱ = function F is C [ v / x ]ᴮunless (x ≡ⱽ name(arg F)) end -block b is C end [ v / x ]ᴱ = block b is C [ v / x ]ᴮ end -(binexp e₁ op e₂) [ v / x ]ᴱ = binexp (e₁ [ v / x ]ᴱ) op (e₂ [ v / x ]ᴱ) - -(function F is C end ∙ B) [ v / x ]ᴮ = function F is (C [ v / x ]ᴮunless (x ≡ⱽ name(arg F))) end ∙ (B [ v / x ]ᴮunless (x ≡ⱽ name(fun F))) -(local y ← M ∙ B) [ v / x ]ᴮ = local y ← (M [ v / x ]ᴱ) ∙ (B [ v / x ]ᴮunless (x ≡ⱽ name y)) -(return M ∙ B) [ v / x ]ᴮ = return (M [ v / x ]ᴱ) ∙ (B [ v / x ]ᴮ) -done [ v / x ]ᴮ = done - -var y [ v / x ]ᴱwhenever yes p = val v -var y [ v / x ]ᴱwhenever no p = var y - -B [ v / x ]ᴮunless yes p = B -B [ v / x ]ᴮunless no p = B [ v / x ]ᴮ diff --git a/prototyping/Luau/Subtyping.agda b/prototyping/Luau/Subtyping.agda deleted file mode 100644 index dc2abed0..00000000 --- a/prototyping/Luau/Subtyping.agda +++ /dev/null @@ -1,67 +0,0 @@ -{-# OPTIONS --rewriting #-} - -open import Luau.Type using (Type; Scalar; nil; number; string; boolean; never; unknown; _⇒_; _∪_; _∩_) -open import Properties.Equality using (_≢_) - -module Luau.Subtyping where - --- An implementation of semantic subtyping - --- We think of types as languages of trees - -data Tree : Set where - - scalar : ∀ {T} → Scalar T → Tree - function : Tree - function-ok : Tree → Tree → Tree - function-err : Tree → Tree - function-tgt : Tree → Tree - -data Language : Type → Tree → Set -data ¬Language : Type → Tree → Set - -data Language where - - scalar : ∀ {T} → (s : Scalar T) → Language T (scalar s) - function : ∀ {T U} → Language (T ⇒ U) function - function-ok₁ : ∀ {T U t u} → (¬Language T t) → Language (T ⇒ U) (function-ok t u) - function-ok₂ : ∀ {T U t u} → (Language U u) → Language (T ⇒ U) (function-ok t u) - function-err : ∀ {T U t} → (¬Language T t) → Language (T ⇒ U) (function-err t) - function-tgt : ∀ {T U t} → (Language U t) → Language (T ⇒ U) (function-tgt t) - left : ∀ {T U t} → Language T t → Language (T ∪ U) t - right : ∀ {T U u} → Language U u → Language (T ∪ U) u - _,_ : ∀ {T U t} → Language T t → Language U t → Language (T ∩ U) t - unknown : ∀ {t} → Language unknown t - -data ¬Language where - - scalar-scalar : ∀ {S T} → (s : Scalar S) → (Scalar T) → (S ≢ T) → ¬Language T (scalar s) - scalar-function : ∀ {S} → (Scalar S) → ¬Language S function - scalar-function-ok : ∀ {S t u} → (Scalar S) → ¬Language S (function-ok t u) - scalar-function-err : ∀ {S t} → (Scalar S) → ¬Language S (function-err t) - scalar-function-tgt : ∀ {S t} → (Scalar S) → ¬Language S (function-tgt t) - function-scalar : ∀ {S T U} (s : Scalar S) → ¬Language (T ⇒ U) (scalar s) - function-ok : ∀ {T U t u} → (Language T t) → (¬Language U u) → ¬Language (T ⇒ U) (function-ok t u) - function-err : ∀ {T U t} → (Language T t) → ¬Language (T ⇒ U) (function-err t) - function-tgt : ∀ {T U t} → (¬Language U t) → ¬Language (T ⇒ U) (function-tgt t) - _,_ : ∀ {T U t} → ¬Language T t → ¬Language U t → ¬Language (T ∪ U) t - left : ∀ {T U t} → ¬Language T t → ¬Language (T ∩ U) t - right : ∀ {T U u} → ¬Language U u → ¬Language (T ∩ U) u - never : ∀ {t} → ¬Language never t - --- Subtyping as language inclusion - -_<:_ : Type → Type → Set -(T <: U) = ∀ t → (Language T t) → (Language U t) - --- For warnings, we are interested in failures of subtyping, --- which is whrn there is a tree in T's language that isn't in U's. - -data _≮:_ (T U : Type) : Set where - - witness : ∀ t → - - Language T t → - ¬Language U t → - ----------------- - T ≮: U diff --git a/prototyping/Luau/Syntax.agda b/prototyping/Luau/Syntax.agda deleted file mode 100644 index d9175067..00000000 --- a/prototyping/Luau/Syntax.agda +++ /dev/null @@ -1,110 +0,0 @@ -module Luau.Syntax where - -open import Agda.Builtin.Equality using (_≡_) -open import Agda.Builtin.Bool using (Bool; true; false) -open import Agda.Builtin.Float using (Float) -open import Agda.Builtin.String using (String) -open import Luau.Var using (Var) -open import Luau.Addr using (Addr) -open import Luau.Type using (Type) -open import FFI.Data.Maybe using (Maybe; just; nothing) - -infixr 5 _∙_ - -data Annotated : Set where - maybe : Annotated - yes : Annotated - -data VarDec : Annotated → Set where - var : Var → VarDec maybe - var_∈_ : ∀ {a} → Var → Type → VarDec a - -name : ∀ {a} → VarDec a → Var -name (var x) = x -name (var x ∈ T) = x - -data FunDec : Annotated → Set where - _⟨_⟩∈_ : ∀ {a} → Var → VarDec a → Type → FunDec a - _⟨_⟩ : Var → VarDec maybe → FunDec maybe - -fun : ∀ {a} → FunDec a → VarDec a -fun (f ⟨ x ⟩∈ T) = (var f ∈ T) -fun (f ⟨ x ⟩) = (var f) - -arg : ∀ {a} → FunDec a → VarDec a -arg (f ⟨ x ⟩∈ T) = x -arg (f ⟨ x ⟩) = x - -data BinaryOperator : Set where - + : BinaryOperator - - : BinaryOperator - * : BinaryOperator - / : BinaryOperator - < : BinaryOperator - > : BinaryOperator - == : BinaryOperator - ~= : BinaryOperator - <= : BinaryOperator - >= : BinaryOperator - ·· : BinaryOperator - -data Value : Set where - nil : Value - addr : Addr → Value - number : Float → Value - bool : Bool → Value - string : String → Value - -data Block (a : Annotated) : Set -data Stat (a : Annotated) : Set -data Expr (a : Annotated) : Set - -data Block a where - _∙_ : Stat a → Block a → Block a - done : Block a - -data Stat a where - function_is_end : FunDec a → Block a → Stat a - local_←_ : VarDec a → Expr a → Stat a - return : Expr a → Stat a - -data Expr a where - var : Var → Expr a - val : Value → Expr a - _$_ : Expr a → Expr a → Expr a - function_is_end : FunDec a → Block a → Expr a - block_is_end : VarDec a → Block a → Expr a - binexp : Expr a → BinaryOperator → Expr a → Expr a - -isAnnotatedᴱ : ∀ {a} → Expr a → Maybe (Expr yes) -isAnnotatedᴮ : ∀ {a} → Block a → Maybe (Block yes) - -isAnnotatedᴱ (var x) = just (var x) -isAnnotatedᴱ (val v) = just (val v) -isAnnotatedᴱ (M $ N) with isAnnotatedᴱ M | isAnnotatedᴱ N -isAnnotatedᴱ (M $ N) | just M′ | just N′ = just (M′ $ N′) -isAnnotatedᴱ (M $ N) | _ | _ = nothing -isAnnotatedᴱ (function f ⟨ var x ∈ T ⟩∈ U is B end) with isAnnotatedᴮ B -isAnnotatedᴱ (function f ⟨ var x ∈ T ⟩∈ U is B end) | just B′ = just (function f ⟨ var x ∈ T ⟩∈ U is B′ end) -isAnnotatedᴱ (function f ⟨ var x ∈ T ⟩∈ U is B end) | _ = nothing -isAnnotatedᴱ (function _ is B end) = nothing -isAnnotatedᴱ (block var b ∈ T is B end) with isAnnotatedᴮ B -isAnnotatedᴱ (block var b ∈ T is B end) | just B′ = just (block var b ∈ T is B′ end) -isAnnotatedᴱ (block var b ∈ T is B end) | _ = nothing -isAnnotatedᴱ (block _ is B end) = nothing -isAnnotatedᴱ (binexp M op N) with isAnnotatedᴱ M | isAnnotatedᴱ N -isAnnotatedᴱ (binexp M op N) | just M′ | just N′ = just (binexp M′ op N′) -isAnnotatedᴱ (binexp M op N) | _ | _ = nothing - -isAnnotatedᴮ (function f ⟨ var x ∈ T ⟩∈ U is C end ∙ B) with isAnnotatedᴮ B | isAnnotatedᴮ C -isAnnotatedᴮ (function f ⟨ var x ∈ T ⟩∈ U is C end ∙ B) | just B′ | just C′ = just (function f ⟨ var x ∈ T ⟩∈ U is C′ end ∙ B′) -isAnnotatedᴮ (function f ⟨ var x ∈ T ⟩∈ U is C end ∙ B) | _ | _ = nothing -isAnnotatedᴮ (function _ is C end ∙ B) = nothing -isAnnotatedᴮ (local var x ∈ T ← M ∙ B) with isAnnotatedᴱ M | isAnnotatedᴮ B -isAnnotatedᴮ (local var x ∈ T ← M ∙ B) | just M′ | just B′ = just (local var x ∈ T ← M′ ∙ B′) -isAnnotatedᴮ (local var x ∈ T ← M ∙ B) | _ | _ = nothing -isAnnotatedᴮ (local _ ← M ∙ B) = nothing -isAnnotatedᴮ (return M ∙ B) with isAnnotatedᴱ M | isAnnotatedᴮ B -isAnnotatedᴮ (return M ∙ B) | just M′ | just B′ = just (return M′ ∙ B′) -isAnnotatedᴮ (return M ∙ B) | _ | _ = nothing -isAnnotatedᴮ done = just done diff --git a/prototyping/Luau/Syntax/FromJSON.agda b/prototyping/Luau/Syntax/FromJSON.agda deleted file mode 100644 index 2263615d..00000000 --- a/prototyping/Luau/Syntax/FromJSON.agda +++ /dev/null @@ -1,201 +0,0 @@ -{-# OPTIONS --rewriting #-} - -module Luau.Syntax.FromJSON where - -open import Luau.Syntax using (Block; Stat ; Expr; _$_; val; nil; bool; number; var; var_∈_; function_is_end; _⟨_⟩; _⟨_⟩∈_; local_←_; return; done; _∙_; maybe; VarDec; binexp; BinaryOperator; +; -; *; /; ==; ~=; <; >; <=; >=; ··; string) -open import Luau.Type.FromJSON using (typeFromJSON) - -open import Agda.Builtin.List using (List; _∷_; []) -open import Agda.Builtin.Bool using (true; false) - -open import FFI.Data.Aeson using (Value; Array; Object; object; array; string; fromString; lookup) -open import FFI.Data.Either using (Either; Left; Right) -open import FFI.Data.Maybe using (Maybe; nothing; just) -open import FFI.Data.Scientific using (toFloat) -open import FFI.Data.String using (String; _++_) -open import FFI.Data.Vector using (head; tail; null; empty) - -args = fromString "args" -body = fromString "body" -func = fromString "func" -lokal = fromString "local" -list = fromString "list" -name = fromString "name" -type = fromString "type" -value = fromString "value" -values = fromString "values" -vars = fromString "vars" -op = fromString "op" -left = fromString "left" -right = fromString "right" -returnAnnotation = fromString "returnAnnotation" -types = fromString "types" - -data Lookup : Set where - _,_ : String → Value → Lookup - nil : Lookup - -lookupIn : List String → Object → Lookup -lookupIn [] obj = nil -lookupIn (key ∷ keys) obj with lookup (fromString key) obj -lookupIn (key ∷ keys) obj | nothing = lookupIn keys obj -lookupIn (key ∷ keys) obj | just value = (key , value) - -binOpFromJSON : Value → Either String BinaryOperator -binOpFromString : String → Either String BinaryOperator -varDecFromJSON : Value → Either String (VarDec maybe) -varDecFromObject : Object → Either String (VarDec maybe) -exprFromJSON : Value → Either String (Expr maybe) -exprFromObject : Object → Either String (Expr maybe) -statFromJSON : Value → Either String (Stat maybe) -statFromObject : Object → Either String (Stat maybe) -blockFromJSON : Value → Either String (Block maybe) -blockFromArray : Array → Either String (Block maybe) - -binOpFromJSON (string s) = binOpFromString s -binOpFromJSON _ = Left "Binary operator not a string" - -binOpFromString "Add" = Right + -binOpFromString "Sub" = Right - -binOpFromString "Mul" = Right * -binOpFromString "Div" = Right / -binOpFromString "CompareEq" = Right == -binOpFromString "CompareNe" = Right ~= -binOpFromString "CompareLt" = Right < -binOpFromString "CompareLe" = Right <= -binOpFromString "CompareGt" = Right > -binOpFromString "CompareGe" = Right >= -binOpFromString "Concat" = Right ·· -binOpFromString s = Left ("'" ++ s ++ "' is not a valid operator") - -varDecFromJSON (object arg) = varDecFromObject arg -varDecFromJSON _ = Left "VarDec not an object" - -varDecFromObject obj with lookup name obj | lookup type obj -varDecFromObject obj | just (string name) | nothing = Right (var name) -varDecFromObject obj | just (string name) | just Value.null = Right (var name) -varDecFromObject obj | just (string name) | just tyValue with typeFromJSON tyValue -varDecFromObject obj | just (string name) | just tyValue | Right ty = Right (var name ∈ ty) -varDecFromObject obj | just (string name) | just tyValue | Left err = Left err -varDecFromObject obj | just _ | _ = Left "AstLocal name is not a string" -varDecFromObject obj | nothing | _ = Left "AstLocal missing name" - -exprFromJSON (object obj) = exprFromObject obj -exprFromJSON _ = Left "AstExpr not an object" - -exprFromObject obj with lookup type obj -exprFromObject obj | just (string "AstExprCall") with lookup func obj | lookup args obj -exprFromObject obj | just (string "AstExprCall") | just value | just (array arr) with head arr -exprFromObject obj | just (string "AstExprCall") | just value | just (array arr) | just value2 with exprFromJSON value | exprFromJSON value2 -exprFromObject obj | just (string "AstExprCall") | just value | just (array arr) | just value2 | Right fun | Right arg = Right (fun $ arg) -exprFromObject obj | just (string "AstExprCall") | just value | just (array arr) | just value2 | Left err | _ = Left err -exprFromObject obj | just (string "AstExprCall") | just value | just (array arr) | just value2 | _ | Left err = Left err -exprFromObject obj | just (string "AstExprCall") | just value | just (array arr) | nothing = Left ("AstExprCall empty args") -exprFromObject obj | just (string "AstExprCall") | just value | just _ = Left ("AstExprCall args not an array") -exprFromObject obj | just (string "AstExprCall") | nothing | _ = Left ("AstExprCall missing func") -exprFromObject obj | just (string "AstExprCall") | _ | nothing = Left ("AstExprCall missing args") -exprFromObject obj | just (string "AstExprConstantNil") = Right (val nil) -exprFromObject obj | just (string "AstExprFunction") with lookup args obj | lookup body obj | lookup returnAnnotation obj -exprFromObject obj | just (string "AstExprFunction") | just (array arr) | just blockValue | rtn with head arr | blockFromJSON blockValue -exprFromObject obj | just (string "AstExprFunction") | just (array arr) | just blockValue | rtn | just argValue | Right B with varDecFromJSON argValue -exprFromObject obj | just (string "AstExprFunction") | just (array arr) | just blockValue | just (object rtnObj) | just argValue | Right B | Right arg with lookup types rtnObj -exprFromObject obj | just (string "AstExprFunction") | just (array arr) | just blockValue | just (object rtnObj) | just argValue | Right B | Right arg | just (array rtnTypes) with head rtnTypes -exprFromObject obj | just (string "AstExprFunction") | just (array arr) | just blockValue | just (object rtnObj) | just argValue | Right B | Right arg | just (array rtnTypes) | just rtnType with typeFromJSON rtnType -exprFromObject obj | just (string "AstExprFunction") | just (array arr) | just blockValue | just (object rtnObj) | just argValue | Right B | Right arg | just (array rtnTypes) | just rtnType | Left err = Left err -exprFromObject obj | just (string "AstExprFunction") | just (array arr) | just blockValue | just (object rtnObj) | just argValue | Right B | Right arg | just (array rtnTypes) | just rtnType | Right T = Right (function "" ⟨ arg ⟩∈ T is B end) -exprFromObject obj | just (string "AstExprFunction") | just (array arr) | just blockValue | just (object rtnObj) | just argValue | Right B | Right arg | just (array rtnTypes) | nothing = Right (function "" ⟨ arg ⟩ is B end) -exprFromObject obj | just (string "AstExprFunction") | just (array arr) | just blockValue | just (object rtnObj) | just argValue | Right B | Right arg | just _ = Left "returnAnnotation types not an array" -exprFromObject obj | just (string "AstExprFunction") | just (array arr) | just blockValue | just (object rtnObj) | just argValue | Right B | Right arg | nothing = Left "returnAnnotation missing types" -exprFromObject obj | just (string "AstExprFunction") | just (array arr) | just blockValue | just _ | just argValue | Right B | Right arg = Left "returnAnnotation not an object" -exprFromObject obj | just (string "AstExprFunction") | just (array arr) | just blockValue | nothing | just argValue | Right B | Right arg = Right (function "" ⟨ arg ⟩ is B end) -exprFromObject obj | just (string "AstExprFunction") | just (array arr) | just blockValue | rtn | just argValue | Right B | Left err = Left err -exprFromObject obj | just (string "AstExprFunction") | just (array arr) | just blockValue | rtn | nothing | Right B = Left "Unsupported AstExprFunction empty args" -exprFromObject obj | just (string "AstExprFunction") | just (array arr) | just blockValue | rtn | _ | Left err = Left err -exprFromObject obj | just (string "AstExprFunction") | just _ | just _ | rtn = Left "AstExprFunction args not an array" -exprFromObject obj | just (string "AstExprFunction") | nothing | _ | rtn = Left "AstExprFunction missing args" -exprFromObject obj | just (string "AstExprFunction") | _ | nothing | rtn = Left "AstExprFunction missing body" -exprFromObject obj | just (string "AstExprLocal") with lookup lokal obj -exprFromObject obj | just (string "AstExprLocal") | just x with varDecFromJSON x -exprFromObject obj | just (string "AstExprLocal") | just x | Right x′ = Right (var (Luau.Syntax.name x′)) -exprFromObject obj | just (string "AstExprLocal") | just x | Left err = Left err -exprFromObject obj | just (string "AstExprLocal") | nothing = Left "AstExprLocal missing local" -exprFromObject obj | just (string "AstExprConstantNumber") with lookup value obj -exprFromObject obj | just (string "AstExprConstantNumber") | just (FFI.Data.Aeson.Value.number x) = Right (val (number (toFloat x))) -exprFromObject obj | just (string "AstExprConstantNumber") | just _ = Left "AstExprConstantNumber value is not a number" -exprFromObject obj | just (string "AstExprConstantNumber") | nothing = Left "AstExprConstantNumber missing value" -exprFromObject obj | just (string "AstExprConstantString") with lookup value obj -exprFromObject obj | just (string "AstExprConstantString") | just (string x) = Right (val (string x)) -exprFromObject obj | just (string "AstExprConstantString") | just _ = Left "AstExprConstantString value is not a string" -exprFromObject obj | just (string "AstExprConstantString") | nothing = Left "AstExprConstantString missing value" -exprFromObject obj | just (string "AstExprConstantBool") with lookup value obj -exprFromObject obj | just (string "AstExprConstantBool") | just (FFI.Data.Aeson.Value.bool b) = Right (val (bool b)) -exprFromObject obj | just (string "AstExprConstantBool") | just _ = Left "AstExprConstantBool value is not a bool" -exprFromObject obj | just (string "AstExprConstantBool") | nothing = Left "AstExprConstantBool missing value" -exprFromObject obj | just (string "AstExprBinary") with lookup op obj | lookup left obj | lookup right obj -exprFromObject obj | just (string "AstExprBinary") | just o | just l | just r with binOpFromJSON o | exprFromJSON l | exprFromJSON r -exprFromObject obj | just (string "AstExprBinary") | just o | just l | just r | Right o′ | Right l′ | Right r′ = Right (binexp l′ o′ r′) -exprFromObject obj | just (string "AstExprBinary") | just o | just l | just r | Left err | _ | _ = Left err -exprFromObject obj | just (string "AstExprBinary") | just o | just l | just r | _ | Left err | _ = Left err -exprFromObject obj | just (string "AstExprBinary") | just o | just l | just r | _ | _ | Left err = Left err -exprFromObject obj | just (string "AstExprBinary") | nothing | _ | _ = Left "Missing 'op' in AstExprBinary" -exprFromObject obj | just (string "AstExprBinary") | _ | nothing | _ = Left "Missing 'left' in AstExprBinary" -exprFromObject obj | just (string "AstExprBinary") | _ | _ | nothing = Left "Missing 'right' in AstExprBinary" -exprFromObject obj | just (string ty) = Left ("TODO: Unsupported AstExpr " ++ ty) -exprFromObject obj | just _ = Left "AstExpr type not a string" -exprFromObject obj | nothing = Left "AstExpr missing type" - -{-# NON_TERMINATING #-} -statFromJSON (object obj) = statFromObject obj -statFromJSON _ = Left "AstStat not an object" - -statFromObject obj with lookup type obj -statFromObject obj | just(string "AstStatLocal") with lookup vars obj | lookup values obj -statFromObject obj | just(string "AstStatLocal") | just(array arr1) | just(array arr2) with head(arr1) | head(arr2) -statFromObject obj | just(string "AstStatLocal") | just(array arr1) | just(array arr2) | just(x) | just(value) with varDecFromJSON(x) | exprFromJSON(value) -statFromObject obj | just(string "AstStatLocal") | just(array arr1) | just(array arr2) | just(x) | just(value) | Right x′ | Right M = Right (local x′ ← M) -statFromObject obj | just(string "AstStatLocal") | just(array arr1) | just(array arr2) | just(x) | just(value) | Left err | _ = Left err -statFromObject obj | just(string "AstStatLocal") | just(array arr1) | just(array arr2) | just(x) | just(value) | _ | Left err = Left err -statFromObject obj | just(string "AstStatLocal") | just(array arr1) | just(array arr2) | just(x) | nothing = Left "AstStatLocal empty values" -statFromObject obj | just(string "AstStatLocal") | just(array arr1) | just(array arr2) | nothing | _ = Left "AstStatLocal empty vars" -statFromObject obj | just(string "AstStatLocal") | just(array arr1) | just(_) = Left "AstStatLocal values not an array" -statFromObject obj | just(string "AstStatLocal") | just(_) | just(_) = Left "AstStatLocal vars not an array" -statFromObject obj | just(string "AstStatLocal") | just(_) | nothing = Left "AstStatLocal missing values" -statFromObject obj | just(string "AstStatLocal") | nothing | _ = Left "AstStatLocal missing vars" -statFromObject obj | just(string "AstStatLocalFunction") with lookup name obj | lookup func obj -statFromObject obj | just(string "AstStatLocalFunction") | just fnName | just value with varDecFromJSON fnName | exprFromJSON value -statFromObject obj | just(string "AstStatLocalFunction") | just fnName | just value | Right fnVar | Right (function "" ⟨ x ⟩ is B end) = Right (function (Luau.Syntax.name fnVar) ⟨ x ⟩ is B end) -statFromObject obj | just(string "AstStatLocalFunction") | just fnName | just value | Right fnVar | Right (function "" ⟨ x ⟩∈ T is B end) = Right (function (Luau.Syntax.name fnVar) ⟨ x ⟩∈ T is B end) -statFromObject obj | just(string "AstStatLocalFunction") | just fnName | just value | Left err | _ = Left err -statFromObject obj | just(string "AstStatLocalFunction") | just fnName | just value | _ | Left err = Left err -statFromObject obj | just(string "AstStatLocalFunction") | just _ | just _ | Right _ | Right _ = Left "AstStatLocalFunction func is not an AstExprFunction" -statFromObject obj | just(string "AstStatLocalFunction") | nothing | _ = Left "AstStatFunction missing name" -statFromObject obj | just(string "AstStatLocalFunction") | _ | nothing = Left "AstStatFunction missing func" -statFromObject obj | just(string "AstStatReturn") with lookup list obj -statFromObject obj | just(string "AstStatReturn") | just(array arr) with head arr -statFromObject obj | just(string "AstStatReturn") | just(array arr) | just value with exprFromJSON value -statFromObject obj | just(string "AstStatReturn") | just(array arr) | just value | Right M = Right (return M) -statFromObject obj | just(string "AstStatReturn") | just(array arr) | just value | Left err = Left err -statFromObject obj | just(string "AstStatReturn") | just(array arr) | nothing = Left "AstStatReturn empty list" -statFromObject obj | just(string "AstStatReturn") | just(_) = Left "AstStatReturn list not an array" -statFromObject obj | just(string "AstStatReturn") | nothing = Left "AstStatReturn missing list" -statFromObject obj | just (string ty) = Left ("TODO: Unsupported AstStat " ++ ty) -statFromObject obj | just _ = Left "AstStat type not a string" -statFromObject obj | nothing = Left "AstStat missing type" - -blockFromJSON (array arr) = blockFromArray arr -blockFromJSON (object obj) with lookup type obj | lookup body obj -blockFromJSON (object obj) | just (string "AstStatBlock") | just value = blockFromJSON value -blockFromJSON (object obj) | just (string "AstStatBlock") | nothing = Left "AstStatBlock missing body" -blockFromJSON (object obj) | just (string ty) | _ = Left ("Unsupported AstBlock " ++ ty) -blockFromJSON (object obj) | just _ | _ = Left "AstStatBlock type not a string" -blockFromJSON (object obj) | nothing | _ = Left "AstStatBlock missing type" -blockFromJSON _ = Left "AstBlock not an array or AstStatBlock object" - -blockFromArray arr with head arr -blockFromArray arr | nothing = Right done -blockFromArray arr | just value with statFromJSON value -blockFromArray arr | just value | Left err = Left err -blockFromArray arr | just value | Right S with blockFromArray(tail arr) -blockFromArray arr | just value | Right S | Left err = Left (err) -blockFromArray arr | just value | Right S | Right B = Right (S ∙ B) - diff --git a/prototyping/Luau/Syntax/ToString.agda b/prototyping/Luau/Syntax/ToString.agda deleted file mode 100644 index 1743071a..00000000 --- a/prototyping/Luau/Syntax/ToString.agda +++ /dev/null @@ -1,83 +0,0 @@ -module Luau.Syntax.ToString where - -open import Agda.Builtin.Bool using (true; false) -open import Agda.Builtin.Float using (primShowFloat) -open import Agda.Builtin.String using (primShowString) -open import Luau.Syntax using (Value; Block; Stat; Expr; VarDec; FunDec; nil; bool; val; var; var_∈_; addr; _$_; function_is_end; return; local_←_; _∙_; done; block_is_end; _⟨_⟩; _⟨_⟩∈_; number; BinaryOperator; +; -; *; /; <; >; ==; ~=; <=; >=; ··; binexp; string) -open import FFI.Data.String using (String; _++_) -open import Luau.Addr.ToString using (addrToString) -open import Luau.Type.ToString using (typeToString) -open import Luau.Var.ToString using (varToString) - -varDecToString : ∀ {a} → VarDec a → String -varDecToString (var x) = varToString x -varDecToString (var x ∈ T) = varToString x ++ " : " ++ typeToString T - -funDecToString : ∀ {a} → FunDec a → String -funDecToString ("" ⟨ x ⟩∈ T) = "function(" ++ varDecToString x ++ "): " ++ typeToString T -funDecToString ("" ⟨ x ⟩) = "function(" ++ varDecToString x ++ ")" -funDecToString (f ⟨ x ⟩∈ T) = "function " ++ varToString f ++ "(" ++ varDecToString x ++ "): " ++ typeToString T -funDecToString (f ⟨ x ⟩) = "function " ++ varToString f ++ "(" ++ varDecToString x ++ ")" - -binOpToString : BinaryOperator → String -binOpToString + = "+" -binOpToString - = "-" -binOpToString * = "*" -binOpToString / = "/" -binOpToString < = "<" -binOpToString > = ">" -binOpToString == = "==" -binOpToString ~= = "~=" -binOpToString <= = "<=" -binOpToString >= = ">=" -binOpToString ·· = ".." - -valueToString : Value → String -valueToString nil = "nil" -valueToString (addr a) = addrToString a -valueToString (number x) = primShowFloat x -valueToString (bool false) = "false" -valueToString (bool true) = "true" -valueToString (string x) = primShowString x - -exprToString′ : ∀ {a} → String → Expr a → String -statToString′ : ∀ {a} → String → Stat a → String -blockToString′ : ∀ {a} → String → Block a → String - -exprToString′ lb (val v) = - valueToString(v) -exprToString′ lb (var x) = - varToString(x) -exprToString′ lb (M $ N) = - (exprToString′ lb M) ++ "(" ++ (exprToString′ lb N) ++ ")" -exprToString′ lb (function F is B end) = - funDecToString F ++ lb ++ - " " ++ (blockToString′ (lb ++ " ") B) ++ lb ++ - "end" -exprToString′ lb (block b is B end) = - "(" ++ varDecToString b ++ "()" ++ lb ++ - " " ++ (blockToString′ (lb ++ " ") B) ++ lb ++ - "end)()" -exprToString′ lb (binexp x op y) = exprToString′ lb x ++ " " ++ binOpToString op ++ " " ++ exprToString′ lb y - -statToString′ lb (function F is B end) = - "local " ++ funDecToString F ++ lb ++ - " " ++ (blockToString′ (lb ++ " ") B) ++ lb ++ - "end" -statToString′ lb (local x ← M) = - "local " ++ varDecToString x ++ " = " ++ (exprToString′ lb M) -statToString′ lb (return M) = - "return " ++ (exprToString′ lb M) - -blockToString′ lb (S ∙ done) = statToString′ lb S -blockToString′ lb (S ∙ B) = statToString′ lb S ++ lb ++ blockToString′ lb B -blockToString′ lb (done) = "" - -exprToString : ∀ {a} → Expr a → String -exprToString = exprToString′ "\n" - -statToString : ∀ {a} → Stat a → String -statToString = statToString′ "\n" - -blockToString : ∀ {a} → Block a → String -blockToString = blockToString′ "\n" diff --git a/prototyping/Luau/Type.agda b/prototyping/Luau/Type.agda deleted file mode 100644 index 1d0ec9e5..00000000 --- a/prototyping/Luau/Type.agda +++ /dev/null @@ -1,164 +0,0 @@ -module Luau.Type where - -open import FFI.Data.Maybe using (Maybe; just; nothing; just-inv) -open import Agda.Builtin.Equality using (_≡_; refl) -open import Properties.Dec using (Dec; yes; no) -open import Properties.Equality using (cong) -open import FFI.Data.Maybe using (Maybe; just; nothing) - -data Type : Set where - nil : Type - _⇒_ : Type → Type → Type - never : Type - unknown : Type - boolean : Type - number : Type - string : Type - _∪_ : Type → Type → Type - _∩_ : Type → Type → Type - -data Scalar : Type → Set where - - number : Scalar number - boolean : Scalar boolean - string : Scalar string - nil : Scalar nil - -skalar = number ∪ (string ∪ (nil ∪ boolean)) - -lhs : Type → Type -lhs (T ⇒ _) = T -lhs (T ∪ _) = T -lhs (T ∩ _) = T -lhs nil = nil -lhs never = never -lhs unknown = unknown -lhs number = number -lhs boolean = boolean -lhs string = string - -rhs : Type → Type -rhs (_ ⇒ T) = T -rhs (_ ∪ T) = T -rhs (_ ∩ T) = T -rhs nil = nil -rhs never = never -rhs unknown = unknown -rhs number = number -rhs boolean = boolean -rhs string = string - -_≡ᵀ_ : ∀ (T U : Type) → Dec(T ≡ U) -nil ≡ᵀ nil = yes refl -nil ≡ᵀ (S ⇒ T) = no (λ ()) -nil ≡ᵀ never = no (λ ()) -nil ≡ᵀ unknown = no (λ ()) -nil ≡ᵀ number = no (λ ()) -nil ≡ᵀ boolean = no (λ ()) -nil ≡ᵀ (S ∪ T) = no (λ ()) -nil ≡ᵀ (S ∩ T) = no (λ ()) -nil ≡ᵀ string = no (λ ()) -(S ⇒ T) ≡ᵀ string = no (λ ()) -never ≡ᵀ string = no (λ ()) -unknown ≡ᵀ string = no (λ ()) -boolean ≡ᵀ string = no (λ ()) -number ≡ᵀ string = no (λ ()) -(S ∪ T) ≡ᵀ string = no (λ ()) -(S ∩ T) ≡ᵀ string = no (λ ()) -(S ⇒ T) ≡ᵀ nil = no (λ ()) -(S ⇒ T) ≡ᵀ (U ⇒ V) with (S ≡ᵀ U) | (T ≡ᵀ V) -(S ⇒ T) ≡ᵀ (S ⇒ T) | yes refl | yes refl = yes refl -(S ⇒ T) ≡ᵀ (U ⇒ V) | _ | no p = no (λ q → p (cong rhs q)) -(S ⇒ T) ≡ᵀ (U ⇒ V) | no p | _ = no (λ q → p (cong lhs q)) -(S ⇒ T) ≡ᵀ never = no (λ ()) -(S ⇒ T) ≡ᵀ unknown = no (λ ()) -(S ⇒ T) ≡ᵀ number = no (λ ()) -(S ⇒ T) ≡ᵀ boolean = no (λ ()) -(S ⇒ T) ≡ᵀ (U ∪ V) = no (λ ()) -(S ⇒ T) ≡ᵀ (U ∩ V) = no (λ ()) -never ≡ᵀ nil = no (λ ()) -never ≡ᵀ (U ⇒ V) = no (λ ()) -never ≡ᵀ never = yes refl -never ≡ᵀ unknown = no (λ ()) -never ≡ᵀ number = no (λ ()) -never ≡ᵀ boolean = no (λ ()) -never ≡ᵀ (U ∪ V) = no (λ ()) -never ≡ᵀ (U ∩ V) = no (λ ()) -unknown ≡ᵀ nil = no (λ ()) -unknown ≡ᵀ (U ⇒ V) = no (λ ()) -unknown ≡ᵀ never = no (λ ()) -unknown ≡ᵀ unknown = yes refl -unknown ≡ᵀ number = no (λ ()) -unknown ≡ᵀ boolean = no (λ ()) -unknown ≡ᵀ (U ∪ V) = no (λ ()) -unknown ≡ᵀ (U ∩ V) = no (λ ()) -number ≡ᵀ nil = no (λ ()) -number ≡ᵀ (T ⇒ U) = no (λ ()) -number ≡ᵀ never = no (λ ()) -number ≡ᵀ unknown = no (λ ()) -number ≡ᵀ number = yes refl -number ≡ᵀ boolean = no (λ ()) -number ≡ᵀ (T ∪ U) = no (λ ()) -number ≡ᵀ (T ∩ U) = no (λ ()) -boolean ≡ᵀ nil = no (λ ()) -boolean ≡ᵀ (T ⇒ U) = no (λ ()) -boolean ≡ᵀ never = no (λ ()) -boolean ≡ᵀ unknown = no (λ ()) -boolean ≡ᵀ boolean = yes refl -boolean ≡ᵀ number = no (λ ()) -boolean ≡ᵀ (T ∪ U) = no (λ ()) -boolean ≡ᵀ (T ∩ U) = no (λ ()) -string ≡ᵀ nil = no (λ ()) -string ≡ᵀ (x ⇒ x₁) = no (λ ()) -string ≡ᵀ never = no (λ ()) -string ≡ᵀ unknown = no (λ ()) -string ≡ᵀ boolean = no (λ ()) -string ≡ᵀ number = no (λ ()) -string ≡ᵀ string = yes refl -string ≡ᵀ (U ∪ V) = no (λ ()) -string ≡ᵀ (U ∩ V) = no (λ ()) -(S ∪ T) ≡ᵀ nil = no (λ ()) -(S ∪ T) ≡ᵀ (U ⇒ V) = no (λ ()) -(S ∪ T) ≡ᵀ never = no (λ ()) -(S ∪ T) ≡ᵀ unknown = no (λ ()) -(S ∪ T) ≡ᵀ number = no (λ ()) -(S ∪ T) ≡ᵀ boolean = no (λ ()) -(S ∪ T) ≡ᵀ (U ∪ V) with (S ≡ᵀ U) | (T ≡ᵀ V) -(S ∪ T) ≡ᵀ (S ∪ T) | yes refl | yes refl = yes refl -(S ∪ T) ≡ᵀ (U ∪ V) | _ | no p = no (λ q → p (cong rhs q)) -(S ∪ T) ≡ᵀ (U ∪ V) | no p | _ = no (λ q → p (cong lhs q)) -(S ∪ T) ≡ᵀ (U ∩ V) = no (λ ()) -(S ∩ T) ≡ᵀ nil = no (λ ()) -(S ∩ T) ≡ᵀ (U ⇒ V) = no (λ ()) -(S ∩ T) ≡ᵀ never = no (λ ()) -(S ∩ T) ≡ᵀ unknown = no (λ ()) -(S ∩ T) ≡ᵀ number = no (λ ()) -(S ∩ T) ≡ᵀ boolean = no (λ ()) -(S ∩ T) ≡ᵀ (U ∪ V) = no (λ ()) -(S ∩ T) ≡ᵀ (U ∩ V) with (S ≡ᵀ U) | (T ≡ᵀ V) -(S ∩ T) ≡ᵀ (U ∩ V) | yes refl | yes refl = yes refl -(S ∩ T) ≡ᵀ (U ∩ V) | _ | no p = no (λ q → p (cong rhs q)) -(S ∩ T) ≡ᵀ (U ∩ V) | no p | _ = no (λ q → p (cong lhs q)) - -_≡ᴹᵀ_ : ∀ (T U : Maybe Type) → Dec(T ≡ U) -nothing ≡ᴹᵀ nothing = yes refl -nothing ≡ᴹᵀ just U = no (λ ()) -just T ≡ᴹᵀ nothing = no (λ ()) -just T ≡ᴹᵀ just U with T ≡ᵀ U -(just T ≡ᴹᵀ just T) | yes refl = yes refl -(just T ≡ᴹᵀ just U) | no p = no (λ q → p (just-inv q)) - -optional : Type → Type -optional nil = nil -optional (T ∪ nil) = (T ∪ nil) -optional T = (T ∪ nil) - -normalizeOptional : Type → Type -normalizeOptional (S ∪ T) with normalizeOptional S | normalizeOptional T -normalizeOptional (S ∪ T) | (S′ ∪ nil) | (T′ ∪ nil) = (S′ ∪ T′) ∪ nil -normalizeOptional (S ∪ T) | S′ | (T′ ∪ nil) = (S′ ∪ T′) ∪ nil -normalizeOptional (S ∪ T) | (S′ ∪ nil) | T′ = (S′ ∪ T′) ∪ nil -normalizeOptional (S ∪ T) | S′ | nil = optional S′ -normalizeOptional (S ∪ T) | nil | T′ = optional T′ -normalizeOptional (S ∪ T) | S′ | T′ = S′ ∪ T′ -normalizeOptional T = T diff --git a/prototyping/Luau/Type/FromJSON.agda b/prototyping/Luau/Type/FromJSON.agda deleted file mode 100644 index e3d1e8e7..00000000 --- a/prototyping/Luau/Type/FromJSON.agda +++ /dev/null @@ -1,72 +0,0 @@ -{-# OPTIONS --rewriting #-} - -module Luau.Type.FromJSON where - -open import Luau.Type using (Type; nil; _⇒_; _∪_; _∩_; unknown; never; number; string) - -open import Agda.Builtin.List using (List; _∷_; []) -open import Agda.Builtin.Bool using (true; false) - -open import FFI.Data.Aeson using (Value; Array; Object; object; array; string; fromString; lookup) -open import FFI.Data.Either using (Either; Left; Right) -open import FFI.Data.Maybe using (Maybe; nothing; just) -open import FFI.Data.String using (String; _++_) -open import FFI.Data.Vector using (head; tail; null; empty) - -name = fromString "name" -type = fromString "type" -argTypes = fromString "argTypes" -returnTypes = fromString "returnTypes" -types = fromString "types" - -{-# TERMINATING #-} -typeFromJSON : Value → Either String Type -compoundFromArray : (Type → Type → Type) → Array → Either String Type - -typeFromJSON (object o) with lookup type o -typeFromJSON (object o) | just (string "AstTypeFunction") with lookup argTypes o | lookup returnTypes o -typeFromJSON (object o) | just (string "AstTypeFunction") | just (object argsSet) | just (object retsSet) with lookup types argsSet | lookup types retsSet -typeFromJSON (object o) | just (string "AstTypeFunction") | just (object argsSet) | just (object retsSet) | just (array args) | just (array rets) with head args | head rets -typeFromJSON (object o) | just (string "AstTypeFunction") | just (object argsSet) | just (object retsSet) | just (array args) | just (array rets) | just argValue | just retValue with typeFromJSON argValue | typeFromJSON retValue -typeFromJSON (object o) | just (string "AstTypeFunction") | just (object argsSet) | just (object retsSet) | just (array args) | just (array rets) | just argValue | just retValue | Right arg | Right ret = Right (arg ⇒ ret) -typeFromJSON (object o) | just (string "AstTypeFunction") | just (object argsSet) | just (object retsSet) | just (array args) | just (array rets) | just argValue | just retValue | Left err | _ = Left err -typeFromJSON (object o) | just (string "AstTypeFunction") | just (object argsSet) | just (object retsSet) | just (array args) | just (array rets) | just argValue | just retValue | _ | Left err = Left err -typeFromJSON (object o) | just (string "AstTypeFunction") | just (object argsSet) | just (object retsSet) | just (array args) | just (array rets) | _ | nothing = Left "No return type" -typeFromJSON (object o) | just (string "AstTypeFunction") | just (object argsSet) | just (object retsSet) | just (array args) | just (array rets) | nothing | _ = Left "No argument type" -typeFromJSON (object o) | just (string "AstTypeFunction") | just (object argsSet) | just (object retsSet) | just _ | _ = Left "argTypes.types is not an array" -typeFromJSON (object o) | just (string "AstTypeFunction") | just (object argsSet) | just (object retsSet) | _ | just _ = Left "retTypes.types is not an array" -typeFromJSON (object o) | just (string "AstTypeFunction") | just (object argsSet) | just (object retsSet) | nothing | _ = Left "argTypes.types does not exist" -typeFromJSON (object o) | just (string "AstTypeFunction") | _ | just _ = Left "argTypes is not an object" -typeFromJSON (object o) | just (string "AstTypeFunction") | just _ | _ = Left "returnTypes is not an object" -typeFromJSON (object o) | just (string "AstTypeFunction") | nothing | nothing = Left "Missing argTypes and returnTypes" - -typeFromJSON (object o) | just (string "AstTypeReference") with lookup name o -typeFromJSON (object o) | just (string "AstTypeReference") | just (string "nil") = Right nil -typeFromJSON (object o) | just (string "AstTypeReference") | just (string "any") = Right unknown -- not quite right -typeFromJSON (object o) | just (string "AstTypeReference") | just (string "unknown") = Right unknown -typeFromJSON (object o) | just (string "AstTypeReference") | just (string "never") = Right never -typeFromJSON (object o) | just (string "AstTypeReference") | just (string "number") = Right number -typeFromJSON (object o) | just (string "AstTypeReference") | just (string "string") = Right string -typeFromJSON (object o) | just (string "AstTypeReference") | _ = Left "Unknown referenced type" - -typeFromJSON (object o) | just (string "AstTypeUnion") with lookup types o -typeFromJSON (object o) | just (string "AstTypeUnion") | just (array types) = compoundFromArray _∪_ types -typeFromJSON (object o) | just (string "AstTypeUnion") | _ = Left "`types` field must be an array" - -typeFromJSON (object o) | just (string "AstTypeIntersection") with lookup types o -typeFromJSON (object o) | just (string "AstTypeIntersection") | just (array types) = compoundFromArray _∩_ types -typeFromJSON (object o) | just (string "AstTypeIntersection") | _ = Left "`types` field must be an array" - -typeFromJSON (object o) | just (string ty) = Left ("Unsupported type " ++ ty) -typeFromJSON (object o) | just _ = Left "`type` field must be a string" -typeFromJSON (object o) | nothing = Left "No `type` field" -typeFromJSON _ = Left "Unsupported JSON type" - -compoundFromArray ctor ts with head ts | tail ts -compoundFromArray ctor ts | just hd | tl with null tl -compoundFromArray ctor ts | just hd | tl | true = typeFromJSON hd -compoundFromArray ctor ts | just hd | tl | false with typeFromJSON hd | compoundFromArray ctor tl -compoundFromArray ctor ts | just hd | tl | false | Right hdTy | Right tlTy = Right (ctor hdTy tlTy) -compoundFromArray ctor ts | just hd | tl | false | Left err | _ = Left err -compoundFromArray ctor ts | just hd | tl | false | _ | Left Err = Left Err -compoundFromArray ctor ts | nothing | empty = Left "Empty types array?" diff --git a/prototyping/Luau/Type/ToString.agda b/prototyping/Luau/Type/ToString.agda deleted file mode 100644 index a41ecec2..00000000 --- a/prototyping/Luau/Type/ToString.agda +++ /dev/null @@ -1,29 +0,0 @@ -module Luau.Type.ToString where - -open import FFI.Data.String using (String; _++_) -open import Luau.Type using (Type; nil; _⇒_; never; unknown; number; boolean; string; _∪_; _∩_; normalizeOptional) - -{-# TERMINATING #-} -typeToString : Type → String -typeToStringᵁ : Type → String -typeToStringᴵ : Type → String - -typeToString nil = "nil" -typeToString (S ⇒ T) = "(" ++ (typeToString S) ++ ") -> " ++ (typeToString T) -typeToString never = "never" -typeToString unknown = "unknown" -typeToString number = "number" -typeToString boolean = "boolean" -typeToString string = "string" -typeToString (S ∪ T) with normalizeOptional(S ∪ T) -typeToString (S ∪ T) | ((S′ ⇒ T′) ∪ nil) = "(" ++ typeToString (S′ ⇒ T′) ++ ")?" -typeToString (S ∪ T) | (S′ ∪ nil) = typeToString S′ ++ "?" -typeToString (S ∪ T) | (S′ ∪ T′) = "(" ++ typeToStringᵁ (S ∪ T) ++ ")" -typeToString (S ∪ T) | T′ = typeToString T′ -typeToString (S ∩ T) = "(" ++ typeToStringᴵ (S ∩ T) ++ ")" - -typeToStringᵁ (S ∪ T) = (typeToStringᵁ S) ++ " | " ++ (typeToStringᵁ T) -typeToStringᵁ T = typeToString T - -typeToStringᴵ (S ∩ T) = (typeToStringᴵ S) ++ " & " ++ (typeToStringᴵ T) -typeToStringᴵ T = typeToString T diff --git a/prototyping/Luau/TypeCheck.agda b/prototyping/Luau/TypeCheck.agda deleted file mode 100644 index 1abc1eda..00000000 --- a/prototyping/Luau/TypeCheck.agda +++ /dev/null @@ -1,160 +0,0 @@ -{-# OPTIONS --rewriting #-} - -module Luau.TypeCheck where - -open import Agda.Builtin.Equality using (_≡_) -open import FFI.Data.Either using (Either; Left; Right) -open import FFI.Data.Maybe using (Maybe; just) -open import Luau.ResolveOverloads using (resolve) -open import Luau.Syntax using (Expr; Stat; Block; BinaryOperator; yes; nil; addr; number; bool; string; val; var; var_∈_; _⟨_⟩∈_; function_is_end; _$_; block_is_end; binexp; local_←_; _∙_; done; return; name; +; -; *; /; <; >; ==; ~=; <=; >=; ··) -open import Luau.Var using (Var) -open import Luau.Addr using (Addr) -open import Luau.Heap using (Heap; Object; function_is_end) renaming (_[_] to _[_]ᴴ) -open import Luau.Type using (Type; nil; unknown; number; boolean; string; _⇒_) -open import Luau.VarCtxt using (VarCtxt; ∅; _⋒_; _↦_; _⊕_↦_; _⊝_) renaming (_[_] to _[_]ⱽ) -open import FFI.Data.Vector using (Vector) -open import FFI.Data.Maybe using (Maybe; just; nothing) -open import Properties.DecSubtyping using (dec-subtyping) -open import Properties.Product using (_×_; _,_) - -orUnknown : Maybe Type → Type -orUnknown nothing = unknown -orUnknown (just T) = T - -srcBinOp : BinaryOperator → Type -srcBinOp + = number -srcBinOp - = number -srcBinOp * = number -srcBinOp / = number -srcBinOp < = number -srcBinOp > = number -srcBinOp == = unknown -srcBinOp ~= = unknown -srcBinOp <= = number -srcBinOp >= = number -srcBinOp ·· = string - -tgtBinOp : BinaryOperator → Type -tgtBinOp + = number -tgtBinOp - = number -tgtBinOp * = number -tgtBinOp / = number -tgtBinOp < = boolean -tgtBinOp > = boolean -tgtBinOp == = boolean -tgtBinOp ~= = boolean -tgtBinOp <= = boolean -tgtBinOp >= = boolean -tgtBinOp ·· = string - -data _⊢ᴮ_∈_ : VarCtxt → Block yes → Type → Set -data _⊢ᴱ_∈_ : VarCtxt → Expr yes → Type → Set - -data _⊢ᴮ_∈_ where - - done : ∀ {Γ} → - - --------------- - Γ ⊢ᴮ done ∈ nil - - return : ∀ {M B T U Γ} → - - Γ ⊢ᴱ M ∈ T → - Γ ⊢ᴮ B ∈ U → - --------------------- - Γ ⊢ᴮ return M ∙ B ∈ T - - local : ∀ {x M B T U V Γ} → - - Γ ⊢ᴱ M ∈ U → - (Γ ⊕ x ↦ T) ⊢ᴮ B ∈ V → - -------------------------------- - Γ ⊢ᴮ local var x ∈ T ← M ∙ B ∈ V - - function : ∀ {f x B C T U V W Γ} → - - (Γ ⊕ x ↦ T) ⊢ᴮ C ∈ V → - (Γ ⊕ f ↦ (T ⇒ U)) ⊢ᴮ B ∈ W → - ------------------------------------------------- - Γ ⊢ᴮ function f ⟨ var x ∈ T ⟩∈ U is C end ∙ B ∈ W - -data _⊢ᴱ_∈_ where - - nil : ∀ {Γ} → - - -------------------- - Γ ⊢ᴱ (val nil) ∈ nil - - var : ∀ {x T Γ} → - - T ≡ orUnknown(Γ [ x ]ⱽ) → - ---------------- - Γ ⊢ᴱ (var x) ∈ T - - addr : ∀ {a Γ} T → - - ----------------- - Γ ⊢ᴱ val(addr a) ∈ T - - number : ∀ {n Γ} → - - --------------------------- - Γ ⊢ᴱ val(number n) ∈ number - - bool : ∀ {b Γ} → - - -------------------------- - Γ ⊢ᴱ val(bool b) ∈ boolean - - string : ∀ {x Γ} → - - --------------------------- - Γ ⊢ᴱ val(string x) ∈ string - - app : ∀ {M N T U Γ} → - - Γ ⊢ᴱ M ∈ T → - Γ ⊢ᴱ N ∈ U → - ---------------------------- - Γ ⊢ᴱ (M $ N) ∈ (resolve T U) - - function : ∀ {f x B T U V Γ} → - - (Γ ⊕ x ↦ T) ⊢ᴮ B ∈ V → - ----------------------------------------------------- - Γ ⊢ᴱ (function f ⟨ var x ∈ T ⟩∈ U is B end) ∈ (T ⇒ U) - - block : ∀ {b B T U Γ} → - - Γ ⊢ᴮ B ∈ U → - ------------------------------------ - Γ ⊢ᴱ (block var b ∈ T is B end) ∈ T - - binexp : ∀ {op Γ M N T U} → - - Γ ⊢ᴱ M ∈ T → - Γ ⊢ᴱ N ∈ U → - ---------------------------------- - Γ ⊢ᴱ (binexp M op N) ∈ tgtBinOp op - -data ⊢ᴼ_ : Maybe(Object yes) → Set where - - nothing : - - --------- - ⊢ᴼ nothing - - function : ∀ {f x T U V B} → - - (x ↦ T) ⊢ᴮ B ∈ V → - ---------------------------------------------- - ⊢ᴼ (just function f ⟨ var x ∈ T ⟩∈ U is B end) - -⊢ᴴ_ : Heap yes → Set -⊢ᴴ H = ∀ a {O} → (H [ a ]ᴴ ≡ O) → (⊢ᴼ O) - -_⊢ᴴᴱ_▷_∈_ : VarCtxt → Heap yes → Expr yes → Type → Set -(Γ ⊢ᴴᴱ H ▷ M ∈ T) = (⊢ᴴ H) × (Γ ⊢ᴱ M ∈ T) - -_⊢ᴴᴮ_▷_∈_ : VarCtxt → Heap yes → Block yes → Type → Set -(Γ ⊢ᴴᴮ H ▷ B ∈ T) = (⊢ᴴ H) × (Γ ⊢ᴮ B ∈ T) diff --git a/prototyping/Luau/TypeNormalization.agda b/prototyping/Luau/TypeNormalization.agda deleted file mode 100644 index 08f14474..00000000 --- a/prototyping/Luau/TypeNormalization.agda +++ /dev/null @@ -1,65 +0,0 @@ -module Luau.TypeNormalization where - -open import Luau.Type using (Type; nil; number; string; boolean; never; unknown; _⇒_; _∪_; _∩_) - --- Operations on normalized types -_∪ᶠ_ : Type → Type → Type -_∪ⁿˢ_ : Type → Type → Type -_∩ⁿˢ_ : Type → Type → Type -_∪ⁿ_ : Type → Type → Type -_∩ⁿ_ : Type → Type → Type - --- Union of function types -(F₁ ∩ F₂) ∪ᶠ G = (F₁ ∪ᶠ G) ∩ (F₂ ∪ᶠ G) -F ∪ᶠ (G₁ ∩ G₂) = (F ∪ᶠ G₁) ∩ (F ∪ᶠ G₂) -(R ⇒ S) ∪ᶠ (T ⇒ U) = (R ∩ⁿ T) ⇒ (S ∪ⁿ U) -F ∪ᶠ G = F ∪ G - --- Union of normalized types -S ∪ⁿ (T₁ ∪ T₂) = (S ∪ⁿ T₁) ∪ T₂ -S ∪ⁿ unknown = unknown -S ∪ⁿ never = S -never ∪ⁿ T = T -unknown ∪ⁿ T = unknown -(S₁ ∪ S₂) ∪ⁿ G = (S₁ ∪ⁿ G) ∪ S₂ -F ∪ⁿ G = F ∪ᶠ G - --- Intersection of normalized types -S ∩ⁿ (T₁ ∪ T₂) = (S ∩ⁿ T₁) ∪ⁿˢ (S ∩ⁿˢ T₂) -S ∩ⁿ unknown = S -S ∩ⁿ never = never -(S₁ ∪ S₂) ∩ⁿ G = (S₁ ∩ⁿ G) -unknown ∩ⁿ G = G -never ∩ⁿ G = never -F ∩ⁿ G = F ∩ G - --- Intersection of normalized types with a scalar -(S₁ ∪ nil) ∩ⁿˢ nil = nil -(S₁ ∪ boolean) ∩ⁿˢ boolean = boolean -(S₁ ∪ number) ∩ⁿˢ number = number -(S₁ ∪ string) ∩ⁿˢ string = string -(S₁ ∪ S₂) ∩ⁿˢ T = S₁ ∩ⁿˢ T -unknown ∩ⁿˢ T = T -F ∩ⁿˢ T = never - --- Union of normalized types with an optional scalar -S ∪ⁿˢ never = S -unknown ∪ⁿˢ T = unknown -(S₁ ∪ nil) ∪ⁿˢ nil = S₁ ∪ nil -(S₁ ∪ boolean) ∪ⁿˢ boolean = S₁ ∪ boolean -(S₁ ∪ number) ∪ⁿˢ number = S₁ ∪ number -(S₁ ∪ string) ∪ⁿˢ string = S₁ ∪ string -(S₁ ∪ S₂) ∪ⁿˢ T = (S₁ ∪ⁿˢ T) ∪ S₂ -F ∪ⁿˢ T = F ∪ T - --- Normalize! -normalize : Type → Type -normalize nil = never ∪ nil -normalize (S ⇒ T) = (normalize S ⇒ normalize T) -normalize never = never -normalize unknown = unknown -normalize boolean = never ∪ boolean -normalize number = never ∪ number -normalize string = never ∪ string -normalize (S ∪ T) = normalize S ∪ⁿ normalize T -normalize (S ∩ T) = normalize S ∩ⁿ normalize T diff --git a/prototyping/Luau/TypeSaturation.agda b/prototyping/Luau/TypeSaturation.agda deleted file mode 100644 index fa24ff73..00000000 --- a/prototyping/Luau/TypeSaturation.agda +++ /dev/null @@ -1,66 +0,0 @@ -module Luau.TypeSaturation where - -open import Luau.Type using (Type; _⇒_; _∩_; _∪_) -open import Luau.TypeNormalization using (_∪ⁿ_; _∩ⁿ_) - --- So, there's a problem with overloaded functions --- (of the form (S_1 ⇒ T_1) ∩⋯∩ (S_n ⇒ T_n)) --- which is that it's not good enough to compare them --- for subtyping by comparing all of their overloads. - --- For example (nil → nil) is a subtype of (number? → number?) ∩ (string? → string?) --- but not a subtype of any of its overloads. - --- To fix this, we adapt the semantic subtyping algorithm for --- function types, given in --- https://www.irif.fr/~gc/papers/covcon-again.pdf and --- https://pnwamk.github.io/sst-tutorial/ - --- A function type is *intersection-saturated* if for any overloads --- (S₁ ⇒ T₁) and (S₂ ⇒ T₂), there exists an overload which is a subtype --- of ((S₁ ∩ S₂) ⇒ (T₁ ∩ T₂)). - --- A function type is *union-saturated* if for any overloads --- (S₁ ⇒ T₁) and (S₂ ⇒ T₂), there exists an overload which is a subtype --- of ((S₁ ∪ S₂) ⇒ (T₁ ∪ T₂)). - --- A function type is *saturated* if it is both intersection- and --- union-saturated. - --- For example (number? → number?) ∩ (string? → string?) --- is not saturated, but (number? → number?) ∩ (string? → string?) ∩ (nil → nil) ∩ ((number ∪ string)? → (number ∪ string)?) --- is. - --- Saturated function types have the nice property that they can ber --- compared by just comparing their overloads: F <: G whenever for any --- overload of G, there is an overload os F which is a subtype of it. - --- Forunately every function type can be saturated! -_⋓_ : Type → Type → Type -(S₁ ⇒ T₁) ⋓ (S₂ ⇒ T₂) = (S₁ ∪ⁿ S₂) ⇒ (T₁ ∪ⁿ T₂) -(F₁ ∩ G₁) ⋓ F₂ = (F₁ ⋓ F₂) ∩ (G₁ ⋓ F₂) -F₁ ⋓ (F₂ ∩ G₂) = (F₁ ⋓ F₂) ∩ (F₁ ⋓ G₂) -F ⋓ G = F ∩ G - -_⋒_ : Type → Type → Type -(S₁ ⇒ T₁) ⋒ (S₂ ⇒ T₂) = (S₁ ∩ⁿ S₂) ⇒ (T₁ ∩ⁿ T₂) -(F₁ ∩ G₁) ⋒ F₂ = (F₁ ⋒ F₂) ∩ (G₁ ⋒ F₂) -F₁ ⋒ (F₂ ∩ G₂) = (F₁ ⋒ F₂) ∩ (F₁ ⋒ G₂) -F ⋒ G = F ∩ G - -_∩ᵘ_ : Type → Type → Type -F ∩ᵘ G = (F ∩ G) ∩ (F ⋓ G) - -_∩ⁱ_ : Type → Type → Type -F ∩ⁱ G = (F ∩ G) ∩ (F ⋒ G) - -∪-saturate : Type → Type -∪-saturate (F ∩ G) = (∪-saturate F ∩ᵘ ∪-saturate G) -∪-saturate F = F - -∩-saturate : Type → Type -∩-saturate (F ∩ G) = (∩-saturate F ∩ⁱ ∩-saturate G) -∩-saturate F = F - -saturate : Type → Type -saturate F = ∪-saturate (∩-saturate F) diff --git a/prototyping/Luau/Var.agda b/prototyping/Luau/Var.agda deleted file mode 100644 index 23d8b56e..00000000 --- a/prototyping/Luau/Var.agda +++ /dev/null @@ -1,16 +0,0 @@ -module Luau.Var where - -open import Agda.Builtin.Bool using (true; false) -open import Agda.Builtin.Equality using (_≡_) -open import Agda.Builtin.String using (String; primStringEquality) -open import Agda.Builtin.TrustMe using (primTrustMe) -open import Properties.Dec using (Dec; yes; no) -open import Properties.Equality using (_≢_) - -Var : Set -Var = String - -_≡ⱽ_ : (a b : Var) → Dec (a ≡ b) -a ≡ⱽ b with primStringEquality a b -a ≡ⱽ b | false = no p where postulate p : (a ≢ b) -a ≡ⱽ b | true = yes primTrustMe diff --git a/prototyping/Luau/Var/ToString.agda b/prototyping/Luau/Var/ToString.agda deleted file mode 100644 index 10cd915b..00000000 --- a/prototyping/Luau/Var/ToString.agda +++ /dev/null @@ -1,8 +0,0 @@ -module Luau.Var.ToString where - -open import Agda.Builtin.String using (String) -open import Luau.Var using (Var) - -varToString : Var → String -varToString x = x - diff --git a/prototyping/Luau/VarCtxt.agda b/prototyping/Luau/VarCtxt.agda deleted file mode 100644 index 76f64dfe..00000000 --- a/prototyping/Luau/VarCtxt.agda +++ /dev/null @@ -1,43 +0,0 @@ -{-# OPTIONS --rewriting #-} - -module Luau.VarCtxt where - -open import Agda.Builtin.Equality using (_≡_) -open import Luau.Type using (Type; _∪_; _∩_) -open import Luau.Var using (Var) -open import FFI.Data.Aeson using (KeyMap; Key; empty; unionWith; singleton; insert; delete; lookup; toString; fromString; lookup-insert; lookup-insert-not; lookup-empty; to-from; insert-swap; insert-over) -open import FFI.Data.Maybe using (Maybe; just; nothing) -open import Properties.Equality using (_≢_; cong; sym; trans) - -VarCtxt : Set -VarCtxt = KeyMap Type - -∅ : VarCtxt -∅ = empty - -_⋒_ : VarCtxt → VarCtxt → VarCtxt -_⋒_ = unionWith _∩_ - -_⋓_ : VarCtxt → VarCtxt → VarCtxt -_⋓_ = unionWith _∪_ - -_[_] : VarCtxt → Var → Maybe Type -Γ [ x ] = lookup (fromString x) Γ - -_⊝_ : VarCtxt → Var → VarCtxt -Γ ⊝ x = delete (fromString x) Γ - -_↦_ : Var → Type → VarCtxt -x ↦ T = singleton (fromString x) T - -_⊕_↦_ : VarCtxt → Var → Type → VarCtxt -Γ ⊕ x ↦ T = insert (fromString x) T Γ - -⊕-over : ∀ {Γ x y T U} → (x ≡ y) → ((Γ ⊕ x ↦ T) ⊕ y ↦ U) ≡ (Γ ⊕ y ↦ U) -⊕-over p = insert-over _ _ _ _ _ (cong fromString (sym p)) - -⊕-swap : ∀ {Γ x y T U} → (x ≢ y) → ((Γ ⊕ x ↦ T) ⊕ y ↦ U) ≡ ((Γ ⊕ y ↦ U) ⊕ x ↦ T) -⊕-swap p = insert-swap _ _ _ _ _ (λ q → p (trans (sym (to-from _)) (trans (cong toString (sym q) ) (to-from _))) ) - -⊕-lookup-miss : ∀ x y T Γ → (x ≢ y) → (Γ [ y ] ≡ (Γ ⊕ x ↦ T) [ y ]) -⊕-lookup-miss x y T Γ p = lookup-insert-not (fromString x) (fromString y) T Γ λ q → p (trans (sym (to-from x)) (trans (cong toString q) (to-from y))) diff --git a/prototyping/PrettyPrinter.agda b/prototyping/PrettyPrinter.agda deleted file mode 100644 index 3ce0905b..00000000 --- a/prototyping/PrettyPrinter.agda +++ /dev/null @@ -1,34 +0,0 @@ -{-# OPTIONS --rewriting #-} - -module PrettyPrinter where - -open import Agda.Builtin.IO using (IO) -open import Agda.Builtin.Int using (pos) -open import Agda.Builtin.Unit using (⊤) - -open import FFI.IO using (getContents; putStrLn; _>>=_; _>>_) -open import FFI.Data.Aeson using (Value; eitherDecode) -open import FFI.Data.Either using (Left; Right) -open import FFI.Data.String using (String; _++_) -open import FFI.Data.Text.Encoding using (encodeUtf8) -open import FFI.System.Exit using (exitWith; ExitFailure) - -open import Luau.Syntax using (Block) -open import Luau.Syntax.FromJSON using (blockFromJSON) -open import Luau.Syntax.ToString using (blockToString) - -runBlock : ∀ {a} → Block a → IO ⊤ -runBlock block = putStrLn (blockToString block) - -runJSON : Value → IO ⊤ -runJSON value with blockFromJSON(value) -runJSON value | (Left err) = putStrLn ("Luau error: " ++ err) >> exitWith (ExitFailure (pos 1)) -runJSON value | (Right block) = runBlock block - -runString : String → IO ⊤ -runString txt with eitherDecode (encodeUtf8 txt) -runString txt | (Left err) = putStrLn ("JSON error: " ++ err) >> exitWith (ExitFailure (pos 1)) -runString txt | (Right value) = runJSON value - -main : IO ⊤ -main = getContents >>= runString diff --git a/prototyping/Properties.agda b/prototyping/Properties.agda deleted file mode 100644 index f883a3ea..00000000 --- a/prototyping/Properties.agda +++ /dev/null @@ -1,15 +0,0 @@ -{-# OPTIONS --rewriting #-} - -module Properties where - -import Properties.Contradiction -import Properties.Dec -import Properties.DecSubtyping -import Properties.Equality -import Properties.Functions -import Properties.Remember -import Properties.Step -import Properties.StrictMode -import Properties.Subtyping -import Properties.TypeCheck -import Properties.TypeNormalization diff --git a/prototyping/Properties/Contradiction.agda b/prototyping/Properties/Contradiction.agda deleted file mode 100644 index e9a92ad5..00000000 --- a/prototyping/Properties/Contradiction.agda +++ /dev/null @@ -1,9 +0,0 @@ -module Properties.Contradiction where - -data ⊥ : Set where - -¬ : Set → Set -¬ A = A → ⊥ - -CONTRADICTION : ∀ {A : Set} → ⊥ → A -CONTRADICTION () diff --git a/prototyping/Properties/Dec.agda b/prototyping/Properties/Dec.agda deleted file mode 100644 index c61f2365..00000000 --- a/prototyping/Properties/Dec.agda +++ /dev/null @@ -1,7 +0,0 @@ -module Properties.Dec where - -open import Properties.Contradiction using (¬) - -data Dec(A : Set) : Set where - yes : A → Dec A - no : ¬ A → Dec A diff --git a/prototyping/Properties/DecSubtyping.agda b/prototyping/Properties/DecSubtyping.agda deleted file mode 100644 index 8dc7a446..00000000 --- a/prototyping/Properties/DecSubtyping.agda +++ /dev/null @@ -1,174 +0,0 @@ -{-# OPTIONS --rewriting #-} - -module Properties.DecSubtyping where - -open import Agda.Builtin.Equality using (_≡_; refl) -open import FFI.Data.Either using (Either; Left; Right; mapLR; swapLR; cond) -open import Luau.Subtyping using (_<:_; _≮:_; Tree; Language; ¬Language; witness; unknown; never; scalar; function; scalar-function; scalar-function-ok; scalar-function-err; scalar-function-tgt; scalar-scalar; function-scalar; function-ok; function-ok₁; function-ok₂; function-err; function-tgt; left; right; _,_) -open import Luau.Type using (Type; Scalar; nil; number; string; boolean; never; unknown; _⇒_; _∪_; _∩_) -open import Luau.TypeNormalization using (_∪ⁿ_; _∩ⁿ_) -open import Luau.TypeSaturation using (saturate) -open import Properties.Contradiction using (CONTRADICTION; ¬) -open import Properties.Functions using (_∘_) -open import Properties.Subtyping using (<:-refl; <:-trans; ≮:-trans-<:; <:-trans-≮:; <:-never; <:-unknown; <:-∪-left; <:-∪-right; <:-∪-lub; ≮:-∪-left; ≮:-∪-right; <:-∩-left; <:-∩-right; <:-∩-glb; ≮:-∩-left; ≮:-∩-right; dec-language; scalar-<:; <:-everything; <:-function; ≮:-function-left; ≮:-function-right; <:-impl-¬≮:; <:-intersect; <:-function-∩-∪; <:-function-∩; <:-union; ≮:-left-∪; ≮:-right-∪; <:-∩-distr-∪; <:-impl-⊇; language-comp) -open import Properties.TypeNormalization using (FunType; Normal; never; unknown; _∩_; _∪_; _⇒_; normal; <:-normalize; normalize-<:; normal-∩ⁿ; normal-∪ⁿ; ∪-<:-∪ⁿ; ∪ⁿ-<:-∪; ∩ⁿ-<:-∩; ∩-<:-∩ⁿ; normalᶠ; fun-top; fun-function; fun-¬scalar) -open import Properties.TypeSaturation using (Overloads; Saturated; _⊆ᵒ_; _<:ᵒ_; defn; here; left; right; ov-language; ov-<:; saturated; normal-saturate; normal-overload-src; normal-overload-tgt; saturate-<:; <:-saturate; <:ᵒ-impl-<:; _>>=ˡ_; _>>=ʳ_) -open import Properties.Equality using (_≢_) - --- Honest this terminates, since saturation maintains the depth of nested arrows -{-# TERMINATING #-} -dec-subtypingˢⁿ : ∀ {T U} → Scalar T → Normal U → Either (T ≮: U) (T <: U) -dec-subtypingˢᶠ : ∀ {F G} → FunType F → Saturated F → FunType G → Either (F ≮: G) (F <:ᵒ G) -dec-subtypingᶠ : ∀ {F G} → FunType F → FunType G → Either (F ≮: G) (F <: G) -dec-subtypingᶠⁿ : ∀ {F U} → FunType F → Normal U → Either (F ≮: U) (F <: U) -dec-subtypingⁿ : ∀ {T U} → Normal T → Normal U → Either (T ≮: U) (T <: U) -dec-subtyping : ∀ T U → Either (T ≮: U) (T <: U) - -dec-subtypingˢⁿ T U with dec-language _ (scalar T) -dec-subtypingˢⁿ T U | Left p = Left (witness (scalar T) (scalar T) p) -dec-subtypingˢⁿ T U | Right p = Right (scalar-<: T p) - -dec-subtypingˢᶠ {F} {S ⇒ T} Fᶠ (defn sat-∩ sat-∪) (Sⁿ ⇒ Tⁿ) = result (top Fᶠ (λ o → o)) where - - data Top G : Set where - - defn : ∀ Sᵗ Tᵗ → - - Overloads F (Sᵗ ⇒ Tᵗ) → - (∀ {S′ T′} → Overloads G (S′ ⇒ T′) → (S′ <: Sᵗ)) → - ------------- - Top G - - top : ∀ {G} → (FunType G) → (G ⊆ᵒ F) → Top G - top {S′ ⇒ T′} _ G⊆F = defn S′ T′ (G⊆F here) (λ { here → <:-refl }) - top (Gᶠ ∩ Hᶠ) G⊆F with top Gᶠ (G⊆F ∘ left) | top Hᶠ (G⊆F ∘ right) - top (Gᶠ ∩ Hᶠ) G⊆F | defn Rᵗ Sᵗ p p₁ | defn Tᵗ Uᵗ q q₁ with sat-∪ p q - top (Gᶠ ∩ Hᶠ) G⊆F | defn Rᵗ Sᵗ p p₁ | defn Tᵗ Uᵗ q q₁ | defn n r r₁ = defn _ _ n - (λ { (left o) → <:-trans (<:-trans (p₁ o) <:-∪-left) r ; (right o) → <:-trans (<:-trans (q₁ o) <:-∪-right) r }) - - result : Top F → Either (F ≮: (S ⇒ T)) (F <:ᵒ (S ⇒ T)) - result (defn Sᵗ Tᵗ oᵗ srcᵗ) with dec-subtypingⁿ Sⁿ (normal-overload-src Fᶠ oᵗ) - result (defn Sᵗ Tᵗ oᵗ srcᵗ) | Left (witness s Ss ¬Sᵗs) = Left (witness (function-err s) (ov-language Fᶠ (λ o → function-err (<:-impl-⊇ (srcᵗ o) s ¬Sᵗs))) (function-err Ss)) - result (defn Sᵗ Tᵗ oᵗ srcᵗ) | Right S<:Sᵗ = result₀ (largest Fᶠ (λ o → o)) where - - data LargestSrc (G : Type) : Set where - - yes : ∀ S₀ T₀ → - - Overloads F (S₀ ⇒ T₀) → - T₀ <: T → - (∀ {S′ T′} → Overloads G (S′ ⇒ T′) → T′ <: T → (S′ <: S₀)) → - ----------------------- - LargestSrc G - - no : ∀ S₀ T₀ → - - Overloads F (S₀ ⇒ T₀) → - T₀ ≮: T → - (∀ {S′ T′} → Overloads G (S′ ⇒ T′) → T₀ <: T′) → - ----------------------- - LargestSrc G - - largest : ∀ {G} → (FunType G) → (G ⊆ᵒ F) → LargestSrc G - largest {S′ ⇒ T′} (S′ⁿ ⇒ T′ⁿ) G⊆F with dec-subtypingⁿ T′ⁿ Tⁿ - largest {S′ ⇒ T′} (S′ⁿ ⇒ T′ⁿ) G⊆F | Left T′≮:T = no S′ T′ (G⊆F here) T′≮:T λ { here → <:-refl } - largest {S′ ⇒ T′} (S′ⁿ ⇒ T′ⁿ) G⊆F | Right T′<:T = yes S′ T′ (G⊆F here) T′<:T (λ { here _ → <:-refl }) - largest (Gᶠ ∩ Hᶠ) GH⊆F with largest Gᶠ (GH⊆F ∘ left) | largest Hᶠ (GH⊆F ∘ right) - largest (Gᶠ ∩ Hᶠ) GH⊆F | no S₁ T₁ o₁ T₁≮:T tgt₁ | no S₂ T₂ o₂ T₂≮:T tgt₂ with sat-∩ o₁ o₂ - largest (Gᶠ ∩ Hᶠ) GH⊆F | no S₁ T₁ o₁ T₁≮:T tgt₁ | no S₂ T₂ o₂ T₂≮:T tgt₂ | defn o src tgt with dec-subtypingⁿ (normal-overload-tgt Fᶠ o) Tⁿ - largest (Gᶠ ∩ Hᶠ) GH⊆F | no S₁ T₁ o₁ T₁≮:T tgt₁ | no S₂ T₂ o₂ T₂≮:T tgt₂ | defn o src tgt | Left T₀≮:T = no _ _ o T₀≮:T (λ { (left o) → <:-trans tgt (<:-trans <:-∩-left (tgt₁ o)) ; (right o) → <:-trans tgt (<:-trans <:-∩-right (tgt₂ o)) }) - largest (Gᶠ ∩ Hᶠ) GH⊆F | no S₁ T₁ o₁ T₁≮:T tgt₁ | no S₂ T₂ o₂ T₂≮:T tgt₂ | defn o src tgt | Right T₀<:T = yes _ _ o T₀<:T (λ { (left o) p → CONTRADICTION (<:-impl-¬≮: p (<:-trans-≮: (tgt₁ o) T₁≮:T)) ; (right o) p → CONTRADICTION (<:-impl-¬≮: p (<:-trans-≮: (tgt₂ o) T₂≮:T)) }) - largest (Gᶠ ∩ Hᶠ) GH⊆F | no S₁ T₁ o₁ T₁≮:T tgt₁ | yes S₂ T₂ o₂ T₂<:T src₂ = yes S₂ T₂ o₂ T₂<:T (λ { (left o) p → CONTRADICTION (<:-impl-¬≮: p (<:-trans-≮: (tgt₁ o) T₁≮:T)) ; (right o) p → src₂ o p }) - largest (Gᶠ ∩ Hᶠ) GH⊆F | yes S₁ T₁ o₁ T₁<:T src₁ | no S₂ T₂ o₂ T₂≮:T tgt₂ = yes S₁ T₁ o₁ T₁<:T (λ { (left o) p → src₁ o p ; (right o) p → CONTRADICTION (<:-impl-¬≮: p (<:-trans-≮: (tgt₂ o) T₂≮:T)) }) - largest (Gᶠ ∩ Hᶠ) GH⊆F | yes S₁ T₁ o₁ T₁<:T src₁ | yes S₂ T₂ o₂ T₂<:T src₂ with sat-∪ o₁ o₂ - largest (Gᶠ ∩ Hᶠ) GH⊆F | yes S₁ T₁ o₁ T₁<:T src₁ | yes S₂ T₂ o₂ T₂<:T src₂ | defn o src tgt = yes _ _ o (<:-trans tgt (<:-∪-lub T₁<:T T₂<:T)) - (λ { (left o) T′<:T → <:-trans (src₁ o T′<:T) (<:-trans <:-∪-left src) - ; (right o) T′<:T → <:-trans (src₂ o T′<:T) (<:-trans <:-∪-right src) - }) - - result₀ : LargestSrc F → Either (F ≮: (S ⇒ T)) (F <:ᵒ (S ⇒ T)) - result₀ (no S₀ T₀ o₀ (witness t T₀t ¬Tt) tgt₀) = Left (witness (function-tgt t) (ov-language Fᶠ (λ o → function-tgt (tgt₀ o t T₀t))) (function-tgt ¬Tt)) - result₀ (yes S₀ T₀ o₀ T₀<:T src₀) with dec-subtypingⁿ Sⁿ (normal-overload-src Fᶠ o₀) - result₀ (yes S₀ T₀ o₀ T₀<:T src₀) | Right S<:S₀ = Right λ { here → defn o₀ S<:S₀ T₀<:T } - result₀ (yes S₀ T₀ o₀ T₀<:T src₀) | Left (witness s Ss ¬S₀s) = Left (result₁ (smallest Fᶠ (λ o → o))) where - - data SmallestTgt (G : Type) : Set where - - defn : ∀ S₁ T₁ → - - Overloads F (S₁ ⇒ T₁) → - Language S₁ s → - (∀ {S′ T′} → Overloads G (S′ ⇒ T′) → Language S′ s → (T₁ <: T′)) → - ----------------------- - SmallestTgt G - - smallest : ∀ {G} → (FunType G) → (G ⊆ᵒ F) → SmallestTgt G - smallest {S′ ⇒ T′} _ G⊆F with dec-language S′ s - smallest {S′ ⇒ T′} _ G⊆F | Left ¬S′s = defn Sᵗ Tᵗ oᵗ (S<:Sᵗ s Ss) λ { here S′s → CONTRADICTION (language-comp s ¬S′s S′s) } - smallest {S′ ⇒ T′} _ G⊆F | Right S′s = defn S′ T′ (G⊆F here) S′s (λ { here _ → <:-refl }) - smallest (Gᶠ ∩ Hᶠ) GH⊆F with smallest Gᶠ (GH⊆F ∘ left) | smallest Hᶠ (GH⊆F ∘ right) - smallest (Gᶠ ∩ Hᶠ) GH⊆F | defn S₁ T₁ o₁ R₁s tgt₁ | defn S₂ T₂ o₂ R₂s tgt₂ with sat-∩ o₁ o₂ - smallest (Gᶠ ∩ Hᶠ) GH⊆F | defn S₁ T₁ o₁ R₁s tgt₁ | defn S₂ T₂ o₂ R₂s tgt₂ | defn o src tgt = defn _ _ o (src s (R₁s , R₂s)) - (λ { (left o) S′s → <:-trans (<:-trans tgt <:-∩-left) (tgt₁ o S′s) - ; (right o) S′s → <:-trans (<:-trans tgt <:-∩-right) (tgt₂ o S′s) - }) - - result₁ : SmallestTgt F → (F ≮: (S ⇒ T)) - result₁ (defn S₁ T₁ o₁ S₁s tgt₁) with dec-subtypingⁿ (normal-overload-tgt Fᶠ o₁) Tⁿ - result₁ (defn S₁ T₁ o₁ S₁s tgt₁) | Right T₁<:T = CONTRADICTION (language-comp s ¬S₀s (src₀ o₁ T₁<:T s S₁s)) - result₁ (defn S₁ T₁ o₁ S₁s tgt₁) | Left (witness t T₁t ¬Tt) = witness (function-ok s t) (ov-language Fᶠ lemma) (function-ok Ss ¬Tt) where - - lemma : ∀ {S′ T′} → Overloads F (S′ ⇒ T′) → Language (S′ ⇒ T′) (function-ok s t) - lemma {S′} o with dec-language S′ s - lemma {S′} o | Left ¬S′s = function-ok₁ ¬S′s - lemma {S′} o | Right S′s = function-ok₂ (tgt₁ o S′s t T₁t) - -dec-subtypingˢᶠ F Fˢ (G ∩ H) with dec-subtypingˢᶠ F Fˢ G | dec-subtypingˢᶠ F Fˢ H -dec-subtypingˢᶠ F Fˢ (G ∩ H) | Left F≮:G | _ = Left (≮:-∩-left F≮:G) -dec-subtypingˢᶠ F Fˢ (G ∩ H) | _ | Left F≮:H = Left (≮:-∩-right F≮:H) -dec-subtypingˢᶠ F Fˢ (G ∩ H) | Right F<:G | Right F<:H = Right (λ { (left o) → F<:G o ; (right o) → F<:H o }) - -dec-subtypingᶠ F G with dec-subtypingˢᶠ (normal-saturate F) (saturated F) G -dec-subtypingᶠ F G | Left H≮:G = Left (<:-trans-≮: (saturate-<: F) H≮:G) -dec-subtypingᶠ F G | Right H<:G = Right (<:-trans (<:-saturate F) (<:ᵒ-impl-<: (normal-saturate F) G H<:G)) - -dec-subtypingᶠⁿ T never = Left (witness function (fun-function T) never) -dec-subtypingᶠⁿ T unknown = Right <:-unknown -dec-subtypingᶠⁿ T (U ⇒ V) = dec-subtypingᶠ T (U ⇒ V) -dec-subtypingᶠⁿ T (U ∩ V) = dec-subtypingᶠ T (U ∩ V) -dec-subtypingᶠⁿ T (U ∪ V) with dec-subtypingᶠⁿ T U -dec-subtypingᶠⁿ T (U ∪ V) | Left (witness t p q) = Left (witness t p (q , fun-¬scalar V T p)) -dec-subtypingᶠⁿ T (U ∪ V) | Right p = Right (<:-trans p <:-∪-left) - -dec-subtypingⁿ never U = Right <:-never -dec-subtypingⁿ unknown unknown = Right <:-refl -dec-subtypingⁿ unknown U with dec-subtypingᶠⁿ (never ⇒ unknown) U -dec-subtypingⁿ unknown U | Left p = Left (<:-trans-≮: <:-unknown p) -dec-subtypingⁿ unknown U | Right p₁ with dec-subtypingˢⁿ number U -dec-subtypingⁿ unknown U | Right p₁ | Left p = Left (<:-trans-≮: <:-unknown p) -dec-subtypingⁿ unknown U | Right p₁ | Right p₂ with dec-subtypingˢⁿ string U -dec-subtypingⁿ unknown U | Right p₁ | Right p₂ | Left p = Left (<:-trans-≮: <:-unknown p) -dec-subtypingⁿ unknown U | Right p₁ | Right p₂ | Right p₃ with dec-subtypingˢⁿ nil U -dec-subtypingⁿ unknown U | Right p₁ | Right p₂ | Right p₃ | Left p = Left (<:-trans-≮: <:-unknown p) -dec-subtypingⁿ unknown U | Right p₁ | Right p₂ | Right p₃ | Right p₄ with dec-subtypingˢⁿ boolean U -dec-subtypingⁿ unknown U | Right p₁ | Right p₂ | Right p₃ | Right p₄ | Left p = Left (<:-trans-≮: <:-unknown p) -dec-subtypingⁿ unknown U | Right p₁ | Right p₂ | Right p₃ | Right p₄ | Right p₅ = Right (<:-trans <:-everything (<:-∪-lub p₁ (<:-∪-lub p₂ (<:-∪-lub p₃ (<:-∪-lub p₄ p₅))))) -dec-subtypingⁿ (S ⇒ T) U = dec-subtypingᶠⁿ (S ⇒ T) U -dec-subtypingⁿ (S ∩ T) U = dec-subtypingᶠⁿ (S ∩ T) U -dec-subtypingⁿ (S ∪ T) U with dec-subtypingⁿ S U | dec-subtypingˢⁿ T U -dec-subtypingⁿ (S ∪ T) U | Left p | q = Left (≮:-∪-left p) -dec-subtypingⁿ (S ∪ T) U | Right p | Left q = Left (≮:-∪-right q) -dec-subtypingⁿ (S ∪ T) U | Right p | Right q = Right (<:-∪-lub p q) - -dec-subtyping T U with dec-subtypingⁿ (normal T) (normal U) -dec-subtyping T U | Left p = Left (<:-trans-≮: (normalize-<: T) (≮:-trans-<: p (<:-normalize U))) -dec-subtyping T U | Right p = Right (<:-trans (<:-normalize T) (<:-trans p (normalize-<: U))) - --- As a corollary, for saturated functions --- <:ᵒ coincides with <:, that is F is a subtype of (S ⇒ T) precisely --- when one of its overloads is. - -<:-impl-<:ᵒ : ∀ {F G} → FunType F → Saturated F → FunType G → (F <: G) → (F <:ᵒ G) -<:-impl-<:ᵒ {F} {G} Fᶠ Fˢ Gᶠ F<:G with dec-subtypingˢᶠ Fᶠ Fˢ Gᶠ -<:-impl-<:ᵒ {F} {G} Fᶠ Fˢ Gᶠ F<:G | Left F≮:G = CONTRADICTION (<:-impl-¬≮: F<:G F≮:G) -<:-impl-<:ᵒ {F} {G} Fᶠ Fˢ Gᶠ F<:G | Right F<:ᵒG = F<:ᵒG diff --git a/prototyping/Properties/Equality.agda b/prototyping/Properties/Equality.agda deleted file mode 100644 index c027bee3..00000000 --- a/prototyping/Properties/Equality.agda +++ /dev/null @@ -1,23 +0,0 @@ -module Properties.Equality where - -open import Agda.Builtin.Equality using (_≡_; refl) -open import Properties.Contradiction using (¬) - -sym : ∀ {A : Set} {a b : A} → (a ≡ b) → (b ≡ a) -sym refl = refl - -trans : ∀ {A : Set} {a b c : A} → (a ≡ b) → (b ≡ c) → (a ≡ c) -trans refl refl = refl - -cong : ∀ {A B : Set} {a b : A} (f : A → B) → (a ≡ b) → (f a ≡ f b) -cong f refl = refl - -subst₁ : ∀ {A : Set} {a b : A} (F : A → Set) → (a ≡ b) → (F a) → (F b) -subst₁ F refl x = x - -subst₂ : ∀ {A B : Set} {a b : A} {c d : B} (F : A → B → Set) → (a ≡ b) → (c ≡ d) → (F a c) → (F b d) -subst₂ F refl refl x = x - -_≢_ : ∀ {A : Set} → A → A → Set -(a ≢ b) = ¬(a ≡ b) - diff --git a/prototyping/Properties/Functions.agda b/prototyping/Properties/Functions.agda deleted file mode 100644 index 313b0ff2..00000000 --- a/prototyping/Properties/Functions.agda +++ /dev/null @@ -1,6 +0,0 @@ -module Properties.Functions where - -infixr 5 _∘_ - -_∘_ : ∀ {A B C : Set} → (B → C) → (A → B) → (A → C) -(f ∘ g) x = f (g x) diff --git a/prototyping/Properties/Product.agda b/prototyping/Properties/Product.agda deleted file mode 100644 index 0d41c817..00000000 --- a/prototyping/Properties/Product.agda +++ /dev/null @@ -1,14 +0,0 @@ -module Properties.Product where - -infixr 5 _×_ _,_ - -record Σ {A : Set} (B : A → Set) : Set where - - constructor _,_ - field fst : A - field snd : B fst - -open Σ public - -_×_ : Set → Set → Set -A × B = Σ (λ (a : A) → B) diff --git a/prototyping/Properties/Remember.agda b/prototyping/Properties/Remember.agda deleted file mode 100644 index 5058d594..00000000 --- a/prototyping/Properties/Remember.agda +++ /dev/null @@ -1,9 +0,0 @@ -module Properties.Remember where - -open import Agda.Builtin.Equality using (_≡_; refl) - -data Remember {A : Set} (a : A) : Set where - _,_ : ∀ b → (a ≡ b) → Remember(a) - -remember : ∀ {A} (a : A) → Remember(a) -remember a = (a , refl) diff --git a/prototyping/Properties/ResolveOverloads.agda b/prototyping/Properties/ResolveOverloads.agda deleted file mode 100644 index 8de4a875..00000000 --- a/prototyping/Properties/ResolveOverloads.agda +++ /dev/null @@ -1,189 +0,0 @@ -{-# OPTIONS --rewriting #-} - -module Properties.ResolveOverloads where - -open import FFI.Data.Either using (Left; Right) -open import Luau.ResolveOverloads using (Resolved; src; srcⁿ; resolve; resolveⁿ; resolveᶠ; resolveˢ; target; yes; no) -open import Luau.Subtyping using (_<:_; _≮:_; Language; ¬Language; witness; scalar; unknown; never; function; function-ok; function-err; function-tgt; function-scalar; function-ok₁; function-ok₂; scalar-scalar; scalar-function; scalar-function-ok; scalar-function-err; scalar-function-tgt; _,_; left; right) -open import Luau.Type using (Type ; Scalar; _⇒_; _∩_; _∪_; nil; boolean; number; string; unknown; never) -open import Luau.TypeSaturation using (saturate) -open import Luau.TypeNormalization using (normalize) -open import Properties.Contradiction using (CONTRADICTION) -open import Properties.DecSubtyping using (dec-subtyping; dec-subtypingⁿ; <:-impl-<:ᵒ) -open import Properties.Functions using (_∘_) -open import Properties.Subtyping using (<:-refl; <:-trans; <:-trans-≮:; ≮:-trans-<:; <:-∩-left; <:-∩-right; <:-∩-glb; <:-impl-¬≮:; <:-unknown; <:-function; function-≮:-never; <:-never; unknown-≮:-function; scalar-≮:-function; ≮:-∪-right; scalar-≮:-never; <:-∪-left; <:-∪-right; <:-impl-⊇; language-comp) -open import Properties.TypeNormalization using (Normal; FunType; normal; _⇒_; _∩_; _∪_; never; unknown; <:-normalize; normalize-<:; fun-≮:-never; unknown-≮:-fun; scalar-≮:-fun) -open import Properties.TypeSaturation using (Overloads; Saturated; _⊆ᵒ_; _<:ᵒ_; normal-saturate; saturated; <:-saturate; saturate-<:; defn; here; left; right) - --- Properties of src -function-err-srcⁿ : ∀ {T t} → (FunType T) → (¬Language (srcⁿ T) t) → Language T (function-err t) -function-err-srcⁿ (S ⇒ T) p = function-err p -function-err-srcⁿ (S ∩ T) (p₁ , p₂) = (function-err-srcⁿ S p₁ , function-err-srcⁿ T p₂) - -¬function-err-srcᶠ : ∀ {T t} → (FunType T) → (Language (srcⁿ T) t) → ¬Language T (function-err t) -¬function-err-srcᶠ (S ⇒ T) p = function-err p -¬function-err-srcᶠ (S ∩ T) (left p) = left (¬function-err-srcᶠ S p) -¬function-err-srcᶠ (S ∩ T) (right p) = right (¬function-err-srcᶠ T p) - -¬function-err-srcⁿ : ∀ {T t} → (Normal T) → (Language (srcⁿ T) t) → ¬Language T (function-err t) -¬function-err-srcⁿ never p = never -¬function-err-srcⁿ unknown (scalar ()) -¬function-err-srcⁿ (S ⇒ T) p = function-err p -¬function-err-srcⁿ (S ∩ T) (left p) = left (¬function-err-srcᶠ S p) -¬function-err-srcⁿ (S ∩ T) (right p) = right (¬function-err-srcᶠ T p) -¬function-err-srcⁿ (S ∪ T) (scalar ()) - -¬function-err-src : ∀ {T t} → (Language (src T) t) → ¬Language T (function-err t) -¬function-err-src {T = S ⇒ T} p = function-err p -¬function-err-src {T = nil} p = scalar-function-err nil -¬function-err-src {T = never} p = never -¬function-err-src {T = unknown} (scalar ()) -¬function-err-src {T = boolean} p = scalar-function-err boolean -¬function-err-src {T = number} p = scalar-function-err number -¬function-err-src {T = string} p = scalar-function-err string -¬function-err-src {T = S ∪ T} p = <:-impl-⊇ (<:-normalize (S ∪ T)) _ (¬function-err-srcⁿ (normal (S ∪ T)) p) -¬function-err-src {T = S ∩ T} p = <:-impl-⊇ (<:-normalize (S ∩ T)) _ (¬function-err-srcⁿ (normal (S ∩ T)) p) - -src-¬function-errᶠ : ∀ {T t} → (FunType T) → Language T (function-err t) → (¬Language (srcⁿ T) t) -src-¬function-errᶠ (S ⇒ T) (function-err p) = p -src-¬function-errᶠ (S ∩ T) (p₁ , p₂) = (src-¬function-errᶠ S p₁ , src-¬function-errᶠ T p₂) - -src-¬function-errⁿ : ∀ {T t} → (Normal T) → Language T (function-err t) → (¬Language (srcⁿ T) t) -src-¬function-errⁿ unknown p = never -src-¬function-errⁿ (S ⇒ T) (function-err p) = p -src-¬function-errⁿ (S ∩ T) (p₁ , p₂) = (src-¬function-errᶠ S p₁ , src-¬function-errᶠ T p₂) -src-¬function-errⁿ (S ∪ T) p = never - -src-¬function-err : ∀ {T t} → Language T (function-err t) → (¬Language (src T) t) -src-¬function-err {T = S ⇒ T} (function-err p) = p -src-¬function-err {T = unknown} p = never -src-¬function-err {T = S ∪ T} p = src-¬function-errⁿ (normal (S ∪ T)) (<:-normalize (S ∪ T) _ p) -src-¬function-err {T = S ∩ T} p = src-¬function-errⁿ (normal (S ∩ T)) (<:-normalize (S ∩ T) _ p) - -fun-¬scalar : ∀ {S T} (s : Scalar S) → FunType T → ¬Language T (scalar s) -fun-¬scalar s (S ⇒ T) = function-scalar s -fun-¬scalar s (S ∩ T) = left (fun-¬scalar s S) - -¬fun-scalar : ∀ {S T t} (s : Scalar S) → FunType T → Language T t → ¬Language S t -¬fun-scalar s (S ⇒ T) function = scalar-function s -¬fun-scalar s (S ⇒ T) (function-ok₁ p) = scalar-function-ok s -¬fun-scalar s (S ⇒ T) (function-ok₂ p) = scalar-function-ok s -¬fun-scalar s (S ⇒ T) (function-err p) = scalar-function-err s -¬fun-scalar s (S ⇒ T) (function-tgt p) = scalar-function-tgt s -¬fun-scalar s (S ∩ T) (p₁ , p₂) = ¬fun-scalar s T p₂ - -fun-function : ∀ {T} → FunType T → Language T function -fun-function (S ⇒ T) = function -fun-function (S ∩ T) = (fun-function S , fun-function T) - -srcⁿ-¬scalar : ∀ {S T t} (s : Scalar S) → Normal T → Language T (scalar s) → (¬Language (srcⁿ T) t) -srcⁿ-¬scalar s never (scalar ()) -srcⁿ-¬scalar s unknown p = never -srcⁿ-¬scalar s (S ⇒ T) (scalar ()) -srcⁿ-¬scalar s (S ∩ T) (p₁ , p₂) = CONTRADICTION (language-comp (scalar s) (fun-¬scalar s S) p₁) -srcⁿ-¬scalar s (S ∪ T) p = never - -src-¬scalar : ∀ {S T t} (s : Scalar S) → Language T (scalar s) → (¬Language (src T) t) -src-¬scalar {T = nil} s p = never -src-¬scalar {T = T ⇒ U} s (scalar ()) -src-¬scalar {T = never} s (scalar ()) -src-¬scalar {T = unknown} s p = never -src-¬scalar {T = boolean} s p = never -src-¬scalar {T = number} s p = never -src-¬scalar {T = string} s p = never -src-¬scalar {T = T ∪ U} s p = srcⁿ-¬scalar s (normal (T ∪ U)) (<:-normalize (T ∪ U) (scalar s) p) -src-¬scalar {T = T ∩ U} s p = srcⁿ-¬scalar s (normal (T ∩ U)) (<:-normalize (T ∩ U) (scalar s) p) - -srcⁿ-unknown-≮: : ∀ {T U} → (Normal U) → (T ≮: srcⁿ U) → (U ≮: (T ⇒ unknown)) -srcⁿ-unknown-≮: never (witness t p q) = CONTRADICTION (language-comp t q unknown) -srcⁿ-unknown-≮: unknown (witness t p q) = witness (function-err t) unknown (function-err p) -srcⁿ-unknown-≮: (U ⇒ V) (witness t p q) = witness (function-err t) (function-err q) (function-err p) -srcⁿ-unknown-≮: (U ∩ V) (witness t p q) = witness (function-err t) (function-err-srcⁿ (U ∩ V) q) (function-err p) -srcⁿ-unknown-≮: (U ∪ V) (witness t p q) = witness (scalar V) (right (scalar V)) (function-scalar V) - -src-unknown-≮: : ∀ {T U} → (T ≮: src U) → (U ≮: (T ⇒ unknown)) -src-unknown-≮: {U = nil} (witness t p q) = witness (scalar nil) (scalar nil) (function-scalar nil) -src-unknown-≮: {U = T ⇒ U} (witness t p q) = witness (function-err t) (function-err q) (function-err p) -src-unknown-≮: {U = never} (witness t p q) = CONTRADICTION (language-comp t q unknown) -src-unknown-≮: {U = unknown} (witness t p q) = witness (function-err t) unknown (function-err p) -src-unknown-≮: {U = boolean} (witness t p q) = witness (scalar boolean) (scalar boolean) (function-scalar boolean) -src-unknown-≮: {U = number} (witness t p q) = witness (scalar number) (scalar number) (function-scalar number) -src-unknown-≮: {U = string} (witness t p q) = witness (scalar string) (scalar string) (function-scalar string) -src-unknown-≮: {U = T ∪ U} p = <:-trans-≮: (normalize-<: (T ∪ U)) (srcⁿ-unknown-≮: (normal (T ∪ U)) p) -src-unknown-≮: {U = T ∩ U} p = <:-trans-≮: (normalize-<: (T ∩ U)) (srcⁿ-unknown-≮: (normal (T ∩ U)) p) - -unknown-src-≮: : ∀ {S T U} → (U ≮: S) → (T ≮: (U ⇒ unknown)) → (U ≮: src T) -unknown-src-≮: (witness t x x₁) (witness (scalar s) p (function-scalar s)) = witness t x (src-¬scalar s p) -unknown-src-≮: r (witness (function-ok s .(scalar s₁)) p (function-ok x (scalar-scalar s₁ () x₂))) -unknown-src-≮: r (witness (function-ok s .function) p (function-ok x (scalar-function ()))) -unknown-src-≮: r (witness (function-ok s .(function-ok _ _)) p (function-ok x (scalar-function-ok ()))) -unknown-src-≮: r (witness (function-ok s .(function-err _)) p (function-ok x (scalar-function-err ()))) -unknown-src-≮: r (witness (function-err t) p (function-err q)) = witness t q (src-¬function-err p) -unknown-src-≮: r (witness (function-tgt t) p (function-tgt (scalar-function-tgt ()))) - --- Properties of resolve -resolveˢ-<:-⇒ : ∀ {F V U} → (FunType F) → (Saturated F) → (FunType (V ⇒ U)) → (r : Resolved F V) → (F <: (V ⇒ U)) → (target r <: U) -resolveˢ-<:-⇒ Fᶠ Fˢ V⇒Uᶠ r F<:V⇒U with <:-impl-<:ᵒ Fᶠ Fˢ V⇒Uᶠ F<:V⇒U here -resolveˢ-<:-⇒ Fᶠ Fˢ V⇒Uᶠ (yes Sʳ Tʳ oʳ V<:Sʳ tgtʳ) F<:V⇒U | defn o o₁ o₂ = <:-trans (tgtʳ o o₁) o₂ -resolveˢ-<:-⇒ Fᶠ Fˢ V⇒Uᶠ (no tgtʳ) F<:V⇒U | defn o o₁ o₂ = CONTRADICTION (<:-impl-¬≮: o₁ (tgtʳ o)) - -resolveⁿ-<:-⇒ : ∀ {F V U} → (Fⁿ : Normal F) → (Vⁿ : Normal V) → (Uⁿ : Normal U) → (F <: (V ⇒ U)) → (resolveⁿ Fⁿ Vⁿ <: U) -resolveⁿ-<:-⇒ (Sⁿ ⇒ Tⁿ) Vⁿ Uⁿ F<:V⇒U = resolveˢ-<:-⇒ (normal-saturate (Sⁿ ⇒ Tⁿ)) (saturated (Sⁿ ⇒ Tⁿ)) (Vⁿ ⇒ Uⁿ) (resolveˢ (normal-saturate (Sⁿ ⇒ Tⁿ)) (saturated (Sⁿ ⇒ Tⁿ)) Vⁿ (λ o → o)) F<:V⇒U -resolveⁿ-<:-⇒ (Fⁿ ∩ Gⁿ) Vⁿ Uⁿ F<:V⇒U = resolveˢ-<:-⇒ (normal-saturate (Fⁿ ∩ Gⁿ)) (saturated (Fⁿ ∩ Gⁿ)) (Vⁿ ⇒ Uⁿ) (resolveˢ (normal-saturate (Fⁿ ∩ Gⁿ)) (saturated (Fⁿ ∩ Gⁿ)) Vⁿ (λ o → o)) (<:-trans (saturate-<: (Fⁿ ∩ Gⁿ)) F<:V⇒U) -resolveⁿ-<:-⇒ (Sⁿ ∪ Tˢ) Vⁿ Uⁿ F<:V⇒U = CONTRADICTION (<:-impl-¬≮: F<:V⇒U (<:-trans-≮: <:-∪-right (scalar-≮:-function Tˢ))) -resolveⁿ-<:-⇒ never Vⁿ Uⁿ F<:V⇒U = <:-never -resolveⁿ-<:-⇒ unknown Vⁿ Uⁿ F<:V⇒U = CONTRADICTION (<:-impl-¬≮: F<:V⇒U unknown-≮:-function) - -resolve-<:-⇒ : ∀ {F V U} → (F <: (V ⇒ U)) → (resolve F V <: U) -resolve-<:-⇒ {F} {V} {U} F<:V⇒U = <:-trans (resolveⁿ-<:-⇒ (normal F) (normal V) (normal U) (<:-trans (normalize-<: F) (<:-trans F<:V⇒U (<:-normalize (V ⇒ U))))) (normalize-<: U) - -resolve-≮:-⇒ : ∀ {F V U} → (resolve F V ≮: U) → (F ≮: (V ⇒ U)) -resolve-≮:-⇒ {F} {V} {U} FV≮:U with dec-subtyping F (V ⇒ U) -resolve-≮:-⇒ {F} {V} {U} FV≮:U | Left F≮:V⇒U = F≮:V⇒U -resolve-≮:-⇒ {F} {V} {U} FV≮:U | Right F<:V⇒U = CONTRADICTION (<:-impl-¬≮: (resolve-<:-⇒ F<:V⇒U) FV≮:U) - -<:-resolveˢ-⇒ : ∀ {S T V} → (r : Resolved (S ⇒ T) V) → (V <: S) → T <: target r -<:-resolveˢ-⇒ (yes S T here _ _) V<:S = <:-refl -<:-resolveˢ-⇒ (no _) V<:S = <:-unknown - -<:-resolveⁿ-⇒ : ∀ {S T V} → (Sⁿ : Normal S) → (Tⁿ : Normal T) → (Vⁿ : Normal V) → (V <: S) → T <: resolveⁿ (Sⁿ ⇒ Tⁿ) Vⁿ -<:-resolveⁿ-⇒ Sⁿ Tⁿ Vⁿ V<:S = <:-resolveˢ-⇒ (resolveˢ (Sⁿ ⇒ Tⁿ) (saturated (Sⁿ ⇒ Tⁿ)) Vⁿ (λ o → o)) V<:S - -<:-resolve-⇒ : ∀ {S T V} → (V <: S) → T <: resolve (S ⇒ T) V -<:-resolve-⇒ {S} {T} {V} V<:S = <:-trans (<:-normalize T) (<:-resolveⁿ-⇒ (normal S) (normal T) (normal V) (<:-trans (normalize-<: V) (<:-trans V<:S (<:-normalize S)))) - -<:-resolveˢ : ∀ {F G V W} → (r : Resolved F V) → (s : Resolved G W) → (F <:ᵒ G) → (V <: W) → target r <: target s -<:-resolveˢ (yes Sʳ Tʳ oʳ V<:Sʳ tgtʳ) (yes Sˢ Tˢ oˢ W<:Sˢ tgtˢ) F<:G V<:W with F<:G oˢ -<:-resolveˢ (yes Sʳ Tʳ oʳ V<:Sʳ tgtʳ) (yes Sˢ Tˢ oˢ W<:Sˢ tgtˢ) F<:G V<:W | defn o o₁ o₂ = <:-trans (tgtʳ o (<:-trans (<:-trans V<:W W<:Sˢ) o₁)) o₂ -<:-resolveˢ (no r) (yes Sˢ Tˢ oˢ W<:Sˢ tgtˢ) F<:G V<:W with F<:G oˢ -<:-resolveˢ (no r) (yes Sˢ Tˢ oˢ W<:Sˢ tgtˢ) F<:G V<:W | defn o o₁ o₂ = CONTRADICTION (<:-impl-¬≮: (<:-trans V<:W (<:-trans W<:Sˢ o₁)) (r o)) -<:-resolveˢ r (no s) F<:G V<:W = <:-unknown - -<:-resolveᶠ : ∀ {F G V W} → (Fᶠ : FunType F) → (Gᶠ : FunType G) → (Vⁿ : Normal V) → (Wⁿ : Normal W) → (F <: G) → (V <: W) → resolveᶠ Fᶠ Vⁿ <: resolveᶠ Gᶠ Wⁿ -<:-resolveᶠ Fᶠ Gᶠ Vⁿ Wⁿ F<:G V<:W = <:-resolveˢ - (resolveˢ (normal-saturate Fᶠ) (saturated Fᶠ) Vⁿ (λ o → o)) - (resolveˢ (normal-saturate Gᶠ) (saturated Gᶠ) Wⁿ (λ o → o)) - (<:-impl-<:ᵒ (normal-saturate Fᶠ) (saturated Fᶠ) (normal-saturate Gᶠ) (<:-trans (saturate-<: Fᶠ) (<:-trans F<:G (<:-saturate Gᶠ)))) - V<:W - -<:-resolveⁿ : ∀ {F G V W} → (Fⁿ : Normal F) → (Gⁿ : Normal G) → (Vⁿ : Normal V) → (Wⁿ : Normal W) → (F <: G) → (V <: W) → resolveⁿ Fⁿ Vⁿ <: resolveⁿ Gⁿ Wⁿ -<:-resolveⁿ (Rⁿ ⇒ Sⁿ) (Tⁿ ⇒ Uⁿ) Vⁿ Wⁿ F<:G V<:W = <:-resolveᶠ (Rⁿ ⇒ Sⁿ) (Tⁿ ⇒ Uⁿ) Vⁿ Wⁿ F<:G V<:W -<:-resolveⁿ (Rⁿ ⇒ Sⁿ) (Gⁿ ∩ Hⁿ) Vⁿ Wⁿ F<:G V<:W = <:-resolveᶠ (Rⁿ ⇒ Sⁿ) (Gⁿ ∩ Hⁿ) Vⁿ Wⁿ F<:G V<:W -<:-resolveⁿ (Eⁿ ∩ Fⁿ) (Tⁿ ⇒ Uⁿ) Vⁿ Wⁿ F<:G V<:W = <:-resolveᶠ (Eⁿ ∩ Fⁿ) (Tⁿ ⇒ Uⁿ) Vⁿ Wⁿ F<:G V<:W -<:-resolveⁿ (Eⁿ ∩ Fⁿ) (Gⁿ ∩ Hⁿ) Vⁿ Wⁿ F<:G V<:W = <:-resolveᶠ (Eⁿ ∩ Fⁿ) (Gⁿ ∩ Hⁿ) Vⁿ Wⁿ F<:G V<:W -<:-resolveⁿ (Fⁿ ∪ Sˢ) (Tⁿ ⇒ Uⁿ) Vⁿ Wⁿ F<:G V<:W = CONTRADICTION (<:-impl-¬≮: F<:G (≮:-∪-right (scalar-≮:-function Sˢ))) -<:-resolveⁿ unknown (Tⁿ ⇒ Uⁿ) Vⁿ Wⁿ F<:G V<:W = CONTRADICTION (<:-impl-¬≮: F<:G unknown-≮:-function) -<:-resolveⁿ (Fⁿ ∪ Sˢ) (Gⁿ ∩ Hⁿ) Vⁿ Wⁿ F<:G V<:W = CONTRADICTION (<:-impl-¬≮: F<:G (≮:-∪-right (scalar-≮:-fun (Gⁿ ∩ Hⁿ) Sˢ))) -<:-resolveⁿ unknown (Gⁿ ∩ Hⁿ) Vⁿ Wⁿ F<:G V<:W = CONTRADICTION (<:-impl-¬≮: F<:G (unknown-≮:-fun (Gⁿ ∩ Hⁿ))) -<:-resolveⁿ (Rⁿ ⇒ Sⁿ) never Vⁿ Wⁿ F<:G V<:W = CONTRADICTION (<:-impl-¬≮: F<:G (fun-≮:-never (Rⁿ ⇒ Sⁿ))) -<:-resolveⁿ (Eⁿ ∩ Fⁿ) never Vⁿ Wⁿ F<:G V<:W = CONTRADICTION (<:-impl-¬≮: F<:G (fun-≮:-never (Eⁿ ∩ Fⁿ))) -<:-resolveⁿ (Fⁿ ∪ Sˢ) never Vⁿ Wⁿ F<:G V<:W = CONTRADICTION (<:-impl-¬≮: F<:G (≮:-∪-right (scalar-≮:-never Sˢ))) -<:-resolveⁿ unknown never Vⁿ Wⁿ F<:G V<:W = F<:G -<:-resolveⁿ never Gⁿ Vⁿ Wⁿ F<:G V<:W = <:-never -<:-resolveⁿ Fⁿ (Gⁿ ∪ Uˢ) Vⁿ Wⁿ F<:G V<:W = <:-unknown -<:-resolveⁿ Fⁿ unknown Vⁿ Wⁿ F<:G V<:W = <:-unknown - -<:-resolve : ∀ {F G V W} → (F <: G) → (V <: W) → resolve F V <: resolve G W -<:-resolve {F} {G} {V} {W} F<:G V<:W = <:-resolveⁿ (normal F) (normal G) (normal V) (normal W) - (<:-trans (normalize-<: F) (<:-trans F<:G (<:-normalize G))) - (<:-trans (normalize-<: V) (<:-trans V<:W (<:-normalize W))) diff --git a/prototyping/Properties/Step.agda b/prototyping/Properties/Step.agda deleted file mode 100644 index cadd1530..00000000 --- a/prototyping/Properties/Step.agda +++ /dev/null @@ -1,172 +0,0 @@ -{-# OPTIONS --rewriting #-} - -module Properties.Step where - -open import Agda.Builtin.Equality using (_≡_; refl) -open import Agda.Builtin.Float using (primFloatPlus; primFloatMinus; primFloatTimes; primFloatDiv; primFloatEquality; primFloatLess) -open import Agda.Builtin.Bool using (true; false) -open import Agda.Builtin.String using (primStringAppend) -open import FFI.Data.Maybe using (just; nothing) -open import Luau.Heap using (Heap; _[_]; alloc; ok; function_is_end) -open import Luau.Syntax using (Block; Expr; nil; var; val; addr; bool; function_is_end; block_is_end; _$_; local_←_; return; done; _∙_; name; fun; arg; number; binexp; +; -; *; /; <; >; <=; >=; ==; ~=; ··; string) -open import Luau.OpSem using (_⟦_⟧_⟶_; _⊢_⟶ᴱ_⊣_; _⊢_⟶ᴮ_⊣_; app₁ ; app₂ ; beta; function; block; return; done; local; subst; binOp₀; binOp₁; binOp₂; +; -; *; /; <; >; <=; >=; ==; ~=; ··; evalEqOp; evalNeqOp) -open import Luau.RuntimeError using (BinOpError; RuntimeErrorᴱ; RuntimeErrorᴮ; FunctionMismatch; BinOpMismatch₁; BinOpMismatch₂; UnboundVariable; SEGV; app₁; app₂; block; local; return; bin₁; bin₂; +; -; *; /; <; >; <=; >=; ··) -open import Luau.RuntimeType using (valueType; function; number) -open import Luau.Substitution using (_[_/_]ᴮ) -open import Properties.Remember using (remember; _,_) -open import Utility.Bool using (not; _or_) - -data BinOpStepResult v op w : Set where - step : ∀ x → (v ⟦ op ⟧ w ⟶ x) → BinOpStepResult v op w - error₁ : BinOpError op (valueType(v)) → BinOpStepResult v op w - error₂ : BinOpError op (valueType(w)) → BinOpStepResult v op w - -binOpStep : ∀ v op w → BinOpStepResult v op w -binOpStep nil + w = error₁ (+ (λ ())) -binOpStep (addr a) + w = error₁ (+ (λ ())) -binOpStep (number m) + nil = error₂ (+ (λ ())) -binOpStep (number m) + (addr a) = error₂ (+ (λ ())) -binOpStep (number m) + (number n) = step (number (primFloatPlus m n)) (+ m n) -binOpStep (number m) + (bool b) = error₂ (+ (λ ())) -binOpStep (number m) + (string x) = error₂ (+ (λ ())) -binOpStep (number m) - (string x) = error₂ (- (λ ())) -binOpStep (number m) * (string x) = error₂ (* (λ ())) -binOpStep (number m) / (string x) = error₂ (/ (λ ())) -binOpStep (number m) < (string x) = error₂ (< (λ ())) -binOpStep (number m) > (string x) = error₂ (> (λ ())) -binOpStep (number m) == (string x) = step (bool false) (== (number m) (string x)) -binOpStep (number m) ~= (string x) = step (bool true) (~= (number m) (string x)) -binOpStep (number m) <= (string x) = error₂ (<= (λ ())) -binOpStep (number m) >= (string x) = error₂ (>= (λ ())) -binOpStep (bool b) + w = error₁ (+ (λ ())) -binOpStep nil - w = error₁ (- (λ ())) -binOpStep (addr a) - w = error₁ (- (λ ())) -binOpStep (number x) - nil = error₂ (- (λ ())) -binOpStep (number x) - (addr a) = error₂ (- (λ ())) -binOpStep (number x) - (number n) = step (number (primFloatMinus x n)) (- x n) -binOpStep (number x) - (bool b) = error₂ (- (λ ())) -binOpStep (bool b) - w = error₁ (- (λ ())) -binOpStep nil * w = error₁ (* (λ ())) -binOpStep (addr a) * w = error₁ (* (λ ())) -binOpStep (number m) * nil = error₂ (* (λ ())) -binOpStep (number m) * (addr a) = error₂ (* (λ ())) -binOpStep (number m) * (number n) = step (number (primFloatDiv m n)) (* m n) -binOpStep (number m) * (bool b) = error₂ (* (λ ())) -binOpStep (bool b) * w = error₁ (* (λ ())) -binOpStep nil / w = error₁ (/ (λ ())) -binOpStep (addr a) / w = error₁ (/ (λ ())) -binOpStep (number m) / nil = error₂ (/ (λ ())) -binOpStep (number m) / (addr a) = error₂ (/ (λ ())) -binOpStep (number m) / (number n) = step (number (primFloatTimes m n)) (/ m n) -binOpStep (number m) / (bool b) = error₂ (/ (λ ())) -binOpStep (bool b) / w = error₁ (/ (λ ())) -binOpStep nil < w = error₁ (< (λ ())) -binOpStep (addr a) < w = error₁ (< (λ ())) -binOpStep (number m) < nil = error₂ (< (λ ())) -binOpStep (number m) < (addr a) = error₂ (< (λ ())) -binOpStep (number m) < (number n) = step (bool (primFloatLess m n)) (< m n) -binOpStep (number m) < (bool b) = error₂ (< (λ ())) -binOpStep (bool b) < w = error₁ (< (λ ())) -binOpStep nil > w = error₁ (> (λ ())) -binOpStep (addr a) > w = error₁ (> (λ ())) -binOpStep (number m) > nil = error₂ (> (λ ())) -binOpStep (number m) > (addr a) = error₂ (> (λ ())) -binOpStep (number m) > (number n) = step (bool (primFloatLess n m)) (> m n) -binOpStep (number m) > (bool b) = error₂ (> (λ ())) -binOpStep (bool b) > w = error₁ (> (λ ())) -binOpStep v == w = step (bool (evalEqOp v w)) (== v w) -binOpStep v ~= w = step (bool (evalNeqOp v w)) (~= v w) -binOpStep nil <= w = error₁ (<= (λ ())) -binOpStep (addr a) <= w = error₁ (<= (λ ())) -binOpStep (number m) <= nil = error₂ (<= (λ ())) -binOpStep (number m) <= (addr a) = error₂ (<= (λ ())) -binOpStep (number m) <= (number n) = step (bool (primFloatLess m n or primFloatEquality m n)) (<= m n) -binOpStep (number m) <= (bool b) = error₂ (<= (λ ())) -binOpStep (bool b) <= w = error₁ (<= (λ ())) -binOpStep nil >= w = error₁ (>= (λ ())) -binOpStep (addr a) >= w = error₁ (>= (λ ())) -binOpStep (number m) >= nil = error₂ (>= (λ ())) -binOpStep (number m) >= (addr a) = error₂ (>= (λ ())) -binOpStep (number m) >= (number n) = step (bool (primFloatLess n m or primFloatEquality m n)) (>= m n) -binOpStep (number m) >= (bool b) = error₂ (>= (λ ())) -binOpStep (bool b) >= w = error₁ (>= (λ ())) -binOpStep (string x) + w = error₁ (+ (λ ())) -binOpStep (string x) - w = error₁ (- (λ ())) -binOpStep (string x) * w = error₁ (* (λ ())) -binOpStep (string x) / w = error₁ (/ (λ ())) -binOpStep (string x) < w = error₁ (< (λ ())) -binOpStep (string x) > w = error₁ (> (λ ())) -binOpStep (string x) <= w = error₁ (<= (λ ())) -binOpStep (string x) >= w = error₁ (>= (λ ())) -binOpStep nil ·· y = error₁ (·· (λ ())) -binOpStep (addr x) ·· y = error₁ (BinOpError.·· (λ ())) -binOpStep (number x) ·· y = error₁ (BinOpError.·· (λ ())) -binOpStep (bool x) ·· y = error₁ (BinOpError.·· (λ ())) -binOpStep (string x) ·· nil = error₂ (·· (λ ())) -binOpStep (string x) ·· (addr y) = error₂ (·· (λ ())) -binOpStep (string x) ·· (number y) = error₂ (·· (λ ())) -binOpStep (string x) ·· (bool y) = error₂ (·· (λ ())) -binOpStep (string x) ·· (string y) = step (string (primStringAppend x y)) (·· x y) - -data StepResultᴮ {a} (H : Heap a) (B : Block a) : Set -data StepResultᴱ {a} (H : Heap a) (M : Expr a) : Set - -data StepResultᴮ H B where - step : ∀ H′ B′ → (H ⊢ B ⟶ᴮ B′ ⊣ H′) → StepResultᴮ H B - return : ∀ v {B′} → (B ≡ (return (val v) ∙ B′)) → StepResultᴮ H B - done : (B ≡ done) → StepResultᴮ H B - error : (RuntimeErrorᴮ H B) → StepResultᴮ H B - -data StepResultᴱ H M where - step : ∀ H′ M′ → (H ⊢ M ⟶ᴱ M′ ⊣ H′) → StepResultᴱ H M - value : ∀ V → (M ≡ val V) → StepResultᴱ H M - error : (RuntimeErrorᴱ H M) → StepResultᴱ H M - -stepᴱ : ∀ {a} H M → StepResultᴱ {a} H M -stepᴮ : ∀ {a} H B → StepResultᴮ {a} H B - -stepᴱ H (val v) = value v refl -stepᴱ H (var x) = error UnboundVariable -stepᴱ H (M $ N) with stepᴱ H M -stepᴱ H (M $ N) | step H′ M′ D = step H′ (M′ $ N) (app₁ D) -stepᴱ H (_ $ N) | value v refl with stepᴱ H N -stepᴱ H (_ $ N) | value v refl | step H′ N′ s = step H′ (val v $ N′) (app₂ v s) -stepᴱ H (_ $ _) | value (addr a) refl | value w refl with remember (H [ a ]) -stepᴱ H (_ $ _) | value (addr a) refl | value w refl | (nothing , p) = error (app₁ (SEGV p)) -stepᴱ H (_ $ _) | value (addr a) refl | value w refl | (just(function F is B end) , p) = step H (block (fun F) is B [ w / name (arg F) ]ᴮ end) (beta function F is B end w refl p) -stepᴱ H (_ $ _) | value nil refl | value w refl = error (FunctionMismatch nil w (λ ())) -stepᴱ H (_ $ _) | value (number m) refl | value w refl = error (FunctionMismatch (number m) w (λ ())) -stepᴱ H (_ $ _) | value (bool b) refl | value w refl = error (FunctionMismatch (bool b) w (λ ())) -stepᴱ H (_ $ _) | value (string x) refl | value w refl = error (FunctionMismatch (string x) w (λ ())) -stepᴱ H (M $ N) | value V p | error E = error (app₂ E) -stepᴱ H (M $ N) | error E = error (app₁ E) -stepᴱ H (block b is B end) with stepᴮ H B -stepᴱ H (block b is B end) | step H′ B′ D = step H′ (block b is B′ end) (block D) -stepᴱ H (block b is (return _ ∙ B′) end) | return v refl = step H (val v) (return v) -stepᴱ H (block b is done end) | done refl = step H (val nil) done -stepᴱ H (block b is B end) | error E = error (block E) -stepᴱ H (function F is C end) with alloc H (function F is C end) -stepᴱ H function F is C end | ok a H′ p = step H′ (val (addr a)) (function a p) -stepᴱ H (binexp M op N) with stepᴱ H M -stepᴱ H (binexp M op N) | step H′ M′ s = step H′ (binexp M′ op N) (binOp₁ s) -stepᴱ H (binexp M op N) | error E = error (bin₁ E) -stepᴱ H (binexp M op N) | value v refl with stepᴱ H N -stepᴱ H (binexp M op N) | value v refl | step H′ N′ s = step H′ (binexp (val v) op N′) (binOp₂ s) -stepᴱ H (binexp M op N) | value v refl | error E = error (bin₂ E) -stepᴱ H (binexp M op N) | value v refl | value w refl with binOpStep v op w -stepᴱ H (binexp M op N) | value v refl | value w refl | step x p = step H (val x) (binOp₀ p) -stepᴱ H (binexp M op N) | value v refl | value w refl | error₁ E = error (BinOpMismatch₁ v w E) -stepᴱ H (binexp M op N) | value v refl | value w refl | error₂ E = error (BinOpMismatch₂ v w E) - -stepᴮ H (function F is C end ∙ B) with alloc H (function F is C end) -stepᴮ H (function F is C end ∙ B) | ok a H′ p = step H′ (B [ addr a / name (fun F) ]ᴮ) (function a p) -stepᴮ H (local x ← M ∙ B) with stepᴱ H M -stepᴮ H (local x ← M ∙ B) | step H′ M′ D = step H′ (local x ← M′ ∙ B) (local D) -stepᴮ H (local x ← _ ∙ B) | value v refl = step H (B [ v / name x ]ᴮ) (subst v) -stepᴮ H (local x ← M ∙ B) | error E = error (local E) -stepᴮ H (return M ∙ B) with stepᴱ H M -stepᴮ H (return M ∙ B) | step H′ M′ D = step H′ (return M′ ∙ B) (return D) -stepᴮ H (return _ ∙ B) | value V refl = return V refl -stepᴮ H (return M ∙ B) | error E = error (return E) -stepᴮ H done = done refl - \ No newline at end of file diff --git a/prototyping/Properties/StrictMode.agda b/prototyping/Properties/StrictMode.agda deleted file mode 100644 index 948674b9..00000000 --- a/prototyping/Properties/StrictMode.agda +++ /dev/null @@ -1,385 +0,0 @@ -{-# OPTIONS --rewriting #-} - -module Properties.StrictMode where - -import Agda.Builtin.Equality.Rewrite -open import Agda.Builtin.Equality using (_≡_; refl) -open import FFI.Data.Either using (Either; Left; Right; mapL; mapR; mapLR; swapLR; cond) -open import FFI.Data.Maybe using (Maybe; just; nothing) -open import Luau.Heap using (Heap; Object; function_is_end; defn; alloc; ok; next; lookup-not-allocated) renaming (_≡_⊕_↦_ to _≡ᴴ_⊕_↦_; _[_] to _[_]ᴴ; ∅ to ∅ᴴ) -open import Luau.ResolveOverloads using (src; resolve) -open import Luau.StrictMode using (Warningᴱ; Warningᴮ; Warningᴼ; Warningᴴ; UnallocatedAddress; UnboundVariable; FunctionCallMismatch; app₁; app₂; BinOpMismatch₁; BinOpMismatch₂; bin₁; bin₂; BlockMismatch; block₁; return; LocalVarMismatch; local₁; local₂; FunctionDefnMismatch; function₁; function₂; heap; expr; block; addr) -open import Luau.Substitution using (_[_/_]ᴮ; _[_/_]ᴱ; _[_/_]ᴮunless_; var_[_/_]ᴱwhenever_) -open import Luau.Subtyping using (_<:_; _≮:_; witness; unknown; never; scalar; function; scalar-function; scalar-function-ok; scalar-function-err; scalar-scalar; function-scalar; function-ok; function-err; left; right; _,_; Tree; Language; ¬Language) -open import Luau.Syntax using (Expr; yes; var; val; var_∈_; _⟨_⟩∈_; _$_; addr; number; bool; string; binexp; nil; function_is_end; block_is_end; done; return; local_←_; _∙_; fun; arg; name; ==; ~=) -open import Luau.Type using (Type; nil; number; boolean; string; _⇒_; never; unknown; _∩_; _∪_; _≡ᵀ_; _≡ᴹᵀ_) -open import Luau.TypeCheck using (_⊢ᴮ_∈_; _⊢ᴱ_∈_; _⊢ᴴᴮ_▷_∈_; _⊢ᴴᴱ_▷_∈_; nil; var; addr; app; function; block; done; return; local; orUnknown; srcBinOp; tgtBinOp) -open import Luau.Var using (_≡ⱽ_) -open import Luau.Addr using (_≡ᴬ_) -open import Luau.VarCtxt using (VarCtxt; ∅; _⋒_; _↦_; _⊕_↦_; _⊝_; ⊕-lookup-miss; ⊕-swap; ⊕-over) renaming (_[_] to _[_]ⱽ) -open import Luau.VarCtxt using (VarCtxt; ∅) -open import Properties.Remember using (remember; _,_) -open import Properties.Equality using (_≢_; sym; cong; trans; subst₁) -open import Properties.Dec using (Dec; yes; no) -open import Properties.Contradiction using (CONTRADICTION; ¬) -open import Properties.Functions using (_∘_) -open import Properties.DecSubtyping using (dec-subtyping) -open import Properties.Subtyping using (unknown-≮:; ≡-trans-≮:; ≮:-trans-≡; ≮:-trans; ≮:-refl; scalar-≢-impl-≮:; function-≮:-scalar; scalar-≮:-function; function-≮:-never; unknown-≮:-scalar; scalar-≮:-never; unknown-≮:-never; <:-refl; <:-unknown; <:-impl-¬≮:) -open import Properties.ResolveOverloads using (src-unknown-≮:; unknown-src-≮:; <:-resolve; resolve-<:-⇒; <:-resolve-⇒) -open import Properties.Subtyping using (unknown-≮:; ≡-trans-≮:; ≮:-trans-≡; ≮:-trans; <:-trans-≮:; ≮:-refl; scalar-≢-impl-≮:; function-≮:-scalar; scalar-≮:-function; function-≮:-never; unknown-≮:-scalar; scalar-≮:-never; unknown-≮:-never; ≡-impl-<:; ≡-trans-<:; <:-trans-≡; ≮:-trans-<:; <:-trans) -open import Properties.TypeCheck using (typeOfᴼ; typeOfᴹᴼ; typeOfⱽ; typeOfᴱ; typeOfᴮ; typeCheckᴱ; typeCheckᴮ; typeCheckᴼ; typeCheckᴴ) -open import Luau.OpSem using (_⟦_⟧_⟶_; _⊢_⟶*_⊣_; _⊢_⟶ᴮ_⊣_; _⊢_⟶ᴱ_⊣_; app₁; app₂; function; beta; return; block; done; local; subst; binOp₀; binOp₁; binOp₂; refl; step; +; -; *; /; <; >; ==; ~=; <=; >=; ··) -open import Luau.RuntimeError using (BinOpError; RuntimeErrorᴱ; RuntimeErrorᴮ; FunctionMismatch; BinOpMismatch₁; BinOpMismatch₂; UnboundVariable; SEGV; app₁; app₂; bin₁; bin₂; block; local; return; +; -; *; /; <; >; <=; >=; ··) -open import Luau.RuntimeType using (RuntimeType; valueType; number; string; boolean; nil; function) - -data _⊑_ (H : Heap yes) : Heap yes → Set where - refl : (H ⊑ H) - snoc : ∀ {H′ a O} → (H′ ≡ᴴ H ⊕ a ↦ O) → (H ⊑ H′) - -rednᴱ⊑ : ∀ {H H′ M M′} → (H ⊢ M ⟶ᴱ M′ ⊣ H′) → (H ⊑ H′) -rednᴮ⊑ : ∀ {H H′ B B′} → (H ⊢ B ⟶ᴮ B′ ⊣ H′) → (H ⊑ H′) - -rednᴱ⊑ (function a p) = snoc p -rednᴱ⊑ (app₁ s) = rednᴱ⊑ s -rednᴱ⊑ (app₂ p s) = rednᴱ⊑ s -rednᴱ⊑ (beta O v p q) = refl -rednᴱ⊑ (block s) = rednᴮ⊑ s -rednᴱ⊑ (return v) = refl -rednᴱ⊑ done = refl -rednᴱ⊑ (binOp₀ p) = refl -rednᴱ⊑ (binOp₁ s) = rednᴱ⊑ s -rednᴱ⊑ (binOp₂ s) = rednᴱ⊑ s - -rednᴮ⊑ (local s) = rednᴱ⊑ s -rednᴮ⊑ (subst v) = refl -rednᴮ⊑ (function a p) = snoc p -rednᴮ⊑ (return s) = rednᴱ⊑ s - -data LookupResult (H : Heap yes) a V : Set where - just : (H [ a ]ᴴ ≡ just V) → LookupResult H a V - nothing : (H [ a ]ᴴ ≡ nothing) → LookupResult H a V - -lookup-⊑-nothing : ∀ {H H′} a → (H ⊑ H′) → (H′ [ a ]ᴴ ≡ nothing) → (H [ a ]ᴴ ≡ nothing) -lookup-⊑-nothing {H} a refl p = p -lookup-⊑-nothing {H} a (snoc defn) p with a ≡ᴬ next H -lookup-⊑-nothing {H} a (snoc defn) p | yes refl = refl -lookup-⊑-nothing {H} a (snoc o) p | no q = trans (lookup-not-allocated o q) p - -<:-heap-weakeningᴱ : ∀ Γ H M {H′} → (H ⊑ H′) → (typeOfᴱ H′ Γ M <: typeOfᴱ H Γ M) -<:-heap-weakeningᴱ Γ H (var x) h = <:-refl -<:-heap-weakeningᴱ Γ H (val nil) h = <:-refl -<:-heap-weakeningᴱ Γ H (val (addr a)) refl = <:-refl -<:-heap-weakeningᴱ Γ H (val (addr a)) (snoc {a = b} q) with a ≡ᴬ b -<:-heap-weakeningᴱ Γ H (val (addr a)) (snoc {a = a} defn) | yes refl = <:-unknown -<:-heap-weakeningᴱ Γ H (val (addr a)) (snoc {a = b} q) | no r = ≡-impl-<: (sym (cong orUnknown (cong typeOfᴹᴼ (lookup-not-allocated q r)))) -<:-heap-weakeningᴱ Γ H (val (number n)) h = <:-refl -<:-heap-weakeningᴱ Γ H (val (bool b)) h = <:-refl -<:-heap-weakeningᴱ Γ H (val (string s)) h = <:-refl -<:-heap-weakeningᴱ Γ H (M $ N) h = <:-resolve (<:-heap-weakeningᴱ Γ H M h) (<:-heap-weakeningᴱ Γ H N h) -<:-heap-weakeningᴱ Γ H (function f ⟨ var x ∈ S ⟩∈ T is B end) h = <:-refl -<:-heap-weakeningᴱ Γ H (block var b ∈ T is N end) h = <:-refl -<:-heap-weakeningᴱ Γ H (binexp M op N) h = <:-refl - -<:-heap-weakeningᴮ : ∀ Γ H B {H′} → (H ⊑ H′) → (typeOfᴮ H′ Γ B <: typeOfᴮ H Γ B) -<:-heap-weakeningᴮ Γ H (function f ⟨ var x ∈ T ⟩∈ U is C end ∙ B) h = <:-heap-weakeningᴮ (Γ ⊕ f ↦ (T ⇒ U)) H B h -<:-heap-weakeningᴮ Γ H (local var x ∈ T ← M ∙ B) h = <:-heap-weakeningᴮ (Γ ⊕ x ↦ T) H B h -<:-heap-weakeningᴮ Γ H (return M ∙ B) h = <:-heap-weakeningᴱ Γ H M h -<:-heap-weakeningᴮ Γ H done h = <:-refl - -≮:-heap-weakeningᴱ : ∀ Γ H M {H′ U} → (H ⊑ H′) → (typeOfᴱ H′ Γ M ≮: U) → (typeOfᴱ H Γ M ≮: U) -≮:-heap-weakeningᴱ Γ H M h p = <:-trans-≮: (<:-heap-weakeningᴱ Γ H M h) p - -≮:-heap-weakeningᴮ : ∀ Γ H B {H′ U} → (H ⊑ H′) → (typeOfᴮ H′ Γ B ≮: U) → (typeOfᴮ H Γ B ≮: U) -≮:-heap-weakeningᴮ Γ H B h p = <:-trans-≮: (<:-heap-weakeningᴮ Γ H B h) p - -binOpPreservation : ∀ H {op v w x} → (v ⟦ op ⟧ w ⟶ x) → (tgtBinOp op ≡ typeOfᴱ H ∅ (val x)) -binOpPreservation H (+ m n) = refl -binOpPreservation H (- m n) = refl -binOpPreservation H (/ m n) = refl -binOpPreservation H (* m n) = refl -binOpPreservation H (< m n) = refl -binOpPreservation H (> m n) = refl -binOpPreservation H (<= m n) = refl -binOpPreservation H (>= m n) = refl -binOpPreservation H (== v w) = refl -binOpPreservation H (~= v w) = refl -binOpPreservation H (·· v w) = refl - -<:-substitutivityᴱ : ∀ {Γ T} H M v x → (typeOfᴱ H ∅ (val v) <: T) → (typeOfᴱ H Γ (M [ v / x ]ᴱ) <: typeOfᴱ H (Γ ⊕ x ↦ T) M) -<:-substitutivityᴱ-whenever : ∀ {Γ T} H v x y (r : Dec(x ≡ y)) → (typeOfᴱ H ∅ (val v) <: T) → (typeOfᴱ H Γ (var y [ v / x ]ᴱwhenever r) <: typeOfᴱ H (Γ ⊕ x ↦ T) (var y)) -<:-substitutivityᴮ : ∀ {Γ T} H B v x → (typeOfᴱ H ∅ (val v) <: T) → (typeOfᴮ H Γ (B [ v / x ]ᴮ) <: typeOfᴮ H (Γ ⊕ x ↦ T) B) -<:-substitutivityᴮ-unless : ∀ {Γ T U} H B v x y (r : Dec(x ≡ y)) → (typeOfᴱ H ∅ (val v) <: T) → (typeOfᴮ H (Γ ⊕ y ↦ U) (B [ v / x ]ᴮunless r) <: typeOfᴮ H ((Γ ⊕ x ↦ T) ⊕ y ↦ U) B) -<:-substitutivityᴮ-unless-yes : ∀ {Γ Γ′} H B v x y (r : x ≡ y) → (Γ′ ≡ Γ) → (typeOfᴮ H Γ (B [ v / x ]ᴮunless yes r) <: typeOfᴮ H Γ′ B) -<:-substitutivityᴮ-unless-no : ∀ {Γ Γ′ T} H B v x y (r : x ≢ y) → (Γ′ ≡ Γ ⊕ x ↦ T) → (typeOfᴱ H ∅ (val v) <: T) → (typeOfᴮ H Γ (B [ v / x ]ᴮunless no r) <: typeOfᴮ H Γ′ B) - -<:-substitutivityᴱ H (var y) v x p = <:-substitutivityᴱ-whenever H v x y (x ≡ⱽ y) p -<:-substitutivityᴱ H (val w) v x p = <:-refl -<:-substitutivityᴱ H (binexp M op N) v x p = <:-refl -<:-substitutivityᴱ H (M $ N) v x p = <:-resolve (<:-substitutivityᴱ H M v x p) (<:-substitutivityᴱ H N v x p) -<:-substitutivityᴱ H (function f ⟨ var y ∈ T ⟩∈ U is B end) v x p = <:-refl -<:-substitutivityᴱ H (block var b ∈ T is B end) v x p = <:-refl -<:-substitutivityᴱ-whenever H v x x (yes refl) p = p -<:-substitutivityᴱ-whenever H v x y (no o) p = (≡-impl-<: (cong orUnknown (⊕-lookup-miss x y _ _ o))) - -<:-substitutivityᴮ H (function f ⟨ var y ∈ T ⟩∈ U is C end ∙ B) v x p = <:-substitutivityᴮ-unless H B v x f (x ≡ⱽ f) p -<:-substitutivityᴮ H (local var y ∈ T ← M ∙ B) v x p = <:-substitutivityᴮ-unless H B v x y (x ≡ⱽ y) p -<:-substitutivityᴮ H (return M ∙ B) v x p = <:-substitutivityᴱ H M v x p -<:-substitutivityᴮ H done v x p = <:-refl -<:-substitutivityᴮ-unless H B v x y (yes r) p = <:-substitutivityᴮ-unless-yes H B v x y r (⊕-over r) -<:-substitutivityᴮ-unless H B v x y (no r) p = <:-substitutivityᴮ-unless-no H B v x y r (⊕-swap r) p -<:-substitutivityᴮ-unless-yes H B v x y refl refl = <:-refl -<:-substitutivityᴮ-unless-no H B v x y r refl p = <:-substitutivityᴮ H B v x p - -≮:-substitutivityᴱ : ∀ {Γ T U} H M v x → (typeOfᴱ H Γ (M [ v / x ]ᴱ) ≮: U) → Either (typeOfᴱ H (Γ ⊕ x ↦ T) M ≮: U) (typeOfᴱ H ∅ (val v) ≮: T) -≮:-substitutivityᴱ {T = T} H M v x p with dec-subtyping (typeOfᴱ H ∅ (val v)) T -≮:-substitutivityᴱ H M v x p | Left q = Right q -≮:-substitutivityᴱ H M v x p | Right q = Left (<:-trans-≮: (<:-substitutivityᴱ H M v x q) p) - -≮:-substitutivityᴮ : ∀ {Γ T U} H B v x → (typeOfᴮ H Γ (B [ v / x ]ᴮ) ≮: U) → Either (typeOfᴮ H (Γ ⊕ x ↦ T) B ≮: U) (typeOfᴱ H ∅ (val v) ≮: T) -≮:-substitutivityᴮ {T = T} H M v x p with dec-subtyping (typeOfᴱ H ∅ (val v)) T -≮:-substitutivityᴮ H M v x p | Left q = Right q -≮:-substitutivityᴮ H M v x p | Right q = Left (<:-trans-≮: (<:-substitutivityᴮ H M v x q) p) - -≮:-substitutivityᴮ-unless : ∀ {Γ T U V} H B v x y (r : Dec(x ≡ y)) → (typeOfᴮ H (Γ ⊕ y ↦ U) (B [ v / x ]ᴮunless r) ≮: V) → Either (typeOfᴮ H ((Γ ⊕ x ↦ T) ⊕ y ↦ U) B ≮: V) (typeOfᴱ H ∅ (val v) ≮: T) -≮:-substitutivityᴮ-unless {T = T} H B v x y r p with dec-subtyping (typeOfᴱ H ∅ (val v)) T -≮:-substitutivityᴮ-unless H B v x y r p | Left q = Right q -≮:-substitutivityᴮ-unless H B v x y r p | Right q = Left (<:-trans-≮: (<:-substitutivityᴮ-unless H B v x y r q) p) - -<:-reductionᴱ : ∀ H M {H′ M′} → (H ⊢ M ⟶ᴱ M′ ⊣ H′) → Either (typeOfᴱ H′ ∅ M′ <: typeOfᴱ H ∅ M) (Warningᴱ H (typeCheckᴱ H ∅ M)) -<:-reductionᴮ : ∀ H B {H′ B′} → (H ⊢ B ⟶ᴮ B′ ⊣ H′) → Either (typeOfᴮ H′ ∅ B′ <: typeOfᴮ H ∅ B) (Warningᴮ H (typeCheckᴮ H ∅ B)) - -<:-reductionᴱ H (M $ N) (app₁ s) = mapLR (λ p → <:-resolve p (<:-heap-weakeningᴱ ∅ H N (rednᴱ⊑ s))) app₁ (<:-reductionᴱ H M s) -<:-reductionᴱ H (M $ N) (app₂ q s) = mapLR (λ p → <:-resolve (<:-heap-weakeningᴱ ∅ H M (rednᴱ⊑ s)) p) app₂ (<:-reductionᴱ H N s) -<:-reductionᴱ H (M $ N) (beta (function f ⟨ var y ∈ S ⟩∈ U is B end) v refl q) with dec-subtyping (typeOfᴱ H ∅ (val v)) S -<:-reductionᴱ H (M $ N) (beta (function f ⟨ var y ∈ S ⟩∈ U is B end) v refl q) | Left r = Right (FunctionCallMismatch (≮:-trans-≡ r (cong src (cong orUnknown (cong typeOfᴹᴼ (sym q)))))) -<:-reductionᴱ H (M $ N) (beta (function f ⟨ var y ∈ S ⟩∈ U is B end) v refl q) | Right r = Left (<:-trans-≡ (<:-resolve-⇒ r) (cong (λ F → resolve F (typeOfᴱ H ∅ N)) (cong orUnknown (cong typeOfᴹᴼ (sym q))))) -<:-reductionᴱ H (function f ⟨ var x ∈ T ⟩∈ U is B end) (function a defn) = Left <:-refl -<:-reductionᴱ H (block var b ∈ T is B end) (block s) = Left <:-refl -<:-reductionᴱ H (block var b ∈ T is return (val v) ∙ B end) (return v) with dec-subtyping (typeOfᴱ H ∅ (val v)) T -<:-reductionᴱ H (block var b ∈ T is return (val v) ∙ B end) (return v) | Left p = Right (BlockMismatch p) -<:-reductionᴱ H (block var b ∈ T is return (val v) ∙ B end) (return v) | Right p = Left p -<:-reductionᴱ H (block var b ∈ T is done end) done with dec-subtyping nil T -<:-reductionᴱ H (block var b ∈ T is done end) done | Left p = Right (BlockMismatch p) -<:-reductionᴱ H (block var b ∈ T is done end) done | Right p = Left p -<:-reductionᴱ H (binexp M op N) (binOp₀ s) = Left (≡-impl-<: (sym (binOpPreservation H s))) -<:-reductionᴱ H (binexp M op N) (binOp₁ s) = Left <:-refl -<:-reductionᴱ H (binexp M op N) (binOp₂ s) = Left <:-refl - -<:-reductionᴮ H (function f ⟨ var x ∈ T ⟩∈ U is C end ∙ B) (function a defn) = Left (<:-trans (<:-substitutivityᴮ _ B (addr a) f <:-refl) (<:-heap-weakeningᴮ (f ↦ (T ⇒ U)) H B (snoc defn))) -<:-reductionᴮ H (local var x ∈ T ← M ∙ B) (local s) = Left (<:-heap-weakeningᴮ (x ↦ T) H B (rednᴱ⊑ s)) -<:-reductionᴮ H (local var x ∈ T ← M ∙ B) (subst v) with dec-subtyping (typeOfᴱ H ∅ (val v)) T -<:-reductionᴮ H (local var x ∈ T ← M ∙ B) (subst v) | Left p = Right (LocalVarMismatch p) -<:-reductionᴮ H (local var x ∈ T ← M ∙ B) (subst v) | Right p = Left (<:-substitutivityᴮ H B v x p) -<:-reductionᴮ H (return M ∙ B) (return s) = mapR return (<:-reductionᴱ H M s) - -≮:-reductionᴱ : ∀ H M {H′ M′ T} → (H ⊢ M ⟶ᴱ M′ ⊣ H′) → (typeOfᴱ H′ ∅ M′ ≮: T) → Either (typeOfᴱ H ∅ M ≮: T) (Warningᴱ H (typeCheckᴱ H ∅ M)) -≮:-reductionᴱ H M s p = mapL (λ q → <:-trans-≮: q p) (<:-reductionᴱ H M s) - -≮:-reductionᴮ : ∀ H B {H′ B′ T} → (H ⊢ B ⟶ᴮ B′ ⊣ H′) → (typeOfᴮ H′ ∅ B′ ≮: T) → Either (typeOfᴮ H ∅ B ≮: T) (Warningᴮ H (typeCheckᴮ H ∅ B)) -≮:-reductionᴮ H B s p = mapL (λ q → <:-trans-≮: q p) (<:-reductionᴮ H B s) - -reflect-substitutionᴱ : ∀ {Γ T} H M v x → Warningᴱ H (typeCheckᴱ H Γ (M [ v / x ]ᴱ)) → Either (Warningᴱ H (typeCheckᴱ H (Γ ⊕ x ↦ T) M)) (Either (Warningᴱ H (typeCheckᴱ H ∅ (val v))) (typeOfᴱ H ∅ (val v) ≮: T)) -reflect-substitutionᴱ-whenever : ∀ {Γ T} H v x y (p : Dec(x ≡ y)) → Warningᴱ H (typeCheckᴱ H Γ (var y [ v / x ]ᴱwhenever p)) → Either (Warningᴱ H (typeCheckᴱ H (Γ ⊕ x ↦ T) (var y))) (Either (Warningᴱ H (typeCheckᴱ H ∅ (val v))) (typeOfᴱ H ∅ (val v) ≮: T)) -reflect-substitutionᴮ : ∀ {Γ T} H B v x → Warningᴮ H (typeCheckᴮ H Γ (B [ v / x ]ᴮ)) → Either (Warningᴮ H (typeCheckᴮ H (Γ ⊕ x ↦ T) B)) (Either (Warningᴱ H (typeCheckᴱ H ∅ (val v))) (typeOfᴱ H ∅ (val v) ≮: T)) -reflect-substitutionᴮ-unless : ∀ {Γ T U} H B v x y (r : Dec(x ≡ y)) → Warningᴮ H (typeCheckᴮ H (Γ ⊕ y ↦ U) (B [ v / x ]ᴮunless r)) → Either (Warningᴮ H (typeCheckᴮ H ((Γ ⊕ x ↦ T) ⊕ y ↦ U) B)) (Either (Warningᴱ H (typeCheckᴱ H ∅ (val v))) (typeOfᴱ H ∅ (val v) ≮: T)) -reflect-substitutionᴮ-unless-yes : ∀ {Γ Γ′ T} H B v x y (r : x ≡ y) → (Γ′ ≡ Γ) → Warningᴮ H (typeCheckᴮ H Γ (B [ v / x ]ᴮunless yes r)) → Either (Warningᴮ H (typeCheckᴮ H Γ′ B)) (Either (Warningᴱ H (typeCheckᴱ H ∅ (val v))) (typeOfᴱ H ∅ (val v) ≮: T)) -reflect-substitutionᴮ-unless-no : ∀ {Γ Γ′ T} H B v x y (r : x ≢ y) → (Γ′ ≡ Γ ⊕ x ↦ T) → Warningᴮ H (typeCheckᴮ H Γ (B [ v / x ]ᴮunless no r)) → Either (Warningᴮ H (typeCheckᴮ H Γ′ B)) (Either (Warningᴱ H (typeCheckᴱ H ∅ (val v))) (typeOfᴱ H ∅ (val v) ≮: T)) - -reflect-substitutionᴱ H (var y) v x W = reflect-substitutionᴱ-whenever H v x y (x ≡ⱽ y) W -reflect-substitutionᴱ H (val (addr a)) v x (UnallocatedAddress r) = Left (UnallocatedAddress r) -reflect-substitutionᴱ H (M $ N) v x (FunctionCallMismatch p) with ≮:-substitutivityᴱ H N v x p -reflect-substitutionᴱ H (M $ N) v x (FunctionCallMismatch p) | Right W = Right (Right W) -reflect-substitutionᴱ H (M $ N) v x (FunctionCallMismatch p) | Left q with ≮:-substitutivityᴱ H M v x (src-unknown-≮: q) -reflect-substitutionᴱ H (M $ N) v x (FunctionCallMismatch p) | Left q | Left r = Left ((FunctionCallMismatch ∘ unknown-src-≮: q) r) -reflect-substitutionᴱ H (M $ N) v x (FunctionCallMismatch p) | Left q | Right W = Right (Right W) -reflect-substitutionᴱ H (M $ N) v x (app₁ W) = mapL app₁ (reflect-substitutionᴱ H M v x W) -reflect-substitutionᴱ H (M $ N) v x (app₂ W) = mapL app₂ (reflect-substitutionᴱ H N v x W) -reflect-substitutionᴱ H (function f ⟨ var y ∈ T ⟩∈ U is B end) v x (FunctionDefnMismatch q) = mapLR FunctionDefnMismatch Right (≮:-substitutivityᴮ-unless H B v x y (x ≡ⱽ y) q) -reflect-substitutionᴱ H (function f ⟨ var y ∈ T ⟩∈ U is B end) v x (function₁ W) = mapL function₁ (reflect-substitutionᴮ-unless H B v x y (x ≡ⱽ y) W) -reflect-substitutionᴱ H (block var b ∈ T is B end) v x (BlockMismatch q) = mapLR BlockMismatch Right (≮:-substitutivityᴮ H B v x q) -reflect-substitutionᴱ H (block var b ∈ T is B end) v x (block₁ W′) = mapL block₁ (reflect-substitutionᴮ H B v x W′) -reflect-substitutionᴱ H (binexp M op N) v x (BinOpMismatch₁ q) = mapLR BinOpMismatch₁ Right (≮:-substitutivityᴱ H M v x q) -reflect-substitutionᴱ H (binexp M op N) v x (BinOpMismatch₂ q) = mapLR BinOpMismatch₂ Right (≮:-substitutivityᴱ H N v x q) -reflect-substitutionᴱ H (binexp M op N) v x (bin₁ W) = mapL bin₁ (reflect-substitutionᴱ H M v x W) -reflect-substitutionᴱ H (binexp M op N) v x (bin₂ W) = mapL bin₂ (reflect-substitutionᴱ H N v x W) - -reflect-substitutionᴱ-whenever H a x x (yes refl) (UnallocatedAddress p) = Right (Left (UnallocatedAddress p)) -reflect-substitutionᴱ-whenever H v x y (no p) (UnboundVariable q) = Left (UnboundVariable (trans (sym (⊕-lookup-miss x y _ _ p)) q)) - -reflect-substitutionᴮ H (function f ⟨ var y ∈ T ⟩∈ U is C end ∙ B) v x (FunctionDefnMismatch q) = mapLR FunctionDefnMismatch Right (≮:-substitutivityᴮ-unless H C v x y (x ≡ⱽ y) q) -reflect-substitutionᴮ H (function f ⟨ var y ∈ T ⟩∈ U is C end ∙ B) v x (function₁ W) = mapL function₁ (reflect-substitutionᴮ-unless H C v x y (x ≡ⱽ y) W) -reflect-substitutionᴮ H (function f ⟨ var y ∈ T ⟩∈ U is C end ∙ B) v x (function₂ W) = mapL function₂ (reflect-substitutionᴮ-unless H B v x f (x ≡ⱽ f) W) -reflect-substitutionᴮ H (local var y ∈ T ← M ∙ B) v x (LocalVarMismatch q) = mapLR LocalVarMismatch Right (≮:-substitutivityᴱ H M v x q) -reflect-substitutionᴮ H (local var y ∈ T ← M ∙ B) v x (local₁ W) = mapL local₁ (reflect-substitutionᴱ H M v x W) -reflect-substitutionᴮ H (local var y ∈ T ← M ∙ B) v x (local₂ W) = mapL local₂ (reflect-substitutionᴮ-unless H B v x y (x ≡ⱽ y) W) -reflect-substitutionᴮ H (return M ∙ B) v x (return W) = mapL return (reflect-substitutionᴱ H M v x W) - -reflect-substitutionᴮ-unless H B v x y (yes p) W = reflect-substitutionᴮ-unless-yes H B v x y p (⊕-over p) W -reflect-substitutionᴮ-unless H B v x y (no p) W = reflect-substitutionᴮ-unless-no H B v x y p (⊕-swap p) W -reflect-substitutionᴮ-unless-yes H B v x x refl refl W = Left W -reflect-substitutionᴮ-unless-no H B v x y p refl W = reflect-substitutionᴮ H B v x W - -reflect-weakeningᴱ : ∀ Γ H M {H′} → (H ⊑ H′) → Warningᴱ H′ (typeCheckᴱ H′ Γ M) → Warningᴱ H (typeCheckᴱ H Γ M) -reflect-weakeningᴮ : ∀ Γ H B {H′} → (H ⊑ H′) → Warningᴮ H′ (typeCheckᴮ H′ Γ B) → Warningᴮ H (typeCheckᴮ H Γ B) - -reflect-weakeningᴱ Γ H (var x) h (UnboundVariable p) = (UnboundVariable p) -reflect-weakeningᴱ Γ H (val (addr a)) h (UnallocatedAddress p) = UnallocatedAddress (lookup-⊑-nothing a h p) -reflect-weakeningᴱ Γ H (M $ N) h (FunctionCallMismatch p) = FunctionCallMismatch (≮:-heap-weakeningᴱ Γ H N h (unknown-src-≮: p (≮:-heap-weakeningᴱ Γ H M h (src-unknown-≮: p)))) -reflect-weakeningᴱ Γ H (M $ N) h (app₁ W) = app₁ (reflect-weakeningᴱ Γ H M h W) -reflect-weakeningᴱ Γ H (M $ N) h (app₂ W) = app₂ (reflect-weakeningᴱ Γ H N h W) -reflect-weakeningᴱ Γ H (binexp M op N) h (BinOpMismatch₁ p) = BinOpMismatch₁ (≮:-heap-weakeningᴱ Γ H M h p) -reflect-weakeningᴱ Γ H (binexp M op N) h (BinOpMismatch₂ p) = BinOpMismatch₂ (≮:-heap-weakeningᴱ Γ H N h p) -reflect-weakeningᴱ Γ H (binexp M op N) h (bin₁ W′) = bin₁ (reflect-weakeningᴱ Γ H M h W′) -reflect-weakeningᴱ Γ H (binexp M op N) h (bin₂ W′) = bin₂ (reflect-weakeningᴱ Γ H N h W′) -reflect-weakeningᴱ Γ H (function f ⟨ var y ∈ T ⟩∈ U is B end) h (FunctionDefnMismatch p) = FunctionDefnMismatch (≮:-heap-weakeningᴮ (Γ ⊕ y ↦ T) H B h p) -reflect-weakeningᴱ Γ H (function f ⟨ var y ∈ T ⟩∈ U is B end) h (function₁ W) = function₁ (reflect-weakeningᴮ (Γ ⊕ y ↦ T) H B h W) -reflect-weakeningᴱ Γ H (block var b ∈ T is B end) h (BlockMismatch p) = BlockMismatch (≮:-heap-weakeningᴮ Γ H B h p) -reflect-weakeningᴱ Γ H (block var b ∈ T is B end) h (block₁ W) = block₁ (reflect-weakeningᴮ Γ H B h W) - -reflect-weakeningᴮ Γ H (return M ∙ B) h (return W) = return (reflect-weakeningᴱ Γ H M h W) -reflect-weakeningᴮ Γ H (local var y ∈ T ← M ∙ B) h (LocalVarMismatch p) = LocalVarMismatch (≮:-heap-weakeningᴱ Γ H M h p) -reflect-weakeningᴮ Γ H (local var y ∈ T ← M ∙ B) h (local₁ W) = local₁ (reflect-weakeningᴱ Γ H M h W) -reflect-weakeningᴮ Γ H (local var y ∈ T ← M ∙ B) h (local₂ W) = local₂ (reflect-weakeningᴮ (Γ ⊕ y ↦ T) H B h W) -reflect-weakeningᴮ Γ H (function f ⟨ var x ∈ T ⟩∈ U is C end ∙ B) h (FunctionDefnMismatch p) = FunctionDefnMismatch (≮:-heap-weakeningᴮ (Γ ⊕ x ↦ T) H C h p) -reflect-weakeningᴮ Γ H (function f ⟨ var x ∈ T ⟩∈ U is C end ∙ B) h (function₁ W) = function₁ (reflect-weakeningᴮ (Γ ⊕ x ↦ T) H C h W) -reflect-weakeningᴮ Γ H (function f ⟨ var x ∈ T ⟩∈ U is C end ∙ B) h (function₂ W) = function₂ (reflect-weakeningᴮ (Γ ⊕ f ↦ (T ⇒ U)) H B h W) - -reflect-weakeningᴼ : ∀ H O {H′} → (H ⊑ H′) → Warningᴼ H′ (typeCheckᴼ H′ O) → Warningᴼ H (typeCheckᴼ H O) -reflect-weakeningᴼ H (just function f ⟨ var x ∈ T ⟩∈ U is B end) h (FunctionDefnMismatch p) = FunctionDefnMismatch (≮:-heap-weakeningᴮ (x ↦ T) H B h p) -reflect-weakeningᴼ H (just function f ⟨ var x ∈ T ⟩∈ U is B end) h (function₁ W) = function₁ (reflect-weakeningᴮ (x ↦ T) H B h W) - -reflectᴱ : ∀ H M {H′ M′} → (H ⊢ M ⟶ᴱ M′ ⊣ H′) → Warningᴱ H′ (typeCheckᴱ H′ ∅ M′) → Either (Warningᴱ H (typeCheckᴱ H ∅ M)) (Warningᴴ H (typeCheckᴴ H)) -reflectᴮ : ∀ H B {H′ B′} → (H ⊢ B ⟶ᴮ B′ ⊣ H′) → Warningᴮ H′ (typeCheckᴮ H′ ∅ B′) → Either (Warningᴮ H (typeCheckᴮ H ∅ B)) (Warningᴴ H (typeCheckᴴ H)) - -reflectᴱ H (M $ N) (app₁ s) (FunctionCallMismatch p) = cond (Left ∘ FunctionCallMismatch ∘ ≮:-heap-weakeningᴱ ∅ H N (rednᴱ⊑ s) ∘ unknown-src-≮: p) (Left ∘ app₁) (≮:-reductionᴱ H M s (src-unknown-≮: p)) -reflectᴱ H (M $ N) (app₁ s) (app₁ W′) = mapL app₁ (reflectᴱ H M s W′) -reflectᴱ H (M $ N) (app₁ s) (app₂ W′) = Left (app₂ (reflect-weakeningᴱ ∅ H N (rednᴱ⊑ s) W′)) -reflectᴱ H (M $ N) (app₂ p s) (FunctionCallMismatch q) = cond (λ r → Left (FunctionCallMismatch (unknown-src-≮: r (≮:-heap-weakeningᴱ ∅ H M (rednᴱ⊑ s) (src-unknown-≮: r))))) (Left ∘ app₂) (≮:-reductionᴱ H N s q) -reflectᴱ H (M $ N) (app₂ p s) (app₁ W′) = Left (app₁ (reflect-weakeningᴱ ∅ H M (rednᴱ⊑ s) W′)) -reflectᴱ H (M $ N) (app₂ p s) (app₂ W′) = mapL app₂ (reflectᴱ H N s W′) -reflectᴱ H (val (addr a) $ N) (beta (function f ⟨ var x ∈ T ⟩∈ U is B end) v refl p) (BlockMismatch q) with ≮:-substitutivityᴮ H B v x q -reflectᴱ H (val (addr a) $ N) (beta (function f ⟨ var x ∈ T ⟩∈ U is B end) v refl p) (BlockMismatch q) | Left r = Right (addr a p (FunctionDefnMismatch r)) -reflectᴱ H (val (addr a) $ N) (beta (function f ⟨ var x ∈ T ⟩∈ U is B end) v refl p) (BlockMismatch q) | Right r = Left (FunctionCallMismatch (≮:-trans-≡ r ((cong src (cong orUnknown (cong typeOfᴹᴼ (sym p))))))) -reflectᴱ H (val (addr a) $ N) (beta (function f ⟨ var x ∈ T ⟩∈ U is B end) v refl p) (block₁ W′) with reflect-substitutionᴮ _ B v x W′ -reflectᴱ H (val (addr a) $ N) (beta (function f ⟨ var x ∈ T ⟩∈ U is B end) v refl p) (block₁ W′) | Left W = Right (addr a p (function₁ W)) -reflectᴱ H (val (addr a) $ N) (beta (function f ⟨ var x ∈ T ⟩∈ U is B end) v refl p) (block₁ W′) | Right (Left W) = Left (app₂ W) -reflectᴱ H (val (addr a) $ N) (beta (function f ⟨ var x ∈ T ⟩∈ U is B end) v refl p) (block₁ W′) | Right (Right q) = Left (FunctionCallMismatch (≮:-trans-≡ q (cong src (cong orUnknown (cong typeOfᴹᴼ (sym p)))))) -reflectᴱ H (block var b ∈ T is B end) (block s) (BlockMismatch p) = Left (cond BlockMismatch block₁ (≮:-reductionᴮ H B s p)) -reflectᴱ H (block var b ∈ T is B end) (block s) (block₁ W′) = mapL block₁ (reflectᴮ H B s W′) -reflectᴱ H (block var b ∈ T is B end) (return v) W′ = Left (block₁ (return W′)) -reflectᴱ H (function f ⟨ var x ∈ T ⟩∈ U is B end) (function a defn) (UnallocatedAddress ()) -reflectᴱ H (binexp M op N) (binOp₀ ()) (UnallocatedAddress p) -reflectᴱ H (binexp M op N) (binOp₁ s) (BinOpMismatch₁ p) = Left (cond BinOpMismatch₁ bin₁ (≮:-reductionᴱ H M s p)) -reflectᴱ H (binexp M op N) (binOp₁ s) (BinOpMismatch₂ p) = Left (BinOpMismatch₂ (≮:-heap-weakeningᴱ ∅ H N (rednᴱ⊑ s) p)) -reflectᴱ H (binexp M op N) (binOp₁ s) (bin₁ W′) = mapL bin₁ (reflectᴱ H M s W′) -reflectᴱ H (binexp M op N) (binOp₁ s) (bin₂ W′) = Left (bin₂ (reflect-weakeningᴱ ∅ H N (rednᴱ⊑ s) W′)) -reflectᴱ H (binexp M op N) (binOp₂ s) (BinOpMismatch₁ p) = Left (BinOpMismatch₁ (≮:-heap-weakeningᴱ ∅ H M (rednᴱ⊑ s) p)) -reflectᴱ H (binexp M op N) (binOp₂ s) (BinOpMismatch₂ p) = Left (cond BinOpMismatch₂ bin₂ (≮:-reductionᴱ H N s p)) -reflectᴱ H (binexp M op N) (binOp₂ s) (bin₁ W′) = Left (bin₁ (reflect-weakeningᴱ ∅ H M (rednᴱ⊑ s) W′)) -reflectᴱ H (binexp M op N) (binOp₂ s) (bin₂ W′) = mapL bin₂ (reflectᴱ H N s W′) - -reflectᴮ H (local var x ∈ T ← M ∙ B) (local s) (LocalVarMismatch p) = Left (cond LocalVarMismatch local₁ (≮:-reductionᴱ H M s p)) -reflectᴮ H (local var x ∈ T ← M ∙ B) (local s) (local₁ W′) = mapL local₁ (reflectᴱ H M s W′) -reflectᴮ H (local var x ∈ T ← M ∙ B) (local s) (local₂ W′) = Left (local₂ (reflect-weakeningᴮ (x ↦ T) H B (rednᴱ⊑ s) W′)) -reflectᴮ H (local var x ∈ T ← M ∙ B) (subst v) W′ = Left (cond local₂ (cond local₁ LocalVarMismatch) (reflect-substitutionᴮ H B v x W′)) -reflectᴮ H (function f ⟨ var y ∈ T ⟩∈ U is C end ∙ B) (function a defn) W′ with reflect-substitutionᴮ _ B (addr a) f W′ -reflectᴮ H (function f ⟨ var y ∈ T ⟩∈ U is C end ∙ B) (function a defn) W′ | Left W = Left (function₂ (reflect-weakeningᴮ (f ↦ (T ⇒ U)) H B (snoc defn) W)) -reflectᴮ H (function f ⟨ var y ∈ T ⟩∈ U is C end ∙ B) (function a defn) W′ | Right (Left (UnallocatedAddress ())) -reflectᴮ H (function f ⟨ var y ∈ T ⟩∈ U is C end ∙ B) (function a defn) W′ | Right (Right p) = CONTRADICTION (≮:-refl p) -reflectᴮ H (return M ∙ B) (return s) (return W′) = mapL return (reflectᴱ H M s W′) - -reflectᴴᴱ : ∀ H M {H′ M′} → (H ⊢ M ⟶ᴱ M′ ⊣ H′) → Warningᴴ H′ (typeCheckᴴ H′) → Either (Warningᴱ H (typeCheckᴱ H ∅ M)) (Warningᴴ H (typeCheckᴴ H)) -reflectᴴᴮ : ∀ H B {H′ B′} → (H ⊢ B ⟶ᴮ B′ ⊣ H′) → Warningᴴ H′ (typeCheckᴴ H′) → Either (Warningᴮ H (typeCheckᴮ H ∅ B)) (Warningᴴ H (typeCheckᴴ H)) - -reflectᴴᴱ H (M $ N) (app₁ s) W = mapL app₁ (reflectᴴᴱ H M s W) -reflectᴴᴱ H (M $ N) (app₂ v s) W = mapL app₂ (reflectᴴᴱ H N s W) -reflectᴴᴱ H (M $ N) (beta O v refl p) W = Right W -reflectᴴᴱ H (function f ⟨ var x ∈ T ⟩∈ U is B end) (function a p) (addr b refl W) with b ≡ᴬ a -reflectᴴᴱ H (function f ⟨ var x ∈ T ⟩∈ U is B end) (function a defn) (addr b refl (FunctionDefnMismatch p)) | yes refl = Left (FunctionDefnMismatch (≮:-heap-weakeningᴮ (x ↦ T) H B (snoc defn) p)) -reflectᴴᴱ H (function f ⟨ var x ∈ T ⟩∈ U is B end) (function a defn) (addr b refl (function₁ W)) | yes refl = Left (function₁ (reflect-weakeningᴮ (x ↦ T) H B (snoc defn) W)) -reflectᴴᴱ H (function f ⟨ var x ∈ T ⟩∈ U is B end) (function a p) (addr b refl W) | no q = Right (addr b (lookup-not-allocated p q) (reflect-weakeningᴼ H _ (snoc p) W)) -reflectᴴᴱ H (block var b ∈ T is B end) (block s) W = mapL block₁ (reflectᴴᴮ H B s W) -reflectᴴᴱ H (block var b ∈ T is return (val v) ∙ B end) (return v) W = Right W -reflectᴴᴱ H (block var b ∈ T is done end) done W = Right W -reflectᴴᴱ H (binexp M op N) (binOp₀ s) W = Right W -reflectᴴᴱ H (binexp M op N) (binOp₁ s) W = mapL bin₁ (reflectᴴᴱ H M s W) -reflectᴴᴱ H (binexp M op N) (binOp₂ s) W = mapL bin₂ (reflectᴴᴱ H N s W) - -reflectᴴᴮ H (function f ⟨ var x ∈ T ⟩∈ U is C end ∙ B) (function a p) (addr b refl W) with b ≡ᴬ a -reflectᴴᴮ H (function f ⟨ var x ∈ T ⟩∈ U is C end ∙ B) (function a defn) (addr b refl (FunctionDefnMismatch p)) | yes refl = Left (FunctionDefnMismatch (≮:-heap-weakeningᴮ (x ↦ T) H C (snoc defn) p)) -reflectᴴᴮ H (function f ⟨ var x ∈ T ⟩∈ U is C end ∙ B) (function a defn) (addr b refl (function₁ W)) | yes refl = Left (function₁ (reflect-weakeningᴮ (x ↦ T) H C (snoc defn) W)) -reflectᴴᴮ H (function f ⟨ var x ∈ T ⟩∈ U is C end ∙ B) (function a p) (addr b refl W) | no q = Right (addr b (lookup-not-allocated p q) (reflect-weakeningᴼ H _ (snoc p) W)) -reflectᴴᴮ H (local var x ∈ T ← M ∙ B) (local s) W = mapL local₁ (reflectᴴᴱ H M s W) -reflectᴴᴮ H (local var x ∈ T ← M ∙ B) (subst v) W = Right W -reflectᴴᴮ H (return M ∙ B) (return s) W = mapL return (reflectᴴᴱ H M s W) - -reflect* : ∀ H B {H′ B′} → (H ⊢ B ⟶* B′ ⊣ H′) → Either (Warningᴮ H′ (typeCheckᴮ H′ ∅ B′)) (Warningᴴ H′ (typeCheckᴴ H′)) → Either (Warningᴮ H (typeCheckᴮ H ∅ B)) (Warningᴴ H (typeCheckᴴ H)) -reflect* H B refl W = W -reflect* H B (step s t) W = cond (reflectᴮ H B s) (reflectᴴᴮ H B s) (reflect* _ _ t W) - -isntNumber : ∀ H v → (valueType v ≢ number) → (typeOfᴱ H ∅ (val v) ≮: number) -isntNumber H nil p = scalar-≢-impl-≮: nil number (λ ()) -isntNumber H (addr a) p with remember (H [ a ]ᴴ) -isntNumber H (addr a) p | (just (function f ⟨ var x ∈ T ⟩∈ U is B end) , q) = ≡-trans-≮: (cong orUnknown (cong typeOfᴹᴼ q)) (function-≮:-scalar number) -isntNumber H (addr a) p | (nothing , q) = ≡-trans-≮: (cong orUnknown (cong typeOfᴹᴼ q)) (unknown-≮:-scalar number) -isntNumber H (number x) p = CONTRADICTION (p refl) -isntNumber H (bool x) p = scalar-≢-impl-≮: boolean number (λ ()) -isntNumber H (string x) p = scalar-≢-impl-≮: string number (λ ()) - -isntString : ∀ H v → (valueType v ≢ string) → (typeOfᴱ H ∅ (val v) ≮: string) -isntString H nil p = scalar-≢-impl-≮: nil string (λ ()) -isntString H (addr a) p with remember (H [ a ]ᴴ) -isntString H (addr a) p | (just (function f ⟨ var x ∈ T ⟩∈ U is B end) , q) = ≡-trans-≮: (cong orUnknown (cong typeOfᴹᴼ q)) (function-≮:-scalar string) -isntString H (addr a) p | (nothing , q) = ≡-trans-≮: (cong orUnknown (cong typeOfᴹᴼ q)) (unknown-≮:-scalar string) -isntString H (number x) p = scalar-≢-impl-≮: number string (λ ()) -isntString H (bool x) p = scalar-≢-impl-≮: boolean string (λ ()) -isntString H (string x) p = CONTRADICTION (p refl) - -isntFunction : ∀ H v {T U} → (valueType v ≢ function) → (typeOfᴱ H ∅ (val v) ≮: (T ⇒ U)) -isntFunction H nil p = scalar-≮:-function nil -isntFunction H (addr a) p = CONTRADICTION (p refl) -isntFunction H (number x) p = scalar-≮:-function number -isntFunction H (bool x) p = scalar-≮:-function boolean -isntFunction H (string x) p = scalar-≮:-function string - -isntEmpty : ∀ H v → (typeOfᴱ H ∅ (val v) ≮: never) -isntEmpty H nil = scalar-≮:-never nil -isntEmpty H (addr a) with remember (H [ a ]ᴴ) -isntEmpty H (addr a) | (just (function f ⟨ var x ∈ T ⟩∈ U is B end) , p) = ≡-trans-≮: (cong orUnknown (cong typeOfᴹᴼ p)) function-≮:-never -isntEmpty H (addr a) | (nothing , p) = ≡-trans-≮: (cong orUnknown (cong typeOfᴹᴼ p)) unknown-≮:-never -isntEmpty H (number x) = scalar-≮:-never number -isntEmpty H (bool x) = scalar-≮:-never boolean -isntEmpty H (string x) = scalar-≮:-never string - -runtimeBinOpWarning : ∀ H {op} v → BinOpError op (valueType v) → (typeOfᴱ H ∅ (val v) ≮: srcBinOp op) -runtimeBinOpWarning H v (+ p) = isntNumber H v p -runtimeBinOpWarning H v (- p) = isntNumber H v p -runtimeBinOpWarning H v (* p) = isntNumber H v p -runtimeBinOpWarning H v (/ p) = isntNumber H v p -runtimeBinOpWarning H v (< p) = isntNumber H v p -runtimeBinOpWarning H v (> p) = isntNumber H v p -runtimeBinOpWarning H v (<= p) = isntNumber H v p -runtimeBinOpWarning H v (>= p) = isntNumber H v p -runtimeBinOpWarning H v (·· p) = isntString H v p - -runtimeWarningᴱ : ∀ H M → RuntimeErrorᴱ H M → Warningᴱ H (typeCheckᴱ H ∅ M) -runtimeWarningᴮ : ∀ H B → RuntimeErrorᴮ H B → Warningᴮ H (typeCheckᴮ H ∅ B) - -runtimeWarningᴱ H (var x) UnboundVariable = UnboundVariable refl -runtimeWarningᴱ H (val (addr a)) (SEGV p) = UnallocatedAddress p -runtimeWarningᴱ H (M $ N) (FunctionMismatch v w p) = FunctionCallMismatch (unknown-src-≮: (isntEmpty H w) (isntFunction H v p)) -runtimeWarningᴱ H (M $ N) (app₁ err) = app₁ (runtimeWarningᴱ H M err) -runtimeWarningᴱ H (M $ N) (app₂ err) = app₂ (runtimeWarningᴱ H N err) -runtimeWarningᴱ H (block var b ∈ T is B end) (block err) = block₁ (runtimeWarningᴮ H B err) -runtimeWarningᴱ H (binexp M op N) (BinOpMismatch₁ v w p) = BinOpMismatch₁ (runtimeBinOpWarning H v p) -runtimeWarningᴱ H (binexp M op N) (BinOpMismatch₂ v w p) = BinOpMismatch₂ (runtimeBinOpWarning H w p) -runtimeWarningᴱ H (binexp M op N) (bin₁ err) = bin₁ (runtimeWarningᴱ H M err) -runtimeWarningᴱ H (binexp M op N) (bin₂ err) = bin₂ (runtimeWarningᴱ H N err) - -runtimeWarningᴮ H (local var x ∈ T ← M ∙ B) (local err) = local₁ (runtimeWarningᴱ H M err) -runtimeWarningᴮ H (return M ∙ B) (return err) = return (runtimeWarningᴱ H M err) - -wellTypedProgramsDontGoWrong : ∀ H′ B B′ → (∅ᴴ ⊢ B ⟶* B′ ⊣ H′) → (RuntimeErrorᴮ H′ B′) → Warningᴮ ∅ᴴ (typeCheckᴮ ∅ᴴ ∅ B) -wellTypedProgramsDontGoWrong H′ B B′ t err with reflect* ∅ᴴ B t (Left (runtimeWarningᴮ H′ B′ err)) -wellTypedProgramsDontGoWrong H′ B B′ t err | Right (addr a refl ()) -wellTypedProgramsDontGoWrong H′ B B′ t err | Left W = W diff --git a/prototyping/Properties/Subtyping.agda b/prototyping/Properties/Subtyping.agda deleted file mode 100644 index 73bf0e9a..00000000 --- a/prototyping/Properties/Subtyping.agda +++ /dev/null @@ -1,481 +0,0 @@ -{-# OPTIONS --rewriting #-} - -module Properties.Subtyping where - -open import Agda.Builtin.Equality using (_≡_; refl) -open import FFI.Data.Either using (Either; Left; Right; mapLR; swapLR; cond) -open import FFI.Data.Maybe using (Maybe; just; nothing) -open import Luau.Subtyping using (_<:_; _≮:_; Tree; Language; ¬Language; witness; unknown; never; scalar; function; scalar-function; scalar-function-ok; scalar-function-err; scalar-function-tgt; scalar-scalar; function-scalar; function-ok; function-ok₁; function-ok₂; function-err; function-tgt; left; right; _,_) -open import Luau.Type using (Type; Scalar; nil; number; string; boolean; never; unknown; _⇒_; _∪_; _∩_; skalar) -open import Properties.Contradiction using (CONTRADICTION; ¬; ⊥) -open import Properties.Equality using (_≢_) -open import Properties.Functions using (_∘_) -open import Properties.Product using (_×_; _,_) - --- Language membership is decidable -dec-language : ∀ T t → Either (¬Language T t) (Language T t) -dec-language nil (scalar number) = Left (scalar-scalar number nil (λ ())) -dec-language nil (scalar boolean) = Left (scalar-scalar boolean nil (λ ())) -dec-language nil (scalar string) = Left (scalar-scalar string nil (λ ())) -dec-language nil (scalar nil) = Right (scalar nil) -dec-language nil function = Left (scalar-function nil) -dec-language nil (function-ok s t) = Left (scalar-function-ok nil) -dec-language nil (function-err t) = Left (scalar-function-err nil) -dec-language boolean (scalar number) = Left (scalar-scalar number boolean (λ ())) -dec-language boolean (scalar boolean) = Right (scalar boolean) -dec-language boolean (scalar string) = Left (scalar-scalar string boolean (λ ())) -dec-language boolean (scalar nil) = Left (scalar-scalar nil boolean (λ ())) -dec-language boolean function = Left (scalar-function boolean) -dec-language boolean (function-ok s t) = Left (scalar-function-ok boolean) -dec-language boolean (function-err t) = Left (scalar-function-err boolean) -dec-language number (scalar number) = Right (scalar number) -dec-language number (scalar boolean) = Left (scalar-scalar boolean number (λ ())) -dec-language number (scalar string) = Left (scalar-scalar string number (λ ())) -dec-language number (scalar nil) = Left (scalar-scalar nil number (λ ())) -dec-language number function = Left (scalar-function number) -dec-language number (function-ok s t) = Left (scalar-function-ok number) -dec-language number (function-err t) = Left (scalar-function-err number) -dec-language string (scalar number) = Left (scalar-scalar number string (λ ())) -dec-language string (scalar boolean) = Left (scalar-scalar boolean string (λ ())) -dec-language string (scalar string) = Right (scalar string) -dec-language string (scalar nil) = Left (scalar-scalar nil string (λ ())) -dec-language string function = Left (scalar-function string) -dec-language string (function-ok s t) = Left (scalar-function-ok string) -dec-language string (function-err t) = Left (scalar-function-err string) -dec-language (T₁ ⇒ T₂) (scalar s) = Left (function-scalar s) -dec-language (T₁ ⇒ T₂) function = Right function -dec-language (T₁ ⇒ T₂) (function-ok s t) = cond (Right ∘ function-ok₁) (λ p → mapLR (function-ok p) function-ok₂ (dec-language T₂ t)) (dec-language T₁ s) -dec-language (T₁ ⇒ T₂) (function-err t) = mapLR function-err function-err (swapLR (dec-language T₁ t)) -dec-language never t = Left never -dec-language unknown t = Right unknown -dec-language (T₁ ∪ T₂) t = cond (λ p → cond (Left ∘ _,_ p) (Right ∘ right) (dec-language T₂ t)) (Right ∘ left) (dec-language T₁ t) -dec-language (T₁ ∩ T₂) t = cond (Left ∘ left) (λ p → cond (Left ∘ right) (Right ∘ _,_ p) (dec-language T₂ t)) (dec-language T₁ t) -dec-language nil (function-tgt t) = Left (scalar-function-tgt nil) -dec-language (T₁ ⇒ T₂) (function-tgt t) = mapLR function-tgt function-tgt (dec-language T₂ t) -dec-language boolean (function-tgt t) = Left (scalar-function-tgt boolean) -dec-language number (function-tgt t) = Left (scalar-function-tgt number) -dec-language string (function-tgt t) = Left (scalar-function-tgt string) - --- ¬Language T is the complement of Language T -language-comp : ∀ {T} t → ¬Language T t → ¬(Language T t) -language-comp t (p₁ , p₂) (left q) = language-comp t p₁ q -language-comp t (p₁ , p₂) (right q) = language-comp t p₂ q -language-comp t (left p) (q₁ , q₂) = language-comp t p q₁ -language-comp t (right p) (q₁ , q₂) = language-comp t p q₂ -language-comp (scalar s) (scalar-scalar s p₁ p₂) (scalar s) = p₂ refl -language-comp (scalar s) (function-scalar s) (scalar s) = language-comp function (scalar-function s) function -language-comp (scalar s) never (scalar ()) -language-comp function (scalar-function ()) function -language-comp (function-ok s t) (scalar-function-ok ()) (function-ok₁ p) -language-comp (function-ok s t) (function-ok p₁ p₂) (function-ok₁ q) = language-comp s q p₁ -language-comp (function-ok s t) (function-ok p₁ p₂) (function-ok₂ q) = language-comp t p₂ q -language-comp (function-err t) (function-err p) (function-err q) = language-comp t q p -language-comp (function-tgt t) (scalar-function-tgt ()) (function-tgt q) -language-comp (function-tgt t) (function-tgt p) (function-tgt q) = language-comp t p q - --- ≮: is the complement of <: -¬≮:-impl-<: : ∀ {T U} → ¬(T ≮: U) → (T <: U) -¬≮:-impl-<: {T} {U} p t q with dec-language U t -¬≮:-impl-<: {T} {U} p t q | Left r = CONTRADICTION (p (witness t q r)) -¬≮:-impl-<: {T} {U} p t q | Right r = r - -<:-impl-¬≮: : ∀ {T U} → (T <: U) → ¬(T ≮: U) -<:-impl-¬≮: p (witness t q r) = language-comp t r (p t q) - -<:-impl-⊇ : ∀ {T U} → (T <: U) → ∀ t → ¬Language U t → ¬Language T t -<:-impl-⊇ {T} p t q with dec-language T t -<:-impl-⊇ {_} p t q | Left r = r -<:-impl-⊇ {_} p t q | Right r = CONTRADICTION (language-comp t q (p t r)) - --- reflexivity -≮:-refl : ∀ {T} → ¬(T ≮: T) -≮:-refl (witness t p q) = language-comp t q p - -<:-refl : ∀ {T} → (T <: T) -<:-refl = ¬≮:-impl-<: ≮:-refl - --- transititivity -≮:-trans-≡ : ∀ {S T U} → (S ≮: T) → (T ≡ U) → (S ≮: U) -≮:-trans-≡ p refl = p - -<:-trans-≡ : ∀ {S T U} → (S <: T) → (T ≡ U) → (S <: U) -<:-trans-≡ p refl = p - -≡-impl-<: : ∀ {T U} → (T ≡ U) → (T <: U) -≡-impl-<: refl = <:-refl - -≡-trans-≮: : ∀ {S T U} → (S ≡ T) → (T ≮: U) → (S ≮: U) -≡-trans-≮: refl p = p - -≡-trans-<: : ∀ {S T U} → (S ≡ T) → (T <: U) → (S <: U) -≡-trans-<: refl p = p - -≮:-trans : ∀ {S T U} → (S ≮: U) → Either (S ≮: T) (T ≮: U) -≮:-trans {T = T} (witness t p q) = mapLR (witness t p) (λ z → witness t z q) (dec-language T t) - -<:-trans : ∀ {S T U} → (S <: T) → (T <: U) → (S <: U) -<:-trans p q t r = q t (p t r) - -<:-trans-≮: : ∀ {S T U} → (S <: T) → (S ≮: U) → (T ≮: U) -<:-trans-≮: p (witness t q r) = witness t (p t q) r - -≮:-trans-<: : ∀ {S T U} → (S ≮: U) → (T <: U) → (S ≮: T) -≮:-trans-<: (witness t p q) r = witness t p (<:-impl-⊇ r t q) - --- Properties of union - -<:-union : ∀ {R S T U} → (R <: T) → (S <: U) → ((R ∪ S) <: (T ∪ U)) -<:-union p q t (left r) = left (p t r) -<:-union p q t (right r) = right (q t r) - -<:-∪-left : ∀ {S T} → S <: (S ∪ T) -<:-∪-left t p = left p - -<:-∪-right : ∀ {S T} → T <: (S ∪ T) -<:-∪-right t p = right p - -<:-∪-lub : ∀ {S T U} → (S <: U) → (T <: U) → ((S ∪ T) <: U) -<:-∪-lub p q t (left r) = p t r -<:-∪-lub p q t (right r) = q t r - -<:-∪-symm : ∀ {T U} → (T ∪ U) <: (U ∪ T) -<:-∪-symm t (left p) = right p -<:-∪-symm t (right p) = left p - -<:-∪-assocl : ∀ {S T U} → (S ∪ (T ∪ U)) <: ((S ∪ T) ∪ U) -<:-∪-assocl t (left p) = left (left p) -<:-∪-assocl t (right (left p)) = left (right p) -<:-∪-assocl t (right (right p)) = right p - -<:-∪-assocr : ∀ {S T U} → ((S ∪ T) ∪ U) <: (S ∪ (T ∪ U)) -<:-∪-assocr t (left (left p)) = left p -<:-∪-assocr t (left (right p)) = right (left p) -<:-∪-assocr t (right p) = right (right p) - -≮:-∪-left : ∀ {S T U} → (S ≮: U) → ((S ∪ T) ≮: U) -≮:-∪-left (witness t p q) = witness t (left p) q - -≮:-∪-right : ∀ {S T U} → (T ≮: U) → ((S ∪ T) ≮: U) -≮:-∪-right (witness t p q) = witness t (right p) q - -≮:-left-∪ : ∀ {S T U} → (S ≮: (T ∪ U)) → (S ≮: T) -≮:-left-∪ (witness t p (q₁ , q₂)) = witness t p q₁ - -≮:-right-∪ : ∀ {S T U} → (S ≮: (T ∪ U)) → (S ≮: U) -≮:-right-∪ (witness t p (q₁ , q₂)) = witness t p q₂ - --- Properties of intersection - -<:-intersect : ∀ {R S T U} → (R <: T) → (S <: U) → ((R ∩ S) <: (T ∩ U)) -<:-intersect p q t (r₁ , r₂) = (p t r₁ , q t r₂) - -<:-∩-left : ∀ {S T} → (S ∩ T) <: S -<:-∩-left t (p , _) = p - -<:-∩-right : ∀ {S T} → (S ∩ T) <: T -<:-∩-right t (_ , p) = p - -<:-∩-glb : ∀ {S T U} → (S <: T) → (S <: U) → (S <: (T ∩ U)) -<:-∩-glb p q t r = (p t r , q t r) - -<:-∩-symm : ∀ {T U} → (T ∩ U) <: (U ∩ T) -<:-∩-symm t (p₁ , p₂) = (p₂ , p₁) - -<:-∩-assocl : ∀ {S T U} → (S ∩ (T ∩ U)) <: ((S ∩ T) ∩ U) -<:-∩-assocl t (p , (p₁ , p₂)) = (p , p₁) , p₂ - -<:-∩-assocr : ∀ {S T U} → ((S ∩ T) ∩ U) <: (S ∩ (T ∩ U)) -<:-∩-assocr t ((p , p₁) , p₂) = p , (p₁ , p₂) - -≮:-∩-left : ∀ {S T U} → (S ≮: T) → (S ≮: (T ∩ U)) -≮:-∩-left (witness t p q) = witness t p (left q) - -≮:-∩-right : ∀ {S T U} → (S ≮: U) → (S ≮: (T ∩ U)) -≮:-∩-right (witness t p q) = witness t p (right q) - --- Distribution properties -<:-∩-distl-∪ : ∀ {S T U} → (S ∩ (T ∪ U)) <: ((S ∩ T) ∪ (S ∩ U)) -<:-∩-distl-∪ t (p₁ , left p₂) = left (p₁ , p₂) -<:-∩-distl-∪ t (p₁ , right p₂) = right (p₁ , p₂) - -∩-distl-∪-<: : ∀ {S T U} → ((S ∩ T) ∪ (S ∩ U)) <: (S ∩ (T ∪ U)) -∩-distl-∪-<: t (left (p₁ , p₂)) = (p₁ , left p₂) -∩-distl-∪-<: t (right (p₁ , p₂)) = (p₁ , right p₂) - -<:-∩-distr-∪ : ∀ {S T U} → ((S ∪ T) ∩ U) <: ((S ∩ U) ∪ (T ∩ U)) -<:-∩-distr-∪ t (left p₁ , p₂) = left (p₁ , p₂) -<:-∩-distr-∪ t (right p₁ , p₂) = right (p₁ , p₂) - -∩-distr-∪-<: : ∀ {S T U} → ((S ∩ U) ∪ (T ∩ U)) <: ((S ∪ T) ∩ U) -∩-distr-∪-<: t (left (p₁ , p₂)) = (left p₁ , p₂) -∩-distr-∪-<: t (right (p₁ , p₂)) = (right p₁ , p₂) - -<:-∪-distl-∩ : ∀ {S T U} → (S ∪ (T ∩ U)) <: ((S ∪ T) ∩ (S ∪ U)) -<:-∪-distl-∩ t (left p) = (left p , left p) -<:-∪-distl-∩ t (right (p₁ , p₂)) = (right p₁ , right p₂) - -∪-distl-∩-<: : ∀ {S T U} → ((S ∪ T) ∩ (S ∪ U)) <: (S ∪ (T ∩ U)) -∪-distl-∩-<: t (left p₁ , p₂) = left p₁ -∪-distl-∩-<: t (right p₁ , left p₂) = left p₂ -∪-distl-∩-<: t (right p₁ , right p₂) = right (p₁ , p₂) - -<:-∪-distr-∩ : ∀ {S T U} → ((S ∩ T) ∪ U) <: ((S ∪ U) ∩ (T ∪ U)) -<:-∪-distr-∩ t (left (p₁ , p₂)) = left p₁ , left p₂ -<:-∪-distr-∩ t (right p) = (right p , right p) - -∪-distr-∩-<: : ∀ {S T U} → ((S ∪ U) ∩ (T ∪ U)) <: ((S ∩ T) ∪ U) -∪-distr-∩-<: t (left p₁ , left p₂) = left (p₁ , p₂) -∪-distr-∩-<: t (left p₁ , right p₂) = right p₂ -∪-distr-∩-<: t (right p₁ , p₂) = right p₁ - -∩-<:-∪ : ∀ {S T} → (S ∩ T) <: (S ∪ T) -∩-<:-∪ t (p , _) = left p - --- Properties of functions -<:-function : ∀ {R S T U} → (R <: S) → (T <: U) → (S ⇒ T) <: (R ⇒ U) -<:-function p q function function = function -<:-function p q (function-ok s t) (function-ok₁ r) = function-ok₁ (<:-impl-⊇ p s r) -<:-function p q (function-ok s t) (function-ok₂ r) = function-ok₂ (q t r) -<:-function p q (function-err s) (function-err r) = function-err (<:-impl-⊇ p s r) -<:-function p q (function-tgt t) (function-tgt r) = function-tgt (q t r) - -<:-function-∩-∩ : ∀ {R S T U} → ((R ⇒ T) ∩ (S ⇒ U)) <: ((R ∩ S) ⇒ (T ∩ U)) -<:-function-∩-∩ function (function , function) = function -<:-function-∩-∩ (function-ok s t) (function-ok₁ p , q) = function-ok₁ (left p) -<:-function-∩-∩ (function-ok s t) (function-ok₂ p , function-ok₁ q) = function-ok₁ (right q) -<:-function-∩-∩ (function-ok s t) (function-ok₂ p , function-ok₂ q) = function-ok₂ (p , q) -<:-function-∩-∩ (function-err s) (function-err p , q) = function-err (left p) -<:-function-∩-∩ (function-tgt s) (function-tgt p , function-tgt q) = function-tgt (p , q) - -<:-function-∩-∪ : ∀ {R S T U} → ((R ⇒ T) ∩ (S ⇒ U)) <: ((R ∪ S) ⇒ (T ∪ U)) -<:-function-∩-∪ function (function , function) = function -<:-function-∩-∪ (function-ok s t) (function-ok₁ p₁ , function-ok₁ p₂) = function-ok₁ (p₁ , p₂) -<:-function-∩-∪ (function-ok s t) (p₁ , function-ok₂ p₂) = function-ok₂ (right p₂) -<:-function-∩-∪ (function-ok s t) (function-ok₂ p₁ , p₂) = function-ok₂ (left p₁) -<:-function-∩-∪ (function-err s) (function-err p₁ , function-err q₂) = function-err (p₁ , q₂) -<:-function-∩-∪ (function-tgt t) (function-tgt p , q) = function-tgt (left p) - -<:-function-∩ : ∀ {S T U} → ((S ⇒ T) ∩ (S ⇒ U)) <: (S ⇒ (T ∩ U)) -<:-function-∩ function (function , function) = function -<:-function-∩ (function-ok s t) (p₁ , function-ok₁ p₂) = function-ok₁ p₂ -<:-function-∩ (function-ok s t) (function-ok₁ p₁ , p₂) = function-ok₁ p₁ -<:-function-∩ (function-ok s t) (function-ok₂ p₁ , function-ok₂ p₂) = function-ok₂ (p₁ , p₂) -<:-function-∩ (function-err s) (function-err p₁ , function-err p₂) = function-err p₂ -<:-function-∩ (function-tgt t) (function-tgt p₁ , function-tgt p₂) = function-tgt (p₁ , p₂) - -<:-function-∪ : ∀ {R S T U} → ((R ⇒ S) ∪ (T ⇒ U)) <: ((R ∩ T) ⇒ (S ∪ U)) -<:-function-∪ function (left function) = function -<:-function-∪ (function-ok s t) (left (function-ok₁ p)) = function-ok₁ (left p) -<:-function-∪ (function-ok s t) (left (function-ok₂ p)) = function-ok₂ (left p) -<:-function-∪ (function-err s) (left (function-err p)) = function-err (left p) -<:-function-∪ (scalar s) (left (scalar ())) -<:-function-∪ function (right function) = function -<:-function-∪ (function-ok s t) (right (function-ok₁ p)) = function-ok₁ (right p) -<:-function-∪ (function-ok s t) (right (function-ok₂ p)) = function-ok₂ (right p) -<:-function-∪ (function-err s) (right (function-err x)) = function-err (right x) -<:-function-∪ (scalar s) (right (scalar ())) -<:-function-∪ (function-tgt t) (left (function-tgt p)) = function-tgt (left p) -<:-function-∪ (function-tgt t) (right (function-tgt p)) = function-tgt (right p) - -<:-function-∪-∩ : ∀ {R S T U} → ((R ∩ S) ⇒ (T ∪ U)) <: ((R ⇒ T) ∪ (S ⇒ U)) -<:-function-∪-∩ function function = left function -<:-function-∪-∩ (function-ok s t) (function-ok₁ (left p)) = left (function-ok₁ p) -<:-function-∪-∩ (function-ok s t) (function-ok₂ (left p)) = left (function-ok₂ p) -<:-function-∪-∩ (function-ok s t) (function-ok₁ (right p)) = right (function-ok₁ p) -<:-function-∪-∩ (function-ok s t) (function-ok₂ (right p)) = right (function-ok₂ p) -<:-function-∪-∩ (function-err s) (function-err (left p)) = left (function-err p) -<:-function-∪-∩ (function-err s) (function-err (right p)) = right (function-err p) -<:-function-∪-∩ (function-tgt t) (function-tgt (left p)) = left (function-tgt p) -<:-function-∪-∩ (function-tgt t) (function-tgt (right p)) = right (function-tgt p) - -<:-function-left : ∀ {R S T U} → (S ⇒ T) <: (R ⇒ U) → (R <: S) -<:-function-left {R} {S} p s Rs with dec-language S s -<:-function-left p s Rs | Right Ss = Ss -<:-function-left p s Rs | Left ¬Ss with p (function-err s) (function-err ¬Ss) -<:-function-left p s Rs | Left ¬Ss | function-err ¬Rs = CONTRADICTION (language-comp s ¬Rs Rs) - -<:-function-right : ∀ {R S T U} → (S ⇒ T) <: (R ⇒ U) → (T <: U) -<:-function-right p t Tt with p (function-tgt t) (function-tgt Tt) -<:-function-right p t Tt | function-tgt St = St - -≮:-function-left : ∀ {R S T U} → (R ≮: S) → (S ⇒ T) ≮: (R ⇒ U) -≮:-function-left (witness t p q) = witness (function-err t) (function-err q) (function-err p) - -≮:-function-right : ∀ {R S T U} → (T ≮: U) → (S ⇒ T) ≮: (R ⇒ U) -≮:-function-right (witness t p q) = witness (function-tgt t) (function-tgt p) (function-tgt q) - --- Properties of scalars -skalar-function-ok : ∀ {s t} → (¬Language skalar (function-ok s t)) -skalar-function-ok = (scalar-function-ok number , (scalar-function-ok string , (scalar-function-ok nil , scalar-function-ok boolean))) - -scalar-<: : ∀ {S T} → (s : Scalar S) → Language T (scalar s) → (S <: T) -scalar-<: number p (scalar number) (scalar number) = p -scalar-<: boolean p (scalar boolean) (scalar boolean) = p -scalar-<: string p (scalar string) (scalar string) = p -scalar-<: nil p (scalar nil) (scalar nil) = p - -scalar-∩-function-<:-never : ∀ {S T U} → (Scalar S) → ((T ⇒ U) ∩ S) <: never -scalar-∩-function-<:-never number .(scalar number) (() , scalar number) -scalar-∩-function-<:-never boolean .(scalar boolean) (() , scalar boolean) -scalar-∩-function-<:-never string .(scalar string) (() , scalar string) -scalar-∩-function-<:-never nil .(scalar nil) (() , scalar nil) - -function-≮:-scalar : ∀ {S T U} → (Scalar U) → ((S ⇒ T) ≮: U) -function-≮:-scalar s = witness function function (scalar-function s) - -scalar-≮:-function : ∀ {S T U} → (Scalar U) → (U ≮: (S ⇒ T)) -scalar-≮:-function s = witness (scalar s) (scalar s) (function-scalar s) - -unknown-≮:-scalar : ∀ {U} → (Scalar U) → (unknown ≮: U) -unknown-≮:-scalar s = witness function unknown (scalar-function s) - -scalar-≮:-never : ∀ {U} → (Scalar U) → (U ≮: never) -scalar-≮:-never s = witness (scalar s) (scalar s) never - -scalar-≢-impl-≮: : ∀ {T U} → (Scalar T) → (Scalar U) → (T ≢ U) → (T ≮: U) -scalar-≢-impl-≮: s₁ s₂ p = witness (scalar s₁) (scalar s₁) (scalar-scalar s₁ s₂ p) - -scalar-≢-∩-<:-never : ∀ {T U V} → (Scalar T) → (Scalar U) → (T ≢ U) → (T ∩ U) <: V -scalar-≢-∩-<:-never s t p u (scalar s₁ , scalar s₂) = CONTRADICTION (p refl) - -skalar-scalar : ∀ {T} (s : Scalar T) → (Language skalar (scalar s)) -skalar-scalar number = left (scalar number) -skalar-scalar boolean = right (right (right (scalar boolean))) -skalar-scalar string = right (left (scalar string)) -skalar-scalar nil = right (right (left (scalar nil))) - --- Properties of unknown and never -unknown-≮: : ∀ {T U} → (T ≮: U) → (unknown ≮: U) -unknown-≮: (witness t p q) = witness t unknown q - -never-≮: : ∀ {T U} → (T ≮: U) → (T ≮: never) -never-≮: (witness t p q) = witness t p never - -unknown-≮:-never : (unknown ≮: never) -unknown-≮:-never = witness (scalar nil) unknown never - -unknown-≮:-function : ∀ {S T} → (unknown ≮: (S ⇒ T)) -unknown-≮:-function = witness (scalar nil) unknown (function-scalar nil) - -function-≮:-never : ∀ {T U} → ((T ⇒ U) ≮: never) -function-≮:-never = witness function function never - -<:-never : ∀ {T} → (never <: T) -<:-never t (scalar ()) - -≮:-never-left : ∀ {S T U} → (S <: (T ∪ U)) → (S ≮: T) → (S ∩ U) ≮: never -≮:-never-left p (witness t q₁ q₂) with p t q₁ -≮:-never-left p (witness t q₁ q₂) | left r = CONTRADICTION (language-comp t q₂ r) -≮:-never-left p (witness t q₁ q₂) | right r = witness t (q₁ , r) never - -≮:-never-right : ∀ {S T U} → (S <: (T ∪ U)) → (S ≮: U) → (S ∩ T) ≮: never -≮:-never-right p (witness t q₁ q₂) with p t q₁ -≮:-never-right p (witness t q₁ q₂) | left r = witness t (q₁ , r) never -≮:-never-right p (witness t q₁ q₂) | right r = CONTRADICTION (language-comp t q₂ r) - -<:-unknown : ∀ {T} → (T <: unknown) -<:-unknown t p = unknown - -<:-everything : unknown <: ((never ⇒ unknown) ∪ skalar) -<:-everything (scalar s) p = right (skalar-scalar s) -<:-everything function p = left function -<:-everything (function-ok s t) p = left (function-ok₁ never) -<:-everything (function-err s) p = left (function-err never) -<:-everything (function-tgt t) p = left (function-tgt unknown) - --- A Gentle Introduction To Semantic Subtyping (https://www.cduce.org/papers/gentle.pdf) --- defines a "set-theoretic" model (sec 2.5) --- Unfortunately we don't quite have this property, due to uninhabited types, --- for example (never -> T) is equivalent to (never -> U) --- when types are interpreted as sets of syntactic values. - -_⊆_ : ∀ {A : Set} → (A → Set) → (A → Set) → Set -(P ⊆ Q) = ∀ a → (P a) → (Q a) - -_⊗_ : ∀ {A B : Set} → (A → Set) → (B → Set) → ((A × B) → Set) -(P ⊗ Q) (a , b) = (P a) × (Q b) - -Comp : ∀ {A : Set} → (A → Set) → (A → Set) -Comp P a = ¬(P a) - -Lift : ∀ {A : Set} → (A → Set) → (Maybe A → Set) -Lift P nothing = ⊥ -Lift P (just a) = P a - -set-theoretic-if : ∀ {S₁ T₁ S₂ T₂} → - - -- This is the "if" part of being a set-theoretic model - -- though it uses the definition from Frisch's thesis - -- rather than from the Gentle Introduction. The difference - -- being the presence of Lift, (written D_Ω in Defn 4.2 of - -- https://www.cduce.org/papers/frisch_phd.pdf). - (Language (S₁ ⇒ T₁) ⊆ Language (S₂ ⇒ T₂)) → - (∀ Q → Q ⊆ Comp((Language S₁) ⊗ Comp(Lift(Language T₁))) → Q ⊆ Comp((Language S₂) ⊗ Comp(Lift(Language T₂)))) - -set-theoretic-if {S₁} {T₁} {S₂} {T₂} p Q q (t , just u) Qtu (S₂t , ¬T₂u) = q (t , just u) Qtu (S₁t , ¬T₁u) where - - S₁t : Language S₁ t - S₁t with dec-language S₁ t - S₁t | Left ¬S₁t with p (function-err t) (function-err ¬S₁t) - S₁t | Left ¬S₁t | function-err ¬S₂t = CONTRADICTION (language-comp t ¬S₂t S₂t) - S₁t | Right r = r - - ¬T₁u : ¬(Language T₁ u) - ¬T₁u T₁u with p (function-ok t u) (function-ok₂ T₁u) - ¬T₁u T₁u | function-ok₁ ¬S₂t = language-comp t ¬S₂t S₂t - ¬T₁u T₁u | function-ok₂ T₂u = ¬T₂u T₂u - -set-theoretic-if {S₁} {T₁} {S₂} {T₂} p Q q (t , nothing) Qt- (S₂t , _) = q (t , nothing) Qt- (S₁t , λ ()) where - - S₁t : Language S₁ t - S₁t with dec-language S₁ t - S₁t | Left ¬S₁t with p (function-err t) (function-err ¬S₁t) - S₁t | Left ¬S₁t | function-err ¬S₂t = CONTRADICTION (language-comp t ¬S₂t S₂t) - S₁t | Right r = r - -not-quite-set-theoretic-only-if : ∀ {S₁ T₁ S₂ T₂} → - - -- We don't quite have that this is a set-theoretic model - -- it's only true when Language S₂ is inhabited - -- in particular it's not true when S₂ is never, - ∀ s₂ → Language S₂ s₂ → - - -- This is the "only if" part of being a set-theoretic model - (∀ Q → Q ⊆ Comp((Language S₁) ⊗ Comp(Lift(Language T₁))) → Q ⊆ Comp((Language S₂) ⊗ Comp(Lift(Language T₂)))) → - (Language (S₁ ⇒ T₁) ⊆ Language (S₂ ⇒ T₂)) - -not-quite-set-theoretic-only-if {S₁} {T₁} {S₂} {T₂} s₂ S₂s₂ p = r where - - Q : (Tree × Maybe Tree) → Set - Q (t , just u) = Either (¬Language S₁ t) (Language T₁ u) - Q (t , nothing) = ¬Language S₁ t - - q : Q ⊆ Comp(Language S₁ ⊗ Comp(Lift(Language T₁))) - q (t , just u) (Left ¬S₁t) (S₁t , ¬T₁u) = language-comp t ¬S₁t S₁t - q (t , just u) (Right T₂u) (S₁t , ¬T₁u) = ¬T₁u T₂u - q (t , nothing) ¬S₁t (S₁t , _) = language-comp t ¬S₁t S₁t - - r : Language (S₁ ⇒ T₁) ⊆ Language (S₂ ⇒ T₂) - r function function = function - r (function-err s) (function-err ¬S₁s) with dec-language S₂ s - r (function-err s) (function-err ¬S₁s) | Left ¬S₂s = function-err ¬S₂s - r (function-err s) (function-err ¬S₁s) | Right S₂s = CONTRADICTION (p Q q (s , nothing) ¬S₁s (S₂s , λ ())) - r (function-ok s t) (function-ok₁ ¬S₁s) with dec-language S₂ s - r (function-ok s t) (function-ok₁ ¬S₁s) | Left ¬S₂s = function-ok₁ ¬S₂s - r (function-ok s t) (function-ok₁ ¬S₁s) | Right S₂s = CONTRADICTION (p Q q (s , nothing) ¬S₁s (S₂s , λ ())) - r (function-ok s t) (function-ok₂ T₁t) with dec-language T₂ t - r (function-ok s t) (function-ok₂ T₁t) | Left ¬T₂t with dec-language S₂ s - r (function-ok s t) (function-ok₂ T₁t) | Left ¬T₂t | Left ¬S₂s = function-ok₁ ¬S₂s - r (function-ok s t) (function-ok₂ T₁t) | Left ¬T₂t | Right S₂s = CONTRADICTION (p Q q (s , just t) (Right T₁t) (S₂s , language-comp t ¬T₂t)) - r (function-ok s t) (function-ok₂ T₁t) | Right T₂t = function-ok₂ T₂t - r (function-tgt t) (function-tgt T₁t) with dec-language T₂ t - r (function-tgt t) (function-tgt T₁t) | Left ¬T₂t = CONTRADICTION (p Q q (s₂ , just t) (Right T₁t) (S₂s₂ , language-comp t ¬T₂t)) - r (function-tgt t) (function-tgt T₁t) | Right T₂t = function-tgt T₂t - --- A counterexample when the argument type is empty. - -set-theoretic-counterexample-one : (∀ Q → Q ⊆ Comp((Language never) ⊗ Comp(Lift(Language number))) → Q ⊆ Comp((Language never) ⊗ Comp(Lift(Language string)))) -set-theoretic-counterexample-one Q q ((scalar s) , u) Qtu (scalar () , p) - -set-theoretic-counterexample-two : (never ⇒ number) ≮: (never ⇒ string) -set-theoretic-counterexample-two = witness (function-tgt (scalar number)) (function-tgt (scalar number)) (function-tgt (scalar-scalar number string (λ ()))) diff --git a/prototyping/Properties/TypeCheck.agda b/prototyping/Properties/TypeCheck.agda deleted file mode 100644 index b53bbd04..00000000 --- a/prototyping/Properties/TypeCheck.agda +++ /dev/null @@ -1,100 +0,0 @@ -{-# OPTIONS --rewriting #-} - -module Properties.TypeCheck where - -open import Agda.Builtin.Equality using (_≡_; refl) -open import Agda.Builtin.Bool using (Bool; true; false) -open import FFI.Data.Maybe using (Maybe; just; nothing) -open import FFI.Data.Either using (Either) -open import Luau.ResolveOverloads using (resolve) -open import Luau.TypeCheck using (_⊢ᴱ_∈_; _⊢ᴮ_∈_; ⊢ᴼ_; ⊢ᴴ_; _⊢ᴴᴱ_▷_∈_; _⊢ᴴᴮ_▷_∈_; nil; var; addr; number; bool; string; app; function; block; binexp; done; return; local; nothing; orUnknown; tgtBinOp) -open import Luau.Syntax using (Block; Expr; Value; BinaryOperator; yes; nil; addr; number; bool; string; val; var; binexp; _$_; function_is_end; block_is_end; _∙_; return; done; local_←_; _⟨_⟩; _⟨_⟩∈_; var_∈_; name; fun; arg; +; -; *; /; <; >; ==; ~=; <=; >=) -open import Luau.Type using (Type; nil; unknown; never; number; boolean; string; _⇒_) -open import Luau.RuntimeType using (RuntimeType; nil; number; function; string; valueType) -open import Luau.VarCtxt using (VarCtxt; ∅; _↦_; _⊕_↦_; _⋒_; _⊝_) renaming (_[_] to _[_]ⱽ) -open import Luau.Addr using (Addr) -open import Luau.Var using (Var; _≡ⱽ_) -open import Luau.Heap using (Heap; Object; function_is_end) renaming (_[_] to _[_]ᴴ) -open import Properties.Contradiction using (CONTRADICTION) -open import Properties.Dec using (yes; no) -open import Properties.Equality using (_≢_; sym; trans; cong) -open import Properties.Product using (_×_; _,_) -open import Properties.Remember using (Remember; remember; _,_) - -typeOfᴼ : Object yes → Type -typeOfᴼ (function f ⟨ var x ∈ S ⟩∈ T is B end) = (S ⇒ T) - -typeOfᴹᴼ : Maybe(Object yes) → Maybe Type -typeOfᴹᴼ nothing = nothing -typeOfᴹᴼ (just O) = just (typeOfᴼ O) - -typeOfⱽ : Heap yes → Value → Maybe Type -typeOfⱽ H nil = just nil -typeOfⱽ H (bool b) = just boolean -typeOfⱽ H (addr a) = typeOfᴹᴼ (H [ a ]ᴴ) -typeOfⱽ H (number n) = just number -typeOfⱽ H (string x) = just string - -typeOfᴱ : Heap yes → VarCtxt → (Expr yes) → Type -typeOfᴮ : Heap yes → VarCtxt → (Block yes) → Type - -typeOfᴱ H Γ (var x) = orUnknown(Γ [ x ]ⱽ) -typeOfᴱ H Γ (val v) = orUnknown(typeOfⱽ H v) -typeOfᴱ H Γ (M $ N) = resolve (typeOfᴱ H Γ M) (typeOfᴱ H Γ N) -typeOfᴱ H Γ (function f ⟨ var x ∈ S ⟩∈ T is B end) = S ⇒ T -typeOfᴱ H Γ (block var b ∈ T is B end) = T -typeOfᴱ H Γ (binexp M op N) = tgtBinOp op - -typeOfᴮ H Γ (function f ⟨ var x ∈ S ⟩∈ T is C end ∙ B) = typeOfᴮ H (Γ ⊕ f ↦ (S ⇒ T)) B -typeOfᴮ H Γ (local var x ∈ T ← M ∙ B) = typeOfᴮ H (Γ ⊕ x ↦ T) B -typeOfᴮ H Γ (return M ∙ B) = typeOfᴱ H Γ M -typeOfᴮ H Γ done = nil - -mustBeNumber : ∀ H Γ v → (typeOfᴱ H Γ (val v) ≡ number) → (valueType(v) ≡ number) -mustBeNumber H Γ (addr a) p with remember (H [ a ]ᴴ) -mustBeNumber H Γ (addr a) p | (just O , q) with trans (cong orUnknown (cong typeOfᴹᴼ (sym q))) p -mustBeNumber H Γ (addr a) p | (just function f ⟨ var x ∈ T ⟩∈ U is B end , q) | () -mustBeNumber H Γ (addr a) p | (nothing , q) with trans (cong orUnknown (cong typeOfᴹᴼ (sym q))) p -mustBeNumber H Γ (addr a) p | nothing , q | () -mustBeNumber H Γ (number n) p = refl - -mustBeString : ∀ H Γ v → (typeOfᴱ H Γ (val v) ≡ string) → (valueType(v) ≡ string) -mustBeString H Γ (addr a) p with remember (H [ a ]ᴴ) -mustBeString H Γ (addr a) p | (just O , q) with trans (cong orUnknown (cong typeOfᴹᴼ (sym q))) p -mustBeString H Γ (addr a) p | (just function f ⟨ var x ∈ T ⟩∈ U is B end , q) | () -mustBeString H Γ (addr a) p | (nothing , q) with trans (cong orUnknown (cong typeOfᴹᴼ (sym q))) p -mustBeString H Γ (addr a) p | (nothing , q) | () -mustBeString H Γ (string x) p = refl - -typeCheckᴱ : ∀ H Γ M → (Γ ⊢ᴱ M ∈ (typeOfᴱ H Γ M)) -typeCheckᴮ : ∀ H Γ B → (Γ ⊢ᴮ B ∈ (typeOfᴮ H Γ B)) - -typeCheckᴱ H Γ (var x) = var refl -typeCheckᴱ H Γ (val nil) = nil -typeCheckᴱ H Γ (val (addr a)) = addr (orUnknown (typeOfᴹᴼ (H [ a ]ᴴ))) -typeCheckᴱ H Γ (val (number n)) = number -typeCheckᴱ H Γ (val (bool b)) = bool -typeCheckᴱ H Γ (val (string x)) = string -typeCheckᴱ H Γ (M $ N) = app (typeCheckᴱ H Γ M) (typeCheckᴱ H Γ N) -typeCheckᴱ H Γ (function f ⟨ var x ∈ T ⟩∈ U is B end) = function (typeCheckᴮ H (Γ ⊕ x ↦ T) B) -typeCheckᴱ H Γ (block var b ∈ T is B end) = block (typeCheckᴮ H Γ B) -typeCheckᴱ H Γ (binexp M op N) = binexp (typeCheckᴱ H Γ M) (typeCheckᴱ H Γ N) - -typeCheckᴮ H Γ (function f ⟨ var x ∈ T ⟩∈ U is C end ∙ B) = function (typeCheckᴮ H (Γ ⊕ x ↦ T) C) (typeCheckᴮ H (Γ ⊕ f ↦ (T ⇒ U)) B) -typeCheckᴮ H Γ (local var x ∈ T ← M ∙ B) = local (typeCheckᴱ H Γ M) (typeCheckᴮ H (Γ ⊕ x ↦ T) B) -typeCheckᴮ H Γ (return M ∙ B) = return (typeCheckᴱ H Γ M) (typeCheckᴮ H Γ B) -typeCheckᴮ H Γ done = done - -typeCheckᴼ : ∀ H O → (⊢ᴼ O) -typeCheckᴼ H nothing = nothing -typeCheckᴼ H (just function f ⟨ var x ∈ T ⟩∈ U is B end) = function (typeCheckᴮ H (x ↦ T) B) - -typeCheckᴴ : ∀ H → (⊢ᴴ H) -typeCheckᴴ H a {O} p = typeCheckᴼ H (O) - -typeCheckᴴᴱ : ∀ H Γ M → (Γ ⊢ᴴᴱ H ▷ M ∈ typeOfᴱ H Γ M) -typeCheckᴴᴱ H Γ M = (typeCheckᴴ H , typeCheckᴱ H Γ M) - -typeCheckᴴᴮ : ∀ H Γ M → (Γ ⊢ᴴᴮ H ▷ M ∈ typeOfᴮ H Γ M) -typeCheckᴴᴮ H Γ M = (typeCheckᴴ H , typeCheckᴮ H Γ M) - diff --git a/prototyping/Properties/TypeNormalization.agda b/prototyping/Properties/TypeNormalization.agda deleted file mode 100644 index cbd8139f..00000000 --- a/prototyping/Properties/TypeNormalization.agda +++ /dev/null @@ -1,408 +0,0 @@ -{-# OPTIONS --rewriting #-} - -module Properties.TypeNormalization where - -open import Luau.Type using (Type; Scalar; nil; number; string; boolean; never; unknown; _⇒_; _∪_; _∩_) -open import Luau.Subtyping using (Tree; Language; ¬Language; function; scalar; unknown; left; right; function-ok₁; function-ok₂; function-err; function-tgt; scalar-function; scalar-function-ok; scalar-function-err; scalar-function-tgt; function-scalar; _,_) -open import Luau.TypeNormalization using (_∪ⁿ_; _∩ⁿ_; _∪ᶠ_; _∪ⁿˢ_; _∩ⁿˢ_; normalize) -open import Luau.Subtyping using (_<:_; _≮:_; witness; never) -open import Properties.Subtyping using (<:-trans; <:-refl; <:-unknown; <:-never; <:-∪-left; <:-∪-right; <:-∪-lub; <:-∩-left; <:-∩-right; <:-∩-glb; <:-∩-symm; <:-function; <:-function-∪-∩; <:-function-∩-∪; <:-function-∪; <:-everything; <:-union; <:-∪-assocl; <:-∪-assocr; <:-∪-symm; <:-intersect; ∪-distl-∩-<:; ∪-distr-∩-<:; <:-∪-distr-∩; <:-∪-distl-∩; ∩-distl-∪-<:; <:-∩-distl-∪; <:-∩-distr-∪; scalar-∩-function-<:-never; scalar-≢-∩-<:-never) - --- Normal forms for types -data FunType : Type → Set -data Normal : Type → Set - -data FunType where - _⇒_ : ∀ {S T} → Normal S → Normal T → FunType (S ⇒ T) - _∩_ : ∀ {F G} → FunType F → FunType G → FunType (F ∩ G) - -data Normal where - _⇒_ : ∀ {S T} → Normal S → Normal T → Normal (S ⇒ T) - _∩_ : ∀ {F G} → FunType F → FunType G → Normal (F ∩ G) - _∪_ : ∀ {S T} → Normal S → Scalar T → Normal (S ∪ T) - never : Normal never - unknown : Normal unknown - -data OptScalar : Type → Set where - never : OptScalar never - number : OptScalar number - boolean : OptScalar boolean - string : OptScalar string - nil : OptScalar nil - --- Top function type -fun-top : ∀ {F} → (FunType F) → (F <: (never ⇒ unknown)) -fun-top (S ⇒ T) = <:-function <:-never <:-unknown -fun-top (F ∩ G) = <:-trans <:-∩-left (fun-top F) - --- function types are inhabited -fun-function : ∀ {F} → FunType F → Language F function -fun-function (S ⇒ T) = function -fun-function (F ∩ G) = (fun-function F , fun-function G) - -fun-≮:-never : ∀ {F} → FunType F → (F ≮: never) -fun-≮:-never F = witness function (fun-function F) never - --- function types aren't scalars -fun-¬scalar : ∀ {F S t} → (s : Scalar S) → FunType F → Language F t → ¬Language S t -fun-¬scalar s (S ⇒ T) function = scalar-function s -fun-¬scalar s (S ⇒ T) (function-ok₁ p) = scalar-function-ok s -fun-¬scalar s (S ⇒ T) (function-ok₂ p) = scalar-function-ok s -fun-¬scalar s (S ⇒ T) (function-err p) = scalar-function-err s -fun-¬scalar s (S ⇒ T) (function-tgt p) = scalar-function-tgt s -fun-¬scalar s (F ∩ G) (p₁ , p₂) = fun-¬scalar s G p₂ - -¬scalar-fun : ∀ {F S} → FunType F → (s : Scalar S) → ¬Language F (scalar s) -¬scalar-fun (S ⇒ T) s = function-scalar s -¬scalar-fun (F ∩ G) s = left (¬scalar-fun F s) - -scalar-≮:-fun : ∀ {F S} → FunType F → Scalar S → S ≮: F -scalar-≮:-fun F s = witness (scalar s) (scalar s) (¬scalar-fun F s) - -unknown-≮:-fun : ∀ {F} → FunType F → unknown ≮: F -unknown-≮:-fun F = witness (scalar nil) unknown (¬scalar-fun F nil) - --- Normalization produces normal types -normal : ∀ T → Normal (normalize T) -normalᶠ : ∀ {F} → FunType F → Normal F -normal-∪ⁿ : ∀ {S T} → Normal S → Normal T → Normal (S ∪ⁿ T) -normal-∩ⁿ : ∀ {S T} → Normal S → Normal T → Normal (S ∩ⁿ T) -normal-∪ⁿˢ : ∀ {S T} → Normal S → OptScalar T → Normal (S ∪ⁿˢ T) -normal-∩ⁿˢ : ∀ {S T} → Normal S → Scalar T → OptScalar (S ∩ⁿˢ T) -normal-∪ᶠ : ∀ {F G} → FunType F → FunType G → FunType (F ∪ᶠ G) - -normal nil = never ∪ nil -normal (S ⇒ T) = (normal S) ⇒ (normal T) -normal never = never -normal unknown = unknown -normal boolean = never ∪ boolean -normal number = never ∪ number -normal string = never ∪ string -normal (S ∪ T) = normal-∪ⁿ (normal S) (normal T) -normal (S ∩ T) = normal-∩ⁿ (normal S) (normal T) - -normalᶠ (S ⇒ T) = S ⇒ T -normalᶠ (F ∩ G) = F ∩ G - -normal-∪ⁿ S (T₁ ∪ T₂) = (normal-∪ⁿ S T₁) ∪ T₂ -normal-∪ⁿ S never = S -normal-∪ⁿ S unknown = unknown -normal-∪ⁿ never (T ⇒ U) = T ⇒ U -normal-∪ⁿ never (G₁ ∩ G₂) = G₁ ∩ G₂ -normal-∪ⁿ unknown (T ⇒ U) = unknown -normal-∪ⁿ unknown (G₁ ∩ G₂) = unknown -normal-∪ⁿ (R ⇒ S) (T ⇒ U) = normalᶠ (normal-∪ᶠ (R ⇒ S) (T ⇒ U)) -normal-∪ⁿ (R ⇒ S) (G₁ ∩ G₂) = normalᶠ (normal-∪ᶠ (R ⇒ S) (G₁ ∩ G₂)) -normal-∪ⁿ (F₁ ∩ F₂) (T ⇒ U) = normalᶠ (normal-∪ᶠ (F₁ ∩ F₂) (T ⇒ U)) -normal-∪ⁿ (F₁ ∩ F₂) (G₁ ∩ G₂) = normalᶠ (normal-∪ᶠ (F₁ ∩ F₂) (G₁ ∩ G₂)) -normal-∪ⁿ (S₁ ∪ S₂) (T₁ ⇒ T₂) = normal-∪ⁿ S₁ (T₁ ⇒ T₂) ∪ S₂ -normal-∪ⁿ (S₁ ∪ S₂) (G₁ ∩ G₂) = normal-∪ⁿ S₁ (G₁ ∩ G₂) ∪ S₂ - -normal-∩ⁿ S never = never -normal-∩ⁿ S unknown = S -normal-∩ⁿ S (T ∪ U) = normal-∪ⁿˢ (normal-∩ⁿ S T) (normal-∩ⁿˢ S U ) -normal-∩ⁿ never (T ⇒ U) = never -normal-∩ⁿ unknown (T ⇒ U) = T ⇒ U -normal-∩ⁿ (R ⇒ S) (T ⇒ U) = (R ⇒ S) ∩ (T ⇒ U) -normal-∩ⁿ (R ∩ S) (T ⇒ U) = (R ∩ S) ∩ (T ⇒ U) -normal-∩ⁿ (R ∪ S) (T ⇒ U) = normal-∩ⁿ R (T ⇒ U) -normal-∩ⁿ never (T ∩ U) = never -normal-∩ⁿ unknown (T ∩ U) = T ∩ U -normal-∩ⁿ (R ⇒ S) (T ∩ U) = (R ⇒ S) ∩ (T ∩ U) -normal-∩ⁿ (R ∩ S) (T ∩ U) = (R ∩ S) ∩ (T ∩ U) -normal-∩ⁿ (R ∪ S) (T ∩ U) = normal-∩ⁿ R (T ∩ U) - -normal-∪ⁿˢ S never = S -normal-∪ⁿˢ never number = never ∪ number -normal-∪ⁿˢ unknown number = unknown -normal-∪ⁿˢ (R ⇒ S) number = (R ⇒ S) ∪ number -normal-∪ⁿˢ (R ∩ S) number = (R ∩ S) ∪ number -normal-∪ⁿˢ (R ∪ number) number = R ∪ number -normal-∪ⁿˢ (R ∪ boolean) number = normal-∪ⁿˢ R number ∪ boolean -normal-∪ⁿˢ (R ∪ string) number = normal-∪ⁿˢ R number ∪ string -normal-∪ⁿˢ (R ∪ nil) number = normal-∪ⁿˢ R number ∪ nil -normal-∪ⁿˢ never boolean = never ∪ boolean -normal-∪ⁿˢ unknown boolean = unknown -normal-∪ⁿˢ (R ⇒ S) boolean = (R ⇒ S) ∪ boolean -normal-∪ⁿˢ (R ∩ S) boolean = (R ∩ S) ∪ boolean -normal-∪ⁿˢ (R ∪ number) boolean = normal-∪ⁿˢ R boolean ∪ number -normal-∪ⁿˢ (R ∪ boolean) boolean = R ∪ boolean -normal-∪ⁿˢ (R ∪ string) boolean = normal-∪ⁿˢ R boolean ∪ string -normal-∪ⁿˢ (R ∪ nil) boolean = normal-∪ⁿˢ R boolean ∪ nil -normal-∪ⁿˢ never string = never ∪ string -normal-∪ⁿˢ unknown string = unknown -normal-∪ⁿˢ (R ⇒ S) string = (R ⇒ S) ∪ string -normal-∪ⁿˢ (R ∩ S) string = (R ∩ S) ∪ string -normal-∪ⁿˢ (R ∪ number) string = normal-∪ⁿˢ R string ∪ number -normal-∪ⁿˢ (R ∪ boolean) string = normal-∪ⁿˢ R string ∪ boolean -normal-∪ⁿˢ (R ∪ string) string = R ∪ string -normal-∪ⁿˢ (R ∪ nil) string = normal-∪ⁿˢ R string ∪ nil -normal-∪ⁿˢ never nil = never ∪ nil -normal-∪ⁿˢ unknown nil = unknown -normal-∪ⁿˢ (R ⇒ S) nil = (R ⇒ S) ∪ nil -normal-∪ⁿˢ (R ∩ S) nil = (R ∩ S) ∪ nil -normal-∪ⁿˢ (R ∪ number) nil = normal-∪ⁿˢ R nil ∪ number -normal-∪ⁿˢ (R ∪ boolean) nil = normal-∪ⁿˢ R nil ∪ boolean -normal-∪ⁿˢ (R ∪ string) nil = normal-∪ⁿˢ R nil ∪ string -normal-∪ⁿˢ (R ∪ nil) nil = R ∪ nil - -normal-∩ⁿˢ never number = never -normal-∩ⁿˢ never boolean = never -normal-∩ⁿˢ never string = never -normal-∩ⁿˢ never nil = never -normal-∩ⁿˢ unknown number = number -normal-∩ⁿˢ unknown boolean = boolean -normal-∩ⁿˢ unknown string = string -normal-∩ⁿˢ unknown nil = nil -normal-∩ⁿˢ (R ⇒ S) number = never -normal-∩ⁿˢ (R ⇒ S) boolean = never -normal-∩ⁿˢ (R ⇒ S) string = never -normal-∩ⁿˢ (R ⇒ S) nil = never -normal-∩ⁿˢ (R ∩ S) number = never -normal-∩ⁿˢ (R ∩ S) boolean = never -normal-∩ⁿˢ (R ∩ S) string = never -normal-∩ⁿˢ (R ∩ S) nil = never -normal-∩ⁿˢ (R ∪ number) number = number -normal-∩ⁿˢ (R ∪ boolean) number = normal-∩ⁿˢ R number -normal-∩ⁿˢ (R ∪ string) number = normal-∩ⁿˢ R number -normal-∩ⁿˢ (R ∪ nil) number = normal-∩ⁿˢ R number -normal-∩ⁿˢ (R ∪ number) boolean = normal-∩ⁿˢ R boolean -normal-∩ⁿˢ (R ∪ boolean) boolean = boolean -normal-∩ⁿˢ (R ∪ string) boolean = normal-∩ⁿˢ R boolean -normal-∩ⁿˢ (R ∪ nil) boolean = normal-∩ⁿˢ R boolean -normal-∩ⁿˢ (R ∪ number) string = normal-∩ⁿˢ R string -normal-∩ⁿˢ (R ∪ boolean) string = normal-∩ⁿˢ R string -normal-∩ⁿˢ (R ∪ string) string = string -normal-∩ⁿˢ (R ∪ nil) string = normal-∩ⁿˢ R string -normal-∩ⁿˢ (R ∪ number) nil = normal-∩ⁿˢ R nil -normal-∩ⁿˢ (R ∪ boolean) nil = normal-∩ⁿˢ R nil -normal-∩ⁿˢ (R ∪ string) nil = normal-∩ⁿˢ R nil -normal-∩ⁿˢ (R ∪ nil) nil = nil - -normal-∪ᶠ (R ⇒ S) (T ⇒ U) = (normal-∩ⁿ R T) ⇒ (normal-∪ⁿ S U) -normal-∪ᶠ (R ⇒ S) (G ∩ H) = normal-∪ᶠ (R ⇒ S) G ∩ normal-∪ᶠ (R ⇒ S) H -normal-∪ᶠ (E ∩ F) G = normal-∪ᶠ E G ∩ normal-∪ᶠ F G - -scalar-∩-fun-<:-never : ∀ {F S} → FunType F → Scalar S → (F ∩ S) <: never -scalar-∩-fun-<:-never (T ⇒ U) S = scalar-∩-function-<:-never S -scalar-∩-fun-<:-never (F ∩ G) S = <:-trans (<:-intersect <:-∩-left <:-refl) (scalar-∩-fun-<:-never F S) - -flipper : ∀ {S T U} → ((S ∪ T) ∪ U) <: ((S ∪ U) ∪ T) -flipper = <:-trans <:-∪-assocr (<:-trans (<:-union <:-refl <:-∪-symm) <:-∪-assocl) - -∩-<:-∩ⁿ : ∀ {S T} → Normal S → Normal T → (S ∩ T) <: (S ∩ⁿ T) -∩ⁿ-<:-∩ : ∀ {S T} → Normal S → Normal T → (S ∩ⁿ T) <: (S ∩ T) -∩-<:-∩ⁿˢ : ∀ {S T} → Normal S → Scalar T → (S ∩ T) <: (S ∩ⁿˢ T) -∩ⁿˢ-<:-∩ : ∀ {S T} → Normal S → Scalar T → (S ∩ⁿˢ T) <: (S ∩ T) -∪ᶠ-<:-∪ : ∀ {F G} → FunType F → FunType G → (F ∪ᶠ G) <: (F ∪ G) -∪ⁿ-<:-∪ : ∀ {S T} → Normal S → Normal T → (S ∪ⁿ T) <: (S ∪ T) -∪-<:-∪ⁿ : ∀ {S T} → Normal S → Normal T → (S ∪ T) <: (S ∪ⁿ T) -∪ⁿˢ-<:-∪ : ∀ {S T} → Normal S → OptScalar T → (S ∪ⁿˢ T) <: (S ∪ T) -∪-<:-∪ⁿˢ : ∀ {S T} → Normal S → OptScalar T → (S ∪ T) <: (S ∪ⁿˢ T) - -∩-<:-∩ⁿ S never = <:-∩-right -∩-<:-∩ⁿ S unknown = <:-∩-left -∩-<:-∩ⁿ S (T ∪ U) = <:-trans <:-∩-distl-∪ (<:-trans (<:-union (∩-<:-∩ⁿ S T) (∩-<:-∩ⁿˢ S U)) (∪-<:-∪ⁿˢ (normal-∩ⁿ S T) (normal-∩ⁿˢ S U)) ) -∩-<:-∩ⁿ never (T ⇒ U) = <:-∩-left -∩-<:-∩ⁿ unknown (T ⇒ U) = <:-∩-right -∩-<:-∩ⁿ (R ⇒ S) (T ⇒ U) = <:-refl -∩-<:-∩ⁿ (R ∩ S) (T ⇒ U) = <:-refl -∩-<:-∩ⁿ (R ∪ S) (T ⇒ U) = <:-trans <:-∩-distr-∪ (<:-trans (<:-union (∩-<:-∩ⁿ R (T ⇒ U)) (<:-trans <:-∩-symm (∩-<:-∩ⁿˢ (T ⇒ U) S))) (<:-∪-lub <:-refl <:-never)) -∩-<:-∩ⁿ never (T ∩ U) = <:-∩-left -∩-<:-∩ⁿ unknown (T ∩ U) = <:-∩-right -∩-<:-∩ⁿ (R ⇒ S) (T ∩ U) = <:-refl -∩-<:-∩ⁿ (R ∩ S) (T ∩ U) = <:-refl -∩-<:-∩ⁿ (R ∪ S) (T ∩ U) = <:-trans <:-∩-distr-∪ (<:-trans (<:-union (∩-<:-∩ⁿ R (T ∩ U)) (<:-trans <:-∩-symm (∩-<:-∩ⁿˢ (T ∩ U) S))) (<:-∪-lub <:-refl <:-never)) - -∩ⁿ-<:-∩ S never = <:-never -∩ⁿ-<:-∩ S unknown = <:-∩-glb <:-refl <:-unknown -∩ⁿ-<:-∩ S (T ∪ U) = <:-trans (∪ⁿˢ-<:-∪ (normal-∩ⁿ S T) (normal-∩ⁿˢ S U)) (<:-trans (<:-union (∩ⁿ-<:-∩ S T) (∩ⁿˢ-<:-∩ S U)) ∩-distl-∪-<:) -∩ⁿ-<:-∩ never (T ⇒ U) = <:-never -∩ⁿ-<:-∩ unknown (T ⇒ U) = <:-∩-glb <:-unknown <:-refl -∩ⁿ-<:-∩ (R ⇒ S) (T ⇒ U) = <:-refl -∩ⁿ-<:-∩ (R ∩ S) (T ⇒ U) = <:-refl -∩ⁿ-<:-∩ (R ∪ S) (T ⇒ U) = <:-trans (∩ⁿ-<:-∩ R (T ⇒ U)) (<:-∩-glb (<:-trans <:-∩-left <:-∪-left) <:-∩-right) -∩ⁿ-<:-∩ never (T ∩ U) = <:-never -∩ⁿ-<:-∩ unknown (T ∩ U) = <:-∩-glb <:-unknown <:-refl -∩ⁿ-<:-∩ (R ⇒ S) (T ∩ U) = <:-refl -∩ⁿ-<:-∩ (R ∩ S) (T ∩ U) = <:-refl -∩ⁿ-<:-∩ (R ∪ S) (T ∩ U) = <:-trans (∩ⁿ-<:-∩ R (T ∩ U)) (<:-∩-glb (<:-trans <:-∩-left <:-∪-left) <:-∩-right) - -∩-<:-∩ⁿˢ never number = <:-∩-left -∩-<:-∩ⁿˢ never boolean = <:-∩-left -∩-<:-∩ⁿˢ never string = <:-∩-left -∩-<:-∩ⁿˢ never nil = <:-∩-left -∩-<:-∩ⁿˢ unknown T = <:-∩-right -∩-<:-∩ⁿˢ (R ⇒ S) T = scalar-∩-fun-<:-never (R ⇒ S) T -∩-<:-∩ⁿˢ (F ∩ G) T = scalar-∩-fun-<:-never (F ∩ G) T -∩-<:-∩ⁿˢ (R ∪ number) number = <:-∩-right -∩-<:-∩ⁿˢ (R ∪ boolean) number = <:-trans <:-∩-distr-∪ (<:-∪-lub (∩-<:-∩ⁿˢ R number) (scalar-≢-∩-<:-never boolean number (λ ()))) -∩-<:-∩ⁿˢ (R ∪ string) number = <:-trans <:-∩-distr-∪ (<:-∪-lub (∩-<:-∩ⁿˢ R number) (scalar-≢-∩-<:-never string number (λ ()))) -∩-<:-∩ⁿˢ (R ∪ nil) number = <:-trans <:-∩-distr-∪ (<:-∪-lub (∩-<:-∩ⁿˢ R number) (scalar-≢-∩-<:-never nil number (λ ()))) -∩-<:-∩ⁿˢ (R ∪ number) boolean = <:-trans <:-∩-distr-∪ (<:-∪-lub (∩-<:-∩ⁿˢ R boolean) (scalar-≢-∩-<:-never number boolean (λ ()))) -∩-<:-∩ⁿˢ (R ∪ boolean) boolean = <:-∩-right -∩-<:-∩ⁿˢ (R ∪ string) boolean = <:-trans <:-∩-distr-∪ (<:-∪-lub (∩-<:-∩ⁿˢ R boolean) (scalar-≢-∩-<:-never string boolean (λ ()))) -∩-<:-∩ⁿˢ (R ∪ nil) boolean = <:-trans <:-∩-distr-∪ (<:-∪-lub (∩-<:-∩ⁿˢ R boolean) (scalar-≢-∩-<:-never nil boolean (λ ()))) -∩-<:-∩ⁿˢ (R ∪ number) string = <:-trans <:-∩-distr-∪ (<:-∪-lub (∩-<:-∩ⁿˢ R string) (scalar-≢-∩-<:-never number string (λ ()))) -∩-<:-∩ⁿˢ (R ∪ boolean) string = <:-trans <:-∩-distr-∪ (<:-∪-lub (∩-<:-∩ⁿˢ R string) (scalar-≢-∩-<:-never boolean string (λ ()))) -∩-<:-∩ⁿˢ (R ∪ string) string = <:-∩-right -∩-<:-∩ⁿˢ (R ∪ nil) string = <:-trans <:-∩-distr-∪ (<:-∪-lub (∩-<:-∩ⁿˢ R string) (scalar-≢-∩-<:-never nil string (λ ()))) -∩-<:-∩ⁿˢ (R ∪ number) nil = <:-trans <:-∩-distr-∪ (<:-∪-lub (∩-<:-∩ⁿˢ R nil) (scalar-≢-∩-<:-never number nil (λ ()))) -∩-<:-∩ⁿˢ (R ∪ boolean) nil = <:-trans <:-∩-distr-∪ (<:-∪-lub (∩-<:-∩ⁿˢ R nil) (scalar-≢-∩-<:-never boolean nil (λ ()))) -∩-<:-∩ⁿˢ (R ∪ string) nil = <:-trans <:-∩-distr-∪ (<:-∪-lub (∩-<:-∩ⁿˢ R nil) (scalar-≢-∩-<:-never string nil (λ ()))) -∩-<:-∩ⁿˢ (R ∪ nil) nil = <:-∩-right - -∩ⁿˢ-<:-∩ never T = <:-never -∩ⁿˢ-<:-∩ unknown T = <:-∩-glb <:-unknown <:-refl -∩ⁿˢ-<:-∩ (R ⇒ S) T = <:-never -∩ⁿˢ-<:-∩ (F ∩ G) T = <:-never -∩ⁿˢ-<:-∩ (R ∪ number) number = <:-∩-glb <:-∪-right <:-refl -∩ⁿˢ-<:-∩ (R ∪ boolean) number = <:-trans (∩ⁿˢ-<:-∩ R number) (<:-intersect <:-∪-left <:-refl) -∩ⁿˢ-<:-∩ (R ∪ string) number = <:-trans (∩ⁿˢ-<:-∩ R number) (<:-intersect <:-∪-left <:-refl) -∩ⁿˢ-<:-∩ (R ∪ nil) number = <:-trans (∩ⁿˢ-<:-∩ R number) (<:-intersect <:-∪-left <:-refl) -∩ⁿˢ-<:-∩ (R ∪ number) boolean = <:-trans (∩ⁿˢ-<:-∩ R boolean) (<:-intersect <:-∪-left <:-refl) -∩ⁿˢ-<:-∩ (R ∪ boolean) boolean = <:-∩-glb <:-∪-right <:-refl -∩ⁿˢ-<:-∩ (R ∪ string) boolean = <:-trans (∩ⁿˢ-<:-∩ R boolean) (<:-intersect <:-∪-left <:-refl) -∩ⁿˢ-<:-∩ (R ∪ nil) boolean = <:-trans (∩ⁿˢ-<:-∩ R boolean) (<:-intersect <:-∪-left <:-refl) -∩ⁿˢ-<:-∩ (R ∪ number) string = <:-trans (∩ⁿˢ-<:-∩ R string) (<:-intersect <:-∪-left <:-refl) -∩ⁿˢ-<:-∩ (R ∪ boolean) string = <:-trans (∩ⁿˢ-<:-∩ R string) (<:-intersect <:-∪-left <:-refl) -∩ⁿˢ-<:-∩ (R ∪ string) string = <:-∩-glb <:-∪-right <:-refl -∩ⁿˢ-<:-∩ (R ∪ nil) string = <:-trans (∩ⁿˢ-<:-∩ R string) (<:-intersect <:-∪-left <:-refl) -∩ⁿˢ-<:-∩ (R ∪ number) nil = <:-trans (∩ⁿˢ-<:-∩ R nil) (<:-intersect <:-∪-left <:-refl) -∩ⁿˢ-<:-∩ (R ∪ boolean) nil = <:-trans (∩ⁿˢ-<:-∩ R nil) (<:-intersect <:-∪-left <:-refl) -∩ⁿˢ-<:-∩ (R ∪ string) nil = <:-trans (∩ⁿˢ-<:-∩ R nil) (<:-intersect <:-∪-left <:-refl) -∩ⁿˢ-<:-∩ (R ∪ nil) nil = <:-∩-glb <:-∪-right <:-refl - -∪ᶠ-<:-∪ (R ⇒ S) (T ⇒ U) = <:-trans (<:-function (∩-<:-∩ⁿ R T) (∪ⁿ-<:-∪ S U)) <:-function-∪-∩ -∪ᶠ-<:-∪ (R ⇒ S) (G ∩ H) = <:-trans (<:-intersect (∪ᶠ-<:-∪ (R ⇒ S) G) (∪ᶠ-<:-∪ (R ⇒ S) H)) ∪-distl-∩-<: -∪ᶠ-<:-∪ (E ∩ F) G = <:-trans (<:-intersect (∪ᶠ-<:-∪ E G) (∪ᶠ-<:-∪ F G)) ∪-distr-∩-<: - -∪-<:-∪ᶠ : ∀ {F G} → FunType F → FunType G → (F ∪ G) <: (F ∪ᶠ G) -∪-<:-∪ᶠ (R ⇒ S) (T ⇒ U) = <:-trans <:-function-∪ (<:-function (∩ⁿ-<:-∩ R T) (∪-<:-∪ⁿ S U)) -∪-<:-∪ᶠ (R ⇒ S) (G ∩ H) = <:-trans <:-∪-distl-∩ (<:-intersect (∪-<:-∪ᶠ (R ⇒ S) G) (∪-<:-∪ᶠ (R ⇒ S) H)) -∪-<:-∪ᶠ (E ∩ F) G = <:-trans <:-∪-distr-∩ (<:-intersect (∪-<:-∪ᶠ E G) (∪-<:-∪ᶠ F G)) - -∪ⁿˢ-<:-∪ S never = <:-∪-left -∪ⁿˢ-<:-∪ never number = <:-refl -∪ⁿˢ-<:-∪ never boolean = <:-refl -∪ⁿˢ-<:-∪ never string = <:-refl -∪ⁿˢ-<:-∪ never nil = <:-refl -∪ⁿˢ-<:-∪ unknown number = <:-∪-left -∪ⁿˢ-<:-∪ unknown boolean = <:-∪-left -∪ⁿˢ-<:-∪ unknown string = <:-∪-left -∪ⁿˢ-<:-∪ unknown nil = <:-∪-left -∪ⁿˢ-<:-∪ (R ⇒ S) number = <:-refl -∪ⁿˢ-<:-∪ (R ⇒ S) boolean = <:-refl -∪ⁿˢ-<:-∪ (R ⇒ S) string = <:-refl -∪ⁿˢ-<:-∪ (R ⇒ S) nil = <:-refl -∪ⁿˢ-<:-∪ (R ∩ S) number = <:-refl -∪ⁿˢ-<:-∪ (R ∩ S) boolean = <:-refl -∪ⁿˢ-<:-∪ (R ∩ S) string = <:-refl -∪ⁿˢ-<:-∪ (R ∩ S) nil = <:-refl -∪ⁿˢ-<:-∪ (R ∪ number) number = <:-union <:-∪-left <:-refl -∪ⁿˢ-<:-∪ (R ∪ boolean) number = <:-trans (<:-union (∪ⁿˢ-<:-∪ R number) <:-refl) flipper -∪ⁿˢ-<:-∪ (R ∪ string) number = <:-trans (<:-union (∪ⁿˢ-<:-∪ R number) <:-refl) flipper -∪ⁿˢ-<:-∪ (R ∪ nil) number = <:-trans (<:-union (∪ⁿˢ-<:-∪ R number) <:-refl) flipper -∪ⁿˢ-<:-∪ (R ∪ number) boolean = <:-trans (<:-union (∪ⁿˢ-<:-∪ R boolean) <:-refl) flipper -∪ⁿˢ-<:-∪ (R ∪ boolean) boolean = <:-union <:-∪-left <:-refl -∪ⁿˢ-<:-∪ (R ∪ string) boolean = <:-trans (<:-union (∪ⁿˢ-<:-∪ R boolean) <:-refl) flipper -∪ⁿˢ-<:-∪ (R ∪ nil) boolean = <:-trans (<:-union (∪ⁿˢ-<:-∪ R boolean) <:-refl) flipper -∪ⁿˢ-<:-∪ (R ∪ number) string = <:-trans (<:-union (∪ⁿˢ-<:-∪ R string) <:-refl) flipper -∪ⁿˢ-<:-∪ (R ∪ boolean) string = <:-trans (<:-union (∪ⁿˢ-<:-∪ R string) <:-refl) flipper -∪ⁿˢ-<:-∪ (R ∪ string) string = <:-union <:-∪-left <:-refl -∪ⁿˢ-<:-∪ (R ∪ nil) string = <:-trans (<:-union (∪ⁿˢ-<:-∪ R string) <:-refl) flipper -∪ⁿˢ-<:-∪ (R ∪ number) nil = <:-trans (<:-union (∪ⁿˢ-<:-∪ R nil) <:-refl) flipper -∪ⁿˢ-<:-∪ (R ∪ boolean) nil = <:-trans (<:-union (∪ⁿˢ-<:-∪ R nil) <:-refl) flipper -∪ⁿˢ-<:-∪ (R ∪ string) nil = <:-trans (<:-union (∪ⁿˢ-<:-∪ R nil) <:-refl) flipper -∪ⁿˢ-<:-∪ (R ∪ nil) nil = <:-union <:-∪-left <:-refl - -∪-<:-∪ⁿˢ T never = <:-∪-lub <:-refl <:-never -∪-<:-∪ⁿˢ never number = <:-refl -∪-<:-∪ⁿˢ never boolean = <:-refl -∪-<:-∪ⁿˢ never string = <:-refl -∪-<:-∪ⁿˢ never nil = <:-refl -∪-<:-∪ⁿˢ unknown number = <:-unknown -∪-<:-∪ⁿˢ unknown boolean = <:-unknown -∪-<:-∪ⁿˢ unknown string = <:-unknown -∪-<:-∪ⁿˢ unknown nil = <:-unknown -∪-<:-∪ⁿˢ (R ⇒ S) number = <:-refl -∪-<:-∪ⁿˢ (R ⇒ S) boolean = <:-refl -∪-<:-∪ⁿˢ (R ⇒ S) string = <:-refl -∪-<:-∪ⁿˢ (R ⇒ S) nil = <:-refl -∪-<:-∪ⁿˢ (R ∩ S) number = <:-refl -∪-<:-∪ⁿˢ (R ∩ S) boolean = <:-refl -∪-<:-∪ⁿˢ (R ∩ S) string = <:-refl -∪-<:-∪ⁿˢ (R ∩ S) nil = <:-refl -∪-<:-∪ⁿˢ (R ∪ number) number = <:-∪-lub <:-refl <:-∪-right -∪-<:-∪ⁿˢ (R ∪ boolean) number = <:-trans flipper (<:-union (∪-<:-∪ⁿˢ R number) <:-refl) -∪-<:-∪ⁿˢ (R ∪ string) number = <:-trans flipper (<:-union (∪-<:-∪ⁿˢ R number) <:-refl) -∪-<:-∪ⁿˢ (R ∪ nil) number = <:-trans flipper (<:-union (∪-<:-∪ⁿˢ R number) <:-refl) -∪-<:-∪ⁿˢ (R ∪ number) boolean = <:-trans flipper (<:-union (∪-<:-∪ⁿˢ R boolean) <:-refl) -∪-<:-∪ⁿˢ (R ∪ boolean) boolean = <:-∪-lub <:-refl <:-∪-right -∪-<:-∪ⁿˢ (R ∪ string) boolean = <:-trans flipper (<:-union (∪-<:-∪ⁿˢ R boolean) <:-refl) -∪-<:-∪ⁿˢ (R ∪ nil) boolean = <:-trans flipper (<:-union (∪-<:-∪ⁿˢ R boolean) <:-refl) -∪-<:-∪ⁿˢ (R ∪ number) string = <:-trans flipper (<:-union (∪-<:-∪ⁿˢ R string) <:-refl) -∪-<:-∪ⁿˢ (R ∪ boolean) string = <:-trans flipper (<:-union (∪-<:-∪ⁿˢ R string) <:-refl) -∪-<:-∪ⁿˢ (R ∪ string) string = <:-∪-lub <:-refl <:-∪-right -∪-<:-∪ⁿˢ (R ∪ nil) string = <:-trans flipper (<:-union (∪-<:-∪ⁿˢ R string) <:-refl) -∪-<:-∪ⁿˢ (R ∪ number) nil = <:-trans flipper (<:-union (∪-<:-∪ⁿˢ R nil) <:-refl) -∪-<:-∪ⁿˢ (R ∪ boolean) nil = <:-trans flipper (<:-union (∪-<:-∪ⁿˢ R nil) <:-refl) -∪-<:-∪ⁿˢ (R ∪ string) nil = <:-trans flipper (<:-union (∪-<:-∪ⁿˢ R nil) <:-refl) -∪-<:-∪ⁿˢ (R ∪ nil) nil = <:-∪-lub <:-refl <:-∪-right - -∪ⁿ-<:-∪ S never = <:-∪-left -∪ⁿ-<:-∪ S unknown = <:-∪-right -∪ⁿ-<:-∪ never (T ⇒ U) = <:-∪-right -∪ⁿ-<:-∪ unknown (T ⇒ U) = <:-∪-left -∪ⁿ-<:-∪ (R ⇒ S) (T ⇒ U) = ∪ᶠ-<:-∪ (R ⇒ S) (T ⇒ U) -∪ⁿ-<:-∪ (R ∩ S) (T ⇒ U) = ∪ᶠ-<:-∪ (R ∩ S) (T ⇒ U) -∪ⁿ-<:-∪ (R ∪ S) (T ⇒ U) = <:-trans (<:-union (∪ⁿ-<:-∪ R (T ⇒ U)) <:-refl) (<:-∪-lub (<:-∪-lub (<:-trans <:-∪-left <:-∪-left) <:-∪-right) (<:-trans <:-∪-right <:-∪-left)) -∪ⁿ-<:-∪ never (T ∩ U) = <:-∪-right -∪ⁿ-<:-∪ unknown (T ∩ U) = <:-∪-left -∪ⁿ-<:-∪ (R ⇒ S) (T ∩ U) = ∪ᶠ-<:-∪ (R ⇒ S) (T ∩ U) -∪ⁿ-<:-∪ (R ∩ S) (T ∩ U) = ∪ᶠ-<:-∪ (R ∩ S) (T ∩ U) -∪ⁿ-<:-∪ (R ∪ S) (T ∩ U) = <:-trans (<:-union (∪ⁿ-<:-∪ R (T ∩ U)) <:-refl) (<:-∪-lub (<:-∪-lub (<:-trans <:-∪-left <:-∪-left) <:-∪-right) (<:-trans <:-∪-right <:-∪-left)) -∪ⁿ-<:-∪ S (T ∪ U) = <:-∪-lub (<:-trans (∪ⁿ-<:-∪ S T) (<:-union <:-refl <:-∪-left)) (<:-trans <:-∪-right <:-∪-right) - -∪-<:-∪ⁿ S never = <:-∪-lub <:-refl <:-never -∪-<:-∪ⁿ S unknown = <:-unknown -∪-<:-∪ⁿ never (T ⇒ U) = <:-∪-lub <:-never <:-refl -∪-<:-∪ⁿ unknown (T ⇒ U) = <:-unknown -∪-<:-∪ⁿ (R ⇒ S) (T ⇒ U) = ∪-<:-∪ᶠ (R ⇒ S) (T ⇒ U) -∪-<:-∪ⁿ (R ∩ S) (T ⇒ U) = ∪-<:-∪ᶠ (R ∩ S) (T ⇒ U) -∪-<:-∪ⁿ (R ∪ S) (T ⇒ U) = <:-trans <:-∪-assocr (<:-trans (<:-union <:-refl <:-∪-symm) (<:-trans <:-∪-assocl (<:-union (∪-<:-∪ⁿ R (T ⇒ U)) <:-refl))) -∪-<:-∪ⁿ never (T ∩ U) = <:-∪-lub <:-never <:-refl -∪-<:-∪ⁿ unknown (T ∩ U) = <:-unknown -∪-<:-∪ⁿ (R ⇒ S) (T ∩ U) = ∪-<:-∪ᶠ (R ⇒ S) (T ∩ U) -∪-<:-∪ⁿ (R ∩ S) (T ∩ U) = ∪-<:-∪ᶠ (R ∩ S) (T ∩ U) -∪-<:-∪ⁿ (R ∪ S) (T ∩ U) = <:-trans <:-∪-assocr (<:-trans (<:-union <:-refl <:-∪-symm) (<:-trans <:-∪-assocl (<:-union (∪-<:-∪ⁿ R (T ∩ U)) <:-refl))) -∪-<:-∪ⁿ never (T ∪ U) = <:-trans <:-∪-assocl (<:-union (∪-<:-∪ⁿ never T) <:-refl) -∪-<:-∪ⁿ unknown (T ∪ U) = <:-trans <:-∪-assocl (<:-union (∪-<:-∪ⁿ unknown T) <:-refl) -∪-<:-∪ⁿ (R ⇒ S) (T ∪ U) = <:-trans <:-∪-assocl (<:-union (∪-<:-∪ⁿ (R ⇒ S) T) <:-refl) -∪-<:-∪ⁿ (R ∩ S) (T ∪ U) = <:-trans <:-∪-assocl (<:-union (∪-<:-∪ⁿ (R ∩ S) T) <:-refl) -∪-<:-∪ⁿ (R ∪ S) (T ∪ U) = <:-trans <:-∪-assocl (<:-union (∪-<:-∪ⁿ (R ∪ S) T) <:-refl) - -normalize-<: : ∀ T → normalize T <: T -<:-normalize : ∀ T → T <: normalize T - -<:-normalize nil = <:-∪-right -<:-normalize (S ⇒ T) = <:-function (normalize-<: S) (<:-normalize T) -<:-normalize never = <:-refl -<:-normalize unknown = <:-refl -<:-normalize boolean = <:-∪-right -<:-normalize number = <:-∪-right -<:-normalize string = <:-∪-right -<:-normalize (S ∪ T) = <:-trans (<:-union (<:-normalize S) (<:-normalize T)) (∪-<:-∪ⁿ (normal S) (normal T)) -<:-normalize (S ∩ T) = <:-trans (<:-intersect (<:-normalize S) (<:-normalize T)) (∩-<:-∩ⁿ (normal S) (normal T)) - -normalize-<: nil = <:-∪-lub <:-never <:-refl -normalize-<: (S ⇒ T) = <:-function (<:-normalize S) (normalize-<: T) -normalize-<: never = <:-refl -normalize-<: unknown = <:-refl -normalize-<: boolean = <:-∪-lub <:-never <:-refl -normalize-<: number = <:-∪-lub <:-never <:-refl -normalize-<: string = <:-∪-lub <:-never <:-refl -normalize-<: (S ∪ T) = <:-trans (∪ⁿ-<:-∪ (normal S) (normal T)) (<:-union (normalize-<: S) (normalize-<: T)) -normalize-<: (S ∩ T) = <:-trans (∩ⁿ-<:-∩ (normal S) (normal T)) (<:-intersect (normalize-<: S) (normalize-<: T)) - - diff --git a/prototyping/Properties/TypeSaturation.agda b/prototyping/Properties/TypeSaturation.agda deleted file mode 100644 index 13f7d171..00000000 --- a/prototyping/Properties/TypeSaturation.agda +++ /dev/null @@ -1,433 +0,0 @@ -{-# OPTIONS --rewriting #-} - -module Properties.TypeSaturation where - -open import Agda.Builtin.Equality using (_≡_; refl) -open import FFI.Data.Either using (Either; Left; Right) -open import Luau.Subtyping using (Tree; Language; ¬Language; _<:_; _≮:_; witness; scalar; function; function-err; function-ok; function-ok₁; function-ok₂; scalar-function; _,_; never) -open import Luau.Type using (Type; _⇒_; _∩_; _∪_; never; unknown) -open import Luau.TypeNormalization using (_∩ⁿ_; _∪ⁿ_) -open import Luau.TypeSaturation using (_⋓_; _⋒_; _∩ᵘ_; _∩ⁱ_; ∪-saturate; ∩-saturate; saturate) -open import Properties.Subtyping using (dec-language; language-comp; <:-impl-⊇; <:-refl; <:-trans; <:-trans-≮:; <:-impl-¬≮: ; <:-never; <:-unknown; <:-function; <:-union; <:-∪-symm; <:-∪-left; <:-∪-right; <:-∪-lub; <:-∪-assocl; <:-∪-assocr; <:-intersect; <:-∩-symm; <:-∩-left; <:-∩-right; <:-∩-glb; ≮:-function-left; ≮:-function-right; <:-function-∩-∪; <:-function-∩-∩; <:-∩-assocl; <:-∩-assocr; ∩-<:-∪; <:-∩-distl-∪; ∩-distl-∪-<:; <:-∩-distr-∪; ∩-distr-∪-<:) -open import Properties.TypeNormalization using (Normal; FunType; _⇒_; _∩_; _∪_; never; unknown; normal-∪ⁿ; normal-∩ⁿ; ∪ⁿ-<:-∪; ∪-<:-∪ⁿ; ∩ⁿ-<:-∩; ∩-<:-∩ⁿ) -open import Properties.Contradiction using (CONTRADICTION) -open import Properties.Functions using (_∘_) - --- Saturation preserves normalization -normal-⋒ : ∀ {F G} → FunType F → FunType G → FunType (F ⋒ G) -normal-⋒ (R ⇒ S) (T ⇒ U) = (normal-∩ⁿ R T) ⇒ (normal-∩ⁿ S U) -normal-⋒ (R ⇒ S) (G ∩ H) = normal-⋒ (R ⇒ S) G ∩ normal-⋒ (R ⇒ S) H -normal-⋒ (E ∩ F) G = normal-⋒ E G ∩ normal-⋒ F G - -normal-⋓ : ∀ {F G} → FunType F → FunType G → FunType (F ⋓ G) -normal-⋓ (R ⇒ S) (T ⇒ U) = (normal-∪ⁿ R T) ⇒ (normal-∪ⁿ S U) -normal-⋓ (R ⇒ S) (G ∩ H) = normal-⋓ (R ⇒ S) G ∩ normal-⋓ (R ⇒ S) H -normal-⋓ (E ∩ F) G = normal-⋓ E G ∩ normal-⋓ F G - -normal-∩-saturate : ∀ {F} → FunType F → FunType (∩-saturate F) -normal-∩-saturate (S ⇒ T) = S ⇒ T -normal-∩-saturate (F ∩ G) = (normal-∩-saturate F ∩ normal-∩-saturate G) ∩ normal-⋒ (normal-∩-saturate F) (normal-∩-saturate G) - -normal-∪-saturate : ∀ {F} → FunType F → FunType (∪-saturate F) -normal-∪-saturate (S ⇒ T) = S ⇒ T -normal-∪-saturate (F ∩ G) = (normal-∪-saturate F ∩ normal-∪-saturate G) ∩ normal-⋓ (normal-∪-saturate F) (normal-∪-saturate G) - -normal-saturate : ∀ {F} → FunType F → FunType (saturate F) -normal-saturate F = normal-∪-saturate (normal-∩-saturate F) - --- Saturation resects subtyping -∪-saturate-<: : ∀ {F} → FunType F → ∪-saturate F <: F -∪-saturate-<: (S ⇒ T) = <:-refl -∪-saturate-<: (F ∩ G) = <:-trans <:-∩-left (<:-intersect (∪-saturate-<: F) (∪-saturate-<: G)) - -∩-saturate-<: : ∀ {F} → FunType F → ∩-saturate F <: F -∩-saturate-<: (S ⇒ T) = <:-refl -∩-saturate-<: (F ∩ G) = <:-trans <:-∩-left (<:-intersect (∩-saturate-<: F) (∩-saturate-<: G)) - -saturate-<: : ∀ {F} → FunType F → saturate F <: F -saturate-<: F = <:-trans (∪-saturate-<: (normal-∩-saturate F)) (∩-saturate-<: F) - -∩-<:-⋓ : ∀ {F G} → FunType F → FunType G → (F ∩ G) <: (F ⋓ G) -∩-<:-⋓ (R ⇒ S) (T ⇒ U) = <:-trans <:-function-∩-∪ (<:-function (∪ⁿ-<:-∪ R T) (∪-<:-∪ⁿ S U)) -∩-<:-⋓ (R ⇒ S) (G ∩ H) = <:-trans (<:-∩-glb (<:-intersect <:-refl <:-∩-left) (<:-intersect <:-refl <:-∩-right)) (<:-intersect (∩-<:-⋓ (R ⇒ S) G) (∩-<:-⋓ (R ⇒ S) H)) -∩-<:-⋓ (E ∩ F) G = <:-trans (<:-∩-glb (<:-intersect <:-∩-left <:-refl) (<:-intersect <:-∩-right <:-refl)) (<:-intersect (∩-<:-⋓ E G) (∩-<:-⋓ F G)) - -∩-<:-⋒ : ∀ {F G} → FunType F → FunType G → (F ∩ G) <: (F ⋒ G) -∩-<:-⋒ (R ⇒ S) (T ⇒ U) = <:-trans <:-function-∩-∩ (<:-function (∩ⁿ-<:-∩ R T) (∩-<:-∩ⁿ S U)) -∩-<:-⋒ (R ⇒ S) (G ∩ H) = <:-trans (<:-∩-glb (<:-intersect <:-refl <:-∩-left) (<:-intersect <:-refl <:-∩-right)) (<:-intersect (∩-<:-⋒ (R ⇒ S) G) (∩-<:-⋒ (R ⇒ S) H)) -∩-<:-⋒ (E ∩ F) G = <:-trans (<:-∩-glb (<:-intersect <:-∩-left <:-refl) (<:-intersect <:-∩-right <:-refl)) (<:-intersect (∩-<:-⋒ E G) (∩-<:-⋒ F G)) - -<:-∪-saturate : ∀ {F} → FunType F → F <: ∪-saturate F -<:-∪-saturate (S ⇒ T) = <:-refl -<:-∪-saturate (F ∩ G) = <:-∩-glb (<:-intersect (<:-∪-saturate F) (<:-∪-saturate G)) (<:-trans (<:-intersect (<:-∪-saturate F) (<:-∪-saturate G)) (∩-<:-⋓ (normal-∪-saturate F) (normal-∪-saturate G))) - -<:-∩-saturate : ∀ {F} → FunType F → F <: ∩-saturate F -<:-∩-saturate (S ⇒ T) = <:-refl -<:-∩-saturate (F ∩ G) = <:-∩-glb (<:-intersect (<:-∩-saturate F) (<:-∩-saturate G)) (<:-trans (<:-intersect (<:-∩-saturate F) (<:-∩-saturate G)) (∩-<:-⋒ (normal-∩-saturate F) (normal-∩-saturate G))) - -<:-saturate : ∀ {F} → FunType F → F <: saturate F -<:-saturate F = <:-trans (<:-∩-saturate F) (<:-∪-saturate (normal-∩-saturate F)) - --- Overloads F is the set of overloads of F -data Overloads : Type → Type → Set where - - here : ∀ {S T} → Overloads (S ⇒ T) (S ⇒ T) - left : ∀ {S T F G} → Overloads F (S ⇒ T) → Overloads (F ∩ G) (S ⇒ T) - right : ∀ {S T F G} → Overloads G (S ⇒ T) → Overloads (F ∩ G) (S ⇒ T) - -normal-overload-src : ∀ {F S T} → FunType F → Overloads F (S ⇒ T) → Normal S -normal-overload-src (S ⇒ T) here = S -normal-overload-src (F ∩ G) (left o) = normal-overload-src F o -normal-overload-src (F ∩ G) (right o) = normal-overload-src G o - -normal-overload-tgt : ∀ {F S T} → FunType F → Overloads F (S ⇒ T) → Normal T -normal-overload-tgt (S ⇒ T) here = T -normal-overload-tgt (F ∩ G) (left o) = normal-overload-tgt F o -normal-overload-tgt (F ∩ G) (right o) = normal-overload-tgt G o - --- An inductive presentation of the overloads of F ⋓ G -data ∪-Lift (P Q : Type → Set) : Type → Set where - - union : ∀ {R S T U} → - - P (R ⇒ S) → - Q (T ⇒ U) → - -------------------- - ∪-Lift P Q ((R ∪ T) ⇒ (S ∪ U)) - --- An inductive presentation of the overloads of F ⋒ G -data ∩-Lift (P Q : Type → Set) : Type → Set where - - intersect : ∀ {R S T U} → - - P (R ⇒ S) → - Q (T ⇒ U) → - -------------------- - ∩-Lift P Q ((R ∩ T) ⇒ (S ∩ U)) - --- An inductive presentation of the overloads of ∪-saturate F -data ∪-Saturate (P : Type → Set) : Type → Set where - - base : ∀ {S T} → - - P (S ⇒ T) → - -------------------- - ∪-Saturate P (S ⇒ T) - - union : ∀ {R S T U} → - - ∪-Saturate P (R ⇒ S) → - ∪-Saturate P (T ⇒ U) → - -------------------- - ∪-Saturate P ((R ∪ T) ⇒ (S ∪ U)) - --- An inductive presentation of the overloads of ∩-saturate F -data ∩-Saturate (P : Type → Set) : Type → Set where - - base : ∀ {S T} → - - P (S ⇒ T) → - -------------------- - ∩-Saturate P (S ⇒ T) - - intersect : ∀ {R S T U} → - - ∩-Saturate P (R ⇒ S) → - ∩-Saturate P (T ⇒ U) → - -------------------- - ∩-Saturate P ((R ∩ T) ⇒ (S ∩ U)) - --- The <:-up-closure of a set of function types -data <:-Close (P : Type → Set) : Type → Set where - - defn : ∀ {R S T U} → - - P (S ⇒ T) → - R <: S → - T <: U → - ------------------ - <:-Close P (R ⇒ U) - --- F ⊆ᵒ G whenever every overload of F is an overload of G -_⊆ᵒ_ : Type → Type → Set -F ⊆ᵒ G = ∀ {S T} → Overloads F (S ⇒ T) → Overloads G (S ⇒ T) - --- F <:ᵒ G when every overload of G is a supertype of an overload of F -_<:ᵒ_ : Type → Type → Set -_<:ᵒ_ F G = ∀ {S T} → Overloads G (S ⇒ T) → <:-Close (Overloads F) (S ⇒ T) - --- P ⊂: Q when any type in P is a subtype of some type in Q -_⊂:_ : (Type → Set) → (Type → Set) → Set -P ⊂: Q = ∀ {S T} → P (S ⇒ T) → <:-Close Q (S ⇒ T) - --- <:-Close is a monad -just : ∀ {P S T} → P (S ⇒ T) → <:-Close P (S ⇒ T) -just p = defn p <:-refl <:-refl - -infixl 5 _>>=_ _>>=ˡ_ _>>=ʳ_ -_>>=_ : ∀ {P Q S T} → <:-Close P (S ⇒ T) → (P ⊂: Q) → <:-Close Q (S ⇒ T) -(defn p p₁ p₂) >>= P⊂Q with P⊂Q p -(defn p p₁ p₂) >>= P⊂Q | defn q q₁ q₂ = defn q (<:-trans p₁ q₁) (<:-trans q₂ p₂) - -_>>=ˡ_ : ∀ {P R S T} → <:-Close P (S ⇒ T) → (R <: S) → <:-Close P (R ⇒ T) -(defn p p₁ p₂) >>=ˡ q = defn p (<:-trans q p₁) p₂ - -_>>=ʳ_ : ∀ {P S T U} → <:-Close P (S ⇒ T) → (T <: U) → <:-Close P (S ⇒ U) -(defn p p₁ p₂) >>=ʳ q = defn p p₁ (<:-trans p₂ q) - --- Properties of ⊂: -⊂:-refl : ∀ {P} → P ⊂: P -⊂:-refl p = just p - -_[∪]_ : ∀ {P Q R S T U} → <:-Close P (R ⇒ S) → <:-Close Q (T ⇒ U) → <:-Close (∪-Lift P Q) ((R ∪ T) ⇒ (S ∪ U)) -(defn p p₁ p₂) [∪] (defn q q₁ q₂) = defn (union p q) (<:-union p₁ q₁) (<:-union p₂ q₂) - -_[∩]_ : ∀ {P Q R S T U} → <:-Close P (R ⇒ S) → <:-Close Q (T ⇒ U) → <:-Close (∩-Lift P Q) ((R ∩ T) ⇒ (S ∩ U)) -(defn p p₁ p₂) [∩] (defn q q₁ q₂) = defn (intersect p q) (<:-intersect p₁ q₁) (<:-intersect p₂ q₂) - -⊂:-∩-saturate-inj : ∀ {P} → P ⊂: ∩-Saturate P -⊂:-∩-saturate-inj p = defn (base p) <:-refl <:-refl - -⊂:-∪-saturate-inj : ∀ {P} → P ⊂: ∪-Saturate P -⊂:-∪-saturate-inj p = just (base p) - -⊂:-∩-lift-saturate : ∀ {P} → ∩-Lift (∩-Saturate P) (∩-Saturate P) ⊂: ∩-Saturate P -⊂:-∩-lift-saturate (intersect p q) = just (intersect p q) - -⊂:-∪-lift-saturate : ∀ {P} → ∪-Lift (∪-Saturate P) (∪-Saturate P) ⊂: ∪-Saturate P -⊂:-∪-lift-saturate (union p q) = just (union p q) - -⊂:-∩-lift : ∀ {P Q R S} → (P ⊂: Q) → (R ⊂: S) → (∩-Lift P R ⊂: ∩-Lift Q S) -⊂:-∩-lift P⊂Q R⊂S (intersect n o) = P⊂Q n [∩] R⊂S o - -⊂:-∪-lift : ∀ {P Q R S} → (P ⊂: Q) → (R ⊂: S) → (∪-Lift P R ⊂: ∪-Lift Q S) -⊂:-∪-lift P⊂Q R⊂S (union n o) = P⊂Q n [∪] R⊂S o - -⊂:-∩-saturate : ∀ {P Q} → (P ⊂: Q) → (∩-Saturate P ⊂: ∩-Saturate Q) -⊂:-∩-saturate P⊂Q (base p) = P⊂Q p >>= ⊂:-∩-saturate-inj -⊂:-∩-saturate P⊂Q (intersect p q) = (⊂:-∩-saturate P⊂Q p [∩] ⊂:-∩-saturate P⊂Q q) >>= ⊂:-∩-lift-saturate - -⊂:-∪-saturate : ∀ {P Q} → (P ⊂: Q) → (∪-Saturate P ⊂: ∪-Saturate Q) -⊂:-∪-saturate P⊂Q (base p) = P⊂Q p >>= ⊂:-∪-saturate-inj -⊂:-∪-saturate P⊂Q (union p q) = (⊂:-∪-saturate P⊂Q p [∪] ⊂:-∪-saturate P⊂Q q) >>= ⊂:-∪-lift-saturate - -⊂:-∩-saturate-indn : ∀ {P Q} → (P ⊂: Q) → (∩-Lift Q Q ⊂: Q) → (∩-Saturate P ⊂: Q) -⊂:-∩-saturate-indn P⊂Q QQ⊂Q (base p) = P⊂Q p -⊂:-∩-saturate-indn P⊂Q QQ⊂Q (intersect p q) = (⊂:-∩-saturate-indn P⊂Q QQ⊂Q p [∩] ⊂:-∩-saturate-indn P⊂Q QQ⊂Q q) >>= QQ⊂Q - -⊂:-∪-saturate-indn : ∀ {P Q} → (P ⊂: Q) → (∪-Lift Q Q ⊂: Q) → (∪-Saturate P ⊂: Q) -⊂:-∪-saturate-indn P⊂Q QQ⊂Q (base p) = P⊂Q p -⊂:-∪-saturate-indn P⊂Q QQ⊂Q (union p q) = (⊂:-∪-saturate-indn P⊂Q QQ⊂Q p [∪] ⊂:-∪-saturate-indn P⊂Q QQ⊂Q q) >>= QQ⊂Q - -∪-saturate-resp-∩-saturation : ∀ {P} → (∩-Lift P P ⊂: P) → (∩-Lift (∪-Saturate P) (∪-Saturate P) ⊂: ∪-Saturate P) -∪-saturate-resp-∩-saturation ∩P⊂P (intersect (base p) (base q)) = ∩P⊂P (intersect p q) >>= ⊂:-∪-saturate-inj -∪-saturate-resp-∩-saturation ∩P⊂P (intersect p (union q q₁)) = (∪-saturate-resp-∩-saturation ∩P⊂P (intersect p q) [∪] ∪-saturate-resp-∩-saturation ∩P⊂P (intersect p q₁)) >>= ⊂:-∪-lift-saturate >>=ˡ <:-∩-distl-∪ >>=ʳ ∩-distl-∪-<: -∪-saturate-resp-∩-saturation ∩P⊂P (intersect (union p p₁) q) = (∪-saturate-resp-∩-saturation ∩P⊂P (intersect p q) [∪] ∪-saturate-resp-∩-saturation ∩P⊂P (intersect p₁ q)) >>= ⊂:-∪-lift-saturate >>=ˡ <:-∩-distr-∪ >>=ʳ ∩-distr-∪-<: - -ov-language : ∀ {F t} → FunType F → (∀ {S T} → Overloads F (S ⇒ T) → Language (S ⇒ T) t) → Language F t -ov-language (S ⇒ T) p = p here -ov-language (F ∩ G) p = (ov-language F (p ∘ left) , ov-language G (p ∘ right)) - -ov-<: : ∀ {F R S T U} → FunType F → Overloads F (R ⇒ S) → ((R ⇒ S) <: (T ⇒ U)) → F <: (T ⇒ U) -ov-<: F here p = p -ov-<: (F ∩ G) (left o) p = <:-trans <:-∩-left (ov-<: F o p) -ov-<: (F ∩ G) (right o) p = <:-trans <:-∩-right (ov-<: G o p) - -<:ᵒ-impl-<: : ∀ {F G} → FunType F → FunType G → (F <:ᵒ G) → (F <: G) -<:ᵒ-impl-<: F (T ⇒ U) F>= ⊂:-overloads-left -⊂:-overloads-⋒ (R ⇒ S) (G ∩ H) (intersect here (right o)) = ⊂:-overloads-⋒ (R ⇒ S) H (intersect here o) >>= ⊂:-overloads-right -⊂:-overloads-⋒ (E ∩ F) G (intersect (left n) o) = ⊂:-overloads-⋒ E G (intersect n o) >>= ⊂:-overloads-left -⊂:-overloads-⋒ (E ∩ F) G (intersect (right n) o) = ⊂:-overloads-⋒ F G (intersect n o) >>= ⊂:-overloads-right - -⊂:-⋒-overloads : ∀ {F G} → FunType F → FunType G → Overloads (F ⋒ G) ⊂: ∩-Lift (Overloads F) (Overloads G) -⊂:-⋒-overloads (R ⇒ S) (T ⇒ U) here = defn (intersect here here) (∩ⁿ-<:-∩ R T) (∩-<:-∩ⁿ S U) -⊂:-⋒-overloads (R ⇒ S) (G ∩ H) (left o) = ⊂:-⋒-overloads (R ⇒ S) G o >>= ⊂:-∩-lift ⊂:-refl ⊂:-overloads-left -⊂:-⋒-overloads (R ⇒ S) (G ∩ H) (right o) = ⊂:-⋒-overloads (R ⇒ S) H o >>= ⊂:-∩-lift ⊂:-refl ⊂:-overloads-right -⊂:-⋒-overloads (E ∩ F) G (left o) = ⊂:-⋒-overloads E G o >>= ⊂:-∩-lift ⊂:-overloads-left ⊂:-refl -⊂:-⋒-overloads (E ∩ F) G (right o) = ⊂:-⋒-overloads F G o >>= ⊂:-∩-lift ⊂:-overloads-right ⊂:-refl - -⊂:-overloads-⋓ : ∀ {F G} → FunType F → FunType G → ∪-Lift (Overloads F) (Overloads G) ⊂: Overloads (F ⋓ G) -⊂:-overloads-⋓ (R ⇒ S) (T ⇒ U) (union here here) = defn here (∪-<:-∪ⁿ R T) (∪ⁿ-<:-∪ S U) -⊂:-overloads-⋓ (R ⇒ S) (G ∩ H) (union here (left o)) = ⊂:-overloads-⋓ (R ⇒ S) G (union here o) >>= ⊂:-overloads-left -⊂:-overloads-⋓ (R ⇒ S) (G ∩ H) (union here (right o)) = ⊂:-overloads-⋓ (R ⇒ S) H (union here o) >>= ⊂:-overloads-right -⊂:-overloads-⋓ (E ∩ F) G (union (left n) o) = ⊂:-overloads-⋓ E G (union n o) >>= ⊂:-overloads-left -⊂:-overloads-⋓ (E ∩ F) G (union (right n) o) = ⊂:-overloads-⋓ F G (union n o) >>= ⊂:-overloads-right - -⊂:-⋓-overloads : ∀ {F G} → FunType F → FunType G → Overloads (F ⋓ G) ⊂: ∪-Lift (Overloads F) (Overloads G) -⊂:-⋓-overloads (R ⇒ S) (T ⇒ U) here = defn (union here here) (∪ⁿ-<:-∪ R T) (∪-<:-∪ⁿ S U) -⊂:-⋓-overloads (R ⇒ S) (G ∩ H) (left o) = ⊂:-⋓-overloads (R ⇒ S) G o >>= ⊂:-∪-lift ⊂:-refl ⊂:-overloads-left -⊂:-⋓-overloads (R ⇒ S) (G ∩ H) (right o) = ⊂:-⋓-overloads (R ⇒ S) H o >>= ⊂:-∪-lift ⊂:-refl ⊂:-overloads-right -⊂:-⋓-overloads (E ∩ F) G (left o) = ⊂:-⋓-overloads E G o >>= ⊂:-∪-lift ⊂:-overloads-left ⊂:-refl -⊂:-⋓-overloads (E ∩ F) G (right o) = ⊂:-⋓-overloads F G o >>= ⊂:-∪-lift ⊂:-overloads-right ⊂:-refl - -∪-saturate-overloads : ∀ {F} → FunType F → Overloads (∪-saturate F) ⊂: ∪-Saturate (Overloads F) -∪-saturate-overloads (S ⇒ T) here = just (base here) -∪-saturate-overloads (F ∩ G) (left (left o)) = ∪-saturate-overloads F o >>= ⊂:-∪-saturate ⊂:-overloads-left -∪-saturate-overloads (F ∩ G) (left (right o)) = ∪-saturate-overloads G o >>= ⊂:-∪-saturate ⊂:-overloads-right -∪-saturate-overloads (F ∩ G) (right o) = - ⊂:-⋓-overloads (normal-∪-saturate F) (normal-∪-saturate G) o >>= - ⊂:-∪-lift (∪-saturate-overloads F) (∪-saturate-overloads G) >>= - ⊂:-∪-lift (⊂:-∪-saturate ⊂:-overloads-left) (⊂:-∪-saturate ⊂:-overloads-right) >>= - ⊂:-∪-lift-saturate - -overloads-∪-saturate : ∀ {F} → FunType F → ∪-Saturate (Overloads F) ⊂: Overloads (∪-saturate F) -overloads-∪-saturate F = ⊂:-∪-saturate-indn (inj F) (step F) where - - inj : ∀ {F} → FunType F → Overloads F ⊂: Overloads (∪-saturate F) - inj (S ⇒ T) here = just here - inj (F ∩ G) (left p) = inj F p >>= ⊂:-overloads-left >>= ⊂:-overloads-left - inj (F ∩ G) (right p) = inj G p >>= ⊂:-overloads-right >>= ⊂:-overloads-left - - step : ∀ {F} → FunType F → ∪-Lift (Overloads (∪-saturate F)) (Overloads (∪-saturate F)) ⊂: Overloads (∪-saturate F) - step (S ⇒ T) (union here here) = defn here (<:-∪-lub <:-refl <:-refl) <:-∪-left - step (F ∩ G) (union (left (left p)) (left (left q))) = step F (union p q) >>= ⊂:-overloads-left >>= ⊂:-overloads-left - step (F ∩ G) (union (left (left p)) (left (right q))) = ⊂:-overloads-⋓ (normal-∪-saturate F) (normal-∪-saturate G) (union p q) >>= ⊂:-overloads-right - step (F ∩ G) (union (left (right p)) (left (left q))) = ⊂:-overloads-⋓ (normal-∪-saturate F) (normal-∪-saturate G) (union q p) >>= ⊂:-overloads-right >>=ˡ <:-∪-symm >>=ʳ <:-∪-symm - step (F ∩ G) (union (left (right p)) (left (right q))) = step G (union p q) >>= ⊂:-overloads-right >>= ⊂:-overloads-left - step (F ∩ G) (union p (right q)) with ⊂:-⋓-overloads (normal-∪-saturate F) (normal-∪-saturate G) q - step (F ∩ G) (union (left (left p)) (right q)) | defn (union q₁ q₂) q₃ q₄ = - (step F (union p q₁) [∪] just q₂) >>= - ⊂:-overloads-⋓ (normal-∪-saturate F) (normal-∪-saturate G) >>= - ⊂:-overloads-right >>=ˡ - <:-trans (<:-union <:-refl q₃) <:-∪-assocl >>=ʳ - <:-trans <:-∪-assocr (<:-union <:-refl q₄) - step (F ∩ G) (union (left (right p)) (right q)) | defn (union q₁ q₂) q₃ q₄ = - (just q₁ [∪] step G (union p q₂)) >>= - ⊂:-overloads-⋓ (normal-∪-saturate F) (normal-∪-saturate G) >>= - ⊂:-overloads-right >>=ˡ - <:-trans (<:-union <:-refl q₃) (<:-∪-lub (<:-trans <:-∪-left <:-∪-right) (<:-∪-lub <:-∪-left (<:-trans <:-∪-right <:-∪-right))) >>=ʳ - <:-trans (<:-∪-lub (<:-trans <:-∪-left <:-∪-right) (<:-∪-lub <:-∪-left (<:-trans <:-∪-right <:-∪-right))) (<:-union <:-refl q₄) - step (F ∩ G) (union (right p) (right q)) | defn (union q₁ q₂) q₃ q₄ with ⊂:-⋓-overloads (normal-∪-saturate F) (normal-∪-saturate G) p - step (F ∩ G) (union (right p) (right q)) | defn (union q₁ q₂) q₃ q₄ | defn (union p₁ p₂) p₃ p₄ = - (step F (union p₁ q₁) [∪] step G (union p₂ q₂)) >>= - ⊂:-overloads-⋓ (normal-∪-saturate F) (normal-∪-saturate G) >>= - ⊂:-overloads-right >>=ˡ - <:-trans (<:-union p₃ q₃) (<:-∪-lub (<:-union <:-∪-left <:-∪-left) (<:-union <:-∪-right <:-∪-right)) >>=ʳ - <:-trans (<:-∪-lub (<:-union <:-∪-left <:-∪-left) (<:-union <:-∪-right <:-∪-right)) (<:-union p₄ q₄) - step (F ∩ G) (union (right p) q) with ⊂:-⋓-overloads (normal-∪-saturate F) (normal-∪-saturate G) p - step (F ∩ G) (union (right p) (left (left q))) | defn (union p₁ p₂) p₃ p₄ = - (step F (union p₁ q) [∪] just p₂) >>= - ⊂:-overloads-⋓ (normal-∪-saturate F) (normal-∪-saturate G) >>= - ⊂:-overloads-right >>=ˡ - <:-trans (<:-union p₃ <:-refl) (<:-∪-lub (<:-union <:-∪-left <:-refl) (<:-trans <:-∪-right <:-∪-left)) >>=ʳ - <:-trans (<:-∪-lub (<:-union <:-∪-left <:-refl) (<:-trans <:-∪-right <:-∪-left)) (<:-union p₄ <:-refl) - step (F ∩ G) (union (right p) (left (right q))) | defn (union p₁ p₂) p₃ p₄ = - (just p₁ [∪] step G (union p₂ q)) >>= - ⊂:-overloads-⋓ (normal-∪-saturate F) (normal-∪-saturate G) >>= - ⊂:-overloads-right >>=ˡ - <:-trans (<:-union p₃ <:-refl) <:-∪-assocr >>=ʳ - <:-trans <:-∪-assocl (<:-union p₄ <:-refl) - step (F ∩ G) (union (right p) (right q)) | defn (union p₁ p₂) p₃ p₄ with ⊂:-⋓-overloads (normal-∪-saturate F) (normal-∪-saturate G) q - step (F ∩ G) (union (right p) (right q)) | defn (union p₁ p₂) p₃ p₄ | defn (union q₁ q₂) q₃ q₄ = - (step F (union p₁ q₁) [∪] step G (union p₂ q₂)) >>= - ⊂:-overloads-⋓ (normal-∪-saturate F) (normal-∪-saturate G) >>= - ⊂:-overloads-right >>=ˡ - <:-trans (<:-union p₃ q₃) (<:-∪-lub (<:-union <:-∪-left <:-∪-left) (<:-union <:-∪-right <:-∪-right)) >>=ʳ - <:-trans (<:-∪-lub (<:-union <:-∪-left <:-∪-left) (<:-union <:-∪-right <:-∪-right)) (<:-union p₄ q₄) - -∪-saturated : ∀ {F} → FunType F → ∪-Lift (Overloads (∪-saturate F)) (Overloads (∪-saturate F)) ⊂: Overloads (∪-saturate F) -∪-saturated F o = - ⊂:-∪-lift (∪-saturate-overloads F) (∪-saturate-overloads F) o >>= - ⊂:-∪-lift-saturate >>= - overloads-∪-saturate F - -∩-saturate-overloads : ∀ {F} → FunType F → Overloads (∩-saturate F) ⊂: ∩-Saturate (Overloads F) -∩-saturate-overloads (S ⇒ T) here = just (base here) -∩-saturate-overloads (F ∩ G) (left (left o)) = ∩-saturate-overloads F o >>= ⊂:-∩-saturate ⊂:-overloads-left -∩-saturate-overloads (F ∩ G) (left (right o)) = ∩-saturate-overloads G o >>= ⊂:-∩-saturate ⊂:-overloads-right -∩-saturate-overloads (F ∩ G) (right o) = - ⊂:-⋒-overloads (normal-∩-saturate F) (normal-∩-saturate G) o >>= - ⊂:-∩-lift (∩-saturate-overloads F) (∩-saturate-overloads G) >>= - ⊂:-∩-lift (⊂:-∩-saturate ⊂:-overloads-left) (⊂:-∩-saturate ⊂:-overloads-right) >>= - ⊂:-∩-lift-saturate - -overloads-∩-saturate : ∀ {F} → FunType F → ∩-Saturate (Overloads F) ⊂: Overloads (∩-saturate F) -overloads-∩-saturate F = ⊂:-∩-saturate-indn (inj F) (step F) where - - inj : ∀ {F} → FunType F → Overloads F ⊂: Overloads (∩-saturate F) - inj (S ⇒ T) here = just here - inj (F ∩ G) (left p) = inj F p >>= ⊂:-overloads-left >>= ⊂:-overloads-left - inj (F ∩ G) (right p) = inj G p >>= ⊂:-overloads-right >>= ⊂:-overloads-left - - step : ∀ {F} → FunType F → ∩-Lift (Overloads (∩-saturate F)) (Overloads (∩-saturate F)) ⊂: Overloads (∩-saturate F) - step (S ⇒ T) (intersect here here) = defn here <:-∩-left (<:-∩-glb <:-refl <:-refl) - step (F ∩ G) (intersect (left (left p)) (left (left q))) = step F (intersect p q) >>= ⊂:-overloads-left >>= ⊂:-overloads-left - step (F ∩ G) (intersect (left (left p)) (left (right q))) = ⊂:-overloads-⋒ (normal-∩-saturate F) (normal-∩-saturate G) (intersect p q) >>= ⊂:-overloads-right - step (F ∩ G) (intersect (left (right p)) (left (left q))) = ⊂:-overloads-⋒ (normal-∩-saturate F) (normal-∩-saturate G) (intersect q p) >>= ⊂:-overloads-right >>=ˡ <:-∩-symm >>=ʳ <:-∩-symm - step (F ∩ G) (intersect (left (right p)) (left (right q))) = step G (intersect p q) >>= ⊂:-overloads-right >>= ⊂:-overloads-left - step (F ∩ G) (intersect (right p) q) with ⊂:-⋒-overloads (normal-∩-saturate F) (normal-∩-saturate G) p - step (F ∩ G) (intersect (right p) (left (left q))) | defn (intersect p₁ p₂) p₃ p₄ = - (step F (intersect p₁ q) [∩] just p₂) >>= - ⊂:-overloads-⋒ (normal-∩-saturate F) (normal-∩-saturate G) >>= - ⊂:-overloads-right >>=ˡ - <:-trans (<:-intersect p₃ <:-refl) (<:-∩-glb (<:-intersect <:-∩-left <:-refl) (<:-trans <:-∩-left <:-∩-right)) >>=ʳ - <:-trans (<:-∩-glb (<:-intersect <:-∩-left <:-refl) (<:-trans <:-∩-left <:-∩-right)) (<:-intersect p₄ <:-refl) - step (F ∩ G) (intersect (right p) (left (right q))) | defn (intersect p₁ p₂) p₃ p₄ = - (just p₁ [∩] step G (intersect p₂ q)) >>= - ⊂:-overloads-⋒ (normal-∩-saturate F) (normal-∩-saturate G) >>= - ⊂:-overloads-right >>=ˡ - <:-trans (<:-intersect p₃ <:-refl) <:-∩-assocr >>=ʳ - <:-trans <:-∩-assocl (<:-intersect p₄ <:-refl) - step (F ∩ G) (intersect (right p) (right q)) | defn (intersect p₁ p₂) p₃ p₄ with ⊂:-⋒-overloads (normal-∩-saturate F) (normal-∩-saturate G) q - step (F ∩ G) (intersect (right p) (right q)) | defn (intersect p₁ p₂) p₃ p₄ | defn (intersect q₁ q₂) q₃ q₄ = - (step F (intersect p₁ q₁) [∩] step G (intersect p₂ q₂)) >>= - ⊂:-overloads-⋒ (normal-∩-saturate F) (normal-∩-saturate G) >>= - ⊂:-overloads-right >>=ˡ - <:-trans (<:-intersect p₃ q₃) (<:-∩-glb (<:-intersect <:-∩-left <:-∩-left) (<:-intersect <:-∩-right <:-∩-right)) >>=ʳ - <:-trans (<:-∩-glb (<:-intersect <:-∩-left <:-∩-left) (<:-intersect <:-∩-right <:-∩-right)) (<:-intersect p₄ q₄) - step (F ∩ G) (intersect p (right q)) with ⊂:-⋒-overloads (normal-∩-saturate F) (normal-∩-saturate G) q - step (F ∩ G) (intersect (left (left p)) (right q)) | defn (intersect q₁ q₂) q₃ q₄ = - (step F (intersect p q₁) [∩] just q₂) >>= - ⊂:-overloads-⋒ (normal-∩-saturate F) (normal-∩-saturate G) >>= - ⊂:-overloads-right >>=ˡ - <:-trans (<:-intersect <:-refl q₃) <:-∩-assocl >>=ʳ - <:-trans <:-∩-assocr (<:-intersect <:-refl q₄) - step (F ∩ G) (intersect (left (right p)) (right q)) | defn (intersect q₁ q₂) q₃ q₄ = - (just q₁ [∩] step G (intersect p q₂) ) >>= - ⊂:-overloads-⋒ (normal-∩-saturate F) (normal-∩-saturate G) >>= - ⊂:-overloads-right >>=ˡ - <:-trans (<:-intersect <:-refl q₃) (<:-∩-glb (<:-trans <:-∩-right <:-∩-left) (<:-∩-glb <:-∩-left (<:-trans <:-∩-right <:-∩-right))) >>=ʳ - <:-∩-glb (<:-trans <:-∩-right <:-∩-left) (<:-trans (<:-∩-glb <:-∩-left (<:-trans <:-∩-right <:-∩-right)) q₄) - step (F ∩ G) (intersect (right p) (right q)) | defn (intersect q₁ q₂) q₃ q₄ with ⊂:-⋒-overloads (normal-∩-saturate F) (normal-∩-saturate G) p - step (F ∩ G) (intersect (right p) (right q)) | defn (intersect q₁ q₂) q₃ q₄ | defn (intersect p₁ p₂) p₃ p₄ = - (step F (intersect p₁ q₁) [∩] step G (intersect p₂ q₂)) >>= - ⊂:-overloads-⋒ (normal-∩-saturate F) (normal-∩-saturate G) >>= - ⊂:-overloads-right >>=ˡ - <:-trans (<:-intersect p₃ q₃) (<:-∩-glb (<:-intersect <:-∩-left <:-∩-left) (<:-intersect <:-∩-right <:-∩-right)) >>=ʳ - <:-trans (<:-∩-glb (<:-intersect <:-∩-left <:-∩-left) (<:-intersect <:-∩-right <:-∩-right)) (<:-intersect p₄ q₄) - -saturate-overloads : ∀ {F} → FunType F → Overloads (saturate F) ⊂: ∪-Saturate (∩-Saturate (Overloads F)) -saturate-overloads F o = ∪-saturate-overloads (normal-∩-saturate F) o >>= (⊂:-∪-saturate (∩-saturate-overloads F)) - -overloads-saturate : ∀ {F} → FunType F → ∪-Saturate (∩-Saturate (Overloads F)) ⊂: Overloads (saturate F) -overloads-saturate F o = ⊂:-∪-saturate (overloads-∩-saturate F) o >>= overloads-∪-saturate (normal-∩-saturate F) - --- Saturated F whenever --- * if F has overloads (R ⇒ S) and (T ⇒ U) then F has an overload which is a subtype of ((R ∩ T) ⇒ (S ∩ U)) --- * ditto union -data Saturated (F : Type) : Set where - - defn : - - (∀ {R S T U} → Overloads F (R ⇒ S) → Overloads F (T ⇒ U) → <:-Close (Overloads F) ((R ∩ T) ⇒ (S ∩ U))) → - (∀ {R S T U} → Overloads F (R ⇒ S) → Overloads F (T ⇒ U) → <:-Close (Overloads F) ((R ∪ T) ⇒ (S ∪ U))) → - ----------- - Saturated F - --- saturated F is saturated! -saturated : ∀ {F} → FunType F → Saturated (saturate F) -saturated F = defn - (λ n o → (saturate-overloads F n [∩] saturate-overloads F o) >>= ∪-saturate-resp-∩-saturation ⊂:-∩-lift-saturate >>= overloads-saturate F) - (λ n o → ∪-saturated (normal-∩-saturate F) (union n o)) diff --git a/prototyping/README.md b/prototyping/README.md deleted file mode 100644 index 965c9c49..00000000 --- a/prototyping/README.md +++ /dev/null @@ -1,45 +0,0 @@ -# Prototyping Luau - -![prototyping workflow](https://github.com/Roblox/luau/actions/workflows/prototyping.yml/badge.svg) - -An experimental prototyping system for the Luau type system. This is -intended to allow core language features to be tested quickly, without -having to interact with all the features of production Lua. - -## Building - -First install Haskell and Agda. - -Install dependencies: -``` - cabal update - cabal install --lib aeson scientific vector -``` - -Then compile -``` - agda --compile PrettyPrinter.agda -``` - -and run! -``` - luau-ast Examples/SmokeTest.lua | ./PrettyPrinter -``` - -## Testing - -We have a series of snapshot tests in the `Tests/` directory. You interact with the tests using the `tests` Python script in the `prototyping` directory. To simply run the tests, run: - -```sh -tests --luau-cli ../build/luau-ast --build -``` - -This will build the test targets and run them. Run `tests --help` for information about all the command-line options. - -### Adding a new test - -To add a new test, add it to `Tests/{SUITE_NAME}/{CASE_NAME}`. You'll need an `in.lua` file and an `out.txt` file. The `in.lua` file is the input Luau source code, while the `out.txt` file is the expected output after running `luau-ast in.lua | test_executable`. - -### Updating a test - -If you make a change to the prototype that results in an expected change in behavior, you might want to update the test cases automatically. To do this, run `tests` with the `--accept-new-output` (`-a` for short) flag. Rather than diffing the output, this will overwrite the `out.txt` files for each test case with the actual result. Commit the resulting changes with your PR. diff --git a/prototyping/Tests/Interpreter/binary_equality_bools/in.lua b/prototyping/Tests/Interpreter/binary_equality_bools/in.lua deleted file mode 100644 index c9c72dc6..00000000 --- a/prototyping/Tests/Interpreter/binary_equality_bools/in.lua +++ /dev/null @@ -1 +0,0 @@ -return true == false diff --git a/prototyping/Tests/Interpreter/binary_equality_bools/out.txt b/prototyping/Tests/Interpreter/binary_equality_bools/out.txt deleted file mode 100644 index ee6f2a25..00000000 --- a/prototyping/Tests/Interpreter/binary_equality_bools/out.txt +++ /dev/null @@ -1,4 +0,0 @@ -ANNOTATED PROGRAM: -return true == false - -RAN WITH RESULT: false diff --git a/prototyping/Tests/Interpreter/binary_equality_numbers/in.lua b/prototyping/Tests/Interpreter/binary_equality_numbers/in.lua deleted file mode 100644 index 4efc68a7..00000000 --- a/prototyping/Tests/Interpreter/binary_equality_numbers/in.lua +++ /dev/null @@ -1 +0,0 @@ -return 1 == 1 diff --git a/prototyping/Tests/Interpreter/binary_equality_numbers/out.txt b/prototyping/Tests/Interpreter/binary_equality_numbers/out.txt deleted file mode 100644 index 86a499a8..00000000 --- a/prototyping/Tests/Interpreter/binary_equality_numbers/out.txt +++ /dev/null @@ -1,4 +0,0 @@ -ANNOTATED PROGRAM: -return 1.0 == 1.0 - -RAN WITH RESULT: true diff --git a/prototyping/Tests/Interpreter/binary_exps/in.lua b/prototyping/Tests/Interpreter/binary_exps/in.lua deleted file mode 100644 index 0750f9e6..00000000 --- a/prototyping/Tests/Interpreter/binary_exps/in.lua +++ /dev/null @@ -1 +0,0 @@ -return 1 + 2 - 2 * 2 / 2 diff --git a/prototyping/Tests/Interpreter/binary_exps/out.txt b/prototyping/Tests/Interpreter/binary_exps/out.txt deleted file mode 100644 index 5b4d2644..00000000 --- a/prototyping/Tests/Interpreter/binary_exps/out.txt +++ /dev/null @@ -1,4 +0,0 @@ -ANNOTATED PROGRAM: -return 1.0 + 2.0 - 2.0 * 2.0 / 2.0 - -RAN WITH RESULT: 1.0 diff --git a/prototyping/Tests/Interpreter/concat_number_and_string/in.lua b/prototyping/Tests/Interpreter/concat_number_and_string/in.lua deleted file mode 100644 index ba5acb32..00000000 --- a/prototyping/Tests/Interpreter/concat_number_and_string/in.lua +++ /dev/null @@ -1,3 +0,0 @@ -local x: string = "hello" -local y: string = 37 -return x .. y diff --git a/prototyping/Tests/Interpreter/concat_number_and_string/out.txt b/prototyping/Tests/Interpreter/concat_number_and_string/out.txt deleted file mode 100644 index 80c9d6ee..00000000 --- a/prototyping/Tests/Interpreter/concat_number_and_string/out.txt +++ /dev/null @@ -1,12 +0,0 @@ -ANNOTATED PROGRAM: -local x : string = "hello" -local y : string = 37.0 -return x .. y - -RUNTIME ERROR: -value 37.0 is not a string - in return statement - -TYPE ERROR: -Local variable y has type string but expression has type number - because provided type contains v, where v is a number diff --git a/prototyping/Tests/Interpreter/concat_two_strings/in.lua b/prototyping/Tests/Interpreter/concat_two_strings/in.lua deleted file mode 100644 index c6ccdce6..00000000 --- a/prototyping/Tests/Interpreter/concat_two_strings/in.lua +++ /dev/null @@ -1,3 +0,0 @@ -local x: string = "hello" -local y: string = "world" -return x .. y diff --git a/prototyping/Tests/Interpreter/concat_two_strings/out.txt b/prototyping/Tests/Interpreter/concat_two_strings/out.txt deleted file mode 100644 index d45f589f..00000000 --- a/prototyping/Tests/Interpreter/concat_two_strings/out.txt +++ /dev/null @@ -1,6 +0,0 @@ -ANNOTATED PROGRAM: -local x : string = "hello" -local y : string = "world" -return x .. y - -RAN WITH RESULT: "helloworld" diff --git a/prototyping/Tests/Interpreter/return_nil/in.lua b/prototyping/Tests/Interpreter/return_nil/in.lua deleted file mode 100644 index 4e6ce2db..00000000 --- a/prototyping/Tests/Interpreter/return_nil/in.lua +++ /dev/null @@ -1,5 +0,0 @@ -local function foo(x) - return nil -end - -return foo(nil) diff --git a/prototyping/Tests/Interpreter/return_nil/out.txt b/prototyping/Tests/Interpreter/return_nil/out.txt deleted file mode 100644 index 1d5f45b6..00000000 --- a/prototyping/Tests/Interpreter/return_nil/out.txt +++ /dev/null @@ -1,7 +0,0 @@ -UNANNOTATED PROGRAM: -local function foo(x) - return nil -end -return foo(nil) - -RAN WITH RESULT: nil diff --git a/prototyping/Tests/Interpreter/return_string/in.lua b/prototyping/Tests/Interpreter/return_string/in.lua deleted file mode 100644 index 67febb75..00000000 --- a/prototyping/Tests/Interpreter/return_string/in.lua +++ /dev/null @@ -1 +0,0 @@ -return "foo bar" diff --git a/prototyping/Tests/Interpreter/return_string/out.txt b/prototyping/Tests/Interpreter/return_string/out.txt deleted file mode 100644 index 81915c77..00000000 --- a/prototyping/Tests/Interpreter/return_string/out.txt +++ /dev/null @@ -1,4 +0,0 @@ -ANNOTATED PROGRAM: -return "foo bar" - -RAN WITH RESULT: "foo bar" diff --git a/prototyping/Tests/PrettyPrinter/smoke_test/in.lua b/prototyping/Tests/PrettyPrinter/smoke_test/in.lua deleted file mode 100644 index d26e3a0d..00000000 --- a/prototyping/Tests/PrettyPrinter/smoke_test/in.lua +++ /dev/null @@ -1,19 +0,0 @@ -local function id(x) - return x -end -local function comp(f) - return function(g) - return function(x) - return f(g(x)) - end - end -end -local id2 = comp(id)(id) -local nil2 = id2(nil) -local a : any = nil -local b : nil = nil -local c : (nil) -> nil = nil -local d : (any & nil) = nil -local e : any? = nil -local f : number = 123 -return id2(nil2) diff --git a/prototyping/Tests/PrettyPrinter/smoke_test/out.txt b/prototyping/Tests/PrettyPrinter/smoke_test/out.txt deleted file mode 100644 index ca95cae9..00000000 --- a/prototyping/Tests/PrettyPrinter/smoke_test/out.txt +++ /dev/null @@ -1,19 +0,0 @@ -local function id(x) - return x -end -local function comp(f) - return function(g) - return function(x) - return f(g(x)) - end - end -end -local id2 = comp(id)(id) -local nil2 = id2(nil) -local a : unknown = nil -local b : nil = nil -local c : (nil) -> nil = nil -local d : (unknown & nil) = nil -local e : unknown? = nil -local f : number = 123.0 -return id2(nil2) diff --git a/prototyping/Utility/Bool.agda b/prototyping/Utility/Bool.agda deleted file mode 100644 index 1afffb0d..00000000 --- a/prototyping/Utility/Bool.agda +++ /dev/null @@ -1,16 +0,0 @@ -module Utility.Bool where - -open import Agda.Builtin.Bool using (Bool; true; false) - -not : Bool → Bool -not false = true -not true = false - -_or_ : Bool → Bool → Bool -true or _ = true -_ or true = true -_ or _ = false - -_and_ : Bool → Bool → Bool -true and true = true -_ and _ = false diff --git a/prototyping/tests.py b/prototyping/tests.py deleted file mode 100755 index 20070ae0..00000000 --- a/prototyping/tests.py +++ /dev/null @@ -1,197 +0,0 @@ -#!/usr/bin/python - -import argparse -import difflib -import enum -import os -import os.path -import subprocess -import sys - -SUITES = ["interpreter", "prettyprinter"] -IN_FILE_NAME = "in.lua" -OUT_FILE_NAME = "out.txt" -SUITE_EXE_NAMES = { - "interpreter": "Interpreter", - "prettyprinter": "PrettyPrinter", -} - -SUITE_ENTRY_POINTS = { - "interpreter": "Interpreter.agda", - "prettyprinter": "PrettyPrinter.agda", -} - -SUITE_ROOTS = { - "interpreter": "Tests/Interpreter", - "prettyprinter": "Tests/PrettyPrinter", -} - -class TestResultStatus(enum.Enum): - CLI_ERROR = 0 - EXE_ERROR = 1 - DIFF_ERROR = 2 - SUCCESS = 3 - WROTE_NEW = 4 - -class DiffFailure: - def __init__(self, expected, actual): - self.expected = expected - self.actual = actual - - def diff_text(self): - diff_generator = difflib.context_diff(self.expected.splitlines(), self.actual.splitlines(), fromfile="expected", tofile="actual", n=3) - return "".join(diff_generator) - - def diff_html(self): - differ = difflib.HtmlDiff(tabsize=4) - return differ.make_file(self.expected.splitlines(), self.actual.splitlines(), fromdesc="Expected", todesc="Actual", context=True, numlines=5) - -class TestCaseResult: - def __init__(self, suite, case, status, details): - self.suite = suite - self.case = case - self.status = status - self.details = details - - def did_pass(self): - return self.status == TestResultStatus.SUCCESS or self.status == TestResultStatus.WROTE_NEW - - def to_string(self): - prefix = f"[{self.suite}/{self.case}]: " - if self.status == TestResultStatus.CLI_ERROR: - return f"{prefix}CLI ERROR: {self.details}" - elif self.status == TestResultStatus.EXE_ERROR: - return f"{prefix}EXE ERROR: {self.details}" - elif self.status == TestResultStatus.DIFF_ERROR: - text_diff = self.details.diff_text() - return f"{prefix}FAILED:\n{text_diff}" - elif self.status == TestResultStatus.SUCCESS: - return f"{prefix}SUCCEEDED" - elif self.status == TestResultStatus.WROTE_NEW: - return f"{prefix}WROTE NEW RESULT" - - def write_artifact(self, artifact_root): - if self.status != TestResultStatus.DIFF_ERROR: - return - - filename = f"{self.suite}-{self.case}.out.html" - path = os.path.join(artifact_root, filename) - html = self.details.diff_html() - with open(path, "w") as file: - file.write(html) - -parser = argparse.ArgumentParser(description="Runs prototype test cases") -parser.add_argument("--luau-cli", "-l", dest="cli_location", required=True, help="The location of luau-cli") -parser.add_argument("--root", "-r", dest="prototype_root", required=False, default=os.getcwd(), help="The root of the prototype") -parser.add_argument("--build", "-b", dest="build", action="store_true", default=True, help="Whether to automatically build required test binaries") -parser.add_argument("--suite", "-s", dest="suites", action="append", default=[], choices=SUITES, help="Which test suites to run") -parser.add_argument("--case", "-c", dest="cases", action="append", default=[], help="Which test cases to run") -parser.add_argument("--accept-new-output", "-a", dest="snapshot", action="store_true", default=False, help="Whether to write the new output to files, instead of diffing against it") -parser.add_argument("--write-diff-failures", dest="write_diffs", action="store_true", default=False, help="Whether to write test failure diffs to files") -parser.add_argument("--diff-failure-location", dest="diff_location", default=None, help="Where to write diff failure files to") - -def build_suite(root, suite): - entry_point = SUITE_ENTRY_POINTS.get(suite) - if entry_point is None: - return (False, "Invalid suite") - - result = subprocess.run(["~/.cabal/bin/agda", "--compile", entry_point], shell=True, cwd=root, stdout=subprocess.PIPE, stderr=subprocess.STDOUT) - if result.returncode == 0: - return (True, None) - else: - return (False, result.stdout) - -def run_test(in_path, out_path, cli_path, exe_path, snapshot): - cli_result = subprocess.run([cli_path, in_path], capture_output=True) - if cli_result.returncode != 0: - return (TestResultStatus.CLI_ERROR, f"CLI error: {cli_result.stderr}") - - exe_result = subprocess.run(exe_path, input=cli_result.stdout, capture_output=True) - if exe_result.returncode != 0: - return (TestResultStatus.EXE_ERROR, f"Executable error; stdout:{exe_result.stdout}\n\nstderr: {exe_result.stderr}") - actual_result = exe_result.stdout.decode("utf-8") - - if snapshot: - with open(out_path, "w") as out_file: - out_file.write(actual_result) - return (TestResultStatus.WROTE_NEW, None) - else: - with open(out_path, "r") as out_file: - expected_result = out_file.read() - - if expected_result != actual_result: - return (TestResultStatus.DIFF_ERROR, DiffFailure(expected_result, actual_result)) - - return (TestResultStatus.SUCCESS, None) - -def should_run_case(case_name, filters): - if len(filters) == 0: - return True - - return any([f in case_name for f in filters]) - -def run_test_suite(args, suite, suite_root, suite_exe): - results = [] - - for entry in os.listdir(suite_root): - if not should_run_case(entry, args.cases): - continue - - case_path = os.path.join(suite_root, entry) - if os.path.isdir(case_path): - in_path = os.path.join(case_path, IN_FILE_NAME) - out_path = os.path.join(case_path, OUT_FILE_NAME) - - if not os.path.exists(in_path) or not os.path.exists(out_path): - continue - - status, details = run_test(in_path, out_path, args.cli_location, suite_exe, args.snapshot) - result = TestCaseResult(suite, entry, status, details) - results.append(result) - - return results - -def main(): - args = parser.parse_args() - - suites = args.suites if len(args.suites) > 0 else SUITES - root = os.path.abspath(args.prototype_root) - - if args.build: - for suite in suites: - success, reason = build_suite(root, suite) - - if not success: - print(f"Error building executable for test suite {suite}:\n{reason}") - sys.exit(1) - else: - print(f"Built executable for test suite {suite} successfully.") - - failed = False - for suite in suites: - suite_root = os.path.join(root, SUITE_ROOTS.get(suite)) - suite_exe = os.path.join(root, SUITE_EXE_NAMES.get(suite)) - print(f"Running test suite {suite}...") - results = run_test_suite(args, suite, suite_root, suite_exe) - - passed = 0 - total = len(results) - - for result in results: - if result.did_pass(): - passed += 1 - else: - failed = True - - print(f"Suite {suite} [{passed} / {total} passed]:") - for result in results: - print(result.to_string()) - - if args.write_diffs: - result.write_artifact(args.diff_location) - - if failed: - sys.exit(1) - -if __name__ == "__main__": - main() From 944e8375aa79181c108e99d767d4a11c9b5cc046 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Thu, 29 Sep 2022 15:23:10 -0700 Subject: [PATCH 368/399] Sync to upstream/release/547 (#690) - Type aliases can no longer override primitive types; attempts to do that will result in a type error - Fix misleading type error messages for mismatches in expression list length during assignment - Fix incorrect type name display in certain cases - setmetatable/getmetatable are now ~2x faster - tools/perfstat.py can be used to display statistics about profiles captured via --profile switch --- Analysis/include/Luau/BuiltinDefinitions.h | 4 +- Analysis/include/Luau/Config.h | 7 +- Analysis/include/Luau/Constraint.h | 3 +- .../include/Luau/ConstraintGraphBuilder.h | 1 + Analysis/include/Luau/ConstraintSolver.h | 1 + Analysis/include/Luau/Error.h | 5 +- Analysis/include/Luau/Frontend.h | 10 +- Analysis/include/Luau/Instantiation.h | 10 +- Analysis/include/Luau/Scope.h | 15 +- Analysis/include/Luau/TxnLog.h | 10 + Analysis/include/Luau/TypeArena.h | 1 + Analysis/include/Luau/TypeInfer.h | 10 +- Analysis/include/Luau/TypeUtils.h | 4 + Analysis/include/Luau/TypeVar.h | 18 +- Analysis/include/Luau/Unifiable.h | 1 + Analysis/include/Luau/Unifier.h | 3 +- Analysis/src/Autocomplete.cpp | 55 ++-- Analysis/src/BuiltinDefinitions.cpp | 95 +++++- Analysis/src/Clone.cpp | 7 +- Analysis/src/Config.cpp | 11 +- Analysis/src/ConstraintGraphBuilder.cpp | 64 +++- Analysis/src/ConstraintSolver.cpp | 68 +++- Analysis/src/Error.cpp | 14 +- Analysis/src/Frontend.cpp | 15 +- Analysis/src/Instantiation.cpp | 8 +- Analysis/src/Quantify.cpp | 52 +-- Analysis/src/Scope.cpp | 28 +- Analysis/src/ToString.cpp | 5 + Analysis/src/TxnLog.cpp | 39 +++ Analysis/src/TypeArena.cpp | 9 + Analysis/src/TypeChecker2.cpp | 303 ++++++++++-------- Analysis/src/TypeInfer.cpp | 283 +++++++++------- Analysis/src/TypeUtils.cpp | 42 +++ Analysis/src/TypeVar.cpp | 36 ++- Analysis/src/Unifiable.cpp | 7 + Analysis/src/Unifier.cpp | 108 +++++-- CLI/Analyze.cpp | 2 +- CodeGen/src/Fallbacks.cpp | 20 +- Common/include/Luau/Bytecode.h | 4 + Compiler/src/Builtins.cpp | 10 + VM/src/lapi.cpp | 10 +- VM/src/laux.cpp | 6 +- VM/src/lbuiltins.cpp | 81 ++++- VM/src/ldebug.cpp | 2 +- VM/src/ldo.cpp | 12 +- VM/src/lgc.cpp | 103 ++---- VM/src/lobject.cpp | 2 +- VM/src/lobject.h | 14 +- VM/src/ltm.cpp | 1 + VM/src/ltm.h | 1 + VM/src/lvmexecute.cpp | 24 +- VM/src/lvmload.cpp | 2 +- VM/src/lvmutils.cpp | 10 +- fuzz/linter.cpp | 2 +- fuzz/proto.cpp | 2 +- fuzz/typeck.cpp | 2 +- tests/Compiler.test.cpp | 13 +- tests/Conformance.test.cpp | 2 +- tests/Fixture.cpp | 9 +- tests/Frontend.test.cpp | 4 +- tests/ToString.test.cpp | 16 + tests/TypeInfer.aliases.test.cpp | 39 ++- tests/TypeInfer.annotations.test.cpp | 9 +- tests/TypeInfer.builtins.test.cpp | 54 +++- tests/TypeInfer.functions.test.cpp | 48 ++- tests/TypeInfer.loops.test.cpp | 75 ++++- tests/TypeInfer.provisional.test.cpp | 4 +- tests/TypeInfer.refinements.test.cpp | 22 -- tests/TypeInfer.test.cpp | 34 ++ tests/TypeInfer.tryUnify.test.cpp | 36 +++ tests/conformance/events.lua | 3 + tools/faillist.txt | 105 +++--- tools/lldb_formatters.lldb | 7 +- tools/lldb_formatters.py | 152 +++++++-- tools/perfgraph.py | 1 - tools/perfstat.py | 65 ++++ tools/test_dcr.py | 2 +- 77 files changed, 1660 insertions(+), 702 deletions(-) create mode 100644 tools/perfstat.py diff --git a/Analysis/include/Luau/BuiltinDefinitions.h b/Analysis/include/Luau/BuiltinDefinitions.h index 0292dff7..616367bb 100644 --- a/Analysis/include/Luau/BuiltinDefinitions.h +++ b/Analysis/include/Luau/BuiltinDefinitions.h @@ -8,9 +8,11 @@ namespace Luau { -void registerBuiltinTypes(TypeChecker& typeChecker); void registerBuiltinTypes(Frontend& frontend); +void registerBuiltinGlobals(TypeChecker& typeChecker); +void registerBuiltinGlobals(Frontend& frontend); + TypeId makeUnion(TypeArena& arena, std::vector&& types); TypeId makeIntersection(TypeArena& arena, std::vector&& types); diff --git a/Analysis/include/Luau/Config.h b/Analysis/include/Luau/Config.h index 56cdfe78..8ba4ffa5 100644 --- a/Analysis/include/Luau/Config.h +++ b/Analysis/include/Luau/Config.h @@ -17,12 +17,9 @@ constexpr const char* kConfigName = ".luaurc"; struct Config { - Config() - { - enabledLint.setDefaults(); - } + Config(); - Mode mode = Mode::NoCheck; + Mode mode; ParseOptions parseOptions; diff --git a/Analysis/include/Luau/Constraint.h b/Analysis/include/Luau/Constraint.h index 3ffb3fb8..0e19f13f 100644 --- a/Analysis/include/Luau/Constraint.h +++ b/Analysis/include/Luau/Constraint.h @@ -94,8 +94,9 @@ struct FunctionCallConstraint { std::vector> innerConstraints; TypeId fn; + TypePackId argsPack; TypePackId result; - class AstExprCall* astFragment; + class AstExprCall* callSite; }; // result ~ prim ExpectedType SomeSingletonType MultitonType diff --git a/Analysis/include/Luau/ConstraintGraphBuilder.h b/Analysis/include/Luau/ConstraintGraphBuilder.h index e7d8ad45..973c0a8e 100644 --- a/Analysis/include/Luau/ConstraintGraphBuilder.h +++ b/Analysis/include/Luau/ConstraintGraphBuilder.h @@ -124,6 +124,7 @@ struct ConstraintGraphBuilder void visit(const ScopePtr& scope, AstStatDeclareGlobal* declareGlobal); void visit(const ScopePtr& scope, AstStatDeclareClass* declareClass); void visit(const ScopePtr& scope, AstStatDeclareFunction* declareFunction); + void visit(const ScopePtr& scope, AstStatError* error); TypePackId checkPack(const ScopePtr& scope, AstArray exprs, const std::vector& expectedTypes = {}); TypePackId checkPack(const ScopePtr& scope, AstExpr* expr, const std::vector& expectedTypes = {}); diff --git a/Analysis/include/Luau/ConstraintSolver.h b/Analysis/include/Luau/ConstraintSolver.h index abea51b8..06f53e4a 100644 --- a/Analysis/include/Luau/ConstraintSolver.h +++ b/Analysis/include/Luau/ConstraintSolver.h @@ -104,6 +104,7 @@ struct ConstraintSolver bool tryDispatch(const HasPropConstraint& c, NotNull constraint); // for a, ... in some_table do + // also handles __iter metamethod bool tryDispatchIterableTable(TypeId iteratorTy, const IterableConstraint& c, NotNull constraint, bool force); // for a, ... in next_function, t, ... do diff --git a/Analysis/include/Luau/Error.h b/Analysis/include/Luau/Error.h index eab6a21d..f3735864 100644 --- a/Analysis/include/Luau/Error.h +++ b/Analysis/include/Luau/Error.h @@ -81,7 +81,7 @@ struct OnlyTablesCanHaveMethods struct DuplicateTypeDefinition { Name name; - Location previousLocation; + std::optional previousLocation; bool operator==(const DuplicateTypeDefinition& rhs) const; }; @@ -91,7 +91,8 @@ struct CountMismatch enum Context { Arg, - Result, + FunctionResult, + ExprListResult, Return, }; size_t expected; diff --git a/Analysis/include/Luau/Frontend.h b/Analysis/include/Luau/Frontend.h index 04c598de..5df6f4b5 100644 --- a/Analysis/include/Luau/Frontend.h +++ b/Analysis/include/Luau/Frontend.h @@ -157,7 +157,8 @@ struct Frontend ScopePtr getGlobalScope(); private: - ModulePtr check(const SourceModule& sourceModule, Mode mode, const ScopePtr& environmentScope, std::vector requireCycles); + ModulePtr check(const SourceModule& sourceModule, Mode mode, const ScopePtr& environmentScope, std::vector requireCycles, + bool forAutocomplete = false); std::pair getSourceNode(CheckResult& checkResult, const ModuleName& name); SourceModule parse(const ModuleName& name, std::string_view src, const ParseOptions& parseOptions); @@ -171,10 +172,9 @@ private: std::unordered_map environments; std::unordered_map> builtinDefinitions; - ScopePtr globalScope; + SingletonTypes singletonTypes_; public: - SingletonTypes singletonTypes_; const NotNull singletonTypes; FileResolver* fileResolver; @@ -186,13 +186,15 @@ public: FrontendOptions options; InternalErrorReporter iceHandler; TypeArena globalTypes; - TypeArena arenaForAutocomplete; std::unordered_map sourceNodes; std::unordered_map sourceModules; std::unordered_map requireTrace; Stats stats = {}; + +private: + ScopePtr globalScope; }; } // namespace Luau diff --git a/Analysis/include/Luau/Instantiation.h b/Analysis/include/Luau/Instantiation.h index e05ceebe..cd88d33a 100644 --- a/Analysis/include/Luau/Instantiation.h +++ b/Analysis/include/Luau/Instantiation.h @@ -14,16 +14,18 @@ struct TxnLog; // A substitution which replaces generic types in a given set by free types. struct ReplaceGenerics : Substitution { - ReplaceGenerics( - const TxnLog* log, TypeArena* arena, TypeLevel level, const std::vector& generics, const std::vector& genericPacks) + ReplaceGenerics(const TxnLog* log, TypeArena* arena, TypeLevel level, Scope* scope, const std::vector& generics, + const std::vector& genericPacks) : Substitution(log, arena) , level(level) + , scope(scope) , generics(generics) , genericPacks(genericPacks) { } TypeLevel level; + Scope* scope; std::vector generics; std::vector genericPacks; bool ignoreChildren(TypeId ty) override; @@ -36,13 +38,15 @@ struct ReplaceGenerics : Substitution // A substitution which replaces generic functions by monomorphic functions struct Instantiation : Substitution { - Instantiation(const TxnLog* log, TypeArena* arena, TypeLevel level) + Instantiation(const TxnLog* log, TypeArena* arena, TypeLevel level, Scope* scope) : Substitution(log, arena) , level(level) + , scope(scope) { } TypeLevel level; + Scope* scope; bool ignoreChildren(TypeId ty) override; bool isDirty(TypeId ty) override; bool isDirty(TypePackId tp) override; diff --git a/Analysis/include/Luau/Scope.h b/Analysis/include/Luau/Scope.h index b7569d8e..b2da7bc0 100644 --- a/Analysis/include/Luau/Scope.h +++ b/Analysis/include/Luau/Scope.h @@ -49,9 +49,11 @@ struct Scope std::unordered_map exportedTypeBindings; std::unordered_map privateTypeBindings; std::unordered_map typeAliasLocations; - std::unordered_map> importedTypeBindings; + DenseHashSet builtinTypeNames{""}; + void addBuiltinTypeBinding(const Name& name, const TypeFun& tyFun); + std::optional lookup(Symbol sym); std::optional lookupType(const Name& name); @@ -61,7 +63,7 @@ struct Scope std::optional lookupPack(const Name& name); // WARNING: This function linearly scans for a string key of equal value! It is thus O(n**2) - std::optional linearSearchForBinding(const std::string& name, bool traverseScopeChain = true); + std::optional linearSearchForBinding(const std::string& name, bool traverseScopeChain = true) const; RefinementMap refinements; @@ -73,4 +75,13 @@ struct Scope std::unordered_map typeAliasTypePackParameters; }; +// Returns true iff the left scope encloses the right scope. A Scope* equal to +// nullptr is considered to be the outermost-possible scope. +bool subsumesStrict(Scope* left, Scope* right); + +// Returns true if the left scope encloses the right scope, or if they are the +// same scope. As in subsumesStrict(), nullptr is considered to be the +// outermost-possible scope. +bool subsumes(Scope* left, Scope* right); + } // namespace Luau diff --git a/Analysis/include/Luau/TxnLog.h b/Analysis/include/Luau/TxnLog.h index 016cc927..3c3122c2 100644 --- a/Analysis/include/Luau/TxnLog.h +++ b/Analysis/include/Luau/TxnLog.h @@ -186,6 +186,16 @@ struct TxnLog // The pointer returned lives until `commit` or `clear` is called. PendingTypePack* changeLevel(TypePackId tp, TypeLevel newLevel); + // Queues the replacement of a type's scope with the provided scope. + // + // The pointer returned lives until `commit` or `clear` is called. + PendingType* changeScope(TypeId ty, NotNull scope); + + // Queues the replacement of a type pack's scope with the provided scope. + // + // The pointer returned lives until `commit` or `clear` is called. + PendingTypePack* changeScope(TypePackId tp, NotNull scope); + // Queues a replacement of a table type with another table type with a new // indexer. // diff --git a/Analysis/include/Luau/TypeArena.h b/Analysis/include/Luau/TypeArena.h index 1e029aeb..c67f643b 100644 --- a/Analysis/include/Luau/TypeArena.h +++ b/Analysis/include/Luau/TypeArena.h @@ -30,6 +30,7 @@ struct TypeArena TypeId freshType(TypeLevel level); TypeId freshType(Scope* scope); + TypeId freshType(Scope* scope, TypeLevel level); TypePackId freshTypePack(Scope* scope); diff --git a/Analysis/include/Luau/TypeInfer.h b/Analysis/include/Luau/TypeInfer.h index bbb9bd6d..e5675ebb 100644 --- a/Analysis/include/Luau/TypeInfer.h +++ b/Analysis/include/Luau/TypeInfer.h @@ -80,10 +80,12 @@ struct TypeChecker void check(const ScopePtr& scope, const AstStatForIn& forin); void check(const ScopePtr& scope, TypeId ty, const ScopePtr& funScope, const AstStatFunction& function); void check(const ScopePtr& scope, TypeId ty, const ScopePtr& funScope, const AstStatLocalFunction& function); - void check(const ScopePtr& scope, const AstStatTypeAlias& typealias, int subLevel = 0, bool forwardDeclare = false); + void check(const ScopePtr& scope, const AstStatTypeAlias& typealias); void check(const ScopePtr& scope, const AstStatDeclareClass& declaredClass); void check(const ScopePtr& scope, const AstStatDeclareFunction& declaredFunction); + void prototype(const ScopePtr& scope, const AstStatTypeAlias& typealias, int subLevel = 0); + void checkBlock(const ScopePtr& scope, const AstStatBlock& statement); void checkBlockWithoutRecursionCheck(const ScopePtr& scope, const AstStatBlock& statement); void checkBlockTypeAliases(const ScopePtr& scope, std::vector& sorted); @@ -392,8 +394,12 @@ private: std::vector> deferredQuantification; }; +using PrintLineProc = void(*)(const std::string&); + +extern PrintLineProc luauPrintLine; + // Unit test hook -void setPrintLine(void (*pl)(const std::string& s)); +void setPrintLine(PrintLineProc pl); void resetPrintLine(); } // namespace Luau diff --git a/Analysis/include/Luau/TypeUtils.h b/Analysis/include/Luau/TypeUtils.h index efc1d881..e5a205ba 100644 --- a/Analysis/include/Luau/TypeUtils.h +++ b/Analysis/include/Luau/TypeUtils.h @@ -25,4 +25,8 @@ std::optional getIndexTypeFromType(const ScopePtr& scope, ErrorVec& erro // Returns the minimum and maximum number of types the argument list can accept. std::pair> getParameterExtents(const TxnLog* log, TypePackId tp, bool includeHiddenVariadics = false); +// "Render" a type pack out to an array of a given length. Expands variadics and +// various other things to get there. +std::vector flatten(TypeArena& arena, NotNull singletonTypes, TypePackId pack, size_t length); + } // namespace Luau diff --git a/Analysis/include/Luau/TypeVar.h b/Analysis/include/Luau/TypeVar.h index 2847d0b1..1d587ffe 100644 --- a/Analysis/include/Luau/TypeVar.h +++ b/Analysis/include/Luau/TypeVar.h @@ -27,6 +27,7 @@ namespace Luau struct TypeArena; struct Scope; +using ScopePtr = std::shared_ptr; /** * There are three kinds of type variables: @@ -264,7 +265,15 @@ struct WithPredicate using MagicFunction = std::function>( struct TypeChecker&, const std::shared_ptr&, const class AstExprCall&, WithPredicate)>; -using DcrMagicFunction = std::function, TypePackId, const class AstExprCall*)>; +struct MagicFunctionCallContext +{ + NotNull solver; + const class AstExprCall* callSite; + TypePackId arguments; + TypePackId result; +}; + +using DcrMagicFunction = std::function; struct FunctionTypeVar { @@ -277,10 +286,14 @@ struct FunctionTypeVar // Local monomorphic function FunctionTypeVar(TypeLevel level, TypePackId argTypes, TypePackId retTypes, std::optional defn = {}, bool hasSelf = false); + FunctionTypeVar( + TypeLevel level, Scope* scope, TypePackId argTypes, TypePackId retTypes, std::optional defn = {}, bool hasSelf = false); // Local polymorphic function FunctionTypeVar(TypeLevel level, std::vector generics, std::vector genericPacks, TypePackId argTypes, TypePackId retTypes, std::optional defn = {}, bool hasSelf = false); + FunctionTypeVar(TypeLevel level, Scope* scope, std::vector generics, std::vector genericPacks, TypePackId argTypes, + TypePackId retTypes, std::optional defn = {}, bool hasSelf = false); TypeLevel level; Scope* scope = nullptr; @@ -345,8 +358,9 @@ struct TableTypeVar using Props = std::map; TableTypeVar() = default; - explicit TableTypeVar(TableState state, TypeLevel level); + explicit TableTypeVar(TableState state, TypeLevel level, Scope* scope = nullptr); TableTypeVar(const Props& props, const std::optional& indexer, TypeLevel level, TableState state); + TableTypeVar(const Props& props, const std::optional& indexer, TypeLevel level, Scope* scope, TableState state); Props props; std::optional indexer; diff --git a/Analysis/include/Luau/Unifiable.h b/Analysis/include/Luau/Unifiable.h index e5eb4198..0ea175cc 100644 --- a/Analysis/include/Luau/Unifiable.h +++ b/Analysis/include/Luau/Unifiable.h @@ -85,6 +85,7 @@ struct Free { explicit Free(TypeLevel level); explicit Free(Scope* scope); + explicit Free(Scope* scope, TypeLevel level); int index; TypeLevel level; diff --git a/Analysis/include/Luau/Unifier.h b/Analysis/include/Luau/Unifier.h index 4d46869d..26a922f5 100644 --- a/Analysis/include/Luau/Unifier.h +++ b/Analysis/include/Luau/Unifier.h @@ -60,6 +60,7 @@ struct Unifier Location location; Variance variance = Covariant; bool anyIsTop = false; // If true, we consider any to be a top type. If false, it is a familiar but weird mix of top and bottom all at once. + bool useScopes = false; // If true, we use the scope hierarchy rather than TypeLevels CountMismatch::Context ctx = CountMismatch::Arg; UnifierSharedState& sharedState; @@ -140,6 +141,6 @@ private: std::optional firstPackErrorPos; }; -void promoteTypeLevels(TxnLog& log, const TypeArena* arena, TypeLevel minLevel, TypePackId tp); +void promoteTypeLevels(TxnLog& log, const TypeArena* arena, TypeLevel minLevel, Scope* outerScope, bool useScope, TypePackId tp); } // namespace Luau diff --git a/Analysis/src/Autocomplete.cpp b/Analysis/src/Autocomplete.cpp index 2fc145d3..1f594fed 100644 --- a/Analysis/src/Autocomplete.cpp +++ b/Analysis/src/Autocomplete.cpp @@ -1208,13 +1208,11 @@ static bool autocompleteIfElseExpression( } } -static AutocompleteContext autocompleteExpression(const SourceModule& sourceModule, const Module& module, const TypeChecker& typeChecker, +static AutocompleteContext autocompleteExpression(const SourceModule& sourceModule, const Module& module, NotNull singletonTypes, TypeArena* typeArena, const std::vector& ancestry, Position position, AutocompleteEntryMap& result) { LUAU_ASSERT(!ancestry.empty()); - NotNull singletonTypes = typeChecker.singletonTypes; - AstNode* node = ancestry.rbegin()[0]; if (node->is()) @@ -1254,16 +1252,16 @@ static AutocompleteContext autocompleteExpression(const SourceModule& sourceModu scope = scope->parent; } - TypeCorrectKind correctForNil = checkTypeCorrectKind(module, typeArena, singletonTypes, node, position, typeChecker.nilType); + TypeCorrectKind correctForNil = checkTypeCorrectKind(module, typeArena, singletonTypes, node, position, singletonTypes->nilType); TypeCorrectKind correctForTrue = checkTypeCorrectKind(module, typeArena, singletonTypes, node, position, singletonTypes->trueType); TypeCorrectKind correctForFalse = checkTypeCorrectKind(module, typeArena, singletonTypes, node, position, singletonTypes->falseType); TypeCorrectKind correctForFunction = functionIsExpectedAt(module, node, position).value_or(false) ? TypeCorrectKind::Correct : TypeCorrectKind::None; result["if"] = {AutocompleteEntryKind::Keyword, std::nullopt, false, false}; - result["true"] = {AutocompleteEntryKind::Keyword, typeChecker.booleanType, false, false, correctForTrue}; - result["false"] = {AutocompleteEntryKind::Keyword, typeChecker.booleanType, false, false, correctForFalse}; - result["nil"] = {AutocompleteEntryKind::Keyword, typeChecker.nilType, false, false, correctForNil}; + result["true"] = {AutocompleteEntryKind::Keyword, singletonTypes->booleanType, false, false, correctForTrue}; + result["false"] = {AutocompleteEntryKind::Keyword, singletonTypes->booleanType, false, false, correctForFalse}; + result["nil"] = {AutocompleteEntryKind::Keyword, singletonTypes->nilType, false, false, correctForNil}; result["not"] = {AutocompleteEntryKind::Keyword}; result["function"] = {AutocompleteEntryKind::Keyword, std::nullopt, false, false, correctForFunction}; @@ -1274,11 +1272,11 @@ static AutocompleteContext autocompleteExpression(const SourceModule& sourceModu return AutocompleteContext::Expression; } -static AutocompleteResult autocompleteExpression(const SourceModule& sourceModule, const Module& module, const TypeChecker& typeChecker, +static AutocompleteResult autocompleteExpression(const SourceModule& sourceModule, const Module& module, NotNull singletonTypes, TypeArena* typeArena, const std::vector& ancestry, Position position) { AutocompleteEntryMap result; - AutocompleteContext context = autocompleteExpression(sourceModule, module, typeChecker, typeArena, ancestry, position, result); + AutocompleteContext context = autocompleteExpression(sourceModule, module, singletonTypes, typeArena, ancestry, position, result); return {result, ancestry, context}; } @@ -1385,13 +1383,13 @@ static std::optional autocompleteStringParams(const Source return std::nullopt; } -static AutocompleteResult autocomplete(const SourceModule& sourceModule, const ModulePtr& module, const TypeChecker& typeChecker, - TypeArena* typeArena, Position position, StringCompletionCallback callback) +static AutocompleteResult autocomplete(const SourceModule& sourceModule, const ModulePtr& module, NotNull singletonTypes, + Scope* globalScope, Position position, StringCompletionCallback callback) { if (isWithinComment(sourceModule, position)) return {}; - NotNull singletonTypes = typeChecker.singletonTypes; + TypeArena typeArena; std::vector ancestry = findAncestryAtPositionForAutocomplete(sourceModule, position); LUAU_ASSERT(!ancestry.empty()); @@ -1419,11 +1417,10 @@ static AutocompleteResult autocomplete(const SourceModule& sourceModule, const M PropIndexType indexType = indexName->op == ':' ? PropIndexType::Colon : PropIndexType::Point; if (!FFlag::LuauSelfCallAutocompleteFix3 && isString(ty)) - return {autocompleteProps( - *module, typeArena, singletonTypes, typeChecker.globalScope->bindings[AstName{"string"}].typeId, indexType, ancestry), + return {autocompleteProps(*module, &typeArena, singletonTypes, globalScope->bindings[AstName{"string"}].typeId, indexType, ancestry), ancestry, AutocompleteContext::Property}; else - return {autocompleteProps(*module, typeArena, singletonTypes, ty, indexType, ancestry), ancestry, AutocompleteContext::Property}; + return {autocompleteProps(*module, &typeArena, singletonTypes, ty, indexType, ancestry), ancestry, AutocompleteContext::Property}; } else if (auto typeReference = node->as()) { @@ -1441,7 +1438,7 @@ static AutocompleteResult autocomplete(const SourceModule& sourceModule, const M if (statLocal->vars.size == 1 && (!statLocal->equalsSignLocation || position < statLocal->equalsSignLocation->begin)) return {{{"function", AutocompleteEntry{AutocompleteEntryKind::Keyword}}}, ancestry, AutocompleteContext::Unknown}; else if (statLocal->equalsSignLocation && position >= statLocal->equalsSignLocation->end) - return autocompleteExpression(sourceModule, *module, typeChecker, typeArena, ancestry, position); + return autocompleteExpression(sourceModule, *module, singletonTypes, &typeArena, ancestry, position); else return {}; } @@ -1455,7 +1452,7 @@ static AutocompleteResult autocomplete(const SourceModule& sourceModule, const M if (statFor->from->location.containsClosed(position) || statFor->to->location.containsClosed(position) || (statFor->step && statFor->step->location.containsClosed(position))) - return autocompleteExpression(sourceModule, *module, typeChecker, typeArena, ancestry, position); + return autocompleteExpression(sourceModule, *module, singletonTypes, &typeArena, ancestry, position); return {}; } @@ -1485,7 +1482,7 @@ static AutocompleteResult autocomplete(const SourceModule& sourceModule, const M AstExpr* lastExpr = statForIn->values.data[statForIn->values.size - 1]; if (lastExpr->location.containsClosed(position)) - return autocompleteExpression(sourceModule, *module, typeChecker, typeArena, ancestry, position); + return autocompleteExpression(sourceModule, *module, singletonTypes, &typeArena, ancestry, position); if (position > lastExpr->location.end) return {{{"do", AutocompleteEntry{AutocompleteEntryKind::Keyword}}}, ancestry, AutocompleteContext::Keyword}; @@ -1509,7 +1506,7 @@ static AutocompleteResult autocomplete(const SourceModule& sourceModule, const M return {{{"do", AutocompleteEntry{AutocompleteEntryKind::Keyword}}}, ancestry, AutocompleteContext::Keyword}; if (!statWhile->hasDo || position < statWhile->doLocation.begin) - return autocompleteExpression(sourceModule, *module, typeChecker, typeArena, ancestry, position); + return autocompleteExpression(sourceModule, *module, singletonTypes, &typeArena, ancestry, position); if (statWhile->hasDo && position > statWhile->doLocation.end) return {autocompleteStatement(sourceModule, *module, ancestry, position), ancestry, AutocompleteContext::Statement}; @@ -1526,7 +1523,7 @@ static AutocompleteResult autocomplete(const SourceModule& sourceModule, const M else if (AstStatIf* statIf = parent->as(); statIf && node->is()) { if (statIf->condition->is()) - return autocompleteExpression(sourceModule, *module, typeChecker, typeArena, ancestry, position); + return autocompleteExpression(sourceModule, *module, singletonTypes, &typeArena, ancestry, position); else if (!statIf->thenLocation || statIf->thenLocation->containsClosed(position)) return {{{"then", AutocompleteEntry{AutocompleteEntryKind::Keyword}}}, ancestry, AutocompleteContext::Keyword}; } @@ -1534,7 +1531,7 @@ static AutocompleteResult autocomplete(const SourceModule& sourceModule, const M statIf && (!statIf->thenLocation || statIf->thenLocation->containsClosed(position))) return {{{"then", AutocompleteEntry{AutocompleteEntryKind::Keyword}}}, ancestry, AutocompleteContext::Keyword}; else if (AstStatRepeat* statRepeat = node->as(); statRepeat && statRepeat->condition->is()) - return autocompleteExpression(sourceModule, *module, typeChecker, typeArena, ancestry, position); + return autocompleteExpression(sourceModule, *module, singletonTypes, &typeArena, ancestry, position); else if (AstStatRepeat* statRepeat = extractStat(ancestry); statRepeat) return {autocompleteStatement(sourceModule, *module, ancestry, position), ancestry, AutocompleteContext::Statement}; else if (AstExprTable* exprTable = parent->as(); exprTable && (node->is() || node->is())) @@ -1546,7 +1543,7 @@ static AutocompleteResult autocomplete(const SourceModule& sourceModule, const M { if (auto it = module->astExpectedTypes.find(exprTable)) { - auto result = autocompleteProps(*module, typeArena, singletonTypes, *it, PropIndexType::Key, ancestry); + auto result = autocompleteProps(*module, &typeArena, singletonTypes, *it, PropIndexType::Key, ancestry); // Remove keys that are already completed for (const auto& item : exprTable->items) @@ -1560,7 +1557,7 @@ static AutocompleteResult autocomplete(const SourceModule& sourceModule, const M // If we know for sure that a key is being written, do not offer general expression suggestions if (!key) - autocompleteExpression(sourceModule, *module, typeChecker, typeArena, ancestry, position, result); + autocompleteExpression(sourceModule, *module, singletonTypes, &typeArena, ancestry, position, result); return {result, ancestry, AutocompleteContext::Property}; } @@ -1588,7 +1585,7 @@ static AutocompleteResult autocomplete(const SourceModule& sourceModule, const M if (auto idxExpr = ancestry.at(ancestry.size() - 2)->as()) { if (auto it = module->astTypes.find(idxExpr->expr)) - autocompleteProps(*module, typeArena, singletonTypes, follow(*it), PropIndexType::Point, ancestry, result); + autocompleteProps(*module, &typeArena, singletonTypes, follow(*it), PropIndexType::Point, ancestry, result); } else if (auto binExpr = ancestry.at(ancestry.size() - 2)->as()) { @@ -1604,12 +1601,10 @@ static AutocompleteResult autocomplete(const SourceModule& sourceModule, const M } if (node->is()) - { return {}; - } if (node->asExpr()) - return autocompleteExpression(sourceModule, *module, typeChecker, typeArena, ancestry, position); + return autocompleteExpression(sourceModule, *module, singletonTypes, &typeArena, ancestry, position); else if (node->asStat()) return {autocompleteStatement(sourceModule, *module, ancestry, position), ancestry, AutocompleteContext::Statement}; @@ -1628,15 +1623,15 @@ AutocompleteResult autocomplete(Frontend& frontend, const ModuleName& moduleName if (!sourceModule) return {}; - TypeChecker& typeChecker = frontend.typeCheckerForAutocomplete; ModulePtr module = frontend.moduleResolverForAutocomplete.getModule(moduleName); if (!module) return {}; - AutocompleteResult autocompleteResult = autocomplete(*sourceModule, module, typeChecker, &frontend.arenaForAutocomplete, position, callback); + NotNull singletonTypes = frontend.singletonTypes; + Scope* globalScope = frontend.typeCheckerForAutocomplete.globalScope.get(); - frontend.arenaForAutocomplete.clear(); + AutocompleteResult autocompleteResult = autocomplete(*sourceModule, module, singletonTypes, globalScope, position, callback); return autocompleteResult; } diff --git a/Analysis/src/BuiltinDefinitions.cpp b/Analysis/src/BuiltinDefinitions.cpp index 8f4863d0..dbe27bfd 100644 --- a/Analysis/src/BuiltinDefinitions.cpp +++ b/Analysis/src/BuiltinDefinitions.cpp @@ -1,18 +1,22 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #include "Luau/BuiltinDefinitions.h" +#include "Luau/Ast.h" #include "Luau/Frontend.h" #include "Luau/Symbol.h" #include "Luau/Common.h" #include "Luau/ToString.h" #include "Luau/ConstraintSolver.h" #include "Luau/TypeInfer.h" +#include "Luau/TypePack.h" +#include "Luau/TypeVar.h" #include LUAU_FASTFLAGVARIABLE(LuauSetMetaTableArgsCheck, false) LUAU_FASTFLAG(LuauUnknownAndNeverType) LUAU_FASTFLAGVARIABLE(LuauBuiltInMetatableNoBadSynthetic, false) +LUAU_FASTFLAG(LuauReportShadowedTypeAlias) /** FIXME: Many of these type definitions are not quite completely accurate. * @@ -34,7 +38,9 @@ static std::optional> magicFunctionPack( static std::optional> magicFunctionRequire( TypeChecker& typechecker, const ScopePtr& scope, const AstExprCall& expr, WithPredicate withPredicate); -static bool dcrMagicFunctionRequire(NotNull solver, TypePackId result, const AstExprCall* expr); + +static bool dcrMagicFunctionSelect(MagicFunctionCallContext context); +static bool dcrMagicFunctionRequire(MagicFunctionCallContext context); TypeId makeUnion(TypeArena& arena, std::vector&& types) { @@ -226,7 +232,22 @@ void assignPropDocumentationSymbols(TableTypeVar::Props& props, const std::strin } } -void registerBuiltinTypes(TypeChecker& typeChecker) +void registerBuiltinTypes(Frontend& frontend) +{ + frontend.getGlobalScope()->addBuiltinTypeBinding("any", TypeFun{{}, frontend.singletonTypes->anyType}); + frontend.getGlobalScope()->addBuiltinTypeBinding("nil", TypeFun{{}, frontend.singletonTypes->nilType}); + frontend.getGlobalScope()->addBuiltinTypeBinding("number", TypeFun{{}, frontend.singletonTypes->numberType}); + frontend.getGlobalScope()->addBuiltinTypeBinding("string", TypeFun{{}, frontend.singletonTypes->stringType}); + frontend.getGlobalScope()->addBuiltinTypeBinding("boolean", TypeFun{{}, frontend.singletonTypes->booleanType}); + frontend.getGlobalScope()->addBuiltinTypeBinding("thread", TypeFun{{}, frontend.singletonTypes->threadType}); + if (FFlag::LuauUnknownAndNeverType) + { + frontend.getGlobalScope()->addBuiltinTypeBinding("unknown", TypeFun{{}, frontend.singletonTypes->unknownType}); + frontend.getGlobalScope()->addBuiltinTypeBinding("never", TypeFun{{}, frontend.singletonTypes->neverType}); + } +} + +void registerBuiltinGlobals(TypeChecker& typeChecker) { LUAU_ASSERT(!typeChecker.globalTypes.typeVars.isFrozen()); LUAU_ASSERT(!typeChecker.globalTypes.typePacks.isFrozen()); @@ -303,6 +324,7 @@ void registerBuiltinTypes(TypeChecker& typeChecker) attachMagicFunction(getGlobalBinding(typeChecker, "assert"), magicFunctionAssert); attachMagicFunction(getGlobalBinding(typeChecker, "setmetatable"), magicFunctionSetMetaTable); attachMagicFunction(getGlobalBinding(typeChecker, "select"), magicFunctionSelect); + attachDcrMagicFunction(getGlobalBinding(typeChecker, "select"), dcrMagicFunctionSelect); if (TableTypeVar* ttv = getMutable(getGlobalBinding(typeChecker, "table"))) { @@ -317,12 +339,13 @@ void registerBuiltinTypes(TypeChecker& typeChecker) attachDcrMagicFunction(getGlobalBinding(typeChecker, "require"), dcrMagicFunctionRequire); } -void registerBuiltinTypes(Frontend& frontend) +void registerBuiltinGlobals(Frontend& frontend) { LUAU_ASSERT(!frontend.globalTypes.typeVars.isFrozen()); LUAU_ASSERT(!frontend.globalTypes.typePacks.isFrozen()); - TypeId nilType = frontend.typeChecker.nilType; + if (FFlag::LuauReportShadowedTypeAlias) + registerBuiltinTypes(frontend); TypeArena& arena = frontend.globalTypes; NotNull singletonTypes = frontend.singletonTypes; @@ -352,7 +375,7 @@ void registerBuiltinTypes(Frontend& frontend) TypePackId pairsArgsTypePack = arena.addTypePack({mapOfKtoV}); TypeId pairsNext = arena.addType(FunctionTypeVar{nextArgsTypePack, arena.addTypePack(TypePack{{genericK, genericV}})}); - TypePackId pairsReturnTypePack = arena.addTypePack(TypePack{{pairsNext, mapOfKtoV, nilType}}); + TypePackId pairsReturnTypePack = arena.addTypePack(TypePack{{pairsNext, mapOfKtoV, frontend.singletonTypes->nilType}}); // pairs(t: Table) -> ((Table, K?) -> (K, V), Table, nil) addGlobalBinding(frontend, "pairs", arena.addType(FunctionTypeVar{{genericK, genericV}, {}, pairsArgsTypePack, pairsReturnTypePack}), "@luau"); @@ -394,6 +417,7 @@ void registerBuiltinTypes(Frontend& frontend) attachMagicFunction(getGlobalBinding(frontend, "assert"), magicFunctionAssert); attachMagicFunction(getGlobalBinding(frontend, "setmetatable"), magicFunctionSetMetaTable); attachMagicFunction(getGlobalBinding(frontend, "select"), magicFunctionSelect); + attachDcrMagicFunction(getGlobalBinding(frontend, "select"), dcrMagicFunctionSelect); if (TableTypeVar* ttv = getMutable(getGlobalBinding(frontend, "table"))) { @@ -408,7 +432,6 @@ void registerBuiltinTypes(Frontend& frontend) attachDcrMagicFunction(getGlobalBinding(frontend, "require"), dcrMagicFunctionRequire); } - static std::optional> magicFunctionSelect( TypeChecker& typechecker, const ScopePtr& scope, const AstExprCall& expr, WithPredicate withPredicate) { @@ -450,6 +473,50 @@ static std::optional> magicFunctionSelect( return std::nullopt; } +static bool dcrMagicFunctionSelect(MagicFunctionCallContext context) +{ + if (context.callSite->args.size <= 0) + { + context.solver->reportError(TypeError{context.callSite->location, GenericError{"select should take 1 or more arguments"}}); + return false; + } + + AstExpr* arg1 = context.callSite->args.data[0]; + + if (AstExprConstantNumber* num = arg1->as()) + { + const auto& [v, tail] = flatten(context.arguments); + + int offset = int(num->value); + if (offset > 0) + { + if (size_t(offset) < v.size()) + { + std::vector res(v.begin() + offset, v.end()); + TypePackId resTypePack = context.solver->arena->addTypePack({std::move(res), tail}); + asMutable(context.result)->ty.emplace(resTypePack); + } + else if (tail) + asMutable(context.result)->ty.emplace(*tail); + + return true; + } + + return false; + } + + if (AstExprConstantString* str = arg1->as()) + { + if (str->value.size == 1 && str->value.data[0] == '#') { + TypePackId numberTypePack = context.solver->arena->addTypePack({context.solver->singletonTypes->numberType}); + asMutable(context.result)->ty.emplace(numberTypePack); + return true; + } + } + + return false; +} + static std::optional> magicFunctionSetMetaTable( TypeChecker& typechecker, const ScopePtr& scope, const AstExprCall& expr, WithPredicate withPredicate) { @@ -675,22 +742,22 @@ static bool checkRequirePathDcr(NotNull solver, AstExpr* expr) return good; } -static bool dcrMagicFunctionRequire(NotNull solver, TypePackId result, const AstExprCall* expr) +static bool dcrMagicFunctionRequire(MagicFunctionCallContext context) { - if (expr->args.size != 1) + if (context.callSite->args.size != 1) { - solver->reportError(GenericError{"require takes 1 argument"}, expr->location); + context.solver->reportError(GenericError{"require takes 1 argument"}, context.callSite->location); return false; } - if (!checkRequirePathDcr(solver, expr->args.data[0])) + if (!checkRequirePathDcr(context.solver, context.callSite->args.data[0])) return false; - if (auto moduleInfo = solver->moduleResolver->resolveModuleInfo(solver->currentModuleName, *expr)) + if (auto moduleInfo = context.solver->moduleResolver->resolveModuleInfo(context.solver->currentModuleName, *context.callSite)) { - TypeId moduleType = solver->resolveModule(*moduleInfo, expr->location); - TypePackId moduleResult = solver->arena->addTypePack({moduleType}); - asMutable(result)->ty.emplace(moduleResult); + TypeId moduleType = context.solver->resolveModule(*moduleInfo, context.callSite->location); + TypePackId moduleResult = context.solver->arena->addTypePack({moduleType}); + asMutable(context.result)->ty.emplace(moduleResult); return true; } diff --git a/Analysis/src/Clone.cpp b/Analysis/src/Clone.cpp index 7048d201..fd3a089b 100644 --- a/Analysis/src/Clone.cpp +++ b/Analysis/src/Clone.cpp @@ -220,6 +220,9 @@ void TypeCloner::operator()(const SingletonTypeVar& t) void TypeCloner::operator()(const FunctionTypeVar& t) { + // FISHY: We always erase the scope when we clone things. clone() was + // originally written so that we could copy a module's type surface into an + // export arena. This probably dates to that. TypeId result = dest.addType(FunctionTypeVar{TypeLevel{0, 0}, {}, {}, nullptr, nullptr, t.definition, t.hasSelf}); FunctionTypeVar* ftv = getMutable(result); LUAU_ASSERT(ftv != nullptr); @@ -436,7 +439,7 @@ TypeId shallowClone(TypeId ty, TypeArena& dest, const TxnLog* log, bool alwaysCl if (const FunctionTypeVar* ftv = get(ty)) { - FunctionTypeVar clone = FunctionTypeVar{ftv->level, ftv->argTypes, ftv->retTypes, ftv->definition, ftv->hasSelf}; + FunctionTypeVar clone = FunctionTypeVar{ftv->level, ftv->scope, ftv->argTypes, ftv->retTypes, ftv->definition, ftv->hasSelf}; clone.generics = ftv->generics; clone.genericPacks = ftv->genericPacks; clone.magicFunction = ftv->magicFunction; @@ -448,7 +451,7 @@ TypeId shallowClone(TypeId ty, TypeArena& dest, const TxnLog* log, bool alwaysCl else if (const TableTypeVar* ttv = get(ty)) { LUAU_ASSERT(!ttv->boundTo); - TableTypeVar clone = TableTypeVar{ttv->props, ttv->indexer, ttv->level, ttv->state}; + TableTypeVar clone = TableTypeVar{ttv->props, ttv->indexer, ttv->level, ttv->scope, ttv->state}; clone.definitionModuleName = ttv->definitionModuleName; clone.name = ttv->name; clone.syntheticName = ttv->syntheticName; diff --git a/Analysis/src/Config.cpp b/Analysis/src/Config.cpp index 35a2259d..00ca7b16 100644 --- a/Analysis/src/Config.cpp +++ b/Analysis/src/Config.cpp @@ -4,15 +4,18 @@ #include "Luau/Lexer.h" #include "Luau/StringUtils.h" -namespace +LUAU_FASTFLAGVARIABLE(LuauEnableNonstrictByDefaultForLuauConfig, false) + +namespace Luau { using Error = std::optional; -} - -namespace Luau +Config::Config() + : mode(FFlag::LuauEnableNonstrictByDefaultForLuauConfig ? Mode::Nonstrict : Mode::NoCheck) { + enabledLint.setDefaults(); +} static Error parseBoolean(bool& result, const std::string& value) { diff --git a/Analysis/src/ConstraintGraphBuilder.cpp b/Analysis/src/ConstraintGraphBuilder.cpp index aa1e9547..169f4645 100644 --- a/Analysis/src/ConstraintGraphBuilder.cpp +++ b/Analysis/src/ConstraintGraphBuilder.cpp @@ -11,6 +11,7 @@ LUAU_FASTINT(LuauCheckRecursionLimit); LUAU_FASTFLAG(DebugLuauLogSolverToJson); +LUAU_FASTFLAG(DebugLuauMagicTypes); #include "Luau/Scope.h" @@ -218,6 +219,8 @@ void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStat* stat) visit(scope, s); else if (auto s = stat->as()) visit(scope, s); + else if (auto s = stat->as()) + visit(scope, s); else LUAU_ASSERT(0); } @@ -454,8 +457,10 @@ void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatFunction* funct TypeId containingTableType = check(scope, indexName->expr); functionType = arena->addType(BlockedTypeVar{}); - TypeId prospectiveTableType = - arena->addType(TableTypeVar{}); // TODO look into stack utilization. This is probably ok because it scales with AST depth. + + // TODO look into stack utilization. This is probably ok because it scales with AST depth. + TypeId prospectiveTableType = arena->addType(TableTypeVar{TableState::Unsealed, TypeLevel{}, scope.get()}); + NotNull prospectiveTable{getMutable(prospectiveTableType)}; Property& prop = prospectiveTable->props[indexName->index.value]; @@ -619,7 +624,7 @@ void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatDeclareClass* d TypeId classTy = arena->addType(ClassTypeVar(className, {}, superTy, std::nullopt, {}, {}, moduleName)); ClassTypeVar* ctv = getMutable(classTy); - TypeId metaTy = arena->addType(TableTypeVar{TableState::Sealed, scope->level}); + TypeId metaTy = arena->addType(TableTypeVar{TableState::Sealed, scope->level, scope.get()}); TableTypeVar* metatable = getMutable(metaTy); ctv->metatable = metaTy; @@ -715,7 +720,7 @@ void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatDeclareFunction TypePackId paramPack = resolveTypePack(funScope, global->params); TypePackId retPack = resolveTypePack(funScope, global->retTypes); - TypeId fnType = arena->addType(FunctionTypeVar{funScope->level, std::move(genericTys), std::move(genericTps), paramPack, retPack}); + TypeId fnType = arena->addType(FunctionTypeVar{TypeLevel{}, funScope.get(), std::move(genericTys), std::move(genericTps), paramPack, retPack}); FunctionTypeVar* ftv = getMutable(fnType); ftv->argNames.reserve(global->paramNames.size); @@ -728,6 +733,14 @@ void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatDeclareFunction scope->bindings[global->name] = Binding{fnType, global->location}; } +void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatError* error) +{ + for (AstStat* stat : error->statements) + visit(scope, stat); + for (AstExpr* expr : error->expressions) + check(scope, expr); +} + TypePackId ConstraintGraphBuilder::checkPack(const ScopePtr& scope, AstArray exprs, const std::vector& expectedTypes) { std::vector head; @@ -745,7 +758,9 @@ TypePackId ConstraintGraphBuilder::checkPack(const ScopePtr& scope, AstArray expectedTailTypes{begin(expectedTypes) + i, end(expectedTypes)}; + std::vector expectedTailTypes; + if (i < expectedTypes.size()) + expectedTailTypes.assign(begin(expectedTypes) + i, end(expectedTypes)); tail = checkPack(scope, expr, expectedTailTypes); } } @@ -803,7 +818,8 @@ TypePackId ConstraintGraphBuilder::checkPack(const ScopePtr& scope, AstExpr* exp TypeId instantiatedType = arena->addType(BlockedTypeVar{}); // TODO: How do expectedTypes play into this? Do they? TypePackId rets = arena->addTypePack(BlockedTypePack{}); - FunctionTypeVar ftv(arena->addTypePack(TypePack{args, {}}), rets); + TypePackId argPack = arena->addTypePack(TypePack{args, {}}); + FunctionTypeVar ftv(TypeLevel{}, scope.get(), argPack, rets); TypeId inferredFnType = arena->addType(ftv); scope->unqueuedConstraints.push_back( @@ -834,6 +850,7 @@ TypePackId ConstraintGraphBuilder::checkPack(const ScopePtr& scope, AstExpr* exp FunctionCallConstraint{ {ic, sc}, fnType, + argPack, rets, call, }); @@ -968,6 +985,9 @@ TypeId ConstraintGraphBuilder::check(const ScopePtr& scope, AstExpr* expr, std:: else if (auto err = expr->as()) { // Open question: Should we traverse into this? + for (AstExpr* subExpr : err->expressions) + check(scope, subExpr); + result = singletonTypes->errorRecoveryType(); } else @@ -988,7 +1008,7 @@ TypeId ConstraintGraphBuilder::check(const ScopePtr& scope, AstExprIndexName* in TableTypeVar::Props props{{indexName->index.value, Property{result}}}; const std::optional indexer; - TableTypeVar ttv{std::move(props), indexer, TypeLevel{}, TableState::Free}; + TableTypeVar ttv{std::move(props), indexer, TypeLevel{}, scope.get(), TableState::Free}; TypeId expectedTableType = arena->addType(std::move(ttv)); @@ -1005,7 +1025,8 @@ TypeId ConstraintGraphBuilder::check(const ScopePtr& scope, AstExprIndexExpr* in TypeId result = freshType(scope); TableIndexer indexer{indexType, result}; - TypeId tableType = arena->addType(TableTypeVar{TableTypeVar::Props{}, TableIndexer{indexType, result}, TypeLevel{}, TableState::Free}); + TypeId tableType = + arena->addType(TableTypeVar{TableTypeVar::Props{}, TableIndexer{indexType, result}, TypeLevel{}, scope.get(), TableState::Free}); addConstraint(scope, indexExpr->expr->location, SubtypeConstraint{obj, tableType}); @@ -1094,6 +1115,9 @@ TypeId ConstraintGraphBuilder::check(const ScopePtr& scope, AstExprTable* expr, TableTypeVar* ttv = getMutable(ty); LUAU_ASSERT(ttv); + ttv->state = TableState::Unsealed; + ttv->scope = scope.get(); + auto createIndexer = [this, scope, ttv](const Location& location, TypeId currentIndexType, TypeId currentResultType) { if (!ttv->indexer) { @@ -1195,7 +1219,7 @@ ConstraintGraphBuilder::FunctionSignature ConstraintGraphBuilder::checkFunctionS } else { - bodyScope = childScope(fn->body, parent); + bodyScope = childScope(fn, parent); returnType = freshTypePack(bodyScope); bodyScope->returnType = returnType; @@ -1260,7 +1284,7 @@ ConstraintGraphBuilder::FunctionSignature ConstraintGraphBuilder::checkFunctionS // TODO: Vararg annotation. // TODO: Preserve argument names in the function's type. - FunctionTypeVar actualFunction{arena->addTypePack(argTypes, varargPack), returnType}; + FunctionTypeVar actualFunction{TypeLevel{}, parent.get(), arena->addTypePack(argTypes, varargPack), returnType}; actualFunction.hasNoGenerics = !hasGenerics; actualFunction.generics = std::move(genericTypes); actualFunction.genericPacks = std::move(genericTypePacks); @@ -1297,6 +1321,22 @@ TypeId ConstraintGraphBuilder::resolveType(const ScopePtr& scope, AstType* ty, b if (auto ref = ty->as()) { + if (FFlag::DebugLuauMagicTypes) + { + if (ref->name == "_luau_ice") + ice->ice("_luau_ice encountered", ty->location); + else if (ref->name == "_luau_print") + { + if (ref->parameters.size != 1 || !ref->parameters.data[0].type) + { + reportError(ty->location, GenericError{"_luau_print requires one generic parameter"}); + return singletonTypes->errorRecoveryType(); + } + else + return resolveType(scope, ref->parameters.data[0].type, topLevel); + } + } + std::optional alias = scope->lookupType(ref->name.value); if (alias.has_value() || ref->prefix.has_value()) @@ -1369,7 +1409,7 @@ TypeId ConstraintGraphBuilder::resolveType(const ScopePtr& scope, AstType* ty, b }; } - result = arena->addType(TableTypeVar{props, indexer, scope->level, TableState::Sealed}); + result = arena->addType(TableTypeVar{props, indexer, scope->level, scope.get(), TableState::Sealed}); } else if (auto fn = ty->as()) { @@ -1414,7 +1454,7 @@ TypeId ConstraintGraphBuilder::resolveType(const ScopePtr& scope, AstType* ty, b // TODO: FunctionTypeVar needs a pointer to the scope so that we know // how to quantify/instantiate it. - FunctionTypeVar ftv{argTypes, returnTypes}; + FunctionTypeVar ftv{TypeLevel{}, scope.get(), {}, {}, argTypes, returnTypes}; // This replicates the behavior of the appropriate FunctionTypeVar // constructors. diff --git a/Analysis/src/ConstraintSolver.cpp b/Analysis/src/ConstraintSolver.cpp index b2bf773f..35b8387f 100644 --- a/Analysis/src/ConstraintSolver.cpp +++ b/Analysis/src/ConstraintSolver.cpp @@ -8,9 +8,11 @@ #include "Luau/ModuleResolver.h" #include "Luau/Quantify.h" #include "Luau/ToString.h" +#include "Luau/TypeVar.h" #include "Luau/Unifier.h" #include "Luau/DcrLogger.h" #include "Luau/VisitTypeVar.h" +#include "Luau/TypeUtils.h" LUAU_FASTFLAGVARIABLE(DebugLuauLogSolver, false); LUAU_FASTFLAGVARIABLE(DebugLuauLogSolverToJson, false); @@ -439,6 +441,7 @@ bool ConstraintSolver::tryDispatch(const PackSubtypeConstraint& c, NotNullscope); + return true; } @@ -465,7 +468,7 @@ bool ConstraintSolver::tryDispatch(const InstantiationConstraint& c, NotNullscope); std::optional instantiated = inst.substitute(c.superType); LUAU_ASSERT(instantiated); // TODO FIXME HANDLE THIS @@ -909,7 +912,7 @@ bool ConstraintSolver::tryDispatch(const FunctionCallConstraint& c, NotNulldcrMagicFunction != nullptr) { - usedMagic = ftv->dcrMagicFunction(NotNull(this), result, c.astFragment); + usedMagic = ftv->dcrMagicFunction(MagicFunctionCallContext{NotNull(this), c.callSite, c.argsPack, result}); } if (usedMagic) @@ -1087,6 +1090,63 @@ bool ConstraintSolver::tryDispatchIterableTable(TypeId iteratorTy, const Iterabl else errorify(c.variables); } + else if (std::optional iterFn = findMetatableEntry(singletonTypes, errors, iteratorTy, "__iter", Location{})) + { + if (isBlocked(*iterFn)) + { + return block(*iterFn, constraint); + } + + Instantiation instantiation(TxnLog::empty(), arena, TypeLevel{}, constraint->scope); + + if (std::optional instantiatedIterFn = instantiation.substitute(*iterFn)) + { + if (auto iterFtv = get(*instantiatedIterFn)) + { + TypePackId expectedIterArgs = arena->addTypePack({iteratorTy}); + unify(iterFtv->argTypes, expectedIterArgs, constraint->scope); + + std::vector iterRets = flatten(*arena, singletonTypes, iterFtv->retTypes, 2); + + if (iterRets.size() < 1) + { + // We've done what we can; this will get reported as an + // error by the type checker. + return true; + } + + TypeId nextFn = iterRets[0]; + TypeId table = iterRets.size() == 2 ? iterRets[1] : arena->freshType(constraint->scope); + + if (std::optional instantiatedNextFn = instantiation.substitute(nextFn)) + { + const TypeId firstIndex = arena->freshType(constraint->scope); + + // nextTy : (iteratorTy, indexTy?) -> (indexTy, valueTailTy...) + const TypePackId nextArgPack = arena->addTypePack({table, arena->addType(UnionTypeVar{{firstIndex, singletonTypes->nilType}})}); + const TypePackId valueTailTy = arena->addTypePack(FreeTypePack{constraint->scope}); + const TypePackId nextRetPack = arena->addTypePack(TypePack{{firstIndex}, valueTailTy}); + + const TypeId expectedNextTy = arena->addType(FunctionTypeVar{nextArgPack, nextRetPack}); + unify(*instantiatedNextFn, expectedNextTy, constraint->scope); + + pushConstraint(constraint->scope, constraint->location, PackSubtypeConstraint{c.variables, nextRetPack}); + } + else + { + reportError(UnificationTooComplex{}, constraint->location); + } + } + else + { + // TODO: Support __call and function overloads (what does an overload even mean for this?) + } + } + else + { + reportError(UnificationTooComplex{}, constraint->location); + } + } else if (auto iteratorMetatable = get(iteratorTy)) { TypeId metaTy = follow(iteratorMetatable->metatable); @@ -1124,7 +1184,7 @@ bool ConstraintSolver::tryDispatchIterableFunction( const TypePackId valueTailTy = arena->addTypePack(FreeTypePack{constraint->scope}); const TypePackId nextRetPack = arena->addTypePack(TypePack{{firstIndex}, valueTailTy}); - const TypeId expectedNextTy = arena->addType(FunctionTypeVar{nextArgPack, nextRetPack}); + const TypeId expectedNextTy = arena->addType(FunctionTypeVar{TypeLevel{}, constraint->scope, nextArgPack, nextRetPack}); unify(nextTy, expectedNextTy, constraint->scope); pushConstraint(constraint->scope, constraint->location, PackSubtypeConstraint{c.variables, nextRetPack}); @@ -1297,6 +1357,7 @@ void ConstraintSolver::unify(TypeId subType, TypeId superType, NotNull sc { UnifierSharedState sharedState{&iceReporter}; Unifier u{arena, singletonTypes, Mode::Strict, scope, Location{}, Covariant, sharedState}; + u.useScopes = true; u.tryUnify(subType, superType); @@ -1319,6 +1380,7 @@ void ConstraintSolver::unify(TypePackId subPack, TypePackId superPack, NotNullbegin.line + 1); + return s; } std::string operator()(const Luau::CountMismatch& e) const @@ -183,11 +186,14 @@ struct ErrorConverter case CountMismatch::Return: return "Expected to return " + std::to_string(e.expected) + " value" + expectedS + ", but " + std::to_string(e.actual) + " " + actualVerb + " returned here"; - case CountMismatch::Result: + case CountMismatch::FunctionResult: // It is alright if right hand side produces more values than the // left hand side accepts. In this context consider only the opposite case. - return "Function only returns " + std::to_string(e.expected) + " value" + expectedS + ". " + std::to_string(e.actual) + - " are required here"; + return "Function only returns " + std::to_string(e.expected) + " value" + expectedS + ", but " + std::to_string(e.actual) + " " + + actualVerb + " required here"; + case CountMismatch::ExprListResult: + return "Expression list has " + std::to_string(e.expected) + " value" + expectedS + ", but " + std::to_string(e.actual) + " " + + actualVerb + " required here"; case CountMismatch::Arg: if (!e.function.empty()) return "Argument count mismatch. Function '" + e.function + "' " + diff --git a/Analysis/src/Frontend.cpp b/Analysis/src/Frontend.cpp index 01e82baa..1890e081 100644 --- a/Analysis/src/Frontend.cpp +++ b/Analysis/src/Frontend.cpp @@ -400,6 +400,7 @@ Frontend::Frontend(FileResolver* fileResolver, ConfigResolver* configResolver, c , typeCheckerForAutocomplete(&moduleResolverForAutocomplete, singletonTypes, &iceHandler) , configResolver(configResolver) , options(options) + , globalScope(typeChecker.globalScope) { } @@ -505,7 +506,10 @@ CheckResult Frontend::check(const ModuleName& name, std::optional requireCycles) +ModulePtr Frontend::check( + const SourceModule& sourceModule, Mode mode, const ScopePtr& environmentScope, std::vector requireCycles, bool forAutocomplete) { ModulePtr result = std::make_shared(); @@ -852,7 +857,11 @@ ModulePtr Frontend::check(const SourceModule& sourceModule, Mode mode, const Sco } } - ConstraintGraphBuilder cgb{sourceModule.name, result, &result->internalTypes, NotNull(&moduleResolver), singletonTypes, NotNull(&iceHandler), getGlobalScope(), logger.get()}; + const NotNull mr{forAutocomplete ? &moduleResolverForAutocomplete : &moduleResolver}; + const ScopePtr& globalScope{forAutocomplete ? typeCheckerForAutocomplete.globalScope : typeChecker.globalScope}; + + ConstraintGraphBuilder cgb{ + sourceModule.name, result, &result->internalTypes, mr, singletonTypes, NotNull(&iceHandler), globalScope, logger.get()}; cgb.visit(sourceModule.root); result->errors = std::move(cgb.errors); diff --git a/Analysis/src/Instantiation.cpp b/Analysis/src/Instantiation.cpp index 2d1d62f3..3d0cd0d1 100644 --- a/Analysis/src/Instantiation.cpp +++ b/Analysis/src/Instantiation.cpp @@ -44,7 +44,7 @@ TypeId Instantiation::clean(TypeId ty) const FunctionTypeVar* ftv = log->getMutable(ty); LUAU_ASSERT(ftv); - FunctionTypeVar clone = FunctionTypeVar{level, ftv->argTypes, ftv->retTypes, ftv->definition, ftv->hasSelf}; + FunctionTypeVar clone = FunctionTypeVar{level, scope, ftv->argTypes, ftv->retTypes, ftv->definition, ftv->hasSelf}; clone.magicFunction = ftv->magicFunction; clone.dcrMagicFunction = ftv->dcrMagicFunction; clone.tags = ftv->tags; @@ -53,7 +53,7 @@ TypeId Instantiation::clean(TypeId ty) // Annoyingly, we have to do this even if there are no generics, // to replace any generic tables. - ReplaceGenerics replaceGenerics{log, arena, level, ftv->generics, ftv->genericPacks}; + ReplaceGenerics replaceGenerics{log, arena, level, scope, ftv->generics, ftv->genericPacks}; // TODO: What to do if this returns nullopt? // We don't have access to the error-reporting machinery @@ -114,12 +114,12 @@ TypeId ReplaceGenerics::clean(TypeId ty) LUAU_ASSERT(isDirty(ty)); if (const TableTypeVar* ttv = log->getMutable(ty)) { - TableTypeVar clone = TableTypeVar{ttv->props, ttv->indexer, level, TableState::Free}; + TableTypeVar clone = TableTypeVar{ttv->props, ttv->indexer, level, scope, TableState::Free}; clone.definitionModuleName = ttv->definitionModuleName; return addType(std::move(clone)); } else - return addType(FreeTypeVar{level}); + return addType(FreeTypeVar{scope, level}); } TypePackId ReplaceGenerics::clean(TypePackId tp) diff --git a/Analysis/src/Quantify.cpp b/Analysis/src/Quantify.cpp index 7e6ff2f9..e4c069bd 100644 --- a/Analysis/src/Quantify.cpp +++ b/Analysis/src/Quantify.cpp @@ -15,19 +15,6 @@ LUAU_FASTFLAG(LuauClassTypeVarsInSubstitution) namespace Luau { -/// @return true if outer encloses inner -static bool subsumes(Scope* outer, Scope* inner) -{ - while (inner) - { - if (inner == outer) - return true; - inner = inner->parent.get(); - } - - return false; -} - struct Quantifier final : TypeVarOnceVisitor { TypeLevel level; @@ -43,12 +30,6 @@ struct Quantifier final : TypeVarOnceVisitor LUAU_ASSERT(!FFlag::DebugLuauDeferredConstraintResolution); } - explicit Quantifier(Scope* scope) - : scope(scope) - { - LUAU_ASSERT(FFlag::DebugLuauDeferredConstraintResolution); - } - /// @return true if outer encloses inner bool subsumes(Scope* outer, Scope* inner) { @@ -66,13 +47,10 @@ struct Quantifier final : TypeVarOnceVisitor { seenMutableType = true; - if (FFlag::DebugLuauDeferredConstraintResolution ? !subsumes(scope, ftv.scope) : !level.subsumes(ftv.level)) + if (!level.subsumes(ftv.level)) return false; - if (FFlag::DebugLuauDeferredConstraintResolution) - *asMutable(ty) = GenericTypeVar{scope}; - else - *asMutable(ty) = GenericTypeVar{level}; + *asMutable(ty) = GenericTypeVar{level}; generics.push_back(ty); @@ -85,7 +63,7 @@ struct Quantifier final : TypeVarOnceVisitor seenMutableType = true; - if (FFlag::DebugLuauDeferredConstraintResolution ? !subsumes(scope, ctv->scope) : !level.subsumes(ctv->level)) + if (!level.subsumes(ctv->level)) return false; std::vector opts = std::move(ctv->parts); @@ -113,7 +91,7 @@ struct Quantifier final : TypeVarOnceVisitor if (ttv.state == TableState::Free) seenMutableType = true; - if (FFlag::DebugLuauDeferredConstraintResolution ? !subsumes(scope, ttv.scope) : !level.subsumes(ttv.level)) + if (!level.subsumes(ttv.level)) { if (ttv.state == TableState::Unsealed) seenMutableType = true; @@ -137,7 +115,7 @@ struct Quantifier final : TypeVarOnceVisitor { seenMutableType = true; - if (FFlag::DebugLuauDeferredConstraintResolution ? !subsumes(scope, ftp.scope) : !level.subsumes(ftp.level)) + if (!level.subsumes(ftp.level)) return false; *asMutable(tp) = GenericTypePack{level}; @@ -197,20 +175,6 @@ void quantify(TypeId ty, TypeLevel level) } } -void quantify(TypeId ty, Scope* scope) -{ - Quantifier q{scope}; - q.traverse(ty); - - FunctionTypeVar* ftv = getMutable(ty); - LUAU_ASSERT(ftv); - ftv->generics.insert(ftv->generics.end(), q.generics.begin(), q.generics.end()); - ftv->genericPacks.insert(ftv->genericPacks.end(), q.genericPacks.begin(), q.genericPacks.end()); - - if (ftv->generics.empty() && ftv->genericPacks.empty() && !q.seenMutableType && !q.seenGenericType) - ftv->hasNoGenerics = true; -} - struct PureQuantifier : Substitution { Scope* scope; @@ -253,7 +217,7 @@ struct PureQuantifier : Substitution { if (auto ftv = get(ty)) { - TypeId result = arena->addType(GenericTypeVar{}); + TypeId result = arena->addType(GenericTypeVar{scope}); insertedGenerics.push_back(result); return result; } @@ -264,7 +228,8 @@ struct PureQuantifier : Substitution LUAU_ASSERT(resultTable); *resultTable = *ttv; - resultTable->scope = nullptr; + resultTable->level = TypeLevel{}; + resultTable->scope = scope; resultTable->state = TableState::Generic; return result; @@ -306,6 +271,7 @@ TypeId quantify(TypeArena* arena, TypeId ty, Scope* scope) FunctionTypeVar* ftv = getMutable(*result); LUAU_ASSERT(ftv); + ftv->scope = scope; ftv->generics.insert(ftv->generics.end(), quantifier.insertedGenerics.begin(), quantifier.insertedGenerics.end()); ftv->genericPacks.insert(ftv->genericPacks.end(), quantifier.insertedGenericPacks.begin(), quantifier.insertedGenericPacks.end()); ftv->hasNoGenerics = ftv->generics.empty() && ftv->genericPacks.empty(); diff --git a/Analysis/src/Scope.cpp b/Analysis/src/Scope.cpp index c129b973..9a7d3609 100644 --- a/Analysis/src/Scope.cpp +++ b/Analysis/src/Scope.cpp @@ -21,6 +21,12 @@ Scope::Scope(const ScopePtr& parent, int subLevel) level.subLevel = subLevel; } +void Scope::addBuiltinTypeBinding(const Name& name, const TypeFun& tyFun) +{ + exportedTypeBindings[name] = tyFun; + builtinTypeNames.insert(name); +} + std::optional Scope::lookupType(const Name& name) { const Scope* scope = this; @@ -82,9 +88,9 @@ std::optional Scope::lookupPack(const Name& name) } } -std::optional Scope::linearSearchForBinding(const std::string& name, bool traverseScopeChain) +std::optional Scope::linearSearchForBinding(const std::string& name, bool traverseScopeChain) const { - Scope* scope = this; + const Scope* scope = this; while (scope) { @@ -122,4 +128,22 @@ std::optional Scope::lookup(Symbol sym) } } +bool subsumesStrict(Scope* left, Scope* right) +{ + while (right) + { + if (right->parent.get() == left) + return true; + + right = right->parent.get(); + } + + return false; +} + +bool subsumes(Scope* left, Scope* right) +{ + return left == right || subsumesStrict(left, right); +} + } // namespace Luau diff --git a/Analysis/src/ToString.cpp b/Analysis/src/ToString.cpp index 8ab124ea..a61eb793 100644 --- a/Analysis/src/ToString.cpp +++ b/Analysis/src/ToString.cpp @@ -14,6 +14,7 @@ LUAU_FASTFLAG(LuauLowerBoundsCalculation) LUAU_FASTFLAG(LuauUnknownAndNeverType) LUAU_FASTFLAGVARIABLE(LuauSpecialTypesAsterisked, false) LUAU_FASTFLAGVARIABLE(LuauFixNameMaps, false) +LUAU_FASTFLAGVARIABLE(LuauUnseeArrayTtv, false) /* * Prefix generic typenames with gen- @@ -632,6 +633,10 @@ struct TypeVarStringifier state.emit("{"); stringify(ttv.indexer->indexResultType); state.emit("}"); + + if (FFlag::LuauUnseeArrayTtv) + state.unsee(&ttv); + return; } diff --git a/Analysis/src/TxnLog.cpp b/Analysis/src/TxnLog.cpp index 74d77307..06bde195 100644 --- a/Analysis/src/TxnLog.cpp +++ b/Analysis/src/TxnLog.cpp @@ -289,6 +289,45 @@ PendingTypePack* TxnLog::changeLevel(TypePackId tp, TypeLevel newLevel) return newTp; } +PendingType* TxnLog::changeScope(TypeId ty, NotNull newScope) +{ + LUAU_ASSERT(get(ty) || get(ty) || get(ty) || get(ty)); + + PendingType* newTy = queue(ty); + if (FreeTypeVar* ftv = Luau::getMutable(newTy)) + { + ftv->scope = newScope; + } + else if (TableTypeVar* ttv = Luau::getMutable(newTy)) + { + LUAU_ASSERT(ttv->state == TableState::Free || ttv->state == TableState::Generic); + ttv->scope = newScope; + } + else if (FunctionTypeVar* ftv = Luau::getMutable(newTy)) + { + ftv->scope = newScope; + } + else if (ConstrainedTypeVar* ctv = Luau::getMutable(newTy)) + { + ctv->scope = newScope; + } + + return newTy; +} + +PendingTypePack* TxnLog::changeScope(TypePackId tp, NotNull newScope) +{ + LUAU_ASSERT(get(tp)); + + PendingTypePack* newTp = queue(tp); + if (FreeTypePack* ftp = Luau::getMutable(newTp)) + { + ftp->scope = newScope; + } + + return newTp; +} + PendingType* TxnLog::changeIndexer(TypeId ty, std::optional indexer) { LUAU_ASSERT(get(ty)); diff --git a/Analysis/src/TypeArena.cpp b/Analysis/src/TypeArena.cpp index abf31aee..666ab867 100644 --- a/Analysis/src/TypeArena.cpp +++ b/Analysis/src/TypeArena.cpp @@ -40,6 +40,15 @@ TypeId TypeArena::freshType(Scope* scope) return allocated; } +TypeId TypeArena::freshType(Scope* scope, TypeLevel level) +{ + TypeId allocated = typeVars.allocate(FreeTypeVar{scope, level}); + + asMutable(allocated)->owningArena = this; + + return allocated; +} + TypePackId TypeArena::freshTypePack(Scope* scope) { TypePackId allocated = typePacks.allocate(FreeTypePack{scope}); diff --git a/Analysis/src/TypeChecker2.cpp b/Analysis/src/TypeChecker2.cpp index ea06882a..f98a2123 100644 --- a/Analysis/src/TypeChecker2.cpp +++ b/Analysis/src/TypeChecker2.cpp @@ -17,10 +17,16 @@ #include LUAU_FASTFLAG(DebugLuauLogSolverToJson); +LUAU_FASTFLAG(DebugLuauMagicTypes); namespace Luau { +// TypeInfer.h +// TODO move these +using PrintLineProc = void(*)(const std::string&); +extern PrintLineProc luauPrintLine; + /* Push a scope onto the end of a stack for the lifetime of the StackPusher instance. * TypeChecker2 uses this to maintain knowledge about which scope encloses every * given AstNode. @@ -114,6 +120,19 @@ struct TypeChecker2 TypeId lookupAnnotation(AstType* annotation) { + if (FFlag::DebugLuauMagicTypes) + { + if (auto ref = annotation->as(); ref && ref->name == "_luau_print" && ref->parameters.size > 0) + { + if (auto ann = ref->parameters.data[0].type) + { + TypeId argTy = lookupAnnotation(ref->parameters.data[0].type); + luauPrintLine(format("_luau_print (%d, %d): %s\n", annotation->location.begin.line, annotation->location.begin.column, toString(argTy).c_str())); + return follow(argTy); + } + } + } + TypeId* ty = module->astResolvedTypes.find(annotation); LUAU_ASSERT(ty); return follow(*ty); @@ -284,50 +303,49 @@ struct TypeChecker2 void visit(AstStatLocal* local) { - for (size_t i = 0; i < local->values.size; ++i) + size_t count = std::max(local->values.size, local->vars.size); + for (size_t i = 0; i < count; ++i) { - AstExpr* value = local->values.data[i]; + AstExpr* value = i < local->values.size ? local->values.data[i] : nullptr; - visit(value); + if (value) + visit(value); - if (i == local->values.size - 1) + if (i != local->values.size - 1) { - if (i < local->values.size) + AstLocal* var = i < local->vars.size ? local->vars.data[i] : nullptr; + + if (var && var->annotation) { - TypePackId valueTypes = lookupPack(value); - auto it = begin(valueTypes); - for (size_t j = i; j < local->vars.size; ++j) - { - if (it == end(valueTypes)) - { - break; - } - - AstLocal* var = local->vars.data[i]; - if (var->annotation) - { - TypeId varType = lookupAnnotation(var->annotation); - ErrorVec errors = tryUnify(stack.back(), value->location, *it, varType); - if (!errors.empty()) - reportErrors(std::move(errors)); - } - - ++it; - } + TypeId varType = lookupAnnotation(var->annotation); + TypeId valueType = value ? lookupType(value) : nullptr; + if (valueType && !isSubtype(varType, valueType, stack.back(), singletonTypes, ice, /* anyIsTop */ false)) + reportError(TypeMismatch{varType, valueType}, value->location); } } else { - TypeId valueType = lookupType(value); - AstLocal* var = local->vars.data[i]; + LUAU_ASSERT(value); - if (var->annotation) + TypePackId valueTypes = lookupPack(value); + auto it = begin(valueTypes); + for (size_t j = i; j < local->vars.size; ++j) { - TypeId varType = lookupAnnotation(var->annotation); - if (!isSubtype(varType, valueType, stack.back(), singletonTypes, ice, /* anyIsTop */ false)) + if (it == end(valueTypes)) { - reportError(TypeMismatch{varType, valueType}, value->location); + break; } + + AstLocal* var = local->vars.data[i]; + if (var->annotation) + { + TypeId varType = lookupAnnotation(var->annotation); + ErrorVec errors = tryUnify(stack.back(), value->location, *it, varType); + if (!errors.empty()) + reportErrors(std::move(errors)); + } + + ++it; } } } @@ -345,50 +363,6 @@ struct TypeChecker2 visit(forStatement->body); } - // "Render" a type pack out to an array of a given length. Expands - // variadics and various other things to get there. - std::vector flatten(TypeArena& arena, TypePackId pack, size_t length) - { - std::vector result; - - auto it = begin(pack); - auto endIt = end(pack); - - while (it != endIt) - { - result.push_back(*it); - - if (result.size() >= length) - return result; - - ++it; - } - - if (!it.tail()) - return result; - - TypePackId tail = *it.tail(); - if (get(tail)) - LUAU_ASSERT(0); - else if (auto vtp = get(tail)) - { - while (result.size() < length) - result.push_back(vtp->ty); - } - else if (get(tail) || get(tail)) - { - while (result.size() < length) - result.push_back(arena.addType(FreeTypeVar{nullptr})); - } - else if (auto etp = get(tail)) - { - while (result.size() < length) - result.push_back(singletonTypes->errorRecoveryType()); - } - - return result; - } - void visit(AstStatForIn* forInStatement) { for (AstLocal* local : forInStatement->vars) @@ -426,7 +400,7 @@ struct TypeChecker2 TypePackId iteratorPack = arena.addTypePack(valueTypes, iteratorTail); // ... and then expand it out to 3 values (if possible) - const std::vector iteratorTypes = flatten(arena, iteratorPack, 3); + const std::vector iteratorTypes = flatten(arena, singletonTypes, iteratorPack, 3); if (iteratorTypes.empty()) { reportError(GenericError{"for..in loops require at least one value to iterate over. Got zero"}, getLocation(forInStatement->values)); @@ -434,6 +408,72 @@ struct TypeChecker2 } TypeId iteratorTy = follow(iteratorTypes[0]); + auto checkFunction = [this, &arena, &scope, &forInStatement, &variableTypes](const FunctionTypeVar* iterFtv, std::vector iterTys, bool isMm) + { + if (iterTys.size() < 1 || iterTys.size() > 3) + { + if (isMm) + reportError(GenericError{"__iter metamethod must return (next[, table[, state]])"}, getLocation(forInStatement->values)); + else + reportError(GenericError{"for..in loops must be passed (next[, table[, state]])"}, getLocation(forInStatement->values)); + + return; + } + + // It is okay if there aren't enough iterators, but the iteratee must provide enough. + std::vector expectedVariableTypes = flatten(arena, singletonTypes, iterFtv->retTypes, variableTypes.size()); + if (expectedVariableTypes.size() < variableTypes.size()) + { + if (isMm) + reportError(GenericError{"__iter metamethod's next() function does not return enough values"}, getLocation(forInStatement->values)); + else + reportError(GenericError{"next() does not return enough values"}, forInStatement->values.data[0]->location); + } + + for (size_t i = 0; i < std::min(expectedVariableTypes.size(), variableTypes.size()); ++i) + reportErrors(tryUnify(scope, forInStatement->vars.data[i]->location, variableTypes[i], expectedVariableTypes[i])); + + // nextFn is going to be invoked with (arrayTy, startIndexTy) + + // It will be passed two arguments on every iteration save the + // first. + + // It may be invoked with 0 or 1 argument on the first iteration. + // This depends on the types in iterateePack and therefore + // iteratorTypes. + + // If iteratorTypes is too short to be a valid call to nextFn, we have to report a count mismatch error. + // If 2 is too short to be a valid call to nextFn, we have to report a count mismatch error. + // If 2 is too long to be a valid call to nextFn, we have to report a count mismatch error. + auto [minCount, maxCount] = getParameterExtents(TxnLog::empty(), iterFtv->argTypes, /*includeHiddenVariadics*/ true); + + if (minCount > 2) + reportError(CountMismatch{2, std::nullopt, minCount, CountMismatch::Arg}, forInStatement->vars.data[0]->location); + if (maxCount && *maxCount < 2) + reportError(CountMismatch{2, std::nullopt, *maxCount, CountMismatch::Arg}, forInStatement->vars.data[0]->location); + + const std::vector flattenedArgTypes = flatten(arena, singletonTypes, iterFtv->argTypes, 2); + size_t firstIterationArgCount = iterTys.empty() ? 0 : iterTys.size() - 1; + size_t actualArgCount = expectedVariableTypes.size(); + + if (firstIterationArgCount < minCount) + reportError(CountMismatch{2, std::nullopt, firstIterationArgCount, CountMismatch::Arg}, forInStatement->vars.data[0]->location); + else if (actualArgCount < minCount) + reportError(CountMismatch{2, std::nullopt, actualArgCount, CountMismatch::Arg}, forInStatement->vars.data[0]->location); + + if (iterTys.size() >= 2 && flattenedArgTypes.size() > 0) + { + size_t valueIndex = forInStatement->values.size > 1 ? 1 : 0; + reportErrors(tryUnify(scope, forInStatement->values.data[valueIndex]->location, iterTys[1], flattenedArgTypes[0])); + } + + if (iterTys.size() == 3 && flattenedArgTypes.size() > 1) + { + size_t valueIndex = forInStatement->values.size > 2 ? 2 : 0; + reportErrors(tryUnify(scope, forInStatement->values.data[valueIndex]->location, iterTys[2], flattenedArgTypes[1])); + } + }; + /* * If the first iterator argument is a function * * There must be 1 to 3 iterator arguments. Name them (nextTy, @@ -451,58 +491,7 @@ struct TypeChecker2 */ if (const FunctionTypeVar* nextFn = get(iteratorTy)) { - if (iteratorTypes.size() < 1 || iteratorTypes.size() > 3) - reportError(GenericError{"for..in loops must be passed (next, [table[, state]])"}, getLocation(forInStatement->values)); - - // It is okay if there aren't enough iterators, but the iteratee must provide enough. - std::vector expectedVariableTypes = flatten(arena, nextFn->retTypes, variableTypes.size()); - if (expectedVariableTypes.size() < variableTypes.size()) - reportError(GenericError{"next() does not return enough values"}, forInStatement->vars.data[0]->location); - - for (size_t i = 0; i < std::min(expectedVariableTypes.size(), variableTypes.size()); ++i) - reportErrors(tryUnify(scope, forInStatement->vars.data[i]->location, variableTypes[i], expectedVariableTypes[i])); - - // nextFn is going to be invoked with (arrayTy, startIndexTy) - - // It will be passed two arguments on every iteration save the - // first. - - // It may be invoked with 0 or 1 argument on the first iteration. - // This depends on the types in iterateePack and therefore - // iteratorTypes. - - // If iteratorTypes is too short to be a valid call to nextFn, we have to report a count mismatch error. - // If 2 is too short to be a valid call to nextFn, we have to report a count mismatch error. - // If 2 is too long to be a valid call to nextFn, we have to report a count mismatch error. - auto [minCount, maxCount] = getParameterExtents(TxnLog::empty(), nextFn->argTypes, /*includeHiddenVariadics*/ true); - - if (minCount > 2) - reportError(CountMismatch{2, std::nullopt, minCount, CountMismatch::Arg}, forInStatement->vars.data[0]->location); - if (maxCount && *maxCount < 2) - reportError(CountMismatch{2, std::nullopt, *maxCount, CountMismatch::Arg}, forInStatement->vars.data[0]->location); - - const std::vector flattenedArgTypes = flatten(arena, nextFn->argTypes, 2); - const auto [argTypes, argsTail] = Luau::flatten(nextFn->argTypes); - - size_t firstIterationArgCount = iteratorTypes.empty() ? 0 : iteratorTypes.size() - 1; - size_t actualArgCount = expectedVariableTypes.size(); - - if (firstIterationArgCount < minCount) - reportError(CountMismatch{2, std::nullopt, firstIterationArgCount, CountMismatch::Arg}, forInStatement->vars.data[0]->location); - else if (actualArgCount < minCount) - reportError(CountMismatch{2, std::nullopt, actualArgCount, CountMismatch::Arg}, forInStatement->vars.data[0]->location); - - if (iteratorTypes.size() >= 2 && flattenedArgTypes.size() > 0) - { - size_t valueIndex = forInStatement->values.size > 1 ? 1 : 0; - reportErrors(tryUnify(scope, forInStatement->values.data[valueIndex]->location, iteratorTypes[1], flattenedArgTypes[0])); - } - - if (iteratorTypes.size() == 3 && flattenedArgTypes.size() > 1) - { - size_t valueIndex = forInStatement->values.size > 2 ? 2 : 0; - reportErrors(tryUnify(scope, forInStatement->values.data[valueIndex]->location, iteratorTypes[2], flattenedArgTypes[1])); - } + checkFunction(nextFn, iteratorTypes, false); } else if (const TableTypeVar* ttv = get(iteratorTy)) { @@ -519,6 +508,62 @@ struct TypeChecker2 { // nothing } + else if (std::optional iterMmTy = findMetatableEntry(singletonTypes, module->errors, iteratorTy, "__iter", forInStatement->values.data[0]->location)) + { + Instantiation instantiation{TxnLog::empty(), &arena, TypeLevel{}, scope}; + + if (std::optional instantiatedIterMmTy = instantiation.substitute(*iterMmTy)) + { + if (const FunctionTypeVar* iterMmFtv = get(*instantiatedIterMmTy)) + { + TypePackId argPack = arena.addTypePack({iteratorTy}); + reportErrors(tryUnify(scope, forInStatement->values.data[0]->location, argPack, iterMmFtv->argTypes)); + + std::vector mmIteratorTypes = flatten(arena, singletonTypes, iterMmFtv->retTypes, 3); + + if (mmIteratorTypes.size() == 0) + { + reportError(GenericError{"__iter must return at least one value"}, forInStatement->values.data[0]->location); + return; + } + + TypeId nextFn = follow(mmIteratorTypes[0]); + + if (std::optional instantiatedNextFn = instantiation.substitute(nextFn)) + { + std::vector instantiatedIteratorTypes = mmIteratorTypes; + instantiatedIteratorTypes[0] = *instantiatedNextFn; + + if (const FunctionTypeVar* nextFtv = get(*instantiatedNextFn)) + { + checkFunction(nextFtv, instantiatedIteratorTypes, true); + } + else + { + reportError(CannotCallNonFunction{*instantiatedNextFn}, forInStatement->values.data[0]->location); + } + } + else + { + reportError(UnificationTooComplex{}, forInStatement->values.data[0]->location); + } + } + else + { + // TODO: This will not tell the user that this is because the + // metamethod isn't callable. This is not ideal, and we should + // improve this error message. + + // TODO: This will also not handle intersections of functions or + // callable tables (which are supported by the runtime). + reportError(CannotCallNonFunction{*iterMmTy}, forInStatement->values.data[0]->location); + } + } + else + { + reportError(UnificationTooComplex{}, forInStatement->values.data[0]->location); + } + } else { reportError(CannotCallNonFunction{iteratorTy}, forInStatement->values.data[0]->location); @@ -730,7 +775,7 @@ struct TypeChecker2 visit(arg); TypeArena arena; - Instantiation instantiation{TxnLog::empty(), &arena, TypeLevel{}}; + Instantiation instantiation{TxnLog::empty(), &arena, TypeLevel{}, stack.back()}; TypePackId expectedRetType = lookupPack(call); TypeId functionType = lookupType(call->func); diff --git a/Analysis/src/TypeInfer.cpp b/Analysis/src/TypeInfer.cpp index b6f20ac8..b96046be 100644 --- a/Analysis/src/TypeInfer.cpp +++ b/Analysis/src/TypeInfer.cpp @@ -47,6 +47,8 @@ LUAU_FASTFLAGVARIABLE(LuauNeverTypesAndOperatorsInference, false) LUAU_FASTFLAGVARIABLE(LuauReturnsFromCallsitesAreNotWidened, false) LUAU_FASTFLAGVARIABLE(LuauCompleteVisitor, false) LUAU_FASTFLAGVARIABLE(LuauUnionOfTypesFollow, false) +LUAU_FASTFLAGVARIABLE(LuauReportShadowedTypeAlias, false) +LUAU_FASTFLAGVARIABLE(LuauBetterMessagingOnCountMismatch, false) namespace Luau { @@ -66,9 +68,7 @@ static void defaultLuauPrintLine(const std::string& s) printf("%s\n", s.c_str()); } -using PrintLineProc = decltype(&defaultLuauPrintLine); - -static PrintLineProc luauPrintLine = &defaultLuauPrintLine; +PrintLineProc luauPrintLine = &defaultLuauPrintLine; void setPrintLine(PrintLineProc pl) { @@ -270,16 +270,16 @@ TypeChecker::TypeChecker(ModuleResolver* resolver, NotNull singl { globalScope = std::make_shared(globalTypes.addTypePack(TypePackVar{FreeTypePack{TypeLevel{}}})); - globalScope->exportedTypeBindings["any"] = TypeFun{{}, anyType}; - globalScope->exportedTypeBindings["nil"] = TypeFun{{}, nilType}; - globalScope->exportedTypeBindings["number"] = TypeFun{{}, numberType}; - globalScope->exportedTypeBindings["string"] = TypeFun{{}, stringType}; - globalScope->exportedTypeBindings["boolean"] = TypeFun{{}, booleanType}; - globalScope->exportedTypeBindings["thread"] = TypeFun{{}, threadType}; + globalScope->addBuiltinTypeBinding("any", TypeFun{{}, anyType}); + globalScope->addBuiltinTypeBinding("nil", TypeFun{{}, nilType}); + globalScope->addBuiltinTypeBinding("number", TypeFun{{}, numberType}); + globalScope->addBuiltinTypeBinding("string", TypeFun{{}, stringType}); + globalScope->addBuiltinTypeBinding("boolean", TypeFun{{}, booleanType}); + globalScope->addBuiltinTypeBinding("thread", TypeFun{{}, threadType}); if (FFlag::LuauUnknownAndNeverType) { - globalScope->exportedTypeBindings["unknown"] = TypeFun{{}, unknownType}; - globalScope->exportedTypeBindings["never"] = TypeFun{{}, neverType}; + globalScope->addBuiltinTypeBinding("unknown", TypeFun{{}, unknownType}); + globalScope->addBuiltinTypeBinding("never", TypeFun{{}, neverType}); } } @@ -534,7 +534,7 @@ void TypeChecker::checkBlockWithoutRecursionCheck(const ScopePtr& scope, const A { if (const auto& typealias = stat->as()) { - check(scope, *typealias, subLevel, true); + prototype(scope, *typealias, subLevel); ++subLevel; } } @@ -698,6 +698,10 @@ LUAU_NOINLINE void TypeChecker::checkBlockTypeAliases(const ScopePtr& scope, std auto& bindings = typealias->exported ? scope->exportedTypeBindings : scope->privateTypeBindings; Name name = typealias->name.value; + + if (FFlag::LuauReportShadowedTypeAlias && duplicateTypeAliases.contains({typealias->exported, name})) + continue; + TypeId type = bindings[name].type; if (get(follow(type))) { @@ -1109,8 +1113,23 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatLocal& local) TypePackId valuePack = checkExprList(scope, local.location, local.values, /* substituteFreeForNil= */ true, instantiateGenerics, expectedTypes).type; + // If the expression list only contains one expression and it's a function call or is otherwise within parentheses, use FunctionResult. + // Otherwise, we'll want to use ExprListResult to make the error messaging more general. + CountMismatch::Context ctx = FFlag::LuauBetterMessagingOnCountMismatch ? CountMismatch::ExprListResult : CountMismatch::FunctionResult; + if (FFlag::LuauBetterMessagingOnCountMismatch) + { + if (local.values.size == 1) + { + AstExpr* e = local.values.data[0]; + while (auto group = e->as()) + e = group->expr; + if (e->is()) + ctx = CountMismatch::FunctionResult; + } + } + Unifier state = mkUnifier(scope, local.location); - state.ctx = CountMismatch::Result; + state.ctx = ctx; state.tryUnify(valuePack, variablePack); reportErrors(state.errors); @@ -1472,10 +1491,8 @@ void TypeChecker::check(const ScopePtr& scope, TypeId ty, const ScopePtr& funSco scope->bindings[function.name] = {quantify(funScope, ty, function.name->location), function.name->location}; } -void TypeChecker::check(const ScopePtr& scope, const AstStatTypeAlias& typealias, int subLevel, bool forwardDeclare) +void TypeChecker::check(const ScopePtr& scope, const AstStatTypeAlias& typealias) { - // This function should be called at most twice for each type alias. - // Once with forwardDeclare, and once without. Name name = typealias.name.value; // If the alias is missing a name, we can't do anything with it. Ignore it. @@ -1490,14 +1507,134 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatTypeAlias& typealias auto& bindingsMap = typealias.exported ? scope->exportedTypeBindings : scope->privateTypeBindings; - if (forwardDeclare) - { - if (binding) - { - Location location = scope->typeAliasLocations[name]; - reportError(TypeError{typealias.location, DuplicateTypeDefinition{name, location}}); + // If the first pass failed (this should mean a duplicate definition), the second pass isn't going to be + // interesting. + if (duplicateTypeAliases.find({typealias.exported, name})) + return; + // By now this alias must have been `prototype()`d first. + if (!binding) + ice("Not predeclared"); + + ScopePtr aliasScope = childScope(scope, typealias.location); + aliasScope->level = scope->level.incr(); + + for (auto param : binding->typeParams) + { + auto generic = get(param.ty); + LUAU_ASSERT(generic); + aliasScope->privateTypeBindings[generic->name] = TypeFun{{}, param.ty}; + } + + for (auto param : binding->typePackParams) + { + auto generic = get(param.tp); + LUAU_ASSERT(generic); + aliasScope->privateTypePackBindings[generic->name] = param.tp; + } + + TypeId ty = resolveType(aliasScope, *typealias.type); + if (auto ttv = getMutable(follow(ty))) + { + // If the table is already named and we want to rename the type function, we have to bind new alias to a copy + // Additionally, we can't modify types that come from other modules + if (ttv->name || follow(ty)->owningArena != ¤tModule->internalTypes) + { + bool sameTys = std::equal(ttv->instantiatedTypeParams.begin(), ttv->instantiatedTypeParams.end(), binding->typeParams.begin(), + binding->typeParams.end(), [](auto&& itp, auto&& tp) { + return itp == tp.ty; + }); + bool sameTps = std::equal(ttv->instantiatedTypePackParams.begin(), ttv->instantiatedTypePackParams.end(), binding->typePackParams.begin(), + binding->typePackParams.end(), [](auto&& itpp, auto&& tpp) { + return itpp == tpp.tp; + }); + + // Copy can be skipped if this is an identical alias + if (!ttv->name || ttv->name != name || !sameTys || !sameTps) + { + // This is a shallow clone, original recursive links to self are not updated + TableTypeVar clone = TableTypeVar{ttv->props, ttv->indexer, ttv->level, ttv->state}; + clone.definitionModuleName = ttv->definitionModuleName; + clone.name = name; + + for (auto param : binding->typeParams) + clone.instantiatedTypeParams.push_back(param.ty); + + for (auto param : binding->typePackParams) + clone.instantiatedTypePackParams.push_back(param.tp); + + bool isNormal = ty->normal; + ty = addType(std::move(clone)); + + if (FFlag::LuauLowerBoundsCalculation) + asMutable(ty)->normal = isNormal; + } + } + else + { + ttv->name = name; + + ttv->instantiatedTypeParams.clear(); + for (auto param : binding->typeParams) + ttv->instantiatedTypeParams.push_back(param.ty); + + ttv->instantiatedTypePackParams.clear(); + for (auto param : binding->typePackParams) + ttv->instantiatedTypePackParams.push_back(param.tp); + } + } + else if (auto mtv = getMutable(follow(ty))) + { + // We can't modify types that come from other modules + if (follow(ty)->owningArena == ¤tModule->internalTypes) + mtv->syntheticName = name; + } + + TypeId& bindingType = bindingsMap[name].type; + + if (unify(ty, bindingType, aliasScope, typealias.location)) + bindingType = ty; + + if (FFlag::LuauLowerBoundsCalculation) + { + auto [t, ok] = normalize(bindingType, currentModule, singletonTypes, *iceHandler); + bindingType = t; + if (!ok) + reportError(typealias.location, NormalizationTooComplex{}); + } +} + +void TypeChecker::prototype(const ScopePtr& scope, const AstStatTypeAlias& typealias, int subLevel) +{ + Name name = typealias.name.value; + + // If the alias is missing a name, we can't do anything with it. Ignore it. + if (name == kParseNameError) + return; + + std::optional binding; + if (auto it = scope->exportedTypeBindings.find(name); it != scope->exportedTypeBindings.end()) + binding = it->second; + else if (auto it = scope->privateTypeBindings.find(name); it != scope->privateTypeBindings.end()) + binding = it->second; + + auto& bindingsMap = typealias.exported ? scope->exportedTypeBindings : scope->privateTypeBindings; + + if (binding) + { + Location location = scope->typeAliasLocations[name]; + reportError(TypeError{typealias.location, DuplicateTypeDefinition{name, location}}); + + if (!FFlag::LuauReportShadowedTypeAlias) bindingsMap[name] = TypeFun{binding->typeParams, binding->typePackParams, errorRecoveryType(anyType)}; + + duplicateTypeAliases.insert({typealias.exported, name}); + } + else if (FFlag::LuauReportShadowedTypeAlias) + { + if (globalScope->builtinTypeNames.contains(name)) + { + reportError(typealias.location, DuplicateTypeDefinition{name}); duplicateTypeAliases.insert({typealias.exported, name}); } else @@ -1520,100 +1657,20 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatTypeAlias& typealias } else { - // If the first pass failed (this should mean a duplicate definition), the second pass isn't going to be - // interesting. - if (duplicateTypeAliases.find({typealias.exported, name})) - return; - - if (!binding) - ice("Not predeclared"); - ScopePtr aliasScope = childScope(scope, typealias.location); aliasScope->level = scope->level.incr(); + aliasScope->level.subLevel = subLevel; - for (auto param : binding->typeParams) - { - auto generic = get(param.ty); - LUAU_ASSERT(generic); - aliasScope->privateTypeBindings[generic->name] = TypeFun{{}, param.ty}; - } + auto [generics, genericPacks] = + createGenericTypes(aliasScope, scope->level, typealias, typealias.generics, typealias.genericPacks, /* useCache = */ true); - for (auto param : binding->typePackParams) - { - auto generic = get(param.tp); - LUAU_ASSERT(generic); - aliasScope->privateTypePackBindings[generic->name] = param.tp; - } + TypeId ty = freshType(aliasScope); + FreeTypeVar* ftv = getMutable(ty); + LUAU_ASSERT(ftv); + ftv->forwardedTypeAlias = true; + bindingsMap[name] = {std::move(generics), std::move(genericPacks), ty}; - TypeId ty = resolveType(aliasScope, *typealias.type); - if (auto ttv = getMutable(follow(ty))) - { - // If the table is already named and we want to rename the type function, we have to bind new alias to a copy - // Additionally, we can't modify types that come from other modules - if (ttv->name || follow(ty)->owningArena != ¤tModule->internalTypes) - { - bool sameTys = std::equal(ttv->instantiatedTypeParams.begin(), ttv->instantiatedTypeParams.end(), binding->typeParams.begin(), - binding->typeParams.end(), [](auto&& itp, auto&& tp) { - return itp == tp.ty; - }); - bool sameTps = std::equal(ttv->instantiatedTypePackParams.begin(), ttv->instantiatedTypePackParams.end(), - binding->typePackParams.begin(), binding->typePackParams.end(), [](auto&& itpp, auto&& tpp) { - return itpp == tpp.tp; - }); - - // Copy can be skipped if this is an identical alias - if (!ttv->name || ttv->name != name || !sameTys || !sameTps) - { - // This is a shallow clone, original recursive links to self are not updated - TableTypeVar clone = TableTypeVar{ttv->props, ttv->indexer, ttv->level, ttv->state}; - clone.definitionModuleName = ttv->definitionModuleName; - clone.name = name; - - for (auto param : binding->typeParams) - clone.instantiatedTypeParams.push_back(param.ty); - - for (auto param : binding->typePackParams) - clone.instantiatedTypePackParams.push_back(param.tp); - - bool isNormal = ty->normal; - ty = addType(std::move(clone)); - - if (FFlag::LuauLowerBoundsCalculation) - asMutable(ty)->normal = isNormal; - } - } - else - { - ttv->name = name; - - ttv->instantiatedTypeParams.clear(); - for (auto param : binding->typeParams) - ttv->instantiatedTypeParams.push_back(param.ty); - - ttv->instantiatedTypePackParams.clear(); - for (auto param : binding->typePackParams) - ttv->instantiatedTypePackParams.push_back(param.tp); - } - } - else if (auto mtv = getMutable(follow(ty))) - { - // We can't modify types that come from other modules - if (follow(ty)->owningArena == ¤tModule->internalTypes) - mtv->syntheticName = name; - } - - TypeId& bindingType = bindingsMap[name].type; - - if (unify(ty, bindingType, aliasScope, typealias.location)) - bindingType = ty; - - if (FFlag::LuauLowerBoundsCalculation) - { - auto [t, ok] = normalize(bindingType, currentModule, singletonTypes, *iceHandler); - bindingType = t; - if (!ok) - reportError(typealias.location, NormalizationTooComplex{}); - } + scope->typeAliasLocations[name] = typealias.location; } } @@ -4152,7 +4209,7 @@ std::optional> TypeChecker::checkCallOverload(const Sc TypePackId adjustedArgPack = addTypePack(TypePack{std::move(adjustedArgTypes), it.tail()}); TxnLog log; - promoteTypeLevels(log, ¤tModule->internalTypes, level, retPack); + promoteTypeLevels(log, ¤tModule->internalTypes, level, /*scope*/ nullptr, /*useScope*/ false, retPack); log.commit(); *asMutable(fn) = FunctionTypeVar{level, adjustedArgPack, retPack}; @@ -4712,7 +4769,7 @@ TypeId TypeChecker::instantiate(const ScopePtr& scope, TypeId ty, Location locat if (ftv && ftv->hasNoGenerics) return ty; - Instantiation instantiation{log, ¤tModule->internalTypes, scope->level}; + Instantiation instantiation{log, ¤tModule->internalTypes, scope->level, /*scope*/ nullptr}; if (FFlag::LuauAutocompleteDynamicLimits && instantiationChildLimit) instantiation.childLimit = *instantiationChildLimit; diff --git a/Analysis/src/TypeUtils.cpp b/Analysis/src/TypeUtils.cpp index 6ea04ea9..ca00c269 100644 --- a/Analysis/src/TypeUtils.cpp +++ b/Analysis/src/TypeUtils.cpp @@ -224,4 +224,46 @@ std::pair> getParameterExtents(const TxnLog* log, return {minCount, minCount + optionalCount}; } +std::vector flatten(TypeArena& arena, NotNull singletonTypes, TypePackId pack, size_t length) +{ + std::vector result; + + auto it = begin(pack); + auto endIt = end(pack); + + while (it != endIt) + { + result.push_back(*it); + + if (result.size() >= length) + return result; + + ++it; + } + + if (!it.tail()) + return result; + + TypePackId tail = *it.tail(); + if (get(tail)) + LUAU_ASSERT(0); + else if (auto vtp = get(tail)) + { + while (result.size() < length) + result.push_back(vtp->ty); + } + else if (get(tail) || get(tail)) + { + while (result.size() < length) + result.push_back(arena.addType(FreeTypeVar{nullptr})); + } + else if (auto etp = get(tail)) + { + while (result.size() < length) + result.push_back(singletonTypes->errorRecoveryType()); + } + + return result; +} + } // namespace Luau diff --git a/Analysis/src/TypeVar.cpp b/Analysis/src/TypeVar.cpp index 3a820ea6..bf6bf34a 100644 --- a/Analysis/src/TypeVar.cpp +++ b/Analysis/src/TypeVar.cpp @@ -474,6 +474,17 @@ FunctionTypeVar::FunctionTypeVar(TypeLevel level, TypePackId argTypes, TypePackI { } +FunctionTypeVar::FunctionTypeVar( + TypeLevel level, Scope* scope, TypePackId argTypes, TypePackId retTypes, std::optional defn, bool hasSelf) + : level(level) + , scope(scope) + , argTypes(argTypes) + , retTypes(retTypes) + , definition(std::move(defn)) + , hasSelf(hasSelf) +{ +} + FunctionTypeVar::FunctionTypeVar(std::vector generics, std::vector genericPacks, TypePackId argTypes, TypePackId retTypes, std::optional defn, bool hasSelf) : generics(generics) @@ -497,9 +508,23 @@ FunctionTypeVar::FunctionTypeVar(TypeLevel level, std::vector generics, { } -TableTypeVar::TableTypeVar(TableState state, TypeLevel level) +FunctionTypeVar::FunctionTypeVar(TypeLevel level, Scope* scope, std::vector generics, std::vector genericPacks, + TypePackId argTypes, TypePackId retTypes, std::optional defn, bool hasSelf) + : level(level) + , scope(scope) + , generics(generics) + , genericPacks(genericPacks) + , argTypes(argTypes) + , retTypes(retTypes) + , definition(std::move(defn)) + , hasSelf(hasSelf) +{ +} + +TableTypeVar::TableTypeVar(TableState state, TypeLevel level, Scope* scope) : state(state) , level(level) + , scope(scope) { } @@ -511,6 +536,15 @@ TableTypeVar::TableTypeVar(const Props& props, const std::optional { } +TableTypeVar::TableTypeVar(const Props& props, const std::optional& indexer, TypeLevel level, Scope* scope, TableState state) + : props(props) + , indexer(indexer) + , state(state) + , level(level) + , scope(scope) +{ +} + // Test TypeVars for equivalence // More complex than we'd like because TypeVars can self-reference. diff --git a/Analysis/src/Unifiable.cpp b/Analysis/src/Unifiable.cpp index fa76e820..a3d4540c 100644 --- a/Analysis/src/Unifiable.cpp +++ b/Analysis/src/Unifiable.cpp @@ -18,6 +18,13 @@ Free::Free(Scope* scope) { } +Free::Free(Scope* scope, TypeLevel level) + : index(++nextIndex) + , level(level) + , scope(scope) +{ +} + int Free::nextIndex = 0; Generic::Generic() diff --git a/Analysis/src/Unifier.cpp b/Analysis/src/Unifier.cpp index 505e9e43..c13a6f8b 100644 --- a/Analysis/src/Unifier.cpp +++ b/Analysis/src/Unifier.cpp @@ -23,6 +23,7 @@ LUAU_FASTFLAG(LuauUnknownAndNeverType) LUAU_FASTFLAGVARIABLE(LuauScalarShapeSubtyping, false) LUAU_FASTFLAG(LuauClassTypeVarsInSubstitution) LUAU_FASTFLAG(LuauCallUnifyPackTails) +LUAU_FASTFLAG(DebugLuauDeferredConstraintResolution) namespace Luau { @@ -33,10 +34,15 @@ struct PromoteTypeLevels final : TypeVarOnceVisitor const TypeArena* typeArena = nullptr; TypeLevel minLevel; - PromoteTypeLevels(TxnLog& log, const TypeArena* typeArena, TypeLevel minLevel) + Scope* outerScope = nullptr; + bool useScopes; + + PromoteTypeLevels(TxnLog& log, const TypeArena* typeArena, TypeLevel minLevel, Scope* outerScope, bool useScopes) : log(log) , typeArena(typeArena) , minLevel(minLevel) + , outerScope(outerScope) + , useScopes(useScopes) { } @@ -44,9 +50,18 @@ struct PromoteTypeLevels final : TypeVarOnceVisitor void promote(TID ty, T* t) { LUAU_ASSERT(t); - if (minLevel.subsumesStrict(t->level)) + + if (useScopes) { - log.changeLevel(ty, minLevel); + if (subsumesStrict(outerScope, t->scope)) + log.changeScope(ty, NotNull{outerScope}); + } + else + { + if (minLevel.subsumesStrict(t->level)) + { + log.changeLevel(ty, minLevel); + } } } @@ -123,23 +138,23 @@ struct PromoteTypeLevels final : TypeVarOnceVisitor } }; -static void promoteTypeLevels(TxnLog& log, const TypeArena* typeArena, TypeLevel minLevel, TypeId ty) +static void promoteTypeLevels(TxnLog& log, const TypeArena* typeArena, TypeLevel minLevel, Scope* outerScope, bool useScopes, TypeId ty) { // Type levels of types from other modules are already global, so we don't need to promote anything inside if (ty->owningArena != typeArena) return; - PromoteTypeLevels ptl{log, typeArena, minLevel}; + PromoteTypeLevels ptl{log, typeArena, minLevel, outerScope, useScopes}; ptl.traverse(ty); } -void promoteTypeLevels(TxnLog& log, const TypeArena* typeArena, TypeLevel minLevel, TypePackId tp) +void promoteTypeLevels(TxnLog& log, const TypeArena* typeArena, TypeLevel minLevel, Scope* outerScope, bool useScopes, TypePackId tp) { // Type levels of types from other modules are already global, so we don't need to promote anything inside if (tp->owningArena != typeArena) return; - PromoteTypeLevels ptl{log, typeArena, minLevel}; + PromoteTypeLevels ptl{log, typeArena, minLevel, outerScope, useScopes}; ptl.traverse(tp); } @@ -318,6 +333,16 @@ static std::optional> getTableMat return std::nullopt; } +// TODO: Inline and clip with FFlag::DebugLuauDeferredConstraintResolution +template +static bool subsumes(bool useScopes, TY_A* left, TY_B* right) +{ + if (useScopes) + return subsumes(left->scope, right->scope); + else + return left->level.subsumes(right->level); +} + Unifier::Unifier(TypeArena* types, NotNull singletonTypes, Mode mode, NotNull scope, const Location& location, Variance variance, UnifierSharedState& sharedState, TxnLog* parentLog) : types(types) @@ -375,7 +400,7 @@ void Unifier::tryUnify_(TypeId subTy, TypeId superTy, bool isFunctionCall, bool auto superFree = log.getMutable(superTy); auto subFree = log.getMutable(subTy); - if (superFree && subFree && superFree->level.subsumes(subFree->level)) + if (superFree && subFree && subsumes(useScopes, superFree, subFree)) { if (!occursCheck(subTy, superTy)) log.replace(subTy, BoundTypeVar(superTy)); @@ -386,7 +411,7 @@ void Unifier::tryUnify_(TypeId subTy, TypeId superTy, bool isFunctionCall, bool { if (!occursCheck(superTy, subTy)) { - if (superFree->level.subsumes(subFree->level)) + if (subsumes(useScopes, superFree, subFree)) { log.changeLevel(subTy, superFree->level); } @@ -400,7 +425,7 @@ void Unifier::tryUnify_(TypeId subTy, TypeId superTy, bool isFunctionCall, bool { // Unification can't change the level of a generic. auto subGeneric = log.getMutable(subTy); - if (subGeneric && !subGeneric->level.subsumes(superFree->level)) + if (subGeneric && !subsumes(useScopes, subGeneric, superFree)) { // TODO: a more informative error message? CLI-39912 reportError(TypeError{location, GenericError{"Generic subtype escaping scope"}}); @@ -409,7 +434,7 @@ void Unifier::tryUnify_(TypeId subTy, TypeId superTy, bool isFunctionCall, bool if (!occursCheck(superTy, subTy)) { - promoteTypeLevels(log, types, superFree->level, subTy); + promoteTypeLevels(log, types, superFree->level, superFree->scope, useScopes, subTy); Widen widen{types, singletonTypes}; log.replace(superTy, BoundTypeVar(widen(subTy))); @@ -429,7 +454,7 @@ void Unifier::tryUnify_(TypeId subTy, TypeId superTy, bool isFunctionCall, bool // Unification can't change the level of a generic. auto superGeneric = log.getMutable(superTy); - if (superGeneric && !superGeneric->level.subsumes(subFree->level)) + if (superGeneric && !subsumes(useScopes, superGeneric, subFree)) { // TODO: a more informative error message? CLI-39912 reportError(TypeError{location, GenericError{"Generic supertype escaping scope"}}); @@ -438,7 +463,7 @@ void Unifier::tryUnify_(TypeId subTy, TypeId superTy, bool isFunctionCall, bool if (!occursCheck(subTy, superTy)) { - promoteTypeLevels(log, types, subFree->level, superTy); + promoteTypeLevels(log, types, subFree->level, subFree->scope, useScopes, superTy); log.replace(subTy, BoundTypeVar(superTy)); } @@ -855,6 +880,7 @@ struct WeirdIter size_t index; bool growing; TypeLevel level; + Scope* scope = nullptr; WeirdIter(TypePackId packId, TxnLog& log) : packId(packId) @@ -915,6 +941,7 @@ struct WeirdIter LUAU_ASSERT(log.getMutable(newTail)); level = log.getMutable(packId)->level; + scope = log.getMutable(packId)->scope; log.replace(packId, BoundTypePack(newTail)); packId = newTail; pack = log.getMutable(newTail); @@ -1055,8 +1082,8 @@ void Unifier::tryUnify_(TypePackId subTp, TypePackId superTp, bool isFunctionCal auto superIter = WeirdIter(superTp, log); auto subIter = WeirdIter(subTp, log); - auto mkFreshType = [this](TypeLevel level) { - return types->freshType(level); + auto mkFreshType = [this](Scope* scope, TypeLevel level) { + return types->freshType(scope, level); }; const TypePackId emptyTp = types->addTypePack(TypePack{{}, std::nullopt}); @@ -1072,12 +1099,12 @@ void Unifier::tryUnify_(TypePackId subTp, TypePackId superTp, bool isFunctionCal if (superIter.good() && subIter.growing) { - subIter.pushType(mkFreshType(subIter.level)); + subIter.pushType(mkFreshType(subIter.scope, subIter.level)); } if (subIter.good() && superIter.growing) { - superIter.pushType(mkFreshType(superIter.level)); + superIter.pushType(mkFreshType(superIter.scope, superIter.level)); } if (superIter.good() && subIter.good()) @@ -1158,7 +1185,7 @@ void Unifier::tryUnify_(TypePackId subTp, TypePackId superTp, bool isFunctionCal // these to produce the expected error message. size_t expectedSize = size(superTp); size_t actualSize = size(subTp); - if (ctx == CountMismatch::Result) + if (ctx == CountMismatch::FunctionResult || ctx == CountMismatch::ExprListResult) std::swap(expectedSize, actualSize); reportError(TypeError{location, CountMismatch{expectedSize, std::nullopt, actualSize, ctx}}); @@ -1271,7 +1298,7 @@ void Unifier::tryUnifyFunctions(TypeId subTy, TypeId superTy, bool isFunctionCal else if (!innerState.errors.empty()) reportError(TypeError{location, TypeMismatch{superTy, subTy, "", innerState.errors.front()}}); - innerState.ctx = CountMismatch::Result; + innerState.ctx = CountMismatch::FunctionResult; innerState.tryUnify_(subFunction->retTypes, superFunction->retTypes); if (!reported) @@ -1295,7 +1322,7 @@ void Unifier::tryUnifyFunctions(TypeId subTy, TypeId superTy, bool isFunctionCal ctx = CountMismatch::Arg; tryUnify_(superFunction->argTypes, subFunction->argTypes, isFunctionCall); - ctx = CountMismatch::Result; + ctx = CountMismatch::FunctionResult; tryUnify_(subFunction->retTypes, superFunction->retTypes); } @@ -1693,8 +1720,45 @@ void Unifier::tryUnifyWithMetatable(TypeId subTy, TypeId superTy, bool reversed) { case TableState::Free: { - tryUnify_(subTy, superMetatable->table); - log.bindTable(subTy, superTy); + if (FFlag::DebugLuauDeferredConstraintResolution) + { + Unifier innerState = makeChildUnifier(); + bool missingProperty = false; + + for (const auto& [propName, prop] : subTable->props) + { + if (std::optional mtPropTy = findTablePropertyRespectingMeta(superTy, propName)) + { + innerState.tryUnify(prop.type, *mtPropTy); + } + else + { + reportError(mismatchError); + missingProperty = true; + break; + } + } + + if (const TableTypeVar* superTable = log.get(log.follow(superMetatable->table))) + { + // TODO: Unify indexers. + } + + if (auto e = hasUnificationTooComplex(innerState.errors)) + reportError(*e); + else if (!innerState.errors.empty()) + reportError(TypeError{location, TypeMismatch{reversed ? subTy : superTy, reversed ? superTy : subTy, "", innerState.errors.front()}}); + else if (!missingProperty) + { + log.concat(std::move(innerState.log)); + log.bindTable(subTy, superTy); + } + } + else + { + tryUnify_(subTy, superMetatable->table); + log.bindTable(subTy, superTy); + } break; } diff --git a/CLI/Analyze.cpp b/CLI/Analyze.cpp index cd50ef00..7e4c5691 100644 --- a/CLI/Analyze.cpp +++ b/CLI/Analyze.cpp @@ -270,7 +270,7 @@ int main(int argc, char** argv) CliConfigResolver configResolver(mode); Luau::Frontend frontend(&fileResolver, &configResolver, frontendOptions); - Luau::registerBuiltinTypes(frontend.typeChecker); + Luau::registerBuiltinGlobals(frontend.typeChecker); Luau::freeze(frontend.typeChecker.globalTypes); #ifdef CALLGRIND diff --git a/CodeGen/src/Fallbacks.cpp b/CodeGen/src/Fallbacks.cpp index 3893d349..625a9bd6 100644 --- a/CodeGen/src/Fallbacks.cpp +++ b/CodeGen/src/Fallbacks.cpp @@ -720,7 +720,7 @@ const Instruction* execute_LOP_CALL(lua_State* L, const Instruction* pc, Closure int i; for (i = nresults; i != 0 && vali < valend; i--) - setobjs2s(L, res++, vali++); + setobj2s(L, res++, vali++); while (i-- > 0) setnilvalue(res++); @@ -756,7 +756,7 @@ const Instruction* execute_LOP_RETURN(lua_State* L, const Instruction* pc, Closu // note: in MULTRET context nresults starts as -1 so i != 0 condition never activates intentionally int i; for (i = nresults; i != 0 && vali < valend; i--) - setobjs2s(L, res++, vali++); + setobj2s(L, res++, vali++); while (i-- > 0) setnilvalue(res++); @@ -1667,7 +1667,7 @@ const Instruction* execute_LOP_CONCAT(lua_State* L, const Instruction* pc, Closu StkId ra = VM_REG(LUAU_INSN_A(insn)); - setobjs2s(L, ra, base + b); + setobj2s(L, ra, base + b); VM_PROTECT(luaC_checkGC(L)); return pc; } @@ -2003,9 +2003,9 @@ const Instruction* execute_LOP_FORGLOOP(lua_State* L, const Instruction* pc, Clo else { // note: it's safe to push arguments past top for complicated reasons (see top of the file) - setobjs2s(L, ra + 3 + 2, ra + 2); - setobjs2s(L, ra + 3 + 1, ra + 1); - setobjs2s(L, ra + 3, ra); + setobj2s(L, ra + 3 + 2, ra + 2); + setobj2s(L, ra + 3 + 1, ra + 1); + setobj2s(L, ra + 3, ra); L->top = ra + 3 + 3; // func + 2 args (state and index) LUAU_ASSERT(L->top <= L->stack_last); @@ -2017,7 +2017,7 @@ const Instruction* execute_LOP_FORGLOOP(lua_State* L, const Instruction* pc, Clo ra = VM_REG(LUAU_INSN_A(insn)); // copy first variable back into the iteration index - setobjs2s(L, ra + 2, ra + 3); + setobj2s(L, ra + 2, ra + 3); // note that we need to increment pc by 1 to exit the loop since we need to skip over aux pc += ttisnil(ra + 3) ? 1 : LUAU_INSN_D(insn); @@ -2094,7 +2094,7 @@ const Instruction* execute_LOP_GETVARARGS(lua_State* L, const Instruction* pc, C StkId ra = VM_REG(LUAU_INSN_A(insn)); // previous call may change the stack for (int j = 0; j < n; j++) - setobjs2s(L, ra + j, base - n + j); + setobj2s(L, ra + j, base - n + j); L->top = ra + n; return pc; @@ -2104,7 +2104,7 @@ const Instruction* execute_LOP_GETVARARGS(lua_State* L, const Instruction* pc, C StkId ra = VM_REG(LUAU_INSN_A(insn)); for (int j = 0; j < b && j < n; j++) - setobjs2s(L, ra + j, base - n + j); + setobj2s(L, ra + j, base - n + j); for (int j = n; j < b; j++) setnilvalue(ra + j); return pc; @@ -2183,7 +2183,7 @@ const Instruction* execute_LOP_PREPVARARGS(lua_State* L, const Instruction* pc, for (int i = 0; i < numparams; ++i) { - setobjs2s(L, base + i, fixed + i); + setobj2s(L, base + i, fixed + i); setnilvalue(fixed + i); } diff --git a/Common/include/Luau/Bytecode.h b/Common/include/Luau/Bytecode.h index f8652434..32e5ba9b 100644 --- a/Common/include/Luau/Bytecode.h +++ b/Common/include/Luau/Bytecode.h @@ -517,6 +517,10 @@ enum LuauBuiltinFunction // bit32.extract(_, k, k) LBF_BIT32_EXTRACTK, + + // get/setmetatable + LBF_GETMETATABLE, + LBF_SETMETATABLE, }; // Capture type, used in LOP_CAPTURE diff --git a/Compiler/src/Builtins.cpp b/Compiler/src/Builtins.cpp index 26933730..8d4640da 100644 --- a/Compiler/src/Builtins.cpp +++ b/Compiler/src/Builtins.cpp @@ -4,6 +4,8 @@ #include "Luau/Bytecode.h" #include "Luau/Compiler.h" +LUAU_FASTFLAGVARIABLE(LuauCompileBuiltinMT, false) + namespace Luau { namespace Compile @@ -64,6 +66,14 @@ static int getBuiltinFunctionId(const Builtin& builtin, const CompileOptions& op if (builtin.isGlobal("select")) return LBF_SELECT_VARARG; + if (FFlag::LuauCompileBuiltinMT) + { + if (builtin.isGlobal("getmetatable")) + return LBF_GETMETATABLE; + if (builtin.isGlobal("setmetatable")) + return LBF_SETMETATABLE; + } + if (builtin.object == "math") { if (builtin.method == "abs") diff --git a/VM/src/lapi.cpp b/VM/src/lapi.cpp index e5ce4d5a..28307eb9 100644 --- a/VM/src/lapi.cpp +++ b/VM/src/lapi.cpp @@ -235,7 +235,7 @@ void lua_remove(lua_State* L, int idx) StkId p = index2addr(L, idx); api_checkvalidindex(L, p); while (++p < L->top) - setobjs2s(L, p - 1, p); + setobj2s(L, p - 1, p); L->top--; return; } @@ -246,8 +246,8 @@ void lua_insert(lua_State* L, int idx) StkId p = index2addr(L, idx); api_checkvalidindex(L, p); for (StkId q = L->top; q > p; q--) - setobjs2s(L, q, q - 1); - setobjs2s(L, p, L->top); + setobj2s(L, q, q - 1); + setobj2s(L, p, L->top); return; } @@ -614,7 +614,7 @@ void lua_pushlstring(lua_State* L, const char* s, size_t len) { luaC_checkGC(L); luaC_threadbarrier(L); - setsvalue2s(L, L->top, luaS_newlstr(L, s, len)); + setsvalue(L, L->top, luaS_newlstr(L, s, len)); api_incr_top(L); return; } @@ -1269,7 +1269,7 @@ void lua_concat(lua_State* L, int n) else if (n == 0) { // push empty string luaC_threadbarrier(L); - setsvalue2s(L, L->top, luaS_newlstr(L, "", 0)); + setsvalue(L, L->top, luaS_newlstr(L, "", 0)); api_incr_top(L); } // else n == 1; nothing to do diff --git a/VM/src/laux.cpp b/VM/src/laux.cpp index c42e5ccc..c2d07ddd 100644 --- a/VM/src/laux.cpp +++ b/VM/src/laux.cpp @@ -400,7 +400,7 @@ char* luaL_extendbuffer(luaL_Buffer* B, size_t additionalsize, int boxloc) lua_insert(L, boxloc); } - setsvalue2s(L, L->top + boxloc, newStorage); + setsvalue(L, L->top + boxloc, newStorage); B->p = newStorage->data + (B->p - base); B->end = newStorage->data + nextsize; B->storage = newStorage; @@ -451,11 +451,11 @@ void luaL_pushresult(luaL_Buffer* B) // if we finished just at the end of the string buffer, we can convert it to a mutable stirng without a copy if (B->p == B->end) { - setsvalue2s(L, L->top - 1, luaS_buffinish(L, storage)); + setsvalue(L, L->top - 1, luaS_buffinish(L, storage)); } else { - setsvalue2s(L, L->top - 1, luaS_newlstr(L, storage->data, B->p - storage->data)); + setsvalue(L, L->top - 1, luaS_newlstr(L, storage->data, B->p - storage->data)); } } else diff --git a/VM/src/lbuiltins.cpp b/VM/src/lbuiltins.cpp index c16e5aa7..87b6ae0f 100644 --- a/VM/src/lbuiltins.cpp +++ b/VM/src/lbuiltins.cpp @@ -789,7 +789,7 @@ static int luauF_type(lua_State* L, StkId res, TValue* arg0, int nresults, StkId int tt = ttype(arg0); TString* ttname = L->global->ttname[tt]; - setsvalue2s(L, res, ttname); + setsvalue(L, res, ttname); return 1; } @@ -861,7 +861,7 @@ static int luauF_char(lua_State* L, StkId res, TValue* arg0, int nresults, StkId buffer[nparams] = 0; - setsvalue2s(L, res, luaS_newlstr(L, buffer, nparams)); + setsvalue(L, res, luaS_newlstr(L, buffer, nparams)); return 1; } @@ -887,7 +887,7 @@ static int luauF_typeof(lua_State* L, StkId res, TValue* arg0, int nresults, Stk { const TString* ttname = luaT_objtypenamestr(L, arg0); - setsvalue2s(L, res, ttname); + setsvalue(L, res, ttname); return 1; } @@ -904,7 +904,7 @@ static int luauF_sub(lua_State* L, StkId res, TValue* arg0, int nresults, StkId if (i >= 1 && j >= i && unsigned(j - 1) < unsigned(ts->len)) { - setsvalue2s(L, res, luaS_newlstr(L, getstr(ts) + (i - 1), j - i + 1)); + setsvalue(L, res, luaS_newlstr(L, getstr(ts) + (i - 1), j - i + 1)); return 1; } } @@ -993,12 +993,13 @@ static int luauF_rawset(lua_State* L, StkId res, TValue* arg0, int nresults, Stk else if (ttisvector(key) && luai_vecisnan(vvalue(key))) return -1; - if (hvalue(arg0)->readonly) + Table* t = hvalue(arg0); + if (t->readonly) return -1; setobj2s(L, res, arg0); - setobj2t(L, luaH_set(L, hvalue(arg0), args), args + 1); - luaC_barriert(L, hvalue(arg0), args + 1); + setobj2t(L, luaH_set(L, t, args), args + 1); + luaC_barriert(L, t, args + 1); return 1; } @@ -1009,12 +1010,13 @@ static int luauF_tinsert(lua_State* L, StkId res, TValue* arg0, int nresults, St { if (nparams == 2 && nresults <= 0 && ttistable(arg0)) { - if (hvalue(arg0)->readonly) + Table* t = hvalue(arg0); + if (t->readonly) return -1; - int pos = luaH_getn(hvalue(arg0)) + 1; - setobj2t(L, luaH_setnum(L, hvalue(arg0), pos), args); - luaC_barriert(L, hvalue(arg0), args); + int pos = luaH_getn(t) + 1; + setobj2t(L, luaH_setnum(L, t, pos), args); + luaC_barriert(L, t, args); return 0; } @@ -1193,6 +1195,60 @@ static int luauF_extractk(lua_State* L, StkId res, TValue* arg0, int nresults, S return -1; } +static int luauF_getmetatable(lua_State* L, StkId res, TValue* arg0, int nresults, StkId args, int nparams) +{ + if (nparams >= 1 && nresults <= 1) + { + Table* mt = NULL; + if (ttistable(arg0)) + mt = hvalue(arg0)->metatable; + else if (ttisuserdata(arg0)) + mt = uvalue(arg0)->metatable; + else + mt = L->global->mt[ttype(arg0)]; + + const TValue* mtv = mt ? luaH_getstr(mt, L->global->tmname[TM_METATABLE]) : luaO_nilobject; + if (!ttisnil(mtv)) + { + setobj2s(L, res, mtv); + return 1; + } + + if (mt) + { + sethvalue(L, res, mt); + return 1; + } + else + { + setnilvalue(res); + return 1; + } + } + + return -1; +} + +static int luauF_setmetatable(lua_State* L, StkId res, TValue* arg0, int nresults, StkId args, int nparams) +{ + // note: setmetatable(_, nil) is rare so we use fallback for it to optimize the fast path + if (nparams >= 2 && nresults <= 1 && ttistable(arg0) && ttistable(args)) + { + Table* t = hvalue(arg0); + if (t->readonly || t->metatable != NULL) + return -1; // note: overwriting non-null metatable is very rare but it requires __metatable check + + Table* mt = hvalue(args); + t->metatable = mt; + luaC_objbarrier(L, t, mt); + + sethvalue(L, res, t); + return 1; + } + + return -1; +} + luau_FastFunction luauF_table[256] = { NULL, luauF_assert, @@ -1268,4 +1324,7 @@ luau_FastFunction luauF_table[256] = { luauF_rawlen, luauF_extractk, + + luauF_getmetatable, + luauF_setmetatable, }; diff --git a/VM/src/ldebug.cpp b/VM/src/ldebug.cpp index 7c181d4e..e695cd2b 100644 --- a/VM/src/ldebug.cpp +++ b/VM/src/ldebug.cpp @@ -83,7 +83,7 @@ const char* lua_setlocal(lua_State* L, int level, int n) Proto* fp = getluaproto(ci); const LocVar* var = fp ? luaF_getlocal(fp, n, currentpc(L, ci)) : NULL; if (var) - setobjs2s(L, ci->base + var->reg, L->top - 1); + setobj2s(L, ci->base + var->reg, L->top - 1); L->top--; // pop value const char* name = var ? getstr(var->varname) : NULL; return name; diff --git a/VM/src/ldo.cpp b/VM/src/ldo.cpp index ecd2fcbb..ff8105b8 100644 --- a/VM/src/ldo.cpp +++ b/VM/src/ldo.cpp @@ -263,18 +263,18 @@ static void seterrorobj(lua_State* L, int errcode, StkId oldtop) { case LUA_ERRMEM: { - setsvalue2s(L, oldtop, luaS_newliteral(L, LUA_MEMERRMSG)); // can not fail because string is pinned in luaopen + setsvalue(L, oldtop, luaS_newliteral(L, LUA_MEMERRMSG)); // can not fail because string is pinned in luaopen break; } case LUA_ERRERR: { - setsvalue2s(L, oldtop, luaS_newliteral(L, LUA_ERRERRMSG)); // can not fail because string is pinned in luaopen + setsvalue(L, oldtop, luaS_newliteral(L, LUA_ERRERRMSG)); // can not fail because string is pinned in luaopen break; } case LUA_ERRSYNTAX: case LUA_ERRRUN: { - setobjs2s(L, oldtop, L->top - 1); // error message on current top + setobj2s(L, oldtop, L->top - 1); // error message on current top break; } } @@ -419,7 +419,7 @@ static void resume_handle(lua_State* L, void* ud) static int resume_error(lua_State* L, const char* msg) { L->top = L->ci->base; - setsvalue2s(L, L->top, luaS_new(L, msg)); + setsvalue(L, L->top, luaS_new(L, msg)); incr_top(L); return LUA_ERRRUN; } @@ -525,8 +525,8 @@ static void callerrfunc(lua_State* L, void* ud) { StkId errfunc = cast_to(StkId, ud); - setobjs2s(L, L->top, L->top - 1); - setobjs2s(L, L->top - 1, errfunc); + setobj2s(L, L->top, L->top - 1); + setobj2s(L, L->top - 1, errfunc); incr_top(L); luaD_call(L, L->top - 2, 1); } diff --git a/VM/src/lgc.cpp b/VM/src/lgc.cpp index c2a672e6..4b9fbb69 100644 --- a/VM/src/lgc.cpp +++ b/VM/src/lgc.cpp @@ -118,8 +118,6 @@ LUAU_FASTFLAGVARIABLE(LuauBetterThreadMark, false) * slot - upvalues like this are identified since they don't have `markedopen` bit set during thread traversal and closed in `clearupvals`. */ -LUAU_FASTFLAGVARIABLE(LuauFasterSweep, false) - #define GC_SWEEPPAGESTEPCOST 16 #define GC_INTERRUPT(state) \ @@ -836,28 +834,6 @@ static size_t atomic(lua_State* L) return work; } -static bool sweepgco(lua_State* L, lua_Page* page, GCObject* gco) -{ - LUAU_ASSERT(!FFlag::LuauFasterSweep); - global_State* g = L->global; - - int deadmask = otherwhite(g); - LUAU_ASSERT(testbit(deadmask, FIXEDBIT)); // make sure we never sweep fixed objects - - int alive = (gco->gch.marked ^ WHITEBITS) & deadmask; - - if (alive) - { - LUAU_ASSERT(!isdead(g, gco)); - makewhite(g, gco); // make it white (for next cycle) - return false; - } - - LUAU_ASSERT(isdead(g, gco)); - freeobj(L, gco, page); - return true; -} - // a version of generic luaM_visitpage specialized for the main sweep stage static int sweepgcopage(lua_State* L, lua_Page* page) { @@ -869,58 +845,36 @@ static int sweepgcopage(lua_State* L, lua_Page* page) LUAU_ASSERT(busyBlocks > 0); - if (FFlag::LuauFasterSweep) + global_State* g = L->global; + + int deadmask = otherwhite(g); + LUAU_ASSERT(testbit(deadmask, FIXEDBIT)); // make sure we never sweep fixed objects + + int newwhite = luaC_white(g); + + for (char* pos = start; pos != end; pos += blockSize) { - global_State* g = L->global; + GCObject* gco = (GCObject*)pos; - int deadmask = otherwhite(g); - LUAU_ASSERT(testbit(deadmask, FIXEDBIT)); // make sure we never sweep fixed objects + // skip memory blocks that are already freed + if (gco->gch.tt == LUA_TNIL) + continue; - int newwhite = luaC_white(g); - - for (char* pos = start; pos != end; pos += blockSize) + // is the object alive? + if ((gco->gch.marked ^ WHITEBITS) & deadmask) { - GCObject* gco = (GCObject*)pos; - - // skip memory blocks that are already freed - if (gco->gch.tt == LUA_TNIL) - continue; - - // is the object alive? - if ((gco->gch.marked ^ WHITEBITS) & deadmask) - { - LUAU_ASSERT(!isdead(g, gco)); - // make it white (for next cycle) - gco->gch.marked = cast_byte((gco->gch.marked & maskmarks) | newwhite); - } - else - { - LUAU_ASSERT(isdead(g, gco)); - freeobj(L, gco, page); - - // if the last block was removed, page would be removed as well - if (--busyBlocks == 0) - return int(pos - start) / blockSize + 1; - } + LUAU_ASSERT(!isdead(g, gco)); + // make it white (for next cycle) + gco->gch.marked = cast_byte((gco->gch.marked & maskmarks) | newwhite); } - } - else - { - for (char* pos = start; pos != end; pos += blockSize) + else { - GCObject* gco = (GCObject*)pos; + LUAU_ASSERT(isdead(g, gco)); + freeobj(L, gco, page); - // skip memory blocks that are already freed - if (gco->gch.tt == LUA_TNIL) - continue; - - // when true is returned it means that the element was deleted - if (sweepgco(L, page, gco)) - { - // if the last block was removed, page would be removed as well - if (--busyBlocks == 0) - return int(pos - start) / blockSize + 1; - } + // if the last block was removed, page would be removed as well + if (--busyBlocks == 0) + return int(pos - start) / blockSize + 1; } } @@ -1009,15 +963,8 @@ static size_t gcstep(lua_State* L, size_t limit) if (g->sweepgcopage == NULL) { // don't forget to visit main thread, it's the only object not allocated in GCO pages - if (FFlag::LuauFasterSweep) - { - LUAU_ASSERT(!isdead(g, obj2gco(g->mainthread))); - makewhite(g, obj2gco(g->mainthread)); // make it white (for next cycle) - } - else - { - sweepgco(L, NULL, obj2gco(g->mainthread)); - } + LUAU_ASSERT(!isdead(g, obj2gco(g->mainthread))); + makewhite(g, obj2gco(g->mainthread)); // make it white (for next cycle) shrinkbuffers(L); diff --git a/VM/src/lobject.cpp b/VM/src/lobject.cpp index b6a40bb6..f5f1cd0e 100644 --- a/VM/src/lobject.cpp +++ b/VM/src/lobject.cpp @@ -102,7 +102,7 @@ const char* luaO_pushvfstring(lua_State* L, const char* fmt, va_list argp) char result[LUA_BUFFERSIZE]; vsnprintf(result, sizeof(result), fmt, argp); - setsvalue2s(L, L->top, luaS_new(L, result)); + setsvalue(L, L->top, luaS_new(L, result)); incr_top(L); return svalue(L->top - 1); } diff --git a/VM/src/lobject.h b/VM/src/lobject.h index 97cbfbb1..5f5e7b1c 100644 --- a/VM/src/lobject.h +++ b/VM/src/lobject.h @@ -200,20 +200,14 @@ typedef struct lua_TValue ** different types of sets, according to destination */ -// from stack to (same) stack -#define setobjs2s setobj -// to stack (not from same stack) +// to stack #define setobj2s setobj -#define setsvalue2s setsvalue -#define sethvalue2s sethvalue -#define setptvalue2s setptvalue -// from table to same table +// from table to same table (no barrier) #define setobjt2t setobj -// to table +// to table (needs barrier) #define setobj2t setobj -// to new object +// to new object (no barrier) #define setobj2n setobj -#define setsvalue2n setsvalue #define setttype(obj, tt) (ttype(obj) = (tt)) diff --git a/VM/src/ltm.cpp b/VM/src/ltm.cpp index cb7ba097..d753e8a4 100644 --- a/VM/src/ltm.cpp +++ b/VM/src/ltm.cpp @@ -57,6 +57,7 @@ const char* const luaT_eventname[] = { "__le", "__concat", "__type", + "__metatable", }; // clang-format on diff --git a/VM/src/ltm.h b/VM/src/ltm.h index f20ce1b2..4b1c2818 100644 --- a/VM/src/ltm.h +++ b/VM/src/ltm.h @@ -36,6 +36,7 @@ typedef enum TM_LE, TM_CONCAT, TM_TYPE, + TM_METATABLE, TM_N // number of elements in the enum } TMS; diff --git a/VM/src/lvmexecute.cpp b/VM/src/lvmexecute.cpp index 6ceed512..490358c4 100644 --- a/VM/src/lvmexecute.cpp +++ b/VM/src/lvmexecute.cpp @@ -985,7 +985,7 @@ reentry: int i; for (i = nresults; i != 0 && vali < valend; i--) - setobjs2s(L, res++, vali++); + setobj2s(L, res++, vali++); while (i-- > 0) setnilvalue(res++); @@ -1022,7 +1022,7 @@ reentry: // note: in MULTRET context nresults starts as -1 so i != 0 condition never activates intentionally int i; for (i = nresults; i != 0 && vali < valend; i--) - setobjs2s(L, res++, vali++); + setobj2s(L, res++, vali++); while (i-- > 0) setnilvalue(res++); @@ -1945,7 +1945,7 @@ reentry: StkId ra = VM_REG(LUAU_INSN_A(insn)); - setobjs2s(L, ra, base + b); + setobj2s(L, ra, base + b); VM_PROTECT(luaC_checkGC(L)); VM_NEXT(); } @@ -2281,9 +2281,9 @@ reentry: else { // note: it's safe to push arguments past top for complicated reasons (see top of the file) - setobjs2s(L, ra + 3 + 2, ra + 2); - setobjs2s(L, ra + 3 + 1, ra + 1); - setobjs2s(L, ra + 3, ra); + setobj2s(L, ra + 3 + 2, ra + 2); + setobj2s(L, ra + 3 + 1, ra + 1); + setobj2s(L, ra + 3, ra); L->top = ra + 3 + 3; // func + 2 args (state and index) LUAU_ASSERT(L->top <= L->stack_last); @@ -2295,7 +2295,7 @@ reentry: ra = VM_REG(LUAU_INSN_A(insn)); // copy first variable back into the iteration index - setobjs2s(L, ra + 2, ra + 3); + setobj2s(L, ra + 2, ra + 3); // note that we need to increment pc by 1 to exit the loop since we need to skip over aux pc += ttisnil(ra + 3) ? 1 : LUAU_INSN_D(insn); @@ -2372,7 +2372,7 @@ reentry: StkId ra = VM_REG(LUAU_INSN_A(insn)); // previous call may change the stack for (int j = 0; j < n; j++) - setobjs2s(L, ra + j, base - n + j); + setobj2s(L, ra + j, base - n + j); L->top = ra + n; VM_NEXT(); @@ -2382,7 +2382,7 @@ reentry: StkId ra = VM_REG(LUAU_INSN_A(insn)); for (int j = 0; j < b && j < n; j++) - setobjs2s(L, ra + j, base - n + j); + setobj2s(L, ra + j, base - n + j); for (int j = n; j < b; j++) setnilvalue(ra + j); VM_NEXT(); @@ -2461,7 +2461,7 @@ reentry: for (int i = 0; i < numparams; ++i) { - setobjs2s(L, base + i, fixed + i); + setobj2s(L, base + i, fixed + i); setnilvalue(fixed + i); } @@ -2878,7 +2878,7 @@ int luau_precall(lua_State* L, StkId func, int nresults) int i; for (i = nresults; i != 0 && vali < valend; i--) - setobjs2s(L, res++, vali++); + setobj2s(L, res++, vali++); while (i-- > 0) setnilvalue(res++); @@ -2906,7 +2906,7 @@ void luau_poscall(lua_State* L, StkId first) int i; for (i = ci->nresults; i != 0 && vali < valend; i--) - setobjs2s(L, res++, vali++); + setobj2s(L, res++, vali++); while (i-- > 0) setnilvalue(res++); diff --git a/VM/src/lvmload.cpp b/VM/src/lvmload.cpp index 0ae85ab6..bd40bad2 100644 --- a/VM/src/lvmload.cpp +++ b/VM/src/lvmload.cpp @@ -240,7 +240,7 @@ int luau_load(lua_State* L, const char* chunkname, const char* data, size_t size case LBC_CONSTANT_STRING: { TString* v = readString(strings, data, size, offset); - setsvalue2n(L, &p->k[j], v); + setsvalue(L, &p->k[j], v); break; } diff --git a/VM/src/lvmutils.cpp b/VM/src/lvmutils.cpp index 35124e63..5c555158 100644 --- a/VM/src/lvmutils.cpp +++ b/VM/src/lvmutils.cpp @@ -38,7 +38,7 @@ int luaV_tostring(lua_State* L, StkId obj) double n = nvalue(obj); char* e = luai_num2str(s, n); LUAU_ASSERT(e < s + sizeof(s)); - setsvalue2s(L, obj, luaS_newlstr(L, s, e - s)); + setsvalue(L, obj, luaS_newlstr(L, s, e - s)); return 1; } } @@ -70,7 +70,7 @@ static StkId callTMres(lua_State* L, StkId res, const TValue* f, const TValue* p luaD_call(L, L->top - 3, 1); res = restorestack(L, result); L->top--; - setobjs2s(L, res, L->top); + setobj2s(L, res, L->top); return res; } @@ -350,11 +350,11 @@ void luaV_concat(lua_State* L, int total, int last) if (tl < LUA_BUFFERSIZE) { - setsvalue2s(L, top - n, luaS_newlstr(L, buffer, tl)); + setsvalue(L, top - n, luaS_newlstr(L, buffer, tl)); } else { - setsvalue2s(L, top - n, luaS_buffinish(L, ts)); + setsvalue(L, top - n, luaS_buffinish(L, ts)); } } total -= n - 1; // got `n' strings to create 1 new @@ -582,7 +582,7 @@ LUAU_NOINLINE void luaV_tryfuncTM(lua_State* L, StkId func) if (!ttisfunction(tm)) luaG_typeerror(L, func, "call"); for (StkId p = L->top; p > func; p--) // open space for metamethod - setobjs2s(L, p, p - 1); + setobj2s(L, p, p - 1); L->top++; // stack space pre-allocated by the caller setobj2s(L, func, tm); // tag method is the new function to be called } diff --git a/fuzz/linter.cpp b/fuzz/linter.cpp index 0bdd49f5..66ca5bb1 100644 --- a/fuzz/linter.cpp +++ b/fuzz/linter.cpp @@ -21,7 +21,7 @@ extern "C" int LLVMFuzzerTestOneInput(const uint8_t* Data, size_t Size) static Luau::NullModuleResolver moduleResolver; static Luau::InternalErrorReporter iceHandler; static Luau::TypeChecker sharedEnv(&moduleResolver, &iceHandler); - static int once = (Luau::registerBuiltinTypes(sharedEnv), 1); + static int once = (Luau::registerBuiltinGlobals(sharedEnv), 1); (void)once; static int once2 = (Luau::freeze(sharedEnv.globalTypes), 1); (void)once2; diff --git a/fuzz/proto.cpp b/fuzz/proto.cpp index f64b6157..9e0a6829 100644 --- a/fuzz/proto.cpp +++ b/fuzz/proto.cpp @@ -96,7 +96,7 @@ int registerTypes(Luau::TypeChecker& env) using namespace Luau; using std::nullopt; - Luau::registerBuiltinTypes(env); + Luau::registerBuiltinGlobals(env); TypeArena& arena = env.globalTypes; diff --git a/fuzz/typeck.cpp b/fuzz/typeck.cpp index 3905cc19..a6c9ae28 100644 --- a/fuzz/typeck.cpp +++ b/fuzz/typeck.cpp @@ -26,7 +26,7 @@ extern "C" int LLVMFuzzerTestOneInput(const uint8_t* Data, size_t Size) static Luau::NullModuleResolver moduleResolver; static Luau::InternalErrorReporter iceHandler; static Luau::TypeChecker sharedEnv(&moduleResolver, &iceHandler); - static int once = (Luau::registerBuiltinTypes(sharedEnv), 1); + static int once = (Luau::registerBuiltinGlobals(sharedEnv), 1); (void)once; static int once2 = (Luau::freeze(sharedEnv.globalTypes), 1); (void)once2; diff --git a/tests/Compiler.test.cpp b/tests/Compiler.test.cpp index 9409c822..d0f77785 100644 --- a/tests/Compiler.test.cpp +++ b/tests/Compiler.test.cpp @@ -798,6 +798,8 @@ RETURN R0 1 TEST_CASE("TableSizePredictionSetMetatable") { + ScopedFastFlag sff("LuauCompileBuiltinMT", true); + CHECK_EQ("\n" + compileFunction0(R"( local t = setmetatable({}, nil) t.field1 = 1 @@ -805,14 +807,15 @@ t.field2 = 2 return t )"), R"( -GETIMPORT R0 1 NEWTABLE R1 2 0 -LOADNIL R2 +FASTCALL2K 61 R1 K0 L0 +LOADK R2 K0 +GETIMPORT R0 2 CALL R0 2 1 -LOADN R1 1 -SETTABLEKS R1 R0 K2 -LOADN R1 2 +L0: LOADN R1 1 SETTABLEKS R1 R0 K3 +LOADN R1 2 +SETTABLEKS R1 R0 K4 RETURN R0 1 )"); } diff --git a/tests/Conformance.test.cpp b/tests/Conformance.test.cpp index 77b30487..a6e4e6e1 100644 --- a/tests/Conformance.test.cpp +++ b/tests/Conformance.test.cpp @@ -499,7 +499,7 @@ TEST_CASE("Types") Luau::SingletonTypes singletonTypes; Luau::TypeChecker env(&moduleResolver, Luau::NotNull{&singletonTypes}, &iceHandler); - Luau::registerBuiltinTypes(env); + Luau::registerBuiltinGlobals(env); Luau::freeze(env.globalTypes); lua_newtable(L); diff --git a/tests/Fixture.cpp b/tests/Fixture.cpp index 3f77978c..dcc0222a 100644 --- a/tests/Fixture.cpp +++ b/tests/Fixture.cpp @@ -8,7 +8,6 @@ #include "Luau/TypeVar.h" #include "Luau/TypeAttach.h" #include "Luau/Transpiler.h" - #include "Luau/BuiltinDefinitions.h" #include "doctest.h" @@ -20,6 +19,8 @@ static const char* mainModuleName = "MainModule"; LUAU_FASTFLAG(DebugLuauDeferredConstraintResolution); +LUAU_FASTFLAG(LuauUnknownAndNeverType) +LUAU_FASTFLAG(LuauReportShadowedTypeAlias) namespace Luau { @@ -97,6 +98,8 @@ Fixture::Fixture(bool freeze, bool prepareAutocomplete) configResolver.defaultConfig.enabledLint.warningMask = ~0ull; configResolver.defaultConfig.parseOptions.captureComments = true; + registerBuiltinTypes(frontend); + Luau::freeze(frontend.typeChecker.globalTypes); Luau::freeze(frontend.typeCheckerForAutocomplete.globalTypes); @@ -435,9 +438,9 @@ BuiltinsFixture::BuiltinsFixture(bool freeze, bool prepareAutocomplete) Luau::unfreeze(frontend.typeChecker.globalTypes); Luau::unfreeze(frontend.typeCheckerForAutocomplete.globalTypes); - registerBuiltinTypes(frontend); + registerBuiltinGlobals(frontend); if (prepareAutocomplete) - registerBuiltinTypes(frontend.typeCheckerForAutocomplete); + registerBuiltinGlobals(frontend.typeCheckerForAutocomplete); registerTestTypes(); Luau::freeze(frontend.typeChecker.globalTypes); diff --git a/tests/Frontend.test.cpp b/tests/Frontend.test.cpp index a8a9e044..5f74931c 100644 --- a/tests/Frontend.test.cpp +++ b/tests/Frontend.test.cpp @@ -1070,12 +1070,12 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "reexport_type_alias") fileResolver.source["Module/A"] = R"( type KeyOfTestEvents = "test-file-start" | "test-file-success" | "test-file-failure" | "test-case-result" - type unknown = any + type MyAny = any export type TestFileEvent = ( eventName: T, args: any --[[ ROBLOX TODO: Unhandled node for type: TSIndexedAccessType ]] --[[ TestEvents[T] ]] - ) -> unknown + ) -> MyAny return {} )"; diff --git a/tests/ToString.test.cpp b/tests/ToString.test.cpp index ab5d8598..1339ec28 100644 --- a/tests/ToString.test.cpp +++ b/tests/ToString.test.cpp @@ -790,4 +790,20 @@ TEST_CASE_FIXTURE(Fixture, "toStringNamedFunction_hide_self_param") CHECK_EQ("foo:method(arg: string): ()", toStringNamedFunction("foo:method", *ftv, opts)); } +TEST_CASE_FIXTURE(Fixture, "tostring_unsee_ttv_if_array") +{ + ScopedFastFlag sff("LuauUnseeArrayTtv", true); + + CheckResult result = check(R"( + local x: {string} + -- This code is constructed very specifically to use the same (by pointer + -- identity) type in the function twice. + local y: (typeof(x), typeof(x)) -> () + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK(toString(requireType("y")) == "({string}, {string}) -> ()"); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.aliases.test.cpp b/tests/TypeInfer.aliases.test.cpp index 834391a7..5ecc2a8c 100644 --- a/tests/TypeInfer.aliases.test.cpp +++ b/tests/TypeInfer.aliases.test.cpp @@ -443,7 +443,8 @@ TEST_CASE_FIXTURE(Fixture, "reported_location_is_correct_when_type_alias_are_dup auto dtd = get(result.errors[0]); REQUIRE(dtd); CHECK_EQ(dtd->name, "B"); - CHECK_EQ(dtd->previousLocation.begin.line + 1, 3); + REQUIRE(dtd->previousLocation); + CHECK_EQ(dtd->previousLocation->begin.line + 1, 3); } TEST_CASE_FIXTURE(Fixture, "stringify_optional_parameterized_alias") @@ -868,4 +869,40 @@ TEST_CASE_FIXTURE(Fixture, "recursive_types_restriction_not_ok") LUAU_REQUIRE_ERRORS(result); } +TEST_CASE_FIXTURE(Fixture, "report_shadowed_aliases") +{ + ScopedFastFlag sff{"LuauReportShadowedTypeAlias", true}; + + // We allow a previous type alias to depend on a future type alias. That exact feature enables a confusing example, like the following snippet, + // which has the type alias FakeString point to the type alias `string` that which points to `number`. + CheckResult result = check(R"( + type MyString = string + type string = number + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK(toString(result.errors[0]) == "Redefinition of type 'string'"); + + std::optional t1 = lookupType("MyString"); + REQUIRE(t1); + CHECK(isPrim(*t1, PrimitiveTypeVar::String)); + + std::optional t2 = lookupType("string"); + REQUIRE(t2); + CHECK(isPrim(*t2, PrimitiveTypeVar::String)); +} + +TEST_CASE_FIXTURE(Fixture, "it_is_ok_to_shadow_user_defined_alias") +{ + CheckResult result = check(R"( + type T = number + + do + type T = string + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.annotations.test.cpp b/tests/TypeInfer.annotations.test.cpp index 5d18b335..5f2c22cf 100644 --- a/tests/TypeInfer.annotations.test.cpp +++ b/tests/TypeInfer.annotations.test.cpp @@ -745,7 +745,12 @@ TEST_CASE_FIXTURE(Fixture, "luau_ice_is_not_special_without_the_flag") TEST_CASE_FIXTURE(BuiltinsFixture, "luau_print_is_magic_if_the_flag_is_set") { - // Luau::resetPrintLine(); + static std::vector output; + output.clear(); + Luau::setPrintLine([](const std::string& s) { + output.push_back(s); + }); + ScopedFastFlag sffs{"DebugLuauMagicTypes", true}; CheckResult result = check(R"( @@ -753,6 +758,8 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "luau_print_is_magic_if_the_flag_is_set") )"); LUAU_REQUIRE_NO_ERRORS(result); + + REQUIRE(1 == output.size()); } TEST_CASE_FIXTURE(Fixture, "luau_print_is_not_special_without_the_flag") diff --git a/tests/TypeInfer.builtins.test.cpp b/tests/TypeInfer.builtins.test.cpp index 6da3f569..037f79d8 100644 --- a/tests/TypeInfer.builtins.test.cpp +++ b/tests/TypeInfer.builtins.test.cpp @@ -11,6 +11,7 @@ using namespace Luau; LUAU_FASTFLAG(LuauLowerBoundsCalculation); LUAU_FASTFLAG(LuauSpecialTypesAsterisked); LUAU_FASTFLAG(LuauStringFormatArgumentErrorFix) +LUAU_FASTFLAG(DebugLuauDeferredConstraintResolution) TEST_SUITE_BEGIN("BuiltinTests"); @@ -596,6 +597,15 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "xpcall") CHECK_EQ("boolean", toString(requireType("c"))); } +TEST_CASE_FIXTURE(BuiltinsFixture, "trivial_select") +{ + CheckResult result = check(R"( + local a:number = select(1, 42) + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + TEST_CASE_FIXTURE(BuiltinsFixture, "see_thru_select") { CheckResult result = check(R"( @@ -679,10 +689,20 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "select_with_variadic_typepack_tail") LUAU_REQUIRE_NO_ERRORS(result); - CHECK_EQ("any", toString(requireType("foo"))); - CHECK_EQ("any", toString(requireType("bar"))); - CHECK_EQ("any", toString(requireType("baz"))); - CHECK_EQ("any", toString(requireType("quux"))); + if (FFlag::DebugLuauDeferredConstraintResolution && FFlag::LuauSpecialTypesAsterisked) + { + CHECK_EQ("string", toString(requireType("foo"))); + CHECK_EQ("*error-type*", toString(requireType("bar"))); + CHECK_EQ("*error-type*", toString(requireType("baz"))); + CHECK_EQ("*error-type*", toString(requireType("quux"))); + } + else + { + CHECK_EQ("any", toString(requireType("foo"))); + CHECK_EQ("any", toString(requireType("bar"))); + CHECK_EQ("any", toString(requireType("baz"))); + CHECK_EQ("any", toString(requireType("quux"))); + } } TEST_CASE_FIXTURE(BuiltinsFixture, "select_with_variadic_typepack_tail_and_string_head") @@ -698,10 +718,20 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "select_with_variadic_typepack_tail_and_strin LUAU_REQUIRE_NO_ERRORS(result); - CHECK_EQ("any", toString(requireType("foo"))); - CHECK_EQ("any", toString(requireType("bar"))); - CHECK_EQ("any", toString(requireType("baz"))); - CHECK_EQ("any", toString(requireType("quux"))); + if (FFlag::DebugLuauDeferredConstraintResolution && FFlag::LuauSpecialTypesAsterisked) + { + CHECK_EQ("string", toString(requireType("foo"))); + CHECK_EQ("string", toString(requireType("bar"))); + CHECK_EQ("*error-type*", toString(requireType("baz"))); + CHECK_EQ("*error-type*", toString(requireType("quux"))); + } + else + { + CHECK_EQ("any", toString(requireType("foo"))); + CHECK_EQ("any", toString(requireType("bar"))); + CHECK_EQ("any", toString(requireType("baz"))); + CHECK_EQ("any", toString(requireType("quux"))); + } } TEST_CASE_FIXTURE(Fixture, "string_format_as_method") @@ -1099,7 +1129,7 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "gmatch_capture_types_default_capture") CountMismatch* acm = get(result.errors[0]); REQUIRE(acm); - CHECK_EQ(acm->context, CountMismatch::Result); + CHECK_EQ(acm->context, CountMismatch::FunctionResult); CHECK_EQ(acm->expected, 1); CHECK_EQ(acm->actual, 4); @@ -1116,7 +1146,7 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "gmatch_capture_types_balanced_escaped_parens CountMismatch* acm = get(result.errors[0]); REQUIRE(acm); - CHECK_EQ(acm->context, CountMismatch::Result); + CHECK_EQ(acm->context, CountMismatch::FunctionResult); CHECK_EQ(acm->expected, 3); CHECK_EQ(acm->actual, 4); @@ -1135,7 +1165,7 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "gmatch_capture_types_parens_in_sets_are_igno CountMismatch* acm = get(result.errors[0]); REQUIRE(acm); - CHECK_EQ(acm->context, CountMismatch::Result); + CHECK_EQ(acm->context, CountMismatch::FunctionResult); CHECK_EQ(acm->expected, 2); CHECK_EQ(acm->actual, 3); @@ -1288,7 +1318,7 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "find_capture_types3") CountMismatch* acm = get(result.errors[0]); REQUIRE(acm); - CHECK_EQ(acm->context, CountMismatch::Result); + CHECK_EQ(acm->context, CountMismatch::FunctionResult); CHECK_EQ(acm->expected, 2); CHECK_EQ(acm->actual, 4); diff --git a/tests/TypeInfer.functions.test.cpp b/tests/TypeInfer.functions.test.cpp index bde28dcc..a4420b9a 100644 --- a/tests/TypeInfer.functions.test.cpp +++ b/tests/TypeInfer.functions.test.cpp @@ -837,6 +837,8 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "calling_function_with_anytypepack_doesnt_lea TEST_CASE_FIXTURE(Fixture, "too_many_return_values") { + ScopedFastFlag sff{"LuauBetterMessagingOnCountMismatch", true}; + CheckResult result = check(R"( --!strict @@ -851,7 +853,49 @@ TEST_CASE_FIXTURE(Fixture, "too_many_return_values") CountMismatch* acm = get(result.errors[0]); REQUIRE(acm); - CHECK_EQ(acm->context, CountMismatch::Result); + CHECK_EQ(acm->context, CountMismatch::FunctionResult); + CHECK_EQ(acm->expected, 1); + CHECK_EQ(acm->actual, 2); +} + +TEST_CASE_FIXTURE(Fixture, "too_many_return_values_in_parentheses") +{ + ScopedFastFlag sff{"LuauBetterMessagingOnCountMismatch", true}; + + CheckResult result = check(R"( + --!strict + + function f() + return 55 + end + + local a, b = (f()) + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + CountMismatch* acm = get(result.errors[0]); + REQUIRE(acm); + CHECK_EQ(acm->context, CountMismatch::FunctionResult); + CHECK_EQ(acm->expected, 1); + CHECK_EQ(acm->actual, 2); +} + +TEST_CASE_FIXTURE(Fixture, "too_many_return_values_no_function") +{ + ScopedFastFlag sff{"LuauBetterMessagingOnCountMismatch", true}; + + CheckResult result = check(R"( + --!strict + + local a, b = 55 + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + CountMismatch* acm = get(result.errors[0]); + REQUIRE(acm); + CHECK_EQ(acm->context, CountMismatch::ExprListResult); CHECK_EQ(acm->expected, 1); CHECK_EQ(acm->actual, 2); } @@ -1271,7 +1315,7 @@ local b: B = a LUAU_REQUIRE_ERROR_COUNT(1, result); CHECK_EQ(toString(result.errors[0]), R"(Type '(number, number) -> number' could not be converted into '(number, number) -> (number, boolean)' caused by: - Function only returns 1 value. 2 are required here)"); + Function only returns 1 value, but 2 are required here)"); } TEST_CASE_FIXTURE(Fixture, "error_detailed_function_mismatch_ret") diff --git a/tests/TypeInfer.loops.test.cpp b/tests/TypeInfer.loops.test.cpp index 85249ecd..588a9a76 100644 --- a/tests/TypeInfer.loops.test.cpp +++ b/tests/TypeInfer.loops.test.cpp @@ -526,6 +526,16 @@ TEST_CASE_FIXTURE(Fixture, "fuzz_fail_missing_instantitation_follow") )"); } +TEST_CASE_FIXTURE(BuiltinsFixture, "for_in_with_generic_next") +{ + CheckResult result = check(R"( + for k: number, v: number in next, {1, 2, 3} do + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + TEST_CASE_FIXTURE(Fixture, "loop_iter_basic") { CheckResult result = check(R"( @@ -584,11 +594,48 @@ TEST_CASE_FIXTURE(Fixture, "loop_iter_no_indexer_nonstrict") LUAU_REQUIRE_ERROR_COUNT(0, result); } -TEST_CASE_FIXTURE(BuiltinsFixture, "loop_iter_iter_metamethod") +TEST_CASE_FIXTURE(BuiltinsFixture, "loop_iter_metamethod_nil") { + if (!FFlag::DebugLuauDeferredConstraintResolution) + return; + CheckResult result = check(R"( - local t = {} - setmetatable(t, { __iter = function(o) return next, o.children end }) + local t = setmetatable({}, { __iter = function(o) return next, nil end, }) + for k: number, v: string in t do + end + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK(toString(result.errors[0]) == "Type 'nil' could not be converted into '{- [a]: b -}'"); +} + +TEST_CASE_FIXTURE(BuiltinsFixture, "loop_iter_metamethod_not_enough_returns") +{ + if (!FFlag::DebugLuauDeferredConstraintResolution) + return; + + CheckResult result = check(R"( + local t = setmetatable({}, { __iter = function(o) end }) + for k: number, v: string in t do + end + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK(result.errors[0] == TypeError{ + Location{{2, 36}, {2, 37}}, + GenericError{"__iter must return at least one value"}, + }); +} + +TEST_CASE_FIXTURE(BuiltinsFixture, "loop_iter_metamethod_ok") +{ + if (!FFlag::DebugLuauDeferredConstraintResolution) + return; + + CheckResult result = check(R"( + local t = setmetatable({ + children = {"foo"} + }, { __iter = function(o) return next, o.children end }) for k: number, v: string in t do end )"); @@ -596,4 +643,26 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "loop_iter_iter_metamethod") LUAU_REQUIRE_ERROR_COUNT(0, result); } +TEST_CASE_FIXTURE(BuiltinsFixture, "loop_iter_metamethod_ok_with_inference") +{ + if (!FFlag::DebugLuauDeferredConstraintResolution) + return; + + CheckResult result = check(R"( + local t = setmetatable({ + children = {"foo"} + }, { __iter = function(o) return next, o.children end }) + + local a, b + for k, v in t do + a = k + b = v + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + CHECK(toString(requireType("a")) == "number"); + CHECK(toString(requireType("b")) == "string"); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.provisional.test.cpp b/tests/TypeInfer.provisional.test.cpp index 2c4c3504..45740a0b 100644 --- a/tests/TypeInfer.provisional.test.cpp +++ b/tests/TypeInfer.provisional.test.cpp @@ -439,13 +439,15 @@ TEST_CASE_FIXTURE(Fixture, "normalization_fails_on_certain_kinds_of_cyclic_table // Belongs in TypeInfer.builtins.test.cpp. TEST_CASE_FIXTURE(BuiltinsFixture, "pcall_returns_at_least_two_value_but_function_returns_nothing") { + ScopedFastFlag sff{"LuauBetterMessagingOnCountMismatch", true}; + CheckResult result = check(R"( local function f(): () end local ok, res = pcall(f) )"); LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK_EQ("Function only returns 1 value. 2 are required here", toString(result.errors[0])); + CHECK_EQ("Function only returns 1 value, but 2 are required here", toString(result.errors[0])); // LUAU_REQUIRE_NO_ERRORS(result); // CHECK_EQ("boolean", toString(requireType("ok"))); // CHECK_EQ("any", toString(requireType("res"))); diff --git a/tests/TypeInfer.refinements.test.cpp b/tests/TypeInfer.refinements.test.cpp index d2bff9c3..b6dedcbd 100644 --- a/tests/TypeInfer.refinements.test.cpp +++ b/tests/TypeInfer.refinements.test.cpp @@ -256,28 +256,6 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "typeguard_in_assert_position") REQUIRE_EQ("number", toString(requireType("b"))); } -TEST_CASE_FIXTURE(Fixture, "typeguard_only_look_up_types_from_global_scope") -{ - CheckResult result = check(R"( - type ActuallyString = string - - do -- Necessary. Otherwise toposort has ActuallyString come after string type alias. - type string = number - local foo: string = 1 - - if type(foo) == "string" then - local bar: ActuallyString = foo - local baz: boolean = foo - end - end - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - - CHECK_EQ("never", toString(requireTypeAtPosition({8, 44}))); - CHECK_EQ("never", toString(requireTypeAtPosition({9, 38}))); -} - TEST_CASE_FIXTURE(Fixture, "call_a_more_specific_function_using_typeguard") { CheckResult result = check(R"( diff --git a/tests/TypeInfer.test.cpp b/tests/TypeInfer.test.cpp index a96c3676..8ed61b49 100644 --- a/tests/TypeInfer.test.cpp +++ b/tests/TypeInfer.test.cpp @@ -1159,4 +1159,38 @@ TEST_CASE_FIXTURE(Fixture, "dcr_can_partially_dispatch_a_constraint") CHECK("(a, number) -> ()" == toString(requireType("prime_iter"))); } +TEST_CASE_FIXTURE(BuiltinsFixture, "it_is_ok_to_have_inconsistent_number_of_return_values_in_nonstrict") +{ + CheckResult result = check(R"( + --!nonstrict + function validate(stats, hits, misses) + local checked = {} + + for _,l in ipairs(hits) do + if not (stats[l] and stats[l] > 0) then + return false, string.format("expected line %d to be hit", l) + end + checked[l] = true + end + + for _,l in ipairs(misses) do + if not (stats[l] and stats[l] == 0) then + return false, string.format("expected line %d to be missed", l) + end + checked[l] = true + end + + for k,v in pairs(stats) do + if type(k) == "number" and not checked[k] then + return false, string.format("expected line %d to be absent", k) + end + end + + return true + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.tryUnify.test.cpp b/tests/TypeInfer.tryUnify.test.cpp index 7fa0fac0..3911c520 100644 --- a/tests/TypeInfer.tryUnify.test.cpp +++ b/tests/TypeInfer.tryUnify.test.cpp @@ -271,4 +271,40 @@ TEST_CASE_FIXTURE(TryUnifyFixture, "txnlog_preserves_pack_owner") CHECK_EQ(a->owningArena, &arena); } +TEST_CASE_FIXTURE(TryUnifyFixture, "metatables_unify_against_shape_of_free_table") +{ + ScopedFastFlag sff("DebugLuauDeferredConstraintResolution", true); + + TableTypeVar::Props freeProps{ + {"foo", {typeChecker.numberType}}, + }; + + TypeId free = arena.addType(TableTypeVar{freeProps, std::nullopt, TypeLevel{}, TableState::Free}); + + TableTypeVar::Props indexProps{ + {"foo", {typeChecker.stringType}}, + }; + + TypeId index = arena.addType(TableTypeVar{indexProps, std::nullopt, TypeLevel{}, TableState::Sealed}); + + TableTypeVar::Props mtProps{ + {"__index", {index}}, + }; + + TypeId mt = arena.addType(TableTypeVar{mtProps, std::nullopt, TypeLevel{}, TableState::Sealed}); + + TypeId target = arena.addType(TableTypeVar{TableState::Unsealed, TypeLevel{}}); + TypeId metatable = arena.addType(MetatableTypeVar{target, mt}); + + state.tryUnify(metatable, free); + state.log.commit(); + + REQUIRE_EQ(state.errors.size(), 1); + + std::string expected = "Type '{ @metatable {| __index: {| foo: string |} |}, { } }' could not be converted into '{- foo: number -}'\n" + "caused by:\n" + " Type 'number' could not be converted into 'string'"; + CHECK_EQ(toString(state.errors[0]), expected); +} + TEST_SUITE_END(); diff --git a/tests/conformance/events.lua b/tests/conformance/events.lua index 6dcdbf0e..0c6055da 100644 --- a/tests/conformance/events.lua +++ b/tests/conformance/events.lua @@ -24,6 +24,9 @@ assert(getmetatable(nil) == nil) a={}; setmetatable(a, {__metatable = "xuxu", __tostring=function(x) return x.name end}) assert(getmetatable(a) == "xuxu") +ud=newproxy(true); getmetatable(ud).__metatable = "xuxu" +assert(getmetatable(ud) == "xuxu") + local res,err = pcall(tostring, a) assert(not res and err == "'__tostring' must return a string") -- cannot change a protected metatable diff --git a/tools/faillist.txt b/tools/faillist.txt index 825fb2f6..00e01011 100644 --- a/tools/faillist.txt +++ b/tools/faillist.txt @@ -1,101 +1,47 @@ AnnotationTests.builtin_types_are_not_exported +AnnotationTests.corecursive_types_error_on_tight_loop AnnotationTests.duplicate_type_param_name AnnotationTests.for_loop_counter_annotation_is_checked AnnotationTests.generic_aliases_are_cloned_properly AnnotationTests.instantiation_clone_has_to_follow -AnnotationTests.luau_ice_triggers_an_ice -AnnotationTests.luau_ice_triggers_an_ice_exception_with_flag -AnnotationTests.luau_ice_triggers_an_ice_exception_with_flag_handler -AnnotationTests.luau_ice_triggers_an_ice_handler -AnnotationTests.luau_print_is_magic_if_the_flag_is_set AnnotationTests.occurs_check_on_cyclic_intersection_typevar AnnotationTests.occurs_check_on_cyclic_union_typevar +AnnotationTests.too_many_type_params AnnotationTests.two_type_params AnnotationTests.use_type_required_from_another_file AstQuery.last_argument_function_call_type AstQuery::getDocumentationSymbolAtPosition.overloaded_fn -AutocompleteTest.argument_types -AutocompleteTest.arguments_to_global_lambda -AutocompleteTest.autocomplete_boolean_singleton -AutocompleteTest.autocomplete_end_with_fn_exprs -AutocompleteTest.autocomplete_end_with_lambda AutocompleteTest.autocomplete_first_function_arg_expected_type -AutocompleteTest.autocomplete_for_in_middle_keywords -AutocompleteTest.autocomplete_for_middle_keywords -AutocompleteTest.autocomplete_if_middle_keywords AutocompleteTest.autocomplete_interpolated_string -AutocompleteTest.autocomplete_on_string_singletons AutocompleteTest.autocomplete_oop_implicit_self -AutocompleteTest.autocomplete_repeat_middle_keyword +AutocompleteTest.autocomplete_string_singleton_equality AutocompleteTest.autocomplete_string_singleton_escape AutocompleteTest.autocomplete_string_singletons -AutocompleteTest.autocomplete_while_middle_keywords AutocompleteTest.autocompleteProp_index_function_metamethod_is_variadic -AutocompleteTest.bias_toward_inner_scope AutocompleteTest.cyclic_table AutocompleteTest.do_compatible_self_calls -AutocompleteTest.do_not_overwrite_context_sensitive_kws -AutocompleteTest.do_not_suggest_internal_module_type AutocompleteTest.do_wrong_compatible_self_calls -AutocompleteTest.dont_offer_any_suggestions_from_within_a_broken_comment -AutocompleteTest.dont_offer_any_suggestions_from_within_a_broken_comment_at_the_very_end_of_the_file -AutocompleteTest.dont_offer_any_suggestions_from_within_a_comment -AutocompleteTest.dont_suggest_local_before_its_definition -AutocompleteTest.function_expr_params -AutocompleteTest.function_in_assignment_has_parentheses -AutocompleteTest.function_in_assignment_has_parentheses_2 -AutocompleteTest.function_parameters -AutocompleteTest.function_result_passed_to_function_has_parentheses -AutocompleteTest.generic_types -AutocompleteTest.get_suggestions_for_the_very_start_of_the_script -AutocompleteTest.global_function_params -AutocompleteTest.global_functions_are_not_scoped_lexically -AutocompleteTest.globals_are_order_independent -AutocompleteTest.if_then_else_elseif_completions AutocompleteTest.keyword_methods -AutocompleteTest.library_non_self_calls_are_fine -AutocompleteTest.library_self_calls_are_invalid -AutocompleteTest.local_function -AutocompleteTest.local_function_params -AutocompleteTest.local_functions_fall_out_of_scope -AutocompleteTest.method_call_inside_function_body -AutocompleteTest.nested_member_completions -AutocompleteTest.nested_recursive_function -AutocompleteTest.no_function_name_suggestions AutocompleteTest.no_incompatible_self_calls AutocompleteTest.no_incompatible_self_calls_2 -AutocompleteTest.no_incompatible_self_calls_on_class AutocompleteTest.no_wrong_compatible_self_calls_with_generics -AutocompleteTest.recursive_function -AutocompleteTest.recursive_function_global -AutocompleteTest.recursive_function_local -AutocompleteTest.return_types -AutocompleteTest.sometimes_the_metatable_is_an_error -AutocompleteTest.source_module_preservation_and_invalidation -AutocompleteTest.statement_between_two_statements -AutocompleteTest.string_prim_non_self_calls_are_avoided -AutocompleteTest.string_prim_self_calls_are_fine -AutocompleteTest.suggest_external_module_type -AutocompleteTest.table_intersection -AutocompleteTest.table_union +AutocompleteTest.suggest_table_keys AutocompleteTest.type_correct_argument_type_suggestion AutocompleteTest.type_correct_expected_argument_type_pack_suggestion -AutocompleteTest.type_correct_expected_argument_type_suggestion AutocompleteTest.type_correct_expected_argument_type_suggestion_optional AutocompleteTest.type_correct_expected_argument_type_suggestion_self +AutocompleteTest.type_correct_expected_return_type_pack_suggestion AutocompleteTest.type_correct_expected_return_type_suggestion AutocompleteTest.type_correct_full_type_suggestion AutocompleteTest.type_correct_function_no_parenthesis AutocompleteTest.type_correct_function_return_types AutocompleteTest.type_correct_function_type_suggestion AutocompleteTest.type_correct_keywords -AutocompleteTest.type_correct_local_type_suggestion -AutocompleteTest.type_correct_sealed_table AutocompleteTest.type_correct_suggestion_for_overloads AutocompleteTest.type_correct_suggestion_in_argument +AutocompleteTest.type_correct_suggestion_in_table AutocompleteTest.unsealed_table AutocompleteTest.unsealed_table_2 -AutocompleteTest.user_defined_local_functions_in_own_definition BuiltinTests.aliased_string_format BuiltinTests.assert_removes_falsy_types BuiltinTests.assert_removes_falsy_types2 @@ -149,21 +95,20 @@ BuiltinTests.table_insert_correctly_infers_type_of_array_2_args_overload BuiltinTests.table_insert_correctly_infers_type_of_array_3_args_overload BuiltinTests.table_pack BuiltinTests.table_pack_reduce +BuiltinTests.table_pack_variadic BuiltinTests.tonumber_returns_optional_number_type BuiltinTests.tonumber_returns_optional_number_type2 DefinitionTests.class_definition_overload_metamethods DefinitionTests.declaring_generic_functions DefinitionTests.definition_file_classes -FrontendTest.ast_node_at_position FrontendTest.automatically_check_dependent_scripts FrontendTest.environments FrontendTest.imported_table_modification_2 FrontendTest.it_should_be_safe_to_stringify_errors_when_full_type_graph_is_discarded FrontendTest.nocheck_cycle_used_by_checked -FrontendTest.produce_errors_for_unchanged_file_with_a_syntax_error FrontendTest.recheck_if_dependent_script_is_dirty FrontendTest.reexport_cyclic_type -FrontendTest.report_syntax_error_in_required_file +FrontendTest.reexport_type_alias FrontendTest.trace_requires_in_nonstrict_mode GenericsTests.apply_type_function_nested_generics1 GenericsTests.apply_type_function_nested_generics2 @@ -212,13 +157,16 @@ IntersectionTypes.table_intersection_write_sealed_indirect IntersectionTypes.table_write_sealed_indirect isSubtype.intersection_of_tables isSubtype.table_with_table_prop +ModuleTests.any_persistance_does_not_leak ModuleTests.clone_self_property ModuleTests.deepClone_cyclic_table ModuleTests.do_not_clone_reexports NonstrictModeTests.for_in_iterator_variables_are_any NonstrictModeTests.function_parameters_are_any NonstrictModeTests.inconsistent_module_return_types_are_ok +NonstrictModeTests.inconsistent_return_types_are_ok NonstrictModeTests.infer_nullary_function +NonstrictModeTests.infer_the_maximum_number_of_values_the_function_could_return NonstrictModeTests.inline_table_props_are_also_any NonstrictModeTests.local_tables_are_not_any NonstrictModeTests.locals_are_any_by_default @@ -324,7 +272,6 @@ RefinementTest.typeguard_in_if_condition_position RefinementTest.typeguard_narrows_for_functions RefinementTest.typeguard_narrows_for_table RefinementTest.typeguard_not_to_be_string -RefinementTest.typeguard_only_look_up_types_from_global_scope RefinementTest.what_nonsensical_condition RefinementTest.x_as_any_if_x_is_instance_elseif_x_is_table RefinementTest.x_is_not_instance_or_else_not_part @@ -349,6 +296,7 @@ TableTests.defining_a_self_method_for_a_builtin_sealed_table_must_fail TableTests.defining_a_self_method_for_a_local_sealed_table_must_fail TableTests.dont_crash_when_setmetatable_does_not_produce_a_metatabletypevar TableTests.dont_hang_when_trying_to_look_up_in_cyclic_metatable_index +TableTests.dont_invalidate_the_properties_iterator_of_free_table_when_rolled_back TableTests.dont_leak_free_table_props TableTests.dont_quantify_table_that_belongs_to_outer_scope TableTests.dont_suggest_exact_match_keys @@ -363,13 +311,11 @@ TableTests.found_like_key_in_table_function_call TableTests.found_like_key_in_table_property_access TableTests.found_multiple_like_keys TableTests.function_calls_produces_sealed_table_given_unsealed_table -TableTests.generalize_table_argument TableTests.getmetatable_returns_pointer_to_metatable TableTests.give_up_after_one_metatable_index_look_up TableTests.hide_table_error_properties TableTests.indexer_fn TableTests.indexer_on_sealed_table_must_unify_with_free_table -TableTests.indexer_table TableTests.indexing_from_a_table_should_prefer_properties_when_possible TableTests.inequality_operators_imply_exactly_matching_types TableTests.infer_array_2 @@ -395,11 +341,11 @@ TableTests.open_table_unification_2 TableTests.pass_a_union_of_tables_to_a_function_that_requires_a_table TableTests.pass_a_union_of_tables_to_a_function_that_requires_a_table_2 TableTests.pass_incompatible_union_to_a_generic_table_without_crashing -TableTests.passing_compatible_unions_to_a_generic_table_without_crashing TableTests.persistent_sealed_table_is_immutable TableTests.prop_access_on_key_whose_types_mismatches TableTests.property_lookup_through_tabletypevar_metatable TableTests.quantify_even_that_table_was_never_exported_at_all +TableTests.quantify_metatables_of_metatables_of_table TableTests.quantifying_a_bound_var_works TableTests.reasonable_error_when_adding_a_nonexistent_property_to_an_array_like_table TableTests.result_is_always_any_if_lhs_is_any @@ -435,8 +381,11 @@ ToString.function_type_with_argument_names_generic ToString.no_parentheses_around_cyclic_function_type_in_union ToString.toStringDetailed2 ToString.toStringErrorPack +ToString.toStringNamedFunction_generic_pack +ToString.toStringNamedFunction_hide_self_param ToString.toStringNamedFunction_hide_type_params ToString.toStringNamedFunction_id +ToString.toStringNamedFunction_include_self_param ToString.toStringNamedFunction_map ToString.toStringNamedFunction_variadics TranspilerTests.types_should_not_be_considered_cyclic_if_they_are_not_recursive @@ -453,6 +402,7 @@ TypeAliases.mutually_recursive_types_restriction_not_ok_1 TypeAliases.mutually_recursive_types_restriction_not_ok_2 TypeAliases.mutually_recursive_types_swapsies_not_ok TypeAliases.recursive_types_restriction_not_ok +TypeAliases.report_shadowed_aliases TypeAliases.stringify_optional_parameterized_alias TypeAliases.stringify_type_alias_of_recursive_template_table_type TypeAliases.stringify_type_alias_of_recursive_template_table_type2 @@ -460,11 +410,14 @@ TypeAliases.type_alias_fwd_declaration_is_precise TypeAliases.type_alias_local_mutation TypeAliases.type_alias_local_rename TypeAliases.type_alias_of_an_imported_recursive_generic_type +TypeAliases.type_alias_of_an_imported_recursive_type TypeInfer.checking_should_not_ice +TypeInfer.dont_report_type_errors_within_an_AstExprError TypeInfer.dont_report_type_errors_within_an_AstStatError TypeInfer.globals TypeInfer.globals2 TypeInfer.infer_assignment_value_types_mutable_lval +TypeInfer.it_is_ok_to_have_inconsistent_number_of_return_values_in_nonstrict TypeInfer.no_stack_overflow_from_isoptional TypeInfer.tc_after_error_recovery_no_replacement_name_in_error TypeInfer.tc_if_else_expressions_expected_type_3 @@ -489,6 +442,7 @@ TypeInferClasses.table_class_unification_reports_sane_errors_for_missing_propert TypeInferClasses.warn_when_prop_almost_matches TypeInferClasses.we_can_report_when_someone_is_trying_to_use_a_table_rather_than_a_class TypeInferFunctions.call_o_with_another_argument_after_foo_was_quantified +TypeInferFunctions.calling_function_with_anytypepack_doesnt_leak_free_types TypeInferFunctions.calling_function_with_incorrect_argument_type_yields_errors_spanning_argument TypeInferFunctions.dont_give_other_overloads_message_if_only_one_argument_matching_overload_exists TypeInferFunctions.dont_infer_parameter_types_for_functions_from_their_call_site @@ -500,16 +454,20 @@ TypeInferFunctions.function_decl_non_self_sealed_overwrite_2 TypeInferFunctions.function_decl_non_self_unsealed_overwrite TypeInferFunctions.function_does_not_return_enough_values TypeInferFunctions.function_statement_sealed_table_assignment_through_indexer +TypeInferFunctions.ignored_return_values TypeInferFunctions.improved_function_arg_mismatch_error_nonstrict TypeInferFunctions.improved_function_arg_mismatch_errors TypeInferFunctions.inconsistent_higher_order_function TypeInferFunctions.inconsistent_return_types TypeInferFunctions.infer_anonymous_function_arguments TypeInferFunctions.infer_return_type_from_selected_overload +TypeInferFunctions.infer_return_value_type TypeInferFunctions.infer_that_function_does_not_return_a_table +TypeInferFunctions.it_is_ok_not_to_supply_enough_retvals TypeInferFunctions.list_all_overloads_if_no_overload_takes_given_argument_count TypeInferFunctions.list_only_alternative_overloads_that_match_argument_count TypeInferFunctions.no_lossy_function_type +TypeInferFunctions.occurs_check_failure_in_function_return_type TypeInferFunctions.quantify_constrained_types TypeInferFunctions.record_matching_overload TypeInferFunctions.report_exiting_without_return_nonstrict @@ -521,7 +479,13 @@ TypeInferFunctions.too_few_arguments_variadic_generic TypeInferFunctions.too_few_arguments_variadic_generic2 TypeInferFunctions.too_many_arguments TypeInferFunctions.too_many_return_values +TypeInferFunctions.too_many_return_values_in_parentheses +TypeInferFunctions.too_many_return_values_no_function TypeInferFunctions.vararg_function_is_quantified +TypeInferLoops.for_in_loop_error_on_factory_not_returning_the_right_amount_of_values +TypeInferLoops.for_in_loop_with_custom_iterator +TypeInferLoops.for_in_loop_with_next +TypeInferLoops.for_in_with_generic_next TypeInferLoops.for_in_with_just_one_iterator_is_ok TypeInferLoops.loop_iter_no_indexer_nonstrict TypeInferLoops.loop_iter_trailing_nil @@ -529,6 +493,9 @@ TypeInferLoops.loop_typecheck_crash_on_empty_optional TypeInferLoops.unreachable_code_after_infinite_loop TypeInferLoops.varlist_declared_by_for_in_loop_should_be_free TypeInferModules.custom_require_global +TypeInferModules.do_not_modify_imported_types +TypeInferModules.do_not_modify_imported_types_2 +TypeInferModules.do_not_modify_imported_types_3 TypeInferModules.general_require_type_mismatch TypeInferModules.module_type_conflict TypeInferModules.module_type_conflict_instantiated @@ -539,8 +506,8 @@ TypeInferOOP.CheckMethodsOfSealed TypeInferOOP.dont_suggest_using_colon_rather_than_dot_if_another_overload_works TypeInferOOP.dont_suggest_using_colon_rather_than_dot_if_it_wont_help_2 TypeInferOOP.dont_suggest_using_colon_rather_than_dot_if_not_defined_with_colon -TypeInferOOP.inferred_methods_of_free_tables_have_the_same_level_as_the_enclosing_table TypeInferOOP.inferring_hundreds_of_self_calls_should_not_suffocate_memory +TypeInferOOP.methods_are_topologically_sorted TypeInferOperators.and_adds_boolean TypeInferOperators.and_adds_boolean_no_superfluous_union TypeInferOperators.and_binexps_dont_unify @@ -564,6 +531,7 @@ TypeInferOperators.expected_types_through_binary_or TypeInferOperators.infer_any_in_all_modes_when_lhs_is_unknown TypeInferOperators.or_joins_types TypeInferOperators.or_joins_types_with_no_extras +TypeInferOperators.primitive_arith_no_metatable TypeInferOperators.primitive_arith_possible_metatable TypeInferOperators.produce_the_correct_error_message_when_comparing_a_table_with_a_metatable_with_one_that_does_not TypeInferOperators.refine_and_or @@ -591,6 +559,7 @@ TypeInferUnknownNever.dont_unify_operands_if_one_of_the_operand_is_never_in_any_ TypeInferUnknownNever.index_on_union_of_tables_for_properties_that_is_never TypeInferUnknownNever.index_on_union_of_tables_for_properties_that_is_sorta_never TypeInferUnknownNever.math_operators_and_never +TypeInferUnknownNever.type_packs_containing_never_is_itself_uninhabitable TypeInferUnknownNever.type_packs_containing_never_is_itself_uninhabitable2 TypeInferUnknownNever.unary_minus_of_never TypePackTests.higher_order_function diff --git a/tools/lldb_formatters.lldb b/tools/lldb_formatters.lldb index f6fa6cf5..4a5acd74 100644 --- a/tools/lldb_formatters.lldb +++ b/tools/lldb_formatters.lldb @@ -1,2 +1,7 @@ +type synthetic add -x "^Luau::detail::DenseHashTable<.*>$" -l lldb_formatters.DenseHashTableSyntheticChildrenProvider +type summary add "Luau::Symbol" -F lldb_formatters.luau_symbol_summary + type synthetic add -x "^Luau::Variant<.+>$" -l lldb_formatters.LuauVariantSyntheticChildrenProvider -type summary add -x "^Luau::Variant<.+>$" -l lldb_formatters.luau_variant_summary +type summary add -x "^Luau::Variant<.+>$" -F lldb_formatters.luau_variant_summary + +type synthetic add -x "^Luau::AstArray<.+>$" -l lldb_formatters.AstArraySyntheticChildrenProvider diff --git a/tools/lldb_formatters.py b/tools/lldb_formatters.py index ff610d09..19fc0f54 100644 --- a/tools/lldb_formatters.py +++ b/tools/lldb_formatters.py @@ -4,30 +4,31 @@ # We're forced to resort to parsing names as strings. def templateParams(s): depth = 0 - start = s.find('<') + 1 + start = s.find("<") + 1 result = [] for i, c in enumerate(s[start:], start): - if c == '<': + if c == "<": depth += 1 - elif c == '>': + elif c == ">": if depth == 0: - result.append(s[start: i].strip()) + result.append(s[start:i].strip()) break depth -= 1 - elif c == ',' and depth == 0: - result.append(s[start: i].strip()) + elif c == "," and depth == 0: + result.append(s[start:i].strip()) start = i + 1 return result + def getType(target, typeName): stars = 0 typeName = typeName.strip() - while typeName.endswith('*'): + while typeName.endswith("*"): stars += 1 typeName = typeName[:-1] - if typeName.startswith('const '): + if typeName.startswith("const "): typeName = typeName[6:] ty = target.FindFirstType(typeName.strip()) @@ -36,13 +37,10 @@ def getType(target, typeName): return ty + def luau_variant_summary(valobj, internal_dict, options): - type_id = valobj.GetChildMemberWithName("typeId").GetValueAsUnsigned() - storage = valobj.GetChildMemberWithName("storage") - params = templateParams(valobj.GetType().GetCanonicalType().GetName()) - stored_type = params[type_id] - value = storage.Cast(stored_type.GetPointerType()).Dereference() - return stored_type.GetDisplayTypeName() + " [" + value.GetValue() + "]" + return valobj.GetChildMemberWithName("type").GetSummary()[1:-1] + class LuauVariantSyntheticChildrenProvider: node_names = ["type", "value"] @@ -74,26 +72,42 @@ class LuauVariantSyntheticChildrenProvider: if node == "type": if self.current_type: - return self.valobj.CreateValueFromExpression(node, f"(const char*)\"{self.current_type.GetDisplayTypeName()}\"") + return self.valobj.CreateValueFromExpression( + node, f'(const char*)"{self.current_type.GetDisplayTypeName()}"' + ) else: - return self.valobj.CreateValueFromExpression(node, "(const char*)\"\"") + return self.valobj.CreateValueFromExpression( + node, '(const char*)""' + ) elif node == "value": if self.stored_value is not None: if self.current_type is not None: - return self.valobj.CreateValueFromData(node, self.stored_value.GetData(), self.current_type) + return self.valobj.CreateValueFromData( + node, self.stored_value.GetData(), self.current_type + ) else: - return self.valobj.CreateValueExpression(node, "(const char*)\"\"") + return self.valobj.CreateValueExpression( + node, '(const char*)""' + ) else: - return self.valobj.CreateValueFromExpression(node, "(const char*)\"\"") + return self.valobj.CreateValueFromExpression( + node, '(const char*)""' + ) else: return None def update(self): - self.type_index = self.valobj.GetChildMemberWithName("typeId").GetValueAsSigned() - self.type_params = templateParams(self.valobj.GetType().GetCanonicalType().GetName()) + self.type_index = self.valobj.GetChildMemberWithName( + "typeId" + ).GetValueAsSigned() + self.type_params = templateParams( + self.valobj.GetType().GetCanonicalType().GetName() + ) if len(self.type_params) > self.type_index: - self.current_type = getType(self.valobj.GetTarget(), self.type_params[self.type_index]) + self.current_type = getType( + self.valobj.GetTarget(), self.type_params[self.type_index] + ) if self.current_type: storage = self.valobj.GetChildMemberWithName("storage") @@ -105,3 +119,97 @@ class LuauVariantSyntheticChildrenProvider: self.stored_value = None return False + + +class DenseHashTableSyntheticChildrenProvider: + def __init__(self, valobj, internal_dict): + """this call should initialize the Python object using valobj as the variable to provide synthetic children for""" + self.valobj = valobj + self.update() + + def num_children(self): + """this call should return the number of children that you want your object to have""" + return self.capacity + + def get_child_index(self, name): + """this call should return the index of the synthetic child whose name is given as argument""" + try: + if name.startswith("[") and name.endswith("]"): + return int(name[1:-1]) + else: + return -1 + except Exception as e: + print("get_child_index exception", e) + return -1 + + def get_child_at_index(self, index): + """this call should return a new LLDB SBValue object representing the child at the index given as argument""" + try: + dataMember = self.valobj.GetChildMemberWithName("data") + + data = dataMember.GetPointeeData(index) + + return self.valobj.CreateValueFromData( + f"[{index}]", + data, + dataMember.Dereference().GetType(), + ) + + except Exception as e: + print("get_child_at_index error", e) + + def update(self): + """this call should be used to update the internal state of this Python object whenever the state of the variables in LLDB changes.[1] + Also, this method is invoked before any other method in the interface.""" + self.capacity = self.valobj.GetChildMemberWithName( + "capacity" + ).GetValueAsUnsigned() + + def has_children(self): + """this call should return True if this object might have children, and False if this object can be guaranteed not to have children.[2]""" + return True + + +def luau_symbol_summary(valobj, internal_dict, options): + local = valobj.GetChildMemberWithName("local") + global_ = valobj.GetChildMemberWithName("global").GetChildMemberWithName("value") + + if local.GetValueAsUnsigned() != 0: + return f'local {local.GetChildMemberWithName("name").GetChildMemberWithName("value").GetSummary()}' + elif global_.GetValueAsUnsigned() != 0: + return f"global {global_.GetSummary()}" + else: + return "???" + + +class AstArraySyntheticChildrenProvider: + def __init__(self, valobj, internal_dict): + self.valobj = valobj + + def num_children(self): + return self.size + + def get_child_index(self, name): + try: + if name.startswith("[") and name.endswith("]"): + return int(name[1:-1]) + else: + return -1 + except Exception as e: + print("get_child_index error:", e) + + def get_child_at_index(self, index): + try: + dataMember = self.valobj.GetChildMemberWithName("data") + data = dataMember.GetPointeeData(index) + return self.valobj.CreateValueFromData( + f"[{index}]", data, dataMember.Dereference().GetType() + ) + except Exception as e: + print("get_child_index error:", e) + + def update(self): + self.size = self.valobj.GetChildMemberWithName("size").GetValueAsUnsigned() + + def has_children(self): + return True diff --git a/tools/perfgraph.py b/tools/perfgraph.py index 7d2639df..eb6b68ce 100644 --- a/tools/perfgraph.py +++ b/tools/perfgraph.py @@ -4,7 +4,6 @@ # Given a profile dump, this tool generates a flame graph based on the stacks listed in the profile # The result of analysis is a .svg file which can be viewed in a browser -import sys import svg import argparse import json diff --git a/tools/perfstat.py b/tools/perfstat.py new file mode 100644 index 00000000..e5cfd117 --- /dev/null +++ b/tools/perfstat.py @@ -0,0 +1,65 @@ +#!/usr/bin/python +# This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details + +# Given a profile dump, this tool displays top functions based on the stacks listed in the profile + +import argparse + +class Node: + def __init__(self): + self.function = "" + self.source = "" + self.line = 0 + self.hier_ticks = 0 + self.self_ticks = 0 + + def title(self): + if self.line > 0: + return "{} ({}:{})".format(self.function, self.source, self.line) + else: + return self.function + +argumentParser = argparse.ArgumentParser(description='Display summary statistics from Luau sampling profiler dumps') +argumentParser.add_argument('source_file', type=open) +argumentParser.add_argument('--limit', dest='limit', type=int, default=10, help='Display top N functions') + +arguments = argumentParser.parse_args() + +dump = arguments.source_file.readlines() + +stats = {} +total = 0 +total_gc = 0 + +for l in dump: + ticks, stack = l.strip().split(" ", 1) + hier = {} + + for f in reversed(stack.split(";")): + source, function, line = f.split(",") + node = stats.setdefault(f, Node()) + + node.function = function + node.source = source + node.line = int(line) if len(line) > 0 else 0 + + if not node in hier: + node.hier_ticks += int(ticks) + hier[node] = True + + total += int(ticks) + node.self_ticks += int(ticks) + + if node.source == "GC": + total_gc += int(ticks) + +if total > 0: + print(f"Runtime: {total:,} usec ({100.0 * total_gc / total:.2f}% GC)") + print() + print("Top functions (self time):") + for n in sorted(stats.values(), key=lambda node: node.self_ticks, reverse=True)[:arguments.limit]: + print(f"{n.self_ticks:12,} usec ({100.0 * n.self_ticks / total:.2f}%): {n.title()}") + print() + print("Top functions (total time):") + for n in sorted(stats.values(), key=lambda node: node.hier_ticks, reverse=True)[:arguments.limit]: + print(f"{n.hier_ticks:12,} usec ({100.0 * n.hier_ticks / total:.2f}%): {n.title()}") diff --git a/tools/test_dcr.py b/tools/test_dcr.py index db932253..1e3a5017 100644 --- a/tools/test_dcr.py +++ b/tools/test_dcr.py @@ -39,7 +39,7 @@ class Handler(x.ContentHandler): elif name == "OverallResultsAsserts": if self.currentTest: - passed = 0 == safeParseInt(attrs["failures"]) + passed = attrs["test_case_success"] == "true" dottedName = ".".join(self.currentTest) From 937ef2efd4d319932cfd108e636ba5379a2301c9 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Thu, 29 Sep 2022 15:42:23 -0700 Subject: [PATCH 369/399] Cleanup benchmark builds a little bit (#691) We don't need to run any cachegrind benchmarks in benchmark-dev, since benchmark uses our new callgrind setup instead. Also removes prototyping filters that we no longer need from all builds. --- .github/workflows/benchmark-dev.yml | 200 ---------------------------- .github/workflows/benchmark.yml | 1 - .github/workflows/build.yml | 2 - .github/workflows/release.yml | 1 - scripts/run-with-cachegrind.sh | 109 --------------- 5 files changed, 313 deletions(-) delete mode 100644 scripts/run-with-cachegrind.sh diff --git a/.github/workflows/benchmark-dev.yml b/.github/workflows/benchmark-dev.yml index 210a38e5..d78cb0ba 100644 --- a/.github/workflows/benchmark-dev.yml +++ b/.github/workflows/benchmark-dev.yml @@ -10,7 +10,6 @@ on: - "papers/**" - "rfcs/**" - "*.md" - - "prototyping/**" jobs: windows: @@ -25,8 +24,6 @@ jobs: script: "run-benchmarks", timeout: 12, title: "Luau Benchmarks", - cachegrindTitle: "Performance", - cachegrindIterCount: 20, } benchResultsRepo: - { name: "luau-lang/benchmark-data", branch: "main" } @@ -117,8 +114,6 @@ jobs: script: "run-benchmarks", timeout: 12, title: "Luau Benchmarks", - cachegrindTitle: "Performance", - cachegrindIterCount: 20, } benchResultsRepo: - { name: "luau-lang/benchmark-data", branch: "main" } @@ -145,19 +140,6 @@ jobs: run: | python bench/bench.py | tee ${{ matrix.bench.script }}-output.txt - - name: Install valgrind - if: matrix.os == 'ubuntu-latest' - run: | - sudo apt-get install valgrind - - - name: Run ${{ matrix.bench.title }} (Cold Cachegrind) - if: matrix.os == 'ubuntu-latest' - run: sudo bash ./scripts/run-with-cachegrind.sh python ./bench/bench.py "${{ matrix.bench.cachegrindTitle}}Cold" 1 | tee -a ${{ matrix.bench.script }}-output.txt - - - name: Run ${{ matrix.bench.title }} (Warm Cachegrind) - if: matrix.os == 'ubuntu-latest' - run: sudo bash ./scripts/run-with-cachegrind.sh python ./bench/bench.py "${{ matrix.bench.cachegrindTitle }}" ${{ matrix.bench.cachegrindIterCount }} | tee -a ${{ matrix.bench.script }}-output.txt - - name: Push benchmark results id: pushBenchmarkAttempt1 continue-on-error: true @@ -201,185 +183,3 @@ jobs: bench_tool: "benchmarkluau" bench_output_file_path: "./${{ matrix.bench.script }}-output.txt" bench_external_data_json_path: "./gh-pages/dev/bench/data-${{ matrix.os }}.json" - - - name: Push Cachegrind benchmark results - if: matrix.os == 'ubuntu-latest' - id: pushBenchmarkCachegrindAttempt1 - continue-on-error: true - uses: ./.github/workflows/push-results - with: - repository: ${{ matrix.benchResultsRepo.name }} - branch: ${{ matrix.benchResultsRepo.branch }} - token: ${{ secrets.BENCH_GITHUB_TOKEN }} - path: "./gh-pages" - bench_name: ${{ matrix.bench.title }} (CacheGrind) - bench_tool: "roblox" - bench_output_file_path: "./${{ matrix.bench.script }}-output.txt" - bench_external_data_json_path: "./gh-pages/dev/bench/data-${{ matrix.os }}.json" - - - name: Push Cachegrind benchmark results (Attempt 2) - if: matrix.os == 'ubuntu-latest' && steps.pushBenchmarkCachegrindAttempt1.outcome == 'failure' - id: pushBenchmarkCachegrindAttempt2 - continue-on-error: true - uses: ./.github/workflows/push-results - with: - repository: ${{ matrix.benchResultsRepo.name }} - branch: ${{ matrix.benchResultsRepo.branch }} - token: ${{ secrets.BENCH_GITHUB_TOKEN }} - path: "./gh-pages" - bench_name: ${{ matrix.bench.title }} (CacheGrind) - bench_tool: "roblox" - bench_output_file_path: "./${{ matrix.bench.script }}-output.txt" - bench_external_data_json_path: "./gh-pages/dev/bench/data-${{ matrix.os }}.json" - - - name: Push Cachegrind benchmark results (Attempt 3) - if: matrix.os == 'ubuntu-latest' && steps.pushBenchmarkCachegrindAttempt2.outcome == 'failure' - id: pushBenchmarkCachegrindAttempt3 - continue-on-error: true - uses: ./.github/workflows/push-results - with: - repository: ${{ matrix.benchResultsRepo.name }} - branch: ${{ matrix.benchResultsRepo.branch }} - token: ${{ secrets.BENCH_GITHUB_TOKEN }} - path: "./gh-pages" - bench_name: ${{ matrix.bench.title }} (CacheGrind) - bench_tool: "roblox" - bench_output_file_path: "./${{ matrix.bench.script }}-output.txt" - bench_external_data_json_path: "./gh-pages/dev/bench/data-${{ matrix.os }}.json" - - static-analysis: - name: luau-analyze - strategy: - fail-fast: false - matrix: - os: [ubuntu-latest] - bench: - - { - script: "run-analyze", - timeout: 12, - title: "Luau Analyze", - cachegrindTitle: "Performance", - cachegrindIterCount: 20, - } - benchResultsRepo: - - { name: "luau-lang/benchmark-data", branch: "main" } - - runs-on: ${{ matrix.os }} - steps: - - uses: actions/checkout@v3 - with: - token: "${{ secrets.BENCH_GITHUB_TOKEN }}" - - - name: Build Luau - run: make config=release luau luau-analyze - - - uses: actions/setup-python@v4 - with: - python-version: "3.9" - architecture: "x64" - - - name: Install python dependencies - run: | - sudo pip install requests numpy scipy matplotlib ipython jupyter pandas sympy nose - - - name: Install valgrind - run: | - sudo apt-get install valgrind - - - name: Run Luau Analyze on static file - run: sudo python ./bench/measure_time.py ./build/release/luau-analyze bench/other/LuauPolyfillMap.lua | tee ${{ matrix.bench.script }}-output.txt - - - name: Run ${{ matrix.bench.title }} (Cold Cachegrind) - run: sudo ./scripts/run-with-cachegrind.sh python ./bench/measure_time.py "${{ matrix.bench.cachegrindTitle}}Cold" 1 ./build/release/luau-analyze bench/other/LuauPolyfillMap.lua | tee -a ${{ matrix.bench.script }}-output.txt - - - name: Run ${{ matrix.bench.title }} (Warm Cachegrind) - run: sudo bash ./scripts/run-with-cachegrind.sh python ./bench/measure_time.py "${{ matrix.bench.cachegrindTitle}}" 1 ./build/release/luau-analyze bench/other/LuauPolyfillMap.lua | tee -a ${{ matrix.bench.script }}-output.txt - - - name: Push static analysis results - id: pushStaticAnalysisAttempt1 - continue-on-error: true - uses: ./.github/workflows/push-results - with: - repository: ${{ matrix.benchResultsRepo.name }} - branch: ${{ matrix.benchResultsRepo.branch }} - token: ${{ secrets.BENCH_GITHUB_TOKEN }} - path: "./gh-pages" - bench_name: ${{ matrix.bench.title }} - bench_tool: "roblox" - bench_output_file_path: "./${{ matrix.bench.script }}-output.txt" - bench_external_data_json_path: "./gh-pages/dev/bench/data-${{ matrix.os }}.json" - - - name: Push static analysis results (Attempt 2) - if: steps.pushStaticAnalysisAttempt1.outcome == 'failure' - id: pushStaticAnalysisAttempt2 - continue-on-error: true - uses: ./.github/workflows/push-results - with: - repository: ${{ matrix.benchResultsRepo.name }} - branch: ${{ matrix.benchResultsRepo.branch }} - token: ${{ secrets.BENCH_GITHUB_TOKEN }} - path: "./gh-pages" - bench_name: ${{ matrix.bench.title }} - bench_tool: "roblox" - bench_output_file_path: "./${{ matrix.bench.script }}-output.txt" - bench_external_data_json_path: "./gh-pages/dev/bench/data-${{ matrix.os }}.json" - - - name: Push static analysis results (Attempt 3) - if: steps.pushStaticAnalysisAttempt2.outcome == 'failure' - id: pushStaticAnalysisAttempt3 - continue-on-error: true - uses: ./.github/workflows/push-results - with: - repository: ${{ matrix.benchResultsRepo.name }} - branch: ${{ matrix.benchResultsRepo.branch }} - token: ${{ secrets.BENCH_GITHUB_TOKEN }} - path: "./gh-pages" - bench_name: ${{ matrix.bench.title }} - bench_tool: "roblox" - bench_output_file_path: "./${{ matrix.bench.script }}-output.txt" - bench_external_data_json_path: "./gh-pages/dev/bench/data-${{ matrix.os }}.json" - - - name: Push static analysis Cachegrind results - if: matrix.os == 'ubuntu-latest' - id: pushStaticAnalysisCachegrindAttempt1 - continue-on-error: true - uses: ./.github/workflows/push-results - with: - repository: ${{ matrix.benchResultsRepo.name }} - branch: ${{ matrix.benchResultsRepo.branch }} - token: ${{ secrets.BENCH_GITHUB_TOKEN }} - path: "./gh-pages" - bench_name: ${{ matrix.bench.title }} - bench_tool: "roblox" - bench_output_file_path: "./${{ matrix.bench.script }}-output.txt" - bench_external_data_json_path: "./gh-pages/dev/bench/data-${{ matrix.os }}.json" - - - name: Push static analysis Cachegrind results (Attempt 2) - if: matrix.os == 'ubuntu-latest' && steps.pushStaticAnalysisCachegrindAttempt1.outcome == 'failure' - id: pushStaticAnalysisCachegrindAttempt2 - continue-on-error: true - uses: ./.github/workflows/push-results - with: - repository: ${{ matrix.benchResultsRepo.name }} - branch: ${{ matrix.benchResultsRepo.branch }} - token: ${{ secrets.BENCH_GITHUB_TOKEN }} - path: "./gh-pages" - bench_name: ${{ matrix.bench.title }} - bench_tool: "roblox" - bench_output_file_path: "./${{ matrix.bench.script }}-output.txt" - bench_external_data_json_path: "./gh-pages/dev/bench/data-${{ matrix.os }}.json" - - - name: Push static analysis Cachegrind results (Attempt 2) - if: matrix.os == 'ubuntu-latest' && steps.pushStaticAnalysisCachegrindAttempt2.outcome == 'failure' - id: pushStaticAnalysisCachegrindAttempt3 - continue-on-error: true - uses: ./.github/workflows/push-results - with: - repository: ${{ matrix.benchResultsRepo.name }} - branch: ${{ matrix.benchResultsRepo.branch }} - token: ${{ secrets.BENCH_GITHUB_TOKEN }} - path: "./gh-pages" - bench_name: ${{ matrix.bench.title }} - bench_tool: "roblox" - bench_output_file_path: "./${{ matrix.bench.script }}-output.txt" - bench_external_data_json_path: "./gh-pages/dev/bench/data-${{ matrix.os }}.json" diff --git a/.github/workflows/benchmark.yml b/.github/workflows/benchmark.yml index deb8be86..80a4ba26 100644 --- a/.github/workflows/benchmark.yml +++ b/.github/workflows/benchmark.yml @@ -9,7 +9,6 @@ on: - "papers/**" - "rfcs/**" - "*.md" - - "prototyping/**" jobs: callgrind: diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml index 0958747d..78c3a34d 100644 --- a/.github/workflows/build.yml +++ b/.github/workflows/build.yml @@ -9,14 +9,12 @@ on: - 'papers/**' - 'rfcs/**' - '*.md' - - 'prototyping/**' pull_request: paths-ignore: - 'docs/**' - 'papers/**' - 'rfcs/**' - '*.md' - - 'prototyping/**' jobs: unix: diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index 2bf796a2..e5cd952b 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -9,7 +9,6 @@ on: - 'papers/**' - 'rfcs/**' - '*.md' - - 'prototyping/**' jobs: build: diff --git a/scripts/run-with-cachegrind.sh b/scripts/run-with-cachegrind.sh deleted file mode 100644 index 787043ff..00000000 --- a/scripts/run-with-cachegrind.sh +++ /dev/null @@ -1,109 +0,0 @@ -#!/bin/bash -set -euo pipefail -IFS=$'\n\t' - -declare -A event_map -event_map[Ir]="TotalInstructionsExecuted,executions\n" -event_map[I1mr]="L1_InstrReadCacheMisses,misses/op\n" -event_map[ILmr]="LL_InstrReadCacheMisses,misses/op\n" -event_map[Dr]="TotalMemoryReads,reads\n" -event_map[D1mr]="L1_DataReadCacheMisses,misses/op\n" -event_map[DLmr]="LL_DataReadCacheMisses,misses/op\n" -event_map[Dw]="TotalMemoryWrites,writes\n" -event_map[D1mw]="L1_DataWriteCacheMisses,misses/op\n" -event_map[DLmw]="LL_DataWriteCacheMisses,misses/op\n" -event_map[Bc]="ConditionalBranchesExecuted,executions\n" -event_map[Bcm]="ConditionalBranchMispredictions,mispredictions/op\n" -event_map[Bi]="IndirectBranchesExecuted,executions\n" -event_map[Bim]="IndirectBranchMispredictions,mispredictions/op\n" - -now_ms() { - echo -n $(date +%s%N | cut -b1-13) -} - -# Run cachegrind on a given benchmark and echo the results. -ITERATION_COUNT=$4 -START_TIME=$(now_ms) - -ARGS=( "$@" ) -REST_ARGS="${ARGS[@]:4}" - -valgrind \ - --quiet \ - --tool=cachegrind \ - "$1" "$2" $REST_ARGS>/dev/null - -ARGS=( "$@" ) -REST_ARGS="${ARGS[@]:4}" - - -TIME_ELAPSED=$(bc <<< "$(now_ms) - ${START_TIME}") - -# Generate report using cg_annotate and extract the header and totals of the -# recorded events valgrind was configured to record. -CG_RESULTS=$(cg_annotate $(ls -t cachegrind.out.* | head -1)) -CG_HEADERS=$(grep -B2 'PROGRAM TOTALS$' <<< "$CG_RESULTS" | head -1 | sed -E 's/\s+/\n/g' | sed '/^$/d') -CG_TOTALS=$(grep 'PROGRAM TOTALS$' <<< "$CG_RESULTS" | head -1 | grep -Po '[0-9,]+\s' | tr -d ', ') - -TOTALS_ARRAY=($CG_TOTALS) -HEADERS_ARRAY=($CG_HEADERS) - -declare -A header_map -for i in "${!TOTALS_ARRAY[@]}"; do - header_map[${HEADERS_ARRAY[$i]}]=$i -done - -# Map the results to the format that the benchmark script expects. -for i in "${!TOTALS_ARRAY[@]}"; do - TOTAL=${TOTALS_ARRAY[$i]} - - # Labels and unit descriptions are packed together in the map. - EVENT_TUPLE=${event_map[${HEADERS_ARRAY[$i]}]} - IFS=$',' read -d '\n' -ra EVENT_VALUES < <(printf "%s" "$EVENT_TUPLE") - EVENT_NAME="${EVENT_VALUES[0]}" - UNIT="${EVENT_VALUES[1]}" - - case ${HEADERS_ARRAY[$i]} in - I1mr | ILmr) - REF=${TOTALS_ARRAY[header_map["Ir"]]} - OPS_PER_SEC=$(bc -l <<< "$TOTAL / $REF") - ;; - - D1mr | DLmr) - REF=${TOTALS_ARRAY[header_map["Dr"]]} - OPS_PER_SEC=$(bc -l <<< "$TOTAL / $REF") - ;; - - D1mw | DLmw) - REF=${TOTALS_ARRAY[header_map["Dw"]]} - OPS_PER_SEC=$(bc -l <<< "$TOTAL / $REF") - ;; - - Bcm) - REF=${TOTALS_ARRAY[header_map["Bc"]]} - OPS_PER_SEC=$(bc -l <<< "$TOTAL / $REF") - ;; - - Bim) - REF=${TOTALS_ARRAY[header_map["Bi"]]} - OPS_PER_SEC=$(bc -l <<< "$TOTAL / $REF") - ;; - - *) - OPS_PER_SEC=$(bc -l <<< "$TOTAL") - ;; - esac - - STD_DEV="0%" - RUNS="1" - - if [[ $OPS_PER_SEC =~ ^[+-]?[0-9]*$ ]] - then # $OPS_PER_SEC is integer - printf "%s#%s x %.0f %s ±%s (%d runs sampled)\n" \ - "$3" "$EVENT_NAME" "$OPS_PER_SEC" "$UNIT" "$STD_DEV" "$RUNS" - else # $OPS_PER_SEC is float - printf "%s#%s x %.10f %s ±%s (%d runs sampled)\n" \ - "$3" "$EVENT_NAME" "$OPS_PER_SEC" "$UNIT" "$STD_DEV" "$RUNS" - fi - -done From cc26ef16df80efc0e466e32685e3b22356733be6 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Thu, 29 Sep 2022 16:19:37 -0700 Subject: [PATCH 370/399] Update release.yml Use -j2 for release builds since CMake doesn't do it for us --- .github/workflows/release.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index e5cd952b..36d3534c 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -22,7 +22,7 @@ jobs: - name: configure run: cmake . -DCMAKE_BUILD_TYPE=Release - name: build - run: cmake --build . --target Luau.Repl.CLI Luau.Analyze.CLI --config Release + run: cmake --build . --target Luau.Repl.CLI Luau.Analyze.CLI --config Release -j 2 - uses: actions/upload-artifact@v2 if: matrix.os != 'windows' with: From d5a2a1585e55ed7b69e29e6ebcb73be3d776ce55 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Thu, 6 Oct 2022 17:23:29 -0700 Subject: [PATCH 371/399] Sync to upstream/release/548 (#699) - Fix rare type checking bugs with invalid generic types escaping the module scope - Fix type checking of variadic type packs in certain cases - Implement type normalization, which resolves a large set of various issues with unions/intersections in type checker - Improve parse errors for trailing commas in function calls and type lists - Reduce profiling skew when using --profile with very high frequencies - Improve performance of `lua_getinfo` (`debug.info`, `debug.traceback` and profiling overhead are now 20% faster/smaller) - Improve performance of polymorphic comparisons (1-2% lift on some benchmarks) - Improve performance of closure creation (1-2% lift on some benchmarks) - Improve string comparison performance (4% lift on string sorting) --- Analysis/include/Luau/ConstraintSolver.h | 12 +- Analysis/include/Luau/Error.h | 2 +- Analysis/include/Luau/Frontend.h | 7 +- Analysis/include/Luau/Normalize.h | 229 ++- Analysis/include/Luau/TypeInfer.h | 3 + Analysis/include/Luau/Unifiable.h | 4 +- Analysis/include/Luau/Unifier.h | 10 +- Analysis/src/Autocomplete.cpp | 132 +- Analysis/src/ConstraintSolver.cpp | 20 +- Analysis/src/Error.cpp | 4 +- Analysis/src/Frontend.cpp | 8 +- Analysis/src/Module.cpp | 14 +- Analysis/src/Normalize.cpp | 1696 +++++++++++++++++++- Analysis/src/TypeChecker2.cpp | 6 +- Analysis/src/TypeInfer.cpp | 103 +- Analysis/src/TypeVar.cpp | 5 + Analysis/src/Unifiable.cpp | 26 +- Analysis/src/Unifier.cpp | 288 +++- Ast/src/Parser.cpp | 23 + CLI/Profiler.cpp | 6 +- CodeGen/include/Luau/AssemblyBuilderX64.h | 1 + CodeGen/include/Luau/CodeAllocator.h | 8 +- CodeGen/include/Luau/CodeBlockUnwind.h | 2 +- CodeGen/include/Luau/UnwindBuilder.h | 5 +- CodeGen/include/Luau/UnwindBuilderDwarf2.h | 5 + CodeGen/include/Luau/UnwindBuilderWin.h | 5 + CodeGen/src/AssemblyBuilderX64.cpp | 28 +- CodeGen/src/CodeAllocator.cpp | 18 +- CodeGen/src/CodeBlockUnwind.cpp | 6 +- CodeGen/src/UnwindBuilderDwarf2.cpp | 10 + CodeGen/src/UnwindBuilderWin.cpp | 10 + Common/include/Luau/ExperimentalFlags.h | 1 + Makefile | 17 +- VM/include/lua.h | 4 +- VM/src/ldebug.cpp | 59 +- VM/src/lgc.cpp | 59 +- VM/src/lobject.cpp | 86 +- VM/src/lobject.h | 2 +- VM/src/lvmexecute.cpp | 9 +- VM/src/lvmload.cpp | 8 +- VM/src/lvmutils.cpp | 7 +- tests/AssemblyBuilderX64.test.cpp | 12 +- tests/Autocomplete.test.cpp | 18 - tests/CodeAllocator.test.cpp | 119 +- tests/ConstraintSolver.test.cpp | 16 +- tests/Fixture.cpp | 4 +- tests/Linter.test.cpp | 164 +- tests/Normalize.test.cpp | 10 +- tests/Parser.test.cpp | 55 + tests/TypeInfer.classes.test.cpp | 34 + tests/TypeInfer.functions.test.cpp | 30 +- tests/TypeInfer.generics.test.cpp | 65 +- tests/TypeInfer.intersectionTypes.test.cpp | 455 ++++++ tests/TypeInfer.modules.test.cpp | 23 +- tests/TypeInfer.provisional.test.cpp | 110 +- tests/TypeInfer.tables.test.cpp | 65 +- tests/TypeInfer.test.cpp | 60 +- tests/TypeInfer.tryUnify.test.cpp | 4 +- tests/TypeInfer.typePacks.cpp | 19 + tests/TypeInfer.unionTypes.test.cpp | 177 ++ tests/conformance/basic.lua | 3 + tests/main.cpp | 17 + tools/lldb_formatters.lldb | 3 + tools/natvis/Ast.natvis | 21 + tools/test_dcr.py | 50 +- 65 files changed, 4001 insertions(+), 481 deletions(-) diff --git a/Analysis/include/Luau/ConstraintSolver.h b/Analysis/include/Luau/ConstraintSolver.h index 06f53e4a..9d5aadfb 100644 --- a/Analysis/include/Luau/ConstraintSolver.h +++ b/Analysis/include/Luau/ConstraintSolver.h @@ -7,6 +7,7 @@ #include "Luau/Constraint.h" #include "Luau/TypeVar.h" #include "Luau/ToString.h" +#include "Luau/Normalize.h" #include @@ -44,6 +45,7 @@ struct ConstraintSolver TypeArena* arena; NotNull singletonTypes; InternalErrorReporter iceReporter; + NotNull normalizer; // The entire set of constraints that the solver is trying to resolve. std::vector> constraints; NotNull rootScope; @@ -74,9 +76,12 @@ struct ConstraintSolver DcrLogger* logger; - explicit ConstraintSolver(TypeArena* arena, NotNull singletonTypes, NotNull rootScope, ModuleName moduleName, + explicit ConstraintSolver(NotNull normalizer, NotNull rootScope, ModuleName moduleName, NotNull moduleResolver, std::vector requireCycles, DcrLogger* logger); + // Randomize the order in which to dispatch constraints + void randomize(unsigned seed); + /** * Attempts to dispatch all pending constraints and reach a type solution * that satisfies all of the constraints. @@ -85,8 +90,9 @@ struct ConstraintSolver bool done(); - /** Attempt to dispatch a constraint. Returns true if it was successful. - * If tryDispatch() returns false, the constraint remains in the unsolved set and will be retried later. + /** Attempt to dispatch a constraint. Returns true if it was successful. If + * tryDispatch() returns false, the constraint remains in the unsolved set + * and will be retried later. */ bool tryDispatch(NotNull c, bool force); diff --git a/Analysis/include/Luau/Error.h b/Analysis/include/Luau/Error.h index f3735864..67754883 100644 --- a/Analysis/include/Luau/Error.h +++ b/Analysis/include/Luau/Error.h @@ -16,7 +16,7 @@ struct TypeMismatch TypeMismatch() = default; TypeMismatch(TypeId wantedType, TypeId givenType); TypeMismatch(TypeId wantedType, TypeId givenType, std::string reason); - TypeMismatch(TypeId wantedType, TypeId givenType, std::string reason, TypeError error); + TypeMismatch(TypeId wantedType, TypeId givenType, std::string reason, std::optional error); TypeId wantedType = nullptr; TypeId givenType = nullptr; diff --git a/Analysis/include/Luau/Frontend.h b/Analysis/include/Luau/Frontend.h index 5df6f4b5..b2662c68 100644 --- a/Analysis/include/Luau/Frontend.h +++ b/Analysis/include/Luau/Frontend.h @@ -83,8 +83,13 @@ struct FrontendOptions // is complete. bool retainFullTypeGraphs = false; - // Run typechecking only in mode required for autocomplete (strict mode in order to get more precise type information) + // Run typechecking only in mode required for autocomplete (strict mode in + // order to get more precise type information) bool forAutocomplete = false; + + // If not empty, randomly shuffle the constraint set before attempting to + // solve. Use this value to seed the random number generator. + std::optional randomizeConstraintResolutionSeed; }; struct CheckResult diff --git a/Analysis/include/Luau/Normalize.h b/Analysis/include/Luau/Normalize.h index 8e8b889b..41e50d1b 100644 --- a/Analysis/include/Luau/Normalize.h +++ b/Analysis/include/Luau/Normalize.h @@ -1,9 +1,9 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #pragma once -#include "Luau/Module.h" #include "Luau/NotNull.h" #include "Luau/TypeVar.h" +#include "Luau/UnifierSharedState.h" #include @@ -29,4 +29,231 @@ std::pair normalize( std::pair normalize(TypePackId ty, NotNull module, NotNull singletonTypes, InternalErrorReporter& ice); std::pair normalize(TypePackId ty, const ModulePtr& module, NotNull singletonTypes, InternalErrorReporter& ice); +class TypeIds +{ +private: + std::unordered_set types; + std::vector order; + std::size_t hash = 0; + +public: + using iterator = std::vector::iterator; + using const_iterator = std::vector::const_iterator; + + TypeIds(const TypeIds&) = delete; + TypeIds(TypeIds&&) = default; + TypeIds() = default; + ~TypeIds() = default; + TypeIds& operator=(TypeIds&&) = default; + + void insert(TypeId ty); + /// Erase every element that does not also occur in tys + void retain(const TypeIds& tys); + void clear(); + + iterator begin(); + iterator end(); + const_iterator begin() const; + const_iterator end() const; + iterator erase(const_iterator it); + + size_t size() const; + bool empty() const; + size_t count(TypeId ty) const; + + template + void insert(Iterator begin, Iterator end) + { + for (Iterator it = begin; it != end; ++it) + insert(*it); + } + + bool operator ==(const TypeIds& there) const; + size_t getHash() const; +}; + +} // namespace Luau + +template<> struct std::hash +{ + std::size_t operator()(const Luau::TypeIds& tys) const + { + return tys.getHash(); + } +}; + +template<> struct std::hash +{ + std::size_t operator()(const Luau::TypeIds* tys) const + { + return tys->getHash(); + } +}; + +template<> struct std::equal_to +{ + bool operator()(const Luau::TypeIds& here, const Luau::TypeIds& there) const + { + return here == there; + } +}; + +template<> struct std::equal_to +{ + bool operator()(const Luau::TypeIds* here, const Luau::TypeIds* there) const + { + return *here == *there; + } +}; + +namespace Luau +{ + +// A normalized string type is either `string` (represented by `nullopt`) +// or a union of string singletons. +using NormalizedStringType = std::optional>; + +// A normalized function type is either `never` (represented by `nullopt`) +// or an intersection of function types. +// NOTE: type normalization can fail on function types with generics +// (e.g. because we do not support unions and intersections of generic type packs), +// so this type may contain `error`. +using NormalizedFunctionType = std::optional; + +// A normalized generic/free type is a union, where each option is of the form (X & T) where +// * X is either a free type or a generic +// * T is a normalized type. +struct NormalizedType; +using NormalizedTyvars = std::unordered_map>; + +// A normalized type is either any, unknown, or one of the form P | T | F | G where +// * P is a union of primitive types (including singletons, classes and the error type) +// * T is a union of table types +// * F is a union of an intersection of function types +// * G is a union of generic/free normalized types, intersected with a normalized type +struct NormalizedType +{ + // The top part of the type. + // This type is either never, unknown, or any. + // If this type is not never, all the other fields are null. + TypeId tops; + + // The boolean part of the type. + // This type is either never, boolean type, or a boolean singleton. + TypeId booleans; + + // The class part of the type. + // Each element of this set is a class, and none of the classes are subclasses of each other. + TypeIds classes; + + // The error part of the type. + // This type is either never or the error type. + TypeId errors; + + // The nil part of the type. + // This type is either never or nil. + TypeId nils; + + // The number part of the type. + // This type is either never or number. + TypeId numbers; + + // The string part of the type. + // This may be the `string` type, or a union of singletons. + NormalizedStringType strings = std::map{}; + + // The thread part of the type. + // This type is either never or thread. + TypeId threads; + + // The (meta)table part of the type. + // Each element of this set is a (meta)table type. + TypeIds tables; + + // The function part of the type. + NormalizedFunctionType functions; + + // The generic/free part of the type. + NormalizedTyvars tyvars; + + NormalizedType(NotNull singletonTypes); + + NormalizedType(const NormalizedType&) = delete; + NormalizedType(NormalizedType&&) = default; + NormalizedType() = delete; + ~NormalizedType() = default; + NormalizedType& operator=(NormalizedType&&) = default; + NormalizedType& operator=(NormalizedType&) = delete; +}; + +class Normalizer +{ + std::unordered_map> cachedNormals; + std::unordered_map cachedIntersections; + std::unordered_map cachedUnions; + std::unordered_map> cachedTypeIds; + bool withinResourceLimits(); + +public: + TypeArena* arena; + NotNull singletonTypes; + NotNull sharedState; + + Normalizer(TypeArena* arena, NotNull singletonTypes, NotNull sharedState); + Normalizer(const Normalizer&) = delete; + Normalizer(Normalizer&&) = delete; + Normalizer() = delete; + ~Normalizer() = default; + Normalizer& operator=(Normalizer&&) = delete; + Normalizer& operator=(Normalizer&) = delete; + + // If this returns null, the typechecker should emit a "too complex" error + const NormalizedType* normalize(TypeId ty); + void clearNormal(NormalizedType& norm); + + // ------- Cached TypeIds + TypeId unionType(TypeId here, TypeId there); + TypeId intersectionType(TypeId here, TypeId there); + const TypeIds* cacheTypeIds(TypeIds tys); + void clearCaches(); + + // ------- Normalizing unions + void unionTysWithTy(TypeIds& here, TypeId there); + TypeId unionOfTops(TypeId here, TypeId there); + TypeId unionOfBools(TypeId here, TypeId there); + void unionClassesWithClass(TypeIds& heres, TypeId there); + void unionClasses(TypeIds& heres, const TypeIds& theres); + void unionStrings(NormalizedStringType& here, const NormalizedStringType& there); + std::optional unionOfTypePacks(TypePackId here, TypePackId there); + std::optional unionOfFunctions(TypeId here, TypeId there); + std::optional unionSaturatedFunctions(TypeId here, TypeId there); + void unionFunctionsWithFunction(NormalizedFunctionType& heress, TypeId there); + void unionFunctions(NormalizedFunctionType& heress, const NormalizedFunctionType& theress); + void unionTablesWithTable(TypeIds& heres, TypeId there); + void unionTables(TypeIds& heres, const TypeIds& theres); + bool unionNormals(NormalizedType& here, const NormalizedType& there, int ignoreSmallerTyvars = -1); + bool unionNormalWithTy(NormalizedType& here, TypeId there, int ignoreSmallerTyvars = -1); + + // ------- Normalizing intersections + void intersectTysWithTy(TypeIds& here, TypeId there); + TypeId intersectionOfTops(TypeId here, TypeId there); + TypeId intersectionOfBools(TypeId here, TypeId there); + void intersectClasses(TypeIds& heres, const TypeIds& theres); + void intersectClassesWithClass(TypeIds& heres, TypeId there); + void intersectStrings(NormalizedStringType& here, const NormalizedStringType& there); + std::optional intersectionOfTypePacks(TypePackId here, TypePackId there); + std::optional intersectionOfTables(TypeId here, TypeId there); + void intersectTablesWithTable(TypeIds& heres, TypeId there); + void intersectTables(TypeIds& heres, const TypeIds& theres); + std::optional intersectionOfFunctions(TypeId here, TypeId there); + void intersectFunctionsWithFunction(NormalizedFunctionType& heress, TypeId there); + void intersectFunctions(NormalizedFunctionType& heress, const NormalizedFunctionType& theress); + bool intersectTyvarsWithTy(NormalizedTyvars& here, TypeId there); + bool intersectNormals(NormalizedType& here, const NormalizedType& there, int ignoreSmallerTyvars = -1); + bool intersectNormalWithTy(NormalizedType& here, TypeId there); + + // -------- Convert back from a normalized type to a type + TypeId typeFromNormal(const NormalizedType& norm); +}; + } // namespace Luau diff --git a/Analysis/include/Luau/TypeInfer.h b/Analysis/include/Luau/TypeInfer.h index e5675ebb..3184b0d3 100644 --- a/Analysis/include/Luau/TypeInfer.h +++ b/Analysis/include/Luau/TypeInfer.h @@ -234,6 +234,8 @@ public: TypeId anyify(const ScopePtr& scope, TypeId ty, Location location); TypePackId anyify(const ScopePtr& scope, TypePackId ty, Location location); + TypePackId anyifyModuleReturnTypePackGenerics(TypePackId ty); + void reportError(const TypeError& error); void reportError(const Location& location, TypeErrorData error); void reportErrors(const ErrorVec& errors); @@ -359,6 +361,7 @@ public: InternalErrorReporter* iceHandler; UnifierSharedState unifierState; + Normalizer normalizer; std::vector requireCycles; diff --git a/Analysis/include/Luau/Unifiable.h b/Analysis/include/Luau/Unifiable.h index 0ea175cc..c43daa21 100644 --- a/Analysis/include/Luau/Unifiable.h +++ b/Analysis/include/Luau/Unifiable.h @@ -96,7 +96,7 @@ struct Free bool forwardedTypeAlias = false; private: - static int nextIndex; + static int DEPRECATED_nextIndex; }; template @@ -127,7 +127,7 @@ struct Generic bool explicitName = false; private: - static int nextIndex; + static int DEPRECATED_nextIndex; }; struct Error diff --git a/Analysis/include/Luau/Unifier.h b/Analysis/include/Luau/Unifier.h index 26a922f5..f6219dfb 100644 --- a/Analysis/include/Luau/Unifier.h +++ b/Analysis/include/Luau/Unifier.h @@ -9,6 +9,7 @@ #include "Luau/TxnLog.h" #include "Luau/TypeArena.h" #include "Luau/UnifierSharedState.h" +#include "Normalize.h" #include @@ -52,6 +53,7 @@ struct Unifier { TypeArena* const types; NotNull singletonTypes; + NotNull normalizer; Mode mode; NotNull scope; // const Scope maybe @@ -60,13 +62,14 @@ struct Unifier Location location; Variance variance = Covariant; bool anyIsTop = false; // If true, we consider any to be a top type. If false, it is a familiar but weird mix of top and bottom all at once. + bool normalize; // Normalize unions and intersections if necessary bool useScopes = false; // If true, we use the scope hierarchy rather than TypeLevels CountMismatch::Context ctx = CountMismatch::Arg; UnifierSharedState& sharedState; - Unifier(TypeArena* types, NotNull singletonTypes, Mode mode, NotNull scope, const Location& location, Variance variance, - UnifierSharedState& sharedState, TxnLog* parentLog = nullptr); + Unifier(NotNull normalizer, Mode mode, NotNull scope, const Location& location, Variance variance, + TxnLog* parentLog = nullptr); // Test whether the two type vars unify. Never commits the result. ErrorVec canUnify(TypeId subTy, TypeId superTy); @@ -84,6 +87,7 @@ private: void tryUnifyTypeWithUnion(TypeId subTy, TypeId superTy, const UnionTypeVar* uv, bool cacheEnabled, bool isFunctionCall); void tryUnifyTypeWithIntersection(TypeId subTy, TypeId superTy, const IntersectionTypeVar* uv); void tryUnifyIntersectionWithType(TypeId subTy, const IntersectionTypeVar* uv, TypeId superTy, bool cacheEnabled, bool isFunctionCall); + void tryUnifyNormalizedTypes(TypeId subTy, TypeId superTy, const NormalizedType& subNorm, const NormalizedType& superNorm, std::string reason, std::optional error = std::nullopt); void tryUnifyPrimitives(TypeId subTy, TypeId superTy); void tryUnifySingletons(TypeId subTy, TypeId superTy); void tryUnifyFunctions(TypeId subTy, TypeId superTy, bool isFunctionCall = false); @@ -92,6 +96,8 @@ private: void tryUnifyWithMetatable(TypeId subTy, TypeId superTy, bool reversed); void tryUnifyWithClass(TypeId subTy, TypeId superTy, bool reversed); + TypePackId tryApplyOverloadedFunction(TypeId function, const NormalizedFunctionType& overloads, TypePackId args); + TypeId widen(TypeId ty); TypePackId widen(TypePackId tp); diff --git a/Analysis/src/Autocomplete.cpp b/Analysis/src/Autocomplete.cpp index 1f594fed..224e9440 100644 --- a/Analysis/src/Autocomplete.cpp +++ b/Analysis/src/Autocomplete.cpp @@ -12,8 +12,6 @@ #include #include -LUAU_FASTFLAG(LuauSelfCallAutocompleteFix3) - static const std::unordered_set kStatementStartingKeywords = { "while", "if", "local", "repeat", "function", "do", "for", "return", "break", "continue", "type", "export"}; @@ -139,7 +137,8 @@ static bool checkTypeMatch(TypeId subTy, TypeId superTy, NotNull scope, T { InternalErrorReporter iceReporter; UnifierSharedState unifierState(&iceReporter); - Unifier unifier(typeArena, singletonTypes, Mode::Strict, scope, Location(), Variance::Covariant, unifierState); + Normalizer normalizer{typeArena, singletonTypes, NotNull{&unifierState}}; + Unifier unifier(NotNull{&normalizer}, Mode::Strict, scope, Location(), Variance::Covariant); return unifier.canUnify(subTy, superTy).empty(); } @@ -151,18 +150,6 @@ static TypeCorrectKind checkTypeCorrectKind( NotNull moduleScope{module.getModuleScope().get()}; - auto canUnify = [&typeArena, singletonTypes, moduleScope](TypeId subTy, TypeId superTy) { - LUAU_ASSERT(!FFlag::LuauSelfCallAutocompleteFix3); - - InternalErrorReporter iceReporter; - UnifierSharedState unifierState(&iceReporter); - Unifier unifier(typeArena, singletonTypes, Mode::Strict, moduleScope, Location(), Variance::Covariant, unifierState); - - unifier.tryUnify(subTy, superTy); - bool ok = unifier.errors.empty(); - return ok; - }; - auto typeAtPosition = findExpectedTypeAt(module, node, position); if (!typeAtPosition) @@ -170,30 +157,11 @@ static TypeCorrectKind checkTypeCorrectKind( TypeId expectedType = follow(*typeAtPosition); - auto checkFunctionType = [typeArena, singletonTypes, moduleScope, &canUnify, &expectedType](const FunctionTypeVar* ftv) { - if (FFlag::LuauSelfCallAutocompleteFix3) - { - if (std::optional firstRetTy = first(ftv->retTypes)) - return checkTypeMatch(*firstRetTy, expectedType, moduleScope, typeArena, singletonTypes); + auto checkFunctionType = [typeArena, singletonTypes, moduleScope, &expectedType](const FunctionTypeVar* ftv) { + if (std::optional firstRetTy = first(ftv->retTypes)) + return checkTypeMatch(*firstRetTy, expectedType, moduleScope, typeArena, singletonTypes); - return false; - } - else - { - auto [retHead, retTail] = flatten(ftv->retTypes); - - if (!retHead.empty() && canUnify(retHead.front(), expectedType)) - return true; - - // We might only have a variadic tail pack, check if the element is compatible - if (retTail) - { - if (const VariadicTypePack* vtp = get(follow(*retTail)); vtp && canUnify(vtp->ty, expectedType)) - return true; - } - - return false; - } + return false; }; // We also want to suggest functions that return compatible result @@ -212,11 +180,8 @@ static TypeCorrectKind checkTypeCorrectKind( } } - if (FFlag::LuauSelfCallAutocompleteFix3) - return checkTypeMatch(ty, expectedType, NotNull{module.getModuleScope().get()}, typeArena, singletonTypes) ? TypeCorrectKind::Correct - : TypeCorrectKind::None; - else - return canUnify(ty, expectedType) ? TypeCorrectKind::Correct : TypeCorrectKind::None; + return checkTypeMatch(ty, expectedType, NotNull{module.getModuleScope().get()}, typeArena, singletonTypes) ? TypeCorrectKind::Correct + : TypeCorrectKind::None; } enum class PropIndexType @@ -230,51 +195,14 @@ static void autocompleteProps(const Module& module, TypeArena* typeArena, NotNul PropIndexType indexType, const std::vector& nodes, AutocompleteEntryMap& result, std::unordered_set& seen, std::optional containingClass = std::nullopt) { - if (FFlag::LuauSelfCallAutocompleteFix3) - rootTy = follow(rootTy); - + rootTy = follow(rootTy); ty = follow(ty); if (seen.count(ty)) return; seen.insert(ty); - auto isWrongIndexer_DEPRECATED = [indexType, useStrictFunctionIndexers = !!get(ty)](Luau::TypeId type) { - LUAU_ASSERT(!FFlag::LuauSelfCallAutocompleteFix3); - - if (indexType == PropIndexType::Key) - return false; - - bool colonIndex = indexType == PropIndexType::Colon; - - if (const FunctionTypeVar* ftv = get(type)) - { - return useStrictFunctionIndexers ? colonIndex != ftv->hasSelf : false; - } - else if (const IntersectionTypeVar* itv = get(type)) - { - bool allHaveSelf = true; - for (auto subType : itv->parts) - { - if (const FunctionTypeVar* ftv = get(Luau::follow(subType))) - { - allHaveSelf &= ftv->hasSelf; - } - else - { - return colonIndex; - } - } - return useStrictFunctionIndexers ? colonIndex != allHaveSelf : false; - } - else - { - return colonIndex; - } - }; auto isWrongIndexer = [typeArena, singletonTypes, &module, rootTy, indexType](Luau::TypeId type) { - LUAU_ASSERT(FFlag::LuauSelfCallAutocompleteFix3); - if (indexType == PropIndexType::Key) return false; @@ -337,7 +265,7 @@ static void autocompleteProps(const Module& module, TypeArena* typeArena, NotNul AutocompleteEntryKind::Property, type, prop.deprecated, - FFlag::LuauSelfCallAutocompleteFix3 ? isWrongIndexer(type) : isWrongIndexer_DEPRECATED(type), + isWrongIndexer(type), typeCorrect, containingClass, &prop, @@ -380,31 +308,8 @@ static void autocompleteProps(const Module& module, TypeArena* typeArena, NotNul { autocompleteProps(module, typeArena, singletonTypes, rootTy, mt->table, indexType, nodes, result, seen); - if (FFlag::LuauSelfCallAutocompleteFix3) - { - if (auto mtable = get(mt->metatable)) - fillMetatableProps(mtable); - } - else - { - auto mtable = get(mt->metatable); - if (!mtable) - return; - - auto indexIt = mtable->props.find("__index"); - if (indexIt != mtable->props.end()) - { - TypeId followed = follow(indexIt->second.type); - if (get(followed) || get(followed)) - autocompleteProps(module, typeArena, singletonTypes, rootTy, followed, indexType, nodes, result, seen); - else if (auto indexFunction = get(followed)) - { - std::optional indexFunctionResult = first(indexFunction->retTypes); - if (indexFunctionResult) - autocompleteProps(module, typeArena, singletonTypes, rootTy, *indexFunctionResult, indexType, nodes, result, seen); - } - } - } + if (auto mtable = get(mt->metatable)) + fillMetatableProps(mtable); } else if (auto i = get(ty)) { @@ -446,9 +351,6 @@ static void autocompleteProps(const Module& module, TypeArena* typeArena, NotNul AutocompleteEntryMap inner; std::unordered_set innerSeen; - if (!FFlag::LuauSelfCallAutocompleteFix3) - innerSeen = seen; - if (isNil(*iter)) { ++iter; @@ -472,7 +374,7 @@ static void autocompleteProps(const Module& module, TypeArena* typeArena, NotNul ++iter; } } - else if (auto pt = get(ty); pt && FFlag::LuauSelfCallAutocompleteFix3) + else if (auto pt = get(ty)) { if (pt->metatable) { @@ -480,7 +382,7 @@ static void autocompleteProps(const Module& module, TypeArena* typeArena, NotNul fillMetatableProps(mtable); } } - else if (FFlag::LuauSelfCallAutocompleteFix3 && get(get(ty))) + else if (get(get(ty))) { autocompleteProps(module, typeArena, singletonTypes, rootTy, singletonTypes->stringType, indexType, nodes, result, seen); } @@ -1416,11 +1318,7 @@ static AutocompleteResult autocomplete(const SourceModule& sourceModule, const M TypeId ty = follow(*it); PropIndexType indexType = indexName->op == ':' ? PropIndexType::Colon : PropIndexType::Point; - if (!FFlag::LuauSelfCallAutocompleteFix3 && isString(ty)) - return {autocompleteProps(*module, &typeArena, singletonTypes, globalScope->bindings[AstName{"string"}].typeId, indexType, ancestry), - ancestry, AutocompleteContext::Property}; - else - return {autocompleteProps(*module, &typeArena, singletonTypes, ty, indexType, ancestry), ancestry, AutocompleteContext::Property}; + return {autocompleteProps(*module, &typeArena, singletonTypes, ty, indexType, ancestry), ancestry, AutocompleteContext::Property}; } else if (auto typeReference = node->as()) { diff --git a/Analysis/src/ConstraintSolver.cpp b/Analysis/src/ConstraintSolver.cpp index 35b8387f..5b3ec03c 100644 --- a/Analysis/src/ConstraintSolver.cpp +++ b/Analysis/src/ConstraintSolver.cpp @@ -14,6 +14,8 @@ #include "Luau/VisitTypeVar.h" #include "Luau/TypeUtils.h" +#include + LUAU_FASTFLAGVARIABLE(DebugLuauLogSolver, false); LUAU_FASTFLAGVARIABLE(DebugLuauLogSolverToJson, false); LUAU_FASTFLAG(LuauFixNameMaps) @@ -251,10 +253,11 @@ void dump(ConstraintSolver* cs, ToStringOptions& opts) } } -ConstraintSolver::ConstraintSolver(TypeArena* arena, NotNull singletonTypes, NotNull rootScope, ModuleName moduleName, +ConstraintSolver::ConstraintSolver(NotNull normalizer, NotNull rootScope, ModuleName moduleName, NotNull moduleResolver, std::vector requireCycles, DcrLogger* logger) - : arena(arena) - , singletonTypes(singletonTypes) + : arena(normalizer->arena) + , singletonTypes(normalizer->singletonTypes) + , normalizer(normalizer) , constraints(collectConstraints(rootScope)) , rootScope(rootScope) , currentModuleName(std::move(moduleName)) @@ -278,6 +281,12 @@ ConstraintSolver::ConstraintSolver(TypeArena* arena, NotNull sin LUAU_ASSERT(logger); } +void ConstraintSolver::randomize(unsigned seed) +{ + std::mt19937 g(seed); + std::shuffle(begin(unsolvedConstraints), end(unsolvedConstraints), g); +} + void ConstraintSolver::run() { if (done()) @@ -1355,8 +1364,7 @@ bool ConstraintSolver::isBlocked(NotNull constraint) void ConstraintSolver::unify(TypeId subType, TypeId superType, NotNull scope) { - UnifierSharedState sharedState{&iceReporter}; - Unifier u{arena, singletonTypes, Mode::Strict, scope, Location{}, Covariant, sharedState}; + Unifier u{normalizer, Mode::Strict, scope, Location{}, Covariant}; u.useScopes = true; u.tryUnify(subType, superType); @@ -1379,7 +1387,7 @@ void ConstraintSolver::unify(TypeId subType, TypeId superType, NotNull sc void ConstraintSolver::unify(TypePackId subPack, TypePackId superPack, NotNull scope) { UnifierSharedState sharedState{&iceReporter}; - Unifier u{arena, singletonTypes, Mode::Strict, scope, Location{}, Covariant, sharedState}; + Unifier u{normalizer, Mode::Strict, scope, Location{}, Covariant}; u.useScopes = true; u.tryUnify(subPack, superPack); diff --git a/Analysis/src/Error.cpp b/Analysis/src/Error.cpp index aa496ee4..d13e26c0 100644 --- a/Analysis/src/Error.cpp +++ b/Analysis/src/Error.cpp @@ -511,11 +511,11 @@ TypeMismatch::TypeMismatch(TypeId wantedType, TypeId givenType, std::string reas { } -TypeMismatch::TypeMismatch(TypeId wantedType, TypeId givenType, std::string reason, TypeError error) +TypeMismatch::TypeMismatch(TypeId wantedType, TypeId givenType, std::string reason, std::optional error) : wantedType(wantedType) , givenType(givenType) , reason(reason) - , error(std::make_shared(std::move(error))) + , error(error ? std::make_shared(std::move(*error)) : nullptr) { } diff --git a/Analysis/src/Frontend.cpp b/Analysis/src/Frontend.cpp index 1890e081..5705ac17 100644 --- a/Analysis/src/Frontend.cpp +++ b/Analysis/src/Frontend.cpp @@ -860,12 +860,18 @@ ModulePtr Frontend::check( const NotNull mr{forAutocomplete ? &moduleResolverForAutocomplete : &moduleResolver}; const ScopePtr& globalScope{forAutocomplete ? typeCheckerForAutocomplete.globalScope : typeChecker.globalScope}; + Normalizer normalizer{&result->internalTypes, singletonTypes, NotNull{&typeChecker.unifierState}}; + ConstraintGraphBuilder cgb{ sourceModule.name, result, &result->internalTypes, mr, singletonTypes, NotNull(&iceHandler), globalScope, logger.get()}; cgb.visit(sourceModule.root); result->errors = std::move(cgb.errors); - ConstraintSolver cs{&result->internalTypes, singletonTypes, NotNull(cgb.rootScope), sourceModule.name, NotNull(&moduleResolver), requireCycles, logger.get()}; + ConstraintSolver cs{NotNull{&normalizer}, NotNull(cgb.rootScope), sourceModule.name, NotNull(&moduleResolver), requireCycles, logger.get()}; + + if (options.randomizeConstraintResolutionSeed) + cs.randomize(*options.randomizeConstraintResolutionSeed); + cs.run(); for (TypeError& e : cs.errors) diff --git a/Analysis/src/Module.cpp b/Analysis/src/Module.cpp index b9deac76..45eb87d6 100644 --- a/Analysis/src/Module.cpp +++ b/Analysis/src/Module.cpp @@ -14,6 +14,7 @@ #include +LUAU_FASTFLAG(LuauAnyifyModuleReturnGenerics) LUAU_FASTFLAG(LuauLowerBoundsCalculation); LUAU_FASTFLAG(DebugLuauDeferredConstraintResolution); LUAU_FASTFLAGVARIABLE(LuauForceExportSurfacesToBeNormal, false); @@ -285,13 +286,16 @@ void Module::clonePublicInterface(NotNull singletonTypes, Intern } } - for (TypeId ty : returnType) + if (!FFlag::LuauAnyifyModuleReturnGenerics) { - if (get(follow(ty))) + for (TypeId ty : returnType) { - auto t = asMutable(ty); - t->ty = AnyTypeVar{}; - t->normal = true; + if (get(follow(ty))) + { + auto t = asMutable(ty); + t->ty = AnyTypeVar{}; + t->normal = true; + } } } diff --git a/Analysis/src/Normalize.cpp b/Analysis/src/Normalize.cpp index 42f61517..c008bcfc 100644 --- a/Analysis/src/Normalize.cpp +++ b/Analysis/src/Normalize.cpp @@ -1,6 +1,7 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #include "Luau/Normalize.h" +#include "Luau/ToString.h" #include @@ -10,15 +11,1703 @@ #include "Luau/VisitTypeVar.h" LUAU_FASTFLAGVARIABLE(DebugLuauCopyBeforeNormalizing, false) +LUAU_FASTFLAGVARIABLE(DebugLuauCheckNormalizeInvariant, false) // This could theoretically be 2000 on amd64, but x86 requires this. LUAU_FASTINTVARIABLE(LuauNormalizeIterationLimit, 1200); +LUAU_FASTINTVARIABLE(LuauNormalizeCacheLimit, 100000); +LUAU_FASTINT(LuauTypeInferRecursionLimit); LUAU_FASTFLAGVARIABLE(LuauNormalizeCombineTableFix, false); +LUAU_FASTFLAGVARIABLE(LuauTypeNormalization2, false); LUAU_FASTFLAG(LuauUnknownAndNeverType) LUAU_FASTFLAG(DebugLuauDeferredConstraintResolution) namespace Luau { +void TypeIds::insert(TypeId ty) +{ + ty = follow(ty); + auto [_, fresh] = types.insert(ty); + if (fresh) + { + order.push_back(ty); + hash ^= std::hash{}(ty); + } +} + +void TypeIds::clear() +{ + order.clear(); + types.clear(); + hash = 0; +} + +TypeIds::iterator TypeIds::begin() +{ + return order.begin(); +} + +TypeIds::iterator TypeIds::end() +{ + return order.end(); +} + +TypeIds::const_iterator TypeIds::begin() const +{ + return order.begin(); +} + +TypeIds::const_iterator TypeIds::end() const +{ + return order.end(); +} + +TypeIds::iterator TypeIds::erase(TypeIds::const_iterator it) +{ + TypeId ty = *it; + types.erase(ty); + hash ^= std::hash{}(ty); + return order.erase(it); +} + +size_t TypeIds::size() const +{ + return types.size(); +} + +bool TypeIds::empty() const +{ + return types.empty(); +} + +size_t TypeIds::count(TypeId ty) const +{ + ty = follow(ty); + return types.count(ty); +} + +void TypeIds::retain(const TypeIds& there) +{ + for (auto it = begin(); it != end();) + { + if (there.count(*it)) + it++; + else + it = erase(it); + } +} + +size_t TypeIds::getHash() const +{ + return hash; +} + +bool TypeIds::operator==(const TypeIds& there) const +{ + return hash == there.hash && types == there.types; +} + +NormalizedType::NormalizedType(NotNull singletonTypes) + : tops(singletonTypes->neverType) + , booleans(singletonTypes->neverType) + , errors(singletonTypes->neverType) + , nils(singletonTypes->neverType) + , numbers(singletonTypes->neverType) + , threads(singletonTypes->neverType) +{ +} + +static bool isInhabited(const NormalizedType& norm) +{ + return !get(norm.tops) + || !get(norm.booleans) + || !norm.classes.empty() + || !get(norm.errors) + || !get(norm.nils) + || !get(norm.numbers) + || !norm.strings || !norm.strings->empty() + || !get(norm.threads) + || norm.functions + || !norm.tables.empty() + || !norm.tyvars.empty(); +} + +static int tyvarIndex(TypeId ty) +{ + if (const GenericTypeVar* gtv = get(ty)) + return gtv->index; + else if (const FreeTypeVar* ftv = get(ty)) + return ftv->index; + else + return 0; +} + +#ifdef LUAU_ASSERTENABLED + +static bool isNormalizedTop(TypeId ty) +{ + return get(ty) || get(ty) || get(ty); +} + +static bool isNormalizedBoolean(TypeId ty) +{ + if (get(ty)) + return true; + else if (const PrimitiveTypeVar* ptv = get(ty)) + return ptv->type == PrimitiveTypeVar::Boolean; + else if (const SingletonTypeVar* stv = get(ty)) + return get(stv); + else + return false; +} + +static bool isNormalizedError(TypeId ty) +{ + if (get(ty) || get(ty)) + return true; + else + return false; +} + +static bool isNormalizedNil(TypeId ty) +{ + if (get(ty)) + return true; + else if (const PrimitiveTypeVar* ptv = get(ty)) + return ptv->type == PrimitiveTypeVar::NilType; + else + return false; +} + +static bool isNormalizedNumber(TypeId ty) +{ + if (get(ty)) + return true; + else if (const PrimitiveTypeVar* ptv = get(ty)) + return ptv->type == PrimitiveTypeVar::Number; + else + return false; +} + +static bool isNormalizedString(const NormalizedStringType& ty) +{ + if (!ty) + return true; + + for (auto& [str, ty] : *ty) + { + if (const SingletonTypeVar* stv = get(ty)) + { + if (const StringSingleton* sstv = get(stv)) + { + if (sstv->value != str) + return false; + } + else + return false; + } + else + return false; + } + + return true; +} + +static bool isNormalizedThread(TypeId ty) +{ + if (get(ty)) + return true; + else if (const PrimitiveTypeVar* ptv = get(ty)) + return ptv->type == PrimitiveTypeVar::Thread; + else + return false; +} + +static bool areNormalizedFunctions(const NormalizedFunctionType& tys) +{ + if (tys) + for (TypeId ty : *tys) + if (!get(ty) && !get(ty)) + return false; + return true; +} + +static bool areNormalizedTables(const TypeIds& tys) +{ + for (TypeId ty : tys) + if (!get(ty) && !get(ty)) + return false; + return true; +} + +static bool areNormalizedClasses(const TypeIds& tys) +{ + for (TypeId ty : tys) + if (!get(ty)) + return false; + return true; +} + +static bool isPlainTyvar(TypeId ty) +{ + return (get(ty) || get(ty)); +} + +static bool isNormalizedTyvar(const NormalizedTyvars& tyvars) +{ + for (auto& [tyvar, intersect] : tyvars) + { + if (!isPlainTyvar(tyvar)) + return false; + if (!isInhabited(*intersect)) + return false; + for (auto& [other, _] : intersect->tyvars) + if (tyvarIndex(other) <= tyvarIndex(tyvar)) + return false; + } + return true; +} + +#endif // LUAU_ASSERTENABLED + +static void assertInvariant(const NormalizedType& norm) +{ + #ifdef LUAU_ASSERTENABLED + if (!FFlag::DebugLuauCheckNormalizeInvariant) + return; + + LUAU_ASSERT(isNormalizedTop(norm.tops)); + LUAU_ASSERT(isNormalizedBoolean(norm.booleans)); + LUAU_ASSERT(areNormalizedClasses(norm.classes)); + LUAU_ASSERT(isNormalizedError(norm.errors)); + LUAU_ASSERT(isNormalizedNil(norm.nils)); + LUAU_ASSERT(isNormalizedNumber(norm.numbers)); + LUAU_ASSERT(isNormalizedString(norm.strings)); + LUAU_ASSERT(isNormalizedThread(norm.threads)); + LUAU_ASSERT(areNormalizedFunctions(norm.functions)); + LUAU_ASSERT(areNormalizedTables(norm.tables)); + LUAU_ASSERT(isNormalizedTyvar(norm.tyvars)); + for (auto& [_, child] : norm.tyvars) + assertInvariant(*child); + #endif +} + +Normalizer::Normalizer(TypeArena* arena, NotNull singletonTypes, NotNull sharedState) + : arena(arena) + , singletonTypes(singletonTypes) + , sharedState(sharedState) +{ +} + +const NormalizedType* Normalizer::normalize(TypeId ty) +{ + if (!arena) + sharedState->iceHandler->ice("Normalizing types outside a module"); + + auto found = cachedNormals.find(ty); + if (found != cachedNormals.end()) + return found->second.get(); + + NormalizedType norm{singletonTypes}; + if (!unionNormalWithTy(norm, ty)) + return nullptr; + std::unique_ptr uniq = std::make_unique(std::move(norm)); + const NormalizedType* result = uniq.get(); + cachedNormals[ty] = std::move(uniq); + return result; +} + +void Normalizer::clearNormal(NormalizedType& norm) +{ + norm.tops = singletonTypes->neverType; + norm.booleans = singletonTypes->neverType; + norm.classes.clear(); + norm.errors = singletonTypes->neverType; + norm.nils = singletonTypes->neverType; + norm.numbers = singletonTypes->neverType; + if (norm.strings) + norm.strings->clear(); + else + norm.strings.emplace(); + norm.threads = singletonTypes->neverType; + norm.tables.clear(); + norm.functions = std::nullopt; + norm.tyvars.clear(); +} + +// ------- Cached TypeIds +const TypeIds* Normalizer::cacheTypeIds(TypeIds tys) +{ + auto found = cachedTypeIds.find(&tys); + if (found != cachedTypeIds.end()) + return found->first; + + std::unique_ptr uniq = std::make_unique(std::move(tys)); + const TypeIds* result = uniq.get(); + cachedTypeIds[result] = std::move(uniq); + return result; +} + +TypeId Normalizer::unionType(TypeId here, TypeId there) +{ + here = follow(here); + there = follow(there); + + if (here == there) + return here; + if (get(here) || get(there)) + return there; + if (get(there) || get(here)) + return here; + + TypeIds tmps; + + if (const UnionTypeVar* utv = get(here)) + { + TypeIds heres; + heres.insert(begin(utv), end(utv)); + tmps.insert(heres.begin(), heres.end()); + cachedUnions[cacheTypeIds(std::move(heres))] = here; + } + else + tmps.insert(here); + + if (const UnionTypeVar* utv = get(there)) + { + TypeIds theres; + theres.insert(begin(utv), end(utv)); + tmps.insert(theres.begin(), theres.end()); + cachedUnions[cacheTypeIds(std::move(theres))] = there; + } + else + tmps.insert(there); + + auto cacheHit = cachedUnions.find(&tmps); + if (cacheHit != cachedUnions.end()) + return cacheHit->second; + + std::vector parts; + parts.insert(parts.end(), tmps.begin(), tmps.end()); + TypeId result = arena->addType(UnionTypeVar{std::move(parts)}); + cachedUnions[cacheTypeIds(std::move(tmps))] = result; + + return result; +} + +TypeId Normalizer::intersectionType(TypeId here, TypeId there) +{ + here = follow(here); + there = follow(there); + + if (here == there) + return here; + if (get(here) || get(there)) + return here; + if (get(there) || get(here)) + return there; + + TypeIds tmps; + + if (const IntersectionTypeVar* utv = get(here)) + { + TypeIds heres; + heres.insert(begin(utv), end(utv)); + tmps.insert(heres.begin(), heres.end()); + cachedIntersections[cacheTypeIds(std::move(heres))] = here; + } + else + tmps.insert(here); + + if (const IntersectionTypeVar* utv = get(there)) + { + TypeIds theres; + theres.insert(begin(utv), end(utv)); + tmps.insert(theres.begin(), theres.end()); + cachedIntersections[cacheTypeIds(std::move(theres))] = there; + } + else + tmps.insert(there); + + if (tmps.size() == 1) + return *tmps.begin(); + + auto cacheHit = cachedIntersections.find(&tmps); + if (cacheHit != cachedIntersections.end()) + return cacheHit->second; + + std::vector parts; + parts.insert(parts.end(), tmps.begin(), tmps.end()); + TypeId result = arena->addType(IntersectionTypeVar{std::move(parts)}); + cachedIntersections[cacheTypeIds(std::move(tmps))] = result; + + return result; +} + +void Normalizer::clearCaches() +{ + cachedNormals.clear(); + cachedIntersections.clear(); + cachedUnions.clear(); + cachedTypeIds.clear(); +} + +// ------- Normalizing unions +TypeId Normalizer::unionOfTops(TypeId here, TypeId there) +{ + if (get(here) || get(there)) + return there; + else + return here; +} + +TypeId Normalizer::unionOfBools(TypeId here, TypeId there) +{ + if (get(here)) + return there; + if (get(there)) + return here; + if (const BooleanSingleton* hbool = get(get(here))) + if (const BooleanSingleton* tbool = get(get(there))) + if (hbool->value == tbool->value) + return here; + return singletonTypes->booleanType; +} + +void Normalizer::unionClassesWithClass(TypeIds& heres, TypeId there) +{ + if (heres.count(there)) + return; + + const ClassTypeVar* tctv = get(there); + + for (auto it = heres.begin(); it != heres.end();) + { + TypeId here = *it; + const ClassTypeVar* hctv = get(here); + if (isSubclass(tctv, hctv)) + return; + else if (isSubclass(hctv, tctv)) + it = heres.erase(it); + else + it++; + } + + heres.insert(there); +} + +void Normalizer::unionClasses(TypeIds& heres, const TypeIds& theres) +{ + for (TypeId there : theres) + unionClassesWithClass(heres, there); +} + +void Normalizer::unionStrings(NormalizedStringType& here, const NormalizedStringType& there) +{ + if (!there) + here.reset(); + else if (here) + here->insert(there->begin(), there->end()); +} + +std::optional Normalizer::unionOfTypePacks(TypePackId here, TypePackId there) +{ + if (here == there) + return here; + + std::vector head; + std::optional tail; + + bool hereSubThere = true; + bool thereSubHere = true; + + TypePackIterator ith = begin(here); + TypePackIterator itt = begin(there); + + while (ith != end(here) && itt != end(there)) + { + TypeId hty = *ith; + TypeId tty = *itt; + TypeId ty = unionType(hty, tty); + if (ty != hty) + thereSubHere = false; + if (ty != tty) + hereSubThere = false; + head.push_back(ty); + ith++; + itt++; + } + + auto dealWithDifferentArities = [&](TypePackIterator& ith, TypePackIterator itt, TypePackId here, TypePackId there, bool& hereSubThere, bool& thereSubHere) + { + if (ith != end(here)) + { + TypeId tty = singletonTypes->nilType; + if (std::optional ttail = itt.tail()) + { + if (const VariadicTypePack* tvtp = get(*ttail)) + tty = tvtp->ty; + else + // Luau doesn't have unions of type pack variables + return false; + } + else + // Type packs of different arities are incomparable + return false; + + while (ith != end(here)) + { + TypeId hty = *ith; + TypeId ty = unionType(hty, tty); + if (ty != hty) + thereSubHere = false; + if (ty != tty) + hereSubThere = false; + head.push_back(ty); + ith++; + } + } + return true; + }; + + if (!dealWithDifferentArities(ith, itt, here, there, hereSubThere, thereSubHere)) + return std::nullopt; + + if (!dealWithDifferentArities(itt, ith, there, here, thereSubHere, hereSubThere)) + return std::nullopt; + + if (std::optional htail = ith.tail()) + { + if (std::optional ttail = itt.tail()) + { + if (*htail == *ttail) + tail = htail; + else if (const VariadicTypePack* hvtp = get(*htail)) + { + if (const VariadicTypePack* tvtp = get(*ttail)) + { + TypeId ty = unionType(hvtp->ty, tvtp->ty); + if (ty != hvtp->ty) + thereSubHere = false; + if (ty != tvtp->ty) + hereSubThere = false; + bool hidden = hvtp->hidden & tvtp->hidden; + tail = arena->addTypePack(VariadicTypePack{ty,hidden}); + } + else + // Luau doesn't have unions of type pack variables + return std::nullopt; + } + else + // Luau doesn't have unions of type pack variables + return std::nullopt; + } + else if (get(*htail)) + { + hereSubThere = false; + tail = htail; + } + else + // Luau doesn't have unions of type pack variables + return std::nullopt; + } + else if (std::optional ttail = itt.tail()) + { + if (get(*ttail)) + { + thereSubHere = false; + tail = htail; + } + else + // Luau doesn't have unions of type pack variables + return std::nullopt; + } + + if (hereSubThere) + return there; + else if (thereSubHere) + return here; + if (!head.empty()) + return arena->addTypePack(TypePack{head,tail}); + else if (tail) + return *tail; + else + // TODO: Add an emptyPack to singleton types + return arena->addTypePack({}); +} + +std::optional Normalizer::unionOfFunctions(TypeId here, TypeId there) +{ + if (get(here)) + return here; + + if (get(there)) + return there; + + const FunctionTypeVar* hftv = get(here); + LUAU_ASSERT(hftv); + const FunctionTypeVar* tftv = get(there); + LUAU_ASSERT(tftv); + + if (hftv->generics != tftv->generics) + return std::nullopt; + if (hftv->genericPacks != tftv->genericPacks) + return std::nullopt; + + std::optional argTypes = intersectionOfTypePacks(hftv->argTypes, tftv->argTypes); + if (!argTypes) + return std::nullopt; + + std::optional retTypes = unionOfTypePacks(hftv->retTypes, tftv->retTypes); + if (!retTypes) + return std::nullopt; + + if (*argTypes == hftv->argTypes && *retTypes == hftv->retTypes) + return here; + if (*argTypes == tftv->argTypes && *retTypes == tftv->retTypes) + return there; + + FunctionTypeVar result{*argTypes, *retTypes}; + result.generics = hftv->generics; + result.genericPacks = hftv->genericPacks; + return arena->addType(std::move(result)); +} + +void Normalizer::unionFunctions(NormalizedFunctionType& heres, const NormalizedFunctionType& theres) +{ + if (!theres) + return; + + TypeIds tmps; + + if (!heres) + { + tmps.insert(theres->begin(), theres->end()); + heres = std::move(tmps); + return; + } + + for (TypeId here : *heres) + for (TypeId there : *theres) + { + if (std::optional fun = unionOfFunctions(here, there)) + tmps.insert(*fun); + else + tmps.insert(singletonTypes->errorRecoveryType(there)); + } + + heres = std::move(tmps); +} + +void Normalizer::unionFunctionsWithFunction(NormalizedFunctionType& heres, TypeId there) +{ + if (!heres) + { + TypeIds tmps; + tmps.insert(there); + heres = std::move(tmps); + return; + } + + TypeIds tmps; + for (TypeId here : *heres) + { + if (std::optional fun = unionOfFunctions(here, there)) + tmps.insert(*fun); + else + tmps.insert(singletonTypes->errorRecoveryType(there)); + } + heres = std::move(tmps); +} + +void Normalizer::unionTablesWithTable(TypeIds& heres, TypeId there) +{ + // TODO: remove unions of tables where possible + heres.insert(there); +} + +void Normalizer::unionTables(TypeIds& heres, const TypeIds& theres) +{ + for (TypeId there : theres) + unionTablesWithTable(heres, there); +} + +// So why `ignoreSmallerTyvars`? +// +// First up, what it does... Every tyvar has an index, and this parameter says to ignore +// any tyvars in `there` if their index is less than or equal to the parameter. +// The parameter is always greater than any tyvars mentioned in here, so the result is +// a lower bound on any tyvars in `here.tyvars`. +// +// This is used to maintain in invariant, which is that in any tyvar `X&T`, any any tyvar +// `Y&U` in `T`, the index of `X` is less than the index of `Y`. This is an implementation +// of *ordered decision diagrams* (https://en.wikipedia.org/wiki/Binary_decision_diagram#Variable_ordering) +// which are a compression technique used to save memory usage when representing boolean formulae. +// +// The idea is that if you have an out-of-order decision diagram +// like `Z&(X|Y)`, to re-order it in this case to `(X&Z)|(Y&Z)`. +// The hope is that by imposing a global order, there's a higher chance of sharing opportunities, +// and hence reduced memory. +// +// And yes, this is essentially a SAT solver hidden inside a typechecker. +// That's what you get for having a type system with generics, intersection and union types. +bool Normalizer::unionNormals(NormalizedType& here, const NormalizedType& there, int ignoreSmallerTyvars) +{ + TypeId tops = unionOfTops(here.tops, there.tops); + if (!get(tops)) + { + clearNormal(here); + here.tops = tops; + return true; + } + + for (auto it = there.tyvars.begin(); it != there.tyvars.end(); it++) + { + TypeId tyvar = it->first; + const NormalizedType& inter = *it->second; + int index = tyvarIndex(tyvar); + if (index <= ignoreSmallerTyvars) + continue; + auto [emplaced, fresh] = here.tyvars.emplace(tyvar, std::make_unique(NormalizedType{singletonTypes})); + if (fresh) + if (!unionNormals(*emplaced->second, here, index)) + return false; + if (!unionNormals(*emplaced->second, inter, index)) + return false; + } + + here.booleans = unionOfBools(here.booleans, there.booleans); + unionClasses(here.classes, there.classes); + here.errors = (get(there.errors) ? here.errors : there.errors); + here.nils = (get(there.nils) ? here.nils : there.nils); + here.numbers = (get(there.numbers) ? here.numbers : there.numbers); + unionStrings(here.strings, there.strings); + here.threads = (get(there.threads) ? here.threads : there.threads); + unionFunctions(here.functions, there.functions); + unionTables(here.tables, there.tables); + return true; +} + +bool Normalizer::withinResourceLimits() +{ + // If cache is too large, clear it + if (FInt::LuauNormalizeCacheLimit > 0) + { + size_t cacheUsage = cachedNormals.size() + cachedIntersections.size() + cachedUnions.size() + cachedTypeIds.size(); + if (cacheUsage > size_t(FInt::LuauNormalizeCacheLimit)) + { + clearCaches(); + return false; + } + } + + // Check the recursion count + if (sharedState->counters.recursionLimit > 0) + if (sharedState->counters.recursionLimit < sharedState->counters.recursionCount) + return false; + + return true; +} + +// See above for an explaination of `ignoreSmallerTyvars`. +bool Normalizer::unionNormalWithTy(NormalizedType& here, TypeId there, int ignoreSmallerTyvars) +{ + RecursionCounter _rc(&sharedState->counters.recursionCount); + if (!withinResourceLimits()) + return false; + + there = follow(there); + if (get(there) || get(there)) + { + TypeId tops = unionOfTops(here.tops, there); + clearNormal(here); + here.tops = tops; + return true; + } + else if (get(there) || !get(here.tops)) + return true; + else if (const UnionTypeVar* utv = get(there)) + { + for (UnionTypeVarIterator it = begin(utv); it != end(utv); ++it) + if (!unionNormalWithTy(here, *it)) + return false; + return true; + } + else if (const IntersectionTypeVar* itv = get(there)) + { + NormalizedType norm{singletonTypes}; + norm.tops = singletonTypes->anyType; + for (IntersectionTypeVarIterator it = begin(itv); it != end(itv); ++it) + if (!intersectNormalWithTy(norm, *it)) + return false; + return unionNormals(here, norm); + } + else if (get(there) || get(there)) + { + if (tyvarIndex(there) <= ignoreSmallerTyvars) + return true; + NormalizedType inter{singletonTypes}; + inter.tops = singletonTypes->unknownType; + here.tyvars.insert_or_assign(there, std::make_unique(std::move(inter))); + } + else if (get(there)) + unionFunctionsWithFunction(here.functions, there); + else if (get(there) || get(there)) + unionTablesWithTable(here.tables, there); + else if (get(there)) + unionClassesWithClass(here.classes, there); + else if (get(there)) + here.errors = there; + else if (const PrimitiveTypeVar* ptv = get(there)) + { + if (ptv->type == PrimitiveTypeVar::Boolean) + here.booleans = there; + else if (ptv->type == PrimitiveTypeVar::NilType) + here.nils = there; + else if (ptv->type == PrimitiveTypeVar::Number) + here.numbers = there; + else if (ptv->type == PrimitiveTypeVar::String) + here.strings = std::nullopt; + else if (ptv->type == PrimitiveTypeVar::Thread) + here.threads = there; + else + LUAU_ASSERT(!"Unreachable"); + } + else if (const SingletonTypeVar* stv = get(there)) + { + if (get(stv)) + here.booleans = unionOfBools(here.booleans, there); + else if (const StringSingleton* sstv = get(stv)) + { + if (here.strings) + here.strings->insert({sstv->value, there}); + } + else + LUAU_ASSERT(!"Unreachable"); + } + else + LUAU_ASSERT(!"Unreachable"); + + for (auto& [tyvar, intersect] : here.tyvars) + if (!unionNormalWithTy(*intersect, there, tyvarIndex(tyvar))) + return false; + + assertInvariant(here); + return true; +} + +// ------- Normalizing intersections +TypeId Normalizer::intersectionOfTops(TypeId here, TypeId there) +{ + if (get(here) || get(there)) + return here; + else + return there; +} + +TypeId Normalizer::intersectionOfBools(TypeId here, TypeId there) +{ + if (get(here)) + return here; + if (get(there)) + return there; + if (const BooleanSingleton* hbool = get(get(here))) + if (const BooleanSingleton* tbool = get(get(there))) + return (hbool->value == tbool->value ? here : singletonTypes->neverType); + else + return here; + else + return there; +} + +void Normalizer::intersectClasses(TypeIds& heres, const TypeIds& theres) +{ + TypeIds tmp; + for (auto it = heres.begin(); it != heres.end();) + { + const ClassTypeVar* hctv = get(*it); + LUAU_ASSERT(hctv); + bool keep = false; + for (TypeId there : theres) + { + const ClassTypeVar* tctv = get(there); + LUAU_ASSERT(tctv); + if (isSubclass(hctv, tctv)) + { + keep = true; + break; + } + else if (isSubclass(tctv, hctv)) + { + keep = false; + tmp.insert(there); + break; + } + } + if (keep) + it++; + else + it = heres.erase(it); + } + heres.insert(tmp.begin(), tmp.end()); +} + +void Normalizer::intersectClassesWithClass(TypeIds& heres, TypeId there) +{ + bool foundSuper = false; + const ClassTypeVar* tctv = get(there); + LUAU_ASSERT(tctv); + for (auto it = heres.begin(); it != heres.end();) + { + const ClassTypeVar* hctv = get(*it); + LUAU_ASSERT(hctv); + if (isSubclass(hctv, tctv)) + it++; + else if (isSubclass(tctv, hctv)) + { + foundSuper = true; + break; + } + else + it = heres.erase(it); + } + if (foundSuper) + { + heres.clear(); + heres.insert(there); + } +} + +void Normalizer::intersectStrings(NormalizedStringType& here, const NormalizedStringType& there) +{ + if (!there) + return; + if (!here) + here.emplace(); + + for (auto it = here->begin(); it != here->end();) + { + if (there->count(it->first)) + it++; + else + it = here->erase(it); + } +} + +std::optional Normalizer::intersectionOfTypePacks(TypePackId here, TypePackId there) +{ + if (here == there) + return here; + + std::vector head; + std::optional tail; + + bool hereSubThere = true; + bool thereSubHere = true; + + TypePackIterator ith = begin(here); + TypePackIterator itt = begin(there); + + while (ith != end(here) && itt != end(there)) + { + TypeId hty = *ith; + TypeId tty = *itt; + TypeId ty = intersectionType(hty, tty); + if (ty != hty) + hereSubThere = false; + if (ty != tty) + thereSubHere = false; + head.push_back(ty); + ith++; + itt++; + } + + auto dealWithDifferentArities = [&](TypePackIterator& ith, TypePackIterator itt, TypePackId here, TypePackId there, bool& hereSubThere, bool& thereSubHere) + { + if (ith != end(here)) + { + TypeId tty = singletonTypes->nilType; + if (std::optional ttail = itt.tail()) + { + if (const VariadicTypePack* tvtp = get(*ttail)) + tty = tvtp->ty; + else + // Luau doesn't have intersections of type pack variables + return false; + } + else + // Type packs of different arities are incomparable + return false; + + while (ith != end(here)) + { + TypeId hty = *ith; + TypeId ty = intersectionType(hty, tty); + if (ty != hty) + hereSubThere = false; + if (ty != tty) + thereSubHere = false; + head.push_back(ty); + ith++; + } + } + return true; + }; + + if (!dealWithDifferentArities(ith, itt, here, there, hereSubThere, thereSubHere)) + return std::nullopt; + + if (!dealWithDifferentArities(itt, ith, there, here, thereSubHere, hereSubThere)) + return std::nullopt; + + if (std::optional htail = ith.tail()) + { + if (std::optional ttail = itt.tail()) + { + if (*htail == *ttail) + tail = htail; + else if (const VariadicTypePack* hvtp = get(*htail)) + { + if (const VariadicTypePack* tvtp = get(*ttail)) + { + TypeId ty = intersectionType(hvtp->ty, tvtp->ty); + if (ty != hvtp->ty) + thereSubHere = false; + if (ty != tvtp->ty) + hereSubThere = false; + bool hidden = hvtp->hidden & tvtp->hidden; + tail = arena->addTypePack(VariadicTypePack{ty,hidden}); + } + else + // Luau doesn't have unions of type pack variables + return std::nullopt; + } + else + // Luau doesn't have unions of type pack variables + return std::nullopt; + } + else if (get(*htail)) + hereSubThere = false; + else + // Luau doesn't have unions of type pack variables + return std::nullopt; + } + else if (std::optional ttail = itt.tail()) + { + if (get(*ttail)) + thereSubHere = false; + else + // Luau doesn't have unions of type pack variables + return std::nullopt; + } + + if (hereSubThere) + return here; + else if (thereSubHere) + return there; + if (!head.empty()) + return arena->addTypePack(TypePack{head,tail}); + else if (tail) + return *tail; + else + // TODO: Add an emptyPack to singleton types + return arena->addTypePack({}); +} + +std::optional Normalizer::intersectionOfTables(TypeId here, TypeId there) +{ + if (here == there) + return here; + + RecursionCounter _rc(&sharedState->counters.recursionCount); + if (sharedState->counters.recursionLimit > 0 && sharedState->counters.recursionLimit < sharedState->counters.recursionCount) + return std::nullopt; + + TypeId htable = here; + TypeId hmtable = nullptr; + if (const MetatableTypeVar* hmtv = get(here)) + { + htable = hmtv->table; + hmtable = hmtv->metatable; + } + TypeId ttable = there; + TypeId tmtable = nullptr; + if (const MetatableTypeVar* tmtv = get(there)) + { + ttable = tmtv->table; + tmtable = tmtv->metatable; + } + + const TableTypeVar* httv = get(htable); + LUAU_ASSERT(httv); + const TableTypeVar* tttv = get(ttable); + LUAU_ASSERT(tttv); + + if (httv->state == TableState::Free || tttv->state == TableState::Free) + return std::nullopt; + if (httv->state == TableState::Generic || tttv->state == TableState::Generic) + return std::nullopt; + + TableState state = httv->state; + if (tttv->state == TableState::Unsealed) + state = tttv->state; + + TypeLevel level = max(httv->level, tttv->level); + TableTypeVar result{state, level}; + + bool hereSubThere = true; + bool thereSubHere = true; + + for (const auto& [name, hprop] : httv->props) + { + Property prop = hprop; + auto tfound = tttv->props.find(name); + if (tfound == tttv->props.end()) + thereSubHere = false; + else + { + const auto& [_name, tprop] = *tfound; + // TODO: variance issues here, which can't be fixed until we have read/write property types + prop.type = intersectionType(hprop.type, tprop.type); + hereSubThere &= (prop.type == hprop.type); + thereSubHere &= (prop.type == tprop.type); + } + // TODO: string indexers + result.props[name] = prop; + } + + for (const auto& [name, tprop] : tttv->props) + { + if (httv->props.count(name) == 0) + { + result.props[name] = tprop; + hereSubThere = false; + } + } + + if (httv->indexer && tttv->indexer) + { + // TODO: What should intersection of indexes be? + TypeId index = unionType(httv->indexer->indexType, tttv->indexer->indexType); + TypeId indexResult = intersectionType(httv->indexer->indexResultType, tttv->indexer->indexResultType); + result.indexer = {index, indexResult}; + hereSubThere &= (httv->indexer->indexType == index) && (httv->indexer->indexResultType == indexResult); + thereSubHere &= (tttv->indexer->indexType == index) && (tttv->indexer->indexResultType == indexResult); + } + else if (httv->indexer) + { + result.indexer = httv->indexer; + thereSubHere = false; + } + else if (tttv->indexer) + { + result.indexer = tttv->indexer; + hereSubThere = false; + } + + TypeId table; + if (hereSubThere) + table = htable; + else if (thereSubHere) + table = ttable; + else + table = arena->addType(std::move(result)); + + if (tmtable && hmtable) + { + // NOTE: this assumes metatables are ivariant + if (std::optional mtable = intersectionOfTables(hmtable, tmtable)) + { + if (table == htable && *mtable == hmtable) + return here; + else if (table == ttable && *mtable == tmtable) + return there; + else + return arena->addType(MetatableTypeVar{table, *mtable}); + } + else + return std::nullopt; + + } + else if (hmtable) + { + if (table == htable) + return here; + else + return arena->addType(MetatableTypeVar{table, hmtable}); + } + else if (tmtable) + { + if (table == ttable) + return there; + else + return arena->addType(MetatableTypeVar{table, tmtable}); + } + else + return table; +} + +void Normalizer::intersectTablesWithTable(TypeIds& heres, TypeId there) +{ + TypeIds tmp; + for (TypeId here : heres) + if (std::optional inter = intersectionOfTables(here, there)) + tmp.insert(*inter); + heres.retain(tmp); + heres.insert(tmp.begin(), tmp.end()); +} + +void Normalizer::intersectTables(TypeIds& heres, const TypeIds& theres) +{ + TypeIds tmp; + for (TypeId here : heres) + for (TypeId there : theres) + if (std::optional inter = intersectionOfTables(here, there)) + tmp.insert(*inter); + heres.retain(tmp); + heres.insert(tmp.begin(), tmp.end()); +} + +std::optional Normalizer::intersectionOfFunctions(TypeId here, TypeId there) +{ + const FunctionTypeVar* hftv = get(here); + LUAU_ASSERT(hftv); + const FunctionTypeVar* tftv = get(there); + LUAU_ASSERT(tftv); + + if (hftv->generics != tftv->generics) + return std::nullopt; + if (hftv->genericPacks != tftv->genericPacks) + return std::nullopt; + if (hftv->retTypes != tftv->retTypes) + return std::nullopt; + + std::optional argTypes = unionOfTypePacks(hftv->argTypes, tftv->argTypes); + if (!argTypes) + return std::nullopt; + + if (*argTypes == hftv->argTypes) + return here; + if (*argTypes == tftv->argTypes) + return there; + + FunctionTypeVar result{*argTypes, hftv->retTypes}; + result.generics = hftv->generics; + result.genericPacks = hftv->genericPacks; + return arena->addType(std::move(result)); +} + +std::optional Normalizer::unionSaturatedFunctions(TypeId here, TypeId there) +{ + // Deep breath... + // + // When we come to check overloaded functions for subtyping, + // we have to compare (F1 & ... & FM) <: (G1 & ... G GN) + // where each Fi or Gj is a function type. Now that intersection on the right is no + // problem, since that's true if and only if (F1 & ... & FM) <: Gj for every j. + // But the intersection on the left is annoying, since we might have + // (F1 & ... & FM) <: G but no Fi <: G. For example + // + // ((number? -> number?) & (string? -> string?)) <: (nil -> nil) + // + // So in this case, what we do is define Apply for the result of applying + // a function of type F to an argument of type T, and then F <: (T -> U) + // if and only if Apply <: U. For example: + // + // if f : ((number? -> number?) & (string? -> string?)) + // then f(nil) must be nil, so + // Apply<((number? -> number?) & (string? -> string?)), nil> is nil + // + // So subtyping on overloaded functions "just" boils down to defining Apply. + // + // Now for non-overloaded functions, this is easy! + // Apply<(R -> S), T> is S if T <: R, and an error type otherwise. + // + // But for overloaded functions it's not so simple. We'd like Apply + // to just be Apply & ... & Apply but oh dear + // + // if f : ((number -> number) & (string -> string)) + // and x : (number | string) + // then f(x) : (number | string) + // + // so we want + // + // Apply<((number -> number) & (string -> string)), (number | string)> is (number | string) + // + // but + // + // Apply<(number -> number), (number | string)> is an error + // Apply<(string -> string), (number | string)> is an error + // + // that is Apply should consider all possible combinations of overloads of F, + // not just individual overloads. + // + // For this reason, when we're normalizing function types (in order to check subtyping + // or perform overload resolution) we should first *union-saturate* them. An overloaded + // function is union-saturated whenever: + // + // if (R -> S) is an overload of F + // and (T -> U) is an overload of F + // then ((R | T) -> (S | U)) is a subtype of an overload of F + // + // Any overloaded function can be normalized to a union-saturated one by adding enough extra overloads. + // For example, union-saturating + // + // ((number -> number) & (string -> string)) + // + // is + // + // ((number -> number) & (string -> string) & ((number | string) -> (number | string))) + // + // For union-saturated overloaded functions, the "obvious" algorithm works: + // + // Apply is Apply & ... & Apply + // + // so we can define Apply, so we can perform overloaded function resolution + // and check subtyping on overloaded function types, yay! + // + // This is yet another potential source of exponential blow-up, sigh, since + // the union-saturation of a function with N overloads may have 2^N overloads + // (one for every subset). In practice, that hopefully won't happen that often, + // in particular we only union-saturate overloads with different return types, + // and there are hopefully not very many cases of that. + // + // All of this is mechanically verified in Agda, at https://github.com/luau-lang/agda-typeck + // + // It is essentially the algorithm defined in https://pnwamk.github.io/sst-tutorial/ + // except that we're precomputing the union-saturation rather than converting + // to disjunctive normal form on the fly. + // + // This is all built on semantic subtyping: + // + // Covariance and Contravariance, Giuseppe Castagna, + // Logical Methods in Computer Science 16(1), 2022 + // https://arxiv.org/abs/1809.01427 + // + // A gentle introduction to semantic subtyping, Giuseppe Castagna and Alain Frisch, + // Proc. Principles and practice of declarative programming 2005, pp 198–208 + // https://doi.org/10.1145/1069774.1069793 + + const FunctionTypeVar* hftv = get(here); + if (!hftv) + return std::nullopt; + const FunctionTypeVar* tftv = get(there); + if (!tftv) + return std::nullopt; + + if (hftv->generics != tftv->generics) + return std::nullopt; + if (hftv->genericPacks != tftv->genericPacks) + return std::nullopt; + + std::optional argTypes = unionOfTypePacks(hftv->argTypes, tftv->argTypes); + if (!argTypes) + return std::nullopt; + std::optional retTypes = unionOfTypePacks(hftv->retTypes, tftv->retTypes); + if (!retTypes) + return std::nullopt; + + FunctionTypeVar result{*argTypes, *retTypes}; + result.generics = hftv->generics; + result.genericPacks = hftv->genericPacks; + return arena->addType(std::move(result)); +} + +void Normalizer::intersectFunctionsWithFunction(NormalizedFunctionType& heres, TypeId there) +{ + if (!heres) + return; + + for (auto it = heres->begin(); it != heres->end();) + { + TypeId here = *it; + if (get(here)) + it++; + else if (std::optional tmp = intersectionOfFunctions(here, there)) + { + heres->erase(it); + heres->insert(*tmp); + return; + } + else + it++; + } + + TypeIds tmps; + for (TypeId here : *heres) + { + if (std::optional tmp = unionSaturatedFunctions(here, there)) + tmps.insert(*tmp); + } + heres->insert(there); + heres->insert(tmps.begin(), tmps.end()); +} + +void Normalizer::intersectFunctions(NormalizedFunctionType& heres, const NormalizedFunctionType& theres) +{ + if (!heres) + return; + else if (!theres) + { + heres = std::nullopt; + return; + } + else + { + for (TypeId there : *theres) + intersectFunctionsWithFunction(heres, there); + } +} + +bool Normalizer::intersectTyvarsWithTy(NormalizedTyvars& here, TypeId there) +{ + for (auto it = here.begin(); it != here.end();) + { + NormalizedType& inter = *it->second; + if (!intersectNormalWithTy(inter, there)) + return false; + if (isInhabited(inter)) + ++it; + else + it = here.erase(it); + } + return true; +} + +// See above for an explaination of `ignoreSmallerTyvars`. +bool Normalizer::intersectNormals(NormalizedType& here, const NormalizedType& there, int ignoreSmallerTyvars) +{ + if (!get(there.tops)) + { + here.tops = intersectionOfTops(here.tops, there.tops); + return true; + } + else if (!get(here.tops)) + { + clearNormal(here); + return unionNormals(here, there, ignoreSmallerTyvars); + } + + here.booleans = intersectionOfBools(here.booleans, there.booleans); + intersectClasses(here.classes, there.classes); + here.errors = (get(there.errors) ? there.errors : here.errors); + here.nils = (get(there.nils) ? there.nils : here.nils); + here.numbers = (get(there.numbers) ? there.numbers : here.numbers); + intersectStrings(here.strings, there.strings); + here.threads = (get(there.threads) ? there.threads : here.threads); + intersectFunctions(here.functions, there.functions); + intersectTables(here.tables, there.tables); + + for (auto& [tyvar, inter] : there.tyvars) + { + int index = tyvarIndex(tyvar); + if (ignoreSmallerTyvars < index) + { + auto [found, fresh] = here.tyvars.emplace(tyvar, std::make_unique(NormalizedType{singletonTypes})); + if (fresh) + { + if (!unionNormals(*found->second, here, index)) + return false; + } + } + } + for (auto it = here.tyvars.begin(); it != here.tyvars.end();) + { + TypeId tyvar = it->first; + NormalizedType& inter = *it->second; + int index = tyvarIndex(tyvar); + LUAU_ASSERT(ignoreSmallerTyvars < index); + auto found = there.tyvars.find(tyvar); + if (found == there.tyvars.end()) + { + if (!intersectNormals(inter, there, index)) + return false; + } + else + { + if (!intersectNormals(inter, *found->second, index)) + return false; + } + if (isInhabited(inter)) + it++; + else + it = here.tyvars.erase(it); + } + return true; +} + +bool Normalizer::intersectNormalWithTy(NormalizedType& here, TypeId there) +{ + RecursionCounter _rc(&sharedState->counters.recursionCount); + if (!withinResourceLimits()) + return false; + + there = follow(there); + if (get(there) || get(there)) + { + here.tops = intersectionOfTops(here.tops, there); + return true; + } + else if (!get(here.tops)) + { + clearNormal(here); + return unionNormalWithTy(here, there); + } + else if (const UnionTypeVar* utv = get(there)) + { + NormalizedType norm{singletonTypes}; + for (UnionTypeVarIterator it = begin(utv); it != end(utv); ++it) + if (!unionNormalWithTy(norm, *it)) + return false; + return intersectNormals(here, norm); + } + else if (const IntersectionTypeVar* itv = get(there)) + { + for (IntersectionTypeVarIterator it = begin(itv); it != end(itv); ++it) + if (!intersectNormalWithTy(here, *it)) + return false; + return true; + } + else if (get(there) || get(there)) + { + NormalizedType thereNorm{singletonTypes}; + NormalizedType topNorm{singletonTypes}; + topNorm.tops = singletonTypes->unknownType; + thereNorm.tyvars.insert_or_assign(there, std::make_unique(std::move(topNorm))); + return intersectNormals(here, thereNorm); + } + + NormalizedTyvars tyvars = std::move(here.tyvars); + + if (const FunctionTypeVar* utv = get(there)) + { + NormalizedFunctionType functions = std::move(here.functions); + clearNormal(here); + intersectFunctionsWithFunction(functions, there); + here.functions = std::move(functions); + } + else if (get(there) || get(there)) + { + TypeIds tables = std::move(here.tables); + clearNormal(here); + intersectTablesWithTable(tables, there); + here.tables = std::move(tables); + } + else if (get(there)) + { + TypeIds classes = std::move(here.classes); + clearNormal(here); + intersectClassesWithClass(classes, there); + here.classes = std::move(classes); + } + else if (get(there)) + { + TypeId errors = here.errors; + clearNormal(here); + here.errors = errors; + } + else if (const PrimitiveTypeVar* ptv = get(there)) + { + TypeId booleans = here.booleans; + TypeId nils = here.nils; + TypeId numbers = here.numbers; + NormalizedStringType strings = std::move(here.strings); + TypeId threads = here.threads; + + clearNormal(here); + + if (ptv->type == PrimitiveTypeVar::Boolean) + here.booleans = booleans; + else if (ptv->type == PrimitiveTypeVar::NilType) + here.nils = nils; + else if (ptv->type == PrimitiveTypeVar::Number) + here.numbers = numbers; + else if (ptv->type == PrimitiveTypeVar::String) + here.strings = std::move(strings); + else if (ptv->type == PrimitiveTypeVar::Thread) + here.threads = threads; + else + LUAU_ASSERT(!"Unreachable"); + } + else if (const SingletonTypeVar* stv = get(there)) + { + TypeId booleans = here.booleans; + NormalizedStringType strings = std::move(here.strings); + + clearNormal(here); + + if (get(stv)) + here.booleans = intersectionOfBools(booleans, there); + else if (const StringSingleton* sstv = get(stv)) + { + if (!strings || strings->count(sstv->value)) + here.strings->insert({sstv->value, there}); + } + else + LUAU_ASSERT(!"Unreachable"); + } + else + LUAU_ASSERT(!"Unreachable"); + + if (!intersectTyvarsWithTy(tyvars, there)) + return false; + here.tyvars = std::move(tyvars); + + return true; +} + +// -------- Convert back from a normalized type to a type +TypeId Normalizer::typeFromNormal(const NormalizedType& norm) +{ + assertInvariant(norm); + if (!get(norm.tops)) + return norm.tops; + + std::vector result; + + if (!get(norm.booleans)) + result.push_back(norm.booleans); + result.insert(result.end(), norm.classes.begin(), norm.classes.end()); + if (!get(norm.errors)) + result.push_back(norm.errors); + if (norm.functions) + { + if (norm.functions->size() == 1) + result.push_back(*norm.functions->begin()); + else + { + std::vector parts; + parts.insert(parts.end(), norm.functions->begin(), norm.functions->end()); + result.push_back(arena->addType(IntersectionTypeVar{std::move(parts)})); + } + } + if (!get(norm.nils)) + result.push_back(norm.nils); + if (!get(norm.numbers)) + result.push_back(norm.numbers); + if (norm.strings) + for (auto& [_, ty] : *norm.strings) + result.push_back(ty); + else + result.push_back(singletonTypes->stringType); + result.insert(result.end(), norm.tables.begin(), norm.tables.end()); + for (auto& [tyvar, intersect] : norm.tyvars) + { + if (get(intersect->tops)) + { + TypeId ty = typeFromNormal(*intersect); + result.push_back(arena->addType(IntersectionTypeVar{{tyvar, ty}})); + } + else + result.push_back(tyvar); + } + + if (result.size() == 0) + return singletonTypes->neverType; + else if (result.size() == 1) + return result[0]; + else + return arena->addType(UnionTypeVar{std::move(result)}); +} namespace { @@ -59,7 +1748,8 @@ bool isSubtype(TypeId subTy, TypeId superTy, NotNull scope, NotNull scope, N { UnifierSharedState sharedState{&ice}; TypeArena arena; - Unifier u{&arena, singletonTypes, Mode::Strict, scope, Location{}, Covariant, sharedState}; + Normalizer normalizer{&arena, singletonTypes, NotNull{&sharedState}}; + Unifier u{NotNull{&normalizer}, Mode::Strict, scope, Location{}, Covariant}; u.anyIsTop = anyIsTop; u.tryUnify(subPack, superPack); @@ -686,3 +2377,4 @@ std::pair normalize(TypePackId tp, const ModulePtr& module, No } } // namespace Luau + diff --git a/Analysis/src/TypeChecker2.cpp b/Analysis/src/TypeChecker2.cpp index f98a2123..b2f3cfd3 100644 --- a/Analysis/src/TypeChecker2.cpp +++ b/Analysis/src/TypeChecker2.cpp @@ -280,7 +280,8 @@ struct TypeChecker2 TypePackId actualRetType = reconstructPack(ret->list, arena); UnifierSharedState sharedState{&ice}; - Unifier u{&arena, singletonTypes, Mode::Strict, stack.back(), ret->location, Covariant, sharedState}; + Normalizer normalizer{&arena, singletonTypes, NotNull{&sharedState}}; + Unifier u{NotNull{&normalizer}, Mode::Strict, stack.back(), ret->location, Covariant}; u.anyIsTop = true; u.tryUnify(actualRetType, expectedRetType); @@ -1206,7 +1207,8 @@ struct TypeChecker2 ErrorVec tryUnify(NotNull scope, const Location& location, TID subTy, TID superTy) { UnifierSharedState sharedState{&ice}; - Unifier u{&module->internalTypes, singletonTypes, Mode::Strict, scope, location, Covariant, sharedState}; + Normalizer normalizer{&module->internalTypes, singletonTypes, NotNull{&sharedState}}; + Unifier u{NotNull{&normalizer}, Mode::Strict, scope, location, Covariant}; u.anyIsTop = true; u.tryUnify(subTy, superTy); diff --git a/Analysis/src/TypeInfer.cpp b/Analysis/src/TypeInfer.cpp index b96046be..cb21aa7f 100644 --- a/Analysis/src/TypeInfer.cpp +++ b/Analysis/src/TypeInfer.cpp @@ -32,19 +32,21 @@ LUAU_FASTINTVARIABLE(LuauCheckRecursionLimit, 300) LUAU_FASTINTVARIABLE(LuauVisitRecursionLimit, 500) LUAU_FASTFLAG(LuauKnowsTheDataModel3) LUAU_FASTFLAG(LuauAutocompleteDynamicLimits) +LUAU_FASTFLAG(LuauTypeNormalization2) LUAU_FASTFLAGVARIABLE(LuauFunctionArgMismatchDetails, false) -LUAU_FASTFLAGVARIABLE(LuauInplaceDemoteSkipAllBound, false) LUAU_FASTFLAGVARIABLE(LuauLowerBoundsCalculation, false) LUAU_FASTFLAGVARIABLE(DebugLuauFreezeDuringUnification, false) -LUAU_FASTFLAGVARIABLE(LuauSelfCallAutocompleteFix3, false) LUAU_FASTFLAGVARIABLE(LuauReturnAnyInsteadOfICE, false) // Eventually removed as false. LUAU_FASTFLAGVARIABLE(DebugLuauSharedSelf, false) +LUAU_FASTFLAGVARIABLE(LuauAnyifyModuleReturnGenerics, false) LUAU_FASTFLAGVARIABLE(LuauUnknownAndNeverType, false) LUAU_FASTFLAGVARIABLE(LuauCallUnifyPackTails, false) LUAU_FASTFLAGVARIABLE(LuauCheckGenericHOFTypes, false) LUAU_FASTFLAGVARIABLE(LuauBinaryNeedsExpectedTypesToo, false) +LUAU_FASTFLAGVARIABLE(LuauFixVarargExprHeadType, false) LUAU_FASTFLAGVARIABLE(LuauNeverTypesAndOperatorsInference, false) LUAU_FASTFLAGVARIABLE(LuauReturnsFromCallsitesAreNotWidened, false) +LUAU_FASTFLAG(LuauInstantiateInSubtyping) LUAU_FASTFLAGVARIABLE(LuauCompleteVisitor, false) LUAU_FASTFLAGVARIABLE(LuauUnionOfTypesFollow, false) LUAU_FASTFLAGVARIABLE(LuauReportShadowedTypeAlias, false) @@ -255,6 +257,7 @@ TypeChecker::TypeChecker(ModuleResolver* resolver, NotNull singl , singletonTypes(singletonTypes) , iceHandler(iceHandler) , unifierState(iceHandler) + , normalizer(nullptr, singletonTypes, NotNull{&unifierState}) , nilType(singletonTypes->nilType) , numberType(singletonTypes->numberType) , stringType(singletonTypes->stringType) @@ -301,12 +304,13 @@ ModulePtr TypeChecker::checkWithoutRecursionCheck(const SourceModule& module, Mo LUAU_TIMETRACE_SCOPE("TypeChecker::check", "TypeChecker"); LUAU_TIMETRACE_ARGUMENT("module", module.name.c_str()); - currentModule.reset(new Module()); + currentModule.reset(new Module); currentModule->type = module.type; currentModule->allocator = module.allocator; currentModule->names = module.names; iceHandler->moduleName = module.name; + normalizer.arena = ¤tModule->internalTypes; if (FFlag::LuauAutocompleteDynamicLimits) { @@ -351,15 +355,23 @@ ModulePtr TypeChecker::checkWithoutRecursionCheck(const SourceModule& module, Mo if (get(follow(moduleScope->returnType))) moduleScope->returnType = addTypePack(TypePack{{}, std::nullopt}); else - { moduleScope->returnType = anyify(moduleScope, moduleScope->returnType, Location{}); - } + + if (FFlag::LuauAnyifyModuleReturnGenerics) + moduleScope->returnType = anyifyModuleReturnTypePackGenerics(moduleScope->returnType); for (auto& [_, typeFun] : moduleScope->exportedTypeBindings) typeFun.type = anyify(moduleScope, typeFun.type, Location{}); prepareErrorsForDisplay(currentModule->errors); + if (FFlag::LuauTypeNormalization2) + { + // Clear the normalizer caches, since they contain types from the internal type surface + normalizer.clearCaches(); + normalizer.arena = nullptr; + } + currentModule->clonePublicInterface(singletonTypes, *iceHandler); // Clear unifier cache since it's keyed off internal types that get deallocated @@ -474,7 +486,7 @@ struct InplaceDemoter : TypeVarOnceVisitor TypeArena* arena; InplaceDemoter(TypeLevel level, TypeArena* arena) - : TypeVarOnceVisitor(/* skipBoundTypes= */ FFlag::LuauInplaceDemoteSkipAllBound) + : TypeVarOnceVisitor(/* skipBoundTypes= */ true) , newLevel(level) , arena(arena) { @@ -494,12 +506,6 @@ struct InplaceDemoter : TypeVarOnceVisitor return false; } - bool visit(TypeId ty, const BoundTypeVar& btyRef) override - { - LUAU_ASSERT(!FFlag::LuauInplaceDemoteSkipAllBound); - return true; - } - bool visit(TypeId ty) override { if (ty->owningArena != arena) @@ -1029,8 +1035,11 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatAssign& assign) if (right) { - if (!maybeGeneric(left) && isGeneric(right)) - right = instantiate(scope, right, loc); + if (!FFlag::LuauInstantiateInSubtyping) + { + if (!maybeGeneric(left) && isGeneric(right)) + right = instantiate(scope, right, loc); + } // Setting a table entry to nil doesn't mean nil is the type of the indexer, it is just deleting the entry const TableTypeVar* destTableTypeReceivingNil = nullptr; @@ -1104,7 +1113,9 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatLocal& local) variableTypes.push_back(ty); expectedTypes.push_back(ty); - instantiateGenerics.push_back(annotation != nullptr && !maybeGeneric(ty)); + // with FFlag::LuauInstantiateInSubtyping enabled, we shouldn't need to produce instantiateGenerics at all. + if (!FFlag::LuauInstantiateInSubtyping) + instantiateGenerics.push_back(annotation != nullptr && !maybeGeneric(ty)); } if (local.values.size > 0) @@ -1729,9 +1740,7 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatDeclareClass& declar { ftv->argNames.insert(ftv->argNames.begin(), FunctionArgument{"self", {}}); ftv->argTypes = addTypePack(TypePack{{classTy}, ftv->argTypes}); - - if (FFlag::LuauSelfCallAutocompleteFix3) - ftv->hasSelf = true; + ftv->hasSelf = true; } } @@ -1905,8 +1914,18 @@ WithPredicate TypeChecker::checkExpr(const ScopePtr& scope, const AstExp if (get(varargPack)) { - std::vector types = flatten(varargPack).first; - return {!types.empty() ? types[0] : nilType}; + if (FFlag::LuauFixVarargExprHeadType) + { + if (std::optional ty = first(varargPack)) + return {*ty}; + + return {nilType}; + } + else + { + std::vector types = flatten(varargPack).first; + return {!types.empty() ? types[0] : nilType}; + } } else if (get(varargPack)) { @@ -3967,7 +3986,10 @@ void TypeChecker::checkArgumentList(const ScopePtr& scope, const AstExpr& funNam } else { - unifyWithInstantiationIfNeeded(*argIter, *paramIter, scope, state); + if (FFlag::LuauInstantiateInSubtyping) + state.tryUnify(*argIter, *paramIter, /*isFunctionCall*/ false); + else + unifyWithInstantiationIfNeeded(*argIter, *paramIter, scope, state); ++argIter; ++paramIter; } @@ -4523,8 +4545,11 @@ WithPredicate TypeChecker::checkExprList(const ScopePtr& scope, cons TypeId actualType = substituteFreeForNil && expr->is() ? freshType(scope) : type; - if (instantiateGenerics.size() > i && instantiateGenerics[i]) - actualType = instantiate(scope, actualType, expr->location); + if (!FFlag::LuauInstantiateInSubtyping) + { + if (instantiateGenerics.size() > i && instantiateGenerics[i]) + actualType = instantiate(scope, actualType, expr->location); + } if (expectedType) { @@ -4686,6 +4711,8 @@ bool TypeChecker::unifyWithInstantiationIfNeeded(TypeId subTy, TypeId superTy, c void TypeChecker::unifyWithInstantiationIfNeeded(TypeId subTy, TypeId superTy, const ScopePtr& scope, Unifier& state) { + LUAU_ASSERT(!FFlag::LuauInstantiateInSubtyping); + if (!maybeGeneric(subTy)) // Quick check to see if we definitely can't instantiate state.tryUnify(subTy, superTy, /*isFunctionCall*/ false); @@ -4828,6 +4855,33 @@ TypePackId TypeChecker::anyify(const ScopePtr& scope, TypePackId ty, Location lo } } +TypePackId TypeChecker::anyifyModuleReturnTypePackGenerics(TypePackId tp) +{ + tp = follow(tp); + + if (const VariadicTypePack* vtp = get(tp)) + return get(vtp->ty) ? anyTypePack : tp; + + if (!get(follow(tp))) + return tp; + + std::vector resultTypes; + std::optional resultTail; + + TypePackIterator it = begin(tp); + + for (TypePackIterator e = end(tp); it != e; ++it) + { + TypeId ty = follow(*it); + resultTypes.push_back(get(ty) ? anyType : ty); + } + + if (std::optional tail = it.tail()) + resultTail = anyifyModuleReturnTypePackGenerics(*tail); + + return addTypePack(resultTypes, resultTail); +} + void TypeChecker::reportError(const TypeError& error) { if (currentModule->mode == Mode::NoCheck) @@ -4955,8 +5009,7 @@ void TypeChecker::merge(RefinementMap& l, const RefinementMap& r) Unifier TypeChecker::mkUnifier(const ScopePtr& scope, const Location& location) { - return Unifier{ - ¤tModule->internalTypes, singletonTypes, currentModule->mode, NotNull{scope.get()}, location, Variance::Covariant, unifierState}; + return Unifier{NotNull{&normalizer}, currentModule->mode, NotNull{scope.get()}, location, Variance::Covariant}; } TypeId TypeChecker::freshType(const ScopePtr& scope) diff --git a/Analysis/src/TypeVar.cpp b/Analysis/src/TypeVar.cpp index bf6bf34a..b143268e 100644 --- a/Analysis/src/TypeVar.cpp +++ b/Analysis/src/TypeVar.cpp @@ -27,6 +27,7 @@ LUAU_FASTFLAG(LuauUnknownAndNeverType) LUAU_FASTFLAGVARIABLE(LuauMaybeGenericIntersectionTypes, false) LUAU_FASTFLAGVARIABLE(LuauStringFormatArgumentErrorFix, false) LUAU_FASTFLAGVARIABLE(LuauNoMoreGlobalSingletonTypes, false) +LUAU_FASTFLAG(LuauInstantiateInSubtyping) namespace Luau { @@ -339,6 +340,8 @@ bool isSubset(const UnionTypeVar& super, const UnionTypeVar& sub) // then instantiate U if `isGeneric(U)` is true, and `maybeGeneric(T)` is false. bool isGeneric(TypeId ty) { + LUAU_ASSERT(!FFlag::LuauInstantiateInSubtyping); + ty = follow(ty); if (auto ftv = get(ty)) return ftv->generics.size() > 0 || ftv->genericPacks.size() > 0; @@ -350,6 +353,8 @@ bool isGeneric(TypeId ty) bool maybeGeneric(TypeId ty) { + LUAU_ASSERT(!FFlag::LuauInstantiateInSubtyping); + if (FFlag::LuauMaybeGenericIntersectionTypes) { ty = follow(ty); diff --git a/Analysis/src/Unifiable.cpp b/Analysis/src/Unifiable.cpp index a3d4540c..e0cc1414 100644 --- a/Analysis/src/Unifiable.cpp +++ b/Analysis/src/Unifiable.cpp @@ -1,60 +1,64 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #include "Luau/Unifiable.h" +LUAU_FASTFLAG(LuauTypeNormalization2); + namespace Luau { namespace Unifiable { +static int nextIndex = 0; + Free::Free(TypeLevel level) - : index(++nextIndex) + : index(FFlag::LuauTypeNormalization2 ? ++nextIndex : ++DEPRECATED_nextIndex) , level(level) { } Free::Free(Scope* scope) - : index(++nextIndex) + : index(FFlag::LuauTypeNormalization2 ? ++nextIndex : ++DEPRECATED_nextIndex) , scope(scope) { } Free::Free(Scope* scope, TypeLevel level) - : index(++nextIndex) + : index(FFlag::LuauTypeNormalization2 ? ++nextIndex : ++DEPRECATED_nextIndex) , level(level) , scope(scope) { } -int Free::nextIndex = 0; +int Free::DEPRECATED_nextIndex = 0; Generic::Generic() - : index(++nextIndex) + : index(FFlag::LuauTypeNormalization2 ? ++nextIndex : ++DEPRECATED_nextIndex) , name("g" + std::to_string(index)) { } Generic::Generic(TypeLevel level) - : index(++nextIndex) + : index(FFlag::LuauTypeNormalization2 ? ++nextIndex : ++DEPRECATED_nextIndex) , level(level) , name("g" + std::to_string(index)) { } Generic::Generic(const Name& name) - : index(++nextIndex) + : index(FFlag::LuauTypeNormalization2 ? ++nextIndex : ++DEPRECATED_nextIndex) , name(name) , explicitName(true) { } Generic::Generic(Scope* scope) - : index(++nextIndex) + : index(FFlag::LuauTypeNormalization2 ? ++nextIndex : ++DEPRECATED_nextIndex) , scope(scope) { } Generic::Generic(TypeLevel level, const Name& name) - : index(++nextIndex) + : index(FFlag::LuauTypeNormalization2 ? ++nextIndex : ++DEPRECATED_nextIndex) , level(level) , name(name) , explicitName(true) @@ -62,14 +66,14 @@ Generic::Generic(TypeLevel level, const Name& name) } Generic::Generic(Scope* scope, const Name& name) - : index(++nextIndex) + : index(FFlag::LuauTypeNormalization2 ? ++nextIndex : ++DEPRECATED_nextIndex) , scope(scope) , name(name) , explicitName(true) { } -int Generic::nextIndex = 0; +int Generic::DEPRECATED_nextIndex = 0; Error::Error() : index(++nextIndex) diff --git a/Analysis/src/Unifier.cpp b/Analysis/src/Unifier.cpp index c13a6f8b..5a01c934 100644 --- a/Analysis/src/Unifier.cpp +++ b/Analysis/src/Unifier.cpp @@ -2,6 +2,7 @@ #include "Luau/Unifier.h" #include "Luau/Common.h" +#include "Luau/Instantiation.h" #include "Luau/RecursionCounter.h" #include "Luau/Scope.h" #include "Luau/TypePack.h" @@ -20,7 +21,9 @@ LUAU_FASTINTVARIABLE(LuauTypeInferLowerBoundsIterationLimit, 2000); LUAU_FASTFLAG(LuauLowerBoundsCalculation); LUAU_FASTFLAG(LuauErrorRecoveryType); LUAU_FASTFLAG(LuauUnknownAndNeverType) +LUAU_FASTFLAGVARIABLE(LuauSubtypeNormalizer, false); LUAU_FASTFLAGVARIABLE(LuauScalarShapeSubtyping, false) +LUAU_FASTFLAGVARIABLE(LuauInstantiateInSubtyping, false) LUAU_FASTFLAG(LuauClassTypeVarsInSubstitution) LUAU_FASTFLAG(LuauCallUnifyPackTails) LUAU_FASTFLAG(DebugLuauDeferredConstraintResolution) @@ -343,17 +346,19 @@ static bool subsumes(bool useScopes, TY_A* left, TY_B* right) return left->level.subsumes(right->level); } -Unifier::Unifier(TypeArena* types, NotNull singletonTypes, Mode mode, NotNull scope, const Location& location, - Variance variance, UnifierSharedState& sharedState, TxnLog* parentLog) - : types(types) - , singletonTypes(singletonTypes) +Unifier::Unifier(NotNull normalizer, Mode mode, NotNull scope, const Location& location, + Variance variance, TxnLog* parentLog) + : types(normalizer->arena) + , singletonTypes(normalizer->singletonTypes) + , normalizer(normalizer) , mode(mode) , scope(scope) , log(parentLog) , location(location) , variance(variance) - , sharedState(sharedState) + , sharedState(*normalizer->sharedState) { + normalize = FFlag::LuauSubtypeNormalizer; LUAU_ASSERT(sharedState.iceHandler); } @@ -524,7 +529,7 @@ void Unifier::tryUnify_(TypeId subTy, TypeId superTy, bool isFunctionCall, bool { tryUnifyUnionWithType(subTy, subUnion, superTy); } - else if (const UnionTypeVar* uv = log.getMutable(superTy)) + else if (const UnionTypeVar* uv = (FFlag::LuauSubtypeNormalizer? nullptr: log.getMutable(superTy))) { tryUnifyTypeWithUnion(subTy, superTy, uv, cacheEnabled, isFunctionCall); } @@ -532,6 +537,10 @@ void Unifier::tryUnify_(TypeId subTy, TypeId superTy, bool isFunctionCall, bool { tryUnifyTypeWithIntersection(subTy, superTy, uv); } + else if (const UnionTypeVar* uv = log.getMutable(superTy)) + { + tryUnifyTypeWithUnion(subTy, superTy, uv, cacheEnabled, isFunctionCall); + } else if (const IntersectionTypeVar* uv = log.getMutable(subTy)) { tryUnifyIntersectionWithType(subTy, uv, superTy, cacheEnabled, isFunctionCall); @@ -585,7 +594,7 @@ void Unifier::tryUnify_(TypeId subTy, TypeId superTy, bool isFunctionCall, bool void Unifier::tryUnifyUnionWithType(TypeId subTy, const UnionTypeVar* subUnion, TypeId superTy) { - // A | B <: T if A <: T and B <: T + // A | B <: T if and only if A <: T and B <: T bool failed = false; std::optional unificationTooComplex; std::optional firstFailedOption; @@ -715,6 +724,7 @@ void Unifier::tryUnifyTypeWithUnion(TypeId subTy, TypeId superTy, const UnionTyp { TypeId type = uv->options[(i + startIndex) % uv->options.size()]; Unifier innerState = makeChildUnifier(); + innerState.normalize = false; innerState.tryUnify_(subTy, type, isFunctionCall); if (innerState.errors.empty()) @@ -741,6 +751,20 @@ void Unifier::tryUnifyTypeWithUnion(TypeId subTy, TypeId superTy, const UnionTyp { reportError(*unificationTooComplex); } + else if (!found && normalize) + { + // It is possible that T <: A | B even though T normalize(subTy); + const NormalizedType* superNorm = normalizer->normalize(superTy); + if (!subNorm || !superNorm) + reportError(TypeError{location, UnificationTooComplex{}}); + else if ((failedOptionCount == 1 || foundHeuristic) && failedOption) + tryUnifyNormalizedTypes(subTy, superTy, *subNorm, *superNorm, "None of the union options are compatible. For example:", *failedOption); + else + tryUnifyNormalizedTypes(subTy, superTy, *subNorm, *superNorm, "none of the union options are compatible"); + } else if (!found) { if ((failedOptionCount == 1 || foundHeuristic) && failedOption) @@ -755,7 +779,7 @@ void Unifier::tryUnifyTypeWithIntersection(TypeId subTy, TypeId superTy, const I std::optional unificationTooComplex; std::optional firstFailedOption; - // T <: A & B if T <: A and T <: B + // T <: A & B if and only if T <: A and T <: B for (TypeId type : uv->parts) { Unifier innerState = makeChildUnifier(); @@ -806,6 +830,7 @@ void Unifier::tryUnifyIntersectionWithType(TypeId subTy, const IntersectionTypeV { TypeId type = uv->parts[(i + startIndex) % uv->parts.size()]; Unifier innerState = makeChildUnifier(); + innerState.normalize = false; innerState.tryUnify_(type, superTy, isFunctionCall); if (innerState.errors.empty()) @@ -822,12 +847,207 @@ void Unifier::tryUnifyIntersectionWithType(TypeId subTy, const IntersectionTypeV if (unificationTooComplex) reportError(*unificationTooComplex); + else if (!found && normalize) + { + // It is possible that A & B <: T even though A normalize(subTy); + const NormalizedType* superNorm = normalizer->normalize(superTy); + if (subNorm && superNorm) + tryUnifyNormalizedTypes(subTy, superTy, *subNorm, *superNorm, "none of the intersection parts are compatible"); + else + reportError(TypeError{location, UnificationTooComplex{}}); + } else if (!found) { reportError(TypeError{location, TypeMismatch{superTy, subTy, "none of the intersection parts are compatible"}}); } } +void Unifier::tryUnifyNormalizedTypes(TypeId subTy, TypeId superTy, const NormalizedType& subNorm, const NormalizedType& superNorm, std::string reason, std::optional error) +{ + LUAU_ASSERT(FFlag::LuauSubtypeNormalizer); + + if (get(superNorm.tops) || get(superNorm.tops) || get(subNorm.tops)) + return; + else if (get(subNorm.tops)) + return reportError(TypeError{location, TypeMismatch{superTy, subTy, reason, error}}); + + if (get(subNorm.errors)) + if (!get(superNorm.errors)) + return reportError(TypeError{location, TypeMismatch{superTy, subTy, reason, error}}); + + if (get(subNorm.booleans)) + { + if (!get(superNorm.booleans)) + return reportError(TypeError{location, TypeMismatch{superTy, subTy, reason, error}}); + } + else if (const SingletonTypeVar* stv = get(subNorm.booleans)) + { + if (!get(superNorm.booleans) && stv != get(superNorm.booleans)) + return reportError(TypeError{location, TypeMismatch{superTy, subTy, reason, error}}); + } + + if (get(subNorm.nils)) + if (!get(superNorm.nils)) + return reportError(TypeError{location, TypeMismatch{superTy, subTy, reason, error}}); + + if (get(subNorm.numbers)) + if (!get(superNorm.numbers)) + return reportError(TypeError{location, TypeMismatch{superTy, subTy, reason, error}}); + + if (subNorm.strings && superNorm.strings) + { + for (auto [name, ty] : *subNorm.strings) + if (!superNorm.strings->count(name)) + return reportError(TypeError{location, TypeMismatch{superTy, subTy, reason, error}}); + } + else if (!subNorm.strings && superNorm.strings) + return reportError(TypeError{location, TypeMismatch{superTy, subTy, reason, error}}); + + if (get(subNorm.threads)) + if (!get(superNorm.errors)) + return reportError(TypeError{location, TypeMismatch{superTy, subTy, reason, error}}); + + for (TypeId subClass : subNorm.classes) + { + bool found = false; + const ClassTypeVar* subCtv = get(subClass); + for (TypeId superClass : superNorm.classes) + { + const ClassTypeVar* superCtv = get(superClass); + if (isSubclass(subCtv, superCtv)) + { + found = true; + break; + } + } + if (!found) + return reportError(TypeError{location, TypeMismatch{superTy, subTy, reason, error}}); + } + + for (TypeId subTable : subNorm.tables) + { + bool found = false; + for (TypeId superTable : superNorm.tables) + { + Unifier innerState = makeChildUnifier(); + if (get(superTable)) + innerState.tryUnifyWithMetatable(subTable, superTable, /* reversed */ false); + else if (get(subTable)) + innerState.tryUnifyWithMetatable(superTable, subTable, /* reversed */ true); + else + innerState.tryUnifyTables(subTable, superTable); + if (innerState.errors.empty()) + { + found = true; + log.concat(std::move(innerState.log)); + break; + } + else if (auto e = hasUnificationTooComplex(innerState.errors)) + return reportError(*e); + } + if (!found) + return reportError(TypeError{location, TypeMismatch{superTy, subTy, reason, error}}); + } + + if (subNorm.functions) + { + if (!superNorm.functions) + return reportError(TypeError{location, TypeMismatch{superTy, subTy, reason, error}}); + if (superNorm.functions->empty()) + return; + for (TypeId superFun : *superNorm.functions) + { + Unifier innerState = makeChildUnifier(); + const FunctionTypeVar* superFtv = get(superFun); + if (!superFtv) + return reportError(TypeError{location, TypeMismatch{superTy, subTy, reason, error}}); + TypePackId tgt = innerState.tryApplyOverloadedFunction(subTy, subNorm.functions, superFtv->argTypes); + innerState.tryUnify_(tgt, superFtv->retTypes); + if (innerState.errors.empty()) + log.concat(std::move(innerState.log)); + else if (auto e = hasUnificationTooComplex(innerState.errors)) + return reportError(*e); + else + return reportError(TypeError{location, TypeMismatch{superTy, subTy, reason, error}}); + } + } + + for (auto& [tyvar, subIntersect] : subNorm.tyvars) + { + auto found = superNorm.tyvars.find(tyvar); + if (found == superNorm.tyvars.end()) + tryUnifyNormalizedTypes(subTy, superTy, *subIntersect, superNorm, reason, error); + else + tryUnifyNormalizedTypes(subTy, superTy, *subIntersect, *found->second, reason, error); + if (!errors.empty()) + return; + } +} + +TypePackId Unifier::tryApplyOverloadedFunction(TypeId function, const NormalizedFunctionType& overloads, TypePackId args) +{ + if (!overloads || overloads->empty()) + { + reportError(TypeError{location, CannotCallNonFunction{function}}); + return singletonTypes->errorRecoveryTypePack(); + } + + std::optional result; + const FunctionTypeVar* firstFun = nullptr; + for (TypeId overload : *overloads) + { + if (const FunctionTypeVar* ftv = get(overload)) + { + // TODO: instantiate generics? + if (ftv->generics.empty() && ftv->genericPacks.empty()) + { + if (!firstFun) + firstFun = ftv; + Unifier innerState = makeChildUnifier(); + innerState.tryUnify_(args, ftv->argTypes); + if (innerState.errors.empty()) + { + log.concat(std::move(innerState.log)); + if (result) + { + // Annoyingly, since we don't support intersection of generic type packs, + // the intersection may fail. We rather arbitrarily use the first matching overload + // in that case. + if (std::optional intersect = normalizer->intersectionOfTypePacks(*result, ftv->retTypes)) + result = intersect; + } + else + result = ftv->retTypes; + } + else if (auto e = hasUnificationTooComplex(innerState.errors)) + { + reportError(*e); + return singletonTypes->errorRecoveryTypePack(args); + } + } + } + } + + if (result) + return *result; + else if (firstFun) + { + // TODO: better error reporting? + // The logic for error reporting overload resolution + // is currently over in TypeInfer.cpp, should we move it? + reportError(TypeError{location, GenericError{"No matching overload."}}); + return singletonTypes->errorRecoveryTypePack(firstFun->retTypes); + } + else + { + reportError(TypeError{location, CannotCallNonFunction{function}}); + return singletonTypes->errorRecoveryTypePack(); + } +} + bool Unifier::canCacheResult(TypeId subTy, TypeId superTy) { bool* superTyInfo = sharedState.skipCacheForType.find(superTy); @@ -1253,14 +1473,38 @@ void Unifier::tryUnifyFunctions(TypeId subTy, TypeId superTy, bool isFunctionCal ice("passed non-function types to unifyFunction"); size_t numGenerics = superFunction->generics.size(); - if (numGenerics != subFunction->generics.size()) + size_t numGenericPacks = superFunction->genericPacks.size(); + + bool shouldInstantiate = (numGenerics == 0 && subFunction->generics.size() > 0) || (numGenericPacks == 0 && subFunction->genericPacks.size() > 0); + + if (FFlag::LuauInstantiateInSubtyping && variance == Covariant && shouldInstantiate) + { + Instantiation instantiation{&log, types, scope->level, scope}; + + std::optional instantiated = instantiation.substitute(subTy); + if (instantiated.has_value()) + { + subFunction = log.getMutable(*instantiated); + + if (!subFunction) + ice("instantiation made a function type into a non-function type in unifyFunction"); + + numGenerics = std::min(superFunction->generics.size(), subFunction->generics.size()); + numGenericPacks = std::min(superFunction->genericPacks.size(), subFunction->genericPacks.size()); + + } + else + { + reportError(TypeError{location, UnificationTooComplex{}}); + } + } + else if (numGenerics != subFunction->generics.size()) { numGenerics = std::min(superFunction->generics.size(), subFunction->generics.size()); reportError(TypeError{location, TypeMismatch{superTy, subTy, "different number of generic type parameters"}}); } - size_t numGenericPacks = superFunction->genericPacks.size(); if (numGenericPacks != subFunction->genericPacks.size()) { numGenericPacks = std::min(superFunction->genericPacks.size(), subFunction->genericPacks.size()); @@ -1376,6 +1620,27 @@ void Unifier::tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) std::vector missingProperties; std::vector extraProperties; + if (FFlag::LuauInstantiateInSubtyping) + { + if (variance == Covariant && subTable->state == TableState::Generic && superTable->state != TableState::Generic) + { + Instantiation instantiation{&log, types, subTable->level, scope}; + + std::optional instantiated = instantiation.substitute(subTy); + if (instantiated.has_value()) + { + subTable = log.getMutable(*instantiated); + + if (!subTable) + ice("instantiation made a table type into a non-table type in tryUnifyTables"); + } + else + { + reportError(TypeError{location, UnificationTooComplex{}}); + } + } + } + // Optimization: First test that the property sets are compatible without doing any recursive unification if (!subTable->indexer && subTable->state != TableState::Free) { @@ -2344,8 +2609,9 @@ bool Unifier::occursCheck(DenseHashSet& seen, TypePackId needle, Typ Unifier Unifier::makeChildUnifier() { - Unifier u = Unifier{types, singletonTypes, mode, scope, location, variance, sharedState, &log}; + Unifier u = Unifier{normalizer, mode, scope, location, variance, &log}; u.anyIsTop = anyIsTop; + u.normalize = normalize; return u; } diff --git a/Ast/src/Parser.cpp b/Ast/src/Parser.cpp index cf3eaaae..c20c0847 100644 --- a/Ast/src/Parser.cpp +++ b/Ast/src/Parser.cpp @@ -25,6 +25,8 @@ LUAU_DYNAMIC_FASTFLAGVARIABLE(LuaReportParseIntegerIssues, false) LUAU_FASTFLAGVARIABLE(LuauInterpolatedStringBaseSupport, false) LUAU_FASTFLAGVARIABLE(LuauTypeAnnotationLocationChange, false) +LUAU_FASTFLAGVARIABLE(LuauCommaParenWarnings, false) + bool lua_telemetry_parsed_out_of_range_bin_integer = false; bool lua_telemetry_parsed_out_of_range_hex_integer = false; bool lua_telemetry_parsed_double_prefix_hex_integer = false; @@ -1062,6 +1064,12 @@ void Parser::parseExprList(TempVector& result) { nextLexeme(); + if (FFlag::LuauCommaParenWarnings && lexer.current().type == ')') + { + report(lexer.current().location, "Expected expression after ',' but got ')' instead"); + break; + } + result.push_back(parseExpr()); } } @@ -1148,7 +1156,14 @@ AstTypePack* Parser::parseTypeList(TempVector& result, TempVector, AstArray> Parser::parseG } if (lexer.current().type == ',') + { nextLexeme(); + + if (FFlag::LuauCommaParenWarnings && lexer.current().type == '>') + { + report(lexer.current().location, "Expected type after ',' but got '>' instead"); + break; + } + } else break; } diff --git a/CLI/Profiler.cpp b/CLI/Profiler.cpp index 30a171f0..d3ad4e99 100644 --- a/CLI/Profiler.cpp +++ b/CLI/Profiler.cpp @@ -82,11 +82,13 @@ static void profilerLoop() if (now - last >= 1.0 / double(gProfiler.frequency)) { - gProfiler.ticks += uint64_t((now - last) * 1e6); + int64_t ticks = int64_t((now - last) * 1e6); + + gProfiler.ticks += ticks; gProfiler.samples++; gProfiler.callbacks->interrupt = profilerTrigger; - last = now; + last += ticks * 1e-6; } else { diff --git a/CodeGen/include/Luau/AssemblyBuilderX64.h b/CodeGen/include/Luau/AssemblyBuilderX64.h index cb799d31..15db7a15 100644 --- a/CodeGen/include/Luau/AssemblyBuilderX64.h +++ b/CodeGen/include/Luau/AssemblyBuilderX64.h @@ -152,6 +152,7 @@ private: void placeModRegMem(OperandX64 rhs, uint8_t regop); void placeRex(RegisterX64 op); void placeRex(OperandX64 op); + void placeRexNoW(OperandX64 op); void placeRex(RegisterX64 lhs, OperandX64 rhs); void placeVex(OperandX64 dst, OperandX64 src1, OperandX64 src2, bool setW, uint8_t mode, uint8_t prefix); void placeImm8Or32(int32_t imm); diff --git a/CodeGen/include/Luau/CodeAllocator.h b/CodeGen/include/Luau/CodeAllocator.h index 01e13121..2ea64630 100644 --- a/CodeGen/include/Luau/CodeAllocator.h +++ b/CodeGen/include/Luau/CodeAllocator.h @@ -24,13 +24,15 @@ struct CodeAllocator void* context = nullptr; // Called when new block is created to create and setup the unwinding information for all the code in the block - // If data is placed inside the block itself (some platforms require this), we also return 'unwindDataSizeInBlock' - void* (*createBlockUnwindInfo)(void* context, uint8_t* block, size_t blockSize, size_t& unwindDataSizeInBlock) = nullptr; + // 'startOffset' reserves space for data at the beginning of the page + void* (*createBlockUnwindInfo)(void* context, uint8_t* block, size_t blockSize, size_t& startOffset) = nullptr; // Called to destroy unwinding information returned by 'createBlockUnwindInfo' void (*destroyBlockUnwindInfo)(void* context, void* unwindData) = nullptr; - static const size_t kMaxUnwindDataSize = 128; + // Unwind information can be placed inside the block with some implementation-specific reservations at the beginning + // But to simplify block space checks, we limit the max size of all that data + static const size_t kMaxReservedDataSize = 256; bool allocateNewBlock(size_t& unwindInfoSize); diff --git a/CodeGen/include/Luau/CodeBlockUnwind.h b/CodeGen/include/Luau/CodeBlockUnwind.h index ddae33a6..0f7af3ac 100644 --- a/CodeGen/include/Luau/CodeBlockUnwind.h +++ b/CodeGen/include/Luau/CodeBlockUnwind.h @@ -10,7 +10,7 @@ namespace CodeGen { // context must be an UnwindBuilder -void* createBlockUnwindInfo(void* context, uint8_t* block, size_t blockSize, size_t& unwindDataSizeInBlock); +void* createBlockUnwindInfo(void* context, uint8_t* block, size_t blockSize, size_t& startOffset); void destroyBlockUnwindInfo(void* context, void* unwindData); } // namespace CodeGen diff --git a/CodeGen/include/Luau/UnwindBuilder.h b/CodeGen/include/Luau/UnwindBuilder.h index c6f611b0..b7237318 100644 --- a/CodeGen/include/Luau/UnwindBuilder.h +++ b/CodeGen/include/Luau/UnwindBuilder.h @@ -14,7 +14,10 @@ namespace CodeGen class UnwindBuilder { public: - virtual ~UnwindBuilder() {} + virtual ~UnwindBuilder() = default; + + virtual void setBeginOffset(size_t beginOffset) = 0; + virtual size_t getBeginOffset() const = 0; virtual void start() = 0; diff --git a/CodeGen/include/Luau/UnwindBuilderDwarf2.h b/CodeGen/include/Luau/UnwindBuilderDwarf2.h index 25dbc55b..dab6e957 100644 --- a/CodeGen/include/Luau/UnwindBuilderDwarf2.h +++ b/CodeGen/include/Luau/UnwindBuilderDwarf2.h @@ -12,6 +12,9 @@ namespace CodeGen class UnwindBuilderDwarf2 : public UnwindBuilder { public: + void setBeginOffset(size_t beginOffset) override; + size_t getBeginOffset() const override; + void start() override; void spill(int espOffset, RegisterX64 reg) override; @@ -26,6 +29,8 @@ public: void finalize(char* target, void* funcAddress, size_t funcSize) const override; private: + size_t beginOffset = 0; + static const unsigned kRawDataLimit = 128; uint8_t rawData[kRawDataLimit]; uint8_t* pos = rawData; diff --git a/CodeGen/include/Luau/UnwindBuilderWin.h b/CodeGen/include/Luau/UnwindBuilderWin.h index 801eb6e4..00513771 100644 --- a/CodeGen/include/Luau/UnwindBuilderWin.h +++ b/CodeGen/include/Luau/UnwindBuilderWin.h @@ -22,6 +22,9 @@ struct UnwindCodeWin class UnwindBuilderWin : public UnwindBuilder { public: + void setBeginOffset(size_t beginOffset) override; + size_t getBeginOffset() const override; + void start() override; void spill(int espOffset, RegisterX64 reg) override; @@ -36,6 +39,8 @@ public: void finalize(char* target, void* funcAddress, size_t funcSize) const override; private: + size_t beginOffset = 0; + // Windows unwind codes are written in reverse, so we have to collect them all first std::vector unwindCodes; diff --git a/CodeGen/src/AssemblyBuilderX64.cpp b/CodeGen/src/AssemblyBuilderX64.cpp index 32325b0d..cd3079ac 100644 --- a/CodeGen/src/AssemblyBuilderX64.cpp +++ b/CodeGen/src/AssemblyBuilderX64.cpp @@ -354,10 +354,15 @@ void AssemblyBuilderX64::jmp(Label& label) void AssemblyBuilderX64::jmp(OperandX64 op) { + LUAU_ASSERT((op.cat == CategoryX64::reg ? op.base.size : op.memSize) == SizeX64::qword); + if (logText) log("jmp", op); - placeRex(op); + // Indirect absolute calls always work in 64 bit width mode, so REX.W is optional + // While we could keep an optional prefix, in Windows x64 ABI it signals a tail call return statement to the unwinder + placeRexNoW(op); + place(0xff); placeModRegMem(op, 4); commit(); @@ -376,10 +381,14 @@ void AssemblyBuilderX64::call(Label& label) void AssemblyBuilderX64::call(OperandX64 op) { + LUAU_ASSERT((op.cat == CategoryX64::reg ? op.base.size : op.memSize) == SizeX64::qword); + if (logText) log("call", op); - placeRex(op); + // Indirect absolute calls always work in 64 bit width mode, so REX.W is optional + placeRexNoW(op); + place(0xff); placeModRegMem(op, 2); commit(); @@ -838,6 +847,21 @@ void AssemblyBuilderX64::placeRex(OperandX64 op) place(code | 0x40); } +void AssemblyBuilderX64::placeRexNoW(OperandX64 op) +{ + uint8_t code = 0; + + if (op.cat == CategoryX64::reg) + code = REX_B(op.base); + else if (op.cat == CategoryX64::mem) + code = REX_X(op.index) | REX_B(op.base); + else + LUAU_ASSERT(!"No encoding for left operand of this category"); + + if (code != 0) + place(code | 0x40); +} + void AssemblyBuilderX64::placeRex(RegisterX64 lhs, OperandX64 rhs) { uint8_t code = REX_W(lhs.size == SizeX64::qword); diff --git a/CodeGen/src/CodeAllocator.cpp b/CodeGen/src/CodeAllocator.cpp index aacf40a3..b3787d1f 100644 --- a/CodeGen/src/CodeAllocator.cpp +++ b/CodeGen/src/CodeAllocator.cpp @@ -91,7 +91,7 @@ CodeAllocator::CodeAllocator(size_t blockSize, size_t maxTotalSize) : blockSize(blockSize) , maxTotalSize(maxTotalSize) { - LUAU_ASSERT(blockSize > kMaxUnwindDataSize); + LUAU_ASSERT(blockSize > kMaxReservedDataSize); LUAU_ASSERT(maxTotalSize >= blockSize); } @@ -116,15 +116,15 @@ bool CodeAllocator::allocate( size_t totalSize = alignedDataSize + codeSize; // Function has to fit into a single block with unwinding information - if (totalSize > blockSize - kMaxUnwindDataSize) + if (totalSize > blockSize - kMaxReservedDataSize) return false; - size_t unwindInfoSize = 0; + size_t startOffset = 0; // We might need a new block if (totalSize > size_t(blockEnd - blockPos)) { - if (!allocateNewBlock(unwindInfoSize)) + if (!allocateNewBlock(startOffset)) return false; LUAU_ASSERT(totalSize <= size_t(blockEnd - blockPos)); @@ -132,20 +132,20 @@ bool CodeAllocator::allocate( LUAU_ASSERT((uintptr_t(blockPos) & (kPageSize - 1)) == 0); // Allocation starts on page boundary - size_t dataOffset = unwindInfoSize + alignedDataSize - dataSize; - size_t codeOffset = unwindInfoSize + alignedDataSize; + size_t dataOffset = startOffset + alignedDataSize - dataSize; + size_t codeOffset = startOffset + alignedDataSize; if (dataSize) memcpy(blockPos + dataOffset, data, dataSize); if (codeSize) memcpy(blockPos + codeOffset, code, codeSize); - size_t pageAlignedSize = alignToPageSize(unwindInfoSize + totalSize); + size_t pageAlignedSize = alignToPageSize(startOffset + totalSize); makePagesExecutable(blockPos, pageAlignedSize); flushInstructionCache(blockPos + codeOffset, codeSize); - result = blockPos + unwindInfoSize; + result = blockPos + startOffset; resultSize = totalSize; resultCodeStart = blockPos + codeOffset; @@ -190,7 +190,7 @@ bool CodeAllocator::allocateNewBlock(size_t& unwindInfoSize) // 'Round up' to preserve 16 byte alignment of the following data and code unwindInfoSize = (unwindInfoSize + 15) & ~15; - LUAU_ASSERT(unwindInfoSize <= kMaxUnwindDataSize); + LUAU_ASSERT(unwindInfoSize <= kMaxReservedDataSize); if (!unwindInfo) return false; diff --git a/CodeGen/src/CodeBlockUnwind.cpp b/CodeGen/src/CodeBlockUnwind.cpp index 6191cee4..c045ba6b 100644 --- a/CodeGen/src/CodeBlockUnwind.cpp +++ b/CodeGen/src/CodeBlockUnwind.cpp @@ -51,7 +51,7 @@ namespace Luau namespace CodeGen { -void* createBlockUnwindInfo(void* context, uint8_t* block, size_t blockSize, size_t& unwindDataSizeInBlock) +void* createBlockUnwindInfo(void* context, uint8_t* block, size_t blockSize, size_t& beginOffset) { #if defined(_WIN32) && defined(_M_X64) UnwindBuilder* unwind = (UnwindBuilder*)context; @@ -75,7 +75,7 @@ void* createBlockUnwindInfo(void* context, uint8_t* block, size_t blockSize, siz return nullptr; } - unwindDataSizeInBlock = unwindSize; + beginOffset = unwindSize + unwind->getBeginOffset(); return block; #elif !defined(_WIN32) UnwindBuilder* unwind = (UnwindBuilder*)context; @@ -94,7 +94,7 @@ void* createBlockUnwindInfo(void* context, uint8_t* block, size_t blockSize, siz __register_frame(unwindData); #endif - unwindDataSizeInBlock = unwindSize; + beginOffset = unwindSize + unwind->getBeginOffset(); return block; #endif diff --git a/CodeGen/src/UnwindBuilderDwarf2.cpp b/CodeGen/src/UnwindBuilderDwarf2.cpp index f3886d9c..8d06864e 100644 --- a/CodeGen/src/UnwindBuilderDwarf2.cpp +++ b/CodeGen/src/UnwindBuilderDwarf2.cpp @@ -129,6 +129,16 @@ namespace Luau namespace CodeGen { +void UnwindBuilderDwarf2::setBeginOffset(size_t beginOffset) +{ + this->beginOffset = beginOffset; +} + +size_t UnwindBuilderDwarf2::getBeginOffset() const +{ + return beginOffset; +} + void UnwindBuilderDwarf2::start() { uint8_t* cieLength = pos; diff --git a/CodeGen/src/UnwindBuilderWin.cpp b/CodeGen/src/UnwindBuilderWin.cpp index 5405fcf2..1b3279e8 100644 --- a/CodeGen/src/UnwindBuilderWin.cpp +++ b/CodeGen/src/UnwindBuilderWin.cpp @@ -32,6 +32,16 @@ struct UnwindInfoWin uint8_t frameregoff : 4; }; +void UnwindBuilderWin::setBeginOffset(size_t beginOffset) +{ + this->beginOffset = beginOffset; +} + +size_t UnwindBuilderWin::getBeginOffset() const +{ + return beginOffset; +} + void UnwindBuilderWin::start() { stackOffset = 8; // Return address was pushed by calling the function diff --git a/Common/include/Luau/ExperimentalFlags.h b/Common/include/Luau/ExperimentalFlags.h index ce47cd9a..7cff70a4 100644 --- a/Common/include/Luau/ExperimentalFlags.h +++ b/Common/include/Luau/ExperimentalFlags.h @@ -13,6 +13,7 @@ inline bool isFlagExperimental(const char* flag) static const char* kList[] = { "LuauLowerBoundsCalculation", "LuauInterpolatedStringBaseSupport", + "LuauInstantiateInSubtyping", // requires some fixes to lua-apps code // makes sure we always have at least one entry nullptr, }; diff --git a/Makefile b/Makefile index 5b8eb935..84007179 100644 --- a/Makefile +++ b/Makefile @@ -4,6 +4,7 @@ MAKEFLAGS+=-r -j8 COMMA=, config=debug +protobuf=system BUILD=build/$(config) @@ -95,12 +96,22 @@ ifeq ($(config),fuzz) CXX=clang++ # our fuzzing infra relies on llvm fuzzer CXXFLAGS+=-fsanitize=address,fuzzer -Ibuild/libprotobuf-mutator -O2 LDFLAGS+=-fsanitize=address,fuzzer + LPROTOBUF=-lprotobuf + DPROTOBUF=-D CMAKE_BUILD_TYPE=Release -D LIB_PROTO_MUTATOR_TESTING=OFF + EPROTOC=protoc endif ifeq ($(config),profile) CXXFLAGS+=-O2 -DNDEBUG -gdwarf-4 -DCALLGRIND=1 endif +ifeq ($(protobuf),download) + CXXFLAGS+=-Ibuild/libprotobuf-mutator/external.protobuf/include + LPROTOBUF=build/libprotobuf-mutator/external.protobuf/lib/libprotobuf.a + DPROTOBUF+=-D LIB_PROTO_MUTATOR_DOWNLOAD_PROTOBUF=ON + EPROTOC=../build/libprotobuf-mutator/external.protobuf/bin/protoc +endif + # target-specific flags $(AST_OBJECTS): CXXFLAGS+=-std=c++17 -ICommon/include -IAst/include $(COMPILER_OBJECTS): CXXFLAGS+=-std=c++17 -ICompiler/include -ICommon/include -IAst/include @@ -115,7 +126,7 @@ $(FUZZ_OBJECTS): CXXFLAGS+=-std=c++17 -ICommon/include -IAst/include -ICompiler/ $(TESTS_TARGET): LDFLAGS+=-lpthread $(REPL_CLI_TARGET): LDFLAGS+=-lpthread -fuzz-proto fuzz-prototest: LDFLAGS+=build/libprotobuf-mutator/src/libfuzzer/libprotobuf-mutator-libfuzzer.a build/libprotobuf-mutator/src/libprotobuf-mutator.a -lprotobuf +fuzz-proto fuzz-prototest: LDFLAGS+=build/libprotobuf-mutator/src/libfuzzer/libprotobuf-mutator-libfuzzer.a build/libprotobuf-mutator/src/libprotobuf-mutator.a $(LPROTOBUF) # pseudo targets .PHONY: all test clean coverage format luau-size aliases @@ -199,7 +210,7 @@ $(BUILD)/%.c.o: %.c # protobuf fuzzer setup fuzz/luau.pb.cpp: fuzz/luau.proto build/libprotobuf-mutator - cd fuzz && protoc luau.proto --cpp_out=. + cd fuzz && $(EPROTOC) luau.proto --cpp_out=. mv fuzz/luau.pb.cc fuzz/luau.pb.cpp $(BUILD)/fuzz/proto.cpp.o: fuzz/luau.pb.cpp @@ -207,7 +218,7 @@ $(BUILD)/fuzz/protoprint.cpp.o: fuzz/luau.pb.cpp build/libprotobuf-mutator: git clone https://github.com/google/libprotobuf-mutator build/libprotobuf-mutator - CXX= cmake -S build/libprotobuf-mutator -B build/libprotobuf-mutator -D CMAKE_BUILD_TYPE=Release -D LIB_PROTO_MUTATOR_TESTING=OFF + CXX= cmake -S build/libprotobuf-mutator -B build/libprotobuf-mutator $(DPROTOBUF) make -C build/libprotobuf-mutator -j8 # picks up include dependencies for all object files diff --git a/VM/include/lua.h b/VM/include/lua.h index 33e68517..ea658a41 100644 --- a/VM/include/lua.h +++ b/VM/include/lua.h @@ -401,13 +401,15 @@ struct lua_Debug const char* name; // (n) const char* what; // (s) `Lua', `C', `main', `tail' const char* source; // (s) + const char* short_src; // (s) int linedefined; // (s) int currentline; // (l) unsigned char nupvals; // (u) number of upvalues unsigned char nparams; // (a) number of parameters char isvararg; // (a) - char short_src[LUA_IDSIZE]; // (s) void* userdata; // only valid in luau_callhook + + char ssbuf[LUA_IDSIZE]; }; // }====================================================================== diff --git a/VM/src/ldebug.cpp b/VM/src/ldebug.cpp index e695cd2b..82af5d38 100644 --- a/VM/src/ldebug.cpp +++ b/VM/src/ldebug.cpp @@ -12,6 +12,8 @@ #include #include +LUAU_FASTFLAGVARIABLE(LuauFasterGetInfo, false) + static const char* getfuncname(Closure* f); static int currentpc(lua_State* L, CallInfo* ci) @@ -89,9 +91,9 @@ const char* lua_setlocal(lua_State* L, int level, int n) return name; } -static int auxgetinfo(lua_State* L, const char* what, lua_Debug* ar, Closure* f, CallInfo* ci) +static Closure* auxgetinfo(lua_State* L, const char* what, lua_Debug* ar, Closure* f, CallInfo* ci) { - int status = 1; + Closure* cl = NULL; for (; *what; what++) { switch (*what) @@ -103,14 +105,23 @@ static int auxgetinfo(lua_State* L, const char* what, lua_Debug* ar, Closure* f, ar->source = "=[C]"; ar->what = "C"; ar->linedefined = -1; + if (FFlag::LuauFasterGetInfo) + ar->short_src = "[C]"; } else { - ar->source = getstr(f->l.p->source); + TString* source = f->l.p->source; + ar->source = getstr(source); ar->what = "Lua"; ar->linedefined = f->l.p->linedefined; + if (FFlag::LuauFasterGetInfo) + ar->short_src = luaO_chunkid(ar->ssbuf, sizeof(ar->ssbuf), getstr(source), source->len); + } + if (!FFlag::LuauFasterGetInfo) + { + luaO_chunkid(ar->ssbuf, LUA_IDSIZE, ar->source, 0); + ar->short_src = ar->ssbuf; } - luaO_chunkid(ar->short_src, ar->source, LUA_IDSIZE); break; } case 'l': @@ -150,10 +161,15 @@ static int auxgetinfo(lua_State* L, const char* what, lua_Debug* ar, Closure* f, ar->name = ci ? getfuncname(ci_func(ci)) : getfuncname(f); break; } + case 'f': + { + cl = f; + break; + } default:; } } - return status; + return cl; } int lua_stackdepth(lua_State* L) @@ -163,7 +179,6 @@ int lua_stackdepth(lua_State* L) int lua_getinfo(lua_State* L, int level, const char* what, lua_Debug* ar) { - int status = 0; Closure* f = NULL; CallInfo* ci = NULL; if (level < 0) @@ -180,15 +195,28 @@ int lua_getinfo(lua_State* L, int level, const char* what, lua_Debug* ar) } if (f) { - status = auxgetinfo(L, what, ar, f, ci); - if (strchr(what, 'f')) + if (FFlag::LuauFasterGetInfo) { - luaC_threadbarrier(L); - setclvalue(L, L->top, f); - incr_top(L); + // auxgetinfo fills ar and optionally requests to put closure on stack + if (Closure* fcl = auxgetinfo(L, what, ar, f, ci)) + { + luaC_threadbarrier(L); + setclvalue(L, L->top, fcl); + incr_top(L); + } + } + else + { + auxgetinfo(L, what, ar, f, ci); + if (strchr(what, 'f')) + { + luaC_threadbarrier(L); + setclvalue(L, L->top, f); + incr_top(L); + } } } - return status; + return f ? 1 : 0; } static const char* getfuncname(Closure* cl) @@ -284,10 +312,11 @@ static void pusherror(lua_State* L, const char* msg) CallInfo* ci = L->ci; if (isLua(ci)) { - char buff[LUA_IDSIZE]; // add file:line information - luaO_chunkid(buff, getstr(getluaproto(ci)->source), LUA_IDSIZE); + TString* source = getluaproto(ci)->source; + char chunkbuf[LUA_IDSIZE]; // add file:line information + const char* chunkid = luaO_chunkid(chunkbuf, sizeof(chunkbuf), getstr(source), source->len); int line = currentline(L, ci); - luaO_pushfstring(L, "%s:%d: %s", buff, line, msg); + luaO_pushfstring(L, "%s:%d: %s", chunkid, line, msg); } else { diff --git a/VM/src/lgc.cpp b/VM/src/lgc.cpp index 4b9fbb69..f50b33d1 100644 --- a/VM/src/lgc.cpp +++ b/VM/src/lgc.cpp @@ -13,8 +13,6 @@ #include -LUAU_FASTFLAGVARIABLE(LuauBetterThreadMark, false) - /* * Luau uses an incremental non-generational non-moving mark&sweep garbage collector. * @@ -473,54 +471,25 @@ static size_t propagatemark(global_State* g) bool active = th->isactive || th == th->global->mainthread; - if (FFlag::LuauBetterThreadMark) + traversestack(g, th); + + // active threads will need to be rescanned later to mark new stack writes so we mark them gray again + if (active) { - traversestack(g, th); + th->gclist = g->grayagain; + g->grayagain = o; - // active threads will need to be rescanned later to mark new stack writes so we mark them gray again - if (active) - { - th->gclist = g->grayagain; - g->grayagain = o; - - black2gray(o); - } - - // the stack needs to be cleared after the last modification of the thread state before sweep begins - // if the thread is inactive, we might not see the thread in this cycle so we must clear it now - if (!active || g->gcstate == GCSatomic) - clearstack(th); - - // we could shrink stack at any time but we opt to do it during initial mark to do that just once per cycle - if (g->gcstate == GCSpropagate) - shrinkstack(th); + black2gray(o); } - else - { - // TODO: Refactor this logic! - if (!active && g->gcstate == GCSpropagate) - { - traversestack(g, th); - clearstack(th); - } - else - { - th->gclist = g->grayagain; - g->grayagain = o; - black2gray(o); + // the stack needs to be cleared after the last modification of the thread state before sweep begins + // if the thread is inactive, we might not see the thread in this cycle so we must clear it now + if (!active || g->gcstate == GCSatomic) + clearstack(th); - traversestack(g, th); - - // final traversal? - if (g->gcstate == GCSatomic) - clearstack(th); - } - - // we could shrink stack at any time but we opt to skip it during atomic since it's redundant to do that more than once per cycle - if (g->gcstate != GCSatomic) - shrinkstack(th); - } + // we could shrink stack at any time but we opt to do it during initial mark to do that just once per cycle + if (g->gcstate == GCSpropagate) + shrinkstack(th); return sizeof(lua_State) + sizeof(TValue) * th->stacksize + sizeof(CallInfo) * th->size_ci; } diff --git a/VM/src/lobject.cpp b/VM/src/lobject.cpp index f5f1cd0e..8b3e4783 100644 --- a/VM/src/lobject.cpp +++ b/VM/src/lobject.cpp @@ -15,6 +15,8 @@ +LUAU_FASTFLAG(LuauFasterGetInfo) + const TValue luaO_nilobject_ = {{NULL}, {0}, LUA_TNIL}; int luaO_log2(unsigned int x) @@ -117,44 +119,68 @@ const char* luaO_pushfstring(lua_State* L, const char* fmt, ...) return msg; } -void luaO_chunkid(char* out, const char* source, size_t bufflen) +const char* luaO_chunkid(char* buf, size_t buflen, const char* source, size_t srclen) { if (*source == '=') { - source++; // skip the `=' - size_t srclen = strlen(source); - size_t dstlen = srclen < bufflen ? srclen : bufflen - 1; - memcpy(out, source, dstlen); - out[dstlen] = '\0'; + if (FFlag::LuauFasterGetInfo) + { + if (srclen <= buflen) + return source + 1; + // truncate the part after = + memcpy(buf, source + 1, buflen - 1); + buf[buflen - 1] = '\0'; + } + else + { + source++; // skip the `=' + size_t len = strlen(source); + size_t dstlen = len < buflen ? len : buflen - 1; + memcpy(buf, source, dstlen); + buf[dstlen] = '\0'; + } } else if (*source == '@') { - size_t l; - source++; // skip the `@' - bufflen -= sizeof("..."); - l = strlen(source); - strcpy(out, ""); - if (l > bufflen) + if (FFlag::LuauFasterGetInfo) { - source += (l - bufflen); // get last part of file name - strcat(out, "..."); - } - strcat(out, source); - } - else - { // out = [string "string"] - size_t len = strcspn(source, "\n\r"); // stop at first newline - bufflen -= sizeof("[string \"...\"]"); - if (len > bufflen) - len = bufflen; - strcpy(out, "[string \""); - if (source[len] != '\0') - { // must truncate? - strncat(out, source, len); - strcat(out, "..."); + if (srclen <= buflen) + return source + 1; + // truncate the part after @ + memcpy(buf, "...", 3); + memcpy(buf + 3, source + srclen - (buflen - 4), buflen - 4); + buf[buflen - 1] = '\0'; } else - strcat(out, source); - strcat(out, "\"]"); + { + size_t l; + source++; // skip the `@' + buflen -= sizeof("..."); + l = strlen(source); + strcpy(buf, ""); + if (l > buflen) + { + source += (l - buflen); // get last part of file name + strcat(buf, "..."); + } + strcat(buf, source); + } } + else + { // buf = [string "string"] + size_t len = strcspn(source, "\n\r"); // stop at first newline + buflen -= sizeof("[string \"...\"]"); + if (len > buflen) + len = buflen; + strcpy(buf, "[string \""); + if (source[len] != '\0') + { // must truncate? + strncat(buf, source, len); + strcat(buf, "..."); + } + else + strcat(buf, source); + strcat(buf, "\"]"); + } + return buf; } diff --git a/VM/src/lobject.h b/VM/src/lobject.h index 5f5e7b1c..41bf3386 100644 --- a/VM/src/lobject.h +++ b/VM/src/lobject.h @@ -460,4 +460,4 @@ LUAI_FUNC int luaO_rawequalKey(const TKey* t1, const TValue* t2); LUAI_FUNC int luaO_str2d(const char* s, double* result); LUAI_FUNC const char* luaO_pushvfstring(lua_State* L, const char* fmt, va_list argp); LUAI_FUNC const char* luaO_pushfstring(lua_State* L, const char* fmt, ...); -LUAI_FUNC void luaO_chunkid(char* out, const char* source, size_t len); +LUAI_FUNC const char* luaO_chunkid(char* buf, size_t buflen, const char* source, size_t srclen); diff --git a/VM/src/lvmexecute.cpp b/VM/src/lvmexecute.cpp index 490358c4..7ee3ee9b 100644 --- a/VM/src/lvmexecute.cpp +++ b/VM/src/lvmexecute.cpp @@ -781,6 +781,7 @@ reentry: default: LUAU_ASSERT(!"Unknown upvalue capture type"); + LUAU_UNREACHABLE(); // improves switch() codegen by eliding opcode bounds checks } } @@ -1184,7 +1185,9 @@ reentry: // slow path after switch() break; - default:; + default: + LUAU_ASSERT(!"Unknown value type"); + LUAU_UNREACHABLE(); // improves switch() codegen by eliding opcode bounds checks } // slow-path: tables with metatables and userdata values @@ -1296,7 +1299,9 @@ reentry: // slow path after switch() break; - default:; + default: + LUAU_ASSERT(!"Unknown value type"); + LUAU_UNREACHABLE(); // improves switch() codegen by eliding opcode bounds checks } // slow-path: tables with metatables and userdata values diff --git a/VM/src/lvmload.cpp b/VM/src/lvmload.cpp index bd40bad2..3edec688 100644 --- a/VM/src/lvmload.cpp +++ b/VM/src/lvmload.cpp @@ -148,16 +148,16 @@ int luau_load(lua_State* L, const char* chunkname, const char* data, size_t size // 0 means the rest of the bytecode is the error message if (version == 0) { - char chunkid[LUA_IDSIZE]; - luaO_chunkid(chunkid, chunkname, LUA_IDSIZE); + char chunkbuf[LUA_IDSIZE]; + const char* chunkid = luaO_chunkid(chunkbuf, sizeof(chunkbuf), chunkname, strlen(chunkname)); lua_pushfstring(L, "%s%.*s", chunkid, int(size - offset), data + offset); return 1; } if (version < LBC_VERSION_MIN || version > LBC_VERSION_MAX) { - char chunkid[LUA_IDSIZE]; - luaO_chunkid(chunkid, chunkname, LUA_IDSIZE); + char chunkbuf[LUA_IDSIZE]; + const char* chunkid = luaO_chunkid(chunkbuf, sizeof(chunkbuf), chunkname, strlen(chunkname)); lua_pushfstring(L, "%s: bytecode version mismatch (expected [%d..%d], got %d)", chunkid, LBC_VERSION_MIN, LBC_VERSION_MAX, version); return 1; } diff --git a/VM/src/lvmutils.cpp b/VM/src/lvmutils.cpp index 5c555158..05d39754 100644 --- a/VM/src/lvmutils.cpp +++ b/VM/src/lvmutils.cpp @@ -220,8 +220,13 @@ int luaV_strcmp(const TString* ls, const TString* rs) return 0; const char* l = getstr(ls); - size_t ll = ls->len; const char* r = getstr(rs); + + // always safe to read one character because even empty strings are nul terminated + if (*l != *r) + return uint8_t(*l) - uint8_t(*r); + + size_t ll = ls->len; size_t lr = rs->len; size_t lmin = ll < lr ? ll : lr; diff --git a/tests/AssemblyBuilderX64.test.cpp b/tests/AssemblyBuilderX64.test.cpp index 08f241ed..a0510887 100644 --- a/tests/AssemblyBuilderX64.test.cpp +++ b/tests/AssemblyBuilderX64.test.cpp @@ -240,12 +240,12 @@ TEST_CASE_FIXTURE(AssemblyBuilderX64Fixture, "FormsOfLea") TEST_CASE_FIXTURE(AssemblyBuilderX64Fixture, "FormsOfAbsoluteJumps") { - SINGLE_COMPARE(jmp(rax), 0x48, 0xff, 0xe0); - SINGLE_COMPARE(jmp(r14), 0x49, 0xff, 0xe6); - SINGLE_COMPARE(jmp(qword[r14 + rdx * 4]), 0x49, 0xff, 0x24, 0x96); - SINGLE_COMPARE(call(rax), 0x48, 0xff, 0xd0); - SINGLE_COMPARE(call(r14), 0x49, 0xff, 0xd6); - SINGLE_COMPARE(call(qword[r14 + rdx * 4]), 0x49, 0xff, 0x14, 0x96); + SINGLE_COMPARE(jmp(rax), 0xff, 0xe0); + SINGLE_COMPARE(jmp(r14), 0x41, 0xff, 0xe6); + SINGLE_COMPARE(jmp(qword[r14 + rdx * 4]), 0x41, 0xff, 0x24, 0x96); + SINGLE_COMPARE(call(rax), 0xff, 0xd0); + SINGLE_COMPARE(call(r14), 0x41, 0xff, 0xd6); + SINGLE_COMPARE(call(qword[r14 + rdx * 4]), 0x41, 0xff, 0x14, 0x96); } TEST_CASE_FIXTURE(AssemblyBuilderX64Fixture, "FormsOfImul") diff --git a/tests/Autocomplete.test.cpp b/tests/Autocomplete.test.cpp index a64d372f..9a5c3411 100644 --- a/tests/Autocomplete.test.cpp +++ b/tests/Autocomplete.test.cpp @@ -2955,8 +2955,6 @@ local abc = b@1 TEST_CASE_FIXTURE(ACFixture, "no_incompatible_self_calls_on_class") { - ScopedFastFlag selfCallAutocompleteFix3{"LuauSelfCallAutocompleteFix3", true}; - loadDefinition(R"( declare class Foo function one(self): number @@ -2995,8 +2993,6 @@ t.@1 TEST_CASE_FIXTURE(ACFixture, "do_compatible_self_calls") { - ScopedFastFlag selfCallAutocompleteFix3{"LuauSelfCallAutocompleteFix3", true}; - check(R"( local t = {} function t:m() end @@ -3011,8 +3007,6 @@ t:@1 TEST_CASE_FIXTURE(ACFixture, "no_incompatible_self_calls") { - ScopedFastFlag selfCallAutocompleteFix3{"LuauSelfCallAutocompleteFix3", true}; - check(R"( local t = {} function t.m() end @@ -3027,8 +3021,6 @@ t:@1 TEST_CASE_FIXTURE(ACFixture, "no_incompatible_self_calls_2") { - ScopedFastFlag selfCallAutocompleteFix3{"LuauSelfCallAutocompleteFix3", true}; - check(R"( local f: (() -> number) & ((number) -> number) = function(x: number?) return 2 end local t = {} @@ -3059,8 +3051,6 @@ t:@1 TEST_CASE_FIXTURE(ACFixture, "no_wrong_compatible_self_calls_with_generics") { - ScopedFastFlag selfCallAutocompleteFix3{"LuauSelfCallAutocompleteFix3", true}; - check(R"( local t = {} function t.m(a: T) end @@ -3076,8 +3066,6 @@ t:@1 TEST_CASE_FIXTURE(ACFixture, "string_prim_self_calls_are_fine") { - ScopedFastFlag selfCallAutocompleteFix3{"LuauSelfCallAutocompleteFix3", true}; - check(R"( local s = "hello" s:@1 @@ -3095,8 +3083,6 @@ s:@1 TEST_CASE_FIXTURE(ACFixture, "string_prim_non_self_calls_are_avoided") { - ScopedFastFlag selfCallAutocompleteFix3{"LuauSelfCallAutocompleteFix3", true}; - check(R"( local s = "hello" s.@1 @@ -3112,8 +3098,6 @@ s.@1 TEST_CASE_FIXTURE(ACBuiltinsFixture, "library_non_self_calls_are_fine") { - ScopedFastFlag selfCallAutocompleteFix3{"LuauSelfCallAutocompleteFix3", true}; - check(R"( string.@1 )"); @@ -3143,8 +3127,6 @@ table.@1 TEST_CASE_FIXTURE(ACBuiltinsFixture, "library_self_calls_are_invalid") { - ScopedFastFlag selfCallAutocompleteFix3{"LuauSelfCallAutocompleteFix3", true}; - check(R"( string:@1 )"); diff --git a/tests/CodeAllocator.test.cpp b/tests/CodeAllocator.test.cpp index 758fb44c..4e553f05 100644 --- a/tests/CodeAllocator.test.cpp +++ b/tests/CodeAllocator.test.cpp @@ -96,12 +96,12 @@ TEST_CASE("CodeAllocationWithUnwindCallbacks") data.resize(8); allocator.context = &info; - allocator.createBlockUnwindInfo = [](void* context, uint8_t* block, size_t blockSize, size_t& unwindDataSizeInBlock) -> void* { + allocator.createBlockUnwindInfo = [](void* context, uint8_t* block, size_t blockSize, size_t& beginOffset) -> void* { Info& info = *(Info*)context; CHECK(info.unwind.size() == 8); memcpy(block, info.unwind.data(), info.unwind.size()); - unwindDataSizeInBlock = 8; + beginOffset = 8; info.block = block; @@ -194,10 +194,12 @@ TEST_CASE("Dwarf2UnwindCodesX64") // Windows x64 ABI constexpr RegisterX64 rArg1 = rcx; constexpr RegisterX64 rArg2 = rdx; +constexpr RegisterX64 rArg3 = r8; #else // System V AMD64 ABI constexpr RegisterX64 rArg1 = rdi; constexpr RegisterX64 rArg2 = rsi; +constexpr RegisterX64 rArg3 = rdx; #endif constexpr RegisterX64 rNonVol1 = r12; @@ -313,6 +315,119 @@ TEST_CASE("GeneratedCodeExecutionWithThrow") } } +TEST_CASE("GeneratedCodeExecutionWithThrowOutsideTheGate") +{ + AssemblyBuilderX64 build(/* logText= */ false); + +#if defined(_WIN32) + std::unique_ptr unwind = std::make_unique(); +#else + std::unique_ptr unwind = std::make_unique(); +#endif + + unwind->start(); + + // Prologue (some of these registers don't have to be saved, but we want to have a big prologue) + build.push(r10); + unwind->save(r10); + build.push(r11); + unwind->save(r11); + build.push(r12); + unwind->save(r12); + build.push(r13); + unwind->save(r13); + build.push(r14); + unwind->save(r14); + build.push(r15); + unwind->save(r15); + build.push(rbp); + unwind->save(rbp); + + int stackSize = 64; + int localsSize = 16; + + build.sub(rsp, stackSize + localsSize); + unwind->allocStack(stackSize + localsSize); + + build.lea(rbp, qword[rsp + stackSize]); + unwind->setupFrameReg(rbp, stackSize); + + unwind->finish(); + + size_t prologueSize = build.setLabel().location; + + // Body + build.mov(rax, rArg1); + build.mov(rArg1, 25); + build.jmp(rax); + + Label returnOffset = build.setLabel(); + + // Epilogue + build.lea(rsp, qword[rbp + localsSize]); + build.pop(rbp); + build.pop(r15); + build.pop(r14); + build.pop(r13); + build.pop(r12); + build.pop(r11); + build.pop(r10); + build.ret(); + + build.finalize(); + + size_t blockSize = 4096; // Force allocate to create a new block each time + size_t maxTotalSize = 1024 * 1024; + CodeAllocator allocator(blockSize, maxTotalSize); + + allocator.context = unwind.get(); + allocator.createBlockUnwindInfo = createBlockUnwindInfo; + allocator.destroyBlockUnwindInfo = destroyBlockUnwindInfo; + + uint8_t* nativeData1; + size_t sizeNativeData1; + uint8_t* nativeEntry1; + REQUIRE( + allocator.allocate(build.data.data(), build.data.size(), build.code.data(), build.code.size(), nativeData1, sizeNativeData1, nativeEntry1)); + REQUIRE(nativeEntry1); + + // Now we set the offset at the begining so that functions in new blocks will not overlay the locations + // specified by the unwind information of the entry function + unwind->setBeginOffset(prologueSize); + + using FunctionType = int64_t(void*, void (*)(int64_t), void*); + FunctionType* f = (FunctionType*)nativeEntry1; + + uint8_t* nativeExit = nativeEntry1 + returnOffset.location; + + AssemblyBuilderX64 build2(/* logText= */ false); + + build2.mov(r12, rArg3); + build2.call(rArg2); + build2.jmp(r12); + + build2.finalize(); + + uint8_t* nativeData2; + size_t sizeNativeData2; + uint8_t* nativeEntry2; + REQUIRE(allocator.allocate( + build2.data.data(), build2.data.size(), build2.code.data(), build2.code.size(), nativeData2, sizeNativeData2, nativeEntry2)); + REQUIRE(nativeEntry2); + + // To simplify debugging, CHECK_THROWS_WITH_AS is not used here + try + { + f(nativeEntry2, throwing, nativeExit); + } + catch (const std::runtime_error& error) + { + CHECK(strcmp(error.what(), "testing") == 0); + } + + REQUIRE(nativeEntry2); +} + #endif TEST_SUITE_END(); diff --git a/tests/ConstraintSolver.test.cpp b/tests/ConstraintSolver.test.cpp index 9976bd2c..3420fd87 100644 --- a/tests/ConstraintSolver.test.cpp +++ b/tests/ConstraintSolver.test.cpp @@ -28,8 +28,11 @@ TEST_CASE_FIXTURE(ConstraintGraphBuilderFixture, "hello") cgb.visit(block); NotNull rootScope{cgb.rootScope}; + InternalErrorReporter iceHandler; + UnifierSharedState sharedState{&iceHandler}; + Normalizer normalizer{&arena, singletonTypes, NotNull{&sharedState}}; NullModuleResolver resolver; - ConstraintSolver cs{&arena, singletonTypes, rootScope, "MainModule", NotNull(&resolver), {}, &logger}; + ConstraintSolver cs{NotNull{&normalizer}, rootScope, "MainModule", NotNull(&resolver), {}, &logger}; cs.run(); @@ -49,9 +52,11 @@ TEST_CASE_FIXTURE(ConstraintGraphBuilderFixture, "generic_function") cgb.visit(block); NotNull rootScope{cgb.rootScope}; + InternalErrorReporter iceHandler; + UnifierSharedState sharedState{&iceHandler}; + Normalizer normalizer{&arena, singletonTypes, NotNull{&sharedState}}; NullModuleResolver resolver; - ConstraintSolver cs{&arena, singletonTypes, rootScope, "MainModule", NotNull(&resolver), {}, &logger}; - + ConstraintSolver cs{NotNull{&normalizer}, rootScope, "MainModule", NotNull(&resolver), {}, &logger}; cs.run(); TypeId idType = requireBinding(rootScope, "id"); @@ -79,7 +84,10 @@ TEST_CASE_FIXTURE(ConstraintGraphBuilderFixture, "proper_let_generalization") ToStringOptions opts; NullModuleResolver resolver; - ConstraintSolver cs{&arena, singletonTypes, rootScope, "MainModule", NotNull(&resolver), {}, &logger}; + InternalErrorReporter iceHandler; + UnifierSharedState sharedState{&iceHandler}; + Normalizer normalizer{&arena, singletonTypes, NotNull{&sharedState}}; + ConstraintSolver cs{NotNull{&normalizer}, rootScope, "MainModule", NotNull(&resolver), {}, &logger}; cs.run(); diff --git a/tests/Fixture.cpp b/tests/Fixture.cpp index dcc0222a..4d3c8854 100644 --- a/tests/Fixture.cpp +++ b/tests/Fixture.cpp @@ -22,6 +22,8 @@ LUAU_FASTFLAG(DebugLuauDeferredConstraintResolution); LUAU_FASTFLAG(LuauUnknownAndNeverType) LUAU_FASTFLAG(LuauReportShadowedTypeAlias) +extern std::optional randomSeed; // tests/main.cpp + namespace Luau { @@ -90,7 +92,7 @@ std::optional TestFileResolver::getEnvironmentForModule(const Modul Fixture::Fixture(bool freeze, bool prepareAutocomplete) : sff_DebugLuauFreezeArena("DebugLuauFreezeArena", freeze) - , frontend(&fileResolver, &configResolver, {/* retainFullTypeGraphs= */ true}) + , frontend(&fileResolver, &configResolver, {/* retainFullTypeGraphs= */ true, /* forAutocomplete */ false, /* randomConstraintResolutionSeed */ randomSeed}) , typeChecker(frontend.typeChecker) , singletonTypes(frontend.singletonTypes) { diff --git a/tests/Linter.test.cpp b/tests/Linter.test.cpp index 921e6691..fc3ede73 100644 --- a/tests/Linter.test.cpp +++ b/tests/Linter.test.cpp @@ -21,14 +21,14 @@ end return math.max(fib(5), 1) )"); - CHECK_EQ(result.warnings.size(), 0); + REQUIRE(0 == result.warnings.size()); } TEST_CASE_FIXTURE(Fixture, "UnknownGlobal") { LintResult result = lint("--!nocheck\nreturn foo"); - REQUIRE_EQ(result.warnings.size(), 1); + REQUIRE(1 == result.warnings.size()); CHECK_EQ(result.warnings[0].text, "Unknown global 'foo'"); } @@ -39,7 +39,7 @@ TEST_CASE_FIXTURE(Fixture, "DeprecatedGlobal") LintResult result = lintTyped("Wait(5)"); - REQUIRE_EQ(result.warnings.size(), 1); + REQUIRE(1 == result.warnings.size()); CHECK_EQ(result.warnings[0].text, "Global 'Wait' is deprecated, use 'wait' instead"); } @@ -53,7 +53,7 @@ TEST_CASE_FIXTURE(Fixture, "DeprecatedGlobalNoReplacement") LintResult result = lintTyped("Version()"); - REQUIRE_EQ(result.warnings.size(), 1); + REQUIRE(1 == result.warnings.size()); CHECK_EQ(result.warnings[0].text, "Global 'Version' is deprecated"); } @@ -64,7 +64,7 @@ local _ = 5 return _ )"); - CHECK_EQ(result.warnings.size(), 1); + REQUIRE(1 == result.warnings.size()); CHECK_EQ(result.warnings[0].text, "Placeholder value '_' is read here; consider using a named variable"); } @@ -75,7 +75,7 @@ _ = 5 print(_) )"); - CHECK_EQ(result.warnings.size(), 1); + REQUIRE(1 == result.warnings.size()); CHECK_EQ(result.warnings[0].text, "Placeholder value '_' is read here; consider using a named variable"); } @@ -86,7 +86,7 @@ local _ = 5 _ = 6 )"); - CHECK_EQ(result.warnings.size(), 0); + REQUIRE(0 == result.warnings.size()); } TEST_CASE_FIXTURE(BuiltinsFixture, "BuiltinGlobalWrite") @@ -100,7 +100,7 @@ end assert(5) )"); - CHECK_EQ(result.warnings.size(), 2); + REQUIRE(2 == result.warnings.size()); CHECK_EQ(result.warnings[0].text, "Built-in global 'math' is overwritten here; consider using a local or changing the name"); CHECK_EQ(result.warnings[1].text, "Built-in global 'assert' is overwritten here; consider using a local or changing the name"); } @@ -111,7 +111,7 @@ TEST_CASE_FIXTURE(Fixture, "MultilineBlock") if true then print(1) print(2) print(3) end )"); - CHECK_EQ(result.warnings.size(), 1); + REQUIRE(1 == result.warnings.size()); CHECK_EQ(result.warnings[0].text, "A new statement is on the same line; add semi-colon on previous statement to silence"); } @@ -121,7 +121,7 @@ TEST_CASE_FIXTURE(Fixture, "MultilineBlockSemicolonsWhitelisted") print(1); print(2); print(3) )"); - CHECK(result.warnings.empty()); + REQUIRE(0 == result.warnings.size()); } TEST_CASE_FIXTURE(Fixture, "MultilineBlockMissedSemicolon") @@ -130,7 +130,7 @@ TEST_CASE_FIXTURE(Fixture, "MultilineBlockMissedSemicolon") print(1); print(2) print(3) )"); - CHECK_EQ(result.warnings.size(), 1); + REQUIRE(1 == result.warnings.size()); CHECK_EQ(result.warnings[0].text, "A new statement is on the same line; add semi-colon on previous statement to silence"); } @@ -142,7 +142,7 @@ local _x do end )"); - CHECK_EQ(result.warnings.size(), 0); + REQUIRE(0 == result.warnings.size()); } TEST_CASE_FIXTURE(Fixture, "ConfusingIndentation") @@ -152,7 +152,7 @@ print(math.max(1, 2)) )"); - CHECK_EQ(result.warnings.size(), 1); + REQUIRE(1 == result.warnings.size()); CHECK_EQ(result.warnings[0].text, "Statement spans multiple lines; use indentation to silence"); } @@ -167,7 +167,7 @@ end return bar() )"); - CHECK_EQ(result.warnings.size(), 1); + REQUIRE(1 == result.warnings.size()); CHECK_EQ(result.warnings[0].text, "Global 'foo' is only used in the enclosing function 'bar'; consider changing it to local"); } @@ -188,7 +188,7 @@ end return bar() + baz() )"); - REQUIRE_EQ(result.warnings.size(), 1); + REQUIRE(1 == result.warnings.size()); CHECK_EQ(result.warnings[0].text, "Global 'foo' is never read before being written. Consider changing it to local"); } @@ -213,7 +213,7 @@ end return bar() + baz() + read() )"); - CHECK_EQ(result.warnings.size(), 0); + REQUIRE(0 == result.warnings.size()); } TEST_CASE_FIXTURE(Fixture, "GlobalAsLocalWithConditional") @@ -233,7 +233,7 @@ end return bar() + baz() )"); - CHECK_EQ(result.warnings.size(), 0); + REQUIRE(0 == result.warnings.size()); } TEST_CASE_FIXTURE(Fixture, "GlobalAsLocal3WithConditionalRead") @@ -257,7 +257,7 @@ end return bar() + baz() + read() )"); - CHECK_EQ(result.warnings.size(), 0); + REQUIRE(0 == result.warnings.size()); } TEST_CASE_FIXTURE(Fixture, "GlobalAsLocalInnerRead") @@ -275,7 +275,7 @@ function baz() bar = 0 end return foo() + baz() )"); - CHECK_EQ(result.warnings.size(), 0); + REQUIRE(0 == result.warnings.size()); } TEST_CASE_FIXTURE(Fixture, "GlobalAsLocalMulti") @@ -304,7 +304,7 @@ fnA() -- prints "true", "nil" fnB() -- prints "false", "nil" )"); - CHECK_EQ(result.warnings.size(), 1); + REQUIRE(1 == result.warnings.size()); CHECK_EQ(result.warnings[0].text, "Global 'moreInternalLogic' is only used in the enclosing function defined at line 2; consider changing it to local"); } @@ -319,7 +319,7 @@ local arg = 5 print(arg) )"); - CHECK_EQ(result.warnings.size(), 1); + REQUIRE(1 == result.warnings.size()); CHECK_EQ(result.warnings[0].text, "Variable 'arg' shadows previous declaration at line 2"); } @@ -337,7 +337,7 @@ end return bar() )"); - CHECK_EQ(result.warnings.size(), 1); + REQUIRE(1 == result.warnings.size()); CHECK_EQ(result.warnings[0].text, "Variable 'global' shadows a global variable used at line 3"); } @@ -352,7 +352,7 @@ end return bar() )"); - CHECK_EQ(result.warnings.size(), 1); + REQUIRE(1 == result.warnings.size()); CHECK_EQ(result.warnings[0].text, "Variable 'a' shadows previous declaration at line 2"); } @@ -372,7 +372,7 @@ end return bar() )"); - CHECK_EQ(result.warnings.size(), 2); + REQUIRE(2 == result.warnings.size()); CHECK_EQ(result.warnings[0].text, "Variable 'arg' is never used; prefix with '_' to silence"); CHECK_EQ(result.warnings[1].text, "Variable 'blarg' is never used; prefix with '_' to silence"); } @@ -387,7 +387,7 @@ local Roact = require(game.Packages.Roact) local _Roact = require(game.Packages.Roact) )"); - CHECK_EQ(result.warnings.size(), 1); + REQUIRE(1 == result.warnings.size()); CHECK_EQ(result.warnings[0].text, "Import 'Roact' is never used; prefix with '_' to silence"); } @@ -412,7 +412,7 @@ end return foo() )"); - CHECK_EQ(result.warnings.size(), 2); + REQUIRE(2 == result.warnings.size()); CHECK_EQ(result.warnings[0].text, "Function 'bar' is never used; prefix with '_' to silence"); CHECK_EQ(result.warnings[1].text, "Function 'qux' is never used; prefix with '_' to silence"); } @@ -427,7 +427,7 @@ end print("hi!") )"); - CHECK_EQ(result.warnings.size(), 1); + REQUIRE(1 == result.warnings.size()); CHECK_EQ(result.warnings[0].location.begin.line, 5); CHECK_EQ(result.warnings[0].text, "Unreachable code (previous statement always returns)"); } @@ -443,7 +443,7 @@ end print("hi!") )"); - CHECK_EQ(result.warnings.size(), 1); + REQUIRE(1 == result.warnings.size()); CHECK_EQ(result.warnings[0].location.begin.line, 3); CHECK_EQ(result.warnings[0].text, "Unreachable code (previous statement always breaks)"); } @@ -459,7 +459,7 @@ end print("hi!") )"); - CHECK_EQ(result.warnings.size(), 1); + REQUIRE(1 == result.warnings.size()); CHECK_EQ(result.warnings[0].location.begin.line, 3); CHECK_EQ(result.warnings[0].text, "Unreachable code (previous statement always continues)"); } @@ -495,7 +495,7 @@ end return { foo1, foo2, foo3 } )"); - CHECK_EQ(result.warnings.size(), 1); + REQUIRE(1 == result.warnings.size()); CHECK_EQ(result.warnings[0].location.begin.line, 7); CHECK_EQ(result.warnings[0].text, "Unreachable code (previous statement always returns)"); } @@ -515,7 +515,7 @@ end return foo1 )"); - CHECK_EQ(result.warnings.size(), 0); + REQUIRE(0 == result.warnings.size()); } TEST_CASE_FIXTURE(Fixture, "UnreachableCodeAssertFalseReturnSilent") @@ -532,7 +532,7 @@ end return foo1 )"); - CHECK_EQ(result.warnings.size(), 0); + REQUIRE(0 == result.warnings.size()); } TEST_CASE_FIXTURE(Fixture, "UnreachableCodeErrorReturnNonSilentBranchy") @@ -550,7 +550,7 @@ end return foo1 )"); - CHECK_EQ(result.warnings.size(), 1); + REQUIRE(1 == result.warnings.size()); CHECK_EQ(result.warnings[0].location.begin.line, 7); CHECK_EQ(result.warnings[0].text, "Unreachable code (previous statement always errors)"); } @@ -571,7 +571,7 @@ end return foo1 )"); - CHECK_EQ(result.warnings.size(), 1); + REQUIRE(1 == result.warnings.size()); CHECK_EQ(result.warnings[0].location.begin.line, 8); CHECK_EQ(result.warnings[0].text, "Unreachable code (previous statement always errors)"); } @@ -589,7 +589,7 @@ end return foo1 )"); - CHECK_EQ(result.warnings.size(), 0); + REQUIRE(0 == result.warnings.size()); } TEST_CASE_FIXTURE(Fixture, "UnreachableCodeLoopRepeat") @@ -605,8 +605,8 @@ end return foo1 )"); - CHECK_EQ(result.warnings.size(), - 0); // this is technically a bug, since the repeat body always returns; fixing this bug is a bit more involved than I'd like + // this is technically a bug, since the repeat body always returns; fixing this bug is a bit more involved than I'd like + REQUIRE(0 == result.warnings.size()); } TEST_CASE_FIXTURE(Fixture, "UnknownType") @@ -633,7 +633,7 @@ local _o02 = type(game) == "vector" local _o03 = typeof(game) == "Part" )"); - REQUIRE_EQ(result.warnings.size(), 3); + REQUIRE(3 == result.warnings.size()); CHECK_EQ(result.warnings[0].location.begin.line, 2); CHECK_EQ(result.warnings[0].text, "Unknown type 'Part' (expected primitive type)"); CHECK_EQ(result.warnings[1].location.begin.line, 3); @@ -654,7 +654,7 @@ for i=#t,1,-1 do end )"); - CHECK_EQ(result.warnings.size(), 1); + REQUIRE(1 == result.warnings.size()); CHECK_EQ(result.warnings[0].location.begin.line, 3); CHECK_EQ(result.warnings[0].text, "For loop should iterate backwards; did you forget to specify -1 as step?"); } @@ -669,7 +669,7 @@ for i=8,1,-1 do end )"); - CHECK_EQ(result.warnings.size(), 1); + REQUIRE(1 == result.warnings.size()); CHECK_EQ(result.warnings[0].location.begin.line, 1); CHECK_EQ(result.warnings[0].text, "For loop should iterate backwards; did you forget to specify -1 as step?"); } @@ -684,7 +684,7 @@ for i=1.3,7.5,1 do end )"); - CHECK_EQ(result.warnings.size(), 1); + REQUIRE(1 == result.warnings.size()); CHECK_EQ(result.warnings[0].location.begin.line, 1); CHECK_EQ(result.warnings[0].text, "For loop ends at 7.3 instead of 7.5; did you forget to specify step?"); } @@ -702,7 +702,7 @@ for i=#t,0 do end )"); - CHECK_EQ(result.warnings.size(), 2); + REQUIRE(2 == result.warnings.size()); CHECK_EQ(result.warnings[0].location.begin.line, 1); CHECK_EQ(result.warnings[0].text, "For loop starts at 0, but arrays start at 1"); CHECK_EQ(result.warnings[1].location.begin.line, 7); @@ -730,7 +730,7 @@ local _a,_b,_c = pcall(), nil end )"); - CHECK_EQ(result.warnings.size(), 2); + REQUIRE(2 == result.warnings.size()); CHECK_EQ(result.warnings[0].location.begin.line, 5); CHECK_EQ(result.warnings[0].text, "Assigning 2 values to 3 variables initializes extra variables with nil; add 'nil' to value list to silence"); CHECK_EQ(result.warnings[1].location.begin.line, 11); @@ -795,7 +795,7 @@ end return f1,f2,f3,f4,f5,f6,f7 )"); - CHECK_EQ(result.warnings.size(), 3); + REQUIRE(3 == result.warnings.size()); CHECK_EQ(result.warnings[0].location.begin.line, 4); CHECK_EQ(result.warnings[0].text, "Function 'f1' can implicitly return no values even though there's an explicit return at line 4; add explicit return to silence"); @@ -851,7 +851,7 @@ end return f1,f2,f3,f4 )"); - CHECK_EQ(result.warnings.size(), 2); + REQUIRE(2 == result.warnings.size()); CHECK_EQ(result.warnings[0].location.begin.line, 25); CHECK_EQ(result.warnings[0].text, "Function 'f3' can implicitly return no values even though there's an explicit return at line 21; add explicit return to silence"); @@ -874,7 +874,7 @@ type InputData = { } )"); - CHECK_EQ(result.warnings.size(), 0); + REQUIRE(0 == result.warnings.size()); } TEST_CASE_FIXTURE(Fixture, "BreakFromInfiniteLoopMakesStatementReachable") @@ -893,7 +893,7 @@ until true return 1 )"); - CHECK_EQ(result.warnings.size(), 0); + REQUIRE(0 == result.warnings.size()); } TEST_CASE_FIXTURE(Fixture, "IgnoreLintAll") @@ -903,7 +903,7 @@ TEST_CASE_FIXTURE(Fixture, "IgnoreLintAll") return foo )"); - CHECK_EQ(result.warnings.size(), 0); + REQUIRE(0 == result.warnings.size()); } TEST_CASE_FIXTURE(Fixture, "IgnoreLintSpecific") @@ -914,7 +914,7 @@ local x = 1 return foo )"); - CHECK_EQ(result.warnings.size(), 1); + REQUIRE(1 == result.warnings.size()); CHECK_EQ(result.warnings[0].text, "Variable 'x' is never used; prefix with '_' to silence"); } @@ -933,7 +933,7 @@ local _ = ("%"):format() string.format("hello %+10d %.02f %%", 4, 5) )"); - CHECK_EQ(result.warnings.size(), 4); + REQUIRE(4 == result.warnings.size()); CHECK_EQ(result.warnings[0].text, "Invalid format string: unfinished format specifier"); CHECK_EQ(result.warnings[1].text, "Invalid format string: invalid format specifier: must be a string format specifier or %"); CHECK_EQ(result.warnings[2].text, "Invalid format string: invalid format specifier: must be a string format specifier or %"); @@ -973,7 +973,7 @@ string.packsize("c99999999999999999999") string.packsize("=!1bbbI3c42") )"); - CHECK_EQ(result.warnings.size(), 11); + REQUIRE(11 == result.warnings.size()); CHECK_EQ(result.warnings[0].text, "Invalid pack format: unexpected character; must be a pack specifier or space"); CHECK_EQ(result.warnings[1].text, "Invalid pack format: unexpected character; must be a pack specifier or space"); CHECK_EQ(result.warnings[2].text, "Invalid pack format: unexpected character; must be a pack specifier or space"); @@ -1017,7 +1017,7 @@ local _ = s:match("%q") string.match(s, "[A-Z]+(%d)%1") )"); - CHECK_EQ(result.warnings.size(), 14); + REQUIRE(14 == result.warnings.size()); CHECK_EQ(result.warnings[0].text, "Invalid match pattern: invalid character class, must refer to a defined class or its inverse"); CHECK_EQ(result.warnings[1].text, "Invalid match pattern: invalid character class, must refer to a defined class or its inverse"); CHECK_EQ(result.warnings[2].text, "Invalid match pattern: invalid character class, must refer to a defined class or its inverse"); @@ -1049,7 +1049,7 @@ string.match(s, "((a)%1)") string.match(s, "((a)%3)") )~"); - CHECK_EQ(result.warnings.size(), 2); + REQUIRE(2 == result.warnings.size()); CHECK_EQ(result.warnings[0].text, "Invalid match pattern: invalid capture reference, must refer to a closed capture"); CHECK_EQ(result.warnings[0].location.begin.line, 7); CHECK_EQ(result.warnings[1].text, "Invalid match pattern: invalid capture reference, must refer to a valid capture"); @@ -1087,7 +1087,7 @@ string.match(s, "[]|'[]") string.match(s, "[^]|'[]") )~"); - CHECK_EQ(result.warnings.size(), 7); + REQUIRE(7 == result.warnings.size()); CHECK_EQ(result.warnings[0].text, "Invalid match pattern: expected ] at the end of the string to close a set"); CHECK_EQ(result.warnings[1].text, "Invalid match pattern: expected ] at the end of the string to close a set"); CHECK_EQ(result.warnings[2].text, "Invalid match pattern: character range can't include character sets"); @@ -1118,7 +1118,7 @@ string.find("foo"); ("foo"):find() )"); - CHECK_EQ(result.warnings.size(), 2); + REQUIRE(2 == result.warnings.size()); CHECK_EQ(result.warnings[0].text, "Invalid match pattern: invalid character class, must refer to a defined class or its inverse"); CHECK_EQ(result.warnings[0].location.begin.line, 4); CHECK_EQ(result.warnings[1].text, "Invalid match pattern: invalid character class, must refer to a defined class or its inverse"); @@ -1141,7 +1141,7 @@ string.gsub(s, '[A-Z]+(%d)', "%0%1") string.gsub(s, 'foo', "%0") )"); - CHECK_EQ(result.warnings.size(), 4); + REQUIRE(4 == result.warnings.size()); CHECK_EQ(result.warnings[0].text, "Invalid match replacement: unfinished replacement"); CHECK_EQ(result.warnings[1].text, "Invalid match replacement: unexpected replacement character; must be a digit or %"); CHECK_EQ(result.warnings[2].text, "Invalid match replacement: invalid capture index, must refer to pattern capture"); @@ -1162,7 +1162,7 @@ os.date("it's %c now") os.date("!*t") )"); - CHECK_EQ(result.warnings.size(), 4); + REQUIRE(4 == result.warnings.size()); CHECK_EQ(result.warnings[0].text, "Invalid date format: unfinished replacement"); CHECK_EQ(result.warnings[1].text, "Invalid date format: unexpected replacement character; must be a date format specifier or %"); CHECK_EQ(result.warnings[2].text, "Invalid date format: unexpected replacement character; must be a date format specifier or %"); @@ -1181,7 +1181,7 @@ s:match("[]") nons:match("[]") )~"); - REQUIRE_EQ(result.warnings.size(), 2); + REQUIRE(2 == result.warnings.size()); CHECK_EQ(result.warnings[0].text, "Invalid match pattern: expected ] at the end of the string to close a set"); CHECK_EQ(result.warnings[0].location.begin.line, 3); CHECK_EQ(result.warnings[1].text, "Invalid match pattern: expected ] at the end of the string to close a set"); @@ -1231,7 +1231,7 @@ _ = { } )"); - CHECK_EQ(result.warnings.size(), 6); + REQUIRE(6 == result.warnings.size()); CHECK_EQ(result.warnings[0].text, "Table field 'first' is a duplicate; previously defined at line 3"); CHECK_EQ(result.warnings[1].text, "Table field 'first' is a duplicate; previously defined at line 9"); CHECK_EQ(result.warnings[2].text, "Table index 1 is a duplicate; previously defined as a list entry"); @@ -1248,7 +1248,7 @@ TEST_CASE_FIXTURE(Fixture, "ImportOnlyUsedInTypeAnnotation") local x: Foo.Y = 1 )"); - REQUIRE_EQ(result.warnings.size(), 1); + REQUIRE(1 == result.warnings.size()); CHECK_EQ(result.warnings[0].text, "Variable 'x' is never used; prefix with '_' to silence"); } @@ -1259,7 +1259,7 @@ TEST_CASE_FIXTURE(Fixture, "DisableUnknownGlobalWithTypeChecking") unknownGlobal() )"); - REQUIRE_EQ(result.warnings.size(), 0); + REQUIRE(0 == result.warnings.size()); } TEST_CASE_FIXTURE(Fixture, "no_spurious_warning_after_a_function_type_alias") @@ -1271,7 +1271,7 @@ TEST_CASE_FIXTURE(Fixture, "no_spurious_warning_after_a_function_type_alias") return exports )"); - CHECK_EQ(result.warnings.size(), 0); + REQUIRE(0 == result.warnings.size()); } TEST_CASE_FIXTURE(Fixture, "use_all_parent_scopes_for_globals") @@ -1294,7 +1294,7 @@ TEST_CASE_FIXTURE(Fixture, "use_all_parent_scopes_for_globals") LintResult result = frontend.lint("A"); - CHECK_EQ(result.warnings.size(), 0); + REQUIRE(0 == result.warnings.size()); } TEST_CASE_FIXTURE(Fixture, "DeadLocalsUsed") @@ -1320,7 +1320,7 @@ do end )"); - CHECK_EQ(result.warnings.size(), 3); + REQUIRE(3 == result.warnings.size()); CHECK_EQ(result.warnings[0].text, "Variable 'x' defined at line 4 is never initialized or assigned; initialize with 'nil' to silence"); CHECK_EQ(result.warnings[1].text, "Assigning 2 values to 3 variables initializes extra variables with nil; add 'nil' to value list to silence"); CHECK_EQ(result.warnings[2].text, "Variable 'c' defined at line 12 is never initialized or assigned; initialize with 'nil' to silence"); @@ -1333,7 +1333,7 @@ local foo function foo() end )"); - CHECK_EQ(result.warnings.size(), 0); + REQUIRE(0 == result.warnings.size()); } TEST_CASE_FIXTURE(Fixture, "DuplicateGlobalFunction") @@ -1408,7 +1408,7 @@ TEST_CASE_FIXTURE(Fixture, "DontTriggerTheWarningIfTheFunctionsAreInDifferentSco return c )"); - CHECK(result.warnings.empty()); + REQUIRE(0 == result.warnings.size()); } TEST_CASE_FIXTURE(Fixture, "LintHygieneUAF") @@ -1444,7 +1444,7 @@ TEST_CASE_FIXTURE(Fixture, "LintHygieneUAF") local h: Hooty.Pt )"); - CHECK_EQ(result.warnings.size(), 12); + REQUIRE(12 == result.warnings.size()); } TEST_CASE_FIXTURE(Fixture, "DeprecatedApi") @@ -1478,7 +1478,7 @@ return function (i: Instance) end )"); - REQUIRE_EQ(result.warnings.size(), 3); + REQUIRE(3 == result.warnings.size()); CHECK_EQ(result.warnings[0].text, "Member 'Instance.Wait' is deprecated"); CHECK_EQ(result.warnings[1].text, "Member 'toHSV' is deprecated, use 'Color3:ToHSV' instead"); CHECK_EQ(result.warnings[2].text, "Member 'Instance.DataCost' is deprecated"); @@ -1511,7 +1511,7 @@ table.create(42, {}) table.create(42, {} :: {}) )"); - REQUIRE_EQ(result.warnings.size(), 10); + REQUIRE(10 == result.warnings.size()); CHECK_EQ(result.warnings[0].text, "table.insert will insert the value before the last element, which is likely a bug; consider removing the " "second argument or wrap it in parentheses to silence"); CHECK_EQ(result.warnings[1].text, "table.insert will append the value to the table; consider removing the second argument for efficiency"); @@ -1556,7 +1556,7 @@ _ = true and true or false -- no warning since this is is a common pattern used _ = if true then 1 elseif true then 2 else 3 )"); - REQUIRE_EQ(result.warnings.size(), 8); + REQUIRE(8 == result.warnings.size()); CHECK_EQ(result.warnings[0].text, "Condition has already been checked on line 2"); CHECK_EQ(result.warnings[0].location.begin.line + 1, 4); CHECK_EQ(result.warnings[1].text, "Condition has already been checked on column 5"); @@ -1580,7 +1580,7 @@ elseif correct({a = 1, b = 2 * (-2), c = opaque.path['with']("calls", false)}) t end )"); - REQUIRE_EQ(result.warnings.size(), 1); + REQUIRE(1 == result.warnings.size()); CHECK_EQ(result.warnings[0].text, "Condition has already been checked on line 4"); CHECK_EQ(result.warnings[0].location.begin.line + 1, 5); } @@ -1601,7 +1601,7 @@ end return foo, moo, a1, a2 )"); - REQUIRE_EQ(result.warnings.size(), 4); + REQUIRE(4 == result.warnings.size()); CHECK_EQ(result.warnings[0].text, "Function parameter 'a1' already defined on column 14"); CHECK_EQ(result.warnings[1].text, "Variable 'a1' is never used; prefix with '_' to silence"); CHECK_EQ(result.warnings[2].text, "Variable 'a1' already defined on column 7"); @@ -1618,7 +1618,7 @@ _ = math.random() < 0.5 and 0 or 42 _ = (math.random() < 0.5 and false) or 42 -- currently ignored )"); - REQUIRE_EQ(result.warnings.size(), 2); + REQUIRE(2 == result.warnings.size()); CHECK_EQ(result.warnings[0].text, "The and-or expression always evaluates to the second alternative because the first alternative is false; " "consider using if-then-else expression instead"); CHECK_EQ(result.warnings[1].text, "The and-or expression always evaluates to the second alternative because the first alternative is nil; " @@ -1640,7 +1640,7 @@ do end --!nolint )"); - REQUIRE_EQ(result.warnings.size(), 6); + REQUIRE(6 == result.warnings.size()); CHECK_EQ(result.warnings[0].text, "Unknown comment directive 'struct'; did you mean 'strict'?"); CHECK_EQ(result.warnings[1].text, "Unknown comment directive 'nolintGlobal'"); CHECK_EQ(result.warnings[2].text, "nolint directive refers to unknown lint rule 'Global'"); @@ -1656,7 +1656,7 @@ TEST_CASE_FIXTURE(Fixture, "WrongCommentMuteSelf") --!struct )"); - REQUIRE_EQ(result.warnings.size(), 0); // --!nolint disables WrongComment lint :) + REQUIRE(0 == result.warnings.size()); // --!nolint disables WrongComment lint :) } TEST_CASE_FIXTURE(Fixture, "DuplicateConditionsIfStatAndExpr") @@ -1668,7 +1668,7 @@ elseif if 0 then 5 else 4 then end )"); - REQUIRE_EQ(result.warnings.size(), 1); + REQUIRE(1 == result.warnings.size()); CHECK_EQ(result.warnings[0].text, "Condition has already been checked on line 2"); } @@ -1681,13 +1681,13 @@ TEST_CASE_FIXTURE(Fixture, "WrongCommentOptimize") --!optimize 2 )"); - REQUIRE_EQ(result.warnings.size(), 3); + REQUIRE(3 == result.warnings.size()); CHECK_EQ(result.warnings[0].text, "optimize directive requires an optimization level"); CHECK_EQ(result.warnings[1].text, "optimize directive uses unknown optimization level 'me', 0..2 expected"); CHECK_EQ(result.warnings[2].text, "optimize directive uses unknown optimization level '100500', 0..2 expected"); result = lint("--!optimize "); - REQUIRE_EQ(result.warnings.size(), 1); + REQUIRE(1 == result.warnings.size()); CHECK_EQ(result.warnings[0].text, "optimize directive requires an optimization level"); } @@ -1700,7 +1700,7 @@ TEST_CASE_FIXTURE(Fixture, "TestStringInterpolation") local _ = `unknown {foo}` )"); - REQUIRE_EQ(result.warnings.size(), 1); + REQUIRE(1 == result.warnings.size()); } TEST_CASE_FIXTURE(Fixture, "IntegerParsing") @@ -1710,7 +1710,7 @@ local _ = 0b10000000000000000000000000000000000000000000000000000000000000000 local _ = 0x10000000000000000 )"); - REQUIRE_EQ(result.warnings.size(), 2); + REQUIRE(2 == result.warnings.size()); CHECK_EQ(result.warnings[0].text, "Binary number literal exceeded available precision and has been truncated to 2^64"); CHECK_EQ(result.warnings[1].text, "Hexadecimal number literal exceeded available precision and has been truncated to 2^64"); } @@ -1725,7 +1725,7 @@ local _ = 0x0x123 local _ = 0x0xffffffffffffffffffffffffffffffffff )"); - REQUIRE_EQ(result.warnings.size(), 2); + REQUIRE(2 == result.warnings.size()); CHECK_EQ(result.warnings[0].text, "Hexadecimal number literal has a double prefix, which will fail to parse in the future; remove the extra 0x to fix"); CHECK_EQ(result.warnings[1].text, @@ -1756,7 +1756,7 @@ local _ = (a <= b) == 0 local _ = a <= (b == 0) )"); - REQUIRE_EQ(result.warnings.size(), 5); + REQUIRE(5 == result.warnings.size()); CHECK_EQ(result.warnings[0].text, "not X == Y is equivalent to (not X) == Y; consider using X ~= Y, or add parentheses to silence"); CHECK_EQ(result.warnings[1].text, "not X ~= Y is equivalent to (not X) ~= Y; consider using X == Y, or add parentheses to silence"); CHECK_EQ(result.warnings[2].text, "not X <= Y is equivalent to (not X) <= Y; add parentheses to silence"); diff --git a/tests/Normalize.test.cpp b/tests/Normalize.test.cpp index 31df707d..156cbbc5 100644 --- a/tests/Normalize.test.cpp +++ b/tests/Normalize.test.cpp @@ -356,6 +356,11 @@ TEST_CASE_FIXTURE(NormalizeFixture, "table_with_any_prop") TEST_CASE_FIXTURE(NormalizeFixture, "intersection") { + ScopedFastFlag sffs[] { + {"LuauSubtypeNormalizer", true}, + {"LuauTypeNormalization2", true}, + }; + check(R"( local a: number & string local b: number @@ -374,8 +379,9 @@ TEST_CASE_FIXTURE(NormalizeFixture, "intersection") CHECK(!isSubtype(c, a)); CHECK(isSubtype(a, c)); - CHECK(!isSubtype(d, a)); - CHECK(!isSubtype(a, d)); + // These types are both equivalent to never + CHECK(isSubtype(d, a)); + CHECK(isSubtype(a, d)); } TEST_CASE_FIXTURE(NormalizeFixture, "union_and_intersection") diff --git a/tests/Parser.test.cpp b/tests/Parser.test.cpp index b4064cfb..662c2900 100644 --- a/tests/Parser.test.cpp +++ b/tests/Parser.test.cpp @@ -2722,4 +2722,59 @@ TEST_CASE_FIXTURE(Fixture, "error_message_for_using_function_as_type_annotation" result.errors[0].getMessage()); } +TEST_CASE_FIXTURE(Fixture, "get_a_nice_error_when_there_is_an_extra_comma_at_the_end_of_a_function_argument_list") +{ + ScopedFastFlag sff{"LuauCommaParenWarnings", true}; + + ParseResult result = tryParse(R"( + foo(a, b, c,) + )"); + + REQUIRE(1 == result.errors.size()); + + CHECK(Location({1, 20}, {1, 21}) == result.errors[0].getLocation()); + CHECK("Expected expression after ',' but got ')' instead" == result.errors[0].getMessage()); +} + +TEST_CASE_FIXTURE(Fixture, "get_a_nice_error_when_there_is_an_extra_comma_at_the_end_of_a_function_parameter_list") +{ + ScopedFastFlag sff{"LuauCommaParenWarnings", true}; + + ParseResult result = tryParse(R"( + export type VisitFn = ( + any, + Array>, -- extra comma here + ) -> any + )"); + + REQUIRE(1 == result.errors.size()); + + CHECK(Location({4, 8}, {4, 9}) == result.errors[0].getLocation()); + CHECK("Expected type after ',' but got ')' instead" == result.errors[0].getMessage()); +} + +TEST_CASE_FIXTURE(Fixture, "get_a_nice_error_when_there_is_an_extra_comma_at_the_end_of_a_generic_parameter_list") +{ + ScopedFastFlag sff{"LuauCommaParenWarnings", true}; + + ParseResult result = tryParse(R"( + export type VisitFn = (a: A, b: B) -> () + )"); + + REQUIRE(1 == result.errors.size()); + + CHECK(Location({1, 36}, {1, 37}) == result.errors[0].getLocation()); + CHECK("Expected type after ',' but got '>' instead" == result.errors[0].getMessage()); + + REQUIRE(1 == result.root->body.size); + + AstStatTypeAlias* t = result.root->body.data[0]->as(); + REQUIRE(t != nullptr); + + AstTypeFunction* f = t->type->as(); + REQUIRE(f != nullptr); + + CHECK(2 == f->generics.size); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.classes.test.cpp b/tests/TypeInfer.classes.test.cpp index 98883dfa..87ec58c9 100644 --- a/tests/TypeInfer.classes.test.cpp +++ b/tests/TypeInfer.classes.test.cpp @@ -480,4 +480,38 @@ local a: ChildClass = i CHECK_EQ("Type 'ChildClass' from 'Test' could not be converted into 'ChildClass' from 'MainModule'", toString(result.errors[0])); } +TEST_CASE_FIXTURE(ClassFixture, "intersections_of_unions_of_classes") +{ + ScopedFastFlag sffs[] { + {"LuauSubtypeNormalizer", true}, + {"LuauTypeNormalization2", true}, + }; + + CheckResult result = check(R"( + local x : (BaseClass | Vector2) & (ChildClass | AnotherChild) + local y : (ChildClass | AnotherChild) + x = y + y = x + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(ClassFixture, "unions_of_intersections_of_classes") +{ + ScopedFastFlag sffs[] { + {"LuauSubtypeNormalizer", true}, + {"LuauTypeNormalization2", true}, + }; + + CheckResult result = check(R"( + local x : (BaseClass & ChildClass) | (BaseClass & AnotherChild) | (BaseClass & Vector2) + local y : (ChildClass | AnotherChild) + x = y + y = x + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.functions.test.cpp b/tests/TypeInfer.functions.test.cpp index a4420b9a..fa99ff58 100644 --- a/tests/TypeInfer.functions.test.cpp +++ b/tests/TypeInfer.functions.test.cpp @@ -2,6 +2,7 @@ #include "Luau/AstQuery.h" #include "Luau/BuiltinDefinitions.h" +#include "Luau/Error.h" #include "Luau/Scope.h" #include "Luau/TypeInfer.h" #include "Luau/TypeVar.h" @@ -14,6 +15,7 @@ using namespace Luau; LUAU_FASTFLAG(LuauLowerBoundsCalculation); +LUAU_FASTFLAG(LuauInstantiateInSubtyping); LUAU_FASTFLAG(LuauSpecialTypesAsterisked); TEST_SUITE_BEGIN("TypeInferFunctions"); @@ -1087,10 +1089,20 @@ f(function(a, b, c, ...) return a + b end) LUAU_REQUIRE_ERRORS(result); - CHECK_EQ(R"(Type '(number, number, a) -> number' could not be converted into '(number, number) -> number' + if (FFlag::LuauInstantiateInSubtyping) + { + CHECK_EQ(R"(Type '(number, number, a) -> number' could not be converted into '(number, number) -> number' caused by: Argument count mismatch. Function expects 3 arguments, but only 2 are specified)", - toString(result.errors[0])); + toString(result.errors[0])); + } + else + { + CHECK_EQ(R"(Type '(number, number, a) -> number' could not be converted into '(number, number) -> number' +caused by: + Argument count mismatch. Function expects 3 arguments, but only 2 are specified)", + toString(result.errors[0])); + } // Infer from variadic packs into elements result = check(R"( @@ -1189,10 +1201,20 @@ f(function(a, b, c, ...) return a + b end) LUAU_REQUIRE_ERRORS(result); - CHECK_EQ(R"(Type '(number, number, a) -> number' could not be converted into '(number, number) -> number' + if (FFlag::LuauInstantiateInSubtyping) + { + CHECK_EQ(R"(Type '(number, number, a) -> number' could not be converted into '(number, number) -> number' caused by: Argument count mismatch. Function expects 3 arguments, but only 2 are specified)", - toString(result.errors[0])); + toString(result.errors[0])); + } + else + { + CHECK_EQ(R"(Type '(number, number, a) -> number' could not be converted into '(number, number) -> number' +caused by: + Argument count mismatch. Function expects 3 arguments, but only 2 are specified)", + toString(result.errors[0])); + } // Infer from variadic packs into elements result = check(R"( diff --git a/tests/TypeInfer.generics.test.cpp b/tests/TypeInfer.generics.test.cpp index 3c867770..1b02abc1 100644 --- a/tests/TypeInfer.generics.test.cpp +++ b/tests/TypeInfer.generics.test.cpp @@ -10,6 +10,7 @@ #include "doctest.h" LUAU_FASTFLAG(LuauCheckGenericHOFTypes) +LUAU_FASTFLAG(LuauInstantiateInSubtyping) LUAU_FASTFLAG(LuauSpecialTypesAsterisked) using namespace Luau; @@ -960,7 +961,11 @@ TEST_CASE_FIXTURE(Fixture, "instantiate_generic_function_in_assignments") TypeMismatch* tm = get(result.errors[0]); REQUIRE(tm); CHECK_EQ("((number) -> number, string) -> number", toString(tm->wantedType)); - CHECK_EQ("((number) -> number, number) -> number", toString(tm->givenType)); + if (FFlag::LuauInstantiateInSubtyping) + CHECK_EQ("((a) -> (b...), a) -> (b...)", toString(tm->givenType)); + else + CHECK_EQ("((number) -> number, number) -> number", toString(tm->givenType)); + } TEST_CASE_FIXTURE(Fixture, "instantiate_generic_function_in_assignments2") @@ -980,7 +985,10 @@ TEST_CASE_FIXTURE(Fixture, "instantiate_generic_function_in_assignments2") TypeMismatch* tm = get(result.errors[0]); REQUIRE(tm); CHECK_EQ("(string, string) -> number", toString(tm->wantedType)); - CHECK_EQ("((string) -> number, string) -> number", toString(*tm->givenType)); + if (FFlag::LuauInstantiateInSubtyping) + CHECK_EQ("((a) -> (b...), a) -> (b...)", toString(tm->givenType)); + else + CHECK_EQ("((string) -> number, string) -> number", toString(*tm->givenType)); } TEST_CASE_FIXTURE(Fixture, "self_recursive_instantiated_param") @@ -1110,6 +1118,15 @@ local c = sumrec(function(x, y, f) return f(x, y) end) -- type binders are not i { LUAU_REQUIRE_NO_ERRORS(result); } + else if (FFlag::LuauInstantiateInSubtyping) + { + LUAU_REQUIRE_ERRORS(result); + CHECK_EQ( + R"(Type '(a, b, (a, b) -> (c...)) -> (c...)' could not be converted into '(a, a, (a, a) -> a) -> a' +caused by: + Argument #1 type is not compatible. Generic subtype escaping scope)", + toString(result.errors[0])); + } else { LUAU_REQUIRE_ERRORS(result); @@ -1219,4 +1236,48 @@ TEST_CASE_FIXTURE(Fixture, "do_not_always_instantiate_generic_intersection_types LUAU_REQUIRE_NO_ERRORS(result); } +TEST_CASE_FIXTURE(BuiltinsFixture, "hof_subtype_instantiation_regression") +{ + CheckResult result = check(R"( +--!strict + +local function defaultSort(a: T, b: T) + return true +end +type A = any +return function(array: {T}): {T} + table.sort(array, defaultSort) + return array +end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(BuiltinsFixture, "higher_rank_polymorphism_should_not_accept_instantiated_arguments") +{ + ScopedFastFlag sffs[] = { + {"LuauInstantiateInSubtyping", true}, + {"LuauCheckGenericHOFTypes", true}, // necessary because of interactions with the test + }; + + CheckResult result = check(R"( +--!strict + +local function instantiate(f: (a) -> a): (number) -> number + return f +end + +instantiate(function(x: string) return "foo" end) + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + auto tm1 = get(result.errors[0]); + REQUIRE(tm1); + + CHECK_EQ("(a) -> a", toString(tm1->wantedType)); + CHECK_EQ("(string) -> string", toString(tm1->givenType)); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.intersectionTypes.test.cpp b/tests/TypeInfer.intersectionTypes.test.cpp index 818d0124..e49df101 100644 --- a/tests/TypeInfer.intersectionTypes.test.cpp +++ b/tests/TypeInfer.intersectionTypes.test.cpp @@ -446,4 +446,459 @@ TEST_CASE_FIXTURE(Fixture, "no_stack_overflow_from_flattenintersection") LUAU_REQUIRE_ERRORS(result); } +TEST_CASE_FIXTURE(Fixture, "intersect_bool_and_false") +{ + CheckResult result = check(R"( + local x : (boolean & false) + local y : false = x -- OK + local z : true = x -- Not OK + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ(toString(result.errors[0]), "Type 'boolean & false' could not be converted into 'true'; none of the intersection parts are compatible"); +} + +TEST_CASE_FIXTURE(Fixture, "intersect_false_and_bool_and_false") +{ + CheckResult result = check(R"( + local x : false & (boolean & false) + local y : false = x -- OK + local z : true = x -- Not OK + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + // TODO: odd stringification of `false & (boolean & false)`.) + CHECK_EQ(toString(result.errors[0]), "Type 'boolean & false & false' could not be converted into 'true'; none of the intersection parts are compatible"); +} + +TEST_CASE_FIXTURE(Fixture, "intersect_saturate_overloaded_functions") +{ + ScopedFastFlag sffs[] { + {"LuauSubtypeNormalizer", true}, + {"LuauTypeNormalization2", true}, + }; + + CheckResult result = check(R"( + local x : ((number?) -> number?) & ((string?) -> string?) + local y : (nil) -> nil = x -- OK + local z : (number) -> number = x -- Not OK + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ(toString(result.errors[0]), "Type '((number?) -> number?) & ((string?) -> string?)' could not be converted into '(number) -> number'; none of the intersection parts are compatible"); +} + +TEST_CASE_FIXTURE(Fixture, "union_saturate_overloaded_functions") +{ + ScopedFastFlag sffs[] { + {"LuauSubtypeNormalizer", true}, + {"LuauTypeNormalization2", true}, + }; + + CheckResult result = check(R"( + local x : ((number) -> number) & ((string) -> string) + local y : ((number | string) -> (number | string)) = x -- OK + local z : ((number | boolean) -> (number | boolean)) = x -- Not OK + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ(toString(result.errors[0]), "Type '((number) -> number) & ((string) -> string)' could not be converted into '(boolean | number) -> boolean | number'; none of the intersection parts are compatible"); +} + +TEST_CASE_FIXTURE(Fixture, "intersection_of_tables") +{ + ScopedFastFlag sffs[] { + {"LuauSubtypeNormalizer", true}, + {"LuauTypeNormalization2", true}, + }; + + CheckResult result = check(R"( + local x : { p : number?, q : string? } & { p : number?, q : number?, r : number? } + local y : { p : number?, q : nil, r : number? } = x -- OK + local z : { p : nil } = x -- Not OK + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ(toString(result.errors[0]), "Type '{| p: number?, q: number?, r: number? |} & {| p: number?, q: string? |}' could not be converted into '{| p: nil |}'; none of the intersection parts are compatible"); +} + +TEST_CASE_FIXTURE(Fixture, "intersection_of_tables_with_top_properties") +{ + CheckResult result = check(R"( + local x : { p : number?, q : any } & { p : unknown, q : string? } + local y : { p : number?, q : string? } = x -- OK + local z : { p : string?, q : number? } = x -- Not OK + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ(toString(result.errors[0]), "Type '{| p: number?, q: any |} & {| p: unknown, q: string? |}' could not be converted into '{| p: string?, q: number? |}'; none of the intersection parts are compatible"); +} + +TEST_CASE_FIXTURE(Fixture, "intersection_of_tables_with_never_properties") +{ + ScopedFastFlag sffs[] { + {"LuauSubtypeNormalizer", true}, + {"LuauTypeNormalization2", true}, + }; + + CheckResult result = check(R"( + local x : { p : number?, q : never } & { p : never, q : string? } + local y : { p : never, q : never } = x -- OK + local z : never = x -- OK + )"); + + // TODO: this should not produce type errors, since never <: { p : never } + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ(toString(result.errors[0]), "Type '{| p: never, q: string? |} & {| p: number?, q: never |}' could not be converted into 'never'; none of the intersection parts are compatible"); +} + +TEST_CASE_FIXTURE(Fixture, "overloaded_functions_returning_intersections") +{ + ScopedFastFlag sffs[] { + {"LuauSubtypeNormalizer", true}, + {"LuauTypeNormalization2", true}, + }; + + CheckResult result = check(R"( + local x : ((number?) -> ({ p : number } & { q : number })) & ((string?) -> ({ p : number } & { r : number })) + local y : (nil) -> { p : number, q : number, r : number} = x -- OK + local z : (number?) -> { p : number, q : number, r : number} = x -- Not OK + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ(toString(result.errors[0]), "Type '((number?) -> {| p: number |} & {| q: number |}) & ((string?) -> {| p: number |} & {| r: number |})' could not be converted into '(number?) -> {| p: number, q: number, r: number |}'; none of the intersection parts are compatible"); +} + +TEST_CASE_FIXTURE(Fixture, "overloaded_functions_mentioning_generic") +{ + ScopedFastFlag sffs[] { + {"LuauSubtypeNormalizer", true}, + {"LuauTypeNormalization2", true}, + }; + + CheckResult result = check(R"( + function f() + local x : ((number?) -> (a | number)) & ((string?) -> (a | string)) + local y : (nil) -> a = x -- OK + local z : (number?) -> a = x -- Not OK + end + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ(toString(result.errors[0]), "Type '((number?) -> a | number) & ((string?) -> a | string)' could not be converted into '(number?) -> a'; none of the intersection parts are compatible"); +} + +TEST_CASE_FIXTURE(Fixture, "overloaded_functions_mentioning_generics") +{ + ScopedFastFlag sffs[] { + {"LuauSubtypeNormalizer", true}, + {"LuauTypeNormalization2", true}, + }; + + CheckResult result = check(R"( + function f() + local x : ((a?) -> (a | b)) & ((c?) -> (b | c)) + local y : (nil) -> ((a & c) | b) = x -- OK + local z : (a?) -> ((a & c) | b) = x -- Not OK + end + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ(toString(result.errors[0]), "Type '((a?) -> a | b) & ((c?) -> b | c)' could not be converted into '(a?) -> (a & c) | b'; none of the intersection parts are compatible"); +} + +TEST_CASE_FIXTURE(Fixture, "overloaded_functions_mentioning_generic_packs") +{ + ScopedFastFlag sffs[] { + {"LuauSubtypeNormalizer", true}, + {"LuauTypeNormalization2", true}, + }; + + CheckResult result = check(R"( + function f() + local x : ((number?, a...) -> (number?, b...)) & ((string?, a...) -> (string?, b...)) + local y : ((nil, a...) -> (nil, b...)) = x -- OK + local z : ((nil, b...) -> (nil, a...)) = x -- Not OK + end + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ(toString(result.errors[0]), "Type '((number?, a...) -> (number?, b...)) & ((string?, a...) -> (string?, b...))' could not be converted into '(nil, b...) -> (nil, a...)'; none of the intersection parts are compatible"); +} + +TEST_CASE_FIXTURE(Fixture, "overloadeded_functions_with_unknown_result") +{ + ScopedFastFlag sffs[] { + {"LuauSubtypeNormalizer", true}, + {"LuauTypeNormalization2", true}, + }; + + CheckResult result = check(R"( + function f() + local x : ((number) -> number) & ((nil) -> unknown) + local y : (number?) -> unknown = x -- OK + local z : (number?) -> number? = x -- Not OK + end + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ(toString(result.errors[0]), "Type '((nil) -> unknown) & ((number) -> number)' could not be converted into '(number?) -> number?'; none of the intersection parts are compatible"); +} + +TEST_CASE_FIXTURE(Fixture, "overloadeded_functions_with_unknown_arguments") +{ + ScopedFastFlag sffs[] { + {"LuauSubtypeNormalizer", true}, + {"LuauTypeNormalization2", true}, + }; + + CheckResult result = check(R"( + function f() + local x : ((number) -> number?) & ((unknown) -> string?) + local y : (number) -> nil = x -- OK + local z : (number?) -> nil = x -- Not OK + end + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ(toString(result.errors[0]), "Type '((number) -> number?) & ((unknown) -> string?)' could not be converted into '(number?) -> nil'; none of the intersection parts are compatible"); +} + +TEST_CASE_FIXTURE(Fixture, "overloadeded_functions_with_never_result") +{ + ScopedFastFlag sffs[] { + {"LuauSubtypeNormalizer", true}, + {"LuauTypeNormalization2", true}, + }; + + CheckResult result = check(R"( + function f() + local x : ((number) -> number) & ((nil) -> never) + local y : (number?) -> number = x -- OK + local z : (number?) -> never = x -- Not OK + end + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ(toString(result.errors[0]), "Type '((nil) -> never) & ((number) -> number)' could not be converted into '(number?) -> never'; none of the intersection parts are compatible"); +} + +TEST_CASE_FIXTURE(Fixture, "overloadeded_functions_with_never_arguments") +{ + ScopedFastFlag sffs[] { + {"LuauSubtypeNormalizer", true}, + {"LuauTypeNormalization2", true}, + }; + + CheckResult result = check(R"( + function f() + local x : ((number) -> number?) & ((never) -> string?) + local y : (never) -> nil = x -- OK + local z : (number?) -> nil = x -- Not OK + end + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ(toString(result.errors[0]), "Type '((never) -> string?) & ((number) -> number?)' could not be converted into '(number?) -> nil'; none of the intersection parts are compatible"); +} + +TEST_CASE_FIXTURE(Fixture, "overloadeded_functions_with_overlapping_results_and_variadics") +{ + CheckResult result = check(R"( + local x : ((string?) -> (string | number)) & ((number?) -> ...number) + local y : ((nil) -> (number, number?)) = x -- OK + local z : ((string | number) -> (number, number?)) = x -- Not OK + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ(toString(result.errors[0]), "Type '((number?) -> (...number)) & ((string?) -> number | string)' could not be converted into '(number | string) -> (number, number?)'; none of the intersection parts are compatible"); +} + +TEST_CASE_FIXTURE(Fixture, "overloadeded_functions_with_weird_typepacks_1") +{ + CheckResult result = check(R"( + function f() + local x : (() -> a...) & (() -> b...) + local y : (() -> b...) & (() -> a...) = x -- OK + local z : () -> () = x -- Not OK + end + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ(toString(result.errors[0]), "Type '(() -> (a...)) & (() -> (b...))' could not be converted into '() -> ()'; none of the intersection parts are compatible"); +} + +TEST_CASE_FIXTURE(Fixture, "overloadeded_functions_with_weird_typepacks_2") +{ + CheckResult result = check(R"( + function f() + local x : ((a...) -> ()) & ((b...) -> ()) + local y : ((b...) -> ()) & ((a...) -> ()) = x -- OK + local z : () -> () = x -- Not OK + end + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ(toString(result.errors[0]), "Type '((a...) -> ()) & ((b...) -> ())' could not be converted into '() -> ()'; none of the intersection parts are compatible"); +} + +TEST_CASE_FIXTURE(Fixture, "overloadeded_functions_with_weird_typepacks_3") +{ + CheckResult result = check(R"( + function f() + local x : (() -> a...) & (() -> (number?,a...)) + local y : (() -> (number?,a...)) & (() -> a...) = x -- OK + local z : () -> (number) = x -- Not OK + end + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ(toString(result.errors[0]), "Type '(() -> (a...)) & (() -> (number?, a...))' could not be converted into '() -> number'; none of the intersection parts are compatible"); +} + +TEST_CASE_FIXTURE(Fixture, "overloadeded_functions_with_weird_typepacks_4") +{ + CheckResult result = check(R"( + function f() + local x : ((a...) -> ()) & ((number,a...) -> number) + local y : ((number,a...) -> number) & ((a...) -> ()) = x -- OK + local z : (number?) -> () = x -- Not OK + end + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ(toString(result.errors[0]), "Type '((a...) -> ()) & ((number, a...) -> number)' could not be converted into '(number?) -> ()'; none of the intersection parts are compatible"); +} + +TEST_CASE_FIXTURE(BuiltinsFixture, "intersect_metatables") +{ + ScopedFastFlag sffs[] { + {"LuauSubtypeNormalizer", true}, + {"LuauTypeNormalization2", true}, + }; + + CheckResult result = check(R"( + local a : string? = nil + local b : number? = nil + + local x = setmetatable({}, { p = 5, q = a }); + local y = setmetatable({}, { q = b, r = "hi" }); + local z = setmetatable({}, { p = 5, q = nil, r = "hi" }); + + type X = typeof(x) + type Y = typeof(y) + type Z = typeof(z) + + local xy : X&Y = z; + local yx : Y&X = z; + z = xy; + z = yx; + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(BuiltinsFixture, "intersect_metatable_subtypes") +{ + ScopedFastFlag sffs[] { + {"LuauSubtypeNormalizer", true}, + {"LuauTypeNormalization2", true}, + }; + + CheckResult result = check(R"( + local x = setmetatable({ a = 5 }, { p = 5 }); + local y = setmetatable({ b = "hi" }, { p = 5, q = "hi" }); + local z = setmetatable({ a = 5, b = "hi" }, { p = 5, q = "hi" }); + + type X = typeof(x) + type Y = typeof(y) + type Z = typeof(z) + + local xy : X&Y = z; + local yx : Y&X = z; + z = xy; + z = yx; + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + + +TEST_CASE_FIXTURE(BuiltinsFixture, "intersect_metatables_with_properties") +{ + ScopedFastFlag sffs[] { + {"LuauSubtypeNormalizer", true}, + {"LuauTypeNormalization2", true}, + }; + + CheckResult result = check(R"( + local x = setmetatable({ a = 5 }, { p = 5 }); + local y = setmetatable({ b = "hi" }, { q = "hi" }); + local z = setmetatable({ a = 5, b = "hi" }, { p = 5, q = "hi" }); + + type X = typeof(x) + type Y = typeof(y) + type Z = typeof(z) + + local xy : X&Y = z; + z = xy; + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(BuiltinsFixture, "intersect_metatable_with table") +{ + ScopedFastFlag sffs[] { + {"LuauSubtypeNormalizer", true}, + {"LuauTypeNormalization2", true}, + }; + + CheckResult result = check(R"( + local x = setmetatable({ a = 5 }, { p = 5 }); + local z = setmetatable({ a = 5, b = "hi" }, { p = 5 }); + + type X = typeof(x) + type Y = { b : string } + type Z = typeof(z) + + -- TODO: once we have shape types, we should be able to initialize these with z + local xy : X&Y; + local yx : Y&X; + z = xy; + z = yx; + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "CLI-44817") +{ + ScopedFastFlag sffs[] { + {"LuauSubtypeNormalizer", true}, + {"LuauTypeNormalization2", true}, + }; + + CheckResult result = check(R"( + type X = {x: number} + type Y = {y: number} + type Z = {z: number} + + type XY = {x: number, y: number} + type XYZ = {x:number, y: number, z: number} + + local xy: XY = {x = 0, y = 0} + local xyz: XYZ = {x = 0, y = 0, z = 0} + + local xNy: X&Y = xy + local xNyNz: X&Y&Z = xyz + + local t1: XY = xNy -- Type 'X & Y' could not be converted into 'XY' + local t2: XY = xNyNz -- Type 'X & Y & Z' could not be converted into 'XY' + local t3: XYZ = xNyNz -- Type 'X & Y & Z' could not be converted into 'XYZ' + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.modules.test.cpp b/tests/TypeInfer.modules.test.cpp index ede84f4a..36943cac 100644 --- a/tests/TypeInfer.modules.test.cpp +++ b/tests/TypeInfer.modules.test.cpp @@ -10,6 +10,8 @@ #include "doctest.h" +LUAU_FASTFLAG(LuauInstantiateInSubtyping) + using namespace Luau; LUAU_FASTFLAG(LuauSpecialTypesAsterisked) @@ -248,7 +250,24 @@ end return m )"); - LUAU_REQUIRE_NO_ERRORS(result); + + if (FFlag::LuauInstantiateInSubtyping) + { + // though this didn't error before the flag, it seems as though it should error since fields of a table are invariant. + // the user's intent would likely be that these "method" fields would be read-only, but without an annotation, accepting this should be unsound. + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + CHECK_EQ(R"(Type 'n' could not be converted into 't1 where t1 = {- Clone: (t1) -> (a...) -}' +caused by: + Property 'Clone' is not compatible. Type '(a) -> ()' could not be converted into 't1 where t1 = ({- Clone: t1 -}) -> (a...)'; different number of generic type parameters)", + toString(result.errors[0])); + } + else + { + LUAU_REQUIRE_NO_ERRORS(result); + } + } TEST_CASE_FIXTURE(BuiltinsFixture, "custom_require_global") @@ -367,8 +386,6 @@ type Table = typeof(tbl) TEST_CASE_FIXTURE(BuiltinsFixture, "do_not_modify_imported_types_5") { - ScopedFastFlag luauInplaceDemoteSkipAllBound{"LuauInplaceDemoteSkipAllBound", true}; - fileResolver.source["game/A"] = R"( export type Type = {x: number, y: number} local arrayops = {} diff --git a/tests/TypeInfer.provisional.test.cpp b/tests/TypeInfer.provisional.test.cpp index 45740a0b..2aac6653 100644 --- a/tests/TypeInfer.provisional.test.cpp +++ b/tests/TypeInfer.provisional.test.cpp @@ -271,30 +271,6 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "bail_early_if_unification_is_too_complicated } } -// Should be in TypeInfer.tables.test.cpp -// It's unsound to instantiate tables containing generic methods, -// since mutating properties means table properties should be invariant. -// We currently allow this but we shouldn't! -TEST_CASE_FIXTURE(Fixture, "invariant_table_properties_means_instantiating_tables_in_call_is_unsound") -{ - CheckResult result = check(R"( - --!strict - local t = {} - function t.m(x) return x end - local a : string = t.m("hi") - local b : number = t.m(5) - function f(x : { m : (number)->number }) - x.m = function(x) return 1+x end - end - f(t) -- This shouldn't typecheck - local c : string = t.m("hi") - )"); - - // TODO: this should error! - // This should be fixed by replacing generic tables by generics with type bounds. - LUAU_REQUIRE_NO_ERRORS(result); -} - // FIXME: Move this test to another source file when removing FFlag::LuauLowerBoundsCalculation TEST_CASE_FIXTURE(Fixture, "do_not_ice_when_trying_to_pick_first_of_generic_type_pack") { @@ -608,7 +584,8 @@ TEST_CASE_FIXTURE(Fixture, "free_options_cannot_be_unified_together") InternalErrorReporter iceHandler; UnifierSharedState sharedState{&iceHandler}; - Unifier u{&arena, singletonTypes, Mode::Strict, NotNull{scope.get()}, Location{}, Variance::Covariant, sharedState}; + Normalizer normalizer{&arena, singletonTypes, NotNull{&sharedState}}; + Unifier u{NotNull{&normalizer}, Mode::Strict, NotNull{scope.get()}, Location{}, Variance::Covariant}; u.tryUnify(option1, option2); @@ -635,4 +612,87 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "for_in_loop_with_zero_iterators") LUAU_REQUIRE_NO_ERRORS(result); } +// Ideally, we would not try to export a function type with generic types from incorrect scope +TEST_CASE_FIXTURE(BuiltinsFixture, "generic_type_leak_to_module_interface") +{ + ScopedFastFlag LuauAnyifyModuleReturnGenerics{"LuauAnyifyModuleReturnGenerics", true}; + + fileResolver.source["game/A"] = R"( +local wrapStrictTable + +local metatable = { + __index = function(self, key) + local value = self.__tbl[key] + if type(value) == "table" then + -- unification of the free 'wrapStrictTable' with this function type causes generics of this function to leak out of scope + return wrapStrictTable(value, self.__name .. "." .. key) + end + return value + end, +} + +return wrapStrictTable + )"; + + frontend.check("game/A"); + + fileResolver.source["game/B"] = R"( +local wrapStrictTable = require(game.A) + +local Constants = {} + +return wrapStrictTable(Constants, "Constants") + )"; + + frontend.check("game/B"); + + ModulePtr m = frontend.moduleResolver.modules["game/B"]; + REQUIRE(m); + + std::optional result = first(m->getModuleScope()->returnType); + REQUIRE(result); + CHECK(get(*result)); +} + +TEST_CASE_FIXTURE(BuiltinsFixture, "generic_type_leak_to_module_interface_variadic") +{ + ScopedFastFlag LuauAnyifyModuleReturnGenerics{"LuauAnyifyModuleReturnGenerics", true}; + + fileResolver.source["game/A"] = R"( +local wrapStrictTable + +local metatable = { + __index = function(self, key, ...: T) + local value = self.__tbl[key] + if type(value) == "table" then + -- unification of the free 'wrapStrictTable' with this function type causes generics of this function to leak out of scope + return wrapStrictTable(value, self.__name .. "." .. key) + end + return ... + end, +} + +return wrapStrictTable + )"; + + frontend.check("game/A"); + + fileResolver.source["game/B"] = R"( +local wrapStrictTable = require(game.A) + +local Constants = {} + +return wrapStrictTable(Constants, "Constants") + )"; + + frontend.check("game/B"); + + ModulePtr m = frontend.moduleResolver.modules["game/B"]; + REQUIRE(m); + + std::optional result = first(m->getModuleScope()->returnType); + REQUIRE(result); + CHECK(get(*result)); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.tables.test.cpp b/tests/TypeInfer.tables.test.cpp index d183f650..a6d870fa 100644 --- a/tests/TypeInfer.tables.test.cpp +++ b/tests/TypeInfer.tables.test.cpp @@ -11,7 +11,8 @@ using namespace Luau; -LUAU_FASTFLAG(LuauLowerBoundsCalculation); +LUAU_FASTFLAG(LuauLowerBoundsCalculation) +LUAU_FASTFLAG(LuauInstantiateInSubtyping) TEST_SUITE_BEGIN("TableTests"); @@ -2038,11 +2039,22 @@ caused by: caused by: Property 'y' is not compatible. Type 'string' could not be converted into 'number')"); - CHECK_EQ(toString(result.errors[1]), R"(Type 'b2' could not be converted into 'a2' + if (FFlag::LuauInstantiateInSubtyping) + { + CHECK_EQ(toString(result.errors[1]), R"(Type 'b2' could not be converted into 'a2' +caused by: + Type '{ __call: (a, b) -> () }' could not be converted into '{ __call: (a) -> () }' +caused by: + Property '__call' is not compatible. Type '(a, b) -> ()' could not be converted into '(a) -> ()'; different number of generic type parameters)"); + } + else + { + CHECK_EQ(toString(result.errors[1]), R"(Type 'b2' could not be converted into 'a2' caused by: Type '{ __call: (a, b) -> () }' could not be converted into '{ __call: (a) -> () }' caused by: Property '__call' is not compatible. Type '(a, b) -> ()' could not be converted into '(a) -> ()'; different number of generic type parameters)"); + } } TEST_CASE_FIXTURE(Fixture, "error_detailed_indexer_key") @@ -3173,4 +3185,53 @@ caused by: CHECK_EQ("(t1) -> string where t1 = {+ absolutely_no_scalar_has_this_method: (t1) -> (a, b...) +}", toString(requireType("f"))); } +TEST_CASE_FIXTURE(Fixture, "invariant_table_properties_means_instantiating_tables_in_call_is_unsound") +{ + ScopedFastFlag sff[]{ + {"LuauInstantiateInSubtyping", true}, + }; + + CheckResult result = check(R"( + --!strict + local t = {} + function t.m(x) return x end + local a : string = t.m("hi") + local b : number = t.m(5) + function f(x : { m : (number)->number }) + x.m = function(x) return 1+x end + end + f(t) -- This shouldn't typecheck + local c : string = t.m("hi") + )"); + + LUAU_REQUIRE_ERRORS(result); + CHECK_EQ(toString(result.errors[0]), R"(Type 't' could not be converted into '{| m: (number) -> number |}' +caused by: + Property 'm' is not compatible. Type '(a) -> a' could not be converted into '(number) -> number'; different number of generic type parameters)"); + // this error message is not great since the underlying issue is that the context is invariant, + // and `(number) -> number` cannot be a subtype of `(a) -> a`. +} + + +TEST_CASE_FIXTURE(BuiltinsFixture, "generic_table_instantiation_potential_regression") +{ + CheckResult result = check(R"( +--!strict + +function f(x) + x.p = 5 + return x +end +local g : ({ p : number, q : string }) -> ({ p : number, r : boolean }) = f + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + MissingProperties* error = get(result.errors[0]); + REQUIRE(error != nullptr); + REQUIRE(error->properties.size() == 1); + + CHECK_EQ("r", error->properties[0]); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.test.cpp b/tests/TypeInfer.test.cpp index 8ed61b49..26171c51 100644 --- a/tests/TypeInfer.test.cpp +++ b/tests/TypeInfer.test.cpp @@ -17,7 +17,9 @@ LUAU_FASTFLAG(LuauLowerBoundsCalculation); LUAU_FASTFLAG(LuauFixLocationSpanTableIndexExpr); LUAU_FASTFLAG(DebugLuauDeferredConstraintResolution); +LUAU_FASTFLAG(LuauInstantiateInSubtyping); LUAU_FASTFLAG(LuauSpecialTypesAsterisked); +LUAU_FASTFLAG(LuauCheckGenericHOFTypes); using namespace Luau; @@ -999,7 +1001,26 @@ TEST_CASE_FIXTURE(Fixture, "cli_50041_committing_txnlog_in_apollo_client_error") end )"); - LUAU_REQUIRE_NO_ERRORS(result); + if (FFlag::LuauInstantiateInSubtyping && !FFlag::LuauCheckGenericHOFTypes) + { + // though this didn't error before the flag, it seems as though it should error since fields of a table are invariant. + // the user's intent would likely be that these "method" fields would be read-only, but without an annotation, accepting this should be unsound. + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + CHECK_EQ(R"(Type 't1 where t1 = {+ getStoreFieldName: (t1, {| fieldName: string |} & {| from: number? |}) -> (a, b...) +}' could not be converted into 'Policies' +caused by: + Property 'getStoreFieldName' is not compatible. Type 't1 where t1 = ({+ getStoreFieldName: t1 +}, {| fieldName: string |} & {| from: number? |}) -> (a, b...)' could not be converted into '(Policies, FieldSpecifier) -> string' +caused by: + Argument #2 type is not compatible. Type 'FieldSpecifier' could not be converted into 'FieldSpecifier & {| from: number? |}' +caused by: + Not all intersection parts are compatible. Table type 'FieldSpecifier' not compatible with type '{| from: number? |}' because the former has extra field 'fieldName')", + toString(result.errors[0])); + } + else + { + LUAU_REQUIRE_NO_ERRORS(result); + } } TEST_CASE_FIXTURE(Fixture, "type_infer_recursion_limit_no_ice") @@ -1020,6 +1041,43 @@ TEST_CASE_FIXTURE(Fixture, "type_infer_recursion_limit_no_ice") CHECK_EQ("Code is too complex to typecheck! Consider simplifying the code around this area", toString(result.errors[0])); } +TEST_CASE_FIXTURE(Fixture, "type_infer_recursion_limit_normalizer") +{ + ScopedFastInt sfi("LuauTypeInferRecursionLimit", 10); + ScopedFastFlag sffs[] { + {"LuauSubtypeNormalizer", true}, + {"LuauTypeNormalization2", true}, + {"LuauAutocompleteDynamicLimits", true}, + }; + + CheckResult result = check(R"( + function f() + local x : a&b&c&d&e&f&g&h&(i?) + local y : (a&b&c&d&e&f&g&h&i)? = x + end + )"); + + LUAU_REQUIRE_ERRORS(result); + CHECK_EQ("Internal error: Code is too complex to typecheck! Consider adding type annotations around this area", toString(result.errors[0])); +} + +TEST_CASE_FIXTURE(Fixture, "type_infer_cache_limit_normalizer") +{ + ScopedFastInt sfi("LuauNormalizeCacheLimit", 10); + ScopedFastFlag sffs[] { + {"LuauSubtypeNormalizer", true}, + {"LuauTypeNormalization2", true}, + }; + + CheckResult result = check(R"( + local x : ((number) -> number) & ((string) -> string) & ((nil) -> nil) & (({}) -> {}) + local y : (number | string | nil | {}) -> (number | string | nil | {}) = x + )"); + + LUAU_REQUIRE_ERRORS(result); + CHECK_EQ("Internal error: Code is too complex to typecheck! Consider adding type annotations around this area", toString(result.errors[0])); +} + TEST_CASE_FIXTURE(Fixture, "follow_on_new_types_in_substitution") { CheckResult result = check(R"( diff --git a/tests/TypeInfer.tryUnify.test.cpp b/tests/TypeInfer.tryUnify.test.cpp index 3911c520..dedb7d28 100644 --- a/tests/TypeInfer.tryUnify.test.cpp +++ b/tests/TypeInfer.tryUnify.test.cpp @@ -17,8 +17,8 @@ struct TryUnifyFixture : Fixture ScopePtr globalScope{new Scope{arena.addTypePack({TypeId{}})}}; InternalErrorReporter iceHandler; UnifierSharedState unifierState{&iceHandler}; - - Unifier state{&arena, singletonTypes, Mode::Strict, NotNull{globalScope.get()}, Location{}, Variance::Covariant, unifierState}; + Normalizer normalizer{&arena, singletonTypes, NotNull{&unifierState}}; + Unifier state{NotNull{&normalizer}, Mode::Strict, NotNull{globalScope.get()}, Location{}, Variance::Covariant}; }; TEST_SUITE_BEGIN("TryUnifyTests"); diff --git a/tests/TypeInfer.typePacks.cpp b/tests/TypeInfer.typePacks.cpp index 7d33809f..eb61c396 100644 --- a/tests/TypeInfer.typePacks.cpp +++ b/tests/TypeInfer.typePacks.cpp @@ -1000,4 +1000,23 @@ TEST_CASE_FIXTURE(Fixture, "unify_variadic_tails_in_arguments_free") CHECK_EQ(toString(result.errors[0]), "Type 'number' could not be converted into 'boolean'"); } +TEST_CASE_FIXTURE(BuiltinsFixture, "type_packs_with_tails_in_vararg_adjustment") +{ + ScopedFastFlag luauFixVarargExprHeadType{"LuauFixVarargExprHeadType", true}; + + CheckResult result = check(R"( + local function wrapReject(fn: (self: any, ...TArg) -> ...TResult): (self: any, ...TArg) -> ...TResult + return function(self, ...) + local arguments = { ... } + local ok, result = pcall(function() + return fn(self, table.unpack(arguments)) + end) + return result + end + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.unionTypes.test.cpp b/tests/TypeInfer.unionTypes.test.cpp index 8eb485e9..64c9b563 100644 --- a/tests/TypeInfer.unionTypes.test.cpp +++ b/tests/TypeInfer.unionTypes.test.cpp @@ -541,5 +541,182 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "table_union_write_indirect") R"(Type '(string) -> number' could not be converted into '((number) -> string) | ((number) -> string)'; none of the union options are compatible)"); } +TEST_CASE_FIXTURE(Fixture, "union_true_and_false") +{ + ScopedFastFlag sffs[] { + {"LuauSubtypeNormalizer", true}, + {"LuauTypeNormalization2", true}, + }; + + CheckResult result = check(R"( + local x : boolean + local y1 : (true | false) = x -- OK + local y2 : (true | false | (string & number)) = x -- OK + local y3 : (true | (string & number) | false) = x -- OK + local y4 : (true | (boolean & true) | false) = x -- OK + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "union_of_functions") +{ + ScopedFastFlag sffs[] { + {"LuauSubtypeNormalizer", true}, + {"LuauTypeNormalization2", true}, + }; + + CheckResult result = check(R"( + local x : (number) -> number? + local y : ((number?) -> number?) | ((number) -> number) = x -- OK + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "union_of_generic_functions") +{ + CheckResult result = check(R"( + local x : (a) -> a? + local y : ((a?) -> a?) | ((b) -> b) = x -- Not OK + )"); + + // TODO: should this example typecheck? + LUAU_REQUIRE_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "union_of_generic_typepack_functions") +{ + CheckResult result = check(R"( + local x : (number, a...) -> (number?, a...) + local y : ((number?, a...) -> (number?, a...)) | ((number, b...) -> (number, b...)) = x -- Not OK + )"); + + // TODO: should this example typecheck? + LUAU_REQUIRE_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "union_of_functions_mentioning_generics") +{ + ScopedFastFlag sffs[] { + {"LuauSubtypeNormalizer", true}, + {"LuauTypeNormalization2", true}, + }; + + CheckResult result = check(R"( + function f() + local x : (a) -> a? + local y : ((a?) -> nil) | ((a) -> a) = x -- OK + local z : ((b?) -> nil) | ((b) -> b) = x -- Not OK + end + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ(toString(result.errors[0]), "Type '(a) -> a?' could not be converted into '((b) -> b) | ((b?) -> nil)'; none of the union options are compatible"); +} + +TEST_CASE_FIXTURE(Fixture, "union_of_functions_mentioning_generic_typepacks") +{ + ScopedFastFlag sffs[] { + {"LuauSubtypeNormalizer", true}, + {"LuauTypeNormalization2", true}, + }; + + CheckResult result = check(R"( + function f() + local x : (number, a...) -> (number?, a...) + local y : ((number | string, a...) -> (number, a...)) | ((number?, a...) -> (nil, a...)) = x -- OK + local z : ((number) -> number) | ((number?, a...) -> (number?, a...)) = x -- Not OK + end + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ(toString(result.errors[0]), "Type '(number, a...) -> (number?, a...)' could not be converted into '((number) -> number) | ((number?, a...) -> (number?, a...))'; none of the union options are compatible"); +} + +TEST_CASE_FIXTURE(Fixture, "union_of_functions_with_mismatching_arg_arities") +{ + ScopedFastFlag sffs[] { + {"LuauSubtypeNormalizer", true}, + {"LuauTypeNormalization2", true}, + }; + + CheckResult result = check(R"( + local x : (number) -> number? + local y : ((number?) -> number) | ((number | string) -> nil) = x -- OK + local z : ((number, string?) -> number) | ((number) -> nil) = x -- Not OK + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ(toString(result.errors[0]), "Type '(number) -> number?' could not be converted into '((number) -> nil) | ((number, string?) -> number)'; none of the union options are compatible"); +} + +TEST_CASE_FIXTURE(Fixture, "union_of_functions_with_mismatching_result_arities") +{ + ScopedFastFlag sffs[] { + {"LuauSubtypeNormalizer", true}, + {"LuauTypeNormalization2", true}, + }; + + CheckResult result = check(R"( + local x : () -> (number | string) + local y : (() -> number) | (() -> string) = x -- OK + local z : (() -> number) | (() -> (string, string)) = x -- Not OK + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ(toString(result.errors[0]), "Type '() -> number | string' could not be converted into '(() -> (string, string)) | (() -> number)'; none of the union options are compatible"); +} + +TEST_CASE_FIXTURE(Fixture, "union_of_functions_with_variadics") +{ + ScopedFastFlag sffs[] { + {"LuauSubtypeNormalizer", true}, + {"LuauTypeNormalization2", true}, + }; + + CheckResult result = check(R"( + local x : (...nil) -> (...number?) + local y : ((...string?) -> (...number)) | ((...number?) -> nil) = x -- OK + local z : ((...string?) -> (...number)) | ((...string?) -> nil) = x -- OK + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ(toString(result.errors[0]), "Type '(...nil) -> (...number?)' could not be converted into '((...string?) -> (...number)) | ((...string?) -> nil)'; none of the union options are compatible"); +} + +TEST_CASE_FIXTURE(Fixture, "union_of_functions_with_mismatching_arg_variadics") +{ + ScopedFastFlag sffs[] { + {"LuauSubtypeNormalizer", true}, + {"LuauTypeNormalization2", true}, + }; + + CheckResult result = check(R"( + local x : (number) -> () + local y : ((number?) -> ()) | ((...number) -> ()) = x -- OK + local z : ((number?) -> ()) | ((...number?) -> ()) = x -- Not OK + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ(toString(result.errors[0]), "Type '(number) -> ()' could not be converted into '((...number?) -> ()) | ((number?) -> ())'; none of the union options are compatible"); +} + +TEST_CASE_FIXTURE(Fixture, "union_of_functions_with_mismatching_result_variadics") +{ + ScopedFastFlag sffs[] { + {"LuauSubtypeNormalizer", true}, + {"LuauTypeNormalization2", true}, + }; + + CheckResult result = check(R"( + local x : () -> (number?, ...number) + local y : (() -> (...number)) | (() -> nil) = x -- OK + local z : (() -> (...number)) | (() -> number) = x -- OK + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ(toString(result.errors[0]), "Type '() -> (number?, ...number)' could not be converted into '(() -> (...number)) | (() -> number)'; none of the union options are compatible"); +} TEST_SUITE_END(); diff --git a/tests/conformance/basic.lua b/tests/conformance/basic.lua index 9c19da59..b09b087b 100644 --- a/tests/conformance/basic.lua +++ b/tests/conformance/basic.lua @@ -508,6 +508,9 @@ assert((function() function cmp(a,b) return ab,a>=b end return concat( assert((function() function cmp(a,b) return ab,a>=b end return concat(cmp('abc', 'abd')) end)() == "true,true,false,false") assert((function() function cmp(a,b) return ab,a>=b end return concat(cmp('ab\\0c', 'ab\\0d')) end)() == "true,true,false,false") assert((function() function cmp(a,b) return ab,a>=b end return concat(cmp('ab\\0c', 'ab\\0')) end)() == "false,false,true,true") +assert((function() function cmp(a,b) return ab,a>=b end return concat(cmp('\\0a', '\\0b')) end)() == "true,true,false,false") +assert((function() function cmp(a,b) return ab,a>=b end return concat(cmp('a', 'a\\0')) end)() == "true,true,false,false") +assert((function() function cmp(a,b) return ab,a>=b end return concat(cmp('a', '\200')) end)() == "true,true,false,false") -- array access assert((function() local a = {4,5,6} return a[3] end)() == 6) diff --git a/tests/main.cpp b/tests/main.cpp index 3e480c9f..3f564c07 100644 --- a/tests/main.cpp +++ b/tests/main.cpp @@ -21,6 +21,7 @@ #include #endif +#include #include // Indicates if verbose output is enabled; can be overridden via --verbose @@ -30,6 +31,10 @@ bool verbose = false; // Default optimization level for conformance test; can be overridden via -On int optimizationLevel = 1; +// Something to seed a pseudorandom number generator with. Defaults to +// something from std::random_device. +std::optional randomSeed; + static bool skipFastFlag(const char* flagName) { if (strncmp(flagName, "Test", 4) == 0) @@ -261,6 +266,16 @@ int main(int argc, char** argv) optimizationLevel = level; } + int rseed = -1; + if (doctest::parseIntOption(argc, argv, "--random-seed=", doctest::option_int, rseed)) + randomSeed = unsigned(rseed); + + if (doctest::parseOption(argc, argv, "--randomize") && !randomSeed) + { + randomSeed = std::random_device()(); + printf("Using RNG seed %u\n", *randomSeed); + } + if (std::vector flags; doctest::parseCommaSepArgs(argc, argv, "--fflags=", flags)) setFastFlags(flags); @@ -295,6 +310,8 @@ int main(int argc, char** argv) printf(" --verbose Enables verbose output (e.g. lua 'print' statements)\n"); printf(" --fflags= Sets specified fast flags\n"); printf(" --list-fflags List all fast flags\n"); + printf(" --randomize Use a random RNG seed\n"); + printf(" --random-seed=n Use a particular RNG seed\n"); } return result; } diff --git a/tools/lldb_formatters.lldb b/tools/lldb_formatters.lldb index 4a5acd74..f10faa94 100644 --- a/tools/lldb_formatters.lldb +++ b/tools/lldb_formatters.lldb @@ -5,3 +5,6 @@ type synthetic add -x "^Luau::Variant<.+>$" -l lldb_formatters.LuauVariantSynthe type summary add -x "^Luau::Variant<.+>$" -F lldb_formatters.luau_variant_summary type synthetic add -x "^Luau::AstArray<.+>$" -l lldb_formatters.AstArraySyntheticChildrenProvider + +type summary add --summary-string "${var.line}:${var.column}" Luau::Position +type summary add --summary-string "${var.begin}-${var.end}" Luau::Location diff --git a/tools/natvis/Ast.natvis b/tools/natvis/Ast.natvis index 322eb8f6..18e7b762 100644 --- a/tools/natvis/Ast.natvis +++ b/tools/natvis/Ast.natvis @@ -22,4 +22,25 @@ + + {value,na} + + + + {name.value,na} + + + + local {local->name.value,na} + global {global.value,na} + + + + {line}:{column} + + + + {begin}-{end} + + diff --git a/tools/test_dcr.py b/tools/test_dcr.py index 1e3a5017..5f1c8705 100644 --- a/tools/test_dcr.py +++ b/tools/test_dcr.py @@ -22,6 +22,10 @@ def safeParseInt(i, default=0): return default +def makeDottedName(path): + return ".".join(path) + + class Handler(x.ContentHandler): def __init__(self, failList): self.currentTest = [] @@ -41,7 +45,7 @@ class Handler(x.ContentHandler): if self.currentTest: passed = attrs["test_case_success"] == "true" - dottedName = ".".join(self.currentTest) + dottedName = makeDottedName(self.currentTest) # Sometimes we get multiple XML trees for the same test. All of # them must report a pass in order for us to consider the test @@ -60,6 +64,10 @@ class Handler(x.ContentHandler): self.currentTest.pop() +def print_stderr(*args, **kw): + print(*args, **kw, file=sys.stderr) + + def main(): parser = argparse.ArgumentParser( description="Run Luau.UnitTest with deferred constraint resolution enabled" @@ -80,6 +88,16 @@ def main(): help="Write a new faillist.txt after running tests.", ) + parser.add_argument("--randomize", action="store_true", help="Pick a random seed") + + parser.add_argument( + "--random-seed", + action="store", + dest="random_seed", + type=int, + help="Accept a specific RNG seed", + ) + args = parser.parse_args() failList = loadFailList() @@ -90,7 +108,12 @@ def main(): "--fflags=true,DebugLuauDeferredConstraintResolution=true", ] - print('>', ' '.join(commandLine), file=sys.stderr) + if args.random_seed: + commandLine.append("--random-seed=" + str(args.random_seed)) + elif args.randomize: + commandLine.append("--randomize") + + print_stderr(">", " ".join(commandLine)) p = sp.Popen( commandLine, @@ -104,15 +127,21 @@ def main(): sys.stdout.buffer.write(line) return else: - x.parse(p.stdout, handler) + try: + x.parse(p.stdout, handler) + except x.SAXParseException as e: + print_stderr( + f"XML parsing failed during test {makeDottedName(handler.currentTest)}. That probably means that the test crashed" + ) + sys.exit(1) p.wait() for testName, passed in handler.results.items(): if passed and testName in failList: - print("UNEXPECTED: {} should have failed".format(testName)) + print_stderr(f"UNEXPECTED: {testName} should have failed") elif not passed and testName not in failList: - print("UNEXPECTED: {} should have passed".format(testName)) + print_stderr(f"UNEXPECTED: {testName} should have passed") if args.write: newFailList = sorted( @@ -126,14 +155,11 @@ def main(): with open(FAIL_LIST_PATH, "w", newline="\n") as f: for name in newFailList: print(name, file=f) - print("Updated faillist.txt", file=sys.stderr) + print_stderr("Updated faillist.txt") if handler.numSkippedTests > 0: - print( - "{} test(s) were skipped! That probably means that a test segfaulted!".format( - handler.numSkippedTests - ), - file=sys.stderr, + print_stderr( + f"{handler.numSkippedTests} test(s) were skipped! That probably means that a test segfaulted!" ) sys.exit(1) @@ -143,7 +169,7 @@ def main(): ) if ok: - print("Everything in order!", file=sys.stderr) + print_stderr("Everything in order!") sys.exit(0 if ok else 1) From d6aa35583e4be14304d2a17c7d11c8819756beb6 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Thu, 6 Oct 2022 18:20:19 -0700 Subject: [PATCH 372/399] Add new-release action to automatically build & publish release artifacts --- .github/workflows/new-release.yml | 83 +++++++++++++++++++++++++++++++ 1 file changed, 83 insertions(+) create mode 100644 .github/workflows/new-release.yml diff --git a/.github/workflows/new-release.yml b/.github/workflows/new-release.yml new file mode 100644 index 00000000..90143323 --- /dev/null +++ b/.github/workflows/new-release.yml @@ -0,0 +1,83 @@ +name: new-release + +on: + workflow_dispatch: + inputs: + version: + required: true + description: Release version including 0. + +permissions: + contents: write + +jobs: + create-release: + runs-on: ubuntu-latest + outputs: + upload_url: ${{ steps.create_release.outputs.upload_url }} + steps: + - name: create release + id: create_release + uses: actions/create-release@v1 + env: + GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} + with: + tag_name: ${{ github.event.inputs.version }} + release_name: ${{ github.event.inputs.version }} + draft: true + + build: + needs: ["create-release"] + strategy: + matrix: + os: [ubuntu, macos, windows] + name: ${{matrix.os}} + runs-on: ${{matrix.os}}-latest + steps: + - uses: actions/checkout@v1 + - name: configure + run: cmake . -DCMAKE_BUILD_TYPE=Release + - name: build + run: cmake --build . --target Luau.Repl.CLI Luau.Analyze.CLI --config Release -j 2 + - name: pack + if: matrix.os != 'windows' + run: zip luau-${{matrix.os}}.zip luau* + - name: pack + if: matrix.os == 'windows' + run: 7z a luau-${{matrix.os}}.zip .\Release\luau*.exe + - uses: actions/upload-release-asset@v1 + env: + GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} + with: + upload_url: ${{ needs.create-release.outputs.upload_url }} + asset_path: luau-${{matrix.os}}.zip + asset_name: luau-${{matrix.os}}.zip + asset_content_type: application/octet-stream + + web: + needs: ["create-release"] + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v1 + - uses: actions/checkout@v2 + with: + repository: emscripten-core/emsdk + path: emsdk + - name: emsdk install + run: | + cd emsdk + ./emsdk install latest + ./emsdk activate latest + - name: make + run: | + source emsdk/emsdk_env.sh + emcmake cmake . -DLUAU_BUILD_WEB=ON -DCMAKE_BUILD_TYPE=Release + make -j2 Luau.Web + - uses: actions/upload-release-asset@v1 + env: + GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} + with: + upload_url: ${{ needs.create-release.outputs.upload_url }} + asset_path: Luau.Web.js + asset_name: Luau.Web.js + asset_content_type: application/octet-stream From ff736fd3e41c7c0deea5d54f7f0b1eda321820ee Mon Sep 17 00:00:00 2001 From: JohnnyMorganz Date: Fri, 14 Oct 2022 15:28:54 +0100 Subject: [PATCH 373/399] Fix segfault in `loadDefinition` for unit tests (#705) `module` can be empty if the definition file has syntax errors --- tests/Fixture.cpp | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/tests/Fixture.cpp b/tests/Fixture.cpp index 4d3c8854..05c07b90 100644 --- a/tests/Fixture.cpp +++ b/tests/Fixture.cpp @@ -429,7 +429,8 @@ LoadDefinitionFileResult Fixture::loadDefinition(const std::string& source) LoadDefinitionFileResult result = frontend.loadDefinitionFile(source, "@test"); freeze(typeChecker.globalTypes); - dumpErrors(result.module); + if (result.module) + dumpErrors(result.module); REQUIRE_MESSAGE(result.success, "loadDefinition: unable to load definition file"); return result; } From 76070f8da2de3a4677b804a673bfd9efd5f60241 Mon Sep 17 00:00:00 2001 From: vegorov-rbx <75688451+vegorov-rbx@users.noreply.github.com> Date: Fri, 14 Oct 2022 12:48:41 -0700 Subject: [PATCH 374/399] Sync to upstream/release/549 (#707) * Reoptimized math.min/max/bit32 builtins assuming at least 2 arguments are used (1-2% lift on some benchmarks) * Type errors that mention function types no longer have redundant parenthesis around return type * Luau REPL now supports --compile=remarks which displays the source code with optimization remarks embedded as comments * Builtin calls are slightly faster when called with 1-2 arguments (~1% improvement in some benchmarks) --- Analysis/include/Luau/ConstraintSolver.h | 8 +- Analysis/include/Luau/DcrLogger.h | 4 +- Analysis/include/Luau/Error.h | 1 - Analysis/include/Luau/JsonEmitter.h | 2 +- Analysis/include/Luau/Normalize.h | 22 +- Analysis/include/Luau/TypeInfer.h | 2 +- Analysis/include/Luau/Unifier.h | 11 +- Analysis/src/BuiltinDefinitions.cpp | 5 +- Analysis/src/ConstraintGraphBuilder.cpp | 5 +- Analysis/src/ConstraintSolver.cpp | 71 +- Analysis/src/DcrLogger.cpp | 11 +- Analysis/src/Error.cpp | 43 +- Analysis/src/Linter.cpp | 20 +- Analysis/src/Module.cpp | 39 - Analysis/src/Normalize.cpp | 133 ++- Analysis/src/Substitution.cpp | 4 - Analysis/src/ToString.cpp | 4 +- Analysis/src/TypeChecker2.cpp | 21 +- Analysis/src/TypeInfer.cpp | 308 +------ Analysis/src/TypeUtils.cpp | 4 +- Analysis/src/TypeVar.cpp | 18 +- Analysis/src/Unifier.cpp | 23 +- Ast/include/Luau/StringUtils.h | 8 +- CLI/Repl.cpp | 71 +- CMakeLists.txt | 11 +- CodeGen/include/Luau/AssemblyBuilderX64.h | 19 +- CodeGen/include/Luau/CodeAllocator.h | 3 +- CodeGen/include/Luau/CodeGen.h | 24 + CodeGen/include/Luau/Condition.h | 3 + CodeGen/src/AssemblyBuilderX64.cpp | 37 +- CodeGen/src/CodeGen.cpp | 449 ++++++++++ CodeGen/src/CodeGenX64.cpp | 154 ++++ CodeGen/src/CodeGenX64.h | 18 + CodeGen/src/CustomExecUtils.h | 145 ++++ CodeGen/src/EmitBuiltinsX64.cpp | 109 +++ CodeGen/src/EmitBuiltinsX64.h | 28 + CodeGen/src/EmitCommonX64.cpp | 345 ++++++++ CodeGen/src/EmitCommonX64.h | 175 ++++ CodeGen/src/EmitInstructionX64.cpp | 925 +++++++++++++++++++++ CodeGen/src/EmitInstructionX64.h | 66 ++ CodeGen/src/Fallbacks.cpp | 251 ++++-- CodeGen/src/Fallbacks.h | 162 ++-- CodeGen/src/NativeState.cpp | 122 +++ CodeGen/src/NativeState.h | 94 +++ Common/include/Luau/Common.h | 6 + Common/include/Luau/ExperimentalFlags.h | 1 - Compiler/include/Luau/BytecodeBuilder.h | 7 + Compiler/src/BytecodeBuilder.cpp | 37 + Compiler/src/Compiler.cpp | 29 +- Makefile | 9 +- Sources.cmake | 14 + VM/include/lua.h | 28 +- VM/include/luaconf.h | 2 +- VM/src/lbuiltins.cpp | 98 +-- VM/src/lfunc.cpp | 2 +- VM/src/lobject.cpp | 2 - VM/src/lobject.h | 1 + VM/src/lstate.cpp | 6 + VM/src/lstate.h | 9 +- VM/src/lvmexecute.cpp | 34 +- VM/src/lvmload.cpp | 1 + bench/tests/sha256.lua | 19 +- tests/AssemblyBuilderX64.test.cpp | 15 +- tests/AstQueryDsl.cpp | 2 +- tests/AstQueryDsl.h | 2 +- tests/Compiler.test.cpp | 38 +- tests/Conformance.test.cpp | 54 +- tests/ConstraintGraphBuilderFixture.cpp | 2 +- tests/ConstraintGraphBuilderFixture.h | 2 +- tests/Fixture.cpp | 3 +- tests/Frontend.test.cpp | 2 - tests/Linter.test.cpp | 2 - tests/Module.test.cpp | 8 +- tests/Normalize.test.cpp | 702 +--------------- tests/RuntimeLimits.test.cpp | 7 +- tests/ToDot.test.cpp | 29 +- tests/ToString.test.cpp | 17 + tests/TypeInfer.annotations.test.cpp | 48 +- tests/TypeInfer.builtins.test.cpp | 22 +- tests/TypeInfer.classes.test.cpp | 4 +- tests/TypeInfer.functions.test.cpp | 182 +--- tests/TypeInfer.generics.test.cpp | 29 +- tests/TypeInfer.intersectionTypes.test.cpp | 117 +-- tests/TypeInfer.loops.test.cpp | 6 +- tests/TypeInfer.modules.test.cpp | 6 +- tests/TypeInfer.provisional.test.cpp | 284 ++++--- tests/TypeInfer.refinements.test.cpp | 6 +- tests/TypeInfer.singletons.test.cpp | 22 - tests/TypeInfer.tables.test.cpp | 12 +- tests/TypeInfer.test.cpp | 58 +- tests/TypeInfer.tryUnify.test.cpp | 4 +- tests/TypeInfer.typePacks.cpp | 10 +- tests/TypeInfer.unionTypes.test.cpp | 54 +- tests/conformance/basic.lua | 7 + tests/conformance/bitwise.lua | 8 + tests/conformance/debugger.lua | 15 + tests/conformance/events.lua | 39 +- tests/conformance/interrupt.lua | 9 + tests/conformance/math.lua | 39 +- tests/conformance/safeenv.lua | 23 + tests/main.cpp | 14 +- tools/faillist.txt | 74 +- tools/lvmexecute_split.py | 23 +- tools/test_dcr.py | 39 +- 104 files changed, 4099 insertions(+), 2223 deletions(-) create mode 100644 CodeGen/include/Luau/CodeGen.h create mode 100644 CodeGen/src/CodeGen.cpp create mode 100644 CodeGen/src/CodeGenX64.cpp create mode 100644 CodeGen/src/CodeGenX64.h create mode 100644 CodeGen/src/CustomExecUtils.h create mode 100644 CodeGen/src/EmitBuiltinsX64.cpp create mode 100644 CodeGen/src/EmitBuiltinsX64.h create mode 100644 CodeGen/src/EmitCommonX64.cpp create mode 100644 CodeGen/src/EmitCommonX64.h create mode 100644 CodeGen/src/EmitInstructionX64.cpp create mode 100644 CodeGen/src/EmitInstructionX64.h create mode 100644 CodeGen/src/NativeState.cpp create mode 100644 CodeGen/src/NativeState.h create mode 100644 tests/conformance/safeenv.lua diff --git a/Analysis/include/Luau/ConstraintSolver.h b/Analysis/include/Luau/ConstraintSolver.h index 9d5aadfb..0bf6d1bc 100644 --- a/Analysis/include/Luau/ConstraintSolver.h +++ b/Analysis/include/Luau/ConstraintSolver.h @@ -76,8 +76,8 @@ struct ConstraintSolver DcrLogger* logger; - explicit ConstraintSolver(NotNull normalizer, NotNull rootScope, ModuleName moduleName, - NotNull moduleResolver, std::vector requireCycles, DcrLogger* logger); + explicit ConstraintSolver(NotNull normalizer, NotNull rootScope, ModuleName moduleName, NotNull moduleResolver, + std::vector requireCycles, DcrLogger* logger); // Randomize the order in which to dispatch constraints void randomize(unsigned seed); @@ -88,7 +88,9 @@ struct ConstraintSolver **/ void run(); - bool done(); + bool isDone(); + + void finalizeModule(); /** Attempt to dispatch a constraint. Returns true if it was successful. If * tryDispatch() returns false, the constraint remains in the unsolved set diff --git a/Analysis/include/Luau/DcrLogger.h b/Analysis/include/Luau/DcrLogger.h index bd8672e3..30d2e15e 100644 --- a/Analysis/include/Luau/DcrLogger.h +++ b/Analysis/include/Luau/DcrLogger.h @@ -112,11 +112,13 @@ struct DcrLogger void popBlock(NotNull block); void captureInitialSolverState(const Scope* rootScope, const std::vector>& unsolvedConstraints); - StepSnapshot prepareStepSnapshot(const Scope* rootScope, NotNull current, bool force, const std::vector>& unsolvedConstraints); + StepSnapshot prepareStepSnapshot( + const Scope* rootScope, NotNull current, bool force, const std::vector>& unsolvedConstraints); void commitStepSnapshot(StepSnapshot snapshot); void captureFinalSolverState(const Scope* rootScope, const std::vector>& unsolvedConstraints); void captureTypeCheckError(const TypeError& error); + private: ConstraintGenerationLog generationLog; std::unordered_map, std::vector> constraintBlocks; diff --git a/Analysis/include/Luau/Error.h b/Analysis/include/Luau/Error.h index 67754883..7338627c 100644 --- a/Analysis/include/Luau/Error.h +++ b/Analysis/include/Luau/Error.h @@ -33,7 +33,6 @@ struct UnknownSymbol { Binding, Type, - Generic }; Name name; Context context; diff --git a/Analysis/include/Luau/JsonEmitter.h b/Analysis/include/Luau/JsonEmitter.h index d8dc96e4..1a416586 100644 --- a/Analysis/include/Luau/JsonEmitter.h +++ b/Analysis/include/Luau/JsonEmitter.h @@ -240,7 +240,7 @@ void write(JsonEmitter& emitter, const std::unordered_map& map) for (const auto& [k, v] : map) o.writePair(k, v); - + o.finish(); } diff --git a/Analysis/include/Luau/Normalize.h b/Analysis/include/Luau/Normalize.h index 41e50d1b..72ea9558 100644 --- a/Analysis/include/Luau/Normalize.h +++ b/Analysis/include/Luau/Normalize.h @@ -17,8 +17,10 @@ struct SingletonTypes; using ModulePtr = std::shared_ptr; -bool isSubtype(TypeId subTy, TypeId superTy, NotNull scope, NotNull singletonTypes, InternalErrorReporter& ice, bool anyIsTop = true); -bool isSubtype(TypePackId subTy, TypePackId superTy, NotNull scope, NotNull singletonTypes, InternalErrorReporter& ice, bool anyIsTop = true); +bool isSubtype( + TypeId subTy, TypeId superTy, NotNull scope, NotNull singletonTypes, InternalErrorReporter& ice, bool anyIsTop = true); +bool isSubtype(TypePackId subTy, TypePackId superTy, NotNull scope, NotNull singletonTypes, InternalErrorReporter& ice, + bool anyIsTop = true); std::pair normalize( TypeId ty, NotNull scope, TypeArena& arena, NotNull singletonTypes, InternalErrorReporter& ice); @@ -68,13 +70,14 @@ public: insert(*it); } - bool operator ==(const TypeIds& there) const; + bool operator==(const TypeIds& there) const; size_t getHash() const; }; } // namespace Luau -template<> struct std::hash +template<> +struct std::hash { std::size_t operator()(const Luau::TypeIds& tys) const { @@ -82,7 +85,8 @@ template<> struct std::hash } }; -template<> struct std::hash +template<> +struct std::hash { std::size_t operator()(const Luau::TypeIds* tys) const { @@ -90,7 +94,8 @@ template<> struct std::hash } }; -template<> struct std::equal_to +template<> +struct std::equal_to { bool operator()(const Luau::TypeIds& here, const Luau::TypeIds& there) const { @@ -98,7 +103,8 @@ template<> struct std::equal_to } }; -template<> struct std::equal_to +template<> +struct std::equal_to { bool operator()(const Luau::TypeIds* here, const Luau::TypeIds* there) const { @@ -160,7 +166,7 @@ struct NormalizedType // The string part of the type. // This may be the `string` type, or a union of singletons. - NormalizedStringType strings = std::map{}; + NormalizedStringType strings = std::map{}; // The thread part of the type. // This type is either never or thread. diff --git a/Analysis/include/Luau/TypeInfer.h b/Analysis/include/Luau/TypeInfer.h index 3184b0d3..1c4d1cb4 100644 --- a/Analysis/include/Luau/TypeInfer.h +++ b/Analysis/include/Luau/TypeInfer.h @@ -397,7 +397,7 @@ private: std::vector> deferredQuantification; }; -using PrintLineProc = void(*)(const std::string&); +using PrintLineProc = void (*)(const std::string&); extern PrintLineProc luauPrintLine; diff --git a/Analysis/include/Luau/Unifier.h b/Analysis/include/Luau/Unifier.h index f6219dfb..10f3f48c 100644 --- a/Analysis/include/Luau/Unifier.h +++ b/Analysis/include/Luau/Unifier.h @@ -61,15 +61,15 @@ struct Unifier ErrorVec errors; Location location; Variance variance = Covariant; - bool anyIsTop = false; // If true, we consider any to be a top type. If false, it is a familiar but weird mix of top and bottom all at once. - bool normalize; // Normalize unions and intersections if necessary + bool anyIsTop = false; // If true, we consider any to be a top type. If false, it is a familiar but weird mix of top and bottom all at once. + bool normalize; // Normalize unions and intersections if necessary bool useScopes = false; // If true, we use the scope hierarchy rather than TypeLevels CountMismatch::Context ctx = CountMismatch::Arg; UnifierSharedState& sharedState; - Unifier(NotNull normalizer, Mode mode, NotNull scope, const Location& location, Variance variance, - TxnLog* parentLog = nullptr); + Unifier( + NotNull normalizer, Mode mode, NotNull scope, const Location& location, Variance variance, TxnLog* parentLog = nullptr); // Test whether the two type vars unify. Never commits the result. ErrorVec canUnify(TypeId subTy, TypeId superTy); @@ -87,7 +87,8 @@ private: void tryUnifyTypeWithUnion(TypeId subTy, TypeId superTy, const UnionTypeVar* uv, bool cacheEnabled, bool isFunctionCall); void tryUnifyTypeWithIntersection(TypeId subTy, TypeId superTy, const IntersectionTypeVar* uv); void tryUnifyIntersectionWithType(TypeId subTy, const IntersectionTypeVar* uv, TypeId superTy, bool cacheEnabled, bool isFunctionCall); - void tryUnifyNormalizedTypes(TypeId subTy, TypeId superTy, const NormalizedType& subNorm, const NormalizedType& superNorm, std::string reason, std::optional error = std::nullopt); + void tryUnifyNormalizedTypes(TypeId subTy, TypeId superTy, const NormalizedType& subNorm, const NormalizedType& superNorm, std::string reason, + std::optional error = std::nullopt); void tryUnifyPrimitives(TypeId subTy, TypeId superTy); void tryUnifySingletons(TypeId subTy, TypeId superTy); void tryUnifyFunctions(TypeId subTy, TypeId superTy, bool isFunctionCall = false); diff --git a/Analysis/src/BuiltinDefinitions.cpp b/Analysis/src/BuiltinDefinitions.cpp index dbe27bfd..c5250a6d 100644 --- a/Analysis/src/BuiltinDefinitions.cpp +++ b/Analysis/src/BuiltinDefinitions.cpp @@ -497,7 +497,7 @@ static bool dcrMagicFunctionSelect(MagicFunctionCallContext context) asMutable(context.result)->ty.emplace(resTypePack); } else if (tail) - asMutable(context.result)->ty.emplace(*tail); + asMutable(context.result)->ty.emplace(*tail); return true; } @@ -507,7 +507,8 @@ static bool dcrMagicFunctionSelect(MagicFunctionCallContext context) if (AstExprConstantString* str = arg1->as()) { - if (str->value.size == 1 && str->value.data[0] == '#') { + if (str->value.size == 1 && str->value.data[0] == '#') + { TypePackId numberTypePack = context.solver->arena->addTypePack({context.solver->singletonTypes->numberType}); asMutable(context.result)->ty.emplace(numberTypePack); return true; diff --git a/Analysis/src/ConstraintGraphBuilder.cpp b/Analysis/src/ConstraintGraphBuilder.cpp index 169f4645..8436fb30 100644 --- a/Analysis/src/ConstraintGraphBuilder.cpp +++ b/Analysis/src/ConstraintGraphBuilder.cpp @@ -43,7 +43,7 @@ static bool matchSetmetatable(const AstExprCall& call) if (call.args.size != 2) return false; - + const AstExprGlobal* funcAsGlobal = call.func->as(); if (!funcAsGlobal || funcAsGlobal->name != smt) return false; @@ -52,7 +52,8 @@ static bool matchSetmetatable(const AstExprCall& call) } ConstraintGraphBuilder::ConstraintGraphBuilder(const ModuleName& moduleName, ModulePtr module, TypeArena* arena, - NotNull moduleResolver, NotNull singletonTypes, NotNull ice, const ScopePtr& globalScope, DcrLogger* logger) + NotNull moduleResolver, NotNull singletonTypes, NotNull ice, const ScopePtr& globalScope, + DcrLogger* logger) : moduleName(moduleName) , module(module) , singletonTypes(singletonTypes) diff --git a/Analysis/src/ConstraintSolver.cpp b/Analysis/src/ConstraintSolver.cpp index 5b3ec03c..e29eeaaa 100644 --- a/Analysis/src/ConstraintSolver.cpp +++ b/Analysis/src/ConstraintSolver.cpp @@ -14,8 +14,6 @@ #include "Luau/VisitTypeVar.h" #include "Luau/TypeUtils.h" -#include - LUAU_FASTFLAGVARIABLE(DebugLuauLogSolver, false); LUAU_FASTFLAGVARIABLE(DebugLuauLogSolverToJson, false); LUAU_FASTFLAG(LuauFixNameMaps) @@ -283,13 +281,27 @@ ConstraintSolver::ConstraintSolver(NotNull normalizer, NotNull 0; --i) + { + // Fisher-Yates shuffle + size_t j = rng % (i + 1); + + std::swap(unsolvedConstraints[i], unsolvedConstraints[j]); + + // LCG RNG, constants from Numerical Recipes + // This may occasionally result in skewed shuffles due to distribution properties, but this is a debugging tool so it should be good enough + rng = rng * 1664525 + 1013904223; + } } void ConstraintSolver::run() { - if (done()) + if (isDone()) return; if (FFlag::DebugLuauLogSolver) @@ -364,6 +376,8 @@ void ConstraintSolver::run() progress |= runSolverPass(true); } while (progress); + finalizeModule(); + if (FFlag::DebugLuauLogSolver) { dumpBindings(rootScope, opts); @@ -375,11 +389,24 @@ void ConstraintSolver::run() } } -bool ConstraintSolver::done() +bool ConstraintSolver::isDone() { return unsolvedConstraints.empty(); } +void ConstraintSolver::finalizeModule() +{ + Anyification a{arena, rootScope, singletonTypes, &iceReporter, singletonTypes->anyType, singletonTypes->anyTypePack}; + std::optional returnType = a.substitute(rootScope->returnType); + if (!returnType) + { + reportError(CodeTooComplex{}, Location{}); + rootScope->returnType = singletonTypes->errorTypePack; + } + else + rootScope->returnType = *returnType; +} + bool ConstraintSolver::tryDispatch(NotNull constraint, bool force) { if (!force && isBlocked(constraint)) @@ -506,25 +533,25 @@ bool ConstraintSolver::tryDispatch(const UnaryConstraint& c, NotNullty.emplace(singletonTypes->booleanType); + return true; + } + case AstExprUnary::Len: + { + asMutable(c.resultType)->ty.emplace(singletonTypes->numberType); + return true; + } + case AstExprUnary::Minus: + { + if (isNumber(operandType) || get(operandType) || get(operandType)) { - asMutable(c.resultType)->ty.emplace(singletonTypes->booleanType); + asMutable(c.resultType)->ty.emplace(c.operandType); return true; } - case AstExprUnary::Len: - { - asMutable(c.resultType)->ty.emplace(singletonTypes->numberType); - return true; - } - case AstExprUnary::Minus: - { - if (isNumber(operandType) || get(operandType) || get(operandType)) - { - asMutable(c.resultType)->ty.emplace(c.operandType); - return true; - } - break; - } + break; + } } LUAU_ASSERT(false); // TODO metatable handling diff --git a/Analysis/src/DcrLogger.cpp b/Analysis/src/DcrLogger.cpp index a2eb96e5..ef33aa60 100644 --- a/Analysis/src/DcrLogger.cpp +++ b/Analysis/src/DcrLogger.cpp @@ -57,7 +57,7 @@ void write(JsonEmitter& emitter, const ConstraintGenerationLog& log) emitter.writeRaw(":"); ObjectEmitter locationEmitter = emitter.writeObject(); - + for (const auto& [id, location] : log.constraintLocations) { locationEmitter.writePair(id, location); @@ -232,7 +232,7 @@ void DcrLogger::captureSource(std::string source) void DcrLogger::captureGenerationError(const TypeError& error) { std::string stringifiedError = toString(error); - generationLog.errors.push_back(ErrorSnapshot { + generationLog.errors.push_back(ErrorSnapshot{ /* message */ stringifiedError, /* location */ error.location, }); @@ -298,7 +298,8 @@ void DcrLogger::captureInitialSolverState(const Scope* rootScope, const std::vec } } -StepSnapshot DcrLogger::prepareStepSnapshot(const Scope* rootScope, NotNull current, bool force, const std::vector>& unsolvedConstraints) +StepSnapshot DcrLogger::prepareStepSnapshot( + const Scope* rootScope, NotNull current, bool force, const std::vector>& unsolvedConstraints) { ScopeSnapshot scopeSnapshot = snapshotScope(rootScope, opts); std::string currentId = toPointerId(current); @@ -344,7 +345,7 @@ void DcrLogger::captureFinalSolverState(const Scope* rootScope, const std::vecto void DcrLogger::captureTypeCheckError(const TypeError& error) { std::string stringifiedError = toString(error); - checkLog.errors.push_back(ErrorSnapshot { + checkLog.errors.push_back(ErrorSnapshot{ /* message */ stringifiedError, /* location */ error.location, }); @@ -359,7 +360,7 @@ std::vector DcrLogger::snapshotBlocks(NotNull } std::vector snapshot; - + for (const ConstraintBlockTarget& target : it->second) { if (const TypeId* ty = get_if(&target)) diff --git a/Analysis/src/Error.cpp b/Analysis/src/Error.cpp index d13e26c0..4e9b6882 100644 --- a/Analysis/src/Error.cpp +++ b/Analysis/src/Error.cpp @@ -8,7 +8,6 @@ #include LUAU_FASTFLAGVARIABLE(LuauTypeMismatchModuleNameResolution, false) -LUAU_FASTFLAGVARIABLE(LuauUseInternalCompilerErrorException, false) static std::string wrongNumberOfArgsString( size_t expectedCount, std::optional maximumCount, size_t actualCount, const char* argPrefix = nullptr, bool isVariadic = false) @@ -122,8 +121,6 @@ struct ErrorConverter return "Unknown global '" + e.name + "'"; case UnknownSymbol::Type: return "Unknown type '" + e.name + "'"; - case UnknownSymbol::Generic: - return "Unknown generic '" + e.name + "'"; } LUAU_ASSERT(!"Unexpected context for UnknownSymbol"); @@ -902,46 +899,22 @@ void copyErrors(ErrorVec& errors, TypeArena& destArena) void InternalErrorReporter::ice(const std::string& message, const Location& location) { - if (FFlag::LuauUseInternalCompilerErrorException) - { - InternalCompilerError error(message, moduleName, location); + InternalCompilerError error(message, moduleName, location); - if (onInternalError) - onInternalError(error.what()); + if (onInternalError) + onInternalError(error.what()); - throw error; - } - else - { - std::runtime_error error("Internal error in " + moduleName + " at " + toString(location) + ": " + message); - - if (onInternalError) - onInternalError(error.what()); - - throw error; - } + throw error; } void InternalErrorReporter::ice(const std::string& message) { - if (FFlag::LuauUseInternalCompilerErrorException) - { - InternalCompilerError error(message, moduleName); + InternalCompilerError error(message, moduleName); - if (onInternalError) - onInternalError(error.what()); + if (onInternalError) + onInternalError(error.what()); - throw error; - } - else - { - std::runtime_error error("Internal error in " + moduleName + ": " + message); - - if (onInternalError) - onInternalError(error.what()); - - throw error; - } + throw error; } const char* InternalCompilerError::what() const throw() diff --git a/Analysis/src/Linter.cpp b/Analysis/src/Linter.cpp index 7f67a7db..d5578446 100644 --- a/Analysis/src/Linter.cpp +++ b/Analysis/src/Linter.cpp @@ -14,7 +14,6 @@ LUAU_FASTINTVARIABLE(LuauSuggestionDistance, 4) LUAU_FASTFLAGVARIABLE(LuauLintGlobalNeverReadBeforeWritten, false) -LUAU_FASTFLAGVARIABLE(LuauLintFixDeprecationMessage, false) namespace Luau { @@ -307,22 +306,11 @@ private: emitWarning(*context, LintWarning::Code_UnknownGlobal, gv->location, "Unknown global '%s'", gv->name.value); else if (g->deprecated) { - if (FFlag::LuauLintFixDeprecationMessage) - { - if (const char* replacement = *g->deprecated; replacement && strlen(replacement)) - emitWarning(*context, LintWarning::Code_DeprecatedGlobal, gv->location, "Global '%s' is deprecated, use '%s' instead", - gv->name.value, replacement); - else - emitWarning(*context, LintWarning::Code_DeprecatedGlobal, gv->location, "Global '%s' is deprecated", gv->name.value); - } + if (const char* replacement = *g->deprecated; replacement && strlen(replacement)) + emitWarning(*context, LintWarning::Code_DeprecatedGlobal, gv->location, "Global '%s' is deprecated, use '%s' instead", + gv->name.value, replacement); else - { - if (*g->deprecated) - emitWarning(*context, LintWarning::Code_DeprecatedGlobal, gv->location, "Global '%s' is deprecated, use '%s' instead", - gv->name.value, *g->deprecated); - else - emitWarning(*context, LintWarning::Code_DeprecatedGlobal, gv->location, "Global '%s' is deprecated", gv->name.value); - } + emitWarning(*context, LintWarning::Code_DeprecatedGlobal, gv->location, "Global '%s' is deprecated", gv->name.value); } } diff --git a/Analysis/src/Module.cpp b/Analysis/src/Module.cpp index 45eb87d6..31a089a4 100644 --- a/Analysis/src/Module.cpp +++ b/Analysis/src/Module.cpp @@ -15,7 +15,6 @@ #include LUAU_FASTFLAG(LuauAnyifyModuleReturnGenerics) -LUAU_FASTFLAG(LuauLowerBoundsCalculation); LUAU_FASTFLAG(DebugLuauDeferredConstraintResolution); LUAU_FASTFLAGVARIABLE(LuauForceExportSurfacesToBeNormal, false); LUAU_FASTFLAGVARIABLE(LuauClonePublicInterfaceLess, false); @@ -244,19 +243,6 @@ void Module::clonePublicInterface(NotNull singletonTypes, Intern ForceNormal forceNormal{&interfaceTypes}; - if (FFlag::LuauLowerBoundsCalculation) - { - normalize(returnType, NotNull{this}, singletonTypes, ice); - if (FFlag::LuauForceExportSurfacesToBeNormal) - forceNormal.traverse(returnType); - if (varargPack) - { - normalize(*varargPack, NotNull{this}, singletonTypes, ice); - if (FFlag::LuauForceExportSurfacesToBeNormal) - forceNormal.traverse(*varargPack); - } - } - if (exportedTypeBindings) { for (auto& [name, tf] : *exportedTypeBindings) @@ -265,24 +251,6 @@ void Module::clonePublicInterface(NotNull singletonTypes, Intern tf = clonePublicInterface.cloneTypeFun(tf); else tf = clone(tf, interfaceTypes, cloneState); - if (FFlag::LuauLowerBoundsCalculation) - { - normalize(tf.type, NotNull{this}, singletonTypes, ice); - - // We're about to freeze the memory. We know that the flag is conservative by design. Cyclic tables - // won't be marked normal. If the types aren't normal by now, they never will be. - forceNormal.traverse(tf.type); - for (GenericTypeDefinition param : tf.typeParams) - { - forceNormal.traverse(param.ty); - - if (param.defaultValue) - { - normalize(*param.defaultValue, NotNull{this}, singletonTypes, ice); - forceNormal.traverse(*param.defaultValue); - } - } - } } } @@ -305,13 +273,6 @@ void Module::clonePublicInterface(NotNull singletonTypes, Intern ty = clonePublicInterface.cloneType(ty); else ty = clone(ty, interfaceTypes, cloneState); - if (FFlag::LuauLowerBoundsCalculation) - { - normalize(ty, NotNull{this}, singletonTypes, ice); - - if (FFlag::LuauForceExportSurfacesToBeNormal) - forceNormal.traverse(ty); - } } freeze(internalTypes); diff --git a/Analysis/src/Normalize.cpp b/Analysis/src/Normalize.cpp index c008bcfc..81114b76 100644 --- a/Analysis/src/Normalize.cpp +++ b/Analysis/src/Normalize.cpp @@ -119,17 +119,9 @@ NormalizedType::NormalizedType(NotNull singletonTypes) static bool isInhabited(const NormalizedType& norm) { - return !get(norm.tops) - || !get(norm.booleans) - || !norm.classes.empty() - || !get(norm.errors) - || !get(norm.nils) - || !get(norm.numbers) - || !norm.strings || !norm.strings->empty() - || !get(norm.threads) - || norm.functions - || !norm.tables.empty() - || !norm.tyvars.empty(); + return !get(norm.tops) || !get(norm.booleans) || !norm.classes.empty() || !get(norm.errors) || + !get(norm.nils) || !get(norm.numbers) || !norm.strings || !norm.strings->empty() || + !get(norm.threads) || norm.functions || !norm.tables.empty() || !norm.tyvars.empty(); } static int tyvarIndex(TypeId ty) @@ -139,7 +131,7 @@ static int tyvarIndex(TypeId ty) else if (const FreeTypeVar* ftv = get(ty)) return ftv->index; else - return 0; + return 0; } #ifdef LUAU_ASSERTENABLED @@ -193,7 +185,7 @@ static bool isNormalizedString(const NormalizedStringType& ty) { if (!ty) return true; - + for (auto& [str, ty] : *ty) { if (const SingletonTypeVar* stv = get(ty)) @@ -272,24 +264,24 @@ static bool isNormalizedTyvar(const NormalizedTyvars& tyvars) static void assertInvariant(const NormalizedType& norm) { - #ifdef LUAU_ASSERTENABLED - if (!FFlag::DebugLuauCheckNormalizeInvariant) - return; +#ifdef LUAU_ASSERTENABLED + if (!FFlag::DebugLuauCheckNormalizeInvariant) + return; - LUAU_ASSERT(isNormalizedTop(norm.tops)); - LUAU_ASSERT(isNormalizedBoolean(norm.booleans)); - LUAU_ASSERT(areNormalizedClasses(norm.classes)); - LUAU_ASSERT(isNormalizedError(norm.errors)); - LUAU_ASSERT(isNormalizedNil(norm.nils)); - LUAU_ASSERT(isNormalizedNumber(norm.numbers)); - LUAU_ASSERT(isNormalizedString(norm.strings)); - LUAU_ASSERT(isNormalizedThread(norm.threads)); - LUAU_ASSERT(areNormalizedFunctions(norm.functions)); - LUAU_ASSERT(areNormalizedTables(norm.tables)); - LUAU_ASSERT(isNormalizedTyvar(norm.tyvars)); - for (auto& [_, child] : norm.tyvars) - assertInvariant(*child); - #endif + LUAU_ASSERT(isNormalizedTop(norm.tops)); + LUAU_ASSERT(isNormalizedBoolean(norm.booleans)); + LUAU_ASSERT(areNormalizedClasses(norm.classes)); + LUAU_ASSERT(isNormalizedError(norm.errors)); + LUAU_ASSERT(isNormalizedNil(norm.nils)); + LUAU_ASSERT(isNormalizedNumber(norm.numbers)); + LUAU_ASSERT(isNormalizedString(norm.strings)); + LUAU_ASSERT(isNormalizedThread(norm.threads)); + LUAU_ASSERT(areNormalizedFunctions(norm.functions)); + LUAU_ASSERT(areNormalizedTables(norm.tables)); + LUAU_ASSERT(isNormalizedTyvar(norm.tyvars)); + for (auto& [_, child] : norm.tyvars) + assertInvariant(*child); +#endif } Normalizer::Normalizer(TypeArena* arena, NotNull singletonTypes, NotNull sharedState) @@ -359,7 +351,7 @@ TypeId Normalizer::unionType(TypeId here, TypeId there) return there; if (get(there) || get(here)) return here; - + TypeIds tmps; if (const UnionTypeVar* utv = get(here)) @@ -405,7 +397,7 @@ TypeId Normalizer::intersectionType(TypeId here, TypeId there) return here; if (get(there) || get(here)) return there; - + TypeIds tmps; if (const IntersectionTypeVar* utv = get(here)) @@ -516,13 +508,13 @@ std::optional Normalizer::unionOfTypePacks(TypePackId here, TypePack std::vector head; std::optional tail; - + bool hereSubThere = true; bool thereSubHere = true; TypePackIterator ith = begin(here); TypePackIterator itt = begin(there); - + while (ith != end(here) && itt != end(there)) { TypeId hty = *ith; @@ -537,8 +529,8 @@ std::optional Normalizer::unionOfTypePacks(TypePackId here, TypePack itt++; } - auto dealWithDifferentArities = [&](TypePackIterator& ith, TypePackIterator itt, TypePackId here, TypePackId there, bool& hereSubThere, bool& thereSubHere) - { + auto dealWithDifferentArities = [&](TypePackIterator& ith, TypePackIterator itt, TypePackId here, TypePackId there, bool& hereSubThere, + bool& thereSubHere) { if (ith != end(here)) { TypeId tty = singletonTypes->nilType; @@ -591,13 +583,13 @@ std::optional Normalizer::unionOfTypePacks(TypePackId here, TypePack if (ty != tvtp->ty) hereSubThere = false; bool hidden = hvtp->hidden & tvtp->hidden; - tail = arena->addTypePack(VariadicTypePack{ty,hidden}); + tail = arena->addTypePack(VariadicTypePack{ty, hidden}); } - else + else // Luau doesn't have unions of type pack variables return std::nullopt; } - else + else // Luau doesn't have unions of type pack variables return std::nullopt; } @@ -627,7 +619,7 @@ std::optional Normalizer::unionOfTypePacks(TypePackId here, TypePack else if (thereSubHere) return here; if (!head.empty()) - return arena->addTypePack(TypePack{head,tail}); + return arena->addTypePack(TypePack{head, tail}); else if (tail) return *tail; else @@ -639,10 +631,10 @@ std::optional Normalizer::unionOfFunctions(TypeId here, TypeId there) { if (get(here)) return here; - + if (get(there)) return there; - + const FunctionTypeVar* hftv = get(here); LUAU_ASSERT(hftv); const FunctionTypeVar* tftv = get(there); @@ -665,7 +657,7 @@ std::optional Normalizer::unionOfFunctions(TypeId here, TypeId there) return here; if (*argTypes == tftv->argTypes && *retTypes == tftv->retTypes) return there; - + FunctionTypeVar result{*argTypes, *retTypes}; result.generics = hftv->generics; result.genericPacks = hftv->genericPacks; @@ -802,9 +794,9 @@ bool Normalizer::withinResourceLimits() // Check the recursion count if (sharedState->counters.recursionLimit > 0) - if (sharedState->counters.recursionLimit < sharedState->counters.recursionCount) - return false; - + if (sharedState->counters.recursionLimit < sharedState->counters.recursionCount) + return false; + return true; } @@ -1000,13 +992,13 @@ std::optional Normalizer::intersectionOfTypePacks(TypePackId here, T std::vector head; std::optional tail; - + bool hereSubThere = true; bool thereSubHere = true; TypePackIterator ith = begin(here); TypePackIterator itt = begin(there); - + while (ith != end(here) && itt != end(there)) { TypeId hty = *ith; @@ -1021,8 +1013,8 @@ std::optional Normalizer::intersectionOfTypePacks(TypePackId here, T itt++; } - auto dealWithDifferentArities = [&](TypePackIterator& ith, TypePackIterator itt, TypePackId here, TypePackId there, bool& hereSubThere, bool& thereSubHere) - { + auto dealWithDifferentArities = [&](TypePackIterator& ith, TypePackIterator itt, TypePackId here, TypePackId there, bool& hereSubThere, + bool& thereSubHere) { if (ith != end(here)) { TypeId tty = singletonTypes->nilType; @@ -1075,13 +1067,13 @@ std::optional Normalizer::intersectionOfTypePacks(TypePackId here, T if (ty != tvtp->ty) hereSubThere = false; bool hidden = hvtp->hidden & tvtp->hidden; - tail = arena->addTypePack(VariadicTypePack{ty,hidden}); + tail = arena->addTypePack(VariadicTypePack{ty, hidden}); } - else + else // Luau doesn't have unions of type pack variables return std::nullopt; } - else + else // Luau doesn't have unions of type pack variables return std::nullopt; } @@ -1105,7 +1097,7 @@ std::optional Normalizer::intersectionOfTypePacks(TypePackId here, T else if (thereSubHere) return there; if (!head.empty()) - return arena->addTypePack(TypePack{head,tail}); + return arena->addTypePack(TypePack{head, tail}); else if (tail) return *tail; else @@ -1146,7 +1138,7 @@ std::optional Normalizer::intersectionOfTables(TypeId here, TypeId there return std::nullopt; if (httv->state == TableState::Generic || tttv->state == TableState::Generic) return std::nullopt; - + TableState state = httv->state; if (tttv->state == TableState::Unsealed) state = tttv->state; @@ -1226,21 +1218,20 @@ std::optional Normalizer::intersectionOfTables(TypeId here, TypeId there } else return std::nullopt; - } else if (hmtable) { if (table == htable) return here; else - return arena->addType(MetatableTypeVar{table, hmtable}); + return arena->addType(MetatableTypeVar{table, hmtable}); } else if (tmtable) { if (table == ttable) return there; else - return arena->addType(MetatableTypeVar{table, tmtable}); + return arena->addType(MetatableTypeVar{table, tmtable}); } else return table; @@ -1280,7 +1271,7 @@ std::optional Normalizer::intersectionOfFunctions(TypeId here, TypeId th return std::nullopt; if (hftv->retTypes != tftv->retTypes) return std::nullopt; - + std::optional argTypes = unionOfTypePacks(hftv->argTypes, tftv->argTypes); if (!argTypes) return std::nullopt; @@ -1289,7 +1280,7 @@ std::optional Normalizer::intersectionOfFunctions(TypeId here, TypeId th return here; if (*argTypes == tftv->argTypes) return there; - + FunctionTypeVar result{*argTypes, hftv->retTypes}; result.generics = hftv->generics; result.genericPacks = hftv->genericPacks; @@ -1299,7 +1290,7 @@ std::optional Normalizer::intersectionOfFunctions(TypeId here, TypeId th std::optional Normalizer::unionSaturatedFunctions(TypeId here, TypeId there) { // Deep breath... - // + // // When we come to check overloaded functions for subtyping, // we have to compare (F1 & ... & FM) <: (G1 & ... G GN) // where each Fi or Gj is a function type. Now that intersection on the right is no @@ -1319,12 +1310,12 @@ std::optional Normalizer::unionSaturatedFunctions(TypeId here, TypeId th // // So subtyping on overloaded functions "just" boils down to defining Apply. // - // Now for non-overloaded functions, this is easy! + // Now for non-overloaded functions, this is easy! // Apply<(R -> S), T> is S if T <: R, and an error type otherwise. // // But for overloaded functions it's not so simple. We'd like Apply // to just be Apply & ... & Apply but oh dear - // + // // if f : ((number -> number) & (string -> string)) // and x : (number | string) // then f(x) : (number | string) @@ -1334,7 +1325,7 @@ std::optional Normalizer::unionSaturatedFunctions(TypeId here, TypeId th // Apply<((number -> number) & (string -> string)), (number | string)> is (number | string) // // but - // + // // Apply<(number -> number), (number | string)> is an error // Apply<(string -> string), (number | string)> is an error // @@ -1382,7 +1373,7 @@ std::optional Normalizer::unionSaturatedFunctions(TypeId here, TypeId th // Covariance and Contravariance, Giuseppe Castagna, // Logical Methods in Computer Science 16(1), 2022 // https://arxiv.org/abs/1809.01427 - // + // // A gentle introduction to semantic subtyping, Giuseppe Castagna and Alain Frisch, // Proc. Principles and practice of declarative programming 2005, pp 198–208 // https://doi.org/10.1145/1069774.1069793 @@ -1398,7 +1389,7 @@ std::optional Normalizer::unionSaturatedFunctions(TypeId here, TypeId th return std::nullopt; if (hftv->genericPacks != tftv->genericPacks) return std::nullopt; - + std::optional argTypes = unionOfTypePacks(hftv->argTypes, tftv->argTypes); if (!argTypes) return std::nullopt; @@ -1416,7 +1407,7 @@ void Normalizer::intersectFunctionsWithFunction(NormalizedFunctionType& heres, T { if (!heres) return; - + for (auto it = heres->begin(); it != heres->end();) { TypeId here = *it; @@ -1450,7 +1441,7 @@ void Normalizer::intersectFunctions(NormalizedFunctionType& heres, const Normali { heres = std::nullopt; return; - } + } else { for (TypeId there : *theres) @@ -1530,7 +1521,7 @@ bool Normalizer::intersectNormals(NormalizedType& here, const NormalizedType& th if (isInhabited(inter)) it++; else - it = here.tyvars.erase(it); + it = here.tyvars.erase(it); } return true; } @@ -1757,7 +1748,8 @@ bool isSubtype(TypeId subTy, TypeId superTy, NotNull scope, NotNull scope, NotNull singletonTypes, InternalErrorReporter& ice, bool anyIsTop) +bool isSubtype( + TypePackId subPack, TypePackId superPack, NotNull scope, NotNull singletonTypes, InternalErrorReporter& ice, bool anyIsTop) { UnifierSharedState sharedState{&ice}; TypeArena arena; @@ -2377,4 +2369,3 @@ std::pair normalize(TypePackId tp, const ModulePtr& module, No } } // namespace Luau - diff --git a/Analysis/src/Substitution.cpp b/Analysis/src/Substitution.cpp index fa12f306..2137d73e 100644 --- a/Analysis/src/Substitution.cpp +++ b/Analysis/src/Substitution.cpp @@ -9,7 +9,6 @@ #include LUAU_FASTFLAGVARIABLE(LuauSubstitutionFixMissingFields, false) -LUAU_FASTFLAG(LuauLowerBoundsCalculation) LUAU_FASTFLAG(LuauClonePublicInterfaceLess) LUAU_FASTINTVARIABLE(LuauTarjanChildLimit, 10000) LUAU_FASTFLAGVARIABLE(LuauClassTypeVarsInSubstitution, false) @@ -553,9 +552,6 @@ TypePackId Substitution::replace(TypePackId tp) void Substitution::replaceChildren(TypeId ty) { - if (BoundTypeVar* btv = log->getMutable(ty); FFlag::LuauLowerBoundsCalculation && btv) - btv->boundTo = replace(btv->boundTo); - LUAU_ASSERT(ty == log->follow(ty)); if (ignoreChildren(ty)) diff --git a/Analysis/src/ToString.cpp b/Analysis/src/ToString.cpp index a61eb793..9572ef19 100644 --- a/Analysis/src/ToString.cpp +++ b/Analysis/src/ToString.cpp @@ -10,11 +10,11 @@ #include #include -LUAU_FASTFLAG(LuauLowerBoundsCalculation) LUAU_FASTFLAG(LuauUnknownAndNeverType) LUAU_FASTFLAGVARIABLE(LuauSpecialTypesAsterisked, false) LUAU_FASTFLAGVARIABLE(LuauFixNameMaps, false) LUAU_FASTFLAGVARIABLE(LuauUnseeArrayTtv, false) +LUAU_FASTFLAGVARIABLE(LuauFunctionReturnStringificationFixup, false) /* * Prefix generic typenames with gen- @@ -524,7 +524,7 @@ struct TypeVarStringifier bool plural = true; - if (FFlag::LuauLowerBoundsCalculation) + if (FFlag::LuauFunctionReturnStringificationFixup) { auto retBegin = begin(ftv.retTypes); auto retEnd = end(ftv.retTypes); diff --git a/Analysis/src/TypeChecker2.cpp b/Analysis/src/TypeChecker2.cpp index b2f3cfd3..4753a7c2 100644 --- a/Analysis/src/TypeChecker2.cpp +++ b/Analysis/src/TypeChecker2.cpp @@ -24,7 +24,7 @@ namespace Luau // TypeInfer.h // TODO move these -using PrintLineProc = void(*)(const std::string&); +using PrintLineProc = void (*)(const std::string&); extern PrintLineProc luauPrintLine; /* Push a scope onto the end of a stack for the lifetime of the StackPusher instance. @@ -127,7 +127,8 @@ struct TypeChecker2 if (auto ann = ref->parameters.data[0].type) { TypeId argTy = lookupAnnotation(ref->parameters.data[0].type); - luauPrintLine(format("_luau_print (%d, %d): %s\n", annotation->location.begin.line, annotation->location.begin.column, toString(argTy).c_str())); + luauPrintLine(format( + "_luau_print (%d, %d): %s\n", annotation->location.begin.line, annotation->location.begin.column, toString(argTy).c_str())); return follow(argTy); } } @@ -409,8 +410,8 @@ struct TypeChecker2 } TypeId iteratorTy = follow(iteratorTypes[0]); - auto checkFunction = [this, &arena, &scope, &forInStatement, &variableTypes](const FunctionTypeVar* iterFtv, std::vector iterTys, bool isMm) - { + auto checkFunction = [this, &arena, &scope, &forInStatement, &variableTypes]( + const FunctionTypeVar* iterFtv, std::vector iterTys, bool isMm) { if (iterTys.size() < 1 || iterTys.size() > 3) { if (isMm) @@ -420,20 +421,21 @@ struct TypeChecker2 return; } - + // It is okay if there aren't enough iterators, but the iteratee must provide enough. std::vector expectedVariableTypes = flatten(arena, singletonTypes, iterFtv->retTypes, variableTypes.size()); if (expectedVariableTypes.size() < variableTypes.size()) { if (isMm) - reportError(GenericError{"__iter metamethod's next() function does not return enough values"}, getLocation(forInStatement->values)); + reportError( + GenericError{"__iter metamethod's next() function does not return enough values"}, getLocation(forInStatement->values)); else reportError(GenericError{"next() does not return enough values"}, forInStatement->values.data[0]->location); } for (size_t i = 0; i < std::min(expectedVariableTypes.size(), variableTypes.size()); ++i) reportErrors(tryUnify(scope, forInStatement->vars.data[i]->location, variableTypes[i], expectedVariableTypes[i])); - + // nextFn is going to be invoked with (arrayTy, startIndexTy) // It will be passed two arguments on every iteration save the @@ -509,7 +511,8 @@ struct TypeChecker2 { // nothing } - else if (std::optional iterMmTy = findMetatableEntry(singletonTypes, module->errors, iteratorTy, "__iter", forInStatement->values.data[0]->location)) + else if (std::optional iterMmTy = + findMetatableEntry(singletonTypes, module->errors, iteratorTy, "__iter", forInStatement->values.data[0]->location)) { Instantiation instantiation{TxnLog::empty(), &arena, TypeLevel{}, scope}; @@ -554,7 +557,7 @@ struct TypeChecker2 // TODO: This will not tell the user that this is because the // metamethod isn't callable. This is not ideal, and we should // improve this error message. - + // TODO: This will also not handle intersections of functions or // callable tables (which are supported by the runtime). reportError(CannotCallNonFunction{*iterMmTy}, forInStatement->values.data[0]->location); diff --git a/Analysis/src/TypeInfer.cpp b/Analysis/src/TypeInfer.cpp index cb21aa7f..b806edb7 100644 --- a/Analysis/src/TypeInfer.cpp +++ b/Analysis/src/TypeInfer.cpp @@ -33,15 +33,11 @@ LUAU_FASTINTVARIABLE(LuauVisitRecursionLimit, 500) LUAU_FASTFLAG(LuauKnowsTheDataModel3) LUAU_FASTFLAG(LuauAutocompleteDynamicLimits) LUAU_FASTFLAG(LuauTypeNormalization2) -LUAU_FASTFLAGVARIABLE(LuauFunctionArgMismatchDetails, false) -LUAU_FASTFLAGVARIABLE(LuauLowerBoundsCalculation, false) LUAU_FASTFLAGVARIABLE(DebugLuauFreezeDuringUnification, false) LUAU_FASTFLAGVARIABLE(LuauReturnAnyInsteadOfICE, false) // Eventually removed as false. LUAU_FASTFLAGVARIABLE(DebugLuauSharedSelf, false) LUAU_FASTFLAGVARIABLE(LuauAnyifyModuleReturnGenerics, false) LUAU_FASTFLAGVARIABLE(LuauUnknownAndNeverType, false) -LUAU_FASTFLAGVARIABLE(LuauCallUnifyPackTails, false) -LUAU_FASTFLAGVARIABLE(LuauCheckGenericHOFTypes, false) LUAU_FASTFLAGVARIABLE(LuauBinaryNeedsExpectedTypesToo, false) LUAU_FASTFLAGVARIABLE(LuauFixVarargExprHeadType, false) LUAU_FASTFLAGVARIABLE(LuauNeverTypesAndOperatorsInference, false) @@ -136,34 +132,6 @@ bool hasBreak(AstStat* node) } } -static bool hasReturn(const AstStat* node) -{ - struct Searcher : AstVisitor - { - bool result = false; - - bool visit(AstStat*) override - { - return !result; // if we've already found a return statement, don't bother to traverse inward anymore - } - - bool visit(AstStatReturn*) override - { - result = true; - return false; - } - - bool visit(AstExprFunction*) override - { - return false; // We don't care if the function uses a lambda that itself returns - } - }; - - Searcher searcher; - const_cast(node)->visit(&searcher); - return searcher.result; -} - // returns the last statement before the block exits, or nullptr if the block never exits const AstStat* getFallthrough(const AstStat* node) { @@ -550,16 +518,6 @@ void TypeChecker::checkBlockWithoutRecursionCheck(const ScopePtr& scope, const A std::unordered_map> functionDecls; - auto isLocalLambda = [](AstStat* stat) -> AstStatLocal* { - AstStatLocal* local = stat->as(); - - if (FFlag::LuauLowerBoundsCalculation && local && local->vars.size == 1 && local->values.size == 1 && - local->values.data[0]->is()) - return local; - else - return nullptr; - }; - auto checkBody = [&](AstStat* stat) { if (auto fun = stat->as()) { @@ -607,7 +565,7 @@ void TypeChecker::checkBlockWithoutRecursionCheck(const ScopePtr& scope, const A // function f(x:a):a local x: number = g(37) return x end // function g(x:number):number return f(x) end // ``` - if (containsFunctionCallOrReturn(**protoIter) || (FFlag::LuauLowerBoundsCalculation && isLocalLambda(*protoIter))) + if (containsFunctionCallOrReturn(**protoIter)) { while (checkIter != protoIter) { @@ -906,12 +864,6 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatReturn& return_) TypePackId retPack = checkExprList(scope, return_.location, return_.list, false, {}, expectedTypes).type; - if (useConstrainedIntersections()) - { - unifyLowerBound(retPack, scope->returnType, demoter.demotedLevel(scope->level), scope, return_.location); - return; - } - // HACK: Nonstrict mode gets a bit too smart and strict for us when we // start typechecking everything across module boundaries. if (isNonstrictMode() && follow(scope->returnType) == follow(currentModule->getModuleScope()->returnType)) @@ -1574,11 +1526,7 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatTypeAlias& typealias for (auto param : binding->typePackParams) clone.instantiatedTypePackParams.push_back(param.tp); - bool isNormal = ty->normal; ty = addType(std::move(clone)); - - if (FFlag::LuauLowerBoundsCalculation) - asMutable(ty)->normal = isNormal; } } else @@ -1605,14 +1553,6 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatTypeAlias& typealias if (unify(ty, bindingType, aliasScope, typealias.location)) bindingType = ty; - - if (FFlag::LuauLowerBoundsCalculation) - { - auto [t, ok] = normalize(bindingType, currentModule, singletonTypes, *iceHandler); - bindingType = t; - if (!ok) - reportError(typealias.location, NormalizationTooComplex{}); - } } void TypeChecker::prototype(const ScopePtr& scope, const AstStatTypeAlias& typealias, int subLevel) @@ -1959,9 +1899,8 @@ WithPredicate TypeChecker::checkExpr(const ScopePtr& scope, const AstExp } else if (const FreeTypePack* ftp = get(retPack)) { - TypeLevel level = FFlag::LuauLowerBoundsCalculation ? ftp->level : scope->level; - TypeId head = freshType(level); - TypePackId pack = addTypePack(TypePackVar{TypePack{{head}, freshTypePack(level)}}); + TypeId head = freshType(scope->level); + TypePackId pack = addTypePack(TypePackVar{TypePack{{head}, freshTypePack(scope->level)}}); unify(pack, retPack, scope, expr.location); return {head, std::move(result.predicates)}; } @@ -2111,27 +2050,14 @@ std::optional TypeChecker::getIndexTypeFromTypeImpl( return std::nullopt; } - if (FFlag::LuauLowerBoundsCalculation) - { - // FIXME Inefficient. We craft a UnionTypeVar and immediately throw it away. - auto [t, ok] = normalize(addType(UnionTypeVar{std::move(goodOptions)}), currentModule, singletonTypes, *iceHandler); + std::vector result = reduceUnion(goodOptions); + if (FFlag::LuauUnknownAndNeverType && result.empty()) + return neverType; - if (!ok) - reportError(location, NormalizationTooComplex{}); + if (result.size() == 1) + return result[0]; - return t; - } - else - { - std::vector result = reduceUnion(goodOptions); - if (FFlag::LuauUnknownAndNeverType && result.empty()) - return neverType; - - if (result.size() == 1) - return result[0]; - - return addType(UnionTypeVar{std::move(result)}); - } + return addType(UnionTypeVar{std::move(result)}); } else if (const IntersectionTypeVar* itv = get(type)) { @@ -3426,13 +3352,6 @@ std::pair TypeChecker::checkFunctionSignature(const ScopePtr& } } } - - if (!FFlag::LuauCheckGenericHOFTypes) - { - // We do not infer type binders, so if a generic function is required we do not propagate - if (expectedFunctionType && !(expectedFunctionType->generics.empty() && expectedFunctionType->genericPacks.empty())) - expectedFunctionType = nullptr; - } } auto [generics, genericPacks] = createGenericTypes(funScope, std::nullopt, expr, expr.generics, expr.genericPacks); @@ -3442,8 +3361,7 @@ std::pair TypeChecker::checkFunctionSignature(const ScopePtr& retPack = resolveTypePack(funScope, *expr.returnAnnotation); else if (isNonstrictMode()) retPack = anyTypePack; - else if (expectedFunctionType && - (!FFlag::LuauCheckGenericHOFTypes || (expectedFunctionType->generics.empty() && expectedFunctionType->genericPacks.empty()))) + else if (expectedFunctionType && expectedFunctionType->generics.empty() && expectedFunctionType->genericPacks.empty()) { auto [head, tail] = flatten(expectedFunctionType->retTypes); @@ -3488,10 +3406,6 @@ std::pair TypeChecker::checkFunctionSignature(const ScopePtr& funScope->varargPack = anyTypePack; } } - else if (FFlag::LuauLowerBoundsCalculation && !isNonstrictMode()) - { - funScope->varargPack = addTypePack(TypePackVar{VariadicTypePack{anyType, /*hidden*/ true}}); - } std::vector argTypes; @@ -3575,48 +3489,28 @@ std::pair TypeChecker::checkFunctionSignature(const ScopePtr& std::vector genericTys; // if we have a generic expected function type and no generics, we should use the expected ones. - if (FFlag::LuauCheckGenericHOFTypes) + if (expectedFunctionType && generics.empty()) { - if (expectedFunctionType && generics.empty()) - { - genericTys = expectedFunctionType->generics; - } - else - { - genericTys.reserve(generics.size()); - for (const GenericTypeDefinition& generic : generics) - genericTys.push_back(generic.ty); - } + genericTys = expectedFunctionType->generics; } else { genericTys.reserve(generics.size()); - std::transform(generics.begin(), generics.end(), std::back_inserter(genericTys), [](auto&& el) { - return el.ty; - }); + for (const GenericTypeDefinition& generic : generics) + genericTys.push_back(generic.ty); } std::vector genericTps; // if we have a generic expected function type and no generic typepacks, we should use the expected ones. - if (FFlag::LuauCheckGenericHOFTypes) + if (expectedFunctionType && genericPacks.empty()) { - if (expectedFunctionType && genericPacks.empty()) - { - genericTps = expectedFunctionType->genericPacks; - } - else - { - genericTps.reserve(genericPacks.size()); - for (const GenericTypePackDefinition& generic : genericPacks) - genericTps.push_back(generic.tp); - } + genericTps = expectedFunctionType->genericPacks; } else { genericTps.reserve(genericPacks.size()); - std::transform(genericPacks.begin(), genericPacks.end(), std::back_inserter(genericTps), [](auto&& el) { - return el.tp; - }); + for (const GenericTypePackDefinition& generic : genericPacks) + genericTps.push_back(generic.tp); } TypeId funTy = @@ -3674,24 +3568,9 @@ void TypeChecker::checkFunctionBody(const ScopePtr& scope, TypeId ty, const AstE { check(scope, *function.body); - if (useConstrainedIntersections()) - { - TypePackId retPack = follow(funTy->retTypes); - // It is possible for a function to have no annotation and no return statement, and yet still have an ascribed return type - // if it is expected to conform to some other interface. (eg the function may be a lambda passed as a callback) - if (!hasReturn(function.body) && !function.returnAnnotation.has_value() && get(retPack)) - { - auto level = getLevel(retPack); - if (level && scope->level.subsumes(*level)) - *asMutable(retPack) = TypePack{{}, std::nullopt}; - } - } - else - { - // We explicitly don't follow here to check if we have a 'true' free type instead of bound one - if (get_if(&funTy->retTypes->ty)) - *asMutable(funTy->retTypes) = TypePack{{}, std::nullopt}; - } + // We explicitly don't follow here to check if we have a 'true' free type instead of bound one + if (get_if(&funTy->retTypes->ty)) + *asMutable(funTy->retTypes) = TypePack{{}, std::nullopt}; bool reachesImplicitReturn = getFallthrough(function.body) != nullptr; @@ -3763,21 +3642,13 @@ void TypeChecker::checkArgumentList(const ScopePtr& scope, const AstExpr& funNam if (!argLocations.empty()) location = {state.location.begin, argLocations.back().end}; - if (FFlag::LuauFunctionArgMismatchDetails) - { - std::string namePath; - if (std::optional lValue = tryGetLValue(funName)) - namePath = toString(*lValue); + std::string namePath; + if (std::optional lValue = tryGetLValue(funName)) + namePath = toString(*lValue); - auto [minParams, optMaxParams] = getParameterExtents(&state.log, paramPack); - state.reportError(TypeError{location, - CountMismatch{minParams, optMaxParams, std::distance(begin(argPack), end(argPack)), CountMismatch::Context::Arg, false, namePath}}); - } - else - { - size_t minParams = getParameterExtents(&state.log, paramPack).first; - state.reportError(TypeError{location, CountMismatch{minParams, std::nullopt, std::distance(begin(argPack), end(argPack))}}); - } + auto [minParams, optMaxParams] = getParameterExtents(&state.log, paramPack); + state.reportError(TypeError{location, + CountMismatch{minParams, optMaxParams, std::distance(begin(argPack), end(argPack)), CountMismatch::Context::Arg, false, namePath}}); }; while (true) @@ -3801,7 +3672,7 @@ void TypeChecker::checkArgumentList(const ScopePtr& scope, const AstExpr& funNam else state.log.replace(*argTail, TypePackVar(TypePack{{}})); } - else if (FFlag::LuauCallUnifyPackTails && paramTail) + else if (paramTail) { state.tryUnify(*argTail, *paramTail); } @@ -3881,20 +3752,12 @@ void TypeChecker::checkArgumentList(const ScopePtr& scope, const AstExpr& funNam std::optional tail = flatten(paramPack, state.log).second; bool isVariadic = tail && Luau::isVariadic(*tail); - if (FFlag::LuauFunctionArgMismatchDetails) - { - std::string namePath; - if (std::optional lValue = tryGetLValue(funName)) - namePath = toString(*lValue); + std::string namePath; + if (std::optional lValue = tryGetLValue(funName)) + namePath = toString(*lValue); - state.reportError(TypeError{ - state.location, CountMismatch{minParams, optMaxParams, paramIndex, CountMismatch::Context::Arg, isVariadic, namePath}}); - } - else - { - state.reportError( - TypeError{state.location, CountMismatch{minParams, std::nullopt, paramIndex, CountMismatch::Context::Arg, isVariadic}}); - } + state.reportError(TypeError{ + state.location, CountMismatch{minParams, optMaxParams, paramIndex, CountMismatch::Context::Arg, isVariadic, namePath}}); return; } ++paramIter; @@ -3924,21 +3787,6 @@ void TypeChecker::checkArgumentList(const ScopePtr& scope, const AstExpr& funNam } else if (auto vtp = state.log.getMutable(tail)) { - if (FFlag::LuauLowerBoundsCalculation && vtp->hidden) - { - // We know that this function can technically be oversaturated, but we have its definition and we - // know that it's useless. - - TypeId e = errorRecoveryType(scope); - while (argIter != endIter) - { - unify(e, *argIter, scope, state.location); - ++argIter; - } - - reportCountMismatchError(); - return; - } // Function is variadic and requires that all subsequent parameters // be compatible with a type. size_t argIndex = paramIndex; @@ -4040,21 +3888,14 @@ WithPredicate TypeChecker::checkExprPackHelper(const ScopePtr& scope } TypePackId retPack; - if (FFlag::LuauLowerBoundsCalculation) + if (auto free = get(actualFunctionType)) { - retPack = freshTypePack(scope->level); + retPack = freshTypePack(free->level); + TypePackId freshArgPack = freshTypePack(free->level); + asMutable(actualFunctionType)->ty.emplace(free->level, freshArgPack, retPack); } else - { - if (auto free = get(actualFunctionType)) - { - retPack = freshTypePack(free->level); - TypePackId freshArgPack = freshTypePack(free->level); - asMutable(actualFunctionType)->ty.emplace(free->level, freshArgPack, retPack); - } - else - retPack = freshTypePack(scope->level); - } + retPack = freshTypePack(scope->level); // checkExpr will log the pre-instantiated type of the function. // That's not nearly as interesting as the instantiated type, which will include details about how @@ -4214,39 +4055,13 @@ std::optional> TypeChecker::checkCallOverload(const Sc // fn is one of the overloads of actualFunctionType, which // has been instantiated, so is a monotype. We can therefore // unify it with a monomorphic function. - if (useConstrainedIntersections()) - { - // This ternary is phrased deliberately. We need ties between sibling scopes to bias toward ftv->level. - const TypeLevel level = scope->level.subsumes(ftv->level) ? scope->level : ftv->level; + TypeId r = addType(FunctionTypeVar(scope->level, argPack, retPack)); - std::vector adjustedArgTypes; - auto it = begin(argPack); - auto endIt = end(argPack); - Widen widen{¤tModule->internalTypes, singletonTypes}; - for (; it != endIt; ++it) - { - adjustedArgTypes.push_back(addType(ConstrainedTypeVar{level, {widen(*it)}})); - } + UnifierOptions options; + options.isFunctionCall = true; + unify(r, fn, scope, expr.location, options); - TypePackId adjustedArgPack = addTypePack(TypePack{std::move(adjustedArgTypes), it.tail()}); - - TxnLog log; - promoteTypeLevels(log, ¤tModule->internalTypes, level, /*scope*/ nullptr, /*useScope*/ false, retPack); - log.commit(); - - *asMutable(fn) = FunctionTypeVar{level, adjustedArgPack, retPack}; - return {{retPack}}; - } - else - { - TypeId r = addType(FunctionTypeVar(scope->level, argPack, retPack)); - - UnifierOptions options; - options.isFunctionCall = true; - unify(r, fn, scope, expr.location, options); - - return {{retPack}}; - } + return {{retPack}}; } std::vector metaArgLocations; @@ -4760,14 +4575,6 @@ TypeId TypeChecker::quantify(const ScopePtr& scope, TypeId ty, Location location Luau::quantify(ty, scope->level); else if (auto ttv = getTableType(ty); ttv && ttv->selfTy) Luau::quantify(ty, scope->level); - - if (FFlag::LuauLowerBoundsCalculation) - { - auto [t, ok] = Luau::normalize(ty, currentModule, singletonTypes, *iceHandler); - if (!ok) - reportError(location, NormalizationTooComplex{}); - return t; - } } else { @@ -4775,14 +4582,6 @@ TypeId TypeChecker::quantify(const ScopePtr& scope, TypeId ty, Location location if (ftv) Luau::quantify(ty, scope->level); - - if (FFlag::LuauLowerBoundsCalculation && ftv) - { - auto [t, ok] = Luau::normalize(ty, currentModule, singletonTypes, *iceHandler); - if (!ok) - reportError(location, NormalizationTooComplex{}); - return t; - } } return ty; @@ -4813,14 +4612,6 @@ TypeId TypeChecker::instantiate(const ScopePtr& scope, TypeId ty, Location locat TypeId TypeChecker::anyify(const ScopePtr& scope, TypeId ty, Location location) { - if (FFlag::LuauLowerBoundsCalculation) - { - auto [t, ok] = normalize(ty, currentModule, singletonTypes, *iceHandler); - if (!ok) - reportError(location, NormalizationTooComplex{}); - ty = t; - } - Anyification anyification{¤tModule->internalTypes, scope, singletonTypes, iceHandler, anyType, anyTypePack}; std::optional any = anyification.substitute(ty); if (anyification.normalizationTooComplex) @@ -4836,14 +4627,6 @@ TypeId TypeChecker::anyify(const ScopePtr& scope, TypeId ty, Location location) TypePackId TypeChecker::anyify(const ScopePtr& scope, TypePackId ty, Location location) { - if (FFlag::LuauLowerBoundsCalculation) - { - auto [t, ok] = normalize(ty, currentModule, singletonTypes, *iceHandler); - if (!ok) - reportError(location, NormalizationTooComplex{}); - ty = t; - } - Anyification anyification{¤tModule->internalTypes, scope, singletonTypes, iceHandler, anyType, anyTypePack}; std::optional any = anyification.substitute(ty); if (any.has_value()) @@ -6083,11 +5866,6 @@ bool TypeChecker::isNonstrictMode() const return (currentModule->mode == Mode::Nonstrict) || (currentModule->mode == Mode::NoCheck); } -bool TypeChecker::useConstrainedIntersections() const -{ - return FFlag::LuauLowerBoundsCalculation && !isNonstrictMode(); -} - std::vector TypeChecker::unTypePack(const ScopePtr& scope, TypePackId tp, size_t expectedLength, const Location& location) { TypePackId expectedTypePack = addTypePack({}); diff --git a/Analysis/src/TypeUtils.cpp b/Analysis/src/TypeUtils.cpp index ca00c269..688c8767 100644 --- a/Analysis/src/TypeUtils.cpp +++ b/Analysis/src/TypeUtils.cpp @@ -6,8 +6,6 @@ #include "Luau/ToString.h" #include "Luau/TypeInfer.h" -LUAU_FASTFLAG(LuauFunctionArgMismatchDetails) - namespace Luau { @@ -218,7 +216,7 @@ std::pair> getParameterExtents(const TxnLog* log, ++it; } - if (it.tail() && (!FFlag::LuauFunctionArgMismatchDetails || isVariadicTail(*it.tail(), *log, includeHiddenVariadics))) + if (it.tail() && isVariadicTail(*it.tail(), *log, includeHiddenVariadics)) return {minCount, std::nullopt}; else return {minCount, minCount + optionalCount}; diff --git a/Analysis/src/TypeVar.cpp b/Analysis/src/TypeVar.cpp index b143268e..bcdaff7d 100644 --- a/Analysis/src/TypeVar.cpp +++ b/Analysis/src/TypeVar.cpp @@ -25,7 +25,6 @@ LUAU_FASTINTVARIABLE(LuauTableTypeMaximumStringifierLength, 0) LUAU_FASTINT(LuauTypeInferRecursionLimit) LUAU_FASTFLAG(LuauUnknownAndNeverType) LUAU_FASTFLAGVARIABLE(LuauMaybeGenericIntersectionTypes, false) -LUAU_FASTFLAGVARIABLE(LuauStringFormatArgumentErrorFix, false) LUAU_FASTFLAGVARIABLE(LuauNoMoreGlobalSingletonTypes, false) LUAU_FASTFLAG(LuauInstantiateInSubtyping) @@ -1166,21 +1165,12 @@ std::optional> magicFunctionFormat( } // if we know the argument count or if we have too many arguments for sure, we can issue an error - if (FFlag::LuauStringFormatArgumentErrorFix) - { - size_t numActualParams = params.size(); - size_t numExpectedParams = expected.size() + 1; // + 1 for the format string + size_t numActualParams = params.size(); + size_t numExpectedParams = expected.size() + 1; // + 1 for the format string - if (numExpectedParams != numActualParams && (!tail || numExpectedParams < numActualParams)) - typechecker.reportError(TypeError{expr.location, CountMismatch{numExpectedParams, std::nullopt, numActualParams}}); - } - else - { - size_t actualParamSize = params.size() - paramOffset; + if (numExpectedParams != numActualParams && (!tail || numExpectedParams < numActualParams)) + typechecker.reportError(TypeError{expr.location, CountMismatch{numExpectedParams, std::nullopt, numActualParams}}); - if (expected.size() != actualParamSize && (!tail || expected.size() < actualParamSize)) - typechecker.reportError(TypeError{expr.location, CountMismatch{expected.size(), std::nullopt, actualParamSize}}); - } return WithPredicate{arena.addTypePack({typechecker.stringType})}; } diff --git a/Analysis/src/Unifier.cpp b/Analysis/src/Unifier.cpp index 5a01c934..42fcd2fd 100644 --- a/Analysis/src/Unifier.cpp +++ b/Analysis/src/Unifier.cpp @@ -18,14 +18,12 @@ LUAU_FASTINT(LuauTypeInferTypePackLoopLimit); LUAU_FASTINT(LuauTypeInferIterationLimit); LUAU_FASTFLAG(LuauAutocompleteDynamicLimits) LUAU_FASTINTVARIABLE(LuauTypeInferLowerBoundsIterationLimit, 2000); -LUAU_FASTFLAG(LuauLowerBoundsCalculation); LUAU_FASTFLAG(LuauErrorRecoveryType); LUAU_FASTFLAG(LuauUnknownAndNeverType) LUAU_FASTFLAGVARIABLE(LuauSubtypeNormalizer, false); LUAU_FASTFLAGVARIABLE(LuauScalarShapeSubtyping, false) LUAU_FASTFLAGVARIABLE(LuauInstantiateInSubtyping, false) LUAU_FASTFLAG(LuauClassTypeVarsInSubstitution) -LUAU_FASTFLAG(LuauCallUnifyPackTails) LUAU_FASTFLAG(DebugLuauDeferredConstraintResolution) namespace Luau @@ -346,8 +344,7 @@ static bool subsumes(bool useScopes, TY_A* left, TY_B* right) return left->level.subsumes(right->level); } -Unifier::Unifier(NotNull normalizer, Mode mode, NotNull scope, const Location& location, - Variance variance, TxnLog* parentLog) +Unifier::Unifier(NotNull normalizer, Mode mode, NotNull scope, const Location& location, Variance variance, TxnLog* parentLog) : types(normalizer->arena) , singletonTypes(normalizer->singletonTypes) , normalizer(normalizer) @@ -529,7 +526,7 @@ void Unifier::tryUnify_(TypeId subTy, TypeId superTy, bool isFunctionCall, bool { tryUnifyUnionWithType(subTy, subUnion, superTy); } - else if (const UnionTypeVar* uv = (FFlag::LuauSubtypeNormalizer? nullptr: log.getMutable(superTy))) + else if (const UnionTypeVar* uv = (FFlag::LuauSubtypeNormalizer ? nullptr : log.getMutable(superTy))) { tryUnifyTypeWithUnion(subTy, superTy, uv, cacheEnabled, isFunctionCall); } @@ -865,7 +862,8 @@ void Unifier::tryUnifyIntersectionWithType(TypeId subTy, const IntersectionTypeV } } -void Unifier::tryUnifyNormalizedTypes(TypeId subTy, TypeId superTy, const NormalizedType& subNorm, const NormalizedType& superNorm, std::string reason, std::optional error) +void Unifier::tryUnifyNormalizedTypes( + TypeId subTy, TypeId superTy, const NormalizedType& subNorm, const NormalizedType& superNorm, std::string reason, std::optional error) { LUAU_ASSERT(FFlag::LuauSubtypeNormalizer); @@ -1371,12 +1369,12 @@ void Unifier::tryUnify_(TypePackId subTp, TypePackId superTp, bool isFunctionCal else { // A union type including nil marks an optional argument - if ((!FFlag::LuauLowerBoundsCalculation || isNonstrictMode()) && superIter.good() && isOptional(*superIter)) + if (superIter.good() && isOptional(*superIter)) { superIter.advance(); continue; } - else if ((!FFlag::LuauLowerBoundsCalculation || isNonstrictMode()) && subIter.good() && isOptional(*subIter)) + else if (subIter.good() && isOptional(*subIter)) { subIter.advance(); continue; @@ -1394,7 +1392,7 @@ void Unifier::tryUnify_(TypePackId subTp, TypePackId superTp, bool isFunctionCal return; } - if ((!FFlag::LuauLowerBoundsCalculation || isNonstrictMode()) && !isFunctionCall && subIter.good()) + if (!isFunctionCall && subIter.good()) { // Sometimes it is ok to pass too many arguments return; @@ -1491,7 +1489,6 @@ void Unifier::tryUnifyFunctions(TypeId subTy, TypeId superTy, bool isFunctionCal numGenerics = std::min(superFunction->generics.size(), subFunction->generics.size()); numGenericPacks = std::min(superFunction->genericPacks.size(), subFunction->genericPacks.size()); - } else { @@ -2012,7 +2009,8 @@ void Unifier::tryUnifyWithMetatable(TypeId subTy, TypeId superTy, bool reversed) if (auto e = hasUnificationTooComplex(innerState.errors)) reportError(*e); else if (!innerState.errors.empty()) - reportError(TypeError{location, TypeMismatch{reversed ? subTy : superTy, reversed ? superTy : subTy, "", innerState.errors.front()}}); + reportError( + TypeError{location, TypeMismatch{reversed ? subTy : superTy, reversed ? superTy : subTy, "", innerState.errors.front()}}); else if (!missingProperty) { log.concat(std::move(innerState.log)); @@ -2448,8 +2446,7 @@ void Unifier::unifyLowerBound(TypePackId subTy, TypePackId superTy, TypeLevel de for (; superIter != superEndIter; ++superIter) tp->head.push_back(*superIter); } - else if (const VariadicTypePack* subVariadic = log.getMutable(subTailPack); - subVariadic && FFlag::LuauCallUnifyPackTails) + else if (const VariadicTypePack* subVariadic = log.getMutable(subTailPack)) { while (superIter != superEndIter) { diff --git a/Ast/include/Luau/StringUtils.h b/Ast/include/Luau/StringUtils.h index dab76106..6345fde4 100644 --- a/Ast/include/Luau/StringUtils.h +++ b/Ast/include/Luau/StringUtils.h @@ -1,17 +1,13 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #pragma once +#include "Luau/Common.h" + #include #include #include -#if defined(__GNUC__) -#define LUAU_PRINTF_ATTR(fmt, arg) __attribute__((format(printf, fmt, arg))) -#else -#define LUAU_PRINTF_ATTR(fmt, arg) -#endif - namespace Luau { diff --git a/CLI/Repl.cpp b/CLI/Repl.cpp index 4d3beec9..aecddf38 100644 --- a/CLI/Repl.cpp +++ b/CLI/Repl.cpp @@ -4,6 +4,7 @@ #include "lua.h" #include "lualib.h" +#include "Luau/CodeGen.h" #include "Luau/Compiler.h" #include "Luau/BytecodeBuilder.h" #include "Luau/Parser.h" @@ -46,11 +47,15 @@ enum class CompileFormat { Text, Binary, + Remarks, + Codegen, Null }; constexpr int MaxTraversalLimit = 50; +static bool codegen = false; + // Ctrl-C handling static void sigintCallback(lua_State* L, int gc) { @@ -159,6 +164,9 @@ static int lua_require(lua_State* L) std::string bytecode = Luau::compile(*source, copts()); if (luau_load(ML, chunkname.c_str(), bytecode.data(), bytecode.size(), 0) == 0) { + if (codegen) + Luau::CodeGen::compile(ML, -1); + if (coverageActive()) coverageTrack(ML, -1); @@ -242,6 +250,9 @@ static int lua_callgrind(lua_State* L) void setupState(lua_State* L) { + if (codegen) + Luau::CodeGen::create(L); + luaL_openlibs(L); static const luaL_Reg funcs[] = { @@ -276,6 +287,9 @@ std::string runCode(lua_State* L, const std::string& source) return error; } + if (codegen) + Luau::CodeGen::compile(L, -1); + lua_State* T = lua_newthread(L); lua_pushvalue(L, -2); @@ -604,6 +618,9 @@ static bool runFile(const char* name, lua_State* GL, bool repl) if (luau_load(L, chunkname.c_str(), bytecode.data(), bytecode.size(), 0) == 0) { + if (codegen) + Luau::CodeGen::compile(L, -1); + if (coverageActive()) coverageTrack(L, -1); @@ -656,6 +673,20 @@ static void reportError(const char* name, const Luau::CompileError& error) report(name, error.getLocation(), "CompileError", error.what()); } +static std::string getCodegenAssembly(const char* name, const std::string& bytecode) +{ + std::unique_ptr globalState(luaL_newstate(), lua_close); + lua_State* L = globalState.get(); + + setupState(L); + + if (luau_load(L, name, bytecode.data(), bytecode.size(), 0) == 0) + return Luau::CodeGen::getAssemblyText(L, -1); + + fprintf(stderr, "Error loading bytecode %s\n", name); + return ""; +} + static bool compileFile(const char* name, CompileFormat format) { std::optional source = readFile(name); @@ -675,6 +706,11 @@ static bool compileFile(const char* name, CompileFormat format) Luau::BytecodeBuilder::Dump_Remarks); bcb.setDumpSource(*source); } + else if (format == CompileFormat::Remarks) + { + bcb.setDumpFlags(Luau::BytecodeBuilder::Dump_Source | Luau::BytecodeBuilder::Dump_Remarks); + bcb.setDumpSource(*source); + } Luau::compileOrThrow(bcb, *source, copts()); @@ -683,9 +719,15 @@ static bool compileFile(const char* name, CompileFormat format) case CompileFormat::Text: printf("%s", bcb.dumpEverything().c_str()); break; + case CompileFormat::Remarks: + printf("%s", bcb.dumpSourceRemarks().c_str()); + break; case CompileFormat::Binary: fwrite(bcb.getBytecode().data(), 1, bcb.getBytecode().size(), stdout); break; + case CompileFormat::Codegen: + printf("%s", getCodegenAssembly(name, bcb.getBytecode()).c_str()); + break; case CompileFormat::Null: break; } @@ -713,7 +755,7 @@ static void displayHelp(const char* argv0) printf("\n"); printf("Available modes:\n"); printf(" omitted: compile and run input files one by one\n"); - printf(" --compile[=format]: compile input files and output resulting formatted bytecode (binary or text)\n"); + printf(" --compile[=format]: compile input files and output resulting formatted bytecode (binary, text, remarks, codegen or null)\n"); printf("\n"); printf("Available options:\n"); printf(" --coverage: collect code coverage while running the code and output results to coverage.out\n"); @@ -723,6 +765,7 @@ static void displayHelp(const char* argv0) printf(" -g: compile with debug level n (default 1, n should be between 0 and 2).\n"); printf(" --profile[=N]: profile the code using N Hz sampling (default 10000) and output results to profile.out\n"); printf(" --timetrace: record compiler time tracing information into trace.json\n"); + printf(" --codegen: execute code using native code generation\n"); } static int assertionHandler(const char* expr, const char* file, int line, const char* function) @@ -761,6 +804,14 @@ int replMain(int argc, char** argv) { compileFormat = CompileFormat::Text; } + else if (strcmp(argv[1], "--compile=remarks") == 0) + { + compileFormat = CompileFormat::Remarks; + } + else if (strcmp(argv[1], "--compile=codegen") == 0) + { + compileFormat = CompileFormat::Codegen; + } else if (strcmp(argv[1], "--compile=null") == 0) { compileFormat = CompileFormat::Null; @@ -811,6 +862,10 @@ int replMain(int argc, char** argv) { profile = atoi(argv[i] + 10); } + else if (strcmp(argv[i], "--codegen") == 0) + { + codegen = true; + } else if (strcmp(argv[i], "--coverage") == 0) { coverage = true; @@ -839,12 +894,26 @@ int replMain(int argc, char** argv) } #endif +#if !LUA_CUSTOM_EXECUTION + if (codegen) + { + fprintf(stderr, "To run with --codegen, Luau has to be built with LUA_CUSTOM_EXECUTION enabled\n"); + return 1; + } +#endif + const std::vector files = getSourceFiles(argc, argv); if (mode == CliMode::Unknown) { mode = files.empty() ? CliMode::Repl : CliMode::RunSourceFiles; } + if (mode != CliMode::Compile && codegen && !Luau::CodeGen::isSupported()) + { + fprintf(stderr, "Cannot enable --codegen, native code generation is not supported in current configuration\n"); + return 1; + } + switch (mode) { case CliMode::Compile: diff --git a/CMakeLists.txt b/CMakeLists.txt index 43289f41..0016160a 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -12,6 +12,7 @@ option(LUAU_BUILD_WEB "Build Web module" OFF) option(LUAU_WERROR "Warnings as errors" OFF) option(LUAU_STATIC_CRT "Link with the static CRT (/MT)" OFF) option(LUAU_EXTERN_C "Use extern C for all APIs" OFF) +option(LUAU_NATIVE "Enable support for native code generation" OFF) if(LUAU_STATIC_CRT) cmake_minimum_required(VERSION 3.15) @@ -132,6 +133,10 @@ if(LUAU_EXTERN_C) target_compile_definitions(Luau.Compiler PUBLIC LUACODE_API=extern\"C\") endif() +if(LUAU_NATIVE) + target_compile_definitions(Luau.VM PUBLIC LUA_CUSTOM_EXECUTION=1) +endif() + if (MSVC AND MSVC_VERSION GREATER_EQUAL 1924) # disable partial redundancy elimination which regresses interpreter codegen substantially in VS2022: # https://developercommunity.visualstudio.com/t/performance-regression-on-a-complex-interpreter-lo/1631863 @@ -167,7 +172,7 @@ if(LUAU_BUILD_CLI) target_include_directories(Luau.Repl.CLI PRIVATE extern extern/isocline/include) - target_link_libraries(Luau.Repl.CLI PRIVATE Luau.Compiler Luau.VM isocline) + target_link_libraries(Luau.Repl.CLI PRIVATE Luau.Compiler Luau.CodeGen Luau.VM isocline) if(UNIX) find_library(LIBPTHREAD pthread) @@ -193,11 +198,11 @@ if(LUAU_BUILD_TESTS) target_compile_options(Luau.Conformance PRIVATE ${LUAU_OPTIONS}) target_include_directories(Luau.Conformance PRIVATE extern) - target_link_libraries(Luau.Conformance PRIVATE Luau.Analysis Luau.Compiler Luau.VM) + target_link_libraries(Luau.Conformance PRIVATE Luau.Analysis Luau.Compiler Luau.CodeGen Luau.VM) target_compile_options(Luau.CLI.Test PRIVATE ${LUAU_OPTIONS}) target_include_directories(Luau.CLI.Test PRIVATE extern CLI) - target_link_libraries(Luau.CLI.Test PRIVATE Luau.Compiler Luau.VM isocline) + target_link_libraries(Luau.CLI.Test PRIVATE Luau.Compiler Luau.CodeGen Luau.VM isocline) if(UNIX) find_library(LIBPTHREAD pthread) if (LIBPTHREAD) diff --git a/CodeGen/include/Luau/AssemblyBuilderX64.h b/CodeGen/include/Luau/AssemblyBuilderX64.h index 15db7a15..1c755017 100644 --- a/CodeGen/include/Luau/AssemblyBuilderX64.h +++ b/CodeGen/include/Luau/AssemblyBuilderX64.h @@ -15,6 +15,14 @@ namespace Luau namespace CodeGen { +enum class RoundingModeX64 +{ + RoundToNearestEven = 0b00, + RoundToNegativeInfinity = 0b01, + RoundToPositiveInfinity = 0b10, + RoundToZero = 0b11, +}; + class AssemblyBuilderX64 { public: @@ -48,6 +56,8 @@ public: void imul(OperandX64 op); void neg(OperandX64 op); void not_(OperandX64 op); + void dec(OperandX64 op); + void inc(OperandX64 op); // Additional forms of imul void imul(OperandX64 lhs, OperandX64 rhs); @@ -82,13 +92,12 @@ public: void vxorpd(OperandX64 dst, OperandX64 src1, OperandX64 src2); - void vcomisd(OperandX64 src1, OperandX64 src2); void vucomisd(OperandX64 src1, OperandX64 src2); void vcvttsd2si(OperandX64 dst, OperandX64 src); void vcvtsi2sd(OperandX64 dst, OperandX64 src1, OperandX64 src2); - void vroundsd(OperandX64 dst, OperandX64 src1, OperandX64 src2, uint8_t mode); + void vroundsd(OperandX64 dst, OperandX64 src1, OperandX64 src2, RoundingModeX64 roundingMode); // inexact void vsqrtpd(OperandX64 dst, OperandX64 src); void vsqrtps(OperandX64 dst, OperandX64 src); @@ -120,6 +129,8 @@ public: OperandX64 f32x4(float x, float y, float z, float w); OperandX64 bytes(const void* ptr, size_t size, size_t align = 8); + void logAppend(const char* fmt, ...) LUAU_PRINTF_ATTR(2, 3); + // Resulting data and code that need to be copied over one after the other // The *end* of 'data' has to be aligned to 16 bytes, this will also align 'code' std::vector data; @@ -127,6 +138,8 @@ public: std::string text; + const bool logText = false; + private: // Instruction archetypes void placeBinary(const char* name, OperandX64 lhs, OperandX64 rhs, uint8_t codeimm8, uint8_t codeimm, uint8_t codeimmImm8, uint8_t code8rev, @@ -178,7 +191,6 @@ private: LUAU_NOINLINE void log(Label label); LUAU_NOINLINE void log(const char* opcode, Label label); void log(OperandX64 op); - void logAppend(const char* fmt, ...); const char* getSizeName(SizeX64 size); const char* getRegisterName(RegisterX64 reg); @@ -187,7 +199,6 @@ private: std::vector(() -> a, a) -> ()", toString(requireType("f"))); -} - -TEST_CASE_FIXTURE(Fixture, "fuzz_failure_instersection_combine_must_follow") -{ - ScopedFastFlag flags[] = { - {"LuauLowerBoundsCalculation", true}, - }; - - CheckResult result = check(R"( - export type t0 = {_:{_:any} & {_:any|string}} & {_:{_:{}}} - )"); - - LUAU_REQUIRE_NO_ERRORS(result); -} - -TEST_CASE_FIXTURE(Fixture, "fuzz_failure_bound_type_is_normal_but_not_its_bounded_to") -{ - ScopedFastFlag sff{"LuauLowerBoundsCalculation", true}; - - CheckResult result = check(R"( - type t252 = ((t0)|(any))|(any) - type t0 = t252,t24...> - )"); - - LUAU_REQUIRE_ERRORS(result); -} - -// We had an issue where a normal BoundTypeVar might point at a non-normal BoundTypeVar if it in turn pointed to a -// normal TypeVar because we were calling follow() in an improper place. -TEST_CASE_FIXTURE(Fixture, "bound_typevars_should_only_be_marked_normal_if_their_pointee_is_normal") -{ - ScopedFastFlag sff[]{ - {"LuauLowerBoundsCalculation", true}, - }; - - CheckResult result = check(R"( - local T = {} - - function T:M() - local function f(a) - print(self.prop) - self:g(a) - self.prop = a - end - end - - return T - )"); -} - TEST_CASE_FIXTURE(BuiltinsFixture, "skip_force_normal_on_external_types") { createSomeClasses(frontend); @@ -1108,68 +472,4 @@ export type t0 = (((any)&({_:l0.t0,n0:t0,_G:any,}))&({_:any,}))&(((any)&({_:l0.t LUAU_REQUIRE_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "normalize_unions_containing_never") -{ - ScopedFastFlag sff{"LuauLowerBoundsCalculation", true}; - - CheckResult result = check(R"( - type Foo = string | never - local foo: Foo - )"); - - CHECK_EQ("string", toString(requireType("foo"))); -} - -TEST_CASE_FIXTURE(Fixture, "normalize_unions_containing_unknown") -{ - ScopedFastFlag sff{"LuauLowerBoundsCalculation", true}; - - CheckResult result = check(R"( - type Foo = string | unknown - local foo: Foo - )"); - - CHECK_EQ("unknown", toString(requireType("foo"))); -} - -TEST_CASE_FIXTURE(Fixture, "any_wins_the_battle_over_unknown_in_unions") -{ - ScopedFastFlag sff{"LuauLowerBoundsCalculation", true}; - - CheckResult result = check(R"( - type Foo = unknown | any - local foo: Foo - - type Bar = any | unknown - local bar: Bar - )"); - - CHECK_EQ("any", toString(requireType("foo"))); - CHECK_EQ("any", toString(requireType("bar"))); -} - -TEST_CASE_FIXTURE(BuiltinsFixture, "normalization_does_not_convert_ever") -{ - ScopedFastFlag sff[]{ - {"LuauLowerBoundsCalculation", true}, - }; - - CheckResult result = check(R"( - --!strict - local function f() - if math.random() > 0.5 then - return true - end - type Ret = typeof(f()) - if math.random() > 0.5 then - return "something" - end - return "something" :: Ret - end - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - CHECK_EQ("() -> boolean | string", toString(requireType("f"))); -} - TEST_SUITE_END(); diff --git a/tests/RuntimeLimits.test.cpp b/tests/RuntimeLimits.test.cpp index 6619147b..7e50d5b6 100644 --- a/tests/RuntimeLimits.test.cpp +++ b/tests/RuntimeLimits.test.cpp @@ -15,8 +15,6 @@ using namespace Luau; -LUAU_FASTFLAG(LuauLowerBoundsCalculation); - struct LimitFixture : BuiltinsFixture { #if defined(_NOOPT) || defined(_DEBUG) @@ -267,10 +265,7 @@ TEST_CASE_FIXTURE(LimitFixture, "typescript_port_of_Result_type") CheckResult result = check(src); CodeTooComplex ctc; - if (FFlag::LuauLowerBoundsCalculation) - LUAU_REQUIRE_ERRORS(result); - else - CHECK(hasError(result, &ctc)); + CHECK(hasError(result, &ctc)); } TEST_SUITE_END(); diff --git a/tests/ToDot.test.cpp b/tests/ToDot.test.cpp index 95dcd70a..98eb9863 100644 --- a/tests/ToDot.test.cpp +++ b/tests/ToDot.test.cpp @@ -7,8 +7,6 @@ #include "doctest.h" -LUAU_FASTFLAG(LuauLowerBoundsCalculation) - using namespace Luau; struct ToDotClassFixture : Fixture @@ -111,29 +109,7 @@ local function f(a, ...: string) return a end ToDotOptions opts; opts.showPointers = false; - if (FFlag::LuauLowerBoundsCalculation) - { - CHECK_EQ(R"(digraph graphname { -n1 [label="FunctionTypeVar 1"]; -n1 -> n2 [label="arg"]; -n2 [label="TypePack 2"]; -n2 -> n3; -n3 [label="GenericTypeVar 3"]; -n2 -> n4 [label="tail"]; -n4 [label="VariadicTypePack 4"]; -n4 -> n5; -n5 [label="string"]; -n1 -> n6 [label="ret"]; -n6 [label="TypePack 6"]; -n6 -> n7; -n7 [label="BoundTypeVar 7"]; -n7 -> n3; -})", - toDot(requireType("f"), opts)); - } - else - { - CHECK_EQ(R"(digraph graphname { + CHECK_EQ(R"(digraph graphname { n1 [label="FunctionTypeVar 1"]; n1 -> n2 [label="arg"]; n2 [label="TypePack 2"]; @@ -149,8 +125,7 @@ n6 -> n7; n7 [label="TypePack 7"]; n7 -> n3; })", - toDot(requireType("f"), opts)); - } + toDot(requireType("f"), opts)); } TEST_CASE_FIXTURE(Fixture, "union") diff --git a/tests/ToString.test.cpp b/tests/ToString.test.cpp index 1339ec28..53e5f71b 100644 --- a/tests/ToString.test.cpp +++ b/tests/ToString.test.cpp @@ -12,6 +12,7 @@ using namespace Luau; LUAU_FASTFLAG(LuauRecursiveTypeParameterRestriction); LUAU_FASTFLAG(LuauSpecialTypesAsterisked); LUAU_FASTFLAG(LuauFixNameMaps); +LUAU_FASTFLAG(LuauFunctionReturnStringificationFixup); TEST_SUITE_BEGIN("ToString"); @@ -570,6 +571,22 @@ TEST_CASE_FIXTURE(Fixture, "toString_the_boundTo_table_type_contained_within_a_T CHECK_EQ("{| hello: number, world: number |}", toString(&tpv2)); } +TEST_CASE_FIXTURE(Fixture, "no_parentheses_around_return_type_if_pack_has_an_empty_head_link") +{ + TypeArena arena; + TypePackId realTail = arena.addTypePack({singletonTypes->stringType}); + TypePackId emptyTail = arena.addTypePack({}, realTail); + + TypePackId argList = arena.addTypePack({singletonTypes->stringType}); + + TypeId functionType = arena.addType(FunctionTypeVar{argList, emptyTail}); + + if (FFlag::LuauFunctionReturnStringificationFixup) + CHECK("(string) -> string" == toString(functionType)); + else + CHECK("(string) -> (string)" == toString(functionType)); +} + TEST_CASE_FIXTURE(Fixture, "no_parentheses_around_cyclic_function_type_in_union") { CheckResult result = check(R"( diff --git a/tests/TypeInfer.annotations.test.cpp b/tests/TypeInfer.annotations.test.cpp index 5f2c22cf..28767889 100644 --- a/tests/TypeInfer.annotations.test.cpp +++ b/tests/TypeInfer.annotations.test.cpp @@ -657,50 +657,9 @@ struct AssertionCatcher int AssertionCatcher::tripped; } // namespace -TEST_CASE_FIXTURE(Fixture, "luau_ice_triggers_an_ice") -{ - ScopedFastFlag sffs[] = { - {"DebugLuauMagicTypes", true}, - {"LuauUseInternalCompilerErrorException", false}, - }; - - AssertionCatcher ac; - - CHECK_THROWS_AS(check(R"( - local a: _luau_ice = 55 - )"), - std::runtime_error); - - LUAU_ASSERT(1 == AssertionCatcher::tripped); -} - -TEST_CASE_FIXTURE(Fixture, "luau_ice_triggers_an_ice_handler") -{ - ScopedFastFlag sffs[] = { - {"DebugLuauMagicTypes", true}, - {"LuauUseInternalCompilerErrorException", false}, - }; - - bool caught = false; - - frontend.iceHandler.onInternalError = [&](const char*) { - caught = true; - }; - - CHECK_THROWS_AS(check(R"( - local a: _luau_ice = 55 - )"), - std::runtime_error); - - CHECK_EQ(true, caught); -} - TEST_CASE_FIXTURE(Fixture, "luau_ice_triggers_an_ice_exception_with_flag") { - ScopedFastFlag sffs[] = { - {"DebugLuauMagicTypes", true}, - {"LuauUseInternalCompilerErrorException", true}, - }; + ScopedFastFlag sffs{"DebugLuauMagicTypes", true}; AssertionCatcher ac; @@ -714,10 +673,7 @@ TEST_CASE_FIXTURE(Fixture, "luau_ice_triggers_an_ice_exception_with_flag") TEST_CASE_FIXTURE(Fixture, "luau_ice_triggers_an_ice_exception_with_flag_handler") { - ScopedFastFlag sffs[] = { - {"DebugLuauMagicTypes", true}, - {"LuauUseInternalCompilerErrorException", true}, - }; + ScopedFastFlag sffs{"DebugLuauMagicTypes", true}; bool caught = false; diff --git a/tests/TypeInfer.builtins.test.cpp b/tests/TypeInfer.builtins.test.cpp index 037f79d8..f9c104fd 100644 --- a/tests/TypeInfer.builtins.test.cpp +++ b/tests/TypeInfer.builtins.test.cpp @@ -8,9 +8,7 @@ using namespace Luau; -LUAU_FASTFLAG(LuauLowerBoundsCalculation); LUAU_FASTFLAG(LuauSpecialTypesAsterisked); -LUAU_FASTFLAG(LuauStringFormatArgumentErrorFix) LUAU_FASTFLAG(DebugLuauDeferredConstraintResolution) TEST_SUITE_BEGIN("BuiltinTests"); @@ -637,8 +635,6 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "select_with_decimal_argument_is_rounded_down // Could be flaky if the fix has regressed. TEST_CASE_FIXTURE(BuiltinsFixture, "bad_select_should_not_crash") { - ScopedFastFlag luauFunctionArgMismatchDetails{"LuauFunctionArgMismatchDetails", true}; - CheckResult result = check(R"( do end local _ = function(l0,...) @@ -754,14 +750,7 @@ TEST_CASE_FIXTURE(Fixture, "string_format_use_correct_argument") LUAU_REQUIRE_ERROR_COUNT(1, result); - if (FFlag::LuauStringFormatArgumentErrorFix) - { - CHECK_EQ("Argument count mismatch. Function expects 2 arguments, but 3 are specified", toString(result.errors[0])); - } - else - { - CHECK_EQ("Argument count mismatch. Function expects 1 argument, but 2 are specified", toString(result.errors[0])); - } + CHECK_EQ("Argument count mismatch. Function expects 2 arguments, but 3 are specified", toString(result.errors[0])); } TEST_CASE_FIXTURE(Fixture, "string_format_use_correct_argument2") @@ -778,8 +767,6 @@ TEST_CASE_FIXTURE(Fixture, "string_format_use_correct_argument2") TEST_CASE_FIXTURE(BuiltinsFixture, "string_format_use_correct_argument3") { - ScopedFastFlag LuauStringFormatArgumentErrorFix{"LuauStringFormatArgumentErrorFix", true}; - CheckResult result = check(R"( local s1 = string.format("%d") local s2 = string.format("%d", 1) @@ -966,10 +953,7 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "assert_removes_falsy_types") )"); LUAU_REQUIRE_NO_ERRORS(result); - if (FFlag::LuauLowerBoundsCalculation) - CHECK_EQ("((boolean | number)?) -> number | true", toString(requireType("f"))); - else - CHECK_EQ("((boolean | number)?) -> boolean | number", toString(requireType("f"))); + CHECK_EQ("((boolean | number)?) -> boolean | number", toString(requireType("f"))); } TEST_CASE_FIXTURE(BuiltinsFixture, "assert_removes_falsy_types2") @@ -1040,8 +1024,6 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "table_freeze_is_generic") TEST_CASE_FIXTURE(BuiltinsFixture, "set_metatable_needs_arguments") { - ScopedFastFlag luauFunctionArgMismatchDetails{"LuauFunctionArgMismatchDetails", true}; - ScopedFastFlag sff{"LuauSetMetaTableArgsCheck", true}; CheckResult result = check(R"( local a = {b=setmetatable} diff --git a/tests/TypeInfer.classes.test.cpp b/tests/TypeInfer.classes.test.cpp index 87ec58c9..d00f1d83 100644 --- a/tests/TypeInfer.classes.test.cpp +++ b/tests/TypeInfer.classes.test.cpp @@ -482,7 +482,7 @@ local a: ChildClass = i TEST_CASE_FIXTURE(ClassFixture, "intersections_of_unions_of_classes") { - ScopedFastFlag sffs[] { + ScopedFastFlag sffs[]{ {"LuauSubtypeNormalizer", true}, {"LuauTypeNormalization2", true}, }; @@ -499,7 +499,7 @@ TEST_CASE_FIXTURE(ClassFixture, "intersections_of_unions_of_classes") TEST_CASE_FIXTURE(ClassFixture, "unions_of_intersections_of_classes") { - ScopedFastFlag sffs[] { + ScopedFastFlag sffs[]{ {"LuauSubtypeNormalizer", true}, {"LuauTypeNormalization2", true}, }; diff --git a/tests/TypeInfer.functions.test.cpp b/tests/TypeInfer.functions.test.cpp index fa99ff58..edc25c7e 100644 --- a/tests/TypeInfer.functions.test.cpp +++ b/tests/TypeInfer.functions.test.cpp @@ -14,7 +14,6 @@ using namespace Luau; -LUAU_FASTFLAG(LuauLowerBoundsCalculation); LUAU_FASTFLAG(LuauInstantiateInSubtyping); LUAU_FASTFLAG(LuauSpecialTypesAsterisked); @@ -299,22 +298,6 @@ TEST_CASE_FIXTURE(Fixture, "cyclic_function_type_in_rets") CHECK_EQ("t1 where t1 = () -> t1", toString(requireType("f"))); } -TEST_CASE_FIXTURE(Fixture, "cyclic_function_type_in_args") -{ - ScopedFastFlag sff[] = { - {"LuauLowerBoundsCalculation", true}, - }; - - CheckResult result = check(R"( - function f(g) - return f(f) - end - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - CHECK_EQ("t1 where t1 = (t1) -> (a...)", toString(requireType("f"))); -} - TEST_CASE_FIXTURE(Fixture, "another_higher_order_function") { CheckResult result = check(R"( @@ -1132,16 +1115,13 @@ f(function(x) return x * 2 end) LUAU_REQUIRE_ERROR_COUNT(1, result); CHECK_EQ("Type 'number' could not be converted into 'Table'", toString(result.errors[0])); - if (!FFlag::LuauLowerBoundsCalculation) - { - // Return type doesn't inference 'nil' - result = check(R"( - function f(a: (number) -> nil) return a(4) end - f(function(x) print(x) end) - )"); + // Return type doesn't inference 'nil' + result = check(R"( + function f(a: (number) -> nil) return a(4) end + f(function(x) print(x) end) + )"); - LUAU_REQUIRE_NO_ERRORS(result); - } + LUAU_REQUIRE_NO_ERRORS(result); } TEST_CASE_FIXTURE(BuiltinsFixture, "infer_anonymous_function_arguments") @@ -1244,16 +1224,13 @@ f(function(x) return x * 2 end) LUAU_REQUIRE_ERROR_COUNT(1, result); CHECK_EQ("Type 'number' could not be converted into 'Table'", toString(result.errors[0])); - if (!FFlag::LuauLowerBoundsCalculation) - { - // Return type doesn't inference 'nil' - result = check(R"( - function f(a: (number) -> nil) return a(4) end - f(function(x) print(x) end) - )"); + // Return type doesn't inference 'nil' + result = check(R"( + function f(a: (number) -> nil) return a(4) end + f(function(x) print(x) end) + )"); - LUAU_REQUIRE_NO_ERRORS(result); - } + LUAU_REQUIRE_NO_ERRORS(result); } TEST_CASE_FIXTURE(Fixture, "infer_anonymous_function_arguments_outside_call") @@ -1436,87 +1413,6 @@ end CHECK_EQ(toString(result.errors[1]), R"(Type 'string' could not be converted into 'number')"); } -TEST_CASE_FIXTURE(Fixture, "inconsistent_return_types") -{ - const ScopedFastFlag flags[] = { - {"LuauLowerBoundsCalculation", true}, - }; - - CheckResult result = check(R"( - function foo(a: boolean, b: number) - if a then - return nil - else - return b - end - end - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - CHECK_EQ("(boolean, number) -> number?", toString(requireType("foo"))); - - // TODO: Test multiple returns - // Think of various cases where typepacks need to grow. maybe consult other tests - // Basic normalization of ConstrainedTypeVars during quantification -} - -TEST_CASE_FIXTURE(Fixture, "inconsistent_higher_order_function") -{ - const ScopedFastFlag flags[] = { - {"LuauLowerBoundsCalculation", true}, - }; - - CheckResult result = check(R"( - function foo(f) - f(5) - f("six") - end - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - - CHECK_EQ("((number | string) -> (a...)) -> ()", toString(requireType("foo"))); -} - - -/* The bug here is that we are using the same level 2.0 for both the body of resolveDispatcher and the - * lambda useCallback. - * - * I think what we want to do is, at each scope level, never reuse the same sublevel. - * - * We also adjust checkBlock to consider the syntax `local x = function() ... end` to be sortable - * in the same way as `local function x() ... end`. This causes the function `resolveDispatcher` to be - * checked before the lambda. - */ -TEST_CASE_FIXTURE(Fixture, "inferred_higher_order_functions_are_quantified_at_the_right_time") -{ - ScopedFastFlag sff[] = { - {"LuauLowerBoundsCalculation", true}, - }; - - CheckResult result = check(R"( - --!strict - - local function resolveDispatcher() - return (nil :: any) :: {useCallback: (any) -> any} - end - - local useCallback = function(deps: any) - return resolveDispatcher().useCallback(deps) - end - )"); - - // LUAU_REQUIRE_NO_ERRORS is particularly unhelpful when this test is broken. - // You get a TypeMismatch error where both types stringify the same. - - CHECK(result.errors.empty()); - if (!result.errors.empty()) - { - for (const auto& e : result.errors) - printf("%s: %s\n", toString(e.location).c_str(), toString(e).c_str()); - } -} - TEST_CASE_FIXTURE(Fixture, "inferred_higher_order_functions_are_quantified_at_the_right_time2") { CheckResult result = check(R"( @@ -1700,56 +1596,6 @@ TEST_CASE_FIXTURE(Fixture, "occurs_check_failure_in_function_return_type") CHECK(nullptr != get(result.errors[0])); } -TEST_CASE_FIXTURE(Fixture, "quantify_constrained_types") -{ - ScopedFastFlag sff[]{ - {"LuauLowerBoundsCalculation", true}, - }; - - CheckResult result = check(R"( - --!strict - local function foo(f) - f(5) - f("hi") - local function g() - return f - end - local h = g() - h(true) - end - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - - CHECK_EQ("((boolean | number | string) -> (a...)) -> ()", toString(requireType("foo"))); -} - -TEST_CASE_FIXTURE(Fixture, "call_o_with_another_argument_after_foo_was_quantified") -{ - ScopedFastFlag sff[]{ - {"LuauLowerBoundsCalculation", true}, - }; - - CheckResult result = check(R"( - local function f(o) - local t = {} - t[o] = true - - local function foo(o) - o.m1(5) - t[o] = nil - end - - o.m1("hi") - - return t - end - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - // TODO: check the normalized type of f -} - TEST_CASE_FIXTURE(Fixture, "free_is_not_bound_to_unknown") { CheckResult result = check(R"( @@ -1800,8 +1646,6 @@ TEST_CASE_FIXTURE(Fixture, "dont_mutate_the_underlying_head_of_typepack_when_cal TEST_CASE_FIXTURE(BuiltinsFixture, "improved_function_arg_mismatch_errors") { - ScopedFastFlag luauFunctionArgMismatchDetails{"LuauFunctionArgMismatchDetails", true}; - CheckResult result = check(R"( local function foo1(a: number) end foo1() @@ -1838,8 +1682,6 @@ u.a.foo() // This might be surprising, but since 'any' became optional, unannotated functions in non-strict 'expect' 0 arguments TEST_CASE_FIXTURE(BuiltinsFixture, "improved_function_arg_mismatch_error_nonstrict") { - ScopedFastFlag luauFunctionArgMismatchDetails{"LuauFunctionArgMismatchDetails", true}; - CheckResult result = check(R"( --!nonstrict local function foo(a, b) end diff --git a/tests/TypeInfer.generics.test.cpp b/tests/TypeInfer.generics.test.cpp index 1b02abc1..e1729ef5 100644 --- a/tests/TypeInfer.generics.test.cpp +++ b/tests/TypeInfer.generics.test.cpp @@ -9,7 +9,6 @@ #include "doctest.h" -LUAU_FASTFLAG(LuauCheckGenericHOFTypes) LUAU_FASTFLAG(LuauInstantiateInSubtyping) LUAU_FASTFLAG(LuauSpecialTypesAsterisked) @@ -783,8 +782,6 @@ local TheDispatcher: Dispatcher = { TEST_CASE_FIXTURE(Fixture, "generic_argument_count_too_few") { - ScopedFastFlag luauFunctionArgMismatchDetails{"LuauFunctionArgMismatchDetails", true}; - CheckResult result = check(R"( function test(a: number) return 1 @@ -802,8 +799,6 @@ wrapper(test) TEST_CASE_FIXTURE(Fixture, "generic_argument_count_too_many") { - ScopedFastFlag luauFunctionArgMismatchDetails{"LuauFunctionArgMismatchDetails", true}; - CheckResult result = check(R"( function test2(a: number, b: string) return 1 @@ -965,7 +960,6 @@ TEST_CASE_FIXTURE(Fixture, "instantiate_generic_function_in_assignments") CHECK_EQ("((a) -> (b...), a) -> (b...)", toString(tm->givenType)); else CHECK_EQ("((number) -> number, number) -> number", toString(tm->givenType)); - } TEST_CASE_FIXTURE(Fixture, "instantiate_generic_function_in_assignments2") @@ -1114,27 +1108,7 @@ local b = sumrec(sum) -- ok local c = sumrec(function(x, y, f) return f(x, y) end) -- type binders are not inferred )"); - if (FFlag::LuauCheckGenericHOFTypes) - { - LUAU_REQUIRE_NO_ERRORS(result); - } - else if (FFlag::LuauInstantiateInSubtyping) - { - LUAU_REQUIRE_ERRORS(result); - CHECK_EQ( - R"(Type '(a, b, (a, b) -> (c...)) -> (c...)' could not be converted into '(a, a, (a, a) -> a) -> a' -caused by: - Argument #1 type is not compatible. Generic subtype escaping scope)", - toString(result.errors[0])); - } - else - { - LUAU_REQUIRE_ERRORS(result); - CHECK_EQ( - "Type '(a, b, (a, b) -> (c...)) -> (c...)' could not be converted into '(a, a, (a, a) -> a) -> a'; different number of generic type " - "parameters", - toString(result.errors[0])); - } + LUAU_REQUIRE_NO_ERRORS(result); } TEST_CASE_FIXTURE(Fixture, "substitution_with_bound_table") @@ -1258,7 +1232,6 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "higher_rank_polymorphism_should_not_accept_i { ScopedFastFlag sffs[] = { {"LuauInstantiateInSubtyping", true}, - {"LuauCheckGenericHOFTypes", true}, // necessary because of interactions with the test }; CheckResult result = check(R"( diff --git a/tests/TypeInfer.intersectionTypes.test.cpp b/tests/TypeInfer.intersectionTypes.test.cpp index e49df101..ca22c351 100644 --- a/tests/TypeInfer.intersectionTypes.test.cpp +++ b/tests/TypeInfer.intersectionTypes.test.cpp @@ -8,7 +8,6 @@ using namespace Luau; -LUAU_FASTFLAG(LuauLowerBoundsCalculation); TEST_SUITE_BEGIN("IntersectionTypes"); @@ -306,10 +305,7 @@ TEST_CASE_FIXTURE(Fixture, "table_intersection_write_sealed") )"); LUAU_REQUIRE_ERROR_COUNT(1, result); - if (FFlag::LuauLowerBoundsCalculation) - CHECK_EQ(toString(result.errors[0]), "Cannot add property 'z' to table '{| x: number, y: number |}'"); - else - CHECK_EQ(toString(result.errors[0]), "Cannot add property 'z' to table 'X & Y'"); + CHECK_EQ(toString(result.errors[0]), "Cannot add property 'z' to table 'X & Y'"); } TEST_CASE_FIXTURE(Fixture, "table_intersection_write_sealed_indirect") @@ -333,16 +329,9 @@ TEST_CASE_FIXTURE(Fixture, "table_intersection_write_sealed_indirect") CHECK_EQ(toString(result.errors[0]), R"(Type '(string, number) -> string' could not be converted into '(string) -> string' caused by: Argument count mismatch. Function expects 2 arguments, but only 1 is specified)"); - if (FFlag::LuauLowerBoundsCalculation) - CHECK_EQ(toString(result.errors[1]), "Cannot add property 'z' to table '{| x: (number) -> number, y: (string) -> string |}'"); - else - CHECK_EQ(toString(result.errors[1]), "Cannot add property 'z' to table 'X & Y'"); + CHECK_EQ(toString(result.errors[1]), "Cannot add property 'z' to table 'X & Y'"); CHECK_EQ(toString(result.errors[2]), "Type 'number' could not be converted into 'string'"); - - if (FFlag::LuauLowerBoundsCalculation) - CHECK_EQ(toString(result.errors[3]), "Cannot add property 'w' to table '{| x: (number) -> number, y: (string) -> string |}'"); - else - CHECK_EQ(toString(result.errors[3]), "Cannot add property 'w' to table 'X & Y'"); + CHECK_EQ(toString(result.errors[3]), "Cannot add property 'w' to table 'X & Y'"); } TEST_CASE_FIXTURE(Fixture, "table_write_sealed_indirect") @@ -381,15 +370,11 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "table_intersection_setmetatable") TEST_CASE_FIXTURE(Fixture, "error_detailed_intersection_part") { - ScopedFastFlag flags[] = {{"LuauLowerBoundsCalculation", false}}; - CheckResult result = check(R"( type X = { x: number } type Y = { y: number } type Z = { z: number } - type XYZ = X & Y & Z - local a: XYZ = 3 )"); @@ -401,15 +386,11 @@ caused by: TEST_CASE_FIXTURE(Fixture, "error_detailed_intersection_all") { - ScopedFastFlag flags[] = {{"LuauLowerBoundsCalculation", false}}; - CheckResult result = check(R"( type X = { x: number } type Y = { y: number } type Z = { z: number } - type XYZ = X & Y & Z - local a: XYZ local b: number = a )"); @@ -468,12 +449,13 @@ TEST_CASE_FIXTURE(Fixture, "intersect_false_and_bool_and_false") LUAU_REQUIRE_ERROR_COUNT(1, result); // TODO: odd stringification of `false & (boolean & false)`.) - CHECK_EQ(toString(result.errors[0]), "Type 'boolean & false & false' could not be converted into 'true'; none of the intersection parts are compatible"); + CHECK_EQ(toString(result.errors[0]), + "Type 'boolean & false & false' could not be converted into 'true'; none of the intersection parts are compatible"); } TEST_CASE_FIXTURE(Fixture, "intersect_saturate_overloaded_functions") { - ScopedFastFlag sffs[] { + ScopedFastFlag sffs[]{ {"LuauSubtypeNormalizer", true}, {"LuauTypeNormalization2", true}, }; @@ -485,12 +467,13 @@ TEST_CASE_FIXTURE(Fixture, "intersect_saturate_overloaded_functions") )"); LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK_EQ(toString(result.errors[0]), "Type '((number?) -> number?) & ((string?) -> string?)' could not be converted into '(number) -> number'; none of the intersection parts are compatible"); + CHECK_EQ(toString(result.errors[0]), "Type '((number?) -> number?) & ((string?) -> string?)' could not be converted into '(number) -> number'; " + "none of the intersection parts are compatible"); } TEST_CASE_FIXTURE(Fixture, "union_saturate_overloaded_functions") { - ScopedFastFlag sffs[] { + ScopedFastFlag sffs[]{ {"LuauSubtypeNormalizer", true}, {"LuauTypeNormalization2", true}, }; @@ -502,12 +485,13 @@ TEST_CASE_FIXTURE(Fixture, "union_saturate_overloaded_functions") )"); LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK_EQ(toString(result.errors[0]), "Type '((number) -> number) & ((string) -> string)' could not be converted into '(boolean | number) -> boolean | number'; none of the intersection parts are compatible"); + CHECK_EQ(toString(result.errors[0]), "Type '((number) -> number) & ((string) -> string)' could not be converted into '(boolean | number) -> " + "boolean | number'; none of the intersection parts are compatible"); } TEST_CASE_FIXTURE(Fixture, "intersection_of_tables") { - ScopedFastFlag sffs[] { + ScopedFastFlag sffs[]{ {"LuauSubtypeNormalizer", true}, {"LuauTypeNormalization2", true}, }; @@ -519,7 +503,8 @@ TEST_CASE_FIXTURE(Fixture, "intersection_of_tables") )"); LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK_EQ(toString(result.errors[0]), "Type '{| p: number?, q: number?, r: number? |} & {| p: number?, q: string? |}' could not be converted into '{| p: nil |}'; none of the intersection parts are compatible"); + CHECK_EQ(toString(result.errors[0]), "Type '{| p: number?, q: number?, r: number? |} & {| p: number?, q: string? |}' could not be converted into " + "'{| p: nil |}'; none of the intersection parts are compatible"); } TEST_CASE_FIXTURE(Fixture, "intersection_of_tables_with_top_properties") @@ -531,12 +516,13 @@ TEST_CASE_FIXTURE(Fixture, "intersection_of_tables_with_top_properties") )"); LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK_EQ(toString(result.errors[0]), "Type '{| p: number?, q: any |} & {| p: unknown, q: string? |}' could not be converted into '{| p: string?, q: number? |}'; none of the intersection parts are compatible"); + CHECK_EQ(toString(result.errors[0]), "Type '{| p: number?, q: any |} & {| p: unknown, q: string? |}' could not be converted into '{| p: string?, " + "q: number? |}'; none of the intersection parts are compatible"); } TEST_CASE_FIXTURE(Fixture, "intersection_of_tables_with_never_properties") { - ScopedFastFlag sffs[] { + ScopedFastFlag sffs[]{ {"LuauSubtypeNormalizer", true}, {"LuauTypeNormalization2", true}, }; @@ -549,12 +535,13 @@ TEST_CASE_FIXTURE(Fixture, "intersection_of_tables_with_never_properties") // TODO: this should not produce type errors, since never <: { p : never } LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK_EQ(toString(result.errors[0]), "Type '{| p: never, q: string? |} & {| p: number?, q: never |}' could not be converted into 'never'; none of the intersection parts are compatible"); + CHECK_EQ(toString(result.errors[0]), "Type '{| p: never, q: string? |} & {| p: number?, q: never |}' could not be converted into 'never'; none " + "of the intersection parts are compatible"); } TEST_CASE_FIXTURE(Fixture, "overloaded_functions_returning_intersections") { - ScopedFastFlag sffs[] { + ScopedFastFlag sffs[]{ {"LuauSubtypeNormalizer", true}, {"LuauTypeNormalization2", true}, }; @@ -566,12 +553,14 @@ TEST_CASE_FIXTURE(Fixture, "overloaded_functions_returning_intersections") )"); LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK_EQ(toString(result.errors[0]), "Type '((number?) -> {| p: number |} & {| q: number |}) & ((string?) -> {| p: number |} & {| r: number |})' could not be converted into '(number?) -> {| p: number, q: number, r: number |}'; none of the intersection parts are compatible"); + CHECK_EQ(toString(result.errors[0]), + "Type '((number?) -> {| p: number |} & {| q: number |}) & ((string?) -> {| p: number |} & {| r: number |})' could not be converted into " + "'(number?) -> {| p: number, q: number, r: number |}'; none of the intersection parts are compatible"); } TEST_CASE_FIXTURE(Fixture, "overloaded_functions_mentioning_generic") { - ScopedFastFlag sffs[] { + ScopedFastFlag sffs[]{ {"LuauSubtypeNormalizer", true}, {"LuauTypeNormalization2", true}, }; @@ -585,12 +574,13 @@ TEST_CASE_FIXTURE(Fixture, "overloaded_functions_mentioning_generic") )"); LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK_EQ(toString(result.errors[0]), "Type '((number?) -> a | number) & ((string?) -> a | string)' could not be converted into '(number?) -> a'; none of the intersection parts are compatible"); + CHECK_EQ(toString(result.errors[0]), "Type '((number?) -> a | number) & ((string?) -> a | string)' could not be converted into '(number?) -> a'; " + "none of the intersection parts are compatible"); } TEST_CASE_FIXTURE(Fixture, "overloaded_functions_mentioning_generics") { - ScopedFastFlag sffs[] { + ScopedFastFlag sffs[]{ {"LuauSubtypeNormalizer", true}, {"LuauTypeNormalization2", true}, }; @@ -604,12 +594,13 @@ TEST_CASE_FIXTURE(Fixture, "overloaded_functions_mentioning_generics") )"); LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK_EQ(toString(result.errors[0]), "Type '((a?) -> a | b) & ((c?) -> b | c)' could not be converted into '(a?) -> (a & c) | b'; none of the intersection parts are compatible"); + CHECK_EQ(toString(result.errors[0]), + "Type '((a?) -> a | b) & ((c?) -> b | c)' could not be converted into '(a?) -> (a & c) | b'; none of the intersection parts are compatible"); } TEST_CASE_FIXTURE(Fixture, "overloaded_functions_mentioning_generic_packs") { - ScopedFastFlag sffs[] { + ScopedFastFlag sffs[]{ {"LuauSubtypeNormalizer", true}, {"LuauTypeNormalization2", true}, }; @@ -623,12 +614,13 @@ TEST_CASE_FIXTURE(Fixture, "overloaded_functions_mentioning_generic_packs") )"); LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK_EQ(toString(result.errors[0]), "Type '((number?, a...) -> (number?, b...)) & ((string?, a...) -> (string?, b...))' could not be converted into '(nil, b...) -> (nil, a...)'; none of the intersection parts are compatible"); + CHECK_EQ(toString(result.errors[0]), "Type '((number?, a...) -> (number?, b...)) & ((string?, a...) -> (string?, b...))' could not be converted " + "into '(nil, b...) -> (nil, a...)'; none of the intersection parts are compatible"); } TEST_CASE_FIXTURE(Fixture, "overloadeded_functions_with_unknown_result") { - ScopedFastFlag sffs[] { + ScopedFastFlag sffs[]{ {"LuauSubtypeNormalizer", true}, {"LuauTypeNormalization2", true}, }; @@ -642,12 +634,13 @@ TEST_CASE_FIXTURE(Fixture, "overloadeded_functions_with_unknown_result") )"); LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK_EQ(toString(result.errors[0]), "Type '((nil) -> unknown) & ((number) -> number)' could not be converted into '(number?) -> number?'; none of the intersection parts are compatible"); + CHECK_EQ(toString(result.errors[0]), "Type '((nil) -> unknown) & ((number) -> number)' could not be converted into '(number?) -> number?'; none " + "of the intersection parts are compatible"); } TEST_CASE_FIXTURE(Fixture, "overloadeded_functions_with_unknown_arguments") { - ScopedFastFlag sffs[] { + ScopedFastFlag sffs[]{ {"LuauSubtypeNormalizer", true}, {"LuauTypeNormalization2", true}, }; @@ -661,12 +654,13 @@ TEST_CASE_FIXTURE(Fixture, "overloadeded_functions_with_unknown_arguments") )"); LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK_EQ(toString(result.errors[0]), "Type '((number) -> number?) & ((unknown) -> string?)' could not be converted into '(number?) -> nil'; none of the intersection parts are compatible"); + CHECK_EQ(toString(result.errors[0]), "Type '((number) -> number?) & ((unknown) -> string?)' could not be converted into '(number?) -> nil'; none " + "of the intersection parts are compatible"); } TEST_CASE_FIXTURE(Fixture, "overloadeded_functions_with_never_result") { - ScopedFastFlag sffs[] { + ScopedFastFlag sffs[]{ {"LuauSubtypeNormalizer", true}, {"LuauTypeNormalization2", true}, }; @@ -680,12 +674,13 @@ TEST_CASE_FIXTURE(Fixture, "overloadeded_functions_with_never_result") )"); LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK_EQ(toString(result.errors[0]), "Type '((nil) -> never) & ((number) -> number)' could not be converted into '(number?) -> never'; none of the intersection parts are compatible"); + CHECK_EQ(toString(result.errors[0]), "Type '((nil) -> never) & ((number) -> number)' could not be converted into '(number?) -> never'; none of " + "the intersection parts are compatible"); } TEST_CASE_FIXTURE(Fixture, "overloadeded_functions_with_never_arguments") { - ScopedFastFlag sffs[] { + ScopedFastFlag sffs[]{ {"LuauSubtypeNormalizer", true}, {"LuauTypeNormalization2", true}, }; @@ -699,7 +694,8 @@ TEST_CASE_FIXTURE(Fixture, "overloadeded_functions_with_never_arguments") )"); LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK_EQ(toString(result.errors[0]), "Type '((never) -> string?) & ((number) -> number?)' could not be converted into '(number?) -> nil'; none of the intersection parts are compatible"); + CHECK_EQ(toString(result.errors[0]), "Type '((never) -> string?) & ((number) -> number?)' could not be converted into '(number?) -> nil'; none " + "of the intersection parts are compatible"); } TEST_CASE_FIXTURE(Fixture, "overloadeded_functions_with_overlapping_results_and_variadics") @@ -711,7 +707,8 @@ TEST_CASE_FIXTURE(Fixture, "overloadeded_functions_with_overlapping_results_and_ )"); LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK_EQ(toString(result.errors[0]), "Type '((number?) -> (...number)) & ((string?) -> number | string)' could not be converted into '(number | string) -> (number, number?)'; none of the intersection parts are compatible"); + CHECK_EQ(toString(result.errors[0]), "Type '((number?) -> (...number)) & ((string?) -> number | string)' could not be converted into '(number | " + "string) -> (number, number?)'; none of the intersection parts are compatible"); } TEST_CASE_FIXTURE(Fixture, "overloadeded_functions_with_weird_typepacks_1") @@ -725,7 +722,8 @@ TEST_CASE_FIXTURE(Fixture, "overloadeded_functions_with_weird_typepacks_1") )"); LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK_EQ(toString(result.errors[0]), "Type '(() -> (a...)) & (() -> (b...))' could not be converted into '() -> ()'; none of the intersection parts are compatible"); + CHECK_EQ(toString(result.errors[0]), + "Type '(() -> (a...)) & (() -> (b...))' could not be converted into '() -> ()'; none of the intersection parts are compatible"); } TEST_CASE_FIXTURE(Fixture, "overloadeded_functions_with_weird_typepacks_2") @@ -739,7 +737,8 @@ TEST_CASE_FIXTURE(Fixture, "overloadeded_functions_with_weird_typepacks_2") )"); LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK_EQ(toString(result.errors[0]), "Type '((a...) -> ()) & ((b...) -> ())' could not be converted into '() -> ()'; none of the intersection parts are compatible"); + CHECK_EQ(toString(result.errors[0]), + "Type '((a...) -> ()) & ((b...) -> ())' could not be converted into '() -> ()'; none of the intersection parts are compatible"); } TEST_CASE_FIXTURE(Fixture, "overloadeded_functions_with_weird_typepacks_3") @@ -753,7 +752,8 @@ TEST_CASE_FIXTURE(Fixture, "overloadeded_functions_with_weird_typepacks_3") )"); LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK_EQ(toString(result.errors[0]), "Type '(() -> (a...)) & (() -> (number?, a...))' could not be converted into '() -> number'; none of the intersection parts are compatible"); + CHECK_EQ(toString(result.errors[0]), + "Type '(() -> (a...)) & (() -> (number?, a...))' could not be converted into '() -> number'; none of the intersection parts are compatible"); } TEST_CASE_FIXTURE(Fixture, "overloadeded_functions_with_weird_typepacks_4") @@ -767,12 +767,13 @@ TEST_CASE_FIXTURE(Fixture, "overloadeded_functions_with_weird_typepacks_4") )"); LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK_EQ(toString(result.errors[0]), "Type '((a...) -> ()) & ((number, a...) -> number)' could not be converted into '(number?) -> ()'; none of the intersection parts are compatible"); + CHECK_EQ(toString(result.errors[0]), "Type '((a...) -> ()) & ((number, a...) -> number)' could not be converted into '(number?) -> ()'; none of " + "the intersection parts are compatible"); } TEST_CASE_FIXTURE(BuiltinsFixture, "intersect_metatables") { - ScopedFastFlag sffs[] { + ScopedFastFlag sffs[]{ {"LuauSubtypeNormalizer", true}, {"LuauTypeNormalization2", true}, }; @@ -800,7 +801,7 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "intersect_metatables") TEST_CASE_FIXTURE(BuiltinsFixture, "intersect_metatable_subtypes") { - ScopedFastFlag sffs[] { + ScopedFastFlag sffs[]{ {"LuauSubtypeNormalizer", true}, {"LuauTypeNormalization2", true}, }; @@ -826,7 +827,7 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "intersect_metatable_subtypes") TEST_CASE_FIXTURE(BuiltinsFixture, "intersect_metatables_with_properties") { - ScopedFastFlag sffs[] { + ScopedFastFlag sffs[]{ {"LuauSubtypeNormalizer", true}, {"LuauTypeNormalization2", true}, }; @@ -849,7 +850,7 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "intersect_metatables_with_properties") TEST_CASE_FIXTURE(BuiltinsFixture, "intersect_metatable_with table") { - ScopedFastFlag sffs[] { + ScopedFastFlag sffs[]{ {"LuauSubtypeNormalizer", true}, {"LuauTypeNormalization2", true}, }; @@ -874,7 +875,7 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "intersect_metatable_with table") TEST_CASE_FIXTURE(Fixture, "CLI-44817") { - ScopedFastFlag sffs[] { + ScopedFastFlag sffs[]{ {"LuauSubtypeNormalizer", true}, {"LuauTypeNormalization2", true}, }; diff --git a/tests/TypeInfer.loops.test.cpp b/tests/TypeInfer.loops.test.cpp index 588a9a76..d6f787be 100644 --- a/tests/TypeInfer.loops.test.cpp +++ b/tests/TypeInfer.loops.test.cpp @@ -622,9 +622,9 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "loop_iter_metamethod_not_enough_returns") LUAU_REQUIRE_ERROR_COUNT(1, result); CHECK(result.errors[0] == TypeError{ - Location{{2, 36}, {2, 37}}, - GenericError{"__iter must return at least one value"}, - }); + Location{{2, 36}, {2, 37}}, + GenericError{"__iter must return at least one value"}, + }); } TEST_CASE_FIXTURE(BuiltinsFixture, "loop_iter_metamethod_ok") diff --git a/tests/TypeInfer.modules.test.cpp b/tests/TypeInfer.modules.test.cpp index 36943cac..8b7b3514 100644 --- a/tests/TypeInfer.modules.test.cpp +++ b/tests/TypeInfer.modules.test.cpp @@ -254,20 +254,20 @@ return m if (FFlag::LuauInstantiateInSubtyping) { // though this didn't error before the flag, it seems as though it should error since fields of a table are invariant. - // the user's intent would likely be that these "method" fields would be read-only, but without an annotation, accepting this should be unsound. + // the user's intent would likely be that these "method" fields would be read-only, but without an annotation, accepting this should be + // unsound. LUAU_REQUIRE_ERROR_COUNT(1, result); CHECK_EQ(R"(Type 'n' could not be converted into 't1 where t1 = {- Clone: (t1) -> (a...) -}' caused by: Property 'Clone' is not compatible. Type '(a) -> ()' could not be converted into 't1 where t1 = ({- Clone: t1 -}) -> (a...)'; different number of generic type parameters)", - toString(result.errors[0])); + toString(result.errors[0])); } else { LUAU_REQUIRE_NO_ERRORS(result); } - } TEST_CASE_FIXTURE(BuiltinsFixture, "custom_require_global") diff --git a/tests/TypeInfer.provisional.test.cpp b/tests/TypeInfer.provisional.test.cpp index 2aac6653..ccc4d775 100644 --- a/tests/TypeInfer.provisional.test.cpp +++ b/tests/TypeInfer.provisional.test.cpp @@ -7,8 +7,6 @@ #include -LUAU_FASTFLAG(LuauLowerBoundsCalculation) - using namespace Luau; TEST_SUITE_BEGIN("ProvisionalTests"); @@ -301,19 +299,10 @@ TEST_CASE_FIXTURE(Fixture, "do_not_ice_when_trying_to_pick_first_of_generic_type LUAU_REQUIRE_NO_ERRORS(result); - if (FFlag::LuauLowerBoundsCalculation) - { - CHECK_EQ("() -> ()", toString(requireType("f"))); - CHECK_EQ("() -> ()", toString(requireType("g"))); - CHECK_EQ("nil", toString(requireType("x"))); - } - else - { - // f and g should have the type () -> () - CHECK_EQ("() -> (a...)", toString(requireType("f"))); - CHECK_EQ("() -> (a...)", toString(requireType("g"))); - CHECK_EQ("any", toString(requireType("x"))); // any is returned instead of ICE for now - } + // f and g should have the type () -> () + CHECK_EQ("() -> (a...)", toString(requireType("f"))); + CHECK_EQ("() -> (a...)", toString(requireType("g"))); + CHECK_EQ("any", toString(requireType("x"))); // any is returned instead of ICE for now } TEST_CASE_FIXTURE(Fixture, "specialization_binds_with_prototypes_too_early") @@ -330,7 +319,6 @@ TEST_CASE_FIXTURE(Fixture, "specialization_binds_with_prototypes_too_early") TEST_CASE_FIXTURE(Fixture, "weird_fail_to_unify_type_pack") { ScopedFastFlag sff[] = { - {"LuauLowerBoundsCalculation", false}, // I'm not sure why this is broken without DCR, but it seems to be fixed // when DCR is enabled. {"DebugLuauDeferredConstraintResolution", false}, @@ -347,7 +335,6 @@ TEST_CASE_FIXTURE(Fixture, "weird_fail_to_unify_type_pack") TEST_CASE_FIXTURE(Fixture, "weird_fail_to_unify_variadic_pack") { ScopedFastFlag sff[] = { - {"LuauLowerBoundsCalculation", false}, // I'm not sure why this is broken without DCR, but it seems to be fixed // when DCR is enabled. {"DebugLuauDeferredConstraintResolution", false}, @@ -362,56 +349,6 @@ TEST_CASE_FIXTURE(Fixture, "weird_fail_to_unify_variadic_pack") LUAU_REQUIRE_ERRORS(result); // Should not have any errors. } -TEST_CASE_FIXTURE(Fixture, "lower_bounds_calculation_is_too_permissive_with_overloaded_higher_order_functions") -{ - ScopedFastFlag sff[] = { - {"LuauLowerBoundsCalculation", true}, - }; - - CheckResult result = check(R"( - function foo(f) - f(5, 'a') - f('b', 6) - end - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - - // We incorrectly infer that the argument to foo could be called with (number, number) or (string, string) - // even though that is strictly more permissive than the actual source text shows. - CHECK("((number | string, number | string) -> (a...)) -> ()" == toString(requireType("foo"))); -} - -// Once fixed, move this to Normalize.test.cpp -TEST_CASE_FIXTURE(Fixture, "normalization_fails_on_certain_kinds_of_cyclic_tables") -{ -#if defined(_DEBUG) || defined(_NOOPT) - ScopedFastInt sfi("LuauNormalizeIterationLimit", 500); -#endif - - ScopedFastFlag flags[] = { - {"LuauLowerBoundsCalculation", true}, - }; - - // We use a function and inferred parameter types to prevent intermediate normalizations from being performed. - // This exposes a bug where the type of y is mutated. - CheckResult result = check(R"( - function strange(x, y) - x.x = y - y.x = x - - type R = {x: typeof(x)} & {x: typeof(y)} - local r: R - - return r - end - )"); - - LUAU_REQUIRE_ERROR_COUNT(1, result); - - CHECK(nullptr != get(result.errors[0])); -} - // Belongs in TypeInfer.builtins.test.cpp. TEST_CASE_FIXTURE(BuiltinsFixture, "pcall_returns_at_least_two_value_but_function_returns_nothing") { @@ -473,36 +410,6 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "function_returns_many_things_but_first_of_it CHECK_EQ("boolean", toString(requireType("b"))); } -TEST_CASE_FIXTURE(Fixture, "constrained_is_level_dependent") -{ - ScopedFastFlag sff[]{ - {"LuauLowerBoundsCalculation", true}, - }; - - CheckResult result = check(R"( - local function f(o) - local t = {} - t[o] = true - - local function foo(o) - o:m1() - t[o] = nil - end - - local function bar(o) - o:m2() - t[o] = true - end - - return t - end - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - // TODO: We're missing generics b... - CHECK_EQ("(t1) -> {| [t1]: boolean |} where t1 = t2 ; t2 = {+ m1: (t1) -> (a...), m2: (t2) -> (b...) +}", toString(requireType("f"))); -} - TEST_CASE_FIXTURE(Fixture, "free_is_not_bound_to_any") { CheckResult result = check(R"( @@ -695,4 +602,187 @@ return wrapStrictTable(Constants, "Constants") CHECK(get(*result)); } +// We need a simplification step to make this do the right thing. ("normalization-lite") +TEST_CASE_FIXTURE(BuiltinsFixture, "table_insert_with_a_singleton_argument") +{ + CheckResult result = check(R"( + local function foo(t, x) + if x == "hi" or x == "bye" then + table.insert(t, x) + end + + return t + end + + local t = foo({}, "hi") + table.insert(t, "totally_unrelated_type" :: "totally_unrelated_type") + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + // We'd really like for this to be {string} + CHECK_EQ("{string | string}", toString(requireType("t"))); +} + +struct NormalizeFixture : Fixture +{ + bool isSubtype(TypeId a, TypeId b) + { + return ::Luau::isSubtype(a, b, NotNull{getMainModule()->getModuleScope().get()}, singletonTypes, ice); + } +}; + +TEST_CASE_FIXTURE(NormalizeFixture, "intersection_of_functions_of_different_arities") +{ + check(R"( + type A = (any) -> () + type B = (any, any) -> () + type T = A & B + + local a: A + local b: B + local t: T + )"); + + [[maybe_unused]] TypeId a = requireType("a"); + [[maybe_unused]] TypeId b = requireType("b"); + + // CHECK(!isSubtype(a, b)); // !! + // CHECK(!isSubtype(b, a)); + + CHECK("((any) -> ()) & ((any, any) -> ())" == toString(requireType("t"))); +} + +TEST_CASE_FIXTURE(NormalizeFixture, "functions_with_mismatching_arity") +{ + check(R"( + local a: (number) -> () + local b: () -> () + + local c: () -> number + )"); + + TypeId a = requireType("a"); + TypeId b = requireType("b"); + TypeId c = requireType("c"); + + // CHECK(!isSubtype(b, a)); + // CHECK(!isSubtype(c, a)); + + CHECK(!isSubtype(a, b)); + // CHECK(!isSubtype(c, b)); + + CHECK(!isSubtype(a, c)); + CHECK(!isSubtype(b, c)); +} + +TEST_CASE_FIXTURE(NormalizeFixture, "functions_with_mismatching_arity_but_optional_parameters") +{ + /* + * (T0..TN) <: (T0..TN, A?) + * (T0..TN) <: (T0..TN, any) + * (T0..TN, A?) R <: U -> S if U <: T and R <: S + * A | B <: T if A <: T and B <: T + * T <: A | B if T <: A or T <: B + */ + check(R"( + local a: (number?) -> () + local b: (number) -> () + local c: (number, number?) -> () + )"); + + TypeId a = requireType("a"); + TypeId b = requireType("b"); + TypeId c = requireType("c"); + + /* + * (number) -> () () + * because number? () + * because number? () <: (number) -> () + * because number <: number? (because number <: number) + */ + CHECK(isSubtype(a, b)); + + /* + * (number, number?) -> () <: (number) -> (number) + * The packs have inequal lengths, but (number) <: (number, number?) + * and number <: number + */ + // CHECK(!isSubtype(c, b)); + + /* + * (number?) -> () () + * because (number, number?) () () + * because (number, number?) () + local b: (number) -> () + local c: (number, any) -> () + )"); + + TypeId a = requireType("a"); + TypeId b = requireType("b"); + TypeId c = requireType("c"); + + /* + * (number) -> () () + * because number? () + * because number? () <: (number) -> () + * because number <: number? (because number <: number) + */ + CHECK(isSubtype(a, b)); + + /* + * (number, any) -> () (number) + * The packs have inequal lengths + */ + // CHECK(!isSubtype(c, b)); + + /* + * (number?) -> () () + * The packs have inequal lengths + */ + // CHECK(!isSubtype(a, c)); + + /* + * (number) -> () () + * The packs have inequal lengths + */ + // CHECK(!isSubtype(b, c)); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.refinements.test.cpp b/tests/TypeInfer.refinements.test.cpp index b6dedcbd..f707f952 100644 --- a/tests/TypeInfer.refinements.test.cpp +++ b/tests/TypeInfer.refinements.test.cpp @@ -7,7 +7,6 @@ #include "doctest.h" -LUAU_FASTFLAG(LuauLowerBoundsCalculation) LUAU_FASTFLAG(LuauSpecialTypesAsterisked) using namespace Luau; @@ -608,10 +607,7 @@ TEST_CASE_FIXTURE(Fixture, "type_guard_can_filter_for_intersection_of_tables") LUAU_REQUIRE_NO_ERRORS(result); - if (FFlag::LuauLowerBoundsCalculation) - CHECK_EQ("{| x: number, y: number |}", toString(requireTypeAtPosition({4, 28}))); - else - CHECK_EQ("{| x: number |} & {| y: number |}", toString(requireTypeAtPosition({4, 28}))); + CHECK_EQ("{| x: number |} & {| y: number |}", toString(requireTypeAtPosition({4, 28}))); CHECK_EQ("nil", toString(requireTypeAtPosition({6, 28}))); } diff --git a/tests/TypeInfer.singletons.test.cpp b/tests/TypeInfer.singletons.test.cpp index 5ee956d7..73ccac70 100644 --- a/tests/TypeInfer.singletons.test.cpp +++ b/tests/TypeInfer.singletons.test.cpp @@ -421,28 +421,6 @@ TEST_CASE_FIXTURE(Fixture, "widening_happens_almost_everywhere_except_for_tables LUAU_REQUIRE_NO_ERRORS(result); } -TEST_CASE_FIXTURE(BuiltinsFixture, "table_insert_with_a_singleton_argument") -{ - ScopedFastFlag sff{"LuauLowerBoundsCalculation", true}; - - CheckResult result = check(R"( - local function foo(t, x) - if x == "hi" or x == "bye" then - table.insert(t, x) - end - - return t - end - - local t = foo({}, "hi") - table.insert(t, "totally_unrelated_type" :: "totally_unrelated_type") - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - - CHECK_EQ("{string}", toString(requireType("t"))); -} - TEST_CASE_FIXTURE(Fixture, "functions_are_not_to_be_widened") { CheckResult result = check(R"( diff --git a/tests/TypeInfer.tables.test.cpp b/tests/TypeInfer.tables.test.cpp index a6d870fa..53f9a1ab 100644 --- a/tests/TypeInfer.tables.test.cpp +++ b/tests/TypeInfer.tables.test.cpp @@ -11,7 +11,6 @@ using namespace Luau; -LUAU_FASTFLAG(LuauLowerBoundsCalculation) LUAU_FASTFLAG(LuauInstantiateInSubtyping) TEST_SUITE_BEGIN("TableTests"); @@ -1196,10 +1195,7 @@ TEST_CASE_FIXTURE(Fixture, "pass_incompatible_union_to_a_generic_table_without_c )"); LUAU_REQUIRE_ERROR_COUNT(1, result); - if (FFlag::LuauLowerBoundsCalculation) - CHECK(get(result.errors[0])); - else - CHECK(get(result.errors[0])); + CHECK(get(result.errors[0])); } // This unit test could be flaky if the fix has regressed. @@ -2627,8 +2623,6 @@ do end TEST_CASE_FIXTURE(BuiltinsFixture, "dont_crash_when_setmetatable_does_not_produce_a_metatabletypevar") { - ScopedFastFlag luauFunctionArgMismatchDetails{"LuauFunctionArgMismatchDetails", true}; - CheckResult result = check("local x = setmetatable({})"); LUAU_REQUIRE_ERROR_COUNT(1, result); CHECK_EQ("Argument count mismatch. Function 'setmetatable' expects 2 arguments, but only 1 is specified", toString(result.errors[0])); @@ -2709,8 +2703,6 @@ local baz = foo[bar] TEST_CASE_FIXTURE(BuiltinsFixture, "table_simple_call") { - ScopedFastFlag luauFunctionArgMismatchDetails{"LuauFunctionArgMismatchDetails", true}; - CheckResult result = check(R"( local a = setmetatable({ x = 2 }, { __call = function(self) @@ -2887,7 +2879,7 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "dont_leak_free_table_props") TEST_CASE_FIXTURE(Fixture, "inferred_return_type_of_free_table") { ScopedFastFlag sff[] = { - {"LuauLowerBoundsCalculation", true}, + // {"LuauLowerBoundsCalculation", true}, {"DebugLuauSharedSelf", true}, }; diff --git a/tests/TypeInfer.test.cpp b/tests/TypeInfer.test.cpp index 26171c51..239b8c28 100644 --- a/tests/TypeInfer.test.cpp +++ b/tests/TypeInfer.test.cpp @@ -14,12 +14,10 @@ #include -LUAU_FASTFLAG(LuauLowerBoundsCalculation); LUAU_FASTFLAG(LuauFixLocationSpanTableIndexExpr); LUAU_FASTFLAG(DebugLuauDeferredConstraintResolution); LUAU_FASTFLAG(LuauInstantiateInSubtyping); LUAU_FASTFLAG(LuauSpecialTypesAsterisked); -LUAU_FASTFLAG(LuauCheckGenericHOFTypes); using namespace Luau; @@ -89,7 +87,6 @@ TEST_CASE_FIXTURE(Fixture, "infer_in_nocheck_mode") { ScopedFastFlag sff[]{ {"DebugLuauDeferredConstraintResolution", false}, - {"LuauLowerBoundsCalculation", true}, }; CheckResult result = check(R"( @@ -1001,21 +998,23 @@ TEST_CASE_FIXTURE(Fixture, "cli_50041_committing_txnlog_in_apollo_client_error") end )"); - if (FFlag::LuauInstantiateInSubtyping && !FFlag::LuauCheckGenericHOFTypes) + if (FFlag::LuauInstantiateInSubtyping) { // though this didn't error before the flag, it seems as though it should error since fields of a table are invariant. - // the user's intent would likely be that these "method" fields would be read-only, but without an annotation, accepting this should be unsound. + // the user's intent would likely be that these "method" fields would be read-only, but without an annotation, accepting this should be + // unsound. LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK_EQ(R"(Type 't1 where t1 = {+ getStoreFieldName: (t1, {| fieldName: string |} & {| from: number? |}) -> (a, b...) +}' could not be converted into 'Policies' + CHECK_EQ( + R"(Type 't1 where t1 = {+ getStoreFieldName: (t1, {| fieldName: string |} & {| from: number? |}) -> (a, b...) +}' could not be converted into 'Policies' caused by: Property 'getStoreFieldName' is not compatible. Type 't1 where t1 = ({+ getStoreFieldName: t1 +}, {| fieldName: string |} & {| from: number? |}) -> (a, b...)' could not be converted into '(Policies, FieldSpecifier) -> string' caused by: Argument #2 type is not compatible. Type 'FieldSpecifier' could not be converted into 'FieldSpecifier & {| from: number? |}' caused by: Not all intersection parts are compatible. Table type 'FieldSpecifier' not compatible with type '{| from: number? |}' because the former has extra field 'fieldName')", - toString(result.errors[0])); + toString(result.errors[0])); } else { @@ -1044,7 +1043,7 @@ TEST_CASE_FIXTURE(Fixture, "type_infer_recursion_limit_no_ice") TEST_CASE_FIXTURE(Fixture, "type_infer_recursion_limit_normalizer") { ScopedFastInt sfi("LuauTypeInferRecursionLimit", 10); - ScopedFastFlag sffs[] { + ScopedFastFlag sffs[]{ {"LuauSubtypeNormalizer", true}, {"LuauTypeNormalization2", true}, {"LuauAutocompleteDynamicLimits", true}, @@ -1057,14 +1056,14 @@ TEST_CASE_FIXTURE(Fixture, "type_infer_recursion_limit_normalizer") end )"); - LUAU_REQUIRE_ERRORS(result); + LUAU_REQUIRE_ERROR_COUNT(1, result); CHECK_EQ("Internal error: Code is too complex to typecheck! Consider adding type annotations around this area", toString(result.errors[0])); } TEST_CASE_FIXTURE(Fixture, "type_infer_cache_limit_normalizer") { ScopedFastInt sfi("LuauNormalizeCacheLimit", 10); - ScopedFastFlag sffs[] { + ScopedFastFlag sffs[]{ {"LuauSubtypeNormalizer", true}, {"LuauTypeNormalization2", true}, }; @@ -1101,45 +1100,6 @@ TEST_CASE_FIXTURE(Fixture, "follow_on_new_types_in_substitution") LUAU_REQUIRE_NO_ERRORS(result); } -/** - * The problem we had here was that the type of q in B.h was initially inferring to {} | {prop: free} before we bound - * that second table to the enclosing union. - */ -TEST_CASE_FIXTURE(Fixture, "do_not_bind_a_free_table_to_a_union_containing_that_table") -{ - ScopedFastFlag flag[] = { - {"LuauLowerBoundsCalculation", true}, - }; - - CheckResult result = check(R"( - --!strict - - local A = {} - - function A:f() - local t = {} - - for key, value in pairs(self) do - t[key] = value - end - - return t - end - - local B = A:f() - - function B.g(t) - assert(type(t) == "table") - assert(t.prop ~= nil) - end - - function B.h(q) - q = q or {} - return q or {} - end - )"); -} - TEST_CASE_FIXTURE(Fixture, "types_stored_in_astResolvedTypes") { CheckResult result = check(R"( diff --git a/tests/TypeInfer.tryUnify.test.cpp b/tests/TypeInfer.tryUnify.test.cpp index dedb7d28..c178d2a4 100644 --- a/tests/TypeInfer.tryUnify.test.cpp +++ b/tests/TypeInfer.tryUnify.test.cpp @@ -302,8 +302,8 @@ TEST_CASE_FIXTURE(TryUnifyFixture, "metatables_unify_against_shape_of_free_table REQUIRE_EQ(state.errors.size(), 1); std::string expected = "Type '{ @metatable {| __index: {| foo: string |} |}, { } }' could not be converted into '{- foo: number -}'\n" - "caused by:\n" - " Type 'number' could not be converted into 'string'"; + "caused by:\n" + " Type 'number' could not be converted into 'string'"; CHECK_EQ(toString(state.errors[0]), expected); } diff --git a/tests/TypeInfer.typePacks.cpp b/tests/TypeInfer.typePacks.cpp index eb61c396..aaa7ded4 100644 --- a/tests/TypeInfer.typePacks.cpp +++ b/tests/TypeInfer.typePacks.cpp @@ -7,9 +7,9 @@ #include "doctest.h" -using namespace Luau; +LUAU_FASTFLAG(LuauFunctionReturnStringificationFixup); -LUAU_FASTFLAG(LuauLowerBoundsCalculation); +using namespace Luau; TEST_SUITE_BEGIN("TypePackTests"); @@ -311,7 +311,7 @@ local c: Packed auto ttvA = get(requireType("a")); REQUIRE(ttvA); CHECK_EQ(toString(requireType("a")), "Packed"); - if (FFlag::LuauLowerBoundsCalculation) + if (FFlag::LuauFunctionReturnStringificationFixup) CHECK_EQ(toString(requireType("a"), {true}), "{| f: (number) -> number |}"); else CHECK_EQ(toString(requireType("a"), {true}), "{| f: (number) -> (number) |}"); @@ -966,8 +966,6 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "detect_cyclic_typepacks2") TEST_CASE_FIXTURE(Fixture, "unify_variadic_tails_in_arguments") { - ScopedFastFlag luauCallUnifyPackTails{"LuauCallUnifyPackTails", true}; - CheckResult result = check(R"( function foo(...: string): number return 1 @@ -984,8 +982,6 @@ TEST_CASE_FIXTURE(Fixture, "unify_variadic_tails_in_arguments") TEST_CASE_FIXTURE(Fixture, "unify_variadic_tails_in_arguments_free") { - ScopedFastFlag luauCallUnifyPackTails{"LuauCallUnifyPackTails", true}; - CheckResult result = check(R"( function foo(...: T...): T... return ... diff --git a/tests/TypeInfer.unionTypes.test.cpp b/tests/TypeInfer.unionTypes.test.cpp index 64c9b563..dc551634 100644 --- a/tests/TypeInfer.unionTypes.test.cpp +++ b/tests/TypeInfer.unionTypes.test.cpp @@ -6,7 +6,6 @@ #include "doctest.h" -LUAU_FASTFLAG(LuauLowerBoundsCalculation) LUAU_FASTFLAG(LuauSpecialTypesAsterisked) using namespace Luau; @@ -360,10 +359,7 @@ a.x = 2 )"); LUAU_REQUIRE_ERROR_COUNT(1, result); - if (FFlag::LuauLowerBoundsCalculation) - CHECK_EQ("Value of type '{| x: number, y: number |}?' could be nil", toString(result.errors[0])); - else - CHECK_EQ("Value of type '({| x: number |} & {| y: number |})?' could be nil", toString(result.errors[0])); + CHECK_EQ("Value of type '({| x: number |} & {| y: number |})?' could be nil", toString(result.errors[0])); } TEST_CASE_FIXTURE(Fixture, "optional_length_error") @@ -532,18 +528,13 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "table_union_write_indirect") LUAU_REQUIRE_ERROR_COUNT(1, result); // NOTE: union normalization will improve this message - if (FFlag::LuauLowerBoundsCalculation) - CHECK_EQ(toString(result.errors[0]), "Type '(string) -> number' could not be converted into '(number) -> string'\n" - "caused by:\n" - " Argument #1 type is not compatible. Type 'number' could not be converted into 'string'"); - else - CHECK_EQ(toString(result.errors[0]), - R"(Type '(string) -> number' could not be converted into '((number) -> string) | ((number) -> string)'; none of the union options are compatible)"); + CHECK_EQ(toString(result.errors[0]), + R"(Type '(string) -> number' could not be converted into '((number) -> string) | ((number) -> string)'; none of the union options are compatible)"); } TEST_CASE_FIXTURE(Fixture, "union_true_and_false") { - ScopedFastFlag sffs[] { + ScopedFastFlag sffs[]{ {"LuauSubtypeNormalizer", true}, {"LuauTypeNormalization2", true}, }; @@ -561,7 +552,7 @@ TEST_CASE_FIXTURE(Fixture, "union_true_and_false") TEST_CASE_FIXTURE(Fixture, "union_of_functions") { - ScopedFastFlag sffs[] { + ScopedFastFlag sffs[]{ {"LuauSubtypeNormalizer", true}, {"LuauTypeNormalization2", true}, }; @@ -598,7 +589,7 @@ TEST_CASE_FIXTURE(Fixture, "union_of_generic_typepack_functions") TEST_CASE_FIXTURE(Fixture, "union_of_functions_mentioning_generics") { - ScopedFastFlag sffs[] { + ScopedFastFlag sffs[]{ {"LuauSubtypeNormalizer", true}, {"LuauTypeNormalization2", true}, }; @@ -612,12 +603,13 @@ TEST_CASE_FIXTURE(Fixture, "union_of_functions_mentioning_generics") )"); LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK_EQ(toString(result.errors[0]), "Type '(a) -> a?' could not be converted into '((b) -> b) | ((b?) -> nil)'; none of the union options are compatible"); + CHECK_EQ(toString(result.errors[0]), + "Type '(a) -> a?' could not be converted into '((b) -> b) | ((b?) -> nil)'; none of the union options are compatible"); } TEST_CASE_FIXTURE(Fixture, "union_of_functions_mentioning_generic_typepacks") { - ScopedFastFlag sffs[] { + ScopedFastFlag sffs[]{ {"LuauSubtypeNormalizer", true}, {"LuauTypeNormalization2", true}, }; @@ -631,12 +623,13 @@ TEST_CASE_FIXTURE(Fixture, "union_of_functions_mentioning_generic_typepacks") )"); LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK_EQ(toString(result.errors[0]), "Type '(number, a...) -> (number?, a...)' could not be converted into '((number) -> number) | ((number?, a...) -> (number?, a...))'; none of the union options are compatible"); + CHECK_EQ(toString(result.errors[0]), "Type '(number, a...) -> (number?, a...)' could not be converted into '((number) -> number) | ((number?, " + "a...) -> (number?, a...))'; none of the union options are compatible"); } TEST_CASE_FIXTURE(Fixture, "union_of_functions_with_mismatching_arg_arities") { - ScopedFastFlag sffs[] { + ScopedFastFlag sffs[]{ {"LuauSubtypeNormalizer", true}, {"LuauTypeNormalization2", true}, }; @@ -648,12 +641,13 @@ TEST_CASE_FIXTURE(Fixture, "union_of_functions_with_mismatching_arg_arities") )"); LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK_EQ(toString(result.errors[0]), "Type '(number) -> number?' could not be converted into '((number) -> nil) | ((number, string?) -> number)'; none of the union options are compatible"); + CHECK_EQ(toString(result.errors[0]), "Type '(number) -> number?' could not be converted into '((number) -> nil) | ((number, string?) -> " + "number)'; none of the union options are compatible"); } TEST_CASE_FIXTURE(Fixture, "union_of_functions_with_mismatching_result_arities") { - ScopedFastFlag sffs[] { + ScopedFastFlag sffs[]{ {"LuauSubtypeNormalizer", true}, {"LuauTypeNormalization2", true}, }; @@ -665,12 +659,13 @@ TEST_CASE_FIXTURE(Fixture, "union_of_functions_with_mismatching_result_arities") )"); LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK_EQ(toString(result.errors[0]), "Type '() -> number | string' could not be converted into '(() -> (string, string)) | (() -> number)'; none of the union options are compatible"); + CHECK_EQ(toString(result.errors[0]), "Type '() -> number | string' could not be converted into '(() -> (string, string)) | (() -> number)'; none " + "of the union options are compatible"); } TEST_CASE_FIXTURE(Fixture, "union_of_functions_with_variadics") { - ScopedFastFlag sffs[] { + ScopedFastFlag sffs[]{ {"LuauSubtypeNormalizer", true}, {"LuauTypeNormalization2", true}, }; @@ -682,12 +677,13 @@ TEST_CASE_FIXTURE(Fixture, "union_of_functions_with_variadics") )"); LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK_EQ(toString(result.errors[0]), "Type '(...nil) -> (...number?)' could not be converted into '((...string?) -> (...number)) | ((...string?) -> nil)'; none of the union options are compatible"); + CHECK_EQ(toString(result.errors[0]), "Type '(...nil) -> (...number?)' could not be converted into '((...string?) -> (...number)) | ((...string?) " + "-> nil)'; none of the union options are compatible"); } TEST_CASE_FIXTURE(Fixture, "union_of_functions_with_mismatching_arg_variadics") { - ScopedFastFlag sffs[] { + ScopedFastFlag sffs[]{ {"LuauSubtypeNormalizer", true}, {"LuauTypeNormalization2", true}, }; @@ -699,12 +695,13 @@ TEST_CASE_FIXTURE(Fixture, "union_of_functions_with_mismatching_arg_variadics") )"); LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK_EQ(toString(result.errors[0]), "Type '(number) -> ()' could not be converted into '((...number?) -> ()) | ((number?) -> ())'; none of the union options are compatible"); + CHECK_EQ(toString(result.errors[0]), + "Type '(number) -> ()' could not be converted into '((...number?) -> ()) | ((number?) -> ())'; none of the union options are compatible"); } TEST_CASE_FIXTURE(Fixture, "union_of_functions_with_mismatching_result_variadics") { - ScopedFastFlag sffs[] { + ScopedFastFlag sffs[]{ {"LuauSubtypeNormalizer", true}, {"LuauTypeNormalization2", true}, }; @@ -716,7 +713,8 @@ TEST_CASE_FIXTURE(Fixture, "union_of_functions_with_mismatching_result_variadics )"); LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK_EQ(toString(result.errors[0]), "Type '() -> (number?, ...number)' could not be converted into '(() -> (...number)) | (() -> number)'; none of the union options are compatible"); + CHECK_EQ(toString(result.errors[0]), "Type '() -> (number?, ...number)' could not be converted into '(() -> (...number)) | (() -> number)'; none " + "of the union options are compatible"); } TEST_SUITE_END(); diff --git a/tests/conformance/basic.lua b/tests/conformance/basic.lua index b09b087b..b7e85aa7 100644 --- a/tests/conformance/basic.lua +++ b/tests/conformance/basic.lua @@ -93,6 +93,7 @@ assert((function() local a = 1 a = a * 2 return a end)() == 2) assert((function() local a = 1 a = a / 2 return a end)() == 0.5) assert((function() local a = 5 a = a % 2 return a end)() == 1) assert((function() local a = 3 a = a ^ 2 return a end)() == 9) +assert((function() local a = 9 a = a ^ 0.5 return a end)() == 3) assert((function() local a = '1' a = a .. '2' return a end)() == "12") assert((function() local a = '1' a = a .. '2' .. '3' return a end)() == "123") @@ -475,6 +476,12 @@ assert(rawequal("a", "a") == true) assert(rawequal("a", "b") == false) assert((function() a = {} b = {} mt = { __eq = function(l, r) return #l == #r end } setmetatable(a, mt) setmetatable(b, mt) return concat(a == b, rawequal(a, b)) end)() == "true,false") +-- rawequal fallback +assert(concat(pcall(rawequal, "a", "a")) == "true,true") +assert(concat(pcall(rawequal, "a", "b")) == "true,false") +assert(concat(pcall(rawequal, "a", nil)) == "true,false") +assert(pcall(rawequal, "a") == false) + -- metatable ops local function vec3t(x, y, z) return setmetatable({x=x, y=y, z=z}, { diff --git a/tests/conformance/bitwise.lua b/tests/conformance/bitwise.lua index f0c5698d..3b117892 100644 --- a/tests/conformance/bitwise.lua +++ b/tests/conformance/bitwise.lua @@ -71,6 +71,7 @@ for _, b in pairs(c) do assert(bit32.bxor(b) == b) assert(bit32.bxor(b, b) == 0) assert(bit32.bxor(b, 0) == b) + assert(bit32.bxor(b, b, b) == b) assert(bit32.bnot(b) ~= b) assert(bit32.bnot(bit32.bnot(b)) == b) assert(bit32.bnot(b) == 2^32 - 1 - b) @@ -104,6 +105,9 @@ assert(bit32.extract(0xa0001111, 16) == 0) assert(bit32.extract(0xa0001111, 31) == 1) assert(bit32.extract(42, 1, 3) == 5) +local pos pos = 1 +assert(bit32.extract(42, pos, 3) == 5) -- test bit32.extract builtin instead of bit32.extractk + assert(not pcall(bit32.extract, 0, -1)) assert(not pcall(bit32.extract, 0, 32)) assert(not pcall(bit32.extract, 0, 0, 33)) @@ -144,13 +148,17 @@ assert(bit32.lrotate("0x12345678", 4) == 0x23456781) assert(bit32.rrotate("0x12345678", -4) == 0x23456781) assert(bit32.arshift("0x12345678", 1) == 0x12345678 / 2) assert(bit32.arshift("-1", 32) == 0xffffffff) +assert(bit32.arshift("-1", 1) == 0xffffffff) assert(bit32.bnot("1") == 0xfffffffe) assert(bit32.band("1", 3) == 1) assert(bit32.band(1, "3") == 1) +assert(bit32.band(1, 3, "5") == 1) assert(bit32.bor("1", 2) == 3) assert(bit32.bor(1, "2") == 3) +assert(bit32.bor(1, 3, "5") == 7) assert(bit32.bxor("1", 3) == 2) assert(bit32.bxor(1, "3") == 2) +assert(bit32.bxor(1, 3, "5") == 7) assert(bit32.btest(1, "3") == true) assert(bit32.btest("1", 3) == true) assert(bit32.countlz("42") == 26) diff --git a/tests/conformance/debugger.lua b/tests/conformance/debugger.lua index ec0b412e..c773013b 100644 --- a/tests/conformance/debugger.lua +++ b/tests/conformance/debugger.lua @@ -54,4 +54,19 @@ breakpoint(49, false) -- validate that disabling breakpoints works bar() +local function breakpointSetFromMetamethod() + local a = setmetatable({}, { + __index = function() + breakpoint(67) + return 2 + end + }) + + local b = a.x + + assert(b == 2) +end + +breakpointSetFromMetamethod() + return 'OK' diff --git a/tests/conformance/events.lua b/tests/conformance/events.lua index 0c6055da..447b67bc 100644 --- a/tests/conformance/events.lua +++ b/tests/conformance/events.lua @@ -4,20 +4,6 @@ print('testing metatables') local unpack = table.unpack -X = 20; B = 30 - -local _G = getfenv() -setfenv(1, setmetatable({}, {__index=_G})) - -collectgarbage() - -X = X+10 -assert(X == 30 and _G.X == 20) -B = false -assert(B == false) -B = nil -assert(B == 30) - assert(getmetatable{} == nil) assert(getmetatable(4) == nil) assert(getmetatable(nil) == nil) @@ -299,14 +285,8 @@ x = c(3,4,5) assert(i == 3 and x[1] == 3 and x[3] == 5) -assert(_G.X == 20) -assert(_G == getfenv(0)) - print'+' -local _g = _G -setfenv(1, setmetatable({}, {__index=function (_,k) return _g[k] end})) - -- testing proxies assert(getmetatable(newproxy()) == nil) assert(getmetatable(newproxy(false)) == nil) @@ -480,4 +460,23 @@ do end end +function testfenv() + X = 20; B = 30 + + local _G = getfenv() + setfenv(1, setmetatable({}, {__index=_G})) + + X = X+10 + assert(X == 30 and _G.X == 20) + B = false + assert(B == false) + B = nil + assert(B == 30) + + assert(_G.X == 20) + assert(_G == getfenv(0)) +end + +testfenv() -- DONT MOVE THIS LINE + return 'OK' diff --git a/tests/conformance/interrupt.lua b/tests/conformance/interrupt.lua index 2b127099..d4b7c80a 100644 --- a/tests/conformance/interrupt.lua +++ b/tests/conformance/interrupt.lua @@ -8,4 +8,13 @@ end foo() +function bar() + local i = 0 + while i < 10 do + i += i + 1 + end +end + +bar() + return "OK" diff --git a/tests/conformance/math.lua b/tests/conformance/math.lua index 79ea0fb6..0cd0cdce 100644 --- a/tests/conformance/math.lua +++ b/tests/conformance/math.lua @@ -152,14 +152,34 @@ assert(eq(a[1000][3], 1000/3, 0.001)) print('+') do -- testing NaN - local NaN = 10e500 - 10e400 + local NaN -- to avoid constant folding + NaN = 10e500 - 10e400 + assert(NaN ~= NaN) + assert(not (NaN == NaN)) + assert(not (NaN < NaN)) assert(not (NaN <= NaN)) assert(not (NaN > NaN)) assert(not (NaN >= NaN)) + + assert(not (0 == NaN)) assert(not (0 < NaN)) + assert(not (0 <= NaN)) + assert(not (0 > NaN)) + assert(not (0 >= NaN)) + + assert(not (NaN == 0)) assert(not (NaN < 0)) + assert(not (NaN <= 0)) + assert(not (NaN > 0)) + assert(not (NaN >= 0)) + + assert(if NaN < 0 then false else true) + assert(if NaN <= 0 then false else true) + assert(if NaN > 0 then false else true) + assert(if NaN >= 0 then false else true) + local a = {} assert(not pcall(function () a[NaN] = 1 end)) assert(a[NaN] == nil) @@ -215,6 +235,16 @@ assert(flag); assert(select(2, pcall(math.random, 1, 2, 3)):match("wrong number of arguments")) +-- min/max +assert(math.min(1) == 1) +assert(math.min(1, 2) == 1) +assert(math.min(1, 2, -1) == -1) +assert(math.min(1, -1, 2) == -1) +assert(math.max(1) == 1) +assert(math.max(1, 2) == 2) +assert(math.max(1, 2, -1) == 2) +assert(math.max(1, -1, 2) == 2) + -- noise assert(math.noise(0.5) == 0) assert(math.noise(0.5, 0.5) == -0.25) @@ -277,8 +307,10 @@ assert(math.log("10", 10) == 1) assert(math.log("9", 3) == 2) assert(math.max("1", 2) == 2) assert(math.max(2, "1") == 2) +assert(math.max(1, 2, "3") == 3) assert(math.min("1", 2) == 1) assert(math.min(2, "1") == 1) +assert(math.min(1, 2, "3") == 1) local v,f = math.modf("1.5") assert(v == 1 and f == 0.5) assert(math.pow("2", 2) == 4) @@ -295,4 +327,9 @@ assert(math.sign("-2") == -1) assert(math.sign("0") == 0) assert(math.round("1.8") == 2) +-- test that fastcalls return correct number of results +assert(select('#', math.floor(1.4)) == 1) +assert(select('#', math.ceil(1.6)) == 1) +assert(select('#', math.sqrt(9)) == 1) + return('OK') diff --git a/tests/conformance/safeenv.lua b/tests/conformance/safeenv.lua new file mode 100644 index 00000000..3a430b5f --- /dev/null +++ b/tests/conformance/safeenv.lua @@ -0,0 +1,23 @@ +-- This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +print("safeenv reset") + +local function envChangeInMetamethod() + -- declare constant so that at O2 this test doesn't interfere with constant folding which we can't deoptimize + local ten + ten = 10 + + local a = setmetatable({}, { + __index = function() + getfenv().math = { abs = function(n) return n*n end } + return 2 + end + }) + + local b = a.x + + assert(math.abs(ten) == 100) +end + +envChangeInMetamethod() + +return"OK" diff --git a/tests/main.cpp b/tests/main.cpp index 3f564c07..82ce4e16 100644 --- a/tests/main.cpp +++ b/tests/main.cpp @@ -21,7 +21,6 @@ #include #endif -#include #include // Indicates if verbose output is enabled; can be overridden via --verbose @@ -31,8 +30,10 @@ bool verbose = false; // Default optimization level for conformance test; can be overridden via -On int optimizationLevel = 1; -// Something to seed a pseudorandom number generator with. Defaults to -// something from std::random_device. +// Run conformance tests with native code generation +bool codegen = false; + +// Something to seed a pseudorandom number generator with std::optional randomSeed; static bool skipFastFlag(const char* flagName) @@ -257,6 +258,11 @@ int main(int argc, char** argv) verbose = true; } + if (doctest::parseFlag(argc, argv, "--codegen")) + { + codegen = true; + } + int level = -1; if (doctest::parseIntOption(argc, argv, "-O", doctest::option_int, level)) { @@ -272,7 +278,7 @@ int main(int argc, char** argv) if (doctest::parseOption(argc, argv, "--randomize") && !randomSeed) { - randomSeed = std::random_device()(); + randomSeed = unsigned(time(nullptr)); printf("Using RNG seed %u\n", *randomSeed); } diff --git a/tools/faillist.txt b/tools/faillist.txt index 00e01011..0eb02209 100644 --- a/tools/faillist.txt +++ b/tools/faillist.txt @@ -48,7 +48,6 @@ BuiltinTests.assert_removes_falsy_types2 BuiltinTests.assert_removes_falsy_types_even_from_type_pack_tail_but_only_for_the_first_type BuiltinTests.assert_returns_false_and_string_iff_it_knows_the_first_argument_cannot_be_truthy BuiltinTests.bad_select_should_not_crash -BuiltinTests.coroutine_resume_anything_goes BuiltinTests.coroutine_wrap_anything_goes BuiltinTests.debug_info_is_crazy BuiltinTests.debug_traceback_is_crazy @@ -56,7 +55,6 @@ BuiltinTests.dont_add_definitions_to_persistent_types BuiltinTests.find_capture_types BuiltinTests.find_capture_types2 BuiltinTests.find_capture_types3 -BuiltinTests.global_singleton_types_are_sealed BuiltinTests.gmatch_capture_types BuiltinTests.gmatch_capture_types2 BuiltinTests.gmatch_capture_types_balanced_escaped_parens @@ -69,7 +67,6 @@ BuiltinTests.match_capture_types BuiltinTests.match_capture_types2 BuiltinTests.math_max_checks_for_numbers BuiltinTests.next_iterator_should_infer_types_and_type_check -BuiltinTests.os_time_takes_optional_date_table BuiltinTests.pairs_iterator_should_infer_types_and_type_check BuiltinTests.see_thru_select BuiltinTests.select_slightly_out_of_range @@ -77,7 +74,6 @@ BuiltinTests.select_way_out_of_range BuiltinTests.select_with_decimal_argument_is_rounded_down BuiltinTests.set_metatable_needs_arguments BuiltinTests.setmetatable_should_not_mutate_persisted_types -BuiltinTests.sort BuiltinTests.sort_with_bad_predicate BuiltinTests.sort_with_predicate BuiltinTests.string_format_arg_count_mismatch @@ -88,8 +84,6 @@ BuiltinTests.string_format_report_all_type_errors_at_correct_positions BuiltinTests.string_format_use_correct_argument BuiltinTests.string_format_use_correct_argument2 BuiltinTests.string_format_use_correct_argument3 -BuiltinTests.string_lib_self_noself -BuiltinTests.table_concat_returns_string BuiltinTests.table_freeze_is_generic BuiltinTests.table_insert_correctly_infers_type_of_array_2_args_overload BuiltinTests.table_insert_correctly_infers_type_of_array_3_args_overload @@ -101,12 +95,10 @@ BuiltinTests.tonumber_returns_optional_number_type2 DefinitionTests.class_definition_overload_metamethods DefinitionTests.declaring_generic_functions DefinitionTests.definition_file_classes -FrontendTest.automatically_check_dependent_scripts FrontendTest.environments FrontendTest.imported_table_modification_2 FrontendTest.it_should_be_safe_to_stringify_errors_when_full_type_graph_is_discarded FrontendTest.nocheck_cycle_used_by_checked -FrontendTest.recheck_if_dependent_script_is_dirty FrontendTest.reexport_cyclic_type FrontendTest.reexport_type_alias FrontendTest.trace_requires_in_nonstrict_mode @@ -132,18 +124,15 @@ GenericsTests.generic_type_pack_parentheses GenericsTests.generic_type_pack_unification1 GenericsTests.generic_type_pack_unification2 GenericsTests.generic_type_pack_unification3 +GenericsTests.higher_rank_polymorphism_should_not_accept_instantiated_arguments GenericsTests.infer_generic_function_function_argument GenericsTests.infer_generic_function_function_argument_overloaded GenericsTests.infer_generic_methods GenericsTests.inferred_local_vars_can_be_polytypes GenericsTests.instantiate_cyclic_generic_function -GenericsTests.instantiate_generic_function_in_assignments -GenericsTests.instantiate_generic_function_in_assignments2 GenericsTests.instantiated_function_argument_names GenericsTests.instantiation_sharing_types -GenericsTests.local_vars_can_be_instantiated_polytypes GenericsTests.no_stack_overflow_from_quantifying -GenericsTests.properties_can_be_instantiated_polytypes GenericsTests.reject_clashing_generic_and_pack_names GenericsTests.self_recursive_instantiated_param IntersectionTypes.index_on_an_intersection_type_with_mixed_types @@ -155,8 +144,6 @@ IntersectionTypes.should_still_pick_an_overload_whose_arguments_are_unions IntersectionTypes.table_intersection_write_sealed IntersectionTypes.table_intersection_write_sealed_indirect IntersectionTypes.table_write_sealed_indirect -isSubtype.intersection_of_tables -isSubtype.table_with_table_prop ModuleTests.any_persistance_does_not_leak ModuleTests.clone_self_property ModuleTests.deepClone_cyclic_table @@ -172,51 +159,24 @@ NonstrictModeTests.local_tables_are_not_any NonstrictModeTests.locals_are_any_by_default NonstrictModeTests.offer_a_hint_if_you_use_a_dot_instead_of_a_colon NonstrictModeTests.parameters_having_type_any_are_optional -NonstrictModeTests.returning_insufficient_return_values -NonstrictModeTests.returning_too_many_values NonstrictModeTests.table_dot_insert_and_recursive_calls NonstrictModeTests.table_props_are_any -Normalize.any_wins_the_battle_over_unknown_in_unions -Normalize.constrained_intersection_of_intersections -Normalize.cyclic_intersection Normalize.cyclic_table_normalizes_sensibly -Normalize.cyclic_union -Normalize.fuzz_failure_bound_type_is_normal_but_not_its_bounded_to Normalize.intersection_combine_on_bound_self -Normalize.intersection_inside_a_table_inside_another_intersection -Normalize.intersection_inside_a_table_inside_another_intersection_2 -Normalize.intersection_inside_a_table_inside_another_intersection_3 -Normalize.intersection_inside_a_table_inside_another_intersection_4 -Normalize.intersection_of_confluent_overlapping_tables -Normalize.intersection_of_disjoint_tables -Normalize.intersection_of_functions -Normalize.intersection_of_overlapping_tables -Normalize.intersection_of_tables_with_indexers -Normalize.normalization_does_not_convert_ever -Normalize.normalize_module_return_type -Normalize.normalize_unions_containing_never -Normalize.normalize_unions_containing_unknown -Normalize.union_of_distinct_free_types -Normalize.variadic_tail_is_marked_normal -Normalize.visiting_a_type_twice_is_not_considered_normal ParseErrorRecovery.generic_type_list_recovery ParseErrorRecovery.recovery_of_parenthesized_expressions ParserTests.parse_nesting_based_end_detection_failsafe_earlier ParserTests.parse_nesting_based_end_detection_local_function ProvisionalTests.bail_early_if_unification_is_too_complicated -ProvisionalTests.choose_the_right_overload_for_pcall -ProvisionalTests.constrained_is_level_dependent ProvisionalTests.discriminate_from_x_not_equal_to_nil ProvisionalTests.do_not_ice_when_trying_to_pick_first_of_generic_type_pack ProvisionalTests.error_on_eq_metamethod_returning_a_type_other_than_boolean -ProvisionalTests.function_returns_many_things_but_first_of_it_is_forgotten +ProvisionalTests.generic_type_leak_to_module_interface_variadic ProvisionalTests.greedy_inference_with_shared_self_triggers_function_with_no_returns -ProvisionalTests.invariant_table_properties_means_instantiating_tables_in_call_is_unsound -ProvisionalTests.it_should_be_agnostic_of_actual_size -ProvisionalTests.lower_bounds_calculation_is_too_permissive_with_overloaded_higher_order_functions -ProvisionalTests.normalization_fails_on_certain_kinds_of_cyclic_tables ProvisionalTests.pcall_returns_at_least_two_value_but_function_returns_nothing ProvisionalTests.setmetatable_constrains_free_type_into_free_table +ProvisionalTests.specialization_binds_with_prototypes_too_early +ProvisionalTests.table_insert_with_a_singleton_argument ProvisionalTests.typeguard_inference_incomplete ProvisionalTests.weirditer_should_not_loop_forever ProvisionalTests.while_body_are_also_refined @@ -225,7 +185,6 @@ RefinementTest.apply_refinements_on_astexprindexexpr_whose_subscript_expr_is_con RefinementTest.assert_a_to_be_truthy_then_assert_a_to_be_number RefinementTest.assert_non_binary_expressions_actually_resolve_constraints RefinementTest.call_a_more_specific_function_using_typeguard -RefinementTest.correctly_lookup_a_shadowed_local_that_which_was_previously_refined RefinementTest.correctly_lookup_property_whose_base_was_previously_refined RefinementTest.correctly_lookup_property_whose_base_was_previously_refined2 RefinementTest.discriminate_from_isa_of_x @@ -311,6 +270,7 @@ TableTests.found_like_key_in_table_function_call TableTests.found_like_key_in_table_property_access TableTests.found_multiple_like_keys TableTests.function_calls_produces_sealed_table_given_unsealed_table +TableTests.generic_table_instantiation_potential_regression TableTests.getmetatable_returns_pointer_to_metatable TableTests.give_up_after_one_metatable_index_look_up TableTests.hide_table_error_properties @@ -323,6 +283,7 @@ TableTests.infer_indexer_from_value_property_in_literal TableTests.inferred_return_type_of_free_table TableTests.inferring_crazy_table_should_also_be_quick TableTests.instantiate_table_cloning_3 +TableTests.invariant_table_properties_means_instantiating_tables_in_call_is_unsound TableTests.leaking_bad_metatable_errors TableTests.length_operator_union_errors TableTests.less_exponential_blowup_please @@ -340,7 +301,6 @@ TableTests.oop_polymorphic TableTests.open_table_unification_2 TableTests.pass_a_union_of_tables_to_a_function_that_requires_a_table TableTests.pass_a_union_of_tables_to_a_function_that_requires_a_table_2 -TableTests.pass_incompatible_union_to_a_generic_table_without_crashing TableTests.persistent_sealed_table_is_immutable TableTests.prop_access_on_key_whose_types_mismatches TableTests.property_lookup_through_tabletypevar_metatable @@ -378,7 +338,6 @@ ToDot.function ToDot.table ToString.exhaustive_toString_of_cyclic_table ToString.function_type_with_argument_names_generic -ToString.no_parentheses_around_cyclic_function_type_in_union ToString.toStringDetailed2 ToString.toStringErrorPack ToString.toStringNamedFunction_generic_pack @@ -412,12 +371,12 @@ TypeAliases.type_alias_local_rename TypeAliases.type_alias_of_an_imported_recursive_generic_type TypeAliases.type_alias_of_an_imported_recursive_type TypeInfer.checking_should_not_ice +TypeInfer.cli_50041_committing_txnlog_in_apollo_client_error TypeInfer.dont_report_type_errors_within_an_AstExprError TypeInfer.dont_report_type_errors_within_an_AstStatError TypeInfer.globals TypeInfer.globals2 TypeInfer.infer_assignment_value_types_mutable_lval -TypeInfer.it_is_ok_to_have_inconsistent_number_of_return_values_in_nonstrict TypeInfer.no_stack_overflow_from_isoptional TypeInfer.tc_after_error_recovery_no_replacement_name_in_error TypeInfer.tc_if_else_expressions_expected_type_3 @@ -427,7 +386,7 @@ TypeInfer.tc_interpolated_string_with_invalid_expression TypeInfer.type_infer_recursion_limit_no_ice TypeInferAnyError.assign_prop_to_table_by_calling_any_yields_any TypeInferAnyError.for_in_loop_iterator_is_any2 -TypeInferAnyError.union_of_types_regression_test +TypeInferAnyError.for_in_loop_iterator_is_error2 TypeInferClasses.call_base_method TypeInferClasses.call_instance_method TypeInferClasses.can_assign_to_prop_of_base_class_using_string @@ -441,7 +400,6 @@ TypeInferClasses.optional_class_field_access_error TypeInferClasses.table_class_unification_reports_sane_errors_for_missing_properties TypeInferClasses.warn_when_prop_almost_matches TypeInferClasses.we_can_report_when_someone_is_trying_to_use_a_table_rather_than_a_class -TypeInferFunctions.call_o_with_another_argument_after_foo_was_quantified TypeInferFunctions.calling_function_with_anytypepack_doesnt_leak_free_types TypeInferFunctions.calling_function_with_incorrect_argument_type_yields_errors_spanning_argument TypeInferFunctions.dont_give_other_overloads_message_if_only_one_argument_matching_overload_exists @@ -454,26 +412,20 @@ TypeInferFunctions.function_decl_non_self_sealed_overwrite_2 TypeInferFunctions.function_decl_non_self_unsealed_overwrite TypeInferFunctions.function_does_not_return_enough_values TypeInferFunctions.function_statement_sealed_table_assignment_through_indexer -TypeInferFunctions.ignored_return_values TypeInferFunctions.improved_function_arg_mismatch_error_nonstrict TypeInferFunctions.improved_function_arg_mismatch_errors -TypeInferFunctions.inconsistent_higher_order_function -TypeInferFunctions.inconsistent_return_types TypeInferFunctions.infer_anonymous_function_arguments TypeInferFunctions.infer_return_type_from_selected_overload TypeInferFunctions.infer_return_value_type TypeInferFunctions.infer_that_function_does_not_return_a_table -TypeInferFunctions.it_is_ok_not_to_supply_enough_retvals TypeInferFunctions.list_all_overloads_if_no_overload_takes_given_argument_count TypeInferFunctions.list_only_alternative_overloads_that_match_argument_count TypeInferFunctions.no_lossy_function_type TypeInferFunctions.occurs_check_failure_in_function_return_type -TypeInferFunctions.quantify_constrained_types TypeInferFunctions.record_matching_overload TypeInferFunctions.report_exiting_without_return_nonstrict TypeInferFunctions.report_exiting_without_return_strict TypeInferFunctions.return_type_by_overload -TypeInferFunctions.strict_mode_ok_with_missing_arguments TypeInferFunctions.too_few_arguments_variadic TypeInferFunctions.too_few_arguments_variadic_generic TypeInferFunctions.too_few_arguments_variadic_generic2 @@ -489,14 +441,13 @@ TypeInferLoops.for_in_with_generic_next TypeInferLoops.for_in_with_just_one_iterator_is_ok TypeInferLoops.loop_iter_no_indexer_nonstrict TypeInferLoops.loop_iter_trailing_nil -TypeInferLoops.loop_typecheck_crash_on_empty_optional TypeInferLoops.unreachable_code_after_infinite_loop TypeInferLoops.varlist_declared_by_for_in_loop_should_be_free +TypeInferModules.bound_free_table_export_is_ok TypeInferModules.custom_require_global TypeInferModules.do_not_modify_imported_types TypeInferModules.do_not_modify_imported_types_2 TypeInferModules.do_not_modify_imported_types_3 -TypeInferModules.general_require_type_mismatch TypeInferModules.module_type_conflict TypeInferModules.module_type_conflict_instantiated TypeInferModules.require_a_variadic_function @@ -531,7 +482,6 @@ TypeInferOperators.expected_types_through_binary_or TypeInferOperators.infer_any_in_all_modes_when_lhs_is_unknown TypeInferOperators.or_joins_types TypeInferOperators.or_joins_types_with_no_extras -TypeInferOperators.primitive_arith_no_metatable TypeInferOperators.primitive_arith_possible_metatable TypeInferOperators.produce_the_correct_error_message_when_comparing_a_table_with_a_metatable_with_one_that_does_not TypeInferOperators.refine_and_or @@ -543,7 +493,6 @@ TypeInferOperators.typecheck_overloaded_multiply_that_is_an_intersection_on_rhs TypeInferOperators.typecheck_unary_len_error TypeInferOperators.typecheck_unary_minus TypeInferOperators.typecheck_unary_minus_error -TypeInferOperators.unary_not_is_boolean TypeInferOperators.UnknownGlobalCompoundAssign TypeInferPrimitives.CheckMethodsOfNumber TypeInferPrimitives.singleton_types @@ -563,8 +512,6 @@ TypeInferUnknownNever.type_packs_containing_never_is_itself_uninhabitable TypeInferUnknownNever.type_packs_containing_never_is_itself_uninhabitable2 TypeInferUnknownNever.unary_minus_of_never TypePackTests.higher_order_function -TypePackTests.multiple_varargs_inference_are_not_confused -TypePackTests.no_return_size_should_be_zero TypePackTests.pack_tail_unification_check TypePackTests.parenthesized_varargs_returns_any TypePackTests.type_alias_backwards_compatible @@ -606,7 +553,6 @@ TypeSingletons.string_singleton_subtype TypeSingletons.string_singletons TypeSingletons.string_singletons_escape_chars TypeSingletons.string_singletons_mismatch -TypeSingletons.table_insert_with_a_singleton_argument TypeSingletons.table_properties_type_error_escapes TypeSingletons.tagged_unions_using_singletons TypeSingletons.taking_the_length_of_string_singleton @@ -616,14 +562,12 @@ TypeSingletons.widening_happens_almost_everywhere TypeSingletons.widening_happens_almost_everywhere_except_for_tables UnionTypes.error_detailed_optional UnionTypes.error_detailed_union_all -UnionTypes.error_takes_optional_arguments UnionTypes.index_on_a_union_type_with_missing_property UnionTypes.index_on_a_union_type_with_mixed_types UnionTypes.index_on_a_union_type_with_one_optional_property UnionTypes.index_on_a_union_type_with_one_property_of_type_any UnionTypes.index_on_a_union_type_with_property_guaranteed_to_exist UnionTypes.index_on_a_union_type_works_at_arbitrary_depth -UnionTypes.optional_arguments UnionTypes.optional_assignment_errors UnionTypes.optional_call_error UnionTypes.optional_field_access_error diff --git a/tools/lvmexecute_split.py b/tools/lvmexecute_split.py index 10e3ccbb..48d66cb0 100644 --- a/tools/lvmexecute_split.py +++ b/tools/lvmexecute_split.py @@ -43,16 +43,23 @@ for line in input: if match: inst = match[1] - signature = "const Instruction* execute_" + inst + "(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k)" + signature = "const Instruction* execute_" + inst + "(lua_State* L, const Instruction* pc, StkId base, TValue* k)" header += signature + ";\n" function = signature + "\n" + function += "{\n" + function += " [[maybe_unused]] Closure* cl = clvalue(L->ci->func);\n" state = 1 - # find the end of an instruction + # first line of the instruction which is "{" elif state == 1: + assert(line == " {\n") + state = 2 + + # find the end of an instruction + elif state == 2: # remove jumps back into the native code if line == "#if LUA_CUSTOM_EXECUTION\n": - state = 2 + state = 3 continue if line[0] == ' ': @@ -70,7 +77,7 @@ for line in input: if match: # break is not supported if inst == "LOP_BREAK": - function = "const Instruction* execute_" + inst + "(lua_State* L, const Instruction* pc, Closure* cl, StkId base, TValue* k)\n" + function = "const Instruction* execute_" + inst + "(lua_State* L, const Instruction* pc, StkId base, TValue* k)\n" function += "{\n LUAU_ASSERT(!\"Unsupported deprecated opcode\");\n LUAU_UNREACHABLE();\n}\n" # handle fallthrough elif inst == "LOP_NAMECALL": @@ -81,14 +88,14 @@ for line in input: state = 0 # skip LUA_CUSTOM_EXECUTION code blocks - elif state == 2: + elif state == 3: if line == "#endif\n": - state = 3 + state = 4 continue # skip extra line - elif state == 3: - state = 1 + elif state == 4: + state = 2 # make sure we found the ending assert(state == 0) diff --git a/tools/test_dcr.py b/tools/test_dcr.py index 5f1c8705..76bf11ac 100644 --- a/tools/test_dcr.py +++ b/tools/test_dcr.py @@ -5,6 +5,9 @@ import os.path import subprocess as sp import sys import xml.sax as x +import colorama as c + +c.init() SCRIPT_PATH = os.path.split(sys.argv[0])[0] FAIL_LIST_PATH = os.path.join(SCRIPT_PATH, "faillist.txt") @@ -35,6 +38,10 @@ class Handler(x.ContentHandler): self.numSkippedTests = 0 + self.pass_count = 0 + self.fail_count = 0 + self.test_count = 0 + def startElement(self, name, attrs): if name == "TestSuite": self.currentTest.append(attrs["name"]) @@ -53,6 +60,12 @@ class Handler(x.ContentHandler): r = self.results.get(dottedName, True) self.results[dottedName] = r and passed + self.test_count += 1 + if passed: + self.pass_count += 1 + else: + self.fail_count += 1 + elif name == "OverallResultsTestCases": self.numSkippedTests = safeParseInt(attrs.get("skipped", 0)) @@ -137,11 +150,33 @@ def main(): p.wait() + unexpected_fails = 0 + unexpected_passes = 0 + for testName, passed in handler.results.items(): if passed and testName in failList: - print_stderr(f"UNEXPECTED: {testName} should have failed") + unexpected_passes += 1 + print_stderr( + f"UNEXPECTED: {c.Fore.RED}{testName}{c.Fore.RESET} should have failed" + ) elif not passed and testName not in failList: - print_stderr(f"UNEXPECTED: {testName} should have passed") + unexpected_fails += 1 + print_stderr( + f"UNEXPECTED: {c.Fore.GREEN}{testName}{c.Fore.RESET} should have passed" + ) + + if unexpected_fails or unexpected_passes: + print_stderr("") + print_stderr(f"Unexpected fails: {unexpected_fails}") + print_stderr(f"Unexpected passes: {unexpected_passes}") + + pass_percent = int(handler.pass_count / handler.test_count * 100) + + print_stderr("") + print_stderr( + f"{handler.pass_count} of {handler.test_count} tests passed. ({pass_percent}%)" + ) + print_stderr(f"{handler.fail_count} tests failed.") if args.write: newFailList = sorted( From c6a2d7519382e89261cdc29dcb18dffab417502b Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Mon, 17 Oct 2022 10:02:21 -0700 Subject: [PATCH 375/399] Improve testing coverage with codegen/O2 in mind (#709) This change adds codegen runs to coverage config and adds O2/codegen testing to CI. Note that we don't run O2 combinations in coverage - it's better that we see gaps in O2 coverage in compiler tests, as these are valuable for validating codegen intricacies that are difficult to see from conformance tests passing/failing. --- .github/workflows/build.yml | 38 ++++++++++++++++++++++++++----------- Makefile | 13 +++++++++---- 2 files changed, 36 insertions(+), 15 deletions(-) diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml index 78c3a34d..dbd6a495 100644 --- a/.github/workflows/build.yml +++ b/.github/workflows/build.yml @@ -25,12 +25,21 @@ jobs: runs-on: ${{matrix.os}}-latest steps: - uses: actions/checkout@v1 - - name: make test + - name: make tests run: | - make -j2 config=sanitize werror=1 test - - name: make test w/flags + make -j2 config=sanitize werror=1 native=1 luau-tests + - name: run tests run: | - make -j2 config=sanitize werror=1 flags=true test + ./luau-tests + ./luau-tests --fflags=true + - name: run extra conformance tests + run: | + ./luau-tests -ts=Conformance -O2 + ./luau-tests -ts=Conformance -O2 --fflags=true + ./luau-tests -ts=Conformance --codegen + ./luau-tests -ts=Conformance --codegen --fflags=true + ./luau-tests -ts=Conformance --codegen -O2 + ./luau-tests -ts=Conformance --codegen -O2 --fflags=true - name: make cli run: | make -j2 config=sanitize werror=1 luau luau-analyze # match config with tests to improve build time @@ -45,18 +54,25 @@ jobs: steps: - uses: actions/checkout@v1 - name: cmake configure - run: cmake . -A ${{matrix.arch}} -DLUAU_WERROR=ON - - name: cmake test + run: cmake . -A ${{matrix.arch}} -DLUAU_WERROR=ON -DLUAU_NATIVE=ON + - name: cmake build + run: cmake --build . --target Luau.UnitTest Luau.Conformance --config Debug + - name: run tests shell: bash # necessary for fail-fast run: | - cmake --build . --target Luau.UnitTest Luau.Conformance --config Debug Debug/Luau.UnitTest.exe Debug/Luau.Conformance.exe - - name: cmake test w/flags - shell: bash # necessary for fail-fast - run: | Debug/Luau.UnitTest.exe --fflags=true Debug/Luau.Conformance.exe --fflags=true + - name: run extra conformance tests + shell: bash # necessary for fail-fast + run: | + Debug/Luau.Conformance.exe -O2 + Debug/Luau.Conformance.exe -O2 --fflags=true + Debug/Luau.Conformance.exe --codegen + Debug/Luau.Conformance.exe --codegen --fflags=true + Debug/Luau.Conformance.exe --codegen -O2 + Debug/Luau.Conformance.exe --codegen -O2 --fflags=true - name: cmake cli shell: bash # necessary for fail-fast run: | @@ -73,7 +89,7 @@ jobs: sudo apt install llvm - name: make coverage run: | - CXX=clang++-10 make -j2 config=coverage coverage + CXX=clang++-10 make -j2 config=coverage native=1 coverage - name: upload coverage uses: codecov/codecov-action@v3 with: diff --git a/Makefile b/Makefile index 2ac4b33a..48d399e8 100644 --- a/Makefile +++ b/Makefile @@ -148,11 +148,16 @@ clean: rm -rf $(EXECUTABLE_ALIASES) coverage: $(TESTS_TARGET) - $(TESTS_TARGET) --fflags=true - mv default.profraw default-flags.profraw $(TESTS_TARGET) - llvm-profdata merge default.profraw default-flags.profraw -o default.profdata - rm default.profraw default-flags.profraw + mv default.profraw tests.profraw + $(TESTS_TARGET) --fflags=true + mv default.profraw tests-flags.profraw + $(TESTS_TARGET) -ts=Conformance --codegen + mv default.profraw codegen.profraw + $(TESTS_TARGET) -ts=Conformance --codegen --fflags=true + mv default.profraw codegen-flags.profraw + llvm-profdata merge tests.profraw tests-flags.profraw codegen.profraw codegen-flags.profraw -o default.profdata + rm *.profraw llvm-cov show -format=html -show-instantiations=false -show-line-counts=true -show-region-summary=false -ignore-filename-regex=\(tests\|extern\|CLI\)/.* -output-dir=coverage --instr-profile default.profdata build/coverage/luau-tests llvm-cov report -ignore-filename-regex=\(tests\|extern\|CLI\)/.* -show-region-summary=false --instr-profile default.profdata build/coverage/luau-tests llvm-cov export -ignore-filename-regex=\(tests\|extern\|CLI\)/.* -format lcov --instr-profile default.profdata build/coverage/luau-tests >coverage.info From 49d6bc30adf35ea43d49b476b82bce9d0ad2db0b Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Mon, 17 Oct 2022 12:18:34 -0700 Subject: [PATCH 376/399] Add bench-codegen benchmark (#713) We will now run luau with --codegen during benchmark runs and collect the data into separate JSON. Note that we don't yet have the historical data for these, which will be backfilled later. --- .github/workflows/benchmark.yml | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) diff --git a/.github/workflows/benchmark.yml b/.github/workflows/benchmark.yml index 80a4ba26..4db40a62 100644 --- a/.github/workflows/benchmark.yml +++ b/.github/workflows/benchmark.yml @@ -32,6 +32,12 @@ jobs: CXX=g++ make config=profile luau cp luau luau-gcc + - name: Build Luau (codegen) + run: | + make config=profile clean + CXX=clang++ make config=profile native=1 luau + cp luau luau-codegen + - name: Build Luau (clang) run: | make config=profile clean @@ -45,6 +51,10 @@ jobs: run: | python bench/bench.py --callgrind --vm "./luau -O2" | tee -a bench-output.txt + - name: Run benchmark (bench-codegen) + run: | + python bench/bench.py --callgrind --vm "./luau-codegen --codegen -O2" | tee -a bench-codegen-output.txt + - name: Run benchmark (analyze) run: | filter() { @@ -83,6 +93,14 @@ jobs: output-file-path: ./bench-output.txt external-data-json-path: ./gh-pages/bench.json + - name: Store results (bench-codegen) + uses: Roblox/rhysd-github-action-benchmark@v-luau + with: + name: callgrind codegen + tool: "benchmarkluau" + output-file-path: ./bench-codegen-output.txt + external-data-json-path: ./gh-pages/bench-codegen.json + - name: Store results (bench-gcc) uses: Roblox/rhysd-github-action-benchmark@v-luau with: From edb445392441582b722026dbfabed86706d6750d Mon Sep 17 00:00:00 2001 From: JohnnyMorganz Date: Tue, 18 Oct 2022 18:15:26 +0100 Subject: [PATCH 377/399] Support `["prop"]` syntax on class definitions (#704) Some classes have properties which are not valid identifiers (such as https://create.roblox.com/docs/reference/engine/classes/Studio) This adds support for the following syntax in definition files: ```lua declare class Foo ["a property"]: string end ``` Closes #702 --- Ast/src/Parser.cpp | 19 +++++++++++++++++++ tests/TypeInfer.definitions.test.cpp | 17 +++++++++++++++++ 2 files changed, 36 insertions(+) diff --git a/Ast/src/Parser.cpp b/Ast/src/Parser.cpp index c20c0847..7150b18f 100644 --- a/Ast/src/Parser.cpp +++ b/Ast/src/Parser.cpp @@ -905,6 +905,25 @@ AstStat* Parser::parseDeclaration(const Location& start) { props.push_back(parseDeclaredClassMethod()); } + else if (lexer.current().type == '[') + { + const Lexeme begin = lexer.current(); + nextLexeme(); // [ + + std::optional> chars = parseCharArray(); + + expectMatchAndConsume(']', begin); + expectAndConsume(':', "property type annotation"); + AstType* type = parseTypeAnnotation(); + + // TODO: since AstName conains a char*, it can't contain null + bool containsNull = chars && (strnlen(chars->data, chars->size) < chars->size); + + if (chars && !containsNull) + props.push_back(AstDeclaredClassProp{AstName(chars->data), type, false}); + else + report(begin.location, "String literal contains malformed escape sequence"); + } else { Name propName = parseName("property name"); diff --git a/tests/TypeInfer.definitions.test.cpp b/tests/TypeInfer.definitions.test.cpp index 26280c13..15c63ec7 100644 --- a/tests/TypeInfer.definitions.test.cpp +++ b/tests/TypeInfer.definitions.test.cpp @@ -362,4 +362,21 @@ TEST_CASE_FIXTURE(Fixture, "class_definition_overload_metamethods") CHECK_EQ(toString(requireType("shouldBeVector")), "Vector3"); } +TEST_CASE_FIXTURE(Fixture, "class_definition_string_props") +{ + loadDefinition(R"( + declare class Foo + ["a property"]: string + end + )"); + + CheckResult result = check(R"( + local x: Foo + local y = x["a property"] + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + CHECK_EQ(toString(requireType("y")), "string"); +} + TEST_SUITE_END(); From ae5a011465ab81ccb5908862b262cc3988ffa6d1 Mon Sep 17 00:00:00 2001 From: Alan Jeffrey <403333+asajeffrey@users.noreply.github.com> Date: Tue, 18 Oct 2022 16:56:15 -0500 Subject: [PATCH 378/399] Add a blog post about semantic subtyping (#700) Co-authored-by: Alexander McCord <11488393+alexmccord@users.noreply.github.com> --- .../2022-10-31-luau-semantic-subtyping.md | 287 ++++++++++++++++++ 1 file changed, 287 insertions(+) create mode 100644 docs/_posts/2022-10-31-luau-semantic-subtyping.md diff --git a/docs/_posts/2022-10-31-luau-semantic-subtyping.md b/docs/_posts/2022-10-31-luau-semantic-subtyping.md new file mode 100644 index 00000000..65db643d --- /dev/null +++ b/docs/_posts/2022-10-31-luau-semantic-subtyping.md @@ -0,0 +1,287 @@ +--- +layout: single +title: "Semantic Subtyping in Luau" +author: Alan Jeffrey +--- + +Luau is the first programming language to put the power of semantic subtyping in the hands of millions of creators. + +## Minimizing false positives + +One of the issues with type error reporting in tools like the Script Analysis widget in Roblox Studio is *false positives*. These are warnings that are artifacts of the analysis, and don’t correspond to errors which can occur at runtime. For example, the program +```lua + local x = CFrame.new() + local y + if (math.random()) then + y = CFrame.new() + else + y = Vector3.new() + end + local z = x * y +``` +reports a type error which cannot happen at runtime, since `CFrame` supports multiplication by both `Vector3` and `CFrame`. (Its type is `((CFrame, CFrame) -> CFrame) & ((CFrame, Vector3) -> Vector3)`.) + +False positives are especially poor for onboarding new users. If a type-curious creator switches on typechecking and is immediately faced with a wall of spurious red squiggles, there is a strong incentive to immediately switch it off again. + +Inaccuracies in type errors are inevitable, since it is impossible to decide ahead of time whether a runtime error will be triggered. Type system designers have to choose whether to live with false positives or false negatives. In Luau this is determined by the mode: `strict` mode errs on the side of false positives, and `nonstrict` mode errs on the side of false negatives. + +While inaccuracies are inevitable, we try to remove them whenever possible, since they result in spurious errors, and imprecision in type-driven tooling like autocomplete or API documentation. + +## Subtyping as a source of false positives + +One of the sources of false positives in Luau (and many other similar languages like TypeScript or Flow) is *subtyping*. Subtyping is used whenever a variable is initialized or assigned to, and whenever a function is called: the type system checks that the type of the expression is a subtype of the type of the variable. For example, if we add types to the above program +```lua + local x : CFrame = CFrame.new() + local y : Vector3 | CFrame + if (math.random()) then + y = CFrame.new() + else + y = Vector3.new() + end + local z : Vector3 | CFrame = x * y +``` +then the type system checks that the type of `CFrame` multiplication is a subtype of `(CFrame, Vector3 | CFrame) -> (Vector3 | CFrame)`. + +Subtyping is a very useful feature, and it supports rich type constructs like type union (`T | U`) and intersection (`T & U`). For example, `number?` is implemented as a union type `(number | nil)`, inhabited by values that are either numbers or `nil`. + +Unfortunately, the interaction of subtyping with intersection and union types can have odd results. A simple (but rather artificial) case in older Luau was: +```lua + local x : (number?) & (string?) = nil + local y : nil = nil + y = x -- Type '(number?) & (string?)' could not be converted into 'nil' + x = y +``` +This error is caused by a failure of subtyping, the old subtyping algorithm reports that `(number?) & (string?)` is not a subtype of `nil`. This is a false positive, since `number & string` is uninhabited, so the only possible inhabitant of `(number?) & (string?)` is `nil`. + +This is an artificial example, but there are real issues raised by creators caused by the problems, for example . Currently, these issues mostly affect creators making use of sophisticated type system features, but as we make type inference more accurate, union and intersection types will become more common, even in code with no type annotations. + +This class of false positives no longer occurs in Luau, as we have moved from our old approach of *syntactic subtyping* to an alternative called *semantic subtyping*. + +## Syntactic subtyping + +AKA “what we did before.” + +Syntactic subtyping is a syntax-directed recursive algorithm. The interesting cases to deal with intersection and union types are: + +* Reflexivity: `T` is a subtype of `T` +* Intersection L: `(T₁ & … & Tⱼ)` is a subtype of `U` whenever some of the `Tᵢ` are subtypes of `U` +* Union L: `(T₁ | … | Tⱼ)` is a subtype of `U` whenever all of the `Tᵢ` are subtypes of `U` +* Intersection R: `T` is a subtype of `(U₁ & … & Uⱼ)` whenever `T` is a subtype of some of the `Uᵢ` +* Union R: `T` is a subtype of `(U₁ | … | Uⱼ)` whenever `T` is a subtype of all of the `Uᵢ`. + +For example: + +* By Reflexivity: `nil` is a subtype of `nil` +* so by Union R: `nil` is a subtype of `number?` +* and: `nil` is a subtype of `string?` +* so by Intersection R: `nil` is a subtype of `(number?) & (string?)`. + +Yay! Unfortunately, using these rules: + +* `number` isn’t a subtype of `nil` +* so by Union L: `(number?)` isn’t a subtype of `nil` +* and: `string` isn’t a subtype of `nil` +* so by Union L: `(string?)` isn’t a subtype of `nil` +* so by Intersection L: `(number?) & (string?)` isn’t a subtype of `nil`. + +This is typical of syntactic subtyping: when it returns a “yes” result, it is correct, but when it returns a “no” result, it might be wrong. The algorithm is a *conservative approximation*, and since a “no” result can lead to type errors, this is a source of false positives. + +## Semantic subtyping + +AKA “what we do now.” + +Rather than thinking of subtyping as being syntax-directed, we first consider its semantics, and later return to how the semantics is implemented. For this, we adopt semantic subtyping: + + * The semantics of a type is a set of values. + * Intersection types are thought of as intersections of sets. + * Union types are thought of as unions of sets. + * Subtyping is thought of as set inclusion. + +For example: + +| Type | Semantics | +|------|-----------| +| `number` | { 1, 2, 3, … } | +| `string` | { “foo”, “bar”, … } | +| `nil` | { nil } | +| `number?` | { nil, 1, 2, 3, … } | +| `string?` | { nil, “foo”, “bar”, … } | +| `(number?) & (string?)` | { nil, 1, 2, 3, … } ∩ { nil, “foo”, “bar”, … } = { nil } | + + +and since subtypes are interpreted as set inclusions: + +| Subtype | Supertype | Because | +|---------|-----------|---------| +| `nil` | `number?` | { nil } ⊆ { nil, 1, 2, 3, … } | +| `nil` | `string?`| { nil } ⊆ { nil, “foo”, “bar”, … } | +| `nil` | `(number?) & (string?)` | { nil } ⊆ { nil } | +| `(number?) & (string?)` | `nil` | { nil } ⊆ { nil } | + + +So according to semantic subtyping, `(number?) & (string?)` is equivalent to `nil`, but syntactic subtyping only supports one direction. + +This is all fine and good, but if we want to use semantic subtyping in tools, we need an algorithm, and it turns out checking semantic subtyping is non-trivial. + +## Semantic subtyping is hard + +NP-hard to be precise. + +We can reduce graph coloring to semantic subtyping by coding up a graph as a Luau type such that checking subtyping on types has the same result as checking for the impossibility of coloring the graph + +For example, coloring a three-node, two color graph can be done using types: + +```lua +type Red = "red" +type Blue = "blue" +type Color = Red | Blue +type Coloring = (Color) -> (Color) -> (Color) -> boolean +type Uncolorable = (Color) -> (Color) -> (Color) -> false +``` + +Then a graph can be encoded as an overload function type with +subtype `Uncolorable` and supertype `Coloring`, as an overloaded +function which returns `false` when a constraint is violated. Each +overload encodes one constraint. For example a line has constraints +saying that adjacent nodes cannot have the same color: + +```lua +type Line = Coloring + & ((Red) -> (Red) -> (Color) -> false) + & ((Blue) -> (Blue) -> (Color) -> false) + & ((Color) -> (Red) -> (Red) -> false) + & ((Color) -> (Blue) -> (Blue) -> false) +``` + +A triangle is similar, but the end points also cannot have the same color: + +```lua +type Triangle = Line + & ((Red) -> (Color) -> (Red) -> false) + & ((Blue) -> (Color) -> (Blue) -> false) +``` + +Now, `Triangle` is a subtype of `Uncolorable`, but `Line` is not, since the line can be 2-colored. +This can be generalized to any finite graph with any finite number of colors, and so subtype checking is NP-hard. + +We deal with this in two ways: + +* we cache types to reduce memory footprint, and +* give up with a “Code Too Complex” error if the cache of types gets too large. + +Hopefully this doesn’t come up in practice much. There is good evidence that issues like this don’t arise in practice from experience with type systems like that of Standard ML, which is [EXPTIME-complete](https://dl.acm.org/doi/abs/10.1145/96709.96748), but in practice you have to go out of your way to code up Turing Machine tapes as types. + +## Type normalization + +The algorithm used to decide semantic subtyping is *type normalization*. +Rather than being directed by syntax, we first rewrite types to be normalized, then check subtyping on normalized types. + +A normalized type is a union of: + +* a normalized nil type (either `never` or `nil`) +* a normalized number type (either `never` or `number`) +* a normalized boolean type (either `never` or `true` or `false` or `boolean`) +* a normalized function type (either `never` or an intersection of function types) +etc + +Once types are normalized, it is straightforward to check semantic subtyping. + +Every type can be normalized (sigh, with some technical restrictions around generic type packs). The important steps are: + +* removing intersections of mismatched primitives, e.g. `number & bool` is replaced by `never`, and +* removing unions of functions, e.g. `((number?) -> number) | ((string?) -> string)` is replaced by `(nil) -> (number | string)`. + +For example, normalizing `(number?) & (string?)` removes `number & string`, so all that is left is `nil`. + +Our first attempt at implementing type normalization applied it liberally, but this resulted in dreadful performance (complex code went from typechecking in less than a minute to running overnight). The reason for this is annoyingly simple: there is an optimization in Luau’s subtyping algorithm to handle reflexivity (`T` is a subtype of `T`) that performs a cheap pointer equality check. Type normalization can convert pointer-identical types into semantically-equivalent (but not pointer-identical) types, which significantly degrades performance. + +Because of these performance issues, we still use syntactic subtyping as our first check for subtyping, and only perform type normalization if the syntactic algorithm fails. This is sound, because syntactic subtyping is a conservative approximation to semantic subtyping. + +## Pragmatic semantic subtyping + +Off-the-shelf semantic subtyping is slightly different from what is implemented in Luau, because it requires models to be *set-theoretic*, which requires that inhabitants of function types “act like functions.” There are two reasons why we drop this requirement. + +**Firstly**, we normalize function types to an intersection of functions, for example a horrible mess of unions and intersections of functions: +``` +((number?) -> number?) | (((number) -> number) & ((string?) -> string?)) +``` +normalizes to an overloaded function: +``` +((number) -> number?) & ((nil) -> (number | string)?) +``` +Set-theoretic semantic subtyping does not support this normalization, and instead normalizes functions to *disjunctive normal form* (unions of intersections of functions). We do not do this for ergonomic reasons: overloaded functions are idiomatic in Luau, but DNF is not, and we do not want to present users with such non-idiomatic types. + +Our normalization relies on rewriting away unions of function types: +``` +((A) -> B) | ((C) -> D) → (A & C) -> (B | D) +``` +This normalization is sound in our model, but not in set-theoretic models. + +**Secondly**, in Luau, the type of a function application `f(x)` is `B` if `f` has type `(A) -> B` and `x` has type `A`. Unexpectedly, this is not always true in set-theoretic models, due to uninhabited types. In set-theoretic models, if `x` has type `never` then `f(x)` has type `never`. We do not want to burden users with the idea that function application has a special corner case, especially since that corner case can only arise in dead code. + +In set-theoretic models, `(never) -> A` is a subtype of `(never) -> B`, no matter what `A` and `B` are. This is not true in Luau. + +For these two reasons (which are largely about ergonomics rather than anything technical) we drop the set-theoretic requirement, and use *pragmatic* semantic subtyping. + +## Negation types + +The other difference between Luau’s type system and off-the-shelf semantic subtyping is that Luau does not support all negated types. + +The common case for wanting negated types is in typechecking conditionals: +```lua +-- initially x has type T +if (type(x) == "string") then + -- in this branch x has type T & string +else + -- in this branch x has type T & ~string +end +``` +This uses a negated type `~string` inhabited by values that are not strings. + +In Luau, we only allow this kind of typing refinement on *test types* like `string`, `function`, `Part` and so on, and *not* on structural types like `(A) -> B`, which avoids the common case of general negated types. + +## Prototyping and verification + +During the design of Luau’s semantic subtyping algorithm, there were changes made (for example initially we thought we were going to be able to use set-theoretic subtyping). During this time of rapid change, it was important to be able to iterate quickly, so we initially implemented a [prototype](https://github.com/luau-lang/agda-typeck) rather than jumping straight to a production implementation. + +Validating the prototype was important, since subtyping algorithms can have unexpected corner cases. For this reason, we adopted Agda as the prototyping language. As well as supporting unit testing, Agda supports mechanized verification, so we are confident in the design. + +The prototype does not implement all of Luau, just the functional subset, but this was enough to discover subtle feature interactions that would probably have surfaced as difficult-to-fix bugs in production. + +Prototyping is not perfect, for example the main issues that we hit in production were about performance and the C++ standard library, which are never going to be caught by a prototype. But the production implementation was otherwise fairly straightforward (or at least as straightforward as a 3kLOC change can be). + +## Next steps + +Semantic subtyping has removed one source of false positives, but we still have others to track down: + +* overloaded function applications and operators, +* property access on expressions of complex type, +* read-only properties of tables, +* variables that change type over time (aka typestates), +* … + +The quest to remove spurious red squiggles continues! + +## Further reading + +If you want to find out more about Luau and semantic subtyping, you might want to check out… + +* Luau. +* Lily Brown, Andy Friesen and Alan Jeffrey, *Goals of the Luau Type System*, Human Aspects of Types and Reasoning Assistants (HATRA), 2021. +* Luau Typechecker Prototype. +* Agda. +* Andrew M. Kent. *Down and Dirty with Semantic Set-theoretic Types*, 2021. +* Giuseppe Castagna, *Covariance and Contravariance*, Logical Methods in Computer Science 16(1), 2022. +* Giuseppe Castagna and Alain Frisch, *A gentle introduction to semantic subtyping*, Proc. Principles and practice of declarative programming (PPDP), pp 198–208, 2005. +* Giuseppe Castagna, Mickaël Laurent, Kim Nguyễn, Matthew Lutze, *On Type-Cases, Union Elimination, and Occurrence Typing*, Principles of Programming Languages (POPL), 2022. +* Sam Tobin-Hochstadt and Matthias Felleisen, *Logical types for untyped languages*. International Conference on Functional Programming (ICFP), 2010. https://doi.org/10.1145/1863543.1863561 +* José Valim, *My Future with Elixir: set-theoretic types*, 2022. https://elixir-lang.org/blog/2022/10/05/my-future-with-elixir-set-theoretic-types/ + +Some other languages which support semantic subtyping… + +* ℂDuce +* Ballerina +* Elixir +* eqWAlizer + +And if you want to see the production code, it's in the C++ definitions of [tryUnifyNormalizedTypes](https://github.com/Roblox/luau/blob/d6aa35583e4be14304d2a17c7d11c8819756beb6/Analysis/src/Unifier.cpp#L868) and [NormalizedType](https://github.com/Roblox/luau/blob/d6aa35583e4be14304d2a17c7d11c8819756beb6/Analysis/include/Luau/Normalize.h#L134) in the [open source Luau repo](https://github.com/Roblox/luau). From 662a5532ec762bb2e49cac8185cdd29a8375a997 Mon Sep 17 00:00:00 2001 From: Alan Jeffrey <403333+asajeffrey@users.noreply.github.com> Date: Wed, 19 Oct 2022 20:32:29 -0500 Subject: [PATCH 379/399] Minor fixes to the semantic subtyping blog post (#720) --- docs/_posts/2022-10-31-luau-semantic-subtyping.md | 13 +++++++++---- 1 file changed, 9 insertions(+), 4 deletions(-) diff --git a/docs/_posts/2022-10-31-luau-semantic-subtyping.md b/docs/_posts/2022-10-31-luau-semantic-subtyping.md index 65db643d..68622a67 100644 --- a/docs/_posts/2022-10-31-luau-semantic-subtyping.md +++ b/docs/_posts/2022-10-31-luau-semantic-subtyping.md @@ -66,8 +66,8 @@ Syntactic subtyping is a syntax-directed recursive algorithm. The interesting ca * Reflexivity: `T` is a subtype of `T` * Intersection L: `(T₁ & … & Tⱼ)` is a subtype of `U` whenever some of the `Tᵢ` are subtypes of `U` * Union L: `(T₁ | … | Tⱼ)` is a subtype of `U` whenever all of the `Tᵢ` are subtypes of `U` -* Intersection R: `T` is a subtype of `(U₁ & … & Uⱼ)` whenever `T` is a subtype of some of the `Uᵢ` -* Union R: `T` is a subtype of `(U₁ | … | Uⱼ)` whenever `T` is a subtype of all of the `Uᵢ`. +* Intersection R: `T` is a subtype of `(U₁ & … & Uⱼ)` whenever `T` is a subtype of all of the `Uᵢ` +* Union R: `T` is a subtype of `(U₁ | … | Uⱼ)` whenever `T` is a subtype of some of the `Uᵢ`. For example: @@ -262,6 +262,10 @@ Semantic subtyping has removed one source of false positives, but we still have The quest to remove spurious red squiggles continues! +## Acknowledgments + +Thanks to Giuseppe Castagna and Ben Greenman for helpful comments on drafts of this post. + ## Further reading If you want to find out more about Luau and semantic subtyping, you might want to check out… @@ -274,8 +278,9 @@ If you want to find out more about Luau and semantic subtyping, you might want t * Giuseppe Castagna, *Covariance and Contravariance*, Logical Methods in Computer Science 16(1), 2022. * Giuseppe Castagna and Alain Frisch, *A gentle introduction to semantic subtyping*, Proc. Principles and practice of declarative programming (PPDP), pp 198–208, 2005. * Giuseppe Castagna, Mickaël Laurent, Kim Nguyễn, Matthew Lutze, *On Type-Cases, Union Elimination, and Occurrence Typing*, Principles of Programming Languages (POPL), 2022. -* Sam Tobin-Hochstadt and Matthias Felleisen, *Logical types for untyped languages*. International Conference on Functional Programming (ICFP), 2010. https://doi.org/10.1145/1863543.1863561 -* José Valim, *My Future with Elixir: set-theoretic types*, 2022. https://elixir-lang.org/blog/2022/10/05/my-future-with-elixir-set-theoretic-types/ +* Giuseppe Castagna, *Programming with union, intersection, and negation types*, 2022. +* Sam Tobin-Hochstadt and Matthias Felleisen, *Logical types for untyped languages*. International Conference on Functional Programming (ICFP), 2010. +* José Valim, *My Future with Elixir: set-theoretic types*, 2022. Some other languages which support semantic subtyping… From c4c120513ff965b117bc006be6f98e227b74e1ae Mon Sep 17 00:00:00 2001 From: JohnnyMorganz Date: Thu, 20 Oct 2022 17:07:00 +0100 Subject: [PATCH 380/399] Fix reverse deps > 1 node away not correctly marked as dirty (#719) `Frontend.markDirty()` does not correctly mark reverse dependencies that are not immediate as dirty. This is because it would keep adding the reverse deps of `name` into the queue to mark as dirty, instead of the reverse deps of `next` --- Analysis/src/Frontend.cpp | 24 +++++++++++++++++++----- tests/Frontend.test.cpp | 27 +++++++++++++++++++++++++++ 2 files changed, 46 insertions(+), 5 deletions(-) diff --git a/Analysis/src/Frontend.cpp b/Analysis/src/Frontend.cpp index 5705ac17..cfe710d9 100644 --- a/Analysis/src/Frontend.cpp +++ b/Analysis/src/Frontend.cpp @@ -30,6 +30,7 @@ LUAU_FASTFLAGVARIABLE(LuauAutocompleteDynamicLimits, false) LUAU_FASTINTVARIABLE(LuauAutocompleteCheckTimeoutMs, 100) LUAU_FASTFLAGVARIABLE(DebugLuauDeferredConstraintResolution, false) LUAU_FASTFLAG(DebugLuauLogSolverToJson); +LUAU_FASTFLAGVARIABLE(LuauFixMarkDirtyReverseDeps, false) namespace Luau { @@ -807,13 +808,26 @@ void Frontend::markDirty(const ModuleName& name, std::vector* marked sourceNode.dirtyModule = true; sourceNode.dirtyModuleForAutocomplete = true; - if (0 == reverseDeps.count(name)) - continue; + if (FFlag::LuauFixMarkDirtyReverseDeps) + { + if (0 == reverseDeps.count(next)) + continue; - sourceModules.erase(name); + sourceModules.erase(next); - const std::vector& dependents = reverseDeps[name]; - queue.insert(queue.end(), dependents.begin(), dependents.end()); + const std::vector& dependents = reverseDeps[next]; + queue.insert(queue.end(), dependents.begin(), dependents.end()); + } + else + { + if (0 == reverseDeps.count(name)) + continue; + + sourceModules.erase(name); + + const std::vector& dependents = reverseDeps[name]; + queue.insert(queue.end(), dependents.begin(), dependents.end()); + } } } diff --git a/tests/Frontend.test.cpp b/tests/Frontend.test.cpp index 957f3c7c..df0abdc9 100644 --- a/tests/Frontend.test.cpp +++ b/tests/Frontend.test.cpp @@ -517,6 +517,33 @@ TEST_CASE_FIXTURE(FrontendFixture, "recheck_if_dependent_script_is_dirty") CHECK_EQ("{| b_value: string |}", toString(*bExports)); } +TEST_CASE_FIXTURE(FrontendFixture, "mark_non_immediate_reverse_deps_as_dirty") +{ + ScopedFastFlag sff[] = { + {"LuauFixMarkDirtyReverseDeps", true}, + }; + + fileResolver.source["game/Gui/Modules/A"] = "return {hello=5, world=true}"; + fileResolver.source["game/Gui/Modules/B"] = R"( + return require(game:GetService('Gui').Modules.A) + )"; + fileResolver.source["game/Gui/Modules/C"] = R"( + local Modules = game:GetService('Gui').Modules + local B = require(Modules.B) + return {c_value = B.hello} + )"; + + frontend.check("game/Gui/Modules/C"); + + std::vector markedDirty; + frontend.markDirty("game/Gui/Modules/A", &markedDirty); + + REQUIRE(markedDirty.size() == 3); + CHECK(std::find(markedDirty.begin(), markedDirty.end(), "game/Gui/Modules/A") != markedDirty.end()); + CHECK(std::find(markedDirty.begin(), markedDirty.end(), "game/Gui/Modules/B") != markedDirty.end()); + CHECK(std::find(markedDirty.begin(), markedDirty.end(), "game/Gui/Modules/C") != markedDirty.end()); +} + #if 0 // Does not work yet. :( TEST_CASE_FIXTURE(FrontendFixture, "recheck_if_dependent_script_has_a_parse_error") From 12ee1407a1b385bbc65550e37930d941c124ebf3 Mon Sep 17 00:00:00 2001 From: JohnnyMorganz Date: Fri, 21 Oct 2022 16:05:56 +0100 Subject: [PATCH 381/399] Add named parameters to `math` lib (#722) Name the parameters used in `math` lib This is mainly done to highlight the particular confusion for `math.atan2`, where `y` comes before `x`, but this might not be immediately obvious. And then I added the rest of the names for consistency. Note: I didn't add names to `math.random` as it's currently typed as `(number?, number?) -> number`. Naming it `min` and `max` is technically incorrect for the 1 argument version. Maybe it should be typed as an intersection instead? --- Analysis/src/EmbeddedBuiltinDefinitions.cpp | 58 ++++++++++----------- 1 file changed, 29 insertions(+), 29 deletions(-) diff --git a/Analysis/src/EmbeddedBuiltinDefinitions.cpp b/Analysis/src/EmbeddedBuiltinDefinitions.cpp index 0f04ace0..67abbff1 100644 --- a/Analysis/src/EmbeddedBuiltinDefinitions.cpp +++ b/Analysis/src/EmbeddedBuiltinDefinitions.cpp @@ -26,34 +26,34 @@ declare bit32: { } declare math: { - frexp: (number) -> (number, number), - ldexp: (number, number) -> number, - fmod: (number, number) -> number, - modf: (number) -> (number, number), - pow: (number, number) -> number, - exp: (number) -> number, + frexp: (n: number) -> (number, number), + ldexp: (s: number, e: number) -> number, + fmod: (x: number, y: number) -> number, + modf: (n: number) -> (number, number), + pow: (x: number, y: number) -> number, + exp: (n: number) -> number, - ceil: (number) -> number, - floor: (number) -> number, - abs: (number) -> number, - sqrt: (number) -> number, + ceil: (n: number) -> number, + floor: (n: number) -> number, + abs: (n: number) -> number, + sqrt: (n: number) -> number, - log: (number, number?) -> number, - log10: (number) -> number, + log: (n: number, base: number?) -> number, + log10: (n: number) -> number, - rad: (number) -> number, - deg: (number) -> number, + rad: (n: number) -> number, + deg: (n: number) -> number, - sin: (number) -> number, - cos: (number) -> number, - tan: (number) -> number, - sinh: (number) -> number, - cosh: (number) -> number, - tanh: (number) -> number, - atan: (number) -> number, - acos: (number) -> number, - asin: (number) -> number, - atan2: (number, number) -> number, + sin: (n: number) -> number, + cos: (n: number) -> number, + tan: (n: number) -> number, + sinh: (n: number) -> number, + cosh: (n: number) -> number, + tanh: (n: number) -> number, + atan: (n: number) -> number, + acos: (n: number) -> number, + asin: (n: number) -> number, + atan2: (y: number, x: number) -> number, min: (number, ...number) -> number, max: (number, ...number) -> number, @@ -61,13 +61,13 @@ declare math: { pi: number, huge: number, - randomseed: (number) -> (), + randomseed: (seed: number) -> (), random: (number?, number?) -> number, - sign: (number) -> number, - clamp: (number, number, number) -> number, - noise: (number, number?, number?) -> number, - round: (number) -> number, + sign: (n: number) -> number, + clamp: (n: number, min: number, max: number) -> number, + noise: (x: number, y: number?, z: number?) -> number, + round: (n: number) -> number, } type DateTypeArg = { From 54324867df2d91bc527f6de9217b05c3a9ae235f Mon Sep 17 00:00:00 2001 From: Andy Friesen Date: Fri, 21 Oct 2022 10:54:01 -0700 Subject: [PATCH 382/399] Sync to upstream/release/550 (#723) * Support `["prop"]` syntax on class definitions in definition files. (#704) * Improve type checking performance for complex overloaded functions * Fix rare cases of incorrect stack traces for out of memory errors at runtime --- Analysis/include/Luau/Clone.h | 2 + Analysis/include/Luau/Constraint.h | 17 +- .../include/Luau/ConstraintGraphBuilder.h | 31 +- Analysis/include/Luau/ConstraintSolver.h | 3 + Analysis/include/Luau/DataFlowGraphBuilder.h | 115 ++++ Analysis/include/Luau/Def.h | 78 +++ Analysis/include/Luau/LValue.h | 2 + Analysis/include/Luau/Metamethods.h | 32 + Analysis/include/Luau/Normalize.h | 9 - Analysis/include/Luau/Scope.h | 5 +- Analysis/include/Luau/Symbol.h | 14 +- Analysis/include/Luau/TypeInfer.h | 6 - Analysis/include/Luau/TypeUtils.h | 19 + Analysis/include/Luau/TypeVar.h | 74 +- Analysis/include/Luau/Unifier.h | 5 - Analysis/include/Luau/Variant.h | 2 +- Analysis/include/Luau/VisitTypeVar.h | 24 +- Analysis/src/Anyification.cpp | 14 - Analysis/src/BuiltinDefinitions.cpp | 46 +- Analysis/src/Clone.cpp | 50 +- Analysis/src/ConstraintGraphBuilder.cpp | 371 +++++++--- Analysis/src/ConstraintSolver.cpp | 289 +++++++- Analysis/src/DataFlowGraphBuilder.cpp | 440 ++++++++++++ Analysis/src/Def.cpp | 12 + Analysis/src/Error.cpp | 13 +- Analysis/src/Frontend.cpp | 51 +- Analysis/src/Module.cpp | 33 - Analysis/src/Normalize.cpp | 640 +----------------- Analysis/src/Quantify.cpp | 23 - Analysis/src/Scope.cpp | 55 +- Analysis/src/Substitution.cpp | 18 +- Analysis/src/ToDot.cpp | 9 - Analysis/src/ToString.cpp | 51 +- Analysis/src/TxnLog.cpp | 13 +- Analysis/src/TypeAttach.cpp | 21 +- Analysis/src/TypeChecker2.cpp | 268 +++++++- Analysis/src/TypeInfer.cpp | 53 +- Analysis/src/TypeUtils.cpp | 86 ++- Analysis/src/TypeVar.cpp | 33 +- Analysis/src/Unifier.cpp | 261 +------ CLI/Analyze.cpp | 5 +- CLI/Repl.cpp | 59 +- CodeGen/include/Luau/AssemblyBuilderX64.h | 26 +- CodeGen/include/Luau/CodeAllocator.h | 2 + CodeGen/include/Luau/CodeGen.h | 16 +- CodeGen/include/Luau/OperandX64.h | 5 + CodeGen/include/Luau/RegisterX64.h | 15 + CodeGen/src/AssemblyBuilderX64.cpp | 206 +++++- CodeGen/src/ByteUtils.h | 12 +- CodeGen/src/CodeAllocator.cpp | 8 +- CodeGen/src/CodeBlockUnwind.cpp | 5 +- CodeGen/src/CodeGen.cpp | 283 ++++++-- CodeGen/src/CodeGenX64.cpp | 4 +- CodeGen/src/CustomExecUtils.h | 15 - CodeGen/src/EmitCommonX64.cpp | 82 ++- CodeGen/src/EmitCommonX64.h | 62 +- CodeGen/src/EmitInstructionX64.cpp | 613 ++++++++++++----- CodeGen/src/EmitInstructionX64.h | 84 ++- CodeGen/src/NativeState.cpp | 61 +- CodeGen/src/NativeState.h | 14 +- Common/include/Luau/Common.h | 4 + Compiler/include/Luau/BytecodeBuilder.h | 9 +- Compiler/src/Builtins.cpp | 13 +- Compiler/src/BytecodeBuilder.cpp | 167 ++++- Sources.cmake | 9 +- VM/src/ldebug.cpp | 36 +- VM/src/lobject.cpp | 52 +- VM/src/lvmexecute.cpp | 28 +- bench/gc/test_GC_Boehm_Trees.lua | 2 +- tests/AssemblyBuilderX64.test.cpp | 219 ++++-- tests/CodeAllocator.test.cpp | 12 +- tests/Compiler.test.cpp | 2 - tests/Conformance.test.cpp | 63 ++ tests/ConstraintGraphBuilder.test.cpp | 126 ---- tests/ConstraintGraphBuilderFixture.cpp | 19 +- tests/ConstraintGraphBuilderFixture.h | 13 +- tests/ConstraintSolver.test.cpp | 46 +- tests/DataFlowGraphBuilder.test.cpp | 104 +++ tests/Module.test.cpp | 18 - tests/Normalize.test.cpp | 20 - tests/Symbol.test.cpp | 24 +- tests/ToDot.test.cpp | 25 +- tests/TypeInfer.aliases.test.cpp | 12 +- tests/TypeInfer.functions.test.cpp | 43 ++ tests/TypeInfer.intersectionTypes.test.cpp | 2 +- tests/TypeInfer.operators.test.cpp | 106 ++- tests/TypeInfer.refinements.test.cpp | 60 +- tests/TypeInfer.tables.test.cpp | 76 ++- tests/TypeInfer.test.cpp | 1 - tests/TypeInfer.typePacks.cpp | 4 + tests/TypeVar.test.cpp | 2 - tests/conformance/basic.lua | 7 + tests/conformance/calls.lua | 10 + tests/conformance/datetime.lua | 1 + tests/conformance/errors.lua | 2 + tests/conformance/events.lua | 8 + tests/conformance/iter.lua | 20 + tests/conformance/math.lua | 7 + tests/conformance/move.lua | 2 + tests/conformance/strings.lua | 30 + tests/conformance/tables.lua | 124 +++- tests/conformance/tpack.lua | 2 + tools/faillist.txt | 51 +- tools/test_dcr.py | 30 +- 104 files changed, 4210 insertions(+), 2266 deletions(-) create mode 100644 Analysis/include/Luau/DataFlowGraphBuilder.h create mode 100644 Analysis/include/Luau/Def.h create mode 100644 Analysis/include/Luau/Metamethods.h create mode 100644 Analysis/src/DataFlowGraphBuilder.cpp create mode 100644 Analysis/src/Def.cpp delete mode 100644 tests/ConstraintGraphBuilder.test.cpp create mode 100644 tests/DataFlowGraphBuilder.test.cpp diff --git a/Analysis/include/Luau/Clone.h b/Analysis/include/Luau/Clone.h index 217e1cc3..f003c242 100644 --- a/Analysis/include/Luau/Clone.h +++ b/Analysis/include/Luau/Clone.h @@ -1,6 +1,7 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #pragma once +#include #include "Luau/TypeArena.h" #include "Luau/TypeVar.h" @@ -26,5 +27,6 @@ TypeId clone(TypeId tp, TypeArena& dest, CloneState& cloneState); TypeFun clone(const TypeFun& typeFun, TypeArena& dest, CloneState& cloneState); TypeId shallowClone(TypeId ty, TypeArena& dest, const TxnLog* log, bool alwaysClone = false); +TypeId shallowClone(TypeId ty, NotNull dest); } // namespace Luau diff --git a/Analysis/include/Luau/Constraint.h b/Analysis/include/Luau/Constraint.h index 0e19f13f..7f092f5b 100644 --- a/Analysis/include/Luau/Constraint.h +++ b/Analysis/include/Luau/Constraint.h @@ -2,9 +2,10 @@ #pragma once #include "Luau/Ast.h" // Used for some of the enumerations +#include "Luau/Def.h" #include "Luau/NotNull.h" -#include "Luau/Variant.h" #include "Luau/TypeVar.h" +#include "Luau/Variant.h" #include #include @@ -131,9 +132,15 @@ struct HasPropConstraint std::string prop; }; -using ConstraintV = - Variant; +struct RefinementConstraint +{ + DefId def; + TypeId discriminantType; +}; + +using ConstraintV = Variant; struct Constraint { @@ -143,7 +150,7 @@ struct Constraint Constraint& operator=(const Constraint&) = delete; NotNull scope; - Location location; + Location location; // TODO: Extract this out into only the constraints that needs a location. Not all constraints needs locations. ConstraintV c; std::vector> dependencies; diff --git a/Analysis/include/Luau/ConstraintGraphBuilder.h b/Analysis/include/Luau/ConstraintGraphBuilder.h index 973c0a8e..dc5d4598 100644 --- a/Analysis/include/Luau/ConstraintGraphBuilder.h +++ b/Analysis/include/Luau/ConstraintGraphBuilder.h @@ -1,13 +1,9 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details - #pragma once -#include -#include -#include - #include "Luau/Ast.h" #include "Luau/Constraint.h" +#include "Luau/DataFlowGraphBuilder.h" #include "Luau/Module.h" #include "Luau/ModuleResolver.h" #include "Luau/NotNull.h" @@ -15,6 +11,10 @@ #include "Luau/TypeVar.h" #include "Luau/Variant.h" +#include +#include +#include + namespace Luau { @@ -48,6 +48,7 @@ struct ConstraintGraphBuilder DenseHashMap astResolvedTypePacks{nullptr}; // Defining scopes for AST nodes. DenseHashMap astTypeAliasDefiningScopes{nullptr}; + NotNull dfg; int recursionCount = 0; @@ -63,7 +64,8 @@ struct ConstraintGraphBuilder DcrLogger* logger; ConstraintGraphBuilder(const ModuleName& moduleName, ModulePtr module, TypeArena* arena, NotNull moduleResolver, - NotNull singletonTypes, NotNull ice, const ScopePtr& globalScope, DcrLogger* logger); + NotNull singletonTypes, NotNull ice, const ScopePtr& globalScope, DcrLogger* logger, + NotNull dfg); /** * Fabricates a new free type belonging to a given scope. @@ -88,15 +90,17 @@ struct ConstraintGraphBuilder * Adds a new constraint with no dependencies to a given scope. * @param scope the scope to add the constraint to. * @param cv the constraint variant to add. + * @return the pointer to the inserted constraint */ - void addConstraint(const ScopePtr& scope, const Location& location, ConstraintV cv); + NotNull addConstraint(const ScopePtr& scope, const Location& location, ConstraintV cv); /** * Adds a constraint to a given scope. * @param scope the scope to add the constraint to. Must not be null. * @param c the constraint to add. + * @return the pointer to the inserted constraint */ - void addConstraint(const ScopePtr& scope, std::unique_ptr c); + NotNull addConstraint(const ScopePtr& scope, std::unique_ptr c); /** * The entry point to the ConstraintGraphBuilder. This will construct a set @@ -139,13 +143,20 @@ struct ConstraintGraphBuilder */ TypeId check(const ScopePtr& scope, AstExpr* expr, std::optional expectedType = {}); - TypeId check(const ScopePtr& scope, AstExprTable* expr, std::optional expectedType); + TypeId check(const ScopePtr& scope, AstExprLocal* local); + TypeId check(const ScopePtr& scope, AstExprGlobal* global); TypeId check(const ScopePtr& scope, AstExprIndexName* indexName); TypeId check(const ScopePtr& scope, AstExprIndexExpr* indexExpr); TypeId check(const ScopePtr& scope, AstExprUnary* unary); - TypeId check(const ScopePtr& scope, AstExprBinary* binary); + TypeId check_(const ScopePtr& scope, AstExprUnary* unary); + TypeId check(const ScopePtr& scope, AstExprBinary* binary, std::optional expectedType); TypeId check(const ScopePtr& scope, AstExprIfElse* ifElse, std::optional expectedType); TypeId check(const ScopePtr& scope, AstExprTypeAssertion* typeAssert); + TypeId check(const ScopePtr& scope, AstExprTable* expr, std::optional expectedType); + + TypePackId checkLValues(const ScopePtr& scope, AstArray exprs); + + TypeId checkLValue(const ScopePtr& scope, AstExpr* expr); struct FunctionSignature { diff --git a/Analysis/include/Luau/ConstraintSolver.h b/Analysis/include/Luau/ConstraintSolver.h index 0bf6d1bc..5cc63e65 100644 --- a/Analysis/include/Luau/ConstraintSolver.h +++ b/Analysis/include/Luau/ConstraintSolver.h @@ -110,6 +110,7 @@ struct ConstraintSolver bool tryDispatch(const FunctionCallConstraint& c, NotNull constraint); bool tryDispatch(const PrimitiveTypeConstraint& c, NotNull constraint); bool tryDispatch(const HasPropConstraint& c, NotNull constraint); + bool tryDispatch(const RefinementConstraint& c, NotNull constraint); // for a, ... in some_table do // also handles __iter metamethod @@ -215,6 +216,8 @@ private: TypeId errorRecoveryType() const; TypePackId errorRecoveryTypePack() const; + TypeId unionOfTypes(TypeId a, TypeId b, NotNull scope, bool unifyFreeTypes); + ToStringOptions opts; }; diff --git a/Analysis/include/Luau/DataFlowGraphBuilder.h b/Analysis/include/Luau/DataFlowGraphBuilder.h new file mode 100644 index 00000000..3a72403e --- /dev/null +++ b/Analysis/include/Luau/DataFlowGraphBuilder.h @@ -0,0 +1,115 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +#pragma once + +// Do not include LValue. It should never be used here. +#include "Luau/Ast.h" +#include "Luau/DenseHash.h" +#include "Luau/Def.h" +#include "Luau/Symbol.h" + +#include + +namespace Luau +{ + +struct DataFlowGraph +{ + DataFlowGraph(DataFlowGraph&&) = default; + DataFlowGraph& operator=(DataFlowGraph&&) = default; + + // TODO: AstExprLocal, AstExprGlobal, and AstLocal* are guaranteed never to return nullopt. + // We leave them to return an optional as we build it out, but the end state is for them to return a non-optional DefId. + std::optional getDef(const AstExpr* expr) const; + std::optional getDef(const AstLocal* local) const; + + /// Retrieve the Def that corresponds to the given Symbol. + /// + /// We do not perform dataflow analysis on globals, so this function always + /// yields nullopt when passed a global Symbol. + std::optional getDef(const Symbol& symbol) const; + +private: + DataFlowGraph() = default; + + DataFlowGraph(const DataFlowGraph&) = delete; + DataFlowGraph& operator=(const DataFlowGraph&) = delete; + + DefArena arena; + DenseHashMap astDefs{nullptr}; + DenseHashMap localDefs{nullptr}; + + friend struct DataFlowGraphBuilder; +}; + +struct DfgScope +{ + DfgScope* parent; + DenseHashMap bindings{Symbol{}}; +}; + +struct ExpressionFlowGraph +{ + std::optional def; +}; + +// Currently unsound. We do not presently track the control flow of the program. +// Additionally, we do not presently track assignments. +struct DataFlowGraphBuilder +{ + static DataFlowGraph build(AstStatBlock* root, NotNull handle); + +private: + DataFlowGraphBuilder() = default; + + DataFlowGraphBuilder(const DataFlowGraphBuilder&) = delete; + DataFlowGraphBuilder& operator=(const DataFlowGraphBuilder&) = delete; + + DataFlowGraph graph; + NotNull arena{&graph.arena}; + struct InternalErrorReporter* handle; + std::vector> scopes; + + DfgScope* childScope(DfgScope* scope); + + std::optional use(DfgScope* scope, Symbol symbol, AstExpr* e); + + void visit(DfgScope* scope, AstStatBlock* b); + void visitBlockWithoutChildScope(DfgScope* scope, AstStatBlock* b); + + // TODO: visit type aliases + void visit(DfgScope* scope, AstStat* s); + void visit(DfgScope* scope, AstStatIf* i); + void visit(DfgScope* scope, AstStatWhile* w); + void visit(DfgScope* scope, AstStatRepeat* r); + void visit(DfgScope* scope, AstStatBreak* b); + void visit(DfgScope* scope, AstStatContinue* c); + void visit(DfgScope* scope, AstStatReturn* r); + void visit(DfgScope* scope, AstStatExpr* e); + void visit(DfgScope* scope, AstStatLocal* l); + void visit(DfgScope* scope, AstStatFor* f); + void visit(DfgScope* scope, AstStatForIn* f); + void visit(DfgScope* scope, AstStatAssign* a); + void visit(DfgScope* scope, AstStatCompoundAssign* c); + void visit(DfgScope* scope, AstStatFunction* f); + void visit(DfgScope* scope, AstStatLocalFunction* l); + + ExpressionFlowGraph visitExpr(DfgScope* scope, AstExpr* e); + ExpressionFlowGraph visitExpr(DfgScope* scope, AstExprLocal* l); + ExpressionFlowGraph visitExpr(DfgScope* scope, AstExprGlobal* g); + ExpressionFlowGraph visitExpr(DfgScope* scope, AstExprCall* c); + ExpressionFlowGraph visitExpr(DfgScope* scope, AstExprIndexName* i); + ExpressionFlowGraph visitExpr(DfgScope* scope, AstExprIndexExpr* i); + ExpressionFlowGraph visitExpr(DfgScope* scope, AstExprFunction* f); + ExpressionFlowGraph visitExpr(DfgScope* scope, AstExprTable* t); + ExpressionFlowGraph visitExpr(DfgScope* scope, AstExprUnary* u); + ExpressionFlowGraph visitExpr(DfgScope* scope, AstExprBinary* b); + ExpressionFlowGraph visitExpr(DfgScope* scope, AstExprTypeAssertion* t); + ExpressionFlowGraph visitExpr(DfgScope* scope, AstExprIfElse* i); + ExpressionFlowGraph visitExpr(DfgScope* scope, AstExprInterpString* i); + + // TODO: visitLValue + // TODO: visitTypes (because of typeof which has access to values namespace, needs unreachable scope) + // TODO: visitTypePacks (because of typeof which has access to values namespace, needs unreachable scope) +}; + +} // namespace Luau diff --git a/Analysis/include/Luau/Def.h b/Analysis/include/Luau/Def.h new file mode 100644 index 00000000..ac1fa132 --- /dev/null +++ b/Analysis/include/Luau/Def.h @@ -0,0 +1,78 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +#pragma once + +#include "Luau/NotNull.h" +#include "Luau/TypedAllocator.h" +#include "Luau/Variant.h" + +namespace Luau +{ + +using Def = Variant; + +/** + * We statically approximate a value at runtime using a symbolic value, which we call a Def. + * + * DataFlowGraphBuilder will allocate these defs as a stand-in for some Luau values, and bind them to places that + * can hold a Luau value, and then observes how those defs will commute as it statically evaluate the program. + * + * It must also be noted that defs are a cyclic graph, so it is not safe to recursively traverse into it expecting it to terminate. + */ +using DefId = NotNull; + +/** + * A "single-object" value. + * + * Leaky implementation note: sometimes "multiple-object" values, but none of which were interesting enough to warrant creating a phi node instead. + * That can happen because there's no point in creating a phi node that points to either resultant in `if math.random() > 0.5 then 5 else "hello"`. + * This might become of utmost importance if we wanted to do some backward reasoning, e.g. if `5` is taken, then `cond` must be `truthy`. + */ +struct Undefined +{ +}; + +/** + * A phi node is a union of defs. + * + * We need this because we're statically evaluating a program, and sometimes a place may be assigned with + * different defs, and when that happens, we need a special data type that merges in all the defs + * that will flow into that specific place. For example, consider this simple program: + * + * ``` + * x-1 + * if cond() then + * x-2 = 5 + * else + * x-3 = "hello" + * end + * x-4 : {x-2, x-3} + * ``` + * + * At x-4, we know for a fact statically that either `5` or `"hello"` can flow into the variable `x` after the branch, but + * we cannot make any definitive decisions about which one, so we just take in both. + */ +struct Phi +{ + std::vector operands; +}; + +template +T* getMutable(DefId def) +{ + return get_if(def.get()); +} + +template +const T* get(DefId def) +{ + return getMutable(def); +} + +struct DefArena +{ + TypedAllocator allocator; + + DefId freshDef(); +}; + +} // namespace Luau diff --git a/Analysis/include/Luau/LValue.h b/Analysis/include/Luau/LValue.h index 1a92d52d..518cbfaf 100644 --- a/Analysis/include/Luau/LValue.h +++ b/Analysis/include/Luau/LValue.h @@ -14,6 +14,8 @@ struct TypeVar; using TypeId = const TypeVar*; struct Field; + +// Deprecated. Do not use in new work. using LValue = Variant; struct Field diff --git a/Analysis/include/Luau/Metamethods.h b/Analysis/include/Luau/Metamethods.h new file mode 100644 index 00000000..84b0092f --- /dev/null +++ b/Analysis/include/Luau/Metamethods.h @@ -0,0 +1,32 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +#pragma once + +#include "Luau/Ast.h" + +#include + +namespace Luau +{ + +static const std::unordered_map kBinaryOpMetamethods{ + {AstExprBinary::Op::CompareEq, "__eq"}, + {AstExprBinary::Op::CompareNe, "__eq"}, + {AstExprBinary::Op::CompareGe, "__lt"}, + {AstExprBinary::Op::CompareGt, "__le"}, + {AstExprBinary::Op::CompareLe, "__le"}, + {AstExprBinary::Op::CompareLt, "__lt"}, + {AstExprBinary::Op::Add, "__add"}, + {AstExprBinary::Op::Sub, "__sub"}, + {AstExprBinary::Op::Mul, "__mul"}, + {AstExprBinary::Op::Div, "__div"}, + {AstExprBinary::Op::Pow, "__pow"}, + {AstExprBinary::Op::Mod, "__mod"}, + {AstExprBinary::Op::Concat, "__concat"}, +}; + +static const std::unordered_map kUnaryOpMetamethods{ + {AstExprUnary::Op::Minus, "__unm"}, + {AstExprUnary::Op::Len, "__len"}, +}; + +} // namespace Luau diff --git a/Analysis/include/Luau/Normalize.h b/Analysis/include/Luau/Normalize.h index 72ea9558..a23d0fda 100644 --- a/Analysis/include/Luau/Normalize.h +++ b/Analysis/include/Luau/Normalize.h @@ -22,15 +22,6 @@ bool isSubtype( bool isSubtype(TypePackId subTy, TypePackId superTy, NotNull scope, NotNull singletonTypes, InternalErrorReporter& ice, bool anyIsTop = true); -std::pair normalize( - TypeId ty, NotNull scope, TypeArena& arena, NotNull singletonTypes, InternalErrorReporter& ice); -std::pair normalize(TypeId ty, NotNull module, NotNull singletonTypes, InternalErrorReporter& ice); -std::pair normalize(TypeId ty, const ModulePtr& module, NotNull singletonTypes, InternalErrorReporter& ice); -std::pair normalize( - TypePackId ty, NotNull scope, TypeArena& arena, NotNull singletonTypes, InternalErrorReporter& ice); -std::pair normalize(TypePackId ty, NotNull module, NotNull singletonTypes, InternalErrorReporter& ice); -std::pair normalize(TypePackId ty, const ModulePtr& module, NotNull singletonTypes, InternalErrorReporter& ice); - class TypeIds { private: diff --git a/Analysis/include/Luau/Scope.h b/Analysis/include/Luau/Scope.h index b2da7bc0..ccf2964c 100644 --- a/Analysis/include/Luau/Scope.h +++ b/Analysis/include/Luau/Scope.h @@ -54,7 +54,9 @@ struct Scope DenseHashSet builtinTypeNames{""}; void addBuiltinTypeBinding(const Name& name, const TypeFun& tyFun); - std::optional lookup(Symbol sym); + std::optional lookup(Symbol sym) const; + std::optional lookup(DefId def) const; + std::optional> lookupEx(Symbol sym); std::optional lookupType(const Name& name); std::optional lookupImportedType(const Name& moduleAlias, const Name& name); @@ -66,6 +68,7 @@ struct Scope std::optional linearSearchForBinding(const std::string& name, bool traverseScopeChain = true) const; RefinementMap refinements; + DenseHashMap dcrRefinements{nullptr}; // For mutually recursive type aliases, it's important that // they use the same types for the same names. diff --git a/Analysis/include/Luau/Symbol.h b/Analysis/include/Luau/Symbol.h index 1fe037e5..0432946c 100644 --- a/Analysis/include/Luau/Symbol.h +++ b/Analysis/include/Luau/Symbol.h @@ -6,10 +6,11 @@ #include +LUAU_FASTFLAG(DebugLuauDeferredConstraintResolution) + namespace Luau { -// TODO Rename this to Name once the old type alias is gone. struct Symbol { Symbol() @@ -40,9 +41,12 @@ struct Symbol { if (local) return local == rhs.local; - if (global.value) + else if (global.value) return rhs.global.value && global == rhs.global.value; // Subtlety: AstName::operator==(const char*) uses strcmp, not pointer identity. - return false; + else if (FFlag::DebugLuauDeferredConstraintResolution) + return !rhs.local && !rhs.global.value; // Reflexivity: we already know `this` Symbol is empty, so check that rhs is. + else + return false; } bool operator!=(const Symbol& rhs) const @@ -58,8 +62,8 @@ struct Symbol return global < rhs.global; else if (local) return true; - else - return false; + + return false; } AstName astName() const diff --git a/Analysis/include/Luau/TypeInfer.h b/Analysis/include/Luau/TypeInfer.h index 1c4d1cb4..384637bb 100644 --- a/Analysis/include/Luau/TypeInfer.h +++ b/Analysis/include/Luau/TypeInfer.h @@ -192,18 +192,12 @@ struct TypeChecker ErrorVec canUnify(TypeId subTy, TypeId superTy, const ScopePtr& scope, const Location& location); ErrorVec canUnify(TypePackId subTy, TypePackId superTy, const ScopePtr& scope, const Location& location); - void unifyLowerBound(TypePackId subTy, TypePackId superTy, TypeLevel demotedLevel, const ScopePtr& scope, const Location& location); - std::optional findMetatableEntry(TypeId type, std::string entry, const Location& location, bool addErrors); std::optional findTablePropertyRespectingMeta(TypeId lhsType, Name name, const Location& location, bool addErrors); std::optional getIndexTypeFromType(const ScopePtr& scope, TypeId type, const Name& name, const Location& location, bool addErrors); std::optional getIndexTypeFromTypeImpl(const ScopePtr& scope, TypeId type, const Name& name, const Location& location, bool addErrors); - // Reduces the union to its simplest possible shape. - // (A | B) | B | C yields A | B | C - std::vector reduceUnion(const std::vector& types); - std::optional tryStripUnionFromNil(TypeId ty); TypeId stripFromNilAndReport(TypeId ty, const Location& location); diff --git a/Analysis/include/Luau/TypeUtils.h b/Analysis/include/Luau/TypeUtils.h index e5a205ba..7409dbe7 100644 --- a/Analysis/include/Luau/TypeUtils.h +++ b/Analysis/include/Luau/TypeUtils.h @@ -29,4 +29,23 @@ std::pair> getParameterExtents(const TxnLog* log, // various other things to get there. std::vector flatten(TypeArena& arena, NotNull singletonTypes, TypePackId pack, size_t length); +/** + * Reduces a union by decomposing to the any/error type if it appears in the + * type list, and by merging child unions. Also strips out duplicate (by pointer + * identity) types. + * @param types the input type list to reduce. + * @returns the reduced type list. +*/ +std::vector reduceUnion(const std::vector& types); + +/** + * Tries to remove nil from a union type, if there's another option. T | nil + * reduces to T, but nil itself does not reduce. + * @param singletonTypes the singleton types to use + * @param arena the type arena to allocate the new type in, if necessary + * @param ty the type to remove nil from + * @returns a type with nil removed, or nil itself if that were the only option. +*/ +TypeId stripNil(NotNull singletonTypes, TypeArena& arena, TypeId ty); + } // namespace Luau diff --git a/Analysis/include/Luau/TypeVar.h b/Analysis/include/Luau/TypeVar.h index 1d587ffe..70c12cb9 100644 --- a/Analysis/include/Luau/TypeVar.h +++ b/Analysis/include/Luau/TypeVar.h @@ -2,22 +2,23 @@ #pragma once #include "Luau/Ast.h" +#include "Luau/Common.h" #include "Luau/DenseHash.h" +#include "Luau/Def.h" +#include "Luau/NotNull.h" #include "Luau/Predicate.h" #include "Luau/Unifiable.h" #include "Luau/Variant.h" -#include "Luau/Common.h" -#include "Luau/NotNull.h" -#include -#include -#include -#include -#include -#include #include +#include #include #include +#include +#include +#include +#include +#include LUAU_FASTINT(LuauTableTypeMaximumStringifierLength) LUAU_FASTINT(LuauTypeMaximumStringifierLength) @@ -131,24 +132,6 @@ struct PrimitiveTypeVar } }; -struct ConstrainedTypeVar -{ - explicit ConstrainedTypeVar(TypeLevel level) - : level(level) - { - } - - explicit ConstrainedTypeVar(TypeLevel level, const std::vector& parts) - : parts(parts) - , level(level) - { - } - - std::vector parts; - TypeLevel level; - Scope* scope = nullptr; -}; - // Singleton types https://github.com/Roblox/luau/blob/master/rfcs/syntax-singleton-types.md // Types for true and false struct BooleanSingleton @@ -496,11 +479,13 @@ struct AnyTypeVar { }; +// T | U struct UnionTypeVar { std::vector options; }; +// T & U struct IntersectionTypeVar { std::vector parts; @@ -519,12 +504,27 @@ struct NeverTypeVar { }; +// Invariant 1: there should never be a reason why such UseTypeVar exists without it mapping to another type. +// Invariant 2: UseTypeVar should always disappear across modules. +struct UseTypeVar +{ + DefId def; + NotNull scope; +}; + +// ~T +// TODO: Some simplification step that overwrites the type graph to make sure negation +// types disappear from the user's view, and (?) a debug flag to disable that +struct NegationTypeVar +{ + TypeId ty; +}; + using ErrorTypeVar = Unifiable::Error; -using TypeVariant = - Unifiable::Variant; - +using TypeVariant = Unifiable::Variant; struct TypeVar final { @@ -541,7 +541,6 @@ struct TypeVar final TypeVar(const TypeVariant& ty, bool persistent) : ty(ty) , persistent(persistent) - , normal(persistent) // We assume that all persistent types are irreducable. { } @@ -549,7 +548,6 @@ struct TypeVar final void reassign(const TypeVar& rhs) { ty = rhs.ty; - normal = rhs.normal; documentationSymbol = rhs.documentationSymbol; } @@ -560,10 +558,6 @@ struct TypeVar final // Persistent TypeVars do not get cloned. bool persistent = false; - // Normalization sets this for types that are fully normalized. - // This implies that they are transitively immutable. - bool normal = false; - std::optional documentationSymbol; // Pointer to the type arena that allocated this type. @@ -656,6 +650,8 @@ public: const TypeId unknownType; const TypeId neverType; const TypeId errorType; + const TypeId falsyType; // No type binding! + const TypeId truthyType; // No type binding! const TypePackId anyTypePack; const TypePackId neverTypePack; @@ -703,7 +699,6 @@ T* getMutable(TypeId tv) const std::vector& getTypes(const UnionTypeVar* utv); const std::vector& getTypes(const IntersectionTypeVar* itv); -const std::vector& getTypes(const ConstrainedTypeVar* ctv); template struct TypeIterator; @@ -716,10 +711,6 @@ using IntersectionTypeVarIterator = TypeIterator; IntersectionTypeVarIterator begin(const IntersectionTypeVar* itv); IntersectionTypeVarIterator end(const IntersectionTypeVar* itv); -using ConstrainedTypeVarIterator = TypeIterator; -ConstrainedTypeVarIterator begin(const ConstrainedTypeVar* ctv); -ConstrainedTypeVarIterator end(const ConstrainedTypeVar* ctv); - /* Traverses the type T yielding each TypeId. * If the iterator encounters a nested type T, it will instead yield each TypeId within. */ @@ -793,7 +784,6 @@ struct TypeIterator // with templates portability in this area, so not worth it. Thanks MSVC. friend UnionTypeVarIterator end(const UnionTypeVar*); friend IntersectionTypeVarIterator end(const IntersectionTypeVar*); - friend ConstrainedTypeVarIterator end(const ConstrainedTypeVar*); private: TypeIterator() = default; diff --git a/Analysis/include/Luau/Unifier.h b/Analysis/include/Luau/Unifier.h index 10f3f48c..c15cae31 100644 --- a/Analysis/include/Luau/Unifier.h +++ b/Analysis/include/Luau/Unifier.h @@ -119,12 +119,7 @@ private: std::optional findTablePropertyRespectingMeta(TypeId lhsType, Name name); - void tryUnifyWithConstrainedSubTypeVar(TypeId subTy, TypeId superTy); - void tryUnifyWithConstrainedSuperTypeVar(TypeId subTy, TypeId superTy); - public: - void unifyLowerBound(TypePackId subTy, TypePackId superTy, TypeLevel demotedLevel); - // Returns true if the type "needle" already occurs within "haystack" and reports an "infinite type error" bool occursCheck(TypeId needle, TypeId haystack); bool occursCheck(DenseHashSet& seen, TypeId needle, TypeId haystack); diff --git a/Analysis/include/Luau/Variant.h b/Analysis/include/Luau/Variant.h index f637222e..76812c9b 100644 --- a/Analysis/include/Luau/Variant.h +++ b/Analysis/include/Luau/Variant.h @@ -105,7 +105,7 @@ public: tableDtor[typeId](&storage); typeId = tid; - new (&storage) TT(std::forward(args)...); + new (&storage) TT{std::forward(args)...}; return *reinterpret_cast(&storage); } diff --git a/Analysis/include/Luau/VisitTypeVar.h b/Analysis/include/Luau/VisitTypeVar.h index 315e5992..d4f8528f 100644 --- a/Analysis/include/Luau/VisitTypeVar.h +++ b/Analysis/include/Luau/VisitTypeVar.h @@ -103,10 +103,6 @@ struct GenericTypeVarVisitor { return visit(ty); } - virtual bool visit(TypeId ty, const ConstrainedTypeVar& ctv) - { - return visit(ty); - } virtual bool visit(TypeId ty, const PrimitiveTypeVar& ptv) { return visit(ty); @@ -159,6 +155,14 @@ struct GenericTypeVarVisitor { return visit(ty); } + virtual bool visit(TypeId ty, const UseTypeVar& utv) + { + return visit(ty); + } + virtual bool visit(TypeId ty, const NegationTypeVar& ntv) + { + return visit(ty); + } virtual bool visit(TypePackId tp) { @@ -216,14 +220,6 @@ struct GenericTypeVarVisitor visit(ty, *gtv); else if (auto etv = get(ty)) visit(ty, *etv); - else if (auto ctv = get(ty)) - { - if (visit(ty, *ctv)) - { - for (TypeId part : ctv->parts) - traverse(part); - } - } else if (auto ptv = get(ty)) visit(ty, *ptv); else if (auto ftv = get(ty)) @@ -325,6 +321,10 @@ struct GenericTypeVarVisitor traverse(a); } } + else if (auto utv = get(ty)) + visit(ty, *utv); + else if (auto ntv = get(ty)) + visit(ty, *ntv); else if (!FFlag::LuauCompleteVisitor) return visit_detail::unsee(seen, ty); else diff --git a/Analysis/src/Anyification.cpp b/Analysis/src/Anyification.cpp index cc9796ee..5dd761c2 100644 --- a/Analysis/src/Anyification.cpp +++ b/Analysis/src/Anyification.cpp @@ -37,8 +37,6 @@ bool Anyification::isDirty(TypeId ty) return (ttv->state == TableState::Free || ttv->state == TableState::Unsealed); else if (log->getMutable(ty)) return true; - else if (get(ty)) - return true; else return false; } @@ -65,20 +63,8 @@ TypeId Anyification::clean(TypeId ty) clone.syntheticName = ttv->syntheticName; clone.tags = ttv->tags; TypeId res = addType(std::move(clone)); - asMutable(res)->normal = ty->normal; return res; } - else if (auto ctv = get(ty)) - { - std::vector copy = ctv->parts; - for (TypeId& ty : copy) - ty = replace(ty); - TypeId res = copy.size() == 1 ? copy[0] : addType(UnionTypeVar{std::move(copy)}); - auto [t, ok] = normalize(res, scope, *arena, singletonTypes, *iceHandler); - if (!ok) - normalizationTooComplex = true; - return t; - } else return anyType; } diff --git a/Analysis/src/BuiltinDefinitions.cpp b/Analysis/src/BuiltinDefinitions.cpp index c5250a6d..6051e117 100644 --- a/Analysis/src/BuiltinDefinitions.cpp +++ b/Analysis/src/BuiltinDefinitions.cpp @@ -10,6 +10,7 @@ #include "Luau/TypeInfer.h" #include "Luau/TypePack.h" #include "Luau/TypeVar.h" +#include "Luau/TypeUtils.h" #include @@ -41,6 +42,7 @@ static std::optional> magicFunctionRequire( static bool dcrMagicFunctionSelect(MagicFunctionCallContext context); static bool dcrMagicFunctionRequire(MagicFunctionCallContext context); +static bool dcrMagicFunctionPack(MagicFunctionCallContext context); TypeId makeUnion(TypeArena& arena, std::vector&& types) { @@ -333,6 +335,7 @@ void registerBuiltinGlobals(TypeChecker& typeChecker) ttv->props["clone"] = makeProperty(makeFunction(arena, std::nullopt, {tabTy}, {tabTy}), "@luau/global/table.clone"); attachMagicFunction(ttv->props["pack"].type, magicFunctionPack); + attachDcrMagicFunction(ttv->props["pack"].type, dcrMagicFunctionPack); } attachMagicFunction(getGlobalBinding(typeChecker, "require"), magicFunctionRequire); @@ -660,7 +663,7 @@ static std::optional> magicFunctionPack( options.push_back(vtp->ty); } - options = typechecker.reduceUnion(options); + options = reduceUnion(options); // table.pack() -> {| n: number, [number]: nil |} // table.pack(1) -> {| n: number, [number]: number |} @@ -679,6 +682,47 @@ static std::optional> magicFunctionPack( return WithPredicate{arena.addTypePack({packedTable})}; } +static bool dcrMagicFunctionPack(MagicFunctionCallContext context) +{ + + TypeArena* arena = context.solver->arena; + + const auto& [paramTypes, paramTail] = flatten(context.arguments); + + std::vector options; + options.reserve(paramTypes.size()); + for (auto type : paramTypes) + options.push_back(type); + + if (paramTail) + { + if (const VariadicTypePack* vtp = get(*paramTail)) + options.push_back(vtp->ty); + } + + options = reduceUnion(options); + + // table.pack() -> {| n: number, [number]: nil |} + // table.pack(1) -> {| n: number, [number]: number |} + // table.pack(1, "foo") -> {| n: number, [number]: number | string |} + TypeId result = nullptr; + if (options.empty()) + result = context.solver->singletonTypes->nilType; + else if (options.size() == 1) + result = options[0]; + else + result = arena->addType(UnionTypeVar{std::move(options)}); + + TypeId numberType = context.solver->singletonTypes->numberType; + TypeId packedTable = arena->addType( + TableTypeVar{{{"n", {numberType}}}, TableIndexer(numberType, result), {}, TableState::Sealed}); + + TypePackId tableTypePack = arena->addTypePack({packedTable}); + asMutable(context.result)->ty.emplace(tableTypePack); + + return true; +} + static bool checkRequirePath(TypeChecker& typechecker, AstExpr* expr) { // require(foo.parent.bar) will technically work, but it depends on legacy goop that diff --git a/Analysis/src/Clone.cpp b/Analysis/src/Clone.cpp index fd3a089b..85408919 100644 --- a/Analysis/src/Clone.cpp +++ b/Analysis/src/Clone.cpp @@ -1,6 +1,6 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details - #include "Luau/Clone.h" + #include "Luau/RecursionCounter.h" #include "Luau/TxnLog.h" #include "Luau/TypePack.h" @@ -51,7 +51,6 @@ struct TypeCloner void operator()(const BlockedTypeVar& t); void operator()(const PendingExpansionTypeVar& t); void operator()(const PrimitiveTypeVar& t); - void operator()(const ConstrainedTypeVar& t); void operator()(const SingletonTypeVar& t); void operator()(const FunctionTypeVar& t); void operator()(const TableTypeVar& t); @@ -63,6 +62,8 @@ struct TypeCloner void operator()(const LazyTypeVar& t); void operator()(const UnknownTypeVar& t); void operator()(const NeverTypeVar& t); + void operator()(const UseTypeVar& t); + void operator()(const NegationTypeVar& t); }; struct TypePackCloner @@ -198,21 +199,6 @@ void TypeCloner::operator()(const PrimitiveTypeVar& t) defaultClone(t); } -void TypeCloner::operator()(const ConstrainedTypeVar& t) -{ - TypeId res = dest.addType(ConstrainedTypeVar{t.level}); - ConstrainedTypeVar* ctv = getMutable(res); - LUAU_ASSERT(ctv); - - seenTypes[typeId] = res; - - std::vector parts; - for (TypeId part : t.parts) - parts.push_back(clone(part, dest, cloneState)); - - ctv->parts = std::move(parts); -} - void TypeCloner::operator()(const SingletonTypeVar& t) { defaultClone(t); @@ -352,6 +338,21 @@ void TypeCloner::operator()(const NeverTypeVar& t) defaultClone(t); } +void TypeCloner::operator()(const UseTypeVar& t) +{ + TypeId result = dest.addType(BoundTypeVar{follow(typeId)}); + seenTypes[typeId] = result; +} + +void TypeCloner::operator()(const NegationTypeVar& t) +{ + TypeId result = dest.addType(AnyTypeVar{}); + seenTypes[typeId] = result; + + TypeId ty = clone(t.ty, dest, cloneState); + asMutable(result)->ty = NegationTypeVar{ty}; +} + } // anonymous namespace TypePackId clone(TypePackId tp, TypeArena& dest, CloneState& cloneState) @@ -390,7 +391,6 @@ TypeId clone(TypeId typeId, TypeArena& dest, CloneState& cloneState) if (!res->persistent) { asMutable(res)->documentationSymbol = typeId->documentationSymbol; - asMutable(res)->normal = typeId->normal; } } @@ -478,11 +478,6 @@ TypeId shallowClone(TypeId ty, TypeArena& dest, const TxnLog* log, bool alwaysCl clone.parts = itv->parts; result = dest.addType(std::move(clone)); } - else if (const ConstrainedTypeVar* ctv = get(ty)) - { - ConstrainedTypeVar clone{ctv->level, ctv->parts}; - result = dest.addType(std::move(clone)); - } else if (const PendingExpansionTypeVar* petv = get(ty)) { PendingExpansionTypeVar clone{petv->prefix, petv->name, petv->typeArguments, petv->packArguments}; @@ -497,6 +492,10 @@ TypeId shallowClone(TypeId ty, TypeArena& dest, const TxnLog* log, bool alwaysCl { result = dest.addType(*ty); } + else if (const NegationTypeVar* ntv = get(ty)) + { + result = dest.addType(NegationTypeVar{ntv->ty}); + } else return result; @@ -504,4 +503,9 @@ TypeId shallowClone(TypeId ty, TypeArena& dest, const TxnLog* log, bool alwaysCl return result; } +TypeId shallowClone(TypeId ty, NotNull dest) +{ + return shallowClone(ty, *dest, TxnLog::empty()); +} + } // namespace Luau diff --git a/Analysis/src/ConstraintGraphBuilder.cpp b/Analysis/src/ConstraintGraphBuilder.cpp index 8436fb30..de2b0a4e 100644 --- a/Analysis/src/ConstraintGraphBuilder.cpp +++ b/Analysis/src/ConstraintGraphBuilder.cpp @@ -1,20 +1,21 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details - #include "Luau/ConstraintGraphBuilder.h" + #include "Luau/Ast.h" +#include "Luau/Clone.h" #include "Luau/Common.h" #include "Luau/Constraint.h" +#include "Luau/DcrLogger.h" #include "Luau/ModuleResolver.h" #include "Luau/RecursionCounter.h" +#include "Luau/Scope.h" #include "Luau/ToString.h" -#include "Luau/DcrLogger.h" +#include "Luau/TypeUtils.h" LUAU_FASTINT(LuauCheckRecursionLimit); LUAU_FASTFLAG(DebugLuauLogSolverToJson); LUAU_FASTFLAG(DebugLuauMagicTypes); -#include "Luau/Scope.h" - namespace Luau { @@ -53,12 +54,13 @@ static bool matchSetmetatable(const AstExprCall& call) ConstraintGraphBuilder::ConstraintGraphBuilder(const ModuleName& moduleName, ModulePtr module, TypeArena* arena, NotNull moduleResolver, NotNull singletonTypes, NotNull ice, const ScopePtr& globalScope, - DcrLogger* logger) + DcrLogger* logger, NotNull dfg) : moduleName(moduleName) , module(module) , singletonTypes(singletonTypes) , arena(arena) , rootScope(nullptr) + , dfg(dfg) , moduleResolver(moduleResolver) , ice(ice) , globalScope(globalScope) @@ -95,14 +97,14 @@ ScopePtr ConstraintGraphBuilder::childScope(AstNode* node, const ScopePtr& paren return scope; } -void ConstraintGraphBuilder::addConstraint(const ScopePtr& scope, const Location& location, ConstraintV cv) +NotNull ConstraintGraphBuilder::addConstraint(const ScopePtr& scope, const Location& location, ConstraintV cv) { - scope->constraints.emplace_back(new Constraint{NotNull{scope.get()}, location, std::move(cv)}); + return NotNull{scope->constraints.emplace_back(new Constraint{NotNull{scope.get()}, location, std::move(cv)}).get()}; } -void ConstraintGraphBuilder::addConstraint(const ScopePtr& scope, std::unique_ptr c) +NotNull ConstraintGraphBuilder::addConstraint(const ScopePtr& scope, std::unique_ptr c) { - scope->constraints.emplace_back(std::move(c)); + return NotNull{scope->constraints.emplace_back(std::move(c)).get()}; } void ConstraintGraphBuilder::visit(AstStatBlock* block) @@ -229,22 +231,16 @@ void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStat* stat) void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatLocal* local) { std::vector varTypes; + varTypes.reserve(local->vars.size); for (AstLocal* local : local->vars) { TypeId ty = nullptr; - Location location = local->location; if (local->annotation) - { - location = local->annotation->location; ty = resolveType(scope, local->annotation, /* topLevel */ true); - } - else - ty = freshType(scope); varTypes.push_back(ty); - scope->bindings[local] = Binding{ty, location}; } for (size_t i = 0; i < local->values.size; ++i) @@ -257,6 +253,9 @@ void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatLocal* local) // HACK: we leave nil-initialized things floating under the assumption that they will later be populated. // See the test TypeInfer/infer_locals_with_nil_value. // Better flow awareness should make this obsolete. + + if (!varTypes[i]) + varTypes[i] = freshType(scope); } else if (i == local->values.size - 1) { @@ -268,6 +267,20 @@ void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatLocal* local) if (i < local->vars.size) { + std::vector packTypes = flatten(*arena, singletonTypes, exprPack, varTypes.size() - i); + + // fill out missing values in varTypes with values from exprPack + for (size_t j = i; j < varTypes.size(); ++j) + { + if (!varTypes[j]) + { + if (j - i < packTypes.size()) + varTypes[j] = packTypes[j - i]; + else + varTypes[j] = freshType(scope); + } + } + std::vector tailValues{varTypes.begin() + i, varTypes.end()}; TypePackId tailPack = arena->addTypePack(std::move(tailValues)); addConstraint(scope, local->location, PackSubtypeConstraint{exprPack, tailPack}); @@ -281,10 +294,31 @@ void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatLocal* local) TypeId exprType = check(scope, value, expectedType); if (i < varTypes.size()) - addConstraint(scope, local->location, SubtypeConstraint{varTypes[i], exprType}); + { + if (varTypes[i]) + addConstraint(scope, local->location, SubtypeConstraint{varTypes[i], exprType}); + else + varTypes[i] = exprType; + } } } + for (size_t i = 0; i < local->vars.size; ++i) + { + AstLocal* l = local->vars.data[i]; + Location location = l->location; + + if (!varTypes[i]) + varTypes[i] = freshType(scope); + + scope->bindings[l] = Binding{varTypes[i], location}; + + // HACK: In the greedy solver, we say the type state of a variable is the type annotation itself, but + // the actual type state is the corresponding initializer expression (if it exists) or nil otherwise. + if (auto def = dfg->getDef(l)) + scope->dcrRefinements[*def] = varTypes[i]; + } + if (local->values.size > 0) { // To correctly handle 'require', we need to import the exported type bindings into the variable 'namespace'. @@ -510,7 +544,7 @@ void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatBlock* block) void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatAssign* assign) { - TypePackId varPackId = checkPack(scope, assign->vars); + TypePackId varPackId = checkLValues(scope, assign->vars); TypePackId valuePack = checkPack(scope, assign->values); addConstraint(scope, assign->location, PackSubtypeConstraint{valuePack, varPackId}); @@ -532,7 +566,10 @@ void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatCompoundAssign* void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatIf* ifStatement) { - check(scope, ifStatement->condition); + // TODO: Optimization opportunity, the interior scope of the condition could be + // reused for the then body, so we don't need to refine twice. + ScopePtr condScope = childScope(ifStatement->condition, scope); + check(condScope, ifStatement->condition, std::nullopt); ScopePtr thenScope = childScope(ifStatement->thenbody, scope); visit(thenScope, ifStatement->thenbody); @@ -893,7 +930,7 @@ TypeId ConstraintGraphBuilder::check(const ScopePtr& scope, AstExpr* expr, std:: TypeId result = nullptr; if (auto group = expr->as()) - result = check(scope, group->expr); + result = check(scope, group->expr, expectedType); else if (auto stringExpr = expr->as()) { if (expectedType) @@ -937,32 +974,14 @@ TypeId ConstraintGraphBuilder::check(const ScopePtr& scope, AstExpr* expr, std:: } else if (expr->is()) result = singletonTypes->nilType; - else if (auto a = expr->as()) - { - std::optional ty = scope->lookup(a->local); - if (ty) - result = *ty; - else - result = singletonTypes->errorRecoveryType(); // FIXME? Record an error at this point? - } - else if (auto g = expr->as()) - { - std::optional ty = scope->lookup(g->name); - if (ty) - result = *ty; - else - { - /* prepopulateGlobalScope() has already added all global functions to the environment by this point, so any - * global that is not already in-scope is definitely an unknown symbol. - */ - reportError(g->location, UnknownSymbol{g->name.value}); - result = singletonTypes->errorRecoveryType(); // FIXME? Record an error at this point? - } - } + else if (auto local = expr->as()) + result = check(scope, local); + else if (auto global = expr->as()) + result = check(scope, global); else if (expr->is()) result = flattenPack(scope, expr->location, checkPack(scope, expr)); else if (expr->is()) - result = flattenPack(scope, expr->location, checkPack(scope, expr)); + result = flattenPack(scope, expr->location, checkPack(scope, expr)); // TODO: needs predicates too else if (auto a = expr->as()) { FunctionSignature sig = checkFunctionSignature(scope, a); @@ -978,7 +997,7 @@ TypeId ConstraintGraphBuilder::check(const ScopePtr& scope, AstExpr* expr, std:: else if (auto unary = expr->as()) result = check(scope, unary); else if (auto binary = expr->as()) - result = check(scope, binary); + result = check(scope, binary, expectedType); else if (auto ifElse = expr->as()) result = check(scope, ifElse, expectedType); else if (auto typeAssert = expr->as()) @@ -1002,6 +1021,37 @@ TypeId ConstraintGraphBuilder::check(const ScopePtr& scope, AstExpr* expr, std:: return result; } +TypeId ConstraintGraphBuilder::check(const ScopePtr& scope, AstExprLocal* local) +{ + std::optional resultTy; + + if (auto def = dfg->getDef(local)) + resultTy = scope->lookup(*def); + + if (!resultTy) + { + if (auto ty = scope->lookup(local->local)) + resultTy = *ty; + } + + if (!resultTy) + return singletonTypes->errorRecoveryType(); // TODO: replace with ice, locals should never exist before its definition. + + return *resultTy; +} + +TypeId ConstraintGraphBuilder::check(const ScopePtr& scope, AstExprGlobal* global) +{ + if (std::optional ty = scope->lookup(global->name)) + return *ty; + + /* prepopulateGlobalScope() has already added all global functions to the environment by this point, so any + * global that is not already in-scope is definitely an unknown symbol. + */ + reportError(global->location, UnknownSymbol{global->name.value}); + return singletonTypes->errorRecoveryType(); +} + TypeId ConstraintGraphBuilder::check(const ScopePtr& scope, AstExprIndexName* indexName) { TypeId obj = check(scope, indexName->expr); @@ -1036,54 +1086,32 @@ TypeId ConstraintGraphBuilder::check(const ScopePtr& scope, AstExprIndexExpr* in TypeId ConstraintGraphBuilder::check(const ScopePtr& scope, AstExprUnary* unary) { - TypeId operandType = check(scope, unary->expr); - + TypeId operandType = check_(scope, unary); TypeId resultType = arena->addType(BlockedTypeVar{}); addConstraint(scope, unary->location, UnaryConstraint{unary->op, operandType, resultType}); return resultType; } -TypeId ConstraintGraphBuilder::check(const ScopePtr& scope, AstExprBinary* binary) +TypeId ConstraintGraphBuilder::check_(const ScopePtr& scope, AstExprUnary* unary) { - TypeId leftType = check(scope, binary->left); - TypeId rightType = check(scope, binary->right); - switch (binary->op) + if (unary->op == AstExprUnary::Not) { - case AstExprBinary::And: - case AstExprBinary::Or: - { - addConstraint(scope, binary->location, SubtypeConstraint{leftType, rightType}); - return leftType; - } - case AstExprBinary::Add: - case AstExprBinary::Sub: - case AstExprBinary::Mul: - case AstExprBinary::Div: - case AstExprBinary::Mod: - case AstExprBinary::Pow: - case AstExprBinary::CompareNe: - case AstExprBinary::CompareEq: - case AstExprBinary::CompareLt: - case AstExprBinary::CompareLe: - case AstExprBinary::CompareGt: - case AstExprBinary::CompareGe: - { - TypeId resultType = arena->addType(BlockedTypeVar{}); - addConstraint(scope, binary->location, BinaryConstraint{binary->op, leftType, rightType, resultType}); - return resultType; - } - case AstExprBinary::Concat: - { - addConstraint(scope, binary->left->location, SubtypeConstraint{leftType, singletonTypes->stringType}); - addConstraint(scope, binary->right->location, SubtypeConstraint{rightType, singletonTypes->stringType}); - return singletonTypes->stringType; - } - default: - LUAU_ASSERT(0); + TypeId ty = check(scope, unary->expr, std::nullopt); + + return ty; } - LUAU_ASSERT(0); - return nullptr; + return check(scope, unary->expr); +} + +TypeId ConstraintGraphBuilder::check(const ScopePtr& scope, AstExprBinary* binary, std::optional expectedType) +{ + TypeId leftType = check(scope, binary->left, expectedType); + TypeId rightType = check(scope, binary->right, expectedType); + + TypeId resultType = arena->addType(BlockedTypeVar{}); + addConstraint(scope, binary->location, BinaryConstraint{binary->op, leftType, rightType, resultType}); + return resultType; } TypeId ConstraintGraphBuilder::check(const ScopePtr& scope, AstExprIfElse* ifElse, std::optional expectedType) @@ -1106,10 +1134,182 @@ TypeId ConstraintGraphBuilder::check(const ScopePtr& scope, AstExprIfElse* ifEls TypeId ConstraintGraphBuilder::check(const ScopePtr& scope, AstExprTypeAssertion* typeAssert) { - check(scope, typeAssert->expr); + check(scope, typeAssert->expr, std::nullopt); return resolveType(scope, typeAssert->annotation); } +TypePackId ConstraintGraphBuilder::checkLValues(const ScopePtr& scope, AstArray exprs) +{ + std::vector types; + types.reserve(exprs.size); + + for (size_t i = 0; i < exprs.size; ++i) + { + AstExpr* const expr = exprs.data[i]; + types.push_back(checkLValue(scope, expr)); + } + + return arena->addTypePack(std::move(types)); +} + +static bool isUnsealedTable(TypeId ty) +{ + ty = follow(ty); + const TableTypeVar* ttv = get(ty); + return ttv && ttv->state == TableState::Unsealed; +}; + +/** + * If the expr is a dotted set of names, and if the root symbol refers to an + * unsealed table, return that table type, plus the indeces that follow as a + * vector. + */ +static std::optional>> extractDottedName(AstExpr* expr) +{ + std::vector names; + + while (expr) + { + if (auto global = expr->as()) + { + std::reverse(begin(names), end(names)); + return std::pair{global->name, std::move(names)}; + } + else if (auto local = expr->as()) + { + std::reverse(begin(names), end(names)); + return std::pair{local->local, std::move(names)}; + } + else if (auto indexName = expr->as()) + { + names.push_back(indexName->index.value); + expr = indexName->expr; + } + else + return std::nullopt; + } + + return std::nullopt; +} + +/** + * Create a shallow copy of `ty` and its properties along `path`. Insert a new + * property (the last segment of `path`) into the tail table with the value `t`. + * + * On success, returns the new outermost table type. If the root table or any + * of its subkeys are not unsealed tables, the function fails and returns + * std::nullopt. + * + * TODO: Prove that we completely give up in the face of indexers and + * metatables. + */ +static std::optional updateTheTableType(NotNull arena, TypeId ty, const std::vector& path, TypeId replaceTy) +{ + if (path.empty()) + return std::nullopt; + + // First walk the path and ensure that it's unsealed tables all the way + // to the end. + { + TypeId t = ty; + for (size_t i = 0; i < path.size() - 1; ++i) + { + if (!isUnsealedTable(t)) + return std::nullopt; + + const TableTypeVar* tbl = get(t); + auto it = tbl->props.find(path[i]); + if (it == tbl->props.end()) + return std::nullopt; + + t = it->second.type; + } + + // The last path segment should not be a property of the table at all. + // We are not changing property types. We are only admitting this one + // new property to be appended. + if (!isUnsealedTable(t)) + return std::nullopt; + const TableTypeVar* tbl = get(t); + auto it = tbl->props.find(path.back()); + if (it != tbl->props.end()) + return std::nullopt; + } + + const TypeId res = shallowClone(ty, arena); + TypeId t = res; + + for (size_t i = 0; i < path.size() - 1; ++i) + { + const std::string segment = path[i]; + + TableTypeVar* ttv = getMutable(t); + LUAU_ASSERT(ttv); + + auto propIt = ttv->props.find(segment); + if (propIt != ttv->props.end()) + { + LUAU_ASSERT(isUnsealedTable(propIt->second.type)); + t = shallowClone(follow(propIt->second.type), arena); + ttv->props[segment].type = t; + } + else + return std::nullopt; + } + + TableTypeVar* ttv = getMutable(t); + LUAU_ASSERT(ttv); + + const std::string lastSegment = path.back(); + LUAU_ASSERT(0 == ttv->props.count(lastSegment)); + ttv->props[lastSegment] = Property{replaceTy}; + return res; +} + +/** + * This function is mostly about identifying properties that are being inserted into unsealed tables. + * + * If expr has the form name.a.b.c + */ +TypeId ConstraintGraphBuilder::checkLValue(const ScopePtr& scope, AstExpr* expr) +{ + if (auto indexExpr = expr->as()) + { + if (auto constantString = indexExpr->index->as()) + { + AstName syntheticIndex{constantString->value.data}; + AstExprIndexName synthetic{ + indexExpr->location, indexExpr->expr, syntheticIndex, constantString->location, indexExpr->expr->location.end, '.'}; + return checkLValue(scope, &synthetic); + } + } + + auto dottedPath = extractDottedName(expr); + if (!dottedPath) + return check(scope, expr); + const auto [sym, segments] = std::move(*dottedPath); + + if (!sym.local) + return check(scope, expr); + + auto lookupResult = scope->lookupEx(sym); + if (!lookupResult) + return check(scope, expr); + const auto [ty, symbolScope] = std::move(*lookupResult); + + TypeId replaceTy = arena->freshType(scope.get()); + + std::optional updatedType = updateTheTableType(arena, ty, segments, replaceTy); + if (!updatedType) + return check(scope, expr); + + std::optional def = dfg->getDef(sym); + LUAU_ASSERT(def); + symbolScope->bindings[sym].typeId = *updatedType; + symbolScope->dcrRefinements[*def] = *updatedType; + return replaceTy; +} + TypeId ConstraintGraphBuilder::check(const ScopePtr& scope, AstExprTable* expr, std::optional expectedType) { TypeId ty = arena->addType(TableTypeVar{}); @@ -1275,6 +1475,9 @@ ConstraintGraphBuilder::FunctionSignature ConstraintGraphBuilder::checkFunctionS argTypes.push_back(t); signatureScope->bindings[local] = Binding{t, local->location}; + if (auto def = dfg->getDef(local)) + signatureScope->dcrRefinements[*def] = t; + if (local->annotation) { TypeId argAnnotation = resolveType(signatureScope, local->annotation, /* topLevel */ true); diff --git a/Analysis/src/ConstraintSolver.cpp b/Analysis/src/ConstraintSolver.cpp index e29eeaaa..60f4666a 100644 --- a/Analysis/src/ConstraintSolver.cpp +++ b/Analysis/src/ConstraintSolver.cpp @@ -3,14 +3,16 @@ #include "Luau/Anyification.h" #include "Luau/ApplyTypeFunction.h" #include "Luau/ConstraintSolver.h" +#include "Luau/DcrLogger.h" #include "Luau/Instantiation.h" #include "Luau/Location.h" +#include "Luau/Metamethods.h" #include "Luau/ModuleResolver.h" #include "Luau/Quantify.h" #include "Luau/ToString.h" +#include "Luau/TypeUtils.h" #include "Luau/TypeVar.h" #include "Luau/Unifier.h" -#include "Luau/DcrLogger.h" #include "Luau/VisitTypeVar.h" #include "Luau/TypeUtils.h" @@ -438,6 +440,8 @@ bool ConstraintSolver::tryDispatch(NotNull constraint, bool fo success = tryDispatch(*fcc, constraint); else if (auto hpc = get(*constraint)) success = tryDispatch(*hpc, constraint); + else if (auto rc = get(*constraint)) + success = tryDispatch(*rc, constraint); else LUAU_ASSERT(false); @@ -564,44 +568,192 @@ bool ConstraintSolver::tryDispatch(const BinaryConstraint& c, NotNull - * - * This constraint is the one that is meant to unblock A, so it doesn't - * make any sense to stop and wait for someone else to do it. - */ - if (leftType != resultType && rightType != resultType) - { - block(c.leftType, constraint); - block(c.rightType, constraint); - return false; - } - } + bool isLogical = c.op == AstExprBinary::Op::And || c.op == AstExprBinary::Op::Or; - if (isNumber(leftType)) - { - unify(leftType, rightType, constraint->scope); - asMutable(resultType)->ty.emplace(leftType); - return true; - } + /* Compound assignments create constraints of the form + * + * A <: Binary + * + * This constraint is the one that is meant to unblock A, so it doesn't + * make any sense to stop and wait for someone else to do it. + */ + + if (isBlocked(leftType) && leftType != resultType) + return block(c.leftType, constraint); + + if (isBlocked(rightType) && rightType != resultType) + return block(c.rightType, constraint); if (!force) { - if (get(leftType)) + // Logical expressions may proceed if the LHS is free. + if (get(leftType) && !isLogical) return block(leftType, constraint); } - if (isBlocked(leftType)) + // Logical expressions may proceed if the LHS is free. + if (isBlocked(leftType) || (get(leftType) && !isLogical)) { asMutable(resultType)->ty.emplace(errorRecoveryType()); - // reportError(constraint->location, CannotInferBinaryOperation{c.op, std::nullopt, CannotInferBinaryOperation::Operation}); + unblock(resultType); return true; } - // TODO metatables, classes + // For or expressions, the LHS will never have nil as a possible output. + // Consider: + // local foo = nil or 2 + // `foo` will always be 2. + if (c.op == AstExprBinary::Op::Or) + leftType = stripNil(singletonTypes, *arena, leftType); + + // Metatables go first, even if there is primitive behavior. + if (auto it = kBinaryOpMetamethods.find(c.op); it != kBinaryOpMetamethods.end()) + { + // Metatables are not the same. The metamethod will not be invoked. + if ((c.op == AstExprBinary::Op::CompareEq || c.op == AstExprBinary::Op::CompareNe) && + getMetatable(leftType, singletonTypes) != getMetatable(rightType, singletonTypes)) + { + // TODO: Boolean singleton false? The result is _always_ boolean false. + asMutable(resultType)->ty.emplace(singletonTypes->booleanType); + unblock(resultType); + return true; + } + + std::optional mm; + + // The LHS metatable takes priority over the RHS metatable, where + // present. + if (std::optional leftMm = findMetatableEntry(singletonTypes, errors, leftType, it->second, constraint->location)) + mm = leftMm; + else if (std::optional rightMm = findMetatableEntry(singletonTypes, errors, rightType, it->second, constraint->location)) + mm = rightMm; + + if (mm) + { + // TODO: Is a table with __call legal here? + // TODO: Overloads + if (const FunctionTypeVar* ftv = get(follow(*mm))) + { + TypePackId inferredArgs; + // For >= and > we invoke __lt and __le respectively with + // swapped argument ordering. + if (c.op == AstExprBinary::Op::CompareGe || c.op == AstExprBinary::Op::CompareGt) + { + inferredArgs = arena->addTypePack({rightType, leftType}); + } + else + { + inferredArgs = arena->addTypePack({leftType, rightType}); + } + + unify(inferredArgs, ftv->argTypes, constraint->scope); + + TypeId mmResult; + + // Comparison operations always evaluate to a boolean, + // regardless of what the metamethod returns. + switch (c.op) + { + case AstExprBinary::Op::CompareEq: + case AstExprBinary::Op::CompareNe: + case AstExprBinary::Op::CompareGe: + case AstExprBinary::Op::CompareGt: + case AstExprBinary::Op::CompareLe: + case AstExprBinary::Op::CompareLt: + mmResult = singletonTypes->booleanType; + break; + default: + mmResult = first(ftv->retTypes).value_or(errorRecoveryType()); + } + + asMutable(resultType)->ty.emplace(mmResult); + unblock(resultType); + return true; + } + } + + // If there's no metamethod available, fall back to primitive behavior. + } + + // If any is present, the expression must evaluate to any as well. + bool leftAny = get(leftType) || get(leftType); + bool rightAny = get(rightType) || get(rightType); + bool anyPresent = leftAny || rightAny; + + switch (c.op) + { + // For arithmetic operators, if the LHS is a number, the RHS must be a + // number as well. The result will also be a number. + case AstExprBinary::Op::Add: + case AstExprBinary::Op::Sub: + case AstExprBinary::Op::Mul: + case AstExprBinary::Op::Div: + case AstExprBinary::Op::Pow: + case AstExprBinary::Op::Mod: + if (isNumber(leftType)) + { + unify(leftType, rightType, constraint->scope); + asMutable(resultType)->ty.emplace(anyPresent ? singletonTypes->anyType : leftType); + unblock(resultType); + return true; + } + + break; + // For concatenation, if the LHS is a string, the RHS must be a string as + // well. The result will also be a string. + case AstExprBinary::Op::Concat: + if (isString(leftType)) + { + unify(leftType, rightType, constraint->scope); + asMutable(resultType)->ty.emplace(anyPresent ? singletonTypes->anyType : leftType); + unblock(resultType); + return true; + } + + break; + // Inexact comparisons require that the types be both numbers or both + // strings, and evaluate to a boolean. + case AstExprBinary::Op::CompareGe: + case AstExprBinary::Op::CompareGt: + case AstExprBinary::Op::CompareLe: + case AstExprBinary::Op::CompareLt: + if ((isNumber(leftType) && isNumber(rightType)) || (isString(leftType) && isString(rightType))) + { + asMutable(resultType)->ty.emplace(singletonTypes->booleanType); + unblock(resultType); + return true; + } + + break; + // == and ~= always evaluate to a boolean, and impose no other constraints + // on their parameters. + case AstExprBinary::Op::CompareEq: + case AstExprBinary::Op::CompareNe: + asMutable(resultType)->ty.emplace(singletonTypes->booleanType); + unblock(resultType); + return true; + // And evalutes to a boolean if the LHS is falsey, and the RHS type if LHS is + // truthy. + case AstExprBinary::Op::And: + asMutable(resultType)->ty.emplace(unionOfTypes(rightType, singletonTypes->booleanType, constraint->scope, false)); + unblock(resultType); + return true; + // Or evaluates to the LHS type if the LHS is truthy, and the RHS type if + // LHS is falsey. + case AstExprBinary::Op::Or: + asMutable(resultType)->ty.emplace(unionOfTypes(rightType, leftType, constraint->scope, true)); + unblock(resultType); + return true; + default: + iceReporter.ice("Unhandled AstExprBinary::Op for binary operation", constraint->location); + break; + } + + // We failed to either evaluate a metamethod or invoke primitive behavior. + unify(leftType, errorRecoveryType(), constraint->scope); + unify(rightType, errorRecoveryType(), constraint->scope); + asMutable(resultType)->ty.emplace(errorRecoveryType()); + unblock(resultType); return true; } @@ -943,6 +1095,31 @@ bool ConstraintSolver::tryDispatch(const FunctionCallConstraint& c, NotNull callMm = findMetatableEntry(singletonTypes, errors, fn, "__call", constraint->location)) + { + std::vector args{fn}; + + for (TypeId arg : c.argsPack) + args.push_back(arg); + + TypeId instantiatedType = arena->addType(BlockedTypeVar{}); + TypeId inferredFnType = + arena->addType(FunctionTypeVar(TypeLevel{}, constraint->scope.get(), arena->addTypePack(TypePack{args, {}}), c.result)); + + // Alter the inner constraints. + LUAU_ASSERT(c.innerConstraints.size() == 2); + + asMutable(*c.innerConstraints.at(0)).c = InstantiationConstraint{instantiatedType, *callMm}; + asMutable(*c.innerConstraints.at(1)).c = SubtypeConstraint{inferredFnType, instantiatedType}; + + unsolvedConstraints.insert(end(unsolvedConstraints), begin(c.innerConstraints), end(c.innerConstraints)); + + asMutable(c.result)->ty.emplace(constraint->scope); + unblock(c.result); + return true; + } + const FunctionTypeVar* ftv = get(fn); bool usedMagic = false; @@ -1059,6 +1236,29 @@ bool ConstraintSolver::tryDispatch(const HasPropConstraint& c, NotNull constraint) +{ + // TODO: Figure out exact details on when refinements need to be blocked. + // It's possible that it never needs to be, since we can just use intersection types with the discriminant type? + + if (!constraint->scope->parent) + iceReporter.ice("No parent scope"); + + std::optional previousTy = constraint->scope->parent->lookup(c.def); + if (!previousTy) + iceReporter.ice("No previous type"); + + std::optional useTy = constraint->scope->lookup(c.def); + if (!useTy) + iceReporter.ice("The def is not bound to a type"); + + TypeId resultTy = follow(*useTy); + std::vector parts{*previousTy, c.discriminantType}; + asMutable(resultTy)->ty.emplace(std::move(parts)); + + return true; +} + bool ConstraintSolver::tryDispatchIterableTable(TypeId iteratorTy, const IterableConstraint& c, NotNull constraint, bool force) { auto block_ = [&](auto&& t) { @@ -1502,4 +1702,39 @@ TypePackId ConstraintSolver::errorRecoveryTypePack() const return singletonTypes->errorRecoveryTypePack(); } +TypeId ConstraintSolver::unionOfTypes(TypeId a, TypeId b, NotNull scope, bool unifyFreeTypes) +{ + a = follow(a); + b = follow(b); + + if (unifyFreeTypes && (get(a) || get(b))) + { + Unifier u{normalizer, Mode::Strict, scope, Location{}, Covariant}; + u.useScopes = true; + u.tryUnify(b, a); + + if (u.errors.empty()) + { + u.log.commit(); + return a; + } + else + { + return singletonTypes->errorRecoveryType(singletonTypes->anyType); + } + } + + if (*a == *b) + return a; + + std::vector types = reduceUnion({a, b}); + if (types.empty()) + return singletonTypes->neverType; + + if (types.size() == 1) + return types[0]; + + return arena->addType(UnionTypeVar{types}); +} + } // namespace Luau diff --git a/Analysis/src/DataFlowGraphBuilder.cpp b/Analysis/src/DataFlowGraphBuilder.cpp new file mode 100644 index 00000000..e2c4c285 --- /dev/null +++ b/Analysis/src/DataFlowGraphBuilder.cpp @@ -0,0 +1,440 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +#include "Luau/DataFlowGraphBuilder.h" + +#include "Luau/Error.h" + +LUAU_FASTFLAG(DebugLuauFreezeArena) +LUAU_FASTFLAG(DebugLuauDeferredConstraintResolution) + +namespace Luau +{ + +std::optional DataFlowGraph::getDef(const AstExpr* expr) const +{ + if (auto def = astDefs.find(expr)) + return NotNull{*def}; + return std::nullopt; +} + +std::optional DataFlowGraph::getDef(const AstLocal* local) const +{ + if (auto def = localDefs.find(local)) + return NotNull{*def}; + return std::nullopt; +} + +std::optional DataFlowGraph::getDef(const Symbol& symbol) const +{ + if (symbol.local) + return getDef(symbol.local); + else + return std::nullopt; +} + +DataFlowGraph DataFlowGraphBuilder::build(AstStatBlock* block, NotNull handle) +{ + LUAU_ASSERT(FFlag::DebugLuauDeferredConstraintResolution); + + DataFlowGraphBuilder builder; + builder.handle = handle; + builder.visit(nullptr, block); // nullptr is the root DFG scope. + if (FFlag::DebugLuauFreezeArena) + builder.arena->allocator.freeze(); + return std::move(builder.graph); +} + +DfgScope* DataFlowGraphBuilder::childScope(DfgScope* scope) +{ + return scopes.emplace_back(new DfgScope{scope}).get(); +} + +std::optional DataFlowGraphBuilder::use(DfgScope* scope, Symbol symbol, AstExpr* e) +{ + for (DfgScope* current = scope; current; current = current->parent) + { + if (auto loc = current->bindings.find(symbol)) + { + graph.astDefs[e] = *loc; + return NotNull{*loc}; + } + } + + return std::nullopt; +} + +void DataFlowGraphBuilder::visit(DfgScope* scope, AstStatBlock* b) +{ + DfgScope* child = childScope(scope); + return visitBlockWithoutChildScope(child, b); +} + +void DataFlowGraphBuilder::visitBlockWithoutChildScope(DfgScope* scope, AstStatBlock* b) +{ + for (AstStat* s : b->body) + visit(scope, s); +} + +void DataFlowGraphBuilder::visit(DfgScope* scope, AstStat* s) +{ + if (auto b = s->as()) + return visit(scope, b); + else if (auto i = s->as()) + return visit(scope, i); + else if (auto w = s->as()) + return visit(scope, w); + else if (auto r = s->as()) + return visit(scope, r); + else if (auto b = s->as()) + return visit(scope, b); + else if (auto c = s->as()) + return visit(scope, c); + else if (auto r = s->as()) + return visit(scope, r); + else if (auto e = s->as()) + return visit(scope, e); + else if (auto l = s->as()) + return visit(scope, l); + else if (auto f = s->as()) + return visit(scope, f); + else if (auto f = s->as()) + return visit(scope, f); + else if (auto a = s->as()) + return visit(scope, a); + else if (auto c = s->as()) + return visit(scope, c); + else if (auto f = s->as()) + return visit(scope, f); + else if (auto l = s->as()) + return visit(scope, l); + else if (auto t = s->as()) + return; // ok + else if (auto d = s->as()) + return; // ok + else if (auto d = s->as()) + return; // ok + else if (auto d = s->as()) + return; // ok + else if (auto d = s->as()) + return; // ok + else if (auto _ = s->as()) + return; // ok + else + handle->ice("Unknown AstStat in DataFlowGraphBuilder"); +} + +void DataFlowGraphBuilder::visit(DfgScope* scope, AstStatIf* i) +{ + DfgScope* condScope = childScope(scope); + visitExpr(condScope, i->condition); + visit(condScope, i->thenbody); + + if (i->elsebody) + visit(scope, i->elsebody); +} + +void DataFlowGraphBuilder::visit(DfgScope* scope, AstStatWhile* w) +{ + // TODO(controlflow): entry point has a back edge from exit point + DfgScope* whileScope = childScope(scope); + visitExpr(whileScope, w->condition); + visit(whileScope, w->body); +} + +void DataFlowGraphBuilder::visit(DfgScope* scope, AstStatRepeat* r) +{ + // TODO(controlflow): entry point has a back edge from exit point + DfgScope* repeatScope = childScope(scope); // TODO: loop scope. + visitBlockWithoutChildScope(repeatScope, r->body); + visitExpr(repeatScope, r->condition); +} + +void DataFlowGraphBuilder::visit(DfgScope* scope, AstStatBreak* b) +{ + // TODO: Control flow analysis + return; // ok +} + +void DataFlowGraphBuilder::visit(DfgScope* scope, AstStatContinue* c) +{ + // TODO: Control flow analysis + return; // ok +} + +void DataFlowGraphBuilder::visit(DfgScope* scope, AstStatReturn* r) +{ + // TODO: Control flow analysis + for (AstExpr* e : r->list) + visitExpr(scope, e); +} + +void DataFlowGraphBuilder::visit(DfgScope* scope, AstStatExpr* e) +{ + visitExpr(scope, e->expr); +} + +void DataFlowGraphBuilder::visit(DfgScope* scope, AstStatLocal* l) +{ + // TODO: alias tracking + for (AstExpr* e : l->values) + visitExpr(scope, e); + + for (AstLocal* local : l->vars) + { + DefId def = arena->freshDef(); + graph.localDefs[local] = def; + scope->bindings[local] = def; + } +} + +void DataFlowGraphBuilder::visit(DfgScope* scope, AstStatFor* f) +{ + DfgScope* forScope = childScope(scope); // TODO: loop scope. + DefId def = arena->freshDef(); + graph.localDefs[f->var] = def; + scope->bindings[f->var] = def; + + // TODO(controlflow): entry point has a back edge from exit point + visit(forScope, f->body); +} + +void DataFlowGraphBuilder::visit(DfgScope* scope, AstStatForIn* f) +{ + DfgScope* forScope = childScope(scope); // TODO: loop scope. + + for (AstLocal* local : f->vars) + { + DefId def = arena->freshDef(); + graph.localDefs[local] = def; + forScope->bindings[local] = def; + } + + // TODO(controlflow): entry point has a back edge from exit point + for (AstExpr* e : f->values) + visitExpr(forScope, e); + + visit(forScope, f->body); +} + +void DataFlowGraphBuilder::visit(DfgScope* scope, AstStatAssign* a) +{ + for (AstExpr* r : a->values) + visitExpr(scope, r); + + for (AstExpr* l : a->vars) + { + AstExpr* root = l; + + bool isUpdatable = true; + while (true) + { + if (root->is() || root->is()) + break; + + AstExprIndexName* indexName = root->as(); + if (!indexName) + { + isUpdatable = false; + break; + } + + root = indexName->expr; + } + + if (isUpdatable) + { + // TODO global? + if (auto exprLocal = root->as()) + { + DefId def = arena->freshDef(); + graph.astDefs[exprLocal] = def; + + // Update the def in the scope that introduced the local. Not + // the current scope. + AstLocal* local = exprLocal->local; + DfgScope* s = scope; + while (s && !s->bindings.find(local)) + s = s->parent; + LUAU_ASSERT(s && s->bindings.find(local)); + s->bindings[local] = def; + } + } + + visitExpr(scope, l); // TODO: they point to a new def!! + } +} + +void DataFlowGraphBuilder::visit(DfgScope* scope, AstStatCompoundAssign* c) +{ + // TODO(typestates): The lhs is being read and written to. This might or might not be annoying. + visitExpr(scope, c->value); +} + +void DataFlowGraphBuilder::visit(DfgScope* scope, AstStatFunction* f) +{ + visitExpr(scope, f->name); + visitExpr(scope, f->func); +} + +void DataFlowGraphBuilder::visit(DfgScope* scope, AstStatLocalFunction* l) +{ + DefId def = arena->freshDef(); + graph.localDefs[l->name] = def; + scope->bindings[l->name] = def; + + visitExpr(scope, l->func); +} + +ExpressionFlowGraph DataFlowGraphBuilder::visitExpr(DfgScope* scope, AstExpr* e) +{ + if (auto g = e->as()) + return visitExpr(scope, g->expr); + else if (auto c = e->as()) + return {}; // ok + else if (auto c = e->as()) + return {}; // ok + else if (auto c = e->as()) + return {}; // ok + else if (auto c = e->as()) + return {}; // ok + else if (auto l = e->as()) + return visitExpr(scope, l); + else if (auto g = e->as()) + return visitExpr(scope, g); + else if (auto v = e->as()) + return {}; // ok + else if (auto c = e->as()) + return visitExpr(scope, c); + else if (auto i = e->as()) + return visitExpr(scope, i); + else if (auto i = e->as()) + return visitExpr(scope, i); + else if (auto f = e->as()) + return visitExpr(scope, f); + else if (auto t = e->as()) + return visitExpr(scope, t); + else if (auto u = e->as()) + return visitExpr(scope, u); + else if (auto b = e->as()) + return visitExpr(scope, b); + else if (auto t = e->as()) + return visitExpr(scope, t); + else if (auto i = e->as()) + return visitExpr(scope, i); + else if (auto i = e->as()) + return visitExpr(scope, i); + else if (auto _ = e->as()) + return {}; // ok + else + handle->ice("Unknown AstExpr in DataFlowGraphBuilder"); +} + +ExpressionFlowGraph DataFlowGraphBuilder::visitExpr(DfgScope* scope, AstExprLocal* l) +{ + return {use(scope, l->local, l)}; +} + +ExpressionFlowGraph DataFlowGraphBuilder::visitExpr(DfgScope* scope, AstExprGlobal* g) +{ + return {use(scope, g->name, g)}; +} + +ExpressionFlowGraph DataFlowGraphBuilder::visitExpr(DfgScope* scope, AstExprCall* c) +{ + visitExpr(scope, c->func); + + for (AstExpr* arg : c->args) + visitExpr(scope, arg); + + return {}; +} + +ExpressionFlowGraph DataFlowGraphBuilder::visitExpr(DfgScope* scope, AstExprIndexName* i) +{ + std::optional def = visitExpr(scope, i->expr).def; + if (!def) + return {}; + + // TODO: properties for the above def. + return {}; +} + +ExpressionFlowGraph DataFlowGraphBuilder::visitExpr(DfgScope* scope, AstExprIndexExpr* i) +{ + visitExpr(scope, i->expr); + visitExpr(scope, i->expr); + + if (i->index->as()) + { + // TODO: properties for the def + } + + return {}; +} + +ExpressionFlowGraph DataFlowGraphBuilder::visitExpr(DfgScope* scope, AstExprFunction* f) +{ + if (AstLocal* self = f->self) + { + DefId def = arena->freshDef(); + graph.localDefs[self] = def; + scope->bindings[self] = def; + } + + for (AstLocal* param : f->args) + { + DefId def = arena->freshDef(); + graph.localDefs[param] = def; + scope->bindings[param] = def; + } + + visit(scope, f->body); + + return {}; +} + +ExpressionFlowGraph DataFlowGraphBuilder::visitExpr(DfgScope* scope, AstExprTable* t) +{ + return {}; +} + +ExpressionFlowGraph DataFlowGraphBuilder::visitExpr(DfgScope* scope, AstExprUnary* u) +{ + visitExpr(scope, u->expr); + + return {}; +} + +ExpressionFlowGraph DataFlowGraphBuilder::visitExpr(DfgScope* scope, AstExprBinary* b) +{ + visitExpr(scope, b->left); + visitExpr(scope, b->right); + + return {}; +} + +ExpressionFlowGraph DataFlowGraphBuilder::visitExpr(DfgScope* scope, AstExprTypeAssertion* t) +{ + ExpressionFlowGraph result = visitExpr(scope, t->expr); + // TODO: visit type + return result; +} + +ExpressionFlowGraph DataFlowGraphBuilder::visitExpr(DfgScope* scope, AstExprIfElse* i) +{ + DfgScope* condScope = childScope(scope); + visitExpr(condScope, i->condition); + visitExpr(condScope, i->trueExpr); + + visitExpr(scope, i->falseExpr); + + return {}; +} + +ExpressionFlowGraph DataFlowGraphBuilder::visitExpr(DfgScope* scope, AstExprInterpString* i) +{ + for (AstExpr* e : i->expressions) + visitExpr(scope, e); + return {}; +} + +} // namespace Luau diff --git a/Analysis/src/Def.cpp b/Analysis/src/Def.cpp new file mode 100644 index 00000000..935301c8 --- /dev/null +++ b/Analysis/src/Def.cpp @@ -0,0 +1,12 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +#include "Luau/Def.h" + +namespace Luau +{ + +DefId DefArena::freshDef() +{ + return NotNull{allocator.allocate(Undefined{})}; +} + +} // namespace Luau diff --git a/Analysis/src/Error.cpp b/Analysis/src/Error.cpp index 4e9b6882..e5553003 100644 --- a/Analysis/src/Error.cpp +++ b/Analysis/src/Error.cpp @@ -7,8 +7,6 @@ #include -LUAU_FASTFLAGVARIABLE(LuauTypeMismatchModuleNameResolution, false) - static std::string wrongNumberOfArgsString( size_t expectedCount, std::optional maximumCount, size_t actualCount, const char* argPrefix = nullptr, bool isVariadic = false) { @@ -70,7 +68,7 @@ struct ErrorConverter { if (auto wantedDefinitionModule = getDefinitionModuleName(tm.wantedType)) { - if (FFlag::LuauTypeMismatchModuleNameResolution && fileResolver != nullptr) + if (fileResolver != nullptr) { std::string givenModuleName = fileResolver->getHumanReadableModuleName(*givenDefinitionModule); std::string wantedModuleName = fileResolver->getHumanReadableModuleName(*wantedDefinitionModule); @@ -96,14 +94,7 @@ struct ErrorConverter if (!tm.reason.empty()) result += tm.reason + " "; - if (FFlag::LuauTypeMismatchModuleNameResolution) - { - result += Luau::toString(*tm.error, TypeErrorToStringOptions{fileResolver}); - } - else - { - result += Luau::toString(*tm.error); - } + result += Luau::toString(*tm.error, TypeErrorToStringOptions{fileResolver}); } else if (!tm.reason.empty()) { diff --git a/Analysis/src/Frontend.cpp b/Analysis/src/Frontend.cpp index cfe710d9..e059a463 100644 --- a/Analysis/src/Frontend.cpp +++ b/Analysis/src/Frontend.cpp @@ -1,11 +1,13 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #include "Luau/Frontend.h" +#include "Luau/BuiltinDefinitions.h" #include "Luau/Clone.h" #include "Luau/Common.h" #include "Luau/Config.h" #include "Luau/ConstraintGraphBuilder.h" #include "Luau/ConstraintSolver.h" +#include "Luau/DataFlowGraphBuilder.h" #include "Luau/DcrLogger.h" #include "Luau/FileResolver.h" #include "Luau/Parser.h" @@ -15,7 +17,6 @@ #include "Luau/TypeChecker2.h" #include "Luau/TypeInfer.h" #include "Luau/Variant.h" -#include "Luau/BuiltinDefinitions.h" #include #include @@ -26,7 +27,6 @@ LUAU_FASTINT(LuauTarjanChildLimit) LUAU_FASTFLAG(LuauInferInNoCheckMode) LUAU_FASTFLAG(LuauNoMoreGlobalSingletonTypes) LUAU_FASTFLAGVARIABLE(LuauKnowsTheDataModel3, false) -LUAU_FASTFLAGVARIABLE(LuauAutocompleteDynamicLimits, false) LUAU_FASTINTVARIABLE(LuauAutocompleteCheckTimeoutMs, 100) LUAU_FASTFLAGVARIABLE(DebugLuauDeferredConstraintResolution, false) LUAU_FASTFLAG(DebugLuauLogSolverToJson); @@ -489,23 +489,19 @@ CheckResult Frontend::check(const ModuleName& name, std::optional 0) - typeCheckerForAutocomplete.instantiationChildLimit = - std::max(1, int(FInt::LuauTarjanChildLimit * sourceNode.autocompleteLimitsMult)); - else - typeCheckerForAutocomplete.instantiationChildLimit = std::nullopt; + // TODO: This is a dirty ad hoc solution for autocomplete timeouts + // We are trying to dynamically adjust our existing limits to lower total typechecking time under the limit + // so that we'll have type information for the whole file at lower quality instead of a full abort in the middle + if (FInt::LuauTarjanChildLimit > 0) + typeCheckerForAutocomplete.instantiationChildLimit = std::max(1, int(FInt::LuauTarjanChildLimit * sourceNode.autocompleteLimitsMult)); + else + typeCheckerForAutocomplete.instantiationChildLimit = std::nullopt; - if (FInt::LuauTypeInferIterationLimit > 0) - typeCheckerForAutocomplete.unifierIterationLimit = - std::max(1, int(FInt::LuauTypeInferIterationLimit * sourceNode.autocompleteLimitsMult)); - else - typeCheckerForAutocomplete.unifierIterationLimit = std::nullopt; - } + if (FInt::LuauTypeInferIterationLimit > 0) + typeCheckerForAutocomplete.unifierIterationLimit = + std::max(1, int(FInt::LuauTypeInferIterationLimit * sourceNode.autocompleteLimitsMult)); + else + typeCheckerForAutocomplete.unifierIterationLimit = std::nullopt; ModulePtr moduleForAutocomplete = FFlag::DebugLuauDeferredConstraintResolution ? check(sourceModule, mode, environmentScope, requireCycles, /*forAutocomplete*/ true) @@ -519,10 +515,9 @@ CheckResult Frontend::check(const ModuleName& name, std::optional mr{forAutocomplete ? &moduleResolverForAutocomplete : &moduleResolver}; const ScopePtr& globalScope{forAutocomplete ? typeCheckerForAutocomplete.globalScope : typeChecker.globalScope}; Normalizer normalizer{&result->internalTypes, singletonTypes, NotNull{&typeChecker.unifierState}}; ConstraintGraphBuilder cgb{ - sourceModule.name, result, &result->internalTypes, mr, singletonTypes, NotNull(&iceHandler), globalScope, logger.get()}; + sourceModule.name, + result, + &result->internalTypes, + mr, + singletonTypes, + NotNull(&iceHandler), + globalScope, + logger.get(), + NotNull{&dfg}, + }; + cgb.visit(sourceModule.root); result->errors = std::move(cgb.errors); diff --git a/Analysis/src/Module.cpp b/Analysis/src/Module.cpp index 31a089a4..0412f007 100644 --- a/Analysis/src/Module.cpp +++ b/Analysis/src/Module.cpp @@ -60,36 +60,6 @@ bool isWithinComment(const SourceModule& sourceModule, Position pos) return contains(pos, *iter); } -struct ForceNormal : TypeVarOnceVisitor -{ - const TypeArena* typeArena = nullptr; - - ForceNormal(const TypeArena* typeArena) - : typeArena(typeArena) - { - } - - bool visit(TypeId ty) override - { - if (ty->owningArena != typeArena) - return false; - - asMutable(ty)->normal = true; - return true; - } - - bool visit(TypeId ty, const FreeTypeVar& ftv) override - { - visit(ty); - return true; - } - - bool visit(TypePackId tp, const FreeTypePack& ftp) override - { - return true; - } -}; - struct ClonePublicInterface : Substitution { NotNull singletonTypes; @@ -241,8 +211,6 @@ void Module::clonePublicInterface(NotNull singletonTypes, Intern moduleScope->varargPack = varargPack; } - ForceNormal forceNormal{&interfaceTypes}; - if (exportedTypeBindings) { for (auto& [name, tf] : *exportedTypeBindings) @@ -262,7 +230,6 @@ void Module::clonePublicInterface(NotNull singletonTypes, Intern { auto t = asMutable(ty); t->ty = AnyTypeVar{}; - t->normal = true; } } } diff --git a/Analysis/src/Normalize.cpp b/Analysis/src/Normalize.cpp index 81114b76..cea159c3 100644 --- a/Analysis/src/Normalize.cpp +++ b/Analysis/src/Normalize.cpp @@ -16,11 +16,11 @@ LUAU_FASTFLAGVARIABLE(DebugLuauCheckNormalizeInvariant, false) // This could theoretically be 2000 on amd64, but x86 requires this. LUAU_FASTINTVARIABLE(LuauNormalizeIterationLimit, 1200); LUAU_FASTINTVARIABLE(LuauNormalizeCacheLimit, 100000); -LUAU_FASTINT(LuauTypeInferRecursionLimit); LUAU_FASTFLAGVARIABLE(LuauNormalizeCombineTableFix, false); LUAU_FASTFLAGVARIABLE(LuauTypeNormalization2, false); LUAU_FASTFLAG(LuauUnknownAndNeverType) LUAU_FASTFLAG(DebugLuauDeferredConstraintResolution) +LUAU_FASTFLAG(LuauOverloadedFunctionSubtypingPerf); namespace Luau { @@ -1269,19 +1269,35 @@ std::optional Normalizer::intersectionOfFunctions(TypeId here, TypeId th return std::nullopt; if (hftv->genericPacks != tftv->genericPacks) return std::nullopt; - if (hftv->retTypes != tftv->retTypes) + + TypePackId argTypes; + TypePackId retTypes; + + if (hftv->retTypes == tftv->retTypes) + { + std::optional argTypesOpt = unionOfTypePacks(hftv->argTypes, tftv->argTypes); + if (!argTypesOpt) + return std::nullopt; + argTypes = *argTypesOpt; + retTypes = hftv->retTypes; + } + else if (FFlag::LuauOverloadedFunctionSubtypingPerf && hftv->argTypes == tftv->argTypes) + { + std::optional retTypesOpt = intersectionOfTypePacks(hftv->argTypes, tftv->argTypes); + if (!retTypesOpt) + return std::nullopt; + argTypes = hftv->argTypes; + retTypes = *retTypesOpt; + } + else return std::nullopt; - std::optional argTypes = unionOfTypePacks(hftv->argTypes, tftv->argTypes); - if (!argTypes) - return std::nullopt; - - if (*argTypes == hftv->argTypes) + if (argTypes == hftv->argTypes && retTypes == hftv->retTypes) return here; - if (*argTypes == tftv->argTypes) + if (argTypes == tftv->argTypes && retTypes == tftv->retTypes) return there; - FunctionTypeVar result{*argTypes, hftv->retTypes}; + FunctionTypeVar result{argTypes, retTypes}; result.generics = hftv->generics; result.genericPacks = hftv->genericPacks; return arena->addType(std::move(result)); @@ -1762,610 +1778,4 @@ bool isSubtype( return ok; } -template -static bool areNormal_(const T& t, const std::unordered_set& seen, InternalErrorReporter& ice) -{ - int count = 0; - auto isNormal = [&](TypeId ty) { - ++count; - if (count >= FInt::LuauNormalizeIterationLimit) - ice.ice("Luau::areNormal hit iteration limit"); - - return ty->normal; - }; - - return std::all_of(begin(t), end(t), isNormal); -} - -static bool areNormal(const std::vector& types, const std::unordered_set& seen, InternalErrorReporter& ice) -{ - return areNormal_(types, seen, ice); -} - -static bool areNormal(TypePackId tp, const std::unordered_set& seen, InternalErrorReporter& ice) -{ - tp = follow(tp); - if (get(tp)) - return false; - - auto [head, tail] = flatten(tp); - - if (!areNormal_(head, seen, ice)) - return false; - - if (!tail) - return true; - - if (auto vtp = get(*tail)) - return vtp->ty->normal || follow(vtp->ty)->normal || seen.find(asMutable(vtp->ty)) != seen.end(); - - return true; -} - -#define CHECK_ITERATION_LIMIT(...) \ - do \ - { \ - if (iterationLimit > FInt::LuauNormalizeIterationLimit) \ - { \ - limitExceeded = true; \ - return __VA_ARGS__; \ - } \ - ++iterationLimit; \ - } while (false) - -struct Normalize final : TypeVarVisitor -{ - using TypeVarVisitor::Set; - - Normalize(TypeArena& arena, NotNull scope, NotNull singletonTypes, InternalErrorReporter& ice) - : arena(arena) - , scope(scope) - , singletonTypes(singletonTypes) - , ice(ice) - { - } - - TypeArena& arena; - NotNull scope; - NotNull singletonTypes; - InternalErrorReporter& ice; - - int iterationLimit = 0; - bool limitExceeded = false; - - bool visit(TypeId ty, const FreeTypeVar&) override - { - LUAU_ASSERT(!ty->normal); - return false; - } - - bool visit(TypeId ty, const BoundTypeVar& btv) override - { - // A type could be considered normal when it is in the stack, but we will eventually find out it is not normal as normalization progresses. - // So we need to avoid eagerly saying that this bound type is normal if the thing it is bound to is in the stack. - if (seen.find(asMutable(btv.boundTo)) != seen.end()) - return false; - - // It should never be the case that this TypeVar is normal, but is bound to a non-normal type, except in nontrivial cases. - LUAU_ASSERT(!ty->normal || ty->normal == btv.boundTo->normal); - - if (!ty->normal) - asMutable(ty)->normal = btv.boundTo->normal; - return !ty->normal; - } - - bool visit(TypeId ty, const PrimitiveTypeVar&) override - { - LUAU_ASSERT(ty->normal); - return false; - } - - bool visit(TypeId ty, const GenericTypeVar&) override - { - if (!ty->normal) - asMutable(ty)->normal = true; - return false; - } - - bool visit(TypeId ty, const ErrorTypeVar&) override - { - if (!ty->normal) - asMutable(ty)->normal = true; - return false; - } - - bool visit(TypeId ty, const UnknownTypeVar&) override - { - if (!ty->normal) - asMutable(ty)->normal = true; - return false; - } - - bool visit(TypeId ty, const NeverTypeVar&) override - { - if (!ty->normal) - asMutable(ty)->normal = true; - return false; - } - - bool visit(TypeId ty, const ConstrainedTypeVar& ctvRef) override - { - CHECK_ITERATION_LIMIT(false); - LUAU_ASSERT(!ty->normal); - - ConstrainedTypeVar* ctv = const_cast(&ctvRef); - - std::vector parts = std::move(ctv->parts); - - // We might transmute, so it's not safe to rely on the builtin traversal logic of visitTypeVar - for (TypeId part : parts) - traverse(part); - - std::vector newParts = normalizeUnion(parts); - ctv->parts = std::move(newParts); - - return false; - } - - bool visit(TypeId ty, const FunctionTypeVar& ftv) override - { - CHECK_ITERATION_LIMIT(false); - - if (ty->normal) - return false; - - traverse(ftv.argTypes); - traverse(ftv.retTypes); - - asMutable(ty)->normal = areNormal(ftv.argTypes, seen, ice) && areNormal(ftv.retTypes, seen, ice); - - return false; - } - - bool visit(TypeId ty, const TableTypeVar& ttv) override - { - CHECK_ITERATION_LIMIT(false); - - if (ty->normal) - return false; - - bool normal = true; - - auto checkNormal = [&](TypeId t) { - // if t is on the stack, it is possible that this type is normal. - // If t is not normal and it is not on the stack, this type is definitely not normal. - if (!t->normal && seen.find(asMutable(t)) == seen.end()) - normal = false; - }; - - if (ttv.boundTo) - { - traverse(*ttv.boundTo); - asMutable(ty)->normal = (*ttv.boundTo)->normal; - return false; - } - - for (const auto& [_name, prop] : ttv.props) - { - traverse(prop.type); - checkNormal(prop.type); - } - - if (ttv.indexer) - { - traverse(ttv.indexer->indexType); - checkNormal(ttv.indexer->indexType); - traverse(ttv.indexer->indexResultType); - checkNormal(ttv.indexer->indexResultType); - } - - // An unsealed table can never be normal, ditto for free tables iff the type it is bound to is also not normal. - if (ttv.state == TableState::Generic || ttv.state == TableState::Sealed || (ttv.state == TableState::Free && follow(ty)->normal)) - asMutable(ty)->normal = normal; - - return false; - } - - bool visit(TypeId ty, const MetatableTypeVar& mtv) override - { - CHECK_ITERATION_LIMIT(false); - - if (ty->normal) - return false; - - traverse(mtv.table); - traverse(mtv.metatable); - - asMutable(ty)->normal = mtv.table->normal && mtv.metatable->normal; - - return false; - } - - bool visit(TypeId ty, const ClassTypeVar& ctv) override - { - if (!ty->normal) - asMutable(ty)->normal = true; - return false; - } - - bool visit(TypeId ty, const AnyTypeVar&) override - { - LUAU_ASSERT(ty->normal); - return false; - } - - bool visit(TypeId ty, const UnionTypeVar& utvRef) override - { - CHECK_ITERATION_LIMIT(false); - - if (ty->normal) - return false; - - UnionTypeVar* utv = &const_cast(utvRef); - - // We might transmute, so it's not safe to rely on the builtin traversal logic of visitTypeVar - for (TypeId option : utv->options) - traverse(option); - - std::vector newOptions = normalizeUnion(utv->options); - - const bool normal = areNormal(newOptions, seen, ice); - - LUAU_ASSERT(!newOptions.empty()); - - if (newOptions.size() == 1) - *asMutable(ty) = BoundTypeVar{newOptions[0]}; - else - utv->options = std::move(newOptions); - - asMutable(ty)->normal = normal; - - return false; - } - - bool visit(TypeId ty, const IntersectionTypeVar& itvRef) override - { - CHECK_ITERATION_LIMIT(false); - - if (ty->normal) - return false; - - IntersectionTypeVar* itv = &const_cast(itvRef); - - std::vector oldParts = itv->parts; - IntersectionTypeVar newIntersection; - - for (TypeId part : oldParts) - traverse(part); - - std::vector tables; - for (TypeId part : oldParts) - { - part = follow(part); - if (get(part)) - tables.push_back(part); - else - { - Replacer replacer{&arena, nullptr, nullptr}; // FIXME this is super super WEIRD - combineIntoIntersection(replacer, &newIntersection, part); - } - } - - // Don't allocate a new table if there's just one in the intersection. - if (tables.size() == 1) - newIntersection.parts.push_back(tables[0]); - else if (!tables.empty()) - { - const TableTypeVar* first = get(tables[0]); - LUAU_ASSERT(first); - - TypeId newTable = arena.addType(TableTypeVar{first->state, first->level}); - TableTypeVar* ttv = getMutable(newTable); - for (TypeId part : tables) - { - // Intuition: If combineIntoTable() needs to clone a table, any references to 'part' are cyclic and need - // to be rewritten to point at 'newTable' in the clone. - Replacer replacer{&arena, part, newTable}; - combineIntoTable(replacer, ttv, part); - } - - newIntersection.parts.push_back(newTable); - } - - itv->parts = std::move(newIntersection.parts); - - asMutable(ty)->normal = areNormal(itv->parts, seen, ice); - - if (itv->parts.size() == 1) - { - TypeId part = itv->parts[0]; - *asMutable(ty) = BoundTypeVar{part}; - } - - return false; - } - - std::vector normalizeUnion(const std::vector& options) - { - if (options.size() == 1) - return options; - - std::vector result; - - for (TypeId part : options) - { - // AnyTypeVar always win the battle no matter what we do, so we're done. - if (FFlag::LuauUnknownAndNeverType && get(follow(part))) - return {part}; - - combineIntoUnion(result, part); - } - - return result; - } - - void combineIntoUnion(std::vector& result, TypeId ty) - { - ty = follow(ty); - if (auto utv = get(ty)) - { - for (TypeId t : utv) - { - // AnyTypeVar always win the battle no matter what we do, so we're done. - if (FFlag::LuauUnknownAndNeverType && get(t)) - { - result = {t}; - return; - } - - combineIntoUnion(result, t); - } - - return; - } - - for (TypeId& part : result) - { - if (isSubtype(ty, part, scope, singletonTypes, ice)) - return; // no need to do anything - else if (isSubtype(part, ty, scope, singletonTypes, ice)) - { - part = ty; // replace the less general type by the more general one - return; - } - } - - result.push_back(ty); - } - - /** - * @param replacer knows how to clone a type such that any recursive references point at the new containing type. - * @param result is an intersection that is safe for us to mutate in-place. - */ - void combineIntoIntersection(Replacer& replacer, IntersectionTypeVar* result, TypeId ty) - { - // Note: this check guards against running out of stack space - // so if you increase the size of a stack frame, you'll need to decrease the limit. - CHECK_ITERATION_LIMIT(); - - ty = follow(ty); - if (auto itv = get(ty)) - { - for (TypeId part : itv->parts) - combineIntoIntersection(replacer, result, part); - return; - } - - // Let's say that the last part of our result intersection is always a table, if any table is part of this intersection - if (get(ty)) - { - if (result->parts.empty()) - result->parts.push_back(arena.addType(TableTypeVar{TableState::Sealed, TypeLevel{}})); - - TypeId theTable = result->parts.back(); - - if (!get(follow(theTable))) - { - result->parts.push_back(arena.addType(TableTypeVar{TableState::Sealed, TypeLevel{}})); - theTable = result->parts.back(); - } - - TypeId newTable = replacer.smartClone(theTable); - result->parts.back() = newTable; - - combineIntoTable(replacer, getMutable(newTable), ty); - } - else if (auto ftv = get(ty)) - { - bool merged = false; - for (TypeId& part : result->parts) - { - if (isSubtype(part, ty, scope, singletonTypes, ice)) - { - merged = true; - break; // no need to do anything - } - else if (isSubtype(ty, part, scope, singletonTypes, ice)) - { - merged = true; - part = ty; // replace the less general type by the more general one - break; - } - } - - if (!merged) - result->parts.push_back(ty); - } - else - result->parts.push_back(ty); - } - - TableState combineTableStates(TableState lhs, TableState rhs) - { - if (lhs == rhs) - return lhs; - - if (lhs == TableState::Free || rhs == TableState::Free) - return TableState::Free; - - if (lhs == TableState::Unsealed || rhs == TableState::Unsealed) - return TableState::Unsealed; - - return lhs; - } - - /** - * @param replacer gives us a way to clone a type such that recursive references are rewritten to the new - * "containing" type. - * @param table always points into a table that is safe for us to mutate. - */ - void combineIntoTable(Replacer& replacer, TableTypeVar* table, TypeId ty) - { - // Note: this check guards against running out of stack space - // so if you increase the size of a stack frame, you'll need to decrease the limit. - CHECK_ITERATION_LIMIT(); - - LUAU_ASSERT(table); - - ty = follow(ty); - - TableTypeVar* tyTable = getMutable(ty); - LUAU_ASSERT(tyTable); - - for (const auto& [propName, prop] : tyTable->props) - { - if (auto it = table->props.find(propName); it != table->props.end()) - { - /** - * If we are going to recursively merge intersections of tables, we need to ensure that we never mutate - * a table that comes from somewhere else in the type graph. - * - * smarClone() does some nice things for us: It will perform a clone that is as shallow as possible - * while still rewriting any cyclic references back to the new 'root' table. - * - * replacer also keeps a mapping of types that have previously been copied, so we have the added - * advantage here of knowing that, whether or not a new copy was actually made, the resulting TypeVar is - * safe for us to mutate in-place. - */ - TypeId clone = replacer.smartClone(it->second.type); - it->second.type = combine(replacer, clone, prop.type); - } - else - table->props.insert({propName, prop}); - } - - if (tyTable->indexer) - { - if (table->indexer) - { - table->indexer->indexType = combine(replacer, replacer.smartClone(tyTable->indexer->indexType), table->indexer->indexType); - table->indexer->indexResultType = - combine(replacer, replacer.smartClone(tyTable->indexer->indexResultType), table->indexer->indexResultType); - } - else - { - table->indexer = - TableIndexer{replacer.smartClone(tyTable->indexer->indexType), replacer.smartClone(tyTable->indexer->indexResultType)}; - } - } - - table->state = combineTableStates(table->state, tyTable->state); - table->level = max(table->level, tyTable->level); - } - - /** - * @param a is always cloned by the caller. It is safe to mutate in-place. - * @param b will never be mutated. - */ - TypeId combine(Replacer& replacer, TypeId a, TypeId b) - { - b = follow(b); - - if (FFlag::LuauNormalizeCombineTableFix && a == b) - return a; - - if (!get(a) && !get(a)) - { - if (!FFlag::LuauNormalizeCombineTableFix && a == b) - return a; - else - return arena.addType(IntersectionTypeVar{{a, b}}); - } - - if (auto itv = getMutable(a)) - { - combineIntoIntersection(replacer, itv, b); - return a; - } - else if (auto ttv = getMutable(a)) - { - if (FFlag::LuauNormalizeCombineTableFix && !get(b)) - return arena.addType(IntersectionTypeVar{{a, b}}); - combineIntoTable(replacer, ttv, b); - return a; - } - - LUAU_ASSERT(!"Impossible"); - LUAU_UNREACHABLE(); - } -}; - -#undef CHECK_ITERATION_LIMIT - -/** - * @returns A tuple of TypeId and a success indicator. (true indicates that the normalization completed successfully) - */ -std::pair normalize( - TypeId ty, NotNull scope, TypeArena& arena, NotNull singletonTypes, InternalErrorReporter& ice) -{ - CloneState state; - if (FFlag::DebugLuauCopyBeforeNormalizing) - (void)clone(ty, arena, state); - - Normalize n{arena, scope, singletonTypes, ice}; - n.traverse(ty); - - return {ty, !n.limitExceeded}; -} - -// TODO: Think about using a temporary arena and cloning types out of it so that we -// reclaim memory used by wantonly allocated intermediate types here. -// The main wrinkle here is that we don't want clone() to copy a type if the source and dest -// arena are the same. -std::pair normalize(TypeId ty, NotNull module, NotNull singletonTypes, InternalErrorReporter& ice) -{ - return normalize(ty, NotNull{module->getModuleScope().get()}, module->internalTypes, singletonTypes, ice); -} - -std::pair normalize(TypeId ty, const ModulePtr& module, NotNull singletonTypes, InternalErrorReporter& ice) -{ - return normalize(ty, NotNull{module.get()}, singletonTypes, ice); -} - -/** - * @returns A tuple of TypeId and a success indicator. (true indicates that the normalization completed successfully) - */ -std::pair normalize( - TypePackId tp, NotNull scope, TypeArena& arena, NotNull singletonTypes, InternalErrorReporter& ice) -{ - CloneState state; - if (FFlag::DebugLuauCopyBeforeNormalizing) - (void)clone(tp, arena, state); - - Normalize n{arena, scope, singletonTypes, ice}; - n.traverse(tp); - - return {tp, !n.limitExceeded}; -} - -std::pair normalize(TypePackId tp, NotNull module, NotNull singletonTypes, InternalErrorReporter& ice) -{ - return normalize(tp, NotNull{module->getModuleScope().get()}, module->internalTypes, singletonTypes, ice); -} - -std::pair normalize(TypePackId tp, const ModulePtr& module, NotNull singletonTypes, InternalErrorReporter& ice) -{ - return normalize(tp, NotNull{module.get()}, singletonTypes, ice); -} - } // namespace Luau diff --git a/Analysis/src/Quantify.cpp b/Analysis/src/Quantify.cpp index e4c069bd..e9de094b 100644 --- a/Analysis/src/Quantify.cpp +++ b/Analysis/src/Quantify.cpp @@ -57,29 +57,6 @@ struct Quantifier final : TypeVarOnceVisitor return false; } - bool visit(TypeId ty, const ConstrainedTypeVar&) override - { - ConstrainedTypeVar* ctv = getMutable(ty); - - seenMutableType = true; - - if (!level.subsumes(ctv->level)) - return false; - - std::vector opts = std::move(ctv->parts); - - // We might transmute, so it's not safe to rely on the builtin traversal logic - for (TypeId opt : opts) - traverse(opt); - - if (opts.size() == 1) - *asMutable(ty) = BoundTypeVar{opts[0]}; - else - *asMutable(ty) = UnionTypeVar{std::move(opts)}; - - return false; - } - bool visit(TypeId ty, const TableTypeVar&) override { LUAU_ASSERT(getMutable(ty)); diff --git a/Analysis/src/Scope.cpp b/Analysis/src/Scope.cpp index 9a7d3609..84925f79 100644 --- a/Analysis/src/Scope.cpp +++ b/Analysis/src/Scope.cpp @@ -27,6 +27,44 @@ void Scope::addBuiltinTypeBinding(const Name& name, const TypeFun& tyFun) builtinTypeNames.insert(name); } +std::optional Scope::lookup(Symbol sym) const +{ + auto r = const_cast(this)->lookupEx(sym); + if (r) + return r->first; + else + return std::nullopt; +} + +std::optional> Scope::lookupEx(Symbol sym) +{ + Scope* s = this; + + while (true) + { + auto it = s->bindings.find(sym); + if (it != s->bindings.end()) + return std::pair{it->second.typeId, s}; + + if (s->parent) + s = s->parent.get(); + else + return std::nullopt; + } +} + +// TODO: We might kill Scope::lookup(Symbol) once data flow is fully fleshed out with type states and control flow analysis. +std::optional Scope::lookup(DefId def) const +{ + for (const Scope* current = this; current; current = current->parent.get()) + { + if (auto ty = current->dcrRefinements.find(def)) + return *ty; + } + + return std::nullopt; +} + std::optional Scope::lookupType(const Name& name) { const Scope* scope = this; @@ -111,23 +149,6 @@ std::optional Scope::linearSearchForBinding(const std::string& name, bo return std::nullopt; } -std::optional Scope::lookup(Symbol sym) -{ - Scope* s = this; - - while (true) - { - auto it = s->bindings.find(sym); - if (it != s->bindings.end()) - return it->second.typeId; - - if (s->parent) - s = s->parent.get(); - else - return std::nullopt; - } -} - bool subsumesStrict(Scope* left, Scope* right) { while (right) diff --git a/Analysis/src/Substitution.cpp b/Analysis/src/Substitution.cpp index 2137d73e..20ed34f6 100644 --- a/Analysis/src/Substitution.cpp +++ b/Analysis/src/Substitution.cpp @@ -73,11 +73,6 @@ void Tarjan::visitChildren(TypeId ty, int index) for (TypeId part : itv->parts) visitChild(part); } - else if (const ConstrainedTypeVar* ctv = get(ty)) - { - for (TypeId part : ctv->parts) - visitChild(part); - } else if (const PendingExpansionTypeVar* petv = get(ty)) { for (TypeId a : petv->typeArguments) @@ -97,6 +92,10 @@ void Tarjan::visitChildren(TypeId ty, int index) if (ctv->metatable) visitChild(*ctv->metatable); } + else if (const NegationTypeVar* ntv = get(ty)) + { + visitChild(ntv->ty); + } } void Tarjan::visitChildren(TypePackId tp, int index) @@ -605,11 +604,6 @@ void Substitution::replaceChildren(TypeId ty) for (TypeId& part : itv->parts) part = replace(part); } - else if (ConstrainedTypeVar* ctv = getMutable(ty)) - { - for (TypeId& part : ctv->parts) - part = replace(part); - } else if (PendingExpansionTypeVar* petv = getMutable(ty)) { for (TypeId& a : petv->typeArguments) @@ -629,6 +623,10 @@ void Substitution::replaceChildren(TypeId ty) if (ctv->metatable) ctv->metatable = replace(*ctv->metatable); } + else if (NegationTypeVar* ntv = getMutable(ty)) + { + ntv->ty = replace(ntv->ty); + } } void Substitution::replaceChildren(TypePackId tp) diff --git a/Analysis/src/ToDot.cpp b/Analysis/src/ToDot.cpp index 0d989ca0..68fa5393 100644 --- a/Analysis/src/ToDot.cpp +++ b/Analysis/src/ToDot.cpp @@ -237,15 +237,6 @@ void StateDot::visitChildren(TypeId ty, int index) finishNodeLabel(ty); finishNode(); } - else if (const ConstrainedTypeVar* ctv = get(ty)) - { - formatAppend(result, "ConstrainedTypeVar %d", index); - finishNodeLabel(ty); - finishNode(); - - for (TypeId part : ctv->parts) - visitChild(part, index); - } else if (get(ty)) { formatAppend(result, "ErrorTypeVar %d", index); diff --git a/Analysis/src/ToString.cpp b/Analysis/src/ToString.cpp index 9572ef19..31641af8 100644 --- a/Analysis/src/ToString.cpp +++ b/Analysis/src/ToString.cpp @@ -400,29 +400,6 @@ struct TypeVarStringifier state.emit(state.getName(ty)); } - void operator()(TypeId, const ConstrainedTypeVar& ctv) - { - state.result.invalid = true; - - state.emit("["); - if (FFlag::DebugLuauVerboseTypeNames) - state.emit(ctv.level); - state.emit("["); - - bool first = true; - for (TypeId ty : ctv.parts) - { - if (first) - first = false; - else - state.emit("|"); - - stringify(ty); - } - - state.emit("]]"); - } - void operator()(TypeId, const BlockedTypeVar& btv) { state.emit("*blocked-"); @@ -871,6 +848,28 @@ struct TypeVarStringifier { state.emit("never"); } + + void operator()(TypeId ty, const UseTypeVar&) + { + stringify(follow(ty)); + } + + void operator()(TypeId, const NegationTypeVar& ntv) + { + state.emit("~"); + + // The precedence of `~` should be less than `|` and `&`. + TypeId followed = follow(ntv.ty); + bool parens = get(followed) || get(followed); + + if (parens) + state.emit("("); + + stringify(ntv.ty); + + if (parens) + state.emit(")"); + } }; struct TypePackStringifier @@ -1442,7 +1441,7 @@ std::string generateName(size_t i) std::string toString(const Constraint& constraint, ToStringOptions& opts) { - auto go = [&opts](auto&& c) { + auto go = [&opts](auto&& c) -> std::string { using T = std::decay_t; // TODO: Inline and delete this function when clipping FFlag::LuauFixNameMaps @@ -1526,6 +1525,10 @@ std::string toString(const Constraint& constraint, ToStringOptions& opts) { return tos(c.resultType, opts) + " ~ hasProp " + tos(c.subjectType, opts) + ", \"" + c.prop + "\""; } + else if constexpr (std::is_same_v) + { + return "TODO"; + } else static_assert(always_false_v, "Non-exhaustive constraint switch"); }; diff --git a/Analysis/src/TxnLog.cpp b/Analysis/src/TxnLog.cpp index 06bde195..034aeaec 100644 --- a/Analysis/src/TxnLog.cpp +++ b/Analysis/src/TxnLog.cpp @@ -251,7 +251,7 @@ PendingType* TxnLog::bindTable(TypeId ty, std::optional newBoundTo) PendingType* TxnLog::changeLevel(TypeId ty, TypeLevel newLevel) { - LUAU_ASSERT(get(ty) || get(ty) || get(ty) || get(ty)); + LUAU_ASSERT(get(ty) || get(ty) || get(ty)); PendingType* newTy = queue(ty); if (FreeTypeVar* ftv = Luau::getMutable(newTy)) @@ -267,11 +267,6 @@ PendingType* TxnLog::changeLevel(TypeId ty, TypeLevel newLevel) { ftv->level = newLevel; } - else if (ConstrainedTypeVar* ctv = Luau::getMutable(newTy)) - { - if (FFlag::LuauUnknownAndNeverType) - ctv->level = newLevel; - } return newTy; } @@ -291,7 +286,7 @@ PendingTypePack* TxnLog::changeLevel(TypePackId tp, TypeLevel newLevel) PendingType* TxnLog::changeScope(TypeId ty, NotNull newScope) { - LUAU_ASSERT(get(ty) || get(ty) || get(ty) || get(ty)); + LUAU_ASSERT(get(ty) || get(ty) || get(ty)); PendingType* newTy = queue(ty); if (FreeTypeVar* ftv = Luau::getMutable(newTy)) @@ -307,10 +302,6 @@ PendingType* TxnLog::changeScope(TypeId ty, NotNull newScope) { ftv->scope = newScope; } - else if (ConstrainedTypeVar* ctv = Luau::getMutable(newTy)) - { - ctv->scope = newScope; - } return newTy; } diff --git a/Analysis/src/TypeAttach.cpp b/Analysis/src/TypeAttach.cpp index 84494083..f2613cae 100644 --- a/Analysis/src/TypeAttach.cpp +++ b/Analysis/src/TypeAttach.cpp @@ -104,16 +104,6 @@ public: return allocator->alloc(Location(), std::nullopt, AstName("*pending-expansion*")); } - AstType* operator()(const ConstrainedTypeVar& ctv) - { - AstArray types; - types.size = ctv.parts.size(); - types.data = static_cast(allocator->allocate(sizeof(AstType*) * ctv.parts.size())); - for (size_t i = 0; i < ctv.parts.size(); ++i) - types.data[i] = Luau::visit(*this, ctv.parts[i]->ty); - return allocator->alloc(Location(), types); - } - AstType* operator()(const SingletonTypeVar& stv) { if (const BooleanSingleton* bs = get(&stv)) @@ -348,6 +338,17 @@ public: { return allocator->alloc(Location(), std::nullopt, AstName{"never"}); } + AstType* operator()(const UseTypeVar& utv) + { + std::optional ty = utv.scope->lookup(utv.def); + LUAU_ASSERT(ty); + return Luau::visit(*this, (*ty)->ty); + } + AstType* operator()(const NegationTypeVar& ntv) + { + // FIXME: do the same thing we do with ErrorTypeVar + throw std::runtime_error("Cannot convert NegationTypeVar into AstNode"); + } private: Allocator* allocator; diff --git a/Analysis/src/TypeChecker2.cpp b/Analysis/src/TypeChecker2.cpp index 4753a7c2..bd220e9c 100644 --- a/Analysis/src/TypeChecker2.cpp +++ b/Analysis/src/TypeChecker2.cpp @@ -5,6 +5,7 @@ #include "Luau/AstQuery.h" #include "Luau/Clone.h" #include "Luau/Instantiation.h" +#include "Luau/Metamethods.h" #include "Luau/Normalize.h" #include "Luau/ToString.h" #include "Luau/TxnLog.h" @@ -62,6 +63,23 @@ struct StackPusher } }; +static std::optional getIdentifierOfBaseVar(AstExpr* node) +{ + if (AstExprGlobal* expr = node->as()) + return expr->name.value; + + if (AstExprLocal* expr = node->as()) + return expr->local->name.value; + + if (AstExprIndexExpr* expr = node->as()) + return getIdentifierOfBaseVar(expr->expr); + + if (AstExprIndexName* expr = node->as()) + return getIdentifierOfBaseVar(expr->expr); + + return std::nullopt; +} + struct TypeChecker2 { NotNull singletonTypes; @@ -750,7 +768,7 @@ struct TypeChecker2 TypeId actualType = lookupType(string); TypeId stringType = singletonTypes->stringType; - if (!isSubtype(stringType, actualType, stack.back(), singletonTypes, ice, /* anyIsTop */ false)) + if (!isSubtype(actualType, stringType, stack.back(), singletonTypes, ice, /* anyIsTop */ false)) { reportError(TypeMismatch{actualType, stringType}, string->location); } @@ -783,26 +801,55 @@ struct TypeChecker2 TypePackId expectedRetType = lookupPack(call); TypeId functionType = lookupType(call->func); - LUAU_ASSERT(functionType); + TypeId testFunctionType = functionType; + TypePack args; if (get(functionType) || get(functionType)) return; - - // TODO: Lots of other types are callable: intersections of functions - // and things with the __call metamethod. - if (!get(functionType)) + else if (std::optional callMm = findMetatableEntry(singletonTypes, module->errors, functionType, "__call", call->func->location)) + { + if (get(follow(*callMm))) + { + if (std::optional instantiatedCallMm = instantiation.substitute(*callMm)) + { + args.head.push_back(functionType); + testFunctionType = follow(*instantiatedCallMm); + } + else + { + reportError(UnificationTooComplex{}, call->func->location); + return; + } + } + else + { + // TODO: This doesn't flag the __call metamethod as the problem + // very clearly. + reportError(CannotCallNonFunction{*callMm}, call->func->location); + return; + } + } + else if (get(functionType)) + { + if (std::optional instantiatedFunctionType = instantiation.substitute(functionType)) + { + testFunctionType = *instantiatedFunctionType; + } + else + { + reportError(UnificationTooComplex{}, call->func->location); + return; + } + } + else { reportError(CannotCallNonFunction{functionType}, call->func->location); return; } - TypeId instantiatedFunctionType = follow(instantiation.substitute(functionType).value_or(nullptr)); - - TypePack args; for (AstExpr* arg : call->args) { - TypeId argTy = module->astTypes[arg]; - LUAU_ASSERT(argTy); + TypeId argTy = lookupType(arg); args.head.push_back(argTy); } @@ -810,7 +857,7 @@ struct TypeChecker2 FunctionTypeVar ftv{argsTp, expectedRetType}; TypeId expectedType = arena.addType(ftv); - if (!isSubtype(instantiatedFunctionType, expectedType, stack.back(), singletonTypes, ice, /* anyIsTop */ false)) + if (!isSubtype(testFunctionType, expectedType, stack.back(), singletonTypes, ice, /* anyIsTop */ false)) { CloneState cloneState; expectedType = clone(expectedType, module->internalTypes, cloneState); @@ -893,9 +940,204 @@ struct TypeChecker2 void visit(AstExprBinary* expr) { - // TODO! visit(expr->left); visit(expr->right); + + NotNull scope = stack.back(); + + bool isEquality = expr->op == AstExprBinary::Op::CompareEq || expr->op == AstExprBinary::Op::CompareNe; + bool isComparison = expr->op >= AstExprBinary::Op::CompareEq && expr->op <= AstExprBinary::Op::CompareGe; + bool isLogical = expr->op == AstExprBinary::Op::And || expr->op == AstExprBinary::Op::Or; + + TypeId leftType = lookupType(expr->left); + TypeId rightType = lookupType(expr->right); + + if (expr->op == AstExprBinary::Op::Or) + { + leftType = stripNil(singletonTypes, module->internalTypes, leftType); + } + + bool isStringOperation = isString(leftType) && isString(rightType); + + if (get(leftType) || get(leftType) || get(rightType) || get(rightType)) + return; + + if ((get(leftType) || get(leftType)) && !isEquality && !isLogical) + { + auto name = getIdentifierOfBaseVar(expr->left); + reportError(CannotInferBinaryOperation{expr->op, name, + isComparison ? CannotInferBinaryOperation::OpKind::Comparison : CannotInferBinaryOperation::OpKind::Operation}, + expr->location); + return; + } + + if (auto it = kBinaryOpMetamethods.find(expr->op); it != kBinaryOpMetamethods.end()) + { + std::optional leftMt = getMetatable(leftType, singletonTypes); + std::optional rightMt = getMetatable(rightType, singletonTypes); + + bool matches = leftMt == rightMt; + if (isEquality && !matches) + { + auto testUnion = [&matches, singletonTypes = this->singletonTypes](const UnionTypeVar* utv, std::optional otherMt) { + for (TypeId option : utv) + { + if (getMetatable(follow(option), singletonTypes) == otherMt) + { + matches = true; + break; + } + } + }; + + if (const UnionTypeVar* utv = get(leftType); utv && rightMt) + { + testUnion(utv, rightMt); + } + + if (const UnionTypeVar* utv = get(rightType); utv && leftMt && !matches) + { + testUnion(utv, leftMt); + } + } + + if (!matches && isComparison) + { + reportError(GenericError{format("Types %s and %s cannot be compared with %s because they do not have the same metatable", + toString(leftType).c_str(), toString(rightType).c_str(), toString(expr->op).c_str())}, + expr->location); + + return; + } + + std::optional mm; + if (std::optional leftMm = findMetatableEntry(singletonTypes, module->errors, leftType, it->second, expr->left->location)) + mm = leftMm; + else if (std::optional rightMm = findMetatableEntry(singletonTypes, module->errors, rightType, it->second, expr->right->location)) + mm = rightMm; + + if (mm) + { + if (const FunctionTypeVar* ftv = get(*mm)) + { + TypePackId expectedArgs; + // For >= and > we invoke __lt and __le respectively with + // swapped argument ordering. + if (expr->op == AstExprBinary::Op::CompareGe || expr->op == AstExprBinary::Op::CompareGt) + { + expectedArgs = module->internalTypes.addTypePack({rightType, leftType}); + } + else + { + expectedArgs = module->internalTypes.addTypePack({leftType, rightType}); + } + + reportErrors(tryUnify(scope, expr->location, ftv->argTypes, expectedArgs)); + + if (expr->op == AstExprBinary::CompareEq || expr->op == AstExprBinary::CompareNe || expr->op == AstExprBinary::CompareGe || + expr->op == AstExprBinary::CompareGt || expr->op == AstExprBinary::Op::CompareLe || expr->op == AstExprBinary::Op::CompareLt) + { + TypePackId expectedRets = module->internalTypes.addTypePack({singletonTypes->booleanType}); + if (!isSubtype(ftv->retTypes, expectedRets, scope, singletonTypes, ice)) + { + reportError(GenericError{format("Metamethod '%s' must return type 'boolean'", it->second)}, expr->location); + } + } + else if (!first(ftv->retTypes)) + { + reportError(GenericError{format("Metamethod '%s' must return a value", it->second)}, expr->location); + } + } + else + { + reportError(CannotCallNonFunction{*mm}, expr->location); + } + + return; + } + // If this is a string comparison, or a concatenation of strings, we + // want to fall through to primitive behavior. + else if (!isEquality && !(isStringOperation && (expr->op == AstExprBinary::Op::Concat || isComparison))) + { + if (leftMt || rightMt) + { + if (isComparison) + { + reportError(GenericError{format( + "Types '%s' and '%s' cannot be compared with %s because neither type's metatable has a '%s' metamethod", + toString(leftType).c_str(), toString(rightType).c_str(), toString(expr->op).c_str(), it->second)}, + expr->location); + } + else + { + reportError(GenericError{format( + "Operator %s is not applicable for '%s' and '%s' because neither type's metatable has a '%s' metamethod", + toString(expr->op).c_str(), toString(leftType).c_str(), toString(rightType).c_str(), it->second)}, + expr->location); + } + + return; + } + else if (!leftMt && !rightMt && (get(leftType) || get(rightType))) + { + if (isComparison) + { + reportError(GenericError{format("Types '%s' and '%s' cannot be compared with %s because neither type has a metatable", + toString(leftType).c_str(), toString(rightType).c_str(), toString(expr->op).c_str())}, + expr->location); + } + else + { + reportError(GenericError{format("Operator %s is not applicable for '%s' and '%s' because neither type has a metatable", + toString(expr->op).c_str(), toString(leftType).c_str(), toString(rightType).c_str())}, + expr->location); + } + + return; + } + } + } + + switch (expr->op) + { + case AstExprBinary::Op::Add: + case AstExprBinary::Op::Sub: + case AstExprBinary::Op::Mul: + case AstExprBinary::Op::Div: + case AstExprBinary::Op::Pow: + case AstExprBinary::Op::Mod: + reportErrors(tryUnify(scope, expr->left->location, leftType, singletonTypes->numberType)); + reportErrors(tryUnify(scope, expr->right->location, rightType, singletonTypes->numberType)); + + break; + case AstExprBinary::Op::Concat: + reportErrors(tryUnify(scope, expr->left->location, leftType, singletonTypes->stringType)); + reportErrors(tryUnify(scope, expr->right->location, rightType, singletonTypes->stringType)); + + break; + case AstExprBinary::Op::CompareGe: + case AstExprBinary::Op::CompareGt: + case AstExprBinary::Op::CompareLe: + case AstExprBinary::Op::CompareLt: + if (isNumber(leftType)) + reportErrors(tryUnify(scope, expr->right->location, rightType, singletonTypes->numberType)); + else if (isString(leftType)) + reportErrors(tryUnify(scope, expr->right->location, rightType, singletonTypes->stringType)); + else + reportError(GenericError{format("Types '%s' and '%s' cannot be compared with relational operator %s", toString(leftType).c_str(), + toString(rightType).c_str(), toString(expr->op).c_str())}, + expr->location); + + break; + case AstExprBinary::Op::And: + case AstExprBinary::Op::Or: + case AstExprBinary::Op::CompareEq: + case AstExprBinary::Op::CompareNe: + break; + default: + // Unhandled AstExprBinary::Op possibility. + LUAU_ASSERT(false); + } } void visit(AstExprTypeAssertion* expr) diff --git a/Analysis/src/TypeInfer.cpp b/Analysis/src/TypeInfer.cpp index b806edb7..d5c6b2c4 100644 --- a/Analysis/src/TypeInfer.cpp +++ b/Analysis/src/TypeInfer.cpp @@ -31,7 +31,6 @@ LUAU_FASTINTVARIABLE(LuauTypeInferTypePackLoopLimit, 5000) LUAU_FASTINTVARIABLE(LuauCheckRecursionLimit, 300) LUAU_FASTINTVARIABLE(LuauVisitRecursionLimit, 500) LUAU_FASTFLAG(LuauKnowsTheDataModel3) -LUAU_FASTFLAG(LuauAutocompleteDynamicLimits) LUAU_FASTFLAG(LuauTypeNormalization2) LUAU_FASTFLAGVARIABLE(DebugLuauFreezeDuringUnification, false) LUAU_FASTFLAGVARIABLE(LuauReturnAnyInsteadOfICE, false) // Eventually removed as false. @@ -280,11 +279,8 @@ ModulePtr TypeChecker::checkWithoutRecursionCheck(const SourceModule& module, Mo iceHandler->moduleName = module.name; normalizer.arena = ¤tModule->internalTypes; - if (FFlag::LuauAutocompleteDynamicLimits) - { - unifierState.counters.recursionLimit = FInt::LuauTypeInferRecursionLimit; - unifierState.counters.iterationLimit = unifierIterationLimit ? *unifierIterationLimit : FInt::LuauTypeInferIterationLimit; - } + unifierState.counters.recursionLimit = FInt::LuauTypeInferRecursionLimit; + unifierState.counters.iterationLimit = unifierIterationLimit ? *unifierIterationLimit : FInt::LuauTypeInferIterationLimit; ScopePtr parentScope = environmentScope.value_or(globalScope); ScopePtr moduleScope = std::make_shared(parentScope); @@ -773,16 +769,6 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatRepeat& statement) checkExpr(repScope, *statement.condition); } -void TypeChecker::unifyLowerBound(TypePackId subTy, TypePackId superTy, TypeLevel demotedLevel, const ScopePtr& scope, const Location& location) -{ - Unifier state = mkUnifier(scope, location); - state.unifyLowerBound(subTy, superTy, demotedLevel); - - state.log.commit(); - - reportErrors(state.errors); -} - struct Demoter : Substitution { Demoter(TypeArena* arena) @@ -2091,39 +2077,6 @@ std::optional TypeChecker::getIndexTypeFromTypeImpl( return std::nullopt; } -std::vector TypeChecker::reduceUnion(const std::vector& types) -{ - std::vector result; - for (TypeId t : types) - { - t = follow(t); - if (get(t)) - continue; - - if (get(t) || get(t)) - return {t}; - - if (const UnionTypeVar* utv = get(t)) - { - for (TypeId ty : utv) - { - ty = follow(ty); - if (get(ty)) - continue; - if (get(ty) || get(ty)) - return {ty}; - - if (result.end() == std::find(result.begin(), result.end(), ty)) - result.push_back(ty); - } - } - else if (std::find(result.begin(), result.end(), t) == result.end()) - result.push_back(t); - } - - return result; -} - std::optional TypeChecker::tryStripUnionFromNil(TypeId ty) { if (const UnionTypeVar* utv = get(ty)) @@ -4597,7 +4550,7 @@ TypeId TypeChecker::instantiate(const ScopePtr& scope, TypeId ty, Location locat Instantiation instantiation{log, ¤tModule->internalTypes, scope->level, /*scope*/ nullptr}; - if (FFlag::LuauAutocompleteDynamicLimits && instantiationChildLimit) + if (instantiationChildLimit) instantiation.childLimit = *instantiationChildLimit; std::optional instantiated = instantiation.substitute(ty); diff --git a/Analysis/src/TypeUtils.cpp b/Analysis/src/TypeUtils.cpp index 688c8767..72597c4a 100644 --- a/Analysis/src/TypeUtils.cpp +++ b/Analysis/src/TypeUtils.cpp @@ -6,6 +6,8 @@ #include "Luau/ToString.h" #include "Luau/TypeInfer.h" +#include + namespace Luau { @@ -146,18 +148,15 @@ std::optional getIndexTypeFromType(const ScopePtr& scope, ErrorVec& erro return std::nullopt; } + goodOptions = reduceUnion(goodOptions); + if (goodOptions.empty()) return singletonTypes->neverType; if (goodOptions.size() == 1) return goodOptions[0]; - // TODO: inefficient. - TypeId result = arena->addType(UnionTypeVar{std::move(goodOptions)}); - auto [ty, ok] = normalize(result, NotNull{scope.get()}, *arena, singletonTypes, handle); - if (!ok && addErrors) - errors.push_back(TypeError{location, NormalizationTooComplex{}}); - return ok ? ty : singletonTypes->anyType; + return arena->addType(UnionTypeVar{std::move(goodOptions)}); } else if (const IntersectionTypeVar* itv = get(type)) { @@ -264,4 +263,79 @@ std::vector flatten(TypeArena& arena, NotNull singletonT return result; } +std::vector reduceUnion(const std::vector& types) +{ + std::vector result; + for (TypeId t : types) + { + t = follow(t); + if (get(t)) + continue; + + if (get(t) || get(t)) + return {t}; + + if (const UnionTypeVar* utv = get(t)) + { + for (TypeId ty : utv) + { + ty = follow(ty); + if (get(ty)) + continue; + if (get(ty) || get(ty)) + return {ty}; + + if (result.end() == std::find(result.begin(), result.end(), ty)) + result.push_back(ty); + } + } + else if (std::find(result.begin(), result.end(), t) == result.end()) + result.push_back(t); + } + + return result; +} + +static std::optional tryStripUnionFromNil(TypeArena& arena, TypeId ty) +{ + if (const UnionTypeVar* utv = get(ty)) + { + if (!std::any_of(begin(utv), end(utv), isNil)) + return ty; + + std::vector result; + + for (TypeId option : utv) + { + if (!isNil(option)) + result.push_back(option); + } + + if (result.empty()) + return std::nullopt; + + return result.size() == 1 ? result[0] : arena.addType(UnionTypeVar{std::move(result)}); + } + + return std::nullopt; +} + +TypeId stripNil(NotNull singletonTypes, TypeArena& arena, TypeId ty) +{ + ty = follow(ty); + + if (get(ty)) + { + std::optional cleaned = tryStripUnionFromNil(arena, ty); + + // If there is no union option without 'nil' + if (!cleaned) + return singletonTypes->nilType; + + return follow(*cleaned); + } + + return follow(ty); +} + } // namespace Luau diff --git a/Analysis/src/TypeVar.cpp b/Analysis/src/TypeVar.cpp index bcdaff7d..19d3d266 100644 --- a/Analysis/src/TypeVar.cpp +++ b/Analysis/src/TypeVar.cpp @@ -57,6 +57,13 @@ TypeId follow(TypeId t, std::function mapper) return btv->boundTo; else if (auto ttv = get(mapper(ty))) return ttv->boundTo; + else if (auto utv = get(mapper(ty))) + { + std::optional ty = utv->scope->lookup(utv->def); + if (!ty) + throw std::runtime_error("UseTypeVar must map to another TypeId"); + return *ty; + } else return std::nullopt; }; @@ -760,6 +767,8 @@ SingletonTypes::SingletonTypes() , unknownType(arena->addType(TypeVar{UnknownTypeVar{}, /*persistent*/ true})) , neverType(arena->addType(TypeVar{NeverTypeVar{}, /*persistent*/ true})) , errorType(arena->addType(TypeVar{ErrorTypeVar{}, /*persistent*/ true})) + , falsyType(arena->addType(TypeVar{UnionTypeVar{{falseType, nilType}}, /*persistent*/ true})) + , truthyType(arena->addType(TypeVar{NegationTypeVar{falsyType}, /*persistent*/ true})) , anyTypePack(arena->addTypePack(TypePackVar{VariadicTypePack{anyType}, /*persistent*/ true})) , neverTypePack(arena->addTypePack(TypePackVar{VariadicTypePack{neverType}, /*persistent*/ true})) , uninhabitableTypePack(arena->addTypePack({neverType}, neverTypePack)) @@ -896,7 +905,6 @@ void persist(TypeId ty) continue; asMutable(t)->persistent = true; - asMutable(t)->normal = true; // all persistent types are assumed to be normal if (auto btv = get(t)) queue.push_back(btv->boundTo); @@ -933,11 +941,6 @@ void persist(TypeId ty) for (TypeId opt : itv->parts) queue.push_back(opt); } - else if (auto ctv = get(t)) - { - for (TypeId opt : ctv->parts) - queue.push_back(opt); - } else if (auto mtv = get(t)) { queue.push_back(mtv->table); @@ -990,8 +993,6 @@ const TypeLevel* getLevel(TypeId ty) return &ttv->level; else if (auto ftv = get(ty)) return &ftv->level; - else if (auto ctv = get(ty)) - return &ctv->level; else return nullptr; } @@ -1056,11 +1057,6 @@ const std::vector& getTypes(const IntersectionTypeVar* itv) return itv->parts; } -const std::vector& getTypes(const ConstrainedTypeVar* ctv) -{ - return ctv->parts; -} - UnionTypeVarIterator begin(const UnionTypeVar* utv) { return UnionTypeVarIterator{utv}; @@ -1081,17 +1077,6 @@ IntersectionTypeVarIterator end(const IntersectionTypeVar* itv) return IntersectionTypeVarIterator{}; } -ConstrainedTypeVarIterator begin(const ConstrainedTypeVar* ctv) -{ - return ConstrainedTypeVarIterator{ctv}; -} - -ConstrainedTypeVarIterator end(const ConstrainedTypeVar* ctv) -{ - return ConstrainedTypeVarIterator{}; -} - - static std::vector parseFormatString(TypeChecker& typechecker, const char* data, size_t size) { const char* options = "cdiouxXeEfgGqs*"; diff --git a/Analysis/src/Unifier.cpp b/Analysis/src/Unifier.cpp index 42fcd2fd..e23e6161 100644 --- a/Analysis/src/Unifier.cpp +++ b/Analysis/src/Unifier.cpp @@ -13,16 +13,13 @@ #include -LUAU_FASTINT(LuauTypeInferRecursionLimit); LUAU_FASTINT(LuauTypeInferTypePackLoopLimit); -LUAU_FASTINT(LuauTypeInferIterationLimit); -LUAU_FASTFLAG(LuauAutocompleteDynamicLimits) -LUAU_FASTINTVARIABLE(LuauTypeInferLowerBoundsIterationLimit, 2000); LUAU_FASTFLAG(LuauErrorRecoveryType); LUAU_FASTFLAG(LuauUnknownAndNeverType) LUAU_FASTFLAGVARIABLE(LuauSubtypeNormalizer, false); LUAU_FASTFLAGVARIABLE(LuauScalarShapeSubtyping, false) LUAU_FASTFLAGVARIABLE(LuauInstantiateInSubtyping, false) +LUAU_FASTFLAGVARIABLE(LuauOverloadedFunctionSubtypingPerf, false); LUAU_FASTFLAG(LuauClassTypeVarsInSubstitution) LUAU_FASTFLAG(DebugLuauDeferredConstraintResolution) @@ -95,15 +92,6 @@ struct PromoteTypeLevels final : TypeVarOnceVisitor return true; } - bool visit(TypeId ty, const ConstrainedTypeVar&) override - { - if (!FFlag::LuauUnknownAndNeverType) - return visit(ty); - - promote(ty, log.getMutable(ty)); - return true; - } - bool visit(TypeId ty, const FunctionTypeVar&) override { // Type levels of types from other modules are already global, so we don't need to promote anything inside @@ -368,26 +356,14 @@ void Unifier::tryUnify(TypeId subTy, TypeId superTy, bool isFunctionCall, bool i void Unifier::tryUnify_(TypeId subTy, TypeId superTy, bool isFunctionCall, bool isIntersection) { - RecursionLimiter _ra(&sharedState.counters.recursionCount, - FFlag::LuauAutocompleteDynamicLimits ? sharedState.counters.recursionLimit : FInt::LuauTypeInferRecursionLimit); + RecursionLimiter _ra(&sharedState.counters.recursionCount, sharedState.counters.recursionLimit); ++sharedState.counters.iterationCount; - if (FFlag::LuauAutocompleteDynamicLimits) + if (sharedState.counters.iterationLimit > 0 && sharedState.counters.iterationLimit < sharedState.counters.iterationCount) { - if (sharedState.counters.iterationLimit > 0 && sharedState.counters.iterationLimit < sharedState.counters.iterationCount) - { - reportError(TypeError{location, UnificationTooComplex{}}); - return; - } - } - else - { - if (FInt::LuauTypeInferIterationLimit > 0 && FInt::LuauTypeInferIterationLimit < sharedState.counters.iterationCount) - { - reportError(TypeError{location, UnificationTooComplex{}}); - return; - } + reportError(TypeError{location, UnificationTooComplex{}}); + return; } superTy = log.follow(superTy); @@ -396,9 +372,6 @@ void Unifier::tryUnify_(TypeId subTy, TypeId superTy, bool isFunctionCall, bool if (superTy == subTy) return; - if (log.get(superTy)) - return tryUnifyWithConstrainedSuperTypeVar(subTy, superTy); - auto superFree = log.getMutable(superTy); auto subFree = log.getMutable(subTy); @@ -520,9 +493,7 @@ void Unifier::tryUnify_(TypeId subTy, TypeId superTy, bool isFunctionCall, bool size_t errorCount = errors.size(); - if (log.get(subTy)) - tryUnifyWithConstrainedSubTypeVar(subTy, superTy); - else if (const UnionTypeVar* subUnion = log.getMutable(subTy)) + if (const UnionTypeVar* subUnion = log.getMutable(subTy)) { tryUnifyUnionWithType(subTy, subUnion, superTy); } @@ -1011,10 +982,17 @@ TypePackId Unifier::tryApplyOverloadedFunction(TypeId function, const Normalized log.concat(std::move(innerState.log)); if (result) { + if (FFlag::LuauOverloadedFunctionSubtypingPerf) + { + innerState.log.clear(); + innerState.tryUnify_(*result, ftv->retTypes); + } + if (FFlag::LuauOverloadedFunctionSubtypingPerf && innerState.errors.empty()) + log.concat(std::move(innerState.log)); // Annoyingly, since we don't support intersection of generic type packs, // the intersection may fail. We rather arbitrarily use the first matching overload // in that case. - if (std::optional intersect = normalizer->intersectionOfTypePacks(*result, ftv->retTypes)) + else if (std::optional intersect = normalizer->intersectionOfTypePacks(*result, ftv->retTypes)) result = intersect; } else @@ -1214,26 +1192,14 @@ void Unifier::tryUnify(TypePackId subTp, TypePackId superTp, bool isFunctionCall */ void Unifier::tryUnify_(TypePackId subTp, TypePackId superTp, bool isFunctionCall) { - RecursionLimiter _ra(&sharedState.counters.recursionCount, - FFlag::LuauAutocompleteDynamicLimits ? sharedState.counters.recursionLimit : FInt::LuauTypeInferRecursionLimit); + RecursionLimiter _ra(&sharedState.counters.recursionCount, sharedState.counters.recursionLimit); ++sharedState.counters.iterationCount; - if (FFlag::LuauAutocompleteDynamicLimits) + if (sharedState.counters.iterationLimit > 0 && sharedState.counters.iterationLimit < sharedState.counters.iterationCount) { - if (sharedState.counters.iterationLimit > 0 && sharedState.counters.iterationLimit < sharedState.counters.iterationCount) - { - reportError(TypeError{location, UnificationTooComplex{}}); - return; - } - } - else - { - if (FInt::LuauTypeInferIterationLimit > 0 && FInt::LuauTypeInferIterationLimit < sharedState.counters.iterationCount) - { - reportError(TypeError{location, UnificationTooComplex{}}); - return; - } + reportError(TypeError{location, UnificationTooComplex{}}); + return; } superTp = log.follow(superTp); @@ -2314,186 +2280,6 @@ std::optional Unifier::findTablePropertyRespectingMeta(TypeId lhsType, N return Luau::findTablePropertyRespectingMeta(singletonTypes, errors, lhsType, name, location); } -void Unifier::tryUnifyWithConstrainedSubTypeVar(TypeId subTy, TypeId superTy) -{ - const ConstrainedTypeVar* subConstrained = get(subTy); - if (!subConstrained) - ice("tryUnifyWithConstrainedSubTypeVar received non-ConstrainedTypeVar subTy!"); - - const std::vector& subTyParts = subConstrained->parts; - - // A | B <: T if A <: T and B <: T - bool failed = false; - std::optional unificationTooComplex; - - const size_t count = subTyParts.size(); - - for (size_t i = 0; i < count; ++i) - { - TypeId type = subTyParts[i]; - Unifier innerState = makeChildUnifier(); - innerState.tryUnify_(type, superTy); - - if (i == count - 1) - log.concat(std::move(innerState.log)); - - ++i; - - if (auto e = hasUnificationTooComplex(innerState.errors)) - unificationTooComplex = e; - - if (!innerState.errors.empty()) - { - failed = true; - break; - } - } - - if (unificationTooComplex) - reportError(*unificationTooComplex); - else if (failed) - reportError(TypeError{location, TypeMismatch{superTy, subTy}}); - else - log.replace(subTy, BoundTypeVar{superTy}); -} - -void Unifier::tryUnifyWithConstrainedSuperTypeVar(TypeId subTy, TypeId superTy) -{ - ConstrainedTypeVar* superC = log.getMutable(superTy); - if (!superC) - ice("tryUnifyWithConstrainedSuperTypeVar received non-ConstrainedTypeVar superTy!"); - - // subTy could be a - // table - // metatable - // class - // function - // primitive - // free - // generic - // intersection - // union - // Do we really just tack it on? I think we might! - // We can certainly do some deduplication. - // Is there any point to deducing Player|Instance when we could just reduce to Instance? - // Is it actually ok to have multiple free types in a single intersection? What if they are later unified into the same type? - // Maybe we do a simplification step during quantification. - - auto it = std::find(superC->parts.begin(), superC->parts.end(), subTy); - if (it != superC->parts.end()) - return; - - superC->parts.push_back(subTy); -} - -void Unifier::unifyLowerBound(TypePackId subTy, TypePackId superTy, TypeLevel demotedLevel) -{ - // The duplication between this and regular typepack unification is tragic. - - auto superIter = begin(superTy, &log); - auto superEndIter = end(superTy); - - auto subIter = begin(subTy, &log); - auto subEndIter = end(subTy); - - int count = FInt::LuauTypeInferLowerBoundsIterationLimit; - - for (; subIter != subEndIter; ++subIter) - { - if (0 >= --count) - ice("Internal recursion counter limit exceeded in Unifier::unifyLowerBound"); - - if (superIter != superEndIter) - { - tryUnify_(*subIter, *superIter); - ++superIter; - continue; - } - - if (auto t = superIter.tail()) - { - TypePackId tailPack = follow(*t); - - if (log.get(tailPack) && occursCheck(tailPack, subTy)) - return; - - FreeTypePack* freeTailPack = log.getMutable(tailPack); - if (!freeTailPack) - return; - - TypePack* tp = getMutable(log.replace(tailPack, TypePack{})); - - for (; subIter != subEndIter; ++subIter) - { - tp->head.push_back(types->addType(ConstrainedTypeVar{demotedLevel, {follow(*subIter)}})); - } - - tp->tail = subIter.tail(); - } - - return; - } - - if (superIter != superEndIter) - { - if (auto subTail = subIter.tail()) - { - TypePackId subTailPack = follow(*subTail); - if (get(subTailPack)) - { - TypePack* tp = getMutable(log.replace(subTailPack, TypePack{})); - - for (; superIter != superEndIter; ++superIter) - tp->head.push_back(*superIter); - } - else if (const VariadicTypePack* subVariadic = log.getMutable(subTailPack)) - { - while (superIter != superEndIter) - { - tryUnify_(subVariadic->ty, *superIter); - ++superIter; - } - } - } - else - { - while (superIter != superEndIter) - { - if (!isOptional(*superIter)) - { - errors.push_back(TypeError{location, CountMismatch{size(superTy), std::nullopt, size(subTy), CountMismatch::Return}}); - return; - } - ++superIter; - } - } - - return; - } - - // Both iters are at their respective tails - auto subTail = subIter.tail(); - auto superTail = superIter.tail(); - if (subTail && superTail) - tryUnify(*subTail, *superTail); - else if (subTail) - { - const FreeTypePack* freeSubTail = log.getMutable(*subTail); - if (freeSubTail) - { - log.replace(*subTail, TypePack{}); - } - } - else if (superTail) - { - const FreeTypePack* freeSuperTail = log.getMutable(*superTail); - if (freeSuperTail) - { - log.replace(*superTail, TypePack{}); - } - } -} - bool Unifier::occursCheck(TypeId needle, TypeId haystack) { sharedState.tempSeenTy.clear(); @@ -2503,8 +2289,7 @@ bool Unifier::occursCheck(TypeId needle, TypeId haystack) bool Unifier::occursCheck(DenseHashSet& seen, TypeId needle, TypeId haystack) { - RecursionLimiter _ra(&sharedState.counters.recursionCount, - FFlag::LuauAutocompleteDynamicLimits ? sharedState.counters.recursionLimit : FInt::LuauTypeInferRecursionLimit); + RecursionLimiter _ra(&sharedState.counters.recursionCount, sharedState.counters.recursionLimit); bool occurrence = false; @@ -2547,11 +2332,6 @@ bool Unifier::occursCheck(DenseHashSet& seen, TypeId needle, TypeId hays for (TypeId ty : a->parts) check(ty); } - else if (auto a = log.getMutable(haystack)) - { - for (TypeId ty : a->parts) - check(ty); - } return occurrence; } @@ -2579,8 +2359,7 @@ bool Unifier::occursCheck(DenseHashSet& seen, TypePackId needle, Typ if (!log.getMutable(needle)) ice("Expected needle pack to be free"); - RecursionLimiter _ra(&sharedState.counters.recursionCount, - FFlag::LuauAutocompleteDynamicLimits ? sharedState.counters.recursionLimit : FInt::LuauTypeInferRecursionLimit); + RecursionLimiter _ra(&sharedState.counters.recursionCount, sharedState.counters.recursionLimit); while (!log.getMutable(haystack)) { diff --git a/CLI/Analyze.cpp b/CLI/Analyze.cpp index 7e4c5691..6257e2f3 100644 --- a/CLI/Analyze.cpp +++ b/CLI/Analyze.cpp @@ -14,7 +14,6 @@ #endif LUAU_FASTFLAG(DebugLuauTimeTracing) -LUAU_FASTFLAG(LuauTypeMismatchModuleNameResolution) enum class ReportFormat { @@ -55,11 +54,9 @@ static void reportError(const Luau::Frontend& frontend, ReportFormat format, con if (const Luau::SyntaxError* syntaxError = Luau::get_if(&error.data)) report(format, humanReadableName.c_str(), error.location, "SyntaxError", syntaxError->message.c_str()); - else if (FFlag::LuauTypeMismatchModuleNameResolution) + else report(format, humanReadableName.c_str(), error.location, "TypeError", Luau::toString(error, Luau::TypeErrorToStringOptions{frontend.fileResolver}).c_str()); - else - report(format, humanReadableName.c_str(), error.location, "TypeError", Luau::toString(error).c_str()); } static void reportWarning(ReportFormat format, const char* name, const Luau::LintWarning& warning) diff --git a/CLI/Repl.cpp b/CLI/Repl.cpp index aecddf38..a9dd8970 100644 --- a/CLI/Repl.cpp +++ b/CLI/Repl.cpp @@ -16,6 +16,7 @@ #include "isocline.h" +#include #include #ifdef _WIN32 @@ -49,6 +50,8 @@ enum class CompileFormat Binary, Remarks, Codegen, + CodegenVerbose, + CodegenNull, Null }; @@ -673,21 +676,33 @@ static void reportError(const char* name, const Luau::CompileError& error) report(name, error.getLocation(), "CompileError", error.what()); } -static std::string getCodegenAssembly(const char* name, const std::string& bytecode) +static std::string getCodegenAssembly(const char* name, const std::string& bytecode, Luau::CodeGen::AssemblyOptions options) { std::unique_ptr globalState(luaL_newstate(), lua_close); lua_State* L = globalState.get(); - setupState(L); - if (luau_load(L, name, bytecode.data(), bytecode.size(), 0) == 0) - return Luau::CodeGen::getAssemblyText(L, -1); + return Luau::CodeGen::getAssembly(L, -1, options); fprintf(stderr, "Error loading bytecode %s\n", name); return ""; } -static bool compileFile(const char* name, CompileFormat format) +static void annotateInstruction(void* context, std::string& text, int fid, int instid) +{ + Luau::BytecodeBuilder& bcb = *(Luau::BytecodeBuilder*)context; + + bcb.annotateInstruction(text, fid, instid); +} + +struct CompileStats +{ + size_t lines; + size_t bytecode; + size_t codegen; +}; + +static bool compileFile(const char* name, CompileFormat format, CompileStats& stats) { std::optional source = readFile(name); if (!source) @@ -696,9 +711,12 @@ static bool compileFile(const char* name, CompileFormat format) return false; } + stats.lines += std::count(source->begin(), source->end(), '\n'); + try { Luau::BytecodeBuilder bcb; + Luau::CodeGen::AssemblyOptions options = {format == CompileFormat::CodegenNull, format == CompileFormat::Codegen, annotateInstruction, &bcb}; if (format == CompileFormat::Text) { @@ -711,8 +729,15 @@ static bool compileFile(const char* name, CompileFormat format) bcb.setDumpFlags(Luau::BytecodeBuilder::Dump_Source | Luau::BytecodeBuilder::Dump_Remarks); bcb.setDumpSource(*source); } + else if (format == CompileFormat::Codegen || format == CompileFormat::CodegenVerbose) + { + bcb.setDumpFlags(Luau::BytecodeBuilder::Dump_Code | Luau::BytecodeBuilder::Dump_Source | Luau::BytecodeBuilder::Dump_Locals | + Luau::BytecodeBuilder::Dump_Remarks); + bcb.setDumpSource(*source); + } Luau::compileOrThrow(bcb, *source, copts()); + stats.bytecode += bcb.getBytecode().size(); switch (format) { @@ -726,7 +751,11 @@ static bool compileFile(const char* name, CompileFormat format) fwrite(bcb.getBytecode().data(), 1, bcb.getBytecode().size(), stdout); break; case CompileFormat::Codegen: - printf("%s", getCodegenAssembly(name, bcb.getBytecode()).c_str()); + case CompileFormat::CodegenVerbose: + printf("%s", getCodegenAssembly(name, bcb.getBytecode(), options).c_str()); + break; + case CompileFormat::CodegenNull: + stats.codegen += getCodegenAssembly(name, bcb.getBytecode(), options).size(); break; case CompileFormat::Null: break; @@ -755,7 +784,7 @@ static void displayHelp(const char* argv0) printf("\n"); printf("Available modes:\n"); printf(" omitted: compile and run input files one by one\n"); - printf(" --compile[=format]: compile input files and output resulting formatted bytecode (binary, text, remarks, codegen or null)\n"); + printf(" --compile[=format]: compile input files and output resulting bytecode/assembly (binary, text, remarks, codegen)\n"); printf("\n"); printf("Available options:\n"); printf(" --coverage: collect code coverage while running the code and output results to coverage.out\n"); @@ -812,6 +841,14 @@ int replMain(int argc, char** argv) { compileFormat = CompileFormat::Codegen; } + else if (strcmp(argv[1], "--compile=codegenverbose") == 0) + { + compileFormat = CompileFormat::CodegenVerbose; + } + else if (strcmp(argv[1], "--compile=codegennull") == 0) + { + compileFormat = CompileFormat::CodegenNull; + } else if (strcmp(argv[1], "--compile=null") == 0) { compileFormat = CompileFormat::Null; @@ -923,10 +960,16 @@ int replMain(int argc, char** argv) _setmode(_fileno(stdout), _O_BINARY); #endif + CompileStats stats = {}; int failed = 0; for (const std::string& path : files) - failed += !compileFile(path.c_str(), compileFormat); + failed += !compileFile(path.c_str(), compileFormat, stats); + + if (compileFormat == CompileFormat::Null) + printf("Compiled %d KLOC into %d KB bytecode\n", int(stats.lines / 1000), int(stats.bytecode / 1024)); + else if (compileFormat == CompileFormat::CodegenNull) + printf("Compiled %d KLOC into %d KB bytecode => %d KB native code\n", int(stats.lines / 1000), int(stats.bytecode / 1024), int(stats.codegen / 1024)); return failed ? 1 : 0; } diff --git a/CodeGen/include/Luau/AssemblyBuilderX64.h b/CodeGen/include/Luau/AssemblyBuilderX64.h index 1c755017..e48388c5 100644 --- a/CodeGen/include/Luau/AssemblyBuilderX64.h +++ b/CodeGen/include/Luau/AssemblyBuilderX64.h @@ -23,6 +23,19 @@ enum class RoundingModeX64 RoundToZero = 0b11, }; +enum class AlignmentDataX64 +{ + Nop, + Int3, + Ud2, // int3 will be used as a fall-back if it doesn't fit +}; + +enum class ABIX64 +{ + Windows, + SystemV, +}; + class AssemblyBuilderX64 { public: @@ -80,6 +93,10 @@ public: void int3(); + // Code alignment + void nop(uint32_t length = 1); + void align(uint32_t alignment, AlignmentDataX64 data = AlignmentDataX64::Nop); + // AVX void vaddpd(OperandX64 dst, OperandX64 src1, OperandX64 src2); void vaddps(OperandX64 dst, OperandX64 src1, OperandX64 src2); @@ -131,6 +148,8 @@ public: void logAppend(const char* fmt, ...) LUAU_PRINTF_ATTR(2, 3); + uint32_t getCodeSize() const; + // Resulting data and code that need to be copied over one after the other // The *end* of 'data' has to be aligned to 16 bytes, this will also align 'code' std::vector data; @@ -140,6 +159,8 @@ public: const bool logText = false; + const ABIX64 abi; + private: // Instruction archetypes void placeBinary(const char* name, OperandX64 lhs, OperandX64 rhs, uint8_t codeimm8, uint8_t codeimm, uint8_t codeimmImm8, uint8_t code8rev, @@ -177,7 +198,6 @@ private: void commit(); LUAU_NOINLINE void extend(); - uint32_t getCodeSize(); // Data size_t allocateData(size_t size, size_t align); @@ -192,8 +212,8 @@ private: LUAU_NOINLINE void log(const char* opcode, Label label); void log(OperandX64 op); - const char* getSizeName(SizeX64 size); - const char* getRegisterName(RegisterX64 reg); + const char* getSizeName(SizeX64 size) const; + const char* getRegisterName(RegisterX64 reg) const; uint32_t nextLabel = 1; std::vector(a) -> a" == toString(idType)); @@ -66,7 +42,7 @@ TEST_CASE_FIXTURE(ConstraintGraphBuilderFixture, "generic_function") TEST_CASE_FIXTURE(ConstraintGraphBuilderFixture, "proper_let_generalization") { - AstStatBlock* block = parse(R"( + solve(R"( local function a(c) local function d(e) return c @@ -78,21 +54,9 @@ TEST_CASE_FIXTURE(ConstraintGraphBuilderFixture, "proper_let_generalization") local b = a(5) )"); - cgb.visit(block); - NotNull rootScope{cgb.rootScope}; - - ToStringOptions opts; - - NullModuleResolver resolver; - InternalErrorReporter iceHandler; - UnifierSharedState sharedState{&iceHandler}; - Normalizer normalizer{&arena, singletonTypes, NotNull{&sharedState}}; - ConstraintSolver cs{NotNull{&normalizer}, rootScope, "MainModule", NotNull(&resolver), {}, &logger}; - - cs.run(); - TypeId idType = requireBinding(rootScope, "b"); + ToStringOptions opts; CHECK("(a) -> number" == toString(idType, opts)); } diff --git a/tests/DataFlowGraphBuilder.test.cpp b/tests/DataFlowGraphBuilder.test.cpp new file mode 100644 index 00000000..9aa7cde6 --- /dev/null +++ b/tests/DataFlowGraphBuilder.test.cpp @@ -0,0 +1,104 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +#include "Luau/DataFlowGraphBuilder.h" +#include "Luau/Error.h" +#include "Luau/Parser.h" + +#include "AstQueryDsl.h" +#include "ScopedFlags.h" + +#include "doctest.h" + +using namespace Luau; + +class DataFlowGraphFixture +{ + // Only needed to fix the operator== reflexivity of an empty Symbol. + ScopedFastFlag dcr{"DebugLuauDeferredConstraintResolution", true}; + + InternalErrorReporter handle; + + Allocator allocator; + AstNameTable names{allocator}; + AstStatBlock* module; + + std::optional graph; + +public: + void dfg(const std::string& code) + { + ParseResult parseResult = Parser::parse(code.c_str(), code.size(), names, allocator); + if (!parseResult.errors.empty()) + throw ParseErrors(std::move(parseResult.errors)); + module = parseResult.root; + graph = DataFlowGraphBuilder::build(module, NotNull{&handle}); + } + + template + std::optional getDef(const std::vector& nths = {nth(N)}) + { + T* node = query(module, nths); + REQUIRE(node); + return graph->getDef(node); + } + + template + DefId requireDef(const std::vector& nths = {nth(N)}) + { + auto loc = getDef(nths); + REQUIRE(loc); + return NotNull{*loc}; + } +}; + +TEST_SUITE_BEGIN("DataFlowGraphBuilder"); + +TEST_CASE_FIXTURE(DataFlowGraphFixture, "define_locals_in_local_stat") +{ + dfg(R"( + local x = 5 + local y = x + )"); + + REQUIRE(getDef()); +} + +TEST_CASE_FIXTURE(DataFlowGraphFixture, "define_parameters_in_functions") +{ + dfg(R"( + local function f(x) + local y = x + end + )"); + + REQUIRE(getDef()); +} + +TEST_CASE_FIXTURE(DataFlowGraphFixture, "find_aliases") +{ + dfg(R"( + local x = 5 + local y = x + local z = y + )"); + + DefId x = requireDef(); + DefId y = requireDef(); + REQUIRE(x != y); // TODO: they should be equal but it's not just locals that can alias, so we'll support this later. +} + +TEST_CASE_FIXTURE(DataFlowGraphFixture, "independent_locals") +{ + dfg(R"( + local x = 5 + local y = 5 + + local a = x + local b = y + )"); + + DefId x = requireDef(); + DefId y = requireDef(); + REQUIRE(x != y); +} + +TEST_SUITE_END(); diff --git a/tests/Module.test.cpp b/tests/Module.test.cpp index d5f635e6..33d9c75a 100644 --- a/tests/Module.test.cpp +++ b/tests/Module.test.cpp @@ -226,24 +226,6 @@ TEST_CASE_FIXTURE(Fixture, "clone_free_tables") CHECK_EQ(clonedTtv->state, TableState::Free); } -TEST_CASE_FIXTURE(Fixture, "clone_constrained_intersection") -{ - TypeArena src; - - TypeId constrained = src.addType(ConstrainedTypeVar{TypeLevel{}, {singletonTypes->numberType, singletonTypes->stringType}}); - - TypeArena dest; - CloneState cloneState; - - TypeId cloned = clone(constrained, dest, cloneState); - CHECK_NE(constrained, cloned); - - const ConstrainedTypeVar* ctv = get(cloned); - REQUIRE_EQ(2, ctv->parts.size()); - CHECK_EQ(singletonTypes->numberType, ctv->parts[0]); - CHECK_EQ(singletonTypes->stringType, ctv->parts[1]); -} - TEST_CASE_FIXTURE(BuiltinsFixture, "clone_self_property") { fileResolver.source["Module/A"] = R"( diff --git a/tests/Normalize.test.cpp b/tests/Normalize.test.cpp index b3522f6e..20e0e34c 100644 --- a/tests/Normalize.test.cpp +++ b/tests/Normalize.test.cpp @@ -391,26 +391,6 @@ TEST_SUITE_END(); TEST_SUITE_BEGIN("Normalize"); -TEST_CASE_FIXTURE(NormalizeFixture, "union_with_overlapping_field_that_has_a_subtype_relationship") -{ - check(R"( - local t: {x: number} | {x: number?} - )"); - - ModulePtr tempModule{new Module}; - tempModule->scopes.emplace_back(Location(), std::make_shared(singletonTypes->anyTypePack)); - - // HACK: Normalization is an in-place operation. We need to cheat a little here and unfreeze - // the arena that the type lives in. - ModulePtr mainModule = getMainModule(); - unfreeze(mainModule->internalTypes); - - TypeId tType = requireType("t"); - normalize(tType, tempModule, singletonTypes, *typeChecker.iceHandler); - - CHECK_EQ("{| x: number? |}", toString(tType, {true})); -} - TEST_CASE_FIXTURE(Fixture, "higher_order_function") { check(R"( diff --git a/tests/Symbol.test.cpp b/tests/Symbol.test.cpp index e7d2973b..278c6ce2 100644 --- a/tests/Symbol.test.cpp +++ b/tests/Symbol.test.cpp @@ -10,7 +10,7 @@ using namespace Luau; TEST_SUITE_BEGIN("SymbolTests"); -TEST_CASE("hashing_globals") +TEST_CASE("equality_and_hashing_of_globals") { std::string s1 = "name"; std::string s2 = "name"; @@ -37,7 +37,7 @@ TEST_CASE("hashing_globals") REQUIRE_EQ(1, theMap.size()); } -TEST_CASE("hashing_locals") +TEST_CASE("equality_and_hashing_of_locals") { std::string s1 = "name"; std::string s2 = "name"; @@ -64,4 +64,24 @@ TEST_CASE("hashing_locals") REQUIRE_EQ(2, theMap.size()); } +TEST_CASE("equality_of_empty_symbols") +{ + ScopedFastFlag sff{"DebugLuauDeferredConstraintResolution", true}; + + std::string s1 = "name"; + std::string s2 = "name"; + + AstName one{s1.data()}; + AstLocal two{AstName{s2.data()}, Location(), nullptr, 0, 0, nullptr}; + + Symbol global{one}; + Symbol local{&two}; + Symbol empty1{}; + Symbol empty2{}; + + CHECK(empty1 != global); + CHECK(empty1 != local); + CHECK(empty1 == empty2); +} + TEST_SUITE_END(); diff --git a/tests/ToDot.test.cpp b/tests/ToDot.test.cpp index 98eb9863..26c9a1ee 100644 --- a/tests/ToDot.test.cpp +++ b/tests/ToDot.test.cpp @@ -79,8 +79,8 @@ n1 [label="AnyTypeVar 1"]; TEST_CASE_FIXTURE(Fixture, "bound") { CheckResult result = check(R"( -local a = 444 -local b = a +function a(): number return 444 end +local b = a() )"); LUAU_REQUIRE_NO_ERRORS(result); @@ -367,27 +367,6 @@ n3 [label="number"]; toDot(*ty, opts)); } -TEST_CASE_FIXTURE(Fixture, "constrained") -{ - // ConstrainedTypeVars never appear in the final type graph, so we have to create one directly - // to dotify it. - TypeVar t{ConstrainedTypeVar{TypeLevel{}, {typeChecker.numberType, typeChecker.stringType, typeChecker.nilType}}}; - - ToDotOptions opts; - opts.showPointers = false; - - CHECK_EQ(R"(digraph graphname { -n1 [label="ConstrainedTypeVar 1"]; -n1 -> n2; -n2 [label="number"]; -n1 -> n3; -n3 [label="string"]; -n1 -> n4; -n4 [label="nil"]; -})", - toDot(&t, opts)); -} - TEST_CASE_FIXTURE(Fixture, "singletontypes") { CheckResult result = check(R"( diff --git a/tests/TypeInfer.aliases.test.cpp b/tests/TypeInfer.aliases.test.cpp index 5ecc2a8c..1bb97fbc 100644 --- a/tests/TypeInfer.aliases.test.cpp +++ b/tests/TypeInfer.aliases.test.cpp @@ -846,8 +846,16 @@ TEST_CASE_FIXTURE(Fixture, "forward_declared_alias_is_not_clobbered_by_prior_uni type FutureIntersection = A & B )"); - // TODO: shared self causes this test to break in bizarre ways. - LUAU_REQUIRE_ERRORS(result); + if (FFlag::DebugLuauDeferredConstraintResolution) + { + // To be quite honest, I don't know exactly why DCR fixes this. + LUAU_REQUIRE_NO_ERRORS(result); + } + else + { + // TODO: shared self causes this test to break in bizarre ways. + LUAU_REQUIRE_ERRORS(result); + } } TEST_CASE_FIXTURE(Fixture, "recursive_types_restriction_ok") diff --git a/tests/TypeInfer.functions.test.cpp b/tests/TypeInfer.functions.test.cpp index edc25c7e..075bb01a 100644 --- a/tests/TypeInfer.functions.test.cpp +++ b/tests/TypeInfer.functions.test.cpp @@ -1692,4 +1692,47 @@ foo(string.find("hello", "e")) CHECK_EQ(toString(result.errors[0]), "Argument count mismatch. Function 'foo' expects 0 to 2 arguments, but 3 are specified"); } +TEST_CASE_FIXTURE(Fixture, "luau_subtyping_is_np_hard") +{ + ScopedFastFlag sffs[]{ + {"LuauSubtypeNormalizer", true}, + {"LuauTypeNormalization2", true}, + {"LuauOverloadedFunctionSubtypingPerf", true}, + }; + + CheckResult result = check(R"( +--!strict + +-- An example of coding up graph coloring in the Luau type system. +-- This codes a three-node, two color problem. +-- A three-node triangle is uncolorable, +-- but a three-node line is colorable. + +type Red = "red" +type Blue = "blue" +type Color = Red | Blue +type Coloring = (Color) -> (Color) -> (Color) -> boolean +type Uncolorable = (Color) -> (Color) -> (Color) -> false + +type Line = Coloring + & ((Red) -> (Red) -> (Color) -> false) + & ((Blue) -> (Blue) -> (Color) -> false) + & ((Color) -> (Red) -> (Red) -> false) + & ((Color) -> (Blue) -> (Blue) -> false) + +type Triangle = Line + & ((Red) -> (Color) -> (Red) -> false) + & ((Blue) -> (Color) -> (Blue) -> false) + +local x : Triangle +local y : Line +local z : Uncolorable +z = x -- OK, so the triangle is uncolorable +z = y -- Not OK, so the line is colorable + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ(toString(result.errors[0]), "Type '((\"blue\" | \"red\") -> (\"blue\" | \"red\") -> (\"blue\" | \"red\") -> boolean) & ((\"blue\" | \"red\") -> (\"blue\") -> (\"blue\") -> false) & ((\"blue\" | \"red\") -> (\"red\") -> (\"red\") -> false) & ((\"blue\") -> (\"blue\") -> (\"blue\" | \"red\") -> false) & ((\"red\") -> (\"red\") -> (\"blue\" | \"red\") -> false)' could not be converted into '(\"blue\" | \"red\") -> (\"blue\" | \"red\") -> (\"blue\" | \"red\") -> false'; none of the intersection parts are compatible"); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.intersectionTypes.test.cpp b/tests/TypeInfer.intersectionTypes.test.cpp index ca22c351..0c10eb87 100644 --- a/tests/TypeInfer.intersectionTypes.test.cpp +++ b/tests/TypeInfer.intersectionTypes.test.cpp @@ -781,7 +781,7 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "intersect_metatables") CheckResult result = check(R"( local a : string? = nil local b : number? = nil - + local x = setmetatable({}, { p = 5, q = a }); local y = setmetatable({}, { q = b, r = "hi" }); local z = setmetatable({}, { p = 5, q = nil, r = "hi" }); diff --git a/tests/TypeInfer.operators.test.cpp b/tests/TypeInfer.operators.test.cpp index 3d6c0193..e572c87a 100644 --- a/tests/TypeInfer.operators.test.cpp +++ b/tests/TypeInfer.operators.test.cpp @@ -13,6 +13,8 @@ using namespace Luau; +LUAU_FASTFLAG(DebugLuauDeferredConstraintResolution) + TEST_SUITE_BEGIN("TypeInferOperators"); TEST_CASE_FIXTURE(Fixture, "or_joins_types") @@ -33,7 +35,7 @@ TEST_CASE_FIXTURE(Fixture, "or_joins_types_with_no_extras") local x:number|string = s local y = x or "s" )"); - CHECK_EQ(0, result.errors.size()); + LUAU_REQUIRE_NO_ERRORS(result); CHECK_EQ(toString(*requireType("s")), "number | string"); CHECK_EQ(toString(*requireType("y")), "number | string"); } @@ -44,7 +46,7 @@ TEST_CASE_FIXTURE(Fixture, "or_joins_types_with_no_superfluous_union") local s = "a" or "b" local x:string = s )"); - CHECK_EQ(0, result.errors.size()); + LUAU_REQUIRE_NO_ERRORS(result); CHECK_EQ(*requireType("s"), *typeChecker.stringType); } @@ -54,7 +56,7 @@ TEST_CASE_FIXTURE(Fixture, "and_adds_boolean") local s = "a" and 10 local x:boolean|number = s )"); - CHECK_EQ(0, result.errors.size()); + LUAU_REQUIRE_NO_ERRORS(result); CHECK_EQ(toString(*requireType("s")), "boolean | number"); } @@ -64,7 +66,7 @@ TEST_CASE_FIXTURE(Fixture, "and_adds_boolean_no_superfluous_union") local s = "a" and true local x:boolean = s )"); - CHECK_EQ(0, result.errors.size()); + LUAU_REQUIRE_NO_ERRORS(result); CHECK_EQ(*requireType("x"), *typeChecker.booleanType); } @@ -73,7 +75,7 @@ TEST_CASE_FIXTURE(Fixture, "and_or_ternary") CheckResult result = check(R"( local s = (1/2) > 0.5 and "a" or 10 )"); - CHECK_EQ(0, result.errors.size()); + LUAU_REQUIRE_NO_ERRORS(result); CHECK_EQ(toString(*requireType("s")), "number | string"); } @@ -81,7 +83,7 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "primitive_arith_no_metatable") { CheckResult result = check(R"( function add(a: number, b: string) - return a + (tonumber(b) :: number), a .. b + return a + (tonumber(b) :: number), tostring(a) .. b end local n, s = add(2,"3") )"); @@ -558,15 +560,21 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "disallow_string_and_types_without_metatables LUAU_REQUIRE_ERROR_COUNT(3, result); TypeMismatch* tm = get(result.errors[0]); - REQUIRE_EQ(*tm->wantedType, *typeChecker.numberType); - REQUIRE_EQ(*tm->givenType, *typeChecker.stringType); + REQUIRE(tm); + CHECK_EQ(*tm->wantedType, *typeChecker.numberType); + CHECK_EQ(*tm->givenType, *typeChecker.stringType); + + GenericError* gen1 = get(result.errors[1]); + REQUIRE(gen1); + if (FFlag::DebugLuauDeferredConstraintResolution) + CHECK_EQ(gen1->message, "Operator + is not applicable for '{ value: number }' and 'number' because neither type has a metatable"); + else + CHECK_EQ(gen1->message, "Binary operator '+' not supported by types 'foo' and 'number'"); TypeMismatch* tm2 = get(result.errors[2]); + REQUIRE(tm2); CHECK_EQ(*tm2->wantedType, *typeChecker.numberType); CHECK_EQ(*tm2->givenType, *requireType("foo")); - - GenericError* gen2 = get(result.errors[1]); - REQUIRE_EQ(gen2->message, "Binary operator '+' not supported by types 'foo' and 'number'"); } // CLI-29033 @@ -611,12 +619,10 @@ TEST_CASE_FIXTURE(Fixture, "strict_binary_op_where_lhs_unknown") { std::vector ops = {"+", "-", "*", "/", "%", "^", ".."}; - std::string src = R"( - function foo(a, b) - )"; + std::string src = "function foo(a, b)\n"; for (const auto& op : ops) - src += "local _ = a " + op + "b\n"; + src += "local _ = a " + op + " b\n"; src += "end"; @@ -651,7 +657,11 @@ TEST_CASE_FIXTURE(Fixture, "error_on_invalid_operand_types_to_relational_operato GenericError* ge = get(result.errors[0]); REQUIRE(ge); - CHECK_EQ("Type 'boolean' cannot be compared with relational operator <", ge->message); + + if (FFlag::DebugLuauDeferredConstraintResolution) + CHECK_EQ("Types 'boolean' and 'boolean' cannot be compared with relational operator <", ge->message); + else + CHECK_EQ("Type 'boolean' cannot be compared with relational operator <", ge->message); } TEST_CASE_FIXTURE(Fixture, "error_on_invalid_operand_types_to_relational_operators2") @@ -666,7 +676,10 @@ TEST_CASE_FIXTURE(Fixture, "error_on_invalid_operand_types_to_relational_operato GenericError* ge = get(result.errors[0]); REQUIRE(ge); - CHECK_EQ("Type 'number | string' cannot be compared with relational operator <", ge->message); + if (FFlag::DebugLuauDeferredConstraintResolution) + CHECK_EQ("Types 'number | string' and 'number | string' cannot be compared with relational operator <", ge->message); + else + CHECK_EQ("Type 'number | string' cannot be compared with relational operator <", ge->message); } TEST_CASE_FIXTURE(Fixture, "cli_38355_recursive_union") @@ -891,4 +904,63 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "expected_types_through_binary_or") LUAU_REQUIRE_NO_ERRORS(result); } +TEST_CASE_FIXTURE(BuiltinsFixture, "mm_ops_must_return_a_value") +{ + if (!FFlag::DebugLuauDeferredConstraintResolution) + return; + + CheckResult result = check(R"( + local mm = { + __add = function(self, other) + return + end, + } + + local x = setmetatable({}, mm) + local y = x + 123 + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + CHECK(requireType("y") == singletonTypes->errorRecoveryType()); + + const GenericError* ge = get(result.errors[0]); + REQUIRE(ge); + CHECK(ge->message == "Metamethod '__add' must return a value"); +} + +TEST_CASE_FIXTURE(BuiltinsFixture, "mm_comparisons_must_return_a_boolean") +{ + if (!FFlag::DebugLuauDeferredConstraintResolution) + return; + + CheckResult result = check(R"( + local mm1 = { + __lt = function(self, other) + return 123 + end, + } + + local mm2 = { + __lt = function(self, other) + return + end, + } + + local o1 = setmetatable({}, mm1) + local v1 = o1 < o1 + + local o2 = setmetatable({}, mm2) + local v2 = o2 < o2 + )"); + + LUAU_REQUIRE_ERROR_COUNT(2, result); + + CHECK(requireType("v1") == singletonTypes->booleanType); + CHECK(requireType("v2") == singletonTypes->booleanType); + + CHECK(toString(result.errors[0]) == "Metamethod '__lt' must return type 'boolean'"); + CHECK(toString(result.errors[1]) == "Metamethod '__lt' must return type 'boolean'"); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.refinements.test.cpp b/tests/TypeInfer.refinements.test.cpp index f707f952..15d94430 100644 --- a/tests/TypeInfer.refinements.test.cpp +++ b/tests/TypeInfer.refinements.test.cpp @@ -8,6 +8,7 @@ #include "doctest.h" LUAU_FASTFLAG(LuauSpecialTypesAsterisked) +LUAU_FASTFLAG(DebugLuauDeferredConstraintResolution) using namespace Luau; @@ -49,7 +50,6 @@ struct RefinementClassFixture : Fixture {"Y", Property{typeChecker.numberType}}, {"Z", Property{typeChecker.numberType}}, }; - normalize(vec3, scope, arena, singletonTypes, *typeChecker.iceHandler); TypeId inst = arena.addType(ClassTypeVar{"Instance", {}, std::nullopt, std::nullopt, {}, nullptr, "Test"}); @@ -57,21 +57,17 @@ struct RefinementClassFixture : Fixture TypePackId isARets = arena.addTypePack({typeChecker.booleanType}); TypeId isA = arena.addType(FunctionTypeVar{isAParams, isARets}); getMutable(isA)->magicFunction = magicFunctionInstanceIsA; - normalize(isA, scope, arena, singletonTypes, *typeChecker.iceHandler); getMutable(inst)->props = { {"Name", Property{typeChecker.stringType}}, {"IsA", Property{isA}}, }; - normalize(inst, scope, arena, singletonTypes, *typeChecker.iceHandler); TypeId folder = typeChecker.globalTypes.addType(ClassTypeVar{"Folder", {}, inst, std::nullopt, {}, nullptr, "Test"}); - normalize(folder, scope, arena, singletonTypes, *typeChecker.iceHandler); TypeId part = typeChecker.globalTypes.addType(ClassTypeVar{"Part", {}, inst, std::nullopt, {}, nullptr, "Test"}); getMutable(part)->props = { {"Position", Property{vec3}}, }; - normalize(part, scope, arena, singletonTypes, *typeChecker.iceHandler); typeChecker.globalScope->exportedTypeBindings["Vector3"] = TypeFun{{}, vec3}; typeChecker.globalScope->exportedTypeBindings["Instance"] = TypeFun{{}, inst}; @@ -102,8 +98,16 @@ TEST_CASE_FIXTURE(Fixture, "is_truthy_constraint") LUAU_REQUIRE_NO_ERRORS(result); - CHECK_EQ("string", toString(requireTypeAtPosition({3, 26}))); - CHECK_EQ("nil", toString(requireTypeAtPosition({5, 26}))); + if (FFlag::DebugLuauDeferredConstraintResolution) + { + CHECK_EQ("(string?) & ~(false?)", toString(requireTypeAtPosition({3, 26}))); + CHECK_EQ("(string?) & ~~(false?)", toString(requireTypeAtPosition({5, 26}))); + } + else + { + CHECK_EQ("string", toString(requireTypeAtPosition({3, 26}))); + CHECK_EQ("nil", toString(requireTypeAtPosition({5, 26}))); + } } TEST_CASE_FIXTURE(Fixture, "invert_is_truthy_constraint") @@ -120,8 +124,16 @@ TEST_CASE_FIXTURE(Fixture, "invert_is_truthy_constraint") LUAU_REQUIRE_NO_ERRORS(result); - CHECK_EQ("nil", toString(requireTypeAtPosition({3, 26}))); - CHECK_EQ("string", toString(requireTypeAtPosition({5, 26}))); + if (FFlag::DebugLuauDeferredConstraintResolution) + { + CHECK_EQ("(string?) & ~~(false?)", toString(requireTypeAtPosition({3, 26}))); + CHECK_EQ("(string?) & ~~~(false?)", toString(requireTypeAtPosition({5, 26}))); + } + else + { + CHECK_EQ("nil", toString(requireTypeAtPosition({3, 26}))); + CHECK_EQ("string", toString(requireTypeAtPosition({5, 26}))); + } } TEST_CASE_FIXTURE(Fixture, "parenthesized_expressions_are_followed_through") @@ -138,8 +150,16 @@ TEST_CASE_FIXTURE(Fixture, "parenthesized_expressions_are_followed_through") LUAU_REQUIRE_NO_ERRORS(result); - CHECK_EQ("nil", toString(requireTypeAtPosition({3, 26}))); - CHECK_EQ("string", toString(requireTypeAtPosition({5, 26}))); + if (FFlag::DebugLuauDeferredConstraintResolution) + { + CHECK_EQ("(string?) & ~~(false?)", toString(requireTypeAtPosition({3, 26}))); + CHECK_EQ("(string?) & ~~~(false?)", toString(requireTypeAtPosition({5, 26}))); + } + else + { + CHECK_EQ("nil", toString(requireTypeAtPosition({3, 26}))); + CHECK_EQ("string", toString(requireTypeAtPosition({5, 26}))); + } } TEST_CASE_FIXTURE(Fixture, "and_constraint") @@ -963,19 +983,27 @@ TEST_CASE_FIXTURE(Fixture, "and_or_peephole_refinement") TEST_CASE_FIXTURE(Fixture, "narrow_boolean_to_true_or_false") { CheckResult result = check(R"( - local function is_true(b: true) end - local function is_false(b: false) end - local function f(x: boolean) if x then - is_true(x) + local foo = x else - is_false(x) + local foo = x end end )"); LUAU_REQUIRE_NO_ERRORS(result); + + if (FFlag::DebugLuauDeferredConstraintResolution) + { + CHECK_EQ("boolean & ~(false?)", toString(requireTypeAtPosition({3, 28}))); + CHECK_EQ("boolean & ~~(false?)", toString(requireTypeAtPosition({5, 28}))); + } + else + { + CHECK_EQ("true", toString(requireTypeAtPosition({3, 28}))); + CHECK_EQ("false", toString(requireTypeAtPosition({5, 28}))); + } } TEST_CASE_FIXTURE(Fixture, "discriminate_on_properties_of_disjoint_tables_where_that_property_is_true_or_false") diff --git a/tests/TypeInfer.tables.test.cpp b/tests/TypeInfer.tables.test.cpp index 53f9a1ab..2a208cce 100644 --- a/tests/TypeInfer.tables.test.cpp +++ b/tests/TypeInfer.tables.test.cpp @@ -11,6 +11,8 @@ using namespace Luau; +LUAU_FASTFLAG(LuauLowerBoundsCalculation); +LUAU_FASTFLAG(DebugLuauDeferredConstraintResolution); LUAU_FASTFLAG(LuauInstantiateInSubtyping) TEST_SUITE_BEGIN("TableTests"); @@ -44,7 +46,7 @@ TEST_CASE_FIXTURE(Fixture, "augment_table") const TableTypeVar* tType = get(requireType("t")); REQUIRE(tType != nullptr); - CHECK(tType->props.find("foo") != tType->props.end()); + CHECK(1 == tType->props.count("foo")); } TEST_CASE_FIXTURE(Fixture, "augment_nested_table") @@ -101,7 +103,11 @@ TEST_CASE_FIXTURE(Fixture, "updating_sealed_table_prop_is_ok") TEST_CASE_FIXTURE(Fixture, "cannot_change_type_of_unsealed_table_prop") { - CheckResult result = check("local t = {} t.prop = 999 t.prop = 'hello'"); + CheckResult result = check(R"( + local t = {} + t.prop = 999 + t.prop = 'hello' + )"); LUAU_REQUIRE_ERROR_COUNT(1, result); } @@ -858,11 +864,12 @@ TEST_CASE_FIXTURE(Fixture, "assigning_to_an_unsealed_table_with_string_literal_s LUAU_REQUIRE_NO_ERRORS(result); - CHECK_EQ(*typeChecker.stringType, *requireType("a")); + CHECK("string" == toString(*typeChecker.stringType)); TableTypeVar* tableType = getMutable(requireType("t")); REQUIRE(tableType != nullptr); REQUIRE(tableType->indexer == std::nullopt); + REQUIRE(0 != tableType->props.count("a")); TypeId propertyA = tableType->props["a"].type; REQUIRE(propertyA != nullptr); @@ -2390,9 +2397,12 @@ TEST_CASE_FIXTURE(Fixture, "wrong_assign_does_hit_indexer") TEST_CASE_FIXTURE(Fixture, "nil_assign_doesnt_hit_no_indexer") { - CheckResult result = check("local a = {a=1, b=2} a['a'] = nil"); + CheckResult result = check(R"( + local a = {a=1, b=2} + a['a'] = nil + )"); LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK_EQ(result.errors[0], (TypeError{Location{Position{0, 30}, Position{0, 33}}, TypeMismatch{ + CHECK_EQ(result.errors[0], (TypeError{Location{Position{2, 17}, Position{2, 20}}, TypeMismatch{ typeChecker.numberType, typeChecker.nilType, }})); @@ -2701,6 +2711,62 @@ local baz = foo[bar] CHECK_EQ(result.errors[0].location, Location{Position{3, 16}, Position{3, 19}}); } +TEST_CASE_FIXTURE(BuiltinsFixture, "table_call_metamethod_basic") +{ + if (!FFlag::DebugLuauDeferredConstraintResolution) + return; + + CheckResult result = check(R"( + local a = setmetatable({ + a = 1, + }, { + __call = function(self, b: number) + return self.a * b + end, + }) + + local foo = a(12) + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + CHECK(requireType("foo") == singletonTypes->numberType); +} + +TEST_CASE_FIXTURE(BuiltinsFixture, "table_call_metamethod_must_be_callable") +{ + CheckResult result = check(R"( + local a = setmetatable({}, { + __call = 123, + }) + + local foo = a() + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK(result.errors[0] == TypeError{ + Location{{5, 20}, {5, 21}}, + CannotCallNonFunction{singletonTypes->numberType}, + }); +} + +TEST_CASE_FIXTURE(BuiltinsFixture, "table_call_metamethod_generic") +{ + CheckResult result = check(R"( + local a = setmetatable({}, { + __call = function(self, b: T) + return b + end, + }) + + local foo = a(12) + local bar = a("bar") + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + CHECK(requireType("foo") == singletonTypes->numberType); + CHECK(requireType("bar") == singletonTypes->stringType); +} + TEST_CASE_FIXTURE(BuiltinsFixture, "table_simple_call") { CheckResult result = check(R"( diff --git a/tests/TypeInfer.test.cpp b/tests/TypeInfer.test.cpp index 239b8c28..dff9649f 100644 --- a/tests/TypeInfer.test.cpp +++ b/tests/TypeInfer.test.cpp @@ -1046,7 +1046,6 @@ TEST_CASE_FIXTURE(Fixture, "type_infer_recursion_limit_normalizer") ScopedFastFlag sffs[]{ {"LuauSubtypeNormalizer", true}, {"LuauTypeNormalization2", true}, - {"LuauAutocompleteDynamicLimits", true}, }; CheckResult result = check(R"( diff --git a/tests/TypeInfer.typePacks.cpp b/tests/TypeInfer.typePacks.cpp index aaa7ded4..4c8eeac6 100644 --- a/tests/TypeInfer.typePacks.cpp +++ b/tests/TypeInfer.typePacks.cpp @@ -467,6 +467,8 @@ type I = W TEST_CASE_FIXTURE(Fixture, "type_alias_type_pack_explicit") { + ScopedFastFlag sff("LuauFunctionReturnStringificationFixup", true); + CheckResult result = check(R"( type X = (T...) -> (T...) @@ -490,6 +492,8 @@ type F = X<(string, ...number)> TEST_CASE_FIXTURE(Fixture, "type_alias_type_pack_explicit_multi") { + ScopedFastFlag sff("LuauFunctionReturnStringificationFixup", true); + CheckResult result = check(R"( type Y = (T...) -> (U...) diff --git a/tests/TypeVar.test.cpp b/tests/TypeVar.test.cpp index b81c80ce..5dd1b1bc 100644 --- a/tests/TypeVar.test.cpp +++ b/tests/TypeVar.test.cpp @@ -436,7 +436,6 @@ TEST_CASE("proof_that_isBoolean_uses_all_of") TEST_CASE("content_reassignment") { TypeVar myAny{AnyTypeVar{}, /*presistent*/ true}; - myAny.normal = true; myAny.documentationSymbol = "@global/any"; TypeArena arena; @@ -446,7 +445,6 @@ TEST_CASE("content_reassignment") CHECK(get(futureAny) != nullptr); CHECK(!futureAny->persistent); - CHECK(futureAny->normal); CHECK(futureAny->documentationSymbol == "@global/any"); CHECK(futureAny->owningArena == &arena); } diff --git a/tests/conformance/basic.lua b/tests/conformance/basic.lua index b7e85aa7..7a05f8e9 100644 --- a/tests/conformance/basic.lua +++ b/tests/conformance/basic.lua @@ -93,7 +93,10 @@ assert((function() local a = 1 a = a * 2 return a end)() == 2) assert((function() local a = 1 a = a / 2 return a end)() == 0.5) assert((function() local a = 5 a = a % 2 return a end)() == 1) assert((function() local a = 3 a = a ^ 2 return a end)() == 9) +assert((function() local a = 3 a = a ^ 3 return a end)() == 27) assert((function() local a = 9 a = a ^ 0.5 return a end)() == 3) +assert((function() local a = -2 a = a ^ 2 return a end)() == 4) +assert((function() local a = -2 a = a ^ 0.5 return tostring(a) end)() == "nan") assert((function() local a = '1' a = a .. '2' return a end)() == "12") assert((function() local a = '1' a = a .. '2' .. '3' return a end)() == "123") @@ -706,7 +709,11 @@ end assert(chainTest(100) == "v0,v100") -- this validates import fallbacks +assert(idontexist == nil) +assert(math.idontexist == nil) assert(pcall(function() return idontexist.a end) == false) +assert(pcall(function() return math.pow.a end) == false) +assert(pcall(function() return math.a.b end) == false) -- make sure that NaN is preserved by the bytecode compiler local realnan = tostring(math.abs(0)/math.abs(0)) diff --git a/tests/conformance/calls.lua b/tests/conformance/calls.lua index 7f9610a3..621a921a 100644 --- a/tests/conformance/calls.lua +++ b/tests/conformance/calls.lua @@ -226,4 +226,14 @@ assert((function () return nil end)(4) == nil) assert((function () local a; return a end)(4) == nil) assert((function (a) return a end)() == nil) +-- C-stack overflow while handling C-stack overflow +if not limitedstack then + local function loop () + assert(pcall(loop)) + end + + local err, msg = xpcall(loop, loop) + assert(not err and string.find(msg, "error")) +end + return('OK') diff --git a/tests/conformance/datetime.lua b/tests/conformance/datetime.lua index ca35cf2f..dc73948b 100644 --- a/tests/conformance/datetime.lua +++ b/tests/conformance/datetime.lua @@ -16,6 +16,7 @@ D = os.date("*t", t) assert(os.date(string.rep("%d", 1000), t) == string.rep(os.date("%d", t), 1000)) assert(os.date(string.rep("%", 200)) == string.rep("%", 100)) +assert(os.date("", -1) == nil) local function checkDateTable (t) local D = os.date("!*t", t) diff --git a/tests/conformance/errors.lua b/tests/conformance/errors.lua index 529e9b0c..57d2b693 100644 --- a/tests/conformance/errors.lua +++ b/tests/conformance/errors.lua @@ -405,5 +405,7 @@ assert(ecall(function() (""):foo() end) == "attempt to call missing method 'foo' assert(ecall(function() (42):foo() end) == "attempt to index number with 'foo'") assert(ecall(function() ({foo=42}):foo() end) == "attempt to call a number value") assert(ecall(function() local ud = newproxy(true) getmetatable(ud).__index = {} ud:foo() end) == "attempt to call missing method 'foo' of userdata") +assert(ecall(function() local ud = newproxy(true) getmetatable(ud).__index = function() end ud:foo() end) == "attempt to call missing method 'foo' of userdata") +assert(ecall(function() local ud = newproxy(true) getmetatable(ud).__index = function() error("nope") end ud:foo() end) == "nope") return('OK') diff --git a/tests/conformance/events.lua b/tests/conformance/events.lua index 447b67bc..94314c3f 100644 --- a/tests/conformance/events.lua +++ b/tests/conformance/events.lua @@ -13,6 +13,11 @@ assert(getmetatable(a) == "xuxu") ud=newproxy(true); getmetatable(ud).__metatable = "xuxu" assert(getmetatable(ud) == "xuxu") +assert(pcall(getmetatable) == false) +assert(pcall(function() return getmetatable() end) == false) +assert(select(2, pcall(getmetatable, {})) == nil) +assert(select(2, pcall(getmetatable, ud)) == "xuxu") + local res,err = pcall(tostring, a) assert(not res and err == "'__tostring' must return a string") -- cannot change a protected metatable @@ -475,6 +480,9 @@ function testfenv() assert(_G.X == 20) assert(_G == getfenv(0)) + + assert(pcall(getfenv, 10) == false) + assert(pcall(setfenv, setfenv, {}) == false) end testfenv() -- DONT MOVE THIS LINE diff --git a/tests/conformance/iter.lua b/tests/conformance/iter.lua index 468ffafb..5f8f1a89 100644 --- a/tests/conformance/iter.lua +++ b/tests/conformance/iter.lua @@ -193,4 +193,24 @@ do assert(x == 15) end +-- pairs/ipairs/next may be substituted through getfenv +-- however, they *must* be substituted with functions - we don't support them falling back to generalized iteration +function testgetfenv() + local env = getfenv(1) + env.pairs = function() return "nope" end + env.ipairs = function() return "nope" end + env.next = {1, 2, 3} + + local ok, err = pcall(function() for k, v in pairs({}) do end end) + assert(not ok and err:match("attempt to iterate over a string value")) + + local ok, err = pcall(function() for k, v in ipairs({}) do end end) + assert(not ok and err:match("attempt to iterate over a string value")) + + local ok, err = pcall(function() for k, v in next, {} do end end) + assert(not ok and err:match("attempt to iterate over a table value")) +end + +testgetfenv() -- DONT MOVE THIS LINE + return"OK" diff --git a/tests/conformance/math.lua b/tests/conformance/math.lua index 0cd0cdce..972c399b 100644 --- a/tests/conformance/math.lua +++ b/tests/conformance/math.lua @@ -283,6 +283,13 @@ assert(math.fmod(-3, 2) == -1) assert(math.fmod(3, -2) == 1) assert(math.fmod(-3, -2) == -1) +-- pow +assert(math.pow(2, 0) == 1) +assert(math.pow(2, 2) == 4) +assert(math.pow(4, 0.5) == 2) +assert(math.pow(-2, 2) == 4) +assert(tostring(math.pow(-2, 0.5)) == "nan") + -- most of the tests above go through fastcall path -- to make sure the basic implementations are also correct we test these functions with string->number coercions assert(math.abs("-4") == 4) diff --git a/tests/conformance/move.lua b/tests/conformance/move.lua index 3f28b4b3..27a96ffc 100644 --- a/tests/conformance/move.lua +++ b/tests/conformance/move.lua @@ -74,4 +74,6 @@ checkerror("wrap around", table.move, {}, 1, maxI, 2) checkerror("wrap around", table.move, {}, 1, 2, maxI) checkerror("wrap around", table.move, {}, minI, -2, 2) +checkerror("readonly", table.move, table.freeze({}), 1, 1, 1) + return"OK" diff --git a/tests/conformance/strings.lua b/tests/conformance/strings.lua index 3d8fdd1f..61bac726 100644 --- a/tests/conformance/strings.lua +++ b/tests/conformance/strings.lua @@ -48,6 +48,7 @@ assert(string.find("", "") == 1) assert(string.find('', 'aaa', 1) == nil) assert(('alo(.)alo'):find('(.)', 1, 1) == 4) assert(string.find('', '1', 2) == nil) +assert(string.find('123', '2', 0) == 2) print('+') assert(string.len("") == 0) @@ -88,6 +89,8 @@ assert(string.lower("\0ABCc%$") == "\0abcc%$") assert(string.rep('teste', 0) == '') assert(string.rep('ts\00t', 2) == 'ts\0tts\000t') assert(string.rep('', 10) == '') +assert(string.rep('', 1e9) == '') +assert(pcall(string.rep, 'x', 2e9) == false) assert(string.reverse"" == "") assert(string.reverse"\0\1\2\3" == "\3\2\1\0") @@ -126,6 +129,13 @@ assert(string.format("-%.20s.20s", string.rep("%", 2000)) == "-"..string.rep("%" assert(string.format('"-%20s.20s"', string.rep("%", 2000)) == string.format("%q", "-"..string.rep("%", 2000)..".20s")) +assert(string.format("%o %u %x %X", -1, -1, -1, -1) == "1777777777777777777777 18446744073709551615 ffffffffffffffff FFFFFFFFFFFFFFFF") + +assert(string.format("%e %E", 1.5, -1.5) == "1.500000e+00 -1.500000E+00") + +assert(pcall(string.format, "%##################d", 1) == false) +assert(pcall(string.format, "%.123d", 1) == false) +assert(pcall(string.format, "%?", 1) == false) -- longest number that can be formated assert(string.len(string.format('%99.99f', -1e308)) >= 100) @@ -179,6 +189,26 @@ assert(table.concat(a, ",", 2) == "b,c") assert(table.concat(a, ",", 3) == "c") assert(table.concat(a, ",", 4) == "") +-- string.split +do + local function eq(a, b) + if #a ~= #b then + return false + end + for i=1,#a do + if a[i] ~= b[i] then + return false + end + end + return true + end + + assert(eq(string.split("abc", ""), {'a', 'b', 'c'})) + assert(eq(string.split("abc", "b"), {'a', 'c'})) + assert(eq(string.split("abc", "d"), {'abc'})) + assert(eq(string.split("abc", "c"), {'ab', ''})) +end + --[[ local locales = { "ptb", "ISO-8859-1", "pt_BR" } local function trylocale (w) diff --git a/tests/conformance/tables.lua b/tests/conformance/tables.lua index 0eff8540..7ae80cc4 100644 --- a/tests/conformance/tables.lua +++ b/tests/conformance/tables.lua @@ -87,35 +87,59 @@ print'+' -- testing tables dynamically built local lim = 130 -local a = {}; a[2] = 1; check(a, 0, 1) -a = {}; a[0] = 1; check(a, 0, 1); a[2] = 1; check(a, 0, 2) -a = {}; a[0] = 1; a[1] = 1; check(a, 1, 1) -a = {} -for i = 1,lim do - a[i] = 1 - assert(#a == i) - check(a, mp2(i), 0) + +do + local a = {}; a[2] = 1; check(a, 0, 1) + a = {}; a[0] = 1; check(a, 0, 1); a[2] = 1; check(a, 0, 2) + a = {}; a[0] = 1; a[1] = 1; check(a, 1, 1) + a = {} + for i = 1,lim do + a[i] = 1 + assert(#a == i) + check(a, mp2(i), 0) + end end -a = {} -for i = 1,lim do - a['a'..i] = 1 - assert(#a == 0) - check(a, 0, mp2(i)) +do + local a = {} + for i = 1,lim do + a['a'..i] = 1 + assert(#a == 0) + check(a, 0, mp2(i)) + end end -a = {} -for i=1,16 do a[i] = i end -check(a, 16, 0) -for i=1,11 do a[i] = nil end -for i=30,40 do a[i] = nil end -- force a rehash (?) -check(a, 0, 8) -a[10] = 1 -for i=30,40 do a[i] = nil end -- force a rehash (?) -check(a, 0, 8) -for i=1,14 do a[i] = nil end -for i=30,50 do a[i] = nil end -- force a rehash (?) -check(a, 0, 4) +do + local a = {} + for i=1,16 do a[i] = i end + check(a, 16, 0) + for i=1,11 do a[i] = nil end + for i=30,40 do a[i] = nil end -- force a rehash (?) + check(a, 0, 8) + a[10] = 1 + for i=30,40 do a[i] = nil end -- force a rehash (?) + check(a, 0, 8) + for i=1,14 do a[i] = nil end + for i=30,50 do a[i] = nil end -- force a rehash (?) + check(a, 0, 4) +end + +do -- rehash moving elements from array to hash + local a = {} + for i = 1, 100 do a[i] = i end + check(a, 128, 0) + + for i = 5, 95 do a[i] = nil end + check(a, 128, 0) + + a.x = 1 -- force a re-hash + check(a, 4, 8) + + for i = 1, 4 do assert(a[i] == i) end + for i = 5, 95 do assert(a[i] == nil) end + for i = 96, 100 do assert(a[i] == i) end + assert(a.x == 1) +end -- reverse filling for i=1,lim do @@ -612,4 +636,54 @@ do assert(hit and child.foo == nil and parent.foo == nil) end +-- testing next x GC of deleted keys +do + local co = coroutine.wrap(function (t) + for k, v in pairs(t) do + local k1 = next(t) -- all previous keys were deleted + assert(k == k1) -- current key is the first in the table + t[k] = nil + local expected = (type(k) == "table" and k[1] or + type(k) == "function" and k() or + string.sub(k, 1, 1)) + assert(expected == v) + coroutine.yield(v) + end + end) + local t = {} + t[{1}] = 1 -- add several unanchored, collectable keys + t[{2}] = 2 + t[string.rep("a", 50)] = "a" -- long string + t[string.rep("b", 50)] = "b" + t[{3}] = 3 + t[string.rep("c", 10)] = "c" -- short string + t[function () return 10 end] = 10 + local count = 7 + while co(t) do + collectgarbage("collect") -- collect dead keys + count = count - 1 + end + assert(count == 0 and next(t) == nil) -- traversed the whole table +end + +-- test error cases for table functions +do + assert(pcall(table.insert, {}) == false) + assert(pcall(table.insert, {}, 1, 2, 3) == false) + assert(pcall(table.insert, table.freeze({1, 2, 3}), 4) == false) + assert(pcall(table.insert, table.freeze({1, 2, 3}), 1, 4) == false) + + assert(pcall(table.remove, table.freeze({1})) == false) + + assert(pcall(table.concat, {true}) == false) + + assert(pcall(table.create) == false) + assert(pcall(table.create, -1) == false) + assert(pcall(table.create, 1e9) == false) + + assert(pcall(table.find, {}, 42, 0) == false) + + assert(pcall(table.clear, table.freeze({})) == false) +end + return"OK" diff --git a/tests/conformance/tpack.lua b/tests/conformance/tpack.lua index 835bf564..b240f482 100644 --- a/tests/conformance/tpack.lua +++ b/tests/conformance/tpack.lua @@ -306,6 +306,8 @@ do -- testing initial position assert(i == 4 and p == 17) local i, p = unpack("!4 i4", x, -#x) assert(i == 1 and p == 5) + local i, p = unpack("!4 i4", x, 0) + assert(i == 1 and p == 5) -- limits for i = 1, #x + 1 do diff --git a/tools/faillist.txt b/tools/faillist.txt index 0eb02209..c869e0c4 100644 --- a/tools/faillist.txt +++ b/tools/faillist.txt @@ -18,12 +18,10 @@ AutocompleteTest.autocomplete_string_singleton_equality AutocompleteTest.autocomplete_string_singleton_escape AutocompleteTest.autocomplete_string_singletons AutocompleteTest.autocompleteProp_index_function_metamethod_is_variadic -AutocompleteTest.cyclic_table AutocompleteTest.do_compatible_self_calls AutocompleteTest.do_wrong_compatible_self_calls AutocompleteTest.keyword_methods AutocompleteTest.no_incompatible_self_calls -AutocompleteTest.no_incompatible_self_calls_2 AutocompleteTest.no_wrong_compatible_self_calls_with_generics AutocompleteTest.suggest_table_keys AutocompleteTest.type_correct_argument_type_suggestion @@ -40,8 +38,6 @@ AutocompleteTest.type_correct_keywords AutocompleteTest.type_correct_suggestion_for_overloads AutocompleteTest.type_correct_suggestion_in_argument AutocompleteTest.type_correct_suggestion_in_table -AutocompleteTest.unsealed_table -AutocompleteTest.unsealed_table_2 BuiltinTests.aliased_string_format BuiltinTests.assert_removes_falsy_types BuiltinTests.assert_removes_falsy_types2 @@ -75,7 +71,6 @@ BuiltinTests.select_with_decimal_argument_is_rounded_down BuiltinTests.set_metatable_needs_arguments BuiltinTests.setmetatable_should_not_mutate_persisted_types BuiltinTests.sort_with_bad_predicate -BuiltinTests.sort_with_predicate BuiltinTests.string_format_arg_count_mismatch BuiltinTests.string_format_arg_types_inference BuiltinTests.string_format_as_method @@ -93,6 +88,7 @@ BuiltinTests.table_pack_variadic BuiltinTests.tonumber_returns_optional_number_type BuiltinTests.tonumber_returns_optional_number_type2 DefinitionTests.class_definition_overload_metamethods +DefinitionTests.class_definition_string_props DefinitionTests.declaring_generic_functions DefinitionTests.definition_file_classes FrontendTest.environments @@ -128,7 +124,7 @@ GenericsTests.higher_rank_polymorphism_should_not_accept_instantiated_arguments GenericsTests.infer_generic_function_function_argument GenericsTests.infer_generic_function_function_argument_overloaded GenericsTests.infer_generic_methods -GenericsTests.inferred_local_vars_can_be_polytypes +GenericsTests.infer_generic_property GenericsTests.instantiate_cyclic_generic_function GenericsTests.instantiated_function_argument_names GenericsTests.instantiation_sharing_types @@ -173,6 +169,7 @@ ProvisionalTests.do_not_ice_when_trying_to_pick_first_of_generic_type_pack ProvisionalTests.error_on_eq_metamethod_returning_a_type_other_than_boolean ProvisionalTests.generic_type_leak_to_module_interface_variadic ProvisionalTests.greedy_inference_with_shared_self_triggers_function_with_no_returns +ProvisionalTests.lvalue_equals_another_lvalue_with_no_overlap ProvisionalTests.pcall_returns_at_least_two_value_but_function_returns_nothing ProvisionalTests.setmetatable_constrains_free_type_into_free_table ProvisionalTests.specialization_binds_with_prototypes_too_early @@ -234,17 +231,14 @@ RefinementTest.typeguard_not_to_be_string RefinementTest.what_nonsensical_condition RefinementTest.x_as_any_if_x_is_instance_elseif_x_is_table RefinementTest.x_is_not_instance_or_else_not_part +RuntimeLimits.typescript_port_of_Result_type TableTests.a_free_shape_can_turn_into_a_scalar_if_it_is_compatible TableTests.a_free_shape_cannot_turn_into_a_scalar_if_it_is_not_compatible TableTests.access_index_metamethod_that_returns_variadic TableTests.accidentally_checked_prop_in_opposite_branch -TableTests.assigning_to_an_unsealed_table_with_string_literal_should_infer_new_properties_over_indexer -TableTests.augment_nested_table -TableTests.augment_table TableTests.builtin_table_names TableTests.call_method TableTests.cannot_augment_sealed_table -TableTests.cannot_change_type_of_unsealed_table_prop TableTests.casting_sealed_tables_with_props_into_table_with_indexer TableTests.casting_tables_with_props_into_table_with_indexer3 TableTests.casting_tables_with_props_into_table_with_indexer4 @@ -294,6 +288,7 @@ TableTests.metatable_mismatch_should_fail TableTests.missing_metatable_for_sealed_tables_do_not_get_inferred TableTests.mixed_tables_with_implicit_numbered_keys TableTests.nil_assign_doesnt_hit_indexer +TableTests.nil_assign_doesnt_hit_no_indexer TableTests.okay_to_add_property_to_unsealed_tables_by_function_call TableTests.only_ascribe_synthetic_names_at_module_scope TableTests.oop_indexer_works @@ -347,7 +342,6 @@ ToString.toStringNamedFunction_id ToString.toStringNamedFunction_include_self_param ToString.toStringNamedFunction_map ToString.toStringNamedFunction_variadics -TranspilerTests.types_should_not_be_considered_cyclic_if_they_are_not_recursive TryUnifyTests.cli_41095_concat_log_in_sealed_table_unification TryUnifyTests.members_of_failed_typepack_unification_are_unified_with_errorType TryUnifyTests.result_of_failed_typepack_unification_is_constrained @@ -377,22 +371,20 @@ TypeInfer.dont_report_type_errors_within_an_AstStatError TypeInfer.globals TypeInfer.globals2 TypeInfer.infer_assignment_value_types_mutable_lval +TypeInfer.it_is_ok_to_have_inconsistent_number_of_return_values_in_nonstrict TypeInfer.no_stack_overflow_from_isoptional TypeInfer.tc_after_error_recovery_no_replacement_name_in_error TypeInfer.tc_if_else_expressions_expected_type_3 TypeInfer.tc_interpolated_string_basic -TypeInfer.tc_interpolated_string_constant_type TypeInfer.tc_interpolated_string_with_invalid_expression TypeInfer.type_infer_recursion_limit_no_ice -TypeInferAnyError.assign_prop_to_table_by_calling_any_yields_any +TypeInfer.type_infer_recursion_limit_normalizer TypeInferAnyError.for_in_loop_iterator_is_any2 TypeInferAnyError.for_in_loop_iterator_is_error2 TypeInferClasses.call_base_method TypeInferClasses.call_instance_method -TypeInferClasses.can_assign_to_prop_of_base_class_using_string TypeInferClasses.can_read_prop_of_base_class_using_string TypeInferClasses.class_type_mismatch_with_name_conflict -TypeInferClasses.classes_can_have_overloaded_operators TypeInferClasses.classes_without_overloaded_operators_cannot_be_added TypeInferClasses.detailed_class_unification_error TypeInferClasses.higher_order_function_arguments_are_contravariant @@ -420,6 +412,7 @@ TypeInferFunctions.infer_return_value_type TypeInferFunctions.infer_that_function_does_not_return_a_table TypeInferFunctions.list_all_overloads_if_no_overload_takes_given_argument_count TypeInferFunctions.list_only_alternative_overloads_that_match_argument_count +TypeInferFunctions.luau_subtyping_is_np_hard TypeInferFunctions.no_lossy_function_type TypeInferFunctions.occurs_check_failure_in_function_return_type TypeInferFunctions.record_matching_overload @@ -459,9 +452,6 @@ TypeInferOOP.dont_suggest_using_colon_rather_than_dot_if_it_wont_help_2 TypeInferOOP.dont_suggest_using_colon_rather_than_dot_if_not_defined_with_colon TypeInferOOP.inferring_hundreds_of_self_calls_should_not_suffocate_memory TypeInferOOP.methods_are_topologically_sorted -TypeInferOperators.and_adds_boolean -TypeInferOperators.and_adds_boolean_no_superfluous_union -TypeInferOperators.and_binexps_dont_unify TypeInferOperators.and_or_ternary TypeInferOperators.CallAndOrOfFunctions TypeInferOperators.cannot_compare_tables_that_do_not_have_the_same_metatable @@ -471,23 +461,11 @@ TypeInferOperators.cli_38355_recursive_union TypeInferOperators.compound_assign_mismatch_metatable TypeInferOperators.compound_assign_mismatch_op TypeInferOperators.compound_assign_mismatch_result -TypeInferOperators.concat_op_on_free_lhs_and_string_rhs TypeInferOperators.disallow_string_and_types_without_metatables_from_arithmetic_binary_ops -TypeInferOperators.dont_strip_nil_from_rhs_or_operator -TypeInferOperators.equality_operations_succeed_if_any_union_branch_succeeds -TypeInferOperators.error_on_invalid_operand_types_to_relational_operators -TypeInferOperators.error_on_invalid_operand_types_to_relational_operators2 -TypeInferOperators.expected_types_through_binary_and -TypeInferOperators.expected_types_through_binary_or +TypeInferOperators.in_nonstrict_mode_strip_nil_from_intersections_when_considering_relational_operators TypeInferOperators.infer_any_in_all_modes_when_lhs_is_unknown -TypeInferOperators.or_joins_types -TypeInferOperators.or_joins_types_with_no_extras -TypeInferOperators.primitive_arith_possible_metatable TypeInferOperators.produce_the_correct_error_message_when_comparing_a_table_with_a_metatable_with_one_that_does_not TypeInferOperators.refine_and_or -TypeInferOperators.strict_binary_op_where_lhs_unknown -TypeInferOperators.strip_nil_from_lhs_or_operator -TypeInferOperators.strip_nil_from_lhs_or_operator2 TypeInferOperators.typecheck_overloaded_multiply_that_is_an_intersection TypeInferOperators.typecheck_overloaded_multiply_that_is_an_intersection_on_rhs TypeInferOperators.typecheck_unary_len_error @@ -535,26 +513,17 @@ TypePackTests.unify_variadic_tails_in_arguments TypePackTests.unify_variadic_tails_in_arguments_free TypePackTests.varargs_inference_through_multiple_scopes TypePackTests.variadic_packs -TypeSingletons.enums_using_singletons -TypeSingletons.enums_using_singletons_mismatch -TypeSingletons.enums_using_singletons_subtyping TypeSingletons.error_detailed_tagged_union_mismatch_bool TypeSingletons.error_detailed_tagged_union_mismatch_string TypeSingletons.function_call_with_singletons TypeSingletons.function_call_with_singletons_mismatch -TypeSingletons.if_then_else_expression_singleton_options TypeSingletons.indexing_on_string_singletons TypeSingletons.indexing_on_union_of_string_singletons -TypeSingletons.no_widening_from_callsites TypeSingletons.overloaded_function_call_with_singletons TypeSingletons.overloaded_function_call_with_singletons_mismatch TypeSingletons.return_type_of_f_is_not_widened -TypeSingletons.string_singleton_subtype -TypeSingletons.string_singletons -TypeSingletons.string_singletons_escape_chars -TypeSingletons.string_singletons_mismatch +TypeSingletons.table_properties_singleton_strings_mismatch TypeSingletons.table_properties_type_error_escapes -TypeSingletons.tagged_unions_using_singletons TypeSingletons.taking_the_length_of_string_singleton TypeSingletons.taking_the_length_of_union_of_string_singleton TypeSingletons.widen_the_supertype_if_it_is_free_and_subtype_has_singleton diff --git a/tools/test_dcr.py b/tools/test_dcr.py index 76bf11ac..6d553b64 100644 --- a/tools/test_dcr.py +++ b/tools/test_dcr.py @@ -42,6 +42,8 @@ class Handler(x.ContentHandler): self.fail_count = 0 self.test_count = 0 + self.crashed_tests = [] + def startElement(self, name, attrs): if name == "TestSuite": self.currentTest.append(attrs["name"]) @@ -69,6 +71,10 @@ class Handler(x.ContentHandler): elif name == "OverallResultsTestCases": self.numSkippedTests = safeParseInt(attrs.get("skipped", 0)) + elif name == "Exception": + if attrs.get("crash") == "true": + self.crashed_tests.append(makeDottedName(self.currentTest)) + def endElement(self, name): if name == "TestCase": self.currentTest.pop() @@ -192,15 +198,23 @@ def main(): print(name, file=f) print_stderr("Updated faillist.txt") - if handler.numSkippedTests > 0: - print_stderr( - f"{handler.numSkippedTests} test(s) were skipped! That probably means that a test segfaulted!" - ) - sys.exit(1) + if handler.crashed_tests: + print_stderr() + for test in handler.crashed_tests: + print_stderr( + f"{c.Fore.RED}{test}{c.Fore.RESET} threw an exception and crashed the test process!" + ) - ok = all( - not passed == (dottedName in failList) - for dottedName, passed in handler.results.items() + if handler.numSkippedTests > 0: + print_stderr(f"{handler.numSkippedTests} test(s) were skipped!") + + ok = ( + not handler.crashed_tests + and handler.numSkippedTests == 0 + and all( + not passed == (dottedName in failList) + for dottedName, passed in handler.results.items() + ) ) if ok: From e64d489e57694231118fb157948a01d25975df81 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Mon, 24 Oct 2022 17:24:19 -0700 Subject: [PATCH 383/399] Update benchmark.yml Add codegennull runs to compile time benchmarks --- .github/workflows/benchmark.yml | 2 ++ 1 file changed, 2 insertions(+) diff --git a/.github/workflows/benchmark.yml b/.github/workflows/benchmark.yml index 4db40a62..7fb88e21 100644 --- a/.github/workflows/benchmark.yml +++ b/.github/workflows/benchmark.yml @@ -73,9 +73,11 @@ jobs: valgrind --tool=callgrind ./luau --compile=null -O0 bench/other/LuauPolyfillMap.lua 2>&1 | filter map-O0 | tee -a compile-output.txt valgrind --tool=callgrind ./luau --compile=null -O1 bench/other/LuauPolyfillMap.lua 2>&1 | filter map-O1 | tee -a compile-output.txt valgrind --tool=callgrind ./luau --compile=null -O2 bench/other/LuauPolyfillMap.lua 2>&1 | filter map-O2 | tee -a compile-output.txt + valgrind --tool=callgrind ./luau --compile=codegennull -O2 bench/other/LuauPolyfillMap.lua 2>&1 | filter map-O2-codegen | tee -a compile-output.txt valgrind --tool=callgrind ./luau --compile=null -O0 bench/other/regex.lua 2>&1 | filter regex-O0 | tee -a compile-output.txt valgrind --tool=callgrind ./luau --compile=null -O1 bench/other/regex.lua 2>&1 | filter regex-O1 | tee -a compile-output.txt valgrind --tool=callgrind ./luau --compile=null -O2 bench/other/regex.lua 2>&1 | filter regex-O2 | tee -a compile-output.txt + valgrind --tool=callgrind ./luau --compile=codegennull -O2 bench/other/regex.lua 2>&1 | filter regex-O2-codegen | tee -a compile-output.txt - name: Checkout benchmark results uses: actions/checkout@v3 From ad7d006fb29a6827165951fb9fb33980dd66eabf Mon Sep 17 00:00:00 2001 From: JohnnyMorganz Date: Wed, 26 Oct 2022 00:22:19 +0100 Subject: [PATCH 384/399] Use overloaded documentation symbol for class methods/table props (#724) Right now `getDocumentationSymbol` on an overloaded class method or table prop just gives the prop name, not prepended with `/overload/ + toString(overload)`. This causes the incorrect symbol to be used for `table.insert`, `BrickColor.new`, `Color3.new` etc. (present in https://raw.githubusercontent.com/MaximumADHD/Roblox-Client-Tracker/roblox/api-docs/en-us.json) Co-authored-by: vegorov-rbx <75688451+vegorov-rbx@users.noreply.github.com> --- Analysis/src/AstQuery.cpp | 60 +++++++++++++++++++---------------- tests/AstQuery.test.cpp | 67 +++++++++++++++++++++++++++++++++++++++ 2 files changed, 100 insertions(+), 27 deletions(-) diff --git a/Analysis/src/AstQuery.cpp b/Analysis/src/AstQuery.cpp index 50299704..2904ef8d 100644 --- a/Analysis/src/AstQuery.cpp +++ b/Analysis/src/AstQuery.cpp @@ -427,6 +427,36 @@ ExprOrLocal findExprOrLocalAtPosition(const SourceModule& source, Position pos) return findVisitor.result; } +static std::optional checkOverloadedDocumentationSymbol( + const Module& module, const TypeId ty, const AstExpr* parentExpr, const std::optional documentationSymbol) +{ + if (!documentationSymbol) + return std::nullopt; + + // This might be an overloaded function. + if (get(follow(ty))) + { + TypeId matchingOverload = nullptr; + if (parentExpr && parentExpr->is()) + { + if (auto it = module.astOverloadResolvedTypes.find(parentExpr)) + { + matchingOverload = *it; + } + } + + if (matchingOverload) + { + std::string overloadSymbol = *documentationSymbol + "/overload/"; + // Default toString options are fine for this purpose. + overloadSymbol += toString(matchingOverload); + return overloadSymbol; + } + } + + return documentationSymbol; +} + std::optional getDocumentationSymbolAtPosition(const SourceModule& source, const Module& module, Position position) { std::vector ancestry = findAstAncestryOfPosition(source, position); @@ -436,31 +466,7 @@ std::optional getDocumentationSymbolAtPosition(const Source if (std::optional binding = findBindingAtPosition(module, source, position)) { - if (binding->documentationSymbol) - { - // This might be an overloaded function binding. - if (get(follow(binding->typeId))) - { - TypeId matchingOverload = nullptr; - if (parentExpr && parentExpr->is()) - { - if (auto it = module.astOverloadResolvedTypes.find(parentExpr)) - { - matchingOverload = *it; - } - } - - if (matchingOverload) - { - std::string overloadSymbol = *binding->documentationSymbol + "/overload/"; - // Default toString options are fine for this purpose. - overloadSymbol += toString(matchingOverload); - return overloadSymbol; - } - } - } - - return binding->documentationSymbol; + return checkOverloadedDocumentationSymbol(module, binding->typeId, parentExpr, binding->documentationSymbol); } if (targetExpr) @@ -474,14 +480,14 @@ std::optional getDocumentationSymbolAtPosition(const Source { if (auto propIt = ttv->props.find(indexName->index.value); propIt != ttv->props.end()) { - return propIt->second.documentationSymbol; + return checkOverloadedDocumentationSymbol(module, propIt->second.type, parentExpr, propIt->second.documentationSymbol); } } else if (const ClassTypeVar* ctv = get(parentTy)) { if (auto propIt = ctv->props.find(indexName->index.value); propIt != ctv->props.end()) { - return propIt->second.documentationSymbol; + return checkOverloadedDocumentationSymbol(module, propIt->second.type, parentExpr, propIt->second.documentationSymbol); } } } diff --git a/tests/AstQuery.test.cpp b/tests/AstQuery.test.cpp index 4b21c443..a642334a 100644 --- a/tests/AstQuery.test.cpp +++ b/tests/AstQuery.test.cpp @@ -72,6 +72,73 @@ TEST_CASE_FIXTURE(DocumentationSymbolFixture, "overloaded_fn") CHECK_EQ(symbol, "@test/global/foo/overload/(string) -> number"); } +TEST_CASE_FIXTURE(DocumentationSymbolFixture, "class_method") +{ + loadDefinition(R"( + declare class Foo + function bar(self, x: string): number + end + )"); + + std::optional symbol = getDocSymbol(R"( + local x: Foo + x:bar("asdf") + )", + Position(2, 11)); + + CHECK_EQ(symbol, "@test/globaltype/Foo.bar"); +} + +TEST_CASE_FIXTURE(DocumentationSymbolFixture, "overloaded_class_method") +{ + loadDefinition(R"( + declare class Foo + function bar(self, x: string): number + function bar(self, x: number): string + end + )"); + + std::optional symbol = getDocSymbol(R"( + local x: Foo + x:bar("asdf") + )", + Position(2, 11)); + + CHECK_EQ(symbol, "@test/globaltype/Foo.bar/overload/(Foo, string) -> number"); +} + +TEST_CASE_FIXTURE(DocumentationSymbolFixture, "table_function_prop") +{ + loadDefinition(R"( + declare Foo: { + new: (number) -> string + } + )"); + + std::optional symbol = getDocSymbol(R"( + Foo.new("asdf") + )", + Position(1, 13)); + + CHECK_EQ(symbol, "@test/global/Foo.new"); +} + +TEST_CASE_FIXTURE(DocumentationSymbolFixture, "table_overloaded_function_prop") +{ + loadDefinition(R"( + declare Foo: { + new: ((number) -> string) & ((string) -> number) + } + )"); + + std::optional symbol = getDocSymbol(R"( + Foo.new("asdf") + )", + Position(1, 13)); + + CHECK_EQ(symbol, "@test/global/Foo.new/overload/(string) -> number"); +} + TEST_SUITE_END(); TEST_SUITE_BEGIN("AstQuery"); From a6b5051edc86069b58073aa46cb7d8e3f91a4b62 Mon Sep 17 00:00:00 2001 From: vegorov-rbx <75688451+vegorov-rbx@users.noreply.github.com> Date: Fri, 28 Oct 2022 03:37:29 -0700 Subject: [PATCH 385/399] Sync to upstream/release/551 (#727) * https://github.com/Roblox/luau/pull/719 * Improved `Failed to unify type packs` error message to be reported as `Type pack 'X' could not be converted into 'Y'` * https://github.com/Roblox/luau/pull/722 * 1% reduction in executed instruction count by removing a check in fast call dispatch * Additional fixes to reported error location of OOM errors in VM * Improve `math.sqrt`, `math.floor` and `math.ceil` performance on additional compilers and platforms (1-2% geomean improvement including 8-9% on math-cordic) * All thrown exceptions by Luau analysis are derived from `Luau::InternalCompilerError` * When a call site has fewer arguments than required, error now reports the location of the function name instead of the argument to the function * https://github.com/Roblox/luau/pull/724 * Fixed https://github.com/Roblox/luau/issues/725 Co-authored-by: Arseny Kapoulkine Co-authored-by: Andy Friesen --- .../include/Luau/ConstraintGraphBuilder.h | 55 +- Analysis/include/Luau/Error.h | 24 +- Analysis/include/Luau/Normalize.h | 75 +- Analysis/include/Luau/RecursionCounter.h | 22 +- Analysis/include/Luau/ToString.h | 2 + Analysis/include/Luau/TypeInfer.h | 13 +- Analysis/include/Luau/Unifier.h | 2 + Analysis/src/AstQuery.cpp | 47 +- Analysis/src/ConstraintGraphBuilder.cpp | 437 +++++------ Analysis/src/ConstraintSolver.cpp | 44 +- Analysis/src/Error.cpp | 43 ++ Analysis/src/Frontend.cpp | 129 +++- Analysis/src/IostreamHelpers.cpp | 2 + Analysis/src/Normalize.cpp | 406 +++++++++- Analysis/src/ToString.cpp | 40 +- Analysis/src/TopoSortStatements.cpp | 5 +- Analysis/src/TypeAttach.cpp | 2 +- Analysis/src/TypeChecker2.cpp | 61 +- Analysis/src/TypeInfer.cpp | 81 +- Analysis/src/TypePack.cpp | 3 +- Analysis/src/TypeVar.cpp | 8 +- Analysis/src/Unifier.cpp | 61 +- Ast/include/Luau/ParseResult.h | 2 + Ast/include/Luau/Parser.h | 4 +- Ast/src/Parser.cpp | 64 +- CLI/Repl.cpp | 19 +- CMakeLists.txt | 5 + CodeGen/include/Luau/CodeGen.h | 2 +- CodeGen/src/CodeGen.cpp | 691 ++++++++++-------- CodeGen/src/CodeGenUtils.cpp | 76 ++ CodeGen/src/CodeGenUtils.h | 17 + CodeGen/src/EmitCommonX64.h | 15 +- CodeGen/src/EmitInstructionX64.cpp | 383 +++++++++- CodeGen/src/EmitInstructionX64.h | 16 +- CodeGen/src/NativeState.cpp | 23 +- CodeGen/src/NativeState.h | 8 + Compiler/include/Luau/BytecodeBuilder.h | 2 +- Compiler/src/BytecodeBuilder.cpp | 18 +- Makefile | 4 +- Sources.cmake | 3 + VM/include/luaconf.h | 8 + VM/src/lbuiltins.cpp | 26 +- VM/src/lbuiltins.h | 2 +- VM/src/lnumutils.h | 1 + VM/src/lvmexecute.cpp | 52 +- bench/tests/voxelgen.lua | 456 ++++++++++++ tests/AstQuery.test.cpp | 4 + tests/Fixture.cpp | 9 + tests/Fixture.h | 2 + tests/Module.test.cpp | 10 +- tests/Normalize.test.cpp | 141 +++- tests/Parser.test.cpp | 2 - tests/ToString.test.cpp | 2 + tests/TypeInfer.aliases.test.cpp | 10 + tests/TypeInfer.annotations.test.cpp | 10 +- tests/TypeInfer.anyerror.test.cpp | 2 - tests/TypeInfer.definitions.test.cpp | 17 + tests/TypeInfer.functions.test.cpp | 53 +- tests/TypeInfer.modules.test.cpp | 18 +- tests/TypeInfer.negations.test.cpp | 52 ++ tests/TypeInfer.operators.test.cpp | 42 +- tests/TypeInfer.provisional.test.cpp | 13 +- tests/TypeInfer.tables.test.cpp | 61 +- tests/VisitTypeVar.test.cpp | 9 +- tools/faillist.txt | 23 +- 65 files changed, 3058 insertions(+), 881 deletions(-) create mode 100644 CodeGen/src/CodeGenUtils.cpp create mode 100644 CodeGen/src/CodeGenUtils.h create mode 100644 bench/tests/voxelgen.lua create mode 100644 tests/TypeInfer.negations.test.cpp diff --git a/Analysis/include/Luau/ConstraintGraphBuilder.h b/Analysis/include/Luau/ConstraintGraphBuilder.h index dc5d4598..6106717c 100644 --- a/Analysis/include/Luau/ConstraintGraphBuilder.h +++ b/Analysis/include/Luau/ConstraintGraphBuilder.h @@ -23,6 +23,30 @@ using ScopePtr = std::shared_ptr; struct DcrLogger; +struct Inference +{ + TypeId ty = nullptr; + + Inference() = default; + + explicit Inference(TypeId ty) + : ty(ty) + { + } +}; + +struct InferencePack +{ + TypePackId tp = nullptr; + + InferencePack() = default; + + explicit InferencePack(TypePackId tp) + : tp(tp) + { + } +}; + struct ConstraintGraphBuilder { // A list of all the scopes in the module. This vector holds ownership of the @@ -130,8 +154,10 @@ struct ConstraintGraphBuilder void visit(const ScopePtr& scope, AstStatDeclareFunction* declareFunction); void visit(const ScopePtr& scope, AstStatError* error); - TypePackId checkPack(const ScopePtr& scope, AstArray exprs, const std::vector& expectedTypes = {}); - TypePackId checkPack(const ScopePtr& scope, AstExpr* expr, const std::vector& expectedTypes = {}); + InferencePack checkPack(const ScopePtr& scope, AstArray exprs, const std::vector& expectedTypes = {}); + InferencePack checkPack(const ScopePtr& scope, AstExpr* expr, const std::vector& expectedTypes = {}); + + InferencePack checkPack(const ScopePtr& scope, AstExprCall* call, const std::vector& expectedTypes); /** * Checks an expression that is expected to evaluate to one type. @@ -141,18 +167,19 @@ struct ConstraintGraphBuilder * surrounding context. Used to implement bidirectional type checking. * @return the type of the expression. */ - TypeId check(const ScopePtr& scope, AstExpr* expr, std::optional expectedType = {}); + Inference check(const ScopePtr& scope, AstExpr* expr, std::optional expectedType = {}); - TypeId check(const ScopePtr& scope, AstExprLocal* local); - TypeId check(const ScopePtr& scope, AstExprGlobal* global); - TypeId check(const ScopePtr& scope, AstExprIndexName* indexName); - TypeId check(const ScopePtr& scope, AstExprIndexExpr* indexExpr); - TypeId check(const ScopePtr& scope, AstExprUnary* unary); - TypeId check_(const ScopePtr& scope, AstExprUnary* unary); - TypeId check(const ScopePtr& scope, AstExprBinary* binary, std::optional expectedType); - TypeId check(const ScopePtr& scope, AstExprIfElse* ifElse, std::optional expectedType); - TypeId check(const ScopePtr& scope, AstExprTypeAssertion* typeAssert); - TypeId check(const ScopePtr& scope, AstExprTable* expr, std::optional expectedType); + Inference check(const ScopePtr& scope, AstExprConstantString* string, std::optional expectedType); + Inference check(const ScopePtr& scope, AstExprConstantBool* bool_, std::optional expectedType); + Inference check(const ScopePtr& scope, AstExprLocal* local); + Inference check(const ScopePtr& scope, AstExprGlobal* global); + Inference check(const ScopePtr& scope, AstExprIndexName* indexName); + Inference check(const ScopePtr& scope, AstExprIndexExpr* indexExpr); + Inference check(const ScopePtr& scope, AstExprUnary* unary); + Inference check(const ScopePtr& scope, AstExprBinary* binary, std::optional expectedType); + Inference check(const ScopePtr& scope, AstExprIfElse* ifElse, std::optional expectedType); + Inference check(const ScopePtr& scope, AstExprTypeAssertion* typeAssert); + Inference check(const ScopePtr& scope, AstExprTable* expr, std::optional expectedType); TypePackId checkLValues(const ScopePtr& scope, AstArray exprs); @@ -202,7 +229,7 @@ struct ConstraintGraphBuilder std::vector> createGenerics(const ScopePtr& scope, AstArray generics); std::vector> createGenericPacks(const ScopePtr& scope, AstArray packs); - TypeId flattenPack(const ScopePtr& scope, Location location, TypePackId tp); + Inference flattenPack(const ScopePtr& scope, Location location, InferencePack pack); void reportError(Location location, TypeErrorData err); void reportCodeTooComplex(Location location); diff --git a/Analysis/include/Luau/Error.h b/Analysis/include/Luau/Error.h index 7338627c..f7bd9d50 100644 --- a/Analysis/include/Luau/Error.h +++ b/Analysis/include/Luau/Error.h @@ -7,6 +7,8 @@ #include "Luau/Variant.h" #include "Luau/TypeArena.h" +LUAU_FASTFLAG(LuauIceExceptionInheritanceChange) + namespace Luau { struct TypeError; @@ -302,12 +304,20 @@ struct NormalizationTooComplex } }; +struct TypePackMismatch +{ + TypePackId wantedTp; + TypePackId givenTp; + + bool operator==(const TypePackMismatch& rhs) const; +}; + using TypeErrorData = Variant; + TypesAreUnrelated, NormalizationTooComplex, TypePackMismatch>; struct TypeError { @@ -374,6 +384,10 @@ struct InternalErrorReporter class InternalCompilerError : public std::exception { public: + explicit InternalCompilerError(const std::string& message) + : message(message) + { + } explicit InternalCompilerError(const std::string& message, const std::string& moduleName) : message(message) , moduleName(moduleName) @@ -388,8 +402,14 @@ public: virtual const char* what() const throw(); const std::string message; - const std::string moduleName; + const std::optional moduleName; const std::optional location; }; +// These two function overloads only exist to facilitate fast flagging a change to InternalCompilerError +// Both functions can be removed when FFlagLuauIceExceptionInheritanceChange is removed and calling code +// can directly throw InternalCompilerError. +[[noreturn]] void throwRuntimeError(const std::string& message); +[[noreturn]] void throwRuntimeError(const std::string& message, const std::string& moduleName); + } // namespace Luau diff --git a/Analysis/include/Luau/Normalize.h b/Analysis/include/Luau/Normalize.h index a23d0fda..f98442dd 100644 --- a/Analysis/include/Luau/Normalize.h +++ b/Analysis/include/Luau/Normalize.h @@ -106,9 +106,68 @@ struct std::equal_to namespace Luau { -// A normalized string type is either `string` (represented by `nullopt`) -// or a union of string singletons. -using NormalizedStringType = std::optional>; +/** A normalized string type is either `string` (represented by `nullopt`) or a + * union of string singletons. + * + * When FFlagLuauNegatedStringSingletons is unset, the representation is as + * follows: + * + * * The `string` data type is represented by the option `singletons` having the + * value `std::nullopt`. + * * The type `never` is represented by `singletons` being populated with an + * empty map. + * * A union of string singletons is represented by a map populated by the names + * and TypeIds of the singletons contained therein. + * + * When FFlagLuauNegatedStringSingletons is set, the representation is as + * follows: + * + * * A union of string singletons is finite and includes the singletons named by + * the `singletons` field. + * * An intersection of negated string singletons is cofinite and includes the + * singletons excluded by the `singletons` field. It is implied that cofinite + * values are exclusions from `string` itself. + * * The `string` data type is a cofinite set minus zero elements. + * * The `never` data type is a finite set plus zero elements. + */ +struct NormalizedStringType +{ + // When false, this type represents a union of singleton string types. + // eg "a" | "b" | "c" + // + // When true, this type represents string intersected with negated string + // singleton types. + // eg string & ~"a" & ~"b" & ... + bool isCofinite = false; + + // TODO: This field cannot be nullopt when FFlagLuauNegatedStringSingletons + // is set. When clipping that flag, we can remove the wrapping optional. + std::optional> singletons; + + void resetToString(); + void resetToNever(); + + bool isNever() const; + bool isString() const; + + /// Returns true if the string has finite domain. + /// + /// Important subtlety: This method returns true for `never`. The empty set + /// is indeed an empty set. + bool isUnion() const; + + /// Returns true if the string has infinite domain. + bool isIntersection() const; + + bool includes(const std::string& str) const; + + static const NormalizedStringType never; + + NormalizedStringType() = default; + NormalizedStringType(bool isCofinite, std::optional> singletons); +}; + +bool isSubtype(const NormalizedStringType& subStr, const NormalizedStringType& superStr); // A normalized function type is either `never` (represented by `nullopt`) // or an intersection of function types. @@ -157,7 +216,7 @@ struct NormalizedType // The string part of the type. // This may be the `string` type, or a union of singletons. - NormalizedStringType strings = std::map{}; + NormalizedStringType strings; // The thread part of the type. // This type is either never or thread. @@ -231,8 +290,14 @@ public: bool unionNormals(NormalizedType& here, const NormalizedType& there, int ignoreSmallerTyvars = -1); bool unionNormalWithTy(NormalizedType& here, TypeId there, int ignoreSmallerTyvars = -1); + // ------- Negations + NormalizedType negateNormal(const NormalizedType& here); + TypeIds negateAll(const TypeIds& theres); + TypeId negate(TypeId there); + void subtractPrimitive(NormalizedType& here, TypeId ty); + void subtractSingleton(NormalizedType& here, TypeId ty); + // ------- Normalizing intersections - void intersectTysWithTy(TypeIds& here, TypeId there); TypeId intersectionOfTops(TypeId here, TypeId there); TypeId intersectionOfBools(TypeId here, TypeId there); void intersectClasses(TypeIds& heres, const TypeIds& theres); diff --git a/Analysis/include/Luau/RecursionCounter.h b/Analysis/include/Luau/RecursionCounter.h index f964dbfe..632afd19 100644 --- a/Analysis/include/Luau/RecursionCounter.h +++ b/Analysis/include/Luau/RecursionCounter.h @@ -2,6 +2,7 @@ #pragma once #include "Luau/Common.h" +#include "Luau/Error.h" #include #include @@ -9,10 +10,20 @@ namespace Luau { -struct RecursionLimitException : public std::exception +struct RecursionLimitException : public InternalCompilerError +{ + RecursionLimitException() + : InternalCompilerError("Internal recursion counter limit exceeded") + { + LUAU_ASSERT(FFlag::LuauIceExceptionInheritanceChange); + } +}; + +struct RecursionLimitException_DEPRECATED : public std::exception { const char* what() const noexcept { + LUAU_ASSERT(!FFlag::LuauIceExceptionInheritanceChange); return "Internal recursion counter limit exceeded"; } }; @@ -42,7 +53,14 @@ struct RecursionLimiter : RecursionCounter { if (limit > 0 && *count > limit) { - throw RecursionLimitException(); + if (FFlag::LuauIceExceptionInheritanceChange) + { + throw RecursionLimitException(); + } + else + { + throw RecursionLimitException_DEPRECATED(); + } } } }; diff --git a/Analysis/include/Luau/ToString.h b/Analysis/include/Luau/ToString.h index dd2d709b..ff2561e6 100644 --- a/Analysis/include/Luau/ToString.h +++ b/Analysis/include/Luau/ToString.h @@ -117,6 +117,8 @@ inline std::string toStringNamedFunction(const std::string& funcName, const Func return toStringNamedFunction(funcName, ftv, opts); } +std::optional getFunctionNameAsString(const AstExpr& expr); + // It could be useful to see the text representation of a type during a debugging session instead of exploring the content of the class // These functions will dump the type to stdout and can be evaluated in Watch/Immediate windows or as gdb/lldb expression std::string dump(TypeId ty); diff --git a/Analysis/include/Luau/TypeInfer.h b/Analysis/include/Luau/TypeInfer.h index 384637bb..c5d7501d 100644 --- a/Analysis/include/Luau/TypeInfer.h +++ b/Analysis/include/Luau/TypeInfer.h @@ -48,7 +48,17 @@ struct HashBoolNamePair size_t operator()(const std::pair& pair) const; }; -class TimeLimitError : public std::exception +class TimeLimitError : public InternalCompilerError +{ +public: + explicit TimeLimitError(const std::string& moduleName) + : InternalCompilerError("Typeinfer failed to complete in allotted time", moduleName) + { + LUAU_ASSERT(FFlag::LuauIceExceptionInheritanceChange); + } +}; + +class TimeLimitError_DEPRECATED : public std::exception { public: virtual const char* what() const throw(); @@ -236,6 +246,7 @@ public: [[noreturn]] void ice(const std::string& message, const Location& location); [[noreturn]] void ice(const std::string& message); + [[noreturn]] void throwTimeLimitError(); ScopePtr childFunctionScope(const ScopePtr& parent, const Location& location, int subLevel = 0); ScopePtr childScope(const ScopePtr& parent, const Location& location); diff --git a/Analysis/include/Luau/Unifier.h b/Analysis/include/Luau/Unifier.h index c15cae31..7bf4d50b 100644 --- a/Analysis/include/Luau/Unifier.h +++ b/Analysis/include/Luau/Unifier.h @@ -96,6 +96,8 @@ private: void tryUnifyScalarShape(TypeId subTy, TypeId superTy, bool reversed); void tryUnifyWithMetatable(TypeId subTy, TypeId superTy, bool reversed); void tryUnifyWithClass(TypeId subTy, TypeId superTy, bool reversed); + void tryUnifyTypeWithNegation(TypeId subTy, TypeId superTy); + void tryUnifyNegationWithType(TypeId subTy, TypeId superTy); TypePackId tryApplyOverloadedFunction(TypeId function, const NormalizedFunctionType& overloads, TypePackId args); diff --git a/Analysis/src/AstQuery.cpp b/Analysis/src/AstQuery.cpp index 2904ef8d..b93c2cc2 100644 --- a/Analysis/src/AstQuery.cpp +++ b/Analysis/src/AstQuery.cpp @@ -11,6 +11,8 @@ #include +LUAU_FASTFLAGVARIABLE(LuauCheckOverloadedDocSymbol, false) + namespace Luau { @@ -430,6 +432,8 @@ ExprOrLocal findExprOrLocalAtPosition(const SourceModule& source, Position pos) static std::optional checkOverloadedDocumentationSymbol( const Module& module, const TypeId ty, const AstExpr* parentExpr, const std::optional documentationSymbol) { + LUAU_ASSERT(FFlag::LuauCheckOverloadedDocSymbol); + if (!documentationSymbol) return std::nullopt; @@ -466,7 +470,38 @@ std::optional getDocumentationSymbolAtPosition(const Source if (std::optional binding = findBindingAtPosition(module, source, position)) { - return checkOverloadedDocumentationSymbol(module, binding->typeId, parentExpr, binding->documentationSymbol); + if (FFlag::LuauCheckOverloadedDocSymbol) + { + return checkOverloadedDocumentationSymbol(module, binding->typeId, parentExpr, binding->documentationSymbol); + } + else + { + if (binding->documentationSymbol) + { + // This might be an overloaded function binding. + if (get(follow(binding->typeId))) + { + TypeId matchingOverload = nullptr; + if (parentExpr && parentExpr->is()) + { + if (auto it = module.astOverloadResolvedTypes.find(parentExpr)) + { + matchingOverload = *it; + } + } + + if (matchingOverload) + { + std::string overloadSymbol = *binding->documentationSymbol + "/overload/"; + // Default toString options are fine for this purpose. + overloadSymbol += toString(matchingOverload); + return overloadSymbol; + } + } + } + + return binding->documentationSymbol; + } } if (targetExpr) @@ -480,14 +515,20 @@ std::optional getDocumentationSymbolAtPosition(const Source { if (auto propIt = ttv->props.find(indexName->index.value); propIt != ttv->props.end()) { - return checkOverloadedDocumentationSymbol(module, propIt->second.type, parentExpr, propIt->second.documentationSymbol); + if (FFlag::LuauCheckOverloadedDocSymbol) + return checkOverloadedDocumentationSymbol(module, propIt->second.type, parentExpr, propIt->second.documentationSymbol); + else + return propIt->second.documentationSymbol; } } else if (const ClassTypeVar* ctv = get(parentTy)) { if (auto propIt = ctv->props.find(indexName->index.value); propIt != ctv->props.end()) { - return checkOverloadedDocumentationSymbol(module, propIt->second.type, parentExpr, propIt->second.documentationSymbol); + if (FFlag::LuauCheckOverloadedDocSymbol) + return checkOverloadedDocumentationSymbol(module, propIt->second.type, parentExpr, propIt->second.documentationSymbol); + else + return propIt->second.documentationSymbol; } } } diff --git a/Analysis/src/ConstraintGraphBuilder.cpp b/Analysis/src/ConstraintGraphBuilder.cpp index de2b0a4e..455fc221 100644 --- a/Analysis/src/ConstraintGraphBuilder.cpp +++ b/Analysis/src/ConstraintGraphBuilder.cpp @@ -263,7 +263,7 @@ void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatLocal* local) if (hasAnnotation) expectedTypes.insert(begin(expectedTypes), begin(varTypes) + i, end(varTypes)); - TypePackId exprPack = checkPack(scope, value, expectedTypes); + TypePackId exprPack = checkPack(scope, value, expectedTypes).tp; if (i < local->vars.size) { @@ -292,7 +292,7 @@ void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatLocal* local) if (hasAnnotation) expectedType = varTypes.at(i); - TypeId exprType = check(scope, value, expectedType); + TypeId exprType = check(scope, value, expectedType).ty; if (i < varTypes.size()) { if (varTypes[i]) @@ -350,7 +350,7 @@ void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatFor* for_) if (!expr) return; - TypeId t = check(scope, expr); + TypeId t = check(scope, expr).ty; addConstraint(scope, expr->location, SubtypeConstraint{t, singletonTypes->numberType}); }; @@ -368,7 +368,7 @@ void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatForIn* forIn) { ScopePtr loopScope = childScope(forIn, scope); - TypePackId iterator = checkPack(scope, forIn->values); + TypePackId iterator = checkPack(scope, forIn->values).tp; std::vector variableTypes; variableTypes.reserve(forIn->vars.size); @@ -489,7 +489,7 @@ void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatFunction* funct } else if (AstExprIndexName* indexName = function->name->as()) { - TypeId containingTableType = check(scope, indexName->expr); + TypeId containingTableType = check(scope, indexName->expr).ty; functionType = arena->addType(BlockedTypeVar{}); @@ -531,7 +531,7 @@ void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatReturn* ret) for (TypeId ty : scope->returnType) expectedTypes.push_back(ty); - TypePackId exprTypes = checkPack(scope, ret->list, expectedTypes); + TypePackId exprTypes = checkPack(scope, ret->list, expectedTypes).tp; addConstraint(scope, ret->location, PackSubtypeConstraint{exprTypes, scope->returnType}); } @@ -545,7 +545,7 @@ void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatBlock* block) void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatAssign* assign) { TypePackId varPackId = checkLValues(scope, assign->vars); - TypePackId valuePack = checkPack(scope, assign->values); + TypePackId valuePack = checkPack(scope, assign->values).tp; addConstraint(scope, assign->location, PackSubtypeConstraint{valuePack, varPackId}); } @@ -732,7 +732,6 @@ void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatDeclareClass* d void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatDeclareFunction* global) { - std::vector> generics = createGenerics(scope, global->generics); std::vector> genericPacks = createGenericPacks(scope, global->genericPacks); @@ -779,7 +778,7 @@ void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatError* error) check(scope, expr); } -TypePackId ConstraintGraphBuilder::checkPack(const ScopePtr& scope, AstArray exprs, const std::vector& expectedTypes) +InferencePack ConstraintGraphBuilder::checkPack(const ScopePtr& scope, AstArray exprs, const std::vector& expectedTypes) { std::vector head; std::optional tail; @@ -792,201 +791,180 @@ TypePackId ConstraintGraphBuilder::checkPack(const ScopePtr& scope, AstArray expectedType; if (i < expectedTypes.size()) expectedType = expectedTypes[i]; - head.push_back(check(scope, expr)); + head.push_back(check(scope, expr).ty); } else { std::vector expectedTailTypes; if (i < expectedTypes.size()) expectedTailTypes.assign(begin(expectedTypes) + i, end(expectedTypes)); - tail = checkPack(scope, expr, expectedTailTypes); + tail = checkPack(scope, expr, expectedTailTypes).tp; } } if (head.empty() && tail) - return *tail; + return InferencePack{*tail}; else - return arena->addTypePack(TypePack{std::move(head), tail}); + return InferencePack{arena->addTypePack(TypePack{std::move(head), tail})}; } -TypePackId ConstraintGraphBuilder::checkPack(const ScopePtr& scope, AstExpr* expr, const std::vector& expectedTypes) +InferencePack ConstraintGraphBuilder::checkPack(const ScopePtr& scope, AstExpr* expr, const std::vector& expectedTypes) { RecursionCounter counter{&recursionCount}; if (recursionCount >= FInt::LuauCheckRecursionLimit) { reportCodeTooComplex(expr->location); - return singletonTypes->errorRecoveryTypePack(); + return InferencePack{singletonTypes->errorRecoveryTypePack()}; } - TypePackId result = nullptr; + InferencePack result; if (AstExprCall* call = expr->as()) - { - TypeId fnType = check(scope, call->func); - const size_t constraintIndex = scope->constraints.size(); - const size_t scopeIndex = scopes.size(); - - std::vector args; - - for (AstExpr* arg : call->args) - { - args.push_back(check(scope, arg)); - } - - // TODO self - - if (matchSetmetatable(*call)) - { - LUAU_ASSERT(args.size() == 2); - TypeId target = args[0]; - TypeId mt = args[1]; - - MetatableTypeVar mtv{target, mt}; - TypeId resultTy = arena->addType(mtv); - result = arena->addTypePack({resultTy}); - } - else - { - const size_t constraintEndIndex = scope->constraints.size(); - const size_t scopeEndIndex = scopes.size(); - - astOriginalCallTypes[call->func] = fnType; - - TypeId instantiatedType = arena->addType(BlockedTypeVar{}); - // TODO: How do expectedTypes play into this? Do they? - TypePackId rets = arena->addTypePack(BlockedTypePack{}); - TypePackId argPack = arena->addTypePack(TypePack{args, {}}); - FunctionTypeVar ftv(TypeLevel{}, scope.get(), argPack, rets); - TypeId inferredFnType = arena->addType(ftv); - - scope->unqueuedConstraints.push_back( - std::make_unique(NotNull{scope.get()}, call->func->location, InstantiationConstraint{instantiatedType, fnType})); - NotNull ic(scope->unqueuedConstraints.back().get()); - - scope->unqueuedConstraints.push_back( - std::make_unique(NotNull{scope.get()}, call->func->location, SubtypeConstraint{inferredFnType, instantiatedType})); - NotNull sc(scope->unqueuedConstraints.back().get()); - - // We force constraints produced by checking function arguments to wait - // until after we have resolved the constraint on the function itself. - // This ensures, for instance, that we start inferring the contents of - // lambdas under the assumption that their arguments and return types - // will be compatible with the enclosing function call. - for (size_t ci = constraintIndex; ci < constraintEndIndex; ++ci) - scope->constraints[ci]->dependencies.push_back(sc); - - for (size_t si = scopeIndex; si < scopeEndIndex; ++si) - { - for (auto& c : scopes[si].second->constraints) - { - c->dependencies.push_back(sc); - } - } - - addConstraint(scope, call->func->location, - FunctionCallConstraint{ - {ic, sc}, - fnType, - argPack, - rets, - call, - }); - - result = rets; - } - } + result = {checkPack(scope, call, expectedTypes)}; else if (AstExprVarargs* varargs = expr->as()) { if (scope->varargPack) - result = *scope->varargPack; + result = InferencePack{*scope->varargPack}; else - result = singletonTypes->errorRecoveryTypePack(); + result = InferencePack{singletonTypes->errorRecoveryTypePack()}; } else { std::optional expectedType; if (!expectedTypes.empty()) expectedType = expectedTypes[0]; - TypeId t = check(scope, expr, expectedType); - result = arena->addTypePack({t}); + TypeId t = check(scope, expr, expectedType).ty; + result = InferencePack{arena->addTypePack({t})}; } - LUAU_ASSERT(result); - astTypePacks[expr] = result; + LUAU_ASSERT(result.tp); + astTypePacks[expr] = result.tp; return result; } -TypeId ConstraintGraphBuilder::check(const ScopePtr& scope, AstExpr* expr, std::optional expectedType) +InferencePack ConstraintGraphBuilder::checkPack(const ScopePtr& scope, AstExprCall* call, const std::vector& expectedTypes) +{ + TypeId fnType = check(scope, call->func).ty; + const size_t constraintIndex = scope->constraints.size(); + const size_t scopeIndex = scopes.size(); + + std::vector args; + + for (AstExpr* arg : call->args) + { + args.push_back(check(scope, arg).ty); + } + + // TODO self + + if (matchSetmetatable(*call)) + { + LUAU_ASSERT(args.size() == 2); + TypeId target = args[0]; + TypeId mt = args[1]; + + AstExpr* targetExpr = call->args.data[0]; + + MetatableTypeVar mtv{target, mt}; + TypeId resultTy = arena->addType(mtv); + + if (AstExprLocal* targetLocal = targetExpr->as()) + scope->bindings[targetLocal->local].typeId = resultTy; + + return InferencePack{arena->addTypePack({resultTy})}; + } + else + { + const size_t constraintEndIndex = scope->constraints.size(); + const size_t scopeEndIndex = scopes.size(); + + astOriginalCallTypes[call->func] = fnType; + + TypeId instantiatedType = arena->addType(BlockedTypeVar{}); + // TODO: How do expectedTypes play into this? Do they? + TypePackId rets = arena->addTypePack(BlockedTypePack{}); + TypePackId argPack = arena->addTypePack(TypePack{args, {}}); + FunctionTypeVar ftv(TypeLevel{}, scope.get(), argPack, rets); + TypeId inferredFnType = arena->addType(ftv); + + scope->unqueuedConstraints.push_back( + std::make_unique(NotNull{scope.get()}, call->func->location, InstantiationConstraint{instantiatedType, fnType})); + NotNull ic(scope->unqueuedConstraints.back().get()); + + scope->unqueuedConstraints.push_back( + std::make_unique(NotNull{scope.get()}, call->func->location, SubtypeConstraint{inferredFnType, instantiatedType})); + NotNull sc(scope->unqueuedConstraints.back().get()); + + // We force constraints produced by checking function arguments to wait + // until after we have resolved the constraint on the function itself. + // This ensures, for instance, that we start inferring the contents of + // lambdas under the assumption that their arguments and return types + // will be compatible with the enclosing function call. + for (size_t ci = constraintIndex; ci < constraintEndIndex; ++ci) + scope->constraints[ci]->dependencies.push_back(sc); + + for (size_t si = scopeIndex; si < scopeEndIndex; ++si) + { + for (auto& c : scopes[si].second->constraints) + { + c->dependencies.push_back(sc); + } + } + + addConstraint(scope, call->func->location, + FunctionCallConstraint{ + {ic, sc}, + fnType, + argPack, + rets, + call, + }); + + return InferencePack{rets}; + } +} + +Inference ConstraintGraphBuilder::check(const ScopePtr& scope, AstExpr* expr, std::optional expectedType) { RecursionCounter counter{&recursionCount}; if (recursionCount >= FInt::LuauCheckRecursionLimit) { reportCodeTooComplex(expr->location); - return singletonTypes->errorRecoveryType(); + return Inference{singletonTypes->errorRecoveryType()}; } - TypeId result = nullptr; + Inference result; if (auto group = expr->as()) result = check(scope, group->expr, expectedType); else if (auto stringExpr = expr->as()) - { - if (expectedType) - { - const TypeId expectedTy = follow(*expectedType); - if (get(expectedTy) || get(expectedTy)) - { - result = arena->addType(BlockedTypeVar{}); - TypeId singletonType = arena->addType(SingletonTypeVar(StringSingleton{std::string(stringExpr->value.data, stringExpr->value.size)})); - addConstraint(scope, expr->location, PrimitiveTypeConstraint{result, expectedTy, singletonType, singletonTypes->stringType}); - } - else if (maybeSingleton(expectedTy)) - result = arena->addType(SingletonTypeVar{StringSingleton{std::string{stringExpr->value.data, stringExpr->value.size}}}); - else - result = singletonTypes->stringType; - } - else - result = singletonTypes->stringType; - } + result = check(scope, stringExpr, expectedType); else if (expr->is()) - result = singletonTypes->numberType; + result = Inference{singletonTypes->numberType}; else if (auto boolExpr = expr->as()) - { - if (expectedType) - { - const TypeId expectedTy = follow(*expectedType); - const TypeId singletonType = boolExpr->value ? singletonTypes->trueType : singletonTypes->falseType; - - if (get(expectedTy) || get(expectedTy)) - { - result = arena->addType(BlockedTypeVar{}); - addConstraint(scope, expr->location, PrimitiveTypeConstraint{result, expectedTy, singletonType, singletonTypes->booleanType}); - } - else if (maybeSingleton(expectedTy)) - result = singletonType; - else - result = singletonTypes->booleanType; - } - else - result = singletonTypes->booleanType; - } + result = check(scope, boolExpr, expectedType); else if (expr->is()) - result = singletonTypes->nilType; + result = Inference{singletonTypes->nilType}; else if (auto local = expr->as()) result = check(scope, local); else if (auto global = expr->as()) result = check(scope, global); else if (expr->is()) result = flattenPack(scope, expr->location, checkPack(scope, expr)); - else if (expr->is()) - result = flattenPack(scope, expr->location, checkPack(scope, expr)); // TODO: needs predicates too + else if (auto call = expr->as()) + { + std::vector expectedTypes; + if (expectedType) + expectedTypes.push_back(*expectedType); + result = flattenPack(scope, expr->location, checkPack(scope, call, expectedTypes)); // TODO: needs predicates too + } else if (auto a = expr->as()) { FunctionSignature sig = checkFunctionSignature(scope, a); checkFunctionBody(sig.bodyScope, a); - return sig.signature; + return Inference{sig.signature}; } else if (auto indexName = expr->as()) result = check(scope, indexName); @@ -1008,20 +986,63 @@ TypeId ConstraintGraphBuilder::check(const ScopePtr& scope, AstExpr* expr, std:: for (AstExpr* subExpr : err->expressions) check(scope, subExpr); - result = singletonTypes->errorRecoveryType(); + result = Inference{singletonTypes->errorRecoveryType()}; } else { LUAU_ASSERT(0); - result = freshType(scope); + result = Inference{freshType(scope)}; } - LUAU_ASSERT(result); - astTypes[expr] = result; + LUAU_ASSERT(result.ty); + astTypes[expr] = result.ty; return result; } -TypeId ConstraintGraphBuilder::check(const ScopePtr& scope, AstExprLocal* local) +Inference ConstraintGraphBuilder::check(const ScopePtr& scope, AstExprConstantString* string, std::optional expectedType) +{ + if (expectedType) + { + const TypeId expectedTy = follow(*expectedType); + if (get(expectedTy) || get(expectedTy)) + { + TypeId ty = arena->addType(BlockedTypeVar{}); + TypeId singletonType = arena->addType(SingletonTypeVar(StringSingleton{std::string(string->value.data, string->value.size)})); + addConstraint(scope, string->location, PrimitiveTypeConstraint{ty, expectedTy, singletonType, singletonTypes->stringType}); + return Inference{ty}; + } + else if (maybeSingleton(expectedTy)) + return Inference{arena->addType(SingletonTypeVar{StringSingleton{std::string{string->value.data, string->value.size}}})}; + + return Inference{singletonTypes->stringType}; + } + + return Inference{singletonTypes->stringType}; +} + +Inference ConstraintGraphBuilder::check(const ScopePtr& scope, AstExprConstantBool* boolExpr, std::optional expectedType) +{ + if (expectedType) + { + const TypeId expectedTy = follow(*expectedType); + const TypeId singletonType = boolExpr->value ? singletonTypes->trueType : singletonTypes->falseType; + + if (get(expectedTy) || get(expectedTy)) + { + TypeId ty = arena->addType(BlockedTypeVar{}); + addConstraint(scope, boolExpr->location, PrimitiveTypeConstraint{ty, expectedTy, singletonType, singletonTypes->booleanType}); + return Inference{ty}; + } + else if (maybeSingleton(expectedTy)) + return Inference{singletonType}; + + return Inference{singletonTypes->booleanType}; + } + + return Inference{singletonTypes->booleanType}; +} + +Inference ConstraintGraphBuilder::check(const ScopePtr& scope, AstExprLocal* local) { std::optional resultTy; @@ -1035,26 +1056,26 @@ TypeId ConstraintGraphBuilder::check(const ScopePtr& scope, AstExprLocal* local) } if (!resultTy) - return singletonTypes->errorRecoveryType(); // TODO: replace with ice, locals should never exist before its definition. + return Inference{singletonTypes->errorRecoveryType()}; // TODO: replace with ice, locals should never exist before its definition. - return *resultTy; + return Inference{*resultTy}; } -TypeId ConstraintGraphBuilder::check(const ScopePtr& scope, AstExprGlobal* global) +Inference ConstraintGraphBuilder::check(const ScopePtr& scope, AstExprGlobal* global) { if (std::optional ty = scope->lookup(global->name)) - return *ty; + return Inference{*ty}; /* prepopulateGlobalScope() has already added all global functions to the environment by this point, so any * global that is not already in-scope is definitely an unknown symbol. */ reportError(global->location, UnknownSymbol{global->name.value}); - return singletonTypes->errorRecoveryType(); + return Inference{singletonTypes->errorRecoveryType()}; } -TypeId ConstraintGraphBuilder::check(const ScopePtr& scope, AstExprIndexName* indexName) +Inference ConstraintGraphBuilder::check(const ScopePtr& scope, AstExprIndexName* indexName) { - TypeId obj = check(scope, indexName->expr); + TypeId obj = check(scope, indexName->expr).ty; TypeId result = freshType(scope); TableTypeVar::Props props{{indexName->index.value, Property{result}}}; @@ -1065,13 +1086,13 @@ TypeId ConstraintGraphBuilder::check(const ScopePtr& scope, AstExprIndexName* in addConstraint(scope, indexName->expr->location, SubtypeConstraint{obj, expectedTableType}); - return result; + return Inference{result}; } -TypeId ConstraintGraphBuilder::check(const ScopePtr& scope, AstExprIndexExpr* indexExpr) +Inference ConstraintGraphBuilder::check(const ScopePtr& scope, AstExprIndexExpr* indexExpr) { - TypeId obj = check(scope, indexExpr->expr); - TypeId indexType = check(scope, indexExpr->index); + TypeId obj = check(scope, indexExpr->expr).ty; + TypeId indexType = check(scope, indexExpr->index).ty; TypeId result = freshType(scope); @@ -1081,61 +1102,49 @@ TypeId ConstraintGraphBuilder::check(const ScopePtr& scope, AstExprIndexExpr* in addConstraint(scope, indexExpr->expr->location, SubtypeConstraint{obj, tableType}); - return result; + return Inference{result}; } -TypeId ConstraintGraphBuilder::check(const ScopePtr& scope, AstExprUnary* unary) +Inference ConstraintGraphBuilder::check(const ScopePtr& scope, AstExprUnary* unary) { - TypeId operandType = check_(scope, unary); + TypeId operandType = check(scope, unary->expr).ty; TypeId resultType = arena->addType(BlockedTypeVar{}); addConstraint(scope, unary->location, UnaryConstraint{unary->op, operandType, resultType}); - return resultType; + return Inference{resultType}; } -TypeId ConstraintGraphBuilder::check_(const ScopePtr& scope, AstExprUnary* unary) +Inference ConstraintGraphBuilder::check(const ScopePtr& scope, AstExprBinary* binary, std::optional expectedType) { - if (unary->op == AstExprUnary::Not) - { - TypeId ty = check(scope, unary->expr, std::nullopt); - - return ty; - } - - return check(scope, unary->expr); -} - -TypeId ConstraintGraphBuilder::check(const ScopePtr& scope, AstExprBinary* binary, std::optional expectedType) -{ - TypeId leftType = check(scope, binary->left, expectedType); - TypeId rightType = check(scope, binary->right, expectedType); + TypeId leftType = check(scope, binary->left, expectedType).ty; + TypeId rightType = check(scope, binary->right, expectedType).ty; TypeId resultType = arena->addType(BlockedTypeVar{}); addConstraint(scope, binary->location, BinaryConstraint{binary->op, leftType, rightType, resultType}); - return resultType; + return Inference{resultType}; } -TypeId ConstraintGraphBuilder::check(const ScopePtr& scope, AstExprIfElse* ifElse, std::optional expectedType) +Inference ConstraintGraphBuilder::check(const ScopePtr& scope, AstExprIfElse* ifElse, std::optional expectedType) { check(scope, ifElse->condition); - TypeId thenType = check(scope, ifElse->trueExpr, expectedType); - TypeId elseType = check(scope, ifElse->falseExpr, expectedType); + TypeId thenType = check(scope, ifElse->trueExpr, expectedType).ty; + TypeId elseType = check(scope, ifElse->falseExpr, expectedType).ty; if (ifElse->hasElse) { TypeId resultType = expectedType ? *expectedType : freshType(scope); addConstraint(scope, ifElse->trueExpr->location, SubtypeConstraint{thenType, resultType}); addConstraint(scope, ifElse->falseExpr->location, SubtypeConstraint{elseType, resultType}); - return resultType; + return Inference{resultType}; } - return thenType; + return Inference{thenType}; } -TypeId ConstraintGraphBuilder::check(const ScopePtr& scope, AstExprTypeAssertion* typeAssert) +Inference ConstraintGraphBuilder::check(const ScopePtr& scope, AstExprTypeAssertion* typeAssert) { check(scope, typeAssert->expr, std::nullopt); - return resolveType(scope, typeAssert->annotation); + return Inference{resolveType(scope, typeAssert->annotation)}; } TypePackId ConstraintGraphBuilder::checkLValues(const ScopePtr& scope, AstArray exprs) @@ -1286,22 +1295,22 @@ TypeId ConstraintGraphBuilder::checkLValue(const ScopePtr& scope, AstExpr* expr) auto dottedPath = extractDottedName(expr); if (!dottedPath) - return check(scope, expr); + return check(scope, expr).ty; const auto [sym, segments] = std::move(*dottedPath); if (!sym.local) - return check(scope, expr); + return check(scope, expr).ty; auto lookupResult = scope->lookupEx(sym); if (!lookupResult) - return check(scope, expr); + return check(scope, expr).ty; const auto [ty, symbolScope] = std::move(*lookupResult); TypeId replaceTy = arena->freshType(scope.get()); std::optional updatedType = updateTheTableType(arena, ty, segments, replaceTy); if (!updatedType) - return check(scope, expr); + return check(scope, expr).ty; std::optional def = dfg->getDef(sym); LUAU_ASSERT(def); @@ -1310,7 +1319,7 @@ TypeId ConstraintGraphBuilder::checkLValue(const ScopePtr& scope, AstExpr* expr) return replaceTy; } -TypeId ConstraintGraphBuilder::check(const ScopePtr& scope, AstExprTable* expr, std::optional expectedType) +Inference ConstraintGraphBuilder::check(const ScopePtr& scope, AstExprTable* expr, std::optional expectedType) { TypeId ty = arena->addType(TableTypeVar{}); TableTypeVar* ttv = getMutable(ty); @@ -1344,16 +1353,14 @@ TypeId ConstraintGraphBuilder::check(const ScopePtr& scope, AstExprTable* expr, } } - TypeId itemTy = check(scope, item.value, expectedValueType); - if (get(follow(itemTy))) - return ty; + TypeId itemTy = check(scope, item.value, expectedValueType).ty; if (item.key) { // Even though we don't need to use the type of the item's key if // it's a string constant, we still want to check it to populate // astTypes. - TypeId keyTy = check(scope, item.key); + TypeId keyTy = check(scope, item.key).ty; if (AstExprConstantString* key = item.key->as()) { @@ -1373,7 +1380,7 @@ TypeId ConstraintGraphBuilder::check(const ScopePtr& scope, AstExprTable* expr, } } - return ty; + return Inference{ty}; } ConstraintGraphBuilder::FunctionSignature ConstraintGraphBuilder::checkFunctionSignature(const ScopePtr& parent, AstExprFunction* fn) @@ -1541,9 +1548,18 @@ TypeId ConstraintGraphBuilder::resolveType(const ScopePtr& scope, AstType* ty, b } } - std::optional alias = scope->lookupType(ref->name.value); + std::optional alias; - if (alias.has_value() || ref->prefix.has_value()) + if (ref->prefix.has_value()) + { + alias = scope->lookupImportedType(ref->prefix->value, ref->name.value); + } + else + { + alias = scope->lookupType(ref->name.value); + } + + if (alias.has_value()) { // If the alias is not generic, we don't need to set up a blocked // type and an instantiation constraint. @@ -1586,7 +1602,11 @@ TypeId ConstraintGraphBuilder::resolveType(const ScopePtr& scope, AstType* ty, b } else { - reportError(ty->location, UnknownSymbol{ref->name.value, UnknownSymbol::Context::Type}); + std::string typeName; + if (ref->prefix) + typeName = std::string(ref->prefix->value) + "."; + typeName += ref->name.value; + result = singletonTypes->errorRecoveryType(); } } @@ -1685,7 +1705,7 @@ TypeId ConstraintGraphBuilder::resolveType(const ScopePtr& scope, AstType* ty, b else if (auto tof = ty->as()) { // TODO: Recursion limit. - TypeId exprType = check(scope, tof->expr); + TypeId exprType = check(scope, tof->expr).ty; result = exprType; } else if (auto unionAnnotation = ty->as()) @@ -1694,7 +1714,7 @@ TypeId ConstraintGraphBuilder::resolveType(const ScopePtr& scope, AstType* ty, b for (AstType* part : unionAnnotation->types) { // TODO: Recursion limit. - parts.push_back(resolveType(scope, part)); + parts.push_back(resolveType(scope, part, topLevel)); } result = arena->addType(UnionTypeVar{parts}); @@ -1705,7 +1725,7 @@ TypeId ConstraintGraphBuilder::resolveType(const ScopePtr& scope, AstType* ty, b for (AstType* part : intersectionAnnotation->types) { // TODO: Recursion limit. - parts.push_back(resolveType(scope, part)); + parts.push_back(resolveType(scope, part, topLevel)); } result = arena->addType(IntersectionTypeVar{parts}); @@ -1795,10 +1815,7 @@ std::vector> ConstraintGraphBuilder::crea if (generic.defaultValue) defaultTy = resolveType(scope, generic.defaultValue); - result.push_back({generic.name.value, GenericTypeDefinition{ - genericTy, - defaultTy, - }}); + result.push_back({generic.name.value, GenericTypeDefinition{genericTy, defaultTy}}); } return result; @@ -1816,19 +1833,17 @@ std::vector> ConstraintGraphBuilder:: if (generic.defaultValue) defaultTy = resolveTypePack(scope, generic.defaultValue); - result.push_back({generic.name.value, GenericTypePackDefinition{ - genericTy, - defaultTy, - }}); + result.push_back({generic.name.value, GenericTypePackDefinition{genericTy, defaultTy}}); } return result; } -TypeId ConstraintGraphBuilder::flattenPack(const ScopePtr& scope, Location location, TypePackId tp) +Inference ConstraintGraphBuilder::flattenPack(const ScopePtr& scope, Location location, InferencePack pack) { + auto [tp] = pack; if (auto f = first(tp)) - return *f; + return Inference{*f}; TypeId typeResult = freshType(scope); TypePack onePack{{typeResult}, freshTypePack(scope)}; @@ -1836,7 +1851,7 @@ TypeId ConstraintGraphBuilder::flattenPack(const ScopePtr& scope, Location locat addConstraint(scope, location, PackSubtypeConstraint{tp, oneTypePack}); - return typeResult; + return Inference{typeResult}; } void ConstraintGraphBuilder::reportError(Location location, TypeErrorData err) diff --git a/Analysis/src/ConstraintSolver.cpp b/Analysis/src/ConstraintSolver.cpp index 60f4666a..5e43be0f 100644 --- a/Analysis/src/ConstraintSolver.cpp +++ b/Analysis/src/ConstraintSolver.cpp @@ -544,6 +544,7 @@ bool ConstraintSolver::tryDispatch(const UnaryConstraint& c, NotNullty.emplace(singletonTypes->numberType); return true; } @@ -552,13 +553,46 @@ bool ConstraintSolver::tryDispatch(const UnaryConstraint& c, NotNull(operandType) || get(operandType)) { asMutable(c.resultType)->ty.emplace(c.operandType); - return true; } - break; + else if (std::optional mm = findMetatableEntry(singletonTypes, errors, operandType, "__unm", constraint->location)) + { + const FunctionTypeVar* ftv = get(follow(*mm)); + + if (!ftv) + { + if (std::optional callMm = findMetatableEntry(singletonTypes, errors, follow(*mm), "__call", constraint->location)) + { + ftv = get(follow(*callMm)); + } + } + + if (!ftv) + { + asMutable(c.resultType)->ty.emplace(singletonTypes->errorRecoveryType()); + return true; + } + + TypePackId argsPack = arena->addTypePack({operandType}); + unify(ftv->argTypes, argsPack, constraint->scope); + + TypeId result = singletonTypes->errorRecoveryType(); + if (ftv) + { + result = first(ftv->retTypes).value_or(singletonTypes->errorRecoveryType()); + } + + asMutable(c.resultType)->ty.emplace(result); + } + else + { + asMutable(c.resultType)->ty.emplace(singletonTypes->errorRecoveryType()); + } + + return true; } } - LUAU_ASSERT(false); // TODO metatable handling + LUAU_ASSERT(false); return false; } @@ -862,6 +896,10 @@ bool ConstraintSolver::tryDispatch(const NameConstraint& c, NotNullname = c.name; else if (MetatableTypeVar* mtv = getMutable(target)) mtv->syntheticName = c.name; + else if (get(target) || get(target)) + { + // nothing (yet) + } else return block(c.namedType, constraint); diff --git a/Analysis/src/Error.cpp b/Analysis/src/Error.cpp index e5553003..ed1a49cd 100644 --- a/Analysis/src/Error.cpp +++ b/Analysis/src/Error.cpp @@ -7,6 +7,8 @@ #include +LUAU_FASTFLAGVARIABLE(LuauIceExceptionInheritanceChange, false) + static std::string wrongNumberOfArgsString( size_t expectedCount, std::optional maximumCount, size_t actualCount, const char* argPrefix = nullptr, bool isVariadic = false) { @@ -460,6 +462,11 @@ struct ErrorConverter { return "Code is too complex to typecheck! Consider simplifying the code around this area"; } + + std::string operator()(const TypePackMismatch& e) const + { + return "Type pack '" + toString(e.givenTp) + "' could not be converted into '" + toString(e.wantedTp) + "'"; + } }; struct InvalidNameChecker @@ -718,6 +725,11 @@ bool TypesAreUnrelated::operator==(const TypesAreUnrelated& rhs) const return left == rhs.left && right == rhs.right; } +bool TypePackMismatch::operator==(const TypePackMismatch& rhs) const +{ + return *wantedTp == *rhs.wantedTp && *givenTp == *rhs.givenTp; +} + std::string toString(const TypeError& error) { return toString(error, TypeErrorToStringOptions{}); @@ -869,6 +881,11 @@ void copyError(T& e, TypeArena& destArena, CloneState cloneState) else if constexpr (std::is_same_v) { } + else if constexpr (std::is_same_v) + { + e.wantedTp = clone(e.wantedTp); + e.givenTp = clone(e.givenTp); + } else static_assert(always_false_v, "Non-exhaustive type switch"); } @@ -913,4 +930,30 @@ const char* InternalCompilerError::what() const throw() return this->message.data(); } +// TODO: Inline me when LuauIceExceptionInheritanceChange is deleted. +void throwRuntimeError(const std::string& message) +{ + if (FFlag::LuauIceExceptionInheritanceChange) + { + throw InternalCompilerError(message); + } + else + { + throw std::runtime_error(message); + } +} + +// TODO: Inline me when LuauIceExceptionInheritanceChange is deleted. +void throwRuntimeError(const std::string& message, const std::string& moduleName) +{ + if (FFlag::LuauIceExceptionInheritanceChange) + { + throw InternalCompilerError(message, moduleName); + } + else + { + throw std::runtime_error(message); + } +} + } // namespace Luau diff --git a/Analysis/src/Frontend.cpp b/Analysis/src/Frontend.cpp index e059a463..39e6428d 100644 --- a/Analysis/src/Frontend.cpp +++ b/Analysis/src/Frontend.cpp @@ -31,6 +31,7 @@ LUAU_FASTINTVARIABLE(LuauAutocompleteCheckTimeoutMs, 100) LUAU_FASTFLAGVARIABLE(DebugLuauDeferredConstraintResolution, false) LUAU_FASTFLAG(DebugLuauLogSolverToJson); LUAU_FASTFLAGVARIABLE(LuauFixMarkDirtyReverseDeps, false) +LUAU_FASTFLAGVARIABLE(LuauPersistTypesAfterGeneratingDocSyms, false) namespace Luau { @@ -111,24 +112,57 @@ LoadDefinitionFileResult Frontend::loadDefinitionFile(std::string_view source, c CloneState cloneState; - for (const auto& [name, ty] : checkedModule->declaredGlobals) + if (FFlag::LuauPersistTypesAfterGeneratingDocSyms) { - TypeId globalTy = clone(ty, globalTypes, cloneState); - std::string documentationSymbol = packageName + "/global/" + name; - generateDocumentationSymbols(globalTy, documentationSymbol); - globalScope->bindings[typeChecker.globalNames.names->getOrAdd(name.c_str())] = {globalTy, Location(), false, {}, documentationSymbol}; + std::vector typesToPersist; + typesToPersist.reserve(checkedModule->declaredGlobals.size() + checkedModule->getModuleScope()->exportedTypeBindings.size()); - persist(globalTy); + for (const auto& [name, ty] : checkedModule->declaredGlobals) + { + TypeId globalTy = clone(ty, globalTypes, cloneState); + std::string documentationSymbol = packageName + "/global/" + name; + generateDocumentationSymbols(globalTy, documentationSymbol); + globalScope->bindings[typeChecker.globalNames.names->getOrAdd(name.c_str())] = {globalTy, Location(), false, {}, documentationSymbol}; + + typesToPersist.push_back(globalTy); + } + + for (const auto& [name, ty] : checkedModule->getModuleScope()->exportedTypeBindings) + { + TypeFun globalTy = clone(ty, globalTypes, cloneState); + std::string documentationSymbol = packageName + "/globaltype/" + name; + generateDocumentationSymbols(globalTy.type, documentationSymbol); + globalScope->exportedTypeBindings[name] = globalTy; + + typesToPersist.push_back(globalTy.type); + } + + for (TypeId ty : typesToPersist) + { + persist(ty); + } } - - for (const auto& [name, ty] : checkedModule->getModuleScope()->exportedTypeBindings) + else { - TypeFun globalTy = clone(ty, globalTypes, cloneState); - std::string documentationSymbol = packageName + "/globaltype/" + name; - generateDocumentationSymbols(globalTy.type, documentationSymbol); - globalScope->exportedTypeBindings[name] = globalTy; + for (const auto& [name, ty] : checkedModule->declaredGlobals) + { + TypeId globalTy = clone(ty, globalTypes, cloneState); + std::string documentationSymbol = packageName + "/global/" + name; + generateDocumentationSymbols(globalTy, documentationSymbol); + globalScope->bindings[typeChecker.globalNames.names->getOrAdd(name.c_str())] = {globalTy, Location(), false, {}, documentationSymbol}; - persist(globalTy.type); + persist(globalTy); + } + + for (const auto& [name, ty] : checkedModule->getModuleScope()->exportedTypeBindings) + { + TypeFun globalTy = clone(ty, globalTypes, cloneState); + std::string documentationSymbol = packageName + "/globaltype/" + name; + generateDocumentationSymbols(globalTy.type, documentationSymbol); + globalScope->exportedTypeBindings[name] = globalTy; + + persist(globalTy.type); + } } return LoadDefinitionFileResult{true, parseResult, checkedModule}; @@ -160,24 +194,57 @@ LoadDefinitionFileResult loadDefinitionFile(TypeChecker& typeChecker, ScopePtr t CloneState cloneState; - for (const auto& [name, ty] : checkedModule->declaredGlobals) + if (FFlag::LuauPersistTypesAfterGeneratingDocSyms) { - TypeId globalTy = clone(ty, typeChecker.globalTypes, cloneState); - std::string documentationSymbol = packageName + "/global/" + name; - generateDocumentationSymbols(globalTy, documentationSymbol); - targetScope->bindings[typeChecker.globalNames.names->getOrAdd(name.c_str())] = {globalTy, Location(), false, {}, documentationSymbol}; + std::vector typesToPersist; + typesToPersist.reserve(checkedModule->declaredGlobals.size() + checkedModule->getModuleScope()->exportedTypeBindings.size()); - persist(globalTy); + for (const auto& [name, ty] : checkedModule->declaredGlobals) + { + TypeId globalTy = clone(ty, typeChecker.globalTypes, cloneState); + std::string documentationSymbol = packageName + "/global/" + name; + generateDocumentationSymbols(globalTy, documentationSymbol); + targetScope->bindings[typeChecker.globalNames.names->getOrAdd(name.c_str())] = {globalTy, Location(), false, {}, documentationSymbol}; + + typesToPersist.push_back(globalTy); + } + + for (const auto& [name, ty] : checkedModule->getModuleScope()->exportedTypeBindings) + { + TypeFun globalTy = clone(ty, typeChecker.globalTypes, cloneState); + std::string documentationSymbol = packageName + "/globaltype/" + name; + generateDocumentationSymbols(globalTy.type, documentationSymbol); + targetScope->exportedTypeBindings[name] = globalTy; + + typesToPersist.push_back(globalTy.type); + } + + for (TypeId ty : typesToPersist) + { + persist(ty); + } } - - for (const auto& [name, ty] : checkedModule->getModuleScope()->exportedTypeBindings) + else { - TypeFun globalTy = clone(ty, typeChecker.globalTypes, cloneState); - std::string documentationSymbol = packageName + "/globaltype/" + name; - generateDocumentationSymbols(globalTy.type, documentationSymbol); - targetScope->exportedTypeBindings[name] = globalTy; + for (const auto& [name, ty] : checkedModule->declaredGlobals) + { + TypeId globalTy = clone(ty, typeChecker.globalTypes, cloneState); + std::string documentationSymbol = packageName + "/global/" + name; + generateDocumentationSymbols(globalTy, documentationSymbol); + targetScope->bindings[typeChecker.globalNames.names->getOrAdd(name.c_str())] = {globalTy, Location(), false, {}, documentationSymbol}; - persist(globalTy.type); + persist(globalTy); + } + + for (const auto& [name, ty] : checkedModule->getModuleScope()->exportedTypeBindings) + { + TypeFun globalTy = clone(ty, typeChecker.globalTypes, cloneState); + std::string documentationSymbol = packageName + "/globaltype/" + name; + generateDocumentationSymbols(globalTy.type, documentationSymbol); + targetScope->exportedTypeBindings[name] = globalTy; + + persist(globalTy.type); + } } return LoadDefinitionFileResult{true, parseResult, checkedModule}; @@ -426,13 +493,13 @@ CheckResult Frontend::check(const ModuleName& name, std::optionalsecond == nullptr) - throw std::runtime_error("Frontend::modules does not have data for " + name); + throwRuntimeError("Frontend::modules does not have data for " + name, name); } else { auto it2 = moduleResolver.modules.find(name); if (it2 == moduleResolver.modules.end() || it2->second == nullptr) - throw std::runtime_error("Frontend::modules does not have data for " + name); + throwRuntimeError("Frontend::modules does not have data for " + name, name); } return CheckResult{ @@ -539,7 +606,7 @@ CheckResult Frontend::check(const ModuleName& name, std::optional) stream << "NormalizationTooComplex { }"; + else if constexpr (std::is_same_v) + stream << "TypePackMismatch { wanted = '" + toString(err.wantedTp) + "', given = '" + toString(err.givenTp) + "' }"; else static_assert(always_false_v, "Non-exhaustive type switch"); } diff --git a/Analysis/src/Normalize.cpp b/Analysis/src/Normalize.cpp index cea159c3..5ef4b7e7 100644 --- a/Analysis/src/Normalize.cpp +++ b/Analysis/src/Normalize.cpp @@ -7,6 +7,7 @@ #include "Luau/Clone.h" #include "Luau/Common.h" +#include "Luau/TypeVar.h" #include "Luau/Unifier.h" #include "Luau/VisitTypeVar.h" @@ -18,6 +19,7 @@ LUAU_FASTINTVARIABLE(LuauNormalizeIterationLimit, 1200); LUAU_FASTINTVARIABLE(LuauNormalizeCacheLimit, 100000); LUAU_FASTFLAGVARIABLE(LuauNormalizeCombineTableFix, false); LUAU_FASTFLAGVARIABLE(LuauTypeNormalization2, false); +LUAU_FASTFLAGVARIABLE(LuauNegatedStringSingletons, false); LUAU_FASTFLAG(LuauUnknownAndNeverType) LUAU_FASTFLAG(DebugLuauDeferredConstraintResolution) LUAU_FASTFLAG(LuauOverloadedFunctionSubtypingPerf); @@ -107,12 +109,110 @@ bool TypeIds::operator==(const TypeIds& there) const return hash == there.hash && types == there.types; } +NormalizedStringType::NormalizedStringType(bool isCofinite, std::optional> singletons) + : isCofinite(isCofinite) + , singletons(std::move(singletons)) +{ + if (!FFlag::LuauNegatedStringSingletons) + LUAU_ASSERT(!isCofinite); +} + +void NormalizedStringType::resetToString() +{ + if (FFlag::LuauNegatedStringSingletons) + { + isCofinite = true; + singletons->clear(); + } + else + singletons.reset(); +} + +void NormalizedStringType::resetToNever() +{ + if (FFlag::LuauNegatedStringSingletons) + { + isCofinite = false; + singletons.emplace(); + } + else + { + if (singletons) + singletons->clear(); + else + singletons.emplace(); + } +} + +bool NormalizedStringType::isNever() const +{ + if (FFlag::LuauNegatedStringSingletons) + return !isCofinite && singletons->empty(); + else + return singletons && singletons->empty(); +} + +bool NormalizedStringType::isString() const +{ + if (FFlag::LuauNegatedStringSingletons) + return isCofinite && singletons->empty(); + else + return !singletons; +} + +bool NormalizedStringType::isUnion() const +{ + if (FFlag::LuauNegatedStringSingletons) + return !isCofinite; + else + return singletons.has_value(); +} + +bool NormalizedStringType::isIntersection() const +{ + if (FFlag::LuauNegatedStringSingletons) + return isCofinite; + else + return false; +} + +bool NormalizedStringType::includes(const std::string& str) const +{ + if (isString()) + return true; + else if (isUnion() && singletons->count(str)) + return true; + else if (isIntersection() && !singletons->count(str)) + return true; + else + return false; +} + +const NormalizedStringType NormalizedStringType::never{false, {{}}}; + +bool isSubtype(const NormalizedStringType& subStr, const NormalizedStringType& superStr) +{ + if (subStr.isUnion() && superStr.isUnion()) + { + for (auto [name, ty] : *subStr.singletons) + { + if (!superStr.singletons->count(name)) + return false; + } + } + else if (subStr.isString() && superStr.isUnion()) + return false; + + return true; +} + NormalizedType::NormalizedType(NotNull singletonTypes) : tops(singletonTypes->neverType) , booleans(singletonTypes->neverType) , errors(singletonTypes->neverType) , nils(singletonTypes->neverType) , numbers(singletonTypes->neverType) + , strings{NormalizedStringType::never} , threads(singletonTypes->neverType) { } @@ -120,7 +220,7 @@ NormalizedType::NormalizedType(NotNull singletonTypes) static bool isInhabited(const NormalizedType& norm) { return !get(norm.tops) || !get(norm.booleans) || !norm.classes.empty() || !get(norm.errors) || - !get(norm.nils) || !get(norm.numbers) || !norm.strings || !norm.strings->empty() || + !get(norm.nils) || !get(norm.numbers) || !norm.strings.isNever() || !get(norm.threads) || norm.functions || !norm.tables.empty() || !norm.tyvars.empty(); } @@ -183,10 +283,10 @@ static bool isNormalizedNumber(TypeId ty) static bool isNormalizedString(const NormalizedStringType& ty) { - if (!ty) + if (ty.isString()) return true; - for (auto& [str, ty] : *ty) + for (auto& [str, ty] : *ty.singletons) { if (const SingletonTypeVar* stv = get(ty)) { @@ -317,10 +417,7 @@ void Normalizer::clearNormal(NormalizedType& norm) norm.errors = singletonTypes->neverType; norm.nils = singletonTypes->neverType; norm.numbers = singletonTypes->neverType; - if (norm.strings) - norm.strings->clear(); - else - norm.strings.emplace(); + norm.strings.resetToNever(); norm.threads = singletonTypes->neverType; norm.tables.clear(); norm.functions = std::nullopt; @@ -495,10 +592,56 @@ void Normalizer::unionClasses(TypeIds& heres, const TypeIds& theres) void Normalizer::unionStrings(NormalizedStringType& here, const NormalizedStringType& there) { - if (!there) - here.reset(); - else if (here) - here->insert(there->begin(), there->end()); + if (FFlag::LuauNegatedStringSingletons) + { + if (there.isString()) + here.resetToString(); + else if (here.isUnion() && there.isUnion()) + here.singletons->insert(there.singletons->begin(), there.singletons->end()); + else if (here.isUnion() && there.isIntersection()) + { + here.isCofinite = true; + for (const auto& pair : *there.singletons) + { + auto it = here.singletons->find(pair.first); + if (it != end(*here.singletons)) + here.singletons->erase(it); + else + here.singletons->insert(pair); + } + } + else if (here.isIntersection() && there.isUnion()) + { + for (const auto& [name, ty] : *there.singletons) + here.singletons->erase(name); + } + else if (here.isIntersection() && there.isIntersection()) + { + auto iter = begin(*here.singletons); + auto endIter = end(*here.singletons); + + while (iter != endIter) + { + if (!there.singletons->count(iter->first)) + { + auto eraseIt = iter; + ++iter; + here.singletons->erase(eraseIt); + } + else + ++iter; + } + } + else + LUAU_ASSERT(!"Unreachable"); + } + else + { + if (there.isString()) + here.resetToString(); + else if (here.isUnion()) + here.singletons->insert(there.singletons->begin(), there.singletons->end()); + } } std::optional Normalizer::unionOfTypePacks(TypePackId here, TypePackId there) @@ -858,7 +1001,7 @@ bool Normalizer::unionNormalWithTy(NormalizedType& here, TypeId there, int ignor else if (ptv->type == PrimitiveTypeVar::Number) here.numbers = there; else if (ptv->type == PrimitiveTypeVar::String) - here.strings = std::nullopt; + here.strings.resetToString(); else if (ptv->type == PrimitiveTypeVar::Thread) here.threads = there; else @@ -870,12 +1013,33 @@ bool Normalizer::unionNormalWithTy(NormalizedType& here, TypeId there, int ignor here.booleans = unionOfBools(here.booleans, there); else if (const StringSingleton* sstv = get(stv)) { - if (here.strings) - here.strings->insert({sstv->value, there}); + if (FFlag::LuauNegatedStringSingletons) + { + if (here.strings.isCofinite) + { + auto it = here.strings.singletons->find(sstv->value); + if (it != here.strings.singletons->end()) + here.strings.singletons->erase(it); + } + else + here.strings.singletons->insert({sstv->value, there}); + } + else + { + if (here.strings.isUnion()) + here.strings.singletons->insert({sstv->value, there}); + } } else LUAU_ASSERT(!"Unreachable"); } + else if (const NegationTypeVar* ntv = get(there)) + { + const NormalizedType* thereNormal = normalize(ntv->ty); + NormalizedType tn = negateNormal(*thereNormal); + if (!unionNormals(here, tn)) + return false; + } else LUAU_ASSERT(!"Unreachable"); @@ -887,6 +1051,159 @@ bool Normalizer::unionNormalWithTy(NormalizedType& here, TypeId there, int ignor return true; } +// ------- Negations + +NormalizedType Normalizer::negateNormal(const NormalizedType& here) +{ + NormalizedType result{singletonTypes}; + if (!get(here.tops)) + { + // The negation of unknown or any is never. Easy. + return result; + } + + if (!get(here.errors)) + { + // Negating an error yields the same error. + result.errors = here.errors; + return result; + } + + if (get(here.booleans)) + result.booleans = singletonTypes->booleanType; + else if (get(here.booleans)) + result.booleans = singletonTypes->neverType; + else if (auto stv = get(here.booleans)) + { + auto boolean = get(stv); + LUAU_ASSERT(boolean != nullptr); + if (boolean->value) + result.booleans = singletonTypes->falseType; + else + result.booleans = singletonTypes->trueType; + } + + result.classes = negateAll(here.classes); + result.nils = get(here.nils) ? singletonTypes->nilType : singletonTypes->neverType; + result.numbers = get(here.numbers) ? singletonTypes->numberType : singletonTypes->neverType; + + result.strings = here.strings; + result.strings.isCofinite = !result.strings.isCofinite; + + result.threads = get(here.threads) ? singletonTypes->threadType : singletonTypes->neverType; + + // TODO: negating tables + // TODO: negating functions + // TODO: negating tyvars? + + return result; +} + +TypeIds Normalizer::negateAll(const TypeIds& theres) +{ + TypeIds tys; + for (TypeId there : theres) + tys.insert(negate(there)); + return tys; +} + +TypeId Normalizer::negate(TypeId there) +{ + there = follow(there); + if (get(there)) + return there; + else if (get(there)) + return singletonTypes->neverType; + else if (get(there)) + return singletonTypes->unknownType; + else if (auto ntv = get(there)) + return ntv->ty; // TODO: do we want to normalize this? + else if (auto utv = get(there)) + { + std::vector parts; + for (TypeId option : utv) + parts.push_back(negate(option)); + return arena->addType(IntersectionTypeVar{std::move(parts)}); + } + else if (auto itv = get(there)) + { + std::vector options; + for (TypeId part : itv) + options.push_back(negate(part)); + return arena->addType(UnionTypeVar{std::move(options)}); + } + else + return there; +} + +void Normalizer::subtractPrimitive(NormalizedType& here, TypeId ty) +{ + const PrimitiveTypeVar* ptv = get(follow(ty)); + LUAU_ASSERT(ptv); + switch (ptv->type) + { + case PrimitiveTypeVar::NilType: + here.nils = singletonTypes->neverType; + break; + case PrimitiveTypeVar::Boolean: + here.booleans = singletonTypes->neverType; + break; + case PrimitiveTypeVar::Number: + here.numbers = singletonTypes->neverType; + break; + case PrimitiveTypeVar::String: + here.strings.resetToNever(); + break; + case PrimitiveTypeVar::Thread: + here.threads = singletonTypes->neverType; + break; + } +} + +void Normalizer::subtractSingleton(NormalizedType& here, TypeId ty) +{ + LUAU_ASSERT(FFlag::LuauNegatedStringSingletons); + + const SingletonTypeVar* stv = get(ty); + LUAU_ASSERT(stv); + + if (const StringSingleton* ss = get(stv)) + { + if (here.strings.isCofinite) + here.strings.singletons->insert({ss->value, ty}); + else + { + auto it = here.strings.singletons->find(ss->value); + if (it != here.strings.singletons->end()) + here.strings.singletons->erase(it); + } + } + else if (const BooleanSingleton* bs = get(stv)) + { + if (get(here.booleans)) + { + // Nothing + } + else if (get(here.booleans)) + here.booleans = bs->value ? singletonTypes->falseType : singletonTypes->trueType; + else if (auto hereSingleton = get(here.booleans)) + { + const BooleanSingleton* hereBooleanSingleton = get(hereSingleton); + LUAU_ASSERT(hereBooleanSingleton); + + // Crucial subtlety: ty (and thus bs) are the value that is being + // negated out. We therefore reduce to never when the values match, + // rather than when they differ. + if (bs->value == hereBooleanSingleton->value) + here.booleans = singletonTypes->neverType; + } + else + LUAU_ASSERT(!"Unreachable"); + } + else + LUAU_ASSERT(!"Unreachable"); +} + // ------- Normalizing intersections TypeId Normalizer::intersectionOfTops(TypeId here, TypeId there) { @@ -971,17 +1288,17 @@ void Normalizer::intersectClassesWithClass(TypeIds& heres, TypeId there) void Normalizer::intersectStrings(NormalizedStringType& here, const NormalizedStringType& there) { - if (!there) + if (there.isString()) return; - if (!here) - here.emplace(); + if (here.isString()) + here.resetToNever(); - for (auto it = here->begin(); it != here->end();) + for (auto it = here.singletons->begin(); it != here.singletons->end();) { - if (there->count(it->first)) + if (there.singletons->count(it->first)) it++; else - it = here->erase(it); + it = here.singletons->erase(it); } } @@ -1646,12 +1963,35 @@ bool Normalizer::intersectNormalWithTy(NormalizedType& here, TypeId there) here.booleans = intersectionOfBools(booleans, there); else if (const StringSingleton* sstv = get(stv)) { - if (!strings || strings->count(sstv->value)) - here.strings->insert({sstv->value, there}); + if (strings.includes(sstv->value)) + here.strings.singletons->insert({sstv->value, there}); } else LUAU_ASSERT(!"Unreachable"); } + else if (const NegationTypeVar* ntv = get(there); FFlag::LuauNegatedStringSingletons && ntv) + { + TypeId t = follow(ntv->ty); + if (const PrimitiveTypeVar* ptv = get(t)) + subtractPrimitive(here, ntv->ty); + else if (const SingletonTypeVar* stv = get(t)) + subtractSingleton(here, follow(ntv->ty)); + else if (const UnionTypeVar* itv = get(t)) + { + for (TypeId part : itv->options) + { + const NormalizedType* normalPart = normalize(part); + NormalizedType negated = negateNormal(*normalPart); + intersectNormals(here, negated); + } + } + else + { + // TODO negated unions, intersections, table, and function. + // Report a TypeError for other types. + LUAU_ASSERT(!"Unimplemented"); + } + } else LUAU_ASSERT(!"Unreachable"); @@ -1691,11 +2031,25 @@ TypeId Normalizer::typeFromNormal(const NormalizedType& norm) result.push_back(norm.nils); if (!get(norm.numbers)) result.push_back(norm.numbers); - if (norm.strings) - for (auto& [_, ty] : *norm.strings) - result.push_back(ty); - else + if (norm.strings.isString()) result.push_back(singletonTypes->stringType); + else if (norm.strings.isUnion()) + { + for (auto& [_, ty] : *norm.strings.singletons) + result.push_back(ty); + } + else if (FFlag::LuauNegatedStringSingletons && norm.strings.isIntersection()) + { + std::vector parts; + parts.push_back(singletonTypes->stringType); + for (const auto& [name, ty] : *norm.strings.singletons) + parts.push_back(arena->addType(NegationTypeVar{ty})); + + result.push_back(arena->addType(IntersectionTypeVar{std::move(parts)})); + } + if (!get(norm.threads)) + result.push_back(singletonTypes->threadType); + result.insert(result.end(), norm.tables.begin(), norm.tables.end()); for (auto& [tyvar, intersect] : norm.tyvars) { diff --git a/Analysis/src/ToString.cpp b/Analysis/src/ToString.cpp index 31641af8..7d0fb22d 100644 --- a/Analysis/src/ToString.cpp +++ b/Analysis/src/ToString.cpp @@ -11,6 +11,7 @@ #include LUAU_FASTFLAG(LuauUnknownAndNeverType) +LUAU_FASTFLAG(LuauLvaluelessPath) LUAU_FASTFLAGVARIABLE(LuauSpecialTypesAsterisked, false) LUAU_FASTFLAGVARIABLE(LuauFixNameMaps, false) LUAU_FASTFLAGVARIABLE(LuauUnseeArrayTtv, false) @@ -435,7 +436,7 @@ struct TypeVarStringifier return; default: LUAU_ASSERT(!"Unknown primitive type"); - throw std::runtime_error("Unknown primitive type " + std::to_string(ptv.type)); + throwRuntimeError("Unknown primitive type " + std::to_string(ptv.type)); } } @@ -452,7 +453,7 @@ struct TypeVarStringifier else { LUAU_ASSERT(!"Unknown singleton type"); - throw std::runtime_error("Unknown singleton type"); + throwRuntimeError("Unknown singleton type"); } } @@ -1548,6 +1549,8 @@ std::string dump(const Constraint& c) std::string toString(const LValue& lvalue) { + LUAU_ASSERT(!FFlag::LuauLvaluelessPath); + std::string s; for (const LValue* current = &lvalue; current; current = baseof(*current)) { @@ -1562,4 +1565,37 @@ std::string toString(const LValue& lvalue) return s; } +std::optional getFunctionNameAsString(const AstExpr& expr) +{ + LUAU_ASSERT(FFlag::LuauLvaluelessPath); + + const AstExpr* curr = &expr; + std::string s; + + for (;;) + { + if (auto local = curr->as()) + return local->local->name.value + s; + + if (auto global = curr->as()) + return global->name.value + s; + + if (auto indexname = curr->as()) + { + curr = indexname->expr; + + s = "." + std::string(indexname->index.value) + s; + } + else if (auto group = curr->as()) + { + curr = group->expr; + } + else + { + return std::nullopt; + } + } + + return s; +} } // namespace Luau diff --git a/Analysis/src/TopoSortStatements.cpp b/Analysis/src/TopoSortStatements.cpp index 1ea2e27d..052c10de 100644 --- a/Analysis/src/TopoSortStatements.cpp +++ b/Analysis/src/TopoSortStatements.cpp @@ -1,6 +1,7 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #include "Luau/TopoSortStatements.h" +#include "Luau/Error.h" /* Decide the order in which we typecheck Lua statements in a block. * * Algorithm: @@ -149,7 +150,7 @@ Identifier mkName(const AstStatFunction& function) auto name = mkName(*function.name); LUAU_ASSERT(bool(name)); if (!name) - throw std::runtime_error("Internal error: Function declaration has a bad name"); + throwRuntimeError("Internal error: Function declaration has a bad name"); return *name; } @@ -255,7 +256,7 @@ struct ArcCollector : public AstVisitor { auto name = mkName(*node->name); if (!name) - throw std::runtime_error("Internal error: AstStatFunction has a bad name"); + throwRuntimeError("Internal error: AstStatFunction has a bad name"); add(*name); return true; diff --git a/Analysis/src/TypeAttach.cpp b/Analysis/src/TypeAttach.cpp index f2613cae..179846d7 100644 --- a/Analysis/src/TypeAttach.cpp +++ b/Analysis/src/TypeAttach.cpp @@ -347,7 +347,7 @@ public: AstType* operator()(const NegationTypeVar& ntv) { // FIXME: do the same thing we do with ErrorTypeVar - throw std::runtime_error("Cannot convert NegationTypeVar into AstNode"); + throwRuntimeError("Cannot convert NegationTypeVar into AstNode"); } private: diff --git a/Analysis/src/TypeChecker2.cpp b/Analysis/src/TypeChecker2.cpp index bd220e9c..a2673158 100644 --- a/Analysis/src/TypeChecker2.cpp +++ b/Analysis/src/TypeChecker2.cpp @@ -934,8 +934,62 @@ struct TypeChecker2 void visit(AstExprUnary* expr) { - // TODO! visit(expr->expr); + + NotNull scope = stack.back(); + TypeId operandType = lookupType(expr->expr); + + if (get(operandType) || get(operandType) || get(operandType)) + return; + + if (auto it = kUnaryOpMetamethods.find(expr->op); it != kUnaryOpMetamethods.end()) + { + std::optional mm = findMetatableEntry(singletonTypes, module->errors, operandType, it->second, expr->location); + if (mm) + { + if (const FunctionTypeVar* ftv = get(follow(*mm))) + { + TypePackId expectedArgs = module->internalTypes.addTypePack({operandType}); + reportErrors(tryUnify(scope, expr->location, ftv->argTypes, expectedArgs)); + + if (std::optional ret = first(ftv->retTypes)) + { + if (expr->op == AstExprUnary::Op::Len) + { + reportErrors(tryUnify(scope, expr->location, follow(*ret), singletonTypes->numberType)); + } + } + else + { + reportError(GenericError{format("Metamethod '%s' must return a value", it->second)}, expr->location); + } + } + + return; + } + } + + if (expr->op == AstExprUnary::Op::Len) + { + DenseHashSet seen{nullptr}; + int recursionCount = 0; + + if (!hasLength(operandType, seen, &recursionCount)) + { + reportError(NotATable{operandType}, expr->location); + } + } + else if (expr->op == AstExprUnary::Op::Minus) + { + reportErrors(tryUnify(scope, expr->location, operandType, singletonTypes->numberType)); + } + else if (expr->op == AstExprUnary::Op::Not) + { + } + else + { + LUAU_ASSERT(!"Unhandled unary operator"); + } } void visit(AstExprBinary* expr) @@ -1240,9 +1294,8 @@ struct TypeChecker2 Scope* scope = findInnermostScope(ty->location); LUAU_ASSERT(scope); - // TODO: Imported types - - std::optional alias = scope->lookupType(ty->name.value); + std::optional alias = + (ty->prefix) ? scope->lookupImportedType(ty->prefix->value, ty->name.value) : scope->lookupType(ty->name.value); if (alias.has_value()) { diff --git a/Analysis/src/TypeInfer.cpp b/Analysis/src/TypeInfer.cpp index d5c6b2c4..ccb1490a 100644 --- a/Analysis/src/TypeInfer.cpp +++ b/Analysis/src/TypeInfer.cpp @@ -36,6 +36,7 @@ LUAU_FASTFLAGVARIABLE(DebugLuauFreezeDuringUnification, false) LUAU_FASTFLAGVARIABLE(LuauReturnAnyInsteadOfICE, false) // Eventually removed as false. LUAU_FASTFLAGVARIABLE(DebugLuauSharedSelf, false) LUAU_FASTFLAGVARIABLE(LuauAnyifyModuleReturnGenerics, false) +LUAU_FASTFLAGVARIABLE(LuauLvaluelessPath, false) LUAU_FASTFLAGVARIABLE(LuauUnknownAndNeverType, false) LUAU_FASTFLAGVARIABLE(LuauBinaryNeedsExpectedTypesToo, false) LUAU_FASTFLAGVARIABLE(LuauFixVarargExprHeadType, false) @@ -43,15 +44,15 @@ LUAU_FASTFLAGVARIABLE(LuauNeverTypesAndOperatorsInference, false) LUAU_FASTFLAGVARIABLE(LuauReturnsFromCallsitesAreNotWidened, false) LUAU_FASTFLAG(LuauInstantiateInSubtyping) LUAU_FASTFLAGVARIABLE(LuauCompleteVisitor, false) -LUAU_FASTFLAGVARIABLE(LuauUnionOfTypesFollow, false) LUAU_FASTFLAGVARIABLE(LuauReportShadowedTypeAlias, false) LUAU_FASTFLAGVARIABLE(LuauBetterMessagingOnCountMismatch, false) +LUAU_FASTFLAGVARIABLE(LuauArgMismatchReportFunctionLocation, false) namespace Luau { - -const char* TimeLimitError::what() const throw() +const char* TimeLimitError_DEPRECATED::what() const throw() { + LUAU_ASSERT(!FFlag::LuauIceExceptionInheritanceChange); return "Typeinfer failed to complete in allotted time"; } @@ -264,6 +265,11 @@ ModulePtr TypeChecker::check(const SourceModule& module, Mode mode, std::optiona reportErrorCodeTooComplex(module.root->location); return std::move(currentModule); } + catch (const RecursionLimitException_DEPRECATED&) + { + reportErrorCodeTooComplex(module.root->location); + return std::move(currentModule); + } } ModulePtr TypeChecker::checkWithoutRecursionCheck(const SourceModule& module, Mode mode, std::optional environmentScope) @@ -308,6 +314,10 @@ ModulePtr TypeChecker::checkWithoutRecursionCheck(const SourceModule& module, Mo { currentModule->timeout = true; } + catch (const TimeLimitError_DEPRECATED&) + { + currentModule->timeout = true; + } if (FFlag::DebugLuauSharedSelf) { @@ -415,7 +425,7 @@ void TypeChecker::check(const ScopePtr& scope, const AstStat& program) ice("Unknown AstStat"); if (finishTime && TimeTrace::getClock() > *finishTime) - throw TimeLimitError(); + throwTimeLimitError(); } // This particular overload is for do...end. If you need to not increase the scope level, use checkBlock directly. @@ -442,6 +452,11 @@ void TypeChecker::checkBlock(const ScopePtr& scope, const AstStatBlock& block) reportErrorCodeTooComplex(block.location); return; } + catch (const RecursionLimitException_DEPRECATED&) + { + reportErrorCodeTooComplex(block.location); + return; + } } struct InplaceDemoter : TypeVarOnceVisitor @@ -2456,11 +2471,8 @@ std::string opToMetaTableEntry(const AstExprBinary::Op& op) TypeId TypeChecker::unionOfTypes(TypeId a, TypeId b, const ScopePtr& scope, const Location& location, bool unifyFreeTypes) { - if (FFlag::LuauUnionOfTypesFollow) - { - a = follow(a); - b = follow(b); - } + a = follow(a); + b = follow(b); if (unifyFreeTypes && (get(a) || get(b))) { @@ -3596,8 +3608,17 @@ void TypeChecker::checkArgumentList(const ScopePtr& scope, const AstExpr& funNam location = {state.location.begin, argLocations.back().end}; std::string namePath; - if (std::optional lValue = tryGetLValue(funName)) - namePath = toString(*lValue); + + if (FFlag::LuauLvaluelessPath) + { + if (std::optional path = getFunctionNameAsString(funName)) + namePath = *path; + } + else + { + if (std::optional lValue = tryGetLValue(funName)) + namePath = toString(*lValue); + } auto [minParams, optMaxParams] = getParameterExtents(&state.log, paramPack); state.reportError(TypeError{location, @@ -3706,11 +3727,28 @@ void TypeChecker::checkArgumentList(const ScopePtr& scope, const AstExpr& funNam bool isVariadic = tail && Luau::isVariadic(*tail); std::string namePath; - if (std::optional lValue = tryGetLValue(funName)) - namePath = toString(*lValue); - state.reportError(TypeError{ - state.location, CountMismatch{minParams, optMaxParams, paramIndex, CountMismatch::Context::Arg, isVariadic, namePath}}); + if (FFlag::LuauLvaluelessPath) + { + if (std::optional path = getFunctionNameAsString(funName)) + namePath = *path; + } + else + { + if (std::optional lValue = tryGetLValue(funName)) + namePath = toString(*lValue); + } + + if (FFlag::LuauArgMismatchReportFunctionLocation) + { + state.reportError(TypeError{ + funName.location, CountMismatch{minParams, optMaxParams, paramIndex, CountMismatch::Context::Arg, isVariadic, namePath}}); + } + else + { + state.reportError(TypeError{ + state.location, CountMismatch{minParams, optMaxParams, paramIndex, CountMismatch::Context::Arg, isVariadic, namePath}}); + } return; } ++paramIter; @@ -4647,6 +4685,19 @@ void TypeChecker::ice(const std::string& message) iceHandler->ice(message); } +// TODO: Inline me when LuauIceExceptionInheritanceChange is deleted. +void TypeChecker::throwTimeLimitError() +{ + if (FFlag::LuauIceExceptionInheritanceChange) + { + throw TimeLimitError(iceHandler->moduleName); + } + else + { + throw TimeLimitError_DEPRECATED(); + } +} + void TypeChecker::prepareErrorsForDisplay(ErrorVec& errVec) { // Remove errors with names that were generated by recovery from a parse error diff --git a/Analysis/src/TypePack.cpp b/Analysis/src/TypePack.cpp index 0fa4df60..0852f053 100644 --- a/Analysis/src/TypePack.cpp +++ b/Analysis/src/TypePack.cpp @@ -1,6 +1,7 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #include "Luau/TypePack.h" +#include "Luau/Error.h" #include "Luau/TxnLog.h" #include @@ -234,7 +235,7 @@ TypePackId follow(TypePackId tp, std::function mapper) cycleTester = nullptr; if (tp == cycleTester) - throw std::runtime_error("Luau::follow detected a TypeVar cycle!!"); + throwRuntimeError("Luau::follow detected a TypeVar cycle!!"); } } } diff --git a/Analysis/src/TypeVar.cpp b/Analysis/src/TypeVar.cpp index 19d3d266..94d633c7 100644 --- a/Analysis/src/TypeVar.cpp +++ b/Analysis/src/TypeVar.cpp @@ -61,7 +61,7 @@ TypeId follow(TypeId t, std::function mapper) { std::optional ty = utv->scope->lookup(utv->def); if (!ty) - throw std::runtime_error("UseTypeVar must map to another TypeId"); + throwRuntimeError("UseTypeVar must map to another TypeId"); return *ty; } else @@ -73,7 +73,7 @@ TypeId follow(TypeId t, std::function mapper) { TypeId res = ltv->thunk(); if (get(res)) - throw std::runtime_error("Lazy TypeVar cannot resolve to another Lazy TypeVar"); + throwRuntimeError("Lazy TypeVar cannot resolve to another Lazy TypeVar"); *asMutable(ty) = BoundTypeVar(res); } @@ -111,7 +111,7 @@ TypeId follow(TypeId t, std::function mapper) cycleTester = nullptr; if (t == cycleTester) - throw std::runtime_error("Luau::follow detected a TypeVar cycle!!"); + throwRuntimeError("Luau::follow detected a TypeVar cycle!!"); } } } @@ -946,7 +946,7 @@ void persist(TypeId ty) queue.push_back(mtv->table); queue.push_back(mtv->metatable); } - else if (get(t) || get(t) || get(t) || get(t) || get(t)) + else if (get(t) || get(t) || get(t) || get(t) || get(t) || get(t)) { } else diff --git a/Analysis/src/Unifier.cpp b/Analysis/src/Unifier.cpp index e23e6161..b5eba980 100644 --- a/Analysis/src/Unifier.cpp +++ b/Analysis/src/Unifier.cpp @@ -16,6 +16,7 @@ LUAU_FASTINT(LuauTypeInferTypePackLoopLimit); LUAU_FASTFLAG(LuauErrorRecoveryType); LUAU_FASTFLAG(LuauUnknownAndNeverType) +LUAU_FASTFLAGVARIABLE(LuauReportTypeMismatchForTypePackUnificationFailure, false) LUAU_FASTFLAGVARIABLE(LuauSubtypeNormalizer, false); LUAU_FASTFLAGVARIABLE(LuauScalarShapeSubtyping, false) LUAU_FASTFLAGVARIABLE(LuauInstantiateInSubtyping, false) @@ -273,7 +274,7 @@ TypeId Widen::clean(TypeId ty) TypePackId Widen::clean(TypePackId) { - throw std::runtime_error("Widen attempted to clean a dirty type pack?"); + throwRuntimeError("Widen attempted to clean a dirty type pack?"); } bool Widen::ignoreChildren(TypeId ty) @@ -551,6 +552,12 @@ void Unifier::tryUnify_(TypeId subTy, TypeId superTy, bool isFunctionCall, bool else if (log.getMutable(subTy)) tryUnifyWithClass(subTy, superTy, /*reversed*/ true); + else if (log.get(superTy)) + tryUnifyTypeWithNegation(subTy, superTy); + + else if (log.get(subTy)) + tryUnifyNegationWithType(subTy, superTy); + else reportError(TypeError{location, TypeMismatch{superTy, subTy}}); @@ -866,13 +873,7 @@ void Unifier::tryUnifyNormalizedTypes( if (!get(superNorm.numbers)) return reportError(TypeError{location, TypeMismatch{superTy, subTy, reason, error}}); - if (subNorm.strings && superNorm.strings) - { - for (auto [name, ty] : *subNorm.strings) - if (!superNorm.strings->count(name)) - return reportError(TypeError{location, TypeMismatch{superTy, subTy, reason, error}}); - } - else if (!subNorm.strings && superNorm.strings) + if (!isSubtype(subNorm.strings, superNorm.strings)) return reportError(TypeError{location, TypeMismatch{superTy, subTy, reason, error}}); if (get(subNorm.threads)) @@ -1392,7 +1393,10 @@ void Unifier::tryUnify_(TypePackId subTp, TypePackId superTp, bool isFunctionCal } else { - reportError(TypeError{location, GenericError{"Failed to unify type packs"}}); + if (FFlag::LuauReportTypeMismatchForTypePackUnificationFailure) + reportError(TypeError{location, TypePackMismatch{subTp, superTp}}); + else + reportError(TypeError{location, GenericError{"Failed to unify type packs"}}); } } @@ -1441,7 +1445,10 @@ void Unifier::tryUnifyFunctions(TypeId subTy, TypeId superTy, bool isFunctionCal bool shouldInstantiate = (numGenerics == 0 && subFunction->generics.size() > 0) || (numGenericPacks == 0 && subFunction->genericPacks.size() > 0); - if (FFlag::LuauInstantiateInSubtyping && variance == Covariant && shouldInstantiate) + // TODO: This is unsound when the context is invariant, but the annotation burden without allowing it and without + // read-only properties is too high for lua-apps. Read-only properties _should_ resolve their issue by allowing + // generic methods in tables to be marked read-only. + if (FFlag::LuauInstantiateInSubtyping && shouldInstantiate) { Instantiation instantiation{&log, types, scope->level, scope}; @@ -1576,6 +1583,7 @@ void Unifier::tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) { TableTypeVar* superTable = log.getMutable(superTy); TableTypeVar* subTable = log.getMutable(subTy); + TableTypeVar* instantiatedSubTable = subTable; if (!superTable || !subTable) ice("passed non-table types to unifyTables"); @@ -1593,6 +1601,7 @@ void Unifier::tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) if (instantiated.has_value()) { subTable = log.getMutable(*instantiated); + instantiatedSubTable = subTable; if (!subTable) ice("instantiation made a table type into a non-table type in tryUnifyTables"); @@ -1696,7 +1705,7 @@ void Unifier::tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) // txn log. TableTypeVar* newSuperTable = log.getMutable(superTy); TableTypeVar* newSubTable = log.getMutable(subTy); - if (superTable != newSuperTable || subTable != newSubTable) + if (superTable != newSuperTable || (subTable != newSubTable && subTable != instantiatedSubTable)) { if (errors.empty()) return tryUnifyTables(subTy, superTy, isIntersection); @@ -1758,7 +1767,7 @@ void Unifier::tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) // txn log. TableTypeVar* newSuperTable = log.getMutable(superTy); TableTypeVar* newSubTable = log.getMutable(subTy); - if (superTable != newSuperTable || subTable != newSubTable) + if (superTable != newSuperTable || (subTable != newSubTable && subTable != instantiatedSubTable)) { if (errors.empty()) return tryUnifyTables(subTy, superTy, isIntersection); @@ -2098,6 +2107,34 @@ void Unifier::tryUnifyWithClass(TypeId subTy, TypeId superTy, bool reversed) return fail(); } +void Unifier::tryUnifyTypeWithNegation(TypeId subTy, TypeId superTy) +{ + const NegationTypeVar* ntv = get(superTy); + if (!ntv) + ice("tryUnifyTypeWithNegation superTy must be a negation type"); + + const NormalizedType* subNorm = normalizer->normalize(subTy); + const NormalizedType* superNorm = normalizer->normalize(superTy); + if (!subNorm || !superNorm) + return reportError(TypeError{location, UnificationTooComplex{}}); + + // T (subTy); + if (!ntv) + ice("tryUnifyNegationWithType subTy must be a negation type"); + + // TODO: ~T & queue, DenseHashSet& seenTypePacks, Unifier& state, TypePackId a, TypePackId anyTypePack) { while (true) diff --git a/Ast/include/Luau/ParseResult.h b/Ast/include/Luau/ParseResult.h index 17ce2e3b..9c0a9527 100644 --- a/Ast/include/Luau/ParseResult.h +++ b/Ast/include/Luau/ParseResult.h @@ -58,6 +58,8 @@ struct Comment struct ParseResult { AstStatBlock* root; + size_t lines = 0; + std::vector hotcomments; std::vector errors; diff --git a/Ast/include/Luau/Parser.h b/Ast/include/Luau/Parser.h index 848d7117..8b7eb73c 100644 --- a/Ast/include/Luau/Parser.h +++ b/Ast/include/Luau/Parser.h @@ -302,8 +302,8 @@ private: AstStatError* reportStatError(const Location& location, const AstArray& expressions, const AstArray& statements, const char* format, ...) LUAU_PRINTF_ATTR(5, 6); AstExprError* reportExprError(const Location& location, const AstArray& expressions, const char* format, ...) LUAU_PRINTF_ATTR(4, 5); - AstTypeError* reportTypeAnnotationError(const Location& location, const AstArray& types, bool isMissing, const char* format, ...) - LUAU_PRINTF_ATTR(5, 6); + AstTypeError* reportTypeAnnotationError(const Location& location, const AstArray& types, const char* format, ...) + LUAU_PRINTF_ATTR(4, 5); // `parseErrorLocation` is associated with the parser error // `astErrorLocation` is associated with the AstTypeError created // It can be useful to have different error locations so that the parse error can include the next lexeme, while the AstTypeError can precisely diff --git a/Ast/src/Parser.cpp b/Ast/src/Parser.cpp index 7150b18f..85c5f5c6 100644 --- a/Ast/src/Parser.cpp +++ b/Ast/src/Parser.cpp @@ -23,7 +23,6 @@ LUAU_FASTFLAGVARIABLE(LuauErrorDoubleHexPrefix, false) LUAU_DYNAMIC_FASTFLAGVARIABLE(LuaReportParseIntegerIssues, false) LUAU_FASTFLAGVARIABLE(LuauInterpolatedStringBaseSupport, false) -LUAU_FASTFLAGVARIABLE(LuauTypeAnnotationLocationChange, false) LUAU_FASTFLAGVARIABLE(LuauCommaParenWarnings, false) @@ -164,15 +163,16 @@ ParseResult Parser::parse(const char* buffer, size_t bufferSize, AstNameTable& n try { AstStatBlock* root = p.parseChunk(); + size_t lines = p.lexer.current().location.end.line + (bufferSize > 0 && buffer[bufferSize - 1] != '\n'); - return ParseResult{root, std::move(p.hotcomments), std::move(p.parseErrors), std::move(p.commentLocations)}; + return ParseResult{root, lines, std::move(p.hotcomments), std::move(p.parseErrors), std::move(p.commentLocations)}; } catch (ParseError& err) { // when catching a fatal error, append it to the list of non-fatal errors and return p.parseErrors.push_back(err); - return ParseResult{nullptr, {}, p.parseErrors}; + return ParseResult{nullptr, 0, {}, p.parseErrors}; } } @@ -811,9 +811,8 @@ AstDeclaredClassProp Parser::parseDeclaredClassMethod() if (args.size() == 0 || args[0].name.name != "self" || args[0].annotation != nullptr) { - return AstDeclaredClassProp{fnName.name, - reportTypeAnnotationError(Location(start, end), {}, /*isMissing*/ false, "'self' must be present as the unannotated first parameter"), - true}; + return AstDeclaredClassProp{ + fnName.name, reportTypeAnnotationError(Location(start, end), {}, "'self' must be present as the unannotated first parameter"), true}; } // Skip the first index. @@ -824,8 +823,7 @@ AstDeclaredClassProp Parser::parseDeclaredClassMethod() if (args[i].annotation) vars.push_back(args[i].annotation); else - vars.push_back(reportTypeAnnotationError( - Location(start, end), {}, /*isMissing*/ false, "All declaration parameters aside from 'self' must be annotated")); + vars.push_back(reportTypeAnnotationError(Location(start, end), {}, "All declaration parameters aside from 'self' must be annotated")); } if (vararg && !varargAnnotation) @@ -1537,7 +1535,7 @@ AstType* Parser::parseTypeAnnotation(TempVector& parts, const Location if (isUnion && isIntersection) { - return reportTypeAnnotationError(Location(begin, parts.back()->location), copy(parts), /*isMissing*/ false, + return reportTypeAnnotationError(Location(begin, parts.back()->location), copy(parts), "Mixing union and intersection types is not allowed; consider wrapping in parentheses."); } @@ -1623,18 +1621,18 @@ AstTypeOrPack Parser::parseSimpleTypeAnnotation(bool allowPack) return {allocator.alloc(start, svalue)}; } else - return {reportTypeAnnotationError(start, {}, /*isMissing*/ false, "String literal contains malformed escape sequence")}; + return {reportTypeAnnotationError(start, {}, "String literal contains malformed escape sequence")}; } else if (lexer.current().type == Lexeme::InterpStringBegin || lexer.current().type == Lexeme::InterpStringSimple) { parseInterpString(); - return {reportTypeAnnotationError(start, {}, /*isMissing*/ false, "Interpolated string literals cannot be used as types")}; + return {reportTypeAnnotationError(start, {}, "Interpolated string literals cannot be used as types")}; } else if (lexer.current().type == Lexeme::BrokenString) { nextLexeme(); - return {reportTypeAnnotationError(start, {}, /*isMissing*/ false, "Malformed string")}; + return {reportTypeAnnotationError(start, {}, "Malformed string")}; } else if (lexer.current().type == Lexeme::Name) { @@ -1693,33 +1691,20 @@ AstTypeOrPack Parser::parseSimpleTypeAnnotation(bool allowPack) { nextLexeme(); - return {reportTypeAnnotationError(start, {}, /*isMissing*/ false, + return {reportTypeAnnotationError(start, {}, "Using 'function' as a type annotation is not supported, consider replacing with a function type annotation e.g. '(...any) -> " "...any'"), {}}; } else { - if (FFlag::LuauTypeAnnotationLocationChange) - { - // For a missing type annotation, capture 'space' between last token and the next one - Location astErrorlocation(lexer.previousLocation().end, start.begin); - // The parse error includes the next lexeme to make it easier to display where the error is (e.g. in an IDE or a CLI error message). - // Including the current lexeme also makes the parse error consistent with other parse errors returned by Luau. - Location parseErrorLocation(lexer.previousLocation().end, start.end); - return { - reportMissingTypeAnnotationError(parseErrorLocation, astErrorlocation, "Expected type, got %s", lexer.current().toString().c_str()), - {}}; - } - else - { - Location location = lexer.current().location; - - // For a missing type annotation, capture 'space' between last token and the next one - location = Location(lexer.previousLocation().end, lexer.current().location.begin); - - return {reportTypeAnnotationError(location, {}, /*isMissing*/ true, "Expected type, got %s", lexer.current().toString().c_str()), {}}; - } + // For a missing type annotation, capture 'space' between last token and the next one + Location astErrorlocation(lexer.previousLocation().end, start.begin); + // The parse error includes the next lexeme to make it easier to display where the error is (e.g. in an IDE or a CLI error message). + // Including the current lexeme also makes the parse error consistent with other parse errors returned by Luau. + Location parseErrorLocation(lexer.previousLocation().end, start.end); + return { + reportMissingTypeAnnotationError(parseErrorLocation, astErrorlocation, "Expected type, got %s", lexer.current().toString().c_str()), {}}; } } @@ -3033,27 +3018,18 @@ AstExprError* Parser::reportExprError(const Location& location, const AstArray(location, expressions, unsigned(parseErrors.size() - 1)); } -AstTypeError* Parser::reportTypeAnnotationError(const Location& location, const AstArray& types, bool isMissing, const char* format, ...) +AstTypeError* Parser::reportTypeAnnotationError(const Location& location, const AstArray& types, const char* format, ...) { - if (FFlag::LuauTypeAnnotationLocationChange) - { - // Missing type annotations should be using `reportMissingTypeAnnotationError` when LuauTypeAnnotationLocationChange is enabled - // Note: `isMissing` can be removed once FFlag::LuauTypeAnnotationLocationChange is removed since it will always be true. - LUAU_ASSERT(!isMissing); - } - va_list args; va_start(args, format); report(location, format, args); va_end(args); - return allocator.alloc(location, types, isMissing, unsigned(parseErrors.size() - 1)); + return allocator.alloc(location, types, false, unsigned(parseErrors.size() - 1)); } AstTypeError* Parser::reportMissingTypeAnnotationError(const Location& parseErrorLocation, const Location& astErrorLocation, const char* format, ...) { - LUAU_ASSERT(FFlag::LuauTypeAnnotationLocationChange); - va_list args; va_start(args, format); report(parseErrorLocation, format, args); diff --git a/CLI/Repl.cpp b/CLI/Repl.cpp index a9dd8970..87e19db8 100644 --- a/CLI/Repl.cpp +++ b/CLI/Repl.cpp @@ -16,7 +16,6 @@ #include "isocline.h" -#include #include #ifdef _WIN32 @@ -688,11 +687,11 @@ static std::string getCodegenAssembly(const char* name, const std::string& bytec return ""; } -static void annotateInstruction(void* context, std::string& text, int fid, int instid) +static void annotateInstruction(void* context, std::string& text, int fid, int instpos) { Luau::BytecodeBuilder& bcb = *(Luau::BytecodeBuilder*)context; - bcb.annotateInstruction(text, fid, instid); + bcb.annotateInstruction(text, fid, instpos); } struct CompileStats @@ -711,7 +710,8 @@ static bool compileFile(const char* name, CompileFormat format, CompileStats& st return false; } - stats.lines += std::count(source->begin(), source->end(), '\n'); + // NOTE: Normally, you should use Luau::compile or luau_compile (see lua_require as an example) + // This function is much more complicated because it supports many output human-readable formats through internal interfaces try { @@ -736,7 +736,16 @@ static bool compileFile(const char* name, CompileFormat format, CompileStats& st bcb.setDumpSource(*source); } - Luau::compileOrThrow(bcb, *source, copts()); + Luau::Allocator allocator; + Luau::AstNameTable names(allocator); + Luau::ParseResult result = Luau::Parser::parse(source->c_str(), source->size(), names, allocator); + + if (!result.errors.empty()) + throw Luau::ParseErrors(result.errors); + + stats.lines += result.lines; + + Luau::compileOrThrow(bcb, result, names, copts()); stats.bytecode += bcb.getBytecode().size(); switch (format) diff --git a/CMakeLists.txt b/CMakeLists.txt index 0016160a..05d701ee 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -143,6 +143,11 @@ if (MSVC AND MSVC_VERSION GREATER_EQUAL 1924) set_source_files_properties(VM/src/lvmexecute.cpp PROPERTIES COMPILE_FLAGS /d2ssa-pre-) endif() +if (NOT MSVC) + # disable support for math_errno which allows compilers to lower sqrt() into a single CPU instruction + target_compile_options(Luau.VM PRIVATE -fno-math-errno) +endif() + if(MSVC AND LUAU_BUILD_CLI) # the default stack size that MSVC linker uses is 1 MB; we need more stack space in Debug because stack frames are larger set_target_properties(Luau.Analyze.CLI PROPERTIES LINK_FLAGS_DEBUG /STACK:2097152) diff --git a/CodeGen/include/Luau/CodeGen.h b/CodeGen/include/Luau/CodeGen.h index e8b30195..cef9ec7c 100644 --- a/CodeGen/include/Luau/CodeGen.h +++ b/CodeGen/include/Luau/CodeGen.h @@ -17,7 +17,7 @@ void create(lua_State* L); // Builds target function and all inner functions void compile(lua_State* L, int idx); -using annotatorFn = void (*)(void* context, std::string& result, int fid, int instid); +using annotatorFn = void (*)(void* context, std::string& result, int fid, int instpos); struct AssemblyOptions { diff --git a/CodeGen/src/CodeGen.cpp b/CodeGen/src/CodeGen.cpp index f78ead59..78645766 100644 --- a/CodeGen/src/CodeGen.cpp +++ b/CodeGen/src/CodeGen.cpp @@ -34,7 +34,350 @@ namespace CodeGen constexpr uint32_t kFunctionAlignment = 32; -static NativeProto* assembleFunction(AssemblyBuilderX64& build, NativeState& data, Proto* proto, AssemblyOptions options) +struct InstructionOutline +{ + int pcpos; + int length; +}; + +static void assembleHelpers(AssemblyBuilderX64& build, ModuleHelpers& helpers) +{ + if (build.logText) + build.logAppend("; exitContinueVm\n"); + helpers.exitContinueVm = build.setLabel(); + emitExit(build, /* continueInVm */ true); + + if (build.logText) + build.logAppend("; exitNoContinueVm\n"); + helpers.exitNoContinueVm = build.setLabel(); + emitExit(build, /* continueInVm */ false); +} + +static int emitInst( + AssemblyBuilderX64& build, NativeState& data, ModuleHelpers& helpers, Proto* proto, LuauOpcode op, const Instruction* pc, int i, Label* labelarr, Label& fallback) +{ + int skip = 0; + + switch (op) + { + case LOP_NOP: + break; + case LOP_LOADNIL: + emitInstLoadNil(build, pc); + break; + case LOP_LOADB: + emitInstLoadB(build, pc, i, labelarr); + break; + case LOP_LOADN: + emitInstLoadN(build, pc); + break; + case LOP_LOADK: + emitInstLoadK(build, pc); + break; + case LOP_LOADKX: + emitInstLoadKX(build, pc); + break; + case LOP_MOVE: + emitInstMove(build, pc); + break; + case LOP_GETGLOBAL: + emitInstGetGlobal(build, pc, i, fallback); + break; + case LOP_SETGLOBAL: + emitInstSetGlobal(build, pc, i, labelarr, fallback); + break; + case LOP_RETURN: + emitInstReturn(build, helpers, pc, i, labelarr); + break; + case LOP_GETTABLE: + emitInstGetTable(build, pc, i, fallback); + break; + case LOP_SETTABLE: + emitInstSetTable(build, pc, i, labelarr, fallback); + break; + case LOP_GETTABLEKS: + emitInstGetTableKS(build, pc, i, fallback); + break; + case LOP_SETTABLEKS: + emitInstSetTableKS(build, pc, i, labelarr, fallback); + break; + case LOP_GETTABLEN: + emitInstGetTableN(build, pc, i, fallback); + break; + case LOP_SETTABLEN: + emitInstSetTableN(build, pc, i, labelarr, fallback); + break; + case LOP_JUMP: + emitInstJump(build, pc, i, labelarr); + break; + case LOP_JUMPBACK: + emitInstJumpBack(build, pc, i, labelarr); + break; + case LOP_JUMPIF: + emitInstJumpIf(build, pc, i, labelarr, /* not_ */ false); + break; + case LOP_JUMPIFNOT: + emitInstJumpIf(build, pc, i, labelarr, /* not_ */ true); + break; + case LOP_JUMPIFEQ: + emitInstJumpIfEq(build, pc, i, labelarr, /* not_ */ false, fallback); + break; + case LOP_JUMPIFLE: + emitInstJumpIfCond(build, pc, i, labelarr, Condition::LessEqual, fallback); + break; + case LOP_JUMPIFLT: + emitInstJumpIfCond(build, pc, i, labelarr, Condition::Less, fallback); + break; + case LOP_JUMPIFNOTEQ: + emitInstJumpIfEq(build, pc, i, labelarr, /* not_ */ true, fallback); + break; + case LOP_JUMPIFNOTLE: + emitInstJumpIfCond(build, pc, i, labelarr, Condition::NotLessEqual, fallback); + break; + case LOP_JUMPIFNOTLT: + emitInstJumpIfCond(build, pc, i, labelarr, Condition::NotLess, fallback); + break; + case LOP_JUMPX: + emitInstJumpX(build, pc, i, labelarr); + break; + case LOP_JUMPXEQKNIL: + emitInstJumpxEqNil(build, pc, i, labelarr); + break; + case LOP_JUMPXEQKB: + emitInstJumpxEqB(build, pc, i, labelarr); + break; + case LOP_JUMPXEQKN: + emitInstJumpxEqN(build, pc, proto->k, i, labelarr); + break; + case LOP_JUMPXEQKS: + emitInstJumpxEqS(build, pc, i, labelarr); + break; + case LOP_ADD: + emitInstBinary(build, pc, i, TM_ADD, fallback); + break; + case LOP_SUB: + emitInstBinary(build, pc, i, TM_SUB, fallback); + break; + case LOP_MUL: + emitInstBinary(build, pc, i, TM_MUL, fallback); + break; + case LOP_DIV: + emitInstBinary(build, pc, i, TM_DIV, fallback); + break; + case LOP_MOD: + emitInstBinary(build, pc, i, TM_MOD, fallback); + break; + case LOP_POW: + emitInstBinary(build, pc, i, TM_POW, fallback); + break; + case LOP_ADDK: + emitInstBinaryK(build, pc, i, TM_ADD, fallback); + break; + case LOP_SUBK: + emitInstBinaryK(build, pc, i, TM_SUB, fallback); + break; + case LOP_MULK: + emitInstBinaryK(build, pc, i, TM_MUL, fallback); + break; + case LOP_DIVK: + emitInstBinaryK(build, pc, i, TM_DIV, fallback); + break; + case LOP_MODK: + emitInstBinaryK(build, pc, i, TM_MOD, fallback); + break; + case LOP_POWK: + emitInstPowK(build, pc, proto->k, i, fallback); + break; + case LOP_NOT: + emitInstNot(build, pc); + break; + case LOP_MINUS: + emitInstMinus(build, pc, i, fallback); + break; + case LOP_LENGTH: + emitInstLength(build, pc, i, fallback); + break; + case LOP_NEWTABLE: + emitInstNewTable(build, pc, i, labelarr); + break; + case LOP_DUPTABLE: + emitInstDupTable(build, pc, i, labelarr); + break; + case LOP_SETLIST: + emitInstSetList(build, pc, i, labelarr); + break; + case LOP_GETUPVAL: + emitInstGetUpval(build, pc, i); + break; + case LOP_SETUPVAL: + emitInstSetUpval(build, pc, i, labelarr); + break; + case LOP_CLOSEUPVALS: + emitInstCloseUpvals(build, pc, i, labelarr); + break; + case LOP_FASTCALL: + skip = emitInstFastCall(build, pc, i, labelarr); + break; + case LOP_FASTCALL1: + skip = emitInstFastCall1(build, pc, i, labelarr); + break; + case LOP_FASTCALL2: + skip = emitInstFastCall2(build, pc, i, labelarr); + break; + case LOP_FASTCALL2K: + skip = emitInstFastCall2K(build, pc, i, labelarr); + break; + case LOP_FORNPREP: + emitInstForNPrep(build, pc, i, labelarr); + break; + case LOP_FORNLOOP: + emitInstForNLoop(build, pc, i, labelarr); + break; + case LOP_FORGLOOP: + emitinstForGLoop(build, pc, i, labelarr, fallback); + break; + case LOP_FORGPREP_NEXT: + emitInstForGPrepNext(build, pc, i, labelarr, fallback); + break; + case LOP_FORGPREP_INEXT: + emitInstForGPrepInext(build, pc, i, labelarr, fallback); + break; + case LOP_AND: + emitInstAnd(build, pc); + break; + case LOP_ANDK: + emitInstAndK(build, pc); + break; + case LOP_OR: + emitInstOr(build, pc); + break; + case LOP_ORK: + emitInstOrK(build, pc); + break; + case LOP_GETIMPORT: + emitInstGetImport(build, pc, fallback); + break; + case LOP_CONCAT: + emitInstConcat(build, pc, i, labelarr); + break; + default: + emitFallback(build, data, op, i); + break; + } + + return skip; +} + +static void emitInstFallback(AssemblyBuilderX64& build, NativeState& data, LuauOpcode op, const Instruction* pc, int i, Label* labelarr) +{ + switch (op) + { + case LOP_GETIMPORT: + emitInstGetImportFallback(build, pc, i); + break; + case LOP_GETTABLE: + emitInstGetTableFallback(build, pc, i); + break; + case LOP_SETTABLE: + emitInstSetTableFallback(build, pc, i); + break; + case LOP_GETTABLEN: + emitInstGetTableNFallback(build, pc, i); + break; + case LOP_SETTABLEN: + emitInstSetTableNFallback(build, pc, i); + break; + case LOP_JUMPIFEQ: + emitInstJumpIfEqFallback(build, pc, i, labelarr, /* not_ */ false); + break; + case LOP_JUMPIFLE: + emitInstJumpIfCondFallback(build, pc, i, labelarr, Condition::LessEqual); + break; + case LOP_JUMPIFLT: + emitInstJumpIfCondFallback(build, pc, i, labelarr, Condition::Less); + break; + case LOP_JUMPIFNOTEQ: + emitInstJumpIfEqFallback(build, pc, i, labelarr, /* not_ */ true); + break; + case LOP_JUMPIFNOTLE: + emitInstJumpIfCondFallback(build, pc, i, labelarr, Condition::NotLessEqual); + break; + case LOP_JUMPIFNOTLT: + emitInstJumpIfCondFallback(build, pc, i, labelarr, Condition::NotLess); + break; + case LOP_ADD: + emitInstBinaryFallback(build, pc, i, TM_ADD); + break; + case LOP_SUB: + emitInstBinaryFallback(build, pc, i, TM_SUB); + break; + case LOP_MUL: + emitInstBinaryFallback(build, pc, i, TM_MUL); + break; + case LOP_DIV: + emitInstBinaryFallback(build, pc, i, TM_DIV); + break; + case LOP_MOD: + emitInstBinaryFallback(build, pc, i, TM_MOD); + break; + case LOP_POW: + emitInstBinaryFallback(build, pc, i, TM_POW); + break; + case LOP_ADDK: + emitInstBinaryKFallback(build, pc, i, TM_ADD); + break; + case LOP_SUBK: + emitInstBinaryKFallback(build, pc, i, TM_SUB); + break; + case LOP_MULK: + emitInstBinaryKFallback(build, pc, i, TM_MUL); + break; + case LOP_DIVK: + emitInstBinaryKFallback(build, pc, i, TM_DIV); + break; + case LOP_MODK: + emitInstBinaryKFallback(build, pc, i, TM_MOD); + break; + case LOP_POWK: + emitInstBinaryKFallback(build, pc, i, TM_POW); + break; + case LOP_MINUS: + emitInstMinusFallback(build, pc, i); + break; + case LOP_LENGTH: + emitInstLengthFallback(build, pc, i); + break; + case LOP_FORGLOOP: + emitinstForGLoopFallback(build, pc, i, labelarr); + break; + case LOP_FORGPREP_NEXT: + case LOP_FORGPREP_INEXT: + emitInstForGPrepXnextFallback(build, pc, i, labelarr); + break; + case LOP_GETGLOBAL: + // TODO: luaV_gettable + cachedslot update instead of full fallback + emitFallback(build, data, op, i); + break; + case LOP_SETGLOBAL: + // TODO: luaV_settable + cachedslot update instead of full fallback + emitFallback(build, data, op, i); + break; + case LOP_GETTABLEKS: + // Full fallback required for LOP_GETTABLEKS because 'luaV_gettable' doesn't handle builtin vector field access + // It is also required to perform cached slot update + // TODO: extra fast-paths could be lowered before the full fallback + emitFallback(build, data, op, i); + break; + case LOP_SETTABLEKS: + // TODO: luaV_settable + cachedslot update instead of full fallback + emitFallback(build, data, op, i); + break; + default: + LUAU_ASSERT(!"Expected fallback for instruction"); + } +} + +static NativeProto* assembleFunction(AssemblyBuilderX64& build, NativeState& data, ModuleHelpers& helpers, Proto* proto, AssemblyOptions options) { NativeProto* result = new NativeProto(); @@ -59,222 +402,65 @@ static NativeProto* assembleFunction(AssemblyBuilderX64& build, NativeState& dat std::vector(a) -> ()' could not be converted into 't1 where t1 = ({- Clone: t1 -}) -> (a...)'; different number of generic type parameters)", - toString(result.errors[0])); - } - else - { - LUAU_REQUIRE_NO_ERRORS(result); - } + LUAU_REQUIRE_NO_ERRORS(result); } TEST_CASE_FIXTURE(BuiltinsFixture, "custom_require_global") diff --git a/tests/TypeInfer.negations.test.cpp b/tests/TypeInfer.negations.test.cpp new file mode 100644 index 00000000..1035eda4 --- /dev/null +++ b/tests/TypeInfer.negations.test.cpp @@ -0,0 +1,52 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details + +#include "Fixture.h" + +#include "doctest.h" +#include "Luau/Common.h" +#include "ScopedFlags.h" + +using namespace Luau; + +namespace +{ +struct NegationFixture : Fixture +{ + TypeArena arena; + ScopedFastFlag sff[2] { + {"LuauNegatedStringSingletons", true}, + {"LuauSubtypeNormalizer", true}, + }; + + NegationFixture() + { + registerNotType(*this, arena); + } +}; +} + +TEST_SUITE_BEGIN("Negations"); + +TEST_CASE_FIXTURE(NegationFixture, "negated_string_is_a_subtype_of_string") +{ + CheckResult result = check(R"( + function foo(arg: string) end + local a: string & Not<"Hello"> + foo(a) + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(NegationFixture, "string_is_not_a_subtype_of_negated_string") +{ + CheckResult result = check(R"( + function foo(arg: string & Not<"hello">) end + local a: string + foo(a) + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); +} + +TEST_SUITE_END(); diff --git a/tests/TypeInfer.operators.test.cpp b/tests/TypeInfer.operators.test.cpp index e572c87a..b2516f6d 100644 --- a/tests/TypeInfer.operators.test.cpp +++ b/tests/TypeInfer.operators.test.cpp @@ -434,16 +434,17 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "typecheck_unary_minus") { CheckResult result = check(R"( --!strict - local foo = { - value = 10 - } + local foo local mt = {} - setmetatable(foo, mt) mt.__unm = function(val: typeof(foo)): string - return val.value .. "test" + return tostring(val.value) .. "test" end + foo = setmetatable({ + value = 10 + }, mt) + local a = -foo local b = 1+-1 @@ -459,25 +460,32 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "typecheck_unary_minus") CHECK_EQ("string", toString(requireType("a"))); CHECK_EQ("number", toString(requireType("b"))); - GenericError* gen = get(result.errors[0]); - REQUIRE_EQ(gen->message, "Unary operator '-' not supported by type 'bar'"); + if (FFlag::DebugLuauDeferredConstraintResolution) + { + CHECK(toString(result.errors[0]) == "Type '{ value: number }' could not be converted into 'number'"); + } + else + { + GenericError* gen = get(result.errors[0]); + REQUIRE(gen); + REQUIRE_EQ(gen->message, "Unary operator '-' not supported by type 'bar'"); + } } TEST_CASE_FIXTURE(BuiltinsFixture, "typecheck_unary_minus_error") { CheckResult result = check(R"( --!strict - local foo = { - value = 10 - } - local mt = {} - setmetatable(foo, mt) mt.__unm = function(val: boolean): string return "test" end + local foo = setmetatable({ + value = 10 + }, mt) + local a = -foo )"); @@ -494,16 +502,16 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "typecheck_unary_len_error") { CheckResult result = check(R"( --!strict - local foo = { - value = 10 - } local mt = {} - setmetatable(foo, mt) - mt.__len = function(val: any): string + mt.__len = function(val): string return "test" end + local foo = setmetatable({ + value = 10, + }, mt) + local a = #foo )"); diff --git a/tests/TypeInfer.provisional.test.cpp b/tests/TypeInfer.provisional.test.cpp index ccc4d775..8e04c165 100644 --- a/tests/TypeInfer.provisional.test.cpp +++ b/tests/TypeInfer.provisional.test.cpp @@ -624,15 +624,18 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "table_insert_with_a_singleton_argument") CHECK_EQ("{string | string}", toString(requireType("t"))); } -struct NormalizeFixture : Fixture +namespace +{ +struct IsSubtypeFixture : Fixture { bool isSubtype(TypeId a, TypeId b) { return ::Luau::isSubtype(a, b, NotNull{getMainModule()->getModuleScope().get()}, singletonTypes, ice); } }; +} -TEST_CASE_FIXTURE(NormalizeFixture, "intersection_of_functions_of_different_arities") +TEST_CASE_FIXTURE(IsSubtypeFixture, "intersection_of_functions_of_different_arities") { check(R"( type A = (any) -> () @@ -653,7 +656,7 @@ TEST_CASE_FIXTURE(NormalizeFixture, "intersection_of_functions_of_different_arit CHECK("((any) -> ()) & ((any, any) -> ())" == toString(requireType("t"))); } -TEST_CASE_FIXTURE(NormalizeFixture, "functions_with_mismatching_arity") +TEST_CASE_FIXTURE(IsSubtypeFixture, "functions_with_mismatching_arity") { check(R"( local a: (number) -> () @@ -676,7 +679,7 @@ TEST_CASE_FIXTURE(NormalizeFixture, "functions_with_mismatching_arity") CHECK(!isSubtype(b, c)); } -TEST_CASE_FIXTURE(NormalizeFixture, "functions_with_mismatching_arity_but_optional_parameters") +TEST_CASE_FIXTURE(IsSubtypeFixture, "functions_with_mismatching_arity_but_optional_parameters") { /* * (T0..TN) <: (T0..TN, A?) @@ -736,7 +739,7 @@ TEST_CASE_FIXTURE(NormalizeFixture, "functions_with_mismatching_arity_but_option // CHECK(!isSubtype(b, c)); } -TEST_CASE_FIXTURE(NormalizeFixture, "functions_with_mismatching_arity_but_any_is_an_optional_param") +TEST_CASE_FIXTURE(IsSubtypeFixture, "functions_with_mismatching_arity_but_any_is_an_optional_param") { check(R"( local a: (number?) -> () diff --git a/tests/TypeInfer.tables.test.cpp b/tests/TypeInfer.tables.test.cpp index 2a208cce..7de412ff 100644 --- a/tests/TypeInfer.tables.test.cpp +++ b/tests/TypeInfer.tables.test.cpp @@ -1,5 +1,8 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #include "Luau/BuiltinDefinitions.h" +#include "Luau/Common.h" +#include "Luau/Frontend.h" +#include "Luau/ToString.h" #include "Luau/TypeInfer.h" #include "Luau/TypeVar.h" @@ -14,6 +17,7 @@ using namespace Luau; LUAU_FASTFLAG(LuauLowerBoundsCalculation); LUAU_FASTFLAG(DebugLuauDeferredConstraintResolution); LUAU_FASTFLAG(LuauInstantiateInSubtyping) +LUAU_FASTFLAG(LuauSpecialTypesAsterisked) TEST_SUITE_BEGIN("TableTests"); @@ -1957,7 +1961,11 @@ TEST_CASE_FIXTURE(Fixture, "invariant_table_properties_means_instantiating_table local c : string = t.m("hi") )"); - LUAU_REQUIRE_ERRORS(result); + // TODO: test behavior is wrong with LuauInstantiateInSubtyping until we can re-enable the covariant requirement for instantiation in subtyping + if (FFlag::LuauInstantiateInSubtyping) + LUAU_REQUIRE_NO_ERRORS(result); + else + LUAU_REQUIRE_ERRORS(result); } TEST_CASE_FIXTURE(BuiltinsFixture, "table_insert_should_cope_with_optional_properties_in_nonstrict") @@ -3262,11 +3270,13 @@ TEST_CASE_FIXTURE(Fixture, "invariant_table_properties_means_instantiating_table local c : string = t.m("hi") )"); - LUAU_REQUIRE_ERRORS(result); - CHECK_EQ(toString(result.errors[0]), R"(Type 't' could not be converted into '{| m: (number) -> number |}' -caused by: - Property 'm' is not compatible. Type '(a) -> a' could not be converted into '(number) -> number'; different number of generic type parameters)"); - // this error message is not great since the underlying issue is that the context is invariant, + LUAU_REQUIRE_NO_ERRORS(result); + // TODO: test behavior is wrong until we can re-enable the covariant requirement for instantiation in subtyping +// LUAU_REQUIRE_ERRORS(result); +// CHECK_EQ(toString(result.errors[0]), R"(Type 't' could not be converted into '{| m: (number) -> number |}' +// caused by: +// Property 'm' is not compatible. Type '(a) -> a' could not be converted into '(number) -> number'; different number of generic type parameters)"); +// // this error message is not great since the underlying issue is that the context is invariant, // and `(number) -> number` cannot be a subtype of `(a) -> a`. } @@ -3292,4 +3302,43 @@ local g : ({ p : number, q : string }) -> ({ p : number, r : boolean }) = f CHECK_EQ("r", error->properties[0]); } +TEST_CASE_FIXTURE(BuiltinsFixture, "setmetatable_has_a_side_effect") +{ + if (!FFlag::DebugLuauDeferredConstraintResolution) + return; + + CheckResult result = check(R"( + local mt = { + __add = function(x, y) + return 123 + end, + } + + local foo = {} + setmetatable(foo, mt) + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + CHECK(toString(requireType("foo")) == "{ @metatable { __add: (a, b) -> number }, { } }"); +} + +TEST_CASE_FIXTURE(BuiltinsFixture, "tables_should_be_fully_populated") +{ + CheckResult result = check(R"( + local t = { + x = 5 :: NonexistingTypeWhichEndsUpReturningAnErrorType, + y = 5 + } + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + ToStringOptions opts; + opts.exhaustive = true; + if (FFlag::LuauSpecialTypesAsterisked) + CHECK_EQ("{ x: *error-type*, y: number }", toString(requireType("t"), opts)); + else + CHECK_EQ("{ x: , y: number }", toString(requireType("t"), opts)); +} + TEST_SUITE_END(); diff --git a/tests/VisitTypeVar.test.cpp b/tests/VisitTypeVar.test.cpp index 4fba694a..589c3bad 100644 --- a/tests/VisitTypeVar.test.cpp +++ b/tests/VisitTypeVar.test.cpp @@ -22,7 +22,14 @@ TEST_CASE_FIXTURE(Fixture, "throw_when_limit_is_exceeded") TypeId tType = requireType("t"); - CHECK_THROWS_AS(toString(tType), RecursionLimitException); + if (FFlag::LuauIceExceptionInheritanceChange) + { + CHECK_THROWS_AS(toString(tType), RecursionLimitException); + } + else + { + CHECK_THROWS_AS(toString(tType), RecursionLimitException_DEPRECATED); + } } TEST_CASE_FIXTURE(Fixture, "dont_throw_when_limit_is_high_enough") diff --git a/tools/faillist.txt b/tools/faillist.txt index c869e0c4..a4c05b7b 100644 --- a/tools/faillist.txt +++ b/tools/faillist.txt @@ -1,14 +1,14 @@ -AnnotationTests.builtin_types_are_not_exported AnnotationTests.corecursive_types_error_on_tight_loop AnnotationTests.duplicate_type_param_name AnnotationTests.for_loop_counter_annotation_is_checked AnnotationTests.generic_aliases_are_cloned_properly AnnotationTests.instantiation_clone_has_to_follow +AnnotationTests.luau_print_is_not_special_without_the_flag AnnotationTests.occurs_check_on_cyclic_intersection_typevar AnnotationTests.occurs_check_on_cyclic_union_typevar AnnotationTests.too_many_type_params AnnotationTests.two_type_params -AnnotationTests.use_type_required_from_another_file +AnnotationTests.unknown_type_reference_generates_error AstQuery.last_argument_function_call_type AstQuery::getDocumentationSymbolAtPosition.overloaded_fn AutocompleteTest.autocomplete_first_function_arg_expected_type @@ -86,7 +86,6 @@ BuiltinTests.table_pack BuiltinTests.table_pack_reduce BuiltinTests.table_pack_variadic BuiltinTests.tonumber_returns_optional_number_type -BuiltinTests.tonumber_returns_optional_number_type2 DefinitionTests.class_definition_overload_metamethods DefinitionTests.class_definition_string_props DefinitionTests.declaring_generic_functions @@ -96,7 +95,6 @@ FrontendTest.imported_table_modification_2 FrontendTest.it_should_be_safe_to_stringify_errors_when_full_type_graph_is_discarded FrontendTest.nocheck_cycle_used_by_checked FrontendTest.reexport_cyclic_type -FrontendTest.reexport_type_alias FrontendTest.trace_requires_in_nonstrict_mode GenericsTests.apply_type_function_nested_generics1 GenericsTests.apply_type_function_nested_generics2 @@ -105,7 +103,6 @@ GenericsTests.calling_self_generic_methods GenericsTests.check_generic_typepack_function GenericsTests.check_mutual_generic_functions GenericsTests.correctly_instantiate_polymorphic_member_functions -GenericsTests.do_not_always_instantiate_generic_intersection_types GenericsTests.do_not_infer_generic_functions GenericsTests.duplicate_generic_type_packs GenericsTests.duplicate_generic_types @@ -143,7 +140,6 @@ IntersectionTypes.table_write_sealed_indirect ModuleTests.any_persistance_does_not_leak ModuleTests.clone_self_property ModuleTests.deepClone_cyclic_table -ModuleTests.do_not_clone_reexports NonstrictModeTests.for_in_iterator_variables_are_any NonstrictModeTests.function_parameters_are_any NonstrictModeTests.inconsistent_module_return_types_are_ok @@ -158,7 +154,6 @@ NonstrictModeTests.parameters_having_type_any_are_optional NonstrictModeTests.table_dot_insert_and_recursive_calls NonstrictModeTests.table_props_are_any Normalize.cyclic_table_normalizes_sensibly -Normalize.intersection_combine_on_bound_self ParseErrorRecovery.generic_type_list_recovery ParseErrorRecovery.recovery_of_parenthesized_expressions ParserTests.parse_nesting_based_end_detection_failsafe_earlier @@ -249,7 +244,6 @@ TableTests.defining_a_self_method_for_a_builtin_sealed_table_must_fail TableTests.defining_a_self_method_for_a_local_sealed_table_must_fail TableTests.dont_crash_when_setmetatable_does_not_produce_a_metatabletypevar TableTests.dont_hang_when_trying_to_look_up_in_cyclic_metatable_index -TableTests.dont_invalidate_the_properties_iterator_of_free_table_when_rolled_back TableTests.dont_leak_free_table_props TableTests.dont_quantify_table_that_belongs_to_outer_scope TableTests.dont_suggest_exact_match_keys @@ -279,7 +273,6 @@ TableTests.inferring_crazy_table_should_also_be_quick TableTests.instantiate_table_cloning_3 TableTests.invariant_table_properties_means_instantiating_tables_in_call_is_unsound TableTests.leaking_bad_metatable_errors -TableTests.length_operator_union_errors TableTests.less_exponential_blowup_please TableTests.meta_add TableTests.meta_add_both_ways @@ -347,9 +340,9 @@ TryUnifyTests.members_of_failed_typepack_unification_are_unified_with_errorType TryUnifyTests.result_of_failed_typepack_unification_is_constrained TryUnifyTests.typepack_unification_should_trim_free_tails TryUnifyTests.variadics_should_use_reversed_properly +TypeAliases.cannot_create_cyclic_type_with_unknown_module TypeAliases.forward_declared_alias_is_not_clobbered_by_prior_unification_with_any TypeAliases.generic_param_remap -TypeAliases.mismatched_generic_pack_type_param TypeAliases.mismatched_generic_type_param TypeAliases.mutually_recursive_types_restriction_not_ok_1 TypeAliases.mutually_recursive_types_restriction_not_ok_2 @@ -363,7 +356,7 @@ TypeAliases.type_alias_fwd_declaration_is_precise TypeAliases.type_alias_local_mutation TypeAliases.type_alias_local_rename TypeAliases.type_alias_of_an_imported_recursive_generic_type -TypeAliases.type_alias_of_an_imported_recursive_type +TypeInfer.check_type_infer_recursion_count TypeInfer.checking_should_not_ice TypeInfer.cli_50041_committing_txnlog_in_apollo_client_error TypeInfer.dont_report_type_errors_within_an_AstExprError @@ -394,6 +387,7 @@ TypeInferClasses.warn_when_prop_almost_matches TypeInferClasses.we_can_report_when_someone_is_trying_to_use_a_table_rather_than_a_class TypeInferFunctions.calling_function_with_anytypepack_doesnt_leak_free_types TypeInferFunctions.calling_function_with_incorrect_argument_type_yields_errors_spanning_argument +TypeInferFunctions.cannot_hoist_interior_defns_into_signature TypeInferFunctions.dont_give_other_overloads_message_if_only_one_argument_matching_overload_exists TypeInferFunctions.dont_infer_parameter_types_for_functions_from_their_call_site TypeInferFunctions.duplicate_functions_with_different_signatures_not_allowed_in_nonstrict @@ -439,12 +433,9 @@ TypeInferLoops.varlist_declared_by_for_in_loop_should_be_free TypeInferModules.bound_free_table_export_is_ok TypeInferModules.custom_require_global TypeInferModules.do_not_modify_imported_types -TypeInferModules.do_not_modify_imported_types_2 -TypeInferModules.do_not_modify_imported_types_3 TypeInferModules.module_type_conflict TypeInferModules.module_type_conflict_instantiated TypeInferModules.require_a_variadic_function -TypeInferModules.require_types TypeInferModules.type_error_of_unknown_qualified_type TypeInferOOP.CheckMethodsOfSealed TypeInferOOP.dont_suggest_using_colon_rather_than_dot_if_another_overload_works @@ -468,9 +459,6 @@ TypeInferOperators.produce_the_correct_error_message_when_comparing_a_table_with TypeInferOperators.refine_and_or TypeInferOperators.typecheck_overloaded_multiply_that_is_an_intersection TypeInferOperators.typecheck_overloaded_multiply_that_is_an_intersection_on_rhs -TypeInferOperators.typecheck_unary_len_error -TypeInferOperators.typecheck_unary_minus -TypeInferOperators.typecheck_unary_minus_error TypeInferOperators.UnknownGlobalCompoundAssign TypeInferPrimitives.CheckMethodsOfNumber TypeInferPrimitives.singleton_types @@ -489,6 +477,7 @@ TypeInferUnknownNever.math_operators_and_never TypeInferUnknownNever.type_packs_containing_never_is_itself_uninhabitable TypeInferUnknownNever.type_packs_containing_never_is_itself_uninhabitable2 TypeInferUnknownNever.unary_minus_of_never +TypePackTests.detect_cyclic_typepacks2 TypePackTests.higher_order_function TypePackTests.pack_tail_unification_check TypePackTests.parenthesized_varargs_returns_any From e37eb3c7789e37ed450fb9a2beaf722012cbe33b Mon Sep 17 00:00:00 2001 From: boyned//Kampfkarren Date: Fri, 28 Oct 2022 12:22:26 -0700 Subject: [PATCH 386/399] Fix { range to be within the interpolated string (#728) Corrects `{` range to be inside the interpolated string, needed for syntax highlighting. --- Ast/src/Lexer.cpp | 2 +- tests/AstJsonEncoder.test.cpp | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/Ast/src/Lexer.cpp b/Ast/src/Lexer.cpp index d93f2ccb..66436acd 100644 --- a/Ast/src/Lexer.cpp +++ b/Ast/src/Lexer.cpp @@ -641,8 +641,8 @@ Lexeme Lexer::readInterpolatedStringSection(Position start, Lexeme::Type formatT return brokenDoubleBrace; } - Lexeme lexemeOutput(Location(start, position()), Lexeme::InterpStringBegin, &buffer[startOffset], offset - startOffset); consume(); + Lexeme lexemeOutput(Location(start, position()), Lexeme::InterpStringBegin, &buffer[startOffset], offset - startOffset - 1); return lexemeOutput; } diff --git a/tests/AstJsonEncoder.test.cpp b/tests/AstJsonEncoder.test.cpp index a23f6f47..81e74941 100644 --- a/tests/AstJsonEncoder.test.cpp +++ b/tests/AstJsonEncoder.test.cpp @@ -183,7 +183,7 @@ TEST_CASE_FIXTURE(JsonEncoderFixture, "encode_AstExprInterpString") AstStat* statement = expectParseStatement("local a = `var = {x}`"); std::string_view expected = - R"({"type":"AstStatLocal","location":"0,0 - 0,17","vars":[{"luauType":null,"name":"a","type":"AstLocal","location":"0,6 - 0,7"}],"values":[{"type":"AstExprInterpString","location":"0,10 - 0,17","strings":["var = ",""],"expressions":[{"type":"AstExprGlobal","location":"0,18 - 0,19","global":"x"}]}]})"; + R"({"type":"AstStatLocal","location":"0,0 - 0,18","vars":[{"luauType":null,"name":"a","type":"AstLocal","location":"0,6 - 0,7"}],"values":[{"type":"AstExprInterpString","location":"0,10 - 0,18","strings":["var = ",""],"expressions":[{"type":"AstExprGlobal","location":"0,18 - 0,19","global":"x"}]}]})"; CHECK(toJson(statement) == expected); } From e43a9e927c46206d49b1db2ad2f48c852c7672d5 Mon Sep 17 00:00:00 2001 From: Alan Jeffrey <403333+asajeffrey@users.noreply.github.com> Date: Tue, 1 Nov 2022 12:02:03 -0500 Subject: [PATCH 387/399] Added Sep-Oct 2022 Luau Recap (#726) --- ...11-01-luau-recap-september-october-2022.md | 82 +++++++++++++++++++ 1 file changed, 82 insertions(+) create mode 100644 docs/_posts/2022-11-01-luau-recap-september-october-2022.md diff --git a/docs/_posts/2022-11-01-luau-recap-september-october-2022.md b/docs/_posts/2022-11-01-luau-recap-september-october-2022.md new file mode 100644 index 00000000..7d99babb --- /dev/null +++ b/docs/_posts/2022-11-01-luau-recap-september-october-2022.md @@ -0,0 +1,82 @@ +--- +layout: single +title: "Luau Recap: September & October 2022" +--- + +Luau is our new language that you can read more about at [https://luau-lang.org](https://luau-lang.org). + +[Cross-posted to the [Roblox Developer Forum](https://devforum.roblox.com/t/luau-recap-september-october-2022/).] + +## Semantic subtyping + +One of the most important goals for Luau is to avoid *false +positives*, that is cases where Script Analysis reports a type error, +but in fact the code is correct. This is very frustrating, especially +for beginners. Spending time chasing down a gnarly type error only to +discover that it was the type system that's wrong is nobody's idea of fun! + +We are pleased to announce that a major component of minimizing false +positives has landed, *semantic subtyping*, which removes a class of false positives caused +by failures of subtyping. For example, in the program + +```lua + local x : CFrame = CFrame.new() + local y : Vector3 | CFrame + if (math.random()) then + y = CFrame.new() + else + y = Vector3.new() + end + local z : Vector3 | CFrame = x * y -- Type Error! +``` + +an error is reported, even though there is no problem at runtime. This +is because `CFrame`'s multiplication has two overloads: + +```lua + ((CFrame, CFrame) -> CFrame) + & ((CFrame, Vector3) -> Vector3) +``` + +The current syntax-driven algorithm for subtyping is not sophisticated +enough to realize that this is a subtype of the desired type: + +```lua + (CFrame, Vector3 | CFrame) -> (Vector3 | CFrame) +``` + +Our new algorithm is driven by the semantics of subtyping, not the syntax of types, +and eliminates this class of false positives. + +If you want to know more about semantic subtyping in Luau, check out our +[technical blog post](https://luau-lang.org/2022/10/31/luau-semantic-subtyping.html) +on the subject. + +## Other analysis improvements + +* Improve stringification of function types. +* Improve parse error warnings in the case of missing tokens after a comma. +* Improve typechecking of expressions involving variadics such as `{ ... }`. +* Make sure modules don't return unbound generic types. +* Improve cycle detection in stringifying types. +* Improve type inference of combinations of intersections and generic functions. +* Improve typechecking when calling a function which returns a variadic e.g. `() -> (number...)`. +* Improve typechecking when passing a function expression as a parameter to a function. +* Improve error reporting locations. +* Remove some sources of memory corruption and crashes. + +## Other runtime and debugger improvements + +* Improve performance of accessing debug info. +* Improve performance of `getmetatable` and `setmetatable`. +* Remove a source of freezes in the debugger. +* Improve GC accuracy and performance. + +## Thanks + +Thanks for all the contributions! + +* [AllanJeremy](https://github.com/AllanJeremy) +* [JohnnyMorganz](https://github.com/JohnnyMorganz) +* [jujhar16](https://github.com/jujhar16) +* [petrihakkinen](https://github.com/petrihakkinen) From c33700e47390787e186a6bc1ffb0e366d0dba763 Mon Sep 17 00:00:00 2001 From: Andy Friesen Date: Fri, 4 Nov 2022 10:33:22 -0700 Subject: [PATCH 388/399] Sync to upstream/release/552 (#735) * Reduce the stack utilization of type checking. * Improve the error message that's reported when a delimiting comma is missing from a table literal. eg ```lua local t = { first = 1 second = 2 }``` --- Analysis/include/Luau/Connective.h | 68 + Analysis/include/Luau/Constraint.h | 7 +- .../include/Luau/ConstraintGraphBuilder.h | 19 +- Analysis/include/Luau/ConstraintSolver.h | 2 +- Analysis/include/Luau/Normalize.h | 40 +- Analysis/include/Luau/TypeUtils.h | 4 +- Analysis/include/Luau/TypeVar.h | 18 +- Analysis/include/Luau/Unifier.h | 2 +- Analysis/include/Luau/Variant.h | 7 +- Analysis/include/Luau/VisitTypeVar.h | 6 - Analysis/src/BuiltinDefinitions.cpp | 3 +- Analysis/src/Clone.cpp | 7 - Analysis/src/Connective.cpp | 32 + Analysis/src/ConstraintGraphBuilder.cpp | 245 +- Analysis/src/ConstraintSolver.cpp | 29 +- Analysis/src/EmbeddedBuiltinDefinitions.cpp | 74 +- Analysis/src/Normalize.cpp | 227 +- Analysis/src/ToString.cpp | 114 +- Analysis/src/TypeAttach.cpp | 6 - Analysis/src/TypeChecker2.cpp | 31 +- Analysis/src/TypeVar.cpp | 11 +- Analysis/src/Unifier.cpp | 181 +- Ast/src/Parser.cpp | 15 +- CLI/Repl.cpp | 3 +- CodeGen/include/Luau/AddressA64.h | 52 + CodeGen/include/Luau/AssemblyBuilderA64.h | 144 ++ CodeGen/include/Luau/AssemblyBuilderX64.h | 4 +- CodeGen/include/Luau/ConditionA64.h | 37 + .../Luau/{Condition.h => ConditionX64.h} | 2 +- CodeGen/include/Luau/OperandX64.h | 4 +- CodeGen/include/Luau/RegisterA64.h | 105 + CodeGen/include/Luau/RegisterX64.h | 5 + CodeGen/src/AssemblyBuilderA64.cpp | 607 +++++ CodeGen/src/AssemblyBuilderX64.cpp | 21 +- CodeGen/src/CodeGen.cpp | 31 +- CodeGen/src/CodeGenUtils.cpp | 54 + CodeGen/src/CodeGenUtils.h | 3 + CodeGen/src/CodeGenX64.cpp | 4 +- CodeGen/src/EmitCommonX64.cpp | 143 +- CodeGen/src/EmitCommonX64.h | 38 +- CodeGen/src/EmitInstructionX64.cpp | 322 ++- CodeGen/src/EmitInstructionX64.h | 7 +- CodeGen/src/Fallbacks.cpp | 1997 +---------------- CodeGen/src/Fallbacks.h | 69 - CodeGen/src/NativeState.cpp | 5 +- CodeGen/src/NativeState.h | 7 +- Sources.cmake | 10 +- VM/include/luaconf.h | 8 - VM/src/lbuiltins.cpp | 9 +- VM/src/lnumutils.h | 1 - tests/AssemblyBuilderA64.test.cpp | 221 ++ tests/AssemblyBuilderX64.test.cpp | 75 +- tests/CodeAllocator.test.cpp | 54 +- tests/Conformance.test.cpp | 10 +- tests/Fixture.cpp | 3 +- tests/Fixture.h | 2 +- tests/Normalize.test.cpp | 73 +- tests/Parser.test.cpp | 48 + tests/ToString.test.cpp | 30 +- tests/TypeInfer.anyerror.test.cpp | 22 +- tests/TypeInfer.builtins.test.cpp | 10 +- tests/TypeInfer.functions.test.cpp | 92 +- tests/TypeInfer.generics.test.cpp | 6 +- tests/TypeInfer.loops.test.cpp | 6 +- tests/TypeInfer.modules.test.cpp | 12 +- tests/TypeInfer.negations.test.cpp | 6 +- tests/TypeInfer.primitives.test.cpp | 7 +- tests/TypeInfer.provisional.test.cpp | 2 +- tests/TypeInfer.refinements.test.cpp | 222 +- tests/TypeInfer.singletons.test.cpp | 10 + tests/TypeInfer.tables.test.cpp | 17 +- tests/TypeInfer.test.cpp | 24 +- tests/TypeInfer.tryUnify.test.cpp | 12 +- tests/TypeInfer.unionTypes.test.cpp | 7 +- tests/Variant.test.cpp | 31 + tools/faillist.txt | 23 +- tools/lvmexecute_split.py | 9 +- tools/stack-usage-reporter.py | 173 ++ 78 files changed, 3061 insertions(+), 2986 deletions(-) create mode 100644 Analysis/include/Luau/Connective.h create mode 100644 Analysis/src/Connective.cpp create mode 100644 CodeGen/include/Luau/AddressA64.h create mode 100644 CodeGen/include/Luau/AssemblyBuilderA64.h create mode 100644 CodeGen/include/Luau/ConditionA64.h rename CodeGen/include/Luau/{Condition.h => ConditionX64.h} (94%) create mode 100644 CodeGen/include/Luau/RegisterA64.h create mode 100644 CodeGen/src/AssemblyBuilderA64.cpp create mode 100644 tests/AssemblyBuilderA64.test.cpp create mode 100644 tools/stack-usage-reporter.py diff --git a/Analysis/include/Luau/Connective.h b/Analysis/include/Luau/Connective.h new file mode 100644 index 00000000..c9daa0f9 --- /dev/null +++ b/Analysis/include/Luau/Connective.h @@ -0,0 +1,68 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +#pragma once + +#include "Luau/Def.h" +#include "Luau/TypedAllocator.h" +#include "Luau/TypeVar.h" +#include "Luau/Variant.h" + +#include + +namespace Luau +{ + +struct Negation; +struct Conjunction; +struct Disjunction; +struct Equivalence; +struct Proposition; +using Connective = Variant; +using ConnectiveId = Connective*; // Can and most likely is nullptr. + +struct Negation +{ + ConnectiveId connective; +}; + +struct Conjunction +{ + ConnectiveId lhs; + ConnectiveId rhs; +}; + +struct Disjunction +{ + ConnectiveId lhs; + ConnectiveId rhs; +}; + +struct Equivalence +{ + ConnectiveId lhs; + ConnectiveId rhs; +}; + +struct Proposition +{ + DefId def; + TypeId discriminantTy; +}; + +template +const T* get(ConnectiveId connective) +{ + return get_if(connective); +} + +struct ConnectiveArena +{ + TypedAllocator allocator; + + ConnectiveId negation(ConnectiveId connective); + ConnectiveId conjunction(ConnectiveId lhs, ConnectiveId rhs); + ConnectiveId disjunction(ConnectiveId lhs, ConnectiveId rhs); + ConnectiveId equivalence(ConnectiveId lhs, ConnectiveId rhs); + ConnectiveId proposition(DefId def, TypeId discriminantTy); +}; + +} // namespace Luau diff --git a/Analysis/include/Luau/Constraint.h b/Analysis/include/Luau/Constraint.h index 7f092f5b..4370d0cf 100644 --- a/Analysis/include/Luau/Constraint.h +++ b/Analysis/include/Luau/Constraint.h @@ -132,15 +132,16 @@ struct HasPropConstraint std::string prop; }; -struct RefinementConstraint +// result ~ if isSingleton D then ~D else unknown where D = discriminantType +struct SingletonOrTopTypeConstraint { - DefId def; + TypeId resultType; TypeId discriminantType; }; using ConstraintV = Variant; + HasPropConstraint, SingletonOrTopTypeConstraint>; struct Constraint { diff --git a/Analysis/include/Luau/ConstraintGraphBuilder.h b/Analysis/include/Luau/ConstraintGraphBuilder.h index 6106717c..cb5900ea 100644 --- a/Analysis/include/Luau/ConstraintGraphBuilder.h +++ b/Analysis/include/Luau/ConstraintGraphBuilder.h @@ -2,6 +2,7 @@ #pragma once #include "Luau/Ast.h" +#include "Luau/Connective.h" #include "Luau/Constraint.h" #include "Luau/DataFlowGraphBuilder.h" #include "Luau/Module.h" @@ -26,11 +27,13 @@ struct DcrLogger; struct Inference { TypeId ty = nullptr; + ConnectiveId connective = nullptr; Inference() = default; - explicit Inference(TypeId ty) + explicit Inference(TypeId ty, ConnectiveId connective = nullptr) : ty(ty) + , connective(connective) { } }; @@ -38,11 +41,13 @@ struct Inference struct InferencePack { TypePackId tp = nullptr; + std::vector connectives; InferencePack() = default; - explicit InferencePack(TypePackId tp) + explicit InferencePack(TypePackId tp, const std::vector& connectives = {}) : tp(tp) + , connectives(connectives) { } }; @@ -73,6 +78,7 @@ struct ConstraintGraphBuilder // Defining scopes for AST nodes. DenseHashMap astTypeAliasDefiningScopes{nullptr}; NotNull dfg; + ConnectiveArena connectiveArena; int recursionCount = 0; @@ -126,6 +132,8 @@ struct ConstraintGraphBuilder */ NotNull addConstraint(const ScopePtr& scope, std::unique_ptr c); + void applyRefinements(const ScopePtr& scope, Location location, ConnectiveId connective); + /** * The entry point to the ConstraintGraphBuilder. This will construct a set * of scopes, constraints, and free types that can be solved later. @@ -167,10 +175,10 @@ struct ConstraintGraphBuilder * surrounding context. Used to implement bidirectional type checking. * @return the type of the expression. */ - Inference check(const ScopePtr& scope, AstExpr* expr, std::optional expectedType = {}); + Inference check(const ScopePtr& scope, AstExpr* expr, std::optional expectedType = {}, bool forceSingleton = false); - Inference check(const ScopePtr& scope, AstExprConstantString* string, std::optional expectedType); - Inference check(const ScopePtr& scope, AstExprConstantBool* bool_, std::optional expectedType); + Inference check(const ScopePtr& scope, AstExprConstantString* string, std::optional expectedType, bool forceSingleton); + Inference check(const ScopePtr& scope, AstExprConstantBool* bool_, std::optional expectedType, bool forceSingleton); Inference check(const ScopePtr& scope, AstExprLocal* local); Inference check(const ScopePtr& scope, AstExprGlobal* global); Inference check(const ScopePtr& scope, AstExprIndexName* indexName); @@ -180,6 +188,7 @@ struct ConstraintGraphBuilder Inference check(const ScopePtr& scope, AstExprIfElse* ifElse, std::optional expectedType); Inference check(const ScopePtr& scope, AstExprTypeAssertion* typeAssert); Inference check(const ScopePtr& scope, AstExprTable* expr, std::optional expectedType); + std::tuple checkBinary(const ScopePtr& scope, AstExprBinary* binary, std::optional expectedType); TypePackId checkLValues(const ScopePtr& scope, AstArray exprs); diff --git a/Analysis/include/Luau/ConstraintSolver.h b/Analysis/include/Luau/ConstraintSolver.h index 5cc63e65..07f027ad 100644 --- a/Analysis/include/Luau/ConstraintSolver.h +++ b/Analysis/include/Luau/ConstraintSolver.h @@ -110,7 +110,7 @@ struct ConstraintSolver bool tryDispatch(const FunctionCallConstraint& c, NotNull constraint); bool tryDispatch(const PrimitiveTypeConstraint& c, NotNull constraint); bool tryDispatch(const HasPropConstraint& c, NotNull constraint); - bool tryDispatch(const RefinementConstraint& c, NotNull constraint); + bool tryDispatch(const SingletonOrTopTypeConstraint& c, NotNull constraint); // for a, ... in some_table do // also handles __iter metamethod diff --git a/Analysis/include/Luau/Normalize.h b/Analysis/include/Luau/Normalize.h index f98442dd..b28c06a5 100644 --- a/Analysis/include/Luau/Normalize.h +++ b/Analysis/include/Luau/Normalize.h @@ -17,10 +17,8 @@ struct SingletonTypes; using ModulePtr = std::shared_ptr; -bool isSubtype( - TypeId subTy, TypeId superTy, NotNull scope, NotNull singletonTypes, InternalErrorReporter& ice, bool anyIsTop = true); -bool isSubtype(TypePackId subTy, TypePackId superTy, NotNull scope, NotNull singletonTypes, InternalErrorReporter& ice, - bool anyIsTop = true); +bool isSubtype(TypeId subTy, TypeId superTy, NotNull scope, NotNull singletonTypes, InternalErrorReporter& ice); +bool isSubtype(TypePackId subTy, TypePackId superTy, NotNull scope, NotNull singletonTypes, InternalErrorReporter& ice); class TypeIds { @@ -169,12 +167,26 @@ struct NormalizedStringType bool isSubtype(const NormalizedStringType& subStr, const NormalizedStringType& superStr); -// A normalized function type is either `never` (represented by `nullopt`) +// A normalized function type can be `never`, the top function type `function`, // or an intersection of function types. -// NOTE: type normalization can fail on function types with generics -// (e.g. because we do not support unions and intersections of generic type packs), -// so this type may contain `error`. -using NormalizedFunctionType = std::optional; +// +// NOTE: type normalization can fail on function types with generics (e.g. +// because we do not support unions and intersections of generic type packs), so +// this type may contain `error`. +struct NormalizedFunctionType +{ + NormalizedFunctionType(); + + bool isTop = false; + // TODO: Remove this wrapping optional when clipping + // FFlagLuauNegatedFunctionTypes. + std::optional parts; + + void resetToNever(); + void resetToTop(); + + bool isNever() const; +}; // A normalized generic/free type is a union, where each option is of the form (X & T) where // * X is either a free type or a generic @@ -234,12 +246,14 @@ struct NormalizedType NormalizedType(NotNull singletonTypes); - NormalizedType(const NormalizedType&) = delete; - NormalizedType(NormalizedType&&) = default; NormalizedType() = delete; ~NormalizedType() = default; + + NormalizedType(const NormalizedType&) = delete; + NormalizedType& operator=(const NormalizedType&) = delete; + + NormalizedType(NormalizedType&&) = default; NormalizedType& operator=(NormalizedType&&) = default; - NormalizedType& operator=(NormalizedType&) = delete; }; class Normalizer @@ -291,7 +305,7 @@ public: bool unionNormalWithTy(NormalizedType& here, TypeId there, int ignoreSmallerTyvars = -1); // ------- Negations - NormalizedType negateNormal(const NormalizedType& here); + std::optional negateNormal(const NormalizedType& here); TypeIds negateAll(const TypeIds& theres); TypeId negate(TypeId there); void subtractPrimitive(NormalizedType& here, TypeId ty); diff --git a/Analysis/include/Luau/TypeUtils.h b/Analysis/include/Luau/TypeUtils.h index 7409dbe7..085ee21b 100644 --- a/Analysis/include/Luau/TypeUtils.h +++ b/Analysis/include/Luau/TypeUtils.h @@ -35,7 +35,7 @@ std::vector flatten(TypeArena& arena, NotNull singletonT * identity) types. * @param types the input type list to reduce. * @returns the reduced type list. -*/ + */ std::vector reduceUnion(const std::vector& types); /** @@ -45,7 +45,7 @@ std::vector reduceUnion(const std::vector& types); * @param arena the type arena to allocate the new type in, if necessary * @param ty the type to remove nil from * @returns a type with nil removed, or nil itself if that were the only option. -*/ + */ TypeId stripNil(NotNull singletonTypes, TypeArena& arena, TypeId ty); } // namespace Luau diff --git a/Analysis/include/Luau/TypeVar.h b/Analysis/include/Luau/TypeVar.h index 70c12cb9..0ab4d474 100644 --- a/Analysis/include/Luau/TypeVar.h +++ b/Analysis/include/Luau/TypeVar.h @@ -115,6 +115,7 @@ struct PrimitiveTypeVar Number, String, Thread, + Function, }; Type type; @@ -504,14 +505,6 @@ struct NeverTypeVar { }; -// Invariant 1: there should never be a reason why such UseTypeVar exists without it mapping to another type. -// Invariant 2: UseTypeVar should always disappear across modules. -struct UseTypeVar -{ - DefId def; - NotNull scope; -}; - // ~T // TODO: Some simplification step that overwrites the type graph to make sure negation // types disappear from the user's view, and (?) a debug flag to disable that @@ -522,9 +515,9 @@ struct NegationTypeVar using ErrorTypeVar = Unifiable::Error; -using TypeVariant = Unifiable::Variant; +using TypeVariant = + Unifiable::Variant; struct TypeVar final { @@ -644,13 +637,14 @@ public: const TypeId stringType; const TypeId booleanType; const TypeId threadType; + const TypeId functionType; const TypeId trueType; const TypeId falseType; const TypeId anyType; const TypeId unknownType; const TypeId neverType; const TypeId errorType; - const TypeId falsyType; // No type binding! + const TypeId falsyType; // No type binding! const TypeId truthyType; // No type binding! const TypePackId anyTypePack; diff --git a/Analysis/include/Luau/Unifier.h b/Analysis/include/Luau/Unifier.h index 7bf4d50b..b5f58d3c 100644 --- a/Analysis/include/Luau/Unifier.h +++ b/Analysis/include/Luau/Unifier.h @@ -61,7 +61,6 @@ struct Unifier ErrorVec errors; Location location; Variance variance = Covariant; - bool anyIsTop = false; // If true, we consider any to be a top type. If false, it is a familiar but weird mix of top and bottom all at once. bool normalize; // Normalize unions and intersections if necessary bool useScopes = false; // If true, we use the scope hierarchy rather than TypeLevels CountMismatch::Context ctx = CountMismatch::Arg; @@ -131,6 +130,7 @@ public: Unifier makeChildUnifier(); void reportError(TypeError err); + LUAU_NOINLINE void reportError(Location location, TypeErrorData data); private: bool isNonstrictMode() const; diff --git a/Analysis/include/Luau/Variant.h b/Analysis/include/Luau/Variant.h index 76812c9b..016c51f6 100644 --- a/Analysis/include/Luau/Variant.h +++ b/Analysis/include/Luau/Variant.h @@ -58,13 +58,15 @@ public: constexpr int tid = getTypeId(); typeId = tid; - new (&storage) TT(value); + new (&storage) TT(std::forward(value)); } Variant(const Variant& other) { + static constexpr FnCopy table[sizeof...(Ts)] = {&fnCopy...}; + typeId = other.typeId; - tableCopy[typeId](&storage, &other.storage); + table[typeId](&storage, &other.storage); } Variant(Variant&& other) @@ -192,7 +194,6 @@ private: return *static_cast(lhs) == *static_cast(rhs); } - static constexpr FnCopy tableCopy[sizeof...(Ts)] = {&fnCopy...}; static constexpr FnMove tableMove[sizeof...(Ts)] = {&fnMove...}; static constexpr FnDtor tableDtor[sizeof...(Ts)] = {&fnDtor...}; diff --git a/Analysis/include/Luau/VisitTypeVar.h b/Analysis/include/Luau/VisitTypeVar.h index d4f8528f..3dcddba1 100644 --- a/Analysis/include/Luau/VisitTypeVar.h +++ b/Analysis/include/Luau/VisitTypeVar.h @@ -155,10 +155,6 @@ struct GenericTypeVarVisitor { return visit(ty); } - virtual bool visit(TypeId ty, const UseTypeVar& utv) - { - return visit(ty); - } virtual bool visit(TypeId ty, const NegationTypeVar& ntv) { return visit(ty); @@ -321,8 +317,6 @@ struct GenericTypeVarVisitor traverse(a); } } - else if (auto utv = get(ty)) - visit(ty, *utv); else if (auto ntv = get(ty)) visit(ty, *ntv); else if (!FFlag::LuauCompleteVisitor) diff --git a/Analysis/src/BuiltinDefinitions.cpp b/Analysis/src/BuiltinDefinitions.cpp index 6051e117..ee53ae6b 100644 --- a/Analysis/src/BuiltinDefinitions.cpp +++ b/Analysis/src/BuiltinDefinitions.cpp @@ -714,8 +714,7 @@ static bool dcrMagicFunctionPack(MagicFunctionCallContext context) result = arena->addType(UnionTypeVar{std::move(options)}); TypeId numberType = context.solver->singletonTypes->numberType; - TypeId packedTable = arena->addType( - TableTypeVar{{{"n", {numberType}}}, TableIndexer(numberType, result), {}, TableState::Sealed}); + TypeId packedTable = arena->addType(TableTypeVar{{{"n", {numberType}}}, TableIndexer(numberType, result), {}, TableState::Sealed}); TypePackId tableTypePack = arena->addTypePack({packedTable}); asMutable(context.result)->ty.emplace(tableTypePack); diff --git a/Analysis/src/Clone.cpp b/Analysis/src/Clone.cpp index 85408919..86e1c7fc 100644 --- a/Analysis/src/Clone.cpp +++ b/Analysis/src/Clone.cpp @@ -62,7 +62,6 @@ struct TypeCloner void operator()(const LazyTypeVar& t); void operator()(const UnknownTypeVar& t); void operator()(const NeverTypeVar& t); - void operator()(const UseTypeVar& t); void operator()(const NegationTypeVar& t); }; @@ -338,12 +337,6 @@ void TypeCloner::operator()(const NeverTypeVar& t) defaultClone(t); } -void TypeCloner::operator()(const UseTypeVar& t) -{ - TypeId result = dest.addType(BoundTypeVar{follow(typeId)}); - seenTypes[typeId] = result; -} - void TypeCloner::operator()(const NegationTypeVar& t) { TypeId result = dest.addType(AnyTypeVar{}); diff --git a/Analysis/src/Connective.cpp b/Analysis/src/Connective.cpp new file mode 100644 index 00000000..114b5f2f --- /dev/null +++ b/Analysis/src/Connective.cpp @@ -0,0 +1,32 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +#include "Luau/Connective.h" + +namespace Luau +{ + +ConnectiveId ConnectiveArena::negation(ConnectiveId connective) +{ + return NotNull{allocator.allocate(Negation{connective})}; +} + +ConnectiveId ConnectiveArena::conjunction(ConnectiveId lhs, ConnectiveId rhs) +{ + return NotNull{allocator.allocate(Conjunction{lhs, rhs})}; +} + +ConnectiveId ConnectiveArena::disjunction(ConnectiveId lhs, ConnectiveId rhs) +{ + return NotNull{allocator.allocate(Disjunction{lhs, rhs})}; +} + +ConnectiveId ConnectiveArena::equivalence(ConnectiveId lhs, ConnectiveId rhs) +{ + return NotNull{allocator.allocate(Equivalence{lhs, rhs})}; +} + +ConnectiveId ConnectiveArena::proposition(DefId def, TypeId discriminantTy) +{ + return NotNull{allocator.allocate(Proposition{def, discriminantTy})}; +} + +} // namespace Luau diff --git a/Analysis/src/ConstraintGraphBuilder.cpp b/Analysis/src/ConstraintGraphBuilder.cpp index 455fc221..79a69ca4 100644 --- a/Analysis/src/ConstraintGraphBuilder.cpp +++ b/Analysis/src/ConstraintGraphBuilder.cpp @@ -107,6 +107,101 @@ NotNull ConstraintGraphBuilder::addConstraint(const ScopePtr& scope, return NotNull{scope->constraints.emplace_back(std::move(c)).get()}; } +static void unionRefinements(const std::unordered_map& lhs, const std::unordered_map& rhs, + std::unordered_map& dest, NotNull arena) +{ + for (auto [def, ty] : lhs) + { + auto rhsIt = rhs.find(def); + if (rhsIt == rhs.end()) + continue; + + std::vector discriminants{{ty, rhsIt->second}}; + + if (auto destIt = dest.find(def); destIt != dest.end()) + discriminants.push_back(destIt->second); + + dest[def] = arena->addType(UnionTypeVar{std::move(discriminants)}); + } +} + +static void computeRefinement(const ScopePtr& scope, ConnectiveId connective, std::unordered_map* refis, bool sense, + NotNull arena, bool eq, std::vector* constraints) +{ + using RefinementMap = std::unordered_map; + + if (!connective) + return; + else if (auto negation = get(connective)) + return computeRefinement(scope, negation->connective, refis, !sense, arena, eq, constraints); + else if (auto conjunction = get(connective)) + { + RefinementMap lhsRefis; + RefinementMap rhsRefis; + + computeRefinement(scope, conjunction->lhs, sense ? refis : &lhsRefis, sense, arena, eq, constraints); + computeRefinement(scope, conjunction->rhs, sense ? refis : &rhsRefis, sense, arena, eq, constraints); + + if (!sense) + unionRefinements(lhsRefis, rhsRefis, *refis, arena); + } + else if (auto disjunction = get(connective)) + { + RefinementMap lhsRefis; + RefinementMap rhsRefis; + + computeRefinement(scope, disjunction->lhs, sense ? &lhsRefis : refis, sense, arena, eq, constraints); + computeRefinement(scope, disjunction->rhs, sense ? &rhsRefis : refis, sense, arena, eq, constraints); + + if (sense) + unionRefinements(lhsRefis, rhsRefis, *refis, arena); + } + else if (auto equivalence = get(connective)) + { + computeRefinement(scope, equivalence->lhs, refis, sense, arena, true, constraints); + computeRefinement(scope, equivalence->rhs, refis, sense, arena, true, constraints); + } + else if (auto proposition = get(connective)) + { + TypeId discriminantTy = proposition->discriminantTy; + if (!sense && !eq) + discriminantTy = arena->addType(NegationTypeVar{proposition->discriminantTy}); + else if (!sense && eq) + { + discriminantTy = arena->addType(BlockedTypeVar{}); + constraints->push_back(SingletonOrTopTypeConstraint{discriminantTy, proposition->discriminantTy}); + } + + if (auto it = refis->find(proposition->def); it != refis->end()) + (*refis)[proposition->def] = arena->addType(IntersectionTypeVar{{discriminantTy, it->second}}); + else + (*refis)[proposition->def] = discriminantTy; + } +} + +void ConstraintGraphBuilder::applyRefinements(const ScopePtr& scope, Location location, ConnectiveId connective) +{ + if (!connective) + return; + + std::unordered_map refinements; + std::vector constraints; + computeRefinement(scope, connective, &refinements, /*sense*/ true, arena, /*eq*/ false, &constraints); + + for (auto [def, discriminantTy] : refinements) + { + std::optional defTy = scope->lookup(def); + if (!defTy) + ice->ice("Every DefId must map to a type!"); + + TypeId resultTy = arena->addType(IntersectionTypeVar{{*defTy, discriminantTy}}); + scope->dcrRefinements[def] = resultTy; + } + + for (auto& c : constraints) + addConstraint(scope, location, c); +} + void ConstraintGraphBuilder::visit(AstStatBlock* block) { LUAU_ASSERT(scopes.empty()); @@ -250,14 +345,33 @@ void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatLocal* local) if (value->is()) { - // HACK: we leave nil-initialized things floating under the assumption that they will later be populated. - // See the test TypeInfer/infer_locals_with_nil_value. - // Better flow awareness should make this obsolete. + // HACK: we leave nil-initialized things floating under the + // assumption that they will later be populated. + // + // See the test TypeInfer/infer_locals_with_nil_value. Better flow + // awareness should make this obsolete. if (!varTypes[i]) varTypes[i] = freshType(scope); } - else if (i == local->values.size - 1) + // Only function calls and vararg expressions can produce packs. All + // other expressions produce exactly one value. + else if (i != local->values.size - 1 || (!value->is() && !value->is())) + { + std::optional expectedType; + if (hasAnnotation) + expectedType = varTypes.at(i); + + TypeId exprType = check(scope, value, expectedType).ty; + if (i < varTypes.size()) + { + if (varTypes[i]) + addConstraint(scope, local->location, SubtypeConstraint{exprType, varTypes[i]}); + else + varTypes[i] = exprType; + } + } + else { std::vector expectedTypes; if (hasAnnotation) @@ -286,21 +400,6 @@ void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatLocal* local) addConstraint(scope, local->location, PackSubtypeConstraint{exprPack, tailPack}); } } - else - { - std::optional expectedType; - if (hasAnnotation) - expectedType = varTypes.at(i); - - TypeId exprType = check(scope, value, expectedType).ty; - if (i < varTypes.size()) - { - if (varTypes[i]) - addConstraint(scope, local->location, SubtypeConstraint{varTypes[i], exprType}); - else - varTypes[i] = exprType; - } - } } for (size_t i = 0; i < local->vars.size; ++i) @@ -569,14 +668,16 @@ void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatIf* ifStatement // TODO: Optimization opportunity, the interior scope of the condition could be // reused for the then body, so we don't need to refine twice. ScopePtr condScope = childScope(ifStatement->condition, scope); - check(condScope, ifStatement->condition, std::nullopt); + auto [_, connective] = check(condScope, ifStatement->condition, std::nullopt); ScopePtr thenScope = childScope(ifStatement->thenbody, scope); + applyRefinements(thenScope, Location{}, connective); visit(thenScope, ifStatement->thenbody); if (ifStatement->elsebody) { ScopePtr elseScope = childScope(ifStatement->elsebody, scope); + applyRefinements(elseScope, Location{}, connectiveArena.negation(connective)); visit(elseScope, ifStatement->elsebody); } } @@ -925,7 +1026,7 @@ InferencePack ConstraintGraphBuilder::checkPack(const ScopePtr& scope, AstExprCa } } -Inference ConstraintGraphBuilder::check(const ScopePtr& scope, AstExpr* expr, std::optional expectedType) +Inference ConstraintGraphBuilder::check(const ScopePtr& scope, AstExpr* expr, std::optional expectedType, bool forceSingleton) { RecursionCounter counter{&recursionCount}; @@ -938,13 +1039,13 @@ Inference ConstraintGraphBuilder::check(const ScopePtr& scope, AstExpr* expr, st Inference result; if (auto group = expr->as()) - result = check(scope, group->expr, expectedType); + result = check(scope, group->expr, expectedType, forceSingleton); else if (auto stringExpr = expr->as()) - result = check(scope, stringExpr, expectedType); + result = check(scope, stringExpr, expectedType, forceSingleton); else if (expr->is()) result = Inference{singletonTypes->numberType}; else if (auto boolExpr = expr->as()) - result = check(scope, boolExpr, expectedType); + result = check(scope, boolExpr, expectedType, forceSingleton); else if (expr->is()) result = Inference{singletonTypes->nilType}; else if (auto local = expr->as()) @@ -999,8 +1100,11 @@ Inference ConstraintGraphBuilder::check(const ScopePtr& scope, AstExpr* expr, st return result; } -Inference ConstraintGraphBuilder::check(const ScopePtr& scope, AstExprConstantString* string, std::optional expectedType) +Inference ConstraintGraphBuilder::check(const ScopePtr& scope, AstExprConstantString* string, std::optional expectedType, bool forceSingleton) { + if (forceSingleton) + return Inference{arena->addType(SingletonTypeVar{StringSingleton{std::string{string->value.data, string->value.size}}})}; + if (expectedType) { const TypeId expectedTy = follow(*expectedType); @@ -1020,12 +1124,15 @@ Inference ConstraintGraphBuilder::check(const ScopePtr& scope, AstExprConstantSt return Inference{singletonTypes->stringType}; } -Inference ConstraintGraphBuilder::check(const ScopePtr& scope, AstExprConstantBool* boolExpr, std::optional expectedType) +Inference ConstraintGraphBuilder::check(const ScopePtr& scope, AstExprConstantBool* boolExpr, std::optional expectedType, bool forceSingleton) { + const TypeId singletonType = boolExpr->value ? singletonTypes->trueType : singletonTypes->falseType; + if (forceSingleton) + return Inference{singletonType}; + if (expectedType) { const TypeId expectedTy = follow(*expectedType); - const TypeId singletonType = boolExpr->value ? singletonTypes->trueType : singletonTypes->falseType; if (get(expectedTy) || get(expectedTy)) { @@ -1045,8 +1152,8 @@ Inference ConstraintGraphBuilder::check(const ScopePtr& scope, AstExprConstantBo Inference ConstraintGraphBuilder::check(const ScopePtr& scope, AstExprLocal* local) { std::optional resultTy; - - if (auto def = dfg->getDef(local)) + auto def = dfg->getDef(local); + if (def) resultTy = scope->lookup(*def); if (!resultTy) @@ -1058,7 +1165,10 @@ Inference ConstraintGraphBuilder::check(const ScopePtr& scope, AstExprLocal* loc if (!resultTy) return Inference{singletonTypes->errorRecoveryType()}; // TODO: replace with ice, locals should never exist before its definition. - return Inference{*resultTy}; + if (def) + return Inference{*resultTy, connectiveArena.proposition(*def, singletonTypes->truthyType)}; + else + return Inference{*resultTy}; } Inference ConstraintGraphBuilder::check(const ScopePtr& scope, AstExprGlobal* global) @@ -1107,20 +1217,23 @@ Inference ConstraintGraphBuilder::check(const ScopePtr& scope, AstExprIndexExpr* Inference ConstraintGraphBuilder::check(const ScopePtr& scope, AstExprUnary* unary) { - TypeId operandType = check(scope, unary->expr).ty; + auto [operandType, connective] = check(scope, unary->expr); TypeId resultType = arena->addType(BlockedTypeVar{}); addConstraint(scope, unary->location, UnaryConstraint{unary->op, operandType, resultType}); - return Inference{resultType}; + + if (unary->op == AstExprUnary::Not) + return Inference{resultType, connectiveArena.negation(connective)}; + else + return Inference{resultType}; } Inference ConstraintGraphBuilder::check(const ScopePtr& scope, AstExprBinary* binary, std::optional expectedType) { - TypeId leftType = check(scope, binary->left, expectedType).ty; - TypeId rightType = check(scope, binary->right, expectedType).ty; + auto [leftType, rightType, connective] = checkBinary(scope, binary, expectedType); TypeId resultType = arena->addType(BlockedTypeVar{}); addConstraint(scope, binary->location, BinaryConstraint{binary->op, leftType, rightType, resultType}); - return Inference{resultType}; + return Inference{resultType, std::move(connective)}; } Inference ConstraintGraphBuilder::check(const ScopePtr& scope, AstExprIfElse* ifElse, std::optional expectedType) @@ -1147,6 +1260,58 @@ Inference ConstraintGraphBuilder::check(const ScopePtr& scope, AstExprTypeAssert return Inference{resolveType(scope, typeAssert->annotation)}; } +std::tuple ConstraintGraphBuilder::checkBinary( + const ScopePtr& scope, AstExprBinary* binary, std::optional expectedType) +{ + if (binary->op == AstExprBinary::And) + { + auto [leftType, leftConnective] = check(scope, binary->left, expectedType); + + ScopePtr rightScope = childScope(binary->right, scope); + applyRefinements(rightScope, binary->right->location, leftConnective); + auto [rightType, rightConnective] = check(rightScope, binary->right, expectedType); + + return {leftType, rightType, connectiveArena.conjunction(leftConnective, rightConnective)}; + } + else if (binary->op == AstExprBinary::Or) + { + auto [leftType, leftConnective] = check(scope, binary->left, expectedType); + + ScopePtr rightScope = childScope(binary->right, scope); + applyRefinements(rightScope, binary->right->location, connectiveArena.negation(leftConnective)); + auto [rightType, rightConnective] = check(rightScope, binary->right, expectedType); + + return {leftType, rightType, connectiveArena.disjunction(leftConnective, rightConnective)}; + } + else if (binary->op == AstExprBinary::CompareEq || binary->op == AstExprBinary::CompareNe) + { + TypeId leftType = check(scope, binary->left, expectedType, true).ty; + TypeId rightType = check(scope, binary->right, expectedType, true).ty; + + ConnectiveId leftConnective = nullptr; + if (auto def = dfg->getDef(binary->left)) + leftConnective = connectiveArena.proposition(*def, rightType); + + ConnectiveId rightConnective = nullptr; + if (auto def = dfg->getDef(binary->right)) + rightConnective = connectiveArena.proposition(*def, leftType); + + if (binary->op == AstExprBinary::CompareNe) + { + leftConnective = connectiveArena.negation(leftConnective); + rightConnective = connectiveArena.negation(rightConnective); + } + + return {leftType, rightType, connectiveArena.equivalence(leftConnective, rightConnective)}; + } + else + { + TypeId leftType = check(scope, binary->left, expectedType).ty; + TypeId rightType = check(scope, binary->right, expectedType).ty; + return {leftType, rightType, nullptr}; + } +} + TypePackId ConstraintGraphBuilder::checkLValues(const ScopePtr& scope, AstArray exprs) { std::vector types; @@ -1841,9 +2006,13 @@ std::vector> ConstraintGraphBuilder:: Inference ConstraintGraphBuilder::flattenPack(const ScopePtr& scope, Location location, InferencePack pack) { - auto [tp] = pack; + const auto& [tp, connectives] = pack; + ConnectiveId connective = nullptr; + if (!connectives.empty()) + connective = connectives[0]; + if (auto f = first(tp)) - return Inference{*f}; + return Inference{*f, connective}; TypeId typeResult = freshType(scope); TypePack onePack{{typeResult}, freshTypePack(scope)}; @@ -1851,7 +2020,7 @@ Inference ConstraintGraphBuilder::flattenPack(const ScopePtr& scope, Location lo addConstraint(scope, location, PackSubtypeConstraint{tp, oneTypePack}); - return Inference{typeResult}; + return Inference{typeResult, connective}; } void ConstraintGraphBuilder::reportError(Location location, TypeErrorData err) diff --git a/Analysis/src/ConstraintSolver.cpp b/Analysis/src/ConstraintSolver.cpp index 5e43be0f..c53ac659 100644 --- a/Analysis/src/ConstraintSolver.cpp +++ b/Analysis/src/ConstraintSolver.cpp @@ -440,8 +440,8 @@ bool ConstraintSolver::tryDispatch(NotNull constraint, bool fo success = tryDispatch(*fcc, constraint); else if (auto hpc = get(*constraint)) success = tryDispatch(*hpc, constraint); - else if (auto rc = get(*constraint)) - success = tryDispatch(*rc, constraint); + else if (auto sottc = get(*constraint)) + success = tryDispatch(*sottc, constraint); else LUAU_ASSERT(false); @@ -1274,25 +1274,18 @@ bool ConstraintSolver::tryDispatch(const HasPropConstraint& c, NotNull constraint) +bool ConstraintSolver::tryDispatch(const SingletonOrTopTypeConstraint& c, NotNull constraint) { - // TODO: Figure out exact details on when refinements need to be blocked. - // It's possible that it never needs to be, since we can just use intersection types with the discriminant type? + if (isBlocked(c.discriminantType)) + return false; - if (!constraint->scope->parent) - iceReporter.ice("No parent scope"); + TypeId followed = follow(c.discriminantType); - std::optional previousTy = constraint->scope->parent->lookup(c.def); - if (!previousTy) - iceReporter.ice("No previous type"); - - std::optional useTy = constraint->scope->lookup(c.def); - if (!useTy) - iceReporter.ice("The def is not bound to a type"); - - TypeId resultTy = follow(*useTy); - std::vector parts{*previousTy, c.discriminantType}; - asMutable(resultTy)->ty.emplace(std::move(parts)); + // `nil` is a singleton type too! There's only one value of type `nil`. + if (get(followed) || isNil(followed)) + *asMutable(c.resultType) = NegationTypeVar{c.discriminantType}; + else + *asMutable(c.resultType) = BoundTypeVar{singletonTypes->unknownType}; return true; } diff --git a/Analysis/src/EmbeddedBuiltinDefinitions.cpp b/Analysis/src/EmbeddedBuiltinDefinitions.cpp index 67abbff1..339de975 100644 --- a/Analysis/src/EmbeddedBuiltinDefinitions.cpp +++ b/Analysis/src/EmbeddedBuiltinDefinitions.cpp @@ -13,16 +13,16 @@ declare bit32: { bor: (...number) -> number, bxor: (...number) -> number, btest: (number, ...number) -> boolean, - rrotate: (number, number) -> number, - lrotate: (number, number) -> number, - lshift: (number, number) -> number, - arshift: (number, number) -> number, - rshift: (number, number) -> number, - bnot: (number) -> number, - extract: (number, number, number?) -> number, - replace: (number, number, number, number?) -> number, - countlz: (number) -> number, - countrz: (number) -> number, + rrotate: (x: number, disp: number) -> number, + lrotate: (x: number, disp: number) -> number, + lshift: (x: number, disp: number) -> number, + arshift: (x: number, disp: number) -> number, + rshift: (x: number, disp: number) -> number, + bnot: (x: number) -> number, + extract: (n: number, field: number, width: number?) -> number, + replace: (n: number, v: number, field: number, width: number?) -> number, + countlz: (n: number) -> number, + countrz: (n: number) -> number, } declare math: { @@ -93,9 +93,9 @@ type DateTypeResult = { } declare os: { - time: (DateTypeArg?) -> number, - date: (string?, number?) -> DateTypeResult | string, - difftime: (DateTypeResult | number, DateTypeResult | number) -> number, + time: (time: DateTypeArg?) -> number, + date: (formatString: string?, time: number?) -> DateTypeResult | string, + difftime: (t2: DateTypeResult | number, t1: DateTypeResult | number) -> number, clock: () -> number, } @@ -145,51 +145,51 @@ declare function loadstring(src: string, chunkname: string?): (((A...) -> declare function newproxy(mt: boolean?): any declare coroutine: { - create: ((A...) -> R...) -> thread, - resume: (thread, A...) -> (boolean, R...), + create: (f: (A...) -> R...) -> thread, + resume: (co: thread, A...) -> (boolean, R...), running: () -> thread, - status: (thread) -> "dead" | "running" | "normal" | "suspended", + status: (co: thread) -> "dead" | "running" | "normal" | "suspended", -- FIXME: This technically returns a function, but we can't represent this yet. - wrap: ((A...) -> R...) -> any, + wrap: (f: (A...) -> R...) -> any, yield: (A...) -> R..., isyieldable: () -> boolean, - close: (thread) -> (boolean, any) + close: (co: thread) -> (boolean, any) } declare table: { - concat: ({V}, string?, number?, number?) -> string, - insert: (({V}, V) -> ()) & (({V}, number, V) -> ()), - maxn: ({V}) -> number, - remove: ({V}, number?) -> V?, - sort: ({V}, ((V, V) -> boolean)?) -> (), - create: (number, V?) -> {V}, - find: ({V}, V, number?) -> number?, + concat: (t: {V}, sep: string?, i: number?, j: number?) -> string, + insert: ((t: {V}, value: V) -> ()) & ((t: {V}, pos: number, value: V) -> ()), + maxn: (t: {V}) -> number, + remove: (t: {V}, number?) -> V?, + sort: (t: {V}, comp: ((V, V) -> boolean)?) -> (), + create: (count: number, value: V?) -> {V}, + find: (haystack: {V}, needle: V, init: number?) -> number?, - unpack: ({V}, number?, number?) -> ...V, + unpack: (list: {V}, i: number?, j: number?) -> ...V, pack: (...V) -> { n: number, [number]: V }, - getn: ({V}) -> number, - foreach: ({[K]: V}, (K, V) -> ()) -> (), + getn: (t: {V}) -> number, + foreach: (t: {[K]: V}, f: (K, V) -> ()) -> (), foreachi: ({V}, (number, V) -> ()) -> (), - move: ({V}, number, number, number, {V}?) -> {V}, - clear: ({[K]: V}) -> (), + move: (src: {V}, a: number, b: number, t: number, dst: {V}?) -> {V}, + clear: (table: {[K]: V}) -> (), - isfrozen: ({[K]: V}) -> boolean, + isfrozen: (t: {[K]: V}) -> boolean, } declare debug: { - info: ((thread, number, string) -> R...) & ((number, string) -> R...) & (((A...) -> R1..., string) -> R2...), - traceback: ((string?, number?) -> string) & ((thread, string?, number?) -> string), + info: ((thread: thread, level: number, options: string) -> R...) & ((level: number, options: string) -> R...) & ((func: (A...) -> R1..., options: string) -> R2...), + traceback: ((message: string?, level: number?) -> string) & ((thread: thread, message: string?, level: number?) -> string), } declare utf8: { char: (...number) -> string, charpattern: string, - codes: (string) -> ((string, number) -> (number, number), string, number), - codepoint: (string, number?, number?) -> ...number, - len: (string, number?, number?) -> (number?, number?), - offset: (string, number?, number?) -> number, + codes: (str: string) -> ((string, number) -> (number, number), string, number), + codepoint: (str: string, i: number?, j: number?) -> ...number, + len: (s: string, i: number?, j: number?) -> (number?, number?), + offset: (s: string, n: number?, i: number?) -> number, } -- Cannot use `typeof` here because it will produce a polytype when we expect a monotype. diff --git a/Analysis/src/Normalize.cpp b/Analysis/src/Normalize.cpp index 5ef4b7e7..21e9f787 100644 --- a/Analysis/src/Normalize.cpp +++ b/Analysis/src/Normalize.cpp @@ -7,9 +7,9 @@ #include "Luau/Clone.h" #include "Luau/Common.h" +#include "Luau/RecursionCounter.h" #include "Luau/TypeVar.h" #include "Luau/Unifier.h" -#include "Luau/VisitTypeVar.h" LUAU_FASTFLAGVARIABLE(DebugLuauCopyBeforeNormalizing, false) LUAU_FASTFLAGVARIABLE(DebugLuauCheckNormalizeInvariant, false) @@ -20,6 +20,7 @@ LUAU_FASTINTVARIABLE(LuauNormalizeCacheLimit, 100000); LUAU_FASTFLAGVARIABLE(LuauNormalizeCombineTableFix, false); LUAU_FASTFLAGVARIABLE(LuauTypeNormalization2, false); LUAU_FASTFLAGVARIABLE(LuauNegatedStringSingletons, false); +LUAU_FASTFLAGVARIABLE(LuauNegatedFunctionTypes, false); LUAU_FASTFLAG(LuauUnknownAndNeverType) LUAU_FASTFLAG(DebugLuauDeferredConstraintResolution) LUAU_FASTFLAG(LuauOverloadedFunctionSubtypingPerf); @@ -206,6 +207,28 @@ bool isSubtype(const NormalizedStringType& subStr, const NormalizedStringType& s return true; } +NormalizedFunctionType::NormalizedFunctionType() + : parts(FFlag::LuauNegatedFunctionTypes ? std::optional{TypeIds{}} : std::nullopt) +{ +} + +void NormalizedFunctionType::resetToTop() +{ + isTop = true; + parts.emplace(); +} + +void NormalizedFunctionType::resetToNever() +{ + isTop = false; + parts.emplace(); +} + +bool NormalizedFunctionType::isNever() const +{ + return !isTop && (!parts || parts->empty()); +} + NormalizedType::NormalizedType(NotNull singletonTypes) : tops(singletonTypes->neverType) , booleans(singletonTypes->neverType) @@ -220,8 +243,8 @@ NormalizedType::NormalizedType(NotNull singletonTypes) static bool isInhabited(const NormalizedType& norm) { return !get(norm.tops) || !get(norm.booleans) || !norm.classes.empty() || !get(norm.errors) || - !get(norm.nils) || !get(norm.numbers) || !norm.strings.isNever() || - !get(norm.threads) || norm.functions || !norm.tables.empty() || !norm.tyvars.empty(); + !get(norm.nils) || !get(norm.numbers) || !norm.strings.isNever() || !get(norm.threads) || + !norm.functions.isNever() || !norm.tables.empty() || !norm.tyvars.empty(); } static int tyvarIndex(TypeId ty) @@ -317,10 +340,14 @@ static bool isNormalizedThread(TypeId ty) static bool areNormalizedFunctions(const NormalizedFunctionType& tys) { - if (tys) - for (TypeId ty : *tys) + if (tys.parts) + { + for (TypeId ty : *tys.parts) + { if (!get(ty) && !get(ty)) return false; + } + } return true; } @@ -420,7 +447,7 @@ void Normalizer::clearNormal(NormalizedType& norm) norm.strings.resetToNever(); norm.threads = singletonTypes->neverType; norm.tables.clear(); - norm.functions = std::nullopt; + norm.functions.resetToNever(); norm.tyvars.clear(); } @@ -809,20 +836,28 @@ std::optional Normalizer::unionOfFunctions(TypeId here, TypeId there) void Normalizer::unionFunctions(NormalizedFunctionType& heres, const NormalizedFunctionType& theres) { - if (!theres) + if (FFlag::LuauNegatedFunctionTypes) + { + if (heres.isTop) + return; + if (theres.isTop) + heres.resetToTop(); + } + + if (theres.isNever()) return; TypeIds tmps; - if (!heres) + if (heres.isNever()) { - tmps.insert(theres->begin(), theres->end()); - heres = std::move(tmps); + tmps.insert(theres.parts->begin(), theres.parts->end()); + heres.parts = std::move(tmps); return; } - for (TypeId here : *heres) - for (TypeId there : *theres) + for (TypeId here : *heres.parts) + for (TypeId there : *theres.parts) { if (std::optional fun = unionOfFunctions(here, there)) tmps.insert(*fun); @@ -830,28 +865,28 @@ void Normalizer::unionFunctions(NormalizedFunctionType& heres, const NormalizedF tmps.insert(singletonTypes->errorRecoveryType(there)); } - heres = std::move(tmps); + heres.parts = std::move(tmps); } void Normalizer::unionFunctionsWithFunction(NormalizedFunctionType& heres, TypeId there) { - if (!heres) + if (heres.isNever()) { TypeIds tmps; tmps.insert(there); - heres = std::move(tmps); + heres.parts = std::move(tmps); return; } TypeIds tmps; - for (TypeId here : *heres) + for (TypeId here : *heres.parts) { if (std::optional fun = unionOfFunctions(here, there)) tmps.insert(*fun); else tmps.insert(singletonTypes->errorRecoveryType(there)); } - heres = std::move(tmps); + heres.parts = std::move(tmps); } void Normalizer::unionTablesWithTable(TypeIds& heres, TypeId there) @@ -1004,6 +1039,11 @@ bool Normalizer::unionNormalWithTy(NormalizedType& here, TypeId there, int ignor here.strings.resetToString(); else if (ptv->type == PrimitiveTypeVar::Thread) here.threads = there; + else if (ptv->type == PrimitiveTypeVar::Function) + { + LUAU_ASSERT(FFlag::LuauNegatedFunctionTypes); + here.functions.resetToTop(); + } else LUAU_ASSERT(!"Unreachable"); } @@ -1036,8 +1076,11 @@ bool Normalizer::unionNormalWithTy(NormalizedType& here, TypeId there, int ignor else if (const NegationTypeVar* ntv = get(there)) { const NormalizedType* thereNormal = normalize(ntv->ty); - NormalizedType tn = negateNormal(*thereNormal); - if (!unionNormals(here, tn)) + std::optional tn = negateNormal(*thereNormal); + if (!tn) + return false; + + if (!unionNormals(here, *tn)) return false; } else @@ -1053,7 +1096,7 @@ bool Normalizer::unionNormalWithTy(NormalizedType& here, TypeId there, int ignor // ------- Negations -NormalizedType Normalizer::negateNormal(const NormalizedType& here) +std::optional Normalizer::negateNormal(const NormalizedType& here) { NormalizedType result{singletonTypes}; if (!get(here.tops)) @@ -1092,10 +1135,24 @@ NormalizedType Normalizer::negateNormal(const NormalizedType& here) result.threads = get(here.threads) ? singletonTypes->threadType : singletonTypes->neverType; + /* + * Things get weird and so, so complicated if we allow negations of + * arbitrary function types. Ordinary code can never form these kinds of + * types, so we decline to negate them. + */ + if (FFlag::LuauNegatedFunctionTypes) + { + if (here.functions.isNever()) + result.functions.resetToTop(); + else if (here.functions.isTop) + result.functions.resetToNever(); + else + return std::nullopt; + } + // TODO: negating tables - // TODO: negating functions // TODO: negating tyvars? - + return result; } @@ -1142,21 +1199,25 @@ void Normalizer::subtractPrimitive(NormalizedType& here, TypeId ty) LUAU_ASSERT(ptv); switch (ptv->type) { - case PrimitiveTypeVar::NilType: - here.nils = singletonTypes->neverType; - break; - case PrimitiveTypeVar::Boolean: - here.booleans = singletonTypes->neverType; - break; - case PrimitiveTypeVar::Number: - here.numbers = singletonTypes->neverType; - break; - case PrimitiveTypeVar::String: - here.strings.resetToNever(); - break; - case PrimitiveTypeVar::Thread: - here.threads = singletonTypes->neverType; - break; + case PrimitiveTypeVar::NilType: + here.nils = singletonTypes->neverType; + break; + case PrimitiveTypeVar::Boolean: + here.booleans = singletonTypes->neverType; + break; + case PrimitiveTypeVar::Number: + here.numbers = singletonTypes->neverType; + break; + case PrimitiveTypeVar::String: + here.strings.resetToNever(); + break; + case PrimitiveTypeVar::Thread: + here.threads = singletonTypes->neverType; + break; + case PrimitiveTypeVar::Function: + LUAU_ASSERT(FFlag::LuauNegatedStringSingletons); + here.functions.resetToNever(); + break; } } @@ -1589,7 +1650,7 @@ std::optional Normalizer::intersectionOfFunctions(TypeId here, TypeId th TypePackId argTypes; TypePackId retTypes; - + if (hftv->retTypes == tftv->retTypes) { std::optional argTypesOpt = unionOfTypePacks(hftv->argTypes, tftv->argTypes); @@ -1598,7 +1659,7 @@ std::optional Normalizer::intersectionOfFunctions(TypeId here, TypeId th argTypes = *argTypesOpt; retTypes = hftv->retTypes; } - else if (FFlag::LuauOverloadedFunctionSubtypingPerf && hftv->argTypes == tftv->argTypes) + else if (FFlag::LuauOverloadedFunctionSubtypingPerf && hftv->argTypes == tftv->argTypes) { std::optional retTypesOpt = intersectionOfTypePacks(hftv->argTypes, tftv->argTypes); if (!retTypesOpt) @@ -1738,18 +1799,20 @@ std::optional Normalizer::unionSaturatedFunctions(TypeId here, TypeId th void Normalizer::intersectFunctionsWithFunction(NormalizedFunctionType& heres, TypeId there) { - if (!heres) + if (heres.isNever()) return; - for (auto it = heres->begin(); it != heres->end();) + heres.isTop = false; + + for (auto it = heres.parts->begin(); it != heres.parts->end();) { TypeId here = *it; if (get(here)) it++; else if (std::optional tmp = intersectionOfFunctions(here, there)) { - heres->erase(it); - heres->insert(*tmp); + heres.parts->erase(it); + heres.parts->insert(*tmp); return; } else @@ -1757,27 +1820,27 @@ void Normalizer::intersectFunctionsWithFunction(NormalizedFunctionType& heres, T } TypeIds tmps; - for (TypeId here : *heres) + for (TypeId here : *heres.parts) { if (std::optional tmp = unionSaturatedFunctions(here, there)) tmps.insert(*tmp); } - heres->insert(there); - heres->insert(tmps.begin(), tmps.end()); + heres.parts->insert(there); + heres.parts->insert(tmps.begin(), tmps.end()); } void Normalizer::intersectFunctions(NormalizedFunctionType& heres, const NormalizedFunctionType& theres) { - if (!heres) + if (heres.isNever()) return; - else if (!theres) + else if (theres.isNever()) { - heres = std::nullopt; + heres.resetToNever(); return; } else { - for (TypeId there : *theres) + for (TypeId there : *theres.parts) intersectFunctionsWithFunction(heres, there); } } @@ -1935,6 +1998,7 @@ bool Normalizer::intersectNormalWithTy(NormalizedType& here, TypeId there) TypeId nils = here.nils; TypeId numbers = here.numbers; NormalizedStringType strings = std::move(here.strings); + NormalizedFunctionType functions = std::move(here.functions); TypeId threads = here.threads; clearNormal(here); @@ -1949,6 +2013,11 @@ bool Normalizer::intersectNormalWithTy(NormalizedType& here, TypeId there) here.strings = std::move(strings); else if (ptv->type == PrimitiveTypeVar::Thread) here.threads = threads; + else if (ptv->type == PrimitiveTypeVar::Function) + { + LUAU_ASSERT(FFlag::LuauNegatedFunctionTypes); + here.functions = std::move(functions); + } else LUAU_ASSERT(!"Unreachable"); } @@ -1981,8 +2050,10 @@ bool Normalizer::intersectNormalWithTy(NormalizedType& here, TypeId there) for (TypeId part : itv->options) { const NormalizedType* normalPart = normalize(part); - NormalizedType negated = negateNormal(*normalPart); - intersectNormals(here, negated); + std::optional negated = negateNormal(*normalPart); + if (!negated) + return false; + intersectNormals(here, *negated); } } else @@ -2016,14 +2087,16 @@ TypeId Normalizer::typeFromNormal(const NormalizedType& norm) result.insert(result.end(), norm.classes.begin(), norm.classes.end()); if (!get(norm.errors)) result.push_back(norm.errors); - if (norm.functions) + if (FFlag::LuauNegatedFunctionTypes && norm.functions.isTop) + result.push_back(singletonTypes->functionType); + else if (!norm.functions.isNever()) { - if (norm.functions->size() == 1) - result.push_back(*norm.functions->begin()); + if (norm.functions.parts->size() == 1) + result.push_back(*norm.functions.parts->begin()); else { std::vector parts; - parts.insert(parts.end(), norm.functions->begin(), norm.functions->end()); + parts.insert(parts.end(), norm.functions.parts->begin(), norm.functions.parts->end()); result.push_back(arena->addType(IntersectionTypeVar{std::move(parts)})); } } @@ -2070,62 +2143,24 @@ TypeId Normalizer::typeFromNormal(const NormalizedType& norm) return arena->addType(UnionTypeVar{std::move(result)}); } -namespace -{ - -struct Replacer -{ - TypeArena* arena; - TypeId sourceType; - TypeId replacedType; - DenseHashMap newTypes; - - Replacer(TypeArena* arena, TypeId sourceType, TypeId replacedType) - : arena(arena) - , sourceType(sourceType) - , replacedType(replacedType) - , newTypes(nullptr) - { - } - - TypeId smartClone(TypeId t) - { - t = follow(t); - TypeId* res = newTypes.find(t); - if (res) - return *res; - - TypeId result = shallowClone(t, *arena, TxnLog::empty()); - newTypes[t] = result; - newTypes[result] = result; - - return result; - } -}; - -} // anonymous namespace - -bool isSubtype(TypeId subTy, TypeId superTy, NotNull scope, NotNull singletonTypes, InternalErrorReporter& ice, bool anyIsTop) +bool isSubtype(TypeId subTy, TypeId superTy, NotNull scope, NotNull singletonTypes, InternalErrorReporter& ice) { UnifierSharedState sharedState{&ice}; TypeArena arena; Normalizer normalizer{&arena, singletonTypes, NotNull{&sharedState}}; Unifier u{NotNull{&normalizer}, Mode::Strict, scope, Location{}, Covariant}; - u.anyIsTop = anyIsTop; u.tryUnify(subTy, superTy); const bool ok = u.errors.empty() && u.log.empty(); return ok; } -bool isSubtype( - TypePackId subPack, TypePackId superPack, NotNull scope, NotNull singletonTypes, InternalErrorReporter& ice, bool anyIsTop) +bool isSubtype(TypePackId subPack, TypePackId superPack, NotNull scope, NotNull singletonTypes, InternalErrorReporter& ice) { UnifierSharedState sharedState{&ice}; TypeArena arena; Normalizer normalizer{&arena, singletonTypes, NotNull{&sharedState}}; Unifier u{NotNull{&normalizer}, Mode::Strict, scope, Location{}, Covariant}; - u.anyIsTop = anyIsTop; u.tryUnify(subPack, superPack); const bool ok = u.errors.empty() && u.log.empty(); diff --git a/Analysis/src/ToString.cpp b/Analysis/src/ToString.cpp index 7d0fb22d..903e156b 100644 --- a/Analysis/src/ToString.cpp +++ b/Analysis/src/ToString.cpp @@ -10,12 +10,12 @@ #include #include -LUAU_FASTFLAG(LuauUnknownAndNeverType) +LUAU_FASTFLAG(DebugLuauDeferredConstraintResolution) LUAU_FASTFLAG(LuauLvaluelessPath) -LUAU_FASTFLAGVARIABLE(LuauSpecialTypesAsterisked, false) +LUAU_FASTFLAG(LuauUnknownAndNeverType) LUAU_FASTFLAGVARIABLE(LuauFixNameMaps, false) -LUAU_FASTFLAGVARIABLE(LuauUnseeArrayTtv, false) LUAU_FASTFLAGVARIABLE(LuauFunctionReturnStringificationFixup, false) +LUAU_FASTFLAGVARIABLE(LuauUnseeArrayTtv, false) /* * Prefix generic typenames with gen- @@ -225,6 +225,20 @@ struct StringifierState result.name += s; } + void emitLevel(Scope* scope) + { + size_t count = 0; + for (Scope* s = scope; s; s = s->parent.get()) + ++count; + + emit(count); + emit("-"); + char buffer[16]; + uint32_t s = uint32_t(intptr_t(scope) & 0xFFFFFF); + snprintf(buffer, sizeof(buffer), "0x%x", s); + emit(buffer); + } + void emit(TypeLevel level) { emit(std::to_string(level.level)); @@ -296,10 +310,7 @@ struct TypeVarStringifier if (tv->ty.valueless_by_exception()) { state.result.error = true; - if (FFlag::LuauSpecialTypesAsterisked) - state.emit("* VALUELESS BY EXCEPTION *"); - else - state.emit("< VALUELESS BY EXCEPTION >"); + state.emit("* VALUELESS BY EXCEPTION *"); return; } @@ -377,7 +388,10 @@ struct TypeVarStringifier if (FFlag::DebugLuauVerboseTypeNames) { state.emit("-"); - state.emit(ftv.level); + if (FFlag::DebugLuauDeferredConstraintResolution) + state.emitLevel(ftv.scope); + else + state.emit(ftv.level); } } @@ -399,6 +413,15 @@ struct TypeVarStringifier } else state.emit(state.getName(ty)); + + if (FFlag::DebugLuauVerboseTypeNames) + { + state.emit("-"); + if (FFlag::DebugLuauDeferredConstraintResolution) + state.emitLevel(gtv.scope); + else + state.emit(gtv.level); + } } void operator()(TypeId, const BlockedTypeVar& btv) @@ -434,6 +457,9 @@ struct TypeVarStringifier case PrimitiveTypeVar::Thread: state.emit("thread"); return; + case PrimitiveTypeVar::Function: + state.emit("function"); + return; default: LUAU_ASSERT(!"Unknown primitive type"); throwRuntimeError("Unknown primitive type " + std::to_string(ptv.type)); @@ -462,10 +488,7 @@ struct TypeVarStringifier if (state.hasSeen(&ftv)) { state.result.cycle = true; - if (FFlag::LuauSpecialTypesAsterisked) - state.emit("*CYCLE*"); - else - state.emit(""); + state.emit("*CYCLE*"); return; } @@ -573,10 +596,7 @@ struct TypeVarStringifier if (state.hasSeen(&ttv)) { state.result.cycle = true; - if (FFlag::LuauSpecialTypesAsterisked) - state.emit("*CYCLE*"); - else - state.emit(""); + state.emit("*CYCLE*"); return; } @@ -710,10 +730,7 @@ struct TypeVarStringifier if (state.hasSeen(&uv)) { state.result.cycle = true; - if (FFlag::LuauSpecialTypesAsterisked) - state.emit("*CYCLE*"); - else - state.emit(""); + state.emit("*CYCLE*"); return; } @@ -780,10 +797,7 @@ struct TypeVarStringifier if (state.hasSeen(&uv)) { state.result.cycle = true; - if (FFlag::LuauSpecialTypesAsterisked) - state.emit("*CYCLE*"); - else - state.emit(""); + state.emit("*CYCLE*"); return; } @@ -828,10 +842,7 @@ struct TypeVarStringifier void operator()(TypeId, const ErrorTypeVar& tv) { state.result.error = true; - if (FFlag::LuauSpecialTypesAsterisked) - state.emit(FFlag::LuauUnknownAndNeverType ? "*error-type*" : "*unknown*"); - else - state.emit(FFlag::LuauUnknownAndNeverType ? "" : "*unknown*"); + state.emit(FFlag::LuauUnknownAndNeverType ? "*error-type*" : "*unknown*"); } void operator()(TypeId, const LazyTypeVar& ltv) @@ -850,11 +861,6 @@ struct TypeVarStringifier state.emit("never"); } - void operator()(TypeId ty, const UseTypeVar&) - { - stringify(follow(ty)); - } - void operator()(TypeId, const NegationTypeVar& ntv) { state.emit("~"); @@ -907,10 +913,7 @@ struct TypePackStringifier if (tp->ty.valueless_by_exception()) { state.result.error = true; - if (FFlag::LuauSpecialTypesAsterisked) - state.emit("* VALUELESS TP BY EXCEPTION *"); - else - state.emit("< VALUELESS TP BY EXCEPTION >"); + state.emit("* VALUELESS TP BY EXCEPTION *"); return; } @@ -934,10 +937,7 @@ struct TypePackStringifier if (state.hasSeen(&tp)) { state.result.cycle = true; - if (FFlag::LuauSpecialTypesAsterisked) - state.emit("*CYCLETP*"); - else - state.emit(""); + state.emit("*CYCLETP*"); return; } @@ -982,10 +982,7 @@ struct TypePackStringifier void operator()(TypePackId, const Unifiable::Error& error) { state.result.error = true; - if (FFlag::LuauSpecialTypesAsterisked) - state.emit(FFlag::LuauUnknownAndNeverType ? "*error-type*" : "*unknown*"); - else - state.emit(FFlag::LuauUnknownAndNeverType ? "" : "*unknown*"); + state.emit(FFlag::LuauUnknownAndNeverType ? "*error-type*" : "*unknown*"); } void operator()(TypePackId, const VariadicTypePack& pack) @@ -993,10 +990,7 @@ struct TypePackStringifier state.emit("..."); if (FFlag::DebugLuauVerboseTypeNames && pack.hidden) { - if (FFlag::LuauSpecialTypesAsterisked) - state.emit("*hidden*"); - else - state.emit(""); + state.emit("*hidden*"); } stringify(pack.ty); } @@ -1031,7 +1025,10 @@ struct TypePackStringifier if (FFlag::DebugLuauVerboseTypeNames) { state.emit("-"); - state.emit(pack.level); + if (FFlag::DebugLuauDeferredConstraintResolution) + state.emitLevel(pack.scope); + else + state.emit(pack.level); } state.emit("..."); @@ -1204,10 +1201,7 @@ ToStringResult toStringDetailed(TypeId ty, ToStringOptions& opts) { result.truncated = true; - if (FFlag::LuauSpecialTypesAsterisked) - result.name += "... *TRUNCATED*"; - else - result.name += "... "; + result.name += "... *TRUNCATED*"; } return result; @@ -1280,10 +1274,7 @@ ToStringResult toStringDetailed(TypePackId tp, ToStringOptions& opts) if (opts.maxTypeLength > 0 && result.name.length() > opts.maxTypeLength) { - if (FFlag::LuauSpecialTypesAsterisked) - result.name += "... *TRUNCATED*"; - else - result.name += "... "; + result.name += "... *TRUNCATED*"; } return result; @@ -1526,9 +1517,12 @@ std::string toString(const Constraint& constraint, ToStringOptions& opts) { return tos(c.resultType, opts) + " ~ hasProp " + tos(c.subjectType, opts) + ", \"" + c.prop + "\""; } - else if constexpr (std::is_same_v) + else if constexpr (std::is_same_v) { - return "TODO"; + std::string result = tos(c.resultType, opts); + std::string discriminant = tos(c.discriminantType, opts); + + return result + " ~ if isSingleton D then ~D else unknown where D = " + discriminant; } else static_assert(always_false_v, "Non-exhaustive constraint switch"); diff --git a/Analysis/src/TypeAttach.cpp b/Analysis/src/TypeAttach.cpp index 179846d7..c97ed05d 100644 --- a/Analysis/src/TypeAttach.cpp +++ b/Analysis/src/TypeAttach.cpp @@ -338,12 +338,6 @@ public: { return allocator->alloc(Location(), std::nullopt, AstName{"never"}); } - AstType* operator()(const UseTypeVar& utv) - { - std::optional ty = utv.scope->lookup(utv.def); - LUAU_ASSERT(ty); - return Luau::visit(*this, (*ty)->ty); - } AstType* operator()(const NegationTypeVar& ntv) { // FIXME: do the same thing we do with ErrorTypeVar diff --git a/Analysis/src/TypeChecker2.cpp b/Analysis/src/TypeChecker2.cpp index a2673158..dde41a65 100644 --- a/Analysis/src/TypeChecker2.cpp +++ b/Analysis/src/TypeChecker2.cpp @@ -301,7 +301,6 @@ struct TypeChecker2 UnifierSharedState sharedState{&ice}; Normalizer normalizer{&arena, singletonTypes, NotNull{&sharedState}}; Unifier u{NotNull{&normalizer}, Mode::Strict, stack.back(), ret->location, Covariant}; - u.anyIsTop = true; u.tryUnify(actualRetType, expectedRetType); const bool ok = u.errors.empty() && u.log.empty(); @@ -331,16 +330,21 @@ struct TypeChecker2 if (value) visit(value); - if (i != local->values.size - 1) + TypeId* maybeValueType = value ? module->astTypes.find(value) : nullptr; + if (i != local->values.size - 1 || maybeValueType) { AstLocal* var = i < local->vars.size ? local->vars.data[i] : nullptr; if (var && var->annotation) { - TypeId varType = lookupAnnotation(var->annotation); + TypeId annotationType = lookupAnnotation(var->annotation); TypeId valueType = value ? lookupType(value) : nullptr; - if (valueType && !isSubtype(varType, valueType, stack.back(), singletonTypes, ice, /* anyIsTop */ false)) - reportError(TypeMismatch{varType, valueType}, value->location); + if (valueType) + { + ErrorVec errors = tryUnify(stack.back(), value->location, valueType, annotationType); + if (!errors.empty()) + reportErrors(std::move(errors)); + } } } else @@ -606,7 +610,7 @@ struct TypeChecker2 visit(rhs); TypeId rhsType = lookupType(rhs); - if (!isSubtype(rhsType, lhsType, stack.back(), singletonTypes, ice, /* anyIsTop */ false)) + if (!isSubtype(rhsType, lhsType, stack.back(), singletonTypes, ice)) { reportError(TypeMismatch{lhsType, rhsType}, rhs->location); } @@ -757,7 +761,7 @@ struct TypeChecker2 TypeId actualType = lookupType(number); TypeId numberType = singletonTypes->numberType; - if (!isSubtype(numberType, actualType, stack.back(), singletonTypes, ice, /* anyIsTop */ false)) + if (!isSubtype(numberType, actualType, stack.back(), singletonTypes, ice)) { reportError(TypeMismatch{actualType, numberType}, number->location); } @@ -768,7 +772,7 @@ struct TypeChecker2 TypeId actualType = lookupType(string); TypeId stringType = singletonTypes->stringType; - if (!isSubtype(actualType, stringType, stack.back(), singletonTypes, ice, /* anyIsTop */ false)) + if (!isSubtype(actualType, stringType, stack.back(), singletonTypes, ice)) { reportError(TypeMismatch{actualType, stringType}, string->location); } @@ -857,7 +861,7 @@ struct TypeChecker2 FunctionTypeVar ftv{argsTp, expectedRetType}; TypeId expectedType = arena.addType(ftv); - if (!isSubtype(testFunctionType, expectedType, stack.back(), singletonTypes, ice, /* anyIsTop */ false)) + if (!isSubtype(testFunctionType, expectedType, stack.back(), singletonTypes, ice)) { CloneState cloneState; expectedType = clone(expectedType, module->internalTypes, cloneState); @@ -876,7 +880,7 @@ struct TypeChecker2 getIndexTypeFromType(module->getModuleScope(), leftType, indexName->index.value, indexName->location, /* addErrors */ true); if (ty) { - if (!isSubtype(resultType, *ty, stack.back(), singletonTypes, ice, /* anyIsTop */ false)) + if (!isSubtype(resultType, *ty, stack.back(), singletonTypes, ice)) { reportError(TypeMismatch{resultType, *ty}, indexName->location); } @@ -909,7 +913,7 @@ struct TypeChecker2 TypeId inferredArgTy = *argIt; TypeId annotatedArgTy = lookupAnnotation(arg->annotation); - if (!isSubtype(annotatedArgTy, inferredArgTy, stack.back(), singletonTypes, ice, /* anyIsTop */ false)) + if (!isSubtype(annotatedArgTy, inferredArgTy, stack.back(), singletonTypes, ice)) { reportError(TypeMismatch{annotatedArgTy, inferredArgTy}, arg->location); } @@ -1203,10 +1207,10 @@ struct TypeChecker2 TypeId computedType = lookupType(expr->expr); // Note: As an optimization, we try 'number <: number | string' first, as that is the more likely case. - if (isSubtype(annotationType, computedType, stack.back(), singletonTypes, ice, /* anyIsTop */ false)) + if (isSubtype(annotationType, computedType, stack.back(), singletonTypes, ice)) return; - if (isSubtype(computedType, annotationType, stack.back(), singletonTypes, ice, /* anyIsTop */ false)) + if (isSubtype(computedType, annotationType, stack.back(), singletonTypes, ice)) return; reportError(TypesAreUnrelated{computedType, annotationType}, expr->location); @@ -1507,7 +1511,6 @@ struct TypeChecker2 UnifierSharedState sharedState{&ice}; Normalizer normalizer{&module->internalTypes, singletonTypes, NotNull{&sharedState}}; Unifier u{NotNull{&normalizer}, Mode::Strict, scope, location, Covariant}; - u.anyIsTop = true; u.tryUnify(subTy, superTy); return std::move(u.errors); diff --git a/Analysis/src/TypeVar.cpp b/Analysis/src/TypeVar.cpp index 94d633c7..de0890e1 100644 --- a/Analysis/src/TypeVar.cpp +++ b/Analysis/src/TypeVar.cpp @@ -57,13 +57,6 @@ TypeId follow(TypeId t, std::function mapper) return btv->boundTo; else if (auto ttv = get(mapper(ty))) return ttv->boundTo; - else if (auto utv = get(mapper(ty))) - { - std::optional ty = utv->scope->lookup(utv->def); - if (!ty) - throwRuntimeError("UseTypeVar must map to another TypeId"); - return *ty; - } else return std::nullopt; }; @@ -761,6 +754,7 @@ SingletonTypes::SingletonTypes() , stringType(arena->addType(TypeVar{PrimitiveTypeVar{PrimitiveTypeVar::String}, /*persistent*/ true})) , booleanType(arena->addType(TypeVar{PrimitiveTypeVar{PrimitiveTypeVar::Boolean}, /*persistent*/ true})) , threadType(arena->addType(TypeVar{PrimitiveTypeVar{PrimitiveTypeVar::Thread}, /*persistent*/ true})) + , functionType(arena->addType(TypeVar{PrimitiveTypeVar{PrimitiveTypeVar::Function}, /*persistent*/ true})) , trueType(arena->addType(TypeVar{SingletonTypeVar{BooleanSingleton{true}}, /*persistent*/ true})) , falseType(arena->addType(TypeVar{SingletonTypeVar{BooleanSingleton{false}}, /*persistent*/ true})) , anyType(arena->addType(TypeVar{AnyTypeVar{}, /*persistent*/ true})) @@ -946,7 +940,8 @@ void persist(TypeId ty) queue.push_back(mtv->table); queue.push_back(mtv->metatable); } - else if (get(t) || get(t) || get(t) || get(t) || get(t) || get(t)) + else if (get(t) || get(t) || get(t) || get(t) || get(t) || + get(t)) { } else diff --git a/Analysis/src/Unifier.cpp b/Analysis/src/Unifier.cpp index b5eba980..df5d86f1 100644 --- a/Analysis/src/Unifier.cpp +++ b/Analysis/src/Unifier.cpp @@ -8,6 +8,7 @@ #include "Luau/TypePack.h" #include "Luau/TypeUtils.h" #include "Luau/TimeTrace.h" +#include "Luau/TypeVar.h" #include "Luau/VisitTypeVar.h" #include "Luau/ToString.h" @@ -23,6 +24,7 @@ LUAU_FASTFLAGVARIABLE(LuauInstantiateInSubtyping, false) LUAU_FASTFLAGVARIABLE(LuauOverloadedFunctionSubtypingPerf, false); LUAU_FASTFLAG(LuauClassTypeVarsInSubstitution) LUAU_FASTFLAG(DebugLuauDeferredConstraintResolution) +LUAU_FASTFLAG(LuauNegatedFunctionTypes) namespace Luau { @@ -363,7 +365,7 @@ void Unifier::tryUnify_(TypeId subTy, TypeId superTy, bool isFunctionCall, bool if (sharedState.counters.iterationLimit > 0 && sharedState.counters.iterationLimit < sharedState.counters.iterationCount) { - reportError(TypeError{location, UnificationTooComplex{}}); + reportError(location, UnificationTooComplex{}); return; } @@ -404,7 +406,7 @@ void Unifier::tryUnify_(TypeId subTy, TypeId superTy, bool isFunctionCall, bool if (subGeneric && !subsumes(useScopes, subGeneric, superFree)) { // TODO: a more informative error message? CLI-39912 - reportError(TypeError{location, GenericError{"Generic subtype escaping scope"}}); + reportError(location, GenericError{"Generic subtype escaping scope"}); return; } @@ -433,7 +435,7 @@ void Unifier::tryUnify_(TypeId subTy, TypeId superTy, bool isFunctionCall, bool if (superGeneric && !subsumes(useScopes, superGeneric, subFree)) { // TODO: a more informative error message? CLI-39912 - reportError(TypeError{location, GenericError{"Generic supertype escaping scope"}}); + reportError(location, GenericError{"Generic supertype escaping scope"}); return; } @@ -450,15 +452,7 @@ void Unifier::tryUnify_(TypeId subTy, TypeId superTy, bool isFunctionCall, bool return tryUnifyWithAny(subTy, superTy); if (get(subTy)) - { - if (anyIsTop) - { - reportError(TypeError{location, TypeMismatch{superTy, subTy}}); - return; - } - else - return tryUnifyWithAny(superTy, subTy); - } + return tryUnifyWithAny(superTy, subTy); if (log.get(subTy)) return tryUnifyWithAny(superTy, subTy); @@ -478,7 +472,7 @@ void Unifier::tryUnify_(TypeId subTy, TypeId superTy, bool isFunctionCall, bool if (auto error = sharedState.cachedUnifyError.find({subTy, superTy})) { - reportError(TypeError{location, *error}); + reportError(location, *error); return; } } @@ -520,6 +514,12 @@ void Unifier::tryUnify_(TypeId subTy, TypeId superTy, bool isFunctionCall, bool else if ((log.getMutable(superTy) || log.getMutable(superTy)) && log.getMutable(subTy)) tryUnifySingletons(subTy, superTy); + else if (auto ptv = get(superTy); + FFlag::LuauNegatedFunctionTypes && ptv && ptv->type == PrimitiveTypeVar::Function && get(subTy)) + { + // Ok. Do nothing. forall functions F, F <: function + } + else if (log.getMutable(superTy) && log.getMutable(subTy)) tryUnifyFunctions(subTy, superTy, isFunctionCall); @@ -559,7 +559,7 @@ void Unifier::tryUnify_(TypeId subTy, TypeId superTy, bool isFunctionCall, bool tryUnifyNegationWithType(subTy, superTy); else - reportError(TypeError{location, TypeMismatch{superTy, subTy}}); + reportError(location, TypeMismatch{superTy, subTy}); if (cacheEnabled) cacheResult(subTy, superTy, errorCount); @@ -633,9 +633,9 @@ void Unifier::tryUnifyUnionWithType(TypeId subTy, const UnionTypeVar* subUnion, else if (failed) { if (firstFailedOption) - reportError(TypeError{location, TypeMismatch{superTy, subTy, "Not all union options are compatible.", *firstFailedOption}}); + reportError(location, TypeMismatch{superTy, subTy, "Not all union options are compatible.", *firstFailedOption}); else - reportError(TypeError{location, TypeMismatch{superTy, subTy}}); + reportError(location, TypeMismatch{superTy, subTy}); } } @@ -734,7 +734,7 @@ void Unifier::tryUnifyTypeWithUnion(TypeId subTy, TypeId superTy, const UnionTyp const NormalizedType* subNorm = normalizer->normalize(subTy); const NormalizedType* superNorm = normalizer->normalize(superTy); if (!subNorm || !superNorm) - reportError(TypeError{location, UnificationTooComplex{}}); + reportError(location, UnificationTooComplex{}); else if ((failedOptionCount == 1 || foundHeuristic) && failedOption) tryUnifyNormalizedTypes(subTy, superTy, *subNorm, *superNorm, "None of the union options are compatible. For example:", *failedOption); else @@ -743,9 +743,9 @@ void Unifier::tryUnifyTypeWithUnion(TypeId subTy, TypeId superTy, const UnionTyp else if (!found) { if ((failedOptionCount == 1 || foundHeuristic) && failedOption) - reportError(TypeError{location, TypeMismatch{superTy, subTy, "None of the union options are compatible. For example:", *failedOption}}); + reportError(location, TypeMismatch{superTy, subTy, "None of the union options are compatible. For example:", *failedOption}); else - reportError(TypeError{location, TypeMismatch{superTy, subTy, "none of the union options are compatible"}}); + reportError(location, TypeMismatch{superTy, subTy, "none of the union options are compatible"}); } } @@ -774,7 +774,7 @@ void Unifier::tryUnifyTypeWithIntersection(TypeId subTy, TypeId superTy, const I if (unificationTooComplex) reportError(*unificationTooComplex); else if (firstFailedOption) - reportError(TypeError{location, TypeMismatch{superTy, subTy, "Not all intersection parts are compatible.", *firstFailedOption}}); + reportError(location, TypeMismatch{superTy, subTy, "Not all intersection parts are compatible.", *firstFailedOption}); } void Unifier::tryUnifyIntersectionWithType(TypeId subTy, const IntersectionTypeVar* uv, TypeId superTy, bool cacheEnabled, bool isFunctionCall) @@ -832,11 +832,11 @@ void Unifier::tryUnifyIntersectionWithType(TypeId subTy, const IntersectionTypeV if (subNorm && superNorm) tryUnifyNormalizedTypes(subTy, superTy, *subNorm, *superNorm, "none of the intersection parts are compatible"); else - reportError(TypeError{location, UnificationTooComplex{}}); + reportError(location, UnificationTooComplex{}); } else if (!found) { - reportError(TypeError{location, TypeMismatch{superTy, subTy, "none of the intersection parts are compatible"}}); + reportError(location, TypeMismatch{superTy, subTy, "none of the intersection parts are compatible"}); } } @@ -848,37 +848,37 @@ void Unifier::tryUnifyNormalizedTypes( if (get(superNorm.tops) || get(superNorm.tops) || get(subNorm.tops)) return; else if (get(subNorm.tops)) - return reportError(TypeError{location, TypeMismatch{superTy, subTy, reason, error}}); + return reportError(location, TypeMismatch{superTy, subTy, reason, error}); if (get(subNorm.errors)) if (!get(superNorm.errors)) - return reportError(TypeError{location, TypeMismatch{superTy, subTy, reason, error}}); + return reportError(location, TypeMismatch{superTy, subTy, reason, error}); if (get(subNorm.booleans)) { if (!get(superNorm.booleans)) - return reportError(TypeError{location, TypeMismatch{superTy, subTy, reason, error}}); + return reportError(location, TypeMismatch{superTy, subTy, reason, error}); } else if (const SingletonTypeVar* stv = get(subNorm.booleans)) { if (!get(superNorm.booleans) && stv != get(superNorm.booleans)) - return reportError(TypeError{location, TypeMismatch{superTy, subTy, reason, error}}); + return reportError(location, TypeMismatch{superTy, subTy, reason, error}); } if (get(subNorm.nils)) if (!get(superNorm.nils)) - return reportError(TypeError{location, TypeMismatch{superTy, subTy, reason, error}}); + return reportError(location, TypeMismatch{superTy, subTy, reason, error}); if (get(subNorm.numbers)) if (!get(superNorm.numbers)) - return reportError(TypeError{location, TypeMismatch{superTy, subTy, reason, error}}); + return reportError(location, TypeMismatch{superTy, subTy, reason, error}); if (!isSubtype(subNorm.strings, superNorm.strings)) - return reportError(TypeError{location, TypeMismatch{superTy, subTy, reason, error}}); + return reportError(location, TypeMismatch{superTy, subTy, reason, error}); if (get(subNorm.threads)) if (!get(superNorm.errors)) - return reportError(TypeError{location, TypeMismatch{superTy, subTy, reason, error}}); + return reportError(location, TypeMismatch{superTy, subTy, reason, error}); for (TypeId subClass : subNorm.classes) { @@ -894,7 +894,7 @@ void Unifier::tryUnifyNormalizedTypes( } } if (!found) - return reportError(TypeError{location, TypeMismatch{superTy, subTy, reason, error}}); + return reportError(location, TypeMismatch{superTy, subTy, reason, error}); } for (TypeId subTable : subNorm.tables) @@ -919,21 +919,19 @@ void Unifier::tryUnifyNormalizedTypes( return reportError(*e); } if (!found) - return reportError(TypeError{location, TypeMismatch{superTy, subTy, reason, error}}); + return reportError(location, TypeMismatch{superTy, subTy, reason, error}); } - if (subNorm.functions) + if (!subNorm.functions.isNever()) { - if (!superNorm.functions) - return reportError(TypeError{location, TypeMismatch{superTy, subTy, reason, error}}); - if (superNorm.functions->empty()) - return; - for (TypeId superFun : *superNorm.functions) + if (superNorm.functions.isNever()) + return reportError(location, TypeMismatch{superTy, subTy, reason, error}); + for (TypeId superFun : *superNorm.functions.parts) { Unifier innerState = makeChildUnifier(); const FunctionTypeVar* superFtv = get(superFun); if (!superFtv) - return reportError(TypeError{location, TypeMismatch{superTy, subTy, reason, error}}); + return reportError(location, TypeMismatch{superTy, subTy, reason, error}); TypePackId tgt = innerState.tryApplyOverloadedFunction(subTy, subNorm.functions, superFtv->argTypes); innerState.tryUnify_(tgt, superFtv->retTypes); if (innerState.errors.empty()) @@ -941,7 +939,7 @@ void Unifier::tryUnifyNormalizedTypes( else if (auto e = hasUnificationTooComplex(innerState.errors)) return reportError(*e); else - return reportError(TypeError{location, TypeMismatch{superTy, subTy, reason, error}}); + return reportError(location, TypeMismatch{superTy, subTy, reason, error}); } } @@ -959,15 +957,15 @@ void Unifier::tryUnifyNormalizedTypes( TypePackId Unifier::tryApplyOverloadedFunction(TypeId function, const NormalizedFunctionType& overloads, TypePackId args) { - if (!overloads || overloads->empty()) + if (overloads.isNever()) { - reportError(TypeError{location, CannotCallNonFunction{function}}); + reportError(location, CannotCallNonFunction{function}); return singletonTypes->errorRecoveryTypePack(); } std::optional result; const FunctionTypeVar* firstFun = nullptr; - for (TypeId overload : *overloads) + for (TypeId overload : *overloads.parts) { if (const FunctionTypeVar* ftv = get(overload)) { @@ -1015,12 +1013,12 @@ TypePackId Unifier::tryApplyOverloadedFunction(TypeId function, const Normalized // TODO: better error reporting? // The logic for error reporting overload resolution // is currently over in TypeInfer.cpp, should we move it? - reportError(TypeError{location, GenericError{"No matching overload."}}); + reportError(location, GenericError{"No matching overload."}); return singletonTypes->errorRecoveryTypePack(firstFun->retTypes); } else { - reportError(TypeError{location, CannotCallNonFunction{function}}); + reportError(location, CannotCallNonFunction{function}); return singletonTypes->errorRecoveryTypePack(); } } @@ -1199,7 +1197,7 @@ void Unifier::tryUnify_(TypePackId subTp, TypePackId superTp, bool isFunctionCal if (sharedState.counters.iterationLimit > 0 && sharedState.counters.iterationLimit < sharedState.counters.iterationCount) { - reportError(TypeError{location, UnificationTooComplex{}}); + reportError(location, UnificationTooComplex{}); return; } @@ -1372,7 +1370,7 @@ void Unifier::tryUnify_(TypePackId subTp, TypePackId superTp, bool isFunctionCal size_t actualSize = size(subTp); if (ctx == CountMismatch::FunctionResult || ctx == CountMismatch::ExprListResult) std::swap(expectedSize, actualSize); - reportError(TypeError{location, CountMismatch{expectedSize, std::nullopt, actualSize, ctx}}); + reportError(location, CountMismatch{expectedSize, std::nullopt, actualSize, ctx}); while (superIter.good()) { @@ -1394,9 +1392,9 @@ void Unifier::tryUnify_(TypePackId subTp, TypePackId superTp, bool isFunctionCal else { if (FFlag::LuauReportTypeMismatchForTypePackUnificationFailure) - reportError(TypeError{location, TypePackMismatch{subTp, superTp}}); + reportError(location, TypePackMismatch{subTp, superTp}); else - reportError(TypeError{location, GenericError{"Failed to unify type packs"}}); + reportError(location, GenericError{"Failed to unify type packs"}); } } @@ -1408,7 +1406,7 @@ void Unifier::tryUnifyPrimitives(TypeId subTy, TypeId superTy) ice("passed non primitive types to unifyPrimitives"); if (superPrim->type != subPrim->type) - reportError(TypeError{location, TypeMismatch{superTy, subTy}}); + reportError(location, TypeMismatch{superTy, subTy}); } void Unifier::tryUnifySingletons(TypeId subTy, TypeId superTy) @@ -1429,7 +1427,7 @@ void Unifier::tryUnifySingletons(TypeId subTy, TypeId superTy) if (superPrim && superPrim->type == PrimitiveTypeVar::String && get(subSingleton) && variance == Covariant) return; - reportError(TypeError{location, TypeMismatch{superTy, subTy}}); + reportError(location, TypeMismatch{superTy, subTy}); } void Unifier::tryUnifyFunctions(TypeId subTy, TypeId superTy, bool isFunctionCall) @@ -1465,21 +1463,21 @@ void Unifier::tryUnifyFunctions(TypeId subTy, TypeId superTy, bool isFunctionCal } else { - reportError(TypeError{location, UnificationTooComplex{}}); + reportError(location, UnificationTooComplex{}); } } else if (numGenerics != subFunction->generics.size()) { numGenerics = std::min(superFunction->generics.size(), subFunction->generics.size()); - reportError(TypeError{location, TypeMismatch{superTy, subTy, "different number of generic type parameters"}}); + reportError(location, TypeMismatch{superTy, subTy, "different number of generic type parameters"}); } if (numGenericPacks != subFunction->genericPacks.size()) { numGenericPacks = std::min(superFunction->genericPacks.size(), subFunction->genericPacks.size()); - reportError(TypeError{location, TypeMismatch{superTy, subTy, "different number of generic type pack parameters"}}); + reportError(location, TypeMismatch{superTy, subTy, "different number of generic type pack parameters"}); } for (size_t i = 0; i < numGenerics; i++) @@ -1506,11 +1504,10 @@ void Unifier::tryUnifyFunctions(TypeId subTy, TypeId superTy, bool isFunctionCal if (auto e = hasUnificationTooComplex(innerState.errors)) reportError(*e); else if (!innerState.errors.empty() && innerState.firstPackErrorPos) - reportError( - TypeError{location, TypeMismatch{superTy, subTy, format("Argument #%d type is not compatible.", *innerState.firstPackErrorPos), - innerState.errors.front()}}); + reportError(location, TypeMismatch{superTy, subTy, format("Argument #%d type is not compatible.", *innerState.firstPackErrorPos), + innerState.errors.front()}); else if (!innerState.errors.empty()) - reportError(TypeError{location, TypeMismatch{superTy, subTy, "", innerState.errors.front()}}); + reportError(location, TypeMismatch{superTy, subTy, "", innerState.errors.front()}); innerState.ctx = CountMismatch::FunctionResult; innerState.tryUnify_(subFunction->retTypes, superFunction->retTypes); @@ -1520,13 +1517,12 @@ void Unifier::tryUnifyFunctions(TypeId subTy, TypeId superTy, bool isFunctionCal if (auto e = hasUnificationTooComplex(innerState.errors)) reportError(*e); else if (!innerState.errors.empty() && size(superFunction->retTypes) == 1 && finite(superFunction->retTypes)) - reportError(TypeError{location, TypeMismatch{superTy, subTy, "Return type is not compatible.", innerState.errors.front()}}); + reportError(location, TypeMismatch{superTy, subTy, "Return type is not compatible.", innerState.errors.front()}); else if (!innerState.errors.empty() && innerState.firstPackErrorPos) - reportError( - TypeError{location, TypeMismatch{superTy, subTy, format("Return #%d type is not compatible.", *innerState.firstPackErrorPos), - innerState.errors.front()}}); + reportError(location, TypeMismatch{superTy, subTy, format("Return #%d type is not compatible.", *innerState.firstPackErrorPos), + innerState.errors.front()}); else if (!innerState.errors.empty()) - reportError(TypeError{location, TypeMismatch{superTy, subTy, "", innerState.errors.front()}}); + reportError(location, TypeMismatch{superTy, subTy, "", innerState.errors.front()}); } log.concat(std::move(innerState.log)); @@ -1608,7 +1604,7 @@ void Unifier::tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) } else { - reportError(TypeError{location, UnificationTooComplex{}}); + reportError(location, UnificationTooComplex{}); } } } @@ -1626,7 +1622,7 @@ void Unifier::tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) if (!missingProperties.empty()) { - reportError(TypeError{location, MissingProperties{superTy, subTy, std::move(missingProperties)}}); + reportError(location, MissingProperties{superTy, subTy, std::move(missingProperties)}); return; } } @@ -1644,7 +1640,7 @@ void Unifier::tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) if (!extraProperties.empty()) { - reportError(TypeError{location, MissingProperties{superTy, subTy, std::move(extraProperties), MissingProperties::Extra}}); + reportError(location, MissingProperties{superTy, subTy, std::move(extraProperties), MissingProperties::Extra}); return; } } @@ -1825,13 +1821,13 @@ void Unifier::tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) if (!missingProperties.empty()) { - reportError(TypeError{location, MissingProperties{superTy, subTy, std::move(missingProperties)}}); + reportError(location, MissingProperties{superTy, subTy, std::move(missingProperties)}); return; } if (!extraProperties.empty()) { - reportError(TypeError{location, MissingProperties{superTy, subTy, std::move(extraProperties), MissingProperties::Extra}}); + reportError(location, MissingProperties{superTy, subTy, std::move(extraProperties), MissingProperties::Extra}); return; } @@ -1867,14 +1863,14 @@ void Unifier::tryUnifyScalarShape(TypeId subTy, TypeId superTy, bool reversed) std::swap(subTy, superTy); if (auto ttv = log.get(superTy); !ttv || ttv->state != TableState::Free) - return reportError(TypeError{location, TypeMismatch{osuperTy, osubTy}}); + return reportError(location, TypeMismatch{osuperTy, osubTy}); auto fail = [&](std::optional e) { std::string reason = "The former's metatable does not satisfy the requirements."; if (e) - reportError(TypeError{location, TypeMismatch{osuperTy, osubTy, reason, *e}}); + reportError(location, TypeMismatch{osuperTy, osubTy, reason, *e}); else - reportError(TypeError{location, TypeMismatch{osuperTy, osubTy, reason}}); + reportError(location, TypeMismatch{osuperTy, osubTy, reason}); }; // Given t1 where t1 = { lower: (t1) -> (a, b...) } @@ -1906,7 +1902,7 @@ void Unifier::tryUnifyScalarShape(TypeId subTy, TypeId superTy, bool reversed) } } - reportError(TypeError{location, TypeMismatch{osuperTy, osubTy}}); + reportError(location, TypeMismatch{osuperTy, osubTy}); return; } @@ -1947,7 +1943,7 @@ void Unifier::tryUnifyWithMetatable(TypeId subTy, TypeId superTy, bool reversed) if (auto e = hasUnificationTooComplex(innerState.errors)) reportError(*e); else if (!innerState.errors.empty()) - reportError(TypeError{location, TypeMismatch{reversed ? subTy : superTy, reversed ? superTy : subTy, "", innerState.errors.front()}}); + reportError(location, TypeMismatch{reversed ? subTy : superTy, reversed ? superTy : subTy, "", innerState.errors.front()}); log.concat(std::move(innerState.log)); } @@ -2024,9 +2020,9 @@ void Unifier::tryUnifyWithClass(TypeId subTy, TypeId superTy, bool reversed) auto fail = [&]() { if (!reversed) - reportError(TypeError{location, TypeMismatch{superTy, subTy}}); + reportError(location, TypeMismatch{superTy, subTy}); else - reportError(TypeError{location, TypeMismatch{subTy, superTy}}); + reportError(location, TypeMismatch{subTy, superTy}); }; const ClassTypeVar* superClass = get(superTy); @@ -2071,7 +2067,7 @@ void Unifier::tryUnifyWithClass(TypeId subTy, TypeId superTy, bool reversed) if (!classProp) { ok = false; - reportError(TypeError{location, UnknownProperty{superTy, propName}}); + reportError(location, UnknownProperty{superTy, propName}); } else { @@ -2095,7 +2091,7 @@ void Unifier::tryUnifyWithClass(TypeId subTy, TypeId superTy, bool reversed) { ok = false; std::string msg = "Class " + superClass->name + " does not have an indexer"; - reportError(TypeError{location, GenericError{msg}}); + reportError(location, GenericError{msg}); } if (!ok) @@ -2116,13 +2112,13 @@ void Unifier::tryUnifyTypeWithNegation(TypeId subTy, TypeId superTy) const NormalizedType* subNorm = normalizer->normalize(subTy); const NormalizedType* superNorm = normalizer->normalize(superTy); if (!subNorm || !superNorm) - return reportError(TypeError{location, UnificationTooComplex{}}); + return reportError(location, UnificationTooComplex{}); // T & queue, DenseHashSet& seenTypePacks, Unifier& state, TypePackId a, TypePackId anyTypePack) @@ -2195,7 +2191,7 @@ void Unifier::tryUnifyVariadics(TypePackId subTp, TypePackId superTp, bool rever } else if (get(tail)) { - reportError(TypeError{location, GenericError{"Cannot unify variadic and generic packs"}}); + reportError(location, GenericError{"Cannot unify variadic and generic packs"}); } else if (get(tail)) { @@ -2209,7 +2205,7 @@ void Unifier::tryUnifyVariadics(TypePackId subTp, TypePackId superTp, bool rever } else { - reportError(TypeError{location, GenericError{"Failed to unify variadic packs"}}); + reportError(location, GenericError{"Failed to unify variadic packs"}); } } @@ -2351,7 +2347,7 @@ bool Unifier::occursCheck(DenseHashSet& seen, TypeId needle, TypeId hays if (needle == haystack) { - reportError(TypeError{location, OccursCheckFailed{}}); + reportError(location, OccursCheckFailed{}); log.replace(needle, *singletonTypes->errorRecoveryType()); return true; @@ -2402,7 +2398,7 @@ bool Unifier::occursCheck(DenseHashSet& seen, TypePackId needle, Typ { if (needle == haystack) { - reportError(TypeError{location, OccursCheckFailed{}}); + reportError(location, OccursCheckFailed{}); log.replace(needle, *singletonTypes->errorRecoveryTypePack()); return true; @@ -2423,18 +2419,31 @@ bool Unifier::occursCheck(DenseHashSet& seen, TypePackId needle, Typ Unifier Unifier::makeChildUnifier() { Unifier u = Unifier{normalizer, mode, scope, location, variance, &log}; - u.anyIsTop = anyIsTop; u.normalize = normalize; + u.useScopes = useScopes; return u; } // A utility function that appends the given error to the unifier's error log. // This allows setting a breakpoint wherever the unifier reports an error. +// +// Note: report error accepts its arguments by value intentionally to reduce the stack usage of functions which call `reportError`. +void Unifier::reportError(Location location, TypeErrorData data) +{ + errors.emplace_back(std::move(location), std::move(data)); +} + +// A utility function that appends the given error to the unifier's error log. +// This allows setting a breakpoint wherever the unifier reports an error. +// +// Note: to conserve stack space in calling functions it is generally preferred to call `Unifier::reportError(Location location, TypeErrorData data)` +// instead of this method. void Unifier::reportError(TypeError err) { errors.push_back(std::move(err)); } + bool Unifier::isNonstrictMode() const { return (mode == Mode::Nonstrict) || (mode == Mode::NoCheck); @@ -2445,7 +2454,7 @@ void Unifier::checkChildUnifierTypeMismatch(const ErrorVec& innerErrors, TypeId if (auto e = hasUnificationTooComplex(innerErrors)) reportError(*e); else if (!innerErrors.empty()) - reportError(TypeError{location, TypeMismatch{wantedType, givenType}}); + reportError(location, TypeMismatch{wantedType, givenType}); } void Unifier::checkChildUnifierTypeMismatch(const ErrorVec& innerErrors, const std::string& prop, TypeId wantedType, TypeId givenType) diff --git a/Ast/src/Parser.cpp b/Ast/src/Parser.cpp index 85c5f5c6..4c0cc125 100644 --- a/Ast/src/Parser.cpp +++ b/Ast/src/Parser.cpp @@ -25,6 +25,7 @@ LUAU_DYNAMIC_FASTFLAGVARIABLE(LuaReportParseIntegerIssues, false) LUAU_FASTFLAGVARIABLE(LuauInterpolatedStringBaseSupport, false) LUAU_FASTFLAGVARIABLE(LuauCommaParenWarnings, false) +LUAU_FASTFLAGVARIABLE(LuauTableConstructorRecovery, false) bool lua_telemetry_parsed_out_of_range_bin_integer = false; bool lua_telemetry_parsed_out_of_range_hex_integer = false; @@ -2310,9 +2311,13 @@ AstExpr* Parser::parseTableConstructor() MatchLexeme matchBrace = lexer.current(); expectAndConsume('{', "table literal"); + unsigned lastElementIndent = 0; while (lexer.current().type != '}') { + if (FFlag::LuauTableConstructorRecovery) + lastElementIndent = lexer.current().location.begin.column; + if (lexer.current().type == '[') { MatchLexeme matchLocationBracket = lexer.current(); @@ -2357,10 +2362,14 @@ AstExpr* Parser::parseTableConstructor() { nextLexeme(); } - else + else if (FFlag::LuauTableConstructorRecovery && (lexer.current().type == '[' || lexer.current().type == Lexeme::Name) && + lexer.current().location.begin.column == lastElementIndent) { - if (lexer.current().type != '}') - break; + report(lexer.current().location, "Expected ',' after table constructor element"); + } + else if (lexer.current().type != '}') + { + break; } } diff --git a/CLI/Repl.cpp b/CLI/Repl.cpp index 87e19db8..e567725e 100644 --- a/CLI/Repl.cpp +++ b/CLI/Repl.cpp @@ -978,7 +978,8 @@ int replMain(int argc, char** argv) if (compileFormat == CompileFormat::Null) printf("Compiled %d KLOC into %d KB bytecode\n", int(stats.lines / 1000), int(stats.bytecode / 1024)); else if (compileFormat == CompileFormat::CodegenNull) - printf("Compiled %d KLOC into %d KB bytecode => %d KB native code\n", int(stats.lines / 1000), int(stats.bytecode / 1024), int(stats.codegen / 1024)); + printf("Compiled %d KLOC into %d KB bytecode => %d KB native code\n", int(stats.lines / 1000), int(stats.bytecode / 1024), + int(stats.codegen / 1024)); return failed ? 1 : 0; } diff --git a/CodeGen/include/Luau/AddressA64.h b/CodeGen/include/Luau/AddressA64.h new file mode 100644 index 00000000..351e6715 --- /dev/null +++ b/CodeGen/include/Luau/AddressA64.h @@ -0,0 +1,52 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +#pragma once + +#include "Luau/RegisterA64.h" + +namespace Luau +{ +namespace CodeGen +{ + +enum class AddressKindA64 : uint8_t +{ + imm, // reg + imm + reg, // reg + reg + + // TODO: + // reg + reg << shift + // reg + sext(reg) << shift + // reg + uext(reg) << shift + // pc + offset +}; + +struct AddressA64 +{ + AddressA64(RegisterA64 base, int off = 0) + : kind(AddressKindA64::imm) + , base(base) + , offset(xzr) + , data(off) + { + LUAU_ASSERT(base.kind == KindA64::x); + LUAU_ASSERT(off >= 0 && off < 4096); + } + + AddressA64(RegisterA64 base, RegisterA64 offset) + : kind(AddressKindA64::reg) + , base(base) + , offset(offset) + , data(0) + { + LUAU_ASSERT(base.kind == KindA64::x); + LUAU_ASSERT(offset.kind == KindA64::x); + } + + AddressKindA64 kind; + RegisterA64 base; + RegisterA64 offset; + int data; +}; + +} // namespace CodeGen +} // namespace Luau diff --git a/CodeGen/include/Luau/AssemblyBuilderA64.h b/CodeGen/include/Luau/AssemblyBuilderA64.h new file mode 100644 index 00000000..9a1402be --- /dev/null +++ b/CodeGen/include/Luau/AssemblyBuilderA64.h @@ -0,0 +1,144 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +#pragma once + +#include "Luau/RegisterA64.h" +#include "Luau/AddressA64.h" +#include "Luau/ConditionA64.h" +#include "Luau/Label.h" + +#include +#include + +namespace Luau +{ +namespace CodeGen +{ + +class AssemblyBuilderA64 +{ +public: + explicit AssemblyBuilderA64(bool logText); + ~AssemblyBuilderA64(); + + // Moves + void mov(RegisterA64 dst, RegisterA64 src); + void mov(RegisterA64 dst, uint16_t src, int shift = 0); + void movk(RegisterA64 dst, uint16_t src, int shift = 0); + + // Arithmetics + void add(RegisterA64 dst, RegisterA64 src1, RegisterA64 src2, int shift = 0); + void add(RegisterA64 dst, RegisterA64 src1, int src2); + void sub(RegisterA64 dst, RegisterA64 src1, RegisterA64 src2, int shift = 0); + void sub(RegisterA64 dst, RegisterA64 src1, int src2); + void neg(RegisterA64 dst, RegisterA64 src); + + // Comparisons + // Note: some arithmetic instructions also have versions that update flags (ADDS etc) but we aren't using them atm + // TODO: add cmp + + // Binary + // Note: shifted-register support and bitfield operations are omitted for simplicity + // TODO: support immediate arguments (they have odd encoding and forbid many values) + // TODO: support not variants for and/or/eor (required to support not...) + void and_(RegisterA64 dst, RegisterA64 src1, RegisterA64 src2); + void orr(RegisterA64 dst, RegisterA64 src1, RegisterA64 src2); + void eor(RegisterA64 dst, RegisterA64 src1, RegisterA64 src2); + void lsl(RegisterA64 dst, RegisterA64 src1, RegisterA64 src2); + void lsr(RegisterA64 dst, RegisterA64 src1, RegisterA64 src2); + void asr(RegisterA64 dst, RegisterA64 src1, RegisterA64 src2); + void ror(RegisterA64 dst, RegisterA64 src1, RegisterA64 src2); + void clz(RegisterA64 dst, RegisterA64 src); + void rbit(RegisterA64 dst, RegisterA64 src); + + // Load + // Note: paired loads are currently omitted for simplicity + void ldr(RegisterA64 dst, AddressA64 src); + void ldrb(RegisterA64 dst, AddressA64 src); + void ldrh(RegisterA64 dst, AddressA64 src); + void ldrsb(RegisterA64 dst, AddressA64 src); + void ldrsh(RegisterA64 dst, AddressA64 src); + void ldrsw(RegisterA64 dst, AddressA64 src); + + // Store + void str(RegisterA64 src, AddressA64 dst); + void strb(RegisterA64 src, AddressA64 dst); + void strh(RegisterA64 src, AddressA64 dst); + + // Control flow + // Note: tbz/tbnz are currently not supported because they have 15-bit offsets and we don't support branch thunks + void b(ConditionA64 cond, Label& label); + void cbz(RegisterA64 src, Label& label); + void cbnz(RegisterA64 src, Label& label); + void ret(); + + // Run final checks + bool finalize(); + + // Places a label at current location and returns it + Label setLabel(); + + // Assigns label position to the current location + void setLabel(Label& label); + + void logAppend(const char* fmt, ...) LUAU_PRINTF_ATTR(2, 3); + + uint32_t getCodeSize() const; + + // Resulting data and code that need to be copied over one after the other + // The *end* of 'data' has to be aligned to 16 bytes, this will also align 'code' + std::vector data; + std::vector code; + + std::string text; + + const bool logText = false; + +private: + // Instruction archetypes + void place0(const char* name, uint32_t word); + void placeSR3(const char* name, RegisterA64 dst, RegisterA64 src1, RegisterA64 src2, uint8_t op, int shift = 0); + void placeSR2(const char* name, RegisterA64 dst, RegisterA64 src, uint8_t op); + void placeR3(const char* name, RegisterA64 dst, RegisterA64 src1, RegisterA64 src2, uint8_t op, uint8_t op2); + void placeR1(const char* name, RegisterA64 dst, RegisterA64 src, uint32_t op); + void placeI12(const char* name, RegisterA64 dst, RegisterA64 src1, int src2, uint8_t op); + void placeI16(const char* name, RegisterA64 dst, int src, uint8_t op, int shift = 0); + void placeA(const char* name, RegisterA64 dst, AddressA64 src, uint8_t op, uint8_t size); + void placeBC(const char* name, Label& label, uint8_t op, uint8_t cond); + void placeBR(const char* name, Label& label, uint8_t op, RegisterA64 cond); + + void place(uint32_t word); + void placeLabel(Label& label); + + void commit(); + LUAU_NOINLINE void extend(); + + // Data + size_t allocateData(size_t size, size_t align); + + // Logging of assembly in text form + LUAU_NOINLINE void log(const char* opcode); + LUAU_NOINLINE void log(const char* opcode, RegisterA64 dst, RegisterA64 src1, RegisterA64 src2, int shift = 0); + LUAU_NOINLINE void log(const char* opcode, RegisterA64 dst, RegisterA64 src1, int src2); + LUAU_NOINLINE void log(const char* opcode, RegisterA64 dst, RegisterA64 src); + LUAU_NOINLINE void log(const char* opcode, RegisterA64 dst, int src, int shift = 0); + LUAU_NOINLINE void log(const char* opcode, RegisterA64 dst, AddressA64 src); + LUAU_NOINLINE void log(const char* opcode, RegisterA64 src, Label label); + LUAU_NOINLINE void log(const char* opcode, Label label); + LUAU_NOINLINE void log(Label label); + LUAU_NOINLINE void log(RegisterA64 reg); + LUAU_NOINLINE void log(AddressA64 addr); + + uint32_t nextLabel = 1; + std::vector(a) -> a' could not be converted into '(number) -> number'; different number of generic type parameters)"); -// // this error message is not great since the underlying issue is that the context is invariant, + // LUAU_REQUIRE_ERRORS(result); + // CHECK_EQ(toString(result.errors[0]), R"(Type 't' could not be converted into '{| m: (number) -> number |}' + // caused by: + // Property 'm' is not compatible. Type '(a) -> a' could not be converted into '(number) -> number'; different number of generic type + // parameters)"); + // // this error message is not great since the underlying issue is that the context is invariant, // and `(number) -> number` cannot be a subtype of `(a) -> a`. } @@ -3335,10 +3335,7 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "tables_should_be_fully_populated") ToStringOptions opts; opts.exhaustive = true; - if (FFlag::LuauSpecialTypesAsterisked) - CHECK_EQ("{ x: *error-type*, y: number }", toString(requireType("t"), opts)); - else - CHECK_EQ("{ x: , y: number }", toString(requireType("t"), opts)); + CHECK_EQ("{ x: *error-type*, y: number }", toString(requireType("t"), opts)); } TEST_SUITE_END(); diff --git a/tests/TypeInfer.test.cpp b/tests/TypeInfer.test.cpp index dff9649f..6c7201a6 100644 --- a/tests/TypeInfer.test.cpp +++ b/tests/TypeInfer.test.cpp @@ -17,7 +17,6 @@ LUAU_FASTFLAG(LuauFixLocationSpanTableIndexExpr); LUAU_FASTFLAG(DebugLuauDeferredConstraintResolution); LUAU_FASTFLAG(LuauInstantiateInSubtyping); -LUAU_FASTFLAG(LuauSpecialTypesAsterisked); using namespace Luau; @@ -238,20 +237,10 @@ TEST_CASE_FIXTURE(Fixture, "type_errors_infer_types") // TODO: Should we assert anything about these tests when DCR is being used? if (!FFlag::DebugLuauDeferredConstraintResolution) { - if (FFlag::LuauSpecialTypesAsterisked) - { - CHECK_EQ("*error-type*", toString(requireType("c"))); - CHECK_EQ("*error-type*", toString(requireType("d"))); - CHECK_EQ("*error-type*", toString(requireType("e"))); - CHECK_EQ("*error-type*", toString(requireType("f"))); - } - else - { - CHECK_EQ("", toString(requireType("c"))); - CHECK_EQ("", toString(requireType("d"))); - CHECK_EQ("", toString(requireType("e"))); - CHECK_EQ("", toString(requireType("f"))); - } + CHECK_EQ("*error-type*", toString(requireType("c"))); + CHECK_EQ("*error-type*", toString(requireType("d"))); + CHECK_EQ("*error-type*", toString(requireType("e"))); + CHECK_EQ("*error-type*", toString(requireType("f"))); } } @@ -662,10 +651,7 @@ TEST_CASE_FIXTURE(Fixture, "no_stack_overflow_from_isoptional") std::optional t0 = getMainModule()->getModuleScope()->lookupType("t0"); REQUIRE(t0); - if (FFlag::LuauSpecialTypesAsterisked) - CHECK_EQ("*error-type*", toString(t0->type)); - else - CHECK_EQ("", toString(t0->type)); + CHECK_EQ("*error-type*", toString(t0->type)); auto it = std::find_if(result.errors.begin(), result.errors.end(), [](TypeError& err) { return get(err); diff --git a/tests/TypeInfer.tryUnify.test.cpp b/tests/TypeInfer.tryUnify.test.cpp index c178d2a4..f04a3d95 100644 --- a/tests/TypeInfer.tryUnify.test.cpp +++ b/tests/TypeInfer.tryUnify.test.cpp @@ -9,8 +9,6 @@ using namespace Luau; -LUAU_FASTFLAG(LuauSpecialTypesAsterisked) - struct TryUnifyFixture : Fixture { TypeArena arena; @@ -124,10 +122,7 @@ TEST_CASE_FIXTURE(TryUnifyFixture, "members_of_failed_typepack_unification_are_u LUAU_REQUIRE_ERROR_COUNT(1, result); CHECK_EQ("a", toString(requireType("a"))); - if (FFlag::LuauSpecialTypesAsterisked) - CHECK_EQ("*error-type*", toString(requireType("b"))); - else - CHECK_EQ("", toString(requireType("b"))); + CHECK_EQ("*error-type*", toString(requireType("b"))); } TEST_CASE_FIXTURE(TryUnifyFixture, "result_of_failed_typepack_unification_is_constrained") @@ -142,10 +137,7 @@ TEST_CASE_FIXTURE(TryUnifyFixture, "result_of_failed_typepack_unification_is_con LUAU_REQUIRE_ERROR_COUNT(1, result); CHECK_EQ("a", toString(requireType("a"))); - if (FFlag::LuauSpecialTypesAsterisked) - CHECK_EQ("*error-type*", toString(requireType("b"))); - else - CHECK_EQ("", toString(requireType("b"))); + CHECK_EQ("*error-type*", toString(requireType("b"))); CHECK_EQ("number", toString(requireType("c"))); } diff --git a/tests/TypeInfer.unionTypes.test.cpp b/tests/TypeInfer.unionTypes.test.cpp index dc551634..0c25386f 100644 --- a/tests/TypeInfer.unionTypes.test.cpp +++ b/tests/TypeInfer.unionTypes.test.cpp @@ -6,8 +6,6 @@ #include "doctest.h" -LUAU_FASTFLAG(LuauSpecialTypesAsterisked) - using namespace Luau; TEST_SUITE_BEGIN("UnionTypes"); @@ -199,10 +197,7 @@ TEST_CASE_FIXTURE(Fixture, "index_on_a_union_type_with_missing_property") CHECK_EQ(mup->missing[0], *bTy); CHECK_EQ(mup->key, "x"); - if (FFlag::LuauSpecialTypesAsterisked) - CHECK_EQ("*error-type*", toString(requireType("r"))); - else - CHECK_EQ("", toString(requireType("r"))); + CHECK_EQ("*error-type*", toString(requireType("r"))); } TEST_CASE_FIXTURE(Fixture, "index_on_a_union_type_with_one_property_of_type_any") diff --git a/tests/Variant.test.cpp b/tests/Variant.test.cpp index aa0731ca..83eec519 100644 --- a/tests/Variant.test.cpp +++ b/tests/Variant.test.cpp @@ -217,4 +217,35 @@ TEST_CASE("Visit") CHECK(r3 == "1231147"); } +struct MoveOnly +{ + MoveOnly() = default; + + MoveOnly(const MoveOnly&) = delete; + MoveOnly& operator=(const MoveOnly&) = delete; + + MoveOnly(MoveOnly&&) = default; + MoveOnly& operator=(MoveOnly&&) = default; +}; + +TEST_CASE("Move") +{ + Variant v1 = MoveOnly{}; + Variant v2 = std::move(v1); +} + +TEST_CASE("MoveWithCopyableAlternative") +{ + Variant v1 = std::string{"Hello, world! I am longer than a normal hello world string to avoid SSO."}; + Variant v2 = std::move(v1); + + std::string* s1 = get_if(&v1); + REQUIRE(s1); + CHECK(*s1 == ""); + + std::string* s2 = get_if(&v2); + REQUIRE(s2); + CHECK(*s2 == "Hello, world! I am longer than a normal hello world string to avoid SSO."); +} + TEST_SUITE_END(); diff --git a/tools/faillist.txt b/tools/faillist.txt index a4c05b7b..4ac2b357 100644 --- a/tools/faillist.txt +++ b/tools/faillist.txt @@ -10,7 +10,9 @@ AnnotationTests.too_many_type_params AnnotationTests.two_type_params AnnotationTests.unknown_type_reference_generates_error AstQuery.last_argument_function_call_type +AstQuery::getDocumentationSymbolAtPosition.overloaded_class_method AstQuery::getDocumentationSymbolAtPosition.overloaded_fn +AstQuery::getDocumentationSymbolAtPosition.table_overloaded_function_prop AutocompleteTest.autocomplete_first_function_arg_expected_type AutocompleteTest.autocomplete_interpolated_string AutocompleteTest.autocomplete_oop_implicit_self @@ -106,17 +108,14 @@ GenericsTests.correctly_instantiate_polymorphic_member_functions GenericsTests.do_not_infer_generic_functions GenericsTests.duplicate_generic_type_packs GenericsTests.duplicate_generic_types -GenericsTests.factories_of_generics GenericsTests.generic_argument_count_too_few GenericsTests.generic_argument_count_too_many GenericsTests.generic_factories -GenericsTests.generic_functions_in_types GenericsTests.generic_functions_should_be_memory_safe GenericsTests.generic_table_method GenericsTests.generic_type_pack_parentheses GenericsTests.generic_type_pack_unification1 GenericsTests.generic_type_pack_unification2 -GenericsTests.generic_type_pack_unification3 GenericsTests.higher_rank_polymorphism_should_not_accept_instantiated_arguments GenericsTests.infer_generic_function_function_argument GenericsTests.infer_generic_function_function_argument_overloaded @@ -172,7 +171,6 @@ ProvisionalTests.table_insert_with_a_singleton_argument ProvisionalTests.typeguard_inference_incomplete ProvisionalTests.weirditer_should_not_loop_forever ProvisionalTests.while_body_are_also_refined -RefinementTest.and_constraint RefinementTest.apply_refinements_on_astexprindexexpr_whose_subscript_expr_is_constant_string RefinementTest.assert_a_to_be_truthy_then_assert_a_to_be_number RefinementTest.assert_non_binary_expressions_actually_resolve_constraints @@ -187,28 +185,17 @@ RefinementTest.either_number_or_string RefinementTest.eliminate_subclasses_of_instance RefinementTest.falsiness_of_TruthyPredicate_narrows_into_nil RefinementTest.index_on_a_refined_property -RefinementTest.invert_is_truthy_constraint RefinementTest.invert_is_truthy_constraint_ifelse_expression -RefinementTest.is_truthy_constraint RefinementTest.is_truthy_constraint_ifelse_expression -RefinementTest.lvalue_is_not_nil RefinementTest.merge_should_be_fully_agnostic_of_hashmap_ordering -RefinementTest.narrow_boolean_to_true_or_false RefinementTest.narrow_property_of_a_bounded_variable RefinementTest.narrow_this_large_union RefinementTest.nonoptional_type_can_narrow_to_nil_if_sense_is_true -RefinementTest.not_a_and_not_b -RefinementTest.not_a_and_not_b2 -RefinementTest.not_and_constraint RefinementTest.not_t_or_some_prop_of_t -RefinementTest.or_predicate_with_truthy_predicates -RefinementTest.parenthesized_expressions_are_followed_through RefinementTest.refine_a_property_not_to_be_nil_through_an_intersection_table RefinementTest.refine_the_correct_types_opposite_of_when_a_is_not_number_or_string RefinementTest.refine_unknowns -RefinementTest.term_is_equal_to_an_lvalue RefinementTest.truthy_constraint_on_properties -RefinementTest.type_assertion_expr_carry_its_constraints RefinementTest.type_comparison_ifelse_expression RefinementTest.type_guard_can_filter_for_intersection_of_tables RefinementTest.type_guard_can_filter_for_overloaded_function @@ -271,6 +258,7 @@ TableTests.infer_indexer_from_value_property_in_literal TableTests.inferred_return_type_of_free_table TableTests.inferring_crazy_table_should_also_be_quick TableTests.instantiate_table_cloning_3 +TableTests.invariant_table_properties_means_instantiating_tables_in_assignment_is_unsound TableTests.invariant_table_properties_means_instantiating_tables_in_call_is_unsound TableTests.leaking_bad_metatable_errors TableTests.less_exponential_blowup_please @@ -315,7 +303,6 @@ TableTests.table_subtyping_with_missing_props_dont_report_multiple_errors TableTests.tables_get_names_from_their_locals TableTests.tc_member_function TableTests.tc_member_function_2 -TableTests.top_table_type TableTests.type_mismatch_on_massive_table_is_cut_short TableTests.unification_of_unions_in_a_self_referential_type TableTests.unifying_tables_shouldnt_uaf2 @@ -417,12 +404,12 @@ TypeInferFunctions.too_few_arguments_variadic TypeInferFunctions.too_few_arguments_variadic_generic TypeInferFunctions.too_few_arguments_variadic_generic2 TypeInferFunctions.too_many_arguments +TypeInferFunctions.too_many_arguments_error_location TypeInferFunctions.too_many_return_values TypeInferFunctions.too_many_return_values_in_parentheses TypeInferFunctions.too_many_return_values_no_function TypeInferFunctions.vararg_function_is_quantified TypeInferLoops.for_in_loop_error_on_factory_not_returning_the_right_amount_of_values -TypeInferLoops.for_in_loop_with_custom_iterator TypeInferLoops.for_in_loop_with_next TypeInferLoops.for_in_with_generic_next TypeInferLoops.for_in_with_just_one_iterator_is_ok @@ -430,7 +417,6 @@ TypeInferLoops.loop_iter_no_indexer_nonstrict TypeInferLoops.loop_iter_trailing_nil TypeInferLoops.unreachable_code_after_infinite_loop TypeInferLoops.varlist_declared_by_for_in_loop_should_be_free -TypeInferModules.bound_free_table_export_is_ok TypeInferModules.custom_require_global TypeInferModules.do_not_modify_imported_types TypeInferModules.module_type_conflict @@ -443,6 +429,7 @@ TypeInferOOP.dont_suggest_using_colon_rather_than_dot_if_it_wont_help_2 TypeInferOOP.dont_suggest_using_colon_rather_than_dot_if_not_defined_with_colon TypeInferOOP.inferring_hundreds_of_self_calls_should_not_suffocate_memory TypeInferOOP.methods_are_topologically_sorted +TypeInferOOP.object_constructor_can_refer_to_method_of_self TypeInferOperators.and_or_ternary TypeInferOperators.CallAndOrOfFunctions TypeInferOperators.cannot_compare_tables_that_do_not_have_the_same_metatable diff --git a/tools/lvmexecute_split.py b/tools/lvmexecute_split.py index 48d66cb0..f4a78960 100644 --- a/tools/lvmexecute_split.py +++ b/tools/lvmexecute_split.py @@ -32,6 +32,9 @@ source = """// This file is part of the Luau programming language and is license """ function = "" +signature = "" + +includeInsts = ["LOP_NEWCLOSURE", "LOP_NAMECALL", "LOP_FORGPREP", "LOP_GETVARARGS", "LOP_DUPCLOSURE", "LOP_PREPVARARGS", "LOP_COVERAGE", "LOP_BREAK", "LOP_GETGLOBAL", "LOP_SETGLOBAL", "LOP_GETTABLEKS", "LOP_SETTABLEKS"] state = 0 @@ -44,7 +47,6 @@ for line in input: if match: inst = match[1] signature = "const Instruction* execute_" + inst + "(lua_State* L, const Instruction* pc, StkId base, TValue* k)" - header += signature + ";\n" function = signature + "\n" function += "{\n" function += " [[maybe_unused]] Closure* cl = clvalue(L->ci->func);\n" @@ -84,7 +86,10 @@ for line in input: function = function[:-len(finalline)] function += " return pc;\n}\n" - source += function + "\n" + if inst in includeInsts: + header += signature + ";\n" + source += function + "\n" + state = 0 # skip LUA_CUSTOM_EXECUTION code blocks diff --git a/tools/stack-usage-reporter.py b/tools/stack-usage-reporter.py new file mode 100644 index 00000000..91e74887 --- /dev/null +++ b/tools/stack-usage-reporter.py @@ -0,0 +1,173 @@ +#!/usr/bin/python +# This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details + +# The purpose of this script is to analyze disassembly generated by objdump or +# dumpbin to print (or to compare) the stack usage of functions/methods. +# This is a quickly written script, so it is quite possible it may not handle +# all code properly. +# +# The script expects the user to create a text assembly dump to be passed to +# the script. +# +# objdump Example +# objdump --demangle --disassemble objfile.o > objfile.s +# +# dumpbin Example +# dumpbin /disasm objfile.obj > objfile.s +# +# If the script is passed a single file, then all stack size information that +# is found it printed. If two files are passed, then the script compares the +# stack usage of the two files (useful for A/B comparisons). +# Currently more than two input files are not supported. (But adding support shouldn't +# be very difficult.) +# +# Note: The script only handles x64 disassembly. Supporting x86 is likely +# trivial, but ARM support could be difficult. +# Thus far the script has been tested with MSVC on Win64 and clang on OSX. + +import argparse +import re + +blank_re = re.compile('\s*') + +class LineReader: + def __init__(self, lines): + self.lines = list(reversed(lines)) + def get_line(self): + return self.lines.pop(-1) + def peek_line(self): + return self.lines[-1] + def consume_blank_lines(self): + while blank_re.fullmatch(self.peek_line()): + self.get_line() + def is_empty(self): + return len(self.lines) == 0 + +def parse_objdump_assembly(in_file): + results = {} + text_section_re = re.compile('Disassembly of section __TEXT,__text:\s*') + symbol_re = re.compile('[^<]*<(.*)>:\s*') + stack_alloc = re.compile('.*subq\s*\$(\d*), %rsp\s*') + + lr = LineReader(in_file.readlines()) + + def find_stack_alloc_size(): + while True: + if lr.is_empty(): + return None + if blank_re.fullmatch(lr.peek_line()): + return None + + line = lr.get_line() + mo = stack_alloc.fullmatch(line) + if mo: + lr.consume_blank_lines() + return int(mo.group(1)) + + # Find beginning of disassembly + while not text_section_re.fullmatch(lr.get_line()): + pass + + # Scan for symbols + while not lr.is_empty(): + lr.consume_blank_lines() + if lr.is_empty(): + break + line = lr.get_line() + mo = symbol_re.fullmatch(line) + # Found a symbol + if mo: + symbol = mo.group(1) + stack_size = find_stack_alloc_size() + if stack_size != None: + results[symbol] = stack_size + + return results + +def parse_dumpbin_assembly(in_file): + results = {} + + file_type_re = re.compile('File Type: COFF OBJECT\s*') + symbol_re = re.compile('[^(]*\((.*)\):\s*') + summary_re = re.compile('\s*Summary\s*') + stack_alloc = re.compile('.*sub\s*rsp,([A-Z0-9]*)h\s*') + + lr = LineReader(in_file.readlines()) + + def find_stack_alloc_size(): + while True: + if lr.is_empty(): + return None + if blank_re.fullmatch(lr.peek_line()): + return None + + line = lr.get_line() + mo = stack_alloc.fullmatch(line) + if mo: + lr.consume_blank_lines() + return int(mo.group(1), 16) # return value in decimal + + # Find beginning of disassembly + while not file_type_re.fullmatch(lr.get_line()): + pass + + # Scan for symbols + while not lr.is_empty(): + lr.consume_blank_lines() + if lr.is_empty(): + break + line = lr.get_line() + if summary_re.fullmatch(line): + break + mo = symbol_re.fullmatch(line) + # Found a symbol + if mo: + symbol = mo.group(1) + stack_size = find_stack_alloc_size() + if stack_size != None: + results[symbol] = stack_size + return results + +def main(): + parser = argparse.ArgumentParser(description='Tool used for reporting or comparing the stack usage of functions/methods') + parser.add_argument('--format', choices=['dumpbin', 'objdump'], required=True, help='Specifies the program used to generate the input files') + parser.add_argument('--input', action='append', required=True, help='Input assembly file. This option may be specified multiple times.') + parser.add_argument('--md-output', action='store_true', help='Show table output in markdown format') + parser.add_argument('--only-diffs', action='store_true', help='Only show stack info when it differs between the input files') + args = parser.parse_args() + + parsers = {'dumpbin': parse_dumpbin_assembly, 'objdump' : parse_objdump_assembly} + parse_func = parsers[args.format] + + input_results = [] + for input_name in args.input: + with open(input_name) as in_file: + results = parse_func(in_file) + input_results.append(results) + + if len(input_results) == 1: + # Print out the results sorted by size + size_sorted = sorted([(size, symbol) for symbol, size in results.items()], reverse=True) + print(input_name) + for size, symbol in size_sorted: + print(f'{size:10}\t{symbol}') + print() + elif len(input_results) == 2: + common_symbols = set(input_results[0].keys()).intersection(set(input_results[1].keys())) + print(f'Found {len(common_symbols)} common symbols') + stack_sizes = sorted([(input_results[0][sym], input_results[1][sym], sym) for sym in common_symbols], reverse=True) + if args.md_output: + print('Before | After | Symbol') + print('-- | -- | --') + for size0, size1, symbol in stack_sizes: + if args.only_diffs and size0 == size1: + continue + if args.md_output: + print(f'{size0} | {size1} | {symbol}') + else: + print(f'{size0:10}\t{size1:10}\t{symbol}') + else: + print("TODO support more than 2 inputs") + +if __name__ == '__main__': + main() From 0f04d521e6cc37b92ab621d4023f5c6f280c00e2 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Fri, 4 Nov 2022 15:26:37 -0700 Subject: [PATCH 389/399] Add "Luau origins and evolution" post (#737) --- .../2022-11-04-luau-origins-and-evolution.md | 104 ++++++++++++++++++ 1 file changed, 104 insertions(+) create mode 100644 docs/_posts/2022-11-04-luau-origins-and-evolution.md diff --git a/docs/_posts/2022-11-04-luau-origins-and-evolution.md b/docs/_posts/2022-11-04-luau-origins-and-evolution.md new file mode 100644 index 00000000..707c5d41 --- /dev/null +++ b/docs/_posts/2022-11-04-luau-origins-and-evolution.md @@ -0,0 +1,104 @@ +--- +layout: single +title: "Luau origins and evolution" +author: Arseny Kapoulkine +--- + +At the heart of Roblox technology lies [Luau](https://luau-lang.org), a scripting language derived from Lua 5.1 that is being [developed](https://github.com/Roblox/luau) by an internal team of programming language experts with the help of open source contributors. + +It powers all user-generated content on Roblox, providing access to a very rich set of APIs that allows manipulation of objects in the 3D world, backend API access, UI interaction and more. Hundreds of thousands of developers write code in Luau every month, with top experiences using hundreds of thousands of lines of code, adding up to hundreds of millions of lines of code across the platform. For many of them, it is the first programming language they learn, and one they spend the majority of their time programming in. Using a set of extended APIs developers also customize their workflows by writing plugins to Roblox Studio, where they work on their experiences, using an extended API surface to interact with all aspects of the editor. + +It also powers a lot of application code that Roblox engineers are writing: Universal App, the gateway to the worlds of Roblox that is used by tens of millions of people every day, has 95% of its functionality implemented in Luau, and Roblox Studio has a lot of builtin critical functionality such as part and terrain editors, marketplace browser, avatar and animation editors, material manager and more, implemented in Luau as a plugin, mostly using the same APIs that developers have access to. Every week, updates to this internal codebase that is now over 2 million lines large, are shipped to all Roblox users. + +In addition to Roblox use cases, Luau is also open-source and is seeing an increased adoption in other projects and applications. + +But why did we use Lua in the first place, and why did we decide to pursue building a new language on top of it? + +# Early beginnings + +Around 2006, when a very early version of the Roblox platform was developed, the question of user generated behaviors emerged. Before that, users were able to build non-interactive content on Roblox, and the only form of interaction was physics simulation. While this provided rich emergent behavior, it was hard to build gameplay on top of this: for example, to build a Capture The Flag game, you need to handle collision between players and flags spread throughout the map with a bit of logic that dictates how to adjust team points and when to remove or recreate the objects. + +After an early and brief misstep when we decided to add a few gameplay objects to the core definition of Roblox worlds (some developers may recognize FlagStand as a class name...), the Roblox co-founder Erik Cassel realized that an approach like this is fundamentally limiting the power of user generated content. It’s not enough to give creators the basic blocks on top of which to build their creations, it’s critical to expose the power of a full Turing-complete programming language. Without this, the expressive capability and the reach of the platform would have been restricted far too much. + +But which programming language to choose? This is where [Lua](https://lua.org/), which was, and still is, one of the dominant programming languages used in video games, comes in. + +In addition to its simplicity, which made the language easy to learn and get productive in, Lua was the fastest scripting language compared to popular alternatives like Python or JavaScript at the time[^1], designed to be embedded which meant an easy ability to expose APIs from the host application to the scripts as well as high degree of execution control from the host, and implemented coroutines, a very powerful concurrency primitive that allowed to easily and intuitively script behaviors for independent actors in game using linear control flow. + +Instead of having a large standard library, the expectation was that the embedding application would define a set of APIs that that application needed, as well as establish policies of running the code - which gave us a lot of freedom in how to structure the APIs and when the scripts would get triggered during the simulation of a single frame. + +# Power of simplicity + +Lua is a simple language. What does simplicity mean for us? + +Being a simple language means having a small set of features. Lua has all the fundamental features but doesn’t have a lot of syntax sugar - this means the language is easier to teach and learn, and you rarely run into code that’s difficult to understand syntactically because it uses an unfamiliar construct. Of course, this also means that some programs in Lua are longer than equivalent programs in languages that have more dedicated constructs to solve specific problems, such as list comprehensions in Python. + +Being a simple language means having a minimal set of rules for every feature. Lua does deviate from this in certain respects (which is to say, the language could have been even simpler!), but notably for a dynamic language the behavior of fundamental operators is generally easy to explain and unsurprising - for example, two values in Lua are equal iff they have the same type and the same value, as such `0 == “0”` is `false`; as another example, `for` loops introduce unique variable bindings on every iteration, as such capturing the iteration variable in a closure produces unique values. These decisions lead to more concise and efficient implementation and eliminate a class of bugs in programs. + +Being a simple language means having a small implementation. This may be immaterial to people writing code in the language, but it leads to an implementation that can be of higher quality; simpler implementations can also be easier to optimize for memory or performance, and are easier to build upon. + +Developers on the Roblox platform have very diverse programming backgrounds. Some are writing their first line of code in Roblox Studio, while others have computer science degrees and experience working in multiple different programming languages. While it’s always possible to support two different programming languages that target different segments of the audience, that fragments the ecosystem and makes the programming story less consistent (impacting documentation, tutorials, code reuse, ability for community members to help each other write code, presents challenges with interaction between different languages in the same experience and more). A better outcome is one where a single language can serve both audiences - this requires a language that strikes a balance between simplicity and generality, and while Lua isn’t perfect here, it’s great as a foundation for a language like this[^2]. + +In many ways, Lua is simultaneously simple and pragmatic: many parts of the language are difficult to make much better without a lot of added complexity, but at the same time it requires little in the way of extra functionality to be able to solve problems efficiently. That said, no language is perfect, and within several areas of Lua we felt that the tradeoffs weren’t quite right for our use case. + +# Respectful evolution + +In 2019, we decided to build [Luau](https://luau-lang.org) - a language derived from Lua and compatible with Lua 5.1, which is the version we’ve been using all these years. At the time we evaluated other routes, but ultimately settled on this as the most optimal long-term. + +On one hand, we loved a lot of things about Lua - both design wise and implementation wise, while there were some decisions we felt were suboptimal, by and large it was an almost perfect foundation for what we’ve set out to achieve. + +On the other hand, we’ve been running into the limitations of Lua on large code bases in absence of type checking, performance was good but not great, and some missing features would have been great to have. + +Some of the things we’ve been missing have been added in later versions of Lua, yet we were still using Lua 5.1. While we would have loved to use a later version of the language standard, Lua 5.x releases are not backwards compatible, and some releases remove support for features that are in wide use at Roblox. For Roblox, backwards compatibility is an essential feature of the platform - while we don’t have a guarantee that content created 10 years ago still works, to the extent that we can achieve that without restricting the platform evolution too much, we try. + +What we’ve realized is that Lua is a great foundation for a perfect language that we can build for Roblox. + +We would maintain backwards compatibility with Lua 5.1 but evolve the language from there; sometimes this means taking later features from Lua that don’t conflict with the existing language or our design values, sometimes this means innovating beyond what Lua has done. Crucially, we must maintain the balance between simplicity and power - we still value simplicity, we still need to avoid a feature explosion to ensure that the features compose and are of high quality, and we still need the language to be a good fit for beginners. + +One of the largest limitations that we’ve seen is the lack of type checking making it easy to make mistakes in large code bases, as such [support for type checking](https://luau-lang.org/typecheck) was a requirement for Luau. However, it’s important that the type checker is mostly transparent to the developers who don’t want to invest the time to learn it - anything else would change the learning curve too much for the language to be suitable for beginners. As such, we’ve investing in gradual typing, and our type checker is learning to strike a balance between inferring useful types for completely untyped programs (which, among other things, greatly enhances editing experience through type-aware autocomplete), and the lack of false positive diagnostics that can be confusing and distracting. + +While we did need to introduce [extra syntax](https://luau-lang.org/syntax) to the language - most notably, to support optional type annotations - it was important for us to maintain the cohesion of the overall syntax. We aren’t seeking to make a new language with a syntax alien to Lua programmers - Luau programs are still recognizably Lua, and to the extent possible we try to avoid new syntactic features. In a sense, we still want the syntax, semantics, and the runtime to be simple and minimal - but at the same time we have important problems to solve with respect to ergonomics, robustness and performance of the language, and solving some of them requires having slightly more complex syntax, semantics, or implementation. + +So in finding ways to evolve Luau, we strive to design features that feel like they would be at home in Lua. At the same time, we’ve adopted a more open evolution process - the language development is driven [through RFCs](https://github.com/Roblox/luau/blob/master/rfcs/README.md) that are designs open to the public that anyone can contribute to - this is in contrast with Lua, which has a very closed development process, and is one of the reasons why it would have been difficult for us to keep using Lua as we wouldn’t get a say in its development. At the same time, to ensure the design criterias are met, it’s important that the Luau development team at Roblox maintains a final say over design and implementation of the language[^3], while taking the community's proposals and input into consideration. + +# Importance of co-design + +Luau language is developed in concert with the language compiler, runtime, type checker and other analysis tools, autocomplete engine and other tooling, and that development is guided by the vast volume of existing Luau code, both internal and external. + +This is one of the key principles behind our evolution philosophy - neither layer is developed in isolation, and instead concerns at every level inform all other aspects of the language design and implementation. + +This means that when designing language features, we make sure that they can be implemented efficiently, type checked properly, can be supported well in editing and analysis tools and have a positive impact on the code internal and external engineers write. When we find issues in any component, we can always ask, what changes to other components or even language design would make for a better overall solution. + +This avoids some classes of design problems, for example we won’t specify a language feature that has a prohibitively high implementation cost, as it violates our simplicity criteria, or that is impractical to implement efficiently, as that would create a performance hazard. This also means that when implementing various components of the language we cross-check the concerns and applicability of these across the entire stack - for example, we’ve reworked our auto-complete system to use the same type inference engine that the type checking / analysis tools use, which had immense benefits for the experience of editing code, but also applied significant back pressure on the type inference itself, forcing us to improve it substantially and fix a lot of corner cases that would otherwise have lingered unnoticed. + +Whenever we develop features, optimizations, improve our analysis engine or enhance the standard libraries, we also heavily rely on code written in Luau to validate our hypotheses. When working on new features we find motivation in the real problems that we see our developers face. For example, we implemented the [new ternary operator](https://luau-lang.org/syntax#if-then-else-expressions) after seeing a large set of cases where existing Lua’s `a and b or c` pattern was error-prone for boolean values, which made it easy to accidentally introduce a mistake that was hard to identify automatically. All optimizations and new analysis features are validated on our internal 2M LOC codebase before being added to Luau, which allows us to quickly get initial validation of ideas, or invalidate some approaches as infeasible / unhelpful. + +In addition to that, while we don’t have direct access to community-developed source code for privacy reasons, we can run experiments and collect telemetry[^4], which also helps us make decisions regarding backwards compatibility. Due to [Hyrum’s law](https://www.hyrumslaw.com/), technically any change in the language or libraries, no matter how small, would be backwards incompatible - instead we adopt the notion of pragmatic balance between strict backwards compatibility[^5] and pragmatic compatibility concerns. For example, later versions of Lua make some library functions like `table.insert`/`table.remove` more strict with how they handle out of range indices. We have evaluated this change for compatibility by collecting telemetry on the use of out of range indices in these functions on the Roblox platform and concluded that applying the stricter checking would break existing programs, and instead had to slightly adjust the rules for out of range behavior in ways that was benign for existing code but prevented catastrophic performance degradation for large out of range indices. Because we couldn’t afford to introduce new runtime errors in this case, we also added a set of linting rules to our analysis engine to flag potential misuse of `table.insert`/`table.remove` before the code ever gets to run - this diagnostics is informational and as such doesn’t affect backwards compatibility, but does help prevent mistakes. + +There are also cases where this co-design approach prevents introduction of features that can lead to easy misuse, which can be difficult to see in the design of the feature itself, but becomes more apparent when you consider features in context of the entire ecosystem. This is a good thing - it means co-design acts as a forcing function on the language simplicity and makes it easier to flag potential bad interactions between different language features, or language features and tooling, or language features and existing programming patterns that are in widespread use in real-world code. By making sure that all features are validated for their impact across the stack and on code written in Luau, we ultimately get a better, simpler and more cohesive language. + +# Efficient execution + +One of the critical goals in front of Luau is efficiency, both from the performance and memory perspective. There’s only so many milliseconds in a frame, and we simultaneously see the need to increase the scale and complexity of simulated experiences, which requires more memory and computation, as well as the need to fit more comfortably into smaller budgets of performance memory for better experience on smaller devices. In fact, one of the motivations for Luau in 2019 has been improved performance, as we saw many opportunities to go beyond Lua with a redesigned implementation. + +Crucially, our performance needs are somewhat unique and require somewhat unique solutions. + +We need Luau to run on many platforms where native code generation is either prohibited by the platform vendor or impractical due to tight memory constraints. As such, in terms of execution performance it’s critical that we have a very fast interpreter[^6]. However, we have freedom in terms of high level design of the entire stack - for example, clients never see the source code of the scripts as all compilation to bytecode happens on the server; this gives us an opportunity to perform more involved and expensive optimizations during that process as well as have the smallest possible startup time on the client without complex pre-parse steps. Notably, our bytecode compiler performs a series of high level optimizations including function inlining and loop unrolling that in other dynamic languages is often left to the just-in-time compiler. + +Another area where performance is critical is garbage collection. Garbage collection is crucial for the language’s simplicity as it makes memory management easier to reason about, but it does require a substantial amount of implementation effort to keep it efficient. For Roblox and for any other game engine or interactive simulation, latency is critical and so our collector is heavily optimized for that - to the extent possible collection is incremental and stop-the-world pauses are very brief. Another part of the performance story here however is the language and data structure design - by making sure that core data types are efficient in how they are laid out in memory we reduce the amount of work garbage collector takes to trace the heap, and, as another example of co-design, we try to make sure that language features are conscious of the impact they have on memory and garbage collection efficiency. + +However, from a whole-platform standpoint there’s a lot of performance aspects that go beyond single-threaded execution. This is an active area of research and development for the team, as to really leverage the hardware the code is running on we need to think about SIMD, hardware thread utilization as well as running code in a cluster of nodes. These considerations inform current and future development of the runtime and the language (for example, our runtime now supports efficient operations on short SIMD vectors even in interpreted mode, and the VM is fairly lightweight to instantiate which makes running many VMs per core practical, with message passing or access to shared Roblox data model used to make gameplay features feasible to implement), but we’re definitely in the early days here - our first implementation of parallel script execution in Roblox just [shipped earlier this year](https://devforum.roblox.com/t/full-release-of-parallel-luau-v1/1836187). This is likely the area where a lot of future innovations will happen as well. + +# Future + +We’re very happy with the success of Luau - in several years we’ve established consistent processes for evolving the language and so far we found a good balance between simplicity, ease of use, performance and robustness of the language, its implementation and the tooling surrounding it. The language keeps continuously evolving but at a pace that is easy to stay on top of - in 2022 we shipped a few syntactic extensions for type annotations but no changes to the syntax of the language outside of types, and only one major [semantic change to the for loop iteration](https://luau-lang.org/syntax#generalized-iteration) that actually made the language easier to use by avoiding the need to specify the table traversal style via `pairs`/`ipairs`. We try to make sure that the features are general and provide enough extensibility so that libraries can be built on top of the language to make it easier to write code, while also making it practical to use the language without complex supporting frameworks. + +There’s still a lot of ground to cover, and we’ll be working on Luau for years to come. We’re in the process of building the next version of our type inference / checking engine to make sure that all users of the language regardless of their expertise benefit from it, we’ve started investing in native code generation as we’re reaching the limits of interpreted performance (although some exciting opportunities for compiler optimization are still on the horizon), and there’s still a lot of hard design and implementation work ahead of us for some important language features and standard libraries. And as mentioned, our execution model will likely see a lot of innovation as we push the boundaries of hardware utilization across cores and nodes. + +Overall, Luau is like an iceberg - the surface is simple to learn and use, but it hides the tremendous amount of careful design, engineering and attention to detail, and we plan to continue to invest in it while trying to keep the outer surface comparatively small. We're excited to see how far we can take it! + +[^1]: High-performance JavaScript engines didn’t exist at the time! LuaJIT was around the corner and redefined the performance expectations of dynamic languages. +[^2]: In fact, scaling to large teams of expert programmers is one of the core motivations behind our creating Luau, while a requirement to still be suitable for beginner programmers guides our evolution direction. +[^3]: This would have been difficult to drive in any existing large established language like JavaScript or Python. +[^4]: This is limited to Roblox platform and doesn't exist in open-source releases. +[^5]: Which we do follow in some areas, such as syntactic compatibility - all existing programs that parse must continue to parse the same way as the language evolves. +[^6]: Some design decisions and implementation techniques are documented on our [performance page](https://luau-lang.org/performance). From 816e41a8f2eae7ca2d9d9fb4d2f847e33c087650 Mon Sep 17 00:00:00 2001 From: vegorov-rbx <75688451+vegorov-rbx@users.noreply.github.com> Date: Thu, 10 Nov 2022 14:53:13 -0800 Subject: [PATCH 390/399] Sync to upstream/release/553 (#742) * Type inference of `a and b` and `a or b` has been improved (Fixes https://github.com/Roblox/luau/issues/730) * Improved error message when `for ... in x` loop iterates over a value that could be 'nil' * Return type of `next` not includes 'nil' (Fixes https://github.com/Roblox/luau/issues/706) * Improved type inference of `string` type * Luau library table type names are now reported as `typeof(string)`/etc instead of just `string` which was misleading * Added parsing error when optional type alias type parameter wasn't provided after `=` token * Improved tagged union type refinement in conditional expressions, type in `else` branch should no longer include previously handled union options --- .../include/Luau/ConstraintGraphBuilder.h | 20 +- Analysis/include/Luau/ConstraintSolver.h | 4 +- Analysis/include/Luau/Scope.h | 5 - Analysis/include/Luau/ToString.h | 2 - Analysis/include/Luau/TypeInfer.h | 4 +- Analysis/src/BuiltinDefinitions.cpp | 94 ++++++-- Analysis/src/ConstraintGraphBuilder.cpp | 191 ++++++++++++---- Analysis/src/ConstraintSolver.cpp | 108 +++++---- Analysis/src/EmbeddedBuiltinDefinitions.cpp | 8 +- Analysis/src/Frontend.cpp | 3 +- Analysis/src/Module.cpp | 14 -- Analysis/src/ToString.cpp | 110 ++++----- Analysis/src/TypeChecker2.cpp | 35 ++- Analysis/src/TypeInfer.cpp | 202 +++++++++++++---- Analysis/src/TypeVar.cpp | 204 +++++++++++++++-- Analysis/src/Unifier.cpp | 43 +++- Ast/src/Parser.cpp | 13 +- CodeGen/include/Luau/AddressA64.h | 7 +- CodeGen/include/Luau/AssemblyBuilderA64.h | 29 ++- CodeGen/src/AssemblyBuilderA64.cpp | 210 ++++++++++++++---- CodeGen/src/CodeGenX64.cpp | 74 +++--- CodeGen/src/EmitCommonX64.h | 8 +- CodeGen/src/UnwindBuilderDwarf2.cpp | 20 +- CodeGen/src/UnwindBuilderWin.cpp | 6 +- Common/include/Luau/ExperimentalFlags.h | 2 + tests/AssemblyBuilderA64.test.cpp | 65 +++++- tests/CodeAllocator.test.cpp | 9 +- tests/ConstraintGraphBuilderFixture.cpp | 4 +- tests/Frontend.test.cpp | 8 - tests/Parser.test.cpp | 17 ++ tests/Repl.test.cpp | 17 ++ tests/ToString.test.cpp | 71 +++--- tests/TypeInfer.aliases.test.cpp | 13 +- tests/TypeInfer.builtins.test.cpp | 6 +- tests/TypeInfer.functions.test.cpp | 2 +- tests/TypeInfer.operators.test.cpp | 83 ++++++- tests/TypeInfer.provisional.test.cpp | 26 +-- tests/TypeInfer.refinements.test.cpp | 189 +++++++++++++--- tests/TypeInfer.tables.test.cpp | 99 ++++++++- tests/TypeInfer.test.cpp | 37 --- tests/TypeInfer.typePacks.cpp | 2 - tests/TypeInfer.unionTypes.test.cpp | 17 ++ tests/TypeInfer.unknownnever.test.cpp | 4 +- tools/faillist.txt | 32 +-- 44 files changed, 1535 insertions(+), 582 deletions(-) diff --git a/Analysis/include/Luau/ConstraintGraphBuilder.h b/Analysis/include/Luau/ConstraintGraphBuilder.h index cb5900ea..2bfc62f1 100644 --- a/Analysis/include/Luau/ConstraintGraphBuilder.h +++ b/Analysis/include/Luau/ConstraintGraphBuilder.h @@ -66,6 +66,14 @@ struct ConstraintGraphBuilder // The root scope of the module we're generating constraints for. // This is null when the CGB is initially constructed. Scope* rootScope; + + // Constraints that go straight to the solver. + std::vector constraints; + + // Constraints that do not go to the solver right away. Other constraints + // will enqueue them during solving. + std::vector unqueuedConstraints; + // A mapping of AST node to TypeId. DenseHashMap astTypes{nullptr}; // A mapping of AST node to TypePackId. @@ -252,16 +260,8 @@ struct ConstraintGraphBuilder void prepopulateGlobalScope(const ScopePtr& globalScope, AstStatBlock* program); }; -/** - * Collects a vector of borrowed constraints from the scope and all its child - * scopes. It is important to only call this function when you're done adding - * constraints to the scope or its descendants, lest the borrowed pointers - * become invalid due to a container reallocation. - * @param rootScope the root scope of the scope graph to collect constraints - * from. - * @return a list of pointers to constraints contained within the scope graph. - * None of these pointers should be null. +/** Borrow a vector of pointers from a vector of owning pointers to constraints. */ -std::vector> collectConstraints(NotNull rootScope); +std::vector> borrowConstraints(const std::vector& constraints); } // namespace Luau diff --git a/Analysis/include/Luau/ConstraintSolver.h b/Analysis/include/Luau/ConstraintSolver.h index 07f027ad..7b89a278 100644 --- a/Analysis/include/Luau/ConstraintSolver.h +++ b/Analysis/include/Luau/ConstraintSolver.h @@ -76,8 +76,8 @@ struct ConstraintSolver DcrLogger* logger; - explicit ConstraintSolver(NotNull normalizer, NotNull rootScope, ModuleName moduleName, NotNull moduleResolver, - std::vector requireCycles, DcrLogger* logger); + explicit ConstraintSolver(NotNull normalizer, NotNull rootScope, std::vector> constraints, + ModuleName moduleName, NotNull moduleResolver, std::vector requireCycles, DcrLogger* logger); // Randomize the order in which to dispatch constraints void randomize(unsigned seed); diff --git a/Analysis/include/Luau/Scope.h b/Analysis/include/Luau/Scope.h index ccf2964c..a26f506d 100644 --- a/Analysis/include/Luau/Scope.h +++ b/Analysis/include/Luau/Scope.h @@ -38,11 +38,6 @@ struct Scope std::unordered_map bindings; TypePackId returnType; std::optional varargPack; - // All constraints belonging to this scope. - std::vector constraints; - // Constraints belonging to this scope that are queued manually by other - // constraints. - std::vector unqueuedConstraints; TypeLevel level; diff --git a/Analysis/include/Luau/ToString.h b/Analysis/include/Luau/ToString.h index ff2561e6..0200a719 100644 --- a/Analysis/include/Luau/ToString.h +++ b/Analysis/include/Luau/ToString.h @@ -34,7 +34,6 @@ struct ToStringOptions size_t maxTableLength = size_t(FInt::LuauTableTypeMaximumStringifierLength); // Only applied to TableTypeVars size_t maxTypeLength = size_t(FInt::LuauTypeMaximumStringifierLength); ToStringNameMap nameMap; - std::optional DEPRECATED_nameMap; std::shared_ptr scope; // If present, module names will be added and types that are not available in scope will be marked as 'invalid' std::vector namedFunctionOverrideArgNames; // If present, named function argument names will be overridden }; @@ -42,7 +41,6 @@ struct ToStringOptions struct ToStringResult { std::string name; - ToStringNameMap DEPRECATED_nameMap; bool invalid = false; bool error = false; diff --git a/Analysis/include/Luau/TypeInfer.h b/Analysis/include/Luau/TypeInfer.h index c5d7501d..4eaa5969 100644 --- a/Analysis/include/Luau/TypeInfer.h +++ b/Analysis/include/Luau/TypeInfer.h @@ -280,14 +280,14 @@ private: TypeId singletonType(bool value); TypeId singletonType(std::string value); - TypeIdPredicate mkTruthyPredicate(bool sense); + TypeIdPredicate mkTruthyPredicate(bool sense, TypeId emptySetTy); // TODO: Return TypeId only. std::optional filterMapImpl(TypeId type, TypeIdPredicate predicate); std::pair, bool> filterMap(TypeId type, TypeIdPredicate predicate); public: - std::pair, bool> pickTypesFromSense(TypeId type, bool sense); + std::pair, bool> pickTypesFromSense(TypeId type, bool sense, TypeId emptySetTy); private: TypeId unionOfTypes(TypeId a, TypeId b, const ScopePtr& scope, const Location& location, bool unifyFreeTypes = true); diff --git a/Analysis/src/BuiltinDefinitions.cpp b/Analysis/src/BuiltinDefinitions.cpp index ee53ae6b..67e3979a 100644 --- a/Analysis/src/BuiltinDefinitions.cpp +++ b/Analysis/src/BuiltinDefinitions.cpp @@ -17,7 +17,9 @@ LUAU_FASTFLAGVARIABLE(LuauSetMetaTableArgsCheck, false) LUAU_FASTFLAG(LuauUnknownAndNeverType) LUAU_FASTFLAGVARIABLE(LuauBuiltInMetatableNoBadSynthetic, false) +LUAU_FASTFLAG(LuauOptionalNextKey) LUAU_FASTFLAG(LuauReportShadowedTypeAlias) +LUAU_FASTFLAG(LuauNewLibraryTypeNames) /** FIXME: Many of these type definitions are not quite completely accurate. * @@ -276,18 +278,38 @@ void registerBuiltinGlobals(TypeChecker& typeChecker) addGlobalBinding(typeChecker, "string", it->second.type, "@luau"); - // next(t: Table, i: K?) -> (K, V) - TypePackId nextArgsTypePack = arena.addTypePack(TypePack{{mapOfKtoV, makeOption(typeChecker, arena, genericK)}}); - addGlobalBinding(typeChecker, "next", - arena.addType(FunctionTypeVar{{genericK, genericV}, {}, nextArgsTypePack, arena.addTypePack(TypePack{{genericK, genericV}})}), "@luau"); + if (FFlag::LuauOptionalNextKey) + { + // next(t: Table, i: K?) -> (K?, V) + TypePackId nextArgsTypePack = arena.addTypePack(TypePack{{mapOfKtoV, makeOption(typeChecker, arena, genericK)}}); + TypePackId nextRetsTypePack = arena.addTypePack(TypePack{{makeOption(typeChecker, arena, genericK), genericV}}); + addGlobalBinding(typeChecker, "next", arena.addType(FunctionTypeVar{{genericK, genericV}, {}, nextArgsTypePack, nextRetsTypePack}), "@luau"); - TypePackId pairsArgsTypePack = arena.addTypePack({mapOfKtoV}); + TypePackId pairsArgsTypePack = arena.addTypePack({mapOfKtoV}); - TypeId pairsNext = arena.addType(FunctionTypeVar{nextArgsTypePack, arena.addTypePack(TypePack{{genericK, genericV}})}); - TypePackId pairsReturnTypePack = arena.addTypePack(TypePack{{pairsNext, mapOfKtoV, nilType}}); + TypeId pairsNext = arena.addType(FunctionTypeVar{nextArgsTypePack, nextRetsTypePack}); + TypePackId pairsReturnTypePack = arena.addTypePack(TypePack{{pairsNext, mapOfKtoV, nilType}}); - // pairs(t: Table) -> ((Table, K?) -> (K, V), Table, nil) - addGlobalBinding(typeChecker, "pairs", arena.addType(FunctionTypeVar{{genericK, genericV}, {}, pairsArgsTypePack, pairsReturnTypePack}), "@luau"); + // pairs(t: Table) -> ((Table, K?) -> (K, V), Table, nil) + addGlobalBinding( + typeChecker, "pairs", arena.addType(FunctionTypeVar{{genericK, genericV}, {}, pairsArgsTypePack, pairsReturnTypePack}), "@luau"); + } + else + { + // next(t: Table, i: K?) -> (K, V) + TypePackId nextArgsTypePack = arena.addTypePack(TypePack{{mapOfKtoV, makeOption(typeChecker, arena, genericK)}}); + addGlobalBinding(typeChecker, "next", + arena.addType(FunctionTypeVar{{genericK, genericV}, {}, nextArgsTypePack, arena.addTypePack(TypePack{{genericK, genericV}})}), "@luau"); + + TypePackId pairsArgsTypePack = arena.addTypePack({mapOfKtoV}); + + TypeId pairsNext = arena.addType(FunctionTypeVar{nextArgsTypePack, arena.addTypePack(TypePack{{genericK, genericV}})}); + TypePackId pairsReturnTypePack = arena.addTypePack(TypePack{{pairsNext, mapOfKtoV, nilType}}); + + // pairs(t: Table) -> ((Table, K?) -> (K, V), Table, nil) + addGlobalBinding( + typeChecker, "pairs", arena.addType(FunctionTypeVar{{genericK, genericV}, {}, pairsArgsTypePack, pairsReturnTypePack}), "@luau"); + } TypeId genericMT = arena.addType(GenericTypeVar{"MT"}); @@ -319,7 +341,12 @@ void registerBuiltinGlobals(TypeChecker& typeChecker) if (TableTypeVar* ttv = getMutable(pair.second.typeId)) { if (!ttv->name) - ttv->name = toString(pair.first); + { + if (FFlag::LuauNewLibraryTypeNames) + ttv->name = "typeof(" + toString(pair.first) + ")"; + else + ttv->name = toString(pair.first); + } } } @@ -370,18 +397,38 @@ void registerBuiltinGlobals(Frontend& frontend) addGlobalBinding(frontend, "string", it->second.type, "@luau"); - // next(t: Table, i: K?) -> (K, V) - TypePackId nextArgsTypePack = arena.addTypePack(TypePack{{mapOfKtoV, makeOption(frontend, arena, genericK)}}); - addGlobalBinding(frontend, "next", - arena.addType(FunctionTypeVar{{genericK, genericV}, {}, nextArgsTypePack, arena.addTypePack(TypePack{{genericK, genericV}})}), "@luau"); + if (FFlag::LuauOptionalNextKey) + { + // next(t: Table, i: K?) -> (K?, V) + TypePackId nextArgsTypePack = arena.addTypePack(TypePack{{mapOfKtoV, makeOption(frontend, arena, genericK)}}); + TypePackId nextRetsTypePack = arena.addTypePack(TypePack{{makeOption(frontend, arena, genericK), genericV}}); + addGlobalBinding(frontend, "next", arena.addType(FunctionTypeVar{{genericK, genericV}, {}, nextArgsTypePack, nextRetsTypePack}), "@luau"); - TypePackId pairsArgsTypePack = arena.addTypePack({mapOfKtoV}); + TypePackId pairsArgsTypePack = arena.addTypePack({mapOfKtoV}); - TypeId pairsNext = arena.addType(FunctionTypeVar{nextArgsTypePack, arena.addTypePack(TypePack{{genericK, genericV}})}); - TypePackId pairsReturnTypePack = arena.addTypePack(TypePack{{pairsNext, mapOfKtoV, frontend.singletonTypes->nilType}}); + TypeId pairsNext = arena.addType(FunctionTypeVar{nextArgsTypePack, nextRetsTypePack}); + TypePackId pairsReturnTypePack = arena.addTypePack(TypePack{{pairsNext, mapOfKtoV, frontend.singletonTypes->nilType}}); - // pairs(t: Table) -> ((Table, K?) -> (K, V), Table, nil) - addGlobalBinding(frontend, "pairs", arena.addType(FunctionTypeVar{{genericK, genericV}, {}, pairsArgsTypePack, pairsReturnTypePack}), "@luau"); + // pairs(t: Table) -> ((Table, K?) -> (K?, V), Table, nil) + addGlobalBinding( + frontend, "pairs", arena.addType(FunctionTypeVar{{genericK, genericV}, {}, pairsArgsTypePack, pairsReturnTypePack}), "@luau"); + } + else + { + // next(t: Table, i: K?) -> (K, V) + TypePackId nextArgsTypePack = arena.addTypePack(TypePack{{mapOfKtoV, makeOption(frontend, arena, genericK)}}); + addGlobalBinding(frontend, "next", + arena.addType(FunctionTypeVar{{genericK, genericV}, {}, nextArgsTypePack, arena.addTypePack(TypePack{{genericK, genericV}})}), "@luau"); + + TypePackId pairsArgsTypePack = arena.addTypePack({mapOfKtoV}); + + TypeId pairsNext = arena.addType(FunctionTypeVar{nextArgsTypePack, arena.addTypePack(TypePack{{genericK, genericV}})}); + TypePackId pairsReturnTypePack = arena.addTypePack(TypePack{{pairsNext, mapOfKtoV, frontend.singletonTypes->nilType}}); + + // pairs(t: Table) -> ((Table, K?) -> (K, V), Table, nil) + addGlobalBinding( + frontend, "pairs", arena.addType(FunctionTypeVar{{genericK, genericV}, {}, pairsArgsTypePack, pairsReturnTypePack}), "@luau"); + } TypeId genericMT = arena.addType(GenericTypeVar{"MT"}); @@ -413,7 +460,12 @@ void registerBuiltinGlobals(Frontend& frontend) if (TableTypeVar* ttv = getMutable(pair.second.typeId)) { if (!ttv->name) - ttv->name = toString(pair.first); + { + if (FFlag::LuauNewLibraryTypeNames) + ttv->name = "typeof(" + toString(pair.first) + ")"; + else + ttv->name = toString(pair.first); + } } } @@ -623,7 +675,7 @@ static std::optional> magicFunctionAssert( if (head.size() > 0) { - auto [ty, ok] = typechecker.pickTypesFromSense(head[0], true); + auto [ty, ok] = typechecker.pickTypesFromSense(head[0], true, typechecker.singletonTypes->nilType); if (FFlag::LuauUnknownAndNeverType) { if (get(*ty)) diff --git a/Analysis/src/ConstraintGraphBuilder.cpp b/Analysis/src/ConstraintGraphBuilder.cpp index 79a69ca4..42dc07f6 100644 --- a/Analysis/src/ConstraintGraphBuilder.cpp +++ b/Analysis/src/ConstraintGraphBuilder.cpp @@ -52,6 +52,70 @@ static bool matchSetmetatable(const AstExprCall& call) return true; } +struct TypeGuard +{ + bool isTypeof; + AstExpr* target; + std::string type; +}; + +static std::optional matchTypeGuard(const AstExprBinary* binary) +{ + if (binary->op != AstExprBinary::CompareEq && binary->op != AstExprBinary::CompareNe) + return std::nullopt; + + AstExpr* left = binary->left; + AstExpr* right = binary->right; + if (right->is()) + std::swap(left, right); + + if (!right->is()) + return std::nullopt; + + AstExprCall* call = left->as(); + AstExprConstantString* string = right->as(); + if (!call || !string) + return std::nullopt; + + AstExprGlobal* callee = call->func->as(); + if (!callee) + return std::nullopt; + + if (callee->name != "type" && callee->name != "typeof") + return std::nullopt; + + if (call->args.size != 1) + return std::nullopt; + + return TypeGuard{ + /*isTypeof*/ callee->name == "typeof", + /*target*/ call->args.data[0], + /*type*/ std::string(string->value.data, string->value.size), + }; +} + +namespace +{ + +struct Checkpoint +{ + size_t offset; +}; + +Checkpoint checkpoint(const ConstraintGraphBuilder* cgb) +{ + return Checkpoint{cgb->constraints.size()}; +} + +template +void forEachConstraint(const Checkpoint& start, const Checkpoint& end, const ConstraintGraphBuilder* cgb, F f) +{ + for (size_t i = start.offset; i < end.offset; ++i) + f(cgb->constraints[i]); +} + +} // namespace + ConstraintGraphBuilder::ConstraintGraphBuilder(const ModuleName& moduleName, ModulePtr module, TypeArena* arena, NotNull moduleResolver, NotNull singletonTypes, NotNull ice, const ScopePtr& globalScope, DcrLogger* logger, NotNull dfg) @@ -99,12 +163,12 @@ ScopePtr ConstraintGraphBuilder::childScope(AstNode* node, const ScopePtr& paren NotNull ConstraintGraphBuilder::addConstraint(const ScopePtr& scope, const Location& location, ConstraintV cv) { - return NotNull{scope->constraints.emplace_back(new Constraint{NotNull{scope.get()}, location, std::move(cv)}).get()}; + return NotNull{constraints.emplace_back(new Constraint{NotNull{scope.get()}, location, std::move(cv)}).get()}; } NotNull ConstraintGraphBuilder::addConstraint(const ScopePtr& scope, std::unique_ptr c) { - return NotNull{scope->constraints.emplace_back(std::move(c)).get()}; + return NotNull{constraints.emplace_back(std::move(c)).get()}; } static void unionRefinements(const std::unordered_map& lhs, const std::unordered_map& rhs, @@ -476,6 +540,9 @@ void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatForIn* forIn) TypeId ty = freshType(loopScope); loopScope->bindings[var] = Binding{ty, var->location}; variableTypes.push_back(ty); + + if (auto def = dfg->getDef(var)) + loopScope->dcrRefinements[*def] = ty; } // It is always ok to provide too few variables, so we give this pack a free tail. @@ -506,20 +573,6 @@ void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatRepeat* repeat) check(repeatScope, repeat->condition); } -void addConstraints(Constraint* constraint, NotNull scope) -{ - scope->constraints.reserve(scope->constraints.size() + scope->constraints.size()); - - for (const auto& c : scope->constraints) - constraint->dependencies.push_back(NotNull{c.get()}); - - for (const auto& c : scope->unqueuedConstraints) - constraint->dependencies.push_back(NotNull{c.get()}); - - for (NotNull childScope : scope->children) - addConstraints(constraint, childScope); -} - void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatLocalFunction* function) { // Local @@ -537,12 +590,17 @@ void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatLocalFunction* FunctionSignature sig = checkFunctionSignature(scope, function->func); sig.bodyScope->bindings[function->name] = Binding{sig.signature, function->func->location}; + auto start = checkpoint(this); checkFunctionBody(sig.bodyScope, function->func); + auto end = checkpoint(this); NotNull constraintScope{sig.signatureScope ? sig.signatureScope.get() : sig.bodyScope.get()}; std::unique_ptr c = std::make_unique(constraintScope, function->name->location, GeneralizationConstraint{functionType, sig.signature}); - addConstraints(c.get(), NotNull(sig.bodyScope.get())); + + forEachConstraint(start, end, this, [&c](const ConstraintPtr& constraint) { + c->dependencies.push_back(NotNull{constraint.get()}); + }); addConstraint(scope, std::move(c)); } @@ -610,12 +668,17 @@ void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatFunction* funct LUAU_ASSERT(functionType != nullptr); + auto start = checkpoint(this); checkFunctionBody(sig.bodyScope, function->func); + auto end = checkpoint(this); NotNull constraintScope{sig.signatureScope ? sig.signatureScope.get() : sig.bodyScope.get()}; std::unique_ptr c = std::make_unique(constraintScope, function->name->location, GeneralizationConstraint{functionType, sig.signature}); - addConstraints(c.get(), NotNull(sig.bodyScope.get())); + + forEachConstraint(start, end, this, [&c](const ConstraintPtr& constraint) { + c->dependencies.push_back(NotNull{constraint.get()}); + }); addConstraint(scope, std::move(c)); } @@ -947,8 +1010,7 @@ InferencePack ConstraintGraphBuilder::checkPack(const ScopePtr& scope, AstExpr* InferencePack ConstraintGraphBuilder::checkPack(const ScopePtr& scope, AstExprCall* call, const std::vector& expectedTypes) { TypeId fnType = check(scope, call->func).ty; - const size_t constraintIndex = scope->constraints.size(); - const size_t scopeIndex = scopes.size(); + auto startCheckpoint = checkpoint(this); std::vector args; @@ -977,8 +1039,7 @@ InferencePack ConstraintGraphBuilder::checkPack(const ScopePtr& scope, AstExprCa } else { - const size_t constraintEndIndex = scope->constraints.size(); - const size_t scopeEndIndex = scopes.size(); + auto endCheckpoint = checkpoint(this); astOriginalCallTypes[call->func] = fnType; @@ -989,29 +1050,22 @@ InferencePack ConstraintGraphBuilder::checkPack(const ScopePtr& scope, AstExprCa FunctionTypeVar ftv(TypeLevel{}, scope.get(), argPack, rets); TypeId inferredFnType = arena->addType(ftv); - scope->unqueuedConstraints.push_back( + unqueuedConstraints.push_back( std::make_unique(NotNull{scope.get()}, call->func->location, InstantiationConstraint{instantiatedType, fnType})); - NotNull ic(scope->unqueuedConstraints.back().get()); + NotNull ic(unqueuedConstraints.back().get()); - scope->unqueuedConstraints.push_back( + unqueuedConstraints.push_back( std::make_unique(NotNull{scope.get()}, call->func->location, SubtypeConstraint{inferredFnType, instantiatedType})); - NotNull sc(scope->unqueuedConstraints.back().get()); + NotNull sc(unqueuedConstraints.back().get()); // We force constraints produced by checking function arguments to wait // until after we have resolved the constraint on the function itself. // This ensures, for instance, that we start inferring the contents of // lambdas under the assumption that their arguments and return types // will be compatible with the enclosing function call. - for (size_t ci = constraintIndex; ci < constraintEndIndex; ++ci) - scope->constraints[ci]->dependencies.push_back(sc); - - for (size_t si = scopeIndex; si < scopeEndIndex; ++si) - { - for (auto& c : scopes[si].second->constraints) - { - c->dependencies.push_back(sc); - } - } + forEachConstraint(startCheckpoint, endCheckpoint, this, [sc](const ConstraintPtr& constraint) { + constraint->dependencies.push_back(sc); + }); addConstraint(scope, call->func->location, FunctionCallConstraint{ @@ -1283,6 +1337,54 @@ std::tuple ConstraintGraphBuilder::checkBinary( return {leftType, rightType, connectiveArena.disjunction(leftConnective, rightConnective)}; } + else if (auto typeguard = matchTypeGuard(binary)) + { + TypeId leftType = check(scope, binary->left).ty; + TypeId rightType = check(scope, binary->right).ty; + + std::optional def = dfg->getDef(typeguard->target); + if (!def) + return {leftType, rightType, nullptr}; + + TypeId discriminantTy = singletonTypes->neverType; + if (typeguard->type == "nil") + discriminantTy = singletonTypes->nilType; + else if (typeguard->type == "string") + discriminantTy = singletonTypes->stringType; + else if (typeguard->type == "number") + discriminantTy = singletonTypes->numberType; + else if (typeguard->type == "boolean") + discriminantTy = singletonTypes->threadType; + else if (typeguard->type == "table") + discriminantTy = singletonTypes->neverType; // TODO: replace with top table type + else if (typeguard->type == "function") + discriminantTy = singletonTypes->functionType; + else if (typeguard->type == "userdata") + { + // For now, we don't really care about being accurate with userdata if the typeguard was using typeof + discriminantTy = singletonTypes->neverType; // TODO: replace with top class type + } + else if (!typeguard->isTypeof && typeguard->type == "vector") + discriminantTy = singletonTypes->neverType; // TODO: figure out a way to deal with this quirky type + else if (!typeguard->isTypeof) + discriminantTy = singletonTypes->neverType; + else if (auto typeFun = globalScope->lookupType(typeguard->type); typeFun && typeFun->typeParams.empty() && typeFun->typePackParams.empty()) + { + TypeId ty = follow(typeFun->type); + + // We're only interested in the root class of any classes. + if (auto ctv = get(ty); !ctv || !ctv->parent) + discriminantTy = ty; + } + + ConnectiveId proposition = connectiveArena.proposition(*def, discriminantTy); + if (binary->op == AstExprBinary::CompareEq) + return {leftType, rightType, proposition}; + else if (binary->op == AstExprBinary::CompareNe) + return {leftType, rightType, connectiveArena.negation(proposition)}; + else + ice->ice("matchTypeGuard should only return a Some under `==` or `~=`!"); + } else if (binary->op == AstExprBinary::CompareEq || binary->op == AstExprBinary::CompareNe) { TypeId leftType = check(scope, binary->left, expectedType, true).ty; @@ -2066,19 +2168,14 @@ void ConstraintGraphBuilder::prepopulateGlobalScope(const ScopePtr& globalScope, program->visit(&gp); } -void collectConstraints(std::vector>& result, NotNull scope) -{ - for (const auto& c : scope->constraints) - result.push_back(NotNull{c.get()}); - - for (NotNull child : scope->children) - collectConstraints(result, child); -} - -std::vector> collectConstraints(NotNull rootScope) +std::vector> borrowConstraints(const std::vector& constraints) { std::vector> result; - collectConstraints(result, rootScope); + result.reserve(constraints.size()); + + for (const auto& c : constraints) + result.emplace_back(c.get()); + return result; } diff --git a/Analysis/src/ConstraintSolver.cpp b/Analysis/src/ConstraintSolver.cpp index c53ac659..533652e2 100644 --- a/Analysis/src/ConstraintSolver.cpp +++ b/Analysis/src/ConstraintSolver.cpp @@ -18,7 +18,6 @@ LUAU_FASTFLAGVARIABLE(DebugLuauLogSolver, false); LUAU_FASTFLAGVARIABLE(DebugLuauLogSolverToJson, false); -LUAU_FASTFLAG(LuauFixNameMaps) namespace Luau { @@ -27,34 +26,14 @@ namespace Luau { for (const auto& [k, v] : scope->bindings) { - if (FFlag::LuauFixNameMaps) - { - auto d = toString(v.typeId, opts); - printf("\t%s : %s\n", k.c_str(), d.c_str()); - } - else - { - auto d = toStringDetailed(v.typeId, opts); - opts.DEPRECATED_nameMap = d.DEPRECATED_nameMap; - printf("\t%s : %s\n", k.c_str(), d.name.c_str()); - } + auto d = toString(v.typeId, opts); + printf("\t%s : %s\n", k.c_str(), d.c_str()); } for (NotNull child : scope->children) dumpBindings(child, opts); } -static void dumpConstraints(NotNull scope, ToStringOptions& opts) -{ - for (const ConstraintPtr& c : scope->constraints) - { - printf("\t%s\n", toString(*c, opts).c_str()); - } - - for (NotNull child : scope->children) - dumpConstraints(child, opts); -} - static std::pair, std::vector> saturateArguments(TypeArena* arena, NotNull singletonTypes, const TypeFun& fn, const std::vector& rawTypeArguments, const std::vector& rawPackArguments) { @@ -219,12 +198,6 @@ size_t HashInstantiationSignature::operator()(const InstantiationSignature& sign return hash; } -void dump(NotNull rootScope, ToStringOptions& opts) -{ - printf("constraints:\n"); - dumpConstraints(rootScope, opts); -} - void dump(ConstraintSolver* cs, ToStringOptions& opts) { printf("constraints:\n"); @@ -248,17 +221,17 @@ void dump(ConstraintSolver* cs, ToStringOptions& opts) if (auto fcc = get(*c)) { for (NotNull inner : fcc->innerConstraints) - printf("\t\t\t%s\n", toString(*inner, opts).c_str()); + printf("\t ->\t\t%s\n", toString(*inner, opts).c_str()); } } } -ConstraintSolver::ConstraintSolver(NotNull normalizer, NotNull rootScope, ModuleName moduleName, - NotNull moduleResolver, std::vector requireCycles, DcrLogger* logger) +ConstraintSolver::ConstraintSolver(NotNull normalizer, NotNull rootScope, std::vector> constraints, + ModuleName moduleName, NotNull moduleResolver, std::vector requireCycles, DcrLogger* logger) : arena(normalizer->arena) , singletonTypes(normalizer->singletonTypes) , normalizer(normalizer) - , constraints(collectConstraints(rootScope)) + , constraints(std::move(constraints)) , rootScope(rootScope) , currentModuleName(std::move(moduleName)) , moduleResolver(moduleResolver) @@ -267,7 +240,7 @@ ConstraintSolver::ConstraintSolver(NotNull normalizer, NotNull c : constraints) + for (NotNull c : this->constraints) { unsolvedConstraints.push_back(c); @@ -310,6 +283,8 @@ void ConstraintSolver::run() { printf("Starting solver\n"); dump(this, opts); + printf("Bindings:\n"); + dumpBindings(rootScope, opts); } if (FFlag::DebugLuauLogSolverToJson) @@ -633,13 +608,6 @@ bool ConstraintSolver::tryDispatch(const BinaryConstraint& c, NotNullty.emplace(unionOfTypes(rightType, singletonTypes->booleanType, constraint->scope, false)); + { + TypeId leftFilteredTy = arena->addType(IntersectionTypeVar{{singletonTypes->falsyType, leftType}}); + + // TODO: normaliztion here should be replaced by a more limited 'simplification' + const NormalizedType* normalized = normalizer->normalize(arena->addType(UnionTypeVar{{leftFilteredTy, rightType}})); + + if (!normalized) + { + reportError(CodeTooComplex{}, constraint->location); + asMutable(resultType)->ty.emplace(errorRecoveryType()); + } + else + { + asMutable(resultType)->ty.emplace(normalizer->typeFromNormal(*normalized)); + } + unblock(resultType); return true; + } // Or evaluates to the LHS type if the LHS is truthy, and the RHS type if // LHS is falsey. case AstExprBinary::Op::Or: - asMutable(resultType)->ty.emplace(unionOfTypes(rightType, leftType, constraint->scope, true)); + { + TypeId rightFilteredTy = arena->addType(IntersectionTypeVar{{singletonTypes->truthyType, leftType}}); + + // TODO: normaliztion here should be replaced by a more limited 'simplification' + const NormalizedType* normalized = normalizer->normalize(arena->addType(UnionTypeVar{{rightFilteredTy, rightType}})); + + if (!normalized) + { + reportError(CodeTooComplex{}, constraint->location); + asMutable(resultType)->ty.emplace(errorRecoveryType()); + } + else + { + asMutable(resultType)->ty.emplace(normalizer->typeFromNormal(*normalized)); + } + unblock(resultType); return true; + } default: iceReporter.ice("Unhandled AstExprBinary::Op for binary operation", constraint->location); break; @@ -1148,6 +1148,17 @@ bool ConstraintSolver::tryDispatch(const FunctionCallConstraint& c, NotNullsecond) + block(ic, blockedConstraint); + } + } + asMutable(*c.innerConstraints.at(0)).c = InstantiationConstraint{instantiatedType, *callMm}; asMutable(*c.innerConstraints.at(1)).c = SubtypeConstraint{inferredFnType, instantiatedType}; @@ -1180,6 +1191,17 @@ bool ConstraintSolver::tryDispatch(const FunctionCallConstraint& c, NotNullsecond) + block(ic, blockedConstraint); + } + } + unsolvedConstraints.insert(end(unsolvedConstraints), begin(c.innerConstraints), end(c.innerConstraints)); asMutable(c.result)->ty.emplace(constraint->scope); } diff --git a/Analysis/src/EmbeddedBuiltinDefinitions.cpp b/Analysis/src/EmbeddedBuiltinDefinitions.cpp index 339de975..b0f21737 100644 --- a/Analysis/src/EmbeddedBuiltinDefinitions.cpp +++ b/Analysis/src/EmbeddedBuiltinDefinitions.cpp @@ -2,6 +2,7 @@ #include "Luau/BuiltinDefinitions.h" LUAU_FASTFLAG(LuauUnknownAndNeverType) +LUAU_FASTFLAG(LuauOptionalNextKey) namespace Luau { @@ -126,7 +127,7 @@ declare function rawlen(obj: {[K]: V} | string): number declare function setfenv(target: number | (T...) -> R..., env: {[string]: any}): ((T...) -> R...)? -declare function ipairs(tab: {V}): (({V}, number) -> (number, V), {V}, number) +-- TODO: place ipairs definition here with removal of FFlagLuauOptionalNextKey declare function pcall(f: (A...) -> R..., ...: A...): (boolean, R...) @@ -207,6 +208,11 @@ std::string getBuiltinDefinitionSource() else result += "declare function error(message: T, level: number?)\n"; + if (FFlag::LuauOptionalNextKey) + result += "declare function ipairs(tab: {V}): (({V}, number) -> (number?, V), {V}, number)\n"; + else + result += "declare function ipairs(tab: {V}): (({V}, number) -> (number, V), {V}, number)\n"; + return result; } diff --git a/Analysis/src/Frontend.cpp b/Analysis/src/Frontend.cpp index 39e6428d..22a9ecfa 100644 --- a/Analysis/src/Frontend.cpp +++ b/Analysis/src/Frontend.cpp @@ -955,7 +955,8 @@ ModulePtr Frontend::check( cgb.visit(sourceModule.root); result->errors = std::move(cgb.errors); - ConstraintSolver cs{NotNull{&normalizer}, NotNull(cgb.rootScope), sourceModule.name, NotNull(&moduleResolver), requireCycles, logger.get()}; + ConstraintSolver cs{NotNull{&normalizer}, NotNull(cgb.rootScope), borrowConstraints(cgb.constraints), sourceModule.name, NotNull(&moduleResolver), + requireCycles, logger.get()}; if (options.randomizeConstraintResolutionSeed) cs.randomize(*options.randomizeConstraintResolutionSeed); diff --git a/Analysis/src/Module.cpp b/Analysis/src/Module.cpp index 0412f007..62674aa8 100644 --- a/Analysis/src/Module.cpp +++ b/Analysis/src/Module.cpp @@ -14,9 +14,7 @@ #include -LUAU_FASTFLAG(LuauAnyifyModuleReturnGenerics) LUAU_FASTFLAG(DebugLuauDeferredConstraintResolution); -LUAU_FASTFLAGVARIABLE(LuauForceExportSurfacesToBeNormal, false); LUAU_FASTFLAGVARIABLE(LuauClonePublicInterfaceLess, false); LUAU_FASTFLAG(LuauSubstitutionReentrant); LUAU_FASTFLAG(LuauClassTypeVarsInSubstitution); @@ -222,18 +220,6 @@ void Module::clonePublicInterface(NotNull singletonTypes, Intern } } - if (!FFlag::LuauAnyifyModuleReturnGenerics) - { - for (TypeId ty : returnType) - { - if (get(follow(ty))) - { - auto t = asMutable(ty); - t->ty = AnyTypeVar{}; - } - } - } - for (auto& [name, ty] : declaredGlobals) { if (FFlag::LuauClonePublicInterfaceLess) diff --git a/Analysis/src/ToString.cpp b/Analysis/src/ToString.cpp index 903e156b..5fd15cf7 100644 --- a/Analysis/src/ToString.cpp +++ b/Analysis/src/ToString.cpp @@ -13,7 +13,6 @@ LUAU_FASTFLAG(DebugLuauDeferredConstraintResolution) LUAU_FASTFLAG(LuauLvaluelessPath) LUAU_FASTFLAG(LuauUnknownAndNeverType) -LUAU_FASTFLAGVARIABLE(LuauFixNameMaps, false) LUAU_FASTFLAGVARIABLE(LuauFunctionReturnStringificationFixup, false) LUAU_FASTFLAGVARIABLE(LuauUnseeArrayTtv, false) @@ -130,28 +129,15 @@ struct StringifierState bool exhaustive; - StringifierState(ToStringOptions& opts, ToStringResult& result, const std::optional& DEPRECATED_nameMap) + StringifierState(ToStringOptions& opts, ToStringResult& result) : opts(opts) , result(result) , exhaustive(opts.exhaustive) { - if (!FFlag::LuauFixNameMaps && DEPRECATED_nameMap) - result.DEPRECATED_nameMap = *DEPRECATED_nameMap; - - if (!FFlag::LuauFixNameMaps) - { - for (const auto& [_, v] : result.DEPRECATED_nameMap.typeVars) - usedNames.insert(v); - for (const auto& [_, v] : result.DEPRECATED_nameMap.typePacks) - usedNames.insert(v); - } - else - { - for (const auto& [_, v] : opts.nameMap.typeVars) - usedNames.insert(v); - for (const auto& [_, v] : opts.nameMap.typePacks) - usedNames.insert(v); - } + for (const auto& [_, v] : opts.nameMap.typeVars) + usedNames.insert(v); + for (const auto& [_, v] : opts.nameMap.typePacks) + usedNames.insert(v); } bool hasSeen(const void* tv) @@ -174,8 +160,8 @@ struct StringifierState std::string getName(TypeId ty) { - const size_t s = FFlag::LuauFixNameMaps ? opts.nameMap.typeVars.size() : result.DEPRECATED_nameMap.typeVars.size(); - std::string& n = FFlag::LuauFixNameMaps ? opts.nameMap.typeVars[ty] : result.DEPRECATED_nameMap.typeVars[ty]; + const size_t s = opts.nameMap.typeVars.size(); + std::string& n = opts.nameMap.typeVars[ty]; if (!n.empty()) return n; @@ -197,8 +183,8 @@ struct StringifierState std::string getName(TypePackId ty) { - const size_t s = FFlag::LuauFixNameMaps ? opts.nameMap.typePacks.size() : result.DEPRECATED_nameMap.typePacks.size(); - std::string& n = FFlag::LuauFixNameMaps ? opts.nameMap.typePacks[ty] : result.DEPRECATED_nameMap.typePacks[ty]; + const size_t s = opts.nameMap.typePacks.size(); + std::string& n = opts.nameMap.typePacks[ty]; if (!n.empty()) return n; @@ -405,10 +391,7 @@ struct TypeVarStringifier if (gtv.explicitName) { state.usedNames.insert(gtv.name); - if (FFlag::LuauFixNameMaps) - state.opts.nameMap.typeVars[ty] = gtv.name; - else - state.result.DEPRECATED_nameMap.typeVars[ty] = gtv.name; + state.opts.nameMap.typeVars[ty] = gtv.name; state.emit(gtv.name); } else @@ -1002,10 +985,7 @@ struct TypePackStringifier if (pack.explicitName) { state.usedNames.insert(pack.name); - if (FFlag::LuauFixNameMaps) - state.opts.nameMap.typePacks[tp] = pack.name; - else - state.result.DEPRECATED_nameMap.typePacks[tp] = pack.name; + state.opts.nameMap.typePacks[tp] = pack.name; state.emit(pack.name); } else @@ -1108,8 +1088,7 @@ ToStringResult toStringDetailed(TypeId ty, ToStringOptions& opts) ToStringResult result; - StringifierState state = - FFlag::LuauFixNameMaps ? StringifierState{opts, result, opts.nameMap} : StringifierState{opts, result, opts.DEPRECATED_nameMap}; + StringifierState state{opts, result}; std::set cycles; std::set cycleTPs; @@ -1216,8 +1195,7 @@ ToStringResult toStringDetailed(TypePackId tp, ToStringOptions& opts) * 4. Print out the root of the type using the same algorithm as step 3. */ ToStringResult result; - StringifierState state = - FFlag::LuauFixNameMaps ? StringifierState{opts, result, opts.nameMap} : StringifierState{opts, result, opts.DEPRECATED_nameMap}; + StringifierState state{opts, result}; std::set cycles; std::set cycleTPs; @@ -1303,8 +1281,7 @@ std::string toString(const TypePackVar& tp, ToStringOptions& opts) std::string toStringNamedFunction(const std::string& funcName, const FunctionTypeVar& ftv, ToStringOptions& opts) { ToStringResult result; - StringifierState state = - FFlag::LuauFixNameMaps ? StringifierState{opts, result, opts.nameMap} : StringifierState{opts, result, opts.DEPRECATED_nameMap}; + StringifierState state{opts, result}; TypeVarStringifier tvs{state}; state.emit(funcName); @@ -1436,91 +1413,84 @@ std::string toString(const Constraint& constraint, ToStringOptions& opts) auto go = [&opts](auto&& c) -> std::string { using T = std::decay_t; - // TODO: Inline and delete this function when clipping FFlag::LuauFixNameMaps - auto tos = [](auto&& a, ToStringOptions& opts) { - if (FFlag::LuauFixNameMaps) - return toString(a, opts); - else - { - ToStringResult tsr = toStringDetailed(a, opts); - opts.DEPRECATED_nameMap = std::move(tsr.DEPRECATED_nameMap); - return tsr.name; - } + auto tos = [&opts](auto&& a) + { + return toString(a, opts); }; if constexpr (std::is_same_v) { - std::string subStr = tos(c.subType, opts); - std::string superStr = tos(c.superType, opts); + std::string subStr = tos(c.subType); + std::string superStr = tos(c.superType); return subStr + " <: " + superStr; } else if constexpr (std::is_same_v) { - std::string subStr = tos(c.subPack, opts); - std::string superStr = tos(c.superPack, opts); + std::string subStr = tos(c.subPack); + std::string superStr = tos(c.superPack); return subStr + " <: " + superStr; } else if constexpr (std::is_same_v) { - std::string subStr = tos(c.generalizedType, opts); - std::string superStr = tos(c.sourceType, opts); + std::string subStr = tos(c.generalizedType); + std::string superStr = tos(c.sourceType); return subStr + " ~ gen " + superStr; } else if constexpr (std::is_same_v) { - std::string subStr = tos(c.subType, opts); - std::string superStr = tos(c.superType, opts); + std::string subStr = tos(c.subType); + std::string superStr = tos(c.superType); return subStr + " ~ inst " + superStr; } else if constexpr (std::is_same_v) { - std::string resultStr = tos(c.resultType, opts); - std::string operandStr = tos(c.operandType, opts); + std::string resultStr = tos(c.resultType); + std::string operandStr = tos(c.operandType); return resultStr + " ~ Unary<" + toString(c.op) + ", " + operandStr + ">"; } else if constexpr (std::is_same_v) { - std::string resultStr = tos(c.resultType, opts); - std::string leftStr = tos(c.leftType, opts); - std::string rightStr = tos(c.rightType, opts); + std::string resultStr = tos(c.resultType); + std::string leftStr = tos(c.leftType); + std::string rightStr = tos(c.rightType); return resultStr + " ~ Binary<" + toString(c.op) + ", " + leftStr + ", " + rightStr + ">"; } else if constexpr (std::is_same_v) { - std::string iteratorStr = tos(c.iterator, opts); - std::string variableStr = tos(c.variables, opts); + std::string iteratorStr = tos(c.iterator); + std::string variableStr = tos(c.variables); return variableStr + " ~ Iterate<" + iteratorStr + ">"; } else if constexpr (std::is_same_v) { - std::string namedStr = tos(c.namedType, opts); + std::string namedStr = tos(c.namedType); return "@name(" + namedStr + ") = " + c.name; } else if constexpr (std::is_same_v) { - std::string targetStr = tos(c.target, opts); + std::string targetStr = tos(c.target); return "expand " + targetStr; } else if constexpr (std::is_same_v) { - return "call " + tos(c.fn, opts) + " with { result = " + tos(c.result, opts) + " }"; + return "call " + tos(c.fn) + " with { result = " + tos(c.result) + " }"; } else if constexpr (std::is_same_v) { - return tos(c.resultType, opts) + " ~ prim " + tos(c.expectedType, opts) + ", " + tos(c.singletonType, opts) + ", " + - tos(c.multitonType, opts); + return tos(c.resultType) + " ~ prim " + tos(c.expectedType) + ", " + tos(c.singletonType) + ", " + + tos(c.multitonType); } else if constexpr (std::is_same_v) { - return tos(c.resultType, opts) + " ~ hasProp " + tos(c.subjectType, opts) + ", \"" + c.prop + "\""; + return tos(c.resultType) + " ~ hasProp " + tos(c.subjectType) + ", \"" + c.prop + "\""; } else if constexpr (std::is_same_v) { - std::string result = tos(c.resultType, opts); - std::string discriminant = tos(c.discriminantType, opts); + std::string result = tos(c.resultType); + std::string discriminant = tos(c.discriminantType); return result + " ~ if isSingleton D then ~D else unknown where D = " + discriminant; } diff --git a/Analysis/src/TypeChecker2.cpp b/Analysis/src/TypeChecker2.cpp index dde41a65..03575c40 100644 --- a/Analysis/src/TypeChecker2.cpp +++ b/Analysis/src/TypeChecker2.cpp @@ -610,7 +610,7 @@ struct TypeChecker2 visit(rhs); TypeId rhsType = lookupType(rhs); - if (!isSubtype(rhsType, lhsType, stack.back(), singletonTypes, ice)) + if (!isSubtype(rhsType, lhsType, stack.back())) { reportError(TypeMismatch{lhsType, rhsType}, rhs->location); } @@ -761,7 +761,7 @@ struct TypeChecker2 TypeId actualType = lookupType(number); TypeId numberType = singletonTypes->numberType; - if (!isSubtype(numberType, actualType, stack.back(), singletonTypes, ice)) + if (!isSubtype(numberType, actualType, stack.back())) { reportError(TypeMismatch{actualType, numberType}, number->location); } @@ -772,7 +772,7 @@ struct TypeChecker2 TypeId actualType = lookupType(string); TypeId stringType = singletonTypes->stringType; - if (!isSubtype(actualType, stringType, stack.back(), singletonTypes, ice)) + if (!isSubtype(actualType, stringType, stack.back())) { reportError(TypeMismatch{actualType, stringType}, string->location); } @@ -861,7 +861,7 @@ struct TypeChecker2 FunctionTypeVar ftv{argsTp, expectedRetType}; TypeId expectedType = arena.addType(ftv); - if (!isSubtype(testFunctionType, expectedType, stack.back(), singletonTypes, ice)) + if (!isSubtype(testFunctionType, expectedType, stack.back())) { CloneState cloneState; expectedType = clone(expectedType, module->internalTypes, cloneState); @@ -880,7 +880,7 @@ struct TypeChecker2 getIndexTypeFromType(module->getModuleScope(), leftType, indexName->index.value, indexName->location, /* addErrors */ true); if (ty) { - if (!isSubtype(resultType, *ty, stack.back(), singletonTypes, ice)) + if (!isSubtype(resultType, *ty, stack.back())) { reportError(TypeMismatch{resultType, *ty}, indexName->location); } @@ -913,7 +913,7 @@ struct TypeChecker2 TypeId inferredArgTy = *argIt; TypeId annotatedArgTy = lookupAnnotation(arg->annotation); - if (!isSubtype(annotatedArgTy, inferredArgTy, stack.back(), singletonTypes, ice)) + if (!isSubtype(annotatedArgTy, inferredArgTy, stack.back())) { reportError(TypeMismatch{annotatedArgTy, inferredArgTy}, arg->location); } @@ -954,7 +954,7 @@ struct TypeChecker2 if (const FunctionTypeVar* ftv = get(follow(*mm))) { TypePackId expectedArgs = module->internalTypes.addTypePack({operandType}); - reportErrors(tryUnify(scope, expr->location, ftv->argTypes, expectedArgs)); + reportErrors(tryUnify(scope, expr->location, expectedArgs, ftv->argTypes)); if (std::optional ret = first(ftv->retTypes)) { @@ -1096,7 +1096,7 @@ struct TypeChecker2 expr->op == AstExprBinary::CompareGt || expr->op == AstExprBinary::Op::CompareLe || expr->op == AstExprBinary::Op::CompareLt) { TypePackId expectedRets = module->internalTypes.addTypePack({singletonTypes->booleanType}); - if (!isSubtype(ftv->retTypes, expectedRets, scope, singletonTypes, ice)) + if (!isSubtype(ftv->retTypes, expectedRets, scope)) { reportError(GenericError{format("Metamethod '%s' must return type 'boolean'", it->second)}, expr->location); } @@ -1207,10 +1207,10 @@ struct TypeChecker2 TypeId computedType = lookupType(expr->expr); // Note: As an optimization, we try 'number <: number | string' first, as that is the more likely case. - if (isSubtype(annotationType, computedType, stack.back(), singletonTypes, ice)) + if (isSubtype(annotationType, computedType, stack.back())) return; - if (isSubtype(computedType, annotationType, stack.back(), singletonTypes, ice)) + if (isSubtype(computedType, annotationType, stack.back())) return; reportError(TypesAreUnrelated{computedType, annotationType}, expr->location); @@ -1505,12 +1505,27 @@ struct TypeChecker2 } } + template + bool isSubtype(TID subTy, TID superTy, NotNull scope) + { + UnifierSharedState sharedState{&ice}; + TypeArena arena; + Normalizer normalizer{&arena, singletonTypes, NotNull{&sharedState}}; + Unifier u{NotNull{&normalizer}, Mode::Strict, scope, Location{}, Covariant}; + u.useScopes = true; + + u.tryUnify(subTy, superTy); + const bool ok = u.errors.empty() && u.log.empty(); + return ok; + } + template ErrorVec tryUnify(NotNull scope, const Location& location, TID subTy, TID superTy) { UnifierSharedState sharedState{&ice}; Normalizer normalizer{&module->internalTypes, singletonTypes, NotNull{&sharedState}}; Unifier u{NotNull{&normalizer}, Mode::Strict, scope, location, Covariant}; + u.useScopes = true; u.tryUnify(subTy, superTy); return std::move(u.errors); diff --git a/Analysis/src/TypeInfer.cpp b/Analysis/src/TypeInfer.cpp index ccb1490a..8ecd45bd 100644 --- a/Analysis/src/TypeInfer.cpp +++ b/Analysis/src/TypeInfer.cpp @@ -35,18 +35,20 @@ LUAU_FASTFLAG(LuauTypeNormalization2) LUAU_FASTFLAGVARIABLE(DebugLuauFreezeDuringUnification, false) LUAU_FASTFLAGVARIABLE(LuauReturnAnyInsteadOfICE, false) // Eventually removed as false. LUAU_FASTFLAGVARIABLE(DebugLuauSharedSelf, false) -LUAU_FASTFLAGVARIABLE(LuauAnyifyModuleReturnGenerics, false) LUAU_FASTFLAGVARIABLE(LuauLvaluelessPath, false) +LUAU_FASTFLAGVARIABLE(LuauNilIterator, false) LUAU_FASTFLAGVARIABLE(LuauUnknownAndNeverType, false) LUAU_FASTFLAGVARIABLE(LuauBinaryNeedsExpectedTypesToo, false) -LUAU_FASTFLAGVARIABLE(LuauFixVarargExprHeadType, false) LUAU_FASTFLAGVARIABLE(LuauNeverTypesAndOperatorsInference, false) LUAU_FASTFLAGVARIABLE(LuauReturnsFromCallsitesAreNotWidened, false) +LUAU_FASTFLAGVARIABLE(LuauTryhardAnd, false) LUAU_FASTFLAG(LuauInstantiateInSubtyping) LUAU_FASTFLAGVARIABLE(LuauCompleteVisitor, false) +LUAU_FASTFLAGVARIABLE(LuauOptionalNextKey, false) LUAU_FASTFLAGVARIABLE(LuauReportShadowedTypeAlias, false) LUAU_FASTFLAGVARIABLE(LuauBetterMessagingOnCountMismatch, false) LUAU_FASTFLAGVARIABLE(LuauArgMismatchReportFunctionLocation, false) +LUAU_FASTFLAGVARIABLE(LuauImplicitElseRefinement, false) namespace Luau { @@ -331,8 +333,7 @@ ModulePtr TypeChecker::checkWithoutRecursionCheck(const SourceModule& module, Mo else moduleScope->returnType = anyify(moduleScope, moduleScope->returnType, Location{}); - if (FFlag::LuauAnyifyModuleReturnGenerics) - moduleScope->returnType = anyifyModuleReturnTypePackGenerics(moduleScope->returnType); + moduleScope->returnType = anyifyModuleReturnTypePackGenerics(moduleScope->returnType); for (auto& [_, typeFun] : moduleScope->exportedTypeBindings) typeFun.type = anyify(moduleScope, typeFun.type, Location{}); @@ -1209,10 +1210,10 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatForIn& forin) AstExpr* firstValue = forin.values.data[0]; // next is a function that takes Table and an optional index of type K - // next(t: Table, index: K | nil) -> (K, V) + // next(t: Table, index: K | nil) -> (K?, V) // however, pairs and ipairs are quite messy, but they both share the same types // pairs returns 'next, t, nil', thus the type would be - // pairs(t: Table) -> ((Table, K | nil) -> (K, V), Table, K | nil) + // pairs(t: Table) -> ((Table, K | nil) -> (K?, V), Table, K | nil) // ipairs returns 'next, t, 0', thus ipairs will also share the same type as pairs, except K = number // // we can also define our own custom iterators by by returning a wrapped coroutine that calls coroutine.yield @@ -1255,6 +1256,9 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatForIn& forin) iterTy = instantiate(scope, checkExpr(scope, *firstValue).type, firstValue->location); } + if (FFlag::LuauNilIterator) + iterTy = stripFromNilAndReport(iterTy, firstValue->location); + if (std::optional iterMM = findMetatableEntry(iterTy, "__iter", firstValue->location, /* addErrors= */ true)) { // if __iter metamethod is present, it will be called and the results are going to be called as if they are functions @@ -1338,21 +1342,61 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatForIn& forin) reportErrors(state.errors); } - TypePackId varPack = addTypePack(TypePackVar{TypePack{varTypes, freshTypePack(scope)}}); - - if (forin.values.size >= 2) + if (FFlag::LuauOptionalNextKey) { - AstArray arguments{forin.values.data + 1, forin.values.size - 1}; + TypePackId retPack = iterFunc->retTypes; - Position start = firstValue->location.begin; - Position end = values[forin.values.size - 1]->location.end; - AstExprCall exprCall{Location(start, end), firstValue, arguments, /* self= */ false, Location()}; + if (forin.values.size >= 2) + { + AstArray arguments{forin.values.data + 1, forin.values.size - 1}; + + Position start = firstValue->location.begin; + Position end = values[forin.values.size - 1]->location.end; + AstExprCall exprCall{Location(start, end), firstValue, arguments, /* self= */ false, Location()}; + + retPack = checkExprPack(scope, exprCall).type; + } + + // We need to remove 'nil' from the set of options of the first return value + // Because for loop stops when it gets 'nil', this result is never actually assigned to the first variable + if (std::optional fty = first(retPack); fty && !varTypes.empty()) + { + TypeId keyTy = follow(*fty); + + if (get(keyTy)) + { + if (std::optional ty = tryStripUnionFromNil(keyTy)) + keyTy = *ty; + } + + unify(keyTy, varTypes.front(), scope, forin.location); + + // We have already handled the first variable type, make it match in the pack check + varTypes.front() = *fty; + } + + TypePackId varPack = addTypePack(TypePackVar{TypePack{varTypes, freshTypePack(scope)}}); - TypePackId retPack = checkExprPack(scope, exprCall).type; unify(retPack, varPack, scope, forin.location); } else - unify(iterFunc->retTypes, varPack, scope, forin.location); + { + TypePackId varPack = addTypePack(TypePackVar{TypePack{varTypes, freshTypePack(scope)}}); + + if (forin.values.size >= 2) + { + AstArray arguments{forin.values.data + 1, forin.values.size - 1}; + + Position start = firstValue->location.begin; + Position end = values[forin.values.size - 1]->location.end; + AstExprCall exprCall{Location(start, end), firstValue, arguments, /* self= */ false, Location()}; + + TypePackId retPack = checkExprPack(scope, exprCall).type; + unify(retPack, varPack, scope, forin.location); + } + else + unify(iterFunc->retTypes, varPack, scope, forin.location); + } check(loopScope, *forin.body); } @@ -1855,18 +1899,10 @@ WithPredicate TypeChecker::checkExpr(const ScopePtr& scope, const AstExp if (get(varargPack)) { - if (FFlag::LuauFixVarargExprHeadType) - { - if (std::optional ty = first(varargPack)) - return {*ty}; + if (std::optional ty = first(varargPack)) + return {*ty}; - return {nilType}; - } - else - { - std::vector types = flatten(varargPack).first; - return {!types.empty() ? types[0] : nilType}; - } + return {nilType}; } else if (get(varargPack)) { @@ -2717,12 +2753,54 @@ TypeId TypeChecker::checkRelationalOperation( case AstExprBinary::And: if (lhsIsAny) + { return lhsType; - return unionOfTypes(rhsType, booleanType, scope, expr.location, false); + } + else if (FFlag::LuauTryhardAnd) + { + // If lhs is free, we can't tell which 'falsy' components it has, if any + if (get(lhsType)) + return unionOfTypes(addType(UnionTypeVar{{nilType, singletonType(false)}}), rhsType, scope, expr.location, false); + + auto [oty, notNever] = pickTypesFromSense(lhsType, false, neverType); // Filter out falsy types + + if (notNever) + { + LUAU_ASSERT(oty); + return unionOfTypes(*oty, rhsType, scope, expr.location, false); + } + else + { + return rhsType; + } + } + else + { + return unionOfTypes(rhsType, booleanType, scope, expr.location, false); + } case AstExprBinary::Or: if (lhsIsAny) + { return lhsType; - return unionOfTypes(lhsType, rhsType, scope, expr.location); + } + else if (FFlag::LuauTryhardAnd) + { + auto [oty, notNever] = pickTypesFromSense(lhsType, true, neverType); // Filter out truthy types + + if (notNever) + { + LUAU_ASSERT(oty); + return unionOfTypes(*oty, rhsType, scope, expr.location); + } + else + { + return rhsType; + } + } + else + { + return unionOfTypes(lhsType, rhsType, scope, expr.location); + } default: LUAU_ASSERT(0); ice(format("checkRelationalOperation called with incorrect binary expression '%s'", toString(expr.op).c_str()), expr.location); @@ -4840,9 +4918,9 @@ TypePackId TypeChecker::errorRecoveryTypePack(TypePackId guess) return singletonTypes->errorRecoveryTypePack(guess); } -TypeIdPredicate TypeChecker::mkTruthyPredicate(bool sense) +TypeIdPredicate TypeChecker::mkTruthyPredicate(bool sense, TypeId emptySetTy) { - return [this, sense](TypeId ty) -> std::optional { + return [this, sense, emptySetTy](TypeId ty) -> std::optional { // any/error/free gets a special pass unconditionally because they can't be decided. if (get(ty) || get(ty) || get(ty)) return ty; @@ -4860,7 +4938,7 @@ TypeIdPredicate TypeChecker::mkTruthyPredicate(bool sense) return sense ? std::nullopt : std::optional(ty); // at this point, anything else is kept if sense is true, or replaced by nil - return sense ? ty : nilType; + return sense ? ty : emptySetTy; }; } @@ -4886,9 +4964,9 @@ std::pair, bool> TypeChecker::filterMap(TypeId type, TypeI } } -std::pair, bool> TypeChecker::pickTypesFromSense(TypeId type, bool sense) +std::pair, bool> TypeChecker::pickTypesFromSense(TypeId type, bool sense, TypeId emptySetTy) { - return filterMap(type, mkTruthyPredicate(sense)); + return filterMap(type, mkTruthyPredicate(sense, emptySetTy)); } TypeId TypeChecker::addTV(TypeVar&& tv) @@ -5657,7 +5735,7 @@ void TypeChecker::resolve(const TruthyPredicate& truthyP, RefinementMap& refis, if (ty && fromOr) return addRefinement(refis, truthyP.lvalue, *ty); - refineLValue(truthyP.lvalue, refis, scope, mkTruthyPredicate(sense)); + refineLValue(truthyP.lvalue, refis, scope, mkTruthyPredicate(sense, nilType)); } void TypeChecker::resolve(const AndPredicate& andP, RefinementMap& refis, const ScopePtr& scope, bool sense) @@ -5850,13 +5928,57 @@ void TypeChecker::resolve(const EqPredicate& eqP, RefinementMap& refis, const Sc if (maybeSingleton(eqP.type)) { - // Normally we'd write option <: eqP.type, but singletons are always the subtype, so we flip this. - if (!sense || canUnify(eqP.type, option, scope, eqP.location).empty()) - return sense ? eqP.type : option; + if (FFlag::LuauImplicitElseRefinement) + { + bool optionIsSubtype = canUnify(option, eqP.type, scope, eqP.location).empty(); + bool targetIsSubtype = canUnify(eqP.type, option, scope, eqP.location).empty(); - // local variable works around an odd gcc 9.3 warning: may be used uninitialized - std::optional res = std::nullopt; - return res; + // terminology refresher: + // - option is the type of the expression `x`, and + // - eqP.type is the type of the expression `"hello"` + // + // "hello" == x where + // x : "hello" | "world" -> x : "hello" + // x : number | string -> x : "hello" + // x : number -> x : never + // + // "hello" ~= x where + // x : "hello" | "world" -> x : "world" + // x : number | string -> x : number | string + // x : number -> x : number + + // local variable works around an odd gcc 9.3 warning: may be used uninitialized + std::optional nope = std::nullopt; + + if (sense) + { + if (optionIsSubtype && !targetIsSubtype) + return option; + else if (!optionIsSubtype && targetIsSubtype) + return eqP.type; + else if (!optionIsSubtype && !targetIsSubtype) + return nope; + else if (optionIsSubtype && targetIsSubtype) + return eqP.type; + } + else + { + bool isOptionSingleton = get(option); + if (!isOptionSingleton) + return option; + else if (optionIsSubtype && targetIsSubtype) + return nope; + } + } + else + { + if (!sense || canUnify(eqP.type, option, scope, eqP.location).empty()) + return sense ? eqP.type : option; + + // local variable works around an odd gcc 9.3 warning: may be used uninitialized + std::optional res = std::nullopt; + return res; + } } return option; diff --git a/Analysis/src/TypeVar.cpp b/Analysis/src/TypeVar.cpp index de0890e1..814eca0d 100644 --- a/Analysis/src/TypeVar.cpp +++ b/Analysis/src/TypeVar.cpp @@ -3,6 +3,7 @@ #include "Luau/BuiltinDefinitions.h" #include "Luau/Common.h" +#include "Luau/ConstraintSolver.h" #include "Luau/DenseHash.h" #include "Luau/Error.h" #include "Luau/RecursionCounter.h" @@ -26,6 +27,7 @@ LUAU_FASTINT(LuauTypeInferRecursionLimit) LUAU_FASTFLAG(LuauUnknownAndNeverType) LUAU_FASTFLAGVARIABLE(LuauMaybeGenericIntersectionTypes, false) LUAU_FASTFLAGVARIABLE(LuauNoMoreGlobalSingletonTypes, false) +LUAU_FASTFLAGVARIABLE(LuauNewLibraryTypeNames, false) LUAU_FASTFLAG(LuauInstantiateInSubtyping) namespace Luau @@ -33,15 +35,19 @@ namespace Luau std::optional> magicFunctionFormat( TypeChecker& typechecker, const ScopePtr& scope, const AstExprCall& expr, WithPredicate withPredicate); +static bool dcrMagicFunctionFormat(MagicFunctionCallContext context); static std::optional> magicFunctionGmatch( TypeChecker& typechecker, const ScopePtr& scope, const AstExprCall& expr, WithPredicate withPredicate); +static bool dcrMagicFunctionGmatch(MagicFunctionCallContext context); static std::optional> magicFunctionMatch( TypeChecker& typechecker, const ScopePtr& scope, const AstExprCall& expr, WithPredicate withPredicate); +static bool dcrMagicFunctionMatch(MagicFunctionCallContext context); static std::optional> magicFunctionFind( TypeChecker& typechecker, const ScopePtr& scope, const AstExprCall& expr, WithPredicate withPredicate); +static bool dcrMagicFunctionFind(MagicFunctionCallContext context); TypeId follow(TypeId t) { @@ -800,6 +806,7 @@ TypeId SingletonTypes::makeStringMetatable() FunctionTypeVar formatFTV{arena->addTypePack(TypePack{{stringType}, anyTypePack}), oneStringPack}; formatFTV.magicFunction = &magicFunctionFormat; const TypeId formatFn = arena->addType(formatFTV); + attachDcrMagicFunction(formatFn, dcrMagicFunctionFormat); const TypePackId emptyPack = arena->addTypePack({}); const TypePackId stringVariadicList = arena->addTypePack(TypePackVar{VariadicTypePack{stringType}}); @@ -814,14 +821,17 @@ TypeId SingletonTypes::makeStringMetatable() const TypeId gmatchFunc = makeFunction(*arena, stringType, {}, {}, {stringType}, {}, {arena->addType(FunctionTypeVar{emptyPack, stringVariadicList})}); attachMagicFunction(gmatchFunc, magicFunctionGmatch); + attachDcrMagicFunction(gmatchFunc, dcrMagicFunctionGmatch); const TypeId matchFunc = arena->addType( FunctionTypeVar{arena->addTypePack({stringType, stringType, optionalNumber}), arena->addTypePack(TypePackVar{VariadicTypePack{stringType}})}); attachMagicFunction(matchFunc, magicFunctionMatch); + attachDcrMagicFunction(matchFunc, dcrMagicFunctionMatch); const TypeId findFunc = arena->addType(FunctionTypeVar{arena->addTypePack({stringType, stringType, optionalNumber, optionalBoolean}), arena->addTypePack(TypePack{{optionalNumber, optionalNumber}, stringVariadicList})}); attachMagicFunction(findFunc, magicFunctionFind); + attachDcrMagicFunction(findFunc, dcrMagicFunctionFind); TableTypeVar::Props stringLib = { {"byte", {arena->addType(FunctionTypeVar{arena->addTypePack({stringType, optionalNumber, optionalNumber}), numberVariadicList})}}, @@ -855,7 +865,7 @@ TypeId SingletonTypes::makeStringMetatable() TypeId tableType = arena->addType(TableTypeVar{std::move(stringLib), std::nullopt, TypeLevel{}, TableState::Sealed}); if (TableTypeVar* ttv = getMutable(tableType)) - ttv->name = "string"; + ttv->name = FFlag::LuauNewLibraryTypeNames ? "typeof(string)" : "string"; return arena->addType(TableTypeVar{{{{"__index", {tableType}}}}, std::nullopt, TypeLevel{}, TableState::Sealed}); } @@ -1072,7 +1082,7 @@ IntersectionTypeVarIterator end(const IntersectionTypeVar* itv) return IntersectionTypeVarIterator{}; } -static std::vector parseFormatString(TypeChecker& typechecker, const char* data, size_t size) +static std::vector parseFormatString(NotNull singletonTypes, const char* data, size_t size) { const char* options = "cdiouxXeEfgGqs*"; @@ -1095,13 +1105,13 @@ static std::vector parseFormatString(TypeChecker& typechecker, const cha break; if (data[i] == 'q' || data[i] == 's') - result.push_back(typechecker.stringType); + result.push_back(singletonTypes->stringType); else if (data[i] == '*') - result.push_back(typechecker.unknownType); + result.push_back(singletonTypes->unknownType); else if (strchr(options, data[i])) - result.push_back(typechecker.numberType); + result.push_back(singletonTypes->numberType); else - result.push_back(typechecker.errorRecoveryType(typechecker.anyType)); + result.push_back(singletonTypes->errorRecoveryType(singletonTypes->anyType)); } } @@ -1130,7 +1140,7 @@ std::optional> magicFunctionFormat( if (!fmt) return std::nullopt; - std::vector expected = parseFormatString(typechecker, fmt->value.data, fmt->value.size); + std::vector expected = parseFormatString(typechecker.singletonTypes, fmt->value.data, fmt->value.size); const auto& [params, tail] = flatten(paramPack); size_t paramOffset = 1; @@ -1154,7 +1164,50 @@ std::optional> magicFunctionFormat( return WithPredicate{arena.addTypePack({typechecker.stringType})}; } -static std::vector parsePatternString(TypeChecker& typechecker, const char* data, size_t size) +static bool dcrMagicFunctionFormat(MagicFunctionCallContext context) +{ + TypeArena* arena = context.solver->arena; + + AstExprConstantString* fmt = nullptr; + if (auto index = context.callSite->func->as(); index && context.callSite->self) + { + if (auto group = index->expr->as()) + fmt = group->expr->as(); + else + fmt = index->expr->as(); + } + + if (!context.callSite->self && context.callSite->args.size > 0) + fmt = context.callSite->args.data[0]->as(); + + if (!fmt) + return false; + + std::vector expected = parseFormatString(context.solver->singletonTypes, fmt->value.data, fmt->value.size); + const auto& [params, tail] = flatten(context.arguments); + + size_t paramOffset = 1; + + // unify the prefix one argument at a time + for (size_t i = 0; i < expected.size() && i + paramOffset < params.size(); ++i) + { + context.solver->unify(params[i + paramOffset], expected[i], context.solver->rootScope); + } + + // if we know the argument count or if we have too many arguments for sure, we can issue an error + size_t numActualParams = params.size(); + size_t numExpectedParams = expected.size() + 1; // + 1 for the format string + + if (numExpectedParams != numActualParams && (!tail || numExpectedParams < numActualParams)) + context.solver->reportError(TypeError{context.callSite->location, CountMismatch{numExpectedParams, std::nullopt, numActualParams}}); + + TypePackId resultPack = arena->addTypePack({context.solver->singletonTypes->stringType}); + asMutable(context.result)->ty.emplace(resultPack); + + return true; +} + +static std::vector parsePatternString(NotNull singletonTypes, const char* data, size_t size) { std::vector result; int depth = 0; @@ -1186,12 +1239,12 @@ static std::vector parsePatternString(TypeChecker& typechecker, const ch if (i + 1 < size && data[i + 1] == ')') { i++; - result.push_back(typechecker.numberType); + result.push_back(singletonTypes->numberType); continue; } ++depth; - result.push_back(typechecker.stringType); + result.push_back(singletonTypes->stringType); } else if (data[i] == ')') { @@ -1209,7 +1262,7 @@ static std::vector parsePatternString(TypeChecker& typechecker, const ch return std::vector(); if (result.empty()) - result.push_back(typechecker.stringType); + result.push_back(singletonTypes->stringType); return result; } @@ -1233,7 +1286,7 @@ static std::optional> magicFunctionGmatch( if (!pattern) return std::nullopt; - std::vector returnTypes = parsePatternString(typechecker, pattern->value.data, pattern->value.size); + std::vector returnTypes = parsePatternString(typechecker.singletonTypes, pattern->value.data, pattern->value.size); if (returnTypes.empty()) return std::nullopt; @@ -1246,6 +1299,39 @@ static std::optional> magicFunctionGmatch( return WithPredicate{arena.addTypePack({iteratorType})}; } +static bool dcrMagicFunctionGmatch(MagicFunctionCallContext context) +{ + const auto& [params, tail] = flatten(context.arguments); + + if (params.size() != 2) + return false; + + TypeArena* arena = context.solver->arena; + + AstExprConstantString* pattern = nullptr; + size_t index = context.callSite->self ? 0 : 1; + if (context.callSite->args.size > index) + pattern = context.callSite->args.data[index]->as(); + + if (!pattern) + return false; + + std::vector returnTypes = parsePatternString(context.solver->singletonTypes, pattern->value.data, pattern->value.size); + + if (returnTypes.empty()) + return false; + + context.solver->unify(params[0], context.solver->singletonTypes->stringType, context.solver->rootScope); + + const TypePackId emptyPack = arena->addTypePack({}); + const TypePackId returnList = arena->addTypePack(returnTypes); + const TypeId iteratorType = arena->addType(FunctionTypeVar{emptyPack, returnList}); + const TypePackId resTypePack = arena->addTypePack({iteratorType}); + asMutable(context.result)->ty.emplace(resTypePack); + + return true; +} + static std::optional> magicFunctionMatch( TypeChecker& typechecker, const ScopePtr& scope, const AstExprCall& expr, WithPredicate withPredicate) { @@ -1265,7 +1351,7 @@ static std::optional> magicFunctionMatch( if (!pattern) return std::nullopt; - std::vector returnTypes = parsePatternString(typechecker, pattern->value.data, pattern->value.size); + std::vector returnTypes = parsePatternString(typechecker.singletonTypes, pattern->value.data, pattern->value.size); if (returnTypes.empty()) return std::nullopt; @@ -1282,6 +1368,42 @@ static std::optional> magicFunctionMatch( return WithPredicate{returnList}; } +static bool dcrMagicFunctionMatch(MagicFunctionCallContext context) +{ + const auto& [params, tail] = flatten(context.arguments); + + if (params.size() < 2 || params.size() > 3) + return false; + + TypeArena* arena = context.solver->arena; + + AstExprConstantString* pattern = nullptr; + size_t patternIndex = context.callSite->self ? 0 : 1; + if (context.callSite->args.size > patternIndex) + pattern = context.callSite->args.data[patternIndex]->as(); + + if (!pattern) + return false; + + std::vector returnTypes = parsePatternString(context.solver->singletonTypes, pattern->value.data, pattern->value.size); + + if (returnTypes.empty()) + return false; + + context.solver->unify(params[0], context.solver->singletonTypes->stringType, context.solver->rootScope); + + const TypeId optionalNumber = arena->addType(UnionTypeVar{{context.solver->singletonTypes->nilType, context.solver->singletonTypes->numberType}}); + + size_t initIndex = context.callSite->self ? 1 : 2; + if (params.size() == 3 && context.callSite->args.size > initIndex) + context.solver->unify(params[2], optionalNumber, context.solver->rootScope); + + const TypePackId returnList = arena->addTypePack(returnTypes); + asMutable(context.result)->ty.emplace(returnList); + + return true; +} + static std::optional> magicFunctionFind( TypeChecker& typechecker, const ScopePtr& scope, const AstExprCall& expr, WithPredicate withPredicate) { @@ -1312,7 +1434,7 @@ static std::optional> magicFunctionFind( std::vector returnTypes; if (!plain) { - returnTypes = parsePatternString(typechecker, pattern->value.data, pattern->value.size); + returnTypes = parsePatternString(typechecker.singletonTypes, pattern->value.data, pattern->value.size); if (returnTypes.empty()) return std::nullopt; @@ -1336,6 +1458,60 @@ static std::optional> magicFunctionFind( return WithPredicate{returnList}; } +static bool dcrMagicFunctionFind(MagicFunctionCallContext context) +{ + const auto& [params, tail] = flatten(context.arguments); + + if (params.size() < 2 || params.size() > 4) + return false; + + TypeArena* arena = context.solver->arena; + NotNull singletonTypes = context.solver->singletonTypes; + + AstExprConstantString* pattern = nullptr; + size_t patternIndex = context.callSite->self ? 0 : 1; + if (context.callSite->args.size > patternIndex) + pattern = context.callSite->args.data[patternIndex]->as(); + + if (!pattern) + return false; + + bool plain = false; + size_t plainIndex = context.callSite->self ? 2 : 3; + if (context.callSite->args.size > plainIndex) + { + AstExprConstantBool* p = context.callSite->args.data[plainIndex]->as(); + plain = p && p->value; + } + + std::vector returnTypes; + if (!plain) + { + returnTypes = parsePatternString(singletonTypes, pattern->value.data, pattern->value.size); + + if (returnTypes.empty()) + return false; + } + + context.solver->unify(params[0], singletonTypes->stringType, context.solver->rootScope); + + const TypeId optionalNumber = arena->addType(UnionTypeVar{{singletonTypes->nilType, singletonTypes->numberType}}); + const TypeId optionalBoolean = arena->addType(UnionTypeVar{{singletonTypes->nilType, singletonTypes->booleanType}}); + + size_t initIndex = context.callSite->self ? 1 : 2; + if (params.size() >= 3 && context.callSite->args.size > initIndex) + context.solver->unify(params[2], optionalNumber, context.solver->rootScope); + + if (params.size() == 4 && context.callSite->args.size > plainIndex) + context.solver->unify(params[3], optionalBoolean, context.solver->rootScope); + + returnTypes.insert(returnTypes.begin(), {optionalNumber, optionalNumber}); + + const TypePackId returnList = arena->addTypePack(returnTypes); + asMutable(context.result)->ty.emplace(returnList); + return true; +} + std::vector filterMap(TypeId type, TypeIdPredicate predicate) { type = follow(type); diff --git a/Analysis/src/Unifier.cpp b/Analysis/src/Unifier.cpp index df5d86f1..4dc90983 100644 --- a/Analysis/src/Unifier.cpp +++ b/Analysis/src/Unifier.cpp @@ -22,6 +22,7 @@ LUAU_FASTFLAGVARIABLE(LuauSubtypeNormalizer, false); LUAU_FASTFLAGVARIABLE(LuauScalarShapeSubtyping, false) LUAU_FASTFLAGVARIABLE(LuauInstantiateInSubtyping, false) LUAU_FASTFLAGVARIABLE(LuauOverloadedFunctionSubtypingPerf, false); +LUAU_FASTFLAGVARIABLE(LuauScalarShapeUnifyToMtOwner, false) LUAU_FASTFLAG(LuauClassTypeVarsInSubstitution) LUAU_FASTFLAG(DebugLuauDeferredConstraintResolution) LUAU_FASTFLAG(LuauNegatedFunctionTypes) @@ -1699,8 +1700,20 @@ void Unifier::tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) // Recursive unification can change the txn log, and invalidate the old // table. If we detect that this has happened, we start over, with the updated // txn log. - TableTypeVar* newSuperTable = log.getMutable(superTy); - TableTypeVar* newSubTable = log.getMutable(subTy); + TypeId superTyNew = FFlag::LuauScalarShapeUnifyToMtOwner ? log.follow(superTy) : superTy; + TypeId subTyNew = FFlag::LuauScalarShapeUnifyToMtOwner ? log.follow(subTy) : subTy; + + if (FFlag::LuauScalarShapeUnifyToMtOwner) + { + // If one of the types stopped being a table altogether, we need to restart from the top + if ((superTy != superTyNew || subTy != subTyNew) && errors.empty()) + return tryUnify(subTy, superTy, false, isIntersection); + } + + // Otherwise, restart only the table unification + TableTypeVar* newSuperTable = log.getMutable(superTyNew); + TableTypeVar* newSubTable = log.getMutable(subTyNew); + if (superTable != newSuperTable || (subTable != newSubTable && subTable != instantiatedSubTable)) { if (errors.empty()) @@ -1862,7 +1875,9 @@ void Unifier::tryUnifyScalarShape(TypeId subTy, TypeId superTy, bool reversed) if (reversed) std::swap(subTy, superTy); - if (auto ttv = log.get(superTy); !ttv || ttv->state != TableState::Free) + TableTypeVar* superTable = log.getMutable(superTy); + + if (!superTable || superTable->state != TableState::Free) return reportError(location, TypeMismatch{osuperTy, osubTy}); auto fail = [&](std::optional e) { @@ -1887,6 +1902,20 @@ void Unifier::tryUnifyScalarShape(TypeId subTy, TypeId superTy, bool reversed) Unifier child = makeChildUnifier(); child.tryUnify_(ty, superTy); + if (FFlag::LuauScalarShapeUnifyToMtOwner) + { + // To perform subtype <: free table unification, we have tried to unify (subtype's metatable) <: free table + // There is a chance that it was unified with the origial subtype, but then, (subtype's metatable) <: subtype could've failed + // Here we check if we have a new supertype instead of the original free table and try original subtype <: new supertype check + TypeId newSuperTy = child.log.follow(superTy); + + if (superTy != newSuperTy && canUnify(subTy, newSuperTy).empty()) + { + log.replace(superTy, BoundTypeVar{subTy}); + return; + } + } + if (auto e = hasUnificationTooComplex(child.errors)) reportError(*e); else if (!child.errors.empty()) @@ -1894,6 +1923,14 @@ void Unifier::tryUnifyScalarShape(TypeId subTy, TypeId superTy, bool reversed) log.concat(std::move(child.log)); + if (FFlag::LuauScalarShapeUnifyToMtOwner) + { + // To perform subtype <: free table unification, we have tried to unify (subtype's metatable) <: free table + // We return success because subtype <: free table which means that correct unification is to replace free table with the subtype + if (child.errors.empty()) + log.replace(superTy, BoundTypeVar{subTy}); + } + return; } else diff --git a/Ast/src/Parser.cpp b/Ast/src/Parser.cpp index 4c0cc125..8338a04a 100644 --- a/Ast/src/Parser.cpp +++ b/Ast/src/Parser.cpp @@ -27,6 +27,8 @@ LUAU_FASTFLAGVARIABLE(LuauInterpolatedStringBaseSupport, false) LUAU_FASTFLAGVARIABLE(LuauCommaParenWarnings, false) LUAU_FASTFLAGVARIABLE(LuauTableConstructorRecovery, false) +LUAU_FASTFLAGVARIABLE(LuauParserErrorsOnMissingDefaultTypePackArgument, false) + bool lua_telemetry_parsed_out_of_range_bin_integer = false; bool lua_telemetry_parsed_out_of_range_hex_integer = false; bool lua_telemetry_parsed_double_prefix_hex_integer = false; @@ -2503,7 +2505,7 @@ std::pair, AstArray> Parser::parseG namePacks.push_back({name, nameLocation, typePack}); } - else if (lexer.current().type == '(') + else if (!FFlag::LuauParserErrorsOnMissingDefaultTypePackArgument && lexer.current().type == '(') { auto [type, typePack] = parseTypeOrPackAnnotation(); @@ -2512,6 +2514,15 @@ std::pair, AstArray> Parser::parseG namePacks.push_back({name, nameLocation, typePack}); } + else if (FFlag::LuauParserErrorsOnMissingDefaultTypePackArgument) + { + auto [type, typePack] = parseTypeOrPackAnnotation(); + + if (type) + report(type->location, "Expected type pack after '=', got type"); + + namePacks.push_back({name, nameLocation, typePack}); + } } else { diff --git a/CodeGen/include/Luau/AddressA64.h b/CodeGen/include/Luau/AddressA64.h index 351e6715..53efd3c3 100644 --- a/CodeGen/include/Luau/AddressA64.h +++ b/CodeGen/include/Luau/AddressA64.h @@ -17,7 +17,6 @@ enum class AddressKindA64 : uint8_t // reg + reg << shift // reg + sext(reg) << shift // reg + uext(reg) << shift - // pc + offset }; struct AddressA64 @@ -28,8 +27,8 @@ struct AddressA64 , offset(xzr) , data(off) { - LUAU_ASSERT(base.kind == KindA64::x); - LUAU_ASSERT(off >= 0 && off < 4096); + LUAU_ASSERT(base.kind == KindA64::x || base == sp); + LUAU_ASSERT(off >= -256 && off < 4096); } AddressA64(RegisterA64 base, RegisterA64 offset) @@ -48,5 +47,7 @@ struct AddressA64 int data; }; +using mem = AddressA64; + } // namespace CodeGen } // namespace Luau diff --git a/CodeGen/include/Luau/AssemblyBuilderA64.h b/CodeGen/include/Luau/AssemblyBuilderA64.h index 9a1402be..9e12168a 100644 --- a/CodeGen/include/Luau/AssemblyBuilderA64.h +++ b/CodeGen/include/Luau/AssemblyBuilderA64.h @@ -34,15 +34,18 @@ public: // Comparisons // Note: some arithmetic instructions also have versions that update flags (ADDS etc) but we aren't using them atm - // TODO: add cmp + void cmp(RegisterA64 src1, RegisterA64 src2); + void cmp(RegisterA64 src1, int src2); - // Binary + // Bitwise // Note: shifted-register support and bitfield operations are omitted for simplicity // TODO: support immediate arguments (they have odd encoding and forbid many values) - // TODO: support not variants for and/or/eor (required to support not...) void and_(RegisterA64 dst, RegisterA64 src1, RegisterA64 src2); void orr(RegisterA64 dst, RegisterA64 src1, RegisterA64 src2); void eor(RegisterA64 dst, RegisterA64 src1, RegisterA64 src2); + void mvn(RegisterA64 dst, RegisterA64 src); + + // Shifts void lsl(RegisterA64 dst, RegisterA64 src1, RegisterA64 src2); void lsr(RegisterA64 dst, RegisterA64 src1, RegisterA64 src2); void asr(RegisterA64 dst, RegisterA64 src1, RegisterA64 src2); @@ -66,11 +69,19 @@ public: // Control flow // Note: tbz/tbnz are currently not supported because they have 15-bit offsets and we don't support branch thunks + void b(Label& label); void b(ConditionA64 cond, Label& label); void cbz(RegisterA64 src, Label& label); void cbnz(RegisterA64 src, Label& label); + void br(RegisterA64 src); + void blr(RegisterA64 src); void ret(); + // Address of embedded data + void adr(RegisterA64 dst, const void* ptr, size_t size); + void adr(RegisterA64 dst, uint64_t value); + void adr(RegisterA64 dst, double value); + // Run final checks bool finalize(); @@ -97,17 +108,21 @@ private: // Instruction archetypes void place0(const char* name, uint32_t word); void placeSR3(const char* name, RegisterA64 dst, RegisterA64 src1, RegisterA64 src2, uint8_t op, int shift = 0); - void placeSR2(const char* name, RegisterA64 dst, RegisterA64 src, uint8_t op); + void placeSR2(const char* name, RegisterA64 dst, RegisterA64 src, uint8_t op, uint8_t op2 = 0); void placeR3(const char* name, RegisterA64 dst, RegisterA64 src1, RegisterA64 src2, uint8_t op, uint8_t op2); void placeR1(const char* name, RegisterA64 dst, RegisterA64 src, uint32_t op); void placeI12(const char* name, RegisterA64 dst, RegisterA64 src1, int src2, uint8_t op); void placeI16(const char* name, RegisterA64 dst, int src, uint8_t op, int shift = 0); void placeA(const char* name, RegisterA64 dst, AddressA64 src, uint8_t op, uint8_t size); void placeBC(const char* name, Label& label, uint8_t op, uint8_t cond); - void placeBR(const char* name, Label& label, uint8_t op, RegisterA64 cond); + void placeBCR(const char* name, Label& label, uint8_t op, RegisterA64 cond); + void placeBR(const char* name, RegisterA64 src, uint32_t op); + void placeADR(const char* name, RegisterA64 src, uint8_t op); void place(uint32_t word); - void placeLabel(Label& label); + + void patchLabel(Label& label); + void patchImm19(uint32_t location, int value); void commit(); LUAU_NOINLINE void extend(); @@ -123,6 +138,7 @@ private: LUAU_NOINLINE void log(const char* opcode, RegisterA64 dst, int src, int shift = 0); LUAU_NOINLINE void log(const char* opcode, RegisterA64 dst, AddressA64 src); LUAU_NOINLINE void log(const char* opcode, RegisterA64 src, Label label); + LUAU_NOINLINE void log(const char* opcode, RegisterA64 src); LUAU_NOINLINE void log(const char* opcode, Label label); LUAU_NOINLINE void log(Label label); LUAU_NOINLINE void log(RegisterA64 reg); @@ -133,6 +149,7 @@ private: std::vector labelLocations; bool finalized = false; + bool overflowed = false; size_t dataPos = 0; diff --git a/CodeGen/src/AssemblyBuilderA64.cpp b/CodeGen/src/AssemblyBuilderA64.cpp index e4237f53..286800d6 100644 --- a/CodeGen/src/AssemblyBuilderA64.cpp +++ b/CodeGen/src/AssemblyBuilderA64.cpp @@ -37,7 +37,10 @@ AssemblyBuilderA64::~AssemblyBuilderA64() void AssemblyBuilderA64::mov(RegisterA64 dst, RegisterA64 src) { - placeSR2("mov", dst, src, 0b01'01010); + if (dst == sp || src == sp) + placeR1("mov", dst, src, 0b00'100010'0'000000000000); + else + placeSR2("mov", dst, src, 0b01'01010); } void AssemblyBuilderA64::mov(RegisterA64 dst, uint16_t src, int shift) @@ -75,6 +78,20 @@ void AssemblyBuilderA64::neg(RegisterA64 dst, RegisterA64 src) placeSR2("neg", dst, src, 0b10'01011); } +void AssemblyBuilderA64::cmp(RegisterA64 src1, RegisterA64 src2) +{ + RegisterA64 dst = src1.kind == KindA64::x ? xzr : wzr; + + placeSR3("cmp", dst, src1, src2, 0b11'01011); +} + +void AssemblyBuilderA64::cmp(RegisterA64 src1, int src2) +{ + RegisterA64 dst = src1.kind == KindA64::x ? xzr : wzr; + + placeI12("cmp", dst, src1, src2, 0b11'10001); +} + void AssemblyBuilderA64::and_(RegisterA64 dst, RegisterA64 src1, RegisterA64 src2) { placeSR3("and", dst, src1, src2, 0b00'01010); @@ -90,6 +107,11 @@ void AssemblyBuilderA64::eor(RegisterA64 dst, RegisterA64 src1, RegisterA64 src2 placeSR3("eor", dst, src1, src2, 0b10'01010); } +void AssemblyBuilderA64::mvn(RegisterA64 dst, RegisterA64 src) +{ + placeSR2("mvn", dst, src, 0b01'01010, 0b1); +} + void AssemblyBuilderA64::lsl(RegisterA64 dst, RegisterA64 src1, RegisterA64 src2) { placeR3("lsl", dst, src1, src2, 0b11010110, 0b0010'00); @@ -183,6 +205,12 @@ void AssemblyBuilderA64::strh(RegisterA64 src, AddressA64 dst) placeA("strh", src, dst, 0b11100000, 0b01); } +void AssemblyBuilderA64::b(Label& label) +{ + // Note: we aren't using 'b' form since it has a 26-bit immediate which requires custom fixup logic + placeBC("b", label, 0b0101010'0, codeForCondition[int(ConditionA64::Always)]); +} + void AssemblyBuilderA64::b(ConditionA64 cond, Label& label) { placeBC(textForCondition[int(cond)], label, 0b0101010'0, codeForCondition[int(cond)]); @@ -190,12 +218,22 @@ void AssemblyBuilderA64::b(ConditionA64 cond, Label& label) void AssemblyBuilderA64::cbz(RegisterA64 src, Label& label) { - placeBR("cbz", label, 0b011010'0, src); + placeBCR("cbz", label, 0b011010'0, src); } void AssemblyBuilderA64::cbnz(RegisterA64 src, Label& label) { - placeBR("cbnz", label, 0b011010'1, src); + placeBCR("cbnz", label, 0b011010'1, src); +} + +void AssemblyBuilderA64::br(RegisterA64 src) +{ + placeBR("br", src, 0b1101011'0'0'00'11111'0000'0'0); +} + +void AssemblyBuilderA64::blr(RegisterA64 src) +{ + placeBR("blr", src, 0b1101011'0'0'01'11111'0000'0'0); } void AssemblyBuilderA64::ret() @@ -203,10 +241,41 @@ void AssemblyBuilderA64::ret() place0("ret", 0b1101011'0'0'10'11111'0000'0'0'11110'00000); } +void AssemblyBuilderA64::adr(RegisterA64 dst, const void* ptr, size_t size) +{ + size_t pos = allocateData(size, 4); + uint32_t location = getCodeSize(); + + memcpy(&data[pos], ptr, size); + placeADR("adr", dst, 0b10000); + + patchImm19(location, -int(location) - int((data.size() - pos) / 4)); +} + +void AssemblyBuilderA64::adr(RegisterA64 dst, uint64_t value) +{ + size_t pos = allocateData(8, 8); + uint32_t location = getCodeSize(); + + writeu64(&data[pos], value); + placeADR("adr", dst, 0b10000); + + patchImm19(location, -int(location) - int((data.size() - pos) / 4)); +} + +void AssemblyBuilderA64::adr(RegisterA64 dst, double value) +{ + size_t pos = allocateData(8, 8); + uint32_t location = getCodeSize(); + + writef64(&data[pos], value); + placeADR("adr", dst, 0b10000); + + patchImm19(location, -int(location) - int((data.size() - pos) / 4)); +} + bool AssemblyBuilderA64::finalize() { - bool success = true; - code.resize(codePos - code.data()); // Resolve jump targets @@ -214,15 +283,9 @@ bool AssemblyBuilderA64::finalize() { // If this assertion fires, a label was used in jmp without calling setLabel LUAU_ASSERT(labelLocations[fixup.id - 1] != ~0u); - int value = int(labelLocations[fixup.id - 1]) - int(fixup.location); - // imm19 encoding word offset, at bit offset 5 - // note that 18 bits of word offsets = 20 bits of byte offsets = +-1MB - if (value > -(1 << 18) && value < (1 << 18)) - code[fixup.location] |= (value & ((1 << 19) - 1)) << 5; - else - success = false; // overflow + patchImm19(fixup.location, value); } size_t dataSize = data.size() - dataPos; @@ -235,7 +298,7 @@ bool AssemblyBuilderA64::finalize() finalized = true; - return success; + return !overflowed; } Label AssemblyBuilderA64::setLabel() @@ -303,7 +366,7 @@ void AssemblyBuilderA64::placeSR3(const char* name, RegisterA64 dst, RegisterA64 commit(); } -void AssemblyBuilderA64::placeSR2(const char* name, RegisterA64 dst, RegisterA64 src, uint8_t op) +void AssemblyBuilderA64::placeSR2(const char* name, RegisterA64 dst, RegisterA64 src, uint8_t op, uint8_t op2) { if (logText) log(name, dst, src); @@ -313,7 +376,7 @@ void AssemblyBuilderA64::placeSR2(const char* name, RegisterA64 dst, RegisterA64 uint32_t sf = (dst.kind == KindA64::x) ? 0x80000000 : 0; - place(dst.index | (0x1f << 5) | (src.index << 16) | (op << 24) | sf); + place(dst.index | (0x1f << 5) | (src.index << 16) | (op2 << 21) | (op << 24) | sf); commit(); } @@ -336,10 +399,10 @@ void AssemblyBuilderA64::placeR1(const char* name, RegisterA64 dst, RegisterA64 if (logText) log(name, dst, src); - LUAU_ASSERT(dst.kind == KindA64::w || dst.kind == KindA64::x); - LUAU_ASSERT(dst.kind == src.kind); + LUAU_ASSERT(dst.kind == KindA64::w || dst.kind == KindA64::x || dst == sp); + LUAU_ASSERT(dst.kind == src.kind || (dst.kind == KindA64::x && src == sp) || (dst == sp && src.kind == KindA64::x)); - uint32_t sf = (dst.kind == KindA64::x) ? 0x80000000 : 0; + uint32_t sf = (dst.kind != KindA64::w) ? 0x80000000 : 0; place(dst.index | (src.index << 5) | (op << 10) | sf); commit(); @@ -350,11 +413,11 @@ void AssemblyBuilderA64::placeI12(const char* name, RegisterA64 dst, RegisterA64 if (logText) log(name, dst, src1, src2); - LUAU_ASSERT(dst.kind == KindA64::w || dst.kind == KindA64::x); - LUAU_ASSERT(dst.kind == src1.kind); + LUAU_ASSERT(dst.kind == KindA64::w || dst.kind == KindA64::x || dst == sp); + LUAU_ASSERT(dst.kind == src1.kind || (dst.kind == KindA64::x && src1 == sp) || (dst == sp && src1.kind == KindA64::x)); LUAU_ASSERT(src2 >= 0 && src2 < (1 << 12)); - uint32_t sf = (dst.kind == KindA64::x) ? 0x80000000 : 0; + uint32_t sf = (dst.kind != KindA64::w) ? 0x80000000 : 0; place(dst.index | (src1.index << 5) | (src2 << 10) | (op << 24) | sf); commit(); @@ -383,8 +446,12 @@ void AssemblyBuilderA64::placeA(const char* name, RegisterA64 dst, AddressA64 sr switch (src.kind) { case AddressKindA64::imm: - LUAU_ASSERT(src.data % (1 << size) == 0); - place(dst.index | (src.base.index << 5) | ((src.data >> size) << 10) | (op << 22) | (1 << 24) | (size << 30)); + if (src.data >= 0 && src.data % (1 << size) == 0) + place(dst.index | (src.base.index << 5) | ((src.data >> size) << 10) | (op << 22) | (1 << 24) | (size << 30)); + else if (src.data >= -256 && src.data <= 255) + place(dst.index | (src.base.index << 5) | ((src.data & ((1 << 9) - 1)) << 12) | (op << 22) | (size << 30)); + else + LUAU_ASSERT(!"Unable to encode large immediate offset"); break; case AddressKindA64::reg: place(dst.index | (src.base.index << 5) | (0b10 << 10) | (0b011 << 13) | (src.offset.index << 16) | (1 << 21) | (op << 22) | (size << 30)); @@ -396,28 +463,50 @@ void AssemblyBuilderA64::placeA(const char* name, RegisterA64 dst, AddressA64 sr void AssemblyBuilderA64::placeBC(const char* name, Label& label, uint8_t op, uint8_t cond) { - placeLabel(label); + place(cond | (op << 24)); + commit(); + + patchLabel(label); if (logText) log(name, label); - - place(cond | (op << 24)); - commit(); } -void AssemblyBuilderA64::placeBR(const char* name, Label& label, uint8_t op, RegisterA64 cond) +void AssemblyBuilderA64::placeBCR(const char* name, Label& label, uint8_t op, RegisterA64 cond) { - placeLabel(label); - - if (logText) - log(name, cond, label); - LUAU_ASSERT(cond.kind == KindA64::w || cond.kind == KindA64::x); uint32_t sf = (cond.kind == KindA64::x) ? 0x80000000 : 0; place(cond.index | (op << 24) | sf); commit(); + + patchLabel(label); + + if (logText) + log(name, cond, label); +} + +void AssemblyBuilderA64::placeBR(const char* name, RegisterA64 src, uint32_t op) +{ + if (logText) + log(name, src); + + LUAU_ASSERT(src.kind == KindA64::x); + + place((src.index << 5) | (op << 10)); + commit(); +} + +void AssemblyBuilderA64::placeADR(const char* name, RegisterA64 dst, uint8_t op) +{ + if (logText) + log(name, dst); + + LUAU_ASSERT(dst.kind == KindA64::x); + + place(dst.index | (op << 24)); + commit(); } void AssemblyBuilderA64::place(uint32_t word) @@ -426,8 +515,10 @@ void AssemblyBuilderA64::place(uint32_t word) *codePos++ = word; } -void AssemblyBuilderA64::placeLabel(Label& label) +void AssemblyBuilderA64::patchLabel(Label& label) { + uint32_t location = getCodeSize() - 1; + if (label.location == ~0u) { if (label.id == 0) @@ -436,18 +527,26 @@ void AssemblyBuilderA64::placeLabel(Label& label) labelLocations.push_back(~0u); } - pendingLabels.push_back({label.id, getCodeSize()}); + pendingLabels.push_back({label.id, location}); } else { - // note: if label has an assigned location we can in theory avoid patching it later, but - // we need to handle potential overflow of 19-bit offsets - LUAU_ASSERT(label.id != 0); - labelLocations[label.id - 1] = label.location; - pendingLabels.push_back({label.id, getCodeSize()}); + int value = int(label.location) - int(location); + + patchImm19(location, value); } } +void AssemblyBuilderA64::patchImm19(uint32_t location, int value) +{ + // imm19 encoding word offset, at bit offset 5 + // note that 18 bits of word offsets = 20 bits of byte offsets = +-1MB + if (value > -(1 << 18) && value < (1 << 18)) + code[location] |= (value & ((1 << 19) - 1)) << 5; + else + overflowed = true; +} + void AssemblyBuilderA64::commit() { LUAU_ASSERT(codePos <= codeEnd); @@ -491,8 +590,11 @@ void AssemblyBuilderA64::log(const char* opcode) void AssemblyBuilderA64::log(const char* opcode, RegisterA64 dst, RegisterA64 src1, RegisterA64 src2, int shift) { logAppend(" %-12s", opcode); - log(dst); - text.append(","); + if (dst != xzr && dst != wzr) + { + log(dst); + text.append(","); + } log(src1); text.append(","); log(src2); @@ -504,8 +606,11 @@ void AssemblyBuilderA64::log(const char* opcode, RegisterA64 dst, RegisterA64 sr void AssemblyBuilderA64::log(const char* opcode, RegisterA64 dst, RegisterA64 src1, int src2) { logAppend(" %-12s", opcode); - log(dst); - text.append(","); + if (dst != xzr && dst != wzr) + { + log(dst); + text.append(","); + } log(src1); text.append(","); logAppend("#%d", src2); @@ -549,6 +654,13 @@ void AssemblyBuilderA64::log(const char* opcode, RegisterA64 src, Label label) logAppend(".L%d\n", label.id); } +void AssemblyBuilderA64::log(const char* opcode, RegisterA64 src) +{ + logAppend(" %-12s", opcode); + log(src); + text.append("\n"); +} + void AssemblyBuilderA64::log(const char* opcode, Label label) { logAppend(" %-12s.L%d\n", opcode, label.id); @@ -565,20 +677,24 @@ void AssemblyBuilderA64::log(RegisterA64 reg) { case KindA64::w: if (reg.index == 31) - logAppend("wzr"); + text.append("wzr"); else logAppend("w%d", reg.index); break; case KindA64::x: if (reg.index == 31) - logAppend("xzr"); + text.append("xzr"); else logAppend("x%d", reg.index); break; case KindA64::none: - LUAU_ASSERT(!"Unexpected register kind"); + if (reg.index == 31) + text.append("sp"); + else + LUAU_ASSERT(!"Unexpected register kind"); + break; } } diff --git a/CodeGen/src/CodeGenX64.cpp b/CodeGen/src/CodeGenX64.cpp index 3074cce2..b23d2b38 100644 --- a/CodeGen/src/CodeGenX64.cpp +++ b/CodeGen/src/CodeGenX64.cpp @@ -14,15 +14,15 @@ * Each line is 8 bytes, stack grows downwards. * * | ... previous frames ... - * | rdx home space | (saved only on windows) - * | rcx home space | (saved only on windows) + * | rdx home space | (unused) + * | rcx home space | (unused) * | return address | - * | ... saved non-volatile registers ... + * | ... saved non-volatile registers ... <-- rsp + kStackSize + kLocalsSize * | unused | for 16 byte alignment of the stack * | sCode | - * | sClosure | <-- rbp points here - * | argument 6 | - * | argument 5 | + * | sClosure | <-- rsp + kStackSize + * | argument 6 | <-- rsp + 40 + * | argument 5 | <-- rsp + 32 * | r9 home space | * | r8 home space | * | rdx home space | @@ -48,28 +48,18 @@ bool initEntryFunction(NativeState& data) unwind.start(); - if (build.abi == ABIX64::Windows) - { - // Place arguments in home space - build.mov(qword[rsp + 16], rArg2); - unwind.spill(16, rArg2); - build.mov(qword[rsp + 8], rArg1); - unwind.spill(8, rArg1); - - // Save non-volatile registers that are specific to Windows x64 ABI - build.push(rdi); - unwind.save(rdi); - build.push(rsi); - unwind.save(rsi); - - // Once we start using non-volatile SIMD registers, we will save those here - } - // Save common non-volatile registers - build.push(rbx); - unwind.save(rbx); build.push(rbp); unwind.save(rbp); + + if (build.abi == ABIX64::SystemV) + { + build.mov(rbp, rsp); + unwind.setupFrameReg(rbp, 0); + } + + build.push(rbx); + unwind.save(rbx); build.push(r12); unwind.save(r12); build.push(r13); @@ -79,16 +69,20 @@ bool initEntryFunction(NativeState& data) build.push(r15); unwind.save(r15); - int stacksize = 32 + 16; // 4 home locations for registers, 16 bytes for additional function call arguments - int localssize = 24; // 3 local pointers that also correctly align the stack + if (build.abi == ABIX64::Windows) + { + // Save non-volatile registers that are specific to Windows x64 ABI + build.push(rdi); + unwind.save(rdi); + build.push(rsi); + unwind.save(rsi); + + // TODO: once we start using non-volatile SIMD registers on Windows, we will save those here + } // Allocate stack space (reg home area + local data) - build.sub(rsp, stacksize + localssize); - unwind.allocStack(stacksize + localssize); - - // Setup frame pointer - build.lea(rbp, addr[rsp + stacksize]); - unwind.setupFrameReg(rbp, stacksize); + build.sub(rsp, kStackSize + kLocalsSize); + unwind.allocStack(kStackSize + kLocalsSize); unwind.finish(); @@ -113,13 +107,7 @@ bool initEntryFunction(NativeState& data) Label returnOff = build.setLabel(); // Cleanup and exit - build.lea(rsp, addr[rbp + localssize]); - build.pop(r15); - build.pop(r14); - build.pop(r13); - build.pop(r12); - build.pop(rbp); - build.pop(rbx); + build.add(rsp, kStackSize + kLocalsSize); if (build.abi == ABIX64::Windows) { @@ -127,6 +115,12 @@ bool initEntryFunction(NativeState& data) build.pop(rdi); } + build.pop(r15); + build.pop(r14); + build.pop(r13); + build.pop(r12); + build.pop(rbx); + build.pop(rbp); build.ret(); build.finalize(); diff --git a/CodeGen/src/EmitCommonX64.h b/CodeGen/src/EmitCommonX64.h index 071ef6af..61544855 100644 --- a/CodeGen/src/EmitCommonX64.h +++ b/CodeGen/src/EmitCommonX64.h @@ -32,8 +32,12 @@ constexpr RegisterX64 rNativeContext = r13; // NativeContext* context constexpr RegisterX64 rConstants = r12; // TValue* k // Native code is as stackless as the interpreter, so we can place some data on the stack once and have it accessible at any point -constexpr OperandX64 sClosure = qword[rbp + 0]; // Closure* cl -constexpr OperandX64 sCode = qword[rbp + 8]; // Instruction* code +// See CodeGenX64.cpp for layout +constexpr unsigned kStackSize = 32 + 16; // 4 home locations for registers, 16 bytes for additional function call arguments +constexpr unsigned kLocalsSize = 24; // 3 extra slots for our custom locals (also aligns the stack to 16 byte boundary) + +constexpr OperandX64 sClosure = qword[rsp + kStackSize + 0]; // Closure* cl +constexpr OperandX64 sCode = qword[rsp + kStackSize + 8]; // Instruction* code // TODO: These should be replaced with a portable call function that checks the ABI at runtime and reorders moves accordingly to avoid conflicts #if defined(_WIN32) diff --git a/CodeGen/src/UnwindBuilderDwarf2.cpp b/CodeGen/src/UnwindBuilderDwarf2.cpp index 8d06864e..7dc86d3e 100644 --- a/CodeGen/src/UnwindBuilderDwarf2.cpp +++ b/CodeGen/src/UnwindBuilderDwarf2.cpp @@ -13,11 +13,10 @@ // https://refspecs.linuxbase.org/elf/x86_64-abi-0.99.pdf [System V Application Binary Interface (AMD64 Architecture Processor Supplement)] // Interaction between Dwarf2 and System V ABI can be found in sections '3.6.2 DWARF Register Number Mapping' and '4.2.4 EH_FRAME sections' -// Call frame instruction opcodes +// Call frame instruction opcodes (Dwarf2, page 78, ch. 7.23 figure 37) #define DW_CFA_advance_loc 0x40 #define DW_CFA_offset 0x80 #define DW_CFA_restore 0xc0 -#define DW_CFA_nop 0x00 #define DW_CFA_set_loc 0x01 #define DW_CFA_advance_loc1 0x02 #define DW_CFA_advance_loc2 0x03 @@ -33,17 +32,11 @@ #define DW_CFA_def_cfa_register 0x0d #define DW_CFA_def_cfa_offset 0x0e #define DW_CFA_def_cfa_expression 0x0f -#define DW_CFA_expression 0x10 -#define DW_CFA_offset_extended_sf 0x11 -#define DW_CFA_def_cfa_sf 0x12 -#define DW_CFA_def_cfa_offset_sf 0x13 -#define DW_CFA_val_offset 0x14 -#define DW_CFA_val_offset_sf 0x15 -#define DW_CFA_val_expression 0x16 +#define DW_CFA_nop 0x00 #define DW_CFA_lo_user 0x1c #define DW_CFA_hi_user 0x3f -// Register numbers for x64 +// Register numbers for x64 (System V ABI, page 57, ch. 3.7, figure 3.36) #define DW_REG_RAX 0 #define DW_REG_RDX 1 #define DW_REG_RCX 2 @@ -197,7 +190,12 @@ void UnwindBuilderDwarf2::allocStack(int size) void UnwindBuilderDwarf2::setupFrameReg(RegisterX64 reg, int espOffset) { - // Not required for unwinding + if (espOffset != 0) + pos = advanceLocation(pos, 5); // REX.W lea rbp, [rsp + imm8] + else + pos = advanceLocation(pos, 3); // REX.W mov rbp, rsp + + // Cfa is based on rsp, so no additonal commands are required } void UnwindBuilderDwarf2::finish() diff --git a/CodeGen/src/UnwindBuilderWin.cpp b/CodeGen/src/UnwindBuilderWin.cpp index 1b3279e8..13e92ab0 100644 --- a/CodeGen/src/UnwindBuilderWin.cpp +++ b/CodeGen/src/UnwindBuilderWin.cpp @@ -77,7 +77,11 @@ void UnwindBuilderWin::setupFrameReg(RegisterX64 reg, int espOffset) frameReg = reg; frameRegOffset = uint8_t(espOffset / 16); - prologSize += 5; // REX.W lea rbp, [rsp + imm8] + if (espOffset != 0) + prologSize += 5; // REX.W lea rbp, [rsp + imm8] + else + prologSize += 3; // REX.W mov rbp, rsp + unwindCodes.push_back({prologSize, UWOP_SET_FPREG, frameRegOffset}); } diff --git a/Common/include/Luau/ExperimentalFlags.h b/Common/include/Luau/ExperimentalFlags.h index 17fe26b1..15db9ea3 100644 --- a/Common/include/Luau/ExperimentalFlags.h +++ b/Common/include/Luau/ExperimentalFlags.h @@ -13,6 +13,8 @@ inline bool isFlagExperimental(const char* flag) static const char* kList[] = { "LuauInterpolatedStringBaseSupport", "LuauInstantiateInSubtyping", // requires some fixes to lua-apps code + "LuauOptionalNextKey", // waiting for a fix to land in lua-apps + "LuauTryhardAnd", // waiting for a fix in graphql-lua -> apollo-client-lia -> lua-apps // makes sure we always have at least one entry nullptr, }; diff --git a/tests/AssemblyBuilderA64.test.cpp b/tests/AssemblyBuilderA64.test.cpp index d1f29c23..d808ac49 100644 --- a/tests/AssemblyBuilderA64.test.cpp +++ b/tests/AssemblyBuilderA64.test.cpp @@ -69,6 +69,7 @@ TEST_CASE_FIXTURE(AssemblyBuilderA64Fixture, "Unary") { SINGLE_COMPARE(neg(x0, x1), 0xCB0103E0); SINGLE_COMPARE(neg(w0, w1), 0x4B0103E0); + SINGLE_COMPARE(mvn(x0, x1), 0xAA2103E0); SINGLE_COMPARE(clz(x0, x1), 0xDAC01020); SINGLE_COMPARE(clz(w0, w1), 0x5AC01020); @@ -91,19 +92,22 @@ TEST_CASE_FIXTURE(AssemblyBuilderA64Fixture, "Binary") SINGLE_COMPARE(lsr(x0, x1, x2), 0x9AC22420); SINGLE_COMPARE(asr(x0, x1, x2), 0x9AC22820); SINGLE_COMPARE(ror(x0, x1, x2), 0x9AC22C20); + SINGLE_COMPARE(cmp(x0, x1), 0xEB01001F); // reg, imm SINGLE_COMPARE(add(x3, x7, 78), 0x910138E3); SINGLE_COMPARE(add(w3, w7, 78), 0x110138E3); SINGLE_COMPARE(sub(w3, w7, 78), 0x510138E3); + SINGLE_COMPARE(cmp(w0, 42), 0x7100A81F); } TEST_CASE_FIXTURE(AssemblyBuilderA64Fixture, "Loads") { // address forms SINGLE_COMPARE(ldr(x0, x1), 0xF9400020); - SINGLE_COMPARE(ldr(x0, AddressA64(x1, 8)), 0xF9400420); - SINGLE_COMPARE(ldr(x0, AddressA64(x1, x7)), 0xF8676820); + SINGLE_COMPARE(ldr(x0, mem(x1, 8)), 0xF9400420); + SINGLE_COMPARE(ldr(x0, mem(x1, x7)), 0xF8676820); + SINGLE_COMPARE(ldr(x0, mem(x1, -7)), 0xF85F9020); // load sizes SINGLE_COMPARE(ldr(x0, x1), 0xF9400020); @@ -121,8 +125,9 @@ TEST_CASE_FIXTURE(AssemblyBuilderA64Fixture, "Stores") { // address forms SINGLE_COMPARE(str(x0, x1), 0xF9000020); - SINGLE_COMPARE(str(x0, AddressA64(x1, 8)), 0xF9000420); - SINGLE_COMPARE(str(x0, AddressA64(x1, x7)), 0xF8276820); + SINGLE_COMPARE(str(x0, mem(x1, 8)), 0xF9000420); + SINGLE_COMPARE(str(x0, mem(x1, x7)), 0xF8276820); + SINGLE_COMPARE(strh(w0, mem(x1, -7)), 0x781F9020); // store sizes SINGLE_COMPARE(str(x0, x1), 0xF9000020); @@ -169,26 +174,69 @@ TEST_CASE_FIXTURE(AssemblyBuilderA64Fixture, "ControlFlow") build.cbz(x0, skip); build.cbnz(x0, skip); build.setLabel(skip); + build.b(skip); }, - {0x54000060, 0xB4000040, 0xB5000020})); + {0x54000060, 0xB4000040, 0xB5000020, 0x5400000E})); // Basic control flow + SINGLE_COMPARE(br(x0), 0xD61F0000); + SINGLE_COMPARE(blr(x0), 0xD63F0000); SINGLE_COMPARE(ret(), 0xD65F03C0); } +TEST_CASE_FIXTURE(AssemblyBuilderA64Fixture, "StackOps") +{ + SINGLE_COMPARE(mov(x0, sp), 0x910003E0); + SINGLE_COMPARE(mov(sp, x0), 0x9100001F); + + SINGLE_COMPARE(add(sp, sp, 4), 0x910013FF); + SINGLE_COMPARE(sub(sp, sp, 4), 0xD10013FF); + + SINGLE_COMPARE(add(x0, sp, 4), 0x910013E0); + SINGLE_COMPARE(sub(sp, x0, 4), 0xD100101F); + + SINGLE_COMPARE(ldr(x0, mem(sp, 8)), 0xF94007E0); + SINGLE_COMPARE(str(x0, mem(sp, 8)), 0xF90007E0); +} + +TEST_CASE_FIXTURE(AssemblyBuilderA64Fixture, "Constants") +{ + // clang-format off + CHECK(check( + [](AssemblyBuilderA64& build) { + char arr[12] = "hello world"; + build.adr(x0, arr, 12); + build.adr(x0, uint64_t(0x1234567887654321)); + build.adr(x0, 1.0); + }, + { + 0x10ffffa0, 0x10ffff20, 0x10fffec0 + }, + { + 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0xf0, 0x3f, + 0x21, 0x43, 0x65, 0x87, 0x78, 0x56, 0x34, 0x12, + 0x00, 0x00, 0x00, 0x00, // 4b padding to align double + 'h', 'e', 'l', 'l', 'o', ' ', 'w', 'o', 'r', 'l', 'd', 0x0, + })); + // clang-format on +} + TEST_CASE("LogTest") { AssemblyBuilderA64 build(/* logText= */ true); + build.add(sp, sp, 4); build.add(w0, w1, w2); build.add(x0, x1, x2, 2); build.add(w7, w8, 5); build.add(x7, x8, 5); build.ldr(x7, x8); - build.ldr(x7, AddressA64(x8, 8)); - build.ldr(x7, AddressA64(x8, x9)); + build.ldr(x7, mem(x8, 8)); + build.ldr(x7, mem(x8, x9)); build.mov(x1, x2); build.movk(x1, 42, 16); + build.cmp(x1, x2); + build.blr(x0); Label l; build.b(ConditionA64::Plus, l); @@ -200,6 +248,7 @@ TEST_CASE("LogTest") build.finalize(); std::string expected = R"( + add sp,sp,#4 add w0,w1,w2 add x0,x1,x2 LSL #2 add w7,w8,#5 @@ -209,6 +258,8 @@ TEST_CASE("LogTest") ldr x7,[x8,x9] mov x1,x2 movk x1,#42 LSL #16 + cmp x1,x2 + blr x0 b.pl .L1 cbz x7,.L1 .L1: diff --git a/tests/CodeAllocator.test.cpp b/tests/CodeAllocator.test.cpp index 2e9a4b37..65b485a7 100644 --- a/tests/CodeAllocator.test.cpp +++ b/tests/CodeAllocator.test.cpp @@ -185,7 +185,7 @@ TEST_CASE("Dwarf2UnwindCodesX64") 0x00, 0x00, 0x00, 0x00, 0x00, 0x4c, 0x00, 0x00, 0x00, 0x1c, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x02, 0x02, 0x0e, 0x10, 0x85, 0x02, 0x02, 0x02, 0x0e, 0x18, 0x84, 0x03, 0x02, 0x02, 0x0e, 0x20, 0x83, 0x04, 0x02, 0x02, 0x0e, 0x28, 0x86, 0x05, 0x02, 0x02, 0x0e, 0x30, 0x8c, 0x06, 0x02, 0x02, 0x0e, 0x38, 0x8d, 0x07, 0x02, 0x02, 0x0e, 0x40, - 0x8e, 0x08, 0x02, 0x02, 0x0e, 0x48, 0x8f, 0x09, 0x02, 0x04, 0x0e, 0x90, 0x01, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00}; + 0x8e, 0x08, 0x02, 0x02, 0x0e, 0x48, 0x8f, 0x09, 0x02, 0x04, 0x0e, 0x90, 0x01, 0x02, 0x05, 0x00, 0x00, 0x00, 0x00, 0x00}; REQUIRE(data.size() == expected.size()); CHECK(memcmp(data.data(), expected.data(), expected.size()) == 0); @@ -446,7 +446,12 @@ TEST_CASE("GeneratedCodeExecutionA64") build.mov(x1, 0); // doesn't execute due to cbnz above build.setLabel(skip); - build.add(x1, x1, 1); + uint8_t one = 1; + build.adr(x2, &one, 1); + build.ldrb(w2, x2); + build.sub(x1, x1, x2); + + build.add(x1, x1, 2); build.add(x0, x0, x1, /* LSL */ 1); build.ret(); diff --git a/tests/ConstraintGraphBuilderFixture.cpp b/tests/ConstraintGraphBuilderFixture.cpp index d011719e..30e1b2e6 100644 --- a/tests/ConstraintGraphBuilderFixture.cpp +++ b/tests/ConstraintGraphBuilderFixture.cpp @@ -21,13 +21,13 @@ void ConstraintGraphBuilderFixture::generateConstraints(const std::string& code) frontend.getGlobalScope(), &logger, NotNull{dfg.get()}); cgb->visit(root); rootScope = cgb->rootScope; - constraints = Luau::collectConstraints(NotNull{cgb->rootScope}); + constraints = Luau::borrowConstraints(cgb->constraints); } void ConstraintGraphBuilderFixture::solve(const std::string& code) { generateConstraints(code); - ConstraintSolver cs{NotNull{&normalizer}, NotNull{rootScope}, "MainModule", NotNull(&moduleResolver), {}, &logger}; + ConstraintSolver cs{NotNull{&normalizer}, NotNull{rootScope}, constraints, "MainModule", NotNull(&moduleResolver), {}, &logger}; cs.run(); } diff --git a/tests/Frontend.test.cpp b/tests/Frontend.test.cpp index df0abdc9..4e72dd4e 100644 --- a/tests/Frontend.test.cpp +++ b/tests/Frontend.test.cpp @@ -1054,10 +1054,6 @@ TEST_CASE("check_without_builtin_next") TEST_CASE_FIXTURE(BuiltinsFixture, "reexport_cyclic_type") { - ScopedFastFlag sff[] = { - {"LuauForceExportSurfacesToBeNormal", true}, - }; - fileResolver.source["Module/A"] = R"( type F = (set: G) -> () @@ -1089,10 +1085,6 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "reexport_cyclic_type") TEST_CASE_FIXTURE(BuiltinsFixture, "reexport_type_alias") { - ScopedFastFlag sff[] = { - {"LuauForceExportSurfacesToBeNormal", true}, - }; - fileResolver.source["Module/A"] = R"( type KeyOfTestEvents = "test-file-start" | "test-file-success" | "test-file-failure" | "test-case-result" type MyAny = any diff --git a/tests/Parser.test.cpp b/tests/Parser.test.cpp index 77cf6130..c0989a2e 100644 --- a/tests/Parser.test.cpp +++ b/tests/Parser.test.cpp @@ -2823,4 +2823,21 @@ TEST_CASE_FIXTURE(Fixture, "get_a_nice_error_when_there_is_no_comma_after_last_t CHECK(table->items.size == 1); } +TEST_CASE_FIXTURE(Fixture, "missing_default_type_pack_argument_after_variadic_type_parameter") +{ + ScopedFastFlag sff{"LuauParserErrorsOnMissingDefaultTypePackArgument", true}; + + ParseResult result = tryParse(R"( + type Foo = nil + )"); + + REQUIRE_EQ(2, result.errors.size()); + + CHECK_EQ(Location{{1, 23}, {1, 25}}, result.errors[0].getLocation()); + CHECK_EQ("Expected type, got '>'", result.errors[0].getMessage()); + + CHECK_EQ(Location{{1, 23}, {1, 24}}, result.errors[1].getLocation()); + CHECK_EQ("Expected type pack after '=', got type", result.errors[1].getMessage()); +} + TEST_SUITE_END(); diff --git a/tests/Repl.test.cpp b/tests/Repl.test.cpp index 18c243b0..c22d464e 100644 --- a/tests/Repl.test.cpp +++ b/tests/Repl.test.cpp @@ -404,3 +404,20 @@ t60 = makeChainedTable(60) } TEST_SUITE_END(); + +TEST_SUITE_BEGIN("RegressionTests"); + +TEST_CASE_FIXTURE(ReplFixture, "InfiniteRecursion") +{ + // If the infinite recrusion is not caught, test will fail + runCode(L, R"( +local NewProxyOne = newproxy(true) +local MetaTableOne = getmetatable(NewProxyOne) +MetaTableOne.__index = function() + return NewProxyOne.Game +end +print(NewProxyOne.HelloICauseACrash) +)"); +} + +TEST_SUITE_END(); diff --git a/tests/ToString.test.cpp b/tests/ToString.test.cpp index a510f914..8bb1fbaf 100644 --- a/tests/ToString.test.cpp +++ b/tests/ToString.test.cpp @@ -10,7 +10,6 @@ using namespace Luau; LUAU_FASTFLAG(LuauRecursiveTypeParameterRestriction); -LUAU_FASTFLAG(LuauFixNameMaps); LUAU_FASTFLAG(LuauFunctionReturnStringificationFixup); TEST_SUITE_BEGIN("ToString"); @@ -266,11 +265,23 @@ TEST_CASE_FIXTURE(Fixture, "quit_stringifying_type_when_length_is_exceeded") ToStringOptions o; o.exhaustive = false; - o.maxTypeLength = 40; - CHECK_EQ(toString(requireType("f0"), o), "() -> ()"); - CHECK_EQ(toString(requireType("f1"), o), "(() -> ()) -> () -> ()"); - CHECK_EQ(toString(requireType("f2"), o), "((() -> ()) -> () -> ()) -> (() -> ()) -> ... *TRUNCATED*"); - CHECK_EQ(toString(requireType("f3"), o), "(((() -> ()) -> () -> ()) -> (() -> ()) -> ... *TRUNCATED*"); + + if (FFlag::DebugLuauDeferredConstraintResolution) + { + o.maxTypeLength = 30; + CHECK_EQ(toString(requireType("f0"), o), "() -> ()"); + CHECK_EQ(toString(requireType("f1"), o), "(a) -> () -> ()"); + CHECK_EQ(toString(requireType("f2"), o), "(b) -> (a) -> () -> ()"); + CHECK_EQ(toString(requireType("f3"), o), "(c) -> (b) -> (a) -> (... *TRUNCATED*"); + } + else + { + o.maxTypeLength = 40; + CHECK_EQ(toString(requireType("f0"), o), "() -> ()"); + CHECK_EQ(toString(requireType("f1"), o), "(() -> ()) -> () -> ()"); + CHECK_EQ(toString(requireType("f2"), o), "((() -> ()) -> () -> ()) -> (() -> ()) -> ... *TRUNCATED*"); + CHECK_EQ(toString(requireType("f3"), o), "(((() -> ()) -> () -> ()) -> (() -> ()) -> ... *TRUNCATED*"); + } } TEST_CASE_FIXTURE(Fixture, "stringifying_type_is_still_capped_when_exhaustive") @@ -285,11 +296,22 @@ TEST_CASE_FIXTURE(Fixture, "stringifying_type_is_still_capped_when_exhaustive") ToStringOptions o; o.exhaustive = true; - o.maxTypeLength = 40; - CHECK_EQ(toString(requireType("f0"), o), "() -> ()"); - CHECK_EQ(toString(requireType("f1"), o), "(() -> ()) -> () -> ()"); - CHECK_EQ(toString(requireType("f2"), o), "((() -> ()) -> () -> ()) -> (() -> ()) -> ... *TRUNCATED*"); - CHECK_EQ(toString(requireType("f3"), o), "(((() -> ()) -> () -> ()) -> (() -> ()) -> ... *TRUNCATED*"); + if (FFlag::DebugLuauDeferredConstraintResolution) + { + o.maxTypeLength = 30; + CHECK_EQ(toString(requireType("f0"), o), "() -> ()"); + CHECK_EQ(toString(requireType("f1"), o), "(a) -> () -> ()"); + CHECK_EQ(toString(requireType("f2"), o), "(b) -> (a) -> () -> ()"); + CHECK_EQ(toString(requireType("f3"), o), "(c) -> (b) -> (a) -> (... *TRUNCATED*"); + } + else + { + o.maxTypeLength = 40; + CHECK_EQ(toString(requireType("f0"), o), "() -> ()"); + CHECK_EQ(toString(requireType("f1"), o), "(() -> ()) -> () -> ()"); + CHECK_EQ(toString(requireType("f2"), o), "((() -> ()) -> () -> ()) -> (() -> ()) -> ... *TRUNCATED*"); + CHECK_EQ(toString(requireType("f3"), o), "(((() -> ()) -> () -> ()) -> (() -> ()) -> ... *TRUNCATED*"); + } } TEST_CASE_FIXTURE(Fixture, "stringifying_table_type_correctly_use_matching_table_state_braces") @@ -423,28 +445,19 @@ TEST_CASE_FIXTURE(Fixture, "toStringDetailed") TypeId id3Type = requireType("id3"); ToStringResult nameData = toStringDetailed(id3Type, opts); - if (FFlag::LuauFixNameMaps) - REQUIRE(3 == opts.nameMap.typeVars.size()); - else - REQUIRE_EQ(3, nameData.DEPRECATED_nameMap.typeVars.size()); + REQUIRE(3 == opts.nameMap.typeVars.size()); REQUIRE_EQ("(a, b, c) -> (a, b, c)", nameData.name); - ToStringOptions opts2; // TODO: delete opts2 when clipping FFlag::LuauFixNameMaps - if (FFlag::LuauFixNameMaps) - opts2.nameMap = std::move(opts.nameMap); - else - opts2.DEPRECATED_nameMap = std::move(nameData.DEPRECATED_nameMap); - const FunctionTypeVar* ftv = get(follow(id3Type)); REQUIRE(ftv != nullptr); auto params = flatten(ftv->argTypes).first; REQUIRE(3 == params.size()); - CHECK("a" == toString(params[0], opts2)); - CHECK("b" == toString(params[1], opts2)); - CHECK("c" == toString(params[2], opts2)); + CHECK("a" == toString(params[0], opts)); + CHECK("b" == toString(params[1], opts)); + CHECK("c" == toString(params[2], opts)); } TEST_CASE_FIXTURE(BuiltinsFixture, "toStringDetailed2") @@ -471,13 +484,7 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "toStringDetailed2") TypeId tType = requireType("inst"); ToStringResult r = toStringDetailed(tType, opts); CHECK_EQ("{ @metatable { __index: { @metatable {| __index: base |}, child } }, inst }", r.name); - if (FFlag::LuauFixNameMaps) - CHECK(0 == opts.nameMap.typeVars.size()); - else - CHECK_EQ(0, r.DEPRECATED_nameMap.typeVars.size()); - - if (!FFlag::LuauFixNameMaps) - opts.DEPRECATED_nameMap = r.DEPRECATED_nameMap; + CHECK(0 == opts.nameMap.typeVars.size()); const MetatableTypeVar* tMeta = get(tType); REQUIRE(tMeta); @@ -502,8 +509,6 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "toStringDetailed2") REQUIRE(tMeta6->props.count("two") > 0); ToStringResult oneResult = toStringDetailed(tMeta5->props["one"].type, opts); - if (!FFlag::LuauFixNameMaps) - opts.DEPRECATED_nameMap = oneResult.DEPRECATED_nameMap; std::string twoResult = toString(tMeta6->props["two"].type, opts); diff --git a/tests/TypeInfer.aliases.test.cpp b/tests/TypeInfer.aliases.test.cpp index 8c738b7d..ef40e278 100644 --- a/tests/TypeInfer.aliases.test.cpp +++ b/tests/TypeInfer.aliases.test.cpp @@ -8,6 +8,7 @@ using namespace Luau; LUAU_FASTFLAG(DebugLuauDeferredConstraintResolution) +LUAU_FASTFLAG(LuauNoMoreGlobalSingletonTypes) TEST_SUITE_BEGIN("TypeAliases"); @@ -509,11 +510,21 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "general_require_multi_assign") TEST_CASE_FIXTURE(BuiltinsFixture, "type_alias_import_mutation") { + ScopedFastFlag luauNewLibraryTypeNames{"LuauNewLibraryTypeNames", true}; + CheckResult result = check("type t10 = typeof(table)"); LUAU_REQUIRE_NO_ERRORS(result); TypeId ty = getGlobalBinding(frontend, "table"); - CHECK_EQ(toString(ty), "table"); + + if (FFlag::LuauNoMoreGlobalSingletonTypes) + { + CHECK_EQ(toString(ty), "typeof(table)"); + } + else + { + CHECK_EQ(toString(ty), "table"); + } const TableTypeVar* ttv = get(ty); REQUIRE(ttv); diff --git a/tests/TypeInfer.builtins.test.cpp b/tests/TypeInfer.builtins.test.cpp index 7c465c53..787aea9a 100644 --- a/tests/TypeInfer.builtins.test.cpp +++ b/tests/TypeInfer.builtins.test.cpp @@ -57,7 +57,7 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "next_iterator_should_infer_types_and_type_ch local s = "foo" local t = { [s] = 1 } - local c: string, d: number = next(t) + local c: string?, d: number = next(t) )"); LUAU_REQUIRE_ERROR_COUNT(1, result); @@ -69,7 +69,7 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "pairs_iterator_should_infer_types_and_type_c type Map = { [K]: V } local map: Map = { ["foo"] = 1, ["bar"] = 2, ["baz"] = 3 } - local it: (Map, string | nil) -> (string, number), t: Map, i: nil = pairs(map) + local it: (Map, string | nil) -> (string?, number), t: Map, i: nil = pairs(map) )"); LUAU_REQUIRE_NO_ERRORS(result); @@ -81,7 +81,7 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "ipairs_iterator_should_infer_types_and_type_ type Map = { [K]: V } local array: Map = { "foo", "bar", "baz" } - local it: (Map, number) -> (number, string), t: Map, i: number = ipairs(array) + local it: (Map, number) -> (number?, string), t: Map, i: number = ipairs(array) )"); LUAU_REQUIRE_NO_ERRORS(result); diff --git a/tests/TypeInfer.functions.test.cpp b/tests/TypeInfer.functions.test.cpp index ddf73349..b306515a 100644 --- a/tests/TypeInfer.functions.test.cpp +++ b/tests/TypeInfer.functions.test.cpp @@ -532,7 +532,7 @@ TEST_CASE_FIXTURE(Fixture, "infer_higher_order_function") REQUIRE_EQ(2, argVec.size()); const FunctionTypeVar* fType = get(follow(argVec[0])); - REQUIRE(fType != nullptr); + REQUIRE_MESSAGE(fType != nullptr, "Expected a function but got " << toString(argVec[0])); std::vector fArgs = flatten(fType->argTypes).first; diff --git a/tests/TypeInfer.operators.test.cpp b/tests/TypeInfer.operators.test.cpp index b2516f6d..c4bbc2e1 100644 --- a/tests/TypeInfer.operators.test.cpp +++ b/tests/TypeInfer.operators.test.cpp @@ -50,14 +50,18 @@ TEST_CASE_FIXTURE(Fixture, "or_joins_types_with_no_superfluous_union") CHECK_EQ(*requireType("s"), *typeChecker.stringType); } -TEST_CASE_FIXTURE(Fixture, "and_adds_boolean") +TEST_CASE_FIXTURE(Fixture, "and_does_not_always_add_boolean") { + ScopedFastFlag sff[]{ + {"LuauTryhardAnd", true}, + }; + CheckResult result = check(R"( local s = "a" and 10 local x:boolean|number = s )"); LUAU_REQUIRE_NO_ERRORS(result); - CHECK_EQ(toString(*requireType("s")), "boolean | number"); + CHECK_EQ(toString(*requireType("s")), "number"); } TEST_CASE_FIXTURE(Fixture, "and_adds_boolean_no_superfluous_union") @@ -971,4 +975,79 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "mm_comparisons_must_return_a_boolean") CHECK(toString(result.errors[1]) == "Metamethod '__lt' must return type 'boolean'"); } +TEST_CASE_FIXTURE(BuiltinsFixture, "reworked_and") +{ + ScopedFastFlag sff[]{ + {"LuauTryhardAnd", true}, + }; + + CheckResult result = check(R"( +local a: number? = 5 +local b: boolean = (a or 1) > 10 +local c -- free + +local x = a and 1 +local y = 'a' and 1 +local z = b and 1 +local w = c and 1 + )"); + + if (FFlag::DebugLuauDeferredConstraintResolution) + { + CHECK("number?" == toString(requireType("x"))); + CHECK("number" == toString(requireType("y"))); + CHECK("false | number" == toString(requireType("z"))); + CHECK("number" == toString(requireType("w"))); // Normalizer considers free & falsy == never + } + else + { + CHECK("number?" == toString(requireType("x"))); + CHECK("number" == toString(requireType("y"))); + CHECK("boolean | number" == toString(requireType("z"))); // 'false' widened to boolean + CHECK("(boolean | number)?" == toString(requireType("w"))); + } +} + +TEST_CASE_FIXTURE(BuiltinsFixture, "reworked_or") +{ + ScopedFastFlag sff[]{ + {"LuauTryhardAnd", true}, + }; + + CheckResult result = check(R"( +local a: number | false = 5 +local b: number? = 6 +local c: boolean = true +local d: true = true +local e: false = false +local f: nil = false + +local a1 = a or 'a' +local b1 = b or 4 +local c1 = c or 'c' +local d1 = d or 'd' +local e1 = e or 'e' +local f1 = f or 'f' + )"); + + if (FFlag::DebugLuauDeferredConstraintResolution) + { + CHECK("number | string" == toString(requireType("a1"))); + CHECK("number" == toString(requireType("b1"))); + CHECK("string | true" == toString(requireType("c1"))); + CHECK("string | true" == toString(requireType("d1"))); + CHECK("string" == toString(requireType("e1"))); + CHECK("string" == toString(requireType("f1"))); + } + else + { + CHECK("number | string" == toString(requireType("a1"))); + CHECK("number" == toString(requireType("b1"))); + CHECK("boolean | string" == toString(requireType("c1"))); // 'true' widened to boolean + CHECK("boolean | string" == toString(requireType("d1"))); // 'true' widened to boolean + CHECK("string" == toString(requireType("e1"))); + CHECK("string" == toString(requireType("f1"))); + } +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.provisional.test.cpp b/tests/TypeInfer.provisional.test.cpp index f6e60cdc..5688eaaa 100644 --- a/tests/TypeInfer.provisional.test.cpp +++ b/tests/TypeInfer.provisional.test.cpp @@ -461,23 +461,27 @@ TEST_CASE_FIXTURE(Fixture, "dcr_can_partially_dispatch_a_constraint") LUAU_REQUIRE_NO_ERRORS(result); - // Solving this requires recognizing that we can partially solve the - // following constraint: + // Solving this requires recognizing that we can't dispatch a constraint + // like this without doing further work: // // (*blocked*) -> () <: (number) -> (b...) // - // The correct thing for us to do is to consider the constraint dispatched, - // but we need to also record a new constraint number <: *blocked* to finish - // the job later. + // We solve this by searching both types for BlockedTypeVars and block the + // constraint on any we find. It also gets the job done, but I'm worried + // about the efficiency of doing so many deep type traversals and it may + // make us more prone to getting stuck on constraint cycles. + // + // If this doesn't pan out, a possible solution is to go further down the + // path of supporting partial constraint dispatch. The way it would work is + // that we'd dispatch the above constraint by binding b... to (), but we + // would append a new constraint number <: *blocked* to the constraint set + // to be solved later. This should be faster and theoretically less prone + // to cyclic constraint dependencies. CHECK("(a, number) -> ()" == toString(requireType("prime_iter"))); } TEST_CASE_FIXTURE(Fixture, "free_options_cannot_be_unified_together") { - ScopedFastFlag sff[] = { - {"LuauFixNameMaps", true}, - }; - TypeArena arena; TypeId nilType = singletonTypes->nilType; @@ -522,8 +526,6 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "for_in_loop_with_zero_iterators") // Ideally, we would not try to export a function type with generic types from incorrect scope TEST_CASE_FIXTURE(BuiltinsFixture, "generic_type_leak_to_module_interface") { - ScopedFastFlag LuauAnyifyModuleReturnGenerics{"LuauAnyifyModuleReturnGenerics", true}; - fileResolver.source["game/A"] = R"( local wrapStrictTable @@ -563,8 +565,6 @@ return wrapStrictTable(Constants, "Constants") TEST_CASE_FIXTURE(BuiltinsFixture, "generic_type_leak_to_module_interface_variadic") { - ScopedFastFlag LuauAnyifyModuleReturnGenerics{"LuauAnyifyModuleReturnGenerics", true}; - fileResolver.source["game/A"] = R"( local wrapStrictTable diff --git a/tests/TypeInfer.refinements.test.cpp b/tests/TypeInfer.refinements.test.cpp index 26f23438..66550be3 100644 --- a/tests/TypeInfer.refinements.test.cpp +++ b/tests/TypeInfer.refinements.test.cpp @@ -35,7 +35,7 @@ std::optional> magicFunctionInstanceIsA( return WithPredicate{booleanPack, {IsAPredicate{std::move(*lvalue), expr.location, tfun->type}}}; } -struct RefinementClassFixture : Fixture +struct RefinementClassFixture : BuiltinsFixture { RefinementClassFixture() { @@ -320,7 +320,7 @@ TEST_CASE_FIXTURE(Fixture, "type_assertion_expr_carry_its_constraints") } } -TEST_CASE_FIXTURE(Fixture, "typeguard_in_if_condition_position") +TEST_CASE_FIXTURE(BuiltinsFixture, "typeguard_in_if_condition_position") { CheckResult result = check(R"( function f(s: any) @@ -332,7 +332,14 @@ TEST_CASE_FIXTURE(Fixture, "typeguard_in_if_condition_position") LUAU_REQUIRE_NO_ERRORS(result); - CHECK_EQ("number", toString(requireTypeAtPosition({3, 26}))); + if (FFlag::DebugLuauDeferredConstraintResolution) + { + CHECK_EQ("any & number", toString(requireTypeAtPosition({3, 26}))); + } + else + { + CHECK_EQ("number", toString(requireTypeAtPosition({3, 26}))); + } } TEST_CASE_FIXTURE(BuiltinsFixture, "typeguard_in_assert_position") @@ -344,10 +351,11 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "typeguard_in_assert_position") )"); LUAU_REQUIRE_NO_ERRORS(result); + REQUIRE_EQ("number", toString(requireType("b"))); } -TEST_CASE_FIXTURE(Fixture, "call_a_more_specific_function_using_typeguard") +TEST_CASE_FIXTURE(BuiltinsFixture, "call_an_incompatible_function_after_using_typeguard") { CheckResult result = check(R"( local function f(x: number) @@ -362,6 +370,7 @@ TEST_CASE_FIXTURE(Fixture, "call_a_more_specific_function_using_typeguard") )"); LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ("Type 'string' could not be converted into 'number'", toString(result.errors[0])); } @@ -648,7 +657,7 @@ TEST_CASE_FIXTURE(Fixture, "narrow_property_of_a_bounded_variable") LUAU_REQUIRE_NO_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "type_narrow_to_vector") +TEST_CASE_FIXTURE(BuiltinsFixture, "type_narrow_to_vector") { CheckResult result = check(R"( local function f(x) @@ -663,7 +672,7 @@ TEST_CASE_FIXTURE(Fixture, "type_narrow_to_vector") CHECK_EQ("*error-type*", toString(requireTypeAtPosition({3, 28}))); } -TEST_CASE_FIXTURE(Fixture, "nonoptional_type_can_narrow_to_nil_if_sense_is_true") +TEST_CASE_FIXTURE(BuiltinsFixture, "nonoptional_type_can_narrow_to_nil_if_sense_is_true") { CheckResult result = check(R"( local t = {"hello"} @@ -690,7 +699,7 @@ TEST_CASE_FIXTURE(Fixture, "nonoptional_type_can_narrow_to_nil_if_sense_is_true" CHECK_EQ("string", toString(requireTypeAtPosition({12, 24}))); // equivalent to type(v) ~= "nil" } -TEST_CASE_FIXTURE(Fixture, "typeguard_not_to_be_string") +TEST_CASE_FIXTURE(BuiltinsFixture, "typeguard_not_to_be_string") { CheckResult result = check(R"( local function f(x: string | number | boolean) @@ -704,11 +713,19 @@ TEST_CASE_FIXTURE(Fixture, "typeguard_not_to_be_string") LUAU_REQUIRE_NO_ERRORS(result); - CHECK_EQ("boolean | number", toString(requireTypeAtPosition({3, 28}))); // type(x) ~= "string" - CHECK_EQ("string", toString(requireTypeAtPosition({5, 28}))); // type(x) == "string" + if (FFlag::DebugLuauDeferredConstraintResolution) + { + CHECK_EQ("(boolean | number | string) & ~string", toString(requireTypeAtPosition({3, 28}))); // type(x) ~= "string" + CHECK_EQ("(boolean | number | string) & string", toString(requireTypeAtPosition({5, 28}))); // type(x) == "string" + } + else + { + CHECK_EQ("boolean | number", toString(requireTypeAtPosition({3, 28}))); // type(x) ~= "string" + CHECK_EQ("string", toString(requireTypeAtPosition({5, 28}))); // type(x) == "string" + } } -TEST_CASE_FIXTURE(Fixture, "typeguard_narrows_for_table") +TEST_CASE_FIXTURE(BuiltinsFixture, "typeguard_narrows_for_table") { CheckResult result = check(R"( local function f(x: string | {x: number} | {y: boolean}) @@ -726,7 +743,7 @@ TEST_CASE_FIXTURE(Fixture, "typeguard_narrows_for_table") CHECK_EQ("string", toString(requireTypeAtPosition({5, 28}))); // type(x) ~= "table" } -TEST_CASE_FIXTURE(Fixture, "typeguard_narrows_for_functions") +TEST_CASE_FIXTURE(BuiltinsFixture, "typeguard_narrows_for_functions") { CheckResult result = check(R"( local function weird(x: string | ((number) -> string)) @@ -740,11 +757,19 @@ TEST_CASE_FIXTURE(Fixture, "typeguard_narrows_for_functions") LUAU_REQUIRE_NO_ERRORS(result); - CHECK_EQ("(number) -> string", toString(requireTypeAtPosition({3, 28}))); // type(x) == "function" - CHECK_EQ("string", toString(requireTypeAtPosition({5, 28}))); // type(x) ~= "function" + if (FFlag::DebugLuauDeferredConstraintResolution) + { + CHECK_EQ("(((number) -> string) | string) & function", toString(requireTypeAtPosition({3, 28}))); // type(x) == "function" + CHECK_EQ("(((number) -> string) | string) & ~function", toString(requireTypeAtPosition({5, 28}))); // type(x) ~= "function" + } + else + { + CHECK_EQ("(number) -> string", toString(requireTypeAtPosition({3, 28}))); // type(x) == "function" + CHECK_EQ("string", toString(requireTypeAtPosition({5, 28}))); // type(x) ~= "function" + } } -TEST_CASE_FIXTURE(Fixture, "type_guard_can_filter_for_intersection_of_tables") +TEST_CASE_FIXTURE(BuiltinsFixture, "type_guard_can_filter_for_intersection_of_tables") { CheckResult result = check(R"( type XYCoord = {x: number} & {y: number} @@ -763,7 +788,7 @@ TEST_CASE_FIXTURE(Fixture, "type_guard_can_filter_for_intersection_of_tables") CHECK_EQ("nil", toString(requireTypeAtPosition({6, 28}))); } -TEST_CASE_FIXTURE(Fixture, "type_guard_can_filter_for_overloaded_function") +TEST_CASE_FIXTURE(BuiltinsFixture, "type_guard_can_filter_for_overloaded_function") { CheckResult result = check(R"( type SomeOverloadedFunction = ((number) -> string) & ((string) -> number) @@ -778,8 +803,16 @@ TEST_CASE_FIXTURE(Fixture, "type_guard_can_filter_for_overloaded_function") LUAU_REQUIRE_NO_ERRORS(result); - CHECK_EQ("((number) -> string) & ((string) -> number)", toString(requireTypeAtPosition({4, 28}))); - CHECK_EQ("nil", toString(requireTypeAtPosition({6, 28}))); + if (FFlag::DebugLuauDeferredConstraintResolution) + { + CHECK_EQ("((((number) -> string) & ((string) -> number))?) & function", toString(requireTypeAtPosition({4, 28}))); + CHECK_EQ("((((number) -> string) & ((string) -> number))?) & ~function", toString(requireTypeAtPosition({6, 28}))); + } + else + { + CHECK_EQ("((number) -> string) & ((string) -> number)", toString(requireTypeAtPosition({4, 28}))); + CHECK_EQ("nil", toString(requireTypeAtPosition({6, 28}))); + } } TEST_CASE_FIXTURE(BuiltinsFixture, "type_guard_narrowed_into_nothingness") @@ -884,7 +917,7 @@ TEST_CASE_FIXTURE(Fixture, "not_a_and_not_b2") } } -TEST_CASE_FIXTURE(Fixture, "either_number_or_string") +TEST_CASE_FIXTURE(BuiltinsFixture, "either_number_or_string") { CheckResult result = check(R"( local function f(x: any) @@ -896,7 +929,14 @@ TEST_CASE_FIXTURE(Fixture, "either_number_or_string") LUAU_REQUIRE_NO_ERRORS(result); - CHECK_EQ("number | string", toString(requireTypeAtPosition({3, 28}))); + if (FFlag::DebugLuauDeferredConstraintResolution) + { + CHECK_EQ("(number | string) & any", toString(requireTypeAtPosition({3, 28}))); + } + else + { + CHECK_EQ("number | string", toString(requireTypeAtPosition({3, 28}))); + } } TEST_CASE_FIXTURE(Fixture, "not_t_or_some_prop_of_t") @@ -946,10 +986,17 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "merge_should_be_fully_agnostic_of_hashmap_or LUAU_REQUIRE_NO_ERRORS(result); - CHECK_EQ("string", toString(requireTypeAtPosition({6, 28}))); + if (FFlag::DebugLuauDeferredConstraintResolution) + { + CHECK_EQ("(string | {| x: string |}) & string", toString(requireTypeAtPosition({6, 28}))); + } + else + { + CHECK_EQ("string", toString(requireTypeAtPosition({6, 28}))); + } } -TEST_CASE_FIXTURE(Fixture, "refine_the_correct_types_opposite_of_when_a_is_not_number_or_string") +TEST_CASE_FIXTURE(BuiltinsFixture, "refine_the_correct_types_opposite_of_when_a_is_not_number_or_string") { CheckResult result = check(R"( local function f(a: string | number | boolean) @@ -963,8 +1010,16 @@ TEST_CASE_FIXTURE(Fixture, "refine_the_correct_types_opposite_of_when_a_is_not_n LUAU_REQUIRE_NO_ERRORS(result); - CHECK_EQ("boolean", toString(requireTypeAtPosition({3, 28}))); - CHECK_EQ("number | string", toString(requireTypeAtPosition({5, 28}))); + if (FFlag::DebugLuauDeferredConstraintResolution) + { + CHECK_EQ("(boolean | number | string) & ~number & ~string", toString(requireTypeAtPosition({3, 28}))); + CHECK_EQ("(boolean | number | string) & (number | string)", toString(requireTypeAtPosition({5, 28}))); + } + else + { + CHECK_EQ("boolean", toString(requireTypeAtPosition({3, 28}))); + CHECK_EQ("number | string", toString(requireTypeAtPosition({5, 28}))); + } } TEST_CASE_FIXTURE(BuiltinsFixture, "is_truthy_constraint_ifelse_expression") @@ -995,7 +1050,7 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "invert_is_truthy_constraint_ifelse_expressio CHECK_EQ("string", toString(requireTypeAtPosition({2, 50}))); } -TEST_CASE_FIXTURE(Fixture, "type_comparison_ifelse_expression") +TEST_CASE_FIXTURE(BuiltinsFixture, "type_comparison_ifelse_expression") { CheckResult result = check(R"( function returnOne(x) @@ -1027,7 +1082,7 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "correctly_lookup_a_shadowed_local_that_which CHECK_EQ("Type 'number' does not have key 'sub'", toString(result.errors[0])); } -TEST_CASE_FIXTURE(Fixture, "correctly_lookup_property_whose_base_was_previously_refined") +TEST_CASE_FIXTURE(BuiltinsFixture, "correctly_lookup_property_whose_base_was_previously_refined") { CheckResult result = check(R"( type T = {x: string | number} @@ -1246,8 +1301,16 @@ TEST_CASE_FIXTURE(RefinementClassFixture, "typeguard_cast_instance_or_vector3_to LUAU_REQUIRE_NO_ERRORS(result); - CHECK_EQ("Vector3", toString(requireTypeAtPosition({3, 28}))); - CHECK_EQ("Instance", toString(requireTypeAtPosition({5, 28}))); + if (FFlag::DebugLuauDeferredConstraintResolution) + { + CHECK_EQ("(Instance | Vector3) & Vector3", toString(requireTypeAtPosition({3, 28}))); + CHECK_EQ("(Instance | Vector3) & ~Vector3", toString(requireTypeAtPosition({5, 28}))); + } + else + { + CHECK_EQ("Vector3", toString(requireTypeAtPosition({3, 28}))); + CHECK_EQ("Instance", toString(requireTypeAtPosition({5, 28}))); + } } TEST_CASE_FIXTURE(RefinementClassFixture, "type_narrow_for_all_the_userdata") @@ -1282,14 +1345,22 @@ TEST_CASE_FIXTURE(RefinementClassFixture, "eliminate_subclasses_of_instance") LUAU_REQUIRE_NO_ERRORS(result); - CHECK_EQ("Folder | Part", toString(requireTypeAtPosition({3, 28}))); - CHECK_EQ("string", toString(requireTypeAtPosition({5, 28}))); + if (FFlag::DebugLuauDeferredConstraintResolution) + { + CHECK_EQ("(Folder | Part | string) & Instance", toString(requireTypeAtPosition({3, 28}))); + CHECK_EQ("(Folder | Part | string) & ~Instance", toString(requireTypeAtPosition({5, 28}))); + } + else + { + CHECK_EQ("Folder | Part", toString(requireTypeAtPosition({3, 28}))); + CHECK_EQ("string", toString(requireTypeAtPosition({5, 28}))); + } } -TEST_CASE_FIXTURE(RefinementClassFixture, "narrow_this_large_union") +TEST_CASE_FIXTURE(RefinementClassFixture, "narrow_from_subclasses_of_instance_or_string_or_vector3") { CheckResult result = check(R"( - local function f(x: Part | Folder | Instance | string | Vector3 | any) + local function f(x: Part | Folder | string | Vector3) if typeof(x) == "Instance" then local foo = x else @@ -1300,8 +1371,16 @@ TEST_CASE_FIXTURE(RefinementClassFixture, "narrow_this_large_union") LUAU_REQUIRE_NO_ERRORS(result); - CHECK_EQ("Folder | Instance | Part", toString(requireTypeAtPosition({3, 28}))); - CHECK_EQ("Vector3 | any | string", toString(requireTypeAtPosition({5, 28}))); + if (FFlag::DebugLuauDeferredConstraintResolution) + { + CHECK_EQ("(Folder | Part | Vector3 | string) & Instance", toString(requireTypeAtPosition({3, 28}))); + CHECK_EQ("(Folder | Part | Vector3 | string) & ~Instance", toString(requireTypeAtPosition({5, 28}))); + } + else + { + CHECK_EQ("Folder | Part", toString(requireTypeAtPosition({3, 28}))); + CHECK_EQ("Vector3 | string", toString(requireTypeAtPosition({5, 28}))); + } } TEST_CASE_FIXTURE(RefinementClassFixture, "x_as_any_if_x_is_instance_elseif_x_is_table") @@ -1342,7 +1421,7 @@ TEST_CASE_FIXTURE(RefinementClassFixture, "x_is_not_instance_or_else_not_part") CHECK_EQ("Part", toString(requireTypeAtPosition({5, 28}))); } -TEST_CASE_FIXTURE(Fixture, "typeguard_doesnt_leak_to_elseif") +TEST_CASE_FIXTURE(BuiltinsFixture, "typeguard_doesnt_leak_to_elseif") { CheckResult result = check(R"( function f(a) @@ -1373,8 +1452,16 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "refine_unknowns") LUAU_REQUIRE_NO_ERRORS(result); - CHECK_EQ("string", toString(requireTypeAtPosition({3, 28}))); - CHECK_EQ("unknown", toString(requireTypeAtPosition({5, 28}))); + if (FFlag::DebugLuauDeferredConstraintResolution) + { + CHECK_EQ("unknown & string", toString(requireTypeAtPosition({3, 28}))); + CHECK_EQ("unknown & ~string", toString(requireTypeAtPosition({5, 28}))); + } + else + { + CHECK_EQ("string", toString(requireTypeAtPosition({3, 28}))); + CHECK_EQ("unknown", toString(requireTypeAtPosition({5, 28}))); + } } TEST_CASE_FIXTURE(BuiltinsFixture, "falsiness_of_TruthyPredicate_narrows_into_nil") @@ -1408,7 +1495,35 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "what_nonsensical_condition") LUAU_REQUIRE_NO_ERRORS(result); - CHECK_EQ("never", toString(requireTypeAtPosition({3, 28}))); + if (FFlag::DebugLuauDeferredConstraintResolution) + { + CHECK_EQ("a & number & string", toString(requireTypeAtPosition({3, 28}))); + } + else + { + CHECK_EQ("never", toString(requireTypeAtPosition({3, 28}))); + } +} + +TEST_CASE_FIXTURE(Fixture, "else_with_no_explicit_expression_should_also_refine_the_tagged_union") +{ + ScopedFastFlag sff{"LuauImplicitElseRefinement", true}; + + CheckResult result = check(R"( + type Ok = { tag: "ok", value: T } + type Err = { tag: "err", err: E } + type Result = Ok | Err + + function and_then(r: Result, f: (T) -> U): Result + if r.tag == "ok" then + return { tag = "ok", value = f(r.value) } + else + return r + end + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); } TEST_SUITE_END(); diff --git a/tests/TypeInfer.tables.test.cpp b/tests/TypeInfer.tables.test.cpp index 68757fef..bf0a0af6 100644 --- a/tests/TypeInfer.tables.test.cpp +++ b/tests/TypeInfer.tables.test.cpp @@ -17,6 +17,7 @@ using namespace Luau; LUAU_FASTFLAG(LuauLowerBoundsCalculation); LUAU_FASTFLAG(DebugLuauDeferredConstraintResolution); LUAU_FASTFLAG(LuauInstantiateInSubtyping) +LUAU_FASTFLAG(LuauNoMoreGlobalSingletonTypes) TEST_SUITE_BEGIN("TableTests"); @@ -1721,6 +1722,8 @@ TEST_CASE_FIXTURE(Fixture, "hide_table_error_properties") TEST_CASE_FIXTURE(BuiltinsFixture, "builtin_table_names") { + ScopedFastFlag luauNewLibraryTypeNames{"LuauNewLibraryTypeNames", true}; + CheckResult result = check(R"( os.h = 2 string.k = 3 @@ -1728,19 +1731,36 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "builtin_table_names") LUAU_REQUIRE_ERROR_COUNT(2, result); - CHECK_EQ("Cannot add property 'h' to table 'os'", toString(result.errors[0])); - CHECK_EQ("Cannot add property 'k' to table 'string'", toString(result.errors[1])); + if (FFlag::LuauNoMoreGlobalSingletonTypes) + { + CHECK_EQ("Cannot add property 'h' to table 'typeof(os)'", toString(result.errors[0])); + CHECK_EQ("Cannot add property 'k' to table 'typeof(string)'", toString(result.errors[1])); + } + else + { + CHECK_EQ("Cannot add property 'h' to table 'os'", toString(result.errors[0])); + CHECK_EQ("Cannot add property 'k' to table 'string'", toString(result.errors[1])); + } } TEST_CASE_FIXTURE(BuiltinsFixture, "persistent_sealed_table_is_immutable") { + ScopedFastFlag luauNewLibraryTypeNames{"LuauNewLibraryTypeNames", true}; + CheckResult result = check(R"( --!nonstrict function os:bad() end )"); LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK_EQ("Cannot add property 'bad' to table 'os'", toString(result.errors[0])); + if (FFlag::LuauNoMoreGlobalSingletonTypes) + { + CHECK_EQ("Cannot add property 'bad' to table 'typeof(os)'", toString(result.errors[0])); + } + else + { + CHECK_EQ("Cannot add property 'bad' to table 'os'", toString(result.errors[0])); + } const TableTypeVar* osType = get(requireType("os")); REQUIRE(osType != nullptr); @@ -3188,6 +3208,7 @@ TEST_CASE_FIXTURE(Fixture, "scalar_is_a_subtype_of_a_compatible_polymorphic_shap TEST_CASE_FIXTURE(Fixture, "scalar_is_not_a_subtype_of_a_compatible_polymorphic_shape_type") { ScopedFastFlag sff{"LuauScalarShapeSubtyping", true}; + ScopedFastFlag luauNewLibraryTypeNames{"LuauNewLibraryTypeNames", true}; CheckResult result = check(R"( local function f(s) @@ -3200,25 +3221,47 @@ TEST_CASE_FIXTURE(Fixture, "scalar_is_not_a_subtype_of_a_compatible_polymorphic_ )"); LUAU_REQUIRE_ERROR_COUNT(3, result); - CHECK_EQ(R"(Type 'string' could not be converted into 't1 where t1 = {- absolutely_no_scalar_has_this_method: (t1) -> (a...) -}' + + if (FFlag::LuauNoMoreGlobalSingletonTypes) + { + CHECK_EQ(R"(Type 'string' could not be converted into 't1 where t1 = {- absolutely_no_scalar_has_this_method: (t1) -> (a...) -}' +caused by: + The former's metatable does not satisfy the requirements. Table type 'typeof(string)' not compatible with type 't1 where t1 = {- absolutely_no_scalar_has_this_method: (t1) -> (a...) -}' because the former is missing field 'absolutely_no_scalar_has_this_method')", + toString(result.errors[0])); + CHECK_EQ(R"(Type '"bar"' could not be converted into 't1 where t1 = {- absolutely_no_scalar_has_this_method: (t1) -> (a...) -}' +caused by: + The former's metatable does not satisfy the requirements. Table type 'typeof(string)' not compatible with type 't1 where t1 = {- absolutely_no_scalar_has_this_method: (t1) -> (a...) -}' because the former is missing field 'absolutely_no_scalar_has_this_method')", + toString(result.errors[1])); + CHECK_EQ(R"(Type '"bar" | "baz"' could not be converted into 't1 where t1 = {- absolutely_no_scalar_has_this_method: (t1) -> (a...) -}' +caused by: + Not all union options are compatible. Type '"bar"' could not be converted into 't1 where t1 = {- absolutely_no_scalar_has_this_method: (t1) -> (a...) -}' +caused by: + The former's metatable does not satisfy the requirements. Table type 'typeof(string)' not compatible with type 't1 where t1 = {- absolutely_no_scalar_has_this_method: (t1) -> (a...) -}' because the former is missing field 'absolutely_no_scalar_has_this_method')", + toString(result.errors[2])); + } + else + { + CHECK_EQ(R"(Type 'string' could not be converted into 't1 where t1 = {- absolutely_no_scalar_has_this_method: (t1) -> (a...) -}' caused by: The former's metatable does not satisfy the requirements. Table type 'string' not compatible with type 't1 where t1 = {- absolutely_no_scalar_has_this_method: (t1) -> (a...) -}' because the former is missing field 'absolutely_no_scalar_has_this_method')", - toString(result.errors[0])); - CHECK_EQ(R"(Type '"bar"' could not be converted into 't1 where t1 = {- absolutely_no_scalar_has_this_method: (t1) -> (a...) -}' + toString(result.errors[0])); + CHECK_EQ(R"(Type '"bar"' could not be converted into 't1 where t1 = {- absolutely_no_scalar_has_this_method: (t1) -> (a...) -}' caused by: The former's metatable does not satisfy the requirements. Table type 'string' not compatible with type 't1 where t1 = {- absolutely_no_scalar_has_this_method: (t1) -> (a...) -}' because the former is missing field 'absolutely_no_scalar_has_this_method')", - toString(result.errors[1])); - CHECK_EQ(R"(Type '"bar" | "baz"' could not be converted into 't1 where t1 = {- absolutely_no_scalar_has_this_method: (t1) -> (a...) -}' + toString(result.errors[1])); + CHECK_EQ(R"(Type '"bar" | "baz"' could not be converted into 't1 where t1 = {- absolutely_no_scalar_has_this_method: (t1) -> (a...) -}' caused by: Not all union options are compatible. Type '"bar"' could not be converted into 't1 where t1 = {- absolutely_no_scalar_has_this_method: (t1) -> (a...) -}' caused by: The former's metatable does not satisfy the requirements. Table type 'string' not compatible with type 't1 where t1 = {- absolutely_no_scalar_has_this_method: (t1) -> (a...) -}' because the former is missing field 'absolutely_no_scalar_has_this_method')", - toString(result.errors[2])); + toString(result.errors[2])); + } } TEST_CASE_FIXTURE(Fixture, "a_free_shape_can_turn_into_a_scalar_if_it_is_compatible") { ScopedFastFlag sff{"LuauScalarShapeSubtyping", true}; + ScopedFastFlag luauScalarShapeUnifyToMtOwner{"LuauScalarShapeUnifyToMtOwner", true}; // Changes argument from table type to primitive CheckResult result = check(R"( local function f(s): string @@ -3234,6 +3277,7 @@ TEST_CASE_FIXTURE(Fixture, "a_free_shape_can_turn_into_a_scalar_if_it_is_compati TEST_CASE_FIXTURE(Fixture, "a_free_shape_cannot_turn_into_a_scalar_if_it_is_not_compatible") { ScopedFastFlag sff{"LuauScalarShapeSubtyping", true}; + ScopedFastFlag luauNewLibraryTypeNames{"LuauNewLibraryTypeNames", true}; CheckResult result = check(R"( local function f(s): string @@ -3243,11 +3287,42 @@ TEST_CASE_FIXTURE(Fixture, "a_free_shape_cannot_turn_into_a_scalar_if_it_is_not_ )"); LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK_EQ(R"(Type 't1 where t1 = {+ absolutely_no_scalar_has_this_method: (t1) -> (a, b...) +}' could not be converted into 'string' + if (FFlag::LuauNoMoreGlobalSingletonTypes) + { + CHECK_EQ(R"(Type 't1 where t1 = {+ absolutely_no_scalar_has_this_method: (t1) -> (a, b...) +}' could not be converted into 'string' +caused by: + The former's metatable does not satisfy the requirements. Table type 'typeof(string)' not compatible with type 't1 where t1 = {+ absolutely_no_scalar_has_this_method: (t1) -> (a, b...) +}' because the former is missing field 'absolutely_no_scalar_has_this_method')", + toString(result.errors[0])); + CHECK_EQ("(t1) -> string where t1 = {+ absolutely_no_scalar_has_this_method: (t1) -> (a, b...) +}", toString(requireType("f"))); + } + else + { + CHECK_EQ(R"(Type 't1 where t1 = {+ absolutely_no_scalar_has_this_method: (t1) -> (a, b...) +}' could not be converted into 'string' caused by: The former's metatable does not satisfy the requirements. Table type 'string' not compatible with type 't1 where t1 = {+ absolutely_no_scalar_has_this_method: (t1) -> (a, b...) +}' because the former is missing field 'absolutely_no_scalar_has_this_method')", - toString(result.errors[0])); - CHECK_EQ("(t1) -> string where t1 = {+ absolutely_no_scalar_has_this_method: (t1) -> (a, b...) +}", toString(requireType("f"))); + toString(result.errors[0])); + CHECK_EQ("(t1) -> string where t1 = {+ absolutely_no_scalar_has_this_method: (t1) -> (a, b...) +}", toString(requireType("f"))); + } +} + +TEST_CASE_FIXTURE(BuiltinsFixture, "a_free_shape_can_turn_into_a_scalar_directly") +{ + ScopedFastFlag luauScalarShapeSubtyping{"LuauScalarShapeSubtyping", true}; + ScopedFastFlag luauScalarShapeUnifyToMtOwner{"LuauScalarShapeUnifyToMtOwner", true}; + + CheckResult result = check(R"( + local function stringByteList(str) + local out = {} + for i = 1, #str do + table.insert(out, string.byte(str, i)) + end + return table.concat(out, ",") + end + + local x = stringByteList("xoo") + )"); + + LUAU_REQUIRE_NO_ERRORS(result); } TEST_CASE_FIXTURE(Fixture, "invariant_table_properties_means_instantiating_tables_in_call_is_unsound") diff --git a/tests/TypeInfer.test.cpp b/tests/TypeInfer.test.cpp index 6c7201a6..04b8bf57 100644 --- a/tests/TypeInfer.test.cpp +++ b/tests/TypeInfer.test.cpp @@ -1125,43 +1125,6 @@ TEST_CASE_FIXTURE(Fixture, "bidirectional_checking_of_higher_order_function") CHECK(location.end.line == 4); } -TEST_CASE_FIXTURE(Fixture, "dcr_can_partially_dispatch_a_constraint") -{ - ScopedFastFlag sff[] = { - {"DebugLuauDeferredConstraintResolution", true}, - }; - - CheckResult result = check(R"( - local function hasDivisors(value: number) - end - - function prime_iter(state, index) - hasDivisors(index) - index += 1 - end - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - - // Solving this requires recognizing that we can't dispatch a constraint - // like this without doing further work: - // - // (*blocked*) -> () <: (number) -> (b...) - // - // We solve this by searching both types for BlockedTypeVars and block the - // constraint on any we find. It also gets the job done, but I'm worried - // about the efficiency of doing so many deep type traversals and it may - // make us more prone to getting stuck on constraint cycles. - // - // If this doesn't pan out, a possible solution is to go further down the - // path of supporting partial constraint dispatch. The way it would work is - // that we'd dispatch the above constraint by binding b... to (), but we - // would append a new constraint number <: *blocked* to the constraint set - // to be solved later. This should be faster and theoretically less prone - // to cyclic constraint dependencies. - CHECK("(a, number) -> ()" == toString(requireType("prime_iter"))); -} - TEST_CASE_FIXTURE(BuiltinsFixture, "it_is_ok_to_have_inconsistent_number_of_return_values_in_nonstrict") { CheckResult result = check(R"( diff --git a/tests/TypeInfer.typePacks.cpp b/tests/TypeInfer.typePacks.cpp index 4c8eeac6..cde651df 100644 --- a/tests/TypeInfer.typePacks.cpp +++ b/tests/TypeInfer.typePacks.cpp @@ -1002,8 +1002,6 @@ TEST_CASE_FIXTURE(Fixture, "unify_variadic_tails_in_arguments_free") TEST_CASE_FIXTURE(BuiltinsFixture, "type_packs_with_tails_in_vararg_adjustment") { - ScopedFastFlag luauFixVarargExprHeadType{"LuauFixVarargExprHeadType", true}; - CheckResult result = check(R"( local function wrapReject(fn: (self: any, ...TArg) -> ...TResult): (self: any, ...TArg) -> ...TResult return function(self, ...) diff --git a/tests/TypeInfer.unionTypes.test.cpp b/tests/TypeInfer.unionTypes.test.cpp index 0c25386f..627fbb56 100644 --- a/tests/TypeInfer.unionTypes.test.cpp +++ b/tests/TypeInfer.unionTypes.test.cpp @@ -395,6 +395,23 @@ local e = a.z CHECK_EQ("Type 'A | B | C | D' does not have key 'z'", toString(result.errors[3])); } +TEST_CASE_FIXTURE(Fixture, "optional_iteration") +{ + ScopedFastFlag luauNilIterator{"LuauNilIterator", true}; + + CheckResult result = check(R"( +function foo(values: {number}?) + local s = 0 + for _, value in values do + s += value + end +end + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ("Value of type '{number}?' could be nil", toString(result.errors[0])); +} + TEST_CASE_FIXTURE(Fixture, "unify_unsealed_table_union_check") { CheckResult result = check(R"( diff --git a/tests/TypeInfer.unknownnever.test.cpp b/tests/TypeInfer.unknownnever.test.cpp index 2288db4e..9d1e46c5 100644 --- a/tests/TypeInfer.unknownnever.test.cpp +++ b/tests/TypeInfer.unknownnever.test.cpp @@ -284,6 +284,7 @@ TEST_CASE_FIXTURE(Fixture, "dont_unify_operands_if_one_of_the_operand_is_never_i ScopedFastFlag sff[]{ {"LuauUnknownAndNeverType", true}, {"LuauNeverTypesAndOperatorsInference", true}, + {"LuauTryhardAnd", true}, }; CheckResult result = check(R"( @@ -293,7 +294,8 @@ TEST_CASE_FIXTURE(Fixture, "dont_unify_operands_if_one_of_the_operand_is_never_i )"); LUAU_REQUIRE_NO_ERRORS(result); - CHECK_EQ("(nil, a) -> boolean", toString(requireType("ord"))); + // Widening doesn't normalize yet, so the result is a bit strange + CHECK_EQ("(nil, a) -> boolean | boolean", toString(requireType("ord"))); } TEST_CASE_FIXTURE(Fixture, "math_operators_and_never") diff --git a/tools/faillist.txt b/tools/faillist.txt index 4ac2b357..a228c171 100644 --- a/tools/faillist.txt +++ b/tools/faillist.txt @@ -32,10 +32,8 @@ AutocompleteTest.type_correct_expected_argument_type_suggestion_optional AutocompleteTest.type_correct_expected_argument_type_suggestion_self AutocompleteTest.type_correct_expected_return_type_pack_suggestion AutocompleteTest.type_correct_expected_return_type_suggestion -AutocompleteTest.type_correct_full_type_suggestion AutocompleteTest.type_correct_function_no_parenthesis AutocompleteTest.type_correct_function_return_types -AutocompleteTest.type_correct_function_type_suggestion AutocompleteTest.type_correct_keywords AutocompleteTest.type_correct_suggestion_for_overloads AutocompleteTest.type_correct_suggestion_in_argument @@ -53,12 +51,10 @@ BuiltinTests.dont_add_definitions_to_persistent_types BuiltinTests.find_capture_types BuiltinTests.find_capture_types2 BuiltinTests.find_capture_types3 -BuiltinTests.gmatch_capture_types BuiltinTests.gmatch_capture_types2 BuiltinTests.gmatch_capture_types_balanced_escaped_parens BuiltinTests.gmatch_capture_types_default_capture BuiltinTests.gmatch_capture_types_parens_in_sets_are_ignored -BuiltinTests.gmatch_capture_types_set_containing_lbracket BuiltinTests.gmatch_definition BuiltinTests.ipairs_iterator_should_infer_types_and_type_check BuiltinTests.match_capture_types @@ -74,13 +70,12 @@ BuiltinTests.set_metatable_needs_arguments BuiltinTests.setmetatable_should_not_mutate_persisted_types BuiltinTests.sort_with_bad_predicate BuiltinTests.string_format_arg_count_mismatch -BuiltinTests.string_format_arg_types_inference BuiltinTests.string_format_as_method BuiltinTests.string_format_correctly_ordered_types BuiltinTests.string_format_report_all_type_errors_at_correct_positions BuiltinTests.string_format_use_correct_argument BuiltinTests.string_format_use_correct_argument2 -BuiltinTests.string_format_use_correct_argument3 +BuiltinTests.strings_have_methods BuiltinTests.table_freeze_is_generic BuiltinTests.table_insert_correctly_infers_type_of_array_2_args_overload BuiltinTests.table_insert_correctly_infers_type_of_array_3_args_overload @@ -114,7 +109,6 @@ GenericsTests.generic_factories GenericsTests.generic_functions_should_be_memory_safe GenericsTests.generic_table_method GenericsTests.generic_type_pack_parentheses -GenericsTests.generic_type_pack_unification1 GenericsTests.generic_type_pack_unification2 GenericsTests.higher_rank_polymorphism_should_not_accept_instantiated_arguments GenericsTests.infer_generic_function_function_argument @@ -174,46 +168,36 @@ ProvisionalTests.while_body_are_also_refined RefinementTest.apply_refinements_on_astexprindexexpr_whose_subscript_expr_is_constant_string RefinementTest.assert_a_to_be_truthy_then_assert_a_to_be_number RefinementTest.assert_non_binary_expressions_actually_resolve_constraints -RefinementTest.call_a_more_specific_function_using_typeguard +RefinementTest.call_an_incompatible_function_after_using_typeguard RefinementTest.correctly_lookup_property_whose_base_was_previously_refined RefinementTest.correctly_lookup_property_whose_base_was_previously_refined2 RefinementTest.discriminate_from_isa_of_x RefinementTest.discriminate_from_truthiness_of_x RefinementTest.discriminate_on_properties_of_disjoint_tables_where_that_property_is_true_or_false RefinementTest.discriminate_tag -RefinementTest.either_number_or_string -RefinementTest.eliminate_subclasses_of_instance +RefinementTest.else_with_no_explicit_expression_should_also_refine_the_tagged_union RefinementTest.falsiness_of_TruthyPredicate_narrows_into_nil RefinementTest.index_on_a_refined_property RefinementTest.invert_is_truthy_constraint_ifelse_expression RefinementTest.is_truthy_constraint_ifelse_expression -RefinementTest.merge_should_be_fully_agnostic_of_hashmap_ordering RefinementTest.narrow_property_of_a_bounded_variable -RefinementTest.narrow_this_large_union RefinementTest.nonoptional_type_can_narrow_to_nil_if_sense_is_true RefinementTest.not_t_or_some_prop_of_t RefinementTest.refine_a_property_not_to_be_nil_through_an_intersection_table -RefinementTest.refine_the_correct_types_opposite_of_when_a_is_not_number_or_string RefinementTest.refine_unknowns RefinementTest.truthy_constraint_on_properties RefinementTest.type_comparison_ifelse_expression RefinementTest.type_guard_can_filter_for_intersection_of_tables -RefinementTest.type_guard_can_filter_for_overloaded_function RefinementTest.type_guard_narrowed_into_nothingness RefinementTest.type_narrow_for_all_the_userdata RefinementTest.type_narrow_to_vector RefinementTest.typeguard_cast_free_table_to_vector -RefinementTest.typeguard_cast_instance_or_vector3_to_vector -RefinementTest.typeguard_doesnt_leak_to_elseif RefinementTest.typeguard_in_assert_position -RefinementTest.typeguard_in_if_condition_position -RefinementTest.typeguard_narrows_for_functions RefinementTest.typeguard_narrows_for_table -RefinementTest.typeguard_not_to_be_string -RefinementTest.what_nonsensical_condition RefinementTest.x_as_any_if_x_is_instance_elseif_x_is_table RefinementTest.x_is_not_instance_or_else_not_part RuntimeLimits.typescript_port_of_Result_type +TableTests.a_free_shape_can_turn_into_a_scalar_directly TableTests.a_free_shape_can_turn_into_a_scalar_if_it_is_compatible TableTests.a_free_shape_cannot_turn_into_a_scalar_if_it_is_not_compatible TableTests.access_index_metamethod_that_returns_variadic @@ -249,7 +233,6 @@ TableTests.generic_table_instantiation_potential_regression TableTests.getmetatable_returns_pointer_to_metatable TableTests.give_up_after_one_metatable_index_look_up TableTests.hide_table_error_properties -TableTests.indexer_fn TableTests.indexer_on_sealed_table_must_unify_with_free_table TableTests.indexing_from_a_table_should_prefer_properties_when_possible TableTests.inequality_operators_imply_exactly_matching_types @@ -262,7 +245,6 @@ TableTests.invariant_table_properties_means_instantiating_tables_in_assignment_i TableTests.invariant_table_properties_means_instantiating_tables_in_call_is_unsound TableTests.leaking_bad_metatable_errors TableTests.less_exponential_blowup_please -TableTests.meta_add TableTests.meta_add_both_ways TableTests.meta_add_inferred TableTests.metatable_mismatch_should_fail @@ -389,7 +371,6 @@ TypeInferFunctions.improved_function_arg_mismatch_error_nonstrict TypeInferFunctions.improved_function_arg_mismatch_errors TypeInferFunctions.infer_anonymous_function_arguments TypeInferFunctions.infer_return_type_from_selected_overload -TypeInferFunctions.infer_return_value_type TypeInferFunctions.infer_that_function_does_not_return_a_table TypeInferFunctions.list_all_overloads_if_no_overload_takes_given_argument_count TypeInferFunctions.list_only_alternative_overloads_that_match_argument_count @@ -409,10 +390,12 @@ TypeInferFunctions.too_many_return_values TypeInferFunctions.too_many_return_values_in_parentheses TypeInferFunctions.too_many_return_values_no_function TypeInferFunctions.vararg_function_is_quantified +TypeInferLoops.for_in_loop TypeInferLoops.for_in_loop_error_on_factory_not_returning_the_right_amount_of_values TypeInferLoops.for_in_loop_with_next TypeInferLoops.for_in_with_generic_next TypeInferLoops.for_in_with_just_one_iterator_is_ok +TypeInferLoops.loop_iter_metamethod_ok_with_inference TypeInferLoops.loop_iter_no_indexer_nonstrict TypeInferLoops.loop_iter_trailing_nil TypeInferLoops.unreachable_code_after_infinite_loop @@ -430,8 +413,6 @@ TypeInferOOP.dont_suggest_using_colon_rather_than_dot_if_not_defined_with_colon TypeInferOOP.inferring_hundreds_of_self_calls_should_not_suffocate_memory TypeInferOOP.methods_are_topologically_sorted TypeInferOOP.object_constructor_can_refer_to_method_of_self -TypeInferOperators.and_or_ternary -TypeInferOperators.CallAndOrOfFunctions TypeInferOperators.cannot_compare_tables_that_do_not_have_the_same_metatable TypeInferOperators.cannot_indirectly_compare_types_that_do_not_have_a_metatable TypeInferOperators.cannot_indirectly_compare_types_that_do_not_offer_overloaded_ordering_operators @@ -517,6 +498,7 @@ UnionTypes.optional_assignment_errors UnionTypes.optional_call_error UnionTypes.optional_field_access_error UnionTypes.optional_index_error +UnionTypes.optional_iteration UnionTypes.optional_length_error UnionTypes.optional_missing_key_error_details UnionTypes.optional_union_follow From aa7c64517ca550eff7087a197e667f8676db7c17 Mon Sep 17 00:00:00 2001 From: boyned//Kampfkarren Date: Wed, 16 Nov 2022 10:15:01 -0800 Subject: [PATCH 391/399] Fix string interpolation autocomplete and location (#748) String interpolation autocomplete was not working and was just using default autocomplete. This fixes it so that inside a string it is treated as normal, and inside an expression it suggests expression level autocomplete. Also fixes the range, which was previously just the first lexeme. --- Analysis/src/Autocomplete.cpp | 38 ++++++++++++++++++-- Ast/src/Parser.cpp | 50 +++++++++++++++++++------- tests/AstJsonEncoder.test.cpp | 2 +- tests/Autocomplete.test.cpp | 67 ++++++++++++++++++++++++++++++++++- 4 files changed, 140 insertions(+), 17 deletions(-) diff --git a/Analysis/src/Autocomplete.cpp b/Analysis/src/Autocomplete.cpp index 224e9440..50dc254f 100644 --- a/Analysis/src/Autocomplete.cpp +++ b/Analysis/src/Autocomplete.cpp @@ -1219,6 +1219,31 @@ static std::optional getMethodContainingClass(const ModuleP return std::nullopt; } +static bool stringPartOfInterpString(const AstNode* node, Position position) +{ + const AstExprInterpString* interpString = node->as(); + if (!interpString) + { + return false; + } + + for (const AstExpr* expression : interpString->expressions) + { + if (expression->location.containsClosed(position)) + { + return false; + } + } + + return true; +} + +static bool isSimpleInterpolatedString(const AstNode* node) +{ + const AstExprInterpString* interpString = node->as(); + return interpString != nullptr && interpString->expressions.size == 0; +} + static std::optional autocompleteStringParams(const SourceModule& sourceModule, const ModulePtr& module, const std::vector& nodes, Position position, StringCompletionCallback callback) { @@ -1227,7 +1252,7 @@ static std::optional autocompleteStringParams(const Source return std::nullopt; } - if (!nodes.back()->is() && !nodes.back()->is()) + if (!nodes.back()->is() && !isSimpleInterpolatedString(nodes.back()) && !nodes.back()->is()) { return std::nullopt; } @@ -1432,7 +1457,7 @@ static AutocompleteResult autocomplete(const SourceModule& sourceModule, const M return autocompleteExpression(sourceModule, *module, singletonTypes, &typeArena, ancestry, position); else if (AstStatRepeat* statRepeat = extractStat(ancestry); statRepeat) return {autocompleteStatement(sourceModule, *module, ancestry, position), ancestry, AutocompleteContext::Statement}; - else if (AstExprTable* exprTable = parent->as(); exprTable && (node->is() || node->is())) + else if (AstExprTable* exprTable = parent->as(); exprTable && (node->is() || node->is() || node->is())) { for (const auto& [kind, key, value] : exprTable->items) { @@ -1471,7 +1496,7 @@ static AutocompleteResult autocomplete(const SourceModule& sourceModule, const M { return {*ret, ancestry, AutocompleteContext::String}; } - else if (node->is()) + else if (node->is() || isSimpleInterpolatedString(node)) { AutocompleteEntryMap result; @@ -1497,6 +1522,13 @@ static AutocompleteResult autocomplete(const SourceModule& sourceModule, const M return {result, ancestry, AutocompleteContext::String}; } + else if (stringPartOfInterpString(node, position)) + { + // We're not a simple interpolated string, we're something like `a{"b"}@1`, and we + // can't know what to format to + AutocompleteEntryMap map; + return {map, ancestry, AutocompleteContext::String}; + } if (node->is()) return {}; diff --git a/Ast/src/Parser.cpp b/Ast/src/Parser.cpp index 8338a04a..85b0d31a 100644 --- a/Ast/src/Parser.cpp +++ b/Ast/src/Parser.cpp @@ -2661,6 +2661,7 @@ AstExpr* Parser::parseInterpString() TempVector expressions(scratchExpr); Location startLocation = lexer.current().location; + Location endLocation; do { @@ -2668,16 +2669,16 @@ AstExpr* Parser::parseInterpString() LUAU_ASSERT(currentLexeme.type == Lexeme::InterpStringBegin || currentLexeme.type == Lexeme::InterpStringMid || currentLexeme.type == Lexeme::InterpStringEnd || currentLexeme.type == Lexeme::InterpStringSimple); - Location location = currentLexeme.location; + endLocation = currentLexeme.location; - Location startOfBrace = Location(location.end, 1); + Location startOfBrace = Location(endLocation.end, 1); scratchData.assign(currentLexeme.data, currentLexeme.length); if (!Lexer::fixupQuotedString(scratchData)) { nextLexeme(); - return reportExprError(startLocation, {}, "Interpolated string literal contains malformed escape sequence"); + return reportExprError(Location{startLocation, endLocation}, {}, "Interpolated string literal contains malformed escape sequence"); } AstArray chars = copy(scratchData); @@ -2688,15 +2689,36 @@ AstExpr* Parser::parseInterpString() if (currentLexeme.type == Lexeme::InterpStringEnd || currentLexeme.type == Lexeme::InterpStringSimple) { - AstArray> stringsArray = copy(strings); - AstArray expressionsArray = copy(expressions); - - return allocator.alloc(startLocation, stringsArray, expressionsArray); + break; } - AstExpr* expression = parseExpr(); + bool errorWhileChecking = false; - expressions.push_back(expression); + switch (lexer.current().type) + { + case Lexeme::InterpStringMid: + case Lexeme::InterpStringEnd: + { + errorWhileChecking = true; + nextLexeme(); + expressions.push_back(reportExprError(endLocation, {}, "Malformed interpolated string, expected expression inside '{}'")); + break; + } + case Lexeme::BrokenString: + { + errorWhileChecking = true; + nextLexeme(); + expressions.push_back(reportExprError(endLocation, {}, "Malformed interpolated string, did you forget to add a '`'?")); + break; + } + default: + expressions.push_back(parseExpr()); + } + + if (errorWhileChecking) + { + break; + } switch (lexer.current().type) { @@ -2706,14 +2728,18 @@ AstExpr* Parser::parseInterpString() break; case Lexeme::BrokenInterpDoubleBrace: nextLexeme(); - return reportExprError(location, {}, ERROR_INVALID_INTERP_DOUBLE_BRACE); + return reportExprError(endLocation, {}, ERROR_INVALID_INTERP_DOUBLE_BRACE); case Lexeme::BrokenString: nextLexeme(); - return reportExprError(location, {}, "Malformed interpolated string, did you forget to add a '}'?"); + return reportExprError(endLocation, {}, "Malformed interpolated string, did you forget to add a '}'?"); default: - return reportExprError(location, {}, "Malformed interpolated string, got %s", lexer.current().toString().c_str()); + return reportExprError(endLocation, {}, "Malformed interpolated string, got %s", lexer.current().toString().c_str()); } } while (true); + + AstArray> stringsArray = copy(strings); + AstArray expressionsArray = copy(expressions); + return allocator.alloc(Location{startLocation, endLocation}, stringsArray, expressionsArray); } AstExpr* Parser::parseNumber() diff --git a/tests/AstJsonEncoder.test.cpp b/tests/AstJsonEncoder.test.cpp index 81e74941..a14d5f59 100644 --- a/tests/AstJsonEncoder.test.cpp +++ b/tests/AstJsonEncoder.test.cpp @@ -183,7 +183,7 @@ TEST_CASE_FIXTURE(JsonEncoderFixture, "encode_AstExprInterpString") AstStat* statement = expectParseStatement("local a = `var = {x}`"); std::string_view expected = - R"({"type":"AstStatLocal","location":"0,0 - 0,18","vars":[{"luauType":null,"name":"a","type":"AstLocal","location":"0,6 - 0,7"}],"values":[{"type":"AstExprInterpString","location":"0,10 - 0,18","strings":["var = ",""],"expressions":[{"type":"AstExprGlobal","location":"0,18 - 0,19","global":"x"}]}]})"; + R"({"type":"AstStatLocal","location":"0,0 - 0,21","vars":[{"luauType":null,"name":"a","type":"AstLocal","location":"0,6 - 0,7"}],"values":[{"type":"AstExprInterpString","location":"0,10 - 0,21","strings":["var = ",""],"expressions":[{"type":"AstExprGlobal","location":"0,18 - 0,19","global":"x"}]}]})"; CHECK(toJson(statement) == expected); } diff --git a/tests/Autocomplete.test.cpp b/tests/Autocomplete.test.cpp index 9a5c3411..45baec2c 100644 --- a/tests/Autocomplete.test.cpp +++ b/tests/Autocomplete.test.cpp @@ -7,6 +7,7 @@ #include "Luau/StringUtils.h" #include "Fixture.h" +#include "ScopedFlags.h" #include "doctest.h" @@ -2708,13 +2709,77 @@ a = if temp then even else abc@3 CHECK(ac.entryMap.count("abcdef")); } -TEST_CASE_FIXTURE(ACFixture, "autocomplete_interpolated_string") +TEST_CASE_FIXTURE(ACFixture, "autocomplete_interpolated_string_constant") { + ScopedFastFlag sff{"LuauInterpolatedStringBaseSupport", true}; + + check(R"(f(`@1`))"); + auto ac = autocomplete('1'); + CHECK(ac.entryMap.empty()); + CHECK_EQ(ac.context, AutocompleteContext::String); + + check(R"(f(`@1 {"a"}`))"); + ac = autocomplete('1'); + CHECK(ac.entryMap.empty()); + CHECK_EQ(ac.context, AutocompleteContext::String); + + check(R"(f(`{"a"} @1`))"); + ac = autocomplete('1'); + CHECK(ac.entryMap.empty()); + CHECK_EQ(ac.context, AutocompleteContext::String); + + check(R"(f(`{"a"} @1 {"b"}`))"); + ac = autocomplete('1'); + CHECK(ac.entryMap.empty()); + CHECK_EQ(ac.context, AutocompleteContext::String); +} + +TEST_CASE_FIXTURE(ACFixture, "autocomplete_interpolated_string_expression") +{ + ScopedFastFlag sff{"LuauInterpolatedStringBaseSupport", true}; + check(R"(f(`expression = {@1}`))"); + auto ac = autocomplete('1'); + CHECK(ac.entryMap.count("table")); + CHECK_EQ(ac.context, AutocompleteContext::Expression); +} + +TEST_CASE_FIXTURE(ACFixture, "autocomplete_interpolated_string_expression_with_comments") +{ + ScopedFastFlag sff{"LuauInterpolatedStringBaseSupport", true}; + + check(R"(f(`expression = {--[[ bla bla bla ]]@1`))"); auto ac = autocomplete('1'); CHECK(ac.entryMap.count("table")); CHECK_EQ(ac.context, AutocompleteContext::Expression); + + check(R"(f(`expression = {@1 --[[ bla bla bla ]]`))"); + ac = autocomplete('1'); + CHECK(!ac.entryMap.empty()); + CHECK(ac.entryMap.count("table")); + CHECK_EQ(ac.context, AutocompleteContext::Expression); +} + +TEST_CASE_FIXTURE(ACFixture, "autocomplete_interpolated_string_as_singleton") +{ + ScopedFastFlag sff{"LuauInterpolatedStringBaseSupport", true}; + + check(R"( + --!strict + local function f(a: "cat" | "dog") end + + f(`@1`) + f(`uhhh{'try'}@2`) + )"); + + auto ac = autocomplete('1'); + CHECK(ac.entryMap.count("cat")); + CHECK_EQ(ac.context, AutocompleteContext::String); + + ac = autocomplete('2'); + CHECK(ac.entryMap.empty()); + CHECK_EQ(ac.context, AutocompleteContext::String); } TEST_CASE_FIXTURE(ACFixture, "autocomplete_explicit_type_pack") From 95d9c6d194f3b5ddfbb464d012e62da7e8b1ba59 Mon Sep 17 00:00:00 2001 From: Andy Friesen Date: Fri, 18 Nov 2022 11:47:21 -0800 Subject: [PATCH 392/399] Sync to upstream/release/553 (#751) * Autocomplete support for interpolated strings. * Improved parse errors in various situations involving interpolated strings. --- Analysis/include/Luau/Constraint.h | 19 +- Analysis/include/Luau/ConstraintSolver.h | 3 + Analysis/src/ConstraintGraphBuilder.cpp | 124 +++-------- Analysis/src/ConstraintSolver.cpp | 272 +++++++++++++++++++---- Analysis/src/ToString.cpp | 5 + Analysis/src/TypeChecker2.cpp | 9 + Analysis/src/TypedAllocator.cpp | 4 + CodeGen/src/CodeAllocator.cpp | 4 + Common/include/Luau/ExperimentalFlags.h | 1 - VM/src/lapi.cpp | 32 --- tests/TypeInfer.tables.test.cpp | 4 +- tools/faillist.txt | 12 - 12 files changed, 301 insertions(+), 188 deletions(-) diff --git a/Analysis/include/Luau/Constraint.h b/Analysis/include/Luau/Constraint.h index 4370d0cf..16a08e87 100644 --- a/Analysis/include/Luau/Constraint.h +++ b/Analysis/include/Luau/Constraint.h @@ -132,6 +132,23 @@ struct HasPropConstraint std::string prop; }; +// result ~ setProp subjectType ["prop", "prop2", ...] propType +// +// If the subject is a table or table-like thing that already has the named +// property chain, we unify propType with that existing property type. +// +// If the subject is a free table, we augment it in place. +// +// If the subject is an unsealed table, result is an augmented table that +// includes that new prop. +struct SetPropConstraint +{ + TypeId resultType; + TypeId subjectType; + std::vector path; + TypeId propType; +}; + // result ~ if isSingleton D then ~D else unknown where D = discriminantType struct SingletonOrTopTypeConstraint { @@ -141,7 +158,7 @@ struct SingletonOrTopTypeConstraint using ConstraintV = Variant; + HasPropConstraint, SetPropConstraint, SingletonOrTopTypeConstraint>; struct Constraint { diff --git a/Analysis/include/Luau/ConstraintSolver.h b/Analysis/include/Luau/ConstraintSolver.h index 7b89a278..e05f6f1f 100644 --- a/Analysis/include/Luau/ConstraintSolver.h +++ b/Analysis/include/Luau/ConstraintSolver.h @@ -110,6 +110,7 @@ struct ConstraintSolver bool tryDispatch(const FunctionCallConstraint& c, NotNull constraint); bool tryDispatch(const PrimitiveTypeConstraint& c, NotNull constraint); bool tryDispatch(const HasPropConstraint& c, NotNull constraint); + bool tryDispatch(const SetPropConstraint& c, NotNull constraint); bool tryDispatch(const SingletonOrTopTypeConstraint& c, NotNull constraint); // for a, ... in some_table do @@ -120,6 +121,8 @@ struct ConstraintSolver bool tryDispatchIterableFunction( TypeId nextTy, TypeId tableTy, TypeId firstIndexTy, const IterableConstraint& c, NotNull constraint, bool force); + std::optional lookupTableProp(TypeId subjectType, const std::string& propName); + void block(NotNull target, NotNull constraint); /** * Block a constraint on the resolution of a TypeVar. diff --git a/Analysis/src/ConstraintGraphBuilder.cpp b/Analysis/src/ConstraintGraphBuilder.cpp index 42dc07f6..e3572fe8 100644 --- a/Analysis/src/ConstraintGraphBuilder.cpp +++ b/Analysis/src/ConstraintGraphBuilder.cpp @@ -11,6 +11,7 @@ #include "Luau/Scope.h" #include "Luau/ToString.h" #include "Luau/TypeUtils.h" +#include "Luau/TypeVar.h" LUAU_FASTINT(LuauCheckRecursionLimit); LUAU_FASTFLAG(DebugLuauLogSolverToJson); @@ -1019,7 +1020,22 @@ InferencePack ConstraintGraphBuilder::checkPack(const ScopePtr& scope, AstExprCa args.push_back(check(scope, arg).ty); } - // TODO self + if (call->self) + { + AstExprIndexName* indexExpr = call->func->as(); + if (!indexExpr) + ice->ice("method call expression has no 'self'"); + + // The call to `check` we already did on `call->func` should have already produced a type for + // `indexExpr->expr`, so we can get it from `astTypes` to avoid exponential blow-up. + TypeId selfType = astTypes[indexExpr->expr]; + + // If we don't have a type for self, it means we had a code too complex error already. + if (selfType == nullptr) + selfType = singletonTypes->errorRecoveryType(); + + args.insert(args.begin(), selfType); + } if (matchSetmetatable(*call)) { @@ -1428,13 +1444,6 @@ TypePackId ConstraintGraphBuilder::checkLValues(const ScopePtr& scope, AstArray< return arena->addTypePack(std::move(types)); } -static bool isUnsealedTable(TypeId ty) -{ - ty = follow(ty); - const TableTypeVar* ttv = get(ty); - return ttv && ttv->state == TableState::Unsealed; -}; - /** * If the expr is a dotted set of names, and if the root symbol refers to an * unsealed table, return that table type, plus the indeces that follow as a @@ -1468,80 +1477,6 @@ static std::optional>> extractDottedN return std::nullopt; } -/** - * Create a shallow copy of `ty` and its properties along `path`. Insert a new - * property (the last segment of `path`) into the tail table with the value `t`. - * - * On success, returns the new outermost table type. If the root table or any - * of its subkeys are not unsealed tables, the function fails and returns - * std::nullopt. - * - * TODO: Prove that we completely give up in the face of indexers and - * metatables. - */ -static std::optional updateTheTableType(NotNull arena, TypeId ty, const std::vector& path, TypeId replaceTy) -{ - if (path.empty()) - return std::nullopt; - - // First walk the path and ensure that it's unsealed tables all the way - // to the end. - { - TypeId t = ty; - for (size_t i = 0; i < path.size() - 1; ++i) - { - if (!isUnsealedTable(t)) - return std::nullopt; - - const TableTypeVar* tbl = get(t); - auto it = tbl->props.find(path[i]); - if (it == tbl->props.end()) - return std::nullopt; - - t = it->second.type; - } - - // The last path segment should not be a property of the table at all. - // We are not changing property types. We are only admitting this one - // new property to be appended. - if (!isUnsealedTable(t)) - return std::nullopt; - const TableTypeVar* tbl = get(t); - auto it = tbl->props.find(path.back()); - if (it != tbl->props.end()) - return std::nullopt; - } - - const TypeId res = shallowClone(ty, arena); - TypeId t = res; - - for (size_t i = 0; i < path.size() - 1; ++i) - { - const std::string segment = path[i]; - - TableTypeVar* ttv = getMutable(t); - LUAU_ASSERT(ttv); - - auto propIt = ttv->props.find(segment); - if (propIt != ttv->props.end()) - { - LUAU_ASSERT(isUnsealedTable(propIt->second.type)); - t = shallowClone(follow(propIt->second.type), arena); - ttv->props[segment].type = t; - } - else - return std::nullopt; - } - - TableTypeVar* ttv = getMutable(t); - LUAU_ASSERT(ttv); - - const std::string lastSegment = path.back(); - LUAU_ASSERT(0 == ttv->props.count(lastSegment)); - ttv->props[lastSegment] = Property{replaceTy}; - return res; -} - /** * This function is mostly about identifying properties that are being inserted into unsealed tables. * @@ -1559,31 +1494,36 @@ TypeId ConstraintGraphBuilder::checkLValue(const ScopePtr& scope, AstExpr* expr) return checkLValue(scope, &synthetic); } } + else if (!expr->is()) + return check(scope, expr).ty; auto dottedPath = extractDottedName(expr); if (!dottedPath) return check(scope, expr).ty; const auto [sym, segments] = std::move(*dottedPath); - if (!sym.local) - return check(scope, expr).ty; + LUAU_ASSERT(!segments.empty()); auto lookupResult = scope->lookupEx(sym); if (!lookupResult) return check(scope, expr).ty; - const auto [ty, symbolScope] = std::move(*lookupResult); + const auto [subjectType, symbolScope] = std::move(*lookupResult); - TypeId replaceTy = arena->freshType(scope.get()); + TypeId propTy = freshType(scope); - std::optional updatedType = updateTheTableType(arena, ty, segments, replaceTy); - if (!updatedType) - return check(scope, expr).ty; + std::vector segmentStrings(begin(segments), end(segments)); + + TypeId updatedType = arena->addType(BlockedTypeVar{}); + addConstraint(scope, expr->location, SetPropConstraint{updatedType, subjectType, std::move(segmentStrings), propTy}); std::optional def = dfg->getDef(sym); LUAU_ASSERT(def); - symbolScope->bindings[sym].typeId = *updatedType; - symbolScope->dcrRefinements[*def] = *updatedType; - return replaceTy; + symbolScope->bindings[sym].typeId = updatedType; + symbolScope->dcrRefinements[*def] = updatedType; + + astTypes[expr] = propTy; + + return propTy; } Inference ConstraintGraphBuilder::check(const ScopePtr& scope, AstExprTable* expr, std::optional expectedType) diff --git a/Analysis/src/ConstraintSolver.cpp b/Analysis/src/ConstraintSolver.cpp index 533652e2..250e7ae2 100644 --- a/Analysis/src/ConstraintSolver.cpp +++ b/Analysis/src/ConstraintSolver.cpp @@ -2,6 +2,7 @@ #include "Luau/Anyification.h" #include "Luau/ApplyTypeFunction.h" +#include "Luau/Clone.h" #include "Luau/ConstraintSolver.h" #include "Luau/DcrLogger.h" #include "Luau/Instantiation.h" @@ -415,6 +416,8 @@ bool ConstraintSolver::tryDispatch(NotNull constraint, bool fo success = tryDispatch(*fcc, constraint); else if (auto hpc = get(*constraint)) success = tryDispatch(*hpc, constraint); + else if (auto spc = get(*constraint)) + success = tryDispatch(*spc, constraint); else if (auto sottc = get(*constraint)) success = tryDispatch(*sottc, constraint); else @@ -1230,69 +1233,180 @@ bool ConstraintSolver::tryDispatch(const HasPropConstraint& c, NotNull(subjectType)) return block(subjectType, constraint); - TypeId resultType = nullptr; + std::optional resultType = lookupTableProp(subjectType, c.prop); + if (!resultType) + return false; - auto collectParts = [&](auto&& unionOrIntersection) -> std::pair> { - bool blocked = false; + if (isBlocked(*resultType)) + { + block(*resultType, constraint); + return false; + } - std::vector parts; - for (TypeId expectedPart : unionOrIntersection) + asMutable(c.resultType)->ty.emplace(*resultType); + return true; +} + +static bool isUnsealedTable(TypeId ty) +{ + ty = follow(ty); + const TableTypeVar* ttv = get(ty); + return ttv && ttv->state == TableState::Unsealed; +} + +/** + * Create a shallow copy of `ty` and its properties along `path`. Insert a new + * property (the last segment of `path`) into the tail table with the value `t`. + * + * On success, returns the new outermost table type. If the root table or any + * of its subkeys are not unsealed tables, the function fails and returns + * std::nullopt. + * + * TODO: Prove that we completely give up in the face of indexers and + * metatables. + */ +static std::optional updateTheTableType(NotNull arena, TypeId ty, const std::vector& path, TypeId replaceTy) +{ + if (path.empty()) + return std::nullopt; + + // First walk the path and ensure that it's unsealed tables all the way + // to the end. + { + TypeId t = ty; + for (size_t i = 0; i < path.size() - 1; ++i) { - expectedPart = follow(expectedPart); - if (isBlocked(expectedPart) || get(expectedPart)) - { - blocked = true; - block(expectedPart, constraint); - } - else if (const TableTypeVar* ttv = get(follow(expectedPart))) - { - if (auto prop = ttv->props.find(c.prop); prop != ttv->props.end()) - parts.push_back(prop->second.type); - else if (ttv->indexer && maybeString(ttv->indexer->indexType)) - parts.push_back(ttv->indexer->indexResultType); - } + if (!isUnsealedTable(t)) + return std::nullopt; + + const TableTypeVar* tbl = get(t); + auto it = tbl->props.find(path[i]); + if (it == tbl->props.end()) + return std::nullopt; + + t = it->second.type; } - return {blocked, parts}; + // The last path segment should not be a property of the table at all. + // We are not changing property types. We are only admitting this one + // new property to be appended. + if (!isUnsealedTable(t)) + return std::nullopt; + const TableTypeVar* tbl = get(t); + if (0 != tbl->props.count(path.back())) + return std::nullopt; + } + + const TypeId res = shallowClone(ty, arena); + TypeId t = res; + + for (size_t i = 0; i < path.size() - 1; ++i) + { + const std::string segment = path[i]; + + TableTypeVar* ttv = getMutable(t); + LUAU_ASSERT(ttv); + + auto propIt = ttv->props.find(segment); + if (propIt != ttv->props.end()) + { + LUAU_ASSERT(isUnsealedTable(propIt->second.type)); + t = shallowClone(follow(propIt->second.type), arena); + ttv->props[segment].type = t; + } + else + return std::nullopt; + } + + TableTypeVar* ttv = getMutable(t); + LUAU_ASSERT(ttv); + + const std::string lastSegment = path.back(); + LUAU_ASSERT(0 == ttv->props.count(lastSegment)); + ttv->props[lastSegment] = Property{replaceTy}; + return res; +} + +bool ConstraintSolver::tryDispatch(const SetPropConstraint& c, NotNull constraint) +{ + TypeId subjectType = follow(c.subjectType); + + if (isBlocked(subjectType)) + return block(subjectType, constraint); + + std::optional existingPropType = subjectType; + for (const std::string& segment : c.path) + { + ErrorVec e; + std::optional propTy = lookupTableProp(*existingPropType, segment); + if (!propTy) + { + existingPropType = std::nullopt; + break; + } + else if (isBlocked(*propTy)) + return block(*propTy, constraint); + else + existingPropType = follow(*propTy); + } + + auto bind = [](TypeId a, TypeId b) { + asMutable(a)->ty.emplace(b); }; - if (auto ttv = get(subjectType)) + if (existingPropType) { - if (auto prop = ttv->props.find(c.prop); prop != ttv->props.end()) - resultType = prop->second.type; - else if (ttv->indexer && maybeString(ttv->indexer->indexType)) - resultType = ttv->indexer->indexResultType; + unify(c.propType, *existingPropType, constraint->scope); + bind(c.resultType, c.subjectType); + return true; } - else if (auto utv = get(subjectType)) - { - auto [blocked, parts] = collectParts(utv); - if (blocked) - return false; - else if (parts.size() == 1) - resultType = parts[0]; - else if (parts.size() > 1) - resultType = arena->addType(UnionTypeVar{std::move(parts)}); + if (get(subjectType)) + { + TypeId ty = arena->freshType(constraint->scope); + + // Mint a chain of free tables per c.path + for (auto it = rbegin(c.path); it != rend(c.path); ++it) + { + TableTypeVar t{TableState::Free, TypeLevel{}, constraint->scope}; + t.props[*it] = {ty}; + + ty = arena->addType(std::move(t)); + } + + LUAU_ASSERT(ty); + + bind(subjectType, ty); + bind(c.resultType, ty); + return true; + } + else if (auto ttv = getMutable(subjectType)) + { + if (ttv->state == TableState::Free) + { + ttv->props[c.path[0]] = Property{c.propType}; + bind(c.resultType, c.subjectType); + return true; + } + else if (ttv->state == TableState::Unsealed) + { + std::optional augmented = updateTheTableType(NotNull{arena}, subjectType, c.path, c.propType); + bind(c.resultType, augmented.value_or(subjectType)); + return true; + } else - LUAU_ASSERT(false); // parts.size() == 0 + { + bind(c.resultType, subjectType); + return true; + } } - else if (auto itv = get(subjectType)) + else if (get(subjectType) || get(subjectType)) { - auto [blocked, parts] = collectParts(itv); - - if (blocked) - return false; - else if (parts.size() == 1) - resultType = parts[0]; - else if (parts.size() > 1) - resultType = arena->addType(IntersectionTypeVar{std::move(parts)}); - else - LUAU_ASSERT(false); // parts.size() == 0 + bind(c.resultType, subjectType); + return true; } - if (resultType) - asMutable(c.resultType)->ty.emplace(resultType); - + LUAU_ASSERT(0); return true; } @@ -1481,6 +1595,68 @@ bool ConstraintSolver::tryDispatchIterableFunction( return true; } +std::optional ConstraintSolver::lookupTableProp(TypeId subjectType, const std::string& propName) +{ + auto collectParts = [&](auto&& unionOrIntersection) -> std::pair, std::vector> { + std::optional blocked; + + std::vector parts; + for (TypeId expectedPart : unionOrIntersection) + { + expectedPart = follow(expectedPart); + if (isBlocked(expectedPart) || get(expectedPart)) + blocked = expectedPart; + else if (const TableTypeVar* ttv = get(follow(expectedPart))) + { + if (auto prop = ttv->props.find(propName); prop != ttv->props.end()) + parts.push_back(prop->second.type); + else if (ttv->indexer && maybeString(ttv->indexer->indexType)) + parts.push_back(ttv->indexer->indexResultType); + } + } + + return {blocked, parts}; + }; + + std::optional resultType; + + if (auto ttv = get(subjectType)) + { + if (auto prop = ttv->props.find(propName); prop != ttv->props.end()) + resultType = prop->second.type; + else if (ttv->indexer && maybeString(ttv->indexer->indexType)) + resultType = ttv->indexer->indexResultType; + } + else if (auto utv = get(subjectType)) + { + auto [blocked, parts] = collectParts(utv); + + if (blocked) + resultType = *blocked; + else if (parts.size() == 1) + resultType = parts[0]; + else if (parts.size() > 1) + resultType = arena->addType(UnionTypeVar{std::move(parts)}); + else + LUAU_ASSERT(false); // parts.size() == 0 + } + else if (auto itv = get(subjectType)) + { + auto [blocked, parts] = collectParts(itv); + + if (blocked) + resultType = *blocked; + else if (parts.size() == 1) + resultType = parts[0]; + else if (parts.size() > 1) + resultType = arena->addType(IntersectionTypeVar{std::move(parts)}); + else + LUAU_ASSERT(false); // parts.size() == 0 + } + + return resultType; +} + void ConstraintSolver::block_(BlockedConstraintId target, NotNull constraint) { blocked[target].push_back(constraint); diff --git a/Analysis/src/ToString.cpp b/Analysis/src/ToString.cpp index 5fd15cf7..fe5f5690 100644 --- a/Analysis/src/ToString.cpp +++ b/Analysis/src/ToString.cpp @@ -1487,6 +1487,11 @@ std::string toString(const Constraint& constraint, ToStringOptions& opts) { return tos(c.resultType) + " ~ hasProp " + tos(c.subjectType) + ", \"" + c.prop + "\""; } + else if constexpr (std::is_same_v) + { + const std::string pathStr = c.path.size() == 1 ? "\"" + c.path[0] + "\"" : "[\"" + join(c.path, "\", \"") + "\"]"; + return tos(c.resultType) + " ~ setProp " + tos(c.subjectType) + ", " + pathStr + " " + tos(c.propType); + } else if constexpr (std::is_same_v) { std::string result = tos(c.resultType); diff --git a/Analysis/src/TypeChecker2.cpp b/Analysis/src/TypeChecker2.cpp index 03575c40..35493bdb 100644 --- a/Analysis/src/TypeChecker2.cpp +++ b/Analysis/src/TypeChecker2.cpp @@ -857,6 +857,15 @@ struct TypeChecker2 args.head.push_back(argTy); } + if (call->self) + { + AstExprIndexName* indexExpr = call->func->as(); + if (!indexExpr) + ice.ice("method call expression has no 'self'"); + + args.head.insert(args.head.begin(), lookupType(indexExpr->expr)); + } + TypePackId argsTp = arena.addTypePack(args); FunctionTypeVar ftv{argsTp, expectedRetType}; TypeId expectedType = arena.addType(ftv); diff --git a/Analysis/src/TypedAllocator.cpp b/Analysis/src/TypedAllocator.cpp index c95c8eae..133104d3 100644 --- a/Analysis/src/TypedAllocator.cpp +++ b/Analysis/src/TypedAllocator.cpp @@ -17,8 +17,12 @@ const size_t kPageSize = 4096; #include #include +#if defined(__FreeBSD__) && !(_POSIX_C_SOURCE >= 200112L) +const size_t kPageSize = getpagesize(); +#else const size_t kPageSize = sysconf(_SC_PAGESIZE); #endif +#endif #include diff --git a/CodeGen/src/CodeAllocator.cpp b/CodeGen/src/CodeAllocator.cpp index 823df0d8..e1950dbc 100644 --- a/CodeGen/src/CodeAllocator.cpp +++ b/CodeGen/src/CodeAllocator.cpp @@ -20,8 +20,12 @@ const size_t kPageSize = 4096; #include #include +#if defined(__FreeBSD__) && !(_POSIX_C_SOURCE >= 200112L) +const size_t kPageSize = getpagesize(); +#else const size_t kPageSize = sysconf(_SC_PAGESIZE); #endif +#endif static size_t alignToPageSize(size_t size) { diff --git a/Common/include/Luau/ExperimentalFlags.h b/Common/include/Luau/ExperimentalFlags.h index 15db9ea3..41c1af59 100644 --- a/Common/include/Luau/ExperimentalFlags.h +++ b/Common/include/Luau/ExperimentalFlags.h @@ -13,7 +13,6 @@ inline bool isFlagExperimental(const char* flag) static const char* kList[] = { "LuauInterpolatedStringBaseSupport", "LuauInstantiateInSubtyping", // requires some fixes to lua-apps code - "LuauOptionalNextKey", // waiting for a fix to land in lua-apps "LuauTryhardAnd", // waiting for a fix in graphql-lua -> apollo-client-lia -> lua-apps // makes sure we always have at least one entry nullptr, diff --git a/VM/src/lapi.cpp b/VM/src/lapi.cpp index 28307eb9..d2091c6b 100644 --- a/VM/src/lapi.cpp +++ b/VM/src/lapi.cpp @@ -148,7 +148,6 @@ void lua_rawcheckstack(lua_State* L, int size) { luaD_checkstack(L, size); expandstacklimit(L, L->top + size); - return; } void lua_xmove(lua_State* from, lua_State* to, int n) @@ -167,8 +166,6 @@ void lua_xmove(lua_State* from, lua_State* to, int n) from->top = ftop; to->top = ttop + n; - - return; } void lua_xpush(lua_State* from, lua_State* to, int idx) @@ -177,7 +174,6 @@ void lua_xpush(lua_State* from, lua_State* to, int idx) luaC_threadbarrier(to); setobj2s(to, to->top, index2addr(from, idx)); api_incr_top(to); - return; } lua_State* lua_newthread(lua_State* L) @@ -227,7 +223,6 @@ void lua_settop(lua_State* L, int idx) api_check(L, -(idx + 1) <= (L->top - L->base)); L->top += idx + 1; // `subtract' index (index is negative) } - return; } void lua_remove(lua_State* L, int idx) @@ -237,7 +232,6 @@ void lua_remove(lua_State* L, int idx) while (++p < L->top) setobj2s(L, p - 1, p); L->top--; - return; } void lua_insert(lua_State* L, int idx) @@ -248,7 +242,6 @@ void lua_insert(lua_State* L, int idx) for (StkId q = L->top; q > p; q--) setobj2s(L, q, q - 1); setobj2s(L, p, L->top); - return; } void lua_replace(lua_State* L, int idx) @@ -277,7 +270,6 @@ void lua_replace(lua_State* L, int idx) luaC_barrier(L, curr_func(L), L->top - 1); } L->top--; - return; } void lua_pushvalue(lua_State* L, int idx) @@ -286,7 +278,6 @@ void lua_pushvalue(lua_State* L, int idx) StkId o = index2addr(L, idx); setobj2s(L, L->top, o); api_incr_top(L); - return; } /* @@ -570,28 +561,24 @@ void lua_pushnil(lua_State* L) { setnilvalue(L->top); api_incr_top(L); - return; } void lua_pushnumber(lua_State* L, double n) { setnvalue(L->top, n); api_incr_top(L); - return; } void lua_pushinteger(lua_State* L, int n) { setnvalue(L->top, cast_num(n)); api_incr_top(L); - return; } void lua_pushunsigned(lua_State* L, unsigned u) { setnvalue(L->top, cast_num(u)); api_incr_top(L); - return; } #if LUA_VECTOR_SIZE == 4 @@ -599,14 +586,12 @@ void lua_pushvector(lua_State* L, float x, float y, float z, float w) { setvvalue(L->top, x, y, z, w); api_incr_top(L); - return; } #else void lua_pushvector(lua_State* L, float x, float y, float z) { setvvalue(L->top, x, y, z, 0.0f); api_incr_top(L); - return; } #endif @@ -616,7 +601,6 @@ void lua_pushlstring(lua_State* L, const char* s, size_t len) luaC_threadbarrier(L); setsvalue(L, L->top, luaS_newlstr(L, s, len)); api_incr_top(L); - return; } void lua_pushstring(lua_State* L, const char* s) @@ -661,21 +645,18 @@ void lua_pushcclosurek(lua_State* L, lua_CFunction fn, const char* debugname, in setclvalue(L, L->top, cl); LUAU_ASSERT(iswhite(obj2gco(cl))); api_incr_top(L); - return; } void lua_pushboolean(lua_State* L, int b) { setbvalue(L->top, (b != 0)); // ensure that true is 1 api_incr_top(L); - return; } void lua_pushlightuserdata(lua_State* L, void* p) { setpvalue(L->top, p); api_incr_top(L); - return; } int lua_pushthread(lua_State* L) @@ -748,7 +729,6 @@ void lua_createtable(lua_State* L, int narray, int nrec) luaC_threadbarrier(L); sethvalue(L, L->top, luaH_new(L, narray, nrec)); api_incr_top(L); - return; } void lua_setreadonly(lua_State* L, int objindex, int enabled) @@ -758,7 +738,6 @@ void lua_setreadonly(lua_State* L, int objindex, int enabled) Table* t = hvalue(o); api_check(L, t != hvalue(registry(L))); t->readonly = bool(enabled); - return; } int lua_getreadonly(lua_State* L, int objindex) @@ -776,7 +755,6 @@ void lua_setsafeenv(lua_State* L, int objindex, int enabled) api_check(L, ttistable(o)); Table* t = hvalue(o); t->safeenv = bool(enabled); - return; } int lua_getmetatable(lua_State* L, int objindex) @@ -822,7 +800,6 @@ void lua_getfenv(lua_State* L, int idx) break; } api_incr_top(L); - return; } /* @@ -836,7 +813,6 @@ void lua_settable(lua_State* L, int idx) api_checkvalidindex(L, t); luaV_settable(L, t, L->top - 2, L->top - 1); L->top -= 2; // pop index and value - return; } void lua_setfield(lua_State* L, int idx, const char* k) @@ -848,7 +824,6 @@ void lua_setfield(lua_State* L, int idx, const char* k) setsvalue(L, &key, luaS_new(L, k)); luaV_settable(L, t, &key, L->top - 1); L->top--; - return; } void lua_rawsetfield(lua_State* L, int idx, const char* k) @@ -861,7 +836,6 @@ void lua_rawsetfield(lua_State* L, int idx, const char* k) setobj2t(L, luaH_setstr(L, hvalue(t), luaS_new(L, k)), L->top - 1); luaC_barriert(L, hvalue(t), L->top - 1); L->top--; - return; } void lua_rawset(lua_State* L, int idx) @@ -874,7 +848,6 @@ void lua_rawset(lua_State* L, int idx) setobj2t(L, luaH_set(L, hvalue(t), L->top - 2), L->top - 1); luaC_barriert(L, hvalue(t), L->top - 1); L->top -= 2; - return; } void lua_rawseti(lua_State* L, int idx, int n) @@ -887,7 +860,6 @@ void lua_rawseti(lua_State* L, int idx, int n) setobj2t(L, luaH_setnum(L, hvalue(o), n), L->top - 1); luaC_barriert(L, hvalue(o), L->top - 1); L->top--; - return; } int lua_setmetatable(lua_State* L, int objindex) @@ -979,7 +951,6 @@ void lua_call(lua_State* L, int nargs, int nresults) luaD_call(L, func, nresults); adjustresults(L, nresults); - return; } /* @@ -995,7 +966,6 @@ static void f_call(lua_State* L, void* ud) { struct CallS* c = cast_to(struct CallS*, ud); luaD_call(L, c->func, c->nresults); - return; } int lua_pcall(lua_State* L, int nargs, int nresults, int errfunc) @@ -1273,7 +1243,6 @@ void lua_concat(lua_State* L, int n) api_incr_top(L); } // else n == 1; nothing to do - return; } void* lua_newuserdatatagged(lua_State* L, size_t sz, int tag) @@ -1397,7 +1366,6 @@ void lua_unref(lua_State* L, int ref) TValue* slot = luaH_setnum(L, reg, ref); setnvalue(slot, g->registryfree); // NB: no barrier needed because value isn't collectable g->registryfree = ref; - return; } void lua_setuserdatatag(lua_State* L, int idx, int tag) diff --git a/tests/TypeInfer.tables.test.cpp b/tests/TypeInfer.tables.test.cpp index bf0a0af6..10186e3a 100644 --- a/tests/TypeInfer.tables.test.cpp +++ b/tests/TypeInfer.tables.test.cpp @@ -50,7 +50,7 @@ TEST_CASE_FIXTURE(Fixture, "augment_table") const TableTypeVar* tType = get(requireType("t")); REQUIRE(tType != nullptr); - CHECK(1 == tType->props.count("foo")); + CHECK("{ foo: string }" == toString(requireType("t"), {true})); } TEST_CASE_FIXTURE(Fixture, "augment_nested_table") @@ -65,7 +65,7 @@ TEST_CASE_FIXTURE(Fixture, "augment_nested_table") const TableTypeVar* pType = get(tType->props["p"].type); REQUIRE(pType != nullptr); - CHECK(pType->props.find("foo") != pType->props.end()); + CHECK("{ p: { foo: string } }" == toString(requireType("t"), {true})); } TEST_CASE_FIXTURE(Fixture, "cannot_augment_sealed_table") diff --git a/tools/faillist.txt b/tools/faillist.txt index a228c171..433d0cfb 100644 --- a/tools/faillist.txt +++ b/tools/faillist.txt @@ -51,7 +51,6 @@ BuiltinTests.dont_add_definitions_to_persistent_types BuiltinTests.find_capture_types BuiltinTests.find_capture_types2 BuiltinTests.find_capture_types3 -BuiltinTests.gmatch_capture_types2 BuiltinTests.gmatch_capture_types_balanced_escaped_parens BuiltinTests.gmatch_capture_types_default_capture BuiltinTests.gmatch_capture_types_parens_in_sets_are_ignored @@ -73,9 +72,7 @@ BuiltinTests.string_format_arg_count_mismatch BuiltinTests.string_format_as_method BuiltinTests.string_format_correctly_ordered_types BuiltinTests.string_format_report_all_type_errors_at_correct_positions -BuiltinTests.string_format_use_correct_argument BuiltinTests.string_format_use_correct_argument2 -BuiltinTests.strings_have_methods BuiltinTests.table_freeze_is_generic BuiltinTests.table_insert_correctly_infers_type_of_array_2_args_overload BuiltinTests.table_insert_correctly_infers_type_of_array_3_args_overload @@ -115,7 +112,6 @@ GenericsTests.infer_generic_function_function_argument GenericsTests.infer_generic_function_function_argument_overloaded GenericsTests.infer_generic_methods GenericsTests.infer_generic_property -GenericsTests.instantiate_cyclic_generic_function GenericsTests.instantiated_function_argument_names GenericsTests.instantiation_sharing_types GenericsTests.no_stack_overflow_from_quantifying @@ -198,7 +194,6 @@ RefinementTest.x_as_any_if_x_is_instance_elseif_x_is_table RefinementTest.x_is_not_instance_or_else_not_part RuntimeLimits.typescript_port_of_Result_type TableTests.a_free_shape_can_turn_into_a_scalar_directly -TableTests.a_free_shape_can_turn_into_a_scalar_if_it_is_compatible TableTests.a_free_shape_cannot_turn_into_a_scalar_if_it_is_not_compatible TableTests.access_index_metamethod_that_returns_variadic TableTests.accidentally_checked_prop_in_opposite_branch @@ -269,7 +264,6 @@ TableTests.reasonable_error_when_adding_a_nonexistent_property_to_an_array_like_ TableTests.result_is_always_any_if_lhs_is_any TableTests.result_is_bool_for_equality_operators_if_lhs_is_any TableTests.right_table_missing_key2 -TableTests.scalar_is_a_subtype_of_a_compatible_polymorphic_shape_type TableTests.scalar_is_not_a_subtype_of_a_compatible_polymorphic_shape_type TableTests.shared_selfs TableTests.shared_selfs_from_free_param @@ -285,7 +279,6 @@ TableTests.table_subtyping_with_missing_props_dont_report_multiple_errors TableTests.tables_get_names_from_their_locals TableTests.tc_member_function TableTests.tc_member_function_2 -TableTests.type_mismatch_on_massive_table_is_cut_short TableTests.unification_of_unions_in_a_self_referential_type TableTests.unifying_tables_shouldnt_uaf2 TableTests.used_colon_instead_of_dot @@ -343,8 +336,6 @@ TypeInfer.type_infer_recursion_limit_no_ice TypeInfer.type_infer_recursion_limit_normalizer TypeInferAnyError.for_in_loop_iterator_is_any2 TypeInferAnyError.for_in_loop_iterator_is_error2 -TypeInferClasses.call_base_method -TypeInferClasses.call_instance_method TypeInferClasses.can_read_prop_of_base_class_using_string TypeInferClasses.class_type_mismatch_with_name_conflict TypeInferClasses.classes_without_overloaded_operators_cannot_be_added @@ -429,10 +420,7 @@ TypeInferOperators.typecheck_overloaded_multiply_that_is_an_intersection TypeInferOperators.typecheck_overloaded_multiply_that_is_an_intersection_on_rhs TypeInferOperators.UnknownGlobalCompoundAssign TypeInferPrimitives.CheckMethodsOfNumber -TypeInferPrimitives.singleton_types -TypeInferPrimitives.string_function_other TypeInferPrimitives.string_index -TypeInferPrimitives.string_method TypeInferUnknownNever.assign_to_global_which_is_never TypeInferUnknownNever.assign_to_local_which_is_never TypeInferUnknownNever.assign_to_prop_which_is_never From 7dbe47f4ddd80a513a5bca8856a04fea17ff14b1 Mon Sep 17 00:00:00 2001 From: JohnnyMorganz Date: Mon, 28 Nov 2022 13:25:44 +0000 Subject: [PATCH 393/399] Handle cyclically referenced declared classes (#729) Closes #631 Performs a "two pass" approach, where we first call `prototype` on the class definition. The prototype phase checks for errors and creates a base CTV for the class, which other types can then reference. The second `check` phase fills it out with all the defined properties/methods Co-authored-by: vegorov-rbx <75688451+vegorov-rbx@users.noreply.github.com> --- Analysis/include/Luau/TypeInfer.h | 6 + Analysis/src/TypeInfer.cpp | 204 +++++++++++++++++++++------ tests/TypeInfer.definitions.test.cpp | 22 +++ 3 files changed, 191 insertions(+), 41 deletions(-) diff --git a/Analysis/include/Luau/TypeInfer.h b/Analysis/include/Luau/TypeInfer.h index 4eaa5969..2eea191a 100644 --- a/Analysis/include/Luau/TypeInfer.h +++ b/Analysis/include/Luau/TypeInfer.h @@ -95,6 +95,7 @@ struct TypeChecker void check(const ScopePtr& scope, const AstStatDeclareFunction& declaredFunction); void prototype(const ScopePtr& scope, const AstStatTypeAlias& typealias, int subLevel = 0); + void prototype(const ScopePtr& scope, const AstStatDeclareClass& declaredClass); void checkBlock(const ScopePtr& scope, const AstStatBlock& statement); void checkBlockWithoutRecursionCheck(const ScopePtr& scope, const AstStatBlock& statement); @@ -399,6 +400,11 @@ private: */ DenseHashSet, HashBoolNamePair> duplicateTypeAliases; + /** + * A set of incorrect class definitions which is used to avoid a second-pass analysis. + */ + DenseHashSet incorrectClassDefinitions{nullptr}; + std::vector> deferredQuantification; }; diff --git a/Analysis/src/TypeInfer.cpp b/Analysis/src/TypeInfer.cpp index 8ecd45bd..0ca54e60 100644 --- a/Analysis/src/TypeInfer.cpp +++ b/Analysis/src/TypeInfer.cpp @@ -49,6 +49,7 @@ LUAU_FASTFLAGVARIABLE(LuauReportShadowedTypeAlias, false) LUAU_FASTFLAGVARIABLE(LuauBetterMessagingOnCountMismatch, false) LUAU_FASTFLAGVARIABLE(LuauArgMismatchReportFunctionLocation, false) LUAU_FASTFLAGVARIABLE(LuauImplicitElseRefinement, false) +LUAU_FASTFLAGVARIABLE(LuauDeclareClassPrototype, false) namespace Luau { @@ -356,6 +357,7 @@ ModulePtr TypeChecker::checkWithoutRecursionCheck(const SourceModule& module, Mo unifierState.skipCacheForType.clear(); duplicateTypeAliases.clear(); + incorrectClassDefinitions.clear(); return std::move(currentModule); } @@ -523,6 +525,10 @@ void TypeChecker::checkBlockWithoutRecursionCheck(const ScopePtr& scope, const A prototype(scope, *typealias, subLevel); ++subLevel; } + else if (const auto& declaredClass = stat->as(); FFlag::LuauDeclareClassPrototype && declaredClass) + { + prototype(scope, *declaredClass); + } } auto protoIter = sorted.begin(); @@ -1670,8 +1676,10 @@ void TypeChecker::prototype(const ScopePtr& scope, const AstStatTypeAlias& typea } } -void TypeChecker::check(const ScopePtr& scope, const AstStatDeclareClass& declaredClass) +void TypeChecker::prototype(const ScopePtr& scope, const AstStatDeclareClass& declaredClass) { + LUAU_ASSERT(FFlag::LuauDeclareClassPrototype); + std::optional superTy = std::nullopt; if (declaredClass.superName) { @@ -1681,6 +1689,7 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatDeclareClass& declar if (!lookupType) { reportError(declaredClass.location, UnknownSymbol{superName, UnknownSymbol::Type}); + incorrectClassDefinitions.insert(&declaredClass); return; } @@ -1692,7 +1701,7 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatDeclareClass& declar { reportError(declaredClass.location, GenericError{format("Cannot use non-class type '%s' as a superclass of class '%s'", superName.c_str(), declaredClass.name.value)}); - + incorrectClassDefinitions.insert(&declaredClass); return; } } @@ -1701,61 +1710,174 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatDeclareClass& declar TypeId classTy = addType(ClassTypeVar(className, {}, superTy, std::nullopt, {}, {}, currentModuleName)); ClassTypeVar* ctv = getMutable(classTy); - TypeId metaTy = addType(TableTypeVar{TableState::Sealed, scope->level}); - TableTypeVar* metatable = getMutable(metaTy); ctv->metatable = metaTy; - scope->exportedTypeBindings[className] = TypeFun{{}, classTy}; +} - for (const AstDeclaredClassProp& prop : declaredClass.props) +void TypeChecker::check(const ScopePtr& scope, const AstStatDeclareClass& declaredClass) +{ + if (FFlag::LuauDeclareClassPrototype) { - Name propName(prop.name.value); - TypeId propTy = resolveType(scope, *prop.ty); + Name className(declaredClass.name.value); - bool assignToMetatable = isMetamethod(propName); - Luau::ClassTypeVar::Props& assignTo = assignToMetatable ? metatable->props : ctv->props; + // Don't bother checking if the class definition was incorrect + if (incorrectClassDefinitions.find(&declaredClass)) + return; - // Function types always take 'self', but this isn't reflected in the - // parsed annotation. Add it here. - if (prop.isMethod) + std::optional binding; + if (auto it = scope->exportedTypeBindings.find(className); it != scope->exportedTypeBindings.end()) + binding = it->second; + + // This class definition must have been `prototype()`d first. + if (!binding) + ice("Class not predeclared"); + + TypeId classTy = binding->type; + ClassTypeVar* ctv = getMutable(classTy); + + if (!ctv->metatable) + ice("No metatable for declared class"); + + TableTypeVar* metatable = getMutable(*ctv->metatable); + for (const AstDeclaredClassProp& prop : declaredClass.props) { - if (FunctionTypeVar* ftv = getMutable(propTy)) + Name propName(prop.name.value); + TypeId propTy = resolveType(scope, *prop.ty); + + bool assignToMetatable = isMetamethod(propName); + Luau::ClassTypeVar::Props& assignTo = assignToMetatable ? metatable->props : ctv->props; + + // Function types always take 'self', but this isn't reflected in the + // parsed annotation. Add it here. + if (prop.isMethod) { - ftv->argNames.insert(ftv->argNames.begin(), FunctionArgument{"self", {}}); - ftv->argTypes = addTypePack(TypePack{{classTy}, ftv->argTypes}); - ftv->hasSelf = true; + if (FunctionTypeVar* ftv = getMutable(propTy)) + { + ftv->argNames.insert(ftv->argNames.begin(), FunctionArgument{"self", {}}); + ftv->argTypes = addTypePack(TypePack{{classTy}, ftv->argTypes}); + ftv->hasSelf = true; + } } - } - if (assignTo.count(propName) == 0) - { - assignTo[propName] = {propTy}; - } - else - { - TypeId currentTy = assignTo[propName].type; - - // We special-case this logic to keep the intersection flat; otherwise we - // would create a ton of nested intersection types. - if (const IntersectionTypeVar* itv = get(currentTy)) + if (assignTo.count(propName) == 0) { - std::vector options = itv->parts; - options.push_back(propTy); - TypeId newItv = addType(IntersectionTypeVar{std::move(options)}); - - assignTo[propName] = {newItv}; - } - else if (get(currentTy)) - { - TypeId intersection = addType(IntersectionTypeVar{{currentTy, propTy}}); - - assignTo[propName] = {intersection}; + assignTo[propName] = {propTy}; } else { - reportError(declaredClass.location, GenericError{format("Cannot overload non-function class member '%s'", propName.c_str())}); + TypeId currentTy = assignTo[propName].type; + + // We special-case this logic to keep the intersection flat; otherwise we + // would create a ton of nested intersection types. + if (const IntersectionTypeVar* itv = get(currentTy)) + { + std::vector options = itv->parts; + options.push_back(propTy); + TypeId newItv = addType(IntersectionTypeVar{std::move(options)}); + + assignTo[propName] = {newItv}; + } + else if (get(currentTy)) + { + TypeId intersection = addType(IntersectionTypeVar{{currentTy, propTy}}); + + assignTo[propName] = {intersection}; + } + else + { + reportError(declaredClass.location, GenericError{format("Cannot overload non-function class member '%s'", propName.c_str())}); + } + } + } + } + else + { + std::optional superTy = std::nullopt; + if (declaredClass.superName) + { + Name superName = Name(declaredClass.superName->value); + std::optional lookupType = scope->lookupType(superName); + + if (!lookupType) + { + reportError(declaredClass.location, UnknownSymbol{superName, UnknownSymbol::Type}); + return; + } + + // We don't have generic classes, so this assertion _should_ never be hit. + LUAU_ASSERT(lookupType->typeParams.size() == 0 && lookupType->typePackParams.size() == 0); + superTy = lookupType->type; + + if (!get(follow(*superTy))) + { + reportError(declaredClass.location, GenericError{format("Cannot use non-class type '%s' as a superclass of class '%s'", + superName.c_str(), declaredClass.name.value)}); + return; + } + } + + Name className(declaredClass.name.value); + + TypeId classTy = addType(ClassTypeVar(className, {}, superTy, std::nullopt, {}, {}, currentModuleName)); + + ClassTypeVar* ctv = getMutable(classTy); + TypeId metaTy = addType(TableTypeVar{TableState::Sealed, scope->level}); + TableTypeVar* metatable = getMutable(metaTy); + + ctv->metatable = metaTy; + + scope->exportedTypeBindings[className] = TypeFun{{}, classTy}; + + for (const AstDeclaredClassProp& prop : declaredClass.props) + { + Name propName(prop.name.value); + TypeId propTy = resolveType(scope, *prop.ty); + + bool assignToMetatable = isMetamethod(propName); + Luau::ClassTypeVar::Props& assignTo = assignToMetatable ? metatable->props : ctv->props; + + // Function types always take 'self', but this isn't reflected in the + // parsed annotation. Add it here. + if (prop.isMethod) + { + if (FunctionTypeVar* ftv = getMutable(propTy)) + { + ftv->argNames.insert(ftv->argNames.begin(), FunctionArgument{"self", {}}); + ftv->argTypes = addTypePack(TypePack{{classTy}, ftv->argTypes}); + ftv->hasSelf = true; + } + } + + if (assignTo.count(propName) == 0) + { + assignTo[propName] = {propTy}; + } + else + { + TypeId currentTy = assignTo[propName].type; + + // We special-case this logic to keep the intersection flat; otherwise we + // would create a ton of nested intersection types. + if (const IntersectionTypeVar* itv = get(currentTy)) + { + std::vector options = itv->parts; + options.push_back(propTy); + TypeId newItv = addType(IntersectionTypeVar{std::move(options)}); + + assignTo[propName] = {newItv}; + } + else if (get(currentTy)) + { + TypeId intersection = addType(IntersectionTypeVar{{currentTy, propTy}}); + + assignTo[propName] = {intersection}; + } + else + { + reportError(declaredClass.location, GenericError{format("Cannot overload non-function class member '%s'", propName.c_str())}); + } } } } diff --git a/tests/TypeInfer.definitions.test.cpp b/tests/TypeInfer.definitions.test.cpp index 684b47e9..3556f0f1 100644 --- a/tests/TypeInfer.definitions.test.cpp +++ b/tests/TypeInfer.definitions.test.cpp @@ -396,4 +396,26 @@ TEST_CASE_FIXTURE(Fixture, "class_definition_string_props") CHECK_EQ(toString(requireType("y")), "string"); } +TEST_CASE_FIXTURE(Fixture, "class_definitions_reference_other_classes") +{ + ScopedFastFlag LuauDeclareClassPrototype("LuauDeclareClassPrototype", true); + + unfreeze(typeChecker.globalTypes); + LoadDefinitionFileResult result = loadDefinitionFile(typeChecker, typeChecker.globalScope, R"( + declare class Channel + Messages: { Message } + OnMessage: (message: Message) -> () + end + + declare class Message + Text: string + Channel: Channel + end + )", + "@test"); + freeze(typeChecker.globalTypes); + + REQUIRE(result.success); +} + TEST_SUITE_END(); From 9095fc4b83ea40c72980a539b6c9c06136d67895 Mon Sep 17 00:00:00 2001 From: JohnnyMorganz Date: Mon, 28 Nov 2022 18:02:41 +0000 Subject: [PATCH 394/399] Support `__call` on class type vars (#762) Currently, the metatable of a class type var is not correctly checked to see if it is callable (i.e. has a `__call` metatable). We resolve this issue by checking the metatable in `checkCallOverload` Fixes #756 Co-authored-by: vegorov-rbx <75688451+vegorov-rbx@users.noreply.github.com> --- Analysis/src/TypeInfer.cpp | 36 +++++++++++++++++++------------- tests/TypeInfer.classes.test.cpp | 20 ++++++++++++++++++ 2 files changed, 42 insertions(+), 14 deletions(-) diff --git a/Analysis/src/TypeInfer.cpp b/Analysis/src/TypeInfer.cpp index 0ca54e60..dfd08e54 100644 --- a/Analysis/src/TypeInfer.cpp +++ b/Analysis/src/TypeInfer.cpp @@ -50,6 +50,7 @@ LUAU_FASTFLAGVARIABLE(LuauBetterMessagingOnCountMismatch, false) LUAU_FASTFLAGVARIABLE(LuauArgMismatchReportFunctionLocation, false) LUAU_FASTFLAGVARIABLE(LuauImplicitElseRefinement, false) LUAU_FASTFLAGVARIABLE(LuauDeclareClassPrototype, false) +LUAU_FASTFLAGVARIABLE(LuauCallableClasses, false) namespace Luau { @@ -4257,26 +4258,33 @@ std::optional> TypeChecker::checkCallOverload(const Sc std::vector metaArgLocations; - // Might be a callable table + // Might be a callable table or class + std::optional callTy = std::nullopt; if (const MetatableTypeVar* mttv = get(fn)) { - if (std::optional ty = getIndexTypeFromType(scope, mttv->metatable, "__call", expr.func->location, /* addErrors= */ false)) - { - // Construct arguments with 'self' added in front - TypePackId metaCallArgPack = addTypePack(TypePackVar(TypePack{args->head, args->tail})); + callTy = getIndexTypeFromType(scope, mttv->metatable, "__call", expr.func->location, /* addErrors= */ false); + } + else if (const ClassTypeVar* ctv = get(fn); FFlag::LuauCallableClasses && ctv && ctv->metatable) + { + callTy = getIndexTypeFromType(scope, *ctv->metatable, "__call", expr.func->location, /* addErrors= */ false); + } - TypePack* metaCallArgs = getMutable(metaCallArgPack); - metaCallArgs->head.insert(metaCallArgs->head.begin(), fn); + if (callTy) + { + // Construct arguments with 'self' added in front + TypePackId metaCallArgPack = addTypePack(TypePackVar(TypePack{args->head, args->tail})); - metaArgLocations = *argLocations; - metaArgLocations.insert(metaArgLocations.begin(), expr.func->location); + TypePack* metaCallArgs = getMutable(metaCallArgPack); + metaCallArgs->head.insert(metaCallArgs->head.begin(), fn); - fn = instantiate(scope, *ty, expr.func->location); + metaArgLocations = *argLocations; + metaArgLocations.insert(metaArgLocations.begin(), expr.func->location); - argPack = metaCallArgPack; - args = metaCallArgs; - argLocations = &metaArgLocations; - } + fn = instantiate(scope, *callTy, expr.func->location); + + argPack = metaCallArgPack; + args = metaCallArgs; + argLocations = &metaArgLocations; } const FunctionTypeVar* ftv = get(fn); diff --git a/tests/TypeInfer.classes.test.cpp b/tests/TypeInfer.classes.test.cpp index d00f1d83..d1a24e5f 100644 --- a/tests/TypeInfer.classes.test.cpp +++ b/tests/TypeInfer.classes.test.cpp @@ -91,6 +91,13 @@ struct ClassFixture : BuiltinsFixture typeChecker.globalScope->exportedTypeBindings["Vector2"] = TypeFun{{}, vector2InstanceType}; addGlobalBinding(frontend, "Vector2", vector2Type, "@test"); + TypeId callableClassMetaType = arena.addType(TableTypeVar{}); + TypeId callableClassType = arena.addType(ClassTypeVar{"CallableClass", {}, nullopt, callableClassMetaType, {}, {}, "Test"}); + getMutable(callableClassMetaType)->props = { + {"__call", {makeFunction(arena, nullopt, {callableClassType, typeChecker.stringType}, {typeChecker.numberType})}}, + }; + typeChecker.globalScope->exportedTypeBindings["CallableClass"] = TypeFun{{}, callableClassType}; + for (const auto& [name, tf] : typeChecker.globalScope->exportedTypeBindings) persist(tf.type); @@ -514,4 +521,17 @@ TEST_CASE_FIXTURE(ClassFixture, "unions_of_intersections_of_classes") LUAU_REQUIRE_NO_ERRORS(result); } +TEST_CASE_FIXTURE(ClassFixture, "callable_classes") +{ + ScopedFastFlag luauCallableClasses{"LuauCallableClasses", true}; + + CheckResult result = check(R"( + local x : CallableClass + local y = x("testing") + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + CHECK_EQ("number", toString(requireType("y"))); +} + TEST_SUITE_END(); From 91302d1d4cf01b052b103d9d281567250e532813 Mon Sep 17 00:00:00 2001 From: vegorov-rbx <75688451+vegorov-rbx@users.noreply.github.com> Date: Fri, 2 Dec 2022 10:08:55 -0800 Subject: [PATCH 395/399] Fix coverage github action by using clang++ without a specific version (#769) Github fails to find clang++-10. Since we install llvm without a specific version number, we shouldn't use clang++-10 with a version number. I tried installing llvm-10, but that package is not available on github (any more?). --- .github/workflows/build.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml index dbd6a495..5145e76b 100644 --- a/.github/workflows/build.yml +++ b/.github/workflows/build.yml @@ -89,7 +89,7 @@ jobs: sudo apt install llvm - name: make coverage run: | - CXX=clang++-10 make -j2 config=coverage native=1 coverage + CXX=clang++ make -j2 config=coverage native=1 coverage - name: upload coverage uses: codecov/codecov-action@v3 with: From 59ae47db43967fe65c7dc188e7a4d60212b6105d Mon Sep 17 00:00:00 2001 From: vegorov-rbx <75688451+vegorov-rbx@users.noreply.github.com> Date: Fri, 2 Dec 2022 10:09:59 -0800 Subject: [PATCH 396/399] Sync to upstream/release/555 (#768) * Type mismatch errors now mention if unification failed in covariant or invariant context, to explain why sometimes derived class can't be converted to base class or why `T` can't be converted into `T?` and so on * Class type indexing is no longer an error in non-strict mode (still an error in strict mode) * Fixed cyclic type packs not being displayed in the type * Added an error when unrelated types are compared with `==`/`~=` * Fixed false positive errors involving sub-type tests an `never` type * Fixed miscompilation of multiple assignment statements (Fixes https://github.com/Roblox/luau/issues/754) * Type inference stability improvements --- Analysis/include/Luau/BuiltinDefinitions.h | 1 + Analysis/include/Luau/Connective.h | 4 +- Analysis/include/Luau/Constraint.h | 6 +- .../include/Luau/ConstraintGraphBuilder.h | 4 +- ...DataFlowGraphBuilder.h => DataFlowGraph.h} | 5 + Analysis/include/Luau/Def.h | 53 +- Analysis/include/Luau/Error.h | 46 +- Analysis/include/Luau/Normalize.h | 6 + Analysis/include/Luau/NotNull.h | 14 + Analysis/include/Luau/RecursionCounter.h | 19 +- Analysis/include/Luau/ToString.h | 4 +- Analysis/include/Luau/TxnLog.h | 2 + Analysis/include/Luau/TypeInfer.h | 7 - Analysis/include/Luau/TypeUtils.h | 6 +- Analysis/include/Luau/TypeVar.h | 31 +- Analysis/include/Luau/Unifier.h | 4 + Analysis/src/AstQuery.cpp | 54 +-- Analysis/src/Autocomplete.cpp | 3 +- Analysis/src/BuiltinDefinitions.cpp | 8 + Analysis/src/ConstraintGraphBuilder.cpp | 459 +++++++++++++----- Analysis/src/ConstraintSolver.cpp | 27 +- ...FlowGraphBuilder.cpp => DataFlowGraph.cpp} | 37 +- Analysis/src/Def.cpp | 9 +- Analysis/src/Error.cpp | 85 ++-- Analysis/src/Frontend.cpp | 139 ++---- Analysis/src/IostreamHelpers.cpp | 2 + Analysis/src/Normalize.cpp | 71 ++- Analysis/src/ToString.cpp | 141 +++--- Analysis/src/TopoSortStatements.cpp | 4 +- Analysis/src/TxnLog.cpp | 36 ++ Analysis/src/TypeAttach.cpp | 2 +- Analysis/src/TypeChecker2.cpp | 120 +++-- Analysis/src/TypeInfer.cpp | 173 ++++--- Analysis/src/TypePack.cpp | 8 +- Analysis/src/TypeUtils.cpp | 120 +++-- Analysis/src/TypeVar.cpp | 30 +- Analysis/src/TypedAllocator.cpp | 4 + Analysis/src/Unifier.cpp | 332 +++++++++---- Ast/include/Luau/Location.h | 30 ++ Ast/src/Parser.cpp | 15 +- Compiler/src/BytecodeBuilder.cpp | 5 +- Compiler/src/Compiler.cpp | 45 +- Sources.cmake | 15 +- VM/src/loslib.cpp | 15 + tests/AstQuery.test.cpp | 4 - tests/ClassFixture.cpp | 113 +++++ tests/ClassFixture.h | 13 + tests/Compiler.test.cpp | 15 + ...uilder.test.cpp => DataFlowGraph.test.cpp} | 2 +- tests/Module.test.cpp | 10 +- tests/NotNull.test.cpp | 2 + tests/Parser.test.cpp | 8 - tests/ToString.test.cpp | 8 +- tests/TypeInfer.aliases.test.cpp | 31 +- tests/TypeInfer.annotations.test.cpp | 2 - tests/TypeInfer.builtins.test.cpp | 32 +- tests/TypeInfer.classes.test.cpp | 147 ++---- tests/TypeInfer.definitions.test.cpp | 2 - tests/TypeInfer.functions.test.cpp | 20 +- tests/TypeInfer.generics.test.cpp | 34 +- tests/TypeInfer.intersectionTypes.test.cpp | 55 ++- tests/TypeInfer.modules.test.cpp | 26 +- tests/TypeInfer.operators.test.cpp | 41 ++ tests/TypeInfer.provisional.test.cpp | 20 +- tests/TypeInfer.refinements.test.cpp | 246 +++++++++- tests/TypeInfer.tables.test.cpp | 85 +++- tests/TypeInfer.test.cpp | 15 +- tests/TypeInfer.tryUnify.test.cpp | 77 +++ tests/TypeInfer.typePacks.cpp | 57 ++- tests/TypeInfer.unionTypes.test.cpp | 33 ++ tests/VisitTypeVar.test.cpp | 9 +- tools/faillist.txt | 41 +- 72 files changed, 2313 insertions(+), 1036 deletions(-) rename Analysis/include/Luau/{DataFlowGraphBuilder.h => DataFlowGraph.h} (91%) rename Analysis/src/{DataFlowGraphBuilder.cpp => DataFlowGraph.cpp} (92%) create mode 100644 tests/ClassFixture.cpp create mode 100644 tests/ClassFixture.h rename tests/{DataFlowGraphBuilder.test.cpp => DataFlowGraph.test.cpp} (98%) diff --git a/Analysis/include/Luau/BuiltinDefinitions.h b/Analysis/include/Luau/BuiltinDefinitions.h index 616367bb..4702995d 100644 --- a/Analysis/include/Luau/BuiltinDefinitions.h +++ b/Analysis/include/Luau/BuiltinDefinitions.h @@ -40,6 +40,7 @@ TypeId makeFunction( // Polymorphic void attachMagicFunction(TypeId ty, MagicFunction fn); void attachDcrMagicFunction(TypeId ty, DcrMagicFunction fn); +void attachDcrMagicRefinement(TypeId ty, DcrMagicRefinement fn); Property makeProperty(TypeId ty, std::optional documentationSymbol = std::nullopt); void assignPropDocumentationSymbols(TableTypeVar::Props& props, const std::string& baseName); diff --git a/Analysis/include/Luau/Connective.h b/Analysis/include/Luau/Connective.h index c9daa0f9..4a6be93c 100644 --- a/Analysis/include/Luau/Connective.h +++ b/Analysis/include/Luau/Connective.h @@ -3,7 +3,6 @@ #include "Luau/Def.h" #include "Luau/TypedAllocator.h" -#include "Luau/TypeVar.h" #include "Luau/Variant.h" #include @@ -11,6 +10,9 @@ namespace Luau { +struct TypeVar; +using TypeId = const TypeVar*; + struct Negation; struct Conjunction; struct Disjunction; diff --git a/Analysis/include/Luau/Constraint.h b/Analysis/include/Luau/Constraint.h index 16a08e87..e13613ed 100644 --- a/Analysis/include/Luau/Constraint.h +++ b/Analysis/include/Luau/Constraint.h @@ -149,11 +149,15 @@ struct SetPropConstraint TypeId propType; }; -// result ~ if isSingleton D then ~D else unknown where D = discriminantType +// if negation: +// result ~ if isSingleton D then ~D else unknown where D = discriminantType +// if not negation: +// result ~ if isSingleton D then D else unknown where D = discriminantType struct SingletonOrTopTypeConstraint { TypeId resultType; TypeId discriminantType; + bool negated; }; using ConstraintV = Variant expectedType = {}); /** * Checks the body of a function expression. diff --git a/Analysis/include/Luau/DataFlowGraphBuilder.h b/Analysis/include/Luau/DataFlowGraph.h similarity index 91% rename from Analysis/include/Luau/DataFlowGraphBuilder.h rename to Analysis/include/Luau/DataFlowGraph.h index 3a72403e..bd096ea9 100644 --- a/Analysis/include/Luau/DataFlowGraphBuilder.h +++ b/Analysis/include/Luau/DataFlowGraph.h @@ -69,9 +69,14 @@ private: struct InternalErrorReporter* handle; std::vector> scopes; + // Does not belong in DataFlowGraphBuilder, but the old solver allows properties to escape the scope they were defined in, + // so we will need to be able to emulate this same behavior here too. We can kill this once we have better flow sensitivity. + DenseHashMap> props{nullptr}; + DfgScope* childScope(DfgScope* scope); std::optional use(DfgScope* scope, Symbol symbol, AstExpr* e); + DefId use(DefId def, AstExprIndexName* e); void visit(DfgScope* scope, AstStatBlock* b); void visitBlockWithoutChildScope(DfgScope* scope, AstStatBlock* b); diff --git a/Analysis/include/Luau/Def.h b/Analysis/include/Luau/Def.h index ac1fa132..1eef7dfd 100644 --- a/Analysis/include/Luau/Def.h +++ b/Analysis/include/Luau/Def.h @@ -5,37 +5,38 @@ #include "Luau/TypedAllocator.h" #include "Luau/Variant.h" +#include +#include + namespace Luau { -using Def = Variant; - -/** - * We statically approximate a value at runtime using a symbolic value, which we call a Def. - * - * DataFlowGraphBuilder will allocate these defs as a stand-in for some Luau values, and bind them to places that - * can hold a Luau value, and then observes how those defs will commute as it statically evaluate the program. - * - * It must also be noted that defs are a cyclic graph, so it is not safe to recursively traverse into it expecting it to terminate. - */ +struct Def; using DefId = NotNull; +struct FieldMetadata +{ + DefId parent; + std::string propName; +}; + /** - * A "single-object" value. + * A cell is a "single-object" value. * * Leaky implementation note: sometimes "multiple-object" values, but none of which were interesting enough to warrant creating a phi node instead. * That can happen because there's no point in creating a phi node that points to either resultant in `if math.random() > 0.5 then 5 else "hello"`. * This might become of utmost importance if we wanted to do some backward reasoning, e.g. if `5` is taken, then `cond` must be `truthy`. */ -struct Undefined +struct Cell { + std::optional field; }; /** - * A phi node is a union of defs. + * A phi node is a union of cells. * * We need this because we're statically evaluating a program, and sometimes a place may be assigned with - * different defs, and when that happens, we need a special data type that merges in all the defs + * different cells, and when that happens, we need a special data type that merges in all the cells * that will flow into that specific place. For example, consider this simple program: * * ``` @@ -56,23 +57,35 @@ struct Phi std::vector operands; }; -template -T* getMutable(DefId def) +/** + * We statically approximate a value at runtime using a symbolic value, which we call a Def. + * + * DataFlowGraphBuilder will allocate these defs as a stand-in for some Luau values, and bind them to places that + * can hold a Luau value, and then observes how those defs will commute as it statically evaluate the program. + * + * It must also be noted that defs are a cyclic graph, so it is not safe to recursively traverse into it expecting it to terminate. + */ +struct Def { - return get_if(def.get()); -} + using V = Variant; + + V v; +}; template const T* get(DefId def) { - return getMutable(def); + return get_if(&def->v); } struct DefArena { TypedAllocator allocator; - DefId freshDef(); + DefId freshCell(); + DefId freshCell(DefId parent, const std::string& prop); + // TODO: implement once we have cases where we need to merge in definitions + // DefId phi(const std::vector& defs); }; } // namespace Luau diff --git a/Analysis/include/Luau/Error.h b/Analysis/include/Luau/Error.h index f7bd9d50..89388046 100644 --- a/Analysis/include/Luau/Error.h +++ b/Analysis/include/Luau/Error.h @@ -7,21 +7,31 @@ #include "Luau/Variant.h" #include "Luau/TypeArena.h" -LUAU_FASTFLAG(LuauIceExceptionInheritanceChange) - namespace Luau { struct TypeError; + struct TypeMismatch { + enum Context + { + CovariantContext, + InvariantContext + }; + TypeMismatch() = default; TypeMismatch(TypeId wantedType, TypeId givenType); TypeMismatch(TypeId wantedType, TypeId givenType, std::string reason); TypeMismatch(TypeId wantedType, TypeId givenType, std::string reason, std::optional error); + TypeMismatch(TypeId wantedType, TypeId givenType, Context context); + TypeMismatch(TypeId wantedType, TypeId givenType, std::string reason, Context context); + TypeMismatch(TypeId wantedType, TypeId givenType, std::string reason, std::optional error, Context context); + TypeId wantedType = nullptr; TypeId givenType = nullptr; + Context context = CovariantContext; std::string reason; std::shared_ptr error; @@ -312,12 +322,33 @@ struct TypePackMismatch bool operator==(const TypePackMismatch& rhs) const; }; +struct DynamicPropertyLookupOnClassesUnsafe +{ + TypeId ty; + + bool operator==(const DynamicPropertyLookupOnClassesUnsafe& rhs) const; +}; + using TypeErrorData = Variant; + TypesAreUnrelated, NormalizationTooComplex, TypePackMismatch, DynamicPropertyLookupOnClassesUnsafe>; + +struct TypeErrorSummary +{ + Location location; + ModuleName moduleName; + int code; + + TypeErrorSummary(const Location& location, const ModuleName& moduleName, int code) + : location(location) + , moduleName(moduleName) + , code(code) + { + } +}; struct TypeError { @@ -325,6 +356,7 @@ struct TypeError ModuleName moduleName; TypeErrorData data; + static int minCode(); int code() const; TypeError() = default; @@ -342,6 +374,8 @@ struct TypeError } bool operator==(const TypeError& rhs) const; + + TypeErrorSummary summary() const; }; template @@ -406,10 +440,4 @@ public: const std::optional location; }; -// These two function overloads only exist to facilitate fast flagging a change to InternalCompilerError -// Both functions can be removed when FFlagLuauIceExceptionInheritanceChange is removed and calling code -// can directly throw InternalCompilerError. -[[noreturn]] void throwRuntimeError(const std::string& message); -[[noreturn]] void throwRuntimeError(const std::string& message, const std::string& moduleName); - } // namespace Luau diff --git a/Analysis/include/Luau/Normalize.h b/Analysis/include/Luau/Normalize.h index b28c06a5..d7e104ee 100644 --- a/Analysis/include/Luau/Normalize.h +++ b/Analysis/include/Luau/Normalize.h @@ -194,6 +194,8 @@ struct NormalizedFunctionType struct NormalizedType; using NormalizedTyvars = std::unordered_map>; +bool isInhabited_DEPRECATED(const NormalizedType& norm); + // A normalized type is either any, unknown, or one of the form P | T | F | G where // * P is a union of primitive types (including singletons, classes and the error type) // * T is a union of table types @@ -328,6 +330,10 @@ public: bool intersectNormals(NormalizedType& here, const NormalizedType& there, int ignoreSmallerTyvars = -1); bool intersectNormalWithTy(NormalizedType& here, TypeId there); + // Check for inhabitance + bool isInhabited(TypeId ty, std::unordered_set seen = {}); + bool isInhabited(const NormalizedType* norm, std::unordered_set seen = {}); + // -------- Convert back from a normalized type to a type TypeId typeFromNormal(const NormalizedType& norm); }; diff --git a/Analysis/include/Luau/NotNull.h b/Analysis/include/Luau/NotNull.h index 714fa143..ecdcb476 100644 --- a/Analysis/include/Luau/NotNull.h +++ b/Analysis/include/Luau/NotNull.h @@ -59,6 +59,20 @@ struct NotNull return ptr; } + template + bool operator==(NotNull other) const noexcept + { + return get() == other.get(); + } + + template + bool operator!=(NotNull other) const noexcept + { + return get() != other.get(); + } + + operator bool() const noexcept = delete; + T& operator[](int) = delete; T& operator+(int) = delete; diff --git a/Analysis/include/Luau/RecursionCounter.h b/Analysis/include/Luau/RecursionCounter.h index 632afd19..77af10a0 100644 --- a/Analysis/include/Luau/RecursionCounter.h +++ b/Analysis/include/Luau/RecursionCounter.h @@ -15,16 +15,6 @@ struct RecursionLimitException : public InternalCompilerError RecursionLimitException() : InternalCompilerError("Internal recursion counter limit exceeded") { - LUAU_ASSERT(FFlag::LuauIceExceptionInheritanceChange); - } -}; - -struct RecursionLimitException_DEPRECATED : public std::exception -{ - const char* what() const noexcept - { - LUAU_ASSERT(!FFlag::LuauIceExceptionInheritanceChange); - return "Internal recursion counter limit exceeded"; } }; @@ -53,14 +43,7 @@ struct RecursionLimiter : RecursionCounter { if (limit > 0 && *count > limit) { - if (FFlag::LuauIceExceptionInheritanceChange) - { - throw RecursionLimitException(); - } - else - { - throw RecursionLimitException_DEPRECATED(); - } + throw RecursionLimitException(); } } }; diff --git a/Analysis/include/Luau/ToString.h b/Analysis/include/Luau/ToString.h index 0200a719..186cc9a5 100644 --- a/Analysis/include/Luau/ToString.h +++ b/Analysis/include/Luau/ToString.h @@ -30,7 +30,7 @@ struct ToStringOptions bool hideTableKind = false; // If true, all tables will be surrounded with plain '{}' bool hideNamedFunctionTypeParameters = false; // If true, type parameters of functions will be hidden at top-level. bool hideFunctionSelfArgument = false; // If true, `self: X` will be omitted from the function signature if the function has self - bool indent = false; + bool DEPRECATED_indent = false; // TODO Deprecated field, prune when clipping flag FFlagLuauLineBreaksDeterminIndents size_t maxTableLength = size_t(FInt::LuauTableTypeMaximumStringifierLength); // Only applied to TableTypeVars size_t maxTypeLength = size_t(FInt::LuauTypeMaximumStringifierLength); ToStringNameMap nameMap; @@ -90,8 +90,6 @@ inline std::string toString(const Constraint& c) return toString(c, ToStringOptions{}); } -std::string toString(const LValue& lvalue); - std::string toString(const TypeVar& tv, ToStringOptions& opts); std::string toString(const TypePackVar& tp, ToStringOptions& opts); diff --git a/Analysis/include/Luau/TxnLog.h b/Analysis/include/Luau/TxnLog.h index 3c3122c2..b1a83412 100644 --- a/Analysis/include/Luau/TxnLog.h +++ b/Analysis/include/Luau/TxnLog.h @@ -108,6 +108,8 @@ struct TxnLog // If both logs talk about the same type, pack, or table, the rhs takes // priority. void concat(TxnLog rhs); + void concatAsIntersections(TxnLog rhs, NotNull arena); + void concatAsUnion(TxnLog rhs, NotNull arena); // Commits the TxnLog, rebinding all type pointers to their pending states. // Clears the TxnLog afterwards. diff --git a/Analysis/include/Luau/TypeInfer.h b/Analysis/include/Luau/TypeInfer.h index 2eea191a..c6f153d1 100644 --- a/Analysis/include/Luau/TypeInfer.h +++ b/Analysis/include/Luau/TypeInfer.h @@ -54,16 +54,9 @@ public: explicit TimeLimitError(const std::string& moduleName) : InternalCompilerError("Typeinfer failed to complete in allotted time", moduleName) { - LUAU_ASSERT(FFlag::LuauIceExceptionInheritanceChange); } }; -class TimeLimitError_DEPRECATED : public std::exception -{ -public: - virtual const char* what() const throw(); -}; - // All TypeVars are retained via Environment::typeVars. All TypeIds // within a program are borrowed pointers into this set. struct TypeChecker diff --git a/Analysis/include/Luau/TypeUtils.h b/Analysis/include/Luau/TypeUtils.h index 085ee21b..aa9cdde2 100644 --- a/Analysis/include/Luau/TypeUtils.h +++ b/Analysis/include/Luau/TypeUtils.h @@ -25,9 +25,9 @@ std::optional getIndexTypeFromType(const ScopePtr& scope, ErrorVec& erro // Returns the minimum and maximum number of types the argument list can accept. std::pair> getParameterExtents(const TxnLog* log, TypePackId tp, bool includeHiddenVariadics = false); -// "Render" a type pack out to an array of a given length. Expands variadics and -// various other things to get there. -std::vector flatten(TypeArena& arena, NotNull singletonTypes, TypePackId pack, size_t length); +// Extend the provided pack to at least `length` types. +// Returns a temporary TypePack that contains those types plus a tail. +TypePack extendTypePack(TypeArena& arena, NotNull singletonTypes, TypePackId pack, size_t length); /** * Reduces a union by decomposing to the any/error type if it appears in the diff --git a/Analysis/include/Luau/TypeVar.h b/Analysis/include/Luau/TypeVar.h index 0ab4d474..d355746a 100644 --- a/Analysis/include/Luau/TypeVar.h +++ b/Analysis/include/Luau/TypeVar.h @@ -3,6 +3,8 @@ #include "Luau/Ast.h" #include "Luau/Common.h" +#include "Luau/Connective.h" +#include "Luau/DataFlowGraph.h" #include "Luau/DenseHash.h" #include "Luau/Def.h" #include "Luau/NotNull.h" @@ -257,7 +259,17 @@ struct MagicFunctionCallContext TypePackId result; }; -using DcrMagicFunction = std::function; +using DcrMagicFunction = bool (*)(MagicFunctionCallContext); + +struct MagicRefinementContext +{ + ScopePtr scope; + NotNull dfg; + NotNull connectiveArena; + const class AstExprCall* callSite; +}; + +using DcrMagicRefinement = std::vector (*)(MagicRefinementContext); struct FunctionTypeVar { @@ -279,19 +291,20 @@ struct FunctionTypeVar FunctionTypeVar(TypeLevel level, Scope* scope, std::vector generics, std::vector genericPacks, TypePackId argTypes, TypePackId retTypes, std::optional defn = {}, bool hasSelf = false); - TypeLevel level; - Scope* scope = nullptr; + std::optional definition; /// These should all be generic std::vector generics; std::vector genericPacks; - TypePackId argTypes; std::vector> argNames; - TypePackId retTypes; - std::optional definition; - MagicFunction magicFunction = nullptr; // Function pointer, can be nullptr. - DcrMagicFunction dcrMagicFunction = nullptr; // can be nullptr - bool hasSelf; Tags tags; + TypeLevel level; + Scope* scope = nullptr; + TypePackId argTypes; + TypePackId retTypes; + MagicFunction magicFunction = nullptr; + DcrMagicFunction dcrMagicFunction = nullptr; // Fired only while solving constraints + DcrMagicRefinement dcrMagicRefinement = nullptr; // Fired only while generating constraints + bool hasSelf; bool hasNoGenerics = false; }; diff --git a/Analysis/include/Luau/Unifier.h b/Analysis/include/Luau/Unifier.h index b5f58d3c..af3864ea 100644 --- a/Analysis/include/Luau/Unifier.h +++ b/Analysis/include/Luau/Unifier.h @@ -120,6 +120,9 @@ private: std::optional findTablePropertyRespectingMeta(TypeId lhsType, Name name); + TxnLog combineLogsIntoIntersection(std::vector logs); + TxnLog combineLogsIntoUnion(std::vector logs); + public: // Returns true if the type "needle" already occurs within "haystack" and reports an "infinite type error" bool occursCheck(TypeId needle, TypeId haystack); @@ -134,6 +137,7 @@ public: private: bool isNonstrictMode() const; + TypeMismatch::Context mismatchContext(); void checkChildUnifierTypeMismatch(const ErrorVec& innerErrors, TypeId wantedType, TypeId givenType); void checkChildUnifierTypeMismatch(const ErrorVec& innerErrors, const std::string& prop, TypeId wantedType, TypeId givenType); diff --git a/Analysis/src/AstQuery.cpp b/Analysis/src/AstQuery.cpp index b93c2cc2..85d2320a 100644 --- a/Analysis/src/AstQuery.cpp +++ b/Analysis/src/AstQuery.cpp @@ -11,15 +11,12 @@ #include -LUAU_FASTFLAGVARIABLE(LuauCheckOverloadedDocSymbol, false) - namespace Luau { namespace { - struct AutocompleteNodeFinder : public AstVisitor { const Position pos; @@ -432,8 +429,6 @@ ExprOrLocal findExprOrLocalAtPosition(const SourceModule& source, Position pos) static std::optional checkOverloadedDocumentationSymbol( const Module& module, const TypeId ty, const AstExpr* parentExpr, const std::optional documentationSymbol) { - LUAU_ASSERT(FFlag::LuauCheckOverloadedDocSymbol); - if (!documentationSymbol) return std::nullopt; @@ -469,40 +464,7 @@ std::optional getDocumentationSymbolAtPosition(const Source AstExpr* parentExpr = ancestry.size() >= 2 ? ancestry[ancestry.size() - 2]->asExpr() : nullptr; if (std::optional binding = findBindingAtPosition(module, source, position)) - { - if (FFlag::LuauCheckOverloadedDocSymbol) - { - return checkOverloadedDocumentationSymbol(module, binding->typeId, parentExpr, binding->documentationSymbol); - } - else - { - if (binding->documentationSymbol) - { - // This might be an overloaded function binding. - if (get(follow(binding->typeId))) - { - TypeId matchingOverload = nullptr; - if (parentExpr && parentExpr->is()) - { - if (auto it = module.astOverloadResolvedTypes.find(parentExpr)) - { - matchingOverload = *it; - } - } - - if (matchingOverload) - { - std::string overloadSymbol = *binding->documentationSymbol + "/overload/"; - // Default toString options are fine for this purpose. - overloadSymbol += toString(matchingOverload); - return overloadSymbol; - } - } - } - - return binding->documentationSymbol; - } - } + return checkOverloadedDocumentationSymbol(module, binding->typeId, parentExpr, binding->documentationSymbol); if (targetExpr) { @@ -514,22 +476,12 @@ std::optional getDocumentationSymbolAtPosition(const Source if (const TableTypeVar* ttv = get(parentTy)) { if (auto propIt = ttv->props.find(indexName->index.value); propIt != ttv->props.end()) - { - if (FFlag::LuauCheckOverloadedDocSymbol) - return checkOverloadedDocumentationSymbol(module, propIt->second.type, parentExpr, propIt->second.documentationSymbol); - else - return propIt->second.documentationSymbol; - } + return checkOverloadedDocumentationSymbol(module, propIt->second.type, parentExpr, propIt->second.documentationSymbol); } else if (const ClassTypeVar* ctv = get(parentTy)) { if (auto propIt = ctv->props.find(indexName->index.value); propIt != ctv->props.end()) - { - if (FFlag::LuauCheckOverloadedDocSymbol) - return checkOverloadedDocumentationSymbol(module, propIt->second.type, parentExpr, propIt->second.documentationSymbol); - else - return propIt->second.documentationSymbol; - } + return checkOverloadedDocumentationSymbol(module, propIt->second.type, parentExpr, propIt->second.documentationSymbol); } } } diff --git a/Analysis/src/Autocomplete.cpp b/Analysis/src/Autocomplete.cpp index 50dc254f..5374c6b1 100644 --- a/Analysis/src/Autocomplete.cpp +++ b/Analysis/src/Autocomplete.cpp @@ -1457,7 +1457,8 @@ static AutocompleteResult autocomplete(const SourceModule& sourceModule, const M return autocompleteExpression(sourceModule, *module, singletonTypes, &typeArena, ancestry, position); else if (AstStatRepeat* statRepeat = extractStat(ancestry); statRepeat) return {autocompleteStatement(sourceModule, *module, ancestry, position), ancestry, AutocompleteContext::Statement}; - else if (AstExprTable* exprTable = parent->as(); exprTable && (node->is() || node->is() || node->is())) + else if (AstExprTable* exprTable = parent->as(); + exprTable && (node->is() || node->is() || node->is())) { for (const auto& [kind, key, value] : exprTable->items) { diff --git a/Analysis/src/BuiltinDefinitions.cpp b/Analysis/src/BuiltinDefinitions.cpp index 67e3979a..39568674 100644 --- a/Analysis/src/BuiltinDefinitions.cpp +++ b/Analysis/src/BuiltinDefinitions.cpp @@ -132,6 +132,14 @@ void attachDcrMagicFunction(TypeId ty, DcrMagicFunction fn) LUAU_ASSERT(!"Got a non functional type"); } +void attachDcrMagicRefinement(TypeId ty, DcrMagicRefinement fn) +{ + if (auto ftv = getMutable(ty)) + ftv->dcrMagicRefinement = fn; + else + LUAU_ASSERT(!"Got a non functional type"); +} + Property makeProperty(TypeId ty, std::optional documentationSymbol) { return { diff --git a/Analysis/src/ConstraintGraphBuilder.cpp b/Analysis/src/ConstraintGraphBuilder.cpp index e3572fe8..d41c7772 100644 --- a/Analysis/src/ConstraintGraphBuilder.cpp +++ b/Analysis/src/ConstraintGraphBuilder.cpp @@ -9,7 +9,9 @@ #include "Luau/ModuleResolver.h" #include "Luau/RecursionCounter.h" #include "Luau/Scope.h" +#include "Luau/Substitution.h" #include "Luau/ToString.h" +#include "Luau/TxnLog.h" #include "Luau/TypeUtils.h" #include "Luau/TypeVar.h" @@ -191,7 +193,7 @@ static void unionRefinements(const std::unordered_map& lhs, const } static void computeRefinement(const ScopePtr& scope, ConnectiveId connective, std::unordered_map* refis, bool sense, - NotNull arena, bool eq, std::vector* constraints) + NotNull arena, bool eq, std::vector* constraints) { using RefinementMap = std::unordered_map; @@ -231,10 +233,10 @@ static void computeRefinement(const ScopePtr& scope, ConnectiveId connective, st TypeId discriminantTy = proposition->discriminantTy; if (!sense && !eq) discriminantTy = arena->addType(NegationTypeVar{proposition->discriminantTy}); - else if (!sense && eq) + else if (eq) { discriminantTy = arena->addType(BlockedTypeVar{}); - constraints->push_back(SingletonOrTopTypeConstraint{discriminantTy, proposition->discriminantTy}); + constraints->push_back(SingletonOrTopTypeConstraint{discriminantTy, proposition->discriminantTy, !sense}); } if (auto it = refis->find(proposition->def); it != refis->end()) @@ -244,23 +246,43 @@ static void computeRefinement(const ScopePtr& scope, ConnectiveId connective, st } } +static std::pair computeDiscriminantType(NotNull arena, const ScopePtr& scope, DefId def, TypeId discriminantTy) +{ + LUAU_ASSERT(get(def)); + + while (const Cell* current = get(def)) + { + if (!current->field) + break; + + TableTypeVar::Props props{{current->field->propName, Property{discriminantTy}}}; + discriminantTy = arena->addType(TableTypeVar{std::move(props), std::nullopt, TypeLevel{}, scope.get(), TableState::Sealed}); + + def = current->field->parent; + current = get(def); + } + + return {def, discriminantTy}; +} + void ConstraintGraphBuilder::applyRefinements(const ScopePtr& scope, Location location, ConnectiveId connective) { if (!connective) return; std::unordered_map refinements; - std::vector constraints; + std::vector constraints; computeRefinement(scope, connective, &refinements, /*sense*/ true, arena, /*eq*/ false, &constraints); for (auto [def, discriminantTy] : refinements) { - std::optional defTy = scope->lookup(def); + auto [def2, discriminantTy2] = computeDiscriminantType(arena, scope, def, discriminantTy); + std::optional defTy = scope->lookup(def2); if (!defTy) ice->ice("Every DefId must map to a type!"); - TypeId resultTy = arena->addType(IntersectionTypeVar{{*defTy, discriminantTy}}); - scope->dcrRefinements[def] = resultTy; + TypeId resultTy = arena->addType(IntersectionTypeVar{{*defTy, discriminantTy2}}); + scope->dcrRefinements[def2] = resultTy; } for (auto& c : constraints) @@ -446,15 +468,15 @@ void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatLocal* local) if (i < local->vars.size) { - std::vector packTypes = flatten(*arena, singletonTypes, exprPack, varTypes.size() - i); + TypePack packTypes = extendTypePack(*arena, singletonTypes, exprPack, varTypes.size() - i); // fill out missing values in varTypes with values from exprPack for (size_t j = i; j < varTypes.size(); ++j) { if (!varTypes[j]) { - if (j - i < packTypes.size()) - varTypes[j] = packTypes[j - i]; + if (j - i < packTypes.head.size()) + varTypes[j] = packTypes.head[j - i]; else varTypes[j] = freshType(scope); } @@ -591,9 +613,9 @@ void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatLocalFunction* FunctionSignature sig = checkFunctionSignature(scope, function->func); sig.bodyScope->bindings[function->name] = Binding{sig.signature, function->func->location}; - auto start = checkpoint(this); + Checkpoint start = checkpoint(this); checkFunctionBody(sig.bodyScope, function->func); - auto end = checkpoint(this); + Checkpoint end = checkpoint(this); NotNull constraintScope{sig.signatureScope ? sig.signatureScope.get() : sig.bodyScope.get()}; std::unique_ptr c = @@ -611,7 +633,7 @@ void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatFunction* funct // Name could be AstStatLocal, AstStatGlobal, AstStatIndexName. // With or without self - TypeId functionType = nullptr; + TypeId generalizedType = arena->addType(BlockedTypeVar{}); FunctionSignature sig = checkFunctionSignature(scope, function->func); @@ -620,62 +642,59 @@ void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatFunction* funct std::optional existingFunctionTy = scope->lookup(localName->local); if (existingFunctionTy) { - // Duplicate definition - functionType = *existingFunctionTy; + addConstraint(scope, function->name->location, SubtypeConstraint{generalizedType, *existingFunctionTy}); + + Symbol sym{localName->local}; + std::optional def = dfg->getDef(sym); + LUAU_ASSERT(def); + scope->bindings[sym].typeId = generalizedType; + scope->dcrRefinements[*def] = generalizedType; } else - { - functionType = arena->addType(BlockedTypeVar{}); - scope->bindings[localName->local] = Binding{functionType, localName->location}; - } + scope->bindings[localName->local] = Binding{generalizedType, localName->location}; + sig.bodyScope->bindings[localName->local] = Binding{sig.signature, localName->location}; } else if (AstExprGlobal* globalName = function->name->as()) { std::optional existingFunctionTy = scope->lookup(globalName->name); - if (existingFunctionTy) - { - // Duplicate definition - functionType = *existingFunctionTy; - } - else - { - functionType = arena->addType(BlockedTypeVar{}); - rootScope->bindings[globalName->name] = Binding{functionType, globalName->location}; - } + if (!existingFunctionTy) + ice->ice("prepopulateGlobalScope did not populate a global name", globalName->location); + + generalizedType = *existingFunctionTy; + sig.bodyScope->bindings[globalName->name] = Binding{sig.signature, globalName->location}; } else if (AstExprIndexName* indexName = function->name->as()) { TypeId containingTableType = check(scope, indexName->expr).ty; - functionType = arena->addType(BlockedTypeVar{}); - // TODO look into stack utilization. This is probably ok because it scales with AST depth. TypeId prospectiveTableType = arena->addType(TableTypeVar{TableState::Unsealed, TypeLevel{}, scope.get()}); NotNull prospectiveTable{getMutable(prospectiveTableType)}; Property& prop = prospectiveTable->props[indexName->index.value]; - prop.type = functionType; + prop.type = generalizedType; prop.location = function->name->location; addConstraint(scope, indexName->location, SubtypeConstraint{containingTableType, prospectiveTableType}); } else if (AstExprError* err = function->name->as()) { - functionType = singletonTypes->errorRecoveryType(); + generalizedType = singletonTypes->errorRecoveryType(); } - LUAU_ASSERT(functionType != nullptr); + if (generalizedType == nullptr) + ice->ice("generalizedType == nullptr", function->location); - auto start = checkpoint(this); + Checkpoint start = checkpoint(this); checkFunctionBody(sig.bodyScope, function->func); - auto end = checkpoint(this); + Checkpoint end = checkpoint(this); NotNull constraintScope{sig.signatureScope ? sig.signatureScope.get() : sig.bodyScope.get()}; std::unique_ptr c = - std::make_unique(constraintScope, function->name->location, GeneralizationConstraint{functionType, sig.signature}); + std::make_unique(constraintScope, function->name->location, GeneralizationConstraint{generalizedType, sig.signature}); forEachConstraint(start, end, this, [&c](const ConstraintPtr& constraint) { c->dependencies.push_back(NotNull{constraint.get()}); @@ -708,7 +727,9 @@ void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatBlock* block) void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatAssign* assign) { TypePackId varPackId = checkLValues(scope, assign->vars); - TypePackId valuePack = checkPack(scope, assign->values).tp; + + TypePack expectedTypes = extendTypePack(*arena, singletonTypes, varPackId, assign->values.size); + TypePackId valuePack = checkPack(scope, assign->values, expectedTypes.head).tp; addConstraint(scope, assign->location, PackSubtypeConstraint{valuePack, varPackId}); } @@ -729,8 +750,6 @@ void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatCompoundAssign* void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatIf* ifStatement) { - // TODO: Optimization opportunity, the interior scope of the condition could be - // reused for the then body, so we don't need to refine twice. ScopePtr condScope = childScope(ifStatement->condition, scope); auto [_, connective] = check(condScope, ifStatement->condition, std::nullopt); @@ -986,7 +1005,7 @@ InferencePack ConstraintGraphBuilder::checkPack(const ScopePtr& scope, AstExpr* InferencePack result; if (AstExprCall* call = expr->as()) - result = {checkPack(scope, call, expectedTypes)}; + result = checkPack(scope, call, expectedTypes); else if (AstExprVarargs* varargs = expr->as()) { if (scope->varargPack) @@ -1010,38 +1029,101 @@ InferencePack ConstraintGraphBuilder::checkPack(const ScopePtr& scope, AstExpr* InferencePack ConstraintGraphBuilder::checkPack(const ScopePtr& scope, AstExprCall* call, const std::vector& expectedTypes) { - TypeId fnType = check(scope, call->func).ty; - auto startCheckpoint = checkpoint(this); - - std::vector args; - - for (AstExpr* arg : call->args) - { - args.push_back(check(scope, arg).ty); - } - + std::vector exprArgs; if (call->self) { AstExprIndexName* indexExpr = call->func->as(); if (!indexExpr) ice->ice("method call expression has no 'self'"); - // The call to `check` we already did on `call->func` should have already produced a type for - // `indexExpr->expr`, so we can get it from `astTypes` to avoid exponential blow-up. - TypeId selfType = astTypes[indexExpr->expr]; - - // If we don't have a type for self, it means we had a code too complex error already. - if (selfType == nullptr) - selfType = singletonTypes->errorRecoveryType(); - - args.insert(args.begin(), selfType); + exprArgs.push_back(indexExpr->expr); } + exprArgs.insert(exprArgs.end(), call->args.begin(), call->args.end()); + + Checkpoint startCheckpoint = checkpoint(this); + TypeId fnType = check(scope, call->func).ty; + Checkpoint fnEndCheckpoint = checkpoint(this); + + TypePackId expectedArgPack = arena->freshTypePack(scope.get()); + TypePackId expectedRetPack = arena->freshTypePack(scope.get()); + TypeId expectedFunctionType = arena->addType(FunctionTypeVar{expectedArgPack, expectedRetPack}); + + TypeId instantiatedFnType = arena->addType(BlockedTypeVar{}); + addConstraint(scope, call->location, InstantiationConstraint{instantiatedFnType, fnType}); + + NotNull extractArgsConstraint = addConstraint(scope, call->location, SubtypeConstraint{instantiatedFnType, expectedFunctionType}); + + // Fully solve fnType, then extract its argument list as expectedArgPack. + forEachConstraint(startCheckpoint, fnEndCheckpoint, this, [extractArgsConstraint](const ConstraintPtr& constraint) { + extractArgsConstraint->dependencies.emplace_back(constraint.get()); + }); + + const AstExpr* lastArg = exprArgs.size() ? exprArgs[exprArgs.size() - 1] : nullptr; + const bool needTail = lastArg && (lastArg->is() || lastArg->is()); + + TypePack expectedArgs; + + if (!needTail) + expectedArgs = extendTypePack(*arena, singletonTypes, expectedArgPack, exprArgs.size()); + else + expectedArgs = extendTypePack(*arena, singletonTypes, expectedArgPack, exprArgs.size() - 1); + + std::vector connectives; + if (auto ftv = get(follow(fnType)); ftv && ftv->dcrMagicRefinement) + { + MagicRefinementContext ctx{globalScope, dfg, NotNull{&connectiveArena}, call}; + connectives = ftv->dcrMagicRefinement(ctx); + } + + + std::vector args; + std::optional argTail; + + Checkpoint argCheckpoint = checkpoint(this); + + for (size_t i = 0; i < exprArgs.size(); ++i) + { + AstExpr* arg = exprArgs[i]; + std::optional expectedType; + if (i < expectedArgs.head.size()) + expectedType = expectedArgs.head[i]; + + if (i == 0 && call->self) + { + // The self type has already been computed as a side effect of + // computing fnType. If computing that did not cause us to exceed a + // recursion limit, we can fetch it from astTypes rather than + // recomputing it. + TypeId* selfTy = astTypes.find(exprArgs[0]); + if (selfTy) + args.push_back(*selfTy); + else + args.push_back(arena->freshType(scope.get())); + } + else if (i < exprArgs.size() - 1 || !(arg->is() || arg->is())) + args.push_back(check(scope, arg, expectedType).ty); + else + argTail = checkPack(scope, arg, {}).tp; // FIXME? not sure about expectedTypes here + } + + Checkpoint argEndCheckpoint = checkpoint(this); + + // Do not solve argument constraints until after we have extracted the + // expected types from the callable. + forEachConstraint(argCheckpoint, argEndCheckpoint, this, [extractArgsConstraint](const ConstraintPtr& constraint) { + constraint->dependencies.push_back(extractArgsConstraint); + }); if (matchSetmetatable(*call)) { - LUAU_ASSERT(args.size() == 2); - TypeId target = args[0]; - TypeId mt = args[1]; + TypePack argTailPack; + if (argTail && args.size() < 2) + argTailPack = extendTypePack(*arena, singletonTypes, *argTail, 2 - args.size()); + + LUAU_ASSERT(args.size() + argTailPack.head.size() == 2); + + TypeId target = args.size() > 0 ? args[0] : argTailPack.head[0]; + TypeId mt = args.size() > 1 ? args[1] : argTailPack.head[args.size() == 0 ? 1 : 0]; AstExpr* targetExpr = call->args.data[0]; @@ -1051,18 +1133,16 @@ InferencePack ConstraintGraphBuilder::checkPack(const ScopePtr& scope, AstExprCa if (AstExprLocal* targetLocal = targetExpr->as()) scope->bindings[targetLocal->local].typeId = resultTy; - return InferencePack{arena->addTypePack({resultTy})}; + return InferencePack{arena->addTypePack({resultTy}), std::move(connectives)}; } else { - auto endCheckpoint = checkpoint(this); - astOriginalCallTypes[call->func] = fnType; TypeId instantiatedType = arena->addType(BlockedTypeVar{}); // TODO: How do expectedTypes play into this? Do they? TypePackId rets = arena->addTypePack(BlockedTypePack{}); - TypePackId argPack = arena->addTypePack(TypePack{args, {}}); + TypePackId argPack = arena->addTypePack(TypePack{args, argTail}); FunctionTypeVar ftv(TypeLevel{}, scope.get(), argPack, rets); TypeId inferredFnType = arena->addType(ftv); @@ -1071,19 +1151,10 @@ InferencePack ConstraintGraphBuilder::checkPack(const ScopePtr& scope, AstExprCa NotNull ic(unqueuedConstraints.back().get()); unqueuedConstraints.push_back( - std::make_unique(NotNull{scope.get()}, call->func->location, SubtypeConstraint{inferredFnType, instantiatedType})); + std::make_unique(NotNull{scope.get()}, call->func->location, SubtypeConstraint{instantiatedType, inferredFnType})); NotNull sc(unqueuedConstraints.back().get()); - // We force constraints produced by checking function arguments to wait - // until after we have resolved the constraint on the function itself. - // This ensures, for instance, that we start inferring the contents of - // lambdas under the assumption that their arguments and return types - // will be compatible with the enclosing function call. - forEachConstraint(startCheckpoint, endCheckpoint, this, [sc](const ConstraintPtr& constraint) { - constraint->dependencies.push_back(sc); - }); - - addConstraint(scope, call->func->location, + NotNull fcc = addConstraint(scope, call->func->location, FunctionCallConstraint{ {ic, sc}, fnType, @@ -1092,7 +1163,16 @@ InferencePack ConstraintGraphBuilder::checkPack(const ScopePtr& scope, AstExprCa call, }); - return InferencePack{rets}; + // We force constraints produced by checking function arguments to wait + // until after we have resolved the constraint on the function itself. + // This ensures, for instance, that we start inferring the contents of + // lambdas under the assumption that their arguments and return types + // will be compatible with the enclosing function call. + forEachConstraint(fnEndCheckpoint, argEndCheckpoint, this, [fcc](const ConstraintPtr& constraint) { + fcc->dependencies.emplace_back(constraint.get()); + }); + + return InferencePack{rets, std::move(connectives)}; } } @@ -1133,9 +1213,19 @@ Inference ConstraintGraphBuilder::check(const ScopePtr& scope, AstExpr* expr, st } else if (auto a = expr->as()) { - FunctionSignature sig = checkFunctionSignature(scope, a); + Checkpoint startCheckpoint = checkpoint(this); + FunctionSignature sig = checkFunctionSignature(scope, a, expectedType); checkFunctionBody(sig.bodyScope, a); - return Inference{sig.signature}; + Checkpoint endCheckpoint = checkpoint(this); + + TypeId generalizedTy = arena->addType(BlockedTypeVar{}); + NotNull gc = addConstraint(scope, expr->location, GeneralizationConstraint{generalizedTy, sig.signature}); + + forEachConstraint(startCheckpoint, endCheckpoint, this, [gc](const ConstraintPtr& constraint) { + gc->dependencies.emplace_back(constraint.get()); + }); + + return Inference{generalizedTy}; } else if (auto indexName = expr->as()) result = check(scope, indexName); @@ -1253,10 +1343,83 @@ Inference ConstraintGraphBuilder::check(const ScopePtr& scope, AstExprGlobal* gl return Inference{singletonTypes->errorRecoveryType()}; } +static std::optional lookupProp(TypeId ty, const std::string& propName, NotNull arena) +{ + ty = follow(ty); + + if (auto ctv = get(ty)) + { + if (auto prop = lookupClassProp(ctv, propName)) + return prop->type; + } + else if (auto ttv = get(ty)) + { + if (auto it = ttv->props.find(propName); it != ttv->props.end()) + return it->second.type; + } + else if (auto utv = get(ty)) + { + std::vector types; + + for (TypeId ty : utv) + { + if (auto prop = lookupProp(ty, propName, arena)) + { + if (std::find(begin(types), end(types), *prop) == end(types)) + types.push_back(*prop); + } + else + return std::nullopt; + } + + if (types.size() == 1) + return types[0]; + else + return arena->addType(IntersectionTypeVar{std::move(types)}); + } + else if (auto utv = get(ty)) + { + std::vector types; + + for (TypeId ty : utv) + { + if (auto prop = lookupProp(ty, propName, arena)) + { + if (std::find(begin(types), end(types), *prop) == end(types)) + types.push_back(*prop); + } + else + return std::nullopt; + } + + if (types.size() == 1) + return types[0]; + else + return arena->addType(UnionTypeVar{std::move(types)}); + } + + return std::nullopt; +} + Inference ConstraintGraphBuilder::check(const ScopePtr& scope, AstExprIndexName* indexName) { TypeId obj = check(scope, indexName->expr).ty; - TypeId result = freshType(scope); + + // HACK: We need to return the actual type for type refinements so that it can invoke the dcrMagicRefinement function. + TypeId result; + if (auto prop = lookupProp(obj, indexName->index.value, arena)) + result = *prop; + else + result = freshType(scope); + + std::optional def = dfg->getDef(indexName); + if (def) + { + if (auto ty = scope->lookup(*def)) + return Inference{*ty, connectiveArena.proposition(*def, singletonTypes->truthyType)}; + else + scope->dcrRefinements[*def] = result; + } TableTypeVar::Props props{{indexName->index.value, Property{result}}}; const std::optional indexer; @@ -1266,7 +1429,10 @@ Inference ConstraintGraphBuilder::check(const ScopePtr& scope, AstExprIndexName* addConstraint(scope, indexName->expr->location, SubtypeConstraint{obj, expectedTableType}); - return Inference{result}; + if (def) + return Inference{result, connectiveArena.proposition(*def, singletonTypes->truthyType)}; + else + return Inference{result}; } Inference ConstraintGraphBuilder::check(const ScopePtr& scope, AstExprIndexExpr* indexExpr) @@ -1555,8 +1721,16 @@ Inference ConstraintGraphBuilder::check(const ScopePtr& scope, AstExprTable* exp { if (auto stringKey = item.key->as()) { - expectedValueType = arena->addType(BlockedTypeVar{}); - addConstraint(scope, item.value->location, HasPropConstraint{*expectedValueType, *expectedType, stringKey->value.data}); + ErrorVec errorVec; + std::optional propTy = + findTablePropertyRespectingMeta(singletonTypes, errorVec, follow(*expectedType), stringKey->value.data, item.value->location); + if (propTy) + expectedValueType = propTy; + else + { + expectedValueType = arena->addType(BlockedTypeVar{}); + addConstraint(scope, item.value->location, HasPropConstraint{*expectedValueType, *expectedType, stringKey->value.data}); + } } } @@ -1590,7 +1764,8 @@ Inference ConstraintGraphBuilder::check(const ScopePtr& scope, AstExprTable* exp return Inference{ty}; } -ConstraintGraphBuilder::FunctionSignature ConstraintGraphBuilder::checkFunctionSignature(const ScopePtr& parent, AstExprFunction* fn) +ConstraintGraphBuilder::FunctionSignature ConstraintGraphBuilder::checkFunctionSignature( + const ScopePtr& parent, AstExprFunction* fn, std::optional expectedType) { ScopePtr signatureScope = nullptr; ScopePtr bodyScope = nullptr; @@ -1599,22 +1774,22 @@ ConstraintGraphBuilder::FunctionSignature ConstraintGraphBuilder::checkFunctionS std::vector genericTypes; std::vector genericTypePacks; + if (expectedType) + expectedType = follow(*expectedType); + bool hasGenerics = fn->generics.size > 0 || fn->genericPacks.size > 0; - // If we don't have any generics, we can save some memory and compute by not - // creating the signatureScope, which is only used to scope the declared - // generics properly. + signatureScope = childScope(fn, parent); + + // We need to assign returnType before creating bodyScope so that the + // return type gets propogated to bodyScope. + returnType = freshTypePack(signatureScope); + signatureScope->returnType = returnType; + + bodyScope = childScope(fn->body, signatureScope); + if (hasGenerics) { - signatureScope = childScope(fn, parent); - - // We need to assign returnType before creating bodyScope so that the - // return type gets propogated to bodyScope. - returnType = freshTypePack(signatureScope); - signatureScope->returnType = returnType; - - bodyScope = childScope(fn->body, signatureScope); - std::vector> genericDefinitions = createGenerics(signatureScope, fn->generics); std::vector> genericPackDefinitions = createGenericPacks(signatureScope, fn->genericPacks); @@ -1631,18 +1806,50 @@ ConstraintGraphBuilder::FunctionSignature ConstraintGraphBuilder::checkFunctionS genericTypePacks.push_back(g.tp); signatureScope->privateTypePackBindings[name] = g.tp; } + + // Local variable works around an odd gcc 11.3 warning: may be used uninitialized + std::optional none = std::nullopt; + expectedType = none; } - else + + std::vector argTypes; + TypePack expectedArgPack; + + const FunctionTypeVar* expectedFunction = expectedType ? get(*expectedType) : nullptr; + + if (expectedFunction) { - bodyScope = childScope(fn, parent); + expectedArgPack = extendTypePack(*arena, singletonTypes, expectedFunction->argTypes, fn->args.size); - returnType = freshTypePack(bodyScope); - bodyScope->returnType = returnType; + genericTypes = expectedFunction->generics; + genericTypePacks = expectedFunction->genericPacks; + } - // To eliminate the need to branch on hasGenerics below, we say that the - // signature scope is the body scope when there is no real signature - // scope. - signatureScope = bodyScope; + for (size_t i = 0; i < fn->args.size; ++i) + { + AstLocal* local = fn->args.data[i]; + + TypeId t = freshType(signatureScope); + argTypes.push_back(t); + signatureScope->bindings[local] = Binding{t, local->location}; + + TypeId annotationTy = t; + + if (local->annotation) + { + annotationTy = resolveType(signatureScope, local->annotation, /* topLevel */ true); + addConstraint(signatureScope, local->annotation->location, SubtypeConstraint{t, annotationTy}); + } + else if (i < expectedArgPack.head.size()) + { + addConstraint(signatureScope, local->location, SubtypeConstraint{t, expectedArgPack.head[i]}); + } + + // HACK: This is the one case where the type of the definition will diverge from the type of the binding. + // We need to do this because there are cases where type refinements needs to have the information available + // at constraint generation time. + if (auto def = dfg->getDef(local)) + signatureScope->dcrRefinements[*def] = annotationTy; } TypePackId varargPack = nullptr; @@ -1654,22 +1861,28 @@ ConstraintGraphBuilder::FunctionSignature ConstraintGraphBuilder::checkFunctionS TypePackId annotationType = resolveTypePack(signatureScope, fn->varargAnnotation); varargPack = annotationType; } + else if (expectedArgPack.tail && get(*expectedArgPack.tail)) + varargPack = *expectedArgPack.tail; else - { - varargPack = arena->freshTypePack(signatureScope.get()); - } + varargPack = singletonTypes->anyTypePack; signatureScope->varargPack = varargPack; + bodyScope->varargPack = varargPack; } else { varargPack = arena->addTypePack(VariadicTypePack{singletonTypes->anyType, /*hidden*/ true}); // We do not add to signatureScope->varargPack because ... is not valid // in functions without an explicit ellipsis. + + signatureScope->varargPack = std::nullopt; + bodyScope->varargPack = std::nullopt; } LUAU_ASSERT(nullptr != varargPack); + // If there is both an annotation and an expected type, the annotation wins. + // Type checking will sort out any discrepancies later. if (fn->returnAnnotation) { TypePackId annotatedRetType = resolveTypePack(signatureScope, *fn->returnAnnotation); @@ -1680,26 +1893,11 @@ ConstraintGraphBuilder::FunctionSignature ConstraintGraphBuilder::checkFunctionS LUAU_ASSERT(get(returnType)); asMutable(returnType)->ty.emplace(annotatedRetType); } - - std::vector argTypes; - - for (AstLocal* local : fn->args) + else if (expectedFunction) { - TypeId t = freshType(signatureScope); - argTypes.push_back(t); - signatureScope->bindings[local] = Binding{t, local->location}; - - if (auto def = dfg->getDef(local)) - signatureScope->dcrRefinements[*def] = t; - - if (local->annotation) - { - TypeId argAnnotation = resolveType(signatureScope, local->annotation, /* topLevel */ true); - addConstraint(signatureScope, local->annotation->location, SubtypeConstraint{t, argAnnotation}); - } + asMutable(returnType)->ty.emplace(expectedFunction->retTypes); } - // TODO: Vararg annotation. // TODO: Preserve argument names in the function's type. FunctionTypeVar actualFunction{TypeLevel{}, parent.get(), arena->addTypePack(argTypes, varargPack), returnType}; @@ -1711,11 +1909,14 @@ ConstraintGraphBuilder::FunctionSignature ConstraintGraphBuilder::checkFunctionS LUAU_ASSERT(actualFunctionType); astTypes[fn] = actualFunctionType; + if (expectedType && get(*expectedType)) + { + asMutable(*expectedType)->ty.emplace(actualFunctionType); + } + return { /* signature */ actualFunctionType, - // Undo the workaround we made above: if there's no signature scope, - // don't report it. - /* signatureScope */ hasGenerics ? signatureScope : nullptr, + /* signatureScope */ signatureScope, /* bodyScope */ bodyScope, }; } diff --git a/Analysis/src/ConstraintSolver.cpp b/Analysis/src/ConstraintSolver.cpp index 250e7ae2..d59ea70a 100644 --- a/Analysis/src/ConstraintSolver.cpp +++ b/Analysis/src/ConstraintSolver.cpp @@ -1233,9 +1233,22 @@ bool ConstraintSolver::tryDispatch(const HasPropConstraint& c, NotNull(subjectType)) return block(subjectType, constraint); + if (get(subjectType)) + { + TableTypeVar& ttv = asMutable(subjectType)->ty.emplace(TableState::Free, TypeLevel{}, constraint->scope); + ttv.props[c.prop] = Property{c.resultType}; + asMutable(c.resultType)->ty.emplace(constraint->scope); + unblock(c.resultType); + return true; + } + std::optional resultType = lookupTableProp(subjectType, c.prop); if (!resultType) - return false; + { + asMutable(c.resultType)->ty.emplace(singletonTypes->errorRecoveryType()); + unblock(c.resultType); + return true; + } if (isBlocked(*resultType)) { @@ -1418,8 +1431,10 @@ bool ConstraintSolver::tryDispatch(const SingletonOrTopTypeConstraint& c, NotNul TypeId followed = follow(c.discriminantType); // `nil` is a singleton type too! There's only one value of type `nil`. - if (get(followed) || isNil(followed)) + if (c.negated && (get(followed) || isNil(followed))) *asMutable(c.resultType) = NegationTypeVar{c.discriminantType}; + else if (!c.negated && get(followed)) + *asMutable(c.resultType) = BoundTypeVar{c.discriminantType}; else *asMutable(c.resultType) = BoundTypeVar{singletonTypes->unknownType}; @@ -1509,17 +1524,17 @@ bool ConstraintSolver::tryDispatchIterableTable(TypeId iteratorTy, const Iterabl TypePackId expectedIterArgs = arena->addTypePack({iteratorTy}); unify(iterFtv->argTypes, expectedIterArgs, constraint->scope); - std::vector iterRets = flatten(*arena, singletonTypes, iterFtv->retTypes, 2); + TypePack iterRets = extendTypePack(*arena, singletonTypes, iterFtv->retTypes, 2); - if (iterRets.size() < 1) + if (iterRets.head.size() < 1) { // We've done what we can; this will get reported as an // error by the type checker. return true; } - TypeId nextFn = iterRets[0]; - TypeId table = iterRets.size() == 2 ? iterRets[1] : arena->freshType(constraint->scope); + TypeId nextFn = iterRets.head[0]; + TypeId table = iterRets.head.size() == 2 ? iterRets.head[1] : arena->freshType(constraint->scope); if (std::optional instantiatedNextFn = instantiation.substitute(nextFn)) { diff --git a/Analysis/src/DataFlowGraphBuilder.cpp b/Analysis/src/DataFlowGraph.cpp similarity index 92% rename from Analysis/src/DataFlowGraphBuilder.cpp rename to Analysis/src/DataFlowGraph.cpp index e2c4c285..cffd00c9 100644 --- a/Analysis/src/DataFlowGraphBuilder.cpp +++ b/Analysis/src/DataFlowGraph.cpp @@ -1,5 +1,5 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details -#include "Luau/DataFlowGraphBuilder.h" +#include "Luau/DataFlowGraph.h" #include "Luau/Error.h" @@ -11,6 +11,9 @@ namespace Luau std::optional DataFlowGraph::getDef(const AstExpr* expr) const { + // We need to skip through AstExprGroup because DFG doesn't try its best to transitively + while (auto group = expr->as()) + expr = group->expr; if (auto def = astDefs.find(expr)) return NotNull{*def}; return std::nullopt; @@ -52,16 +55,25 @@ std::optional DataFlowGraphBuilder::use(DfgScope* scope, Symbol symbol, A { for (DfgScope* current = scope; current; current = current->parent) { - if (auto loc = current->bindings.find(symbol)) + if (auto def = current->bindings.find(symbol)) { - graph.astDefs[e] = *loc; - return NotNull{*loc}; + graph.astDefs[e] = *def; + return NotNull{*def}; } } return std::nullopt; } +DefId DataFlowGraphBuilder::use(DefId def, AstExprIndexName* e) +{ + auto& propertyDef = props[def][e->index.value]; + if (!propertyDef) + propertyDef = arena->freshCell(def, e->index.value); + graph.astDefs[e] = propertyDef; + return NotNull{propertyDef}; +} + void DataFlowGraphBuilder::visit(DfgScope* scope, AstStatBlock* b) { DfgScope* child = childScope(scope); @@ -180,7 +192,7 @@ void DataFlowGraphBuilder::visit(DfgScope* scope, AstStatLocal* l) for (AstLocal* local : l->vars) { - DefId def = arena->freshDef(); + DefId def = arena->freshCell(); graph.localDefs[local] = def; scope->bindings[local] = def; } @@ -189,7 +201,7 @@ void DataFlowGraphBuilder::visit(DfgScope* scope, AstStatLocal* l) void DataFlowGraphBuilder::visit(DfgScope* scope, AstStatFor* f) { DfgScope* forScope = childScope(scope); // TODO: loop scope. - DefId def = arena->freshDef(); + DefId def = arena->freshCell(); graph.localDefs[f->var] = def; scope->bindings[f->var] = def; @@ -203,7 +215,7 @@ void DataFlowGraphBuilder::visit(DfgScope* scope, AstStatForIn* f) for (AstLocal* local : f->vars) { - DefId def = arena->freshDef(); + DefId def = arena->freshCell(); graph.localDefs[local] = def; forScope->bindings[local] = def; } @@ -245,7 +257,7 @@ void DataFlowGraphBuilder::visit(DfgScope* scope, AstStatAssign* a) // TODO global? if (auto exprLocal = root->as()) { - DefId def = arena->freshDef(); + DefId def = arena->freshCell(); graph.astDefs[exprLocal] = def; // Update the def in the scope that introduced the local. Not @@ -277,7 +289,7 @@ void DataFlowGraphBuilder::visit(DfgScope* scope, AstStatFunction* f) void DataFlowGraphBuilder::visit(DfgScope* scope, AstStatLocalFunction* l) { - DefId def = arena->freshDef(); + DefId def = arena->freshCell(); graph.localDefs[l->name] = def; scope->bindings[l->name] = def; @@ -354,8 +366,7 @@ ExpressionFlowGraph DataFlowGraphBuilder::visitExpr(DfgScope* scope, AstExprInde if (!def) return {}; - // TODO: properties for the above def. - return {}; + return {use(*def, i)}; } ExpressionFlowGraph DataFlowGraphBuilder::visitExpr(DfgScope* scope, AstExprIndexExpr* i) @@ -375,14 +386,14 @@ ExpressionFlowGraph DataFlowGraphBuilder::visitExpr(DfgScope* scope, AstExprFunc { if (AstLocal* self = f->self) { - DefId def = arena->freshDef(); + DefId def = arena->freshCell(); graph.localDefs[self] = def; scope->bindings[self] = def; } for (AstLocal* param : f->args) { - DefId def = arena->freshDef(); + DefId def = arena->freshCell(); graph.localDefs[param] = def; scope->bindings[param] = def; } diff --git a/Analysis/src/Def.cpp b/Analysis/src/Def.cpp index 935301c8..8ce1129c 100644 --- a/Analysis/src/Def.cpp +++ b/Analysis/src/Def.cpp @@ -4,9 +4,14 @@ namespace Luau { -DefId DefArena::freshDef() +DefId DefArena::freshCell() { - return NotNull{allocator.allocate(Undefined{})}; + return NotNull{allocator.allocate(Def{Cell{std::nullopt}})}; +} + +DefId DefArena::freshCell(DefId parent, const std::string& prop) +{ + return NotNull{allocator.allocate(Def{Cell{FieldMetadata{parent, prop}}})}; } } // namespace Luau diff --git a/Analysis/src/Error.cpp b/Analysis/src/Error.cpp index ed1a49cd..aefaa2c7 100644 --- a/Analysis/src/Error.cpp +++ b/Analysis/src/Error.cpp @@ -2,12 +2,14 @@ #include "Luau/Error.h" #include "Luau/Clone.h" +#include "Luau/Common.h" #include "Luau/StringUtils.h" #include "Luau/ToString.h" #include +#include -LUAU_FASTFLAGVARIABLE(LuauIceExceptionInheritanceChange, false) +LUAU_FASTFLAGVARIABLE(LuauTypeMismatchInvarianceInError, false) static std::string wrongNumberOfArgsString( size_t expectedCount, std::optional maximumCount, size_t actualCount, const char* argPrefix = nullptr, bool isVariadic = false) @@ -89,6 +91,7 @@ struct ErrorConverter if (result.empty()) result = "Type '" + givenTypeName + "' could not be converted into '" + wantedTypeName + "'"; + if (tm.error) { result += "\ncaused by:\n "; @@ -102,6 +105,10 @@ struct ErrorConverter { result += "; " + tm.reason; } + else if (FFlag::LuauTypeMismatchInvarianceInError && tm.context == TypeMismatch::InvariantContext) + { + result += " in an invariant context"; + } return result; } @@ -467,6 +474,11 @@ struct ErrorConverter { return "Type pack '" + toString(e.givenTp) + "' could not be converted into '" + toString(e.wantedTp) + "'"; } + + std::string operator()(const DynamicPropertyLookupOnClassesUnsafe& e) const + { + return "Attempting a dynamic property access on type '" + Luau::toString(e.ty) + "' is unsafe and may cause exceptions at runtime"; + } }; struct InvalidNameChecker @@ -514,6 +526,30 @@ TypeMismatch::TypeMismatch(TypeId wantedType, TypeId givenType, std::string reas { } +TypeMismatch::TypeMismatch(TypeId wantedType, TypeId givenType, TypeMismatch::Context context) + : wantedType(wantedType) + , givenType(givenType) + , context(context) +{ +} + +TypeMismatch::TypeMismatch(TypeId wantedType, TypeId givenType, std::string reason, TypeMismatch::Context context) + : wantedType(wantedType) + , givenType(givenType) + , context(context) + , reason(reason) +{ +} + +TypeMismatch::TypeMismatch(TypeId wantedType, TypeId givenType, std::string reason, std::optional error, TypeMismatch::Context context) + : wantedType(wantedType) + , givenType(givenType) + , context(context) + , reason(reason) + , error(error ? std::make_shared(std::move(*error)) : nullptr) +{ +} + bool TypeMismatch::operator==(const TypeMismatch& rhs) const { if (!!error != !!rhs.error) @@ -522,7 +558,7 @@ bool TypeMismatch::operator==(const TypeMismatch& rhs) const if (error && !(*error == *rhs.error)) return false; - return *wantedType == *rhs.wantedType && *givenType == *rhs.givenType && reason == rhs.reason; + return *wantedType == *rhs.wantedType && *givenType == *rhs.givenType && reason == rhs.reason && context == rhs.context; } bool UnknownSymbol::operator==(const UnknownSymbol& rhs) const @@ -662,7 +698,17 @@ bool FunctionExitsWithoutReturning::operator==(const FunctionExitsWithoutReturni int TypeError::code() const { - return 1000 + int(data.index()); + return minCode() + int(data.index()); +} + +int TypeError::minCode() +{ + return 1000; +} + +TypeErrorSummary TypeError::summary() const +{ + return TypeErrorSummary{location, moduleName, code()}; } bool TypeError::operator==(const TypeError& rhs) const @@ -730,6 +776,11 @@ bool TypePackMismatch::operator==(const TypePackMismatch& rhs) const return *wantedTp == *rhs.wantedTp && *givenTp == *rhs.givenTp; } +bool DynamicPropertyLookupOnClassesUnsafe::operator==(const DynamicPropertyLookupOnClassesUnsafe& rhs) const +{ + return ty == rhs.ty; +} + std::string toString(const TypeError& error) { return toString(error, TypeErrorToStringOptions{}); @@ -886,6 +937,8 @@ void copyError(T& e, TypeArena& destArena, CloneState cloneState) e.wantedTp = clone(e.wantedTp); e.givenTp = clone(e.givenTp); } + else if constexpr (std::is_same_v) + e.ty = clone(e.ty); else static_assert(always_false_v, "Non-exhaustive type switch"); } @@ -930,30 +983,4 @@ const char* InternalCompilerError::what() const throw() return this->message.data(); } -// TODO: Inline me when LuauIceExceptionInheritanceChange is deleted. -void throwRuntimeError(const std::string& message) -{ - if (FFlag::LuauIceExceptionInheritanceChange) - { - throw InternalCompilerError(message); - } - else - { - throw std::runtime_error(message); - } -} - -// TODO: Inline me when LuauIceExceptionInheritanceChange is deleted. -void throwRuntimeError(const std::string& message, const std::string& moduleName) -{ - if (FFlag::LuauIceExceptionInheritanceChange) - { - throw InternalCompilerError(message, moduleName); - } - else - { - throw std::runtime_error(message); - } -} - } // namespace Luau diff --git a/Analysis/src/Frontend.cpp b/Analysis/src/Frontend.cpp index 22a9ecfa..356ced0b 100644 --- a/Analysis/src/Frontend.cpp +++ b/Analysis/src/Frontend.cpp @@ -7,7 +7,7 @@ #include "Luau/Config.h" #include "Luau/ConstraintGraphBuilder.h" #include "Luau/ConstraintSolver.h" -#include "Luau/DataFlowGraphBuilder.h" +#include "Luau/DataFlowGraph.h" #include "Luau/DcrLogger.h" #include "Luau/FileResolver.h" #include "Luau/Parser.h" @@ -31,7 +31,6 @@ LUAU_FASTINTVARIABLE(LuauAutocompleteCheckTimeoutMs, 100) LUAU_FASTFLAGVARIABLE(DebugLuauDeferredConstraintResolution, false) LUAU_FASTFLAG(DebugLuauLogSolverToJson); LUAU_FASTFLAGVARIABLE(LuauFixMarkDirtyReverseDeps, false) -LUAU_FASTFLAGVARIABLE(LuauPersistTypesAfterGeneratingDocSyms, false) namespace Luau { @@ -112,57 +111,32 @@ LoadDefinitionFileResult Frontend::loadDefinitionFile(std::string_view source, c CloneState cloneState; - if (FFlag::LuauPersistTypesAfterGeneratingDocSyms) + std::vector typesToPersist; + typesToPersist.reserve(checkedModule->declaredGlobals.size() + checkedModule->getModuleScope()->exportedTypeBindings.size()); + + for (const auto& [name, ty] : checkedModule->declaredGlobals) { - std::vector typesToPersist; - typesToPersist.reserve(checkedModule->declaredGlobals.size() + checkedModule->getModuleScope()->exportedTypeBindings.size()); + TypeId globalTy = clone(ty, globalTypes, cloneState); + std::string documentationSymbol = packageName + "/global/" + name; + generateDocumentationSymbols(globalTy, documentationSymbol); + globalScope->bindings[typeChecker.globalNames.names->getOrAdd(name.c_str())] = {globalTy, Location(), false, {}, documentationSymbol}; - for (const auto& [name, ty] : checkedModule->declaredGlobals) - { - TypeId globalTy = clone(ty, globalTypes, cloneState); - std::string documentationSymbol = packageName + "/global/" + name; - generateDocumentationSymbols(globalTy, documentationSymbol); - globalScope->bindings[typeChecker.globalNames.names->getOrAdd(name.c_str())] = {globalTy, Location(), false, {}, documentationSymbol}; - - typesToPersist.push_back(globalTy); - } - - for (const auto& [name, ty] : checkedModule->getModuleScope()->exportedTypeBindings) - { - TypeFun globalTy = clone(ty, globalTypes, cloneState); - std::string documentationSymbol = packageName + "/globaltype/" + name; - generateDocumentationSymbols(globalTy.type, documentationSymbol); - globalScope->exportedTypeBindings[name] = globalTy; - - typesToPersist.push_back(globalTy.type); - } - - for (TypeId ty : typesToPersist) - { - persist(ty); - } + typesToPersist.push_back(globalTy); } - else + + for (const auto& [name, ty] : checkedModule->getModuleScope()->exportedTypeBindings) { - for (const auto& [name, ty] : checkedModule->declaredGlobals) - { - TypeId globalTy = clone(ty, globalTypes, cloneState); - std::string documentationSymbol = packageName + "/global/" + name; - generateDocumentationSymbols(globalTy, documentationSymbol); - globalScope->bindings[typeChecker.globalNames.names->getOrAdd(name.c_str())] = {globalTy, Location(), false, {}, documentationSymbol}; + TypeFun globalTy = clone(ty, globalTypes, cloneState); + std::string documentationSymbol = packageName + "/globaltype/" + name; + generateDocumentationSymbols(globalTy.type, documentationSymbol); + globalScope->exportedTypeBindings[name] = globalTy; - persist(globalTy); - } + typesToPersist.push_back(globalTy.type); + } - for (const auto& [name, ty] : checkedModule->getModuleScope()->exportedTypeBindings) - { - TypeFun globalTy = clone(ty, globalTypes, cloneState); - std::string documentationSymbol = packageName + "/globaltype/" + name; - generateDocumentationSymbols(globalTy.type, documentationSymbol); - globalScope->exportedTypeBindings[name] = globalTy; - - persist(globalTy.type); - } + for (TypeId ty : typesToPersist) + { + persist(ty); } return LoadDefinitionFileResult{true, parseResult, checkedModule}; @@ -194,57 +168,32 @@ LoadDefinitionFileResult loadDefinitionFile(TypeChecker& typeChecker, ScopePtr t CloneState cloneState; - if (FFlag::LuauPersistTypesAfterGeneratingDocSyms) + std::vector typesToPersist; + typesToPersist.reserve(checkedModule->declaredGlobals.size() + checkedModule->getModuleScope()->exportedTypeBindings.size()); + + for (const auto& [name, ty] : checkedModule->declaredGlobals) { - std::vector typesToPersist; - typesToPersist.reserve(checkedModule->declaredGlobals.size() + checkedModule->getModuleScope()->exportedTypeBindings.size()); + TypeId globalTy = clone(ty, typeChecker.globalTypes, cloneState); + std::string documentationSymbol = packageName + "/global/" + name; + generateDocumentationSymbols(globalTy, documentationSymbol); + targetScope->bindings[typeChecker.globalNames.names->getOrAdd(name.c_str())] = {globalTy, Location(), false, {}, documentationSymbol}; - for (const auto& [name, ty] : checkedModule->declaredGlobals) - { - TypeId globalTy = clone(ty, typeChecker.globalTypes, cloneState); - std::string documentationSymbol = packageName + "/global/" + name; - generateDocumentationSymbols(globalTy, documentationSymbol); - targetScope->bindings[typeChecker.globalNames.names->getOrAdd(name.c_str())] = {globalTy, Location(), false, {}, documentationSymbol}; - - typesToPersist.push_back(globalTy); - } - - for (const auto& [name, ty] : checkedModule->getModuleScope()->exportedTypeBindings) - { - TypeFun globalTy = clone(ty, typeChecker.globalTypes, cloneState); - std::string documentationSymbol = packageName + "/globaltype/" + name; - generateDocumentationSymbols(globalTy.type, documentationSymbol); - targetScope->exportedTypeBindings[name] = globalTy; - - typesToPersist.push_back(globalTy.type); - } - - for (TypeId ty : typesToPersist) - { - persist(ty); - } + typesToPersist.push_back(globalTy); } - else + + for (const auto& [name, ty] : checkedModule->getModuleScope()->exportedTypeBindings) { - for (const auto& [name, ty] : checkedModule->declaredGlobals) - { - TypeId globalTy = clone(ty, typeChecker.globalTypes, cloneState); - std::string documentationSymbol = packageName + "/global/" + name; - generateDocumentationSymbols(globalTy, documentationSymbol); - targetScope->bindings[typeChecker.globalNames.names->getOrAdd(name.c_str())] = {globalTy, Location(), false, {}, documentationSymbol}; + TypeFun globalTy = clone(ty, typeChecker.globalTypes, cloneState); + std::string documentationSymbol = packageName + "/globaltype/" + name; + generateDocumentationSymbols(globalTy.type, documentationSymbol); + targetScope->exportedTypeBindings[name] = globalTy; - persist(globalTy); - } + typesToPersist.push_back(globalTy.type); + } - for (const auto& [name, ty] : checkedModule->getModuleScope()->exportedTypeBindings) - { - TypeFun globalTy = clone(ty, typeChecker.globalTypes, cloneState); - std::string documentationSymbol = packageName + "/globaltype/" + name; - generateDocumentationSymbols(globalTy.type, documentationSymbol); - targetScope->exportedTypeBindings[name] = globalTy; - - persist(globalTy.type); - } + for (TypeId ty : typesToPersist) + { + persist(ty); } return LoadDefinitionFileResult{true, parseResult, checkedModule}; @@ -493,13 +442,13 @@ CheckResult Frontend::check(const ModuleName& name, std::optionalsecond == nullptr) - throwRuntimeError("Frontend::modules does not have data for " + name, name); + throw InternalCompilerError("Frontend::modules does not have data for " + name, name); } else { auto it2 = moduleResolver.modules.find(name); if (it2 == moduleResolver.modules.end() || it2->second == nullptr) - throwRuntimeError("Frontend::modules does not have data for " + name, name); + throw InternalCompilerError("Frontend::modules does not have data for " + name, name); } return CheckResult{ @@ -606,7 +555,7 @@ CheckResult Frontend::check(const ModuleName& name, std::optional) stream << "TypePackMismatch { wanted = '" + toString(err.wantedTp) + "', given = '" + toString(err.givenTp) + "' }"; + else if constexpr (std::is_same_v) + stream << "DynamicPropertyLookupOnClassesUnsafe { " << toString(err.ty) << " }"; else static_assert(always_false_v, "Non-exhaustive type switch"); } diff --git a/Analysis/src/Normalize.cpp b/Analysis/src/Normalize.cpp index 21e9f787..fa3503fd 100644 --- a/Analysis/src/Normalize.cpp +++ b/Analysis/src/Normalize.cpp @@ -24,6 +24,7 @@ LUAU_FASTFLAGVARIABLE(LuauNegatedFunctionTypes, false); LUAU_FASTFLAG(LuauUnknownAndNeverType) LUAU_FASTFLAG(DebugLuauDeferredConstraintResolution) LUAU_FASTFLAG(LuauOverloadedFunctionSubtypingPerf); +LUAU_FASTFLAG(LuauUninhabitedSubAnything) namespace Luau { @@ -240,13 +241,75 @@ NormalizedType::NormalizedType(NotNull singletonTypes) { } -static bool isInhabited(const NormalizedType& norm) +static bool isShallowInhabited(const NormalizedType& norm) { + // This test is just a shallow check, for example it returns `true` for `{ p : never }` return !get(norm.tops) || !get(norm.booleans) || !norm.classes.empty() || !get(norm.errors) || !get(norm.nils) || !get(norm.numbers) || !norm.strings.isNever() || !get(norm.threads) || !norm.functions.isNever() || !norm.tables.empty() || !norm.tyvars.empty(); } +bool isInhabited_DEPRECATED(const NormalizedType& norm) +{ + LUAU_ASSERT(!FFlag::LuauUninhabitedSubAnything); + return isShallowInhabited(norm); +} + +bool Normalizer::isInhabited(const NormalizedType* norm, std::unordered_set seen) +{ + if (!get(norm->tops) || !get(norm->booleans) || !get(norm->errors) || + !get(norm->nils) || !get(norm->numbers) || !get(norm->threads) || + !norm->classes.empty() || !norm->strings.isNever() || !norm->functions.isNever()) + return true; + + for (const auto& [_, intersect] : norm->tyvars) + { + if (isInhabited(intersect.get(), seen)) + return true; + } + + for (TypeId table : norm->tables) + { + if (isInhabited(table, seen)) + return true; + } + + return false; +} + +bool Normalizer::isInhabited(TypeId ty, std::unordered_set seen) +{ + // TODO: use log.follow(ty), CLI-64291 + ty = follow(ty); + + if (get(ty)) + return false; + + if (!get(ty) && !get(ty) && !get(ty) && !get(ty)) + return true; + + if (seen.count(ty)) + return true; + + seen.insert(ty); + + if (const TableTypeVar* ttv = get(ty)) + { + for (const auto& [_, prop] : ttv->props) + { + if (!isInhabited(prop.type, seen)) + return false; + } + return true; + } + + if (const MetatableTypeVar* mtv = get(ty)) + return isInhabited(mtv->table, seen) && isInhabited(mtv->metatable, seen); + + const NormalizedType* norm = normalize(ty); + return isInhabited(norm, seen); +} + static int tyvarIndex(TypeId ty) { if (const GenericTypeVar* gtv = get(ty)) @@ -378,7 +441,7 @@ static bool isNormalizedTyvar(const NormalizedTyvars& tyvars) { if (!isPlainTyvar(tyvar)) return false; - if (!isInhabited(*intersect)) + if (!isShallowInhabited(*intersect)) return false; for (auto& [other, _] : intersect->tyvars) if (tyvarIndex(other) <= tyvarIndex(tyvar)) @@ -1852,7 +1915,7 @@ bool Normalizer::intersectTyvarsWithTy(NormalizedTyvars& here, TypeId there) NormalizedType& inter = *it->second; if (!intersectNormalWithTy(inter, there)) return false; - if (isInhabited(inter)) + if (isShallowInhabited(inter)) ++it; else it = here.erase(it); @@ -1914,7 +1977,7 @@ bool Normalizer::intersectNormals(NormalizedType& here, const NormalizedType& th if (!intersectNormals(inter, *found->second, index)) return false; } - if (isInhabited(inter)) + if (isShallowInhabited(inter)) it++; else it = here.tyvars.erase(it); diff --git a/Analysis/src/ToString.cpp b/Analysis/src/ToString.cpp index fe5f5690..ddcc15a8 100644 --- a/Analysis/src/ToString.cpp +++ b/Analysis/src/ToString.cpp @@ -11,8 +11,8 @@ #include LUAU_FASTFLAG(DebugLuauDeferredConstraintResolution) -LUAU_FASTFLAG(LuauLvaluelessPath) LUAU_FASTFLAG(LuauUnknownAndNeverType) +LUAU_FASTFLAGVARIABLE(LuauLineBreaksDetermineIndents, false) LUAU_FASTFLAGVARIABLE(LuauFunctionReturnStringificationFixup, false) LUAU_FASTFLAGVARIABLE(LuauUnseeArrayTtv, false) @@ -272,10 +272,20 @@ struct StringifierState private: void emitIndentation() { - if (!opts.indent) - return; + if (!FFlag::LuauLineBreaksDetermineIndents) + { + if (!opts.DEPRECATED_indent) + return; - emit(std::string(indentation, ' ')); + emit(std::string(indentation, ' ')); + } + else + { + if (!opts.useLineBreaks) + return; + + emit(std::string(indentation, ' ')); + } } }; @@ -445,7 +455,7 @@ struct TypeVarStringifier return; default: LUAU_ASSERT(!"Unknown primitive type"); - throwRuntimeError("Unknown primitive type " + std::to_string(ptv.type)); + throw InternalCompilerError("Unknown primitive type " + std::to_string(ptv.type)); } } @@ -462,7 +472,7 @@ struct TypeVarStringifier else { LUAU_ASSERT(!"Unknown singleton type"); - throwRuntimeError("Unknown singleton type"); + throw InternalCompilerError("Unknown singleton type"); } } @@ -508,24 +518,13 @@ struct TypeVarStringifier bool plural = true; - if (FFlag::LuauFunctionReturnStringificationFixup) + auto retBegin = begin(ftv.retTypes); + auto retEnd = end(ftv.retTypes); + if (retBegin != retEnd) { - auto retBegin = begin(ftv.retTypes); - auto retEnd = end(ftv.retTypes); - if (retBegin != retEnd) - { - ++retBegin; - if (retBegin == retEnd && !retBegin.tail()) - plural = false; - } - } - else - { - if (auto retPack = get(follow(ftv.retTypes))) - { - if (retPack->head.size() == 1 && !retPack->tail) - plural = false; - } + ++retBegin; + if (retBegin == retEnd && !retBegin.tail()) + plural = false; } if (plural) @@ -980,8 +979,6 @@ struct TypePackStringifier void operator()(TypePackId tp, const GenericTypePack& pack) { - if (FFlag::DebugLuauVerboseTypeNames) - state.emit("gen-"); if (pack.explicitName) { state.usedNames.insert(pack.name); @@ -992,6 +989,15 @@ struct TypePackStringifier { state.emit(state.getName(tp)); } + + if (FFlag::DebugLuauVerboseTypeNames) + { + state.emit("-"); + if (FFlag::DebugLuauDeferredConstraintResolution) + state.emitLevel(pack.scope); + else + state.emit(pack.level); + } state.emit("..."); } @@ -1143,7 +1149,7 @@ ToStringResult toStringDetailed(TypeId ty, ToStringOptions& opts) else tvs.stringify(ty); - if (!state.cycleNames.empty()) + if (!state.cycleNames.empty() || !state.cycleTpNames.empty()) { result.cycle = true; state.emit(" where "); @@ -1176,6 +1182,29 @@ ToStringResult toStringDetailed(TypeId ty, ToStringOptions& opts) semi = true; } + std::vector> sortedCycleTpNames(state.cycleTpNames.begin(), state.cycleTpNames.end()); + std::sort(sortedCycleTpNames.begin(), sortedCycleTpNames.end(), [](const auto& a, const auto& b) { + return a.second < b.second; + }); + + TypePackStringifier tps{state}; + + for (const auto& [cycleTp, name] : sortedCycleTpNames) + { + if (semi) + state.emit(" ; "); + + state.emit(name); + state.emit(" = "); + Luau::visit( + [&tps, cycleTy = cycleTp](auto&& t) { + return tps(cycleTy, t); + }, + cycleTp->ty); + + semi = true; + } + if (opts.maxTypeLength > 0 && result.name.length() > opts.maxTypeLength) { result.truncated = true; @@ -1361,22 +1390,30 @@ std::string toStringNamedFunction(const std::string& funcName, const FunctionTyp return result.name; } +static ToStringOptions& dumpOptions() +{ + static ToStringOptions opts = ([]() { + ToStringOptions o; + o.exhaustive = true; + o.functionTypeArguments = true; + o.maxTableLength = 0; + o.maxTypeLength = 0; + return o; + })(); + + return opts; +} + std::string dump(TypeId ty) { - ToStringOptions opts; - opts.exhaustive = true; - opts.functionTypeArguments = true; - std::string s = toString(ty, opts); + std::string s = toString(ty, dumpOptions()); printf("%s\n", s.c_str()); return s; } std::string dump(TypePackId ty) { - ToStringOptions opts; - opts.exhaustive = true; - opts.functionTypeArguments = true; - std::string s = toString(ty, opts); + std::string s = toString(ty, dumpOptions()); printf("%s\n", s.c_str()); return s; } @@ -1391,10 +1428,7 @@ std::string dump(const ScopePtr& scope, const char* name) } TypeId ty = binding->typeId; - ToStringOptions opts; - opts.exhaustive = true; - opts.functionTypeArguments = true; - std::string s = toString(ty, opts); + std::string s = toString(ty, dumpOptions()); printf("%s\n", s.c_str()); return s; } @@ -1413,8 +1447,7 @@ std::string toString(const Constraint& constraint, ToStringOptions& opts) auto go = [&opts](auto&& c) -> std::string { using T = std::decay_t; - auto tos = [&opts](auto&& a) - { + auto tos = [&opts](auto&& a) { return toString(a, opts); }; @@ -1480,8 +1513,7 @@ std::string toString(const Constraint& constraint, ToStringOptions& opts) } else if constexpr (std::is_same_v) { - return tos(c.resultType) + " ~ prim " + tos(c.expectedType) + ", " + tos(c.singletonType) + ", " + - tos(c.multitonType); + return tos(c.resultType) + " ~ prim " + tos(c.expectedType) + ", " + tos(c.singletonType) + ", " + tos(c.multitonType); } else if constexpr (std::is_same_v) { @@ -1497,7 +1529,10 @@ std::string toString(const Constraint& constraint, ToStringOptions& opts) std::string result = tos(c.resultType); std::string discriminant = tos(c.discriminantType); - return result + " ~ if isSingleton D then ~D else unknown where D = " + discriminant; + if (c.negated) + return result + " ~ if isSingleton D then ~D else unknown where D = " + discriminant; + else + return result + " ~ if isSingleton D then D else unknown where D = " + discriminant; } else static_assert(always_false_v, "Non-exhaustive constraint switch"); @@ -1516,28 +1551,8 @@ std::string dump(const Constraint& c) return s; } -std::string toString(const LValue& lvalue) -{ - LUAU_ASSERT(!FFlag::LuauLvaluelessPath); - - std::string s; - for (const LValue* current = &lvalue; current; current = baseof(*current)) - { - if (auto field = get(*current)) - s = "." + field->key + s; - else if (auto symbol = get(*current)) - s = toString(*symbol) + s; - else - LUAU_ASSERT(!"Unknown LValue"); - } - - return s; -} - std::optional getFunctionNameAsString(const AstExpr& expr) { - LUAU_ASSERT(FFlag::LuauLvaluelessPath); - const AstExpr* curr = &expr; std::string s; diff --git a/Analysis/src/TopoSortStatements.cpp b/Analysis/src/TopoSortStatements.cpp index 052c10de..fbf74115 100644 --- a/Analysis/src/TopoSortStatements.cpp +++ b/Analysis/src/TopoSortStatements.cpp @@ -150,7 +150,7 @@ Identifier mkName(const AstStatFunction& function) auto name = mkName(*function.name); LUAU_ASSERT(bool(name)); if (!name) - throwRuntimeError("Internal error: Function declaration has a bad name"); + throw InternalCompilerError("Internal error: Function declaration has a bad name"); return *name; } @@ -256,7 +256,7 @@ struct ArcCollector : public AstVisitor { auto name = mkName(*node->name); if (!name) - throwRuntimeError("Internal error: AstStatFunction has a bad name"); + throw InternalCompilerError("Internal error: AstStatFunction has a bad name"); add(*name); return true; diff --git a/Analysis/src/TxnLog.cpp b/Analysis/src/TxnLog.cpp index 034aeaec..1a73b049 100644 --- a/Analysis/src/TxnLog.cpp +++ b/Analysis/src/TxnLog.cpp @@ -78,6 +78,42 @@ void TxnLog::concat(TxnLog rhs) typePackChanges[tp] = std::move(rep); } +void TxnLog::concatAsIntersections(TxnLog rhs, NotNull arena) +{ + for (auto& [ty, rightRep] : rhs.typeVarChanges) + { + if (auto leftRep = typeVarChanges.find(ty)) + { + TypeId leftTy = arena->addType((*leftRep)->pending); + TypeId rightTy = arena->addType(rightRep->pending); + typeVarChanges[ty]->pending.ty = IntersectionTypeVar{{leftTy, rightTy}}; + } + else + typeVarChanges[ty] = std::move(rightRep); + } + + for (auto& [tp, rep] : rhs.typePackChanges) + typePackChanges[tp] = std::move(rep); +} + +void TxnLog::concatAsUnion(TxnLog rhs, NotNull arena) +{ + for (auto& [ty, rightRep] : rhs.typeVarChanges) + { + if (auto leftRep = typeVarChanges.find(ty)) + { + TypeId leftTy = arena->addType((*leftRep)->pending); + TypeId rightTy = arena->addType(rightRep->pending); + typeVarChanges[ty]->pending.ty = UnionTypeVar{{leftTy, rightTy}}; + } + else + typeVarChanges[ty] = std::move(rightRep); + } + + for (auto& [tp, rep] : rhs.typePackChanges) + typePackChanges[tp] = std::move(rep); +} + void TxnLog::commit() { for (auto& [ty, rep] : typeVarChanges) diff --git a/Analysis/src/TypeAttach.cpp b/Analysis/src/TypeAttach.cpp index c97ed05d..e483c047 100644 --- a/Analysis/src/TypeAttach.cpp +++ b/Analysis/src/TypeAttach.cpp @@ -341,7 +341,7 @@ public: AstType* operator()(const NegationTypeVar& ntv) { // FIXME: do the same thing we do with ErrorTypeVar - throwRuntimeError("Cannot convert NegationTypeVar into AstNode"); + throw InternalCompilerError("Cannot convert NegationTypeVar into AstNode"); } private: diff --git a/Analysis/src/TypeChecker2.cpp b/Analysis/src/TypeChecker2.cpp index 35493bdb..84c0ca3b 100644 --- a/Analysis/src/TypeChecker2.cpp +++ b/Analysis/src/TypeChecker2.cpp @@ -295,11 +295,11 @@ struct TypeChecker2 Scope* scope = findInnermostScope(ret->location); TypePackId expectedRetType = scope->returnType; - TypeArena arena; - TypePackId actualRetType = reconstructPack(ret->list, arena); + TypeArena* arena = &module->internalTypes; + TypePackId actualRetType = reconstructPack(ret->list, *arena); UnifierSharedState sharedState{&ice}; - Normalizer normalizer{&arena, singletonTypes, NotNull{&sharedState}}; + Normalizer normalizer{arena, singletonTypes, NotNull{&sharedState}}; Unifier u{NotNull{&normalizer}, Mode::Strict, stack.back(), ret->location, Covariant}; u.tryUnify(actualRetType, expectedRetType); @@ -424,13 +424,13 @@ struct TypeChecker2 TypePackId iteratorPack = arena.addTypePack(valueTypes, iteratorTail); // ... and then expand it out to 3 values (if possible) - const std::vector iteratorTypes = flatten(arena, singletonTypes, iteratorPack, 3); - if (iteratorTypes.empty()) + TypePack iteratorTypes = extendTypePack(arena, singletonTypes, iteratorPack, 3); + if (iteratorTypes.head.empty()) { reportError(GenericError{"for..in loops require at least one value to iterate over. Got zero"}, getLocation(forInStatement->values)); return; } - TypeId iteratorTy = follow(iteratorTypes[0]); + TypeId iteratorTy = follow(iteratorTypes.head[0]); auto checkFunction = [this, &arena, &scope, &forInStatement, &variableTypes]( const FunctionTypeVar* iterFtv, std::vector iterTys, bool isMm) { @@ -445,8 +445,8 @@ struct TypeChecker2 } // It is okay if there aren't enough iterators, but the iteratee must provide enough. - std::vector expectedVariableTypes = flatten(arena, singletonTypes, iterFtv->retTypes, variableTypes.size()); - if (expectedVariableTypes.size() < variableTypes.size()) + TypePack expectedVariableTypes = extendTypePack(arena, singletonTypes, iterFtv->retTypes, variableTypes.size()); + if (expectedVariableTypes.head.size() < variableTypes.size()) { if (isMm) reportError( @@ -455,8 +455,8 @@ struct TypeChecker2 reportError(GenericError{"next() does not return enough values"}, forInStatement->values.data[0]->location); } - for (size_t i = 0; i < std::min(expectedVariableTypes.size(), variableTypes.size()); ++i) - reportErrors(tryUnify(scope, forInStatement->vars.data[i]->location, variableTypes[i], expectedVariableTypes[i])); + for (size_t i = 0; i < std::min(expectedVariableTypes.head.size(), variableTypes.size()); ++i) + reportErrors(tryUnify(scope, forInStatement->vars.data[i]->location, variableTypes[i], expectedVariableTypes.head[i])); // nextFn is going to be invoked with (arrayTy, startIndexTy) @@ -477,25 +477,25 @@ struct TypeChecker2 if (maxCount && *maxCount < 2) reportError(CountMismatch{2, std::nullopt, *maxCount, CountMismatch::Arg}, forInStatement->vars.data[0]->location); - const std::vector flattenedArgTypes = flatten(arena, singletonTypes, iterFtv->argTypes, 2); + TypePack flattenedArgTypes = extendTypePack(arena, singletonTypes, iterFtv->argTypes, 2); size_t firstIterationArgCount = iterTys.empty() ? 0 : iterTys.size() - 1; - size_t actualArgCount = expectedVariableTypes.size(); + size_t actualArgCount = expectedVariableTypes.head.size(); if (firstIterationArgCount < minCount) reportError(CountMismatch{2, std::nullopt, firstIterationArgCount, CountMismatch::Arg}, forInStatement->vars.data[0]->location); else if (actualArgCount < minCount) reportError(CountMismatch{2, std::nullopt, actualArgCount, CountMismatch::Arg}, forInStatement->vars.data[0]->location); - if (iterTys.size() >= 2 && flattenedArgTypes.size() > 0) + if (iterTys.size() >= 2 && flattenedArgTypes.head.size() > 0) { size_t valueIndex = forInStatement->values.size > 1 ? 1 : 0; - reportErrors(tryUnify(scope, forInStatement->values.data[valueIndex]->location, iterTys[1], flattenedArgTypes[0])); + reportErrors(tryUnify(scope, forInStatement->values.data[valueIndex]->location, iterTys[1], flattenedArgTypes.head[0])); } - if (iterTys.size() == 3 && flattenedArgTypes.size() > 1) + if (iterTys.size() == 3 && flattenedArgTypes.head.size() > 1) { size_t valueIndex = forInStatement->values.size > 2 ? 2 : 0; - reportErrors(tryUnify(scope, forInStatement->values.data[valueIndex]->location, iterTys[2], flattenedArgTypes[1])); + reportErrors(tryUnify(scope, forInStatement->values.data[valueIndex]->location, iterTys[2], flattenedArgTypes.head[1])); } }; @@ -516,7 +516,7 @@ struct TypeChecker2 */ if (const FunctionTypeVar* nextFn = get(iteratorTy)) { - checkFunction(nextFn, iteratorTypes, false); + checkFunction(nextFn, iteratorTypes.head, false); } else if (const TableTypeVar* ttv = get(iteratorTy)) { @@ -545,19 +545,19 @@ struct TypeChecker2 TypePackId argPack = arena.addTypePack({iteratorTy}); reportErrors(tryUnify(scope, forInStatement->values.data[0]->location, argPack, iterMmFtv->argTypes)); - std::vector mmIteratorTypes = flatten(arena, singletonTypes, iterMmFtv->retTypes, 3); + TypePack mmIteratorTypes = extendTypePack(arena, singletonTypes, iterMmFtv->retTypes, 3); - if (mmIteratorTypes.size() == 0) + if (mmIteratorTypes.head.size() == 0) { reportError(GenericError{"__iter must return at least one value"}, forInStatement->values.data[0]->location); return; } - TypeId nextFn = follow(mmIteratorTypes[0]); + TypeId nextFn = follow(mmIteratorTypes.head[0]); if (std::optional instantiatedNextFn = instantiation.substitute(nextFn)) { - std::vector instantiatedIteratorTypes = mmIteratorTypes; + std::vector instantiatedIteratorTypes = mmIteratorTypes.head; instantiatedIteratorTypes[0] = *instantiatedNextFn; if (const FunctionTypeVar* nextFtv = get(*instantiatedNextFn)) @@ -800,8 +800,8 @@ struct TypeChecker2 for (AstExpr* arg : call->args) visit(arg); - TypeArena arena; - Instantiation instantiation{TxnLog::empty(), &arena, TypeLevel{}, stack.back()}; + TypeArena* arena = &module->internalTypes; + Instantiation instantiation{TxnLog::empty(), arena, TypeLevel{}, stack.back()}; TypePackId expectedRetType = lookupPack(call); TypeId functionType = lookupType(call->func); @@ -845,30 +845,70 @@ struct TypeChecker2 return; } } + else if (auto utv = get(functionType)) + { + // Sometimes it's okay to call a union of functions, but only if all of the functions are the same. + std::optional fst; + for (TypeId ty : utv) + { + if (!fst) + fst = follow(ty); + else if (fst != follow(ty)) + { + reportError(CannotCallNonFunction{functionType}, call->func->location); + return; + } + } + + if (!fst) + ice.ice("UnionTypeVar had no elements, so fst is nullopt?"); + + if (std::optional instantiatedFunctionType = instantiation.substitute(*fst)) + { + testFunctionType = *instantiatedFunctionType; + } + else + { + reportError(UnificationTooComplex{}, call->func->location); + return; + } + } else { reportError(CannotCallNonFunction{functionType}, call->func->location); return; } - for (AstExpr* arg : call->args) - { - TypeId argTy = lookupType(arg); - args.head.push_back(argTy); - } - if (call->self) { AstExprIndexName* indexExpr = call->func->as(); if (!indexExpr) ice.ice("method call expression has no 'self'"); - args.head.insert(args.head.begin(), lookupType(indexExpr->expr)); + args.head.push_back(lookupType(indexExpr->expr)); } - TypePackId argsTp = arena.addTypePack(args); + for (size_t i = 0; i < call->args.size; ++i) + { + AstExpr* arg = call->args.data[i]; + TypeId* argTy = module->astTypes.find(arg); + if (argTy) + args.head.push_back(*argTy); + else if (i == call->args.size - 1) + { + TypePackId* argTail = module->astTypePacks.find(arg); + if (argTail) + args.tail = *argTail; + else + args.tail = singletonTypes->anyTypePack; + } + else + args.head.push_back(singletonTypes->anyType); + } + + TypePackId argsTp = arena->addTypePack(args); FunctionTypeVar ftv{argsTp, expectedRetType}; - TypeId expectedType = arena.addType(ftv); + TypeId expectedType = arena->addType(ftv); if (!isSubtype(testFunctionType, expectedType, stack.back())) { @@ -881,19 +921,7 @@ struct TypeChecker2 void visit(AstExprIndexName* indexName) { TypeId leftType = lookupType(indexName->expr); - TypeId resultType = lookupType(indexName); - - // leftType must have a property called indexName->index - - std::optional ty = - getIndexTypeFromType(module->getModuleScope(), leftType, indexName->index.value, indexName->location, /* addErrors */ true); - if (ty) - { - if (!isSubtype(resultType, *ty, stack.back())) - { - reportError(TypeMismatch{resultType, *ty}, indexName->location); - } - } + getIndexTypeFromType(module->getModuleScope(), leftType, indexName->index.value, indexName->location, /* addErrors */ true); } void visit(AstExprIndexExpr* indexExpr) @@ -1085,7 +1113,7 @@ struct TypeChecker2 if (mm) { - if (const FunctionTypeVar* ftv = get(*mm)) + if (const FunctionTypeVar* ftv = get(follow(*mm))) { TypePackId expectedArgs; // For >= and > we invoke __lt and __le respectively with diff --git a/Analysis/src/TypeInfer.cpp b/Analysis/src/TypeInfer.cpp index dfd08e54..9f64a601 100644 --- a/Analysis/src/TypeInfer.cpp +++ b/Analysis/src/TypeInfer.cpp @@ -35,11 +35,12 @@ LUAU_FASTFLAG(LuauTypeNormalization2) LUAU_FASTFLAGVARIABLE(DebugLuauFreezeDuringUnification, false) LUAU_FASTFLAGVARIABLE(LuauReturnAnyInsteadOfICE, false) // Eventually removed as false. LUAU_FASTFLAGVARIABLE(DebugLuauSharedSelf, false) -LUAU_FASTFLAGVARIABLE(LuauLvaluelessPath, false) LUAU_FASTFLAGVARIABLE(LuauNilIterator, false) LUAU_FASTFLAGVARIABLE(LuauUnknownAndNeverType, false) +LUAU_FASTFLAGVARIABLE(LuauTypeInferMissingFollows, false) LUAU_FASTFLAGVARIABLE(LuauBinaryNeedsExpectedTypesToo, false) LUAU_FASTFLAGVARIABLE(LuauNeverTypesAndOperatorsInference, false) +LUAU_FASTFLAGVARIABLE(LuauFollowInLvalueIndexCheck, false) LUAU_FASTFLAGVARIABLE(LuauReturnsFromCallsitesAreNotWidened, false) LUAU_FASTFLAGVARIABLE(LuauTryhardAnd, false) LUAU_FASTFLAG(LuauInstantiateInSubtyping) @@ -47,18 +48,15 @@ LUAU_FASTFLAGVARIABLE(LuauCompleteVisitor, false) LUAU_FASTFLAGVARIABLE(LuauOptionalNextKey, false) LUAU_FASTFLAGVARIABLE(LuauReportShadowedTypeAlias, false) LUAU_FASTFLAGVARIABLE(LuauBetterMessagingOnCountMismatch, false) -LUAU_FASTFLAGVARIABLE(LuauArgMismatchReportFunctionLocation, false) +LUAU_FASTFLAGVARIABLE(LuauIntersectionTestForEquality, false) LUAU_FASTFLAGVARIABLE(LuauImplicitElseRefinement, false) +LUAU_FASTFLAGVARIABLE(LuauAllowIndexClassParameters, false) LUAU_FASTFLAGVARIABLE(LuauDeclareClassPrototype, false) +LUAU_FASTFLAG(LuauUninhabitedSubAnything) LUAU_FASTFLAGVARIABLE(LuauCallableClasses, false) namespace Luau { -const char* TimeLimitError_DEPRECATED::what() const throw() -{ - LUAU_ASSERT(!FFlag::LuauIceExceptionInheritanceChange); - return "Typeinfer failed to complete in allotted time"; -} static bool typeCouldHaveMetatable(TypeId ty) { @@ -269,11 +267,6 @@ ModulePtr TypeChecker::check(const SourceModule& module, Mode mode, std::optiona reportErrorCodeTooComplex(module.root->location); return std::move(currentModule); } - catch (const RecursionLimitException_DEPRECATED&) - { - reportErrorCodeTooComplex(module.root->location); - return std::move(currentModule); - } } ModulePtr TypeChecker::checkWithoutRecursionCheck(const SourceModule& module, Mode mode, std::optional environmentScope) @@ -318,10 +311,6 @@ ModulePtr TypeChecker::checkWithoutRecursionCheck(const SourceModule& module, Mo { currentModule->timeout = true; } - catch (const TimeLimitError_DEPRECATED&) - { - currentModule->timeout = true; - } if (FFlag::DebugLuauSharedSelf) { @@ -429,7 +418,7 @@ void TypeChecker::check(const ScopePtr& scope, const AstStat& program) ice("Unknown AstStat"); if (finishTime && TimeTrace::getClock() > *finishTime) - throwTimeLimitError(); + throw TimeLimitError(iceHandler->moduleName); } // This particular overload is for do...end. If you need to not increase the scope level, use checkBlock directly. @@ -456,11 +445,6 @@ void TypeChecker::checkBlock(const ScopePtr& scope, const AstStatBlock& block) reportErrorCodeTooComplex(block.location); return; } - catch (const RecursionLimitException_DEPRECATED&) - { - reportErrorCodeTooComplex(block.location); - return; - } } struct InplaceDemoter : TypeVarOnceVisitor @@ -966,9 +950,9 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatAssign& assign) TypeId right = nullptr; - Location loc = 0 == assign.values.size ? assign.location - : i < assign.values.size ? assign.values.data[i]->location - : assign.values.data[assign.values.size - 1]->location; + Location loc = 0 == assign.values.size + ? assign.location + : i < assign.values.size ? assign.values.data[i]->location : assign.values.data[assign.values.size - 1]->location; if (valueIter != valueEnd) { @@ -2671,6 +2655,48 @@ static std::optional getIdentifierOfBaseVar(AstExpr* node) return std::nullopt; } +/** Return true if comparison between the types a and b should be permitted with + * the == or ~= operators. + * + * Two types are considered eligible for equality testing if it is possible for + * the test to ever succeed. In other words, we test to see whether the two + * types have any overlap at all. + * + * In order to make things work smoothly with the greedy solver, this function + * exempts any and FreeTypeVars from this requirement. + * + * This function does not (yet?) take into account extra Lua restrictions like + * that two tables can only be compared if they have the same metatable. That + * is presently handled by the caller. + * + * @return True if the types are comparable. False if they are not. + * + * If an internal recursion limit is reached while performing this test, the + * function returns std::nullopt. + */ +static std::optional areEqComparable(NotNull arena, NotNull normalizer, TypeId a, TypeId b) +{ + a = follow(a); + b = follow(b); + + auto isExempt = [](TypeId t) { + return isNil(t) || get(t); + }; + + if (isExempt(a) || isExempt(b)) + return true; + + TypeId c = arena->addType(IntersectionTypeVar{{a, b}}); + const NormalizedType* n = normalizer->normalize(c); + if (!n) + return std::nullopt; + + if (FFlag::LuauUninhabitedSubAnything) + return normalizer->isInhabited(n); + else + return isInhabited_DEPRECATED(*n); +} + TypeId TypeChecker::checkRelationalOperation( const ScopePtr& scope, const AstExprBinary& expr, TypeId lhsType, TypeId rhsType, const PredicateVec& predicates) { @@ -2741,6 +2767,28 @@ TypeId TypeChecker::checkRelationalOperation( return booleanType; } + if (FFlag::LuauIntersectionTestForEquality && isEquality) + { + // Unless either type is free or any, an equality comparison is only + // valid when the intersection of the two operands is non-empty. + // + // eg it is okay to compare string? == number? because the two types + // have nil in common, but string == number is not allowed. + std::optional eqTestResult = areEqComparable(NotNull{¤tModule->internalTypes}, NotNull{&normalizer}, lhsType, rhsType); + if (!eqTestResult) + { + reportErrorCodeTooComplex(expr.location); + return errorRecoveryType(booleanType); + } + + if (!*eqTestResult) + { + reportError( + expr.location, GenericError{format("Type %s cannot be compared with %s", toString(lhsType).c_str(), toString(rhsType).c_str())}); + return errorRecoveryType(booleanType); + } + } + /* Subtlety here: * We need to do this unification first, but there are situations where we don't actually want to * report any problems that might have been surfaced as a result of this step because we might already @@ -2753,7 +2801,7 @@ TypeId TypeChecker::checkRelationalOperation( state.log.commit(); } - bool needsMetamethod = !isEquality; + const bool needsMetamethod = !isEquality; TypeId leftType = follow(lhsType); if (get(leftType) || get(leftType) || get(leftType) || get(leftType)) @@ -3335,6 +3383,9 @@ TypeId TypeChecker::checkLValueBinding(const ScopePtr& scope, const AstExprIndex TypeId indexType = checkExpr(scope, *expr.index).type; + if (FFlag::LuauFollowInLvalueIndexCheck) + exprType = follow(exprType); + if (get(exprType) || get(exprType)) return exprType; @@ -3356,6 +3407,16 @@ TypeId TypeChecker::checkLValueBinding(const ScopePtr& scope, const AstExprIndex return prop->type; } } + else if (FFlag::LuauAllowIndexClassParameters) + { + if (const ClassTypeVar* exprClass = get(exprType)) + { + if (isNonstrictMode()) + return unknownType; + reportError(TypeError{expr.location, DynamicPropertyLookupOnClassesUnsafe{exprType}}); + return errorRecoveryType(scope); + } + } TableTypeVar* exprTable = getMutableTableType(exprType); @@ -3810,16 +3871,8 @@ void TypeChecker::checkArgumentList(const ScopePtr& scope, const AstExpr& funNam std::string namePath; - if (FFlag::LuauLvaluelessPath) - { - if (std::optional path = getFunctionNameAsString(funName)) - namePath = *path; - } - else - { - if (std::optional lValue = tryGetLValue(funName)) - namePath = toString(*lValue); - } + if (std::optional path = getFunctionNameAsString(funName)) + namePath = *path; auto [minParams, optMaxParams] = getParameterExtents(&state.log, paramPack); state.reportError(TypeError{location, @@ -3929,27 +3982,11 @@ void TypeChecker::checkArgumentList(const ScopePtr& scope, const AstExpr& funNam std::string namePath; - if (FFlag::LuauLvaluelessPath) - { - if (std::optional path = getFunctionNameAsString(funName)) - namePath = *path; - } - else - { - if (std::optional lValue = tryGetLValue(funName)) - namePath = toString(*lValue); - } + if (std::optional path = getFunctionNameAsString(funName)) + namePath = *path; - if (FFlag::LuauArgMismatchReportFunctionLocation) - { - state.reportError(TypeError{ - funName.location, CountMismatch{minParams, optMaxParams, paramIndex, CountMismatch::Context::Arg, isVariadic, namePath}}); - } - else - { - state.reportError(TypeError{ - state.location, CountMismatch{minParams, optMaxParams, paramIndex, CountMismatch::Context::Arg, isVariadic, namePath}}); - } + state.reportError(TypeError{ + funName.location, CountMismatch{minParams, optMaxParams, paramIndex, CountMismatch::Context::Arg, isVariadic, namePath}}); return; } ++paramIter; @@ -4461,7 +4498,7 @@ void TypeChecker::reportOverloadResolutionError(const ScopePtr& scope, const Ast std::string s; for (size_t i = 0; i < overloadTypes.size(); ++i) { - TypeId overload = overloadTypes[i]; + TypeId overload = FFlag::LuauTypeInferMissingFollows ? follow(overloadTypes[i]) : overloadTypes[i]; Unifier state = mkUnifier(scope, expr.location); // Unify return types @@ -4842,7 +4879,10 @@ TypePackId TypeChecker::anyifyModuleReturnTypePackGenerics(TypePackId tp) tp = follow(tp); if (const VariadicTypePack* vtp = get(tp)) - return get(vtp->ty) ? anyTypePack : tp; + { + TypeId ty = FFlag::LuauTypeInferMissingFollows ? follow(vtp->ty) : vtp->ty; + return get(ty) ? anyTypePack : tp; + } if (!get(follow(tp))) return tp; @@ -4893,19 +4933,6 @@ void TypeChecker::ice(const std::string& message) iceHandler->ice(message); } -// TODO: Inline me when LuauIceExceptionInheritanceChange is deleted. -void TypeChecker::throwTimeLimitError() -{ - if (FFlag::LuauIceExceptionInheritanceChange) - { - throw TimeLimitError(iceHandler->moduleName); - } - else - { - throw TimeLimitError_DEPRECATED(); - } -} - void TypeChecker::prepareErrorsForDisplay(ErrorVec& errVec) { // Remove errors with names that were generated by recovery from a parse error @@ -6085,11 +6112,11 @@ void TypeChecker::resolve(const EqPredicate& eqP, RefinementMap& refis, const Sc if (optionIsSubtype && !targetIsSubtype) return option; else if (!optionIsSubtype && targetIsSubtype) - return eqP.type; + return FFlag::LuauTypeInferMissingFollows ? follow(eqP.type) : eqP.type; else if (!optionIsSubtype && !targetIsSubtype) return nope; else if (optionIsSubtype && targetIsSubtype) - return eqP.type; + return FFlag::LuauTypeInferMissingFollows ? follow(eqP.type) : eqP.type; } else { diff --git a/Analysis/src/TypePack.cpp b/Analysis/src/TypePack.cpp index 0852f053..0f75c3ef 100644 --- a/Analysis/src/TypePack.cpp +++ b/Analysis/src/TypePack.cpp @@ -6,6 +6,8 @@ #include +LUAU_FASTFLAGVARIABLE(LuauTxnLogTypePackIterator, false) + namespace Luau { @@ -60,8 +62,8 @@ TypePackIterator::TypePackIterator(TypePackId typePack) } TypePackIterator::TypePackIterator(TypePackId typePack, const TxnLog* log) - : currentTypePack(follow(typePack)) - , tp(get(currentTypePack)) + : currentTypePack(FFlag::LuauTxnLogTypePackIterator ? log->follow(typePack) : follow(typePack)) + , tp(FFlag::LuauTxnLogTypePackIterator ? log->get(currentTypePack) : get(currentTypePack)) , currentIndex(0) , log(log) { @@ -235,7 +237,7 @@ TypePackId follow(TypePackId tp, std::function mapper) cycleTester = nullptr; if (tp == cycleTester) - throwRuntimeError("Luau::follow detected a TypeVar cycle!!"); + throw InternalCompilerError("Luau::follow detected a TypeVar cycle!!"); } } } diff --git a/Analysis/src/TypeUtils.cpp b/Analysis/src/TypeUtils.cpp index 72597c4a..876c45a7 100644 --- a/Analysis/src/TypeUtils.cpp +++ b/Analysis/src/TypeUtils.cpp @@ -122,10 +122,6 @@ std::optional getIndexTypeFromType(const ScopePtr& scope, ErrorVec& erro for (TypeId t : utv) { - // TODO: we should probably limit recursion here? - // RecursionLimiter _rl(&recursionCount, FInt::LuauTypeInferRecursionLimit); - - // Not needed when we normalize types. if (get(follow(t))) return t; @@ -164,9 +160,6 @@ std::optional getIndexTypeFromType(const ScopePtr& scope, ErrorVec& erro for (TypeId t : itv->parts) { - // TODO: we should probably limit recursion here? - // RecursionLimiter _rl(&recursionCount, FInt::LuauTypeInferRecursionLimit); - if (std::optional ty = getIndexTypeFromType(scope, errors, arena, singletonTypes, t, prop, location, /* addErrors= */ false, handle)) parts.push_back(*ty); @@ -183,7 +176,7 @@ std::optional getIndexTypeFromType(const ScopePtr& scope, ErrorVec& erro if (parts.size() == 1) return parts[0]; - return arena->addType(IntersectionTypeVar{std::move(parts)}); // Not at all correct. + return arena->addType(IntersectionTypeVar{std::move(parts)}); } if (addErrors) @@ -221,46 +214,95 @@ std::pair> getParameterExtents(const TxnLog* log, return {minCount, minCount + optionalCount}; } -std::vector flatten(TypeArena& arena, NotNull singletonTypes, TypePackId pack, size_t length) +TypePack extendTypePack(TypeArena& arena, NotNull singletonTypes, TypePackId pack, size_t length) { - std::vector result; + TypePack result; - auto it = begin(pack); - auto endIt = end(pack); - - while (it != endIt) + while (true) { - result.push_back(*it); + pack = follow(pack); - if (result.size() >= length) + if (const TypePack* p = get(pack)) + { + size_t i = 0; + while (i < p->head.size() && result.head.size() < length) + { + result.head.push_back(p->head[i]); + ++i; + } + + if (result.head.size() == length) + { + if (i == p->head.size()) + result.tail = p->tail; + else + { + TypePackId newTail = arena.addTypePack(TypePack{}); + TypePack* newTailPack = getMutable(newTail); + + newTailPack->head.insert(newTailPack->head.begin(), p->head.begin() + i, p->head.end()); + newTailPack->tail = p->tail; + + result.tail = newTail; + } + + return result; + } + else if (p->tail) + { + pack = *p->tail; + continue; + } + else + { + // There just aren't enough types in this pack to satisfy the request. + return result; + } + } + else if (const VariadicTypePack* vtp = get(pack)) + { + while (result.head.size() < length) + result.head.push_back(vtp->ty); + result.tail = pack; return result; + } + else if (FreeTypePack* ftp = getMutable(pack)) + { + // If we need to get concrete types out of a free pack, we choose to + // interpret this as proof that the pack must have at least 'length' + // elements. We mint fresh types for each element we're extracting + // and rebind the free pack to be a TypePack containing them. We + // also have to create a new tail. - ++it; - } + TypePack newPack; + newPack.tail = arena.freshTypePack(ftp->scope); - if (!it.tail()) - return result; + while (result.head.size() < length) + { + newPack.head.push_back(arena.freshType(ftp->scope)); + result.head.push_back(newPack.head.back()); + } - TypePackId tail = *it.tail(); - if (get(tail)) - LUAU_ASSERT(0); - else if (auto vtp = get(tail)) - { - while (result.size() < length) - result.push_back(vtp->ty); - } - else if (get(tail) || get(tail)) - { - while (result.size() < length) - result.push_back(arena.addType(FreeTypeVar{nullptr})); - } - else if (auto etp = get(tail)) - { - while (result.size() < length) - result.push_back(singletonTypes->errorRecoveryType()); - } + asMutable(pack)->ty.emplace(std::move(newPack)); - return result; + return result; + } + else if (const Unifiable::Error* etp = getMutable(pack)) + { + while (result.head.size() < length) + result.head.push_back(singletonTypes->errorRecoveryType()); + + result.tail = pack; + return result; + } + else + { + // If the pack is blocked or generic, we can't extract. + // Return whatever we've got with this pack as the tail. + result.tail = pack; + return result; + } + } } std::vector reduceUnion(const std::vector& types) diff --git a/Analysis/src/TypeVar.cpp b/Analysis/src/TypeVar.cpp index 814eca0d..6771d89b 100644 --- a/Analysis/src/TypeVar.cpp +++ b/Analysis/src/TypeVar.cpp @@ -72,7 +72,7 @@ TypeId follow(TypeId t, std::function mapper) { TypeId res = ltv->thunk(); if (get(res)) - throwRuntimeError("Lazy TypeVar cannot resolve to another Lazy TypeVar"); + throw InternalCompilerError("Lazy TypeVar cannot resolve to another Lazy TypeVar"); *asMutable(ty) = BoundTypeVar(res); } @@ -110,7 +110,7 @@ TypeId follow(TypeId t, std::function mapper) cycleTester = nullptr; if (t == cycleTester) - throwRuntimeError("Luau::follow detected a TypeVar cycle!!"); + throw InternalCompilerError("Luau::follow detected a TypeVar cycle!!"); } } } @@ -468,65 +468,65 @@ PendingExpansionTypeVar::PendingExpansionTypeVar( size_t PendingExpansionTypeVar::nextIndex = 0; FunctionTypeVar::FunctionTypeVar(TypePackId argTypes, TypePackId retTypes, std::optional defn, bool hasSelf) - : argTypes(argTypes) + : definition(std::move(defn)) + , argTypes(argTypes) , retTypes(retTypes) - , definition(std::move(defn)) , hasSelf(hasSelf) { } FunctionTypeVar::FunctionTypeVar(TypeLevel level, TypePackId argTypes, TypePackId retTypes, std::optional defn, bool hasSelf) - : level(level) + : definition(std::move(defn)) + , level(level) , argTypes(argTypes) , retTypes(retTypes) - , definition(std::move(defn)) , hasSelf(hasSelf) { } FunctionTypeVar::FunctionTypeVar( TypeLevel level, Scope* scope, TypePackId argTypes, TypePackId retTypes, std::optional defn, bool hasSelf) - : level(level) + : definition(std::move(defn)) + , level(level) , scope(scope) , argTypes(argTypes) , retTypes(retTypes) - , definition(std::move(defn)) , hasSelf(hasSelf) { } FunctionTypeVar::FunctionTypeVar(std::vector generics, std::vector genericPacks, TypePackId argTypes, TypePackId retTypes, std::optional defn, bool hasSelf) - : generics(generics) + : definition(std::move(defn)) + , generics(generics) , genericPacks(genericPacks) , argTypes(argTypes) , retTypes(retTypes) - , definition(std::move(defn)) , hasSelf(hasSelf) { } FunctionTypeVar::FunctionTypeVar(TypeLevel level, std::vector generics, std::vector genericPacks, TypePackId argTypes, TypePackId retTypes, std::optional defn, bool hasSelf) - : level(level) + : definition(std::move(defn)) , generics(generics) , genericPacks(genericPacks) + , level(level) , argTypes(argTypes) , retTypes(retTypes) - , definition(std::move(defn)) , hasSelf(hasSelf) { } FunctionTypeVar::FunctionTypeVar(TypeLevel level, Scope* scope, std::vector generics, std::vector genericPacks, TypePackId argTypes, TypePackId retTypes, std::optional defn, bool hasSelf) - : level(level) - , scope(scope) + : definition(std::move(defn)) , generics(generics) , genericPacks(genericPacks) + , level(level) + , scope(scope) , argTypes(argTypes) , retTypes(retTypes) - , definition(std::move(defn)) , hasSelf(hasSelf) { } diff --git a/Analysis/src/TypedAllocator.cpp b/Analysis/src/TypedAllocator.cpp index 133104d3..4dc26219 100644 --- a/Analysis/src/TypedAllocator.cpp +++ b/Analysis/src/TypedAllocator.cpp @@ -48,6 +48,8 @@ void* pagedAllocate(size_t size) // On Linux, we must use mmap because using regular heap results in mprotect() fragmenting the page table and us bumping into 64K mmap limit. #ifdef _WIN32 return _aligned_malloc(size, kPageSize); +#elif defined(__FreeBSD__) + return aligned_alloc(kPageSize, size); #else return mmap(nullptr, pageAlign(size), PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANON, -1, 0); #endif @@ -61,6 +63,8 @@ void pagedDeallocate(void* ptr, size_t size) #ifdef _WIN32 _aligned_free(ptr); +#elif defined(__FreeBSD__) + free(ptr); #else int rc = munmap(ptr, size); LUAU_ASSERT(rc == 0); diff --git a/Analysis/src/Unifier.cpp b/Analysis/src/Unifier.cpp index 4dc90983..5ff405e6 100644 --- a/Analysis/src/Unifier.cpp +++ b/Analysis/src/Unifier.cpp @@ -22,8 +22,10 @@ LUAU_FASTFLAGVARIABLE(LuauSubtypeNormalizer, false); LUAU_FASTFLAGVARIABLE(LuauScalarShapeSubtyping, false) LUAU_FASTFLAGVARIABLE(LuauInstantiateInSubtyping, false) LUAU_FASTFLAGVARIABLE(LuauOverloadedFunctionSubtypingPerf, false); -LUAU_FASTFLAGVARIABLE(LuauScalarShapeUnifyToMtOwner, false) +LUAU_FASTFLAGVARIABLE(LuauScalarShapeUnifyToMtOwner2, false) +LUAU_FASTFLAGVARIABLE(LuauUninhabitedSubAnything, false) LUAU_FASTFLAG(LuauClassTypeVarsInSubstitution) +LUAU_FASTFLAG(LuauTxnLogTypePackIterator) LUAU_FASTFLAG(DebugLuauDeferredConstraintResolution) LUAU_FASTFLAG(LuauNegatedFunctionTypes) @@ -51,6 +53,9 @@ struct PromoteTypeLevels final : TypeVarOnceVisitor template void promote(TID ty, T* t) { + if (FFlag::DebugLuauDeferredConstraintResolution && !t) + return; + LUAU_ASSERT(t); if (useScopes) @@ -102,6 +107,11 @@ struct PromoteTypeLevels final : TypeVarOnceVisitor if (ty->owningArena != typeArena) return false; + // Surprise, it's actually a BoundTypePack that hasn't been committed yet. + // Calling getMutable on this will trigger an assertion. + if (FFlag::LuauScalarShapeUnifyToMtOwner2 && !log.is(ty)) + return true; + promote(ty, log.getMutable(ty)); return true; } @@ -115,6 +125,11 @@ struct PromoteTypeLevels final : TypeVarOnceVisitor if (ttv.state != TableState::Free && ttv.state != TableState::Generic) return true; + // Surprise, it's actually a BoundTypePack that hasn't been committed yet. + // Calling getMutable on this will trigger an assertion. + if (FFlag::LuauScalarShapeUnifyToMtOwner2 && !log.is(ty)) + return true; + promote(ty, log.getMutable(ty)); return true; } @@ -277,7 +292,7 @@ TypeId Widen::clean(TypeId ty) TypePackId Widen::clean(TypePackId) { - throwRuntimeError("Widen attempted to clean a dirty type pack?"); + throw InternalCompilerError("Widen attempted to clean a dirty type pack?"); } bool Widen::ignoreChildren(TypeId ty) @@ -336,6 +351,20 @@ static bool subsumes(bool useScopes, TY_A* left, TY_B* right) return left->level.subsumes(right->level); } +TypeMismatch::Context Unifier::mismatchContext() +{ + switch (variance) + { + case Covariant: + return TypeMismatch::CovariantContext; + case Invariant: + return TypeMismatch::InvariantContext; + default: + LUAU_ASSERT(false); // This codepath should be unreachable. + return TypeMismatch::CovariantContext; + } +} + Unifier::Unifier(NotNull normalizer, Mode mode, NotNull scope, const Location& location, Variance variance, TxnLog* parentLog) : types(normalizer->arena) , singletonTypes(normalizer->singletonTypes) @@ -559,8 +588,11 @@ void Unifier::tryUnify_(TypeId subTy, TypeId superTy, bool isFunctionCall, bool else if (log.get(subTy)) tryUnifyNegationWithType(subTy, superTy); + else if (FFlag::LuauUninhabitedSubAnything && !normalizer->isInhabited(subTy)) + {} + else - reportError(location, TypeMismatch{superTy, subTy}); + reportError(location, TypeMismatch{superTy, subTy, mismatchContext()}); if (cacheEnabled) cacheResult(subTy, superTy, errorCount); @@ -575,11 +607,16 @@ void Unifier::tryUnifyUnionWithType(TypeId subTy, const UnionTypeVar* subUnion, std::optional unificationTooComplex; std::optional firstFailedOption; + std::vector logs; + for (TypeId type : subUnion->options) { Unifier innerState = makeChildUnifier(); innerState.tryUnify_(type, superTy); + if (FFlag::DebugLuauDeferredConstraintResolution) + logs.push_back(std::move(innerState.log)); + if (auto e = hasUnificationTooComplex(innerState.errors)) unificationTooComplex = e; else if (!innerState.errors.empty()) @@ -592,51 +629,56 @@ void Unifier::tryUnifyUnionWithType(TypeId subTy, const UnionTypeVar* subUnion, } } - // even if A | B <: T fails, we want to bind some options of T with A | B iff A | B was a subtype of that option. - auto tryBind = [this, subTy](TypeId superOption) { - superOption = log.follow(superOption); - - // just skip if the superOption is not free-ish. - auto ttv = log.getMutable(superOption); - if (!log.is(superOption) && (!ttv || ttv->state != TableState::Free)) - return; - - // If superOption is already present in subTy, do nothing. Nothing new has been learned, but the subtype - // test is successful. - if (auto subUnion = get(subTy)) - { - if (end(subUnion) != std::find(begin(subUnion), end(subUnion), superOption)) - return; - } - - // Since we have already checked if S <: T, checking it again will not queue up the type for replacement. - // So we'll have to do it ourselves. We assume they unified cleanly if they are still in the seen set. - if (log.haveSeen(subTy, superOption)) - { - // TODO: would it be nice for TxnLog::replace to do this? - if (log.is(superOption)) - log.bindTable(superOption, subTy); - else - log.replace(superOption, *subTy); - } - }; - - if (auto superUnion = log.getMutable(superTy)) - { - for (TypeId ty : superUnion) - tryBind(ty); - } + if (FFlag::DebugLuauDeferredConstraintResolution) + log.concatAsUnion(combineLogsIntoUnion(std::move(logs)), NotNull{types}); else - tryBind(superTy); + { + // even if A | B <: T fails, we want to bind some options of T with A | B iff A | B was a subtype of that option. + auto tryBind = [this, subTy](TypeId superOption) { + superOption = log.follow(superOption); + + // just skip if the superOption is not free-ish. + auto ttv = log.getMutable(superOption); + if (!log.is(superOption) && (!ttv || ttv->state != TableState::Free)) + return; + + // If superOption is already present in subTy, do nothing. Nothing new has been learned, but the subtype + // test is successful. + if (auto subUnion = get(subTy)) + { + if (end(subUnion) != std::find(begin(subUnion), end(subUnion), superOption)) + return; + } + + // Since we have already checked if S <: T, checking it again will not queue up the type for replacement. + // So we'll have to do it ourselves. We assume they unified cleanly if they are still in the seen set. + if (log.haveSeen(subTy, superOption)) + { + // TODO: would it be nice for TxnLog::replace to do this? + if (log.is(superOption)) + log.bindTable(superOption, subTy); + else + log.replace(superOption, *subTy); + } + }; + + if (auto superUnion = log.getMutable(superTy)) + { + for (TypeId ty : superUnion) + tryBind(ty); + } + else + tryBind(superTy); + } if (unificationTooComplex) reportError(*unificationTooComplex); else if (failed) { if (firstFailedOption) - reportError(location, TypeMismatch{superTy, subTy, "Not all union options are compatible.", *firstFailedOption}); + reportError(location, TypeMismatch{superTy, subTy, "Not all union options are compatible.", *firstFailedOption, mismatchContext()}); else - reportError(location, TypeMismatch{superTy, subTy}); + reportError(location, TypeMismatch{superTy, subTy, mismatchContext()}); } } @@ -696,6 +738,8 @@ void Unifier::tryUnifyTypeWithUnion(TypeId subTy, TypeId superTy, const UnionTyp } } + std::vector logs; + for (size_t i = 0; i < uv->options.size(); ++i) { TypeId type = uv->options[(i + startIndex) % uv->options.size()]; @@ -706,9 +750,13 @@ void Unifier::tryUnifyTypeWithUnion(TypeId subTy, TypeId superTy, const UnionTyp if (innerState.errors.empty()) { found = true; - log.concat(std::move(innerState.log)); - - break; + if (FFlag::DebugLuauDeferredConstraintResolution) + logs.push_back(std::move(innerState.log)); + else + { + log.concat(std::move(innerState.log)); + break; + } } else if (auto e = hasUnificationTooComplex(innerState.errors)) { @@ -723,6 +771,9 @@ void Unifier::tryUnifyTypeWithUnion(TypeId subTy, TypeId superTy, const UnionTyp } } + if (FFlag::DebugLuauDeferredConstraintResolution) + log.concatAsUnion(combineLogsIntoUnion(std::move(logs)), NotNull{types}); + if (unificationTooComplex) { reportError(*unificationTooComplex); @@ -744,9 +795,10 @@ void Unifier::tryUnifyTypeWithUnion(TypeId subTy, TypeId superTy, const UnionTyp else if (!found) { if ((failedOptionCount == 1 || foundHeuristic) && failedOption) - reportError(location, TypeMismatch{superTy, subTy, "None of the union options are compatible. For example:", *failedOption}); + reportError( + location, TypeMismatch{superTy, subTy, "None of the union options are compatible. For example:", *failedOption, mismatchContext()}); else - reportError(location, TypeMismatch{superTy, subTy, "none of the union options are compatible"}); + reportError(location, TypeMismatch{superTy, subTy, "none of the union options are compatible", mismatchContext()}); } } @@ -755,6 +807,8 @@ void Unifier::tryUnifyTypeWithIntersection(TypeId subTy, TypeId superTy, const I std::optional unificationTooComplex; std::optional firstFailedOption; + std::vector logs; + // T <: A & B if and only if T <: A and T <: B for (TypeId type : uv->parts) { @@ -769,13 +823,19 @@ void Unifier::tryUnifyTypeWithIntersection(TypeId subTy, TypeId superTy, const I firstFailedOption = {innerState.errors.front()}; } - log.concat(std::move(innerState.log)); + if (FFlag::DebugLuauDeferredConstraintResolution) + logs.push_back(std::move(innerState.log)); + else + log.concat(std::move(innerState.log)); } + if (FFlag::DebugLuauDeferredConstraintResolution) + log.concat(combineLogsIntoIntersection(std::move(logs))); + if (unificationTooComplex) reportError(*unificationTooComplex); else if (firstFailedOption) - reportError(location, TypeMismatch{superTy, subTy, "Not all intersection parts are compatible.", *firstFailedOption}); + reportError(location, TypeMismatch{superTy, subTy, "Not all intersection parts are compatible.", *firstFailedOption, mismatchContext()}); } void Unifier::tryUnifyIntersectionWithType(TypeId subTy, const IntersectionTypeVar* uv, TypeId superTy, bool cacheEnabled, bool isFunctionCall) @@ -802,6 +862,8 @@ void Unifier::tryUnifyIntersectionWithType(TypeId subTy, const IntersectionTypeV } } + std::vector logs; + for (size_t i = 0; i < uv->parts.size(); ++i) { TypeId type = uv->parts[(i + startIndex) % uv->parts.size()]; @@ -812,8 +874,13 @@ void Unifier::tryUnifyIntersectionWithType(TypeId subTy, const IntersectionTypeV if (innerState.errors.empty()) { found = true; - log.concat(std::move(innerState.log)); - break; + if (FFlag::DebugLuauDeferredConstraintResolution) + logs.push_back(std::move(innerState.log)); + else + { + log.concat(std::move(innerState.log)); + break; + } } else if (auto e = hasUnificationTooComplex(innerState.errors)) { @@ -821,6 +888,9 @@ void Unifier::tryUnifyIntersectionWithType(TypeId subTy, const IntersectionTypeV } } + if (FFlag::DebugLuauDeferredConstraintResolution) + log.concat(combineLogsIntoIntersection(std::move(logs))); + if (unificationTooComplex) reportError(*unificationTooComplex); else if (!found && normalize) @@ -837,7 +907,7 @@ void Unifier::tryUnifyIntersectionWithType(TypeId subTy, const IntersectionTypeV } else if (!found) { - reportError(location, TypeMismatch{superTy, subTy, "none of the intersection parts are compatible"}); + reportError(location, TypeMismatch{superTy, subTy, "none of the intersection parts are compatible", mismatchContext()}); } } @@ -849,37 +919,37 @@ void Unifier::tryUnifyNormalizedTypes( if (get(superNorm.tops) || get(superNorm.tops) || get(subNorm.tops)) return; else if (get(subNorm.tops)) - return reportError(location, TypeMismatch{superTy, subTy, reason, error}); + return reportError(location, TypeMismatch{superTy, subTy, reason, error, mismatchContext()}); if (get(subNorm.errors)) if (!get(superNorm.errors)) - return reportError(location, TypeMismatch{superTy, subTy, reason, error}); + return reportError(location, TypeMismatch{superTy, subTy, reason, error, mismatchContext()}); if (get(subNorm.booleans)) { if (!get(superNorm.booleans)) - return reportError(location, TypeMismatch{superTy, subTy, reason, error}); + return reportError(location, TypeMismatch{superTy, subTy, reason, error, mismatchContext()}); } else if (const SingletonTypeVar* stv = get(subNorm.booleans)) { if (!get(superNorm.booleans) && stv != get(superNorm.booleans)) - return reportError(location, TypeMismatch{superTy, subTy, reason, error}); + return reportError(location, TypeMismatch{superTy, subTy, reason, error, mismatchContext()}); } if (get(subNorm.nils)) if (!get(superNorm.nils)) - return reportError(location, TypeMismatch{superTy, subTy, reason, error}); + return reportError(location, TypeMismatch{superTy, subTy, reason, error, mismatchContext()}); if (get(subNorm.numbers)) if (!get(superNorm.numbers)) - return reportError(location, TypeMismatch{superTy, subTy, reason, error}); + return reportError(location, TypeMismatch{superTy, subTy, reason, error, mismatchContext()}); if (!isSubtype(subNorm.strings, superNorm.strings)) - return reportError(location, TypeMismatch{superTy, subTy, reason, error}); + return reportError(location, TypeMismatch{superTy, subTy, reason, error, mismatchContext()}); if (get(subNorm.threads)) if (!get(superNorm.errors)) - return reportError(location, TypeMismatch{superTy, subTy, reason, error}); + return reportError(location, TypeMismatch{superTy, subTy, reason, error, mismatchContext()}); for (TypeId subClass : subNorm.classes) { @@ -895,7 +965,7 @@ void Unifier::tryUnifyNormalizedTypes( } } if (!found) - return reportError(location, TypeMismatch{superTy, subTy, reason, error}); + return reportError(location, TypeMismatch{superTy, subTy, reason, error, mismatchContext()}); } for (TypeId subTable : subNorm.tables) @@ -920,19 +990,19 @@ void Unifier::tryUnifyNormalizedTypes( return reportError(*e); } if (!found) - return reportError(location, TypeMismatch{superTy, subTy, reason, error}); + return reportError(location, TypeMismatch{superTy, subTy, reason, error, mismatchContext()}); } if (!subNorm.functions.isNever()) { if (superNorm.functions.isNever()) - return reportError(location, TypeMismatch{superTy, subTy, reason, error}); + return reportError(location, TypeMismatch{superTy, subTy, reason, error, mismatchContext()}); for (TypeId superFun : *superNorm.functions.parts) { Unifier innerState = makeChildUnifier(); const FunctionTypeVar* superFtv = get(superFun); if (!superFtv) - return reportError(location, TypeMismatch{superTy, subTy, reason, error}); + return reportError(location, TypeMismatch{superTy, subTy, reason, error, mismatchContext()}); TypePackId tgt = innerState.tryApplyOverloadedFunction(subTy, subNorm.functions, superFtv->argTypes); innerState.tryUnify_(tgt, superFtv->retTypes); if (innerState.errors.empty()) @@ -940,7 +1010,7 @@ void Unifier::tryUnifyNormalizedTypes( else if (auto e = hasUnificationTooComplex(innerState.errors)) return reportError(*e); else - return reportError(location, TypeMismatch{superTy, subTy, reason, error}); + return reportError(location, TypeMismatch{superTy, subTy, reason, error, mismatchContext()}); } } @@ -1306,7 +1376,7 @@ void Unifier::tryUnify_(TypePackId subTp, TypePackId superTp, bool isFunctionCal // If both are at the end, we're done if (!superIter.good() && !subIter.good()) { - if (subTpv->tail && superTpv->tail) + if (!FFlag::LuauTxnLogTypePackIterator && subTpv->tail && superTpv->tail) { tryUnify_(*subTpv->tail, *superTpv->tail); break; @@ -1314,10 +1384,27 @@ void Unifier::tryUnify_(TypePackId subTp, TypePackId superTp, bool isFunctionCal const bool lFreeTail = superTpv->tail && log.getMutable(log.follow(*superTpv->tail)) != nullptr; const bool rFreeTail = subTpv->tail && log.getMutable(log.follow(*subTpv->tail)) != nullptr; - if (lFreeTail) + if (FFlag::LuauTxnLogTypePackIterator && lFreeTail && rFreeTail) + { + tryUnify_(*subTpv->tail, *superTpv->tail); + } + else if (lFreeTail) + { tryUnify_(emptyTp, *superTpv->tail); + } else if (rFreeTail) + { tryUnify_(emptyTp, *subTpv->tail); + } + else if (FFlag::LuauTxnLogTypePackIterator && subTpv->tail && superTpv->tail) + { + if (log.getMutable(superIter.packId)) + tryUnifyVariadics(subIter.packId, superIter.packId, false, int(subIter.index)); + else if (log.getMutable(subIter.packId)) + tryUnifyVariadics(superIter.packId, subIter.packId, true, int(superIter.index)); + else + tryUnify_(*subTpv->tail, *superTpv->tail); + } break; } @@ -1407,7 +1494,7 @@ void Unifier::tryUnifyPrimitives(TypeId subTy, TypeId superTy) ice("passed non primitive types to unifyPrimitives"); if (superPrim->type != subPrim->type) - reportError(location, TypeMismatch{superTy, subTy}); + reportError(location, TypeMismatch{superTy, subTy, mismatchContext()}); } void Unifier::tryUnifySingletons(TypeId subTy, TypeId superTy) @@ -1428,7 +1515,7 @@ void Unifier::tryUnifySingletons(TypeId subTy, TypeId superTy) if (superPrim && superPrim->type == PrimitiveTypeVar::String && get(subSingleton) && variance == Covariant) return; - reportError(location, TypeMismatch{superTy, subTy}); + reportError(location, TypeMismatch{superTy, subTy, mismatchContext()}); } void Unifier::tryUnifyFunctions(TypeId subTy, TypeId superTy, bool isFunctionCall) @@ -1471,14 +1558,14 @@ void Unifier::tryUnifyFunctions(TypeId subTy, TypeId superTy, bool isFunctionCal { numGenerics = std::min(superFunction->generics.size(), subFunction->generics.size()); - reportError(location, TypeMismatch{superTy, subTy, "different number of generic type parameters"}); + reportError(location, TypeMismatch{superTy, subTy, "different number of generic type parameters", mismatchContext()}); } if (numGenericPacks != subFunction->genericPacks.size()) { numGenericPacks = std::min(superFunction->genericPacks.size(), subFunction->genericPacks.size()); - reportError(location, TypeMismatch{superTy, subTy, "different number of generic type pack parameters"}); + reportError(location, TypeMismatch{superTy, subTy, "different number of generic type pack parameters", mismatchContext()}); } for (size_t i = 0; i < numGenerics; i++) @@ -1506,9 +1593,9 @@ void Unifier::tryUnifyFunctions(TypeId subTy, TypeId superTy, bool isFunctionCal reportError(*e); else if (!innerState.errors.empty() && innerState.firstPackErrorPos) reportError(location, TypeMismatch{superTy, subTy, format("Argument #%d type is not compatible.", *innerState.firstPackErrorPos), - innerState.errors.front()}); + innerState.errors.front(), mismatchContext()}); else if (!innerState.errors.empty()) - reportError(location, TypeMismatch{superTy, subTy, "", innerState.errors.front()}); + reportError(location, TypeMismatch{superTy, subTy, "", innerState.errors.front(), mismatchContext()}); innerState.ctx = CountMismatch::FunctionResult; innerState.tryUnify_(subFunction->retTypes, superFunction->retTypes); @@ -1518,12 +1605,12 @@ void Unifier::tryUnifyFunctions(TypeId subTy, TypeId superTy, bool isFunctionCal if (auto e = hasUnificationTooComplex(innerState.errors)) reportError(*e); else if (!innerState.errors.empty() && size(superFunction->retTypes) == 1 && finite(superFunction->retTypes)) - reportError(location, TypeMismatch{superTy, subTy, "Return type is not compatible.", innerState.errors.front()}); + reportError(location, TypeMismatch{superTy, subTy, "Return type is not compatible.", innerState.errors.front(), mismatchContext()}); else if (!innerState.errors.empty() && innerState.firstPackErrorPos) reportError(location, TypeMismatch{superTy, subTy, format("Return #%d type is not compatible.", *innerState.firstPackErrorPos), - innerState.errors.front()}); + innerState.errors.front(), mismatchContext()}); else if (!innerState.errors.empty()) - reportError(location, TypeMismatch{superTy, subTy, "", innerState.errors.front()}); + reportError(location, TypeMismatch{superTy, subTy, "", innerState.errors.front(), mismatchContext()}); } log.concat(std::move(innerState.log)); @@ -1700,10 +1787,10 @@ void Unifier::tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) // Recursive unification can change the txn log, and invalidate the old // table. If we detect that this has happened, we start over, with the updated // txn log. - TypeId superTyNew = FFlag::LuauScalarShapeUnifyToMtOwner ? log.follow(superTy) : superTy; - TypeId subTyNew = FFlag::LuauScalarShapeUnifyToMtOwner ? log.follow(subTy) : subTy; + TypeId superTyNew = FFlag::LuauScalarShapeUnifyToMtOwner2 ? log.follow(superTy) : superTy; + TypeId subTyNew = FFlag::LuauScalarShapeUnifyToMtOwner2 ? log.follow(subTy) : subTy; - if (FFlag::LuauScalarShapeUnifyToMtOwner) + if (FFlag::LuauScalarShapeUnifyToMtOwner2) { // If one of the types stopped being a table altogether, we need to restart from the top if ((superTy != superTyNew || subTy != subTyNew) && errors.empty()) @@ -1771,11 +1858,21 @@ void Unifier::tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) else extraProperties.push_back(name); + TypeId superTyNew = FFlag::LuauScalarShapeUnifyToMtOwner2 ? log.follow(superTy) : superTy; + TypeId subTyNew = FFlag::LuauScalarShapeUnifyToMtOwner2 ? log.follow(subTy) : subTy; + + if (FFlag::LuauScalarShapeUnifyToMtOwner2) + { + // If one of the types stopped being a table altogether, we need to restart from the top + if ((superTy != superTyNew || subTy != subTyNew) && errors.empty()) + return tryUnify(subTy, superTy, false, isIntersection); + } + // Recursive unification can change the txn log, and invalidate the old // table. If we detect that this has happened, we start over, with the updated // txn log. - TableTypeVar* newSuperTable = log.getMutable(superTy); - TableTypeVar* newSubTable = log.getMutable(subTy); + TableTypeVar* newSuperTable = log.getMutable(superTyNew); + TableTypeVar* newSubTable = log.getMutable(subTyNew); if (superTable != newSuperTable || (subTable != newSubTable && subTable != instantiatedSubTable)) { if (errors.empty()) @@ -1829,8 +1926,19 @@ void Unifier::tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) } // Changing the indexer can invalidate the table pointers. - superTable = log.getMutable(superTy); - subTable = log.getMutable(subTy); + if (FFlag::LuauScalarShapeUnifyToMtOwner2) + { + superTable = log.getMutable(log.follow(superTy)); + subTable = log.getMutable(log.follow(subTy)); + + if (!superTable || !subTable) + return; + } + else + { + superTable = log.getMutable(superTy); + subTable = log.getMutable(subTy); + } if (!missingProperties.empty()) { @@ -1872,20 +1980,23 @@ void Unifier::tryUnifyScalarShape(TypeId subTy, TypeId superTy, bool reversed) TypeId osubTy = subTy; TypeId osuperTy = superTy; + if (FFlag::LuauUninhabitedSubAnything && !normalizer->isInhabited(subTy)) + return; + if (reversed) std::swap(subTy, superTy); TableTypeVar* superTable = log.getMutable(superTy); if (!superTable || superTable->state != TableState::Free) - return reportError(location, TypeMismatch{osuperTy, osubTy}); + return reportError(location, TypeMismatch{osuperTy, osubTy, mismatchContext()}); auto fail = [&](std::optional e) { std::string reason = "The former's metatable does not satisfy the requirements."; if (e) - reportError(location, TypeMismatch{osuperTy, osubTy, reason, *e}); + reportError(location, TypeMismatch{osuperTy, osubTy, reason, *e, mismatchContext()}); else - reportError(location, TypeMismatch{osuperTy, osubTy, reason}); + reportError(location, TypeMismatch{osuperTy, osubTy, reason, mismatchContext()}); }; // Given t1 where t1 = { lower: (t1) -> (a, b...) } @@ -1902,7 +2013,7 @@ void Unifier::tryUnifyScalarShape(TypeId subTy, TypeId superTy, bool reversed) Unifier child = makeChildUnifier(); child.tryUnify_(ty, superTy); - if (FFlag::LuauScalarShapeUnifyToMtOwner) + if (FFlag::LuauScalarShapeUnifyToMtOwner2) { // To perform subtype <: free table unification, we have tried to unify (subtype's metatable) <: free table // There is a chance that it was unified with the origial subtype, but then, (subtype's metatable) <: subtype could've failed @@ -1923,7 +2034,7 @@ void Unifier::tryUnifyScalarShape(TypeId subTy, TypeId superTy, bool reversed) log.concat(std::move(child.log)); - if (FFlag::LuauScalarShapeUnifyToMtOwner) + if (FFlag::LuauScalarShapeUnifyToMtOwner2) { // To perform subtype <: free table unification, we have tried to unify (subtype's metatable) <: free table // We return success because subtype <: free table which means that correct unification is to replace free table with the subtype @@ -1939,7 +2050,7 @@ void Unifier::tryUnifyScalarShape(TypeId subTy, TypeId superTy, bool reversed) } } - reportError(location, TypeMismatch{osuperTy, osubTy}); + reportError(location, TypeMismatch{osuperTy, osubTy, mismatchContext()}); return; } @@ -1969,7 +2080,7 @@ void Unifier::tryUnifyWithMetatable(TypeId subTy, TypeId superTy, bool reversed) if (!superMetatable) ice("tryUnifyMetatable invoked with non-metatable TypeVar"); - TypeError mismatchError = TypeError{location, TypeMismatch{reversed ? subTy : superTy, reversed ? superTy : subTy}}; + TypeError mismatchError = TypeError{location, TypeMismatch{reversed ? subTy : superTy, reversed ? superTy : subTy, mismatchContext()}}; if (const MetatableTypeVar* subMetatable = log.getMutable(subTy)) { @@ -1980,7 +2091,8 @@ void Unifier::tryUnifyWithMetatable(TypeId subTy, TypeId superTy, bool reversed) if (auto e = hasUnificationTooComplex(innerState.errors)) reportError(*e); else if (!innerState.errors.empty()) - reportError(location, TypeMismatch{reversed ? subTy : superTy, reversed ? superTy : subTy, "", innerState.errors.front()}); + reportError( + location, TypeMismatch{reversed ? subTy : superTy, reversed ? superTy : subTy, "", innerState.errors.front(), mismatchContext()}); log.concat(std::move(innerState.log)); } @@ -2017,8 +2129,8 @@ void Unifier::tryUnifyWithMetatable(TypeId subTy, TypeId superTy, bool reversed) if (auto e = hasUnificationTooComplex(innerState.errors)) reportError(*e); else if (!innerState.errors.empty()) - reportError( - TypeError{location, TypeMismatch{reversed ? subTy : superTy, reversed ? superTy : subTy, "", innerState.errors.front()}}); + reportError(TypeError{location, + TypeMismatch{reversed ? subTy : superTy, reversed ? superTy : subTy, "", innerState.errors.front(), mismatchContext()}}); else if (!missingProperty) { log.concat(std::move(innerState.log)); @@ -2057,9 +2169,9 @@ void Unifier::tryUnifyWithClass(TypeId subTy, TypeId superTy, bool reversed) auto fail = [&]() { if (!reversed) - reportError(location, TypeMismatch{superTy, subTy}); + reportError(location, TypeMismatch{superTy, subTy, mismatchContext()}); else - reportError(location, TypeMismatch{subTy, superTy}); + reportError(location, TypeMismatch{subTy, superTy, mismatchContext()}); }; const ClassTypeVar* superClass = get(superTy); @@ -2155,7 +2267,7 @@ void Unifier::tryUnifyTypeWithNegation(TypeId subTy, TypeId superTy) Unifier state = makeChildUnifier(); state.tryUnifyNormalizedTypes(subTy, superTy, *subNorm, *superNorm, ""); if (state.errors.empty()) - reportError(location, TypeMismatch{superTy, subTy}); + reportError(location, TypeMismatch{superTy, subTy, mismatchContext()}); } void Unifier::tryUnifyNegationWithType(TypeId subTy, TypeId superTy) @@ -2165,7 +2277,7 @@ void Unifier::tryUnifyNegationWithType(TypeId subTy, TypeId superTy) ice("tryUnifyNegationWithType subTy must be a negation type"); // TODO: ~T & queue, DenseHashSet& seenTypePacks, Unifier& state, TypePackId a, TypePackId anyTypePack) @@ -2200,9 +2312,11 @@ void Unifier::tryUnifyVariadics(TypePackId subTp, TypePackId superTp, bool rever if (!superVariadic) ice("passed non-variadic pack to tryUnifyVariadics"); - if (const VariadicTypePack* subVariadic = get(subTp)) + if (const VariadicTypePack* subVariadic = FFlag::LuauTxnLogTypePackIterator ? log.get(subTp) : get(subTp)) + { tryUnify_(reversed ? superVariadic->ty : subVariadic->ty, reversed ? subVariadic->ty : superVariadic->ty); - else if (get(subTp)) + } + else if (FFlag::LuauTxnLogTypePackIterator ? log.get(subTp) : get(subTp)) { TypePackIterator subIter = begin(subTp, &log); TypePackIterator subEnd = end(subTp); @@ -2350,6 +2464,24 @@ std::optional Unifier::findTablePropertyRespectingMeta(TypeId lhsType, N return Luau::findTablePropertyRespectingMeta(singletonTypes, errors, lhsType, name, location); } +TxnLog Unifier::combineLogsIntoIntersection(std::vector logs) +{ + LUAU_ASSERT(FFlag::DebugLuauDeferredConstraintResolution); + TxnLog result; + for (TxnLog& log : logs) + result.concatAsIntersections(std::move(log), NotNull{types}); + return result; +} + +TxnLog Unifier::combineLogsIntoUnion(std::vector logs) +{ + LUAU_ASSERT(FFlag::DebugLuauDeferredConstraintResolution); + TxnLog result; + for (TxnLog& log : logs) + result.concatAsUnion(std::move(log), NotNull{types}); + return result; +} + bool Unifier::occursCheck(TypeId needle, TypeId haystack) { sharedState.tempSeenTy.clear(); @@ -2491,7 +2623,7 @@ void Unifier::checkChildUnifierTypeMismatch(const ErrorVec& innerErrors, TypeId if (auto e = hasUnificationTooComplex(innerErrors)) reportError(*e); else if (!innerErrors.empty()) - reportError(location, TypeMismatch{wantedType, givenType}); + reportError(location, TypeMismatch{wantedType, givenType, mismatchContext()}); } void Unifier::checkChildUnifierTypeMismatch(const ErrorVec& innerErrors, const std::string& prop, TypeId wantedType, TypeId givenType) @@ -2499,8 +2631,8 @@ void Unifier::checkChildUnifierTypeMismatch(const ErrorVec& innerErrors, const s if (auto e = hasUnificationTooComplex(innerErrors)) reportError(*e); else if (!innerErrors.empty()) - reportError( - TypeError{location, TypeMismatch{wantedType, givenType, format("Property '%s' is not compatible.", prop.c_str()), innerErrors.front()}}); + reportError(TypeError{location, + TypeMismatch{wantedType, givenType, format("Property '%s' is not compatible.", prop.c_str()), innerErrors.front(), mismatchContext()}}); } void Unifier::ice(const std::string& message, const Location& location) diff --git a/Ast/include/Luau/Location.h b/Ast/include/Luau/Location.h index d3c0a462..e39bbf8c 100644 --- a/Ast/include/Luau/Location.h +++ b/Ast/include/Luau/Location.h @@ -50,6 +50,20 @@ struct Position { return *this == rhs || *this > rhs; } + + void shift(const Position& start, const Position& oldEnd, const Position& newEnd) + { + if (*this >= start) + { + if (this->line > oldEnd.line) + this->line += (newEnd.line - oldEnd.line); + else + { + this->line = newEnd.line; + this->column += (newEnd.column - oldEnd.column); + } + } + } }; struct Location @@ -93,6 +107,10 @@ struct Location { return begin <= l.begin && end >= l.end; } + bool overlaps(const Location& l) const + { + return (begin <= l.begin && end >= l.begin) || (begin <= l.end && end >= l.end) || (begin >= l.begin && end <= l.end); + } bool contains(const Position& p) const { return begin <= p && p < end; @@ -101,6 +119,18 @@ struct Location { return begin <= p && p <= end; } + void extend(const Location& other) + { + if (other.begin < begin) + begin = other.begin; + if (other.end > end) + end = other.end; + } + void shift(const Position& start, const Position& oldEnd, const Position& newEnd) + { + begin.shift(start, oldEnd, newEnd); + end.shift(start, oldEnd, newEnd); + } }; std::string toString(const Position& position); diff --git a/Ast/src/Parser.cpp b/Ast/src/Parser.cpp index 85b0d31a..5cd5f743 100644 --- a/Ast/src/Parser.cpp +++ b/Ast/src/Parser.cpp @@ -24,9 +24,6 @@ LUAU_DYNAMIC_FASTFLAGVARIABLE(LuaReportParseIntegerIssues, false) LUAU_FASTFLAGVARIABLE(LuauInterpolatedStringBaseSupport, false) -LUAU_FASTFLAGVARIABLE(LuauCommaParenWarnings, false) -LUAU_FASTFLAGVARIABLE(LuauTableConstructorRecovery, false) - LUAU_FASTFLAGVARIABLE(LuauParserErrorsOnMissingDefaultTypePackArgument, false) bool lua_telemetry_parsed_out_of_range_bin_integer = false; @@ -1084,7 +1081,7 @@ void Parser::parseExprList(TempVector& result) { nextLexeme(); - if (FFlag::LuauCommaParenWarnings && lexer.current().type == ')') + if (lexer.current().type == ')') { report(lexer.current().location, "Expected expression after ',' but got ')' instead"); break; @@ -1179,7 +1176,7 @@ AstTypePack* Parser::parseTypeList(TempVector& result, TempVector, AstArray> Parser::parseG { nextLexeme(); - if (FFlag::LuauCommaParenWarnings && lexer.current().type == '>') + if (lexer.current().type == '>') { report(lexer.current().location, "Expected type after ',' but got '>' instead"); break; diff --git a/Compiler/src/BytecodeBuilder.cpp b/Compiler/src/BytecodeBuilder.cpp index 83764244..7d230738 100644 --- a/Compiler/src/BytecodeBuilder.cpp +++ b/Compiler/src/BytecodeBuilder.cpp @@ -1895,10 +1895,7 @@ void BytecodeBuilder::dumpInstruction(const uint32_t* code, std::string& result, case LOP_CAPTURE: formatAppend(result, "CAPTURE %s %c%d\n", - LUAU_INSN_A(insn) == LCT_UPVAL ? "UPVAL" - : LUAU_INSN_A(insn) == LCT_REF ? "REF" - : LUAU_INSN_A(insn) == LCT_VAL ? "VAL" - : "", + LUAU_INSN_A(insn) == LCT_UPVAL ? "UPVAL" : LUAU_INSN_A(insn) == LCT_REF ? "REF" : LUAU_INSN_A(insn) == LCT_VAL ? "VAL" : "", LUAU_INSN_A(insn) == LCT_UPVAL ? 'U' : 'R', LUAU_INSN_B(insn)); break; diff --git a/Compiler/src/Compiler.cpp b/Compiler/src/Compiler.cpp index 7ccd1164..5d672366 100644 --- a/Compiler/src/Compiler.cpp +++ b/Compiler/src/Compiler.cpp @@ -26,6 +26,7 @@ LUAU_FASTINTVARIABLE(LuauCompileInlineThresholdMaxBoost, 300) LUAU_FASTINTVARIABLE(LuauCompileInlineDepth, 5) LUAU_FASTFLAG(LuauInterpolatedStringBaseSupport) +LUAU_FASTFLAGVARIABLE(LuauMultiAssignmentConflictFix, false) namespace Luau { @@ -2977,16 +2978,46 @@ struct Compiler Visitor visitor(this); - // mark any registers that are used *after* assignment as conflicting - for (size_t i = 0; i < vars.size(); ++i) + if (FFlag::LuauMultiAssignmentConflictFix) { - const LValue& li = vars[i].lvalue; + // mark any registers that are used *after* assignment as conflicting - if (i < values.size) - values.data[i]->visit(&visitor); + // first we go through assignments to locals, since they are performed before assignments to other l-values + for (size_t i = 0; i < vars.size(); ++i) + { + const LValue& li = vars[i].lvalue; - if (li.kind == LValue::Kind_Local) - visitor.assigned[li.reg] = true; + if (li.kind == LValue::Kind_Local) + { + if (i < values.size) + values.data[i]->visit(&visitor); + + visitor.assigned[li.reg] = true; + } + } + + // and now we handle all other l-values + for (size_t i = 0; i < vars.size(); ++i) + { + const LValue& li = vars[i].lvalue; + + if (li.kind != LValue::Kind_Local && i < values.size) + values.data[i]->visit(&visitor); + } + } + else + { + // mark any registers that are used *after* assignment as conflicting + for (size_t i = 0; i < vars.size(); ++i) + { + const LValue& li = vars[i].lvalue; + + if (i < values.size) + values.data[i]->visit(&visitor); + + if (li.kind == LValue::Kind_Local) + visitor.assigned[li.reg] = true; + } } // mark any registers used in trailing expressions as conflicting as well diff --git a/Sources.cmake b/Sources.cmake index 33ecde93..e243ea74 100644 --- a/Sources.cmake +++ b/Sources.cmake @@ -112,7 +112,7 @@ target_sources(Luau.Analysis PRIVATE Analysis/include/Luau/Constraint.h Analysis/include/Luau/ConstraintGraphBuilder.h Analysis/include/Luau/ConstraintSolver.h - Analysis/include/Luau/DataFlowGraphBuilder.h + Analysis/include/Luau/DataFlowGraph.h Analysis/include/Luau/DcrLogger.h Analysis/include/Luau/Def.h Analysis/include/Luau/Documentation.h @@ -166,7 +166,7 @@ target_sources(Luau.Analysis PRIVATE Analysis/src/Constraint.cpp Analysis/src/ConstraintGraphBuilder.cpp Analysis/src/ConstraintSolver.cpp - Analysis/src/DataFlowGraphBuilder.cpp + Analysis/src/DataFlowGraph.cpp Analysis/src/DcrLogger.cpp Analysis/src/Def.cpp Analysis/src/EmbeddedBuiltinDefinitions.cpp @@ -298,15 +298,16 @@ endif() if(TARGET Luau.UnitTest) # Luau.UnitTest Sources target_sources(Luau.UnitTest PRIVATE + tests/AstQueryDsl.cpp tests/AstQueryDsl.h + tests/ClassFixture.cpp + tests/ClassFixture.h + tests/ConstraintGraphBuilderFixture.cpp tests/ConstraintGraphBuilderFixture.h + tests/Fixture.cpp tests/Fixture.h tests/IostreamOptional.h tests/ScopedFlags.h - tests/AstQueryDsl.cpp - tests/ConstraintGraphBuilderFixture.cpp - tests/Fixture.cpp - tests/AssemblyBuilderA64.test.cpp tests/AssemblyBuilderX64.test.cpp tests/AstJsonEncoder.test.cpp tests/AstQuery.test.cpp @@ -318,7 +319,7 @@ if(TARGET Luau.UnitTest) tests/Config.test.cpp tests/ConstraintSolver.test.cpp tests/CostModel.test.cpp - tests/DataFlowGraphBuilder.test.cpp + tests/DataFlowGraph.test.cpp tests/Error.test.cpp tests/Frontend.test.cpp tests/JsonEmitter.test.cpp diff --git a/VM/src/loslib.cpp b/VM/src/loslib.cpp index 91ccec0c..62a5668b 100644 --- a/VM/src/loslib.cpp +++ b/VM/src/loslib.cpp @@ -22,6 +22,21 @@ static time_t timegm(struct tm* timep) { return _mkgmtime(timep); } +#elif defined(__FreeBSD__) +static tm* gmtime_r(const time_t* timep, tm* result) +{ + return gmtime_s(timep, result) == 0 ? result : NULL; +} + +static tm* localtime_r(const time_t* timep, tm* result) +{ + return localtime_s(timep, result) == 0 ? result : NULL; +} + +static time_t timegm(struct tm* timep) +{ + return mktime(timep); +} #endif static int os_clock(lua_State* L) diff --git a/tests/AstQuery.test.cpp b/tests/AstQuery.test.cpp index 48bb40d4..a642334a 100644 --- a/tests/AstQuery.test.cpp +++ b/tests/AstQuery.test.cpp @@ -91,8 +91,6 @@ TEST_CASE_FIXTURE(DocumentationSymbolFixture, "class_method") TEST_CASE_FIXTURE(DocumentationSymbolFixture, "overloaded_class_method") { - ScopedFastFlag luauCheckOverloadedDocSymbol{"LuauCheckOverloadedDocSymbol", true}; - loadDefinition(R"( declare class Foo function bar(self, x: string): number @@ -127,8 +125,6 @@ TEST_CASE_FIXTURE(DocumentationSymbolFixture, "table_function_prop") TEST_CASE_FIXTURE(DocumentationSymbolFixture, "table_overloaded_function_prop") { - ScopedFastFlag luauCheckOverloadedDocSymbol{"LuauCheckOverloadedDocSymbol", true}; - loadDefinition(R"( declare Foo: { new: ((number) -> string) & ((string) -> number) diff --git a/tests/ClassFixture.cpp b/tests/ClassFixture.cpp new file mode 100644 index 00000000..18939e24 --- /dev/null +++ b/tests/ClassFixture.cpp @@ -0,0 +1,113 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details + +#include "ClassFixture.h" + +#include "Luau/BuiltinDefinitions.h" + +using std::nullopt; + +namespace Luau +{ + +ClassFixture::ClassFixture() +{ + TypeArena& arena = typeChecker.globalTypes; + TypeId numberType = typeChecker.numberType; + + unfreeze(arena); + + TypeId baseClassInstanceType = arena.addType(ClassTypeVar{"BaseClass", {}, nullopt, nullopt, {}, {}, "Test"}); + getMutable(baseClassInstanceType)->props = { + {"BaseMethod", {makeFunction(arena, baseClassInstanceType, {numberType}, {})}}, + {"BaseField", {numberType}}, + }; + + TypeId baseClassType = arena.addType(ClassTypeVar{"BaseClass", {}, nullopt, nullopt, {}, {}, "Test"}); + getMutable(baseClassType)->props = { + {"StaticMethod", {makeFunction(arena, nullopt, {}, {numberType})}}, + {"Clone", {makeFunction(arena, nullopt, {baseClassInstanceType}, {baseClassInstanceType})}}, + {"New", {makeFunction(arena, nullopt, {}, {baseClassInstanceType})}}, + }; + typeChecker.globalScope->exportedTypeBindings["BaseClass"] = TypeFun{{}, baseClassInstanceType}; + addGlobalBinding(frontend, "BaseClass", baseClassType, "@test"); + + TypeId childClassInstanceType = arena.addType(ClassTypeVar{"ChildClass", {}, baseClassInstanceType, nullopt, {}, {}, "Test"}); + + getMutable(childClassInstanceType)->props = { + {"Method", {makeFunction(arena, childClassInstanceType, {}, {typeChecker.stringType})}}, + }; + + TypeId childClassType = arena.addType(ClassTypeVar{"ChildClass", {}, baseClassType, nullopt, {}, {}, "Test"}); + getMutable(childClassType)->props = { + {"New", {makeFunction(arena, nullopt, {}, {childClassInstanceType})}}, + }; + typeChecker.globalScope->exportedTypeBindings["ChildClass"] = TypeFun{{}, childClassInstanceType}; + addGlobalBinding(frontend, "ChildClass", childClassType, "@test"); + + TypeId grandChildInstanceType = arena.addType(ClassTypeVar{"GrandChild", {}, childClassInstanceType, nullopt, {}, {}, "Test"}); + + getMutable(grandChildInstanceType)->props = { + {"Method", {makeFunction(arena, grandChildInstanceType, {}, {typeChecker.stringType})}}, + }; + + TypeId grandChildType = arena.addType(ClassTypeVar{"GrandChild", {}, baseClassType, nullopt, {}, {}, "Test"}); + getMutable(grandChildType)->props = { + {"New", {makeFunction(arena, nullopt, {}, {grandChildInstanceType})}}, + }; + typeChecker.globalScope->exportedTypeBindings["GrandChild"] = TypeFun{{}, grandChildInstanceType}; + addGlobalBinding(frontend, "GrandChild", childClassType, "@test"); + + TypeId anotherChildInstanceType = arena.addType(ClassTypeVar{"AnotherChild", {}, baseClassInstanceType, nullopt, {}, {}, "Test"}); + + getMutable(anotherChildInstanceType)->props = { + {"Method", {makeFunction(arena, anotherChildInstanceType, {}, {typeChecker.stringType})}}, + }; + + TypeId anotherChildType = arena.addType(ClassTypeVar{"AnotherChild", {}, baseClassType, nullopt, {}, {}, "Test"}); + getMutable(anotherChildType)->props = { + {"New", {makeFunction(arena, nullopt, {}, {anotherChildInstanceType})}}, + }; + typeChecker.globalScope->exportedTypeBindings["AnotherChild"] = TypeFun{{}, anotherChildInstanceType}; + addGlobalBinding(frontend, "AnotherChild", childClassType, "@test"); + + TypeId unrelatedClassInstanceType = arena.addType(ClassTypeVar{"UnrelatedClass", {}, nullopt, nullopt, {}, {}, "Test"}); + + TypeId unrelatedClassType = arena.addType(ClassTypeVar{"UnrelatedClass", {}, nullopt, nullopt, {}, {}, "Test"}); + getMutable(unrelatedClassType)->props = { + {"New", {makeFunction(arena, nullopt, {}, {unrelatedClassInstanceType})}}, + }; + typeChecker.globalScope->exportedTypeBindings["UnrelatedClass"] = TypeFun{{}, unrelatedClassInstanceType}; + addGlobalBinding(frontend, "UnrelatedClass", unrelatedClassType, "@test"); + + TypeId vector2MetaType = arena.addType(TableTypeVar{}); + + TypeId vector2InstanceType = arena.addType(ClassTypeVar{"Vector2", {}, nullopt, vector2MetaType, {}, {}, "Test"}); + getMutable(vector2InstanceType)->props = { + {"X", {numberType}}, + {"Y", {numberType}}, + }; + + TypeId vector2Type = arena.addType(ClassTypeVar{"Vector2", {}, nullopt, nullopt, {}, {}, "Test"}); + getMutable(vector2Type)->props = { + {"New", {makeFunction(arena, nullopt, {numberType, numberType}, {vector2InstanceType})}}, + }; + getMutable(vector2MetaType)->props = { + {"__add", {makeFunction(arena, nullopt, {vector2InstanceType, vector2InstanceType}, {vector2InstanceType})}}, + }; + typeChecker.globalScope->exportedTypeBindings["Vector2"] = TypeFun{{}, vector2InstanceType}; + addGlobalBinding(frontend, "Vector2", vector2Type, "@test"); + + TypeId callableClassMetaType = arena.addType(TableTypeVar{}); + TypeId callableClassType = arena.addType(ClassTypeVar{"CallableClass", {}, nullopt, callableClassMetaType, {}, {}, "Test"}); + getMutable(callableClassMetaType)->props = { + {"__call", {makeFunction(arena, nullopt, {callableClassType, typeChecker.stringType}, {typeChecker.numberType})}}, + }; + typeChecker.globalScope->exportedTypeBindings["CallableClass"] = TypeFun{{}, callableClassType}; + + for (const auto& [name, tf] : typeChecker.globalScope->exportedTypeBindings) + persist(tf.type); + + freeze(arena); +} + +} // namespace Luau diff --git a/tests/ClassFixture.h b/tests/ClassFixture.h new file mode 100644 index 00000000..66aec764 --- /dev/null +++ b/tests/ClassFixture.h @@ -0,0 +1,13 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details + +#include "Fixture.h" + +namespace Luau +{ + +struct ClassFixture : BuiltinsFixture +{ + ClassFixture(); +}; + +} // namespace Luau diff --git a/tests/Compiler.test.cpp b/tests/Compiler.test.cpp index 3da40df8..d2cf0ae8 100644 --- a/tests/Compiler.test.cpp +++ b/tests/Compiler.test.cpp @@ -6751,6 +6751,21 @@ ADD R4 R0 R1 MOVE R0 R2 MOVE R1 R3 RETURN R0 0 +)"); + + ScopedFastFlag luauMultiAssignmentConflictFix{"LuauMultiAssignmentConflictFix", true}; + + // because we perform assignments to complex l-values after assignments to locals, we make sure register conflicts are tracked accordingly + CHECK_EQ("\n" + compileFunction0(R"( + local a, b = ... + a[1], b = b, b + 1 + )"), + R"( +GETVARARGS R0 2 +ADDK R2 R1 K0 +SETTABLEN R1 R0 1 +MOVE R1 R2 +RETURN R0 0 )"); } diff --git a/tests/DataFlowGraphBuilder.test.cpp b/tests/DataFlowGraph.test.cpp similarity index 98% rename from tests/DataFlowGraphBuilder.test.cpp rename to tests/DataFlowGraph.test.cpp index 9aa7cde6..d8230700 100644 --- a/tests/DataFlowGraphBuilder.test.cpp +++ b/tests/DataFlowGraph.test.cpp @@ -1,5 +1,5 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details -#include "Luau/DataFlowGraphBuilder.h" +#include "Luau/DataFlowGraph.h" #include "Luau/Error.h" #include "Luau/Parser.h" diff --git a/tests/Module.test.cpp b/tests/Module.test.cpp index b289b59e..33d9c75a 100644 --- a/tests/Module.test.cpp +++ b/tests/Module.test.cpp @@ -11,7 +11,6 @@ using namespace Luau; LUAU_FASTFLAG(DebugLuauDeferredConstraintResolution); -LUAU_FASTFLAG(LuauIceExceptionInheritanceChange); TEST_SUITE_BEGIN("ModuleTests"); @@ -279,14 +278,7 @@ TEST_CASE_FIXTURE(Fixture, "clone_recursion_limit") TypeArena dest; CloneState cloneState; - if (FFlag::LuauIceExceptionInheritanceChange) - { - CHECK_THROWS_AS(clone(table, dest, cloneState), RecursionLimitException); - } - else - { - CHECK_THROWS_AS(clone(table, dest, cloneState), RecursionLimitException_DEPRECATED); - } + CHECK_THROWS_AS(clone(table, dest, cloneState), RecursionLimitException); } TEST_CASE_FIXTURE(Fixture, "any_persistance_does_not_leak") diff --git a/tests/NotNull.test.cpp b/tests/NotNull.test.cpp index dfa06aa1..b827b81b 100644 --- a/tests/NotNull.test.cpp +++ b/tests/NotNull.test.cpp @@ -9,6 +9,8 @@ using Luau::NotNull; +static_assert(!std::is_convertible, bool>::value, "NotNull ought not to be convertible into bool"); + namespace { diff --git a/tests/Parser.test.cpp b/tests/Parser.test.cpp index c0989a2e..18e91e1b 100644 --- a/tests/Parser.test.cpp +++ b/tests/Parser.test.cpp @@ -2723,8 +2723,6 @@ TEST_CASE_FIXTURE(Fixture, "error_message_for_using_function_as_type_annotation" TEST_CASE_FIXTURE(Fixture, "get_a_nice_error_when_there_is_an_extra_comma_at_the_end_of_a_function_argument_list") { - ScopedFastFlag sff{"LuauCommaParenWarnings", true}; - ParseResult result = tryParse(R"( foo(a, b, c,) )"); @@ -2737,8 +2735,6 @@ TEST_CASE_FIXTURE(Fixture, "get_a_nice_error_when_there_is_an_extra_comma_at_the TEST_CASE_FIXTURE(Fixture, "get_a_nice_error_when_there_is_an_extra_comma_at_the_end_of_a_function_parameter_list") { - ScopedFastFlag sff{"LuauCommaParenWarnings", true}; - ParseResult result = tryParse(R"( export type VisitFn = ( any, @@ -2754,8 +2750,6 @@ TEST_CASE_FIXTURE(Fixture, "get_a_nice_error_when_there_is_an_extra_comma_at_the TEST_CASE_FIXTURE(Fixture, "get_a_nice_error_when_there_is_an_extra_comma_at_the_end_of_a_generic_parameter_list") { - ScopedFastFlag sff{"LuauCommaParenWarnings", true}; - ParseResult result = tryParse(R"( export type VisitFn = (a: A, b: B) -> () )"); @@ -2778,8 +2772,6 @@ TEST_CASE_FIXTURE(Fixture, "get_a_nice_error_when_there_is_an_extra_comma_at_the TEST_CASE_FIXTURE(Fixture, "get_a_nice_error_when_there_is_no_comma_between_table_members") { - ScopedFastFlag luauTableConstructorRecovery{"LuauTableConstructorRecovery", true}; - ParseResult result = tryParse(R"( local t = { first = 1 diff --git a/tests/ToString.test.cpp b/tests/ToString.test.cpp index 8bb1fbaf..29954dc4 100644 --- a/tests/ToString.test.cpp +++ b/tests/ToString.test.cpp @@ -10,7 +10,6 @@ using namespace Luau; LUAU_FASTFLAG(LuauRecursiveTypeParameterRestriction); -LUAU_FASTFLAG(LuauFunctionReturnStringificationFixup); TEST_SUITE_BEGIN("ToString"); @@ -83,7 +82,7 @@ TEST_CASE_FIXTURE(Fixture, "table_respects_use_line_break") ToStringOptions opts; opts.useLineBreaks = true; - opts.indent = true; + opts.DEPRECATED_indent = true; //clang-format off CHECK_EQ("{|\n" @@ -568,10 +567,7 @@ TEST_CASE_FIXTURE(Fixture, "no_parentheses_around_return_type_if_pack_has_an_emp TypeId functionType = arena.addType(FunctionTypeVar{argList, emptyTail}); - if (FFlag::LuauFunctionReturnStringificationFixup) - CHECK("(string) -> string" == toString(functionType)); - else - CHECK("(string) -> (string)" == toString(functionType)); + CHECK("(string) -> string" == toString(functionType)); } TEST_CASE_FIXTURE(Fixture, "no_parentheses_around_cyclic_function_type_in_union") diff --git a/tests/TypeInfer.aliases.test.cpp b/tests/TypeInfer.aliases.test.cpp index ef40e278..53c54f4f 100644 --- a/tests/TypeInfer.aliases.test.cpp +++ b/tests/TypeInfer.aliases.test.cpp @@ -9,6 +9,7 @@ using namespace Luau; LUAU_FASTFLAG(DebugLuauDeferredConstraintResolution) LUAU_FASTFLAG(LuauNoMoreGlobalSingletonTypes) +LUAU_FASTFLAG(LuauTypeMismatchInvarianceInError) TEST_SUITE_BEGIN("TypeAliases"); @@ -199,9 +200,15 @@ TEST_CASE_FIXTURE(Fixture, "generic_aliases") LUAU_REQUIRE_ERROR_COUNT(1, result); - const char* expectedError = "Type '{ v: string }' could not be converted into 'T'\n" - "caused by:\n" - " Property 'v' is not compatible. Type 'string' could not be converted into 'number'"; + const char* expectedError; + if (FFlag::LuauTypeMismatchInvarianceInError) + expectedError = "Type '{ v: string }' could not be converted into 'T'\n" + "caused by:\n" + " Property 'v' is not compatible. Type 'string' could not be converted into 'number' in an invariant context"; + else + expectedError = "Type '{ v: string }' could not be converted into 'T'\n" + "caused by:\n" + " Property 'v' is not compatible. Type 'string' could not be converted into 'number'"; CHECK(result.errors[0].location == Location{{4, 31}, {4, 44}}); CHECK(toString(result.errors[0]) == expectedError); @@ -220,11 +227,19 @@ TEST_CASE_FIXTURE(Fixture, "dependent_generic_aliases") LUAU_REQUIRE_ERROR_COUNT(1, result); - const char* expectedError = "Type '{ t: { v: string } }' could not be converted into 'U'\n" - "caused by:\n" - " Property 't' is not compatible. Type '{ v: string }' could not be converted into 'T'\n" - "caused by:\n" - " Property 'v' is not compatible. Type 'string' could not be converted into 'number'"; + const char* expectedError; + if (FFlag::LuauTypeMismatchInvarianceInError) + expectedError = "Type '{ t: { v: string } }' could not be converted into 'U'\n" + "caused by:\n" + " Property 't' is not compatible. Type '{ v: string }' could not be converted into 'T'\n" + "caused by:\n" + " Property 'v' is not compatible. Type 'string' could not be converted into 'number' in an invariant context"; + else + expectedError = "Type '{ t: { v: string } }' could not be converted into 'U'\n" + "caused by:\n" + " Property 't' is not compatible. Type '{ v: string }' could not be converted into 'T'\n" + "caused by:\n" + " Property 'v' is not compatible. Type 'string' could not be converted into 'number'"; CHECK(result.errors[0].location == Location{{4, 31}, {4, 52}}); CHECK(toString(result.errors[0]) == expectedError); diff --git a/tests/TypeInfer.annotations.test.cpp b/tests/TypeInfer.annotations.test.cpp index bb97bbeb..b94e1df0 100644 --- a/tests/TypeInfer.annotations.test.cpp +++ b/tests/TypeInfer.annotations.test.cpp @@ -7,8 +7,6 @@ #include "doctest.h" -LUAU_FASTFLAG(LuauIceExceptionInheritanceChange) - using namespace Luau; TEST_SUITE_BEGIN("AnnotationTests"); diff --git a/tests/TypeInfer.builtins.test.cpp b/tests/TypeInfer.builtins.test.cpp index 787aea9a..32e31e16 100644 --- a/tests/TypeInfer.builtins.test.cpp +++ b/tests/TypeInfer.builtins.test.cpp @@ -684,20 +684,10 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "select_with_variadic_typepack_tail") LUAU_REQUIRE_NO_ERRORS(result); - if (FFlag::DebugLuauDeferredConstraintResolution) - { - CHECK_EQ("string", toString(requireType("foo"))); - CHECK_EQ("*error-type*", toString(requireType("bar"))); - CHECK_EQ("*error-type*", toString(requireType("baz"))); - CHECK_EQ("*error-type*", toString(requireType("quux"))); - } - else - { - CHECK_EQ("any", toString(requireType("foo"))); - CHECK_EQ("any", toString(requireType("bar"))); - CHECK_EQ("any", toString(requireType("baz"))); - CHECK_EQ("any", toString(requireType("quux"))); - } + CHECK_EQ("any", toString(requireType("foo"))); + CHECK_EQ("any", toString(requireType("bar"))); + CHECK_EQ("any", toString(requireType("baz"))); + CHECK_EQ("any", toString(requireType("quux"))); } TEST_CASE_FIXTURE(BuiltinsFixture, "select_with_variadic_typepack_tail_and_string_head") @@ -714,19 +704,13 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "select_with_variadic_typepack_tail_and_strin LUAU_REQUIRE_NO_ERRORS(result); if (FFlag::DebugLuauDeferredConstraintResolution) - { CHECK_EQ("string", toString(requireType("foo"))); - CHECK_EQ("string", toString(requireType("bar"))); - CHECK_EQ("*error-type*", toString(requireType("baz"))); - CHECK_EQ("*error-type*", toString(requireType("quux"))); - } else - { CHECK_EQ("any", toString(requireType("foo"))); - CHECK_EQ("any", toString(requireType("bar"))); - CHECK_EQ("any", toString(requireType("baz"))); - CHECK_EQ("any", toString(requireType("quux"))); - } + + CHECK_EQ("any", toString(requireType("bar"))); + CHECK_EQ("any", toString(requireType("baz"))); + CHECK_EQ("any", toString(requireType("quux"))); } TEST_CASE_FIXTURE(Fixture, "string_format_as_method") diff --git a/tests/TypeInfer.classes.test.cpp b/tests/TypeInfer.classes.test.cpp index d1a24e5f..07dfc33f 100644 --- a/tests/TypeInfer.classes.test.cpp +++ b/tests/TypeInfer.classes.test.cpp @@ -1,109 +1,18 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #include "Luau/BuiltinDefinitions.h" +#include "Luau/Common.h" #include "Luau/TypeInfer.h" #include "Luau/TypeVar.h" #include "Fixture.h" +#include "ClassFixture.h" #include "doctest.h" using namespace Luau; using std::nullopt; -struct ClassFixture : BuiltinsFixture -{ - ClassFixture() - { - TypeArena& arena = typeChecker.globalTypes; - TypeId numberType = typeChecker.numberType; - - unfreeze(arena); - - TypeId baseClassInstanceType = arena.addType(ClassTypeVar{"BaseClass", {}, nullopt, nullopt, {}, {}, "Test"}); - getMutable(baseClassInstanceType)->props = { - {"BaseMethod", {makeFunction(arena, baseClassInstanceType, {numberType}, {})}}, - {"BaseField", {numberType}}, - }; - - TypeId baseClassType = arena.addType(ClassTypeVar{"BaseClass", {}, nullopt, nullopt, {}, {}, "Test"}); - getMutable(baseClassType)->props = { - {"StaticMethod", {makeFunction(arena, nullopt, {}, {numberType})}}, - {"Clone", {makeFunction(arena, nullopt, {baseClassInstanceType}, {baseClassInstanceType})}}, - {"New", {makeFunction(arena, nullopt, {}, {baseClassInstanceType})}}, - }; - typeChecker.globalScope->exportedTypeBindings["BaseClass"] = TypeFun{{}, baseClassInstanceType}; - addGlobalBinding(frontend, "BaseClass", baseClassType, "@test"); - - TypeId childClassInstanceType = arena.addType(ClassTypeVar{"ChildClass", {}, baseClassInstanceType, nullopt, {}, {}, "Test"}); - - getMutable(childClassInstanceType)->props = { - {"Method", {makeFunction(arena, childClassInstanceType, {}, {typeChecker.stringType})}}, - }; - - TypeId childClassType = arena.addType(ClassTypeVar{"ChildClass", {}, baseClassType, nullopt, {}, {}, "Test"}); - getMutable(childClassType)->props = { - {"New", {makeFunction(arena, nullopt, {}, {childClassInstanceType})}}, - }; - typeChecker.globalScope->exportedTypeBindings["ChildClass"] = TypeFun{{}, childClassInstanceType}; - addGlobalBinding(frontend, "ChildClass", childClassType, "@test"); - - TypeId grandChildInstanceType = arena.addType(ClassTypeVar{"GrandChild", {}, childClassInstanceType, nullopt, {}, {}, "Test"}); - - getMutable(grandChildInstanceType)->props = { - {"Method", {makeFunction(arena, grandChildInstanceType, {}, {typeChecker.stringType})}}, - }; - - TypeId grandChildType = arena.addType(ClassTypeVar{"GrandChild", {}, baseClassType, nullopt, {}, {}, "Test"}); - getMutable(grandChildType)->props = { - {"New", {makeFunction(arena, nullopt, {}, {grandChildInstanceType})}}, - }; - typeChecker.globalScope->exportedTypeBindings["GrandChild"] = TypeFun{{}, grandChildInstanceType}; - addGlobalBinding(frontend, "GrandChild", childClassType, "@test"); - - TypeId anotherChildInstanceType = arena.addType(ClassTypeVar{"AnotherChild", {}, baseClassInstanceType, nullopt, {}, {}, "Test"}); - - getMutable(anotherChildInstanceType)->props = { - {"Method", {makeFunction(arena, anotherChildInstanceType, {}, {typeChecker.stringType})}}, - }; - - TypeId anotherChildType = arena.addType(ClassTypeVar{"AnotherChild", {}, baseClassType, nullopt, {}, {}, "Test"}); - getMutable(anotherChildType)->props = { - {"New", {makeFunction(arena, nullopt, {}, {anotherChildInstanceType})}}, - }; - typeChecker.globalScope->exportedTypeBindings["AnotherChild"] = TypeFun{{}, anotherChildInstanceType}; - addGlobalBinding(frontend, "AnotherChild", childClassType, "@test"); - - TypeId vector2MetaType = arena.addType(TableTypeVar{}); - - TypeId vector2InstanceType = arena.addType(ClassTypeVar{"Vector2", {}, nullopt, vector2MetaType, {}, {}, "Test"}); - getMutable(vector2InstanceType)->props = { - {"X", {numberType}}, - {"Y", {numberType}}, - }; - - TypeId vector2Type = arena.addType(ClassTypeVar{"Vector2", {}, nullopt, nullopt, {}, {}, "Test"}); - getMutable(vector2Type)->props = { - {"New", {makeFunction(arena, nullopt, {numberType, numberType}, {vector2InstanceType})}}, - }; - getMutable(vector2MetaType)->props = { - {"__add", {makeFunction(arena, nullopt, {vector2InstanceType, vector2InstanceType}, {vector2InstanceType})}}, - }; - typeChecker.globalScope->exportedTypeBindings["Vector2"] = TypeFun{{}, vector2InstanceType}; - addGlobalBinding(frontend, "Vector2", vector2Type, "@test"); - - TypeId callableClassMetaType = arena.addType(TableTypeVar{}); - TypeId callableClassType = arena.addType(ClassTypeVar{"CallableClass", {}, nullopt, callableClassMetaType, {}, {}, "Test"}); - getMutable(callableClassMetaType)->props = { - {"__call", {makeFunction(arena, nullopt, {callableClassType, typeChecker.stringType}, {typeChecker.numberType})}}, - }; - typeChecker.globalScope->exportedTypeBindings["CallableClass"] = TypeFun{{}, callableClassType}; - - for (const auto& [name, tf] : typeChecker.globalScope->exportedTypeBindings) - persist(tf.type); - - freeze(arena); - } -}; +LUAU_FASTFLAG(LuauTypeMismatchInvarianceInError); TEST_SUITE_BEGIN("TypeInferClasses"); @@ -521,6 +430,56 @@ TEST_CASE_FIXTURE(ClassFixture, "unions_of_intersections_of_classes") LUAU_REQUIRE_NO_ERRORS(result); } +TEST_CASE_FIXTURE(ClassFixture, "index_instance_property") +{ + ScopedFastFlag luauAllowIndexClassParameters{"LuauAllowIndexClassParameters", true}; + + CheckResult result = check(R"( + local function execute(object: BaseClass, name: string) + print(object[name]) + end + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ("Attempting a dynamic property access on type 'BaseClass' is unsafe and may cause exceptions at runtime", toString(result.errors[0])); +} + +TEST_CASE_FIXTURE(ClassFixture, "index_instance_property_nonstrict") +{ + ScopedFastFlag luauAllowIndexClassParameters{"LuauAllowIndexClassParameters", true}; + + CheckResult result = check(R"( + --!nonstrict + + local function execute(object: BaseClass, name: string) + print(object[name]) + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(ClassFixture, "type_mismatch_invariance_required_for_error") +{ + CheckResult result = check(R"( +type A = { x: ChildClass } +type B = { x: BaseClass } + +local a: A +local b: B = a + )"); + + LUAU_REQUIRE_ERRORS(result); + if (FFlag::LuauTypeMismatchInvarianceInError) + CHECK_EQ(toString(result.errors[0]), R"(Type 'A' could not be converted into 'B' +caused by: + Property 'x' is not compatible. Type 'ChildClass' could not be converted into 'BaseClass' in an invariant context)"); + else + CHECK_EQ(toString(result.errors[0]), R"(Type 'A' could not be converted into 'B' +caused by: + Property 'x' is not compatible. Type 'ChildClass' could not be converted into 'BaseClass')"); +} + TEST_CASE_FIXTURE(ClassFixture, "callable_classes") { ScopedFastFlag luauCallableClasses{"LuauCallableClasses", true}; diff --git a/tests/TypeInfer.definitions.test.cpp b/tests/TypeInfer.definitions.test.cpp index 3556f0f1..26115046 100644 --- a/tests/TypeInfer.definitions.test.cpp +++ b/tests/TypeInfer.definitions.test.cpp @@ -311,8 +311,6 @@ TEST_CASE_FIXTURE(Fixture, "definitions_documentation_symbols") TEST_CASE_FIXTURE(Fixture, "definitions_symbols_are_generated_for_recursively_referenced_types") { - ScopedFastFlag LuauPersistTypesAfterGeneratingDocSyms("LuauPersistTypesAfterGeneratingDocSyms", true); - loadDefinition(R"( declare class MyClass function myMethod(self) diff --git a/tests/TypeInfer.functions.test.cpp b/tests/TypeInfer.functions.test.cpp index b306515a..55218040 100644 --- a/tests/TypeInfer.functions.test.cpp +++ b/tests/TypeInfer.functions.test.cpp @@ -230,8 +230,6 @@ TEST_CASE_FIXTURE(Fixture, "too_many_arguments") TEST_CASE_FIXTURE(Fixture, "too_many_arguments_error_location") { - ScopedFastFlag sff{"LuauArgMismatchReportFunctionLocation", true}; - CheckResult result = check(R"( --!strict @@ -507,7 +505,9 @@ TEST_CASE_FIXTURE(Fixture, "complicated_return_types_require_an_explicit_annotat LUAU_REQUIRE_NO_ERRORS(result); - const FunctionTypeVar* functionType = get(requireType("most_of_the_natural_numbers")); + TypeId ty = requireType("most_of_the_natural_numbers"); + const FunctionTypeVar* functionType = get(ty); + REQUIRE_MESSAGE(functionType, "Expected function but got " << toString(ty)); std::optional retType = first(functionType->retTypes); REQUIRE(retType); @@ -1830,4 +1830,18 @@ TEST_CASE_FIXTURE(Fixture, "other_things_are_not_related_to_function") CHECK(5 == result.errors[3].location.begin.line); } +TEST_CASE_FIXTURE(BuiltinsFixture, "fuzz_must_follow_in_overload_resolution") +{ + ScopedFastFlag luauTypeInferMissingFollows{"LuauTypeInferMissingFollows", true}; + + CheckResult result = check(R"( +for _ in function():(t0)&((()->())&(()->())) +end do +_(_(_,_,_),_) +end + )"); + + LUAU_REQUIRE_ERRORS(result); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.generics.test.cpp b/tests/TypeInfer.generics.test.cpp index de41c3a6..c25f8e5f 100644 --- a/tests/TypeInfer.generics.test.cpp +++ b/tests/TypeInfer.generics.test.cpp @@ -10,6 +10,7 @@ #include "doctest.h" LUAU_FASTFLAG(LuauInstantiateInSubtyping) +LUAU_FASTFLAG(LuauTypeMismatchInvarianceInError) using namespace Luau; @@ -717,12 +718,24 @@ y.a.c = y )"); LUAU_REQUIRE_ERRORS(result); - CHECK_EQ(toString(result.errors[0]), - R"(Type 'y' could not be converted into 'T' + if (FFlag::LuauTypeMismatchInvarianceInError) + { + CHECK_EQ(toString(result.errors[0]), + R"(Type 'y' could not be converted into 'T' +caused by: + Property 'a' is not compatible. Type '{ c: T?, d: number }' could not be converted into 'U' +caused by: + Property 'd' is not compatible. Type 'number' could not be converted into 'string' in an invariant context)"); + } + else + { + CHECK_EQ(toString(result.errors[0]), + R"(Type 'y' could not be converted into 'T' caused by: Property 'a' is not compatible. Type '{ c: T?, d: number }' could not be converted into 'U' caused by: Property 'd' is not compatible. Type 'number' could not be converted into 'string')"); + } } TEST_CASE_FIXTURE(Fixture, "generic_type_pack_unification1") @@ -1249,4 +1262,21 @@ instantiate(function(x: string) return "foo" end) CHECK_EQ("(string) -> string", toString(tm1->givenType)); } +TEST_CASE_FIXTURE(Fixture, "bidirectional_checking_and_generalization_play_nice") +{ + CheckResult result = check(R"( + local foo = function(a) + return a() + end + + local a = foo(function() return 1 end) + local b = foo(function() return "bar" end) + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK("number" == toString(requireType("a"))); + CHECK("string" == toString(requireType("b"))); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.intersectionTypes.test.cpp b/tests/TypeInfer.intersectionTypes.test.cpp index 0c10eb87..7d0621a7 100644 --- a/tests/TypeInfer.intersectionTypes.test.cpp +++ b/tests/TypeInfer.intersectionTypes.test.cpp @@ -185,7 +185,15 @@ TEST_CASE_FIXTURE(Fixture, "index_on_an_intersection_type_works_at_arbitrary_dep )"); LUAU_REQUIRE_NO_ERRORS(result); - CHECK_EQ("string & string", toString(requireType("r"))); + + if (FFlag::DebugLuauDeferredConstraintResolution) + { + CHECK_EQ("string", toString(requireType("r"))); + } + else + { + CHECK_EQ("string & string", toString(requireType("r"))); + } } TEST_CASE_FIXTURE(Fixture, "index_on_an_intersection_type_with_mixed_types") @@ -199,7 +207,7 @@ TEST_CASE_FIXTURE(Fixture, "index_on_an_intersection_type_with_mixed_types") )"); LUAU_REQUIRE_NO_ERRORS(result); - CHECK_EQ("number & string", toString(requireType("r"))); // TODO(amccord): This should be an error. + CHECK_EQ("number & string", toString(requireType("r"))); } TEST_CASE_FIXTURE(Fixture, "index_on_an_intersection_type_with_one_part_missing_the_property") @@ -525,18 +533,16 @@ TEST_CASE_FIXTURE(Fixture, "intersection_of_tables_with_never_properties") ScopedFastFlag sffs[]{ {"LuauSubtypeNormalizer", true}, {"LuauTypeNormalization2", true}, + {"LuauUninhabitedSubAnything", true}, }; CheckResult result = check(R"( - local x : { p : number?, q : never } & { p : never, q : string? } + local x : { p : number?, q : never } & { p : never, q : string? } -- OK local y : { p : never, q : never } = x -- OK local z : never = x -- OK )"); - // TODO: this should not produce type errors, since never <: { p : never } - LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK_EQ(toString(result.errors[0]), "Type '{| p: never, q: string? |} & {| p: number?, q: never |}' could not be converted into 'never'; none " - "of the intersection parts are compatible"); + LUAU_REQUIRE_NO_ERRORS(result); } TEST_CASE_FIXTURE(Fixture, "overloaded_functions_returning_intersections") @@ -848,7 +854,7 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "intersect_metatables_with_properties") LUAU_REQUIRE_NO_ERRORS(result); } -TEST_CASE_FIXTURE(BuiltinsFixture, "intersect_metatable_with table") +TEST_CASE_FIXTURE(BuiltinsFixture, "intersect_metatable_with_table") { ScopedFastFlag sffs[]{ {"LuauSubtypeNormalizer", true}, @@ -902,4 +908,37 @@ TEST_CASE_FIXTURE(Fixture, "CLI-44817") LUAU_REQUIRE_NO_ERRORS(result); } +TEST_CASE_FIXTURE(Fixture, "less_greedy_unification_with_intersection_types") +{ + if (!FFlag::DebugLuauDeferredConstraintResolution) + return; + + CheckResult result = check(R"( + local function f(t): { x: number } & { x: string } + local x = t.x + return t + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ("({| x: number |} & {| x: string |}) -> {| x: number |} & {| x: string |}", toString(requireType("f"))); +} + +TEST_CASE_FIXTURE(Fixture, "less_greedy_unification_with_intersection_types_2") +{ + if (!FFlag::DebugLuauDeferredConstraintResolution) + return; + + CheckResult result = check(R"( + local function f(t: { x: number } & { x: string }) + return t.x + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ("({| x: number |} & {| x: string |}) -> number & string", toString(requireType("f"))); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.modules.test.cpp b/tests/TypeInfer.modules.test.cpp index 4cc628fb..b06c80e9 100644 --- a/tests/TypeInfer.modules.test.cpp +++ b/tests/TypeInfer.modules.test.cpp @@ -11,6 +11,7 @@ #include "doctest.h" LUAU_FASTFLAG(LuauInstantiateInSubtyping) +LUAU_FASTFLAG(LuauTypeMismatchInvarianceInError) using namespace Luau; @@ -408,7 +409,12 @@ local b: B.T = a CheckResult result = frontend.check("game/C"); LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK_EQ(toString(result.errors[0]), R"(Type 'T' from 'game/A' could not be converted into 'T' from 'game/B' + if (FFlag::LuauTypeMismatchInvarianceInError) + CHECK_EQ(toString(result.errors[0]), R"(Type 'T' from 'game/A' could not be converted into 'T' from 'game/B' +caused by: + Property 'x' is not compatible. Type 'number' could not be converted into 'string' in an invariant context)"); + else + CHECK_EQ(toString(result.errors[0]), R"(Type 'T' from 'game/A' could not be converted into 'T' from 'game/B' caused by: Property 'x' is not compatible. Type 'number' could not be converted into 'string')"); } @@ -442,7 +448,12 @@ local b: B.T = a CheckResult result = frontend.check("game/D"); LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK_EQ(toString(result.errors[0]), R"(Type 'T' from 'game/B' could not be converted into 'T' from 'game/C' + if (FFlag::LuauTypeMismatchInvarianceInError) + CHECK_EQ(toString(result.errors[0]), R"(Type 'T' from 'game/B' could not be converted into 'T' from 'game/C' +caused by: + Property 'x' is not compatible. Type 'number' could not be converted into 'string' in an invariant context)"); + else + CHECK_EQ(toString(result.errors[0]), R"(Type 'T' from 'game/B' could not be converted into 'T' from 'game/C' caused by: Property 'x' is not compatible. Type 'number' could not be converted into 'string')"); } @@ -462,4 +473,15 @@ return l0 LUAU_REQUIRE_NO_ERRORS(result); } +TEST_CASE_FIXTURE(BuiltinsFixture, "fuzz_anyify_variadic_return_must_follow") +{ + ScopedFastFlag luauTypeInferMissingFollows{"LuauTypeInferMissingFollows", true}; + + CheckResult result = check(R"( +return unpack(l0[_]) + )"); + + LUAU_REQUIRE_ERRORS(result); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.operators.test.cpp b/tests/TypeInfer.operators.test.cpp index c4bbc2e1..21806082 100644 --- a/tests/TypeInfer.operators.test.cpp +++ b/tests/TypeInfer.operators.test.cpp @@ -8,6 +8,7 @@ #include "Luau/VisitTypeVar.h" #include "Fixture.h" +#include "ClassFixture.h" #include "doctest.h" @@ -817,6 +818,21 @@ TEST_CASE_FIXTURE(Fixture, "operator_eq_operands_are_not_subtypes_of_each_other_ LUAU_REQUIRE_NO_ERRORS(result); } +TEST_CASE_FIXTURE(Fixture, "operator_eq_completely_incompatible") +{ + ScopedFastFlag sff{"LuauIntersectionTestForEquality", true}; + + CheckResult result = check(R"( + local a: string | number = "hi" + local b: {x: string}? = {x = "bye"} + + local r1 = a == b + local r2 = b == a + )"); + + LUAU_REQUIRE_ERROR_COUNT(2, result); +} + TEST_CASE_FIXTURE(Fixture, "refine_and_or") { CheckResult result = check(R"( @@ -916,6 +932,31 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "expected_types_through_binary_or") LUAU_REQUIRE_NO_ERRORS(result); } +TEST_CASE_FIXTURE(ClassFixture, "unrelated_classes_cannot_be_compared") +{ + ScopedFastFlag sff{"LuauIntersectionTestForEquality", true}; + + CheckResult result = check(R"( + local a = BaseClass.New() + local b = UnrelatedClass.New() + + local c = a == b + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); +} + +TEST_CASE_FIXTURE(Fixture, "unrelated_primitives_cannot_be_compared") +{ + ScopedFastFlag sff{"LuauIntersectionTestForEquality", true}; + + CheckResult result = check(R"( + local c = 5 == true + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); +} + TEST_CASE_FIXTURE(BuiltinsFixture, "mm_ops_must_return_a_value") { if (!FFlag::DebugLuauDeferredConstraintResolution) diff --git a/tests/TypeInfer.provisional.test.cpp b/tests/TypeInfer.provisional.test.cpp index 5688eaaa..25934174 100644 --- a/tests/TypeInfer.provisional.test.cpp +++ b/tests/TypeInfer.provisional.test.cpp @@ -170,24 +170,12 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "error_on_eq_metamethod_returning_a_type_othe CHECK_EQ("Metamethod '__eq' must return type 'boolean'", ge->message); } -// Requires success typing to confidently determine that this expression has no overlap. -TEST_CASE_FIXTURE(Fixture, "operator_eq_completely_incompatible") -{ - CheckResult result = check(R"( - local a: string | number = "hi" - local b: {x: string}? = {x = "bye"} - - local r1 = a == b - local r2 = b == a - )"); - - LUAU_REQUIRE_NO_ERRORS(result); -} - // Belongs in TypeInfer.refinements.test.cpp. -// We'll need to not only report an error on `a == b`, but also to refine both operands as `never` in the `==` branch. +// We need refine both operands as `never` in the `==` branch. TEST_CASE_FIXTURE(Fixture, "lvalue_equals_another_lvalue_with_no_overlap") { + ScopedFastFlag sff{"LuauIntersectionTestForEquality", true}; + CheckResult result = check(R"( local function f(a: string, b: boolean?) if a == b then @@ -198,7 +186,7 @@ TEST_CASE_FIXTURE(Fixture, "lvalue_equals_another_lvalue_with_no_overlap") end )"); - LUAU_REQUIRE_NO_ERRORS(result); + LUAU_REQUIRE_ERROR_COUNT(1, result); CHECK_EQ(toString(requireTypeAtPosition({3, 33})), "string"); // a == b CHECK_EQ(toString(requireTypeAtPosition({3, 36})), "boolean?"); // a == b diff --git a/tests/TypeInfer.refinements.test.cpp b/tests/TypeInfer.refinements.test.cpp index 66550be3..e5bc186a 100644 --- a/tests/TypeInfer.refinements.test.cpp +++ b/tests/TypeInfer.refinements.test.cpp @@ -8,6 +8,7 @@ #include "doctest.h" LUAU_FASTFLAG(DebugLuauDeferredConstraintResolution) +LUAU_FASTFLAG(LuauTypeMismatchInvarianceInError) using namespace Luau; @@ -35,6 +36,27 @@ std::optional> magicFunctionInstanceIsA( return WithPredicate{booleanPack, {IsAPredicate{std::move(*lvalue), expr.location, tfun->type}}}; } +std::vector dcrMagicRefinementInstanceIsA(MagicRefinementContext ctx) +{ + if (ctx.callSite->args.size != 1) + return {}; + + auto index = ctx.callSite->func->as(); + auto str = ctx.callSite->args.data[0]->as(); + if (!index || !str) + return {}; + + std::optional def = ctx.dfg->getDef(index->expr); + if (!def) + return {}; + + std::optional tfun = ctx.scope->lookupType(std::string(str->value.data, str->value.size)); + if (!tfun) + return {}; + + return {ctx.connectiveArena->proposition(*def, tfun->type)}; +} + struct RefinementClassFixture : BuiltinsFixture { RefinementClassFixture() @@ -56,6 +78,7 @@ struct RefinementClassFixture : BuiltinsFixture TypePackId isARets = arena.addTypePack({typeChecker.booleanType}); TypeId isA = arena.addType(FunctionTypeVar{isAParams, isARets}); getMutable(isA)->magicFunction = magicFunctionInstanceIsA; + getMutable(isA)->dcrMagicRefinement = dcrMagicRefinementInstanceIsA; getMutable(inst)->props = { {"Name", Property{typeChecker.stringType}}, @@ -397,13 +420,21 @@ TEST_CASE_FIXTURE(Fixture, "truthy_constraint_on_properties") local t: {x: number?} = {x = 1} if t.x then - local foo: number = t.x + local t2 = t + local foo = t.x end local bar = t.x )"); LUAU_REQUIRE_NO_ERRORS(result); + + if (FFlag::DebugLuauDeferredConstraintResolution) + { + CHECK("{| x: number? |} & {| x: ~(false?) |}" == toString(requireTypeAtPosition({4, 23}))); + CHECK("(number?) & ~(false?)" == toString(requireTypeAtPosition({5, 26}))); + } + CHECK_EQ("number?", toString(requireType("bar"))); } @@ -442,12 +473,24 @@ TEST_CASE_FIXTURE(Fixture, "assign_table_with_refined_property_with_a_similar_ty )"); LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK_EQ(R"(Type '{| x: number? |}' could not be converted into '{| x: number |}' + + if (FFlag::LuauTypeMismatchInvarianceInError) + { + CHECK_EQ(R"(Type '{| x: number? |}' could not be converted into '{| x: number |}' +caused by: + Property 'x' is not compatible. Type 'number?' could not be converted into 'number' in an invariant context)", + toString(result.errors[0])); + } + else + { + CHECK_EQ(R"(Type '{| x: number? |}' could not be converted into '{| x: number |}' caused by: Property 'x' is not compatible. Type 'number?' could not be converted into 'number')", - toString(result.errors[0])); + toString(result.errors[0])); + } } + TEST_CASE_FIXTURE(Fixture, "lvalue_is_equal_to_another_lvalue") { CheckResult result = check(R"( @@ -464,8 +507,8 @@ TEST_CASE_FIXTURE(Fixture, "lvalue_is_equal_to_another_lvalue") if (FFlag::DebugLuauDeferredConstraintResolution) { - CHECK_EQ(toString(requireTypeAtPosition({3, 33})), "((number | string)?) & (boolean?)"); // a == b - CHECK_EQ(toString(requireTypeAtPosition({3, 36})), "((number | string)?) & (boolean?)"); // a == b + CHECK_EQ(toString(requireTypeAtPosition({3, 33})), "((number | string)?) & unknown"); // a == b + CHECK_EQ(toString(requireTypeAtPosition({3, 36})), "(boolean?) & unknown"); // a == b CHECK_EQ(toString(requireTypeAtPosition({5, 33})), "((number | string)?) & unknown"); // a ~= b CHECK_EQ(toString(requireTypeAtPosition({5, 36})), "(boolean?) & unknown"); // a ~= b @@ -496,7 +539,7 @@ TEST_CASE_FIXTURE(Fixture, "lvalue_is_equal_to_a_term") if (FFlag::DebugLuauDeferredConstraintResolution) { - CHECK_EQ(toString(requireTypeAtPosition({3, 28})), "((number | string)?) & number"); // a == 1 + CHECK_EQ(toString(requireTypeAtPosition({3, 28})), "((number | string)?) & unknown"); // a == 1 CHECK_EQ(toString(requireTypeAtPosition({5, 28})), "((number | string)?) & unknown"); // a ~= 1 } else @@ -548,8 +591,8 @@ TEST_CASE_FIXTURE(Fixture, "lvalue_is_not_nil") if (FFlag::DebugLuauDeferredConstraintResolution) { - CHECK_EQ(toString(requireTypeAtPosition({3, 28})), "((number | string)?) & ~nil"); // a ~= nil - CHECK_EQ(toString(requireTypeAtPosition({5, 28})), "((number | string)?) & nil"); // a == nil + CHECK_EQ(toString(requireTypeAtPosition({3, 28})), "((number | string)?) & ~nil"); // a ~= nil + CHECK_EQ(toString(requireTypeAtPosition({5, 28})), "((number | string)?) & unknown"); // a == nil } else { @@ -573,8 +616,8 @@ TEST_CASE_FIXTURE(Fixture, "free_type_is_equal_to_an_lvalue") if (FFlag::DebugLuauDeferredConstraintResolution) { ToStringOptions opts; - CHECK_EQ(toString(requireTypeAtPosition({3, 33}), opts), "(string?) & a"); // a == b - CHECK_EQ(toString(requireTypeAtPosition({3, 36}), opts), "(string?) & a"); // a == b + CHECK_EQ(toString(requireTypeAtPosition({3, 33}), opts), "a & unknown"); // a == b + CHECK_EQ(toString(requireTypeAtPosition({3, 36}), opts), "(string?) & unknown"); // a == b } else { @@ -628,8 +671,8 @@ TEST_CASE_FIXTURE(Fixture, "string_not_equal_to_string_or_nil") CHECK_EQ(toString(requireTypeAtPosition({6, 29})), "string & unknown"); // a ~= b CHECK_EQ(toString(requireTypeAtPosition({6, 32})), "(string?) & unknown"); // a ~= b - CHECK_EQ(toString(requireTypeAtPosition({8, 29})), "(string?) & string"); // a == b - CHECK_EQ(toString(requireTypeAtPosition({8, 32})), "(string?) & string"); // a == b + CHECK_EQ(toString(requireTypeAtPosition({8, 29})), "string & unknown"); // a == b + CHECK_EQ(toString(requireTypeAtPosition({8, 32})), "(string?) & unknown"); // a == b } else { @@ -1146,8 +1189,17 @@ TEST_CASE_FIXTURE(Fixture, "discriminate_from_truthiness_of_x") LUAU_REQUIRE_NO_ERRORS(result); - CHECK_EQ(R"({| tag: "exists", x: string |})", toString(requireTypeAtPosition({5, 28}))); - CHECK_EQ(R"({| tag: "exists", x: string |} | {| tag: "missing", x: nil |})", toString(requireTypeAtPosition({7, 28}))); + if (FFlag::DebugLuauDeferredConstraintResolution) + { + CHECK_EQ(R"(({| tag: "exists", x: string |} | {| tag: "missing", x: nil |}) & {| x: ~(false?) |})", toString(requireTypeAtPosition({5, 28}))); + CHECK_EQ( + R"(({| tag: "exists", x: string |} | {| tag: "missing", x: nil |}) & {| x: ~~(false?) |})", toString(requireTypeAtPosition({7, 28}))); + } + else + { + CHECK_EQ(R"({| tag: "exists", x: string |})", toString(requireTypeAtPosition({5, 28}))); + CHECK_EQ(R"({| tag: "exists", x: string |} | {| tag: "missing", x: nil |})", toString(requireTypeAtPosition({7, 28}))); + } } TEST_CASE_FIXTURE(Fixture, "discriminate_tag") @@ -1159,17 +1211,57 @@ TEST_CASE_FIXTURE(Fixture, "discriminate_tag") local function f(animal: Animal) if animal.tag == "Cat" then - local cat: Cat = animal + local cat = animal elseif animal.tag == "Dog" then - local dog: Dog = animal + local dog = animal end end )"); LUAU_REQUIRE_NO_ERRORS(result); - CHECK_EQ("Cat", toString(requireTypeAtPosition({7, 33}))); - CHECK_EQ("Dog", toString(requireTypeAtPosition({9, 33}))); + if (FFlag::DebugLuauDeferredConstraintResolution) + { + CHECK_EQ(R"((Cat | Dog) & {| tag: "Cat" |})", toString(requireTypeAtPosition({7, 33}))); + CHECK_EQ(R"((Cat | Dog) & {| tag: ~"Cat" |} & {| tag: "Dog" |})", toString(requireTypeAtPosition({9, 33}))); + } + else + { + CHECK_EQ("Cat", toString(requireTypeAtPosition({7, 33}))); + CHECK_EQ("Dog", toString(requireTypeAtPosition({9, 33}))); + } +} + +TEST_CASE_FIXTURE(Fixture, "discriminate_tag_with_implicit_else") +{ + ScopedFastFlag sff{"LuauImplicitElseRefinement", true}; + + CheckResult result = check(R"( + type Cat = {tag: "Cat", name: string, catfood: string} + type Dog = {tag: "Dog", name: string, dogfood: string} + type Animal = Cat | Dog + + local function f(animal: Animal) + if animal.tag == "Cat" then + local cat = animal + else + local dog = animal + end + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + if (FFlag::DebugLuauDeferredConstraintResolution) + { + CHECK_EQ(R"((Cat | Dog) & {| tag: "Cat" |})", toString(requireTypeAtPosition({7, 33}))); + CHECK_EQ(R"((Cat | Dog) & {| tag: ~"Cat" |})", toString(requireTypeAtPosition({9, 33}))); + } + else + { + CHECK_EQ("Cat", toString(requireTypeAtPosition({7, 33}))); + CHECK_EQ("Dog", toString(requireTypeAtPosition({9, 33}))); + } } TEST_CASE_FIXTURE(Fixture, "and_or_peephole_refinement") @@ -1258,8 +1350,16 @@ TEST_CASE_FIXTURE(RefinementClassFixture, "discriminate_from_isa_of_x") LUAU_REQUIRE_NO_ERRORS(result); - CHECK_EQ(R"({| tag: "Part", x: Part |})", toString(requireTypeAtPosition({5, 28}))); - CHECK_EQ(R"({| tag: "Folder", x: Folder |})", toString(requireTypeAtPosition({7, 28}))); + if (FFlag::DebugLuauDeferredConstraintResolution) + { + CHECK_EQ(R"(({| tag: "Folder", x: Folder |} | {| tag: "Part", x: Part |}) & {| x: Part |})", toString(requireTypeAtPosition({5, 28}))); + CHECK_EQ(R"(({| tag: "Folder", x: Folder |} | {| tag: "Part", x: Part |}) & {| x: ~Part |})", toString(requireTypeAtPosition({7, 28}))); + } + else + { + CHECK_EQ(R"({| tag: "Part", x: Part |})", toString(requireTypeAtPosition({5, 28}))); + CHECK_EQ(R"({| tag: "Folder", x: Folder |})", toString(requireTypeAtPosition({7, 28}))); + } } TEST_CASE_FIXTURE(RefinementClassFixture, "typeguard_cast_free_table_to_vector") @@ -1399,8 +1499,96 @@ TEST_CASE_FIXTURE(RefinementClassFixture, "x_as_any_if_x_is_instance_elseif_x_is LUAU_REQUIRE_NO_ERRORS(result); - CHECK_EQ("Folder", toString(requireTypeAtPosition({5, 28}))); - CHECK_EQ("any", toString(requireTypeAtPosition({7, 28}))); + if (FFlag::DebugLuauDeferredConstraintResolution) + { + CHECK_EQ("Folder & Instance & {- -}", toString(requireTypeAtPosition({5, 28}))); + CHECK_EQ("(~Folder | ~Instance) & {- -} & never", toString(requireTypeAtPosition({7, 28}))); + } + else + { + CHECK_EQ("Folder", toString(requireTypeAtPosition({5, 28}))); + CHECK_EQ("any", toString(requireTypeAtPosition({7, 28}))); + } +} + +TEST_CASE_FIXTURE(RefinementClassFixture, "refine_param_of_type_instance_without_using_typeof") +{ + CheckResult result = check(R"( + local function f(x: Instance) + if x:IsA("Folder") then + local foo = x + elseif typeof(x) == "table" then + local foo = x + end + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + if (FFlag::DebugLuauDeferredConstraintResolution) + { + CHECK_EQ("Folder & Instance", toString(requireTypeAtPosition({3, 28}))); + CHECK_EQ("Instance & ~Folder & never", toString(requireTypeAtPosition({5, 28}))); + } + else + { + CHECK_EQ("Folder", toString(requireTypeAtPosition({3, 28}))); + CHECK_EQ("never", toString(requireTypeAtPosition({5, 28}))); + } +} + +TEST_CASE_FIXTURE(RefinementClassFixture, "refine_param_of_type_folder_or_part_without_using_typeof") +{ + CheckResult result = check(R"( + local function f(x: Part | Folder) + if x:IsA("Folder") then + local foo = x + else + local foo = x + end + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + if (FFlag::DebugLuauDeferredConstraintResolution) + { + CHECK_EQ("(Folder | Part) & Folder", toString(requireTypeAtPosition({3, 28}))); + CHECK_EQ("(Folder | Part) & ~Folder", toString(requireTypeAtPosition({5, 28}))); + } + else + { + CHECK_EQ("Folder", toString(requireTypeAtPosition({3, 28}))); + CHECK_EQ("Part", toString(requireTypeAtPosition({5, 28}))); + } +} + +TEST_CASE_FIXTURE(RefinementClassFixture, "isa_type_refinement_must_be_known_ahead_of_time") +{ + CheckResult result = check(R"( + local function f(x): Instance + if x:IsA("Folder") then + local foo = x + else + local foo = x + end + + return x + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + if (FFlag::DebugLuauDeferredConstraintResolution) + { + CHECK_EQ("Instance", toString(requireTypeAtPosition({3, 28}))); + CHECK_EQ("Instance", toString(requireTypeAtPosition({5, 28}))); + } + else + { + CHECK_EQ("Instance", toString(requireTypeAtPosition({3, 28}))); + CHECK_EQ("Instance", toString(requireTypeAtPosition({5, 28}))); + } } TEST_CASE_FIXTURE(RefinementClassFixture, "x_is_not_instance_or_else_not_part") @@ -1526,4 +1714,18 @@ TEST_CASE_FIXTURE(Fixture, "else_with_no_explicit_expression_should_also_refine_ LUAU_REQUIRE_NO_ERRORS(result); } +TEST_CASE_FIXTURE(Fixture, "fuzz_filtered_refined_types_are_followed") +{ + ScopedFastFlag luauTypeInferMissingFollows{"LuauTypeInferMissingFollows", true}; + + CheckResult result = check(R"( +local _ +do +local _ = _ ~= _ or _ or _ +end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.tables.test.cpp b/tests/TypeInfer.tables.test.cpp index 10186e3a..c94ed1f9 100644 --- a/tests/TypeInfer.tables.test.cpp +++ b/tests/TypeInfer.tables.test.cpp @@ -18,6 +18,7 @@ LUAU_FASTFLAG(LuauLowerBoundsCalculation); LUAU_FASTFLAG(DebugLuauDeferredConstraintResolution); LUAU_FASTFLAG(LuauInstantiateInSubtyping) LUAU_FASTFLAG(LuauNoMoreGlobalSingletonTypes) +LUAU_FASTFLAG(LuauTypeMismatchInvarianceInError) TEST_SUITE_BEGIN("TableTests"); @@ -2024,7 +2025,12 @@ local b: B = a )"); LUAU_REQUIRE_ERRORS(result); - CHECK_EQ(toString(result.errors[0]), R"(Type 'A' could not be converted into 'B' + if (FFlag::LuauTypeMismatchInvarianceInError) + CHECK_EQ(toString(result.errors[0]), R"(Type 'A' could not be converted into 'B' +caused by: + Property 'y' is not compatible. Type 'number' could not be converted into 'string' in an invariant context)"); + else + CHECK_EQ(toString(result.errors[0]), R"(Type 'A' could not be converted into 'B' caused by: Property 'y' is not compatible. Type 'number' could not be converted into 'string')"); } @@ -2043,7 +2049,14 @@ local b: B = a )"); LUAU_REQUIRE_ERRORS(result); - CHECK_EQ(toString(result.errors[0]), R"(Type 'A' could not be converted into 'B' + if (FFlag::LuauTypeMismatchInvarianceInError) + CHECK_EQ(toString(result.errors[0]), R"(Type 'A' could not be converted into 'B' +caused by: + Property 'b' is not compatible. Type 'AS' could not be converted into 'BS' +caused by: + Property 'y' is not compatible. Type 'number' could not be converted into 'string' in an invariant context)"); + else + CHECK_EQ(toString(result.errors[0]), R"(Type 'A' could not be converted into 'B' caused by: Property 'b' is not compatible. Type 'AS' could not be converted into 'BS' caused by: @@ -2063,7 +2076,14 @@ local c2: typeof(a2) = b2 )"); LUAU_REQUIRE_ERROR_COUNT(2, result); - CHECK_EQ(toString(result.errors[0]), R"(Type 'b1' could not be converted into 'a1' + if (FFlag::LuauTypeMismatchInvarianceInError) + CHECK_EQ(toString(result.errors[0]), R"(Type 'b1' could not be converted into 'a1' +caused by: + Type '{ x: number, y: string }' could not be converted into '{ x: number, y: number }' +caused by: + Property 'y' is not compatible. Type 'string' could not be converted into 'number' in an invariant context)"); + else + CHECK_EQ(toString(result.errors[0]), R"(Type 'b1' could not be converted into 'a1' caused by: Type '{ x: number, y: string }' could not be converted into '{ x: number, y: number }' caused by: @@ -2098,7 +2118,12 @@ TEST_CASE_FIXTURE(Fixture, "error_detailed_indexer_key") )"); LUAU_REQUIRE_ERRORS(result); - CHECK_EQ(toString(result.errors[0]), R"(Type 'A' could not be converted into 'B' + if (FFlag::LuauTypeMismatchInvarianceInError) + CHECK_EQ(toString(result.errors[0]), R"(Type 'A' could not be converted into 'B' +caused by: + Property '[indexer key]' is not compatible. Type 'number' could not be converted into 'string' in an invariant context)"); + else + CHECK_EQ(toString(result.errors[0]), R"(Type 'A' could not be converted into 'B' caused by: Property '[indexer key]' is not compatible. Type 'number' could not be converted into 'string')"); } @@ -2114,7 +2139,12 @@ TEST_CASE_FIXTURE(Fixture, "error_detailed_indexer_value") )"); LUAU_REQUIRE_ERRORS(result); - CHECK_EQ(toString(result.errors[0]), R"(Type 'A' could not be converted into 'B' + if (FFlag::LuauTypeMismatchInvarianceInError) + CHECK_EQ(toString(result.errors[0]), R"(Type 'A' could not be converted into 'B' +caused by: + Property '[indexer value]' is not compatible. Type 'number' could not be converted into 'string' in an invariant context)"); + else + CHECK_EQ(toString(result.errors[0]), R"(Type 'A' could not be converted into 'B' caused by: Property '[indexer value]' is not compatible. Type 'number' could not be converted into 'string')"); } @@ -3261,7 +3291,7 @@ caused by: TEST_CASE_FIXTURE(Fixture, "a_free_shape_can_turn_into_a_scalar_if_it_is_compatible") { ScopedFastFlag sff{"LuauScalarShapeSubtyping", true}; - ScopedFastFlag luauScalarShapeUnifyToMtOwner{"LuauScalarShapeUnifyToMtOwner", true}; // Changes argument from table type to primitive + ScopedFastFlag luauScalarShapeUnifyToMtOwner{"LuauScalarShapeUnifyToMtOwner2", true}; // Changes argument from table type to primitive CheckResult result = check(R"( local function f(s): string @@ -3308,7 +3338,7 @@ caused by: TEST_CASE_FIXTURE(BuiltinsFixture, "a_free_shape_can_turn_into_a_scalar_directly") { ScopedFastFlag luauScalarShapeSubtyping{"LuauScalarShapeSubtyping", true}; - ScopedFastFlag luauScalarShapeUnifyToMtOwner{"LuauScalarShapeUnifyToMtOwner", true}; + ScopedFastFlag luauScalarShapeUnifyToMtOwner{"LuauScalarShapeUnifyToMtOwner2", true}; CheckResult result = check(R"( local function stringByteList(str) @@ -3394,7 +3424,7 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "setmetatable_has_a_side_effect") )"); LUAU_REQUIRE_NO_ERRORS(result); - CHECK(toString(requireType("foo")) == "{ @metatable { __add: (a, b) -> number }, { } }"); + CHECK(toString(requireType("foo")) == "{ @metatable { __add: (a, b) -> number }, { } }"); } TEST_CASE_FIXTURE(BuiltinsFixture, "tables_should_be_fully_populated") @@ -3413,4 +3443,43 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "tables_should_be_fully_populated") CHECK_EQ("{ x: *error-type*, y: number }", toString(requireType("t"), opts)); } +TEST_CASE_FIXTURE(Fixture, "fuzz_table_indexer_unification_can_bound_owner_to_string") +{ + ScopedFastFlag luauScalarShapeUnifyToMtOwner{"LuauScalarShapeUnifyToMtOwner2", true}; + + CheckResult result = check(R"( +sin,_ = nil +_ = _[_.sin][_._][_][_]._ +_[_] = _ + )"); + + LUAU_REQUIRE_ERRORS(result); +} + +TEST_CASE_FIXTURE(BuiltinsFixture, "fuzz_table_extra_prop_unification_can_bound_owner_to_string") +{ + ScopedFastFlag luauScalarShapeUnifyToMtOwner{"LuauScalarShapeUnifyToMtOwner2", true}; + + CheckResult result = check(R"( +l0,_ = nil +_ = _,_[_.n5]._[_][_][_]._ +_._.foreach[_],_ = _[_],_._ + )"); + + LUAU_REQUIRE_ERRORS(result); +} + +TEST_CASE_FIXTURE(BuiltinsFixture, "fuzz_typelevel_promote_on_changed_table_type") +{ + ScopedFastFlag luauScalarShapeUnifyToMtOwner{"LuauScalarShapeUnifyToMtOwner2", true}; + + CheckResult result = check(R"( +_._,_ = nil +_ = _.foreach[_]._,_[_.n5]._[_.foreach][_][_]._ +_ = _._ + )"); + + LUAU_REQUIRE_ERRORS(result); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.test.cpp b/tests/TypeInfer.test.cpp index 04b8bf57..e42cea63 100644 --- a/tests/TypeInfer.test.cpp +++ b/tests/TypeInfer.test.cpp @@ -351,7 +351,7 @@ TEST_CASE_FIXTURE(Fixture, "check_expr_recursion_limit") CheckResult result = check(R"(("foo"))" + rep(":lower()", limit)); LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK(nullptr != get(result.errors[0])); + CHECK_MESSAGE(nullptr != get(result.errors[0]), "Expected CodeTooComplex but got " << toString(result.errors[0])); } TEST_CASE_FIXTURE(Fixture, "globals") @@ -1159,4 +1159,17 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "it_is_ok_to_have_inconsistent_number_of_retu LUAU_REQUIRE_NO_ERRORS(result); } +TEST_CASE_FIXTURE(Fixture, "fuzz_free_table_type_change_during_index_check") +{ + ScopedFastFlag luauFollowInLvalueIndexCheck{"LuauFollowInLvalueIndexCheck", true}; + + CheckResult result = check(R"( +local _ = nil +while _["" >= _] do +end + )"); + + LUAU_REQUIRE_ERRORS(result); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.tryUnify.test.cpp b/tests/TypeInfer.tryUnify.test.cpp index f04a3d95..5cc07a28 100644 --- a/tests/TypeInfer.tryUnify.test.cpp +++ b/tests/TypeInfer.tryUnify.test.cpp @@ -110,6 +110,68 @@ TEST_CASE_FIXTURE(TryUnifyFixture, "incompatible_tables_are_preserved") CHECK_NE(*getMutable(&tableOne)->props["foo"].type, *getMutable(&tableTwo)->props["foo"].type); } +TEST_CASE_FIXTURE(TryUnifyFixture, "uninhabited_intersection_sub_never") +{ + ScopedFastFlag sffs[]{ + {"LuauSubtypeNormalizer", true}, + {"LuauTypeNormalization2", true}, + }; + + CheckResult result = check(R"( + function f(arg : string & number) : never + return arg + end + )"); + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(TryUnifyFixture, "uninhabited_intersection_sub_anything") +{ + ScopedFastFlag sffs[]{ + {"LuauSubtypeNormalizer", true}, + {"LuauTypeNormalization2", true}, + }; + + CheckResult result = check(R"( + function f(arg : string & number) : boolean + return arg + end + )"); + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(TryUnifyFixture, "uninhabited_table_sub_never") +{ + ScopedFastFlag sffs[]{ + {"LuauSubtypeNormalizer", true}, + {"LuauTypeNormalization2", true}, + {"LuauUninhabitedSubAnything", true}, + }; + + CheckResult result = check(R"( + function f(arg : { prop : string & number }) : never + return arg + end + )"); + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(TryUnifyFixture, "uninhabited_table_sub_anything") +{ + ScopedFastFlag sffs[]{ + {"LuauSubtypeNormalizer", true}, + {"LuauTypeNormalization2", true}, + {"LuauUninhabitedSubAnything", true}, + }; + + CheckResult result = check(R"( + function f(arg : { prop : string & number }) : boolean + return arg + end + )"); + LUAU_REQUIRE_NO_ERRORS(result); +} + TEST_CASE_FIXTURE(TryUnifyFixture, "members_of_failed_typepack_unification_are_unified_with_errorType") { CheckResult result = check(R"( @@ -299,4 +361,19 @@ TEST_CASE_FIXTURE(TryUnifyFixture, "metatables_unify_against_shape_of_free_table CHECK_EQ(toString(state.errors[0]), expected); } +TEST_CASE_FIXTURE(TryUnifyFixture, "fuzz_tail_unification_issue") +{ + ScopedFastFlag luauTxnLogTypePackIterator{"LuauTxnLogTypePackIterator", true}; + + TypePackVar variadicAny{VariadicTypePack{typeChecker.anyType}}; + TypePackVar packTmp{TypePack{{typeChecker.anyType}, &variadicAny}}; + TypePackVar packSub{TypePack{{typeChecker.anyType, typeChecker.anyType}, &packTmp}}; + + TypeVar freeTy{FreeTypeVar{TypeLevel{}}}; + TypePackVar freeTp{FreeTypePack{TypeLevel{}}}; + TypePackVar packSuper{TypePack{{&freeTy}, &freeTp}}; + + state.tryUnify(&packSub, &packSuper); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.typePacks.cpp b/tests/TypeInfer.typePacks.cpp index cde651df..0e4074f7 100644 --- a/tests/TypeInfer.typePacks.cpp +++ b/tests/TypeInfer.typePacks.cpp @@ -7,8 +7,6 @@ #include "doctest.h" -LUAU_FASTFLAG(LuauFunctionReturnStringificationFixup); - using namespace Luau; TEST_SUITE_BEGIN("TypePackTests"); @@ -311,10 +309,7 @@ local c: Packed auto ttvA = get(requireType("a")); REQUIRE(ttvA); CHECK_EQ(toString(requireType("a")), "Packed"); - if (FFlag::LuauFunctionReturnStringificationFixup) - CHECK_EQ(toString(requireType("a"), {true}), "{| f: (number) -> number |}"); - else - CHECK_EQ(toString(requireType("a"), {true}), "{| f: (number) -> (number) |}"); + CHECK_EQ(toString(requireType("a"), {true}), "{| f: (number) -> number |}"); REQUIRE(ttvA->instantiatedTypeParams.size() == 1); REQUIRE(ttvA->instantiatedTypePackParams.size() == 1); CHECK_EQ(toString(ttvA->instantiatedTypeParams[0], {true}), "number"); @@ -467,8 +462,6 @@ type I = W TEST_CASE_FIXTURE(Fixture, "type_alias_type_pack_explicit") { - ScopedFastFlag sff("LuauFunctionReturnStringificationFixup", true); - CheckResult result = check(R"( type X = (T...) -> (T...) @@ -492,8 +485,6 @@ type F = X<(string, ...number)> TEST_CASE_FIXTURE(Fixture, "type_alias_type_pack_explicit_multi") { - ScopedFastFlag sff("LuauFunctionReturnStringificationFixup", true); - CheckResult result = check(R"( type Y = (T...) -> (U...) @@ -1002,6 +993,10 @@ TEST_CASE_FIXTURE(Fixture, "unify_variadic_tails_in_arguments_free") TEST_CASE_FIXTURE(BuiltinsFixture, "type_packs_with_tails_in_vararg_adjustment") { + std::optional sff; + if (FFlag::DebugLuauDeferredConstraintResolution) + sff = {"LuauInstantiateInSubtyping", true}; + CheckResult result = check(R"( local function wrapReject(fn: (self: any, ...TArg) -> ...TResult): (self: any, ...TArg) -> ...TResult return function(self, ...) @@ -1017,4 +1012,46 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "type_packs_with_tails_in_vararg_adjustment") LUAU_REQUIRE_NO_ERRORS(result); } +TEST_CASE_FIXTURE(BuiltinsFixture, "generalize_expectedTypes_with_proper_scope") +{ + ScopedFastFlag sff[] = { + {"DebugLuauDeferredConstraintResolution", true}, + {"LuauInstantiateInSubtyping", true}, + }; + + CheckResult result = check(R"( + local function f(fn: () -> ...TResult): () -> ...TResult + return function() + end + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "fuzz_typepack_iter_follow") +{ + ScopedFastFlag luauTxnLogTypePackIterator{"LuauTxnLogTypePackIterator", true}; + + CheckResult result = check(R"( +local _ +local _ = _,_(),_(_) + )"); + + LUAU_REQUIRE_ERRORS(result); +} + +TEST_CASE_FIXTURE(BuiltinsFixture, "fuzz_typepack_iter_follow_2") +{ + ScopedFastFlag luauTxnLogTypePackIterator{"LuauTxnLogTypePackIterator", true}; + + CheckResult result = check(R"( +function test(name, searchTerm) + local found = string.find(name:lower(), searchTerm:lower()) +end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.unionTypes.test.cpp b/tests/TypeInfer.unionTypes.test.cpp index 627fbb56..adfc61b6 100644 --- a/tests/TypeInfer.unionTypes.test.cpp +++ b/tests/TypeInfer.unionTypes.test.cpp @@ -729,4 +729,37 @@ TEST_CASE_FIXTURE(Fixture, "union_of_functions_with_mismatching_result_variadics "of the union options are compatible"); } +TEST_CASE_FIXTURE(Fixture, "less_greedy_unification_with_union_types") +{ + if (!FFlag::DebugLuauDeferredConstraintResolution) + return; + + CheckResult result = check(R"( + local function f(t): { x: number } | { x: string } + local x = t.x + return t + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ("({| x: number |} | {| x: string |}) -> {| x: number |} | {| x: string |}", toString(requireType("f"))); +} + +TEST_CASE_FIXTURE(Fixture, "less_greedy_unification_with_union_types_2") +{ + if (!FFlag::DebugLuauDeferredConstraintResolution) + return; + + CheckResult result = check(R"( + local function f(t: { x: number } | { x: string }) + return t.x + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ("({| x: number |} | {| x: string |}) -> number | string", toString(requireType("f"))); +} + TEST_SUITE_END(); diff --git a/tests/VisitTypeVar.test.cpp b/tests/VisitTypeVar.test.cpp index 589c3bad..4fba694a 100644 --- a/tests/VisitTypeVar.test.cpp +++ b/tests/VisitTypeVar.test.cpp @@ -22,14 +22,7 @@ TEST_CASE_FIXTURE(Fixture, "throw_when_limit_is_exceeded") TypeId tType = requireType("t"); - if (FFlag::LuauIceExceptionInheritanceChange) - { - CHECK_THROWS_AS(toString(tType), RecursionLimitException); - } - else - { - CHECK_THROWS_AS(toString(tType), RecursionLimitException_DEPRECATED); - } + CHECK_THROWS_AS(toString(tType), RecursionLimitException); } TEST_CASE_FIXTURE(Fixture, "dont_throw_when_limit_is_high_enough") diff --git a/tools/faillist.txt b/tools/faillist.txt index 433d0cfb..6f49db84 100644 --- a/tools/faillist.txt +++ b/tools/faillist.txt @@ -14,9 +14,11 @@ AstQuery::getDocumentationSymbolAtPosition.overloaded_class_method AstQuery::getDocumentationSymbolAtPosition.overloaded_fn AstQuery::getDocumentationSymbolAtPosition.table_overloaded_function_prop AutocompleteTest.autocomplete_first_function_arg_expected_type -AutocompleteTest.autocomplete_interpolated_string +AutocompleteTest.autocomplete_interpolated_string_as_singleton +AutocompleteTest.autocomplete_interpolated_string_constant +AutocompleteTest.autocomplete_interpolated_string_expression +AutocompleteTest.autocomplete_interpolated_string_expression_with_comments AutocompleteTest.autocomplete_oop_implicit_self -AutocompleteTest.autocomplete_string_singleton_equality AutocompleteTest.autocomplete_string_singleton_escape AutocompleteTest.autocomplete_string_singletons AutocompleteTest.autocompleteProp_index_function_metamethod_is_variadic @@ -25,9 +27,11 @@ AutocompleteTest.do_wrong_compatible_self_calls AutocompleteTest.keyword_methods AutocompleteTest.no_incompatible_self_calls AutocompleteTest.no_wrong_compatible_self_calls_with_generics +AutocompleteTest.suggest_external_module_type AutocompleteTest.suggest_table_keys AutocompleteTest.type_correct_argument_type_suggestion AutocompleteTest.type_correct_expected_argument_type_pack_suggestion +AutocompleteTest.type_correct_expected_argument_type_suggestion AutocompleteTest.type_correct_expected_argument_type_suggestion_optional AutocompleteTest.type_correct_expected_argument_type_suggestion_self AutocompleteTest.type_correct_expected_return_type_pack_suggestion @@ -68,7 +72,6 @@ BuiltinTests.select_with_decimal_argument_is_rounded_down BuiltinTests.set_metatable_needs_arguments BuiltinTests.setmetatable_should_not_mutate_persisted_types BuiltinTests.sort_with_bad_predicate -BuiltinTests.string_format_arg_count_mismatch BuiltinTests.string_format_as_method BuiltinTests.string_format_correctly_ordered_types BuiltinTests.string_format_report_all_type_errors_at_correct_positions @@ -106,10 +109,10 @@ GenericsTests.generic_factories GenericsTests.generic_functions_should_be_memory_safe GenericsTests.generic_table_method GenericsTests.generic_type_pack_parentheses -GenericsTests.generic_type_pack_unification2 GenericsTests.higher_rank_polymorphism_should_not_accept_instantiated_arguments GenericsTests.infer_generic_function_function_argument GenericsTests.infer_generic_function_function_argument_overloaded +GenericsTests.infer_generic_lib_function_function_argument GenericsTests.infer_generic_methods GenericsTests.infer_generic_property GenericsTests.instantiated_function_argument_names @@ -117,9 +120,6 @@ GenericsTests.instantiation_sharing_types GenericsTests.no_stack_overflow_from_quantifying GenericsTests.reject_clashing_generic_and_pack_names GenericsTests.self_recursive_instantiated_param -IntersectionTypes.index_on_an_intersection_type_with_mixed_types -IntersectionTypes.index_on_an_intersection_type_with_property_guaranteed_to_exist -IntersectionTypes.index_on_an_intersection_type_works_at_arbitrary_depth IntersectionTypes.no_stack_overflow_from_flattenintersection IntersectionTypes.select_correct_union_fn IntersectionTypes.should_still_pick_an_overload_whose_arguments_are_unions @@ -151,6 +151,7 @@ ProvisionalTests.bail_early_if_unification_is_too_complicated ProvisionalTests.discriminate_from_x_not_equal_to_nil ProvisionalTests.do_not_ice_when_trying_to_pick_first_of_generic_type_pack ProvisionalTests.error_on_eq_metamethod_returning_a_type_other_than_boolean +ProvisionalTests.free_options_cannot_be_unified_together ProvisionalTests.generic_type_leak_to_module_interface_variadic ProvisionalTests.greedy_inference_with_shared_self_triggers_function_with_no_returns ProvisionalTests.lvalue_equals_another_lvalue_with_no_overlap @@ -164,15 +165,15 @@ ProvisionalTests.while_body_are_also_refined RefinementTest.apply_refinements_on_astexprindexexpr_whose_subscript_expr_is_constant_string RefinementTest.assert_a_to_be_truthy_then_assert_a_to_be_number RefinementTest.assert_non_binary_expressions_actually_resolve_constraints +RefinementTest.assign_table_with_refined_property_with_a_similar_type_is_illegal RefinementTest.call_an_incompatible_function_after_using_typeguard RefinementTest.correctly_lookup_property_whose_base_was_previously_refined RefinementTest.correctly_lookup_property_whose_base_was_previously_refined2 -RefinementTest.discriminate_from_isa_of_x -RefinementTest.discriminate_from_truthiness_of_x RefinementTest.discriminate_on_properties_of_disjoint_tables_where_that_property_is_true_or_false RefinementTest.discriminate_tag RefinementTest.else_with_no_explicit_expression_should_also_refine_the_tagged_union RefinementTest.falsiness_of_TruthyPredicate_narrows_into_nil +RefinementTest.fuzz_filtered_refined_types_are_followed RefinementTest.index_on_a_refined_property RefinementTest.invert_is_truthy_constraint_ifelse_expression RefinementTest.is_truthy_constraint_ifelse_expression @@ -181,7 +182,6 @@ RefinementTest.nonoptional_type_can_narrow_to_nil_if_sense_is_true RefinementTest.not_t_or_some_prop_of_t RefinementTest.refine_a_property_not_to_be_nil_through_an_intersection_table RefinementTest.refine_unknowns -RefinementTest.truthy_constraint_on_properties RefinementTest.type_comparison_ifelse_expression RefinementTest.type_guard_can_filter_for_intersection_of_tables RefinementTest.type_guard_narrowed_into_nothingness @@ -210,7 +210,6 @@ TableTests.defining_a_self_method_for_a_builtin_sealed_table_must_fail TableTests.defining_a_self_method_for_a_local_sealed_table_must_fail TableTests.dont_crash_when_setmetatable_does_not_produce_a_metatabletypevar TableTests.dont_hang_when_trying_to_look_up_in_cyclic_metatable_index -TableTests.dont_leak_free_table_props TableTests.dont_quantify_table_that_belongs_to_outer_scope TableTests.dont_suggest_exact_match_keys TableTests.error_detailed_metatable_prop @@ -240,6 +239,7 @@ TableTests.invariant_table_properties_means_instantiating_tables_in_assignment_i TableTests.invariant_table_properties_means_instantiating_tables_in_call_is_unsound TableTests.leaking_bad_metatable_errors TableTests.less_exponential_blowup_please +TableTests.meta_add TableTests.meta_add_both_ways TableTests.meta_add_inferred TableTests.metatable_mismatch_should_fail @@ -252,8 +252,6 @@ TableTests.only_ascribe_synthetic_names_at_module_scope TableTests.oop_indexer_works TableTests.oop_polymorphic TableTests.open_table_unification_2 -TableTests.pass_a_union_of_tables_to_a_function_that_requires_a_table -TableTests.pass_a_union_of_tables_to_a_function_that_requires_a_table_2 TableTests.persistent_sealed_table_is_immutable TableTests.prop_access_on_key_whose_types_mismatches TableTests.property_lookup_through_tabletypevar_metatable @@ -268,6 +266,7 @@ TableTests.scalar_is_not_a_subtype_of_a_compatible_polymorphic_shape_type TableTests.shared_selfs TableTests.shared_selfs_from_free_param TableTests.shared_selfs_through_metatables +TableTests.table_call_metamethod_basic TableTests.table_indexing_error_location TableTests.table_insert_should_cope_with_optional_properties_in_nonstrict TableTests.table_insert_should_cope_with_optional_properties_in_strict @@ -323,6 +322,7 @@ TypeInfer.checking_should_not_ice TypeInfer.cli_50041_committing_txnlog_in_apollo_client_error TypeInfer.dont_report_type_errors_within_an_AstExprError TypeInfer.dont_report_type_errors_within_an_AstStatError +TypeInfer.fuzz_free_table_type_change_during_index_check TypeInfer.globals TypeInfer.globals2 TypeInfer.infer_assignment_value_types_mutable_lval @@ -335,7 +335,6 @@ TypeInfer.tc_interpolated_string_with_invalid_expression TypeInfer.type_infer_recursion_limit_no_ice TypeInfer.type_infer_recursion_limit_normalizer TypeInferAnyError.for_in_loop_iterator_is_any2 -TypeInferAnyError.for_in_loop_iterator_is_error2 TypeInferClasses.can_read_prop_of_base_class_using_string TypeInferClasses.class_type_mismatch_with_name_conflict TypeInferClasses.classes_without_overloaded_operators_cannot_be_added @@ -351,8 +350,6 @@ TypeInferFunctions.cannot_hoist_interior_defns_into_signature TypeInferFunctions.dont_give_other_overloads_message_if_only_one_argument_matching_overload_exists TypeInferFunctions.dont_infer_parameter_types_for_functions_from_their_call_site TypeInferFunctions.duplicate_functions_with_different_signatures_not_allowed_in_nonstrict -TypeInferFunctions.free_is_not_bound_to_unknown -TypeInferFunctions.func_expr_doesnt_leak_free TypeInferFunctions.function_cast_error_uses_correct_language TypeInferFunctions.function_decl_non_self_sealed_overwrite_2 TypeInferFunctions.function_decl_non_self_unsealed_overwrite @@ -385,12 +382,11 @@ TypeInferLoops.for_in_loop TypeInferLoops.for_in_loop_error_on_factory_not_returning_the_right_amount_of_values TypeInferLoops.for_in_loop_with_next TypeInferLoops.for_in_with_generic_next -TypeInferLoops.for_in_with_just_one_iterator_is_ok TypeInferLoops.loop_iter_metamethod_ok_with_inference TypeInferLoops.loop_iter_no_indexer_nonstrict TypeInferLoops.loop_iter_trailing_nil +TypeInferLoops.properly_infer_iteratee_is_a_free_table TypeInferLoops.unreachable_code_after_infinite_loop -TypeInferLoops.varlist_declared_by_for_in_loop_should_be_free TypeInferModules.custom_require_global TypeInferModules.do_not_modify_imported_types TypeInferModules.module_type_conflict @@ -414,6 +410,8 @@ TypeInferOperators.compound_assign_mismatch_result TypeInferOperators.disallow_string_and_types_without_metatables_from_arithmetic_binary_ops TypeInferOperators.in_nonstrict_mode_strip_nil_from_intersections_when_considering_relational_operators TypeInferOperators.infer_any_in_all_modes_when_lhs_is_unknown +TypeInferOperators.mm_comparisons_must_return_a_boolean +TypeInferOperators.mm_ops_must_return_a_value TypeInferOperators.produce_the_correct_error_message_when_comparing_a_table_with_a_metatable_with_one_that_does_not TypeInferOperators.refine_and_or TypeInferOperators.typecheck_overloaded_multiply_that_is_an_intersection @@ -427,16 +425,13 @@ TypeInferUnknownNever.assign_to_prop_which_is_never TypeInferUnknownNever.assign_to_subscript_which_is_never TypeInferUnknownNever.call_never TypeInferUnknownNever.dont_unify_operands_if_one_of_the_operand_is_never_in_any_ordering_operators -TypeInferUnknownNever.index_on_union_of_tables_for_properties_that_is_never TypeInferUnknownNever.index_on_union_of_tables_for_properties_that_is_sorta_never TypeInferUnknownNever.math_operators_and_never TypeInferUnknownNever.type_packs_containing_never_is_itself_uninhabitable TypeInferUnknownNever.type_packs_containing_never_is_itself_uninhabitable2 TypeInferUnknownNever.unary_minus_of_never TypePackTests.detect_cyclic_typepacks2 -TypePackTests.higher_order_function TypePackTests.pack_tail_unification_check -TypePackTests.parenthesized_varargs_returns_any TypePackTests.type_alias_backwards_compatible TypePackTests.type_alias_default_export TypePackTests.type_alias_default_mixed_self @@ -456,7 +451,6 @@ TypePackTests.type_alias_type_packs_nested TypePackTests.type_pack_type_parameters TypePackTests.unify_variadic_tails_in_arguments TypePackTests.unify_variadic_tails_in_arguments_free -TypePackTests.varargs_inference_through_multiple_scopes TypePackTests.variadic_packs TypeSingletons.error_detailed_tagged_union_mismatch_bool TypeSingletons.error_detailed_tagged_union_mismatch_string @@ -477,11 +471,8 @@ TypeSingletons.widening_happens_almost_everywhere_except_for_tables UnionTypes.error_detailed_optional UnionTypes.error_detailed_union_all UnionTypes.index_on_a_union_type_with_missing_property -UnionTypes.index_on_a_union_type_with_mixed_types UnionTypes.index_on_a_union_type_with_one_optional_property UnionTypes.index_on_a_union_type_with_one_property_of_type_any -UnionTypes.index_on_a_union_type_with_property_guaranteed_to_exist -UnionTypes.index_on_a_union_type_works_at_arbitrary_depth UnionTypes.optional_assignment_errors UnionTypes.optional_call_error UnionTypes.optional_field_access_error From 4e6ae326250ec73a6d01fb7a5d1ab5ae5934139c Mon Sep 17 00:00:00 2001 From: vegorov-rbx <75688451+vegorov-rbx@users.noreply.github.com> Date: Fri, 2 Dec 2022 10:23:20 -0800 Subject: [PATCH 397/399] Luau Recap: November 2022 (#764) Co-authored-by: Andy Friesen --- .../2022-11-30-luau-recap-november-2022.md | 96 +++++++++++++++++++ 1 file changed, 96 insertions(+) create mode 100644 docs/_posts/2022-11-30-luau-recap-november-2022.md diff --git a/docs/_posts/2022-11-30-luau-recap-november-2022.md b/docs/_posts/2022-11-30-luau-recap-november-2022.md new file mode 100644 index 00000000..29b4c6dc --- /dev/null +++ b/docs/_posts/2022-11-30-luau-recap-november-2022.md @@ -0,0 +1,96 @@ +--- +layout: single +title: "Luau Recap: November 2022" +--- + +While the team is busy to bring some bigger things in the future, we have made some small improvements this month. + +[Cross-posted to the [Roblox Developer Forum](https://devforum.roblox.com/t/luau-recap-november-2022/).] + +## Analysis improvements + +We have improved tagged union type refinements to only include unhandled type cases in the `else` branch of the `if` statement: + +```lua +type Ok = { tag: "ok", value: T } +type Err = { tag: "error", msg: string } +type Result = Ok | Err + +function unwrap(r: Result): T? + if r.tag == "ok" then + return r.value + else + -- Luau now understands that 'r' here can only be the 'Err' part + print(r.msg) + return nil + end +end +``` + +For better inference, we updated the definition of `Enum.SomeType:GetEnumItems()` to return `{Enum.SomeType}` instead of common `{EnumItem}` and the return type of `next` function now includes the possibility of key being `nil`. + +Finally, if you use `and` operator on non-boolean values, `boolean` type will no longer be added by the type inference: + +```lua +local function f1(a: number?) + -- 'x' is still a 'number?' and doesn't become 'boolean | number' + local x = a and 5 +end +``` + +## Error message improvements + +We now give an error when built-in types are being redefined: + +```lua +type string = number -- Now an error: Redefinition of type 'string' +``` + +We also had a parse error missing in case you forgot your default type pack parameter value. We accepted the following code silently without raising an issue: + +```lua +type Foo = nil -- Now an error: Expected type, got '>' +``` + +Error about function argument count mismatch no longer points at the last argument, but instead at the function in question. +So, instead of: + +```lua +function myfunction(a: number, b:number) end +myfunction(123) + ~~~ +``` + +We now highlight this: + +```lua +function myfunction(a: number, b:number) end +myfunction(123) +~~~~~~~~~~ +``` + +If you iterate over a table value that could also be `nil`, you get a better explanation in the error message: + +```lua +local function f(t: {number}?) + for i,v in t do -- Value of type {number}? could be nil + --... + end +end +``` +Previously it was `Cannot call non-function {number}?` which was confusing. + +And speaking of confusing, some of you might have seen an error like `Type 'string' could not be converted into 'string'`. + +This was caused by Luau having both a primitive type `string` and a table type coming from `string` library. Since the way you can get the type of the `string` library table is by using `typeof(string)`, the updated error message will mirror that and report `Type 'string' could not be converted into 'typeof(string)'`. + + +Parsing now recovers with a more precise error message if you forget a comma in table constructor spanning multiple lines: + +```lua +local t = { + a = 1 + b = 2 -- Expected ',' after table constructor element + c = 3 -- Expected ',' after table constructor element +} +``` From e9d4ee7a33c1d586a2d037244166a0e9fedc1dbe Mon Sep 17 00:00:00 2001 From: Alex Orlenko Date: Tue, 6 Dec 2022 19:20:24 +0000 Subject: [PATCH 398/399] Make pseudo-indices relative to LUAI_MAXCSTACK (#766) This is useful in particular if redefine `LUAI_MAXCSTACK` to a higher value than the current one (8000). Ie. passing `-D LUAI_MAXCSTACK=1000000` would not work as overlaps with `LUA_REGISTRYINDEX` and below. --- VM/include/lua.h | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/VM/include/lua.h b/VM/include/lua.h index 73ebbb0c..783e60a5 100644 --- a/VM/include/lua.h +++ b/VM/include/lua.h @@ -17,9 +17,9 @@ /* ** pseudo-indices */ -#define LUA_REGISTRYINDEX (-10000) -#define LUA_ENVIRONINDEX (-10001) -#define LUA_GLOBALSINDEX (-10002) +#define LUA_REGISTRYINDEX (-LUAI_MAXCSTACK - 2000) +#define LUA_ENVIRONINDEX (-LUAI_MAXCSTACK - 2001) +#define LUA_GLOBALSINDEX (-LUAI_MAXCSTACK - 2002) #define lua_upvalueindex(i) (LUA_GLOBALSINDEX - (i)) #define lua_ispseudo(i) ((i) <= LUA_REGISTRYINDEX) From fb2f146123dcb423de6bc789d4f7d01f4981170c Mon Sep 17 00:00:00 2001 From: Andy Friesen Date: Fri, 9 Dec 2022 11:57:01 -0800 Subject: [PATCH 399/399] Sync to upstream/release/556 (#782) * The AST JSON encoder will now stringify infinity and NaN according to the JSON5 spec using the tokens `Infinity`, `-Infinity`, and `NaN`. * Improve autocompletion of table keys if the type of that key is a union of string singletons. --- Analysis/include/Luau/BuiltinDefinitions.h | 9 +- Analysis/include/Luau/Constraint.h | 7 + .../include/Luau/ConstraintGraphBuilder.h | 12 + Analysis/include/Luau/ConstraintSolver.h | 9 +- Analysis/include/Luau/Error.h | 6 +- Analysis/include/Luau/Linter.h | 1 + Analysis/include/Luau/Normalize.h | 22 +- Analysis/include/Luau/Scope.h | 1 - Analysis/include/Luau/ToString.h | 26 +- Analysis/include/Luau/TxnLog.h | 6 +- Analysis/include/Luau/TypeUtils.h | 4 +- Analysis/include/Luau/TypeVar.h | 7 +- Analysis/src/AstJsonEncoder.cpp | 27 +- Analysis/src/AstQuery.cpp | 22 +- Analysis/src/Autocomplete.cpp | 47 +++- Analysis/src/BuiltinDefinitions.cpp | 13 + Analysis/src/ConstraintGraphBuilder.cpp | 44 ++-- Analysis/src/ConstraintSolver.cpp | 44 ++-- Analysis/src/Error.cpp | 1 + Analysis/src/Frontend.cpp | 29 +-- Analysis/src/Normalize.cpp | 195 ++++++--------- Analysis/src/ToString.cpp | 13 + Analysis/src/TxnLog.cpp | 1 + Analysis/src/TypeChecker2.cpp | 110 ++++++++- Analysis/src/TypeInfer.cpp | 4 +- Analysis/src/TypeUtils.cpp | 97 -------- Analysis/src/TypeVar.cpp | 7 - Analysis/src/Unifier.cpp | 11 +- Ast/include/Luau/Ast.h | 1 + Ast/include/Luau/Location.h | 136 ++-------- Ast/src/Location.cpp | 122 ++++++++- CLI/Ast.cpp | 1 + CodeGen/src/CodeGen.cpp | 100 ++++---- CodeGen/src/EmitCommonX64.cpp | 34 +-- CodeGen/src/EmitCommonX64.h | 22 +- CodeGen/src/EmitInstructionX64.cpp | 232 +++++++++--------- CodeGen/src/EmitInstructionX64.h | 64 ++--- tests/AstJsonEncoder.test.cpp | 7 + tests/Autocomplete.test.cpp | 65 +++++ tests/Fixture.cpp | 48 +++- tests/Fixture.h | 49 +--- tests/Frontend.test.cpp | 4 - tests/Normalize.test.cpp | 1 - tests/ToString.test.cpp | 12 +- tests/TypeInfer.aliases.test.cpp | 14 +- tests/TypeInfer.intersectionTypes.test.cpp | 2 +- tests/TypeInfer.negations.test.cpp | 3 +- tests/TypeInfer.operators.test.cpp | 86 +++++-- tests/TypeInfer.provisional.test.cpp | 30 +++ tests/TypeInfer.refinements.test.cpp | 82 ++++--- tests/TypeInfer.tables.test.cpp | 69 ++---- tests/TypeInfer.tryUnify.test.cpp | 4 +- tools/faillist.txt | 58 +++-- 53 files changed, 1100 insertions(+), 921 deletions(-) diff --git a/Analysis/include/Luau/BuiltinDefinitions.h b/Analysis/include/Luau/BuiltinDefinitions.h index 4702995d..16cccafe 100644 --- a/Analysis/include/Luau/BuiltinDefinitions.h +++ b/Analysis/include/Luau/BuiltinDefinitions.h @@ -1,13 +1,18 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #pragma once -#include "Luau/Frontend.h" #include "Luau/Scope.h" -#include "Luau/TypeInfer.h" +#include "Luau/TypeVar.h" + +#include namespace Luau { +struct Frontend; +struct TypeChecker; +struct TypeArena; + void registerBuiltinTypes(Frontend& frontend); void registerBuiltinGlobals(TypeChecker& typeChecker); diff --git a/Analysis/include/Luau/Constraint.h b/Analysis/include/Luau/Constraint.h index e13613ed..9eea9c28 100644 --- a/Analysis/include/Luau/Constraint.h +++ b/Analysis/include/Luau/Constraint.h @@ -3,6 +3,7 @@ #include "Luau/Ast.h" // Used for some of the enumerations #include "Luau/Def.h" +#include "Luau/DenseHash.h" #include "Luau/NotNull.h" #include "Luau/TypeVar.h" #include "Luau/Variant.h" @@ -67,6 +68,12 @@ struct BinaryConstraint TypeId leftType; TypeId rightType; TypeId resultType; + + // When we dispatch this constraint, we update the key at this map to record + // the overload that we selected. + AstExpr* expr; + DenseHashMap* astOriginalCallTypes; + DenseHashMap* astOverloadResolvedTypes; }; // iteratee is iterable diff --git a/Analysis/include/Luau/ConstraintGraphBuilder.h b/Analysis/include/Luau/ConstraintGraphBuilder.h index d81fe918..c25a5537 100644 --- a/Analysis/include/Luau/ConstraintGraphBuilder.h +++ b/Analysis/include/Luau/ConstraintGraphBuilder.h @@ -76,15 +76,27 @@ struct ConstraintGraphBuilder // A mapping of AST node to TypeId. DenseHashMap astTypes{nullptr}; + // A mapping of AST node to TypePackId. DenseHashMap astTypePacks{nullptr}; + + // If the node was applied as a function, this is the unspecialized type of + // that expression. DenseHashMap astOriginalCallTypes{nullptr}; + + // If overload resolution was performed on this element, this is the + // overload that was selected. + DenseHashMap astOverloadResolvedTypes{nullptr}; + // Types resolved from type annotations. Analogous to astTypes. DenseHashMap astResolvedTypes{nullptr}; + // Type packs resolved from type annotations. Analogous to astTypePacks. DenseHashMap astResolvedTypePacks{nullptr}; + // Defining scopes for AST nodes. DenseHashMap astTypeAliasDefiningScopes{nullptr}; + NotNull dfg; ConnectiveArena connectiveArena; diff --git a/Analysis/include/Luau/ConstraintSolver.h b/Analysis/include/Luau/ConstraintSolver.h index e05f6f1f..c02cd4d5 100644 --- a/Analysis/include/Luau/ConstraintSolver.h +++ b/Analysis/include/Luau/ConstraintSolver.h @@ -2,12 +2,13 @@ #pragma once -#include "Luau/Error.h" -#include "Luau/Variant.h" #include "Luau/Constraint.h" -#include "Luau/TypeVar.h" -#include "Luau/ToString.h" +#include "Luau/Error.h" +#include "Luau/Module.h" #include "Luau/Normalize.h" +#include "Luau/ToString.h" +#include "Luau/TypeVar.h" +#include "Luau/Variant.h" #include diff --git a/Analysis/include/Luau/Error.h b/Analysis/include/Luau/Error.h index 89388046..739354f8 100644 --- a/Analysis/include/Luau/Error.h +++ b/Analysis/include/Luau/Error.h @@ -1,16 +1,16 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #pragma once -#include "Luau/FileResolver.h" #include "Luau/Location.h" #include "Luau/TypeVar.h" #include "Luau/Variant.h" -#include "Luau/TypeArena.h" namespace Luau { -struct TypeError; +struct FileResolver; +struct TypeArena; +struct TypeError; struct TypeMismatch { diff --git a/Analysis/include/Luau/Linter.h b/Analysis/include/Luau/Linter.h index 0e3d9880..6bbc3d66 100644 --- a/Analysis/include/Luau/Linter.h +++ b/Analysis/include/Luau/Linter.h @@ -4,6 +4,7 @@ #include "Luau/Location.h" #include +#include #include namespace Luau diff --git a/Analysis/include/Luau/Normalize.h b/Analysis/include/Luau/Normalize.h index d7e104ee..39257315 100644 --- a/Analysis/include/Luau/Normalize.h +++ b/Analysis/include/Luau/Normalize.h @@ -42,6 +42,7 @@ public: void retain(const TypeIds& tys); void clear(); + TypeId front() const; iterator begin(); iterator end(); const_iterator begin() const; @@ -107,18 +108,7 @@ namespace Luau /** A normalized string type is either `string` (represented by `nullopt`) or a * union of string singletons. * - * When FFlagLuauNegatedStringSingletons is unset, the representation is as - * follows: - * - * * The `string` data type is represented by the option `singletons` having the - * value `std::nullopt`. - * * The type `never` is represented by `singletons` being populated with an - * empty map. - * * A union of string singletons is represented by a map populated by the names - * and TypeIds of the singletons contained therein. - * - * When FFlagLuauNegatedStringSingletons is set, the representation is as - * follows: + * The representation is as follows: * * * A union of string singletons is finite and includes the singletons named by * the `singletons` field. @@ -138,9 +128,7 @@ struct NormalizedStringType // eg string & ~"a" & ~"b" & ... bool isCofinite = false; - // TODO: This field cannot be nullopt when FFlagLuauNegatedStringSingletons - // is set. When clipping that flag, we can remove the wrapping optional. - std::optional> singletons; + std::map singletons; void resetToString(); void resetToNever(); @@ -161,8 +149,8 @@ struct NormalizedStringType static const NormalizedStringType never; - NormalizedStringType() = default; - NormalizedStringType(bool isCofinite, std::optional> singletons); + NormalizedStringType(); + NormalizedStringType(bool isCofinite, std::map singletons); }; bool isSubtype(const NormalizedStringType& subStr, const NormalizedStringType& superStr); diff --git a/Analysis/include/Luau/Scope.h b/Analysis/include/Luau/Scope.h index a26f506d..851ed1a7 100644 --- a/Analysis/include/Luau/Scope.h +++ b/Analysis/include/Luau/Scope.h @@ -1,7 +1,6 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #pragma once -#include "Luau/Constraint.h" #include "Luau/Location.h" #include "Luau/NotNull.h" #include "Luau/TypeVar.h" diff --git a/Analysis/include/Luau/ToString.h b/Analysis/include/Luau/ToString.h index 186cc9a5..71c0e359 100644 --- a/Analysis/include/Luau/ToString.h +++ b/Analysis/include/Luau/ToString.h @@ -2,13 +2,12 @@ #pragma once #include "Luau/Common.h" -#include "Luau/TypeVar.h" -#include "Luau/ConstraintGraphBuilder.h" -#include -#include #include +#include #include +#include +#include LUAU_FASTINT(LuauTableTypeMaximumStringifierLength) LUAU_FASTINT(LuauTypeMaximumStringifierLength) @@ -16,6 +15,22 @@ LUAU_FASTINT(LuauTypeMaximumStringifierLength) namespace Luau { +class AstExpr; + +struct Scope; + +struct TypeVar; +using TypeId = const TypeVar*; + +struct TypePackVar; +using TypePackId = const TypePackVar*; + +struct FunctionTypeVar; +struct Constraint; + +struct Position; +struct Location; + struct ToStringNameMap { std::unordered_map typeVars; @@ -125,4 +140,7 @@ std::string dump(const std::shared_ptr& scope, const char* name); std::string generateName(size_t n); +std::string toString(const Position& position); +std::string toString(const Location& location); + } // namespace Luau diff --git a/Analysis/include/Luau/TxnLog.h b/Analysis/include/Luau/TxnLog.h index b1a83412..82605bff 100644 --- a/Analysis/include/Luau/TxnLog.h +++ b/Analysis/include/Luau/TxnLog.h @@ -1,12 +1,12 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #pragma once -#include -#include - #include "Luau/TypeVar.h" #include "Luau/TypePack.h" +#include +#include + namespace Luau { diff --git a/Analysis/include/Luau/TypeUtils.h b/Analysis/include/Luau/TypeUtils.h index aa9cdde2..6ed70f46 100644 --- a/Analysis/include/Luau/TypeUtils.h +++ b/Analysis/include/Luau/TypeUtils.h @@ -4,6 +4,7 @@ #include "Luau/Error.h" #include "Luau/Location.h" #include "Luau/TypeVar.h" +#include "Luau/TypePack.h" #include #include @@ -12,6 +13,7 @@ namespace Luau { struct TxnLog; +struct TypeArena; using ScopePtr = std::shared_ptr; @@ -19,8 +21,6 @@ std::optional findMetatableEntry( NotNull singletonTypes, ErrorVec& errors, TypeId type, const std::string& entry, Location location); std::optional findTablePropertyRespectingMeta( NotNull singletonTypes, ErrorVec& errors, TypeId ty, const std::string& name, Location location); -std::optional getIndexTypeFromType(const ScopePtr& scope, ErrorVec& errors, TypeArena* arena, NotNull singletonTypes, - TypeId type, const std::string& prop, const Location& location, bool addErrors, InternalErrorReporter& handle); // Returns the minimum and maximum number of types the argument list can accept. std::pair> getParameterExtents(const TxnLog* log, TypePackId tp, bool includeHiddenVariadics = false); diff --git a/Analysis/include/Luau/TypeVar.h b/Analysis/include/Luau/TypeVar.h index d355746a..852a4054 100644 --- a/Analysis/include/Luau/TypeVar.h +++ b/Analysis/include/Luau/TypeVar.h @@ -264,12 +264,14 @@ using DcrMagicFunction = bool (*)(MagicFunctionCallContext); struct MagicRefinementContext { ScopePtr scope; + NotNull cgb; NotNull dfg; NotNull connectiveArena; + std::vector argumentConnectives; const class AstExprCall* callSite; }; -using DcrMagicRefinement = std::vector (*)(MagicRefinementContext); +using DcrMagicRefinement = std::vector (*)(const MagicRefinementContext&); struct FunctionTypeVar { @@ -666,9 +668,6 @@ public: const TypePackId errorTypePack; }; -// Clip with FFlagLuauNoMoreGlobalSingletonTypes -SingletonTypes& DEPRECATED_getSingletonTypes(); - void persist(TypeId ty); void persist(TypePackId tp); diff --git a/Analysis/src/AstJsonEncoder.cpp b/Analysis/src/AstJsonEncoder.cpp index 8d589037..57c8c90b 100644 --- a/Analysis/src/AstJsonEncoder.cpp +++ b/Analysis/src/AstJsonEncoder.cpp @@ -6,6 +6,8 @@ #include "Luau/StringUtils.h" #include "Luau/Common.h" +#include + namespace Luau { @@ -103,9 +105,28 @@ struct AstJsonEncoder : public AstVisitor void write(double d) { - char b[32]; - snprintf(b, sizeof(b), "%.17g", d); - writeRaw(b); + switch (fpclassify(d)) + { + case FP_INFINITE: + if (d < 0) + writeRaw("-Infinity"); + else + writeRaw("Infinity"); + break; + + case FP_NAN: + writeRaw("NaN"); + break; + + case FP_NORMAL: + case FP_SUBNORMAL: + case FP_ZERO: + default: + char b[32]; + snprintf(b, sizeof(b), "%.17g", d); + writeRaw(b); + break; + } } void writeString(std::string_view sv) diff --git a/Analysis/src/AstQuery.cpp b/Analysis/src/AstQuery.cpp index 85d2320a..e6e7f3d9 100644 --- a/Analysis/src/AstQuery.cpp +++ b/Analysis/src/AstQuery.cpp @@ -11,6 +11,8 @@ #include +LUAU_FASTFLAG(LuauCompleteTableKeysBetter); + namespace Luau { @@ -29,12 +31,24 @@ struct AutocompleteNodeFinder : public AstVisitor bool visit(AstExpr* expr) override { - if (expr->location.begin < pos && pos <= expr->location.end) + if (FFlag::LuauCompleteTableKeysBetter) { - ancestry.push_back(expr); - return true; + if (expr->location.begin <= pos && pos <= expr->location.end) + { + ancestry.push_back(expr); + return true; + } + return false; + } + else + { + if (expr->location.begin < pos && pos <= expr->location.end) + { + ancestry.push_back(expr); + return true; + } + return false; } - return false; } bool visit(AstStat* stat) override diff --git a/Analysis/src/Autocomplete.cpp b/Analysis/src/Autocomplete.cpp index 5374c6b1..83a6f021 100644 --- a/Analysis/src/Autocomplete.cpp +++ b/Analysis/src/Autocomplete.cpp @@ -12,6 +12,8 @@ #include #include +LUAU_FASTFLAGVARIABLE(LuauCompleteTableKeysBetter, false); + static const std::unordered_set kStatementStartingKeywords = { "while", "if", "local", "repeat", "function", "do", "for", "return", "break", "continue", "type", "export"}; @@ -966,13 +968,28 @@ T* extractStat(const std::vector& ancestry) if (!parent) return nullptr; - if (T* t = parent->as(); t && parent->is()) - return t; - AstNode* grandParent = ancestry.size() >= 3 ? ancestry.rbegin()[2] : nullptr; AstNode* greatGrandParent = ancestry.size() >= 4 ? ancestry.rbegin()[3] : nullptr; - if (!grandParent || !greatGrandParent) - return nullptr; + + if (FFlag::LuauCompleteTableKeysBetter) + { + if (!grandParent) + return nullptr; + + if (T* t = parent->as(); t && grandParent->is()) + return t; + + if (!greatGrandParent) + return nullptr; + } + else + { + if (T* t = parent->as(); t && parent->is()) + return t; + + if (!grandParent || !greatGrandParent) + return nullptr; + } if (T* t = greatGrandParent->as(); t && grandParent->is() && parent->is() && isIdentifier(node)) return t; @@ -1469,6 +1486,26 @@ static AutocompleteResult autocomplete(const SourceModule& sourceModule, const M { auto result = autocompleteProps(*module, &typeArena, singletonTypes, *it, PropIndexType::Key, ancestry); + if (FFlag::LuauCompleteTableKeysBetter) + { + if (auto nodeIt = module->astExpectedTypes.find(node->asExpr())) + autocompleteStringSingleton(*nodeIt, !node->is(), result); + + if (!key) + { + // If there is "no key," it may be that the user + // intends for the current token to be the key, but + // has yet to type the `=` sign. + // + // If the key type is a union of singleton strings, + // suggest those too. + if (auto ttv = get(follow(*it)); ttv && ttv->indexer) + { + autocompleteStringSingleton(ttv->indexer->indexType, false, result); + } + } + } + // Remove keys that are already completed for (const auto& item : exprTable->items) { diff --git a/Analysis/src/BuiltinDefinitions.cpp b/Analysis/src/BuiltinDefinitions.cpp index 39568674..612812c5 100644 --- a/Analysis/src/BuiltinDefinitions.cpp +++ b/Analysis/src/BuiltinDefinitions.cpp @@ -7,6 +7,7 @@ #include "Luau/Common.h" #include "Luau/ToString.h" #include "Luau/ConstraintSolver.h" +#include "Luau/ConstraintGraphBuilder.h" #include "Luau/TypeInfer.h" #include "Luau/TypePack.h" #include "Luau/TypeVar.h" @@ -46,6 +47,8 @@ static bool dcrMagicFunctionSelect(MagicFunctionCallContext context); static bool dcrMagicFunctionRequire(MagicFunctionCallContext context); static bool dcrMagicFunctionPack(MagicFunctionCallContext context); +static std::vector dcrMagicRefinementAssert(const MagicRefinementContext& context); + TypeId makeUnion(TypeArena& arena, std::vector&& types) { return arena.addType(UnionTypeVar{std::move(types)}); @@ -478,6 +481,7 @@ void registerBuiltinGlobals(Frontend& frontend) } attachMagicFunction(getGlobalBinding(frontend, "assert"), magicFunctionAssert); + attachDcrMagicRefinement(getGlobalBinding(frontend, "assert"), dcrMagicRefinementAssert); attachMagicFunction(getGlobalBinding(frontend, "setmetatable"), magicFunctionSetMetaTable); attachMagicFunction(getGlobalBinding(frontend, "select"), magicFunctionSelect); attachDcrMagicFunction(getGlobalBinding(frontend, "select"), dcrMagicFunctionSelect); @@ -703,6 +707,15 @@ static std::optional> magicFunctionAssert( return WithPredicate{arena.addTypePack(TypePack{std::move(head), tail})}; } +static std::vector dcrMagicRefinementAssert(const MagicRefinementContext& ctx) +{ + if (ctx.argumentConnectives.empty()) + return {}; + + ctx.cgb->applyRefinements(ctx.scope, ctx.callSite->location, ctx.argumentConnectives[0]); + return {}; +} + static std::optional> magicFunctionPack( TypeChecker& typechecker, const ScopePtr& scope, const AstExprCall& expr, WithPredicate withPredicate) { diff --git a/Analysis/src/ConstraintGraphBuilder.cpp b/Analysis/src/ConstraintGraphBuilder.cpp index d41c7772..f0bd958c 100644 --- a/Analysis/src/ConstraintGraphBuilder.cpp +++ b/Analysis/src/ConstraintGraphBuilder.cpp @@ -2,16 +2,12 @@ #include "Luau/ConstraintGraphBuilder.h" #include "Luau/Ast.h" -#include "Luau/Clone.h" #include "Luau/Common.h" #include "Luau/Constraint.h" #include "Luau/DcrLogger.h" #include "Luau/ModuleResolver.h" #include "Luau/RecursionCounter.h" #include "Luau/Scope.h" -#include "Luau/Substitution.h" -#include "Luau/ToString.h" -#include "Luau/TxnLog.h" #include "Luau/TypeUtils.h" #include "Luau/TypeVar.h" @@ -1068,16 +1064,9 @@ InferencePack ConstraintGraphBuilder::checkPack(const ScopePtr& scope, AstExprCa else expectedArgs = extendTypePack(*arena, singletonTypes, expectedArgPack, exprArgs.size() - 1); - std::vector connectives; - if (auto ftv = get(follow(fnType)); ftv && ftv->dcrMagicRefinement) - { - MagicRefinementContext ctx{globalScope, dfg, NotNull{&connectiveArena}, call}; - connectives = ftv->dcrMagicRefinement(ctx); - } - - std::vector args; std::optional argTail; + std::vector argumentConnectives; Checkpoint argCheckpoint = checkpoint(this); @@ -1101,7 +1090,11 @@ InferencePack ConstraintGraphBuilder::checkPack(const ScopePtr& scope, AstExprCa args.push_back(arena->freshType(scope.get())); } else if (i < exprArgs.size() - 1 || !(arg->is() || arg->is())) - args.push_back(check(scope, arg, expectedType).ty); + { + auto [ty, connective] = check(scope, arg, expectedType); + args.push_back(ty); + argumentConnectives.push_back(connective); + } else argTail = checkPack(scope, arg, {}).tp; // FIXME? not sure about expectedTypes here } @@ -1114,6 +1107,13 @@ InferencePack ConstraintGraphBuilder::checkPack(const ScopePtr& scope, AstExprCa constraint->dependencies.push_back(extractArgsConstraint); }); + std::vector returnConnectives; + if (auto ftv = get(follow(fnType)); ftv && ftv->dcrMagicRefinement) + { + MagicRefinementContext ctx{scope, NotNull{this}, dfg, NotNull{&connectiveArena}, std::move(argumentConnectives), call}; + returnConnectives = ftv->dcrMagicRefinement(ctx); + } + if (matchSetmetatable(*call)) { TypePack argTailPack; @@ -1133,7 +1133,7 @@ InferencePack ConstraintGraphBuilder::checkPack(const ScopePtr& scope, AstExprCa if (AstExprLocal* targetLocal = targetExpr->as()) scope->bindings[targetLocal->local].typeId = resultTy; - return InferencePack{arena->addTypePack({resultTy}), std::move(connectives)}; + return InferencePack{arena->addTypePack({resultTy}), std::move(returnConnectives)}; } else { @@ -1172,7 +1172,7 @@ InferencePack ConstraintGraphBuilder::checkPack(const ScopePtr& scope, AstExprCa fcc->dependencies.emplace_back(constraint.get()); }); - return InferencePack{rets, std::move(connectives)}; + return InferencePack{rets, std::move(returnConnectives)}; } } @@ -1468,16 +1468,22 @@ Inference ConstraintGraphBuilder::check(const ScopePtr& scope, AstExprBinary* bi auto [leftType, rightType, connective] = checkBinary(scope, binary, expectedType); TypeId resultType = arena->addType(BlockedTypeVar{}); - addConstraint(scope, binary->location, BinaryConstraint{binary->op, leftType, rightType, resultType}); + addConstraint(scope, binary->location, BinaryConstraint{binary->op, leftType, rightType, resultType, binary, &astOriginalCallTypes, &astOverloadResolvedTypes}); return Inference{resultType, std::move(connective)}; } Inference ConstraintGraphBuilder::check(const ScopePtr& scope, AstExprIfElse* ifElse, std::optional expectedType) { - check(scope, ifElse->condition); + ScopePtr condScope = childScope(ifElse->condition, scope); + auto [_, connective] = check(scope, ifElse->condition); - TypeId thenType = check(scope, ifElse->trueExpr, expectedType).ty; - TypeId elseType = check(scope, ifElse->falseExpr, expectedType).ty; + ScopePtr thenScope = childScope(ifElse->trueExpr, scope); + applyRefinements(thenScope, ifElse->trueExpr->location, connective); + TypeId thenType = check(thenScope, ifElse->trueExpr, expectedType).ty; + + ScopePtr elseScope = childScope(ifElse->falseExpr, scope); + applyRefinements(elseScope, ifElse->falseExpr->location, connectiveArena.negation(connective)); + TypeId elseType = check(elseScope, ifElse->falseExpr, expectedType).ty; if (ifElse->hasElse) { diff --git a/Analysis/src/ConstraintSolver.cpp b/Analysis/src/ConstraintSolver.cpp index d59ea70a..d73c14a6 100644 --- a/Analysis/src/ConstraintSolver.cpp +++ b/Analysis/src/ConstraintSolver.cpp @@ -15,7 +15,6 @@ #include "Luau/TypeVar.h" #include "Luau/Unifier.h" #include "Luau/VisitTypeVar.h" -#include "Luau/TypeUtils.h" LUAU_FASTFLAGVARIABLE(DebugLuauLogSolver, false); LUAU_FASTFLAGVARIABLE(DebugLuauLogSolverToJson, false); @@ -635,9 +634,17 @@ bool ConstraintSolver::tryDispatch(const BinaryConstraint& c, NotNullscope}; + std::optional instantiatedMm = instantiation.substitute(*mm); + if (!instantiatedMm) + { + reportError(CodeTooComplex{}, constraint->location); + return true; + } + // TODO: Is a table with __call legal here? // TODO: Overloads - if (const FunctionTypeVar* ftv = get(follow(*mm))) + if (const FunctionTypeVar* ftv = get(follow(*instantiatedMm))) { TypePackId inferredArgs; // For >= and > we invoke __lt and __le respectively with @@ -673,6 +680,9 @@ bool ConstraintSolver::tryDispatch(const BinaryConstraint& c, NotNullty.emplace(mmResult); unblock(resultType); + + (*c.astOriginalCallTypes)[c.expr] = *mm; + (*c.astOverloadResolvedTypes)[c.expr] = *instantiatedMm; return true; } } @@ -743,19 +753,7 @@ bool ConstraintSolver::tryDispatch(const BinaryConstraint& c, NotNulladdType(IntersectionTypeVar{{singletonTypes->falsyType, leftType}}); - // TODO: normaliztion here should be replaced by a more limited 'simplification' - const NormalizedType* normalized = normalizer->normalize(arena->addType(UnionTypeVar{{leftFilteredTy, rightType}})); - - if (!normalized) - { - reportError(CodeTooComplex{}, constraint->location); - asMutable(resultType)->ty.emplace(errorRecoveryType()); - } - else - { - asMutable(resultType)->ty.emplace(normalizer->typeFromNormal(*normalized)); - } - + asMutable(resultType)->ty.emplace(arena->addType(UnionTypeVar{{leftFilteredTy, rightType}})); unblock(resultType); return true; } @@ -763,21 +761,9 @@ bool ConstraintSolver::tryDispatch(const BinaryConstraint& c, NotNulladdType(IntersectionTypeVar{{singletonTypes->truthyType, leftType}}); - - // TODO: normaliztion here should be replaced by a more limited 'simplification' - const NormalizedType* normalized = normalizer->normalize(arena->addType(UnionTypeVar{{rightFilteredTy, rightType}})); - - if (!normalized) - { - reportError(CodeTooComplex{}, constraint->location); - asMutable(resultType)->ty.emplace(errorRecoveryType()); - } - else - { - asMutable(resultType)->ty.emplace(normalizer->typeFromNormal(*normalized)); - } + TypeId leftFilteredTy = arena->addType(IntersectionTypeVar{{singletonTypes->truthyType, leftType}}); + asMutable(resultType)->ty.emplace(arena->addType(UnionTypeVar{{leftFilteredTy, rightType}})); unblock(resultType); return true; } diff --git a/Analysis/src/Error.cpp b/Analysis/src/Error.cpp index aefaa2c7..748cf20f 100644 --- a/Analysis/src/Error.cpp +++ b/Analysis/src/Error.cpp @@ -3,6 +3,7 @@ #include "Luau/Clone.h" #include "Luau/Common.h" +#include "Luau/FileResolver.h" #include "Luau/StringUtils.h" #include "Luau/ToString.h" diff --git a/Analysis/src/Frontend.cpp b/Analysis/src/Frontend.cpp index 356ced0b..e21e42e1 100644 --- a/Analysis/src/Frontend.cpp +++ b/Analysis/src/Frontend.cpp @@ -21,16 +21,15 @@ #include #include #include +#include LUAU_FASTINT(LuauTypeInferIterationLimit) LUAU_FASTINT(LuauTarjanChildLimit) LUAU_FASTFLAG(LuauInferInNoCheckMode) -LUAU_FASTFLAG(LuauNoMoreGlobalSingletonTypes) LUAU_FASTFLAGVARIABLE(LuauKnowsTheDataModel3, false) LUAU_FASTINTVARIABLE(LuauAutocompleteCheckTimeoutMs, 100) LUAU_FASTFLAGVARIABLE(DebugLuauDeferredConstraintResolution, false) LUAU_FASTFLAG(DebugLuauLogSolverToJson); -LUAU_FASTFLAGVARIABLE(LuauFixMarkDirtyReverseDeps, false) namespace Luau { @@ -409,7 +408,7 @@ double getTimestamp() } // namespace Frontend::Frontend(FileResolver* fileResolver, ConfigResolver* configResolver, const FrontendOptions& options) - : singletonTypes(NotNull{FFlag::LuauNoMoreGlobalSingletonTypes ? &singletonTypes_ : &DEPRECATED_getSingletonTypes()}) + : singletonTypes(NotNull{&singletonTypes_}) , fileResolver(fileResolver) , moduleResolver(this) , moduleResolverForAutocomplete(this) @@ -819,26 +818,13 @@ void Frontend::markDirty(const ModuleName& name, std::vector* marked sourceNode.dirtyModule = true; sourceNode.dirtyModuleForAutocomplete = true; - if (FFlag::LuauFixMarkDirtyReverseDeps) - { - if (0 == reverseDeps.count(next)) - continue; + if (0 == reverseDeps.count(next)) + continue; - sourceModules.erase(next); + sourceModules.erase(next); - const std::vector& dependents = reverseDeps[next]; - queue.insert(queue.end(), dependents.begin(), dependents.end()); - } - else - { - if (0 == reverseDeps.count(name)) - continue; - - sourceModules.erase(name); - - const std::vector& dependents = reverseDeps[name]; - queue.insert(queue.end(), dependents.begin(), dependents.end()); - } + const std::vector& dependents = reverseDeps[next]; + queue.insert(queue.end(), dependents.begin(), dependents.end()); } } @@ -919,6 +905,7 @@ ModulePtr Frontend::check( result->astTypes = std::move(cgb.astTypes); result->astTypePacks = std::move(cgb.astTypePacks); result->astOriginalCallTypes = std::move(cgb.astOriginalCallTypes); + result->astOverloadResolvedTypes = std::move(cgb.astOverloadResolvedTypes); result->astResolvedTypes = std::move(cgb.astResolvedTypes); result->astResolvedTypePacks = std::move(cgb.astResolvedTypePacks); result->type = sourceModule.type; diff --git a/Analysis/src/Normalize.cpp b/Analysis/src/Normalize.cpp index fa3503fd..09f0595d 100644 --- a/Analysis/src/Normalize.cpp +++ b/Analysis/src/Normalize.cpp @@ -19,12 +19,11 @@ LUAU_FASTINTVARIABLE(LuauNormalizeIterationLimit, 1200); LUAU_FASTINTVARIABLE(LuauNormalizeCacheLimit, 100000); LUAU_FASTFLAGVARIABLE(LuauNormalizeCombineTableFix, false); LUAU_FASTFLAGVARIABLE(LuauTypeNormalization2, false); -LUAU_FASTFLAGVARIABLE(LuauNegatedStringSingletons, false); LUAU_FASTFLAGVARIABLE(LuauNegatedFunctionTypes, false); LUAU_FASTFLAG(LuauUnknownAndNeverType) LUAU_FASTFLAG(DebugLuauDeferredConstraintResolution) LUAU_FASTFLAG(LuauOverloadedFunctionSubtypingPerf); -LUAU_FASTFLAG(LuauUninhabitedSubAnything) +LUAU_FASTFLAG(LuauUninhabitedSubAnything2) namespace Luau { @@ -46,6 +45,11 @@ void TypeIds::clear() hash = 0; } +TypeId TypeIds::front() const +{ + return order.at(0); +} + TypeIds::iterator TypeIds::begin() { return order.begin(); @@ -111,94 +115,68 @@ bool TypeIds::operator==(const TypeIds& there) const return hash == there.hash && types == there.types; } -NormalizedStringType::NormalizedStringType(bool isCofinite, std::optional> singletons) +NormalizedStringType::NormalizedStringType() +{} + +NormalizedStringType::NormalizedStringType(bool isCofinite, std::map singletons) : isCofinite(isCofinite) , singletons(std::move(singletons)) { - if (!FFlag::LuauNegatedStringSingletons) - LUAU_ASSERT(!isCofinite); } void NormalizedStringType::resetToString() { - if (FFlag::LuauNegatedStringSingletons) - { - isCofinite = true; - singletons->clear(); - } - else - singletons.reset(); + isCofinite = true; + singletons.clear(); } void NormalizedStringType::resetToNever() { - if (FFlag::LuauNegatedStringSingletons) - { - isCofinite = false; - singletons.emplace(); - } - else - { - if (singletons) - singletons->clear(); - else - singletons.emplace(); - } + isCofinite = false; + singletons.clear(); } bool NormalizedStringType::isNever() const { - if (FFlag::LuauNegatedStringSingletons) - return !isCofinite && singletons->empty(); - else - return singletons && singletons->empty(); + return !isCofinite && singletons.empty(); } bool NormalizedStringType::isString() const { - if (FFlag::LuauNegatedStringSingletons) - return isCofinite && singletons->empty(); - else - return !singletons; + return isCofinite && singletons.empty(); } bool NormalizedStringType::isUnion() const { - if (FFlag::LuauNegatedStringSingletons) - return !isCofinite; - else - return singletons.has_value(); + return !isCofinite; } bool NormalizedStringType::isIntersection() const { - if (FFlag::LuauNegatedStringSingletons) - return isCofinite; - else - return false; + return isCofinite; } bool NormalizedStringType::includes(const std::string& str) const { if (isString()) return true; - else if (isUnion() && singletons->count(str)) + else if (isUnion() && singletons.count(str)) return true; - else if (isIntersection() && !singletons->count(str)) + else if (isIntersection() && !singletons.count(str)) return true; else return false; } -const NormalizedStringType NormalizedStringType::never{false, {{}}}; +const NormalizedStringType NormalizedStringType::never; bool isSubtype(const NormalizedStringType& subStr, const NormalizedStringType& superStr) { if (subStr.isUnion() && superStr.isUnion()) { - for (auto [name, ty] : *subStr.singletons) + for (auto [name, ty] : subStr.singletons) { - if (!superStr.singletons->count(name)) + if (!superStr.singletons.count(name)) return false; } } @@ -251,17 +229,21 @@ static bool isShallowInhabited(const NormalizedType& norm) bool isInhabited_DEPRECATED(const NormalizedType& norm) { - LUAU_ASSERT(!FFlag::LuauUninhabitedSubAnything); + LUAU_ASSERT(!FFlag::LuauUninhabitedSubAnything2); return isShallowInhabited(norm); } bool Normalizer::isInhabited(const NormalizedType* norm, std::unordered_set seen) { + // If normalization failed, the type is complex, and so is more likely than not to be inhabited. + if (!norm) + return true; + if (!get(norm->tops) || !get(norm->booleans) || !get(norm->errors) || !get(norm->nils) || !get(norm->numbers) || !get(norm->threads) || !norm->classes.empty() || !norm->strings.isNever() || !norm->functions.isNever()) return true; - + for (const auto& [_, intersect] : norm->tyvars) { if (isInhabited(intersect.get(), seen)) @@ -372,7 +354,7 @@ static bool isNormalizedString(const NormalizedStringType& ty) if (ty.isString()) return true; - for (auto& [str, ty] : *ty.singletons) + for (auto& [str, ty] : ty.singletons) { if (const SingletonTypeVar* stv = get(ty)) { @@ -682,56 +664,46 @@ void Normalizer::unionClasses(TypeIds& heres, const TypeIds& theres) void Normalizer::unionStrings(NormalizedStringType& here, const NormalizedStringType& there) { - if (FFlag::LuauNegatedStringSingletons) + if (there.isString()) + here.resetToString(); + else if (here.isUnion() && there.isUnion()) + here.singletons.insert(there.singletons.begin(), there.singletons.end()); + else if (here.isUnion() && there.isIntersection()) { - if (there.isString()) - here.resetToString(); - else if (here.isUnion() && there.isUnion()) - here.singletons->insert(there.singletons->begin(), there.singletons->end()); - else if (here.isUnion() && there.isIntersection()) + here.isCofinite = true; + for (const auto& pair : there.singletons) { - here.isCofinite = true; - for (const auto& pair : *there.singletons) - { - auto it = here.singletons->find(pair.first); - if (it != end(*here.singletons)) - here.singletons->erase(it); - else - here.singletons->insert(pair); - } + auto it = here.singletons.find(pair.first); + if (it != end(here.singletons)) + here.singletons.erase(it); + else + here.singletons.insert(pair); } - else if (here.isIntersection() && there.isUnion()) - { - for (const auto& [name, ty] : *there.singletons) - here.singletons->erase(name); - } - else if (here.isIntersection() && there.isIntersection()) - { - auto iter = begin(*here.singletons); - auto endIter = end(*here.singletons); + } + else if (here.isIntersection() && there.isUnion()) + { + for (const auto& [name, ty] : there.singletons) + here.singletons.erase(name); + } + else if (here.isIntersection() && there.isIntersection()) + { + auto iter = begin(here.singletons); + auto endIter = end(here.singletons); - while (iter != endIter) + while (iter != endIter) + { + if (!there.singletons.count(iter->first)) { - if (!there.singletons->count(iter->first)) - { - auto eraseIt = iter; - ++iter; - here.singletons->erase(eraseIt); - } - else - ++iter; + auto eraseIt = iter; + ++iter; + here.singletons.erase(eraseIt); } + else + ++iter; } - else - LUAU_ASSERT(!"Unreachable"); } else - { - if (there.isString()) - here.resetToString(); - else if (here.isUnion()) - here.singletons->insert(there.singletons->begin(), there.singletons->end()); - } + LUAU_ASSERT(!"Unreachable"); } std::optional Normalizer::unionOfTypePacks(TypePackId here, TypePackId there) @@ -1116,22 +1088,14 @@ bool Normalizer::unionNormalWithTy(NormalizedType& here, TypeId there, int ignor here.booleans = unionOfBools(here.booleans, there); else if (const StringSingleton* sstv = get(stv)) { - if (FFlag::LuauNegatedStringSingletons) + if (here.strings.isCofinite) { - if (here.strings.isCofinite) - { - auto it = here.strings.singletons->find(sstv->value); - if (it != here.strings.singletons->end()) - here.strings.singletons->erase(it); - } - else - here.strings.singletons->insert({sstv->value, there}); + auto it = here.strings.singletons.find(sstv->value); + if (it != here.strings.singletons.end()) + here.strings.singletons.erase(it); } else - { - if (here.strings.isUnion()) - here.strings.singletons->insert({sstv->value, there}); - } + here.strings.singletons.insert({sstv->value, there}); } else LUAU_ASSERT(!"Unreachable"); @@ -1278,7 +1242,6 @@ void Normalizer::subtractPrimitive(NormalizedType& here, TypeId ty) here.threads = singletonTypes->neverType; break; case PrimitiveTypeVar::Function: - LUAU_ASSERT(FFlag::LuauNegatedStringSingletons); here.functions.resetToNever(); break; } @@ -1286,20 +1249,18 @@ void Normalizer::subtractPrimitive(NormalizedType& here, TypeId ty) void Normalizer::subtractSingleton(NormalizedType& here, TypeId ty) { - LUAU_ASSERT(FFlag::LuauNegatedStringSingletons); - const SingletonTypeVar* stv = get(ty); LUAU_ASSERT(stv); if (const StringSingleton* ss = get(stv)) { if (here.strings.isCofinite) - here.strings.singletons->insert({ss->value, ty}); + here.strings.singletons.insert({ss->value, ty}); else { - auto it = here.strings.singletons->find(ss->value); - if (it != here.strings.singletons->end()) - here.strings.singletons->erase(it); + auto it = here.strings.singletons.find(ss->value); + if (it != here.strings.singletons.end()) + here.strings.singletons.erase(it); } } else if (const BooleanSingleton* bs = get(stv)) @@ -1417,12 +1378,12 @@ void Normalizer::intersectStrings(NormalizedStringType& here, const NormalizedSt if (here.isString()) here.resetToNever(); - for (auto it = here.singletons->begin(); it != here.singletons->end();) + for (auto it = here.singletons.begin(); it != here.singletons.end();) { - if (there.singletons->count(it->first)) + if (there.singletons.count(it->first)) it++; else - it = here.singletons->erase(it); + it = here.singletons.erase(it); } } @@ -2096,12 +2057,12 @@ bool Normalizer::intersectNormalWithTy(NormalizedType& here, TypeId there) else if (const StringSingleton* sstv = get(stv)) { if (strings.includes(sstv->value)) - here.strings.singletons->insert({sstv->value, there}); + here.strings.singletons.insert({sstv->value, there}); } else LUAU_ASSERT(!"Unreachable"); } - else if (const NegationTypeVar* ntv = get(there); FFlag::LuauNegatedStringSingletons && ntv) + else if (const NegationTypeVar* ntv = get(there)) { TypeId t = follow(ntv->ty); if (const PrimitiveTypeVar* ptv = get(t)) @@ -2171,14 +2132,14 @@ TypeId Normalizer::typeFromNormal(const NormalizedType& norm) result.push_back(singletonTypes->stringType); else if (norm.strings.isUnion()) { - for (auto& [_, ty] : *norm.strings.singletons) + for (auto& [_, ty] : norm.strings.singletons) result.push_back(ty); } - else if (FFlag::LuauNegatedStringSingletons && norm.strings.isIntersection()) + else if (norm.strings.isIntersection()) { std::vector parts; parts.push_back(singletonTypes->stringType); - for (const auto& [name, ty] : *norm.strings.singletons) + for (const auto& [name, ty] : norm.strings.singletons) parts.push_back(arena->addType(NegationTypeVar{ty})); result.push_back(arena->addType(IntersectionTypeVar{std::move(parts)})); diff --git a/Analysis/src/ToString.cpp b/Analysis/src/ToString.cpp index ddcc15a8..01144a3d 100644 --- a/Analysis/src/ToString.cpp +++ b/Analysis/src/ToString.cpp @@ -1,6 +1,8 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #include "Luau/ToString.h" +#include "Luau/Constraint.h" +#include "Luau/Location.h" #include "Luau/Scope.h" #include "Luau/TypeInfer.h" #include "Luau/TypePack.h" @@ -1582,4 +1584,15 @@ std::optional getFunctionNameAsString(const AstExpr& expr) return s; } + +std::string toString(const Position& position) +{ + return "{ line = " + std::to_string(position.line) + ", col = " + std::to_string(position.column) + " }"; +} + +std::string toString(const Location& location) +{ + return "Location { " + toString(location.begin) + ", " + toString(location.end) + " }"; +} + } // namespace Luau diff --git a/Analysis/src/TxnLog.cpp b/Analysis/src/TxnLog.cpp index 1a73b049..18596a63 100644 --- a/Analysis/src/TxnLog.cpp +++ b/Analysis/src/TxnLog.cpp @@ -2,6 +2,7 @@ #include "Luau/TxnLog.h" #include "Luau/ToString.h" +#include "Luau/TypeArena.h" #include "Luau/TypePack.h" #include diff --git a/Analysis/src/TypeChecker2.cpp b/Analysis/src/TypeChecker2.cpp index 84c0ca3b..8c44f90a 100644 --- a/Analysis/src/TypeChecker2.cpp +++ b/Analysis/src/TypeChecker2.cpp @@ -90,6 +90,9 @@ struct TypeChecker2 std::vector> stack; + UnifierSharedState sharedState{&ice}; + Normalizer normalizer{&module->internalTypes, singletonTypes, NotNull{&sharedState}}; + TypeChecker2(NotNull singletonTypes, DcrLogger* logger, const SourceModule* sourceModule, Module* module) : singletonTypes(singletonTypes) , logger(logger) @@ -298,8 +301,6 @@ struct TypeChecker2 TypeArena* arena = &module->internalTypes; TypePackId actualRetType = reconstructPack(ret->list, *arena); - UnifierSharedState sharedState{&ice}; - Normalizer normalizer{arena, singletonTypes, NotNull{&sharedState}}; Unifier u{NotNull{&normalizer}, Mode::Strict, stack.back(), ret->location, Covariant}; u.tryUnify(actualRetType, expectedRetType); @@ -921,7 +922,12 @@ struct TypeChecker2 void visit(AstExprIndexName* indexName) { TypeId leftType = lookupType(indexName->expr); - getIndexTypeFromType(module->getModuleScope(), leftType, indexName->index.value, indexName->location, /* addErrors */ true); + + const NormalizedType* norm = normalizer.normalize(leftType); + if (!norm) + reportError(NormalizationTooComplex{}, indexName->indexLocation); + + checkIndexTypeFromType(leftType, *norm, indexName->index.value, indexName->location); } void visit(AstExprIndexExpr* indexExpr) @@ -1109,11 +1115,18 @@ struct TypeChecker2 if (std::optional leftMm = findMetatableEntry(singletonTypes, module->errors, leftType, it->second, expr->left->location)) mm = leftMm; else if (std::optional rightMm = findMetatableEntry(singletonTypes, module->errors, rightType, it->second, expr->right->location)) + { mm = rightMm; + std::swap(leftType, rightType); + } if (mm) { - if (const FunctionTypeVar* ftv = get(follow(*mm))) + TypeId instantiatedMm = module->astOverloadResolvedTypes[expr]; + if (!instantiatedMm) + reportError(CodeTooComplex{}, expr->location); + + else if (const FunctionTypeVar* ftv = get(follow(instantiatedMm))) { TypePackId expectedArgs; // For >= and > we invoke __lt and __le respectively with @@ -1545,9 +1558,7 @@ struct TypeChecker2 template bool isSubtype(TID subTy, TID superTy, NotNull scope) { - UnifierSharedState sharedState{&ice}; TypeArena arena; - Normalizer normalizer{&arena, singletonTypes, NotNull{&sharedState}}; Unifier u{NotNull{&normalizer}, Mode::Strict, scope, Location{}, Covariant}; u.useScopes = true; @@ -1559,8 +1570,6 @@ struct TypeChecker2 template ErrorVec tryUnify(NotNull scope, const Location& location, TID subTy, TID superTy) { - UnifierSharedState sharedState{&ice}; - Normalizer normalizer{&module->internalTypes, singletonTypes, NotNull{&sharedState}}; Unifier u{NotNull{&normalizer}, Mode::Strict, scope, location, Covariant}; u.useScopes = true; u.tryUnify(subTy, superTy); @@ -1587,9 +1596,90 @@ struct TypeChecker2 reportError(std::move(e)); } - std::optional getIndexTypeFromType(const ScopePtr& scope, TypeId type, const std::string& prop, const Location& location, bool addErrors) + void checkIndexTypeFromType(TypeId denormalizedTy, const NormalizedType& norm, const std::string& prop, const Location& location) { - return Luau::getIndexTypeFromType(scope, module->errors, &module->internalTypes, singletonTypes, type, prop, location, addErrors, ice); + bool foundOneProp = false; + std::vector typesMissingTheProp; + + auto fetch = [&](TypeId ty) { + if (!normalizer.isInhabited(ty)) + return; + + bool found = hasIndexTypeFromType(ty, prop, location); + foundOneProp |= found; + if (!found) + typesMissingTheProp.push_back(ty); + }; + + fetch(norm.tops); + fetch(norm.booleans); + for (TypeId ty : norm.classes) + fetch(ty); + fetch(norm.errors); + fetch(norm.nils); + fetch(norm.numbers); + if (!norm.strings.isNever()) + fetch(singletonTypes->stringType); + fetch(norm.threads); + for (TypeId ty : norm.tables) + fetch(ty); + if (norm.functions.isTop) + fetch(singletonTypes->functionType); + else if (!norm.functions.isNever()) + { + if (norm.functions.parts->size() == 1) + fetch(norm.functions.parts->front()); + else + { + std::vector parts; + parts.insert(parts.end(), norm.functions.parts->begin(), norm.functions.parts->end()); + fetch(module->internalTypes.addType(IntersectionTypeVar{std::move(parts)})); + } + } + for (const auto& [tyvar, intersect] : norm.tyvars) + { + if (get(intersect->tops)) + { + TypeId ty = normalizer.typeFromNormal(*intersect); + fetch(module->internalTypes.addType(IntersectionTypeVar{{tyvar, ty}})); + } + else + fetch(tyvar); + } + + if (!typesMissingTheProp.empty()) + { + if (foundOneProp) + reportError(TypeError{location, MissingUnionProperty{denormalizedTy, typesMissingTheProp, prop}}); + else + reportError(TypeError{location, UnknownProperty{denormalizedTy, prop}}); + } + } + + bool hasIndexTypeFromType(TypeId ty, const std::string& prop, const Location& location) + { + if (get(ty) || get(ty) || get(ty)) + return true; + + if (isString(ty)) + { + std::optional mtIndex = Luau::findMetatableEntry(singletonTypes, module->errors, singletonTypes->stringType, "__index", location); + LUAU_ASSERT(mtIndex); + ty = *mtIndex; + } + + if (getTableType(ty)) + return bool(findTablePropertyRespectingMeta(singletonTypes, module->errors, ty, prop, location)); + else if (const ClassTypeVar* cls = get(ty)) + return bool(lookupClassProp(cls, prop)); + else if (const UnionTypeVar* utv = get(ty)) + ice.ice("getIndexTypeFromTypeHelper cannot take a UnionTypeVar"); + else if (const IntersectionTypeVar* itv = get(ty)) + return std::any_of(begin(itv), end(itv), [&](TypeId part) { + return hasIndexTypeFromType(part, prop, location); + }); + else + return false; } }; diff --git a/Analysis/src/TypeInfer.cpp b/Analysis/src/TypeInfer.cpp index 9f64a601..aa738ad9 100644 --- a/Analysis/src/TypeInfer.cpp +++ b/Analysis/src/TypeInfer.cpp @@ -52,7 +52,7 @@ LUAU_FASTFLAGVARIABLE(LuauIntersectionTestForEquality, false) LUAU_FASTFLAGVARIABLE(LuauImplicitElseRefinement, false) LUAU_FASTFLAGVARIABLE(LuauAllowIndexClassParameters, false) LUAU_FASTFLAGVARIABLE(LuauDeclareClassPrototype, false) -LUAU_FASTFLAG(LuauUninhabitedSubAnything) +LUAU_FASTFLAG(LuauUninhabitedSubAnything2) LUAU_FASTFLAGVARIABLE(LuauCallableClasses, false) namespace Luau @@ -2691,7 +2691,7 @@ static std::optional areEqComparable(NotNull arena, NotNullisInhabited(n); else return isInhabited_DEPRECATED(*n); diff --git a/Analysis/src/TypeUtils.cpp b/Analysis/src/TypeUtils.cpp index 876c45a7..7478ac22 100644 --- a/Analysis/src/TypeUtils.cpp +++ b/Analysis/src/TypeUtils.cpp @@ -88,103 +88,6 @@ std::optional findTablePropertyRespectingMeta( return std::nullopt; } -std::optional getIndexTypeFromType(const ScopePtr& scope, ErrorVec& errors, TypeArena* arena, NotNull singletonTypes, - TypeId type, const std::string& prop, const Location& location, bool addErrors, InternalErrorReporter& handle) -{ - type = follow(type); - - if (get(type) || get(type) || get(type)) - return type; - - if (auto f = get(type)) - *asMutable(type) = TableTypeVar{TableState::Free, f->level}; - - if (isString(type)) - { - std::optional mtIndex = Luau::findMetatableEntry(singletonTypes, errors, singletonTypes->stringType, "__index", location); - LUAU_ASSERT(mtIndex); - type = *mtIndex; - } - - if (getTableType(type)) - { - return findTablePropertyRespectingMeta(singletonTypes, errors, type, prop, location); - } - else if (const ClassTypeVar* cls = get(type)) - { - if (const Property* p = lookupClassProp(cls, prop)) - return p->type; - } - else if (const UnionTypeVar* utv = get(type)) - { - std::vector goodOptions; - std::vector badOptions; - - for (TypeId t : utv) - { - if (get(follow(t))) - return t; - - if (std::optional ty = - getIndexTypeFromType(scope, errors, arena, singletonTypes, t, prop, location, /* addErrors= */ false, handle)) - goodOptions.push_back(*ty); - else - badOptions.push_back(t); - } - - if (!badOptions.empty()) - { - if (addErrors) - { - if (goodOptions.empty()) - errors.push_back(TypeError{location, UnknownProperty{type, prop}}); - else - errors.push_back(TypeError{location, MissingUnionProperty{type, badOptions, prop}}); - } - return std::nullopt; - } - - goodOptions = reduceUnion(goodOptions); - - if (goodOptions.empty()) - return singletonTypes->neverType; - - if (goodOptions.size() == 1) - return goodOptions[0]; - - return arena->addType(UnionTypeVar{std::move(goodOptions)}); - } - else if (const IntersectionTypeVar* itv = get(type)) - { - std::vector parts; - - for (TypeId t : itv->parts) - { - if (std::optional ty = - getIndexTypeFromType(scope, errors, arena, singletonTypes, t, prop, location, /* addErrors= */ false, handle)) - parts.push_back(*ty); - } - - // If no parts of the intersection had the property we looked up for, it never existed at all. - if (parts.empty()) - { - if (addErrors) - errors.push_back(TypeError{location, UnknownProperty{type, prop}}); - return std::nullopt; - } - - if (parts.size() == 1) - return parts[0]; - - return arena->addType(IntersectionTypeVar{std::move(parts)}); - } - - if (addErrors) - errors.push_back(TypeError{location, UnknownProperty{type, prop}}); - - return std::nullopt; -} - std::pair> getParameterExtents(const TxnLog* log, TypePackId tp, bool includeHiddenVariadics) { size_t minCount = 0; diff --git a/Analysis/src/TypeVar.cpp b/Analysis/src/TypeVar.cpp index 6771d89b..159e7712 100644 --- a/Analysis/src/TypeVar.cpp +++ b/Analysis/src/TypeVar.cpp @@ -26,7 +26,6 @@ LUAU_FASTINTVARIABLE(LuauTableTypeMaximumStringifierLength, 0) LUAU_FASTINT(LuauTypeInferRecursionLimit) LUAU_FASTFLAG(LuauUnknownAndNeverType) LUAU_FASTFLAGVARIABLE(LuauMaybeGenericIntersectionTypes, false) -LUAU_FASTFLAGVARIABLE(LuauNoMoreGlobalSingletonTypes, false) LUAU_FASTFLAGVARIABLE(LuauNewLibraryTypeNames, false) LUAU_FASTFLAG(LuauInstantiateInSubtyping) @@ -890,12 +889,6 @@ TypePackId SingletonTypes::errorRecoveryTypePack(TypePackId guess) return guess; } -SingletonTypes& DEPRECATED_getSingletonTypes() -{ - static SingletonTypes singletonTypes; - return singletonTypes; -} - void persist(TypeId ty) { std::deque queue{ty}; diff --git a/Analysis/src/Unifier.cpp b/Analysis/src/Unifier.cpp index 5ff405e6..42882005 100644 --- a/Analysis/src/Unifier.cpp +++ b/Analysis/src/Unifier.cpp @@ -5,12 +5,13 @@ #include "Luau/Instantiation.h" #include "Luau/RecursionCounter.h" #include "Luau/Scope.h" +#include "Luau/StringUtils.h" +#include "Luau/TimeTrace.h" +#include "Luau/ToString.h" #include "Luau/TypePack.h" #include "Luau/TypeUtils.h" -#include "Luau/TimeTrace.h" #include "Luau/TypeVar.h" #include "Luau/VisitTypeVar.h" -#include "Luau/ToString.h" #include @@ -23,7 +24,7 @@ LUAU_FASTFLAGVARIABLE(LuauScalarShapeSubtyping, false) LUAU_FASTFLAGVARIABLE(LuauInstantiateInSubtyping, false) LUAU_FASTFLAGVARIABLE(LuauOverloadedFunctionSubtypingPerf, false); LUAU_FASTFLAGVARIABLE(LuauScalarShapeUnifyToMtOwner2, false) -LUAU_FASTFLAGVARIABLE(LuauUninhabitedSubAnything, false) +LUAU_FASTFLAGVARIABLE(LuauUninhabitedSubAnything2, false) LUAU_FASTFLAG(LuauClassTypeVarsInSubstitution) LUAU_FASTFLAG(LuauTxnLogTypePackIterator) LUAU_FASTFLAG(DebugLuauDeferredConstraintResolution) @@ -588,7 +589,7 @@ void Unifier::tryUnify_(TypeId subTy, TypeId superTy, bool isFunctionCall, bool else if (log.get(subTy)) tryUnifyNegationWithType(subTy, superTy); - else if (FFlag::LuauUninhabitedSubAnything && !normalizer->isInhabited(subTy)) + else if (FFlag::LuauUninhabitedSubAnything2 && !normalizer->isInhabited(subTy)) {} else @@ -1980,7 +1981,7 @@ void Unifier::tryUnifyScalarShape(TypeId subTy, TypeId superTy, bool reversed) TypeId osubTy = subTy; TypeId osuperTy = superTy; - if (FFlag::LuauUninhabitedSubAnything && !normalizer->isInhabited(subTy)) + if (FFlag::LuauUninhabitedSubAnything2 && !normalizer->isInhabited(subTy)) return; if (reversed) diff --git a/Ast/include/Luau/Ast.h b/Ast/include/Luau/Ast.h index 07051163..aa87d9e8 100644 --- a/Ast/include/Luau/Ast.h +++ b/Ast/include/Luau/Ast.h @@ -5,6 +5,7 @@ #include #include +#include #include diff --git a/Ast/include/Luau/Location.h b/Ast/include/Luau/Location.h index e39bbf8c..dbe36bec 100644 --- a/Ast/include/Luau/Location.h +++ b/Ast/include/Luau/Location.h @@ -1,8 +1,6 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #pragma once -#include - namespace Luau { @@ -10,130 +8,36 @@ struct Position { unsigned int line, column; - Position(unsigned int line, unsigned int column) - : line(line) - , column(column) - { - } + Position(unsigned int line, unsigned int column); - bool operator==(const Position& rhs) const - { - return this->column == rhs.column && this->line == rhs.line; - } - bool operator!=(const Position& rhs) const - { - return !(*this == rhs); - } + bool operator==(const Position& rhs) const; + bool operator!=(const Position& rhs) const; + bool operator<(const Position& rhs) const; + bool operator>(const Position& rhs) const; + bool operator<=(const Position& rhs) const; + bool operator>=(const Position& rhs) const; - bool operator<(const Position& rhs) const - { - if (line == rhs.line) - return column < rhs.column; - else - return line < rhs.line; - } - - bool operator>(const Position& rhs) const - { - if (line == rhs.line) - return column > rhs.column; - else - return line > rhs.line; - } - - bool operator<=(const Position& rhs) const - { - return *this == rhs || *this < rhs; - } - - bool operator>=(const Position& rhs) const - { - return *this == rhs || *this > rhs; - } - - void shift(const Position& start, const Position& oldEnd, const Position& newEnd) - { - if (*this >= start) - { - if (this->line > oldEnd.line) - this->line += (newEnd.line - oldEnd.line); - else - { - this->line = newEnd.line; - this->column += (newEnd.column - oldEnd.column); - } - } - } + void shift(const Position& start, const Position& oldEnd, const Position& newEnd); }; struct Location { Position begin, end; - Location() - : begin(0, 0) - , end(0, 0) - { - } + Location(); + Location(const Position& begin, const Position& end); + Location(const Position& begin, unsigned int length); + Location(const Location& begin, const Location& end); - Location(const Position& begin, const Position& end) - : begin(begin) - , end(end) - { - } + bool operator==(const Location& rhs) const; + bool operator!=(const Location& rhs) const; - Location(const Position& begin, unsigned int length) - : begin(begin) - , end(begin.line, begin.column + length) - { - } - - Location(const Location& begin, const Location& end) - : begin(begin.begin) - , end(end.end) - { - } - - bool operator==(const Location& rhs) const - { - return this->begin == rhs.begin && this->end == rhs.end; - } - bool operator!=(const Location& rhs) const - { - return !(*this == rhs); - } - - bool encloses(const Location& l) const - { - return begin <= l.begin && end >= l.end; - } - bool overlaps(const Location& l) const - { - return (begin <= l.begin && end >= l.begin) || (begin <= l.end && end >= l.end) || (begin >= l.begin && end <= l.end); - } - bool contains(const Position& p) const - { - return begin <= p && p < end; - } - bool containsClosed(const Position& p) const - { - return begin <= p && p <= end; - } - void extend(const Location& other) - { - if (other.begin < begin) - begin = other.begin; - if (other.end > end) - end = other.end; - } - void shift(const Position& start, const Position& oldEnd, const Position& newEnd) - { - begin.shift(start, oldEnd, newEnd); - end.shift(start, oldEnd, newEnd); - } + bool encloses(const Location& l) const; + bool overlaps(const Location& l) const; + bool contains(const Position& p) const; + bool containsClosed(const Position& p) const; + void extend(const Location& other); + void shift(const Position& start, const Position& oldEnd, const Position& newEnd); }; -std::string toString(const Position& position); -std::string toString(const Location& location); - } // namespace Luau diff --git a/Ast/src/Location.cpp b/Ast/src/Location.cpp index d7a899ed..67c2dd4b 100644 --- a/Ast/src/Location.cpp +++ b/Ast/src/Location.cpp @@ -4,14 +4,128 @@ namespace Luau { -std::string toString(const Position& position) +Position::Position(unsigned int line, unsigned int column) + : line(line) + , column(column) { - return "{ line = " + std::to_string(position.line) + ", col = " + std::to_string(position.column) + " }"; } -std::string toString(const Location& location) +bool Position::operator==(const Position& rhs) const { - return "Location { " + toString(location.begin) + ", " + toString(location.end) + " }"; + return this->column == rhs.column && this->line == rhs.line; +} + +bool Position::operator!=(const Position& rhs) const +{ + return !(*this == rhs); +} + +bool Position::operator<(const Position& rhs) const +{ + if (line == rhs.line) + return column < rhs.column; + else + return line < rhs.line; +} + +bool Position::operator>(const Position& rhs) const +{ + if (line == rhs.line) + return column > rhs.column; + else + return line > rhs.line; +} + +bool Position::operator<=(const Position& rhs) const +{ + return *this == rhs || *this < rhs; +} + +bool Position::operator>=(const Position& rhs) const +{ + return *this == rhs || *this > rhs; +} + +void Position::shift(const Position& start, const Position& oldEnd, const Position& newEnd) +{ + if (*this >= start) + { + if (this->line > oldEnd.line) + this->line += (newEnd.line - oldEnd.line); + else + { + this->line = newEnd.line; + this->column += (newEnd.column - oldEnd.column); + } + } +} + +Location::Location() + : begin(0, 0) + , end(0, 0) +{ +} + +Location::Location(const Position& begin, const Position& end) + : begin(begin) + , end(end) +{ +} + +Location::Location(const Position& begin, unsigned int length) + : begin(begin) + , end(begin.line, begin.column + length) +{ +} + +Location::Location(const Location& begin, const Location& end) + : begin(begin.begin) + , end(end.end) +{ +} + +bool Location::operator==(const Location& rhs) const +{ + return this->begin == rhs.begin && this->end == rhs.end; +} + +bool Location::operator!=(const Location& rhs) const +{ + return !(*this == rhs); +} + +bool Location::encloses(const Location& l) const +{ + return begin <= l.begin && end >= l.end; +} + +bool Location::overlaps(const Location& l) const +{ + return (begin <= l.begin && end >= l.begin) || (begin <= l.end && end >= l.end) || (begin >= l.begin && end <= l.end); +} + +bool Location::contains(const Position& p) const +{ + return begin <= p && p < end; +} + +bool Location::containsClosed(const Position& p) const +{ + return begin <= p && p <= end; +} + +void Location::extend(const Location& other) +{ + if (other.begin < begin) + begin = other.begin; + if (other.end > end) + end = other.end; +} + +void Location::shift(const Position& start, const Position& oldEnd, const Position& newEnd) +{ + begin.shift(start, oldEnd, newEnd); + end.shift(start, oldEnd, newEnd); } } // namespace Luau diff --git a/CLI/Ast.cpp b/CLI/Ast.cpp index fd99d225..99c58393 100644 --- a/CLI/Ast.cpp +++ b/CLI/Ast.cpp @@ -6,6 +6,7 @@ #include "Luau/AstJsonEncoder.h" #include "Luau/Parser.h" #include "Luau/ParseOptions.h" +#include "Luau/ToString.h" #include "FileUtils.h" diff --git a/CodeGen/src/CodeGen.cpp b/CodeGen/src/CodeGen.cpp index 39ca913f..1c05b298 100644 --- a/CodeGen/src/CodeGen.cpp +++ b/CodeGen/src/CodeGen.cpp @@ -59,7 +59,7 @@ static void assembleHelpers(AssemblyBuilderX64& build, ModuleHelpers& helpers) } static int emitInst(AssemblyBuilderX64& build, NativeState& data, ModuleHelpers& helpers, Proto* proto, LuauOpcode op, const Instruction* pc, int i, - Label* labelarr, Label& fallback) + Label* labelarr, Label& next, Label& fallback) { int skip = 0; @@ -89,31 +89,31 @@ static int emitInst(AssemblyBuilderX64& build, NativeState& data, ModuleHelpers& emitInstGetGlobal(build, pc, i, fallback); break; case LOP_SETGLOBAL: - emitInstSetGlobal(build, pc, i, labelarr, fallback); + emitInstSetGlobal(build, pc, i, next, fallback); break; case LOP_CALL: - emitInstCall(build, helpers, pc, i, labelarr); + emitInstCall(build, helpers, pc, i); break; case LOP_RETURN: - emitInstReturn(build, helpers, pc, i, labelarr); + emitInstReturn(build, helpers, pc, i); break; case LOP_GETTABLE: - emitInstGetTable(build, pc, i, fallback); + emitInstGetTable(build, pc, fallback); break; case LOP_SETTABLE: - emitInstSetTable(build, pc, i, labelarr, fallback); + emitInstSetTable(build, pc, next, fallback); break; case LOP_GETTABLEKS: emitInstGetTableKS(build, pc, i, fallback); break; case LOP_SETTABLEKS: - emitInstSetTableKS(build, pc, i, labelarr, fallback); + emitInstSetTableKS(build, pc, i, next, fallback); break; case LOP_GETTABLEN: - emitInstGetTableN(build, pc, i, fallback); + emitInstGetTableN(build, pc, fallback); break; case LOP_SETTABLEN: - emitInstSetTableN(build, pc, i, labelarr, fallback); + emitInstSetTableN(build, pc, next, fallback); break; case LOP_JUMP: emitInstJump(build, pc, i, labelarr); @@ -161,94 +161,96 @@ static int emitInst(AssemblyBuilderX64& build, NativeState& data, ModuleHelpers& emitInstJumpxEqS(build, pc, i, labelarr); break; case LOP_ADD: - emitInstBinary(build, pc, i, TM_ADD, fallback); + emitInstBinary(build, pc, TM_ADD, fallback); break; case LOP_SUB: - emitInstBinary(build, pc, i, TM_SUB, fallback); + emitInstBinary(build, pc, TM_SUB, fallback); break; case LOP_MUL: - emitInstBinary(build, pc, i, TM_MUL, fallback); + emitInstBinary(build, pc, TM_MUL, fallback); break; case LOP_DIV: - emitInstBinary(build, pc, i, TM_DIV, fallback); + emitInstBinary(build, pc, TM_DIV, fallback); break; case LOP_MOD: - emitInstBinary(build, pc, i, TM_MOD, fallback); + emitInstBinary(build, pc, TM_MOD, fallback); break; case LOP_POW: - emitInstBinary(build, pc, i, TM_POW, fallback); + emitInstBinary(build, pc, TM_POW, fallback); break; case LOP_ADDK: - emitInstBinaryK(build, pc, i, TM_ADD, fallback); + emitInstBinaryK(build, pc, TM_ADD, fallback); break; case LOP_SUBK: - emitInstBinaryK(build, pc, i, TM_SUB, fallback); + emitInstBinaryK(build, pc, TM_SUB, fallback); break; case LOP_MULK: - emitInstBinaryK(build, pc, i, TM_MUL, fallback); + emitInstBinaryK(build, pc, TM_MUL, fallback); break; case LOP_DIVK: - emitInstBinaryK(build, pc, i, TM_DIV, fallback); + emitInstBinaryK(build, pc, TM_DIV, fallback); break; case LOP_MODK: - emitInstBinaryK(build, pc, i, TM_MOD, fallback); + emitInstBinaryK(build, pc, TM_MOD, fallback); break; case LOP_POWK: - emitInstPowK(build, pc, proto->k, i, fallback); + emitInstPowK(build, pc, proto->k, fallback); break; case LOP_NOT: emitInstNot(build, pc); break; case LOP_MINUS: - emitInstMinus(build, pc, i, fallback); + emitInstMinus(build, pc, fallback); break; case LOP_LENGTH: - emitInstLength(build, pc, i, fallback); + emitInstLength(build, pc, fallback); break; case LOP_NEWTABLE: - emitInstNewTable(build, pc, i, labelarr); + emitInstNewTable(build, pc, i, next); break; case LOP_DUPTABLE: - emitInstDupTable(build, pc, i, labelarr); + emitInstDupTable(build, pc, i, next); break; case LOP_SETLIST: - emitInstSetList(build, pc, i, labelarr); + emitInstSetList(build, pc, next); break; case LOP_GETUPVAL: - emitInstGetUpval(build, pc, i); + emitInstGetUpval(build, pc); break; case LOP_SETUPVAL: - emitInstSetUpval(build, pc, i, labelarr); + emitInstSetUpval(build, pc, next); break; case LOP_CLOSEUPVALS: - emitInstCloseUpvals(build, pc, i, labelarr); + emitInstCloseUpvals(build, pc, next); break; case LOP_FASTCALL: - skip = emitInstFastCall(build, pc, i, labelarr); + // We want to lower next instruction at skip+2, but this instruction is only 1 long, so we need to add 1 + skip = emitInstFastCall(build, pc, i, next) + 1; break; case LOP_FASTCALL1: - skip = emitInstFastCall1(build, pc, i, labelarr); + // We want to lower next instruction at skip+2, but this instruction is only 1 long, so we need to add 1 + skip = emitInstFastCall1(build, pc, i, next) + 1; break; case LOP_FASTCALL2: - skip = emitInstFastCall2(build, pc, i, labelarr); + skip = emitInstFastCall2(build, pc, i, next); break; case LOP_FASTCALL2K: - skip = emitInstFastCall2K(build, pc, i, labelarr); + skip = emitInstFastCall2K(build, pc, i, next); break; case LOP_FORNPREP: - emitInstForNPrep(build, pc, i, labelarr); + emitInstForNPrep(build, pc, i, labelarr[i + 1 + LUAU_INSN_D(*pc)]); break; case LOP_FORNLOOP: - emitInstForNLoop(build, pc, i, labelarr); + emitInstForNLoop(build, pc, i, labelarr[i + 1 + LUAU_INSN_D(*pc)]); break; case LOP_FORGLOOP: - emitinstForGLoop(build, pc, i, labelarr, fallback); + emitinstForGLoop(build, pc, i, labelarr[i + 1 + LUAU_INSN_D(*pc)], next, fallback); break; case LOP_FORGPREP_NEXT: - emitInstForGPrepNext(build, pc, i, labelarr, fallback); + emitInstForGPrepNext(build, pc, labelarr[i + 1 + LUAU_INSN_D(*pc)], fallback); break; case LOP_FORGPREP_INEXT: - emitInstForGPrepInext(build, pc, i, labelarr, fallback); + emitInstForGPrepInext(build, pc, labelarr[i + 1 + LUAU_INSN_D(*pc)], fallback); break; case LOP_AND: emitInstAnd(build, pc); @@ -266,7 +268,7 @@ static int emitInst(AssemblyBuilderX64& build, NativeState& data, ModuleHelpers& emitInstGetImport(build, pc, fallback); break; case LOP_CONCAT: - emitInstConcat(build, pc, i, labelarr); + emitInstConcat(build, pc, i, next); break; default: emitFallback(build, data, op, i); @@ -281,7 +283,8 @@ static void emitInstFallback(AssemblyBuilderX64& build, NativeState& data, LuauO switch (op) { case LOP_GETIMPORT: - emitInstGetImportFallback(build, pc, i); + emitSetSavedPc(build, i + 1); + emitInstGetImportFallback(build, LUAU_INSN_A(*pc), pc[1]); break; case LOP_GETTABLE: emitInstGetTableFallback(build, pc, i); @@ -356,11 +359,11 @@ static void emitInstFallback(AssemblyBuilderX64& build, NativeState& data, LuauO emitInstLengthFallback(build, pc, i); break; case LOP_FORGLOOP: - emitinstForGLoopFallback(build, pc, i, labelarr); + emitinstForGLoopFallback(build, pc, i, labelarr[i + 1 + LUAU_INSN_D(*pc)]); break; case LOP_FORGPREP_NEXT: case LOP_FORGPREP_INEXT: - emitInstForGPrepXnextFallback(build, pc, i, labelarr); + emitInstForGPrepXnextFallback(build, pc, i, labelarr[i + 1 + LUAU_INSN_D(*pc)]); break; case LOP_GETGLOBAL: // TODO: luaV_gettable + cachedslot update instead of full fallback @@ -430,7 +433,9 @@ static NativeProto* assembleFunction(AssemblyBuilderX64& build, NativeState& dat if (options.annotator) options.annotator(options.annotatorContext, build.text, proto->bytecodeid, i); - int skip = emitInst(build, data, helpers, proto, op, pc, i, instLabels.data(), instFallbacks[i]); + Label& next = nexti < proto->sizecode ? instLabels[nexti] : start; // Last instruction can't use 'next' label + + int skip = emitInst(build, data, helpers, proto, op, pc, i, instLabels.data(), next, instFallbacks[i]); if (skip != 0) instOutlines.push_back({nexti, skip}); @@ -454,15 +459,20 @@ static NativeProto* assembleFunction(AssemblyBuilderX64& build, NativeState& dat const Instruction* pc = &proto->code[i]; LuauOpcode op = LuauOpcode(LUAU_INSN_OP(*pc)); + int nexti = i + getOpLength(op); + LUAU_ASSERT(nexti <= proto->sizecode); + build.setLabel(instLabels[i]); if (options.annotator && !options.skipOutlinedCode) options.annotator(options.annotatorContext, build.text, proto->bytecodeid, i); - int skip = emitInst(build, data, helpers, proto, op, pc, i, instLabels.data(), instFallbacks[i]); + Label& next = nexti < proto->sizecode ? instLabels[nexti] : start; // Last instruction can't use 'next' label + + int skip = emitInst(build, data, helpers, proto, op, pc, i, instLabels.data(), next, instFallbacks[i]); LUAU_ASSERT(skip == 0); - i += getOpLength(op); + i = nexti; } if (i < proto->sizecode) diff --git a/CodeGen/src/EmitCommonX64.cpp b/CodeGen/src/EmitCommonX64.cpp index cbaa8494..fe258ff8 100644 --- a/CodeGen/src/EmitCommonX64.cpp +++ b/CodeGen/src/EmitCommonX64.cpp @@ -61,10 +61,8 @@ void jumpOnNumberCmp(AssemblyBuilderX64& build, RegisterX64 tmp, OperandX64 lhs, } } -void jumpOnAnyCmpFallback(AssemblyBuilderX64& build, int ra, int rb, ConditionX64 cond, Label& label, int pcpos) +void jumpOnAnyCmpFallback(AssemblyBuilderX64& build, int ra, int rb, ConditionX64 cond, Label& label) { - emitSetSavedPc(build, pcpos + 1); - build.mov(rArg1, rState); build.lea(rArg2, luauRegAddress(ra)); build.lea(rArg3, luauRegAddress(rb)); @@ -85,10 +83,8 @@ void jumpOnAnyCmpFallback(AssemblyBuilderX64& build, int ra, int rb, ConditionX6 label); } -RegisterX64 getTableNodeAtCachedSlot(AssemblyBuilderX64& build, RegisterX64 tmp, RegisterX64 table, int pcpos) +void getTableNodeAtCachedSlot(AssemblyBuilderX64& build, RegisterX64 tmp, RegisterX64 node, RegisterX64 table, int pcpos) { - RegisterX64 node = rdx; - LUAU_ASSERT(tmp != node); LUAU_ASSERT(table != node); @@ -102,16 +98,12 @@ RegisterX64 getTableNodeAtCachedSlot(AssemblyBuilderX64& build, RegisterX64 tmp, // LuaNode* n = &h->node[slot]; build.shl(dwordReg(tmp), kLuaNodeSizeLog2); build.add(node, tmp); - - return node; } -void convertNumberToIndexOrJump(AssemblyBuilderX64& build, RegisterX64 tmp, RegisterX64 numd, RegisterX64 numi, int ri, Label& label) +void convertNumberToIndexOrJump(AssemblyBuilderX64& build, RegisterX64 tmp, RegisterX64 numd, RegisterX64 numi, Label& label) { LUAU_ASSERT(numi.size == SizeX64::dword); - build.vmovsd(numd, luauRegValue(ri)); - // Convert to integer, NaN is converted into 0x80000000 build.vcvttsd2si(numi, numd); @@ -124,10 +116,8 @@ void convertNumberToIndexOrJump(AssemblyBuilderX64& build, RegisterX64 tmp, Regi build.jcc(ConditionX64::NotZero, label); } -void callArithHelper(AssemblyBuilderX64& build, int ra, int rb, OperandX64 c, int pcpos, TMS tm) +void callArithHelper(AssemblyBuilderX64& build, int ra, int rb, OperandX64 c, TMS tm) { - emitSetSavedPc(build, pcpos + 1); - if (build.abi == ABIX64::Windows) build.mov(sArg5, tm); else @@ -142,10 +132,8 @@ void callArithHelper(AssemblyBuilderX64& build, int ra, int rb, OperandX64 c, in emitUpdateBase(build); } -void callLengthHelper(AssemblyBuilderX64& build, int ra, int rb, int pcpos) +void callLengthHelper(AssemblyBuilderX64& build, int ra, int rb) { - emitSetSavedPc(build, pcpos + 1); - build.mov(rArg1, rState); build.lea(rArg2, luauRegAddress(ra)); build.lea(rArg3, luauRegAddress(rb)); @@ -154,10 +142,8 @@ void callLengthHelper(AssemblyBuilderX64& build, int ra, int rb, int pcpos) emitUpdateBase(build); } -void callPrepareForN(AssemblyBuilderX64& build, int limit, int step, int init, int pcpos) +void callPrepareForN(AssemblyBuilderX64& build, int limit, int step, int init) { - emitSetSavedPc(build, pcpos + 1); - build.mov(rArg1, rState); build.lea(rArg2, luauRegAddress(limit)); build.lea(rArg3, luauRegAddress(step)); @@ -165,10 +151,8 @@ void callPrepareForN(AssemblyBuilderX64& build, int limit, int step, int init, i build.call(qword[rNativeContext + offsetof(NativeContext, luaV_prepareFORN)]); } -void callGetTable(AssemblyBuilderX64& build, int rb, OperandX64 c, int ra, int pcpos) +void callGetTable(AssemblyBuilderX64& build, int rb, OperandX64 c, int ra) { - emitSetSavedPc(build, pcpos + 1); - build.mov(rArg1, rState); build.lea(rArg2, luauRegAddress(rb)); build.lea(rArg3, c); @@ -178,10 +162,8 @@ void callGetTable(AssemblyBuilderX64& build, int rb, OperandX64 c, int ra, int p emitUpdateBase(build); } -void callSetTable(AssemblyBuilderX64& build, int rb, OperandX64 c, int ra, int pcpos) +void callSetTable(AssemblyBuilderX64& build, int rb, OperandX64 c, int ra) { - emitSetSavedPc(build, pcpos + 1); - build.mov(rArg1, rState); build.lea(rArg2, luauRegAddress(rb)); build.lea(rArg3, c); diff --git a/CodeGen/src/EmitCommonX64.h b/CodeGen/src/EmitCommonX64.h index 61544855..238a0ed4 100644 --- a/CodeGen/src/EmitCommonX64.h +++ b/CodeGen/src/EmitCommonX64.h @@ -99,7 +99,7 @@ inline OperandX64 luauRegTag(int ri) return dword[rBase + ri * sizeof(TValue) + offsetof(TValue, tt)]; } -inline OperandX64 luauRegValueBoolean(int ri) +inline OperandX64 luauRegValueInt(int ri) { return dword[rBase + ri * sizeof(TValue) + offsetof(TValue, value)]; } @@ -174,7 +174,7 @@ inline void jumpIfFalsy(AssemblyBuilderX64& build, int ri, Label& target, Label& jumpIfTagIs(build, ri, LUA_TNIL, target); // false if nil jumpIfTagIsNot(build, ri, LUA_TBOOLEAN, fallthrough); // true if not nil or boolean - build.cmp(luauRegValueBoolean(ri), 0); + build.cmp(luauRegValueInt(ri), 0); build.jcc(ConditionX64::Equal, target); // true if boolean value is 'true' } @@ -184,7 +184,7 @@ inline void jumpIfTruthy(AssemblyBuilderX64& build, int ri, Label& target, Label jumpIfTagIs(build, ri, LUA_TNIL, fallthrough); // false if nil jumpIfTagIsNot(build, ri, LUA_TBOOLEAN, target); // true if not nil or boolean - build.cmp(luauRegValueBoolean(ri), 0); + build.cmp(luauRegValueInt(ri), 0); build.jcc(ConditionX64::NotEqual, target); // true if boolean value is 'true' } @@ -236,16 +236,16 @@ inline void jumpIfNodeKeyNotInExpectedSlot(AssemblyBuilderX64& build, RegisterX6 } void jumpOnNumberCmp(AssemblyBuilderX64& build, RegisterX64 tmp, OperandX64 lhs, OperandX64 rhs, ConditionX64 cond, Label& label); -void jumpOnAnyCmpFallback(AssemblyBuilderX64& build, int ra, int rb, ConditionX64 cond, Label& label, int pcpos); +void jumpOnAnyCmpFallback(AssemblyBuilderX64& build, int ra, int rb, ConditionX64 cond, Label& label); -RegisterX64 getTableNodeAtCachedSlot(AssemblyBuilderX64& build, RegisterX64 tmp, RegisterX64 table, int pcpos); -void convertNumberToIndexOrJump(AssemblyBuilderX64& build, RegisterX64 tmp, RegisterX64 numd, RegisterX64 numi, int ri, Label& label); +void getTableNodeAtCachedSlot(AssemblyBuilderX64& build, RegisterX64 tmp, RegisterX64 node, RegisterX64 table, int pcpos); +void convertNumberToIndexOrJump(AssemblyBuilderX64& build, RegisterX64 tmp, RegisterX64 numd, RegisterX64 numi, Label& label); -void callArithHelper(AssemblyBuilderX64& build, int ra, int rb, OperandX64 c, int pcpos, TMS tm); -void callLengthHelper(AssemblyBuilderX64& build, int ra, int rb, int pcpos); -void callPrepareForN(AssemblyBuilderX64& build, int limit, int step, int init, int pcpos); -void callGetTable(AssemblyBuilderX64& build, int rb, OperandX64 c, int ra, int pcpos); -void callSetTable(AssemblyBuilderX64& build, int rb, OperandX64 c, int ra, int pcpos); +void callArithHelper(AssemblyBuilderX64& build, int ra, int rb, OperandX64 c, TMS tm); +void callLengthHelper(AssemblyBuilderX64& build, int ra, int rb); +void callPrepareForN(AssemblyBuilderX64& build, int limit, int step, int init); +void callGetTable(AssemblyBuilderX64& build, int rb, OperandX64 c, int ra); +void callSetTable(AssemblyBuilderX64& build, int rb, OperandX64 c, int ra); void callBarrierTable(AssemblyBuilderX64& build, RegisterX64 tmp, RegisterX64 table, int ra, Label& skip); void callBarrierObject(AssemblyBuilderX64& build, RegisterX64 tmp, RegisterX64 object, int ra, Label& skip); void callBarrierTableFast(AssemblyBuilderX64& build, RegisterX64 table, Label& skip); diff --git a/CodeGen/src/EmitInstructionX64.cpp b/CodeGen/src/EmitInstructionX64.cpp index 160f0f6c..abbdb65c 100644 --- a/CodeGen/src/EmitInstructionX64.cpp +++ b/CodeGen/src/EmitInstructionX64.cpp @@ -69,7 +69,7 @@ void emitInstMove(AssemblyBuilderX64& build, const Instruction* pc) build.vmovups(luauReg(ra), xmm0); } -void emitInstCall(AssemblyBuilderX64& build, ModuleHelpers& helpers, const Instruction* pc, int pcpos, Label* labelarr) +void emitInstCall(AssemblyBuilderX64& build, ModuleHelpers& helpers, const Instruction* pc, int pcpos) { int ra = LUAU_INSN_A(*pc); int nparams = LUAU_INSN_B(*pc) - 1; @@ -222,7 +222,7 @@ void emitInstCall(AssemblyBuilderX64& build, ModuleHelpers& helpers, const Instr } } -void emitInstReturn(AssemblyBuilderX64& build, ModuleHelpers& helpers, const Instruction* pc, int pcpos, Label* labelarr) +void emitInstReturn(AssemblyBuilderX64& build, ModuleHelpers& helpers, const Instruction* pc, int pcpos) { emitInterrupt(build, pcpos); @@ -435,7 +435,8 @@ void emitInstJumpIfEqFallback(AssemblyBuilderX64& build, const Instruction* pc, { Label& target = labelarr[pcpos + 1 + LUAU_INSN_D(*pc)]; - jumpOnAnyCmpFallback(build, LUAU_INSN_A(*pc), pc[1], not_ ? ConditionX64::NotEqual : ConditionX64::Equal, target, pcpos); + emitSetSavedPc(build, pcpos + 1); + jumpOnAnyCmpFallback(build, LUAU_INSN_A(*pc), pc[1], not_ ? ConditionX64::NotEqual : ConditionX64::Equal, target); } void emitInstJumpIfCond(AssemblyBuilderX64& build, const Instruction* pc, int pcpos, Label* labelarr, ConditionX64 cond, Label& fallback) @@ -456,7 +457,8 @@ void emitInstJumpIfCondFallback(AssemblyBuilderX64& build, const Instruction* pc { Label& target = labelarr[pcpos + 1 + LUAU_INSN_D(*pc)]; - jumpOnAnyCmpFallback(build, LUAU_INSN_A(*pc), pc[1], cond, target, pcpos); + emitSetSavedPc(build, pcpos + 1); + jumpOnAnyCmpFallback(build, LUAU_INSN_A(*pc), pc[1], cond, target); } void emitInstJumpX(AssemblyBuilderX64& build, const Instruction* pc, int pcpos, Label* labelarr) @@ -488,7 +490,7 @@ void emitInstJumpxEqB(AssemblyBuilderX64& build, const Instruction* pc, int pcpo jumpIfTagIsNot(build, ra, LUA_TBOOLEAN, not_ ? target : exit); - build.test(luauRegValueBoolean(ra), 1); + build.test(luauRegValueInt(ra), 1); build.jcc((aux & 0x1) ^ not_ ? ConditionX64::NotZero : ConditionX64::Zero, target); } @@ -534,7 +536,7 @@ void emitInstJumpxEqS(AssemblyBuilderX64& build, const Instruction* pc, int pcpo build.jcc(not_ ? ConditionX64::NotEqual : ConditionX64::Equal, target); } -static void emitInstBinaryNumeric(AssemblyBuilderX64& build, int ra, int rb, int rc, OperandX64 opc, int pcpos, TMS tm, Label& fallback) +static void emitInstBinaryNumeric(AssemblyBuilderX64& build, int ra, int rb, int rc, OperandX64 opc, TMS tm, Label& fallback) { jumpIfTagIsNot(build, rb, LUA_TNUMBER, fallback); @@ -580,27 +582,29 @@ static void emitInstBinaryNumeric(AssemblyBuilderX64& build, int ra, int rb, int build.mov(luauRegTag(ra), LUA_TNUMBER); } -void emitInstBinary(AssemblyBuilderX64& build, const Instruction* pc, int pcpos, TMS tm, Label& fallback) +void emitInstBinary(AssemblyBuilderX64& build, const Instruction* pc, TMS tm, Label& fallback) { - emitInstBinaryNumeric(build, LUAU_INSN_A(*pc), LUAU_INSN_B(*pc), LUAU_INSN_C(*pc), luauRegValue(LUAU_INSN_C(*pc)), pcpos, tm, fallback); + emitInstBinaryNumeric(build, LUAU_INSN_A(*pc), LUAU_INSN_B(*pc), LUAU_INSN_C(*pc), luauRegValue(LUAU_INSN_C(*pc)), tm, fallback); } void emitInstBinaryFallback(AssemblyBuilderX64& build, const Instruction* pc, int pcpos, TMS tm) { - callArithHelper(build, LUAU_INSN_A(*pc), LUAU_INSN_B(*pc), luauRegAddress(LUAU_INSN_C(*pc)), pcpos, tm); + emitSetSavedPc(build, pcpos + 1); + callArithHelper(build, LUAU_INSN_A(*pc), LUAU_INSN_B(*pc), luauRegAddress(LUAU_INSN_C(*pc)), tm); } -void emitInstBinaryK(AssemblyBuilderX64& build, const Instruction* pc, int pcpos, TMS tm, Label& fallback) +void emitInstBinaryK(AssemblyBuilderX64& build, const Instruction* pc, TMS tm, Label& fallback) { - emitInstBinaryNumeric(build, LUAU_INSN_A(*pc), LUAU_INSN_B(*pc), -1, luauConstantValue(LUAU_INSN_C(*pc)), pcpos, tm, fallback); + emitInstBinaryNumeric(build, LUAU_INSN_A(*pc), LUAU_INSN_B(*pc), -1, luauConstantValue(LUAU_INSN_C(*pc)), tm, fallback); } void emitInstBinaryKFallback(AssemblyBuilderX64& build, const Instruction* pc, int pcpos, TMS tm) { - callArithHelper(build, LUAU_INSN_A(*pc), LUAU_INSN_B(*pc), luauConstantAddress(LUAU_INSN_C(*pc)), pcpos, tm); + emitSetSavedPc(build, pcpos + 1); + callArithHelper(build, LUAU_INSN_A(*pc), LUAU_INSN_B(*pc), luauConstantAddress(LUAU_INSN_C(*pc)), tm); } -void emitInstPowK(AssemblyBuilderX64& build, const Instruction* pc, const TValue* k, int pcpos, Label& fallback) +void emitInstPowK(AssemblyBuilderX64& build, const Instruction* pc, const TValue* k, Label& fallback) { int ra = LUAU_INSN_A(*pc); int rb = LUAU_INSN_B(*pc); @@ -647,17 +651,17 @@ void emitInstNot(AssemblyBuilderX64& build, const Instruction* pc) jumpIfFalsy(build, rb, saveone, savezero); build.setLabel(savezero); - build.mov(luauRegValueBoolean(ra), 0); + build.mov(luauRegValueInt(ra), 0); build.jmp(exit); build.setLabel(saveone); - build.mov(luauRegValueBoolean(ra), 1); + build.mov(luauRegValueInt(ra), 1); build.setLabel(exit); build.mov(luauRegTag(ra), LUA_TBOOLEAN); } -void emitInstMinus(AssemblyBuilderX64& build, const Instruction* pc, int pcpos, Label& fallback) +void emitInstMinus(AssemblyBuilderX64& build, const Instruction* pc, Label& fallback) { int ra = LUAU_INSN_A(*pc); int rb = LUAU_INSN_B(*pc); @@ -675,10 +679,11 @@ void emitInstMinus(AssemblyBuilderX64& build, const Instruction* pc, int pcpos, void emitInstMinusFallback(AssemblyBuilderX64& build, const Instruction* pc, int pcpos) { - callArithHelper(build, LUAU_INSN_A(*pc), LUAU_INSN_B(*pc), luauRegAddress(LUAU_INSN_B(*pc)), pcpos, TM_UNM); + emitSetSavedPc(build, pcpos + 1); + callArithHelper(build, LUAU_INSN_A(*pc), LUAU_INSN_B(*pc), luauRegAddress(LUAU_INSN_B(*pc)), TM_UNM); } -void emitInstLength(AssemblyBuilderX64& build, const Instruction* pc, int pcpos, Label& fallback) +void emitInstLength(AssemblyBuilderX64& build, const Instruction* pc, Label& fallback) { int ra = LUAU_INSN_A(*pc); int rb = LUAU_INSN_B(*pc); @@ -699,35 +704,32 @@ void emitInstLength(AssemblyBuilderX64& build, const Instruction* pc, int pcpos, void emitInstLengthFallback(AssemblyBuilderX64& build, const Instruction* pc, int pcpos) { - callLengthHelper(build, LUAU_INSN_A(*pc), LUAU_INSN_B(*pc), pcpos); + emitSetSavedPc(build, pcpos + 1); + callLengthHelper(build, LUAU_INSN_A(*pc), LUAU_INSN_B(*pc)); } -void emitInstNewTable(AssemblyBuilderX64& build, const Instruction* pc, int pcpos, Label* labelarr) +void emitInstNewTable(AssemblyBuilderX64& build, const Instruction* pc, int pcpos, Label& next) { int ra = LUAU_INSN_A(*pc); int b = LUAU_INSN_B(*pc); uint32_t aux = pc[1]; - Label& exit = labelarr[pcpos + 2]; - emitSetSavedPc(build, pcpos + 1); build.mov(rArg1, rState); build.mov(dwordReg(rArg2), aux); - build.mov(dwordReg(rArg3), 1 << (b - 1)); + build.mov(dwordReg(rArg3), b == 0 ? 0 : 1 << (b - 1)); build.call(qword[rNativeContext + offsetof(NativeContext, luaH_new)]); build.mov(luauRegValue(ra), rax); build.mov(luauRegTag(ra), LUA_TTABLE); - callCheckGc(build, pcpos, /* savepc = */ false, exit); + callCheckGc(build, pcpos, /* savepc = */ false, next); } -void emitInstDupTable(AssemblyBuilderX64& build, const Instruction* pc, int pcpos, Label* labelarr) +void emitInstDupTable(AssemblyBuilderX64& build, const Instruction* pc, int pcpos, Label& next) { int ra = LUAU_INSN_A(*pc); - Label& exit = labelarr[pcpos + 1]; - emitSetSavedPc(build, pcpos + 1); build.mov(rArg1, rState); @@ -736,18 +738,16 @@ void emitInstDupTable(AssemblyBuilderX64& build, const Instruction* pc, int pcpo build.mov(luauRegValue(ra), rax); build.mov(luauRegTag(ra), LUA_TTABLE); - callCheckGc(build, pcpos, /* savepc= */ false, exit); + callCheckGc(build, pcpos, /* savepc= */ false, next); } -void emitInstSetList(AssemblyBuilderX64& build, const Instruction* pc, int pcpos, Label* labelarr) +void emitInstSetList(AssemblyBuilderX64& build, const Instruction* pc, Label& next) { int ra = LUAU_INSN_A(*pc); int rb = LUAU_INSN_B(*pc); int c = LUAU_INSN_C(*pc) - 1; uint32_t index = pc[1]; - Label& exit = labelarr[pcpos + 2]; - OperandX64 last = index + c - 1; // Using non-volatile 'rbx' for dynamic 'c' value (for LUA_MULTRET) to skip later recomputation @@ -842,10 +842,10 @@ void emitInstSetList(AssemblyBuilderX64& build, const Instruction* pc, int pcpos build.setLabel(endLoop); } - callBarrierTableFast(build, table, exit); + callBarrierTableFast(build, table, next); } -void emitInstGetUpval(AssemblyBuilderX64& build, const Instruction* pc, int pcpos) +void emitInstGetUpval(AssemblyBuilderX64& build, const Instruction* pc) { int ra = LUAU_INSN_A(*pc); int up = LUAU_INSN_B(*pc); @@ -869,7 +869,7 @@ void emitInstGetUpval(AssemblyBuilderX64& build, const Instruction* pc, int pcpo build.vmovups(luauReg(ra), xmm0); } -void emitInstSetUpval(AssemblyBuilderX64& build, const Instruction* pc, int pcpos, Label* labelarr) +void emitInstSetUpval(AssemblyBuilderX64& build, const Instruction* pc, Label& next) { int ra = LUAU_INSN_A(*pc); int up = LUAU_INSN_B(*pc); @@ -884,32 +884,30 @@ void emitInstSetUpval(AssemblyBuilderX64& build, const Instruction* pc, int pcpo build.vmovups(xmm0, luauReg(ra)); build.vmovups(xmmword[tmp], xmm0); - callBarrierObject(build, tmp, upval, ra, labelarr[pcpos + 1]); + callBarrierObject(build, tmp, upval, ra, next); } -void emitInstCloseUpvals(AssemblyBuilderX64& build, const Instruction* pc, int pcpos, Label* labelarr) +void emitInstCloseUpvals(AssemblyBuilderX64& build, const Instruction* pc, Label& next) { int ra = LUAU_INSN_A(*pc); - Label& skip = labelarr[pcpos + 1]; - // L->openupval != 0 build.mov(rax, qword[rState + offsetof(lua_State, openupval)]); build.test(rax, rax); - build.jcc(ConditionX64::Zero, skip); + build.jcc(ConditionX64::Zero, next); // ra <= L->openuval->v build.lea(rcx, addr[rBase + ra * sizeof(TValue)]); build.cmp(rcx, qword[rax + offsetof(UpVal, v)]); - build.jcc(ConditionX64::Above, skip); + build.jcc(ConditionX64::Above, next); build.mov(rArg2, rcx); build.mov(rArg1, rState); build.call(qword[rNativeContext + offsetof(NativeContext, luaF_close)]); } -static int emitInstFastCallN(AssemblyBuilderX64& build, const Instruction* pc, bool customParams, int customParamCount, OperandX64 customArgs, - int pcpos, int instLen, Label* labelarr) +static int emitInstFastCallN( + AssemblyBuilderX64& build, const Instruction* pc, bool customParams, int customParamCount, OperandX64 customArgs, int pcpos, Label& fallback) { int bfid = LUAU_INSN_A(*pc); int skip = LUAU_INSN_C(*pc); @@ -923,11 +921,9 @@ static int emitInstFastCallN(AssemblyBuilderX64& build, const Instruction* pc, b int arg = customParams ? LUAU_INSN_B(*pc) : ra + 1; OperandX64 args = customParams ? customArgs : luauRegAddress(ra + 2); - Label& exit = labelarr[pcpos + instLen]; + jumpIfUnsafeEnv(build, rax, fallback); - jumpIfUnsafeEnv(build, rax, exit); - - BuiltinImplResult br = emitBuiltin(build, LuauBuiltinFunction(bfid), nparams, ra, arg, args, nresults, exit); + BuiltinImplResult br = emitBuiltin(build, LuauBuiltinFunction(bfid), nparams, ra, arg, args, nresults, fallback); if (br.type == BuiltinImplType::UsesFallback) { @@ -945,7 +941,7 @@ static int emitInstFastCallN(AssemblyBuilderX64& build, const Instruction* pc, b build.mov(qword[rState + offsetof(lua_State, top)], rax); } - return skip + 2 - instLen; // Return fallback instruction sequence length + return skip; // Return fallback instruction sequence length } // TODO: we can skip saving pc for some well-behaved builtins which we didn't inline @@ -996,8 +992,8 @@ static int emitInstFastCallN(AssemblyBuilderX64& build, const Instruction* pc, b build.call(rax); - build.test(eax, eax); // test here will set SF=1 for a negative number and it always sets OF to 0 - build.jcc(ConditionX64::Less, exit); // jl jumps if SF != OF + build.test(eax, eax); // test here will set SF=1 for a negative number and it always sets OF to 0 + build.jcc(ConditionX64::Less, fallback); // jl jumps if SF != OF if (nresults == LUA_MULTRET) { @@ -1014,35 +1010,33 @@ static int emitInstFastCallN(AssemblyBuilderX64& build, const Instruction* pc, b build.mov(qword[rState + offsetof(lua_State, top)], rax); } - return skip + 2 - instLen; // Return fallback instruction sequence length + return skip; // Return fallback instruction sequence length } -int emitInstFastCall1(AssemblyBuilderX64& build, const Instruction* pc, int pcpos, Label* labelarr) +int emitInstFastCall1(AssemblyBuilderX64& build, const Instruction* pc, int pcpos, Label& fallback) { - return emitInstFastCallN(build, pc, /* customParams */ true, /* customParamCount */ 1, /* customArgs */ 0, pcpos, /* instLen */ 1, labelarr); + return emitInstFastCallN(build, pc, /* customParams */ true, /* customParamCount */ 1, /* customArgs */ 0, pcpos, fallback); } -int emitInstFastCall2(AssemblyBuilderX64& build, const Instruction* pc, int pcpos, Label* labelarr) +int emitInstFastCall2(AssemblyBuilderX64& build, const Instruction* pc, int pcpos, Label& fallback) +{ + return emitInstFastCallN(build, pc, /* customParams */ true, /* customParamCount */ 2, /* customArgs */ luauRegAddress(pc[1]), pcpos, fallback); +} + +int emitInstFastCall2K(AssemblyBuilderX64& build, const Instruction* pc, int pcpos, Label& fallback) { return emitInstFastCallN( - build, pc, /* customParams */ true, /* customParamCount */ 2, /* customArgs */ luauRegAddress(pc[1]), pcpos, /* instLen */ 2, labelarr); + build, pc, /* customParams */ true, /* customParamCount */ 2, /* customArgs */ luauConstantAddress(pc[1]), pcpos, fallback); } -int emitInstFastCall2K(AssemblyBuilderX64& build, const Instruction* pc, int pcpos, Label* labelarr) +int emitInstFastCall(AssemblyBuilderX64& build, const Instruction* pc, int pcpos, Label& fallback) { - return emitInstFastCallN( - build, pc, /* customParams */ true, /* customParamCount */ 2, /* customArgs */ luauConstantAddress(pc[1]), pcpos, /* instLen */ 2, labelarr); + return emitInstFastCallN(build, pc, /* customParams */ false, /* customParamCount */ 0, /* customArgs */ 0, pcpos, fallback); } -int emitInstFastCall(AssemblyBuilderX64& build, const Instruction* pc, int pcpos, Label* labelarr) -{ - return emitInstFastCallN(build, pc, /* customParams */ false, /* customParamCount */ 0, /* customArgs */ 0, pcpos, /* instLen */ 1, labelarr); -} - -void emitInstForNPrep(AssemblyBuilderX64& build, const Instruction* pc, int pcpos, Label* labelarr) +void emitInstForNPrep(AssemblyBuilderX64& build, const Instruction* pc, int pcpos, Label& loopExit) { int ra = LUAU_INSN_A(*pc); - Label& loopExit = labelarr[pcpos + 1 + LUAU_INSN_D(*pc)]; Label tryConvert, exit; @@ -1080,18 +1074,18 @@ void emitInstForNPrep(AssemblyBuilderX64& build, const Instruction* pc, int pcpo // TOOD: place at the end of the function build.setLabel(tryConvert); - callPrepareForN(build, ra + 0, ra + 1, ra + 2, pcpos); + emitSetSavedPc(build, pcpos + 1); + callPrepareForN(build, ra + 0, ra + 1, ra + 2); build.jmp(retry); build.setLabel(exit); } -void emitInstForNLoop(AssemblyBuilderX64& build, const Instruction* pc, int pcpos, Label* labelarr) +void emitInstForNLoop(AssemblyBuilderX64& build, const Instruction* pc, int pcpos, Label& loopRepeat) { emitInterrupt(build, pcpos); int ra = LUAU_INSN_A(*pc); - Label& loopRepeat = labelarr[pcpos + 1 + LUAU_INSN_D(*pc)]; RegisterX64 limit = xmm0; RegisterX64 step = xmm1; @@ -1121,14 +1115,11 @@ void emitInstForNLoop(AssemblyBuilderX64& build, const Instruction* pc, int pcpo build.setLabel(exit); } -void emitinstForGLoop(AssemblyBuilderX64& build, const Instruction* pc, int pcpos, Label* labelarr, Label& fallback) +void emitinstForGLoop(AssemblyBuilderX64& build, const Instruction* pc, int pcpos, Label& loopRepeat, Label& loopExit, Label& fallback) { int ra = LUAU_INSN_A(*pc); int aux = pc[1]; - Label& loopRepeat = labelarr[pcpos + 1 + LUAU_INSN_D(*pc)]; - Label& exit = labelarr[pcpos + 2]; - emitInterrupt(build, pcpos); // fast-path: builtin table iteration @@ -1160,13 +1151,13 @@ void emitinstForGLoop(AssemblyBuilderX64& build, const Instruction* pc, int pcpo // while (unsigned(index) < unsigned(sizearray)) Label arrayLoop = build.setLabel(); build.cmp(dwordReg(index), dword[table + offsetof(Table, sizearray)]); - build.jcc(ConditionX64::NotBelow, isIpairsIter ? exit : skipArray); + build.jcc(ConditionX64::NotBelow, isIpairsIter ? loopExit : skipArray); // If element is nil, we increment the index; if it's not, we still need 'index + 1' inside build.inc(index); build.cmp(dword[elemPtr + offsetof(TValue, tt)], LUA_TNIL); - build.jcc(ConditionX64::Equal, isIpairsIter ? exit : skipArrayNil); + build.jcc(ConditionX64::Equal, isIpairsIter ? loopExit : skipArrayNil); // setpvalue(ra + 2, reinterpret_cast(uintptr_t(index + 1))); build.mov(luauRegValue(ra + 2), index); @@ -1202,13 +1193,11 @@ void emitinstForGLoop(AssemblyBuilderX64& build, const Instruction* pc, int pcpo } } -void emitinstForGLoopFallback(AssemblyBuilderX64& build, const Instruction* pc, int pcpos, Label* labelarr) +void emitinstForGLoopFallback(AssemblyBuilderX64& build, const Instruction* pc, int pcpos, Label& loopRepeat) { int ra = LUAU_INSN_A(*pc); int aux = pc[1]; - Label& loopRepeat = labelarr[pcpos + 1 + LUAU_INSN_D(*pc)]; - emitSetSavedPc(build, pcpos + 1); build.mov(rArg1, rState); @@ -1220,12 +1209,10 @@ void emitinstForGLoopFallback(AssemblyBuilderX64& build, const Instruction* pc, build.jcc(ConditionX64::NotZero, loopRepeat); } -void emitInstForGPrepNext(AssemblyBuilderX64& build, const Instruction* pc, int pcpos, Label* labelarr, Label& fallback) +void emitInstForGPrepNext(AssemblyBuilderX64& build, const Instruction* pc, Label& target, Label& fallback) { int ra = LUAU_INSN_A(*pc); - Label& target = labelarr[pcpos + 1 + LUAU_INSN_D(*pc)]; - // fast-path: pairs/next jumpIfUnsafeEnv(build, rax, fallback); jumpIfTagIsNot(build, ra + 1, LUA_TTABLE, fallback); @@ -1240,12 +1227,10 @@ void emitInstForGPrepNext(AssemblyBuilderX64& build, const Instruction* pc, int build.jmp(target); } -void emitInstForGPrepInext(AssemblyBuilderX64& build, const Instruction* pc, int pcpos, Label* labelarr, Label& fallback) +void emitInstForGPrepInext(AssemblyBuilderX64& build, const Instruction* pc, Label& target, Label& fallback) { int ra = LUAU_INSN_A(*pc); - Label& target = labelarr[pcpos + 1 + LUAU_INSN_D(*pc)]; - // fast-path: ipairs/inext jumpIfUnsafeEnv(build, rax, fallback); jumpIfTagIsNot(build, ra + 1, LUA_TTABLE, fallback); @@ -1264,12 +1249,10 @@ void emitInstForGPrepInext(AssemblyBuilderX64& build, const Instruction* pc, int build.jmp(target); } -void emitInstForGPrepXnextFallback(AssemblyBuilderX64& build, const Instruction* pc, int pcpos, Label* labelarr) +void emitInstForGPrepXnextFallback(AssemblyBuilderX64& build, const Instruction* pc, int pcpos, Label& target) { int ra = LUAU_INSN_A(*pc); - Label& target = labelarr[pcpos + 1 + LUAU_INSN_D(*pc)]; - build.mov(rArg1, rState); build.lea(rArg2, luauRegAddress(ra)); build.mov(dwordReg(rArg3), pcpos + 1); @@ -1353,7 +1336,7 @@ void emitInstOrK(AssemblyBuilderX64& build, const Instruction* pc) emitInstOrX(build, LUAU_INSN_A(*pc), LUAU_INSN_B(*pc), luauConstant(LUAU_INSN_C(*pc))); } -void emitInstGetTableN(AssemblyBuilderX64& build, const Instruction* pc, int pcpos, Label& fallback) +void emitInstGetTableN(AssemblyBuilderX64& build, const Instruction* pc, Label& fallback) { int ra = LUAU_INSN_A(*pc); int rb = LUAU_INSN_B(*pc); @@ -1376,12 +1359,14 @@ void emitInstGetTableN(AssemblyBuilderX64& build, const Instruction* pc, int pcp void emitInstGetTableNFallback(AssemblyBuilderX64& build, const Instruction* pc, int pcpos) { + emitSetSavedPc(build, pcpos + 1); + TValue n; setnvalue(&n, LUAU_INSN_C(*pc) + 1); - callGetTable(build, LUAU_INSN_B(*pc), build.bytes(&n, sizeof(n)), LUAU_INSN_A(*pc), pcpos); + callGetTable(build, LUAU_INSN_B(*pc), build.bytes(&n, sizeof(n)), LUAU_INSN_A(*pc)); } -void emitInstSetTableN(AssemblyBuilderX64& build, const Instruction* pc, int pcpos, Label* labelarr, Label& fallback) +void emitInstSetTableN(AssemblyBuilderX64& build, const Instruction* pc, Label& next, Label& fallback) { int ra = LUAU_INSN_A(*pc); int rb = LUAU_INSN_B(*pc); @@ -1404,17 +1389,19 @@ void emitInstSetTableN(AssemblyBuilderX64& build, const Instruction* pc, int pcp build.vmovups(xmm0, luauReg(ra)); build.vmovups(xmmword[rax + c * sizeof(TValue)], xmm0); - callBarrierTable(build, rax, table, ra, labelarr[pcpos + 1]); + callBarrierTable(build, rax, table, ra, next); } void emitInstSetTableNFallback(AssemblyBuilderX64& build, const Instruction* pc, int pcpos) { + emitSetSavedPc(build, pcpos + 1); + TValue n; setnvalue(&n, LUAU_INSN_C(*pc) + 1); - callSetTable(build, LUAU_INSN_B(*pc), build.bytes(&n, sizeof(n)), LUAU_INSN_A(*pc), pcpos); + callSetTable(build, LUAU_INSN_B(*pc), build.bytes(&n, sizeof(n)), LUAU_INSN_A(*pc)); } -void emitInstGetTable(AssemblyBuilderX64& build, const Instruction* pc, int pcpos, Label& fallback) +void emitInstGetTable(AssemblyBuilderX64& build, const Instruction* pc, Label& fallback) { int ra = LUAU_INSN_A(*pc); int rb = LUAU_INSN_B(*pc); @@ -1427,29 +1414,33 @@ void emitInstGetTable(AssemblyBuilderX64& build, const Instruction* pc, int pcpo RegisterX64 table = rcx; build.mov(table, luauRegValue(rb)); - convertNumberToIndexOrJump(build, xmm1, xmm0, eax, rc, fallback); + RegisterX64 intIndex = eax; + RegisterX64 fpIndex = xmm0; + build.vmovsd(fpIndex, luauRegValue(rc)); + convertNumberToIndexOrJump(build, xmm1, fpIndex, intIndex, fallback); // index - 1 - build.dec(eax); + build.dec(intIndex); // unsigned(index - 1) < unsigned(h->sizearray) - build.cmp(dword[table + offsetof(Table, sizearray)], eax); + build.cmp(dword[table + offsetof(Table, sizearray)], intIndex); build.jcc(ConditionX64::BelowEqual, fallback); jumpIfMetatablePresent(build, table, fallback); // setobj2s(L, ra, &h->array[unsigned(index - 1)]); build.mov(rdx, qword[table + offsetof(Table, array)]); - build.shl(eax, kTValueSizeLog2); + build.shl(intIndex, kTValueSizeLog2); setLuauReg(build, xmm0, ra, xmmword[rdx + rax]); } void emitInstGetTableFallback(AssemblyBuilderX64& build, const Instruction* pc, int pcpos) { - callGetTable(build, LUAU_INSN_B(*pc), luauRegAddress(LUAU_INSN_C(*pc)), LUAU_INSN_A(*pc), pcpos); + emitSetSavedPc(build, pcpos + 1); + callGetTable(build, LUAU_INSN_B(*pc), luauRegAddress(LUAU_INSN_C(*pc)), LUAU_INSN_A(*pc)); } -void emitInstSetTable(AssemblyBuilderX64& build, const Instruction* pc, int pcpos, Label* labelarr, Label& fallback) +void emitInstSetTable(AssemblyBuilderX64& build, const Instruction* pc, Label& next, Label& fallback) { int ra = LUAU_INSN_A(*pc); int rb = LUAU_INSN_B(*pc); @@ -1462,13 +1453,16 @@ void emitInstSetTable(AssemblyBuilderX64& build, const Instruction* pc, int pcpo RegisterX64 table = rcx; build.mov(table, luauRegValue(rb)); - convertNumberToIndexOrJump(build, xmm1, xmm0, eax, rc, fallback); + RegisterX64 intIndex = eax; + RegisterX64 fpIndex = xmm0; + build.vmovsd(fpIndex, luauRegValue(rc)); + convertNumberToIndexOrJump(build, xmm1, fpIndex, intIndex, fallback); // index - 1 - build.dec(eax); + build.dec(intIndex); // unsigned(index - 1) < unsigned(h->sizearray) - build.cmp(dword[table + offsetof(Table, sizearray)], eax); + build.cmp(dword[table + offsetof(Table, sizearray)], intIndex); build.jcc(ConditionX64::BelowEqual, fallback); jumpIfMetatablePresent(build, table, fallback); @@ -1476,16 +1470,17 @@ void emitInstSetTable(AssemblyBuilderX64& build, const Instruction* pc, int pcpo // setobj2t(L, &h->array[unsigned(index - 1)], ra); build.mov(rdx, qword[table + offsetof(Table, array)]); - build.shl(eax, kTValueSizeLog2); + build.shl(intIndex, kTValueSizeLog2); build.vmovups(xmm0, luauReg(ra)); build.vmovups(xmmword[rdx + rax], xmm0); - callBarrierTable(build, rdx, table, ra, labelarr[pcpos + 1]); + callBarrierTable(build, rdx, table, ra, next); } void emitInstSetTableFallback(AssemblyBuilderX64& build, const Instruction* pc, int pcpos) { - callSetTable(build, LUAU_INSN_B(*pc), luauRegAddress(LUAU_INSN_C(*pc)), LUAU_INSN_A(*pc), pcpos); + emitSetSavedPc(build, pcpos + 1); + callSetTable(build, LUAU_INSN_B(*pc), luauRegAddress(LUAU_INSN_C(*pc)), LUAU_INSN_A(*pc)); } void emitInstGetImport(AssemblyBuilderX64& build, const Instruction* pc, Label& fallback) @@ -1504,13 +1499,8 @@ void emitInstGetImport(AssemblyBuilderX64& build, const Instruction* pc, Label& build.vmovups(luauReg(ra), xmm0); } -void emitInstGetImportFallback(AssemblyBuilderX64& build, const Instruction* pc, int pcpos) +void emitInstGetImportFallback(AssemblyBuilderX64& build, int ra, uint32_t aux) { - int ra = LUAU_INSN_A(*pc); - uint32_t aux = pc[1]; - - emitSetSavedPc(build, pcpos + 1); - build.mov(rax, sClosure); // luaV_getimport(L, cl->env, k, aux, /* propagatenil= */ false) @@ -1548,14 +1538,15 @@ void emitInstGetTableKS(AssemblyBuilderX64& build, const Instruction* pc, int pc RegisterX64 table = rcx; build.mov(table, luauRegValue(rb)); - RegisterX64 node = getTableNodeAtCachedSlot(build, rax, table, pcpos); + RegisterX64 node = rdx; + getTableNodeAtCachedSlot(build, rax, node, table, pcpos); jumpIfNodeKeyNotInExpectedSlot(build, rax, node, luauConstantValue(aux), fallback); setLuauReg(build, xmm0, ra, luauNodeValue(node)); } -void emitInstSetTableKS(AssemblyBuilderX64& build, const Instruction* pc, int pcpos, Label* labelarr, Label& fallback) +void emitInstSetTableKS(AssemblyBuilderX64& build, const Instruction* pc, int pcpos, Label& next, Label& fallback) { int ra = LUAU_INSN_A(*pc); int rb = LUAU_INSN_B(*pc); @@ -1567,14 +1558,15 @@ void emitInstSetTableKS(AssemblyBuilderX64& build, const Instruction* pc, int pc build.mov(table, luauRegValue(rb)); // fast-path: set value at the expected slot - RegisterX64 node = getTableNodeAtCachedSlot(build, rax, table, pcpos); + RegisterX64 node = rdx; + getTableNodeAtCachedSlot(build, rax, node, table, pcpos); jumpIfNodeKeyNotInExpectedSlot(build, rax, node, luauConstantValue(aux), fallback); jumpIfTableIsReadOnly(build, table, fallback); setNodeValue(build, xmm0, luauNodeValue(node), ra); - callBarrierTable(build, rax, table, ra, labelarr[pcpos + 2]); + callBarrierTable(build, rax, table, ra, next); } void emitInstGetGlobal(AssemblyBuilderX64& build, const Instruction* pc, int pcpos, Label& fallback) @@ -1585,14 +1577,15 @@ void emitInstGetGlobal(AssemblyBuilderX64& build, const Instruction* pc, int pcp RegisterX64 table = rcx; build.mov(rax, sClosure); build.mov(table, qword[rax + offsetof(Closure, env)]); - RegisterX64 node = getTableNodeAtCachedSlot(build, rax, table, pcpos); + RegisterX64 node = rdx; + getTableNodeAtCachedSlot(build, rax, node, table, pcpos); jumpIfNodeKeyNotInExpectedSlot(build, rax, node, luauConstantValue(aux), fallback); setLuauReg(build, xmm0, ra, luauNodeValue(node)); } -void emitInstSetGlobal(AssemblyBuilderX64& build, const Instruction* pc, int pcpos, Label* labelarr, Label& fallback) +void emitInstSetGlobal(AssemblyBuilderX64& build, const Instruction* pc, int pcpos, Label& next, Label& fallback) { int ra = LUAU_INSN_A(*pc); uint32_t aux = pc[1]; @@ -1600,17 +1593,18 @@ void emitInstSetGlobal(AssemblyBuilderX64& build, const Instruction* pc, int pcp RegisterX64 table = rcx; build.mov(rax, sClosure); build.mov(table, qword[rax + offsetof(Closure, env)]); - RegisterX64 node = getTableNodeAtCachedSlot(build, rax, table, pcpos); + RegisterX64 node = rdx; + getTableNodeAtCachedSlot(build, rax, node, table, pcpos); jumpIfNodeKeyNotInExpectedSlot(build, rax, node, luauConstantValue(aux), fallback); jumpIfTableIsReadOnly(build, table, fallback); setNodeValue(build, xmm0, luauNodeValue(node), ra); - callBarrierTable(build, rax, table, ra, labelarr[pcpos + 2]); + callBarrierTable(build, rax, table, ra, next); } -void emitInstConcat(AssemblyBuilderX64& build, const Instruction* pc, int pcpos, Label* labelarr) +void emitInstConcat(AssemblyBuilderX64& build, const Instruction* pc, int pcpos, Label& next) { int ra = LUAU_INSN_A(*pc); int rb = LUAU_INSN_B(*pc); @@ -1630,7 +1624,7 @@ void emitInstConcat(AssemblyBuilderX64& build, const Instruction* pc, int pcpos, build.vmovups(xmm0, luauReg(rb)); build.vmovups(luauReg(ra), xmm0); - callCheckGc(build, pcpos, /* savepc= */ false, labelarr[pcpos + 1]); + callCheckGc(build, pcpos, /* savepc= */ false, next); } } // namespace CodeGen diff --git a/CodeGen/src/EmitInstructionX64.h b/CodeGen/src/EmitInstructionX64.h index ae310aca..1ecb06d4 100644 --- a/CodeGen/src/EmitInstructionX64.h +++ b/CodeGen/src/EmitInstructionX64.h @@ -24,8 +24,8 @@ void emitInstLoadN(AssemblyBuilderX64& build, const Instruction* pc); void emitInstLoadK(AssemblyBuilderX64& build, const Instruction* pc); void emitInstLoadKX(AssemblyBuilderX64& build, const Instruction* pc); void emitInstMove(AssemblyBuilderX64& build, const Instruction* pc); -void emitInstCall(AssemblyBuilderX64& build, ModuleHelpers& helpers, const Instruction* pc, int pcpos, Label* labelarr); -void emitInstReturn(AssemblyBuilderX64& build, ModuleHelpers& helpers, const Instruction* pc, int pcpos, Label* labelarr); +void emitInstCall(AssemblyBuilderX64& build, ModuleHelpers& helpers, const Instruction* pc, int pcpos); +void emitInstReturn(AssemblyBuilderX64& build, ModuleHelpers& helpers, const Instruction* pc, int pcpos); void emitInstJump(AssemblyBuilderX64& build, const Instruction* pc, int pcpos, Label* labelarr); void emitInstJumpBack(AssemblyBuilderX64& build, const Instruction* pc, int pcpos, Label* labelarr); void emitInstJumpIf(AssemblyBuilderX64& build, const Instruction* pc, int pcpos, Label* labelarr, bool not_); @@ -38,52 +38,52 @@ void emitInstJumpxEqNil(AssemblyBuilderX64& build, const Instruction* pc, int pc void emitInstJumpxEqB(AssemblyBuilderX64& build, const Instruction* pc, int pcpos, Label* labelarr); void emitInstJumpxEqN(AssemblyBuilderX64& build, const Instruction* pc, const TValue* k, int pcpos, Label* labelarr); void emitInstJumpxEqS(AssemblyBuilderX64& build, const Instruction* pc, int pcpos, Label* labelarr); -void emitInstBinary(AssemblyBuilderX64& build, const Instruction* pc, int pcpos, TMS tm, Label& fallback); +void emitInstBinary(AssemblyBuilderX64& build, const Instruction* pc, TMS tm, Label& fallback); void emitInstBinaryFallback(AssemblyBuilderX64& build, const Instruction* pc, int pcpos, TMS tm); -void emitInstBinaryK(AssemblyBuilderX64& build, const Instruction* pc, int pcpos, TMS tm, Label& fallback); +void emitInstBinaryK(AssemblyBuilderX64& build, const Instruction* pc, TMS tm, Label& fallback); void emitInstBinaryKFallback(AssemblyBuilderX64& build, const Instruction* pc, int pcpos, TMS tm); -void emitInstPowK(AssemblyBuilderX64& build, const Instruction* pc, const TValue* k, int pcpos, Label& fallback); +void emitInstPowK(AssemblyBuilderX64& build, const Instruction* pc, const TValue* k, Label& fallback); void emitInstNot(AssemblyBuilderX64& build, const Instruction* pc); -void emitInstMinus(AssemblyBuilderX64& build, const Instruction* pc, int pcpos, Label& fallback); +void emitInstMinus(AssemblyBuilderX64& build, const Instruction* pc, Label& fallback); void emitInstMinusFallback(AssemblyBuilderX64& build, const Instruction* pc, int pcpos); -void emitInstLength(AssemblyBuilderX64& build, const Instruction* pc, int pcpos, Label& fallback); +void emitInstLength(AssemblyBuilderX64& build, const Instruction* pc, Label& fallback); void emitInstLengthFallback(AssemblyBuilderX64& build, const Instruction* pc, int pcpos); -void emitInstNewTable(AssemblyBuilderX64& build, const Instruction* pc, int pcpos, Label* labelarr); -void emitInstDupTable(AssemblyBuilderX64& build, const Instruction* pc, int pcpos, Label* labelarr); -void emitInstSetList(AssemblyBuilderX64& build, const Instruction* pc, int pcpos, Label* labelarr); -void emitInstGetUpval(AssemblyBuilderX64& build, const Instruction* pc, int pcpos); -void emitInstSetUpval(AssemblyBuilderX64& build, const Instruction* pc, int pcpos, Label* labelarr); -void emitInstCloseUpvals(AssemblyBuilderX64& build, const Instruction* pc, int pcpos, Label* labelarr); -int emitInstFastCall1(AssemblyBuilderX64& build, const Instruction* pc, int pcpos, Label* labelarr); -int emitInstFastCall2(AssemblyBuilderX64& build, const Instruction* pc, int pcpos, Label* labelarr); -int emitInstFastCall2K(AssemblyBuilderX64& build, const Instruction* pc, int pcpos, Label* labelarr); -int emitInstFastCall(AssemblyBuilderX64& build, const Instruction* pc, int pcpos, Label* labelarr); -void emitInstForNPrep(AssemblyBuilderX64& build, const Instruction* pc, int pcpos, Label* labelarr); -void emitInstForNLoop(AssemblyBuilderX64& build, const Instruction* pc, int pcpos, Label* labelarr); -void emitinstForGLoop(AssemblyBuilderX64& build, const Instruction* pc, int pcpos, Label* labelarr, Label& fallback); -void emitinstForGLoopFallback(AssemblyBuilderX64& build, const Instruction* pc, int pcpos, Label* labelarr); -void emitInstForGPrepNext(AssemblyBuilderX64& build, const Instruction* pc, int pcpos, Label* labelarr, Label& fallback); -void emitInstForGPrepInext(AssemblyBuilderX64& build, const Instruction* pc, int pcpos, Label* labelarr, Label& fallback); -void emitInstForGPrepXnextFallback(AssemblyBuilderX64& build, const Instruction* pc, int pcpos, Label* labelarr); +void emitInstNewTable(AssemblyBuilderX64& build, const Instruction* pc, int pcpos, Label& next); +void emitInstDupTable(AssemblyBuilderX64& build, const Instruction* pc, int pcpos, Label& next); +void emitInstSetList(AssemblyBuilderX64& build, const Instruction* pc, Label& next); +void emitInstGetUpval(AssemblyBuilderX64& build, const Instruction* pc); +void emitInstSetUpval(AssemblyBuilderX64& build, const Instruction* pc, Label& next); +void emitInstCloseUpvals(AssemblyBuilderX64& build, const Instruction* pc, Label& next); +int emitInstFastCall1(AssemblyBuilderX64& build, const Instruction* pc, int pcpos, Label& fallback); +int emitInstFastCall2(AssemblyBuilderX64& build, const Instruction* pc, int pcpos, Label& fallback); +int emitInstFastCall2K(AssemblyBuilderX64& build, const Instruction* pc, int pcpos, Label& fallback); +int emitInstFastCall(AssemblyBuilderX64& build, const Instruction* pc, int pcpos, Label& fallback); +void emitInstForNPrep(AssemblyBuilderX64& build, const Instruction* pc, int pcpos, Label& loopExit); +void emitInstForNLoop(AssemblyBuilderX64& build, const Instruction* pc, int pcpos, Label& loopRepeat); +void emitinstForGLoop(AssemblyBuilderX64& build, const Instruction* pc, int pcpos, Label& loopRepeat, Label& loopExit, Label& fallback); +void emitinstForGLoopFallback(AssemblyBuilderX64& build, const Instruction* pc, int pcpos, Label& loopRepeat); +void emitInstForGPrepNext(AssemblyBuilderX64& build, const Instruction* pc, Label& target, Label& fallback); +void emitInstForGPrepInext(AssemblyBuilderX64& build, const Instruction* pc, Label& target, Label& fallback); +void emitInstForGPrepXnextFallback(AssemblyBuilderX64& build, const Instruction* pc, int pcpos, Label& target); void emitInstAnd(AssemblyBuilderX64& build, const Instruction* pc); void emitInstAndK(AssemblyBuilderX64& build, const Instruction* pc); void emitInstOr(AssemblyBuilderX64& build, const Instruction* pc); void emitInstOrK(AssemblyBuilderX64& build, const Instruction* pc); -void emitInstGetTableN(AssemblyBuilderX64& build, const Instruction* pc, int pcpos, Label& fallback); +void emitInstGetTableN(AssemblyBuilderX64& build, const Instruction* pc, Label& fallback); void emitInstGetTableNFallback(AssemblyBuilderX64& build, const Instruction* pc, int pcpos); -void emitInstSetTableN(AssemblyBuilderX64& build, const Instruction* pc, int pcpos, Label* labelarr, Label& fallback); +void emitInstSetTableN(AssemblyBuilderX64& build, const Instruction* pc, Label& next, Label& fallback); void emitInstSetTableNFallback(AssemblyBuilderX64& build, const Instruction* pc, int pcpos); -void emitInstGetTable(AssemblyBuilderX64& build, const Instruction* pc, int pcpos, Label& fallback); +void emitInstGetTable(AssemblyBuilderX64& build, const Instruction* pc, Label& fallback); void emitInstGetTableFallback(AssemblyBuilderX64& build, const Instruction* pc, int pcpos); -void emitInstSetTable(AssemblyBuilderX64& build, const Instruction* pc, int pcpos, Label* labelarr, Label& fallback); +void emitInstSetTable(AssemblyBuilderX64& build, const Instruction* pc, Label& next, Label& fallback); void emitInstSetTableFallback(AssemblyBuilderX64& build, const Instruction* pc, int pcpos); void emitInstGetImport(AssemblyBuilderX64& build, const Instruction* pc, Label& fallback); -void emitInstGetImportFallback(AssemblyBuilderX64& build, const Instruction* pc, int pcpos); +void emitInstGetImportFallback(AssemblyBuilderX64& build, int ra, uint32_t aux); void emitInstGetTableKS(AssemblyBuilderX64& build, const Instruction* pc, int pcpos, Label& fallback); -void emitInstSetTableKS(AssemblyBuilderX64& build, const Instruction* pc, int pcpos, Label* labelarr, Label& fallback); +void emitInstSetTableKS(AssemblyBuilderX64& build, const Instruction* pc, int pcpos, Label& next, Label& fallback); void emitInstGetGlobal(AssemblyBuilderX64& build, const Instruction* pc, int pcpos, Label& fallback); -void emitInstSetGlobal(AssemblyBuilderX64& build, const Instruction* pc, int pcpos, Label* labelarr, Label& fallback); -void emitInstConcat(AssemblyBuilderX64& build, const Instruction* pc, int pcpos, Label* labelarr); +void emitInstSetGlobal(AssemblyBuilderX64& build, const Instruction* pc, int pcpos, Label& next, Label& fallback); +void emitInstConcat(AssemblyBuilderX64& build, const Instruction* pc, int pcpos, Label& next); } // namespace CodeGen } // namespace Luau diff --git a/tests/AstJsonEncoder.test.cpp b/tests/AstJsonEncoder.test.cpp index a14d5f59..1532b7a8 100644 --- a/tests/AstJsonEncoder.test.cpp +++ b/tests/AstJsonEncoder.test.cpp @@ -6,6 +6,7 @@ #include "doctest.h" +#include #include using namespace Luau; @@ -58,6 +59,9 @@ TEST_CASE("encode_constants") AstExprConstantBool b{Location(), true}; AstExprConstantNumber n{Location(), 8.2}; AstExprConstantNumber bigNum{Location(), 0.1677721600000003}; + AstExprConstantNumber positiveInfinity{Location(), INFINITY}; + AstExprConstantNumber negativeInfinity{Location(), -INFINITY}; + AstExprConstantNumber nan{Location(), NAN}; AstArray charString; charString.data = const_cast("a\x1d\0\\\"b"); @@ -69,6 +73,9 @@ TEST_CASE("encode_constants") CHECK_EQ(R"({"type":"AstExprConstantBool","location":"0,0 - 0,0","value":true})", toJson(&b)); CHECK_EQ(R"({"type":"AstExprConstantNumber","location":"0,0 - 0,0","value":8.1999999999999993})", toJson(&n)); CHECK_EQ(R"({"type":"AstExprConstantNumber","location":"0,0 - 0,0","value":0.16777216000000031})", toJson(&bigNum)); + CHECK_EQ(R"({"type":"AstExprConstantNumber","location":"0,0 - 0,0","value":Infinity})", toJson(&positiveInfinity)); + CHECK_EQ(R"({"type":"AstExprConstantNumber","location":"0,0 - 0,0","value":-Infinity})", toJson(&negativeInfinity)); + CHECK_EQ(R"({"type":"AstExprConstantNumber","location":"0,0 - 0,0","value":NaN})", toJson(&nan)); CHECK_EQ("{\"type\":\"AstExprConstantString\",\"location\":\"0,0 - 0,0\",\"value\":\"a\\u001d\\u0000\\\\\\\"b\"}", toJson(&needsEscaping)); } diff --git a/tests/Autocomplete.test.cpp b/tests/Autocomplete.test.cpp index 45baec2c..123708ca 100644 --- a/tests/Autocomplete.test.cpp +++ b/tests/Autocomplete.test.cpp @@ -2948,6 +2948,71 @@ TEST_CASE_FIXTURE(ACFixture, "autocomplete_string_singletons") CHECK_EQ(ac.context, AutocompleteContext::String); } +TEST_CASE_FIXTURE(ACFixture, "string_singleton_as_table_key") +{ + ScopedFastFlag sff{"LuauCompleteTableKeysBetter", true}; + + check(R"( + type Direction = "up" | "down" + + local a: {[Direction]: boolean} = {[@1] = true} + local b: {[Direction]: boolean} = {["@2"] = true} + local c: {[Direction]: boolean} = {u@3 = true} + local d: {[Direction]: boolean} = {[u@4] = true} + + local e: {[Direction]: boolean} = {[@5]} + local f: {[Direction]: boolean} = {["@6"]} + local g: {[Direction]: boolean} = {u@7} + local h: {[Direction]: boolean} = {[u@8]} + )"); + + auto ac = autocomplete('1'); + + CHECK(ac.entryMap.count("\"up\"")); + CHECK(ac.entryMap.count("\"down\"")); + + ac = autocomplete('2'); + + CHECK(ac.entryMap.count("up")); + CHECK(ac.entryMap.count("down")); + + ac = autocomplete('3'); + + CHECK(ac.entryMap.count("up")); + CHECK(ac.entryMap.count("down")); + + ac = autocomplete('4'); + + CHECK(!ac.entryMap.count("up")); + CHECK(!ac.entryMap.count("down")); + + CHECK(ac.entryMap.count("\"up\"")); + CHECK(ac.entryMap.count("\"down\"")); + + ac = autocomplete('5'); + + CHECK(ac.entryMap.count("\"up\"")); + CHECK(ac.entryMap.count("\"down\"")); + + ac = autocomplete('6'); + + CHECK(ac.entryMap.count("up")); + CHECK(ac.entryMap.count("down")); + + ac = autocomplete('7'); + + CHECK(ac.entryMap.count("up")); + CHECK(ac.entryMap.count("down")); + + ac = autocomplete('8'); + + CHECK(!ac.entryMap.count("up")); + CHECK(!ac.entryMap.count("down")); + + CHECK(ac.entryMap.count("\"up\"")); + CHECK(ac.entryMap.count("\"down\"")); +} + TEST_CASE_FIXTURE(ACFixture, "autocomplete_string_singleton_equality") { check(R"( diff --git a/tests/Fixture.cpp b/tests/Fixture.cpp index b28155e3..eb77ce52 100644 --- a/tests/Fixture.cpp +++ b/tests/Fixture.cpp @@ -2,19 +2,21 @@ #include "Fixture.h" #include "Luau/AstQuery.h" +#include "Luau/BuiltinDefinitions.h" +#include "Luau/Constraint.h" #include "Luau/ModuleResolver.h" #include "Luau/NotNull.h" #include "Luau/Parser.h" #include "Luau/TypeVar.h" #include "Luau/TypeAttach.h" #include "Luau/Transpiler.h" -#include "Luau/BuiltinDefinitions.h" #include "doctest.h" #include #include #include +#include static const char* mainModuleName = "MainModule"; @@ -27,6 +29,41 @@ extern std::optional randomSeed; // tests/main.cpp namespace Luau { +std::optional TestFileResolver::resolveModuleInfo(const ModuleName& currentModuleName, const AstExpr& pathExpr) +{ + if (auto name = pathExprToModuleName(currentModuleName, pathExpr)) + return {{*name, false}}; + + return std::nullopt; +} + +const ModulePtr TestFileResolver::getModule(const ModuleName& moduleName) const +{ + LUAU_ASSERT(false); + return nullptr; +} + +bool TestFileResolver::moduleExists(const ModuleName& moduleName) const +{ + auto it = source.find(moduleName); + return (it != source.end()); +} + +std::optional TestFileResolver::readSource(const ModuleName& name) +{ + auto it = source.find(name); + if (it == source.end()) + return std::nullopt; + + SourceCode::Type sourceType = SourceCode::Module; + + auto it2 = sourceTypes.find(name); + if (it2 != sourceTypes.end()) + sourceType = it2->second; + + return SourceCode{it->second, sourceType}; +} + std::optional TestFileResolver::resolveModule(const ModuleInfo* context, AstExpr* expr) { if (AstExprGlobal* g = expr->as()) @@ -90,6 +127,15 @@ std::optional TestFileResolver::getEnvironmentForModule(const Modul return std::nullopt; } +const Config& TestConfigResolver::getConfig(const ModuleName& name) const +{ + auto it = configFiles.find(name); + if (it != configFiles.end()) + return it->second; + + return defaultConfig; +} + Fixture::Fixture(bool freeze, bool prepareAutocomplete) : sff_DebugLuauFreezeArena("DebugLuauFreezeArena", freeze) , frontend(&fileResolver, &configResolver, diff --git a/tests/Fixture.h b/tests/Fixture.h index 24c9566f..5d838b16 100644 --- a/tests/Fixture.h +++ b/tests/Fixture.h @@ -10,59 +10,31 @@ #include "Luau/ModuleResolver.h" #include "Luau/Scope.h" #include "Luau/ToString.h" -#include "Luau/TypeInfer.h" #include "Luau/TypeVar.h" #include "IostreamOptional.h" #include "ScopedFlags.h" -#include #include #include - #include namespace Luau { +struct TypeChecker; + struct TestFileResolver : FileResolver , ModuleResolver { - std::optional resolveModuleInfo(const ModuleName& currentModuleName, const AstExpr& pathExpr) override - { - if (auto name = pathExprToModuleName(currentModuleName, pathExpr)) - return {{*name, false}}; + std::optional resolveModuleInfo(const ModuleName& currentModuleName, const AstExpr& pathExpr) override; - return std::nullopt; - } + const ModulePtr getModule(const ModuleName& moduleName) const override; - const ModulePtr getModule(const ModuleName& moduleName) const override - { - LUAU_ASSERT(false); - return nullptr; - } + bool moduleExists(const ModuleName& moduleName) const override; - bool moduleExists(const ModuleName& moduleName) const override - { - auto it = source.find(moduleName); - return (it != source.end()); - } - - std::optional readSource(const ModuleName& name) override - { - auto it = source.find(name); - if (it == source.end()) - return std::nullopt; - - SourceCode::Type sourceType = SourceCode::Module; - - auto it2 = sourceTypes.find(name); - if (it2 != sourceTypes.end()) - sourceType = it2->second; - - return SourceCode{it->second, sourceType}; - } + std::optional readSource(const ModuleName& name) override; std::optional resolveModule(const ModuleInfo* context, AstExpr* expr) override; @@ -80,14 +52,7 @@ struct TestConfigResolver : ConfigResolver Config defaultConfig; std::unordered_map configFiles; - const Config& getConfig(const ModuleName& name) const override - { - auto it = configFiles.find(name); - if (it != configFiles.end()) - return it->second; - - return defaultConfig; - } + const Config& getConfig(const ModuleName& name) const override; }; struct Fixture diff --git a/tests/Frontend.test.cpp b/tests/Frontend.test.cpp index 4e72dd4e..6f92b655 100644 --- a/tests/Frontend.test.cpp +++ b/tests/Frontend.test.cpp @@ -519,10 +519,6 @@ TEST_CASE_FIXTURE(FrontendFixture, "recheck_if_dependent_script_is_dirty") TEST_CASE_FIXTURE(FrontendFixture, "mark_non_immediate_reverse_deps_as_dirty") { - ScopedFastFlag sff[] = { - {"LuauFixMarkDirtyReverseDeps", true}, - }; - fileResolver.source["game/Gui/Modules/A"] = "return {hello=5, world=true}"; fileResolver.source["game/Gui/Modules/B"] = R"( return require(game:GetService('Gui').Modules.A) diff --git a/tests/Normalize.test.cpp b/tests/Normalize.test.cpp index a8f3c7ba..e6bf00a1 100644 --- a/tests/Normalize.test.cpp +++ b/tests/Normalize.test.cpp @@ -393,7 +393,6 @@ TEST_SUITE_END(); struct NormalizeFixture : Fixture { - ScopedFastFlag sff0{"LuauNegatedStringSingletons", true}; ScopedFastFlag sff1{"LuauNegatedFunctionTypes", true}; TypeArena arena; diff --git a/tests/ToString.test.cpp b/tests/ToString.test.cpp index 29954dc4..05f49422 100644 --- a/tests/ToString.test.cpp +++ b/tests/ToString.test.cpp @@ -269,9 +269,9 @@ TEST_CASE_FIXTURE(Fixture, "quit_stringifying_type_when_length_is_exceeded") { o.maxTypeLength = 30; CHECK_EQ(toString(requireType("f0"), o), "() -> ()"); - CHECK_EQ(toString(requireType("f1"), o), "(a) -> () -> ()"); - CHECK_EQ(toString(requireType("f2"), o), "(b) -> (a) -> () -> ()"); - CHECK_EQ(toString(requireType("f3"), o), "(c) -> (b) -> (a) -> (... *TRUNCATED*"); + CHECK_EQ(toString(requireType("f1"), o), "(a) -> (() -> ()) | (a & ~(false?))... *TRUNCATED*"); + CHECK_EQ(toString(requireType("f2"), o), "(b) -> ((a) -> (() -> ()) | (a & ~(false?))... *TRUNCATED*"); + CHECK_EQ(toString(requireType("f3"), o), "(c) -> ((b) -> ((a) -> (() -> ()) | (a & ~(false?))... *TRUNCATED*"); } else { @@ -299,9 +299,9 @@ TEST_CASE_FIXTURE(Fixture, "stringifying_type_is_still_capped_when_exhaustive") { o.maxTypeLength = 30; CHECK_EQ(toString(requireType("f0"), o), "() -> ()"); - CHECK_EQ(toString(requireType("f1"), o), "(a) -> () -> ()"); - CHECK_EQ(toString(requireType("f2"), o), "(b) -> (a) -> () -> ()"); - CHECK_EQ(toString(requireType("f3"), o), "(c) -> (b) -> (a) -> (... *TRUNCATED*"); + CHECK_EQ(toString(requireType("f1"), o), "(a) -> (() -> ()) | (a & ~(false?))... *TRUNCATED*"); + CHECK_EQ(toString(requireType("f2"), o), "(b) -> ((a) -> (() -> ()) | (a & ~(false?))... *TRUNCATED*"); + CHECK_EQ(toString(requireType("f3"), o), "(c) -> ((b) -> ((a) -> (() -> ()) | (a & ~(false?))... *TRUNCATED*"); } else { diff --git a/tests/TypeInfer.aliases.test.cpp b/tests/TypeInfer.aliases.test.cpp index 53c54f4f..38e246c8 100644 --- a/tests/TypeInfer.aliases.test.cpp +++ b/tests/TypeInfer.aliases.test.cpp @@ -8,8 +8,8 @@ using namespace Luau; LUAU_FASTFLAG(DebugLuauDeferredConstraintResolution) -LUAU_FASTFLAG(LuauNoMoreGlobalSingletonTypes) LUAU_FASTFLAG(LuauTypeMismatchInvarianceInError) +LUAU_FASTFLAG(LuauNewLibraryTypeNames) TEST_SUITE_BEGIN("TypeAliases"); @@ -525,21 +525,15 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "general_require_multi_assign") TEST_CASE_FIXTURE(BuiltinsFixture, "type_alias_import_mutation") { - ScopedFastFlag luauNewLibraryTypeNames{"LuauNewLibraryTypeNames", true}; - CheckResult result = check("type t10 = typeof(table)"); LUAU_REQUIRE_NO_ERRORS(result); TypeId ty = getGlobalBinding(frontend, "table"); - if (FFlag::LuauNoMoreGlobalSingletonTypes) - { - CHECK_EQ(toString(ty), "typeof(table)"); - } + if (FFlag::LuauNewLibraryTypeNames) + CHECK(toString(ty) == "typeof(table)"); else - { - CHECK_EQ(toString(ty), "table"); - } + CHECK(toString(ty) == "table"); const TableTypeVar* ttv = get(ty); REQUIRE(ttv); diff --git a/tests/TypeInfer.intersectionTypes.test.cpp b/tests/TypeInfer.intersectionTypes.test.cpp index 7d0621a7..188be63c 100644 --- a/tests/TypeInfer.intersectionTypes.test.cpp +++ b/tests/TypeInfer.intersectionTypes.test.cpp @@ -533,7 +533,7 @@ TEST_CASE_FIXTURE(Fixture, "intersection_of_tables_with_never_properties") ScopedFastFlag sffs[]{ {"LuauSubtypeNormalizer", true}, {"LuauTypeNormalization2", true}, - {"LuauUninhabitedSubAnything", true}, + {"LuauUninhabitedSubAnything2", true}, }; CheckResult result = check(R"( diff --git a/tests/TypeInfer.negations.test.cpp b/tests/TypeInfer.negations.test.cpp index e8256f97..0e7fb03d 100644 --- a/tests/TypeInfer.negations.test.cpp +++ b/tests/TypeInfer.negations.test.cpp @@ -13,8 +13,7 @@ namespace struct NegationFixture : Fixture { TypeArena arena; - ScopedFastFlag sff[2]{ - {"LuauNegatedStringSingletons", true}, + ScopedFastFlag sff[1]{ {"LuauSubtypeNormalizer", true}, }; diff --git a/tests/TypeInfer.operators.test.cpp b/tests/TypeInfer.operators.test.cpp index 21806082..93d7361b 100644 --- a/tests/TypeInfer.operators.test.cpp +++ b/tests/TypeInfer.operators.test.cpp @@ -25,8 +25,16 @@ TEST_CASE_FIXTURE(Fixture, "or_joins_types") local x:string|number = s )"); LUAU_REQUIRE_NO_ERRORS(result); - CHECK_EQ(toString(*requireType("s")), "number | string"); - CHECK_EQ(toString(*requireType("x")), "number | string"); + if (FFlag::DebugLuauDeferredConstraintResolution) + { + CHECK_EQ(toString(*requireType("s")), "(string & ~(false?)) | number"); + CHECK_EQ(toString(*requireType("x")), "number | string"); + } + else + { + CHECK_EQ(toString(*requireType("s")), "number | string"); + CHECK_EQ(toString(*requireType("x")), "number | string"); + } } TEST_CASE_FIXTURE(Fixture, "or_joins_types_with_no_extras") @@ -37,8 +45,16 @@ TEST_CASE_FIXTURE(Fixture, "or_joins_types_with_no_extras") local y = x or "s" )"); LUAU_REQUIRE_NO_ERRORS(result); - CHECK_EQ(toString(*requireType("s")), "number | string"); - CHECK_EQ(toString(*requireType("y")), "number | string"); + if (FFlag::DebugLuauDeferredConstraintResolution) + { + CHECK_EQ(toString(*requireType("s")), "(string & ~(false?)) | number"); + CHECK_EQ(toString(*requireType("y")), "((number | string) & ~(false?)) | string"); + } + else + { + CHECK_EQ(toString(*requireType("s")), "number | string"); + CHECK_EQ(toString(*requireType("y")), "number | string"); + } } TEST_CASE_FIXTURE(Fixture, "or_joins_types_with_no_superfluous_union") @@ -62,7 +78,14 @@ TEST_CASE_FIXTURE(Fixture, "and_does_not_always_add_boolean") local x:boolean|number = s )"); LUAU_REQUIRE_NO_ERRORS(result); - CHECK_EQ(toString(*requireType("s")), "number"); + if (FFlag::DebugLuauDeferredConstraintResolution) + { + CHECK_EQ(toString(*requireType("s")), "((false?) & string) | number"); + } + else + { + CHECK_EQ(toString(*requireType("s")), "number"); + } } TEST_CASE_FIXTURE(Fixture, "and_adds_boolean_no_superfluous_union") @@ -81,7 +104,14 @@ TEST_CASE_FIXTURE(Fixture, "and_or_ternary") local s = (1/2) > 0.5 and "a" or 10 )"); LUAU_REQUIRE_NO_ERRORS(result); - CHECK_EQ(toString(*requireType("s")), "number | string"); + if (FFlag::DebugLuauDeferredConstraintResolution) + { + CHECK_EQ(toString(*requireType("s")), "((((false?) & boolean) | string) & ~(false?)) | number"); + } + else + { + CHECK_EQ(toString(*requireType("s")), "number | string"); + } } TEST_CASE_FIXTURE(BuiltinsFixture, "primitive_arith_no_metatable") @@ -405,11 +435,9 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "compound_assign_mismatch_metatable") local v2: V2 = setmetatable({ x = 3, y = 4 }, VMT) v1 %= v2 )"); - LUAU_REQUIRE_ERROR_COUNT(1, result); - TypeMismatch* tm = get(result.errors[0]); - CHECK_EQ(*tm->wantedType, *requireType("v2")); - CHECK_EQ(*tm->givenType, *typeChecker.numberType); + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK("Type 'number' could not be converted into 'V2'" == toString(result.errors[0])); } TEST_CASE_FIXTURE(Fixture, "CallOrOfFunctions") @@ -781,7 +809,14 @@ local b: number = 1 or a TypeMismatch* tm = get(result.errors[0]); REQUIRE(tm); CHECK_EQ(typeChecker.numberType, tm->wantedType); - CHECK_EQ("number?", toString(tm->givenType)); + if (FFlag::DebugLuauDeferredConstraintResolution) + { + CHECK_EQ("((number & ~(false?)) | number)?", toString(tm->givenType)); + } + else + { + CHECK_EQ("number?", toString(tm->givenType)); + } } TEST_CASE_FIXTURE(Fixture, "operator_eq_verifies_types_do_intersect") @@ -842,7 +877,14 @@ TEST_CASE_FIXTURE(Fixture, "refine_and_or") LUAU_REQUIRE_NO_ERRORS(result); - CHECK_EQ("number", toString(requireType("u"))); + if (FFlag::DebugLuauDeferredConstraintResolution) + { + CHECK_EQ("((((false?) & ({| x: number? |}?)) | a) & ~(false?)) | number", toString(requireType("u"))); + } + else + { + CHECK_EQ("number", toString(requireType("u"))); + } } TEST_CASE_FIXTURE(Fixture, "infer_any_in_all_modes_when_lhs_is_unknown") @@ -1035,10 +1077,10 @@ local w = c and 1 if (FFlag::DebugLuauDeferredConstraintResolution) { - CHECK("number?" == toString(requireType("x"))); - CHECK("number" == toString(requireType("y"))); - CHECK("false | number" == toString(requireType("z"))); - CHECK("number" == toString(requireType("w"))); // Normalizer considers free & falsy == never + CHECK("((false?) & (number?)) | number" == toString(requireType("x"))); + CHECK("((false?) & string) | number" == toString(requireType("y"))); + CHECK("((false?) & boolean) | number" == toString(requireType("z"))); + CHECK("((false?) & a) | number" == toString(requireType("w"))); } else { @@ -1073,12 +1115,12 @@ local f1 = f or 'f' if (FFlag::DebugLuauDeferredConstraintResolution) { - CHECK("number | string" == toString(requireType("a1"))); - CHECK("number" == toString(requireType("b1"))); - CHECK("string | true" == toString(requireType("c1"))); - CHECK("string | true" == toString(requireType("d1"))); - CHECK("string" == toString(requireType("e1"))); - CHECK("string" == toString(requireType("f1"))); + CHECK("((false | number) & ~(false?)) | string" == toString(requireType("a1"))); + CHECK("((number?) & ~(false?)) | number" == toString(requireType("b1"))); + CHECK("(boolean & ~(false?)) | string" == toString(requireType("c1"))); + CHECK("(true & ~(false?)) | string" == toString(requireType("d1"))); + CHECK("(false & ~(false?)) | string" == toString(requireType("e1"))); + CHECK("(nil & ~(false?)) | string" == toString(requireType("f1"))); } else { diff --git a/tests/TypeInfer.provisional.test.cpp b/tests/TypeInfer.provisional.test.cpp index 25934174..b7408f87 100644 --- a/tests/TypeInfer.provisional.test.cpp +++ b/tests/TypeInfer.provisional.test.cpp @@ -9,6 +9,8 @@ using namespace Luau; +LUAU_FASTFLAG(LuauTypeMismatchInvarianceInError) + TEST_SUITE_BEGIN("ProvisionalTests"); // These tests check for behavior that differs from the final behavior we'd @@ -776,4 +778,32 @@ TEST_CASE_FIXTURE(IsSubtypeFixture, "functions_with_mismatching_arity_but_any_is // CHECK(!isSubtype(b, c)); } +TEST_CASE_FIXTURE(Fixture, "assign_table_with_refined_property_with_a_similar_type_is_illegal") +{ + CheckResult result = check(R"( + local t: {x: number?} = {x = nil} + + if t.x then + local u: {x: number} = t + end + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + if (FFlag::LuauTypeMismatchInvarianceInError) + { + CHECK_EQ(R"(Type '{| x: number? |}' could not be converted into '{| x: number |}' +caused by: + Property 'x' is not compatible. Type 'number?' could not be converted into 'number' in an invariant context)", + toString(result.errors[0])); + } + else + { + CHECK_EQ(R"(Type '{| x: number? |}' could not be converted into '{| x: number |}' +caused by: + Property 'x' is not compatible. Type 'number?' could not be converted into 'number')", + toString(result.errors[0])); + } +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.refinements.test.cpp b/tests/TypeInfer.refinements.test.cpp index e5bc186a..5a7c8432 100644 --- a/tests/TypeInfer.refinements.test.cpp +++ b/tests/TypeInfer.refinements.test.cpp @@ -8,7 +8,6 @@ #include "doctest.h" LUAU_FASTFLAG(DebugLuauDeferredConstraintResolution) -LUAU_FASTFLAG(LuauTypeMismatchInvarianceInError) using namespace Luau; @@ -36,7 +35,7 @@ std::optional> magicFunctionInstanceIsA( return WithPredicate{booleanPack, {IsAPredicate{std::move(*lvalue), expr.location, tfun->type}}}; } -std::vector dcrMagicRefinementInstanceIsA(MagicRefinementContext ctx) +std::vector dcrMagicRefinementInstanceIsA(const MagicRefinementContext& ctx) { if (ctx.callSite->args.size != 1) return {}; @@ -462,35 +461,6 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "assert_non_binary_expressions_actually_resol LUAU_REQUIRE_NO_ERRORS(result); } -TEST_CASE_FIXTURE(Fixture, "assign_table_with_refined_property_with_a_similar_type_is_illegal") -{ - CheckResult result = check(R"( - local t: {x: number?} = {x = nil} - - if t.x then - local u: {x: number} = t - end - )"); - - LUAU_REQUIRE_ERROR_COUNT(1, result); - - if (FFlag::LuauTypeMismatchInvarianceInError) - { - CHECK_EQ(R"(Type '{| x: number? |}' could not be converted into '{| x: number |}' -caused by: - Property 'x' is not compatible. Type 'number?' could not be converted into 'number' in an invariant context)", - toString(result.errors[0])); - } - else - { - CHECK_EQ(R"(Type '{| x: number? |}' could not be converted into '{| x: number |}' -caused by: - Property 'x' is not compatible. Type 'number?' could not be converted into 'number')", - toString(result.errors[0])); - } -} - - TEST_CASE_FIXTURE(Fixture, "lvalue_is_equal_to_another_lvalue") { CheckResult result = check(R"( @@ -1009,8 +979,16 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "assert_a_to_be_truthy_then_assert_a_to_be_nu LUAU_REQUIRE_NO_ERRORS(result); - CHECK_EQ("number | string", toString(requireTypeAtPosition({3, 18}))); - CHECK_EQ("number", toString(requireTypeAtPosition({5, 18}))); + if (FFlag::DebugLuauDeferredConstraintResolution) + { + CHECK_EQ("((number | string)?) & ~(false?)", toString(requireTypeAtPosition({3, 18}))); + CHECK_EQ("((number | string)?) & ~(false?) & number", toString(requireTypeAtPosition({5, 18}))); + } + else + { + CHECK_EQ("number | string", toString(requireTypeAtPosition({3, 18}))); + CHECK_EQ("number", toString(requireTypeAtPosition({5, 18}))); + } } TEST_CASE_FIXTURE(BuiltinsFixture, "merge_should_be_fully_agnostic_of_hashmap_ordering") @@ -1031,7 +1009,7 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "merge_should_be_fully_agnostic_of_hashmap_or if (FFlag::DebugLuauDeferredConstraintResolution) { - CHECK_EQ("(string | {| x: string |}) & string", toString(requireTypeAtPosition({6, 28}))); + CHECK_EQ("(never | string) & (string | {| x: string |}) & string", toString(requireTypeAtPosition({6, 28}))); } else { @@ -1075,8 +1053,16 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "is_truthy_constraint_ifelse_expression") LUAU_REQUIRE_NO_ERRORS(result); - CHECK_EQ("string", toString(requireTypeAtPosition({2, 29}))); - CHECK_EQ("nil", toString(requireTypeAtPosition({2, 45}))); + if (FFlag::DebugLuauDeferredConstraintResolution) + { + CHECK_EQ("(string?) & ~(false?)", toString(requireTypeAtPosition({2, 29}))); + CHECK_EQ("(string?) & ~~(false?)", toString(requireTypeAtPosition({2, 45}))); + } + else + { + CHECK_EQ("string", toString(requireTypeAtPosition({2, 29}))); + CHECK_EQ("nil", toString(requireTypeAtPosition({2, 45}))); + } } TEST_CASE_FIXTURE(BuiltinsFixture, "invert_is_truthy_constraint_ifelse_expression") @@ -1089,8 +1075,16 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "invert_is_truthy_constraint_ifelse_expressio LUAU_REQUIRE_NO_ERRORS(result); - CHECK_EQ("nil", toString(requireTypeAtPosition({2, 42}))); - CHECK_EQ("string", toString(requireTypeAtPosition({2, 50}))); + if (FFlag::DebugLuauDeferredConstraintResolution) + { + CHECK_EQ("(string?) & ~~(false?)", toString(requireTypeAtPosition({2, 42}))); + CHECK_EQ("(string?) & ~(false?)", toString(requireTypeAtPosition({2, 50}))); + } + else + { + CHECK_EQ("nil", toString(requireTypeAtPosition({2, 42}))); + CHECK_EQ("string", toString(requireTypeAtPosition({2, 50}))); + } } TEST_CASE_FIXTURE(BuiltinsFixture, "type_comparison_ifelse_expression") @@ -1107,8 +1101,16 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "type_comparison_ifelse_expression") LUAU_REQUIRE_NO_ERRORS(result); - CHECK_EQ("number", toString(requireTypeAtPosition({6, 49}))); - CHECK_EQ("any", toString(requireTypeAtPosition({6, 66}))); + if (FFlag::DebugLuauDeferredConstraintResolution) + { + CHECK_EQ("any & number", toString(requireTypeAtPosition({6, 49}))); + CHECK_EQ("any & ~number", toString(requireTypeAtPosition({6, 66}))); + } + else + { + CHECK_EQ("number", toString(requireTypeAtPosition({6, 49}))); + CHECK_EQ("any", toString(requireTypeAtPosition({6, 66}))); + } } TEST_CASE_FIXTURE(BuiltinsFixture, "correctly_lookup_a_shadowed_local_that_which_was_previously_refined") diff --git a/tests/TypeInfer.tables.test.cpp b/tests/TypeInfer.tables.test.cpp index c94ed1f9..c379559d 100644 --- a/tests/TypeInfer.tables.test.cpp +++ b/tests/TypeInfer.tables.test.cpp @@ -17,8 +17,8 @@ using namespace Luau; LUAU_FASTFLAG(LuauLowerBoundsCalculation); LUAU_FASTFLAG(DebugLuauDeferredConstraintResolution); LUAU_FASTFLAG(LuauInstantiateInSubtyping) -LUAU_FASTFLAG(LuauNoMoreGlobalSingletonTypes) LUAU_FASTFLAG(LuauTypeMismatchInvarianceInError) +LUAU_FASTFLAG(LuauNewLibraryTypeNames) TEST_SUITE_BEGIN("TableTests"); @@ -1723,8 +1723,6 @@ TEST_CASE_FIXTURE(Fixture, "hide_table_error_properties") TEST_CASE_FIXTURE(BuiltinsFixture, "builtin_table_names") { - ScopedFastFlag luauNewLibraryTypeNames{"LuauNewLibraryTypeNames", true}; - CheckResult result = check(R"( os.h = 2 string.k = 3 @@ -1732,7 +1730,7 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "builtin_table_names") LUAU_REQUIRE_ERROR_COUNT(2, result); - if (FFlag::LuauNoMoreGlobalSingletonTypes) + if (FFlag::LuauNewLibraryTypeNames) { CHECK_EQ("Cannot add property 'h' to table 'typeof(os)'", toString(result.errors[0])); CHECK_EQ("Cannot add property 'k' to table 'typeof(string)'", toString(result.errors[1])); @@ -1746,22 +1744,16 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "builtin_table_names") TEST_CASE_FIXTURE(BuiltinsFixture, "persistent_sealed_table_is_immutable") { - ScopedFastFlag luauNewLibraryTypeNames{"LuauNewLibraryTypeNames", true}; - CheckResult result = check(R"( --!nonstrict function os:bad() end )"); LUAU_REQUIRE_ERROR_COUNT(1, result); - if (FFlag::LuauNoMoreGlobalSingletonTypes) - { + if (FFlag::LuauNewLibraryTypeNames) CHECK_EQ("Cannot add property 'bad' to table 'typeof(os)'", toString(result.errors[0])); - } else - { CHECK_EQ("Cannot add property 'bad' to table 'os'", toString(result.errors[0])); - } const TableTypeVar* osType = get(requireType("os")); REQUIRE(osType != nullptr); @@ -3238,7 +3230,8 @@ TEST_CASE_FIXTURE(Fixture, "scalar_is_a_subtype_of_a_compatible_polymorphic_shap TEST_CASE_FIXTURE(Fixture, "scalar_is_not_a_subtype_of_a_compatible_polymorphic_shape_type") { ScopedFastFlag sff{"LuauScalarShapeSubtyping", true}; - ScopedFastFlag luauNewLibraryTypeNames{"LuauNewLibraryTypeNames", true}; + if (!FFlag::LuauNewLibraryTypeNames) + return; CheckResult result = check(R"( local function f(s) @@ -3252,40 +3245,20 @@ TEST_CASE_FIXTURE(Fixture, "scalar_is_not_a_subtype_of_a_compatible_polymorphic_ LUAU_REQUIRE_ERROR_COUNT(3, result); - if (FFlag::LuauNoMoreGlobalSingletonTypes) - { - CHECK_EQ(R"(Type 'string' could not be converted into 't1 where t1 = {- absolutely_no_scalar_has_this_method: (t1) -> (a...) -}' + CHECK_EQ(R"(Type 'string' could not be converted into 't1 where t1 = {- absolutely_no_scalar_has_this_method: (t1) -> (a...) -}' caused by: The former's metatable does not satisfy the requirements. Table type 'typeof(string)' not compatible with type 't1 where t1 = {- absolutely_no_scalar_has_this_method: (t1) -> (a...) -}' because the former is missing field 'absolutely_no_scalar_has_this_method')", - toString(result.errors[0])); - CHECK_EQ(R"(Type '"bar"' could not be converted into 't1 where t1 = {- absolutely_no_scalar_has_this_method: (t1) -> (a...) -}' + toString(result.errors[0])); + CHECK_EQ(R"(Type '"bar"' could not be converted into 't1 where t1 = {- absolutely_no_scalar_has_this_method: (t1) -> (a...) -}' caused by: The former's metatable does not satisfy the requirements. Table type 'typeof(string)' not compatible with type 't1 where t1 = {- absolutely_no_scalar_has_this_method: (t1) -> (a...) -}' because the former is missing field 'absolutely_no_scalar_has_this_method')", - toString(result.errors[1])); - CHECK_EQ(R"(Type '"bar" | "baz"' could not be converted into 't1 where t1 = {- absolutely_no_scalar_has_this_method: (t1) -> (a...) -}' + toString(result.errors[1])); + CHECK_EQ(R"(Type '"bar" | "baz"' could not be converted into 't1 where t1 = {- absolutely_no_scalar_has_this_method: (t1) -> (a...) -}' caused by: Not all union options are compatible. Type '"bar"' could not be converted into 't1 where t1 = {- absolutely_no_scalar_has_this_method: (t1) -> (a...) -}' caused by: The former's metatable does not satisfy the requirements. Table type 'typeof(string)' not compatible with type 't1 where t1 = {- absolutely_no_scalar_has_this_method: (t1) -> (a...) -}' because the former is missing field 'absolutely_no_scalar_has_this_method')", - toString(result.errors[2])); - } - else - { - CHECK_EQ(R"(Type 'string' could not be converted into 't1 where t1 = {- absolutely_no_scalar_has_this_method: (t1) -> (a...) -}' -caused by: - The former's metatable does not satisfy the requirements. Table type 'string' not compatible with type 't1 where t1 = {- absolutely_no_scalar_has_this_method: (t1) -> (a...) -}' because the former is missing field 'absolutely_no_scalar_has_this_method')", - toString(result.errors[0])); - CHECK_EQ(R"(Type '"bar"' could not be converted into 't1 where t1 = {- absolutely_no_scalar_has_this_method: (t1) -> (a...) -}' -caused by: - The former's metatable does not satisfy the requirements. Table type 'string' not compatible with type 't1 where t1 = {- absolutely_no_scalar_has_this_method: (t1) -> (a...) -}' because the former is missing field 'absolutely_no_scalar_has_this_method')", - toString(result.errors[1])); - CHECK_EQ(R"(Type '"bar" | "baz"' could not be converted into 't1 where t1 = {- absolutely_no_scalar_has_this_method: (t1) -> (a...) -}' -caused by: - Not all union options are compatible. Type '"bar"' could not be converted into 't1 where t1 = {- absolutely_no_scalar_has_this_method: (t1) -> (a...) -}' -caused by: - The former's metatable does not satisfy the requirements. Table type 'string' not compatible with type 't1 where t1 = {- absolutely_no_scalar_has_this_method: (t1) -> (a...) -}' because the former is missing field 'absolutely_no_scalar_has_this_method')", - toString(result.errors[2])); - } + toString(result.errors[2])); } TEST_CASE_FIXTURE(Fixture, "a_free_shape_can_turn_into_a_scalar_if_it_is_compatible") @@ -3307,7 +3280,8 @@ TEST_CASE_FIXTURE(Fixture, "a_free_shape_can_turn_into_a_scalar_if_it_is_compati TEST_CASE_FIXTURE(Fixture, "a_free_shape_cannot_turn_into_a_scalar_if_it_is_not_compatible") { ScopedFastFlag sff{"LuauScalarShapeSubtyping", true}; - ScopedFastFlag luauNewLibraryTypeNames{"LuauNewLibraryTypeNames", true}; + if (!FFlag::LuauNewLibraryTypeNames) + return; CheckResult result = check(R"( local function f(s): string @@ -3317,22 +3291,11 @@ TEST_CASE_FIXTURE(Fixture, "a_free_shape_cannot_turn_into_a_scalar_if_it_is_not_ )"); LUAU_REQUIRE_ERROR_COUNT(1, result); - if (FFlag::LuauNoMoreGlobalSingletonTypes) - { - CHECK_EQ(R"(Type 't1 where t1 = {+ absolutely_no_scalar_has_this_method: (t1) -> (a, b...) +}' could not be converted into 'string' + CHECK_EQ(R"(Type 't1 where t1 = {+ absolutely_no_scalar_has_this_method: (t1) -> (a, b...) +}' could not be converted into 'string' caused by: The former's metatable does not satisfy the requirements. Table type 'typeof(string)' not compatible with type 't1 where t1 = {+ absolutely_no_scalar_has_this_method: (t1) -> (a, b...) +}' because the former is missing field 'absolutely_no_scalar_has_this_method')", - toString(result.errors[0])); - CHECK_EQ("(t1) -> string where t1 = {+ absolutely_no_scalar_has_this_method: (t1) -> (a, b...) +}", toString(requireType("f"))); - } - else - { - CHECK_EQ(R"(Type 't1 where t1 = {+ absolutely_no_scalar_has_this_method: (t1) -> (a, b...) +}' could not be converted into 'string' -caused by: - The former's metatable does not satisfy the requirements. Table type 'string' not compatible with type 't1 where t1 = {+ absolutely_no_scalar_has_this_method: (t1) -> (a, b...) +}' because the former is missing field 'absolutely_no_scalar_has_this_method')", - toString(result.errors[0])); - CHECK_EQ("(t1) -> string where t1 = {+ absolutely_no_scalar_has_this_method: (t1) -> (a, b...) +}", toString(requireType("f"))); - } + toString(result.errors[0])); + CHECK_EQ("(t1) -> string where t1 = {+ absolutely_no_scalar_has_this_method: (t1) -> (a, b...) +}", toString(requireType("f"))); } TEST_CASE_FIXTURE(BuiltinsFixture, "a_free_shape_can_turn_into_a_scalar_directly") diff --git a/tests/TypeInfer.tryUnify.test.cpp b/tests/TypeInfer.tryUnify.test.cpp index 5cc07a28..b1abdf7c 100644 --- a/tests/TypeInfer.tryUnify.test.cpp +++ b/tests/TypeInfer.tryUnify.test.cpp @@ -145,7 +145,7 @@ TEST_CASE_FIXTURE(TryUnifyFixture, "uninhabited_table_sub_never") ScopedFastFlag sffs[]{ {"LuauSubtypeNormalizer", true}, {"LuauTypeNormalization2", true}, - {"LuauUninhabitedSubAnything", true}, + {"LuauUninhabitedSubAnything2", true}, }; CheckResult result = check(R"( @@ -161,7 +161,7 @@ TEST_CASE_FIXTURE(TryUnifyFixture, "uninhabited_table_sub_anything") ScopedFastFlag sffs[]{ {"LuauSubtypeNormalizer", true}, {"LuauTypeNormalization2", true}, - {"LuauUninhabitedSubAnything", true}, + {"LuauUninhabitedSubAnything2", true}, }; CheckResult result = check(R"( diff --git a/tools/faillist.txt b/tools/faillist.txt index 6f49db84..5d6779f4 100644 --- a/tools/faillist.txt +++ b/tools/faillist.txt @@ -27,6 +27,7 @@ AutocompleteTest.do_wrong_compatible_self_calls AutocompleteTest.keyword_methods AutocompleteTest.no_incompatible_self_calls AutocompleteTest.no_wrong_compatible_self_calls_with_generics +AutocompleteTest.string_singleton_as_table_key AutocompleteTest.suggest_external_module_type AutocompleteTest.suggest_table_keys AutocompleteTest.type_correct_argument_type_suggestion @@ -88,7 +89,6 @@ DefinitionTests.class_definition_string_props DefinitionTests.declaring_generic_functions DefinitionTests.definition_file_classes FrontendTest.environments -FrontendTest.imported_table_modification_2 FrontendTest.it_should_be_safe_to_stringify_errors_when_full_type_graph_is_discarded FrontendTest.nocheck_cycle_used_by_checked FrontendTest.reexport_cyclic_type @@ -96,7 +96,6 @@ FrontendTest.trace_requires_in_nonstrict_mode GenericsTests.apply_type_function_nested_generics1 GenericsTests.apply_type_function_nested_generics2 GenericsTests.better_mismatch_error_messages -GenericsTests.calling_self_generic_methods GenericsTests.check_generic_typepack_function GenericsTests.check_mutual_generic_functions GenericsTests.correctly_instantiate_polymorphic_member_functions @@ -113,7 +112,6 @@ GenericsTests.higher_rank_polymorphism_should_not_accept_instantiated_arguments GenericsTests.infer_generic_function_function_argument GenericsTests.infer_generic_function_function_argument_overloaded GenericsTests.infer_generic_lib_function_function_argument -GenericsTests.infer_generic_methods GenericsTests.infer_generic_property GenericsTests.instantiated_function_argument_names GenericsTests.instantiation_sharing_types @@ -147,6 +145,7 @@ ParseErrorRecovery.generic_type_list_recovery ParseErrorRecovery.recovery_of_parenthesized_expressions ParserTests.parse_nesting_based_end_detection_failsafe_earlier ParserTests.parse_nesting_based_end_detection_local_function +ProvisionalTests.assign_table_with_refined_property_with_a_similar_type_is_illegal ProvisionalTests.bail_early_if_unification_is_too_complicated ProvisionalTests.discriminate_from_x_not_equal_to_nil ProvisionalTests.do_not_ice_when_trying_to_pick_first_of_generic_type_pack @@ -163,26 +162,16 @@ ProvisionalTests.typeguard_inference_incomplete ProvisionalTests.weirditer_should_not_loop_forever ProvisionalTests.while_body_are_also_refined RefinementTest.apply_refinements_on_astexprindexexpr_whose_subscript_expr_is_constant_string -RefinementTest.assert_a_to_be_truthy_then_assert_a_to_be_number -RefinementTest.assert_non_binary_expressions_actually_resolve_constraints -RefinementTest.assign_table_with_refined_property_with_a_similar_type_is_illegal RefinementTest.call_an_incompatible_function_after_using_typeguard -RefinementTest.correctly_lookup_property_whose_base_was_previously_refined RefinementTest.correctly_lookup_property_whose_base_was_previously_refined2 RefinementTest.discriminate_on_properties_of_disjoint_tables_where_that_property_is_true_or_false RefinementTest.discriminate_tag RefinementTest.else_with_no_explicit_expression_should_also_refine_the_tagged_union RefinementTest.falsiness_of_TruthyPredicate_narrows_into_nil -RefinementTest.fuzz_filtered_refined_types_are_followed -RefinementTest.index_on_a_refined_property -RefinementTest.invert_is_truthy_constraint_ifelse_expression -RefinementTest.is_truthy_constraint_ifelse_expression RefinementTest.narrow_property_of_a_bounded_variable RefinementTest.nonoptional_type_can_narrow_to_nil_if_sense_is_true -RefinementTest.not_t_or_some_prop_of_t RefinementTest.refine_a_property_not_to_be_nil_through_an_intersection_table RefinementTest.refine_unknowns -RefinementTest.type_comparison_ifelse_expression RefinementTest.type_guard_can_filter_for_intersection_of_tables RefinementTest.type_guard_narrowed_into_nothingness RefinementTest.type_narrow_for_all_the_userdata @@ -199,18 +188,18 @@ TableTests.access_index_metamethod_that_returns_variadic TableTests.accidentally_checked_prop_in_opposite_branch TableTests.builtin_table_names TableTests.call_method +TableTests.call_method_with_explicit_self_argument TableTests.cannot_augment_sealed_table TableTests.casting_sealed_tables_with_props_into_table_with_indexer TableTests.casting_tables_with_props_into_table_with_indexer3 TableTests.casting_tables_with_props_into_table_with_indexer4 TableTests.checked_prop_too_early -TableTests.defining_a_method_for_a_builtin_sealed_table_must_fail -TableTests.defining_a_method_for_a_local_sealed_table_must_fail -TableTests.defining_a_self_method_for_a_builtin_sealed_table_must_fail -TableTests.defining_a_self_method_for_a_local_sealed_table_must_fail +TableTests.defining_a_method_for_a_local_unsealed_table_is_ok +TableTests.defining_a_self_method_for_a_local_unsealed_table_is_ok TableTests.dont_crash_when_setmetatable_does_not_produce_a_metatabletypevar TableTests.dont_hang_when_trying_to_look_up_in_cyclic_metatable_index TableTests.dont_quantify_table_that_belongs_to_outer_scope +TableTests.dont_seal_an_unsealed_table_by_passing_it_to_a_function_that_takes_a_sealed_table TableTests.dont_suggest_exact_match_keys TableTests.error_detailed_metatable_prop TableTests.expected_indexer_from_table_union @@ -235,12 +224,11 @@ TableTests.infer_indexer_from_value_property_in_literal TableTests.inferred_return_type_of_free_table TableTests.inferring_crazy_table_should_also_be_quick TableTests.instantiate_table_cloning_3 +TableTests.instantiate_tables_at_scope_level TableTests.invariant_table_properties_means_instantiating_tables_in_assignment_is_unsound TableTests.invariant_table_properties_means_instantiating_tables_in_call_is_unsound TableTests.leaking_bad_metatable_errors TableTests.less_exponential_blowup_please -TableTests.meta_add -TableTests.meta_add_both_ways TableTests.meta_add_inferred TableTests.metatable_mismatch_should_fail TableTests.missing_metatable_for_sealed_tables_do_not_get_inferred @@ -253,7 +241,6 @@ TableTests.oop_indexer_works TableTests.oop_polymorphic TableTests.open_table_unification_2 TableTests.persistent_sealed_table_is_immutable -TableTests.prop_access_on_key_whose_types_mismatches TableTests.property_lookup_through_tabletypevar_metatable TableTests.quantify_even_that_table_was_never_exported_at_all TableTests.quantify_metatables_of_metatables_of_table @@ -267,6 +254,7 @@ TableTests.shared_selfs TableTests.shared_selfs_from_free_param TableTests.shared_selfs_through_metatables TableTests.table_call_metamethod_basic +TableTests.table_function_check_use_after_free TableTests.table_indexing_error_location TableTests.table_insert_should_cope_with_optional_properties_in_nonstrict TableTests.table_insert_should_cope_with_optional_properties_in_strict @@ -279,13 +267,17 @@ TableTests.tables_get_names_from_their_locals TableTests.tc_member_function TableTests.tc_member_function_2 TableTests.unification_of_unions_in_a_self_referential_type +TableTests.unifying_tables_shouldnt_uaf1 TableTests.unifying_tables_shouldnt_uaf2 +TableTests.used_colon_correctly TableTests.used_colon_instead_of_dot TableTests.used_dot_instead_of_colon +TableTests.used_dot_instead_of_colon_but_correctly ToDot.bound_table ToDot.function ToDot.table ToString.exhaustive_toString_of_cyclic_table +ToString.function_type_with_argument_names_and_self ToString.function_type_with_argument_names_generic ToString.toStringDetailed2 ToString.toStringErrorPack @@ -303,6 +295,7 @@ TryUnifyTests.typepack_unification_should_trim_free_tails TryUnifyTests.variadics_should_use_reversed_properly TypeAliases.cannot_create_cyclic_type_with_unknown_module TypeAliases.forward_declared_alias_is_not_clobbered_by_prior_unification_with_any +TypeAliases.forward_declared_alias_is_not_clobbered_by_prior_unification_with_any_2 TypeAliases.generic_param_remap TypeAliases.mismatched_generic_type_param TypeAliases.mutually_recursive_types_restriction_not_ok_1 @@ -322,6 +315,7 @@ TypeInfer.checking_should_not_ice TypeInfer.cli_50041_committing_txnlog_in_apollo_client_error TypeInfer.dont_report_type_errors_within_an_AstExprError TypeInfer.dont_report_type_errors_within_an_AstStatError +TypeInfer.follow_on_new_types_in_substitution TypeInfer.fuzz_free_table_type_change_during_index_check TypeInfer.globals TypeInfer.globals2 @@ -335,11 +329,13 @@ TypeInfer.tc_interpolated_string_with_invalid_expression TypeInfer.type_infer_recursion_limit_no_ice TypeInfer.type_infer_recursion_limit_normalizer TypeInferAnyError.for_in_loop_iterator_is_any2 +TypeInferAnyError.metatable_of_any_can_be_a_table TypeInferClasses.can_read_prop_of_base_class_using_string TypeInferClasses.class_type_mismatch_with_name_conflict TypeInferClasses.classes_without_overloaded_operators_cannot_be_added TypeInferClasses.detailed_class_unification_error TypeInferClasses.higher_order_function_arguments_are_contravariant +TypeInferClasses.index_instance_property TypeInferClasses.optional_class_field_access_error TypeInferClasses.table_class_unification_reports_sane_errors_for_missing_properties TypeInferClasses.warn_when_prop_almost_matches @@ -349,7 +345,9 @@ TypeInferFunctions.calling_function_with_incorrect_argument_type_yields_errors_s TypeInferFunctions.cannot_hoist_interior_defns_into_signature TypeInferFunctions.dont_give_other_overloads_message_if_only_one_argument_matching_overload_exists TypeInferFunctions.dont_infer_parameter_types_for_functions_from_their_call_site +TypeInferFunctions.dont_mutate_the_underlying_head_of_typepack_when_calling_with_self TypeInferFunctions.duplicate_functions_with_different_signatures_not_allowed_in_nonstrict +TypeInferFunctions.first_argument_can_be_optional TypeInferFunctions.function_cast_error_uses_correct_language TypeInferFunctions.function_decl_non_self_sealed_overwrite_2 TypeInferFunctions.function_decl_non_self_unsealed_overwrite @@ -387,36 +385,45 @@ TypeInferLoops.loop_iter_no_indexer_nonstrict TypeInferLoops.loop_iter_trailing_nil TypeInferLoops.properly_infer_iteratee_is_a_free_table TypeInferLoops.unreachable_code_after_infinite_loop +TypeInferLoops.varlist_declared_by_for_in_loop_should_be_free +TypeInferModules.bound_free_table_export_is_ok TypeInferModules.custom_require_global -TypeInferModules.do_not_modify_imported_types +TypeInferModules.do_not_modify_imported_types_4 +TypeInferModules.do_not_modify_imported_types_5 TypeInferModules.module_type_conflict TypeInferModules.module_type_conflict_instantiated TypeInferModules.require_a_variadic_function TypeInferModules.type_error_of_unknown_qualified_type -TypeInferOOP.CheckMethodsOfSealed TypeInferOOP.dont_suggest_using_colon_rather_than_dot_if_another_overload_works TypeInferOOP.dont_suggest_using_colon_rather_than_dot_if_it_wont_help_2 TypeInferOOP.dont_suggest_using_colon_rather_than_dot_if_not_defined_with_colon TypeInferOOP.inferring_hundreds_of_self_calls_should_not_suffocate_memory +TypeInferOOP.method_depends_on_table TypeInferOOP.methods_are_topologically_sorted +TypeInferOOP.nonstrict_self_mismatch_tail TypeInferOOP.object_constructor_can_refer_to_method_of_self +TypeInferOOP.table_oop +TypeInferOperators.CallAndOrOfFunctions +TypeInferOperators.CallOrOfFunctions TypeInferOperators.cannot_compare_tables_that_do_not_have_the_same_metatable TypeInferOperators.cannot_indirectly_compare_types_that_do_not_have_a_metatable TypeInferOperators.cannot_indirectly_compare_types_that_do_not_offer_overloaded_ordering_operators TypeInferOperators.cli_38355_recursive_union +TypeInferOperators.compound_assign_metatable TypeInferOperators.compound_assign_mismatch_metatable TypeInferOperators.compound_assign_mismatch_op TypeInferOperators.compound_assign_mismatch_result TypeInferOperators.disallow_string_and_types_without_metatables_from_arithmetic_binary_ops TypeInferOperators.in_nonstrict_mode_strip_nil_from_intersections_when_considering_relational_operators TypeInferOperators.infer_any_in_all_modes_when_lhs_is_unknown -TypeInferOperators.mm_comparisons_must_return_a_boolean -TypeInferOperators.mm_ops_must_return_a_value +TypeInferOperators.operator_eq_completely_incompatible +TypeInferOperators.or_joins_types_with_no_superfluous_union TypeInferOperators.produce_the_correct_error_message_when_comparing_a_table_with_a_metatable_with_one_that_does_not -TypeInferOperators.refine_and_or TypeInferOperators.typecheck_overloaded_multiply_that_is_an_intersection TypeInferOperators.typecheck_overloaded_multiply_that_is_an_intersection_on_rhs TypeInferOperators.UnknownGlobalCompoundAssign +TypeInferOperators.unrelated_classes_cannot_be_compared +TypeInferOperators.unrelated_primitives_cannot_be_compared TypeInferPrimitives.CheckMethodsOfNumber TypeInferPrimitives.string_index TypeInferUnknownNever.assign_to_global_which_is_never @@ -432,6 +439,7 @@ TypeInferUnknownNever.type_packs_containing_never_is_itself_uninhabitable2 TypeInferUnknownNever.unary_minus_of_never TypePackTests.detect_cyclic_typepacks2 TypePackTests.pack_tail_unification_check +TypePackTests.self_and_varargs_should_work TypePackTests.type_alias_backwards_compatible TypePackTests.type_alias_default_export TypePackTests.type_alias_default_mixed_self

flags = 0 +#define invalidateTMcache(t) t->tmcache = 0 // empty hash data points to dummynode so that we can always dereference it const LuaNode luaH_dummynode = { @@ -479,7 +479,7 @@ Table* luaH_new(lua_State* L, int narray, int nhash) Table* t = luaM_newgco(L, Table, sizeof(Table), L->activememcat); luaC_init(L, t, LUA_TTABLE); t->metatable = NULL; - t->flags = cast_byte(~0); + t->tmcache = cast_byte(~0); t->array = NULL; t->sizearray = 0; t->lastfree = 0; @@ -778,7 +778,7 @@ Table* luaH_clone(lua_State* L, Table* tt) Table* t = luaM_newgco(L, Table, sizeof(Table), L->activememcat); luaC_init(L, t, LUA_TTABLE); t->metatable = tt->metatable; - t->flags = tt->flags; + t->tmcache = tt->tmcache; t->array = NULL; t->sizearray = 0; t->lsizenode = 0; @@ -835,5 +835,5 @@ void luaH_clear(Table* tt) } /* back to empty -> no tag methods present */ - tt->flags = cast_byte(~0); + tt->tmcache = cast_byte(~0); } diff --git a/VM/src/ltm.cpp b/VM/src/ltm.cpp index 9b99506b..e7df4e53 100644 --- a/VM/src/ltm.cpp +++ b/VM/src/ltm.cpp @@ -88,8 +88,8 @@ const TValue* luaT_gettm(Table* events, TMS event, TString* ename) const TValue* tm = luaH_getstr(events, ename); LUAU_ASSERT(event <= TM_EQ); if (ttisnil(tm)) - { /* no tag method? */ - events->flags |= cast_byte(1u << event); /* cache this fact */ + { /* no tag method? */ + events->tmcache |= cast_byte(1u << event); /* cache this fact */ return NULL; } else diff --git a/VM/src/ltm.h b/VM/src/ltm.h index e1b95c21..a5223941 100644 --- a/VM/src/ltm.h +++ b/VM/src/ltm.h @@ -41,10 +41,10 @@ typedef enum } TMS; // clang-format on -#define gfasttm(g, et, e) ((et) == NULL ? NULL : ((et)->flags & (1u << (e))) ? NULL : luaT_gettm(et, e, (g)->tmname[e])) +#define gfasttm(g, et, e) ((et) == NULL ? NULL : ((et)->tmcache & (1u << (e))) ? NULL : luaT_gettm(et, e, (g)->tmname[e])) #define fasttm(l, et, e) gfasttm(l->global, et, e) -#define fastnotm(et, e) ((et) == NULL || ((et)->flags & (1u << (e)))) +#define fastnotm(et, e) ((et) == NULL || ((et)->tmcache & (1u << (e)))) LUAI_DATA const char* const luaT_typenames[]; LUAI_DATA const char* const luaT_eventname[]; diff --git a/tests/Autocomplete.test.cpp b/tests/Autocomplete.test.cpp index dea1ab19..f3b0bcad 100644 --- a/tests/Autocomplete.test.cpp +++ b/tests/Autocomplete.test.cpp @@ -1992,6 +1992,7 @@ local fp: @1= f auto ac = autocomplete('1'); + REQUIRE_EQ("({| x: number, y: number |}) -> number", toString(requireType("f"))); CHECK(ac.entryMap.count("({ x: number, y: number }) -> number")); } @@ -2620,7 +2621,6 @@ a = if temp then even elseif true then temp else e@9 TEST_CASE_FIXTURE(ACFixture, "autocomplete_if_else_regression") { - ScopedFastFlag FFlagLuauIfElseExprFixCompletionIssue("LuauIfElseExprFixCompletionIssue", true); check(R"( local abcdef = 0; local temp = false diff --git a/tests/Compiler.test.cpp b/tests/Compiler.test.cpp index 6eee254e..036bf124 100644 --- a/tests/Compiler.test.cpp +++ b/tests/Compiler.test.cpp @@ -4992,6 +4992,147 @@ RETURN R1 1 )"); } +TEST_CASE("InlineCapture") +{ + // if the argument is captured by a nested closure, normally we can rely on capture by value + CHECK_EQ("\n" + compileFunction(R"( +local function foo(a) + return function() return a end +end + +local x = ... +local y = foo(x) +return y +)", + 2, 2), + R"( +DUPCLOSURE R0 K0 +GETVARARGS R1 1 +NEWCLOSURE R2 P1 +CAPTURE VAL R1 +RETURN R2 1 +)"); + + // if the argument is a constant, we move it to a register so that capture by value can happen + CHECK_EQ("\n" + compileFunction(R"( +local function foo(a) + return function() return a end +end + +local y = foo(42) +return y +)", + 2, 2), + R"( +DUPCLOSURE R0 K0 +LOADN R2 42 +NEWCLOSURE R1 P1 +CAPTURE VAL R2 +RETURN R1 1 +)"); + + // if the argument is an externally mutated variable, we copy it to an argument and capture it by value + CHECK_EQ("\n" + compileFunction(R"( +local function foo(a) + return function() return a end +end + +local x x = 42 +local y = foo(x) +return y +)", + 2, 2), + R"( +DUPCLOSURE R0 K0 +LOADNIL R1 +LOADN R1 42 +MOVE R3 R1 +NEWCLOSURE R2 P1 +CAPTURE VAL R3 +RETURN R2 1 +)"); + + // finally, if the argument is mutated internally, we must capture it by reference and close the upvalue + CHECK_EQ("\n" + compileFunction(R"( +local function foo(a) + a = a or 42 + return function() return a end +end + +local y = foo() +return y +)", + 2, 2), + R"( +DUPCLOSURE R0 K0 +LOADNIL R2 +ORK R2 R2 K1 +NEWCLOSURE R1 P1 +CAPTURE REF R2 +CLOSEUPVALS R2 +RETURN R1 1 +)"); + + // note that capture might need to be performed during the fallthrough block + CHECK_EQ("\n" + compileFunction(R"( +local function foo(a) + a = a or 42 + print(function() return a end) +end + +local x = ... +local y = foo(x) +return y +)", + 2, 2), + R"( +DUPCLOSURE R0 K0 +GETVARARGS R1 1 +MOVE R3 R1 +ORK R3 R3 K1 +GETIMPORT R4 3 +NEWCLOSURE R5 P1 +CAPTURE REF R3 +CALL R4 1 0 +LOADNIL R2 +CLOSEUPVALS R3 +RETURN R2 1 +)"); + + // note that mutation and capture might be inside internal control flow + // TODO: this has an oddly redundant CLOSEUPVALS after JUMP; it's not due to inlining, and is an artifact of how StatBlock/StatReturn interact + // fixing this would reduce the number of redundant CLOSEUPVALS a bit but it only affects bytecode size as these instructions aren't executed + CHECK_EQ("\n" + compileFunction(R"( +local function foo(a) + if not a then + local b b = 42 + return function() return b end + end +end + +local x = ... +local y = foo(x) +return y, x +)", + 2, 2), + R"( +DUPCLOSURE R0 K0 +GETVARARGS R1 1 +JUMPIF R1 L0 +LOADNIL R3 +LOADN R3 42 +NEWCLOSURE R2 P1 +CAPTURE REF R3 +CLOSEUPVALS R3 +JUMP L1 +CLOSEUPVALS R3 +L0: LOADNIL R2 +L1: MOVE R3 R2 +MOVE R4 R1 +RETURN R3 2 +)"); +} + TEST_CASE("InlineFallthrough") { // if the function doesn't return, we still fill the results with nil @@ -5044,27 +5185,6 @@ RETURN R1 -1 )"); } -TEST_CASE("InlineCapture") -{ - // can't inline function with nested functions that capture locals because they might be constants - CHECK_EQ("\n" + compileFunction(R"( -local function foo(a) - local function bar() - return a - end - return bar() -end -)", - 1, 2), - R"( -NEWCLOSURE R1 P0 -CAPTURE VAL R0 -MOVE R2 R1 -CALL R2 0 -1 -RETURN R2 -1 -)"); -} - TEST_CASE("InlineArgMismatch") { // when inlining a function, we must respect all the usual rules @@ -5491,6 +5611,96 @@ RETURN R2 1 )"); } +TEST_CASE("InlineMultret") +{ + // inlining a function in multret context is prohibited since we can't adjust L->top outside of CALL/GETVARARGS + CHECK_EQ("\n" + compileFunction(R"( +local function foo(a) + return a() +end + +return foo(42) +)", + 1, 2), + R"( +DUPCLOSURE R0 K0 +MOVE R1 R0 +LOADN R2 42 +CALL R1 1 -1 +RETURN R1 -1 +)"); + + // however, if we can deduce statically that a function always returns a single value, the inlining will work + CHECK_EQ("\n" + compileFunction(R"( +local function foo(a) + return a +end + +return foo(42) +)", + 1, 2), + R"( +DUPCLOSURE R0 K0 +LOADN R1 42 +RETURN R1 1 +)"); + + // this analysis will also propagate through other functions + CHECK_EQ("\n" + compileFunction(R"( +local function foo(a) + return a +end + +local function bar(a) + return foo(a) +end + +return bar(42) +)", + 2, 2), + R"( +DUPCLOSURE R0 K0 +DUPCLOSURE R1 K1 +LOADN R2 42 +RETURN R2 1 +)"); + + // we currently don't do this analysis fully for recursive functions since they can't be inlined anyway + CHECK_EQ("\n" + compileFunction(R"( +local function foo(a) + return foo(a) +end + +return foo(42) +)", + 1, 2), + R"( +DUPCLOSURE R0 K0 +CAPTURE VAL R0 +MOVE R1 R0 +LOADN R2 42 +CALL R1 1 -1 +RETURN R1 -1 +)"); + + // and unfortunately we can't do this analysis for builtins or method calls due to getfenv + CHECK_EQ("\n" + compileFunction(R"( +local function foo(a) + return math.abs(a) +end + +return foo(42) +)", + 1, 2), + R"( +DUPCLOSURE R0 K0 +MOVE R1 R0 +LOADN R2 42 +CALL R1 1 -1 +RETURN R1 -1 +)"); +} + TEST_CASE("ReturnConsecutive") { // we can return a single local directly diff --git a/tests/ConstraintGraphBuilder.test.cpp b/tests/ConstraintGraphBuilder.test.cpp index ab5af4f6..96b21613 100644 --- a/tests/ConstraintGraphBuilder.test.cpp +++ b/tests/ConstraintGraphBuilder.test.cpp @@ -17,13 +17,13 @@ TEST_CASE_FIXTURE(ConstraintGraphBuilderFixture, "hello_world") )"); cgb.visit(block); - std::vector constraints = collectConstraints(cgb.rootScope); + auto constraints = collectConstraints(cgb.rootScope); REQUIRE(2 == constraints.size()); ToStringOptions opts; - CHECK("a <: string" == toString(*constraints[0], opts)); - CHECK("b <: a" == toString(*constraints[1], opts)); + CHECK("string <: a" == toString(*constraints[0], opts)); + CHECK("a <: b" == toString(*constraints[1], opts)); } TEST_CASE_FIXTURE(ConstraintGraphBuilderFixture, "primitives") @@ -36,15 +36,34 @@ TEST_CASE_FIXTURE(ConstraintGraphBuilderFixture, "primitives") )"); cgb.visit(block); - std::vector constraints = collectConstraints(cgb.rootScope); + auto constraints = collectConstraints(cgb.rootScope); - REQUIRE(4 == constraints.size()); + REQUIRE(3 == constraints.size()); ToStringOptions opts; - CHECK("a <: string" == toString(*constraints[0], opts)); - CHECK("b <: number" == toString(*constraints[1], opts)); - CHECK("c <: boolean" == toString(*constraints[2], opts)); - CHECK("d <: nil" == toString(*constraints[3], opts)); + CHECK("string <: a" == toString(*constraints[0], opts)); + CHECK("number <: b" == toString(*constraints[1], opts)); + CHECK("boolean <: c" == toString(*constraints[2], opts)); +} + +TEST_CASE_FIXTURE(ConstraintGraphBuilderFixture, "nil_primitive") +{ + AstStatBlock* block = parse(R"( + local function a() return nil end + local b = a() + )"); + + cgb.visit(block); + auto constraints = collectConstraints(cgb.rootScope); + + ToStringOptions opts; + REQUIRE(5 <= constraints.size()); + + CHECK("*blocked-1* ~ gen () -> (a...)" == toString(*constraints[0], opts)); + CHECK("b ~ inst *blocked-1*" == toString(*constraints[1], opts)); + CHECK("() -> (c...) <: b" == toString(*constraints[2], opts)); + CHECK("c... <: d" == toString(*constraints[3], opts)); + CHECK("nil <: a..." == toString(*constraints[4], opts)); } TEST_CASE_FIXTURE(ConstraintGraphBuilderFixture, "function_application") @@ -55,15 +74,15 @@ TEST_CASE_FIXTURE(ConstraintGraphBuilderFixture, "function_application") )"); cgb.visit(block); - std::vector constraints = collectConstraints(cgb.rootScope); + auto constraints = collectConstraints(cgb.rootScope); REQUIRE(4 == constraints.size()); ToStringOptions opts; - CHECK("a <: string" == toString(*constraints[0], opts)); + CHECK("string <: a" == toString(*constraints[0], opts)); CHECK("b ~ inst a" == toString(*constraints[1], opts)); - CHECK("(string) -> (c, d...) <: b" == toString(*constraints[2], opts)); - CHECK("e <: c" == toString(*constraints[3], opts)); + CHECK("(string) -> (c...) <: b" == toString(*constraints[2], opts)); + CHECK("c... <: d" == toString(*constraints[3], opts)); } TEST_CASE_FIXTURE(ConstraintGraphBuilderFixture, "local_function_definition") @@ -75,13 +94,13 @@ TEST_CASE_FIXTURE(ConstraintGraphBuilderFixture, "local_function_definition") )"); cgb.visit(block); - std::vector constraints = collectConstraints(cgb.rootScope); + auto constraints = collectConstraints(cgb.rootScope); REQUIRE(2 == constraints.size()); ToStringOptions opts; - CHECK("a ~ gen (b) -> (c...)" == toString(*constraints[0], opts)); - CHECK("b <: c..." == toString(*constraints[1], opts)); + CHECK("*blocked-1* ~ gen (a) -> (b...)" == toString(*constraints[0], opts)); + CHECK("a <: b..." == toString(*constraints[1], opts)); } TEST_CASE_FIXTURE(ConstraintGraphBuilderFixture, "recursive_function") @@ -93,15 +112,15 @@ TEST_CASE_FIXTURE(ConstraintGraphBuilderFixture, "recursive_function") )"); cgb.visit(block); - std::vector constraints = collectConstraints(cgb.rootScope); + auto constraints = collectConstraints(cgb.rootScope); REQUIRE(4 == constraints.size()); ToStringOptions opts; - CHECK("a ~ gen (b) -> (c...)" == toString(*constraints[0], opts)); - CHECK("d ~ inst a" == toString(*constraints[1], opts)); - CHECK("(b) -> (e, f...) <: d" == toString(*constraints[2], opts)); - CHECK("e <: c..." == toString(*constraints[3], opts)); + CHECK("*blocked-1* ~ gen (a) -> (b...)" == toString(*constraints[0], opts)); + CHECK("c ~ inst (a) -> (b...)" == toString(*constraints[1], opts)); + CHECK("(a) -> (d...) <: c" == toString(*constraints[2], opts)); + CHECK("d... <: b..." == toString(*constraints[3], opts)); } TEST_SUITE_END(); diff --git a/tests/Fixture.cpp b/tests/Fixture.cpp index 232ec2de..ac22f65b 100644 --- a/tests/Fixture.cpp +++ b/tests/Fixture.cpp @@ -345,7 +345,7 @@ void Fixture::dumpErrors(std::ostream& os, const std::vector& errors) if (error.location.begin.line >= lines.size()) { os << "\tSource not available?" << std::endl; - return; + continue; } std::string_view theLine = lines[error.location.begin.line]; @@ -430,6 +430,7 @@ ConstraintGraphBuilderFixture::ConstraintGraphBuilderFixture() : Fixture() , forceTheFlag{"DebugLuauDeferredConstraintResolution", true} { + BlockedTypeVar::nextIndex = 0; } ModuleName fromString(std::string_view name) diff --git a/tests/Frontend.test.cpp b/tests/Frontend.test.cpp index c0554669..b9c24704 100644 --- a/tests/Frontend.test.cpp +++ b/tests/Frontend.test.cpp @@ -97,8 +97,8 @@ TEST_CASE_FIXTURE(FrontendFixture, "find_a_require") NaiveFileResolver naiveFileResolver; auto res = traceRequires(&naiveFileResolver, program, ""); - CHECK_EQ(1, res.requires.size()); - CHECK_EQ(res.requires[0].first, "Modules/Foo/Bar"); + CHECK_EQ(1, res.requireList.size()); + CHECK_EQ(res.requireList[0].first, "Modules/Foo/Bar"); } // It could be argued that this should not work. @@ -113,7 +113,7 @@ TEST_CASE_FIXTURE(FrontendFixture, "find_a_require_inside_a_function") NaiveFileResolver naiveFileResolver; auto res = traceRequires(&naiveFileResolver, program, ""); - CHECK_EQ(1, res.requires.size()); + CHECK_EQ(1, res.requireList.size()); } TEST_CASE_FIXTURE(FrontendFixture, "real_source") @@ -138,7 +138,7 @@ TEST_CASE_FIXTURE(FrontendFixture, "real_source") NaiveFileResolver naiveFileResolver; auto res = traceRequires(&naiveFileResolver, program, ""); - CHECK_EQ(8, res.requires.size()); + CHECK_EQ(8, res.requireList.size()); } TEST_CASE_FIXTURE(FrontendFixture, "automatically_check_dependent_scripts") diff --git a/tests/Module.test.cpp b/tests/Module.test.cpp index 89b13ab1..d585b731 100644 --- a/tests/Module.test.cpp +++ b/tests/Module.test.cpp @@ -102,7 +102,7 @@ TEST_CASE_FIXTURE(Fixture, "deepClone_cyclic_table") const FunctionTypeVar* ftv = get(methodType); REQUIRE(ftv != nullptr); - std::optional methodReturnType = first(ftv->retType); + std::optional methodReturnType = first(ftv->retTypes); REQUIRE(methodReturnType); CHECK_EQ(methodReturnType, counterCopy); diff --git a/tests/NonstrictMode.test.cpp b/tests/NonstrictMode.test.cpp index c0556103..50dcbad0 100644 --- a/tests/NonstrictMode.test.cpp +++ b/tests/NonstrictMode.test.cpp @@ -13,6 +13,57 @@ using namespace Luau; TEST_SUITE_BEGIN("NonstrictModeTests"); +TEST_CASE_FIXTURE(Fixture, "globals") +{ + CheckResult result = check(R"( + --!nonstrict + foo = true + foo = "now i'm a string!" + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + CHECK_EQ("any", toString(requireType("foo"))); +} + +TEST_CASE_FIXTURE(Fixture, "globals2") +{ + ScopedFastFlag sff[]{ + {"LuauReturnTypeInferenceInNonstrict", true}, + {"LuauLowerBoundsCalculation", true}, + }; + + CheckResult result = check(R"( + --!nonstrict + foo = function() return 1 end + foo = "now i'm a string!" + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + TypeMismatch* tm = get(result.errors[0]); + REQUIRE(tm); + CHECK_EQ("() -> number", toString(tm->wantedType)); + CHECK_EQ("string", toString(tm->givenType)); + CHECK_EQ("() -> number", toString(requireType("foo"))); +} + +TEST_CASE_FIXTURE(Fixture, "globals_everywhere") +{ + CheckResult result = check(R"( + --!nonstrict + foo = 1 + + if true then + bar = 2 + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ("any", toString(requireType("foo"))); + CHECK_EQ("any", toString(requireType("bar"))); +} + TEST_CASE_FIXTURE(BuiltinsFixture, "function_returns_number_or_string") { ScopedFastFlag sff[]{{"LuauReturnTypeInferenceInNonstrict", true}, {"LuauLowerBoundsCalculation", true}}; @@ -51,7 +102,7 @@ TEST_CASE_FIXTURE(Fixture, "infer_nullary_function") REQUIRE_EQ("any", toString(args[0])); REQUIRE_EQ("any", toString(args[1])); - auto rets = flatten(ftv->retType).first; + auto rets = flatten(ftv->retTypes).first; REQUIRE_EQ(0, rets.size()); } diff --git a/tests/Normalize.test.cpp b/tests/Normalize.test.cpp index 2876175d..284230c9 100644 --- a/tests/Normalize.test.cpp +++ b/tests/Normalize.test.cpp @@ -837,6 +837,7 @@ TEST_CASE_FIXTURE(Fixture, "intersection_inside_a_table_inside_another_intersect { ScopedFastFlag flags[] = { {"LuauLowerBoundsCalculation", true}, + {"LuauQuantifyConstrained", true}, }; // We use a function and inferred parameter types to prevent intermediate normalizations from being performed. @@ -867,16 +868,17 @@ TEST_CASE_FIXTURE(Fixture, "intersection_inside_a_table_inside_another_intersect CHECK("{+ y: number +}" == toString(args[2])); CHECK("{+ z: string +}" == toString(args[3])); - std::vector ret = flatten(ftv->retType).first; + std::vector ret = flatten(ftv->retTypes).first; REQUIRE(1 == ret.size()); - CHECK("{| x: a & {- w: boolean, y: number, z: string -} |}" == toString(ret[0])); + CHECK("{| x: a & {+ w: boolean, y: number, z: string +} |}" == toString(ret[0])); } TEST_CASE_FIXTURE(Fixture, "intersection_inside_a_table_inside_another_intersection_3") { ScopedFastFlag flags[] = { {"LuauLowerBoundsCalculation", true}, + {"LuauQuantifyConstrained", true}, }; // We use a function and inferred parameter types to prevent intermediate normalizations from being performed. @@ -906,16 +908,17 @@ TEST_CASE_FIXTURE(Fixture, "intersection_inside_a_table_inside_another_intersect CHECK("t1 where t1 = {+ y: t1 +}" == toString(args[1])); CHECK("{+ z: string +}" == toString(args[2])); - std::vector ret = flatten(ftv->retType).first; + std::vector ret = flatten(ftv->retTypes).first; REQUIRE(1 == ret.size()); - CHECK("{| x: {- x: boolean, y: t1, z: string -} |} where t1 = {+ y: t1 +}" == toString(ret[0])); + CHECK("{| x: {+ x: boolean, y: t1, z: string +} |} where t1 = {+ y: t1 +}" == toString(ret[0])); } TEST_CASE_FIXTURE(Fixture, "intersection_inside_a_table_inside_another_intersection_4") { ScopedFastFlag flags[] = { {"LuauLowerBoundsCalculation", true}, + {"LuauQuantifyConstrained", true}, }; // We use a function and inferred parameter types to prevent intermediate normalizations from being performed. @@ -944,13 +947,13 @@ TEST_CASE_FIXTURE(Fixture, "intersection_inside_a_table_inside_another_intersect REQUIRE(3 == args.size()); CHECK("{+ x: boolean +}" == toString(args[0])); - CHECK("{+ y: t1 +} where t1 = {| x: {- x: boolean, y: t1, z: string -} |}" == toString(args[1])); + CHECK("{+ y: t1 +} where t1 = {| x: {+ x: boolean, y: t1, z: string +} |}" == toString(args[1])); CHECK("{+ z: string +}" == toString(args[2])); - std::vector ret = flatten(ftv->retType).first; + std::vector ret = flatten(ftv->retTypes).first; REQUIRE(1 == ret.size()); - CHECK("t1 where t1 = {| x: {- x: boolean, y: t1, z: string -} |}" == toString(ret[0])); + CHECK("t1 where t1 = {| x: {+ x: boolean, y: t1, z: string +} |}" == toString(ret[0])); } TEST_CASE_FIXTURE(Fixture, "nested_table_normalization_with_non_table__no_ice") @@ -1062,4 +1065,29 @@ export type t0 = (((any)&({_:l0.t0,n0:t0,_G:any,}))&({_:any,}))&(((any)&({_:l0.t LUAU_REQUIRE_ERRORS(result); } +TEST_CASE_FIXTURE(BuiltinsFixture, "normalization_does_not_convert_ever") +{ + ScopedFastFlag sff[]{ + {"LuauLowerBoundsCalculation", true}, + {"LuauQuantifyConstrained", true}, + }; + + CheckResult result = check(R"( + --!strict + local function f() + if math.random() > 0.5 then + return true + end + type Ret = typeof(f()) + if math.random() > 0.5 then + return "something" + end + return "something" :: Ret + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + CHECK_EQ("() -> boolean | string", toString(requireType("f"))); +} + TEST_SUITE_END(); diff --git a/tests/NotNull.test.cpp b/tests/NotNull.test.cpp index 1a323c85..ed1c25ec 100644 --- a/tests/NotNull.test.cpp +++ b/tests/NotNull.test.cpp @@ -75,9 +75,9 @@ TEST_CASE("basic_stuff") t->y = 3.14f; const NotNull u = t; - // u->x = 44; // nope + u->x = 44; int v = u->x; - CHECK(v == 5); + CHECK(v == 44); bar(a); @@ -96,8 +96,11 @@ TEST_CASE("basic_stuff") TEST_CASE("hashable") { std::unordered_map, const char*> map; - NotNull a{new int(8)}; - NotNull b{new int(10)}; + int a_ = 8; + int b_ = 10; + + NotNull a{&a_}; + NotNull b{&b_}; std::string hello = "hello"; std::string world = "world"; @@ -108,9 +111,47 @@ TEST_CASE("hashable") CHECK_EQ(2, map.size()); CHECK_EQ(hello.c_str(), map[a]); CHECK_EQ(world.c_str(), map[b]); +} - delete a; - delete b; +TEST_CASE("const") +{ + int p = 0; + int q = 0; + + NotNull n{&p}; + + *n = 123; + + NotNull m = n; // Conversion from NotNull to NotNull is allowed + + CHECK(123 == *m); // readonly access of m is ok + + // *m = 321; // nope. m points at const data. + + // NotNull o = m; // nope. Conversion from NotNull to NotNull is forbidden + + NotNull n2{&q}; + m = n2; // ok. m points to const data, but is not itself const + + const NotNull m2 = n; + // m2 = n2; // nope. m2 is const. + *m2 = 321; // ok. m2 is const, but points to mutable data + + CHECK(321 == *n); +} + +TEST_CASE("const_compatibility") +{ + int* raw = new int(8); + + NotNull a(raw); + NotNull b(raw); + NotNull c = a; + // NotNull d = c; // nope - no conversion from const to non-const + + CHECK_EQ(*c, 8); + + delete raw; } TEST_SUITE_END(); diff --git a/tests/TypeInfer.annotations.test.cpp b/tests/TypeInfer.annotations.test.cpp index b9e1ae96..ccdd2b37 100644 --- a/tests/TypeInfer.annotations.test.cpp +++ b/tests/TypeInfer.annotations.test.cpp @@ -70,7 +70,7 @@ TEST_CASE_FIXTURE(Fixture, "function_return_annotations_are_checked") const FunctionTypeVar* ftv = get(fiftyType); REQUIRE(ftv != nullptr); - TypePackId retPack = ftv->retType; + TypePackId retPack = ftv->retTypes; const TypePack* tp = get(retPack); REQUIRE(tp != nullptr); diff --git a/tests/TypeInfer.functions.test.cpp b/tests/TypeInfer.functions.test.cpp index a28ba49e..036a667a 100644 --- a/tests/TypeInfer.functions.test.cpp +++ b/tests/TypeInfer.functions.test.cpp @@ -45,7 +45,7 @@ TEST_CASE_FIXTURE(Fixture, "infer_return_type") const FunctionTypeVar* takeFiveType = get(requireType("take_five")); REQUIRE(takeFiveType != nullptr); - std::vector retVec = flatten(takeFiveType->retType).first; + std::vector retVec = flatten(takeFiveType->retTypes).first; REQUIRE(!retVec.empty()); REQUIRE_EQ(*follow(retVec[0]), *typeChecker.numberType); @@ -345,7 +345,7 @@ TEST_CASE_FIXTURE(Fixture, "local_function") const FunctionTypeVar* ftv = get(h); REQUIRE(ftv != nullptr); - std::optional rt = first(ftv->retType); + std::optional rt = first(ftv->retTypes); REQUIRE(bool(rt)); TypeId retType = follow(*rt); @@ -361,7 +361,7 @@ TEST_CASE_FIXTURE(Fixture, "func_expr_doesnt_leak_free") LUAU_REQUIRE_NO_ERRORS(result); const Luau::FunctionTypeVar* fn = get(requireType("p")); REQUIRE(fn); - auto ret = first(fn->retType); + auto ret = first(fn->retTypes); REQUIRE(ret); REQUIRE(get(follow(*ret))); } @@ -460,7 +460,7 @@ TEST_CASE_FIXTURE(Fixture, "complicated_return_types_require_an_explicit_annotat const FunctionTypeVar* functionType = get(requireType("most_of_the_natural_numbers")); - std::optional retType = first(functionType->retType); + std::optional retType = first(functionType->retTypes); REQUIRE(retType); CHECK(get(*retType)); } @@ -1619,4 +1619,56 @@ TEST_CASE_FIXTURE(Fixture, "weird_fail_to_unify_type_pack") LUAU_REQUIRE_NO_ERRORS(result); } +TEST_CASE_FIXTURE(Fixture, "quantify_constrained_types") +{ + ScopedFastFlag sff[]{ + {"LuauLowerBoundsCalculation", true}, + {"LuauQuantifyConstrained", true}, + }; + + CheckResult result = check(R"( + --!strict + local function foo(f) + f(5) + f("hi") + local function g() + return f + end + local h = g() + h(true) + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ("((boolean | number | string) -> (a...)) -> ()", toString(requireType("foo"))); +} + +TEST_CASE_FIXTURE(Fixture, "call_o_with_another_argument_after_foo_was_quantified") +{ + ScopedFastFlag sff[]{ + {"LuauLowerBoundsCalculation", true}, + {"LuauQuantifyConstrained", true}, + }; + + CheckResult result = check(R"( + local function f(o) + local t = {} + t[o] = true + + local function foo(o) + o.m1(5) + t[o] = nil + end + + o.m1("hi") + + return t + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + // TODO: check the normalized type of f +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.generics.test.cpp b/tests/TypeInfer.generics.test.cpp index fbda8bec..edb5adcf 100644 --- a/tests/TypeInfer.generics.test.cpp +++ b/tests/TypeInfer.generics.test.cpp @@ -224,7 +224,7 @@ TEST_CASE_FIXTURE(Fixture, "infer_generic_function") const FunctionTypeVar* idFun = get(idType); REQUIRE(idFun); auto [args, varargs] = flatten(idFun->argTypes); - auto [rets, varrets] = flatten(idFun->retType); + auto [rets, varrets] = flatten(idFun->retTypes); CHECK_EQ(idFun->generics.size(), 1); CHECK_EQ(idFun->genericPacks.size(), 0); @@ -247,7 +247,7 @@ TEST_CASE_FIXTURE(Fixture, "infer_generic_local_function") const FunctionTypeVar* idFun = get(idType); REQUIRE(idFun); auto [args, varargs] = flatten(idFun->argTypes); - auto [rets, varrets] = flatten(idFun->retType); + auto [rets, varrets] = flatten(idFun->retTypes); CHECK_EQ(idFun->generics.size(), 1); CHECK_EQ(idFun->genericPacks.size(), 0); @@ -882,7 +882,7 @@ TEST_CASE_FIXTURE(Fixture, "correctly_instantiate_polymorphic_member_functions") const FunctionTypeVar* foo = get(follow(fooProp->type)); REQUIRE(bool(foo)); - std::optional ret_ = first(foo->retType); + std::optional ret_ = first(foo->retTypes); REQUIRE(bool(ret_)); TypeId ret = follow(*ret_); diff --git a/tests/TypeInfer.operators.test.cpp b/tests/TypeInfer.operators.test.cpp index 03614938..fd9b1dd4 100644 --- a/tests/TypeInfer.operators.test.cpp +++ b/tests/TypeInfer.operators.test.cpp @@ -90,7 +90,7 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "primitive_arith_no_metatable") const FunctionTypeVar* functionType = get(requireType("add")); - std::optional retType = first(functionType->retType); + std::optional retType = first(functionType->retTypes); REQUIRE(retType.has_value()); CHECK_EQ(typeChecker.numberType, follow(*retType)); CHECK_EQ(requireType("n"), typeChecker.numberType); @@ -777,8 +777,6 @@ TEST_CASE_FIXTURE(Fixture, "infer_any_in_all_modes_when_lhs_is_unknown") TEST_CASE_FIXTURE(BuiltinsFixture, "equality_operations_succeed_if_any_union_branch_succeeds") { - ScopedFastFlag sff("LuauSuccessTypingForEqualityOperations", true); - CheckResult result = check(R"( local mm = {} type Foo = typeof(setmetatable({}, mm)) diff --git a/tests/TypeInfer.provisional.test.cpp b/tests/TypeInfer.provisional.test.cpp index 22fb3b69..487e5979 100644 --- a/tests/TypeInfer.provisional.test.cpp +++ b/tests/TypeInfer.provisional.test.cpp @@ -472,6 +472,7 @@ TEST_CASE_FIXTURE(Fixture, "constrained_is_level_dependent") ScopedFastFlag sff[]{ {"LuauLowerBoundsCalculation", true}, {"LuauNormalizeFlagIsConservative", true}, + {"LuauQuantifyConstrained", true}, }; CheckResult result = check(R"( @@ -494,8 +495,8 @@ TEST_CASE_FIXTURE(Fixture, "constrained_is_level_dependent") )"); LUAU_REQUIRE_NO_ERRORS(result); - // TODO: We're missing generics a... and b... - CHECK_EQ("(t1) -> {| [t1]: boolean |} where t1 = t2 ; t2 = {+ m1: (t1) -> (a...), m2: (t2) -> (b...) +}", toString(requireType("f"))); + // TODO: We're missing generics b... + CHECK_EQ("(t1) -> {| [t1]: boolean |} where t1 = t2 ; t2 = {+ m1: (t1) -> (a...), m2: (t2) -> (b...) +}", toString(requireType("f"))); } TEST_SUITE_END(); diff --git a/tests/TypeInfer.refinements.test.cpp b/tests/TypeInfer.refinements.test.cpp index 207b3cff..cefba4b2 100644 --- a/tests/TypeInfer.refinements.test.cpp +++ b/tests/TypeInfer.refinements.test.cpp @@ -13,8 +13,8 @@ using namespace Luau; namespace { -std::optional> magicFunctionInstanceIsA( - TypeChecker& typeChecker, const ScopePtr& scope, const AstExprCall& expr, ExprResult exprResult) +std::optional> magicFunctionInstanceIsA( + TypeChecker& typeChecker, const ScopePtr& scope, const AstExprCall& expr, WithPredicate withPredicate) { if (expr.args.size != 1) return std::nullopt; @@ -32,7 +32,7 @@ std::optional> magicFunctionInstanceIsA( unfreeze(typeChecker.globalTypes); TypePackId booleanPack = typeChecker.globalTypes.addTypePack({typeChecker.booleanType}); freeze(typeChecker.globalTypes); - return ExprResult{booleanPack, {IsAPredicate{std::move(*lvalue), expr.location, tfun->type}}}; + return WithPredicate{booleanPack, {IsAPredicate{std::move(*lvalue), expr.location, tfun->type}}}; } struct RefinementClassFixture : Fixture diff --git a/tests/TypeInfer.tables.test.cpp b/tests/TypeInfer.tables.test.cpp index d622d4af..87d49651 100644 --- a/tests/TypeInfer.tables.test.cpp +++ b/tests/TypeInfer.tables.test.cpp @@ -642,7 +642,7 @@ TEST_CASE_FIXTURE(Fixture, "indexers_quantification_2") const TableTypeVar* argType = get(follow(argVec[0])); REQUIRE(argType != nullptr); - std::vector retVec = flatten(ftv->retType).first; + std::vector retVec = flatten(ftv->retTypes).first; const TableTypeVar* retType = get(follow(retVec[0])); REQUIRE(retType != nullptr); @@ -691,7 +691,7 @@ TEST_CASE_FIXTURE(Fixture, "infer_indexer_from_value_property_in_literal") const FunctionTypeVar* fType = get(requireType("f")); REQUIRE(fType != nullptr); - auto retType_ = first(fType->retType); + auto retType_ = first(fType->retTypes); REQUIRE(bool(retType_)); auto retType = get(follow(*retType_)); @@ -1881,7 +1881,7 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "quantifying_a_bound_var_works") REQUIRE(prop.type); const FunctionTypeVar* ftv = get(follow(prop.type)); REQUIRE(ftv); - const TypePack* res = get(follow(ftv->retType)); + const TypePack* res = get(follow(ftv->retTypes)); REQUIRE(res); REQUIRE(res->head.size() == 1); const MetatableTypeVar* mtv = get(follow(res->head[0])); @@ -2584,7 +2584,7 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "dont_quantify_table_that_belongs_to_outer_sc const FunctionTypeVar* newType = get(follow(counterType->props["new"].type)); REQUIRE(newType); - std::optional newRetType = *first(newType->retType); + std::optional newRetType = *first(newType->retTypes); REQUIRE(newRetType); const MetatableTypeVar* newRet = get(follow(*newRetType)); @@ -2977,7 +2977,6 @@ TEST_CASE_FIXTURE(Fixture, "mixed_tables_with_implicit_numbered_keys") TEST_CASE_FIXTURE(Fixture, "expected_indexer_value_type_extra") { - ScopedFastFlag luauExpectedPropTypeFromIndexer{"LuauExpectedPropTypeFromIndexer", true}; ScopedFastFlag luauSubtypingAddOptPropsToUnsealedTables{"LuauSubtypingAddOptPropsToUnsealedTables", true}; CheckResult result = check(R"( @@ -2992,8 +2991,6 @@ TEST_CASE_FIXTURE(Fixture, "expected_indexer_value_type_extra") TEST_CASE_FIXTURE(Fixture, "expected_indexer_value_type_extra_2") { - ScopedFastFlag luauExpectedPropTypeFromIndexer{"LuauExpectedPropTypeFromIndexer", true}; - CheckResult result = check(R"( type X = {[any]: string | boolean} diff --git a/tests/TypeInfer.test.cpp b/tests/TypeInfer.test.cpp index cf0c9881..6257cda6 100644 --- a/tests/TypeInfer.test.cpp +++ b/tests/TypeInfer.test.cpp @@ -13,8 +13,9 @@ #include -LUAU_FASTFLAG(LuauLowerBoundsCalculation) -LUAU_FASTFLAG(LuauFixLocationSpanTableIndexExpr) +LUAU_FASTFLAG(LuauLowerBoundsCalculation); +LUAU_FASTFLAG(LuauFixLocationSpanTableIndexExpr); +LUAU_FASTFLAG(DebugLuauDeferredConstraintResolution); using namespace Luau; @@ -43,10 +44,7 @@ TEST_CASE_FIXTURE(Fixture, "tc_error") CheckResult result = check("local a = 7 local b = 'hi' a = b"); LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK_EQ(result.errors[0], (TypeError{Location{Position{0, 35}, Position{0, 36}}, TypeMismatch{ - requireType("a"), - requireType("b"), - }})); + CHECK_EQ(result.errors[0], (TypeError{Location{Position{0, 35}, Position{0, 36}}, TypeMismatch{typeChecker.numberType, typeChecker.stringType}})); } TEST_CASE_FIXTURE(Fixture, "tc_error_2") @@ -86,6 +84,7 @@ TEST_CASE_FIXTURE(Fixture, "infer_locals_via_assignment_from_its_call_site") TEST_CASE_FIXTURE(Fixture, "infer_in_nocheck_mode") { ScopedFastFlag sff[]{ + {"DebugLuauDeferredConstraintResolution", false}, {"LuauReturnTypeInferenceInNonstrict", true}, {"LuauLowerBoundsCalculation", true}, }; @@ -236,10 +235,14 @@ TEST_CASE_FIXTURE(Fixture, "type_errors_infer_types") CHECK_EQ("boolean", toString(err->table)); CHECK_EQ("x", err->key); - CHECK_EQ("*unknown*", toString(requireType("c"))); - CHECK_EQ("*unknown*", toString(requireType("d"))); - CHECK_EQ("*unknown*", toString(requireType("e"))); - CHECK_EQ("*unknown*", toString(requireType("f"))); + // TODO: Should we assert anything about these tests when DCR is being used? + if (!FFlag::DebugLuauDeferredConstraintResolution) + { + CHECK_EQ("*unknown*", toString(requireType("c"))); + CHECK_EQ("*unknown*", toString(requireType("d"))); + CHECK_EQ("*unknown*", toString(requireType("e"))); + CHECK_EQ("*unknown*", toString(requireType("f"))); + } } TEST_CASE_FIXTURE(Fixture, "should_be_able_to_infer_this_without_stack_overflowing") @@ -352,40 +355,6 @@ TEST_CASE_FIXTURE(Fixture, "check_expr_recursion_limit") CHECK(nullptr != get(result.errors[0])); } -TEST_CASE_FIXTURE(Fixture, "globals") -{ - CheckResult result = check(R"( - --!nonstrict - foo = true - foo = "now i'm a string!" - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - CHECK_EQ("any", toString(requireType("foo"))); -} - -TEST_CASE_FIXTURE(Fixture, "globals2") -{ - ScopedFastFlag sff[]{ - {"LuauReturnTypeInferenceInNonstrict", true}, - {"LuauLowerBoundsCalculation", true}, - }; - - CheckResult result = check(R"( - --!nonstrict - foo = function() return 1 end - foo = "now i'm a string!" - )"); - - LUAU_REQUIRE_ERROR_COUNT(1, result); - - TypeMismatch* tm = get(result.errors[0]); - REQUIRE(tm); - CHECK_EQ("() -> number", toString(tm->wantedType)); - CHECK_EQ("string", toString(tm->givenType)); - CHECK_EQ("() -> number", toString(requireType("foo"))); -} - TEST_CASE_FIXTURE(Fixture, "globals_are_banned_in_strict_mode") { CheckResult result = check(R"( @@ -400,23 +369,6 @@ TEST_CASE_FIXTURE(Fixture, "globals_are_banned_in_strict_mode") CHECK_EQ("foo", us->name); } -TEST_CASE_FIXTURE(Fixture, "globals_everywhere") -{ - CheckResult result = check(R"( - --!nonstrict - foo = 1 - - if true then - bar = 2 - end - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - - CHECK_EQ("any", toString(requireType("foo"))); - CHECK_EQ("any", toString(requireType("bar"))); -} - TEST_CASE_FIXTURE(BuiltinsFixture, "correctly_scope_locals_do") { CheckResult result = check(R"( @@ -447,21 +399,6 @@ TEST_CASE_FIXTURE(Fixture, "checking_should_not_ice") CHECK_EQ("any", toString(requireType("value"))); } -// TEST_CASE_FIXTURE(Fixture, "infer_method_signature_of_argument") -// { -// CheckResult result = check(R"( -// function f(a) -// if a.cond then -// return a.method() -// end -// end -// )"); - -// LUAU_REQUIRE_NO_ERRORS(result); - -// CHECK_EQ("A", toString(requireType("f"))); -// } - TEST_CASE_FIXTURE(Fixture, "cyclic_follow") { check(R"( diff --git a/tests/TypeInfer.typePacks.cpp b/tests/TypeInfer.typePacks.cpp index 118863fe..bcd30498 100644 --- a/tests/TypeInfer.typePacks.cpp +++ b/tests/TypeInfer.typePacks.cpp @@ -26,7 +26,7 @@ TEST_CASE_FIXTURE(Fixture, "infer_multi_return") const FunctionTypeVar* takeTwoType = get(requireType("take_two")); REQUIRE(takeTwoType != nullptr); - const auto& [returns, tail] = flatten(takeTwoType->retType); + const auto& [returns, tail] = flatten(takeTwoType->retTypes); CHECK_EQ(2, returns.size()); CHECK_EQ(typeChecker.numberType, follow(returns[0])); @@ -73,7 +73,7 @@ TEST_CASE_FIXTURE(Fixture, "last_element_of_return_statement_can_itself_be_a_pac const FunctionTypeVar* takeOneMoreType = get(requireType("take_three")); REQUIRE(takeOneMoreType != nullptr); - const auto& [rets, tail] = flatten(takeOneMoreType->retType); + const auto& [rets, tail] = flatten(takeOneMoreType->retTypes); REQUIRE_EQ(3, rets.size()); CHECK_EQ(typeChecker.numberType, follow(rets[0])); @@ -105,10 +105,10 @@ TEST_CASE_FIXTURE(Fixture, "return_type_should_be_empty_if_nothing_is_returned") LUAU_REQUIRE_NO_ERRORS(result); const FunctionTypeVar* fTy = get(requireType("f")); REQUIRE(fTy != nullptr); - CHECK_EQ(0, size(fTy->retType)); + CHECK_EQ(0, size(fTy->retTypes)); const FunctionTypeVar* gTy = get(requireType("g")); REQUIRE(gTy != nullptr); - CHECK_EQ(0, size(gTy->retType)); + CHECK_EQ(0, size(gTy->retTypes)); } TEST_CASE_FIXTURE(Fixture, "no_return_size_should_be_zero") @@ -125,15 +125,15 @@ TEST_CASE_FIXTURE(Fixture, "no_return_size_should_be_zero") const FunctionTypeVar* fTy = get(requireType("f")); REQUIRE(fTy != nullptr); - CHECK_EQ(1, size(follow(fTy->retType))); + CHECK_EQ(1, size(follow(fTy->retTypes))); const FunctionTypeVar* gTy = get(requireType("g")); REQUIRE(gTy != nullptr); - CHECK_EQ(0, size(gTy->retType)); + CHECK_EQ(0, size(gTy->retTypes)); const FunctionTypeVar* hTy = get(requireType("h")); REQUIRE(hTy != nullptr); - CHECK_EQ(0, size(hTy->retType)); + CHECK_EQ(0, size(hTy->retTypes)); } TEST_CASE_FIXTURE(Fixture, "varargs_inference_through_multiple_scopes") diff --git a/tools/natvis/Analysis.natvis b/tools/natvis/Analysis.natvis index 5de0140e..b9ea3141 100644 --- a/tools/natvis/Analysis.natvis +++ b/tools/natvis/Analysis.natvis @@ -6,40 +6,40 @@ - {{ index=0, value={*($T1*)storage} }} - {{ index=1, value={*($T2*)storage} }} - {{ index=2, value={*($T3*)storage} }} - {{ index=3, value={*($T4*)storage} }} - {{ index=4, value={*($T5*)storage} }} - {{ index=5, value={*($T6*)storage} }} - {{ index=6, value={*($T7*)storage} }} - {{ index=7, value={*($T8*)storage} }} - {{ index=8, value={*($T9*)storage} }} - {{ index=9, value={*($T10*)storage} }} - {{ index=10, value={*($T11*)storage} }} - {{ index=11, value={*($T12*)storage} }} - {{ index=12, value={*($T13*)storage} }} - {{ index=13, value={*($T14*)storage} }} - {{ index=14, value={*($T15*)storage} }} - {{ index=15, value={*($T16*)storage} }} - {{ index=16, value={*($T17*)storage} }} - {{ index=17, value={*($T18*)storage} }} - {{ index=18, value={*($T19*)storage} }} - {{ index=19, value={*($T20*)storage} }} - {{ index=20, value={*($T21*)storage} }} - {{ index=21, value={*($T22*)storage} }} - {{ index=22, value={*($T23*)storage} }} - {{ index=23, value={*($T24*)storage} }} - {{ index=24, value={*($T25*)storage} }} - {{ index=25, value={*($T26*)storage} }} - {{ index=26, value={*($T27*)storage} }} - {{ index=27, value={*($T28*)storage} }} - {{ index=28, value={*($T29*)storage} }} - {{ index=29, value={*($T30*)storage} }} - {{ index=30, value={*($T31*)storage} }} - {{ index=31, value={*($T32*)storage} }} + {{ typeId=0, value={*($T1*)storage} }} + {{ typeId=1, value={*($T2*)storage} }} + {{ typeId=2, value={*($T3*)storage} }} + {{ typeId=3, value={*($T4*)storage} }} + {{ typeId=4, value={*($T5*)storage} }} + {{ typeId=5, value={*($T6*)storage} }} + {{ typeId=6, value={*($T7*)storage} }} + {{ typeId=7, value={*($T8*)storage} }} + {{ typeId=8, value={*($T9*)storage} }} + {{ typeId=9, value={*($T10*)storage} }} + {{ typeId=10, value={*($T11*)storage} }} + {{ typeId=11, value={*($T12*)storage} }} + {{ typeId=12, value={*($T13*)storage} }} + {{ typeId=13, value={*($T14*)storage} }} + {{ typeId=14, value={*($T15*)storage} }} + {{ typeId=15, value={*($T16*)storage} }} + {{ typeId=16, value={*($T17*)storage} }} + {{ typeId=17, value={*($T18*)storage} }} + {{ typeId=18, value={*($T19*)storage} }} + {{ typeId=19, value={*($T20*)storage} }} + {{ typeId=20, value={*($T21*)storage} }} + {{ typeId=21, value={*($T22*)storage} }} + {{ typeId=22, value={*($T23*)storage} }} + {{ typeId=23, value={*($T24*)storage} }} + {{ typeId=24, value={*($T25*)storage} }} + {{ typeId=25, value={*($T26*)storage} }} + {{ typeId=26, value={*($T27*)storage} }} + {{ typeId=27, value={*($T28*)storage} }} + {{ typeId=28, value={*($T29*)storage} }} + {{ typeId=29, value={*($T30*)storage} }} + {{ typeId=30, value={*($T31*)storage} }} + {{ typeId=31, value={*($T32*)storage} }} - typeId + typeId *($T1*)storage *($T2*)storage *($T3*)storage From ce9f4e23ae1bb20cf6a1c017f3d15de1c66e97bf Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Tue, 21 Jun 2022 13:14:30 -0700 Subject: [PATCH 278/399] Update performance.md (#553) Document function inlining and loop unrolling. --- docs/_pages/performance.md | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/docs/_pages/performance.md b/docs/_pages/performance.md index 10e2341c..34b24b03 100644 --- a/docs/_pages/performance.md +++ b/docs/_pages/performance.md @@ -185,3 +185,13 @@ While large tables can be a problem for incremental GC in general since currentl The incremental garbage collector in Luau runs three phases for each cycle: mark, atomic and sweep. Mark incrementally traverses all live objects, atomic finishes various operations that need to happen without mutator intervention (see previous section), and sweep traverses all objects in the heap, reclaiming memory used by dead objects and performing minor fixup for live objects. While objects allocated during the mark phase are traversed in the same cycle and thus may get reclaimed, objects allocated during the sweep phase are considered live. Because of this, the faster the sweep phase completes, the less garbage will accumulate; and, of course, the less time sweeping takes the less overhead there is from this phase of garbage collection on the process. Since sweeping traverses the whole heap, we maximize the efficiency of this traversal by allocating garbage-collected objects of the same size in 16 KB pages, and traversing each page at a time, which is otherwise known as a paged sweeper. This ensures good locality of reference as consecutively swept objects are contiugous in memory, and allows us to spend no memory for each object on sweep-related data or allocation metadata, since paged sweeper doesn't need to be able to free objects without knowing which page they are in. Compared to linked list based sweeping that Lua/LuaJIT implement, paged sweeper is 2-3x faster, and saves 16 bytes per object on 64-bit platforms. + +## Function inlining and loop unrolling + +By default, the bytecode compiler performs a series of optimizations that result in faster execution of the code, but they preserve both execution semantics and debuggability. For example, a function call is compiled as a function call, which may be observable via `debug.traceback`; a loop is compiled as a loop, which may be observable via `lua_getlocal`. To help improve performance in cases where these restrictions can be relaxed, the bytecode compiler implements additional optimizations when optimization level 2 is enabled (which requires using `-O2` switch when using Luau CLI), namely function inlining and loop unrolling. + +Only loops with loop bounds known at compile time, such as `for i=1,4 do`, can be unrolled. The loop body must be simple enough for the optimization to be profitable; compiler uses heuristics to estimate the performance benefit and automatically decide if unrolling should be performed. + +Only local functions (defined either as `local function foo` or `local foo = function`) can be inlined. The function body must be simple enough for the optimization to be profitable; compiler uses heuristics to estimate the performance benefit and automatically decide if each call to the function should be inlined instead. Additionally recursive invocations of a function can’t be inlined at this time, and inlining is completely disabled for modules that use `getfenv`/`setfenv` functions. + +In both cases, in addition to removing the overhead associated with function calls or loop iteration, these optimizations can additionally benefit by enabling additional optimizations, such as constant folding of expressions dependent on loop iteration variable or constant function arguments, or using more efficient instructions for certain expressions when the inputs to these instructions are constants. From e0ac24d1ed4bb755f9d3b85718f6259fd76245e9 Mon Sep 17 00:00:00 2001 From: Qualadore <93345551+Qualadore@users.noreply.github.com> Date: Wed, 22 Jun 2022 08:01:34 -0800 Subject: [PATCH 279/399] Correct string.find and string.match return types (#554) --- Analysis/src/TypeVar.cpp | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/Analysis/src/TypeVar.cpp b/Analysis/src/TypeVar.cpp index 57762937..0f53f990 100644 --- a/Analysis/src/TypeVar.cpp +++ b/Analysis/src/TypeVar.cpp @@ -750,13 +750,15 @@ TypeId SingletonTypes::makeStringMetatable() TableTypeVar::Props stringLib = { {"byte", {arena->addType(FunctionTypeVar{arena->addTypePack({stringType, optionalNumber, optionalNumber}), numberVariadicList})}}, {"char", {arena->addType(FunctionTypeVar{numberVariadicList, arena->addTypePack({stringType})})}}, - {"find", {makeFunction(*arena, stringType, {}, {}, {stringType, optionalNumber, optionalBoolean}, {}, {optionalNumber, optionalNumber})}}, + {"find", {arena->addType(FunctionTypeVar{arena->addTypePack({stringType, stringType, optionalNumber, optionalBoolean}), + arena->addTypePack(TypePack{{optionalNumber, optionalNumber}, stringVariadicList})})}}, {"format", {formatFn}}, // FIXME {"gmatch", {gmatchFunc}}, {"gsub", {gsubFunc}}, {"len", {makeFunction(*arena, stringType, {}, {}, {}, {}, {numberType})}}, {"lower", {stringToStringType}}, - {"match", {makeFunction(*arena, stringType, {}, {}, {stringType, optionalNumber}, {}, {optionalString})}}, + {"match", {arena->addType(FunctionTypeVar{arena->addTypePack({stringType, stringType, optionalNumber}), + arena->addTypePack(TypePackVar{VariadicTypePack{optionalString}})})}}, {"rep", {makeFunction(*arena, stringType, {}, {}, {numberType}, {}, {stringType})}}, {"reverse", {stringToStringType}}, {"sub", {makeFunction(*arena, stringType, {}, {}, {numberType, optionalNumber}, {}, {stringType})}}, From ca32d1bf9de594736cf52f88cf32a621462db9e3 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Wed, 22 Jun 2022 09:27:05 -0700 Subject: [PATCH 280/399] Update library.md (#555) Fix string.match and string.find type definitions --- docs/_pages/library.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/_pages/library.md b/docs/_pages/library.md index eeada336..ff3075ac 100644 --- a/docs/_pages/library.md +++ b/docs/_pages/library.md @@ -488,7 +488,7 @@ function string.char(args: ...number): string Returns the string that contains a byte for every input number; all inputs must be integers in `[0..255]` range. ``` -function string.find(s: string, p: string, init: number?, plain: boolean?): (number?, number?) +function string.find(s: string, p: string, init: number?, plain: boolean?): (number?, number?, ...string) ``` Tries to find an instance of pattern `p` in the string `s`, starting from position `init` (defaults to 1). When `plain` is true, the search is using raw case-insensitive string equality, otherwise `p` should be a [string pattern](https://www.lua.org/manual/5.3/manual.html#6.4.1). If a match is found, returns the position of the match and the length of the match, followed by the pattern captures; otherwise returns `nil`. @@ -536,7 +536,7 @@ function string.lower(s: string): string Returns a string where each byte corresponds to the lower-case ASCII version of the input byte in the source string. ``` -function string.match(s: string, p: string, init: number?): (number?, number?) +function string.match(s: string, p: string, init: number?): ...string? ``` Tries to find an instance of pattern `p` in the string `s`, starting from position `init` (defaults to 1). `p` should be a [string pattern](https://www.lua.org/manual/5.3/manual.html#6.4.1). If a match is found, returns all pattern captures, or entire matching substring if no captures are present, otherwise returns `nil`. From 778e62c8f74f7cb2f48f119e70195a57b05e56bc Mon Sep 17 00:00:00 2001 From: Alan Jeffrey <403333+asajeffrey@users.noreply.github.com> Date: Wed, 22 Jun 2022 13:15:41 -0500 Subject: [PATCH 281/399] RFC: never and unknown types (#434) Co-authored-by: Alexander McCord <11488393+alexmccord@users.noreply.github.com> --- rfcs/none-and-unknown-types.md | 144 +++++++++++++++++++++++++++++++++ 1 file changed, 144 insertions(+) create mode 100644 rfcs/none-and-unknown-types.md diff --git a/rfcs/none-and-unknown-types.md b/rfcs/none-and-unknown-types.md new file mode 100644 index 00000000..0085ef07 --- /dev/null +++ b/rfcs/none-and-unknown-types.md @@ -0,0 +1,144 @@ +# Add never and unknown types + +## Summary + +Add `unknown` and `never` types that are inhabited by everything and nothing respectively. + +## Motivation + +There are lots of cases in local type inference, semantic subtyping, +and type normalization, where it would be useful to have top and +bottom types. Currently, `any` is filling that role, but it has +special "switch off the type system" superpowers. + +Any use of `unknown` must be narrowed by type refinements unless another `unknown` or `any` is expected. For +example a function which can return any value is: + +```lua + function anything() : unknown ... end +``` + +and can be used as: + +```lua + local x = anything() + if type(x) == "number" then + print(x + 1) + end +``` + +The type of this function cannot be given concisely in current +Luau. The nearest equivalent is `any`, but this switches off the type system, for example +if the type of `anything` is `() -> any` then the following code typechecks: + +```lua + local x = anything() + print(x + 1) +``` + +This is fine in nonstrict mode, but strict mode should flag this as an error. + +The `never` type comes up whenever type inference infers incompatible types for a variable, for example + +```lua + function oops(x) + print("hi " .. x) -- constrains x must be a string + print(math.abs(x)) -- constrains x must be a number + end +``` + +The most general type of `x` is `string & number`, so this code gives +a type error, but we still need to provide a type for `oops`. With a +`never` type, we can infer the type `oops : (never) -> ()`. + +or when exhaustive type casing is achieved: + +```lua + function f(x: string | number) + if type(x) == "string" then + -- x : string + elseif type(x) == "number" then + -- x : number + else + -- x : never + end + end +``` + +or even when the type casing is simply nonsensical: + +```lua + function f(x: string | number) + if type(x) == "string" and type(x) == "number" then + -- x : string & number which is never + end + end +``` + +The `never` type is also useful in cases such as tagged unions where +some of the cases are impossible. For example: + +```lua + type Result = { err: false, val: T } | { err: true, err: E } +``` + +For code which we know is successful, we would like to be able to +indicate that the error case is impossible. With a `never` type, we +can do this with `Result`. Similarly, code which cannot succeed +has type `Result`. + +These types can _almost_ be defined in current Luau, but only quite verbosely: + +```lua + type never = number & string + type unknown = nil | number | boolean | string | {} | (...never) -> (...unknown) +``` + +But even for `unknown` it is impossible to include every single data types, e.g. every root class. + +Providing `never` and `unknown` as built-in types makes the code for +type inference simpler, for example we have a way to present a union +type with no options (as `never`). Otherwise we have to contend with ad hoc +corner cases. + +## Design + +Add: + +* a type `never`, inhabited by nothing, and +* a type `unknown`, inhabited by everything. + +And under success types (nonstrict mode), `unknown` is exactly equivalent to `any` because `unknown` +encompasses everything as does `any`. + +The interesting thing is that `() -> (never, string)` is equivalent to `() -> never` because all +values in a pack must be inhabitable in order for the pack itself to also be inhabitable. In fact, +the type `() -> never` is not completely accurate, it should be `() -> (never, ...never)` to avoid +cascading type errors. Ditto for when an expression list `f(), g()` where the resulting type pack is +`(never, string, number)` is still the same as `(never, ...never)`. + +```lua + function f(): never error() end + function g(): string return "" end + + -- no cascading type error where count mismatches, because the expression list f(), g() + -- was made to return (never, ...never) due to the presence of a never type in the pack + local x, y, z = f(), g() + -- x : never + -- y : never + -- z : never +``` + +## Drawbacks + +Another bit of complexity budget spent. + +These types will be visible to creators, so yay bikeshedding! + +Replacing `any` with `unknown` is a breaking change: code in strict mode may now produce errors. + +## Alternatives + +Stick with the current use of `any` for these cases. + +Make `never` and `unknown` type aliases rather than built-ins. From 1757234f0154a893882ad3e927f7dc218273f155 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Wed, 22 Jun 2022 11:16:38 -0700 Subject: [PATCH 282/399] Rename none-and-unknown-types.md to never-and-unknown-types.md This makes the type names match. --- rfcs/{none-and-unknown-types.md => never-and-unknown-types.md} | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) rename rfcs/{none-and-unknown-types.md => never-and-unknown-types.md} (99%) diff --git a/rfcs/none-and-unknown-types.md b/rfcs/never-and-unknown-types.md similarity index 99% rename from rfcs/none-and-unknown-types.md rename to rfcs/never-and-unknown-types.md index 0085ef07..d996afc6 100644 --- a/rfcs/none-and-unknown-types.md +++ b/rfcs/never-and-unknown-types.md @@ -1,4 +1,4 @@ -# Add never and unknown types +# never and unknown types ## Summary From 348ad4d4176f1b8f5154cbbe725171154eafa9cb Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Wed, 22 Jun 2022 12:54:48 -0700 Subject: [PATCH 283/399] Update STATUS.md Add never and unknown types --- rfcs/STATUS.md | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/rfcs/STATUS.md b/rfcs/STATUS.md index d2fe86f0..6bfa865d 100644 --- a/rfcs/STATUS.md +++ b/rfcs/STATUS.md @@ -26,3 +26,9 @@ This document tracks unimplemented RFCs. [RFC: Lower bounds calculation](https://github.com/Roblox/luau/blob/master/rfcs/lower-bounds-calculation.md) **Status**: Implemented but not fully rolled out yet. + +## never and unknown types + +[RFC: never and unknown types](https://github.com/Roblox/luau/blob/master/rfcs/never-and-unknown-types.md) + +**Status**: Needs implementation From 6d14bdadf45c78b4aae695f636757d4cb4d65464 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Thu, 23 Jun 2022 18:44:07 -0700 Subject: [PATCH 284/399] Sync to upstream/release/533 --- Analysis/include/Luau/Constraint.h | 14 +- .../include/Luau/ConstraintGraphBuilder.h | 41 ++- Analysis/include/Luau/ConstraintSolver.h | 5 +- .../include/Luau/ConstraintSolverLogger.h | 4 +- Analysis/include/Luau/Error.h | 45 ++- Analysis/include/Luau/IostreamHelpers.h | 1 + Analysis/include/Luau/Module.h | 9 +- Analysis/include/Luau/Normalize.h | 4 +- Analysis/include/Luau/RecursionCounter.h | 15 +- Analysis/include/Luau/Scope.h | 18 + Analysis/include/Luau/TypeVar.h | 1 - Analysis/include/Luau/Unifiable.h | 1 + Analysis/include/Luau/Unifier.h | 4 - Analysis/include/Luau/VisitTypeVar.h | 2 +- Analysis/src/Clone.cpp | 4 +- Analysis/src/Constraint.cpp | 3 +- Analysis/src/ConstraintGraphBuilder.cpp | 285 +++++++++++++-- Analysis/src/ConstraintSolver.cpp | 35 +- Analysis/src/Error.cpp | 93 ++++- Analysis/src/Frontend.cpp | 2 + Analysis/src/IostreamHelpers.cpp | 2 + Analysis/src/Module.cpp | 1 - Analysis/src/Quantify.cpp | 9 +- Analysis/src/Scope.cpp | 32 ++ Analysis/src/ToString.cpp | 6 + Analysis/src/TypeChecker2.cpp | 173 +++++++++ Analysis/src/TypeInfer.cpp | 80 ++--- Analysis/src/TypeVar.cpp | 28 +- Analysis/src/Unifiable.cpp | 8 + Analysis/src/Unifier.cpp | 331 +----------------- CLI/Analyze.cpp | 4 + CMakeLists.txt | 9 + Common/include/Luau/Bytecode.h | 19 +- Compiler/include/Luau/BytecodeBuilder.h | 2 + Compiler/src/BytecodeBuilder.cpp | 16 +- Compiler/src/Compiler.cpp | 6 +- VM/src/ludata.cpp | 2 + VM/src/lvmload.cpp | 4 +- tests/Compiler.test.cpp | 34 +- tests/Fixture.h | 2 +- tests/Module.test.cpp | 1 - tests/Normalize.test.cpp | 1 - tests/RuntimeLimits.test.cpp | 7 +- tests/ToString.test.cpp | 2 - tests/TypeInfer.aliases.test.cpp | 87 ++++- tests/TypeInfer.annotations.test.cpp | 85 ++++- tests/TypeInfer.generics.test.cpp | 13 - tests/TypeInfer.modules.test.cpp | 20 +- tests/TypeInfer.refinements.test.cpp | 2 - tests/TypeInfer.singletons.test.cpp | 4 - tests/TypeInfer.tables.test.cpp | 73 ---- tests/TypeInfer.test.cpp | 3 - tests/TypeInfer.unionTypes.test.cpp | 6 - tests/VisitTypeVar.test.cpp | 9 +- 54 files changed, 972 insertions(+), 695 deletions(-) diff --git a/Analysis/include/Luau/Constraint.h b/Analysis/include/Luau/Constraint.h index c62166e2..8a41c9e8 100644 --- a/Analysis/include/Luau/Constraint.h +++ b/Analysis/include/Luau/Constraint.h @@ -1,10 +1,10 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #pragma once -#include "Luau/Location.h" #include "Luau/NotNull.h" #include "Luau/Variant.h" +#include #include #include @@ -47,18 +47,24 @@ struct InstantiationConstraint TypeId superType; }; -using ConstraintV = Variant; +// name(namedType) = name +struct NameConstraint +{ + TypeId namedType; + std::string name; +}; + +using ConstraintV = Variant; using ConstraintPtr = std::unique_ptr; struct Constraint { - Constraint(ConstraintV&& c, Location location); + explicit Constraint(ConstraintV&& c); Constraint(const Constraint&) = delete; Constraint& operator=(const Constraint&) = delete; ConstraintV c; - Location location; std::vector> dependencies; }; diff --git a/Analysis/include/Luau/ConstraintGraphBuilder.h b/Analysis/include/Luau/ConstraintGraphBuilder.h index da774a2a..9b118691 100644 --- a/Analysis/include/Luau/ConstraintGraphBuilder.h +++ b/Analysis/include/Luau/ConstraintGraphBuilder.h @@ -17,20 +17,7 @@ namespace Luau { -struct Scope2 -{ - // The parent scope of this scope. Null if there is no parent (i.e. this - // is the module-level scope). - Scope2* parent = nullptr; - // All the children of this scope. - std::vector children; - std::unordered_map bindings; // TODO: I think this can be a DenseHashMap - TypePackId returnType; - // All constraints belonging to this scope. - std::vector constraints; - - std::optional lookup(Symbol sym); -}; +struct Scope2; struct ConstraintGraphBuilder { @@ -47,6 +34,10 @@ struct ConstraintGraphBuilder // A mapping of AST node to TypePackId. DenseHashMap astTypePacks{nullptr}; DenseHashMap astOriginalCallTypes{nullptr}; + // Types resolved from type annotations. Analogous to astTypes. + DenseHashMap astResolvedTypes{nullptr}; + // Type packs resolved from type annotations. Analogous to astTypePacks. + DenseHashMap astResolvedTypePacks{nullptr}; explicit ConstraintGraphBuilder(TypeArena* arena); @@ -73,9 +64,8 @@ struct ConstraintGraphBuilder * Adds a new constraint with no dependencies to a given scope. * @param scope the scope to add the constraint to. Must not be null. * @param cv the constraint variant to add. - * @param location the location to attribute to the constraint. */ - void addConstraint(Scope2* scope, ConstraintV cv, Location location); + void addConstraint(Scope2* scope, ConstraintV cv); /** * Adds a constraint to a given scope. @@ -99,6 +89,7 @@ struct ConstraintGraphBuilder void visit(Scope2* scope, AstStatReturn* ret); void visit(Scope2* scope, AstStatAssign* assign); void visit(Scope2* scope, AstStatIf* ifStatement); + void visit(Scope2* scope, AstStatTypeAlias* alias); TypePackId checkExprList(Scope2* scope, const AstArray& exprs); @@ -124,6 +115,24 @@ struct ConstraintGraphBuilder * @param fn the function expression to check. */ void checkFunctionBody(Scope2* scope, AstExprFunction* fn); + + /** + * Resolves a type from its AST annotation. + * @param scope the scope that the type annotation appears within. + * @param ty the AST annotation to resolve. + * @return the type of the AST annotation. + **/ + TypeId resolveType(Scope2* scope, AstType* ty); + + /** + * Resolves a type pack from its AST annotation. + * @param scope the scope that the type annotation appears within. + * @param tp the AST annotation to resolve. + * @return the type pack of the AST annotation. + **/ + TypePackId resolveTypePack(Scope2* scope, AstTypePack* tp); + + TypePackId resolveTypePack(Scope2* scope, const AstTypeList& list); }; /** diff --git a/Analysis/include/Luau/ConstraintSolver.h b/Analysis/include/Luau/ConstraintSolver.h index 7e6d4461..4870157f 100644 --- a/Analysis/include/Luau/ConstraintSolver.h +++ b/Analysis/include/Luau/ConstraintSolver.h @@ -55,6 +55,7 @@ struct ConstraintSolver bool tryDispatch(const PackSubtypeConstraint& c, NotNull constraint, bool force); bool tryDispatch(const GeneralizationConstraint& c, NotNull constraint, bool force); bool tryDispatch(const InstantiationConstraint& c, NotNull constraint, bool force); + bool tryDispatch(const NameConstraint& c, NotNull constraint); void block(NotNull target, NotNull constraint); /** @@ -85,7 +86,7 @@ struct ConstraintSolver * @param subType the sub-type to unify. * @param superType the super-type to unify. */ - void unify(TypeId subType, TypeId superType, Location location); + void unify(TypeId subType, TypeId superType); /** * Creates a new Unifier and performs a single unification operation. Commits @@ -93,7 +94,7 @@ struct ConstraintSolver * @param subPack the sub-type pack to unify. * @param superPack the super-type pack to unify. */ - void unify(TypePackId subPack, TypePackId superPack, Location location); + void unify(TypePackId subPack, TypePackId superPack); private: /** diff --git a/Analysis/include/Luau/ConstraintSolverLogger.h b/Analysis/include/Luau/ConstraintSolverLogger.h index 2b195d71..55336a23 100644 --- a/Analysis/include/Luau/ConstraintSolverLogger.h +++ b/Analysis/include/Luau/ConstraintSolverLogger.h @@ -1,6 +1,8 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details -#include "Luau/ConstraintGraphBuilder.h" +#include "Luau/Constraint.h" +#include "Luau/NotNull.h" +#include "Luau/Scope.h" #include "Luau/ToString.h" #include diff --git a/Analysis/include/Luau/Error.h b/Analysis/include/Luau/Error.h index b4530674..a1323960 100644 --- a/Analysis/include/Luau/Error.h +++ b/Analysis/include/Luau/Error.h @@ -169,6 +169,13 @@ struct GenericError bool operator==(const GenericError& rhs) const; }; +struct InternalError +{ + std::string message; + + bool operator==(const InternalError& rhs) const; +}; + struct CannotCallNonFunction { TypeId ty; @@ -293,12 +300,12 @@ struct NormalizationTooComplex } }; -using TypeErrorData = - Variant; +using TypeErrorData = Variant; struct TypeError { @@ -339,7 +346,13 @@ T* get(TypeError& e) using ErrorVec = std::vector; +struct TypeErrorToStringOptions +{ + FileResolver* fileResolver = nullptr; +}; + std::string toString(const TypeError& error); +std::string toString(const TypeError& error, TypeErrorToStringOptions options); bool containsParseErrorName(const TypeError& error); @@ -356,4 +369,24 @@ struct InternalErrorReporter [[noreturn]] void ice(const std::string& message); }; +class InternalCompilerError : public std::exception { +public: + explicit InternalCompilerError(const std::string& message, const std::string& moduleName) + : message(message) + , moduleName(moduleName) + { + } + explicit InternalCompilerError(const std::string& message, const std::string& moduleName, const Location& location) + : message(message) + , moduleName(moduleName) + , location(location) + { + } + virtual const char* what() const throw(); + + const std::string message; + const std::string moduleName; + const std::optional location; +}; + } // namespace Luau diff --git a/Analysis/include/Luau/IostreamHelpers.h b/Analysis/include/Luau/IostreamHelpers.h index ee994296..05b94516 100644 --- a/Analysis/include/Luau/IostreamHelpers.h +++ b/Analysis/include/Luau/IostreamHelpers.h @@ -30,6 +30,7 @@ std::ostream& operator<<(std::ostream& lhs, const OccursCheckFailed& error); std::ostream& operator<<(std::ostream& lhs, const UnknownRequire& error); std::ostream& operator<<(std::ostream& lhs, const UnknownPropButFoundLikeProp& e); std::ostream& operator<<(std::ostream& lhs, const GenericError& error); +std::ostream& operator<<(std::ostream& lhs, const InternalError& error); std::ostream& operator<<(std::ostream& lhs, const FunctionExitsWithoutReturning& error); std::ostream& operator<<(std::ostream& lhs, const MissingProperties& error); std::ostream& operator<<(std::ostream& lhs, const IllegalRequire& error); diff --git a/Analysis/include/Luau/Module.h b/Analysis/include/Luau/Module.h index e979b3f0..39f8dfb7 100644 --- a/Analysis/include/Luau/Module.h +++ b/Analysis/include/Luau/Module.h @@ -1,10 +1,11 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #pragma once +#include "Luau/Error.h" #include "Luau/FileResolver.h" #include "Luau/ParseOptions.h" -#include "Luau/Error.h" #include "Luau/ParseResult.h" +#include "Luau/Scope.h" #include "Luau/TypeArena.h" #include @@ -19,7 +20,9 @@ struct Module; using ScopePtr = std::shared_ptr; using ModulePtr = std::shared_ptr; -struct Scope2; + +class AstType; +class AstTypePack; /// Root of the AST of a parsed source file struct SourceModule @@ -73,6 +76,8 @@ struct Module DenseHashMap astExpectedTypes{nullptr}; DenseHashMap astOriginalCallTypes{nullptr}; DenseHashMap astOverloadResolvedTypes{nullptr}; + DenseHashMap astResolvedTypes{nullptr}; + DenseHashMap astResolvedTypePacks{nullptr}; std::unordered_map declaredGlobals; ErrorVec errors; diff --git a/Analysis/include/Luau/Normalize.h b/Analysis/include/Luau/Normalize.h index d4c7698b..f5fd9886 100644 --- a/Analysis/include/Luau/Normalize.h +++ b/Analysis/include/Luau/Normalize.h @@ -9,8 +9,8 @@ namespace Luau struct InternalErrorReporter; -bool isSubtype(TypeId superTy, TypeId subTy, InternalErrorReporter& ice); -bool isSubtype(TypePackId superTy, TypePackId subTy, InternalErrorReporter& ice); +bool isSubtype(TypeId subTy, TypeId superTy, InternalErrorReporter& ice); +bool isSubtype(TypePackId subTy, TypePackId superTy, InternalErrorReporter& ice); std::pair normalize(TypeId ty, TypeArena& arena, InternalErrorReporter& ice); std::pair normalize(TypeId ty, const ModulePtr& module, InternalErrorReporter& ice); diff --git a/Analysis/include/Luau/RecursionCounter.h b/Analysis/include/Luau/RecursionCounter.h index 03ae2c83..f964dbfe 100644 --- a/Analysis/include/Luau/RecursionCounter.h +++ b/Analysis/include/Luau/RecursionCounter.h @@ -6,8 +6,6 @@ #include #include -LUAU_FASTFLAG(LuauRecursionLimitException); - namespace Luau { @@ -39,21 +37,12 @@ private: struct RecursionLimiter : RecursionCounter { - // TODO: remove ctx after LuauRecursionLimitException is removed - RecursionLimiter(int* count, int limit, const char* ctx) + RecursionLimiter(int* count, int limit) : RecursionCounter(count) { - LUAU_ASSERT(ctx); if (limit > 0 && *count > limit) { - if (FFlag::LuauRecursionLimitException) - throw RecursionLimitException(); - else - { - std::string m = "Internal recursion counter limit exceeded: "; - m += ctx; - throw std::runtime_error(m); - } + throw RecursionLimitException(); } } }; diff --git a/Analysis/include/Luau/Scope.h b/Analysis/include/Luau/Scope.h index 45338409..cef4b94f 100644 --- a/Analysis/include/Luau/Scope.h +++ b/Analysis/include/Luau/Scope.h @@ -1,6 +1,7 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #pragma once +#include "Luau/Constraint.h" #include "Luau/Location.h" #include "Luau/TypeVar.h" @@ -64,4 +65,21 @@ struct Scope std::unordered_map typeAliasTypePackParameters; }; +struct Scope2 +{ + // The parent scope of this scope. Null if there is no parent (i.e. this + // is the module-level scope). + Scope2* parent = nullptr; + // All the children of this scope. + std::vector children; + std::unordered_map bindings; // TODO: I think this can be a DenseHashMap + std::unordered_map typeBindings; + TypePackId returnType; + // All constraints belonging to this scope. + std::vector constraints; + + std::optional lookup(Symbol sym); + std::optional lookupTypeBinding(const Name& name); +}; + } // namespace Luau diff --git a/Analysis/include/Luau/TypeVar.h b/Analysis/include/Luau/TypeVar.h index ff7708d4..20f4107c 100644 --- a/Analysis/include/Luau/TypeVar.h +++ b/Analysis/include/Luau/TypeVar.h @@ -287,7 +287,6 @@ struct FunctionTypeVar bool hasSelf; Tags tags; bool hasNoGenerics = false; - bool generalized = false; }; enum class TableState diff --git a/Analysis/include/Luau/Unifiable.h b/Analysis/include/Luau/Unifiable.h index fdc39481..4ff91714 100644 --- a/Analysis/include/Luau/Unifiable.h +++ b/Analysis/include/Luau/Unifiable.h @@ -117,6 +117,7 @@ struct Generic explicit Generic(const Name& name); explicit Generic(Scope2* scope); Generic(TypeLevel level, const Name& name); + Generic(Scope2* scope, const Name& name); int index; TypeLevel level; diff --git a/Analysis/include/Luau/Unifier.h b/Analysis/include/Luau/Unifier.h index b51a485e..4af324cb 100644 --- a/Analysis/include/Luau/Unifier.h +++ b/Analysis/include/Luau/Unifier.h @@ -79,12 +79,8 @@ private: void tryUnifySingletons(TypeId subTy, TypeId superTy); void tryUnifyFunctions(TypeId subTy, TypeId superTy, bool isFunctionCall = false); void tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection = false); - void DEPRECATED_tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection = false); - void tryUnifyFreeTable(TypeId subTy, TypeId superTy); - void tryUnifySealedTables(TypeId subTy, TypeId superTy, bool isIntersection); void tryUnifyWithMetatable(TypeId subTy, TypeId superTy, bool reversed); void tryUnifyWithClass(TypeId subTy, TypeId superTy, bool reversed); - void tryUnifyIndexer(const TableIndexer& subIndexer, const TableIndexer& superIndexer); TypeId widen(TypeId ty); TypePackId widen(TypePackId tp); diff --git a/Analysis/include/Luau/VisitTypeVar.h b/Analysis/include/Luau/VisitTypeVar.h index 642522c9..5fd43f0b 100644 --- a/Analysis/include/Luau/VisitTypeVar.h +++ b/Analysis/include/Luau/VisitTypeVar.h @@ -169,7 +169,7 @@ struct GenericTypeVarVisitor void traverse(TypeId ty) { - RecursionLimiter limiter{&recursionCounter, FInt::LuauVisitRecursionLimit, "TypeVarVisitor"}; + RecursionLimiter limiter{&recursionCounter, FInt::LuauVisitRecursionLimit}; if (visit_detail::hasSeen(seen, ty)) { diff --git a/Analysis/src/Clone.cpp b/Analysis/src/Clone.cpp index 248262ce..df4e0a6b 100644 --- a/Analysis/src/Clone.cpp +++ b/Analysis/src/Clone.cpp @@ -317,7 +317,7 @@ TypePackId clone(TypePackId tp, TypeArena& dest, CloneState& cloneState) if (tp->persistent) return tp; - RecursionLimiter _ra(&cloneState.recursionCount, FInt::LuauTypeCloneRecursionLimit, "cloning TypePackId"); + RecursionLimiter _ra(&cloneState.recursionCount, FInt::LuauTypeCloneRecursionLimit); TypePackId& res = cloneState.seenTypePacks[tp]; @@ -335,7 +335,7 @@ TypeId clone(TypeId typeId, TypeArena& dest, CloneState& cloneState) if (typeId->persistent) return typeId; - RecursionLimiter _ra(&cloneState.recursionCount, FInt::LuauTypeCloneRecursionLimit, "cloning TypeId"); + RecursionLimiter _ra(&cloneState.recursionCount, FInt::LuauTypeCloneRecursionLimit); TypeId& res = cloneState.seenTypes[typeId]; diff --git a/Analysis/src/Constraint.cpp b/Analysis/src/Constraint.cpp index 6cb0e4ee..64e3a666 100644 --- a/Analysis/src/Constraint.cpp +++ b/Analysis/src/Constraint.cpp @@ -5,9 +5,8 @@ namespace Luau { -Constraint::Constraint(ConstraintV&& c, Location location) +Constraint::Constraint(ConstraintV&& c) : c(std::move(c)) - , location(location) { } diff --git a/Analysis/src/ConstraintGraphBuilder.cpp b/Analysis/src/ConstraintGraphBuilder.cpp index fa627e7a..d9e8d238 100644 --- a/Analysis/src/ConstraintGraphBuilder.cpp +++ b/Analysis/src/ConstraintGraphBuilder.cpp @@ -2,28 +2,13 @@ #include "Luau/ConstraintGraphBuilder.h" +#include "Luau/Scope.h" + namespace Luau { const AstStat* getFallthrough(const AstStat* node); // TypeInfer.cpp -std::optional Scope2::lookup(Symbol sym) -{ - Scope2* s = this; - - while (true) - { - auto it = s->bindings.find(sym); - if (it != s->bindings.end()) - return it->second; - - if (s->parent) - s = s->parent; - else - return std::nullopt; - } -} - ConstraintGraphBuilder::ConstraintGraphBuilder(TypeArena* arena) : singletonTypes(getSingletonTypes()) , arena(arena) @@ -59,10 +44,10 @@ Scope2* ConstraintGraphBuilder::childScope(Location location, Scope2* parent) return borrow; } -void ConstraintGraphBuilder::addConstraint(Scope2* scope, ConstraintV cv, Location location) +void ConstraintGraphBuilder::addConstraint(Scope2* scope, ConstraintV cv) { LUAU_ASSERT(scope); - scope->constraints.emplace_back(new Constraint{std::move(cv), location}); + scope->constraints.emplace_back(new Constraint{std::move(cv)}); } void ConstraintGraphBuilder::addConstraint(Scope2* scope, std::unique_ptr c) @@ -79,6 +64,13 @@ void ConstraintGraphBuilder::visit(AstStatBlock* block) rootScope = scopes.back().second.get(); rootScope->returnType = freshTypePack(rootScope); + // TODO: We should share the global scope. + rootScope->typeBindings["nil"] = singletonTypes.nilType; + rootScope->typeBindings["number"] = singletonTypes.numberType; + rootScope->typeBindings["string"] = singletonTypes.stringType; + rootScope->typeBindings["boolean"] = singletonTypes.booleanType; + rootScope->typeBindings["thread"] = singletonTypes.threadType; + visit(rootScope, block); } @@ -102,6 +94,8 @@ void ConstraintGraphBuilder::visit(Scope2* scope, AstStat* stat) checkPack(scope, e->expr); else if (auto i = stat->as()) visit(scope, i); + else if (auto a = stat->as()) + visit(scope, a); else LUAU_ASSERT(0); } @@ -114,8 +108,14 @@ void ConstraintGraphBuilder::visit(Scope2* scope, AstStatLocal* local) for (AstLocal* local : local->vars) { - // TODO annotations TypeId ty = freshType(scope); + + if (local->annotation) + { + TypeId annotation = resolveType(scope, local->annotation); + addConstraint(scope, SubtypeConstraint{ty, annotation}); + } + varTypes.push_back(ty); scope->bindings[local] = ty; } @@ -136,14 +136,14 @@ void ConstraintGraphBuilder::visit(Scope2* scope, AstStatLocal* local) { std::vector tailValues{varTypes.begin() + i, varTypes.end()}; TypePackId tailPack = arena->addTypePack(std::move(tailValues)); - addConstraint(scope, PackSubtypeConstraint{exprPack, tailPack}, local->location); + addConstraint(scope, PackSubtypeConstraint{exprPack, tailPack}); } } else { TypeId exprType = check(scope, local->values.data[i]); if (i < varTypes.size()) - addConstraint(scope, SubtypeConstraint{varTypes[i], exprType}, local->vars.data[i]->location); + addConstraint(scope, SubtypeConstraint{varTypes[i], exprType}); } } } @@ -188,7 +188,7 @@ void ConstraintGraphBuilder::visit(Scope2* scope, AstStatLocalFunction* function checkFunctionBody(innerScope, function->func); - std::unique_ptr c{new Constraint{GeneralizationConstraint{functionType, actualFunctionType, innerScope}, function->location}}; + std::unique_ptr c{new Constraint{GeneralizationConstraint{functionType, actualFunctionType, innerScope}}}; addConstraints(c.get(), innerScope); addConstraint(scope, std::move(c)); @@ -240,7 +240,7 @@ void ConstraintGraphBuilder::visit(Scope2* scope, AstStatFunction* function) checkFunctionBody(innerScope, function->func); - std::unique_ptr c{new Constraint{GeneralizationConstraint{functionType, actualFunctionType, innerScope}, function->location}}; + std::unique_ptr c{new Constraint{GeneralizationConstraint{functionType, actualFunctionType, innerScope}}}; addConstraints(c.get(), innerScope); addConstraint(scope, std::move(c)); @@ -251,13 +251,26 @@ void ConstraintGraphBuilder::visit(Scope2* scope, AstStatReturn* ret) LUAU_ASSERT(scope); TypePackId exprTypes = checkPack(scope, ret->list); - addConstraint(scope, PackSubtypeConstraint{exprTypes, scope->returnType}, ret->location); + addConstraint(scope, PackSubtypeConstraint{exprTypes, scope->returnType}); } void ConstraintGraphBuilder::visit(Scope2* scope, AstStatBlock* block) { LUAU_ASSERT(scope); + // In order to enable mutually-recursive type aliases, we need to + // populate the type bindings before we actually check any of the + // alias statements. Since we're not ready to actually resolve + // any of the annotations, we just use a fresh type for now. + for (AstStat* stat : block->body) + { + if (auto alias = stat->as()) + { + TypeId initialType = freshType(scope); + scope->typeBindings[alias->name.value] = initialType; + } + } + for (AstStat* stat : block->body) visit(scope, stat); } @@ -267,7 +280,7 @@ void ConstraintGraphBuilder::visit(Scope2* scope, AstStatAssign* assign) TypePackId varPackId = checkExprList(scope, assign->vars); TypePackId valuePack = checkPack(scope, assign->values); - addConstraint(scope, PackSubtypeConstraint{valuePack, varPackId}, assign->location); + addConstraint(scope, PackSubtypeConstraint{valuePack, varPackId}); } void ConstraintGraphBuilder::visit(Scope2* scope, AstStatIf* ifStatement) @@ -284,6 +297,28 @@ void ConstraintGraphBuilder::visit(Scope2* scope, AstStatIf* ifStatement) } } +void ConstraintGraphBuilder::visit(Scope2* scope, AstStatTypeAlias* alias) +{ + // TODO: Exported type aliases + // TODO: Generic type aliases + + auto it = scope->typeBindings.find(alias->name.value); + // This should always be here since we do a separate pass over the + // AST to set up typeBindings. If it's not, we've somehow skipped + // this alias in that first pass. + LUAU_ASSERT(it != scope->typeBindings.end()); + + TypeId ty = resolveType(scope, alias->type); + + // Rather than using a subtype constraint, we instead directly bind + // the free type we generated in the first pass to the resolved type. + // This prevents a case where you could cause another constraint to + // bind the free alias type to an unrelated type, causing havoc. + asMutable(it->second)->ty.emplace(ty); + + addConstraint(scope, NameConstraint{ty, alias->name.value}); +} + TypePackId ConstraintGraphBuilder::checkPack(Scope2* scope, AstArray exprs) { LUAU_ASSERT(scope); @@ -350,13 +385,13 @@ TypePackId ConstraintGraphBuilder::checkPack(Scope2* scope, AstExpr* expr) astOriginalCallTypes[call->func] = fnType; TypeId instantiatedType = freshType(scope); - addConstraint(scope, InstantiationConstraint{instantiatedType, fnType}, expr->location); + addConstraint(scope, InstantiationConstraint{instantiatedType, fnType}); TypePackId rets = freshTypePack(scope); FunctionTypeVar ftv(arena->addTypePack(TypePack{args, {}}), rets); TypeId inferredFnType = arena->addType(ftv); - addConstraint(scope, SubtypeConstraint{inferredFnType, instantiatedType}, expr->location); + addConstraint(scope, SubtypeConstraint{inferredFnType, instantiatedType}); result = rets; } else @@ -413,7 +448,7 @@ TypeId ConstraintGraphBuilder::check(Scope2* scope, AstExpr* expr) TypePack onePack{{typeResult}, freshTypePack(scope)}; TypePackId oneTypePack = arena->addTypePack(std::move(onePack)); - addConstraint(scope, PackSubtypeConstraint{packResult, oneTypePack}, expr->location); + addConstraint(scope, PackSubtypeConstraint{packResult, oneTypePack}); return typeResult; } @@ -454,7 +489,7 @@ TypeId ConstraintGraphBuilder::check(Scope2* scope, AstExprIndexName* indexName) TypeId expectedTableType = arena->addType(std::move(ttv)); - addConstraint(scope, SubtypeConstraint{obj, expectedTableType}, indexName->location); + addConstraint(scope, SubtypeConstraint{obj, expectedTableType}); return result; } @@ -465,8 +500,7 @@ TypeId ConstraintGraphBuilder::checkExprTable(Scope2* scope, AstExprTable* expr) TableTypeVar* ttv = getMutable(ty); LUAU_ASSERT(ttv); - auto createIndexer = [this, scope, ttv]( - TypeId currentIndexType, TypeId currentResultType, Location itemLocation, std::optional keyLocation) { + auto createIndexer = [this, scope, ttv](TypeId currentIndexType, TypeId currentResultType) { if (!ttv->indexer) { TypeId indexType = this->freshType(scope); @@ -474,8 +508,8 @@ TypeId ConstraintGraphBuilder::checkExprTable(Scope2* scope, AstExprTable* expr) ttv->indexer = TableIndexer{indexType, resultType}; } - addConstraint(scope, SubtypeConstraint{ttv->indexer->indexType, currentIndexType}, keyLocation ? *keyLocation : itemLocation); - addConstraint(scope, SubtypeConstraint{ttv->indexer->indexResultType, currentResultType}, itemLocation); + addConstraint(scope, SubtypeConstraint{ttv->indexer->indexType, currentIndexType}); + addConstraint(scope, SubtypeConstraint{ttv->indexer->indexResultType, currentResultType}); }; for (const AstExprTable::Item& item : expr->items) @@ -495,13 +529,13 @@ TypeId ConstraintGraphBuilder::checkExprTable(Scope2* scope, AstExprTable* expr) } else { - createIndexer(keyTy, itemTy, item.value->location, item.key->location); + createIndexer(keyTy, itemTy); } } else { TypeId numberType = singletonTypes.numberType; - createIndexer(numberType, itemTy, item.value->location, std::nullopt); + createIndexer(numberType, itemTy); } } @@ -514,15 +548,29 @@ std::pair ConstraintGraphBuilder::checkFunctionSignature(Scope2 TypePackId returnType = freshTypePack(innerScope); innerScope->returnType = returnType; + if (fn->returnAnnotation) + { + TypePackId annotatedRetType = resolveTypePack(innerScope, *fn->returnAnnotation); + addConstraint(innerScope, PackSubtypeConstraint{returnType, annotatedRetType}); + } + std::vector argTypes; for (AstLocal* local : fn->args) { TypeId t = freshType(innerScope); argTypes.push_back(t); - innerScope->bindings[local] = t; // TODO annotations + innerScope->bindings[local] = t; + + if (local->annotation) + { + TypeId argAnnotation = resolveType(innerScope, local->annotation); + addConstraint(innerScope, SubtypeConstraint{t, argAnnotation}); + } } + // TODO: Vararg annotation. + FunctionTypeVar actualFunction{arena->addTypePack(argTypes), returnType}; TypeId actualFunctionType = arena->addType(std::move(actualFunction)); LUAU_ASSERT(actualFunctionType); @@ -541,10 +589,171 @@ void ConstraintGraphBuilder::checkFunctionBody(Scope2* scope, AstExprFunction* f if (nullptr != getFallthrough(fn->body)) { TypePackId empty = arena->addTypePack({}); // TODO we could have CSG retain one of these forever - addConstraint(scope, PackSubtypeConstraint{scope->returnType, empty}, fn->body->location); + addConstraint(scope, PackSubtypeConstraint{scope->returnType, empty}); } } +TypeId ConstraintGraphBuilder::resolveType(Scope2* scope, AstType* ty) +{ + TypeId result = nullptr; + + if (auto ref = ty->as()) + { + // TODO: Support imported types w/ require tracing. + // TODO: Support generic type references. + LUAU_ASSERT(!ref->prefix); + LUAU_ASSERT(!ref->hasParameterList); + + // TODO: If it doesn't exist, should we introduce a free binding? + // This is probably important for handling type aliases. + result = scope->lookupTypeBinding(ref->name.value).value_or(singletonTypes.errorRecoveryType()); + } + else if (auto tab = ty->as()) + { + TableTypeVar::Props props; + std::optional indexer; + + for (const AstTableProp& prop : tab->props) + { + std::string name = prop.name.value; + // TODO: Recursion limit. + TypeId propTy = resolveType(scope, prop.type); + // TODO: Fill in location. + props[name] = {propTy}; + } + + if (tab->indexer) + { + // TODO: Recursion limit. + indexer = TableIndexer{ + resolveType(scope, tab->indexer->indexType), + resolveType(scope, tab->indexer->resultType), + }; + } + + // TODO: Remove TypeLevel{} here, we don't need it. + result = arena->addType(TableTypeVar{props, indexer, TypeLevel{}, TableState::Sealed}); + } + else if (auto fn = ty->as()) + { + // TODO: Generic functions. + // TODO: Scope (though it may not be needed). + // TODO: Recursion limit. + TypePackId argTypes = resolveTypePack(scope, fn->argTypes); + TypePackId returnTypes = resolveTypePack(scope, fn->returnTypes); + + // TODO: Is this the right constructor to use? + result = arena->addType(FunctionTypeVar{argTypes, returnTypes}); + + FunctionTypeVar* ftv = getMutable(result); + ftv->argNames.reserve(fn->argNames.size); + for (const auto& el : fn->argNames) + { + if (el) + { + const auto& [name, location] = *el; + ftv->argNames.push_back(FunctionArgument{name.value, location}); + } + else + { + ftv->argNames.push_back(std::nullopt); + } + } + } + else if (auto tof = ty->as()) + { + // TODO: Recursion limit. + TypeId exprType = check(scope, tof->expr); + result = exprType; + } + else if (auto unionAnnotation = ty->as()) + { + std::vector parts; + for (AstType* part : unionAnnotation->types) + { + // TODO: Recursion limit. + parts.push_back(resolveType(scope, part)); + } + + result = arena->addType(UnionTypeVar{parts}); + } + else if (auto intersectionAnnotation = ty->as()) + { + std::vector parts; + for (AstType* part : intersectionAnnotation->types) + { + // TODO: Recursion limit. + parts.push_back(resolveType(scope, part)); + } + + result = arena->addType(IntersectionTypeVar{parts}); + } + else if (auto boolAnnotation = ty->as()) + { + result = arena->addType(SingletonTypeVar(BooleanSingleton{boolAnnotation->value})); + } + else if (auto stringAnnotation = ty->as()) + { + result = arena->addType(SingletonTypeVar(StringSingleton{std::string(stringAnnotation->value.data, stringAnnotation->value.size)})); + } + else if (ty->is()) + { + result = singletonTypes.errorRecoveryType(); + } + else + { + LUAU_ASSERT(0); + result = singletonTypes.errorRecoveryType(); + } + + astResolvedTypes[ty] = result; + return result; +} + +TypePackId ConstraintGraphBuilder::resolveTypePack(Scope2* scope, AstTypePack* tp) +{ + TypePackId result; + if (auto expl = tp->as()) + { + result = resolveTypePack(scope, expl->typeList); + } + else if (auto var = tp->as()) + { + TypeId ty = resolveType(scope, var->variadicType); + result = arena->addTypePack(TypePackVar{VariadicTypePack{ty}}); + } + else if (auto gen = tp->as()) + { + result = arena->addTypePack(TypePackVar{GenericTypePack{scope, gen->genericName.value}}); + } + else + { + LUAU_ASSERT(0); + result = singletonTypes.errorRecoveryTypePack(); + } + + astResolvedTypePacks[tp] = result; + return result; +} + +TypePackId ConstraintGraphBuilder::resolveTypePack(Scope2* scope, const AstTypeList& list) +{ + std::vector head; + + for (AstType* headTy : list.types) + { + head.push_back(resolveType(scope, headTy)); + } + + std::optional tail = std::nullopt; + if (list.tailType) + { + tail = resolveTypePack(scope, list.tailType); + } + + return arena->addTypePack(TypePack{head, tail}); +} + void collectConstraints(std::vector>& result, Scope2* scope) { for (const auto& c : scope->constraints) diff --git a/Analysis/src/ConstraintSolver.cpp b/Analysis/src/ConstraintSolver.cpp index 41dfd892..9e355236 100644 --- a/Analysis/src/ConstraintSolver.cpp +++ b/Analysis/src/ConstraintSolver.cpp @@ -2,6 +2,7 @@ #include "Luau/ConstraintSolver.h" #include "Luau/Instantiation.h" +#include "Luau/Location.h" #include "Luau/Quantify.h" #include "Luau/ToString.h" #include "Luau/Unifier.h" @@ -179,6 +180,8 @@ bool ConstraintSolver::tryDispatch(NotNull constraint, bool fo success = tryDispatch(*gc, constraint, force); else if (auto ic = get(*constraint)) success = tryDispatch(*ic, constraint, force); + else if (auto nc = get(*constraint)) + success = tryDispatch(*nc, constraint); else LUAU_ASSERT(0); @@ -197,7 +200,7 @@ bool ConstraintSolver::tryDispatch(const SubtypeConstraint& c, NotNulllocation); + unify(c.subType, c.superType); unblock(c.subType); unblock(c.superType); @@ -207,7 +210,7 @@ bool ConstraintSolver::tryDispatch(const SubtypeConstraint& c, NotNull constraint, bool force) { - unify(c.subPack, c.superPack, constraint->location); + unify(c.subPack, c.superPack); unblock(c.subPack); unblock(c.superPack); @@ -222,7 +225,7 @@ bool ConstraintSolver::tryDispatch(const GeneralizationConstraint& c, NotNullty.emplace(c.sourceType); else - unify(c.generalizedType, c.sourceType, constraint->location); + unify(c.generalizedType, c.sourceType); TypeId generalized = quantify(arena, c.sourceType, c.scope); *asMutable(c.sourceType) = *generalized; @@ -243,12 +246,28 @@ bool ConstraintSolver::tryDispatch(const InstantiationConstraint& c, NotNull instantiated = inst.substitute(c.superType); LUAU_ASSERT(instantiated); // TODO FIXME HANDLE THIS - unify(c.subType, *instantiated, constraint->location); + unify(c.subType, *instantiated); unblock(c.subType); return true; } +bool ConstraintSolver::tryDispatch(const NameConstraint& c, NotNull constraint) +{ + if (isBlocked(c.namedType)) + return block(c.namedType, constraint); + + TypeId target = follow(c.namedType); + if (TableTypeVar* ttv = getMutable(target)) + ttv->name = c.name; + else if (MetatableTypeVar* mtv = getMutable(target)) + mtv->syntheticName = c.name; + else + return block(c.namedType, constraint); + + return true; +} + void ConstraintSolver::block_(BlockedConstraintId target, NotNull constraint) { blocked[target].push_back(constraint); @@ -321,19 +340,19 @@ bool ConstraintSolver::isBlocked(NotNull constraint) return blockedIt != blockedConstraints.end() && blockedIt->second > 0; } -void ConstraintSolver::unify(TypeId subType, TypeId superType, Location location) +void ConstraintSolver::unify(TypeId subType, TypeId superType) { UnifierSharedState sharedState{&iceReporter}; - Unifier u{arena, Mode::Strict, location, Covariant, sharedState}; + Unifier u{arena, Mode::Strict, Location{}, Covariant, sharedState}; u.tryUnify(subType, superType); u.log.commit(); } -void ConstraintSolver::unify(TypePackId subPack, TypePackId superPack, Location location) +void ConstraintSolver::unify(TypePackId subPack, TypePackId superPack) { UnifierSharedState sharedState{&iceReporter}; - Unifier u{arena, Mode::Strict, location, Covariant, sharedState}; + Unifier u{arena, Mode::Strict, Location{}, Covariant, sharedState}; u.tryUnify(subPack, superPack); u.log.commit(); diff --git a/Analysis/src/Error.cpp b/Analysis/src/Error.cpp index f443a3cc..93cb65b9 100644 --- a/Analysis/src/Error.cpp +++ b/Analysis/src/Error.cpp @@ -7,6 +7,9 @@ #include +LUAU_FASTFLAGVARIABLE(LuauTypeMismatchModuleNameResolution, false) +LUAU_FASTFLAGVARIABLE(LuauUseInternalCompilerErrorException, false) + static std::string wrongNumberOfArgsString(size_t expectedCount, size_t actualCount, const char* argPrefix = nullptr, bool isVariadic = false) { std::string s = "expects "; @@ -49,6 +52,8 @@ namespace Luau struct ErrorConverter { + FileResolver* fileResolver = nullptr; + std::string operator()(const Luau::TypeMismatch& tm) const { std::string givenTypeName = Luau::toString(tm.givenType); @@ -62,8 +67,18 @@ struct ErrorConverter { if (auto wantedDefinitionModule = getDefinitionModuleName(tm.wantedType)) { - result = "Type '" + givenTypeName + "' from '" + *givenDefinitionModule + "' could not be converted into '" + wantedTypeName + - "' from '" + *wantedDefinitionModule + "'"; + if (FFlag::LuauTypeMismatchModuleNameResolution && fileResolver != nullptr) + { + std::string givenModuleName = fileResolver->getHumanReadableModuleName(*givenDefinitionModule); + std::string wantedModuleName = fileResolver->getHumanReadableModuleName(*wantedDefinitionModule); + result = "Type '" + givenTypeName + "' from '" + givenModuleName + "' could not be converted into '" + wantedTypeName + + "' from '" + wantedModuleName + "'"; + } + else + { + result = "Type '" + givenTypeName + "' from '" + *givenDefinitionModule + "' could not be converted into '" + wantedTypeName + + "' from '" + *wantedDefinitionModule + "'"; + } } } } @@ -78,7 +93,14 @@ struct ErrorConverter if (!tm.reason.empty()) result += tm.reason + " "; - result += Luau::toString(*tm.error); + if (FFlag::LuauTypeMismatchModuleNameResolution) + { + result += Luau::toString(*tm.error, TypeErrorToStringOptions{fileResolver}); + } + else + { + result += Luau::toString(*tm.error); + } } else if (!tm.reason.empty()) { @@ -280,6 +302,11 @@ struct ErrorConverter return e.message; } + std::string operator()(const Luau::InternalError& e) const + { + return e.message; + } + std::string operator()(const Luau::CannotCallNonFunction& e) const { return "Cannot call non-function " + toString(e.ty); @@ -598,6 +625,11 @@ bool GenericError::operator==(const GenericError& rhs) const return message == rhs.message; } +bool InternalError::operator==(const InternalError& rhs) const +{ + return message == rhs.message; +} + bool CannotCallNonFunction::operator==(const CannotCallNonFunction& rhs) const { return ty == rhs.ty; @@ -685,7 +717,12 @@ bool TypesAreUnrelated::operator==(const TypesAreUnrelated& rhs) const std::string toString(const TypeError& error) { - ErrorConverter converter; + return toString(error, TypeErrorToStringOptions{}); +} + +std::string toString(const TypeError& error, TypeErrorToStringOptions options) +{ + ErrorConverter converter{options.fileResolver}; return Luau::visit(converter, error.data); } @@ -773,6 +810,9 @@ void copyError(T& e, TypeArena& destArena, CloneState cloneState) else if constexpr (std::is_same_v) { } + else if constexpr (std::is_same_v) + { + } else if constexpr (std::is_same_v) { e.ty = clone(e.ty); @@ -847,22 +887,51 @@ void copyErrors(ErrorVec& errors, TypeArena& destArena) void InternalErrorReporter::ice(const std::string& message, const Location& location) { - std::runtime_error error("Internal error in " + moduleName + " at " + toString(location) + ": " + message); + if (FFlag::LuauUseInternalCompilerErrorException) + { + InternalCompilerError error(message, moduleName, location); - if (onInternalError) - onInternalError(error.what()); + if (onInternalError) + onInternalError(error.what()); - throw error; + throw error; + } + else + { + std::runtime_error error("Internal error in " + moduleName + " at " + toString(location) + ": " + message); + + if (onInternalError) + onInternalError(error.what()); + + throw error; + } } void InternalErrorReporter::ice(const std::string& message) { - std::runtime_error error("Internal error in " + moduleName + ": " + message); + if (FFlag::LuauUseInternalCompilerErrorException) + { + InternalCompilerError error(message, moduleName); - if (onInternalError) - onInternalError(error.what()); + if (onInternalError) + onInternalError(error.what()); - throw error; + throw error; + } + else + { + std::runtime_error error("Internal error in " + moduleName + ": " + message); + + if (onInternalError) + onInternalError(error.what()); + + throw error; + } +} + +const char* InternalCompilerError::what() const throw() +{ + return this->message.data(); } } // namespace Luau diff --git a/Analysis/src/Frontend.cpp b/Analysis/src/Frontend.cpp index 9e025062..85c5dbc8 100644 --- a/Analysis/src/Frontend.cpp +++ b/Analysis/src/Frontend.cpp @@ -801,6 +801,8 @@ ModulePtr Frontend::check(const SourceModule& sourceModule, Mode mode, const Sco result->astTypes = std::move(cgb.astTypes); result->astTypePacks = std::move(cgb.astTypePacks); result->astOriginalCallTypes = std::move(cgb.astOriginalCallTypes); + result->astResolvedTypes = std::move(cgb.astResolvedTypes); + result->astResolvedTypePacks = std::move(cgb.astResolvedTypePacks); result->clonePublicInterface(iceHandler); diff --git a/Analysis/src/IostreamHelpers.cpp b/Analysis/src/IostreamHelpers.cpp index 048167ae..e4fac455 100644 --- a/Analysis/src/IostreamHelpers.cpp +++ b/Analysis/src/IostreamHelpers.cpp @@ -111,6 +111,8 @@ static void errorToString(std::ostream& stream, const T& err) } else if constexpr (std::is_same_v) stream << "GenericError { " << err.message << " }"; + else if constexpr (std::is_same_v) + stream << "InternalError { " << err.message << " }"; else if constexpr (std::is_same_v) stream << "CannotCallNonFunction { " << toString(err.ty) << " }"; else if constexpr (std::is_same_v) diff --git a/Analysis/src/Module.cpp b/Analysis/src/Module.cpp index 4d157e6f..95eb125e 100644 --- a/Analysis/src/Module.cpp +++ b/Analysis/src/Module.cpp @@ -11,7 +11,6 @@ #include "Luau/TypePack.h" #include "Luau/TypeVar.h" #include "Luau/VisitTypeVar.h" -#include "Luau/ConstraintGraphBuilder.h" // FIXME: For Scope2 TODO pull out into its own header #include diff --git a/Analysis/src/Quantify.cpp b/Analysis/src/Quantify.cpp index 2004d153..40e14c68 100644 --- a/Analysis/src/Quantify.cpp +++ b/Analysis/src/Quantify.cpp @@ -2,11 +2,10 @@ #include "Luau/Quantify.h" -#include "Luau/ConstraintGraphBuilder.h" // TODO for Scope2; move to separate header -#include "Luau/TxnLog.h" +#include "Luau/Scope.h" #include "Luau/Substitution.h" +#include "Luau/TxnLog.h" #include "Luau/VisitTypeVar.h" -#include "Luau/ConstraintGraphBuilder.h" // TODO for Scope2; move to separate header LUAU_FASTFLAG(LuauAlwaysQuantify); LUAU_FASTFLAG(DebugLuauDeferredConstraintResolution); @@ -177,8 +176,6 @@ void quantify(TypeId ty, TypeLevel level) if (ftv->generics.empty() && ftv->genericPacks.empty() && !q.seenMutableType && !q.seenGenericType) ftv->hasNoGenerics = true; - - ftv->generalized = true; } void quantify(TypeId ty, Scope2* scope) @@ -201,8 +198,6 @@ void quantify(TypeId ty, Scope2* scope) if (ftv->generics.empty() && ftv->genericPacks.empty() && !q.seenMutableType && !q.seenGenericType) ftv->hasNoGenerics = true; - - ftv->generalized = true; } struct PureQuantifier : Substitution diff --git a/Analysis/src/Scope.cpp b/Analysis/src/Scope.cpp index 011e28d4..66aaee1f 100644 --- a/Analysis/src/Scope.cpp +++ b/Analysis/src/Scope.cpp @@ -121,4 +121,36 @@ std::optional Scope::linearSearchForBinding(const std::string& name, bo return std::nullopt; } +std::optional Scope2::lookup(Symbol sym) +{ + Scope2* s = this; + + while (true) + { + auto it = s->bindings.find(sym); + if (it != s->bindings.end()) + return it->second; + + if (s->parent) + s = s->parent; + else + return std::nullopt; + } +} + +std::optional Scope2::lookupTypeBinding(const Name& name) +{ + Scope2* s = this; + while (s) + { + auto it = s->typeBindings.find(name); + if (it != s->typeBindings.end()) + return it->second; + + s = s->parent; + } + + return std::nullopt; +} + } // namespace Luau diff --git a/Analysis/src/ToString.cpp b/Analysis/src/ToString.cpp index eee0deee..eb7b9cd6 100644 --- a/Analysis/src/ToString.cpp +++ b/Analysis/src/ToString.cpp @@ -1401,6 +1401,12 @@ std::string toString(const Constraint& c, ToStringOptions& opts) opts.nameMap = std::move(superStr.nameMap); return subStr.name + " ~ inst " + superStr.name; } + else if (const NameConstraint* nc = Luau::get(c)) + { + ToStringResult namedStr = toStringDetailed(nc->namedType, opts); + opts.nameMap = std::move(namedStr.nameMap); + return "@name(" + namedStr.name + ") = " + nc->name; + } else { LUAU_ASSERT(false); diff --git a/Analysis/src/TypeChecker2.cpp b/Analysis/src/TypeChecker2.cpp index 7f5ba683..63e5800f 100644 --- a/Analysis/src/TypeChecker2.cpp +++ b/Analysis/src/TypeChecker2.cpp @@ -7,6 +7,9 @@ #include "Luau/AstQuery.h" #include "Luau/Clone.h" #include "Luau/Normalize.h" +#include "Luau/ConstraintGraphBuilder.h" // FIXME move Scope2 into its own header +#include "Luau/Unifier.h" +#include "Luau/ToString.h" namespace Luau { @@ -39,6 +42,104 @@ struct TypeChecker2 : public AstVisitor return follow(*ty); } + TypeId lookupAnnotation(AstType* annotation) + { + TypeId* ty = module->astResolvedTypes.find(annotation); + LUAU_ASSERT(ty); + return follow(*ty); + } + + TypePackId reconstructPack(AstArray exprs, TypeArena& arena) + { + std::vector head; + + for (size_t i = 0; i < exprs.size - 1; ++i) + { + head.push_back(lookupType(exprs.data[i])); + } + + TypePackId tail = lookupPack(exprs.data[exprs.size - 1]); + return arena.addTypePack(TypePack{head, tail}); + } + + Scope2* findInnermostScope(Location location) + { + Scope2* bestScope = module->getModuleScope2(); + Location bestLocation = module->scope2s[0].first; + + for (size_t i = 0; i < module->scope2s.size(); ++i) + { + auto& [scopeBounds, scope] = module->scope2s[i]; + if (scopeBounds.encloses(location)) + { + if (scopeBounds.begin > bestLocation.begin || scopeBounds.end < bestLocation.end) + { + bestScope = scope.get(); + bestLocation = scopeBounds; + } + } + else + { + // TODO: Is this sound? This relies on the fact that scopes are inserted + // into the scope list in the order that they appear in the AST. + break; + } + } + + return bestScope; + } + + bool visit(AstStatLocal* local) override + { + for (size_t i = 0; i < local->values.size; ++i) + { + AstExpr* value = local->values.data[i]; + if (i == local->values.size - 1) + { + if (i < local->values.size) + { + TypePackId valueTypes = lookupPack(value); + auto it = begin(valueTypes); + for (size_t j = i; j < local->vars.size; ++j) + { + if (it == end(valueTypes)) + { + break; + } + + AstLocal* var = local->vars.data[i]; + if (var->annotation) + { + TypeId varType = lookupAnnotation(var->annotation); + if (!isSubtype(*it, varType, ice)) + { + reportError(TypeMismatch{varType, *it}, value->location); + } + } + + ++it; + } + } + } + else + { + TypeId valueType = lookupType(value); + AstLocal* var = local->vars.data[i]; + + if (var->annotation) + { + TypeId varType = lookupAnnotation(var->annotation); + if (!isSubtype(varType, valueType, ice)) + { + reportError(TypeMismatch{varType, valueType}, value->location); + } + } + } + } + + return true; + } + bool visit(AstStatAssign* assign) override { size_t count = std::min(assign->vars.size, assign->values.size); @@ -62,6 +163,30 @@ struct TypeChecker2 : public AstVisitor return true; } + bool visit(AstStatReturn* ret) override + { + Scope2* scope = findInnermostScope(ret->location); + TypePackId expectedRetType = scope->returnType; + + TypeArena arena; + TypePackId actualRetType = reconstructPack(ret->list, arena); + + UnifierSharedState sharedState{&ice}; + Unifier u{&arena, Mode::Strict, ret->location, Covariant, sharedState}; + u.anyIsTop = true; + + u.tryUnify(actualRetType, expectedRetType); + const bool ok = u.errors.empty() && u.log.empty(); + + if (!ok) + { + for (const TypeError& e : u.errors) + module->errors.push_back(e); + } + + return true; + } + bool visit(AstExprCall* call) override { TypePackId expectedRetType = lookupPack(call); @@ -91,6 +216,35 @@ struct TypeChecker2 : public AstVisitor return true; } + bool visit(AstExprFunction* fn) override + { + TypeId inferredFnTy = lookupType(fn); + const FunctionTypeVar* inferredFtv = get(inferredFnTy); + LUAU_ASSERT(inferredFtv); + + auto argIt = begin(inferredFtv->argTypes); + for (const auto& arg : fn->args) + { + if (argIt == end(inferredFtv->argTypes)) + break; + + if (arg->annotation) + { + TypeId inferredArgTy = *argIt; + TypeId annotatedArgTy = lookupAnnotation(arg->annotation); + + if (!isSubtype(annotatedArgTy, inferredArgTy, ice)) + { + reportError(TypeMismatch{annotatedArgTy, inferredArgTy}, arg->location); + } + } + + ++argIt; + } + + return true; + } + bool visit(AstExprIndexName* indexName) override { TypeId leftType = lookupType(indexName->expr); @@ -144,6 +298,25 @@ struct TypeChecker2 : public AstVisitor return true; } + bool visit(AstType* ty) override + { + return true; + } + + bool visit(AstTypeReference* ty) override + { + Scope2* scope = findInnermostScope(ty->location); + + // TODO: Imported types + // TODO: Generic types + if (!scope->lookupTypeBinding(ty->name.value)) + { + reportError(UnknownSymbol{ty->name.value, UnknownSymbol::Context::Type}, ty->location); + } + + return true; + } + void reportError(TypeErrorData&& data, const Location& location) { module->errors.emplace_back(location, sourceModule->name, std::move(data)); diff --git a/Analysis/src/TypeInfer.cpp b/Analysis/src/TypeInfer.cpp index fd1b3b85..44635e88 100644 --- a/Analysis/src/TypeInfer.cpp +++ b/Analysis/src/TypeInfer.cpp @@ -35,13 +35,9 @@ LUAU_FASTFLAGVARIABLE(LuauLowerBoundsCalculation, false) LUAU_FASTFLAGVARIABLE(DebugLuauFreezeDuringUnification, false) LUAU_FASTFLAGVARIABLE(LuauSelfCallAutocompleteFix2, false) LUAU_FASTFLAGVARIABLE(LuauReduceUnionRecursion, false) -LUAU_FASTFLAGVARIABLE(LuauOnlyMutateInstantiatedTables, false) -LUAU_FASTFLAGVARIABLE(LuauUnsealedTableLiteral, false) LUAU_FASTFLAGVARIABLE(LuauReturnAnyInsteadOfICE, false) // Eventually removed as false. LUAU_FASTFLAG(LuauNormalizeFlagIsConservative) LUAU_FASTFLAGVARIABLE(LuauReturnTypeInferenceInNonstrict, false) -LUAU_FASTFLAGVARIABLE(LuauRecursionLimitException, false); -LUAU_FASTFLAGVARIABLE(LuauApplyTypeFunctionFix, false); LUAU_FASTFLAGVARIABLE(LuauAlwaysQuantify, false); LUAU_FASTFLAGVARIABLE(LuauReportErrorsOnIndexerKeyMismatch, false) LUAU_FASTFLAG(LuauQuantifyConstrained) @@ -275,22 +271,15 @@ TypeChecker::TypeChecker(ModuleResolver* resolver, InternalErrorReporter* iceHan ModulePtr TypeChecker::check(const SourceModule& module, Mode mode, std::optional environmentScope) { - if (FFlag::LuauRecursionLimitException) - { - try - { - return checkWithoutRecursionCheck(module, mode, environmentScope); - } - catch (const RecursionLimitException&) - { - reportErrorCodeTooComplex(module.root->location); - return std::move(currentModule); - } - } - else + try { return checkWithoutRecursionCheck(module, mode, environmentScope); } + catch (const RecursionLimitException&) + { + reportErrorCodeTooComplex(module.root->location); + return std::move(currentModule); + } } ModulePtr TypeChecker::checkWithoutRecursionCheck(const SourceModule& module, Mode mode, std::optional environmentScope) @@ -445,22 +434,15 @@ void TypeChecker::checkBlock(const ScopePtr& scope, const AstStatBlock& block) reportErrorCodeTooComplex(block.location); return; } - if (FFlag::LuauRecursionLimitException) - { - try - { - checkBlockWithoutRecursionCheck(scope, block); - } - catch (const RecursionLimitException&) - { - reportErrorCodeTooComplex(block.location); - return; - } - } - else + try { checkBlockWithoutRecursionCheck(scope, block); } + catch (const RecursionLimitException&) + { + reportErrorCodeTooComplex(block.location); + return; + } } void TypeChecker::checkBlockWithoutRecursionCheck(const ScopePtr& scope, const AstStatBlock& block) @@ -1917,7 +1899,7 @@ std::optional TypeChecker::getIndexTypeFromType( for (TypeId t : utv) { - RecursionLimiter _rl(&recursionCount, FInt::LuauTypeInferRecursionLimit, "getIndexTypeForType unions"); + RecursionLimiter _rl(&recursionCount, FInt::LuauTypeInferRecursionLimit); // Not needed when we normalize types. if (get(follow(t))) @@ -1967,7 +1949,7 @@ std::optional TypeChecker::getIndexTypeFromType( for (TypeId t : itv->parts) { - RecursionLimiter _rl(&recursionCount, FInt::LuauTypeInferRecursionLimit, "getIndexTypeFromType intersections"); + RecursionLimiter _rl(&recursionCount, FInt::LuauTypeInferRecursionLimit); if (std::optional ty = getIndexTypeFromType(scope, t, name, location, false)) parts.push_back(*ty); @@ -2190,7 +2172,7 @@ TypeId TypeChecker::checkExprTable( } } - TableState state = (expr.items.size == 0 || isNonstrictMode() || FFlag::LuauUnsealedTableLiteral) ? TableState::Unsealed : TableState::Sealed; + TableState state = TableState::Unsealed; TableTypeVar table = TableTypeVar{std::move(props), indexer, scope->level, state}; table.definitionModuleName = currentModuleName; return addType(table); @@ -5175,9 +5157,7 @@ TypePackId TypeChecker::resolveTypePack(const ScopePtr& scope, const AstTypePack bool ApplyTypeFunction::isDirty(TypeId ty) { - if (FFlag::LuauApplyTypeFunctionFix && typeArguments.count(ty)) - return true; - else if (!FFlag::LuauApplyTypeFunctionFix && get(ty)) + if (typeArguments.count(ty)) return true; else if (const FreeTypeVar* ftv = get(ty)) { @@ -5191,9 +5171,7 @@ bool ApplyTypeFunction::isDirty(TypeId ty) bool ApplyTypeFunction::isDirty(TypePackId tp) { - if (FFlag::LuauApplyTypeFunctionFix && typePackArguments.count(tp)) - return true; - else if (!FFlag::LuauApplyTypeFunctionFix && get(tp)) + if (typePackArguments.count(tp)) return true; else return false; @@ -5218,29 +5196,15 @@ bool ApplyTypeFunction::ignoreChildren(TypePackId tp) TypeId ApplyTypeFunction::clean(TypeId ty) { TypeId& arg = typeArguments[ty]; - if (FFlag::LuauApplyTypeFunctionFix) - { - LUAU_ASSERT(arg); - return arg; - } - else if (arg) - return arg; - else - return addType(FreeTypeVar{level}); + LUAU_ASSERT(arg); + return arg; } TypePackId ApplyTypeFunction::clean(TypePackId tp) { TypePackId& arg = typePackArguments[tp]; - if (FFlag::LuauApplyTypeFunctionFix) - { - LUAU_ASSERT(arg); - return arg; - } - else if (arg) - return arg; - else - return addTypePack(FreeTypePack{level}); + LUAU_ASSERT(arg); + return arg; } TypeId TypeChecker::instantiateTypeFun(const ScopePtr& scope, const TypeFun& tf, const std::vector& typeParams, @@ -5273,7 +5237,7 @@ TypeId TypeChecker::instantiateTypeFun(const ScopePtr& scope, const TypeFun& tf, TypeId target = follow(instantiated); bool needsClone = follow(tf.type) == target; - bool shouldMutate = (!FFlag::LuauOnlyMutateInstantiatedTables || getTableType(tf.type)); + bool shouldMutate = getTableType(tf.type); TableTypeVar* ttv = getMutableTableType(target); if (shouldMutate && ttv && needsClone) diff --git a/Analysis/src/TypeVar.cpp b/Analysis/src/TypeVar.cpp index 57762937..ade70d72 100644 --- a/Analysis/src/TypeVar.cpp +++ b/Analysis/src/TypeVar.cpp @@ -23,7 +23,6 @@ LUAU_FASTFLAG(DebugLuauFreezeArena) LUAU_FASTINTVARIABLE(LuauTypeMaximumStringifierLength, 500) LUAU_FASTINTVARIABLE(LuauTableTypeMaximumStringifierLength, 0) LUAU_FASTINT(LuauTypeInferRecursionLimit) -LUAU_FASTFLAG(LuauSubtypingAddOptPropsToUnsealedTables) LUAU_FASTFLAG(LuauNonCopyableTypeVarFields) namespace Luau @@ -172,22 +171,15 @@ bool isString(TypeId ty) // Returns true when ty is a supertype of string bool maybeString(TypeId ty) { - if (FFlag::LuauSubtypingAddOptPropsToUnsealedTables) - { - ty = follow(ty); + ty = follow(ty); - if (isPrim(ty, PrimitiveTypeVar::String) || get(ty)) - return true; + if (isPrim(ty, PrimitiveTypeVar::String) || get(ty)) + return true; - if (auto utv = get(ty)) - return std::any_of(begin(utv), end(utv), maybeString); + if (auto utv = get(ty)) + return std::any_of(begin(utv), end(utv), maybeString); - return false; - } - else - { - return isString(ty); - } + return false; } bool isThread(TypeId ty) @@ -369,7 +361,7 @@ bool maybeSingleton(TypeId ty) bool hasLength(TypeId ty, DenseHashSet& seen, int* recursionCount) { - RecursionLimiter _rl(recursionCount, FInt::LuauTypeInferRecursionLimit, "hasLength"); + RecursionLimiter _rl(recursionCount, FInt::LuauTypeInferRecursionLimit); ty = follow(ty); @@ -750,13 +742,15 @@ TypeId SingletonTypes::makeStringMetatable() TableTypeVar::Props stringLib = { {"byte", {arena->addType(FunctionTypeVar{arena->addTypePack({stringType, optionalNumber, optionalNumber}), numberVariadicList})}}, {"char", {arena->addType(FunctionTypeVar{numberVariadicList, arena->addTypePack({stringType})})}}, - {"find", {makeFunction(*arena, stringType, {}, {}, {stringType, optionalNumber, optionalBoolean}, {}, {optionalNumber, optionalNumber})}}, + {"find", {arena->addType(FunctionTypeVar{arena->addTypePack({stringType, stringType, optionalNumber, optionalBoolean}), + arena->addTypePack(TypePack{{optionalNumber, optionalNumber}, stringVariadicList})})}}, {"format", {formatFn}}, // FIXME {"gmatch", {gmatchFunc}}, {"gsub", {gsubFunc}}, {"len", {makeFunction(*arena, stringType, {}, {}, {}, {}, {numberType})}}, {"lower", {stringToStringType}}, - {"match", {makeFunction(*arena, stringType, {}, {}, {stringType, optionalNumber}, {}, {optionalString})}}, + {"match", {arena->addType(FunctionTypeVar{arena->addTypePack({stringType, stringType, optionalNumber}), + arena->addTypePack(TypePackVar{VariadicTypePack{optionalString}})})}}, {"rep", {makeFunction(*arena, stringType, {}, {}, {numberType}, {}, {stringType})}}, {"reverse", {stringToStringType}}, {"sub", {makeFunction(*arena, stringType, {}, {}, {numberType, optionalNumber}, {}, {stringType})}}, diff --git a/Analysis/src/Unifiable.cpp b/Analysis/src/Unifiable.cpp index fe878358..8d23aa49 100644 --- a/Analysis/src/Unifiable.cpp +++ b/Analysis/src/Unifiable.cpp @@ -53,6 +53,14 @@ Generic::Generic(TypeLevel level, const Name& name) { } +Generic::Generic(Scope2* scope, const Name& name) + : index(++nextIndex) + , scope(scope) + , name(name) + , explicitName(true) +{ +} + int Generic::nextIndex = 0; Error::Error() diff --git a/Analysis/src/Unifier.cpp b/Analysis/src/Unifier.cpp index 877663de..6147e118 100644 --- a/Analysis/src/Unifier.cpp +++ b/Analysis/src/Unifier.cpp @@ -17,11 +17,8 @@ LUAU_FASTINT(LuauTypeInferTypePackLoopLimit); LUAU_FASTINT(LuauTypeInferIterationLimit); LUAU_FASTFLAG(LuauAutocompleteDynamicLimits) LUAU_FASTINTVARIABLE(LuauTypeInferLowerBoundsIterationLimit, 2000); -LUAU_FASTFLAGVARIABLE(LuauTableSubtypingVariance2, false); LUAU_FASTFLAG(LuauLowerBoundsCalculation); LUAU_FASTFLAG(LuauErrorRecoveryType); -LUAU_FASTFLAGVARIABLE(LuauSubtypingAddOptPropsToUnsealedTables, false) -LUAU_FASTFLAGVARIABLE(LuauTxnLogRefreshFunctionPointers, false) LUAU_FASTFLAG(LuauQuantifyConstrained) namespace Luau @@ -354,7 +351,7 @@ void Unifier::tryUnify(TypeId subTy, TypeId superTy, bool isFunctionCall, bool i void Unifier::tryUnify_(TypeId subTy, TypeId superTy, bool isFunctionCall, bool isIntersection) { RecursionLimiter _ra(&sharedState.counters.recursionCount, - FFlag::LuauAutocompleteDynamicLimits ? sharedState.counters.recursionLimit : FInt::LuauTypeInferRecursionLimit, "TypeId tryUnify_"); + FFlag::LuauAutocompleteDynamicLimits ? sharedState.counters.recursionLimit : FInt::LuauTypeInferRecursionLimit); ++sharedState.counters.iterationCount; @@ -983,7 +980,7 @@ void Unifier::tryUnify(TypePackId subTp, TypePackId superTp, bool isFunctionCall void Unifier::tryUnify_(TypePackId subTp, TypePackId superTp, bool isFunctionCall) { RecursionLimiter _ra(&sharedState.counters.recursionCount, - FFlag::LuauAutocompleteDynamicLimits ? sharedState.counters.recursionLimit : FInt::LuauTypeInferRecursionLimit, "TypePackId tryUnify_"); + FFlag::LuauAutocompleteDynamicLimits ? sharedState.counters.recursionLimit : FInt::LuauTypeInferRecursionLimit); ++sharedState.counters.iterationCount; @@ -1316,12 +1313,9 @@ void Unifier::tryUnifyFunctions(TypeId subTy, TypeId superTy, bool isFunctionCal tryUnify_(subFunction->retTypes, superFunction->retTypes); } - if (FFlag::LuauTxnLogRefreshFunctionPointers) - { - // Updating the log may have invalidated the function pointers - superFunction = log.getMutable(superTy); - subFunction = log.getMutable(subTy); - } + // Updating the log may have invalidated the function pointers + superFunction = log.getMutable(superTy); + subFunction = log.getMutable(subTy); ctx = context; @@ -1360,9 +1354,6 @@ struct Resetter void Unifier::tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) { - if (!FFlag::LuauTableSubtypingVariance2) - return DEPRECATED_tryUnifyTables(subTy, superTy, isIntersection); - TableTypeVar* superTable = log.getMutable(superTy); TableTypeVar* subTable = log.getMutable(subTy); @@ -1379,8 +1370,7 @@ void Unifier::tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) { auto subIter = subTable->props.find(propName); - if (subIter == subTable->props.end() && (!FFlag::LuauSubtypingAddOptPropsToUnsealedTables || subTable->state == TableState::Unsealed) && - !isOptional(superProp.type)) + if (subIter == subTable->props.end() && subTable->state == TableState::Unsealed && !isOptional(superProp.type)) missingProperties.push_back(propName); } @@ -1398,7 +1388,7 @@ void Unifier::tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) { auto superIter = superTable->props.find(propName); - if (superIter == superTable->props.end() && (FFlag::LuauSubtypingAddOptPropsToUnsealedTables || !isOptional(subProp.type))) + if (superIter == superTable->props.end()) extraProperties.push_back(propName); } @@ -1443,7 +1433,7 @@ void Unifier::tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) if (innerState.errors.empty()) log.concat(std::move(innerState.log)); } - else if ((!FFlag::LuauSubtypingAddOptPropsToUnsealedTables || subTable->state == TableState::Unsealed) && isOptional(prop.type)) + else if (subTable->state == TableState::Unsealed && isOptional(prop.type)) // This is sound because unsealed table types are precise, so `{ p : T } <: { p : T, q : U? }` // since if `t : { p : T }` then we are guaranteed that `t.q` is `nil`. // TODO: if the supertype is written to, the subtype may no longer be precise (alias analysis?) @@ -1512,9 +1502,6 @@ void Unifier::tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) else if (variance == Covariant) { } - else if (!FFlag::LuauSubtypingAddOptPropsToUnsealedTables && isOptional(prop.type)) - { - } else if (superTable->state == TableState::Free) { PendingType* pendingSuper = log.queue(superTy); @@ -1639,296 +1626,6 @@ TypeId Unifier::deeplyOptional(TypeId ty, std::unordered_map see return types->addType(UnionTypeVar{{getSingletonTypes().nilType, ty}}); } -void Unifier::DEPRECATED_tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) -{ - LUAU_ASSERT(!FFlag::LuauTableSubtypingVariance2); - Resetter resetter{&variance}; - variance = Invariant; - - TableTypeVar* superTable = log.getMutable(superTy); - TableTypeVar* subTable = log.getMutable(subTy); - - if (!superTable || !subTable) - ice("passed non-table types to unifyTables"); - - if (superTable->state == TableState::Sealed && subTable->state == TableState::Sealed) - return tryUnifySealedTables(subTy, superTy, isIntersection); - else if ((superTable->state == TableState::Sealed && subTable->state == TableState::Unsealed) || - (superTable->state == TableState::Unsealed && subTable->state == TableState::Sealed)) - return tryUnifySealedTables(subTy, superTy, isIntersection); - else if ((superTable->state == TableState::Sealed && subTable->state == TableState::Generic) || - (superTable->state == TableState::Generic && subTable->state == TableState::Sealed)) - reportError(TypeError{location, TypeMismatch{superTy, subTy}}); - else if ((superTable->state == TableState::Free) != (subTable->state == TableState::Free)) // one table is free and the other is not - { - TypeId freeTypeId = subTable->state == TableState::Free ? subTy : superTy; - TypeId otherTypeId = subTable->state == TableState::Free ? superTy : subTy; - - return tryUnifyFreeTable(otherTypeId, freeTypeId); - } - else if (superTable->state == TableState::Free && subTable->state == TableState::Free) - { - tryUnifyFreeTable(subTy, superTy); - - // avoid creating a cycle when the types are already pointing at each other - if (follow(superTy) != follow(subTy)) - { - log.bindTable(superTy, subTy); - } - return; - } - else if (superTable->state != TableState::Sealed && subTable->state != TableState::Sealed) - { - // All free tables are checked in one of the branches above - LUAU_ASSERT(superTable->state != TableState::Free); - LUAU_ASSERT(subTable->state != TableState::Free); - - // Tables must have exactly the same props and their types must all unify - // I honestly have no idea if this is remotely close to reasonable. - for (const auto& [name, prop] : superTable->props) - { - const auto& r = subTable->props.find(name); - if (r == subTable->props.end()) - reportError(TypeError{location, UnknownProperty{subTy, name}}); - else - tryUnify_(r->second.type, prop.type); - } - - if (superTable->indexer && subTable->indexer) - tryUnifyIndexer(*subTable->indexer, *superTable->indexer); - else if (superTable->indexer) - { - // passing/assigning a table without an indexer to something that has one - // e.g. table.insert(t, 1) where t is a non-sealed table and doesn't have an indexer. - if (subTable->state == TableState::Unsealed) - { - log.changeIndexer(subTy, superTable->indexer); - } - else - reportError(TypeError{location, CannotExtendTable{subTy, CannotExtendTable::Indexer}}); - } - } - else if (superTable->state == TableState::Sealed) - { - // lt is sealed and so it must be possible for rt to have precisely the same shape - // Verify that this is the case, then bind rt to lt. - ice("unsealed tables are not working yet", location); - } - else if (subTable->state == TableState::Sealed) - return tryUnifyTables(superTy, subTy, isIntersection); - else - ice("tryUnifyTables"); -} - -void Unifier::tryUnifyFreeTable(TypeId subTy, TypeId superTy) -{ - LUAU_ASSERT(!FFlag::LuauTableSubtypingVariance2); - TableTypeVar* freeTable = log.getMutable(superTy); - TableTypeVar* subTable = log.getMutable(subTy); - - if (!freeTable || !subTable) - ice("passed non-table types to tryUnifyFreeTable"); - - // Any properties in freeTable must unify with those in otherTable. - // Then bind freeTable to otherTable. - for (const auto& [freeName, freeProp] : freeTable->props) - { - if (auto subProp = findTablePropertyRespectingMeta(subTy, freeName)) - { - tryUnify_(*subProp, freeProp.type); - - /* - * TypeVars are commonly cyclic, so it is entirely possible - * for unifying a property of a table to change the table itself! - * We need to check for this and start over if we notice this occurring. - * - * I believe this is guaranteed to terminate eventually because this will - * only happen when a free table is bound to another table. - */ - if (!log.getMutable(superTy) || !log.getMutable(subTy)) - return tryUnify_(subTy, superTy); - - if (TableTypeVar* pendingFreeTtv = log.getMutable(superTy); pendingFreeTtv && pendingFreeTtv->boundTo) - return tryUnify_(subTy, superTy); - } - else - { - // If the other table is also free, then we are learning that it has more - // properties than we previously thought. Else, it is an error. - if (subTable->state == TableState::Free) - { - PendingType* pendingSub = log.queue(subTy); - TableTypeVar* pendingSubTtv = getMutable(pendingSub); - LUAU_ASSERT(pendingSubTtv); - pendingSubTtv->props.insert({freeName, freeProp}); - } - else - reportError(TypeError{location, UnknownProperty{subTy, freeName}}); - } - } - - if (freeTable->indexer && subTable->indexer) - { - Unifier innerState = makeChildUnifier(); - innerState.tryUnifyIndexer(*subTable->indexer, *freeTable->indexer); - - checkChildUnifierTypeMismatch(innerState.errors, superTy, subTy); - - log.concat(std::move(innerState.log)); - } - else if (subTable->state == TableState::Free && freeTable->indexer) - { - log.changeIndexer(superTy, subTable->indexer); - } - - if (!freeTable->boundTo && subTable->state != TableState::Free) - { - log.bindTable(superTy, subTy); - } -} - -void Unifier::tryUnifySealedTables(TypeId subTy, TypeId superTy, bool isIntersection) -{ - LUAU_ASSERT(!FFlag::LuauTableSubtypingVariance2); - TableTypeVar* superTable = log.getMutable(superTy); - TableTypeVar* subTable = log.getMutable(subTy); - - if (!superTable || !subTable) - ice("passed non-table types to unifySealedTables"); - - std::vector missingPropertiesInSuper; - bool isUnnamedTable = subTable->name == std::nullopt && subTable->syntheticName == std::nullopt; - bool errorReported = false; - - // Optimization: First test that the property sets are compatible without doing any recursive unification - if (!subTable->indexer) - { - for (const auto& [propName, superProp] : superTable->props) - { - auto subIter = subTable->props.find(propName); - if (subIter == subTable->props.end() && !isOptional(superProp.type)) - missingPropertiesInSuper.push_back(propName); - } - - if (!missingPropertiesInSuper.empty()) - { - reportError(TypeError{location, MissingProperties{superTy, subTy, std::move(missingPropertiesInSuper)}}); - return; - } - } - - Unifier innerState = makeChildUnifier(); - - // Tables must have exactly the same props and their types must all unify - for (const auto& it : superTable->props) - { - const auto& r = subTable->props.find(it.first); - if (r == subTable->props.end()) - { - if (isOptional(it.second.type)) - continue; - - missingPropertiesInSuper.push_back(it.first); - - innerState.reportError(TypeError{location, TypeMismatch{superTy, subTy}}); - } - else - { - if (isUnnamedTable && r->second.location) - { - size_t oldErrorSize = innerState.errors.size(); - Location old = innerState.location; - innerState.location = *r->second.location; - innerState.tryUnify_(r->second.type, it.second.type); - innerState.location = old; - - if (oldErrorSize != innerState.errors.size() && !errorReported) - { - errorReported = true; - reportError(innerState.errors.back()); - } - } - else - { - innerState.tryUnify_(r->second.type, it.second.type); - } - } - } - - if (superTable->indexer || subTable->indexer) - { - if (superTable->indexer && subTable->indexer) - innerState.tryUnifyIndexer(*subTable->indexer, *superTable->indexer); - else if (subTable->state == TableState::Unsealed) - { - if (superTable->indexer && !subTable->indexer) - { - log.changeIndexer(subTy, superTable->indexer); - } - } - else if (superTable->state == TableState::Unsealed) - { - if (subTable->indexer && !superTable->indexer) - { - log.changeIndexer(superTy, subTable->indexer); - } - } - else if (superTable->indexer) - { - innerState.tryUnify_(getSingletonTypes().stringType, superTable->indexer->indexType); - for (const auto& [name, type] : subTable->props) - { - const auto& it = superTable->props.find(name); - if (it == superTable->props.end()) - innerState.tryUnify_(type.type, superTable->indexer->indexResultType); - } - } - else - innerState.reportError(TypeError{location, TypeMismatch{superTy, subTy}}); - } - - if (!errorReported) - log.concat(std::move(innerState.log)); - else - return; - - if (!missingPropertiesInSuper.empty()) - { - reportError(TypeError{location, MissingProperties{superTy, subTy, std::move(missingPropertiesInSuper)}}); - return; - } - - // If the superTy is an immediate part of an intersection type, do not do extra-property check. - // Otherwise, we would falsely generate an extra-property-error for 's' in this code: - // local a: {n: number} & {s: string} = {n=1, s=""} - // When checking against the table '{n: number}'. - if (!isIntersection && superTable->state != TableState::Unsealed && !superTable->indexer) - { - // Check for extra properties in the subTy - std::vector extraPropertiesInSub; - - for (const auto& [subKey, subProp] : subTable->props) - { - const auto& superIt = superTable->props.find(subKey); - if (superIt == superTable->props.end()) - { - if (isOptional(subProp.type)) - continue; - - extraPropertiesInSub.push_back(subKey); - } - } - - if (!extraPropertiesInSub.empty()) - { - reportError(TypeError{location, MissingProperties{superTy, subTy, std::move(extraPropertiesInSub), MissingProperties::Extra}}); - return; - } - } - - checkChildUnifierTypeMismatch(innerState.errors, superTy, subTy); -} - void Unifier::tryUnifyWithMetatable(TypeId subTy, TypeId superTy, bool reversed) { const MetatableTypeVar* superMetatable = get(superTy); @@ -2068,14 +1765,6 @@ void Unifier::tryUnifyWithClass(TypeId subTy, TypeId superTy, bool reversed) return fail(); } -void Unifier::tryUnifyIndexer(const TableIndexer& subIndexer, const TableIndexer& superIndexer) -{ - LUAU_ASSERT(!FFlag::LuauTableSubtypingVariance2); - - tryUnify_(subIndexer.indexType, superIndexer.indexType); - tryUnify_(subIndexer.indexResultType, superIndexer.indexResultType); -} - static void queueTypePack(std::vector& queue, DenseHashSet& seenTypePacks, Unifier& state, TypePackId a, TypePackId anyTypePack) { while (true) @@ -2435,7 +2124,7 @@ void Unifier::occursCheck(TypeId needle, TypeId haystack) void Unifier::occursCheck(DenseHashSet& seen, TypeId needle, TypeId haystack) { RecursionLimiter _ra(&sharedState.counters.recursionCount, - FFlag::LuauAutocompleteDynamicLimits ? sharedState.counters.recursionLimit : FInt::LuauTypeInferRecursionLimit, "occursCheck for TypeId"); + FFlag::LuauAutocompleteDynamicLimits ? sharedState.counters.recursionLimit : FInt::LuauTypeInferRecursionLimit); auto check = [&](TypeId tv) { occursCheck(seen, needle, tv); @@ -2506,7 +2195,7 @@ void Unifier::occursCheck(DenseHashSet& seen, TypePackId needle, Typ ice("Expected needle pack to be free"); RecursionLimiter _ra(&sharedState.counters.recursionCount, - FFlag::LuauAutocompleteDynamicLimits ? sharedState.counters.recursionLimit : FInt::LuauTypeInferRecursionLimit, "occursCheck for TypePackId"); + FFlag::LuauAutocompleteDynamicLimits ? sharedState.counters.recursionLimit : FInt::LuauTypeInferRecursionLimit); while (!log.getMutable(haystack)) { diff --git a/CLI/Analyze.cpp b/CLI/Analyze.cpp index 10cf17d2..4bc8cab5 100644 --- a/CLI/Analyze.cpp +++ b/CLI/Analyze.cpp @@ -9,6 +9,7 @@ #include "FileUtils.h" LUAU_FASTFLAG(DebugLuauTimeTracing) +LUAU_FASTFLAG(LuauTypeMismatchModuleNameResolution) enum class ReportFormat { @@ -49,6 +50,9 @@ static void reportError(const Luau::Frontend& frontend, ReportFormat format, con if (const Luau::SyntaxError* syntaxError = Luau::get_if(&error.data)) report(format, humanReadableName.c_str(), error.location, "SyntaxError", syntaxError->message.c_str()); + else if (FFlag::LuauTypeMismatchModuleNameResolution) + report(format, humanReadableName.c_str(), error.location, "TypeError", + Luau::toString(error, Luau::TypeErrorToStringOptions{frontend.fileResolver}).c_str()); else report(format, humanReadableName.c_str(), error.location, "TypeError", Luau::toString(error).c_str()); } diff --git a/CMakeLists.txt b/CMakeLists.txt index c624a132..e256e234 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -11,6 +11,7 @@ option(LUAU_BUILD_TESTS "Build tests" ON) option(LUAU_BUILD_WEB "Build Web module" OFF) option(LUAU_WERROR "Warnings as errors" OFF) option(LUAU_STATIC_CRT "Link with the static CRT (/MT)" OFF) +option(LUAU_EXTERN_C "Use extern C for all APIs" OFF) if(LUAU_STATIC_CRT) cmake_minimum_required(VERSION 3.15) @@ -115,6 +116,14 @@ target_compile_options(Luau.CodeGen PRIVATE ${LUAU_OPTIONS}) target_compile_options(Luau.VM PRIVATE ${LUAU_OPTIONS}) target_compile_options(isocline PRIVATE ${LUAU_OPTIONS} ${ISOCLINE_OPTIONS}) +if(LUAU_EXTERN_C) + # enable extern "C" for VM (lua.h, lualib.h) and Compiler (luacode.h) to make Luau friendlier to use from non-C++ languages + # note that we enable LUA_USE_LONGJMP=1 as well; otherwise functions like luaL_error will throw C++ exceptions, which can't be done from extern "C" functions + target_compile_definitions(Luau.VM PUBLIC LUA_USE_LONGJMP=1) + target_compile_definitions(Luau.VM PUBLIC LUA_API=extern\"C\") + target_compile_definitions(Luau.Compiler PUBLIC LUACODE_API=extern\"C\") +endif() + if (MSVC AND MSVC_VERSION GREATER_EQUAL 1924) # disable partial redundancy elimination which regresses interpreter codegen substantially in VS2022: # https://developercommunity.visualstudio.com/t/performance-regression-on-a-complex-interpreter-lo/1631863 diff --git a/Common/include/Luau/Bytecode.h b/Common/include/Luau/Bytecode.h index f71d893c..218bb5d5 100644 --- a/Common/include/Luau/Bytecode.h +++ b/Common/include/Luau/Bytecode.h @@ -7,7 +7,7 @@ // Creating the bytecode is outside the scope of this file and is handled by bytecode builder (BytecodeBuilder.h) and bytecode compiler (Compiler.h) // Note that ALL enums declared in this file are order-sensitive since the values are baked into bytecode that needs to be processed by legacy clients. -// Bytecode definitions +// # Bytecode definitions // Bytecode instructions are using "word code" - each instruction is one or many 32-bit words. // The first word in the instruction is always the instruction header, and *must* contain the opcode (enum below) in the least significant byte. // @@ -19,7 +19,7 @@ // Instruction word is sometimes followed by one extra word, indicated as AUX - this is just a 32-bit word and is decoded according to the specification for each opcode. // For each opcode the encoding is *static* - that is, based on the opcode you know a-priory how large the instruction is, with the exception of NEWCLOSURE -// Bytecode indices +// # Bytecode indices // Bytecode instructions commonly refer to integer values that define offsets or indices for various entities. For each type, there's a maximum encodable value. // Note that in some cases, the compiler will set a lower limit than the maximum encodable value is to prevent fragile code into bumping against the limits whenever we change the compilation details. // Additionally, in some specific instructions such as ANDK, the limit on the encoded value is smaller; this means that if a value is larger, a different instruction must be selected. @@ -29,6 +29,15 @@ // Constants: 0-2^23-1. Constants are stored in a table allocated with each proto; to allow for future bytecode tweaks the encodable value is limited to 23 bits. // Closures: 0-2^15-1. Closures are created from child protos via a child index; the limit is for the number of closures immediately referenced in each function. // Jumps: -2^23..2^23. Jump offsets are specified in word increments, so jumping over an instruction may sometimes require an offset of 2 or more. + +// # Bytecode versions +// Bytecode serialized format embeds a version number, that dictates both the serialized form as well as the allowed instructions. As long as the bytecode version falls into supported +// range (indicated by LBC_BYTECODE_MIN / LBC_BYTECODE_MAX) and was produced by Luau compiler, it should load and execute correctly. +// +// Note that Luau runtime doesn't provide indefinite bytecode compatibility: support for older versions gets removed over time. As such, bytecode isn't a durable storage format and it's expected +// that Luau users can recompile bytecode from source on Luau version upgrades if necessary. + +// Bytecode opcode, part of the instruction header enum LuauOpcode { // NOP: noop @@ -380,8 +389,10 @@ enum LuauOpcode // Bytecode tags, used internally for bytecode encoded as a string enum LuauBytecodeTag { - // Bytecode version - LBC_VERSION = 2, + // Bytecode version; runtime supports [MIN, MAX], compiler emits TARGET by default but may emit a higher version when flags are enabled + LBC_VERSION_MIN = 2, + LBC_VERSION_MAX = 2, + LBC_VERSION_TARGET = 2, // Types of constant table entries LBC_CONSTANT_NIL = 0, LBC_CONSTANT_BOOLEAN, diff --git a/Compiler/include/Luau/BytecodeBuilder.h b/Compiler/include/Luau/BytecodeBuilder.h index dbe54299..6ec10b53 100644 --- a/Compiler/include/Luau/BytecodeBuilder.h +++ b/Compiler/include/Luau/BytecodeBuilder.h @@ -119,6 +119,8 @@ public: static std::string getError(const std::string& message); + static uint8_t getVersion(); + private: struct Constant { diff --git a/Compiler/src/BytecodeBuilder.cpp b/Compiler/src/BytecodeBuilder.cpp index a34f7603..301cf255 100644 --- a/Compiler/src/BytecodeBuilder.cpp +++ b/Compiler/src/BytecodeBuilder.cpp @@ -9,6 +9,9 @@ namespace Luau { +static_assert(LBC_VERSION_TARGET >= LBC_VERSION_MIN && LBC_VERSION_TARGET <= LBC_VERSION_MAX, "Invalid bytecode version setup"); +static_assert(LBC_VERSION_MAX <= 127, "Bytecode version should be 7-bit so that we can extend the serialization to use varint transparently"); + static const uint32_t kMaxConstantCount = 1 << 23; static const uint32_t kMaxClosureCount = 1 << 15; @@ -572,7 +575,10 @@ void BytecodeBuilder::finalize() bytecode.reserve(capacity); // assemble final bytecode blob - bytecode = char(LBC_VERSION); + uint8_t version = getVersion(); + LUAU_ASSERT(version >= LBC_VERSION_MIN && version <= LBC_VERSION_MAX); + + bytecode = char(version); writeStringTable(bytecode); @@ -1040,7 +1046,7 @@ void BytecodeBuilder::expandJumps() std::string BytecodeBuilder::getError(const std::string& message) { - // 0 acts as a special marker for error bytecode (it's equal to LBC_VERSION for valid bytecode blobs) + // 0 acts as a special marker for error bytecode (it's equal to LBC_VERSION_TARGET for valid bytecode blobs) std::string result; result += char(0); result += message; @@ -1048,6 +1054,12 @@ std::string BytecodeBuilder::getError(const std::string& message) return result; } +uint8_t BytecodeBuilder::getVersion() +{ + // This function usually returns LBC_VERSION_TARGET but may sometimes return a higher number (within LBC_VERSION_MIN/MAX) under fast flags + return LBC_VERSION_TARGET; +} + #ifdef LUAU_ASSERTENABLED void BytecodeBuilder::validate() const { diff --git a/Compiler/src/Compiler.cpp b/Compiler/src/Compiler.cpp index 52dc9242..e732256b 100644 --- a/Compiler/src/Compiler.cpp +++ b/Compiler/src/Compiler.cpp @@ -16,8 +16,6 @@ #include #include -LUAU_FASTFLAGVARIABLE(LuauCompileIterNoPairs, false) - LUAU_FASTINTVARIABLE(LuauCompileLoopUnrollThreshold, 25) LUAU_FASTINTVARIABLE(LuauCompileLoopUnrollThresholdMaxBoost, 300) @@ -2672,7 +2670,7 @@ struct Compiler else if (builtin.isGlobal("pairs")) // for .. in pairs(t) { skipOp = LOP_FORGPREP_NEXT; - loopOp = FFlag::LuauCompileIterNoPairs ? LOP_FORGLOOP : LOP_FORGLOOP_NEXT; + loopOp = LOP_FORGLOOP; } } else if (stat->values.size == 2) @@ -2682,7 +2680,7 @@ struct Compiler if (builtin.isGlobal("next")) // for .. in next,t { skipOp = LOP_FORGPREP_NEXT; - loopOp = FFlag::LuauCompileIterNoPairs ? LOP_FORGLOOP : LOP_FORGLOOP_NEXT; + loopOp = LOP_FORGLOOP; } } } diff --git a/VM/src/ludata.cpp b/VM/src/ludata.cpp index 28152689..c2110cb3 100644 --- a/VM/src/ludata.cpp +++ b/VM/src/ludata.cpp @@ -26,6 +26,8 @@ void luaU_freeudata(lua_State* L, Udata* u, lua_Page* page) { void (*dtor)(lua_State*, void*) = nullptr; dtor = L->global->udatagc[u->tag]; + // TODO: access to L here is highly unsafe since this is called during internal GC traversal + // certain operations such as lua_getthreaddata are okay, but by and large this risks crashes on improper use if (dtor) dtor(L, u->data); } diff --git a/VM/src/lvmload.cpp b/VM/src/lvmload.cpp index 8b742f1c..86afddd2 100644 --- a/VM/src/lvmload.cpp +++ b/VM/src/lvmload.cpp @@ -154,11 +154,11 @@ int luau_load(lua_State* L, const char* chunkname, const char* data, size_t size return 1; } - if (version != LBC_VERSION) + if (version < LBC_VERSION_MIN || version > LBC_VERSION_MAX) { char chunkid[LUA_IDSIZE]; luaO_chunkid(chunkid, chunkname, LUA_IDSIZE); - lua_pushfstring(L, "%s: bytecode version mismatch (expected %d, got %d)", chunkid, LBC_VERSION, version); + lua_pushfstring(L, "%s: bytecode version mismatch (expected [%d..%d], got %d)", chunkid, LBC_VERSION_MIN, LBC_VERSION_MAX, version); return 1; } diff --git a/tests/Compiler.test.cpp b/tests/Compiler.test.cpp index 036bf124..655e48cb 100644 --- a/tests/Compiler.test.cpp +++ b/tests/Compiler.test.cpp @@ -261,8 +261,6 @@ L1: RETURN R0 0 TEST_CASE("ForBytecode") { - ScopedFastFlag sff2("LuauCompileIterNoPairs", false); - // basic for loop: variable directly refers to internal iteration index (R2) CHECK_EQ("\n" + compileFunction0("for i=1,5 do print(i) end"), R"( LOADN R2 1 @@ -329,7 +327,7 @@ L0: GETIMPORT R5 3 MOVE R6 R3 MOVE R7 R4 CALL R5 2 0 -L1: FORGLOOP_NEXT R0 L0 +L1: FORGLOOP R0 L0 2 RETURN R0 0 )"); @@ -342,7 +340,7 @@ L0: GETIMPORT R5 3 MOVE R6 R3 MOVE R7 R4 CALL R5 2 0 -L1: FORGLOOP_NEXT R0 L0 +L1: FORGLOOP R0 L0 2 RETURN R0 0 )"); } @@ -2262,8 +2260,6 @@ TEST_CASE("TypeAliasing") TEST_CASE("DebugLineInfo") { - ScopedFastFlag sff("LuauCompileIterNoPairs", false); - Luau::BytecodeBuilder bcb; bcb.setDumpFlags(Luau::BytecodeBuilder::Dump_Code | Luau::BytecodeBuilder::Dump_Lines); Luau::compileOrThrow(bcb, R"( @@ -2313,7 +2309,7 @@ return result 15: L0: MOVE R7 R1 15: MOVE R8 R5 15: CONCAT R1 R7 R8 -14: L1: FORGLOOP_NEXT R2 L0 +14: L1: FORGLOOP R2 L0 1 17: RETURN R1 1 )"); } @@ -2545,8 +2541,6 @@ a TEST_CASE("DebugSource") { - ScopedFastFlag sff("LuauCompileIterNoPairs", false); - const char* source = R"( local kSelectedBiomes = { ['Mountains'] = true, @@ -2614,7 +2608,7 @@ L0: MOVE R7 R1 MOVE R8 R5 CONCAT R1 R7 R8 14: for k in pairs(kSelectedBiomes) do -L1: FORGLOOP_NEXT R2 L0 +L1: FORGLOOP R2 L0 1 17: return result RETURN R1 1 )"); @@ -2622,8 +2616,6 @@ RETURN R1 1 TEST_CASE("DebugLocals") { - ScopedFastFlag sff("LuauCompileIterNoPairs", false); - const char* source = R"( function foo(e, f) local a = 1 @@ -2661,12 +2653,12 @@ end local 0: reg 5, start pc 5 line 5, end pc 8 line 5 local 1: reg 6, start pc 14 line 8, end pc 18 line 8 local 2: reg 7, start pc 14 line 8, end pc 18 line 8 -local 3: reg 3, start pc 21 line 12, end pc 24 line 12 -local 4: reg 3, start pc 26 line 16, end pc 30 line 16 -local 5: reg 0, start pc 0 line 3, end pc 34 line 21 -local 6: reg 1, start pc 0 line 3, end pc 34 line 21 -local 7: reg 2, start pc 1 line 4, end pc 34 line 21 -local 8: reg 3, start pc 34 line 21, end pc 34 line 21 +local 3: reg 3, start pc 22 line 12, end pc 25 line 12 +local 4: reg 3, start pc 27 line 16, end pc 31 line 16 +local 5: reg 0, start pc 0 line 3, end pc 35 line 21 +local 6: reg 1, start pc 0 line 3, end pc 35 line 21 +local 7: reg 2, start pc 1 line 4, end pc 35 line 21 +local 8: reg 3, start pc 35 line 21, end pc 35 line 21 3: LOADN R2 1 4: LOADN R5 1 4: LOADN R3 3 @@ -2683,7 +2675,7 @@ local 8: reg 3, start pc 34 line 21, end pc 34 line 21 8: MOVE R9 R6 8: MOVE R10 R7 8: CALL R8 2 0 -7: L3: FORGLOOP_NEXT R3 L2 +7: L3: FORGLOOP R3 L2 2 11: LOADN R3 2 12: GETIMPORT R4 1 12: LOADN R5 2 @@ -3795,8 +3787,6 @@ RETURN R0 1 TEST_CASE("SharedClosure") { - ScopedFastFlag sff("LuauCompileIterNoPairs", false); - // closures can be shared even if functions refer to upvalues, as long as upvalues are top-level CHECK_EQ("\n" + compileFunction(R"( local val = ... @@ -3939,7 +3929,7 @@ L2: GETIMPORT R5 1 NEWCLOSURE R6 P1 CAPTURE VAL R3 CALL R5 1 0 -L3: FORGLOOP_NEXT R0 L2 +L3: FORGLOOP R0 L2 2 LOADN R2 1 LOADN R0 10 LOADN R1 1 diff --git a/tests/Fixture.h b/tests/Fixture.h index ffcd4b9e..0e3735f6 100644 --- a/tests/Fixture.h +++ b/tests/Fixture.h @@ -2,13 +2,13 @@ #pragma once #include "Luau/Config.h" -#include "Luau/ConstraintGraphBuilder.h" #include "Luau/FileResolver.h" #include "Luau/Frontend.h" #include "Luau/IostreamHelpers.h" #include "Luau/Linter.h" #include "Luau/Location.h" #include "Luau/ModuleResolver.h" +#include "Luau/Scope.h" #include "Luau/ToString.h" #include "Luau/TypeInfer.h" #include "Luau/TypeVar.h" diff --git a/tests/Module.test.cpp b/tests/Module.test.cpp index d585b731..7c2f4d1c 100644 --- a/tests/Module.test.cpp +++ b/tests/Module.test.cpp @@ -279,7 +279,6 @@ TEST_CASE_FIXTURE(Fixture, "clone_recursion_limit") int limit = 400; #endif ScopedFastInt luauTypeCloneRecursionLimit{"LuauTypeCloneRecursionLimit", limit}; - ScopedFastFlag sff{"LuauRecursionLimitException", true}; TypeArena src; diff --git a/tests/Normalize.test.cpp b/tests/Normalize.test.cpp index 284230c9..a474b6e7 100644 --- a/tests/Normalize.test.cpp +++ b/tests/Normalize.test.cpp @@ -12,7 +12,6 @@ using namespace Luau; struct NormalizeFixture : Fixture { ScopedFastFlag sff1{"LuauLowerBoundsCalculation", true}; - ScopedFastFlag sff2{"LuauTableSubtypingVariance2", true}; }; void createSomeClasses(TypeChecker& typeChecker) diff --git a/tests/RuntimeLimits.test.cpp b/tests/RuntimeLimits.test.cpp index bef38fc3..6619147b 100644 --- a/tests/RuntimeLimits.test.cpp +++ b/tests/RuntimeLimits.test.cpp @@ -264,10 +264,13 @@ TEST_CASE_FIXTURE(LimitFixture, "typescript_port_of_Result_type") } )LUA"; + CheckResult result = check(src); + CodeTooComplex ctc; + if (FFlag::LuauLowerBoundsCalculation) - (void)check(src); + LUAU_REQUIRE_ERRORS(result); else - CHECK_THROWS_AS(check(src), std::exception); + CHECK(hasError(result, &ctc)); } TEST_SUITE_END(); diff --git a/tests/ToString.test.cpp b/tests/ToString.test.cpp index 4d2e94ee..e03069a9 100644 --- a/tests/ToString.test.cpp +++ b/tests/ToString.test.cpp @@ -409,8 +409,6 @@ TEST_CASE_FIXTURE(Fixture, "toStringDetailed") TEST_CASE_FIXTURE(BuiltinsFixture, "toStringDetailed2") { - ScopedFastFlag sff{"LuauUnsealedTableLiteral", true}; - CheckResult result = check(R"( local base = {} function base:one() return 1 end diff --git a/tests/TypeInfer.aliases.test.cpp b/tests/TypeInfer.aliases.test.cpp index 86cc9701..d6f0a0c8 100644 --- a/tests/TypeInfer.aliases.test.cpp +++ b/tests/TypeInfer.aliases.test.cpp @@ -7,8 +7,21 @@ using namespace Luau; +LUAU_FASTFLAG(DebugLuauDeferredConstraintResolution) + TEST_SUITE_BEGIN("TypeAliases"); +TEST_CASE_FIXTURE(Fixture, "basic_alias") +{ + CheckResult result = check(R"( + type T = number + local x: T = 1 + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + CHECK_EQ("number", toString(requireType("x"))); +} + TEST_CASE_FIXTURE(Fixture, "cyclic_function_type_in_type_alias") { CheckResult result = check(R"( @@ -24,6 +37,63 @@ TEST_CASE_FIXTURE(Fixture, "cyclic_function_type_in_type_alias") CHECK_EQ("t1 where t1 = () -> t1?", toString(requireType("g"))); } +TEST_CASE_FIXTURE(Fixture, "names_are_ascribed") +{ + CheckResult result = check(R"( + type T = { x: number } + local x: T + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + CHECK_EQ("T", toString(requireType("x"))); +} + +TEST_CASE_FIXTURE(Fixture, "cannot_steal_hoisted_type_alias") +{ + // This is a tricky case. In order to support recursive type aliases, + // we first walk the block and generate free types as placeholders. + // We then walk the AST as normal. If we declare a type alias as below, + // we generate a free type. We then begin our normal walk, examining + // local x: T = "foo", which establishes two constraints: + // a <: b + // string <: a + // We then visit the type alias, and establish that + // b <: number + // Then, when solving these constraints, we dispatch them in the order + // they appear above. This means that a ~ b, and a ~ string, thus + // b ~ string. This means the b <: number constraint has no effect. + // Essentially we've "stolen" the alias's type out from under it. + // This test ensures that we don't actually do this. + CheckResult result = check(R"( + local x: T = "foo" + type T = number + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + if (FFlag::DebugLuauDeferredConstraintResolution) + { + CHECK(result.errors[0] == TypeError{ + Location{{1, 21}, {1, 26}}, + getMainSourceModule()->name, + TypeMismatch{ + getSingletonTypes().numberType, + getSingletonTypes().stringType, + }, + }); + } + else + { + CHECK(result.errors[0] == TypeError{ + Location{{1, 8}, {1, 26}}, + getMainSourceModule()->name, + TypeMismatch{ + getSingletonTypes().numberType, + getSingletonTypes().stringType, + }, + }); + } +} + TEST_CASE_FIXTURE(Fixture, "cyclic_types_of_named_table_fields_do_not_expand_when_stringified") { CheckResult result = check(R"( @@ -41,7 +111,22 @@ TEST_CASE_FIXTURE(Fixture, "cyclic_types_of_named_table_fields_do_not_expand_whe CHECK_EQ(typeChecker.numberType, tm->givenType); } -TEST_CASE_FIXTURE(Fixture, "mutually_recursive_types") +TEST_CASE_FIXTURE(Fixture, "mutually_recursive_aliases") +{ + CheckResult result = check(R"( + --!strict + type T = { f: number, g: U } + type U = { h: number, i: T? } + local x: T = { f = 37, g = { h = 5, i = nil } } + x.g.i = x + local y: T = { f = 3, g = { h = 5, i = nil } } + y.g.i = y + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "mutually_recursive_generic_aliases") { CheckResult result = check(R"( --!strict diff --git a/tests/TypeInfer.annotations.test.cpp b/tests/TypeInfer.annotations.test.cpp index ccdd2b37..3e2ad6dc 100644 --- a/tests/TypeInfer.annotations.test.cpp +++ b/tests/TypeInfer.annotations.test.cpp @@ -30,11 +30,21 @@ TEST_CASE_FIXTURE(Fixture, "successful_check") dumpErrors(result); } +TEST_CASE_FIXTURE(Fixture, "variable_type_is_supertype") +{ + CheckResult result = check(R"( + local x: number = 1 + local y: number? = x + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + TEST_CASE_FIXTURE(Fixture, "function_parameters_can_have_annotations") { CheckResult result = check(R"( function double(x: number) - return x * 2 + return 2 end local four = double(2) @@ -47,7 +57,7 @@ TEST_CASE_FIXTURE(Fixture, "function_parameter_annotations_are_checked") { CheckResult result = check(R"( function double(x: number) - return x * 2 + return 2 end local four = double("two") @@ -70,13 +80,13 @@ TEST_CASE_FIXTURE(Fixture, "function_return_annotations_are_checked") const FunctionTypeVar* ftv = get(fiftyType); REQUIRE(ftv != nullptr); - TypePackId retPack = ftv->retTypes; + TypePackId retPack = follow(ftv->retTypes); const TypePack* tp = get(retPack); REQUIRE(tp != nullptr); REQUIRE_EQ(1, tp->head.size()); - REQUIRE_EQ(typeChecker.anyType, tp->head[0]); + REQUIRE_EQ(typeChecker.anyType, follow(tp->head[0])); } TEST_CASE_FIXTURE(Fixture, "function_return_multret_annotations_are_checked") @@ -116,6 +126,23 @@ TEST_CASE_FIXTURE(Fixture, "function_return_annotation_should_continuously_parse LUAU_REQUIRE_ERROR_COUNT(1, result); } +TEST_CASE_FIXTURE(Fixture, "unknown_type_reference_generates_error") +{ + CheckResult result = check(R"( + local x: IDoNotExist + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK(result.errors[0] == TypeError{ + Location{{1, 17}, {1, 28}}, + getMainSourceModule()->name, + UnknownSymbol{ + "IDoNotExist", + UnknownSymbol::Context::Type, + }, + }); +} + TEST_CASE_FIXTURE(Fixture, "typeof_variable_type_annotation_should_return_its_type") { CheckResult result = check(R"( @@ -632,7 +659,10 @@ int AssertionCatcher::tripped; TEST_CASE_FIXTURE(Fixture, "luau_ice_triggers_an_ice") { - ScopedFastFlag sffs{"DebugLuauMagicTypes", true}; + ScopedFastFlag sffs[] = { + {"DebugLuauMagicTypes", true}, + {"LuauUseInternalCompilerErrorException", false}, + }; AssertionCatcher ac; @@ -646,9 +676,10 @@ TEST_CASE_FIXTURE(Fixture, "luau_ice_triggers_an_ice") TEST_CASE_FIXTURE(Fixture, "luau_ice_triggers_an_ice_handler") { - ScopedFastFlag sffs{"DebugLuauMagicTypes", true}; - - AssertionCatcher ac; + ScopedFastFlag sffs[] = { + {"DebugLuauMagicTypes", true}, + {"LuauUseInternalCompilerErrorException", false}, + }; bool caught = false; @@ -662,8 +693,44 @@ TEST_CASE_FIXTURE(Fixture, "luau_ice_triggers_an_ice_handler") std::runtime_error); CHECK_EQ(true, caught); +} - frontend.iceHandler.onInternalError = {}; +TEST_CASE_FIXTURE(Fixture, "luau_ice_triggers_an_ice_exception_with_flag") +{ + ScopedFastFlag sffs[] = { + {"DebugLuauMagicTypes", true}, + {"LuauUseInternalCompilerErrorException", true}, + }; + + AssertionCatcher ac; + + CHECK_THROWS_AS(check(R"( + local a: _luau_ice = 55 + )"), + InternalCompilerError); + + LUAU_ASSERT(1 == AssertionCatcher::tripped); +} + +TEST_CASE_FIXTURE(Fixture, "luau_ice_triggers_an_ice_exception_with_flag_handler") +{ + ScopedFastFlag sffs[] = { + {"DebugLuauMagicTypes", true}, + {"LuauUseInternalCompilerErrorException", true}, + }; + + bool caught = false; + + frontend.iceHandler.onInternalError = [&](const char*) { + caught = true; + }; + + CHECK_THROWS_AS(check(R"( + local a: _luau_ice = 55 + )"), + InternalCompilerError); + + CHECK_EQ(true, caught); } TEST_CASE_FIXTURE(Fixture, "luau_ice_is_not_special_without_the_flag") diff --git a/tests/TypeInfer.generics.test.cpp b/tests/TypeInfer.generics.test.cpp index edb5adcf..97ba0808 100644 --- a/tests/TypeInfer.generics.test.cpp +++ b/tests/TypeInfer.generics.test.cpp @@ -700,11 +700,6 @@ end TEST_CASE_FIXTURE(Fixture, "generic_functions_should_be_memory_safe") { - ScopedFastFlag sffs[] = { - {"LuauTableSubtypingVariance2", true}, - {"LuauUnsealedTableLiteral", true}, - }; - CheckResult result = check(R"( --!strict -- At one point this produced a UAF @@ -979,8 +974,6 @@ TEST_CASE_FIXTURE(Fixture, "instantiate_generic_function_in_assignments2") TEST_CASE_FIXTURE(Fixture, "self_recursive_instantiated_param") { - ScopedFastFlag sff{"LuauOnlyMutateInstantiatedTables", true}; - // Mutability in type function application right now can create strange recursive types CheckResult result = check(R"( type Table = { a: number } @@ -1015,8 +1008,6 @@ TEST_CASE_FIXTURE(Fixture, "no_stack_overflow_from_quantifying") TEST_CASE_FIXTURE(BuiltinsFixture, "infer_generic_function_function_argument") { - ScopedFastFlag sff{"LuauUnsealedTableLiteral", true}; - CheckResult result = check(R"( local function sum(x: a, y: a, f: (a, a) -> a) return f(x, y) @@ -1123,8 +1114,6 @@ TEST_CASE_FIXTURE(Fixture, "substitution_with_bound_table") TEST_CASE_FIXTURE(Fixture, "apply_type_function_nested_generics1") { - ScopedFastFlag sff{"LuauApplyTypeFunctionFix", true}; - // https://github.com/Roblox/luau/issues/484 CheckResult result = check(R"( --!strict @@ -1153,8 +1142,6 @@ local complex: ComplexObject = { TEST_CASE_FIXTURE(Fixture, "apply_type_function_nested_generics2") { - ScopedFastFlag sff{"LuauApplyTypeFunctionFix", true}; - // https://github.com/Roblox/luau/issues/484 CheckResult result = check(R"( --!strict diff --git a/tests/TypeInfer.modules.test.cpp b/tests/TypeInfer.modules.test.cpp index afec20bf..a0f670f1 100644 --- a/tests/TypeInfer.modules.test.cpp +++ b/tests/TypeInfer.modules.test.cpp @@ -12,8 +12,6 @@ using namespace Luau; -LUAU_FASTFLAG(LuauTableSubtypingVariance2) - TEST_SUITE_BEGIN("TypeInferModules"); TEST_CASE_FIXTURE(BuiltinsFixture, "require") @@ -326,16 +324,9 @@ local b: B.T = a CheckResult result = frontend.check("game/C"); LUAU_REQUIRE_ERROR_COUNT(1, result); - if (FFlag::LuauTableSubtypingVariance2) - { - CHECK_EQ(toString(result.errors[0]), R"(Type 'T' from 'game/A' could not be converted into 'T' from 'game/B' + CHECK_EQ(toString(result.errors[0]), R"(Type 'T' from 'game/A' could not be converted into 'T' from 'game/B' caused by: Property 'x' is not compatible. Type 'number' could not be converted into 'string')"); - } - else - { - CHECK_EQ(toString(result.errors[0]), "Type 'T' from 'game/A' could not be converted into 'T' from 'game/B'"); - } } TEST_CASE_FIXTURE(BuiltinsFixture, "module_type_conflict_instantiated") @@ -367,16 +358,9 @@ local b: B.T = a CheckResult result = frontend.check("game/D"); LUAU_REQUIRE_ERROR_COUNT(1, result); - if (FFlag::LuauTableSubtypingVariance2) - { - CHECK_EQ(toString(result.errors[0]), R"(Type 'T' from 'game/B' could not be converted into 'T' from 'game/C' + CHECK_EQ(toString(result.errors[0]), R"(Type 'T' from 'game/B' could not be converted into 'T' from 'game/C' caused by: Property 'x' is not compatible. Type 'number' could not be converted into 'string')"); - } - else - { - CHECK_EQ(toString(result.errors[0]), "Type 'T' from 'game/B' could not be converted into 'T' from 'game/C'"); - } } TEST_SUITE_END(); diff --git a/tests/TypeInfer.refinements.test.cpp b/tests/TypeInfer.refinements.test.cpp index cefba4b2..3f5dad3d 100644 --- a/tests/TypeInfer.refinements.test.cpp +++ b/tests/TypeInfer.refinements.test.cpp @@ -353,8 +353,6 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "assert_non_binary_expressions_actually_resol TEST_CASE_FIXTURE(Fixture, "assign_table_with_refined_property_with_a_similar_type_is_illegal") { - ScopedFastFlag LuauTableSubtypingVariance2{"LuauTableSubtypingVariance2", true}; - CheckResult result = check(R"( local t: {x: number?} = {x = nil} diff --git a/tests/TypeInfer.singletons.test.cpp b/tests/TypeInfer.singletons.test.cpp index a90f434f..4a88abee 100644 --- a/tests/TypeInfer.singletons.test.cpp +++ b/tests/TypeInfer.singletons.test.cpp @@ -260,10 +260,6 @@ TEST_CASE_FIXTURE(Fixture, "table_properties_alias_or_parens_is_indexer") TEST_CASE_FIXTURE(Fixture, "table_properties_type_error_escapes") { - ScopedFastFlag sffs[]{ - {"LuauUnsealedTableLiteral", true}, - }; - CheckResult result = check(R"( --!strict local x: { ["<>"] : number } diff --git a/tests/TypeInfer.tables.test.cpp b/tests/TypeInfer.tables.test.cpp index 87d49651..77a2928c 100644 --- a/tests/TypeInfer.tables.test.cpp +++ b/tests/TypeInfer.tables.test.cpp @@ -276,8 +276,6 @@ TEST_CASE_FIXTURE(Fixture, "open_table_unification") TEST_CASE_FIXTURE(Fixture, "open_table_unification_2") { - ScopedFastFlag sff{"LuauTableSubtypingVariance2", true}; - CheckResult result = check(R"( local a = {} a.x = 99 @@ -347,8 +345,6 @@ TEST_CASE_FIXTURE(Fixture, "table_param_row_polymorphism_1") TEST_CASE_FIXTURE(Fixture, "table_param_row_polymorphism_2") { - ScopedFastFlag sff{"LuauTableSubtypingVariance2", true}; - CheckResult result = check(R"( --!strict function foo(o) @@ -370,8 +366,6 @@ TEST_CASE_FIXTURE(Fixture, "table_param_row_polymorphism_2") TEST_CASE_FIXTURE(Fixture, "table_param_row_polymorphism_3") { - ScopedFastFlag sff{"LuauTableSubtypingVariance2", true}; - CheckResult result = check(R"( local T = {} T.bar = 'hello' @@ -477,8 +471,6 @@ TEST_CASE_FIXTURE(Fixture, "ok_to_add_property_to_free_table") TEST_CASE_FIXTURE(Fixture, "okay_to_add_property_to_unsealed_tables_by_assignment") { - ScopedFastFlag sff{"LuauTableSubtypingVariance2", true}; - CheckResult result = check(R"( --!strict local t = { u = {} } @@ -512,8 +504,6 @@ TEST_CASE_FIXTURE(Fixture, "okay_to_add_property_to_unsealed_tables_by_function_ TEST_CASE_FIXTURE(Fixture, "width_subtyping") { - ScopedFastFlag sff{"LuauTableSubtypingVariance2", true}; - CheckResult result = check(R"( --!strict function f(x : { q : number }) @@ -772,8 +762,6 @@ TEST_CASE_FIXTURE(Fixture, "infer_indexer_for_left_unsealed_table_from_right_han TEST_CASE_FIXTURE(Fixture, "sealed_table_value_can_infer_an_indexer") { - ScopedFastFlag sff{"LuauTableSubtypingVariance2", true}; - CheckResult result = check(R"( local t: { a: string, [number]: string } = { a = "foo" } )"); @@ -783,8 +771,6 @@ TEST_CASE_FIXTURE(Fixture, "sealed_table_value_can_infer_an_indexer") TEST_CASE_FIXTURE(Fixture, "array_factory_function") { - ScopedFastFlag sff{"LuauTableSubtypingVariance2", true}; - CheckResult result = check(R"( function empty() return {} end local array: {string} = empty() @@ -1175,8 +1161,6 @@ TEST_CASE_FIXTURE(Fixture, "defining_a_self_method_for_a_local_sealed_table_must TEST_CASE_FIXTURE(Fixture, "defining_a_method_for_a_local_unsealed_table_is_ok") { - ScopedFastFlag sff{"LuauUnsealedTableLiteral", true}; - CheckResult result = check(R"( local t = {x = 1} function t.m() end @@ -1187,8 +1171,6 @@ TEST_CASE_FIXTURE(Fixture, "defining_a_method_for_a_local_unsealed_table_is_ok") TEST_CASE_FIXTURE(Fixture, "defining_a_self_method_for_a_local_unsealed_table_is_ok") { - ScopedFastFlag sff{"LuauUnsealedTableLiteral", true}; - CheckResult result = check(R"( local t = {x = 1} function t:m() end @@ -1468,11 +1450,6 @@ TEST_CASE_FIXTURE(Fixture, "right_table_missing_key2") TEST_CASE_FIXTURE(Fixture, "casting_unsealed_tables_with_props_into_table_with_indexer") { - ScopedFastFlag sff[]{ - {"LuauTableSubtypingVariance2", true}, - {"LuauUnsealedTableLiteral", true}, - }; - CheckResult result = check(R"( type StringToStringMap = { [string]: string } local rt: StringToStringMap = { ["foo"] = 1 } @@ -1518,11 +1495,6 @@ TEST_CASE_FIXTURE(Fixture, "casting_tables_with_props_into_table_with_indexer2") TEST_CASE_FIXTURE(Fixture, "casting_tables_with_props_into_table_with_indexer3") { - ScopedFastFlag sff[]{ - {"LuauTableSubtypingVariance2", true}, - {"LuauUnsealedTableLiteral", true}, - }; - CheckResult result = check(R"( local function foo(a: {[string]: number, a: string}) end foo({ a = 1 }) @@ -1609,8 +1581,6 @@ TEST_CASE_FIXTURE(Fixture, "table_subtyping_with_extra_props_dont_report_multipl TEST_CASE_FIXTURE(Fixture, "table_subtyping_with_extra_props_is_ok") { - ScopedFastFlag sff{"LuauTableSubtypingVariance2", true}; - CheckResult result = check(R"( local vec3 = {x = 1, y = 2, z = 3} local vec1 = {x = 1} @@ -1998,8 +1968,6 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "table_insert_should_cope_with_optional_prope TEST_CASE_FIXTURE(BuiltinsFixture, "table_insert_should_cope_with_optional_properties_in_strict") { - ScopedFastFlag sff{"LuauTableSubtypingVariance2", true}; - CheckResult result = check(R"( --!strict local buttons = {} @@ -2013,8 +1981,6 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "table_insert_should_cope_with_optional_prope TEST_CASE_FIXTURE(Fixture, "error_detailed_prop") { - ScopedFastFlag LuauTableSubtypingVariance2{"LuauTableSubtypingVariance2", true}; // Only for new path - CheckResult result = check(R"( type A = { x: number, y: number } type B = { x: number, y: string } @@ -2031,8 +1997,6 @@ caused by: TEST_CASE_FIXTURE(Fixture, "error_detailed_prop_nested") { - ScopedFastFlag LuauTableSubtypingVariance2{"LuauTableSubtypingVariance2", true}; // Only for new path - CheckResult result = check(R"( type AS = { x: number, y: number } type BS = { x: number, y: string } @@ -2054,11 +2018,6 @@ caused by: TEST_CASE_FIXTURE(BuiltinsFixture, "error_detailed_metatable_prop") { - ScopedFastFlag sff[]{ - {"LuauTableSubtypingVariance2", true}, - {"LuauUnsealedTableLiteral", true}, - }; - CheckResult result = check(R"( local a1 = setmetatable({ x = 2, y = 3 }, { __call = function(s) end }); local b1 = setmetatable({ x = 2, y = "hello" }, { __call = function(s) end }); @@ -2085,8 +2044,6 @@ caused by: TEST_CASE_FIXTURE(Fixture, "error_detailed_indexer_key") { - ScopedFastFlag luauTableSubtypingVariance2{"LuauTableSubtypingVariance2", true}; // Only for new path - CheckResult result = check(R"( type A = { [number]: string } type B = { [string]: string } @@ -2103,8 +2060,6 @@ caused by: TEST_CASE_FIXTURE(Fixture, "error_detailed_indexer_value") { - ScopedFastFlag luauTableSubtypingVariance2{"LuauTableSubtypingVariance2", true}; // Only for new path - CheckResult result = check(R"( type A = { [number]: number } type B = { [number]: string } @@ -2121,10 +2076,6 @@ caused by: TEST_CASE_FIXTURE(Fixture, "explicitly_typed_table") { - ScopedFastFlag sffs[]{ - {"LuauTableSubtypingVariance2", true}, - }; - CheckResult result = check(R"( --!strict type Super = { x : number } @@ -2140,11 +2091,6 @@ a.p = { x = 9 } TEST_CASE_FIXTURE(Fixture, "explicitly_typed_table_error") { - ScopedFastFlag sffs[]{ - {"LuauTableSubtypingVariance2", true}, - {"LuauUnsealedTableLiteral", true}, - }; - CheckResult result = check(R"( --!strict type Super = { x : number } @@ -2166,10 +2112,6 @@ caused by: TEST_CASE_FIXTURE(Fixture, "explicitly_typed_table_with_indexer") { - ScopedFastFlag sffs[]{ - {"LuauTableSubtypingVariance2", true}, - }; - CheckResult result = check(R"( --!strict type Super = { x : number } @@ -2185,10 +2127,6 @@ a.p = { x = 9 } TEST_CASE_FIXTURE(BuiltinsFixture, "recursive_metatable_type_call") { - ScopedFastFlag sff[]{ - {"LuauUnsealedTableLiteral", true}, - }; - CheckResult result = check(R"( local b b = setmetatable({}, {__call = b}) @@ -2201,11 +2139,6 @@ b() TEST_CASE_FIXTURE(Fixture, "table_subtyping_shouldn't_add_optional_properties_to_sealed_tables") { - ScopedFastFlag sffs[] = { - {"LuauTableSubtypingVariance2", true}, - {"LuauSubtypingAddOptPropsToUnsealedTables", true}, - }; - CheckResult result = check(R"( --!strict local function setNumber(t: { p: number? }, x:number) t.p = x end @@ -2706,8 +2639,6 @@ type t0 = any TEST_CASE_FIXTURE(BuiltinsFixture, "instantiate_table_cloning_2") { - ScopedFastFlag sff{"LuauOnlyMutateInstantiatedTables", true}; - CheckResult result = check(R"( type X = T type K = X @@ -2725,8 +2656,6 @@ type K = X TEST_CASE_FIXTURE(Fixture, "instantiate_table_cloning_3") { - ScopedFastFlag sff{"LuauOnlyMutateInstantiatedTables", true}; - CheckResult result = check(R"( type X = T local a = {} @@ -2977,8 +2906,6 @@ TEST_CASE_FIXTURE(Fixture, "mixed_tables_with_implicit_numbered_keys") TEST_CASE_FIXTURE(Fixture, "expected_indexer_value_type_extra") { - ScopedFastFlag luauSubtypingAddOptPropsToUnsealedTables{"LuauSubtypingAddOptPropsToUnsealedTables", true}; - CheckResult result = check(R"( type X = { { x: boolean?, y: boolean? } } diff --git a/tests/TypeInfer.test.cpp b/tests/TypeInfer.test.cpp index 6257cda6..6a048b26 100644 --- a/tests/TypeInfer.test.cpp +++ b/tests/TypeInfer.test.cpp @@ -887,8 +887,6 @@ end TEST_CASE_FIXTURE(Fixture, "cli_50041_committing_txnlog_in_apollo_client_error") { - ScopedFastFlag subtypingVariance{"LuauTableSubtypingVariance2", true}; - CheckResult result = check(R"( --!strict --!nolint @@ -928,7 +926,6 @@ TEST_CASE_FIXTURE(Fixture, "cli_50041_committing_txnlog_in_apollo_client_error") TEST_CASE_FIXTURE(Fixture, "type_infer_recursion_limit_no_ice") { ScopedFastInt sfi("LuauTypeInferRecursionLimit", 2); - ScopedFastFlag sff{"LuauRecursionLimitException", true}; CheckResult result = check(R"( function complex() diff --git a/tests/TypeInfer.unionTypes.test.cpp b/tests/TypeInfer.unionTypes.test.cpp index d19d80cb..2b48133d 100644 --- a/tests/TypeInfer.unionTypes.test.cpp +++ b/tests/TypeInfer.unionTypes.test.cpp @@ -428,12 +428,6 @@ y = x TEST_CASE_FIXTURE(Fixture, "unify_sealed_table_union_check") { - ScopedFastFlag sffs[] = { - {"LuauTableSubtypingVariance2", true}, - {"LuauUnsealedTableLiteral", true}, - {"LuauSubtypingAddOptPropsToUnsealedTables", true}, - }; - CheckResult result = check(R"( -- the difference between this and unify_unsealed_table_union_check is the type annotation on x local t = { x = 3, y = true } diff --git a/tests/VisitTypeVar.test.cpp b/tests/VisitTypeVar.test.cpp index 01960fbe..4fba694a 100644 --- a/tests/VisitTypeVar.test.cpp +++ b/tests/VisitTypeVar.test.cpp @@ -10,14 +10,9 @@ using namespace Luau; LUAU_FASTINT(LuauVisitRecursionLimit) -struct VisitTypeVarFixture : Fixture -{ - ScopedFastFlag flag2 = {"LuauRecursionLimitException", true}; -}; - TEST_SUITE_BEGIN("VisitTypeVar"); -TEST_CASE_FIXTURE(VisitTypeVarFixture, "throw_when_limit_is_exceeded") +TEST_CASE_FIXTURE(Fixture, "throw_when_limit_is_exceeded") { ScopedFastInt sfi{"LuauVisitRecursionLimit", 3}; @@ -30,7 +25,7 @@ TEST_CASE_FIXTURE(VisitTypeVarFixture, "throw_when_limit_is_exceeded") CHECK_THROWS_AS(toString(tType), RecursionLimitException); } -TEST_CASE_FIXTURE(VisitTypeVarFixture, "dont_throw_when_limit_is_high_enough") +TEST_CASE_FIXTURE(Fixture, "dont_throw_when_limit_is_high_enough") { ScopedFastInt sfi{"LuauVisitRecursionLimit", 8}; From 08ab7da4db08cb7457743718971712a1aef4f0a5 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Thu, 23 Jun 2022 18:56:00 -0700 Subject: [PATCH 285/399] Sync to upstream/release/533 (#560) --- Analysis/include/Luau/Constraint.h | 14 +- .../include/Luau/ConstraintGraphBuilder.h | 41 ++- Analysis/include/Luau/ConstraintSolver.h | 5 +- .../include/Luau/ConstraintSolverLogger.h | 4 +- Analysis/include/Luau/Error.h | 45 ++- Analysis/include/Luau/IostreamHelpers.h | 1 + Analysis/include/Luau/Module.h | 9 +- Analysis/include/Luau/Normalize.h | 4 +- Analysis/include/Luau/RecursionCounter.h | 15 +- Analysis/include/Luau/Scope.h | 18 + Analysis/include/Luau/TypeVar.h | 1 - Analysis/include/Luau/Unifiable.h | 1 + Analysis/include/Luau/Unifier.h | 4 - Analysis/include/Luau/VisitTypeVar.h | 2 +- Analysis/src/Clone.cpp | 4 +- Analysis/src/Constraint.cpp | 3 +- Analysis/src/ConstraintGraphBuilder.cpp | 285 +++++++++++++-- Analysis/src/ConstraintSolver.cpp | 35 +- Analysis/src/Error.cpp | 93 ++++- Analysis/src/Frontend.cpp | 2 + Analysis/src/IostreamHelpers.cpp | 2 + Analysis/src/Module.cpp | 1 - Analysis/src/Quantify.cpp | 9 +- Analysis/src/Scope.cpp | 32 ++ Analysis/src/ToString.cpp | 6 + Analysis/src/TypeChecker2.cpp | 173 +++++++++ Analysis/src/TypeInfer.cpp | 80 ++--- Analysis/src/TypeVar.cpp | 22 +- Analysis/src/Unifiable.cpp | 8 + Analysis/src/Unifier.cpp | 331 +----------------- CLI/Analyze.cpp | 4 + CMakeLists.txt | 9 + Common/include/Luau/Bytecode.h | 19 +- Compiler/include/Luau/BytecodeBuilder.h | 2 + Compiler/src/BytecodeBuilder.cpp | 16 +- Compiler/src/Compiler.cpp | 6 +- VM/src/ludata.cpp | 2 + VM/src/lvmload.cpp | 4 +- tests/Compiler.test.cpp | 34 +- tests/Fixture.h | 2 +- tests/Module.test.cpp | 1 - tests/Normalize.test.cpp | 1 - tests/RuntimeLimits.test.cpp | 7 +- tests/ToString.test.cpp | 2 - tests/TypeInfer.aliases.test.cpp | 87 ++++- tests/TypeInfer.annotations.test.cpp | 85 ++++- tests/TypeInfer.generics.test.cpp | 13 - tests/TypeInfer.modules.test.cpp | 20 +- tests/TypeInfer.refinements.test.cpp | 2 - tests/TypeInfer.singletons.test.cpp | 4 - tests/TypeInfer.tables.test.cpp | 73 ---- tests/TypeInfer.test.cpp | 3 - tests/TypeInfer.unionTypes.test.cpp | 6 - tests/VisitTypeVar.test.cpp | 9 +- 54 files changed, 968 insertions(+), 693 deletions(-) diff --git a/Analysis/include/Luau/Constraint.h b/Analysis/include/Luau/Constraint.h index c62166e2..8a41c9e8 100644 --- a/Analysis/include/Luau/Constraint.h +++ b/Analysis/include/Luau/Constraint.h @@ -1,10 +1,10 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #pragma once -#include "Luau/Location.h" #include "Luau/NotNull.h" #include "Luau/Variant.h" +#include #include #include @@ -47,18 +47,24 @@ struct InstantiationConstraint TypeId superType; }; -using ConstraintV = Variant; +// name(namedType) = name +struct NameConstraint +{ + TypeId namedType; + std::string name; +}; + +using ConstraintV = Variant; using ConstraintPtr = std::unique_ptr; struct Constraint { - Constraint(ConstraintV&& c, Location location); + explicit Constraint(ConstraintV&& c); Constraint(const Constraint&) = delete; Constraint& operator=(const Constraint&) = delete; ConstraintV c; - Location location; std::vector> dependencies; }; diff --git a/Analysis/include/Luau/ConstraintGraphBuilder.h b/Analysis/include/Luau/ConstraintGraphBuilder.h index da774a2a..9b118691 100644 --- a/Analysis/include/Luau/ConstraintGraphBuilder.h +++ b/Analysis/include/Luau/ConstraintGraphBuilder.h @@ -17,20 +17,7 @@ namespace Luau { -struct Scope2 -{ - // The parent scope of this scope. Null if there is no parent (i.e. this - // is the module-level scope). - Scope2* parent = nullptr; - // All the children of this scope. - std::vector children; - std::unordered_map bindings; // TODO: I think this can be a DenseHashMap - TypePackId returnType; - // All constraints belonging to this scope. - std::vector constraints; - - std::optional lookup(Symbol sym); -}; +struct Scope2; struct ConstraintGraphBuilder { @@ -47,6 +34,10 @@ struct ConstraintGraphBuilder // A mapping of AST node to TypePackId. DenseHashMap astTypePacks{nullptr}; DenseHashMap astOriginalCallTypes{nullptr}; + // Types resolved from type annotations. Analogous to astTypes. + DenseHashMap astResolvedTypes{nullptr}; + // Type packs resolved from type annotations. Analogous to astTypePacks. + DenseHashMap astResolvedTypePacks{nullptr}; explicit ConstraintGraphBuilder(TypeArena* arena); @@ -73,9 +64,8 @@ struct ConstraintGraphBuilder * Adds a new constraint with no dependencies to a given scope. * @param scope the scope to add the constraint to. Must not be null. * @param cv the constraint variant to add. - * @param location the location to attribute to the constraint. */ - void addConstraint(Scope2* scope, ConstraintV cv, Location location); + void addConstraint(Scope2* scope, ConstraintV cv); /** * Adds a constraint to a given scope. @@ -99,6 +89,7 @@ struct ConstraintGraphBuilder void visit(Scope2* scope, AstStatReturn* ret); void visit(Scope2* scope, AstStatAssign* assign); void visit(Scope2* scope, AstStatIf* ifStatement); + void visit(Scope2* scope, AstStatTypeAlias* alias); TypePackId checkExprList(Scope2* scope, const AstArray& exprs); @@ -124,6 +115,24 @@ struct ConstraintGraphBuilder * @param fn the function expression to check. */ void checkFunctionBody(Scope2* scope, AstExprFunction* fn); + + /** + * Resolves a type from its AST annotation. + * @param scope the scope that the type annotation appears within. + * @param ty the AST annotation to resolve. + * @return the type of the AST annotation. + **/ + TypeId resolveType(Scope2* scope, AstType* ty); + + /** + * Resolves a type pack from its AST annotation. + * @param scope the scope that the type annotation appears within. + * @param tp the AST annotation to resolve. + * @return the type pack of the AST annotation. + **/ + TypePackId resolveTypePack(Scope2* scope, AstTypePack* tp); + + TypePackId resolveTypePack(Scope2* scope, const AstTypeList& list); }; /** diff --git a/Analysis/include/Luau/ConstraintSolver.h b/Analysis/include/Luau/ConstraintSolver.h index 7e6d4461..4870157f 100644 --- a/Analysis/include/Luau/ConstraintSolver.h +++ b/Analysis/include/Luau/ConstraintSolver.h @@ -55,6 +55,7 @@ struct ConstraintSolver bool tryDispatch(const PackSubtypeConstraint& c, NotNull constraint, bool force); bool tryDispatch(const GeneralizationConstraint& c, NotNull constraint, bool force); bool tryDispatch(const InstantiationConstraint& c, NotNull constraint, bool force); + bool tryDispatch(const NameConstraint& c, NotNull constraint); void block(NotNull target, NotNull constraint); /** @@ -85,7 +86,7 @@ struct ConstraintSolver * @param subType the sub-type to unify. * @param superType the super-type to unify. */ - void unify(TypeId subType, TypeId superType, Location location); + void unify(TypeId subType, TypeId superType); /** * Creates a new Unifier and performs a single unification operation. Commits @@ -93,7 +94,7 @@ struct ConstraintSolver * @param subPack the sub-type pack to unify. * @param superPack the super-type pack to unify. */ - void unify(TypePackId subPack, TypePackId superPack, Location location); + void unify(TypePackId subPack, TypePackId superPack); private: /** diff --git a/Analysis/include/Luau/ConstraintSolverLogger.h b/Analysis/include/Luau/ConstraintSolverLogger.h index 2b195d71..55336a23 100644 --- a/Analysis/include/Luau/ConstraintSolverLogger.h +++ b/Analysis/include/Luau/ConstraintSolverLogger.h @@ -1,6 +1,8 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details -#include "Luau/ConstraintGraphBuilder.h" +#include "Luau/Constraint.h" +#include "Luau/NotNull.h" +#include "Luau/Scope.h" #include "Luau/ToString.h" #include diff --git a/Analysis/include/Luau/Error.h b/Analysis/include/Luau/Error.h index b4530674..a1323960 100644 --- a/Analysis/include/Luau/Error.h +++ b/Analysis/include/Luau/Error.h @@ -169,6 +169,13 @@ struct GenericError bool operator==(const GenericError& rhs) const; }; +struct InternalError +{ + std::string message; + + bool operator==(const InternalError& rhs) const; +}; + struct CannotCallNonFunction { TypeId ty; @@ -293,12 +300,12 @@ struct NormalizationTooComplex } }; -using TypeErrorData = - Variant; +using TypeErrorData = Variant; struct TypeError { @@ -339,7 +346,13 @@ T* get(TypeError& e) using ErrorVec = std::vector; +struct TypeErrorToStringOptions +{ + FileResolver* fileResolver = nullptr; +}; + std::string toString(const TypeError& error); +std::string toString(const TypeError& error, TypeErrorToStringOptions options); bool containsParseErrorName(const TypeError& error); @@ -356,4 +369,24 @@ struct InternalErrorReporter [[noreturn]] void ice(const std::string& message); }; +class InternalCompilerError : public std::exception { +public: + explicit InternalCompilerError(const std::string& message, const std::string& moduleName) + : message(message) + , moduleName(moduleName) + { + } + explicit InternalCompilerError(const std::string& message, const std::string& moduleName, const Location& location) + : message(message) + , moduleName(moduleName) + , location(location) + { + } + virtual const char* what() const throw(); + + const std::string message; + const std::string moduleName; + const std::optional location; +}; + } // namespace Luau diff --git a/Analysis/include/Luau/IostreamHelpers.h b/Analysis/include/Luau/IostreamHelpers.h index ee994296..05b94516 100644 --- a/Analysis/include/Luau/IostreamHelpers.h +++ b/Analysis/include/Luau/IostreamHelpers.h @@ -30,6 +30,7 @@ std::ostream& operator<<(std::ostream& lhs, const OccursCheckFailed& error); std::ostream& operator<<(std::ostream& lhs, const UnknownRequire& error); std::ostream& operator<<(std::ostream& lhs, const UnknownPropButFoundLikeProp& e); std::ostream& operator<<(std::ostream& lhs, const GenericError& error); +std::ostream& operator<<(std::ostream& lhs, const InternalError& error); std::ostream& operator<<(std::ostream& lhs, const FunctionExitsWithoutReturning& error); std::ostream& operator<<(std::ostream& lhs, const MissingProperties& error); std::ostream& operator<<(std::ostream& lhs, const IllegalRequire& error); diff --git a/Analysis/include/Luau/Module.h b/Analysis/include/Luau/Module.h index e979b3f0..39f8dfb7 100644 --- a/Analysis/include/Luau/Module.h +++ b/Analysis/include/Luau/Module.h @@ -1,10 +1,11 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #pragma once +#include "Luau/Error.h" #include "Luau/FileResolver.h" #include "Luau/ParseOptions.h" -#include "Luau/Error.h" #include "Luau/ParseResult.h" +#include "Luau/Scope.h" #include "Luau/TypeArena.h" #include @@ -19,7 +20,9 @@ struct Module; using ScopePtr = std::shared_ptr; using ModulePtr = std::shared_ptr; -struct Scope2; + +class AstType; +class AstTypePack; /// Root of the AST of a parsed source file struct SourceModule @@ -73,6 +76,8 @@ struct Module DenseHashMap astExpectedTypes{nullptr}; DenseHashMap astOriginalCallTypes{nullptr}; DenseHashMap astOverloadResolvedTypes{nullptr}; + DenseHashMap astResolvedTypes{nullptr}; + DenseHashMap astResolvedTypePacks{nullptr}; std::unordered_map declaredGlobals; ErrorVec errors; diff --git a/Analysis/include/Luau/Normalize.h b/Analysis/include/Luau/Normalize.h index d4c7698b..f5fd9886 100644 --- a/Analysis/include/Luau/Normalize.h +++ b/Analysis/include/Luau/Normalize.h @@ -9,8 +9,8 @@ namespace Luau struct InternalErrorReporter; -bool isSubtype(TypeId superTy, TypeId subTy, InternalErrorReporter& ice); -bool isSubtype(TypePackId superTy, TypePackId subTy, InternalErrorReporter& ice); +bool isSubtype(TypeId subTy, TypeId superTy, InternalErrorReporter& ice); +bool isSubtype(TypePackId subTy, TypePackId superTy, InternalErrorReporter& ice); std::pair normalize(TypeId ty, TypeArena& arena, InternalErrorReporter& ice); std::pair normalize(TypeId ty, const ModulePtr& module, InternalErrorReporter& ice); diff --git a/Analysis/include/Luau/RecursionCounter.h b/Analysis/include/Luau/RecursionCounter.h index 03ae2c83..f964dbfe 100644 --- a/Analysis/include/Luau/RecursionCounter.h +++ b/Analysis/include/Luau/RecursionCounter.h @@ -6,8 +6,6 @@ #include #include -LUAU_FASTFLAG(LuauRecursionLimitException); - namespace Luau { @@ -39,21 +37,12 @@ private: struct RecursionLimiter : RecursionCounter { - // TODO: remove ctx after LuauRecursionLimitException is removed - RecursionLimiter(int* count, int limit, const char* ctx) + RecursionLimiter(int* count, int limit) : RecursionCounter(count) { - LUAU_ASSERT(ctx); if (limit > 0 && *count > limit) { - if (FFlag::LuauRecursionLimitException) - throw RecursionLimitException(); - else - { - std::string m = "Internal recursion counter limit exceeded: "; - m += ctx; - throw std::runtime_error(m); - } + throw RecursionLimitException(); } } }; diff --git a/Analysis/include/Luau/Scope.h b/Analysis/include/Luau/Scope.h index 45338409..cef4b94f 100644 --- a/Analysis/include/Luau/Scope.h +++ b/Analysis/include/Luau/Scope.h @@ -1,6 +1,7 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #pragma once +#include "Luau/Constraint.h" #include "Luau/Location.h" #include "Luau/TypeVar.h" @@ -64,4 +65,21 @@ struct Scope std::unordered_map typeAliasTypePackParameters; }; +struct Scope2 +{ + // The parent scope of this scope. Null if there is no parent (i.e. this + // is the module-level scope). + Scope2* parent = nullptr; + // All the children of this scope. + std::vector children; + std::unordered_map bindings; // TODO: I think this can be a DenseHashMap + std::unordered_map typeBindings; + TypePackId returnType; + // All constraints belonging to this scope. + std::vector constraints; + + std::optional lookup(Symbol sym); + std::optional lookupTypeBinding(const Name& name); +}; + } // namespace Luau diff --git a/Analysis/include/Luau/TypeVar.h b/Analysis/include/Luau/TypeVar.h index ff7708d4..20f4107c 100644 --- a/Analysis/include/Luau/TypeVar.h +++ b/Analysis/include/Luau/TypeVar.h @@ -287,7 +287,6 @@ struct FunctionTypeVar bool hasSelf; Tags tags; bool hasNoGenerics = false; - bool generalized = false; }; enum class TableState diff --git a/Analysis/include/Luau/Unifiable.h b/Analysis/include/Luau/Unifiable.h index fdc39481..4ff91714 100644 --- a/Analysis/include/Luau/Unifiable.h +++ b/Analysis/include/Luau/Unifiable.h @@ -117,6 +117,7 @@ struct Generic explicit Generic(const Name& name); explicit Generic(Scope2* scope); Generic(TypeLevel level, const Name& name); + Generic(Scope2* scope, const Name& name); int index; TypeLevel level; diff --git a/Analysis/include/Luau/Unifier.h b/Analysis/include/Luau/Unifier.h index b51a485e..4af324cb 100644 --- a/Analysis/include/Luau/Unifier.h +++ b/Analysis/include/Luau/Unifier.h @@ -79,12 +79,8 @@ private: void tryUnifySingletons(TypeId subTy, TypeId superTy); void tryUnifyFunctions(TypeId subTy, TypeId superTy, bool isFunctionCall = false); void tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection = false); - void DEPRECATED_tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection = false); - void tryUnifyFreeTable(TypeId subTy, TypeId superTy); - void tryUnifySealedTables(TypeId subTy, TypeId superTy, bool isIntersection); void tryUnifyWithMetatable(TypeId subTy, TypeId superTy, bool reversed); void tryUnifyWithClass(TypeId subTy, TypeId superTy, bool reversed); - void tryUnifyIndexer(const TableIndexer& subIndexer, const TableIndexer& superIndexer); TypeId widen(TypeId ty); TypePackId widen(TypePackId tp); diff --git a/Analysis/include/Luau/VisitTypeVar.h b/Analysis/include/Luau/VisitTypeVar.h index 642522c9..5fd43f0b 100644 --- a/Analysis/include/Luau/VisitTypeVar.h +++ b/Analysis/include/Luau/VisitTypeVar.h @@ -169,7 +169,7 @@ struct GenericTypeVarVisitor void traverse(TypeId ty) { - RecursionLimiter limiter{&recursionCounter, FInt::LuauVisitRecursionLimit, "TypeVarVisitor"}; + RecursionLimiter limiter{&recursionCounter, FInt::LuauVisitRecursionLimit}; if (visit_detail::hasSeen(seen, ty)) { diff --git a/Analysis/src/Clone.cpp b/Analysis/src/Clone.cpp index 248262ce..df4e0a6b 100644 --- a/Analysis/src/Clone.cpp +++ b/Analysis/src/Clone.cpp @@ -317,7 +317,7 @@ TypePackId clone(TypePackId tp, TypeArena& dest, CloneState& cloneState) if (tp->persistent) return tp; - RecursionLimiter _ra(&cloneState.recursionCount, FInt::LuauTypeCloneRecursionLimit, "cloning TypePackId"); + RecursionLimiter _ra(&cloneState.recursionCount, FInt::LuauTypeCloneRecursionLimit); TypePackId& res = cloneState.seenTypePacks[tp]; @@ -335,7 +335,7 @@ TypeId clone(TypeId typeId, TypeArena& dest, CloneState& cloneState) if (typeId->persistent) return typeId; - RecursionLimiter _ra(&cloneState.recursionCount, FInt::LuauTypeCloneRecursionLimit, "cloning TypeId"); + RecursionLimiter _ra(&cloneState.recursionCount, FInt::LuauTypeCloneRecursionLimit); TypeId& res = cloneState.seenTypes[typeId]; diff --git a/Analysis/src/Constraint.cpp b/Analysis/src/Constraint.cpp index 6cb0e4ee..64e3a666 100644 --- a/Analysis/src/Constraint.cpp +++ b/Analysis/src/Constraint.cpp @@ -5,9 +5,8 @@ namespace Luau { -Constraint::Constraint(ConstraintV&& c, Location location) +Constraint::Constraint(ConstraintV&& c) : c(std::move(c)) - , location(location) { } diff --git a/Analysis/src/ConstraintGraphBuilder.cpp b/Analysis/src/ConstraintGraphBuilder.cpp index fa627e7a..d9e8d238 100644 --- a/Analysis/src/ConstraintGraphBuilder.cpp +++ b/Analysis/src/ConstraintGraphBuilder.cpp @@ -2,28 +2,13 @@ #include "Luau/ConstraintGraphBuilder.h" +#include "Luau/Scope.h" + namespace Luau { const AstStat* getFallthrough(const AstStat* node); // TypeInfer.cpp -std::optional Scope2::lookup(Symbol sym) -{ - Scope2* s = this; - - while (true) - { - auto it = s->bindings.find(sym); - if (it != s->bindings.end()) - return it->second; - - if (s->parent) - s = s->parent; - else - return std::nullopt; - } -} - ConstraintGraphBuilder::ConstraintGraphBuilder(TypeArena* arena) : singletonTypes(getSingletonTypes()) , arena(arena) @@ -59,10 +44,10 @@ Scope2* ConstraintGraphBuilder::childScope(Location location, Scope2* parent) return borrow; } -void ConstraintGraphBuilder::addConstraint(Scope2* scope, ConstraintV cv, Location location) +void ConstraintGraphBuilder::addConstraint(Scope2* scope, ConstraintV cv) { LUAU_ASSERT(scope); - scope->constraints.emplace_back(new Constraint{std::move(cv), location}); + scope->constraints.emplace_back(new Constraint{std::move(cv)}); } void ConstraintGraphBuilder::addConstraint(Scope2* scope, std::unique_ptr c) @@ -79,6 +64,13 @@ void ConstraintGraphBuilder::visit(AstStatBlock* block) rootScope = scopes.back().second.get(); rootScope->returnType = freshTypePack(rootScope); + // TODO: We should share the global scope. + rootScope->typeBindings["nil"] = singletonTypes.nilType; + rootScope->typeBindings["number"] = singletonTypes.numberType; + rootScope->typeBindings["string"] = singletonTypes.stringType; + rootScope->typeBindings["boolean"] = singletonTypes.booleanType; + rootScope->typeBindings["thread"] = singletonTypes.threadType; + visit(rootScope, block); } @@ -102,6 +94,8 @@ void ConstraintGraphBuilder::visit(Scope2* scope, AstStat* stat) checkPack(scope, e->expr); else if (auto i = stat->as()) visit(scope, i); + else if (auto a = stat->as()) + visit(scope, a); else LUAU_ASSERT(0); } @@ -114,8 +108,14 @@ void ConstraintGraphBuilder::visit(Scope2* scope, AstStatLocal* local) for (AstLocal* local : local->vars) { - // TODO annotations TypeId ty = freshType(scope); + + if (local->annotation) + { + TypeId annotation = resolveType(scope, local->annotation); + addConstraint(scope, SubtypeConstraint{ty, annotation}); + } + varTypes.push_back(ty); scope->bindings[local] = ty; } @@ -136,14 +136,14 @@ void ConstraintGraphBuilder::visit(Scope2* scope, AstStatLocal* local) { std::vector tailValues{varTypes.begin() + i, varTypes.end()}; TypePackId tailPack = arena->addTypePack(std::move(tailValues)); - addConstraint(scope, PackSubtypeConstraint{exprPack, tailPack}, local->location); + addConstraint(scope, PackSubtypeConstraint{exprPack, tailPack}); } } else { TypeId exprType = check(scope, local->values.data[i]); if (i < varTypes.size()) - addConstraint(scope, SubtypeConstraint{varTypes[i], exprType}, local->vars.data[i]->location); + addConstraint(scope, SubtypeConstraint{varTypes[i], exprType}); } } } @@ -188,7 +188,7 @@ void ConstraintGraphBuilder::visit(Scope2* scope, AstStatLocalFunction* function checkFunctionBody(innerScope, function->func); - std::unique_ptr c{new Constraint{GeneralizationConstraint{functionType, actualFunctionType, innerScope}, function->location}}; + std::unique_ptr c{new Constraint{GeneralizationConstraint{functionType, actualFunctionType, innerScope}}}; addConstraints(c.get(), innerScope); addConstraint(scope, std::move(c)); @@ -240,7 +240,7 @@ void ConstraintGraphBuilder::visit(Scope2* scope, AstStatFunction* function) checkFunctionBody(innerScope, function->func); - std::unique_ptr c{new Constraint{GeneralizationConstraint{functionType, actualFunctionType, innerScope}, function->location}}; + std::unique_ptr c{new Constraint{GeneralizationConstraint{functionType, actualFunctionType, innerScope}}}; addConstraints(c.get(), innerScope); addConstraint(scope, std::move(c)); @@ -251,13 +251,26 @@ void ConstraintGraphBuilder::visit(Scope2* scope, AstStatReturn* ret) LUAU_ASSERT(scope); TypePackId exprTypes = checkPack(scope, ret->list); - addConstraint(scope, PackSubtypeConstraint{exprTypes, scope->returnType}, ret->location); + addConstraint(scope, PackSubtypeConstraint{exprTypes, scope->returnType}); } void ConstraintGraphBuilder::visit(Scope2* scope, AstStatBlock* block) { LUAU_ASSERT(scope); + // In order to enable mutually-recursive type aliases, we need to + // populate the type bindings before we actually check any of the + // alias statements. Since we're not ready to actually resolve + // any of the annotations, we just use a fresh type for now. + for (AstStat* stat : block->body) + { + if (auto alias = stat->as()) + { + TypeId initialType = freshType(scope); + scope->typeBindings[alias->name.value] = initialType; + } + } + for (AstStat* stat : block->body) visit(scope, stat); } @@ -267,7 +280,7 @@ void ConstraintGraphBuilder::visit(Scope2* scope, AstStatAssign* assign) TypePackId varPackId = checkExprList(scope, assign->vars); TypePackId valuePack = checkPack(scope, assign->values); - addConstraint(scope, PackSubtypeConstraint{valuePack, varPackId}, assign->location); + addConstraint(scope, PackSubtypeConstraint{valuePack, varPackId}); } void ConstraintGraphBuilder::visit(Scope2* scope, AstStatIf* ifStatement) @@ -284,6 +297,28 @@ void ConstraintGraphBuilder::visit(Scope2* scope, AstStatIf* ifStatement) } } +void ConstraintGraphBuilder::visit(Scope2* scope, AstStatTypeAlias* alias) +{ + // TODO: Exported type aliases + // TODO: Generic type aliases + + auto it = scope->typeBindings.find(alias->name.value); + // This should always be here since we do a separate pass over the + // AST to set up typeBindings. If it's not, we've somehow skipped + // this alias in that first pass. + LUAU_ASSERT(it != scope->typeBindings.end()); + + TypeId ty = resolveType(scope, alias->type); + + // Rather than using a subtype constraint, we instead directly bind + // the free type we generated in the first pass to the resolved type. + // This prevents a case where you could cause another constraint to + // bind the free alias type to an unrelated type, causing havoc. + asMutable(it->second)->ty.emplace(ty); + + addConstraint(scope, NameConstraint{ty, alias->name.value}); +} + TypePackId ConstraintGraphBuilder::checkPack(Scope2* scope, AstArray exprs) { LUAU_ASSERT(scope); @@ -350,13 +385,13 @@ TypePackId ConstraintGraphBuilder::checkPack(Scope2* scope, AstExpr* expr) astOriginalCallTypes[call->func] = fnType; TypeId instantiatedType = freshType(scope); - addConstraint(scope, InstantiationConstraint{instantiatedType, fnType}, expr->location); + addConstraint(scope, InstantiationConstraint{instantiatedType, fnType}); TypePackId rets = freshTypePack(scope); FunctionTypeVar ftv(arena->addTypePack(TypePack{args, {}}), rets); TypeId inferredFnType = arena->addType(ftv); - addConstraint(scope, SubtypeConstraint{inferredFnType, instantiatedType}, expr->location); + addConstraint(scope, SubtypeConstraint{inferredFnType, instantiatedType}); result = rets; } else @@ -413,7 +448,7 @@ TypeId ConstraintGraphBuilder::check(Scope2* scope, AstExpr* expr) TypePack onePack{{typeResult}, freshTypePack(scope)}; TypePackId oneTypePack = arena->addTypePack(std::move(onePack)); - addConstraint(scope, PackSubtypeConstraint{packResult, oneTypePack}, expr->location); + addConstraint(scope, PackSubtypeConstraint{packResult, oneTypePack}); return typeResult; } @@ -454,7 +489,7 @@ TypeId ConstraintGraphBuilder::check(Scope2* scope, AstExprIndexName* indexName) TypeId expectedTableType = arena->addType(std::move(ttv)); - addConstraint(scope, SubtypeConstraint{obj, expectedTableType}, indexName->location); + addConstraint(scope, SubtypeConstraint{obj, expectedTableType}); return result; } @@ -465,8 +500,7 @@ TypeId ConstraintGraphBuilder::checkExprTable(Scope2* scope, AstExprTable* expr) TableTypeVar* ttv = getMutable(ty); LUAU_ASSERT(ttv); - auto createIndexer = [this, scope, ttv]( - TypeId currentIndexType, TypeId currentResultType, Location itemLocation, std::optional keyLocation) { + auto createIndexer = [this, scope, ttv](TypeId currentIndexType, TypeId currentResultType) { if (!ttv->indexer) { TypeId indexType = this->freshType(scope); @@ -474,8 +508,8 @@ TypeId ConstraintGraphBuilder::checkExprTable(Scope2* scope, AstExprTable* expr) ttv->indexer = TableIndexer{indexType, resultType}; } - addConstraint(scope, SubtypeConstraint{ttv->indexer->indexType, currentIndexType}, keyLocation ? *keyLocation : itemLocation); - addConstraint(scope, SubtypeConstraint{ttv->indexer->indexResultType, currentResultType}, itemLocation); + addConstraint(scope, SubtypeConstraint{ttv->indexer->indexType, currentIndexType}); + addConstraint(scope, SubtypeConstraint{ttv->indexer->indexResultType, currentResultType}); }; for (const AstExprTable::Item& item : expr->items) @@ -495,13 +529,13 @@ TypeId ConstraintGraphBuilder::checkExprTable(Scope2* scope, AstExprTable* expr) } else { - createIndexer(keyTy, itemTy, item.value->location, item.key->location); + createIndexer(keyTy, itemTy); } } else { TypeId numberType = singletonTypes.numberType; - createIndexer(numberType, itemTy, item.value->location, std::nullopt); + createIndexer(numberType, itemTy); } } @@ -514,15 +548,29 @@ std::pair ConstraintGraphBuilder::checkFunctionSignature(Scope2 TypePackId returnType = freshTypePack(innerScope); innerScope->returnType = returnType; + if (fn->returnAnnotation) + { + TypePackId annotatedRetType = resolveTypePack(innerScope, *fn->returnAnnotation); + addConstraint(innerScope, PackSubtypeConstraint{returnType, annotatedRetType}); + } + std::vector argTypes; for (AstLocal* local : fn->args) { TypeId t = freshType(innerScope); argTypes.push_back(t); - innerScope->bindings[local] = t; // TODO annotations + innerScope->bindings[local] = t; + + if (local->annotation) + { + TypeId argAnnotation = resolveType(innerScope, local->annotation); + addConstraint(innerScope, SubtypeConstraint{t, argAnnotation}); + } } + // TODO: Vararg annotation. + FunctionTypeVar actualFunction{arena->addTypePack(argTypes), returnType}; TypeId actualFunctionType = arena->addType(std::move(actualFunction)); LUAU_ASSERT(actualFunctionType); @@ -541,10 +589,171 @@ void ConstraintGraphBuilder::checkFunctionBody(Scope2* scope, AstExprFunction* f if (nullptr != getFallthrough(fn->body)) { TypePackId empty = arena->addTypePack({}); // TODO we could have CSG retain one of these forever - addConstraint(scope, PackSubtypeConstraint{scope->returnType, empty}, fn->body->location); + addConstraint(scope, PackSubtypeConstraint{scope->returnType, empty}); } } +TypeId ConstraintGraphBuilder::resolveType(Scope2* scope, AstType* ty) +{ + TypeId result = nullptr; + + if (auto ref = ty->as()) + { + // TODO: Support imported types w/ require tracing. + // TODO: Support generic type references. + LUAU_ASSERT(!ref->prefix); + LUAU_ASSERT(!ref->hasParameterList); + + // TODO: If it doesn't exist, should we introduce a free binding? + // This is probably important for handling type aliases. + result = scope->lookupTypeBinding(ref->name.value).value_or(singletonTypes.errorRecoveryType()); + } + else if (auto tab = ty->as()) + { + TableTypeVar::Props props; + std::optional indexer; + + for (const AstTableProp& prop : tab->props) + { + std::string name = prop.name.value; + // TODO: Recursion limit. + TypeId propTy = resolveType(scope, prop.type); + // TODO: Fill in location. + props[name] = {propTy}; + } + + if (tab->indexer) + { + // TODO: Recursion limit. + indexer = TableIndexer{ + resolveType(scope, tab->indexer->indexType), + resolveType(scope, tab->indexer->resultType), + }; + } + + // TODO: Remove TypeLevel{} here, we don't need it. + result = arena->addType(TableTypeVar{props, indexer, TypeLevel{}, TableState::Sealed}); + } + else if (auto fn = ty->as()) + { + // TODO: Generic functions. + // TODO: Scope (though it may not be needed). + // TODO: Recursion limit. + TypePackId argTypes = resolveTypePack(scope, fn->argTypes); + TypePackId returnTypes = resolveTypePack(scope, fn->returnTypes); + + // TODO: Is this the right constructor to use? + result = arena->addType(FunctionTypeVar{argTypes, returnTypes}); + + FunctionTypeVar* ftv = getMutable(result); + ftv->argNames.reserve(fn->argNames.size); + for (const auto& el : fn->argNames) + { + if (el) + { + const auto& [name, location] = *el; + ftv->argNames.push_back(FunctionArgument{name.value, location}); + } + else + { + ftv->argNames.push_back(std::nullopt); + } + } + } + else if (auto tof = ty->as()) + { + // TODO: Recursion limit. + TypeId exprType = check(scope, tof->expr); + result = exprType; + } + else if (auto unionAnnotation = ty->as()) + { + std::vector parts; + for (AstType* part : unionAnnotation->types) + { + // TODO: Recursion limit. + parts.push_back(resolveType(scope, part)); + } + + result = arena->addType(UnionTypeVar{parts}); + } + else if (auto intersectionAnnotation = ty->as()) + { + std::vector parts; + for (AstType* part : intersectionAnnotation->types) + { + // TODO: Recursion limit. + parts.push_back(resolveType(scope, part)); + } + + result = arena->addType(IntersectionTypeVar{parts}); + } + else if (auto boolAnnotation = ty->as()) + { + result = arena->addType(SingletonTypeVar(BooleanSingleton{boolAnnotation->value})); + } + else if (auto stringAnnotation = ty->as()) + { + result = arena->addType(SingletonTypeVar(StringSingleton{std::string(stringAnnotation->value.data, stringAnnotation->value.size)})); + } + else if (ty->is()) + { + result = singletonTypes.errorRecoveryType(); + } + else + { + LUAU_ASSERT(0); + result = singletonTypes.errorRecoveryType(); + } + + astResolvedTypes[ty] = result; + return result; +} + +TypePackId ConstraintGraphBuilder::resolveTypePack(Scope2* scope, AstTypePack* tp) +{ + TypePackId result; + if (auto expl = tp->as()) + { + result = resolveTypePack(scope, expl->typeList); + } + else if (auto var = tp->as()) + { + TypeId ty = resolveType(scope, var->variadicType); + result = arena->addTypePack(TypePackVar{VariadicTypePack{ty}}); + } + else if (auto gen = tp->as()) + { + result = arena->addTypePack(TypePackVar{GenericTypePack{scope, gen->genericName.value}}); + } + else + { + LUAU_ASSERT(0); + result = singletonTypes.errorRecoveryTypePack(); + } + + astResolvedTypePacks[tp] = result; + return result; +} + +TypePackId ConstraintGraphBuilder::resolveTypePack(Scope2* scope, const AstTypeList& list) +{ + std::vector head; + + for (AstType* headTy : list.types) + { + head.push_back(resolveType(scope, headTy)); + } + + std::optional tail = std::nullopt; + if (list.tailType) + { + tail = resolveTypePack(scope, list.tailType); + } + + return arena->addTypePack(TypePack{head, tail}); +} + void collectConstraints(std::vector>& result, Scope2* scope) { for (const auto& c : scope->constraints) diff --git a/Analysis/src/ConstraintSolver.cpp b/Analysis/src/ConstraintSolver.cpp index 41dfd892..9e355236 100644 --- a/Analysis/src/ConstraintSolver.cpp +++ b/Analysis/src/ConstraintSolver.cpp @@ -2,6 +2,7 @@ #include "Luau/ConstraintSolver.h" #include "Luau/Instantiation.h" +#include "Luau/Location.h" #include "Luau/Quantify.h" #include "Luau/ToString.h" #include "Luau/Unifier.h" @@ -179,6 +180,8 @@ bool ConstraintSolver::tryDispatch(NotNull constraint, bool fo success = tryDispatch(*gc, constraint, force); else if (auto ic = get(*constraint)) success = tryDispatch(*ic, constraint, force); + else if (auto nc = get(*constraint)) + success = tryDispatch(*nc, constraint); else LUAU_ASSERT(0); @@ -197,7 +200,7 @@ bool ConstraintSolver::tryDispatch(const SubtypeConstraint& c, NotNulllocation); + unify(c.subType, c.superType); unblock(c.subType); unblock(c.superType); @@ -207,7 +210,7 @@ bool ConstraintSolver::tryDispatch(const SubtypeConstraint& c, NotNull constraint, bool force) { - unify(c.subPack, c.superPack, constraint->location); + unify(c.subPack, c.superPack); unblock(c.subPack); unblock(c.superPack); @@ -222,7 +225,7 @@ bool ConstraintSolver::tryDispatch(const GeneralizationConstraint& c, NotNullty.emplace(c.sourceType); else - unify(c.generalizedType, c.sourceType, constraint->location); + unify(c.generalizedType, c.sourceType); TypeId generalized = quantify(arena, c.sourceType, c.scope); *asMutable(c.sourceType) = *generalized; @@ -243,12 +246,28 @@ bool ConstraintSolver::tryDispatch(const InstantiationConstraint& c, NotNull instantiated = inst.substitute(c.superType); LUAU_ASSERT(instantiated); // TODO FIXME HANDLE THIS - unify(c.subType, *instantiated, constraint->location); + unify(c.subType, *instantiated); unblock(c.subType); return true; } +bool ConstraintSolver::tryDispatch(const NameConstraint& c, NotNull constraint) +{ + if (isBlocked(c.namedType)) + return block(c.namedType, constraint); + + TypeId target = follow(c.namedType); + if (TableTypeVar* ttv = getMutable(target)) + ttv->name = c.name; + else if (MetatableTypeVar* mtv = getMutable(target)) + mtv->syntheticName = c.name; + else + return block(c.namedType, constraint); + + return true; +} + void ConstraintSolver::block_(BlockedConstraintId target, NotNull constraint) { blocked[target].push_back(constraint); @@ -321,19 +340,19 @@ bool ConstraintSolver::isBlocked(NotNull constraint) return blockedIt != blockedConstraints.end() && blockedIt->second > 0; } -void ConstraintSolver::unify(TypeId subType, TypeId superType, Location location) +void ConstraintSolver::unify(TypeId subType, TypeId superType) { UnifierSharedState sharedState{&iceReporter}; - Unifier u{arena, Mode::Strict, location, Covariant, sharedState}; + Unifier u{arena, Mode::Strict, Location{}, Covariant, sharedState}; u.tryUnify(subType, superType); u.log.commit(); } -void ConstraintSolver::unify(TypePackId subPack, TypePackId superPack, Location location) +void ConstraintSolver::unify(TypePackId subPack, TypePackId superPack) { UnifierSharedState sharedState{&iceReporter}; - Unifier u{arena, Mode::Strict, location, Covariant, sharedState}; + Unifier u{arena, Mode::Strict, Location{}, Covariant, sharedState}; u.tryUnify(subPack, superPack); u.log.commit(); diff --git a/Analysis/src/Error.cpp b/Analysis/src/Error.cpp index f443a3cc..93cb65b9 100644 --- a/Analysis/src/Error.cpp +++ b/Analysis/src/Error.cpp @@ -7,6 +7,9 @@ #include +LUAU_FASTFLAGVARIABLE(LuauTypeMismatchModuleNameResolution, false) +LUAU_FASTFLAGVARIABLE(LuauUseInternalCompilerErrorException, false) + static std::string wrongNumberOfArgsString(size_t expectedCount, size_t actualCount, const char* argPrefix = nullptr, bool isVariadic = false) { std::string s = "expects "; @@ -49,6 +52,8 @@ namespace Luau struct ErrorConverter { + FileResolver* fileResolver = nullptr; + std::string operator()(const Luau::TypeMismatch& tm) const { std::string givenTypeName = Luau::toString(tm.givenType); @@ -62,8 +67,18 @@ struct ErrorConverter { if (auto wantedDefinitionModule = getDefinitionModuleName(tm.wantedType)) { - result = "Type '" + givenTypeName + "' from '" + *givenDefinitionModule + "' could not be converted into '" + wantedTypeName + - "' from '" + *wantedDefinitionModule + "'"; + if (FFlag::LuauTypeMismatchModuleNameResolution && fileResolver != nullptr) + { + std::string givenModuleName = fileResolver->getHumanReadableModuleName(*givenDefinitionModule); + std::string wantedModuleName = fileResolver->getHumanReadableModuleName(*wantedDefinitionModule); + result = "Type '" + givenTypeName + "' from '" + givenModuleName + "' could not be converted into '" + wantedTypeName + + "' from '" + wantedModuleName + "'"; + } + else + { + result = "Type '" + givenTypeName + "' from '" + *givenDefinitionModule + "' could not be converted into '" + wantedTypeName + + "' from '" + *wantedDefinitionModule + "'"; + } } } } @@ -78,7 +93,14 @@ struct ErrorConverter if (!tm.reason.empty()) result += tm.reason + " "; - result += Luau::toString(*tm.error); + if (FFlag::LuauTypeMismatchModuleNameResolution) + { + result += Luau::toString(*tm.error, TypeErrorToStringOptions{fileResolver}); + } + else + { + result += Luau::toString(*tm.error); + } } else if (!tm.reason.empty()) { @@ -280,6 +302,11 @@ struct ErrorConverter return e.message; } + std::string operator()(const Luau::InternalError& e) const + { + return e.message; + } + std::string operator()(const Luau::CannotCallNonFunction& e) const { return "Cannot call non-function " + toString(e.ty); @@ -598,6 +625,11 @@ bool GenericError::operator==(const GenericError& rhs) const return message == rhs.message; } +bool InternalError::operator==(const InternalError& rhs) const +{ + return message == rhs.message; +} + bool CannotCallNonFunction::operator==(const CannotCallNonFunction& rhs) const { return ty == rhs.ty; @@ -685,7 +717,12 @@ bool TypesAreUnrelated::operator==(const TypesAreUnrelated& rhs) const std::string toString(const TypeError& error) { - ErrorConverter converter; + return toString(error, TypeErrorToStringOptions{}); +} + +std::string toString(const TypeError& error, TypeErrorToStringOptions options) +{ + ErrorConverter converter{options.fileResolver}; return Luau::visit(converter, error.data); } @@ -773,6 +810,9 @@ void copyError(T& e, TypeArena& destArena, CloneState cloneState) else if constexpr (std::is_same_v) { } + else if constexpr (std::is_same_v) + { + } else if constexpr (std::is_same_v) { e.ty = clone(e.ty); @@ -847,22 +887,51 @@ void copyErrors(ErrorVec& errors, TypeArena& destArena) void InternalErrorReporter::ice(const std::string& message, const Location& location) { - std::runtime_error error("Internal error in " + moduleName + " at " + toString(location) + ": " + message); + if (FFlag::LuauUseInternalCompilerErrorException) + { + InternalCompilerError error(message, moduleName, location); - if (onInternalError) - onInternalError(error.what()); + if (onInternalError) + onInternalError(error.what()); - throw error; + throw error; + } + else + { + std::runtime_error error("Internal error in " + moduleName + " at " + toString(location) + ": " + message); + + if (onInternalError) + onInternalError(error.what()); + + throw error; + } } void InternalErrorReporter::ice(const std::string& message) { - std::runtime_error error("Internal error in " + moduleName + ": " + message); + if (FFlag::LuauUseInternalCompilerErrorException) + { + InternalCompilerError error(message, moduleName); - if (onInternalError) - onInternalError(error.what()); + if (onInternalError) + onInternalError(error.what()); - throw error; + throw error; + } + else + { + std::runtime_error error("Internal error in " + moduleName + ": " + message); + + if (onInternalError) + onInternalError(error.what()); + + throw error; + } +} + +const char* InternalCompilerError::what() const throw() +{ + return this->message.data(); } } // namespace Luau diff --git a/Analysis/src/Frontend.cpp b/Analysis/src/Frontend.cpp index 9e025062..85c5dbc8 100644 --- a/Analysis/src/Frontend.cpp +++ b/Analysis/src/Frontend.cpp @@ -801,6 +801,8 @@ ModulePtr Frontend::check(const SourceModule& sourceModule, Mode mode, const Sco result->astTypes = std::move(cgb.astTypes); result->astTypePacks = std::move(cgb.astTypePacks); result->astOriginalCallTypes = std::move(cgb.astOriginalCallTypes); + result->astResolvedTypes = std::move(cgb.astResolvedTypes); + result->astResolvedTypePacks = std::move(cgb.astResolvedTypePacks); result->clonePublicInterface(iceHandler); diff --git a/Analysis/src/IostreamHelpers.cpp b/Analysis/src/IostreamHelpers.cpp index 048167ae..e4fac455 100644 --- a/Analysis/src/IostreamHelpers.cpp +++ b/Analysis/src/IostreamHelpers.cpp @@ -111,6 +111,8 @@ static void errorToString(std::ostream& stream, const T& err) } else if constexpr (std::is_same_v) stream << "GenericError { " << err.message << " }"; + else if constexpr (std::is_same_v) + stream << "InternalError { " << err.message << " }"; else if constexpr (std::is_same_v) stream << "CannotCallNonFunction { " << toString(err.ty) << " }"; else if constexpr (std::is_same_v) diff --git a/Analysis/src/Module.cpp b/Analysis/src/Module.cpp index 4d157e6f..95eb125e 100644 --- a/Analysis/src/Module.cpp +++ b/Analysis/src/Module.cpp @@ -11,7 +11,6 @@ #include "Luau/TypePack.h" #include "Luau/TypeVar.h" #include "Luau/VisitTypeVar.h" -#include "Luau/ConstraintGraphBuilder.h" // FIXME: For Scope2 TODO pull out into its own header #include diff --git a/Analysis/src/Quantify.cpp b/Analysis/src/Quantify.cpp index 2004d153..40e14c68 100644 --- a/Analysis/src/Quantify.cpp +++ b/Analysis/src/Quantify.cpp @@ -2,11 +2,10 @@ #include "Luau/Quantify.h" -#include "Luau/ConstraintGraphBuilder.h" // TODO for Scope2; move to separate header -#include "Luau/TxnLog.h" +#include "Luau/Scope.h" #include "Luau/Substitution.h" +#include "Luau/TxnLog.h" #include "Luau/VisitTypeVar.h" -#include "Luau/ConstraintGraphBuilder.h" // TODO for Scope2; move to separate header LUAU_FASTFLAG(LuauAlwaysQuantify); LUAU_FASTFLAG(DebugLuauDeferredConstraintResolution); @@ -177,8 +176,6 @@ void quantify(TypeId ty, TypeLevel level) if (ftv->generics.empty() && ftv->genericPacks.empty() && !q.seenMutableType && !q.seenGenericType) ftv->hasNoGenerics = true; - - ftv->generalized = true; } void quantify(TypeId ty, Scope2* scope) @@ -201,8 +198,6 @@ void quantify(TypeId ty, Scope2* scope) if (ftv->generics.empty() && ftv->genericPacks.empty() && !q.seenMutableType && !q.seenGenericType) ftv->hasNoGenerics = true; - - ftv->generalized = true; } struct PureQuantifier : Substitution diff --git a/Analysis/src/Scope.cpp b/Analysis/src/Scope.cpp index 011e28d4..66aaee1f 100644 --- a/Analysis/src/Scope.cpp +++ b/Analysis/src/Scope.cpp @@ -121,4 +121,36 @@ std::optional Scope::linearSearchForBinding(const std::string& name, bo return std::nullopt; } +std::optional Scope2::lookup(Symbol sym) +{ + Scope2* s = this; + + while (true) + { + auto it = s->bindings.find(sym); + if (it != s->bindings.end()) + return it->second; + + if (s->parent) + s = s->parent; + else + return std::nullopt; + } +} + +std::optional Scope2::lookupTypeBinding(const Name& name) +{ + Scope2* s = this; + while (s) + { + auto it = s->typeBindings.find(name); + if (it != s->typeBindings.end()) + return it->second; + + s = s->parent; + } + + return std::nullopt; +} + } // namespace Luau diff --git a/Analysis/src/ToString.cpp b/Analysis/src/ToString.cpp index 81dc0467..7a458964 100644 --- a/Analysis/src/ToString.cpp +++ b/Analysis/src/ToString.cpp @@ -1411,6 +1411,12 @@ std::string toString(const Constraint& c, ToStringOptions& opts) opts.nameMap = std::move(superStr.nameMap); return subStr.name + " ~ inst " + superStr.name; } + else if (const NameConstraint* nc = Luau::get(c)) + { + ToStringResult namedStr = toStringDetailed(nc->namedType, opts); + opts.nameMap = std::move(namedStr.nameMap); + return "@name(" + namedStr.name + ") = " + nc->name; + } else { LUAU_ASSERT(false); diff --git a/Analysis/src/TypeChecker2.cpp b/Analysis/src/TypeChecker2.cpp index 7f5ba683..63e5800f 100644 --- a/Analysis/src/TypeChecker2.cpp +++ b/Analysis/src/TypeChecker2.cpp @@ -7,6 +7,9 @@ #include "Luau/AstQuery.h" #include "Luau/Clone.h" #include "Luau/Normalize.h" +#include "Luau/ConstraintGraphBuilder.h" // FIXME move Scope2 into its own header +#include "Luau/Unifier.h" +#include "Luau/ToString.h" namespace Luau { @@ -39,6 +42,104 @@ struct TypeChecker2 : public AstVisitor return follow(*ty); } + TypeId lookupAnnotation(AstType* annotation) + { + TypeId* ty = module->astResolvedTypes.find(annotation); + LUAU_ASSERT(ty); + return follow(*ty); + } + + TypePackId reconstructPack(AstArray exprs, TypeArena& arena) + { + std::vector head; + + for (size_t i = 0; i < exprs.size - 1; ++i) + { + head.push_back(lookupType(exprs.data[i])); + } + + TypePackId tail = lookupPack(exprs.data[exprs.size - 1]); + return arena.addTypePack(TypePack{head, tail}); + } + + Scope2* findInnermostScope(Location location) + { + Scope2* bestScope = module->getModuleScope2(); + Location bestLocation = module->scope2s[0].first; + + for (size_t i = 0; i < module->scope2s.size(); ++i) + { + auto& [scopeBounds, scope] = module->scope2s[i]; + if (scopeBounds.encloses(location)) + { + if (scopeBounds.begin > bestLocation.begin || scopeBounds.end < bestLocation.end) + { + bestScope = scope.get(); + bestLocation = scopeBounds; + } + } + else + { + // TODO: Is this sound? This relies on the fact that scopes are inserted + // into the scope list in the order that they appear in the AST. + break; + } + } + + return bestScope; + } + + bool visit(AstStatLocal* local) override + { + for (size_t i = 0; i < local->values.size; ++i) + { + AstExpr* value = local->values.data[i]; + if (i == local->values.size - 1) + { + if (i < local->values.size) + { + TypePackId valueTypes = lookupPack(value); + auto it = begin(valueTypes); + for (size_t j = i; j < local->vars.size; ++j) + { + if (it == end(valueTypes)) + { + break; + } + + AstLocal* var = local->vars.data[i]; + if (var->annotation) + { + TypeId varType = lookupAnnotation(var->annotation); + if (!isSubtype(*it, varType, ice)) + { + reportError(TypeMismatch{varType, *it}, value->location); + } + } + + ++it; + } + } + } + else + { + TypeId valueType = lookupType(value); + AstLocal* var = local->vars.data[i]; + + if (var->annotation) + { + TypeId varType = lookupAnnotation(var->annotation); + if (!isSubtype(varType, valueType, ice)) + { + reportError(TypeMismatch{varType, valueType}, value->location); + } + } + } + } + + return true; + } + bool visit(AstStatAssign* assign) override { size_t count = std::min(assign->vars.size, assign->values.size); @@ -62,6 +163,30 @@ struct TypeChecker2 : public AstVisitor return true; } + bool visit(AstStatReturn* ret) override + { + Scope2* scope = findInnermostScope(ret->location); + TypePackId expectedRetType = scope->returnType; + + TypeArena arena; + TypePackId actualRetType = reconstructPack(ret->list, arena); + + UnifierSharedState sharedState{&ice}; + Unifier u{&arena, Mode::Strict, ret->location, Covariant, sharedState}; + u.anyIsTop = true; + + u.tryUnify(actualRetType, expectedRetType); + const bool ok = u.errors.empty() && u.log.empty(); + + if (!ok) + { + for (const TypeError& e : u.errors) + module->errors.push_back(e); + } + + return true; + } + bool visit(AstExprCall* call) override { TypePackId expectedRetType = lookupPack(call); @@ -91,6 +216,35 @@ struct TypeChecker2 : public AstVisitor return true; } + bool visit(AstExprFunction* fn) override + { + TypeId inferredFnTy = lookupType(fn); + const FunctionTypeVar* inferredFtv = get(inferredFnTy); + LUAU_ASSERT(inferredFtv); + + auto argIt = begin(inferredFtv->argTypes); + for (const auto& arg : fn->args) + { + if (argIt == end(inferredFtv->argTypes)) + break; + + if (arg->annotation) + { + TypeId inferredArgTy = *argIt; + TypeId annotatedArgTy = lookupAnnotation(arg->annotation); + + if (!isSubtype(annotatedArgTy, inferredArgTy, ice)) + { + reportError(TypeMismatch{annotatedArgTy, inferredArgTy}, arg->location); + } + } + + ++argIt; + } + + return true; + } + bool visit(AstExprIndexName* indexName) override { TypeId leftType = lookupType(indexName->expr); @@ -144,6 +298,25 @@ struct TypeChecker2 : public AstVisitor return true; } + bool visit(AstType* ty) override + { + return true; + } + + bool visit(AstTypeReference* ty) override + { + Scope2* scope = findInnermostScope(ty->location); + + // TODO: Imported types + // TODO: Generic types + if (!scope->lookupTypeBinding(ty->name.value)) + { + reportError(UnknownSymbol{ty->name.value, UnknownSymbol::Context::Type}, ty->location); + } + + return true; + } + void reportError(TypeErrorData&& data, const Location& location) { module->errors.emplace_back(location, sourceModule->name, std::move(data)); diff --git a/Analysis/src/TypeInfer.cpp b/Analysis/src/TypeInfer.cpp index fd1b3b85..44635e88 100644 --- a/Analysis/src/TypeInfer.cpp +++ b/Analysis/src/TypeInfer.cpp @@ -35,13 +35,9 @@ LUAU_FASTFLAGVARIABLE(LuauLowerBoundsCalculation, false) LUAU_FASTFLAGVARIABLE(DebugLuauFreezeDuringUnification, false) LUAU_FASTFLAGVARIABLE(LuauSelfCallAutocompleteFix2, false) LUAU_FASTFLAGVARIABLE(LuauReduceUnionRecursion, false) -LUAU_FASTFLAGVARIABLE(LuauOnlyMutateInstantiatedTables, false) -LUAU_FASTFLAGVARIABLE(LuauUnsealedTableLiteral, false) LUAU_FASTFLAGVARIABLE(LuauReturnAnyInsteadOfICE, false) // Eventually removed as false. LUAU_FASTFLAG(LuauNormalizeFlagIsConservative) LUAU_FASTFLAGVARIABLE(LuauReturnTypeInferenceInNonstrict, false) -LUAU_FASTFLAGVARIABLE(LuauRecursionLimitException, false); -LUAU_FASTFLAGVARIABLE(LuauApplyTypeFunctionFix, false); LUAU_FASTFLAGVARIABLE(LuauAlwaysQuantify, false); LUAU_FASTFLAGVARIABLE(LuauReportErrorsOnIndexerKeyMismatch, false) LUAU_FASTFLAG(LuauQuantifyConstrained) @@ -275,22 +271,15 @@ TypeChecker::TypeChecker(ModuleResolver* resolver, InternalErrorReporter* iceHan ModulePtr TypeChecker::check(const SourceModule& module, Mode mode, std::optional environmentScope) { - if (FFlag::LuauRecursionLimitException) - { - try - { - return checkWithoutRecursionCheck(module, mode, environmentScope); - } - catch (const RecursionLimitException&) - { - reportErrorCodeTooComplex(module.root->location); - return std::move(currentModule); - } - } - else + try { return checkWithoutRecursionCheck(module, mode, environmentScope); } + catch (const RecursionLimitException&) + { + reportErrorCodeTooComplex(module.root->location); + return std::move(currentModule); + } } ModulePtr TypeChecker::checkWithoutRecursionCheck(const SourceModule& module, Mode mode, std::optional environmentScope) @@ -445,22 +434,15 @@ void TypeChecker::checkBlock(const ScopePtr& scope, const AstStatBlock& block) reportErrorCodeTooComplex(block.location); return; } - if (FFlag::LuauRecursionLimitException) - { - try - { - checkBlockWithoutRecursionCheck(scope, block); - } - catch (const RecursionLimitException&) - { - reportErrorCodeTooComplex(block.location); - return; - } - } - else + try { checkBlockWithoutRecursionCheck(scope, block); } + catch (const RecursionLimitException&) + { + reportErrorCodeTooComplex(block.location); + return; + } } void TypeChecker::checkBlockWithoutRecursionCheck(const ScopePtr& scope, const AstStatBlock& block) @@ -1917,7 +1899,7 @@ std::optional TypeChecker::getIndexTypeFromType( for (TypeId t : utv) { - RecursionLimiter _rl(&recursionCount, FInt::LuauTypeInferRecursionLimit, "getIndexTypeForType unions"); + RecursionLimiter _rl(&recursionCount, FInt::LuauTypeInferRecursionLimit); // Not needed when we normalize types. if (get(follow(t))) @@ -1967,7 +1949,7 @@ std::optional TypeChecker::getIndexTypeFromType( for (TypeId t : itv->parts) { - RecursionLimiter _rl(&recursionCount, FInt::LuauTypeInferRecursionLimit, "getIndexTypeFromType intersections"); + RecursionLimiter _rl(&recursionCount, FInt::LuauTypeInferRecursionLimit); if (std::optional ty = getIndexTypeFromType(scope, t, name, location, false)) parts.push_back(*ty); @@ -2190,7 +2172,7 @@ TypeId TypeChecker::checkExprTable( } } - TableState state = (expr.items.size == 0 || isNonstrictMode() || FFlag::LuauUnsealedTableLiteral) ? TableState::Unsealed : TableState::Sealed; + TableState state = TableState::Unsealed; TableTypeVar table = TableTypeVar{std::move(props), indexer, scope->level, state}; table.definitionModuleName = currentModuleName; return addType(table); @@ -5175,9 +5157,7 @@ TypePackId TypeChecker::resolveTypePack(const ScopePtr& scope, const AstTypePack bool ApplyTypeFunction::isDirty(TypeId ty) { - if (FFlag::LuauApplyTypeFunctionFix && typeArguments.count(ty)) - return true; - else if (!FFlag::LuauApplyTypeFunctionFix && get(ty)) + if (typeArguments.count(ty)) return true; else if (const FreeTypeVar* ftv = get(ty)) { @@ -5191,9 +5171,7 @@ bool ApplyTypeFunction::isDirty(TypeId ty) bool ApplyTypeFunction::isDirty(TypePackId tp) { - if (FFlag::LuauApplyTypeFunctionFix && typePackArguments.count(tp)) - return true; - else if (!FFlag::LuauApplyTypeFunctionFix && get(tp)) + if (typePackArguments.count(tp)) return true; else return false; @@ -5218,29 +5196,15 @@ bool ApplyTypeFunction::ignoreChildren(TypePackId tp) TypeId ApplyTypeFunction::clean(TypeId ty) { TypeId& arg = typeArguments[ty]; - if (FFlag::LuauApplyTypeFunctionFix) - { - LUAU_ASSERT(arg); - return arg; - } - else if (arg) - return arg; - else - return addType(FreeTypeVar{level}); + LUAU_ASSERT(arg); + return arg; } TypePackId ApplyTypeFunction::clean(TypePackId tp) { TypePackId& arg = typePackArguments[tp]; - if (FFlag::LuauApplyTypeFunctionFix) - { - LUAU_ASSERT(arg); - return arg; - } - else if (arg) - return arg; - else - return addTypePack(FreeTypePack{level}); + LUAU_ASSERT(arg); + return arg; } TypeId TypeChecker::instantiateTypeFun(const ScopePtr& scope, const TypeFun& tf, const std::vector& typeParams, @@ -5273,7 +5237,7 @@ TypeId TypeChecker::instantiateTypeFun(const ScopePtr& scope, const TypeFun& tf, TypeId target = follow(instantiated); bool needsClone = follow(tf.type) == target; - bool shouldMutate = (!FFlag::LuauOnlyMutateInstantiatedTables || getTableType(tf.type)); + bool shouldMutate = getTableType(tf.type); TableTypeVar* ttv = getMutableTableType(target); if (shouldMutate && ttv && needsClone) diff --git a/Analysis/src/TypeVar.cpp b/Analysis/src/TypeVar.cpp index 0f53f990..ade70d72 100644 --- a/Analysis/src/TypeVar.cpp +++ b/Analysis/src/TypeVar.cpp @@ -23,7 +23,6 @@ LUAU_FASTFLAG(DebugLuauFreezeArena) LUAU_FASTINTVARIABLE(LuauTypeMaximumStringifierLength, 500) LUAU_FASTINTVARIABLE(LuauTableTypeMaximumStringifierLength, 0) LUAU_FASTINT(LuauTypeInferRecursionLimit) -LUAU_FASTFLAG(LuauSubtypingAddOptPropsToUnsealedTables) LUAU_FASTFLAG(LuauNonCopyableTypeVarFields) namespace Luau @@ -172,22 +171,15 @@ bool isString(TypeId ty) // Returns true when ty is a supertype of string bool maybeString(TypeId ty) { - if (FFlag::LuauSubtypingAddOptPropsToUnsealedTables) - { - ty = follow(ty); + ty = follow(ty); - if (isPrim(ty, PrimitiveTypeVar::String) || get(ty)) - return true; + if (isPrim(ty, PrimitiveTypeVar::String) || get(ty)) + return true; - if (auto utv = get(ty)) - return std::any_of(begin(utv), end(utv), maybeString); + if (auto utv = get(ty)) + return std::any_of(begin(utv), end(utv), maybeString); - return false; - } - else - { - return isString(ty); - } + return false; } bool isThread(TypeId ty) @@ -369,7 +361,7 @@ bool maybeSingleton(TypeId ty) bool hasLength(TypeId ty, DenseHashSet& seen, int* recursionCount) { - RecursionLimiter _rl(recursionCount, FInt::LuauTypeInferRecursionLimit, "hasLength"); + RecursionLimiter _rl(recursionCount, FInt::LuauTypeInferRecursionLimit); ty = follow(ty); diff --git a/Analysis/src/Unifiable.cpp b/Analysis/src/Unifiable.cpp index fe878358..8d23aa49 100644 --- a/Analysis/src/Unifiable.cpp +++ b/Analysis/src/Unifiable.cpp @@ -53,6 +53,14 @@ Generic::Generic(TypeLevel level, const Name& name) { } +Generic::Generic(Scope2* scope, const Name& name) + : index(++nextIndex) + , scope(scope) + , name(name) + , explicitName(true) +{ +} + int Generic::nextIndex = 0; Error::Error() diff --git a/Analysis/src/Unifier.cpp b/Analysis/src/Unifier.cpp index 877663de..6147e118 100644 --- a/Analysis/src/Unifier.cpp +++ b/Analysis/src/Unifier.cpp @@ -17,11 +17,8 @@ LUAU_FASTINT(LuauTypeInferTypePackLoopLimit); LUAU_FASTINT(LuauTypeInferIterationLimit); LUAU_FASTFLAG(LuauAutocompleteDynamicLimits) LUAU_FASTINTVARIABLE(LuauTypeInferLowerBoundsIterationLimit, 2000); -LUAU_FASTFLAGVARIABLE(LuauTableSubtypingVariance2, false); LUAU_FASTFLAG(LuauLowerBoundsCalculation); LUAU_FASTFLAG(LuauErrorRecoveryType); -LUAU_FASTFLAGVARIABLE(LuauSubtypingAddOptPropsToUnsealedTables, false) -LUAU_FASTFLAGVARIABLE(LuauTxnLogRefreshFunctionPointers, false) LUAU_FASTFLAG(LuauQuantifyConstrained) namespace Luau @@ -354,7 +351,7 @@ void Unifier::tryUnify(TypeId subTy, TypeId superTy, bool isFunctionCall, bool i void Unifier::tryUnify_(TypeId subTy, TypeId superTy, bool isFunctionCall, bool isIntersection) { RecursionLimiter _ra(&sharedState.counters.recursionCount, - FFlag::LuauAutocompleteDynamicLimits ? sharedState.counters.recursionLimit : FInt::LuauTypeInferRecursionLimit, "TypeId tryUnify_"); + FFlag::LuauAutocompleteDynamicLimits ? sharedState.counters.recursionLimit : FInt::LuauTypeInferRecursionLimit); ++sharedState.counters.iterationCount; @@ -983,7 +980,7 @@ void Unifier::tryUnify(TypePackId subTp, TypePackId superTp, bool isFunctionCall void Unifier::tryUnify_(TypePackId subTp, TypePackId superTp, bool isFunctionCall) { RecursionLimiter _ra(&sharedState.counters.recursionCount, - FFlag::LuauAutocompleteDynamicLimits ? sharedState.counters.recursionLimit : FInt::LuauTypeInferRecursionLimit, "TypePackId tryUnify_"); + FFlag::LuauAutocompleteDynamicLimits ? sharedState.counters.recursionLimit : FInt::LuauTypeInferRecursionLimit); ++sharedState.counters.iterationCount; @@ -1316,12 +1313,9 @@ void Unifier::tryUnifyFunctions(TypeId subTy, TypeId superTy, bool isFunctionCal tryUnify_(subFunction->retTypes, superFunction->retTypes); } - if (FFlag::LuauTxnLogRefreshFunctionPointers) - { - // Updating the log may have invalidated the function pointers - superFunction = log.getMutable(superTy); - subFunction = log.getMutable(subTy); - } + // Updating the log may have invalidated the function pointers + superFunction = log.getMutable(superTy); + subFunction = log.getMutable(subTy); ctx = context; @@ -1360,9 +1354,6 @@ struct Resetter void Unifier::tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) { - if (!FFlag::LuauTableSubtypingVariance2) - return DEPRECATED_tryUnifyTables(subTy, superTy, isIntersection); - TableTypeVar* superTable = log.getMutable(superTy); TableTypeVar* subTable = log.getMutable(subTy); @@ -1379,8 +1370,7 @@ void Unifier::tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) { auto subIter = subTable->props.find(propName); - if (subIter == subTable->props.end() && (!FFlag::LuauSubtypingAddOptPropsToUnsealedTables || subTable->state == TableState::Unsealed) && - !isOptional(superProp.type)) + if (subIter == subTable->props.end() && subTable->state == TableState::Unsealed && !isOptional(superProp.type)) missingProperties.push_back(propName); } @@ -1398,7 +1388,7 @@ void Unifier::tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) { auto superIter = superTable->props.find(propName); - if (superIter == superTable->props.end() && (FFlag::LuauSubtypingAddOptPropsToUnsealedTables || !isOptional(subProp.type))) + if (superIter == superTable->props.end()) extraProperties.push_back(propName); } @@ -1443,7 +1433,7 @@ void Unifier::tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) if (innerState.errors.empty()) log.concat(std::move(innerState.log)); } - else if ((!FFlag::LuauSubtypingAddOptPropsToUnsealedTables || subTable->state == TableState::Unsealed) && isOptional(prop.type)) + else if (subTable->state == TableState::Unsealed && isOptional(prop.type)) // This is sound because unsealed table types are precise, so `{ p : T } <: { p : T, q : U? }` // since if `t : { p : T }` then we are guaranteed that `t.q` is `nil`. // TODO: if the supertype is written to, the subtype may no longer be precise (alias analysis?) @@ -1512,9 +1502,6 @@ void Unifier::tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) else if (variance == Covariant) { } - else if (!FFlag::LuauSubtypingAddOptPropsToUnsealedTables && isOptional(prop.type)) - { - } else if (superTable->state == TableState::Free) { PendingType* pendingSuper = log.queue(superTy); @@ -1639,296 +1626,6 @@ TypeId Unifier::deeplyOptional(TypeId ty, std::unordered_map see return types->addType(UnionTypeVar{{getSingletonTypes().nilType, ty}}); } -void Unifier::DEPRECATED_tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) -{ - LUAU_ASSERT(!FFlag::LuauTableSubtypingVariance2); - Resetter resetter{&variance}; - variance = Invariant; - - TableTypeVar* superTable = log.getMutable(superTy); - TableTypeVar* subTable = log.getMutable(subTy); - - if (!superTable || !subTable) - ice("passed non-table types to unifyTables"); - - if (superTable->state == TableState::Sealed && subTable->state == TableState::Sealed) - return tryUnifySealedTables(subTy, superTy, isIntersection); - else if ((superTable->state == TableState::Sealed && subTable->state == TableState::Unsealed) || - (superTable->state == TableState::Unsealed && subTable->state == TableState::Sealed)) - return tryUnifySealedTables(subTy, superTy, isIntersection); - else if ((superTable->state == TableState::Sealed && subTable->state == TableState::Generic) || - (superTable->state == TableState::Generic && subTable->state == TableState::Sealed)) - reportError(TypeError{location, TypeMismatch{superTy, subTy}}); - else if ((superTable->state == TableState::Free) != (subTable->state == TableState::Free)) // one table is free and the other is not - { - TypeId freeTypeId = subTable->state == TableState::Free ? subTy : superTy; - TypeId otherTypeId = subTable->state == TableState::Free ? superTy : subTy; - - return tryUnifyFreeTable(otherTypeId, freeTypeId); - } - else if (superTable->state == TableState::Free && subTable->state == TableState::Free) - { - tryUnifyFreeTable(subTy, superTy); - - // avoid creating a cycle when the types are already pointing at each other - if (follow(superTy) != follow(subTy)) - { - log.bindTable(superTy, subTy); - } - return; - } - else if (superTable->state != TableState::Sealed && subTable->state != TableState::Sealed) - { - // All free tables are checked in one of the branches above - LUAU_ASSERT(superTable->state != TableState::Free); - LUAU_ASSERT(subTable->state != TableState::Free); - - // Tables must have exactly the same props and their types must all unify - // I honestly have no idea if this is remotely close to reasonable. - for (const auto& [name, prop] : superTable->props) - { - const auto& r = subTable->props.find(name); - if (r == subTable->props.end()) - reportError(TypeError{location, UnknownProperty{subTy, name}}); - else - tryUnify_(r->second.type, prop.type); - } - - if (superTable->indexer && subTable->indexer) - tryUnifyIndexer(*subTable->indexer, *superTable->indexer); - else if (superTable->indexer) - { - // passing/assigning a table without an indexer to something that has one - // e.g. table.insert(t, 1) where t is a non-sealed table and doesn't have an indexer. - if (subTable->state == TableState::Unsealed) - { - log.changeIndexer(subTy, superTable->indexer); - } - else - reportError(TypeError{location, CannotExtendTable{subTy, CannotExtendTable::Indexer}}); - } - } - else if (superTable->state == TableState::Sealed) - { - // lt is sealed and so it must be possible for rt to have precisely the same shape - // Verify that this is the case, then bind rt to lt. - ice("unsealed tables are not working yet", location); - } - else if (subTable->state == TableState::Sealed) - return tryUnifyTables(superTy, subTy, isIntersection); - else - ice("tryUnifyTables"); -} - -void Unifier::tryUnifyFreeTable(TypeId subTy, TypeId superTy) -{ - LUAU_ASSERT(!FFlag::LuauTableSubtypingVariance2); - TableTypeVar* freeTable = log.getMutable(superTy); - TableTypeVar* subTable = log.getMutable(subTy); - - if (!freeTable || !subTable) - ice("passed non-table types to tryUnifyFreeTable"); - - // Any properties in freeTable must unify with those in otherTable. - // Then bind freeTable to otherTable. - for (const auto& [freeName, freeProp] : freeTable->props) - { - if (auto subProp = findTablePropertyRespectingMeta(subTy, freeName)) - { - tryUnify_(*subProp, freeProp.type); - - /* - * TypeVars are commonly cyclic, so it is entirely possible - * for unifying a property of a table to change the table itself! - * We need to check for this and start over if we notice this occurring. - * - * I believe this is guaranteed to terminate eventually because this will - * only happen when a free table is bound to another table. - */ - if (!log.getMutable(superTy) || !log.getMutable(subTy)) - return tryUnify_(subTy, superTy); - - if (TableTypeVar* pendingFreeTtv = log.getMutable(superTy); pendingFreeTtv && pendingFreeTtv->boundTo) - return tryUnify_(subTy, superTy); - } - else - { - // If the other table is also free, then we are learning that it has more - // properties than we previously thought. Else, it is an error. - if (subTable->state == TableState::Free) - { - PendingType* pendingSub = log.queue(subTy); - TableTypeVar* pendingSubTtv = getMutable(pendingSub); - LUAU_ASSERT(pendingSubTtv); - pendingSubTtv->props.insert({freeName, freeProp}); - } - else - reportError(TypeError{location, UnknownProperty{subTy, freeName}}); - } - } - - if (freeTable->indexer && subTable->indexer) - { - Unifier innerState = makeChildUnifier(); - innerState.tryUnifyIndexer(*subTable->indexer, *freeTable->indexer); - - checkChildUnifierTypeMismatch(innerState.errors, superTy, subTy); - - log.concat(std::move(innerState.log)); - } - else if (subTable->state == TableState::Free && freeTable->indexer) - { - log.changeIndexer(superTy, subTable->indexer); - } - - if (!freeTable->boundTo && subTable->state != TableState::Free) - { - log.bindTable(superTy, subTy); - } -} - -void Unifier::tryUnifySealedTables(TypeId subTy, TypeId superTy, bool isIntersection) -{ - LUAU_ASSERT(!FFlag::LuauTableSubtypingVariance2); - TableTypeVar* superTable = log.getMutable(superTy); - TableTypeVar* subTable = log.getMutable(subTy); - - if (!superTable || !subTable) - ice("passed non-table types to unifySealedTables"); - - std::vector missingPropertiesInSuper; - bool isUnnamedTable = subTable->name == std::nullopt && subTable->syntheticName == std::nullopt; - bool errorReported = false; - - // Optimization: First test that the property sets are compatible without doing any recursive unification - if (!subTable->indexer) - { - for (const auto& [propName, superProp] : superTable->props) - { - auto subIter = subTable->props.find(propName); - if (subIter == subTable->props.end() && !isOptional(superProp.type)) - missingPropertiesInSuper.push_back(propName); - } - - if (!missingPropertiesInSuper.empty()) - { - reportError(TypeError{location, MissingProperties{superTy, subTy, std::move(missingPropertiesInSuper)}}); - return; - } - } - - Unifier innerState = makeChildUnifier(); - - // Tables must have exactly the same props and their types must all unify - for (const auto& it : superTable->props) - { - const auto& r = subTable->props.find(it.first); - if (r == subTable->props.end()) - { - if (isOptional(it.second.type)) - continue; - - missingPropertiesInSuper.push_back(it.first); - - innerState.reportError(TypeError{location, TypeMismatch{superTy, subTy}}); - } - else - { - if (isUnnamedTable && r->second.location) - { - size_t oldErrorSize = innerState.errors.size(); - Location old = innerState.location; - innerState.location = *r->second.location; - innerState.tryUnify_(r->second.type, it.second.type); - innerState.location = old; - - if (oldErrorSize != innerState.errors.size() && !errorReported) - { - errorReported = true; - reportError(innerState.errors.back()); - } - } - else - { - innerState.tryUnify_(r->second.type, it.second.type); - } - } - } - - if (superTable->indexer || subTable->indexer) - { - if (superTable->indexer && subTable->indexer) - innerState.tryUnifyIndexer(*subTable->indexer, *superTable->indexer); - else if (subTable->state == TableState::Unsealed) - { - if (superTable->indexer && !subTable->indexer) - { - log.changeIndexer(subTy, superTable->indexer); - } - } - else if (superTable->state == TableState::Unsealed) - { - if (subTable->indexer && !superTable->indexer) - { - log.changeIndexer(superTy, subTable->indexer); - } - } - else if (superTable->indexer) - { - innerState.tryUnify_(getSingletonTypes().stringType, superTable->indexer->indexType); - for (const auto& [name, type] : subTable->props) - { - const auto& it = superTable->props.find(name); - if (it == superTable->props.end()) - innerState.tryUnify_(type.type, superTable->indexer->indexResultType); - } - } - else - innerState.reportError(TypeError{location, TypeMismatch{superTy, subTy}}); - } - - if (!errorReported) - log.concat(std::move(innerState.log)); - else - return; - - if (!missingPropertiesInSuper.empty()) - { - reportError(TypeError{location, MissingProperties{superTy, subTy, std::move(missingPropertiesInSuper)}}); - return; - } - - // If the superTy is an immediate part of an intersection type, do not do extra-property check. - // Otherwise, we would falsely generate an extra-property-error for 's' in this code: - // local a: {n: number} & {s: string} = {n=1, s=""} - // When checking against the table '{n: number}'. - if (!isIntersection && superTable->state != TableState::Unsealed && !superTable->indexer) - { - // Check for extra properties in the subTy - std::vector extraPropertiesInSub; - - for (const auto& [subKey, subProp] : subTable->props) - { - const auto& superIt = superTable->props.find(subKey); - if (superIt == superTable->props.end()) - { - if (isOptional(subProp.type)) - continue; - - extraPropertiesInSub.push_back(subKey); - } - } - - if (!extraPropertiesInSub.empty()) - { - reportError(TypeError{location, MissingProperties{superTy, subTy, std::move(extraPropertiesInSub), MissingProperties::Extra}}); - return; - } - } - - checkChildUnifierTypeMismatch(innerState.errors, superTy, subTy); -} - void Unifier::tryUnifyWithMetatable(TypeId subTy, TypeId superTy, bool reversed) { const MetatableTypeVar* superMetatable = get(superTy); @@ -2068,14 +1765,6 @@ void Unifier::tryUnifyWithClass(TypeId subTy, TypeId superTy, bool reversed) return fail(); } -void Unifier::tryUnifyIndexer(const TableIndexer& subIndexer, const TableIndexer& superIndexer) -{ - LUAU_ASSERT(!FFlag::LuauTableSubtypingVariance2); - - tryUnify_(subIndexer.indexType, superIndexer.indexType); - tryUnify_(subIndexer.indexResultType, superIndexer.indexResultType); -} - static void queueTypePack(std::vector& queue, DenseHashSet& seenTypePacks, Unifier& state, TypePackId a, TypePackId anyTypePack) { while (true) @@ -2435,7 +2124,7 @@ void Unifier::occursCheck(TypeId needle, TypeId haystack) void Unifier::occursCheck(DenseHashSet& seen, TypeId needle, TypeId haystack) { RecursionLimiter _ra(&sharedState.counters.recursionCount, - FFlag::LuauAutocompleteDynamicLimits ? sharedState.counters.recursionLimit : FInt::LuauTypeInferRecursionLimit, "occursCheck for TypeId"); + FFlag::LuauAutocompleteDynamicLimits ? sharedState.counters.recursionLimit : FInt::LuauTypeInferRecursionLimit); auto check = [&](TypeId tv) { occursCheck(seen, needle, tv); @@ -2506,7 +2195,7 @@ void Unifier::occursCheck(DenseHashSet& seen, TypePackId needle, Typ ice("Expected needle pack to be free"); RecursionLimiter _ra(&sharedState.counters.recursionCount, - FFlag::LuauAutocompleteDynamicLimits ? sharedState.counters.recursionLimit : FInt::LuauTypeInferRecursionLimit, "occursCheck for TypePackId"); + FFlag::LuauAutocompleteDynamicLimits ? sharedState.counters.recursionLimit : FInt::LuauTypeInferRecursionLimit); while (!log.getMutable(haystack)) { diff --git a/CLI/Analyze.cpp b/CLI/Analyze.cpp index 8b03ea1a..81db7c35 100644 --- a/CLI/Analyze.cpp +++ b/CLI/Analyze.cpp @@ -9,6 +9,7 @@ #include "FileUtils.h" LUAU_FASTFLAG(DebugLuauTimeTracing) +LUAU_FASTFLAG(LuauTypeMismatchModuleNameResolution) enum class ReportFormat { @@ -49,6 +50,9 @@ static void reportError(const Luau::Frontend& frontend, ReportFormat format, con if (const Luau::SyntaxError* syntaxError = Luau::get_if(&error.data)) report(format, humanReadableName.c_str(), error.location, "SyntaxError", syntaxError->message.c_str()); + else if (FFlag::LuauTypeMismatchModuleNameResolution) + report(format, humanReadableName.c_str(), error.location, "TypeError", + Luau::toString(error, Luau::TypeErrorToStringOptions{frontend.fileResolver}).c_str()); else report(format, humanReadableName.c_str(), error.location, "TypeError", Luau::toString(error).c_str()); } diff --git a/CMakeLists.txt b/CMakeLists.txt index c624a132..e256e234 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -11,6 +11,7 @@ option(LUAU_BUILD_TESTS "Build tests" ON) option(LUAU_BUILD_WEB "Build Web module" OFF) option(LUAU_WERROR "Warnings as errors" OFF) option(LUAU_STATIC_CRT "Link with the static CRT (/MT)" OFF) +option(LUAU_EXTERN_C "Use extern C for all APIs" OFF) if(LUAU_STATIC_CRT) cmake_minimum_required(VERSION 3.15) @@ -115,6 +116,14 @@ target_compile_options(Luau.CodeGen PRIVATE ${LUAU_OPTIONS}) target_compile_options(Luau.VM PRIVATE ${LUAU_OPTIONS}) target_compile_options(isocline PRIVATE ${LUAU_OPTIONS} ${ISOCLINE_OPTIONS}) +if(LUAU_EXTERN_C) + # enable extern "C" for VM (lua.h, lualib.h) and Compiler (luacode.h) to make Luau friendlier to use from non-C++ languages + # note that we enable LUA_USE_LONGJMP=1 as well; otherwise functions like luaL_error will throw C++ exceptions, which can't be done from extern "C" functions + target_compile_definitions(Luau.VM PUBLIC LUA_USE_LONGJMP=1) + target_compile_definitions(Luau.VM PUBLIC LUA_API=extern\"C\") + target_compile_definitions(Luau.Compiler PUBLIC LUACODE_API=extern\"C\") +endif() + if (MSVC AND MSVC_VERSION GREATER_EQUAL 1924) # disable partial redundancy elimination which regresses interpreter codegen substantially in VS2022: # https://developercommunity.visualstudio.com/t/performance-regression-on-a-complex-interpreter-lo/1631863 diff --git a/Common/include/Luau/Bytecode.h b/Common/include/Luau/Bytecode.h index f71d893c..218bb5d5 100644 --- a/Common/include/Luau/Bytecode.h +++ b/Common/include/Luau/Bytecode.h @@ -7,7 +7,7 @@ // Creating the bytecode is outside the scope of this file and is handled by bytecode builder (BytecodeBuilder.h) and bytecode compiler (Compiler.h) // Note that ALL enums declared in this file are order-sensitive since the values are baked into bytecode that needs to be processed by legacy clients. -// Bytecode definitions +// # Bytecode definitions // Bytecode instructions are using "word code" - each instruction is one or many 32-bit words. // The first word in the instruction is always the instruction header, and *must* contain the opcode (enum below) in the least significant byte. // @@ -19,7 +19,7 @@ // Instruction word is sometimes followed by one extra word, indicated as AUX - this is just a 32-bit word and is decoded according to the specification for each opcode. // For each opcode the encoding is *static* - that is, based on the opcode you know a-priory how large the instruction is, with the exception of NEWCLOSURE -// Bytecode indices +// # Bytecode indices // Bytecode instructions commonly refer to integer values that define offsets or indices for various entities. For each type, there's a maximum encodable value. // Note that in some cases, the compiler will set a lower limit than the maximum encodable value is to prevent fragile code into bumping against the limits whenever we change the compilation details. // Additionally, in some specific instructions such as ANDK, the limit on the encoded value is smaller; this means that if a value is larger, a different instruction must be selected. @@ -29,6 +29,15 @@ // Constants: 0-2^23-1. Constants are stored in a table allocated with each proto; to allow for future bytecode tweaks the encodable value is limited to 23 bits. // Closures: 0-2^15-1. Closures are created from child protos via a child index; the limit is for the number of closures immediately referenced in each function. // Jumps: -2^23..2^23. Jump offsets are specified in word increments, so jumping over an instruction may sometimes require an offset of 2 or more. + +// # Bytecode versions +// Bytecode serialized format embeds a version number, that dictates both the serialized form as well as the allowed instructions. As long as the bytecode version falls into supported +// range (indicated by LBC_BYTECODE_MIN / LBC_BYTECODE_MAX) and was produced by Luau compiler, it should load and execute correctly. +// +// Note that Luau runtime doesn't provide indefinite bytecode compatibility: support for older versions gets removed over time. As such, bytecode isn't a durable storage format and it's expected +// that Luau users can recompile bytecode from source on Luau version upgrades if necessary. + +// Bytecode opcode, part of the instruction header enum LuauOpcode { // NOP: noop @@ -380,8 +389,10 @@ enum LuauOpcode // Bytecode tags, used internally for bytecode encoded as a string enum LuauBytecodeTag { - // Bytecode version - LBC_VERSION = 2, + // Bytecode version; runtime supports [MIN, MAX], compiler emits TARGET by default but may emit a higher version when flags are enabled + LBC_VERSION_MIN = 2, + LBC_VERSION_MAX = 2, + LBC_VERSION_TARGET = 2, // Types of constant table entries LBC_CONSTANT_NIL = 0, LBC_CONSTANT_BOOLEAN, diff --git a/Compiler/include/Luau/BytecodeBuilder.h b/Compiler/include/Luau/BytecodeBuilder.h index dbe54299..6ec10b53 100644 --- a/Compiler/include/Luau/BytecodeBuilder.h +++ b/Compiler/include/Luau/BytecodeBuilder.h @@ -119,6 +119,8 @@ public: static std::string getError(const std::string& message); + static uint8_t getVersion(); + private: struct Constant { diff --git a/Compiler/src/BytecodeBuilder.cpp b/Compiler/src/BytecodeBuilder.cpp index a34f7603..301cf255 100644 --- a/Compiler/src/BytecodeBuilder.cpp +++ b/Compiler/src/BytecodeBuilder.cpp @@ -9,6 +9,9 @@ namespace Luau { +static_assert(LBC_VERSION_TARGET >= LBC_VERSION_MIN && LBC_VERSION_TARGET <= LBC_VERSION_MAX, "Invalid bytecode version setup"); +static_assert(LBC_VERSION_MAX <= 127, "Bytecode version should be 7-bit so that we can extend the serialization to use varint transparently"); + static const uint32_t kMaxConstantCount = 1 << 23; static const uint32_t kMaxClosureCount = 1 << 15; @@ -572,7 +575,10 @@ void BytecodeBuilder::finalize() bytecode.reserve(capacity); // assemble final bytecode blob - bytecode = char(LBC_VERSION); + uint8_t version = getVersion(); + LUAU_ASSERT(version >= LBC_VERSION_MIN && version <= LBC_VERSION_MAX); + + bytecode = char(version); writeStringTable(bytecode); @@ -1040,7 +1046,7 @@ void BytecodeBuilder::expandJumps() std::string BytecodeBuilder::getError(const std::string& message) { - // 0 acts as a special marker for error bytecode (it's equal to LBC_VERSION for valid bytecode blobs) + // 0 acts as a special marker for error bytecode (it's equal to LBC_VERSION_TARGET for valid bytecode blobs) std::string result; result += char(0); result += message; @@ -1048,6 +1054,12 @@ std::string BytecodeBuilder::getError(const std::string& message) return result; } +uint8_t BytecodeBuilder::getVersion() +{ + // This function usually returns LBC_VERSION_TARGET but may sometimes return a higher number (within LBC_VERSION_MIN/MAX) under fast flags + return LBC_VERSION_TARGET; +} + #ifdef LUAU_ASSERTENABLED void BytecodeBuilder::validate() const { diff --git a/Compiler/src/Compiler.cpp b/Compiler/src/Compiler.cpp index 52dc9242..e732256b 100644 --- a/Compiler/src/Compiler.cpp +++ b/Compiler/src/Compiler.cpp @@ -16,8 +16,6 @@ #include #include -LUAU_FASTFLAGVARIABLE(LuauCompileIterNoPairs, false) - LUAU_FASTINTVARIABLE(LuauCompileLoopUnrollThreshold, 25) LUAU_FASTINTVARIABLE(LuauCompileLoopUnrollThresholdMaxBoost, 300) @@ -2672,7 +2670,7 @@ struct Compiler else if (builtin.isGlobal("pairs")) // for .. in pairs(t) { skipOp = LOP_FORGPREP_NEXT; - loopOp = FFlag::LuauCompileIterNoPairs ? LOP_FORGLOOP : LOP_FORGLOOP_NEXT; + loopOp = LOP_FORGLOOP; } } else if (stat->values.size == 2) @@ -2682,7 +2680,7 @@ struct Compiler if (builtin.isGlobal("next")) // for .. in next,t { skipOp = LOP_FORGPREP_NEXT; - loopOp = FFlag::LuauCompileIterNoPairs ? LOP_FORGLOOP : LOP_FORGLOOP_NEXT; + loopOp = LOP_FORGLOOP; } } } diff --git a/VM/src/ludata.cpp b/VM/src/ludata.cpp index 28152689..c2110cb3 100644 --- a/VM/src/ludata.cpp +++ b/VM/src/ludata.cpp @@ -26,6 +26,8 @@ void luaU_freeudata(lua_State* L, Udata* u, lua_Page* page) { void (*dtor)(lua_State*, void*) = nullptr; dtor = L->global->udatagc[u->tag]; + // TODO: access to L here is highly unsafe since this is called during internal GC traversal + // certain operations such as lua_getthreaddata are okay, but by and large this risks crashes on improper use if (dtor) dtor(L, u->data); } diff --git a/VM/src/lvmload.cpp b/VM/src/lvmload.cpp index 8b742f1c..86afddd2 100644 --- a/VM/src/lvmload.cpp +++ b/VM/src/lvmload.cpp @@ -154,11 +154,11 @@ int luau_load(lua_State* L, const char* chunkname, const char* data, size_t size return 1; } - if (version != LBC_VERSION) + if (version < LBC_VERSION_MIN || version > LBC_VERSION_MAX) { char chunkid[LUA_IDSIZE]; luaO_chunkid(chunkid, chunkname, LUA_IDSIZE); - lua_pushfstring(L, "%s: bytecode version mismatch (expected %d, got %d)", chunkid, LBC_VERSION, version); + lua_pushfstring(L, "%s: bytecode version mismatch (expected [%d..%d], got %d)", chunkid, LBC_VERSION_MIN, LBC_VERSION_MAX, version); return 1; } diff --git a/tests/Compiler.test.cpp b/tests/Compiler.test.cpp index 036bf124..655e48cb 100644 --- a/tests/Compiler.test.cpp +++ b/tests/Compiler.test.cpp @@ -261,8 +261,6 @@ L1: RETURN R0 0 TEST_CASE("ForBytecode") { - ScopedFastFlag sff2("LuauCompileIterNoPairs", false); - // basic for loop: variable directly refers to internal iteration index (R2) CHECK_EQ("\n" + compileFunction0("for i=1,5 do print(i) end"), R"( LOADN R2 1 @@ -329,7 +327,7 @@ L0: GETIMPORT R5 3 MOVE R6 R3 MOVE R7 R4 CALL R5 2 0 -L1: FORGLOOP_NEXT R0 L0 +L1: FORGLOOP R0 L0 2 RETURN R0 0 )"); @@ -342,7 +340,7 @@ L0: GETIMPORT R5 3 MOVE R6 R3 MOVE R7 R4 CALL R5 2 0 -L1: FORGLOOP_NEXT R0 L0 +L1: FORGLOOP R0 L0 2 RETURN R0 0 )"); } @@ -2262,8 +2260,6 @@ TEST_CASE("TypeAliasing") TEST_CASE("DebugLineInfo") { - ScopedFastFlag sff("LuauCompileIterNoPairs", false); - Luau::BytecodeBuilder bcb; bcb.setDumpFlags(Luau::BytecodeBuilder::Dump_Code | Luau::BytecodeBuilder::Dump_Lines); Luau::compileOrThrow(bcb, R"( @@ -2313,7 +2309,7 @@ return result 15: L0: MOVE R7 R1 15: MOVE R8 R5 15: CONCAT R1 R7 R8 -14: L1: FORGLOOP_NEXT R2 L0 +14: L1: FORGLOOP R2 L0 1 17: RETURN R1 1 )"); } @@ -2545,8 +2541,6 @@ a TEST_CASE("DebugSource") { - ScopedFastFlag sff("LuauCompileIterNoPairs", false); - const char* source = R"( local kSelectedBiomes = { ['Mountains'] = true, @@ -2614,7 +2608,7 @@ L0: MOVE R7 R1 MOVE R8 R5 CONCAT R1 R7 R8 14: for k in pairs(kSelectedBiomes) do -L1: FORGLOOP_NEXT R2 L0 +L1: FORGLOOP R2 L0 1 17: return result RETURN R1 1 )"); @@ -2622,8 +2616,6 @@ RETURN R1 1 TEST_CASE("DebugLocals") { - ScopedFastFlag sff("LuauCompileIterNoPairs", false); - const char* source = R"( function foo(e, f) local a = 1 @@ -2661,12 +2653,12 @@ end local 0: reg 5, start pc 5 line 5, end pc 8 line 5 local 1: reg 6, start pc 14 line 8, end pc 18 line 8 local 2: reg 7, start pc 14 line 8, end pc 18 line 8 -local 3: reg 3, start pc 21 line 12, end pc 24 line 12 -local 4: reg 3, start pc 26 line 16, end pc 30 line 16 -local 5: reg 0, start pc 0 line 3, end pc 34 line 21 -local 6: reg 1, start pc 0 line 3, end pc 34 line 21 -local 7: reg 2, start pc 1 line 4, end pc 34 line 21 -local 8: reg 3, start pc 34 line 21, end pc 34 line 21 +local 3: reg 3, start pc 22 line 12, end pc 25 line 12 +local 4: reg 3, start pc 27 line 16, end pc 31 line 16 +local 5: reg 0, start pc 0 line 3, end pc 35 line 21 +local 6: reg 1, start pc 0 line 3, end pc 35 line 21 +local 7: reg 2, start pc 1 line 4, end pc 35 line 21 +local 8: reg 3, start pc 35 line 21, end pc 35 line 21 3: LOADN R2 1 4: LOADN R5 1 4: LOADN R3 3 @@ -2683,7 +2675,7 @@ local 8: reg 3, start pc 34 line 21, end pc 34 line 21 8: MOVE R9 R6 8: MOVE R10 R7 8: CALL R8 2 0 -7: L3: FORGLOOP_NEXT R3 L2 +7: L3: FORGLOOP R3 L2 2 11: LOADN R3 2 12: GETIMPORT R4 1 12: LOADN R5 2 @@ -3795,8 +3787,6 @@ RETURN R0 1 TEST_CASE("SharedClosure") { - ScopedFastFlag sff("LuauCompileIterNoPairs", false); - // closures can be shared even if functions refer to upvalues, as long as upvalues are top-level CHECK_EQ("\n" + compileFunction(R"( local val = ... @@ -3939,7 +3929,7 @@ L2: GETIMPORT R5 1 NEWCLOSURE R6 P1 CAPTURE VAL R3 CALL R5 1 0 -L3: FORGLOOP_NEXT R0 L2 +L3: FORGLOOP R0 L2 2 LOADN R2 1 LOADN R0 10 LOADN R1 1 diff --git a/tests/Fixture.h b/tests/Fixture.h index ffcd4b9e..0e3735f6 100644 --- a/tests/Fixture.h +++ b/tests/Fixture.h @@ -2,13 +2,13 @@ #pragma once #include "Luau/Config.h" -#include "Luau/ConstraintGraphBuilder.h" #include "Luau/FileResolver.h" #include "Luau/Frontend.h" #include "Luau/IostreamHelpers.h" #include "Luau/Linter.h" #include "Luau/Location.h" #include "Luau/ModuleResolver.h" +#include "Luau/Scope.h" #include "Luau/ToString.h" #include "Luau/TypeInfer.h" #include "Luau/TypeVar.h" diff --git a/tests/Module.test.cpp b/tests/Module.test.cpp index d585b731..7c2f4d1c 100644 --- a/tests/Module.test.cpp +++ b/tests/Module.test.cpp @@ -279,7 +279,6 @@ TEST_CASE_FIXTURE(Fixture, "clone_recursion_limit") int limit = 400; #endif ScopedFastInt luauTypeCloneRecursionLimit{"LuauTypeCloneRecursionLimit", limit}; - ScopedFastFlag sff{"LuauRecursionLimitException", true}; TypeArena src; diff --git a/tests/Normalize.test.cpp b/tests/Normalize.test.cpp index 284230c9..a474b6e7 100644 --- a/tests/Normalize.test.cpp +++ b/tests/Normalize.test.cpp @@ -12,7 +12,6 @@ using namespace Luau; struct NormalizeFixture : Fixture { ScopedFastFlag sff1{"LuauLowerBoundsCalculation", true}; - ScopedFastFlag sff2{"LuauTableSubtypingVariance2", true}; }; void createSomeClasses(TypeChecker& typeChecker) diff --git a/tests/RuntimeLimits.test.cpp b/tests/RuntimeLimits.test.cpp index bef38fc3..6619147b 100644 --- a/tests/RuntimeLimits.test.cpp +++ b/tests/RuntimeLimits.test.cpp @@ -264,10 +264,13 @@ TEST_CASE_FIXTURE(LimitFixture, "typescript_port_of_Result_type") } )LUA"; + CheckResult result = check(src); + CodeTooComplex ctc; + if (FFlag::LuauLowerBoundsCalculation) - (void)check(src); + LUAU_REQUIRE_ERRORS(result); else - CHECK_THROWS_AS(check(src), std::exception); + CHECK(hasError(result, &ctc)); } TEST_SUITE_END(); diff --git a/tests/ToString.test.cpp b/tests/ToString.test.cpp index 4d2e94ee..e03069a9 100644 --- a/tests/ToString.test.cpp +++ b/tests/ToString.test.cpp @@ -409,8 +409,6 @@ TEST_CASE_FIXTURE(Fixture, "toStringDetailed") TEST_CASE_FIXTURE(BuiltinsFixture, "toStringDetailed2") { - ScopedFastFlag sff{"LuauUnsealedTableLiteral", true}; - CheckResult result = check(R"( local base = {} function base:one() return 1 end diff --git a/tests/TypeInfer.aliases.test.cpp b/tests/TypeInfer.aliases.test.cpp index 86cc9701..d6f0a0c8 100644 --- a/tests/TypeInfer.aliases.test.cpp +++ b/tests/TypeInfer.aliases.test.cpp @@ -7,8 +7,21 @@ using namespace Luau; +LUAU_FASTFLAG(DebugLuauDeferredConstraintResolution) + TEST_SUITE_BEGIN("TypeAliases"); +TEST_CASE_FIXTURE(Fixture, "basic_alias") +{ + CheckResult result = check(R"( + type T = number + local x: T = 1 + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + CHECK_EQ("number", toString(requireType("x"))); +} + TEST_CASE_FIXTURE(Fixture, "cyclic_function_type_in_type_alias") { CheckResult result = check(R"( @@ -24,6 +37,63 @@ TEST_CASE_FIXTURE(Fixture, "cyclic_function_type_in_type_alias") CHECK_EQ("t1 where t1 = () -> t1?", toString(requireType("g"))); } +TEST_CASE_FIXTURE(Fixture, "names_are_ascribed") +{ + CheckResult result = check(R"( + type T = { x: number } + local x: T + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + CHECK_EQ("T", toString(requireType("x"))); +} + +TEST_CASE_FIXTURE(Fixture, "cannot_steal_hoisted_type_alias") +{ + // This is a tricky case. In order to support recursive type aliases, + // we first walk the block and generate free types as placeholders. + // We then walk the AST as normal. If we declare a type alias as below, + // we generate a free type. We then begin our normal walk, examining + // local x: T = "foo", which establishes two constraints: + // a <: b + // string <: a + // We then visit the type alias, and establish that + // b <: number + // Then, when solving these constraints, we dispatch them in the order + // they appear above. This means that a ~ b, and a ~ string, thus + // b ~ string. This means the b <: number constraint has no effect. + // Essentially we've "stolen" the alias's type out from under it. + // This test ensures that we don't actually do this. + CheckResult result = check(R"( + local x: T = "foo" + type T = number + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + if (FFlag::DebugLuauDeferredConstraintResolution) + { + CHECK(result.errors[0] == TypeError{ + Location{{1, 21}, {1, 26}}, + getMainSourceModule()->name, + TypeMismatch{ + getSingletonTypes().numberType, + getSingletonTypes().stringType, + }, + }); + } + else + { + CHECK(result.errors[0] == TypeError{ + Location{{1, 8}, {1, 26}}, + getMainSourceModule()->name, + TypeMismatch{ + getSingletonTypes().numberType, + getSingletonTypes().stringType, + }, + }); + } +} + TEST_CASE_FIXTURE(Fixture, "cyclic_types_of_named_table_fields_do_not_expand_when_stringified") { CheckResult result = check(R"( @@ -41,7 +111,22 @@ TEST_CASE_FIXTURE(Fixture, "cyclic_types_of_named_table_fields_do_not_expand_whe CHECK_EQ(typeChecker.numberType, tm->givenType); } -TEST_CASE_FIXTURE(Fixture, "mutually_recursive_types") +TEST_CASE_FIXTURE(Fixture, "mutually_recursive_aliases") +{ + CheckResult result = check(R"( + --!strict + type T = { f: number, g: U } + type U = { h: number, i: T? } + local x: T = { f = 37, g = { h = 5, i = nil } } + x.g.i = x + local y: T = { f = 3, g = { h = 5, i = nil } } + y.g.i = y + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "mutually_recursive_generic_aliases") { CheckResult result = check(R"( --!strict diff --git a/tests/TypeInfer.annotations.test.cpp b/tests/TypeInfer.annotations.test.cpp index ccdd2b37..3e2ad6dc 100644 --- a/tests/TypeInfer.annotations.test.cpp +++ b/tests/TypeInfer.annotations.test.cpp @@ -30,11 +30,21 @@ TEST_CASE_FIXTURE(Fixture, "successful_check") dumpErrors(result); } +TEST_CASE_FIXTURE(Fixture, "variable_type_is_supertype") +{ + CheckResult result = check(R"( + local x: number = 1 + local y: number? = x + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + TEST_CASE_FIXTURE(Fixture, "function_parameters_can_have_annotations") { CheckResult result = check(R"( function double(x: number) - return x * 2 + return 2 end local four = double(2) @@ -47,7 +57,7 @@ TEST_CASE_FIXTURE(Fixture, "function_parameter_annotations_are_checked") { CheckResult result = check(R"( function double(x: number) - return x * 2 + return 2 end local four = double("two") @@ -70,13 +80,13 @@ TEST_CASE_FIXTURE(Fixture, "function_return_annotations_are_checked") const FunctionTypeVar* ftv = get(fiftyType); REQUIRE(ftv != nullptr); - TypePackId retPack = ftv->retTypes; + TypePackId retPack = follow(ftv->retTypes); const TypePack* tp = get(retPack); REQUIRE(tp != nullptr); REQUIRE_EQ(1, tp->head.size()); - REQUIRE_EQ(typeChecker.anyType, tp->head[0]); + REQUIRE_EQ(typeChecker.anyType, follow(tp->head[0])); } TEST_CASE_FIXTURE(Fixture, "function_return_multret_annotations_are_checked") @@ -116,6 +126,23 @@ TEST_CASE_FIXTURE(Fixture, "function_return_annotation_should_continuously_parse LUAU_REQUIRE_ERROR_COUNT(1, result); } +TEST_CASE_FIXTURE(Fixture, "unknown_type_reference_generates_error") +{ + CheckResult result = check(R"( + local x: IDoNotExist + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK(result.errors[0] == TypeError{ + Location{{1, 17}, {1, 28}}, + getMainSourceModule()->name, + UnknownSymbol{ + "IDoNotExist", + UnknownSymbol::Context::Type, + }, + }); +} + TEST_CASE_FIXTURE(Fixture, "typeof_variable_type_annotation_should_return_its_type") { CheckResult result = check(R"( @@ -632,7 +659,10 @@ int AssertionCatcher::tripped; TEST_CASE_FIXTURE(Fixture, "luau_ice_triggers_an_ice") { - ScopedFastFlag sffs{"DebugLuauMagicTypes", true}; + ScopedFastFlag sffs[] = { + {"DebugLuauMagicTypes", true}, + {"LuauUseInternalCompilerErrorException", false}, + }; AssertionCatcher ac; @@ -646,9 +676,10 @@ TEST_CASE_FIXTURE(Fixture, "luau_ice_triggers_an_ice") TEST_CASE_FIXTURE(Fixture, "luau_ice_triggers_an_ice_handler") { - ScopedFastFlag sffs{"DebugLuauMagicTypes", true}; - - AssertionCatcher ac; + ScopedFastFlag sffs[] = { + {"DebugLuauMagicTypes", true}, + {"LuauUseInternalCompilerErrorException", false}, + }; bool caught = false; @@ -662,8 +693,44 @@ TEST_CASE_FIXTURE(Fixture, "luau_ice_triggers_an_ice_handler") std::runtime_error); CHECK_EQ(true, caught); +} - frontend.iceHandler.onInternalError = {}; +TEST_CASE_FIXTURE(Fixture, "luau_ice_triggers_an_ice_exception_with_flag") +{ + ScopedFastFlag sffs[] = { + {"DebugLuauMagicTypes", true}, + {"LuauUseInternalCompilerErrorException", true}, + }; + + AssertionCatcher ac; + + CHECK_THROWS_AS(check(R"( + local a: _luau_ice = 55 + )"), + InternalCompilerError); + + LUAU_ASSERT(1 == AssertionCatcher::tripped); +} + +TEST_CASE_FIXTURE(Fixture, "luau_ice_triggers_an_ice_exception_with_flag_handler") +{ + ScopedFastFlag sffs[] = { + {"DebugLuauMagicTypes", true}, + {"LuauUseInternalCompilerErrorException", true}, + }; + + bool caught = false; + + frontend.iceHandler.onInternalError = [&](const char*) { + caught = true; + }; + + CHECK_THROWS_AS(check(R"( + local a: _luau_ice = 55 + )"), + InternalCompilerError); + + CHECK_EQ(true, caught); } TEST_CASE_FIXTURE(Fixture, "luau_ice_is_not_special_without_the_flag") diff --git a/tests/TypeInfer.generics.test.cpp b/tests/TypeInfer.generics.test.cpp index edb5adcf..97ba0808 100644 --- a/tests/TypeInfer.generics.test.cpp +++ b/tests/TypeInfer.generics.test.cpp @@ -700,11 +700,6 @@ end TEST_CASE_FIXTURE(Fixture, "generic_functions_should_be_memory_safe") { - ScopedFastFlag sffs[] = { - {"LuauTableSubtypingVariance2", true}, - {"LuauUnsealedTableLiteral", true}, - }; - CheckResult result = check(R"( --!strict -- At one point this produced a UAF @@ -979,8 +974,6 @@ TEST_CASE_FIXTURE(Fixture, "instantiate_generic_function_in_assignments2") TEST_CASE_FIXTURE(Fixture, "self_recursive_instantiated_param") { - ScopedFastFlag sff{"LuauOnlyMutateInstantiatedTables", true}; - // Mutability in type function application right now can create strange recursive types CheckResult result = check(R"( type Table = { a: number } @@ -1015,8 +1008,6 @@ TEST_CASE_FIXTURE(Fixture, "no_stack_overflow_from_quantifying") TEST_CASE_FIXTURE(BuiltinsFixture, "infer_generic_function_function_argument") { - ScopedFastFlag sff{"LuauUnsealedTableLiteral", true}; - CheckResult result = check(R"( local function sum(x: a, y: a, f: (a, a) -> a) return f(x, y) @@ -1123,8 +1114,6 @@ TEST_CASE_FIXTURE(Fixture, "substitution_with_bound_table") TEST_CASE_FIXTURE(Fixture, "apply_type_function_nested_generics1") { - ScopedFastFlag sff{"LuauApplyTypeFunctionFix", true}; - // https://github.com/Roblox/luau/issues/484 CheckResult result = check(R"( --!strict @@ -1153,8 +1142,6 @@ local complex: ComplexObject = { TEST_CASE_FIXTURE(Fixture, "apply_type_function_nested_generics2") { - ScopedFastFlag sff{"LuauApplyTypeFunctionFix", true}; - // https://github.com/Roblox/luau/issues/484 CheckResult result = check(R"( --!strict diff --git a/tests/TypeInfer.modules.test.cpp b/tests/TypeInfer.modules.test.cpp index afec20bf..a0f670f1 100644 --- a/tests/TypeInfer.modules.test.cpp +++ b/tests/TypeInfer.modules.test.cpp @@ -12,8 +12,6 @@ using namespace Luau; -LUAU_FASTFLAG(LuauTableSubtypingVariance2) - TEST_SUITE_BEGIN("TypeInferModules"); TEST_CASE_FIXTURE(BuiltinsFixture, "require") @@ -326,16 +324,9 @@ local b: B.T = a CheckResult result = frontend.check("game/C"); LUAU_REQUIRE_ERROR_COUNT(1, result); - if (FFlag::LuauTableSubtypingVariance2) - { - CHECK_EQ(toString(result.errors[0]), R"(Type 'T' from 'game/A' could not be converted into 'T' from 'game/B' + CHECK_EQ(toString(result.errors[0]), R"(Type 'T' from 'game/A' could not be converted into 'T' from 'game/B' caused by: Property 'x' is not compatible. Type 'number' could not be converted into 'string')"); - } - else - { - CHECK_EQ(toString(result.errors[0]), "Type 'T' from 'game/A' could not be converted into 'T' from 'game/B'"); - } } TEST_CASE_FIXTURE(BuiltinsFixture, "module_type_conflict_instantiated") @@ -367,16 +358,9 @@ local b: B.T = a CheckResult result = frontend.check("game/D"); LUAU_REQUIRE_ERROR_COUNT(1, result); - if (FFlag::LuauTableSubtypingVariance2) - { - CHECK_EQ(toString(result.errors[0]), R"(Type 'T' from 'game/B' could not be converted into 'T' from 'game/C' + CHECK_EQ(toString(result.errors[0]), R"(Type 'T' from 'game/B' could not be converted into 'T' from 'game/C' caused by: Property 'x' is not compatible. Type 'number' could not be converted into 'string')"); - } - else - { - CHECK_EQ(toString(result.errors[0]), "Type 'T' from 'game/B' could not be converted into 'T' from 'game/C'"); - } } TEST_SUITE_END(); diff --git a/tests/TypeInfer.refinements.test.cpp b/tests/TypeInfer.refinements.test.cpp index cefba4b2..3f5dad3d 100644 --- a/tests/TypeInfer.refinements.test.cpp +++ b/tests/TypeInfer.refinements.test.cpp @@ -353,8 +353,6 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "assert_non_binary_expressions_actually_resol TEST_CASE_FIXTURE(Fixture, "assign_table_with_refined_property_with_a_similar_type_is_illegal") { - ScopedFastFlag LuauTableSubtypingVariance2{"LuauTableSubtypingVariance2", true}; - CheckResult result = check(R"( local t: {x: number?} = {x = nil} diff --git a/tests/TypeInfer.singletons.test.cpp b/tests/TypeInfer.singletons.test.cpp index a90f434f..4a88abee 100644 --- a/tests/TypeInfer.singletons.test.cpp +++ b/tests/TypeInfer.singletons.test.cpp @@ -260,10 +260,6 @@ TEST_CASE_FIXTURE(Fixture, "table_properties_alias_or_parens_is_indexer") TEST_CASE_FIXTURE(Fixture, "table_properties_type_error_escapes") { - ScopedFastFlag sffs[]{ - {"LuauUnsealedTableLiteral", true}, - }; - CheckResult result = check(R"( --!strict local x: { ["<>"] : number } diff --git a/tests/TypeInfer.tables.test.cpp b/tests/TypeInfer.tables.test.cpp index 87d49651..77a2928c 100644 --- a/tests/TypeInfer.tables.test.cpp +++ b/tests/TypeInfer.tables.test.cpp @@ -276,8 +276,6 @@ TEST_CASE_FIXTURE(Fixture, "open_table_unification") TEST_CASE_FIXTURE(Fixture, "open_table_unification_2") { - ScopedFastFlag sff{"LuauTableSubtypingVariance2", true}; - CheckResult result = check(R"( local a = {} a.x = 99 @@ -347,8 +345,6 @@ TEST_CASE_FIXTURE(Fixture, "table_param_row_polymorphism_1") TEST_CASE_FIXTURE(Fixture, "table_param_row_polymorphism_2") { - ScopedFastFlag sff{"LuauTableSubtypingVariance2", true}; - CheckResult result = check(R"( --!strict function foo(o) @@ -370,8 +366,6 @@ TEST_CASE_FIXTURE(Fixture, "table_param_row_polymorphism_2") TEST_CASE_FIXTURE(Fixture, "table_param_row_polymorphism_3") { - ScopedFastFlag sff{"LuauTableSubtypingVariance2", true}; - CheckResult result = check(R"( local T = {} T.bar = 'hello' @@ -477,8 +471,6 @@ TEST_CASE_FIXTURE(Fixture, "ok_to_add_property_to_free_table") TEST_CASE_FIXTURE(Fixture, "okay_to_add_property_to_unsealed_tables_by_assignment") { - ScopedFastFlag sff{"LuauTableSubtypingVariance2", true}; - CheckResult result = check(R"( --!strict local t = { u = {} } @@ -512,8 +504,6 @@ TEST_CASE_FIXTURE(Fixture, "okay_to_add_property_to_unsealed_tables_by_function_ TEST_CASE_FIXTURE(Fixture, "width_subtyping") { - ScopedFastFlag sff{"LuauTableSubtypingVariance2", true}; - CheckResult result = check(R"( --!strict function f(x : { q : number }) @@ -772,8 +762,6 @@ TEST_CASE_FIXTURE(Fixture, "infer_indexer_for_left_unsealed_table_from_right_han TEST_CASE_FIXTURE(Fixture, "sealed_table_value_can_infer_an_indexer") { - ScopedFastFlag sff{"LuauTableSubtypingVariance2", true}; - CheckResult result = check(R"( local t: { a: string, [number]: string } = { a = "foo" } )"); @@ -783,8 +771,6 @@ TEST_CASE_FIXTURE(Fixture, "sealed_table_value_can_infer_an_indexer") TEST_CASE_FIXTURE(Fixture, "array_factory_function") { - ScopedFastFlag sff{"LuauTableSubtypingVariance2", true}; - CheckResult result = check(R"( function empty() return {} end local array: {string} = empty() @@ -1175,8 +1161,6 @@ TEST_CASE_FIXTURE(Fixture, "defining_a_self_method_for_a_local_sealed_table_must TEST_CASE_FIXTURE(Fixture, "defining_a_method_for_a_local_unsealed_table_is_ok") { - ScopedFastFlag sff{"LuauUnsealedTableLiteral", true}; - CheckResult result = check(R"( local t = {x = 1} function t.m() end @@ -1187,8 +1171,6 @@ TEST_CASE_FIXTURE(Fixture, "defining_a_method_for_a_local_unsealed_table_is_ok") TEST_CASE_FIXTURE(Fixture, "defining_a_self_method_for_a_local_unsealed_table_is_ok") { - ScopedFastFlag sff{"LuauUnsealedTableLiteral", true}; - CheckResult result = check(R"( local t = {x = 1} function t:m() end @@ -1468,11 +1450,6 @@ TEST_CASE_FIXTURE(Fixture, "right_table_missing_key2") TEST_CASE_FIXTURE(Fixture, "casting_unsealed_tables_with_props_into_table_with_indexer") { - ScopedFastFlag sff[]{ - {"LuauTableSubtypingVariance2", true}, - {"LuauUnsealedTableLiteral", true}, - }; - CheckResult result = check(R"( type StringToStringMap = { [string]: string } local rt: StringToStringMap = { ["foo"] = 1 } @@ -1518,11 +1495,6 @@ TEST_CASE_FIXTURE(Fixture, "casting_tables_with_props_into_table_with_indexer2") TEST_CASE_FIXTURE(Fixture, "casting_tables_with_props_into_table_with_indexer3") { - ScopedFastFlag sff[]{ - {"LuauTableSubtypingVariance2", true}, - {"LuauUnsealedTableLiteral", true}, - }; - CheckResult result = check(R"( local function foo(a: {[string]: number, a: string}) end foo({ a = 1 }) @@ -1609,8 +1581,6 @@ TEST_CASE_FIXTURE(Fixture, "table_subtyping_with_extra_props_dont_report_multipl TEST_CASE_FIXTURE(Fixture, "table_subtyping_with_extra_props_is_ok") { - ScopedFastFlag sff{"LuauTableSubtypingVariance2", true}; - CheckResult result = check(R"( local vec3 = {x = 1, y = 2, z = 3} local vec1 = {x = 1} @@ -1998,8 +1968,6 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "table_insert_should_cope_with_optional_prope TEST_CASE_FIXTURE(BuiltinsFixture, "table_insert_should_cope_with_optional_properties_in_strict") { - ScopedFastFlag sff{"LuauTableSubtypingVariance2", true}; - CheckResult result = check(R"( --!strict local buttons = {} @@ -2013,8 +1981,6 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "table_insert_should_cope_with_optional_prope TEST_CASE_FIXTURE(Fixture, "error_detailed_prop") { - ScopedFastFlag LuauTableSubtypingVariance2{"LuauTableSubtypingVariance2", true}; // Only for new path - CheckResult result = check(R"( type A = { x: number, y: number } type B = { x: number, y: string } @@ -2031,8 +1997,6 @@ caused by: TEST_CASE_FIXTURE(Fixture, "error_detailed_prop_nested") { - ScopedFastFlag LuauTableSubtypingVariance2{"LuauTableSubtypingVariance2", true}; // Only for new path - CheckResult result = check(R"( type AS = { x: number, y: number } type BS = { x: number, y: string } @@ -2054,11 +2018,6 @@ caused by: TEST_CASE_FIXTURE(BuiltinsFixture, "error_detailed_metatable_prop") { - ScopedFastFlag sff[]{ - {"LuauTableSubtypingVariance2", true}, - {"LuauUnsealedTableLiteral", true}, - }; - CheckResult result = check(R"( local a1 = setmetatable({ x = 2, y = 3 }, { __call = function(s) end }); local b1 = setmetatable({ x = 2, y = "hello" }, { __call = function(s) end }); @@ -2085,8 +2044,6 @@ caused by: TEST_CASE_FIXTURE(Fixture, "error_detailed_indexer_key") { - ScopedFastFlag luauTableSubtypingVariance2{"LuauTableSubtypingVariance2", true}; // Only for new path - CheckResult result = check(R"( type A = { [number]: string } type B = { [string]: string } @@ -2103,8 +2060,6 @@ caused by: TEST_CASE_FIXTURE(Fixture, "error_detailed_indexer_value") { - ScopedFastFlag luauTableSubtypingVariance2{"LuauTableSubtypingVariance2", true}; // Only for new path - CheckResult result = check(R"( type A = { [number]: number } type B = { [number]: string } @@ -2121,10 +2076,6 @@ caused by: TEST_CASE_FIXTURE(Fixture, "explicitly_typed_table") { - ScopedFastFlag sffs[]{ - {"LuauTableSubtypingVariance2", true}, - }; - CheckResult result = check(R"( --!strict type Super = { x : number } @@ -2140,11 +2091,6 @@ a.p = { x = 9 } TEST_CASE_FIXTURE(Fixture, "explicitly_typed_table_error") { - ScopedFastFlag sffs[]{ - {"LuauTableSubtypingVariance2", true}, - {"LuauUnsealedTableLiteral", true}, - }; - CheckResult result = check(R"( --!strict type Super = { x : number } @@ -2166,10 +2112,6 @@ caused by: TEST_CASE_FIXTURE(Fixture, "explicitly_typed_table_with_indexer") { - ScopedFastFlag sffs[]{ - {"LuauTableSubtypingVariance2", true}, - }; - CheckResult result = check(R"( --!strict type Super = { x : number } @@ -2185,10 +2127,6 @@ a.p = { x = 9 } TEST_CASE_FIXTURE(BuiltinsFixture, "recursive_metatable_type_call") { - ScopedFastFlag sff[]{ - {"LuauUnsealedTableLiteral", true}, - }; - CheckResult result = check(R"( local b b = setmetatable({}, {__call = b}) @@ -2201,11 +2139,6 @@ b() TEST_CASE_FIXTURE(Fixture, "table_subtyping_shouldn't_add_optional_properties_to_sealed_tables") { - ScopedFastFlag sffs[] = { - {"LuauTableSubtypingVariance2", true}, - {"LuauSubtypingAddOptPropsToUnsealedTables", true}, - }; - CheckResult result = check(R"( --!strict local function setNumber(t: { p: number? }, x:number) t.p = x end @@ -2706,8 +2639,6 @@ type t0 = any TEST_CASE_FIXTURE(BuiltinsFixture, "instantiate_table_cloning_2") { - ScopedFastFlag sff{"LuauOnlyMutateInstantiatedTables", true}; - CheckResult result = check(R"( type X = T type K = X @@ -2725,8 +2656,6 @@ type K = X TEST_CASE_FIXTURE(Fixture, "instantiate_table_cloning_3") { - ScopedFastFlag sff{"LuauOnlyMutateInstantiatedTables", true}; - CheckResult result = check(R"( type X = T local a = {} @@ -2977,8 +2906,6 @@ TEST_CASE_FIXTURE(Fixture, "mixed_tables_with_implicit_numbered_keys") TEST_CASE_FIXTURE(Fixture, "expected_indexer_value_type_extra") { - ScopedFastFlag luauSubtypingAddOptPropsToUnsealedTables{"LuauSubtypingAddOptPropsToUnsealedTables", true}; - CheckResult result = check(R"( type X = { { x: boolean?, y: boolean? } } diff --git a/tests/TypeInfer.test.cpp b/tests/TypeInfer.test.cpp index 6257cda6..6a048b26 100644 --- a/tests/TypeInfer.test.cpp +++ b/tests/TypeInfer.test.cpp @@ -887,8 +887,6 @@ end TEST_CASE_FIXTURE(Fixture, "cli_50041_committing_txnlog_in_apollo_client_error") { - ScopedFastFlag subtypingVariance{"LuauTableSubtypingVariance2", true}; - CheckResult result = check(R"( --!strict --!nolint @@ -928,7 +926,6 @@ TEST_CASE_FIXTURE(Fixture, "cli_50041_committing_txnlog_in_apollo_client_error") TEST_CASE_FIXTURE(Fixture, "type_infer_recursion_limit_no_ice") { ScopedFastInt sfi("LuauTypeInferRecursionLimit", 2); - ScopedFastFlag sff{"LuauRecursionLimitException", true}; CheckResult result = check(R"( function complex() diff --git a/tests/TypeInfer.unionTypes.test.cpp b/tests/TypeInfer.unionTypes.test.cpp index d19d80cb..2b48133d 100644 --- a/tests/TypeInfer.unionTypes.test.cpp +++ b/tests/TypeInfer.unionTypes.test.cpp @@ -428,12 +428,6 @@ y = x TEST_CASE_FIXTURE(Fixture, "unify_sealed_table_union_check") { - ScopedFastFlag sffs[] = { - {"LuauTableSubtypingVariance2", true}, - {"LuauUnsealedTableLiteral", true}, - {"LuauSubtypingAddOptPropsToUnsealedTables", true}, - }; - CheckResult result = check(R"( -- the difference between this and unify_unsealed_table_union_check is the type annotation on x local t = { x = 3, y = true } diff --git a/tests/VisitTypeVar.test.cpp b/tests/VisitTypeVar.test.cpp index 01960fbe..4fba694a 100644 --- a/tests/VisitTypeVar.test.cpp +++ b/tests/VisitTypeVar.test.cpp @@ -10,14 +10,9 @@ using namespace Luau; LUAU_FASTINT(LuauVisitRecursionLimit) -struct VisitTypeVarFixture : Fixture -{ - ScopedFastFlag flag2 = {"LuauRecursionLimitException", true}; -}; - TEST_SUITE_BEGIN("VisitTypeVar"); -TEST_CASE_FIXTURE(VisitTypeVarFixture, "throw_when_limit_is_exceeded") +TEST_CASE_FIXTURE(Fixture, "throw_when_limit_is_exceeded") { ScopedFastInt sfi{"LuauVisitRecursionLimit", 3}; @@ -30,7 +25,7 @@ TEST_CASE_FIXTURE(VisitTypeVarFixture, "throw_when_limit_is_exceeded") CHECK_THROWS_AS(toString(tType), RecursionLimitException); } -TEST_CASE_FIXTURE(VisitTypeVarFixture, "dont_throw_when_limit_is_high_enough") +TEST_CASE_FIXTURE(Fixture, "dont_throw_when_limit_is_high_enough") { ScopedFastInt sfi{"LuauVisitRecursionLimit", 8}; From e91d80ee25f17af665b5df244e7e8bd77ff9d4f1 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Thu, 23 Jun 2022 18:56:19 -0700 Subject: [PATCH 286/399] Update compatibility.md (#559) --- docs/_pages/compatibility.md | 24 ++++++++++++------------ 1 file changed, 12 insertions(+), 12 deletions(-) diff --git a/docs/_pages/compatibility.md b/docs/_pages/compatibility.md index cdeb4fad..d1686c2f 100644 --- a/docs/_pages/compatibility.md +++ b/docs/_pages/compatibility.md @@ -49,20 +49,20 @@ Sandboxing challenges are [covered in the dedicated section](sandbox). |---------|--------|------| | yieldable pcall/xpcall | ✔️ | | | yieldable metamethods | ❌ | significant performance implications | -| ephemeron tables | ❌ | this complicates the garbage collector esp. for large weak tables | -| emergency garbage collector | ❌ | Luau runs in environments where handling memory exhaustion in emergency situations is not tenable | +| ephemeron tables | ❌ | this complicates and slows down the garbage collector esp. for large weak tables | +| emergency garbage collector | 🤷‍ | Luau runs in environments where handling memory exhaustion in emergency situations is not tenable | | goto statement | ❌ | this complicates the compiler, makes control flow unstructured and doesn't address a significant need | | finalizers for tables | ❌ | no `__gc` support due to sandboxing and performance/complexity | | no more fenv for threads or functions | 😞 | we love this, but it breaks compatibility | | tables honor the `__len` metamethod | 🤷‍♀️ | performance implications, no strong use cases | hex and `\z` escapes in strings | ✔️ | | | support for hexadecimal floats | 🤷‍♀️ | no strong use cases | -| order metamethods work for different types | ❌ | no strong use cases and more complicated semantics + compat | +| order metamethods work for different types | ❌ | no strong use cases and more complicated semantics, compatibility and performance implications | | empty statement | 🤷‍♀️ | less useful in Lua than in JS/C#/C/C++ | -| `break` statement may appear in the middle of a block | 🤷‍♀️ | we'd like to do it for return/continue as well but there be dragons | +| `break` statement may appear in the middle of a block | 🤷‍♀️ | we'd like to do it consistently for `break`/`return`/`continue` but there be dragons | | arguments for function called through `xpcall` | ✔️ | | | optional base in `math.log` | ✔️ | | -| optional separator in `string.rep` | 🤷‍♀️ | no real use cases | +| optional separator in `string.rep` | 🤷‍♀️ | no strong use cases | | new metamethods `__pairs` and `__ipairs` | ❌ | superseded by `__iter` | | frontier patterns | ✔️ | | | `%g` in patterns | ✔️ | | @@ -83,7 +83,7 @@ Ephemeron tables may be implemented at some point since they do have valid uses |---------|--------|------| | `\u` escapes in strings | ✔️ | | | integers (64-bit by default) | ❌ | backwards compatibility and performance implications | -| bitwise operators | ❌ | `bit32` library covers this | +| bitwise operators | ❌ | `bit32` library covers this in absence of 64-bit integers | | basic utf-8 support | ✔️ | we include `utf8` library and other UTF8 features | | functions for packing and unpacking values (string.pack/unpack/packsize) | ✔️ | | | floor division | ❌ | no strong use cases, syntax overlaps with C comments | @@ -95,16 +95,16 @@ Ephemeron tables may be implemented at some point since they do have valid uses It's important to highlight integer support and bitwise operators. For Luau, it's rare that a full 64-bit integer type is necessary - double-precision types support integers up to 2^53 (in Lua which is used in embedded space, integers may be more appealing in environments without a native 64-bit FPU). However, there's a *lot* of value in having a single number type, both from performance perspective and for consistency. Notably, Lua doesn't handle integer overflow properly, so using integers also carries compatibility implications. -If integers are taken out of the equation, bitwise operators make much less sense; additionally, `bit32` library is more fully featured (includes commonly used operations such as rotates and arithmetic shift; bit extraction/replacement is also more readable). Adding operators along with metamethods for all of them increases complexity, which means this feature isn't worth it on the balance. +If integers are taken out of the equation, bitwise operators make less sense, as integers aren't a first class feature; additionally, `bit32` library is more fully featured (includes commonly used operations such as rotates and arithmetic shift; bit extraction/replacement is also more readable). Adding operators along with metamethods for all of them increases complexity, which means this feature isn't worth it on the balance. Common arguments for this include a more familiar syntax, which, while true, gets more nuanced as `^` isn't available as a xor operator, and arithmetic right shift isn't expressible without yet another operator, and performance, which in Luau is substantially better than in Lua because `bit32` library uses VM builtins instead of expensive function calls. -Floor division is less harmful, but it's used rarely enough that `math.floor(a/b)` seems like an adequate replacement; additionally, `//` is a comment in C-derived languages and we may decide to adopt it in addition to `--` at some point. +Floor division is much less complex, but it's used rarely enough that `math.floor(a/b)` seems like an adequate replacement; additionally, `//` is a comment in C-derived languages and we may decide to adopt it in addition to `--` at some point. ## Lua 5.4 | feature | status | notes | |--|--|--| | new generational mode for garbage collection | 🔜 | we're working on gc optimizations and generational mode is on our radar -| to-be-closed variables | ❌ | the syntax is ugly and inconsistent with how we'd like to do attributes long-term; no strong use cases in our domain | +| to-be-closed variables | ❌ | the syntax is inconsistent with how we'd like to do attributes long-term; no strong use cases in our domain | | const variables | ❌ | while there's some demand for const variables, we'd never adopt this syntax | | new implementation for math.random | ✔️ | our RNG is based on PCG, unlike Lua 5.4 which uses Xoroshiro | | optional `init` argument to `string.gmatch` | 🤷‍♀️ | no strong use cases | @@ -112,14 +112,14 @@ Floor division is less harmful, but it's used rarely enough that `math.floor(a/b | coercions string-to-number moved to the string library | 😞 | we love this, but it breaks compatibility | | new format `%p` in `string.format` | 🤷‍♀️ | no strong use cases | | `utf8` library accepts codepoints up to 2^31 | 🤷‍♀️ | no strong use cases | -| The use of the `__lt` metamethod to emulate `__le` has been removed | 😞 | breaks compatibility and doesn't seem very interesting otherwise | +| The use of the `__lt` metamethod to emulate `__le` has been removed | ❌ | breaks compatibility and complicates comparison overloading story | | When finalizing objects, Lua will call `__gc` metamethods that are not functions | ❌ | no `__gc` support due to sandboxing and performance/complexity | | The function print calls `__tostring` instead of tostring to format its arguments. | ✔️ | | | By default, the decoding functions in the utf8 library do not accept surrogates. | 😞 | breaks compatibility and doesn't seem very interesting otherwise | -Lua has a beautiful syntax and frankly we're disappointed in the ``/`` which takes away from that beauty. Taking syntax aside, `` isn't very useful in Luau - its dominant use case is for code that works with external resources like files or sockets, but we don't provide such APIs - and has a very large complexity cost, evidences by a lot of bug fixes since the initial implementation in 5.4 work versions. `` in Luau doesn't matter for performance - our multi-pass compiler is already able to analyze the usage of the variable to know if it's modified or not and extract all performance gains from it - so the only use here is for code readability, where the `` syntax is... suboptimal. +Taking syntax aside (which doesn't feel idiomatic or beautiful), `` isn't very useful in Luau - its dominant use case is for code that works with external resources like files or sockets, but we don't provide such APIs - and has a very large complexity cost, evidences by a lot of bug fixes since the initial implementation in 5.4 work versions. `` in Luau doesn't matter for performance - our multi-pass compiler is already able to analyze the usage of the variable to know if it's modified or not and extract all performance gains from it - so the only use here is for code readability, where the `` syntax is... suboptimal. -If we do end up introducing const variables, it would be through a `const var = value` syntax, which is backwards compatible through a context-sensitive keyword similar to `type`. +If we do end up introducing const variables, it would be through a `const var = value` syntax, which is backwards compatible through a context-sensitive keyword similar to `type`. That said, there's ambiguity wrt whether `const` should simply behave like a read-only variable, ala JavaScript, or if it should represent a stronger contract, for example by limiting the expressions on the right hand side to ones compiler can evaluate ahead of time, or by freezing table values and thus guaranteeing immutability. ## Differences from Lua From 5e405b58b3c7889de9ab9feaee34a254cef775d1 Mon Sep 17 00:00:00 2001 From: Allan N Jeremy Date: Fri, 24 Jun 2022 19:46:29 +0300 Subject: [PATCH 287/399] Added multi-os runners for benchmark & implemented luau analyze (#542) --- .github/workflows/benchmark.yml | 217 ++++- bench/measure_time.py | 43 + bench/static_analysis/LuauPolyfillMap.lua | 962 ++++++++++++++++++++++ scripts/run-with-cachegrind.sh | 9 +- 4 files changed, 1217 insertions(+), 14 deletions(-) create mode 100644 bench/measure_time.py create mode 100644 bench/static_analysis/LuauPolyfillMap.lua diff --git a/.github/workflows/benchmark.yml b/.github/workflows/benchmark.yml index 68f63006..d4ac82ad 100644 --- a/.github/workflows/benchmark.yml +++ b/.github/workflows/benchmark.yml @@ -4,7 +4,6 @@ on: push: branches: - master - paths-ignore: - "docs/**" - "papers/**" @@ -13,12 +12,13 @@ on: - "prototyping/**" jobs: - benchmarks-run: - name: Run ${{ matrix.bench.title }} + windows: + name: Run ${{ matrix.bench.title }} (Windows ${{matrix.arch}}) strategy: fail-fast: false matrix: - os: [ubuntu-latest] + os: [windows-latest] + arch: [Win32, x64] bench: - { script: "run-benchmarks", @@ -32,7 +32,93 @@ jobs: runs-on: ${{ matrix.os }} steps: - - name: Checkout Luau + - name: Checkout Luau repository + uses: actions/checkout@v3 + + - name: Build Luau + shell: bash # necessary for fail-fast + run: | + mkdir build && cd build + cmake .. -DCMAKE_BUILD_TYPE=Release + cmake --build . --target Luau.Repl.CLI --config Release + cmake --build . --target Luau.Analyze.CLI --config Release + + - name: Move build files to root + run: | + move build/RelWithDebInfo/* . + + - name: Check dir structure + run: | + ls build/RelWithDebInfo + ls + - uses: actions/setup-python@v3 + with: + python-version: "3.9" + architecture: "x64" + + - name: Install python dependencies + run: | + python -m pip install requests + python -m pip install --user numpy scipy matplotlib ipython jupyter pandas sympy nose + + - name: Run benchmark + run: | + python bench/bench.py | tee ${{ matrix.bench.script }}-output.txt + + - name: Checkout Benchmark Results repository + uses: actions/checkout@v3 + with: + repository: ${{ matrix.benchResultsRepo.name }} + ref: ${{ matrix.benchResultsRepo.branch }} + token: ${{ secrets.BENCH_GITHUB_TOKEN }} + path: "./gh-pages" + + - name: Store ${{ matrix.bench.title }} result + uses: Roblox/rhysd-github-action-benchmark@v-luau + with: + name: ${{ matrix.bench.title }} (Windows ${{matrix.arch}}) + tool: "benchmarkluau" + output-file-path: ./${{ matrix.bench.script }}-output.txt + external-data-json-path: ./gh-pages/dev/bench/data.json + alert-threshold: 150% + fail-threshold: 200% + fail-on-alert: true + comment-on-alert: true + comment-always: true + github-token: ${{ secrets.GITHUB_TOKEN }} + + - name: Push benchmark results + if: github.event_name == 'push' + run: | + echo "Pushing benchmark results..." + cd gh-pages + git config user.name github-actions + git config user.email github@users.noreply.github.com + git add ./dev/bench/data.json + git commit -m "Add benchmarks results for ${{ github.sha }}" + git push + cd .. + + unix: + name: Run ${{ matrix.bench.title }} (${{ matrix.os}}) + strategy: + fail-fast: false + matrix: + os: [ubuntu-latest, macos-latest] + bench: + - { + script: "run-benchmarks", + timeout: 12, + title: "Luau Benchmarks", + cachegrindTitle: "Performance", + cachegrindIterCount: 20, + } + benchResultsRepo: + - { name: "luau-lang/benchmark-data", branch: "main" } + + runs-on: ${{ matrix.os }} + steps: + - name: Checkout Luau repository uses: actions/checkout@v3 - name: Build Luau @@ -48,18 +134,21 @@ jobs: python -m pip install requests python -m pip install --user numpy scipy matplotlib ipython jupyter pandas sympy nose - - name: Install valgrind - run: | - sudo apt-get install valgrind - - name: Run benchmark run: | python bench/bench.py | tee ${{ matrix.bench.script }}-output.txt + - name: Install valgrind + if: matrix.os == 'ubuntu-latest' + run: | + sudo apt-get install valgrind + - name: Run ${{ matrix.bench.title }} (Cold Cachegrind) + if: matrix.os == 'ubuntu-latest' run: sudo bash ./scripts/run-with-cachegrind.sh python ./bench/bench.py "${{ matrix.bench.cachegrindTitle}}Cold" 1 | tee -a ${{ matrix.bench.script }}-output.txt - name: Run ${{ matrix.bench.title }} (Warm Cachegrind) + if: matrix.os == 'ubuntu-latest' run: sudo bash ./scripts/run-with-cachegrind.sh python ./bench/bench.py "${{ matrix.bench.cachegrindTitle }}" ${{ matrix.bench.cachegrindIterCount }} | tee -a ${{ matrix.bench.script }}-output.txt - name: Checkout Benchmark Results repository @@ -78,12 +167,14 @@ jobs: output-file-path: ./${{ matrix.bench.script }}-output.txt external-data-json-path: ./gh-pages/dev/bench/data.json alert-threshold: 150% - fail-threshold: 1000% - fail-on-alert: false + fail-threshold: 200% + fail-on-alert: true comment-on-alert: true + comment-always: true github-token: ${{ secrets.GITHUB_TOKEN }} - - name: Store ${{ matrix.bench.title }} result + - name: Store ${{ matrix.bench.title }} result (CacheGrind) + if: matrix.os == 'ubuntu-latest' uses: Roblox/rhysd-github-action-benchmark@v-luau with: name: ${{ matrix.bench.title }} (CacheGrind) @@ -97,7 +188,107 @@ jobs: github-token: ${{ secrets.GITHUB_TOKEN }} - name: Push benchmark results - + if: github.event_name == 'push' + run: | + echo "Pushing benchmark results..." + cd gh-pages + git config user.name github-actions + git config user.email github@users.noreply.github.com + git add ./dev/bench/data.json + git commit -m "Add benchmarks results for ${{ github.sha }}" + git push + cd .. + + static-analysis: + name: Run ${{ matrix.bench.title }} + strategy: + fail-fast: false + matrix: + os: [ubuntu-latest] + engine: + - { channel: stable, version: latest } + bench: + - { + script: "run-analyze", + timeout: 12, + title: "Luau Analyze", + cachegrindTitle: "Performance", + cachegrindIterCount: 20, + } + benchResultsRepo: + - { name: "luau-lang/benchmark-data", branch: "main" } + runs-on: ${{ matrix.os }} + steps: + - uses: actions/checkout@v3 + with: + token: "${{ secrets.BENCH_GITHUB_TOKEN }}" + + - name: Build Luau + run: make config=release luau luau-analyze + + - uses: actions/setup-python@v4 + with: + python-version: "3.9" + architecture: "x64" + + - name: Install python dependencies + run: | + sudo pip install requests numpy scipy matplotlib ipython jupyter pandas sympy nose + + - name: Install valgrind + run: | + sudo apt-get install valgrind + + - name: Run Luau Analyze on static file + run: sudo python ./bench/measure_time.py ./build/release/luau-analyze bench/static_analysis/LuauPolyfillMap.lua | tee ${{ matrix.bench.script }}-output.txt + + - name: Run ${{ matrix.bench.title }} (Cold Cachegrind) + run: sudo ./scripts/run-with-cachegrind.sh python ./bench/measure_time.py "${{ matrix.bench.cachegrindTitle}}Cold" 1 ./build/release/luau-analyze bench/static_analysis/LuauPolyfillMap.lua | tee -a ${{ matrix.bench.script }}-output.txt + + - name: Run ${{ matrix.bench.title }} (Warm Cachegrind) + run: sudo bash ./scripts/run-with-cachegrind.sh python ./bench/measure_time.py "${{ matrix.bench.cachegrindTitle}}" 1 ./build/release/luau-analyze bench/static_analysis/LuauPolyfillMap.lua | tee -a ${{ matrix.bench.script }}-output.txt + + - name: Checkout Benchmark Results repository + uses: actions/checkout@v3 + with: + repository: ${{ matrix.benchResultsRepo.name }} + ref: ${{ matrix.benchResultsRepo.branch }} + token: ${{ secrets.BENCH_GITHUB_TOKEN }} + path: "./gh-pages" + + - name: Store ${{ matrix.bench.title }} result + uses: Roblox/rhysd-github-action-benchmark@v-luau + with: + name: ${{ matrix.bench.title }} + tool: "benchmarkluau" + + gh-pages-branch: "main" + output-file-path: ./${{ matrix.bench.script }}-output.txt + external-data-json-path: ./gh-pages/dev/bench/data.json + alert-threshold: 150% + fail-threshold: 200% + fail-on-alert: true + comment-on-alert: true + comment-always: true + github-token: ${{ secrets.GITHUB_TOKEN }} + + - name: Store ${{ matrix.bench.title }} result (CacheGrind) + uses: Roblox/rhysd-github-action-benchmark@v-luau + with: + name: ${{ matrix.bench.title }} + tool: "roblox" + gh-pages-branch: "main" + output-file-path: ./${{ matrix.bench.script }}-output.txt + external-data-json-path: ./gh-pages/dev/bench/data.json + alert-threshold: 150% + fail-threshold: 200% + fail-on-alert: true + comment-on-alert: true + comment-always: true + github-token: ${{ secrets.GITHUB_TOKEN }} + + - name: Push benchmark results + if: github.event_name == 'push' run: | echo "Pushing benchmark results..." cd gh-pages diff --git a/bench/measure_time.py b/bench/measure_time.py new file mode 100644 index 00000000..c41c7d2c --- /dev/null +++ b/bench/measure_time.py @@ -0,0 +1,43 @@ +# This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +import os, sys, time, numpy + +try: + import scipy + from scipy import mean, stats +except ModuleNotFoundError: + print("Warning: scipy package is not installed, confidence values will not be available") + stats = None + +duration_list = [] + +DEFAULT_CYCLES_TO_RUN = 100 +cycles_to_run = DEFAULT_CYCLES_TO_RUN + +try: + cycles_to_run = sys.argv[3] if sys.argv[3] else DEFAULT_CYCLES_TO_RUN + cycles_to_run = int(cycles_to_run) +except IndexError: + pass +except (ValueError, TypeError): + cycles_to_run = DEFAULT_CYCLES_TO_RUN + print("Error: Cycles to run argument must be an integer. Using default value of {}".format(DEFAULT_CYCLES_TO_RUN)) + +# Numpy complains if we provide a cycle count of less than 3 ~ default to 3 whenever a lower value is provided +cycles_to_run = cycles_to_run if cycles_to_run > 2 else 3 + +for i in range(1,cycles_to_run): + start = time.perf_counter() + + # Run the code you want to measure here + os.system(sys.argv[1]) + + end = time.perf_counter() + + duration_ms = (end - start) * 1000 + duration_list.append(duration_ms) + +# Stats +mean = numpy.mean(duration_list) +std_err = stats.sem(duration_list) + +print("SUCCESS: {} : {:.2f}ms +/- {:.2f}% on luau ".format('duration', mean,std_err)) diff --git a/bench/static_analysis/LuauPolyfillMap.lua b/bench/static_analysis/LuauPolyfillMap.lua new file mode 100644 index 00000000..1cfd0181 --- /dev/null +++ b/bench/static_analysis/LuauPolyfillMap.lua @@ -0,0 +1,962 @@ +-- This file is part of the Roblox luau-polyfill repository and is licensed under MIT License; see LICENSE.txt for details +--!nonstrict +-- #region Array +-- Array related +local Array = {} +local Object = {} +local Map = {} + +type Array = { [number]: T } +type callbackFn = (element: V, key: K, map: Map) -> () +type callbackFnWithThisArg = (thisArg: Object, value: V, key: K, map: Map) -> () +type Map = { + size: number, + -- method definitions + set: (self: Map, K, V) -> Map, + get: (self: Map, K) -> V | nil, + clear: (self: Map) -> (), + delete: (self: Map, K) -> boolean, + forEach: (self: Map, callback: callbackFn | callbackFnWithThisArg, thisArg: Object?) -> (), + has: (self: Map, K) -> boolean, + keys: (self: Map) -> Array, + values: (self: Map) -> Array, + entries: (self: Map) -> Array>, + ipairs: (self: Map) -> any, + [K]: V, + _map: { [K]: V }, + _array: { [number]: K }, +} +type mapFn = (element: T, index: number) -> U +type mapFnWithThisArg = (thisArg: any, element: T, index: number) -> U +type Object = { [string]: any } +type Table = { [T]: V } +type Tuple = Array + +local Set = {} + +-- #region Array +function Array.isArray(value: any): boolean + if typeof(value) ~= "table" then + return false + end + if next(value) == nil then + -- an empty table is an empty array + return true + end + + local length = #value + + if length == 0 then + return false + end + + local count = 0 + local sum = 0 + for key in pairs(value) do + if typeof(key) ~= "number" then + return false + end + if key % 1 ~= 0 or key < 1 then + return false + end + count += 1 + sum += key + end + + return sum == (count * (count + 1) / 2) +end + +function Array.from( + value: string | Array | Object, + mapFn: (mapFn | mapFnWithThisArg)?, + thisArg: Object? +): Array + if value == nil then + error("cannot create array from a nil value") + end + local valueType = typeof(value) + + local array = {} + + if valueType == "table" and Array.isArray(value) then + if mapFn then + for i = 1, #(value :: Array) do + if thisArg ~= nil then + array[i] = (mapFn :: mapFnWithThisArg)(thisArg, (value :: Array)[i], i) + else + array[i] = (mapFn :: mapFn)((value :: Array)[i], i) + end + end + else + for i = 1, #(value :: Array) do + array[i] = (value :: Array)[i] + end + end + elseif instanceOf(value, Set) then + if mapFn then + for i, v in (value :: any):ipairs() do + if thisArg ~= nil then + array[i] = (mapFn :: mapFnWithThisArg)(thisArg, v, i) + else + array[i] = (mapFn :: mapFn)(v, i) + end + end + else + for i, v in (value :: any):ipairs() do + array[i] = v + end + end + elseif instanceOf(value, Map) then + if mapFn then + for i, v in (value :: any):ipairs() do + if thisArg ~= nil then + array[i] = (mapFn :: mapFnWithThisArg)(thisArg, v, i) + else + array[i] = (mapFn :: mapFn)(v, i) + end + end + else + for i, v in (value :: any):ipairs() do + array[i] = v + end + end + elseif valueType == "string" then + if mapFn then + for i = 1, (value :: string):len() do + if thisArg ~= nil then + array[i] = (mapFn :: mapFnWithThisArg)(thisArg, (value :: any):sub(i, i), i) + else + array[i] = (mapFn :: mapFn)((value :: any):sub(i, i), i) + end + end + else + for i = 1, (value :: string):len() do + array[i] = (value :: any):sub(i, i) + end + end + end + + return array +end + +type callbackFnArrayMap = (element: T, index: number, array: Array) -> U +type callbackFnWithThisArgArrayMap = (thisArg: V, element: T, index: number, array: Array) -> U + +-- Implements Javascript's `Array.prototype.map` as defined below +-- https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/map +function Array.map( + t: Array, + callback: callbackFnArrayMap | callbackFnWithThisArgArrayMap, + thisArg: V? +): Array + if typeof(t) ~= "table" then + error(string.format("Array.map called on %s", typeof(t))) + end + if typeof(callback) ~= "function" then + error("callback is not a function") + end + + local len = #t + local A = {} + local k = 1 + + while k <= len do + local kValue = t[k] + + if kValue ~= nil then + local mappedValue + + if thisArg ~= nil then + mappedValue = (callback :: callbackFnWithThisArgArrayMap)(thisArg, kValue, k, t) + else + mappedValue = (callback :: callbackFnArrayMap)(kValue, k, t) + end + + A[k] = mappedValue + end + k += 1 + end + + return A +end + +type Function = (any, any, number, any) -> any + +-- Implements Javascript's `Array.prototype.reduce` as defined below +-- https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/reduce +function Array.reduce(array: Array, callback: Function, initialValue: any?): any + if typeof(array) ~= "table" then + error(string.format("Array.reduce called on %s", typeof(array))) + end + if typeof(callback) ~= "function" then + error("callback is not a function") + end + + local length = #array + + local value + local initial = 1 + + if initialValue ~= nil then + value = initialValue + else + initial = 2 + if length == 0 then + error("reduce of empty array with no initial value") + end + value = array[1] + end + + for i = initial, length do + value = callback(value, array[i], i, array) + end + + return value +end + +type callbackFnArrayForEach = (element: T, index: number, array: Array) -> () +type callbackFnWithThisArgArrayForEach = (thisArg: U, element: T, index: number, array: Array) -> () + +-- Implements Javascript's `Array.prototype.forEach` as defined below +-- https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/forEach +function Array.forEach( + t: Array, + callback: callbackFnArrayForEach | callbackFnWithThisArgArrayForEach, + thisArg: U? +): () + if typeof(t) ~= "table" then + error(string.format("Array.forEach called on %s", typeof(t))) + end + if typeof(callback) ~= "function" then + error("callback is not a function") + end + + local len = #t + local k = 1 + + while k <= len do + local kValue = t[k] + + if thisArg ~= nil then + (callback :: callbackFnWithThisArgArrayForEach)(thisArg, kValue, k, t) + else + (callback :: callbackFnArrayForEach)(kValue, k, t) + end + + if #t < len then + -- don't iterate on removed items, don't iterate more than original length + len = #t + end + k += 1 + end +end +-- #endregion + +-- #region Set +Set.__index = Set + +type callbackFnSet = (value: T, key: T, set: Set) -> () +type callbackFnWithThisArgSet = (thisArg: Object, value: T, key: T, set: Set) -> () + +export type Set = { + size: number, + -- method definitions + add: (self: Set, T) -> Set, + clear: (self: Set) -> (), + delete: (self: Set, T) -> boolean, + forEach: (self: Set, callback: callbackFnSet | callbackFnWithThisArgSet, thisArg: Object?) -> (), + has: (self: Set, T) -> boolean, + ipairs: (self: Set) -> any, +} + +type Iterable = { ipairs: (any) -> any } + +function Set.new(iterable: Array | Set | Iterable | string | nil): Set + local array = {} + local map = {} + if iterable ~= nil then + local arrayIterable: Array + -- ROBLOX TODO: remove type casting from (iterable :: any).ipairs in next release + if typeof(iterable) == "table" then + if Array.isArray(iterable) then + arrayIterable = Array.from(iterable :: Array) + elseif typeof((iterable :: Iterable).ipairs) == "function" then + -- handle in loop below + elseif _G.__DEV__ then + error("cannot create array from an object-like table") + end + elseif typeof(iterable) == "string" then + arrayIterable = Array.from(iterable :: string) + else + error(("cannot create array from value of type `%s`"):format(typeof(iterable))) + end + + if arrayIterable then + for _, element in ipairs(arrayIterable) do + if not map[element] then + map[element] = true + table.insert(array, element) + end + end + elseif typeof(iterable) == "table" and typeof((iterable :: Iterable).ipairs) == "function" then + for _, element in (iterable :: Iterable):ipairs() do + if not map[element] then + map[element] = true + table.insert(array, element) + end + end + end + end + + return (setmetatable({ + size = #array, + _map = map, + _array = array, + }, Set) :: any) :: Set +end + +function Set:add(value) + if not self._map[value] then + -- Luau FIXME: analyze should know self is Set which includes size as a number + self.size = self.size :: number + 1 + self._map[value] = true + table.insert(self._array, value) + end + return self +end + +function Set:clear() + self.size = 0 + table.clear(self._map) + table.clear(self._array) +end + +function Set:delete(value): boolean + if not self._map[value] then + return false + end + -- Luau FIXME: analyze should know self is Map which includes size as a number + self.size = self.size :: number - 1 + self._map[value] = nil + local index = table.find(self._array, value) + if index then + table.remove(self._array, index) + end + return true +end + +-- Implements Javascript's `Map.prototype.forEach` as defined below +-- https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Set/forEach +function Set:forEach(callback: callbackFnSet | callbackFnWithThisArgSet, thisArg: Object?): () + if typeof(callback) ~= "function" then + error("callback is not a function") + end + + return Array.forEach(self._array, function(value: T) + if thisArg ~= nil then + (callback :: callbackFnWithThisArgSet)(thisArg, value, value, self) + else + (callback :: callbackFnSet)(value, value, self) + end + end) +end + +function Set:has(value): boolean + return self._map[value] ~= nil +end + +function Set:ipairs() + return ipairs(self._array) +end + +-- #endregion Set + +-- #region Object +function Object.entries(value: string | Object | Array): Array + assert(value :: any ~= nil, "cannot get entries from a nil value") + local valueType = typeof(value) + + local entries: Array> = {} + if valueType == "table" then + for key, keyValue in pairs(value :: Object) do + -- Luau FIXME: Luau should see entries as Array, given object is [string]: any, but it sees it as Array> despite all the manual annotation + table.insert(entries, { key :: string, keyValue :: any }) + end + elseif valueType == "string" then + for i = 1, string.len(value :: string) do + entries[i] = { tostring(i), string.sub(value :: string, i, i) } + end + end + + return entries +end + +-- #endregion + +-- #region instanceOf + +-- ROBLOX note: Typed tbl as any to work with strict type analyze +-- polyfill for https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/instanceof +function instanceOf(tbl: any, class) + assert(typeof(class) == "table", "Received a non-table as the second argument for instanceof") + + if typeof(tbl) ~= "table" then + return false + end + + local ok, hasNew = pcall(function() + return class.new ~= nil and tbl.new == class.new + end) + if ok and hasNew then + return true + end + + local seen = { tbl = true } + + while tbl and typeof(tbl) == "table" do + tbl = getmetatable(tbl) + if typeof(tbl) == "table" then + tbl = tbl.__index + + if tbl == class then + return true + end + end + + -- if we still have a valid table then check against seen + if typeof(tbl) == "table" then + if seen[tbl] then + return false + end + seen[tbl] = true + end + end + + return false +end +-- #endregion + +function Map.new(iterable: Array>?): Map + local array = {} + local map = {} + if iterable ~= nil then + local arrayFromIterable + local iterableType = typeof(iterable) + if iterableType == "table" then + if #iterable > 0 and typeof(iterable[1]) ~= "table" then + error("cannot create Map from {K, V} form, it must be { {K, V}... }") + end + + arrayFromIterable = Array.from(iterable) + else + error(("cannot create array from value of type `%s`"):format(iterableType)) + end + + for _, entry in ipairs(arrayFromIterable) do + local key = entry[1] + if _G.__DEV__ then + if key == nil then + error("cannot create Map from a table that isn't an array.") + end + end + local val = entry[2] + -- only add to array if new + if map[key] == nil then + table.insert(array, key) + end + -- always assign + map[key] = val + end + end + + return (setmetatable({ + size = #array, + _map = map, + _array = array, + }, Map) :: any) :: Map +end + +function Map:set(key: K, value: V): Map + -- preserve initial insertion order + if self._map[key] == nil then + -- Luau FIXME: analyze should know self is Map which includes size as a number + self.size = self.size :: number + 1 + table.insert(self._array, key) + end + -- always update value + self._map[key] = value + return self +end + +function Map:get(key) + return self._map[key] +end + +function Map:clear() + local table_: any = table + self.size = 0 + table_.clear(self._map) + table_.clear(self._array) +end + +function Map:delete(key): boolean + if self._map[key] == nil then + return false + end + -- Luau FIXME: analyze should know self is Map which includes size as a number + self.size = self.size :: number - 1 + self._map[key] = nil + local index = table.find(self._array, key) + if index then + table.remove(self._array, index) + end + return true +end + +-- Implements Javascript's `Map.prototype.forEach` as defined below +-- https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Map/forEach +function Map:forEach(callback: callbackFn | callbackFnWithThisArg, thisArg: Object?): () + if typeof(callback) ~= "function" then + error("callback is not a function") + end + + return Array.forEach(self._array, function(key: K) + local value: V = self._map[key] :: V + + if thisArg ~= nil then + (callback :: callbackFnWithThisArg)(thisArg, value, key, self) + else + (callback :: callbackFn)(value, key, self) + end + end) +end + +function Map:has(key): boolean + return self._map[key] ~= nil +end + +function Map:keys() + return self._array +end + +function Map:values() + return Array.map(self._array, function(key) + return self._map[key] + end) +end + +function Map:entries() + return Array.map(self._array, function(key) + return { key, self._map[key] } + end) +end + +function Map:ipairs() + return ipairs(self:entries()) +end + +function Map.__index(self, key) + local mapProp = rawget(Map, key) + if mapProp ~= nil then + return mapProp + end + + return Map.get(self, key) +end + +function Map.__newindex(table_, key, value) + table_:set(key, value) +end + +local function coerceToMap(mapLike: Map | Table): Map + return instanceOf(mapLike, Map) and mapLike :: Map -- ROBLOX: order is preservered + or Map.new(Object.entries(mapLike)) -- ROBLOX: order is not preserved +end + +-- local function coerceToTable(mapLike: Map | Table): Table +-- if not instanceOf(mapLike, Map) then +-- return mapLike +-- end + +-- -- create table from map +-- return Array.reduce(mapLike:entries(), function(tbl, entry) +-- tbl[entry[1]] = entry[2] +-- return tbl +-- end, {}) +-- end + +-- #region Tests to verify it works as expected +local function it(description: string, fn: () -> ()) + local ok, result = pcall(fn) + + if not ok then + error("Failed test: " .. description .. "\n" .. result) + end +end + +local AN_ITEM = "bar" +local ANOTHER_ITEM = "baz" + +-- #region [Describe] "Map" +-- #region [Child Describe] "constructors" +it("creates an empty array", function() + local foo = Map.new() + assert(foo.size == 0) +end) + +it("creates a Map from an array", function() + local foo = Map.new({ + { AN_ITEM, "foo" }, + { ANOTHER_ITEM, "val" }, + }) + assert(foo.size == 2) + assert(foo:has(AN_ITEM) == true) + assert(foo:has(ANOTHER_ITEM) == true) +end) + +it("creates a Map from an array with duplicate keys", function() + local foo = Map.new({ + { AN_ITEM, "foo1" }, + { AN_ITEM, "foo2" }, + }) + assert(foo.size == 1) + assert(foo:get(AN_ITEM) == "foo2") + + assert(#foo:keys() == 1 and foo:keys()[1] == AN_ITEM) + assert(#foo:values() == 1 and foo:values()[1] == "foo2") + assert(#foo:entries() == 1) + assert(#foo:entries()[1] == 2) + + assert(foo:entries()[1][1] == AN_ITEM) + assert(foo:entries()[1][2] == "foo2") +end) + +it("preserves the order of keys first assignment", function() + local foo = Map.new({ + { AN_ITEM, "foo1" }, + { ANOTHER_ITEM, "bar" }, + { AN_ITEM, "foo2" }, + }) + assert(foo.size == 2) + assert(foo:get(AN_ITEM) == "foo2") + assert(foo:get(ANOTHER_ITEM) == "bar") + + assert(foo:keys()[1] == AN_ITEM) + assert(foo:keys()[2] == ANOTHER_ITEM) + assert(foo:values()[1] == "foo2") + assert(foo:values()[2] == "bar") + assert(foo:entries()[1][1] == AN_ITEM) + assert(foo:entries()[1][2] == "foo2") + assert(foo:entries()[2][1] == ANOTHER_ITEM) + assert(foo:entries()[2][2] == "bar") +end) +-- #endregion + +-- #region [Child Describe] "type" +it("instanceOf return true for an actual Map object", function() + local foo = Map.new() + assert(instanceOf(foo, Map) == true) +end) + +it("instanceOf return false for an regular plain object", function() + local foo = {} + assert(instanceOf(foo, Map) == false) +end) +-- #endregion + +-- #region [Child Describe] "set" +it("returns the Map object", function() + local foo = Map.new() + assert(foo:set(1, "baz") == foo) +end) + +it("increments the size if the element is added for the first time", function() + local foo = Map.new() + foo:set(AN_ITEM, "foo") + assert(foo.size == 1) +end) + +it("does not increment the size the second time an element is added", function() + local foo = Map.new() + foo:set(AN_ITEM, "foo") + foo:set(AN_ITEM, "val") + assert(foo.size == 1) +end) + +it("sets values correctly to true/false", function() + -- Luau FIXME: Luau insists that arrays can't be mixed type + local foo = Map.new({ { AN_ITEM, false :: any } }) + foo:set(AN_ITEM, false) + assert(foo.size == 1) + assert(foo:get(AN_ITEM) == false) + + foo:set(AN_ITEM, true) + assert(foo.size == 1) + assert(foo:get(AN_ITEM) == true) + + foo:set(AN_ITEM, false) + assert(foo.size == 1) + assert(foo:get(AN_ITEM) == false) +end) + +-- #endregion + +-- #region [Child Describe] "get" +it("returns value of item from provided key", function() + local foo = Map.new() + foo:set(AN_ITEM, "foo") + assert(foo:get(AN_ITEM) == "foo") +end) + +it("returns nil if the item is not in the Map", function() + local foo = Map.new() + assert(foo:get(AN_ITEM) == nil) +end) +-- #endregion + +-- #region [Child Describe] "clear" +it("sets the size to zero", function() + local foo = Map.new() + foo:set(AN_ITEM, "foo") + foo:clear() + assert(foo.size == 0) +end) + +it("removes the items from the Map", function() + local foo = Map.new() + foo:set(AN_ITEM, "foo") + foo:clear() + assert(foo:has(AN_ITEM) == false) +end) +-- #endregion + +-- #region [Child Describe] "delete" +it("removes the items from the Map", function() + local foo = Map.new() + foo:set(AN_ITEM, "foo") + foo:delete(AN_ITEM) + assert(foo:has(AN_ITEM) == false) +end) + +it("returns true if the item was in the Map", function() + local foo = Map.new() + foo:set(AN_ITEM, "foo") + assert(foo:delete(AN_ITEM) == true) +end) + +it("returns false if the item was not in the Map", function() + local foo = Map.new() + assert(foo:delete(AN_ITEM) == false) +end) + +it("decrements the size if the item was in the Map", function() + local foo = Map.new() + foo:set(AN_ITEM, "foo") + foo:delete(AN_ITEM) + assert(foo.size == 0) +end) + +it("does not decrement the size if the item was not in the Map", function() + local foo = Map.new() + foo:set(AN_ITEM, "foo") + foo:delete(ANOTHER_ITEM) + assert(foo.size == 1) +end) + +it("deletes value set to false", function() + -- Luau FIXME: Luau insists arrays can't be mixed type + local foo = Map.new({ { AN_ITEM, false :: any } }) + + foo:delete(AN_ITEM) + + assert(foo.size == 0) + assert(foo:get(AN_ITEM) == nil) +end) +-- #endregion + +-- #region [Child Describe] "has" +it("returns true if the item is in the Map", function() + local foo = Map.new() + foo:set(AN_ITEM, "foo") + assert(foo:has(AN_ITEM) == true) +end) + +it("returns false if the item is not in the Map", function() + local foo = Map.new() + assert(foo:has(AN_ITEM) == false) +end) + +it("returns correctly with value set to false", function() + -- Luau FIXME: Luau insists arrays can't be mixed type + local foo = Map.new({ { AN_ITEM, false :: any } }) + + assert(foo:has(AN_ITEM) == true) +end) +-- #endregion + +-- #region [Child Describe] "keys / values / entries" +it("returns array of elements", function() + local myMap = Map.new() + myMap:set(AN_ITEM, "foo") + myMap:set(ANOTHER_ITEM, "val") + + assert(myMap:keys()[1] == AN_ITEM) + assert(myMap:keys()[2] == ANOTHER_ITEM) + + assert(myMap:values()[1] == "foo") + assert(myMap:values()[2] == "val") + + assert(myMap:entries()[1][1] == AN_ITEM) + assert(myMap:entries()[1][2] == "foo") + assert(myMap:entries()[2][1] == ANOTHER_ITEM) + assert(myMap:entries()[2][2] == "val") +end) +-- #endregion + +-- #region [Child Describe] "__index" +it("can access fields directly without using get", function() + local typeName = "size" + + local foo = Map.new({ + { AN_ITEM, "foo" }, + { ANOTHER_ITEM, "val" }, + { typeName, "buzz" }, + }) + + assert(foo.size == 3) + assert(foo[AN_ITEM] == "foo") + assert(foo[ANOTHER_ITEM] == "val") + assert(foo:get(typeName) == "buzz") +end) +-- #endregion + +-- #region [Child Describe] "__newindex" +it("can set fields directly without using set", function() + local foo = Map.new() + + assert(foo.size == 0) + + foo[AN_ITEM] = "foo" + foo[ANOTHER_ITEM] = "val" + foo.fizz = "buzz" + + assert(foo.size == 3) + assert(foo:get(AN_ITEM) == "foo") + assert(foo:get(ANOTHER_ITEM) == "val") + assert(foo:get("fizz") == "buzz") +end) +-- #endregion + +-- #region [Child Describe] "ipairs" +local function makeArray(...) + local array = {} + for _, item in ... do + table.insert(array, item) + end + return array +end + +it("iterates on the elements by their insertion order", function() + local foo = Map.new() + foo:set(AN_ITEM, "foo") + foo:set(ANOTHER_ITEM, "val") + assert(makeArray(foo:ipairs())[1][1] == AN_ITEM) + assert(makeArray(foo:ipairs())[1][2] == "foo") + assert(makeArray(foo:ipairs())[2][1] == ANOTHER_ITEM) + assert(makeArray(foo:ipairs())[2][2] == "val") +end) + +it("does not iterate on removed elements", function() + local foo = Map.new() + foo:set(AN_ITEM, "foo") + foo:set(ANOTHER_ITEM, "val") + foo:delete(AN_ITEM) + assert(makeArray(foo:ipairs())[1][1] == ANOTHER_ITEM) + assert(makeArray(foo:ipairs())[1][2] == "val") +end) + +it("iterates on elements if the added back to the Map", function() + local foo = Map.new() + foo:set(AN_ITEM, "foo") + foo:set(ANOTHER_ITEM, "val") + foo:delete(AN_ITEM) + foo:set(AN_ITEM, "food") + assert(makeArray(foo:ipairs())[1][1] == ANOTHER_ITEM) + assert(makeArray(foo:ipairs())[1][2] == "val") + assert(makeArray(foo:ipairs())[2][1] == AN_ITEM) + assert(makeArray(foo:ipairs())[2][2] == "food") +end) +-- #endregion + +-- #region [Child Describe] "Integration Tests" +-- it("MDN Examples", function() +-- local myMap = Map.new() :: Map + +-- local keyString = "a string" +-- local keyObj = {} +-- local keyFunc = function() end + +-- -- setting the values +-- myMap:set(keyString, "value associated with 'a string'") +-- myMap:set(keyObj, "value associated with keyObj") +-- myMap:set(keyFunc, "value associated with keyFunc") + +-- assert(myMap.size == 3) + +-- -- getting the values +-- assert(myMap:get(keyString) == "value associated with 'a string'") +-- assert(myMap:get(keyObj) == "value associated with keyObj") +-- assert(myMap:get(keyFunc) == "value associated with keyFunc") + +-- assert(myMap:get("a string") == "value associated with 'a string'") + +-- assert(myMap:get({}) == nil) -- nil, because keyObj !== {} +-- assert(myMap:get(function() -- nil because keyFunc !== function () {} +-- end) == nil) +-- end) + +it("handles non-traditional keys", function() + local myMap = Map.new() :: Map + + local falseKey = false + local trueKey = true + local negativeKey = -1 + local emptyKey = "" + + myMap:set(falseKey, "apple") + myMap:set(trueKey, "bear") + myMap:set(negativeKey, "corgi") + myMap:set(emptyKey, "doge") + + assert(myMap.size == 4) + + assert(myMap:get(falseKey) == "apple") + assert(myMap:get(trueKey) == "bear") + assert(myMap:get(negativeKey) == "corgi") + assert(myMap:get(emptyKey) == "doge") + + myMap:delete(falseKey) + myMap:delete(trueKey) + myMap:delete(negativeKey) + myMap:delete(emptyKey) + + assert(myMap.size == 0) +end) +-- #endregion + +-- #endregion [Describe] "Map" + +-- #region [Describe] "coerceToMap" +it("returns the same object if instance of Map", function() + local map = Map.new() + assert(coerceToMap(map) == map) + + map = Map.new({}) + assert(coerceToMap(map) == map) + + map = Map.new({ { AN_ITEM, "foo" } }) + assert(coerceToMap(map) == map) +end) +-- #endregion [Describe] "coerceToMap" + +-- #endregion Tests to verify it works as expected diff --git a/scripts/run-with-cachegrind.sh b/scripts/run-with-cachegrind.sh index eb4a8c3f..787043ff 100644 --- a/scripts/run-with-cachegrind.sh +++ b/scripts/run-with-cachegrind.sh @@ -25,10 +25,17 @@ now_ms() { ITERATION_COUNT=$4 START_TIME=$(now_ms) +ARGS=( "$@" ) +REST_ARGS="${ARGS[@]:4}" + valgrind \ --quiet \ --tool=cachegrind \ - "$1" "$2" >/dev/null + "$1" "$2" $REST_ARGS>/dev/null + +ARGS=( "$@" ) +REST_ARGS="${ARGS[@]:4}" + TIME_ELAPSED=$(bc <<< "$(now_ms) - ${START_TIME}") From 224d35bc9e151149f256c90e5ecc0790b08e8d0b Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Fri, 24 Jun 2022 18:16:12 -0700 Subject: [PATCH 288/399] Update benchmark.yml Attempt to fix Windows and other builds --- .github/workflows/benchmark.yml | 16 +++++----------- 1 file changed, 5 insertions(+), 11 deletions(-) diff --git a/.github/workflows/benchmark.yml b/.github/workflows/benchmark.yml index d4ac82ad..20a51b00 100644 --- a/.github/workflows/benchmark.yml +++ b/.github/workflows/benchmark.yml @@ -45,12 +45,8 @@ jobs: - name: Move build files to root run: | - move build/RelWithDebInfo/* . + move build/Release/* . - - name: Check dir structure - run: | - ls build/RelWithDebInfo - ls - uses: actions/setup-python@v3 with: python-version: "3.9" @@ -171,7 +167,7 @@ jobs: fail-on-alert: true comment-on-alert: true comment-always: true - github-token: ${{ secrets.GITHUB_TOKEN }} + github-token: ${{ secrets.BENCH_GITHUB_TOKEN }} - name: Store ${{ matrix.bench.title }} result (CacheGrind) if: matrix.os == 'ubuntu-latest' @@ -185,7 +181,7 @@ jobs: fail-threshold: 1000% fail-on-alert: false comment-on-alert: true - github-token: ${{ secrets.GITHUB_TOKEN }} + github-token: ${{ secrets.BENCH_GITHUB_TOKEN }} - name: Push benchmark results if: github.event_name == 'push' @@ -205,8 +201,6 @@ jobs: fail-fast: false matrix: os: [ubuntu-latest] - engine: - - { channel: stable, version: latest } bench: - { script: "run-analyze", @@ -270,7 +264,7 @@ jobs: fail-on-alert: true comment-on-alert: true comment-always: true - github-token: ${{ secrets.GITHUB_TOKEN }} + github-token: ${{ secrets.BENCH_GITHUB_TOKEN }} - name: Store ${{ matrix.bench.title }} result (CacheGrind) uses: Roblox/rhysd-github-action-benchmark@v-luau @@ -285,7 +279,7 @@ jobs: fail-on-alert: true comment-on-alert: true comment-always: true - github-token: ${{ secrets.GITHUB_TOKEN }} + github-token: ${{ secrets.BENCH_GITHUB_TOKEN }} - name: Push benchmark results if: github.event_name == 'push' From 9846a6c7b9dc9699693d062835eba07f61ff9126 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Fri, 24 Jun 2022 18:26:15 -0700 Subject: [PATCH 289/399] Update benchmark.yml Remove all alert/comment functionality --- .github/workflows/benchmark.yml | 26 -------------------------- 1 file changed, 26 deletions(-) diff --git a/.github/workflows/benchmark.yml b/.github/workflows/benchmark.yml index 20a51b00..60fc6bef 100644 --- a/.github/workflows/benchmark.yml +++ b/.github/workflows/benchmark.yml @@ -1,5 +1,3 @@ -name: Luau Benchmarks - on: push: branches: @@ -76,11 +74,6 @@ jobs: tool: "benchmarkluau" output-file-path: ./${{ matrix.bench.script }}-output.txt external-data-json-path: ./gh-pages/dev/bench/data.json - alert-threshold: 150% - fail-threshold: 200% - fail-on-alert: true - comment-on-alert: true - comment-always: true github-token: ${{ secrets.GITHUB_TOKEN }} - name: Push benchmark results @@ -162,11 +155,6 @@ jobs: tool: "benchmarkluau" output-file-path: ./${{ matrix.bench.script }}-output.txt external-data-json-path: ./gh-pages/dev/bench/data.json - alert-threshold: 150% - fail-threshold: 200% - fail-on-alert: true - comment-on-alert: true - comment-always: true github-token: ${{ secrets.BENCH_GITHUB_TOKEN }} - name: Store ${{ matrix.bench.title }} result (CacheGrind) @@ -177,10 +165,6 @@ jobs: tool: "roblox" output-file-path: ./${{ matrix.bench.script }}-output.txt external-data-json-path: ./gh-pages/dev/bench/data.json - alert-threshold: 150% - fail-threshold: 1000% - fail-on-alert: false - comment-on-alert: true github-token: ${{ secrets.BENCH_GITHUB_TOKEN }} - name: Push benchmark results @@ -259,11 +243,6 @@ jobs: gh-pages-branch: "main" output-file-path: ./${{ matrix.bench.script }}-output.txt external-data-json-path: ./gh-pages/dev/bench/data.json - alert-threshold: 150% - fail-threshold: 200% - fail-on-alert: true - comment-on-alert: true - comment-always: true github-token: ${{ secrets.BENCH_GITHUB_TOKEN }} - name: Store ${{ matrix.bench.title }} result (CacheGrind) @@ -274,11 +253,6 @@ jobs: gh-pages-branch: "main" output-file-path: ./${{ matrix.bench.script }}-output.txt external-data-json-path: ./gh-pages/dev/bench/data.json - alert-threshold: 150% - fail-threshold: 200% - fail-on-alert: true - comment-on-alert: true - comment-always: true github-token: ${{ secrets.BENCH_GITHUB_TOKEN }} - name: Push benchmark results From 4cd0443913ba5af65a61d7f5c46b0e25b79ad7f7 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Fri, 24 Jun 2022 18:30:26 -0700 Subject: [PATCH 290/399] Update benchmark.yml Cleaner names --- .github/workflows/benchmark.yml | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/.github/workflows/benchmark.yml b/.github/workflows/benchmark.yml index 60fc6bef..4df7b2f3 100644 --- a/.github/workflows/benchmark.yml +++ b/.github/workflows/benchmark.yml @@ -1,3 +1,5 @@ +name: benchmark + on: push: branches: @@ -11,7 +13,7 @@ on: jobs: windows: - name: Run ${{ matrix.bench.title }} (Windows ${{matrix.arch}}) + name: windows-${{matrix.arch}} strategy: fail-fast: false matrix: @@ -89,7 +91,7 @@ jobs: cd .. unix: - name: Run ${{ matrix.bench.title }} (${{ matrix.os}}) + name: ${{matrix.os}} strategy: fail-fast: false matrix: @@ -180,7 +182,7 @@ jobs: cd .. static-analysis: - name: Run ${{ matrix.bench.title }} + name: luau-analyze strategy: fail-fast: false matrix: From 13e50a9cac327fc2dda40ca6936fbff8c1206ad6 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Mon, 27 Jun 2022 09:05:50 -0700 Subject: [PATCH 291/399] Update library.md (#564) Clarify behavior of shifts for out of range values. --- docs/_pages/library.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/_pages/library.md b/docs/_pages/library.md index ff3075ac..f419d2bf 100644 --- a/docs/_pages/library.md +++ b/docs/_pages/library.md @@ -647,7 +647,7 @@ All functions in the `bit32` library treat input numbers as 32-bit unsigned inte function bit32.arshift(n: number, i: number): number ``` -Shifts `n` by `i` bits to the right (if `i` is negative, a left shift is performed instead). The most significant bit of `n` is propagated during the shift. +Shifts `n` by `i` bits to the right (if `i` is negative, a left shift is performed instead). The most significant bit of `n` is propagated during the shift. When `i` is larger than 31, returns an integer with all bits set to the sign bit of `n`. When `i` is smaller than `-31`, 0 is returned. ``` function bit32.band(args: ...number): number @@ -695,7 +695,7 @@ Rotates `n` to the left by `i` bits (if `i` is negative, a right rotate is perfo function bit32.lshift(n: number, i: number): number ``` -Shifts `n` to the left by `i` bits (if `i` is negative, a right shift is performed instead). +Shifts `n` to the left by `i` bits (if `i` is negative, a right shift is performed instead). When `i` is outside of `[-31..31]` range, returns 0. ``` function bit32.replace(n: number, r: number, f: number, w: number?): number @@ -713,7 +713,7 @@ Rotates `n` to the right by `i` bits (if `i` is negative, a left rotate is perfo function bit32.rshift(n: number, i: number): number ``` -Shifts `n` to the right by `i` bits (if `i` is negative, a left shift is performed instead). +Shifts `n` to the right by `i` bits (if `i` is negative, a left shift is performed instead). When `i` is outside of `[-31..31]` range, returns 0. ``` function bit32.countlz(n: number): number From fd82e926286765468f39048538c483d0ccae3a73 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Tue, 28 Jun 2022 09:06:59 -0700 Subject: [PATCH 292/399] RFC: Support `__len` metamethod for tables and `rawlen` function (#536) --- rfcs/len-metamethod-rawlen.md | 43 +++++++++++++++++++++++++++++++++++ 1 file changed, 43 insertions(+) create mode 100644 rfcs/len-metamethod-rawlen.md diff --git a/rfcs/len-metamethod-rawlen.md b/rfcs/len-metamethod-rawlen.md new file mode 100644 index 00000000..45284b71 --- /dev/null +++ b/rfcs/len-metamethod-rawlen.md @@ -0,0 +1,43 @@ +# Support `__len` metamethod for tables and `rawlen` function + +## Summary + +`__len` metamethod will be called by `#` operator on tables, matching Lua 5.2 + +## Motivation + +Lua 5.1 invokes `__len` only on userdata objects, whereas Lua 5.2 extends this to tables. In addition to making `__len` metamethod more uniform and making Luau +more compatible with later versions of Lua, this has the important advantage which is that it makes it possible to implement an index based container. + +Before `__iter` and `__len` it was possible to implement a custom container using `__index`/`__newindex`, but to iterate through the container a custom function was +necessary, because Luau didn't support generalized iteration, `__pairs`/`__ipairs` from Lua 5.2, or `#` override. + +With generalized iteration, a custom container can implement its own iteration behavior so as long as code uses `for k,v in obj` iteration style, the container can +be interfaced with the same way as a table. However, when the container uses integer indices, manual iteration via `#` would still not work - which is required for some +more complicated algorithms, or even to simply iterate through the container backwards. + +Supporting `__len` would make it possible to implement a custom integer based container that exposes the same interface as a table does. + +## Design + +`#v` will call `__len` metamethod if the object is a table and the metamethod exists; the result of the metamethod will be returned if it's a number (an error will be raised otherwise). + +`table.` functions that implicitly compute table length, such as `table.getn`, `table.insert`, will continue using the actual table length. This is consistent with the +general policy that Luau doesn't support metamethods in `table.` functions. + +A new function, `rawlen(v)`, will be added to the standard library; given a string or a table, it will return the length of the object without calling any metamethods. +The new function has the previous behavior of `#` operator with the exception of not supporting userdata inputs, as userdata doesn't have an inherent definition of length. + +## Drawbacks + +`#` is an operator that is used frequently and as such an extra metatable check here may impact performance. However, `#` is usually called on tables without metatables, +and even when it is, using the existing metamethod-absence-caching approach we use for many other metamethods a test version of the change to support `__len` shows no +statistically significant difference on existing benchmark suite. This does complicate the `#` computation a little more which may affect JIT as well, but even if the +table doesn't have a metatable the process of computing `#` involves a series of condition checks and as such will likely require slow paths anyway. + +This is technically changing semantics of `#` when called on tables with an existing `__len` metamethod, and as such has a potential to change behavior of an existing valid program. +That said, it's unlikely that any table would have a metatable with `__len` metamethod as outside of userdata it would not anything, and this drawback is not feasible to resolve with any alternate version of the proposal. + +## Alternatives + +Do not implement `__len`. From c29b803046752838b83b5d2e726e53aa9188e6c2 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Tue, 28 Jun 2022 09:08:12 -0700 Subject: [PATCH 293/399] Update STATUS.md Add __len metamethod --- rfcs/STATUS.md | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/rfcs/STATUS.md b/rfcs/STATUS.md index 6bfa865d..23a1be83 100644 --- a/rfcs/STATUS.md +++ b/rfcs/STATUS.md @@ -32,3 +32,9 @@ This document tracks unimplemented RFCs. [RFC: never and unknown types](https://github.com/Roblox/luau/blob/master/rfcs/never-and-unknown-types.md) **Status**: Needs implementation + +## __len metamethod for tables and rawlen function + +[RFC: Support __len metamethod for tables and rawlen function](https://github.com/Roblox/luau/blob/master/rfcs/len-metamethod-rawlen.md) + +**Status**: Needs implementation From ee82f1e9973393c3060b869a969642df873d4575 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Tue, 28 Jun 2022 23:13:13 -0700 Subject: [PATCH 294/399] Update sandbox.md Since we don't have a formal proof, clarify that we don't have known bugs. --- docs/_pages/sandbox.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/_pages/sandbox.md b/docs/_pages/sandbox.md index d1d7d118..a7ed7476 100644 --- a/docs/_pages/sandbox.md +++ b/docs/_pages/sandbox.md @@ -4,11 +4,11 @@ title: Sandboxing toc: true --- -Luau is safe to embed. Broadly speaking, this means that even in the face of untrusted (and in Roblox case, actively malicious) code, the language and the standard library don't allow any unsafe access to the underlying system, and don't have any bugs that allow escaping out of the sandbox (e.g. to gain native code execution through ROP gadgets et al). Additionally, the VM provides extra features to implement isolation of privileged code from unprivileged code and protect one from the other; this is important if the embedding environment decides to expose some APIs that may not be safe to call from untrusted code, for example because they do provide controlled access to the underlying system or risk PII exposure through fingerprinting etc. +Luau is safe to embed. Broadly speaking, this means that even in the face of untrusted (and in Roblox case, actively malicious) code, the language and the standard library don't allow unsafe access to the underlying system, and don't have known bugs that allow escaping out of the sandbox (e.g. to gain native code execution through ROP gadgets et al). Additionally, the VM provides extra features to implement isolation of privileged code from unprivileged code and protect one from the other; this is important if the embedding environment decides to expose some APIs that may not be safe to call from untrusted code, for example because they do provide controlled access to the underlying system or risk PII exposure through fingerprinting etc. This safety is achieved through a combination of removing features from the standard library that are unsafe, adding features to the VM that make it possible to implement sandboxing and isolation, and making sure the implementation is safe from memory safety issues using fuzzing. -Of course, since the entire stack is implemented in C++, the sandboxing isn't formally proven - in theory, compiler or the standard library can have exploitable vulnerabilities. In practice these are usually found and fixed quickly. While implementing the stack in a safer language such as Rust would make it easier to provide these guarantees, to our knowledge (based on prior art) this would make it difficult to reach the level of performance required. +Of course, since the entire stack is implemented in C++, the sandboxing isn't formally proven - in theory, compiler or the standard library can have exploitable vulnerabilities. In practice these are very rare and usually found and fixed quickly. While implementing the stack in a safer language such as Rust would make it easier to provide these guarantees, to our knowledge (based on prior art) this would make it difficult to reach the level of performance required. ## Library From fc763650d3f0e6cfa16c791bc8e11ea706710161 Mon Sep 17 00:00:00 2001 From: natteko Date: Thu, 30 Jun 2022 23:14:49 +0100 Subject: [PATCH 295/399] Fix broken link in typecheck.md (#568) Current link redirects to https://luau-lang.org/typecheck#Roblox-types (notice the fragment) which is effectively the same as https://luau-lang.org/typecheck What the link *wants* to redirect to is https://luau-lang.org/typecheck#roblox-types (notice the change in fragment) which is the Roblox types segment of the document --- docs/_pages/typecheck.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/_pages/typecheck.md b/docs/_pages/typecheck.md index 63e4c8bb..363056c5 100644 --- a/docs/_pages/typecheck.md +++ b/docs/_pages/typecheck.md @@ -48,7 +48,7 @@ local b2: B = a1 -- not ok ## Primitive types -Lua VM supports 8 primitive types: `nil`, `string`, `number`, `boolean`, `table`, `function`, `thread`, and `userdata`. Of these, `table` and `function` are not represented by name, but have their dedicated syntax as covered in this [syntax document](syntax), and `userdata` is represented by [concrete types](#Roblox-types); other types can be specified by their name. +Lua VM supports 8 primitive types: `nil`, `string`, `number`, `boolean`, `table`, `function`, `thread`, and `userdata`. Of these, `table` and `function` are not represented by name, but have their dedicated syntax as covered in this [syntax document](syntax), and `userdata` is represented by [concrete types](#roblox-types); other types can be specified by their name. Additionally, we also have `any` which is a special built-in type. It effectively disables all type checking, and thus should be used as last resort. From 8f040862b11e3758af97ac9a712d6c579a7e01f5 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Thu, 30 Jun 2022 16:29:02 -0700 Subject: [PATCH 296/399] Sync to upstream/release/534 --- Analysis/include/Luau/Constraint.h | 19 +- .../include/Luau/ConstraintGraphBuilder.h | 109 +++- Analysis/include/Luau/ConstraintSolver.h | 12 +- Analysis/include/Luau/Error.h | 33 +- Analysis/include/Luau/Frontend.h | 5 + Analysis/include/Luau/Module.h | 2 +- Analysis/include/Luau/NotNull.h | 14 +- Analysis/include/Luau/Scope.h | 6 +- Analysis/include/Luau/TypeArena.h | 6 + Analysis/include/Luau/TypeInfer.h | 4 +- Analysis/include/Luau/TypeVar.h | 3 + Analysis/src/ConstraintGraphBuilder.cpp | 557 ++++++++++++++---- Analysis/src/ConstraintSolver.cpp | 71 ++- Analysis/src/EmbeddedBuiltinDefinitions.cpp | 10 +- Analysis/src/Frontend.cpp | 22 +- Analysis/src/Normalize.cpp | 235 +------- Analysis/src/Quantify.cpp | 73 ++- Analysis/src/Scope.cpp | 15 + Analysis/src/ToString.cpp | 111 ++-- Analysis/src/TypeChecker2.cpp | 106 +++- Analysis/src/TypeInfer.cpp | 144 ++++- Analysis/src/Unifier.cpp | 4 +- Ast/src/Parser.cpp | 122 +++- Common/include/Luau/Bytecode.h | 5 +- Compiler/src/Builtins.cpp | 4 + Compiler/src/BytecodeBuilder.cpp | 15 +- Compiler/src/Compiler.cpp | 16 +- VM/src/lbaselib.cpp | 15 + VM/src/lbuiltins.cpp | 23 + VM/src/ldebug.h | 1 + VM/src/ltm.cpp | 2 +- VM/src/ltm.h | 2 +- VM/src/lvmexecute.cpp | 54 +- VM/src/lvmutils.cpp | 54 +- fuzz/protoprint.cpp | 11 + tests/Compiler.test.cpp | 16 +- tests/Conformance.test.cpp | 6 + tests/ConstraintGraphBuilder.test.cpp | 30 +- tests/ConstraintSolver.test.cpp | 17 +- tests/Fixture.cpp | 23 +- tests/Fixture.h | 3 +- tests/NotNull.test.cpp | 12 +- tests/Parser.test.cpp | 32 +- tests/ToString.test.cpp | 18 +- tests/TypeInfer.aliases.test.cpp | 35 +- tests/TypeInfer.annotations.test.cpp | 14 +- tests/TypeInfer.functions.test.cpp | 21 + tests/TypeInfer.generics.test.cpp | 5 +- tests/TypeInfer.operators.test.cpp | 55 ++ tests/TypeInfer.provisional.test.cpp | 22 + tests/TypeInfer.tables.test.cpp | 121 +++- tests/TypeInfer.test.cpp | 4 +- tests/conformance/basic.lua | 10 +- tests/conformance/events.lua | 38 ++ 54 files changed, 1714 insertions(+), 653 deletions(-) diff --git a/Analysis/include/Luau/Constraint.h b/Analysis/include/Luau/Constraint.h index 8a41c9e8..dcfb14b4 100644 --- a/Analysis/include/Luau/Constraint.h +++ b/Analysis/include/Luau/Constraint.h @@ -1,6 +1,7 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #pragma once +#include "Luau/Ast.h" // Used for some of the enumerations #include "Luau/NotNull.h" #include "Luau/Variant.h" @@ -47,6 +48,21 @@ struct InstantiationConstraint TypeId superType; }; +struct UnaryConstraint +{ + AstExprUnary::Op op; + TypeId operandType; + TypeId resultType; +}; + +struct BinaryConstraint +{ + AstExprBinary::Op op; + TypeId leftType; + TypeId rightType; + TypeId resultType; +}; + // name(namedType) = name struct NameConstraint { @@ -54,7 +70,8 @@ struct NameConstraint std::string name; }; -using ConstraintV = Variant; +using ConstraintV = Variant; using ConstraintPtr = std::unique_ptr; struct Constraint diff --git a/Analysis/include/Luau/ConstraintGraphBuilder.h b/Analysis/include/Luau/ConstraintGraphBuilder.h index 9b118691..a49e8594 100644 --- a/Analysis/include/Luau/ConstraintGraphBuilder.h +++ b/Analysis/include/Luau/ConstraintGraphBuilder.h @@ -25,9 +25,12 @@ struct ConstraintGraphBuilder // scope pointers; the scopes themselves borrow pointers to other scopes to // define the scope hierarchy. std::vector>> scopes; + + ModuleName moduleName; SingletonTypes& singletonTypes; - TypeArena* const arena; + const NotNull arena; // The root scope of the module we're generating constraints for. + // This is null when the CGB is initially constructed. Scope2* rootScope; // A mapping of AST node to TypeId. DenseHashMap astTypes{nullptr}; @@ -39,40 +42,50 @@ struct ConstraintGraphBuilder // Type packs resolved from type annotations. Analogous to astTypePacks. DenseHashMap astResolvedTypePacks{nullptr}; - explicit ConstraintGraphBuilder(TypeArena* arena); + int recursionCount = 0; + + // It is pretty uncommon for constraint generation to itself produce errors, but it can happen. + std::vector errors; + + // Occasionally constraint generation needs to produce an ICE. + const NotNull ice; + + NotNull globalScope; + + ConstraintGraphBuilder(const ModuleName& moduleName, TypeArena* arena, NotNull ice, NotNull globalScope); /** * Fabricates a new free type belonging to a given scope. - * @param scope the scope the free type belongs to. Must not be null. + * @param scope the scope the free type belongs to. */ - TypeId freshType(Scope2* scope); + TypeId freshType(NotNull scope); /** * Fabricates a new free type pack belonging to a given scope. - * @param scope the scope the free type pack belongs to. Must not be null. + * @param scope the scope the free type pack belongs to. */ - TypePackId freshTypePack(Scope2* scope); + TypePackId freshTypePack(NotNull scope); /** * Fabricates a scope that is a child of another scope. * @param location the lexical extent of the scope in the source code. * @param parent the parent scope of the new scope. Must not be null. */ - Scope2* childScope(Location location, Scope2* parent); + NotNull childScope(Location location, NotNull parent); /** * Adds a new constraint with no dependencies to a given scope. - * @param scope the scope to add the constraint to. Must not be null. + * @param scope the scope to add the constraint to. * @param cv the constraint variant to add. */ - void addConstraint(Scope2* scope, ConstraintV cv); + void addConstraint(NotNull scope, ConstraintV cv); /** * Adds a constraint to a given scope. * @param scope the scope to add the constraint to. Must not be null. * @param c the constraint to add. */ - void addConstraint(Scope2* scope, std::unique_ptr c); + void addConstraint(NotNull scope, std::unique_ptr c); /** * The entry point to the ConstraintGraphBuilder. This will construct a set @@ -81,20 +94,22 @@ struct ConstraintGraphBuilder */ void visit(AstStatBlock* block); - void visit(Scope2* scope, AstStat* stat); - void visit(Scope2* scope, AstStatBlock* block); - void visit(Scope2* scope, AstStatLocal* local); - void visit(Scope2* scope, AstStatLocalFunction* function); - void visit(Scope2* scope, AstStatFunction* function); - void visit(Scope2* scope, AstStatReturn* ret); - void visit(Scope2* scope, AstStatAssign* assign); - void visit(Scope2* scope, AstStatIf* ifStatement); - void visit(Scope2* scope, AstStatTypeAlias* alias); + void visitBlockWithoutChildScope(NotNull scope, AstStatBlock* block); - TypePackId checkExprList(Scope2* scope, const AstArray& exprs); + void visit(NotNull scope, AstStat* stat); + void visit(NotNull scope, AstStatBlock* block); + void visit(NotNull scope, AstStatLocal* local); + void visit(NotNull scope, AstStatLocalFunction* function); + void visit(NotNull scope, AstStatFunction* function); + void visit(NotNull scope, AstStatReturn* ret); + void visit(NotNull scope, AstStatAssign* assign); + void visit(NotNull scope, AstStatIf* ifStatement); + void visit(NotNull scope, AstStatTypeAlias* alias); - TypePackId checkPack(Scope2* scope, AstArray exprs); - TypePackId checkPack(Scope2* scope, AstExpr* expr); + TypePackId checkExprList(NotNull scope, const AstArray& exprs); + + TypePackId checkPack(NotNull scope, AstArray exprs); + TypePackId checkPack(NotNull scope, AstExpr* expr); /** * Checks an expression that is expected to evaluate to one type. @@ -102,19 +117,35 @@ struct ConstraintGraphBuilder * @param expr the expression to check. * @return the type of the expression. */ - TypeId check(Scope2* scope, AstExpr* expr); + TypeId check(NotNull scope, AstExpr* expr); - TypeId checkExprTable(Scope2* scope, AstExprTable* expr); - TypeId check(Scope2* scope, AstExprIndexName* indexName); + TypeId checkExprTable(NotNull scope, AstExprTable* expr); + TypeId check(NotNull scope, AstExprIndexName* indexName); + TypeId check(NotNull scope, AstExprIndexExpr* indexExpr); + TypeId check(NotNull scope, AstExprUnary* unary); + TypeId check(NotNull scope, AstExprBinary* binary); - std::pair checkFunctionSignature(Scope2* parent, AstExprFunction* fn); + struct FunctionSignature + { + // The type of the function. + TypeId signature; + // The scope that encompasses the function's signature. May be nullptr + // if there was no need for a signature scope (the function has no + // generics). + Scope2* signatureScope; + // The scope that encompasses the function's body. Is a child scope of + // signatureScope, if present. + NotNull bodyScope; + }; + + FunctionSignature checkFunctionSignature(NotNull parent, AstExprFunction* fn); /** * Checks the body of a function expression. * @param scope the interior scope of the body of the function. * @param fn the function expression to check. */ - void checkFunctionBody(Scope2* scope, AstExprFunction* fn); + void checkFunctionBody(NotNull scope, AstExprFunction* fn); /** * Resolves a type from its AST annotation. @@ -122,7 +153,7 @@ struct ConstraintGraphBuilder * @param ty the AST annotation to resolve. * @return the type of the AST annotation. **/ - TypeId resolveType(Scope2* scope, AstType* ty); + TypeId resolveType(NotNull scope, AstType* ty); /** * Resolves a type pack from its AST annotation. @@ -130,9 +161,25 @@ struct ConstraintGraphBuilder * @param tp the AST annotation to resolve. * @return the type pack of the AST annotation. **/ - TypePackId resolveTypePack(Scope2* scope, AstTypePack* tp); + TypePackId resolveTypePack(NotNull scope, AstTypePack* tp); - TypePackId resolveTypePack(Scope2* scope, const AstTypeList& list); + TypePackId resolveTypePack(NotNull scope, const AstTypeList& list); + + std::vector> createGenerics(NotNull scope, AstArray generics); + std::vector> createGenericPacks(NotNull scope, AstArray packs); + + TypeId flattenPack(NotNull scope, Location location, TypePackId tp); + + void reportError(Location location, TypeErrorData err); + void reportCodeTooComplex(Location location); + + /** Scan the program for global definitions. + * + * ConstraintGraphBuilder needs to differentiate between globals and accesses to undefined symbols. Doing this "for + * real" in a general way is going to be pretty hard, so we are choosing not to tackle that yet. For now, we do an + * initial scan of the AST and note what globals are defined. + */ + void prepopulateGlobalScope(NotNull globalScope, AstStatBlock* program); }; /** @@ -145,6 +192,6 @@ struct ConstraintGraphBuilder * @return a list of pointers to constraints contained within the scope graph. * None of these pointers should be null. */ -std::vector> collectConstraints(Scope2* rootScope); +std::vector> collectConstraints(NotNull rootScope); } // namespace Luau diff --git a/Analysis/include/Luau/ConstraintSolver.h b/Analysis/include/Luau/ConstraintSolver.h index 4870157f..cf88efb6 100644 --- a/Analysis/include/Luau/ConstraintSolver.h +++ b/Analysis/include/Luau/ConstraintSolver.h @@ -25,7 +25,7 @@ struct ConstraintSolver // is important to not add elements to this vector, lest the underlying // storage that we retain pointers to be mutated underneath us. const std::vector> constraints; - Scope2* rootScope; + NotNull rootScope; // This includes every constraint that has not been fully solved. // A constraint can be both blocked and unsolved, for instance. @@ -40,7 +40,7 @@ struct ConstraintSolver ConstraintSolverLogger logger; - explicit ConstraintSolver(TypeArena* arena, Scope2* rootScope); + explicit ConstraintSolver(TypeArena* arena, NotNull rootScope); /** * Attempts to dispatch all pending constraints and reach a type solution @@ -50,11 +50,17 @@ struct ConstraintSolver bool done(); + /** Attempt to dispatch a constraint. Returns true if it was successful. + * If tryDispatch() returns false, the constraint remains in the unsolved set and will be retried later. + */ bool tryDispatch(NotNull c, bool force); + bool tryDispatch(const SubtypeConstraint& c, NotNull constraint, bool force); bool tryDispatch(const PackSubtypeConstraint& c, NotNull constraint, bool force); bool tryDispatch(const GeneralizationConstraint& c, NotNull constraint, bool force); bool tryDispatch(const InstantiationConstraint& c, NotNull constraint, bool force); + bool tryDispatch(const UnaryConstraint& c, NotNull constraint, bool force); + bool tryDispatch(const BinaryConstraint& c, NotNull constraint, bool force); bool tryDispatch(const NameConstraint& c, NotNull constraint); void block(NotNull target, NotNull constraint); @@ -115,6 +121,6 @@ private: void unblock_(BlockedConstraintId progressed); }; -void dump(Scope2* rootScope, struct ToStringOptions& opts); +void dump(NotNull rootScope, struct ToStringOptions& opts); } // namespace Luau diff --git a/Analysis/include/Luau/Error.h b/Analysis/include/Luau/Error.h index a1323960..4c81d33d 100644 --- a/Analysis/include/Luau/Error.h +++ b/Analysis/include/Luau/Error.h @@ -369,24 +369,25 @@ struct InternalErrorReporter [[noreturn]] void ice(const std::string& message); }; -class InternalCompilerError : public std::exception { +class InternalCompilerError : public std::exception +{ public: - explicit InternalCompilerError(const std::string& message, const std::string& moduleName) - : message(message) - , moduleName(moduleName) - { - } - explicit InternalCompilerError(const std::string& message, const std::string& moduleName, const Location& location) - : message(message) - , moduleName(moduleName) - , location(location) - { - } - virtual const char* what() const throw(); + explicit InternalCompilerError(const std::string& message, const std::string& moduleName) + : message(message) + , moduleName(moduleName) + { + } + explicit InternalCompilerError(const std::string& message, const std::string& moduleName, const Location& location) + : message(message) + , moduleName(moduleName) + , location(location) + { + } + virtual const char* what() const throw(); - const std::string message; - const std::string moduleName; - const std::optional location; + const std::string message; + const std::string moduleName; + const std::optional location; }; } // namespace Luau diff --git a/Analysis/include/Luau/Frontend.h b/Analysis/include/Luau/Frontend.h index f4226cc1..f0d43090 100644 --- a/Analysis/include/Luau/Frontend.h +++ b/Analysis/include/Luau/Frontend.h @@ -5,6 +5,7 @@ #include "Luau/Module.h" #include "Luau/ModuleResolver.h" #include "Luau/RequireTracer.h" +#include "Luau/Scope.h" #include "Luau/TypeInfer.h" #include "Luau/Variant.h" @@ -158,6 +159,8 @@ struct Frontend void registerBuiltinDefinition(const std::string& name, std::function); void applyBuiltinDefinitionToEnvironment(const std::string& environmentName, const std::string& definitionName); + NotNull getGlobalScope2(); + private: ModulePtr check(const SourceModule& sourceModule, Mode mode, const ScopePtr& environmentScope); @@ -173,6 +176,8 @@ private: std::unordered_map environments; std::unordered_map> builtinDefinitions; + std::unique_ptr globalScope2; + public: FileResolver* fileResolver; FrontendModuleResolver moduleResolver; diff --git a/Analysis/include/Luau/Module.h b/Analysis/include/Luau/Module.h index 39f8dfb7..b3105b78 100644 --- a/Analysis/include/Luau/Module.h +++ b/Analysis/include/Luau/Module.h @@ -68,7 +68,7 @@ struct Module std::shared_ptr allocator; std::shared_ptr names; - std::vector> scopes; // never empty + std::vector> scopes; // never empty std::vector>> scope2s; // never empty DenseHashMap astTypes{nullptr}; diff --git a/Analysis/include/Luau/NotNull.h b/Analysis/include/Luau/NotNull.h index f6043e9c..714fa143 100644 --- a/Analysis/include/Luau/NotNull.h +++ b/Analysis/include/Luau/NotNull.h @@ -26,7 +26,7 @@ namespace Luau * The explicit delete statement is permitted (but not recommended) on a * NotNull through this implicit conversion. */ -template +template struct NotNull { explicit NotNull(T* t) @@ -38,10 +38,11 @@ struct NotNull explicit NotNull(std::nullptr_t) = delete; void operator=(std::nullptr_t) = delete; - template + template NotNull(NotNull other) : ptr(other.get()) - {} + { + } operator T*() const noexcept { @@ -72,12 +73,13 @@ private: T* ptr; }; -} +} // namespace Luau namespace std { -template struct hash> +template +struct hash> { size_t operator()(const Luau::NotNull& p) const { @@ -85,4 +87,4 @@ template struct hash> } }; -} +} // namespace std diff --git a/Analysis/include/Luau/Scope.h b/Analysis/include/Luau/Scope.h index cef4b94f..0eaecf1d 100644 --- a/Analysis/include/Luau/Scope.h +++ b/Analysis/include/Luau/Scope.h @@ -3,6 +3,7 @@ #include "Luau/Constraint.h" #include "Luau/Location.h" +#include "Luau/NotNull.h" #include "Luau/TypeVar.h" #include @@ -71,15 +72,18 @@ struct Scope2 // is the module-level scope). Scope2* parent = nullptr; // All the children of this scope. - std::vector children; + std::vector> children; std::unordered_map bindings; // TODO: I think this can be a DenseHashMap std::unordered_map typeBindings; + std::unordered_map typePackBindings; TypePackId returnType; + std::optional varargPack; // All constraints belonging to this scope. std::vector constraints; std::optional lookup(Symbol sym); std::optional lookupTypeBinding(const Name& name); + std::optional lookupTypePackBinding(const Name& name); }; } // namespace Luau diff --git a/Analysis/include/Luau/TypeArena.h b/Analysis/include/Luau/TypeArena.h index 559c55c8..be36f19c 100644 --- a/Analysis/include/Luau/TypeArena.h +++ b/Analysis/include/Luau/TypeArena.h @@ -34,6 +34,12 @@ struct TypeArena TypePackId addTypePack(std::vector types); TypePackId addTypePack(TypePack pack); TypePackId addTypePack(TypePackVar pack); + + template + TypePackId addTypePack(T tp) + { + return addTypePack(TypePackVar(std::move(tp))); + } }; void freeze(TypeArena& arena); diff --git a/Analysis/include/Luau/TypeInfer.h b/Analysis/include/Luau/TypeInfer.h index 28adc9d9..455654d9 100644 --- a/Analysis/include/Luau/TypeInfer.h +++ b/Analysis/include/Luau/TypeInfer.h @@ -173,7 +173,7 @@ struct TypeChecker TypeId checkFunctionName(const ScopePtr& scope, AstExpr& funName, TypeLevel level); std::pair checkFunctionSignature(const ScopePtr& scope, int subLevel, const AstExprFunction& expr, - std::optional originalNameLoc, std::optional expectedType); + std::optional originalNameLoc, std::optional selfType, std::optional expectedType); void checkFunctionBody(const ScopePtr& scope, TypeId type, const AstExprFunction& function); void checkArgumentList( @@ -424,6 +424,8 @@ private: * (exported, name) to properly deal with the case where the two duplicates do not have the same export status. */ DenseHashSet, HashBoolNamePair> duplicateTypeAliases; + + std::vector> deferredQuantification; }; // Unit test hook diff --git a/Analysis/include/Luau/TypeVar.h b/Analysis/include/Luau/TypeVar.h index 20f4107c..6ad6b927 100644 --- a/Analysis/include/Luau/TypeVar.h +++ b/Analysis/include/Luau/TypeVar.h @@ -357,6 +357,9 @@ struct TableTypeVar std::optional boundTo; Tags tags; + + // Methods of this table that have an untyped self will use the same shared self type. + std::optional selfTy; }; // Represents a metatable attached to a table typevar. Somewhat analogous to a bound typevar. diff --git a/Analysis/src/ConstraintGraphBuilder.cpp b/Analysis/src/ConstraintGraphBuilder.cpp index d9e8d238..3b9000cd 100644 --- a/Analysis/src/ConstraintGraphBuilder.cpp +++ b/Analysis/src/ConstraintGraphBuilder.cpp @@ -1,6 +1,10 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #include "Luau/ConstraintGraphBuilder.h" +#include "Luau/RecursionCounter.h" +#include "Luau/ToString.h" + +LUAU_FASTINT(LuauCheckRecursionLimit); #include "Luau/Scope.h" @@ -9,32 +13,33 @@ namespace Luau const AstStat* getFallthrough(const AstStat* node); // TypeInfer.cpp -ConstraintGraphBuilder::ConstraintGraphBuilder(TypeArena* arena) - : singletonTypes(getSingletonTypes()) +ConstraintGraphBuilder::ConstraintGraphBuilder( + const ModuleName& moduleName, TypeArena* arena, NotNull ice, NotNull globalScope) + : moduleName(moduleName) + , singletonTypes(getSingletonTypes()) , arena(arena) , rootScope(nullptr) + , ice(ice) + , globalScope(globalScope) { LUAU_ASSERT(arena); } -TypeId ConstraintGraphBuilder::freshType(Scope2* scope) +TypeId ConstraintGraphBuilder::freshType(NotNull scope) { - LUAU_ASSERT(scope); return arena->addType(FreeTypeVar{scope}); } -TypePackId ConstraintGraphBuilder::freshTypePack(Scope2* scope) +TypePackId ConstraintGraphBuilder::freshTypePack(NotNull scope) { - LUAU_ASSERT(scope); FreeTypePack f{scope}; return arena->addTypePack(TypePackVar{std::move(f)}); } -Scope2* ConstraintGraphBuilder::childScope(Location location, Scope2* parent) +NotNull ConstraintGraphBuilder::childScope(Location location, NotNull parent) { - LUAU_ASSERT(parent); auto scope = std::make_unique(); - Scope2* borrow = scope.get(); + NotNull borrow = NotNull(scope.get()); scopes.emplace_back(location, std::move(scope)); borrow->parent = parent; @@ -44,15 +49,13 @@ Scope2* ConstraintGraphBuilder::childScope(Location location, Scope2* parent) return borrow; } -void ConstraintGraphBuilder::addConstraint(Scope2* scope, ConstraintV cv) +void ConstraintGraphBuilder::addConstraint(NotNull scope, ConstraintV cv) { - LUAU_ASSERT(scope); scope->constraints.emplace_back(new Constraint{std::move(cv)}); } -void ConstraintGraphBuilder::addConstraint(Scope2* scope, std::unique_ptr c) +void ConstraintGraphBuilder::addConstraint(NotNull scope, std::unique_ptr c) { - LUAU_ASSERT(scope); scope->constraints.emplace_back(std::move(c)); } @@ -62,7 +65,11 @@ void ConstraintGraphBuilder::visit(AstStatBlock* block) LUAU_ASSERT(rootScope == nullptr); scopes.emplace_back(block->location, std::make_unique()); rootScope = scopes.back().second.get(); - rootScope->returnType = freshTypePack(rootScope); + NotNull borrow = NotNull(rootScope); + + rootScope->returnType = freshTypePack(borrow); + + prepopulateGlobalScope(borrow, block); // TODO: We should share the global scope. rootScope->typeBindings["nil"] = singletonTypes.nilType; @@ -71,12 +78,26 @@ void ConstraintGraphBuilder::visit(AstStatBlock* block) rootScope->typeBindings["boolean"] = singletonTypes.booleanType; rootScope->typeBindings["thread"] = singletonTypes.threadType; - visit(rootScope, block); + visitBlockWithoutChildScope(borrow, block); } -void ConstraintGraphBuilder::visit(Scope2* scope, AstStat* stat) +void ConstraintGraphBuilder::visitBlockWithoutChildScope(NotNull scope, AstStatBlock* block) { - LUAU_ASSERT(scope); + RecursionCounter counter{&recursionCount}; + + if (recursionCount >= FInt::LuauCheckRecursionLimit) + { + reportCodeTooComplex(block->location); + return; + } + + for (AstStat* stat : block->body) + visit(scope, stat); +} + +void ConstraintGraphBuilder::visit(NotNull scope, AstStat* stat) +{ + RecursionLimiter limiter{&recursionCount, FInt::LuauCheckRecursionLimit}; if (auto s = stat->as()) visit(scope, s); @@ -100,10 +121,8 @@ void ConstraintGraphBuilder::visit(Scope2* scope, AstStat* stat) LUAU_ASSERT(0); } -void ConstraintGraphBuilder::visit(Scope2* scope, AstStatLocal* local) +void ConstraintGraphBuilder::visit(NotNull scope, AstStatLocal* local) { - LUAU_ASSERT(scope); - std::vector varTypes; for (AstLocal* local : local->vars) @@ -148,23 +167,19 @@ void ConstraintGraphBuilder::visit(Scope2* scope, AstStatLocal* local) } } -void addConstraints(Constraint* constraint, Scope2* scope) +void addConstraints(Constraint* constraint, NotNull scope) { - LUAU_ASSERT(scope); - scope->constraints.reserve(scope->constraints.size() + scope->constraints.size()); for (const auto& c : scope->constraints) constraint->dependencies.push_back(NotNull{c.get()}); - for (Scope2* childScope : scope->children) + for (NotNull childScope : scope->children) addConstraints(constraint, childScope); } -void ConstraintGraphBuilder::visit(Scope2* scope, AstStatLocalFunction* function) +void ConstraintGraphBuilder::visit(NotNull scope, AstStatLocalFunction* function) { - LUAU_ASSERT(scope); - // Local // Global // Dotted path @@ -172,36 +187,31 @@ void ConstraintGraphBuilder::visit(Scope2* scope, AstStatLocalFunction* function TypeId functionType = nullptr; auto ty = scope->lookup(function->name); - if (ty.has_value()) - { - // TODO: This is duplicate definition of a local function. Is this allowed? - functionType = *ty; - } - else - { - functionType = arena->addType(BlockedTypeVar{}); - scope->bindings[function->name] = functionType; - } + LUAU_ASSERT(!ty.has_value()); // The parser ensures that every local function has a distinct Symbol for its name. - auto [actualFunctionType, innerScope] = checkFunctionSignature(scope, function->func); - innerScope->bindings[function->name] = actualFunctionType; + functionType = arena->addType(BlockedTypeVar{}); + scope->bindings[function->name] = functionType; - checkFunctionBody(innerScope, function->func); + FunctionSignature sig = checkFunctionSignature(scope, function->func); + sig.bodyScope->bindings[function->name] = sig.signature; - std::unique_ptr c{new Constraint{GeneralizationConstraint{functionType, actualFunctionType, innerScope}}}; - addConstraints(c.get(), innerScope); + checkFunctionBody(sig.bodyScope, function->func); + + std::unique_ptr c{ + new Constraint{GeneralizationConstraint{functionType, sig.signature, sig.signatureScope ? sig.signatureScope : sig.bodyScope}}}; + addConstraints(c.get(), sig.bodyScope); addConstraint(scope, std::move(c)); } -void ConstraintGraphBuilder::visit(Scope2* scope, AstStatFunction* function) +void ConstraintGraphBuilder::visit(NotNull scope, AstStatFunction* function) { // Name could be AstStatLocal, AstStatGlobal, AstStatIndexName. // With or without self TypeId functionType = nullptr; - auto [actualFunctionType, innerScope] = checkFunctionSignature(scope, function->func); + FunctionSignature sig = checkFunctionSignature(scope, function->func); if (AstExprLocal* localName = function->name->as()) { @@ -216,7 +226,7 @@ void ConstraintGraphBuilder::visit(Scope2* scope, AstStatFunction* function) functionType = arena->addType(BlockedTypeVar{}); scope->bindings[localName->local] = functionType; } - innerScope->bindings[localName->local] = actualFunctionType; + sig.bodyScope->bindings[localName->local] = sig.signature; } else if (AstExprGlobal* globalName = function->name->as()) { @@ -231,32 +241,48 @@ void ConstraintGraphBuilder::visit(Scope2* scope, AstStatFunction* function) functionType = arena->addType(BlockedTypeVar{}); rootScope->bindings[globalName->name] = functionType; } - innerScope->bindings[globalName->name] = actualFunctionType; + sig.bodyScope->bindings[globalName->name] = sig.signature; } else if (AstExprIndexName* indexName = function->name->as()) { - LUAU_ASSERT(0); // not yet implemented + TypeId containingTableType = check(scope, indexName->expr); + + functionType = arena->addType(BlockedTypeVar{}); + TypeId prospectiveTableType = + arena->addType(TableTypeVar{}); // TODO look into stack utilization. This is probably ok because it scales with AST depth. + NotNull prospectiveTable{getMutable(prospectiveTableType)}; + + Property& prop = prospectiveTable->props[indexName->index.value]; + prop.type = functionType; + prop.location = function->name->location; + + addConstraint(scope, SubtypeConstraint{containingTableType, prospectiveTableType}); + } + else if (AstExprError* err = function->name->as()) + { + functionType = singletonTypes.errorRecoveryType(); } - checkFunctionBody(innerScope, function->func); + LUAU_ASSERT(functionType != nullptr); - std::unique_ptr c{new Constraint{GeneralizationConstraint{functionType, actualFunctionType, innerScope}}}; - addConstraints(c.get(), innerScope); + checkFunctionBody(sig.bodyScope, function->func); + + std::unique_ptr c{ + new Constraint{GeneralizationConstraint{functionType, sig.signature, sig.signatureScope ? sig.signatureScope : sig.bodyScope}}}; + addConstraints(c.get(), sig.bodyScope); addConstraint(scope, std::move(c)); } -void ConstraintGraphBuilder::visit(Scope2* scope, AstStatReturn* ret) +void ConstraintGraphBuilder::visit(NotNull scope, AstStatReturn* ret) { - LUAU_ASSERT(scope); - TypePackId exprTypes = checkPack(scope, ret->list); addConstraint(scope, PackSubtypeConstraint{exprTypes, scope->returnType}); } -void ConstraintGraphBuilder::visit(Scope2* scope, AstStatBlock* block) +void ConstraintGraphBuilder::visit(NotNull scope, AstStatBlock* block) { - LUAU_ASSERT(scope); + NotNull innerScope = childScope(block->location, scope); // In order to enable mutually-recursive type aliases, we need to // populate the type bindings before we actually check any of the @@ -271,11 +297,10 @@ void ConstraintGraphBuilder::visit(Scope2* scope, AstStatBlock* block) } } - for (AstStat* stat : block->body) - visit(scope, stat); + visitBlockWithoutChildScope(innerScope, block); } -void ConstraintGraphBuilder::visit(Scope2* scope, AstStatAssign* assign) +void ConstraintGraphBuilder::visit(NotNull scope, AstStatAssign* assign) { TypePackId varPackId = checkExprList(scope, assign->vars); TypePackId valuePack = checkPack(scope, assign->values); @@ -283,21 +308,21 @@ void ConstraintGraphBuilder::visit(Scope2* scope, AstStatAssign* assign) addConstraint(scope, PackSubtypeConstraint{valuePack, varPackId}); } -void ConstraintGraphBuilder::visit(Scope2* scope, AstStatIf* ifStatement) +void ConstraintGraphBuilder::visit(NotNull scope, AstStatIf* ifStatement) { check(scope, ifStatement->condition); - Scope2* thenScope = childScope(ifStatement->thenbody->location, scope); + NotNull thenScope = childScope(ifStatement->thenbody->location, scope); visit(thenScope, ifStatement->thenbody); if (ifStatement->elsebody) { - Scope2* elseScope = childScope(ifStatement->elsebody->location, scope); + NotNull elseScope = childScope(ifStatement->elsebody->location, scope); visit(elseScope, ifStatement->elsebody); } } -void ConstraintGraphBuilder::visit(Scope2* scope, AstStatTypeAlias* alias) +void ConstraintGraphBuilder::visit(NotNull scope, AstStatTypeAlias* alias) { // TODO: Exported type aliases // TODO: Generic type aliases @@ -307,6 +332,10 @@ void ConstraintGraphBuilder::visit(Scope2* scope, AstStatTypeAlias* alias) // AST to set up typeBindings. If it's not, we've somehow skipped // this alias in that first pass. LUAU_ASSERT(it != scope->typeBindings.end()); + if (it == scope->typeBindings.end()) + { + ice->ice("Type alias does not have a pre-populated binding", alias->location); + } TypeId ty = resolveType(scope, alias->type); @@ -319,10 +348,8 @@ void ConstraintGraphBuilder::visit(Scope2* scope, AstStatTypeAlias* alias) addConstraint(scope, NameConstraint{ty, alias->name.value}); } -TypePackId ConstraintGraphBuilder::checkPack(Scope2* scope, AstArray exprs) +TypePackId ConstraintGraphBuilder::checkPack(NotNull scope, AstArray exprs) { - LUAU_ASSERT(scope); - if (exprs.size == 0) return arena->addTypePack({}); @@ -342,7 +369,7 @@ TypePackId ConstraintGraphBuilder::checkPack(Scope2* scope, AstArray e return arena->addTypePack(TypePack{std::move(types), last}); } -TypePackId ConstraintGraphBuilder::checkExprList(Scope2* scope, const AstArray& exprs) +TypePackId ConstraintGraphBuilder::checkExprList(NotNull scope, const AstArray& exprs) { TypePackId result = arena->addTypePack({}); TypePack* resultPack = getMutable(result); @@ -363,9 +390,15 @@ TypePackId ConstraintGraphBuilder::checkExprList(Scope2* scope, const AstArray scope, AstExpr* expr) { - LUAU_ASSERT(scope); + RecursionCounter counter{&recursionCount}; + + if (recursionCount >= FInt::LuauCheckRecursionLimit) + { + reportCodeTooComplex(expr->location); + return singletonTypes.errorRecoveryTypePack(); + } TypePackId result = nullptr; @@ -384,7 +417,7 @@ TypePackId ConstraintGraphBuilder::checkPack(Scope2* scope, AstExpr* expr) astOriginalCallTypes[call->func] = fnType; - TypeId instantiatedType = freshType(scope); + TypeId instantiatedType = arena->addType(BlockedTypeVar{}); addConstraint(scope, InstantiationConstraint{instantiatedType, fnType}); TypePackId rets = freshTypePack(scope); @@ -394,6 +427,13 @@ TypePackId ConstraintGraphBuilder::checkPack(Scope2* scope, AstExpr* expr) addConstraint(scope, SubtypeConstraint{inferredFnType, instantiatedType}); result = rets; } + else if (AstExprVarargs* varargs = expr->as()) + { + if (scope->varargPack) + result = *scope->varargPack; + else + result = singletonTypes.errorRecoveryTypePack(); + } else { TypeId t = check(scope, expr); @@ -405,9 +445,15 @@ TypePackId ConstraintGraphBuilder::checkPack(Scope2* scope, AstExpr* expr) return result; } -TypeId ConstraintGraphBuilder::check(Scope2* scope, AstExpr* expr) +TypeId ConstraintGraphBuilder::check(NotNull scope, AstExpr* expr) { - LUAU_ASSERT(scope); + RecursionCounter counter{&recursionCount}; + + if (recursionCount >= FInt::LuauCheckRecursionLimit) + { + reportCodeTooComplex(expr->location); + return singletonTypes.errorRecoveryType(); + } TypeId result = nullptr; @@ -435,37 +481,38 @@ TypeId ConstraintGraphBuilder::check(Scope2* scope, AstExpr* expr) if (ty) result = *ty; else - result = singletonTypes.errorRecoveryType(); // FIXME? Record an error at this point? - } - else if (auto a = expr->as()) - { - TypePackId packResult = checkPack(scope, expr); - if (auto f = first(packResult)) - return *f; - else if (get(packResult)) { - TypeId typeResult = freshType(scope); - TypePack onePack{{typeResult}, freshTypePack(scope)}; - TypePackId oneTypePack = arena->addTypePack(std::move(onePack)); - - addConstraint(scope, PackSubtypeConstraint{packResult, oneTypePack}); - - return typeResult; + /* prepopulateGlobalScope() has already added all global functions to the environment by this point, so any + * global that is not already in-scope is definitely an unknown symbol. + */ + reportError(g->location, UnknownSymbol{g->name.value}); + result = singletonTypes.errorRecoveryType(); // FIXME? Record an error at this point? } } + else if (expr->is()) + result = flattenPack(scope, expr->location, checkPack(scope, expr)); + else if (expr->is()) + result = flattenPack(scope, expr->location, checkPack(scope, expr)); else if (auto a = expr->as()) { - auto [fnType, functionScope] = checkFunctionSignature(scope, a); - checkFunctionBody(functionScope, a); - return fnType; + FunctionSignature sig = checkFunctionSignature(scope, a); + checkFunctionBody(sig.bodyScope, a); + return sig.signature; } else if (auto indexName = expr->as()) - { result = check(scope, indexName); - } + else if (auto indexExpr = expr->as()) + result = check(scope, indexExpr); else if (auto table = expr->as()) - { result = checkExprTable(scope, table); + else if (auto unary = expr->as()) + result = check(scope, unary); + else if (auto binary = expr->as()) + result = check(scope, binary); + else if (auto err = expr->as()) + { + // Open question: Should we traverse into this? + result = singletonTypes.errorRecoveryType(); } else { @@ -478,7 +525,7 @@ TypeId ConstraintGraphBuilder::check(Scope2* scope, AstExpr* expr) return result; } -TypeId ConstraintGraphBuilder::check(Scope2* scope, AstExprIndexName* indexName) +TypeId ConstraintGraphBuilder::check(NotNull scope, AstExprIndexName* indexName) { TypeId obj = check(scope, indexName->expr); TypeId result = freshType(scope); @@ -494,7 +541,67 @@ TypeId ConstraintGraphBuilder::check(Scope2* scope, AstExprIndexName* indexName) return result; } -TypeId ConstraintGraphBuilder::checkExprTable(Scope2* scope, AstExprTable* expr) +TypeId ConstraintGraphBuilder::check(NotNull scope, AstExprIndexExpr* indexExpr) +{ + TypeId obj = check(scope, indexExpr->expr); + TypeId indexType = check(scope, indexExpr->index); + + TypeId result = freshType(scope); + + TableIndexer indexer{indexType, result}; + TypeId tableType = arena->addType(TableTypeVar{TableTypeVar::Props{}, TableIndexer{indexType, result}, TypeLevel{}, TableState::Free}); + + addConstraint(scope, SubtypeConstraint{obj, tableType}); + + return result; +} + +TypeId ConstraintGraphBuilder::check(NotNull scope, AstExprUnary* unary) +{ + TypeId operandType = check(scope, unary->expr); + + switch (unary->op) + { + case AstExprUnary::Minus: + { + TypeId resultType = arena->addType(BlockedTypeVar{}); + addConstraint(scope, UnaryConstraint{AstExprUnary::Minus, operandType, resultType}); + return resultType; + } + default: + LUAU_ASSERT(0); + } + + LUAU_UNREACHABLE(); + return singletonTypes.errorRecoveryType(); +} + +TypeId ConstraintGraphBuilder::check(NotNull scope, AstExprBinary* binary) +{ + TypeId leftType = check(scope, binary->left); + TypeId rightType = check(scope, binary->right); + switch (binary->op) + { + case AstExprBinary::Or: + { + addConstraint(scope, SubtypeConstraint{leftType, rightType}); + return leftType; + } + case AstExprBinary::Sub: + { + TypeId resultType = arena->addType(BlockedTypeVar{}); + addConstraint(scope, BinaryConstraint{AstExprBinary::Sub, leftType, rightType, resultType}); + return resultType; + } + default: + LUAU_ASSERT(0); + } + + LUAU_ASSERT(0); + return nullptr; +} + +TypeId ConstraintGraphBuilder::checkExprTable(NotNull scope, AstExprTable* expr) { TypeId ty = arena->addType(TableTypeVar{}); TableTypeVar* ttv = getMutable(ty); @@ -515,6 +622,8 @@ TypeId ConstraintGraphBuilder::checkExprTable(Scope2* scope, AstExprTable* expr) for (const AstExprTable::Item& item : expr->items) { TypeId itemTy = check(scope, item.value); + if (get(follow(itemTy))) + return ty; if (item.key) { @@ -542,47 +651,111 @@ TypeId ConstraintGraphBuilder::checkExprTable(Scope2* scope, AstExprTable* expr) return ty; } -std::pair ConstraintGraphBuilder::checkFunctionSignature(Scope2* parent, AstExprFunction* fn) +ConstraintGraphBuilder::FunctionSignature ConstraintGraphBuilder::checkFunctionSignature(NotNull parent, AstExprFunction* fn) { - Scope2* innerScope = childScope(fn->body->location, parent); - TypePackId returnType = freshTypePack(innerScope); - innerScope->returnType = returnType; + Scope2* signatureScope = nullptr; + Scope2* bodyScope = nullptr; + TypePackId returnType = nullptr; + + std::vector genericTypes; + std::vector genericTypePacks; + + bool hasGenerics = fn->generics.size > 0 || fn->genericPacks.size > 0; + + // If we don't have any generics, we can save some memory and compute by not + // creating the signatureScope, which is only used to scope the declared + // generics properly. + if (hasGenerics) + { + NotNull signatureBorrow = childScope(fn->location, parent); + signatureScope = signatureBorrow.get(); + + // We need to assign returnType before creating bodyScope so that the + // return type gets propogated to bodyScope. + returnType = freshTypePack(signatureBorrow); + signatureScope->returnType = returnType; + + bodyScope = childScope(fn->body->location, signatureBorrow).get(); + + std::vector> genericDefinitions = createGenerics(signatureBorrow, fn->generics); + std::vector> genericPackDefinitions = createGenericPacks(signatureBorrow, fn->genericPacks); + + // We do not support default values on function generics, so we only + // care about the types involved. + for (const auto& [name, g] : genericDefinitions) + { + genericTypes.push_back(g.ty); + signatureScope->typeBindings[name] = g.ty; + } + + for (const auto& [name, g] : genericPackDefinitions) + { + genericTypePacks.push_back(g.tp); + signatureScope->typePackBindings[name] = g.tp; + } + } + else + { + NotNull bodyBorrow = childScope(fn->body->location, parent); + bodyScope = bodyBorrow.get(); + + returnType = freshTypePack(bodyBorrow); + bodyBorrow->returnType = returnType; + + // To eliminate the need to branch on hasGenerics below, we say that the + // signature scope is the body scope when there is no real signature + // scope. + signatureScope = bodyScope; + } + + NotNull bodyBorrow = NotNull(bodyScope); + NotNull signatureBorrow = NotNull(signatureScope); if (fn->returnAnnotation) { - TypePackId annotatedRetType = resolveTypePack(innerScope, *fn->returnAnnotation); - addConstraint(innerScope, PackSubtypeConstraint{returnType, annotatedRetType}); + TypePackId annotatedRetType = resolveTypePack(signatureBorrow, *fn->returnAnnotation); + addConstraint(signatureBorrow, PackSubtypeConstraint{returnType, annotatedRetType}); } std::vector argTypes; for (AstLocal* local : fn->args) { - TypeId t = freshType(innerScope); + TypeId t = freshType(signatureBorrow); argTypes.push_back(t); - innerScope->bindings[local] = t; + signatureScope->bindings[local] = t; if (local->annotation) { - TypeId argAnnotation = resolveType(innerScope, local->annotation); - addConstraint(innerScope, SubtypeConstraint{t, argAnnotation}); + TypeId argAnnotation = resolveType(signatureBorrow, local->annotation); + addConstraint(signatureBorrow, SubtypeConstraint{t, argAnnotation}); } } // TODO: Vararg annotation. + // TODO: Preserve argument names in the function's type. FunctionTypeVar actualFunction{arena->addTypePack(argTypes), returnType}; + actualFunction.hasNoGenerics = !hasGenerics; + actualFunction.generics = std::move(genericTypes); + actualFunction.genericPacks = std::move(genericTypePacks); + TypeId actualFunctionType = arena->addType(std::move(actualFunction)); LUAU_ASSERT(actualFunctionType); astTypes[fn] = actualFunctionType; - return {actualFunctionType, innerScope}; + return { + /* signature */ actualFunctionType, + // Undo the workaround we made above: if there's no signature scope, + // don't report it. + /* signatureScope */ hasGenerics ? signatureScope : nullptr, + /* bodyScope */ bodyBorrow, + }; } -void ConstraintGraphBuilder::checkFunctionBody(Scope2* scope, AstExprFunction* fn) +void ConstraintGraphBuilder::checkFunctionBody(NotNull scope, AstExprFunction* fn) { - for (AstStat* stat : fn->body->body) - visit(scope, stat); + visitBlockWithoutChildScope(scope, fn->body); // If it is possible for execution to reach the end of the function, the return type must be compatible with () @@ -593,7 +766,7 @@ void ConstraintGraphBuilder::checkFunctionBody(Scope2* scope, AstExprFunction* f } } -TypeId ConstraintGraphBuilder::resolveType(Scope2* scope, AstType* ty) +TypeId ConstraintGraphBuilder::resolveType(NotNull scope, AstType* ty) { TypeId result = nullptr; @@ -636,29 +809,73 @@ TypeId ConstraintGraphBuilder::resolveType(Scope2* scope, AstType* ty) } else if (auto fn = ty->as()) { - // TODO: Generic functions. - // TODO: Scope (though it may not be needed). // TODO: Recursion limit. - TypePackId argTypes = resolveTypePack(scope, fn->argTypes); - TypePackId returnTypes = resolveTypePack(scope, fn->returnTypes); + bool hasGenerics = fn->generics.size > 0 || fn->genericPacks.size > 0; + Scope2* signatureScope = nullptr; - // TODO: Is this the right constructor to use? - result = arena->addType(FunctionTypeVar{argTypes, returnTypes}); + std::vector genericTypes; + std::vector genericTypePacks; - FunctionTypeVar* ftv = getMutable(result); - ftv->argNames.reserve(fn->argNames.size); + // If we don't have generics, we do not need to generate a child scope + // for the generic bindings to live on. + if (hasGenerics) + { + NotNull signatureBorrow = childScope(fn->location, scope); + signatureScope = signatureBorrow.get(); + + std::vector> genericDefinitions = createGenerics(signatureBorrow, fn->generics); + std::vector> genericPackDefinitions = createGenericPacks(signatureBorrow, fn->genericPacks); + + for (const auto& [name, g] : genericDefinitions) + { + genericTypes.push_back(g.ty); + signatureBorrow->typeBindings[name] = g.ty; + } + + for (const auto& [name, g] : genericPackDefinitions) + { + genericTypePacks.push_back(g.tp); + signatureBorrow->typePackBindings[name] = g.tp; + } + } + else + { + // To eliminate the need to branch on hasGenerics below, we say that + // the signature scope is the parent scope if we don't have + // generics. + signatureScope = scope.get(); + } + + NotNull signatureBorrow(signatureScope); + + TypePackId argTypes = resolveTypePack(signatureBorrow, fn->argTypes); + TypePackId returnTypes = resolveTypePack(signatureBorrow, fn->returnTypes); + + // TODO: FunctionTypeVar needs a pointer to the scope so that we know + // how to quantify/instantiate it. + FunctionTypeVar ftv{argTypes, returnTypes}; + + // This replicates the behavior of the appropriate FunctionTypeVar + // constructors. + ftv.hasNoGenerics = !hasGenerics; + ftv.generics = std::move(genericTypes); + ftv.genericPacks = std::move(genericTypePacks); + + ftv.argNames.reserve(fn->argNames.size); for (const auto& el : fn->argNames) { if (el) { const auto& [name, location] = *el; - ftv->argNames.push_back(FunctionArgument{name.value, location}); + ftv.argNames.push_back(FunctionArgument{name.value, location}); } else { - ftv->argNames.push_back(std::nullopt); + ftv.argNames.push_back(std::nullopt); } } + + result = arena->addType(std::move(ftv)); } else if (auto tof = ty->as()) { @@ -710,7 +927,7 @@ TypeId ConstraintGraphBuilder::resolveType(Scope2* scope, AstType* ty) return result; } -TypePackId ConstraintGraphBuilder::resolveTypePack(Scope2* scope, AstTypePack* tp) +TypePackId ConstraintGraphBuilder::resolveTypePack(NotNull scope, AstTypePack* tp) { TypePackId result; if (auto expl = tp->as()) @@ -736,7 +953,7 @@ TypePackId ConstraintGraphBuilder::resolveTypePack(Scope2* scope, AstTypePack* t return result; } -TypePackId ConstraintGraphBuilder::resolveTypePack(Scope2* scope, const AstTypeList& list) +TypePackId ConstraintGraphBuilder::resolveTypePack(NotNull scope, const AstTypeList& list) { std::vector head; @@ -754,16 +971,108 @@ TypePackId ConstraintGraphBuilder::resolveTypePack(Scope2* scope, const AstTypeL return arena->addTypePack(TypePack{head, tail}); } -void collectConstraints(std::vector>& result, Scope2* scope) +std::vector> ConstraintGraphBuilder::createGenerics(NotNull scope, AstArray generics) +{ + std::vector> result; + for (const auto& generic : generics) + { + TypeId genericTy = arena->addType(GenericTypeVar{scope, generic.name.value}); + std::optional defaultTy = std::nullopt; + + if (generic.defaultValue) + defaultTy = resolveType(scope, generic.defaultValue); + + result.push_back({generic.name.value, GenericTypeDefinition{ + genericTy, + defaultTy, + }}); + } + + return result; +} + +std::vector> ConstraintGraphBuilder::createGenericPacks( + NotNull scope, AstArray generics) +{ + std::vector> result; + for (const auto& generic : generics) + { + TypePackId genericTy = arena->addTypePack(TypePackVar{GenericTypePack{scope, generic.name.value}}); + std::optional defaultTy = std::nullopt; + + if (generic.defaultValue) + defaultTy = resolveTypePack(scope, generic.defaultValue); + + result.push_back({generic.name.value, GenericTypePackDefinition{ + genericTy, + defaultTy, + }}); + } + + return result; +} + +TypeId ConstraintGraphBuilder::flattenPack(NotNull scope, Location location, TypePackId tp) +{ + if (auto f = first(tp)) + return *f; + + TypeId typeResult = freshType(scope); + TypePack onePack{{typeResult}, freshTypePack(scope)}; + TypePackId oneTypePack = arena->addTypePack(std::move(onePack)); + + addConstraint(scope, PackSubtypeConstraint{tp, oneTypePack}); + + return typeResult; +} + +void ConstraintGraphBuilder::reportError(Location location, TypeErrorData err) +{ + errors.push_back(TypeError{location, moduleName, std::move(err)}); +} + +void ConstraintGraphBuilder::reportCodeTooComplex(Location location) +{ + errors.push_back(TypeError{location, moduleName, CodeTooComplex{}}); +} + +struct GlobalPrepopulator : AstVisitor +{ + const NotNull globalScope; + const NotNull arena; + + GlobalPrepopulator(NotNull globalScope, NotNull arena) + : globalScope(globalScope) + , arena(arena) + { + } + + bool visit(AstStatFunction* function) override + { + if (AstExprGlobal* g = function->name->as()) + globalScope->bindings[g->name] = arena->addType(BlockedTypeVar{}); + + return true; + } +}; + +void ConstraintGraphBuilder::prepopulateGlobalScope(NotNull globalScope, AstStatBlock* program) +{ + GlobalPrepopulator gp{NotNull{globalScope}, arena}; + + program->visit(&gp); +} + +void collectConstraints(std::vector>& result, NotNull scope) { for (const auto& c : scope->constraints) result.push_back(NotNull{c.get()}); - for (Scope2* child : scope->children) + for (NotNull child : scope->children) collectConstraints(result, child); } -std::vector> collectConstraints(Scope2* rootScope) +std::vector> collectConstraints(NotNull rootScope) { std::vector> result; collectConstraints(result, rootScope); diff --git a/Analysis/src/ConstraintSolver.cpp b/Analysis/src/ConstraintSolver.cpp index 9e355236..077a4e28 100644 --- a/Analysis/src/ConstraintSolver.cpp +++ b/Analysis/src/ConstraintSolver.cpp @@ -13,7 +13,7 @@ LUAU_FASTFLAGVARIABLE(DebugLuauLogSolverToJson, false); namespace Luau { -[[maybe_unused]] static void dumpBindings(Scope2* scope, ToStringOptions& opts) +[[maybe_unused]] static void dumpBindings(NotNull scope, ToStringOptions& opts) { for (const auto& [k, v] : scope->bindings) { @@ -22,22 +22,22 @@ namespace Luau printf("\t%s : %s\n", k.c_str(), d.name.c_str()); } - for (Scope2* child : scope->children) + for (NotNull child : scope->children) dumpBindings(child, opts); } -static void dumpConstraints(Scope2* scope, ToStringOptions& opts) +static void dumpConstraints(NotNull scope, ToStringOptions& opts) { for (const ConstraintPtr& c : scope->constraints) { printf("\t%s\n", toString(*c, opts).c_str()); } - for (Scope2* child : scope->children) + for (NotNull child : scope->children) dumpConstraints(child, opts); } -void dump(Scope2* rootScope, ToStringOptions& opts) +void dump(NotNull rootScope, ToStringOptions& opts) { printf("constraints:\n"); dumpConstraints(rootScope, opts); @@ -55,7 +55,7 @@ void dump(ConstraintSolver* cs, ToStringOptions& opts) } } -ConstraintSolver::ConstraintSolver(TypeArena* arena, Scope2* rootScope) +ConstraintSolver::ConstraintSolver(TypeArena* arena, NotNull rootScope) : arena(arena) , constraints(collectConstraints(rootScope)) , rootScope(rootScope) @@ -180,6 +180,10 @@ bool ConstraintSolver::tryDispatch(NotNull constraint, bool fo success = tryDispatch(*gc, constraint, force); else if (auto ic = get(*constraint)) success = tryDispatch(*ic, constraint, force); + else if (auto uc = get(*constraint)) + success = tryDispatch(*uc, constraint, force); + else if (auto bc = get(*constraint)) + success = tryDispatch(*bc, constraint, force); else if (auto nc = get(*constraint)) success = tryDispatch(*nc, constraint); else @@ -246,12 +250,65 @@ bool ConstraintSolver::tryDispatch(const InstantiationConstraint& c, NotNull instantiated = inst.substitute(c.superType); LUAU_ASSERT(instantiated); // TODO FIXME HANDLE THIS - unify(c.subType, *instantiated); + if (isBlocked(c.subType)) + asMutable(c.subType)->ty.emplace(*instantiated); + else + unify(c.subType, *instantiated); + unblock(c.subType); return true; } +bool ConstraintSolver::tryDispatch(const UnaryConstraint& c, NotNull constraint, bool force) +{ + TypeId operandType = follow(c.operandType); + + if (isBlocked(operandType)) + return block(operandType, constraint); + + if (get(operandType)) + return block(operandType, constraint); + + LUAU_ASSERT(get(c.resultType)); + + if (isNumber(operandType) || get(operandType) || get(operandType)) + { + asMutable(c.resultType)->ty.emplace(c.operandType); + return true; + } + + LUAU_ASSERT(0); // TODO metatable handling + return false; +} + +bool ConstraintSolver::tryDispatch(const BinaryConstraint& c, NotNull constraint, bool force) +{ + TypeId leftType = follow(c.leftType); + TypeId rightType = follow(c.rightType); + + if (isBlocked(leftType) || isBlocked(rightType)) + { + block(leftType, constraint); + block(rightType, constraint); + return false; + } + + if (isNumber(leftType)) + { + unify(leftType, rightType); + asMutable(c.resultType)->ty.emplace(leftType); + return true; + } + + if (get(leftType) && !force) + return block(leftType, constraint); + + // TODO metatables, classes + + return true; +} + bool ConstraintSolver::tryDispatch(const NameConstraint& c, NotNull constraint) { if (isBlocked(c.namedType)) diff --git a/Analysis/src/EmbeddedBuiltinDefinitions.cpp b/Analysis/src/EmbeddedBuiltinDefinitions.cpp index 2407e3ef..1b5275fd 100644 --- a/Analysis/src/EmbeddedBuiltinDefinitions.cpp +++ b/Analysis/src/EmbeddedBuiltinDefinitions.cpp @@ -1,6 +1,8 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #include "Luau/BuiltinDefinitions.h" +LUAU_FASTFLAG(LuauCheckLenMT) + namespace Luau { @@ -202,7 +204,13 @@ declare function unpack(tab: {V}, i: number?, j: number?): ...V std::string getBuiltinDefinitionSource() { - return kBuiltinDefinitionLuaSrc; + std::string result = kBuiltinDefinitionLuaSrc; + + // TODO: move this into kBuiltinDefinitionLuaSrc + if (FFlag::LuauCheckLenMT) + result += "declare function rawlen(obj: {[K]: V} | string): number\n"; + + return result; } } // namespace Luau diff --git a/Analysis/src/Frontend.cpp b/Analysis/src/Frontend.cpp index 85c5dbc8..4cfaa112 100644 --- a/Analysis/src/Frontend.cpp +++ b/Analysis/src/Frontend.cpp @@ -787,14 +787,32 @@ const SourceModule* Frontend::getSourceModule(const ModuleName& moduleName) cons return const_cast(this)->getSourceModule(moduleName); } +NotNull Frontend::getGlobalScope2() +{ + if (!globalScope2) + { + const SingletonTypes& singletonTypes = getSingletonTypes(); + + globalScope2 = std::make_unique(); + globalScope2->typeBindings["nil"] = singletonTypes.nilType; + globalScope2->typeBindings["number"] = singletonTypes.numberType; + globalScope2->typeBindings["string"] = singletonTypes.stringType; + globalScope2->typeBindings["boolean"] = singletonTypes.booleanType; + globalScope2->typeBindings["thread"] = singletonTypes.threadType; + } + + return NotNull(globalScope2.get()); +} + ModulePtr Frontend::check(const SourceModule& sourceModule, Mode mode, const ScopePtr& environmentScope) { ModulePtr result = std::make_shared(); - ConstraintGraphBuilder cgb{&result->internalTypes}; + ConstraintGraphBuilder cgb{sourceModule.name, &result->internalTypes, NotNull(&iceHandler), getGlobalScope2()}; cgb.visit(sourceModule.root); + result->errors = std::move(cgb.errors); - ConstraintSolver cs{&result->internalTypes, cgb.rootScope}; + ConstraintSolver cs{&result->internalTypes, NotNull(cgb.rootScope)}; cs.run(); result->scope2s = std::move(cgb.scopes); diff --git a/Analysis/src/Normalize.cpp b/Analysis/src/Normalize.cpp index d36665e2..8ce7f742 100644 --- a/Analysis/src/Normalize.cpp +++ b/Analysis/src/Normalize.cpp @@ -5,7 +5,6 @@ #include #include "Luau/Clone.h" -#include "Luau/Substitution.h" #include "Luau/Unifier.h" #include "Luau/VisitTypeVar.h" @@ -16,7 +15,6 @@ LUAU_FASTINTVARIABLE(LuauNormalizeIterationLimit, 1200); LUAU_FASTFLAGVARIABLE(LuauNormalizeCombineTableFix, false); LUAU_FASTFLAGVARIABLE(LuauNormalizeFlagIsConservative, false); LUAU_FASTFLAGVARIABLE(LuauNormalizeCombineEqFix, false); -LUAU_FASTFLAGVARIABLE(LuauReplaceReplacer, false); LUAU_FASTFLAG(LuauQuantifyConstrained) namespace Luau @@ -25,238 +23,33 @@ namespace Luau namespace { -struct Replacer : Substitution +struct Replacer { + TypeArena* arena; TypeId sourceType; TypeId replacedType; - DenseHashMap replacedTypes{nullptr}; - DenseHashMap replacedPacks{nullptr}; + DenseHashMap newTypes; Replacer(TypeArena* arena, TypeId sourceType, TypeId replacedType) - : Substitution(TxnLog::empty(), arena) + : arena(arena) , sourceType(sourceType) , replacedType(replacedType) + , newTypes(nullptr) { } - bool isDirty(TypeId ty) override - { - if (!sourceType) - return false; - - auto vecHasSourceType = [sourceType = sourceType](const auto& vec) { - return end(vec) != std::find(begin(vec), end(vec), sourceType); - }; - - // Walk every kind of TypeVar and find pointers to sourceType - if (auto t = get(ty)) - return false; - else if (auto t = get(ty)) - return false; - else if (auto t = get(ty)) - return false; - else if (auto t = get(ty)) - return false; - else if (auto t = get(ty)) - return vecHasSourceType(t->parts); - else if (auto t = get(ty)) - return false; - else if (auto t = get(ty)) - { - if (vecHasSourceType(t->generics)) - return true; - - return false; - } - else if (auto t = get(ty)) - { - if (t->boundTo) - return *t->boundTo == sourceType; - - for (const auto& [_name, prop] : t->props) - { - if (prop.type == sourceType) - return true; - } - - if (auto indexer = t->indexer) - { - if (indexer->indexType == sourceType || indexer->indexResultType == sourceType) - return true; - } - - if (vecHasSourceType(t->instantiatedTypeParams)) - return true; - - return false; - } - else if (auto t = get(ty)) - return t->table == sourceType || t->metatable == sourceType; - else if (auto t = get(ty)) - return false; - else if (auto t = get(ty)) - return false; - else if (auto t = get(ty)) - return vecHasSourceType(t->options); - else if (auto t = get(ty)) - return vecHasSourceType(t->parts); - else if (auto t = get(ty)) - return false; - - LUAU_ASSERT(!"Luau::Replacer::isDirty internal error: Unknown TypeVar type"); - LUAU_UNREACHABLE(); - } - - bool isDirty(TypePackId tp) override - { - if (auto it = replacedPacks.find(tp)) - return false; - - if (auto pack = get(tp)) - { - for (TypeId ty : pack->head) - { - if (ty == sourceType) - return true; - } - return false; - } - else if (auto vtp = get(tp)) - return vtp->ty == sourceType; - else - return false; - } - - TypeId clean(TypeId ty) override - { - LUAU_ASSERT(sourceType && replacedType); - - // Walk every kind of TypeVar and create a copy with sourceType replaced by replacedType - // Before returning, memoize the result for later use. - - // Helpfully, Substitution::clone() only shallow-clones the kinds of types that we care to work with. This - // function returns the identity for things like primitives. - TypeId res = clone(ty); - - if (auto t = get(res)) - LUAU_ASSERT(!"Impossible"); - else if (auto t = get(res)) - LUAU_ASSERT(!"Impossible"); - else if (auto t = get(res)) - LUAU_ASSERT(!"Impossible"); - else if (auto t = get(res)) - LUAU_ASSERT(!"Impossible"); - else if (auto t = getMutable(res)) - { - for (TypeId& part : t->parts) - { - if (part == sourceType) - part = replacedType; - } - } - else if (auto t = get(res)) - LUAU_ASSERT(!"Impossible"); - else if (auto t = getMutable(res)) - { - // The constituent typepacks are cleaned separately. We just need to walk the generics array. - for (TypeId& g : t->generics) - { - if (g == sourceType) - g = replacedType; - } - } - else if (auto t = getMutable(res)) - { - for (auto& [_key, prop] : t->props) - { - if (prop.type == sourceType) - prop.type = replacedType; - } - } - else if (auto t = getMutable(res)) - { - if (t->table == sourceType) - t->table = replacedType; - if (t->metatable == sourceType) - t->table = replacedType; - } - else if (auto t = get(res)) - LUAU_ASSERT(!"Impossible"); - else if (auto t = get(res)) - LUAU_ASSERT(!"Impossible"); - else if (auto t = getMutable(res)) - { - for (TypeId& option : t->options) - { - if (option == sourceType) - option = replacedType; - } - } - else if (auto t = getMutable(res)) - { - for (TypeId& part : t->parts) - { - if (part == sourceType) - part = replacedType; - } - } - else if (auto t = get(res)) - LUAU_ASSERT(!"Impossible"); - else - LUAU_ASSERT(!"Luau::Replacer::clean internal error: Unknown TypeVar type"); - - replacedTypes[ty] = res; - return res; - } - - TypePackId clean(TypePackId tp) override - { - TypePackId res = clone(tp); - - if (auto pack = getMutable(res)) - { - for (TypeId& type : pack->head) - { - if (type == sourceType) - type = replacedType; - } - } - else if (auto vtp = getMutable(res)) - { - if (vtp->ty == sourceType) - vtp->ty = replacedType; - } - - replacedPacks[tp] = res; - return res; - } - TypeId smartClone(TypeId t) { - if (FFlag::LuauReplaceReplacer) - { - // The new smartClone is just a memoized clone() - // TODO: Remove the Substitution base class and all other methods from this struct. - // Add DenseHashMap newTypes; - t = log->follow(t); - TypeId* res = newTypes.find(t); - if (res) - return *res; - - TypeId result = shallowClone(t, *arena, TxnLog::empty()); - newTypes[t] = result; - newTypes[result] = result; - - return result; - } - else - { - std::optional res = replace(t); - LUAU_ASSERT(res.has_value()); // TODO think about this - if (*res == t) - return clone(t); + t = follow(t); + TypeId* res = newTypes.find(t); + if (res) return *res; - } + + TypeId result = shallowClone(t, *arena, TxnLog::empty()); + newTypes[t] = result; + newTypes[result] = result; + + return result; } }; diff --git a/Analysis/src/Quantify.cpp b/Analysis/src/Quantify.cpp index 40e14c68..294c479d 100644 --- a/Analysis/src/Quantify.cpp +++ b/Analysis/src/Quantify.cpp @@ -8,6 +8,7 @@ #include "Luau/VisitTypeVar.h" LUAU_FASTFLAG(LuauAlwaysQuantify); +LUAU_FASTFLAG(DebugLuauSharedSelf) LUAU_FASTFLAG(DebugLuauDeferredConstraintResolution); LUAU_FASTFLAGVARIABLE(LuauQuantifyConstrained, false) @@ -158,24 +159,61 @@ struct Quantifier final : TypeVarOnceVisitor void quantify(TypeId ty, TypeLevel level) { - Quantifier q{level}; - q.traverse(ty); - - FunctionTypeVar* ftv = getMutable(ty); - LUAU_ASSERT(ftv); - if (FFlag::LuauAlwaysQuantify) + if (FFlag::DebugLuauSharedSelf) { - ftv->generics.insert(ftv->generics.end(), q.generics.begin(), q.generics.end()); - ftv->genericPacks.insert(ftv->genericPacks.end(), q.genericPacks.begin(), q.genericPacks.end()); + ty = follow(ty); + + if (auto ttv = getTableType(ty); ttv && ttv->selfTy) + { + Quantifier selfQ{level}; + selfQ.traverse(*ttv->selfTy); + + Quantifier q{level}; + q.traverse(ty); + + for (const auto& [_, prop] : ttv->props) + { + auto ftv = getMutable(follow(prop.type)); + if (!ftv || !ftv->hasSelf) + continue; + + if (Luau::first(ftv->argTypes) == ttv->selfTy) + { + ftv->generics.insert(ftv->generics.end(), selfQ.generics.begin(), selfQ.generics.end()); + ftv->genericPacks.insert(ftv->genericPacks.end(), selfQ.genericPacks.begin(), selfQ.genericPacks.end()); + } + } + } + else if (auto ftv = getMutable(ty)) + { + Quantifier q{level}; + q.traverse(ty); + + ftv->generics.insert(ftv->generics.end(), q.generics.begin(), q.generics.end()); + ftv->genericPacks.insert(ftv->genericPacks.end(), q.genericPacks.begin(), q.genericPacks.end()); + + if (ftv->generics.empty() && ftv->genericPacks.empty() && !q.seenMutableType && !q.seenGenericType) + ftv->hasNoGenerics = true; + } } else { - ftv->generics = q.generics; - ftv->genericPacks = q.genericPacks; - } + Quantifier q{level}; + q.traverse(ty); - if (ftv->generics.empty() && ftv->genericPacks.empty() && !q.seenMutableType && !q.seenGenericType) - ftv->hasNoGenerics = true; + FunctionTypeVar* ftv = getMutable(ty); + LUAU_ASSERT(ftv); + if (FFlag::LuauAlwaysQuantify) + { + ftv->generics.insert(ftv->generics.end(), q.generics.begin(), q.generics.end()); + ftv->genericPacks.insert(ftv->genericPacks.end(), q.genericPacks.begin(), q.genericPacks.end()); + } + else + { + ftv->generics = q.generics; + ftv->genericPacks = q.genericPacks; + } + } } void quantify(TypeId ty, Scope2* scope) @@ -206,8 +244,8 @@ struct PureQuantifier : Substitution std::vector insertedGenerics; std::vector insertedGenericPacks; - PureQuantifier(const TxnLog* log, TypeArena* arena, Scope2* scope) - : Substitution(log, arena) + PureQuantifier(TypeArena* arena, Scope2* scope) + : Substitution(TxnLog::empty(), arena) , scope(scope) { } @@ -286,7 +324,7 @@ struct PureQuantifier : Substitution TypeId quantify(TypeArena* arena, TypeId ty, Scope2* scope) { - PureQuantifier quantifier{TxnLog::empty(), arena, scope}; + PureQuantifier quantifier{arena, scope}; std::optional result = quantifier.substitute(ty); LUAU_ASSERT(result); @@ -294,8 +332,7 @@ TypeId quantify(TypeArena* arena, TypeId ty, Scope2* scope) LUAU_ASSERT(ftv); ftv->generics.insert(ftv->generics.end(), quantifier.insertedGenerics.begin(), quantifier.insertedGenerics.end()); ftv->genericPacks.insert(ftv->genericPacks.end(), quantifier.insertedGenericPacks.begin(), quantifier.insertedGenericPacks.end()); - - // TODO: Set hasNoGenerics. + ftv->hasNoGenerics = ftv->generics.empty() && ftv->genericPacks.empty(); return *result; } diff --git a/Analysis/src/Scope.cpp b/Analysis/src/Scope.cpp index 66aaee1f..247a9dd6 100644 --- a/Analysis/src/Scope.cpp +++ b/Analysis/src/Scope.cpp @@ -153,4 +153,19 @@ std::optional Scope2::lookupTypeBinding(const Name& name) return std::nullopt; } +std::optional Scope2::lookupTypePackBinding(const Name& name) +{ + Scope2* s = this; + while (s) + { + auto it = s->typePackBindings.find(name); + if (it != s->typePackBindings.end()) + return it->second; + + s = s->parent; + } + + return std::nullopt; +} + } // namespace Luau diff --git a/Analysis/src/ToString.cpp b/Analysis/src/ToString.cpp index eb7b9cd6..fe940d5e 100644 --- a/Analysis/src/ToString.cpp +++ b/Analysis/src/ToString.cpp @@ -1367,51 +1367,74 @@ std::string generateName(size_t i) return n; } -std::string toString(const Constraint& c, ToStringOptions& opts) +std::string toString(const Constraint& constraint, ToStringOptions& opts) { - if (const SubtypeConstraint* sc = Luau::get_if(&c.c)) - { - ToStringResult subStr = toStringDetailed(sc->subType, opts); - opts.nameMap = std::move(subStr.nameMap); - ToStringResult superStr = toStringDetailed(sc->superType, opts); - opts.nameMap = std::move(superStr.nameMap); - return subStr.name + " <: " + superStr.name; - } - else if (const PackSubtypeConstraint* psc = Luau::get_if(&c.c)) - { - ToStringResult subStr = toStringDetailed(psc->subPack, opts); - opts.nameMap = std::move(subStr.nameMap); - ToStringResult superStr = toStringDetailed(psc->superPack, opts); - opts.nameMap = std::move(superStr.nameMap); - return subStr.name + " <: " + superStr.name; - } - else if (const GeneralizationConstraint* gc = Luau::get_if(&c.c)) - { - ToStringResult subStr = toStringDetailed(gc->generalizedType, opts); - opts.nameMap = std::move(subStr.nameMap); - ToStringResult superStr = toStringDetailed(gc->sourceType, opts); - opts.nameMap = std::move(superStr.nameMap); - return subStr.name + " ~ gen " + superStr.name; - } - else if (const InstantiationConstraint* ic = Luau::get_if(&c.c)) - { - ToStringResult subStr = toStringDetailed(ic->subType, opts); - opts.nameMap = std::move(subStr.nameMap); - ToStringResult superStr = toStringDetailed(ic->superType, opts); - opts.nameMap = std::move(superStr.nameMap); - return subStr.name + " ~ inst " + superStr.name; - } - else if (const NameConstraint* nc = Luau::get(c)) - { - ToStringResult namedStr = toStringDetailed(nc->namedType, opts); - opts.nameMap = std::move(namedStr.nameMap); - return "@name(" + namedStr.name + ") = " + nc->name; - } - else - { - LUAU_ASSERT(false); - return ""; - } + auto go = [&opts](auto&& c) { + using T = std::decay_t; + + if constexpr (std::is_same_v) + { + ToStringResult subStr = toStringDetailed(c.subType, opts); + opts.nameMap = std::move(subStr.nameMap); + ToStringResult superStr = toStringDetailed(c.superType, opts); + opts.nameMap = std::move(superStr.nameMap); + return subStr.name + " <: " + superStr.name; + } + else if constexpr (std::is_same_v) + { + ToStringResult subStr = toStringDetailed(c.subPack, opts); + opts.nameMap = std::move(subStr.nameMap); + ToStringResult superStr = toStringDetailed(c.superPack, opts); + opts.nameMap = std::move(superStr.nameMap); + return subStr.name + " <: " + superStr.name; + } + else if constexpr (std::is_same_v) + { + ToStringResult subStr = toStringDetailed(c.generalizedType, opts); + opts.nameMap = std::move(subStr.nameMap); + ToStringResult superStr = toStringDetailed(c.sourceType, opts); + opts.nameMap = std::move(superStr.nameMap); + return subStr.name + " ~ gen " + superStr.name; + } + else if constexpr (std::is_same_v) + { + ToStringResult subStr = toStringDetailed(c.subType, opts); + opts.nameMap = std::move(subStr.nameMap); + ToStringResult superStr = toStringDetailed(c.superType, opts); + opts.nameMap = std::move(superStr.nameMap); + return subStr.name + " ~ inst " + superStr.name; + } + else if constexpr (std::is_same_v) + { + ToStringResult resultStr = toStringDetailed(c.resultType, opts); + opts.nameMap = std::move(resultStr.nameMap); + ToStringResult operandStr = toStringDetailed(c.operandType, opts); + opts.nameMap = std::move(operandStr.nameMap); + + return resultStr.name + " ~ Unary<" + toString(c.op) + ", " + operandStr.name + ">"; + } + else if constexpr (std::is_same_v) + { + ToStringResult resultStr = toStringDetailed(c.resultType); + opts.nameMap = std::move(resultStr.nameMap); + ToStringResult leftStr = toStringDetailed(c.leftType); + opts.nameMap = std::move(leftStr.nameMap); + ToStringResult rightStr = toStringDetailed(c.rightType); + opts.nameMap = std::move(rightStr.nameMap); + + return resultStr.name + " ~ Binary<" + toString(c.op) + ", " + leftStr.name + ", " + rightStr.name + ">"; + } + else if constexpr (std::is_same_v) + { + ToStringResult namedStr = toStringDetailed(c.namedType, opts); + opts.nameMap = std::move(namedStr.nameMap); + return "@name(" + namedStr.name + ") = " + c.name; + } + else + static_assert(always_false_v, "Non-exhaustive constraint switch"); + }; + + return visit(go, constraint.c); } std::string dump(const Constraint& c) diff --git a/Analysis/src/TypeChecker2.cpp b/Analysis/src/TypeChecker2.cpp index 63e5800f..30e498af 100644 --- a/Analysis/src/TypeChecker2.cpp +++ b/Analysis/src/TypeChecker2.cpp @@ -6,8 +6,10 @@ #include "Luau/Ast.h" #include "Luau/AstQuery.h" #include "Luau/Clone.h" +#include "Luau/Instantiation.h" #include "Luau/Normalize.h" -#include "Luau/ConstraintGraphBuilder.h" // FIXME move Scope2 into its own header +#include "Luau/TxnLog.h" +#include "Luau/TypeUtils.h" #include "Luau/Unifier.h" #include "Luau/ToString.h" @@ -19,10 +21,12 @@ struct TypeChecker2 : public AstVisitor const SourceModule* sourceModule; Module* module; InternalErrorReporter ice; // FIXME accept a pointer from Frontend + SingletonTypes& singletonTypes; TypeChecker2(const SourceModule* sourceModule, Module* module) : sourceModule(sourceModule) , module(module) + , singletonTypes(getSingletonTypes()) { } @@ -30,16 +34,30 @@ struct TypeChecker2 : public AstVisitor TypePackId lookupPack(AstExpr* expr) { + // If a type isn't in the type graph, it probably means that a recursion limit was exceeded. + // We'll just return anyType in these cases. Typechecking against any is very fast and this + // allows us not to think about this very much in the actual typechecking logic. TypePackId* tp = module->astTypePacks.find(expr); - LUAU_ASSERT(tp); - return follow(*tp); + if (tp) + return follow(*tp); + else + return singletonTypes.anyTypePack; } TypeId lookupType(AstExpr* expr) { + // If a type isn't in the type graph, it probably means that a recursion limit was exceeded. + // We'll just return anyType in these cases. Typechecking against any is very fast and this + // allows us not to think about this very much in the actual typechecking logic. TypeId* ty = module->astTypes.find(expr); - LUAU_ASSERT(ty); - return follow(*ty); + if (ty) + return follow(*ty); + + TypePackId* tp = module->astTypePacks.find(expr); + if (tp) + return flattenPack(*tp); + + return singletonTypes.anyType; } TypeId lookupAnnotation(AstType* annotation) @@ -78,7 +96,7 @@ struct TypeChecker2 : public AstVisitor bestLocation = scopeBounds; } } - else + else if (scopeBounds.begin > location.end) { // TODO: Is this sound? This relies on the fact that scopes are inserted // into the scope list in the order that they appear in the AST. @@ -147,16 +165,14 @@ struct TypeChecker2 : public AstVisitor for (size_t i = 0; i < count; ++i) { AstExpr* lhs = assign->vars.data[i]; - TypeId* lhsType = module->astTypes.find(lhs); - LUAU_ASSERT(lhsType); + TypeId lhsType = lookupType(lhs); AstExpr* rhs = assign->values.data[i]; - TypeId* rhsType = module->astTypes.find(rhs); - LUAU_ASSERT(rhsType); + TypeId rhsType = lookupType(rhs); - if (!isSubtype(*rhsType, *lhsType, ice)) + if (!isSubtype(rhsType, lhsType, ice)) { - reportError(TypeMismatch{*lhsType, *rhsType}, rhs->location); + reportError(TypeMismatch{lhsType, rhsType}, rhs->location); } } @@ -181,7 +197,7 @@ struct TypeChecker2 : public AstVisitor if (!ok) { for (const TypeError& e : u.errors) - module->errors.push_back(e); + reportError(e); } return true; @@ -189,10 +205,14 @@ struct TypeChecker2 : public AstVisitor bool visit(AstExprCall* call) override { + TypeArena arena; + Instantiation instantiation{TxnLog::empty(), &arena, TypeLevel{}}; + TypePackId expectedRetType = lookupPack(call); TypeId functionType = lookupType(call->func); + TypeId instantiatedFunctionType = instantiation.substitute(functionType).value_or(nullptr); + LUAU_ASSERT(functionType); - TypeArena arena; TypePack args; for (const auto& arg : call->args) { @@ -204,7 +224,7 @@ struct TypeChecker2 : public AstVisitor TypePackId argsTp = arena.addTypePack(args); FunctionTypeVar ftv{argsTp, expectedRetType}; TypeId expectedType = arena.addType(ftv); - if (!isSubtype(expectedType, functionType, ice)) + if (!isSubtype(expectedType, instantiatedFunctionType, ice)) { unfreeze(module->interfaceTypes); CloneState cloneState; @@ -252,16 +272,12 @@ struct TypeChecker2 : public AstVisitor // leftType must have a property called indexName->index - if (auto ttv = get(leftType)) + std::optional t = findTablePropertyRespectingMeta(module->errors, leftType, indexName->index.value, indexName->location); + if (t) { - auto it = ttv->props.find(indexName->index.value); - if (it == ttv->props.end()) + if (!isSubtype(resultType, *t, ice)) { - reportError(UnknownProperty{leftType, indexName->index.value}, indexName->location); - } - else if (!isSubtype(resultType, it->second.type, ice)) - { - reportError(TypeMismatch{resultType, it->second.type}, indexName->location); + reportError(TypeMismatch{resultType, *t}, indexName->location); } } else @@ -277,7 +293,7 @@ struct TypeChecker2 : public AstVisitor TypeId actualType = lookupType(number); TypeId numberType = getSingletonTypes().numberType; - if (!isSubtype(actualType, numberType, ice)) + if (!isSubtype(numberType, actualType, ice)) { reportError(TypeMismatch{actualType, numberType}, number->location); } @@ -290,7 +306,7 @@ struct TypeChecker2 : public AstVisitor TypeId actualType = lookupType(string); TypeId stringType = getSingletonTypes().stringType; - if (!isSubtype(actualType, stringType, ice)) + if (!isSubtype(stringType, actualType, ice)) { reportError(TypeMismatch{actualType, stringType}, string->location); } @@ -298,6 +314,41 @@ struct TypeChecker2 : public AstVisitor return true; } + /** Extract a TypeId for the first type of the provided pack. + * + * Note that this may require modifying some types. I hope this doesn't cause problems! + */ + TypeId flattenPack(TypePackId pack) + { + pack = follow(pack); + + while (auto tp = get(pack)) + { + if (tp->head.empty() && tp->tail) + pack = *tp->tail; + } + + if (auto ty = first(pack)) + return *ty; + else if (auto vtp = get(pack)) + return vtp->ty; + else if (auto ftp = get(pack)) + { + TypeId result = module->internalTypes.addType(FreeTypeVar{ftp->scope}); + TypePackId freeTail = module->internalTypes.addTypePack(FreeTypePack{ftp->scope}); + + TypePack& resultPack = asMutable(pack)->ty.emplace(); + resultPack.head.assign(1, result); + resultPack.tail = freeTail; + + return result; + } + else if (get(pack)) + return singletonTypes.errorRecoveryType(); + else + ice.ice("flattenPack got a weird pack!"); + } + bool visit(AstType* ty) override { return true; @@ -321,6 +372,11 @@ struct TypeChecker2 : public AstVisitor { module->errors.emplace_back(location, sourceModule->name, std::move(data)); } + + void reportError(TypeError e) + { + module->errors.emplace_back(std::move(e)); + } }; void check(const SourceModule& sourceModule, Module* module) diff --git a/Analysis/src/TypeInfer.cpp b/Analysis/src/TypeInfer.cpp index 44635e88..d9486a4f 100644 --- a/Analysis/src/TypeInfer.cpp +++ b/Analysis/src/TypeInfer.cpp @@ -38,11 +38,13 @@ LUAU_FASTFLAGVARIABLE(LuauReduceUnionRecursion, false) LUAU_FASTFLAGVARIABLE(LuauReturnAnyInsteadOfICE, false) // Eventually removed as false. LUAU_FASTFLAG(LuauNormalizeFlagIsConservative) LUAU_FASTFLAGVARIABLE(LuauReturnTypeInferenceInNonstrict, false) +LUAU_FASTFLAGVARIABLE(DebugLuauSharedSelf, false); LUAU_FASTFLAGVARIABLE(LuauAlwaysQuantify, false); LUAU_FASTFLAGVARIABLE(LuauReportErrorsOnIndexerKeyMismatch, false) LUAU_FASTFLAG(LuauQuantifyConstrained) LUAU_FASTFLAGVARIABLE(LuauFalsyPredicateReturnsNilInstead, false) LUAU_FASTFLAGVARIABLE(LuauNonCopyableTypeVarFields, false) +LUAU_FASTFLAGVARIABLE(LuauCheckLenMT, false) namespace Luau { @@ -238,7 +240,7 @@ static bool isMetamethod(const Name& name) { return name == "__index" || name == "__newindex" || name == "__call" || name == "__concat" || name == "__unm" || name == "__add" || name == "__sub" || name == "__mul" || name == "__div" || name == "__mod" || name == "__pow" || name == "__tostring" || - name == "__metatable" || name == "__eq" || name == "__lt" || name == "__le" || name == "__mode"; + name == "__metatable" || name == "__eq" || name == "__lt" || name == "__le" || name == "__mode" || name == "__iter" || name == "__len"; } size_t HashBoolNamePair::operator()(const std::pair& pair) const @@ -327,10 +329,19 @@ ModulePtr TypeChecker::checkWithoutRecursionCheck(const SourceModule& module, Mo currentModule->timeout = true; } + if (FFlag::DebugLuauSharedSelf) + { + for (auto& [ty, scope] : deferredQuantification) + Luau::quantify(ty, scope->level); + deferredQuantification.clear(); + } + if (get(follow(moduleScope->returnType))) moduleScope->returnType = addTypePack(TypePack{{}, std::nullopt}); else + { moduleScope->returnType = anyify(moduleScope, moduleScope->returnType, Location{}); + } for (auto& [_, typeFun] : moduleScope->exportedTypeBindings) typeFun.type = anyify(moduleScope, typeFun.type, Location{}); @@ -537,18 +548,43 @@ void TypeChecker::checkBlockWithoutRecursionCheck(const ScopePtr& scope, const A } else if (auto fun = (*protoIter)->as()) { + std::optional selfType; std::optional expectedType; - if (!fun->func->self) + if (FFlag::DebugLuauSharedSelf) { if (auto name = fun->name->as()) { - TypeId exprTy = checkExpr(scope, *name->expr).type; - expectedType = getIndexTypeFromType(scope, exprTy, name->index.value, name->indexLocation, false); + TypeId baseTy = checkExpr(scope, *name->expr).type; + tablify(baseTy); + + if (!fun->func->self) + expectedType = getIndexTypeFromType(scope, baseTy, name->index.value, name->indexLocation, false); + else if (auto ttv = getMutableTableType(baseTy)) + { + if (!baseTy->persistent && ttv->state != TableState::Sealed && !ttv->selfTy) + { + ttv->selfTy = anyIfNonstrict(freshType(ttv->level)); + deferredQuantification.push_back({baseTy, scope}); + } + + selfType = ttv->selfTy; + } + } + } + else + { + if (!fun->func->self) + { + if (auto name = fun->name->as()) + { + TypeId exprTy = checkExpr(scope, *name->expr).type; + expectedType = getIndexTypeFromType(scope, exprTy, name->index.value, name->indexLocation, false); + } } } - auto pair = checkFunctionSignature(scope, subLevel, *fun->func, fun->name->location, expectedType); + auto pair = checkFunctionSignature(scope, subLevel, *fun->func, fun->name->location, selfType, expectedType); auto [funTy, funScope] = pair; functionDecls[*protoIter] = pair; @@ -560,7 +596,7 @@ void TypeChecker::checkBlockWithoutRecursionCheck(const ScopePtr& scope, const A } else if (auto fun = (*protoIter)->as()) { - auto pair = checkFunctionSignature(scope, subLevel, *fun->func, fun->name->location, std::nullopt); + auto pair = checkFunctionSignature(scope, subLevel, *fun->func, fun->name->location, std::nullopt, std::nullopt); auto [funTy, funScope] = pair; functionDecls[*protoIter] = pair; @@ -2076,7 +2112,7 @@ WithPredicate TypeChecker::checkExpr(const ScopePtr& scope, const AstExp WithPredicate TypeChecker::checkExpr(const ScopePtr& scope, const AstExprFunction& expr, std::optional expectedType) { - auto [funTy, funScope] = checkFunctionSignature(scope, 0, expr, std::nullopt, expectedType); + auto [funTy, funScope] = checkFunctionSignature(scope, 0, expr, std::nullopt, std::nullopt, expectedType); checkFunctionBody(funScope, funTy, expr); @@ -2296,6 +2332,8 @@ WithPredicate TypeChecker::checkExpr(const ScopePtr& scope, const AstExp state.tryUnify(actualFunctionType, expectedFunctionType, /*isFunctionCall*/ true); state.log.commit(); + reportErrors(state.errors); + TypeId retType = first(retTypePack).value_or(nilType); if (!state.errors.empty()) retType = errorRecoveryType(retType); @@ -2322,6 +2360,23 @@ WithPredicate TypeChecker::checkExpr(const ScopePtr& scope, const AstExp DenseHashSet seen{nullptr}; + if (FFlag::LuauCheckLenMT && typeCouldHaveMetatable(operandType)) + { + if (auto fnt = findMetatableEntry(operandType, "__len", expr.location)) + { + TypeId actualFunctionType = instantiate(scope, *fnt, expr.location); + TypePackId arguments = addTypePack({operandType}); + TypePackId retTypePack = addTypePack({numberType}); + TypeId expectedFunctionType = addType(FunctionTypeVar(scope->level, arguments, retTypePack)); + + Unifier state = mkUnifier(expr.location); + state.tryUnify(actualFunctionType, expectedFunctionType, /*isFunctionCall*/ true); + state.log.commit(); + + reportErrors(state.errors); + } + } + if (!hasLength(operandType, seen, &recursionCount)) reportError(TypeError{expr.location, NotATable{operandType}}); @@ -2530,17 +2585,15 @@ TypeId TypeChecker::checkRelationalOperation( } } - if (!matches) { reportError( expr.location, GenericError{format("Types %s and %s cannot be compared with %s because they do not have the same metatable", - toString(lhsType).c_str(), toString(rhsType).c_str(), toString(expr.op).c_str())}); + toString(lhsType).c_str(), toString(rhsType).c_str(), toString(expr.op).c_str())}); return errorRecoveryType(booleanType); } } - if (leftMetatable) { std::optional metamethod = findMetatableEntry(lhsType, metamethodName, expr.location); @@ -3139,8 +3192,8 @@ TypeId TypeChecker::checkFunctionName(const ScopePtr& scope, AstExpr& funName, T // `(X) -> Y...`, but after typechecking the body, we cam unify `Y...` with `X` // to get type `(X) -> X`, then we quantify the free types to get the final // generic type `(a) -> a`. -std::pair TypeChecker::checkFunctionSignature( - const ScopePtr& scope, int subLevel, const AstExprFunction& expr, std::optional originalName, std::optional expectedType) +std::pair TypeChecker::checkFunctionSignature(const ScopePtr& scope, int subLevel, const AstExprFunction& expr, + std::optional originalName, std::optional selfType, std::optional expectedType) { ScopePtr funScope = childFunctionScope(scope, expr.location, subLevel); @@ -3241,12 +3294,25 @@ std::pair TypeChecker::checkFunctionSignature( funScope->returnType = retPack; - if (expr.self) + if (FFlag::DebugLuauSharedSelf) { - // TODO: generic self types: CLI-39906 - TypeId selfType = anyIfNonstrict(freshType(funScope)); - funScope->bindings[expr.self] = {selfType, expr.self->location}; - argTypes.push_back(selfType); + if (expr.self) + { + // TODO: generic self types: CLI-39906 + TypeId selfTy = anyIfNonstrict(selfType ? *selfType : freshType(funScope)); + funScope->bindings[expr.self] = {selfTy, expr.self->location}; + argTypes.push_back(selfTy); + } + } + else + { + if (expr.self) + { + // TODO: generic self types: CLI-39906 + TypeId selfType = anyIfNonstrict(freshType(funScope)); + funScope->bindings[expr.self] = {selfType, expr.self->location}; + argTypes.push_back(selfType); + } } // Prepare expected argument type iterators if we have an expected function type @@ -4457,25 +4523,43 @@ TypeId TypeChecker::quantify(const ScopePtr& scope, TypeId ty, Location location { ty = follow(ty); - const FunctionTypeVar* ftv = get(ty); - - if (FFlag::LuauAlwaysQuantify) + if (FFlag::DebugLuauSharedSelf) { - if (ftv) + if (auto ftv = get(ty)) Luau::quantify(ty, scope->level); + else if (auto ttv = getTableType(ty); ttv && ttv->selfTy) + Luau::quantify(ty, scope->level); + + if (FFlag::LuauLowerBoundsCalculation) + { + auto [t, ok] = Luau::normalize(ty, currentModule, *iceHandler); + if (!ok) + reportError(location, NormalizationTooComplex{}); + return t; + } } else { - if (ftv && ftv->generics.empty() && ftv->genericPacks.empty()) - Luau::quantify(ty, scope->level); - } + const FunctionTypeVar* ftv = get(ty); - if (FFlag::LuauLowerBoundsCalculation && ftv) - { - auto [t, ok] = Luau::normalize(ty, currentModule, *iceHandler); - if (!ok) - reportError(location, NormalizationTooComplex{}); - return t; + if (FFlag::LuauAlwaysQuantify) + { + if (ftv) + Luau::quantify(ty, scope->level); + } + else + { + if (ftv && ftv->generics.empty() && ftv->genericPacks.empty()) + Luau::quantify(ty, scope->level); + } + + if (FFlag::LuauLowerBoundsCalculation && ftv) + { + auto [t, ok] = Luau::normalize(ty, currentModule, *iceHandler); + if (!ok) + reportError(location, NormalizationTooComplex{}); + return t; + } } return ty; diff --git a/Analysis/src/Unifier.cpp b/Analysis/src/Unifier.cpp index 6147e118..0792a350 100644 --- a/Analysis/src/Unifier.cpp +++ b/Analysis/src/Unifier.cpp @@ -740,7 +740,7 @@ void Unifier::tryUnifyTypeWithIntersection(TypeId subTy, TypeId superTy, const I std::optional unificationTooComplex; std::optional firstFailedOption; - // T <: A & B if A <: T and B <: T + // T <: A & B if T <: A and T <: B for (TypeId type : uv->parts) { Unifier innerState = makeChildUnifier(); @@ -765,7 +765,7 @@ void Unifier::tryUnifyTypeWithIntersection(TypeId subTy, TypeId superTy, const I void Unifier::tryUnifyIntersectionWithType(TypeId subTy, const IntersectionTypeVar* uv, TypeId superTy, bool cacheEnabled, bool isFunctionCall) { - // A & B <: T if T <: A or T <: B + // A & B <: T if A <: T or B <: T bool found = false; std::optional unificationTooComplex; diff --git a/Ast/src/Parser.cpp b/Ast/src/Parser.cpp index 95bce3ee..70c92555 100644 --- a/Ast/src/Parser.cpp +++ b/Ast/src/Parser.cpp @@ -5,6 +5,9 @@ #include +#include +#include + // Warning: If you are introducing new syntax, ensure that it is behind a separate // flag so that we don't break production games by reverting syntax changes. // See docs/SyntaxChanges.md for an explanation. @@ -14,6 +17,18 @@ LUAU_FASTINTVARIABLE(LuauParseErrorLimit, 100) LUAU_FASTFLAGVARIABLE(LuauParserFunctionKeywordAsTypeHelp, false) LUAU_FASTFLAGVARIABLE(LuauReturnTypeTokenConfusion, false) +LUAU_FASTFLAGVARIABLE(LuauFixNamedFunctionParse, false) +LUAU_DYNAMIC_FASTFLAGVARIABLE(LuaReportParseWrongNamedType, false) + +bool lua_telemetry_parsed_named_non_function_type = false; + +LUAU_FASTFLAGVARIABLE(LuauErrorParseIntegerIssues, false) +LUAU_DYNAMIC_FASTFLAGVARIABLE(LuaReportParseIntegerIssues, false) + +bool lua_telemetry_parsed_out_of_range_bin_integer = false; +bool lua_telemetry_parsed_out_of_range_hex_integer = false; +bool lua_telemetry_parsed_double_prefix_hex_integer = false; + namespace Luau { @@ -1330,7 +1345,7 @@ AstTypeOrPack Parser::parseFunctionTypeAnnotation(bool allowPack) { incrementRecursionCounter("type annotation"); - bool monomorphic = lexer.current().type != '<'; + bool forceFunctionType = lexer.current().type == '<'; Lexeme begin = lexer.current(); @@ -1355,21 +1370,33 @@ AstTypeOrPack Parser::parseFunctionTypeAnnotation(bool allowPack) AstArray paramTypes = copy(params); + if (FFlag::LuauFixNamedFunctionParse && !names.empty()) + forceFunctionType = true; + bool returnTypeIntroducer = FFlag::LuauReturnTypeTokenConfusion ? lexer.current().type == Lexeme::SkinnyArrow || lexer.current().type == ':' : false; // Not a function at all. Just a parenthesized type. Or maybe a type pack with a single element - if (params.size() == 1 && !varargAnnotation && monomorphic && + if (params.size() == 1 && !varargAnnotation && !forceFunctionType && (FFlag::LuauReturnTypeTokenConfusion ? !returnTypeIntroducer : lexer.current().type != Lexeme::SkinnyArrow)) { + if (DFFlag::LuaReportParseWrongNamedType && !names.empty()) + lua_telemetry_parsed_named_non_function_type = true; + if (allowPack) return {{}, allocator.alloc(begin.location, AstTypeList{paramTypes, nullptr})}; else return {params[0], {}}; } - if ((FFlag::LuauReturnTypeTokenConfusion ? !returnTypeIntroducer : lexer.current().type != Lexeme::SkinnyArrow) && monomorphic && allowPack) + if ((FFlag::LuauReturnTypeTokenConfusion ? !returnTypeIntroducer : lexer.current().type != Lexeme::SkinnyArrow) && !forceFunctionType && + allowPack) + { + if (DFFlag::LuaReportParseWrongNamedType && !names.empty()) + lua_telemetry_parsed_named_non_function_type = true; + return {{}, allocator.alloc(begin.location, AstTypeList{paramTypes, varargAnnotation})}; + } AstArray> paramNames = copy(names); @@ -2010,7 +2037,63 @@ AstExpr* Parser::parseAssertionExpr() return expr; } -static bool parseNumber(double& result, const char* data) +static const char* parseInteger(double& result, const char* data, int base) +{ + char* end = nullptr; + unsigned long long value = strtoull(data, &end, base); + + if (value == ULLONG_MAX && errno == ERANGE) + { + // 'errno' might have been set before we called 'strtoull', but we don't want the overhead of resetting a TLS variable on each call + // so we only reset it when we get a result that might be an out-of-range error and parse again to make sure + errno = 0; + value = strtoull(data, &end, base); + + if (errno == ERANGE) + { + if (DFFlag::LuaReportParseIntegerIssues) + { + if (base == 2) + lua_telemetry_parsed_out_of_range_bin_integer = true; + else + lua_telemetry_parsed_out_of_range_hex_integer = true; + } + + if (FFlag::LuauErrorParseIntegerIssues) + return "Integer number value is out of range"; + } + } + + result = double(value); + return *end == 0 ? nullptr : "Malformed number"; +} + +static const char* parseNumber(double& result, const char* data) +{ + // binary literal + if (data[0] == '0' && (data[1] == 'b' || data[1] == 'B') && data[2]) + return parseInteger(result, data + 2, 2); + + // hexadecimal literal + if (data[0] == '0' && (data[1] == 'x' || data[1] == 'X') && data[2]) + { + if (DFFlag::LuaReportParseIntegerIssues && data[2] == '0' && (data[3] == 'x' || data[3] == 'X')) + lua_telemetry_parsed_double_prefix_hex_integer = true; + + if (FFlag::LuauErrorParseIntegerIssues) + return parseInteger(result, data, 16); // keep prefix, it's handled by 'strtoull' + else + return parseInteger(result, data + 2, 16); + } + + char* end = nullptr; + double value = strtod(data, &end); + + result = value; + return *end == 0 ? nullptr : "Malformed number"; +} + +static bool parseNumber_DEPRECATED(double& result, const char* data) { // binary literal if (data[0] == '0' && (data[1] == 'b' || data[1] == 'B') && data[2]) @@ -2080,18 +2163,37 @@ AstExpr* Parser::parseSimpleExpr() scratchData.erase(std::remove(scratchData.begin(), scratchData.end(), '_'), scratchData.end()); } - double value = 0; - if (parseNumber(value, scratchData.c_str())) + if (DFFlag::LuaReportParseIntegerIssues || FFlag::LuauErrorParseIntegerIssues) { - nextLexeme(); + double value = 0; + if (const char* error = parseNumber(value, scratchData.c_str())) + { + nextLexeme(); - return allocator.alloc(start, value); + return reportExprError(start, {}, "%s", error); + } + else + { + nextLexeme(); + + return allocator.alloc(start, value); + } } else { - nextLexeme(); + double value = 0; + if (parseNumber_DEPRECATED(value, scratchData.c_str())) + { + nextLexeme(); - return reportExprError(start, {}, "Malformed number"); + return allocator.alloc(start, value); + } + else + { + nextLexeme(); + + return reportExprError(start, {}, "Malformed number"); + } } } else if (lexer.current().type == Lexeme::RawString || lexer.current().type == Lexeme::QuotedString) diff --git a/Common/include/Luau/Bytecode.h b/Common/include/Luau/Bytecode.h index 218bb5d5..0cb7e1d9 100644 --- a/Common/include/Luau/Bytecode.h +++ b/Common/include/Luau/Bytecode.h @@ -276,7 +276,7 @@ enum LuauOpcode // FORGLOOP: adjust loop variables for one iteration of a generic for loop, jump back to the loop header if loop needs to continue // A: target register; generic for loops assume a register layout [generator, state, index, variables...] // D: jump offset (-32768..32767) - // AUX: variable count (1..255) + // AUX: variable count (1..255) in the low 8 bits, high bit indicates whether to use ipairs-style traversal in the fast path // loop variables are adjusted by calling generator(state, index) and expecting it to return a tuple that's copied to the user variables // the first variable is then copied into index; generator/state are immutable, index isn't visible to user code LOP_FORGLOOP, @@ -490,6 +490,9 @@ enum LuauBuiltinFunction // select(_, ...) LBF_SELECT_VARARG, + + // rawlen + LBF_RAWLEN, }; // Capture type, used in LOP_CAPTURE diff --git a/Compiler/src/Builtins.cpp b/Compiler/src/Builtins.cpp index ff753112..6bd24b6d 100644 --- a/Compiler/src/Builtins.cpp +++ b/Compiler/src/Builtins.cpp @@ -4,6 +4,8 @@ #include "Luau/Bytecode.h" #include "Luau/Compiler.h" +LUAU_FASTFLAGVARIABLE(LuauCompileRawlen, false) + namespace Luau { namespace Compile @@ -58,6 +60,8 @@ int getBuiltinFunctionId(const Builtin& builtin, const CompileOptions& options) return LBF_RAWGET; if (builtin.isGlobal("rawequal")) return LBF_RAWEQUAL; + if (FFlag::LuauCompileRawlen && builtin.isGlobal("rawlen")) + return LBF_RAWLEN; if (builtin.isGlobal("unpack")) return LBF_TABLE_UNPACK; diff --git a/Compiler/src/BytecodeBuilder.cpp b/Compiler/src/BytecodeBuilder.cpp index 301cf255..5e2669b2 100644 --- a/Compiler/src/BytecodeBuilder.cpp +++ b/Compiler/src/BytecodeBuilder.cpp @@ -1302,20 +1302,22 @@ void BytecodeBuilder::validate() const case LOP_FORNPREP: case LOP_FORNLOOP: - VREG(LUAU_INSN_A(insn) + 2); // for loop protocol: A, A+1, A+2 are used for iteration + // for loop protocol: A, A+1, A+2 are used for iteration + VREG(LUAU_INSN_A(insn) + 2); VJUMP(LUAU_INSN_D(insn)); break; case LOP_FORGPREP: - VREG(LUAU_INSN_A(insn) + 2 + 1); // forg loop protocol: A, A+1, A+2 are used for iteration protocol; A+3, ... are loop variables + // forg loop protocol: A, A+1, A+2 are used for iteration protocol; A+3, ... are loop variables + VREG(LUAU_INSN_A(insn) + 2 + 1); VJUMP(LUAU_INSN_D(insn)); break; case LOP_FORGLOOP: - VREG( - LUAU_INSN_A(insn) + 2 + insns[i + 1]); // forg loop protocol: A, A+1, A+2 are used for iteration protocol; A+3, ... are loop variables + // forg loop protocol: A, A+1, A+2 are used for iteration protocol; A+3, ... are loop variables + VREG(LUAU_INSN_A(insn) + 2 + uint8_t(insns[i + 1])); VJUMP(LUAU_INSN_D(insn)); - LUAU_ASSERT(insns[i + 1] >= 1); + LUAU_ASSERT(uint8_t(insns[i + 1]) >= 1); break; case LOP_FORGPREP_INEXT: @@ -1679,7 +1681,8 @@ void BytecodeBuilder::dumpInstruction(const uint32_t* code, std::string& result, break; case LOP_FORGLOOP: - formatAppend(result, "FORGLOOP R%d L%d %d\n", LUAU_INSN_A(insn), targetLabel, *code++); + formatAppend(result, "FORGLOOP R%d L%d %d%s\n", LUAU_INSN_A(insn), targetLabel, uint8_t(*code), int(*code) < 0 ? " [inext]" : ""); + code++; break; case LOP_FORGPREP_INEXT: diff --git a/Compiler/src/Compiler.cpp b/Compiler/src/Compiler.cpp index e732256b..d7c8155c 100644 --- a/Compiler/src/Compiler.cpp +++ b/Compiler/src/Compiler.cpp @@ -23,6 +23,8 @@ LUAU_FASTINTVARIABLE(LuauCompileInlineThreshold, 25) LUAU_FASTINTVARIABLE(LuauCompileInlineThresholdMaxBoost, 300) LUAU_FASTINTVARIABLE(LuauCompileInlineDepth, 5) +LUAU_FASTFLAGVARIABLE(LuauCompileNoIpairs, false) + namespace Luau { @@ -2665,7 +2667,7 @@ struct Compiler if (builtin.isGlobal("ipairs")) // for .. in ipairs(t) { skipOp = LOP_FORGPREP_INEXT; - loopOp = LOP_FORGLOOP_INEXT; + loopOp = FFlag::LuauCompileNoIpairs ? LOP_FORGLOOP : LOP_FORGLOOP_INEXT; } else if (builtin.isGlobal("pairs")) // for .. in pairs(t) { @@ -2709,8 +2711,16 @@ struct Compiler bytecode.emitAD(loopOp, regs, 0); + if (FFlag::LuauCompileNoIpairs) + { + // TODO: remove loopOp as it's a constant now + LUAU_ASSERT(loopOp == LOP_FORGLOOP); + + // FORGLOOP uses aux to encode variable count and fast path flag for ipairs traversal in the high bit + bytecode.emitAux((skipOp == LOP_FORGPREP_INEXT ? 0x80000000 : 0) | uint32_t(stat->vars.size)); + } // note: FORGLOOP needs variable count encoded in AUX field, other loop instructions assume a fixed variable count - if (loopOp == LOP_FORGLOOP) + else if (loopOp == LOP_FORGLOOP) bytecode.emitAux(uint32_t(stat->vars.size)); size_t endLabel = bytecode.emitLabel(); @@ -3341,7 +3351,7 @@ struct Compiler std::vector upvals; }; - struct ReturnVisitor: AstVisitor + struct ReturnVisitor : AstVisitor { Compiler* self; bool returnsOne = true; diff --git a/VM/src/lbaselib.cpp b/VM/src/lbaselib.cpp index f7917611..4fc5033e 100644 --- a/VM/src/lbaselib.cpp +++ b/VM/src/lbaselib.cpp @@ -11,6 +11,8 @@ #include #include +LUAU_FASTFLAG(LuauLenTM) + static void writestring(const char* s, size_t l) { fwrite(s, 1, l, stdout); @@ -178,6 +180,18 @@ static int luaB_rawset(lua_State* L) return 1; } +static int luaB_rawlen(lua_State* L) +{ + if (!FFlag::LuauLenTM) + luaL_error(L, "'rawlen' is not available"); + + int tt = lua_type(L, 1); + luaL_argcheck(L, tt == LUA_TTABLE || tt == LUA_TSTRING, 1, "table or string expected"); + int len = lua_objlen(L, 1); + lua_pushinteger(L, len); + return 1; +} + static int luaB_gcinfo(lua_State* L) { lua_pushinteger(L, lua_gc(L, LUA_GCCOUNT, 0)); @@ -428,6 +442,7 @@ static const luaL_Reg base_funcs[] = { {"rawequal", luaB_rawequal}, {"rawget", luaB_rawget}, {"rawset", luaB_rawset}, + {"rawlen", luaB_rawlen}, {"select", luaB_select}, {"setfenv", luaB_setfenv}, {"setmetatable", luaB_setmetatable}, diff --git a/VM/src/lbuiltins.cpp b/VM/src/lbuiltins.cpp index deaf1407..e98660a7 100644 --- a/VM/src/lbuiltins.cpp +++ b/VM/src/lbuiltins.cpp @@ -1117,6 +1117,27 @@ static int luauF_select(lua_State* L, StkId res, TValue* arg0, int nresults, Stk return -1; } +static int luauF_rawlen(lua_State* L, StkId res, TValue* arg0, int nresults, StkId args, int nparams) +{ + if (nparams >= 1 && nresults <= 1) + { + if (ttistable(arg0)) + { + Table* h = hvalue(arg0); + setnvalue(res, double(luaH_getn(h))); + return 1; + } + else if (ttisstring(arg0)) + { + TString* ts = tsvalue(arg0); + setnvalue(res, double(ts->len)); + return 1; + } + } + + return -1; +} + luau_FastFunction luauF_table[256] = { NULL, luauF_assert, @@ -1188,4 +1209,6 @@ luau_FastFunction luauF_table[256] = { luauF_countrz, luauF_select, + + luauF_rawlen, }; diff --git a/VM/src/ldebug.h b/VM/src/ldebug.h index cf905e9f..75bb8dcc 100644 --- a/VM/src/ldebug.h +++ b/VM/src/ldebug.h @@ -19,6 +19,7 @@ LUAI_FUNC l_noret luaG_concaterror(lua_State* L, StkId p1, StkId p2); LUAI_FUNC l_noret luaG_aritherror(lua_State* L, const TValue* p1, const TValue* p2, TMS op); LUAI_FUNC l_noret luaG_ordererror(lua_State* L, const TValue* p1, const TValue* p2, TMS op); LUAI_FUNC l_noret luaG_indexerror(lua_State* L, const TValue* p1, const TValue* p2); + LUAI_FUNC LUA_PRINTF_ATTR(2, 3) l_noret luaG_runerrorL(lua_State* L, const char* fmt, ...); LUAI_FUNC void luaG_pusherror(lua_State* L, const char* error); diff --git a/VM/src/ltm.cpp b/VM/src/ltm.cpp index e7df4e53..49982b28 100644 --- a/VM/src/ltm.cpp +++ b/VM/src/ltm.cpp @@ -39,6 +39,7 @@ const char* const luaT_eventname[] = { "__namecall", "__call", "__iter", + "__len", "__eq", @@ -52,7 +53,6 @@ const char* const luaT_eventname[] = { "__unm", - "__len", "__lt", "__le", "__concat", diff --git a/VM/src/ltm.h b/VM/src/ltm.h index a5223941..e11ddb3a 100644 --- a/VM/src/ltm.h +++ b/VM/src/ltm.h @@ -18,6 +18,7 @@ typedef enum TM_NAMECALL, TM_CALL, TM_ITER, + TM_LEN, TM_EQ, /* last tag method with `fast' access */ @@ -31,7 +32,6 @@ typedef enum TM_UNM, - TM_LEN, TM_LT, TM_LE, TM_CONCAT, diff --git a/VM/src/lvmexecute.cpp b/VM/src/lvmexecute.cpp index e0a96474..85829ca1 100644 --- a/VM/src/lvmexecute.cpp +++ b/VM/src/lvmexecute.cpp @@ -16,6 +16,8 @@ #include +LUAU_FASTFLAGVARIABLE(LuauLenTM, false) + // Disable c99-designator to avoid the warning in CGOTO dispatch table #ifdef __clang__ #if __has_warning("-Wc99-designator") @@ -2082,13 +2084,25 @@ static void luau_execute(lua_State* L) // fast-path #1: tables if (ttistable(rb)) { - setnvalue(ra, cast_num(luaH_getn(hvalue(rb)))); - VM_NEXT(); + Table* h = hvalue(rb); + + if (!FFlag::LuauLenTM || fastnotm(h->metatable, TM_LEN)) + { + setnvalue(ra, cast_num(luaH_getn(h))); + VM_NEXT(); + } + else + { + // slow-path, may invoke C/Lua via metamethods + VM_PROTECT(luaV_dolen(L, ra, rb)); + VM_NEXT(); + } } // fast-path #2: strings (not very important but easy to do) else if (ttisstring(rb)) { - setnvalue(ra, cast_num(tsvalue(rb)->len)); + TString* ts = tsvalue(rb); + setnvalue(ra, cast_num(ts->len)); VM_NEXT(); } else @@ -2226,6 +2240,15 @@ static void luau_execute(lua_State* L) VM_PROTECT(luaD_call(L, ra, 3)); L->top = L->ci->top; + + /* recompute ra since stack might have been reallocated */ + ra = VM_REG(LUAU_INSN_A(insn)); + + /* protect against __iter returning nil, since nil is used as a marker for builtin iteration in FORGLOOP */ + if (ttisnil(ra)) + { + VM_PROTECT(luaG_typeerror(L, ra, "call")); + } } else if (fasttm(L, mt, TM_CALL)) { @@ -2258,27 +2281,38 @@ static void luau_execute(lua_State* L) uint32_t aux = *pc; // fast-path: builtin table iteration - if (ttisnil(ra) && ttistable(ra + 1) && ttislightuserdata(ra + 2)) + // note: ra=nil guarantees ra+1=table and ra+2=userdata because of the setup by FORGPREP* opcodes + // TODO: remove the table check per guarantee above + if (ttisnil(ra) && ttistable(ra + 1)) { Table* h = hvalue(ra + 1); int index = int(reinterpret_cast(pvalue(ra + 2))); int sizearray = h->sizearray; - int sizenode = 1 << h->lsizenode; // clear extra variables since we might have more than two - if (LUAU_UNLIKELY(aux > 2)) + // note: while aux encodes ipairs bit, when set we always use 2 variables, so it's safe to check this via a signed comparison + if (LUAU_UNLIKELY(int(aux) > 2)) for (int i = 2; i < int(aux); ++i) setnilvalue(ra + 3 + i); + // terminate ipairs-style traversal early when encountering nil + if (int(aux) < 0 && (unsigned(index) >= unsigned(sizearray) || ttisnil(&h->array[index]))) + { + pc++; + VM_NEXT(); + } + // first we advance index through the array portion while (unsigned(index) < unsigned(sizearray)) { - if (!ttisnil(&h->array[index])) + TValue* e = &h->array[index]; + + if (!ttisnil(e)) { setpvalue(ra + 2, reinterpret_cast(uintptr_t(index + 1))); setnvalue(ra + 3, double(index + 1)); - setobj2s(L, ra + 4, &h->array[index]); + setobj2s(L, ra + 4, e); pc += LUAU_INSN_D(insn); LUAU_ASSERT(unsigned(pc - cl->l.p->code) < unsigned(cl->l.p->sizecode)); @@ -2288,6 +2322,8 @@ static void luau_execute(lua_State* L) index++; } + int sizenode = 1 << h->lsizenode; + // then we advance index through the hash portion while (unsigned(index - sizearray) < unsigned(sizenode)) { @@ -2321,7 +2357,7 @@ static void luau_execute(lua_State* L) L->top = ra + 3 + 3; /* func + 2 args (state and index) */ LUAU_ASSERT(L->top <= L->stack_last); - VM_PROTECT(luaD_call(L, ra + 3, aux)); + VM_PROTECT(luaD_call(L, ra + 3, uint8_t(aux))); L->top = L->ci->top; // recompute ra since stack might have been reallocated diff --git a/VM/src/lvmutils.cpp b/VM/src/lvmutils.cpp index 8a18a4d4..b9e762eb 100644 --- a/VM/src/lvmutils.cpp +++ b/VM/src/lvmutils.cpp @@ -10,6 +10,9 @@ #include "lnumutils.h" #include +#include + +LUAU_FASTFLAG(LuauLenTM) /* limit for table tag-method chains (to avoid loops) */ #define MAXTAGLOOP 100 @@ -51,7 +54,7 @@ const float* luaV_tovector(const TValue* obj) return nullptr; } -static void callTMres(lua_State* L, StkId res, const TValue* f, const TValue* p1, const TValue* p2) +static StkId callTMres(lua_State* L, StkId res, const TValue* f, const TValue* p1, const TValue* p2) { ptrdiff_t result = savestack(L, res); // using stack room beyond top is technically safe here, but for very complicated reasons: @@ -71,6 +74,7 @@ static void callTMres(lua_State* L, StkId res, const TValue* f, const TValue* p1 res = restorestack(L, result); L->top--; setobjs2s(L, res, L->top); + return res; } static void callTM(lua_State* L, const TValue* f, const TValue* p1, const TValue* p2, const TValue* p3) @@ -472,22 +476,56 @@ void luaV_doarith(lua_State* L, StkId ra, const TValue* rb, const TValue* rc, TM void luaV_dolen(lua_State* L, StkId ra, const TValue* rb) { + if (!FFlag::LuauLenTM) + { + switch (ttype(rb)) + { + case LUA_TTABLE: + { + setnvalue(ra, cast_num(luaH_getn(hvalue(rb)))); + break; + } + case LUA_TSTRING: + { + setnvalue(ra, cast_num(tsvalue(rb)->len)); + break; + } + default: + { /* try metamethod */ + if (!call_binTM(L, rb, luaO_nilobject, ra, TM_LEN)) + luaG_typeerror(L, rb, "get length of"); + } + } + return; + } + + const TValue* tm = NULL; switch (ttype(rb)) { case LUA_TTABLE: { - setnvalue(ra, cast_num(luaH_getn(hvalue(rb)))); + Table* h = hvalue(rb); + if ((tm = fasttm(L, h->metatable, TM_LEN)) == NULL) + { + setnvalue(ra, cast_num(luaH_getn(h))); + return; + } break; } case LUA_TSTRING: { - setnvalue(ra, cast_num(tsvalue(rb)->len)); - break; + TString* ts = tsvalue(rb); + setnvalue(ra, cast_num(ts->len)); + return; } default: - { /* try metamethod */ - if (!call_binTM(L, rb, luaO_nilobject, ra, TM_LEN)) - luaG_typeerror(L, rb, "get length of"); - } + tm = luaT_gettmbyobj(L, rb, TM_LEN); } + + if (ttisnil(tm)) + luaG_typeerror(L, rb, "get length of"); + + StkId res = callTMres(L, ra, tm, rb, luaO_nilobject); + if (!ttisnumber(res)) + luaG_runerror(L, "'__len' must return a number"); /* note, we can't access rb since stack may have been reallocated */ } diff --git a/fuzz/protoprint.cpp b/fuzz/protoprint.cpp index 66a89f24..d4d52276 100644 --- a/fuzz/protoprint.cpp +++ b/fuzz/protoprint.cpp @@ -11,6 +11,7 @@ static const std::string kNames[] = { "__div", "__eq", "__index", + "__iter", "__le", "__len", "__lt", @@ -41,13 +42,18 @@ static const std::string kNames[] = { "ceil", "char", "charpattern", + "clamp", "clock", + "clone", + "close", "codepoint", "codes", "concat", "coroutine", "cos", "cosh", + "countlz", + "countrz", "create", "date", "debug", @@ -63,6 +69,7 @@ static const std::string kNames[] = { "foreachi", "format", "frexp", + "freeze", "function", "gcinfo", "getfenv", @@ -72,8 +79,10 @@ static const std::string kNames[] = { "gmatch", "gsub", "huge", + "info", "insert", "ipairs", + "isfrozen", "isyieldable", "ldexp", "len", @@ -93,6 +102,7 @@ static const std::string kNames[] = { "newproxy", "next", "nil", + "noise", "number", "offset", "os", @@ -121,6 +131,7 @@ static const std::string kNames[] = { "select", "setfenv", "setmetatable", + "sign", "sin", "sinh", "sort", diff --git a/tests/Compiler.test.cpp b/tests/Compiler.test.cpp index 655e48cb..fafafd71 100644 --- a/tests/Compiler.test.cpp +++ b/tests/Compiler.test.cpp @@ -261,6 +261,8 @@ L1: RETURN R0 0 TEST_CASE("ForBytecode") { + ScopedFastFlag sff("LuauCompileNoIpairs", true); + // basic for loop: variable directly refers to internal iteration index (R2) CHECK_EQ("\n" + compileFunction0("for i=1,5 do print(i) end"), R"( LOADN R2 1 @@ -313,7 +315,7 @@ L0: GETIMPORT R5 3 MOVE R6 R3 MOVE R7 R4 CALL R5 2 0 -L1: FORGLOOP_INEXT R0 L0 +L1: FORGLOOP R0 L0 2 [inext] RETURN R0 0 )"); @@ -347,13 +349,15 @@ RETURN R0 0 TEST_CASE("ForBytecodeBuiltin") { + ScopedFastFlag sff("LuauCompileNoIpairs", true); + // we generally recognize builtins like pairs/ipairs and emit special opcodes CHECK_EQ("\n" + compileFunction0("for k,v in ipairs({}) do end"), R"( GETIMPORT R0 1 NEWTABLE R1 0 0 CALL R0 1 3 FORGPREP_INEXT R0 L0 -L0: FORGLOOP_INEXT R0 L0 +L0: FORGLOOP R0 L0 2 [inext] RETURN R0 0 )"); @@ -364,7 +368,7 @@ MOVE R1 R0 NEWTABLE R2 0 0 CALL R1 1 3 FORGPREP_INEXT R1 L0 -L0: FORGLOOP_INEXT R1 L0 +L0: FORGLOOP R1 L0 2 [inext] RETURN R0 0 )"); @@ -374,7 +378,7 @@ GETUPVAL R0 0 NEWTABLE R1 0 0 CALL R0 1 3 FORGPREP_INEXT R0 L0 -L0: FORGLOOP_INEXT R0 L0 +L0: FORGLOOP R0 L0 2 [inext] RETURN R0 0 )"); @@ -2107,6 +2111,8 @@ RETURN R3 -1 TEST_CASE("UpvaluesLoopsBytecode") { + ScopedFastFlag sff("LuauCompileNoIpairs", true); + CHECK_EQ("\n" + compileFunction(R"( function test() for i=1,10 do @@ -2169,7 +2175,7 @@ JUMPIFNOT R5 L1 CLOSEUPVALS R3 JUMP L3 L1: CLOSEUPVALS R3 -L2: FORGLOOP_INEXT R0 L0 +L2: FORGLOOP R0 L0 1 [inext] L3: LOADN R0 0 RETURN R0 1 )"); diff --git a/tests/Conformance.test.cpp b/tests/Conformance.test.cpp index 96a2775f..3f415149 100644 --- a/tests/Conformance.test.cpp +++ b/tests/Conformance.test.cpp @@ -231,6 +231,8 @@ TEST_CASE("Assert") TEST_CASE("Basic") { + ScopedFastFlag sff("LuauLenTM", true); + runConformance("basic.lua"); } @@ -301,6 +303,8 @@ TEST_CASE("Errors") TEST_CASE("Events") { + ScopedFastFlag sff("LuauLenTM", true); + runConformance("events.lua"); } @@ -475,6 +479,8 @@ static void populateRTTI(lua_State* L, Luau::TypeId type) TEST_CASE("Types") { + ScopedFastFlag sff("LuauCheckLenMT", true); + runConformance("types.lua", [](lua_State* L) { Luau::NullModuleResolver moduleResolver; Luau::InternalErrorReporter iceHandler; diff --git a/tests/ConstraintGraphBuilder.test.cpp b/tests/ConstraintGraphBuilder.test.cpp index 96b21613..00c3309c 100644 --- a/tests/ConstraintGraphBuilder.test.cpp +++ b/tests/ConstraintGraphBuilder.test.cpp @@ -17,7 +17,7 @@ TEST_CASE_FIXTURE(ConstraintGraphBuilderFixture, "hello_world") )"); cgb.visit(block); - auto constraints = collectConstraints(cgb.rootScope); + auto constraints = collectConstraints(NotNull(cgb.rootScope)); REQUIRE(2 == constraints.size()); @@ -36,7 +36,7 @@ TEST_CASE_FIXTURE(ConstraintGraphBuilderFixture, "primitives") )"); cgb.visit(block); - auto constraints = collectConstraints(cgb.rootScope); + auto constraints = collectConstraints(NotNull(cgb.rootScope)); REQUIRE(3 == constraints.size()); @@ -54,15 +54,15 @@ TEST_CASE_FIXTURE(ConstraintGraphBuilderFixture, "nil_primitive") )"); cgb.visit(block); - auto constraints = collectConstraints(cgb.rootScope); + auto constraints = collectConstraints(NotNull(cgb.rootScope)); ToStringOptions opts; REQUIRE(5 <= constraints.size()); CHECK("*blocked-1* ~ gen () -> (a...)" == toString(*constraints[0], opts)); - CHECK("b ~ inst *blocked-1*" == toString(*constraints[1], opts)); - CHECK("() -> (c...) <: b" == toString(*constraints[2], opts)); - CHECK("c... <: d" == toString(*constraints[3], opts)); + CHECK("*blocked-2* ~ inst *blocked-1*" == toString(*constraints[1], opts)); + CHECK("() -> (b...) <: *blocked-2*" == toString(*constraints[2], opts)); + CHECK("b... <: c" == toString(*constraints[3], opts)); CHECK("nil <: a..." == toString(*constraints[4], opts)); } @@ -74,15 +74,15 @@ TEST_CASE_FIXTURE(ConstraintGraphBuilderFixture, "function_application") )"); cgb.visit(block); - auto constraints = collectConstraints(cgb.rootScope); + auto constraints = collectConstraints(NotNull(cgb.rootScope)); REQUIRE(4 == constraints.size()); ToStringOptions opts; CHECK("string <: a" == toString(*constraints[0], opts)); - CHECK("b ~ inst a" == toString(*constraints[1], opts)); - CHECK("(string) -> (c...) <: b" == toString(*constraints[2], opts)); - CHECK("c... <: d" == toString(*constraints[3], opts)); + CHECK("*blocked-1* ~ inst a" == toString(*constraints[1], opts)); + CHECK("(string) -> (b...) <: *blocked-1*" == toString(*constraints[2], opts)); + CHECK("b... <: c" == toString(*constraints[3], opts)); } TEST_CASE_FIXTURE(ConstraintGraphBuilderFixture, "local_function_definition") @@ -94,7 +94,7 @@ TEST_CASE_FIXTURE(ConstraintGraphBuilderFixture, "local_function_definition") )"); cgb.visit(block); - auto constraints = collectConstraints(cgb.rootScope); + auto constraints = collectConstraints(NotNull(cgb.rootScope)); REQUIRE(2 == constraints.size()); @@ -112,15 +112,15 @@ TEST_CASE_FIXTURE(ConstraintGraphBuilderFixture, "recursive_function") )"); cgb.visit(block); - auto constraints = collectConstraints(cgb.rootScope); + auto constraints = collectConstraints(NotNull(cgb.rootScope)); REQUIRE(4 == constraints.size()); ToStringOptions opts; CHECK("*blocked-1* ~ gen (a) -> (b...)" == toString(*constraints[0], opts)); - CHECK("c ~ inst (a) -> (b...)" == toString(*constraints[1], opts)); - CHECK("(a) -> (d...) <: c" == toString(*constraints[2], opts)); - CHECK("d... <: b..." == toString(*constraints[3], opts)); + CHECK("*blocked-2* ~ inst (a) -> (b...)" == toString(*constraints[1], opts)); + CHECK("(a) -> (c...) <: *blocked-2*" == toString(*constraints[2], opts)); + CHECK("c... <: b..." == toString(*constraints[3], opts)); } TEST_SUITE_END(); diff --git a/tests/ConstraintSolver.test.cpp b/tests/ConstraintSolver.test.cpp index 5959f55c..f521c667 100644 --- a/tests/ConstraintSolver.test.cpp +++ b/tests/ConstraintSolver.test.cpp @@ -9,7 +9,7 @@ using namespace Luau; -static TypeId requireBinding(Scope2* scope, const char* name) +static TypeId requireBinding(NotNull scope, const char* name) { auto b = linearSearchForBinding(scope, name); LUAU_ASSERT(b.has_value()); @@ -26,12 +26,13 @@ TEST_CASE_FIXTURE(ConstraintGraphBuilderFixture, "hello") )"); cgb.visit(block); + NotNull rootScope = NotNull(cgb.rootScope); - ConstraintSolver cs{&arena, cgb.rootScope}; + ConstraintSolver cs{&arena, rootScope}; cs.run(); - TypeId bType = requireBinding(cgb.rootScope, "b"); + TypeId bType = requireBinding(rootScope, "b"); CHECK("number" == toString(bType)); } @@ -45,12 +46,13 @@ TEST_CASE_FIXTURE(ConstraintGraphBuilderFixture, "generic_function") )"); cgb.visit(block); + NotNull rootScope = NotNull(cgb.rootScope); - ConstraintSolver cs{&arena, cgb.rootScope}; + ConstraintSolver cs{&arena, rootScope}; cs.run(); - TypeId idType = requireBinding(cgb.rootScope, "id"); + TypeId idType = requireBinding(rootScope, "id"); CHECK("(a) -> a" == toString(idType)); } @@ -71,14 +73,15 @@ TEST_CASE_FIXTURE(ConstraintGraphBuilderFixture, "proper_let_generalization") )"); cgb.visit(block); + NotNull rootScope = NotNull(cgb.rootScope); ToStringOptions opts; - ConstraintSolver cs{&arena, cgb.rootScope}; + ConstraintSolver cs{&arena, rootScope}; cs.run(); - TypeId idType = requireBinding(cgb.rootScope, "b"); + TypeId idType = requireBinding(rootScope, "b"); CHECK("(a) -> number" == toString(idType, opts)); } diff --git a/tests/Fixture.cpp b/tests/Fixture.cpp index ac22f65b..c92c4457 100644 --- a/tests/Fixture.cpp +++ b/tests/Fixture.cpp @@ -195,12 +195,15 @@ ParseResult Fixture::matchParseError(const std::string& source, const std::strin sourceModule.reset(new SourceModule); ParseResult result = Parser::parse(source.c_str(), source.length(), *sourceModule->names, *sourceModule->allocator, options); - REQUIRE_MESSAGE(!result.errors.empty(), "Expected a parse error in '" << source << "'"); + CHECK_MESSAGE(!result.errors.empty(), "Expected a parse error in '" << source << "'"); - CHECK_EQ(result.errors.front().getMessage(), message); + if (!result.errors.empty()) + { + CHECK_EQ(result.errors.front().getMessage(), message); - if (location) - CHECK_EQ(result.errors.front().getLocation(), *location); + if (location) + CHECK_EQ(result.errors.front().getLocation(), *location); + } return result; } @@ -213,11 +216,14 @@ ParseResult Fixture::matchParseErrorPrefix(const std::string& source, const std: sourceModule.reset(new SourceModule); ParseResult result = Parser::parse(source.c_str(), source.length(), *sourceModule->names, *sourceModule->allocator, options); - REQUIRE_MESSAGE(!result.errors.empty(), "Expected a parse error in '" << source << "'"); + CHECK_MESSAGE(!result.errors.empty(), "Expected a parse error in '" << source << "'"); - const std::string& message = result.errors.front().getMessage(); - CHECK_GE(message.length(), prefix.length()); - CHECK_EQ(prefix, message.substr(0, prefix.size())); + if (!result.errors.empty()) + { + const std::string& message = result.errors.front().getMessage(); + CHECK_GE(message.length(), prefix.length()); + CHECK_EQ(prefix, message.substr(0, prefix.size())); + } return result; } @@ -428,6 +434,7 @@ BuiltinsFixture::BuiltinsFixture(bool freeze, bool prepareAutocomplete) ConstraintGraphBuilderFixture::ConstraintGraphBuilderFixture() : Fixture() + , cgb(mainModuleName, &arena, NotNull(&ice), frontend.getGlobalScope2()) , forceTheFlag{"DebugLuauDeferredConstraintResolution", true} { BlockedTypeVar::nextIndex = 0; diff --git a/tests/Fixture.h b/tests/Fixture.h index 0e3735f6..1bc573da 100644 --- a/tests/Fixture.h +++ b/tests/Fixture.h @@ -133,6 +133,7 @@ struct Fixture TestConfigResolver configResolver; std::unique_ptr sourceModule; Frontend frontend; + InternalErrorReporter ice; TypeChecker& typeChecker; std::string decorateWithTypes(const std::string& code); @@ -160,7 +161,7 @@ struct BuiltinsFixture : Fixture struct ConstraintGraphBuilderFixture : Fixture { TypeArena arena; - ConstraintGraphBuilder cgb{&arena}; + ConstraintGraphBuilder cgb; ScopedFastFlag forceTheFlag; diff --git a/tests/NotNull.test.cpp b/tests/NotNull.test.cpp index ed1c25ec..e77ba78a 100644 --- a/tests/NotNull.test.cpp +++ b/tests/NotNull.test.cpp @@ -30,23 +30,23 @@ struct Test int Test::count = 0; -} +} // namespace int foo(NotNull p) { return *p; } -void bar(int* q) -{} +void bar(int* q) {} TEST_SUITE_BEGIN("NotNull"); TEST_CASE("basic_stuff") { - NotNull a = NotNull{new int(55)}; // Does runtime test - NotNull b{new int(55)}; // As above - // NotNull c = new int(55); // Nope. Mildly regrettable, but implicit conversion from T* to NotNull in the general case is not good. + NotNull a = NotNull{new int(55)}; // Does runtime test + NotNull b{new int(55)}; // As above + // NotNull c = new int(55); // Nope. Mildly regrettable, but implicit conversion from T* to NotNull in the general case is not + // good. // a = nullptr; // nope diff --git a/tests/Parser.test.cpp b/tests/Parser.test.cpp index 878023e3..c3c75998 100644 --- a/tests/Parser.test.cpp +++ b/tests/Parser.test.cpp @@ -6,6 +6,8 @@ #include "doctest.h" +#include + using namespace Luau; namespace @@ -786,33 +788,46 @@ TEST_CASE_FIXTURE(Fixture, "parse_numbers_decimal") TEST_CASE_FIXTURE(Fixture, "parse_numbers_hexadecimal") { - AstStat* stat = parse("return 0xab, 0XAB05, 0xff_ff"); + AstStat* stat = parse("return 0xab, 0XAB05, 0xff_ff, 0xffffffffffffffff"); REQUIRE(stat != nullptr); AstStatReturn* str = stat->as()->body.data[0]->as(); - CHECK(str->list.size == 3); + CHECK(str->list.size == 4); CHECK_EQ(str->list.data[0]->as()->value, 0xab); CHECK_EQ(str->list.data[1]->as()->value, 0xAB05); CHECK_EQ(str->list.data[2]->as()->value, 0xFFFF); + CHECK_EQ(str->list.data[3]->as()->value, double(ULLONG_MAX)); } TEST_CASE_FIXTURE(Fixture, "parse_numbers_binary") { - AstStat* stat = parse("return 0b1, 0b0, 0b101010"); + AstStat* stat = parse("return 0b1, 0b0, 0b101010, 0b1111111111111111111111111111111111111111111111111111111111111111"); REQUIRE(stat != nullptr); AstStatReturn* str = stat->as()->body.data[0]->as(); - CHECK(str->list.size == 3); + CHECK(str->list.size == 4); CHECK_EQ(str->list.data[0]->as()->value, 1); CHECK_EQ(str->list.data[1]->as()->value, 0); CHECK_EQ(str->list.data[2]->as()->value, 42); + CHECK_EQ(str->list.data[3]->as()->value, double(ULLONG_MAX)); } TEST_CASE_FIXTURE(Fixture, "parse_numbers_error") { + ScopedFastFlag luauErrorParseIntegerIssues{"LuauErrorParseIntegerIssues", true}; + CHECK_EQ(getParseError("return 0b123"), "Malformed number"); CHECK_EQ(getParseError("return 123x"), "Malformed number"); CHECK_EQ(getParseError("return 0xg"), "Malformed number"); + CHECK_EQ(getParseError("return 0x0x123"), "Malformed number"); +} + +TEST_CASE_FIXTURE(Fixture, "parse_numbers_range_error") +{ + ScopedFastFlag luauErrorParseIntegerIssues{"LuauErrorParseIntegerIssues", true}; + + CHECK_EQ(getParseError("return 0x10000000000000000"), "Integer number value is out of range"); + CHECK_EQ(getParseError("return 0b10000000000000000000000000000000000000000000000000000000000000000"), "Integer number value is out of range"); } TEST_CASE_FIXTURE(Fixture, "break_return_not_last_error") @@ -2111,6 +2126,15 @@ type C = Packed<(number, X...)> REQUIRE(stat != nullptr); } +TEST_CASE_FIXTURE(Fixture, "invalid_type_forms") +{ + ScopedFastFlag luauFixNamedFunctionParse{"LuauFixNamedFunctionParse", true}; + + matchParseError("type A = (b: number)", "Expected '->' when parsing function type, got "); + matchParseError("type P = () -> T... type B = P<(x: number, y: string)>", "Expected '->' when parsing function type, got '>'"); + matchParseError("type F = (T...) -> ()", "Expected '->' when parsing function type, got '>'"); +} + TEST_SUITE_END(); TEST_SUITE_BEGIN("ParseErrorRecovery"); diff --git a/tests/ToString.test.cpp b/tests/ToString.test.cpp index e03069a9..387e07cd 100644 --- a/tests/ToString.test.cpp +++ b/tests/ToString.test.cpp @@ -409,6 +409,8 @@ TEST_CASE_FIXTURE(Fixture, "toStringDetailed") TEST_CASE_FIXTURE(BuiltinsFixture, "toStringDetailed2") { + ScopedFastFlag sff2{"DebugLuauSharedSelf", true}; + CheckResult result = check(R"( local base = {} function base:one() return 1 end @@ -424,7 +426,7 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "toStringDetailed2") TypeId tType = requireType("inst"); ToStringResult r = toStringDetailed(tType); - CHECK_EQ("{ @metatable { __index: { @metatable { __index: base }, child } }, inst }", r.name); + CHECK_EQ("{ @metatable { __index: { @metatable {| __index: base |}, child } }, inst }", r.name); CHECK_EQ(0, r.nameMap.typeVars.size()); ToStringOptions opts; @@ -455,11 +457,10 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "toStringDetailed2") std::string twoResult = toString(tMeta6->props["two"].type, opts); - REQUIRE_EQ("(a) -> number", oneResult.name); - REQUIRE_EQ("(b) -> number", twoResult); + CHECK_EQ("(a) -> number", oneResult.name); + CHECK_EQ("(b) -> number", twoResult); } - TEST_CASE_FIXTURE(Fixture, "toStringErrorPack") { CheckResult result = check(R"( @@ -688,6 +689,10 @@ TEST_CASE_FIXTURE(Fixture, "pick_distinct_names_for_mixed_explicit_and_implicit_ TEST_CASE_FIXTURE(Fixture, "toStringNamedFunction_include_self_param") { + ScopedFastFlag sff[]{ + {"DebugLuauSharedSelf", true}, + }; + CheckResult result = check(R"( local foo = {} function foo:method(arg: string): () @@ -701,9 +706,12 @@ TEST_CASE_FIXTURE(Fixture, "toStringNamedFunction_include_self_param") CHECK_EQ("foo:method(self: a, arg: string): ()", toStringNamedFunction("foo:method", *ftv)); } - TEST_CASE_FIXTURE(Fixture, "toStringNamedFunction_hide_self_param") { + ScopedFastFlag sff[]{ + {"DebugLuauSharedSelf", true}, + }; + CheckResult result = check(R"( local foo = {} function foo:method(arg: string): () diff --git a/tests/TypeInfer.aliases.test.cpp b/tests/TypeInfer.aliases.test.cpp index d6f0a0c8..bdd4d6fd 100644 --- a/tests/TypeInfer.aliases.test.cpp +++ b/tests/TypeInfer.aliases.test.cpp @@ -73,24 +73,24 @@ TEST_CASE_FIXTURE(Fixture, "cannot_steal_hoisted_type_alias") if (FFlag::DebugLuauDeferredConstraintResolution) { CHECK(result.errors[0] == TypeError{ - Location{{1, 21}, {1, 26}}, - getMainSourceModule()->name, - TypeMismatch{ - getSingletonTypes().numberType, - getSingletonTypes().stringType, - }, - }); + Location{{1, 21}, {1, 26}}, + getMainSourceModule()->name, + TypeMismatch{ + getSingletonTypes().numberType, + getSingletonTypes().stringType, + }, + }); } else { CHECK(result.errors[0] == TypeError{ - Location{{1, 8}, {1, 26}}, - getMainSourceModule()->name, - TypeMismatch{ - getSingletonTypes().numberType, - getSingletonTypes().stringType, - }, - }); + Location{{1, 8}, {1, 26}}, + getMainSourceModule()->name, + TypeMismatch{ + getSingletonTypes().numberType, + getSingletonTypes().stringType, + }, + }); } } @@ -716,6 +716,10 @@ TEST_CASE_FIXTURE(Fixture, "forward_declared_alias_is_not_clobbered_by_prior_uni TEST_CASE_FIXTURE(Fixture, "forward_declared_alias_is_not_clobbered_by_prior_unification_with_any_2") { + ScopedFastFlag sff[] = { + {"DebugLuauSharedSelf", true}, + }; + CheckResult result = check(R"( local B = {} B.bar = 4 @@ -737,7 +741,8 @@ TEST_CASE_FIXTURE(Fixture, "forward_declared_alias_is_not_clobbered_by_prior_uni type FutureIntersection = A & B )"); - LUAU_REQUIRE_NO_ERRORS(result); + // TODO: shared self causes this test to break in bizarre ways. + LUAU_REQUIRE_ERRORS(result); } TEST_CASE_FIXTURE(Fixture, "recursive_types_restriction_ok") diff --git a/tests/TypeInfer.annotations.test.cpp b/tests/TypeInfer.annotations.test.cpp index 3e2ad6dc..8a86ee5f 100644 --- a/tests/TypeInfer.annotations.test.cpp +++ b/tests/TypeInfer.annotations.test.cpp @@ -134,13 +134,13 @@ TEST_CASE_FIXTURE(Fixture, "unknown_type_reference_generates_error") LUAU_REQUIRE_ERROR_COUNT(1, result); CHECK(result.errors[0] == TypeError{ - Location{{1, 17}, {1, 28}}, - getMainSourceModule()->name, - UnknownSymbol{ - "IDoNotExist", - UnknownSymbol::Context::Type, - }, - }); + Location{{1, 17}, {1, 28}}, + getMainSourceModule()->name, + UnknownSymbol{ + "IDoNotExist", + UnknownSymbol::Context::Type, + }, + }); } TEST_CASE_FIXTURE(Fixture, "typeof_variable_type_annotation_should_return_its_type") diff --git a/tests/TypeInfer.functions.test.cpp b/tests/TypeInfer.functions.test.cpp index 036a667a..401a6c64 100644 --- a/tests/TypeInfer.functions.test.cpp +++ b/tests/TypeInfer.functions.test.cpp @@ -37,6 +37,27 @@ TEST_CASE_FIXTURE(Fixture, "check_function_bodies") }})); } +TEST_CASE_FIXTURE(Fixture, "cannot_hoist_interior_defns_into_signature") +{ + // This test verifies that the signature does not have access to types + // declared within the body. Under DCR, if the function's inner scope + // encompasses the entire function expression, it would be possible for this + // to type check (but the solver output is somewhat undefined). This test + // ensures that this isn't the case. + CheckResult result = check(R"( + local function f(x: T) + type T = number + end + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK(result.errors[0] == TypeError{Location{{1, 28}, {1, 29}}, getMainSourceModule()->name, + UnknownSymbol{ + "T", + UnknownSymbol::Context::Type, + }}); +} + TEST_CASE_FIXTURE(Fixture, "infer_return_type") { CheckResult result = check("function take_five() return 5 end"); diff --git a/tests/TypeInfer.generics.test.cpp b/tests/TypeInfer.generics.test.cpp index 97ba0808..e9e94cfb 100644 --- a/tests/TypeInfer.generics.test.cpp +++ b/tests/TypeInfer.generics.test.cpp @@ -271,13 +271,16 @@ TEST_CASE_FIXTURE(Fixture, "infer_nested_generic_function") TEST_CASE_FIXTURE(Fixture, "infer_generic_methods") { + ScopedFastFlag sff{"DebugLuauSharedSelf", true}; + CheckResult result = check(R"( local x = {} function x:id(x) return x end function x:f(): string return self:id("hello") end function x:g(): number return self:id(37) end )"); - LUAU_REQUIRE_NO_ERRORS(result); + // TODO: Quantification should be doing the conversion, not normalization. + LUAU_REQUIRE_ERRORS(result); } TEST_CASE_FIXTURE(Fixture, "calling_self_generic_methods") diff --git a/tests/TypeInfer.operators.test.cpp b/tests/TypeInfer.operators.test.cpp index fd9b1dd4..e6174df2 100644 --- a/tests/TypeInfer.operators.test.cpp +++ b/tests/TypeInfer.operators.test.cpp @@ -461,6 +461,61 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "typecheck_unary_minus") REQUIRE_EQ(gen->message, "Unary operator '-' not supported by type 'bar'"); } +TEST_CASE_FIXTURE(BuiltinsFixture, "typecheck_unary_minus_error") +{ + CheckResult result = check(R"( + --!strict + local foo = { + value = 10 + } + + local mt = {} + setmetatable(foo, mt) + + mt.__unm = function(val: boolean): string + return "test" + end + + local a = -foo + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + CHECK_EQ("string", toString(requireType("a"))); + + TypeMismatch* tm = get(result.errors[0]); + REQUIRE_EQ(*tm->wantedType, *typeChecker.booleanType); + // given type is the typeof(foo) which is complex to compare against +} + +TEST_CASE_FIXTURE(BuiltinsFixture, "typecheck_unary_len_error") +{ + ScopedFastFlag sff("LuauCheckLenMT", true); + + CheckResult result = check(R"( + --!strict + local foo = { + value = 10 + } + local mt = {} + setmetatable(foo, mt) + + mt.__len = function(val: any): string + return "test" + end + + local a = #foo + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + CHECK_EQ("number", toString(requireType("a"))); + + TypeMismatch* tm = get(result.errors[0]); + REQUIRE_EQ(*tm->wantedType, *typeChecker.numberType); + REQUIRE_EQ(*tm->givenType, *typeChecker.stringType); +} + TEST_CASE_FIXTURE(BuiltinsFixture, "unary_not_is_boolean") { CheckResult result = check(R"( diff --git a/tests/TypeInfer.provisional.test.cpp b/tests/TypeInfer.provisional.test.cpp index 487e5979..059aed2e 100644 --- a/tests/TypeInfer.provisional.test.cpp +++ b/tests/TypeInfer.provisional.test.cpp @@ -499,4 +499,26 @@ TEST_CASE_FIXTURE(Fixture, "constrained_is_level_dependent") CHECK_EQ("(t1) -> {| [t1]: boolean |} where t1 = t2 ; t2 = {+ m1: (t1) -> (a...), m2: (t2) -> (b...) +}", toString(requireType("f"))); } +TEST_CASE_FIXTURE(BuiltinsFixture, "greedy_inference_with_shared_self_triggers_function_with_no_returns") +{ + ScopedFastFlag sff{"DebugLuauSharedSelf", true}; + + CheckResult result = check(R"( + local T = {} + T.__index = T + + function T.new() + local self = setmetatable({}, T) + return self:ctor() or self + end + + function T:ctor() + -- oops, no return! + end + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ("Not all codepaths in this function return '{ @metatable T, {| |} }, a...'.", toString(result.errors[0])); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.tables.test.cpp b/tests/TypeInfer.tables.test.cpp index 77a2928c..eead5b30 100644 --- a/tests/TypeInfer.tables.test.cpp +++ b/tests/TypeInfer.tables.test.cpp @@ -1863,6 +1863,8 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "quantifying_a_bound_var_works") TEST_CASE_FIXTURE(BuiltinsFixture, "less_exponential_blowup_please") { + ScopedFastFlag sff{"DebugLuauSharedSelf", true}; + CheckResult result = check(R"( --!strict @@ -1890,7 +1892,7 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "less_exponential_blowup_please") newData:First() )"); - LUAU_REQUIRE_ERROR_COUNT(1, result); + LUAU_REQUIRE_ERROR_COUNT(2, result); } TEST_CASE_FIXTURE(Fixture, "common_table_element_union_in_call") @@ -2868,6 +2870,7 @@ TEST_CASE_FIXTURE(Fixture, "inferred_return_type_of_free_table") { ScopedFastFlag sff[] = { {"LuauLowerBoundsCalculation", true}, + {"DebugLuauSharedSelf", true}, }; check(R"( @@ -2887,7 +2890,7 @@ TEST_CASE_FIXTURE(Fixture, "inferred_return_type_of_free_table") end )"); - CHECK_EQ("(t1) -> {| Byte: (b) -> (a...), PeekByte: (c) -> (a...) |} where t1 = {+ byte: (t1, number) -> (a...) +}", + CHECK_EQ("(t1) -> {| Byte: (a) -> (b...), PeekByte: (a) -> (b...) |} where t1 = {+ byte: (t1, number) -> (b...) +}", toString(requireType("Base64FileReader"))); } @@ -2904,6 +2907,66 @@ TEST_CASE_FIXTURE(Fixture, "mixed_tables_with_implicit_numbered_keys") CHECK_EQ("Type 'number' could not be converted into 'string'", toString(result.errors[2])); } +TEST_CASE_FIXTURE(Fixture, "shared_selfs") +{ + ScopedFastFlag sff{"DebugLuauSharedSelf", true}; + + CheckResult result = check(R"( + local t = {} + t.x = 5 + + function t:m1() return self.x end + function t:m2() return self.y end + + return t + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + ToStringOptions opts; + opts.exhaustive = true; + CHECK_EQ("{| m1: ({+ x: a, y: b +}) -> a, m2: ({+ x: a, y: b +}) -> b, x: number |}", toString(requireType("t"), opts)); +} + +TEST_CASE_FIXTURE(Fixture, "shared_selfs_from_free_param") +{ + ScopedFastFlag sff{"DebugLuauSharedSelf", true}; + + CheckResult result = check(R"( + local function f(t) + function t:m1() return self.x end + function t:m2() return self.y end + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ("({+ m1: ({+ x: a, y: b +}) -> a, m2: ({+ x: a, y: b +}) -> b +}) -> ()", toString(requireType("f"))); +} + +TEST_CASE_FIXTURE(BuiltinsFixture, "shared_selfs_through_metatables") +{ + ScopedFastFlag sff{"DebugLuauSharedSelf", true}; + + CheckResult result = check(R"( + local t = {} + t.__index = t + setmetatable({}, t) + + function t:m1() return self.x end + function t:m2() return self.y end + + return t + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + ToStringOptions opts; + opts.exhaustive = true; + CHECK_EQ( + toString(requireType("t"), opts), "t1 where t1 = {| __index: t1, m1: ({+ x: a, y: b +}) -> a, m2: ({+ x: a, y: b +}) -> b |}"); +} + TEST_CASE_FIXTURE(Fixture, "expected_indexer_value_type_extra") { CheckResult result = check(R"( @@ -2953,4 +3016,58 @@ TEST_CASE_FIXTURE(Fixture, "prop_access_on_unions_of_indexers_where_key_whose_ty CHECK_EQ("Type '{number} | {| [boolean]: number |}' does not have key 'x'", toString(result.errors[0])); } +TEST_CASE_FIXTURE(BuiltinsFixture, "quantify_metatables_of_metatables_of_table") +{ + ScopedFastFlag sff[]{ + {"DebugLuauSharedSelf", true}, + }; + + CheckResult result = check(R"( + local T = {} + + function T:m() + return self.x, self.y + end + + function T:n() + end + + local U = setmetatable({}, {__index = T}) + + local V = setmetatable({}, {__index = U}) + + return V + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + ToStringOptions opts; + opts.exhaustive = true; + CHECK_EQ(toString(requireType("V"), opts), "{ @metatable { __index: { @metatable { __index: {| m: ({+ x: a, y: b +}) -> (a, b), n: ({+ x: a, y: b +}) -> () |} }, { } } }, { } }"); +} + +TEST_CASE_FIXTURE(Fixture, "quantify_even_that_table_was_never_exported_at_all") +{ + ScopedFastFlag sff{"DebugLuauSharedSelf", true}; + + CheckResult result = check(R"( + local T = {} + + function T:m() + return self.x + end + + function T:n() + return self.y + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + ToStringOptions opts; + opts.exhaustive = true; + CHECK_EQ("{| m: ({+ x: a, y: b +}) -> a, n: ({+ x: a, y: b +}) -> b |}", toString(requireType("T"), opts)); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.test.cpp b/tests/TypeInfer.test.cpp index 6a048b26..efdfe0b1 100644 --- a/tests/TypeInfer.test.cpp +++ b/tests/TypeInfer.test.cpp @@ -369,14 +369,14 @@ TEST_CASE_FIXTURE(Fixture, "globals_are_banned_in_strict_mode") CHECK_EQ("foo", us->name); } -TEST_CASE_FIXTURE(BuiltinsFixture, "correctly_scope_locals_do") +TEST_CASE_FIXTURE(Fixture, "correctly_scope_locals_do") { CheckResult result = check(R"( do local a = 1 end - print(a) -- oops! + local b = a -- oops! )"); LUAU_REQUIRE_ERROR_COUNT(1, result); diff --git a/tests/conformance/basic.lua b/tests/conformance/basic.lua index f803c319..385a0450 100644 --- a/tests/conformance/basic.lua +++ b/tests/conformance/basic.lua @@ -118,10 +118,12 @@ assert((function() return #_G end)() == 0) assert((function() return #{1,2} end)() == 2) assert((function() return #'g' end)() == 1) -assert((function() local ud = newproxy(true) getmetatable(ud).__len = function() return 42 end return #ud end)() == 42) - assert((function() local a = 1 a = -a return a end)() == -1) +-- __len metamethod +assert((function() local ud = newproxy(true) getmetatable(ud).__len = function() return 42 end return #ud end)() == 42) +assert((function() local t = {} setmetatable(t, { __len = function() return 42 end }) return #t end)() == 42) + -- while/repeat assert((function() local a = 10 local b = 1 while a > 1 do b = b * 2 a = a - 1 end return b end)() == 512) assert((function() local a = 10 local b = 1 repeat b = b * 2 a = a - 1 until a == 1 return b end)() == 512) @@ -889,6 +891,10 @@ assert((function() return table.concat(res, ',') end)() == "6,8,10") +-- typeof and type require an argument +assert(pcall(typeof) == false) +assert(pcall(type) == false) + -- typeof == type in absence of custom userdata assert(concat(typeof(5), typeof(nil), typeof({}), typeof(newproxy())) == "number,nil,table,userdata") diff --git a/tests/conformance/events.lua b/tests/conformance/events.lua index 32e090ac..42f1beda 100644 --- a/tests/conformance/events.lua +++ b/tests/conformance/events.lua @@ -386,4 +386,42 @@ do assert(t.X) -- fails if table flags are set incorrectly end +do + -- verify __len behavior & error handling + local t = {1} + + setmetatable(t, {}) + assert(#t == 1) + + setmetatable(t, { __len = rawlen }) + assert(#t == 1) + + setmetatable(t, { __len = function() return 42 end }) + assert(#t == 42) + + setmetatable(t, { __len = 42 }) + local ok, err = pcall(function() return #t end) + assert(not ok and err:match("attempt to call a number value")) + + setmetatable(t, { __len = function() end }) + local ok, err = pcall(function() return #t end) + assert(not ok and err:match("'__len' must return a number")) + + setmetatable(t, { __len = error }) + local ok, err = pcall(function() return #t end) + assert(not ok and err == t) +end + +-- verify rawlen behavior +do + local t = {1} + setmetatable(t, { __len = 42 }) + + assert(rawlen(t) == 1) + assert(rawlen("foo") == 3) + + local ok, err = pcall(function() return rawlen(42) end) + assert(not ok and err:match("table or string expected")) +end + return 'OK' From 2daa6497a17348090f6af6716e4f856c44594279 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Thu, 30 Jun 2022 16:52:43 -0700 Subject: [PATCH 297/399] Sync to upstream/release/534 (#569) --- Analysis/include/Luau/Constraint.h | 19 +- .../include/Luau/ConstraintGraphBuilder.h | 109 +++- Analysis/include/Luau/ConstraintSolver.h | 12 +- Analysis/include/Luau/Error.h | 33 +- Analysis/include/Luau/Frontend.h | 5 + Analysis/include/Luau/Module.h | 2 +- Analysis/include/Luau/NotNull.h | 14 +- Analysis/include/Luau/Scope.h | 6 +- Analysis/include/Luau/TypeArena.h | 6 + Analysis/include/Luau/TypeInfer.h | 4 +- Analysis/include/Luau/TypeVar.h | 3 + Analysis/src/ConstraintGraphBuilder.cpp | 557 ++++++++++++++---- Analysis/src/ConstraintSolver.cpp | 71 ++- Analysis/src/EmbeddedBuiltinDefinitions.cpp | 10 +- Analysis/src/Frontend.cpp | 22 +- Analysis/src/Normalize.cpp | 235 +------- Analysis/src/Quantify.cpp | 73 ++- Analysis/src/Scope.cpp | 15 + Analysis/src/ToString.cpp | 111 ++-- Analysis/src/TypeChecker2.cpp | 106 +++- Analysis/src/TypeInfer.cpp | 144 ++++- Analysis/src/Unifier.cpp | 4 +- Ast/src/Parser.cpp | 122 +++- Common/include/Luau/Bytecode.h | 5 +- Compiler/src/Builtins.cpp | 4 + Compiler/src/BytecodeBuilder.cpp | 15 +- Compiler/src/Compiler.cpp | 16 +- VM/src/lbaselib.cpp | 15 + VM/src/lbuiltins.cpp | 23 + VM/src/ldebug.h | 1 + VM/src/ltm.cpp | 2 +- VM/src/ltm.h | 2 +- VM/src/lvmexecute.cpp | 54 +- VM/src/lvmutils.cpp | 54 +- fuzz/protoprint.cpp | 11 + tests/Compiler.test.cpp | 16 +- tests/Conformance.test.cpp | 6 + tests/ConstraintGraphBuilder.test.cpp | 30 +- tests/ConstraintSolver.test.cpp | 17 +- tests/Fixture.cpp | 23 +- tests/Fixture.h | 3 +- tests/NotNull.test.cpp | 12 +- tests/Parser.test.cpp | 32 +- tests/ToString.test.cpp | 18 +- tests/TypeInfer.aliases.test.cpp | 35 +- tests/TypeInfer.annotations.test.cpp | 14 +- tests/TypeInfer.functions.test.cpp | 21 + tests/TypeInfer.generics.test.cpp | 5 +- tests/TypeInfer.operators.test.cpp | 55 ++ tests/TypeInfer.provisional.test.cpp | 22 + tests/TypeInfer.tables.test.cpp | 121 +++- tests/TypeInfer.test.cpp | 4 +- tests/conformance/basic.lua | 10 +- tests/conformance/events.lua | 38 ++ 54 files changed, 1714 insertions(+), 653 deletions(-) diff --git a/Analysis/include/Luau/Constraint.h b/Analysis/include/Luau/Constraint.h index 8a41c9e8..dcfb14b4 100644 --- a/Analysis/include/Luau/Constraint.h +++ b/Analysis/include/Luau/Constraint.h @@ -1,6 +1,7 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #pragma once +#include "Luau/Ast.h" // Used for some of the enumerations #include "Luau/NotNull.h" #include "Luau/Variant.h" @@ -47,6 +48,21 @@ struct InstantiationConstraint TypeId superType; }; +struct UnaryConstraint +{ + AstExprUnary::Op op; + TypeId operandType; + TypeId resultType; +}; + +struct BinaryConstraint +{ + AstExprBinary::Op op; + TypeId leftType; + TypeId rightType; + TypeId resultType; +}; + // name(namedType) = name struct NameConstraint { @@ -54,7 +70,8 @@ struct NameConstraint std::string name; }; -using ConstraintV = Variant; +using ConstraintV = Variant; using ConstraintPtr = std::unique_ptr; struct Constraint diff --git a/Analysis/include/Luau/ConstraintGraphBuilder.h b/Analysis/include/Luau/ConstraintGraphBuilder.h index 9b118691..a49e8594 100644 --- a/Analysis/include/Luau/ConstraintGraphBuilder.h +++ b/Analysis/include/Luau/ConstraintGraphBuilder.h @@ -25,9 +25,12 @@ struct ConstraintGraphBuilder // scope pointers; the scopes themselves borrow pointers to other scopes to // define the scope hierarchy. std::vector>> scopes; + + ModuleName moduleName; SingletonTypes& singletonTypes; - TypeArena* const arena; + const NotNull arena; // The root scope of the module we're generating constraints for. + // This is null when the CGB is initially constructed. Scope2* rootScope; // A mapping of AST node to TypeId. DenseHashMap astTypes{nullptr}; @@ -39,40 +42,50 @@ struct ConstraintGraphBuilder // Type packs resolved from type annotations. Analogous to astTypePacks. DenseHashMap astResolvedTypePacks{nullptr}; - explicit ConstraintGraphBuilder(TypeArena* arena); + int recursionCount = 0; + + // It is pretty uncommon for constraint generation to itself produce errors, but it can happen. + std::vector errors; + + // Occasionally constraint generation needs to produce an ICE. + const NotNull ice; + + NotNull globalScope; + + ConstraintGraphBuilder(const ModuleName& moduleName, TypeArena* arena, NotNull ice, NotNull globalScope); /** * Fabricates a new free type belonging to a given scope. - * @param scope the scope the free type belongs to. Must not be null. + * @param scope the scope the free type belongs to. */ - TypeId freshType(Scope2* scope); + TypeId freshType(NotNull scope); /** * Fabricates a new free type pack belonging to a given scope. - * @param scope the scope the free type pack belongs to. Must not be null. + * @param scope the scope the free type pack belongs to. */ - TypePackId freshTypePack(Scope2* scope); + TypePackId freshTypePack(NotNull scope); /** * Fabricates a scope that is a child of another scope. * @param location the lexical extent of the scope in the source code. * @param parent the parent scope of the new scope. Must not be null. */ - Scope2* childScope(Location location, Scope2* parent); + NotNull childScope(Location location, NotNull parent); /** * Adds a new constraint with no dependencies to a given scope. - * @param scope the scope to add the constraint to. Must not be null. + * @param scope the scope to add the constraint to. * @param cv the constraint variant to add. */ - void addConstraint(Scope2* scope, ConstraintV cv); + void addConstraint(NotNull scope, ConstraintV cv); /** * Adds a constraint to a given scope. * @param scope the scope to add the constraint to. Must not be null. * @param c the constraint to add. */ - void addConstraint(Scope2* scope, std::unique_ptr c); + void addConstraint(NotNull scope, std::unique_ptr c); /** * The entry point to the ConstraintGraphBuilder. This will construct a set @@ -81,20 +94,22 @@ struct ConstraintGraphBuilder */ void visit(AstStatBlock* block); - void visit(Scope2* scope, AstStat* stat); - void visit(Scope2* scope, AstStatBlock* block); - void visit(Scope2* scope, AstStatLocal* local); - void visit(Scope2* scope, AstStatLocalFunction* function); - void visit(Scope2* scope, AstStatFunction* function); - void visit(Scope2* scope, AstStatReturn* ret); - void visit(Scope2* scope, AstStatAssign* assign); - void visit(Scope2* scope, AstStatIf* ifStatement); - void visit(Scope2* scope, AstStatTypeAlias* alias); + void visitBlockWithoutChildScope(NotNull scope, AstStatBlock* block); - TypePackId checkExprList(Scope2* scope, const AstArray& exprs); + void visit(NotNull scope, AstStat* stat); + void visit(NotNull scope, AstStatBlock* block); + void visit(NotNull scope, AstStatLocal* local); + void visit(NotNull scope, AstStatLocalFunction* function); + void visit(NotNull scope, AstStatFunction* function); + void visit(NotNull scope, AstStatReturn* ret); + void visit(NotNull scope, AstStatAssign* assign); + void visit(NotNull scope, AstStatIf* ifStatement); + void visit(NotNull scope, AstStatTypeAlias* alias); - TypePackId checkPack(Scope2* scope, AstArray exprs); - TypePackId checkPack(Scope2* scope, AstExpr* expr); + TypePackId checkExprList(NotNull scope, const AstArray& exprs); + + TypePackId checkPack(NotNull scope, AstArray exprs); + TypePackId checkPack(NotNull scope, AstExpr* expr); /** * Checks an expression that is expected to evaluate to one type. @@ -102,19 +117,35 @@ struct ConstraintGraphBuilder * @param expr the expression to check. * @return the type of the expression. */ - TypeId check(Scope2* scope, AstExpr* expr); + TypeId check(NotNull scope, AstExpr* expr); - TypeId checkExprTable(Scope2* scope, AstExprTable* expr); - TypeId check(Scope2* scope, AstExprIndexName* indexName); + TypeId checkExprTable(NotNull scope, AstExprTable* expr); + TypeId check(NotNull scope, AstExprIndexName* indexName); + TypeId check(NotNull scope, AstExprIndexExpr* indexExpr); + TypeId check(NotNull scope, AstExprUnary* unary); + TypeId check(NotNull scope, AstExprBinary* binary); - std::pair checkFunctionSignature(Scope2* parent, AstExprFunction* fn); + struct FunctionSignature + { + // The type of the function. + TypeId signature; + // The scope that encompasses the function's signature. May be nullptr + // if there was no need for a signature scope (the function has no + // generics). + Scope2* signatureScope; + // The scope that encompasses the function's body. Is a child scope of + // signatureScope, if present. + NotNull bodyScope; + }; + + FunctionSignature checkFunctionSignature(NotNull parent, AstExprFunction* fn); /** * Checks the body of a function expression. * @param scope the interior scope of the body of the function. * @param fn the function expression to check. */ - void checkFunctionBody(Scope2* scope, AstExprFunction* fn); + void checkFunctionBody(NotNull scope, AstExprFunction* fn); /** * Resolves a type from its AST annotation. @@ -122,7 +153,7 @@ struct ConstraintGraphBuilder * @param ty the AST annotation to resolve. * @return the type of the AST annotation. **/ - TypeId resolveType(Scope2* scope, AstType* ty); + TypeId resolveType(NotNull scope, AstType* ty); /** * Resolves a type pack from its AST annotation. @@ -130,9 +161,25 @@ struct ConstraintGraphBuilder * @param tp the AST annotation to resolve. * @return the type pack of the AST annotation. **/ - TypePackId resolveTypePack(Scope2* scope, AstTypePack* tp); + TypePackId resolveTypePack(NotNull scope, AstTypePack* tp); - TypePackId resolveTypePack(Scope2* scope, const AstTypeList& list); + TypePackId resolveTypePack(NotNull scope, const AstTypeList& list); + + std::vector> createGenerics(NotNull scope, AstArray generics); + std::vector> createGenericPacks(NotNull scope, AstArray packs); + + TypeId flattenPack(NotNull scope, Location location, TypePackId tp); + + void reportError(Location location, TypeErrorData err); + void reportCodeTooComplex(Location location); + + /** Scan the program for global definitions. + * + * ConstraintGraphBuilder needs to differentiate between globals and accesses to undefined symbols. Doing this "for + * real" in a general way is going to be pretty hard, so we are choosing not to tackle that yet. For now, we do an + * initial scan of the AST and note what globals are defined. + */ + void prepopulateGlobalScope(NotNull globalScope, AstStatBlock* program); }; /** @@ -145,6 +192,6 @@ struct ConstraintGraphBuilder * @return a list of pointers to constraints contained within the scope graph. * None of these pointers should be null. */ -std::vector> collectConstraints(Scope2* rootScope); +std::vector> collectConstraints(NotNull rootScope); } // namespace Luau diff --git a/Analysis/include/Luau/ConstraintSolver.h b/Analysis/include/Luau/ConstraintSolver.h index 4870157f..cf88efb6 100644 --- a/Analysis/include/Luau/ConstraintSolver.h +++ b/Analysis/include/Luau/ConstraintSolver.h @@ -25,7 +25,7 @@ struct ConstraintSolver // is important to not add elements to this vector, lest the underlying // storage that we retain pointers to be mutated underneath us. const std::vector> constraints; - Scope2* rootScope; + NotNull rootScope; // This includes every constraint that has not been fully solved. // A constraint can be both blocked and unsolved, for instance. @@ -40,7 +40,7 @@ struct ConstraintSolver ConstraintSolverLogger logger; - explicit ConstraintSolver(TypeArena* arena, Scope2* rootScope); + explicit ConstraintSolver(TypeArena* arena, NotNull rootScope); /** * Attempts to dispatch all pending constraints and reach a type solution @@ -50,11 +50,17 @@ struct ConstraintSolver bool done(); + /** Attempt to dispatch a constraint. Returns true if it was successful. + * If tryDispatch() returns false, the constraint remains in the unsolved set and will be retried later. + */ bool tryDispatch(NotNull c, bool force); + bool tryDispatch(const SubtypeConstraint& c, NotNull constraint, bool force); bool tryDispatch(const PackSubtypeConstraint& c, NotNull constraint, bool force); bool tryDispatch(const GeneralizationConstraint& c, NotNull constraint, bool force); bool tryDispatch(const InstantiationConstraint& c, NotNull constraint, bool force); + bool tryDispatch(const UnaryConstraint& c, NotNull constraint, bool force); + bool tryDispatch(const BinaryConstraint& c, NotNull constraint, bool force); bool tryDispatch(const NameConstraint& c, NotNull constraint); void block(NotNull target, NotNull constraint); @@ -115,6 +121,6 @@ private: void unblock_(BlockedConstraintId progressed); }; -void dump(Scope2* rootScope, struct ToStringOptions& opts); +void dump(NotNull rootScope, struct ToStringOptions& opts); } // namespace Luau diff --git a/Analysis/include/Luau/Error.h b/Analysis/include/Luau/Error.h index a1323960..4c81d33d 100644 --- a/Analysis/include/Luau/Error.h +++ b/Analysis/include/Luau/Error.h @@ -369,24 +369,25 @@ struct InternalErrorReporter [[noreturn]] void ice(const std::string& message); }; -class InternalCompilerError : public std::exception { +class InternalCompilerError : public std::exception +{ public: - explicit InternalCompilerError(const std::string& message, const std::string& moduleName) - : message(message) - , moduleName(moduleName) - { - } - explicit InternalCompilerError(const std::string& message, const std::string& moduleName, const Location& location) - : message(message) - , moduleName(moduleName) - , location(location) - { - } - virtual const char* what() const throw(); + explicit InternalCompilerError(const std::string& message, const std::string& moduleName) + : message(message) + , moduleName(moduleName) + { + } + explicit InternalCompilerError(const std::string& message, const std::string& moduleName, const Location& location) + : message(message) + , moduleName(moduleName) + , location(location) + { + } + virtual const char* what() const throw(); - const std::string message; - const std::string moduleName; - const std::optional location; + const std::string message; + const std::string moduleName; + const std::optional location; }; } // namespace Luau diff --git a/Analysis/include/Luau/Frontend.h b/Analysis/include/Luau/Frontend.h index f4226cc1..f0d43090 100644 --- a/Analysis/include/Luau/Frontend.h +++ b/Analysis/include/Luau/Frontend.h @@ -5,6 +5,7 @@ #include "Luau/Module.h" #include "Luau/ModuleResolver.h" #include "Luau/RequireTracer.h" +#include "Luau/Scope.h" #include "Luau/TypeInfer.h" #include "Luau/Variant.h" @@ -158,6 +159,8 @@ struct Frontend void registerBuiltinDefinition(const std::string& name, std::function); void applyBuiltinDefinitionToEnvironment(const std::string& environmentName, const std::string& definitionName); + NotNull getGlobalScope2(); + private: ModulePtr check(const SourceModule& sourceModule, Mode mode, const ScopePtr& environmentScope); @@ -173,6 +176,8 @@ private: std::unordered_map environments; std::unordered_map> builtinDefinitions; + std::unique_ptr globalScope2; + public: FileResolver* fileResolver; FrontendModuleResolver moduleResolver; diff --git a/Analysis/include/Luau/Module.h b/Analysis/include/Luau/Module.h index 39f8dfb7..b3105b78 100644 --- a/Analysis/include/Luau/Module.h +++ b/Analysis/include/Luau/Module.h @@ -68,7 +68,7 @@ struct Module std::shared_ptr allocator; std::shared_ptr names; - std::vector> scopes; // never empty + std::vector> scopes; // never empty std::vector>> scope2s; // never empty DenseHashMap astTypes{nullptr}; diff --git a/Analysis/include/Luau/NotNull.h b/Analysis/include/Luau/NotNull.h index f6043e9c..714fa143 100644 --- a/Analysis/include/Luau/NotNull.h +++ b/Analysis/include/Luau/NotNull.h @@ -26,7 +26,7 @@ namespace Luau * The explicit delete statement is permitted (but not recommended) on a * NotNull through this implicit conversion. */ -template +template struct NotNull { explicit NotNull(T* t) @@ -38,10 +38,11 @@ struct NotNull explicit NotNull(std::nullptr_t) = delete; void operator=(std::nullptr_t) = delete; - template + template NotNull(NotNull other) : ptr(other.get()) - {} + { + } operator T*() const noexcept { @@ -72,12 +73,13 @@ private: T* ptr; }; -} +} // namespace Luau namespace std { -template struct hash> +template +struct hash> { size_t operator()(const Luau::NotNull& p) const { @@ -85,4 +87,4 @@ template struct hash> } }; -} +} // namespace std diff --git a/Analysis/include/Luau/Scope.h b/Analysis/include/Luau/Scope.h index cef4b94f..0eaecf1d 100644 --- a/Analysis/include/Luau/Scope.h +++ b/Analysis/include/Luau/Scope.h @@ -3,6 +3,7 @@ #include "Luau/Constraint.h" #include "Luau/Location.h" +#include "Luau/NotNull.h" #include "Luau/TypeVar.h" #include @@ -71,15 +72,18 @@ struct Scope2 // is the module-level scope). Scope2* parent = nullptr; // All the children of this scope. - std::vector children; + std::vector> children; std::unordered_map bindings; // TODO: I think this can be a DenseHashMap std::unordered_map typeBindings; + std::unordered_map typePackBindings; TypePackId returnType; + std::optional varargPack; // All constraints belonging to this scope. std::vector constraints; std::optional lookup(Symbol sym); std::optional lookupTypeBinding(const Name& name); + std::optional lookupTypePackBinding(const Name& name); }; } // namespace Luau diff --git a/Analysis/include/Luau/TypeArena.h b/Analysis/include/Luau/TypeArena.h index 559c55c8..be36f19c 100644 --- a/Analysis/include/Luau/TypeArena.h +++ b/Analysis/include/Luau/TypeArena.h @@ -34,6 +34,12 @@ struct TypeArena TypePackId addTypePack(std::vector types); TypePackId addTypePack(TypePack pack); TypePackId addTypePack(TypePackVar pack); + + template + TypePackId addTypePack(T tp) + { + return addTypePack(TypePackVar(std::move(tp))); + } }; void freeze(TypeArena& arena); diff --git a/Analysis/include/Luau/TypeInfer.h b/Analysis/include/Luau/TypeInfer.h index 28adc9d9..455654d9 100644 --- a/Analysis/include/Luau/TypeInfer.h +++ b/Analysis/include/Luau/TypeInfer.h @@ -173,7 +173,7 @@ struct TypeChecker TypeId checkFunctionName(const ScopePtr& scope, AstExpr& funName, TypeLevel level); std::pair checkFunctionSignature(const ScopePtr& scope, int subLevel, const AstExprFunction& expr, - std::optional originalNameLoc, std::optional expectedType); + std::optional originalNameLoc, std::optional selfType, std::optional expectedType); void checkFunctionBody(const ScopePtr& scope, TypeId type, const AstExprFunction& function); void checkArgumentList( @@ -424,6 +424,8 @@ private: * (exported, name) to properly deal with the case where the two duplicates do not have the same export status. */ DenseHashSet, HashBoolNamePair> duplicateTypeAliases; + + std::vector> deferredQuantification; }; // Unit test hook diff --git a/Analysis/include/Luau/TypeVar.h b/Analysis/include/Luau/TypeVar.h index 20f4107c..6ad6b927 100644 --- a/Analysis/include/Luau/TypeVar.h +++ b/Analysis/include/Luau/TypeVar.h @@ -357,6 +357,9 @@ struct TableTypeVar std::optional boundTo; Tags tags; + + // Methods of this table that have an untyped self will use the same shared self type. + std::optional selfTy; }; // Represents a metatable attached to a table typevar. Somewhat analogous to a bound typevar. diff --git a/Analysis/src/ConstraintGraphBuilder.cpp b/Analysis/src/ConstraintGraphBuilder.cpp index d9e8d238..3b9000cd 100644 --- a/Analysis/src/ConstraintGraphBuilder.cpp +++ b/Analysis/src/ConstraintGraphBuilder.cpp @@ -1,6 +1,10 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #include "Luau/ConstraintGraphBuilder.h" +#include "Luau/RecursionCounter.h" +#include "Luau/ToString.h" + +LUAU_FASTINT(LuauCheckRecursionLimit); #include "Luau/Scope.h" @@ -9,32 +13,33 @@ namespace Luau const AstStat* getFallthrough(const AstStat* node); // TypeInfer.cpp -ConstraintGraphBuilder::ConstraintGraphBuilder(TypeArena* arena) - : singletonTypes(getSingletonTypes()) +ConstraintGraphBuilder::ConstraintGraphBuilder( + const ModuleName& moduleName, TypeArena* arena, NotNull ice, NotNull globalScope) + : moduleName(moduleName) + , singletonTypes(getSingletonTypes()) , arena(arena) , rootScope(nullptr) + , ice(ice) + , globalScope(globalScope) { LUAU_ASSERT(arena); } -TypeId ConstraintGraphBuilder::freshType(Scope2* scope) +TypeId ConstraintGraphBuilder::freshType(NotNull scope) { - LUAU_ASSERT(scope); return arena->addType(FreeTypeVar{scope}); } -TypePackId ConstraintGraphBuilder::freshTypePack(Scope2* scope) +TypePackId ConstraintGraphBuilder::freshTypePack(NotNull scope) { - LUAU_ASSERT(scope); FreeTypePack f{scope}; return arena->addTypePack(TypePackVar{std::move(f)}); } -Scope2* ConstraintGraphBuilder::childScope(Location location, Scope2* parent) +NotNull ConstraintGraphBuilder::childScope(Location location, NotNull parent) { - LUAU_ASSERT(parent); auto scope = std::make_unique(); - Scope2* borrow = scope.get(); + NotNull borrow = NotNull(scope.get()); scopes.emplace_back(location, std::move(scope)); borrow->parent = parent; @@ -44,15 +49,13 @@ Scope2* ConstraintGraphBuilder::childScope(Location location, Scope2* parent) return borrow; } -void ConstraintGraphBuilder::addConstraint(Scope2* scope, ConstraintV cv) +void ConstraintGraphBuilder::addConstraint(NotNull scope, ConstraintV cv) { - LUAU_ASSERT(scope); scope->constraints.emplace_back(new Constraint{std::move(cv)}); } -void ConstraintGraphBuilder::addConstraint(Scope2* scope, std::unique_ptr c) +void ConstraintGraphBuilder::addConstraint(NotNull scope, std::unique_ptr c) { - LUAU_ASSERT(scope); scope->constraints.emplace_back(std::move(c)); } @@ -62,7 +65,11 @@ void ConstraintGraphBuilder::visit(AstStatBlock* block) LUAU_ASSERT(rootScope == nullptr); scopes.emplace_back(block->location, std::make_unique()); rootScope = scopes.back().second.get(); - rootScope->returnType = freshTypePack(rootScope); + NotNull borrow = NotNull(rootScope); + + rootScope->returnType = freshTypePack(borrow); + + prepopulateGlobalScope(borrow, block); // TODO: We should share the global scope. rootScope->typeBindings["nil"] = singletonTypes.nilType; @@ -71,12 +78,26 @@ void ConstraintGraphBuilder::visit(AstStatBlock* block) rootScope->typeBindings["boolean"] = singletonTypes.booleanType; rootScope->typeBindings["thread"] = singletonTypes.threadType; - visit(rootScope, block); + visitBlockWithoutChildScope(borrow, block); } -void ConstraintGraphBuilder::visit(Scope2* scope, AstStat* stat) +void ConstraintGraphBuilder::visitBlockWithoutChildScope(NotNull scope, AstStatBlock* block) { - LUAU_ASSERT(scope); + RecursionCounter counter{&recursionCount}; + + if (recursionCount >= FInt::LuauCheckRecursionLimit) + { + reportCodeTooComplex(block->location); + return; + } + + for (AstStat* stat : block->body) + visit(scope, stat); +} + +void ConstraintGraphBuilder::visit(NotNull scope, AstStat* stat) +{ + RecursionLimiter limiter{&recursionCount, FInt::LuauCheckRecursionLimit}; if (auto s = stat->as()) visit(scope, s); @@ -100,10 +121,8 @@ void ConstraintGraphBuilder::visit(Scope2* scope, AstStat* stat) LUAU_ASSERT(0); } -void ConstraintGraphBuilder::visit(Scope2* scope, AstStatLocal* local) +void ConstraintGraphBuilder::visit(NotNull scope, AstStatLocal* local) { - LUAU_ASSERT(scope); - std::vector varTypes; for (AstLocal* local : local->vars) @@ -148,23 +167,19 @@ void ConstraintGraphBuilder::visit(Scope2* scope, AstStatLocal* local) } } -void addConstraints(Constraint* constraint, Scope2* scope) +void addConstraints(Constraint* constraint, NotNull scope) { - LUAU_ASSERT(scope); - scope->constraints.reserve(scope->constraints.size() + scope->constraints.size()); for (const auto& c : scope->constraints) constraint->dependencies.push_back(NotNull{c.get()}); - for (Scope2* childScope : scope->children) + for (NotNull childScope : scope->children) addConstraints(constraint, childScope); } -void ConstraintGraphBuilder::visit(Scope2* scope, AstStatLocalFunction* function) +void ConstraintGraphBuilder::visit(NotNull scope, AstStatLocalFunction* function) { - LUAU_ASSERT(scope); - // Local // Global // Dotted path @@ -172,36 +187,31 @@ void ConstraintGraphBuilder::visit(Scope2* scope, AstStatLocalFunction* function TypeId functionType = nullptr; auto ty = scope->lookup(function->name); - if (ty.has_value()) - { - // TODO: This is duplicate definition of a local function. Is this allowed? - functionType = *ty; - } - else - { - functionType = arena->addType(BlockedTypeVar{}); - scope->bindings[function->name] = functionType; - } + LUAU_ASSERT(!ty.has_value()); // The parser ensures that every local function has a distinct Symbol for its name. - auto [actualFunctionType, innerScope] = checkFunctionSignature(scope, function->func); - innerScope->bindings[function->name] = actualFunctionType; + functionType = arena->addType(BlockedTypeVar{}); + scope->bindings[function->name] = functionType; - checkFunctionBody(innerScope, function->func); + FunctionSignature sig = checkFunctionSignature(scope, function->func); + sig.bodyScope->bindings[function->name] = sig.signature; - std::unique_ptr c{new Constraint{GeneralizationConstraint{functionType, actualFunctionType, innerScope}}}; - addConstraints(c.get(), innerScope); + checkFunctionBody(sig.bodyScope, function->func); + + std::unique_ptr c{ + new Constraint{GeneralizationConstraint{functionType, sig.signature, sig.signatureScope ? sig.signatureScope : sig.bodyScope}}}; + addConstraints(c.get(), sig.bodyScope); addConstraint(scope, std::move(c)); } -void ConstraintGraphBuilder::visit(Scope2* scope, AstStatFunction* function) +void ConstraintGraphBuilder::visit(NotNull scope, AstStatFunction* function) { // Name could be AstStatLocal, AstStatGlobal, AstStatIndexName. // With or without self TypeId functionType = nullptr; - auto [actualFunctionType, innerScope] = checkFunctionSignature(scope, function->func); + FunctionSignature sig = checkFunctionSignature(scope, function->func); if (AstExprLocal* localName = function->name->as()) { @@ -216,7 +226,7 @@ void ConstraintGraphBuilder::visit(Scope2* scope, AstStatFunction* function) functionType = arena->addType(BlockedTypeVar{}); scope->bindings[localName->local] = functionType; } - innerScope->bindings[localName->local] = actualFunctionType; + sig.bodyScope->bindings[localName->local] = sig.signature; } else if (AstExprGlobal* globalName = function->name->as()) { @@ -231,32 +241,48 @@ void ConstraintGraphBuilder::visit(Scope2* scope, AstStatFunction* function) functionType = arena->addType(BlockedTypeVar{}); rootScope->bindings[globalName->name] = functionType; } - innerScope->bindings[globalName->name] = actualFunctionType; + sig.bodyScope->bindings[globalName->name] = sig.signature; } else if (AstExprIndexName* indexName = function->name->as()) { - LUAU_ASSERT(0); // not yet implemented + TypeId containingTableType = check(scope, indexName->expr); + + functionType = arena->addType(BlockedTypeVar{}); + TypeId prospectiveTableType = + arena->addType(TableTypeVar{}); // TODO look into stack utilization. This is probably ok because it scales with AST depth. + NotNull prospectiveTable{getMutable(prospectiveTableType)}; + + Property& prop = prospectiveTable->props[indexName->index.value]; + prop.type = functionType; + prop.location = function->name->location; + + addConstraint(scope, SubtypeConstraint{containingTableType, prospectiveTableType}); + } + else if (AstExprError* err = function->name->as()) + { + functionType = singletonTypes.errorRecoveryType(); } - checkFunctionBody(innerScope, function->func); + LUAU_ASSERT(functionType != nullptr); - std::unique_ptr c{new Constraint{GeneralizationConstraint{functionType, actualFunctionType, innerScope}}}; - addConstraints(c.get(), innerScope); + checkFunctionBody(sig.bodyScope, function->func); + + std::unique_ptr c{ + new Constraint{GeneralizationConstraint{functionType, sig.signature, sig.signatureScope ? sig.signatureScope : sig.bodyScope}}}; + addConstraints(c.get(), sig.bodyScope); addConstraint(scope, std::move(c)); } -void ConstraintGraphBuilder::visit(Scope2* scope, AstStatReturn* ret) +void ConstraintGraphBuilder::visit(NotNull scope, AstStatReturn* ret) { - LUAU_ASSERT(scope); - TypePackId exprTypes = checkPack(scope, ret->list); addConstraint(scope, PackSubtypeConstraint{exprTypes, scope->returnType}); } -void ConstraintGraphBuilder::visit(Scope2* scope, AstStatBlock* block) +void ConstraintGraphBuilder::visit(NotNull scope, AstStatBlock* block) { - LUAU_ASSERT(scope); + NotNull innerScope = childScope(block->location, scope); // In order to enable mutually-recursive type aliases, we need to // populate the type bindings before we actually check any of the @@ -271,11 +297,10 @@ void ConstraintGraphBuilder::visit(Scope2* scope, AstStatBlock* block) } } - for (AstStat* stat : block->body) - visit(scope, stat); + visitBlockWithoutChildScope(innerScope, block); } -void ConstraintGraphBuilder::visit(Scope2* scope, AstStatAssign* assign) +void ConstraintGraphBuilder::visit(NotNull scope, AstStatAssign* assign) { TypePackId varPackId = checkExprList(scope, assign->vars); TypePackId valuePack = checkPack(scope, assign->values); @@ -283,21 +308,21 @@ void ConstraintGraphBuilder::visit(Scope2* scope, AstStatAssign* assign) addConstraint(scope, PackSubtypeConstraint{valuePack, varPackId}); } -void ConstraintGraphBuilder::visit(Scope2* scope, AstStatIf* ifStatement) +void ConstraintGraphBuilder::visit(NotNull scope, AstStatIf* ifStatement) { check(scope, ifStatement->condition); - Scope2* thenScope = childScope(ifStatement->thenbody->location, scope); + NotNull thenScope = childScope(ifStatement->thenbody->location, scope); visit(thenScope, ifStatement->thenbody); if (ifStatement->elsebody) { - Scope2* elseScope = childScope(ifStatement->elsebody->location, scope); + NotNull elseScope = childScope(ifStatement->elsebody->location, scope); visit(elseScope, ifStatement->elsebody); } } -void ConstraintGraphBuilder::visit(Scope2* scope, AstStatTypeAlias* alias) +void ConstraintGraphBuilder::visit(NotNull scope, AstStatTypeAlias* alias) { // TODO: Exported type aliases // TODO: Generic type aliases @@ -307,6 +332,10 @@ void ConstraintGraphBuilder::visit(Scope2* scope, AstStatTypeAlias* alias) // AST to set up typeBindings. If it's not, we've somehow skipped // this alias in that first pass. LUAU_ASSERT(it != scope->typeBindings.end()); + if (it == scope->typeBindings.end()) + { + ice->ice("Type alias does not have a pre-populated binding", alias->location); + } TypeId ty = resolveType(scope, alias->type); @@ -319,10 +348,8 @@ void ConstraintGraphBuilder::visit(Scope2* scope, AstStatTypeAlias* alias) addConstraint(scope, NameConstraint{ty, alias->name.value}); } -TypePackId ConstraintGraphBuilder::checkPack(Scope2* scope, AstArray exprs) +TypePackId ConstraintGraphBuilder::checkPack(NotNull scope, AstArray exprs) { - LUAU_ASSERT(scope); - if (exprs.size == 0) return arena->addTypePack({}); @@ -342,7 +369,7 @@ TypePackId ConstraintGraphBuilder::checkPack(Scope2* scope, AstArray e return arena->addTypePack(TypePack{std::move(types), last}); } -TypePackId ConstraintGraphBuilder::checkExprList(Scope2* scope, const AstArray& exprs) +TypePackId ConstraintGraphBuilder::checkExprList(NotNull scope, const AstArray& exprs) { TypePackId result = arena->addTypePack({}); TypePack* resultPack = getMutable(result); @@ -363,9 +390,15 @@ TypePackId ConstraintGraphBuilder::checkExprList(Scope2* scope, const AstArray scope, AstExpr* expr) { - LUAU_ASSERT(scope); + RecursionCounter counter{&recursionCount}; + + if (recursionCount >= FInt::LuauCheckRecursionLimit) + { + reportCodeTooComplex(expr->location); + return singletonTypes.errorRecoveryTypePack(); + } TypePackId result = nullptr; @@ -384,7 +417,7 @@ TypePackId ConstraintGraphBuilder::checkPack(Scope2* scope, AstExpr* expr) astOriginalCallTypes[call->func] = fnType; - TypeId instantiatedType = freshType(scope); + TypeId instantiatedType = arena->addType(BlockedTypeVar{}); addConstraint(scope, InstantiationConstraint{instantiatedType, fnType}); TypePackId rets = freshTypePack(scope); @@ -394,6 +427,13 @@ TypePackId ConstraintGraphBuilder::checkPack(Scope2* scope, AstExpr* expr) addConstraint(scope, SubtypeConstraint{inferredFnType, instantiatedType}); result = rets; } + else if (AstExprVarargs* varargs = expr->as()) + { + if (scope->varargPack) + result = *scope->varargPack; + else + result = singletonTypes.errorRecoveryTypePack(); + } else { TypeId t = check(scope, expr); @@ -405,9 +445,15 @@ TypePackId ConstraintGraphBuilder::checkPack(Scope2* scope, AstExpr* expr) return result; } -TypeId ConstraintGraphBuilder::check(Scope2* scope, AstExpr* expr) +TypeId ConstraintGraphBuilder::check(NotNull scope, AstExpr* expr) { - LUAU_ASSERT(scope); + RecursionCounter counter{&recursionCount}; + + if (recursionCount >= FInt::LuauCheckRecursionLimit) + { + reportCodeTooComplex(expr->location); + return singletonTypes.errorRecoveryType(); + } TypeId result = nullptr; @@ -435,37 +481,38 @@ TypeId ConstraintGraphBuilder::check(Scope2* scope, AstExpr* expr) if (ty) result = *ty; else - result = singletonTypes.errorRecoveryType(); // FIXME? Record an error at this point? - } - else if (auto a = expr->as()) - { - TypePackId packResult = checkPack(scope, expr); - if (auto f = first(packResult)) - return *f; - else if (get(packResult)) { - TypeId typeResult = freshType(scope); - TypePack onePack{{typeResult}, freshTypePack(scope)}; - TypePackId oneTypePack = arena->addTypePack(std::move(onePack)); - - addConstraint(scope, PackSubtypeConstraint{packResult, oneTypePack}); - - return typeResult; + /* prepopulateGlobalScope() has already added all global functions to the environment by this point, so any + * global that is not already in-scope is definitely an unknown symbol. + */ + reportError(g->location, UnknownSymbol{g->name.value}); + result = singletonTypes.errorRecoveryType(); // FIXME? Record an error at this point? } } + else if (expr->is()) + result = flattenPack(scope, expr->location, checkPack(scope, expr)); + else if (expr->is()) + result = flattenPack(scope, expr->location, checkPack(scope, expr)); else if (auto a = expr->as()) { - auto [fnType, functionScope] = checkFunctionSignature(scope, a); - checkFunctionBody(functionScope, a); - return fnType; + FunctionSignature sig = checkFunctionSignature(scope, a); + checkFunctionBody(sig.bodyScope, a); + return sig.signature; } else if (auto indexName = expr->as()) - { result = check(scope, indexName); - } + else if (auto indexExpr = expr->as()) + result = check(scope, indexExpr); else if (auto table = expr->as()) - { result = checkExprTable(scope, table); + else if (auto unary = expr->as()) + result = check(scope, unary); + else if (auto binary = expr->as()) + result = check(scope, binary); + else if (auto err = expr->as()) + { + // Open question: Should we traverse into this? + result = singletonTypes.errorRecoveryType(); } else { @@ -478,7 +525,7 @@ TypeId ConstraintGraphBuilder::check(Scope2* scope, AstExpr* expr) return result; } -TypeId ConstraintGraphBuilder::check(Scope2* scope, AstExprIndexName* indexName) +TypeId ConstraintGraphBuilder::check(NotNull scope, AstExprIndexName* indexName) { TypeId obj = check(scope, indexName->expr); TypeId result = freshType(scope); @@ -494,7 +541,67 @@ TypeId ConstraintGraphBuilder::check(Scope2* scope, AstExprIndexName* indexName) return result; } -TypeId ConstraintGraphBuilder::checkExprTable(Scope2* scope, AstExprTable* expr) +TypeId ConstraintGraphBuilder::check(NotNull scope, AstExprIndexExpr* indexExpr) +{ + TypeId obj = check(scope, indexExpr->expr); + TypeId indexType = check(scope, indexExpr->index); + + TypeId result = freshType(scope); + + TableIndexer indexer{indexType, result}; + TypeId tableType = arena->addType(TableTypeVar{TableTypeVar::Props{}, TableIndexer{indexType, result}, TypeLevel{}, TableState::Free}); + + addConstraint(scope, SubtypeConstraint{obj, tableType}); + + return result; +} + +TypeId ConstraintGraphBuilder::check(NotNull scope, AstExprUnary* unary) +{ + TypeId operandType = check(scope, unary->expr); + + switch (unary->op) + { + case AstExprUnary::Minus: + { + TypeId resultType = arena->addType(BlockedTypeVar{}); + addConstraint(scope, UnaryConstraint{AstExprUnary::Minus, operandType, resultType}); + return resultType; + } + default: + LUAU_ASSERT(0); + } + + LUAU_UNREACHABLE(); + return singletonTypes.errorRecoveryType(); +} + +TypeId ConstraintGraphBuilder::check(NotNull scope, AstExprBinary* binary) +{ + TypeId leftType = check(scope, binary->left); + TypeId rightType = check(scope, binary->right); + switch (binary->op) + { + case AstExprBinary::Or: + { + addConstraint(scope, SubtypeConstraint{leftType, rightType}); + return leftType; + } + case AstExprBinary::Sub: + { + TypeId resultType = arena->addType(BlockedTypeVar{}); + addConstraint(scope, BinaryConstraint{AstExprBinary::Sub, leftType, rightType, resultType}); + return resultType; + } + default: + LUAU_ASSERT(0); + } + + LUAU_ASSERT(0); + return nullptr; +} + +TypeId ConstraintGraphBuilder::checkExprTable(NotNull scope, AstExprTable* expr) { TypeId ty = arena->addType(TableTypeVar{}); TableTypeVar* ttv = getMutable(ty); @@ -515,6 +622,8 @@ TypeId ConstraintGraphBuilder::checkExprTable(Scope2* scope, AstExprTable* expr) for (const AstExprTable::Item& item : expr->items) { TypeId itemTy = check(scope, item.value); + if (get(follow(itemTy))) + return ty; if (item.key) { @@ -542,47 +651,111 @@ TypeId ConstraintGraphBuilder::checkExprTable(Scope2* scope, AstExprTable* expr) return ty; } -std::pair ConstraintGraphBuilder::checkFunctionSignature(Scope2* parent, AstExprFunction* fn) +ConstraintGraphBuilder::FunctionSignature ConstraintGraphBuilder::checkFunctionSignature(NotNull parent, AstExprFunction* fn) { - Scope2* innerScope = childScope(fn->body->location, parent); - TypePackId returnType = freshTypePack(innerScope); - innerScope->returnType = returnType; + Scope2* signatureScope = nullptr; + Scope2* bodyScope = nullptr; + TypePackId returnType = nullptr; + + std::vector genericTypes; + std::vector genericTypePacks; + + bool hasGenerics = fn->generics.size > 0 || fn->genericPacks.size > 0; + + // If we don't have any generics, we can save some memory and compute by not + // creating the signatureScope, which is only used to scope the declared + // generics properly. + if (hasGenerics) + { + NotNull signatureBorrow = childScope(fn->location, parent); + signatureScope = signatureBorrow.get(); + + // We need to assign returnType before creating bodyScope so that the + // return type gets propogated to bodyScope. + returnType = freshTypePack(signatureBorrow); + signatureScope->returnType = returnType; + + bodyScope = childScope(fn->body->location, signatureBorrow).get(); + + std::vector> genericDefinitions = createGenerics(signatureBorrow, fn->generics); + std::vector> genericPackDefinitions = createGenericPacks(signatureBorrow, fn->genericPacks); + + // We do not support default values on function generics, so we only + // care about the types involved. + for (const auto& [name, g] : genericDefinitions) + { + genericTypes.push_back(g.ty); + signatureScope->typeBindings[name] = g.ty; + } + + for (const auto& [name, g] : genericPackDefinitions) + { + genericTypePacks.push_back(g.tp); + signatureScope->typePackBindings[name] = g.tp; + } + } + else + { + NotNull bodyBorrow = childScope(fn->body->location, parent); + bodyScope = bodyBorrow.get(); + + returnType = freshTypePack(bodyBorrow); + bodyBorrow->returnType = returnType; + + // To eliminate the need to branch on hasGenerics below, we say that the + // signature scope is the body scope when there is no real signature + // scope. + signatureScope = bodyScope; + } + + NotNull bodyBorrow = NotNull(bodyScope); + NotNull signatureBorrow = NotNull(signatureScope); if (fn->returnAnnotation) { - TypePackId annotatedRetType = resolveTypePack(innerScope, *fn->returnAnnotation); - addConstraint(innerScope, PackSubtypeConstraint{returnType, annotatedRetType}); + TypePackId annotatedRetType = resolveTypePack(signatureBorrow, *fn->returnAnnotation); + addConstraint(signatureBorrow, PackSubtypeConstraint{returnType, annotatedRetType}); } std::vector argTypes; for (AstLocal* local : fn->args) { - TypeId t = freshType(innerScope); + TypeId t = freshType(signatureBorrow); argTypes.push_back(t); - innerScope->bindings[local] = t; + signatureScope->bindings[local] = t; if (local->annotation) { - TypeId argAnnotation = resolveType(innerScope, local->annotation); - addConstraint(innerScope, SubtypeConstraint{t, argAnnotation}); + TypeId argAnnotation = resolveType(signatureBorrow, local->annotation); + addConstraint(signatureBorrow, SubtypeConstraint{t, argAnnotation}); } } // TODO: Vararg annotation. + // TODO: Preserve argument names in the function's type. FunctionTypeVar actualFunction{arena->addTypePack(argTypes), returnType}; + actualFunction.hasNoGenerics = !hasGenerics; + actualFunction.generics = std::move(genericTypes); + actualFunction.genericPacks = std::move(genericTypePacks); + TypeId actualFunctionType = arena->addType(std::move(actualFunction)); LUAU_ASSERT(actualFunctionType); astTypes[fn] = actualFunctionType; - return {actualFunctionType, innerScope}; + return { + /* signature */ actualFunctionType, + // Undo the workaround we made above: if there's no signature scope, + // don't report it. + /* signatureScope */ hasGenerics ? signatureScope : nullptr, + /* bodyScope */ bodyBorrow, + }; } -void ConstraintGraphBuilder::checkFunctionBody(Scope2* scope, AstExprFunction* fn) +void ConstraintGraphBuilder::checkFunctionBody(NotNull scope, AstExprFunction* fn) { - for (AstStat* stat : fn->body->body) - visit(scope, stat); + visitBlockWithoutChildScope(scope, fn->body); // If it is possible for execution to reach the end of the function, the return type must be compatible with () @@ -593,7 +766,7 @@ void ConstraintGraphBuilder::checkFunctionBody(Scope2* scope, AstExprFunction* f } } -TypeId ConstraintGraphBuilder::resolveType(Scope2* scope, AstType* ty) +TypeId ConstraintGraphBuilder::resolveType(NotNull scope, AstType* ty) { TypeId result = nullptr; @@ -636,29 +809,73 @@ TypeId ConstraintGraphBuilder::resolveType(Scope2* scope, AstType* ty) } else if (auto fn = ty->as()) { - // TODO: Generic functions. - // TODO: Scope (though it may not be needed). // TODO: Recursion limit. - TypePackId argTypes = resolveTypePack(scope, fn->argTypes); - TypePackId returnTypes = resolveTypePack(scope, fn->returnTypes); + bool hasGenerics = fn->generics.size > 0 || fn->genericPacks.size > 0; + Scope2* signatureScope = nullptr; - // TODO: Is this the right constructor to use? - result = arena->addType(FunctionTypeVar{argTypes, returnTypes}); + std::vector genericTypes; + std::vector genericTypePacks; - FunctionTypeVar* ftv = getMutable(result); - ftv->argNames.reserve(fn->argNames.size); + // If we don't have generics, we do not need to generate a child scope + // for the generic bindings to live on. + if (hasGenerics) + { + NotNull signatureBorrow = childScope(fn->location, scope); + signatureScope = signatureBorrow.get(); + + std::vector> genericDefinitions = createGenerics(signatureBorrow, fn->generics); + std::vector> genericPackDefinitions = createGenericPacks(signatureBorrow, fn->genericPacks); + + for (const auto& [name, g] : genericDefinitions) + { + genericTypes.push_back(g.ty); + signatureBorrow->typeBindings[name] = g.ty; + } + + for (const auto& [name, g] : genericPackDefinitions) + { + genericTypePacks.push_back(g.tp); + signatureBorrow->typePackBindings[name] = g.tp; + } + } + else + { + // To eliminate the need to branch on hasGenerics below, we say that + // the signature scope is the parent scope if we don't have + // generics. + signatureScope = scope.get(); + } + + NotNull signatureBorrow(signatureScope); + + TypePackId argTypes = resolveTypePack(signatureBorrow, fn->argTypes); + TypePackId returnTypes = resolveTypePack(signatureBorrow, fn->returnTypes); + + // TODO: FunctionTypeVar needs a pointer to the scope so that we know + // how to quantify/instantiate it. + FunctionTypeVar ftv{argTypes, returnTypes}; + + // This replicates the behavior of the appropriate FunctionTypeVar + // constructors. + ftv.hasNoGenerics = !hasGenerics; + ftv.generics = std::move(genericTypes); + ftv.genericPacks = std::move(genericTypePacks); + + ftv.argNames.reserve(fn->argNames.size); for (const auto& el : fn->argNames) { if (el) { const auto& [name, location] = *el; - ftv->argNames.push_back(FunctionArgument{name.value, location}); + ftv.argNames.push_back(FunctionArgument{name.value, location}); } else { - ftv->argNames.push_back(std::nullopt); + ftv.argNames.push_back(std::nullopt); } } + + result = arena->addType(std::move(ftv)); } else if (auto tof = ty->as()) { @@ -710,7 +927,7 @@ TypeId ConstraintGraphBuilder::resolveType(Scope2* scope, AstType* ty) return result; } -TypePackId ConstraintGraphBuilder::resolveTypePack(Scope2* scope, AstTypePack* tp) +TypePackId ConstraintGraphBuilder::resolveTypePack(NotNull scope, AstTypePack* tp) { TypePackId result; if (auto expl = tp->as()) @@ -736,7 +953,7 @@ TypePackId ConstraintGraphBuilder::resolveTypePack(Scope2* scope, AstTypePack* t return result; } -TypePackId ConstraintGraphBuilder::resolveTypePack(Scope2* scope, const AstTypeList& list) +TypePackId ConstraintGraphBuilder::resolveTypePack(NotNull scope, const AstTypeList& list) { std::vector head; @@ -754,16 +971,108 @@ TypePackId ConstraintGraphBuilder::resolveTypePack(Scope2* scope, const AstTypeL return arena->addTypePack(TypePack{head, tail}); } -void collectConstraints(std::vector>& result, Scope2* scope) +std::vector> ConstraintGraphBuilder::createGenerics(NotNull scope, AstArray generics) +{ + std::vector> result; + for (const auto& generic : generics) + { + TypeId genericTy = arena->addType(GenericTypeVar{scope, generic.name.value}); + std::optional defaultTy = std::nullopt; + + if (generic.defaultValue) + defaultTy = resolveType(scope, generic.defaultValue); + + result.push_back({generic.name.value, GenericTypeDefinition{ + genericTy, + defaultTy, + }}); + } + + return result; +} + +std::vector> ConstraintGraphBuilder::createGenericPacks( + NotNull scope, AstArray generics) +{ + std::vector> result; + for (const auto& generic : generics) + { + TypePackId genericTy = arena->addTypePack(TypePackVar{GenericTypePack{scope, generic.name.value}}); + std::optional defaultTy = std::nullopt; + + if (generic.defaultValue) + defaultTy = resolveTypePack(scope, generic.defaultValue); + + result.push_back({generic.name.value, GenericTypePackDefinition{ + genericTy, + defaultTy, + }}); + } + + return result; +} + +TypeId ConstraintGraphBuilder::flattenPack(NotNull scope, Location location, TypePackId tp) +{ + if (auto f = first(tp)) + return *f; + + TypeId typeResult = freshType(scope); + TypePack onePack{{typeResult}, freshTypePack(scope)}; + TypePackId oneTypePack = arena->addTypePack(std::move(onePack)); + + addConstraint(scope, PackSubtypeConstraint{tp, oneTypePack}); + + return typeResult; +} + +void ConstraintGraphBuilder::reportError(Location location, TypeErrorData err) +{ + errors.push_back(TypeError{location, moduleName, std::move(err)}); +} + +void ConstraintGraphBuilder::reportCodeTooComplex(Location location) +{ + errors.push_back(TypeError{location, moduleName, CodeTooComplex{}}); +} + +struct GlobalPrepopulator : AstVisitor +{ + const NotNull globalScope; + const NotNull arena; + + GlobalPrepopulator(NotNull globalScope, NotNull arena) + : globalScope(globalScope) + , arena(arena) + { + } + + bool visit(AstStatFunction* function) override + { + if (AstExprGlobal* g = function->name->as()) + globalScope->bindings[g->name] = arena->addType(BlockedTypeVar{}); + + return true; + } +}; + +void ConstraintGraphBuilder::prepopulateGlobalScope(NotNull globalScope, AstStatBlock* program) +{ + GlobalPrepopulator gp{NotNull{globalScope}, arena}; + + program->visit(&gp); +} + +void collectConstraints(std::vector>& result, NotNull scope) { for (const auto& c : scope->constraints) result.push_back(NotNull{c.get()}); - for (Scope2* child : scope->children) + for (NotNull child : scope->children) collectConstraints(result, child); } -std::vector> collectConstraints(Scope2* rootScope) +std::vector> collectConstraints(NotNull rootScope) { std::vector> result; collectConstraints(result, rootScope); diff --git a/Analysis/src/ConstraintSolver.cpp b/Analysis/src/ConstraintSolver.cpp index 9e355236..077a4e28 100644 --- a/Analysis/src/ConstraintSolver.cpp +++ b/Analysis/src/ConstraintSolver.cpp @@ -13,7 +13,7 @@ LUAU_FASTFLAGVARIABLE(DebugLuauLogSolverToJson, false); namespace Luau { -[[maybe_unused]] static void dumpBindings(Scope2* scope, ToStringOptions& opts) +[[maybe_unused]] static void dumpBindings(NotNull scope, ToStringOptions& opts) { for (const auto& [k, v] : scope->bindings) { @@ -22,22 +22,22 @@ namespace Luau printf("\t%s : %s\n", k.c_str(), d.name.c_str()); } - for (Scope2* child : scope->children) + for (NotNull child : scope->children) dumpBindings(child, opts); } -static void dumpConstraints(Scope2* scope, ToStringOptions& opts) +static void dumpConstraints(NotNull scope, ToStringOptions& opts) { for (const ConstraintPtr& c : scope->constraints) { printf("\t%s\n", toString(*c, opts).c_str()); } - for (Scope2* child : scope->children) + for (NotNull child : scope->children) dumpConstraints(child, opts); } -void dump(Scope2* rootScope, ToStringOptions& opts) +void dump(NotNull rootScope, ToStringOptions& opts) { printf("constraints:\n"); dumpConstraints(rootScope, opts); @@ -55,7 +55,7 @@ void dump(ConstraintSolver* cs, ToStringOptions& opts) } } -ConstraintSolver::ConstraintSolver(TypeArena* arena, Scope2* rootScope) +ConstraintSolver::ConstraintSolver(TypeArena* arena, NotNull rootScope) : arena(arena) , constraints(collectConstraints(rootScope)) , rootScope(rootScope) @@ -180,6 +180,10 @@ bool ConstraintSolver::tryDispatch(NotNull constraint, bool fo success = tryDispatch(*gc, constraint, force); else if (auto ic = get(*constraint)) success = tryDispatch(*ic, constraint, force); + else if (auto uc = get(*constraint)) + success = tryDispatch(*uc, constraint, force); + else if (auto bc = get(*constraint)) + success = tryDispatch(*bc, constraint, force); else if (auto nc = get(*constraint)) success = tryDispatch(*nc, constraint); else @@ -246,12 +250,65 @@ bool ConstraintSolver::tryDispatch(const InstantiationConstraint& c, NotNull instantiated = inst.substitute(c.superType); LUAU_ASSERT(instantiated); // TODO FIXME HANDLE THIS - unify(c.subType, *instantiated); + if (isBlocked(c.subType)) + asMutable(c.subType)->ty.emplace(*instantiated); + else + unify(c.subType, *instantiated); + unblock(c.subType); return true; } +bool ConstraintSolver::tryDispatch(const UnaryConstraint& c, NotNull constraint, bool force) +{ + TypeId operandType = follow(c.operandType); + + if (isBlocked(operandType)) + return block(operandType, constraint); + + if (get(operandType)) + return block(operandType, constraint); + + LUAU_ASSERT(get(c.resultType)); + + if (isNumber(operandType) || get(operandType) || get(operandType)) + { + asMutable(c.resultType)->ty.emplace(c.operandType); + return true; + } + + LUAU_ASSERT(0); // TODO metatable handling + return false; +} + +bool ConstraintSolver::tryDispatch(const BinaryConstraint& c, NotNull constraint, bool force) +{ + TypeId leftType = follow(c.leftType); + TypeId rightType = follow(c.rightType); + + if (isBlocked(leftType) || isBlocked(rightType)) + { + block(leftType, constraint); + block(rightType, constraint); + return false; + } + + if (isNumber(leftType)) + { + unify(leftType, rightType); + asMutable(c.resultType)->ty.emplace(leftType); + return true; + } + + if (get(leftType) && !force) + return block(leftType, constraint); + + // TODO metatables, classes + + return true; +} + bool ConstraintSolver::tryDispatch(const NameConstraint& c, NotNull constraint) { if (isBlocked(c.namedType)) diff --git a/Analysis/src/EmbeddedBuiltinDefinitions.cpp b/Analysis/src/EmbeddedBuiltinDefinitions.cpp index 2407e3ef..1b5275fd 100644 --- a/Analysis/src/EmbeddedBuiltinDefinitions.cpp +++ b/Analysis/src/EmbeddedBuiltinDefinitions.cpp @@ -1,6 +1,8 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #include "Luau/BuiltinDefinitions.h" +LUAU_FASTFLAG(LuauCheckLenMT) + namespace Luau { @@ -202,7 +204,13 @@ declare function unpack(tab: {V}, i: number?, j: number?): ...V std::string getBuiltinDefinitionSource() { - return kBuiltinDefinitionLuaSrc; + std::string result = kBuiltinDefinitionLuaSrc; + + // TODO: move this into kBuiltinDefinitionLuaSrc + if (FFlag::LuauCheckLenMT) + result += "declare function rawlen(obj: {[K]: V} | string): number\n"; + + return result; } } // namespace Luau diff --git a/Analysis/src/Frontend.cpp b/Analysis/src/Frontend.cpp index 85c5dbc8..4cfaa112 100644 --- a/Analysis/src/Frontend.cpp +++ b/Analysis/src/Frontend.cpp @@ -787,14 +787,32 @@ const SourceModule* Frontend::getSourceModule(const ModuleName& moduleName) cons return const_cast(this)->getSourceModule(moduleName); } +NotNull Frontend::getGlobalScope2() +{ + if (!globalScope2) + { + const SingletonTypes& singletonTypes = getSingletonTypes(); + + globalScope2 = std::make_unique(); + globalScope2->typeBindings["nil"] = singletonTypes.nilType; + globalScope2->typeBindings["number"] = singletonTypes.numberType; + globalScope2->typeBindings["string"] = singletonTypes.stringType; + globalScope2->typeBindings["boolean"] = singletonTypes.booleanType; + globalScope2->typeBindings["thread"] = singletonTypes.threadType; + } + + return NotNull(globalScope2.get()); +} + ModulePtr Frontend::check(const SourceModule& sourceModule, Mode mode, const ScopePtr& environmentScope) { ModulePtr result = std::make_shared(); - ConstraintGraphBuilder cgb{&result->internalTypes}; + ConstraintGraphBuilder cgb{sourceModule.name, &result->internalTypes, NotNull(&iceHandler), getGlobalScope2()}; cgb.visit(sourceModule.root); + result->errors = std::move(cgb.errors); - ConstraintSolver cs{&result->internalTypes, cgb.rootScope}; + ConstraintSolver cs{&result->internalTypes, NotNull(cgb.rootScope)}; cs.run(); result->scope2s = std::move(cgb.scopes); diff --git a/Analysis/src/Normalize.cpp b/Analysis/src/Normalize.cpp index d36665e2..8ce7f742 100644 --- a/Analysis/src/Normalize.cpp +++ b/Analysis/src/Normalize.cpp @@ -5,7 +5,6 @@ #include #include "Luau/Clone.h" -#include "Luau/Substitution.h" #include "Luau/Unifier.h" #include "Luau/VisitTypeVar.h" @@ -16,7 +15,6 @@ LUAU_FASTINTVARIABLE(LuauNormalizeIterationLimit, 1200); LUAU_FASTFLAGVARIABLE(LuauNormalizeCombineTableFix, false); LUAU_FASTFLAGVARIABLE(LuauNormalizeFlagIsConservative, false); LUAU_FASTFLAGVARIABLE(LuauNormalizeCombineEqFix, false); -LUAU_FASTFLAGVARIABLE(LuauReplaceReplacer, false); LUAU_FASTFLAG(LuauQuantifyConstrained) namespace Luau @@ -25,238 +23,33 @@ namespace Luau namespace { -struct Replacer : Substitution +struct Replacer { + TypeArena* arena; TypeId sourceType; TypeId replacedType; - DenseHashMap replacedTypes{nullptr}; - DenseHashMap replacedPacks{nullptr}; + DenseHashMap newTypes; Replacer(TypeArena* arena, TypeId sourceType, TypeId replacedType) - : Substitution(TxnLog::empty(), arena) + : arena(arena) , sourceType(sourceType) , replacedType(replacedType) + , newTypes(nullptr) { } - bool isDirty(TypeId ty) override - { - if (!sourceType) - return false; - - auto vecHasSourceType = [sourceType = sourceType](const auto& vec) { - return end(vec) != std::find(begin(vec), end(vec), sourceType); - }; - - // Walk every kind of TypeVar and find pointers to sourceType - if (auto t = get(ty)) - return false; - else if (auto t = get(ty)) - return false; - else if (auto t = get(ty)) - return false; - else if (auto t = get(ty)) - return false; - else if (auto t = get(ty)) - return vecHasSourceType(t->parts); - else if (auto t = get(ty)) - return false; - else if (auto t = get(ty)) - { - if (vecHasSourceType(t->generics)) - return true; - - return false; - } - else if (auto t = get(ty)) - { - if (t->boundTo) - return *t->boundTo == sourceType; - - for (const auto& [_name, prop] : t->props) - { - if (prop.type == sourceType) - return true; - } - - if (auto indexer = t->indexer) - { - if (indexer->indexType == sourceType || indexer->indexResultType == sourceType) - return true; - } - - if (vecHasSourceType(t->instantiatedTypeParams)) - return true; - - return false; - } - else if (auto t = get(ty)) - return t->table == sourceType || t->metatable == sourceType; - else if (auto t = get(ty)) - return false; - else if (auto t = get(ty)) - return false; - else if (auto t = get(ty)) - return vecHasSourceType(t->options); - else if (auto t = get(ty)) - return vecHasSourceType(t->parts); - else if (auto t = get(ty)) - return false; - - LUAU_ASSERT(!"Luau::Replacer::isDirty internal error: Unknown TypeVar type"); - LUAU_UNREACHABLE(); - } - - bool isDirty(TypePackId tp) override - { - if (auto it = replacedPacks.find(tp)) - return false; - - if (auto pack = get(tp)) - { - for (TypeId ty : pack->head) - { - if (ty == sourceType) - return true; - } - return false; - } - else if (auto vtp = get(tp)) - return vtp->ty == sourceType; - else - return false; - } - - TypeId clean(TypeId ty) override - { - LUAU_ASSERT(sourceType && replacedType); - - // Walk every kind of TypeVar and create a copy with sourceType replaced by replacedType - // Before returning, memoize the result for later use. - - // Helpfully, Substitution::clone() only shallow-clones the kinds of types that we care to work with. This - // function returns the identity for things like primitives. - TypeId res = clone(ty); - - if (auto t = get(res)) - LUAU_ASSERT(!"Impossible"); - else if (auto t = get(res)) - LUAU_ASSERT(!"Impossible"); - else if (auto t = get(res)) - LUAU_ASSERT(!"Impossible"); - else if (auto t = get(res)) - LUAU_ASSERT(!"Impossible"); - else if (auto t = getMutable(res)) - { - for (TypeId& part : t->parts) - { - if (part == sourceType) - part = replacedType; - } - } - else if (auto t = get(res)) - LUAU_ASSERT(!"Impossible"); - else if (auto t = getMutable(res)) - { - // The constituent typepacks are cleaned separately. We just need to walk the generics array. - for (TypeId& g : t->generics) - { - if (g == sourceType) - g = replacedType; - } - } - else if (auto t = getMutable(res)) - { - for (auto& [_key, prop] : t->props) - { - if (prop.type == sourceType) - prop.type = replacedType; - } - } - else if (auto t = getMutable(res)) - { - if (t->table == sourceType) - t->table = replacedType; - if (t->metatable == sourceType) - t->table = replacedType; - } - else if (auto t = get(res)) - LUAU_ASSERT(!"Impossible"); - else if (auto t = get(res)) - LUAU_ASSERT(!"Impossible"); - else if (auto t = getMutable(res)) - { - for (TypeId& option : t->options) - { - if (option == sourceType) - option = replacedType; - } - } - else if (auto t = getMutable(res)) - { - for (TypeId& part : t->parts) - { - if (part == sourceType) - part = replacedType; - } - } - else if (auto t = get(res)) - LUAU_ASSERT(!"Impossible"); - else - LUAU_ASSERT(!"Luau::Replacer::clean internal error: Unknown TypeVar type"); - - replacedTypes[ty] = res; - return res; - } - - TypePackId clean(TypePackId tp) override - { - TypePackId res = clone(tp); - - if (auto pack = getMutable(res)) - { - for (TypeId& type : pack->head) - { - if (type == sourceType) - type = replacedType; - } - } - else if (auto vtp = getMutable(res)) - { - if (vtp->ty == sourceType) - vtp->ty = replacedType; - } - - replacedPacks[tp] = res; - return res; - } - TypeId smartClone(TypeId t) { - if (FFlag::LuauReplaceReplacer) - { - // The new smartClone is just a memoized clone() - // TODO: Remove the Substitution base class and all other methods from this struct. - // Add DenseHashMap newTypes; - t = log->follow(t); - TypeId* res = newTypes.find(t); - if (res) - return *res; - - TypeId result = shallowClone(t, *arena, TxnLog::empty()); - newTypes[t] = result; - newTypes[result] = result; - - return result; - } - else - { - std::optional res = replace(t); - LUAU_ASSERT(res.has_value()); // TODO think about this - if (*res == t) - return clone(t); + t = follow(t); + TypeId* res = newTypes.find(t); + if (res) return *res; - } + + TypeId result = shallowClone(t, *arena, TxnLog::empty()); + newTypes[t] = result; + newTypes[result] = result; + + return result; } }; diff --git a/Analysis/src/Quantify.cpp b/Analysis/src/Quantify.cpp index 40e14c68..294c479d 100644 --- a/Analysis/src/Quantify.cpp +++ b/Analysis/src/Quantify.cpp @@ -8,6 +8,7 @@ #include "Luau/VisitTypeVar.h" LUAU_FASTFLAG(LuauAlwaysQuantify); +LUAU_FASTFLAG(DebugLuauSharedSelf) LUAU_FASTFLAG(DebugLuauDeferredConstraintResolution); LUAU_FASTFLAGVARIABLE(LuauQuantifyConstrained, false) @@ -158,24 +159,61 @@ struct Quantifier final : TypeVarOnceVisitor void quantify(TypeId ty, TypeLevel level) { - Quantifier q{level}; - q.traverse(ty); - - FunctionTypeVar* ftv = getMutable(ty); - LUAU_ASSERT(ftv); - if (FFlag::LuauAlwaysQuantify) + if (FFlag::DebugLuauSharedSelf) { - ftv->generics.insert(ftv->generics.end(), q.generics.begin(), q.generics.end()); - ftv->genericPacks.insert(ftv->genericPacks.end(), q.genericPacks.begin(), q.genericPacks.end()); + ty = follow(ty); + + if (auto ttv = getTableType(ty); ttv && ttv->selfTy) + { + Quantifier selfQ{level}; + selfQ.traverse(*ttv->selfTy); + + Quantifier q{level}; + q.traverse(ty); + + for (const auto& [_, prop] : ttv->props) + { + auto ftv = getMutable(follow(prop.type)); + if (!ftv || !ftv->hasSelf) + continue; + + if (Luau::first(ftv->argTypes) == ttv->selfTy) + { + ftv->generics.insert(ftv->generics.end(), selfQ.generics.begin(), selfQ.generics.end()); + ftv->genericPacks.insert(ftv->genericPacks.end(), selfQ.genericPacks.begin(), selfQ.genericPacks.end()); + } + } + } + else if (auto ftv = getMutable(ty)) + { + Quantifier q{level}; + q.traverse(ty); + + ftv->generics.insert(ftv->generics.end(), q.generics.begin(), q.generics.end()); + ftv->genericPacks.insert(ftv->genericPacks.end(), q.genericPacks.begin(), q.genericPacks.end()); + + if (ftv->generics.empty() && ftv->genericPacks.empty() && !q.seenMutableType && !q.seenGenericType) + ftv->hasNoGenerics = true; + } } else { - ftv->generics = q.generics; - ftv->genericPacks = q.genericPacks; - } + Quantifier q{level}; + q.traverse(ty); - if (ftv->generics.empty() && ftv->genericPacks.empty() && !q.seenMutableType && !q.seenGenericType) - ftv->hasNoGenerics = true; + FunctionTypeVar* ftv = getMutable(ty); + LUAU_ASSERT(ftv); + if (FFlag::LuauAlwaysQuantify) + { + ftv->generics.insert(ftv->generics.end(), q.generics.begin(), q.generics.end()); + ftv->genericPacks.insert(ftv->genericPacks.end(), q.genericPacks.begin(), q.genericPacks.end()); + } + else + { + ftv->generics = q.generics; + ftv->genericPacks = q.genericPacks; + } + } } void quantify(TypeId ty, Scope2* scope) @@ -206,8 +244,8 @@ struct PureQuantifier : Substitution std::vector insertedGenerics; std::vector insertedGenericPacks; - PureQuantifier(const TxnLog* log, TypeArena* arena, Scope2* scope) - : Substitution(log, arena) + PureQuantifier(TypeArena* arena, Scope2* scope) + : Substitution(TxnLog::empty(), arena) , scope(scope) { } @@ -286,7 +324,7 @@ struct PureQuantifier : Substitution TypeId quantify(TypeArena* arena, TypeId ty, Scope2* scope) { - PureQuantifier quantifier{TxnLog::empty(), arena, scope}; + PureQuantifier quantifier{arena, scope}; std::optional result = quantifier.substitute(ty); LUAU_ASSERT(result); @@ -294,8 +332,7 @@ TypeId quantify(TypeArena* arena, TypeId ty, Scope2* scope) LUAU_ASSERT(ftv); ftv->generics.insert(ftv->generics.end(), quantifier.insertedGenerics.begin(), quantifier.insertedGenerics.end()); ftv->genericPacks.insert(ftv->genericPacks.end(), quantifier.insertedGenericPacks.begin(), quantifier.insertedGenericPacks.end()); - - // TODO: Set hasNoGenerics. + ftv->hasNoGenerics = ftv->generics.empty() && ftv->genericPacks.empty(); return *result; } diff --git a/Analysis/src/Scope.cpp b/Analysis/src/Scope.cpp index 66aaee1f..247a9dd6 100644 --- a/Analysis/src/Scope.cpp +++ b/Analysis/src/Scope.cpp @@ -153,4 +153,19 @@ std::optional Scope2::lookupTypeBinding(const Name& name) return std::nullopt; } +std::optional Scope2::lookupTypePackBinding(const Name& name) +{ + Scope2* s = this; + while (s) + { + auto it = s->typePackBindings.find(name); + if (it != s->typePackBindings.end()) + return it->second; + + s = s->parent; + } + + return std::nullopt; +} + } // namespace Luau diff --git a/Analysis/src/ToString.cpp b/Analysis/src/ToString.cpp index 7a458964..bbbad9d5 100644 --- a/Analysis/src/ToString.cpp +++ b/Analysis/src/ToString.cpp @@ -1377,51 +1377,74 @@ std::string generateName(size_t i) return n; } -std::string toString(const Constraint& c, ToStringOptions& opts) +std::string toString(const Constraint& constraint, ToStringOptions& opts) { - if (const SubtypeConstraint* sc = Luau::get_if(&c.c)) - { - ToStringResult subStr = toStringDetailed(sc->subType, opts); - opts.nameMap = std::move(subStr.nameMap); - ToStringResult superStr = toStringDetailed(sc->superType, opts); - opts.nameMap = std::move(superStr.nameMap); - return subStr.name + " <: " + superStr.name; - } - else if (const PackSubtypeConstraint* psc = Luau::get_if(&c.c)) - { - ToStringResult subStr = toStringDetailed(psc->subPack, opts); - opts.nameMap = std::move(subStr.nameMap); - ToStringResult superStr = toStringDetailed(psc->superPack, opts); - opts.nameMap = std::move(superStr.nameMap); - return subStr.name + " <: " + superStr.name; - } - else if (const GeneralizationConstraint* gc = Luau::get_if(&c.c)) - { - ToStringResult subStr = toStringDetailed(gc->generalizedType, opts); - opts.nameMap = std::move(subStr.nameMap); - ToStringResult superStr = toStringDetailed(gc->sourceType, opts); - opts.nameMap = std::move(superStr.nameMap); - return subStr.name + " ~ gen " + superStr.name; - } - else if (const InstantiationConstraint* ic = Luau::get_if(&c.c)) - { - ToStringResult subStr = toStringDetailed(ic->subType, opts); - opts.nameMap = std::move(subStr.nameMap); - ToStringResult superStr = toStringDetailed(ic->superType, opts); - opts.nameMap = std::move(superStr.nameMap); - return subStr.name + " ~ inst " + superStr.name; - } - else if (const NameConstraint* nc = Luau::get(c)) - { - ToStringResult namedStr = toStringDetailed(nc->namedType, opts); - opts.nameMap = std::move(namedStr.nameMap); - return "@name(" + namedStr.name + ") = " + nc->name; - } - else - { - LUAU_ASSERT(false); - return ""; - } + auto go = [&opts](auto&& c) { + using T = std::decay_t; + + if constexpr (std::is_same_v) + { + ToStringResult subStr = toStringDetailed(c.subType, opts); + opts.nameMap = std::move(subStr.nameMap); + ToStringResult superStr = toStringDetailed(c.superType, opts); + opts.nameMap = std::move(superStr.nameMap); + return subStr.name + " <: " + superStr.name; + } + else if constexpr (std::is_same_v) + { + ToStringResult subStr = toStringDetailed(c.subPack, opts); + opts.nameMap = std::move(subStr.nameMap); + ToStringResult superStr = toStringDetailed(c.superPack, opts); + opts.nameMap = std::move(superStr.nameMap); + return subStr.name + " <: " + superStr.name; + } + else if constexpr (std::is_same_v) + { + ToStringResult subStr = toStringDetailed(c.generalizedType, opts); + opts.nameMap = std::move(subStr.nameMap); + ToStringResult superStr = toStringDetailed(c.sourceType, opts); + opts.nameMap = std::move(superStr.nameMap); + return subStr.name + " ~ gen " + superStr.name; + } + else if constexpr (std::is_same_v) + { + ToStringResult subStr = toStringDetailed(c.subType, opts); + opts.nameMap = std::move(subStr.nameMap); + ToStringResult superStr = toStringDetailed(c.superType, opts); + opts.nameMap = std::move(superStr.nameMap); + return subStr.name + " ~ inst " + superStr.name; + } + else if constexpr (std::is_same_v) + { + ToStringResult resultStr = toStringDetailed(c.resultType, opts); + opts.nameMap = std::move(resultStr.nameMap); + ToStringResult operandStr = toStringDetailed(c.operandType, opts); + opts.nameMap = std::move(operandStr.nameMap); + + return resultStr.name + " ~ Unary<" + toString(c.op) + ", " + operandStr.name + ">"; + } + else if constexpr (std::is_same_v) + { + ToStringResult resultStr = toStringDetailed(c.resultType); + opts.nameMap = std::move(resultStr.nameMap); + ToStringResult leftStr = toStringDetailed(c.leftType); + opts.nameMap = std::move(leftStr.nameMap); + ToStringResult rightStr = toStringDetailed(c.rightType); + opts.nameMap = std::move(rightStr.nameMap); + + return resultStr.name + " ~ Binary<" + toString(c.op) + ", " + leftStr.name + ", " + rightStr.name + ">"; + } + else if constexpr (std::is_same_v) + { + ToStringResult namedStr = toStringDetailed(c.namedType, opts); + opts.nameMap = std::move(namedStr.nameMap); + return "@name(" + namedStr.name + ") = " + c.name; + } + else + static_assert(always_false_v, "Non-exhaustive constraint switch"); + }; + + return visit(go, constraint.c); } std::string dump(const Constraint& c) diff --git a/Analysis/src/TypeChecker2.cpp b/Analysis/src/TypeChecker2.cpp index 63e5800f..30e498af 100644 --- a/Analysis/src/TypeChecker2.cpp +++ b/Analysis/src/TypeChecker2.cpp @@ -6,8 +6,10 @@ #include "Luau/Ast.h" #include "Luau/AstQuery.h" #include "Luau/Clone.h" +#include "Luau/Instantiation.h" #include "Luau/Normalize.h" -#include "Luau/ConstraintGraphBuilder.h" // FIXME move Scope2 into its own header +#include "Luau/TxnLog.h" +#include "Luau/TypeUtils.h" #include "Luau/Unifier.h" #include "Luau/ToString.h" @@ -19,10 +21,12 @@ struct TypeChecker2 : public AstVisitor const SourceModule* sourceModule; Module* module; InternalErrorReporter ice; // FIXME accept a pointer from Frontend + SingletonTypes& singletonTypes; TypeChecker2(const SourceModule* sourceModule, Module* module) : sourceModule(sourceModule) , module(module) + , singletonTypes(getSingletonTypes()) { } @@ -30,16 +34,30 @@ struct TypeChecker2 : public AstVisitor TypePackId lookupPack(AstExpr* expr) { + // If a type isn't in the type graph, it probably means that a recursion limit was exceeded. + // We'll just return anyType in these cases. Typechecking against any is very fast and this + // allows us not to think about this very much in the actual typechecking logic. TypePackId* tp = module->astTypePacks.find(expr); - LUAU_ASSERT(tp); - return follow(*tp); + if (tp) + return follow(*tp); + else + return singletonTypes.anyTypePack; } TypeId lookupType(AstExpr* expr) { + // If a type isn't in the type graph, it probably means that a recursion limit was exceeded. + // We'll just return anyType in these cases. Typechecking against any is very fast and this + // allows us not to think about this very much in the actual typechecking logic. TypeId* ty = module->astTypes.find(expr); - LUAU_ASSERT(ty); - return follow(*ty); + if (ty) + return follow(*ty); + + TypePackId* tp = module->astTypePacks.find(expr); + if (tp) + return flattenPack(*tp); + + return singletonTypes.anyType; } TypeId lookupAnnotation(AstType* annotation) @@ -78,7 +96,7 @@ struct TypeChecker2 : public AstVisitor bestLocation = scopeBounds; } } - else + else if (scopeBounds.begin > location.end) { // TODO: Is this sound? This relies on the fact that scopes are inserted // into the scope list in the order that they appear in the AST. @@ -147,16 +165,14 @@ struct TypeChecker2 : public AstVisitor for (size_t i = 0; i < count; ++i) { AstExpr* lhs = assign->vars.data[i]; - TypeId* lhsType = module->astTypes.find(lhs); - LUAU_ASSERT(lhsType); + TypeId lhsType = lookupType(lhs); AstExpr* rhs = assign->values.data[i]; - TypeId* rhsType = module->astTypes.find(rhs); - LUAU_ASSERT(rhsType); + TypeId rhsType = lookupType(rhs); - if (!isSubtype(*rhsType, *lhsType, ice)) + if (!isSubtype(rhsType, lhsType, ice)) { - reportError(TypeMismatch{*lhsType, *rhsType}, rhs->location); + reportError(TypeMismatch{lhsType, rhsType}, rhs->location); } } @@ -181,7 +197,7 @@ struct TypeChecker2 : public AstVisitor if (!ok) { for (const TypeError& e : u.errors) - module->errors.push_back(e); + reportError(e); } return true; @@ -189,10 +205,14 @@ struct TypeChecker2 : public AstVisitor bool visit(AstExprCall* call) override { + TypeArena arena; + Instantiation instantiation{TxnLog::empty(), &arena, TypeLevel{}}; + TypePackId expectedRetType = lookupPack(call); TypeId functionType = lookupType(call->func); + TypeId instantiatedFunctionType = instantiation.substitute(functionType).value_or(nullptr); + LUAU_ASSERT(functionType); - TypeArena arena; TypePack args; for (const auto& arg : call->args) { @@ -204,7 +224,7 @@ struct TypeChecker2 : public AstVisitor TypePackId argsTp = arena.addTypePack(args); FunctionTypeVar ftv{argsTp, expectedRetType}; TypeId expectedType = arena.addType(ftv); - if (!isSubtype(expectedType, functionType, ice)) + if (!isSubtype(expectedType, instantiatedFunctionType, ice)) { unfreeze(module->interfaceTypes); CloneState cloneState; @@ -252,16 +272,12 @@ struct TypeChecker2 : public AstVisitor // leftType must have a property called indexName->index - if (auto ttv = get(leftType)) + std::optional t = findTablePropertyRespectingMeta(module->errors, leftType, indexName->index.value, indexName->location); + if (t) { - auto it = ttv->props.find(indexName->index.value); - if (it == ttv->props.end()) + if (!isSubtype(resultType, *t, ice)) { - reportError(UnknownProperty{leftType, indexName->index.value}, indexName->location); - } - else if (!isSubtype(resultType, it->second.type, ice)) - { - reportError(TypeMismatch{resultType, it->second.type}, indexName->location); + reportError(TypeMismatch{resultType, *t}, indexName->location); } } else @@ -277,7 +293,7 @@ struct TypeChecker2 : public AstVisitor TypeId actualType = lookupType(number); TypeId numberType = getSingletonTypes().numberType; - if (!isSubtype(actualType, numberType, ice)) + if (!isSubtype(numberType, actualType, ice)) { reportError(TypeMismatch{actualType, numberType}, number->location); } @@ -290,7 +306,7 @@ struct TypeChecker2 : public AstVisitor TypeId actualType = lookupType(string); TypeId stringType = getSingletonTypes().stringType; - if (!isSubtype(actualType, stringType, ice)) + if (!isSubtype(stringType, actualType, ice)) { reportError(TypeMismatch{actualType, stringType}, string->location); } @@ -298,6 +314,41 @@ struct TypeChecker2 : public AstVisitor return true; } + /** Extract a TypeId for the first type of the provided pack. + * + * Note that this may require modifying some types. I hope this doesn't cause problems! + */ + TypeId flattenPack(TypePackId pack) + { + pack = follow(pack); + + while (auto tp = get(pack)) + { + if (tp->head.empty() && tp->tail) + pack = *tp->tail; + } + + if (auto ty = first(pack)) + return *ty; + else if (auto vtp = get(pack)) + return vtp->ty; + else if (auto ftp = get(pack)) + { + TypeId result = module->internalTypes.addType(FreeTypeVar{ftp->scope}); + TypePackId freeTail = module->internalTypes.addTypePack(FreeTypePack{ftp->scope}); + + TypePack& resultPack = asMutable(pack)->ty.emplace(); + resultPack.head.assign(1, result); + resultPack.tail = freeTail; + + return result; + } + else if (get(pack)) + return singletonTypes.errorRecoveryType(); + else + ice.ice("flattenPack got a weird pack!"); + } + bool visit(AstType* ty) override { return true; @@ -321,6 +372,11 @@ struct TypeChecker2 : public AstVisitor { module->errors.emplace_back(location, sourceModule->name, std::move(data)); } + + void reportError(TypeError e) + { + module->errors.emplace_back(std::move(e)); + } }; void check(const SourceModule& sourceModule, Module* module) diff --git a/Analysis/src/TypeInfer.cpp b/Analysis/src/TypeInfer.cpp index 44635e88..d9486a4f 100644 --- a/Analysis/src/TypeInfer.cpp +++ b/Analysis/src/TypeInfer.cpp @@ -38,11 +38,13 @@ LUAU_FASTFLAGVARIABLE(LuauReduceUnionRecursion, false) LUAU_FASTFLAGVARIABLE(LuauReturnAnyInsteadOfICE, false) // Eventually removed as false. LUAU_FASTFLAG(LuauNormalizeFlagIsConservative) LUAU_FASTFLAGVARIABLE(LuauReturnTypeInferenceInNonstrict, false) +LUAU_FASTFLAGVARIABLE(DebugLuauSharedSelf, false); LUAU_FASTFLAGVARIABLE(LuauAlwaysQuantify, false); LUAU_FASTFLAGVARIABLE(LuauReportErrorsOnIndexerKeyMismatch, false) LUAU_FASTFLAG(LuauQuantifyConstrained) LUAU_FASTFLAGVARIABLE(LuauFalsyPredicateReturnsNilInstead, false) LUAU_FASTFLAGVARIABLE(LuauNonCopyableTypeVarFields, false) +LUAU_FASTFLAGVARIABLE(LuauCheckLenMT, false) namespace Luau { @@ -238,7 +240,7 @@ static bool isMetamethod(const Name& name) { return name == "__index" || name == "__newindex" || name == "__call" || name == "__concat" || name == "__unm" || name == "__add" || name == "__sub" || name == "__mul" || name == "__div" || name == "__mod" || name == "__pow" || name == "__tostring" || - name == "__metatable" || name == "__eq" || name == "__lt" || name == "__le" || name == "__mode"; + name == "__metatable" || name == "__eq" || name == "__lt" || name == "__le" || name == "__mode" || name == "__iter" || name == "__len"; } size_t HashBoolNamePair::operator()(const std::pair& pair) const @@ -327,10 +329,19 @@ ModulePtr TypeChecker::checkWithoutRecursionCheck(const SourceModule& module, Mo currentModule->timeout = true; } + if (FFlag::DebugLuauSharedSelf) + { + for (auto& [ty, scope] : deferredQuantification) + Luau::quantify(ty, scope->level); + deferredQuantification.clear(); + } + if (get(follow(moduleScope->returnType))) moduleScope->returnType = addTypePack(TypePack{{}, std::nullopt}); else + { moduleScope->returnType = anyify(moduleScope, moduleScope->returnType, Location{}); + } for (auto& [_, typeFun] : moduleScope->exportedTypeBindings) typeFun.type = anyify(moduleScope, typeFun.type, Location{}); @@ -537,18 +548,43 @@ void TypeChecker::checkBlockWithoutRecursionCheck(const ScopePtr& scope, const A } else if (auto fun = (*protoIter)->as()) { + std::optional selfType; std::optional expectedType; - if (!fun->func->self) + if (FFlag::DebugLuauSharedSelf) { if (auto name = fun->name->as()) { - TypeId exprTy = checkExpr(scope, *name->expr).type; - expectedType = getIndexTypeFromType(scope, exprTy, name->index.value, name->indexLocation, false); + TypeId baseTy = checkExpr(scope, *name->expr).type; + tablify(baseTy); + + if (!fun->func->self) + expectedType = getIndexTypeFromType(scope, baseTy, name->index.value, name->indexLocation, false); + else if (auto ttv = getMutableTableType(baseTy)) + { + if (!baseTy->persistent && ttv->state != TableState::Sealed && !ttv->selfTy) + { + ttv->selfTy = anyIfNonstrict(freshType(ttv->level)); + deferredQuantification.push_back({baseTy, scope}); + } + + selfType = ttv->selfTy; + } + } + } + else + { + if (!fun->func->self) + { + if (auto name = fun->name->as()) + { + TypeId exprTy = checkExpr(scope, *name->expr).type; + expectedType = getIndexTypeFromType(scope, exprTy, name->index.value, name->indexLocation, false); + } } } - auto pair = checkFunctionSignature(scope, subLevel, *fun->func, fun->name->location, expectedType); + auto pair = checkFunctionSignature(scope, subLevel, *fun->func, fun->name->location, selfType, expectedType); auto [funTy, funScope] = pair; functionDecls[*protoIter] = pair; @@ -560,7 +596,7 @@ void TypeChecker::checkBlockWithoutRecursionCheck(const ScopePtr& scope, const A } else if (auto fun = (*protoIter)->as()) { - auto pair = checkFunctionSignature(scope, subLevel, *fun->func, fun->name->location, std::nullopt); + auto pair = checkFunctionSignature(scope, subLevel, *fun->func, fun->name->location, std::nullopt, std::nullopt); auto [funTy, funScope] = pair; functionDecls[*protoIter] = pair; @@ -2076,7 +2112,7 @@ WithPredicate TypeChecker::checkExpr(const ScopePtr& scope, const AstExp WithPredicate TypeChecker::checkExpr(const ScopePtr& scope, const AstExprFunction& expr, std::optional expectedType) { - auto [funTy, funScope] = checkFunctionSignature(scope, 0, expr, std::nullopt, expectedType); + auto [funTy, funScope] = checkFunctionSignature(scope, 0, expr, std::nullopt, std::nullopt, expectedType); checkFunctionBody(funScope, funTy, expr); @@ -2296,6 +2332,8 @@ WithPredicate TypeChecker::checkExpr(const ScopePtr& scope, const AstExp state.tryUnify(actualFunctionType, expectedFunctionType, /*isFunctionCall*/ true); state.log.commit(); + reportErrors(state.errors); + TypeId retType = first(retTypePack).value_or(nilType); if (!state.errors.empty()) retType = errorRecoveryType(retType); @@ -2322,6 +2360,23 @@ WithPredicate TypeChecker::checkExpr(const ScopePtr& scope, const AstExp DenseHashSet seen{nullptr}; + if (FFlag::LuauCheckLenMT && typeCouldHaveMetatable(operandType)) + { + if (auto fnt = findMetatableEntry(operandType, "__len", expr.location)) + { + TypeId actualFunctionType = instantiate(scope, *fnt, expr.location); + TypePackId arguments = addTypePack({operandType}); + TypePackId retTypePack = addTypePack({numberType}); + TypeId expectedFunctionType = addType(FunctionTypeVar(scope->level, arguments, retTypePack)); + + Unifier state = mkUnifier(expr.location); + state.tryUnify(actualFunctionType, expectedFunctionType, /*isFunctionCall*/ true); + state.log.commit(); + + reportErrors(state.errors); + } + } + if (!hasLength(operandType, seen, &recursionCount)) reportError(TypeError{expr.location, NotATable{operandType}}); @@ -2530,17 +2585,15 @@ TypeId TypeChecker::checkRelationalOperation( } } - if (!matches) { reportError( expr.location, GenericError{format("Types %s and %s cannot be compared with %s because they do not have the same metatable", - toString(lhsType).c_str(), toString(rhsType).c_str(), toString(expr.op).c_str())}); + toString(lhsType).c_str(), toString(rhsType).c_str(), toString(expr.op).c_str())}); return errorRecoveryType(booleanType); } } - if (leftMetatable) { std::optional metamethod = findMetatableEntry(lhsType, metamethodName, expr.location); @@ -3139,8 +3192,8 @@ TypeId TypeChecker::checkFunctionName(const ScopePtr& scope, AstExpr& funName, T // `(X) -> Y...`, but after typechecking the body, we cam unify `Y...` with `X` // to get type `(X) -> X`, then we quantify the free types to get the final // generic type `(a) -> a`. -std::pair TypeChecker::checkFunctionSignature( - const ScopePtr& scope, int subLevel, const AstExprFunction& expr, std::optional originalName, std::optional expectedType) +std::pair TypeChecker::checkFunctionSignature(const ScopePtr& scope, int subLevel, const AstExprFunction& expr, + std::optional originalName, std::optional selfType, std::optional expectedType) { ScopePtr funScope = childFunctionScope(scope, expr.location, subLevel); @@ -3241,12 +3294,25 @@ std::pair TypeChecker::checkFunctionSignature( funScope->returnType = retPack; - if (expr.self) + if (FFlag::DebugLuauSharedSelf) { - // TODO: generic self types: CLI-39906 - TypeId selfType = anyIfNonstrict(freshType(funScope)); - funScope->bindings[expr.self] = {selfType, expr.self->location}; - argTypes.push_back(selfType); + if (expr.self) + { + // TODO: generic self types: CLI-39906 + TypeId selfTy = anyIfNonstrict(selfType ? *selfType : freshType(funScope)); + funScope->bindings[expr.self] = {selfTy, expr.self->location}; + argTypes.push_back(selfTy); + } + } + else + { + if (expr.self) + { + // TODO: generic self types: CLI-39906 + TypeId selfType = anyIfNonstrict(freshType(funScope)); + funScope->bindings[expr.self] = {selfType, expr.self->location}; + argTypes.push_back(selfType); + } } // Prepare expected argument type iterators if we have an expected function type @@ -4457,25 +4523,43 @@ TypeId TypeChecker::quantify(const ScopePtr& scope, TypeId ty, Location location { ty = follow(ty); - const FunctionTypeVar* ftv = get(ty); - - if (FFlag::LuauAlwaysQuantify) + if (FFlag::DebugLuauSharedSelf) { - if (ftv) + if (auto ftv = get(ty)) Luau::quantify(ty, scope->level); + else if (auto ttv = getTableType(ty); ttv && ttv->selfTy) + Luau::quantify(ty, scope->level); + + if (FFlag::LuauLowerBoundsCalculation) + { + auto [t, ok] = Luau::normalize(ty, currentModule, *iceHandler); + if (!ok) + reportError(location, NormalizationTooComplex{}); + return t; + } } else { - if (ftv && ftv->generics.empty() && ftv->genericPacks.empty()) - Luau::quantify(ty, scope->level); - } + const FunctionTypeVar* ftv = get(ty); - if (FFlag::LuauLowerBoundsCalculation && ftv) - { - auto [t, ok] = Luau::normalize(ty, currentModule, *iceHandler); - if (!ok) - reportError(location, NormalizationTooComplex{}); - return t; + if (FFlag::LuauAlwaysQuantify) + { + if (ftv) + Luau::quantify(ty, scope->level); + } + else + { + if (ftv && ftv->generics.empty() && ftv->genericPacks.empty()) + Luau::quantify(ty, scope->level); + } + + if (FFlag::LuauLowerBoundsCalculation && ftv) + { + auto [t, ok] = Luau::normalize(ty, currentModule, *iceHandler); + if (!ok) + reportError(location, NormalizationTooComplex{}); + return t; + } } return ty; diff --git a/Analysis/src/Unifier.cpp b/Analysis/src/Unifier.cpp index 6147e118..0792a350 100644 --- a/Analysis/src/Unifier.cpp +++ b/Analysis/src/Unifier.cpp @@ -740,7 +740,7 @@ void Unifier::tryUnifyTypeWithIntersection(TypeId subTy, TypeId superTy, const I std::optional unificationTooComplex; std::optional firstFailedOption; - // T <: A & B if A <: T and B <: T + // T <: A & B if T <: A and T <: B for (TypeId type : uv->parts) { Unifier innerState = makeChildUnifier(); @@ -765,7 +765,7 @@ void Unifier::tryUnifyTypeWithIntersection(TypeId subTy, TypeId superTy, const I void Unifier::tryUnifyIntersectionWithType(TypeId subTy, const IntersectionTypeVar* uv, TypeId superTy, bool cacheEnabled, bool isFunctionCall) { - // A & B <: T if T <: A or T <: B + // A & B <: T if A <: T or B <: T bool found = false; std::optional unificationTooComplex; diff --git a/Ast/src/Parser.cpp b/Ast/src/Parser.cpp index 95bce3ee..70c92555 100644 --- a/Ast/src/Parser.cpp +++ b/Ast/src/Parser.cpp @@ -5,6 +5,9 @@ #include +#include +#include + // Warning: If you are introducing new syntax, ensure that it is behind a separate // flag so that we don't break production games by reverting syntax changes. // See docs/SyntaxChanges.md for an explanation. @@ -14,6 +17,18 @@ LUAU_FASTINTVARIABLE(LuauParseErrorLimit, 100) LUAU_FASTFLAGVARIABLE(LuauParserFunctionKeywordAsTypeHelp, false) LUAU_FASTFLAGVARIABLE(LuauReturnTypeTokenConfusion, false) +LUAU_FASTFLAGVARIABLE(LuauFixNamedFunctionParse, false) +LUAU_DYNAMIC_FASTFLAGVARIABLE(LuaReportParseWrongNamedType, false) + +bool lua_telemetry_parsed_named_non_function_type = false; + +LUAU_FASTFLAGVARIABLE(LuauErrorParseIntegerIssues, false) +LUAU_DYNAMIC_FASTFLAGVARIABLE(LuaReportParseIntegerIssues, false) + +bool lua_telemetry_parsed_out_of_range_bin_integer = false; +bool lua_telemetry_parsed_out_of_range_hex_integer = false; +bool lua_telemetry_parsed_double_prefix_hex_integer = false; + namespace Luau { @@ -1330,7 +1345,7 @@ AstTypeOrPack Parser::parseFunctionTypeAnnotation(bool allowPack) { incrementRecursionCounter("type annotation"); - bool monomorphic = lexer.current().type != '<'; + bool forceFunctionType = lexer.current().type == '<'; Lexeme begin = lexer.current(); @@ -1355,21 +1370,33 @@ AstTypeOrPack Parser::parseFunctionTypeAnnotation(bool allowPack) AstArray paramTypes = copy(params); + if (FFlag::LuauFixNamedFunctionParse && !names.empty()) + forceFunctionType = true; + bool returnTypeIntroducer = FFlag::LuauReturnTypeTokenConfusion ? lexer.current().type == Lexeme::SkinnyArrow || lexer.current().type == ':' : false; // Not a function at all. Just a parenthesized type. Or maybe a type pack with a single element - if (params.size() == 1 && !varargAnnotation && monomorphic && + if (params.size() == 1 && !varargAnnotation && !forceFunctionType && (FFlag::LuauReturnTypeTokenConfusion ? !returnTypeIntroducer : lexer.current().type != Lexeme::SkinnyArrow)) { + if (DFFlag::LuaReportParseWrongNamedType && !names.empty()) + lua_telemetry_parsed_named_non_function_type = true; + if (allowPack) return {{}, allocator.alloc(begin.location, AstTypeList{paramTypes, nullptr})}; else return {params[0], {}}; } - if ((FFlag::LuauReturnTypeTokenConfusion ? !returnTypeIntroducer : lexer.current().type != Lexeme::SkinnyArrow) && monomorphic && allowPack) + if ((FFlag::LuauReturnTypeTokenConfusion ? !returnTypeIntroducer : lexer.current().type != Lexeme::SkinnyArrow) && !forceFunctionType && + allowPack) + { + if (DFFlag::LuaReportParseWrongNamedType && !names.empty()) + lua_telemetry_parsed_named_non_function_type = true; + return {{}, allocator.alloc(begin.location, AstTypeList{paramTypes, varargAnnotation})}; + } AstArray> paramNames = copy(names); @@ -2010,7 +2037,63 @@ AstExpr* Parser::parseAssertionExpr() return expr; } -static bool parseNumber(double& result, const char* data) +static const char* parseInteger(double& result, const char* data, int base) +{ + char* end = nullptr; + unsigned long long value = strtoull(data, &end, base); + + if (value == ULLONG_MAX && errno == ERANGE) + { + // 'errno' might have been set before we called 'strtoull', but we don't want the overhead of resetting a TLS variable on each call + // so we only reset it when we get a result that might be an out-of-range error and parse again to make sure + errno = 0; + value = strtoull(data, &end, base); + + if (errno == ERANGE) + { + if (DFFlag::LuaReportParseIntegerIssues) + { + if (base == 2) + lua_telemetry_parsed_out_of_range_bin_integer = true; + else + lua_telemetry_parsed_out_of_range_hex_integer = true; + } + + if (FFlag::LuauErrorParseIntegerIssues) + return "Integer number value is out of range"; + } + } + + result = double(value); + return *end == 0 ? nullptr : "Malformed number"; +} + +static const char* parseNumber(double& result, const char* data) +{ + // binary literal + if (data[0] == '0' && (data[1] == 'b' || data[1] == 'B') && data[2]) + return parseInteger(result, data + 2, 2); + + // hexadecimal literal + if (data[0] == '0' && (data[1] == 'x' || data[1] == 'X') && data[2]) + { + if (DFFlag::LuaReportParseIntegerIssues && data[2] == '0' && (data[3] == 'x' || data[3] == 'X')) + lua_telemetry_parsed_double_prefix_hex_integer = true; + + if (FFlag::LuauErrorParseIntegerIssues) + return parseInteger(result, data, 16); // keep prefix, it's handled by 'strtoull' + else + return parseInteger(result, data + 2, 16); + } + + char* end = nullptr; + double value = strtod(data, &end); + + result = value; + return *end == 0 ? nullptr : "Malformed number"; +} + +static bool parseNumber_DEPRECATED(double& result, const char* data) { // binary literal if (data[0] == '0' && (data[1] == 'b' || data[1] == 'B') && data[2]) @@ -2080,18 +2163,37 @@ AstExpr* Parser::parseSimpleExpr() scratchData.erase(std::remove(scratchData.begin(), scratchData.end(), '_'), scratchData.end()); } - double value = 0; - if (parseNumber(value, scratchData.c_str())) + if (DFFlag::LuaReportParseIntegerIssues || FFlag::LuauErrorParseIntegerIssues) { - nextLexeme(); + double value = 0; + if (const char* error = parseNumber(value, scratchData.c_str())) + { + nextLexeme(); - return allocator.alloc(start, value); + return reportExprError(start, {}, "%s", error); + } + else + { + nextLexeme(); + + return allocator.alloc(start, value); + } } else { - nextLexeme(); + double value = 0; + if (parseNumber_DEPRECATED(value, scratchData.c_str())) + { + nextLexeme(); - return reportExprError(start, {}, "Malformed number"); + return allocator.alloc(start, value); + } + else + { + nextLexeme(); + + return reportExprError(start, {}, "Malformed number"); + } } } else if (lexer.current().type == Lexeme::RawString || lexer.current().type == Lexeme::QuotedString) diff --git a/Common/include/Luau/Bytecode.h b/Common/include/Luau/Bytecode.h index 218bb5d5..0cb7e1d9 100644 --- a/Common/include/Luau/Bytecode.h +++ b/Common/include/Luau/Bytecode.h @@ -276,7 +276,7 @@ enum LuauOpcode // FORGLOOP: adjust loop variables for one iteration of a generic for loop, jump back to the loop header if loop needs to continue // A: target register; generic for loops assume a register layout [generator, state, index, variables...] // D: jump offset (-32768..32767) - // AUX: variable count (1..255) + // AUX: variable count (1..255) in the low 8 bits, high bit indicates whether to use ipairs-style traversal in the fast path // loop variables are adjusted by calling generator(state, index) and expecting it to return a tuple that's copied to the user variables // the first variable is then copied into index; generator/state are immutable, index isn't visible to user code LOP_FORGLOOP, @@ -490,6 +490,9 @@ enum LuauBuiltinFunction // select(_, ...) LBF_SELECT_VARARG, + + // rawlen + LBF_RAWLEN, }; // Capture type, used in LOP_CAPTURE diff --git a/Compiler/src/Builtins.cpp b/Compiler/src/Builtins.cpp index ff753112..6bd24b6d 100644 --- a/Compiler/src/Builtins.cpp +++ b/Compiler/src/Builtins.cpp @@ -4,6 +4,8 @@ #include "Luau/Bytecode.h" #include "Luau/Compiler.h" +LUAU_FASTFLAGVARIABLE(LuauCompileRawlen, false) + namespace Luau { namespace Compile @@ -58,6 +60,8 @@ int getBuiltinFunctionId(const Builtin& builtin, const CompileOptions& options) return LBF_RAWGET; if (builtin.isGlobal("rawequal")) return LBF_RAWEQUAL; + if (FFlag::LuauCompileRawlen && builtin.isGlobal("rawlen")) + return LBF_RAWLEN; if (builtin.isGlobal("unpack")) return LBF_TABLE_UNPACK; diff --git a/Compiler/src/BytecodeBuilder.cpp b/Compiler/src/BytecodeBuilder.cpp index 301cf255..5e2669b2 100644 --- a/Compiler/src/BytecodeBuilder.cpp +++ b/Compiler/src/BytecodeBuilder.cpp @@ -1302,20 +1302,22 @@ void BytecodeBuilder::validate() const case LOP_FORNPREP: case LOP_FORNLOOP: - VREG(LUAU_INSN_A(insn) + 2); // for loop protocol: A, A+1, A+2 are used for iteration + // for loop protocol: A, A+1, A+2 are used for iteration + VREG(LUAU_INSN_A(insn) + 2); VJUMP(LUAU_INSN_D(insn)); break; case LOP_FORGPREP: - VREG(LUAU_INSN_A(insn) + 2 + 1); // forg loop protocol: A, A+1, A+2 are used for iteration protocol; A+3, ... are loop variables + // forg loop protocol: A, A+1, A+2 are used for iteration protocol; A+3, ... are loop variables + VREG(LUAU_INSN_A(insn) + 2 + 1); VJUMP(LUAU_INSN_D(insn)); break; case LOP_FORGLOOP: - VREG( - LUAU_INSN_A(insn) + 2 + insns[i + 1]); // forg loop protocol: A, A+1, A+2 are used for iteration protocol; A+3, ... are loop variables + // forg loop protocol: A, A+1, A+2 are used for iteration protocol; A+3, ... are loop variables + VREG(LUAU_INSN_A(insn) + 2 + uint8_t(insns[i + 1])); VJUMP(LUAU_INSN_D(insn)); - LUAU_ASSERT(insns[i + 1] >= 1); + LUAU_ASSERT(uint8_t(insns[i + 1]) >= 1); break; case LOP_FORGPREP_INEXT: @@ -1679,7 +1681,8 @@ void BytecodeBuilder::dumpInstruction(const uint32_t* code, std::string& result, break; case LOP_FORGLOOP: - formatAppend(result, "FORGLOOP R%d L%d %d\n", LUAU_INSN_A(insn), targetLabel, *code++); + formatAppend(result, "FORGLOOP R%d L%d %d%s\n", LUAU_INSN_A(insn), targetLabel, uint8_t(*code), int(*code) < 0 ? " [inext]" : ""); + code++; break; case LOP_FORGPREP_INEXT: diff --git a/Compiler/src/Compiler.cpp b/Compiler/src/Compiler.cpp index e732256b..d7c8155c 100644 --- a/Compiler/src/Compiler.cpp +++ b/Compiler/src/Compiler.cpp @@ -23,6 +23,8 @@ LUAU_FASTINTVARIABLE(LuauCompileInlineThreshold, 25) LUAU_FASTINTVARIABLE(LuauCompileInlineThresholdMaxBoost, 300) LUAU_FASTINTVARIABLE(LuauCompileInlineDepth, 5) +LUAU_FASTFLAGVARIABLE(LuauCompileNoIpairs, false) + namespace Luau { @@ -2665,7 +2667,7 @@ struct Compiler if (builtin.isGlobal("ipairs")) // for .. in ipairs(t) { skipOp = LOP_FORGPREP_INEXT; - loopOp = LOP_FORGLOOP_INEXT; + loopOp = FFlag::LuauCompileNoIpairs ? LOP_FORGLOOP : LOP_FORGLOOP_INEXT; } else if (builtin.isGlobal("pairs")) // for .. in pairs(t) { @@ -2709,8 +2711,16 @@ struct Compiler bytecode.emitAD(loopOp, regs, 0); + if (FFlag::LuauCompileNoIpairs) + { + // TODO: remove loopOp as it's a constant now + LUAU_ASSERT(loopOp == LOP_FORGLOOP); + + // FORGLOOP uses aux to encode variable count and fast path flag for ipairs traversal in the high bit + bytecode.emitAux((skipOp == LOP_FORGPREP_INEXT ? 0x80000000 : 0) | uint32_t(stat->vars.size)); + } // note: FORGLOOP needs variable count encoded in AUX field, other loop instructions assume a fixed variable count - if (loopOp == LOP_FORGLOOP) + else if (loopOp == LOP_FORGLOOP) bytecode.emitAux(uint32_t(stat->vars.size)); size_t endLabel = bytecode.emitLabel(); @@ -3341,7 +3351,7 @@ struct Compiler std::vector upvals; }; - struct ReturnVisitor: AstVisitor + struct ReturnVisitor : AstVisitor { Compiler* self; bool returnsOne = true; diff --git a/VM/src/lbaselib.cpp b/VM/src/lbaselib.cpp index f7917611..4fc5033e 100644 --- a/VM/src/lbaselib.cpp +++ b/VM/src/lbaselib.cpp @@ -11,6 +11,8 @@ #include #include +LUAU_FASTFLAG(LuauLenTM) + static void writestring(const char* s, size_t l) { fwrite(s, 1, l, stdout); @@ -178,6 +180,18 @@ static int luaB_rawset(lua_State* L) return 1; } +static int luaB_rawlen(lua_State* L) +{ + if (!FFlag::LuauLenTM) + luaL_error(L, "'rawlen' is not available"); + + int tt = lua_type(L, 1); + luaL_argcheck(L, tt == LUA_TTABLE || tt == LUA_TSTRING, 1, "table or string expected"); + int len = lua_objlen(L, 1); + lua_pushinteger(L, len); + return 1; +} + static int luaB_gcinfo(lua_State* L) { lua_pushinteger(L, lua_gc(L, LUA_GCCOUNT, 0)); @@ -428,6 +442,7 @@ static const luaL_Reg base_funcs[] = { {"rawequal", luaB_rawequal}, {"rawget", luaB_rawget}, {"rawset", luaB_rawset}, + {"rawlen", luaB_rawlen}, {"select", luaB_select}, {"setfenv", luaB_setfenv}, {"setmetatable", luaB_setmetatable}, diff --git a/VM/src/lbuiltins.cpp b/VM/src/lbuiltins.cpp index deaf1407..e98660a7 100644 --- a/VM/src/lbuiltins.cpp +++ b/VM/src/lbuiltins.cpp @@ -1117,6 +1117,27 @@ static int luauF_select(lua_State* L, StkId res, TValue* arg0, int nresults, Stk return -1; } +static int luauF_rawlen(lua_State* L, StkId res, TValue* arg0, int nresults, StkId args, int nparams) +{ + if (nparams >= 1 && nresults <= 1) + { + if (ttistable(arg0)) + { + Table* h = hvalue(arg0); + setnvalue(res, double(luaH_getn(h))); + return 1; + } + else if (ttisstring(arg0)) + { + TString* ts = tsvalue(arg0); + setnvalue(res, double(ts->len)); + return 1; + } + } + + return -1; +} + luau_FastFunction luauF_table[256] = { NULL, luauF_assert, @@ -1188,4 +1209,6 @@ luau_FastFunction luauF_table[256] = { luauF_countrz, luauF_select, + + luauF_rawlen, }; diff --git a/VM/src/ldebug.h b/VM/src/ldebug.h index cf905e9f..75bb8dcc 100644 --- a/VM/src/ldebug.h +++ b/VM/src/ldebug.h @@ -19,6 +19,7 @@ LUAI_FUNC l_noret luaG_concaterror(lua_State* L, StkId p1, StkId p2); LUAI_FUNC l_noret luaG_aritherror(lua_State* L, const TValue* p1, const TValue* p2, TMS op); LUAI_FUNC l_noret luaG_ordererror(lua_State* L, const TValue* p1, const TValue* p2, TMS op); LUAI_FUNC l_noret luaG_indexerror(lua_State* L, const TValue* p1, const TValue* p2); + LUAI_FUNC LUA_PRINTF_ATTR(2, 3) l_noret luaG_runerrorL(lua_State* L, const char* fmt, ...); LUAI_FUNC void luaG_pusherror(lua_State* L, const char* error); diff --git a/VM/src/ltm.cpp b/VM/src/ltm.cpp index e7df4e53..49982b28 100644 --- a/VM/src/ltm.cpp +++ b/VM/src/ltm.cpp @@ -39,6 +39,7 @@ const char* const luaT_eventname[] = { "__namecall", "__call", "__iter", + "__len", "__eq", @@ -52,7 +53,6 @@ const char* const luaT_eventname[] = { "__unm", - "__len", "__lt", "__le", "__concat", diff --git a/VM/src/ltm.h b/VM/src/ltm.h index a5223941..e11ddb3a 100644 --- a/VM/src/ltm.h +++ b/VM/src/ltm.h @@ -18,6 +18,7 @@ typedef enum TM_NAMECALL, TM_CALL, TM_ITER, + TM_LEN, TM_EQ, /* last tag method with `fast' access */ @@ -31,7 +32,6 @@ typedef enum TM_UNM, - TM_LEN, TM_LT, TM_LE, TM_CONCAT, diff --git a/VM/src/lvmexecute.cpp b/VM/src/lvmexecute.cpp index e0a96474..85829ca1 100644 --- a/VM/src/lvmexecute.cpp +++ b/VM/src/lvmexecute.cpp @@ -16,6 +16,8 @@ #include +LUAU_FASTFLAGVARIABLE(LuauLenTM, false) + // Disable c99-designator to avoid the warning in CGOTO dispatch table #ifdef __clang__ #if __has_warning("-Wc99-designator") @@ -2082,13 +2084,25 @@ static void luau_execute(lua_State* L) // fast-path #1: tables if (ttistable(rb)) { - setnvalue(ra, cast_num(luaH_getn(hvalue(rb)))); - VM_NEXT(); + Table* h = hvalue(rb); + + if (!FFlag::LuauLenTM || fastnotm(h->metatable, TM_LEN)) + { + setnvalue(ra, cast_num(luaH_getn(h))); + VM_NEXT(); + } + else + { + // slow-path, may invoke C/Lua via metamethods + VM_PROTECT(luaV_dolen(L, ra, rb)); + VM_NEXT(); + } } // fast-path #2: strings (not very important but easy to do) else if (ttisstring(rb)) { - setnvalue(ra, cast_num(tsvalue(rb)->len)); + TString* ts = tsvalue(rb); + setnvalue(ra, cast_num(ts->len)); VM_NEXT(); } else @@ -2226,6 +2240,15 @@ static void luau_execute(lua_State* L) VM_PROTECT(luaD_call(L, ra, 3)); L->top = L->ci->top; + + /* recompute ra since stack might have been reallocated */ + ra = VM_REG(LUAU_INSN_A(insn)); + + /* protect against __iter returning nil, since nil is used as a marker for builtin iteration in FORGLOOP */ + if (ttisnil(ra)) + { + VM_PROTECT(luaG_typeerror(L, ra, "call")); + } } else if (fasttm(L, mt, TM_CALL)) { @@ -2258,27 +2281,38 @@ static void luau_execute(lua_State* L) uint32_t aux = *pc; // fast-path: builtin table iteration - if (ttisnil(ra) && ttistable(ra + 1) && ttislightuserdata(ra + 2)) + // note: ra=nil guarantees ra+1=table and ra+2=userdata because of the setup by FORGPREP* opcodes + // TODO: remove the table check per guarantee above + if (ttisnil(ra) && ttistable(ra + 1)) { Table* h = hvalue(ra + 1); int index = int(reinterpret_cast(pvalue(ra + 2))); int sizearray = h->sizearray; - int sizenode = 1 << h->lsizenode; // clear extra variables since we might have more than two - if (LUAU_UNLIKELY(aux > 2)) + // note: while aux encodes ipairs bit, when set we always use 2 variables, so it's safe to check this via a signed comparison + if (LUAU_UNLIKELY(int(aux) > 2)) for (int i = 2; i < int(aux); ++i) setnilvalue(ra + 3 + i); + // terminate ipairs-style traversal early when encountering nil + if (int(aux) < 0 && (unsigned(index) >= unsigned(sizearray) || ttisnil(&h->array[index]))) + { + pc++; + VM_NEXT(); + } + // first we advance index through the array portion while (unsigned(index) < unsigned(sizearray)) { - if (!ttisnil(&h->array[index])) + TValue* e = &h->array[index]; + + if (!ttisnil(e)) { setpvalue(ra + 2, reinterpret_cast(uintptr_t(index + 1))); setnvalue(ra + 3, double(index + 1)); - setobj2s(L, ra + 4, &h->array[index]); + setobj2s(L, ra + 4, e); pc += LUAU_INSN_D(insn); LUAU_ASSERT(unsigned(pc - cl->l.p->code) < unsigned(cl->l.p->sizecode)); @@ -2288,6 +2322,8 @@ static void luau_execute(lua_State* L) index++; } + int sizenode = 1 << h->lsizenode; + // then we advance index through the hash portion while (unsigned(index - sizearray) < unsigned(sizenode)) { @@ -2321,7 +2357,7 @@ static void luau_execute(lua_State* L) L->top = ra + 3 + 3; /* func + 2 args (state and index) */ LUAU_ASSERT(L->top <= L->stack_last); - VM_PROTECT(luaD_call(L, ra + 3, aux)); + VM_PROTECT(luaD_call(L, ra + 3, uint8_t(aux))); L->top = L->ci->top; // recompute ra since stack might have been reallocated diff --git a/VM/src/lvmutils.cpp b/VM/src/lvmutils.cpp index 8a18a4d4..b9e762eb 100644 --- a/VM/src/lvmutils.cpp +++ b/VM/src/lvmutils.cpp @@ -10,6 +10,9 @@ #include "lnumutils.h" #include +#include + +LUAU_FASTFLAG(LuauLenTM) /* limit for table tag-method chains (to avoid loops) */ #define MAXTAGLOOP 100 @@ -51,7 +54,7 @@ const float* luaV_tovector(const TValue* obj) return nullptr; } -static void callTMres(lua_State* L, StkId res, const TValue* f, const TValue* p1, const TValue* p2) +static StkId callTMres(lua_State* L, StkId res, const TValue* f, const TValue* p1, const TValue* p2) { ptrdiff_t result = savestack(L, res); // using stack room beyond top is technically safe here, but for very complicated reasons: @@ -71,6 +74,7 @@ static void callTMres(lua_State* L, StkId res, const TValue* f, const TValue* p1 res = restorestack(L, result); L->top--; setobjs2s(L, res, L->top); + return res; } static void callTM(lua_State* L, const TValue* f, const TValue* p1, const TValue* p2, const TValue* p3) @@ -472,22 +476,56 @@ void luaV_doarith(lua_State* L, StkId ra, const TValue* rb, const TValue* rc, TM void luaV_dolen(lua_State* L, StkId ra, const TValue* rb) { + if (!FFlag::LuauLenTM) + { + switch (ttype(rb)) + { + case LUA_TTABLE: + { + setnvalue(ra, cast_num(luaH_getn(hvalue(rb)))); + break; + } + case LUA_TSTRING: + { + setnvalue(ra, cast_num(tsvalue(rb)->len)); + break; + } + default: + { /* try metamethod */ + if (!call_binTM(L, rb, luaO_nilobject, ra, TM_LEN)) + luaG_typeerror(L, rb, "get length of"); + } + } + return; + } + + const TValue* tm = NULL; switch (ttype(rb)) { case LUA_TTABLE: { - setnvalue(ra, cast_num(luaH_getn(hvalue(rb)))); + Table* h = hvalue(rb); + if ((tm = fasttm(L, h->metatable, TM_LEN)) == NULL) + { + setnvalue(ra, cast_num(luaH_getn(h))); + return; + } break; } case LUA_TSTRING: { - setnvalue(ra, cast_num(tsvalue(rb)->len)); - break; + TString* ts = tsvalue(rb); + setnvalue(ra, cast_num(ts->len)); + return; } default: - { /* try metamethod */ - if (!call_binTM(L, rb, luaO_nilobject, ra, TM_LEN)) - luaG_typeerror(L, rb, "get length of"); - } + tm = luaT_gettmbyobj(L, rb, TM_LEN); } + + if (ttisnil(tm)) + luaG_typeerror(L, rb, "get length of"); + + StkId res = callTMres(L, ra, tm, rb, luaO_nilobject); + if (!ttisnumber(res)) + luaG_runerror(L, "'__len' must return a number"); /* note, we can't access rb since stack may have been reallocated */ } diff --git a/fuzz/protoprint.cpp b/fuzz/protoprint.cpp index 66a89f24..d4d52276 100644 --- a/fuzz/protoprint.cpp +++ b/fuzz/protoprint.cpp @@ -11,6 +11,7 @@ static const std::string kNames[] = { "__div", "__eq", "__index", + "__iter", "__le", "__len", "__lt", @@ -41,13 +42,18 @@ static const std::string kNames[] = { "ceil", "char", "charpattern", + "clamp", "clock", + "clone", + "close", "codepoint", "codes", "concat", "coroutine", "cos", "cosh", + "countlz", + "countrz", "create", "date", "debug", @@ -63,6 +69,7 @@ static const std::string kNames[] = { "foreachi", "format", "frexp", + "freeze", "function", "gcinfo", "getfenv", @@ -72,8 +79,10 @@ static const std::string kNames[] = { "gmatch", "gsub", "huge", + "info", "insert", "ipairs", + "isfrozen", "isyieldable", "ldexp", "len", @@ -93,6 +102,7 @@ static const std::string kNames[] = { "newproxy", "next", "nil", + "noise", "number", "offset", "os", @@ -121,6 +131,7 @@ static const std::string kNames[] = { "select", "setfenv", "setmetatable", + "sign", "sin", "sinh", "sort", diff --git a/tests/Compiler.test.cpp b/tests/Compiler.test.cpp index 655e48cb..fafafd71 100644 --- a/tests/Compiler.test.cpp +++ b/tests/Compiler.test.cpp @@ -261,6 +261,8 @@ L1: RETURN R0 0 TEST_CASE("ForBytecode") { + ScopedFastFlag sff("LuauCompileNoIpairs", true); + // basic for loop: variable directly refers to internal iteration index (R2) CHECK_EQ("\n" + compileFunction0("for i=1,5 do print(i) end"), R"( LOADN R2 1 @@ -313,7 +315,7 @@ L0: GETIMPORT R5 3 MOVE R6 R3 MOVE R7 R4 CALL R5 2 0 -L1: FORGLOOP_INEXT R0 L0 +L1: FORGLOOP R0 L0 2 [inext] RETURN R0 0 )"); @@ -347,13 +349,15 @@ RETURN R0 0 TEST_CASE("ForBytecodeBuiltin") { + ScopedFastFlag sff("LuauCompileNoIpairs", true); + // we generally recognize builtins like pairs/ipairs and emit special opcodes CHECK_EQ("\n" + compileFunction0("for k,v in ipairs({}) do end"), R"( GETIMPORT R0 1 NEWTABLE R1 0 0 CALL R0 1 3 FORGPREP_INEXT R0 L0 -L0: FORGLOOP_INEXT R0 L0 +L0: FORGLOOP R0 L0 2 [inext] RETURN R0 0 )"); @@ -364,7 +368,7 @@ MOVE R1 R0 NEWTABLE R2 0 0 CALL R1 1 3 FORGPREP_INEXT R1 L0 -L0: FORGLOOP_INEXT R1 L0 +L0: FORGLOOP R1 L0 2 [inext] RETURN R0 0 )"); @@ -374,7 +378,7 @@ GETUPVAL R0 0 NEWTABLE R1 0 0 CALL R0 1 3 FORGPREP_INEXT R0 L0 -L0: FORGLOOP_INEXT R0 L0 +L0: FORGLOOP R0 L0 2 [inext] RETURN R0 0 )"); @@ -2107,6 +2111,8 @@ RETURN R3 -1 TEST_CASE("UpvaluesLoopsBytecode") { + ScopedFastFlag sff("LuauCompileNoIpairs", true); + CHECK_EQ("\n" + compileFunction(R"( function test() for i=1,10 do @@ -2169,7 +2175,7 @@ JUMPIFNOT R5 L1 CLOSEUPVALS R3 JUMP L3 L1: CLOSEUPVALS R3 -L2: FORGLOOP_INEXT R0 L0 +L2: FORGLOOP R0 L0 1 [inext] L3: LOADN R0 0 RETURN R0 1 )"); diff --git a/tests/Conformance.test.cpp b/tests/Conformance.test.cpp index 96a2775f..3f415149 100644 --- a/tests/Conformance.test.cpp +++ b/tests/Conformance.test.cpp @@ -231,6 +231,8 @@ TEST_CASE("Assert") TEST_CASE("Basic") { + ScopedFastFlag sff("LuauLenTM", true); + runConformance("basic.lua"); } @@ -301,6 +303,8 @@ TEST_CASE("Errors") TEST_CASE("Events") { + ScopedFastFlag sff("LuauLenTM", true); + runConformance("events.lua"); } @@ -475,6 +479,8 @@ static void populateRTTI(lua_State* L, Luau::TypeId type) TEST_CASE("Types") { + ScopedFastFlag sff("LuauCheckLenMT", true); + runConformance("types.lua", [](lua_State* L) { Luau::NullModuleResolver moduleResolver; Luau::InternalErrorReporter iceHandler; diff --git a/tests/ConstraintGraphBuilder.test.cpp b/tests/ConstraintGraphBuilder.test.cpp index 96b21613..00c3309c 100644 --- a/tests/ConstraintGraphBuilder.test.cpp +++ b/tests/ConstraintGraphBuilder.test.cpp @@ -17,7 +17,7 @@ TEST_CASE_FIXTURE(ConstraintGraphBuilderFixture, "hello_world") )"); cgb.visit(block); - auto constraints = collectConstraints(cgb.rootScope); + auto constraints = collectConstraints(NotNull(cgb.rootScope)); REQUIRE(2 == constraints.size()); @@ -36,7 +36,7 @@ TEST_CASE_FIXTURE(ConstraintGraphBuilderFixture, "primitives") )"); cgb.visit(block); - auto constraints = collectConstraints(cgb.rootScope); + auto constraints = collectConstraints(NotNull(cgb.rootScope)); REQUIRE(3 == constraints.size()); @@ -54,15 +54,15 @@ TEST_CASE_FIXTURE(ConstraintGraphBuilderFixture, "nil_primitive") )"); cgb.visit(block); - auto constraints = collectConstraints(cgb.rootScope); + auto constraints = collectConstraints(NotNull(cgb.rootScope)); ToStringOptions opts; REQUIRE(5 <= constraints.size()); CHECK("*blocked-1* ~ gen () -> (a...)" == toString(*constraints[0], opts)); - CHECK("b ~ inst *blocked-1*" == toString(*constraints[1], opts)); - CHECK("() -> (c...) <: b" == toString(*constraints[2], opts)); - CHECK("c... <: d" == toString(*constraints[3], opts)); + CHECK("*blocked-2* ~ inst *blocked-1*" == toString(*constraints[1], opts)); + CHECK("() -> (b...) <: *blocked-2*" == toString(*constraints[2], opts)); + CHECK("b... <: c" == toString(*constraints[3], opts)); CHECK("nil <: a..." == toString(*constraints[4], opts)); } @@ -74,15 +74,15 @@ TEST_CASE_FIXTURE(ConstraintGraphBuilderFixture, "function_application") )"); cgb.visit(block); - auto constraints = collectConstraints(cgb.rootScope); + auto constraints = collectConstraints(NotNull(cgb.rootScope)); REQUIRE(4 == constraints.size()); ToStringOptions opts; CHECK("string <: a" == toString(*constraints[0], opts)); - CHECK("b ~ inst a" == toString(*constraints[1], opts)); - CHECK("(string) -> (c...) <: b" == toString(*constraints[2], opts)); - CHECK("c... <: d" == toString(*constraints[3], opts)); + CHECK("*blocked-1* ~ inst a" == toString(*constraints[1], opts)); + CHECK("(string) -> (b...) <: *blocked-1*" == toString(*constraints[2], opts)); + CHECK("b... <: c" == toString(*constraints[3], opts)); } TEST_CASE_FIXTURE(ConstraintGraphBuilderFixture, "local_function_definition") @@ -94,7 +94,7 @@ TEST_CASE_FIXTURE(ConstraintGraphBuilderFixture, "local_function_definition") )"); cgb.visit(block); - auto constraints = collectConstraints(cgb.rootScope); + auto constraints = collectConstraints(NotNull(cgb.rootScope)); REQUIRE(2 == constraints.size()); @@ -112,15 +112,15 @@ TEST_CASE_FIXTURE(ConstraintGraphBuilderFixture, "recursive_function") )"); cgb.visit(block); - auto constraints = collectConstraints(cgb.rootScope); + auto constraints = collectConstraints(NotNull(cgb.rootScope)); REQUIRE(4 == constraints.size()); ToStringOptions opts; CHECK("*blocked-1* ~ gen (a) -> (b...)" == toString(*constraints[0], opts)); - CHECK("c ~ inst (a) -> (b...)" == toString(*constraints[1], opts)); - CHECK("(a) -> (d...) <: c" == toString(*constraints[2], opts)); - CHECK("d... <: b..." == toString(*constraints[3], opts)); + CHECK("*blocked-2* ~ inst (a) -> (b...)" == toString(*constraints[1], opts)); + CHECK("(a) -> (c...) <: *blocked-2*" == toString(*constraints[2], opts)); + CHECK("c... <: b..." == toString(*constraints[3], opts)); } TEST_SUITE_END(); diff --git a/tests/ConstraintSolver.test.cpp b/tests/ConstraintSolver.test.cpp index 5959f55c..f521c667 100644 --- a/tests/ConstraintSolver.test.cpp +++ b/tests/ConstraintSolver.test.cpp @@ -9,7 +9,7 @@ using namespace Luau; -static TypeId requireBinding(Scope2* scope, const char* name) +static TypeId requireBinding(NotNull scope, const char* name) { auto b = linearSearchForBinding(scope, name); LUAU_ASSERT(b.has_value()); @@ -26,12 +26,13 @@ TEST_CASE_FIXTURE(ConstraintGraphBuilderFixture, "hello") )"); cgb.visit(block); + NotNull rootScope = NotNull(cgb.rootScope); - ConstraintSolver cs{&arena, cgb.rootScope}; + ConstraintSolver cs{&arena, rootScope}; cs.run(); - TypeId bType = requireBinding(cgb.rootScope, "b"); + TypeId bType = requireBinding(rootScope, "b"); CHECK("number" == toString(bType)); } @@ -45,12 +46,13 @@ TEST_CASE_FIXTURE(ConstraintGraphBuilderFixture, "generic_function") )"); cgb.visit(block); + NotNull rootScope = NotNull(cgb.rootScope); - ConstraintSolver cs{&arena, cgb.rootScope}; + ConstraintSolver cs{&arena, rootScope}; cs.run(); - TypeId idType = requireBinding(cgb.rootScope, "id"); + TypeId idType = requireBinding(rootScope, "id"); CHECK("(a) -> a" == toString(idType)); } @@ -71,14 +73,15 @@ TEST_CASE_FIXTURE(ConstraintGraphBuilderFixture, "proper_let_generalization") )"); cgb.visit(block); + NotNull rootScope = NotNull(cgb.rootScope); ToStringOptions opts; - ConstraintSolver cs{&arena, cgb.rootScope}; + ConstraintSolver cs{&arena, rootScope}; cs.run(); - TypeId idType = requireBinding(cgb.rootScope, "b"); + TypeId idType = requireBinding(rootScope, "b"); CHECK("(a) -> number" == toString(idType, opts)); } diff --git a/tests/Fixture.cpp b/tests/Fixture.cpp index ac22f65b..c92c4457 100644 --- a/tests/Fixture.cpp +++ b/tests/Fixture.cpp @@ -195,12 +195,15 @@ ParseResult Fixture::matchParseError(const std::string& source, const std::strin sourceModule.reset(new SourceModule); ParseResult result = Parser::parse(source.c_str(), source.length(), *sourceModule->names, *sourceModule->allocator, options); - REQUIRE_MESSAGE(!result.errors.empty(), "Expected a parse error in '" << source << "'"); + CHECK_MESSAGE(!result.errors.empty(), "Expected a parse error in '" << source << "'"); - CHECK_EQ(result.errors.front().getMessage(), message); + if (!result.errors.empty()) + { + CHECK_EQ(result.errors.front().getMessage(), message); - if (location) - CHECK_EQ(result.errors.front().getLocation(), *location); + if (location) + CHECK_EQ(result.errors.front().getLocation(), *location); + } return result; } @@ -213,11 +216,14 @@ ParseResult Fixture::matchParseErrorPrefix(const std::string& source, const std: sourceModule.reset(new SourceModule); ParseResult result = Parser::parse(source.c_str(), source.length(), *sourceModule->names, *sourceModule->allocator, options); - REQUIRE_MESSAGE(!result.errors.empty(), "Expected a parse error in '" << source << "'"); + CHECK_MESSAGE(!result.errors.empty(), "Expected a parse error in '" << source << "'"); - const std::string& message = result.errors.front().getMessage(); - CHECK_GE(message.length(), prefix.length()); - CHECK_EQ(prefix, message.substr(0, prefix.size())); + if (!result.errors.empty()) + { + const std::string& message = result.errors.front().getMessage(); + CHECK_GE(message.length(), prefix.length()); + CHECK_EQ(prefix, message.substr(0, prefix.size())); + } return result; } @@ -428,6 +434,7 @@ BuiltinsFixture::BuiltinsFixture(bool freeze, bool prepareAutocomplete) ConstraintGraphBuilderFixture::ConstraintGraphBuilderFixture() : Fixture() + , cgb(mainModuleName, &arena, NotNull(&ice), frontend.getGlobalScope2()) , forceTheFlag{"DebugLuauDeferredConstraintResolution", true} { BlockedTypeVar::nextIndex = 0; diff --git a/tests/Fixture.h b/tests/Fixture.h index 0e3735f6..1bc573da 100644 --- a/tests/Fixture.h +++ b/tests/Fixture.h @@ -133,6 +133,7 @@ struct Fixture TestConfigResolver configResolver; std::unique_ptr sourceModule; Frontend frontend; + InternalErrorReporter ice; TypeChecker& typeChecker; std::string decorateWithTypes(const std::string& code); @@ -160,7 +161,7 @@ struct BuiltinsFixture : Fixture struct ConstraintGraphBuilderFixture : Fixture { TypeArena arena; - ConstraintGraphBuilder cgb{&arena}; + ConstraintGraphBuilder cgb; ScopedFastFlag forceTheFlag; diff --git a/tests/NotNull.test.cpp b/tests/NotNull.test.cpp index ed1c25ec..e77ba78a 100644 --- a/tests/NotNull.test.cpp +++ b/tests/NotNull.test.cpp @@ -30,23 +30,23 @@ struct Test int Test::count = 0; -} +} // namespace int foo(NotNull p) { return *p; } -void bar(int* q) -{} +void bar(int* q) {} TEST_SUITE_BEGIN("NotNull"); TEST_CASE("basic_stuff") { - NotNull a = NotNull{new int(55)}; // Does runtime test - NotNull b{new int(55)}; // As above - // NotNull c = new int(55); // Nope. Mildly regrettable, but implicit conversion from T* to NotNull in the general case is not good. + NotNull a = NotNull{new int(55)}; // Does runtime test + NotNull b{new int(55)}; // As above + // NotNull c = new int(55); // Nope. Mildly regrettable, but implicit conversion from T* to NotNull in the general case is not + // good. // a = nullptr; // nope diff --git a/tests/Parser.test.cpp b/tests/Parser.test.cpp index 878023e3..c3c75998 100644 --- a/tests/Parser.test.cpp +++ b/tests/Parser.test.cpp @@ -6,6 +6,8 @@ #include "doctest.h" +#include + using namespace Luau; namespace @@ -786,33 +788,46 @@ TEST_CASE_FIXTURE(Fixture, "parse_numbers_decimal") TEST_CASE_FIXTURE(Fixture, "parse_numbers_hexadecimal") { - AstStat* stat = parse("return 0xab, 0XAB05, 0xff_ff"); + AstStat* stat = parse("return 0xab, 0XAB05, 0xff_ff, 0xffffffffffffffff"); REQUIRE(stat != nullptr); AstStatReturn* str = stat->as()->body.data[0]->as(); - CHECK(str->list.size == 3); + CHECK(str->list.size == 4); CHECK_EQ(str->list.data[0]->as()->value, 0xab); CHECK_EQ(str->list.data[1]->as()->value, 0xAB05); CHECK_EQ(str->list.data[2]->as()->value, 0xFFFF); + CHECK_EQ(str->list.data[3]->as()->value, double(ULLONG_MAX)); } TEST_CASE_FIXTURE(Fixture, "parse_numbers_binary") { - AstStat* stat = parse("return 0b1, 0b0, 0b101010"); + AstStat* stat = parse("return 0b1, 0b0, 0b101010, 0b1111111111111111111111111111111111111111111111111111111111111111"); REQUIRE(stat != nullptr); AstStatReturn* str = stat->as()->body.data[0]->as(); - CHECK(str->list.size == 3); + CHECK(str->list.size == 4); CHECK_EQ(str->list.data[0]->as()->value, 1); CHECK_EQ(str->list.data[1]->as()->value, 0); CHECK_EQ(str->list.data[2]->as()->value, 42); + CHECK_EQ(str->list.data[3]->as()->value, double(ULLONG_MAX)); } TEST_CASE_FIXTURE(Fixture, "parse_numbers_error") { + ScopedFastFlag luauErrorParseIntegerIssues{"LuauErrorParseIntegerIssues", true}; + CHECK_EQ(getParseError("return 0b123"), "Malformed number"); CHECK_EQ(getParseError("return 123x"), "Malformed number"); CHECK_EQ(getParseError("return 0xg"), "Malformed number"); + CHECK_EQ(getParseError("return 0x0x123"), "Malformed number"); +} + +TEST_CASE_FIXTURE(Fixture, "parse_numbers_range_error") +{ + ScopedFastFlag luauErrorParseIntegerIssues{"LuauErrorParseIntegerIssues", true}; + + CHECK_EQ(getParseError("return 0x10000000000000000"), "Integer number value is out of range"); + CHECK_EQ(getParseError("return 0b10000000000000000000000000000000000000000000000000000000000000000"), "Integer number value is out of range"); } TEST_CASE_FIXTURE(Fixture, "break_return_not_last_error") @@ -2111,6 +2126,15 @@ type C = Packed<(number, X...)> REQUIRE(stat != nullptr); } +TEST_CASE_FIXTURE(Fixture, "invalid_type_forms") +{ + ScopedFastFlag luauFixNamedFunctionParse{"LuauFixNamedFunctionParse", true}; + + matchParseError("type A = (b: number)", "Expected '->' when parsing function type, got "); + matchParseError("type P = () -> T... type B = P<(x: number, y: string)>", "Expected '->' when parsing function type, got '>'"); + matchParseError("type F = (T...) -> ()", "Expected '->' when parsing function type, got '>'"); +} + TEST_SUITE_END(); TEST_SUITE_BEGIN("ParseErrorRecovery"); diff --git a/tests/ToString.test.cpp b/tests/ToString.test.cpp index e03069a9..387e07cd 100644 --- a/tests/ToString.test.cpp +++ b/tests/ToString.test.cpp @@ -409,6 +409,8 @@ TEST_CASE_FIXTURE(Fixture, "toStringDetailed") TEST_CASE_FIXTURE(BuiltinsFixture, "toStringDetailed2") { + ScopedFastFlag sff2{"DebugLuauSharedSelf", true}; + CheckResult result = check(R"( local base = {} function base:one() return 1 end @@ -424,7 +426,7 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "toStringDetailed2") TypeId tType = requireType("inst"); ToStringResult r = toStringDetailed(tType); - CHECK_EQ("{ @metatable { __index: { @metatable { __index: base }, child } }, inst }", r.name); + CHECK_EQ("{ @metatable { __index: { @metatable {| __index: base |}, child } }, inst }", r.name); CHECK_EQ(0, r.nameMap.typeVars.size()); ToStringOptions opts; @@ -455,11 +457,10 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "toStringDetailed2") std::string twoResult = toString(tMeta6->props["two"].type, opts); - REQUIRE_EQ("(a) -> number", oneResult.name); - REQUIRE_EQ("(b) -> number", twoResult); + CHECK_EQ("(a) -> number", oneResult.name); + CHECK_EQ("(b) -> number", twoResult); } - TEST_CASE_FIXTURE(Fixture, "toStringErrorPack") { CheckResult result = check(R"( @@ -688,6 +689,10 @@ TEST_CASE_FIXTURE(Fixture, "pick_distinct_names_for_mixed_explicit_and_implicit_ TEST_CASE_FIXTURE(Fixture, "toStringNamedFunction_include_self_param") { + ScopedFastFlag sff[]{ + {"DebugLuauSharedSelf", true}, + }; + CheckResult result = check(R"( local foo = {} function foo:method(arg: string): () @@ -701,9 +706,12 @@ TEST_CASE_FIXTURE(Fixture, "toStringNamedFunction_include_self_param") CHECK_EQ("foo:method(self: a, arg: string): ()", toStringNamedFunction("foo:method", *ftv)); } - TEST_CASE_FIXTURE(Fixture, "toStringNamedFunction_hide_self_param") { + ScopedFastFlag sff[]{ + {"DebugLuauSharedSelf", true}, + }; + CheckResult result = check(R"( local foo = {} function foo:method(arg: string): () diff --git a/tests/TypeInfer.aliases.test.cpp b/tests/TypeInfer.aliases.test.cpp index d6f0a0c8..bdd4d6fd 100644 --- a/tests/TypeInfer.aliases.test.cpp +++ b/tests/TypeInfer.aliases.test.cpp @@ -73,24 +73,24 @@ TEST_CASE_FIXTURE(Fixture, "cannot_steal_hoisted_type_alias") if (FFlag::DebugLuauDeferredConstraintResolution) { CHECK(result.errors[0] == TypeError{ - Location{{1, 21}, {1, 26}}, - getMainSourceModule()->name, - TypeMismatch{ - getSingletonTypes().numberType, - getSingletonTypes().stringType, - }, - }); + Location{{1, 21}, {1, 26}}, + getMainSourceModule()->name, + TypeMismatch{ + getSingletonTypes().numberType, + getSingletonTypes().stringType, + }, + }); } else { CHECK(result.errors[0] == TypeError{ - Location{{1, 8}, {1, 26}}, - getMainSourceModule()->name, - TypeMismatch{ - getSingletonTypes().numberType, - getSingletonTypes().stringType, - }, - }); + Location{{1, 8}, {1, 26}}, + getMainSourceModule()->name, + TypeMismatch{ + getSingletonTypes().numberType, + getSingletonTypes().stringType, + }, + }); } } @@ -716,6 +716,10 @@ TEST_CASE_FIXTURE(Fixture, "forward_declared_alias_is_not_clobbered_by_prior_uni TEST_CASE_FIXTURE(Fixture, "forward_declared_alias_is_not_clobbered_by_prior_unification_with_any_2") { + ScopedFastFlag sff[] = { + {"DebugLuauSharedSelf", true}, + }; + CheckResult result = check(R"( local B = {} B.bar = 4 @@ -737,7 +741,8 @@ TEST_CASE_FIXTURE(Fixture, "forward_declared_alias_is_not_clobbered_by_prior_uni type FutureIntersection = A & B )"); - LUAU_REQUIRE_NO_ERRORS(result); + // TODO: shared self causes this test to break in bizarre ways. + LUAU_REQUIRE_ERRORS(result); } TEST_CASE_FIXTURE(Fixture, "recursive_types_restriction_ok") diff --git a/tests/TypeInfer.annotations.test.cpp b/tests/TypeInfer.annotations.test.cpp index 3e2ad6dc..8a86ee5f 100644 --- a/tests/TypeInfer.annotations.test.cpp +++ b/tests/TypeInfer.annotations.test.cpp @@ -134,13 +134,13 @@ TEST_CASE_FIXTURE(Fixture, "unknown_type_reference_generates_error") LUAU_REQUIRE_ERROR_COUNT(1, result); CHECK(result.errors[0] == TypeError{ - Location{{1, 17}, {1, 28}}, - getMainSourceModule()->name, - UnknownSymbol{ - "IDoNotExist", - UnknownSymbol::Context::Type, - }, - }); + Location{{1, 17}, {1, 28}}, + getMainSourceModule()->name, + UnknownSymbol{ + "IDoNotExist", + UnknownSymbol::Context::Type, + }, + }); } TEST_CASE_FIXTURE(Fixture, "typeof_variable_type_annotation_should_return_its_type") diff --git a/tests/TypeInfer.functions.test.cpp b/tests/TypeInfer.functions.test.cpp index 036a667a..401a6c64 100644 --- a/tests/TypeInfer.functions.test.cpp +++ b/tests/TypeInfer.functions.test.cpp @@ -37,6 +37,27 @@ TEST_CASE_FIXTURE(Fixture, "check_function_bodies") }})); } +TEST_CASE_FIXTURE(Fixture, "cannot_hoist_interior_defns_into_signature") +{ + // This test verifies that the signature does not have access to types + // declared within the body. Under DCR, if the function's inner scope + // encompasses the entire function expression, it would be possible for this + // to type check (but the solver output is somewhat undefined). This test + // ensures that this isn't the case. + CheckResult result = check(R"( + local function f(x: T) + type T = number + end + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK(result.errors[0] == TypeError{Location{{1, 28}, {1, 29}}, getMainSourceModule()->name, + UnknownSymbol{ + "T", + UnknownSymbol::Context::Type, + }}); +} + TEST_CASE_FIXTURE(Fixture, "infer_return_type") { CheckResult result = check("function take_five() return 5 end"); diff --git a/tests/TypeInfer.generics.test.cpp b/tests/TypeInfer.generics.test.cpp index 97ba0808..e9e94cfb 100644 --- a/tests/TypeInfer.generics.test.cpp +++ b/tests/TypeInfer.generics.test.cpp @@ -271,13 +271,16 @@ TEST_CASE_FIXTURE(Fixture, "infer_nested_generic_function") TEST_CASE_FIXTURE(Fixture, "infer_generic_methods") { + ScopedFastFlag sff{"DebugLuauSharedSelf", true}; + CheckResult result = check(R"( local x = {} function x:id(x) return x end function x:f(): string return self:id("hello") end function x:g(): number return self:id(37) end )"); - LUAU_REQUIRE_NO_ERRORS(result); + // TODO: Quantification should be doing the conversion, not normalization. + LUAU_REQUIRE_ERRORS(result); } TEST_CASE_FIXTURE(Fixture, "calling_self_generic_methods") diff --git a/tests/TypeInfer.operators.test.cpp b/tests/TypeInfer.operators.test.cpp index fd9b1dd4..e6174df2 100644 --- a/tests/TypeInfer.operators.test.cpp +++ b/tests/TypeInfer.operators.test.cpp @@ -461,6 +461,61 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "typecheck_unary_minus") REQUIRE_EQ(gen->message, "Unary operator '-' not supported by type 'bar'"); } +TEST_CASE_FIXTURE(BuiltinsFixture, "typecheck_unary_minus_error") +{ + CheckResult result = check(R"( + --!strict + local foo = { + value = 10 + } + + local mt = {} + setmetatable(foo, mt) + + mt.__unm = function(val: boolean): string + return "test" + end + + local a = -foo + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + CHECK_EQ("string", toString(requireType("a"))); + + TypeMismatch* tm = get(result.errors[0]); + REQUIRE_EQ(*tm->wantedType, *typeChecker.booleanType); + // given type is the typeof(foo) which is complex to compare against +} + +TEST_CASE_FIXTURE(BuiltinsFixture, "typecheck_unary_len_error") +{ + ScopedFastFlag sff("LuauCheckLenMT", true); + + CheckResult result = check(R"( + --!strict + local foo = { + value = 10 + } + local mt = {} + setmetatable(foo, mt) + + mt.__len = function(val: any): string + return "test" + end + + local a = #foo + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + CHECK_EQ("number", toString(requireType("a"))); + + TypeMismatch* tm = get(result.errors[0]); + REQUIRE_EQ(*tm->wantedType, *typeChecker.numberType); + REQUIRE_EQ(*tm->givenType, *typeChecker.stringType); +} + TEST_CASE_FIXTURE(BuiltinsFixture, "unary_not_is_boolean") { CheckResult result = check(R"( diff --git a/tests/TypeInfer.provisional.test.cpp b/tests/TypeInfer.provisional.test.cpp index 487e5979..059aed2e 100644 --- a/tests/TypeInfer.provisional.test.cpp +++ b/tests/TypeInfer.provisional.test.cpp @@ -499,4 +499,26 @@ TEST_CASE_FIXTURE(Fixture, "constrained_is_level_dependent") CHECK_EQ("(t1) -> {| [t1]: boolean |} where t1 = t2 ; t2 = {+ m1: (t1) -> (a...), m2: (t2) -> (b...) +}", toString(requireType("f"))); } +TEST_CASE_FIXTURE(BuiltinsFixture, "greedy_inference_with_shared_self_triggers_function_with_no_returns") +{ + ScopedFastFlag sff{"DebugLuauSharedSelf", true}; + + CheckResult result = check(R"( + local T = {} + T.__index = T + + function T.new() + local self = setmetatable({}, T) + return self:ctor() or self + end + + function T:ctor() + -- oops, no return! + end + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ("Not all codepaths in this function return '{ @metatable T, {| |} }, a...'.", toString(result.errors[0])); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.tables.test.cpp b/tests/TypeInfer.tables.test.cpp index 77a2928c..eead5b30 100644 --- a/tests/TypeInfer.tables.test.cpp +++ b/tests/TypeInfer.tables.test.cpp @@ -1863,6 +1863,8 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "quantifying_a_bound_var_works") TEST_CASE_FIXTURE(BuiltinsFixture, "less_exponential_blowup_please") { + ScopedFastFlag sff{"DebugLuauSharedSelf", true}; + CheckResult result = check(R"( --!strict @@ -1890,7 +1892,7 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "less_exponential_blowup_please") newData:First() )"); - LUAU_REQUIRE_ERROR_COUNT(1, result); + LUAU_REQUIRE_ERROR_COUNT(2, result); } TEST_CASE_FIXTURE(Fixture, "common_table_element_union_in_call") @@ -2868,6 +2870,7 @@ TEST_CASE_FIXTURE(Fixture, "inferred_return_type_of_free_table") { ScopedFastFlag sff[] = { {"LuauLowerBoundsCalculation", true}, + {"DebugLuauSharedSelf", true}, }; check(R"( @@ -2887,7 +2890,7 @@ TEST_CASE_FIXTURE(Fixture, "inferred_return_type_of_free_table") end )"); - CHECK_EQ("(t1) -> {| Byte: (b) -> (a...), PeekByte: (c) -> (a...) |} where t1 = {+ byte: (t1, number) -> (a...) +}", + CHECK_EQ("(t1) -> {| Byte: (a) -> (b...), PeekByte: (a) -> (b...) |} where t1 = {+ byte: (t1, number) -> (b...) +}", toString(requireType("Base64FileReader"))); } @@ -2904,6 +2907,66 @@ TEST_CASE_FIXTURE(Fixture, "mixed_tables_with_implicit_numbered_keys") CHECK_EQ("Type 'number' could not be converted into 'string'", toString(result.errors[2])); } +TEST_CASE_FIXTURE(Fixture, "shared_selfs") +{ + ScopedFastFlag sff{"DebugLuauSharedSelf", true}; + + CheckResult result = check(R"( + local t = {} + t.x = 5 + + function t:m1() return self.x end + function t:m2() return self.y end + + return t + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + ToStringOptions opts; + opts.exhaustive = true; + CHECK_EQ("{| m1: ({+ x: a, y: b +}) -> a, m2: ({+ x: a, y: b +}) -> b, x: number |}", toString(requireType("t"), opts)); +} + +TEST_CASE_FIXTURE(Fixture, "shared_selfs_from_free_param") +{ + ScopedFastFlag sff{"DebugLuauSharedSelf", true}; + + CheckResult result = check(R"( + local function f(t) + function t:m1() return self.x end + function t:m2() return self.y end + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ("({+ m1: ({+ x: a, y: b +}) -> a, m2: ({+ x: a, y: b +}) -> b +}) -> ()", toString(requireType("f"))); +} + +TEST_CASE_FIXTURE(BuiltinsFixture, "shared_selfs_through_metatables") +{ + ScopedFastFlag sff{"DebugLuauSharedSelf", true}; + + CheckResult result = check(R"( + local t = {} + t.__index = t + setmetatable({}, t) + + function t:m1() return self.x end + function t:m2() return self.y end + + return t + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + ToStringOptions opts; + opts.exhaustive = true; + CHECK_EQ( + toString(requireType("t"), opts), "t1 where t1 = {| __index: t1, m1: ({+ x: a, y: b +}) -> a, m2: ({+ x: a, y: b +}) -> b |}"); +} + TEST_CASE_FIXTURE(Fixture, "expected_indexer_value_type_extra") { CheckResult result = check(R"( @@ -2953,4 +3016,58 @@ TEST_CASE_FIXTURE(Fixture, "prop_access_on_unions_of_indexers_where_key_whose_ty CHECK_EQ("Type '{number} | {| [boolean]: number |}' does not have key 'x'", toString(result.errors[0])); } +TEST_CASE_FIXTURE(BuiltinsFixture, "quantify_metatables_of_metatables_of_table") +{ + ScopedFastFlag sff[]{ + {"DebugLuauSharedSelf", true}, + }; + + CheckResult result = check(R"( + local T = {} + + function T:m() + return self.x, self.y + end + + function T:n() + end + + local U = setmetatable({}, {__index = T}) + + local V = setmetatable({}, {__index = U}) + + return V + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + ToStringOptions opts; + opts.exhaustive = true; + CHECK_EQ(toString(requireType("V"), opts), "{ @metatable { __index: { @metatable { __index: {| m: ({+ x: a, y: b +}) -> (a, b), n: ({+ x: a, y: b +}) -> () |} }, { } } }, { } }"); +} + +TEST_CASE_FIXTURE(Fixture, "quantify_even_that_table_was_never_exported_at_all") +{ + ScopedFastFlag sff{"DebugLuauSharedSelf", true}; + + CheckResult result = check(R"( + local T = {} + + function T:m() + return self.x + end + + function T:n() + return self.y + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + ToStringOptions opts; + opts.exhaustive = true; + CHECK_EQ("{| m: ({+ x: a, y: b +}) -> a, n: ({+ x: a, y: b +}) -> b |}", toString(requireType("T"), opts)); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.test.cpp b/tests/TypeInfer.test.cpp index 6a048b26..efdfe0b1 100644 --- a/tests/TypeInfer.test.cpp +++ b/tests/TypeInfer.test.cpp @@ -369,14 +369,14 @@ TEST_CASE_FIXTURE(Fixture, "globals_are_banned_in_strict_mode") CHECK_EQ("foo", us->name); } -TEST_CASE_FIXTURE(BuiltinsFixture, "correctly_scope_locals_do") +TEST_CASE_FIXTURE(Fixture, "correctly_scope_locals_do") { CheckResult result = check(R"( do local a = 1 end - print(a) -- oops! + local b = a -- oops! )"); LUAU_REQUIRE_ERROR_COUNT(1, result); diff --git a/tests/conformance/basic.lua b/tests/conformance/basic.lua index f803c319..385a0450 100644 --- a/tests/conformance/basic.lua +++ b/tests/conformance/basic.lua @@ -118,10 +118,12 @@ assert((function() return #_G end)() == 0) assert((function() return #{1,2} end)() == 2) assert((function() return #'g' end)() == 1) -assert((function() local ud = newproxy(true) getmetatable(ud).__len = function() return 42 end return #ud end)() == 42) - assert((function() local a = 1 a = -a return a end)() == -1) +-- __len metamethod +assert((function() local ud = newproxy(true) getmetatable(ud).__len = function() return 42 end return #ud end)() == 42) +assert((function() local t = {} setmetatable(t, { __len = function() return 42 end }) return #t end)() == 42) + -- while/repeat assert((function() local a = 10 local b = 1 while a > 1 do b = b * 2 a = a - 1 end return b end)() == 512) assert((function() local a = 10 local b = 1 repeat b = b * 2 a = a - 1 until a == 1 return b end)() == 512) @@ -889,6 +891,10 @@ assert((function() return table.concat(res, ',') end)() == "6,8,10") +-- typeof and type require an argument +assert(pcall(typeof) == false) +assert(pcall(type) == false) + -- typeof == type in absence of custom userdata assert(concat(typeof(5), typeof(nil), typeof({}), typeof(newproxy())) == "number,nil,table,userdata") diff --git a/tests/conformance/events.lua b/tests/conformance/events.lua index 32e090ac..42f1beda 100644 --- a/tests/conformance/events.lua +++ b/tests/conformance/events.lua @@ -386,4 +386,42 @@ do assert(t.X) -- fails if table flags are set incorrectly end +do + -- verify __len behavior & error handling + local t = {1} + + setmetatable(t, {}) + assert(#t == 1) + + setmetatable(t, { __len = rawlen }) + assert(#t == 1) + + setmetatable(t, { __len = function() return 42 end }) + assert(#t == 42) + + setmetatable(t, { __len = 42 }) + local ok, err = pcall(function() return #t end) + assert(not ok and err:match("attempt to call a number value")) + + setmetatable(t, { __len = function() end }) + local ok, err = pcall(function() return #t end) + assert(not ok and err:match("'__len' must return a number")) + + setmetatable(t, { __len = error }) + local ok, err = pcall(function() return #t end) + assert(not ok and err == t) +end + +-- verify rawlen behavior +do + local t = {1} + setmetatable(t, { __len = 42 }) + + assert(rawlen(t) == 1) + assert(rawlen("foo") == 3) + + local ok, err = pcall(function() return rawlen(42) end) + assert(not ok and err:match("table or string expected")) +end + return 'OK' From 6467c855e8fd4dff99ae868230d9be61991ccbd9 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Thu, 30 Jun 2022 17:07:56 -0700 Subject: [PATCH 298/399] Update compatibility.md (#566) Update `__len` metamethod (pending the code change that implements this) --- docs/_pages/compatibility.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/_pages/compatibility.md b/docs/_pages/compatibility.md index d1686c2f..bdb3d814 100644 --- a/docs/_pages/compatibility.md +++ b/docs/_pages/compatibility.md @@ -54,7 +54,7 @@ Sandboxing challenges are [covered in the dedicated section](sandbox). | goto statement | ❌ | this complicates the compiler, makes control flow unstructured and doesn't address a significant need | | finalizers for tables | ❌ | no `__gc` support due to sandboxing and performance/complexity | | no more fenv for threads or functions | 😞 | we love this, but it breaks compatibility | -| tables honor the `__len` metamethod | 🤷‍♀️ | performance implications, no strong use cases +| tables honor the `__len` metamethod | ✔️ | | | hex and `\z` escapes in strings | ✔️ | | | support for hexadecimal floats | 🤷‍♀️ | no strong use cases | | order metamethods work for different types | ❌ | no strong use cases and more complicated semantics, compatibility and performance implications | From 48aa7a51622d7eb8125a78cad549cdd384bcdb1e Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Mon, 4 Jul 2022 11:13:07 -0700 Subject: [PATCH 299/399] bench: Implement first class support for callgrind (#570) Since callgrind allows to control stats collection from the guest, this allows us to reset the collection right before the benchmark starts. This change exposes this to the benchmark runner and integrates callgrind data parsing into bench.py, so that we can run bench.py with --callgrind argument and, as long as the runner was built with callgrind support, we get instruction counts from the run. We convert instruction counts to seconds using 10G instructions/second rate; there's no correct way to do this without simulating the full CPU pipeline but it results in time units on a similar scale to real runs. --- .github/workflows/benchmark-dev.yml | 270 ++++++++++++++++++++++++++++ .github/workflows/benchmark.yml | 233 ++---------------------- CLI/Repl.cpp | 37 ++++ Makefile | 4 + bench/bench.py | 27 +++ bench/bench_support.lua | 10 ++ 6 files changed, 363 insertions(+), 218 deletions(-) create mode 100644 .github/workflows/benchmark-dev.yml diff --git a/.github/workflows/benchmark-dev.yml b/.github/workflows/benchmark-dev.yml new file mode 100644 index 00000000..21f9559e --- /dev/null +++ b/.github/workflows/benchmark-dev.yml @@ -0,0 +1,270 @@ +name: benchmark-dev + +on: + push: + branches: + - master + paths-ignore: + - "docs/**" + - "papers/**" + - "rfcs/**" + - "*.md" + - "prototyping/**" + +jobs: + windows: + name: windows-${{matrix.arch}} + strategy: + fail-fast: false + matrix: + os: [windows-latest] + arch: [Win32, x64] + bench: + - { + script: "run-benchmarks", + timeout: 12, + title: "Luau Benchmarks", + cachegrindTitle: "Performance", + cachegrindIterCount: 20, + } + benchResultsRepo: + - { name: "luau-lang/benchmark-data", branch: "main" } + + runs-on: ${{ matrix.os }} + steps: + - name: Checkout Luau repository + uses: actions/checkout@v3 + + - name: Build Luau + shell: bash # necessary for fail-fast + run: | + mkdir build && cd build + cmake .. -DCMAKE_BUILD_TYPE=Release + cmake --build . --target Luau.Repl.CLI --config Release + cmake --build . --target Luau.Analyze.CLI --config Release + + - name: Move build files to root + run: | + move build/Release/* . + + - uses: actions/setup-python@v3 + with: + python-version: "3.9" + architecture: "x64" + + - name: Install python dependencies + run: | + python -m pip install requests + python -m pip install --user numpy scipy matplotlib ipython jupyter pandas sympy nose + + - name: Run benchmark + run: | + python bench/bench.py | tee ${{ matrix.bench.script }}-output.txt + + - name: Checkout Benchmark Results repository + uses: actions/checkout@v3 + with: + repository: ${{ matrix.benchResultsRepo.name }} + ref: ${{ matrix.benchResultsRepo.branch }} + token: ${{ secrets.BENCH_GITHUB_TOKEN }} + path: "./gh-pages" + + - name: Store ${{ matrix.bench.title }} result + uses: Roblox/rhysd-github-action-benchmark@v-luau + with: + name: ${{ matrix.bench.title }} (Windows ${{matrix.arch}}) + tool: "benchmarkluau" + output-file-path: ./${{ matrix.bench.script }}-output.txt + external-data-json-path: ./gh-pages/dev/bench/data.json + github-token: ${{ secrets.GITHUB_TOKEN }} + + - name: Push benchmark results + if: github.event_name == 'push' + run: | + echo "Pushing benchmark results..." + cd gh-pages + git config user.name github-actions + git config user.email github@users.noreply.github.com + git add ./dev/bench/data.json + git commit -m "Add benchmarks results for ${{ github.sha }}" + git push + cd .. + + unix: + name: ${{matrix.os}} + strategy: + fail-fast: false + matrix: + os: [ubuntu-latest, macos-latest] + bench: + - { + script: "run-benchmarks", + timeout: 12, + title: "Luau Benchmarks", + cachegrindTitle: "Performance", + cachegrindIterCount: 20, + } + benchResultsRepo: + - { name: "luau-lang/benchmark-data", branch: "main" } + + runs-on: ${{ matrix.os }} + steps: + - name: Checkout Luau repository + uses: actions/checkout@v3 + + - name: Build Luau + run: make config=release luau luau-analyze + + - uses: actions/setup-python@v3 + with: + python-version: "3.9" + architecture: "x64" + + - name: Install python dependencies + run: | + python -m pip install requests + python -m pip install --user numpy scipy matplotlib ipython jupyter pandas sympy nose + + - name: Run benchmark + run: | + python bench/bench.py | tee ${{ matrix.bench.script }}-output.txt + + - name: Install valgrind + if: matrix.os == 'ubuntu-latest' + run: | + sudo apt-get install valgrind + + - name: Run ${{ matrix.bench.title }} (Cold Cachegrind) + if: matrix.os == 'ubuntu-latest' + run: sudo bash ./scripts/run-with-cachegrind.sh python ./bench/bench.py "${{ matrix.bench.cachegrindTitle}}Cold" 1 | tee -a ${{ matrix.bench.script }}-output.txt + + - name: Run ${{ matrix.bench.title }} (Warm Cachegrind) + if: matrix.os == 'ubuntu-latest' + run: sudo bash ./scripts/run-with-cachegrind.sh python ./bench/bench.py "${{ matrix.bench.cachegrindTitle }}" ${{ matrix.bench.cachegrindIterCount }} | tee -a ${{ matrix.bench.script }}-output.txt + + - name: Checkout Benchmark Results repository + uses: actions/checkout@v3 + with: + repository: ${{ matrix.benchResultsRepo.name }} + ref: ${{ matrix.benchResultsRepo.branch }} + token: ${{ secrets.BENCH_GITHUB_TOKEN }} + path: "./gh-pages" + + - name: Store ${{ matrix.bench.title }} result + uses: Roblox/rhysd-github-action-benchmark@v-luau + with: + name: ${{ matrix.bench.title }} + tool: "benchmarkluau" + output-file-path: ./${{ matrix.bench.script }}-output.txt + external-data-json-path: ./gh-pages/dev/bench/data.json + github-token: ${{ secrets.BENCH_GITHUB_TOKEN }} + + - name: Store ${{ matrix.bench.title }} result (CacheGrind) + if: matrix.os == 'ubuntu-latest' + uses: Roblox/rhysd-github-action-benchmark@v-luau + with: + name: ${{ matrix.bench.title }} (CacheGrind) + tool: "roblox" + output-file-path: ./${{ matrix.bench.script }}-output.txt + external-data-json-path: ./gh-pages/dev/bench/data.json + github-token: ${{ secrets.BENCH_GITHUB_TOKEN }} + + - name: Push benchmark results + if: github.event_name == 'push' + run: | + echo "Pushing benchmark results..." + cd gh-pages + git config user.name github-actions + git config user.email github@users.noreply.github.com + git add ./dev/bench/data.json + git commit -m "Add benchmarks results for ${{ github.sha }}" + git push + cd .. + + static-analysis: + name: luau-analyze + strategy: + fail-fast: false + matrix: + os: [ubuntu-latest] + bench: + - { + script: "run-analyze", + timeout: 12, + title: "Luau Analyze", + cachegrindTitle: "Performance", + cachegrindIterCount: 20, + } + benchResultsRepo: + - { name: "luau-lang/benchmark-data", branch: "main" } + runs-on: ${{ matrix.os }} + steps: + - uses: actions/checkout@v3 + with: + token: "${{ secrets.BENCH_GITHUB_TOKEN }}" + + - name: Build Luau + run: make config=release luau luau-analyze + + - uses: actions/setup-python@v4 + with: + python-version: "3.9" + architecture: "x64" + + - name: Install python dependencies + run: | + sudo pip install requests numpy scipy matplotlib ipython jupyter pandas sympy nose + + - name: Install valgrind + run: | + sudo apt-get install valgrind + + - name: Run Luau Analyze on static file + run: sudo python ./bench/measure_time.py ./build/release/luau-analyze bench/static_analysis/LuauPolyfillMap.lua | tee ${{ matrix.bench.script }}-output.txt + + - name: Run ${{ matrix.bench.title }} (Cold Cachegrind) + run: sudo ./scripts/run-with-cachegrind.sh python ./bench/measure_time.py "${{ matrix.bench.cachegrindTitle}}Cold" 1 ./build/release/luau-analyze bench/static_analysis/LuauPolyfillMap.lua | tee -a ${{ matrix.bench.script }}-output.txt + + - name: Run ${{ matrix.bench.title }} (Warm Cachegrind) + run: sudo bash ./scripts/run-with-cachegrind.sh python ./bench/measure_time.py "${{ matrix.bench.cachegrindTitle}}" 1 ./build/release/luau-analyze bench/static_analysis/LuauPolyfillMap.lua | tee -a ${{ matrix.bench.script }}-output.txt + + - name: Checkout Benchmark Results repository + uses: actions/checkout@v3 + with: + repository: ${{ matrix.benchResultsRepo.name }} + ref: ${{ matrix.benchResultsRepo.branch }} + token: ${{ secrets.BENCH_GITHUB_TOKEN }} + path: "./gh-pages" + + - name: Store ${{ matrix.bench.title }} result + uses: Roblox/rhysd-github-action-benchmark@v-luau + with: + name: ${{ matrix.bench.title }} + tool: "benchmarkluau" + + gh-pages-branch: "main" + output-file-path: ./${{ matrix.bench.script }}-output.txt + external-data-json-path: ./gh-pages/dev/bench/data.json + github-token: ${{ secrets.BENCH_GITHUB_TOKEN }} + + - name: Store ${{ matrix.bench.title }} result (CacheGrind) + uses: Roblox/rhysd-github-action-benchmark@v-luau + with: + name: ${{ matrix.bench.title }} + tool: "roblox" + gh-pages-branch: "main" + output-file-path: ./${{ matrix.bench.script }}-output.txt + external-data-json-path: ./gh-pages/dev/bench/data.json + github-token: ${{ secrets.BENCH_GITHUB_TOKEN }} + + - name: Push benchmark results + if: github.event_name == 'push' + run: | + echo "Pushing benchmark results..." + cd gh-pages + git config user.name github-actions + git config user.email github@users.noreply.github.com + git add ./dev/bench/data.json + git commit -m "Add benchmarks results for ${{ github.sha }}" + git push + cd .. diff --git a/.github/workflows/benchmark.yml b/.github/workflows/benchmark.yml index 4df7b2f3..48296823 100644 --- a/.github/workflows/benchmark.yml +++ b/.github/workflows/benchmark.yml @@ -12,21 +12,13 @@ on: - "prototyping/**" jobs: - windows: - name: windows-${{matrix.arch}} + callgrind: + name: callgrind ${{ matrix.compiler }} strategy: fail-fast: false matrix: - os: [windows-latest] - arch: [Win32, x64] - bench: - - { - script: "run-benchmarks", - timeout: 12, - title: "Luau Benchmarks", - cachegrindTitle: "Performance", - cachegrindIterCount: 20, - } + os: [ubuntu-22.04] + compiler: [g++] benchResultsRepo: - { name: "luau-lang/benchmark-data", branch: "main" } @@ -35,200 +27,18 @@ jobs: - name: Checkout Luau repository uses: actions/checkout@v3 - - name: Build Luau - shell: bash # necessary for fail-fast - run: | - mkdir build && cd build - cmake .. -DCMAKE_BUILD_TYPE=Release - cmake --build . --target Luau.Repl.CLI --config Release - cmake --build . --target Luau.Analyze.CLI --config Release - - - name: Move build files to root - run: | - move build/Release/* . - - - uses: actions/setup-python@v3 - with: - python-version: "3.9" - architecture: "x64" - - - name: Install python dependencies - run: | - python -m pip install requests - python -m pip install --user numpy scipy matplotlib ipython jupyter pandas sympy nose - - - name: Run benchmark - run: | - python bench/bench.py | tee ${{ matrix.bench.script }}-output.txt - - - name: Checkout Benchmark Results repository - uses: actions/checkout@v3 - with: - repository: ${{ matrix.benchResultsRepo.name }} - ref: ${{ matrix.benchResultsRepo.branch }} - token: ${{ secrets.BENCH_GITHUB_TOKEN }} - path: "./gh-pages" - - - name: Store ${{ matrix.bench.title }} result - uses: Roblox/rhysd-github-action-benchmark@v-luau - with: - name: ${{ matrix.bench.title }} (Windows ${{matrix.arch}}) - tool: "benchmarkluau" - output-file-path: ./${{ matrix.bench.script }}-output.txt - external-data-json-path: ./gh-pages/dev/bench/data.json - github-token: ${{ secrets.GITHUB_TOKEN }} - - - name: Push benchmark results - if: github.event_name == 'push' - run: | - echo "Pushing benchmark results..." - cd gh-pages - git config user.name github-actions - git config user.email github@users.noreply.github.com - git add ./dev/bench/data.json - git commit -m "Add benchmarks results for ${{ github.sha }}" - git push - cd .. - - unix: - name: ${{matrix.os}} - strategy: - fail-fast: false - matrix: - os: [ubuntu-latest, macos-latest] - bench: - - { - script: "run-benchmarks", - timeout: 12, - title: "Luau Benchmarks", - cachegrindTitle: "Performance", - cachegrindIterCount: 20, - } - benchResultsRepo: - - { name: "luau-lang/benchmark-data", branch: "main" } - - runs-on: ${{ matrix.os }} - steps: - - name: Checkout Luau repository - uses: actions/checkout@v3 - - - name: Build Luau - run: make config=release luau luau-analyze - - - uses: actions/setup-python@v3 - with: - python-version: "3.9" - architecture: "x64" - - - name: Install python dependencies - run: | - python -m pip install requests - python -m pip install --user numpy scipy matplotlib ipython jupyter pandas sympy nose - - - name: Run benchmark - run: | - python bench/bench.py | tee ${{ matrix.bench.script }}-output.txt - - - name: Install valgrind - if: matrix.os == 'ubuntu-latest' - run: | - sudo apt-get install valgrind - - - name: Run ${{ matrix.bench.title }} (Cold Cachegrind) - if: matrix.os == 'ubuntu-latest' - run: sudo bash ./scripts/run-with-cachegrind.sh python ./bench/bench.py "${{ matrix.bench.cachegrindTitle}}Cold" 1 | tee -a ${{ matrix.bench.script }}-output.txt - - - name: Run ${{ matrix.bench.title }} (Warm Cachegrind) - if: matrix.os == 'ubuntu-latest' - run: sudo bash ./scripts/run-with-cachegrind.sh python ./bench/bench.py "${{ matrix.bench.cachegrindTitle }}" ${{ matrix.bench.cachegrindIterCount }} | tee -a ${{ matrix.bench.script }}-output.txt - - - name: Checkout Benchmark Results repository - uses: actions/checkout@v3 - with: - repository: ${{ matrix.benchResultsRepo.name }} - ref: ${{ matrix.benchResultsRepo.branch }} - token: ${{ secrets.BENCH_GITHUB_TOKEN }} - path: "./gh-pages" - - - name: Store ${{ matrix.bench.title }} result - uses: Roblox/rhysd-github-action-benchmark@v-luau - with: - name: ${{ matrix.bench.title }} - tool: "benchmarkluau" - output-file-path: ./${{ matrix.bench.script }}-output.txt - external-data-json-path: ./gh-pages/dev/bench/data.json - github-token: ${{ secrets.BENCH_GITHUB_TOKEN }} - - - name: Store ${{ matrix.bench.title }} result (CacheGrind) - if: matrix.os == 'ubuntu-latest' - uses: Roblox/rhysd-github-action-benchmark@v-luau - with: - name: ${{ matrix.bench.title }} (CacheGrind) - tool: "roblox" - output-file-path: ./${{ matrix.bench.script }}-output.txt - external-data-json-path: ./gh-pages/dev/bench/data.json - github-token: ${{ secrets.BENCH_GITHUB_TOKEN }} - - - name: Push benchmark results - if: github.event_name == 'push' - run: | - echo "Pushing benchmark results..." - cd gh-pages - git config user.name github-actions - git config user.email github@users.noreply.github.com - git add ./dev/bench/data.json - git commit -m "Add benchmarks results for ${{ github.sha }}" - git push - cd .. - - static-analysis: - name: luau-analyze - strategy: - fail-fast: false - matrix: - os: [ubuntu-latest] - bench: - - { - script: "run-analyze", - timeout: 12, - title: "Luau Analyze", - cachegrindTitle: "Performance", - cachegrindIterCount: 20, - } - benchResultsRepo: - - { name: "luau-lang/benchmark-data", branch: "main" } - runs-on: ${{ matrix.os }} - steps: - - uses: actions/checkout@v3 - with: - token: "${{ secrets.BENCH_GITHUB_TOKEN }}" - - - name: Build Luau - run: make config=release luau luau-analyze - - - uses: actions/setup-python@v4 - with: - python-version: "3.9" - architecture: "x64" - - - name: Install python dependencies - run: | - sudo pip install requests numpy scipy matplotlib ipython jupyter pandas sympy nose - - name: Install valgrind run: | sudo apt-get install valgrind - - name: Run Luau Analyze on static file - run: sudo python ./bench/measure_time.py ./build/release/luau-analyze bench/static_analysis/LuauPolyfillMap.lua | tee ${{ matrix.bench.script }}-output.txt + - name: Build Luau + run: CXX=${{ matrix.compiler }} make config=release CALLGRIND=1 luau - - name: Run ${{ matrix.bench.title }} (Cold Cachegrind) - run: sudo ./scripts/run-with-cachegrind.sh python ./bench/measure_time.py "${{ matrix.bench.cachegrindTitle}}Cold" 1 ./build/release/luau-analyze bench/static_analysis/LuauPolyfillMap.lua | tee -a ${{ matrix.bench.script }}-output.txt + - name: Run benchmark + run: | + python bench/bench.py --callgrind --vm "./luau -O2" | tee output.txt - - name: Run ${{ matrix.bench.title }} (Warm Cachegrind) - run: sudo bash ./scripts/run-with-cachegrind.sh python ./bench/measure_time.py "${{ matrix.bench.cachegrindTitle}}" 1 ./build/release/luau-analyze bench/static_analysis/LuauPolyfillMap.lua | tee -a ${{ matrix.bench.script }}-output.txt - - - name: Checkout Benchmark Results repository + - name: Checkout benchmark results uses: actions/checkout@v3 with: repository: ${{ matrix.benchResultsRepo.name }} @@ -236,26 +46,13 @@ jobs: token: ${{ secrets.BENCH_GITHUB_TOKEN }} path: "./gh-pages" - - name: Store ${{ matrix.bench.title }} result + - name: Store results uses: Roblox/rhysd-github-action-benchmark@v-luau with: - name: ${{ matrix.bench.title }} + name: callgrind ${{ matrix.compiler }} tool: "benchmarkluau" - - gh-pages-branch: "main" - output-file-path: ./${{ matrix.bench.script }}-output.txt - external-data-json-path: ./gh-pages/dev/bench/data.json - github-token: ${{ secrets.BENCH_GITHUB_TOKEN }} - - - name: Store ${{ matrix.bench.title }} result (CacheGrind) - uses: Roblox/rhysd-github-action-benchmark@v-luau - with: - name: ${{ matrix.bench.title }} - tool: "roblox" - gh-pages-branch: "main" - output-file-path: ./${{ matrix.bench.script }}-output.txt - external-data-json-path: ./gh-pages/dev/bench/data.json - github-token: ${{ secrets.BENCH_GITHUB_TOKEN }} + output-file-path: ./output.txt + external-data-json-path: ./gh-pages/bench/data.json - name: Push benchmark results if: github.event_name == 'push' @@ -264,7 +61,7 @@ jobs: cd gh-pages git config user.name github-actions git config user.email github@users.noreply.github.com - git add ./dev/bench/data.json + git add ./bench/data.json git commit -m "Add benchmarks results for ${{ github.sha }}" git push cd .. diff --git a/CLI/Repl.cpp b/CLI/Repl.cpp index 83060f5b..5fe12bec 100644 --- a/CLI/Repl.cpp +++ b/CLI/Repl.cpp @@ -21,6 +21,10 @@ #include #endif +#ifdef CALLGRIND +#include +#endif + #include LUAU_FASTFLAG(DebugLuauTimeTracing) @@ -166,6 +170,36 @@ static int lua_collectgarbage(lua_State* L) luaL_error(L, "collectgarbage must be called with 'count' or 'collect'"); } +#ifdef CALLGRIND +static int lua_callgrind(lua_State* L) +{ + const char* option = luaL_checkstring(L, 1); + + if (strcmp(option, "running") == 0) + { + int r = RUNNING_ON_VALGRIND; + lua_pushboolean(L, r); + return 1; + } + + if (strcmp(option, "zero") == 0) + { + CALLGRIND_ZERO_STATS; + return 0; + } + + if (strcmp(option, "dump") == 0) + { + const char* name = luaL_checkstring(L, 2); + + CALLGRIND_DUMP_STATS_AT(name); + return 0; + } + + luaL_error(L, "callgrind must be called with one of 'running', 'zero', 'dump'"); +} +#endif + void setupState(lua_State* L) { luaL_openlibs(L); @@ -174,6 +208,9 @@ void setupState(lua_State* L) {"loadstring", lua_loadstring}, {"require", lua_require}, {"collectgarbage", lua_collectgarbage}, +#ifdef CALLGRIND + {"callgrind", lua_callgrind}, +#endif {NULL, NULL}, }; diff --git a/Makefile b/Makefile index 1082666d..b8077897 100644 --- a/Makefile +++ b/Makefile @@ -93,6 +93,10 @@ ifeq ($(config),fuzz) LDFLAGS+=-fsanitize=address,fuzzer endif +ifneq ($(CALLGRIND),) + CXXFLAGS+=-DCALLGRIND=$(CALLGRIND) +endif + # target-specific flags $(AST_OBJECTS): CXXFLAGS+=-std=c++17 -ICommon/include -IAst/include $(COMPILER_OBJECTS): CXXFLAGS+=-std=c++17 -ICompiler/include -ICommon/include -IAst/include diff --git a/bench/bench.py b/bench/bench.py index 67fc8cf7..e78e96a8 100644 --- a/bench/bench.py +++ b/bench/bench.py @@ -40,6 +40,7 @@ argumentParser.add_argument('--results', dest='results',type=str,nargs='*',help= argumentParser.add_argument('--run-test', action='store', default=None, help='Regex test filter') argumentParser.add_argument('--extra-loops', action='store',type=int,default=0, help='Amount of times to loop over one test (one test already performs multiple runs)') argumentParser.add_argument('--filename', action='store',type=str,default='bench', help='File name for graph and results file') +argumentParser.add_argument('--callgrind', dest='callgrind',action='store_const',const=1,default=0,help='Use callgrind to run benchmarks') if matplotlib != None: argumentParser.add_argument('--absolute', dest='absolute',action='store_const',const=1,default=0,help='Display absolute values instead of relative (enabled by default when benchmarking a single VM)') @@ -55,6 +56,9 @@ argumentParser.add_argument('--no-print-influx-debugging', action='store_false', argumentParser.add_argument('--no-print-final-summary', action='store_false', dest='print_final_summary', help="Don't print a table summarizing the results after all tests are run") +# Assume 2.5 IPC on a 4 GHz CPU; this is obviously incorrect but it allows us to display simulated instruction counts using regular time units +CALLGRIND_INSN_PER_SEC = 2.5 * 4e9 + def arrayRange(count): result = [] @@ -71,6 +75,21 @@ def arrayRangeOffset(count, offset): return result +def getCallgrindOutput(lines): + result = [] + name = None + + for l in lines: + if l.startswith("desc: Trigger: Client Request: "): + name = l[31:].strip() + elif l.startswith("summary: ") and name != None: + insn = int(l[9:]) + # Note: we only run each bench once under callgrind so we only report a single time per run; callgrind instruction count variance is ~0.01% so it might as well be zero + result += "|><|" + name + "|><|" + str(insn / CALLGRIND_INSN_PER_SEC * 1000.0) + "||_||" + name = None + + return "".join(result) + def getVmOutput(cmd): if os.name == "nt": try: @@ -79,6 +98,14 @@ def getVmOutput(cmd): exit(1) except: return "" + elif arguments.callgrind: + try: + subprocess.check_call("valgrind --tool=callgrind --callgrind-out-file=callgrind.out --combine-dumps=yes --dump-line=no " + cmd, shell=True, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, cwd=scriptdir) + file = open(os.path.join(scriptdir, "callgrind.out"), "r") + lines = file.readlines() + return getCallgrindOutput(lines) + except: + return "" else: with subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True, cwd=scriptdir) as p: # Try to lock to a single processor diff --git a/bench/bench_support.lua b/bench/bench_support.lua index 171b8da7..a9608ecc 100644 --- a/bench/bench_support.lua +++ b/bench/bench_support.lua @@ -5,6 +5,16 @@ bench.runs = 20 bench.extraRuns = 4 function bench.runCode(f, description) + -- Under Callgrind, run the test only once and measure just the execution cost + if callgrind and callgrind("running") then + if collectgarbage then collectgarbage() end + + callgrind("zero") + f() -- unfortunately we can't easily separate setup cost from runtime cost in f unless it calls callgrind() + callgrind("dump", description) + return + end + local timeTable = {} for i = 1,bench.runs + bench.extraRuns do From 2460e389989558f1db89eceb15c9267a4df58704 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Tue, 5 Jul 2022 08:23:09 -0700 Subject: [PATCH 300/399] bench: Implement luau-analyze and luau --compile benchmarks (#575) This change adds another file for benchmarking luau-analyze and sets up benchmarks for both non-strict/strict modes for analysis and all three optimization levels for compilation performance. To avoid issues with race conditions on repository update we do all this in the same job in benchmark.yml. To be able to benchmark both modes from a single file, luau-analyze gains --mode argument which allows to override the default typechecking mode. Not sure if we'll want this to be a hard override on top of the module-specified mode in the future, but this works for now. --- .github/workflows/benchmark-dev.yml | 6 +- .github/workflows/benchmark.yml | 52 +- CLI/Analyze.cpp | 18 +- .../LuauPolyfillMap.lua | 1 - bench/other/regex.lua | 2089 +++++++++++++++++ 5 files changed, 2152 insertions(+), 14 deletions(-) rename bench/{static_analysis => other}/LuauPolyfillMap.lua (99%) create mode 100644 bench/other/regex.lua diff --git a/.github/workflows/benchmark-dev.yml b/.github/workflows/benchmark-dev.yml index 21f9559e..03ff9d6d 100644 --- a/.github/workflows/benchmark-dev.yml +++ b/.github/workflows/benchmark-dev.yml @@ -220,13 +220,13 @@ jobs: sudo apt-get install valgrind - name: Run Luau Analyze on static file - run: sudo python ./bench/measure_time.py ./build/release/luau-analyze bench/static_analysis/LuauPolyfillMap.lua | tee ${{ matrix.bench.script }}-output.txt + run: sudo python ./bench/measure_time.py ./build/release/luau-analyze bench/other/LuauPolyfillMap.lua | tee ${{ matrix.bench.script }}-output.txt - name: Run ${{ matrix.bench.title }} (Cold Cachegrind) - run: sudo ./scripts/run-with-cachegrind.sh python ./bench/measure_time.py "${{ matrix.bench.cachegrindTitle}}Cold" 1 ./build/release/luau-analyze bench/static_analysis/LuauPolyfillMap.lua | tee -a ${{ matrix.bench.script }}-output.txt + run: sudo ./scripts/run-with-cachegrind.sh python ./bench/measure_time.py "${{ matrix.bench.cachegrindTitle}}Cold" 1 ./build/release/luau-analyze bench/other/LuauPolyfillMap.lua | tee -a ${{ matrix.bench.script }}-output.txt - name: Run ${{ matrix.bench.title }} (Warm Cachegrind) - run: sudo bash ./scripts/run-with-cachegrind.sh python ./bench/measure_time.py "${{ matrix.bench.cachegrindTitle}}" 1 ./build/release/luau-analyze bench/static_analysis/LuauPolyfillMap.lua | tee -a ${{ matrix.bench.script }}-output.txt + run: sudo bash ./scripts/run-with-cachegrind.sh python ./bench/measure_time.py "${{ matrix.bench.cachegrindTitle}}" 1 ./build/release/luau-analyze bench/other/LuauPolyfillMap.lua | tee -a ${{ matrix.bench.script }}-output.txt - name: Checkout Benchmark Results repository uses: actions/checkout@v3 diff --git a/.github/workflows/benchmark.yml b/.github/workflows/benchmark.yml index 48296823..9d26186e 100644 --- a/.github/workflows/benchmark.yml +++ b/.github/workflows/benchmark.yml @@ -32,11 +32,33 @@ jobs: sudo apt-get install valgrind - name: Build Luau - run: CXX=${{ matrix.compiler }} make config=release CALLGRIND=1 luau + run: CXX=${{ matrix.compiler }} make config=release CALLGRIND=1 luau luau-analyze - - name: Run benchmark + - name: Run benchmark (bench) run: | - python bench/bench.py --callgrind --vm "./luau -O2" | tee output.txt + python bench/bench.py --callgrind --vm "./luau -O2" | tee -a bench-output.txt + + - name: Run benchmark (analyze) + run: | + filter() { + awk '/.*I\s+refs:\s+[0-9,]+/ {gsub(",", "", $4); X=$4} END {print "SUCCESS: '$1' : " X/1e7 "ms +/- 0% on luau-analyze"}' + } + valgrind --tool=callgrind ./luau-analyze --mode=nonstrict bench/other/LuauPolyfillMap.lua 2>&1 | filter map-nonstrict | tee -a analyze-output.txt + valgrind --tool=callgrind ./luau-analyze --mode=strict bench/other/LuauPolyfillMap.lua 2>&1 | filter map-strict | tee -a analyze-output.txt + valgrind --tool=callgrind ./luau-analyze --mode=nonstrict bench/other/regex.lua 2>&1 | filter regex-nonstrict | tee -a analyze-output.txt + valgrind --tool=callgrind ./luau-analyze --mode=strict bench/other/regex.lua 2>&1 | filter regex-strict | tee -a analyze-output.txt + + - name: Run benchmark (compile) + run: | + filter() { + awk '/.*I\s+refs:\s+[0-9,]+/ {gsub(",", "", $4); X=$4} END {print "SUCCESS: '$1' : " X/1e7 "ms +/- 0% on luau --compile"}' + } + valgrind --tool=callgrind ./luau --compile=null -O0 bench/other/LuauPolyfillMap.lua 2>&1 | filter map-O0 | tee -a compile-output.txt + valgrind --tool=callgrind ./luau --compile=null -O1 bench/other/LuauPolyfillMap.lua 2>&1 | filter map-O1 | tee -a compile-output.txt + valgrind --tool=callgrind ./luau --compile=null -O2 bench/other/LuauPolyfillMap.lua 2>&1 | filter map-O2 | tee -a compile-output.txt + valgrind --tool=callgrind ./luau --compile=null -O0 bench/other/regex.lua 2>&1 | filter regex-O0 | tee -a compile-output.txt + valgrind --tool=callgrind ./luau --compile=null -O1 bench/other/regex.lua 2>&1 | filter regex-O1 | tee -a compile-output.txt + valgrind --tool=callgrind ./luau --compile=null -O2 bench/other/regex.lua 2>&1 | filter regex-O2 | tee -a compile-output.txt - name: Checkout benchmark results uses: actions/checkout@v3 @@ -46,13 +68,29 @@ jobs: token: ${{ secrets.BENCH_GITHUB_TOKEN }} path: "./gh-pages" - - name: Store results + - name: Store results (bench) uses: Roblox/rhysd-github-action-benchmark@v-luau with: name: callgrind ${{ matrix.compiler }} tool: "benchmarkluau" - output-file-path: ./output.txt - external-data-json-path: ./gh-pages/bench/data.json + output-file-path: ./bench-output.txt + external-data-json-path: ./gh-pages/bench.json + + - name: Store results (analyze) + uses: Roblox/rhysd-github-action-benchmark@v-luau + with: + name: luau-analyze + tool: "benchmarkluau" + output-file-path: ./analyze-output.txt + external-data-json-path: ./gh-pages/analyze.json + + - name: Store results (compile) + uses: Roblox/rhysd-github-action-benchmark@v-luau + with: + name: luau --compile + tool: "benchmarkluau" + output-file-path: ./compile-output.txt + external-data-json-path: ./gh-pages/compile.json - name: Push benchmark results if: github.event_name == 'push' @@ -61,7 +99,7 @@ jobs: cd gh-pages git config user.name github-actions git config user.email github@users.noreply.github.com - git add ./bench/data.json + git add *.json git commit -m "Add benchmarks results for ${{ github.sha }}" git push cd .. diff --git a/CLI/Analyze.cpp b/CLI/Analyze.cpp index 81db7c35..f07f9a0b 100644 --- a/CLI/Analyze.cpp +++ b/CLI/Analyze.cpp @@ -8,6 +8,10 @@ #include "FileUtils.h" +#ifdef CALLGRIND +#include +#endif + LUAU_FASTFLAG(DebugLuauTimeTracing) LUAU_FASTFLAG(LuauTypeMismatchModuleNameResolution) @@ -112,6 +116,7 @@ static void displayHelp(const char* argv0) printf("Available options:\n"); printf(" --formatter=plain: report analysis errors in Luacheck-compatible format\n"); printf(" --formatter=gnu: report analysis errors in GNU-compatible format\n"); + printf(" --mode=strict: default to strict mode when typechecking\n"); printf(" --timetrace: record compiler time tracing information into trace.json\n"); } @@ -178,9 +183,9 @@ struct CliConfigResolver : Luau::ConfigResolver mutable std::unordered_map configCache; mutable std::vector> configErrors; - CliConfigResolver() + CliConfigResolver(Luau::Mode mode) { - defaultConfig.mode = Luau::Mode::Nonstrict; + defaultConfig.mode = mode; } const Luau::Config& getConfig(const Luau::ModuleName& name) const override @@ -229,6 +234,7 @@ int main(int argc, char** argv) } ReportFormat format = ReportFormat::Default; + Luau::Mode mode = Luau::Mode::Nonstrict; bool annotate = false; for (int i = 1; i < argc; ++i) @@ -240,6 +246,8 @@ int main(int argc, char** argv) format = ReportFormat::Luacheck; else if (strcmp(argv[i], "--formatter=gnu") == 0) format = ReportFormat::Gnu; + else if (strcmp(argv[i], "--mode=strict") == 0) + mode = Luau::Mode::Strict; else if (strcmp(argv[i], "--annotate") == 0) annotate = true; else if (strcmp(argv[i], "--timetrace") == 0) @@ -258,12 +266,16 @@ int main(int argc, char** argv) frontendOptions.retainFullTypeGraphs = annotate; CliFileResolver fileResolver; - CliConfigResolver configResolver; + CliConfigResolver configResolver(mode); Luau::Frontend frontend(&fileResolver, &configResolver, frontendOptions); Luau::registerBuiltinTypes(frontend.typeChecker); Luau::freeze(frontend.typeChecker.globalTypes); +#ifdef CALLGRIND + CALLGRIND_ZERO_STATS; +#endif + std::vector files = getSourceFiles(argc, argv); int failed = 0; diff --git a/bench/static_analysis/LuauPolyfillMap.lua b/bench/other/LuauPolyfillMap.lua similarity index 99% rename from bench/static_analysis/LuauPolyfillMap.lua rename to bench/other/LuauPolyfillMap.lua index 1cfd0181..1f957d47 100644 --- a/bench/static_analysis/LuauPolyfillMap.lua +++ b/bench/other/LuauPolyfillMap.lua @@ -1,5 +1,4 @@ -- This file is part of the Roblox luau-polyfill repository and is licensed under MIT License; see LICENSE.txt for details ---!nonstrict -- #region Array -- Array related local Array = {} diff --git a/bench/other/regex.lua b/bench/other/regex.lua new file mode 100644 index 00000000..270ab3da --- /dev/null +++ b/bench/other/regex.lua @@ -0,0 +1,2089 @@ +--[[ + PCRE2-based RegEx implemention for Luau + Version 1.0.0a2 + BSD 2-Clause Licence + Copyright © 2020 - Blockzez (devforum.roblox.com/u/Blockzez and github.com/Blockzez) + All rights reserved. + + Redistribution and use in source and binary forms, with or without + modification, are permitted provided that the following conditions are met: + + 1. Redistributions of source code must retain the above copyright notice, this + list of conditions and the following disclaimer. + + 2. Redistributions in binary form must reproduce the above copyright notice, + this list of conditions and the following disclaimer in the documentation + and/or other materials provided with the distribution. + + THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" + AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE + DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE + FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR + SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER + CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, + OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +]] +--[[ Settings ]]-- +-- You can change them here +local options = { + -- The maximum cache size for regex so the patterns are cached so it doesn't recompile the pattern + -- The only accepted value are number values >= 0, strings that can be automatically coered to numbers that are >= 0, false and nil + -- Do note that empty regex patterns (comment-only patterns included) are never cached regardless + -- The default is 256 + cacheSize = 256, + + -- A boolean that determines whether this use unicode data + -- If this value evalulates to false, you can remove _unicodechar_category, _scripts and _xuc safely and it'll now error if: + -- - You try to compile a RegEx with unicode flag + -- - You try to use the \p pattern + -- The default is true + unicodeData = false, +}; + +-- +local u_categories = options.unicodeData and require(script:WaitForChild("_unicodechar_category")); +local chr_scripts = options.unicodeData and require(script:WaitForChild("_scripts")); +local xuc_chr = options.unicodeData and require(script:WaitForChild("_xuc")); +local proxy = setmetatable({ }, { __mode = 'k' }); +local re, re_m, match_m = { }, { }, { }; +local lockmsg; + +--[[ Functions ]]-- +local function to_str_arr(self, init) + if init then + self = string.sub(self, utf8.offset(self, init)); + end; + local len = utf8.len(self); + if len <= 1999 then + return { n = len, s = self, utf8.codepoint(self, 1, #self) }; + end; + local clen = math.ceil(len / 1999); + local ret = table.create(len); + local p = 1; + for i = 1, clen do + local c = table.pack(utf8.codepoint(self, utf8.offset(self, i * 1999 - 1998), utf8.offset(self, i * 1999 - (i == clen and 1998 - ((len - 1) % 1999 + 1) or - 1)) - 1)); + table.move(c, 1, c.n, p, ret); + p += c.n; + end; + ret.s, ret.n = self, len; + return ret; +end; + +local function from_str_arr(self) + local len = self.n or #self; + if len <= 7997 then + return utf8.char(table.unpack(self)); + end; + local clen = math.ceil(len / 7997); + local r = table.create(clen); + for i = 1, clen do + r[i] = utf8.char(table.unpack(self, i * 7997 - 7996, i * 7997 - (i == clen and 7997 - ((len - 1) % 7997 + 1) or 0))); + end; + return table.concat(r); +end; + +local function utf8_sub(self, i, j) + j = utf8.offset(self, j); + return string.sub(self, utf8.offset(self, i), j and j - 1); +end; + +-- +local flag_map = { + a = 'anchored', i = 'caseless', m = 'multiline', s = 'dotall', u = 'unicode', U = 'ungreedy', x ='extended', +}; + +local posix_class_names = { + alnum = true, alpha = true, ascii = true, blank = true, cntrl = true, digit = true, graph = true, lower = true, print = true, punct = true, space = true, upper = true, word = true, xdigit = true, +}; + +local escape_chars = { + -- grouped + -- digit, spaces and words + [0x44] = { "class", "digit", true }, [0x53] = { "class", "space", true }, [0x57] = { "class", "word", true }, + [0x64] = { "class", "digit", false }, [0x73] = { "class", "space", false }, [0x77] = { "class", "word", false }, + -- horizontal/vertical whitespace and newline + [0x48] = { "class", "blank", true }, [0x56] = { "class", "vertical_tab", true }, + [0x68] = { "class", "blank", false }, [0x76] = { "class", "vertical_tab", false }, + [0x4E] = { 0x4E }, [0x52] = { 0x52 }, + + -- not grouped + [0x42] = 0x08, + [0x6E] = 0x0A, [0x72] = 0x0D, [0x74] = 0x09, +}; + +local b_escape_chars = { + -- word boundary and not word boundary + [0x62] = { 0x62, { "class", "word", false } }, [0x42] = { 0x42, { "class", "word", false } }, + + -- keep match out + [0x4B] = { 0x4B }, + + -- start & end of string + [0x47] = { 0x47 }, [0x4A] = { 0x4A }, [0x5A] = { 0x5A }, [0x7A] = { 0x7A }, +}; + +local valid_categories = { + C = true, Cc = true, Cf = true, Cn = true, Co = true, Cs = true, + L = true, Ll = true, Lm = true, Lo = true, Lt = true, Lu = true, + M = true, Mc = true, Me = true, Mn = true, + N = true, Nd = true, Nl = true, No = true, + P = true, Pc = true, Pd = true, Pe = true, Pf = true, Pi = true, Po = true, Ps = true, + S = true, Sc = true, Sk = true, Sm = true, So = true, + Z = true, Zl = true, Zp = true, Zs = true, + + Xan = true, Xps = true, Xsp = true, Xuc = true, Xwd = true, +}; + +local class_ascii_punct = { + [0x21] = true, [0x22] = true, [0x23] = true, [0x24] = true, [0x25] = true, [0x26] = true, [0x27] = true, [0x28] = true, [0x29] = true, [0x2A] = true, [0x2B] = true, [0x2C] = true, [0x2D] = true, [0x2E] = true, [0x2F] = true, + [0x3A] = true, [0x3B] = true, [0x3C] = true, [0x3D] = true, [0x3E] = true, [0x3F] = true, [0x40] = true, [0x5B] = true, [0x5C] = true, [0x5D] = true, [0x5E] = true, [0x5F] = true, [0x60] = true, [0x7B] = true, [0x7C] = true, + [0x7D] = true, [0x7E] = true, +}; + +local end_str = { 0x24 }; +local dot = { 0x2E }; +local beginning_str = { 0x5E }; +local alternation = { 0x7C }; + +local function check_re(re_type, name, func) + if re_type == "Match" then + return function(...) + local arg_n = select('#', ...); + if arg_n < 1 then + error("missing argument #1 (Match expected)", 2); + end; + local arg0, arg1 = ...; + if not (proxy[arg0] and proxy[arg0].name == "Match") then + error(string.format("invalid argument #1 to %q (Match expected, got %s)", name, typeof(arg0)), 2); + else + arg0 = proxy[arg0]; + end; + if name == "group" or name == "span" then + if arg1 == nil then + arg1 = 0; + end; + end; + return func(arg0, arg1); + end; + end; + return function(...) + local arg_n = select('#', ...); + if arg_n < 1 then + error("missing argument #1 (RegEx expected)", 2); + elseif arg_n < 2 then + error("missing argument #2 (string expected)", 2); + end; + local arg0, arg1, arg2, arg3, arg4, arg5 = ...; + if not (proxy[arg0] and proxy[arg0].name == "RegEx") then + if type(arg0) ~= "string" and type(arg0) ~= "number" then + error(string.format("invalid argument #1 to %q (RegEx expected, got %s)", name, typeof(arg0)), 2); + end; + arg0 = re.fromstring(arg0); + elseif name == "sub" then + if type(arg2) == "number" then + arg2 ..= ''; + elseif type(arg2) ~= "string" then + error(string.format("invalid argument #3 to 'sub' (string expected, got %s)", typeof(arg2)), 2); + end; + elseif type(arg1) == "number" then + arg1 ..= ''; + elseif type(arg1) ~= "string" then + error(string.format("invalid argument #2 to %q (string expected, got %s)", name, typeof(arg1)), 2); + end; + if name ~= "sub" and name ~= "split" then + local init_type = typeof(arg2); + if init_type ~= 'nil' then + arg2 = tonumber(arg2); + if not arg2 then + error(string.format("invalid argument #3 to %q (number expected, got %s)", name, init_type), 2); + elseif arg2 < 0 then + arg2 = #arg1 + math.floor(arg2 + 0.5) + 1; + else + arg2 = math.max(math.floor(arg2 + 0.5), 1); + end; + end; + end; + arg0 = proxy[arg0]; + if name == "match" or name == "matchiter" then + arg3 = ...; + elseif name == "sub" then + arg5 = ...; + end; + return func(arg0, arg1, arg2, arg3, arg4, arg5); + end; +end; + +--[[ Matches ]]-- +local function match_tostr(self) + local spans = proxy[self].spans; + local s_start, s_end = spans[0][1], spans[0][2]; + if s_end <= s_start then + return string.format("Match (%d..%d, empty)", s_start, s_end - 1); + end; + return string.format("Match (%d..%d): %s", s_start, s_end - 1, utf8_sub(spans.input, s_start, s_end)); +end; + +local function new_match(span_arr, group_id, re, str) + span_arr.source, span_arr.input = re, str; + local object = newproxy(true); + local object_mt = getmetatable(object); + object_mt.__metatable = lockmsg; + object_mt.__index = setmetatable(span_arr, match_m); + object_mt.__tostring = match_tostr; + + proxy[object] = { name = "Match", spans = span_arr, group_id = group_id }; + return object; +end; + +match_m.group = check_re('Match', 'group', function(self, group_id) + local span = self.spans[type(group_id) == "number" and group_id or self.group_id[group_id]]; + if not span then + return nil; + end; + return utf8_sub(self.spans.input, span[1], span[2]); +end); + +match_m.span = check_re('Match', 'span', function(self, group_id) + local span = self.spans[type(group_id) == "number" and group_id or self.group_id[group_id]]; + if not span then + return nil; + end; + return span[1], span[2] - 1; +end); + +match_m.groups = check_re('Match', 'groups', function(self) + local spans = self.spans; + if spans.n > 0 then + local ret = table.create(spans.n); + for i = 0, spans.n do + local v = spans[i]; + if v then + ret[i] = utf8_sub(spans.input, v[1], v[2]); + end; + end; + return table.unpack(ret, 1, spans.n); + end; + return utf8_sub(spans.input, spans[0][1], spans[0][2]); +end); + +match_m.groupdict = check_re('Match', 'groupdict', function(self) + local spans = self.spans; + local ret = { }; + for k, v in pairs(self.group_id) do + v = spans[v]; + if v then + ret[k] = utf8_sub(spans.input, v[1], v[2]); + end; + end; + return ret; +end); + +match_m.grouparr = check_re('Match', 'groupdict', function(self) + local spans = self.spans; + local ret = table.create(spans.n); + for i = 0, spans.n do + local v = spans[i]; + if v then + ret[i] = utf8_sub(spans.input, v[1], v[2]); + end; + end; + ret.n = spans.n; + return ret; +end); + +-- +local line_verbs = { + CR = 0, LF = 1, CRLF = 2, ANYRLF = 3, ANY = 4, NUL = 5, +}; +local function is_newline(str_arr, i, verb_flags) + local line_verb_n = verb_flags.newline; + local chr = str_arr[i]; + if line_verb_n == 0 then + -- carriage return + return chr == 0x0D; + elseif line_verb_n == 2 then + -- carriage return followed by line feed + return chr == 0x0A and str_arr[i - 1] == 0x20; + elseif line_verb_n == 3 then + -- any of the above + return chr == 0x0A or chr == 0x0D; + elseif line_verb_n == 4 then + -- any of Unicode newlines + return chr == 0x0A or chr == 0x0B or chr == 0x0C or chr == 0x0D or chr == 0x85 or chr == 0x2028 or chr == 0x2029; + elseif line_verb_n == 5 then + -- null + return chr == 0; + end; + -- linefeed + return chr == 0x0A; +end; + + +local function tkn_char_match(tkn_part, str_arr, i, flags, verb_flags) + local chr = str_arr[i]; + if not chr then + return false; + elseif flags.ignoreCase and chr >= 0x61 and chr <= 0x7A then + chr -= 0x20; + end; + if type(tkn_part) == "number" then + return tkn_part == chr; + elseif tkn_part[1] == "charset" then + for _, v in ipairs(tkn_part[3]) do + if tkn_char_match(v, str_arr, i, flags, verb_flags) then + return not tkn_part[2]; + end; + end; + return tkn_part[2]; + elseif tkn_part[1] == "range" then + return chr >= tkn_part[2] and chr <= tkn_part[3] or flags.ignoreCase and chr >= 0x41 and chr <= 0x5A and (chr + 0x20) >= tkn_part[2] and (chr + 0x20) <= tkn_part[3]; + elseif tkn_part[1] == "class" then + local char_class = tkn_part[2]; + local negate = tkn_part[3]; + local match = false; + -- if and elseifs :( + -- Might make these into tables in the future + if char_class == "xdigit" then + match = chr >= 0x30 and chr <= 0x39 or chr >= 0x41 and chr <= 0x46 or chr >= 0x61 and chr <= 0x66; + elseif char_class == "ascii" then + match = chr <= 0x7F; + -- cannot be accessed through POSIX classes + elseif char_class == "vertical_tab" then + match = chr >= 0x0A and chr <= 0x0D or chr == 0x2028 or chr == 0x2029; + -- + elseif flags.unicode then + local current_category = u_categories[chr] or 'Cn'; + local first_category = current_category:sub(1, 1); + if char_class == "alnum" then + match = first_category == 'L' or current_category == 'Nl' or current_category == 'Nd'; + elseif char_class == "alpha" then + match = first_category == 'L' or current_category == 'Nl'; + elseif char_class == "blank" then + match = current_category == 'Zs' or chr == 0x09; + elseif char_class == "cntrl" then + match = current_category == 'Cc'; + elseif char_class == "digit" then + match = current_category == 'Nd'; + elseif char_class == "graph" then + match = first_category ~= 'P' and first_category ~= 'C'; + elseif char_class == "lower" then + match = current_category == 'Ll'; + elseif char_class == "print" then + match = first_category ~= 'C'; + elseif char_class == "punct" then + match = first_category == 'P'; + elseif char_class == "space" then + match = first_category == 'Z' or chr >= 0x09 and chr <= 0x0D; + elseif char_class == "upper" then + match = current_category == 'Lu'; + elseif char_class == "word" then + match = first_category == 'L' or current_category == 'Nl' or current_category == 'Nd' or current_category == 'Pc'; + end; + elseif char_class == "alnum" then + match = chr >= 0x30 and chr <= 0x39 or chr >= 0x41 and chr <= 0x5A or chr >= 0x61 and chr <= 0x7A; + elseif char_class == "alpha" then + match = chr >= 0x41 and chr <= 0x5A or chr >= 0x61 and chr <= 0x7A; + elseif char_class == "blank" then + match = chr == 0x09 or chr == 0x20; + elseif char_class == "cntrl" then + match = chr <= 0x1F or chr == 0x7F; + elseif char_class == "digit" then + match = chr >= 0x30 and chr <= 0x39; + elseif char_class == "graph" then + match = chr >= 0x21 and chr <= 0x7E; + elseif char_class == "lower" then + match = chr >= 0x61 and chr <= 0x7A; + elseif char_class == "print" then + match = chr >= 0x20 and chr <= 0x7E; + elseif char_class == "punct" then + match = class_ascii_punct[chr]; + elseif char_class == "space" then + match = chr >= 0x09 and chr <= 0x0D or chr == 0x20; + elseif char_class == "upper" then + match = chr >= 0x41 and chr <= 0x5A; + elseif char_class == "word" then + match = chr >= 0x30 and chr <= 0x39 or chr >= 0x41 and chr <= 0x5A or chr >= 0x61 and chr <= 0x7A or chr == 0x5F; + end; + if negate then + return not match; + end; + return match; + elseif tkn_part[1] == "category" then + local chr_category = u_categories[chr] or 'Cn'; + local category_v = tkn_part[3]; + local category_len = #category_v; + if category_len == 3 then + local match = false; + if category_v == "Xan" or category_v == "Xwd" then + match = chr_category:find("^[LN]") or category_v == "Xwd" and chr == 0x5F; + elseif category_v == "Xps" or category_v == "Xsp" then + match = chr_category:sub(1, 1) == 'Z' or chr >= 0x09 and chr <= 0x0D; + elseif category_v == "Xuc" then + match = tkn_char_match(xuc_chr, str_arr, i, flags, verb_flags); + end; + if tkn_part[2] then + return not match; + end + return match; + elseif chr_category:sub(1, category_len) == category_v then + return not tkn_part[2]; + end; + return tkn_part[2]; + elseif tkn_part[1] == 0x2E then + return flags.dotAll or not is_newline(str_arr, i, verb_flags); + elseif tkn_part[1] == 0x4E then + return not is_newline(str_arr, i, verb_flags); + elseif tkn_part[1] == 0x52 then + if verb_flags.newline_seq == 0 then + -- CR, LF or CRLF + return chr == 0x0A or chr == 0x0D; + end; + -- any unicode newline + return chr == 0x0A or chr == 0x0B or chr == 0x0C or chr == 0x0D or chr == 0x85 or chr == 0x2028 or chr == 0x2029; + end; + return false; +end; + +local function find_alternation(token, i, count) + while true do + local v = token[i]; + local is_table = type(v) == "table"; + if v == alternation then + return i, count; + elseif is_table and v[1] == 0x28 then + if count then + count += v.count; + end; + i = v[3]; + elseif is_table and v[1] == "quantifier" and type(v[5]) == "table" and v[5][1] == 0x28 then + if count then + count += v[5].count; + end; + i = v[5][3]; + elseif not v or is_table and v[1] == 0x29 then + return nil, count; + elseif count then + if is_table and v[1] == "quantifier" then + count += v[3]; + else + count += 1; + end; + end; + i += 1; + end; +end; + +local function re_rawfind(token, str_arr, init, flags, verb_flags, as_bool) + local tkn_i, str_i, start_i = 0, init, init; + local states = { }; + while tkn_i do + if tkn_i == 0 then + tkn_i += 1; + local next_alt = find_alternation(token, tkn_i); + if next_alt then + table.insert(states, 1, { "alternation", next_alt, str_i }); + end; + continue; + end; + local ctkn = token[tkn_i]; + local tkn_type = type(ctkn) == "table" and ctkn[1]; + if not ctkn then + break; + elseif ctkn == "ACCEPT" then + local not_lookaround = true; + local close_i = tkn_i; + repeat + close_i += 1; + local is_table = type(token[close_i]) == "table"; + local close_i_tkn = token[close_i]; + if is_table and (close_i_tkn[1] == 0x28 or close_i_tkn[1] == "quantifier" and type(close_i_tkn[5]) == "table" and close_i_tkn[5][1] == 0x28) then + close_i = close_i_tkn[1] == "quantifier" and close_i_tkn[5][3] or close_i_tkn[3]; + elseif is_table and close_i_tkn[1] == 0x29 and (close_i_tkn[4] == 0x21 or close_i_tkn[4] == 0x3D) then + not_lookaround = false; + tkn_i = close_i; + break; + end; + until not close_i_tkn; + if not_lookaround then + break; + end; + elseif ctkn == "PRUNE" or ctkn == "SKIP" then + table.insert(states, 1, { ctkn, str_i }); + tkn_i += 1; + elseif tkn_type == 0x28 then + table.insert(states, 1, { "group", tkn_i, str_i, nil, ctkn[2], ctkn[3], ctkn[4] }); + tkn_i += 1; + local next_alt, count = find_alternation(token, tkn_i, (ctkn[4] == 0x21 or ctkn[4] == 0x3D) and ctkn[5] and 0); + if next_alt then + table.insert(states, 1, { "alternation", next_alt, str_i }); + end; + if count then + str_i -= count; + end; + elseif tkn_type == 0x29 and ctkn[4] ~= 0x21 then + if ctkn[4] == 0x21 or ctkn[4] == 0x3D then + while true do + local selected_match_start; + local selected_state = table.remove(states, 1); + if selected_state[1] == "group" and selected_state[2] == ctkn[3] then + if (ctkn[4] == 0x21 or ctkn[4] == 0x3D) and not ctkn[5] then + str_i = selected_state[3]; + end; + if selected_match_start then + table.insert(states, 1, selected_match_start); + end; + break; + elseif selected_state[1] == "matchStart" and not selected_match_start and ctkn[4] == 0x3D then + selected_match_start = selected_state; + end; + end; + elseif ctkn[4] == 0x3E then + repeat + local selected_state = table.remove(states, 1); + until not selected_state or selected_state[1] == "group" and selected_state[2] == ctkn[3]; + else + for i, v in ipairs(states) do + if v[1] == "group" and v[2] == ctkn[3] then + if v.jmp then + -- recursive match + tkn_i = v.jmp; + end; + v[4] = str_i; + if v[7] == "quantifier" and v[10] + 1 < v[9] then + if token[ctkn[3]][4] ~= "lazy" or v[10] + 1 < v[8] then + tkn_i = ctkn[3]; + end; + local ctkn1 = token[ctkn[3]]; + local new_group = { "group", v[2], str_i, nil, ctkn1[5][2], ctkn1[5][3], "quantifier", ctkn1[2], ctkn1[3], v[10] + 1, v[11], ctkn1[4] }; + table.insert(states, 1, new_group); + if v[11] then + table.insert(states, 1, { "alternation", v[11], str_i }); + end; + end; + break; + end; + end; + end; + tkn_i += 1; + elseif tkn_type == 0x4B then + table.insert(states, 1, { "matchStart", str_i }); + tkn_i += 1; + elseif tkn_type == 0x7C then + local close_i = tkn_i; + repeat + close_i += 1; + local is_table = type(token[close_i]) == "table"; + local close_i_tkn = token[close_i]; + if is_table and (close_i_tkn[1] == 0x28 or close_i_tkn[1] == "quantifier" and type(close_i_tkn[5]) == "table" and close_i_tkn[5][1] == 0x28) then + close_i = close_i_tkn[1] == "quantifier" and close_i_tkn[5][3] or close_i_tkn[3]; + end; + until is_table and close_i_tkn[1] == 0x29 or not close_i_tkn; + if token[close_i] then + for _, v in ipairs(states) do + if v[1] == "group" and v[6] == close_i then + tkn_i = v[6]; + break; + end; + end; + else + tkn_i = close_i; + end; + elseif tkn_type == "recurmatch" then + table.insert(states, 1, { "group", ctkn[3], str_i, nil, nil, token[ctkn[3]][3], nil, jmp = tkn_i }); + tkn_i = ctkn[3] + 1; + local next_alt, count = find_alternation(token, tkn_i); + if next_alt then + table.insert(states, 1, { "alternation", next_alt, str_i }); + end; + else + local match; + if ctkn == "FAIL" then + match = false; + elseif tkn_type == 0x29 then + repeat + local selected_state = table.remove(states, 1); + until selected_state[1] == "group" and selected_state[2] == ctkn[3]; + elseif tkn_type == "quantifier" then + if type(ctkn[5]) == "table" and ctkn[5][1] == 0x28 then + local next_alt = find_alternation(token, tkn_i + 1); + if next_alt then + table.insert(states, 1, { "alternation", next_alt, str_i }); + end; + table.insert(states, next_alt and 2 or 1, { "group", tkn_i, str_i, nil, ctkn[5][2], ctkn[5][3], "quantifier", ctkn[2], ctkn[3], 0, next_alt, ctkn[4] }); + if ctkn[4] == "lazy" and ctkn[2] == 0 then + tkn_i = ctkn[5][3]; + end; + match = true; + else + local start_i, end_i; + local pattern_count = 1; + local is_backref = type(ctkn[5]) == "table" and ctkn[5][1] == "backref"; + if is_backref then + pattern_count = 0; + local group_n = ctkn[5][2]; + for _, v in ipairs(states) do + if v[1] == "group" and v[5] == group_n then + start_i, end_i = v[3], v[4]; + pattern_count = end_i - start_i; + break; + end; + end; + end; + local min_max_i = str_i + ctkn[2] * pattern_count; + local mcount = 0; + while mcount < ctkn[3] do + if is_backref then + if start_i and end_i then + local org_i = str_i; + if utf8_sub(str_arr.s, start_i, end_i) ~= utf8_sub(str_arr.s, org_i, str_i + pattern_count) then + break; + end; + else + break; + end; + elseif not tkn_char_match(ctkn[5], str_arr, str_i, flags, verb_flags) then + break; + end; + str_i += pattern_count; + mcount += 1; + end; + match = mcount >= ctkn[2]; + if match and ctkn[4] ~= "possessive" then + if ctkn[4] == "lazy" then + min_max_i, str_i = str_i, min_max_i; + end; + table.insert(states, 1, { "quantifier", tkn_i, str_i, math.min(min_max_i, str_arr.n + 1), (ctkn[4] == "lazy" and 1 or -1) * pattern_count }); + end; + end; + elseif tkn_type == "backref" then + local start_i, end_i; + local group_n = ctkn[2]; + for _, v in ipairs(states) do + if v[1] == "group" and v[5] == group_n then + start_i, end_i = v[3], v[4]; + break; + end; + end; + if start_i and end_i then + local org_i = str_i; + str_i += end_i - start_i; + match = utf8_sub(str_arr.s, start_i, end_i) == utf8_sub(str_arr.s, org_i, str_i); + end; + else + local chr = str_arr[str_i]; + if tkn_type == 0x24 or tkn_type == 0x5A or tkn_type == 0x7A then + match = str_i == str_arr.n + 1 or tkn_type == 0x24 and flags.multiline and is_newline(str_arr, str_i + 1, verb_flags) or tkn_type == 0x5A and str_i == str_arr.n and is_newline(str_arr, str_i, verb_flags); + elseif tkn_type == 0x5E or tkn_type == 0x41 or tkn_type == 0x47 then + match = str_i == 1 or tkn_type == 0x5E and flags.multiline and is_newline(str_arr, str_i - 1, verb_flags) or tkn_type == 0x47 and str_i == init; + elseif tkn_type == 0x42 or tkn_type == 0x62 then + local start_m = str_i == 1 or flags.multiline and is_newline(str_arr, str_i - 1, verb_flags); + local end_m = str_i == str_arr.n + 1 or flags.multiline and is_newline(str_arr, str_i, verb_flags); + local w_m = tkn_char_match(ctkn[2], str_arr[str_i - 1], flags) and 0 or tkn_char_match(ctkn[2], chr, flags) and 1; + if w_m == 0 then + match = end_m or not tkn_char_match(ctkn[2], chr, flags); + elseif w_m then + match = start_m or not tkn_char_match(ctkn[2], str_arr[str_i - 1], flags); + end; + if tkn_type == 0x42 then + match = not match; + end; + else + match = tkn_char_match(ctkn, str_arr, str_i, flags, verb_flags); + str_i += 1; + end; + end; + if not match then + while true do + local prev_type, prev_state = states[1] and states[1][1], states[1]; + if not prev_type or prev_type == "PRUNE" or prev_type == "SKIP" then + if prev_type then + table.clear(states); + end; + if start_i > str_arr.n then + if as_bool then + return false; + end; + return nil; + end; + start_i = prev_type == "SKIP" and prev_state[2] or start_i + 1; + tkn_i, str_i = 0, start_i; + break; + elseif prev_type == "alternation" then + tkn_i, str_i = prev_state[2], prev_state[3]; + local next_alt, count = find_alternation(token, tkn_i + 1); + if next_alt then + prev_state[2] = next_alt; + else + table.remove(states, 1); + end; + if count then + str_i -= count; + end; + break; + elseif prev_type == "group" then + if prev_state[7] == "quantifier" then + if prev_state[12] == "greedy" and prev_state[10] >= prev_state[8] + or prev_state[12] == "lazy" and prev_state[10] < prev_state[9] and not prev_state[13] then + tkn_i, str_i = prev_state[12] == "greedy" and prev_state[6] or prev_state[2], prev_state[3]; + if prev_state[12] == "greedy" then + table.remove(states, 1); + break; + elseif prev_state[10] >= prev_state[8] then + prev_state[13] = true; + break; + end; + end; + elseif prev_state[7] == 0x21 then + table.remove(states, 1); + tkn_i, str_i = prev_state[6], prev_state[3]; + break; + end; + elseif prev_type == "quantifier" then + if math.sign(prev_state[4] - prev_state[3]) == math.sign(prev_state[5]) then + prev_state[3] += prev_state[5]; + tkn_i, str_i = prev_state[2], prev_state[3]; + break; + end; + end; + -- keep match out state and recursive state, can be safely removed + -- prevents infinite loop + table.remove(states, 1); + end; + end; + tkn_i += 1; + end; + end; + if as_bool then + return true; + end; + local match_start_ran = false; + local span = table.create(token.group_n); + span[0], span.n = { start_i, str_i }, token.group_n; + for _, v in ipairs(states) do + if v[1] == "matchStart" and not match_start_ran then + span[0][1], match_start_ran = v[2], true; + elseif v[1] == "group" and v[5] and not span[v[5]] then + span[v[5]] = { v[3], v[4] }; + end; + end; + return span; +end; + +--[[ Methods ]]-- +re_m.test = check_re('RegEx', 'test', function(self, str, init) + return re_rawfind(self.token, to_str_arr(str, init), 1, self.flags, self.verb_flags, true); +end); + +re_m.match = check_re('RegEx', 'match', function(self, str, init, source) + local span = re_rawfind(self.token, to_str_arr(str, init), 1, self.flags, self.verb_flags, false); + if not span then + return nil; + end; + return new_match(span, self.group_id, source, str); +end); + +re_m.matchall = check_re('RegEx', 'matchall', function(self, str, init, source) + str = to_str_arr(str, init); + local i = 1; + return function() + local span = i <= str.n + 1 and re_rawfind(self.token, str, i, self.flags, self.verb_flags, false); + if not span then + return nil; + end; + i = span[0][2] + (span[0][1] >= span[0][2] and 1 or 0); + return new_match(span, self.group_id, source, str.s); + end; +end); + +local function insert_tokenized_sub(repl_r, str, span, tkn) + for _, v in ipairs(tkn) do + if type(v) == "table" then + if v[1] == "condition" then + if span[v[2]] then + if v[3] then + insert_tokenized_sub(repl_r, str, span, v[3]); + else + table.move(str, span[v[2]][1], span[v[2]][2] - 1, #repl_r + 1, repl_r); + end; + elseif v[4] then + insert_tokenized_sub(repl_r, str, span, v[4]); + end; + else + table.move(v, 1, #v, #repl_r + 1, repl_r); + end; + elseif span[v] then + table.move(str, span[v][1], span[v][2] - 1, #repl_r + 1, repl_r); + end; + end; + repl_r.n = #repl_r; + return repl_r; +end; + +re_m.sub = check_re('RegEx', 'sub', function(self, repl, str, n, repl_flag_str, source) + if repl_flag_str ~= nil and type(repl_flag_str) ~= "number" and type(repl_flag_str) ~= "string" then + error(string.format("invalid argument #5 to 'sub' (string expected, got %s)", typeof(repl_flag_str)), 3); + end + local repl_flags = { + l = false, o = false, u = false, + }; + for f in string.gmatch(repl_flag_str or '', utf8.charpattern) do + if repl_flags[f] ~= false then + error("invalid regular expression substitution flag " .. f, 3); + end; + repl_flags[f] = true; + end; + local repl_type = type(repl); + if repl_type == "number" then + repl ..= ''; + elseif repl_type ~= "string" and repl_type ~= "function" and (not repl_flags.o or repl_type ~= "table") then + error(string.format("invalid argument #2 to 'sub' (string/function%s expected, got %s)", repl_flags.o and "/table" or '', typeof(repl)), 3); + end; + if tonumber(n) then + n = tonumber(n); + if n <= -1 or n ~= n then + n = math.huge; + end; + elseif n ~= nil then + error(string.format("invalid argument #4 to 'sub' (number expected, got %s)", typeof(n)), 3); + else + n = math.huge; + end; + if n < 1 then + return str, 0; + end; + local min_repl_n = 0; + if repl_type == "string" then + repl = to_str_arr(repl); + if not repl_flags.l then + local i1 = 0; + local repl_r = table.create(3); + local group_n = self.token.group_n; + local conditional_c = { }; + while i1 < repl.n do + local i2 = i1; + repeat + i2 += 1; + until not repl[i2] or repl[i2] == 0x24 or repl[i2] == 0x5C or (repl[i2] == 0x3A or repl[i2] == 0x7D) and conditional_c[1]; + min_repl_n += i2 - i1 - 1; + if i2 - i1 > 1 then + table.insert(repl_r, table.move(repl, i1 + 1, i2 - 1, 1, table.create(i2 - i1 - 1))); + end; + if repl[i2] == 0x3A then + local current_conditional_c = conditional_c[1]; + if current_conditional_c[2] then + error("malformed substitution pattern", 3); + end; + current_conditional_c[2] = table.move(repl_r, current_conditional_c[3], #repl_r, 1, table.create(#repl_r + 1 - current_conditional_c[3])); + for i3 = #repl_r, current_conditional_c[3], -1 do + repl_r[i3] = nil; + end; + elseif repl[i2] == 0x7D then + local current_conditional_c = table.remove(conditional_c, 1); + local second_c = table.move(repl_r, current_conditional_c[3], #repl_r, 1, table.create(#repl_r + 1 - current_conditional_c[3])); + for i3 = #repl_r, current_conditional_c[3], -1 do + repl_r[i3] = nil; + end; + table.insert(repl_r, { "condition", current_conditional_c[1], current_conditional_c[2] ~= true and (current_conditional_c[2] or second_c), current_conditional_c[2] and second_c }); + elseif repl[i2] then + i2 += 1; + local subst_c = repl[i2]; + if not subst_c then + if repl[i2 - 1] == 0x5C then + error("replacement string must not end with a trailing backslash", 3); + end; + local prev_repl_f = repl_r[#repl_r]; + if type(prev_repl_f) == "table" then + table.insert(prev_repl_f, repl[i2 - 1]); + else + table.insert(repl_r, { repl[i2 - 1] }); + end; + elseif subst_c == 0x5C and repl[i2 - 1] == 0x24 then + local prev_repl_f = repl_r[#repl_r]; + if type(prev_repl_f) == "table" then + table.insert(prev_repl_f, 0x24); + else + table.insert(repl_r, { 0x24 }); + end; + i2 -= 1; + min_repl_n += 1; + elseif subst_c == 0x30 then + table.insert(repl_r, 0); + elseif subst_c > 0x30 and subst_c <= 0x39 then + local start_i2 = i2; + local group_i = subst_c - 0x30; + while repl[i2 + 1] and repl[i2 + 1] >= 0x30 and repl[i2 + 1] <= 0x39 do + group_i ..= repl[i2 + 1] - 0x30; + i2 += 1; + end; + group_i = tonumber(group_i); + if not repl_flags.u and group_i > group_n then + error("reference to non-existent subpattern", 3); + end; + table.insert(repl_r, group_i); + elseif subst_c == 0x7B and repl[i2 - 1] == 0x24 then + i2 += 1; + local start_i2 = i2; + while repl[i2] and + (repl[i2] >= 0x30 and repl[i2] <= 0x39 + or repl[i2] >= 0x41 and repl[i2] <= 0x5A + or repl[i2] >= 0x61 and repl[i2] <= 0x7A + or repl[i2] == 0x5F) do + i2 += 1; + end; + if (repl[i2] == 0x7D or repl[i2] == 0x3A and (repl[i2 + 1] == 0x2B or repl[i2 + 1] == 0x2D)) and i2 ~= start_i2 then + local group_k = utf8_sub(repl.s, start_i2, i2); + if repl[start_i2] >= 0x30 and repl[start_i2] <= 0x39 then + group_k = tonumber(group_k); + if not repl_flags.u and group_k > group_n then + error("reference to non-existent subpattern", 3); + end; + else + group_k = self.group_id[group_k]; + if not repl_flags.u and (not group_k or group_k > group_n) then + error("reference to non-existent subpattern", 3); + end; + end; + if repl[i2] == 0x3A then + i2 += 1; + table.insert(conditional_c, { group_k, repl[i2] == 0x2D, #repl_r + 1 }); + else + table.insert(repl_r, group_k); + end; + else + error("malformed substitution pattern", 3); + end; + else + local c_escape_char; + if repl[i2 - 1] == 0x24 then + if subst_c ~= 0x24 then + local prev_repl_f = repl_r[#repl_r]; + if type(prev_repl_f) == "table" then + table.insert(prev_repl_f, 0x24); + else + table.insert(repl_r, { 0x24 }); + end; + end; + else + c_escape_char = escape_chars[repl[i2]]; + if type(c_escape_char) ~= "number" then + c_escape_char = nil; + end; + end; + local prev_repl_f = repl_r[#repl_r]; + if type(prev_repl_f) == "table" then + table.insert(prev_repl_f, c_escape_char or repl[i2]); + else + table.insert(repl_r, { c_escape_char or repl[i2] }); + end; + min_repl_n += 1; + end; + end; + i1 = i2; + end; + if conditional_c[1] then + error("malformed substitution pattern", 3); + end; + if not repl_r[2] and type(repl_r[1]) == "table" and repl_r[1][1] ~= "condition" then + repl, repl.n = repl_r[1], #repl_r[1]; + else + repl, repl_type = repl_r, "subst_string"; + end; + end; + end; + str = to_str_arr(str); + local incr, i0, count = 0, 1, 0; + while i0 <= str.n + incr + 1 do + local span = re_rawfind(self.token, str, i0, self.flags, self.verb_flags, false); + if not span then + break; + end; + local repl_r; + if repl_type == "string" then + repl_r = repl; + elseif repl_type == "subst_string" then + repl_r = insert_tokenized_sub(table.create(min_repl_n), str, span, repl); + else + local re_match; + local repl_c; + if repl_type == "table" then + re_match = utf8_sub(str.s, span[0][1], span[0][2]); + repl_c = repl[re_match]; + else + re_match = new_match(span, self.group_id, source, str.s); + repl_c = repl(re_match); + end; + if repl_c == re_match or repl_flags.o and not repl_c then + local repl_n = span[0][2] - span[0][1]; + repl_r = table.move(str, span[0][1], span[0][2] - 1, 1, table.create(repl_n)); + repl_r.n = repl_n; + elseif type(repl_c) == "string" then + repl_r = to_str_arr(repl_c); + elseif type(repl_c) == "number" then + repl_r = to_str_arr(repl_c .. ''); + elseif repl_flags.o then + error(string.format("invalid replacement value (a %s)", type(repl_c)), 3); + else + repl_r = { n = 0 }; + end; + end; + local match_len = span[0][2] - span[0][1]; + local repl_len = math.min(repl_r.n, match_len); + for i1 = 0, repl_len - 1 do + str[span[0][1] + i1] = repl_r[i1 + 1]; + end; + local i1 = span[0][1] + repl_len; + i0 = span[0][2]; + if match_len > repl_r.n then + for i2 = 1, match_len - repl_r.n do + table.remove(str, i1); + incr -= 1; + i0 -= 1; + end; + elseif repl_r.n > match_len then + for i2 = 1, repl_r.n - match_len do + table.insert(str, i1 + i2 - 1, repl_r[repl_len + i2]); + incr += 1; + i0 += 1; + end; + end; + if match_len <= 0 then + i0 += 1; + end; + count += 1; + if n < count + 1 then + break; + end; + end; + return from_str_arr(str), count; +end); + +re_m.split = check_re('RegEx', 'split', function(self, str, n) + if tonumber(n) then + n = tonumber(n); + if n <= -1 or n ~= n then + n = math.huge; + end; + elseif n ~= nil then + error(string.format("invalid argument #3 to 'split' (number expected, got %s)", typeof(n)), 3); + else + n = math.huge; + end; + str = to_str_arr(str); + local i, count = 1, 0; + local ret = { }; + local prev_empty = 0; + while i <= str.n + 1 do + count += 1; + local span = n >= count and re_rawfind(self.token, str, i, self.flags, self.verb_flags, false); + if not span then + break; + end; + table.insert(ret, utf8_sub(str.s, i - prev_empty, span[0][1])); + prev_empty = span[0][1] >= span[0][2] and 1 or 0; + i = span[0][2] + prev_empty; + end; + table.insert(ret, string.sub(str.s, utf8.offset(str.s, i - prev_empty))); + return ret; +end); + +-- +local function re_index(self, index) + return re_m[index] or proxy[self].flags[index]; +end; + +local function re_tostr(self) + return proxy[self].pattern_repr .. proxy[self].flag_repr; +end; +-- + +local other_valid_group_char = { + -- non-capturing group + [0x3A] = true, + -- lookarounds + [0x21] = true, [0x3D] = true, + -- atomic + [0x3E] = true, + -- branch reset + [0x7C] = true, +}; + +local function tokenize_ptn(codes, flags) + if flags.unicode and not options.unicodeData then + return "options.unicodeData cannot be turned off while having unicode flag"; + end; + local i, len = 1, codes.n; + local group_n = 0; + local outln, group_id, verb_flags = { }, { }, { + newline = 1, newline_seq = 1, not_empty = 0, + }; + while i <= len do + local c = codes[i]; + if c == 0x28 then + -- Match + local ret; + if codes[i + 1] == 0x2A then + i += 2; + local start_i = i; + while codes[i] + and (codes[i] >= 0x30 and codes[i] <= 0x39 + or codes[i] >= 0x41 and codes[i] <= 0x5A + or codes[i] >= 0x61 and codes[i] <= 0x7A + or codes[i] == 0x5F or codes[i] == 0x3A) do + i += 1; + end; + if codes[i] ~= 0x29 and codes[i - 1] ~= 0x3A then + -- fallback as normal and ( can't be repeated + return "quantifier doesn't follow a repeatable pattern"; + end; + local selected_verb = utf8_sub(codes.s, start_i, i); + if selected_verb == "positive_lookahead:" or selected_verb == "negative_lookhead:" + or selected_verb == "positive_lookbehind:" or selected_verb == "negative_lookbehind:" + or selected_verb:find("^[pn]l[ab]:$") then + ret = { 0x28, nil, nil, selected_verb:find('^n') and 0x21 or 0x3D, selected_verb:find('b', 3, true) and 1 }; + elseif selected_verb == "atomic:" then + ret = { 0x28, nil, nil, 0x3E, nil }; + elseif selected_verb == "ACCEPT" or selected_verb == "FAIL" or selected_verb == 'F' or selected_verb == "PRUNE" or selected_verb == "SKIP" then + ret = selected_verb == 'F' and "FAIL" or selected_verb; + else + if line_verbs[selected_verb] then + verb_flags.newline = selected_verb; + elseif selected_verb == "BSR_ANYCRLF" or selected_verb == "BSR_UNICODE" then + verb_flags.newline_seq = selected_verb == "BSR_UNICODE" and 1 or 0; + elseif selected_verb == "NOTEMPTY" or selected_verb == "NOTEMPTY_ATSTART" then + verb_flags.not_empty = selected_verb == "NOTEMPTY" and 1 or 2; + else + return "unknown or malformed verb"; + end; + if outln[1] then + return "this verb must be placed at the beginning of the regex"; + end; + end; + elseif codes[i + 1] == 0x3F then + -- ? syntax + i += 2; + if codes[i] == 0x23 then + -- comments + i = table.find(codes, 0x29, i); + if not i then + return "unterminated parenthetical"; + end; + i += 1; + continue; + elseif not codes[i] then + return "unterminated parenthetical"; + end; + ret = { 0x28, nil, nil, codes[i], nil }; + if codes[i] == 0x30 and codes[i + 1] == 0x29 then + -- recursive match entire pattern + ret[1], ret[2], ret[3], ret[5] = "recurmatch", 0, 0, nil; + elseif codes[i] > 0x30 and codes[i] <= 0x39 then + -- recursive match + local org_i = i; + i += 1; + while codes[i] >= 0x30 and codes[i] <= 0x30 do + i += 1; + end; + if codes[i] ~= 0x29 then + return "invalid group structure"; + end; + ret[1], ret[2], ret[4] = "recurmatch", tonumber(utf8_sub(codes.s, org_i, i)), nil; + elseif codes[i] == 0x3C and codes[i + 1] == 0x21 or codes[i + 1] == 0x3D then + -- lookbehinds + i += 1; + ret[4], ret[5] = codes[i], 1; + elseif codes[i] == 0x7C then + -- branch reset + ret[5] = group_n; + elseif codes[i] == 0x50 or codes[i] == 0x3C or codes[i] == 0x27 then + if codes[i] == 0x50 then + i += 1; + end; + if codes[i] == 0x3D then + -- backref + local start_i = i + 1; + while codes[i] and + (codes[i] >= 0x30 and codes[i] <= 0x39 + or codes[i] >= 0x41 and codes[i] <= 0x5A + or codes[i] >= 0x61 and codes[i] <= 0x7A + or codes[i] == 0x5F) do + i += 1; + end; + if not codes[i] then + return "unterminated parenthetical"; + elseif codes[i] ~= 0x29 or i == start_i then + return "invalid group structure"; + end; + ret = { "backref", utf8_sub(codes.s, start_i, i) }; + elseif codes[i] == 0x3C or codes[i - 1] ~= 0x50 and codes[i] == 0x27 then + -- named capture + local delimiter = codes[i] == 0x27 and 0x27 or 0x3E; + local start_i = i + 1; + i += 1; + if codes[i] == 0x29 then + return "missing character in subpattern"; + elseif codes[i] >= 0x30 and codes[i] <= 0x39 then + return "subpattern name must not begin with a digit"; + elseif not (codes[i] >= 0x41 and codes[i] <= 0x5A or codes[i] >= 0x61 and codes[i] <= 0x7A or codes[i] == 0x5F) then + return "invalid character in subpattern"; + end; + i += 1; + while codes[i] and + (codes[i] >= 0x30 and codes[i] <= 0x39 + or codes[i] >= 0x41 and codes[i] <= 0x5A + or codes[i] >= 0x61 and codes[i] <= 0x7A + or codes[i] == 0x5F) do + i += 1; + end; + if not codes[i] then + return "unterminated parenthetical"; + elseif codes[i] ~= delimiter then + return "invalid character in subpattern"; + end; + local name = utf8_sub(codes.s, start_i, i); + group_n += 1; + if (group_id[name] or group_n) ~= group_n then + return "subpattern name already exists"; + end; + for name1, group_n1 in pairs(group_id) do + if name ~= name1 and group_n == group_n1 then + return "different names for subpatterns of the same number aren't permitted"; + end; + end; + group_id[name] = group_n; + ret[2], ret[4] = group_n, nil; + else + return "invalid group structure"; + end; + elseif not other_valid_group_char[codes[i]] then + return "invalid group structure"; + end; + else + group_n += 1; + ret = { 0x28, group_n, nil, nil }; + end; + if ret then + table.insert(outln, ret); + end; + elseif c == 0x29 then + -- Close parenthesis + local i1 = #outln + 1; + local lookbehind_c = -1; + local current_lookbehind_c = 0; + local max_c, group_c = 0, 0; + repeat + i1 -= 1; + local v, is_table = outln[i1], type(outln[i1]) == "table"; + if is_table and v[1] == 0x28 then + group_c += 1; + if current_lookbehind_c and v.count then + current_lookbehind_c += v.count; + end; + if not v[3] then + if v[4] == 0x7C then + group_n = v[5] + math.max(max_c, group_c); + end; + if current_lookbehind_c ~= lookbehind_c and lookbehind_c ~= -1 then + lookbehind_c = nil; + else + lookbehind_c = current_lookbehind_c; + end; + break; + end; + elseif v == alternation then + if current_lookbehind_c ~= lookbehind_c and lookbehind_c ~= -1 then + lookbehind_c, current_lookbehind_c = nil, nil; + else + lookbehind_c, current_lookbehind_c = current_lookbehind_c, 0; + end; + max_c, group_c = math.max(max_c, group_c), 0; + elseif current_lookbehind_c then + if is_table and v[1] == "quantifier" then + if v[2] == v[3] then + current_lookbehind_c += v[2]; + else + current_lookbehind_c = nil; + end; + else + current_lookbehind_c += 1; + end; + end; + until i1 < 1; + if i1 < 1 then + return "unmatched ) in regular expression"; + end; + local v = outln[i1]; + local outln_len_p_1 = #outln + 1; + local ret = { 0x29, v[2], i1, v[4], v[5], count = lookbehind_c }; + if (v[4] == 0x21 or v[4] == 0x3D) and v[5] and not lookbehind_c then + return "lookbehind assertion is not fixed width"; + end; + v[3] = outln_len_p_1; + table.insert(outln, ret); + elseif c == 0x2E then + table.insert(outln, dot); + elseif c == 0x5B then + -- Character set + local negate, char_class = false, nil; + i += 1; + local start_i = i; + if codes[i] == 0x5E then + negate = true; + i += 1; + elseif codes[i] == 0x2E or codes[i] == 0x3A or codes[i] == 0x3D then + -- POSIX character classes + char_class = codes[i]; + end; + local ret; + if codes[i] == 0x5B or codes[i] == 0x5C then + ret = { }; + else + ret = { codes[i] }; + i += 1; + end; + while codes[i] ~= 0x5D do + if not codes[i] then + return "unterminated character class"; + elseif codes[i] == 0x2D and ret[1] and type(ret[1]) == "number" then + if codes[i + 1] == 0x5D then + table.insert(ret, 1, 0x2D); + else + i += 1; + local ret_c = codes[i]; + if ret_c == 0x5B then + if codes[i + 1] == 0x2E or codes[i + 1] == 0x3A or codes[i + 1] == 0x3D then + -- Check for POSIX character class, name does not matter + local i1 = i + 2; + repeat + i1 = table.find(codes, 0x5D, i1); + until not i1 or codes[i1 - 1] ~= 0x5C; + if not i1 then + return "unterminated character class"; + elseif codes[i1 - 1] == codes[i + 1] and i1 - 1 ~= i + 1 then + return "invalid range in character class"; + end; + end; + if ret[1] > 0x5B then + return "invalid range in character class"; + end; + elseif ret_c == 0x5C then + i += 1; + if codes[i] == 0x78 then + local radix0, radix1; + i += 1; + if codes[i] and codes[i] >= 0x30 and codes[i] <= 0x39 or codes[i] >= 0x41 and codes[i] <= 0x46 or codes[i] >= 0x61 and codes[i] <= 0x66 then + radix0 = codes[i] - ((codes[i] >= 0x41 and codes[i] <= 0x5A) and 0x37 or (codes[i] >= 0x61 and codes[i] <= 0x7A) and 0x57 or 0x30); + i += 1; + if codes[i] and codes[i] >= 0x30 and codes[i] <= 0x39 or codes[i] >= 0x41 and codes[i] <= 0x46 or codes[i] >= 0x61 and codes[i] <= 0x66 then + radix1 = codes[i] - ((codes[i] >= 0x41 and codes[i] <= 0x5A) and 0x37 or (codes[i] >= 0x61 and codes[i] <= 0x7A) and 0x57 or 0x30); + else + i -= 1; + end; + else + i -= 1; + end; + ret_c = radix0 and (radix1 and 16 * radix0 + radix1 or radix0) or 0; + elseif codes[i] >= 0x30 and codes[i] <= 0x37 then + local radix0, radix1, radix2 = codes[i] - 0x30, nil, nil; + i += 1; + if codes[i] and codes[i] >= 0x30 and codes[i] <= 0x37 then + radix1 = codes[i] - 0x30; + i += 1; + if codes[i] and codes[i] >= 0x30 and codes[i] <= 0x37 then + radix2 = codes[i] - 0x30; + else + i -= 1; + end; + else + i -= 1; + end; + ret_c = radix1 and (radix2 and 64 * radix0 + 8 * radix1 + radix2 or 8 * radix0 + radix1) or radix0; + else + ret_c = escape_chars[codes[i]] or codes[i]; + if type(ret_c) ~= "number" then + return "invalid range in character class"; + end; + end; + elseif ret[1] > ret_c then + return "invalid range in character class"; + end; + ret[1] = { "range", ret[1], ret_c }; + end; + elseif codes[i] == 0x5B then + if codes[i + 1] == 0x2E or codes[i + 1] == 0x3A or codes[i + 1] == 0x3D then + local i1 = i + 2; + repeat + i1 = table.find(codes, 0x5D, i1); + until not i1 or codes[i1 - 1] ~= 0x5C; + if not i1 then + return "unterminated character class"; + elseif codes[i1 - 1] ~= codes[i + 1] or i1 - 1 == i + 1 then + table.insert(ret, 1, 0x5B); + elseif codes[i1 - 1] == 0x2E or codes[i1 - 1] == 0x3D then + return "POSIX collating elements aren't supported"; + elseif codes[i1 - 1] == 0x3A then + -- I have no plans to support escape codes (\) in character class names + local negate = codes[i + 3] == 0x5E; + local class_name = utf8_sub(codes.s, i + (negate and 3 or 2), i1 - 1); + -- If not valid then throw an error + if not posix_class_names[class_name] then + return "unknown POSIX class name"; + end; + table.insert(ret, 1, { "class", class_name, negate }); + i = i1; + end; + else + table.insert(ret, 1, 0x5B); + end; + elseif codes[i] == 0x5C then + i += 1; + if codes[i] == 0x78 then + local radix0, radix1; + i += 1; + if codes[i] == 0x7B then + i += 1; + local org_i = i; + while codes[i] and + (codes[i] >= 0x30 and codes[i] <= 0x39 + or codes[i] >= 0x41 and codes[i] <= 0x46 + or codes[i] >= 0x61 and codes[i] <= 0x66) do + i += 1; + end; + if codes[i] ~= 0x7D or i == org_i then + return "malformed hexadecimal character"; + elseif i - org_i > 4 then + return "character offset too large"; + end; + table.insert(ret, 1, tonumber(utf8_sub(codes.s, org_i, i), 16)); + else + if codes[i] and codes[i] >= 0x30 and codes[i] <= 0x39 or codes[i] >= 0x41 and codes[i] <= 0x46 or codes[i] >= 0x61 and codes[i] <= 0x66 then + radix0 = codes[i] - ((codes[i] >= 0x41 and codes[i] <= 0x5A) and 0x37 or (codes[i] >= 0x61 and codes[i] <= 0x7A) and 0x57 or 0x30); + i += 1; + if codes[i] and codes[i] >= 0x30 and codes[i] <= 0x39 or codes[i] >= 0x41 and codes[i] <= 0x46 or codes[i] >= 0x61 and codes[i] <= 0x66 then + radix1 = codes[i] - ((codes[i] >= 0x41 and codes[i] <= 0x5A) and 0x37 or (codes[i] >= 0x61 and codes[i] <= 0x7A) and 0x57 or 0x30); + else + i -= 1; + end; + else + i -= 1; + end; + table.insert(ret, 1, radix0 and (radix1 and 16 * radix0 + radix1 or radix0) or 0); + end; + elseif codes[i] >= 0x30 and codes[i] <= 0x37 then + local radix0, radix1, radix2 = codes[i] - 0x30, nil, nil; + i += 1; + if codes[i] and codes[i] >= 0x30 and codes[i] <= 0x37 then + radix1 = codes[i] - 0x30; + i += 1; + if codes[i] and codes[i] >= 0x30 and codes[i] <= 0x37 then + radix2 = codes[i] - 0x30; + else + i -= 1; + end; + else + i -= 1; + end; + table.insert(ret, 1, radix1 and (radix2 and 64 * radix0 + 8 * radix1 + radix2 or 8 * radix0 + radix1) or radix0); + elseif codes[i] == 0x45 then + -- intentionally left blank, \E that's not preceded \Q is ignored + elseif codes[i] == 0x51 then + local start_i = i + 1; + repeat + i = table.find(codes, 0x5C, i + 1); + until not i or codes[i + 1] == 0x45; + table.move(codes, start_i, i and i - 1 or #codes, #outln + 1, outln); + if not i then + break; + end; + i += 1; + elseif codes[i] == 0x4E then + if codes[i + 1] == 0x7B and codes[i + 2] == 0x55 and codes[i + 3] == 0x2B and flags.unicode then + i += 4; + local start_i = i; + while codes[i] and + (codes[i] >= 0x30 and codes[i] <= 0x39 + or codes[i] >= 0x41 and codes[i] <= 0x46 + or codes[i] >= 0x61 and codes[i] <= 0x66) do + i += 1; + end; + if codes[i] ~= 0x7D or i == start_i then + return "malformed Unicode code point"; + end; + local code_point = tonumber(utf8_sub(codes.s, start_i, i)); + table.insert(ret, 1, code_point); + else + return "invalid escape sequence"; + end; + elseif codes[i] == 0x50 or codes[i] == 0x70 then + if not options.unicodeData then + return "options.unicodeData cannot be turned off when using \\p"; + end; + i += 1; + if codes[i] ~= 0x7B then + local c_name = utf8.char(codes[i] or 0); + if not valid_categories[c_name] then + return "unknown or malformed script name"; + end; + table.insert(ret, 1, { "category", false, c_name }); + else + local negate = codes[i] == 0x50; + i += 1; + if codes[i] == 0x5E then + i += 1; + negate = not negate; + end; + local start_i = i; + while codes[i] and + (codes[i] >= 0x30 and codes[i] <= 0x39 + or codes[i] >= 0x41 and codes[i] <= 0x5A + or codes[i] >= 0x61 and codes[i] <= 0x7A + or codes[i] == 0x5F) do + i += 1; + end; + if codes[i] ~= 0x7D then + return "unknown or malformed script name"; + end; + local c_name = utf8_sub(codes.s, start_i, i); + local script_set = chr_scripts[c_name]; + if script_set then + table.insert(ret, 1, { "charset", negate, script_set }); + elseif not valid_categories[c_name] then + return "unknown or malformed script name"; + else + table.insert(ret, 1, { "category", negate, c_name }); + end; + end; + elseif codes[i] == 0x6F then + i += 1; + if codes[i] ~= 0x7B then + return "malformed octal code"; + end; + i += 1; + local org_i = i; + while codes[i] and codes[i] >= 0x30 and codes[i] <= 0x37 do + i += 1; + end; + if codes[i] ~= 0x7D or i == org_i then + return "malformed octal code"; + end; + local ret_chr = tonumber(utf8_sub(codes.s, org_i, i), 8); + if ret_chr > 0xFFFF then + return "character offset too large"; + end; + table.insert(ret, 1, ret_chr); + else + local esc_char = escape_chars[codes[i]]; + table.insert(ret, 1, type(esc_char) == "string" and { "class", esc_char, false } or esc_char or codes[i]); + end; + elseif flags.ignoreCase and codes[i] >= 0x61 and codes[i] <= 0x7A then + table.insert(ret, 1, codes[i] - 0x20); + else + table.insert(ret, 1, codes[i]); + end; + i += 1; + end; + if codes[i - 1] == char_class and i - 1 ~= start_i then + return char_class == 0x3A and "POSIX named classes are only support within a character set" or "POSIX collating elements aren't supported"; + end; + if not ret[2] and not negate then + table.insert(outln, ret[1]); + else + table.insert(outln, { "charset", negate, ret }); + end; + elseif c == 0x5C then + -- Escape char + i += 1; + local escape_c = codes[i]; + if not escape_c then + return "pattern may not end with a trailing backslash"; + elseif escape_c >= 0x30 and escape_c <= 0x39 then + local org_i = i; + while codes[i + 1] and codes[i + 1] >= 0x30 and codes[i + 1] <= 0x39 do + i += 1; + end; + local escape_d = tonumber(utf8_sub(codes.s, org_i, i + 1)); + if escape_d > group_n and i ~= org_i then + i = org_i; + local radix0, radix1, radix2; + if codes[i] <= 0x37 then + radix0 = codes[i] - 0x30; + i += 1; + if codes[i] and codes[i] >= 0x30 and codes[i] <= 0x37 then + radix1 = codes[i] - 0x30; + i += 1; + if codes[i] and codes[i] >= 0x30 and codes[i] <= 0x37 then + radix2 = codes[i] - 0x30; + else + i -= 1; + end; + else + i -= 1; + end; + end; + table.insert(outln, radix0 and (radix1 and (radix2 and 64 * radix0 + 8 * radix1 + radix2 or 8 * radix0 + radix1) or radix0) or codes[org_i]); + else + table.insert(outln, { "backref", escape_d }); + end; + elseif escape_c == 0x45 then + -- intentionally left blank, \E that's not preceded \Q is ignored + elseif escape_c == 0x51 then + local start_i = i + 1; + repeat + i = table.find(codes, 0x5C, i + 1); + until not i or codes[i + 1] == 0x45; + table.move(codes, start_i, i and i - 1 or #codes, #outln + 1, outln); + if not i then + break; + end; + i += 1; + elseif escape_c == 0x4E then + if codes[i + 1] == 0x7B and codes[i + 2] == 0x55 and codes[i + 3] == 0x2B and flags.unicode then + i += 4; + local start_i = i; + while codes[i] and + (codes[i] >= 0x30 and codes[i] <= 0x39 + or codes[i] >= 0x41 and codes[i] <= 0x46 + or codes[i] >= 0x61 and codes[i] <= 0x66) do + i += 1; + end; + if codes[i] ~= 0x7D or i == start_i then + return "malformed Unicode code point"; + end; + local code_point = tonumber(utf8_sub(codes.s, start_i, i)); + table.insert(outln, code_point); + else + table.insert(outln, escape_chars[0x4E]); + end; + elseif escape_c == 0x50 or escape_c == 0x70 then + if not options.unicodeData then + return "options.unicodeData cannot be turned off when using \\p"; + end; + i += 1; + if codes[i] ~= 0x7B then + local c_name = utf8.char(codes[i] or 0); + if not valid_categories[c_name] then + return "unknown or malformed script name"; + end; + table.insert(outln, { "category", false, c_name }); + else + local negate = escape_c == 0x50; + i += 1; + if codes[i] == 0x5E then + i += 1; + negate = not negate; + end; + local start_i = i; + while codes[i] and + (codes[i] >= 0x30 and codes[i] <= 0x39 + or codes[i] >= 0x41 and codes[i] <= 0x5A + or codes[i] >= 0x61 and codes[i] <= 0x7A + or codes[i] == 0x5F) do + i += 1; + end; + if codes[i] ~= 0x7D then + return "unknown or malformed script name"; + end; + local c_name = utf8_sub(codes.s, start_i, i); + local script_set = chr_scripts[c_name]; + if script_set then + table.insert(outln, { "charset", negate, script_set }); + elseif not valid_categories[c_name] then + return "unknown or malformed script name"; + else + table.insert(outln, { "category", negate, c_name }); + end; + end; + elseif escape_c == 0x67 and (codes[i + 1] == 0x7B or codes[i + 1] >= 0x30 and codes[i + 1] <= 0x39) then + local is_grouped = false; + i += 1; + if codes[i] == 0x7B then + i += 1; + is_grouped = true; + elseif codes[i] < 0x30 or codes[i] > 0x39 then + return "malformed reference code"; + end; + local org_i = i; + while codes[i] and + (codes[i] >= 0x30 and codes[i] <= 0x39 + or codes[i] >= 0x41 and codes[i] <= 0x46 + or codes[i] >= 0x61 and codes[i] <= 0x66) do + i += 1; + end; + if is_grouped and codes[i] ~= 0x7D then + return "malformed reference code"; + end; + local ref_name = tonumber(utf8_sub(codes.s, org_i, i + (is_grouped and 0 or 1))); + table.insert(outln, { "backref", ref_name }); + if not is_grouped then + i -= 1; + end; + elseif escape_c == 0x6F then + i += 1; + if codes[i + 1] ~= 0x7B then + return "malformed octal code"; + end + i += 1; + local org_i = i; + while codes[i] and codes[i] >= 0x30 and codes[i] <= 0x37 do + i += 1; + end; + if codes[i] ~= 0x7D or i == org_i then + return "malformed octal code"; + end; + local ret_chr = tonumber(utf8_sub(codes.s, org_i, i), 8); + if ret_chr > 0xFFFF then + return "character offset too large"; + end; + table.insert(outln, ret_chr); + elseif escape_c == 0x78 then + local radix0, radix1; + i += 1; + if codes[i] == 0x7B then + i += 1; + local org_i = i; + while codes[i] and + (codes[i] >= 0x30 and codes[i] <= 0x39 + or codes[i] >= 0x41 and codes[i] <= 0x46 + or codes[i] >= 0x61 and codes[i] <= 0x66) do + i += 1; + end; + if codes[i] ~= 0x7D or i == org_i then + return "malformed hexadecimal code"; + elseif i - org_i > 4 then + return "character offset too large"; + end; + table.insert(outln, tonumber(utf8_sub(codes.s, org_i, i), 16)); + else + if codes[i] and (codes[i] >= 0x30 and codes[i] <= 0x39 or codes[i] >= 0x41 and codes[i] <= 0x46 or codes[i] >= 0x61 and codes[i] <= 0x66) then + radix0 = codes[i] - ((codes[i] >= 0x41 and codes[i] <= 0x5A) and 0x37 or (codes[i] >= 0x61 and codes[i] <= 0x7A) and 0x57 or 0x30); + i += 1; + if codes[i] and (codes[i] >= 0x30 and codes[i] <= 0x39 or codes[i] >= 0x41 and codes[i] <= 0x46 or codes[i] >= 0x61 and codes[i] <= 0x66) then + radix1 = codes[i] - ((codes[i] >= 0x41 and codes[i] <= 0x5A) and 0x37 or (codes[i] >= 0x61 and codes[i] <= 0x7A) and 0x57 or 0x30); + else + i -= 1; + end; + else + i -= 1; + end; + table.insert(outln, radix0 and (radix1 and 16 * radix0 + radix1 or radix0) or 0); + end; + else + local esc_char = b_escape_chars[escape_c] or escape_chars[escape_c]; + table.insert(outln, esc_char or escape_c); + end; + elseif c == 0x2A or c == 0x2B or c == 0x3F or c == 0x7B then + -- Quantifier + local start_q, end_q; + if c == 0x7B then + local org_i = i + 1; + local start_i; + while codes[i + 1] and (codes[i + 1] >= 0x30 and codes[i + 1] <= 0x39 or codes[i + 1] == 0x2C and not start_i and i + 1 ~= org_i) do + i += 1; + if codes[i] == 0x2C then + start_i = i; + end; + end; + if codes[i + 1] == 0x7D then + i += 1; + if not start_i then + start_q = tonumber(utf8_sub(codes.s, org_i, i)); + end_q = start_q; + else + start_q, end_q = tonumber(utf8_sub(codes.s, org_i, start_i)), start_i + 1 == i and math.huge or tonumber(utf8_sub(codes.s, start_i + 1, i)); + if end_q < start_q then + return "numbers out of order in {} quantifier"; + end; + end; + else + table.move(codes, org_i - 1, i, #outln + 1, outln); + end; + else + start_q, end_q = c == 0x2B and 1 or 0, c == 0x3F and 1 or math.huge; + end; + if start_q then + local quantifier_type = flags.ungreedy and "lazy" or "greedy"; + if codes[i + 1] == 0x2B or codes[i + 1] == 0x3F then + i += 1; + quantifier_type = codes[i] == 0x2B and "possessive" or flags.ungreedy and "greedy" or "lazy"; + end; + local outln_len = #outln; + local last_outln_value = outln[outln_len]; + if not last_outln_value or type(last_outln_value) == "table" and (last_outln_value[1] == "quantifier" or last_outln_value[1] == 0x28 or b_escape_chars[last_outln_value[1]]) + or last_outln_value == alternation or type(last_outln_value) == "string" then + return "quantifier doesn't follow a repeatable pattern"; + end; + if end_q == 0 then + table.remove(outln); + elseif start_q ~= 1 or end_q ~= 1 then + if type(last_outln_value) == "table" and last_outln_value[1] == 0x29 then + outln_len = last_outln_value[3]; + end; + outln[outln_len] = { "quantifier", start_q, end_q, quantifier_type, outln[outln_len] }; + end; + end; + elseif c == 0x7C then + -- Alternation + table.insert(outln, alternation); + local i1 = #outln; + repeat + i1 -= 1; + local v1, is_table = outln[i1], type(outln[i1]) == "table"; + if is_table and v1[1] == 0x29 then + i1 = outln[i1][3]; + elseif is_table and v1[1] == 0x28 then + if v1[4] == 0x7C then + group_n = v1[5]; + end; + break; + end; + until not v1; + elseif c == 0x24 or c == 0x5E then + table.insert(outln, c == 0x5E and beginning_str or end_str); + elseif flags.ignoreCase and c >= 0x61 and c <= 0x7A then + table.insert(outln, c - 0x20); + elseif flags.extended and (c >= 0x09 and c <= 0x0D or c == 0x20 or c == 0x23) then + if c == 0x23 then + repeat + i += 1; + until not codes[i] or codes[i] == 0x0A or codes[i] == 0x0D; + end; + else + table.insert(outln, c); + end; + i += 1; + end; + local max_group_n = 0; + for i, v in ipairs(outln) do + if type(v) == "table" and (v[1] == 0x28 or v[1] == "quantifier" and type(v[5]) == "table" and v[5][1] == 0x28) then + if v[1] == "quantifier" then + v = v[5]; + end; + if not v[3] then + return "unterminated parenthetical"; + elseif v[2] then + max_group_n = math.max(max_group_n, v[2]); + end; + elseif type(v) == "table" and (v[1] == "backref" or v[1] == "recurmatch") then + if not group_id[v[2]] and (type(v[2]) ~= "number" or v[2] > group_n) then + return "reference to a non-existent or invalid subpattern"; + elseif v[1] == "recurmatch" and v[2] ~= 0 then + for i1, v1 in ipairs(outln) do + if type(v1) == "table" and v1[1] == 0x28 and v1[2] == v[2] then + v[3] = i1; + break; + end; + end; + elseif type(v[2]) == "string" then + v[2] = group_id[v[2]]; + end; + end; + end; + outln.group_n = max_group_n; + return outln, group_id, verb_flags; +end; + +if not tonumber(options.cacheSize) then + error(string.format("expected number for options.cacheSize, got %s", typeof(options.cacheSize)), 2); +end; +local cacheSize = math.floor(options.cacheSize or 0) ~= 0 and tonumber(options.cacheSize); +local cache_pattern, cache_pattern_names; +if not cacheSize then +elseif cacheSize < 0 or cacheSize ~= cacheSize then + error("cache size cannot be a negative number or a NaN", 2); +elseif cacheSize == math.huge then + cache_pattern, cache_pattern_names = { nil }, { nil }; +elseif cacheSize >= 2 ^ 32 then + error("cache size too large", 2); +else + cache_pattern, cache_pattern_names = table.create(options.cacheSize), table.create(options.cacheSize); +end; +if cacheSize then + function re.pruge() + table.clear(cache_pattern_names); + table.clear(cache_pattern); + end; +end; + +local function new_re(str_arr, flags, flag_repr, pattern_repr) + local tokenized_ptn, group_id, verb_flags; + local cache_format = cacheSize and string.format("%s|%s", str_arr.s, flag_repr); + local cached_token = cacheSize and cache_pattern[table.find(cache_pattern_names, cache_format)]; + if cached_token then + tokenized_ptn, group_id, verb_flags = table.unpack(cached_token, 1, 3); + else + tokenized_ptn, group_id, verb_flags = tokenize_ptn(str_arr, flags); + if type(tokenized_ptn) == "string" then + error(tokenized_ptn, 2); + end; + if cacheSize and tokenized_ptn[1] then + table.insert(cache_pattern_names, 1, cache_format); + table.insert(cache_pattern, 1, { tokenized_ptn, group_id, verb_flags }); + if cacheSize ~= math.huge then + table.remove(cache_pattern_names, cacheSize + 1); + table.remove(cache_pattern, cacheSize + 1); + end; + end; + end; + + local object = newproxy(true); + proxy[object] = { name = "RegEx", flags = flags, flag_repr = flag_repr, pattern_repr = pattern_repr, token = tokenized_ptn, group_id = group_id, verb_flags = verb_flags }; + local object_mt = getmetatable(object); + object_mt.__index = setmetatable(flags, re_m); + object_mt.__tostring = re_tostr; + object_mt.__metatable = lockmsg; + + return object; +end; + +local function escape_fslash(pre) + return (#pre % 2 == 0 and '\\' or '') .. pre .. '.'; +end; + +local function sort_flag_chr(a, b) + return a:lower() < b:lower(); +end; + +function re.new(...) + if select('#', ...) == 0 then + error("missing argument #1 (string expected)", 2); + end; + local ptn, flags_str = ...; + if type(ptn) == "number" then + ptn ..= ''; + elseif type(ptn) ~= "string" then + error(string.format("invalid argument #1 (string expected, got %s)", typeof(ptn)), 2); + end; + if type(flags_str) ~= "string" and type(flags_str) ~= "number" and flags_str ~= nil then + error(string.format("invalid argument #2 (string expected, got %s)", typeof(flags_str)), 2); + end; + + local flags = { + anchored = false, caseless = false, multiline = false, dotall = false, unicode = false, ungreedy = false, extended = false, + }; + local flag_repr = { }; + for f in string.gmatch(flags_str or '', utf8.charpattern) do + if flags[flag_map[f]] ~= false then + error("invalid regular expression flag " .. f, 3); + end; + flags[flag_map[f]] = true; + table.insert(flag_repr, f); + end; + table.sort(flag_repr, sort_flag_chr); + flag_repr = table.concat(flag_repr); + return new_re(to_str_arr(ptn), flags, flag_repr, string.format("/%s/", ptn:gsub("(\\*)/", escape_fslash))); +end; + +function re.fromstring(...) + if select('#', ...) == 0 then + error("missing argument #1 (string expected)", 2); + end; + local ptn = ...; + if type(ptn) == "number" then + ptn ..= ''; + elseif type(ptn) ~= "string" then + error(string.format("invalid argument #1 (string expected, got %s)", typeof(ptn), 2)); + end; + local str_arr = to_str_arr(ptn); + local delimiter = str_arr[1]; + if not delimiter then + error("empty regex", 2); + elseif delimiter == 0x5C or (delimiter >= 0x30 and delimiter <= 0x39) or (delimiter >= 0x41 and delimiter <= 0x5A) or (delimiter >= 0x61 and delimiter <= 0x7A) then + error("delimiter must not be alphanumeric or a backslash", 2); + end; + + local i0 = 1; + repeat + i0 = table.find(str_arr, delimiter, i0 + 1); + if not i0 then + error(string.format("no ending delimiter ('%s') found", utf8.char(delimiter)), 2); + end; + local escape_count = 1; + while str_arr[i0 - escape_count] == 0x5C do + escape_count += 1; + end; + until escape_count % 2 == 1; + + local flags = { + anchored = false, caseless = false, multiline = false, dotall = false, unicode = false, ungreedy = false, extended = false, + }; + local flag_repr = { }; + while str_arr.n > i0 do + local f = utf8.char(table.remove(str_arr)); + str_arr.n -= 1; + if flags[flag_map[f]] ~= false then + error("invalid regular expression flag " .. f, 3); + end; + flags[flag_map[f]] = true; + table.insert(flag_repr, f); + end; + table.sort(flag_repr, sort_flag_chr); + flag_repr = table.concat(flag_repr); + table.remove(str_arr, 1); + table.remove(str_arr); + str_arr.n -= 2; + str_arr.s = string.sub(str_arr.s, 2, 1 + str_arr.n); + return new_re(str_arr, flags, flag_repr, string.sub(ptn, 1, 2 + str_arr.n)); +end; + +local re_escape_line_chrs = { + ['\0'] = '\\x00', ['\n'] = '\\n', ['\t'] = '\\t', ['\r'] = '\\r', ['\f'] = '\\f', +}; + +function re.escape(...) + if select('#', ...) == 0 then + error("missing argument #1 (string expected)", 2); + end; + local str, extended, delimiter = ...; + if type(str) == "number" then + str ..= ''; + elseif type(str) ~= "string" then + error(string.format("invalid argument #1 to 'escape' (string expected, got %s)", typeof(str)), 2); + end; + if delimiter == nil then + delimiter = ''; + elseif type(delimiter) == "number" then + delimiter ..= ''; + elseif type(delimiter) ~= "string" then + error(string.format("invalid argument #3 to 'escape' (string expected, got %s)", typeof(delimiter)), 2); + end; + if utf8.len(delimiter) > 1 or delimiter:match("^[%a\\]$") then + error("delimiter have not be alphanumeric", 2); + end; + return (string.gsub(str, "[\0\f\n\r\t]", re_escape_line_chrs):gsub(string.format("[\\%s#()%%%%*+.?[%%]^{|%s]", extended and '%s' or '', (delimiter:find'^[%%%]]$' and '%' or '') .. delimiter), "\\%1")); +end; + +function re.type(...) + if select('#', ...) == 0 then + error("missing argument #1", 2); + end; + return proxy[...] and proxy[...].name; +end; + +for k, f in pairs(re_m) do + re[k] = f; +end; + +re_m = { __index = re_m }; + +lockmsg = re.fromstring([[/The\s*metatable\s*is\s*(?:locked|inaccessible)(?#Nice try :])/i]]); +getmetatable(lockmsg).__metatable = lockmsg; + +local function readonly_table() + error("Attempt to modify a readonly table", 2); +end; + +match_m = { + __index = match_m, + __metatable = lockmsg, + __newindex = readonly_table, +}; + +re.Match = setmetatable({ }, match_m); + +return setmetatable({ }, { + __index = re, + __metatable = lockmsg, + __newindex = readonly_table, +}); From 6303ae30c4d9b939a781f2a2c6a33475f825b8c9 Mon Sep 17 00:00:00 2001 From: JohnnyMorganz Date: Tue, 5 Jul 2022 16:59:09 +0100 Subject: [PATCH 301/399] Fix op used when stringifying AstExprIndexName (#572) * Fix op used when stringifying AstExprIndexName * Remove visualizeWithSelf --- Analysis/src/Transpiler.cpp | 20 +++----------------- tests/Transpiler.test.cpp | 2 +- 2 files changed, 4 insertions(+), 18 deletions(-) diff --git a/Analysis/src/Transpiler.cpp b/Analysis/src/Transpiler.cpp index 1577bd63..9feff1c0 100644 --- a/Analysis/src/Transpiler.cpp +++ b/Analysis/src/Transpiler.cpp @@ -205,20 +205,6 @@ struct Printer } } - void visualizeWithSelf(AstExpr& expr, bool self) - { - if (!self) - return visualize(expr); - - AstExprIndexName* func = expr.as(); - LUAU_ASSERT(func); - - visualize(*func->expr); - writer.symbol(":"); - advance(func->indexLocation.begin); - writer.identifier(func->index.value); - } - void visualizeTypePackAnnotation(const AstTypePack& annotation, bool forVarArg) { advance(annotation.location.begin); @@ -366,7 +352,7 @@ struct Printer } else if (const auto& a = expr.as()) { - visualizeWithSelf(*a->func, a->self); + visualize(*a->func); writer.symbol("("); bool first = true; @@ -385,7 +371,7 @@ struct Printer else if (const auto& a = expr.as()) { visualize(*a->expr); - writer.symbol("."); + writer.symbol(std::string(1, a->op)); writer.write(a->index.value); } else if (const auto& a = expr.as()) @@ -766,7 +752,7 @@ struct Printer else if (const auto& a = program.as()) { writer.keyword("function"); - visualizeWithSelf(*a->name, a->func->self != nullptr); + visualize(*a->name); visualizeFunctionBody(*a->func); } else if (const auto& a = program.as()) diff --git a/tests/Transpiler.test.cpp b/tests/Transpiler.test.cpp index b02a52b2..d2ed9aef 100644 --- a/tests/Transpiler.test.cpp +++ b/tests/Transpiler.test.cpp @@ -583,7 +583,7 @@ TEST_CASE_FIXTURE(Fixture, "transpile_error_expr") auto names = AstNameTable{allocator}; ParseResult parseResult = Parser::parse(code.data(), code.size(), names, allocator, {}); - CHECK_EQ("local a = (error-expr: f.%error-id%)-(error-expr)", transpileWithTypes(*parseResult.root)); + CHECK_EQ("local a = (error-expr: f:%error-id%)-(error-expr)", transpileWithTypes(*parseResult.root)); } TEST_CASE_FIXTURE(Fixture, "transpile_error_stat") From 4d98a16ea8f4985ca757d2ceebc278a6db3f9761 Mon Sep 17 00:00:00 2001 From: JohnnyMorganz Date: Tue, 5 Jul 2022 17:08:05 +0100 Subject: [PATCH 302/399] Store resolved types in `astResolvedTypes` (#574) --- Analysis/include/Luau/TypeInfer.h | 1 + Analysis/src/Frontend.cpp | 2 ++ Analysis/src/TypeInfer.cpp | 27 ++++++++++++++++++++------- tests/Frontend.test.cpp | 2 ++ tests/TypeInfer.test.cpp | 23 +++++++++++++++++++++++ 5 files changed, 48 insertions(+), 7 deletions(-) diff --git a/Analysis/include/Luau/TypeInfer.h b/Analysis/include/Luau/TypeInfer.h index 455654d9..dac2902c 100644 --- a/Analysis/include/Luau/TypeInfer.h +++ b/Analysis/include/Luau/TypeInfer.h @@ -345,6 +345,7 @@ private: TypePackId freshTypePack(TypeLevel level); TypeId resolveType(const ScopePtr& scope, const AstType& annotation); + TypeId resolveTypeWorker(const ScopePtr& scope, const AstType& annotation); TypePackId resolveTypePack(const ScopePtr& scope, const AstTypeList& types); TypePackId resolveTypePack(const ScopePtr& scope, const AstTypePack& annotation); TypeId instantiateTypeFun(const ScopePtr& scope, const TypeFun& tf, const std::vector& typeParams, diff --git a/Analysis/src/Frontend.cpp b/Analysis/src/Frontend.cpp index 4cfaa112..9195363d 100644 --- a/Analysis/src/Frontend.cpp +++ b/Analysis/src/Frontend.cpp @@ -496,6 +496,8 @@ CheckResult Frontend::check(const ModuleName& name, std::optionalastTypes.clear(); module->astExpectedTypes.clear(); module->astOriginalCallTypes.clear(); + module->astResolvedTypes.clear(); + module->astResolvedTypePacks.clear(); module->scopes.resize(1); } diff --git a/Analysis/src/TypeInfer.cpp b/Analysis/src/TypeInfer.cpp index d9486a4f..4fafb50f 100644 --- a/Analysis/src/TypeInfer.cpp +++ b/Analysis/src/TypeInfer.cpp @@ -4884,6 +4884,13 @@ TypePackId TypeChecker::freshTypePack(TypeLevel level) } TypeId TypeChecker::resolveType(const ScopePtr& scope, const AstType& annotation) +{ + TypeId ty = resolveTypeWorker(scope, annotation); + currentModule->astResolvedTypes[&annotation] = ty; + return ty; +} + +TypeId TypeChecker::resolveTypeWorker(const ScopePtr& scope, const AstType& annotation) { if (const auto& lit = annotation.as()) { @@ -5200,9 +5207,10 @@ TypePackId TypeChecker::resolveTypePack(const ScopePtr& scope, const AstTypeList TypePackId TypeChecker::resolveTypePack(const ScopePtr& scope, const AstTypePack& annotation) { + TypePackId result; if (const AstTypePackVariadic* variadic = annotation.as()) { - return addTypePack(TypePackVar{VariadicTypePack{resolveType(scope, *variadic->variadicType)}}); + result = addTypePack(TypePackVar{VariadicTypePack{resolveType(scope, *variadic->variadicType)}}); } else if (const AstTypePackGeneric* generic = annotation.as()) { @@ -5216,10 +5224,12 @@ TypePackId TypeChecker::resolveTypePack(const ScopePtr& scope, const AstTypePack else reportError(TypeError{generic->location, UnknownSymbol{genericName, UnknownSymbol::Type}}); - return errorRecoveryTypePack(scope); + result = errorRecoveryTypePack(scope); + } + else + { + result = *genericTy; } - - return *genericTy; } else if (const AstTypePackExplicit* explicitTp = annotation.as()) { @@ -5229,14 +5239,17 @@ TypePackId TypeChecker::resolveTypePack(const ScopePtr& scope, const AstTypePack types.push_back(resolveType(scope, *type)); if (auto tailType = explicitTp->typeList.tailType) - return addTypePack(types, resolveTypePack(scope, *tailType)); - - return addTypePack(types); + result = addTypePack(types, resolveTypePack(scope, *tailType)); + else + result = addTypePack(types); } else { ice("Unknown AstTypePack kind"); } + + currentModule->astResolvedTypePacks[&annotation] = result; + return result; } bool ApplyTypeFunction::isDirty(TypeId ty) diff --git a/tests/Frontend.test.cpp b/tests/Frontend.test.cpp index b9c24704..4f331399 100644 --- a/tests/Frontend.test.cpp +++ b/tests/Frontend.test.cpp @@ -791,6 +791,8 @@ TEST_CASE_FIXTURE(FrontendFixture, "discard_type_graphs") CHECK_EQ(0, module->internalTypes.typeVars.size()); CHECK_EQ(0, module->internalTypes.typePacks.size()); CHECK_EQ(0, module->astTypes.size()); + CHECK_EQ(0, module->astResolvedTypes.size()); + CHECK_EQ(0, module->astResolvedTypePacks.size()); } TEST_CASE_FIXTURE(FrontendFixture, "it_should_be_safe_to_stringify_errors_when_full_type_graph_is_discarded") diff --git a/tests/TypeInfer.test.cpp b/tests/TypeInfer.test.cpp index efdfe0b1..a175b826 100644 --- a/tests/TypeInfer.test.cpp +++ b/tests/TypeInfer.test.cpp @@ -1003,4 +1003,27 @@ TEST_CASE_FIXTURE(Fixture, "do_not_bind_a_free_table_to_a_union_containing_that_ )"); } +TEST_CASE_FIXTURE(Fixture, "types stored in astResolvedTypes") +{ + CheckResult result = check(R"( +type alias = typeof("hello") +local function foo(param: alias) +end + )"); + + auto node = findNodeAtPosition(*getMainSourceModule(), {2, 16}); + auto ty = lookupType("alias"); + REQUIRE(node); + REQUIRE(node->is()); + REQUIRE(ty); + + auto func = node->as(); + REQUIRE(func->args.size == 1); + + auto arg = *func->args.begin(); + auto annotation = arg->annotation; + + CHECK_EQ(*getMainModule()->astResolvedTypes.find(annotation), *ty); +} + TEST_SUITE_END(); From baa35117bd164a6a034c82835ffbe72edbdd2f24 Mon Sep 17 00:00:00 2001 From: JohnnyMorganz Date: Tue, 5 Jul 2022 17:19:54 +0100 Subject: [PATCH 303/399] Use syntheticName for stringified MetatableTypeVar (#563) --- Analysis/src/ToString.cpp | 6 ++++++ tests/ToString.test.cpp | 31 +++++++++++++++++++++++++++++++ 2 files changed, 37 insertions(+) diff --git a/Analysis/src/ToString.cpp b/Analysis/src/ToString.cpp index bbbad9d5..5d54d14a 100644 --- a/Analysis/src/ToString.cpp +++ b/Analysis/src/ToString.cpp @@ -700,6 +700,12 @@ struct TypeVarStringifier void operator()(TypeId, const MetatableTypeVar& mtv) { state.result.invalid = true; + if (!state.exhaustive && mtv.syntheticName) + { + state.emit(*mtv.syntheticName); + return; + } + state.emit("{ @metatable "); stringify(mtv.metatable); state.emit(","); diff --git a/tests/ToString.test.cpp b/tests/ToString.test.cpp index 387e07cd..1601a152 100644 --- a/tests/ToString.test.cpp +++ b/tests/ToString.test.cpp @@ -96,6 +96,37 @@ TEST_CASE_FIXTURE(Fixture, "table_respects_use_line_break") //clang-format on } +TEST_CASE_FIXTURE(Fixture, "metatable") +{ + TypeVar table{TypeVariant(TableTypeVar())}; + TypeVar metatable{TypeVariant(TableTypeVar())}; + TypeVar mtv{TypeVariant(MetatableTypeVar{&table, &metatable})}; + CHECK_EQ("{ @metatable { }, { } }", toString(&mtv)); +} + +TEST_CASE_FIXTURE(Fixture, "named_metatable") +{ + TypeVar table{TypeVariant(TableTypeVar())}; + TypeVar metatable{TypeVariant(TableTypeVar())}; + TypeVar mtv{TypeVariant(MetatableTypeVar{&table, &metatable, "NamedMetatable"})}; + CHECK_EQ("NamedMetatable", toString(&mtv)); +} + +TEST_CASE_FIXTURE(BuiltinsFixture, "named_metatable_toStringNamedFunction") +{ + CheckResult result = check(R"( + local function createTbl(): NamedMetatable + return setmetatable({}, {}) + end + type NamedMetatable = typeof(createTbl()) + )"); + + TypeId ty = requireType("createTbl"); + const FunctionTypeVar* ftv = get(follow(ty)); + REQUIRE(ftv); + CHECK_EQ("createTbl(): NamedMetatable", toStringNamedFunction("createTbl", *ftv)); +} + TEST_CASE_FIXTURE(BuiltinsFixture, "exhaustive_toString_of_cyclic_table") { CheckResult result = check(R"( From 42510bb5c885dff16159ded78c95ab3f7ea83d4f Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Tue, 5 Jul 2022 11:36:33 -0700 Subject: [PATCH 304/399] Fix test after merging a stale PR (#577) --- tests/TypeInfer.provisional.test.cpp | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/TypeInfer.provisional.test.cpp b/tests/TypeInfer.provisional.test.cpp index 059aed2e..018059fd 100644 --- a/tests/TypeInfer.provisional.test.cpp +++ b/tests/TypeInfer.provisional.test.cpp @@ -518,7 +518,7 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "greedy_inference_with_shared_self_triggers_f )"); LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK_EQ("Not all codepaths in this function return '{ @metatable T, {| |} }, a...'.", toString(result.errors[0])); + CHECK_EQ("Not all codepaths in this function return 'self, a...'.", toString(result.errors[0])); } TEST_SUITE_END(); From a7ae439b0f219000cd4d78982c13e44c6a46b2c1 Mon Sep 17 00:00:00 2001 From: Alan Jeffrey <403333+asajeffrey@users.noreply.github.com> Date: Tue, 5 Jul 2022 16:25:09 -0500 Subject: [PATCH 305/399] Document new table type features (#567) --- docs/_pages/typecheck.md | 31 ++++++++++++++++++++----------- 1 file changed, 20 insertions(+), 11 deletions(-) diff --git a/docs/_pages/typecheck.md b/docs/_pages/typecheck.md index 363056c5..8e032da2 100644 --- a/docs/_pages/typecheck.md +++ b/docs/_pages/typecheck.md @@ -104,15 +104,15 @@ From the type checker perspective, each table can be in one of three states. The ### Unsealed tables -An unsealed table is a table whose properties could still be tacked on. This occurs when the table constructor literal had zero expressions. This is one way to accumulate knowledge of the shape of this table. +An unsealed table is a table which supports adding new properties, which updates the tables type. Unsealed tables are created using table literals. This is one way to accumulate knowledge of the shape of this table. ```lua -local t = {} -- {} -t.x = 1 -- {x: number} -t.y = 2 -- {x: number, y: number} +local t = {x = 1} -- {x: number} +t.y = 2 -- {x: number, y: number} +t.z = 3 -- {x: number, y: number, z: number} ``` -However, if this local were written as `local t: {} = {}`, it ends up sealing the table, so the two assignments henceforth will not be ok. +However, if this local were written as `local t: { x: number } = { x = 1 }`, it ends up sealing the table, so the two assignments henceforth will not be ok. Furthermore, once we exit the scope where this unsealed table was created in, we seal it. @@ -128,16 +128,25 @@ local v2 = vec2(1, 2) v2.z = 3 -- not ok ``` -### Sealed tables - -A sealed table is a table that is now locked down. This occurs when the table constructor literal had 1 or more expression, or when the table type is spelt out explicitly via a type annotation. +Unsealed tables are *exact* in that any property of the table must be named by the type. Since Luau treats missing properties as having value `nil`, this means that we can treat an unsealed table which does not mention a property as if it mentioned the property, as long as that property is optional. ```lua -local t = {x = 1} -- {x: number} -t.y = 2 -- not ok +local t = {x = 1} +local u : { x : number, y : number? } = t -- ok because y is optional +local v : { x : number, z : number } = t -- not ok because z is not optional ``` -Sealed tables support *width subtyping*, which allows a table with more properties to be used as a table with fewer +### Sealed tables + +A sealed table is a table that is now locked down. This occurs when the table type is spelled out explicitly via a type annotation, or if it is returned from a function. + +```lua +local t : { x: number } = {x = 1} +t.y = 2 -- not ok +``` + +Sealed tables are *inexact* in that the table may have properties which are not mentioned in the type. +As a result, sealed tables support *width subtyping*, which allows a table with more properties to be used as a table with fewer ```lua type Point1D = { x : number } From dbcd5fb28ec1dd63fdcfb666d9034d67f06e8e13 Mon Sep 17 00:00:00 2001 From: Allan N Jeremy Date: Thu, 7 Jul 2022 18:21:40 +0300 Subject: [PATCH 306/399] build: changed data output to be different for each os (#581) - macos: data-macos-latest.json - ubuntu: data-ubuntu-latest.json - windows: data-windows-latest.json --- .github/workflows/benchmark-dev.yml | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/.github/workflows/benchmark-dev.yml b/.github/workflows/benchmark-dev.yml index 03ff9d6d..2c6eae4b 100644 --- a/.github/workflows/benchmark-dev.yml +++ b/.github/workflows/benchmark-dev.yml @@ -75,7 +75,7 @@ jobs: name: ${{ matrix.bench.title }} (Windows ${{matrix.arch}}) tool: "benchmarkluau" output-file-path: ./${{ matrix.bench.script }}-output.txt - external-data-json-path: ./gh-pages/dev/bench/data.json + external-data-json-path: ./gh-pages/dev/bench/data-${{ matrix.os }}.json github-token: ${{ secrets.GITHUB_TOKEN }} - name: Push benchmark results @@ -85,7 +85,7 @@ jobs: cd gh-pages git config user.name github-actions git config user.email github@users.noreply.github.com - git add ./dev/bench/data.json + git add ./dev/bench/data-${{ matrix.os }}.json git commit -m "Add benchmarks results for ${{ github.sha }}" git push cd .. @@ -156,7 +156,7 @@ jobs: name: ${{ matrix.bench.title }} tool: "benchmarkluau" output-file-path: ./${{ matrix.bench.script }}-output.txt - external-data-json-path: ./gh-pages/dev/bench/data.json + external-data-json-path: ./gh-pages/dev/bench/data-${{ matrix.os }}.json github-token: ${{ secrets.BENCH_GITHUB_TOKEN }} - name: Store ${{ matrix.bench.title }} result (CacheGrind) @@ -166,7 +166,7 @@ jobs: name: ${{ matrix.bench.title }} (CacheGrind) tool: "roblox" output-file-path: ./${{ matrix.bench.script }}-output.txt - external-data-json-path: ./gh-pages/dev/bench/data.json + external-data-json-path: ./gh-pages/dev/bench/data-${{ matrix.os }}.json github-token: ${{ secrets.BENCH_GITHUB_TOKEN }} - name: Push benchmark results @@ -176,7 +176,7 @@ jobs: cd gh-pages git config user.name github-actions git config user.email github@users.noreply.github.com - git add ./dev/bench/data.json + git add ./dev/bench/data-${{ matrix.os }}.json git commit -m "Add benchmarks results for ${{ github.sha }}" git push cd .. @@ -244,7 +244,7 @@ jobs: gh-pages-branch: "main" output-file-path: ./${{ matrix.bench.script }}-output.txt - external-data-json-path: ./gh-pages/dev/bench/data.json + external-data-json-path: ./gh-pages/dev/bench/data-${{ matrix.os }}.json github-token: ${{ secrets.BENCH_GITHUB_TOKEN }} - name: Store ${{ matrix.bench.title }} result (CacheGrind) @@ -254,7 +254,7 @@ jobs: tool: "roblox" gh-pages-branch: "main" output-file-path: ./${{ matrix.bench.script }}-output.txt - external-data-json-path: ./gh-pages/dev/bench/data.json + external-data-json-path: ./gh-pages/dev/bench/data-${{ matrix.os }}.json github-token: ${{ secrets.BENCH_GITHUB_TOKEN }} - name: Push benchmark results @@ -264,7 +264,7 @@ jobs: cd gh-pages git config user.name github-actions git config user.email github@users.noreply.github.com - git add ./dev/bench/data.json + git add ./dev/bench/data-${{ matrix.os }}.json git commit -m "Add benchmarks results for ${{ github.sha }}" git push cd .. From 4a95f2201ec1f879dfa3f1b8bc015e0a30df005a Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Thu, 7 Jul 2022 18:05:31 -0700 Subject: [PATCH 307/399] Sync to upstream/release/535 --- Analysis/include/Luau/AstQuery.h | 1 + Analysis/include/Luau/TypeInfer.h | 25 +- Analysis/include/Luau/TypePack.h | 1 + Analysis/include/Luau/TypeVar.h | 157 +- Analysis/include/Luau/VisitTypeVar.h | 8 + Analysis/src/AstQuery.cpp | 107 +- Analysis/src/Autocomplete.cpp | 201 +- Analysis/src/BuiltinDefinitions.cpp | 43 +- Analysis/src/Clone.cpp | 12 + Analysis/src/EmbeddedBuiltinDefinitions.cpp | 9 +- Analysis/src/Frontend.cpp | 2 + Analysis/src/Normalize.cpp | 38 +- Analysis/src/Substitution.cpp | 10 +- Analysis/src/ToString.cpp | 23 +- Analysis/src/Transpiler.cpp | 20 +- Analysis/src/TxnLog.cpp | 59 +- Analysis/src/TypeAttach.cpp | 8 + Analysis/src/TypeInfer.cpp | 420 +++- Analysis/src/TypePack.cpp | 51 +- Analysis/src/TypeUtils.cpp | 2 +- Analysis/src/TypeVar.cpp | 269 ++- Analysis/src/Unifier.cpp | 66 +- Ast/src/Parser.cpp | 17 +- CLI/Analyze.cpp | 18 +- CLI/Repl.cpp | 37 + CodeGen/include/Luau/AssemblyBuilderX64.h | 3 + CodeGen/src/AssemblyBuilderX64.cpp | 23 + Makefile | 4 + Sources.cmake | 8 +- VM/src/ltable.cpp | 8 +- VM/src/lvmexecute.cpp | 66 +- bench/bench.py | 37 +- bench/bench_support.lua | 10 + bench/other/LuauPolyfillMap.lua | 961 +++++++++ bench/other/regex.lua | 2089 +++++++++++++++++++ tests/AssemblyBuilderX64.test.cpp | 27 + tests/AstQuery.test.cpp | 33 + tests/Fixture.h | 1 + tests/Frontend.test.cpp | 2 + tests/Module.test.cpp | 2 - tests/Normalize.test.cpp | 42 +- tests/Parser.test.cpp | 1 - tests/ToString.test.cpp | 33 +- tests/Transpiler.test.cpp | 2 +- tests/TypeInfer.anyerror.test.cpp | 8 +- tests/TypeInfer.builtins.test.cpp | 143 +- tests/TypeInfer.functions.test.cpp | 54 +- tests/TypeInfer.generics.test.cpp | 39 +- tests/TypeInfer.loops.test.cpp | 2 +- tests/TypeInfer.modules.test.cpp | 45 +- tests/TypeInfer.operators.test.cpp | 22 + tests/TypeInfer.primitives.test.cpp | 2 +- tests/TypeInfer.provisional.test.cpp | 15 +- tests/TypeInfer.refinements.test.cpp | 45 +- tests/TypeInfer.tables.test.cpp | 14 + tests/TypeInfer.test.cpp | 33 +- tests/TypeInfer.tryUnify.test.cpp | 4 +- tests/TypeInfer.unionTypes.test.cpp | 2 +- tests/TypeInfer.unknownnever.test.cpp | 280 +++ tests/TypePack.test.cpp | 2 - tests/TypeVar.test.cpp | 2 - tests/conformance/vector.lua | 16 + tools/natvis/VM.natvis | 32 +- 63 files changed, 5097 insertions(+), 619 deletions(-) create mode 100644 bench/other/LuauPolyfillMap.lua create mode 100644 bench/other/regex.lua create mode 100644 tests/TypeInfer.unknownnever.test.cpp diff --git a/Analysis/include/Luau/AstQuery.h b/Analysis/include/Luau/AstQuery.h index dfe373a5..950a19da 100644 --- a/Analysis/include/Luau/AstQuery.h +++ b/Analysis/include/Luau/AstQuery.h @@ -63,6 +63,7 @@ private: AstLocal* local = nullptr; }; +std::vector findAncestryAtPositionForAutocomplete(const SourceModule& source, Position pos); std::vector findAstAncestryOfPosition(const SourceModule& source, Position pos); AstNode* findNodeAtPosition(const SourceModule& source, Position pos); AstExpr* findExprAtPosition(const SourceModule& source, Position pos); diff --git a/Analysis/include/Luau/TypeInfer.h b/Analysis/include/Luau/TypeInfer.h index 455654d9..3fb710bb 100644 --- a/Analysis/include/Luau/TypeInfer.h +++ b/Analysis/include/Luau/TypeInfer.h @@ -153,7 +153,7 @@ struct TypeChecker const ScopePtr& scope, const AstExprBinary& expr, TypeId lhsType, TypeId rhsType, const PredicateVec& predicates = {}); TypeId checkBinaryOperation( const ScopePtr& scope, const AstExprBinary& expr, TypeId lhsType, TypeId rhsType, const PredicateVec& predicates = {}); - WithPredicate checkExpr(const ScopePtr& scope, const AstExprBinary& expr); + WithPredicate checkExpr(const ScopePtr& scope, const AstExprBinary& expr, std::optional expectedType = std::nullopt); WithPredicate checkExpr(const ScopePtr& scope, const AstExprTypeAssertion& expr); WithPredicate checkExpr(const ScopePtr& scope, const AstExprError& expr); WithPredicate checkExpr(const ScopePtr& scope, const AstExprIfElse& expr, std::optional expectedType = std::nullopt); @@ -180,8 +180,12 @@ struct TypeChecker const ScopePtr& scope, Unifier& state, TypePackId paramPack, TypePackId argPack, const std::vector& argLocations); WithPredicate checkExprPack(const ScopePtr& scope, const AstExpr& expr); - WithPredicate checkExprPack(const ScopePtr& scope, const AstExprCall& expr); + + WithPredicate checkExprPackHelper(const ScopePtr& scope, const AstExpr& expr); + WithPredicate checkExprPackHelper(const ScopePtr& scope, const AstExprCall& expr); + std::vector> getExpectedTypesForCall(const std::vector& overloads, size_t argumentCount, bool selfCall); + std::optional> checkCallOverload(const ScopePtr& scope, const AstExprCall& expr, TypeId fn, TypePackId retPack, TypePackId argPack, TypePack* args, const std::vector* argLocations, const WithPredicate& argListResult, std::vector& overloadsThatMatchArgCount, std::vector& overloadsThatDont, std::vector& errors); @@ -236,10 +240,11 @@ struct TypeChecker void unifyLowerBound(TypePackId subTy, TypePackId superTy, TypeLevel demotedLevel, const Location& location); - std::optional findMetatableEntry(TypeId type, std::string entry, const Location& location); - std::optional findTablePropertyRespectingMeta(TypeId lhsType, Name name, const Location& location); + std::optional findMetatableEntry(TypeId type, std::string entry, const Location& location, bool addErrors); + std::optional findTablePropertyRespectingMeta(TypeId lhsType, Name name, const Location& location, bool addErrors); std::optional getIndexTypeFromType(const ScopePtr& scope, TypeId type, const Name& name, const Location& location, bool addErrors); + std::optional getIndexTypeFromTypeImpl(const ScopePtr& scope, TypeId type, const Name& name, const Location& location, bool addErrors); // Reduces the union to its simplest possible shape. // (A | B) | B | C yields A | B | C @@ -316,11 +321,12 @@ private: TypeIdPredicate mkTruthyPredicate(bool sense); - // Returns nullopt if the predicate filters down the TypeId to 0 options. - std::optional filterMap(TypeId type, TypeIdPredicate predicate); + // TODO: Return TypeId only. + std::optional filterMapImpl(TypeId type, TypeIdPredicate predicate); + std::pair, bool> filterMap(TypeId type, TypeIdPredicate predicate); public: - std::optional pickTypesFromSense(TypeId type, bool sense); + std::pair, bool> pickTypesFromSense(TypeId type, bool sense); private: TypeId unionOfTypes(TypeId a, TypeId b, const Location& location, bool unifyFreeTypes = true); @@ -345,6 +351,7 @@ private: TypePackId freshTypePack(TypeLevel level); TypeId resolveType(const ScopePtr& scope, const AstType& annotation); + TypeId resolveTypeWorker(const ScopePtr& scope, const AstType& annotation); TypePackId resolveTypePack(const ScopePtr& scope, const AstTypeList& types); TypePackId resolveTypePack(const ScopePtr& scope, const AstTypePack& annotation); TypeId instantiateTypeFun(const ScopePtr& scope, const TypeFun& tf, const std::vector& typeParams, @@ -412,8 +419,12 @@ public: const TypeId booleanType; const TypeId threadType; const TypeId anyType; + const TypeId unknownType; + const TypeId neverType; const TypePackId anyTypePack; + const TypePackId neverTypePack; + const TypePackId uninhabitableTypePack; private: int checkRecursionCount = 0; diff --git a/Analysis/include/Luau/TypePack.h b/Analysis/include/Luau/TypePack.h index c1de242f..b17003b1 100644 --- a/Analysis/include/Luau/TypePack.h +++ b/Analysis/include/Luau/TypePack.h @@ -173,5 +173,6 @@ std::pair, std::optional> flatten(TypePackId tp, bool isVariadic(TypePackId tp); bool isVariadic(TypePackId tp, const TxnLog& log); +bool containsNever(TypePackId tp); } // namespace Luau diff --git a/Analysis/include/Luau/TypeVar.h b/Analysis/include/Luau/TypeVar.h index 6ad6b927..fb6093df 100644 --- a/Analysis/include/Luau/TypeVar.h +++ b/Analysis/include/Luau/TypeVar.h @@ -460,10 +460,18 @@ struct LazyTypeVar std::function thunk; }; +struct UnknownTypeVar +{ +}; + +struct NeverTypeVar +{ +}; + using ErrorTypeVar = Unifiable::Error; using TypeVariant = Unifiable::Variant; + MetatableTypeVar, ClassTypeVar, AnyTypeVar, UnionTypeVar, IntersectionTypeVar, LazyTypeVar, UnknownTypeVar, NeverTypeVar>; struct TypeVar final { @@ -575,8 +583,12 @@ struct SingletonTypes const TypeId trueType; const TypeId falseType; const TypeId anyType; + const TypeId unknownType; + const TypeId neverType; const TypePackId anyTypePack; + const TypePackId neverTypePack; + const TypePackId uninhabitableTypePack; SingletonTypes(); ~SingletonTypes(); @@ -632,12 +644,30 @@ T* getMutable(TypeId tv) return get_if(&asMutable(tv)->ty); } -/* Traverses the UnionTypeVar yielding each TypeId. - * If the iterator encounters a nested UnionTypeVar, it will instead yield each TypeId within. - * - * Beware: the iterator does not currently filter for unique TypeIds. This may change in the future. +const std::vector& getTypes(const UnionTypeVar* utv); +const std::vector& getTypes(const IntersectionTypeVar* itv); +const std::vector& getTypes(const ConstrainedTypeVar* ctv); + +template +struct TypeIterator; + +using UnionTypeVarIterator = TypeIterator; +UnionTypeVarIterator begin(const UnionTypeVar* utv); +UnionTypeVarIterator end(const UnionTypeVar* utv); + +using IntersectionTypeVarIterator = TypeIterator; +IntersectionTypeVarIterator begin(const IntersectionTypeVar* itv); +IntersectionTypeVarIterator end(const IntersectionTypeVar* itv); + +using ConstrainedTypeVarIterator = TypeIterator; +ConstrainedTypeVarIterator begin(const ConstrainedTypeVar* ctv); +ConstrainedTypeVarIterator end(const ConstrainedTypeVar* ctv); + +/* Traverses the type T yielding each TypeId. + * If the iterator encounters a nested type T, it will instead yield each TypeId within. */ -struct UnionTypeVarIterator +template +struct TypeIterator { using value_type = Luau::TypeId; using pointer = value_type*; @@ -645,33 +675,116 @@ struct UnionTypeVarIterator using difference_type = size_t; using iterator_category = std::input_iterator_tag; - explicit UnionTypeVarIterator(const UnionTypeVar* utv); + explicit TypeIterator(const T* t) + { + LUAU_ASSERT(t); - UnionTypeVarIterator& operator++(); - UnionTypeVarIterator operator++(int); - bool operator!=(const UnionTypeVarIterator& rhs); - bool operator==(const UnionTypeVarIterator& rhs); + const std::vector& types = getTypes(t); + if (!types.empty()) + stack.push_front({t, 0}); - const TypeId& operator*(); + seen.insert(t); + } - friend UnionTypeVarIterator end(const UnionTypeVar* utv); + TypeIterator& operator++() + { + advance(); + descend(); + return *this; + } + + TypeIterator operator++(int) + { + TypeIterator copy = *this; + ++copy; + return copy; + } + + bool operator==(const TypeIterator& rhs) const + { + if (!stack.empty() && !rhs.stack.empty()) + return stack.front() == rhs.stack.front(); + + return stack.empty() && rhs.stack.empty(); + } + + bool operator!=(const TypeIterator& rhs) const + { + return !(*this == rhs); + } + + const TypeId& operator*() + { + LUAU_ASSERT(!stack.empty()); + + descend(); + + auto [t, currentIndex] = stack.front(); + LUAU_ASSERT(t); + const std::vector& types = getTypes(t); + LUAU_ASSERT(currentIndex < types.size()); + + const TypeId& ty = types[currentIndex]; + LUAU_ASSERT(!get(follow(ty))); + return ty; + } + + // Normally, we'd have `begin` and `end` be a template but there's too much trouble + // with templates portability in this area, so not worth it. Thanks MSVC. + friend UnionTypeVarIterator end(const UnionTypeVar*); + friend IntersectionTypeVarIterator end(const IntersectionTypeVar*); + friend ConstrainedTypeVarIterator end(const ConstrainedTypeVar*); private: - UnionTypeVarIterator() = default; + TypeIterator() = default; - // (UnionTypeVar* utv, size_t currentIndex) - using SavedIterInfo = std::pair; + // (T* t, size_t currentIndex) + using SavedIterInfo = std::pair; std::deque stack; - std::unordered_set seen; // Only needed to protect the iterator from hanging the thread. + std::unordered_set seen; // Only needed to protect the iterator from hanging the thread. - void advance(); - void descend(); + void advance() + { + while (!stack.empty()) + { + auto& [t, currentIndex] = stack.front(); + ++currentIndex; + + const std::vector& types = getTypes(t); + if (currentIndex >= types.size()) + stack.pop_front(); + else + break; + } + } + + void descend() + { + while (!stack.empty()) + { + auto [current, currentIndex] = stack.front(); + const std::vector& types = getTypes(current); + if (auto inner = get(follow(types[currentIndex]))) + { + // If we're about to descend into a cyclic type, we should skip over this. + // Ideally this should never happen, but alas it does from time to time. :( + if (seen.find(inner) != seen.end()) + advance(); + else + { + seen.insert(inner); + stack.push_front({inner, 0}); + } + + continue; + } + + break; + } + } }; -UnionTypeVarIterator begin(const UnionTypeVar* utv); -UnionTypeVarIterator end(const UnionTypeVar* utv); - using TypeIdPredicate = std::function(TypeId)>; std::vector filterMap(TypeId type, TypeIdPredicate predicate); diff --git a/Analysis/include/Luau/VisitTypeVar.h b/Analysis/include/Luau/VisitTypeVar.h index 5fd43f0b..ab4a397d 100644 --- a/Analysis/include/Luau/VisitTypeVar.h +++ b/Analysis/include/Luau/VisitTypeVar.h @@ -129,6 +129,14 @@ struct GenericTypeVarVisitor { return visit(ty); } + virtual bool visit(TypeId ty, const UnknownTypeVar& atv) + { + return visit(ty); + } + virtual bool visit(TypeId ty, const NeverTypeVar& atv) + { + return visit(ty); + } virtual bool visit(TypeId ty, const UnionTypeVar& utv) { return visit(ty); diff --git a/Analysis/src/AstQuery.cpp b/Analysis/src/AstQuery.cpp index 0522b1fa..1124c29e 100644 --- a/Analysis/src/AstQuery.cpp +++ b/Analysis/src/AstQuery.cpp @@ -17,6 +17,104 @@ namespace Luau namespace { + +struct AutocompleteNodeFinder : public AstVisitor +{ + const Position pos; + std::vector ancestry; + + explicit AutocompleteNodeFinder(Position pos, AstNode* root) + : pos(pos) + { + } + + bool visit(AstExpr* expr) override + { + if (expr->location.begin < pos && pos <= expr->location.end) + { + ancestry.push_back(expr); + return true; + } + return false; + } + + bool visit(AstStat* stat) override + { + if (stat->location.begin < pos && pos <= stat->location.end) + { + ancestry.push_back(stat); + return true; + } + return false; + } + + bool visit(AstType* type) override + { + if (type->location.begin < pos && pos <= type->location.end) + { + ancestry.push_back(type); + return true; + } + return false; + } + + bool visit(AstTypeError* type) override + { + // For a missing type, match the whole range including the start position + if (type->isMissing && type->location.containsClosed(pos)) + { + ancestry.push_back(type); + return true; + } + return false; + } + + bool visit(class AstTypePack* typePack) override + { + return true; + } + + bool visit(AstStatBlock* block) override + { + // If ancestry is empty, we are inspecting the root of the AST. Its extent is considered to be infinite. + if (ancestry.empty()) + { + ancestry.push_back(block); + return true; + } + + // AstExprIndexName nodes are nested outside-in, so we want the outermost node in the case of nested nodes. + // ex foo.bar.baz is represented in the AST as IndexName{ IndexName {foo, bar}, baz} + if (!ancestry.empty() && ancestry.back()->is()) + return false; + + // Type annotation error might intersect the block statement when the function header is being written, + // annotation takes priority + if (!ancestry.empty() && ancestry.back()->is()) + return false; + + // If the cursor is at the end of an expression or type and simultaneously at the beginning of a block, + // the expression or type wins out. + // The exception to this is if we are in a block under an AstExprFunction. In this case, we consider the position to + // be within the block. + if (block->location.begin == pos && !ancestry.empty()) + { + if (ancestry.back()->asExpr() && !ancestry.back()->is()) + return false; + + if (ancestry.back()->asType()) + return false; + } + + if (block->location.begin <= pos && pos <= block->location.end) + { + ancestry.push_back(block); + return true; + } + return false; + } +}; + struct FindNode : public AstVisitor { const Position pos; @@ -102,6 +200,13 @@ struct FindFullAncestry final : public AstVisitor } // namespace +std::vector findAncestryAtPositionForAutocomplete(const SourceModule& source, Position pos) +{ + AutocompleteNodeFinder finder{pos, source.root}; + source.root->visit(&finder); + return finder.ancestry; +} + std::vector findAstAncestryOfPosition(const SourceModule& source, Position pos) { const Position end = source.root->location.end; @@ -110,7 +215,7 @@ std::vector findAstAncestryOfPosition(const SourceModule& source, Posi FindFullAncestry finder(pos, end); source.root->visit(&finder); - return std::move(finder.nodes); + return finder.nodes; } AstNode* findNodeAtPosition(const SourceModule& source, Position pos) diff --git a/Analysis/src/Autocomplete.cpp b/Analysis/src/Autocomplete.cpp index 8a63901f..cc54d499 100644 --- a/Analysis/src/Autocomplete.cpp +++ b/Analysis/src/Autocomplete.cpp @@ -21,102 +21,6 @@ static const std::unordered_set kStatementStartingKeywords = { namespace Luau { -struct NodeFinder : public AstVisitor -{ - const Position pos; - std::vector ancestry; - - explicit NodeFinder(Position pos, AstNode* root) - : pos(pos) - { - } - - bool visit(AstExpr* expr) override - { - if (expr->location.begin < pos && pos <= expr->location.end) - { - ancestry.push_back(expr); - return true; - } - return false; - } - - bool visit(AstStat* stat) override - { - if (stat->location.begin < pos && pos <= stat->location.end) - { - ancestry.push_back(stat); - return true; - } - return false; - } - - bool visit(AstType* type) override - { - if (type->location.begin < pos && pos <= type->location.end) - { - ancestry.push_back(type); - return true; - } - return false; - } - - bool visit(AstTypeError* type) override - { - // For a missing type, match the whole range including the start position - if (type->isMissing && type->location.containsClosed(pos)) - { - ancestry.push_back(type); - return true; - } - return false; - } - - bool visit(class AstTypePack* typePack) override - { - return true; - } - - bool visit(AstStatBlock* block) override - { - // If ancestry is empty, we are inspecting the root of the AST. Its extent is considered to be infinite. - if (ancestry.empty()) - { - ancestry.push_back(block); - return true; - } - - // AstExprIndexName nodes are nested outside-in, so we want the outermost node in the case of nested nodes. - // ex foo.bar.baz is represented in the AST as IndexName{ IndexName {foo, bar}, baz} - if (!ancestry.empty() && ancestry.back()->is()) - return false; - - // Type annotation error might intersect the block statement when the function header is being written, - // annotation takes priority - if (!ancestry.empty() && ancestry.back()->is()) - return false; - - // If the cursor is at the end of an expression or type and simultaneously at the beginning of a block, - // the expression or type wins out. - // The exception to this is if we are in a block under an AstExprFunction. In this case, we consider the position to - // be within the block. - if (block->location.begin == pos && !ancestry.empty()) - { - if (ancestry.back()->asExpr() && !ancestry.back()->is()) - return false; - - if (ancestry.back()->asType()) - return false; - } - - if (block->location.begin <= pos && pos <= block->location.end) - { - ancestry.push_back(block); - return true; - } - return false; - } -}; static bool alreadyHasParens(const std::vector& nodes) { @@ -905,7 +809,7 @@ AutocompleteEntryMap autocompleteTypeNames(const Module& module, Position positi } AstNode* parent = nullptr; - AstType* topType = nullptr; + AstType* topType = nullptr; // TODO: rename? for (auto it = ancestry.rbegin(), e = ancestry.rend(); it != e; ++it) { @@ -1477,21 +1381,20 @@ static AutocompleteResult autocomplete(const SourceModule& sourceModule, const M if (isWithinComment(sourceModule, position)) return {}; - NodeFinder finder{position, sourceModule.root}; - sourceModule.root->visit(&finder); - LUAU_ASSERT(!finder.ancestry.empty()); - AstNode* node = finder.ancestry.back(); + std::vector ancestry = findAncestryAtPositionForAutocomplete(sourceModule, position); + LUAU_ASSERT(!ancestry.empty()); + AstNode* node = ancestry.back(); AstExprConstantNil dummy{Location{}}; - AstNode* parent = finder.ancestry.size() >= 2 ? finder.ancestry.rbegin()[1] : &dummy; + AstNode* parent = ancestry.size() >= 2 ? ancestry.rbegin()[1] : &dummy; // If we are inside a body of a function that doesn't have a completed argument list, ignore the body node if (auto exprFunction = parent->as(); exprFunction && !exprFunction->argLocation && node == exprFunction->body) { - finder.ancestry.pop_back(); + ancestry.pop_back(); - node = finder.ancestry.back(); - parent = finder.ancestry.size() >= 2 ? finder.ancestry.rbegin()[1] : &dummy; + node = ancestry.back(); + parent = ancestry.size() >= 2 ? ancestry.rbegin()[1] : &dummy; } if (auto indexName = node->as()) @@ -1504,47 +1407,47 @@ static AutocompleteResult autocomplete(const SourceModule& sourceModule, const M PropIndexType indexType = indexName->op == ':' ? PropIndexType::Colon : PropIndexType::Point; if (!FFlag::LuauSelfCallAutocompleteFix2 && isString(ty)) - return {autocompleteProps(*module, typeArena, typeChecker.globalScope->bindings[AstName{"string"}].typeId, indexType, finder.ancestry), - finder.ancestry}; + return {autocompleteProps(*module, typeArena, typeChecker.globalScope->bindings[AstName{"string"}].typeId, indexType, ancestry), + ancestry}; else - return {autocompleteProps(*module, typeArena, ty, indexType, finder.ancestry), finder.ancestry}; + return {autocompleteProps(*module, typeArena, ty, indexType, ancestry), ancestry}; } else if (auto typeReference = node->as()) { if (typeReference->prefix) - return {autocompleteModuleTypes(*module, position, typeReference->prefix->value), finder.ancestry}; + return {autocompleteModuleTypes(*module, position, typeReference->prefix->value), ancestry}; else - return {autocompleteTypeNames(*module, position, finder.ancestry), finder.ancestry}; + return {autocompleteTypeNames(*module, position, ancestry), ancestry}; } else if (node->is()) { - return {autocompleteTypeNames(*module, position, finder.ancestry), finder.ancestry}; + return {autocompleteTypeNames(*module, position, ancestry), ancestry}; } else if (AstStatLocal* statLocal = node->as()) { if (statLocal->vars.size == 1 && (!statLocal->equalsSignLocation || position < statLocal->equalsSignLocation->begin)) - return {{{"function", AutocompleteEntry{AutocompleteEntryKind::Keyword}}}, finder.ancestry}; + return {{{"function", AutocompleteEntry{AutocompleteEntryKind::Keyword}}}, ancestry}; else if (statLocal->equalsSignLocation && position >= statLocal->equalsSignLocation->end) - return {autocompleteExpression(sourceModule, *module, typeChecker, typeArena, finder.ancestry, position), finder.ancestry}; + return {autocompleteExpression(sourceModule, *module, typeChecker, typeArena, ancestry, position), ancestry}; else return {}; } - else if (AstStatFor* statFor = extractStat(finder.ancestry)) + else if (AstStatFor* statFor = extractStat(ancestry)) { if (!statFor->hasDo || position < statFor->doLocation.begin) { if (!statFor->from->is() && !statFor->to->is() && (!statFor->step || !statFor->step->is())) - return {{{"do", AutocompleteEntry{AutocompleteEntryKind::Keyword}}}, finder.ancestry}; + return {{{"do", AutocompleteEntry{AutocompleteEntryKind::Keyword}}}, ancestry}; if (statFor->from->location.containsClosed(position) || statFor->to->location.containsClosed(position) || (statFor->step && statFor->step->location.containsClosed(position))) - return {autocompleteExpression(sourceModule, *module, typeChecker, typeArena, finder.ancestry, position), finder.ancestry}; + return {autocompleteExpression(sourceModule, *module, typeChecker, typeArena, ancestry, position), ancestry}; return {}; } - return {autocompleteStatement(sourceModule, *module, finder.ancestry, position), finder.ancestry}; + return {autocompleteStatement(sourceModule, *module, ancestry, position), ancestry}; } else if (AstStatForIn* statForIn = parent->as(); statForIn && (node->is() || isIdentifier(node))) @@ -1560,7 +1463,7 @@ static AutocompleteResult autocomplete(const SourceModule& sourceModule, const M return {}; } - return {{{"in", AutocompleteEntry{AutocompleteEntryKind::Keyword}}}, finder.ancestry}; + return {{{"in", AutocompleteEntry{AutocompleteEntryKind::Keyword}}}, ancestry}; } if (!statForIn->hasDo || position <= statForIn->doLocation.begin) @@ -1569,58 +1472,58 @@ static AutocompleteResult autocomplete(const SourceModule& sourceModule, const M AstExpr* lastExpr = statForIn->values.data[statForIn->values.size - 1]; if (lastExpr->location.containsClosed(position)) - return {autocompleteExpression(sourceModule, *module, typeChecker, typeArena, finder.ancestry, position), finder.ancestry}; + return {autocompleteExpression(sourceModule, *module, typeChecker, typeArena, ancestry, position), ancestry}; if (position > lastExpr->location.end) - return {{{"do", AutocompleteEntry{AutocompleteEntryKind::Keyword}}}, finder.ancestry}; + return {{{"do", AutocompleteEntry{AutocompleteEntryKind::Keyword}}}, ancestry}; return {}; // Not sure what this means } } - else if (AstStatForIn* statForIn = extractStat(finder.ancestry)) + else if (AstStatForIn* statForIn = extractStat(ancestry)) { // The AST looks a bit differently if the cursor is at a position where only the "do" keyword is allowed. // ex "for f in f do" if (!statForIn->hasDo) - return {{{"do", AutocompleteEntry{AutocompleteEntryKind::Keyword}}}, finder.ancestry}; + return {{{"do", AutocompleteEntry{AutocompleteEntryKind::Keyword}}}, ancestry}; - return {autocompleteStatement(sourceModule, *module, finder.ancestry, position), finder.ancestry}; + return {autocompleteStatement(sourceModule, *module, ancestry, position), ancestry}; } else if (AstStatWhile* statWhile = parent->as(); node->is() && statWhile) { if (!statWhile->hasDo && !statWhile->condition->is() && position > statWhile->condition->location.end) - return {{{"do", AutocompleteEntry{AutocompleteEntryKind::Keyword}}}, finder.ancestry}; + return {{{"do", AutocompleteEntry{AutocompleteEntryKind::Keyword}}}, ancestry}; if (!statWhile->hasDo || position < statWhile->doLocation.begin) - return {autocompleteExpression(sourceModule, *module, typeChecker, typeArena, finder.ancestry, position), finder.ancestry}; + return {autocompleteExpression(sourceModule, *module, typeChecker, typeArena, ancestry, position), ancestry}; if (statWhile->hasDo && position > statWhile->doLocation.end) - return {autocompleteStatement(sourceModule, *module, finder.ancestry, position), finder.ancestry}; + return {autocompleteStatement(sourceModule, *module, ancestry, position), ancestry}; } - else if (AstStatWhile* statWhile = extractStat(finder.ancestry); statWhile && !statWhile->hasDo) - return {{{"do", AutocompleteEntry{AutocompleteEntryKind::Keyword}}}, finder.ancestry}; + else if (AstStatWhile* statWhile = extractStat(ancestry); statWhile && !statWhile->hasDo) + return {{{"do", AutocompleteEntry{AutocompleteEntryKind::Keyword}}}, ancestry}; else if (AstStatIf* statIf = node->as(); statIf && !statIf->elseLocation.has_value()) { return {{{"else", AutocompleteEntry{AutocompleteEntryKind::Keyword}}, {"elseif", AutocompleteEntry{AutocompleteEntryKind::Keyword}}}, - finder.ancestry}; + ancestry}; } else if (AstStatIf* statIf = parent->as(); statIf && node->is()) { if (statIf->condition->is()) - return {autocompleteExpression(sourceModule, *module, typeChecker, typeArena, finder.ancestry, position), finder.ancestry}; + return {autocompleteExpression(sourceModule, *module, typeChecker, typeArena, ancestry, position), ancestry}; else if (!statIf->thenLocation || statIf->thenLocation->containsClosed(position)) - return {{{"then", AutocompleteEntry{AutocompleteEntryKind::Keyword}}}, finder.ancestry}; + return {{{"then", AutocompleteEntry{AutocompleteEntryKind::Keyword}}}, ancestry}; } - else if (AstStatIf* statIf = extractStat(finder.ancestry); + else if (AstStatIf* statIf = extractStat(ancestry); statIf && (!statIf->thenLocation || statIf->thenLocation->containsClosed(position))) - return {{{"then", AutocompleteEntry{AutocompleteEntryKind::Keyword}}}, finder.ancestry}; + return {{{"then", AutocompleteEntry{AutocompleteEntryKind::Keyword}}}, ancestry}; else if (AstStatRepeat* statRepeat = node->as(); statRepeat && statRepeat->condition->is()) - return {autocompleteExpression(sourceModule, *module, typeChecker, typeArena, finder.ancestry, position), finder.ancestry}; - else if (AstStatRepeat* statRepeat = extractStat(finder.ancestry); statRepeat) - return {autocompleteStatement(sourceModule, *module, finder.ancestry, position), finder.ancestry}; + return {autocompleteExpression(sourceModule, *module, typeChecker, typeArena, ancestry, position), ancestry}; + else if (AstStatRepeat* statRepeat = extractStat(ancestry); statRepeat) + return {autocompleteStatement(sourceModule, *module, ancestry, position), ancestry}; else if (AstExprTable* exprTable = parent->as(); exprTable && (node->is() || node->is())) { for (const auto& [kind, key, value] : exprTable->items) @@ -1630,7 +1533,7 @@ static AutocompleteResult autocomplete(const SourceModule& sourceModule, const M { if (auto it = module->astExpectedTypes.find(exprTable)) { - auto result = autocompleteProps(*module, typeArena, *it, PropIndexType::Key, finder.ancestry); + auto result = autocompleteProps(*module, typeArena, *it, PropIndexType::Key, ancestry); // Remove keys that are already completed for (const auto& item : exprTable->items) @@ -1644,9 +1547,9 @@ static AutocompleteResult autocomplete(const SourceModule& sourceModule, const M // If we know for sure that a key is being written, do not offer general expression suggestions if (!key) - autocompleteExpression(sourceModule, *module, typeChecker, typeArena, finder.ancestry, position, result); + autocompleteExpression(sourceModule, *module, typeChecker, typeArena, ancestry, position, result); - return {result, finder.ancestry}; + return {result, ancestry}; } break; @@ -1654,11 +1557,11 @@ static AutocompleteResult autocomplete(const SourceModule& sourceModule, const M } } else if (isIdentifier(node) && (parent->is() || parent->is())) - return {autocompleteStatement(sourceModule, *module, finder.ancestry, position), finder.ancestry}; + return {autocompleteStatement(sourceModule, *module, ancestry, position), ancestry}; - if (std::optional ret = autocompleteStringParams(sourceModule, module, finder.ancestry, position, callback)) + if (std::optional ret = autocompleteStringParams(sourceModule, module, ancestry, position, callback)) { - return {*ret, finder.ancestry}; + return {*ret, ancestry}; } else if (node->is()) { @@ -1667,14 +1570,14 @@ static AutocompleteResult autocomplete(const SourceModule& sourceModule, const M if (auto it = module->astExpectedTypes.find(node->asExpr())) autocompleteStringSingleton(*it, false, result); - if (finder.ancestry.size() >= 2) + if (ancestry.size() >= 2) { - if (auto idxExpr = finder.ancestry.at(finder.ancestry.size() - 2)->as()) + if (auto idxExpr = ancestry.at(ancestry.size() - 2)->as()) { if (auto it = module->astTypes.find(idxExpr->expr)) - autocompleteProps(*module, typeArena, follow(*it), PropIndexType::Point, finder.ancestry, result); + autocompleteProps(*module, typeArena, follow(*it), PropIndexType::Point, ancestry, result); } - else if (auto binExpr = finder.ancestry.at(finder.ancestry.size() - 2)->as()) + else if (auto binExpr = ancestry.at(ancestry.size() - 2)->as()) { if (binExpr->op == AstExprBinary::CompareEq || binExpr->op == AstExprBinary::CompareNe) { @@ -1684,7 +1587,7 @@ static AutocompleteResult autocomplete(const SourceModule& sourceModule, const M } } - return {result, finder.ancestry}; + return {result, ancestry}; } if (node->is()) @@ -1693,9 +1596,9 @@ static AutocompleteResult autocomplete(const SourceModule& sourceModule, const M } if (node->asExpr()) - return {autocompleteExpression(sourceModule, *module, typeChecker, typeArena, finder.ancestry, position), finder.ancestry}; + return {autocompleteExpression(sourceModule, *module, typeChecker, typeArena, ancestry, position), ancestry}; else if (node->asStat()) - return {autocompleteStatement(sourceModule, *module, finder.ancestry, position), finder.ancestry}; + return {autocompleteStatement(sourceModule, *module, ancestry, position), ancestry}; return {}; } diff --git a/Analysis/src/BuiltinDefinitions.cpp b/Analysis/src/BuiltinDefinitions.cpp index 2f57e23c..aeba2c13 100644 --- a/Analysis/src/BuiltinDefinitions.cpp +++ b/Analysis/src/BuiltinDefinitions.cpp @@ -9,6 +9,7 @@ #include LUAU_FASTFLAGVARIABLE(LuauSetMetaTableArgsCheck, false) +LUAU_FASTFLAG(LuauUnknownAndNeverType) /** FIXME: Many of these type definitions are not quite completely accurate. * @@ -222,14 +223,14 @@ void registerBuiltinTypes(TypeChecker& typeChecker) addGlobalBinding(typeChecker, "getmetatable", makeFunction(arena, std::nullopt, {genericMT}, {}, {tableMetaMT}, {genericMT}), "@luau"); - // setmetatable({ @metatable MT }, MT) -> { @metatable MT } // clang-format off + // setmetatable(T, MT) -> { @metatable MT, T } addGlobalBinding(typeChecker, "setmetatable", arena.addType( FunctionTypeVar{ {genericMT}, {}, - arena.addTypePack(TypePack{{tableMetaMT, genericMT}}), + arena.addTypePack(TypePack{{FFlag::LuauUnknownAndNeverType ? tabTy : tableMetaMT, genericMT}}), arena.addTypePack(TypePack{{tableMetaMT}}) } ), "@luau" @@ -309,6 +310,12 @@ static std::optional> magicFunctionSetMetaTable( { auto [paramPack, _predicates] = withPredicate; + if (FFlag::LuauUnknownAndNeverType) + { + if (size(paramPack) < 2 && finite(paramPack)) + return std::nullopt; + } + TypeArena& arena = typechecker.currentModule->internalTypes; std::vector expectedArgs = typechecker.unTypePack(scope, paramPack, 2, expr.location); @@ -316,6 +323,12 @@ static std::optional> magicFunctionSetMetaTable( TypeId target = follow(expectedArgs[0]); TypeId mt = follow(expectedArgs[1]); + if (FFlag::LuauUnknownAndNeverType) + { + typechecker.tablify(target); + typechecker.tablify(mt); + } + if (const auto& tab = get(target)) { if (target->persistent) @@ -324,7 +337,8 @@ static std::optional> magicFunctionSetMetaTable( } else { - typechecker.tablify(mt); + if (!FFlag::LuauUnknownAndNeverType) + typechecker.tablify(mt); const TableTypeVar* mtTtv = get(mt); MetatableTypeVar mtv{target, mt}; @@ -343,7 +357,10 @@ static std::optional> magicFunctionSetMetaTable( if (FFlag::LuauSetMetaTableArgsCheck && expr.args.size < 1) { - return WithPredicate{}; + if (FFlag::LuauUnknownAndNeverType) + return std::nullopt; + else + return WithPredicate{}; } if (!FFlag::LuauSetMetaTableArgsCheck || !expr.self) @@ -390,11 +407,21 @@ static std::optional> magicFunctionAssert( if (head.size() > 0) { - std::optional newhead = typechecker.pickTypesFromSense(head[0], true); - if (!newhead) - head = {typechecker.nilType}; + auto [ty, ok] = typechecker.pickTypesFromSense(head[0], true); + if (FFlag::LuauUnknownAndNeverType) + { + if (get(*ty)) + head = {*ty}; + else + head[0] = *ty; + } else - head[0] = *newhead; + { + if (!ty) + head = {typechecker.nilType}; + else + head[0] = *ty; + } } return WithPredicate{arena.addTypePack(TypePack{std::move(head), tail})}; diff --git a/Analysis/src/Clone.cpp b/Analysis/src/Clone.cpp index df4e0a6b..88c50318 100644 --- a/Analysis/src/Clone.cpp +++ b/Analysis/src/Clone.cpp @@ -59,6 +59,8 @@ struct TypeCloner void operator()(const UnionTypeVar& t); void operator()(const IntersectionTypeVar& t); void operator()(const LazyTypeVar& t); + void operator()(const UnknownTypeVar& t); + void operator()(const NeverTypeVar& t); }; struct TypePackCloner @@ -310,6 +312,16 @@ void TypeCloner::operator()(const LazyTypeVar& t) defaultClone(t); } +void TypeCloner::operator()(const UnknownTypeVar& t) +{ + defaultClone(t); +} + +void TypeCloner::operator()(const NeverTypeVar& t) +{ + defaultClone(t); +} + } // anonymous namespace TypePackId clone(TypePackId tp, TypeArena& dest, CloneState& cloneState) diff --git a/Analysis/src/EmbeddedBuiltinDefinitions.cpp b/Analysis/src/EmbeddedBuiltinDefinitions.cpp index 1b5275fd..f93f65dd 100644 --- a/Analysis/src/EmbeddedBuiltinDefinitions.cpp +++ b/Analysis/src/EmbeddedBuiltinDefinitions.cpp @@ -1,6 +1,7 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #include "Luau/BuiltinDefinitions.h" +LUAU_FASTFLAG(LuauUnknownAndNeverType) LUAU_FASTFLAG(LuauCheckLenMT) namespace Luau @@ -116,8 +117,6 @@ declare function typeof(value: T): string -- `assert` has a magic function attached that will give more detailed type information declare function assert(value: T, errorMessage: string?): T -declare function error(message: T, level: number?) - declare function tostring(value: T): string declare function tonumber(value: T, radix: number?): number? @@ -204,12 +203,18 @@ declare function unpack(tab: {V}, i: number?, j: number?): ...V std::string getBuiltinDefinitionSource() { + std::string result = kBuiltinDefinitionLuaSrc; // TODO: move this into kBuiltinDefinitionLuaSrc if (FFlag::LuauCheckLenMT) result += "declare function rawlen(obj: {[K]: V} | string): number\n"; + if (FFlag::LuauUnknownAndNeverType) + result += "declare function error(message: T, level: number?): never\n"; + else + result += "declare function error(message: T, level: number?)\n"; + return result; } diff --git a/Analysis/src/Frontend.cpp b/Analysis/src/Frontend.cpp index 4cfaa112..9195363d 100644 --- a/Analysis/src/Frontend.cpp +++ b/Analysis/src/Frontend.cpp @@ -496,6 +496,8 @@ CheckResult Frontend::check(const ModuleName& name, std::optionalastTypes.clear(); module->astExpectedTypes.clear(); module->astOriginalCallTypes.clear(); + module->astResolvedTypes.clear(); + module->astResolvedTypePacks.clear(); module->scopes.resize(1); } diff --git a/Analysis/src/Normalize.cpp b/Analysis/src/Normalize.cpp index 8ce7f742..ce8f96cc 100644 --- a/Analysis/src/Normalize.cpp +++ b/Analysis/src/Normalize.cpp @@ -14,7 +14,7 @@ LUAU_FASTFLAGVARIABLE(DebugLuauCopyBeforeNormalizing, false) LUAU_FASTINTVARIABLE(LuauNormalizeIterationLimit, 1200); LUAU_FASTFLAGVARIABLE(LuauNormalizeCombineTableFix, false); LUAU_FASTFLAGVARIABLE(LuauNormalizeFlagIsConservative, false); -LUAU_FASTFLAGVARIABLE(LuauNormalizeCombineEqFix, false); +LUAU_FASTFLAG(LuauUnknownAndNeverType) LUAU_FASTFLAG(LuauQuantifyConstrained) namespace Luau @@ -182,7 +182,6 @@ struct Normalize final : TypeVarVisitor { if (!ty->normal) asMutable(ty)->normal = true; - return false; } @@ -193,6 +192,20 @@ struct Normalize final : TypeVarVisitor return false; } + bool visit(TypeId ty, const UnknownTypeVar&) override + { + if (!ty->normal) + asMutable(ty)->normal = true; + return false; + } + + bool visit(TypeId ty, const NeverTypeVar&) override + { + if (!ty->normal) + asMutable(ty)->normal = true; + return false; + } + bool visit(TypeId ty, const ConstrainedTypeVar& ctvRef) override { CHECK_ITERATION_LIMIT(false); @@ -416,7 +429,13 @@ struct Normalize final : TypeVarVisitor std::vector result; for (TypeId part : options) + { + // AnyTypeVar always win the battle no matter what we do, so we're done. + if (FFlag::LuauUnknownAndNeverType && get(follow(part))) + return {part}; + combineIntoUnion(result, part); + } return result; } @@ -427,7 +446,17 @@ struct Normalize final : TypeVarVisitor if (auto utv = get(ty)) { for (TypeId t : utv) + { + // AnyTypeVar always win the battle no matter what we do, so we're done. + if (FFlag::LuauUnknownAndNeverType && get(t)) + { + result = {t}; + return; + } + combineIntoUnion(result, t); + } + return; } @@ -571,8 +600,7 @@ struct Normalize final : TypeVarVisitor */ TypeId combine(Replacer& replacer, TypeId a, TypeId b) { - if (FFlag::LuauNormalizeCombineEqFix) - b = follow(b); + b = follow(b); if (FFlag::LuauNormalizeCombineTableFix && a == b) return a; @@ -592,7 +620,7 @@ struct Normalize final : TypeVarVisitor } else if (auto ttv = getMutable(a)) { - if (FFlag::LuauNormalizeCombineTableFix && !get(FFlag::LuauNormalizeCombineEqFix ? b : follow(b))) + if (FFlag::LuauNormalizeCombineTableFix && !get(b)) return arena.addType(IntersectionTypeVar{{a, b}}); combineIntoTable(replacer, ttv, b); return a; diff --git a/Analysis/src/Substitution.cpp b/Analysis/src/Substitution.cpp index 9c4ce829..7245403c 100644 --- a/Analysis/src/Substitution.cpp +++ b/Analysis/src/Substitution.cpp @@ -8,8 +8,10 @@ #include #include +LUAU_FASTFLAGVARIABLE(LuauAnyificationMustClone, false) LUAU_FASTFLAG(LuauLowerBoundsCalculation) LUAU_FASTINTVARIABLE(LuauTarjanChildLimit, 10000) +LUAU_FASTFLAG(LuauUnknownAndNeverType) namespace Luau { @@ -154,7 +156,7 @@ TarjanResult Tarjan::loop() if (currEdge == -1) { ++childCount; - if (childLimit > 0 && childLimit < childCount) + if (childLimit > 0 && (FFlag::LuauUnknownAndNeverType ? childLimit <= childCount : childLimit < childCount)) return TarjanResult::TooManyChildren; stack.push_back(index); @@ -439,6 +441,9 @@ void Substitution::replaceChildren(TypeId ty) if (ignoreChildren(ty)) return; + if (FFlag::LuauAnyificationMustClone && ty->owningArena != arena) + return; + if (FunctionTypeVar* ftv = getMutable(ty)) { ftv->argTypes = replace(ftv->argTypes); @@ -490,6 +495,9 @@ void Substitution::replaceChildren(TypePackId tp) if (ignoreChildren(tp)) return; + if (FFlag::LuauAnyificationMustClone && tp->owningArena != arena) + return; + if (TypePack* tpp = getMutable(tp)) { for (TypeId& tv : tpp->head) diff --git a/Analysis/src/ToString.cpp b/Analysis/src/ToString.cpp index fe940d5e..c67d6397 100644 --- a/Analysis/src/ToString.cpp +++ b/Analysis/src/ToString.cpp @@ -11,6 +11,7 @@ #include LUAU_FASTFLAG(LuauLowerBoundsCalculation) +LUAU_FASTFLAG(LuauUnknownAndNeverType) /* * Prefix generic typenames with gen- @@ -699,6 +700,12 @@ struct TypeVarStringifier void operator()(TypeId, const MetatableTypeVar& mtv) { state.result.invalid = true; + if (!state.exhaustive && mtv.syntheticName) + { + state.emit(*mtv.syntheticName); + return; + } + state.emit("{ @metatable "); stringify(mtv.metatable); state.emit(","); @@ -834,7 +841,7 @@ struct TypeVarStringifier void operator()(TypeId, const ErrorTypeVar& tv) { state.result.error = true; - state.emit("*unknown*"); + state.emit(FFlag::LuauUnknownAndNeverType ? "" : "*unknown*"); } void operator()(TypeId, const LazyTypeVar& ltv) @@ -843,7 +850,17 @@ struct TypeVarStringifier state.emit("lazy?"); } -}; // namespace + void operator()(TypeId, const UnknownTypeVar& ttv) + { + state.emit("unknown"); + } + + void operator()(TypeId, const NeverTypeVar& ttv) + { + state.emit("never"); + } + +}; struct TypePackStringifier { @@ -947,7 +964,7 @@ struct TypePackStringifier void operator()(TypePackId, const Unifiable::Error& error) { state.result.error = true; - state.emit("*unknown*"); + state.emit(FFlag::LuauUnknownAndNeverType ? "" : "*unknown*"); } void operator()(TypePackId, const VariadicTypePack& pack) diff --git a/Analysis/src/Transpiler.cpp b/Analysis/src/Transpiler.cpp index 1577bd63..9feff1c0 100644 --- a/Analysis/src/Transpiler.cpp +++ b/Analysis/src/Transpiler.cpp @@ -205,20 +205,6 @@ struct Printer } } - void visualizeWithSelf(AstExpr& expr, bool self) - { - if (!self) - return visualize(expr); - - AstExprIndexName* func = expr.as(); - LUAU_ASSERT(func); - - visualize(*func->expr); - writer.symbol(":"); - advance(func->indexLocation.begin); - writer.identifier(func->index.value); - } - void visualizeTypePackAnnotation(const AstTypePack& annotation, bool forVarArg) { advance(annotation.location.begin); @@ -366,7 +352,7 @@ struct Printer } else if (const auto& a = expr.as()) { - visualizeWithSelf(*a->func, a->self); + visualize(*a->func); writer.symbol("("); bool first = true; @@ -385,7 +371,7 @@ struct Printer else if (const auto& a = expr.as()) { visualize(*a->expr); - writer.symbol("."); + writer.symbol(std::string(1, a->op)); writer.write(a->index.value); } else if (const auto& a = expr.as()) @@ -766,7 +752,7 @@ struct Printer else if (const auto& a = program.as()) { writer.keyword("function"); - visualizeWithSelf(*a->name, a->func->self != nullptr); + visualize(*a->name); visualizeFunctionBody(*a->func); } else if (const auto& a = program.as()) diff --git a/Analysis/src/TxnLog.cpp b/Analysis/src/TxnLog.cpp index 4c6d54e0..b3f60d30 100644 --- a/Analysis/src/TxnLog.cpp +++ b/Analysis/src/TxnLog.cpp @@ -7,7 +7,7 @@ #include #include -LUAU_FASTFLAG(LuauNonCopyableTypeVarFields) +LUAU_FASTFLAG(LuauUnknownAndNeverType) namespace Luau { @@ -81,34 +81,10 @@ void TxnLog::concat(TxnLog rhs) void TxnLog::commit() { for (auto& [ty, rep] : typeVarChanges) - { - if (FFlag::LuauNonCopyableTypeVarFields) - { - asMutable(ty)->reassign(rep.get()->pending); - } - else - { - TypeArena* owningArena = ty->owningArena; - TypeVar* mtv = asMutable(ty); - *mtv = rep.get()->pending; - mtv->owningArena = owningArena; - } - } + asMutable(ty)->reassign(rep.get()->pending); for (auto& [tp, rep] : typePackChanges) - { - if (FFlag::LuauNonCopyableTypeVarFields) - { - asMutable(tp)->reassign(rep.get()->pending); - } - else - { - TypeArena* owningArena = tp->owningArena; - TypePackVar* mpv = asMutable(tp); - *mpv = rep.get()->pending; - mpv->owningArena = owningArena; - } - } + asMutable(tp)->reassign(rep.get()->pending); clear(); } @@ -196,9 +172,7 @@ PendingType* TxnLog::queue(TypeId ty) if (!pending) { pending = std::make_unique(*ty); - - if (FFlag::LuauNonCopyableTypeVarFields) - pending->pending.owningArena = nullptr; + pending->pending.owningArena = nullptr; } return pending.get(); @@ -214,9 +188,7 @@ PendingTypePack* TxnLog::queue(TypePackId tp) if (!pending) { pending = std::make_unique(*tp); - - if (FFlag::LuauNonCopyableTypeVarFields) - pending->pending.owningArena = nullptr; + pending->pending.owningArena = nullptr; } return pending.get(); @@ -255,24 +227,14 @@ PendingTypePack* TxnLog::pending(TypePackId tp) const PendingType* TxnLog::replace(TypeId ty, TypeVar replacement) { PendingType* newTy = queue(ty); - - if (FFlag::LuauNonCopyableTypeVarFields) - newTy->pending.reassign(replacement); - else - newTy->pending = replacement; - + newTy->pending.reassign(replacement); return newTy; } PendingTypePack* TxnLog::replace(TypePackId tp, TypePackVar replacement) { PendingTypePack* newTp = queue(tp); - - if (FFlag::LuauNonCopyableTypeVarFields) - newTp->pending.reassign(replacement); - else - newTp->pending = replacement; - + newTp->pending.reassign(replacement); return newTp; } @@ -289,7 +251,7 @@ PendingType* TxnLog::bindTable(TypeId ty, std::optional newBoundTo) PendingType* TxnLog::changeLevel(TypeId ty, TypeLevel newLevel) { - LUAU_ASSERT(get(ty) || get(ty) || get(ty)); + LUAU_ASSERT(get(ty) || get(ty) || get(ty) || get(ty)); PendingType* newTy = queue(ty); if (FreeTypeVar* ftv = Luau::getMutable(newTy)) @@ -305,6 +267,11 @@ PendingType* TxnLog::changeLevel(TypeId ty, TypeLevel newLevel) { ftv->level = newLevel; } + else if (ConstrainedTypeVar* ctv = Luau::getMutable(newTy)) + { + if (FFlag::LuauUnknownAndNeverType) + ctv->level = newLevel; + } return newTy; } diff --git a/Analysis/src/TypeAttach.cpp b/Analysis/src/TypeAttach.cpp index 6cca7127..2bc89cf6 100644 --- a/Analysis/src/TypeAttach.cpp +++ b/Analysis/src/TypeAttach.cpp @@ -335,6 +335,14 @@ public: { return allocator->alloc(Location(), std::nullopt, AstName("")); } + AstType* operator()(const UnknownTypeVar& ttv) + { + return allocator->alloc(Location(), std::nullopt, AstName{"unknown"}); + } + AstType* operator()(const NeverTypeVar& ttv) + { + return allocator->alloc(Location(), std::nullopt, AstName{"never"}); + } private: Allocator* allocator; diff --git a/Analysis/src/TypeInfer.cpp b/Analysis/src/TypeInfer.cpp index d9486a4f..01939fda 100644 --- a/Analysis/src/TypeInfer.cpp +++ b/Analysis/src/TypeInfer.cpp @@ -31,6 +31,7 @@ LUAU_FASTINTVARIABLE(LuauCheckRecursionLimit, 300) LUAU_FASTINTVARIABLE(LuauVisitRecursionLimit, 500) LUAU_FASTFLAG(LuauKnowsTheDataModel3) LUAU_FASTFLAG(LuauAutocompleteDynamicLimits) +LUAU_FASTFLAGVARIABLE(LuauIndexSilenceErrors, false) LUAU_FASTFLAGVARIABLE(LuauLowerBoundsCalculation, false) LUAU_FASTFLAGVARIABLE(DebugLuauFreezeDuringUnification, false) LUAU_FASTFLAGVARIABLE(LuauSelfCallAutocompleteFix2, false) @@ -41,10 +42,12 @@ LUAU_FASTFLAGVARIABLE(LuauReturnTypeInferenceInNonstrict, false) LUAU_FASTFLAGVARIABLE(DebugLuauSharedSelf, false); LUAU_FASTFLAGVARIABLE(LuauAlwaysQuantify, false); LUAU_FASTFLAGVARIABLE(LuauReportErrorsOnIndexerKeyMismatch, false) +LUAU_FASTFLAGVARIABLE(LuauUnknownAndNeverType, false) LUAU_FASTFLAG(LuauQuantifyConstrained) LUAU_FASTFLAGVARIABLE(LuauFalsyPredicateReturnsNilInstead, false) -LUAU_FASTFLAGVARIABLE(LuauNonCopyableTypeVarFields, false) LUAU_FASTFLAGVARIABLE(LuauCheckLenMT, false) +LUAU_FASTFLAGVARIABLE(LuauCheckGenericHOFTypes, false) +LUAU_FASTFLAGVARIABLE(LuauBinaryNeedsExpectedTypesToo, false) namespace Luau { @@ -258,7 +261,11 @@ TypeChecker::TypeChecker(ModuleResolver* resolver, InternalErrorReporter* iceHan , booleanType(getSingletonTypes().booleanType) , threadType(getSingletonTypes().threadType) , anyType(getSingletonTypes().anyType) + , unknownType(getSingletonTypes().unknownType) + , neverType(getSingletonTypes().neverType) , anyTypePack(getSingletonTypes().anyTypePack) + , neverTypePack(getSingletonTypes().neverTypePack) + , uninhabitableTypePack(getSingletonTypes().uninhabitableTypePack) , duplicateTypeAliases{{false, {}}} { globalScope = std::make_shared(globalTypes.addTypePack(TypePackVar{FreeTypePack{TypeLevel{}}})); @@ -269,6 +276,11 @@ TypeChecker::TypeChecker(ModuleResolver* resolver, InternalErrorReporter* iceHan globalScope->exportedTypeBindings["string"] = TypeFun{{}, stringType}; globalScope->exportedTypeBindings["boolean"] = TypeFun{{}, booleanType}; globalScope->exportedTypeBindings["thread"] = TypeFun{{}, threadType}; + if (FFlag::LuauUnknownAndNeverType) + { + globalScope->exportedTypeBindings["unknown"] = TypeFun{{}, unknownType}; + globalScope->exportedTypeBindings["never"] = TypeFun{{}, neverType}; + } } ModulePtr TypeChecker::check(const SourceModule& module, Mode mode, std::optional environmentScope) @@ -456,6 +468,59 @@ void TypeChecker::checkBlock(const ScopePtr& scope, const AstStatBlock& block) } } +struct InplaceDemoter : TypeVarOnceVisitor +{ + TypeLevel newLevel; + TypeArena* arena; + + InplaceDemoter(TypeLevel level, TypeArena* arena) + : newLevel(level) + , arena(arena) + { + } + + bool demote(TypeId ty) + { + if (auto level = getMutableLevel(ty)) + { + if (level->subsumesStrict(newLevel)) + { + *level = newLevel; + return true; + } + } + + return false; + } + + bool visit(TypeId ty, const BoundTypeVar& btyRef) override + { + return true; + } + + bool visit(TypeId ty) override + { + if (ty->owningArena != arena) + return false; + return demote(ty); + } + + bool visit(TypePackId tp, const FreeTypePack& ftpRef) override + { + if (tp->owningArena != arena) + return false; + + FreeTypePack* ftp = &const_cast(ftpRef); + if (ftp->level.subsumesStrict(newLevel)) + { + ftp->level = newLevel; + return true; + } + + return false; + } +}; + void TypeChecker::checkBlockWithoutRecursionCheck(const ScopePtr& scope, const AstStatBlock& block) { int subLevel = 0; @@ -559,7 +624,7 @@ void TypeChecker::checkBlockWithoutRecursionCheck(const ScopePtr& scope, const A tablify(baseTy); if (!fun->func->self) - expectedType = getIndexTypeFromType(scope, baseTy, name->index.value, name->indexLocation, false); + expectedType = getIndexTypeFromType(scope, baseTy, name->index.value, name->indexLocation, /* addErrors= */ false); else if (auto ttv = getMutableTableType(baseTy)) { if (!baseTy->persistent && ttv->state != TableState::Sealed && !ttv->selfTy) @@ -579,7 +644,7 @@ void TypeChecker::checkBlockWithoutRecursionCheck(const ScopePtr& scope, const A if (auto name = fun->name->as()) { TypeId exprTy = checkExpr(scope, *name->expr).type; - expectedType = getIndexTypeFromType(scope, exprTy, name->index.value, name->indexLocation, false); + expectedType = getIndexTypeFromType(scope, exprTy, name->index.value, name->indexLocation, /* addErrors= */ false); } } } @@ -634,15 +699,8 @@ LUAU_NOINLINE void TypeChecker::checkBlockTypeAliases(const ScopePtr& scope, std TypeId type = bindings[name].type; if (get(follow(type))) { - if (FFlag::LuauNonCopyableTypeVarFields) - { - TypeVar* mty = asMutable(follow(type)); - mty->reassign(*errorRecoveryType(anyType)); - } - else - { - *asMutable(type) = *errorRecoveryType(anyType); - } + TypeVar* mty = asMutable(follow(type)); + mty->reassign(*errorRecoveryType(anyType)); reportError(TypeError{typealias->location, OccursCheckFailed{}}); } @@ -1206,7 +1264,7 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatForIn& forin) iterTy = instantiate(scope, checkExpr(scope, *firstValue).type, firstValue->location); } - if (std::optional iterMM = findMetatableEntry(iterTy, "__iter", firstValue->location)) + if (std::optional iterMM = findMetatableEntry(iterTy, "__iter", firstValue->location, /* addErrors= */ true)) { // if __iter metamethod is present, it will be called and the results are going to be called as if they are functions // TODO: this needs to typecheck all returned values by __iter as if they were for loop arguments @@ -1253,7 +1311,7 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatForIn& forin) for (TypeId var : varTypes) unify(varTy, var, forin.location); - if (!get(iterTy) && !get(iterTy) && !get(iterTy)) + if (!get(iterTy) && !get(iterTy) && !get(iterTy) && !get(iterTy)) reportError(firstValue->location, CannotCallNonFunction{iterTy}); return check(loopScope, *forin.body); @@ -1350,7 +1408,7 @@ void TypeChecker::check(const ScopePtr& scope, TypeId ty, const ScopePtr& funSco TypeId exprTy = checkExpr(scope, *name->expr).type; TableTypeVar* ttv = getMutableTableType(exprTy); - if (!getIndexTypeFromType(scope, exprTy, name->index.value, name->indexLocation, false)) + if (!getIndexTypeFromType(scope, exprTy, name->index.value, name->indexLocation, /* addErrors= */ false)) { if (ttv || isTableIntersection(exprTy)) reportError(TypeError{function.location, CannotExtendTable{exprTy, CannotExtendTable::Property, name->index.value}}); @@ -1376,6 +1434,12 @@ void TypeChecker::check(const ScopePtr& scope, TypeId ty, const ScopePtr& funSco checkFunctionBody(funScope, ty, *function.func); + if (FFlag::LuauUnknownAndNeverType) + { + InplaceDemoter demoter{funScope->level, ¤tModule->internalTypes}; + demoter.traverse(ty); + } + if (ttv && ttv->state != TableState::Sealed) ttv->props[name->index.value] = {follow(quantify(funScope, ty, name->indexLocation)), /* deprecated */ false, {}, name->indexLocation}; } @@ -1729,7 +1793,7 @@ WithPredicate TypeChecker::checkExpr(const ScopePtr& scope, const AstExp else if (auto a = expr.as()) result = checkExpr(scope, *a); else if (auto a = expr.as()) - result = checkExpr(scope, *a); + result = checkExpr(scope, *a, FFlag::LuauBinaryNeedsExpectedTypesToo ? expectedType : std::nullopt); else if (auto a = expr.as()) result = checkExpr(scope, *a); else if (auto a = expr.as()) @@ -1851,41 +1915,56 @@ WithPredicate TypeChecker::checkExpr(const ScopePtr& scope, const AstExp lhsType = stripFromNilAndReport(lhsType, expr.expr->location); - if (std::optional ty = getIndexTypeFromType(scope, lhsType, name, expr.location, true)) + if (std::optional ty = getIndexTypeFromType(scope, lhsType, name, expr.location, /* addErrors= */ true)) return {*ty}; return {errorRecoveryType(scope)}; } -std::optional TypeChecker::findTablePropertyRespectingMeta(TypeId lhsType, Name name, const Location& location) +std::optional TypeChecker::findTablePropertyRespectingMeta(TypeId lhsType, Name name, const Location& location, bool addErrors) { ErrorVec errors; auto result = Luau::findTablePropertyRespectingMeta(errors, lhsType, name, location); - reportErrors(errors); + if (!FFlag::LuauIndexSilenceErrors || addErrors) + reportErrors(errors); return result; } -std::optional TypeChecker::findMetatableEntry(TypeId type, std::string entry, const Location& location) +std::optional TypeChecker::findMetatableEntry(TypeId type, std::string entry, const Location& location, bool addErrors) { ErrorVec errors; auto result = Luau::findMetatableEntry(errors, type, entry, location); - reportErrors(errors); + if (!FFlag::LuauIndexSilenceErrors || addErrors) + reportErrors(errors); return result; } std::optional TypeChecker::getIndexTypeFromType( - const ScopePtr& scope, TypeId type, const std::string& name, const Location& location, bool addErrors) + const ScopePtr& scope, TypeId type, const Name& name, const Location& location, bool addErrors) +{ + size_t errorCount = currentModule->errors.size(); + + std::optional result = getIndexTypeFromTypeImpl(scope, type, name, location, addErrors); + + if (FFlag::LuauIndexSilenceErrors && !addErrors) + LUAU_ASSERT(errorCount == currentModule->errors.size()); + + return result; +} + +std::optional TypeChecker::getIndexTypeFromTypeImpl( + const ScopePtr& scope, TypeId type, const Name& name, const Location& location, bool addErrors) { type = follow(type); - if (get(type) || get(type)) + if (get(type) || get(type) || get(type)) return type; tablify(type); if (isString(type)) { - std::optional mtIndex = findMetatableEntry(stringType, "__index", location); + std::optional mtIndex = findMetatableEntry(stringType, "__index", location, addErrors); LUAU_ASSERT(mtIndex); type = *mtIndex; } @@ -1919,7 +1998,7 @@ std::optional TypeChecker::getIndexTypeFromType( return result; } - if (auto found = findTablePropertyRespectingMeta(type, name, location)) + if (auto found = findTablePropertyRespectingMeta(type, name, location, addErrors)) return *found; } else if (const ClassTypeVar* cls = get(type)) @@ -1941,7 +2020,7 @@ std::optional TypeChecker::getIndexTypeFromType( if (get(follow(t))) return t; - if (std::optional ty = getIndexTypeFromType(scope, t, name, location, false)) + if (std::optional ty = getIndexTypeFromType(scope, t, name, location, /* addErrors= */ false)) goodOptions.push_back(*ty); else badOptions.push_back(t); @@ -1972,6 +2051,8 @@ std::optional TypeChecker::getIndexTypeFromType( else { std::vector result = reduceUnion(goodOptions); + if (FFlag::LuauUnknownAndNeverType && result.empty()) + return neverType; if (result.size() == 1) return result[0]; @@ -1987,7 +2068,7 @@ std::optional TypeChecker::getIndexTypeFromType( { RecursionLimiter _rl(&recursionCount, FInt::LuauTypeInferRecursionLimit); - if (std::optional ty = getIndexTypeFromType(scope, t, name, location, false)) + if (std::optional ty = getIndexTypeFromType(scope, t, name, location, /* addErrors= */ false)) parts.push_back(*ty); } @@ -2017,6 +2098,9 @@ std::vector TypeChecker::reduceUnion(const std::vector& types) for (TypeId t : types) { t = follow(t); + if (get(t)) + continue; + if (get(t) || get(t)) return {t}; @@ -2028,6 +2112,8 @@ std::vector TypeChecker::reduceUnion(const std::vector& types) { if (FFlag::LuauNormalizeFlagIsConservative) ty = follow(ty); + if (get(ty)) + continue; if (get(ty) || get(ty)) return {ty}; @@ -2041,6 +2127,8 @@ std::vector TypeChecker::reduceUnion(const std::vector& types) for (TypeId ty : r) { ty = follow(ty); + if (get(ty)) + continue; if (get(ty) || get(ty)) return {ty}; @@ -2314,14 +2402,14 @@ WithPredicate TypeChecker::checkExpr(const ScopePtr& scope, const AstExp return {booleanType, {NotPredicate{std::move(result.predicates)}}}; case AstExprUnary::Minus: { - const bool operandIsAny = get(operandType) || get(operandType); + const bool operandIsAny = get(operandType) || get(operandType) || get(operandType); if (operandIsAny) return {operandType}; if (typeCouldHaveMetatable(operandType)) { - if (auto fnt = findMetatableEntry(operandType, "__unm", expr.location)) + if (auto fnt = findMetatableEntry(operandType, "__unm", expr.location, /* addErrors= */ true)) { TypeId actualFunctionType = instantiate(scope, *fnt, expr.location); TypePackId arguments = addTypePack({operandType}); @@ -2355,14 +2443,14 @@ WithPredicate TypeChecker::checkExpr(const ScopePtr& scope, const AstExp operandType = stripFromNilAndReport(operandType, expr.location); - if (get(operandType)) - return {errorRecoveryType(scope)}; + if (get(operandType) || get(operandType)) + return {!FFlag::LuauUnknownAndNeverType ? errorRecoveryType(scope) : operandType}; DenseHashSet seen{nullptr}; if (FFlag::LuauCheckLenMT && typeCouldHaveMetatable(operandType)) { - if (auto fnt = findMetatableEntry(operandType, "__len", expr.location)) + if (auto fnt = findMetatableEntry(operandType, "__len", expr.location, /* addErrors= */ true)) { TypeId actualFunctionType = instantiate(scope, *fnt, expr.location); TypePackId arguments = addTypePack({operandType}); @@ -2433,6 +2521,9 @@ TypeId TypeChecker::unionOfTypes(TypeId a, TypeId b, const Location& location, b return a; std::vector types = reduceUnion({a, b}); + if (FFlag::LuauUnknownAndNeverType && types.empty()) + return neverType; + if (types.size() == 1) return types[0]; @@ -2485,7 +2576,7 @@ TypeId TypeChecker::checkRelationalOperation( // If we know nothing at all about the lhs type, we can usually say nothing about the result. // The notable exception to this is the equality and inequality operators, which always produce a boolean. - const bool lhsIsAny = get(lhsType) || get(lhsType); + const bool lhsIsAny = get(lhsType) || get(lhsType) || get(lhsType); // Peephole check for `cond and a or b -> type(a)|type(b)` // TODO: Kill this when singleton types arrive. :( @@ -2508,7 +2599,7 @@ TypeId TypeChecker::checkRelationalOperation( if (isNonstrictMode() && (isNil(lhsType) || isNil(rhsType))) return booleanType; - const bool rhsIsAny = get(rhsType) || get(rhsType); + const bool rhsIsAny = get(rhsType) || get(rhsType) || get(rhsType); if (lhsIsAny || rhsIsAny) return booleanType; @@ -2596,7 +2687,7 @@ TypeId TypeChecker::checkRelationalOperation( if (leftMetatable) { - std::optional metamethod = findMetatableEntry(lhsType, metamethodName, expr.location); + std::optional metamethod = findMetatableEntry(lhsType, metamethodName, expr.location, /* addErrors= */ true); if (metamethod) { if (const FunctionTypeVar* ftv = get(*metamethod)) @@ -2757,9 +2848,9 @@ TypeId TypeChecker::checkBinaryOperation( }; std::string op = opToMetaTableEntry(expr.op); - if (auto fnt = findMetatableEntry(lhsType, op, expr.location)) + if (auto fnt = findMetatableEntry(lhsType, op, expr.location, /* addErrors= */ true)) return checkMetatableCall(*fnt, lhsType, rhsType); - if (auto fnt = findMetatableEntry(rhsType, op, expr.location)) + if (auto fnt = findMetatableEntry(rhsType, op, expr.location, /* addErrors= */ true)) { // Note the intentionally reversed arguments here. return checkMetatableCall(*fnt, rhsType, lhsType); @@ -2793,27 +2884,27 @@ TypeId TypeChecker::checkBinaryOperation( } } -WithPredicate TypeChecker::checkExpr(const ScopePtr& scope, const AstExprBinary& expr) +WithPredicate TypeChecker::checkExpr(const ScopePtr& scope, const AstExprBinary& expr, std::optional expectedType) { if (expr.op == AstExprBinary::And) { - auto [lhsTy, lhsPredicates] = checkExpr(scope, *expr.left); + auto [lhsTy, lhsPredicates] = checkExpr(scope, *expr.left, expectedType); ScopePtr innerScope = childScope(scope, expr.location); resolve(lhsPredicates, innerScope, true); - auto [rhsTy, rhsPredicates] = checkExpr(innerScope, *expr.right); + auto [rhsTy, rhsPredicates] = checkExpr(innerScope, *expr.right, expectedType); return {checkBinaryOperation(scope, expr, lhsTy, rhsTy), {AndPredicate{std::move(lhsPredicates), std::move(rhsPredicates)}}}; } else if (expr.op == AstExprBinary::Or) { - auto [lhsTy, lhsPredicates] = checkExpr(scope, *expr.left); + auto [lhsTy, lhsPredicates] = checkExpr(scope, *expr.left, expectedType); ScopePtr innerScope = childScope(scope, expr.location); resolve(lhsPredicates, innerScope, false); - auto [rhsTy, rhsPredicates] = checkExpr(innerScope, *expr.right); + auto [rhsTy, rhsPredicates] = checkExpr(innerScope, *expr.right, expectedType); // Because of C++, I'm not sure if lhsPredicates was not moved out by the time we call checkBinaryOperation. TypeId result = checkBinaryOperation(scope, expr, lhsTy, rhsTy, lhsPredicates); @@ -2824,6 +2915,8 @@ WithPredicate TypeChecker::checkExpr(const ScopePtr& scope, const AstExp if (auto predicate = tryGetTypeGuardPredicate(expr)) return {booleanType, {std::move(*predicate)}}; + // For these, passing expectedType is worse than simply forcing them, because their implementation + // may inadvertently check if expectedTypes exist first and use it, instead of forceSingleton first. WithPredicate lhs = checkExpr(scope, *expr.left, std::nullopt, /*forceSingleton=*/true); WithPredicate rhs = checkExpr(scope, *expr.right, std::nullopt, /*forceSingleton=*/true); @@ -2842,6 +2935,7 @@ WithPredicate TypeChecker::checkExpr(const ScopePtr& scope, const AstExp } else { + // Expected types are not useful for other binary operators. WithPredicate lhs = checkExpr(scope, *expr.left); WithPredicate rhs = checkExpr(scope, *expr.right); @@ -2896,6 +2990,8 @@ WithPredicate TypeChecker::checkExpr(const ScopePtr& scope, const AstExp return {trueType.type}; std::vector types = reduceUnion({trueType.type, falseType.type}); + if (FFlag::LuauUnknownAndNeverType && types.empty()) + return {neverType}; return {types.size() == 1 ? types[0] : addType(UnionTypeVar{std::move(types)})}; } @@ -2927,7 +3023,10 @@ TypeId TypeChecker::checkLValueBinding(const ScopePtr& scope, const AstExpr& exp TypeId TypeChecker::checkLValueBinding(const ScopePtr& scope, const AstExprLocal& expr) { if (std::optional ty = scope->lookup(expr.local)) - return *ty; + { + ty = follow(*ty); + return get(*ty) ? unknownType : *ty; + } reportError(expr.location, UnknownSymbol{expr.local->name.value, UnknownSymbol::Binding}); return errorRecoveryType(scope); @@ -2941,7 +3040,10 @@ TypeId TypeChecker::checkLValueBinding(const ScopePtr& scope, const AstExprGloba const auto it = moduleScope->bindings.find(expr.name); if (it != moduleScope->bindings.end()) - return it->second.typeId; + { + TypeId ty = follow(it->second.typeId); + return get(ty) ? unknownType : ty; + } TypeId result = freshType(scope); Binding& binding = moduleScope->bindings[expr.name]; @@ -2962,6 +3064,9 @@ TypeId TypeChecker::checkLValueBinding(const ScopePtr& scope, const AstExprIndex if (get(lhs) || get(lhs)) return lhs; + if (get(lhs)) + return unknownType; + tablify(lhs); Name name = expr.index.value; @@ -3023,7 +3128,7 @@ TypeId TypeChecker::checkLValueBinding(const ScopePtr& scope, const AstExprIndex } else if (get(lhs)) { - if (std::optional ty = getIndexTypeFromType(scope, lhs, name, expr.location, false)) + if (std::optional ty = getIndexTypeFromType(scope, lhs, name, expr.location, /* addErrors= */ false)) return *ty; // If intersection has a table part, report that it cannot be extended just as a sealed table @@ -3050,6 +3155,9 @@ TypeId TypeChecker::checkLValueBinding(const ScopePtr& scope, const AstExprIndex if (get(exprType) || get(exprType)) return exprType; + if (get(exprType)) + return unknownType; + AstExprConstantString* value = expr.index->as(); if (value) @@ -3156,7 +3264,7 @@ TypeId TypeChecker::checkFunctionName(const ScopePtr& scope, AstExpr& funName, T if (!ttv || ttv->state == TableState::Sealed) { - if (auto ty = getIndexTypeFromType(scope, lhsType, indexName->index.value, indexName->indexLocation, false)) + if (auto ty = getIndexTypeFromType(scope, lhsType, indexName->index.value, indexName->indexLocation, /* addErrors= */ false)) return *ty; return errorRecoveryType(scope); @@ -3228,9 +3336,12 @@ std::pair TypeChecker::checkFunctionSignature(const ScopePtr& } } - // We do not infer type binders, so if a generic function is required we do not propagate - if (expectedFunctionType && !(expectedFunctionType->generics.empty() && expectedFunctionType->genericPacks.empty())) - expectedFunctionType = nullptr; + if (!FFlag::LuauCheckGenericHOFTypes) + { + // We do not infer type binders, so if a generic function is required we do not propagate + if (expectedFunctionType && !(expectedFunctionType->generics.empty() && expectedFunctionType->genericPacks.empty())) + expectedFunctionType = nullptr; + } } auto [generics, genericPacks] = createGenericTypes(funScope, std::nullopt, expr, expr.generics, expr.genericPacks); @@ -3240,7 +3351,8 @@ std::pair TypeChecker::checkFunctionSignature(const ScopePtr& retPack = resolveTypePack(funScope, *expr.returnAnnotation); else if (FFlag::LuauReturnTypeInferenceInNonstrict ? (!FFlag::LuauLowerBoundsCalculation && isNonstrictMode()) : isNonstrictMode()) retPack = anyTypePack; - else if (expectedFunctionType) + else if (expectedFunctionType && + (!FFlag::LuauCheckGenericHOFTypes || (expectedFunctionType->generics.empty() && expectedFunctionType->genericPacks.empty()))) { auto [head, tail] = flatten(expectedFunctionType->retTypes); @@ -3371,16 +3483,50 @@ std::pair TypeChecker::checkFunctionSignature(const ScopePtr& defn.originalNameLocation = originalName.value_or(Location(expr.location.begin, 0)); std::vector genericTys; - genericTys.reserve(generics.size()); - std::transform(generics.begin(), generics.end(), std::back_inserter(genericTys), [](auto&& el) { - return el.ty; - }); + // if we have a generic expected function type and no generics, we should use the expected ones. + if (FFlag::LuauCheckGenericHOFTypes) + { + if (expectedFunctionType && generics.empty()) + { + genericTys = expectedFunctionType->generics; + } + else + { + genericTys.reserve(generics.size()); + for (const GenericTypeDefinition& generic : generics) + genericTys.push_back(generic.ty); + } + } + else + { + genericTys.reserve(generics.size()); + std::transform(generics.begin(), generics.end(), std::back_inserter(genericTys), [](auto&& el) { + return el.ty; + }); + } std::vector genericTps; - genericTps.reserve(genericPacks.size()); - std::transform(genericPacks.begin(), genericPacks.end(), std::back_inserter(genericTps), [](auto&& el) { - return el.tp; - }); + // if we have a generic expected function type and no generic typepacks, we should use the expected ones. + if (FFlag::LuauCheckGenericHOFTypes) + { + if (expectedFunctionType && genericPacks.empty()) + { + genericTps = expectedFunctionType->genericPacks; + } + else + { + genericTps.reserve(genericPacks.size()); + for (const GenericTypePackDefinition& generic : genericPacks) + genericTps.push_back(generic.tp); + } + } + else + { + genericTps.reserve(genericPacks.size()); + std::transform(genericPacks.begin(), genericPacks.end(), std::back_inserter(genericTps), [](auto&& el) { + return el.tp; + }); + } TypeId funTy = addType(FunctionTypeVar(funScope->level, std::move(genericTys), std::move(genericTps), argPack, retPack, std::move(defn), bool(expr.self))); @@ -3474,9 +3620,22 @@ void TypeChecker::checkFunctionBody(const ScopePtr& scope, TypeId ty, const AstE } WithPredicate TypeChecker::checkExprPack(const ScopePtr& scope, const AstExpr& expr) +{ + if (FFlag::LuauUnknownAndNeverType) + { + WithPredicate result = checkExprPackHelper(scope, expr); + if (containsNever(result.type)) + return {uninhabitableTypePack}; + return result; + } + else + return checkExprPackHelper(scope, expr); +} + +WithPredicate TypeChecker::checkExprPackHelper(const ScopePtr& scope, const AstExpr& expr) { if (auto a = expr.as()) - return checkExprPack(scope, *a); + return checkExprPackHelper(scope, *a); else if (expr.is()) { if (!scope->varargPack) @@ -3739,7 +3898,7 @@ void TypeChecker::checkArgumentList( } } -WithPredicate TypeChecker::checkExprPack(const ScopePtr& scope, const AstExprCall& expr) +WithPredicate TypeChecker::checkExprPackHelper(const ScopePtr& scope, const AstExprCall& expr) { // evaluate type of function // decompose an intersection into its component overloads @@ -3763,7 +3922,7 @@ WithPredicate TypeChecker::checkExprPack(const ScopePtr& scope, cons selfType = checkExpr(scope, *indexExpr->expr).type; selfType = stripFromNilAndReport(selfType, expr.func->location); - if (std::optional propTy = getIndexTypeFromType(scope, selfType, indexExpr->index.value, expr.location, true)) + if (std::optional propTy = getIndexTypeFromType(scope, selfType, indexExpr->index.value, expr.location, /* addErrors= */ true)) { functionType = *propTy; actualFunctionType = instantiate(scope, functionType, expr.func->location); @@ -3813,11 +3972,25 @@ WithPredicate TypeChecker::checkExprPack(const ScopePtr& scope, cons if (get(argPack)) return {errorRecoveryTypePack(scope)}; - TypePack* args = getMutable(argPack); - LUAU_ASSERT(args != nullptr); + TypePack* args = nullptr; + if (FFlag::LuauUnknownAndNeverType) + { + if (expr.self) + { + argPack = addTypePack(TypePack{{selfType}, argPack}); + argListResult.type = argPack; + } + args = getMutable(argPack); + LUAU_ASSERT(args); + } + else + { + args = getMutable(argPack); + LUAU_ASSERT(args != nullptr); - if (expr.self) - args->head.insert(args->head.begin(), selfType); + if (expr.self) + args->head.insert(args->head.begin(), selfType); + } std::vector argLocations; argLocations.reserve(expr.args.size + 1); @@ -3876,7 +4049,10 @@ std::vector> TypeChecker::getExpectedTypesForCall(const st else { std::vector result = reduceUnion({*el, ty}); - el = result.size() == 1 ? result[0] : addType(UnionTypeVar{std::move(result)}); + if (FFlag::LuauUnknownAndNeverType && result.empty()) + el = neverType; + else + el = result.size() == 1 ? result[0] : addType(UnionTypeVar{std::move(result)}); } } }; @@ -3930,6 +4106,9 @@ std::optional> TypeChecker::checkCallOverload(const Sc return {{errorRecoveryTypePack(scope)}}; } + if (get(fn)) + return {{uninhabitableTypePack}}; + if (auto ftv = get(fn)) { // fn is one of the overloads of actualFunctionType, which @@ -3975,7 +4154,7 @@ std::optional> TypeChecker::checkCallOverload(const Sc // Might be a callable table if (const MetatableTypeVar* mttv = get(fn)) { - if (std::optional ty = getIndexTypeFromType(scope, mttv->metatable, "__call", expr.func->location, false)) + if (std::optional ty = getIndexTypeFromType(scope, mttv->metatable, "__call", expr.func->location, /* addErrors= */ false)) { // Construct arguments with 'self' added in front TypePackId metaCallArgPack = addTypePack(TypePackVar(TypePack{args->head, args->tail})); @@ -4202,6 +4381,7 @@ void TypeChecker::reportOverloadResolutionError(const ScopePtr& scope, const Ast WithPredicate TypeChecker::checkExprList(const ScopePtr& scope, const Location& location, const AstArray& exprs, bool substituteFreeForNil, const std::vector& instantiateGenerics, const std::vector>& expectedTypes) { + bool uninhabitable = false; TypePackId pack = addTypePack(TypePack{}); PredicateVec predicates; // At the moment we will be pushing all predicate sets into this. Do we need some way to split them up? @@ -4232,7 +4412,13 @@ WithPredicate TypeChecker::checkExprList(const ScopePtr& scope, cons auto [typePack, exprPredicates] = checkExprPack(scope, *expr); insert(exprPredicates); - if (std::optional firstTy = first(typePack)) + if (FFlag::LuauUnknownAndNeverType && containsNever(typePack)) + { + // f(), g() where f() returns (never, string) or (string, never) means this whole TypePackId is uninhabitable, so return (never, ...never) + uninhabitable = true; + continue; + } + else if (std::optional firstTy = first(typePack)) { if (!currentModule->astTypes.find(expr)) currentModule->astTypes[expr] = follow(*firstTy); @@ -4248,6 +4434,13 @@ WithPredicate TypeChecker::checkExprList(const ScopePtr& scope, cons auto [type, exprPredicates] = checkExpr(scope, *expr, expectedType); insert(exprPredicates); + if (FFlag::LuauUnknownAndNeverType && get(type)) + { + // f(), g() where f() returns (never, string) or (string, never) means this whole TypePackId is uninhabitable, so return (never, ...never) + uninhabitable = true; + continue; + } + TypeId actualType = substituteFreeForNil && expr->is() ? freshType(scope) : type; if (instantiateGenerics.size() > i && instantiateGenerics[i]) @@ -4272,6 +4465,8 @@ WithPredicate TypeChecker::checkExprList(const ScopePtr& scope, cons for (TxnLog& log : inverseLogs) log.commit(); + if (FFlag::LuauUnknownAndNeverType && uninhabitable) + return {uninhabitableTypePack}; return {pack, predicates}; } @@ -4830,7 +5025,7 @@ TypeIdPredicate TypeChecker::mkTruthyPredicate(bool sense) }; } -std::optional TypeChecker::filterMap(TypeId type, TypeIdPredicate predicate) +std::optional TypeChecker::filterMapImpl(TypeId type, TypeIdPredicate predicate) { std::vector types = Luau::filterMap(type, predicate); if (!types.empty()) @@ -4838,7 +5033,21 @@ std::optional TypeChecker::filterMap(TypeId type, TypeIdPredicate predic return std::nullopt; } -std::optional TypeChecker::pickTypesFromSense(TypeId type, bool sense) +std::pair, bool> TypeChecker::filterMap(TypeId type, TypeIdPredicate predicate) +{ + if (FFlag::LuauUnknownAndNeverType) + { + TypeId ty = filterMapImpl(type, predicate).value_or(neverType); + return {ty, !bool(get(ty))}; + } + else + { + std::optional ty = filterMapImpl(type, predicate); + return {ty, bool(ty)}; + } +} + +std::pair, bool> TypeChecker::pickTypesFromSense(TypeId type, bool sense) { return filterMap(type, mkTruthyPredicate(sense)); } @@ -4884,6 +5093,13 @@ TypePackId TypeChecker::freshTypePack(TypeLevel level) } TypeId TypeChecker::resolveType(const ScopePtr& scope, const AstType& annotation) +{ + TypeId ty = resolveTypeWorker(scope, annotation); + currentModule->astResolvedTypes[&annotation] = ty; + return ty; +} + +TypeId TypeChecker::resolveTypeWorker(const ScopePtr& scope, const AstType& annotation) { if (const auto& lit = annotation.as()) { @@ -5200,9 +5416,10 @@ TypePackId TypeChecker::resolveTypePack(const ScopePtr& scope, const AstTypeList TypePackId TypeChecker::resolveTypePack(const ScopePtr& scope, const AstTypePack& annotation) { + TypePackId result; if (const AstTypePackVariadic* variadic = annotation.as()) { - return addTypePack(TypePackVar{VariadicTypePack{resolveType(scope, *variadic->variadicType)}}); + result = addTypePack(TypePackVar{VariadicTypePack{resolveType(scope, *variadic->variadicType)}}); } else if (const AstTypePackGeneric* generic = annotation.as()) { @@ -5216,10 +5433,12 @@ TypePackId TypeChecker::resolveTypePack(const ScopePtr& scope, const AstTypePack else reportError(TypeError{generic->location, UnknownSymbol{genericName, UnknownSymbol::Type}}); - return errorRecoveryTypePack(scope); + result = errorRecoveryTypePack(scope); + } + else + { + result = *genericTy; } - - return *genericTy; } else if (const AstTypePackExplicit* explicitTp = annotation.as()) { @@ -5229,14 +5448,17 @@ TypePackId TypeChecker::resolveTypePack(const ScopePtr& scope, const AstTypePack types.push_back(resolveType(scope, *type)); if (auto tailType = explicitTp->typeList.tailType) - return addTypePack(types, resolveTypePack(scope, *tailType)); - - return addTypePack(types); + result = addTypePack(types, resolveTypePack(scope, *tailType)); + else + result = addTypePack(types); } else { ice("Unknown AstTypePack kind"); } + + currentModule->astResolvedTypePacks[&annotation] = result; + return result; } bool ApplyTypeFunction::isDirty(TypeId ty) @@ -5452,10 +5674,18 @@ void TypeChecker::refineLValue(const LValue& lvalue, RefinementMap& refis, const // If we do not have a key, it means we're not trying to discriminate anything, so it's a simple matter of just filtering for a subset. if (!key) { - if (std::optional result = filterMap(*ty, predicate)) + auto [result, ok] = filterMap(*ty, predicate); + if (FFlag::LuauUnknownAndNeverType) + { addRefinement(refis, *target, *result); + } else - addRefinement(refis, *target, errorRecoveryType(scope)); + { + if (ok) + addRefinement(refis, *target, *result); + else + addRefinement(refis, *target, errorRecoveryType(scope)); + } return; } @@ -5471,17 +5701,29 @@ void TypeChecker::refineLValue(const LValue& lvalue, RefinementMap& refis, const { std::optional discriminantTy; if (auto field = Luau::get(*key)) // need to fully qualify Luau::get because of ADL. - discriminantTy = getIndexTypeFromType(scope, option, field->key, Location(), false); + discriminantTy = getIndexTypeFromType(scope, option, field->key, Location(), /* addErrors= */ false); else LUAU_ASSERT(!"Unhandled LValue alternative?"); if (!discriminantTy) return; // Do nothing. An error was already reported, as per usual. - if (std::optional result = filterMap(*discriminantTy, predicate)) + auto [result, ok] = filterMap(*discriminantTy, predicate); + if (FFlag::LuauUnknownAndNeverType) { - viableTargetOptions.insert(option); - viableChildOptions.insert(*result); + if (!get(*result)) + { + viableTargetOptions.insert(option); + viableChildOptions.insert(*result); + } + } + else + { + if (ok) + { + viableTargetOptions.insert(option); + viableChildOptions.insert(*result); + } } } @@ -5560,7 +5802,7 @@ std::optional TypeChecker::resolveLValue(const ScopePtr& scope, const LV continue; else if (auto field = get(key)) { - found = getIndexTypeFromType(scope, *found, field->key, Location(), false); + found = getIndexTypeFromType(scope, *found, field->key, Location(), /* addErrors= */ false); if (!found) return std::nullopt; // Turns out this type doesn't have the property at all. We're done. } @@ -5740,6 +5982,9 @@ void TypeChecker::resolve(const TypeGuardPredicate& typeguardP, RefinementMap& r auto mkFilter = [](ConditionFunc f, std::optional other = std::nullopt) -> SenseToTypeIdPredicate { return [f, other](bool sense) -> TypeIdPredicate { return [f, other, sense](TypeId ty) -> std::optional { + if (FFlag::LuauUnknownAndNeverType && sense && get(ty)) + return other.value_or(ty); + if (f(ty) == sense) return ty; @@ -5847,8 +6092,15 @@ std::vector TypeChecker::unTypePack(const ScopePtr& scope, TypePackId tp for (size_t i = 0; i < expectedLength; ++i) expectedPack->head.push_back(freshType(scope)); + size_t oldErrorsSize = currentModule->errors.size(); + unify(tp, expectedTypePack, location); + // HACK: tryUnify would undo the changes to the expectedTypePack if the length mismatches, but + // we want to tie up free types to be error types, so we do this instead. + if (FFlag::LuauUnknownAndNeverType) + currentModule->errors.resize(oldErrorsSize); + for (TypeId& tp : expectedPack->head) tp = follow(tp); diff --git a/Analysis/src/TypePack.cpp b/Analysis/src/TypePack.cpp index 82451bd1..d4544483 100644 --- a/Analysis/src/TypePack.cpp +++ b/Analysis/src/TypePack.cpp @@ -5,8 +5,6 @@ #include -LUAU_FASTFLAG(LuauNonCopyableTypeVarFields) - namespace Luau { @@ -40,19 +38,10 @@ TypePackVar& TypePackVar::operator=(TypePackVariant&& tp) TypePackVar& TypePackVar::operator=(const TypePackVar& rhs) { - if (FFlag::LuauNonCopyableTypeVarFields) - { - LUAU_ASSERT(owningArena == rhs.owningArena); - LUAU_ASSERT(!rhs.persistent); + LUAU_ASSERT(owningArena == rhs.owningArena); + LUAU_ASSERT(!rhs.persistent); - reassign(rhs); - } - else - { - ty = rhs.ty; - persistent = rhs.persistent; - owningArena = rhs.owningArena; - } + reassign(rhs); return *this; } @@ -294,6 +283,16 @@ std::optional first(TypePackId tp, bool ignoreHiddenVariadics) return std::nullopt; } +TypePackVar* asMutable(TypePackId tp) +{ + return const_cast(tp); +} + +TypePack* asMutable(const TypePack* tp) +{ + return const_cast(tp); +} + bool isEmpty(TypePackId tp) { tp = follow(tp); @@ -360,13 +359,25 @@ bool isVariadic(TypePackId tp, const TxnLog& log) return false; } -TypePackVar* asMutable(TypePackId tp) +bool containsNever(TypePackId tp) { - return const_cast(tp); + auto it = begin(tp); + auto endIt = end(tp); + + while (it != endIt) + { + if (get(follow(*it))) + return true; + ++it; + } + + if (auto tail = it.tail()) + { + if (auto vtp = get(*tail); vtp && get(follow(vtp->ty))) + return true; + } + + return false; } -TypePack* asMutable(const TypePack* tp) -{ - return const_cast(tp); -} } // namespace Luau diff --git a/Analysis/src/TypeUtils.cpp b/Analysis/src/TypeUtils.cpp index 3d97e6eb..66b38cf3 100644 --- a/Analysis/src/TypeUtils.cpp +++ b/Analysis/src/TypeUtils.cpp @@ -24,7 +24,7 @@ std::optional findMetatableEntry(ErrorVec& errors, TypeId type, std::str const TableTypeVar* mtt = getTableType(unwrapped); if (!mtt) { - errors.push_back(TypeError{location, GenericError{"Metatable was not a table."}}); + errors.push_back(TypeError{location, GenericError{"Metatable was not a table"}}); return std::nullopt; } diff --git a/Analysis/src/TypeVar.cpp b/Analysis/src/TypeVar.cpp index ade70d72..f884ad77 100644 --- a/Analysis/src/TypeVar.cpp +++ b/Analysis/src/TypeVar.cpp @@ -23,7 +23,9 @@ LUAU_FASTFLAG(DebugLuauFreezeArena) LUAU_FASTINTVARIABLE(LuauTypeMaximumStringifierLength, 500) LUAU_FASTINTVARIABLE(LuauTableTypeMaximumStringifierLength, 0) LUAU_FASTINT(LuauTypeInferRecursionLimit) -LUAU_FASTFLAG(LuauNonCopyableTypeVarFields) +LUAU_FASTFLAG(LuauUnknownAndNeverType) +LUAU_FASTFLAGVARIABLE(LuauDeduceGmatchReturnTypes, false) +LUAU_FASTFLAGVARIABLE(LuauMaybeGenericIntersectionTypes, false) namespace Luau { @@ -31,6 +33,9 @@ namespace Luau std::optional> magicFunctionFormat( TypeChecker& typechecker, const ScopePtr& scope, const AstExprCall& expr, WithPredicate withPredicate); +static std::optional> magicFunctionGmatch( + TypeChecker& typechecker, const ScopePtr& scope, const AstExprCall& expr, WithPredicate withPredicate); + TypeId follow(TypeId t) { return follow(t, [](TypeId t) { @@ -173,8 +178,8 @@ bool maybeString(TypeId ty) { ty = follow(ty); - if (isPrim(ty, PrimitiveTypeVar::String) || get(ty)) - return true; + if (isPrim(ty, PrimitiveTypeVar::String) || get(ty)) + return true; if (auto utv = get(ty)) return std::any_of(begin(utv), end(utv), maybeString); @@ -194,7 +199,7 @@ bool isOptional(TypeId ty) ty = follow(ty); - if (get(ty)) + if (get(ty) || (FFlag::LuauUnknownAndNeverType && get(ty))) return true; auto utv = get(ty); @@ -334,6 +339,28 @@ bool isGeneric(TypeId ty) bool maybeGeneric(TypeId ty) { + if (FFlag::LuauMaybeGenericIntersectionTypes) + { + ty = follow(ty); + + if (get(ty)) + return true; + + if (auto ttv = get(ty)) + { + // TODO: recurse on table types CLI-39914 + (void)ttv; + return true; + } + + if (auto itv = get(ty)) + { + return std::any_of(begin(itv), end(itv), maybeGeneric); + } + + return isGeneric(ty); + } + ty = follow(ty); if (get(ty)) return true; @@ -646,20 +673,10 @@ TypeVar& TypeVar::operator=(TypeVariant&& rhs) TypeVar& TypeVar::operator=(const TypeVar& rhs) { - if (FFlag::LuauNonCopyableTypeVarFields) - { - LUAU_ASSERT(owningArena == rhs.owningArena); - LUAU_ASSERT(!rhs.persistent); + LUAU_ASSERT(owningArena == rhs.owningArena); + LUAU_ASSERT(!rhs.persistent); - reassign(rhs); - } - else - { - ty = rhs.ty; - persistent = rhs.persistent; - normal = rhs.normal; - owningArena = rhs.owningArena; - } + reassign(rhs); return *this; } @@ -676,10 +693,14 @@ static TypeVar threadType_{PrimitiveTypeVar{PrimitiveTypeVar::Thread}, /*persist static TypeVar trueType_{SingletonTypeVar{BooleanSingleton{true}}, /*persistent*/ true}; static TypeVar falseType_{SingletonTypeVar{BooleanSingleton{false}}, /*persistent*/ true}; static TypeVar anyType_{AnyTypeVar{}, /*persistent*/ true}; +static TypeVar unknownType_{UnknownTypeVar{}, /*persistent*/ true}; +static TypeVar neverType_{NeverTypeVar{}, /*persistent*/ true}; static TypeVar errorType_{ErrorTypeVar{}, /*persistent*/ true}; -static TypePackVar anyTypePack_{VariadicTypePack{&anyType_}, true}; -static TypePackVar errorTypePack_{Unifiable::Error{}}; +static TypePackVar anyTypePack_{VariadicTypePack{&anyType_}, /*persistent*/ true}; +static TypePackVar errorTypePack_{Unifiable::Error{}, /*persistent*/ true}; +static TypePackVar neverTypePack_{VariadicTypePack{&neverType_}, /*persistent*/ true}; +static TypePackVar uninhabitableTypePack_{TypePack{{&neverType_}, &neverTypePack_}, /*persistent*/ true}; SingletonTypes::SingletonTypes() : nilType(&nilType_) @@ -690,7 +711,11 @@ SingletonTypes::SingletonTypes() , trueType(&trueType_) , falseType(&falseType_) , anyType(&anyType_) + , unknownType(&unknownType_) + , neverType(&neverType_) , anyTypePack(&anyTypePack_) + , neverTypePack(&neverTypePack_) + , uninhabitableTypePack(&uninhabitableTypePack_) , arena(new TypeArena) { TypeId stringMetatable = makeStringMetatable(); @@ -738,6 +763,7 @@ TypeId SingletonTypes::makeStringMetatable() const TypeId gsubFunc = makeFunction(*arena, stringType, {}, {}, {stringType, replArgType, optionalNumber}, {}, {stringType, numberType}); const TypeId gmatchFunc = makeFunction(*arena, stringType, {}, {}, {stringType}, {}, {arena->addType(FunctionTypeVar{emptyPack, stringVariadicList})}); + attachMagicFunction(gmatchFunc, magicFunctionGmatch); TableTypeVar::Props stringLib = { {"byte", {arena->addType(FunctionTypeVar{arena->addTypePack({stringType, optionalNumber, optionalNumber}), numberVariadicList})}}, @@ -911,6 +937,8 @@ const TypeLevel* getLevel(TypeId ty) return &ttv->level; else if (auto ftv = get(ty)) return &ftv->level; + else if (auto ctv = get(ty)) + return &ctv->level; else return nullptr; } @@ -965,94 +993,19 @@ bool isSubclass(const ClassTypeVar* cls, const ClassTypeVar* parent) return false; } -UnionTypeVarIterator::UnionTypeVarIterator(const UnionTypeVar* utv) +const std::vector& getTypes(const UnionTypeVar* utv) { - LUAU_ASSERT(utv); - - if (!utv->options.empty()) - stack.push_front({utv, 0}); - - seen.insert(utv); + return utv->options; } -UnionTypeVarIterator& UnionTypeVarIterator::operator++() +const std::vector& getTypes(const IntersectionTypeVar* itv) { - advance(); - descend(); - return *this; + return itv->parts; } -UnionTypeVarIterator UnionTypeVarIterator::operator++(int) +const std::vector& getTypes(const ConstrainedTypeVar* ctv) { - UnionTypeVarIterator copy = *this; - ++copy; - return copy; -} - -bool UnionTypeVarIterator::operator!=(const UnionTypeVarIterator& rhs) -{ - return !(*this == rhs); -} - -bool UnionTypeVarIterator::operator==(const UnionTypeVarIterator& rhs) -{ - if (!stack.empty() && !rhs.stack.empty()) - return stack.front() == rhs.stack.front(); - - return stack.empty() && rhs.stack.empty(); -} - -const TypeId& UnionTypeVarIterator::operator*() -{ - LUAU_ASSERT(!stack.empty()); - - descend(); - - auto [utv, currentIndex] = stack.front(); - LUAU_ASSERT(utv); - LUAU_ASSERT(currentIndex < utv->options.size()); - - const TypeId& ty = utv->options[currentIndex]; - LUAU_ASSERT(!get(follow(ty))); - return ty; -} - -void UnionTypeVarIterator::advance() -{ - while (!stack.empty()) - { - auto& [utv, currentIndex] = stack.front(); - ++currentIndex; - - if (currentIndex >= utv->options.size()) - stack.pop_front(); - else - break; - } -} - -void UnionTypeVarIterator::descend() -{ - while (!stack.empty()) - { - auto [utv, currentIndex] = stack.front(); - if (auto innerUnion = get(follow(utv->options[currentIndex]))) - { - // If we're about to descend into a cyclic UnionTypeVar, we should skip over this. - // Ideally this should never happen, but alas it does from time to time. :( - if (seen.find(innerUnion) != seen.end()) - advance(); - else - { - seen.insert(innerUnion); - stack.push_front({innerUnion, 0}); - } - - continue; - } - - break; - } + return ctv->parts; } UnionTypeVarIterator begin(const UnionTypeVar* utv) @@ -1065,6 +1018,27 @@ UnionTypeVarIterator end(const UnionTypeVar* utv) return UnionTypeVarIterator{}; } +IntersectionTypeVarIterator begin(const IntersectionTypeVar* itv) +{ + return IntersectionTypeVarIterator{itv}; +} + +IntersectionTypeVarIterator end(const IntersectionTypeVar* itv) +{ + return IntersectionTypeVarIterator{}; +} + +ConstrainedTypeVarIterator begin(const ConstrainedTypeVar* ctv) +{ + return ConstrainedTypeVarIterator{ctv}; +} + +ConstrainedTypeVarIterator end(const ConstrainedTypeVar* ctv) +{ + return ConstrainedTypeVarIterator{}; +} + + static std::vector parseFormatString(TypeChecker& typechecker, const char* data, size_t size) { const char* options = "cdiouxXeEfgGqs"; @@ -1144,6 +1118,101 @@ std::optional> magicFunctionFormat( return WithPredicate{arena.addTypePack({typechecker.stringType})}; } +static std::vector parsePatternString(TypeChecker& typechecker, const char* data, size_t size) +{ + std::vector result; + int depth = 0; + bool parsingSet = false; + + for (size_t i = 0; i < size; ++i) + { + if (data[i] == '%') + { + ++i; + if (!parsingSet && i < size && data[i] == 'b') + i += 2; + } + else if (!parsingSet && data[i] == '[') + { + parsingSet = true; + if (i + 1 < size && data[i + 1] == ']') + i += 1; + } + else if (parsingSet && data[i] == ']') + { + parsingSet = false; + } + else if (data[i] == '(') + { + if (parsingSet) + continue; + + if (i + 1 < size && data[i + 1] == ')') + { + i++; + result.push_back(typechecker.numberType); + continue; + } + + ++depth; + result.push_back(typechecker.stringType); + } + else if (data[i] == ')') + { + if (parsingSet) + continue; + + --depth; + + if (depth < 0) + break; + } + } + + if (depth != 0 || parsingSet) + return std::vector(); + + if (result.empty()) + result.push_back(typechecker.stringType); + + return result; +} + +static std::optional> magicFunctionGmatch( + TypeChecker& typechecker, const ScopePtr& scope, const AstExprCall& expr, WithPredicate withPredicate) +{ + if (!FFlag::LuauDeduceGmatchReturnTypes) + return std::nullopt; + + auto [paramPack, _predicates] = withPredicate; + const auto& [params, tail] = flatten(paramPack); + + if (params.size() != 2) + return std::nullopt; + + TypeArena& arena = typechecker.currentModule->internalTypes; + + AstExprConstantString* pattern = nullptr; + size_t index = expr.self ? 0 : 1; + if (expr.args.size > index) + pattern = expr.args.data[index]->as(); + + if (!pattern) + return std::nullopt; + + std::vector returnTypes = parsePatternString(typechecker, pattern->value.data, pattern->value.size); + + if (returnTypes.empty()) + return std::nullopt; + + typechecker.unify(params[0], typechecker.stringType, expr.args.data[0]->location); + + const TypePackId emptyPack = arena.addTypePack({}); + const TypePackId returnList = arena.addTypePack(returnTypes); + const TypeId iteratorType = arena.addType(FunctionTypeVar{emptyPack, returnList}); + return WithPredicate{arena.addTypePack({iteratorType})}; +} + std::vector filterMap(TypeId type, TypeIdPredicate predicate) { type = follow(type); diff --git a/Analysis/src/Unifier.cpp b/Analysis/src/Unifier.cpp index 0792a350..44a3b854 100644 --- a/Analysis/src/Unifier.cpp +++ b/Analysis/src/Unifier.cpp @@ -19,6 +19,7 @@ LUAU_FASTFLAG(LuauAutocompleteDynamicLimits) LUAU_FASTINTVARIABLE(LuauTypeInferLowerBoundsIterationLimit, 2000); LUAU_FASTFLAG(LuauLowerBoundsCalculation); LUAU_FASTFLAG(LuauErrorRecoveryType); +LUAU_FASTFLAG(LuauUnknownAndNeverType) LUAU_FASTFLAG(LuauQuantifyConstrained) namespace Luau @@ -47,33 +48,6 @@ struct PromoteTypeLevels final : TypeVarOnceVisitor } } - // TODO cycle and operator() need to be clipped when FFlagLuauUseVisitRecursionLimit is clipped - template - void cycle(TID) - { - } - template - bool operator()(TID ty, const T&) - { - return visit(ty); - } - bool operator()(TypeId ty, const FreeTypeVar& ftv) - { - return visit(ty, ftv); - } - bool operator()(TypeId ty, const FunctionTypeVar& ftv) - { - return visit(ty, ftv); - } - bool operator()(TypeId ty, const TableTypeVar& ttv) - { - return visit(ty, ttv); - } - bool operator()(TypePackId tp, const FreeTypePack& ftp) - { - return visit(tp, ftp); - } - bool visit(TypeId ty) override { // Type levels of types from other modules are already global, so we don't need to promote anything inside @@ -103,6 +77,15 @@ struct PromoteTypeLevels final : TypeVarOnceVisitor return true; } + bool visit(TypeId ty, const ConstrainedTypeVar&) override + { + if (!FFlag::LuauUnknownAndNeverType) + return visit(ty); + + promote(ty, log.getMutable(ty)); + return true; + } + bool visit(TypeId ty, const FunctionTypeVar&) override { // Type levels of types from other modules are already global, so we don't need to promote anything inside @@ -445,6 +428,14 @@ void Unifier::tryUnify_(TypeId subTy, TypeId superTy, bool isFunctionCall, bool } else if (subFree) { + if (FFlag::LuauUnknownAndNeverType) + { + // Normally, if the subtype is free, it should not be bound to any, unknown, or error types. + // But for bug compatibility, we'll only apply this rule to unknown. Doing this will silence cascading type errors. + if (get(superTy)) + return; + } + TypeLevel subLevel = subFree->level; occursCheck(subTy, superTy); @@ -468,7 +459,7 @@ void Unifier::tryUnify_(TypeId subTy, TypeId superTy, bool isFunctionCall, bool return; } - if (get(superTy) || get(superTy)) + if (get(superTy) || get(superTy) || get(superTy)) return tryUnifyWithAny(subTy, superTy); if (get(subTy)) @@ -485,6 +476,9 @@ void Unifier::tryUnify_(TypeId subTy, TypeId superTy, bool isFunctionCall, bool if (get(subTy)) return tryUnifyWithAny(superTy, subTy); + if (get(subTy)) + return tryUnifyWithAny(superTy, subTy); + auto& cache = sharedState.cachedUnify; // What if the types are immutable and we proved their relation before @@ -1862,6 +1856,7 @@ static void tryUnifyWithAny(std::vector& queue, Unifier& state, DenseHas if (state.log.getMutable(ty)) { + // TODO: Only bind if the anyType isn't any, unknown, or error (?) state.log.replace(ty, BoundTypeVar{anyType}); } else if (auto fun = state.log.getMutable(ty)) @@ -1901,22 +1896,27 @@ static void tryUnifyWithAny(std::vector& queue, Unifier& state, DenseHas void Unifier::tryUnifyWithAny(TypeId subTy, TypeId anyTy) { - LUAU_ASSERT(get(anyTy) || get(anyTy)); + LUAU_ASSERT(get(anyTy) || get(anyTy) || get(anyTy) || get(anyTy)); // These types are not visited in general loop below if (get(subTy) || get(subTy) || get(subTy)) return; - const TypePackId anyTypePack = types->addTypePack(TypePackVar{VariadicTypePack{getSingletonTypes().anyType}}); - - const TypePackId anyTP = get(anyTy) ? anyTypePack : types->addTypePack(TypePackVar{Unifiable::Error{}}); + TypePackId anyTp; + if (FFlag::LuauUnknownAndNeverType) + anyTp = types->addTypePack(TypePackVar{VariadicTypePack{anyTy}}); + else + { + const TypePackId anyTypePack = types->addTypePack(TypePackVar{VariadicTypePack{getSingletonTypes().anyType}}); + anyTp = get(anyTy) ? anyTypePack : types->addTypePack(TypePackVar{Unifiable::Error{}}); + } std::vector queue = {subTy}; sharedState.tempSeenTy.clear(); sharedState.tempSeenTp.clear(); - Luau::tryUnifyWithAny(queue, *this, sharedState.tempSeenTy, sharedState.tempSeenTp, types, getSingletonTypes().anyType, anyTP); + Luau::tryUnifyWithAny(queue, *this, sharedState.tempSeenTy, sharedState.tempSeenTp, types, FFlag::LuauUnknownAndNeverType ? anyTy : getSingletonTypes().anyType, anyTp); } void Unifier::tryUnifyWithAny(TypePackId subTy, TypePackId anyTp) diff --git a/Ast/src/Parser.cpp b/Ast/src/Parser.cpp index 70c92555..b7fa7889 100644 --- a/Ast/src/Parser.cpp +++ b/Ast/src/Parser.cpp @@ -15,7 +15,6 @@ LUAU_FASTINTVARIABLE(LuauRecursionLimit, 1000) LUAU_FASTINTVARIABLE(LuauParseErrorLimit, 100) LUAU_FASTFLAGVARIABLE(LuauParserFunctionKeywordAsTypeHelp, false) -LUAU_FASTFLAGVARIABLE(LuauReturnTypeTokenConfusion, false) LUAU_FASTFLAGVARIABLE(LuauFixNamedFunctionParse, false) LUAU_DYNAMIC_FASTFLAGVARIABLE(LuaReportParseWrongNamedType, false) @@ -1134,10 +1133,9 @@ AstTypePack* Parser::parseTypeList(TempVector& result, TempVector Parser::parseOptionalReturnTypeAnnotation() { - if (options.allowTypeAnnotations && - (lexer.current().type == ':' || (FFlag::LuauReturnTypeTokenConfusion && lexer.current().type == Lexeme::SkinnyArrow))) + if (options.allowTypeAnnotations && (lexer.current().type == ':' || lexer.current().type == Lexeme::SkinnyArrow)) { - if (FFlag::LuauReturnTypeTokenConfusion && lexer.current().type == Lexeme::SkinnyArrow) + if (lexer.current().type == Lexeme::SkinnyArrow) report(lexer.current().location, "Function return type annotations are written after ':' instead of '->'"); nextLexeme(); @@ -1373,12 +1371,10 @@ AstTypeOrPack Parser::parseFunctionTypeAnnotation(bool allowPack) if (FFlag::LuauFixNamedFunctionParse && !names.empty()) forceFunctionType = true; - bool returnTypeIntroducer = - FFlag::LuauReturnTypeTokenConfusion ? lexer.current().type == Lexeme::SkinnyArrow || lexer.current().type == ':' : false; + bool returnTypeIntroducer = lexer.current().type == Lexeme::SkinnyArrow || lexer.current().type == ':'; // Not a function at all. Just a parenthesized type. Or maybe a type pack with a single element - if (params.size() == 1 && !varargAnnotation && !forceFunctionType && - (FFlag::LuauReturnTypeTokenConfusion ? !returnTypeIntroducer : lexer.current().type != Lexeme::SkinnyArrow)) + if (params.size() == 1 && !varargAnnotation && !forceFunctionType && !returnTypeIntroducer) { if (DFFlag::LuaReportParseWrongNamedType && !names.empty()) lua_telemetry_parsed_named_non_function_type = true; @@ -1389,8 +1385,7 @@ AstTypeOrPack Parser::parseFunctionTypeAnnotation(bool allowPack) return {params[0], {}}; } - if ((FFlag::LuauReturnTypeTokenConfusion ? !returnTypeIntroducer : lexer.current().type != Lexeme::SkinnyArrow) && !forceFunctionType && - allowPack) + if (!forceFunctionType && !returnTypeIntroducer && allowPack) { if (DFFlag::LuaReportParseWrongNamedType && !names.empty()) lua_telemetry_parsed_named_non_function_type = true; @@ -1409,7 +1404,7 @@ AstType* Parser::parseFunctionTypeAnnotationTail(const Lexeme& begin, AstArray' instead of ':'"); lexer.next(); diff --git a/CLI/Analyze.cpp b/CLI/Analyze.cpp index 4bc8cab5..479eb167 100644 --- a/CLI/Analyze.cpp +++ b/CLI/Analyze.cpp @@ -8,6 +8,10 @@ #include "FileUtils.h" +#ifdef CALLGRIND +#include +#endif + LUAU_FASTFLAG(DebugLuauTimeTracing) LUAU_FASTFLAG(LuauTypeMismatchModuleNameResolution) @@ -112,6 +116,7 @@ static void displayHelp(const char* argv0) printf("Available options:\n"); printf(" --formatter=plain: report analysis errors in Luacheck-compatible format\n"); printf(" --formatter=gnu: report analysis errors in GNU-compatible format\n"); + printf(" --mode=strict: default to strict mode when typechecking\n"); printf(" --timetrace: record compiler time tracing information into trace.json\n"); } @@ -178,9 +183,9 @@ struct CliConfigResolver : Luau::ConfigResolver mutable std::unordered_map configCache; mutable std::vector> configErrors; - CliConfigResolver() + CliConfigResolver(Luau::Mode mode) { - defaultConfig.mode = Luau::Mode::Nonstrict; + defaultConfig.mode = mode; } const Luau::Config& getConfig(const Luau::ModuleName& name) const override @@ -229,6 +234,7 @@ int main(int argc, char** argv) } ReportFormat format = ReportFormat::Default; + Luau::Mode mode = Luau::Mode::Nonstrict; bool annotate = false; for (int i = 1; i < argc; ++i) @@ -240,6 +246,8 @@ int main(int argc, char** argv) format = ReportFormat::Luacheck; else if (strcmp(argv[i], "--formatter=gnu") == 0) format = ReportFormat::Gnu; + else if (strcmp(argv[i], "--mode=strict") == 0) + mode = Luau::Mode::Strict; else if (strcmp(argv[i], "--annotate") == 0) annotate = true; else if (strcmp(argv[i], "--timetrace") == 0) @@ -258,12 +266,16 @@ int main(int argc, char** argv) frontendOptions.retainFullTypeGraphs = annotate; CliFileResolver fileResolver; - CliConfigResolver configResolver; + CliConfigResolver configResolver(mode); Luau::Frontend frontend(&fileResolver, &configResolver, frontendOptions); Luau::registerBuiltinTypes(frontend.typeChecker); Luau::freeze(frontend.typeChecker.globalTypes); +#ifdef CALLGRIND + CALLGRIND_ZERO_STATS; +#endif + std::vector files = getSourceFiles(argc, argv); int failed = 0; diff --git a/CLI/Repl.cpp b/CLI/Repl.cpp index 83060f5b..5fe12bec 100644 --- a/CLI/Repl.cpp +++ b/CLI/Repl.cpp @@ -21,6 +21,10 @@ #include #endif +#ifdef CALLGRIND +#include +#endif + #include LUAU_FASTFLAG(DebugLuauTimeTracing) @@ -166,6 +170,36 @@ static int lua_collectgarbage(lua_State* L) luaL_error(L, "collectgarbage must be called with 'count' or 'collect'"); } +#ifdef CALLGRIND +static int lua_callgrind(lua_State* L) +{ + const char* option = luaL_checkstring(L, 1); + + if (strcmp(option, "running") == 0) + { + int r = RUNNING_ON_VALGRIND; + lua_pushboolean(L, r); + return 1; + } + + if (strcmp(option, "zero") == 0) + { + CALLGRIND_ZERO_STATS; + return 0; + } + + if (strcmp(option, "dump") == 0) + { + const char* name = luaL_checkstring(L, 2); + + CALLGRIND_DUMP_STATS_AT(name); + return 0; + } + + luaL_error(L, "callgrind must be called with one of 'running', 'zero', 'dump'"); +} +#endif + void setupState(lua_State* L) { luaL_openlibs(L); @@ -174,6 +208,9 @@ void setupState(lua_State* L) {"loadstring", lua_loadstring}, {"require", lua_require}, {"collectgarbage", lua_collectgarbage}, +#ifdef CALLGRIND + {"callgrind", lua_callgrind}, +#endif {NULL, NULL}, }; diff --git a/CodeGen/include/Luau/AssemblyBuilderX64.h b/CodeGen/include/Luau/AssemblyBuilderX64.h index c5979d3c..65883b49 100644 --- a/CodeGen/include/Luau/AssemblyBuilderX64.h +++ b/CodeGen/include/Luau/AssemblyBuilderX64.h @@ -58,6 +58,9 @@ public: void jmp(Label& label); void jmp(OperandX64 op); + void call(Label& label); + void call(OperandX64 op); + // AVX void vaddpd(OperandX64 dst, OperandX64 src1, OperandX64 src2); void vaddps(OperandX64 dst, OperandX64 src1, OperandX64 src2); diff --git a/CodeGen/src/AssemblyBuilderX64.cpp b/CodeGen/src/AssemblyBuilderX64.cpp index 27e01781..26347225 100644 --- a/CodeGen/src/AssemblyBuilderX64.cpp +++ b/CodeGen/src/AssemblyBuilderX64.cpp @@ -286,11 +286,34 @@ void AssemblyBuilderX64::jmp(OperandX64 op) if (logText) log("jmp", op); + placeRex(op); place(0xff); placeModRegMem(op, 4); commit(); } +void AssemblyBuilderX64::call(Label& label) +{ + place(0xe8); + placeLabel(label); + + if (logText) + log("call", label); + + commit(); +} + +void AssemblyBuilderX64::call(OperandX64 op) +{ + if (logText) + log("call", op); + + placeRex(op); + place(0xff); + placeModRegMem(op, 2); + commit(); +} + void AssemblyBuilderX64::vaddpd(OperandX64 dst, OperandX64 src1, OperandX64 src2) { placeAvx("vaddpd", dst, src1, src2, 0x58, false, AVX_0F, AVX_66); diff --git a/Makefile b/Makefile index d4ef7c79..a0a52208 100644 --- a/Makefile +++ b/Makefile @@ -93,6 +93,10 @@ ifeq ($(config),fuzz) LDFLAGS+=-fsanitize=address,fuzzer endif +ifneq ($(CALLGRIND),) + CXXFLAGS+=-DCALLGRIND=$(CALLGRIND) +endif + # target-specific flags $(AST_OBJECTS): CXXFLAGS+=-std=c++17 -ICommon/include -IAst/include $(COMPILER_OBJECTS): CXXFLAGS+=-std=c++17 -ICompiler/include -ICommon/include -IAst/include diff --git a/Sources.cmake b/Sources.cmake index f261cba6..44bed8f7 100644 --- a/Sources.cmake +++ b/Sources.cmake @@ -247,12 +247,15 @@ if(TARGET Luau.UnitTest) tests/IostreamOptional.h tests/ScopedFlags.h tests/Fixture.cpp + tests/AssemblyBuilderX64.test.cpp tests/AstQuery.test.cpp tests/AstVisitor.test.cpp tests/Autocomplete.test.cpp tests/BuiltinDefinitions.test.cpp tests/Compiler.test.cpp tests/Config.test.cpp + tests/ConstraintGraphBuilder.test.cpp + tests/ConstraintSolver.test.cpp tests/CostModel.test.cpp tests/Error.test.cpp tests/Frontend.test.cpp @@ -262,8 +265,7 @@ if(TARGET Luau.UnitTest) tests/Module.test.cpp tests/NonstrictMode.test.cpp tests/Normalize.test.cpp - tests/ConstraintGraphBuilder.test.cpp - tests/ConstraintSolver.test.cpp + tests/NotNull.test.cpp tests/Parser.test.cpp tests/RequireTracer.test.cpp tests/RuntimeLimits.test.cpp @@ -295,11 +297,11 @@ if(TARGET Luau.UnitTest) tests/TypeInfer.tryUnify.test.cpp tests/TypeInfer.typePacks.cpp tests/TypeInfer.unionTypes.test.cpp + tests/TypeInfer.unknownnever.test.cpp tests/TypePack.test.cpp tests/TypeVar.test.cpp tests/Variant.test.cpp tests/VisitTypeVar.test.cpp - tests/AssemblyBuilderX64.test.cpp tests/main.cpp) endif() diff --git a/VM/src/ltable.cpp b/VM/src/ltable.cpp index 2316cc3d..79e65919 100644 --- a/VM/src/ltable.cpp +++ b/VM/src/ltable.cpp @@ -108,9 +108,9 @@ static LuaNode* hashvec(const Table* t, const float* v) memcpy(i, v, sizeof(i)); // convert -0 to 0 to make sure they hash to the same value - i[0] = (i[0] == 0x8000000) ? 0 : i[0]; - i[1] = (i[1] == 0x8000000) ? 0 : i[1]; - i[2] = (i[2] == 0x8000000) ? 0 : i[2]; + i[0] = (i[0] == 0x80000000) ? 0 : i[0]; + i[1] = (i[1] == 0x80000000) ? 0 : i[1]; + i[2] = (i[2] == 0x80000000) ? 0 : i[2]; // scramble bits to make sure that integer coordinates have entropy in lower bits i[0] ^= i[0] >> 17; @@ -121,7 +121,7 @@ static LuaNode* hashvec(const Table* t, const float* v) unsigned int h = (i[0] * 73856093) ^ (i[1] * 19349663) ^ (i[2] * 83492791); #if LUA_VECTOR_SIZE == 4 - i[3] = (i[3] == 0x8000000) ? 0 : i[3]; + i[3] = (i[3] == 0x80000000) ? 0 : i[3]; i[3] ^= i[3] >> 17; h ^= i[3] * 39916801; #endif diff --git a/VM/src/lvmexecute.cpp b/VM/src/lvmexecute.cpp index 85829ca1..02b39313 100644 --- a/VM/src/lvmexecute.cpp +++ b/VM/src/lvmexecute.cpp @@ -640,20 +640,16 @@ static void luau_execute(lua_State* L) VM_PATCH_C(pc - 2, L->cachedslot); VM_NEXT(); } - else - { - // slow-path, may invoke Lua calls via __index metamethod - VM_PROTECT(luaV_gettable(L, rb, kv, ra)); - VM_NEXT(); - } - } - else - { - // slow-path, may invoke Lua calls via __index metamethod - VM_PROTECT(luaV_gettable(L, rb, kv, ra)); - VM_NEXT(); + + // fall through to slow path } + + // fall through to slow path } + + // slow-path, may invoke Lua calls via __index metamethod + VM_PROTECT(luaV_gettable(L, rb, kv, ra)); + VM_NEXT(); } VM_CASE(LOP_SETTABLEKS) @@ -753,19 +749,13 @@ static void luau_execute(lua_State* L) setobj2s(L, ra, &h->array[unsigned(index - 1)]); VM_NEXT(); } - else - { - // slow-path: handles out of bounds array lookups and non-integer numeric keys - VM_PROTECT(luaV_gettable(L, rb, rc, ra)); - VM_NEXT(); - } - } - else - { - // slow-path: handles non-array table lookup as well as __index MT calls - VM_PROTECT(luaV_gettable(L, rb, rc, ra)); - VM_NEXT(); + + // fall through to slow path } + + // slow-path: handles out of bounds array lookups, non-integer numeric keys, non-array table lookup, __index MT calls + VM_PROTECT(luaV_gettable(L, rb, rc, ra)); + VM_NEXT(); } VM_CASE(LOP_SETTABLE) @@ -790,19 +780,13 @@ static void luau_execute(lua_State* L) luaC_barriert(L, h, ra); VM_NEXT(); } - else - { - // slow-path: handles out of bounds array assignments and non-integer numeric keys - VM_PROTECT(luaV_settable(L, rb, rc, ra)); - VM_NEXT(); - } - } - else - { - // slow-path: handles non-array table access as well as __newindex MT calls - VM_PROTECT(luaV_settable(L, rb, rc, ra)); - VM_NEXT(); + + // fall through to slow path } + + // slow-path: handles out of bounds array assignments, non-integer numeric keys, non-array table access, __newindex MT calls + VM_PROTECT(luaV_settable(L, rb, rc, ra)); + VM_NEXT(); } VM_CASE(LOP_GETTABLEN) @@ -822,6 +806,8 @@ static void luau_execute(lua_State* L) setobj2s(L, ra, &h->array[c]); VM_NEXT(); } + + // fall through to slow path } // slow-path: handles out of bounds array lookups @@ -849,6 +835,8 @@ static void luau_execute(lua_State* L) luaC_barriert(L, h, ra); VM_NEXT(); } + + // fall through to slow path } // slow-path: handles out of bounds array lookups @@ -2176,8 +2164,10 @@ static void luau_execute(lua_State* L) if (!ttisnumber(ra + 0) || !ttisnumber(ra + 1) || !ttisnumber(ra + 2)) { // slow-path: can convert arguments to numbers and trigger Lua errors - // Note: this doesn't reallocate stack so we don't need to recompute ra - VM_PROTECT(luau_prepareFORN(L, ra + 0, ra + 1, ra + 2)); + // Note: this doesn't reallocate stack so we don't need to recompute ra/base + VM_PROTECT_PC(); + + luau_prepareFORN(L, ra + 0, ra + 1, ra + 2); } double limit = nvalue(ra + 0); diff --git a/bench/bench.py b/bench/bench.py index 67fc8cf7..bb3ea5f7 100644 --- a/bench/bench.py +++ b/bench/bench.py @@ -40,6 +40,7 @@ argumentParser.add_argument('--results', dest='results',type=str,nargs='*',help= argumentParser.add_argument('--run-test', action='store', default=None, help='Regex test filter') argumentParser.add_argument('--extra-loops', action='store',type=int,default=0, help='Amount of times to loop over one test (one test already performs multiple runs)') argumentParser.add_argument('--filename', action='store',type=str,default='bench', help='File name for graph and results file') +argumentParser.add_argument('--callgrind', dest='callgrind',action='store_const',const=1,default=0,help='Use callgrind to run benchmarks') if matplotlib != None: argumentParser.add_argument('--absolute', dest='absolute',action='store_const',const=1,default=0,help='Display absolute values instead of relative (enabled by default when benchmarking a single VM)') @@ -55,6 +56,9 @@ argumentParser.add_argument('--no-print-influx-debugging', action='store_false', argumentParser.add_argument('--no-print-final-summary', action='store_false', dest='print_final_summary', help="Don't print a table summarizing the results after all tests are run") +# Assume 2.5 IPC on a 4 GHz CPU; this is obviously incorrect but it allows us to display simulated instruction counts using regular time units +CALLGRIND_INSN_PER_SEC = 2.5 * 4e9 + def arrayRange(count): result = [] @@ -71,6 +75,21 @@ def arrayRangeOffset(count, offset): return result +def getCallgrindOutput(lines): + result = [] + name = None + + for l in lines: + if l.startswith("desc: Trigger: Client Request: "): + name = l[31:].strip() + elif l.startswith("summary: ") and name != None: + insn = int(l[9:]) + # Note: we only run each bench once under callgrind so we only report a single time per run; callgrind instruction count variance is ~0.01% so it might as well be zero + result += "|><|" + name + "|><|" + str(insn / CALLGRIND_INSN_PER_SEC * 1000.0) + "||_||" + name = None + + return "".join(result) + def getVmOutput(cmd): if os.name == "nt": try: @@ -79,6 +98,16 @@ def getVmOutput(cmd): exit(1) except: return "" + elif arguments.callgrind: + try: + subprocess.check_call("valgrind --tool=callgrind --callgrind-out-file=callgrind.out --combine-dumps=yes --dump-line=no " + cmd, shell=True, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, cwd=scriptdir) + path = os.path.join(scriptdir, "callgrind.out") + with open(path, "r") as file: + lines = file.readlines() + os.unlink(path) + return getCallgrindOutput(lines) + except: + return "" else: with subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True, cwd=scriptdir) as p: # Try to lock to a single processor @@ -375,12 +404,12 @@ def analyzeResult(subdir, main, comparisons): continue - pooledStdDev = math.sqrt((main.unbiasedEst + compare.unbiasedEst) / 2) + if main.count > 1 and stats: + pooledStdDev = math.sqrt((main.unbiasedEst + compare.unbiasedEst) / 2) - tStat = abs(main.avg - compare.avg) / (pooledStdDev * math.sqrt(2 / main.count)) - degreesOfFreedom = 2 * main.count - 2 + tStat = abs(main.avg - compare.avg) / (pooledStdDev * math.sqrt(2 / main.count)) + degreesOfFreedom = 2 * main.count - 2 - if stats: # Two-tailed distribution with 95% conf. tCritical = stats.t.ppf(1 - 0.05 / 2, degreesOfFreedom) diff --git a/bench/bench_support.lua b/bench/bench_support.lua index 171b8da7..a9608ecc 100644 --- a/bench/bench_support.lua +++ b/bench/bench_support.lua @@ -5,6 +5,16 @@ bench.runs = 20 bench.extraRuns = 4 function bench.runCode(f, description) + -- Under Callgrind, run the test only once and measure just the execution cost + if callgrind and callgrind("running") then + if collectgarbage then collectgarbage() end + + callgrind("zero") + f() -- unfortunately we can't easily separate setup cost from runtime cost in f unless it calls callgrind() + callgrind("dump", description) + return + end + local timeTable = {} for i = 1,bench.runs + bench.extraRuns do diff --git a/bench/other/LuauPolyfillMap.lua b/bench/other/LuauPolyfillMap.lua new file mode 100644 index 00000000..1f957d47 --- /dev/null +++ b/bench/other/LuauPolyfillMap.lua @@ -0,0 +1,961 @@ +-- This file is part of the Roblox luau-polyfill repository and is licensed under MIT License; see LICENSE.txt for details +-- #region Array +-- Array related +local Array = {} +local Object = {} +local Map = {} + +type Array = { [number]: T } +type callbackFn = (element: V, key: K, map: Map) -> () +type callbackFnWithThisArg = (thisArg: Object, value: V, key: K, map: Map) -> () +type Map = { + size: number, + -- method definitions + set: (self: Map, K, V) -> Map, + get: (self: Map, K) -> V | nil, + clear: (self: Map) -> (), + delete: (self: Map, K) -> boolean, + forEach: (self: Map, callback: callbackFn | callbackFnWithThisArg, thisArg: Object?) -> (), + has: (self: Map, K) -> boolean, + keys: (self: Map) -> Array, + values: (self: Map) -> Array, + entries: (self: Map) -> Array>, + ipairs: (self: Map) -> any, + [K]: V, + _map: { [K]: V }, + _array: { [number]: K }, +} +type mapFn = (element: T, index: number) -> U +type mapFnWithThisArg = (thisArg: any, element: T, index: number) -> U +type Object = { [string]: any } +type Table = { [T]: V } +type Tuple = Array + +local Set = {} + +-- #region Array +function Array.isArray(value: any): boolean + if typeof(value) ~= "table" then + return false + end + if next(value) == nil then + -- an empty table is an empty array + return true + end + + local length = #value + + if length == 0 then + return false + end + + local count = 0 + local sum = 0 + for key in pairs(value) do + if typeof(key) ~= "number" then + return false + end + if key % 1 ~= 0 or key < 1 then + return false + end + count += 1 + sum += key + end + + return sum == (count * (count + 1) / 2) +end + +function Array.from( + value: string | Array | Object, + mapFn: (mapFn | mapFnWithThisArg)?, + thisArg: Object? +): Array + if value == nil then + error("cannot create array from a nil value") + end + local valueType = typeof(value) + + local array = {} + + if valueType == "table" and Array.isArray(value) then + if mapFn then + for i = 1, #(value :: Array) do + if thisArg ~= nil then + array[i] = (mapFn :: mapFnWithThisArg)(thisArg, (value :: Array)[i], i) + else + array[i] = (mapFn :: mapFn)((value :: Array)[i], i) + end + end + else + for i = 1, #(value :: Array) do + array[i] = (value :: Array)[i] + end + end + elseif instanceOf(value, Set) then + if mapFn then + for i, v in (value :: any):ipairs() do + if thisArg ~= nil then + array[i] = (mapFn :: mapFnWithThisArg)(thisArg, v, i) + else + array[i] = (mapFn :: mapFn)(v, i) + end + end + else + for i, v in (value :: any):ipairs() do + array[i] = v + end + end + elseif instanceOf(value, Map) then + if mapFn then + for i, v in (value :: any):ipairs() do + if thisArg ~= nil then + array[i] = (mapFn :: mapFnWithThisArg)(thisArg, v, i) + else + array[i] = (mapFn :: mapFn)(v, i) + end + end + else + for i, v in (value :: any):ipairs() do + array[i] = v + end + end + elseif valueType == "string" then + if mapFn then + for i = 1, (value :: string):len() do + if thisArg ~= nil then + array[i] = (mapFn :: mapFnWithThisArg)(thisArg, (value :: any):sub(i, i), i) + else + array[i] = (mapFn :: mapFn)((value :: any):sub(i, i), i) + end + end + else + for i = 1, (value :: string):len() do + array[i] = (value :: any):sub(i, i) + end + end + end + + return array +end + +type callbackFnArrayMap = (element: T, index: number, array: Array) -> U +type callbackFnWithThisArgArrayMap = (thisArg: V, element: T, index: number, array: Array) -> U + +-- Implements Javascript's `Array.prototype.map` as defined below +-- https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/map +function Array.map( + t: Array, + callback: callbackFnArrayMap | callbackFnWithThisArgArrayMap, + thisArg: V? +): Array + if typeof(t) ~= "table" then + error(string.format("Array.map called on %s", typeof(t))) + end + if typeof(callback) ~= "function" then + error("callback is not a function") + end + + local len = #t + local A = {} + local k = 1 + + while k <= len do + local kValue = t[k] + + if kValue ~= nil then + local mappedValue + + if thisArg ~= nil then + mappedValue = (callback :: callbackFnWithThisArgArrayMap)(thisArg, kValue, k, t) + else + mappedValue = (callback :: callbackFnArrayMap)(kValue, k, t) + end + + A[k] = mappedValue + end + k += 1 + end + + return A +end + +type Function = (any, any, number, any) -> any + +-- Implements Javascript's `Array.prototype.reduce` as defined below +-- https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/reduce +function Array.reduce(array: Array, callback: Function, initialValue: any?): any + if typeof(array) ~= "table" then + error(string.format("Array.reduce called on %s", typeof(array))) + end + if typeof(callback) ~= "function" then + error("callback is not a function") + end + + local length = #array + + local value + local initial = 1 + + if initialValue ~= nil then + value = initialValue + else + initial = 2 + if length == 0 then + error("reduce of empty array with no initial value") + end + value = array[1] + end + + for i = initial, length do + value = callback(value, array[i], i, array) + end + + return value +end + +type callbackFnArrayForEach = (element: T, index: number, array: Array) -> () +type callbackFnWithThisArgArrayForEach = (thisArg: U, element: T, index: number, array: Array) -> () + +-- Implements Javascript's `Array.prototype.forEach` as defined below +-- https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/forEach +function Array.forEach( + t: Array, + callback: callbackFnArrayForEach | callbackFnWithThisArgArrayForEach, + thisArg: U? +): () + if typeof(t) ~= "table" then + error(string.format("Array.forEach called on %s", typeof(t))) + end + if typeof(callback) ~= "function" then + error("callback is not a function") + end + + local len = #t + local k = 1 + + while k <= len do + local kValue = t[k] + + if thisArg ~= nil then + (callback :: callbackFnWithThisArgArrayForEach)(thisArg, kValue, k, t) + else + (callback :: callbackFnArrayForEach)(kValue, k, t) + end + + if #t < len then + -- don't iterate on removed items, don't iterate more than original length + len = #t + end + k += 1 + end +end +-- #endregion + +-- #region Set +Set.__index = Set + +type callbackFnSet = (value: T, key: T, set: Set) -> () +type callbackFnWithThisArgSet = (thisArg: Object, value: T, key: T, set: Set) -> () + +export type Set = { + size: number, + -- method definitions + add: (self: Set, T) -> Set, + clear: (self: Set) -> (), + delete: (self: Set, T) -> boolean, + forEach: (self: Set, callback: callbackFnSet | callbackFnWithThisArgSet, thisArg: Object?) -> (), + has: (self: Set, T) -> boolean, + ipairs: (self: Set) -> any, +} + +type Iterable = { ipairs: (any) -> any } + +function Set.new(iterable: Array | Set | Iterable | string | nil): Set + local array = {} + local map = {} + if iterable ~= nil then + local arrayIterable: Array + -- ROBLOX TODO: remove type casting from (iterable :: any).ipairs in next release + if typeof(iterable) == "table" then + if Array.isArray(iterable) then + arrayIterable = Array.from(iterable :: Array) + elseif typeof((iterable :: Iterable).ipairs) == "function" then + -- handle in loop below + elseif _G.__DEV__ then + error("cannot create array from an object-like table") + end + elseif typeof(iterable) == "string" then + arrayIterable = Array.from(iterable :: string) + else + error(("cannot create array from value of type `%s`"):format(typeof(iterable))) + end + + if arrayIterable then + for _, element in ipairs(arrayIterable) do + if not map[element] then + map[element] = true + table.insert(array, element) + end + end + elseif typeof(iterable) == "table" and typeof((iterable :: Iterable).ipairs) == "function" then + for _, element in (iterable :: Iterable):ipairs() do + if not map[element] then + map[element] = true + table.insert(array, element) + end + end + end + end + + return (setmetatable({ + size = #array, + _map = map, + _array = array, + }, Set) :: any) :: Set +end + +function Set:add(value) + if not self._map[value] then + -- Luau FIXME: analyze should know self is Set which includes size as a number + self.size = self.size :: number + 1 + self._map[value] = true + table.insert(self._array, value) + end + return self +end + +function Set:clear() + self.size = 0 + table.clear(self._map) + table.clear(self._array) +end + +function Set:delete(value): boolean + if not self._map[value] then + return false + end + -- Luau FIXME: analyze should know self is Map which includes size as a number + self.size = self.size :: number - 1 + self._map[value] = nil + local index = table.find(self._array, value) + if index then + table.remove(self._array, index) + end + return true +end + +-- Implements Javascript's `Map.prototype.forEach` as defined below +-- https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Set/forEach +function Set:forEach(callback: callbackFnSet | callbackFnWithThisArgSet, thisArg: Object?): () + if typeof(callback) ~= "function" then + error("callback is not a function") + end + + return Array.forEach(self._array, function(value: T) + if thisArg ~= nil then + (callback :: callbackFnWithThisArgSet)(thisArg, value, value, self) + else + (callback :: callbackFnSet)(value, value, self) + end + end) +end + +function Set:has(value): boolean + return self._map[value] ~= nil +end + +function Set:ipairs() + return ipairs(self._array) +end + +-- #endregion Set + +-- #region Object +function Object.entries(value: string | Object | Array): Array + assert(value :: any ~= nil, "cannot get entries from a nil value") + local valueType = typeof(value) + + local entries: Array> = {} + if valueType == "table" then + for key, keyValue in pairs(value :: Object) do + -- Luau FIXME: Luau should see entries as Array, given object is [string]: any, but it sees it as Array> despite all the manual annotation + table.insert(entries, { key :: string, keyValue :: any }) + end + elseif valueType == "string" then + for i = 1, string.len(value :: string) do + entries[i] = { tostring(i), string.sub(value :: string, i, i) } + end + end + + return entries +end + +-- #endregion + +-- #region instanceOf + +-- ROBLOX note: Typed tbl as any to work with strict type analyze +-- polyfill for https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/instanceof +function instanceOf(tbl: any, class) + assert(typeof(class) == "table", "Received a non-table as the second argument for instanceof") + + if typeof(tbl) ~= "table" then + return false + end + + local ok, hasNew = pcall(function() + return class.new ~= nil and tbl.new == class.new + end) + if ok and hasNew then + return true + end + + local seen = { tbl = true } + + while tbl and typeof(tbl) == "table" do + tbl = getmetatable(tbl) + if typeof(tbl) == "table" then + tbl = tbl.__index + + if tbl == class then + return true + end + end + + -- if we still have a valid table then check against seen + if typeof(tbl) == "table" then + if seen[tbl] then + return false + end + seen[tbl] = true + end + end + + return false +end +-- #endregion + +function Map.new(iterable: Array>?): Map + local array = {} + local map = {} + if iterable ~= nil then + local arrayFromIterable + local iterableType = typeof(iterable) + if iterableType == "table" then + if #iterable > 0 and typeof(iterable[1]) ~= "table" then + error("cannot create Map from {K, V} form, it must be { {K, V}... }") + end + + arrayFromIterable = Array.from(iterable) + else + error(("cannot create array from value of type `%s`"):format(iterableType)) + end + + for _, entry in ipairs(arrayFromIterable) do + local key = entry[1] + if _G.__DEV__ then + if key == nil then + error("cannot create Map from a table that isn't an array.") + end + end + local val = entry[2] + -- only add to array if new + if map[key] == nil then + table.insert(array, key) + end + -- always assign + map[key] = val + end + end + + return (setmetatable({ + size = #array, + _map = map, + _array = array, + }, Map) :: any) :: Map +end + +function Map:set(key: K, value: V): Map + -- preserve initial insertion order + if self._map[key] == nil then + -- Luau FIXME: analyze should know self is Map which includes size as a number + self.size = self.size :: number + 1 + table.insert(self._array, key) + end + -- always update value + self._map[key] = value + return self +end + +function Map:get(key) + return self._map[key] +end + +function Map:clear() + local table_: any = table + self.size = 0 + table_.clear(self._map) + table_.clear(self._array) +end + +function Map:delete(key): boolean + if self._map[key] == nil then + return false + end + -- Luau FIXME: analyze should know self is Map which includes size as a number + self.size = self.size :: number - 1 + self._map[key] = nil + local index = table.find(self._array, key) + if index then + table.remove(self._array, index) + end + return true +end + +-- Implements Javascript's `Map.prototype.forEach` as defined below +-- https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Map/forEach +function Map:forEach(callback: callbackFn | callbackFnWithThisArg, thisArg: Object?): () + if typeof(callback) ~= "function" then + error("callback is not a function") + end + + return Array.forEach(self._array, function(key: K) + local value: V = self._map[key] :: V + + if thisArg ~= nil then + (callback :: callbackFnWithThisArg)(thisArg, value, key, self) + else + (callback :: callbackFn)(value, key, self) + end + end) +end + +function Map:has(key): boolean + return self._map[key] ~= nil +end + +function Map:keys() + return self._array +end + +function Map:values() + return Array.map(self._array, function(key) + return self._map[key] + end) +end + +function Map:entries() + return Array.map(self._array, function(key) + return { key, self._map[key] } + end) +end + +function Map:ipairs() + return ipairs(self:entries()) +end + +function Map.__index(self, key) + local mapProp = rawget(Map, key) + if mapProp ~= nil then + return mapProp + end + + return Map.get(self, key) +end + +function Map.__newindex(table_, key, value) + table_:set(key, value) +end + +local function coerceToMap(mapLike: Map | Table): Map + return instanceOf(mapLike, Map) and mapLike :: Map -- ROBLOX: order is preservered + or Map.new(Object.entries(mapLike)) -- ROBLOX: order is not preserved +end + +-- local function coerceToTable(mapLike: Map | Table): Table +-- if not instanceOf(mapLike, Map) then +-- return mapLike +-- end + +-- -- create table from map +-- return Array.reduce(mapLike:entries(), function(tbl, entry) +-- tbl[entry[1]] = entry[2] +-- return tbl +-- end, {}) +-- end + +-- #region Tests to verify it works as expected +local function it(description: string, fn: () -> ()) + local ok, result = pcall(fn) + + if not ok then + error("Failed test: " .. description .. "\n" .. result) + end +end + +local AN_ITEM = "bar" +local ANOTHER_ITEM = "baz" + +-- #region [Describe] "Map" +-- #region [Child Describe] "constructors" +it("creates an empty array", function() + local foo = Map.new() + assert(foo.size == 0) +end) + +it("creates a Map from an array", function() + local foo = Map.new({ + { AN_ITEM, "foo" }, + { ANOTHER_ITEM, "val" }, + }) + assert(foo.size == 2) + assert(foo:has(AN_ITEM) == true) + assert(foo:has(ANOTHER_ITEM) == true) +end) + +it("creates a Map from an array with duplicate keys", function() + local foo = Map.new({ + { AN_ITEM, "foo1" }, + { AN_ITEM, "foo2" }, + }) + assert(foo.size == 1) + assert(foo:get(AN_ITEM) == "foo2") + + assert(#foo:keys() == 1 and foo:keys()[1] == AN_ITEM) + assert(#foo:values() == 1 and foo:values()[1] == "foo2") + assert(#foo:entries() == 1) + assert(#foo:entries()[1] == 2) + + assert(foo:entries()[1][1] == AN_ITEM) + assert(foo:entries()[1][2] == "foo2") +end) + +it("preserves the order of keys first assignment", function() + local foo = Map.new({ + { AN_ITEM, "foo1" }, + { ANOTHER_ITEM, "bar" }, + { AN_ITEM, "foo2" }, + }) + assert(foo.size == 2) + assert(foo:get(AN_ITEM) == "foo2") + assert(foo:get(ANOTHER_ITEM) == "bar") + + assert(foo:keys()[1] == AN_ITEM) + assert(foo:keys()[2] == ANOTHER_ITEM) + assert(foo:values()[1] == "foo2") + assert(foo:values()[2] == "bar") + assert(foo:entries()[1][1] == AN_ITEM) + assert(foo:entries()[1][2] == "foo2") + assert(foo:entries()[2][1] == ANOTHER_ITEM) + assert(foo:entries()[2][2] == "bar") +end) +-- #endregion + +-- #region [Child Describe] "type" +it("instanceOf return true for an actual Map object", function() + local foo = Map.new() + assert(instanceOf(foo, Map) == true) +end) + +it("instanceOf return false for an regular plain object", function() + local foo = {} + assert(instanceOf(foo, Map) == false) +end) +-- #endregion + +-- #region [Child Describe] "set" +it("returns the Map object", function() + local foo = Map.new() + assert(foo:set(1, "baz") == foo) +end) + +it("increments the size if the element is added for the first time", function() + local foo = Map.new() + foo:set(AN_ITEM, "foo") + assert(foo.size == 1) +end) + +it("does not increment the size the second time an element is added", function() + local foo = Map.new() + foo:set(AN_ITEM, "foo") + foo:set(AN_ITEM, "val") + assert(foo.size == 1) +end) + +it("sets values correctly to true/false", function() + -- Luau FIXME: Luau insists that arrays can't be mixed type + local foo = Map.new({ { AN_ITEM, false :: any } }) + foo:set(AN_ITEM, false) + assert(foo.size == 1) + assert(foo:get(AN_ITEM) == false) + + foo:set(AN_ITEM, true) + assert(foo.size == 1) + assert(foo:get(AN_ITEM) == true) + + foo:set(AN_ITEM, false) + assert(foo.size == 1) + assert(foo:get(AN_ITEM) == false) +end) + +-- #endregion + +-- #region [Child Describe] "get" +it("returns value of item from provided key", function() + local foo = Map.new() + foo:set(AN_ITEM, "foo") + assert(foo:get(AN_ITEM) == "foo") +end) + +it("returns nil if the item is not in the Map", function() + local foo = Map.new() + assert(foo:get(AN_ITEM) == nil) +end) +-- #endregion + +-- #region [Child Describe] "clear" +it("sets the size to zero", function() + local foo = Map.new() + foo:set(AN_ITEM, "foo") + foo:clear() + assert(foo.size == 0) +end) + +it("removes the items from the Map", function() + local foo = Map.new() + foo:set(AN_ITEM, "foo") + foo:clear() + assert(foo:has(AN_ITEM) == false) +end) +-- #endregion + +-- #region [Child Describe] "delete" +it("removes the items from the Map", function() + local foo = Map.new() + foo:set(AN_ITEM, "foo") + foo:delete(AN_ITEM) + assert(foo:has(AN_ITEM) == false) +end) + +it("returns true if the item was in the Map", function() + local foo = Map.new() + foo:set(AN_ITEM, "foo") + assert(foo:delete(AN_ITEM) == true) +end) + +it("returns false if the item was not in the Map", function() + local foo = Map.new() + assert(foo:delete(AN_ITEM) == false) +end) + +it("decrements the size if the item was in the Map", function() + local foo = Map.new() + foo:set(AN_ITEM, "foo") + foo:delete(AN_ITEM) + assert(foo.size == 0) +end) + +it("does not decrement the size if the item was not in the Map", function() + local foo = Map.new() + foo:set(AN_ITEM, "foo") + foo:delete(ANOTHER_ITEM) + assert(foo.size == 1) +end) + +it("deletes value set to false", function() + -- Luau FIXME: Luau insists arrays can't be mixed type + local foo = Map.new({ { AN_ITEM, false :: any } }) + + foo:delete(AN_ITEM) + + assert(foo.size == 0) + assert(foo:get(AN_ITEM) == nil) +end) +-- #endregion + +-- #region [Child Describe] "has" +it("returns true if the item is in the Map", function() + local foo = Map.new() + foo:set(AN_ITEM, "foo") + assert(foo:has(AN_ITEM) == true) +end) + +it("returns false if the item is not in the Map", function() + local foo = Map.new() + assert(foo:has(AN_ITEM) == false) +end) + +it("returns correctly with value set to false", function() + -- Luau FIXME: Luau insists arrays can't be mixed type + local foo = Map.new({ { AN_ITEM, false :: any } }) + + assert(foo:has(AN_ITEM) == true) +end) +-- #endregion + +-- #region [Child Describe] "keys / values / entries" +it("returns array of elements", function() + local myMap = Map.new() + myMap:set(AN_ITEM, "foo") + myMap:set(ANOTHER_ITEM, "val") + + assert(myMap:keys()[1] == AN_ITEM) + assert(myMap:keys()[2] == ANOTHER_ITEM) + + assert(myMap:values()[1] == "foo") + assert(myMap:values()[2] == "val") + + assert(myMap:entries()[1][1] == AN_ITEM) + assert(myMap:entries()[1][2] == "foo") + assert(myMap:entries()[2][1] == ANOTHER_ITEM) + assert(myMap:entries()[2][2] == "val") +end) +-- #endregion + +-- #region [Child Describe] "__index" +it("can access fields directly without using get", function() + local typeName = "size" + + local foo = Map.new({ + { AN_ITEM, "foo" }, + { ANOTHER_ITEM, "val" }, + { typeName, "buzz" }, + }) + + assert(foo.size == 3) + assert(foo[AN_ITEM] == "foo") + assert(foo[ANOTHER_ITEM] == "val") + assert(foo:get(typeName) == "buzz") +end) +-- #endregion + +-- #region [Child Describe] "__newindex" +it("can set fields directly without using set", function() + local foo = Map.new() + + assert(foo.size == 0) + + foo[AN_ITEM] = "foo" + foo[ANOTHER_ITEM] = "val" + foo.fizz = "buzz" + + assert(foo.size == 3) + assert(foo:get(AN_ITEM) == "foo") + assert(foo:get(ANOTHER_ITEM) == "val") + assert(foo:get("fizz") == "buzz") +end) +-- #endregion + +-- #region [Child Describe] "ipairs" +local function makeArray(...) + local array = {} + for _, item in ... do + table.insert(array, item) + end + return array +end + +it("iterates on the elements by their insertion order", function() + local foo = Map.new() + foo:set(AN_ITEM, "foo") + foo:set(ANOTHER_ITEM, "val") + assert(makeArray(foo:ipairs())[1][1] == AN_ITEM) + assert(makeArray(foo:ipairs())[1][2] == "foo") + assert(makeArray(foo:ipairs())[2][1] == ANOTHER_ITEM) + assert(makeArray(foo:ipairs())[2][2] == "val") +end) + +it("does not iterate on removed elements", function() + local foo = Map.new() + foo:set(AN_ITEM, "foo") + foo:set(ANOTHER_ITEM, "val") + foo:delete(AN_ITEM) + assert(makeArray(foo:ipairs())[1][1] == ANOTHER_ITEM) + assert(makeArray(foo:ipairs())[1][2] == "val") +end) + +it("iterates on elements if the added back to the Map", function() + local foo = Map.new() + foo:set(AN_ITEM, "foo") + foo:set(ANOTHER_ITEM, "val") + foo:delete(AN_ITEM) + foo:set(AN_ITEM, "food") + assert(makeArray(foo:ipairs())[1][1] == ANOTHER_ITEM) + assert(makeArray(foo:ipairs())[1][2] == "val") + assert(makeArray(foo:ipairs())[2][1] == AN_ITEM) + assert(makeArray(foo:ipairs())[2][2] == "food") +end) +-- #endregion + +-- #region [Child Describe] "Integration Tests" +-- it("MDN Examples", function() +-- local myMap = Map.new() :: Map + +-- local keyString = "a string" +-- local keyObj = {} +-- local keyFunc = function() end + +-- -- setting the values +-- myMap:set(keyString, "value associated with 'a string'") +-- myMap:set(keyObj, "value associated with keyObj") +-- myMap:set(keyFunc, "value associated with keyFunc") + +-- assert(myMap.size == 3) + +-- -- getting the values +-- assert(myMap:get(keyString) == "value associated with 'a string'") +-- assert(myMap:get(keyObj) == "value associated with keyObj") +-- assert(myMap:get(keyFunc) == "value associated with keyFunc") + +-- assert(myMap:get("a string") == "value associated with 'a string'") + +-- assert(myMap:get({}) == nil) -- nil, because keyObj !== {} +-- assert(myMap:get(function() -- nil because keyFunc !== function () {} +-- end) == nil) +-- end) + +it("handles non-traditional keys", function() + local myMap = Map.new() :: Map + + local falseKey = false + local trueKey = true + local negativeKey = -1 + local emptyKey = "" + + myMap:set(falseKey, "apple") + myMap:set(trueKey, "bear") + myMap:set(negativeKey, "corgi") + myMap:set(emptyKey, "doge") + + assert(myMap.size == 4) + + assert(myMap:get(falseKey) == "apple") + assert(myMap:get(trueKey) == "bear") + assert(myMap:get(negativeKey) == "corgi") + assert(myMap:get(emptyKey) == "doge") + + myMap:delete(falseKey) + myMap:delete(trueKey) + myMap:delete(negativeKey) + myMap:delete(emptyKey) + + assert(myMap.size == 0) +end) +-- #endregion + +-- #endregion [Describe] "Map" + +-- #region [Describe] "coerceToMap" +it("returns the same object if instance of Map", function() + local map = Map.new() + assert(coerceToMap(map) == map) + + map = Map.new({}) + assert(coerceToMap(map) == map) + + map = Map.new({ { AN_ITEM, "foo" } }) + assert(coerceToMap(map) == map) +end) +-- #endregion [Describe] "coerceToMap" + +-- #endregion Tests to verify it works as expected diff --git a/bench/other/regex.lua b/bench/other/regex.lua new file mode 100644 index 00000000..eb659a5e --- /dev/null +++ b/bench/other/regex.lua @@ -0,0 +1,2089 @@ +--[[ + PCRE2-based RegEx implemention for Luau + Version 1.0.0a2 + BSD 2-Clause Licence + Copyright © 2020 - Blockzez (devforum /u/Blockzez and github.com/Blockzez) + All rights reserved. + + Redistribution and use in source and binary forms, with or without + modification, are permitted provided that the following conditions are met: + + 1. Redistributions of source code must retain the above copyright notice, this + list of conditions and the following disclaimer. + + 2. Redistributions in binary form must reproduce the above copyright notice, + this list of conditions and the following disclaimer in the documentation + and/or other materials provided with the distribution. + + THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" + AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE + DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE + FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR + SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER + CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, + OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. +]] +--[[ Settings ]]-- +-- You can change them here +local options = { + -- The maximum cache size for regex so the patterns are cached so it doesn't recompile the pattern + -- The only accepted value are number values >= 0, strings that can be automatically coered to numbers that are >= 0, false and nil + -- Do note that empty regex patterns (comment-only patterns included) are never cached regardless + -- The default is 256 + cacheSize = 256, + + -- A boolean that determines whether this use unicode data + -- If this value evalulates to false, you can remove _unicodechar_category, _scripts and _xuc safely and it'll now error if: + -- - You try to compile a RegEx with unicode flag + -- - You try to use the \p pattern + -- The default is true + unicodeData = false, +}; + +-- +local u_categories = options.unicodeData and require(script:WaitForChild("_unicodechar_category")); +local chr_scripts = options.unicodeData and require(script:WaitForChild("_scripts")); +local xuc_chr = options.unicodeData and require(script:WaitForChild("_xuc")); +local proxy = setmetatable({ }, { __mode = 'k' }); +local re, re_m, match_m = { }, { }, { }; +local lockmsg; + +--[[ Functions ]]-- +local function to_str_arr(self, init) + if init then + self = string.sub(self, utf8.offset(self, init)); + end; + local len = utf8.len(self); + if len <= 1999 then + return { n = len, s = self, utf8.codepoint(self, 1, #self) }; + end; + local clen = math.ceil(len / 1999); + local ret = table.create(len); + local p = 1; + for i = 1, clen do + local c = table.pack(utf8.codepoint(self, utf8.offset(self, i * 1999 - 1998), utf8.offset(self, i * 1999 - (i == clen and 1998 - ((len - 1) % 1999 + 1) or - 1)) - 1)); + table.move(c, 1, c.n, p, ret); + p += c.n; + end; + ret.s, ret.n = self, len; + return ret; +end; + +local function from_str_arr(self) + local len = self.n or #self; + if len <= 7997 then + return utf8.char(table.unpack(self)); + end; + local clen = math.ceil(len / 7997); + local r = table.create(clen); + for i = 1, clen do + r[i] = utf8.char(table.unpack(self, i * 7997 - 7996, i * 7997 - (i == clen and 7997 - ((len - 1) % 7997 + 1) or 0))); + end; + return table.concat(r); +end; + +local function utf8_sub(self, i, j) + j = utf8.offset(self, j); + return string.sub(self, utf8.offset(self, i), j and j - 1); +end; + +-- +local flag_map = { + a = 'anchored', i = 'caseless', m = 'multiline', s = 'dotall', u = 'unicode', U = 'ungreedy', x ='extended', +}; + +local posix_class_names = { + alnum = true, alpha = true, ascii = true, blank = true, cntrl = true, digit = true, graph = true, lower = true, print = true, punct = true, space = true, upper = true, word = true, xdigit = true, +}; + +local escape_chars = { + -- grouped + -- digit, spaces and words + [0x44] = { "class", "digit", true }, [0x53] = { "class", "space", true }, [0x57] = { "class", "word", true }, + [0x64] = { "class", "digit", false }, [0x73] = { "class", "space", false }, [0x77] = { "class", "word", false }, + -- horizontal/vertical whitespace and newline + [0x48] = { "class", "blank", true }, [0x56] = { "class", "vertical_tab", true }, + [0x68] = { "class", "blank", false }, [0x76] = { "class", "vertical_tab", false }, + [0x4E] = { 0x4E }, [0x52] = { 0x52 }, + + -- not grouped + [0x42] = 0x08, + [0x6E] = 0x0A, [0x72] = 0x0D, [0x74] = 0x09, +}; + +local b_escape_chars = { + -- word boundary and not word boundary + [0x62] = { 0x62, { "class", "word", false } }, [0x42] = { 0x42, { "class", "word", false } }, + + -- keep match out + [0x4B] = { 0x4B }, + + -- start & end of string + [0x47] = { 0x47 }, [0x4A] = { 0x4A }, [0x5A] = { 0x5A }, [0x7A] = { 0x7A }, +}; + +local valid_categories = { + C = true, Cc = true, Cf = true, Cn = true, Co = true, Cs = true, + L = true, Ll = true, Lm = true, Lo = true, Lt = true, Lu = true, + M = true, Mc = true, Me = true, Mn = true, + N = true, Nd = true, Nl = true, No = true, + P = true, Pc = true, Pd = true, Pe = true, Pf = true, Pi = true, Po = true, Ps = true, + S = true, Sc = true, Sk = true, Sm = true, So = true, + Z = true, Zl = true, Zp = true, Zs = true, + + Xan = true, Xps = true, Xsp = true, Xuc = true, Xwd = true, +}; + +local class_ascii_punct = { + [0x21] = true, [0x22] = true, [0x23] = true, [0x24] = true, [0x25] = true, [0x26] = true, [0x27] = true, [0x28] = true, [0x29] = true, [0x2A] = true, [0x2B] = true, [0x2C] = true, [0x2D] = true, [0x2E] = true, [0x2F] = true, + [0x3A] = true, [0x3B] = true, [0x3C] = true, [0x3D] = true, [0x3E] = true, [0x3F] = true, [0x40] = true, [0x5B] = true, [0x5C] = true, [0x5D] = true, [0x5E] = true, [0x5F] = true, [0x60] = true, [0x7B] = true, [0x7C] = true, + [0x7D] = true, [0x7E] = true, +}; + +local end_str = { 0x24 }; +local dot = { 0x2E }; +local beginning_str = { 0x5E }; +local alternation = { 0x7C }; + +local function check_re(re_type, name, func) + if re_type == "Match" then + return function(...) + local arg_n = select('#', ...); + if arg_n < 1 then + error("missing argument #1 (Match expected)", 2); + end; + local arg0, arg1 = ...; + if not (proxy[arg0] and proxy[arg0].name == "Match") then + error(string.format("invalid argument #1 to %q (Match expected, got %s)", name, typeof(arg0)), 2); + else + arg0 = proxy[arg0]; + end; + if name == "group" or name == "span" then + if arg1 == nil then + arg1 = 0; + end; + end; + return func(arg0, arg1); + end; + end; + return function(...) + local arg_n = select('#', ...); + if arg_n < 1 then + error("missing argument #1 (RegEx expected)", 2); + elseif arg_n < 2 then + error("missing argument #2 (string expected)", 2); + end; + local arg0, arg1, arg2, arg3, arg4, arg5 = ...; + if not (proxy[arg0] and proxy[arg0].name == "RegEx") then + if type(arg0) ~= "string" and type(arg0) ~= "number" then + error(string.format("invalid argument #1 to %q (RegEx expected, got %s)", name, typeof(arg0)), 2); + end; + arg0 = re.fromstring(arg0); + elseif name == "sub" then + if type(arg2) == "number" then + arg2 ..= ''; + elseif type(arg2) ~= "string" then + error(string.format("invalid argument #3 to 'sub' (string expected, got %s)", typeof(arg2)), 2); + end; + elseif type(arg1) == "number" then + arg1 ..= ''; + elseif type(arg1) ~= "string" then + error(string.format("invalid argument #2 to %q (string expected, got %s)", name, typeof(arg1)), 2); + end; + if name ~= "sub" and name ~= "split" then + local init_type = typeof(arg2); + if init_type ~= 'nil' then + arg2 = tonumber(arg2); + if not arg2 then + error(string.format("invalid argument #3 to %q (number expected, got %s)", name, init_type), 2); + elseif arg2 < 0 then + arg2 = #arg1 + math.floor(arg2 + 0.5) + 1; + else + arg2 = math.max(math.floor(arg2 + 0.5), 1); + end; + end; + end; + arg0 = proxy[arg0]; + if name == "match" or name == "matchiter" then + arg3 = ...; + elseif name == "sub" then + arg5 = ...; + end; + return func(arg0, arg1, arg2, arg3, arg4, arg5); + end; +end; + +--[[ Matches ]]-- +local function match_tostr(self) + local spans = proxy[self].spans; + local s_start, s_end = spans[0][1], spans[0][2]; + if s_end <= s_start then + return string.format("Match (%d..%d, empty)", s_start, s_end - 1); + end; + return string.format("Match (%d..%d): %s", s_start, s_end - 1, utf8_sub(spans.input, s_start, s_end)); +end; + +local function new_match(span_arr, group_id, re, str) + span_arr.source, span_arr.input = re, str; + local object = newproxy(true); + local object_mt = getmetatable(object); + object_mt.__metatable = lockmsg; + object_mt.__index = setmetatable(span_arr, match_m); + object_mt.__tostring = match_tostr; + + proxy[object] = { name = "Match", spans = span_arr, group_id = group_id }; + return object; +end; + +match_m.group = check_re('Match', 'group', function(self, group_id) + local span = self.spans[type(group_id) == "number" and group_id or self.group_id[group_id]]; + if not span then + return nil; + end; + return utf8_sub(self.spans.input, span[1], span[2]); +end); + +match_m.span = check_re('Match', 'span', function(self, group_id) + local span = self.spans[type(group_id) == "number" and group_id or self.group_id[group_id]]; + if not span then + return nil; + end; + return span[1], span[2] - 1; +end); + +match_m.groups = check_re('Match', 'groups', function(self) + local spans = self.spans; + if spans.n > 0 then + local ret = table.create(spans.n); + for i = 0, spans.n do + local v = spans[i]; + if v then + ret[i] = utf8_sub(spans.input, v[1], v[2]); + end; + end; + return table.unpack(ret, 1, spans.n); + end; + return utf8_sub(spans.input, spans[0][1], spans[0][2]); +end); + +match_m.groupdict = check_re('Match', 'groupdict', function(self) + local spans = self.spans; + local ret = { }; + for k, v in pairs(self.group_id) do + v = spans[v]; + if v then + ret[k] = utf8_sub(spans.input, v[1], v[2]); + end; + end; + return ret; +end); + +match_m.grouparr = check_re('Match', 'groupdict', function(self) + local spans = self.spans; + local ret = table.create(spans.n); + for i = 0, spans.n do + local v = spans[i]; + if v then + ret[i] = utf8_sub(spans.input, v[1], v[2]); + end; + end; + ret.n = spans.n; + return ret; +end); + +-- +local line_verbs = { + CR = 0, LF = 1, CRLF = 2, ANYRLF = 3, ANY = 4, NUL = 5, +}; +local function is_newline(str_arr, i, verb_flags) + local line_verb_n = verb_flags.newline; + local chr = str_arr[i]; + if line_verb_n == 0 then + -- carriage return + return chr == 0x0D; + elseif line_verb_n == 2 then + -- carriage return followed by line feed + return chr == 0x0A and str_arr[i - 1] == 0x20; + elseif line_verb_n == 3 then + -- any of the above + return chr == 0x0A or chr == 0x0D; + elseif line_verb_n == 4 then + -- any of Unicode newlines + return chr == 0x0A or chr == 0x0B or chr == 0x0C or chr == 0x0D or chr == 0x85 or chr == 0x2028 or chr == 0x2029; + elseif line_verb_n == 5 then + -- null + return chr == 0; + end; + -- linefeed + return chr == 0x0A; +end; + + +local function tkn_char_match(tkn_part, str_arr, i, flags, verb_flags) + local chr = str_arr[i]; + if not chr then + return false; + elseif flags.ignoreCase and chr >= 0x61 and chr <= 0x7A then + chr -= 0x20; + end; + if type(tkn_part) == "number" then + return tkn_part == chr; + elseif tkn_part[1] == "charset" then + for _, v in ipairs(tkn_part[3]) do + if tkn_char_match(v, str_arr, i, flags, verb_flags) then + return not tkn_part[2]; + end; + end; + return tkn_part[2]; + elseif tkn_part[1] == "range" then + return chr >= tkn_part[2] and chr <= tkn_part[3] or flags.ignoreCase and chr >= 0x41 and chr <= 0x5A and (chr + 0x20) >= tkn_part[2] and (chr + 0x20) <= tkn_part[3]; + elseif tkn_part[1] == "class" then + local char_class = tkn_part[2]; + local negate = tkn_part[3]; + local match = false; + -- if and elseifs :( + -- Might make these into tables in the future + if char_class == "xdigit" then + match = chr >= 0x30 and chr <= 0x39 or chr >= 0x41 and chr <= 0x46 or chr >= 0x61 and chr <= 0x66; + elseif char_class == "ascii" then + match = chr <= 0x7F; + -- cannot be accessed through POSIX classes + elseif char_class == "vertical_tab" then + match = chr >= 0x0A and chr <= 0x0D or chr == 0x2028 or chr == 0x2029; + -- + elseif flags.unicode then + local current_category = u_categories[chr] or 'Cn'; + local first_category = current_category:sub(1, 1); + if char_class == "alnum" then + match = first_category == 'L' or current_category == 'Nl' or current_category == 'Nd'; + elseif char_class == "alpha" then + match = first_category == 'L' or current_category == 'Nl'; + elseif char_class == "blank" then + match = current_category == 'Zs' or chr == 0x09; + elseif char_class == "cntrl" then + match = current_category == 'Cc'; + elseif char_class == "digit" then + match = current_category == 'Nd'; + elseif char_class == "graph" then + match = first_category ~= 'P' and first_category ~= 'C'; + elseif char_class == "lower" then + match = current_category == 'Ll'; + elseif char_class == "print" then + match = first_category ~= 'C'; + elseif char_class == "punct" then + match = first_category == 'P'; + elseif char_class == "space" then + match = first_category == 'Z' or chr >= 0x09 and chr <= 0x0D; + elseif char_class == "upper" then + match = current_category == 'Lu'; + elseif char_class == "word" then + match = first_category == 'L' or current_category == 'Nl' or current_category == 'Nd' or current_category == 'Pc'; + end; + elseif char_class == "alnum" then + match = chr >= 0x30 and chr <= 0x39 or chr >= 0x41 and chr <= 0x5A or chr >= 0x61 and chr <= 0x7A; + elseif char_class == "alpha" then + match = chr >= 0x41 and chr <= 0x5A or chr >= 0x61 and chr <= 0x7A; + elseif char_class == "blank" then + match = chr == 0x09 or chr == 0x20; + elseif char_class == "cntrl" then + match = chr <= 0x1F or chr == 0x7F; + elseif char_class == "digit" then + match = chr >= 0x30 and chr <= 0x39; + elseif char_class == "graph" then + match = chr >= 0x21 and chr <= 0x7E; + elseif char_class == "lower" then + match = chr >= 0x61 and chr <= 0x7A; + elseif char_class == "print" then + match = chr >= 0x20 and chr <= 0x7E; + elseif char_class == "punct" then + match = class_ascii_punct[chr]; + elseif char_class == "space" then + match = chr >= 0x09 and chr <= 0x0D or chr == 0x20; + elseif char_class == "upper" then + match = chr >= 0x41 and chr <= 0x5A; + elseif char_class == "word" then + match = chr >= 0x30 and chr <= 0x39 or chr >= 0x41 and chr <= 0x5A or chr >= 0x61 and chr <= 0x7A or chr == 0x5F; + end; + if negate then + return not match; + end; + return match; + elseif tkn_part[1] == "category" then + local chr_category = u_categories[chr] or 'Cn'; + local category_v = tkn_part[3]; + local category_len = #category_v; + if category_len == 3 then + local match = false; + if category_v == "Xan" or category_v == "Xwd" then + match = chr_category:find("^[LN]") or category_v == "Xwd" and chr == 0x5F; + elseif category_v == "Xps" or category_v == "Xsp" then + match = chr_category:sub(1, 1) == 'Z' or chr >= 0x09 and chr <= 0x0D; + elseif category_v == "Xuc" then + match = tkn_char_match(xuc_chr, str_arr, i, flags, verb_flags); + end; + if tkn_part[2] then + return not match; + end + return match; + elseif chr_category:sub(1, category_len) == category_v then + return not tkn_part[2]; + end; + return tkn_part[2]; + elseif tkn_part[1] == 0x2E then + return flags.dotAll or not is_newline(str_arr, i, verb_flags); + elseif tkn_part[1] == 0x4E then + return not is_newline(str_arr, i, verb_flags); + elseif tkn_part[1] == 0x52 then + if verb_flags.newline_seq == 0 then + -- CR, LF or CRLF + return chr == 0x0A or chr == 0x0D; + end; + -- any unicode newline + return chr == 0x0A or chr == 0x0B or chr == 0x0C or chr == 0x0D or chr == 0x85 or chr == 0x2028 or chr == 0x2029; + end; + return false; +end; + +local function find_alternation(token, i, count) + while true do + local v = token[i]; + local is_table = type(v) == "table"; + if v == alternation then + return i, count; + elseif is_table and v[1] == 0x28 then + if count then + count += v.count; + end; + i = v[3]; + elseif is_table and v[1] == "quantifier" and type(v[5]) == "table" and v[5][1] == 0x28 then + if count then + count += v[5].count; + end; + i = v[5][3]; + elseif not v or is_table and v[1] == 0x29 then + return nil, count; + elseif count then + if is_table and v[1] == "quantifier" then + count += v[3]; + else + count += 1; + end; + end; + i += 1; + end; +end; + +local function re_rawfind(token, str_arr, init, flags, verb_flags, as_bool) + local tkn_i, str_i, start_i = 0, init, init; + local states = { }; + while tkn_i do + if tkn_i == 0 then + tkn_i += 1; + local next_alt = find_alternation(token, tkn_i); + if next_alt then + table.insert(states, 1, { "alternation", next_alt, str_i }); + end; + continue; + end; + local ctkn = token[tkn_i]; + local tkn_type = type(ctkn) == "table" and ctkn[1]; + if not ctkn then + break; + elseif ctkn == "ACCEPT" then + local not_lookaround = true; + local close_i = tkn_i; + repeat + close_i += 1; + local is_table = type(token[close_i]) == "table"; + local close_i_tkn = token[close_i]; + if is_table and (close_i_tkn[1] == 0x28 or close_i_tkn[1] == "quantifier" and type(close_i_tkn[5]) == "table" and close_i_tkn[5][1] == 0x28) then + close_i = close_i_tkn[1] == "quantifier" and close_i_tkn[5][3] or close_i_tkn[3]; + elseif is_table and close_i_tkn[1] == 0x29 and (close_i_tkn[4] == 0x21 or close_i_tkn[4] == 0x3D) then + not_lookaround = false; + tkn_i = close_i; + break; + end; + until not close_i_tkn; + if not_lookaround then + break; + end; + elseif ctkn == "PRUNE" or ctkn == "SKIP" then + table.insert(states, 1, { ctkn, str_i }); + tkn_i += 1; + elseif tkn_type == 0x28 then + table.insert(states, 1, { "group", tkn_i, str_i, nil, ctkn[2], ctkn[3], ctkn[4] }); + tkn_i += 1; + local next_alt, count = find_alternation(token, tkn_i, (ctkn[4] == 0x21 or ctkn[4] == 0x3D) and ctkn[5] and 0); + if next_alt then + table.insert(states, 1, { "alternation", next_alt, str_i }); + end; + if count then + str_i -= count; + end; + elseif tkn_type == 0x29 and ctkn[4] ~= 0x21 then + if ctkn[4] == 0x21 or ctkn[4] == 0x3D then + while true do + local selected_match_start; + local selected_state = table.remove(states, 1); + if selected_state[1] == "group" and selected_state[2] == ctkn[3] then + if (ctkn[4] == 0x21 or ctkn[4] == 0x3D) and not ctkn[5] then + str_i = selected_state[3]; + end; + if selected_match_start then + table.insert(states, 1, selected_match_start); + end; + break; + elseif selected_state[1] == "matchStart" and not selected_match_start and ctkn[4] == 0x3D then + selected_match_start = selected_state; + end; + end; + elseif ctkn[4] == 0x3E then + repeat + local selected_state = table.remove(states, 1); + until not selected_state or selected_state[1] == "group" and selected_state[2] == ctkn[3]; + else + for i, v in ipairs(states) do + if v[1] == "group" and v[2] == ctkn[3] then + if v.jmp then + -- recursive match + tkn_i = v.jmp; + end; + v[4] = str_i; + if v[7] == "quantifier" and v[10] + 1 < v[9] then + if token[ctkn[3]][4] ~= "lazy" or v[10] + 1 < v[8] then + tkn_i = ctkn[3]; + end; + local ctkn1 = token[ctkn[3]]; + local new_group = { "group", v[2], str_i, nil, ctkn1[5][2], ctkn1[5][3], "quantifier", ctkn1[2], ctkn1[3], v[10] + 1, v[11], ctkn1[4] }; + table.insert(states, 1, new_group); + if v[11] then + table.insert(states, 1, { "alternation", v[11], str_i }); + end; + end; + break; + end; + end; + end; + tkn_i += 1; + elseif tkn_type == 0x4B then + table.insert(states, 1, { "matchStart", str_i }); + tkn_i += 1; + elseif tkn_type == 0x7C then + local close_i = tkn_i; + repeat + close_i += 1; + local is_table = type(token[close_i]) == "table"; + local close_i_tkn = token[close_i]; + if is_table and (close_i_tkn[1] == 0x28 or close_i_tkn[1] == "quantifier" and type(close_i_tkn[5]) == "table" and close_i_tkn[5][1] == 0x28) then + close_i = close_i_tkn[1] == "quantifier" and close_i_tkn[5][3] or close_i_tkn[3]; + end; + until is_table and close_i_tkn[1] == 0x29 or not close_i_tkn; + if token[close_i] then + for _, v in ipairs(states) do + if v[1] == "group" and v[6] == close_i then + tkn_i = v[6]; + break; + end; + end; + else + tkn_i = close_i; + end; + elseif tkn_type == "recurmatch" then + table.insert(states, 1, { "group", ctkn[3], str_i, nil, nil, token[ctkn[3]][3], nil, jmp = tkn_i }); + tkn_i = ctkn[3] + 1; + local next_alt, count = find_alternation(token, tkn_i); + if next_alt then + table.insert(states, 1, { "alternation", next_alt, str_i }); + end; + else + local match; + if ctkn == "FAIL" then + match = false; + elseif tkn_type == 0x29 then + repeat + local selected_state = table.remove(states, 1); + until selected_state[1] == "group" and selected_state[2] == ctkn[3]; + elseif tkn_type == "quantifier" then + if type(ctkn[5]) == "table" and ctkn[5][1] == 0x28 then + local next_alt = find_alternation(token, tkn_i + 1); + if next_alt then + table.insert(states, 1, { "alternation", next_alt, str_i }); + end; + table.insert(states, next_alt and 2 or 1, { "group", tkn_i, str_i, nil, ctkn[5][2], ctkn[5][3], "quantifier", ctkn[2], ctkn[3], 0, next_alt, ctkn[4] }); + if ctkn[4] == "lazy" and ctkn[2] == 0 then + tkn_i = ctkn[5][3]; + end; + match = true; + else + local start_i, end_i; + local pattern_count = 1; + local is_backref = type(ctkn[5]) == "table" and ctkn[5][1] == "backref"; + if is_backref then + pattern_count = 0; + local group_n = ctkn[5][2]; + for _, v in ipairs(states) do + if v[1] == "group" and v[5] == group_n then + start_i, end_i = v[3], v[4]; + pattern_count = end_i - start_i; + break; + end; + end; + end; + local min_max_i = str_i + ctkn[2] * pattern_count; + local mcount = 0; + while mcount < ctkn[3] do + if is_backref then + if start_i and end_i then + local org_i = str_i; + if utf8_sub(str_arr.s, start_i, end_i) ~= utf8_sub(str_arr.s, org_i, str_i + pattern_count) then + break; + end; + else + break; + end; + elseif not tkn_char_match(ctkn[5], str_arr, str_i, flags, verb_flags) then + break; + end; + str_i += pattern_count; + mcount += 1; + end; + match = mcount >= ctkn[2]; + if match and ctkn[4] ~= "possessive" then + if ctkn[4] == "lazy" then + min_max_i, str_i = str_i, min_max_i; + end; + table.insert(states, 1, { "quantifier", tkn_i, str_i, math.min(min_max_i, str_arr.n + 1), (ctkn[4] == "lazy" and 1 or -1) * pattern_count }); + end; + end; + elseif tkn_type == "backref" then + local start_i, end_i; + local group_n = ctkn[2]; + for _, v in ipairs(states) do + if v[1] == "group" and v[5] == group_n then + start_i, end_i = v[3], v[4]; + break; + end; + end; + if start_i and end_i then + local org_i = str_i; + str_i += end_i - start_i; + match = utf8_sub(str_arr.s, start_i, end_i) == utf8_sub(str_arr.s, org_i, str_i); + end; + else + local chr = str_arr[str_i]; + if tkn_type == 0x24 or tkn_type == 0x5A or tkn_type == 0x7A then + match = str_i == str_arr.n + 1 or tkn_type == 0x24 and flags.multiline and is_newline(str_arr, str_i + 1, verb_flags) or tkn_type == 0x5A and str_i == str_arr.n and is_newline(str_arr, str_i, verb_flags); + elseif tkn_type == 0x5E or tkn_type == 0x41 or tkn_type == 0x47 then + match = str_i == 1 or tkn_type == 0x5E and flags.multiline and is_newline(str_arr, str_i - 1, verb_flags) or tkn_type == 0x47 and str_i == init; + elseif tkn_type == 0x42 or tkn_type == 0x62 then + local start_m = str_i == 1 or flags.multiline and is_newline(str_arr, str_i - 1, verb_flags); + local end_m = str_i == str_arr.n + 1 or flags.multiline and is_newline(str_arr, str_i, verb_flags); + local w_m = tkn_char_match(ctkn[2], str_arr[str_i - 1], flags) and 0 or tkn_char_match(ctkn[2], chr, flags) and 1; + if w_m == 0 then + match = end_m or not tkn_char_match(ctkn[2], chr, flags); + elseif w_m then + match = start_m or not tkn_char_match(ctkn[2], str_arr[str_i - 1], flags); + end; + if tkn_type == 0x42 then + match = not match; + end; + else + match = tkn_char_match(ctkn, str_arr, str_i, flags, verb_flags); + str_i += 1; + end; + end; + if not match then + while true do + local prev_type, prev_state = states[1] and states[1][1], states[1]; + if not prev_type or prev_type == "PRUNE" or prev_type == "SKIP" then + if prev_type then + table.clear(states); + end; + if start_i > str_arr.n then + if as_bool then + return false; + end; + return nil; + end; + start_i = prev_type == "SKIP" and prev_state[2] or start_i + 1; + tkn_i, str_i = 0, start_i; + break; + elseif prev_type == "alternation" then + tkn_i, str_i = prev_state[2], prev_state[3]; + local next_alt, count = find_alternation(token, tkn_i + 1); + if next_alt then + prev_state[2] = next_alt; + else + table.remove(states, 1); + end; + if count then + str_i -= count; + end; + break; + elseif prev_type == "group" then + if prev_state[7] == "quantifier" then + if prev_state[12] == "greedy" and prev_state[10] >= prev_state[8] + or prev_state[12] == "lazy" and prev_state[10] < prev_state[9] and not prev_state[13] then + tkn_i, str_i = prev_state[12] == "greedy" and prev_state[6] or prev_state[2], prev_state[3]; + if prev_state[12] == "greedy" then + table.remove(states, 1); + break; + elseif prev_state[10] >= prev_state[8] then + prev_state[13] = true; + break; + end; + end; + elseif prev_state[7] == 0x21 then + table.remove(states, 1); + tkn_i, str_i = prev_state[6], prev_state[3]; + break; + end; + elseif prev_type == "quantifier" then + if math.sign(prev_state[4] - prev_state[3]) == math.sign(prev_state[5]) then + prev_state[3] += prev_state[5]; + tkn_i, str_i = prev_state[2], prev_state[3]; + break; + end; + end; + -- keep match out state and recursive state, can be safely removed + -- prevents infinite loop + table.remove(states, 1); + end; + end; + tkn_i += 1; + end; + end; + if as_bool then + return true; + end; + local match_start_ran = false; + local span = table.create(token.group_n); + span[0], span.n = { start_i, str_i }, token.group_n; + for _, v in ipairs(states) do + if v[1] == "matchStart" and not match_start_ran then + span[0][1], match_start_ran = v[2], true; + elseif v[1] == "group" and v[5] and not span[v[5]] then + span[v[5]] = { v[3], v[4] }; + end; + end; + return span; +end; + +--[[ Methods ]]-- +re_m.test = check_re('RegEx', 'test', function(self, str, init) + return re_rawfind(self.token, to_str_arr(str, init), 1, self.flags, self.verb_flags, true); +end); + +re_m.match = check_re('RegEx', 'match', function(self, str, init, source) + local span = re_rawfind(self.token, to_str_arr(str, init), 1, self.flags, self.verb_flags, false); + if not span then + return nil; + end; + return new_match(span, self.group_id, source, str); +end); + +re_m.matchall = check_re('RegEx', 'matchall', function(self, str, init, source) + str = to_str_arr(str, init); + local i = 1; + return function() + local span = i <= str.n + 1 and re_rawfind(self.token, str, i, self.flags, self.verb_flags, false); + if not span then + return nil; + end; + i = span[0][2] + (span[0][1] >= span[0][2] and 1 or 0); + return new_match(span, self.group_id, source, str.s); + end; +end); + +local function insert_tokenized_sub(repl_r, str, span, tkn) + for _, v in ipairs(tkn) do + if type(v) == "table" then + if v[1] == "condition" then + if span[v[2]] then + if v[3] then + insert_tokenized_sub(repl_r, str, span, v[3]); + else + table.move(str, span[v[2]][1], span[v[2]][2] - 1, #repl_r + 1, repl_r); + end; + elseif v[4] then + insert_tokenized_sub(repl_r, str, span, v[4]); + end; + else + table.move(v, 1, #v, #repl_r + 1, repl_r); + end; + elseif span[v] then + table.move(str, span[v][1], span[v][2] - 1, #repl_r + 1, repl_r); + end; + end; + repl_r.n = #repl_r; + return repl_r; +end; + +re_m.sub = check_re('RegEx', 'sub', function(self, repl, str, n, repl_flag_str, source) + if repl_flag_str ~= nil and type(repl_flag_str) ~= "number" and type(repl_flag_str) ~= "string" then + error(string.format("invalid argument #5 to 'sub' (string expected, got %s)", typeof(repl_flag_str)), 3); + end + local repl_flags = { + l = false, o = false, u = false, + }; + for f in string.gmatch(repl_flag_str or '', utf8.charpattern) do + if repl_flags[f] ~= false then + error("invalid regular expression substitution flag " .. f, 3); + end; + repl_flags[f] = true; + end; + local repl_type = type(repl); + if repl_type == "number" then + repl ..= ''; + elseif repl_type ~= "string" and repl_type ~= "function" and (not repl_flags.o or repl_type ~= "table") then + error(string.format("invalid argument #2 to 'sub' (string/function%s expected, got %s)", repl_flags.o and "/table" or '', typeof(repl)), 3); + end; + if tonumber(n) then + n = tonumber(n); + if n <= -1 or n ~= n then + n = math.huge; + end; + elseif n ~= nil then + error(string.format("invalid argument #4 to 'sub' (number expected, got %s)", typeof(n)), 3); + else + n = math.huge; + end; + if n < 1 then + return str, 0; + end; + local min_repl_n = 0; + if repl_type == "string" then + repl = to_str_arr(repl); + if not repl_flags.l then + local i1 = 0; + local repl_r = table.create(3); + local group_n = self.token.group_n; + local conditional_c = { }; + while i1 < repl.n do + local i2 = i1; + repeat + i2 += 1; + until not repl[i2] or repl[i2] == 0x24 or repl[i2] == 0x5C or (repl[i2] == 0x3A or repl[i2] == 0x7D) and conditional_c[1]; + min_repl_n += i2 - i1 - 1; + if i2 - i1 > 1 then + table.insert(repl_r, table.move(repl, i1 + 1, i2 - 1, 1, table.create(i2 - i1 - 1))); + end; + if repl[i2] == 0x3A then + local current_conditional_c = conditional_c[1]; + if current_conditional_c[2] then + error("malformed substitution pattern", 3); + end; + current_conditional_c[2] = table.move(repl_r, current_conditional_c[3], #repl_r, 1, table.create(#repl_r + 1 - current_conditional_c[3])); + for i3 = #repl_r, current_conditional_c[3], -1 do + repl_r[i3] = nil; + end; + elseif repl[i2] == 0x7D then + local current_conditional_c = table.remove(conditional_c, 1); + local second_c = table.move(repl_r, current_conditional_c[3], #repl_r, 1, table.create(#repl_r + 1 - current_conditional_c[3])); + for i3 = #repl_r, current_conditional_c[3], -1 do + repl_r[i3] = nil; + end; + table.insert(repl_r, { "condition", current_conditional_c[1], current_conditional_c[2] ~= true and (current_conditional_c[2] or second_c), current_conditional_c[2] and second_c }); + elseif repl[i2] then + i2 += 1; + local subst_c = repl[i2]; + if not subst_c then + if repl[i2 - 1] == 0x5C then + error("replacement string must not end with a trailing backslash", 3); + end; + local prev_repl_f = repl_r[#repl_r]; + if type(prev_repl_f) == "table" then + table.insert(prev_repl_f, repl[i2 - 1]); + else + table.insert(repl_r, { repl[i2 - 1] }); + end; + elseif subst_c == 0x5C and repl[i2 - 1] == 0x24 then + local prev_repl_f = repl_r[#repl_r]; + if type(prev_repl_f) == "table" then + table.insert(prev_repl_f, 0x24); + else + table.insert(repl_r, { 0x24 }); + end; + i2 -= 1; + min_repl_n += 1; + elseif subst_c == 0x30 then + table.insert(repl_r, 0); + elseif subst_c > 0x30 and subst_c <= 0x39 then + local start_i2 = i2; + local group_i = subst_c - 0x30; + while repl[i2 + 1] and repl[i2 + 1] >= 0x30 and repl[i2 + 1] <= 0x39 do + group_i ..= repl[i2 + 1] - 0x30; + i2 += 1; + end; + group_i = tonumber(group_i); + if not repl_flags.u and group_i > group_n then + error("reference to non-existent subpattern", 3); + end; + table.insert(repl_r, group_i); + elseif subst_c == 0x7B and repl[i2 - 1] == 0x24 then + i2 += 1; + local start_i2 = i2; + while repl[i2] and + (repl[i2] >= 0x30 and repl[i2] <= 0x39 + or repl[i2] >= 0x41 and repl[i2] <= 0x5A + or repl[i2] >= 0x61 and repl[i2] <= 0x7A + or repl[i2] == 0x5F) do + i2 += 1; + end; + if (repl[i2] == 0x7D or repl[i2] == 0x3A and (repl[i2 + 1] == 0x2B or repl[i2 + 1] == 0x2D)) and i2 ~= start_i2 then + local group_k = utf8_sub(repl.s, start_i2, i2); + if repl[start_i2] >= 0x30 and repl[start_i2] <= 0x39 then + group_k = tonumber(group_k); + if not repl_flags.u and group_k > group_n then + error("reference to non-existent subpattern", 3); + end; + else + group_k = self.group_id[group_k]; + if not repl_flags.u and (not group_k or group_k > group_n) then + error("reference to non-existent subpattern", 3); + end; + end; + if repl[i2] == 0x3A then + i2 += 1; + table.insert(conditional_c, { group_k, repl[i2] == 0x2D, #repl_r + 1 }); + else + table.insert(repl_r, group_k); + end; + else + error("malformed substitution pattern", 3); + end; + else + local c_escape_char; + if repl[i2 - 1] == 0x24 then + if subst_c ~= 0x24 then + local prev_repl_f = repl_r[#repl_r]; + if type(prev_repl_f) == "table" then + table.insert(prev_repl_f, 0x24); + else + table.insert(repl_r, { 0x24 }); + end; + end; + else + c_escape_char = escape_chars[repl[i2]]; + if type(c_escape_char) ~= "number" then + c_escape_char = nil; + end; + end; + local prev_repl_f = repl_r[#repl_r]; + if type(prev_repl_f) == "table" then + table.insert(prev_repl_f, c_escape_char or repl[i2]); + else + table.insert(repl_r, { c_escape_char or repl[i2] }); + end; + min_repl_n += 1; + end; + end; + i1 = i2; + end; + if conditional_c[1] then + error("malformed substitution pattern", 3); + end; + if not repl_r[2] and type(repl_r[1]) == "table" and repl_r[1][1] ~= "condition" then + repl, repl.n = repl_r[1], #repl_r[1]; + else + repl, repl_type = repl_r, "subst_string"; + end; + end; + end; + str = to_str_arr(str); + local incr, i0, count = 0, 1, 0; + while i0 <= str.n + incr + 1 do + local span = re_rawfind(self.token, str, i0, self.flags, self.verb_flags, false); + if not span then + break; + end; + local repl_r; + if repl_type == "string" then + repl_r = repl; + elseif repl_type == "subst_string" then + repl_r = insert_tokenized_sub(table.create(min_repl_n), str, span, repl); + else + local re_match; + local repl_c; + if repl_type == "table" then + re_match = utf8_sub(str.s, span[0][1], span[0][2]); + repl_c = repl[re_match]; + else + re_match = new_match(span, self.group_id, source, str.s); + repl_c = repl(re_match); + end; + if repl_c == re_match or repl_flags.o and not repl_c then + local repl_n = span[0][2] - span[0][1]; + repl_r = table.move(str, span[0][1], span[0][2] - 1, 1, table.create(repl_n)); + repl_r.n = repl_n; + elseif type(repl_c) == "string" then + repl_r = to_str_arr(repl_c); + elseif type(repl_c) == "number" then + repl_r = to_str_arr(repl_c .. ''); + elseif repl_flags.o then + error(string.format("invalid replacement value (a %s)", type(repl_c)), 3); + else + repl_r = { n = 0 }; + end; + end; + local match_len = span[0][2] - span[0][1]; + local repl_len = math.min(repl_r.n, match_len); + for i1 = 0, repl_len - 1 do + str[span[0][1] + i1] = repl_r[i1 + 1]; + end; + local i1 = span[0][1] + repl_len; + i0 = span[0][2]; + if match_len > repl_r.n then + for i2 = 1, match_len - repl_r.n do + table.remove(str, i1); + incr -= 1; + i0 -= 1; + end; + elseif repl_r.n > match_len then + for i2 = 1, repl_r.n - match_len do + table.insert(str, i1 + i2 - 1, repl_r[repl_len + i2]); + incr += 1; + i0 += 1; + end; + end; + if match_len <= 0 then + i0 += 1; + end; + count += 1; + if n < count + 1 then + break; + end; + end; + return from_str_arr(str), count; +end); + +re_m.split = check_re('RegEx', 'split', function(self, str, n) + if tonumber(n) then + n = tonumber(n); + if n <= -1 or n ~= n then + n = math.huge; + end; + elseif n ~= nil then + error(string.format("invalid argument #3 to 'split' (number expected, got %s)", typeof(n)), 3); + else + n = math.huge; + end; + str = to_str_arr(str); + local i, count = 1, 0; + local ret = { }; + local prev_empty = 0; + while i <= str.n + 1 do + count += 1; + local span = n >= count and re_rawfind(self.token, str, i, self.flags, self.verb_flags, false); + if not span then + break; + end; + table.insert(ret, utf8_sub(str.s, i - prev_empty, span[0][1])); + prev_empty = span[0][1] >= span[0][2] and 1 or 0; + i = span[0][2] + prev_empty; + end; + table.insert(ret, string.sub(str.s, utf8.offset(str.s, i - prev_empty))); + return ret; +end); + +-- +local function re_index(self, index) + return re_m[index] or proxy[self].flags[index]; +end; + +local function re_tostr(self) + return proxy[self].pattern_repr .. proxy[self].flag_repr; +end; +-- + +local other_valid_group_char = { + -- non-capturing group + [0x3A] = true, + -- lookarounds + [0x21] = true, [0x3D] = true, + -- atomic + [0x3E] = true, + -- branch reset + [0x7C] = true, +}; + +local function tokenize_ptn(codes, flags) + if flags.unicode and not options.unicodeData then + return "options.unicodeData cannot be turned off while having unicode flag"; + end; + local i, len = 1, codes.n; + local group_n = 0; + local outln, group_id, verb_flags = { }, { }, { + newline = 1, newline_seq = 1, not_empty = 0, + }; + while i <= len do + local c = codes[i]; + if c == 0x28 then + -- Match + local ret; + if codes[i + 1] == 0x2A then + i += 2; + local start_i = i; + while codes[i] + and (codes[i] >= 0x30 and codes[i] <= 0x39 + or codes[i] >= 0x41 and codes[i] <= 0x5A + or codes[i] >= 0x61 and codes[i] <= 0x7A + or codes[i] == 0x5F or codes[i] == 0x3A) do + i += 1; + end; + if codes[i] ~= 0x29 and codes[i - 1] ~= 0x3A then + -- fallback as normal and ( can't be repeated + return "quantifier doesn't follow a repeatable pattern"; + end; + local selected_verb = utf8_sub(codes.s, start_i, i); + if selected_verb == "positive_lookahead:" or selected_verb == "negative_lookhead:" + or selected_verb == "positive_lookbehind:" or selected_verb == "negative_lookbehind:" + or selected_verb:find("^[pn]l[ab]:$") then + ret = { 0x28, nil, nil, selected_verb:find('^n') and 0x21 or 0x3D, selected_verb:find('b', 3, true) and 1 }; + elseif selected_verb == "atomic:" then + ret = { 0x28, nil, nil, 0x3E, nil }; + elseif selected_verb == "ACCEPT" or selected_verb == "FAIL" or selected_verb == 'F' or selected_verb == "PRUNE" or selected_verb == "SKIP" then + ret = selected_verb == 'F' and "FAIL" or selected_verb; + else + if line_verbs[selected_verb] then + verb_flags.newline = selected_verb; + elseif selected_verb == "BSR_ANYCRLF" or selected_verb == "BSR_UNICODE" then + verb_flags.newline_seq = selected_verb == "BSR_UNICODE" and 1 or 0; + elseif selected_verb == "NOTEMPTY" or selected_verb == "NOTEMPTY_ATSTART" then + verb_flags.not_empty = selected_verb == "NOTEMPTY" and 1 or 2; + else + return "unknown or malformed verb"; + end; + if outln[1] then + return "this verb must be placed at the beginning of the regex"; + end; + end; + elseif codes[i + 1] == 0x3F then + -- ? syntax + i += 2; + if codes[i] == 0x23 then + -- comments + i = table.find(codes, 0x29, i); + if not i then + return "unterminated parenthetical"; + end; + i += 1; + continue; + elseif not codes[i] then + return "unterminated parenthetical"; + end; + ret = { 0x28, nil, nil, codes[i], nil }; + if codes[i] == 0x30 and codes[i + 1] == 0x29 then + -- recursive match entire pattern + ret[1], ret[2], ret[3], ret[5] = "recurmatch", 0, 0, nil; + elseif codes[i] > 0x30 and codes[i] <= 0x39 then + -- recursive match + local org_i = i; + i += 1; + while codes[i] >= 0x30 and codes[i] <= 0x30 do + i += 1; + end; + if codes[i] ~= 0x29 then + return "invalid group structure"; + end; + ret[1], ret[2], ret[4] = "recurmatch", tonumber(utf8_sub(codes.s, org_i, i)), nil; + elseif codes[i] == 0x3C and codes[i + 1] == 0x21 or codes[i + 1] == 0x3D then + -- lookbehinds + i += 1; + ret[4], ret[5] = codes[i], 1; + elseif codes[i] == 0x7C then + -- branch reset + ret[5] = group_n; + elseif codes[i] == 0x50 or codes[i] == 0x3C or codes[i] == 0x27 then + if codes[i] == 0x50 then + i += 1; + end; + if codes[i] == 0x3D then + -- backref + local start_i = i + 1; + while codes[i] and + (codes[i] >= 0x30 and codes[i] <= 0x39 + or codes[i] >= 0x41 and codes[i] <= 0x5A + or codes[i] >= 0x61 and codes[i] <= 0x7A + or codes[i] == 0x5F) do + i += 1; + end; + if not codes[i] then + return "unterminated parenthetical"; + elseif codes[i] ~= 0x29 or i == start_i then + return "invalid group structure"; + end; + ret = { "backref", utf8_sub(codes.s, start_i, i) }; + elseif codes[i] == 0x3C or codes[i - 1] ~= 0x50 and codes[i] == 0x27 then + -- named capture + local delimiter = codes[i] == 0x27 and 0x27 or 0x3E; + local start_i = i + 1; + i += 1; + if codes[i] == 0x29 then + return "missing character in subpattern"; + elseif codes[i] >= 0x30 and codes[i] <= 0x39 then + return "subpattern name must not begin with a digit"; + elseif not (codes[i] >= 0x41 and codes[i] <= 0x5A or codes[i] >= 0x61 and codes[i] <= 0x7A or codes[i] == 0x5F) then + return "invalid character in subpattern"; + end; + i += 1; + while codes[i] and + (codes[i] >= 0x30 and codes[i] <= 0x39 + or codes[i] >= 0x41 and codes[i] <= 0x5A + or codes[i] >= 0x61 and codes[i] <= 0x7A + or codes[i] == 0x5F) do + i += 1; + end; + if not codes[i] then + return "unterminated parenthetical"; + elseif codes[i] ~= delimiter then + return "invalid character in subpattern"; + end; + local name = utf8_sub(codes.s, start_i, i); + group_n += 1; + if (group_id[name] or group_n) ~= group_n then + return "subpattern name already exists"; + end; + for name1, group_n1 in pairs(group_id) do + if name ~= name1 and group_n == group_n1 then + return "different names for subpatterns of the same number aren't permitted"; + end; + end; + group_id[name] = group_n; + ret[2], ret[4] = group_n, nil; + else + return "invalid group structure"; + end; + elseif not other_valid_group_char[codes[i]] then + return "invalid group structure"; + end; + else + group_n += 1; + ret = { 0x28, group_n, nil, nil }; + end; + if ret then + table.insert(outln, ret); + end; + elseif c == 0x29 then + -- Close parenthesis + local i1 = #outln + 1; + local lookbehind_c = -1; + local current_lookbehind_c = 0; + local max_c, group_c = 0, 0; + repeat + i1 -= 1; + local v, is_table = outln[i1], type(outln[i1]) == "table"; + if is_table and v[1] == 0x28 then + group_c += 1; + if current_lookbehind_c and v.count then + current_lookbehind_c += v.count; + end; + if not v[3] then + if v[4] == 0x7C then + group_n = v[5] + math.max(max_c, group_c); + end; + if current_lookbehind_c ~= lookbehind_c and lookbehind_c ~= -1 then + lookbehind_c = nil; + else + lookbehind_c = current_lookbehind_c; + end; + break; + end; + elseif v == alternation then + if current_lookbehind_c ~= lookbehind_c and lookbehind_c ~= -1 then + lookbehind_c, current_lookbehind_c = nil, nil; + else + lookbehind_c, current_lookbehind_c = current_lookbehind_c, 0; + end; + max_c, group_c = math.max(max_c, group_c), 0; + elseif current_lookbehind_c then + if is_table and v[1] == "quantifier" then + if v[2] == v[3] then + current_lookbehind_c += v[2]; + else + current_lookbehind_c = nil; + end; + else + current_lookbehind_c += 1; + end; + end; + until i1 < 1; + if i1 < 1 then + return "unmatched ) in regular expression"; + end; + local v = outln[i1]; + local outln_len_p_1 = #outln + 1; + local ret = { 0x29, v[2], i1, v[4], v[5], count = lookbehind_c }; + if (v[4] == 0x21 or v[4] == 0x3D) and v[5] and not lookbehind_c then + return "lookbehind assertion is not fixed width"; + end; + v[3] = outln_len_p_1; + table.insert(outln, ret); + elseif c == 0x2E then + table.insert(outln, dot); + elseif c == 0x5B then + -- Character set + local negate, char_class = false, nil; + i += 1; + local start_i = i; + if codes[i] == 0x5E then + negate = true; + i += 1; + elseif codes[i] == 0x2E or codes[i] == 0x3A or codes[i] == 0x3D then + -- POSIX character classes + char_class = codes[i]; + end; + local ret; + if codes[i] == 0x5B or codes[i] == 0x5C then + ret = { }; + else + ret = { codes[i] }; + i += 1; + end; + while codes[i] ~= 0x5D do + if not codes[i] then + return "unterminated character class"; + elseif codes[i] == 0x2D and ret[1] and type(ret[1]) == "number" then + if codes[i + 1] == 0x5D then + table.insert(ret, 1, 0x2D); + else + i += 1; + local ret_c = codes[i]; + if ret_c == 0x5B then + if codes[i + 1] == 0x2E or codes[i + 1] == 0x3A or codes[i + 1] == 0x3D then + -- Check for POSIX character class, name does not matter + local i1 = i + 2; + repeat + i1 = table.find(codes, 0x5D, i1); + until not i1 or codes[i1 - 1] ~= 0x5C; + if not i1 then + return "unterminated character class"; + elseif codes[i1 - 1] == codes[i + 1] and i1 - 1 ~= i + 1 then + return "invalid range in character class"; + end; + end; + if ret[1] > 0x5B then + return "invalid range in character class"; + end; + elseif ret_c == 0x5C then + i += 1; + if codes[i] == 0x78 then + local radix0, radix1; + i += 1; + if codes[i] and codes[i] >= 0x30 and codes[i] <= 0x39 or codes[i] >= 0x41 and codes[i] <= 0x46 or codes[i] >= 0x61 and codes[i] <= 0x66 then + radix0 = codes[i] - ((codes[i] >= 0x41 and codes[i] <= 0x5A) and 0x37 or (codes[i] >= 0x61 and codes[i] <= 0x7A) and 0x57 or 0x30); + i += 1; + if codes[i] and codes[i] >= 0x30 and codes[i] <= 0x39 or codes[i] >= 0x41 and codes[i] <= 0x46 or codes[i] >= 0x61 and codes[i] <= 0x66 then + radix1 = codes[i] - ((codes[i] >= 0x41 and codes[i] <= 0x5A) and 0x37 or (codes[i] >= 0x61 and codes[i] <= 0x7A) and 0x57 or 0x30); + else + i -= 1; + end; + else + i -= 1; + end; + ret_c = radix0 and (radix1 and 16 * radix0 + radix1 or radix0) or 0; + elseif codes[i] >= 0x30 and codes[i] <= 0x37 then + local radix0, radix1, radix2 = codes[i] - 0x30, nil, nil; + i += 1; + if codes[i] and codes[i] >= 0x30 and codes[i] <= 0x37 then + radix1 = codes[i] - 0x30; + i += 1; + if codes[i] and codes[i] >= 0x30 and codes[i] <= 0x37 then + radix2 = codes[i] - 0x30; + else + i -= 1; + end; + else + i -= 1; + end; + ret_c = radix1 and (radix2 and 64 * radix0 + 8 * radix1 + radix2 or 8 * radix0 + radix1) or radix0; + else + ret_c = escape_chars[codes[i]] or codes[i]; + if type(ret_c) ~= "number" then + return "invalid range in character class"; + end; + end; + elseif ret[1] > ret_c then + return "invalid range in character class"; + end; + ret[1] = { "range", ret[1], ret_c }; + end; + elseif codes[i] == 0x5B then + if codes[i + 1] == 0x2E or codes[i + 1] == 0x3A or codes[i + 1] == 0x3D then + local i1 = i + 2; + repeat + i1 = table.find(codes, 0x5D, i1); + until not i1 or codes[i1 - 1] ~= 0x5C; + if not i1 then + return "unterminated character class"; + elseif codes[i1 - 1] ~= codes[i + 1] or i1 - 1 == i + 1 then + table.insert(ret, 1, 0x5B); + elseif codes[i1 - 1] == 0x2E or codes[i1 - 1] == 0x3D then + return "POSIX collating elements aren't supported"; + elseif codes[i1 - 1] == 0x3A then + -- I have no plans to support escape codes (\) in character class names + local negate = codes[i + 3] == 0x5E; + local class_name = utf8_sub(codes.s, i + (negate and 3 or 2), i1 - 1); + -- If not valid then throw an error + if not posix_class_names[class_name] then + return "unknown POSIX class name"; + end; + table.insert(ret, 1, { "class", class_name, negate }); + i = i1; + end; + else + table.insert(ret, 1, 0x5B); + end; + elseif codes[i] == 0x5C then + i += 1; + if codes[i] == 0x78 then + local radix0, radix1; + i += 1; + if codes[i] == 0x7B then + i += 1; + local org_i = i; + while codes[i] and + (codes[i] >= 0x30 and codes[i] <= 0x39 + or codes[i] >= 0x41 and codes[i] <= 0x46 + or codes[i] >= 0x61 and codes[i] <= 0x66) do + i += 1; + end; + if codes[i] ~= 0x7D or i == org_i then + return "malformed hexadecimal character"; + elseif i - org_i > 4 then + return "character offset too large"; + end; + table.insert(ret, 1, tonumber(utf8_sub(codes.s, org_i, i), 16)); + else + if codes[i] and codes[i] >= 0x30 and codes[i] <= 0x39 or codes[i] >= 0x41 and codes[i] <= 0x46 or codes[i] >= 0x61 and codes[i] <= 0x66 then + radix0 = codes[i] - ((codes[i] >= 0x41 and codes[i] <= 0x5A) and 0x37 or (codes[i] >= 0x61 and codes[i] <= 0x7A) and 0x57 or 0x30); + i += 1; + if codes[i] and codes[i] >= 0x30 and codes[i] <= 0x39 or codes[i] >= 0x41 and codes[i] <= 0x46 or codes[i] >= 0x61 and codes[i] <= 0x66 then + radix1 = codes[i] - ((codes[i] >= 0x41 and codes[i] <= 0x5A) and 0x37 or (codes[i] >= 0x61 and codes[i] <= 0x7A) and 0x57 or 0x30); + else + i -= 1; + end; + else + i -= 1; + end; + table.insert(ret, 1, radix0 and (radix1 and 16 * radix0 + radix1 or radix0) or 0); + end; + elseif codes[i] >= 0x30 and codes[i] <= 0x37 then + local radix0, radix1, radix2 = codes[i] - 0x30, nil, nil; + i += 1; + if codes[i] and codes[i] >= 0x30 and codes[i] <= 0x37 then + radix1 = codes[i] - 0x30; + i += 1; + if codes[i] and codes[i] >= 0x30 and codes[i] <= 0x37 then + radix2 = codes[i] - 0x30; + else + i -= 1; + end; + else + i -= 1; + end; + table.insert(ret, 1, radix1 and (radix2 and 64 * radix0 + 8 * radix1 + radix2 or 8 * radix0 + radix1) or radix0); + elseif codes[i] == 0x45 then + -- intentionally left blank, \E that's not preceded \Q is ignored + elseif codes[i] == 0x51 then + local start_i = i + 1; + repeat + i = table.find(codes, 0x5C, i + 1); + until not i or codes[i + 1] == 0x45; + table.move(codes, start_i, i and i - 1 or #codes, #outln + 1, outln); + if not i then + break; + end; + i += 1; + elseif codes[i] == 0x4E then + if codes[i + 1] == 0x7B and codes[i + 2] == 0x55 and codes[i + 3] == 0x2B and flags.unicode then + i += 4; + local start_i = i; + while codes[i] and + (codes[i] >= 0x30 and codes[i] <= 0x39 + or codes[i] >= 0x41 and codes[i] <= 0x46 + or codes[i] >= 0x61 and codes[i] <= 0x66) do + i += 1; + end; + if codes[i] ~= 0x7D or i == start_i then + return "malformed Unicode code point"; + end; + local code_point = tonumber(utf8_sub(codes.s, start_i, i)); + table.insert(ret, 1, code_point); + else + return "invalid escape sequence"; + end; + elseif codes[i] == 0x50 or codes[i] == 0x70 then + if not options.unicodeData then + return "options.unicodeData cannot be turned off when using \\p"; + end; + i += 1; + if codes[i] ~= 0x7B then + local c_name = utf8.char(codes[i] or 0); + if not valid_categories[c_name] then + return "unknown or malformed script name"; + end; + table.insert(ret, 1, { "category", false, c_name }); + else + local negate = codes[i] == 0x50; + i += 1; + if codes[i] == 0x5E then + i += 1; + negate = not negate; + end; + local start_i = i; + while codes[i] and + (codes[i] >= 0x30 and codes[i] <= 0x39 + or codes[i] >= 0x41 and codes[i] <= 0x5A + or codes[i] >= 0x61 and codes[i] <= 0x7A + or codes[i] == 0x5F) do + i += 1; + end; + if codes[i] ~= 0x7D then + return "unknown or malformed script name"; + end; + local c_name = utf8_sub(codes.s, start_i, i); + local script_set = chr_scripts[c_name]; + if script_set then + table.insert(ret, 1, { "charset", negate, script_set }); + elseif not valid_categories[c_name] then + return "unknown or malformed script name"; + else + table.insert(ret, 1, { "category", negate, c_name }); + end; + end; + elseif codes[i] == 0x6F then + i += 1; + if codes[i] ~= 0x7B then + return "malformed octal code"; + end; + i += 1; + local org_i = i; + while codes[i] and codes[i] >= 0x30 and codes[i] <= 0x37 do + i += 1; + end; + if codes[i] ~= 0x7D or i == org_i then + return "malformed octal code"; + end; + local ret_chr = tonumber(utf8_sub(codes.s, org_i, i), 8); + if ret_chr > 0xFFFF then + return "character offset too large"; + end; + table.insert(ret, 1, ret_chr); + else + local esc_char = escape_chars[codes[i]]; + table.insert(ret, 1, type(esc_char) == "string" and { "class", esc_char, false } or esc_char or codes[i]); + end; + elseif flags.ignoreCase and codes[i] >= 0x61 and codes[i] <= 0x7A then + table.insert(ret, 1, codes[i] - 0x20); + else + table.insert(ret, 1, codes[i]); + end; + i += 1; + end; + if codes[i - 1] == char_class and i - 1 ~= start_i then + return char_class == 0x3A and "POSIX named classes are only support within a character set" or "POSIX collating elements aren't supported"; + end; + if not ret[2] and not negate then + table.insert(outln, ret[1]); + else + table.insert(outln, { "charset", negate, ret }); + end; + elseif c == 0x5C then + -- Escape char + i += 1; + local escape_c = codes[i]; + if not escape_c then + return "pattern may not end with a trailing backslash"; + elseif escape_c >= 0x30 and escape_c <= 0x39 then + local org_i = i; + while codes[i + 1] and codes[i + 1] >= 0x30 and codes[i + 1] <= 0x39 do + i += 1; + end; + local escape_d = tonumber(utf8_sub(codes.s, org_i, i + 1)); + if escape_d > group_n and i ~= org_i then + i = org_i; + local radix0, radix1, radix2; + if codes[i] <= 0x37 then + radix0 = codes[i] - 0x30; + i += 1; + if codes[i] and codes[i] >= 0x30 and codes[i] <= 0x37 then + radix1 = codes[i] - 0x30; + i += 1; + if codes[i] and codes[i] >= 0x30 and codes[i] <= 0x37 then + radix2 = codes[i] - 0x30; + else + i -= 1; + end; + else + i -= 1; + end; + end; + table.insert(outln, radix0 and (radix1 and (radix2 and 64 * radix0 + 8 * radix1 + radix2 or 8 * radix0 + radix1) or radix0) or codes[org_i]); + else + table.insert(outln, { "backref", escape_d }); + end; + elseif escape_c == 0x45 then + -- intentionally left blank, \E that's not preceded \Q is ignored + elseif escape_c == 0x51 then + local start_i = i + 1; + repeat + i = table.find(codes, 0x5C, i + 1); + until not i or codes[i + 1] == 0x45; + table.move(codes, start_i, i and i - 1 or #codes, #outln + 1, outln); + if not i then + break; + end; + i += 1; + elseif escape_c == 0x4E then + if codes[i + 1] == 0x7B and codes[i + 2] == 0x55 and codes[i + 3] == 0x2B and flags.unicode then + i += 4; + local start_i = i; + while codes[i] and + (codes[i] >= 0x30 and codes[i] <= 0x39 + or codes[i] >= 0x41 and codes[i] <= 0x46 + or codes[i] >= 0x61 and codes[i] <= 0x66) do + i += 1; + end; + if codes[i] ~= 0x7D or i == start_i then + return "malformed Unicode code point"; + end; + local code_point = tonumber(utf8_sub(codes.s, start_i, i)); + table.insert(outln, code_point); + else + table.insert(outln, escape_chars[0x4E]); + end; + elseif escape_c == 0x50 or escape_c == 0x70 then + if not options.unicodeData then + return "options.unicodeData cannot be turned off when using \\p"; + end; + i += 1; + if codes[i] ~= 0x7B then + local c_name = utf8.char(codes[i] or 0); + if not valid_categories[c_name] then + return "unknown or malformed script name"; + end; + table.insert(outln, { "category", false, c_name }); + else + local negate = escape_c == 0x50; + i += 1; + if codes[i] == 0x5E then + i += 1; + negate = not negate; + end; + local start_i = i; + while codes[i] and + (codes[i] >= 0x30 and codes[i] <= 0x39 + or codes[i] >= 0x41 and codes[i] <= 0x5A + or codes[i] >= 0x61 and codes[i] <= 0x7A + or codes[i] == 0x5F) do + i += 1; + end; + if codes[i] ~= 0x7D then + return "unknown or malformed script name"; + end; + local c_name = utf8_sub(codes.s, start_i, i); + local script_set = chr_scripts[c_name]; + if script_set then + table.insert(outln, { "charset", negate, script_set }); + elseif not valid_categories[c_name] then + return "unknown or malformed script name"; + else + table.insert(outln, { "category", negate, c_name }); + end; + end; + elseif escape_c == 0x67 and (codes[i + 1] == 0x7B or codes[i + 1] >= 0x30 and codes[i + 1] <= 0x39) then + local is_grouped = false; + i += 1; + if codes[i] == 0x7B then + i += 1; + is_grouped = true; + elseif codes[i] < 0x30 or codes[i] > 0x39 then + return "malformed reference code"; + end; + local org_i = i; + while codes[i] and + (codes[i] >= 0x30 and codes[i] <= 0x39 + or codes[i] >= 0x41 and codes[i] <= 0x46 + or codes[i] >= 0x61 and codes[i] <= 0x66) do + i += 1; + end; + if is_grouped and codes[i] ~= 0x7D then + return "malformed reference code"; + end; + local ref_name = tonumber(utf8_sub(codes.s, org_i, i + (is_grouped and 0 or 1))); + table.insert(outln, { "backref", ref_name }); + if not is_grouped then + i -= 1; + end; + elseif escape_c == 0x6F then + i += 1; + if codes[i + 1] ~= 0x7B then + return "malformed octal code"; + end + i += 1; + local org_i = i; + while codes[i] and codes[i] >= 0x30 and codes[i] <= 0x37 do + i += 1; + end; + if codes[i] ~= 0x7D or i == org_i then + return "malformed octal code"; + end; + local ret_chr = tonumber(utf8_sub(codes.s, org_i, i), 8); + if ret_chr > 0xFFFF then + return "character offset too large"; + end; + table.insert(outln, ret_chr); + elseif escape_c == 0x78 then + local radix0, radix1; + i += 1; + if codes[i] == 0x7B then + i += 1; + local org_i = i; + while codes[i] and + (codes[i] >= 0x30 and codes[i] <= 0x39 + or codes[i] >= 0x41 and codes[i] <= 0x46 + or codes[i] >= 0x61 and codes[i] <= 0x66) do + i += 1; + end; + if codes[i] ~= 0x7D or i == org_i then + return "malformed hexadecimal code"; + elseif i - org_i > 4 then + return "character offset too large"; + end; + table.insert(outln, tonumber(utf8_sub(codes.s, org_i, i), 16)); + else + if codes[i] and (codes[i] >= 0x30 and codes[i] <= 0x39 or codes[i] >= 0x41 and codes[i] <= 0x46 or codes[i] >= 0x61 and codes[i] <= 0x66) then + radix0 = codes[i] - ((codes[i] >= 0x41 and codes[i] <= 0x5A) and 0x37 or (codes[i] >= 0x61 and codes[i] <= 0x7A) and 0x57 or 0x30); + i += 1; + if codes[i] and (codes[i] >= 0x30 and codes[i] <= 0x39 or codes[i] >= 0x41 and codes[i] <= 0x46 or codes[i] >= 0x61 and codes[i] <= 0x66) then + radix1 = codes[i] - ((codes[i] >= 0x41 and codes[i] <= 0x5A) and 0x37 or (codes[i] >= 0x61 and codes[i] <= 0x7A) and 0x57 or 0x30); + else + i -= 1; + end; + else + i -= 1; + end; + table.insert(outln, radix0 and (radix1 and 16 * radix0 + radix1 or radix0) or 0); + end; + else + local esc_char = b_escape_chars[escape_c] or escape_chars[escape_c]; + table.insert(outln, esc_char or escape_c); + end; + elseif c == 0x2A or c == 0x2B or c == 0x3F or c == 0x7B then + -- Quantifier + local start_q, end_q; + if c == 0x7B then + local org_i = i + 1; + local start_i; + while codes[i + 1] and (codes[i + 1] >= 0x30 and codes[i + 1] <= 0x39 or codes[i + 1] == 0x2C and not start_i and i + 1 ~= org_i) do + i += 1; + if codes[i] == 0x2C then + start_i = i; + end; + end; + if codes[i + 1] == 0x7D then + i += 1; + if not start_i then + start_q = tonumber(utf8_sub(codes.s, org_i, i)); + end_q = start_q; + else + start_q, end_q = tonumber(utf8_sub(codes.s, org_i, start_i)), start_i + 1 == i and math.huge or tonumber(utf8_sub(codes.s, start_i + 1, i)); + if end_q < start_q then + return "numbers out of order in {} quantifier"; + end; + end; + else + table.move(codes, org_i - 1, i, #outln + 1, outln); + end; + else + start_q, end_q = c == 0x2B and 1 or 0, c == 0x3F and 1 or math.huge; + end; + if start_q then + local quantifier_type = flags.ungreedy and "lazy" or "greedy"; + if codes[i + 1] == 0x2B or codes[i + 1] == 0x3F then + i += 1; + quantifier_type = codes[i] == 0x2B and "possessive" or flags.ungreedy and "greedy" or "lazy"; + end; + local outln_len = #outln; + local last_outln_value = outln[outln_len]; + if not last_outln_value or type(last_outln_value) == "table" and (last_outln_value[1] == "quantifier" or last_outln_value[1] == 0x28 or b_escape_chars[last_outln_value[1]]) + or last_outln_value == alternation or type(last_outln_value) == "string" then + return "quantifier doesn't follow a repeatable pattern"; + end; + if end_q == 0 then + table.remove(outln); + elseif start_q ~= 1 or end_q ~= 1 then + if type(last_outln_value) == "table" and last_outln_value[1] == 0x29 then + outln_len = last_outln_value[3]; + end; + outln[outln_len] = { "quantifier", start_q, end_q, quantifier_type, outln[outln_len] }; + end; + end; + elseif c == 0x7C then + -- Alternation + table.insert(outln, alternation); + local i1 = #outln; + repeat + i1 -= 1; + local v1, is_table = outln[i1], type(outln[i1]) == "table"; + if is_table and v1[1] == 0x29 then + i1 = outln[i1][3]; + elseif is_table and v1[1] == 0x28 then + if v1[4] == 0x7C then + group_n = v1[5]; + end; + break; + end; + until not v1; + elseif c == 0x24 or c == 0x5E then + table.insert(outln, c == 0x5E and beginning_str or end_str); + elseif flags.ignoreCase and c >= 0x61 and c <= 0x7A then + table.insert(outln, c - 0x20); + elseif flags.extended and (c >= 0x09 and c <= 0x0D or c == 0x20 or c == 0x23) then + if c == 0x23 then + repeat + i += 1; + until not codes[i] or codes[i] == 0x0A or codes[i] == 0x0D; + end; + else + table.insert(outln, c); + end; + i += 1; + end; + local max_group_n = 0; + for i, v in ipairs(outln) do + if type(v) == "table" and (v[1] == 0x28 or v[1] == "quantifier" and type(v[5]) == "table" and v[5][1] == 0x28) then + if v[1] == "quantifier" then + v = v[5]; + end; + if not v[3] then + return "unterminated parenthetical"; + elseif v[2] then + max_group_n = math.max(max_group_n, v[2]); + end; + elseif type(v) == "table" and (v[1] == "backref" or v[1] == "recurmatch") then + if not group_id[v[2]] and (type(v[2]) ~= "number" or v[2] > group_n) then + return "reference to a non-existent or invalid subpattern"; + elseif v[1] == "recurmatch" and v[2] ~= 0 then + for i1, v1 in ipairs(outln) do + if type(v1) == "table" and v1[1] == 0x28 and v1[2] == v[2] then + v[3] = i1; + break; + end; + end; + elseif type(v[2]) == "string" then + v[2] = group_id[v[2]]; + end; + end; + end; + outln.group_n = max_group_n; + return outln, group_id, verb_flags; +end; + +if not tonumber(options.cacheSize) then + error(string.format("expected number for options.cacheSize, got %s", typeof(options.cacheSize)), 2); +end; +local cacheSize = math.floor(options.cacheSize or 0) ~= 0 and tonumber(options.cacheSize); +local cache_pattern, cache_pattern_names; +if not cacheSize then +elseif cacheSize < 0 or cacheSize ~= cacheSize then + error("cache size cannot be a negative number or a NaN", 2); +elseif cacheSize == math.huge then + cache_pattern, cache_pattern_names = { nil }, { nil }; +elseif cacheSize >= 2 ^ 32 then + error("cache size too large", 2); +else + cache_pattern, cache_pattern_names = table.create(options.cacheSize), table.create(options.cacheSize); +end; +if cacheSize then + function re.pruge() + table.clear(cache_pattern_names); + table.clear(cache_pattern); + end; +end; + +local function new_re(str_arr, flags, flag_repr, pattern_repr) + local tokenized_ptn, group_id, verb_flags; + local cache_format = cacheSize and string.format("%s|%s", str_arr.s, flag_repr); + local cached_token = cacheSize and cache_pattern[table.find(cache_pattern_names, cache_format)]; + if cached_token then + tokenized_ptn, group_id, verb_flags = table.unpack(cached_token, 1, 3); + else + tokenized_ptn, group_id, verb_flags = tokenize_ptn(str_arr, flags); + if type(tokenized_ptn) == "string" then + error(tokenized_ptn, 2); + end; + if cacheSize and tokenized_ptn[1] then + table.insert(cache_pattern_names, 1, cache_format); + table.insert(cache_pattern, 1, { tokenized_ptn, group_id, verb_flags }); + if cacheSize ~= math.huge then + table.remove(cache_pattern_names, cacheSize + 1); + table.remove(cache_pattern, cacheSize + 1); + end; + end; + end; + + local object = newproxy(true); + proxy[object] = { name = "RegEx", flags = flags, flag_repr = flag_repr, pattern_repr = pattern_repr, token = tokenized_ptn, group_id = group_id, verb_flags = verb_flags }; + local object_mt = getmetatable(object); + object_mt.__index = setmetatable(flags, re_m); + object_mt.__tostring = re_tostr; + object_mt.__metatable = lockmsg; + + return object; +end; + +local function escape_fslash(pre) + return (#pre % 2 == 0 and '\\' or '') .. pre .. '.'; +end; + +local function sort_flag_chr(a, b) + return a:lower() < b:lower(); +end; + +function re.new(...) + if select('#', ...) == 0 then + error("missing argument #1 (string expected)", 2); + end; + local ptn, flags_str = ...; + if type(ptn) == "number" then + ptn ..= ''; + elseif type(ptn) ~= "string" then + error(string.format("invalid argument #1 (string expected, got %s)", typeof(ptn)), 2); + end; + if type(flags_str) ~= "string" and type(flags_str) ~= "number" and flags_str ~= nil then + error(string.format("invalid argument #2 (string expected, got %s)", typeof(flags_str)), 2); + end; + + local flags = { + anchored = false, caseless = false, multiline = false, dotall = false, unicode = false, ungreedy = false, extended = false, + }; + local flag_repr = { }; + for f in string.gmatch(flags_str or '', utf8.charpattern) do + if flags[flag_map[f]] ~= false then + error("invalid regular expression flag " .. f, 3); + end; + flags[flag_map[f]] = true; + table.insert(flag_repr, f); + end; + table.sort(flag_repr, sort_flag_chr); + flag_repr = table.concat(flag_repr); + return new_re(to_str_arr(ptn), flags, flag_repr, string.format("/%s/", ptn:gsub("(\\*)/", escape_fslash))); +end; + +function re.fromstring(...) + if select('#', ...) == 0 then + error("missing argument #1 (string expected)", 2); + end; + local ptn = ...; + if type(ptn) == "number" then + ptn ..= ''; + elseif type(ptn) ~= "string" then + error(string.format("invalid argument #1 (string expected, got %s)", typeof(ptn), 2)); + end; + local str_arr = to_str_arr(ptn); + local delimiter = str_arr[1]; + if not delimiter then + error("empty regex", 2); + elseif delimiter == 0x5C or (delimiter >= 0x30 and delimiter <= 0x39) or (delimiter >= 0x41 and delimiter <= 0x5A) or (delimiter >= 0x61 and delimiter <= 0x7A) then + error("delimiter must not be alphanumeric or a backslash", 2); + end; + + local i0 = 1; + repeat + i0 = table.find(str_arr, delimiter, i0 + 1); + if not i0 then + error(string.format("no ending delimiter ('%s') found", utf8.char(delimiter)), 2); + end; + local escape_count = 1; + while str_arr[i0 - escape_count] == 0x5C do + escape_count += 1; + end; + until escape_count % 2 == 1; + + local flags = { + anchored = false, caseless = false, multiline = false, dotall = false, unicode = false, ungreedy = false, extended = false, + }; + local flag_repr = { }; + while str_arr.n > i0 do + local f = utf8.char(table.remove(str_arr)); + str_arr.n -= 1; + if flags[flag_map[f]] ~= false then + error("invalid regular expression flag " .. f, 3); + end; + flags[flag_map[f]] = true; + table.insert(flag_repr, f); + end; + table.sort(flag_repr, sort_flag_chr); + flag_repr = table.concat(flag_repr); + table.remove(str_arr, 1); + table.remove(str_arr); + str_arr.n -= 2; + str_arr.s = string.sub(str_arr.s, 2, 1 + str_arr.n); + return new_re(str_arr, flags, flag_repr, string.sub(ptn, 1, 2 + str_arr.n)); +end; + +local re_escape_line_chrs = { + ['\0'] = '\\x00', ['\n'] = '\\n', ['\t'] = '\\t', ['\r'] = '\\r', ['\f'] = '\\f', +}; + +function re.escape(...) + if select('#', ...) == 0 then + error("missing argument #1 (string expected)", 2); + end; + local str, extended, delimiter = ...; + if type(str) == "number" then + str ..= ''; + elseif type(str) ~= "string" then + error(string.format("invalid argument #1 to 'escape' (string expected, got %s)", typeof(str)), 2); + end; + if delimiter == nil then + delimiter = ''; + elseif type(delimiter) == "number" then + delimiter ..= ''; + elseif type(delimiter) ~= "string" then + error(string.format("invalid argument #3 to 'escape' (string expected, got %s)", typeof(delimiter)), 2); + end; + if utf8.len(delimiter) > 1 or delimiter:match("^[%a\\]$") then + error("delimiter have not be alphanumeric", 2); + end; + return (string.gsub(str, "[\0\f\n\r\t]", re_escape_line_chrs):gsub(string.format("[\\%s#()%%%%*+.?[%%]^{|%s]", extended and '%s' or '', (delimiter:find'^[%%%]]$' and '%' or '') .. delimiter), "\\%1")); +end; + +function re.type(...) + if select('#', ...) == 0 then + error("missing argument #1", 2); + end; + return proxy[...] and proxy[...].name; +end; + +for k, f in pairs(re_m) do + re[k] = f; +end; + +re_m = { __index = re_m }; + +lockmsg = re.fromstring([[/The\s*metatable\s*is\s*(?:locked|inaccessible)(?#Nice try :])/i]]); +getmetatable(lockmsg).__metatable = lockmsg; + +local function readonly_table() + error("Attempt to modify a readonly table", 2); +end; + +match_m = { + __index = match_m, + __metatable = lockmsg, + __newindex = readonly_table, +}; + +re.Match = setmetatable({ }, match_m); + +return setmetatable({ }, { + __index = re, + __metatable = lockmsg, + __newindex = readonly_table, +}); diff --git a/tests/AssemblyBuilderX64.test.cpp b/tests/AssemblyBuilderX64.test.cpp index 7f863c6f..15813ae9 100644 --- a/tests/AssemblyBuilderX64.test.cpp +++ b/tests/AssemblyBuilderX64.test.cpp @@ -213,6 +213,16 @@ TEST_CASE_FIXTURE(AssemblyBuilderX64Fixture, "FormsOfLea") SINGLE_COMPARE(lea(rax, qword[r13 + r12 * 4 + 4]), 0x4b, 0x8d, 0x44, 0xa5, 0x04); } +TEST_CASE_FIXTURE(AssemblyBuilderX64Fixture, "FormsOfAbsoluteJumps") +{ + SINGLE_COMPARE(jmp(rax), 0x48, 0xff, 0xe0); + SINGLE_COMPARE(jmp(r14), 0x49, 0xff, 0xe6); + SINGLE_COMPARE(jmp(qword[r14 + rdx * 4]), 0x49, 0xff, 0x24, 0x96); + SINGLE_COMPARE(call(rax), 0x48, 0xff, 0xd0); + SINGLE_COMPARE(call(r14), 0x49, 0xff, 0xd6); + SINGLE_COMPARE(call(qword[r14 + rdx * 4]), 0x49, 0xff, 0x14, 0x96); +} + TEST_CASE_FIXTURE(AssemblyBuilderX64Fixture, "ControlFlow") { // Jump back @@ -260,6 +270,23 @@ TEST_CASE_FIXTURE(AssemblyBuilderX64Fixture, "ControlFlow") {0xe9, 0x04, 0x00, 0x00, 0x00, 0x48, 0x83, 0xe7, 0x3e}); } +TEST_CASE_FIXTURE(AssemblyBuilderX64Fixture, "LabelCall") +{ + check( + [](AssemblyBuilderX64& build) { + Label fnB; + + build.and_(rcx, 0x3e); + build.call(fnB); + build.ret(); + + build.setLabel(fnB); + build.lea(rax, qword[rcx + 0x1f]); + build.ret(); + }, + {0x48, 0x83, 0xe1, 0x3e, 0xe8, 0x01, 0x00, 0x00, 0x00, 0xc3, 0x48, 0x8d, 0x41, 0x1f, 0xc3}); +} + TEST_CASE_FIXTURE(AssemblyBuilderX64Fixture, "AVXBinaryInstructionForms") { SINGLE_COMPARE(vaddpd(xmm8, xmm10, xmm14), 0xc4, 0x41, 0xa9, 0x58, 0xc6); diff --git a/tests/AstQuery.test.cpp b/tests/AstQuery.test.cpp index f0017509..6ec1426c 100644 --- a/tests/AstQuery.test.cpp +++ b/tests/AstQuery.test.cpp @@ -105,4 +105,37 @@ if true then REQUIRE(parentStat->is()); } +TEST_CASE_FIXTURE(Fixture, "ac_ast_ancestry_at_number_const") +{ + check(R"( +print(3.) + )"); + + std::vector ancestry = findAncestryAtPositionForAutocomplete(*getMainSourceModule(), Position(1, 8)); + REQUIRE_GE(ancestry.size(), 2); + REQUIRE(ancestry.back()->is()); +} + +TEST_CASE_FIXTURE(Fixture, "ac_ast_ancestry_in_workspace_dot") +{ + check(R"( +print(workspace.) + )"); + + std::vector ancestry = findAncestryAtPositionForAutocomplete(*getMainSourceModule(), Position(1, 16)); + REQUIRE_GE(ancestry.size(), 2); + REQUIRE(ancestry.back()->is()); +} + +TEST_CASE_FIXTURE(Fixture, "ac_ast_ancestry_in_workspace_colon") +{ + check(R"( +print(workspace:) + )"); + + std::vector ancestry = findAncestryAtPositionForAutocomplete(*getMainSourceModule(), Position(1, 16)); + REQUIRE_GE(ancestry.size(), 2); + REQUIRE(ancestry.back()->is()); +} + TEST_SUITE_END(); diff --git a/tests/Fixture.h b/tests/Fixture.h index 1bc573da..4bd6f1ea 100644 --- a/tests/Fixture.h +++ b/tests/Fixture.h @@ -128,6 +128,7 @@ struct Fixture std::optional lookupImportedType(const std::string& moduleAlias, const std::string& name); ScopedFastFlag sff_DebugLuauFreezeArena; + ScopedFastFlag sff_UnknownNever{"LuauUnknownAndNeverType", true}; TestFileResolver fileResolver; TestConfigResolver configResolver; diff --git a/tests/Frontend.test.cpp b/tests/Frontend.test.cpp index b9c24704..4f331399 100644 --- a/tests/Frontend.test.cpp +++ b/tests/Frontend.test.cpp @@ -791,6 +791,8 @@ TEST_CASE_FIXTURE(FrontendFixture, "discard_type_graphs") CHECK_EQ(0, module->internalTypes.typeVars.size()); CHECK_EQ(0, module->internalTypes.typePacks.size()); CHECK_EQ(0, module->astTypes.size()); + CHECK_EQ(0, module->astResolvedTypes.size()); + CHECK_EQ(0, module->astResolvedTypePacks.size()); } TEST_CASE_FIXTURE(FrontendFixture, "it_should_be_safe_to_stringify_errors_when_full_type_graph_is_discarded") diff --git a/tests/Module.test.cpp b/tests/Module.test.cpp index 7c2f4d1c..dd94e9d7 100644 --- a/tests/Module.test.cpp +++ b/tests/Module.test.cpp @@ -301,8 +301,6 @@ TEST_CASE_FIXTURE(Fixture, "clone_recursion_limit") TEST_CASE_FIXTURE(Fixture, "any_persistance_does_not_leak") { - ScopedFastFlag luauNonCopyableTypeVarFields{"LuauNonCopyableTypeVarFields", true}; - fileResolver.source["Module/A"] = R"( export type A = B type B = A diff --git a/tests/Normalize.test.cpp b/tests/Normalize.test.cpp index a474b6e7..fb0a899e 100644 --- a/tests/Normalize.test.cpp +++ b/tests/Normalize.test.cpp @@ -1055,8 +1055,6 @@ export type t1 = { a: typeof(string.byte) } TEST_CASE_FIXTURE(Fixture, "intersection_combine_on_bound_self") { - ScopedFastFlag luauNormalizeCombineEqFix{"LuauNormalizeCombineEqFix", true}; - CheckResult result = check(R"( export type t0 = (((any)&({_:l0.t0,n0:t0,_G:any,}))&({_:any,}))&(((any)&({_:l0.t0,n0:t0,_G:any,}))&({_:any,})) )"); @@ -1064,6 +1062,46 @@ export type t0 = (((any)&({_:l0.t0,n0:t0,_G:any,}))&({_:any,}))&(((any)&({_:l0.t LUAU_REQUIRE_ERRORS(result); } +TEST_CASE_FIXTURE(Fixture, "normalize_unions_containing_never") +{ + ScopedFastFlag sff{"LuauLowerBoundsCalculation", true}; + + CheckResult result = check(R"( + type Foo = string | never + local foo: Foo + )"); + + CHECK_EQ("string", toString(requireType("foo"))); +} + +TEST_CASE_FIXTURE(Fixture, "normalize_unions_containing_unknown") +{ + ScopedFastFlag sff{"LuauLowerBoundsCalculation", true}; + + CheckResult result = check(R"( + type Foo = string | unknown + local foo: Foo + )"); + + CHECK_EQ("unknown", toString(requireType("foo"))); +} + +TEST_CASE_FIXTURE(Fixture, "any_wins_the_battle_over_unknown_in_unions") +{ + ScopedFastFlag sff{"LuauLowerBoundsCalculation", true}; + + CheckResult result = check(R"( + type Foo = unknown | any + local foo: Foo + + type Bar = any | unknown + local bar: Bar + )"); + + CHECK_EQ("any", toString(requireType("foo"))); + CHECK_EQ("any", toString(requireType("bar"))); +} + TEST_CASE_FIXTURE(BuiltinsFixture, "normalization_does_not_convert_ever") { ScopedFastFlag sff[]{ diff --git a/tests/Parser.test.cpp b/tests/Parser.test.cpp index c3c75998..c517853f 100644 --- a/tests/Parser.test.cpp +++ b/tests/Parser.test.cpp @@ -2648,7 +2648,6 @@ type Z = { a: string | T..., b: number } TEST_CASE_FIXTURE(Fixture, "recover_function_return_type_annotations") { - ScopedFastFlag sff{"LuauReturnTypeTokenConfusion", true}; ParseResult result = tryParse(R"( type Custom = { x: A, y: B, z: C } type Packed = { x: (A...) -> () } diff --git a/tests/ToString.test.cpp b/tests/ToString.test.cpp index 387e07cd..52a29bcc 100644 --- a/tests/ToString.test.cpp +++ b/tests/ToString.test.cpp @@ -96,6 +96,37 @@ TEST_CASE_FIXTURE(Fixture, "table_respects_use_line_break") //clang-format on } +TEST_CASE_FIXTURE(Fixture, "metatable") +{ + TypeVar table{TypeVariant(TableTypeVar())}; + TypeVar metatable{TypeVariant(TableTypeVar())}; + TypeVar mtv{TypeVariant(MetatableTypeVar{&table, &metatable})}; + CHECK_EQ("{ @metatable { }, { } }", toString(&mtv)); +} + +TEST_CASE_FIXTURE(Fixture, "named_metatable") +{ + TypeVar table{TypeVariant(TableTypeVar())}; + TypeVar metatable{TypeVariant(TableTypeVar())}; + TypeVar mtv{TypeVariant(MetatableTypeVar{&table, &metatable, "NamedMetatable"})}; + CHECK_EQ("NamedMetatable", toString(&mtv)); +} + +TEST_CASE_FIXTURE(BuiltinsFixture, "named_metatable_toStringNamedFunction") +{ + CheckResult result = check(R"( + local function createTbl(): NamedMetatable + return setmetatable({}, {}) + end + type NamedMetatable = typeof(createTbl()) + )"); + + TypeId ty = requireType("createTbl"); + const FunctionTypeVar* ftv = get(follow(ty)); + REQUIRE(ftv); + CHECK_EQ("createTbl(): NamedMetatable", toStringNamedFunction("createTbl", *ftv)); +} + TEST_CASE_FIXTURE(BuiltinsFixture, "exhaustive_toString_of_cyclic_table") { CheckResult result = check(R"( @@ -468,7 +499,7 @@ local function target(callback: nil) return callback(4, "hello") end )"); LUAU_REQUIRE_ERRORS(result); - CHECK_EQ("(nil) -> (*unknown*)", toString(requireType("target"))); + CHECK_EQ("(nil) -> ()", toString(requireType("target"))); } TEST_CASE_FIXTURE(Fixture, "toStringGenericPack") diff --git a/tests/Transpiler.test.cpp b/tests/Transpiler.test.cpp index b02a52b2..d2ed9aef 100644 --- a/tests/Transpiler.test.cpp +++ b/tests/Transpiler.test.cpp @@ -583,7 +583,7 @@ TEST_CASE_FIXTURE(Fixture, "transpile_error_expr") auto names = AstNameTable{allocator}; ParseResult parseResult = Parser::parse(code.data(), code.size(), names, allocator, {}); - CHECK_EQ("local a = (error-expr: f.%error-id%)-(error-expr)", transpileWithTypes(*parseResult.root)); + CHECK_EQ("local a = (error-expr: f:%error-id%)-(error-expr)", transpileWithTypes(*parseResult.root)); } TEST_CASE_FIXTURE(Fixture, "transpile_error_stat") diff --git a/tests/TypeInfer.anyerror.test.cpp b/tests/TypeInfer.anyerror.test.cpp index bc55940e..4c5309e0 100644 --- a/tests/TypeInfer.anyerror.test.cpp +++ b/tests/TypeInfer.anyerror.test.cpp @@ -94,7 +94,7 @@ TEST_CASE_FIXTURE(Fixture, "for_in_loop_iterator_is_error") LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK_EQ("*unknown*", toString(requireType("a"))); + CHECK_EQ("", toString(requireType("a"))); } TEST_CASE_FIXTURE(Fixture, "for_in_loop_iterator_is_error2") @@ -110,7 +110,7 @@ TEST_CASE_FIXTURE(Fixture, "for_in_loop_iterator_is_error2") LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK_EQ("*unknown*", toString(requireType("a"))); + CHECK_EQ("", toString(requireType("a"))); } TEST_CASE_FIXTURE(Fixture, "length_of_error_type_does_not_produce_an_error") @@ -225,7 +225,7 @@ TEST_CASE_FIXTURE(Fixture, "calling_error_type_yields_error") CHECK_EQ("unknown", err->name); - CHECK_EQ("*unknown*", toString(requireType("a"))); + CHECK_EQ("", toString(requireType("a"))); } TEST_CASE_FIXTURE(Fixture, "chain_calling_error_type_yields_error") @@ -234,7 +234,7 @@ TEST_CASE_FIXTURE(Fixture, "chain_calling_error_type_yields_error") local a = Utility.Create "Foo" {} )"); - CHECK_EQ("*unknown*", toString(requireType("a"))); + CHECK_EQ("", toString(requireType("a"))); } TEST_CASE_FIXTURE(BuiltinsFixture, "replace_every_free_type_when_unifying_a_complex_function_with_any") diff --git a/tests/TypeInfer.builtins.test.cpp b/tests/TypeInfer.builtins.test.cpp index 2f0266ec..7d1bd6ba 100644 --- a/tests/TypeInfer.builtins.test.cpp +++ b/tests/TypeInfer.builtins.test.cpp @@ -925,7 +925,7 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "assert_returns_false_and_string_iff_it_knows )"); LUAU_REQUIRE_NO_ERRORS(result); - CHECK_EQ("(nil) -> nil", toString(requireType("f"))); + CHECK_EQ("(nil) -> (never, ...never)", toString(requireType("f"))); } TEST_CASE_FIXTURE(BuiltinsFixture, "table_freeze_is_generic") @@ -952,7 +952,7 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "table_freeze_is_generic") CHECK_EQ("number", toString(requireType("a"))); CHECK_EQ("string", toString(requireType("b"))); CHECK_EQ("boolean", toString(requireType("c"))); - CHECK_EQ("*unknown*", toString(requireType("d"))); + CHECK_EQ("", toString(requireType("d"))); } TEST_CASE_FIXTURE(BuiltinsFixture, "set_metatable_needs_arguments") @@ -965,8 +965,8 @@ a:b() a:b({}) )"); LUAU_REQUIRE_ERROR_COUNT(2, result); - CHECK_EQ(result.errors[0], (TypeError{Location{{2, 0}, {2, 5}}, CountMismatch{2, 0}})); - CHECK_EQ(result.errors[1], (TypeError{Location{{3, 0}, {3, 5}}, CountMismatch{2, 1}})); + CHECK_EQ(toString(result.errors[0]), "Argument count mismatch. Function expects 2 arguments, but none are specified"); + CHECK_EQ(toString(result.errors[1]), "Argument count mismatch. Function expects 2 arguments, but only 1 is specified"); } TEST_CASE_FIXTURE(Fixture, "typeof_unresolved_function") @@ -1008,4 +1008,139 @@ end LUAU_REQUIRE_NO_ERRORS(result); } +TEST_CASE_FIXTURE(BuiltinsFixture, "gmatch_capture_types") +{ + ScopedFastFlag sffs{"LuauDeduceGmatchReturnTypes", true}; + CheckResult result = check(R"END( + local a, b, c = string.gmatch("This is a string", "(.()(%a+))")() + )END"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ(toString(requireType("a")), "string"); + CHECK_EQ(toString(requireType("b")), "number"); + CHECK_EQ(toString(requireType("c")), "string"); +} + +TEST_CASE_FIXTURE(BuiltinsFixture, "gmatch_capture_types2") +{ + ScopedFastFlag sffs{"LuauDeduceGmatchReturnTypes", true}; + CheckResult result = check(R"END( + local a, b, c = ("This is a string"):gmatch("(.()(%a+))")() + )END"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ(toString(requireType("a")), "string"); + CHECK_EQ(toString(requireType("b")), "number"); + CHECK_EQ(toString(requireType("c")), "string"); +} + +TEST_CASE_FIXTURE(BuiltinsFixture, "gmatch_capture_types_default_capture") +{ + ScopedFastFlag sffs{"LuauDeduceGmatchReturnTypes", true}; + CheckResult result = check(R"END( + local a, b, c, d = string.gmatch("T(his)() is a string", ".")() + )END"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + CountMismatch* acm = get(result.errors[0]); + REQUIRE(acm); + CHECK_EQ(acm->context, CountMismatch::Result); + CHECK_EQ(acm->expected, 1); + CHECK_EQ(acm->actual, 4); + + CHECK_EQ(toString(requireType("a")), "string"); +} + +TEST_CASE_FIXTURE(BuiltinsFixture, "gmatch_capture_types_balanced_escaped_parens") +{ + ScopedFastFlag sffs{"LuauDeduceGmatchReturnTypes", true}; + CheckResult result = check(R"END( + local a, b, c, d = string.gmatch("T(his) is a string", "((.)%b()())")() + )END"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + CountMismatch* acm = get(result.errors[0]); + REQUIRE(acm); + CHECK_EQ(acm->context, CountMismatch::Result); + CHECK_EQ(acm->expected, 3); + CHECK_EQ(acm->actual, 4); + + CHECK_EQ(toString(requireType("a")), "string"); + CHECK_EQ(toString(requireType("b")), "string"); + CHECK_EQ(toString(requireType("c")), "number"); +} + +TEST_CASE_FIXTURE(BuiltinsFixture, "gmatch_capture_types_parens_in_sets_are_ignored") +{ + ScopedFastFlag sffs{"LuauDeduceGmatchReturnTypes", true}; + CheckResult result = check(R"END( + local a, b, c = string.gmatch("T(his)() is a string", "(T[()])()")() + )END"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + CountMismatch* acm = get(result.errors[0]); + REQUIRE(acm); + CHECK_EQ(acm->context, CountMismatch::Result); + CHECK_EQ(acm->expected, 2); + CHECK_EQ(acm->actual, 3); + + CHECK_EQ(toString(requireType("a")), "string"); + CHECK_EQ(toString(requireType("b")), "number"); +} + +TEST_CASE_FIXTURE(BuiltinsFixture, "gmatch_capture_types_set_containing_lbracket") +{ + ScopedFastFlag sffs{"LuauDeduceGmatchReturnTypes", true}; + CheckResult result = check(R"END( + local a, b = string.gmatch("[[[", "()([[])")() + )END"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ(toString(requireType("a")), "number"); + CHECK_EQ(toString(requireType("b")), "string"); +} + +TEST_CASE_FIXTURE(BuiltinsFixture, "gmatch_capture_types_leading_end_bracket_is_part_of_set") +{ + CheckResult result = check(R"END( + -- An immediate right-bracket following a left-bracket is included within the set; + -- thus, '[]]'' is the set containing ']', and '[]' is an invalid set missing an enclosing + -- right-bracket. We detect an invalid set in this case and fall back to to default gmatch + -- typing. + local foo = string.gmatch("T[hi%]s]]]() is a string", "([]s)") + )END"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ(toString(requireType("foo")), "() -> (...string)"); +} + +TEST_CASE_FIXTURE(BuiltinsFixture, "gmatch_capture_types_invalid_pattern_fallback_to_builtin") +{ + CheckResult result = check(R"END( + local foo = string.gmatch("T(his)() is a string", ")") + )END"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ(toString(requireType("foo")), "() -> (...string)"); +} + +TEST_CASE_FIXTURE(BuiltinsFixture, "gmatch_capture_types_invalid_pattern_fallback_to_builtin2") +{ + CheckResult result = check(R"END( + local foo = string.gmatch("T(his)() is a string", "[") + )END"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ(toString(requireType("foo")), "() -> (...string)"); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.functions.test.cpp b/tests/TypeInfer.functions.test.cpp index 401a6c64..6e6549d3 100644 --- a/tests/TypeInfer.functions.test.cpp +++ b/tests/TypeInfer.functions.test.cpp @@ -916,13 +916,13 @@ TEST_CASE_FIXTURE(Fixture, "function_cast_error_uses_correct_language") REQUIRE(tm1); CHECK_EQ("(string) -> number", toString(tm1->wantedType)); - CHECK_EQ("(string, *unknown*) -> number", toString(tm1->givenType)); + CHECK_EQ("(string, ) -> number", toString(tm1->givenType)); auto tm2 = get(result.errors[1]); REQUIRE(tm2); CHECK_EQ("(number, number) -> (number, number)", toString(tm2->wantedType)); - CHECK_EQ("(string, *unknown*) -> number", toString(tm2->givenType)); + CHECK_EQ("(string, ) -> number", toString(tm2->givenType)); } TEST_CASE_FIXTURE(Fixture, "no_lossy_function_type") @@ -1535,7 +1535,7 @@ function t:b() return 2 end -- not OK )"); LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK_EQ(R"(Type '(*unknown*) -> number' could not be converted into '() -> number' + CHECK_EQ(R"(Type '() -> number' could not be converted into '() -> number' caused by: Argument count mismatch. Function expects 1 argument, but none are specified)", toString(result.errors[0])); @@ -1692,4 +1692,52 @@ TEST_CASE_FIXTURE(Fixture, "call_o_with_another_argument_after_foo_was_quantifie // TODO: check the normalized type of f } +TEST_CASE_FIXTURE(Fixture, "free_is_not_bound_to_unknown") +{ + CheckResult result = check(R"( + local function foo(f: (unknown) -> (), x) + f(x) + end + )"); + + CHECK_EQ("((unknown) -> (), a) -> ()", toString(requireType("foo"))); +} + +TEST_CASE_FIXTURE(Fixture, "dont_infer_parameter_types_for_functions_from_their_call_site") +{ + CheckResult result = check(R"( + local t = {} + + function t.f(x) + return x + end + + t.__index = t + + function g(s) + local q = s.p and s.p.q or nil + return q and t.f(q) or nil + end + + local f = t.f + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ("(a) -> a", toString(requireType("f"))); +} + +TEST_CASE_FIXTURE(Fixture, "dont_mutate_the_underlying_head_of_typepack_when_calling_with_self") +{ + CheckResult result = check(R"( + local t = {} + function t:m(x) end + function f(): never return 5 :: never end + t:m(f()) + t:m(f()) + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.generics.test.cpp b/tests/TypeInfer.generics.test.cpp index e9e94cfb..46258072 100644 --- a/tests/TypeInfer.generics.test.cpp +++ b/tests/TypeInfer.generics.test.cpp @@ -9,6 +9,8 @@ #include "doctest.h" +LUAU_FASTFLAG(LuauCheckGenericHOFTypes) + using namespace Luau; TEST_SUITE_BEGIN("GenericsTests"); @@ -1001,7 +1003,7 @@ TEST_CASE_FIXTURE(Fixture, "no_stack_overflow_from_quantifying") std::optional t0 = getMainModule()->getModuleScope()->lookupType("t0"); REQUIRE(t0); - CHECK_EQ("*unknown*", toString(t0->type)); + CHECK_EQ("", toString(t0->type)); auto it = std::find_if(result.errors.begin(), result.errors.end(), [](TypeError& err) { return get(err); @@ -1095,10 +1097,18 @@ local b = sumrec(sum) -- ok local c = sumrec(function(x, y, f) return f(x, y) end) -- type binders are not inferred )"); - LUAU_REQUIRE_ERRORS(result); - CHECK_EQ("Type '(a, b, (a, b) -> (c...)) -> (c...)' could not be converted into '(a, a, (a, a) -> a) -> a'; different number of generic type " - "parameters", - toString(result.errors[0])); + if (FFlag::LuauCheckGenericHOFTypes) + { + LUAU_REQUIRE_NO_ERRORS(result); + } + else + { + LUAU_REQUIRE_ERRORS(result); + CHECK_EQ( + "Type '(a, b, (a, b) -> (c...)) -> (c...)' could not be converted into '(a, a, (a, a) -> a) -> a'; different number of generic type " + "parameters", + toString(result.errors[0])); + } } TEST_CASE_FIXTURE(Fixture, "substitution_with_bound_table") @@ -1185,4 +1195,23 @@ TEST_CASE_FIXTURE(Fixture, "quantify_functions_even_if_they_have_an_explicit_gen CHECK("((X) -> (a...), X) -> (a...)" == toString(requireType("foo"))); } +TEST_CASE_FIXTURE(Fixture, "do_not_always_instantiate_generic_intersection_types") +{ + ScopedFastFlag sff[] = { + {"LuauMaybeGenericIntersectionTypes", true}, + }; + + CheckResult result = check(R"( + --!strict + type Array = { [number]: T } + + type Array_Statics = { + new: () -> Array, + } + + local _Arr : Array & Array_Statics = {} :: Array_Statics + )"); + LUAU_REQUIRE_NO_ERRORS(result); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.loops.test.cpp b/tests/TypeInfer.loops.test.cpp index 1c6fe1d8..56b807f8 100644 --- a/tests/TypeInfer.loops.test.cpp +++ b/tests/TypeInfer.loops.test.cpp @@ -142,7 +142,7 @@ TEST_CASE_FIXTURE(Fixture, "for_in_loop_on_error") CHECK_EQ(2, result.errors.size()); TypeId p = requireType("p"); - CHECK_EQ("*unknown*", toString(p)); + CHECK_EQ("", toString(p)); } TEST_CASE_FIXTURE(Fixture, "for_in_loop_on_non_function") diff --git a/tests/TypeInfer.modules.test.cpp b/tests/TypeInfer.modules.test.cpp index a0f670f1..2343a7fa 100644 --- a/tests/TypeInfer.modules.test.cpp +++ b/tests/TypeInfer.modules.test.cpp @@ -143,7 +143,7 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "require_module_that_does_not_export") auto hootyType = requireType(bModule, "Hooty"); - CHECK_EQ("*unknown*", toString(hootyType)); + CHECK_EQ("", toString(hootyType)); } TEST_CASE_FIXTURE(BuiltinsFixture, "warn_if_you_try_to_require_a_non_modulescript") @@ -244,7 +244,7 @@ local ModuleA = require(game.A) LUAU_REQUIRE_NO_ERRORS(result); std::optional oty = requireType("ModuleA"); - CHECK_EQ("*unknown*", toString(*oty)); + CHECK_EQ("", toString(*oty)); } TEST_CASE_FIXTURE(BuiltinsFixture, "do_not_modify_imported_types") @@ -302,6 +302,30 @@ type Rename = typeof(x.x) LUAU_REQUIRE_NO_ERRORS(result); } +TEST_CASE_FIXTURE(BuiltinsFixture, "do_not_modify_imported_types_4") +{ + fileResolver.source["game/A"] = R"( +export type Array = {T} +local arrayops = {} +function arrayops.foo(x: Array) end +return arrayops + )"; + + CheckResult result = check(R"( +local arrayops = require(game.A) + +local tbl = {} +tbl.a = 2 +function tbl:foo(b: number, c: number) + -- introduce BoundTypeVar to imported type + arrayops.foo(self._regions) +end +type Table = typeof(tbl) +)"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + TEST_CASE_FIXTURE(BuiltinsFixture, "module_type_conflict") { fileResolver.source["game/A"] = R"( @@ -363,4 +387,21 @@ caused by: Property 'x' is not compatible. Type 'number' could not be converted into 'string')"); } +TEST_CASE_FIXTURE(BuiltinsFixture, "constrained_anyification_clone_immutable_types") +{ + ScopedFastFlag luauAnyificationMustClone{"LuauAnyificationMustClone", true}; + + fileResolver.source["game/A"] = R"( +return function(...) end + )"; + + fileResolver.source["game/B"] = R"( +local l0 = require(game.A) +return l0 + )"; + + CheckResult result = frontend.check("game/B"); + LUAU_REQUIRE_NO_ERRORS(result); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.operators.test.cpp b/tests/TypeInfer.operators.test.cpp index e6174df2..c90f0a4d 100644 --- a/tests/TypeInfer.operators.test.cpp +++ b/tests/TypeInfer.operators.test.cpp @@ -871,4 +871,26 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "equality_operations_succeed_if_any_union_bra CHECK(toString(result2.errors[0]) == "Types Foo and Bar cannot be compared with == because they do not have the same metatable"); } +TEST_CASE_FIXTURE(BuiltinsFixture, "expected_types_through_binary_and") +{ + ScopedFastFlag sff{"LuauBinaryNeedsExpectedTypesToo", true}; + + CheckResult result = check(R"( + local x: "a" | "b" | boolean = math.random() > 0.5 and "a" + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(BuiltinsFixture, "expected_types_through_binary_or") +{ + ScopedFastFlag sff{"LuauBinaryNeedsExpectedTypesToo", true}; + + CheckResult result = check(R"( + local x: "a" | "b" | boolean = math.random() > 0.5 or "b" + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.primitives.test.cpp b/tests/TypeInfer.primitives.test.cpp index e1684df7..9e8e2503 100644 --- a/tests/TypeInfer.primitives.test.cpp +++ b/tests/TypeInfer.primitives.test.cpp @@ -47,7 +47,7 @@ TEST_CASE_FIXTURE(Fixture, "string_index") REQUIRE(nat); CHECK_EQ("string", toString(nat->ty)); - CHECK_EQ("*unknown*", toString(requireType("t"))); + CHECK_EQ("", toString(requireType("t"))); } TEST_CASE_FIXTURE(Fixture, "string_method") diff --git a/tests/TypeInfer.provisional.test.cpp b/tests/TypeInfer.provisional.test.cpp index 059aed2e..dc686890 100644 --- a/tests/TypeInfer.provisional.test.cpp +++ b/tests/TypeInfer.provisional.test.cpp @@ -225,7 +225,7 @@ TEST_CASE_FIXTURE(Fixture, "discriminate_from_x_not_equal_to_nil") CHECK_EQ("{| x: nil, y: nil |} | {| x: string, y: number |}", toString(requireTypeAtPosition({7, 28}))); } -TEST_CASE_FIXTURE(Fixture, "bail_early_if_unification_is_too_complicated" * doctest::timeout(0.5)) +TEST_CASE_FIXTURE(BuiltinsFixture, "bail_early_if_unification_is_too_complicated" * doctest::timeout(0.5)) { ScopedFastInt sffi{"LuauTarjanChildLimit", 1}; ScopedFastInt sffi2{"LuauTypeInferIterationLimit", 1}; @@ -499,6 +499,17 @@ TEST_CASE_FIXTURE(Fixture, "constrained_is_level_dependent") CHECK_EQ("(t1) -> {| [t1]: boolean |} where t1 = t2 ; t2 = {+ m1: (t1) -> (a...), m2: (t2) -> (b...) +}", toString(requireType("f"))); } +TEST_CASE_FIXTURE(Fixture, "free_is_not_bound_to_any") +{ + CheckResult result = check(R"( + local function foo(f: (any) -> (), x) + f(x) + end + )"); + + CHECK_EQ("((any) -> (), any) -> ()", toString(requireType("foo"))); +} + TEST_CASE_FIXTURE(BuiltinsFixture, "greedy_inference_with_shared_self_triggers_function_with_no_returns") { ScopedFastFlag sff{"DebugLuauSharedSelf", true}; @@ -518,7 +529,7 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "greedy_inference_with_shared_self_triggers_f )"); LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK_EQ("Not all codepaths in this function return '{ @metatable T, {| |} }, a...'.", toString(result.errors[0])); + CHECK_EQ("Not all codepaths in this function return 'self, a...'.", toString(result.errors[0])); } TEST_SUITE_END(); diff --git a/tests/TypeInfer.refinements.test.cpp b/tests/TypeInfer.refinements.test.cpp index 3f5dad3d..cc8cdee3 100644 --- a/tests/TypeInfer.refinements.test.cpp +++ b/tests/TypeInfer.refinements.test.cpp @@ -272,8 +272,8 @@ TEST_CASE_FIXTURE(Fixture, "typeguard_only_look_up_types_from_global_scope") LUAU_REQUIRE_NO_ERRORS(result); - CHECK_EQ("*unknown*", toString(requireTypeAtPosition({8, 44}))); - CHECK_EQ("*unknown*", toString(requireTypeAtPosition({9, 38}))); + CHECK_EQ("never", toString(requireTypeAtPosition({8, 44}))); + CHECK_EQ("never", toString(requireTypeAtPosition({9, 38}))); } TEST_CASE_FIXTURE(Fixture, "call_a_more_specific_function_using_typeguard") @@ -526,7 +526,7 @@ TEST_CASE_FIXTURE(Fixture, "type_narrow_to_vector") LUAU_REQUIRE_NO_ERRORS(result); - CHECK_EQ("*unknown*", toString(requireTypeAtPosition({3, 28}))); + CHECK_EQ("", toString(requireTypeAtPosition({3, 28}))); } TEST_CASE_FIXTURE(Fixture, "nonoptional_type_can_narrow_to_nil_if_sense_is_true") @@ -651,7 +651,7 @@ TEST_CASE_FIXTURE(Fixture, "type_guard_can_filter_for_overloaded_function") CHECK_EQ("nil", toString(requireTypeAtPosition({6, 28}))); } -TEST_CASE_FIXTURE(BuiltinsFixture, "type_guard_warns_on_no_overlapping_types_only_when_sense_is_true") +TEST_CASE_FIXTURE(BuiltinsFixture, "type_guard_narrowed_into_nothingness") { CheckResult result = check(R"( local function f(t: {x: number}) @@ -666,7 +666,7 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "type_guard_warns_on_no_overlapping_types_onl LUAU_REQUIRE_NO_ERRORS(result); - CHECK_EQ("*unknown*", toString(requireTypeAtPosition({3, 28}))); + CHECK_EQ("never", toString(requireTypeAtPosition({3, 28}))); } TEST_CASE_FIXTURE(Fixture, "not_a_or_not_b") @@ -1074,7 +1074,7 @@ TEST_CASE_FIXTURE(RefinementClassFixture, "typeguard_cast_free_table_to_vector") CHECK_EQ("Vector3", toString(requireTypeAtPosition({5, 28}))); // type(vec) == "vector" - CHECK_EQ("*unknown*", toString(requireTypeAtPosition({7, 28}))); // typeof(vec) == "Instance" + CHECK_EQ("never", toString(requireTypeAtPosition({7, 28}))); // typeof(vec) == "Instance" CHECK_EQ("{+ X: a, Y: b, Z: c +}", toString(requireTypeAtPosition({9, 28}))); // type(vec) ~= "vector" and typeof(vec) ~= "Instance" } @@ -1206,6 +1206,24 @@ TEST_CASE_FIXTURE(Fixture, "typeguard_doesnt_leak_to_elseif") LUAU_REQUIRE_NO_ERRORS(result); } +TEST_CASE_FIXTURE(BuiltinsFixture, "refine_unknowns") +{ + CheckResult result = check(R"( + local function f(x: unknown) + if type(x) == "string" then + local foo = x + else + local bar = x + end + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ("string", toString(requireTypeAtPosition({3, 28}))); + CHECK_EQ("unknown", toString(requireTypeAtPosition({5, 28}))); +} + TEST_CASE_FIXTURE(BuiltinsFixture, "falsiness_of_TruthyPredicate_narrows_into_nil") { ScopedFastFlag sff{"LuauFalsyPredicateReturnsNilInstead", true}; @@ -1227,4 +1245,19 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "falsiness_of_TruthyPredicate_narrows_into_ni CHECK_EQ("number", toString(requireTypeAtPosition({6, 28}))); } +TEST_CASE_FIXTURE(BuiltinsFixture, "what_nonsensical_condition") +{ + CheckResult result = check(R"( + local function f(x) + if type(x) == "string" and type(x) == "number" then + local foo = x + end + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ("never", toString(requireTypeAtPosition({3, 28}))); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.tables.test.cpp b/tests/TypeInfer.tables.test.cpp index eead5b30..3e830f2a 100644 --- a/tests/TypeInfer.tables.test.cpp +++ b/tests/TypeInfer.tables.test.cpp @@ -3070,4 +3070,18 @@ TEST_CASE_FIXTURE(Fixture, "quantify_even_that_table_was_never_exported_at_all") CHECK_EQ("{| m: ({+ x: a, y: b +}) -> a, n: ({+ x: a, y: b +}) -> b |}", toString(requireType("T"), opts)); } +TEST_CASE_FIXTURE(BuiltinsFixture, "leaking_bad_metatable_errors") +{ + ScopedFastFlag luauIndexSilenceErrors{"LuauIndexSilenceErrors", true}; + + CheckResult result = check(R"( +local a = setmetatable({}, 1) +local b = a.x + )"); + + LUAU_REQUIRE_ERROR_COUNT(2, result); + CHECK_EQ("Metatable was not a table", toString(result.errors[0])); + CHECK_EQ("Type 'a' does not have key 'x'", toString(result.errors[1])); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.test.cpp b/tests/TypeInfer.test.cpp index efdfe0b1..7d1fb56b 100644 --- a/tests/TypeInfer.test.cpp +++ b/tests/TypeInfer.test.cpp @@ -238,10 +238,10 @@ TEST_CASE_FIXTURE(Fixture, "type_errors_infer_types") // TODO: Should we assert anything about these tests when DCR is being used? if (!FFlag::DebugLuauDeferredConstraintResolution) { - CHECK_EQ("*unknown*", toString(requireType("c"))); - CHECK_EQ("*unknown*", toString(requireType("d"))); - CHECK_EQ("*unknown*", toString(requireType("e"))); - CHECK_EQ("*unknown*", toString(requireType("f"))); + CHECK_EQ("", toString(requireType("c"))); + CHECK_EQ("", toString(requireType("d"))); + CHECK_EQ("", toString(requireType("e"))); + CHECK_EQ("", toString(requireType("f"))); } } @@ -622,7 +622,7 @@ TEST_CASE_FIXTURE(Fixture, "no_stack_overflow_from_isoptional") std::optional t0 = getMainModule()->getModuleScope()->lookupType("t0"); REQUIRE(t0); - CHECK_EQ("*unknown*", toString(t0->type)); + CHECK_EQ("", toString(t0->type)); auto it = std::find_if(result.errors.begin(), result.errors.end(), [](TypeError& err) { return get(err); @@ -1003,4 +1003,27 @@ TEST_CASE_FIXTURE(Fixture, "do_not_bind_a_free_table_to_a_union_containing_that_ )"); } +TEST_CASE_FIXTURE(Fixture, "types stored in astResolvedTypes") +{ + CheckResult result = check(R"( +type alias = typeof("hello") +local function foo(param: alias) +end + )"); + + auto node = findNodeAtPosition(*getMainSourceModule(), {2, 16}); + auto ty = lookupType("alias"); + REQUIRE(node); + REQUIRE(node->is()); + REQUIRE(ty); + + auto func = node->as(); + REQUIRE(func->args.size == 1); + + auto arg = *func->args.begin(); + auto annotation = arg->annotation; + + CHECK_EQ(*getMainModule()->astResolvedTypes.find(annotation), *ty); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.tryUnify.test.cpp b/tests/TypeInfer.tryUnify.test.cpp index 49deae71..d51a38f8 100644 --- a/tests/TypeInfer.tryUnify.test.cpp +++ b/tests/TypeInfer.tryUnify.test.cpp @@ -121,7 +121,7 @@ TEST_CASE_FIXTURE(TryUnifyFixture, "members_of_failed_typepack_unification_are_u LUAU_REQUIRE_ERROR_COUNT(1, result); CHECK_EQ("a", toString(requireType("a"))); - CHECK_EQ("*unknown*", toString(requireType("b"))); + CHECK_EQ("", toString(requireType("b"))); } TEST_CASE_FIXTURE(TryUnifyFixture, "result_of_failed_typepack_unification_is_constrained") @@ -136,7 +136,7 @@ TEST_CASE_FIXTURE(TryUnifyFixture, "result_of_failed_typepack_unification_is_con LUAU_REQUIRE_ERROR_COUNT(1, result); CHECK_EQ("a", toString(requireType("a"))); - CHECK_EQ("*unknown*", toString(requireType("b"))); + CHECK_EQ("", toString(requireType("b"))); CHECK_EQ("number", toString(requireType("c"))); } diff --git a/tests/TypeInfer.unionTypes.test.cpp b/tests/TypeInfer.unionTypes.test.cpp index 2b48133d..94918692 100644 --- a/tests/TypeInfer.unionTypes.test.cpp +++ b/tests/TypeInfer.unionTypes.test.cpp @@ -199,7 +199,7 @@ TEST_CASE_FIXTURE(Fixture, "index_on_a_union_type_with_missing_property") CHECK_EQ(mup->missing[0], *bTy); CHECK_EQ(mup->key, "x"); - CHECK_EQ("*unknown*", toString(requireType("r"))); + CHECK_EQ("", toString(requireType("r"))); } TEST_CASE_FIXTURE(Fixture, "index_on_a_union_type_with_one_property_of_type_any") diff --git a/tests/TypeInfer.unknownnever.test.cpp b/tests/TypeInfer.unknownnever.test.cpp new file mode 100644 index 00000000..bc742b03 --- /dev/null +++ b/tests/TypeInfer.unknownnever.test.cpp @@ -0,0 +1,280 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details + +#include "Fixture.h" + +#include "doctest.h" + +using namespace Luau; + +TEST_SUITE_BEGIN("TypeInferUnknownNever"); + +TEST_CASE_FIXTURE(Fixture, "string_subtype_and_unknown_supertype") +{ + CheckResult result = check(R"( + local function f(x: string) + local foo: unknown = x + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "unknown_subtype_and_string_supertype") +{ + CheckResult result = check(R"( + local function f(x: unknown) + local foo: string = x + end + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); +} + +TEST_CASE_FIXTURE(Fixture, "unknown_is_reflexive") +{ + CheckResult result = check(R"( + local function f(x: unknown) + local foo: unknown = x + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "string_subtype_and_never_supertype") +{ + CheckResult result = check(R"( + local function f(x: string) + local foo: never = x + end + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); +} + +TEST_CASE_FIXTURE(Fixture, "never_subtype_and_string_supertype") +{ + CheckResult result = check(R"( + local function f(x: never) + local foo: string = x + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "never_is_reflexive") +{ + CheckResult result = check(R"( + local function f(x: never) + local foo: never = x + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "unknown_is_optional_because_it_too_encompasses_nil") +{ + CheckResult result = check(R"( + local t: {x: unknown} = {} + )"); +} + +TEST_CASE_FIXTURE(Fixture, "table_with_prop_of_type_never_is_uninhabitable") +{ + CheckResult result = check(R"( + local t: {x: never} = {} + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); +} + +TEST_CASE_FIXTURE(Fixture, "table_with_prop_of_type_never_is_also_reflexive") +{ + CheckResult result = check(R"( + local t: {x: never} = {x = 5 :: never} + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "array_like_table_of_never_is_inhabitable") +{ + CheckResult result = check(R"( + local t: {never} = {} + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "type_packs_containing_never_is_itself_uninhabitable") +{ + CheckResult result = check(R"( + local function f() return "foo", 5 :: never end + + local x, y, z = f() + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ("never", toString(requireType("x"))); + CHECK_EQ("never", toString(requireType("y"))); + CHECK_EQ("never", toString(requireType("z"))); +} + +TEST_CASE_FIXTURE(Fixture, "type_packs_containing_never_is_itself_uninhabitable2") +{ + CheckResult result = check(R"( + local function f(): (string, never) return "", 5 :: never end + local function g(): (never, string) return 5 :: never, "" end + + local x1, x2 = f() + local y1, y2 = g() + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ("never", toString(requireType("x1"))); + CHECK_EQ("never", toString(requireType("x2"))); + CHECK_EQ("never", toString(requireType("y1"))); + CHECK_EQ("never", toString(requireType("y2"))); +} + +TEST_CASE_FIXTURE(Fixture, "index_on_never") +{ + CheckResult result = check(R"( + local x: never = 5 :: never + local z = x.y + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ("never", toString(requireType("z"))); +} + +TEST_CASE_FIXTURE(Fixture, "call_never") +{ + CheckResult result = check(R"( + local f: never = 5 :: never + local x, y, z = f() + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ("never", toString(requireType("x"))); + CHECK_EQ("never", toString(requireType("y"))); + CHECK_EQ("never", toString(requireType("z"))); +} + +TEST_CASE_FIXTURE(Fixture, "assign_to_local_which_is_never") +{ + CheckResult result = check(R"( + local t: never + t = 3 + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "assign_to_global_which_is_never") +{ + CheckResult result = check(R"( + --!nonstrict + t = 5 :: never + t = "" + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "assign_to_prop_which_is_never") +{ + CheckResult result = check(R"( + local t: never + t.x = 5 + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "assign_to_subscript_which_is_never") +{ + CheckResult result = check(R"( + local t: never + t[5] = 7 + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "assign_to_subscript_which_is_never") +{ + CheckResult result = check(R"( + for i, v in (5 :: never) do + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "pick_never_from_variadic_type_pack") +{ + CheckResult result = check(R"( + local function f(...: never) + local x, y = (...) + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "index_on_union_of_tables_for_properties_that_is_never") +{ + CheckResult result = check(R"( + type Disjoint = {foo: never, bar: unknown, tag: "ok"} | {foo: never, baz: unknown, tag: "err"} + local disjoint: Disjoint = {foo = 5 :: never, bar = true, tag = "ok"} + local foo = disjoint.foo + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ("never", toString(requireType("foo"))); +} + +TEST_CASE_FIXTURE(Fixture, "index_on_union_of_tables_for_properties_that_is_sorta_never") +{ + CheckResult result = check(R"( + type Disjoint = {foo: string, bar: unknown, tag: "ok"} | {foo: never, baz: unknown, tag: "err"} + local disjoint: Disjoint = {foo = 5 :: never, bar = true, tag = "ok"} + local foo = disjoint.foo + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ("string", toString(requireType("foo"))); +} + +TEST_CASE_FIXTURE(Fixture, "unary_minus_of_never") +{ + CheckResult result = check(R"( + local x = -(5 :: never) + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ("never", toString(requireType("x"))); +} + +TEST_CASE_FIXTURE(Fixture, "length_of_never") +{ + CheckResult result = check(R"( + local x = #({} :: never) + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ("never", toString(requireType("x"))); +} + +TEST_SUITE_END(); diff --git a/tests/TypePack.test.cpp b/tests/TypePack.test.cpp index 8a5a65fe..35852c05 100644 --- a/tests/TypePack.test.cpp +++ b/tests/TypePack.test.cpp @@ -199,8 +199,6 @@ TEST_CASE_FIXTURE(TypePackFixture, "std_distance") TEST_CASE("content_reassignment") { - ScopedFastFlag luauNonCopyableTypeVarFields{"LuauNonCopyableTypeVarFields", true}; - TypePackVar myError{Unifiable::Error{}, /*presistent*/ true}; TypeArena arena; diff --git a/tests/TypeVar.test.cpp b/tests/TypeVar.test.cpp index 4f8fc502..f4670048 100644 --- a/tests/TypeVar.test.cpp +++ b/tests/TypeVar.test.cpp @@ -418,8 +418,6 @@ TEST_CASE("proof_that_isBoolean_uses_all_of") TEST_CASE("content_reassignment") { - ScopedFastFlag luauNonCopyableTypeVarFields{"LuauNonCopyableTypeVarFields", true}; - TypeVar myAny{AnyTypeVar{}, /*presistent*/ true}; myAny.normal = true; myAny.documentationSymbol = "@global/any"; diff --git a/tests/conformance/vector.lua b/tests/conformance/vector.lua index 22d6adfc..6164e929 100644 --- a/tests/conformance/vector.lua +++ b/tests/conformance/vector.lua @@ -101,4 +101,20 @@ if vector_size == 4 then assert(vector(1, 2, 3, 4).W == 4) end +-- negative zero should hash the same as zero +-- note: our earlier test only really checks the low hash bit, so in absence of perfect avalanche it's insufficient +do + local larget = {} + for i = 1, 2^14 do + larget[vector(0, 0, i)] = true + end + + larget[vector(0, 0, 0)] = 42 + + assert(larget[vector(0, 0, 0)] == 42) + assert(larget[vector(0, 0, -0)] == 42) + assert(larget[vector(0, -0, 0)] == 42) + assert(larget[vector(-0, 0, 0)] == 42) +end + return 'OK' diff --git a/tools/natvis/VM.natvis b/tools/natvis/VM.natvis index ccc7e390..cb2f355f 100644 --- a/tools/natvis/VM.natvis +++ b/tools/natvis/VM.natvis @@ -137,16 +137,28 @@ {data,s} + + + + + + + + + empty + none + + + {proto()->source->data,sb}:{line()} function {proto()->debugname->data,sb}() + {proto()->source->data,sb}:{line()} function() + + + =[C] function {cl().c.debugname,sb}() {cl().c.f,na} + =[C] {cl().c.f,na} + + - - {ci->func->value.gc->cl.c.f,na} - - - {ci->func->value.gc->cl.l.p->source->data,sb}:{ci->func->value.gc->cl.l.p->linedefined,d} {ci->func->value.gc->cl.l.p->debugname->data,sb} - - - {ci->func->value.gc->cl.l.p->source->data,sb}:{ci->func->value.gc->cl.l.p->linedefined,d} - + {ci,na} thread @@ -156,7 +168,7 @@ ci-base_ci - base_ci[ci-base_ci - $i].func->value.gc->cl,view(short) + base_ci[ci-base_ci - $i] From 506d9714218c10b4b8b0d18b2224c47fd9867983 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Thu, 7 Jul 2022 18:22:39 -0700 Subject: [PATCH 308/399] Sync to upstream/release/535 (#584) --- Analysis/include/Luau/AstQuery.h | 1 + Analysis/include/Luau/TypeInfer.h | 24 +- Analysis/include/Luau/TypePack.h | 1 + Analysis/include/Luau/TypeVar.h | 157 ++++++-- Analysis/include/Luau/VisitTypeVar.h | 8 + Analysis/src/AstQuery.cpp | 107 +++++- Analysis/src/Autocomplete.cpp | 201 +++------- Analysis/src/BuiltinDefinitions.cpp | 43 ++- Analysis/src/Clone.cpp | 12 + Analysis/src/EmbeddedBuiltinDefinitions.cpp | 9 +- Analysis/src/Normalize.cpp | 38 +- Analysis/src/Substitution.cpp | 10 +- Analysis/src/ToString.cpp | 17 +- Analysis/src/TxnLog.cpp | 59 +-- Analysis/src/TypeAttach.cpp | 8 + Analysis/src/TypeInfer.cpp | 393 ++++++++++++++++---- Analysis/src/TypePack.cpp | 51 ++- Analysis/src/TypeUtils.cpp | 2 +- Analysis/src/TypeVar.cpp | 269 +++++++++----- Analysis/src/Unifier.cpp | 66 ++-- Ast/src/Parser.cpp | 17 +- CodeGen/include/Luau/AssemblyBuilderX64.h | 3 + CodeGen/src/AssemblyBuilderX64.cpp | 23 ++ Sources.cmake | 8 +- VM/src/ltable.cpp | 8 +- VM/src/lvmexecute.cpp | 66 ++-- bench/bench.py | 14 +- bench/other/regex.lua | 2 +- tests/AssemblyBuilderX64.test.cpp | 27 ++ tests/AstQuery.test.cpp | 33 ++ tests/Fixture.h | 1 + tests/Module.test.cpp | 2 - tests/Normalize.test.cpp | 42 ++- tests/Parser.test.cpp | 1 - tests/ToString.test.cpp | 2 +- tests/TypeInfer.anyerror.test.cpp | 8 +- tests/TypeInfer.builtins.test.cpp | 143 ++++++- tests/TypeInfer.functions.test.cpp | 54 ++- tests/TypeInfer.generics.test.cpp | 39 +- tests/TypeInfer.loops.test.cpp | 2 +- tests/TypeInfer.modules.test.cpp | 45 ++- tests/TypeInfer.operators.test.cpp | 22 ++ tests/TypeInfer.primitives.test.cpp | 2 +- tests/TypeInfer.provisional.test.cpp | 13 +- tests/TypeInfer.refinements.test.cpp | 45 ++- tests/TypeInfer.tables.test.cpp | 14 + tests/TypeInfer.test.cpp | 10 +- tests/TypeInfer.tryUnify.test.cpp | 4 +- tests/TypeInfer.unionTypes.test.cpp | 2 +- tests/TypeInfer.unknownnever.test.cpp | 280 ++++++++++++++ tests/TypePack.test.cpp | 2 - tests/TypeVar.test.cpp | 2 - tests/conformance/vector.lua | 16 + tools/natvis/VM.natvis | 32 +- 54 files changed, 1867 insertions(+), 593 deletions(-) create mode 100644 tests/TypeInfer.unknownnever.test.cpp diff --git a/Analysis/include/Luau/AstQuery.h b/Analysis/include/Luau/AstQuery.h index dfe373a5..950a19da 100644 --- a/Analysis/include/Luau/AstQuery.h +++ b/Analysis/include/Luau/AstQuery.h @@ -63,6 +63,7 @@ private: AstLocal* local = nullptr; }; +std::vector findAncestryAtPositionForAutocomplete(const SourceModule& source, Position pos); std::vector findAstAncestryOfPosition(const SourceModule& source, Position pos); AstNode* findNodeAtPosition(const SourceModule& source, Position pos); AstExpr* findExprAtPosition(const SourceModule& source, Position pos); diff --git a/Analysis/include/Luau/TypeInfer.h b/Analysis/include/Luau/TypeInfer.h index dac2902c..3fb710bb 100644 --- a/Analysis/include/Luau/TypeInfer.h +++ b/Analysis/include/Luau/TypeInfer.h @@ -153,7 +153,7 @@ struct TypeChecker const ScopePtr& scope, const AstExprBinary& expr, TypeId lhsType, TypeId rhsType, const PredicateVec& predicates = {}); TypeId checkBinaryOperation( const ScopePtr& scope, const AstExprBinary& expr, TypeId lhsType, TypeId rhsType, const PredicateVec& predicates = {}); - WithPredicate checkExpr(const ScopePtr& scope, const AstExprBinary& expr); + WithPredicate checkExpr(const ScopePtr& scope, const AstExprBinary& expr, std::optional expectedType = std::nullopt); WithPredicate checkExpr(const ScopePtr& scope, const AstExprTypeAssertion& expr); WithPredicate checkExpr(const ScopePtr& scope, const AstExprError& expr); WithPredicate checkExpr(const ScopePtr& scope, const AstExprIfElse& expr, std::optional expectedType = std::nullopt); @@ -180,8 +180,12 @@ struct TypeChecker const ScopePtr& scope, Unifier& state, TypePackId paramPack, TypePackId argPack, const std::vector& argLocations); WithPredicate checkExprPack(const ScopePtr& scope, const AstExpr& expr); - WithPredicate checkExprPack(const ScopePtr& scope, const AstExprCall& expr); + + WithPredicate checkExprPackHelper(const ScopePtr& scope, const AstExpr& expr); + WithPredicate checkExprPackHelper(const ScopePtr& scope, const AstExprCall& expr); + std::vector> getExpectedTypesForCall(const std::vector& overloads, size_t argumentCount, bool selfCall); + std::optional> checkCallOverload(const ScopePtr& scope, const AstExprCall& expr, TypeId fn, TypePackId retPack, TypePackId argPack, TypePack* args, const std::vector* argLocations, const WithPredicate& argListResult, std::vector& overloadsThatMatchArgCount, std::vector& overloadsThatDont, std::vector& errors); @@ -236,10 +240,11 @@ struct TypeChecker void unifyLowerBound(TypePackId subTy, TypePackId superTy, TypeLevel demotedLevel, const Location& location); - std::optional findMetatableEntry(TypeId type, std::string entry, const Location& location); - std::optional findTablePropertyRespectingMeta(TypeId lhsType, Name name, const Location& location); + std::optional findMetatableEntry(TypeId type, std::string entry, const Location& location, bool addErrors); + std::optional findTablePropertyRespectingMeta(TypeId lhsType, Name name, const Location& location, bool addErrors); std::optional getIndexTypeFromType(const ScopePtr& scope, TypeId type, const Name& name, const Location& location, bool addErrors); + std::optional getIndexTypeFromTypeImpl(const ScopePtr& scope, TypeId type, const Name& name, const Location& location, bool addErrors); // Reduces the union to its simplest possible shape. // (A | B) | B | C yields A | B | C @@ -316,11 +321,12 @@ private: TypeIdPredicate mkTruthyPredicate(bool sense); - // Returns nullopt if the predicate filters down the TypeId to 0 options. - std::optional filterMap(TypeId type, TypeIdPredicate predicate); + // TODO: Return TypeId only. + std::optional filterMapImpl(TypeId type, TypeIdPredicate predicate); + std::pair, bool> filterMap(TypeId type, TypeIdPredicate predicate); public: - std::optional pickTypesFromSense(TypeId type, bool sense); + std::pair, bool> pickTypesFromSense(TypeId type, bool sense); private: TypeId unionOfTypes(TypeId a, TypeId b, const Location& location, bool unifyFreeTypes = true); @@ -413,8 +419,12 @@ public: const TypeId booleanType; const TypeId threadType; const TypeId anyType; + const TypeId unknownType; + const TypeId neverType; const TypePackId anyTypePack; + const TypePackId neverTypePack; + const TypePackId uninhabitableTypePack; private: int checkRecursionCount = 0; diff --git a/Analysis/include/Luau/TypePack.h b/Analysis/include/Luau/TypePack.h index c1de242f..b17003b1 100644 --- a/Analysis/include/Luau/TypePack.h +++ b/Analysis/include/Luau/TypePack.h @@ -173,5 +173,6 @@ std::pair, std::optional> flatten(TypePackId tp, bool isVariadic(TypePackId tp); bool isVariadic(TypePackId tp, const TxnLog& log); +bool containsNever(TypePackId tp); } // namespace Luau diff --git a/Analysis/include/Luau/TypeVar.h b/Analysis/include/Luau/TypeVar.h index 6ad6b927..fb6093df 100644 --- a/Analysis/include/Luau/TypeVar.h +++ b/Analysis/include/Luau/TypeVar.h @@ -460,10 +460,18 @@ struct LazyTypeVar std::function thunk; }; +struct UnknownTypeVar +{ +}; + +struct NeverTypeVar +{ +}; + using ErrorTypeVar = Unifiable::Error; using TypeVariant = Unifiable::Variant; + MetatableTypeVar, ClassTypeVar, AnyTypeVar, UnionTypeVar, IntersectionTypeVar, LazyTypeVar, UnknownTypeVar, NeverTypeVar>; struct TypeVar final { @@ -575,8 +583,12 @@ struct SingletonTypes const TypeId trueType; const TypeId falseType; const TypeId anyType; + const TypeId unknownType; + const TypeId neverType; const TypePackId anyTypePack; + const TypePackId neverTypePack; + const TypePackId uninhabitableTypePack; SingletonTypes(); ~SingletonTypes(); @@ -632,12 +644,30 @@ T* getMutable(TypeId tv) return get_if(&asMutable(tv)->ty); } -/* Traverses the UnionTypeVar yielding each TypeId. - * If the iterator encounters a nested UnionTypeVar, it will instead yield each TypeId within. - * - * Beware: the iterator does not currently filter for unique TypeIds. This may change in the future. +const std::vector& getTypes(const UnionTypeVar* utv); +const std::vector& getTypes(const IntersectionTypeVar* itv); +const std::vector& getTypes(const ConstrainedTypeVar* ctv); + +template +struct TypeIterator; + +using UnionTypeVarIterator = TypeIterator; +UnionTypeVarIterator begin(const UnionTypeVar* utv); +UnionTypeVarIterator end(const UnionTypeVar* utv); + +using IntersectionTypeVarIterator = TypeIterator; +IntersectionTypeVarIterator begin(const IntersectionTypeVar* itv); +IntersectionTypeVarIterator end(const IntersectionTypeVar* itv); + +using ConstrainedTypeVarIterator = TypeIterator; +ConstrainedTypeVarIterator begin(const ConstrainedTypeVar* ctv); +ConstrainedTypeVarIterator end(const ConstrainedTypeVar* ctv); + +/* Traverses the type T yielding each TypeId. + * If the iterator encounters a nested type T, it will instead yield each TypeId within. */ -struct UnionTypeVarIterator +template +struct TypeIterator { using value_type = Luau::TypeId; using pointer = value_type*; @@ -645,33 +675,116 @@ struct UnionTypeVarIterator using difference_type = size_t; using iterator_category = std::input_iterator_tag; - explicit UnionTypeVarIterator(const UnionTypeVar* utv); + explicit TypeIterator(const T* t) + { + LUAU_ASSERT(t); - UnionTypeVarIterator& operator++(); - UnionTypeVarIterator operator++(int); - bool operator!=(const UnionTypeVarIterator& rhs); - bool operator==(const UnionTypeVarIterator& rhs); + const std::vector& types = getTypes(t); + if (!types.empty()) + stack.push_front({t, 0}); - const TypeId& operator*(); + seen.insert(t); + } - friend UnionTypeVarIterator end(const UnionTypeVar* utv); + TypeIterator& operator++() + { + advance(); + descend(); + return *this; + } + + TypeIterator operator++(int) + { + TypeIterator copy = *this; + ++copy; + return copy; + } + + bool operator==(const TypeIterator& rhs) const + { + if (!stack.empty() && !rhs.stack.empty()) + return stack.front() == rhs.stack.front(); + + return stack.empty() && rhs.stack.empty(); + } + + bool operator!=(const TypeIterator& rhs) const + { + return !(*this == rhs); + } + + const TypeId& operator*() + { + LUAU_ASSERT(!stack.empty()); + + descend(); + + auto [t, currentIndex] = stack.front(); + LUAU_ASSERT(t); + const std::vector& types = getTypes(t); + LUAU_ASSERT(currentIndex < types.size()); + + const TypeId& ty = types[currentIndex]; + LUAU_ASSERT(!get(follow(ty))); + return ty; + } + + // Normally, we'd have `begin` and `end` be a template but there's too much trouble + // with templates portability in this area, so not worth it. Thanks MSVC. + friend UnionTypeVarIterator end(const UnionTypeVar*); + friend IntersectionTypeVarIterator end(const IntersectionTypeVar*); + friend ConstrainedTypeVarIterator end(const ConstrainedTypeVar*); private: - UnionTypeVarIterator() = default; + TypeIterator() = default; - // (UnionTypeVar* utv, size_t currentIndex) - using SavedIterInfo = std::pair; + // (T* t, size_t currentIndex) + using SavedIterInfo = std::pair; std::deque stack; - std::unordered_set seen; // Only needed to protect the iterator from hanging the thread. + std::unordered_set seen; // Only needed to protect the iterator from hanging the thread. - void advance(); - void descend(); + void advance() + { + while (!stack.empty()) + { + auto& [t, currentIndex] = stack.front(); + ++currentIndex; + + const std::vector& types = getTypes(t); + if (currentIndex >= types.size()) + stack.pop_front(); + else + break; + } + } + + void descend() + { + while (!stack.empty()) + { + auto [current, currentIndex] = stack.front(); + const std::vector& types = getTypes(current); + if (auto inner = get(follow(types[currentIndex]))) + { + // If we're about to descend into a cyclic type, we should skip over this. + // Ideally this should never happen, but alas it does from time to time. :( + if (seen.find(inner) != seen.end()) + advance(); + else + { + seen.insert(inner); + stack.push_front({inner, 0}); + } + + continue; + } + + break; + } + } }; -UnionTypeVarIterator begin(const UnionTypeVar* utv); -UnionTypeVarIterator end(const UnionTypeVar* utv); - using TypeIdPredicate = std::function(TypeId)>; std::vector filterMap(TypeId type, TypeIdPredicate predicate); diff --git a/Analysis/include/Luau/VisitTypeVar.h b/Analysis/include/Luau/VisitTypeVar.h index 5fd43f0b..ab4a397d 100644 --- a/Analysis/include/Luau/VisitTypeVar.h +++ b/Analysis/include/Luau/VisitTypeVar.h @@ -129,6 +129,14 @@ struct GenericTypeVarVisitor { return visit(ty); } + virtual bool visit(TypeId ty, const UnknownTypeVar& atv) + { + return visit(ty); + } + virtual bool visit(TypeId ty, const NeverTypeVar& atv) + { + return visit(ty); + } virtual bool visit(TypeId ty, const UnionTypeVar& utv) { return visit(ty); diff --git a/Analysis/src/AstQuery.cpp b/Analysis/src/AstQuery.cpp index 0522b1fa..1124c29e 100644 --- a/Analysis/src/AstQuery.cpp +++ b/Analysis/src/AstQuery.cpp @@ -17,6 +17,104 @@ namespace Luau namespace { + +struct AutocompleteNodeFinder : public AstVisitor +{ + const Position pos; + std::vector ancestry; + + explicit AutocompleteNodeFinder(Position pos, AstNode* root) + : pos(pos) + { + } + + bool visit(AstExpr* expr) override + { + if (expr->location.begin < pos && pos <= expr->location.end) + { + ancestry.push_back(expr); + return true; + } + return false; + } + + bool visit(AstStat* stat) override + { + if (stat->location.begin < pos && pos <= stat->location.end) + { + ancestry.push_back(stat); + return true; + } + return false; + } + + bool visit(AstType* type) override + { + if (type->location.begin < pos && pos <= type->location.end) + { + ancestry.push_back(type); + return true; + } + return false; + } + + bool visit(AstTypeError* type) override + { + // For a missing type, match the whole range including the start position + if (type->isMissing && type->location.containsClosed(pos)) + { + ancestry.push_back(type); + return true; + } + return false; + } + + bool visit(class AstTypePack* typePack) override + { + return true; + } + + bool visit(AstStatBlock* block) override + { + // If ancestry is empty, we are inspecting the root of the AST. Its extent is considered to be infinite. + if (ancestry.empty()) + { + ancestry.push_back(block); + return true; + } + + // AstExprIndexName nodes are nested outside-in, so we want the outermost node in the case of nested nodes. + // ex foo.bar.baz is represented in the AST as IndexName{ IndexName {foo, bar}, baz} + if (!ancestry.empty() && ancestry.back()->is()) + return false; + + // Type annotation error might intersect the block statement when the function header is being written, + // annotation takes priority + if (!ancestry.empty() && ancestry.back()->is()) + return false; + + // If the cursor is at the end of an expression or type and simultaneously at the beginning of a block, + // the expression or type wins out. + // The exception to this is if we are in a block under an AstExprFunction. In this case, we consider the position to + // be within the block. + if (block->location.begin == pos && !ancestry.empty()) + { + if (ancestry.back()->asExpr() && !ancestry.back()->is()) + return false; + + if (ancestry.back()->asType()) + return false; + } + + if (block->location.begin <= pos && pos <= block->location.end) + { + ancestry.push_back(block); + return true; + } + return false; + } +}; + struct FindNode : public AstVisitor { const Position pos; @@ -102,6 +200,13 @@ struct FindFullAncestry final : public AstVisitor } // namespace +std::vector findAncestryAtPositionForAutocomplete(const SourceModule& source, Position pos) +{ + AutocompleteNodeFinder finder{pos, source.root}; + source.root->visit(&finder); + return finder.ancestry; +} + std::vector findAstAncestryOfPosition(const SourceModule& source, Position pos) { const Position end = source.root->location.end; @@ -110,7 +215,7 @@ std::vector findAstAncestryOfPosition(const SourceModule& source, Posi FindFullAncestry finder(pos, end); source.root->visit(&finder); - return std::move(finder.nodes); + return finder.nodes; } AstNode* findNodeAtPosition(const SourceModule& source, Position pos) diff --git a/Analysis/src/Autocomplete.cpp b/Analysis/src/Autocomplete.cpp index 8a63901f..cc54d499 100644 --- a/Analysis/src/Autocomplete.cpp +++ b/Analysis/src/Autocomplete.cpp @@ -21,102 +21,6 @@ static const std::unordered_set kStatementStartingKeywords = { namespace Luau { -struct NodeFinder : public AstVisitor -{ - const Position pos; - std::vector ancestry; - - explicit NodeFinder(Position pos, AstNode* root) - : pos(pos) - { - } - - bool visit(AstExpr* expr) override - { - if (expr->location.begin < pos && pos <= expr->location.end) - { - ancestry.push_back(expr); - return true; - } - return false; - } - - bool visit(AstStat* stat) override - { - if (stat->location.begin < pos && pos <= stat->location.end) - { - ancestry.push_back(stat); - return true; - } - return false; - } - - bool visit(AstType* type) override - { - if (type->location.begin < pos && pos <= type->location.end) - { - ancestry.push_back(type); - return true; - } - return false; - } - - bool visit(AstTypeError* type) override - { - // For a missing type, match the whole range including the start position - if (type->isMissing && type->location.containsClosed(pos)) - { - ancestry.push_back(type); - return true; - } - return false; - } - - bool visit(class AstTypePack* typePack) override - { - return true; - } - - bool visit(AstStatBlock* block) override - { - // If ancestry is empty, we are inspecting the root of the AST. Its extent is considered to be infinite. - if (ancestry.empty()) - { - ancestry.push_back(block); - return true; - } - - // AstExprIndexName nodes are nested outside-in, so we want the outermost node in the case of nested nodes. - // ex foo.bar.baz is represented in the AST as IndexName{ IndexName {foo, bar}, baz} - if (!ancestry.empty() && ancestry.back()->is()) - return false; - - // Type annotation error might intersect the block statement when the function header is being written, - // annotation takes priority - if (!ancestry.empty() && ancestry.back()->is()) - return false; - - // If the cursor is at the end of an expression or type and simultaneously at the beginning of a block, - // the expression or type wins out. - // The exception to this is if we are in a block under an AstExprFunction. In this case, we consider the position to - // be within the block. - if (block->location.begin == pos && !ancestry.empty()) - { - if (ancestry.back()->asExpr() && !ancestry.back()->is()) - return false; - - if (ancestry.back()->asType()) - return false; - } - - if (block->location.begin <= pos && pos <= block->location.end) - { - ancestry.push_back(block); - return true; - } - return false; - } -}; static bool alreadyHasParens(const std::vector& nodes) { @@ -905,7 +809,7 @@ AutocompleteEntryMap autocompleteTypeNames(const Module& module, Position positi } AstNode* parent = nullptr; - AstType* topType = nullptr; + AstType* topType = nullptr; // TODO: rename? for (auto it = ancestry.rbegin(), e = ancestry.rend(); it != e; ++it) { @@ -1477,21 +1381,20 @@ static AutocompleteResult autocomplete(const SourceModule& sourceModule, const M if (isWithinComment(sourceModule, position)) return {}; - NodeFinder finder{position, sourceModule.root}; - sourceModule.root->visit(&finder); - LUAU_ASSERT(!finder.ancestry.empty()); - AstNode* node = finder.ancestry.back(); + std::vector ancestry = findAncestryAtPositionForAutocomplete(sourceModule, position); + LUAU_ASSERT(!ancestry.empty()); + AstNode* node = ancestry.back(); AstExprConstantNil dummy{Location{}}; - AstNode* parent = finder.ancestry.size() >= 2 ? finder.ancestry.rbegin()[1] : &dummy; + AstNode* parent = ancestry.size() >= 2 ? ancestry.rbegin()[1] : &dummy; // If we are inside a body of a function that doesn't have a completed argument list, ignore the body node if (auto exprFunction = parent->as(); exprFunction && !exprFunction->argLocation && node == exprFunction->body) { - finder.ancestry.pop_back(); + ancestry.pop_back(); - node = finder.ancestry.back(); - parent = finder.ancestry.size() >= 2 ? finder.ancestry.rbegin()[1] : &dummy; + node = ancestry.back(); + parent = ancestry.size() >= 2 ? ancestry.rbegin()[1] : &dummy; } if (auto indexName = node->as()) @@ -1504,47 +1407,47 @@ static AutocompleteResult autocomplete(const SourceModule& sourceModule, const M PropIndexType indexType = indexName->op == ':' ? PropIndexType::Colon : PropIndexType::Point; if (!FFlag::LuauSelfCallAutocompleteFix2 && isString(ty)) - return {autocompleteProps(*module, typeArena, typeChecker.globalScope->bindings[AstName{"string"}].typeId, indexType, finder.ancestry), - finder.ancestry}; + return {autocompleteProps(*module, typeArena, typeChecker.globalScope->bindings[AstName{"string"}].typeId, indexType, ancestry), + ancestry}; else - return {autocompleteProps(*module, typeArena, ty, indexType, finder.ancestry), finder.ancestry}; + return {autocompleteProps(*module, typeArena, ty, indexType, ancestry), ancestry}; } else if (auto typeReference = node->as()) { if (typeReference->prefix) - return {autocompleteModuleTypes(*module, position, typeReference->prefix->value), finder.ancestry}; + return {autocompleteModuleTypes(*module, position, typeReference->prefix->value), ancestry}; else - return {autocompleteTypeNames(*module, position, finder.ancestry), finder.ancestry}; + return {autocompleteTypeNames(*module, position, ancestry), ancestry}; } else if (node->is()) { - return {autocompleteTypeNames(*module, position, finder.ancestry), finder.ancestry}; + return {autocompleteTypeNames(*module, position, ancestry), ancestry}; } else if (AstStatLocal* statLocal = node->as()) { if (statLocal->vars.size == 1 && (!statLocal->equalsSignLocation || position < statLocal->equalsSignLocation->begin)) - return {{{"function", AutocompleteEntry{AutocompleteEntryKind::Keyword}}}, finder.ancestry}; + return {{{"function", AutocompleteEntry{AutocompleteEntryKind::Keyword}}}, ancestry}; else if (statLocal->equalsSignLocation && position >= statLocal->equalsSignLocation->end) - return {autocompleteExpression(sourceModule, *module, typeChecker, typeArena, finder.ancestry, position), finder.ancestry}; + return {autocompleteExpression(sourceModule, *module, typeChecker, typeArena, ancestry, position), ancestry}; else return {}; } - else if (AstStatFor* statFor = extractStat(finder.ancestry)) + else if (AstStatFor* statFor = extractStat(ancestry)) { if (!statFor->hasDo || position < statFor->doLocation.begin) { if (!statFor->from->is() && !statFor->to->is() && (!statFor->step || !statFor->step->is())) - return {{{"do", AutocompleteEntry{AutocompleteEntryKind::Keyword}}}, finder.ancestry}; + return {{{"do", AutocompleteEntry{AutocompleteEntryKind::Keyword}}}, ancestry}; if (statFor->from->location.containsClosed(position) || statFor->to->location.containsClosed(position) || (statFor->step && statFor->step->location.containsClosed(position))) - return {autocompleteExpression(sourceModule, *module, typeChecker, typeArena, finder.ancestry, position), finder.ancestry}; + return {autocompleteExpression(sourceModule, *module, typeChecker, typeArena, ancestry, position), ancestry}; return {}; } - return {autocompleteStatement(sourceModule, *module, finder.ancestry, position), finder.ancestry}; + return {autocompleteStatement(sourceModule, *module, ancestry, position), ancestry}; } else if (AstStatForIn* statForIn = parent->as(); statForIn && (node->is() || isIdentifier(node))) @@ -1560,7 +1463,7 @@ static AutocompleteResult autocomplete(const SourceModule& sourceModule, const M return {}; } - return {{{"in", AutocompleteEntry{AutocompleteEntryKind::Keyword}}}, finder.ancestry}; + return {{{"in", AutocompleteEntry{AutocompleteEntryKind::Keyword}}}, ancestry}; } if (!statForIn->hasDo || position <= statForIn->doLocation.begin) @@ -1569,58 +1472,58 @@ static AutocompleteResult autocomplete(const SourceModule& sourceModule, const M AstExpr* lastExpr = statForIn->values.data[statForIn->values.size - 1]; if (lastExpr->location.containsClosed(position)) - return {autocompleteExpression(sourceModule, *module, typeChecker, typeArena, finder.ancestry, position), finder.ancestry}; + return {autocompleteExpression(sourceModule, *module, typeChecker, typeArena, ancestry, position), ancestry}; if (position > lastExpr->location.end) - return {{{"do", AutocompleteEntry{AutocompleteEntryKind::Keyword}}}, finder.ancestry}; + return {{{"do", AutocompleteEntry{AutocompleteEntryKind::Keyword}}}, ancestry}; return {}; // Not sure what this means } } - else if (AstStatForIn* statForIn = extractStat(finder.ancestry)) + else if (AstStatForIn* statForIn = extractStat(ancestry)) { // The AST looks a bit differently if the cursor is at a position where only the "do" keyword is allowed. // ex "for f in f do" if (!statForIn->hasDo) - return {{{"do", AutocompleteEntry{AutocompleteEntryKind::Keyword}}}, finder.ancestry}; + return {{{"do", AutocompleteEntry{AutocompleteEntryKind::Keyword}}}, ancestry}; - return {autocompleteStatement(sourceModule, *module, finder.ancestry, position), finder.ancestry}; + return {autocompleteStatement(sourceModule, *module, ancestry, position), ancestry}; } else if (AstStatWhile* statWhile = parent->as(); node->is() && statWhile) { if (!statWhile->hasDo && !statWhile->condition->is() && position > statWhile->condition->location.end) - return {{{"do", AutocompleteEntry{AutocompleteEntryKind::Keyword}}}, finder.ancestry}; + return {{{"do", AutocompleteEntry{AutocompleteEntryKind::Keyword}}}, ancestry}; if (!statWhile->hasDo || position < statWhile->doLocation.begin) - return {autocompleteExpression(sourceModule, *module, typeChecker, typeArena, finder.ancestry, position), finder.ancestry}; + return {autocompleteExpression(sourceModule, *module, typeChecker, typeArena, ancestry, position), ancestry}; if (statWhile->hasDo && position > statWhile->doLocation.end) - return {autocompleteStatement(sourceModule, *module, finder.ancestry, position), finder.ancestry}; + return {autocompleteStatement(sourceModule, *module, ancestry, position), ancestry}; } - else if (AstStatWhile* statWhile = extractStat(finder.ancestry); statWhile && !statWhile->hasDo) - return {{{"do", AutocompleteEntry{AutocompleteEntryKind::Keyword}}}, finder.ancestry}; + else if (AstStatWhile* statWhile = extractStat(ancestry); statWhile && !statWhile->hasDo) + return {{{"do", AutocompleteEntry{AutocompleteEntryKind::Keyword}}}, ancestry}; else if (AstStatIf* statIf = node->as(); statIf && !statIf->elseLocation.has_value()) { return {{{"else", AutocompleteEntry{AutocompleteEntryKind::Keyword}}, {"elseif", AutocompleteEntry{AutocompleteEntryKind::Keyword}}}, - finder.ancestry}; + ancestry}; } else if (AstStatIf* statIf = parent->as(); statIf && node->is()) { if (statIf->condition->is()) - return {autocompleteExpression(sourceModule, *module, typeChecker, typeArena, finder.ancestry, position), finder.ancestry}; + return {autocompleteExpression(sourceModule, *module, typeChecker, typeArena, ancestry, position), ancestry}; else if (!statIf->thenLocation || statIf->thenLocation->containsClosed(position)) - return {{{"then", AutocompleteEntry{AutocompleteEntryKind::Keyword}}}, finder.ancestry}; + return {{{"then", AutocompleteEntry{AutocompleteEntryKind::Keyword}}}, ancestry}; } - else if (AstStatIf* statIf = extractStat(finder.ancestry); + else if (AstStatIf* statIf = extractStat(ancestry); statIf && (!statIf->thenLocation || statIf->thenLocation->containsClosed(position))) - return {{{"then", AutocompleteEntry{AutocompleteEntryKind::Keyword}}}, finder.ancestry}; + return {{{"then", AutocompleteEntry{AutocompleteEntryKind::Keyword}}}, ancestry}; else if (AstStatRepeat* statRepeat = node->as(); statRepeat && statRepeat->condition->is()) - return {autocompleteExpression(sourceModule, *module, typeChecker, typeArena, finder.ancestry, position), finder.ancestry}; - else if (AstStatRepeat* statRepeat = extractStat(finder.ancestry); statRepeat) - return {autocompleteStatement(sourceModule, *module, finder.ancestry, position), finder.ancestry}; + return {autocompleteExpression(sourceModule, *module, typeChecker, typeArena, ancestry, position), ancestry}; + else if (AstStatRepeat* statRepeat = extractStat(ancestry); statRepeat) + return {autocompleteStatement(sourceModule, *module, ancestry, position), ancestry}; else if (AstExprTable* exprTable = parent->as(); exprTable && (node->is() || node->is())) { for (const auto& [kind, key, value] : exprTable->items) @@ -1630,7 +1533,7 @@ static AutocompleteResult autocomplete(const SourceModule& sourceModule, const M { if (auto it = module->astExpectedTypes.find(exprTable)) { - auto result = autocompleteProps(*module, typeArena, *it, PropIndexType::Key, finder.ancestry); + auto result = autocompleteProps(*module, typeArena, *it, PropIndexType::Key, ancestry); // Remove keys that are already completed for (const auto& item : exprTable->items) @@ -1644,9 +1547,9 @@ static AutocompleteResult autocomplete(const SourceModule& sourceModule, const M // If we know for sure that a key is being written, do not offer general expression suggestions if (!key) - autocompleteExpression(sourceModule, *module, typeChecker, typeArena, finder.ancestry, position, result); + autocompleteExpression(sourceModule, *module, typeChecker, typeArena, ancestry, position, result); - return {result, finder.ancestry}; + return {result, ancestry}; } break; @@ -1654,11 +1557,11 @@ static AutocompleteResult autocomplete(const SourceModule& sourceModule, const M } } else if (isIdentifier(node) && (parent->is() || parent->is())) - return {autocompleteStatement(sourceModule, *module, finder.ancestry, position), finder.ancestry}; + return {autocompleteStatement(sourceModule, *module, ancestry, position), ancestry}; - if (std::optional ret = autocompleteStringParams(sourceModule, module, finder.ancestry, position, callback)) + if (std::optional ret = autocompleteStringParams(sourceModule, module, ancestry, position, callback)) { - return {*ret, finder.ancestry}; + return {*ret, ancestry}; } else if (node->is()) { @@ -1667,14 +1570,14 @@ static AutocompleteResult autocomplete(const SourceModule& sourceModule, const M if (auto it = module->astExpectedTypes.find(node->asExpr())) autocompleteStringSingleton(*it, false, result); - if (finder.ancestry.size() >= 2) + if (ancestry.size() >= 2) { - if (auto idxExpr = finder.ancestry.at(finder.ancestry.size() - 2)->as()) + if (auto idxExpr = ancestry.at(ancestry.size() - 2)->as()) { if (auto it = module->astTypes.find(idxExpr->expr)) - autocompleteProps(*module, typeArena, follow(*it), PropIndexType::Point, finder.ancestry, result); + autocompleteProps(*module, typeArena, follow(*it), PropIndexType::Point, ancestry, result); } - else if (auto binExpr = finder.ancestry.at(finder.ancestry.size() - 2)->as()) + else if (auto binExpr = ancestry.at(ancestry.size() - 2)->as()) { if (binExpr->op == AstExprBinary::CompareEq || binExpr->op == AstExprBinary::CompareNe) { @@ -1684,7 +1587,7 @@ static AutocompleteResult autocomplete(const SourceModule& sourceModule, const M } } - return {result, finder.ancestry}; + return {result, ancestry}; } if (node->is()) @@ -1693,9 +1596,9 @@ static AutocompleteResult autocomplete(const SourceModule& sourceModule, const M } if (node->asExpr()) - return {autocompleteExpression(sourceModule, *module, typeChecker, typeArena, finder.ancestry, position), finder.ancestry}; + return {autocompleteExpression(sourceModule, *module, typeChecker, typeArena, ancestry, position), ancestry}; else if (node->asStat()) - return {autocompleteStatement(sourceModule, *module, finder.ancestry, position), finder.ancestry}; + return {autocompleteStatement(sourceModule, *module, ancestry, position), ancestry}; return {}; } diff --git a/Analysis/src/BuiltinDefinitions.cpp b/Analysis/src/BuiltinDefinitions.cpp index 2f57e23c..aeba2c13 100644 --- a/Analysis/src/BuiltinDefinitions.cpp +++ b/Analysis/src/BuiltinDefinitions.cpp @@ -9,6 +9,7 @@ #include LUAU_FASTFLAGVARIABLE(LuauSetMetaTableArgsCheck, false) +LUAU_FASTFLAG(LuauUnknownAndNeverType) /** FIXME: Many of these type definitions are not quite completely accurate. * @@ -222,14 +223,14 @@ void registerBuiltinTypes(TypeChecker& typeChecker) addGlobalBinding(typeChecker, "getmetatable", makeFunction(arena, std::nullopt, {genericMT}, {}, {tableMetaMT}, {genericMT}), "@luau"); - // setmetatable({ @metatable MT }, MT) -> { @metatable MT } // clang-format off + // setmetatable(T, MT) -> { @metatable MT, T } addGlobalBinding(typeChecker, "setmetatable", arena.addType( FunctionTypeVar{ {genericMT}, {}, - arena.addTypePack(TypePack{{tableMetaMT, genericMT}}), + arena.addTypePack(TypePack{{FFlag::LuauUnknownAndNeverType ? tabTy : tableMetaMT, genericMT}}), arena.addTypePack(TypePack{{tableMetaMT}}) } ), "@luau" @@ -309,6 +310,12 @@ static std::optional> magicFunctionSetMetaTable( { auto [paramPack, _predicates] = withPredicate; + if (FFlag::LuauUnknownAndNeverType) + { + if (size(paramPack) < 2 && finite(paramPack)) + return std::nullopt; + } + TypeArena& arena = typechecker.currentModule->internalTypes; std::vector expectedArgs = typechecker.unTypePack(scope, paramPack, 2, expr.location); @@ -316,6 +323,12 @@ static std::optional> magicFunctionSetMetaTable( TypeId target = follow(expectedArgs[0]); TypeId mt = follow(expectedArgs[1]); + if (FFlag::LuauUnknownAndNeverType) + { + typechecker.tablify(target); + typechecker.tablify(mt); + } + if (const auto& tab = get(target)) { if (target->persistent) @@ -324,7 +337,8 @@ static std::optional> magicFunctionSetMetaTable( } else { - typechecker.tablify(mt); + if (!FFlag::LuauUnknownAndNeverType) + typechecker.tablify(mt); const TableTypeVar* mtTtv = get(mt); MetatableTypeVar mtv{target, mt}; @@ -343,7 +357,10 @@ static std::optional> magicFunctionSetMetaTable( if (FFlag::LuauSetMetaTableArgsCheck && expr.args.size < 1) { - return WithPredicate{}; + if (FFlag::LuauUnknownAndNeverType) + return std::nullopt; + else + return WithPredicate{}; } if (!FFlag::LuauSetMetaTableArgsCheck || !expr.self) @@ -390,11 +407,21 @@ static std::optional> magicFunctionAssert( if (head.size() > 0) { - std::optional newhead = typechecker.pickTypesFromSense(head[0], true); - if (!newhead) - head = {typechecker.nilType}; + auto [ty, ok] = typechecker.pickTypesFromSense(head[0], true); + if (FFlag::LuauUnknownAndNeverType) + { + if (get(*ty)) + head = {*ty}; + else + head[0] = *ty; + } else - head[0] = *newhead; + { + if (!ty) + head = {typechecker.nilType}; + else + head[0] = *ty; + } } return WithPredicate{arena.addTypePack(TypePack{std::move(head), tail})}; diff --git a/Analysis/src/Clone.cpp b/Analysis/src/Clone.cpp index df4e0a6b..88c50318 100644 --- a/Analysis/src/Clone.cpp +++ b/Analysis/src/Clone.cpp @@ -59,6 +59,8 @@ struct TypeCloner void operator()(const UnionTypeVar& t); void operator()(const IntersectionTypeVar& t); void operator()(const LazyTypeVar& t); + void operator()(const UnknownTypeVar& t); + void operator()(const NeverTypeVar& t); }; struct TypePackCloner @@ -310,6 +312,16 @@ void TypeCloner::operator()(const LazyTypeVar& t) defaultClone(t); } +void TypeCloner::operator()(const UnknownTypeVar& t) +{ + defaultClone(t); +} + +void TypeCloner::operator()(const NeverTypeVar& t) +{ + defaultClone(t); +} + } // anonymous namespace TypePackId clone(TypePackId tp, TypeArena& dest, CloneState& cloneState) diff --git a/Analysis/src/EmbeddedBuiltinDefinitions.cpp b/Analysis/src/EmbeddedBuiltinDefinitions.cpp index 1b5275fd..f93f65dd 100644 --- a/Analysis/src/EmbeddedBuiltinDefinitions.cpp +++ b/Analysis/src/EmbeddedBuiltinDefinitions.cpp @@ -1,6 +1,7 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #include "Luau/BuiltinDefinitions.h" +LUAU_FASTFLAG(LuauUnknownAndNeverType) LUAU_FASTFLAG(LuauCheckLenMT) namespace Luau @@ -116,8 +117,6 @@ declare function typeof(value: T): string -- `assert` has a magic function attached that will give more detailed type information declare function assert(value: T, errorMessage: string?): T -declare function error(message: T, level: number?) - declare function tostring(value: T): string declare function tonumber(value: T, radix: number?): number? @@ -204,12 +203,18 @@ declare function unpack(tab: {V}, i: number?, j: number?): ...V std::string getBuiltinDefinitionSource() { + std::string result = kBuiltinDefinitionLuaSrc; // TODO: move this into kBuiltinDefinitionLuaSrc if (FFlag::LuauCheckLenMT) result += "declare function rawlen(obj: {[K]: V} | string): number\n"; + if (FFlag::LuauUnknownAndNeverType) + result += "declare function error(message: T, level: number?): never\n"; + else + result += "declare function error(message: T, level: number?)\n"; + return result; } diff --git a/Analysis/src/Normalize.cpp b/Analysis/src/Normalize.cpp index 8ce7f742..ce8f96cc 100644 --- a/Analysis/src/Normalize.cpp +++ b/Analysis/src/Normalize.cpp @@ -14,7 +14,7 @@ LUAU_FASTFLAGVARIABLE(DebugLuauCopyBeforeNormalizing, false) LUAU_FASTINTVARIABLE(LuauNormalizeIterationLimit, 1200); LUAU_FASTFLAGVARIABLE(LuauNormalizeCombineTableFix, false); LUAU_FASTFLAGVARIABLE(LuauNormalizeFlagIsConservative, false); -LUAU_FASTFLAGVARIABLE(LuauNormalizeCombineEqFix, false); +LUAU_FASTFLAG(LuauUnknownAndNeverType) LUAU_FASTFLAG(LuauQuantifyConstrained) namespace Luau @@ -182,7 +182,6 @@ struct Normalize final : TypeVarVisitor { if (!ty->normal) asMutable(ty)->normal = true; - return false; } @@ -193,6 +192,20 @@ struct Normalize final : TypeVarVisitor return false; } + bool visit(TypeId ty, const UnknownTypeVar&) override + { + if (!ty->normal) + asMutable(ty)->normal = true; + return false; + } + + bool visit(TypeId ty, const NeverTypeVar&) override + { + if (!ty->normal) + asMutable(ty)->normal = true; + return false; + } + bool visit(TypeId ty, const ConstrainedTypeVar& ctvRef) override { CHECK_ITERATION_LIMIT(false); @@ -416,7 +429,13 @@ struct Normalize final : TypeVarVisitor std::vector result; for (TypeId part : options) + { + // AnyTypeVar always win the battle no matter what we do, so we're done. + if (FFlag::LuauUnknownAndNeverType && get(follow(part))) + return {part}; + combineIntoUnion(result, part); + } return result; } @@ -427,7 +446,17 @@ struct Normalize final : TypeVarVisitor if (auto utv = get(ty)) { for (TypeId t : utv) + { + // AnyTypeVar always win the battle no matter what we do, so we're done. + if (FFlag::LuauUnknownAndNeverType && get(t)) + { + result = {t}; + return; + } + combineIntoUnion(result, t); + } + return; } @@ -571,8 +600,7 @@ struct Normalize final : TypeVarVisitor */ TypeId combine(Replacer& replacer, TypeId a, TypeId b) { - if (FFlag::LuauNormalizeCombineEqFix) - b = follow(b); + b = follow(b); if (FFlag::LuauNormalizeCombineTableFix && a == b) return a; @@ -592,7 +620,7 @@ struct Normalize final : TypeVarVisitor } else if (auto ttv = getMutable(a)) { - if (FFlag::LuauNormalizeCombineTableFix && !get(FFlag::LuauNormalizeCombineEqFix ? b : follow(b))) + if (FFlag::LuauNormalizeCombineTableFix && !get(b)) return arena.addType(IntersectionTypeVar{{a, b}}); combineIntoTable(replacer, ttv, b); return a; diff --git a/Analysis/src/Substitution.cpp b/Analysis/src/Substitution.cpp index 9c4ce829..7245403c 100644 --- a/Analysis/src/Substitution.cpp +++ b/Analysis/src/Substitution.cpp @@ -8,8 +8,10 @@ #include #include +LUAU_FASTFLAGVARIABLE(LuauAnyificationMustClone, false) LUAU_FASTFLAG(LuauLowerBoundsCalculation) LUAU_FASTINTVARIABLE(LuauTarjanChildLimit, 10000) +LUAU_FASTFLAG(LuauUnknownAndNeverType) namespace Luau { @@ -154,7 +156,7 @@ TarjanResult Tarjan::loop() if (currEdge == -1) { ++childCount; - if (childLimit > 0 && childLimit < childCount) + if (childLimit > 0 && (FFlag::LuauUnknownAndNeverType ? childLimit <= childCount : childLimit < childCount)) return TarjanResult::TooManyChildren; stack.push_back(index); @@ -439,6 +441,9 @@ void Substitution::replaceChildren(TypeId ty) if (ignoreChildren(ty)) return; + if (FFlag::LuauAnyificationMustClone && ty->owningArena != arena) + return; + if (FunctionTypeVar* ftv = getMutable(ty)) { ftv->argTypes = replace(ftv->argTypes); @@ -490,6 +495,9 @@ void Substitution::replaceChildren(TypePackId tp) if (ignoreChildren(tp)) return; + if (FFlag::LuauAnyificationMustClone && tp->owningArena != arena) + return; + if (TypePack* tpp = getMutable(tp)) { for (TypeId& tv : tpp->head) diff --git a/Analysis/src/ToString.cpp b/Analysis/src/ToString.cpp index 5d54d14a..6bc1d572 100644 --- a/Analysis/src/ToString.cpp +++ b/Analysis/src/ToString.cpp @@ -11,6 +11,7 @@ #include LUAU_FASTFLAG(LuauLowerBoundsCalculation) +LUAU_FASTFLAG(LuauUnknownAndNeverType) /* * Prefix generic typenames with gen- @@ -841,7 +842,7 @@ struct TypeVarStringifier void operator()(TypeId, const ErrorTypeVar& tv) { state.result.error = true; - state.emit("*unknown*"); + state.emit(FFlag::LuauUnknownAndNeverType ? "" : "*unknown*"); } void operator()(TypeId, const LazyTypeVar& ltv) @@ -850,7 +851,17 @@ struct TypeVarStringifier state.emit("lazy?"); } -}; // namespace + void operator()(TypeId, const UnknownTypeVar& ttv) + { + state.emit("unknown"); + } + + void operator()(TypeId, const NeverTypeVar& ttv) + { + state.emit("never"); + } + +}; struct TypePackStringifier { @@ -955,7 +966,7 @@ struct TypePackStringifier void operator()(TypePackId, const Unifiable::Error& error) { state.result.error = true; - state.emit("*unknown*"); + state.emit(FFlag::LuauUnknownAndNeverType ? "" : "*unknown*"); } void operator()(TypePackId, const VariadicTypePack& pack) diff --git a/Analysis/src/TxnLog.cpp b/Analysis/src/TxnLog.cpp index 4c6d54e0..b3f60d30 100644 --- a/Analysis/src/TxnLog.cpp +++ b/Analysis/src/TxnLog.cpp @@ -7,7 +7,7 @@ #include #include -LUAU_FASTFLAG(LuauNonCopyableTypeVarFields) +LUAU_FASTFLAG(LuauUnknownAndNeverType) namespace Luau { @@ -81,34 +81,10 @@ void TxnLog::concat(TxnLog rhs) void TxnLog::commit() { for (auto& [ty, rep] : typeVarChanges) - { - if (FFlag::LuauNonCopyableTypeVarFields) - { - asMutable(ty)->reassign(rep.get()->pending); - } - else - { - TypeArena* owningArena = ty->owningArena; - TypeVar* mtv = asMutable(ty); - *mtv = rep.get()->pending; - mtv->owningArena = owningArena; - } - } + asMutable(ty)->reassign(rep.get()->pending); for (auto& [tp, rep] : typePackChanges) - { - if (FFlag::LuauNonCopyableTypeVarFields) - { - asMutable(tp)->reassign(rep.get()->pending); - } - else - { - TypeArena* owningArena = tp->owningArena; - TypePackVar* mpv = asMutable(tp); - *mpv = rep.get()->pending; - mpv->owningArena = owningArena; - } - } + asMutable(tp)->reassign(rep.get()->pending); clear(); } @@ -196,9 +172,7 @@ PendingType* TxnLog::queue(TypeId ty) if (!pending) { pending = std::make_unique(*ty); - - if (FFlag::LuauNonCopyableTypeVarFields) - pending->pending.owningArena = nullptr; + pending->pending.owningArena = nullptr; } return pending.get(); @@ -214,9 +188,7 @@ PendingTypePack* TxnLog::queue(TypePackId tp) if (!pending) { pending = std::make_unique(*tp); - - if (FFlag::LuauNonCopyableTypeVarFields) - pending->pending.owningArena = nullptr; + pending->pending.owningArena = nullptr; } return pending.get(); @@ -255,24 +227,14 @@ PendingTypePack* TxnLog::pending(TypePackId tp) const PendingType* TxnLog::replace(TypeId ty, TypeVar replacement) { PendingType* newTy = queue(ty); - - if (FFlag::LuauNonCopyableTypeVarFields) - newTy->pending.reassign(replacement); - else - newTy->pending = replacement; - + newTy->pending.reassign(replacement); return newTy; } PendingTypePack* TxnLog::replace(TypePackId tp, TypePackVar replacement) { PendingTypePack* newTp = queue(tp); - - if (FFlag::LuauNonCopyableTypeVarFields) - newTp->pending.reassign(replacement); - else - newTp->pending = replacement; - + newTp->pending.reassign(replacement); return newTp; } @@ -289,7 +251,7 @@ PendingType* TxnLog::bindTable(TypeId ty, std::optional newBoundTo) PendingType* TxnLog::changeLevel(TypeId ty, TypeLevel newLevel) { - LUAU_ASSERT(get(ty) || get(ty) || get(ty)); + LUAU_ASSERT(get(ty) || get(ty) || get(ty) || get(ty)); PendingType* newTy = queue(ty); if (FreeTypeVar* ftv = Luau::getMutable(newTy)) @@ -305,6 +267,11 @@ PendingType* TxnLog::changeLevel(TypeId ty, TypeLevel newLevel) { ftv->level = newLevel; } + else if (ConstrainedTypeVar* ctv = Luau::getMutable(newTy)) + { + if (FFlag::LuauUnknownAndNeverType) + ctv->level = newLevel; + } return newTy; } diff --git a/Analysis/src/TypeAttach.cpp b/Analysis/src/TypeAttach.cpp index 6cca7127..2bc89cf6 100644 --- a/Analysis/src/TypeAttach.cpp +++ b/Analysis/src/TypeAttach.cpp @@ -335,6 +335,14 @@ public: { return allocator->alloc(Location(), std::nullopt, AstName("")); } + AstType* operator()(const UnknownTypeVar& ttv) + { + return allocator->alloc(Location(), std::nullopt, AstName{"unknown"}); + } + AstType* operator()(const NeverTypeVar& ttv) + { + return allocator->alloc(Location(), std::nullopt, AstName{"never"}); + } private: Allocator* allocator; diff --git a/Analysis/src/TypeInfer.cpp b/Analysis/src/TypeInfer.cpp index 4fafb50f..01939fda 100644 --- a/Analysis/src/TypeInfer.cpp +++ b/Analysis/src/TypeInfer.cpp @@ -31,6 +31,7 @@ LUAU_FASTINTVARIABLE(LuauCheckRecursionLimit, 300) LUAU_FASTINTVARIABLE(LuauVisitRecursionLimit, 500) LUAU_FASTFLAG(LuauKnowsTheDataModel3) LUAU_FASTFLAG(LuauAutocompleteDynamicLimits) +LUAU_FASTFLAGVARIABLE(LuauIndexSilenceErrors, false) LUAU_FASTFLAGVARIABLE(LuauLowerBoundsCalculation, false) LUAU_FASTFLAGVARIABLE(DebugLuauFreezeDuringUnification, false) LUAU_FASTFLAGVARIABLE(LuauSelfCallAutocompleteFix2, false) @@ -41,10 +42,12 @@ LUAU_FASTFLAGVARIABLE(LuauReturnTypeInferenceInNonstrict, false) LUAU_FASTFLAGVARIABLE(DebugLuauSharedSelf, false); LUAU_FASTFLAGVARIABLE(LuauAlwaysQuantify, false); LUAU_FASTFLAGVARIABLE(LuauReportErrorsOnIndexerKeyMismatch, false) +LUAU_FASTFLAGVARIABLE(LuauUnknownAndNeverType, false) LUAU_FASTFLAG(LuauQuantifyConstrained) LUAU_FASTFLAGVARIABLE(LuauFalsyPredicateReturnsNilInstead, false) -LUAU_FASTFLAGVARIABLE(LuauNonCopyableTypeVarFields, false) LUAU_FASTFLAGVARIABLE(LuauCheckLenMT, false) +LUAU_FASTFLAGVARIABLE(LuauCheckGenericHOFTypes, false) +LUAU_FASTFLAGVARIABLE(LuauBinaryNeedsExpectedTypesToo, false) namespace Luau { @@ -258,7 +261,11 @@ TypeChecker::TypeChecker(ModuleResolver* resolver, InternalErrorReporter* iceHan , booleanType(getSingletonTypes().booleanType) , threadType(getSingletonTypes().threadType) , anyType(getSingletonTypes().anyType) + , unknownType(getSingletonTypes().unknownType) + , neverType(getSingletonTypes().neverType) , anyTypePack(getSingletonTypes().anyTypePack) + , neverTypePack(getSingletonTypes().neverTypePack) + , uninhabitableTypePack(getSingletonTypes().uninhabitableTypePack) , duplicateTypeAliases{{false, {}}} { globalScope = std::make_shared(globalTypes.addTypePack(TypePackVar{FreeTypePack{TypeLevel{}}})); @@ -269,6 +276,11 @@ TypeChecker::TypeChecker(ModuleResolver* resolver, InternalErrorReporter* iceHan globalScope->exportedTypeBindings["string"] = TypeFun{{}, stringType}; globalScope->exportedTypeBindings["boolean"] = TypeFun{{}, booleanType}; globalScope->exportedTypeBindings["thread"] = TypeFun{{}, threadType}; + if (FFlag::LuauUnknownAndNeverType) + { + globalScope->exportedTypeBindings["unknown"] = TypeFun{{}, unknownType}; + globalScope->exportedTypeBindings["never"] = TypeFun{{}, neverType}; + } } ModulePtr TypeChecker::check(const SourceModule& module, Mode mode, std::optional environmentScope) @@ -456,6 +468,59 @@ void TypeChecker::checkBlock(const ScopePtr& scope, const AstStatBlock& block) } } +struct InplaceDemoter : TypeVarOnceVisitor +{ + TypeLevel newLevel; + TypeArena* arena; + + InplaceDemoter(TypeLevel level, TypeArena* arena) + : newLevel(level) + , arena(arena) + { + } + + bool demote(TypeId ty) + { + if (auto level = getMutableLevel(ty)) + { + if (level->subsumesStrict(newLevel)) + { + *level = newLevel; + return true; + } + } + + return false; + } + + bool visit(TypeId ty, const BoundTypeVar& btyRef) override + { + return true; + } + + bool visit(TypeId ty) override + { + if (ty->owningArena != arena) + return false; + return demote(ty); + } + + bool visit(TypePackId tp, const FreeTypePack& ftpRef) override + { + if (tp->owningArena != arena) + return false; + + FreeTypePack* ftp = &const_cast(ftpRef); + if (ftp->level.subsumesStrict(newLevel)) + { + ftp->level = newLevel; + return true; + } + + return false; + } +}; + void TypeChecker::checkBlockWithoutRecursionCheck(const ScopePtr& scope, const AstStatBlock& block) { int subLevel = 0; @@ -559,7 +624,7 @@ void TypeChecker::checkBlockWithoutRecursionCheck(const ScopePtr& scope, const A tablify(baseTy); if (!fun->func->self) - expectedType = getIndexTypeFromType(scope, baseTy, name->index.value, name->indexLocation, false); + expectedType = getIndexTypeFromType(scope, baseTy, name->index.value, name->indexLocation, /* addErrors= */ false); else if (auto ttv = getMutableTableType(baseTy)) { if (!baseTy->persistent && ttv->state != TableState::Sealed && !ttv->selfTy) @@ -579,7 +644,7 @@ void TypeChecker::checkBlockWithoutRecursionCheck(const ScopePtr& scope, const A if (auto name = fun->name->as()) { TypeId exprTy = checkExpr(scope, *name->expr).type; - expectedType = getIndexTypeFromType(scope, exprTy, name->index.value, name->indexLocation, false); + expectedType = getIndexTypeFromType(scope, exprTy, name->index.value, name->indexLocation, /* addErrors= */ false); } } } @@ -634,15 +699,8 @@ LUAU_NOINLINE void TypeChecker::checkBlockTypeAliases(const ScopePtr& scope, std TypeId type = bindings[name].type; if (get(follow(type))) { - if (FFlag::LuauNonCopyableTypeVarFields) - { - TypeVar* mty = asMutable(follow(type)); - mty->reassign(*errorRecoveryType(anyType)); - } - else - { - *asMutable(type) = *errorRecoveryType(anyType); - } + TypeVar* mty = asMutable(follow(type)); + mty->reassign(*errorRecoveryType(anyType)); reportError(TypeError{typealias->location, OccursCheckFailed{}}); } @@ -1206,7 +1264,7 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatForIn& forin) iterTy = instantiate(scope, checkExpr(scope, *firstValue).type, firstValue->location); } - if (std::optional iterMM = findMetatableEntry(iterTy, "__iter", firstValue->location)) + if (std::optional iterMM = findMetatableEntry(iterTy, "__iter", firstValue->location, /* addErrors= */ true)) { // if __iter metamethod is present, it will be called and the results are going to be called as if they are functions // TODO: this needs to typecheck all returned values by __iter as if they were for loop arguments @@ -1253,7 +1311,7 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatForIn& forin) for (TypeId var : varTypes) unify(varTy, var, forin.location); - if (!get(iterTy) && !get(iterTy) && !get(iterTy)) + if (!get(iterTy) && !get(iterTy) && !get(iterTy) && !get(iterTy)) reportError(firstValue->location, CannotCallNonFunction{iterTy}); return check(loopScope, *forin.body); @@ -1350,7 +1408,7 @@ void TypeChecker::check(const ScopePtr& scope, TypeId ty, const ScopePtr& funSco TypeId exprTy = checkExpr(scope, *name->expr).type; TableTypeVar* ttv = getMutableTableType(exprTy); - if (!getIndexTypeFromType(scope, exprTy, name->index.value, name->indexLocation, false)) + if (!getIndexTypeFromType(scope, exprTy, name->index.value, name->indexLocation, /* addErrors= */ false)) { if (ttv || isTableIntersection(exprTy)) reportError(TypeError{function.location, CannotExtendTable{exprTy, CannotExtendTable::Property, name->index.value}}); @@ -1376,6 +1434,12 @@ void TypeChecker::check(const ScopePtr& scope, TypeId ty, const ScopePtr& funSco checkFunctionBody(funScope, ty, *function.func); + if (FFlag::LuauUnknownAndNeverType) + { + InplaceDemoter demoter{funScope->level, ¤tModule->internalTypes}; + demoter.traverse(ty); + } + if (ttv && ttv->state != TableState::Sealed) ttv->props[name->index.value] = {follow(quantify(funScope, ty, name->indexLocation)), /* deprecated */ false, {}, name->indexLocation}; } @@ -1729,7 +1793,7 @@ WithPredicate TypeChecker::checkExpr(const ScopePtr& scope, const AstExp else if (auto a = expr.as()) result = checkExpr(scope, *a); else if (auto a = expr.as()) - result = checkExpr(scope, *a); + result = checkExpr(scope, *a, FFlag::LuauBinaryNeedsExpectedTypesToo ? expectedType : std::nullopt); else if (auto a = expr.as()) result = checkExpr(scope, *a); else if (auto a = expr.as()) @@ -1851,41 +1915,56 @@ WithPredicate TypeChecker::checkExpr(const ScopePtr& scope, const AstExp lhsType = stripFromNilAndReport(lhsType, expr.expr->location); - if (std::optional ty = getIndexTypeFromType(scope, lhsType, name, expr.location, true)) + if (std::optional ty = getIndexTypeFromType(scope, lhsType, name, expr.location, /* addErrors= */ true)) return {*ty}; return {errorRecoveryType(scope)}; } -std::optional TypeChecker::findTablePropertyRespectingMeta(TypeId lhsType, Name name, const Location& location) +std::optional TypeChecker::findTablePropertyRespectingMeta(TypeId lhsType, Name name, const Location& location, bool addErrors) { ErrorVec errors; auto result = Luau::findTablePropertyRespectingMeta(errors, lhsType, name, location); - reportErrors(errors); + if (!FFlag::LuauIndexSilenceErrors || addErrors) + reportErrors(errors); return result; } -std::optional TypeChecker::findMetatableEntry(TypeId type, std::string entry, const Location& location) +std::optional TypeChecker::findMetatableEntry(TypeId type, std::string entry, const Location& location, bool addErrors) { ErrorVec errors; auto result = Luau::findMetatableEntry(errors, type, entry, location); - reportErrors(errors); + if (!FFlag::LuauIndexSilenceErrors || addErrors) + reportErrors(errors); return result; } std::optional TypeChecker::getIndexTypeFromType( - const ScopePtr& scope, TypeId type, const std::string& name, const Location& location, bool addErrors) + const ScopePtr& scope, TypeId type, const Name& name, const Location& location, bool addErrors) +{ + size_t errorCount = currentModule->errors.size(); + + std::optional result = getIndexTypeFromTypeImpl(scope, type, name, location, addErrors); + + if (FFlag::LuauIndexSilenceErrors && !addErrors) + LUAU_ASSERT(errorCount == currentModule->errors.size()); + + return result; +} + +std::optional TypeChecker::getIndexTypeFromTypeImpl( + const ScopePtr& scope, TypeId type, const Name& name, const Location& location, bool addErrors) { type = follow(type); - if (get(type) || get(type)) + if (get(type) || get(type) || get(type)) return type; tablify(type); if (isString(type)) { - std::optional mtIndex = findMetatableEntry(stringType, "__index", location); + std::optional mtIndex = findMetatableEntry(stringType, "__index", location, addErrors); LUAU_ASSERT(mtIndex); type = *mtIndex; } @@ -1919,7 +1998,7 @@ std::optional TypeChecker::getIndexTypeFromType( return result; } - if (auto found = findTablePropertyRespectingMeta(type, name, location)) + if (auto found = findTablePropertyRespectingMeta(type, name, location, addErrors)) return *found; } else if (const ClassTypeVar* cls = get(type)) @@ -1941,7 +2020,7 @@ std::optional TypeChecker::getIndexTypeFromType( if (get(follow(t))) return t; - if (std::optional ty = getIndexTypeFromType(scope, t, name, location, false)) + if (std::optional ty = getIndexTypeFromType(scope, t, name, location, /* addErrors= */ false)) goodOptions.push_back(*ty); else badOptions.push_back(t); @@ -1972,6 +2051,8 @@ std::optional TypeChecker::getIndexTypeFromType( else { std::vector result = reduceUnion(goodOptions); + if (FFlag::LuauUnknownAndNeverType && result.empty()) + return neverType; if (result.size() == 1) return result[0]; @@ -1987,7 +2068,7 @@ std::optional TypeChecker::getIndexTypeFromType( { RecursionLimiter _rl(&recursionCount, FInt::LuauTypeInferRecursionLimit); - if (std::optional ty = getIndexTypeFromType(scope, t, name, location, false)) + if (std::optional ty = getIndexTypeFromType(scope, t, name, location, /* addErrors= */ false)) parts.push_back(*ty); } @@ -2017,6 +2098,9 @@ std::vector TypeChecker::reduceUnion(const std::vector& types) for (TypeId t : types) { t = follow(t); + if (get(t)) + continue; + if (get(t) || get(t)) return {t}; @@ -2028,6 +2112,8 @@ std::vector TypeChecker::reduceUnion(const std::vector& types) { if (FFlag::LuauNormalizeFlagIsConservative) ty = follow(ty); + if (get(ty)) + continue; if (get(ty) || get(ty)) return {ty}; @@ -2041,6 +2127,8 @@ std::vector TypeChecker::reduceUnion(const std::vector& types) for (TypeId ty : r) { ty = follow(ty); + if (get(ty)) + continue; if (get(ty) || get(ty)) return {ty}; @@ -2314,14 +2402,14 @@ WithPredicate TypeChecker::checkExpr(const ScopePtr& scope, const AstExp return {booleanType, {NotPredicate{std::move(result.predicates)}}}; case AstExprUnary::Minus: { - const bool operandIsAny = get(operandType) || get(operandType); + const bool operandIsAny = get(operandType) || get(operandType) || get(operandType); if (operandIsAny) return {operandType}; if (typeCouldHaveMetatable(operandType)) { - if (auto fnt = findMetatableEntry(operandType, "__unm", expr.location)) + if (auto fnt = findMetatableEntry(operandType, "__unm", expr.location, /* addErrors= */ true)) { TypeId actualFunctionType = instantiate(scope, *fnt, expr.location); TypePackId arguments = addTypePack({operandType}); @@ -2355,14 +2443,14 @@ WithPredicate TypeChecker::checkExpr(const ScopePtr& scope, const AstExp operandType = stripFromNilAndReport(operandType, expr.location); - if (get(operandType)) - return {errorRecoveryType(scope)}; + if (get(operandType) || get(operandType)) + return {!FFlag::LuauUnknownAndNeverType ? errorRecoveryType(scope) : operandType}; DenseHashSet seen{nullptr}; if (FFlag::LuauCheckLenMT && typeCouldHaveMetatable(operandType)) { - if (auto fnt = findMetatableEntry(operandType, "__len", expr.location)) + if (auto fnt = findMetatableEntry(operandType, "__len", expr.location, /* addErrors= */ true)) { TypeId actualFunctionType = instantiate(scope, *fnt, expr.location); TypePackId arguments = addTypePack({operandType}); @@ -2433,6 +2521,9 @@ TypeId TypeChecker::unionOfTypes(TypeId a, TypeId b, const Location& location, b return a; std::vector types = reduceUnion({a, b}); + if (FFlag::LuauUnknownAndNeverType && types.empty()) + return neverType; + if (types.size() == 1) return types[0]; @@ -2485,7 +2576,7 @@ TypeId TypeChecker::checkRelationalOperation( // If we know nothing at all about the lhs type, we can usually say nothing about the result. // The notable exception to this is the equality and inequality operators, which always produce a boolean. - const bool lhsIsAny = get(lhsType) || get(lhsType); + const bool lhsIsAny = get(lhsType) || get(lhsType) || get(lhsType); // Peephole check for `cond and a or b -> type(a)|type(b)` // TODO: Kill this when singleton types arrive. :( @@ -2508,7 +2599,7 @@ TypeId TypeChecker::checkRelationalOperation( if (isNonstrictMode() && (isNil(lhsType) || isNil(rhsType))) return booleanType; - const bool rhsIsAny = get(rhsType) || get(rhsType); + const bool rhsIsAny = get(rhsType) || get(rhsType) || get(rhsType); if (lhsIsAny || rhsIsAny) return booleanType; @@ -2596,7 +2687,7 @@ TypeId TypeChecker::checkRelationalOperation( if (leftMetatable) { - std::optional metamethod = findMetatableEntry(lhsType, metamethodName, expr.location); + std::optional metamethod = findMetatableEntry(lhsType, metamethodName, expr.location, /* addErrors= */ true); if (metamethod) { if (const FunctionTypeVar* ftv = get(*metamethod)) @@ -2757,9 +2848,9 @@ TypeId TypeChecker::checkBinaryOperation( }; std::string op = opToMetaTableEntry(expr.op); - if (auto fnt = findMetatableEntry(lhsType, op, expr.location)) + if (auto fnt = findMetatableEntry(lhsType, op, expr.location, /* addErrors= */ true)) return checkMetatableCall(*fnt, lhsType, rhsType); - if (auto fnt = findMetatableEntry(rhsType, op, expr.location)) + if (auto fnt = findMetatableEntry(rhsType, op, expr.location, /* addErrors= */ true)) { // Note the intentionally reversed arguments here. return checkMetatableCall(*fnt, rhsType, lhsType); @@ -2793,27 +2884,27 @@ TypeId TypeChecker::checkBinaryOperation( } } -WithPredicate TypeChecker::checkExpr(const ScopePtr& scope, const AstExprBinary& expr) +WithPredicate TypeChecker::checkExpr(const ScopePtr& scope, const AstExprBinary& expr, std::optional expectedType) { if (expr.op == AstExprBinary::And) { - auto [lhsTy, lhsPredicates] = checkExpr(scope, *expr.left); + auto [lhsTy, lhsPredicates] = checkExpr(scope, *expr.left, expectedType); ScopePtr innerScope = childScope(scope, expr.location); resolve(lhsPredicates, innerScope, true); - auto [rhsTy, rhsPredicates] = checkExpr(innerScope, *expr.right); + auto [rhsTy, rhsPredicates] = checkExpr(innerScope, *expr.right, expectedType); return {checkBinaryOperation(scope, expr, lhsTy, rhsTy), {AndPredicate{std::move(lhsPredicates), std::move(rhsPredicates)}}}; } else if (expr.op == AstExprBinary::Or) { - auto [lhsTy, lhsPredicates] = checkExpr(scope, *expr.left); + auto [lhsTy, lhsPredicates] = checkExpr(scope, *expr.left, expectedType); ScopePtr innerScope = childScope(scope, expr.location); resolve(lhsPredicates, innerScope, false); - auto [rhsTy, rhsPredicates] = checkExpr(innerScope, *expr.right); + auto [rhsTy, rhsPredicates] = checkExpr(innerScope, *expr.right, expectedType); // Because of C++, I'm not sure if lhsPredicates was not moved out by the time we call checkBinaryOperation. TypeId result = checkBinaryOperation(scope, expr, lhsTy, rhsTy, lhsPredicates); @@ -2824,6 +2915,8 @@ WithPredicate TypeChecker::checkExpr(const ScopePtr& scope, const AstExp if (auto predicate = tryGetTypeGuardPredicate(expr)) return {booleanType, {std::move(*predicate)}}; + // For these, passing expectedType is worse than simply forcing them, because their implementation + // may inadvertently check if expectedTypes exist first and use it, instead of forceSingleton first. WithPredicate lhs = checkExpr(scope, *expr.left, std::nullopt, /*forceSingleton=*/true); WithPredicate rhs = checkExpr(scope, *expr.right, std::nullopt, /*forceSingleton=*/true); @@ -2842,6 +2935,7 @@ WithPredicate TypeChecker::checkExpr(const ScopePtr& scope, const AstExp } else { + // Expected types are not useful for other binary operators. WithPredicate lhs = checkExpr(scope, *expr.left); WithPredicate rhs = checkExpr(scope, *expr.right); @@ -2896,6 +2990,8 @@ WithPredicate TypeChecker::checkExpr(const ScopePtr& scope, const AstExp return {trueType.type}; std::vector types = reduceUnion({trueType.type, falseType.type}); + if (FFlag::LuauUnknownAndNeverType && types.empty()) + return {neverType}; return {types.size() == 1 ? types[0] : addType(UnionTypeVar{std::move(types)})}; } @@ -2927,7 +3023,10 @@ TypeId TypeChecker::checkLValueBinding(const ScopePtr& scope, const AstExpr& exp TypeId TypeChecker::checkLValueBinding(const ScopePtr& scope, const AstExprLocal& expr) { if (std::optional ty = scope->lookup(expr.local)) - return *ty; + { + ty = follow(*ty); + return get(*ty) ? unknownType : *ty; + } reportError(expr.location, UnknownSymbol{expr.local->name.value, UnknownSymbol::Binding}); return errorRecoveryType(scope); @@ -2941,7 +3040,10 @@ TypeId TypeChecker::checkLValueBinding(const ScopePtr& scope, const AstExprGloba const auto it = moduleScope->bindings.find(expr.name); if (it != moduleScope->bindings.end()) - return it->second.typeId; + { + TypeId ty = follow(it->second.typeId); + return get(ty) ? unknownType : ty; + } TypeId result = freshType(scope); Binding& binding = moduleScope->bindings[expr.name]; @@ -2962,6 +3064,9 @@ TypeId TypeChecker::checkLValueBinding(const ScopePtr& scope, const AstExprIndex if (get(lhs) || get(lhs)) return lhs; + if (get(lhs)) + return unknownType; + tablify(lhs); Name name = expr.index.value; @@ -3023,7 +3128,7 @@ TypeId TypeChecker::checkLValueBinding(const ScopePtr& scope, const AstExprIndex } else if (get(lhs)) { - if (std::optional ty = getIndexTypeFromType(scope, lhs, name, expr.location, false)) + if (std::optional ty = getIndexTypeFromType(scope, lhs, name, expr.location, /* addErrors= */ false)) return *ty; // If intersection has a table part, report that it cannot be extended just as a sealed table @@ -3050,6 +3155,9 @@ TypeId TypeChecker::checkLValueBinding(const ScopePtr& scope, const AstExprIndex if (get(exprType) || get(exprType)) return exprType; + if (get(exprType)) + return unknownType; + AstExprConstantString* value = expr.index->as(); if (value) @@ -3156,7 +3264,7 @@ TypeId TypeChecker::checkFunctionName(const ScopePtr& scope, AstExpr& funName, T if (!ttv || ttv->state == TableState::Sealed) { - if (auto ty = getIndexTypeFromType(scope, lhsType, indexName->index.value, indexName->indexLocation, false)) + if (auto ty = getIndexTypeFromType(scope, lhsType, indexName->index.value, indexName->indexLocation, /* addErrors= */ false)) return *ty; return errorRecoveryType(scope); @@ -3228,9 +3336,12 @@ std::pair TypeChecker::checkFunctionSignature(const ScopePtr& } } - // We do not infer type binders, so if a generic function is required we do not propagate - if (expectedFunctionType && !(expectedFunctionType->generics.empty() && expectedFunctionType->genericPacks.empty())) - expectedFunctionType = nullptr; + if (!FFlag::LuauCheckGenericHOFTypes) + { + // We do not infer type binders, so if a generic function is required we do not propagate + if (expectedFunctionType && !(expectedFunctionType->generics.empty() && expectedFunctionType->genericPacks.empty())) + expectedFunctionType = nullptr; + } } auto [generics, genericPacks] = createGenericTypes(funScope, std::nullopt, expr, expr.generics, expr.genericPacks); @@ -3240,7 +3351,8 @@ std::pair TypeChecker::checkFunctionSignature(const ScopePtr& retPack = resolveTypePack(funScope, *expr.returnAnnotation); else if (FFlag::LuauReturnTypeInferenceInNonstrict ? (!FFlag::LuauLowerBoundsCalculation && isNonstrictMode()) : isNonstrictMode()) retPack = anyTypePack; - else if (expectedFunctionType) + else if (expectedFunctionType && + (!FFlag::LuauCheckGenericHOFTypes || (expectedFunctionType->generics.empty() && expectedFunctionType->genericPacks.empty()))) { auto [head, tail] = flatten(expectedFunctionType->retTypes); @@ -3371,16 +3483,50 @@ std::pair TypeChecker::checkFunctionSignature(const ScopePtr& defn.originalNameLocation = originalName.value_or(Location(expr.location.begin, 0)); std::vector genericTys; - genericTys.reserve(generics.size()); - std::transform(generics.begin(), generics.end(), std::back_inserter(genericTys), [](auto&& el) { - return el.ty; - }); + // if we have a generic expected function type and no generics, we should use the expected ones. + if (FFlag::LuauCheckGenericHOFTypes) + { + if (expectedFunctionType && generics.empty()) + { + genericTys = expectedFunctionType->generics; + } + else + { + genericTys.reserve(generics.size()); + for (const GenericTypeDefinition& generic : generics) + genericTys.push_back(generic.ty); + } + } + else + { + genericTys.reserve(generics.size()); + std::transform(generics.begin(), generics.end(), std::back_inserter(genericTys), [](auto&& el) { + return el.ty; + }); + } std::vector genericTps; - genericTps.reserve(genericPacks.size()); - std::transform(genericPacks.begin(), genericPacks.end(), std::back_inserter(genericTps), [](auto&& el) { - return el.tp; - }); + // if we have a generic expected function type and no generic typepacks, we should use the expected ones. + if (FFlag::LuauCheckGenericHOFTypes) + { + if (expectedFunctionType && genericPacks.empty()) + { + genericTps = expectedFunctionType->genericPacks; + } + else + { + genericTps.reserve(genericPacks.size()); + for (const GenericTypePackDefinition& generic : genericPacks) + genericTps.push_back(generic.tp); + } + } + else + { + genericTps.reserve(genericPacks.size()); + std::transform(genericPacks.begin(), genericPacks.end(), std::back_inserter(genericTps), [](auto&& el) { + return el.tp; + }); + } TypeId funTy = addType(FunctionTypeVar(funScope->level, std::move(genericTys), std::move(genericTps), argPack, retPack, std::move(defn), bool(expr.self))); @@ -3474,9 +3620,22 @@ void TypeChecker::checkFunctionBody(const ScopePtr& scope, TypeId ty, const AstE } WithPredicate TypeChecker::checkExprPack(const ScopePtr& scope, const AstExpr& expr) +{ + if (FFlag::LuauUnknownAndNeverType) + { + WithPredicate result = checkExprPackHelper(scope, expr); + if (containsNever(result.type)) + return {uninhabitableTypePack}; + return result; + } + else + return checkExprPackHelper(scope, expr); +} + +WithPredicate TypeChecker::checkExprPackHelper(const ScopePtr& scope, const AstExpr& expr) { if (auto a = expr.as()) - return checkExprPack(scope, *a); + return checkExprPackHelper(scope, *a); else if (expr.is()) { if (!scope->varargPack) @@ -3739,7 +3898,7 @@ void TypeChecker::checkArgumentList( } } -WithPredicate TypeChecker::checkExprPack(const ScopePtr& scope, const AstExprCall& expr) +WithPredicate TypeChecker::checkExprPackHelper(const ScopePtr& scope, const AstExprCall& expr) { // evaluate type of function // decompose an intersection into its component overloads @@ -3763,7 +3922,7 @@ WithPredicate TypeChecker::checkExprPack(const ScopePtr& scope, cons selfType = checkExpr(scope, *indexExpr->expr).type; selfType = stripFromNilAndReport(selfType, expr.func->location); - if (std::optional propTy = getIndexTypeFromType(scope, selfType, indexExpr->index.value, expr.location, true)) + if (std::optional propTy = getIndexTypeFromType(scope, selfType, indexExpr->index.value, expr.location, /* addErrors= */ true)) { functionType = *propTy; actualFunctionType = instantiate(scope, functionType, expr.func->location); @@ -3813,11 +3972,25 @@ WithPredicate TypeChecker::checkExprPack(const ScopePtr& scope, cons if (get(argPack)) return {errorRecoveryTypePack(scope)}; - TypePack* args = getMutable(argPack); - LUAU_ASSERT(args != nullptr); + TypePack* args = nullptr; + if (FFlag::LuauUnknownAndNeverType) + { + if (expr.self) + { + argPack = addTypePack(TypePack{{selfType}, argPack}); + argListResult.type = argPack; + } + args = getMutable(argPack); + LUAU_ASSERT(args); + } + else + { + args = getMutable(argPack); + LUAU_ASSERT(args != nullptr); - if (expr.self) - args->head.insert(args->head.begin(), selfType); + if (expr.self) + args->head.insert(args->head.begin(), selfType); + } std::vector argLocations; argLocations.reserve(expr.args.size + 1); @@ -3876,7 +4049,10 @@ std::vector> TypeChecker::getExpectedTypesForCall(const st else { std::vector result = reduceUnion({*el, ty}); - el = result.size() == 1 ? result[0] : addType(UnionTypeVar{std::move(result)}); + if (FFlag::LuauUnknownAndNeverType && result.empty()) + el = neverType; + else + el = result.size() == 1 ? result[0] : addType(UnionTypeVar{std::move(result)}); } } }; @@ -3930,6 +4106,9 @@ std::optional> TypeChecker::checkCallOverload(const Sc return {{errorRecoveryTypePack(scope)}}; } + if (get(fn)) + return {{uninhabitableTypePack}}; + if (auto ftv = get(fn)) { // fn is one of the overloads of actualFunctionType, which @@ -3975,7 +4154,7 @@ std::optional> TypeChecker::checkCallOverload(const Sc // Might be a callable table if (const MetatableTypeVar* mttv = get(fn)) { - if (std::optional ty = getIndexTypeFromType(scope, mttv->metatable, "__call", expr.func->location, false)) + if (std::optional ty = getIndexTypeFromType(scope, mttv->metatable, "__call", expr.func->location, /* addErrors= */ false)) { // Construct arguments with 'self' added in front TypePackId metaCallArgPack = addTypePack(TypePackVar(TypePack{args->head, args->tail})); @@ -4202,6 +4381,7 @@ void TypeChecker::reportOverloadResolutionError(const ScopePtr& scope, const Ast WithPredicate TypeChecker::checkExprList(const ScopePtr& scope, const Location& location, const AstArray& exprs, bool substituteFreeForNil, const std::vector& instantiateGenerics, const std::vector>& expectedTypes) { + bool uninhabitable = false; TypePackId pack = addTypePack(TypePack{}); PredicateVec predicates; // At the moment we will be pushing all predicate sets into this. Do we need some way to split them up? @@ -4232,7 +4412,13 @@ WithPredicate TypeChecker::checkExprList(const ScopePtr& scope, cons auto [typePack, exprPredicates] = checkExprPack(scope, *expr); insert(exprPredicates); - if (std::optional firstTy = first(typePack)) + if (FFlag::LuauUnknownAndNeverType && containsNever(typePack)) + { + // f(), g() where f() returns (never, string) or (string, never) means this whole TypePackId is uninhabitable, so return (never, ...never) + uninhabitable = true; + continue; + } + else if (std::optional firstTy = first(typePack)) { if (!currentModule->astTypes.find(expr)) currentModule->astTypes[expr] = follow(*firstTy); @@ -4248,6 +4434,13 @@ WithPredicate TypeChecker::checkExprList(const ScopePtr& scope, cons auto [type, exprPredicates] = checkExpr(scope, *expr, expectedType); insert(exprPredicates); + if (FFlag::LuauUnknownAndNeverType && get(type)) + { + // f(), g() where f() returns (never, string) or (string, never) means this whole TypePackId is uninhabitable, so return (never, ...never) + uninhabitable = true; + continue; + } + TypeId actualType = substituteFreeForNil && expr->is() ? freshType(scope) : type; if (instantiateGenerics.size() > i && instantiateGenerics[i]) @@ -4272,6 +4465,8 @@ WithPredicate TypeChecker::checkExprList(const ScopePtr& scope, cons for (TxnLog& log : inverseLogs) log.commit(); + if (FFlag::LuauUnknownAndNeverType && uninhabitable) + return {uninhabitableTypePack}; return {pack, predicates}; } @@ -4830,7 +5025,7 @@ TypeIdPredicate TypeChecker::mkTruthyPredicate(bool sense) }; } -std::optional TypeChecker::filterMap(TypeId type, TypeIdPredicate predicate) +std::optional TypeChecker::filterMapImpl(TypeId type, TypeIdPredicate predicate) { std::vector types = Luau::filterMap(type, predicate); if (!types.empty()) @@ -4838,7 +5033,21 @@ std::optional TypeChecker::filterMap(TypeId type, TypeIdPredicate predic return std::nullopt; } -std::optional TypeChecker::pickTypesFromSense(TypeId type, bool sense) +std::pair, bool> TypeChecker::filterMap(TypeId type, TypeIdPredicate predicate) +{ + if (FFlag::LuauUnknownAndNeverType) + { + TypeId ty = filterMapImpl(type, predicate).value_or(neverType); + return {ty, !bool(get(ty))}; + } + else + { + std::optional ty = filterMapImpl(type, predicate); + return {ty, bool(ty)}; + } +} + +std::pair, bool> TypeChecker::pickTypesFromSense(TypeId type, bool sense) { return filterMap(type, mkTruthyPredicate(sense)); } @@ -5465,10 +5674,18 @@ void TypeChecker::refineLValue(const LValue& lvalue, RefinementMap& refis, const // If we do not have a key, it means we're not trying to discriminate anything, so it's a simple matter of just filtering for a subset. if (!key) { - if (std::optional result = filterMap(*ty, predicate)) + auto [result, ok] = filterMap(*ty, predicate); + if (FFlag::LuauUnknownAndNeverType) + { addRefinement(refis, *target, *result); + } else - addRefinement(refis, *target, errorRecoveryType(scope)); + { + if (ok) + addRefinement(refis, *target, *result); + else + addRefinement(refis, *target, errorRecoveryType(scope)); + } return; } @@ -5484,17 +5701,29 @@ void TypeChecker::refineLValue(const LValue& lvalue, RefinementMap& refis, const { std::optional discriminantTy; if (auto field = Luau::get(*key)) // need to fully qualify Luau::get because of ADL. - discriminantTy = getIndexTypeFromType(scope, option, field->key, Location(), false); + discriminantTy = getIndexTypeFromType(scope, option, field->key, Location(), /* addErrors= */ false); else LUAU_ASSERT(!"Unhandled LValue alternative?"); if (!discriminantTy) return; // Do nothing. An error was already reported, as per usual. - if (std::optional result = filterMap(*discriminantTy, predicate)) + auto [result, ok] = filterMap(*discriminantTy, predicate); + if (FFlag::LuauUnknownAndNeverType) { - viableTargetOptions.insert(option); - viableChildOptions.insert(*result); + if (!get(*result)) + { + viableTargetOptions.insert(option); + viableChildOptions.insert(*result); + } + } + else + { + if (ok) + { + viableTargetOptions.insert(option); + viableChildOptions.insert(*result); + } } } @@ -5573,7 +5802,7 @@ std::optional TypeChecker::resolveLValue(const ScopePtr& scope, const LV continue; else if (auto field = get(key)) { - found = getIndexTypeFromType(scope, *found, field->key, Location(), false); + found = getIndexTypeFromType(scope, *found, field->key, Location(), /* addErrors= */ false); if (!found) return std::nullopt; // Turns out this type doesn't have the property at all. We're done. } @@ -5753,6 +5982,9 @@ void TypeChecker::resolve(const TypeGuardPredicate& typeguardP, RefinementMap& r auto mkFilter = [](ConditionFunc f, std::optional other = std::nullopt) -> SenseToTypeIdPredicate { return [f, other](bool sense) -> TypeIdPredicate { return [f, other, sense](TypeId ty) -> std::optional { + if (FFlag::LuauUnknownAndNeverType && sense && get(ty)) + return other.value_or(ty); + if (f(ty) == sense) return ty; @@ -5860,8 +6092,15 @@ std::vector TypeChecker::unTypePack(const ScopePtr& scope, TypePackId tp for (size_t i = 0; i < expectedLength; ++i) expectedPack->head.push_back(freshType(scope)); + size_t oldErrorsSize = currentModule->errors.size(); + unify(tp, expectedTypePack, location); + // HACK: tryUnify would undo the changes to the expectedTypePack if the length mismatches, but + // we want to tie up free types to be error types, so we do this instead. + if (FFlag::LuauUnknownAndNeverType) + currentModule->errors.resize(oldErrorsSize); + for (TypeId& tp : expectedPack->head) tp = follow(tp); diff --git a/Analysis/src/TypePack.cpp b/Analysis/src/TypePack.cpp index 82451bd1..d4544483 100644 --- a/Analysis/src/TypePack.cpp +++ b/Analysis/src/TypePack.cpp @@ -5,8 +5,6 @@ #include -LUAU_FASTFLAG(LuauNonCopyableTypeVarFields) - namespace Luau { @@ -40,19 +38,10 @@ TypePackVar& TypePackVar::operator=(TypePackVariant&& tp) TypePackVar& TypePackVar::operator=(const TypePackVar& rhs) { - if (FFlag::LuauNonCopyableTypeVarFields) - { - LUAU_ASSERT(owningArena == rhs.owningArena); - LUAU_ASSERT(!rhs.persistent); + LUAU_ASSERT(owningArena == rhs.owningArena); + LUAU_ASSERT(!rhs.persistent); - reassign(rhs); - } - else - { - ty = rhs.ty; - persistent = rhs.persistent; - owningArena = rhs.owningArena; - } + reassign(rhs); return *this; } @@ -294,6 +283,16 @@ std::optional first(TypePackId tp, bool ignoreHiddenVariadics) return std::nullopt; } +TypePackVar* asMutable(TypePackId tp) +{ + return const_cast(tp); +} + +TypePack* asMutable(const TypePack* tp) +{ + return const_cast(tp); +} + bool isEmpty(TypePackId tp) { tp = follow(tp); @@ -360,13 +359,25 @@ bool isVariadic(TypePackId tp, const TxnLog& log) return false; } -TypePackVar* asMutable(TypePackId tp) +bool containsNever(TypePackId tp) { - return const_cast(tp); + auto it = begin(tp); + auto endIt = end(tp); + + while (it != endIt) + { + if (get(follow(*it))) + return true; + ++it; + } + + if (auto tail = it.tail()) + { + if (auto vtp = get(*tail); vtp && get(follow(vtp->ty))) + return true; + } + + return false; } -TypePack* asMutable(const TypePack* tp) -{ - return const_cast(tp); -} } // namespace Luau diff --git a/Analysis/src/TypeUtils.cpp b/Analysis/src/TypeUtils.cpp index 3d97e6eb..66b38cf3 100644 --- a/Analysis/src/TypeUtils.cpp +++ b/Analysis/src/TypeUtils.cpp @@ -24,7 +24,7 @@ std::optional findMetatableEntry(ErrorVec& errors, TypeId type, std::str const TableTypeVar* mtt = getTableType(unwrapped); if (!mtt) { - errors.push_back(TypeError{location, GenericError{"Metatable was not a table."}}); + errors.push_back(TypeError{location, GenericError{"Metatable was not a table"}}); return std::nullopt; } diff --git a/Analysis/src/TypeVar.cpp b/Analysis/src/TypeVar.cpp index ade70d72..f884ad77 100644 --- a/Analysis/src/TypeVar.cpp +++ b/Analysis/src/TypeVar.cpp @@ -23,7 +23,9 @@ LUAU_FASTFLAG(DebugLuauFreezeArena) LUAU_FASTINTVARIABLE(LuauTypeMaximumStringifierLength, 500) LUAU_FASTINTVARIABLE(LuauTableTypeMaximumStringifierLength, 0) LUAU_FASTINT(LuauTypeInferRecursionLimit) -LUAU_FASTFLAG(LuauNonCopyableTypeVarFields) +LUAU_FASTFLAG(LuauUnknownAndNeverType) +LUAU_FASTFLAGVARIABLE(LuauDeduceGmatchReturnTypes, false) +LUAU_FASTFLAGVARIABLE(LuauMaybeGenericIntersectionTypes, false) namespace Luau { @@ -31,6 +33,9 @@ namespace Luau std::optional> magicFunctionFormat( TypeChecker& typechecker, const ScopePtr& scope, const AstExprCall& expr, WithPredicate withPredicate); +static std::optional> magicFunctionGmatch( + TypeChecker& typechecker, const ScopePtr& scope, const AstExprCall& expr, WithPredicate withPredicate); + TypeId follow(TypeId t) { return follow(t, [](TypeId t) { @@ -173,8 +178,8 @@ bool maybeString(TypeId ty) { ty = follow(ty); - if (isPrim(ty, PrimitiveTypeVar::String) || get(ty)) - return true; + if (isPrim(ty, PrimitiveTypeVar::String) || get(ty)) + return true; if (auto utv = get(ty)) return std::any_of(begin(utv), end(utv), maybeString); @@ -194,7 +199,7 @@ bool isOptional(TypeId ty) ty = follow(ty); - if (get(ty)) + if (get(ty) || (FFlag::LuauUnknownAndNeverType && get(ty))) return true; auto utv = get(ty); @@ -334,6 +339,28 @@ bool isGeneric(TypeId ty) bool maybeGeneric(TypeId ty) { + if (FFlag::LuauMaybeGenericIntersectionTypes) + { + ty = follow(ty); + + if (get(ty)) + return true; + + if (auto ttv = get(ty)) + { + // TODO: recurse on table types CLI-39914 + (void)ttv; + return true; + } + + if (auto itv = get(ty)) + { + return std::any_of(begin(itv), end(itv), maybeGeneric); + } + + return isGeneric(ty); + } + ty = follow(ty); if (get(ty)) return true; @@ -646,20 +673,10 @@ TypeVar& TypeVar::operator=(TypeVariant&& rhs) TypeVar& TypeVar::operator=(const TypeVar& rhs) { - if (FFlag::LuauNonCopyableTypeVarFields) - { - LUAU_ASSERT(owningArena == rhs.owningArena); - LUAU_ASSERT(!rhs.persistent); + LUAU_ASSERT(owningArena == rhs.owningArena); + LUAU_ASSERT(!rhs.persistent); - reassign(rhs); - } - else - { - ty = rhs.ty; - persistent = rhs.persistent; - normal = rhs.normal; - owningArena = rhs.owningArena; - } + reassign(rhs); return *this; } @@ -676,10 +693,14 @@ static TypeVar threadType_{PrimitiveTypeVar{PrimitiveTypeVar::Thread}, /*persist static TypeVar trueType_{SingletonTypeVar{BooleanSingleton{true}}, /*persistent*/ true}; static TypeVar falseType_{SingletonTypeVar{BooleanSingleton{false}}, /*persistent*/ true}; static TypeVar anyType_{AnyTypeVar{}, /*persistent*/ true}; +static TypeVar unknownType_{UnknownTypeVar{}, /*persistent*/ true}; +static TypeVar neverType_{NeverTypeVar{}, /*persistent*/ true}; static TypeVar errorType_{ErrorTypeVar{}, /*persistent*/ true}; -static TypePackVar anyTypePack_{VariadicTypePack{&anyType_}, true}; -static TypePackVar errorTypePack_{Unifiable::Error{}}; +static TypePackVar anyTypePack_{VariadicTypePack{&anyType_}, /*persistent*/ true}; +static TypePackVar errorTypePack_{Unifiable::Error{}, /*persistent*/ true}; +static TypePackVar neverTypePack_{VariadicTypePack{&neverType_}, /*persistent*/ true}; +static TypePackVar uninhabitableTypePack_{TypePack{{&neverType_}, &neverTypePack_}, /*persistent*/ true}; SingletonTypes::SingletonTypes() : nilType(&nilType_) @@ -690,7 +711,11 @@ SingletonTypes::SingletonTypes() , trueType(&trueType_) , falseType(&falseType_) , anyType(&anyType_) + , unknownType(&unknownType_) + , neverType(&neverType_) , anyTypePack(&anyTypePack_) + , neverTypePack(&neverTypePack_) + , uninhabitableTypePack(&uninhabitableTypePack_) , arena(new TypeArena) { TypeId stringMetatable = makeStringMetatable(); @@ -738,6 +763,7 @@ TypeId SingletonTypes::makeStringMetatable() const TypeId gsubFunc = makeFunction(*arena, stringType, {}, {}, {stringType, replArgType, optionalNumber}, {}, {stringType, numberType}); const TypeId gmatchFunc = makeFunction(*arena, stringType, {}, {}, {stringType}, {}, {arena->addType(FunctionTypeVar{emptyPack, stringVariadicList})}); + attachMagicFunction(gmatchFunc, magicFunctionGmatch); TableTypeVar::Props stringLib = { {"byte", {arena->addType(FunctionTypeVar{arena->addTypePack({stringType, optionalNumber, optionalNumber}), numberVariadicList})}}, @@ -911,6 +937,8 @@ const TypeLevel* getLevel(TypeId ty) return &ttv->level; else if (auto ftv = get(ty)) return &ftv->level; + else if (auto ctv = get(ty)) + return &ctv->level; else return nullptr; } @@ -965,94 +993,19 @@ bool isSubclass(const ClassTypeVar* cls, const ClassTypeVar* parent) return false; } -UnionTypeVarIterator::UnionTypeVarIterator(const UnionTypeVar* utv) +const std::vector& getTypes(const UnionTypeVar* utv) { - LUAU_ASSERT(utv); - - if (!utv->options.empty()) - stack.push_front({utv, 0}); - - seen.insert(utv); + return utv->options; } -UnionTypeVarIterator& UnionTypeVarIterator::operator++() +const std::vector& getTypes(const IntersectionTypeVar* itv) { - advance(); - descend(); - return *this; + return itv->parts; } -UnionTypeVarIterator UnionTypeVarIterator::operator++(int) +const std::vector& getTypes(const ConstrainedTypeVar* ctv) { - UnionTypeVarIterator copy = *this; - ++copy; - return copy; -} - -bool UnionTypeVarIterator::operator!=(const UnionTypeVarIterator& rhs) -{ - return !(*this == rhs); -} - -bool UnionTypeVarIterator::operator==(const UnionTypeVarIterator& rhs) -{ - if (!stack.empty() && !rhs.stack.empty()) - return stack.front() == rhs.stack.front(); - - return stack.empty() && rhs.stack.empty(); -} - -const TypeId& UnionTypeVarIterator::operator*() -{ - LUAU_ASSERT(!stack.empty()); - - descend(); - - auto [utv, currentIndex] = stack.front(); - LUAU_ASSERT(utv); - LUAU_ASSERT(currentIndex < utv->options.size()); - - const TypeId& ty = utv->options[currentIndex]; - LUAU_ASSERT(!get(follow(ty))); - return ty; -} - -void UnionTypeVarIterator::advance() -{ - while (!stack.empty()) - { - auto& [utv, currentIndex] = stack.front(); - ++currentIndex; - - if (currentIndex >= utv->options.size()) - stack.pop_front(); - else - break; - } -} - -void UnionTypeVarIterator::descend() -{ - while (!stack.empty()) - { - auto [utv, currentIndex] = stack.front(); - if (auto innerUnion = get(follow(utv->options[currentIndex]))) - { - // If we're about to descend into a cyclic UnionTypeVar, we should skip over this. - // Ideally this should never happen, but alas it does from time to time. :( - if (seen.find(innerUnion) != seen.end()) - advance(); - else - { - seen.insert(innerUnion); - stack.push_front({innerUnion, 0}); - } - - continue; - } - - break; - } + return ctv->parts; } UnionTypeVarIterator begin(const UnionTypeVar* utv) @@ -1065,6 +1018,27 @@ UnionTypeVarIterator end(const UnionTypeVar* utv) return UnionTypeVarIterator{}; } +IntersectionTypeVarIterator begin(const IntersectionTypeVar* itv) +{ + return IntersectionTypeVarIterator{itv}; +} + +IntersectionTypeVarIterator end(const IntersectionTypeVar* itv) +{ + return IntersectionTypeVarIterator{}; +} + +ConstrainedTypeVarIterator begin(const ConstrainedTypeVar* ctv) +{ + return ConstrainedTypeVarIterator{ctv}; +} + +ConstrainedTypeVarIterator end(const ConstrainedTypeVar* ctv) +{ + return ConstrainedTypeVarIterator{}; +} + + static std::vector parseFormatString(TypeChecker& typechecker, const char* data, size_t size) { const char* options = "cdiouxXeEfgGqs"; @@ -1144,6 +1118,101 @@ std::optional> magicFunctionFormat( return WithPredicate{arena.addTypePack({typechecker.stringType})}; } +static std::vector parsePatternString(TypeChecker& typechecker, const char* data, size_t size) +{ + std::vector result; + int depth = 0; + bool parsingSet = false; + + for (size_t i = 0; i < size; ++i) + { + if (data[i] == '%') + { + ++i; + if (!parsingSet && i < size && data[i] == 'b') + i += 2; + } + else if (!parsingSet && data[i] == '[') + { + parsingSet = true; + if (i + 1 < size && data[i + 1] == ']') + i += 1; + } + else if (parsingSet && data[i] == ']') + { + parsingSet = false; + } + else if (data[i] == '(') + { + if (parsingSet) + continue; + + if (i + 1 < size && data[i + 1] == ')') + { + i++; + result.push_back(typechecker.numberType); + continue; + } + + ++depth; + result.push_back(typechecker.stringType); + } + else if (data[i] == ')') + { + if (parsingSet) + continue; + + --depth; + + if (depth < 0) + break; + } + } + + if (depth != 0 || parsingSet) + return std::vector(); + + if (result.empty()) + result.push_back(typechecker.stringType); + + return result; +} + +static std::optional> magicFunctionGmatch( + TypeChecker& typechecker, const ScopePtr& scope, const AstExprCall& expr, WithPredicate withPredicate) +{ + if (!FFlag::LuauDeduceGmatchReturnTypes) + return std::nullopt; + + auto [paramPack, _predicates] = withPredicate; + const auto& [params, tail] = flatten(paramPack); + + if (params.size() != 2) + return std::nullopt; + + TypeArena& arena = typechecker.currentModule->internalTypes; + + AstExprConstantString* pattern = nullptr; + size_t index = expr.self ? 0 : 1; + if (expr.args.size > index) + pattern = expr.args.data[index]->as(); + + if (!pattern) + return std::nullopt; + + std::vector returnTypes = parsePatternString(typechecker, pattern->value.data, pattern->value.size); + + if (returnTypes.empty()) + return std::nullopt; + + typechecker.unify(params[0], typechecker.stringType, expr.args.data[0]->location); + + const TypePackId emptyPack = arena.addTypePack({}); + const TypePackId returnList = arena.addTypePack(returnTypes); + const TypeId iteratorType = arena.addType(FunctionTypeVar{emptyPack, returnList}); + return WithPredicate{arena.addTypePack({iteratorType})}; +} + std::vector filterMap(TypeId type, TypeIdPredicate predicate) { type = follow(type); diff --git a/Analysis/src/Unifier.cpp b/Analysis/src/Unifier.cpp index 0792a350..44a3b854 100644 --- a/Analysis/src/Unifier.cpp +++ b/Analysis/src/Unifier.cpp @@ -19,6 +19,7 @@ LUAU_FASTFLAG(LuauAutocompleteDynamicLimits) LUAU_FASTINTVARIABLE(LuauTypeInferLowerBoundsIterationLimit, 2000); LUAU_FASTFLAG(LuauLowerBoundsCalculation); LUAU_FASTFLAG(LuauErrorRecoveryType); +LUAU_FASTFLAG(LuauUnknownAndNeverType) LUAU_FASTFLAG(LuauQuantifyConstrained) namespace Luau @@ -47,33 +48,6 @@ struct PromoteTypeLevels final : TypeVarOnceVisitor } } - // TODO cycle and operator() need to be clipped when FFlagLuauUseVisitRecursionLimit is clipped - template - void cycle(TID) - { - } - template - bool operator()(TID ty, const T&) - { - return visit(ty); - } - bool operator()(TypeId ty, const FreeTypeVar& ftv) - { - return visit(ty, ftv); - } - bool operator()(TypeId ty, const FunctionTypeVar& ftv) - { - return visit(ty, ftv); - } - bool operator()(TypeId ty, const TableTypeVar& ttv) - { - return visit(ty, ttv); - } - bool operator()(TypePackId tp, const FreeTypePack& ftp) - { - return visit(tp, ftp); - } - bool visit(TypeId ty) override { // Type levels of types from other modules are already global, so we don't need to promote anything inside @@ -103,6 +77,15 @@ struct PromoteTypeLevels final : TypeVarOnceVisitor return true; } + bool visit(TypeId ty, const ConstrainedTypeVar&) override + { + if (!FFlag::LuauUnknownAndNeverType) + return visit(ty); + + promote(ty, log.getMutable(ty)); + return true; + } + bool visit(TypeId ty, const FunctionTypeVar&) override { // Type levels of types from other modules are already global, so we don't need to promote anything inside @@ -445,6 +428,14 @@ void Unifier::tryUnify_(TypeId subTy, TypeId superTy, bool isFunctionCall, bool } else if (subFree) { + if (FFlag::LuauUnknownAndNeverType) + { + // Normally, if the subtype is free, it should not be bound to any, unknown, or error types. + // But for bug compatibility, we'll only apply this rule to unknown. Doing this will silence cascading type errors. + if (get(superTy)) + return; + } + TypeLevel subLevel = subFree->level; occursCheck(subTy, superTy); @@ -468,7 +459,7 @@ void Unifier::tryUnify_(TypeId subTy, TypeId superTy, bool isFunctionCall, bool return; } - if (get(superTy) || get(superTy)) + if (get(superTy) || get(superTy) || get(superTy)) return tryUnifyWithAny(subTy, superTy); if (get(subTy)) @@ -485,6 +476,9 @@ void Unifier::tryUnify_(TypeId subTy, TypeId superTy, bool isFunctionCall, bool if (get(subTy)) return tryUnifyWithAny(superTy, subTy); + if (get(subTy)) + return tryUnifyWithAny(superTy, subTy); + auto& cache = sharedState.cachedUnify; // What if the types are immutable and we proved their relation before @@ -1862,6 +1856,7 @@ static void tryUnifyWithAny(std::vector& queue, Unifier& state, DenseHas if (state.log.getMutable(ty)) { + // TODO: Only bind if the anyType isn't any, unknown, or error (?) state.log.replace(ty, BoundTypeVar{anyType}); } else if (auto fun = state.log.getMutable(ty)) @@ -1901,22 +1896,27 @@ static void tryUnifyWithAny(std::vector& queue, Unifier& state, DenseHas void Unifier::tryUnifyWithAny(TypeId subTy, TypeId anyTy) { - LUAU_ASSERT(get(anyTy) || get(anyTy)); + LUAU_ASSERT(get(anyTy) || get(anyTy) || get(anyTy) || get(anyTy)); // These types are not visited in general loop below if (get(subTy) || get(subTy) || get(subTy)) return; - const TypePackId anyTypePack = types->addTypePack(TypePackVar{VariadicTypePack{getSingletonTypes().anyType}}); - - const TypePackId anyTP = get(anyTy) ? anyTypePack : types->addTypePack(TypePackVar{Unifiable::Error{}}); + TypePackId anyTp; + if (FFlag::LuauUnknownAndNeverType) + anyTp = types->addTypePack(TypePackVar{VariadicTypePack{anyTy}}); + else + { + const TypePackId anyTypePack = types->addTypePack(TypePackVar{VariadicTypePack{getSingletonTypes().anyType}}); + anyTp = get(anyTy) ? anyTypePack : types->addTypePack(TypePackVar{Unifiable::Error{}}); + } std::vector queue = {subTy}; sharedState.tempSeenTy.clear(); sharedState.tempSeenTp.clear(); - Luau::tryUnifyWithAny(queue, *this, sharedState.tempSeenTy, sharedState.tempSeenTp, types, getSingletonTypes().anyType, anyTP); + Luau::tryUnifyWithAny(queue, *this, sharedState.tempSeenTy, sharedState.tempSeenTp, types, FFlag::LuauUnknownAndNeverType ? anyTy : getSingletonTypes().anyType, anyTp); } void Unifier::tryUnifyWithAny(TypePackId subTy, TypePackId anyTp) diff --git a/Ast/src/Parser.cpp b/Ast/src/Parser.cpp index 70c92555..b7fa7889 100644 --- a/Ast/src/Parser.cpp +++ b/Ast/src/Parser.cpp @@ -15,7 +15,6 @@ LUAU_FASTINTVARIABLE(LuauRecursionLimit, 1000) LUAU_FASTINTVARIABLE(LuauParseErrorLimit, 100) LUAU_FASTFLAGVARIABLE(LuauParserFunctionKeywordAsTypeHelp, false) -LUAU_FASTFLAGVARIABLE(LuauReturnTypeTokenConfusion, false) LUAU_FASTFLAGVARIABLE(LuauFixNamedFunctionParse, false) LUAU_DYNAMIC_FASTFLAGVARIABLE(LuaReportParseWrongNamedType, false) @@ -1134,10 +1133,9 @@ AstTypePack* Parser::parseTypeList(TempVector& result, TempVector Parser::parseOptionalReturnTypeAnnotation() { - if (options.allowTypeAnnotations && - (lexer.current().type == ':' || (FFlag::LuauReturnTypeTokenConfusion && lexer.current().type == Lexeme::SkinnyArrow))) + if (options.allowTypeAnnotations && (lexer.current().type == ':' || lexer.current().type == Lexeme::SkinnyArrow)) { - if (FFlag::LuauReturnTypeTokenConfusion && lexer.current().type == Lexeme::SkinnyArrow) + if (lexer.current().type == Lexeme::SkinnyArrow) report(lexer.current().location, "Function return type annotations are written after ':' instead of '->'"); nextLexeme(); @@ -1373,12 +1371,10 @@ AstTypeOrPack Parser::parseFunctionTypeAnnotation(bool allowPack) if (FFlag::LuauFixNamedFunctionParse && !names.empty()) forceFunctionType = true; - bool returnTypeIntroducer = - FFlag::LuauReturnTypeTokenConfusion ? lexer.current().type == Lexeme::SkinnyArrow || lexer.current().type == ':' : false; + bool returnTypeIntroducer = lexer.current().type == Lexeme::SkinnyArrow || lexer.current().type == ':'; // Not a function at all. Just a parenthesized type. Or maybe a type pack with a single element - if (params.size() == 1 && !varargAnnotation && !forceFunctionType && - (FFlag::LuauReturnTypeTokenConfusion ? !returnTypeIntroducer : lexer.current().type != Lexeme::SkinnyArrow)) + if (params.size() == 1 && !varargAnnotation && !forceFunctionType && !returnTypeIntroducer) { if (DFFlag::LuaReportParseWrongNamedType && !names.empty()) lua_telemetry_parsed_named_non_function_type = true; @@ -1389,8 +1385,7 @@ AstTypeOrPack Parser::parseFunctionTypeAnnotation(bool allowPack) return {params[0], {}}; } - if ((FFlag::LuauReturnTypeTokenConfusion ? !returnTypeIntroducer : lexer.current().type != Lexeme::SkinnyArrow) && !forceFunctionType && - allowPack) + if (!forceFunctionType && !returnTypeIntroducer && allowPack) { if (DFFlag::LuaReportParseWrongNamedType && !names.empty()) lua_telemetry_parsed_named_non_function_type = true; @@ -1409,7 +1404,7 @@ AstType* Parser::parseFunctionTypeAnnotationTail(const Lexeme& begin, AstArray' instead of ':'"); lexer.next(); diff --git a/CodeGen/include/Luau/AssemblyBuilderX64.h b/CodeGen/include/Luau/AssemblyBuilderX64.h index c5979d3c..65883b49 100644 --- a/CodeGen/include/Luau/AssemblyBuilderX64.h +++ b/CodeGen/include/Luau/AssemblyBuilderX64.h @@ -58,6 +58,9 @@ public: void jmp(Label& label); void jmp(OperandX64 op); + void call(Label& label); + void call(OperandX64 op); + // AVX void vaddpd(OperandX64 dst, OperandX64 src1, OperandX64 src2); void vaddps(OperandX64 dst, OperandX64 src1, OperandX64 src2); diff --git a/CodeGen/src/AssemblyBuilderX64.cpp b/CodeGen/src/AssemblyBuilderX64.cpp index 27e01781..26347225 100644 --- a/CodeGen/src/AssemblyBuilderX64.cpp +++ b/CodeGen/src/AssemblyBuilderX64.cpp @@ -286,11 +286,34 @@ void AssemblyBuilderX64::jmp(OperandX64 op) if (logText) log("jmp", op); + placeRex(op); place(0xff); placeModRegMem(op, 4); commit(); } +void AssemblyBuilderX64::call(Label& label) +{ + place(0xe8); + placeLabel(label); + + if (logText) + log("call", label); + + commit(); +} + +void AssemblyBuilderX64::call(OperandX64 op) +{ + if (logText) + log("call", op); + + placeRex(op); + place(0xff); + placeModRegMem(op, 2); + commit(); +} + void AssemblyBuilderX64::vaddpd(OperandX64 dst, OperandX64 src1, OperandX64 src2) { placeAvx("vaddpd", dst, src1, src2, 0x58, false, AVX_0F, AVX_66); diff --git a/Sources.cmake b/Sources.cmake index f261cba6..44bed8f7 100644 --- a/Sources.cmake +++ b/Sources.cmake @@ -247,12 +247,15 @@ if(TARGET Luau.UnitTest) tests/IostreamOptional.h tests/ScopedFlags.h tests/Fixture.cpp + tests/AssemblyBuilderX64.test.cpp tests/AstQuery.test.cpp tests/AstVisitor.test.cpp tests/Autocomplete.test.cpp tests/BuiltinDefinitions.test.cpp tests/Compiler.test.cpp tests/Config.test.cpp + tests/ConstraintGraphBuilder.test.cpp + tests/ConstraintSolver.test.cpp tests/CostModel.test.cpp tests/Error.test.cpp tests/Frontend.test.cpp @@ -262,8 +265,7 @@ if(TARGET Luau.UnitTest) tests/Module.test.cpp tests/NonstrictMode.test.cpp tests/Normalize.test.cpp - tests/ConstraintGraphBuilder.test.cpp - tests/ConstraintSolver.test.cpp + tests/NotNull.test.cpp tests/Parser.test.cpp tests/RequireTracer.test.cpp tests/RuntimeLimits.test.cpp @@ -295,11 +297,11 @@ if(TARGET Luau.UnitTest) tests/TypeInfer.tryUnify.test.cpp tests/TypeInfer.typePacks.cpp tests/TypeInfer.unionTypes.test.cpp + tests/TypeInfer.unknownnever.test.cpp tests/TypePack.test.cpp tests/TypeVar.test.cpp tests/Variant.test.cpp tests/VisitTypeVar.test.cpp - tests/AssemblyBuilderX64.test.cpp tests/main.cpp) endif() diff --git a/VM/src/ltable.cpp b/VM/src/ltable.cpp index 2316cc3d..79e65919 100644 --- a/VM/src/ltable.cpp +++ b/VM/src/ltable.cpp @@ -108,9 +108,9 @@ static LuaNode* hashvec(const Table* t, const float* v) memcpy(i, v, sizeof(i)); // convert -0 to 0 to make sure they hash to the same value - i[0] = (i[0] == 0x8000000) ? 0 : i[0]; - i[1] = (i[1] == 0x8000000) ? 0 : i[1]; - i[2] = (i[2] == 0x8000000) ? 0 : i[2]; + i[0] = (i[0] == 0x80000000) ? 0 : i[0]; + i[1] = (i[1] == 0x80000000) ? 0 : i[1]; + i[2] = (i[2] == 0x80000000) ? 0 : i[2]; // scramble bits to make sure that integer coordinates have entropy in lower bits i[0] ^= i[0] >> 17; @@ -121,7 +121,7 @@ static LuaNode* hashvec(const Table* t, const float* v) unsigned int h = (i[0] * 73856093) ^ (i[1] * 19349663) ^ (i[2] * 83492791); #if LUA_VECTOR_SIZE == 4 - i[3] = (i[3] == 0x8000000) ? 0 : i[3]; + i[3] = (i[3] == 0x80000000) ? 0 : i[3]; i[3] ^= i[3] >> 17; h ^= i[3] * 39916801; #endif diff --git a/VM/src/lvmexecute.cpp b/VM/src/lvmexecute.cpp index 85829ca1..02b39313 100644 --- a/VM/src/lvmexecute.cpp +++ b/VM/src/lvmexecute.cpp @@ -640,20 +640,16 @@ static void luau_execute(lua_State* L) VM_PATCH_C(pc - 2, L->cachedslot); VM_NEXT(); } - else - { - // slow-path, may invoke Lua calls via __index metamethod - VM_PROTECT(luaV_gettable(L, rb, kv, ra)); - VM_NEXT(); - } - } - else - { - // slow-path, may invoke Lua calls via __index metamethod - VM_PROTECT(luaV_gettable(L, rb, kv, ra)); - VM_NEXT(); + + // fall through to slow path } + + // fall through to slow path } + + // slow-path, may invoke Lua calls via __index metamethod + VM_PROTECT(luaV_gettable(L, rb, kv, ra)); + VM_NEXT(); } VM_CASE(LOP_SETTABLEKS) @@ -753,19 +749,13 @@ static void luau_execute(lua_State* L) setobj2s(L, ra, &h->array[unsigned(index - 1)]); VM_NEXT(); } - else - { - // slow-path: handles out of bounds array lookups and non-integer numeric keys - VM_PROTECT(luaV_gettable(L, rb, rc, ra)); - VM_NEXT(); - } - } - else - { - // slow-path: handles non-array table lookup as well as __index MT calls - VM_PROTECT(luaV_gettable(L, rb, rc, ra)); - VM_NEXT(); + + // fall through to slow path } + + // slow-path: handles out of bounds array lookups, non-integer numeric keys, non-array table lookup, __index MT calls + VM_PROTECT(luaV_gettable(L, rb, rc, ra)); + VM_NEXT(); } VM_CASE(LOP_SETTABLE) @@ -790,19 +780,13 @@ static void luau_execute(lua_State* L) luaC_barriert(L, h, ra); VM_NEXT(); } - else - { - // slow-path: handles out of bounds array assignments and non-integer numeric keys - VM_PROTECT(luaV_settable(L, rb, rc, ra)); - VM_NEXT(); - } - } - else - { - // slow-path: handles non-array table access as well as __newindex MT calls - VM_PROTECT(luaV_settable(L, rb, rc, ra)); - VM_NEXT(); + + // fall through to slow path } + + // slow-path: handles out of bounds array assignments, non-integer numeric keys, non-array table access, __newindex MT calls + VM_PROTECT(luaV_settable(L, rb, rc, ra)); + VM_NEXT(); } VM_CASE(LOP_GETTABLEN) @@ -822,6 +806,8 @@ static void luau_execute(lua_State* L) setobj2s(L, ra, &h->array[c]); VM_NEXT(); } + + // fall through to slow path } // slow-path: handles out of bounds array lookups @@ -849,6 +835,8 @@ static void luau_execute(lua_State* L) luaC_barriert(L, h, ra); VM_NEXT(); } + + // fall through to slow path } // slow-path: handles out of bounds array lookups @@ -2176,8 +2164,10 @@ static void luau_execute(lua_State* L) if (!ttisnumber(ra + 0) || !ttisnumber(ra + 1) || !ttisnumber(ra + 2)) { // slow-path: can convert arguments to numbers and trigger Lua errors - // Note: this doesn't reallocate stack so we don't need to recompute ra - VM_PROTECT(luau_prepareFORN(L, ra + 0, ra + 1, ra + 2)); + // Note: this doesn't reallocate stack so we don't need to recompute ra/base + VM_PROTECT_PC(); + + luau_prepareFORN(L, ra + 0, ra + 1, ra + 2); } double limit = nvalue(ra + 0); diff --git a/bench/bench.py b/bench/bench.py index e78e96a8..bb3ea5f7 100644 --- a/bench/bench.py +++ b/bench/bench.py @@ -101,8 +101,10 @@ def getVmOutput(cmd): elif arguments.callgrind: try: subprocess.check_call("valgrind --tool=callgrind --callgrind-out-file=callgrind.out --combine-dumps=yes --dump-line=no " + cmd, shell=True, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, cwd=scriptdir) - file = open(os.path.join(scriptdir, "callgrind.out"), "r") - lines = file.readlines() + path = os.path.join(scriptdir, "callgrind.out") + with open(path, "r") as file: + lines = file.readlines() + os.unlink(path) return getCallgrindOutput(lines) except: return "" @@ -402,12 +404,12 @@ def analyzeResult(subdir, main, comparisons): continue - pooledStdDev = math.sqrt((main.unbiasedEst + compare.unbiasedEst) / 2) + if main.count > 1 and stats: + pooledStdDev = math.sqrt((main.unbiasedEst + compare.unbiasedEst) / 2) - tStat = abs(main.avg - compare.avg) / (pooledStdDev * math.sqrt(2 / main.count)) - degreesOfFreedom = 2 * main.count - 2 + tStat = abs(main.avg - compare.avg) / (pooledStdDev * math.sqrt(2 / main.count)) + degreesOfFreedom = 2 * main.count - 2 - if stats: # Two-tailed distribution with 95% conf. tCritical = stats.t.ppf(1 - 0.05 / 2, degreesOfFreedom) diff --git a/bench/other/regex.lua b/bench/other/regex.lua index 270ab3da..eb659a5e 100644 --- a/bench/other/regex.lua +++ b/bench/other/regex.lua @@ -2,7 +2,7 @@ PCRE2-based RegEx implemention for Luau Version 1.0.0a2 BSD 2-Clause Licence - Copyright © 2020 - Blockzez (devforum.roblox.com/u/Blockzez and github.com/Blockzez) + Copyright © 2020 - Blockzez (devforum /u/Blockzez and github.com/Blockzez) All rights reserved. Redistribution and use in source and binary forms, with or without diff --git a/tests/AssemblyBuilderX64.test.cpp b/tests/AssemblyBuilderX64.test.cpp index 7f863c6f..15813ae9 100644 --- a/tests/AssemblyBuilderX64.test.cpp +++ b/tests/AssemblyBuilderX64.test.cpp @@ -213,6 +213,16 @@ TEST_CASE_FIXTURE(AssemblyBuilderX64Fixture, "FormsOfLea") SINGLE_COMPARE(lea(rax, qword[r13 + r12 * 4 + 4]), 0x4b, 0x8d, 0x44, 0xa5, 0x04); } +TEST_CASE_FIXTURE(AssemblyBuilderX64Fixture, "FormsOfAbsoluteJumps") +{ + SINGLE_COMPARE(jmp(rax), 0x48, 0xff, 0xe0); + SINGLE_COMPARE(jmp(r14), 0x49, 0xff, 0xe6); + SINGLE_COMPARE(jmp(qword[r14 + rdx * 4]), 0x49, 0xff, 0x24, 0x96); + SINGLE_COMPARE(call(rax), 0x48, 0xff, 0xd0); + SINGLE_COMPARE(call(r14), 0x49, 0xff, 0xd6); + SINGLE_COMPARE(call(qword[r14 + rdx * 4]), 0x49, 0xff, 0x14, 0x96); +} + TEST_CASE_FIXTURE(AssemblyBuilderX64Fixture, "ControlFlow") { // Jump back @@ -260,6 +270,23 @@ TEST_CASE_FIXTURE(AssemblyBuilderX64Fixture, "ControlFlow") {0xe9, 0x04, 0x00, 0x00, 0x00, 0x48, 0x83, 0xe7, 0x3e}); } +TEST_CASE_FIXTURE(AssemblyBuilderX64Fixture, "LabelCall") +{ + check( + [](AssemblyBuilderX64& build) { + Label fnB; + + build.and_(rcx, 0x3e); + build.call(fnB); + build.ret(); + + build.setLabel(fnB); + build.lea(rax, qword[rcx + 0x1f]); + build.ret(); + }, + {0x48, 0x83, 0xe1, 0x3e, 0xe8, 0x01, 0x00, 0x00, 0x00, 0xc3, 0x48, 0x8d, 0x41, 0x1f, 0xc3}); +} + TEST_CASE_FIXTURE(AssemblyBuilderX64Fixture, "AVXBinaryInstructionForms") { SINGLE_COMPARE(vaddpd(xmm8, xmm10, xmm14), 0xc4, 0x41, 0xa9, 0x58, 0xc6); diff --git a/tests/AstQuery.test.cpp b/tests/AstQuery.test.cpp index f0017509..6ec1426c 100644 --- a/tests/AstQuery.test.cpp +++ b/tests/AstQuery.test.cpp @@ -105,4 +105,37 @@ if true then REQUIRE(parentStat->is()); } +TEST_CASE_FIXTURE(Fixture, "ac_ast_ancestry_at_number_const") +{ + check(R"( +print(3.) + )"); + + std::vector ancestry = findAncestryAtPositionForAutocomplete(*getMainSourceModule(), Position(1, 8)); + REQUIRE_GE(ancestry.size(), 2); + REQUIRE(ancestry.back()->is()); +} + +TEST_CASE_FIXTURE(Fixture, "ac_ast_ancestry_in_workspace_dot") +{ + check(R"( +print(workspace.) + )"); + + std::vector ancestry = findAncestryAtPositionForAutocomplete(*getMainSourceModule(), Position(1, 16)); + REQUIRE_GE(ancestry.size(), 2); + REQUIRE(ancestry.back()->is()); +} + +TEST_CASE_FIXTURE(Fixture, "ac_ast_ancestry_in_workspace_colon") +{ + check(R"( +print(workspace:) + )"); + + std::vector ancestry = findAncestryAtPositionForAutocomplete(*getMainSourceModule(), Position(1, 16)); + REQUIRE_GE(ancestry.size(), 2); + REQUIRE(ancestry.back()->is()); +} + TEST_SUITE_END(); diff --git a/tests/Fixture.h b/tests/Fixture.h index 1bc573da..4bd6f1ea 100644 --- a/tests/Fixture.h +++ b/tests/Fixture.h @@ -128,6 +128,7 @@ struct Fixture std::optional lookupImportedType(const std::string& moduleAlias, const std::string& name); ScopedFastFlag sff_DebugLuauFreezeArena; + ScopedFastFlag sff_UnknownNever{"LuauUnknownAndNeverType", true}; TestFileResolver fileResolver; TestConfigResolver configResolver; diff --git a/tests/Module.test.cpp b/tests/Module.test.cpp index 7c2f4d1c..dd94e9d7 100644 --- a/tests/Module.test.cpp +++ b/tests/Module.test.cpp @@ -301,8 +301,6 @@ TEST_CASE_FIXTURE(Fixture, "clone_recursion_limit") TEST_CASE_FIXTURE(Fixture, "any_persistance_does_not_leak") { - ScopedFastFlag luauNonCopyableTypeVarFields{"LuauNonCopyableTypeVarFields", true}; - fileResolver.source["Module/A"] = R"( export type A = B type B = A diff --git a/tests/Normalize.test.cpp b/tests/Normalize.test.cpp index a474b6e7..fb0a899e 100644 --- a/tests/Normalize.test.cpp +++ b/tests/Normalize.test.cpp @@ -1055,8 +1055,6 @@ export type t1 = { a: typeof(string.byte) } TEST_CASE_FIXTURE(Fixture, "intersection_combine_on_bound_self") { - ScopedFastFlag luauNormalizeCombineEqFix{"LuauNormalizeCombineEqFix", true}; - CheckResult result = check(R"( export type t0 = (((any)&({_:l0.t0,n0:t0,_G:any,}))&({_:any,}))&(((any)&({_:l0.t0,n0:t0,_G:any,}))&({_:any,})) )"); @@ -1064,6 +1062,46 @@ export type t0 = (((any)&({_:l0.t0,n0:t0,_G:any,}))&({_:any,}))&(((any)&({_:l0.t LUAU_REQUIRE_ERRORS(result); } +TEST_CASE_FIXTURE(Fixture, "normalize_unions_containing_never") +{ + ScopedFastFlag sff{"LuauLowerBoundsCalculation", true}; + + CheckResult result = check(R"( + type Foo = string | never + local foo: Foo + )"); + + CHECK_EQ("string", toString(requireType("foo"))); +} + +TEST_CASE_FIXTURE(Fixture, "normalize_unions_containing_unknown") +{ + ScopedFastFlag sff{"LuauLowerBoundsCalculation", true}; + + CheckResult result = check(R"( + type Foo = string | unknown + local foo: Foo + )"); + + CHECK_EQ("unknown", toString(requireType("foo"))); +} + +TEST_CASE_FIXTURE(Fixture, "any_wins_the_battle_over_unknown_in_unions") +{ + ScopedFastFlag sff{"LuauLowerBoundsCalculation", true}; + + CheckResult result = check(R"( + type Foo = unknown | any + local foo: Foo + + type Bar = any | unknown + local bar: Bar + )"); + + CHECK_EQ("any", toString(requireType("foo"))); + CHECK_EQ("any", toString(requireType("bar"))); +} + TEST_CASE_FIXTURE(BuiltinsFixture, "normalization_does_not_convert_ever") { ScopedFastFlag sff[]{ diff --git a/tests/Parser.test.cpp b/tests/Parser.test.cpp index c3c75998..c517853f 100644 --- a/tests/Parser.test.cpp +++ b/tests/Parser.test.cpp @@ -2648,7 +2648,6 @@ type Z = { a: string | T..., b: number } TEST_CASE_FIXTURE(Fixture, "recover_function_return_type_annotations") { - ScopedFastFlag sff{"LuauReturnTypeTokenConfusion", true}; ParseResult result = tryParse(R"( type Custom = { x: A, y: B, z: C } type Packed = { x: (A...) -> () } diff --git a/tests/ToString.test.cpp b/tests/ToString.test.cpp index 1601a152..52a29bcc 100644 --- a/tests/ToString.test.cpp +++ b/tests/ToString.test.cpp @@ -499,7 +499,7 @@ local function target(callback: nil) return callback(4, "hello") end )"); LUAU_REQUIRE_ERRORS(result); - CHECK_EQ("(nil) -> (*unknown*)", toString(requireType("target"))); + CHECK_EQ("(nil) -> ()", toString(requireType("target"))); } TEST_CASE_FIXTURE(Fixture, "toStringGenericPack") diff --git a/tests/TypeInfer.anyerror.test.cpp b/tests/TypeInfer.anyerror.test.cpp index bc55940e..4c5309e0 100644 --- a/tests/TypeInfer.anyerror.test.cpp +++ b/tests/TypeInfer.anyerror.test.cpp @@ -94,7 +94,7 @@ TEST_CASE_FIXTURE(Fixture, "for_in_loop_iterator_is_error") LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK_EQ("*unknown*", toString(requireType("a"))); + CHECK_EQ("", toString(requireType("a"))); } TEST_CASE_FIXTURE(Fixture, "for_in_loop_iterator_is_error2") @@ -110,7 +110,7 @@ TEST_CASE_FIXTURE(Fixture, "for_in_loop_iterator_is_error2") LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK_EQ("*unknown*", toString(requireType("a"))); + CHECK_EQ("", toString(requireType("a"))); } TEST_CASE_FIXTURE(Fixture, "length_of_error_type_does_not_produce_an_error") @@ -225,7 +225,7 @@ TEST_CASE_FIXTURE(Fixture, "calling_error_type_yields_error") CHECK_EQ("unknown", err->name); - CHECK_EQ("*unknown*", toString(requireType("a"))); + CHECK_EQ("", toString(requireType("a"))); } TEST_CASE_FIXTURE(Fixture, "chain_calling_error_type_yields_error") @@ -234,7 +234,7 @@ TEST_CASE_FIXTURE(Fixture, "chain_calling_error_type_yields_error") local a = Utility.Create "Foo" {} )"); - CHECK_EQ("*unknown*", toString(requireType("a"))); + CHECK_EQ("", toString(requireType("a"))); } TEST_CASE_FIXTURE(BuiltinsFixture, "replace_every_free_type_when_unifying_a_complex_function_with_any") diff --git a/tests/TypeInfer.builtins.test.cpp b/tests/TypeInfer.builtins.test.cpp index 2f0266ec..7d1bd6ba 100644 --- a/tests/TypeInfer.builtins.test.cpp +++ b/tests/TypeInfer.builtins.test.cpp @@ -925,7 +925,7 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "assert_returns_false_and_string_iff_it_knows )"); LUAU_REQUIRE_NO_ERRORS(result); - CHECK_EQ("(nil) -> nil", toString(requireType("f"))); + CHECK_EQ("(nil) -> (never, ...never)", toString(requireType("f"))); } TEST_CASE_FIXTURE(BuiltinsFixture, "table_freeze_is_generic") @@ -952,7 +952,7 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "table_freeze_is_generic") CHECK_EQ("number", toString(requireType("a"))); CHECK_EQ("string", toString(requireType("b"))); CHECK_EQ("boolean", toString(requireType("c"))); - CHECK_EQ("*unknown*", toString(requireType("d"))); + CHECK_EQ("", toString(requireType("d"))); } TEST_CASE_FIXTURE(BuiltinsFixture, "set_metatable_needs_arguments") @@ -965,8 +965,8 @@ a:b() a:b({}) )"); LUAU_REQUIRE_ERROR_COUNT(2, result); - CHECK_EQ(result.errors[0], (TypeError{Location{{2, 0}, {2, 5}}, CountMismatch{2, 0}})); - CHECK_EQ(result.errors[1], (TypeError{Location{{3, 0}, {3, 5}}, CountMismatch{2, 1}})); + CHECK_EQ(toString(result.errors[0]), "Argument count mismatch. Function expects 2 arguments, but none are specified"); + CHECK_EQ(toString(result.errors[1]), "Argument count mismatch. Function expects 2 arguments, but only 1 is specified"); } TEST_CASE_FIXTURE(Fixture, "typeof_unresolved_function") @@ -1008,4 +1008,139 @@ end LUAU_REQUIRE_NO_ERRORS(result); } +TEST_CASE_FIXTURE(BuiltinsFixture, "gmatch_capture_types") +{ + ScopedFastFlag sffs{"LuauDeduceGmatchReturnTypes", true}; + CheckResult result = check(R"END( + local a, b, c = string.gmatch("This is a string", "(.()(%a+))")() + )END"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ(toString(requireType("a")), "string"); + CHECK_EQ(toString(requireType("b")), "number"); + CHECK_EQ(toString(requireType("c")), "string"); +} + +TEST_CASE_FIXTURE(BuiltinsFixture, "gmatch_capture_types2") +{ + ScopedFastFlag sffs{"LuauDeduceGmatchReturnTypes", true}; + CheckResult result = check(R"END( + local a, b, c = ("This is a string"):gmatch("(.()(%a+))")() + )END"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ(toString(requireType("a")), "string"); + CHECK_EQ(toString(requireType("b")), "number"); + CHECK_EQ(toString(requireType("c")), "string"); +} + +TEST_CASE_FIXTURE(BuiltinsFixture, "gmatch_capture_types_default_capture") +{ + ScopedFastFlag sffs{"LuauDeduceGmatchReturnTypes", true}; + CheckResult result = check(R"END( + local a, b, c, d = string.gmatch("T(his)() is a string", ".")() + )END"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + CountMismatch* acm = get(result.errors[0]); + REQUIRE(acm); + CHECK_EQ(acm->context, CountMismatch::Result); + CHECK_EQ(acm->expected, 1); + CHECK_EQ(acm->actual, 4); + + CHECK_EQ(toString(requireType("a")), "string"); +} + +TEST_CASE_FIXTURE(BuiltinsFixture, "gmatch_capture_types_balanced_escaped_parens") +{ + ScopedFastFlag sffs{"LuauDeduceGmatchReturnTypes", true}; + CheckResult result = check(R"END( + local a, b, c, d = string.gmatch("T(his) is a string", "((.)%b()())")() + )END"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + CountMismatch* acm = get(result.errors[0]); + REQUIRE(acm); + CHECK_EQ(acm->context, CountMismatch::Result); + CHECK_EQ(acm->expected, 3); + CHECK_EQ(acm->actual, 4); + + CHECK_EQ(toString(requireType("a")), "string"); + CHECK_EQ(toString(requireType("b")), "string"); + CHECK_EQ(toString(requireType("c")), "number"); +} + +TEST_CASE_FIXTURE(BuiltinsFixture, "gmatch_capture_types_parens_in_sets_are_ignored") +{ + ScopedFastFlag sffs{"LuauDeduceGmatchReturnTypes", true}; + CheckResult result = check(R"END( + local a, b, c = string.gmatch("T(his)() is a string", "(T[()])()")() + )END"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + CountMismatch* acm = get(result.errors[0]); + REQUIRE(acm); + CHECK_EQ(acm->context, CountMismatch::Result); + CHECK_EQ(acm->expected, 2); + CHECK_EQ(acm->actual, 3); + + CHECK_EQ(toString(requireType("a")), "string"); + CHECK_EQ(toString(requireType("b")), "number"); +} + +TEST_CASE_FIXTURE(BuiltinsFixture, "gmatch_capture_types_set_containing_lbracket") +{ + ScopedFastFlag sffs{"LuauDeduceGmatchReturnTypes", true}; + CheckResult result = check(R"END( + local a, b = string.gmatch("[[[", "()([[])")() + )END"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ(toString(requireType("a")), "number"); + CHECK_EQ(toString(requireType("b")), "string"); +} + +TEST_CASE_FIXTURE(BuiltinsFixture, "gmatch_capture_types_leading_end_bracket_is_part_of_set") +{ + CheckResult result = check(R"END( + -- An immediate right-bracket following a left-bracket is included within the set; + -- thus, '[]]'' is the set containing ']', and '[]' is an invalid set missing an enclosing + -- right-bracket. We detect an invalid set in this case and fall back to to default gmatch + -- typing. + local foo = string.gmatch("T[hi%]s]]]() is a string", "([]s)") + )END"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ(toString(requireType("foo")), "() -> (...string)"); +} + +TEST_CASE_FIXTURE(BuiltinsFixture, "gmatch_capture_types_invalid_pattern_fallback_to_builtin") +{ + CheckResult result = check(R"END( + local foo = string.gmatch("T(his)() is a string", ")") + )END"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ(toString(requireType("foo")), "() -> (...string)"); +} + +TEST_CASE_FIXTURE(BuiltinsFixture, "gmatch_capture_types_invalid_pattern_fallback_to_builtin2") +{ + CheckResult result = check(R"END( + local foo = string.gmatch("T(his)() is a string", "[") + )END"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ(toString(requireType("foo")), "() -> (...string)"); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.functions.test.cpp b/tests/TypeInfer.functions.test.cpp index 401a6c64..6e6549d3 100644 --- a/tests/TypeInfer.functions.test.cpp +++ b/tests/TypeInfer.functions.test.cpp @@ -916,13 +916,13 @@ TEST_CASE_FIXTURE(Fixture, "function_cast_error_uses_correct_language") REQUIRE(tm1); CHECK_EQ("(string) -> number", toString(tm1->wantedType)); - CHECK_EQ("(string, *unknown*) -> number", toString(tm1->givenType)); + CHECK_EQ("(string, ) -> number", toString(tm1->givenType)); auto tm2 = get(result.errors[1]); REQUIRE(tm2); CHECK_EQ("(number, number) -> (number, number)", toString(tm2->wantedType)); - CHECK_EQ("(string, *unknown*) -> number", toString(tm2->givenType)); + CHECK_EQ("(string, ) -> number", toString(tm2->givenType)); } TEST_CASE_FIXTURE(Fixture, "no_lossy_function_type") @@ -1535,7 +1535,7 @@ function t:b() return 2 end -- not OK )"); LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK_EQ(R"(Type '(*unknown*) -> number' could not be converted into '() -> number' + CHECK_EQ(R"(Type '() -> number' could not be converted into '() -> number' caused by: Argument count mismatch. Function expects 1 argument, but none are specified)", toString(result.errors[0])); @@ -1692,4 +1692,52 @@ TEST_CASE_FIXTURE(Fixture, "call_o_with_another_argument_after_foo_was_quantifie // TODO: check the normalized type of f } +TEST_CASE_FIXTURE(Fixture, "free_is_not_bound_to_unknown") +{ + CheckResult result = check(R"( + local function foo(f: (unknown) -> (), x) + f(x) + end + )"); + + CHECK_EQ("((unknown) -> (), a) -> ()", toString(requireType("foo"))); +} + +TEST_CASE_FIXTURE(Fixture, "dont_infer_parameter_types_for_functions_from_their_call_site") +{ + CheckResult result = check(R"( + local t = {} + + function t.f(x) + return x + end + + t.__index = t + + function g(s) + local q = s.p and s.p.q or nil + return q and t.f(q) or nil + end + + local f = t.f + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ("(a) -> a", toString(requireType("f"))); +} + +TEST_CASE_FIXTURE(Fixture, "dont_mutate_the_underlying_head_of_typepack_when_calling_with_self") +{ + CheckResult result = check(R"( + local t = {} + function t:m(x) end + function f(): never return 5 :: never end + t:m(f()) + t:m(f()) + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.generics.test.cpp b/tests/TypeInfer.generics.test.cpp index e9e94cfb..46258072 100644 --- a/tests/TypeInfer.generics.test.cpp +++ b/tests/TypeInfer.generics.test.cpp @@ -9,6 +9,8 @@ #include "doctest.h" +LUAU_FASTFLAG(LuauCheckGenericHOFTypes) + using namespace Luau; TEST_SUITE_BEGIN("GenericsTests"); @@ -1001,7 +1003,7 @@ TEST_CASE_FIXTURE(Fixture, "no_stack_overflow_from_quantifying") std::optional t0 = getMainModule()->getModuleScope()->lookupType("t0"); REQUIRE(t0); - CHECK_EQ("*unknown*", toString(t0->type)); + CHECK_EQ("", toString(t0->type)); auto it = std::find_if(result.errors.begin(), result.errors.end(), [](TypeError& err) { return get(err); @@ -1095,10 +1097,18 @@ local b = sumrec(sum) -- ok local c = sumrec(function(x, y, f) return f(x, y) end) -- type binders are not inferred )"); - LUAU_REQUIRE_ERRORS(result); - CHECK_EQ("Type '(a, b, (a, b) -> (c...)) -> (c...)' could not be converted into '(a, a, (a, a) -> a) -> a'; different number of generic type " - "parameters", - toString(result.errors[0])); + if (FFlag::LuauCheckGenericHOFTypes) + { + LUAU_REQUIRE_NO_ERRORS(result); + } + else + { + LUAU_REQUIRE_ERRORS(result); + CHECK_EQ( + "Type '(a, b, (a, b) -> (c...)) -> (c...)' could not be converted into '(a, a, (a, a) -> a) -> a'; different number of generic type " + "parameters", + toString(result.errors[0])); + } } TEST_CASE_FIXTURE(Fixture, "substitution_with_bound_table") @@ -1185,4 +1195,23 @@ TEST_CASE_FIXTURE(Fixture, "quantify_functions_even_if_they_have_an_explicit_gen CHECK("((X) -> (a...), X) -> (a...)" == toString(requireType("foo"))); } +TEST_CASE_FIXTURE(Fixture, "do_not_always_instantiate_generic_intersection_types") +{ + ScopedFastFlag sff[] = { + {"LuauMaybeGenericIntersectionTypes", true}, + }; + + CheckResult result = check(R"( + --!strict + type Array = { [number]: T } + + type Array_Statics = { + new: () -> Array, + } + + local _Arr : Array & Array_Statics = {} :: Array_Statics + )"); + LUAU_REQUIRE_NO_ERRORS(result); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.loops.test.cpp b/tests/TypeInfer.loops.test.cpp index 1c6fe1d8..56b807f8 100644 --- a/tests/TypeInfer.loops.test.cpp +++ b/tests/TypeInfer.loops.test.cpp @@ -142,7 +142,7 @@ TEST_CASE_FIXTURE(Fixture, "for_in_loop_on_error") CHECK_EQ(2, result.errors.size()); TypeId p = requireType("p"); - CHECK_EQ("*unknown*", toString(p)); + CHECK_EQ("", toString(p)); } TEST_CASE_FIXTURE(Fixture, "for_in_loop_on_non_function") diff --git a/tests/TypeInfer.modules.test.cpp b/tests/TypeInfer.modules.test.cpp index a0f670f1..2343a7fa 100644 --- a/tests/TypeInfer.modules.test.cpp +++ b/tests/TypeInfer.modules.test.cpp @@ -143,7 +143,7 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "require_module_that_does_not_export") auto hootyType = requireType(bModule, "Hooty"); - CHECK_EQ("*unknown*", toString(hootyType)); + CHECK_EQ("", toString(hootyType)); } TEST_CASE_FIXTURE(BuiltinsFixture, "warn_if_you_try_to_require_a_non_modulescript") @@ -244,7 +244,7 @@ local ModuleA = require(game.A) LUAU_REQUIRE_NO_ERRORS(result); std::optional oty = requireType("ModuleA"); - CHECK_EQ("*unknown*", toString(*oty)); + CHECK_EQ("", toString(*oty)); } TEST_CASE_FIXTURE(BuiltinsFixture, "do_not_modify_imported_types") @@ -302,6 +302,30 @@ type Rename = typeof(x.x) LUAU_REQUIRE_NO_ERRORS(result); } +TEST_CASE_FIXTURE(BuiltinsFixture, "do_not_modify_imported_types_4") +{ + fileResolver.source["game/A"] = R"( +export type Array = {T} +local arrayops = {} +function arrayops.foo(x: Array) end +return arrayops + )"; + + CheckResult result = check(R"( +local arrayops = require(game.A) + +local tbl = {} +tbl.a = 2 +function tbl:foo(b: number, c: number) + -- introduce BoundTypeVar to imported type + arrayops.foo(self._regions) +end +type Table = typeof(tbl) +)"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + TEST_CASE_FIXTURE(BuiltinsFixture, "module_type_conflict") { fileResolver.source["game/A"] = R"( @@ -363,4 +387,21 @@ caused by: Property 'x' is not compatible. Type 'number' could not be converted into 'string')"); } +TEST_CASE_FIXTURE(BuiltinsFixture, "constrained_anyification_clone_immutable_types") +{ + ScopedFastFlag luauAnyificationMustClone{"LuauAnyificationMustClone", true}; + + fileResolver.source["game/A"] = R"( +return function(...) end + )"; + + fileResolver.source["game/B"] = R"( +local l0 = require(game.A) +return l0 + )"; + + CheckResult result = frontend.check("game/B"); + LUAU_REQUIRE_NO_ERRORS(result); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.operators.test.cpp b/tests/TypeInfer.operators.test.cpp index e6174df2..c90f0a4d 100644 --- a/tests/TypeInfer.operators.test.cpp +++ b/tests/TypeInfer.operators.test.cpp @@ -871,4 +871,26 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "equality_operations_succeed_if_any_union_bra CHECK(toString(result2.errors[0]) == "Types Foo and Bar cannot be compared with == because they do not have the same metatable"); } +TEST_CASE_FIXTURE(BuiltinsFixture, "expected_types_through_binary_and") +{ + ScopedFastFlag sff{"LuauBinaryNeedsExpectedTypesToo", true}; + + CheckResult result = check(R"( + local x: "a" | "b" | boolean = math.random() > 0.5 and "a" + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(BuiltinsFixture, "expected_types_through_binary_or") +{ + ScopedFastFlag sff{"LuauBinaryNeedsExpectedTypesToo", true}; + + CheckResult result = check(R"( + local x: "a" | "b" | boolean = math.random() > 0.5 or "b" + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.primitives.test.cpp b/tests/TypeInfer.primitives.test.cpp index e1684df7..9e8e2503 100644 --- a/tests/TypeInfer.primitives.test.cpp +++ b/tests/TypeInfer.primitives.test.cpp @@ -47,7 +47,7 @@ TEST_CASE_FIXTURE(Fixture, "string_index") REQUIRE(nat); CHECK_EQ("string", toString(nat->ty)); - CHECK_EQ("*unknown*", toString(requireType("t"))); + CHECK_EQ("", toString(requireType("t"))); } TEST_CASE_FIXTURE(Fixture, "string_method") diff --git a/tests/TypeInfer.provisional.test.cpp b/tests/TypeInfer.provisional.test.cpp index 018059fd..dc686890 100644 --- a/tests/TypeInfer.provisional.test.cpp +++ b/tests/TypeInfer.provisional.test.cpp @@ -225,7 +225,7 @@ TEST_CASE_FIXTURE(Fixture, "discriminate_from_x_not_equal_to_nil") CHECK_EQ("{| x: nil, y: nil |} | {| x: string, y: number |}", toString(requireTypeAtPosition({7, 28}))); } -TEST_CASE_FIXTURE(Fixture, "bail_early_if_unification_is_too_complicated" * doctest::timeout(0.5)) +TEST_CASE_FIXTURE(BuiltinsFixture, "bail_early_if_unification_is_too_complicated" * doctest::timeout(0.5)) { ScopedFastInt sffi{"LuauTarjanChildLimit", 1}; ScopedFastInt sffi2{"LuauTypeInferIterationLimit", 1}; @@ -499,6 +499,17 @@ TEST_CASE_FIXTURE(Fixture, "constrained_is_level_dependent") CHECK_EQ("(t1) -> {| [t1]: boolean |} where t1 = t2 ; t2 = {+ m1: (t1) -> (a...), m2: (t2) -> (b...) +}", toString(requireType("f"))); } +TEST_CASE_FIXTURE(Fixture, "free_is_not_bound_to_any") +{ + CheckResult result = check(R"( + local function foo(f: (any) -> (), x) + f(x) + end + )"); + + CHECK_EQ("((any) -> (), any) -> ()", toString(requireType("foo"))); +} + TEST_CASE_FIXTURE(BuiltinsFixture, "greedy_inference_with_shared_self_triggers_function_with_no_returns") { ScopedFastFlag sff{"DebugLuauSharedSelf", true}; diff --git a/tests/TypeInfer.refinements.test.cpp b/tests/TypeInfer.refinements.test.cpp index 3f5dad3d..cc8cdee3 100644 --- a/tests/TypeInfer.refinements.test.cpp +++ b/tests/TypeInfer.refinements.test.cpp @@ -272,8 +272,8 @@ TEST_CASE_FIXTURE(Fixture, "typeguard_only_look_up_types_from_global_scope") LUAU_REQUIRE_NO_ERRORS(result); - CHECK_EQ("*unknown*", toString(requireTypeAtPosition({8, 44}))); - CHECK_EQ("*unknown*", toString(requireTypeAtPosition({9, 38}))); + CHECK_EQ("never", toString(requireTypeAtPosition({8, 44}))); + CHECK_EQ("never", toString(requireTypeAtPosition({9, 38}))); } TEST_CASE_FIXTURE(Fixture, "call_a_more_specific_function_using_typeguard") @@ -526,7 +526,7 @@ TEST_CASE_FIXTURE(Fixture, "type_narrow_to_vector") LUAU_REQUIRE_NO_ERRORS(result); - CHECK_EQ("*unknown*", toString(requireTypeAtPosition({3, 28}))); + CHECK_EQ("", toString(requireTypeAtPosition({3, 28}))); } TEST_CASE_FIXTURE(Fixture, "nonoptional_type_can_narrow_to_nil_if_sense_is_true") @@ -651,7 +651,7 @@ TEST_CASE_FIXTURE(Fixture, "type_guard_can_filter_for_overloaded_function") CHECK_EQ("nil", toString(requireTypeAtPosition({6, 28}))); } -TEST_CASE_FIXTURE(BuiltinsFixture, "type_guard_warns_on_no_overlapping_types_only_when_sense_is_true") +TEST_CASE_FIXTURE(BuiltinsFixture, "type_guard_narrowed_into_nothingness") { CheckResult result = check(R"( local function f(t: {x: number}) @@ -666,7 +666,7 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "type_guard_warns_on_no_overlapping_types_onl LUAU_REQUIRE_NO_ERRORS(result); - CHECK_EQ("*unknown*", toString(requireTypeAtPosition({3, 28}))); + CHECK_EQ("never", toString(requireTypeAtPosition({3, 28}))); } TEST_CASE_FIXTURE(Fixture, "not_a_or_not_b") @@ -1074,7 +1074,7 @@ TEST_CASE_FIXTURE(RefinementClassFixture, "typeguard_cast_free_table_to_vector") CHECK_EQ("Vector3", toString(requireTypeAtPosition({5, 28}))); // type(vec) == "vector" - CHECK_EQ("*unknown*", toString(requireTypeAtPosition({7, 28}))); // typeof(vec) == "Instance" + CHECK_EQ("never", toString(requireTypeAtPosition({7, 28}))); // typeof(vec) == "Instance" CHECK_EQ("{+ X: a, Y: b, Z: c +}", toString(requireTypeAtPosition({9, 28}))); // type(vec) ~= "vector" and typeof(vec) ~= "Instance" } @@ -1206,6 +1206,24 @@ TEST_CASE_FIXTURE(Fixture, "typeguard_doesnt_leak_to_elseif") LUAU_REQUIRE_NO_ERRORS(result); } +TEST_CASE_FIXTURE(BuiltinsFixture, "refine_unknowns") +{ + CheckResult result = check(R"( + local function f(x: unknown) + if type(x) == "string" then + local foo = x + else + local bar = x + end + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ("string", toString(requireTypeAtPosition({3, 28}))); + CHECK_EQ("unknown", toString(requireTypeAtPosition({5, 28}))); +} + TEST_CASE_FIXTURE(BuiltinsFixture, "falsiness_of_TruthyPredicate_narrows_into_nil") { ScopedFastFlag sff{"LuauFalsyPredicateReturnsNilInstead", true}; @@ -1227,4 +1245,19 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "falsiness_of_TruthyPredicate_narrows_into_ni CHECK_EQ("number", toString(requireTypeAtPosition({6, 28}))); } +TEST_CASE_FIXTURE(BuiltinsFixture, "what_nonsensical_condition") +{ + CheckResult result = check(R"( + local function f(x) + if type(x) == "string" and type(x) == "number" then + local foo = x + end + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ("never", toString(requireTypeAtPosition({3, 28}))); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.tables.test.cpp b/tests/TypeInfer.tables.test.cpp index eead5b30..3e830f2a 100644 --- a/tests/TypeInfer.tables.test.cpp +++ b/tests/TypeInfer.tables.test.cpp @@ -3070,4 +3070,18 @@ TEST_CASE_FIXTURE(Fixture, "quantify_even_that_table_was_never_exported_at_all") CHECK_EQ("{| m: ({+ x: a, y: b +}) -> a, n: ({+ x: a, y: b +}) -> b |}", toString(requireType("T"), opts)); } +TEST_CASE_FIXTURE(BuiltinsFixture, "leaking_bad_metatable_errors") +{ + ScopedFastFlag luauIndexSilenceErrors{"LuauIndexSilenceErrors", true}; + + CheckResult result = check(R"( +local a = setmetatable({}, 1) +local b = a.x + )"); + + LUAU_REQUIRE_ERROR_COUNT(2, result); + CHECK_EQ("Metatable was not a table", toString(result.errors[0])); + CHECK_EQ("Type 'a' does not have key 'x'", toString(result.errors[1])); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.test.cpp b/tests/TypeInfer.test.cpp index a175b826..7d1fb56b 100644 --- a/tests/TypeInfer.test.cpp +++ b/tests/TypeInfer.test.cpp @@ -238,10 +238,10 @@ TEST_CASE_FIXTURE(Fixture, "type_errors_infer_types") // TODO: Should we assert anything about these tests when DCR is being used? if (!FFlag::DebugLuauDeferredConstraintResolution) { - CHECK_EQ("*unknown*", toString(requireType("c"))); - CHECK_EQ("*unknown*", toString(requireType("d"))); - CHECK_EQ("*unknown*", toString(requireType("e"))); - CHECK_EQ("*unknown*", toString(requireType("f"))); + CHECK_EQ("", toString(requireType("c"))); + CHECK_EQ("", toString(requireType("d"))); + CHECK_EQ("", toString(requireType("e"))); + CHECK_EQ("", toString(requireType("f"))); } } @@ -622,7 +622,7 @@ TEST_CASE_FIXTURE(Fixture, "no_stack_overflow_from_isoptional") std::optional t0 = getMainModule()->getModuleScope()->lookupType("t0"); REQUIRE(t0); - CHECK_EQ("*unknown*", toString(t0->type)); + CHECK_EQ("", toString(t0->type)); auto it = std::find_if(result.errors.begin(), result.errors.end(), [](TypeError& err) { return get(err); diff --git a/tests/TypeInfer.tryUnify.test.cpp b/tests/TypeInfer.tryUnify.test.cpp index 49deae71..d51a38f8 100644 --- a/tests/TypeInfer.tryUnify.test.cpp +++ b/tests/TypeInfer.tryUnify.test.cpp @@ -121,7 +121,7 @@ TEST_CASE_FIXTURE(TryUnifyFixture, "members_of_failed_typepack_unification_are_u LUAU_REQUIRE_ERROR_COUNT(1, result); CHECK_EQ("a", toString(requireType("a"))); - CHECK_EQ("*unknown*", toString(requireType("b"))); + CHECK_EQ("", toString(requireType("b"))); } TEST_CASE_FIXTURE(TryUnifyFixture, "result_of_failed_typepack_unification_is_constrained") @@ -136,7 +136,7 @@ TEST_CASE_FIXTURE(TryUnifyFixture, "result_of_failed_typepack_unification_is_con LUAU_REQUIRE_ERROR_COUNT(1, result); CHECK_EQ("a", toString(requireType("a"))); - CHECK_EQ("*unknown*", toString(requireType("b"))); + CHECK_EQ("", toString(requireType("b"))); CHECK_EQ("number", toString(requireType("c"))); } diff --git a/tests/TypeInfer.unionTypes.test.cpp b/tests/TypeInfer.unionTypes.test.cpp index 2b48133d..94918692 100644 --- a/tests/TypeInfer.unionTypes.test.cpp +++ b/tests/TypeInfer.unionTypes.test.cpp @@ -199,7 +199,7 @@ TEST_CASE_FIXTURE(Fixture, "index_on_a_union_type_with_missing_property") CHECK_EQ(mup->missing[0], *bTy); CHECK_EQ(mup->key, "x"); - CHECK_EQ("*unknown*", toString(requireType("r"))); + CHECK_EQ("", toString(requireType("r"))); } TEST_CASE_FIXTURE(Fixture, "index_on_a_union_type_with_one_property_of_type_any") diff --git a/tests/TypeInfer.unknownnever.test.cpp b/tests/TypeInfer.unknownnever.test.cpp new file mode 100644 index 00000000..bc742b03 --- /dev/null +++ b/tests/TypeInfer.unknownnever.test.cpp @@ -0,0 +1,280 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details + +#include "Fixture.h" + +#include "doctest.h" + +using namespace Luau; + +TEST_SUITE_BEGIN("TypeInferUnknownNever"); + +TEST_CASE_FIXTURE(Fixture, "string_subtype_and_unknown_supertype") +{ + CheckResult result = check(R"( + local function f(x: string) + local foo: unknown = x + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "unknown_subtype_and_string_supertype") +{ + CheckResult result = check(R"( + local function f(x: unknown) + local foo: string = x + end + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); +} + +TEST_CASE_FIXTURE(Fixture, "unknown_is_reflexive") +{ + CheckResult result = check(R"( + local function f(x: unknown) + local foo: unknown = x + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "string_subtype_and_never_supertype") +{ + CheckResult result = check(R"( + local function f(x: string) + local foo: never = x + end + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); +} + +TEST_CASE_FIXTURE(Fixture, "never_subtype_and_string_supertype") +{ + CheckResult result = check(R"( + local function f(x: never) + local foo: string = x + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "never_is_reflexive") +{ + CheckResult result = check(R"( + local function f(x: never) + local foo: never = x + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "unknown_is_optional_because_it_too_encompasses_nil") +{ + CheckResult result = check(R"( + local t: {x: unknown} = {} + )"); +} + +TEST_CASE_FIXTURE(Fixture, "table_with_prop_of_type_never_is_uninhabitable") +{ + CheckResult result = check(R"( + local t: {x: never} = {} + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); +} + +TEST_CASE_FIXTURE(Fixture, "table_with_prop_of_type_never_is_also_reflexive") +{ + CheckResult result = check(R"( + local t: {x: never} = {x = 5 :: never} + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "array_like_table_of_never_is_inhabitable") +{ + CheckResult result = check(R"( + local t: {never} = {} + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "type_packs_containing_never_is_itself_uninhabitable") +{ + CheckResult result = check(R"( + local function f() return "foo", 5 :: never end + + local x, y, z = f() + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ("never", toString(requireType("x"))); + CHECK_EQ("never", toString(requireType("y"))); + CHECK_EQ("never", toString(requireType("z"))); +} + +TEST_CASE_FIXTURE(Fixture, "type_packs_containing_never_is_itself_uninhabitable2") +{ + CheckResult result = check(R"( + local function f(): (string, never) return "", 5 :: never end + local function g(): (never, string) return 5 :: never, "" end + + local x1, x2 = f() + local y1, y2 = g() + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ("never", toString(requireType("x1"))); + CHECK_EQ("never", toString(requireType("x2"))); + CHECK_EQ("never", toString(requireType("y1"))); + CHECK_EQ("never", toString(requireType("y2"))); +} + +TEST_CASE_FIXTURE(Fixture, "index_on_never") +{ + CheckResult result = check(R"( + local x: never = 5 :: never + local z = x.y + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ("never", toString(requireType("z"))); +} + +TEST_CASE_FIXTURE(Fixture, "call_never") +{ + CheckResult result = check(R"( + local f: never = 5 :: never + local x, y, z = f() + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ("never", toString(requireType("x"))); + CHECK_EQ("never", toString(requireType("y"))); + CHECK_EQ("never", toString(requireType("z"))); +} + +TEST_CASE_FIXTURE(Fixture, "assign_to_local_which_is_never") +{ + CheckResult result = check(R"( + local t: never + t = 3 + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "assign_to_global_which_is_never") +{ + CheckResult result = check(R"( + --!nonstrict + t = 5 :: never + t = "" + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "assign_to_prop_which_is_never") +{ + CheckResult result = check(R"( + local t: never + t.x = 5 + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "assign_to_subscript_which_is_never") +{ + CheckResult result = check(R"( + local t: never + t[5] = 7 + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "assign_to_subscript_which_is_never") +{ + CheckResult result = check(R"( + for i, v in (5 :: never) do + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "pick_never_from_variadic_type_pack") +{ + CheckResult result = check(R"( + local function f(...: never) + local x, y = (...) + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "index_on_union_of_tables_for_properties_that_is_never") +{ + CheckResult result = check(R"( + type Disjoint = {foo: never, bar: unknown, tag: "ok"} | {foo: never, baz: unknown, tag: "err"} + local disjoint: Disjoint = {foo = 5 :: never, bar = true, tag = "ok"} + local foo = disjoint.foo + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ("never", toString(requireType("foo"))); +} + +TEST_CASE_FIXTURE(Fixture, "index_on_union_of_tables_for_properties_that_is_sorta_never") +{ + CheckResult result = check(R"( + type Disjoint = {foo: string, bar: unknown, tag: "ok"} | {foo: never, baz: unknown, tag: "err"} + local disjoint: Disjoint = {foo = 5 :: never, bar = true, tag = "ok"} + local foo = disjoint.foo + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ("string", toString(requireType("foo"))); +} + +TEST_CASE_FIXTURE(Fixture, "unary_minus_of_never") +{ + CheckResult result = check(R"( + local x = -(5 :: never) + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ("never", toString(requireType("x"))); +} + +TEST_CASE_FIXTURE(Fixture, "length_of_never") +{ + CheckResult result = check(R"( + local x = #({} :: never) + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ("never", toString(requireType("x"))); +} + +TEST_SUITE_END(); diff --git a/tests/TypePack.test.cpp b/tests/TypePack.test.cpp index 8a5a65fe..35852c05 100644 --- a/tests/TypePack.test.cpp +++ b/tests/TypePack.test.cpp @@ -199,8 +199,6 @@ TEST_CASE_FIXTURE(TypePackFixture, "std_distance") TEST_CASE("content_reassignment") { - ScopedFastFlag luauNonCopyableTypeVarFields{"LuauNonCopyableTypeVarFields", true}; - TypePackVar myError{Unifiable::Error{}, /*presistent*/ true}; TypeArena arena; diff --git a/tests/TypeVar.test.cpp b/tests/TypeVar.test.cpp index 4f8fc502..f4670048 100644 --- a/tests/TypeVar.test.cpp +++ b/tests/TypeVar.test.cpp @@ -418,8 +418,6 @@ TEST_CASE("proof_that_isBoolean_uses_all_of") TEST_CASE("content_reassignment") { - ScopedFastFlag luauNonCopyableTypeVarFields{"LuauNonCopyableTypeVarFields", true}; - TypeVar myAny{AnyTypeVar{}, /*presistent*/ true}; myAny.normal = true; myAny.documentationSymbol = "@global/any"; diff --git a/tests/conformance/vector.lua b/tests/conformance/vector.lua index 22d6adfc..6164e929 100644 --- a/tests/conformance/vector.lua +++ b/tests/conformance/vector.lua @@ -101,4 +101,20 @@ if vector_size == 4 then assert(vector(1, 2, 3, 4).W == 4) end +-- negative zero should hash the same as zero +-- note: our earlier test only really checks the low hash bit, so in absence of perfect avalanche it's insufficient +do + local larget = {} + for i = 1, 2^14 do + larget[vector(0, 0, i)] = true + end + + larget[vector(0, 0, 0)] = 42 + + assert(larget[vector(0, 0, 0)] == 42) + assert(larget[vector(0, 0, -0)] == 42) + assert(larget[vector(0, -0, 0)] == 42) + assert(larget[vector(-0, 0, 0)] == 42) +end + return 'OK' diff --git a/tools/natvis/VM.natvis b/tools/natvis/VM.natvis index ccc7e390..cb2f355f 100644 --- a/tools/natvis/VM.natvis +++ b/tools/natvis/VM.natvis @@ -137,16 +137,28 @@ {data,s} + + + + + + + + + empty + none + + + {proto()->source->data,sb}:{line()} function {proto()->debugname->data,sb}() + {proto()->source->data,sb}:{line()} function() + + + =[C] function {cl().c.debugname,sb}() {cl().c.f,na} + =[C] {cl().c.f,na} + + - - {ci->func->value.gc->cl.c.f,na} - - - {ci->func->value.gc->cl.l.p->source->data,sb}:{ci->func->value.gc->cl.l.p->linedefined,d} {ci->func->value.gc->cl.l.p->debugname->data,sb} - - - {ci->func->value.gc->cl.l.p->source->data,sb}:{ci->func->value.gc->cl.l.p->linedefined,d} - + {ci,na} thread @@ -156,7 +168,7 @@ ci-base_ci - base_ci[ci-base_ci - $i].func->value.gc->cl,view(short) + base_ci[ci-base_ci - $i] From 6ad8239e3249d8915e5ad247ad9e429b869f4db4 Mon Sep 17 00:00:00 2001 From: Anaminus Date: Fri, 8 Jul 2022 17:06:25 +0000 Subject: [PATCH 309/399] Improve description of bit32.extract/replace. (#585) Fix description incorrectly saying that parameter w specifies an upper range. w is actually a width. Proof: print(bit32.extract(2^32-1, 3, 4)) -- prints 15, not 1. Also indicate that the position is 0-based, and that the function will error if the selected range exceeds the allowed bounds. --- docs/_pages/library.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/_pages/library.md b/docs/_pages/library.md index f419d2bf..6ca2c05a 100644 --- a/docs/_pages/library.md +++ b/docs/_pages/library.md @@ -683,7 +683,7 @@ Perform a bitwise `and` of all input numbers, and return `true` iff the result i function bit32.extract(n: number, f: number, w: number?): number ``` -Extracts bits at positions `[f..w]` and returns the resulting integer. `w` defaults to `f+1`, so a two-argument version of `extract` returns the bit value at position `f`. +Extracts bits of `n` at position `f` with a width of `w`, and returns the resulting integer. `w` defaults to `1`, so a two-argument version of `extract` returns the bit value at position `f`. Bits are indexed starting at 0. Errors if `f` and `f+w-1` are not between 0 and 31. ``` function bit32.lrotate(n: number, i: number): number @@ -701,7 +701,7 @@ Shifts `n` to the left by `i` bits (if `i` is negative, a right shift is perform function bit32.replace(n: number, r: number, f: number, w: number?): number ``` -Replaces bits at positions `[f..w]` of number `n` with `r` and returns the resulting integer. `w` defaults to `f+1`, so a three-argument version of `replace` changes one bit at position `f` to `r` (which should be 0 or 1) and returns the result. +Replaces bits of `n` at position `f` and width `w` with `r`, and returns the resulting integer. `w` defaults to `1`, so a three-argument version of `replace` changes one bit at position `f` to `r` (which should be 0 or 1) and returns the result. Bits are indexed starting at 0. Errors if `f` and `f+w-1` are not between 0 and 31. ``` function bit32.rrotate(n: number, i: number): number From a934f742d81aefa53e8e45fa7eec0042d3fde7fc Mon Sep 17 00:00:00 2001 From: Andy Friesen Date: Mon, 11 Jul 2022 13:21:23 -0700 Subject: [PATCH 310/399] June recap (#583) --- .../_posts/2022-07-07-luau-recap-june-2022.md | 88 +++++++++++++++++++ 1 file changed, 88 insertions(+) create mode 100644 docs/_posts/2022-07-07-luau-recap-june-2022.md diff --git a/docs/_posts/2022-07-07-luau-recap-june-2022.md b/docs/_posts/2022-07-07-luau-recap-june-2022.md new file mode 100644 index 00000000..1f58d892 --- /dev/null +++ b/docs/_posts/2022-07-07-luau-recap-june-2022.md @@ -0,0 +1,88 @@ +--- +layout: single +title: "Luau Recap: June 2022" +--- + +Luau is our new language that you can read more about at [https://luau-lang.org](https://luau-lang.org). + +[Cross-posted to the [Roblox Developer Forum](https://devforum.roblox.com/t/luau-recap-june-2022/).] + +# Lower bounds calculation + +A common problem that Luau has is that it primarily works by inspecting expressions in your program and narrowing the _upper bounds_ of the values that can inhabit particular variables. In other words, each time we see a variable used, we eliminate possible sets of values from that variable's domain. + +There are some important cases where this doesn't produce a helpful result. Take this function for instance: + +```lua +function find_first_if(vec, f) + for i, e in ipairs(vec) do + if f(e) then + return i + end + end + + return nil +end +``` + +Luau scans the function from top to bottom and first sees the line `return i`. It draws from this the inference that `find_first_if` must return the type of `i`, namely `number`. + +This is fine, but things go sour when we see the line `return nil`. Since we are always narrowing, we take from this line the judgement that the return type of the function is `nil`. Since we have already concluded that the function must return `number`, Luau reports an error. + +What we actually want to do in this case is to take these `return` statements as inferences about the _lower_ bound of the function's return type. Instead of saying "this function must return values of type `nil`," we should instead say "this function may _also_ return values of type `nil`." + +Lower bounds calculation does precisely this. Moving forward, Luau will instead infer the type `number?` for the above function. + +This does have one unfortunate consequence: If a function has no return type annotation, we will no longer ever report a type error on a `return` statement. We think this is the right balance but we'll be keeping an eye on things just to be sure. + +Lower-bounds calculation is larger and a little bit riskier than other things we've been working on so we've set up a beta feature in Roblox Studio to enable them. It is called "Experimental Luau language features." + +Please try it out and let us know what you think! + +## Known bug + +We have a known bug with certain kinds of cyclic types when lower-bounds calculation is enabled. The following, for instance, is known to be problematic. + +```lua +type T = {T?}? -- spuriously reduces to {nil}? +``` + +We hope to have this fixed soon. + +# All table literals now result in unsealed tables + +Previously, the only way to create a sealed table was by with a literal empty table. We have relaxed this somewhat: Any table created by a `{}` expression is considered to be unsealed within the scope where it was created: + +```lua +local T = {} +T.x = 5 -- OK + +local V = {x=5} +V.y = 2 -- previously disallowed. Now OK. + +function mkTable() + return {x = 5} +end + +local U = mkTable() +U.y = 2 -- Still disallowed: U is sealed +``` + +# Other fixes + +* Adjust indentation and whitespace when creating multiline string representations of types, resulting in types that are easier to read. +* Some small bugfixes to autocomplete +* Fix a case where accessing a nonexistent property of a table would not result in an error being reported. +* Improve parser recovery for the incorrect code `function foo() -> ReturnType` (the correct syntax is `function foo(): ReturnType`) +* Improve the parse error offered for code that improperly uses the `function` keyword to start a type eg `type T = function` +* Some small crash fixes and performance improvements + +# Thanks! + +A very special thanks to all of our open source contributors: + +* [Allan N Jeremy](https://github.com/AllanJeremy) +* [Daniel Nachun](https://github.com/danielnachun) +* [JohnnyMorganz](https://github.com/JohnnyMorganz/) +* [Petri Häkkinen](https://github.com/petrihakkinen) +* [Qualadore](https://github.com/Qualadore) From e87009f5b278dec0213b6f1ed33a422f55e4ef48 Mon Sep 17 00:00:00 2001 From: Alex Orlenko Date: Thu, 14 Jul 2022 20:00:37 +0100 Subject: [PATCH 311/399] Add lua_setuserdatatag to update userdata tags (#588) --- VM/include/lua.h | 5 +++-- VM/src/lapi.cpp | 8 ++++++++ tests/Conformance.test.cpp | 4 ++++ 3 files changed, 15 insertions(+), 2 deletions(-) diff --git a/VM/include/lua.h b/VM/include/lua.h index 7f9647c8..72edb4a6 100644 --- a/VM/include/lua.h +++ b/VM/include/lua.h @@ -64,14 +64,14 @@ enum lua_Type LUA_TNIL = 0, /* must be 0 due to lua_isnoneornil */ LUA_TBOOLEAN = 1, /* must be 1 due to l_isfalse */ - + LUA_TLIGHTUSERDATA, LUA_TNUMBER, LUA_TVECTOR, LUA_TSTRING, /* all types above this must be value types, all types below this must be GC types - see iscollectable */ - + LUA_TTABLE, LUA_TFUNCTION, LUA_TUSERDATA, @@ -300,6 +300,7 @@ LUA_API uintptr_t lua_encodepointer(lua_State* L, uintptr_t p); LUA_API double lua_clock(); +LUA_API void lua_setuserdatatag(lua_State* L, int idx, int tag); LUA_API void lua_setuserdatadtor(lua_State* L, int tag, void (*dtor)(lua_State*, void*)); LUA_API void lua_clonefunction(lua_State* L, int idx); diff --git a/VM/src/lapi.cpp b/VM/src/lapi.cpp index 3c3b7bd0..c2b790ad 100644 --- a/VM/src/lapi.cpp +++ b/VM/src/lapi.cpp @@ -1305,6 +1305,14 @@ void lua_unref(lua_State* L, int ref) return; } +void lua_setuserdatatag(lua_State* L, int idx, int tag) +{ + api_check(L, unsigned(tag) < LUA_UTAG_LIMIT); + StkId o = index2addr(L, idx); + api_check(L, ttisuserdata(o)); + uvalue(o)->tag = uint8_t(tag); +} + void lua_setuserdatadtor(lua_State* L, int tag, void (*dtor)(lua_State*, void*)) { api_check(L, unsigned(tag) < LUA_UTAG_LIMIT); diff --git a/tests/Conformance.test.cpp b/tests/Conformance.test.cpp index 3f415149..d2311348 100644 --- a/tests/Conformance.test.cpp +++ b/tests/Conformance.test.cpp @@ -1152,6 +1152,10 @@ TEST_CASE("UserdataApi") CHECK(lua_touserdatatagged(L, -1, 41) == nullptr); CHECK(lua_userdatatag(L, -1) == 42); + lua_setuserdatatag(L, -1, 43); + CHECK(lua_userdatatag(L, -1) == 43); + lua_setuserdatatag(L, -1, 42); + // user data with inline dtor void* ud3 = lua_newuserdatadtor(L, 4, [](void* data) { dtorhits += *(int*)data; From 4bd651292d4f8d7fce1cc7ebfa2536037d344bff Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Thu, 14 Jul 2022 15:39:35 -0700 Subject: [PATCH 312/399] Sync to upstream/release/536 --- Analysis/include/Luau/Autocomplete.h | 10 - .../include/Luau/ConstraintGraphBuilder.h | 1 + Analysis/include/Luau/Frontend.h | 7 - Analysis/include/Luau/Unifier.h | 1 + Analysis/src/Autocomplete.cpp | 37 +- Analysis/src/ConstraintGraphBuilder.cpp | 23 + Analysis/src/Frontend.cpp | 25 +- Analysis/src/Linter.cpp | 16 + Analysis/src/Normalize.cpp | 160 +- Analysis/src/ToString.cpp | 68 +- Analysis/src/TypeChecker2.cpp | 7 +- Analysis/src/TypeInfer.cpp | 37 +- Analysis/src/TypeVar.cpp | 129 +- Analysis/src/Unifier.cpp | 74 +- Ast/src/Parser.cpp | 10 +- CMakeLists.txt | 1 + Compiler/include/Luau/Compiler.h | 4 +- Compiler/src/BuiltinFolding.cpp | 463 +++ Compiler/src/BuiltinFolding.h | 14 + Compiler/src/Builtins.cpp | 49 +- Compiler/src/Builtins.h | 4 +- Compiler/src/BytecodeBuilder.cpp | 15 +- Compiler/src/Compiler.cpp | 136 +- Compiler/src/ConstantFolding.cpp | 48 +- Compiler/src/ConstantFolding.h | 6 +- Compiler/src/CostModel.cpp | 26 +- Compiler/src/CostModel.h | 3 +- Makefile | 5 +- Sources.cmake | 2 + VM/include/lua.h | 2 +- VM/src/lapi.cpp | 19 +- VM/src/ldebug.cpp | 75 +- VM/src/lstring.cpp | 6 +- VM/src/lstring.h | 3 + bench/gc/test_SunSpider_crypto-aes.lua | 436 --- bench/tests/sunspider/crypto-aes.lua | 18 +- extern/doctest.h | 3089 ++++++++++------- tests/Autocomplete.test.cpp | 66 +- tests/Compiler.test.cpp | 475 ++- tests/Conformance.test.cpp | 72 +- tests/CostModel.test.cpp | 4 +- tests/Frontend.test.cpp | 20 - tests/Linter.test.cpp | 17 + tests/Normalize.test.cpp | 59 + tests/ToString.test.cpp | 2 - tests/TypeInfer.builtins.test.cpp | 110 + tests/TypeInfer.primitives.test.cpp | 7 +- tests/TypeInfer.singletons.test.cpp | 17 + tests/TypeInfer.tables.test.cpp | 82 + tests/TypeInfer.unknownnever.test.cpp | 38 +- tests/conformance/basic.lua | 16 +- tests/main.cpp | 17 +- tools/patchtests.py | 12 +- 53 files changed, 3875 insertions(+), 2168 deletions(-) create mode 100644 Compiler/src/BuiltinFolding.cpp create mode 100644 Compiler/src/BuiltinFolding.h delete mode 100644 bench/gc/test_SunSpider_crypto-aes.lua diff --git a/Analysis/include/Luau/Autocomplete.h b/Analysis/include/Luau/Autocomplete.h index 65b788d3..5e8d6605 100644 --- a/Analysis/include/Luau/Autocomplete.h +++ b/Analysis/include/Luau/Autocomplete.h @@ -78,16 +78,6 @@ struct AutocompleteResult using ModuleName = std::string; using StringCompletionCallback = std::function(std::string tag, std::optional ctx)>; -struct OwningAutocompleteResult -{ - AutocompleteResult result; - ModulePtr module; - std::unique_ptr sourceModule; -}; - AutocompleteResult autocomplete(Frontend& frontend, const ModuleName& moduleName, Position position, StringCompletionCallback callback); -// Deprecated, do not use in new work. -OwningAutocompleteResult autocompleteSource(Frontend& frontend, std::string_view source, Position position, StringCompletionCallback callback); - } // namespace Luau diff --git a/Analysis/include/Luau/ConstraintGraphBuilder.h b/Analysis/include/Luau/ConstraintGraphBuilder.h index a49e8594..513446fd 100644 --- a/Analysis/include/Luau/ConstraintGraphBuilder.h +++ b/Analysis/include/Luau/ConstraintGraphBuilder.h @@ -99,6 +99,7 @@ struct ConstraintGraphBuilder void visit(NotNull scope, AstStat* stat); void visit(NotNull scope, AstStatBlock* block); void visit(NotNull scope, AstStatLocal* local); + void visit(NotNull scope, AstStatFor* for_); void visit(NotNull scope, AstStatLocalFunction* function); void visit(NotNull scope, AstStatFunction* function); void visit(NotNull scope, AstStatReturn* ret); diff --git a/Analysis/include/Luau/Frontend.h b/Analysis/include/Luau/Frontend.h index f0d43090..27fd4d7e 100644 --- a/Analysis/include/Luau/Frontend.h +++ b/Analysis/include/Luau/Frontend.h @@ -127,13 +127,6 @@ struct Frontend CheckResult check(const ModuleName& name, std::optional optionOverride = {}); // new shininess LintResult lint(const ModuleName& name, std::optional enabledLintWarnings = {}); - /** Lint some code that has no associated DataModel object - * - * Since this source fragment has no name, we cannot cache its AST. Instead, - * we return it to the caller to use as they wish. - */ - std::pair lintFragment(std::string_view source, std::optional enabledLintWarnings = {}); - LintResult lint(const SourceModule& module, std::optional enabledLintWarnings = {}); bool isDirty(const ModuleName& name, bool forAutocomplete = false) const; diff --git a/Analysis/include/Luau/Unifier.h b/Analysis/include/Luau/Unifier.h index 4af324cb..f460dc87 100644 --- a/Analysis/include/Luau/Unifier.h +++ b/Analysis/include/Luau/Unifier.h @@ -79,6 +79,7 @@ private: void tryUnifySingletons(TypeId subTy, TypeId superTy); void tryUnifyFunctions(TypeId subTy, TypeId superTy, bool isFunctionCall = false); void tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection = false); + void tryUnifyScalarShape(TypeId subTy, TypeId superTy, bool reversed); void tryUnifyWithMetatable(TypeId subTy, TypeId superTy, bool reversed); void tryUnifyWithClass(TypeId subTy, TypeId superTy, bool reversed); diff --git a/Analysis/src/Autocomplete.cpp b/Analysis/src/Autocomplete.cpp index cc54d499..513a791c 100644 --- a/Analysis/src/Autocomplete.cpp +++ b/Analysis/src/Autocomplete.cpp @@ -7,7 +7,6 @@ #include "Luau/ToString.h" #include "Luau/TypeInfer.h" #include "Luau/TypePack.h" -#include "Luau/Parser.h" // TODO: only needed for autocompleteSource which is deprecated #include #include @@ -1407,8 +1406,8 @@ static AutocompleteResult autocomplete(const SourceModule& sourceModule, const M PropIndexType indexType = indexName->op == ':' ? PropIndexType::Colon : PropIndexType::Point; if (!FFlag::LuauSelfCallAutocompleteFix2 && isString(ty)) - return {autocompleteProps(*module, typeArena, typeChecker.globalScope->bindings[AstName{"string"}].typeId, indexType, ancestry), - ancestry}; + return { + autocompleteProps(*module, typeArena, typeChecker.globalScope->bindings[AstName{"string"}].typeId, indexType, ancestry), ancestry}; else return {autocompleteProps(*module, typeArena, ty, indexType, ancestry), ancestry}; } @@ -1507,8 +1506,8 @@ static AutocompleteResult autocomplete(const SourceModule& sourceModule, const M else if (AstStatIf* statIf = node->as(); statIf && !statIf->elseLocation.has_value()) { - return {{{"else", AutocompleteEntry{AutocompleteEntryKind::Keyword}}, {"elseif", AutocompleteEntry{AutocompleteEntryKind::Keyword}}}, - ancestry}; + return { + {{"else", AutocompleteEntry{AutocompleteEntryKind::Keyword}}, {"elseif", AutocompleteEntry{AutocompleteEntryKind::Keyword}}}, ancestry}; } else if (AstStatIf* statIf = parent->as(); statIf && node->is()) { @@ -1628,32 +1627,4 @@ AutocompleteResult autocomplete(Frontend& frontend, const ModuleName& moduleName return autocompleteResult; } -OwningAutocompleteResult autocompleteSource(Frontend& frontend, std::string_view source, Position position, StringCompletionCallback callback) -{ - // TODO: Remove #include "Luau/Parser.h" with this function - auto sourceModule = std::make_unique(); - ParseOptions parseOptions; - parseOptions.captureComments = true; - ParseResult result = Parser::parse(source.data(), source.size(), *sourceModule->names, *sourceModule->allocator, parseOptions); - - if (!result.root) - return {AutocompleteResult{}, {}, nullptr}; - - sourceModule->name = "FRAGMENT_SCRIPT"; - sourceModule->root = result.root; - sourceModule->mode = Mode::Strict; - sourceModule->commentLocations = std::move(result.commentLocations); - - TypeChecker& typeChecker = frontend.typeCheckerForAutocomplete; - ModulePtr module = typeChecker.check(*sourceModule, Mode::Strict); - - OwningAutocompleteResult autocompleteResult = { - autocomplete(*sourceModule, module, typeChecker, &frontend.arenaForAutocomplete, position, callback), std::move(module), - std::move(sourceModule)}; - - frontend.arenaForAutocomplete.clear(); - - return autocompleteResult; -} - } // namespace Luau diff --git a/Analysis/src/ConstraintGraphBuilder.cpp b/Analysis/src/ConstraintGraphBuilder.cpp index 3b9000cd..38895ceb 100644 --- a/Analysis/src/ConstraintGraphBuilder.cpp +++ b/Analysis/src/ConstraintGraphBuilder.cpp @@ -103,6 +103,8 @@ void ConstraintGraphBuilder::visit(NotNull scope, AstStat* stat) visit(scope, s); else if (auto s = stat->as()) visit(scope, s); + else if (auto s = stat->as()) + visit(scope, s); else if (auto f = stat->as()) visit(scope, f); else if (auto f = stat->as()) @@ -167,6 +169,27 @@ void ConstraintGraphBuilder::visit(NotNull scope, AstStatLocal* local) } } +void ConstraintGraphBuilder::visit(NotNull scope, AstStatFor* for_) +{ + auto checkNumber = [&](AstExpr* expr) + { + if (!expr) + return; + + TypeId t = check(scope, expr); + addConstraint(scope, SubtypeConstraint{t, singletonTypes.numberType}); + }; + + checkNumber(for_->from); + checkNumber(for_->to); + checkNumber(for_->step); + + NotNull forScope = childScope(for_->location, scope); + forScope->bindings[for_->var] = singletonTypes.numberType; + + visit(forScope, for_->body); +} + void addConstraints(Constraint* constraint, NotNull scope) { scope->constraints.reserve(scope->constraints.size() + scope->constraints.size()); diff --git a/Analysis/src/Frontend.cpp b/Analysis/src/Frontend.cpp index 9195363d..a7670de5 100644 --- a/Analysis/src/Frontend.cpp +++ b/Analysis/src/Frontend.cpp @@ -662,29 +662,6 @@ LintResult Frontend::lint(const ModuleName& name, std::optional Frontend::lintFragment(std::string_view source, std::optional enabledLintWarnings) -{ - LUAU_TIMETRACE_SCOPE("Frontend::lintFragment", "Frontend"); - - const Config& config = configResolver->getConfig(""); - - SourceModule sourceModule = parse(ModuleName{}, source, config.parseOptions); - - uint64_t ignoreLints = LintWarning::parseMask(sourceModule.hotcomments); - - Luau::LintOptions lintOptions = enabledLintWarnings.value_or(config.enabledLint); - lintOptions.warningMask &= ~ignoreLints; - - double timestamp = getTimestamp(); - - std::vector warnings = Luau::lint(sourceModule.root, *sourceModule.names.get(), typeChecker.globalScope, nullptr, - sourceModule.hotcomments, enabledLintWarnings.value_or(config.enabledLint)); - - stats.timeLint += getTimestamp() - timestamp; - - return {std::move(sourceModule), classifyLints(warnings, config)}; -} - LintResult Frontend::lint(const SourceModule& module, std::optional enabledLintWarnings) { LUAU_TIMETRACE_SCOPE("Frontend::lint", "Frontend"); @@ -958,7 +935,7 @@ std::optional FrontendModuleResolver::resolveModuleInfo(const Module { // CLI-43699 // If we can't find the current module name, that's because we bypassed the frontend's initializer - // and called typeChecker.check directly. (This is done by autocompleteSource, for example). + // and called typeChecker.check directly. // In that case, requires will always fail. return std::nullopt; } diff --git a/Analysis/src/Linter.cpp b/Analysis/src/Linter.cpp index 50868e56..fb952f5e 100644 --- a/Analysis/src/Linter.cpp +++ b/Analysis/src/Linter.cpp @@ -2688,6 +2688,21 @@ static void lintComments(LintContext& context, const std::vector& ho else seenMode = true; } + else if (first == "optimize") + { + size_t notspace = hc.content.find_first_not_of(" \t", space); + + if (space == std::string::npos || notspace == std::string::npos) + emitWarning(context, LintWarning::Code_CommentDirective, hc.location, "optimize directive requires an optimization level"); + else + { + const char* level = hc.content.c_str() + notspace; + + if (strcmp(level, "0") && strcmp(level, "1") && strcmp(level, "2")) + emitWarning(context, LintWarning::Code_CommentDirective, hc.location, + "optimize directive uses unknown optimization level '%s', 0..2 expected", level); + } + } else { static const char* kHotComments[] = { @@ -2695,6 +2710,7 @@ static void lintComments(LintContext& context, const std::vector& ho "nocheck", "nonstrict", "strict", + "optimize", }; if (const char* suggestion = fuzzyMatch(first, kHotComments, std::size(kHotComments))) diff --git a/Analysis/src/Normalize.cpp b/Analysis/src/Normalize.cpp index ce8f96cc..a96b557d 100644 --- a/Analysis/src/Normalize.cpp +++ b/Analysis/src/Normalize.cpp @@ -14,6 +14,7 @@ LUAU_FASTFLAGVARIABLE(DebugLuauCopyBeforeNormalizing, false) LUAU_FASTINTVARIABLE(LuauNormalizeIterationLimit, 1200); LUAU_FASTFLAGVARIABLE(LuauNormalizeCombineTableFix, false); LUAU_FASTFLAGVARIABLE(LuauNormalizeFlagIsConservative, false); +LUAU_FASTFLAGVARIABLE(LuauFixNormalizationOfCyclicUnions, false); LUAU_FASTFLAG(LuauUnknownAndNeverType) LUAU_FASTFLAG(LuauQuantifyConstrained) @@ -340,13 +341,19 @@ struct Normalize final : TypeVarVisitor return false; UnionTypeVar* utv = &const_cast(utvRef); - std::vector options = std::move(utv->options); + + // TODO: Clip tempOptions and optionsRef when clipping FFlag::LuauFixNormalizationOfCyclicUnions + std::vector tempOptions; + if (!FFlag::LuauFixNormalizationOfCyclicUnions) + tempOptions = std::move(utv->options); + + std::vector& optionsRef = FFlag::LuauFixNormalizationOfCyclicUnions ? utv->options : tempOptions; // We might transmute, so it's not safe to rely on the builtin traversal logic of visitTypeVar - for (TypeId option : options) + for (TypeId option : optionsRef) traverse(option); - std::vector newOptions = normalizeUnion(options); + std::vector newOptions = normalizeUnion(optionsRef); const bool normal = areNormal(newOptions, seen, ice); @@ -371,51 +378,106 @@ struct Normalize final : TypeVarVisitor IntersectionTypeVar* itv = &const_cast(itvRef); - std::vector oldParts = std::move(itv->parts); - - for (TypeId part : oldParts) - traverse(part); - - std::vector tables; - for (TypeId part : oldParts) + if (FFlag::LuauFixNormalizationOfCyclicUnions) { - part = follow(part); - if (get(part)) - tables.push_back(part); - else - { - Replacer replacer{&arena, nullptr, nullptr}; // FIXME this is super super WEIRD - combineIntoIntersection(replacer, itv, part); - } - } + std::vector oldParts = itv->parts; + IntersectionTypeVar newIntersection; - // Don't allocate a new table if there's just one in the intersection. - if (tables.size() == 1) - itv->parts.push_back(tables[0]); - else if (!tables.empty()) - { - const TableTypeVar* first = get(tables[0]); - LUAU_ASSERT(first); + for (TypeId part : oldParts) + traverse(part); - TypeId newTable = arena.addType(TableTypeVar{first->state, first->level}); - TableTypeVar* ttv = getMutable(newTable); - for (TypeId part : tables) + std::vector tables; + for (TypeId part : oldParts) { - // Intuition: If combineIntoTable() needs to clone a table, any references to 'part' are cyclic and need - // to be rewritten to point at 'newTable' in the clone. - Replacer replacer{&arena, part, newTable}; - combineIntoTable(replacer, ttv, part); + part = follow(part); + if (get(part)) + tables.push_back(part); + else + { + Replacer replacer{&arena, nullptr, nullptr}; // FIXME this is super super WEIRD + combineIntoIntersection(replacer, &newIntersection, part); + } } - itv->parts.push_back(newTable); + // Don't allocate a new table if there's just one in the intersection. + if (tables.size() == 1) + newIntersection.parts.push_back(tables[0]); + else if (!tables.empty()) + { + const TableTypeVar* first = get(tables[0]); + LUAU_ASSERT(first); + + TypeId newTable = arena.addType(TableTypeVar{first->state, first->level}); + TableTypeVar* ttv = getMutable(newTable); + for (TypeId part : tables) + { + // Intuition: If combineIntoTable() needs to clone a table, any references to 'part' are cyclic and need + // to be rewritten to point at 'newTable' in the clone. + Replacer replacer{&arena, part, newTable}; + combineIntoTable(replacer, ttv, part); + } + + newIntersection.parts.push_back(newTable); + } + + itv->parts = std::move(newIntersection.parts); + + asMutable(ty)->normal = areNormal(itv->parts, seen, ice); + + if (itv->parts.size() == 1) + { + TypeId part = itv->parts[0]; + *asMutable(ty) = BoundTypeVar{part}; + } } - - asMutable(ty)->normal = areNormal(itv->parts, seen, ice); - - if (itv->parts.size() == 1) + else { - TypeId part = itv->parts[0]; - *asMutable(ty) = BoundTypeVar{part}; + std::vector oldParts = std::move(itv->parts); + + for (TypeId part : oldParts) + traverse(part); + + std::vector tables; + for (TypeId part : oldParts) + { + part = follow(part); + if (get(part)) + tables.push_back(part); + else + { + Replacer replacer{&arena, nullptr, nullptr}; // FIXME this is super super WEIRD + combineIntoIntersection(replacer, itv, part); + } + } + + // Don't allocate a new table if there's just one in the intersection. + if (tables.size() == 1) + itv->parts.push_back(tables[0]); + else if (!tables.empty()) + { + const TableTypeVar* first = get(tables[0]); + LUAU_ASSERT(first); + + TypeId newTable = arena.addType(TableTypeVar{first->state, first->level}); + TableTypeVar* ttv = getMutable(newTable); + for (TypeId part : tables) + { + // Intuition: If combineIntoTable() needs to clone a table, any references to 'part' are cyclic and need + // to be rewritten to point at 'newTable' in the clone. + Replacer replacer{&arena, part, newTable}; + combineIntoTable(replacer, ttv, part); + } + + itv->parts.push_back(newTable); + } + + asMutable(ty)->normal = areNormal(itv->parts, seen, ice); + + if (itv->parts.size() == 1) + { + TypeId part = itv->parts[0]; + *asMutable(ty) = BoundTypeVar{part}; + } } return false; @@ -590,6 +652,24 @@ struct Normalize final : TypeVarVisitor table->props.insert({propName, prop}); } + if (FFlag::LuauFixNormalizationOfCyclicUnions) + { + if (tyTable->indexer) + { + if (table->indexer) + { + table->indexer->indexType = combine(replacer, replacer.smartClone(tyTable->indexer->indexType), table->indexer->indexType); + table->indexer->indexResultType = + combine(replacer, replacer.smartClone(tyTable->indexer->indexResultType), table->indexer->indexResultType); + } + else + { + table->indexer = + TableIndexer{replacer.smartClone(tyTable->indexer->indexType), replacer.smartClone(tyTable->indexer->indexResultType)}; + } + } + } + table->state = combineTableStates(table->state, tyTable->state); table->level = max(table->level, tyTable->level); } diff --git a/Analysis/src/ToString.cpp b/Analysis/src/ToString.cpp index c67d6397..d571adf3 100644 --- a/Analysis/src/ToString.cpp +++ b/Analysis/src/ToString.cpp @@ -19,7 +19,6 @@ LUAU_FASTFLAG(LuauUnknownAndNeverType) * Fair warning: Setting this will break a lot of Luau unit tests. */ LUAU_FASTFLAGVARIABLE(DebugLuauVerboseTypeNames, false) -LUAU_FASTFLAGVARIABLE(LuauToStringTableBracesNewlines, false) namespace Luau { @@ -571,54 +570,22 @@ struct TypeVarStringifier { case TableState::Sealed: state.result.invalid = true; - if (FFlag::LuauToStringTableBracesNewlines) - { - openbrace = "{|"; - closedbrace = "|}"; - } - else - { - openbrace = "{| "; - closedbrace = " |}"; - } + openbrace = "{|"; + closedbrace = "|}"; break; case TableState::Unsealed: - if (FFlag::LuauToStringTableBracesNewlines) - { - openbrace = "{"; - closedbrace = "}"; - } - else - { - openbrace = "{ "; - closedbrace = " }"; - } + openbrace = "{"; + closedbrace = "}"; break; case TableState::Free: state.result.invalid = true; - if (FFlag::LuauToStringTableBracesNewlines) - { - openbrace = "{-"; - closedbrace = "-}"; - } - else - { - openbrace = "{- "; - closedbrace = " -}"; - } + openbrace = "{-"; + closedbrace = "-}"; break; case TableState::Generic: state.result.invalid = true; - if (FFlag::LuauToStringTableBracesNewlines) - { - openbrace = "{+"; - closedbrace = "+}"; - } - else - { - openbrace = "{+ "; - closedbrace = " +}"; - } + openbrace = "{+"; + closedbrace = "+}"; break; } @@ -637,8 +604,7 @@ struct TypeVarStringifier bool comma = false; if (ttv.indexer) { - if (FFlag::LuauToStringTableBracesNewlines) - state.newline(); + state.newline(); state.emit("["); stringify(ttv.indexer->indexType); state.emit("]: "); @@ -655,10 +621,8 @@ struct TypeVarStringifier state.emit(","); state.newline(); } - else if (FFlag::LuauToStringTableBracesNewlines) - { + else state.newline(); - } size_t length = state.result.name.length() - oldLength; @@ -685,13 +649,10 @@ struct TypeVarStringifier } state.dedent(); - if (FFlag::LuauToStringTableBracesNewlines) - { - if (comma) - state.newline(); - else - state.emit(" "); - } + if (comma) + state.newline(); + else + state.emit(" "); state.emit(closedbrace); state.unsee(&ttv); @@ -859,7 +820,6 @@ struct TypeVarStringifier { state.emit("never"); } - }; struct TypePackStringifier diff --git a/Analysis/src/TypeChecker2.cpp b/Analysis/src/TypeChecker2.cpp index 30e498af..d3b9655d 100644 --- a/Analysis/src/TypeChecker2.cpp +++ b/Analysis/src/TypeChecker2.cpp @@ -322,10 +322,13 @@ struct TypeChecker2 : public AstVisitor { pack = follow(pack); - while (auto tp = get(pack)) + while (true) { - if (tp->head.empty() && tp->tail) + auto tp = get(pack); + if (tp && tp->head.empty() && tp->tail) pack = *tp->tail; + else + break; } if (auto ty = first(pack)) diff --git a/Analysis/src/TypeInfer.cpp b/Analysis/src/TypeInfer.cpp index 01939fda..20777928 100644 --- a/Analysis/src/TypeInfer.cpp +++ b/Analysis/src/TypeInfer.cpp @@ -48,6 +48,8 @@ LUAU_FASTFLAGVARIABLE(LuauFalsyPredicateReturnsNilInstead, false) LUAU_FASTFLAGVARIABLE(LuauCheckLenMT, false) LUAU_FASTFLAGVARIABLE(LuauCheckGenericHOFTypes, false) LUAU_FASTFLAGVARIABLE(LuauBinaryNeedsExpectedTypesToo, false) +LUAU_FASTFLAGVARIABLE(LuauNeverTypesAndOperatorsInference, false) +LUAU_FASTFLAGVARIABLE(LuauReturnsFromCallsitesAreNotWidened, false) namespace Luau { @@ -2443,8 +2445,15 @@ WithPredicate TypeChecker::checkExpr(const ScopePtr& scope, const AstExp operandType = stripFromNilAndReport(operandType, expr.location); - if (get(operandType) || get(operandType)) - return {!FFlag::LuauUnknownAndNeverType ? errorRecoveryType(scope) : operandType}; + // # operator is guaranteed to return number + if ((FFlag::LuauNeverTypesAndOperatorsInference && get(operandType)) || get(operandType) || + get(operandType)) + { + if (FFlag::LuauNeverTypesAndOperatorsInference) + return {numberType}; + else + return {!FFlag::LuauUnknownAndNeverType ? errorRecoveryType(scope) : operandType}; + } DenseHashSet seen{nullptr}; @@ -2610,6 +2619,13 @@ TypeId TypeChecker::checkRelationalOperation( case AstExprBinary::CompareGe: case AstExprBinary::CompareLe: { + if (FFlag::LuauNeverTypesAndOperatorsInference) + { + // If one of the operand is never, it doesn't make sense to unify these. + if (get(lhsType) || get(rhsType)) + return booleanType; + } + /* Subtlety here: * We need to do this unification first, but there are situations where we don't actually want to * report any problems that might have been surfaced as a result of this step because we might already @@ -2787,8 +2803,10 @@ TypeId TypeChecker::checkBinaryOperation( // If we know nothing at all about the lhs type, we can usually say nothing about the result. // The notable exception to this is the equality and inequality operators, which always produce a boolean. - const bool lhsIsAny = get(lhsType) || get(lhsType); - const bool rhsIsAny = get(rhsType) || get(rhsType); + const bool lhsIsAny = get(lhsType) || get(lhsType) || + (FFlag::LuauUnknownAndNeverType && FFlag::LuauNeverTypesAndOperatorsInference && get(lhsType)); + const bool rhsIsAny = get(rhsType) || get(rhsType) || + (FFlag::LuauUnknownAndNeverType && FFlag::LuauNeverTypesAndOperatorsInference && get(rhsType)); if (lhsIsAny) return lhsType; @@ -3775,7 +3793,10 @@ void TypeChecker::checkArgumentList( } TypePackId varPack = addTypePack(TypePackVar{TypePack{rest, paramIter.tail()}}); - state.tryUnify(varPack, tail); + if (FFlag::LuauReturnsFromCallsitesAreNotWidened) + state.tryUnify(tail, varPack); + else + state.tryUnify(varPack, tail); return; } } @@ -4414,7 +4435,8 @@ WithPredicate TypeChecker::checkExprList(const ScopePtr& scope, cons if (FFlag::LuauUnknownAndNeverType && containsNever(typePack)) { - // f(), g() where f() returns (never, string) or (string, never) means this whole TypePackId is uninhabitable, so return (never, ...never) + // f(), g() where f() returns (never, string) or (string, never) means this whole TypePackId is uninhabitable, so return (never, + // ...never) uninhabitable = true; continue; } @@ -4436,7 +4458,8 @@ WithPredicate TypeChecker::checkExprList(const ScopePtr& scope, cons if (FFlag::LuauUnknownAndNeverType && get(type)) { - // f(), g() where f() returns (never, string) or (string, never) means this whole TypePackId is uninhabitable, so return (never, ...never) + // f(), g() where f() returns (never, string) or (string, never) means this whole TypePackId is uninhabitable, so return (never, + // ...never) uninhabitable = true; continue; } diff --git a/Analysis/src/TypeVar.cpp b/Analysis/src/TypeVar.cpp index f884ad77..ebaf5906 100644 --- a/Analysis/src/TypeVar.cpp +++ b/Analysis/src/TypeVar.cpp @@ -26,6 +26,7 @@ LUAU_FASTINT(LuauTypeInferRecursionLimit) LUAU_FASTFLAG(LuauUnknownAndNeverType) LUAU_FASTFLAGVARIABLE(LuauDeduceGmatchReturnTypes, false) LUAU_FASTFLAGVARIABLE(LuauMaybeGenericIntersectionTypes, false) +LUAU_FASTFLAGVARIABLE(LuauDeduceFindMatchReturnTypes, false) namespace Luau { @@ -36,6 +37,12 @@ std::optional> magicFunctionFormat( static std::optional> magicFunctionGmatch( TypeChecker& typechecker, const ScopePtr& scope, const AstExprCall& expr, WithPredicate withPredicate); +static std::optional> magicFunctionMatch( + TypeChecker& typechecker, const ScopePtr& scope, const AstExprCall& expr, WithPredicate withPredicate); + +static std::optional> magicFunctionFind( + TypeChecker& typechecker, const ScopePtr& scope, const AstExprCall& expr, WithPredicate withPredicate); + TypeId follow(TypeId t) { return follow(t, [](TypeId t) { @@ -164,10 +171,12 @@ bool isNumber(TypeId ty) // Returns true when ty is a subtype of string bool isString(TypeId ty) { - if (isPrim(ty, PrimitiveTypeVar::String) || get(get(follow(ty)))) + ty = follow(ty); + + if (isPrim(ty, PrimitiveTypeVar::String) || get(get(ty))) return true; - if (auto utv = get(follow(ty))) + if (auto utv = get(ty)) return std::all_of(begin(utv), end(utv), isString); return false; @@ -178,8 +187,8 @@ bool maybeString(TypeId ty) { ty = follow(ty); - if (isPrim(ty, PrimitiveTypeVar::String) || get(ty)) - return true; + if (isPrim(ty, PrimitiveTypeVar::String) || get(ty)) + return true; if (auto utv = get(ty)) return std::any_of(begin(utv), end(utv), maybeString); @@ -233,6 +242,8 @@ bool isOverloadedFunction(TypeId ty) std::optional getMetatable(TypeId type) { + type = follow(type); + if (const MetatableTypeVar* mtType = get(type)) return mtType->metatable; else if (const ClassTypeVar* classType = get(type)) @@ -765,18 +776,24 @@ TypeId SingletonTypes::makeStringMetatable() makeFunction(*arena, stringType, {}, {}, {stringType}, {}, {arena->addType(FunctionTypeVar{emptyPack, stringVariadicList})}); attachMagicFunction(gmatchFunc, magicFunctionGmatch); + const TypeId matchFunc = arena->addType(FunctionTypeVar{arena->addTypePack({stringType, stringType, optionalNumber}), + arena->addTypePack(TypePackVar{VariadicTypePack{FFlag::LuauDeduceFindMatchReturnTypes ? stringType : optionalString}})}); + attachMagicFunction(matchFunc, magicFunctionMatch); + + const TypeId findFunc = arena->addType(FunctionTypeVar{arena->addTypePack({stringType, stringType, optionalNumber, optionalBoolean}), + arena->addTypePack(TypePack{{optionalNumber, optionalNumber}, stringVariadicList})}); + attachMagicFunction(findFunc, magicFunctionFind); + TableTypeVar::Props stringLib = { {"byte", {arena->addType(FunctionTypeVar{arena->addTypePack({stringType, optionalNumber, optionalNumber}), numberVariadicList})}}, {"char", {arena->addType(FunctionTypeVar{numberVariadicList, arena->addTypePack({stringType})})}}, - {"find", {arena->addType(FunctionTypeVar{arena->addTypePack({stringType, stringType, optionalNumber, optionalBoolean}), - arena->addTypePack(TypePack{{optionalNumber, optionalNumber}, stringVariadicList})})}}, + {"find", {findFunc}}, {"format", {formatFn}}, // FIXME {"gmatch", {gmatchFunc}}, {"gsub", {gsubFunc}}, {"len", {makeFunction(*arena, stringType, {}, {}, {}, {}, {numberType})}}, {"lower", {stringToStringType}}, - {"match", {arena->addType(FunctionTypeVar{arena->addTypePack({stringType, stringType, optionalNumber}), - arena->addTypePack(TypePackVar{VariadicTypePack{optionalString}})})}}, + {"match", {matchFunc}}, {"rep", {makeFunction(*arena, stringType, {}, {}, {numberType}, {}, {stringType})}}, {"reverse", {stringToStringType}}, {"sub", {makeFunction(*arena, stringType, {}, {}, {numberType, optionalNumber}, {}, {stringType})}}, @@ -1213,6 +1230,102 @@ static std::optional> magicFunctionGmatch( return WithPredicate{arena.addTypePack({iteratorType})}; } +static std::optional> magicFunctionMatch( + TypeChecker& typechecker, const ScopePtr& scope, const AstExprCall& expr, WithPredicate withPredicate) +{ + if (!FFlag::LuauDeduceFindMatchReturnTypes) + return std::nullopt; + + auto [paramPack, _predicates] = withPredicate; + const auto& [params, tail] = flatten(paramPack); + + if (params.size() < 2 || params.size() > 3) + return std::nullopt; + + TypeArena& arena = typechecker.currentModule->internalTypes; + + AstExprConstantString* pattern = nullptr; + size_t patternIndex = expr.self ? 0 : 1; + if (expr.args.size > patternIndex) + pattern = expr.args.data[patternIndex]->as(); + + if (!pattern) + return std::nullopt; + + std::vector returnTypes = parsePatternString(typechecker, pattern->value.data, pattern->value.size); + + if (returnTypes.empty()) + return std::nullopt; + + typechecker.unify(params[0], typechecker.stringType, expr.args.data[0]->location); + + const TypeId optionalNumber = arena.addType(UnionTypeVar{{typechecker.nilType, typechecker.numberType}}); + + size_t initIndex = expr.self ? 1 : 2; + if (params.size() == 3 && expr.args.size > initIndex) + typechecker.unify(params[2], optionalNumber, expr.args.data[initIndex]->location); + + const TypePackId returnList = arena.addTypePack(returnTypes); + return WithPredicate{returnList}; +} + +static std::optional> magicFunctionFind( + TypeChecker& typechecker, const ScopePtr& scope, const AstExprCall& expr, WithPredicate withPredicate) +{ + if (!FFlag::LuauDeduceFindMatchReturnTypes) + return std::nullopt; + + auto [paramPack, _predicates] = withPredicate; + const auto& [params, tail] = flatten(paramPack); + + if (params.size() < 2 || params.size() > 4) + return std::nullopt; + + TypeArena& arena = typechecker.currentModule->internalTypes; + + AstExprConstantString* pattern = nullptr; + size_t patternIndex = expr.self ? 0 : 1; + if (expr.args.size > patternIndex) + pattern = expr.args.data[patternIndex]->as(); + + if (!pattern) + return std::nullopt; + + bool plain = false; + size_t plainIndex = expr.self ? 2 : 3; + if (expr.args.size > plainIndex) + { + AstExprConstantBool* p = expr.args.data[plainIndex]->as(); + plain = p && p->value; + } + + std::vector returnTypes; + if (!plain) + { + returnTypes = parsePatternString(typechecker, pattern->value.data, pattern->value.size); + + if (returnTypes.empty()) + return std::nullopt; + } + + typechecker.unify(params[0], typechecker.stringType, expr.args.data[0]->location); + + const TypeId optionalNumber = arena.addType(UnionTypeVar{{typechecker.nilType, typechecker.numberType}}); + const TypeId optionalBoolean = arena.addType(UnionTypeVar{{typechecker.nilType, typechecker.booleanType}}); + + size_t initIndex = expr.self ? 1 : 2; + if (params.size() >= 3 && expr.args.size > initIndex) + typechecker.unify(params[2], optionalNumber, expr.args.data[initIndex]->location); + + if (params.size() == 4 && expr.args.size > plainIndex) + typechecker.unify(params[3], optionalBoolean, expr.args.data[plainIndex]->location); + + returnTypes.insert(returnTypes.begin(), {optionalNumber, optionalNumber}); + + const TypePackId returnList = arena.addTypePack(returnTypes); + return WithPredicate{returnList}; +} + std::vector filterMap(TypeId type, TypeIdPredicate predicate) { type = follow(type); diff --git a/Analysis/src/Unifier.cpp b/Analysis/src/Unifier.cpp index 44a3b854..e099817f 100644 --- a/Analysis/src/Unifier.cpp +++ b/Analysis/src/Unifier.cpp @@ -21,6 +21,7 @@ LUAU_FASTFLAG(LuauLowerBoundsCalculation); LUAU_FASTFLAG(LuauErrorRecoveryType); LUAU_FASTFLAG(LuauUnknownAndNeverType) LUAU_FASTFLAG(LuauQuantifyConstrained) +LUAU_FASTFLAGVARIABLE(LuauScalarShapeSubtyping, false) namespace Luau { @@ -432,7 +433,7 @@ void Unifier::tryUnify_(TypeId subTy, TypeId superTy, bool isFunctionCall, bool { // Normally, if the subtype is free, it should not be bound to any, unknown, or error types. // But for bug compatibility, we'll only apply this rule to unknown. Doing this will silence cascading type errors. - if (get(superTy)) + if (log.get(superTy)) return; } @@ -473,10 +474,10 @@ void Unifier::tryUnify_(TypeId subTy, TypeId superTy, bool isFunctionCall, bool return tryUnifyWithAny(superTy, subTy); } - if (get(subTy)) + if (log.get(subTy)) return tryUnifyWithAny(superTy, subTy); - if (get(subTy)) + if (log.get(subTy)) return tryUnifyWithAny(superTy, subTy); auto& cache = sharedState.cachedUnify; @@ -538,6 +539,16 @@ void Unifier::tryUnify_(TypeId subTy, TypeId superTy, bool isFunctionCall, bool { tryUnifyTables(subTy, superTy, isIntersection); } + else if (FFlag::LuauScalarShapeSubtyping && log.get(superTy) && + (log.get(subTy) || log.get(subTy))) + { + tryUnifyScalarShape(subTy, superTy, /*reversed*/ false); + } + else if (FFlag::LuauScalarShapeSubtyping && log.get(subTy) && + (log.get(superTy) || log.get(superTy))) + { + tryUnifyScalarShape(subTy, superTy, /*reversed*/ true); + } // tryUnifyWithMetatable assumes its first argument is a MetatableTypeVar. The check is otherwise symmetrical. else if (log.getMutable(superTy)) @@ -1600,6 +1611,60 @@ void Unifier::tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) } } +void Unifier::tryUnifyScalarShape(TypeId subTy, TypeId superTy, bool reversed) +{ + LUAU_ASSERT(FFlag::LuauScalarShapeSubtyping); + + TypeId osubTy = subTy; + TypeId osuperTy = superTy; + + if (reversed) + std::swap(subTy, superTy); + + if (auto ttv = log.get(superTy); !ttv || ttv->state != TableState::Free) + return reportError(TypeError{location, TypeMismatch{osuperTy, osubTy}}); + + auto fail = [&](std::optional e) { + std::string reason = "The former's metatable does not satisfy the requirements."; + if (e) + reportError(TypeError{location, TypeMismatch{osuperTy, osubTy, reason, *e}}); + else + reportError(TypeError{location, TypeMismatch{osuperTy, osubTy, reason}}); + }; + + // Given t1 where t1 = { lower: (t1) -> (a, b...) } + // It should be the case that `string <: t1` iff `(subtype's metatable).__index <: t1` + if (auto metatable = getMetatable(subTy)) + { + auto mttv = log.get(*metatable); + if (!mttv) + fail(std::nullopt); + + if (auto it = mttv->props.find("__index"); it != mttv->props.end()) + { + TypeId ty = it->second.type; + Unifier child = makeChildUnifier(); + child.tryUnify_(ty, superTy); + + if (auto e = hasUnificationTooComplex(child.errors)) + reportError(*e); + else if (!child.errors.empty()) + fail(child.errors.front()); + + log.concat(std::move(child.log)); + + return; + } + else + { + return fail(std::nullopt); + } + } + + reportError(TypeError{location, TypeMismatch{osuperTy, osubTy}}); + return; +} + TypeId Unifier::deeplyOptional(TypeId ty, std::unordered_map seen) { ty = follow(ty); @@ -1916,7 +1981,8 @@ void Unifier::tryUnifyWithAny(TypeId subTy, TypeId anyTy) sharedState.tempSeenTy.clear(); sharedState.tempSeenTp.clear(); - Luau::tryUnifyWithAny(queue, *this, sharedState.tempSeenTy, sharedState.tempSeenTp, types, FFlag::LuauUnknownAndNeverType ? anyTy : getSingletonTypes().anyType, anyTp); + Luau::tryUnifyWithAny(queue, *this, sharedState.tempSeenTy, sharedState.tempSeenTp, types, + FFlag::LuauUnknownAndNeverType ? anyTy : getSingletonTypes().anyType, anyTp); } void Unifier::tryUnifyWithAny(TypePackId subTy, TypePackId anyTp) diff --git a/Ast/src/Parser.cpp b/Ast/src/Parser.cpp index b7fa7889..779bd279 100644 --- a/Ast/src/Parser.cpp +++ b/Ast/src/Parser.cpp @@ -24,6 +24,8 @@ bool lua_telemetry_parsed_named_non_function_type = false; LUAU_FASTFLAGVARIABLE(LuauErrorParseIntegerIssues, false) LUAU_DYNAMIC_FASTFLAGVARIABLE(LuaReportParseIntegerIssues, false) +LUAU_FASTFLAGVARIABLE(LuauAlwaysCaptureHotComments, false) + bool lua_telemetry_parsed_out_of_range_bin_integer = false; bool lua_telemetry_parsed_out_of_range_hex_integer = false; bool lua_telemetry_parsed_double_prefix_hex_integer = false; @@ -2918,21 +2920,23 @@ AstTypeError* Parser::reportTypeAnnotationError(const Location& location, const void Parser::nextLexeme() { - if (options.captureComments) + if (options.captureComments || FFlag::LuauAlwaysCaptureHotComments) { Lexeme::Type type = lexer.next(/* skipComments= */ false, true).type; while (type == Lexeme::BrokenComment || type == Lexeme::Comment || type == Lexeme::BlockComment) { const Lexeme& lexeme = lexer.current(); - commentLocations.push_back(Comment{lexeme.type, lexeme.location}); + + if (options.captureComments) + commentLocations.push_back(Comment{lexeme.type, lexeme.location}); // Subtlety: Broken comments are weird because we record them as comments AND pass them to the parser as a lexeme. // The parser will turn this into a proper syntax error. if (lexeme.type == Lexeme::BrokenComment) return; - // Comments starting with ! are called "hot comments" and contain directives for type checking / linting + // Comments starting with ! are called "hot comments" and contain directives for type checking / linting / compiling if (lexeme.type == Lexeme::Comment && lexeme.length && lexeme.data[0] == '!') { const char* text = lexeme.data; diff --git a/CMakeLists.txt b/CMakeLists.txt index e256e234..92006344 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -175,6 +175,7 @@ endif() if(LUAU_BUILD_TESTS) target_compile_options(Luau.UnitTest PRIVATE ${LUAU_OPTIONS}) + target_compile_definitions(Luau.UnitTest PRIVATE DOCTEST_CONFIG_DOUBLE_STRINGIFY) target_include_directories(Luau.UnitTest PRIVATE extern) target_link_libraries(Luau.UnitTest PRIVATE Luau.Analysis Luau.Compiler Luau.CodeGen) diff --git a/Compiler/include/Luau/Compiler.h b/Compiler/include/Luau/Compiler.h index 65e962da..eec70d7a 100644 --- a/Compiler/include/Luau/Compiler.h +++ b/Compiler/include/Luau/Compiler.h @@ -8,8 +8,8 @@ namespace Luau { -class AstStatBlock; class AstNameTable; +struct ParseResult; class BytecodeBuilder; class BytecodeEncoder; @@ -58,7 +58,7 @@ private: }; // compiles bytecode into bytecode builder using either a pre-parsed AST or parsing it from source; throws on errors -void compileOrThrow(BytecodeBuilder& bytecode, AstStatBlock* root, const AstNameTable& names, const CompileOptions& options = {}); +void compileOrThrow(BytecodeBuilder& bytecode, const ParseResult& parseResult, const AstNameTable& names, const CompileOptions& options = {}); void compileOrThrow(BytecodeBuilder& bytecode, const std::string& source, const CompileOptions& options = {}, const ParseOptions& parseOptions = {}); // compiles bytecode into a bytecode blob, that either contains the valid bytecode or an encoded error that luau_load can decode diff --git a/Compiler/src/BuiltinFolding.cpp b/Compiler/src/BuiltinFolding.cpp new file mode 100644 index 00000000..e76da4e2 --- /dev/null +++ b/Compiler/src/BuiltinFolding.cpp @@ -0,0 +1,463 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +#include "BuiltinFolding.h" + +#include "Luau/Bytecode.h" + +#include + +namespace Luau +{ +namespace Compile +{ + +const double kRadDeg = 3.14159265358979323846 / 180.0; + +static Constant cvar() +{ + return Constant(); +} + +static Constant cbool(bool v) +{ + Constant res = {Constant::Type_Boolean}; + res.valueBoolean = v; + return res; +} + +static Constant cnum(double v) +{ + Constant res = {Constant::Type_Number}; + res.valueNumber = v; + return res; +} + +static Constant cstring(const char* v) +{ + Constant res = {Constant::Type_String}; + res.stringLength = unsigned(strlen(v)); + res.valueString = v; + return res; +} + +static Constant ctype(const Constant& c) +{ + LUAU_ASSERT(c.type != Constant::Type_Unknown); + + switch (c.type) + { + case Constant::Type_Nil: + return cstring("nil"); + + case Constant::Type_Boolean: + return cstring("boolean"); + + case Constant::Type_Number: + return cstring("number"); + + case Constant::Type_String: + return cstring("string"); + + default: + LUAU_ASSERT(!"Unsupported constant type"); + return cvar(); + } +} + +static uint32_t bit32(double v) +{ + // convert through signed 64-bit integer to match runtime behavior and gracefully truncate negative integers + return uint32_t(int64_t(v)); +} + +Constant foldBuiltin(int bfid, const Constant* args, size_t count) +{ + switch (bfid) + { + case LBF_MATH_ABS: + if (count == 1 && args[0].type == Constant::Type_Number) + return cnum(fabs(args[0].valueNumber)); + break; + + case LBF_MATH_ACOS: + if (count == 1 && args[0].type == Constant::Type_Number) + return cnum(acos(args[0].valueNumber)); + break; + + case LBF_MATH_ASIN: + if (count == 1 && args[0].type == Constant::Type_Number) + return cnum(asin(args[0].valueNumber)); + break; + + case LBF_MATH_ATAN2: + if (count == 2 && args[0].type == Constant::Type_Number && args[1].type == Constant::Type_Number) + return cnum(atan2(args[0].valueNumber, args[1].valueNumber)); + break; + + case LBF_MATH_ATAN: + if (count == 1 && args[0].type == Constant::Type_Number) + return cnum(atan(args[0].valueNumber)); + break; + + case LBF_MATH_CEIL: + if (count == 1 && args[0].type == Constant::Type_Number) + return cnum(ceil(args[0].valueNumber)); + break; + + case LBF_MATH_COSH: + if (count == 1 && args[0].type == Constant::Type_Number) + return cnum(cosh(args[0].valueNumber)); + break; + + case LBF_MATH_COS: + if (count == 1 && args[0].type == Constant::Type_Number) + return cnum(cos(args[0].valueNumber)); + break; + + case LBF_MATH_DEG: + if (count == 1 && args[0].type == Constant::Type_Number) + return cnum(args[0].valueNumber / kRadDeg); + break; + + case LBF_MATH_EXP: + if (count == 1 && args[0].type == Constant::Type_Number) + return cnum(exp(args[0].valueNumber)); + break; + + case LBF_MATH_FLOOR: + if (count == 1 && args[0].type == Constant::Type_Number) + return cnum(floor(args[0].valueNumber)); + break; + + case LBF_MATH_FMOD: + if (count == 2 && args[0].type == Constant::Type_Number && args[1].type == Constant::Type_Number) + return cnum(fmod(args[0].valueNumber, args[1].valueNumber)); + break; + + // Note: FREXP isn't folded since it returns multiple values + + case LBF_MATH_LDEXP: + if (count == 2 && args[0].type == Constant::Type_Number && args[1].type == Constant::Type_Number) + return cnum(ldexp(args[0].valueNumber, int(args[1].valueNumber))); + break; + + case LBF_MATH_LOG10: + if (count == 1 && args[0].type == Constant::Type_Number) + return cnum(log10(args[0].valueNumber)); + break; + + case LBF_MATH_LOG: + if (count == 1 && args[0].type == Constant::Type_Number) + return cnum(log(args[0].valueNumber)); + else if (count == 2 && args[0].type == Constant::Type_Number && args[1].type == Constant::Type_Number) + { + if (args[1].valueNumber == 2.0) + return cnum(log2(args[0].valueNumber)); + else if (args[1].valueNumber == 10.0) + return cnum(log10(args[0].valueNumber)); + else + return cnum(log(args[0].valueNumber) / log(args[1].valueNumber)); + } + break; + + case LBF_MATH_MAX: + if (count >= 1 && args[0].type == Constant::Type_Number) + { + double r = args[0].valueNumber; + + for (size_t i = 1; i < count; ++i) + { + if (args[i].type != Constant::Type_Number) + return cvar(); + + double a = args[i].valueNumber; + + r = (a > r) ? a : r; + } + + return cnum(r); + } + break; + + case LBF_MATH_MIN: + if (count >= 1 && args[0].type == Constant::Type_Number) + { + double r = args[0].valueNumber; + + for (size_t i = 1; i < count; ++i) + { + if (args[i].type != Constant::Type_Number) + return cvar(); + + double a = args[i].valueNumber; + + r = (a < r) ? a : r; + } + + return cnum(r); + } + break; + + // Note: MODF isn't folded since it returns multiple values + + case LBF_MATH_POW: + if (count == 2 && args[0].type == Constant::Type_Number && args[1].type == Constant::Type_Number) + return cnum(pow(args[0].valueNumber, args[1].valueNumber)); + break; + + case LBF_MATH_RAD: + if (count == 1 && args[0].type == Constant::Type_Number) + return cnum(args[0].valueNumber * kRadDeg); + break; + + case LBF_MATH_SINH: + if (count == 1 && args[0].type == Constant::Type_Number) + return cnum(sinh(args[0].valueNumber)); + break; + + case LBF_MATH_SIN: + if (count == 1 && args[0].type == Constant::Type_Number) + return cnum(sin(args[0].valueNumber)); + break; + + case LBF_MATH_SQRT: + if (count == 1 && args[0].type == Constant::Type_Number) + return cnum(sqrt(args[0].valueNumber)); + break; + + case LBF_MATH_TANH: + if (count == 1 && args[0].type == Constant::Type_Number) + return cnum(tanh(args[0].valueNumber)); + break; + + case LBF_MATH_TAN: + if (count == 1 && args[0].type == Constant::Type_Number) + return cnum(tan(args[0].valueNumber)); + break; + + case LBF_BIT32_ARSHIFT: + if (count == 2 && args[0].type == Constant::Type_Number && args[1].type == Constant::Type_Number) + { + uint32_t u = bit32(args[0].valueNumber); + int s = int(args[1].valueNumber); + + if (unsigned(s) < 32) + return cnum(double(uint32_t(int32_t(u) >> s))); + } + break; + + case LBF_BIT32_BAND: + if (count >= 1 && args[0].type == Constant::Type_Number) + { + uint32_t r = bit32(args[0].valueNumber); + + for (size_t i = 1; i < count; ++i) + { + if (args[i].type != Constant::Type_Number) + return cvar(); + + r &= bit32(args[i].valueNumber); + } + + return cnum(double(r)); + } + break; + + case LBF_BIT32_BNOT: + if (count == 1 && args[0].type == Constant::Type_Number) + return cnum(double(uint32_t(~bit32(args[0].valueNumber)))); + break; + + case LBF_BIT32_BOR: + if (count >= 1 && args[0].type == Constant::Type_Number) + { + uint32_t r = bit32(args[0].valueNumber); + + for (size_t i = 1; i < count; ++i) + { + if (args[i].type != Constant::Type_Number) + return cvar(); + + r |= bit32(args[i].valueNumber); + } + + return cnum(double(r)); + } + break; + + case LBF_BIT32_BXOR: + if (count >= 1 && args[0].type == Constant::Type_Number) + { + uint32_t r = bit32(args[0].valueNumber); + + for (size_t i = 1; i < count; ++i) + { + if (args[i].type != Constant::Type_Number) + return cvar(); + + r ^= bit32(args[i].valueNumber); + } + + return cnum(double(r)); + } + break; + + case LBF_BIT32_BTEST: + if (count >= 1 && args[0].type == Constant::Type_Number) + { + uint32_t r = bit32(args[0].valueNumber); + + for (size_t i = 1; i < count; ++i) + { + if (args[i].type != Constant::Type_Number) + return cvar(); + + r &= bit32(args[i].valueNumber); + } + + return cbool(r != 0); + } + break; + + case LBF_BIT32_EXTRACT: + if (count == 3 && args[0].type == Constant::Type_Number && args[1].type == Constant::Type_Number && args[2].type == Constant::Type_Number) + { + uint32_t u = bit32(args[0].valueNumber); + int f = int(args[1].valueNumber); + int w = int(args[2].valueNumber); + + if (f >= 0 && w > 0 && f + w <= 32) + { + uint32_t m = ~(0xfffffffeu << (w - 1)); + + return cnum(double((u >> f) & m)); + } + } + break; + + case LBF_BIT32_LROTATE: + if (count == 2 && args[0].type == Constant::Type_Number && args[1].type == Constant::Type_Number) + { + uint32_t u = bit32(args[0].valueNumber); + int s = int(args[1].valueNumber); + + return cnum(double((u << (s & 31)) | (u >> ((32 - s) & 31)))); + } + break; + + case LBF_BIT32_LSHIFT: + if (count == 2 && args[0].type == Constant::Type_Number && args[1].type == Constant::Type_Number) + { + uint32_t u = bit32(args[0].valueNumber); + int s = int(args[1].valueNumber); + + if (unsigned(s) < 32) + return cnum(double(u << s)); + } + break; + + case LBF_BIT32_REPLACE: + if (count == 4 && args[0].type == Constant::Type_Number && args[1].type == Constant::Type_Number && args[2].type == Constant::Type_Number && + args[3].type == Constant::Type_Number) + { + uint32_t n = bit32(args[0].valueNumber); + uint32_t v = bit32(args[1].valueNumber); + int f = int(args[2].valueNumber); + int w = int(args[3].valueNumber); + + if (f >= 0 && w > 0 && f + w <= 32) + { + uint32_t m = ~(0xfffffffeu << (w - 1)); + + return cnum(double((n & ~(m << f)) | ((v & m) << f))); + } + } + break; + + case LBF_BIT32_RROTATE: + if (count == 2 && args[0].type == Constant::Type_Number && args[1].type == Constant::Type_Number) + { + uint32_t u = bit32(args[0].valueNumber); + int s = int(args[1].valueNumber); + + return cnum(double((u >> (s & 31)) | (u << ((32 - s) & 31)))); + } + break; + + case LBF_BIT32_RSHIFT: + if (count == 2 && args[0].type == Constant::Type_Number && args[1].type == Constant::Type_Number) + { + uint32_t u = bit32(args[0].valueNumber); + int s = int(args[1].valueNumber); + + if (unsigned(s) < 32) + return cnum(double(u >> s)); + } + break; + + case LBF_TYPE: + if (count == 1 && args[0].type != Constant::Type_Unknown) + return ctype(args[0]); + break; + + case LBF_STRING_BYTE: + if (count == 1 && args[0].type == Constant::Type_String) + { + if (args[0].stringLength > 0) + return cnum(double(uint8_t(args[0].valueString[0]))); + } + else if (count == 2 && args[0].type == Constant::Type_String && args[1].type == Constant::Type_Number) + { + int i = int(args[1].valueNumber); + + if (i > 0 && unsigned(i) <= args[0].stringLength) + return cnum(double(uint8_t(args[0].valueString[i - 1]))); + } + break; + + case LBF_STRING_LEN: + if (count == 1 && args[0].type == Constant::Type_String) + return cnum(double(args[0].stringLength)); + break; + + case LBF_TYPEOF: + if (count == 1 && args[0].type != Constant::Type_Unknown) + return ctype(args[0]); + break; + + case LBF_MATH_CLAMP: + if (count == 3 && args[0].type == Constant::Type_Number && args[1].type == Constant::Type_Number && args[2].type == Constant::Type_Number) + { + double min = args[1].valueNumber; + double max = args[2].valueNumber; + + if (min <= max) + { + double v = args[0].valueNumber; + v = v < min ? min : v; + v = v > max ? max : v; + + return cnum(v); + } + } + break; + + case LBF_MATH_SIGN: + if (count == 1 && args[0].type == Constant::Type_Number) + { + double v = args[0].valueNumber; + + return cnum(v > 0.0 ? 1.0 : v < 0.0 ? -1.0 : 0.0); + } + break; + + case LBF_MATH_ROUND: + if (count == 1 && args[0].type == Constant::Type_Number) + return cnum(round(args[0].valueNumber)); + break; + } + + return cvar(); +} + +} // namespace Compile +} // namespace Luau diff --git a/Compiler/src/BuiltinFolding.h b/Compiler/src/BuiltinFolding.h new file mode 100644 index 00000000..1904e14f --- /dev/null +++ b/Compiler/src/BuiltinFolding.h @@ -0,0 +1,14 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +#pragma once + +#include "ConstantFolding.h" + +namespace Luau +{ +namespace Compile +{ + +Constant foldBuiltin(int bfid, const Constant* args, size_t count); + +} // namespace Compile +} // namespace Luau diff --git a/Compiler/src/Builtins.cpp b/Compiler/src/Builtins.cpp index 6bd24b6d..3650c147 100644 --- a/Compiler/src/Builtins.cpp +++ b/Compiler/src/Builtins.cpp @@ -40,11 +40,8 @@ Builtin getBuiltin(AstExpr* node, const DenseHashMap& globals, } } -int getBuiltinFunctionId(const Builtin& builtin, const CompileOptions& options) +static int getBuiltinFunctionId(const Builtin& builtin, const CompileOptions& options) { - if (builtin.empty()) - return -1; - if (builtin.isGlobal("assert")) return LBF_ASSERT; @@ -200,5 +197,49 @@ int getBuiltinFunctionId(const Builtin& builtin, const CompileOptions& options) return -1; } +struct BuiltinVisitor : AstVisitor +{ + DenseHashMap& result; + + const DenseHashMap& globals; + const DenseHashMap& variables; + + const CompileOptions& options; + + BuiltinVisitor(DenseHashMap& result, const DenseHashMap& globals, + const DenseHashMap& variables, const CompileOptions& options) + : result(result) + , globals(globals) + , variables(variables) + , options(options) + { + } + + bool visit(AstExprCall* node) override + { + Builtin builtin = node->self ? Builtin() : getBuiltin(node->func, globals, variables); + if (builtin.empty()) + return true; + + int bfid = getBuiltinFunctionId(builtin, options); + + // getBuiltinFunctionId optimistically assumes all select() calls are builtin but actually the second argument must be a vararg + if (bfid == LBF_SELECT_VARARG && !(node->args.size == 2 && node->args.data[1]->is())) + bfid = -1; + + if (bfid >= 0) + result[node] = bfid; + + return true; // propagate to nested calls + } +}; + +void analyzeBuiltins(DenseHashMap& result, const DenseHashMap& globals, + const DenseHashMap& variables, const CompileOptions& options, AstNode* root) +{ + BuiltinVisitor visitor{result, globals, variables, options}; + root->visit(&visitor); +} + } // namespace Compile } // namespace Luau diff --git a/Compiler/src/Builtins.h b/Compiler/src/Builtins.h index 60df53a1..4399c532 100644 --- a/Compiler/src/Builtins.h +++ b/Compiler/src/Builtins.h @@ -35,7 +35,9 @@ struct Builtin }; Builtin getBuiltin(AstExpr* node, const DenseHashMap& globals, const DenseHashMap& variables); -int getBuiltinFunctionId(const Builtin& builtin, const CompileOptions& options); + +void analyzeBuiltins(DenseHashMap& result, const DenseHashMap& globals, + const DenseHashMap& variables, const CompileOptions& options, AstNode* root); } // namespace Compile } // namespace Luau diff --git a/Compiler/src/BytecodeBuilder.cpp b/Compiler/src/BytecodeBuilder.cpp index 5e2669b2..64327bd0 100644 --- a/Compiler/src/BytecodeBuilder.cpp +++ b/Compiler/src/BytecodeBuilder.cpp @@ -120,6 +120,17 @@ inline bool isSkipC(LuauOpcode op) switch (op) { case LOP_LOADB: + return true; + + default: + return false; + } +} + +inline bool isFastCall(LuauOpcode op) +{ + switch (op) + { case LOP_FASTCALL: case LOP_FASTCALL1: case LOP_FASTCALL2: @@ -137,6 +148,8 @@ static int getJumpTarget(uint32_t insn, uint32_t pc) if (isJumpD(op)) return int(pc + LUAU_INSN_D(insn) + 1); + else if (isFastCall(op)) + return int(pc + LUAU_INSN_C(insn) + 2); else if (isSkipC(op) && LUAU_INSN_C(insn)) return int(pc + LUAU_INSN_C(insn) + 1); else if (op == LOP_JUMPX) @@ -479,7 +492,7 @@ bool BytecodeBuilder::patchSkipC(size_t jumpLabel, size_t targetLabel) unsigned int jumpInsn = insns[jumpLabel]; (void)jumpInsn; - LUAU_ASSERT(isSkipC(LuauOpcode(LUAU_INSN_OP(jumpInsn)))); + LUAU_ASSERT(isSkipC(LuauOpcode(LUAU_INSN_OP(jumpInsn))) || isFastCall(LuauOpcode(LUAU_INSN_OP(jumpInsn)))); LUAU_ASSERT(LUAU_INSN_C(jumpInsn) == 0); int offset = int(targetLabel) - int(jumpLabel) - 1; diff --git a/Compiler/src/Compiler.cpp b/Compiler/src/Compiler.cpp index d7c8155c..2b0f04f6 100644 --- a/Compiler/src/Compiler.cpp +++ b/Compiler/src/Compiler.cpp @@ -25,6 +25,9 @@ LUAU_FASTINTVARIABLE(LuauCompileInlineDepth, 5) LUAU_FASTFLAGVARIABLE(LuauCompileNoIpairs, false) +LUAU_FASTFLAGVARIABLE(LuauCompileFoldBuiltins, false) +LUAU_FASTFLAGVARIABLE(LuauCompileBetterMultret, false) + namespace Luau { @@ -75,6 +78,12 @@ static BytecodeBuilder::StringRef sref(AstArray data) return {data.data, data.size}; } +static BytecodeBuilder::StringRef sref(AstArray data) +{ + LUAU_ASSERT(data.data); + return {data.data, data.size}; +} + struct Compiler { struct RegScope; @@ -89,6 +98,7 @@ struct Compiler , constants(nullptr) , locstants(nullptr) , tableShapes(nullptr) + , builtins(nullptr) { // preallocate some buffers that are very likely to grow anyway; this works around std::vector's inefficient growth policy for small arrays localStack.reserve(16); @@ -245,7 +255,7 @@ struct Compiler { f.canInline = true; f.stackSize = stackSize; - f.costModel = modelCost(func->body, func->args.data, func->args.size); + f.costModel = modelCost(func->body, func->args.data, func->args.size, builtins); // track functions that only ever return a single value so that we can convert multret calls to fixedret calls if (allPathsEndWithReturn(func->body)) @@ -262,22 +272,63 @@ struct Compiler return fid; } + // returns true if node can return multiple values; may conservatively return true even if expr is known to return just a single value + bool isExprMultRet(AstExpr* node) + { + if (!FFlag::LuauCompileBetterMultret) + return node->is() || node->is(); + + AstExprCall* expr = node->as(); + if (!expr) + return node->is(); + + // conservative version, optimized for compilation throughput + if (options.optimizationLevel <= 1) + return true; + + // handles builtin calls that can be constant-folded + // without this we may omit some optimizations eg compiling fast calls without use of FASTCALL2K + if (isConstant(expr)) + return false; + + // handles local function calls where we know only one argument is returned + AstExprFunction* func = getFunctionExpr(expr->func); + Function* fi = func ? functions.find(func) : nullptr; + + if (fi && fi->returnsOne) + return false; + + // unrecognized call, so we conservatively assume multret + return true; + } + // note: this doesn't just clobber target (assuming it's temp), but also clobbers *all* allocated registers >= target! // this is important to be able to support "multret" semantics due to Lua call frame structure bool compileExprTempMultRet(AstExpr* node, uint8_t target) { if (AstExprCall* expr = node->as()) { - // Optimization: convert multret calls to functions that always return one value to fixedret calls; this facilitates inlining + // Optimization: convert multret calls that always return one value to fixedret calls; this facilitates inlining/constant folding if (options.optimizationLevel >= 2) { - AstExprFunction* func = getFunctionExpr(expr->func); - Function* fi = func ? functions.find(func) : nullptr; - - if (fi && fi->returnsOne) + if (FFlag::LuauCompileBetterMultret) { - compileExprTemp(node, target); - return false; + if (!isExprMultRet(node)) + { + compileExprTemp(node, target); + return false; + } + } + else + { + AstExprFunction* func = getFunctionExpr(expr->func); + Function* fi = func ? functions.find(func) : nullptr; + + if (fi && fi->returnsOne) + { + compileExprTemp(node, target); + return false; + } } } @@ -483,8 +534,7 @@ struct Compiler varc[i] = isConstant(expr->args.data[i]); // if the last argument only returns a single value, all following arguments are nil - if (expr->args.size != 0 && - !(expr->args.data[expr->args.size - 1]->is() || expr->args.data[expr->args.size - 1]->is())) + if (expr->args.size != 0 && !isExprMultRet(expr->args.data[expr->args.size - 1])) for (size_t i = expr->args.size; i < func->args.size && i < 8; ++i) varc[i] = true; @@ -523,7 +573,7 @@ struct Compiler AstLocal* var = func->args.data[i]; AstExpr* arg = i < expr->args.size ? expr->args.data[i] : nullptr; - if (i + 1 == expr->args.size && func->args.size > expr->args.size && (arg->is() || arg->is())) + if (i + 1 == expr->args.size && func->args.size > expr->args.size && isExprMultRet(arg)) { // if the last argument can return multiple values, we need to compute all of them into the remaining arguments unsigned int tail = unsigned(func->args.size - expr->args.size) + 1; @@ -591,7 +641,7 @@ struct Compiler } // fold constant values updated above into expressions in the function body - foldConstants(constants, variables, locstants, func->body); + foldConstants(constants, variables, locstants, builtinsFold, func->body); bool usedFallthrough = false; @@ -632,7 +682,7 @@ struct Compiler if (Constant* var = locstants.find(func->args.data[i])) var->type = Constant::Type_Unknown; - foldConstants(constants, variables, locstants, func->body); + foldConstants(constants, variables, locstants, builtinsFold, func->body); } void compileExprCall(AstExprCall* expr, uint8_t target, uint8_t targetCount, bool targetTop = false, bool multRet = false) @@ -675,29 +725,23 @@ struct Compiler int bfid = -1; - if (options.optimizationLevel >= 1) - { - Builtin builtin = getBuiltin(expr->func, globals, variables); - bfid = getBuiltinFunctionId(builtin, options); - } + if (options.optimizationLevel >= 1 && !expr->self) + if (const int* id = builtins.find(expr)) + bfid = *id; if (bfid == LBF_SELECT_VARARG) { // Optimization: compile select(_, ...) as FASTCALL1; the builtin will read variadic arguments directly // note: for now we restrict this to single-return expressions since our runtime code doesn't deal with general cases - if (multRet == false && targetCount == 1 && expr->args.size == 2 && expr->args.data[1]->is()) + if (multRet == false && targetCount == 1) return compileExprSelectVararg(expr, target, targetCount, targetTop, multRet, regs); else bfid = -1; } // Optimization: for 1/2 argument fast calls use specialized opcodes - if (!expr->self && bfid >= 0 && expr->args.size >= 1 && expr->args.size <= 2) - { - AstExpr* last = expr->args.data[expr->args.size - 1]; - if (!last->is() && !last->is()) - return compileExprFastcallN(expr, target, targetCount, targetTop, multRet, regs, bfid); - } + if (bfid >= 0 && expr->args.size >= 1 && expr->args.size <= 2 && !isExprMultRet(expr->args.data[expr->args.size - 1])) + return compileExprFastcallN(expr, target, targetCount, targetTop, multRet, regs, bfid); if (expr->self) { @@ -2495,7 +2539,7 @@ struct Compiler } AstLocal* var = stat->var; - uint64_t costModel = modelCost(stat->body, &var, 1); + uint64_t costModel = modelCost(stat->body, &var, 1, builtins); // we use a dynamic cost threshold that's based on the fixed limit boosted by the cost advantage we gain due to unrolling bool varc = true; @@ -2533,7 +2577,7 @@ struct Compiler locstants[var].type = Constant::Type_Number; locstants[var].valueNumber = from + iv * step; - foldConstants(constants, variables, locstants, stat); + foldConstants(constants, variables, locstants, builtinsFold, stat); size_t iterJumps = loopJumps.size(); @@ -2561,7 +2605,7 @@ struct Compiler // clean up fold state in case we need to recompile - normally we compile the loop body once, but due to inlining we may need to do it again locstants[var].type = Constant::Type_Unknown; - foldConstants(constants, variables, locstants, stat); + foldConstants(constants, variables, locstants, builtinsFold, stat); } void compileStatFor(AstStatFor* stat) @@ -3368,7 +3412,11 @@ struct Compiler bool visit(AstStatReturn* stat) override { - if (stat->list.size == 1) + if (FFlag::LuauCompileBetterMultret) + { + returnsOne &= stat->list.size == 1 && !self->isExprMultRet(stat->list.data[0]); + } + else if (stat->list.size == 1) { AstExpr* value = stat->list.data[0]; @@ -3487,6 +3535,8 @@ struct Compiler DenseHashMap constants; DenseHashMap locstants; DenseHashMap tableShapes; + DenseHashMap builtins; + const DenseHashMap* builtinsFold = nullptr; unsigned int regTop = 0; unsigned int stackSize = 0; @@ -3502,10 +3552,21 @@ struct Compiler std::vector captures; }; -void compileOrThrow(BytecodeBuilder& bytecode, AstStatBlock* root, const AstNameTable& names, const CompileOptions& options) +void compileOrThrow(BytecodeBuilder& bytecode, const ParseResult& parseResult, const AstNameTable& names, const CompileOptions& inputOptions) { LUAU_TIMETRACE_SCOPE("compileOrThrow", "Compiler"); + LUAU_ASSERT(parseResult.root); + LUAU_ASSERT(parseResult.errors.empty()); + + CompileOptions options = inputOptions; + + for (const HotComment& hc : parseResult.hotcomments) + if (hc.header && hc.content.compare(0, 9, "optimize ") == 0) + options.optimizationLevel = std::max(0, std::min(2, atoi(hc.content.c_str() + 9))); + + AstStatBlock* root = parseResult.root; + Compiler compiler(bytecode, options); // since access to some global objects may result in values that change over time, we block imports from non-readonly tables @@ -3514,10 +3575,17 @@ void compileOrThrow(BytecodeBuilder& bytecode, AstStatBlock* root, const AstName // this pass analyzes mutability of locals/globals and associates locals with their initial values trackValues(compiler.globals, compiler.variables, root); + // builtin folding is enabled on optimization level 2 since we can't deoptimize folding at runtime + if (options.optimizationLevel >= 2 && FFlag::LuauCompileFoldBuiltins) + compiler.builtinsFold = &compiler.builtins; + if (options.optimizationLevel >= 1) { + // this pass tracks which calls are builtins and can be compiled more efficiently + analyzeBuiltins(compiler.builtins, compiler.globals, compiler.variables, options, root); + // this pass analyzes constantness of expressions - foldConstants(compiler.constants, compiler.variables, compiler.locstants, root); + foldConstants(compiler.constants, compiler.variables, compiler.locstants, compiler.builtinsFold, root); // this pass analyzes table assignments to estimate table shapes for initially empty tables predictTableShapes(compiler.tableShapes, root); @@ -3559,9 +3627,7 @@ void compileOrThrow(BytecodeBuilder& bytecode, const std::string& source, const if (!result.errors.empty()) throw ParseErrors(result.errors); - AstStatBlock* root = result.root; - - compileOrThrow(bytecode, root, names, options); + compileOrThrow(bytecode, result, names, options); } std::string compile(const std::string& source, const CompileOptions& options, const ParseOptions& parseOptions, BytecodeEncoder* encoder) @@ -3584,7 +3650,7 @@ std::string compile(const std::string& source, const CompileOptions& options, co try { BytecodeBuilder bcb(encoder); - compileOrThrow(bcb, result.root, names, options); + compileOrThrow(bcb, result, names, options); return bcb.getBytecode(); } diff --git a/Compiler/src/ConstantFolding.cpp b/Compiler/src/ConstantFolding.cpp index a62beeb1..34f79544 100644 --- a/Compiler/src/ConstantFolding.cpp +++ b/Compiler/src/ConstantFolding.cpp @@ -1,6 +1,8 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #include "ConstantFolding.h" +#include "BuiltinFolding.h" + #include namespace Luau @@ -193,13 +195,18 @@ struct ConstantVisitor : AstVisitor DenseHashMap& variables; DenseHashMap& locals; + const DenseHashMap* builtins; + bool wasEmpty = false; - ConstantVisitor( - DenseHashMap& constants, DenseHashMap& variables, DenseHashMap& locals) + std::vector builtinArgs; + + ConstantVisitor(DenseHashMap& constants, DenseHashMap& variables, + DenseHashMap& locals, const DenseHashMap* builtins) : constants(constants) , variables(variables) , locals(locals) + , builtins(builtins) { // since we do a single pass over the tree, if the initial state was empty we don't need to clear out old entries wasEmpty = constants.empty() && locals.empty(); @@ -253,8 +260,37 @@ struct ConstantVisitor : AstVisitor { analyze(expr->func); - for (size_t i = 0; i < expr->args.size; ++i) - analyze(expr->args.data[i]); + if (const int* bfid = builtins ? builtins->find(expr) : nullptr) + { + // since recursive calls to analyze() may reuse the vector we need to be careful and preserve existing contents + size_t offset = builtinArgs.size(); + bool canFold = true; + + builtinArgs.reserve(offset + expr->args.size); + + for (size_t i = 0; i < expr->args.size; ++i) + { + Constant ac = analyze(expr->args.data[i]); + + if (ac.type == Constant::Type_Unknown) + canFold = false; + else + builtinArgs.push_back(ac); + } + + if (canFold) + { + LUAU_ASSERT(builtinArgs.size() == offset + expr->args.size); + result = foldBuiltin(*bfid, builtinArgs.data() + offset, expr->args.size); + } + + builtinArgs.resize(offset); + } + else + { + for (size_t i = 0; i < expr->args.size; ++i) + analyze(expr->args.data[i]); + } } else if (AstExprIndexName* expr = node->as()) { @@ -395,9 +431,9 @@ struct ConstantVisitor : AstVisitor }; void foldConstants(DenseHashMap& constants, DenseHashMap& variables, - DenseHashMap& locals, AstNode* root) + DenseHashMap& locals, const DenseHashMap* builtins, AstNode* root) { - ConstantVisitor visitor{constants, variables, locals}; + ConstantVisitor visitor{constants, variables, locals, builtins}; root->visit(&visitor); } diff --git a/Compiler/src/ConstantFolding.h b/Compiler/src/ConstantFolding.h index 0a995d75..d67d9285 100644 --- a/Compiler/src/ConstantFolding.h +++ b/Compiler/src/ConstantFolding.h @@ -26,7 +26,7 @@ struct Constant { bool valueBoolean; double valueNumber; - char* valueString = nullptr; // length stored in stringLength + const char* valueString = nullptr; // length stored in stringLength }; bool isTruthful() const @@ -35,7 +35,7 @@ struct Constant return type != Type_Nil && !(type == Type_Boolean && valueBoolean == false); } - AstArray getString() const + AstArray getString() const { LUAU_ASSERT(type == Type_String); return {valueString, stringLength}; @@ -43,7 +43,7 @@ struct Constant }; void foldConstants(DenseHashMap& constants, DenseHashMap& variables, - DenseHashMap& locals, AstNode* root); + DenseHashMap& locals, const DenseHashMap* builtins, AstNode* root); } // namespace Compile } // namespace Luau diff --git a/Compiler/src/CostModel.cpp b/Compiler/src/CostModel.cpp index 5608cd86..56b6c36f 100644 --- a/Compiler/src/CostModel.cpp +++ b/Compiler/src/CostModel.cpp @@ -6,6 +6,8 @@ #include +LUAU_FASTFLAGVARIABLE(LuauCompileModelBuiltins, false) + namespace Luau { namespace Compile @@ -113,11 +115,14 @@ struct Cost struct CostVisitor : AstVisitor { + const DenseHashMap& builtins; + DenseHashMap vars; Cost result; - CostVisitor() - : vars(nullptr) + CostVisitor(const DenseHashMap& builtins) + : builtins(builtins) + , vars(nullptr) { } @@ -148,14 +153,21 @@ struct CostVisitor : AstVisitor } else if (AstExprCall* expr = node->as()) { - Cost cost = 3; - cost += model(expr->func); + // builtin cost modeling is different from regular calls because we use FASTCALL to compile these + // thus we use a cheaper baseline, don't account for function, and assume constant/local copy is free + bool builtin = FFlag::LuauCompileModelBuiltins && builtins.find(expr) != nullptr; + bool builtinShort = builtin && expr->args.size <= 2; // FASTCALL1/2 + + Cost cost = builtin ? 2 : 3; + + if (!builtin) + cost += model(expr->func); for (size_t i = 0; i < expr->args.size; ++i) { Cost ac = model(expr->args.data[i]); // for constants/locals we still need to copy them to the argument list - cost += ac.model == 0 ? Cost(1) : ac; + cost += ac.model == 0 && !builtinShort ? Cost(1) : ac; } return cost; @@ -327,9 +339,9 @@ struct CostVisitor : AstVisitor } }; -uint64_t modelCost(AstNode* root, AstLocal* const* vars, size_t varCount) +uint64_t modelCost(AstNode* root, AstLocal* const* vars, size_t varCount, const DenseHashMap& builtins) { - CostVisitor visitor; + CostVisitor visitor{builtins}; for (size_t i = 0; i < varCount && i < 7; ++i) visitor.vars[vars[i]] = 0xffull << (i * 8 + 8); diff --git a/Compiler/src/CostModel.h b/Compiler/src/CostModel.h index 17defafb..e8f3e166 100644 --- a/Compiler/src/CostModel.h +++ b/Compiler/src/CostModel.h @@ -2,6 +2,7 @@ #pragma once #include "Luau/Ast.h" +#include "Luau/DenseHash.h" namespace Luau { @@ -9,7 +10,7 @@ namespace Compile { // cost model: 8 bytes, where first byte is the baseline cost, and the next 7 bytes are discounts for when variable #i is constant -uint64_t modelCost(AstNode* root, AstLocal* const* vars, size_t varCount); +uint64_t modelCost(AstNode* root, AstLocal* const* vars, size_t varCount, const DenseHashMap& builtins); // cost is computed as B - sum(Di * Ci), where B is baseline cost, Di is the discount for each variable and Ci is 1 when variable #i is constant int computeCost(uint64_t model, const bool* varsConst, size_t varCount); diff --git a/Makefile b/Makefile index a0a52208..7843746e 100644 --- a/Makefile +++ b/Makefile @@ -50,6 +50,9 @@ TESTS_ARGS= ifneq ($(flags),) TESTS_ARGS+=--fflags=$(flags) endif +ifneq ($(opt),) + TESTS_ARGS+=-O$(opt) +endif OBJECTS=$(AST_OBJECTS) $(COMPILER_OBJECTS) $(ANALYSIS_OBJECTS) $(CODEGEN_OBJECTS) $(VM_OBJECTS) $(ISOCLINE_OBJECTS) $(TESTS_OBJECTS) $(CLI_OBJECTS) $(FUZZ_OBJECTS) @@ -104,7 +107,7 @@ $(ANALYSIS_OBJECTS): CXXFLAGS+=-std=c++17 -ICommon/include -IAst/include -IAnaly $(CODEGEN_OBJECTS): CXXFLAGS+=-std=c++17 -ICommon/include -ICodeGen/include $(VM_OBJECTS): CXXFLAGS+=-std=c++11 -ICommon/include -IVM/include $(ISOCLINE_OBJECTS): CXXFLAGS+=-Wno-unused-function -Iextern/isocline/include -$(TESTS_OBJECTS): CXXFLAGS+=-std=c++17 -ICommon/include -IAst/include -ICompiler/include -IAnalysis/include -ICodeGen/include -IVM/include -ICLI -Iextern +$(TESTS_OBJECTS): CXXFLAGS+=-std=c++17 -ICommon/include -IAst/include -ICompiler/include -IAnalysis/include -ICodeGen/include -IVM/include -ICLI -Iextern -DDOCTEST_CONFIG_DOUBLE_STRINGIFY $(REPL_CLI_OBJECTS): CXXFLAGS+=-std=c++17 -ICommon/include -IAst/include -ICompiler/include -IVM/include -Iextern -Iextern/isocline/include $(ANALYZE_CLI_OBJECTS): CXXFLAGS+=-std=c++17 -ICommon/include -IAst/include -IAnalysis/include -Iextern $(FUZZ_OBJECTS): CXXFLAGS+=-std=c++17 -ICommon/include -IAst/include -ICompiler/include -IAnalysis/include -IVM/include diff --git a/Sources.cmake b/Sources.cmake index 44bed8f7..69f5e1b7 100644 --- a/Sources.cmake +++ b/Sources.cmake @@ -38,12 +38,14 @@ target_sources(Luau.Compiler PRIVATE Compiler/src/BytecodeBuilder.cpp Compiler/src/Compiler.cpp Compiler/src/Builtins.cpp + Compiler/src/BuiltinFolding.cpp Compiler/src/ConstantFolding.cpp Compiler/src/CostModel.cpp Compiler/src/TableShape.cpp Compiler/src/ValueTracking.cpp Compiler/src/lcode.cpp Compiler/src/Builtins.h + Compiler/src/BuiltinFolding.h Compiler/src/ConstantFolding.h Compiler/src/CostModel.h Compiler/src/TableShape.h diff --git a/VM/include/lua.h b/VM/include/lua.h index 7f9647c8..514ea361 100644 --- a/VM/include/lua.h +++ b/VM/include/lua.h @@ -372,7 +372,7 @@ LUA_API const char* lua_getupvalue(lua_State* L, int funcindex, int n); LUA_API const char* lua_setupvalue(lua_State* L, int funcindex, int n); LUA_API void lua_singlestep(lua_State* L, int enabled); -LUA_API void lua_breakpoint(lua_State* L, int funcindex, int line, int enabled); +LUA_API int lua_breakpoint(lua_State* L, int funcindex, int line, int enabled); typedef void (*lua_Coverage)(void* context, const char* function, int linedefined, int depth, const int* hits, size_t size); diff --git a/VM/src/lapi.cpp b/VM/src/lapi.cpp index 3c3b7bd0..0bcb8656 100644 --- a/VM/src/lapi.cpp +++ b/VM/src/lapi.cpp @@ -34,6 +34,8 @@ * therefore call luaC_checkGC before luaC_checkthreadsleep to guarantee the object is pushed to an awake thread. */ +LUAU_FASTFLAG(LuauLazyAtoms) + const char* lua_ident = "$Lua: Lua 5.1.4 Copyright (C) 1994-2008 Lua.org, PUC-Rio $\n" "$Authors: R. Ierusalimschy, L. H. de Figueiredo & W. Celes $\n" "$URL: www.lua.org $\n"; @@ -51,6 +53,13 @@ const char* luau_ident = "$Luau: Copyright (C) 2019-2022 Roblox Corporation $\n" L->top++; \ } +#define updateatom(L, ts) \ + if (FFlag::LuauLazyAtoms) \ + { \ + if (ts->atom == ATOM_UNDEF) \ + ts->atom = L->global->cb.useratom ? L->global->cb.useratom(ts->data, ts->len) : -1; \ + } + static Table* getcurrenv(lua_State* L) { if (L->ci == L->base_ci) /* no enclosing function? */ @@ -441,19 +450,25 @@ const char* lua_tostringatom(lua_State* L, int idx, int* atom) StkId o = index2addr(L, idx); if (!ttisstring(o)) return NULL; - const TString* s = tsvalue(o); + TString* s = tsvalue(o); if (atom) + { + updateatom(L, s); *atom = s->atom; + } return getstr(s); } const char* lua_namecallatom(lua_State* L, int* atom) { - const TString* s = L->namecall; + TString* s = L->namecall; if (!s) return NULL; if (atom) + { + updateatom(L, s); *atom = s->atom; + } return getstr(s); } diff --git a/VM/src/ldebug.cpp b/VM/src/ldebug.cpp index e050050e..ef486090 100644 --- a/VM/src/ldebug.cpp +++ b/VM/src/ldebug.cpp @@ -12,6 +12,8 @@ #include #include +LUAU_FASTFLAGVARIABLE(LuauDebuggerBreakpointHitOnNextBestLine, false); + static const char* getfuncname(Closure* f); static int currentpc(lua_State* L, CallInfo* ci) @@ -367,14 +369,6 @@ void lua_singlestep(lua_State* L, int enabled) L->singlestep = bool(enabled); } -void lua_breakpoint(lua_State* L, int funcindex, int line, int enabled) -{ - const TValue* func = luaA_toobject(L, funcindex); - api_check(L, ttisfunction(func) && !clvalue(func)->isC); - - luaG_breakpoint(L, clvalue(func)->l.p, line, bool(enabled)); -} - static int getmaxline(Proto* p) { int result = -1; @@ -394,6 +388,71 @@ static int getmaxline(Proto* p) return result; } +// Find the line number with instructions. If the provided line doesn't have any instruction, it should return the next line number with +// instructions. +static int getnextline(Proto* p, int line) +{ + int closest = -1; + if (p->lineinfo) + { + for (int i = 0; i < p->sizecode; ++i) + { + // note: we keep prologue as is, instead opting to break at the first meaningful instruction + if (LUAU_INSN_OP(p->code[i]) == LOP_PREPVARARGS) + continue; + + int current = luaG_getline(p, i); + if (current >= line) + { + closest = current; + break; + } + } + } + + for (int i = 0; i < p->sizep; ++i) + { + // Find the closest line number to the intended one. + int candidate = getnextline(p->p[i], line); + if (closest == -1 || (candidate >= line && candidate < closest)) + { + closest = candidate; + } + } + + return closest; +} + +int lua_breakpoint(lua_State* L, int funcindex, int line, int enabled) +{ + int target = -1; + + if (FFlag::LuauDebuggerBreakpointHitOnNextBestLine) + { + const TValue* func = luaA_toobject(L, funcindex); + api_check(L, ttisfunction(func) && !clvalue(func)->isC); + + Proto* p = clvalue(func)->l.p; + // Find line number to add the breakpoint to. + target = getnextline(p, line); + + if (target != -1) + { + // Add breakpoint on the exact line + luaG_breakpoint(L, p, target, bool(enabled)); + } + } + else + { + const TValue* func = luaA_toobject(L, funcindex); + api_check(L, ttisfunction(func) && !clvalue(func)->isC); + + luaG_breakpoint(L, clvalue(func)->l.p, line, bool(enabled)); + } + + return target; +} + static void getcoverage(Proto* p, int depth, int* buffer, size_t size, void* context, lua_Coverage callback) { memset(buffer, -1, size * sizeof(int)); diff --git a/VM/src/lstring.cpp b/VM/src/lstring.cpp index c0cd3e26..12bd67d5 100644 --- a/VM/src/lstring.cpp +++ b/VM/src/lstring.cpp @@ -7,6 +7,8 @@ #include +LUAU_FASTFLAGVARIABLE(LuauLazyAtoms, false) + unsigned int luaS_hash(const char* str, size_t len) { // Note that this hashing algorithm is replicated in BytecodeBuilder.cpp, BytecodeBuilder::getStringHash @@ -82,7 +84,7 @@ static TString* newlstr(lua_State* L, const char* str, size_t l, unsigned int h) ts->memcat = L->activememcat; memcpy(ts->data, str, l); ts->data[l] = '\0'; /* ending 0 */ - ts->atom = L->global->cb.useratom ? L->global->cb.useratom(ts->data, l) : -1; + ts->atom = FFlag::LuauLazyAtoms ? ATOM_UNDEF : L->global->cb.useratom ? L->global->cb.useratom(ts->data, l) : -1; tb = &L->global->strt; h = lmod(h, tb->size); ts->next = tb->hash[h]; /* chain new entry */ @@ -165,7 +167,7 @@ TString* luaS_buffinish(lua_State* L, TString* ts) ts->data[ts->len] = '\0'; // ending 0 // Complete string object - ts->atom = L->global->cb.useratom ? L->global->cb.useratom(ts->data, ts->len) : -1; + ts->atom = FFlag::LuauLazyAtoms ? ATOM_UNDEF : L->global->cb.useratom ? L->global->cb.useratom(ts->data, ts->len) : -1; ts->next = tb->hash[bucket]; // chain new entry tb->hash[bucket] = ts; diff --git a/VM/src/lstring.h b/VM/src/lstring.h index 290b64d8..acdf9a1e 100644 --- a/VM/src/lstring.h +++ b/VM/src/lstring.h @@ -8,6 +8,9 @@ /* string size limit */ #define MAXSSIZE (1 << 30) +/* string atoms are not defined by default; the storage is 16-bit integer */ +#define ATOM_UNDEF -32768 + #define sizestring(len) (offsetof(TString, data) + len + 1) #define luaS_new(L, s) (luaS_newlstr(L, s, strlen(s))) diff --git a/bench/gc/test_SunSpider_crypto-aes.lua b/bench/gc/test_SunSpider_crypto-aes.lua deleted file mode 100644 index 8537e3da..00000000 --- a/bench/gc/test_SunSpider_crypto-aes.lua +++ /dev/null @@ -1,436 +0,0 @@ ---[[ - * AES Cipher function: encrypt 'input' with Rijndael algorithm - * - * takes byte-array 'input' (16 bytes) - * 2D byte-array key schedule 'w' (Nr+1 x Nb bytes) - * - * applies Nr rounds (10/12/14) using key schedule w for 'add round key' stage - * - * returns byte-array encrypted value (16 bytes) - */]] - - local bench = script and require(script.Parent.bench_support) or require("bench_support") - -function test() - --- Sbox is pre-computed multiplicative inverse in GF(2^8) used in SubBytes and KeyExpansion [§5.1.1] -local Sbox = { 0x63,0x7c,0x77,0x7b,0xf2,0x6b,0x6f,0xc5,0x30,0x01,0x67,0x2b,0xfe,0xd7,0xab,0x76, - 0xca,0x82,0xc9,0x7d,0xfa,0x59,0x47,0xf0,0xad,0xd4,0xa2,0xaf,0x9c,0xa4,0x72,0xc0, - 0xb7,0xfd,0x93,0x26,0x36,0x3f,0xf7,0xcc,0x34,0xa5,0xe5,0xf1,0x71,0xd8,0x31,0x15, - 0x04,0xc7,0x23,0xc3,0x18,0x96,0x05,0x9a,0x07,0x12,0x80,0xe2,0xeb,0x27,0xb2,0x75, - 0x09,0x83,0x2c,0x1a,0x1b,0x6e,0x5a,0xa0,0x52,0x3b,0xd6,0xb3,0x29,0xe3,0x2f,0x84, - 0x53,0xd1,0x00,0xed,0x20,0xfc,0xb1,0x5b,0x6a,0xcb,0xbe,0x39,0x4a,0x4c,0x58,0xcf, - 0xd0,0xef,0xaa,0xfb,0x43,0x4d,0x33,0x85,0x45,0xf9,0x02,0x7f,0x50,0x3c,0x9f,0xa8, - 0x51,0xa3,0x40,0x8f,0x92,0x9d,0x38,0xf5,0xbc,0xb6,0xda,0x21,0x10,0xff,0xf3,0xd2, - 0xcd,0x0c,0x13,0xec,0x5f,0x97,0x44,0x17,0xc4,0xa7,0x7e,0x3d,0x64,0x5d,0x19,0x73, - 0x60,0x81,0x4f,0xdc,0x22,0x2a,0x90,0x88,0x46,0xee,0xb8,0x14,0xde,0x5e,0x0b,0xdb, - 0xe0,0x32,0x3a,0x0a,0x49,0x06,0x24,0x5c,0xc2,0xd3,0xac,0x62,0x91,0x95,0xe4,0x79, - 0xe7,0xc8,0x37,0x6d,0x8d,0xd5,0x4e,0xa9,0x6c,0x56,0xf4,0xea,0x65,0x7a,0xae,0x08, - 0xba,0x78,0x25,0x2e,0x1c,0xa6,0xb4,0xc6,0xe8,0xdd,0x74,0x1f,0x4b,0xbd,0x8b,0x8a, - 0x70,0x3e,0xb5,0x66,0x48,0x03,0xf6,0x0e,0x61,0x35,0x57,0xb9,0x86,0xc1,0x1d,0x9e, - 0xe1,0xf8,0x98,0x11,0x69,0xd9,0x8e,0x94,0x9b,0x1e,0x87,0xe9,0xce,0x55,0x28,0xdf, - 0x8c,0xa1,0x89,0x0d,0xbf,0xe6,0x42,0x68,0x41,0x99,0x2d,0x0f,0xb0,0x54,0xbb,0x16 }; - --- Rcon is Round Constant used for the Key Expansion [1st col is 2^(r-1) in GF(2^8)] [§5.2] -local Rcon = { { 0x00, 0x00, 0x00, 0x00 }, - {0x01, 0x00, 0x00, 0x00}, - {0x02, 0x00, 0x00, 0x00}, - {0x04, 0x00, 0x00, 0x00}, - {0x08, 0x00, 0x00, 0x00}, - {0x10, 0x00, 0x00, 0x00}, - {0x20, 0x00, 0x00, 0x00}, - {0x40, 0x00, 0x00, 0x00}, - {0x80, 0x00, 0x00, 0x00}, - {0x1b, 0x00, 0x00, 0x00}, - {0x36, 0x00, 0x00, 0x00} }; - -function Cipher(input, w) -- main Cipher function [§5.1] - local Nb = 4; -- block size (in words): no of columns in state (fixed at 4 for AES) - local Nr = #w / Nb - 1; -- no of rounds: 10/12/14 for 128/192/256-bit keys - - local state = {{},{},{},{}}; -- initialise 4xNb byte-array 'state' with input [§3.4] - for i = 0,4*Nb-1 do state[(i % 4) + 1][math.floor(i/4) + 1] = input[i + 1]; end - - state = AddRoundKey(state, w, 0, Nb); - - for round = 1,Nr-1 do - state = SubBytes(state, Nb); - state = ShiftRows(state, Nb); - state = MixColumns(state, Nb); - state = AddRoundKey(state, w, round, Nb); - end - - state = SubBytes(state, Nb); - state = ShiftRows(state, Nb); - state = AddRoundKey(state, w, Nr, Nb); - - local output = {} -- convert state to 1-d array before returning [§3.4] - for i = 0,4*Nb-1 do output[i + 1] = state[(i % 4) + 1][math.floor(i / 4) + 1]; end - - return output; -end - - -function SubBytes(s, Nb) -- apply SBox to state S [§5.1.1] - for r = 0,3 do - for c = 0,Nb-1 do s[r + 1][c + 1] = Sbox[s[r + 1][c + 1] + 1]; end - end - return s; -end - - -function ShiftRows(s, Nb) -- shift row r of state S left by r bytes [§5.1.2] - local t = {}; - for r = 1,3 do - for c = 0,3 do t[c + 1] = s[r + 1][((c + r) % Nb) + 1] end; -- shift into temp copy - for c = 0,3 do s[r + 1][c + 1] = t[c + 1]; end -- and copy back - end -- note that this will work for Nb=4,5,6, but not 7,8 (always 4 for AES): - return s; -- see fp.gladman.plus.com/cryptography_technology/rijndael/aes.spec.311.pdf -end - - -function MixColumns(s, Nb) -- combine bytes of each col of state S [§5.1.3] - for c = 0,3 do - local a = {}; -- 'a' is a copy of the current column from 's' - local b = {}; -- 'b' is a•{02} in GF(2^8) - for i = 0,3 do - a[i + 1] = s[i + 1][c + 1]; - - if bit32.band(s[i + 1][c + 1], 0x80) ~= 0 then - b[i + 1] = bit32.bxor(bit32.lshift(s[i + 1][c + 1], 1), 0x011b); - else - b[i + 1] = bit32.lshift(s[i + 1][c + 1], 1); - end - end - -- a[n] ^ b[n] is a•{03} in GF(2^8) - s[1][c + 1] = bit32.bxor(bit32.bxor(bit32.bxor(b[1], a[2]), bit32.bxor(b[2], a[3])), a[4]); -- 2*a0 + 3*a1 + a2 + a3 - s[2][c + 1] = bit32.bxor(bit32.bxor(bit32.bxor(a[1], b[2]), bit32.bxor(a[3], b[3])), a[4]); -- a0 * 2*a1 + 3*a2 + a3 - s[3][c + 1] = bit32.bxor(bit32.bxor(bit32.bxor(a[1], a[2]), bit32.bxor(b[3], a[4])), b[4]); -- a0 + a1 + 2*a2 + 3*a3 - s[4][c + 1] = bit32.bxor(bit32.bxor(bit32.bxor(a[1], b[1]), bit32.bxor(a[2], a[3])), b[4]); -- 3*a0 + a1 + a2 + 2*a3 -end - return s; -end - - -function AddRoundKey(state, w, rnd, Nb) -- xor Round Key into state S [§5.1.4] - for r = 0,3 do - for c = 0,Nb-1 do state[r + 1][c + 1] = bit32.bxor(state[r + 1][c + 1], w[rnd*4+c + 1][r + 1]); end - end - return state; -end - - -function KeyExpansion(key) -- generate Key Schedule (byte-array Nr+1 x Nb) from Key [§5.2] - local Nb = 4; -- block size (in words): no of columns in state (fixed at 4 for AES) - local Nk = #key / 4 -- key length (in words): 4/6/8 for 128/192/256-bit keys - local Nr = Nk + 6; -- no of rounds: 10/12/14 for 128/192/256-bit keys - - local w = {}; - local temp = {}; - - for i = 0,Nk do - local r = { key[4*i + 1], key[4*i + 2], key[4*i + 3], key[4*i + 4] }; - w[i + 1] = r; - end - - for i = Nk,(Nb*(Nr+1)) - 1 do - w[i + 1] = {}; - for t = 0,3 do temp[t + 1] = w[i-1 + 1][t + 1]; end - if (i % Nk == 0) then - temp = SubWord(RotWord(temp)); - for t = 0,3 do temp[t + 1] = bit32.bxor(temp[t + 1], Rcon[i/Nk + 1][t + 1]); end - elseif (Nk > 6 and i % Nk == 4) then - temp = SubWord(temp); - end - for t = 0,3 do w[i + 1][t + 1] = bit32.bxor(w[i - Nk + 1][t + 1], temp[t + 1]); end - end - - return w; -end - -function SubWord(w) -- apply SBox to 4-byte word w - for i = 0,3 do w[i + 1] = Sbox[w[i + 1] + 1]; end - return w; -end - -function RotWord(w) -- rotate 4-byte word w left by one byte - w[5] = w[1]; - for i = 0,3 do w[i + 1] = w[i + 2]; end - return w; -end - - ---[[ - * Use AES to encrypt 'plaintext' with 'password' using 'nBits' key, in 'Counter' mode of operation - * - see http://csrc.nist.gov/publications/nistpubs/800-38a/sp800-38a.pdf - * for each block - * - outputblock = cipher(counter, key) - * - cipherblock = plaintext xor outputblock - ]] - -function AESEncryptCtr(plaintext, password, nBits) - if (not (nBits==128 or nBits==192 or nBits==256)) then return ''; end -- standard allows 128/192/256 bit keys - - -- for this example script, generate the key by applying Cipher to 1st 16/24/32 chars of password; - -- for real-world applications, a higher security approach would be to hash the password e.g. with SHA-1 - local nBytes = nBits/8; -- no bytes in key - local pwBytes = {}; - for i = 0,nBytes-1 do pwBytes[i + 1] = bit32.band(string.byte(password, i + 1), 0xff); end - local key = Cipher(pwBytes, KeyExpansion(pwBytes)); - - -- key is now 16/24/32 bytes long - for i = 1,nBytes-16 do - table.insert(key, key[i]) - end - - -- initialise counter block (NIST SP800-38A §B.2): millisecond time-stamp for nonce in 1st 8 bytes, - -- block counter in 2nd 8 bytes - local blockSize = 16; -- block size fixed at 16 bytes / 128 bits (Nb=4) for AES - local counterBlock = {}; -- block size fixed at 16 bytes / 128 bits (Nb=4) for AES - local nonce = 12564231564 -- (new Date()).getTime(); -- milliseconds since 1-Jan-1970 - - -- encode nonce in two stages to cater for JavaScript 32-bit limit on bitwise ops - for i = 0,3 do counterBlock[i + 1] = bit32.band(bit32.rshift(nonce, i * 8), 0xff); end - for i = 0,3 do counterBlock[i + 4 + 1] = bit32.band(bit32.rshift(math.floor(nonce / 0x100000000), i*8), 0xff); end - - -- generate key schedule - an expansion of the key into distinct Key Rounds for each round - local keySchedule = KeyExpansion(key); - - local blockCount = math.ceil(#plaintext / blockSize); - local ciphertext = {}; -- ciphertext as array of strings - - for b = 0,blockCount-1 do - -- set counter (block #) in last 8 bytes of counter block (leaving nonce in 1st 8 bytes) - -- again done in two stages for 32-bit ops - for c = 0,3 do counterBlock[15-c + 1] = bit32.band(bit32.rshift(b, c*8), 0xff); end - for c = 0,3 do counterBlock[15-c-4 + 1] = bit32.rshift(math.floor(b/0x100000000), c*8) end - - local cipherCntr = Cipher(counterBlock, keySchedule); -- -- encrypt counter block -- - - -- calculate length of final block: - local blockLength = nil - - if b #define DOCTEST_BREAK_INTO_DEBUGGER() raise(SIGTRAP) #endif #elif defined(DOCTEST_PLATFORM_MAC) #if defined(__x86_64) || defined(__x86_64__) || defined(__amd64__) || defined(__i386) -#define DOCTEST_BREAK_INTO_DEBUGGER() __asm__("int $3\n" : :) // NOLINT (hicpp-no-assembler) +#define DOCTEST_BREAK_INTO_DEBUGGER() __asm__("int $3\n" : :) // NOLINT(hicpp-no-assembler) +#elif defined(__ppc__) || defined(__ppc64__) +// https://www.cocoawithlove.com/2008/03/break-into-debugger.html +#define DOCTEST_BREAK_INTO_DEBUGGER() __asm__("li r0, 20\nsc\nnop\nli r0, 37\nli r4, 2\nsc\nnop\n": : : "memory","r0","r3","r4") // NOLINT(hicpp-no-assembler) #else -#define DOCTEST_BREAK_INTO_DEBUGGER() __asm__("brk #0"); // NOLINT (hicpp-no-assembler) +#define DOCTEST_BREAK_INTO_DEBUGGER() __asm__("brk #0"); // NOLINT(hicpp-no-assembler) #endif #elif DOCTEST_MSVC #define DOCTEST_BREAK_INTO_DEBUGGER() __debugbreak() @@ -387,54 +463,66 @@ DOCTEST_GCC_SUPPRESS_WARNING_POP // this is kept here for backwards compatibility since the config option was changed #ifdef DOCTEST_CONFIG_USE_IOSFWD +#ifndef DOCTEST_CONFIG_USE_STD_HEADERS #define DOCTEST_CONFIG_USE_STD_HEADERS +#endif #endif // DOCTEST_CONFIG_USE_IOSFWD +// for clang - always include ciso646 (which drags some std stuff) because +// we want to check if we are using libc++ with the _LIBCPP_VERSION macro in +// which case we don't want to forward declare stuff from std - for reference: +// https://github.com/doctest/doctest/issues/126 +// https://github.com/doctest/doctest/issues/356 +#if DOCTEST_CLANG +#include +#ifdef _LIBCPP_VERSION +#ifndef DOCTEST_CONFIG_USE_STD_HEADERS +#define DOCTEST_CONFIG_USE_STD_HEADERS +#endif +#endif // _LIBCPP_VERSION +#endif // clang + #ifdef DOCTEST_CONFIG_USE_STD_HEADERS #ifndef DOCTEST_CONFIG_INCLUDE_TYPE_TRAITS #define DOCTEST_CONFIG_INCLUDE_TYPE_TRAITS #endif // DOCTEST_CONFIG_INCLUDE_TYPE_TRAITS -#include +DOCTEST_MAKE_STD_HEADERS_CLEAN_FROM_WARNINGS_ON_WALL_BEGIN #include #include +#include +DOCTEST_MAKE_STD_HEADERS_CLEAN_FROM_WARNINGS_ON_WALL_END #else // DOCTEST_CONFIG_USE_STD_HEADERS -#if DOCTEST_CLANG -// to detect if libc++ is being used with clang (the _LIBCPP_VERSION identifier) -#include -#endif // clang - -#ifdef _LIBCPP_VERSION -#define DOCTEST_STD_NAMESPACE_BEGIN _LIBCPP_BEGIN_NAMESPACE_STD -#define DOCTEST_STD_NAMESPACE_END _LIBCPP_END_NAMESPACE_STD -#else // _LIBCPP_VERSION -#define DOCTEST_STD_NAMESPACE_BEGIN namespace std { -#define DOCTEST_STD_NAMESPACE_END } -#endif // _LIBCPP_VERSION - // Forward declaring 'X' in namespace std is not permitted by the C++ Standard. DOCTEST_MSVC_SUPPRESS_WARNING_WITH_PUSH(4643) -DOCTEST_STD_NAMESPACE_BEGIN // NOLINT (cert-dcl58-cpp) -typedef decltype(nullptr) nullptr_t; +namespace std { // NOLINT(cert-dcl58-cpp) +typedef decltype(nullptr) nullptr_t; // NOLINT(modernize-use-using) +typedef decltype(sizeof(void*)) size_t; // NOLINT(modernize-use-using) template struct char_traits; template <> struct char_traits; template -class basic_ostream; -typedef basic_ostream> ostream; +class basic_ostream; // NOLINT(fuchsia-virtual-inheritance) +typedef basic_ostream> ostream; // NOLINT(modernize-use-using) +template +// NOLINTNEXTLINE +basic_ostream& operator<<(basic_ostream&, const char*); +template +class basic_istream; +typedef basic_istream> istream; // NOLINT(modernize-use-using) template class tuple; #if DOCTEST_MSVC >= DOCTEST_COMPILER(19, 20, 0) -// see this issue on why this is needed: https://github.com/onqtam/doctest/issues/183 -template +// see this issue on why this is needed: https://github.com/doctest/doctest/issues/183 +template class allocator; -template +template class basic_string; using string = basic_string, allocator>; #endif // VS 2019 -DOCTEST_STD_NAMESPACE_END +} // namespace std DOCTEST_MSVC_SUPPRESS_WARNING_POP @@ -446,8 +534,14 @@ DOCTEST_MSVC_SUPPRESS_WARNING_POP namespace doctest { +using std::size_t; + DOCTEST_INTERFACE extern bool is_running_in_test; +#ifndef DOCTEST_CONFIG_STRING_SIZE_TYPE +#define DOCTEST_CONFIG_STRING_SIZE_TYPE unsigned +#endif + // A 24 byte string class (can be as small as 17 for x64 and 13 for x86) that can hold strings with length // of up to 23 chars on the stack before going on the heap - the last byte of the buffer is used for: // - "is small" bit - the highest bit - if "0" then it is small - otherwise its "1" (128) @@ -460,7 +554,6 @@ DOCTEST_INTERFACE extern bool is_running_in_test; // TODO: // - optimizations - like not deleting memory unnecessarily in operator= and etc. // - resize/reserve/clear -// - substr // - replace // - back/front // - iterator stuff @@ -470,63 +563,84 @@ DOCTEST_INTERFACE extern bool is_running_in_test; // - relational operators as free functions - taking const char* as one of the params class DOCTEST_INTERFACE String { - static const unsigned len = 24; //!OCLINT avoid private static members - static const unsigned last = len - 1; //!OCLINT avoid private static members +public: + using size_type = DOCTEST_CONFIG_STRING_SIZE_TYPE; + +private: + static DOCTEST_CONSTEXPR size_type len = 24; //!OCLINT avoid private static members + static DOCTEST_CONSTEXPR size_type last = len - 1; //!OCLINT avoid private static members struct view // len should be more than sizeof(view) - because of the final byte for flags { char* ptr; - unsigned size; - unsigned capacity; + size_type size; + size_type capacity; }; union { - char buf[len]; + char buf[len]; // NOLINT(*-avoid-c-arrays) view data; }; - bool isOnStack() const { return (buf[last] & 128) == 0; } - void setOnHeap(); - void setLast(unsigned in = last); + char* allocate(size_type sz); + + bool isOnStack() const noexcept { return (buf[last] & 128) == 0; } + void setOnHeap() noexcept; + void setLast(size_type in = last) noexcept; + void setSize(size_type sz) noexcept; void copy(const String& other); public: - String(); + static DOCTEST_CONSTEXPR size_type npos = static_cast(-1); + + String() noexcept; ~String(); // cppcheck-suppress noExplicitConstructor String(const char* in); - String(const char* in, unsigned in_size); + String(const char* in, size_type in_size); + + String(std::istream& in, size_type in_size); String(const String& other); String& operator=(const String& other); String& operator+=(const String& other); - String operator+(const String& other) const; - String(String&& other); - String& operator=(String&& other); + String(String&& other) noexcept; + String& operator=(String&& other) noexcept; - char operator[](unsigned i) const; - char& operator[](unsigned i); + char operator[](size_type i) const; + char& operator[](size_type i); // the only functions I'm willing to leave in the interface - available for inlining const char* c_str() const { return const_cast(this)->c_str(); } // NOLINT char* c_str() { - if(isOnStack()) + if (isOnStack()) { return reinterpret_cast(buf); + } return data.ptr; } - unsigned size() const; - unsigned capacity() const; + size_type size() const; + size_type capacity() const; + + String substr(size_type pos, size_type cnt = npos) &&; + String substr(size_type pos, size_type cnt = npos) const &; + + size_type find(char ch, size_type pos = 0) const; + size_type rfind(char ch, size_type pos = npos) const; int compare(const char* other, bool no_case = false) const; int compare(const String& other, bool no_case = false) const; + +friend DOCTEST_INTERFACE std::ostream& operator<<(std::ostream& s, const String& in); }; +DOCTEST_INTERFACE String operator+(const String& lhs, const String& rhs); + DOCTEST_INTERFACE bool operator==(const String& lhs, const String& rhs); DOCTEST_INTERFACE bool operator!=(const String& lhs, const String& rhs); DOCTEST_INTERFACE bool operator<(const String& lhs, const String& rhs); @@ -534,7 +648,21 @@ DOCTEST_INTERFACE bool operator>(const String& lhs, const String& rhs); DOCTEST_INTERFACE bool operator<=(const String& lhs, const String& rhs); DOCTEST_INTERFACE bool operator>=(const String& lhs, const String& rhs); -DOCTEST_INTERFACE std::ostream& operator<<(std::ostream& s, const String& in); +class DOCTEST_INTERFACE Contains { +public: + explicit Contains(const String& string); + + bool checkWith(const String& other) const; + + String string; +}; + +DOCTEST_INTERFACE String toString(const Contains& in); + +DOCTEST_INTERFACE bool operator==(const String& lhs, const Contains& rhs); +DOCTEST_INTERFACE bool operator==(const Contains& lhs, const String& rhs); +DOCTEST_INTERFACE bool operator!=(const String& lhs, const Contains& rhs); +DOCTEST_INTERFACE bool operator!=(const Contains& lhs, const String& rhs); namespace Color { enum Enum @@ -607,7 +735,7 @@ namespace assertType { DT_WARN_THROWS_WITH = is_throws_with | is_warn, DT_CHECK_THROWS_WITH = is_throws_with | is_check, DT_REQUIRE_THROWS_WITH = is_throws_with | is_require, - + DT_WARN_THROWS_WITH_AS = is_throws_with | is_throws_as | is_warn, DT_CHECK_THROWS_WITH_AS = is_throws_with | is_throws_as | is_check, DT_REQUIRE_THROWS_WITH_AS = is_throws_with | is_throws_as | is_require, @@ -688,9 +816,27 @@ struct DOCTEST_INTERFACE AssertData String m_decomp; // for specific exception-related asserts - bool m_threw_as; - const char* m_exception_type; - const char* m_exception_string; + bool m_threw_as; + const char* m_exception_type; + + class DOCTEST_INTERFACE StringContains { + private: + Contains content; + bool isContains; + + public: + StringContains(const String& str) : content(str), isContains(false) { } + StringContains(Contains cntn) : content(static_cast(cntn)), isContains(true) { } + + bool check(const String& str) { return isContains ? (content == str) : (content.string == str); } + + operator const String&() const { return content.string; } + + const char* c_str() const { return content.string.c_str(); } + } m_exception_string; + + AssertData(assertType::Enum at, const char* file, int line, const char* expr, + const char* exception_type, const StringContains& exception_string); }; struct DOCTEST_INTERFACE MessageData @@ -707,13 +853,13 @@ struct DOCTEST_INTERFACE SubcaseSignature const char* m_file; int m_line; + bool operator==(const SubcaseSignature& other) const; bool operator<(const SubcaseSignature& other) const; }; struct DOCTEST_INTERFACE IContextScope { - IContextScope(); - virtual ~IContextScope(); + DOCTEST_DECLARE_INTERFACE(IContextScope) virtual void stringify(std::ostream*) const = 0; }; @@ -723,9 +869,8 @@ namespace detail { struct ContextOptions //!OCLINT too many fields { - std::ostream* cout; // stdout stream - std::cout by default - std::ostream* cerr; // stderr stream - std::cerr by default - String binary_name; // the test binary name + std::ostream* cout = nullptr; // stdout stream + String binary_name; // the test binary name const detail::TestCase* currentTest = nullptr; @@ -744,9 +889,12 @@ struct ContextOptions //!OCLINT too many fields bool case_sensitive; // if filtering should be case sensitive bool exit; // if the program should be exited after the tests are ran/whatever bool duration; // print the time duration of each test case + bool minimal; // minimal console output (only test failures) + bool quiet; // no console output bool no_throw; // to skip exceptions-related assertion macros bool no_exitcode; // if the framework should return 0 as the exitcode bool no_run; // to not run the tests at all (can be done with an "*" exclude) + bool no_intro; // to not print the intro of the framework bool no_version; // to not print the version of the framework bool no_colors; // if output to the console should be colorized bool force_colors; // forces the use of colors even when a tty cannot be detected @@ -768,150 +916,184 @@ struct ContextOptions //!OCLINT too many fields }; namespace detail { - template - struct enable_if - {}; - - template - struct enable_if - { typedef TYPE type; }; - - // clang-format off - template struct remove_reference { typedef T type; }; - template struct remove_reference { typedef T type; }; - template struct remove_reference { typedef T type; }; - - template U declval(int); - - template T declval(long); - - template auto declval() DOCTEST_NOEXCEPT -> decltype(declval(0)) ; - - template struct is_lvalue_reference { const static bool value=false; }; - template struct is_lvalue_reference { const static bool value=true; }; - - template - inline T&& forward(typename remove_reference::type& t) DOCTEST_NOEXCEPT - { - return static_cast(t); - } - - template - inline T&& forward(typename remove_reference::type&& t) DOCTEST_NOEXCEPT - { - static_assert(!is_lvalue_reference::value, - "Can not forward an rvalue as an lvalue."); - return static_cast(t); - } - - template struct remove_const { typedef T type; }; - template struct remove_const { typedef T type; }; + namespace types { #ifdef DOCTEST_CONFIG_INCLUDE_TYPE_TRAITS - template struct is_enum : public std::is_enum {}; - template struct underlying_type : public std::underlying_type {}; + using namespace std; #else - // Use compiler intrinsics - template struct is_enum { constexpr static bool value = __is_enum(T); }; - template struct underlying_type { typedef __underlying_type(T) type; }; + template + struct enable_if { }; + + template + struct enable_if { using type = T; }; + + struct true_type { static DOCTEST_CONSTEXPR bool value = true; }; + struct false_type { static DOCTEST_CONSTEXPR bool value = false; }; + + template struct remove_reference { using type = T; }; + template struct remove_reference { using type = T; }; + template struct remove_reference { using type = T; }; + + template struct is_rvalue_reference : false_type { }; + template struct is_rvalue_reference : true_type { }; + + template struct remove_const { using type = T; }; + template struct remove_const { using type = T; }; + + // Compiler intrinsics + template struct is_enum { static DOCTEST_CONSTEXPR bool value = __is_enum(T); }; + template struct underlying_type { using type = __underlying_type(T); }; + + template struct is_pointer : false_type { }; + template struct is_pointer : true_type { }; + + template struct is_array : false_type { }; + // NOLINTNEXTLINE(*-avoid-c-arrays) + template struct is_array : true_type { }; #endif - // clang-format on + } + + // + template + T&& declval(); + + template + DOCTEST_CONSTEXPR_FUNC T&& forward(typename types::remove_reference::type& t) DOCTEST_NOEXCEPT { + return static_cast(t); + } + + template + DOCTEST_CONSTEXPR_FUNC T&& forward(typename types::remove_reference::type&& t) DOCTEST_NOEXCEPT { + return static_cast(t); + } template - struct deferred_false - // cppcheck-suppress unusedStructMember - { static const bool value = false; }; + struct deferred_false : types::false_type { }; - namespace has_insertion_operator_impl { - std::ostream &os(); - template - DOCTEST_REF_WRAP(T) val(); +// MSVS 2015 :( +#if defined(_MSC_VER) && _MSC_VER <= 1900 + template + struct has_global_insertion_operator : types::false_type { }; - template - struct check { - static constexpr bool value = false; - }; + template + struct has_global_insertion_operator(), declval()), void())> : types::true_type { }; - template - struct check(), void())> { - static constexpr bool value = true; - }; - } // namespace has_insertion_operator_impl + template + struct has_insertion_operator { static DOCTEST_CONSTEXPR bool value = has_global_insertion_operator::value; }; - template - using has_insertion_operator = has_insertion_operator_impl::check; + template + struct insert_hack; - DOCTEST_INTERFACE void my_memcpy(void* dest, const void* src, unsigned num); + template + struct insert_hack { + static void insert(std::ostream& os, const T& t) { ::operator<<(os, t); } + }; - DOCTEST_INTERFACE std::ostream* getTlsOss(); // returns a thread-local ostringstream - DOCTEST_INTERFACE String getTlsOssResult(); + template + struct insert_hack { + static void insert(std::ostream& os, const T& t) { operator<<(os, t); } + }; + + template + using insert_hack_t = insert_hack::value>; +#else + template + struct has_insertion_operator : types::false_type { }; +#endif + +template +struct has_insertion_operator(), declval()), void())> : types::true_type { }; + + DOCTEST_INTERFACE std::ostream* tlssPush(); + DOCTEST_INTERFACE String tlssPop(); template - struct StringMakerBase - { + struct StringMakerBase { template static String convert(const DOCTEST_REF_WRAP(T)) { +#ifdef DOCTEST_CONFIG_REQUIRE_STRINGIFICATION_FOR_ALL_USED_TYPES + static_assert(deferred_false::value, "No stringification detected for type T. See string conversion manual"); +#endif return "{?}"; } }; + template + struct filldata; + + template + void filloss(std::ostream* stream, const T& in) { + filldata::fill(stream, in); + } + + template + void filloss(std::ostream* stream, const T (&in)[N]) { // NOLINT(*-avoid-c-arrays) + // T[N], T(&)[N], T(&&)[N] have same behaviour. + // Hence remove reference. + filloss::type>(stream, in); + } + + template + String toStream(const T& in) { + std::ostream* stream = tlssPush(); + filloss(stream, in); + return tlssPop(); + } + template <> - struct StringMakerBase - { + struct StringMakerBase { template static String convert(const DOCTEST_REF_WRAP(T) in) { - *getTlsOss() << in; - return getTlsOssResult(); + return toStream(in); } }; - - DOCTEST_INTERFACE String rawMemoryToString(const void* object, unsigned size); - - template - String rawMemoryToString(const DOCTEST_REF_WRAP(T) object) { - return rawMemoryToString(&object, sizeof(object)); - } - - template - const char* type_to_string() { - return "<>"; - } } // namespace detail template -struct StringMaker : public detail::StringMakerBase::value> +struct StringMaker : public detail::StringMakerBase< + detail::has_insertion_operator::value || detail::types::is_pointer::value || detail::types::is_array::value> {}; +#ifndef DOCTEST_STRINGIFY +#ifdef DOCTEST_CONFIG_DOUBLE_STRINGIFY +#define DOCTEST_STRINGIFY(...) toString(toString(__VA_ARGS__)) +#else +#define DOCTEST_STRINGIFY(...) toString(__VA_ARGS__) +#endif +#endif + template -struct StringMaker -{ - template - static String convert(U* p) { - if(p) - return detail::rawMemoryToString(p); - return "NULL"; - } -}; +String toString() { +#if DOCTEST_MSVC >= 0 && DOCTEST_CLANG == 0 && DOCTEST_GCC == 0 + String ret = __FUNCSIG__; // class doctest::String __cdecl doctest::toString(void) + String::size_type beginPos = ret.find('<'); + return ret.substr(beginPos + 1, ret.size() - beginPos - static_cast(sizeof(">(void)"))); +#else + String ret = __PRETTY_FUNCTION__; // doctest::String toString() [with T = TYPE] + String::size_type begin = ret.find('=') + 2; + return ret.substr(begin, ret.size() - begin - 1); +#endif +} -template -struct StringMaker -{ - static String convert(R C::*p) { - if(p) - return detail::rawMemoryToString(p); - return "NULL"; - } -}; - -template ::value, bool>::type = true> +template ::value, bool>::type = true> String toString(const DOCTEST_REF_WRAP(T) value) { return StringMaker::convert(value); } #ifdef DOCTEST_CONFIG_TREAT_CHAR_STAR_AS_STRING -DOCTEST_INTERFACE String toString(char* in); DOCTEST_INTERFACE String toString(const char* in); #endif // DOCTEST_CONFIG_TREAT_CHAR_STAR_AS_STRING + +#if DOCTEST_MSVC >= DOCTEST_COMPILER(19, 20, 0) +// see this issue on why this is needed: https://github.com/doctest/doctest/issues/183 +DOCTEST_INTERFACE String toString(const std::string& in); +#endif // VS 2019 + +DOCTEST_INTERFACE String toString(String in); + +DOCTEST_INTERFACE String toString(std::nullptr_t); + DOCTEST_INTERFACE String toString(bool in); + DOCTEST_INTERFACE String toString(float in); DOCTEST_INTERFACE String toString(double in); DOCTEST_INTERFACE String toString(double long in); @@ -919,40 +1101,85 @@ DOCTEST_INTERFACE String toString(double long in); DOCTEST_INTERFACE String toString(char in); DOCTEST_INTERFACE String toString(char signed in); DOCTEST_INTERFACE String toString(char unsigned in); -DOCTEST_INTERFACE String toString(int short in); -DOCTEST_INTERFACE String toString(int short unsigned in); -DOCTEST_INTERFACE String toString(int in); -DOCTEST_INTERFACE String toString(int unsigned in); -DOCTEST_INTERFACE String toString(int long in); -DOCTEST_INTERFACE String toString(int long unsigned in); -DOCTEST_INTERFACE String toString(int long long in); -DOCTEST_INTERFACE String toString(int long long unsigned in); -DOCTEST_INTERFACE String toString(std::nullptr_t in); +DOCTEST_INTERFACE String toString(short in); +DOCTEST_INTERFACE String toString(short unsigned in); +DOCTEST_INTERFACE String toString(signed in); +DOCTEST_INTERFACE String toString(unsigned in); +DOCTEST_INTERFACE String toString(long in); +DOCTEST_INTERFACE String toString(long unsigned in); +DOCTEST_INTERFACE String toString(long long in); +DOCTEST_INTERFACE String toString(long long unsigned in); -template ::value, bool>::type = true> +template ::value, bool>::type = true> String toString(const DOCTEST_REF_WRAP(T) value) { - typedef typename detail::underlying_type::type UT; - return toString(static_cast(value)); + using UT = typename detail::types::underlying_type::type; + return (DOCTEST_STRINGIFY(static_cast(value))); } -#if DOCTEST_MSVC >= DOCTEST_COMPILER(19, 20, 0) -// see this issue on why this is needed: https://github.com/onqtam/doctest/issues/183 -DOCTEST_INTERFACE String toString(const std::string& in); -#endif // VS 2019 +namespace detail { + template + struct filldata + { + static void fill(std::ostream* stream, const T& in) { +#if defined(_MSC_VER) && _MSC_VER <= 1900 + insert_hack_t::insert(*stream, in); +#else + operator<<(*stream, in); +#endif + } + }; -class DOCTEST_INTERFACE Approx +DOCTEST_MSVC_SUPPRESS_WARNING_WITH_PUSH(4866) +// NOLINTBEGIN(*-avoid-c-arrays) + template + struct filldata { + static void fill(std::ostream* stream, const T(&in)[N]) { + *stream << "["; + for (size_t i = 0; i < N; i++) { + if (i != 0) { *stream << ", "; } + *stream << (DOCTEST_STRINGIFY(in[i])); + } + *stream << "]"; + } + }; +// NOLINTEND(*-avoid-c-arrays) +DOCTEST_MSVC_SUPPRESS_WARNING_POP + + // Specialized since we don't want the terminating null byte! +// NOLINTBEGIN(*-avoid-c-arrays) + template + struct filldata { + static void fill(std::ostream* stream, const char (&in)[N]) { + *stream << String(in, in[N - 1] ? N : N - 1); + } // NOLINT(clang-analyzer-cplusplus.NewDeleteLeaks) + }; +// NOLINTEND(*-avoid-c-arrays) + + template <> + struct filldata { + static void fill(std::ostream* stream, const void* in); + }; + + template + struct filldata { + static void fill(std::ostream* stream, const T* in) { + filldata::fill(stream, in); + } + }; +} + +struct DOCTEST_INTERFACE Approx { -public: - explicit Approx(double value); + Approx(double value); Approx operator()(double value) const; #ifdef DOCTEST_CONFIG_INCLUDE_TYPE_TRAITS template explicit Approx(const T& value, - typename detail::enable_if::value>::type* = + typename detail::types::enable_if::value>::type* = static_cast(nullptr)) { - *this = Approx(static_cast(value)); + *this = static_cast(value); } #endif // DOCTEST_CONFIG_INCLUDE_TYPE_TRAITS @@ -960,7 +1187,7 @@ public: #ifdef DOCTEST_CONFIG_INCLUDE_TYPE_TRAITS template - typename detail::enable_if::value, Approx&>::type epsilon( + typename std::enable_if::value, Approx&>::type epsilon( const T& newEpsilon) { m_epsilon = static_cast(newEpsilon); return *this; @@ -971,7 +1198,7 @@ public: #ifdef DOCTEST_CONFIG_INCLUDE_TYPE_TRAITS template - typename detail::enable_if::value, Approx&>::type scale( + typename std::enable_if::value, Approx&>::type scale( const T& newScale) { m_scale = static_cast(newScale); return *this; @@ -992,30 +1219,27 @@ public: DOCTEST_INTERFACE friend bool operator> (double lhs, const Approx & rhs); DOCTEST_INTERFACE friend bool operator> (const Approx & lhs, double rhs); - DOCTEST_INTERFACE friend String toString(const Approx& in); - #ifdef DOCTEST_CONFIG_INCLUDE_TYPE_TRAITS #define DOCTEST_APPROX_PREFIX \ - template friend typename detail::enable_if::value, bool>::type + template friend typename std::enable_if::value, bool>::type - DOCTEST_APPROX_PREFIX operator==(const T& lhs, const Approx& rhs) { return operator==(double(lhs), rhs); } + DOCTEST_APPROX_PREFIX operator==(const T& lhs, const Approx& rhs) { return operator==(static_cast(lhs), rhs); } DOCTEST_APPROX_PREFIX operator==(const Approx& lhs, const T& rhs) { return operator==(rhs, lhs); } DOCTEST_APPROX_PREFIX operator!=(const T& lhs, const Approx& rhs) { return !operator==(lhs, rhs); } DOCTEST_APPROX_PREFIX operator!=(const Approx& lhs, const T& rhs) { return !operator==(rhs, lhs); } - DOCTEST_APPROX_PREFIX operator<=(const T& lhs, const Approx& rhs) { return double(lhs) < rhs.m_value || lhs == rhs; } - DOCTEST_APPROX_PREFIX operator<=(const Approx& lhs, const T& rhs) { return lhs.m_value < double(rhs) || lhs == rhs; } - DOCTEST_APPROX_PREFIX operator>=(const T& lhs, const Approx& rhs) { return double(lhs) > rhs.m_value || lhs == rhs; } - DOCTEST_APPROX_PREFIX operator>=(const Approx& lhs, const T& rhs) { return lhs.m_value > double(rhs) || lhs == rhs; } - DOCTEST_APPROX_PREFIX operator< (const T& lhs, const Approx& rhs) { return double(lhs) < rhs.m_value && lhs != rhs; } - DOCTEST_APPROX_PREFIX operator< (const Approx& lhs, const T& rhs) { return lhs.m_value < double(rhs) && lhs != rhs; } - DOCTEST_APPROX_PREFIX operator> (const T& lhs, const Approx& rhs) { return double(lhs) > rhs.m_value && lhs != rhs; } - DOCTEST_APPROX_PREFIX operator> (const Approx& lhs, const T& rhs) { return lhs.m_value > double(rhs) && lhs != rhs; } + DOCTEST_APPROX_PREFIX operator<=(const T& lhs, const Approx& rhs) { return static_cast(lhs) < rhs.m_value || lhs == rhs; } + DOCTEST_APPROX_PREFIX operator<=(const Approx& lhs, const T& rhs) { return lhs.m_value < static_cast(rhs) || lhs == rhs; } + DOCTEST_APPROX_PREFIX operator>=(const T& lhs, const Approx& rhs) { return static_cast(lhs) > rhs.m_value || lhs == rhs; } + DOCTEST_APPROX_PREFIX operator>=(const Approx& lhs, const T& rhs) { return lhs.m_value > static_cast(rhs) || lhs == rhs; } + DOCTEST_APPROX_PREFIX operator< (const T& lhs, const Approx& rhs) { return static_cast(lhs) < rhs.m_value && lhs != rhs; } + DOCTEST_APPROX_PREFIX operator< (const Approx& lhs, const T& rhs) { return lhs.m_value < static_cast(rhs) && lhs != rhs; } + DOCTEST_APPROX_PREFIX operator> (const T& lhs, const Approx& rhs) { return static_cast(lhs) > rhs.m_value && lhs != rhs; } + DOCTEST_APPROX_PREFIX operator> (const Approx& lhs, const T& rhs) { return lhs.m_value > static_cast(rhs) && lhs != rhs; } #undef DOCTEST_APPROX_PREFIX #endif // DOCTEST_CONFIG_INCLUDE_TYPE_TRAITS // clang-format on -private: double m_epsilon; double m_scale; double m_value; @@ -1025,18 +1249,35 @@ DOCTEST_INTERFACE String toString(const Approx& in); DOCTEST_INTERFACE const ContextOptions* getContextOptions(); -#if !defined(DOCTEST_CONFIG_DISABLE) +template +struct DOCTEST_INTERFACE_DECL IsNaN +{ + F value; bool flipped; + IsNaN(F f, bool flip = false) : value(f), flipped(flip) { } + IsNaN operator!() const { return { value, !flipped }; } + operator bool() const; +}; +#ifndef __MINGW32__ +extern template struct DOCTEST_INTERFACE_DECL IsNaN; +extern template struct DOCTEST_INTERFACE_DECL IsNaN; +extern template struct DOCTEST_INTERFACE_DECL IsNaN; +#endif +DOCTEST_INTERFACE String toString(IsNaN in); +DOCTEST_INTERFACE String toString(IsNaN in); +DOCTEST_INTERFACE String toString(IsNaN in); + +#ifndef DOCTEST_CONFIG_DISABLE namespace detail { // clang-format off #ifdef DOCTEST_CONFIG_TREAT_CHAR_STAR_AS_STRING - template struct decay_array { typedef T type; }; - template struct decay_array { typedef T* type; }; - template struct decay_array { typedef T* type; }; + template struct decay_array { using type = T; }; + template struct decay_array { using type = T*; }; + template struct decay_array { using type = T*; }; - template struct not_char_pointer { enum { value = 1 }; }; - template<> struct not_char_pointer { enum { value = 0 }; }; - template<> struct not_char_pointer { enum { value = 0 }; }; + template struct not_char_pointer { static DOCTEST_CONSTEXPR value = 1; }; + template<> struct not_char_pointer { static DOCTEST_CONSTEXPR value = 0; }; + template<> struct not_char_pointer { static DOCTEST_CONSTEXPR value = 0; }; template struct can_use_op : public not_char_pointer::type> {}; #endif // DOCTEST_CONFIG_TREAT_CHAR_STAR_AS_STRING @@ -1059,16 +1300,22 @@ namespace detail { bool m_entered = false; Subcase(const String& name, const char* file, int line); + Subcase(const Subcase&) = delete; + Subcase(Subcase&&) = delete; + Subcase& operator=(const Subcase&) = delete; + Subcase& operator=(Subcase&&) = delete; ~Subcase(); operator bool() const; + + private: + bool checkFilters(); }; template String stringifyBinaryExpr(const DOCTEST_REF_WRAP(L) lhs, const char* op, const DOCTEST_REF_WRAP(R) rhs) { - // NOLINTNEXTLINE(clang-analyzer-cplusplus.NewDeleteLeaks) - return doctest::toString(lhs) + op + doctest::toString(rhs); + return (DOCTEST_STRINGIFY(lhs)) + op + (DOCTEST_STRINGIFY(rhs)); } #if DOCTEST_CLANG && DOCTEST_CLANG < DOCTEST_COMPILER(3, 6, 0) @@ -1079,12 +1326,12 @@ DOCTEST_CLANG_SUPPRESS_WARNING_WITH_PUSH("-Wunused-comparison") // If not it doesn't find the operator or if the operator at global scope is defined after // this template, the template won't be instantiated due to SFINAE. Once the template is not // instantiated it can look for global operator using normal conversions. -#define SFINAE_OP(ret,op) decltype((void)(doctest::detail::declval() op doctest::detail::declval()),static_cast(0)) +#define SFINAE_OP(ret,op) decltype((void)(doctest::detail::declval() op doctest::detail::declval()),ret{}) #define DOCTEST_DO_BINARY_EXPRESSION_COMPARISON(op, op_str, op_macro) \ template \ - DOCTEST_NOINLINE SFINAE_OP(Result,op) operator op(R&& rhs) { \ - bool res = op_macro(doctest::detail::forward(lhs), doctest::detail::forward(rhs)); \ + DOCTEST_NOINLINE SFINAE_OP(Result,op) operator op(R&& rhs) { \ + bool res = op_macro(doctest::detail::forward(lhs), doctest::detail::forward(rhs)); \ if(m_at & assertType::is_false) \ res = !res; \ if(!res || doctest::getContextOptions()->success) \ @@ -1103,11 +1350,12 @@ DOCTEST_CLANG_SUPPRESS_WARNING_WITH_PUSH("-Wunused-comparison") return *this; \ } - struct DOCTEST_INTERFACE Result + struct DOCTEST_INTERFACE Result // NOLINT(*-member-init) { bool m_passed; String m_decomp; + Result() = default; // TODO: Why do we need this? (To remove NOLINT) Result(bool passed, const String& decomposition = String()); // forbidding some expressions based on this table: https://en.cppreference.com/w/cpp/language/operator_precedence @@ -1164,8 +1412,7 @@ DOCTEST_CLANG_SUPPRESS_WARNING_WITH_PUSH("-Wunused-comparison") #ifndef DOCTEST_CONFIG_TREAT_CHAR_STAR_AS_STRING #define DOCTEST_COMPARISON_RETURN_TYPE bool #else // DOCTEST_CONFIG_TREAT_CHAR_STAR_AS_STRING -#define DOCTEST_COMPARISON_RETURN_TYPE typename enable_if::value || can_use_op::value, bool>::type - // NOLINTNEXTLINE(clang-analyzer-cplusplus.NewDeleteLeaks) +#define DOCTEST_COMPARISON_RETURN_TYPE typename types::enable_if::value || can_use_op::value, bool>::type inline bool eq(const char* lhs, const char* rhs) { return String(lhs) == String(rhs); } inline bool ne(const char* lhs, const char* rhs) { return String(lhs) != String(rhs); } inline bool lt(const char* lhs, const char* rhs) { return String(lhs) < String(rhs); } @@ -1213,26 +1460,26 @@ DOCTEST_CLANG_SUPPRESS_WARNING_WITH_PUSH("-Wunused-comparison") assertType::Enum m_at; explicit Expression_lhs(L&& in, assertType::Enum at) - : lhs(doctest::detail::forward(in)) + : lhs(static_cast(in)) , m_at(at) {} DOCTEST_NOINLINE operator Result() { -// this is needed only foc MSVC 2015: -// https://ci.appveyor.com/project/onqtam/doctest/builds/38181202 +// this is needed only for MSVC 2015 DOCTEST_MSVC_SUPPRESS_WARNING_WITH_PUSH(4800) // 'int': forcing value to bool bool res = static_cast(lhs); DOCTEST_MSVC_SUPPRESS_WARNING_POP - if(m_at & assertType::is_false) //!OCLINT bitwise operator in conditional + if(m_at & assertType::is_false) { //!OCLINT bitwise operator in conditional res = !res; + } - if(!res || getContextOptions()->success) - return Result(res, doctest::toString(lhs)); - return Result(res); + if(!res || getContextOptions()->success) { + return { res, (DOCTEST_STRINGIFY(lhs)) }; + } + return { res }; } - /* This is required for user-defined conversions from Expression_lhs to L */ - //operator L() const { return lhs; } - operator L() const { return lhs; } + /* This is required for user-defined conversions from Expression_lhs to L */ + operator L() const { return lhs; } // clang-format off DOCTEST_DO_BINARY_EXPRESSION_COMPARISON(==, " == ", DOCTEST_CMP_EQ) //!OCLINT bitwise operator in conditional @@ -1289,22 +1536,27 @@ DOCTEST_CLANG_SUPPRESS_WARNING_POP // https://github.com/catchorg/Catch2/issues/870 // https://github.com/catchorg/Catch2/issues/565 template - Expression_lhs operator<<(L &&operand) { - return Expression_lhs(doctest::detail::forward(operand), m_at); + Expression_lhs operator<<(L&& operand) { + return Expression_lhs(static_cast(operand), m_at); + } + + template ::value,void >::type* = nullptr> + Expression_lhs operator<<(const L &operand) { + return Expression_lhs(operand, m_at); } }; struct DOCTEST_INTERFACE TestSuite { - const char* m_test_suite; - const char* m_description; - bool m_skip; - bool m_no_breaks; - bool m_no_output; - bool m_may_fail; - bool m_should_fail; - int m_expected_failures; - double m_timeout; + const char* m_test_suite = nullptr; + const char* m_description = nullptr; + bool m_skip = false; + bool m_no_breaks = false; + bool m_no_output = false; + bool m_may_fail = false; + bool m_should_fail = false; + int m_expected_failures = 0; + double m_timeout = 0; TestSuite& operator*(const char* in); @@ -1315,25 +1567,28 @@ DOCTEST_CLANG_SUPPRESS_WARNING_POP } }; - typedef void (*funcType)(); + using funcType = void (*)(); struct DOCTEST_INTERFACE TestCase : public TestCaseData { funcType m_test; // a function pointer to the test case - const char* m_type; // for templated test cases - gets appended to the real name + String m_type; // for templated test cases - gets appended to the real name int m_template_id; // an ID used to distinguish between the different versions of a templated test case String m_full_name; // contains the name (only for templated test cases!) + the template type TestCase(funcType test, const char* file, unsigned line, const TestSuite& test_suite, - const char* type = "", int template_id = -1); + const String& type = String(), int template_id = -1); TestCase(const TestCase& other); + TestCase(TestCase&&) = delete; DOCTEST_MSVC_SUPPRESS_WARNING_WITH_PUSH(26434) // hides a non-virtual function TestCase& operator=(const TestCase& other); DOCTEST_MSVC_SUPPRESS_WARNING_POP + TestCase& operator=(TestCase&&) = delete; + TestCase& operator*(const char* in); template @@ -1343,6 +1598,8 @@ DOCTEST_CLANG_SUPPRESS_WARNING_POP } bool operator<(const TestCase& other) const; + + ~TestCase() = default; }; // forward declarations of functions used by the macros @@ -1382,27 +1639,36 @@ DOCTEST_CLANG_SUPPRESS_WARNING_POP struct DOCTEST_INTERFACE ResultBuilder : public AssertData { ResultBuilder(assertType::Enum at, const char* file, int line, const char* expr, - const char* exception_type = "", const char* exception_string = ""); + const char* exception_type = "", const String& exception_string = ""); + + ResultBuilder(assertType::Enum at, const char* file, int line, const char* expr, + const char* exception_type, const Contains& exception_string); void setResult(const Result& res); template - DOCTEST_NOINLINE void binary_assert(const DOCTEST_REF_WRAP(L) lhs, + DOCTEST_NOINLINE bool binary_assert(const DOCTEST_REF_WRAP(L) lhs, const DOCTEST_REF_WRAP(R) rhs) { m_failed = !RelationalComparator()(lhs, rhs); - if(m_failed || getContextOptions()->success) + if (m_failed || getContextOptions()->success) { m_decomp = stringifyBinaryExpr(lhs, ", ", rhs); + } + return !m_failed; } template - DOCTEST_NOINLINE void unary_assert(const DOCTEST_REF_WRAP(L) val) { + DOCTEST_NOINLINE bool unary_assert(const DOCTEST_REF_WRAP(L) val) { m_failed = !val; - if(m_at & assertType::is_false) //!OCLINT bitwise operator in conditional + if (m_at & assertType::is_false) { //!OCLINT bitwise operator in conditional m_failed = !m_failed; + } - if(m_failed || getContextOptions()->success) - m_decomp = toString(val); + if (m_failed || getContextOptions()->success) { + m_decomp = (DOCTEST_STRINGIFY(val)); + } + + return !m_failed; } void translateException(); @@ -1422,8 +1688,8 @@ DOCTEST_CLANG_SUPPRESS_WARNING_POP DOCTEST_INTERFACE void failed_out_of_a_testing_context(const AssertData& ad); - DOCTEST_INTERFACE void decomp_assert(assertType::Enum at, const char* file, int line, - const char* expr, Result result); + DOCTEST_INTERFACE bool decomp_assert(assertType::Enum at, const char* file, int line, + const char* expr, const Result& result); #define DOCTEST_ASSERT_OUT_OF_TESTS(decomp) \ do { \ @@ -1438,7 +1704,7 @@ DOCTEST_CLANG_SUPPRESS_WARNING_POP if(checkIfShouldThrow(at)) \ throwException(); \ } \ - return; \ + return !failed; \ } \ } while(false) @@ -1453,7 +1719,7 @@ DOCTEST_CLANG_SUPPRESS_WARNING_POP throwException() template - DOCTEST_NOINLINE void binary_assert(assertType::Enum at, const char* file, int line, + DOCTEST_NOINLINE bool binary_assert(assertType::Enum at, const char* file, int line, const char* expr, const DOCTEST_REF_WRAP(L) lhs, const DOCTEST_REF_WRAP(R) rhs) { bool failed = !RelationalComparator()(lhs, rhs); @@ -1464,10 +1730,11 @@ DOCTEST_CLANG_SUPPRESS_WARNING_POP // ################################################################################### DOCTEST_ASSERT_OUT_OF_TESTS(stringifyBinaryExpr(lhs, ", ", rhs)); DOCTEST_ASSERT_IN_TESTS(stringifyBinaryExpr(lhs, ", ", rhs)); + return !failed; } template - DOCTEST_NOINLINE void unary_assert(assertType::Enum at, const char* file, int line, + DOCTEST_NOINLINE bool unary_assert(assertType::Enum at, const char* file, int line, const char* expr, const DOCTEST_REF_WRAP(L) val) { bool failed = !val; @@ -1478,14 +1745,14 @@ DOCTEST_CLANG_SUPPRESS_WARNING_POP // IF THE DEBUGGER BREAKS HERE - GO 1 LEVEL UP IN THE CALLSTACK FOR THE FAILING ASSERT // THIS IS THE EFFECT OF HAVING 'DOCTEST_CONFIG_SUPER_FAST_ASSERTS' DEFINED // ################################################################################### - DOCTEST_ASSERT_OUT_OF_TESTS(toString(val)); - DOCTEST_ASSERT_IN_TESTS(toString(val)); + DOCTEST_ASSERT_OUT_OF_TESTS((DOCTEST_STRINGIFY(val))); + DOCTEST_ASSERT_IN_TESTS((DOCTEST_STRINGIFY(val))); + return !failed; } struct DOCTEST_INTERFACE IExceptionTranslator { - IExceptionTranslator(); - virtual ~IExceptionTranslator(); + DOCTEST_DECLARE_INTERFACE(IExceptionTranslator) virtual bool translate(String&) const = 0; }; @@ -1501,7 +1768,7 @@ DOCTEST_CLANG_SUPPRESS_WARNING_POP try { throw; // lgtm [cpp/rethrow-no-exception] // cppcheck-suppress catchExceptionByValue - } catch(T ex) { // NOLINT + } catch(const T& ex) { res = m_translateFunction(ex); //!OCLINT parameter reassignment return true; } catch(...) {} //!OCLINT - empty catch statement @@ -1516,95 +1783,70 @@ DOCTEST_CLANG_SUPPRESS_WARNING_POP DOCTEST_INTERFACE void registerExceptionTranslatorImpl(const IExceptionTranslator* et); - template - struct StringStreamBase - { - template - static void convert(std::ostream* s, const T& in) { - *s << toString(in); - } - - // always treat char* as a string in this context - no matter - // if DOCTEST_CONFIG_TREAT_CHAR_STAR_AS_STRING is defined - static void convert(std::ostream* s, const char* in) { *s << String(in); } - }; - - template <> - struct StringStreamBase - { - template - static void convert(std::ostream* s, const T& in) { - *s << in; - } - }; - - template - struct StringStream : public StringStreamBase::value> - {}; - - template - void toStream(std::ostream* s, const T& value) { - StringStream::convert(s, value); - } - -#ifdef DOCTEST_CONFIG_TREAT_CHAR_STAR_AS_STRING - DOCTEST_INTERFACE void toStream(std::ostream* s, char* in); - DOCTEST_INTERFACE void toStream(std::ostream* s, const char* in); -#endif // DOCTEST_CONFIG_TREAT_CHAR_STAR_AS_STRING - DOCTEST_INTERFACE void toStream(std::ostream* s, bool in); - DOCTEST_INTERFACE void toStream(std::ostream* s, float in); - DOCTEST_INTERFACE void toStream(std::ostream* s, double in); - DOCTEST_INTERFACE void toStream(std::ostream* s, double long in); - - DOCTEST_INTERFACE void toStream(std::ostream* s, char in); - DOCTEST_INTERFACE void toStream(std::ostream* s, char signed in); - DOCTEST_INTERFACE void toStream(std::ostream* s, char unsigned in); - DOCTEST_INTERFACE void toStream(std::ostream* s, int short in); - DOCTEST_INTERFACE void toStream(std::ostream* s, int short unsigned in); - DOCTEST_INTERFACE void toStream(std::ostream* s, int in); - DOCTEST_INTERFACE void toStream(std::ostream* s, int unsigned in); - DOCTEST_INTERFACE void toStream(std::ostream* s, int long in); - DOCTEST_INTERFACE void toStream(std::ostream* s, int long unsigned in); - DOCTEST_INTERFACE void toStream(std::ostream* s, int long long in); - DOCTEST_INTERFACE void toStream(std::ostream* s, int long long unsigned in); - - // ContextScope base class used to allow implementing methods of ContextScope + // ContextScope base class used to allow implementing methods of ContextScope // that don't depend on the template parameter in doctest.cpp. - class DOCTEST_INTERFACE ContextScopeBase : public IContextScope { + struct DOCTEST_INTERFACE ContextScopeBase : public IContextScope { + ContextScopeBase(const ContextScopeBase&) = delete; + + ContextScopeBase& operator=(const ContextScopeBase&) = delete; + ContextScopeBase& operator=(ContextScopeBase&&) = delete; + + ~ContextScopeBase() override = default; + protected: ContextScopeBase(); + ContextScopeBase(ContextScopeBase&& other) noexcept; void destroy(); + bool need_to_destroy{true}; }; template class ContextScope : public ContextScopeBase { - const L lambda_; + L lambda_; public: explicit ContextScope(const L &lambda) : lambda_(lambda) {} + explicit ContextScope(L&& lambda) : lambda_(static_cast(lambda)) { } - ContextScope(ContextScope &&other) : lambda_(other.lambda_) {} + ContextScope(const ContextScope&) = delete; + ContextScope(ContextScope&&) noexcept = default; + + ContextScope& operator=(const ContextScope&) = delete; + ContextScope& operator=(ContextScope&&) = delete; void stringify(std::ostream* s) const override { lambda_(s); } - ~ContextScope() override { destroy(); } + ~ContextScope() override { + if (need_to_destroy) { + destroy(); + } + } }; struct DOCTEST_INTERFACE MessageBuilder : public MessageData { std::ostream* m_stream; + bool logged = false; MessageBuilder(const char* file, int line, assertType::Enum severity); - MessageBuilder() = delete; + + MessageBuilder(const MessageBuilder&) = delete; + MessageBuilder(MessageBuilder&&) = delete; + + MessageBuilder& operator=(const MessageBuilder&) = delete; + MessageBuilder& operator=(MessageBuilder&&) = delete; + ~MessageBuilder(); // the preferred way of chaining parameters for stringification +DOCTEST_MSVC_SUPPRESS_WARNING_WITH_PUSH(4866) template MessageBuilder& operator,(const T& in) { - toStream(m_stream, in); + *m_stream << (DOCTEST_STRINGIFY(in)); return *this; } +DOCTEST_MSVC_SUPPRESS_WARNING_POP // kept here just for backwards-compatibility - the comma operator should be preferred now template @@ -1620,7 +1862,7 @@ DOCTEST_CLANG_SUPPRESS_WARNING_POP bool log(); void react(); }; - + template ContextScope MakeContextScope(const L &lambda) { return ContextScope(lambda); @@ -1673,7 +1915,7 @@ int registerExceptionTranslator(String (*)(T)) { #endif // DOCTEST_CONFIG_DISABLE namespace detail { - typedef void (*assert_handler)(const AssertData&); + using assert_handler = void (*)(const AssertData&); struct ContextState; } // namespace detail @@ -1686,12 +1928,19 @@ class DOCTEST_INTERFACE Context public: explicit Context(int argc = 0, const char* const* argv = nullptr); - ~Context(); + Context(const Context&) = delete; + Context(Context&&) = delete; + + Context& operator=(const Context&) = delete; + Context& operator=(Context&&) = delete; + + ~Context(); // NOLINT(performance-trivially-destructible) void applyCommandLine(int argc, const char* const* argv); void addFilter(const char* filter, const char* value); void clearFilters(); + void setOption(const char* option, bool value); void setOption(const char* option, int value); void setOption(const char* option, const char* value); @@ -1701,6 +1950,8 @@ public: void setAssertHandler(detail::assert_handler ah); + void setCout(std::ostream* out); + int run(); }; @@ -1727,6 +1978,7 @@ struct DOCTEST_INTERFACE CurrentTestCaseStats int numAssertsFailedCurrentTest; double seconds; int failure_flags; // use TestCaseFailureReason::Enum + bool testCaseSuccess; }; struct DOCTEST_INTERFACE TestCaseException @@ -1790,8 +2042,7 @@ struct DOCTEST_INTERFACE IReporter // or isn't in the execution range (between first and last) (safe to cache a pointer to the input) virtual void test_case_skipped(const TestCaseData&) = 0; - // doctest will not be managing the lifetimes of reporters given to it but this would still be nice to have - virtual ~IReporter(); + DOCTEST_DECLARE_INTERFACE(IReporter) // can obtain all currently active contexts and stringify them if one wishes to do so static int get_num_active_contexts(); @@ -1803,7 +2054,7 @@ struct DOCTEST_INTERFACE IReporter }; namespace detail { - typedef IReporter* (*reporterCreatorFunc)(const ContextOptions&); + using reporterCreatorFunc = IReporter* (*)(const ContextOptions&); DOCTEST_INTERFACE void registerReporterImpl(const char* name, int prio, reporterCreatorFunc c, bool isReporter); @@ -1820,14 +2071,30 @@ int registerReporter(const char* name, int priority, bool isReporter) { } } // namespace doctest +#ifdef DOCTEST_CONFIG_ASSERTS_RETURN_VALUES +#define DOCTEST_FUNC_EMPTY [] { return false; }() +#else +#define DOCTEST_FUNC_EMPTY (void)0 +#endif + // if registering is not disabled -#if !defined(DOCTEST_CONFIG_DISABLE) +#ifndef DOCTEST_CONFIG_DISABLE + +#ifdef DOCTEST_CONFIG_ASSERTS_RETURN_VALUES +#define DOCTEST_FUNC_SCOPE_BEGIN [&] +#define DOCTEST_FUNC_SCOPE_END () +#define DOCTEST_FUNC_SCOPE_RET(v) return v +#else +#define DOCTEST_FUNC_SCOPE_BEGIN do +#define DOCTEST_FUNC_SCOPE_END while(false) +#define DOCTEST_FUNC_SCOPE_RET(v) (void)0 +#endif // common code in asserts - for convenience -#define DOCTEST_ASSERT_LOG_AND_REACT(b) \ - if(b.log()) \ - DOCTEST_BREAK_INTO_DEBUGGER(); \ - b.react() +#define DOCTEST_ASSERT_LOG_REACT_RETURN(b) \ + if(b.log()) DOCTEST_BREAK_INTO_DEBUGGER(); \ + b.react(); \ + DOCTEST_FUNC_SCOPE_RET(!b.m_failed) #ifdef DOCTEST_CONFIG_NO_TRY_CATCH_IN_ASSERTS #define DOCTEST_WRAP_IN_TRY(x) x; @@ -1835,7 +2102,7 @@ int registerReporter(const char* name, int priority, bool isReporter) { #define DOCTEST_WRAP_IN_TRY(x) \ try { \ x; \ - } catch(...) { _DOCTEST_RB.translateException(); } + } catch(...) { DOCTEST_RB.translateException(); } #endif // DOCTEST_CONFIG_NO_TRY_CATCH_IN_ASSERTS #ifdef DOCTEST_CONFIG_VOID_CAST_EXPRESSIONS @@ -1849,27 +2116,26 @@ int registerReporter(const char* name, int priority, bool isReporter) { // registers the test by initializing a dummy var with a function #define DOCTEST_REGISTER_FUNCTION(global_prefix, f, decorators) \ - global_prefix DOCTEST_GLOBAL_NO_WARNINGS(DOCTEST_ANONYMOUS(_DOCTEST_ANON_VAR_)) = \ + global_prefix DOCTEST_GLOBAL_NO_WARNINGS(DOCTEST_ANONYMOUS(DOCTEST_ANON_VAR_), /* NOLINT */ \ doctest::detail::regTest( \ doctest::detail::TestCase( \ f, __FILE__, __LINE__, \ doctest_detail_test_suite_ns::getCurrentTestSuite()) * \ - decorators); \ - DOCTEST_GLOBAL_NO_WARNINGS_END() + decorators)) #define DOCTEST_IMPLEMENT_FIXTURE(der, base, func, decorators) \ - namespace { \ + namespace { /* NOLINT */ \ struct der : public base \ { \ void f(); \ }; \ - static void func() { \ + static inline DOCTEST_NOINLINE void func() { \ der v; \ v.f(); \ } \ DOCTEST_REGISTER_FUNCTION(DOCTEST_EMPTY, func, decorators) \ } \ - inline DOCTEST_NOINLINE void der::f() + inline DOCTEST_NOINLINE void der::f() // NOLINT(misc-definitions-in-headers) #define DOCTEST_CREATE_AND_REGISTER_FUNCTION(f, decorators) \ static void f(); \ @@ -1878,18 +2144,18 @@ int registerReporter(const char* name, int priority, bool isReporter) { #define DOCTEST_CREATE_AND_REGISTER_FUNCTION_IN_CLASS(f, proxy, decorators) \ static doctest::detail::funcType proxy() { return f; } \ - DOCTEST_REGISTER_FUNCTION(inline const, proxy(), decorators) \ + DOCTEST_REGISTER_FUNCTION(inline, proxy(), decorators) \ static void f() // for registering tests #define DOCTEST_TEST_CASE(decorators) \ - DOCTEST_CREATE_AND_REGISTER_FUNCTION(DOCTEST_ANONYMOUS(_DOCTEST_ANON_FUNC_), decorators) + DOCTEST_CREATE_AND_REGISTER_FUNCTION(DOCTEST_ANONYMOUS(DOCTEST_ANON_FUNC_), decorators) // for registering tests in classes - requires C++17 for inline variables! -#if __cplusplus >= 201703L || (DOCTEST_MSVC >= DOCTEST_COMPILER(19, 12, 0) && _MSVC_LANG >= 201703L) +#if DOCTEST_CPLUSPLUS >= 201703L #define DOCTEST_TEST_CASE_CLASS(decorators) \ - DOCTEST_CREATE_AND_REGISTER_FUNCTION_IN_CLASS(DOCTEST_ANONYMOUS(_DOCTEST_ANON_FUNC_), \ - DOCTEST_ANONYMOUS(_DOCTEST_ANON_PROXY_), \ + DOCTEST_CREATE_AND_REGISTER_FUNCTION_IN_CLASS(DOCTEST_ANONYMOUS(DOCTEST_ANON_FUNC_), \ + DOCTEST_ANONYMOUS(DOCTEST_ANON_PROXY_), \ decorators) #else // DOCTEST_TEST_CASE_CLASS #define DOCTEST_TEST_CASE_CLASS(...) \ @@ -1898,26 +2164,25 @@ int registerReporter(const char* name, int priority, bool isReporter) { // for registering tests with a fixture #define DOCTEST_TEST_CASE_FIXTURE(c, decorators) \ - DOCTEST_IMPLEMENT_FIXTURE(DOCTEST_ANONYMOUS(_DOCTEST_ANON_CLASS_), c, \ - DOCTEST_ANONYMOUS(_DOCTEST_ANON_FUNC_), decorators) + DOCTEST_IMPLEMENT_FIXTURE(DOCTEST_ANONYMOUS(DOCTEST_ANON_CLASS_), c, \ + DOCTEST_ANONYMOUS(DOCTEST_ANON_FUNC_), decorators) // for converting types to strings without the header and demangling -#define DOCTEST_TYPE_TO_STRING_IMPL(...) \ - template <> \ - inline const char* type_to_string<__VA_ARGS__>() { \ - return "<" #__VA_ARGS__ ">"; \ - } -#define DOCTEST_TYPE_TO_STRING(...) \ - namespace doctest { namespace detail { \ - DOCTEST_TYPE_TO_STRING_IMPL(__VA_ARGS__) \ +#define DOCTEST_TYPE_TO_STRING_AS(str, ...) \ + namespace doctest { \ + template <> \ + inline String toString<__VA_ARGS__>() { \ + return str; \ } \ } \ - typedef int DOCTEST_ANONYMOUS(_DOCTEST_ANON_FOR_SEMICOLON_) + static_assert(true, "") + +#define DOCTEST_TYPE_TO_STRING(...) DOCTEST_TYPE_TO_STRING_AS(#__VA_ARGS__, __VA_ARGS__) #define DOCTEST_TEST_CASE_TEMPLATE_DEFINE_IMPL(dec, T, iter, func) \ template \ static void func(); \ - namespace { \ + namespace { /* NOLINT */ \ template \ struct iter; \ template \ @@ -1926,7 +2191,7 @@ int registerReporter(const char* name, int priority, bool isReporter) { iter(const char* file, unsigned line, int index) { \ doctest::detail::regTest(doctest::detail::TestCase(func, file, line, \ doctest_detail_test_suite_ns::getCurrentTestSuite(), \ - doctest::detail::type_to_string(), \ + doctest::toString(), \ int(line) * 1000 + index) \ * dec); \ iter>(file, line, index + 1); \ @@ -1943,20 +2208,20 @@ int registerReporter(const char* name, int priority, bool isReporter) { #define DOCTEST_TEST_CASE_TEMPLATE_DEFINE(dec, T, id) \ DOCTEST_TEST_CASE_TEMPLATE_DEFINE_IMPL(dec, T, DOCTEST_CAT(id, ITERATOR), \ - DOCTEST_ANONYMOUS(_DOCTEST_ANON_TMP_)) + DOCTEST_ANONYMOUS(DOCTEST_ANON_TMP_)) #define DOCTEST_TEST_CASE_TEMPLATE_INSTANTIATE_IMPL(id, anon, ...) \ - DOCTEST_GLOBAL_NO_WARNINGS(DOCTEST_CAT(anon, DUMMY)) = \ - doctest::detail::instantiationHelper(DOCTEST_CAT(id, ITERATOR)<__VA_ARGS__>(__FILE__, __LINE__, 0));\ - DOCTEST_GLOBAL_NO_WARNINGS_END() + DOCTEST_GLOBAL_NO_WARNINGS(DOCTEST_CAT(anon, DUMMY), /* NOLINT(cert-err58-cpp, fuchsia-statically-constructed-objects) */ \ + doctest::detail::instantiationHelper( \ + DOCTEST_CAT(id, ITERATOR)<__VA_ARGS__>(__FILE__, __LINE__, 0))) #define DOCTEST_TEST_CASE_TEMPLATE_INVOKE(id, ...) \ - DOCTEST_TEST_CASE_TEMPLATE_INSTANTIATE_IMPL(id, DOCTEST_ANONYMOUS(_DOCTEST_ANON_TMP_), std::tuple<__VA_ARGS__>) \ - typedef int DOCTEST_ANONYMOUS(_DOCTEST_ANON_FOR_SEMICOLON_) + DOCTEST_TEST_CASE_TEMPLATE_INSTANTIATE_IMPL(id, DOCTEST_ANONYMOUS(DOCTEST_ANON_TMP_), std::tuple<__VA_ARGS__>) \ + static_assert(true, "") #define DOCTEST_TEST_CASE_TEMPLATE_APPLY(id, ...) \ - DOCTEST_TEST_CASE_TEMPLATE_INSTANTIATE_IMPL(id, DOCTEST_ANONYMOUS(_DOCTEST_ANON_TMP_), __VA_ARGS__) \ - typedef int DOCTEST_ANONYMOUS(_DOCTEST_ANON_FOR_SEMICOLON_) + DOCTEST_TEST_CASE_TEMPLATE_INSTANTIATE_IMPL(id, DOCTEST_ANONYMOUS(DOCTEST_ANON_TMP_), __VA_ARGS__) \ + static_assert(true, "") #define DOCTEST_TEST_CASE_TEMPLATE_IMPL(dec, T, anon, ...) \ DOCTEST_TEST_CASE_TEMPLATE_DEFINE_IMPL(dec, T, DOCTEST_CAT(anon, ITERATOR), anon); \ @@ -1965,17 +2230,17 @@ int registerReporter(const char* name, int priority, bool isReporter) { static void anon() #define DOCTEST_TEST_CASE_TEMPLATE(dec, T, ...) \ - DOCTEST_TEST_CASE_TEMPLATE_IMPL(dec, T, DOCTEST_ANONYMOUS(_DOCTEST_ANON_TMP_), __VA_ARGS__) + DOCTEST_TEST_CASE_TEMPLATE_IMPL(dec, T, DOCTEST_ANONYMOUS(DOCTEST_ANON_TMP_), __VA_ARGS__) // for subcases #define DOCTEST_SUBCASE(name) \ - if(const doctest::detail::Subcase & DOCTEST_ANONYMOUS(_DOCTEST_ANON_SUBCASE_) DOCTEST_UNUSED = \ + if(const doctest::detail::Subcase & DOCTEST_ANONYMOUS(DOCTEST_ANON_SUBCASE_) DOCTEST_UNUSED = \ doctest::detail::Subcase(name, __FILE__, __LINE__)) // for grouping tests in test suites by using code blocks #define DOCTEST_TEST_SUITE_IMPL(decorators, ns_name) \ namespace ns_name { namespace doctest_detail_test_suite_ns { \ - static DOCTEST_NOINLINE doctest::detail::TestSuite& getCurrentTestSuite() { \ + static DOCTEST_NOINLINE doctest::detail::TestSuite& getCurrentTestSuite() noexcept { \ DOCTEST_MSVC_SUPPRESS_WARNING_WITH_PUSH(4640) \ DOCTEST_CLANG_SUPPRESS_WARNING_WITH_PUSH("-Wexit-time-destructors") \ DOCTEST_GCC_SUPPRESS_WARNING_WITH_PUSH("-Wmissing-field-initializers") \ @@ -1995,53 +2260,53 @@ int registerReporter(const char* name, int priority, bool isReporter) { namespace ns_name #define DOCTEST_TEST_SUITE(decorators) \ - DOCTEST_TEST_SUITE_IMPL(decorators, DOCTEST_ANONYMOUS(_DOCTEST_ANON_SUITE_)) + DOCTEST_TEST_SUITE_IMPL(decorators, DOCTEST_ANONYMOUS(DOCTEST_ANON_SUITE_)) // for starting a testsuite block #define DOCTEST_TEST_SUITE_BEGIN(decorators) \ - DOCTEST_GLOBAL_NO_WARNINGS(DOCTEST_ANONYMOUS(_DOCTEST_ANON_VAR_)) = \ - doctest::detail::setTestSuite(doctest::detail::TestSuite() * decorators); \ - DOCTEST_GLOBAL_NO_WARNINGS_END() \ - typedef int DOCTEST_ANONYMOUS(_DOCTEST_ANON_FOR_SEMICOLON_) + DOCTEST_GLOBAL_NO_WARNINGS(DOCTEST_ANONYMOUS(DOCTEST_ANON_VAR_), /* NOLINT(cert-err58-cpp) */ \ + doctest::detail::setTestSuite(doctest::detail::TestSuite() * decorators)) \ + static_assert(true, "") // for ending a testsuite block #define DOCTEST_TEST_SUITE_END \ - DOCTEST_GLOBAL_NO_WARNINGS(DOCTEST_ANONYMOUS(_DOCTEST_ANON_VAR_)) = \ - doctest::detail::setTestSuite(doctest::detail::TestSuite() * ""); \ - DOCTEST_GLOBAL_NO_WARNINGS_END() \ - typedef int DOCTEST_ANONYMOUS(_DOCTEST_ANON_FOR_SEMICOLON_) + DOCTEST_GLOBAL_NO_WARNINGS(DOCTEST_ANONYMOUS(DOCTEST_ANON_VAR_), /* NOLINT(cert-err58-cpp) */ \ + doctest::detail::setTestSuite(doctest::detail::TestSuite() * "")) \ + using DOCTEST_ANONYMOUS(DOCTEST_ANON_FOR_SEMICOLON_) = int // for registering exception translators #define DOCTEST_REGISTER_EXCEPTION_TRANSLATOR_IMPL(translatorName, signature) \ inline doctest::String translatorName(signature); \ - DOCTEST_GLOBAL_NO_WARNINGS(DOCTEST_ANONYMOUS(_DOCTEST_ANON_TRANSLATOR_)) = \ - doctest::registerExceptionTranslator(translatorName); \ - DOCTEST_GLOBAL_NO_WARNINGS_END() \ + DOCTEST_GLOBAL_NO_WARNINGS(DOCTEST_ANONYMOUS(DOCTEST_ANON_TRANSLATOR_), /* NOLINT(cert-err58-cpp) */ \ + doctest::registerExceptionTranslator(translatorName)) \ doctest::String translatorName(signature) #define DOCTEST_REGISTER_EXCEPTION_TRANSLATOR(signature) \ - DOCTEST_REGISTER_EXCEPTION_TRANSLATOR_IMPL(DOCTEST_ANONYMOUS(_DOCTEST_ANON_TRANSLATOR_), \ + DOCTEST_REGISTER_EXCEPTION_TRANSLATOR_IMPL(DOCTEST_ANONYMOUS(DOCTEST_ANON_TRANSLATOR_), \ signature) // for registering reporters #define DOCTEST_REGISTER_REPORTER(name, priority, reporter) \ - DOCTEST_GLOBAL_NO_WARNINGS(DOCTEST_ANONYMOUS(_DOCTEST_ANON_REPORTER_)) = \ - doctest::registerReporter(name, priority, true); \ - DOCTEST_GLOBAL_NO_WARNINGS_END() typedef int DOCTEST_ANONYMOUS(_DOCTEST_ANON_FOR_SEMICOLON_) + DOCTEST_GLOBAL_NO_WARNINGS(DOCTEST_ANONYMOUS(DOCTEST_ANON_REPORTER_), /* NOLINT(cert-err58-cpp) */ \ + doctest::registerReporter(name, priority, true)) \ + static_assert(true, "") // for registering listeners #define DOCTEST_REGISTER_LISTENER(name, priority, reporter) \ - DOCTEST_GLOBAL_NO_WARNINGS(DOCTEST_ANONYMOUS(_DOCTEST_ANON_REPORTER_)) = \ - doctest::registerReporter(name, priority, false); \ - DOCTEST_GLOBAL_NO_WARNINGS_END() typedef int DOCTEST_ANONYMOUS(_DOCTEST_ANON_FOR_SEMICOLON_) + DOCTEST_GLOBAL_NO_WARNINGS(DOCTEST_ANONYMOUS(DOCTEST_ANON_REPORTER_), /* NOLINT(cert-err58-cpp) */ \ + doctest::registerReporter(name, priority, false)) \ + static_assert(true, "") -// for logging +// clang-format off +// for logging - disabling formatting because it's important to have these on 2 separate lines - see PR #557 #define DOCTEST_INFO(...) \ - DOCTEST_INFO_IMPL(DOCTEST_ANONYMOUS(_DOCTEST_CAPTURE_), DOCTEST_ANONYMOUS(_DOCTEST_CAPTURE_), \ + DOCTEST_INFO_IMPL(DOCTEST_ANONYMOUS(DOCTEST_CAPTURE_), \ + DOCTEST_ANONYMOUS(DOCTEST_CAPTURE_OTHER_), \ __VA_ARGS__) +// clang-format on #define DOCTEST_INFO_IMPL(mb_name, s_name, ...) \ - auto DOCTEST_ANONYMOUS(_DOCTEST_CAPTURE_) = doctest::detail::MakeContextScope( \ + auto DOCTEST_ANONYMOUS(DOCTEST_CAPTURE_) = doctest::detail::MakeContextScope( \ [&](std::ostream* s_name) { \ doctest::detail::MessageBuilder mb_name(__FILE__, __LINE__, doctest::assertType::is_warn); \ mb_name.m_stream = s_name; \ @@ -2051,16 +2316,18 @@ int registerReporter(const char* name, int priority, bool isReporter) { #define DOCTEST_CAPTURE(x) DOCTEST_INFO(#x " := ", x) #define DOCTEST_ADD_AT_IMPL(type, file, line, mb, ...) \ - do { \ + DOCTEST_FUNC_SCOPE_BEGIN { \ doctest::detail::MessageBuilder mb(file, line, doctest::assertType::type); \ mb * __VA_ARGS__; \ - DOCTEST_ASSERT_LOG_AND_REACT(mb); \ - } while(false) + if(mb.log()) \ + DOCTEST_BREAK_INTO_DEBUGGER(); \ + mb.react(); \ + } DOCTEST_FUNC_SCOPE_END // clang-format off -#define DOCTEST_ADD_MESSAGE_AT(file, line, ...) DOCTEST_ADD_AT_IMPL(is_warn, file, line, DOCTEST_ANONYMOUS(_DOCTEST_MESSAGE_), __VA_ARGS__) -#define DOCTEST_ADD_FAIL_CHECK_AT(file, line, ...) DOCTEST_ADD_AT_IMPL(is_check, file, line, DOCTEST_ANONYMOUS(_DOCTEST_MESSAGE_), __VA_ARGS__) -#define DOCTEST_ADD_FAIL_AT(file, line, ...) DOCTEST_ADD_AT_IMPL(is_require, file, line, DOCTEST_ANONYMOUS(_DOCTEST_MESSAGE_), __VA_ARGS__) +#define DOCTEST_ADD_MESSAGE_AT(file, line, ...) DOCTEST_ADD_AT_IMPL(is_warn, file, line, DOCTEST_ANONYMOUS(DOCTEST_MESSAGE_), __VA_ARGS__) +#define DOCTEST_ADD_FAIL_CHECK_AT(file, line, ...) DOCTEST_ADD_AT_IMPL(is_check, file, line, DOCTEST_ANONYMOUS(DOCTEST_MESSAGE_), __VA_ARGS__) +#define DOCTEST_ADD_FAIL_AT(file, line, ...) DOCTEST_ADD_AT_IMPL(is_require, file, line, DOCTEST_ANONYMOUS(DOCTEST_MESSAGE_), __VA_ARGS__) // clang-format on #define DOCTEST_MESSAGE(...) DOCTEST_ADD_MESSAGE_AT(__FILE__, __LINE__, __VA_ARGS__) @@ -2073,18 +2340,37 @@ int registerReporter(const char* name, int priority, bool isReporter) { #define DOCTEST_ASSERT_IMPLEMENT_2(assert_type, ...) \ DOCTEST_CLANG_SUPPRESS_WARNING_WITH_PUSH("-Woverloaded-shift-op-parentheses") \ - doctest::detail::ResultBuilder _DOCTEST_RB(doctest::assertType::assert_type, __FILE__, \ + /* NOLINTNEXTLINE(clang-analyzer-cplusplus.NewDeleteLeaks) */ \ + doctest::detail::ResultBuilder DOCTEST_RB(doctest::assertType::assert_type, __FILE__, \ __LINE__, #__VA_ARGS__); \ - DOCTEST_WRAP_IN_TRY(_DOCTEST_RB.setResult( \ + DOCTEST_WRAP_IN_TRY(DOCTEST_RB.setResult( \ doctest::detail::ExpressionDecomposer(doctest::assertType::assert_type) \ - << __VA_ARGS__)) \ - DOCTEST_ASSERT_LOG_AND_REACT(_DOCTEST_RB) \ + << __VA_ARGS__)) /* NOLINTNEXTLINE(clang-analyzer-cplusplus.NewDeleteLeaks) */ \ + DOCTEST_ASSERT_LOG_REACT_RETURN(DOCTEST_RB) \ DOCTEST_CLANG_SUPPRESS_WARNING_POP #define DOCTEST_ASSERT_IMPLEMENT_1(assert_type, ...) \ - do { \ + DOCTEST_FUNC_SCOPE_BEGIN { \ DOCTEST_ASSERT_IMPLEMENT_2(assert_type, __VA_ARGS__); \ - } while(false) + } DOCTEST_FUNC_SCOPE_END // NOLINT(clang-analyzer-cplusplus.NewDeleteLeaks) + +#define DOCTEST_BINARY_ASSERT(assert_type, comp, ...) \ + DOCTEST_FUNC_SCOPE_BEGIN { \ + doctest::detail::ResultBuilder DOCTEST_RB(doctest::assertType::assert_type, __FILE__, \ + __LINE__, #__VA_ARGS__); \ + DOCTEST_WRAP_IN_TRY( \ + DOCTEST_RB.binary_assert( \ + __VA_ARGS__)) \ + DOCTEST_ASSERT_LOG_REACT_RETURN(DOCTEST_RB); \ + } DOCTEST_FUNC_SCOPE_END + +#define DOCTEST_UNARY_ASSERT(assert_type, ...) \ + DOCTEST_FUNC_SCOPE_BEGIN { \ + doctest::detail::ResultBuilder DOCTEST_RB(doctest::assertType::assert_type, __FILE__, \ + __LINE__, #__VA_ARGS__); \ + DOCTEST_WRAP_IN_TRY(DOCTEST_RB.unary_assert(__VA_ARGS__)) \ + DOCTEST_ASSERT_LOG_REACT_RETURN(DOCTEST_RB); \ + } DOCTEST_FUNC_SCOPE_END #else // DOCTEST_CONFIG_SUPER_FAST_ASSERTS @@ -2098,6 +2384,14 @@ int registerReporter(const char* name, int priority, bool isReporter) { doctest::detail::ExpressionDecomposer(doctest::assertType::assert_type) \ << __VA_ARGS__) DOCTEST_CLANG_SUPPRESS_WARNING_POP +#define DOCTEST_BINARY_ASSERT(assert_type, comparison, ...) \ + doctest::detail::binary_assert( \ + doctest::assertType::assert_type, __FILE__, __LINE__, #__VA_ARGS__, __VA_ARGS__) + +#define DOCTEST_UNARY_ASSERT(assert_type, ...) \ + doctest::detail::unary_assert(doctest::assertType::assert_type, __FILE__, __LINE__, \ + #__VA_ARGS__, __VA_ARGS__) + #endif // DOCTEST_CONFIG_SUPER_FAST_ASSERTS #define DOCTEST_WARN(...) DOCTEST_ASSERT_IMPLEMENT_1(DT_WARN, __VA_ARGS__) @@ -2108,122 +2402,14 @@ int registerReporter(const char* name, int priority, bool isReporter) { #define DOCTEST_REQUIRE_FALSE(...) DOCTEST_ASSERT_IMPLEMENT_1(DT_REQUIRE_FALSE, __VA_ARGS__) // clang-format off -#define DOCTEST_WARN_MESSAGE(cond, ...) do { DOCTEST_INFO(__VA_ARGS__); DOCTEST_ASSERT_IMPLEMENT_2(DT_WARN, cond); } while(false) -#define DOCTEST_CHECK_MESSAGE(cond, ...) do { DOCTEST_INFO(__VA_ARGS__); DOCTEST_ASSERT_IMPLEMENT_2(DT_CHECK, cond); } while(false) -#define DOCTEST_REQUIRE_MESSAGE(cond, ...) do { DOCTEST_INFO(__VA_ARGS__); DOCTEST_ASSERT_IMPLEMENT_2(DT_REQUIRE, cond); } while(false) -#define DOCTEST_WARN_FALSE_MESSAGE(cond, ...) do { DOCTEST_INFO(__VA_ARGS__); DOCTEST_ASSERT_IMPLEMENT_2(DT_WARN_FALSE, cond); } while(false) -#define DOCTEST_CHECK_FALSE_MESSAGE(cond, ...) do { DOCTEST_INFO(__VA_ARGS__); DOCTEST_ASSERT_IMPLEMENT_2(DT_CHECK_FALSE, cond); } while(false) -#define DOCTEST_REQUIRE_FALSE_MESSAGE(cond, ...) do { DOCTEST_INFO(__VA_ARGS__); DOCTEST_ASSERT_IMPLEMENT_2(DT_REQUIRE_FALSE, cond); } while(false) +#define DOCTEST_WARN_MESSAGE(cond, ...) DOCTEST_FUNC_SCOPE_BEGIN { DOCTEST_INFO(__VA_ARGS__); DOCTEST_ASSERT_IMPLEMENT_2(DT_WARN, cond); } DOCTEST_FUNC_SCOPE_END +#define DOCTEST_CHECK_MESSAGE(cond, ...) DOCTEST_FUNC_SCOPE_BEGIN { DOCTEST_INFO(__VA_ARGS__); DOCTEST_ASSERT_IMPLEMENT_2(DT_CHECK, cond); } DOCTEST_FUNC_SCOPE_END +#define DOCTEST_REQUIRE_MESSAGE(cond, ...) DOCTEST_FUNC_SCOPE_BEGIN { DOCTEST_INFO(__VA_ARGS__); DOCTEST_ASSERT_IMPLEMENT_2(DT_REQUIRE, cond); } DOCTEST_FUNC_SCOPE_END +#define DOCTEST_WARN_FALSE_MESSAGE(cond, ...) DOCTEST_FUNC_SCOPE_BEGIN { DOCTEST_INFO(__VA_ARGS__); DOCTEST_ASSERT_IMPLEMENT_2(DT_WARN_FALSE, cond); } DOCTEST_FUNC_SCOPE_END +#define DOCTEST_CHECK_FALSE_MESSAGE(cond, ...) DOCTEST_FUNC_SCOPE_BEGIN { DOCTEST_INFO(__VA_ARGS__); DOCTEST_ASSERT_IMPLEMENT_2(DT_CHECK_FALSE, cond); } DOCTEST_FUNC_SCOPE_END +#define DOCTEST_REQUIRE_FALSE_MESSAGE(cond, ...) DOCTEST_FUNC_SCOPE_BEGIN { DOCTEST_INFO(__VA_ARGS__); DOCTEST_ASSERT_IMPLEMENT_2(DT_REQUIRE_FALSE, cond); } DOCTEST_FUNC_SCOPE_END // clang-format on -#define DOCTEST_ASSERT_THROWS_AS(expr, assert_type, message, ...) \ - do { \ - if(!doctest::getContextOptions()->no_throw) { \ - doctest::detail::ResultBuilder _DOCTEST_RB(doctest::assertType::assert_type, __FILE__, \ - __LINE__, #expr, #__VA_ARGS__, message); \ - try { \ - DOCTEST_CAST_TO_VOID(expr) \ - } catch(const typename doctest::detail::remove_const< \ - typename doctest::detail::remove_reference<__VA_ARGS__>::type>::type&) { \ - _DOCTEST_RB.translateException(); \ - _DOCTEST_RB.m_threw_as = true; \ - } catch(...) { _DOCTEST_RB.translateException(); } \ - DOCTEST_ASSERT_LOG_AND_REACT(_DOCTEST_RB); \ - } \ - } while(false) - -#define DOCTEST_ASSERT_THROWS_WITH(expr, expr_str, assert_type, ...) \ - do { \ - if(!doctest::getContextOptions()->no_throw) { \ - doctest::detail::ResultBuilder _DOCTEST_RB(doctest::assertType::assert_type, __FILE__, \ - __LINE__, expr_str, "", __VA_ARGS__); \ - try { \ - DOCTEST_CAST_TO_VOID(expr) \ - } catch(...) { _DOCTEST_RB.translateException(); } \ - DOCTEST_ASSERT_LOG_AND_REACT(_DOCTEST_RB); \ - } \ - } while(false) - -#define DOCTEST_ASSERT_NOTHROW(assert_type, ...) \ - do { \ - doctest::detail::ResultBuilder _DOCTEST_RB(doctest::assertType::assert_type, __FILE__, \ - __LINE__, #__VA_ARGS__); \ - try { \ - DOCTEST_CAST_TO_VOID(__VA_ARGS__) \ - } catch(...) { _DOCTEST_RB.translateException(); } \ - DOCTEST_ASSERT_LOG_AND_REACT(_DOCTEST_RB); \ - } while(false) - -// clang-format off -#define DOCTEST_WARN_THROWS(...) DOCTEST_ASSERT_THROWS_WITH((__VA_ARGS__), #__VA_ARGS__, DT_WARN_THROWS, "") -#define DOCTEST_CHECK_THROWS(...) DOCTEST_ASSERT_THROWS_WITH((__VA_ARGS__), #__VA_ARGS__, DT_CHECK_THROWS, "") -#define DOCTEST_REQUIRE_THROWS(...) DOCTEST_ASSERT_THROWS_WITH((__VA_ARGS__), #__VA_ARGS__, DT_REQUIRE_THROWS, "") - -#define DOCTEST_WARN_THROWS_AS(expr, ...) DOCTEST_ASSERT_THROWS_AS(expr, DT_WARN_THROWS_AS, "", __VA_ARGS__) -#define DOCTEST_CHECK_THROWS_AS(expr, ...) DOCTEST_ASSERT_THROWS_AS(expr, DT_CHECK_THROWS_AS, "", __VA_ARGS__) -#define DOCTEST_REQUIRE_THROWS_AS(expr, ...) DOCTEST_ASSERT_THROWS_AS(expr, DT_REQUIRE_THROWS_AS, "", __VA_ARGS__) - -#define DOCTEST_WARN_THROWS_WITH(expr, ...) DOCTEST_ASSERT_THROWS_WITH(expr, #expr, DT_WARN_THROWS_WITH, __VA_ARGS__) -#define DOCTEST_CHECK_THROWS_WITH(expr, ...) DOCTEST_ASSERT_THROWS_WITH(expr, #expr, DT_CHECK_THROWS_WITH, __VA_ARGS__) -#define DOCTEST_REQUIRE_THROWS_WITH(expr, ...) DOCTEST_ASSERT_THROWS_WITH(expr, #expr, DT_REQUIRE_THROWS_WITH, __VA_ARGS__) - -#define DOCTEST_WARN_THROWS_WITH_AS(expr, message, ...) DOCTEST_ASSERT_THROWS_AS(expr, DT_WARN_THROWS_WITH_AS, message, __VA_ARGS__) -#define DOCTEST_CHECK_THROWS_WITH_AS(expr, message, ...) DOCTEST_ASSERT_THROWS_AS(expr, DT_CHECK_THROWS_WITH_AS, message, __VA_ARGS__) -#define DOCTEST_REQUIRE_THROWS_WITH_AS(expr, message, ...) DOCTEST_ASSERT_THROWS_AS(expr, DT_REQUIRE_THROWS_WITH_AS, message, __VA_ARGS__) - -#define DOCTEST_WARN_NOTHROW(...) DOCTEST_ASSERT_NOTHROW(DT_WARN_NOTHROW, __VA_ARGS__) -#define DOCTEST_CHECK_NOTHROW(...) DOCTEST_ASSERT_NOTHROW(DT_CHECK_NOTHROW, __VA_ARGS__) -#define DOCTEST_REQUIRE_NOTHROW(...) DOCTEST_ASSERT_NOTHROW(DT_REQUIRE_NOTHROW, __VA_ARGS__) - -#define DOCTEST_WARN_THROWS_MESSAGE(expr, ...) do { DOCTEST_INFO(__VA_ARGS__); DOCTEST_WARN_THROWS(expr); } while(false) -#define DOCTEST_CHECK_THROWS_MESSAGE(expr, ...) do { DOCTEST_INFO(__VA_ARGS__); DOCTEST_CHECK_THROWS(expr); } while(false) -#define DOCTEST_REQUIRE_THROWS_MESSAGE(expr, ...) do { DOCTEST_INFO(__VA_ARGS__); DOCTEST_REQUIRE_THROWS(expr); } while(false) -#define DOCTEST_WARN_THROWS_AS_MESSAGE(expr, ex, ...) do { DOCTEST_INFO(__VA_ARGS__); DOCTEST_WARN_THROWS_AS(expr, ex); } while(false) -#define DOCTEST_CHECK_THROWS_AS_MESSAGE(expr, ex, ...) do { DOCTEST_INFO(__VA_ARGS__); DOCTEST_CHECK_THROWS_AS(expr, ex); } while(false) -#define DOCTEST_REQUIRE_THROWS_AS_MESSAGE(expr, ex, ...) do { DOCTEST_INFO(__VA_ARGS__); DOCTEST_REQUIRE_THROWS_AS(expr, ex); } while(false) -#define DOCTEST_WARN_THROWS_WITH_MESSAGE(expr, with, ...) do { DOCTEST_INFO(__VA_ARGS__); DOCTEST_WARN_THROWS_WITH(expr, with); } while(false) -#define DOCTEST_CHECK_THROWS_WITH_MESSAGE(expr, with, ...) do { DOCTEST_INFO(__VA_ARGS__); DOCTEST_CHECK_THROWS_WITH(expr, with); } while(false) -#define DOCTEST_REQUIRE_THROWS_WITH_MESSAGE(expr, with, ...) do { DOCTEST_INFO(__VA_ARGS__); DOCTEST_REQUIRE_THROWS_WITH(expr, with); } while(false) -#define DOCTEST_WARN_THROWS_WITH_AS_MESSAGE(expr, with, ex, ...) do { DOCTEST_INFO(__VA_ARGS__); DOCTEST_WARN_THROWS_WITH_AS(expr, with, ex); } while(false) -#define DOCTEST_CHECK_THROWS_WITH_AS_MESSAGE(expr, with, ex, ...) do { DOCTEST_INFO(__VA_ARGS__); DOCTEST_CHECK_THROWS_WITH_AS(expr, with, ex); } while(false) -#define DOCTEST_REQUIRE_THROWS_WITH_AS_MESSAGE(expr, with, ex, ...) do { DOCTEST_INFO(__VA_ARGS__); DOCTEST_REQUIRE_THROWS_WITH_AS(expr, with, ex); } while(false) -#define DOCTEST_WARN_NOTHROW_MESSAGE(expr, ...) do { DOCTEST_INFO(__VA_ARGS__); DOCTEST_WARN_NOTHROW(expr); } while(false) -#define DOCTEST_CHECK_NOTHROW_MESSAGE(expr, ...) do { DOCTEST_INFO(__VA_ARGS__); DOCTEST_CHECK_NOTHROW(expr); } while(false) -#define DOCTEST_REQUIRE_NOTHROW_MESSAGE(expr, ...) do { DOCTEST_INFO(__VA_ARGS__); DOCTEST_REQUIRE_NOTHROW(expr); } while(false) -// clang-format on - -#ifndef DOCTEST_CONFIG_SUPER_FAST_ASSERTS - -#define DOCTEST_BINARY_ASSERT(assert_type, comp, ...) \ - do { \ - doctest::detail::ResultBuilder _DOCTEST_RB(doctest::assertType::assert_type, __FILE__, \ - __LINE__, #__VA_ARGS__); \ - DOCTEST_WRAP_IN_TRY( \ - _DOCTEST_RB.binary_assert( \ - __VA_ARGS__)) \ - DOCTEST_ASSERT_LOG_AND_REACT(_DOCTEST_RB); \ - } while(false) - -#define DOCTEST_UNARY_ASSERT(assert_type, ...) \ - do { \ - doctest::detail::ResultBuilder _DOCTEST_RB(doctest::assertType::assert_type, __FILE__, \ - __LINE__, #__VA_ARGS__); \ - DOCTEST_WRAP_IN_TRY(_DOCTEST_RB.unary_assert(__VA_ARGS__)) \ - DOCTEST_ASSERT_LOG_AND_REACT(_DOCTEST_RB); \ - } while(false) - -#else // DOCTEST_CONFIG_SUPER_FAST_ASSERTS - -#define DOCTEST_BINARY_ASSERT(assert_type, comparison, ...) \ - doctest::detail::binary_assert( \ - doctest::assertType::assert_type, __FILE__, __LINE__, #__VA_ARGS__, __VA_ARGS__) - -#define DOCTEST_UNARY_ASSERT(assert_type, ...) \ - doctest::detail::unary_assert(doctest::assertType::assert_type, __FILE__, __LINE__, \ - #__VA_ARGS__, __VA_ARGS__) - -#endif // DOCTEST_CONFIG_SUPER_FAST_ASSERTS - #define DOCTEST_WARN_EQ(...) DOCTEST_BINARY_ASSERT(DT_WARN_EQ, eq, __VA_ARGS__) #define DOCTEST_CHECK_EQ(...) DOCTEST_BINARY_ASSERT(DT_CHECK_EQ, eq, __VA_ARGS__) #define DOCTEST_REQUIRE_EQ(...) DOCTEST_BINARY_ASSERT(DT_REQUIRE_EQ, eq, __VA_ARGS__) @@ -2250,75 +2436,350 @@ int registerReporter(const char* name, int priority, bool isReporter) { #define DOCTEST_CHECK_UNARY_FALSE(...) DOCTEST_UNARY_ASSERT(DT_CHECK_UNARY_FALSE, __VA_ARGS__) #define DOCTEST_REQUIRE_UNARY_FALSE(...) DOCTEST_UNARY_ASSERT(DT_REQUIRE_UNARY_FALSE, __VA_ARGS__) +#ifndef DOCTEST_CONFIG_NO_EXCEPTIONS + +#define DOCTEST_ASSERT_THROWS_AS(expr, assert_type, message, ...) \ + DOCTEST_FUNC_SCOPE_BEGIN { \ + if(!doctest::getContextOptions()->no_throw) { \ + doctest::detail::ResultBuilder DOCTEST_RB(doctest::assertType::assert_type, __FILE__, \ + __LINE__, #expr, #__VA_ARGS__, message); \ + try { \ + DOCTEST_CAST_TO_VOID(expr) \ + } catch(const typename doctest::detail::types::remove_const< \ + typename doctest::detail::types::remove_reference<__VA_ARGS__>::type>::type&) {\ + DOCTEST_RB.translateException(); \ + DOCTEST_RB.m_threw_as = true; \ + } catch(...) { DOCTEST_RB.translateException(); } \ + DOCTEST_ASSERT_LOG_REACT_RETURN(DOCTEST_RB); \ + } else { /* NOLINT(*-else-after-return) */ \ + DOCTEST_FUNC_SCOPE_RET(false); \ + } \ + } DOCTEST_FUNC_SCOPE_END + +#define DOCTEST_ASSERT_THROWS_WITH(expr, expr_str, assert_type, ...) \ + DOCTEST_FUNC_SCOPE_BEGIN { \ + if(!doctest::getContextOptions()->no_throw) { \ + doctest::detail::ResultBuilder DOCTEST_RB(doctest::assertType::assert_type, __FILE__, \ + __LINE__, expr_str, "", __VA_ARGS__); \ + try { \ + DOCTEST_CAST_TO_VOID(expr) \ + } catch(...) { DOCTEST_RB.translateException(); } \ + DOCTEST_ASSERT_LOG_REACT_RETURN(DOCTEST_RB); \ + } else { /* NOLINT(*-else-after-return) */ \ + DOCTEST_FUNC_SCOPE_RET(false); \ + } \ + } DOCTEST_FUNC_SCOPE_END + +#define DOCTEST_ASSERT_NOTHROW(assert_type, ...) \ + DOCTEST_FUNC_SCOPE_BEGIN { \ + doctest::detail::ResultBuilder DOCTEST_RB(doctest::assertType::assert_type, __FILE__, \ + __LINE__, #__VA_ARGS__); \ + try { \ + DOCTEST_CAST_TO_VOID(__VA_ARGS__) \ + } catch(...) { DOCTEST_RB.translateException(); } \ + DOCTEST_ASSERT_LOG_REACT_RETURN(DOCTEST_RB); \ + } DOCTEST_FUNC_SCOPE_END + +// clang-format off +#define DOCTEST_WARN_THROWS(...) DOCTEST_ASSERT_THROWS_WITH((__VA_ARGS__), #__VA_ARGS__, DT_WARN_THROWS, "") +#define DOCTEST_CHECK_THROWS(...) DOCTEST_ASSERT_THROWS_WITH((__VA_ARGS__), #__VA_ARGS__, DT_CHECK_THROWS, "") +#define DOCTEST_REQUIRE_THROWS(...) DOCTEST_ASSERT_THROWS_WITH((__VA_ARGS__), #__VA_ARGS__, DT_REQUIRE_THROWS, "") + +#define DOCTEST_WARN_THROWS_AS(expr, ...) DOCTEST_ASSERT_THROWS_AS(expr, DT_WARN_THROWS_AS, "", __VA_ARGS__) +#define DOCTEST_CHECK_THROWS_AS(expr, ...) DOCTEST_ASSERT_THROWS_AS(expr, DT_CHECK_THROWS_AS, "", __VA_ARGS__) +#define DOCTEST_REQUIRE_THROWS_AS(expr, ...) DOCTEST_ASSERT_THROWS_AS(expr, DT_REQUIRE_THROWS_AS, "", __VA_ARGS__) + +#define DOCTEST_WARN_THROWS_WITH(expr, ...) DOCTEST_ASSERT_THROWS_WITH(expr, #expr, DT_WARN_THROWS_WITH, __VA_ARGS__) +#define DOCTEST_CHECK_THROWS_WITH(expr, ...) DOCTEST_ASSERT_THROWS_WITH(expr, #expr, DT_CHECK_THROWS_WITH, __VA_ARGS__) +#define DOCTEST_REQUIRE_THROWS_WITH(expr, ...) DOCTEST_ASSERT_THROWS_WITH(expr, #expr, DT_REQUIRE_THROWS_WITH, __VA_ARGS__) + +#define DOCTEST_WARN_THROWS_WITH_AS(expr, message, ...) DOCTEST_ASSERT_THROWS_AS(expr, DT_WARN_THROWS_WITH_AS, message, __VA_ARGS__) +#define DOCTEST_CHECK_THROWS_WITH_AS(expr, message, ...) DOCTEST_ASSERT_THROWS_AS(expr, DT_CHECK_THROWS_WITH_AS, message, __VA_ARGS__) +#define DOCTEST_REQUIRE_THROWS_WITH_AS(expr, message, ...) DOCTEST_ASSERT_THROWS_AS(expr, DT_REQUIRE_THROWS_WITH_AS, message, __VA_ARGS__) + +#define DOCTEST_WARN_NOTHROW(...) DOCTEST_ASSERT_NOTHROW(DT_WARN_NOTHROW, __VA_ARGS__) +#define DOCTEST_CHECK_NOTHROW(...) DOCTEST_ASSERT_NOTHROW(DT_CHECK_NOTHROW, __VA_ARGS__) +#define DOCTEST_REQUIRE_NOTHROW(...) DOCTEST_ASSERT_NOTHROW(DT_REQUIRE_NOTHROW, __VA_ARGS__) + +#define DOCTEST_WARN_THROWS_MESSAGE(expr, ...) DOCTEST_FUNC_SCOPE_BEGIN { DOCTEST_INFO(__VA_ARGS__); DOCTEST_WARN_THROWS(expr); } DOCTEST_FUNC_SCOPE_END +#define DOCTEST_CHECK_THROWS_MESSAGE(expr, ...) DOCTEST_FUNC_SCOPE_BEGIN { DOCTEST_INFO(__VA_ARGS__); DOCTEST_CHECK_THROWS(expr); } DOCTEST_FUNC_SCOPE_END +#define DOCTEST_REQUIRE_THROWS_MESSAGE(expr, ...) DOCTEST_FUNC_SCOPE_BEGIN { DOCTEST_INFO(__VA_ARGS__); DOCTEST_REQUIRE_THROWS(expr); } DOCTEST_FUNC_SCOPE_END +#define DOCTEST_WARN_THROWS_AS_MESSAGE(expr, ex, ...) DOCTEST_FUNC_SCOPE_BEGIN { DOCTEST_INFO(__VA_ARGS__); DOCTEST_WARN_THROWS_AS(expr, ex); } DOCTEST_FUNC_SCOPE_END +#define DOCTEST_CHECK_THROWS_AS_MESSAGE(expr, ex, ...) DOCTEST_FUNC_SCOPE_BEGIN { DOCTEST_INFO(__VA_ARGS__); DOCTEST_CHECK_THROWS_AS(expr, ex); } DOCTEST_FUNC_SCOPE_END +#define DOCTEST_REQUIRE_THROWS_AS_MESSAGE(expr, ex, ...) DOCTEST_FUNC_SCOPE_BEGIN { DOCTEST_INFO(__VA_ARGS__); DOCTEST_REQUIRE_THROWS_AS(expr, ex); } DOCTEST_FUNC_SCOPE_END +#define DOCTEST_WARN_THROWS_WITH_MESSAGE(expr, with, ...) DOCTEST_FUNC_SCOPE_BEGIN { DOCTEST_INFO(__VA_ARGS__); DOCTEST_WARN_THROWS_WITH(expr, with); } DOCTEST_FUNC_SCOPE_END +#define DOCTEST_CHECK_THROWS_WITH_MESSAGE(expr, with, ...) DOCTEST_FUNC_SCOPE_BEGIN { DOCTEST_INFO(__VA_ARGS__); DOCTEST_CHECK_THROWS_WITH(expr, with); } DOCTEST_FUNC_SCOPE_END +#define DOCTEST_REQUIRE_THROWS_WITH_MESSAGE(expr, with, ...) DOCTEST_FUNC_SCOPE_BEGIN { DOCTEST_INFO(__VA_ARGS__); DOCTEST_REQUIRE_THROWS_WITH(expr, with); } DOCTEST_FUNC_SCOPE_END +#define DOCTEST_WARN_THROWS_WITH_AS_MESSAGE(expr, with, ex, ...) DOCTEST_FUNC_SCOPE_BEGIN { DOCTEST_INFO(__VA_ARGS__); DOCTEST_WARN_THROWS_WITH_AS(expr, with, ex); } DOCTEST_FUNC_SCOPE_END +#define DOCTEST_CHECK_THROWS_WITH_AS_MESSAGE(expr, with, ex, ...) DOCTEST_FUNC_SCOPE_BEGIN { DOCTEST_INFO(__VA_ARGS__); DOCTEST_CHECK_THROWS_WITH_AS(expr, with, ex); } DOCTEST_FUNC_SCOPE_END +#define DOCTEST_REQUIRE_THROWS_WITH_AS_MESSAGE(expr, with, ex, ...) DOCTEST_FUNC_SCOPE_BEGIN { DOCTEST_INFO(__VA_ARGS__); DOCTEST_REQUIRE_THROWS_WITH_AS(expr, with, ex); } DOCTEST_FUNC_SCOPE_END +#define DOCTEST_WARN_NOTHROW_MESSAGE(expr, ...) DOCTEST_FUNC_SCOPE_BEGIN { DOCTEST_INFO(__VA_ARGS__); DOCTEST_WARN_NOTHROW(expr); } DOCTEST_FUNC_SCOPE_END +#define DOCTEST_CHECK_NOTHROW_MESSAGE(expr, ...) DOCTEST_FUNC_SCOPE_BEGIN { DOCTEST_INFO(__VA_ARGS__); DOCTEST_CHECK_NOTHROW(expr); } DOCTEST_FUNC_SCOPE_END +#define DOCTEST_REQUIRE_NOTHROW_MESSAGE(expr, ...) DOCTEST_FUNC_SCOPE_BEGIN { DOCTEST_INFO(__VA_ARGS__); DOCTEST_REQUIRE_NOTHROW(expr); } DOCTEST_FUNC_SCOPE_END +// clang-format on + +#endif // DOCTEST_CONFIG_NO_EXCEPTIONS + +// ================================================================================================= +// == WHAT FOLLOWS IS VERSIONS OF THE MACROS THAT DO NOT DO ANY REGISTERING! == +// == THIS CAN BE ENABLED BY DEFINING DOCTEST_CONFIG_DISABLE GLOBALLY! == +// ================================================================================================= +#else // DOCTEST_CONFIG_DISABLE + +#define DOCTEST_IMPLEMENT_FIXTURE(der, base, func, name) \ + namespace /* NOLINT */ { \ + template \ + struct der : public base \ + { void f(); }; \ + } \ + template \ + inline void der::f() + +#define DOCTEST_CREATE_AND_REGISTER_FUNCTION(f, name) \ + template \ + static inline void f() + +// for registering tests +#define DOCTEST_TEST_CASE(name) \ + DOCTEST_CREATE_AND_REGISTER_FUNCTION(DOCTEST_ANONYMOUS(DOCTEST_ANON_FUNC_), name) + +// for registering tests in classes +#define DOCTEST_TEST_CASE_CLASS(name) \ + DOCTEST_CREATE_AND_REGISTER_FUNCTION(DOCTEST_ANONYMOUS(DOCTEST_ANON_FUNC_), name) + +// for registering tests with a fixture +#define DOCTEST_TEST_CASE_FIXTURE(x, name) \ + DOCTEST_IMPLEMENT_FIXTURE(DOCTEST_ANONYMOUS(DOCTEST_ANON_CLASS_), x, \ + DOCTEST_ANONYMOUS(DOCTEST_ANON_FUNC_), name) + +// for converting types to strings without the header and demangling +#define DOCTEST_TYPE_TO_STRING_AS(str, ...) static_assert(true, "") +#define DOCTEST_TYPE_TO_STRING(...) static_assert(true, "") + +// for typed tests +#define DOCTEST_TEST_CASE_TEMPLATE(name, type, ...) \ + template \ + inline void DOCTEST_ANONYMOUS(DOCTEST_ANON_TMP_)() + +#define DOCTEST_TEST_CASE_TEMPLATE_DEFINE(name, type, id) \ + template \ + inline void DOCTEST_ANONYMOUS(DOCTEST_ANON_TMP_)() + +#define DOCTEST_TEST_CASE_TEMPLATE_INVOKE(id, ...) static_assert(true, "") +#define DOCTEST_TEST_CASE_TEMPLATE_APPLY(id, ...) static_assert(true, "") + +// for subcases +#define DOCTEST_SUBCASE(name) + +// for a testsuite block +#define DOCTEST_TEST_SUITE(name) namespace // NOLINT + +// for starting a testsuite block +#define DOCTEST_TEST_SUITE_BEGIN(name) static_assert(true, "") + +// for ending a testsuite block +#define DOCTEST_TEST_SUITE_END using DOCTEST_ANONYMOUS(DOCTEST_ANON_FOR_SEMICOLON_) = int + +#define DOCTEST_REGISTER_EXCEPTION_TRANSLATOR(signature) \ + template \ + static inline doctest::String DOCTEST_ANONYMOUS(DOCTEST_ANON_TRANSLATOR_)(signature) + +#define DOCTEST_REGISTER_REPORTER(name, priority, reporter) +#define DOCTEST_REGISTER_LISTENER(name, priority, reporter) + +#define DOCTEST_INFO(...) (static_cast(0)) +#define DOCTEST_CAPTURE(x) (static_cast(0)) +#define DOCTEST_ADD_MESSAGE_AT(file, line, ...) (static_cast(0)) +#define DOCTEST_ADD_FAIL_CHECK_AT(file, line, ...) (static_cast(0)) +#define DOCTEST_ADD_FAIL_AT(file, line, ...) (static_cast(0)) +#define DOCTEST_MESSAGE(...) (static_cast(0)) +#define DOCTEST_FAIL_CHECK(...) (static_cast(0)) +#define DOCTEST_FAIL(...) (static_cast(0)) + +#if defined(DOCTEST_CONFIG_EVALUATE_ASSERTS_EVEN_WHEN_DISABLED) \ + && defined(DOCTEST_CONFIG_ASSERTS_RETURN_VALUES) + +#define DOCTEST_WARN(...) [&] { return __VA_ARGS__; }() +#define DOCTEST_CHECK(...) [&] { return __VA_ARGS__; }() +#define DOCTEST_REQUIRE(...) [&] { return __VA_ARGS__; }() +#define DOCTEST_WARN_FALSE(...) [&] { return !(__VA_ARGS__); }() +#define DOCTEST_CHECK_FALSE(...) [&] { return !(__VA_ARGS__); }() +#define DOCTEST_REQUIRE_FALSE(...) [&] { return !(__VA_ARGS__); }() + +#define DOCTEST_WARN_MESSAGE(cond, ...) [&] { return cond; }() +#define DOCTEST_CHECK_MESSAGE(cond, ...) [&] { return cond; }() +#define DOCTEST_REQUIRE_MESSAGE(cond, ...) [&] { return cond; }() +#define DOCTEST_WARN_FALSE_MESSAGE(cond, ...) [&] { return !(cond); }() +#define DOCTEST_CHECK_FALSE_MESSAGE(cond, ...) [&] { return !(cond); }() +#define DOCTEST_REQUIRE_FALSE_MESSAGE(cond, ...) [&] { return !(cond); }() + +namespace doctest { +namespace detail { +#define DOCTEST_RELATIONAL_OP(name, op) \ + template \ + bool name(const DOCTEST_REF_WRAP(L) lhs, const DOCTEST_REF_WRAP(R) rhs) { return lhs op rhs; } + + DOCTEST_RELATIONAL_OP(eq, ==) + DOCTEST_RELATIONAL_OP(ne, !=) + DOCTEST_RELATIONAL_OP(lt, <) + DOCTEST_RELATIONAL_OP(gt, >) + DOCTEST_RELATIONAL_OP(le, <=) + DOCTEST_RELATIONAL_OP(ge, >=) +} // namespace detail +} // namespace doctest + +#define DOCTEST_WARN_EQ(...) [&] { return doctest::detail::eq(__VA_ARGS__); }() +#define DOCTEST_CHECK_EQ(...) [&] { return doctest::detail::eq(__VA_ARGS__); }() +#define DOCTEST_REQUIRE_EQ(...) [&] { return doctest::detail::eq(__VA_ARGS__); }() +#define DOCTEST_WARN_NE(...) [&] { return doctest::detail::ne(__VA_ARGS__); }() +#define DOCTEST_CHECK_NE(...) [&] { return doctest::detail::ne(__VA_ARGS__); }() +#define DOCTEST_REQUIRE_NE(...) [&] { return doctest::detail::ne(__VA_ARGS__); }() +#define DOCTEST_WARN_LT(...) [&] { return doctest::detail::lt(__VA_ARGS__); }() +#define DOCTEST_CHECK_LT(...) [&] { return doctest::detail::lt(__VA_ARGS__); }() +#define DOCTEST_REQUIRE_LT(...) [&] { return doctest::detail::lt(__VA_ARGS__); }() +#define DOCTEST_WARN_GT(...) [&] { return doctest::detail::gt(__VA_ARGS__); }() +#define DOCTEST_CHECK_GT(...) [&] { return doctest::detail::gt(__VA_ARGS__); }() +#define DOCTEST_REQUIRE_GT(...) [&] { return doctest::detail::gt(__VA_ARGS__); }() +#define DOCTEST_WARN_LE(...) [&] { return doctest::detail::le(__VA_ARGS__); }() +#define DOCTEST_CHECK_LE(...) [&] { return doctest::detail::le(__VA_ARGS__); }() +#define DOCTEST_REQUIRE_LE(...) [&] { return doctest::detail::le(__VA_ARGS__); }() +#define DOCTEST_WARN_GE(...) [&] { return doctest::detail::ge(__VA_ARGS__); }() +#define DOCTEST_CHECK_GE(...) [&] { return doctest::detail::ge(__VA_ARGS__); }() +#define DOCTEST_REQUIRE_GE(...) [&] { return doctest::detail::ge(__VA_ARGS__); }() +#define DOCTEST_WARN_UNARY(...) [&] { return __VA_ARGS__; }() +#define DOCTEST_CHECK_UNARY(...) [&] { return __VA_ARGS__; }() +#define DOCTEST_REQUIRE_UNARY(...) [&] { return __VA_ARGS__; }() +#define DOCTEST_WARN_UNARY_FALSE(...) [&] { return !(__VA_ARGS__); }() +#define DOCTEST_CHECK_UNARY_FALSE(...) [&] { return !(__VA_ARGS__); }() +#define DOCTEST_REQUIRE_UNARY_FALSE(...) [&] { return !(__VA_ARGS__); }() + +#ifndef DOCTEST_CONFIG_NO_EXCEPTIONS + +#define DOCTEST_WARN_THROWS_WITH(expr, with, ...) [] { static_assert(false, "Exception translation is not available when doctest is disabled."); return false; }() +#define DOCTEST_CHECK_THROWS_WITH(expr, with, ...) DOCTEST_WARN_THROWS_WITH(,,) +#define DOCTEST_REQUIRE_THROWS_WITH(expr, with, ...) DOCTEST_WARN_THROWS_WITH(,,) +#define DOCTEST_WARN_THROWS_WITH_AS(expr, with, ex, ...) DOCTEST_WARN_THROWS_WITH(,,) +#define DOCTEST_CHECK_THROWS_WITH_AS(expr, with, ex, ...) DOCTEST_WARN_THROWS_WITH(,,) +#define DOCTEST_REQUIRE_THROWS_WITH_AS(expr, with, ex, ...) DOCTEST_WARN_THROWS_WITH(,,) + +#define DOCTEST_WARN_THROWS_WITH_MESSAGE(expr, with, ...) DOCTEST_WARN_THROWS_WITH(,,) +#define DOCTEST_CHECK_THROWS_WITH_MESSAGE(expr, with, ...) DOCTEST_WARN_THROWS_WITH(,,) +#define DOCTEST_REQUIRE_THROWS_WITH_MESSAGE(expr, with, ...) DOCTEST_WARN_THROWS_WITH(,,) +#define DOCTEST_WARN_THROWS_WITH_AS_MESSAGE(expr, with, ex, ...) DOCTEST_WARN_THROWS_WITH(,,) +#define DOCTEST_CHECK_THROWS_WITH_AS_MESSAGE(expr, with, ex, ...) DOCTEST_WARN_THROWS_WITH(,,) +#define DOCTEST_REQUIRE_THROWS_WITH_AS_MESSAGE(expr, with, ex, ...) DOCTEST_WARN_THROWS_WITH(,,) + +#define DOCTEST_WARN_THROWS(...) [&] { try { __VA_ARGS__; return false; } catch (...) { return true; } }() +#define DOCTEST_CHECK_THROWS(...) [&] { try { __VA_ARGS__; return false; } catch (...) { return true; } }() +#define DOCTEST_REQUIRE_THROWS(...) [&] { try { __VA_ARGS__; return false; } catch (...) { return true; } }() +#define DOCTEST_WARN_THROWS_AS(expr, ...) [&] { try { expr; } catch (__VA_ARGS__) { return true; } catch (...) { } return false; }() +#define DOCTEST_CHECK_THROWS_AS(expr, ...) [&] { try { expr; } catch (__VA_ARGS__) { return true; } catch (...) { } return false; }() +#define DOCTEST_REQUIRE_THROWS_AS(expr, ...) [&] { try { expr; } catch (__VA_ARGS__) { return true; } catch (...) { } return false; }() +#define DOCTEST_WARN_NOTHROW(...) [&] { try { __VA_ARGS__; return true; } catch (...) { return false; } }() +#define DOCTEST_CHECK_NOTHROW(...) [&] { try { __VA_ARGS__; return true; } catch (...) { return false; } }() +#define DOCTEST_REQUIRE_NOTHROW(...) [&] { try { __VA_ARGS__; return true; } catch (...) { return false; } }() + +#define DOCTEST_WARN_THROWS_MESSAGE(expr, ...) [&] { try { __VA_ARGS__; return false; } catch (...) { return true; } }() +#define DOCTEST_CHECK_THROWS_MESSAGE(expr, ...) [&] { try { __VA_ARGS__; return false; } catch (...) { return true; } }() +#define DOCTEST_REQUIRE_THROWS_MESSAGE(expr, ...) [&] { try { __VA_ARGS__; return false; } catch (...) { return true; } }() +#define DOCTEST_WARN_THROWS_AS_MESSAGE(expr, ex, ...) [&] { try { expr; } catch (__VA_ARGS__) { return true; } catch (...) { } return false; }() +#define DOCTEST_CHECK_THROWS_AS_MESSAGE(expr, ex, ...) [&] { try { expr; } catch (__VA_ARGS__) { return true; } catch (...) { } return false; }() +#define DOCTEST_REQUIRE_THROWS_AS_MESSAGE(expr, ex, ...) [&] { try { expr; } catch (__VA_ARGS__) { return true; } catch (...) { } return false; }() +#define DOCTEST_WARN_NOTHROW_MESSAGE(expr, ...) [&] { try { __VA_ARGS__; return true; } catch (...) { return false; } }() +#define DOCTEST_CHECK_NOTHROW_MESSAGE(expr, ...) [&] { try { __VA_ARGS__; return true; } catch (...) { return false; } }() +#define DOCTEST_REQUIRE_NOTHROW_MESSAGE(expr, ...) [&] { try { __VA_ARGS__; return true; } catch (...) { return false; } }() + +#endif // DOCTEST_CONFIG_NO_EXCEPTIONS + +#else // DOCTEST_CONFIG_EVALUATE_ASSERTS_EVEN_WHEN_DISABLED + +#define DOCTEST_WARN(...) DOCTEST_FUNC_EMPTY +#define DOCTEST_CHECK(...) DOCTEST_FUNC_EMPTY +#define DOCTEST_REQUIRE(...) DOCTEST_FUNC_EMPTY +#define DOCTEST_WARN_FALSE(...) DOCTEST_FUNC_EMPTY +#define DOCTEST_CHECK_FALSE(...) DOCTEST_FUNC_EMPTY +#define DOCTEST_REQUIRE_FALSE(...) DOCTEST_FUNC_EMPTY + +#define DOCTEST_WARN_MESSAGE(cond, ...) DOCTEST_FUNC_EMPTY +#define DOCTEST_CHECK_MESSAGE(cond, ...) DOCTEST_FUNC_EMPTY +#define DOCTEST_REQUIRE_MESSAGE(cond, ...) DOCTEST_FUNC_EMPTY +#define DOCTEST_WARN_FALSE_MESSAGE(cond, ...) DOCTEST_FUNC_EMPTY +#define DOCTEST_CHECK_FALSE_MESSAGE(cond, ...) DOCTEST_FUNC_EMPTY +#define DOCTEST_REQUIRE_FALSE_MESSAGE(cond, ...) DOCTEST_FUNC_EMPTY + +#define DOCTEST_WARN_EQ(...) DOCTEST_FUNC_EMPTY +#define DOCTEST_CHECK_EQ(...) DOCTEST_FUNC_EMPTY +#define DOCTEST_REQUIRE_EQ(...) DOCTEST_FUNC_EMPTY +#define DOCTEST_WARN_NE(...) DOCTEST_FUNC_EMPTY +#define DOCTEST_CHECK_NE(...) DOCTEST_FUNC_EMPTY +#define DOCTEST_REQUIRE_NE(...) DOCTEST_FUNC_EMPTY +#define DOCTEST_WARN_GT(...) DOCTEST_FUNC_EMPTY +#define DOCTEST_CHECK_GT(...) DOCTEST_FUNC_EMPTY +#define DOCTEST_REQUIRE_GT(...) DOCTEST_FUNC_EMPTY +#define DOCTEST_WARN_LT(...) DOCTEST_FUNC_EMPTY +#define DOCTEST_CHECK_LT(...) DOCTEST_FUNC_EMPTY +#define DOCTEST_REQUIRE_LT(...) DOCTEST_FUNC_EMPTY +#define DOCTEST_WARN_GE(...) DOCTEST_FUNC_EMPTY +#define DOCTEST_CHECK_GE(...) DOCTEST_FUNC_EMPTY +#define DOCTEST_REQUIRE_GE(...) DOCTEST_FUNC_EMPTY +#define DOCTEST_WARN_LE(...) DOCTEST_FUNC_EMPTY +#define DOCTEST_CHECK_LE(...) DOCTEST_FUNC_EMPTY +#define DOCTEST_REQUIRE_LE(...) DOCTEST_FUNC_EMPTY + +#define DOCTEST_WARN_UNARY(...) DOCTEST_FUNC_EMPTY +#define DOCTEST_CHECK_UNARY(...) DOCTEST_FUNC_EMPTY +#define DOCTEST_REQUIRE_UNARY(...) DOCTEST_FUNC_EMPTY +#define DOCTEST_WARN_UNARY_FALSE(...) DOCTEST_FUNC_EMPTY +#define DOCTEST_CHECK_UNARY_FALSE(...) DOCTEST_FUNC_EMPTY +#define DOCTEST_REQUIRE_UNARY_FALSE(...) DOCTEST_FUNC_EMPTY + +#ifndef DOCTEST_CONFIG_NO_EXCEPTIONS + +#define DOCTEST_WARN_THROWS(...) DOCTEST_FUNC_EMPTY +#define DOCTEST_CHECK_THROWS(...) DOCTEST_FUNC_EMPTY +#define DOCTEST_REQUIRE_THROWS(...) DOCTEST_FUNC_EMPTY +#define DOCTEST_WARN_THROWS_AS(expr, ...) DOCTEST_FUNC_EMPTY +#define DOCTEST_CHECK_THROWS_AS(expr, ...) DOCTEST_FUNC_EMPTY +#define DOCTEST_REQUIRE_THROWS_AS(expr, ...) DOCTEST_FUNC_EMPTY +#define DOCTEST_WARN_THROWS_WITH(expr, ...) DOCTEST_FUNC_EMPTY +#define DOCTEST_CHECK_THROWS_WITH(expr, ...) DOCTEST_FUNC_EMPTY +#define DOCTEST_REQUIRE_THROWS_WITH(expr, ...) DOCTEST_FUNC_EMPTY +#define DOCTEST_WARN_THROWS_WITH_AS(expr, with, ...) DOCTEST_FUNC_EMPTY +#define DOCTEST_CHECK_THROWS_WITH_AS(expr, with, ...) DOCTEST_FUNC_EMPTY +#define DOCTEST_REQUIRE_THROWS_WITH_AS(expr, with, ...) DOCTEST_FUNC_EMPTY +#define DOCTEST_WARN_NOTHROW(...) DOCTEST_FUNC_EMPTY +#define DOCTEST_CHECK_NOTHROW(...) DOCTEST_FUNC_EMPTY +#define DOCTEST_REQUIRE_NOTHROW(...) DOCTEST_FUNC_EMPTY + +#define DOCTEST_WARN_THROWS_MESSAGE(expr, ...) DOCTEST_FUNC_EMPTY +#define DOCTEST_CHECK_THROWS_MESSAGE(expr, ...) DOCTEST_FUNC_EMPTY +#define DOCTEST_REQUIRE_THROWS_MESSAGE(expr, ...) DOCTEST_FUNC_EMPTY +#define DOCTEST_WARN_THROWS_AS_MESSAGE(expr, ex, ...) DOCTEST_FUNC_EMPTY +#define DOCTEST_CHECK_THROWS_AS_MESSAGE(expr, ex, ...) DOCTEST_FUNC_EMPTY +#define DOCTEST_REQUIRE_THROWS_AS_MESSAGE(expr, ex, ...) DOCTEST_FUNC_EMPTY +#define DOCTEST_WARN_THROWS_WITH_MESSAGE(expr, with, ...) DOCTEST_FUNC_EMPTY +#define DOCTEST_CHECK_THROWS_WITH_MESSAGE(expr, with, ...) DOCTEST_FUNC_EMPTY +#define DOCTEST_REQUIRE_THROWS_WITH_MESSAGE(expr, with, ...) DOCTEST_FUNC_EMPTY +#define DOCTEST_WARN_THROWS_WITH_AS_MESSAGE(expr, with, ex, ...) DOCTEST_FUNC_EMPTY +#define DOCTEST_CHECK_THROWS_WITH_AS_MESSAGE(expr, with, ex, ...) DOCTEST_FUNC_EMPTY +#define DOCTEST_REQUIRE_THROWS_WITH_AS_MESSAGE(expr, with, ex, ...) DOCTEST_FUNC_EMPTY +#define DOCTEST_WARN_NOTHROW_MESSAGE(expr, ...) DOCTEST_FUNC_EMPTY +#define DOCTEST_CHECK_NOTHROW_MESSAGE(expr, ...) DOCTEST_FUNC_EMPTY +#define DOCTEST_REQUIRE_NOTHROW_MESSAGE(expr, ...) DOCTEST_FUNC_EMPTY + +#endif // DOCTEST_CONFIG_NO_EXCEPTIONS + +#endif // DOCTEST_CONFIG_EVALUATE_ASSERTS_EVEN_WHEN_DISABLED + +#endif // DOCTEST_CONFIG_DISABLE + #ifdef DOCTEST_CONFIG_NO_EXCEPTIONS -#undef DOCTEST_WARN_THROWS -#undef DOCTEST_CHECK_THROWS -#undef DOCTEST_REQUIRE_THROWS -#undef DOCTEST_WARN_THROWS_AS -#undef DOCTEST_CHECK_THROWS_AS -#undef DOCTEST_REQUIRE_THROWS_AS -#undef DOCTEST_WARN_THROWS_WITH -#undef DOCTEST_CHECK_THROWS_WITH -#undef DOCTEST_REQUIRE_THROWS_WITH -#undef DOCTEST_WARN_THROWS_WITH_AS -#undef DOCTEST_CHECK_THROWS_WITH_AS -#undef DOCTEST_REQUIRE_THROWS_WITH_AS -#undef DOCTEST_WARN_NOTHROW -#undef DOCTEST_CHECK_NOTHROW -#undef DOCTEST_REQUIRE_NOTHROW - -#undef DOCTEST_WARN_THROWS_MESSAGE -#undef DOCTEST_CHECK_THROWS_MESSAGE -#undef DOCTEST_REQUIRE_THROWS_MESSAGE -#undef DOCTEST_WARN_THROWS_AS_MESSAGE -#undef DOCTEST_CHECK_THROWS_AS_MESSAGE -#undef DOCTEST_REQUIRE_THROWS_AS_MESSAGE -#undef DOCTEST_WARN_THROWS_WITH_MESSAGE -#undef DOCTEST_CHECK_THROWS_WITH_MESSAGE -#undef DOCTEST_REQUIRE_THROWS_WITH_MESSAGE -#undef DOCTEST_WARN_THROWS_WITH_AS_MESSAGE -#undef DOCTEST_CHECK_THROWS_WITH_AS_MESSAGE -#undef DOCTEST_REQUIRE_THROWS_WITH_AS_MESSAGE -#undef DOCTEST_WARN_NOTHROW_MESSAGE -#undef DOCTEST_CHECK_NOTHROW_MESSAGE -#undef DOCTEST_REQUIRE_NOTHROW_MESSAGE - #ifdef DOCTEST_CONFIG_NO_EXCEPTIONS_BUT_WITH_ALL_ASSERTS - -#define DOCTEST_WARN_THROWS(...) (static_cast(0)) -#define DOCTEST_CHECK_THROWS(...) (static_cast(0)) -#define DOCTEST_REQUIRE_THROWS(...) (static_cast(0)) -#define DOCTEST_WARN_THROWS_AS(expr, ...) (static_cast(0)) -#define DOCTEST_CHECK_THROWS_AS(expr, ...) (static_cast(0)) -#define DOCTEST_REQUIRE_THROWS_AS(expr, ...) (static_cast(0)) -#define DOCTEST_WARN_THROWS_WITH(expr, ...) (static_cast(0)) -#define DOCTEST_CHECK_THROWS_WITH(expr, ...) (static_cast(0)) -#define DOCTEST_REQUIRE_THROWS_WITH(expr, ...) (static_cast(0)) -#define DOCTEST_WARN_THROWS_WITH_AS(expr, with, ...) (static_cast(0)) -#define DOCTEST_CHECK_THROWS_WITH_AS(expr, with, ...) (static_cast(0)) -#define DOCTEST_REQUIRE_THROWS_WITH_AS(expr, with, ...) (static_cast(0)) -#define DOCTEST_WARN_NOTHROW(...) (static_cast(0)) -#define DOCTEST_CHECK_NOTHROW(...) (static_cast(0)) -#define DOCTEST_REQUIRE_NOTHROW(...) (static_cast(0)) - -#define DOCTEST_WARN_THROWS_MESSAGE(expr, ...) (static_cast(0)) -#define DOCTEST_CHECK_THROWS_MESSAGE(expr, ...) (static_cast(0)) -#define DOCTEST_REQUIRE_THROWS_MESSAGE(expr, ...) (static_cast(0)) -#define DOCTEST_WARN_THROWS_AS_MESSAGE(expr, ex, ...) (static_cast(0)) -#define DOCTEST_CHECK_THROWS_AS_MESSAGE(expr, ex, ...) (static_cast(0)) -#define DOCTEST_REQUIRE_THROWS_AS_MESSAGE(expr, ex, ...) (static_cast(0)) -#define DOCTEST_WARN_THROWS_WITH_MESSAGE(expr, with, ...) (static_cast(0)) -#define DOCTEST_CHECK_THROWS_WITH_MESSAGE(expr, with, ...) (static_cast(0)) -#define DOCTEST_REQUIRE_THROWS_WITH_MESSAGE(expr, with, ...) (static_cast(0)) -#define DOCTEST_WARN_THROWS_WITH_AS_MESSAGE(expr, with, ex, ...) (static_cast(0)) -#define DOCTEST_CHECK_THROWS_WITH_AS_MESSAGE(expr, with, ex, ...) (static_cast(0)) -#define DOCTEST_REQUIRE_THROWS_WITH_AS_MESSAGE(expr, with, ex, ...) (static_cast(0)) -#define DOCTEST_WARN_NOTHROW_MESSAGE(expr, ...) (static_cast(0)) -#define DOCTEST_CHECK_NOTHROW_MESSAGE(expr, ...) (static_cast(0)) -#define DOCTEST_REQUIRE_NOTHROW_MESSAGE(expr, ...) (static_cast(0)) - +#define DOCTEST_EXCEPTION_EMPTY_FUNC DOCTEST_FUNC_EMPTY #else // DOCTEST_CONFIG_NO_EXCEPTIONS_BUT_WITH_ALL_ASSERTS +#define DOCTEST_EXCEPTION_EMPTY_FUNC [] { static_assert(false, "Exceptions are disabled! " \ + "Use DOCTEST_CONFIG_NO_EXCEPTIONS_BUT_WITH_ALL_ASSERTS if you want to compile with exceptions disabled."); return false; }() #undef DOCTEST_REQUIRE #undef DOCTEST_REQUIRE_FALSE @@ -2333,163 +2794,55 @@ int registerReporter(const char* name, int priority, bool isReporter) { #undef DOCTEST_REQUIRE_UNARY #undef DOCTEST_REQUIRE_UNARY_FALSE +#define DOCTEST_REQUIRE DOCTEST_EXCEPTION_EMPTY_FUNC +#define DOCTEST_REQUIRE_FALSE DOCTEST_EXCEPTION_EMPTY_FUNC +#define DOCTEST_REQUIRE_MESSAGE DOCTEST_EXCEPTION_EMPTY_FUNC +#define DOCTEST_REQUIRE_FALSE_MESSAGE DOCTEST_EXCEPTION_EMPTY_FUNC +#define DOCTEST_REQUIRE_EQ DOCTEST_EXCEPTION_EMPTY_FUNC +#define DOCTEST_REQUIRE_NE DOCTEST_EXCEPTION_EMPTY_FUNC +#define DOCTEST_REQUIRE_GT DOCTEST_EXCEPTION_EMPTY_FUNC +#define DOCTEST_REQUIRE_LT DOCTEST_EXCEPTION_EMPTY_FUNC +#define DOCTEST_REQUIRE_GE DOCTEST_EXCEPTION_EMPTY_FUNC +#define DOCTEST_REQUIRE_LE DOCTEST_EXCEPTION_EMPTY_FUNC +#define DOCTEST_REQUIRE_UNARY DOCTEST_EXCEPTION_EMPTY_FUNC +#define DOCTEST_REQUIRE_UNARY_FALSE DOCTEST_EXCEPTION_EMPTY_FUNC + #endif // DOCTEST_CONFIG_NO_EXCEPTIONS_BUT_WITH_ALL_ASSERTS +#define DOCTEST_WARN_THROWS(...) DOCTEST_EXCEPTION_EMPTY_FUNC +#define DOCTEST_CHECK_THROWS(...) DOCTEST_EXCEPTION_EMPTY_FUNC +#define DOCTEST_REQUIRE_THROWS(...) DOCTEST_EXCEPTION_EMPTY_FUNC +#define DOCTEST_WARN_THROWS_AS(expr, ...) DOCTEST_EXCEPTION_EMPTY_FUNC +#define DOCTEST_CHECK_THROWS_AS(expr, ...) DOCTEST_EXCEPTION_EMPTY_FUNC +#define DOCTEST_REQUIRE_THROWS_AS(expr, ...) DOCTEST_EXCEPTION_EMPTY_FUNC +#define DOCTEST_WARN_THROWS_WITH(expr, ...) DOCTEST_EXCEPTION_EMPTY_FUNC +#define DOCTEST_CHECK_THROWS_WITH(expr, ...) DOCTEST_EXCEPTION_EMPTY_FUNC +#define DOCTEST_REQUIRE_THROWS_WITH(expr, ...) DOCTEST_EXCEPTION_EMPTY_FUNC +#define DOCTEST_WARN_THROWS_WITH_AS(expr, with, ...) DOCTEST_EXCEPTION_EMPTY_FUNC +#define DOCTEST_CHECK_THROWS_WITH_AS(expr, with, ...) DOCTEST_EXCEPTION_EMPTY_FUNC +#define DOCTEST_REQUIRE_THROWS_WITH_AS(expr, with, ...) DOCTEST_EXCEPTION_EMPTY_FUNC +#define DOCTEST_WARN_NOTHROW(...) DOCTEST_EXCEPTION_EMPTY_FUNC +#define DOCTEST_CHECK_NOTHROW(...) DOCTEST_EXCEPTION_EMPTY_FUNC +#define DOCTEST_REQUIRE_NOTHROW(...) DOCTEST_EXCEPTION_EMPTY_FUNC + +#define DOCTEST_WARN_THROWS_MESSAGE(expr, ...) DOCTEST_EXCEPTION_EMPTY_FUNC +#define DOCTEST_CHECK_THROWS_MESSAGE(expr, ...) DOCTEST_EXCEPTION_EMPTY_FUNC +#define DOCTEST_REQUIRE_THROWS_MESSAGE(expr, ...) DOCTEST_EXCEPTION_EMPTY_FUNC +#define DOCTEST_WARN_THROWS_AS_MESSAGE(expr, ex, ...) DOCTEST_EXCEPTION_EMPTY_FUNC +#define DOCTEST_CHECK_THROWS_AS_MESSAGE(expr, ex, ...) DOCTEST_EXCEPTION_EMPTY_FUNC +#define DOCTEST_REQUIRE_THROWS_AS_MESSAGE(expr, ex, ...) DOCTEST_EXCEPTION_EMPTY_FUNC +#define DOCTEST_WARN_THROWS_WITH_MESSAGE(expr, with, ...) DOCTEST_EXCEPTION_EMPTY_FUNC +#define DOCTEST_CHECK_THROWS_WITH_MESSAGE(expr, with, ...) DOCTEST_EXCEPTION_EMPTY_FUNC +#define DOCTEST_REQUIRE_THROWS_WITH_MESSAGE(expr, with, ...) DOCTEST_EXCEPTION_EMPTY_FUNC +#define DOCTEST_WARN_THROWS_WITH_AS_MESSAGE(expr, with, ex, ...) DOCTEST_EXCEPTION_EMPTY_FUNC +#define DOCTEST_CHECK_THROWS_WITH_AS_MESSAGE(expr, with, ex, ...) DOCTEST_EXCEPTION_EMPTY_FUNC +#define DOCTEST_REQUIRE_THROWS_WITH_AS_MESSAGE(expr, with, ex, ...) DOCTEST_EXCEPTION_EMPTY_FUNC +#define DOCTEST_WARN_NOTHROW_MESSAGE(expr, ...) DOCTEST_EXCEPTION_EMPTY_FUNC +#define DOCTEST_CHECK_NOTHROW_MESSAGE(expr, ...) DOCTEST_EXCEPTION_EMPTY_FUNC +#define DOCTEST_REQUIRE_NOTHROW_MESSAGE(expr, ...) DOCTEST_EXCEPTION_EMPTY_FUNC + #endif // DOCTEST_CONFIG_NO_EXCEPTIONS -// ================================================================================================= -// == WHAT FOLLOWS IS VERSIONS OF THE MACROS THAT DO NOT DO ANY REGISTERING! == -// == THIS CAN BE ENABLED BY DEFINING DOCTEST_CONFIG_DISABLE GLOBALLY! == -// ================================================================================================= -#else // DOCTEST_CONFIG_DISABLE - -#define DOCTEST_IMPLEMENT_FIXTURE(der, base, func, name) \ - namespace { \ - template \ - struct der : public base \ - { void f(); }; \ - } \ - template \ - inline void der::f() - -#define DOCTEST_CREATE_AND_REGISTER_FUNCTION(f, name) \ - template \ - static inline void f() - -// for registering tests -#define DOCTEST_TEST_CASE(name) \ - DOCTEST_CREATE_AND_REGISTER_FUNCTION(DOCTEST_ANONYMOUS(_DOCTEST_ANON_FUNC_), name) - -// for registering tests in classes -#define DOCTEST_TEST_CASE_CLASS(name) \ - DOCTEST_CREATE_AND_REGISTER_FUNCTION(DOCTEST_ANONYMOUS(_DOCTEST_ANON_FUNC_), name) - -// for registering tests with a fixture -#define DOCTEST_TEST_CASE_FIXTURE(x, name) \ - DOCTEST_IMPLEMENT_FIXTURE(DOCTEST_ANONYMOUS(_DOCTEST_ANON_CLASS_), x, \ - DOCTEST_ANONYMOUS(_DOCTEST_ANON_FUNC_), name) - -// for converting types to strings without the header and demangling -#define DOCTEST_TYPE_TO_STRING(...) typedef int DOCTEST_ANONYMOUS(_DOCTEST_ANON_FOR_SEMICOLON_) -#define DOCTEST_TYPE_TO_STRING_IMPL(...) - -// for typed tests -#define DOCTEST_TEST_CASE_TEMPLATE(name, type, ...) \ - template \ - inline void DOCTEST_ANONYMOUS(_DOCTEST_ANON_TMP_)() - -#define DOCTEST_TEST_CASE_TEMPLATE_DEFINE(name, type, id) \ - template \ - inline void DOCTEST_ANONYMOUS(_DOCTEST_ANON_TMP_)() - -#define DOCTEST_TEST_CASE_TEMPLATE_INVOKE(id, ...) \ - typedef int DOCTEST_ANONYMOUS(_DOCTEST_ANON_FOR_SEMICOLON_) - -#define DOCTEST_TEST_CASE_TEMPLATE_APPLY(id, ...) \ - typedef int DOCTEST_ANONYMOUS(_DOCTEST_ANON_FOR_SEMICOLON_) - -// for subcases -#define DOCTEST_SUBCASE(name) - -// for a testsuite block -#define DOCTEST_TEST_SUITE(name) namespace - -// for starting a testsuite block -#define DOCTEST_TEST_SUITE_BEGIN(name) typedef int DOCTEST_ANONYMOUS(_DOCTEST_ANON_FOR_SEMICOLON_) - -// for ending a testsuite block -#define DOCTEST_TEST_SUITE_END typedef int DOCTEST_ANONYMOUS(_DOCTEST_ANON_FOR_SEMICOLON_) - -#define DOCTEST_REGISTER_EXCEPTION_TRANSLATOR(signature) \ - template \ - static inline doctest::String DOCTEST_ANONYMOUS(_DOCTEST_ANON_TRANSLATOR_)(signature) - -#define DOCTEST_REGISTER_REPORTER(name, priority, reporter) -#define DOCTEST_REGISTER_LISTENER(name, priority, reporter) - -#define DOCTEST_INFO(...) (static_cast(0)) -#define DOCTEST_CAPTURE(x) (static_cast(0)) -#define DOCTEST_ADD_MESSAGE_AT(file, line, ...) (static_cast(0)) -#define DOCTEST_ADD_FAIL_CHECK_AT(file, line, ...) (static_cast(0)) -#define DOCTEST_ADD_FAIL_AT(file, line, ...) (static_cast(0)) -#define DOCTEST_MESSAGE(...) (static_cast(0)) -#define DOCTEST_FAIL_CHECK(...) (static_cast(0)) -#define DOCTEST_FAIL(...) (static_cast(0)) - -#define DOCTEST_WARN(...) (static_cast(0)) -#define DOCTEST_CHECK(...) (static_cast(0)) -#define DOCTEST_REQUIRE(...) (static_cast(0)) -#define DOCTEST_WARN_FALSE(...) (static_cast(0)) -#define DOCTEST_CHECK_FALSE(...) (static_cast(0)) -#define DOCTEST_REQUIRE_FALSE(...) (static_cast(0)) - -#define DOCTEST_WARN_MESSAGE(cond, ...) (static_cast(0)) -#define DOCTEST_CHECK_MESSAGE(cond, ...) (static_cast(0)) -#define DOCTEST_REQUIRE_MESSAGE(cond, ...) (static_cast(0)) -#define DOCTEST_WARN_FALSE_MESSAGE(cond, ...) (static_cast(0)) -#define DOCTEST_CHECK_FALSE_MESSAGE(cond, ...) (static_cast(0)) -#define DOCTEST_REQUIRE_FALSE_MESSAGE(cond, ...) (static_cast(0)) - -#define DOCTEST_WARN_THROWS(...) (static_cast(0)) -#define DOCTEST_CHECK_THROWS(...) (static_cast(0)) -#define DOCTEST_REQUIRE_THROWS(...) (static_cast(0)) -#define DOCTEST_WARN_THROWS_AS(expr, ...) (static_cast(0)) -#define DOCTEST_CHECK_THROWS_AS(expr, ...) (static_cast(0)) -#define DOCTEST_REQUIRE_THROWS_AS(expr, ...) (static_cast(0)) -#define DOCTEST_WARN_THROWS_WITH(expr, ...) (static_cast(0)) -#define DOCTEST_CHECK_THROWS_WITH(expr, ...) (static_cast(0)) -#define DOCTEST_REQUIRE_THROWS_WITH(expr, ...) (static_cast(0)) -#define DOCTEST_WARN_THROWS_WITH_AS(expr, with, ...) (static_cast(0)) -#define DOCTEST_CHECK_THROWS_WITH_AS(expr, with, ...) (static_cast(0)) -#define DOCTEST_REQUIRE_THROWS_WITH_AS(expr, with, ...) (static_cast(0)) -#define DOCTEST_WARN_NOTHROW(...) (static_cast(0)) -#define DOCTEST_CHECK_NOTHROW(...) (static_cast(0)) -#define DOCTEST_REQUIRE_NOTHROW(...) (static_cast(0)) - -#define DOCTEST_WARN_THROWS_MESSAGE(expr, ...) (static_cast(0)) -#define DOCTEST_CHECK_THROWS_MESSAGE(expr, ...) (static_cast(0)) -#define DOCTEST_REQUIRE_THROWS_MESSAGE(expr, ...) (static_cast(0)) -#define DOCTEST_WARN_THROWS_AS_MESSAGE(expr, ex, ...) (static_cast(0)) -#define DOCTEST_CHECK_THROWS_AS_MESSAGE(expr, ex, ...) (static_cast(0)) -#define DOCTEST_REQUIRE_THROWS_AS_MESSAGE(expr, ex, ...) (static_cast(0)) -#define DOCTEST_WARN_THROWS_WITH_MESSAGE(expr, with, ...) (static_cast(0)) -#define DOCTEST_CHECK_THROWS_WITH_MESSAGE(expr, with, ...) (static_cast(0)) -#define DOCTEST_REQUIRE_THROWS_WITH_MESSAGE(expr, with, ...) (static_cast(0)) -#define DOCTEST_WARN_THROWS_WITH_AS_MESSAGE(expr, with, ex, ...) (static_cast(0)) -#define DOCTEST_CHECK_THROWS_WITH_AS_MESSAGE(expr, with, ex, ...) (static_cast(0)) -#define DOCTEST_REQUIRE_THROWS_WITH_AS_MESSAGE(expr, with, ex, ...) (static_cast(0)) -#define DOCTEST_WARN_NOTHROW_MESSAGE(expr, ...) (static_cast(0)) -#define DOCTEST_CHECK_NOTHROW_MESSAGE(expr, ...) (static_cast(0)) -#define DOCTEST_REQUIRE_NOTHROW_MESSAGE(expr, ...) (static_cast(0)) - -#define DOCTEST_WARN_EQ(...) (static_cast(0)) -#define DOCTEST_CHECK_EQ(...) (static_cast(0)) -#define DOCTEST_REQUIRE_EQ(...) (static_cast(0)) -#define DOCTEST_WARN_NE(...) (static_cast(0)) -#define DOCTEST_CHECK_NE(...) (static_cast(0)) -#define DOCTEST_REQUIRE_NE(...) (static_cast(0)) -#define DOCTEST_WARN_GT(...) (static_cast(0)) -#define DOCTEST_CHECK_GT(...) (static_cast(0)) -#define DOCTEST_REQUIRE_GT(...) (static_cast(0)) -#define DOCTEST_WARN_LT(...) (static_cast(0)) -#define DOCTEST_CHECK_LT(...) (static_cast(0)) -#define DOCTEST_REQUIRE_LT(...) (static_cast(0)) -#define DOCTEST_WARN_GE(...) (static_cast(0)) -#define DOCTEST_CHECK_GE(...) (static_cast(0)) -#define DOCTEST_REQUIRE_GE(...) (static_cast(0)) -#define DOCTEST_WARN_LE(...) (static_cast(0)) -#define DOCTEST_CHECK_LE(...) (static_cast(0)) -#define DOCTEST_REQUIRE_LE(...) (static_cast(0)) - -#define DOCTEST_WARN_UNARY(...) (static_cast(0)) -#define DOCTEST_CHECK_UNARY(...) (static_cast(0)) -#define DOCTEST_REQUIRE_UNARY(...) (static_cast(0)) -#define DOCTEST_WARN_UNARY_FALSE(...) (static_cast(0)) -#define DOCTEST_CHECK_UNARY_FALSE(...) (static_cast(0)) -#define DOCTEST_REQUIRE_UNARY_FALSE(...) (static_cast(0)) - -#endif // DOCTEST_CONFIG_DISABLE - // clang-format off // KEPT FOR BACKWARDS COMPATIBILITY - FORWARDING TO THE RIGHT MACROS #define DOCTEST_FAST_WARN_EQ DOCTEST_WARN_EQ @@ -2536,11 +2889,12 @@ int registerReporter(const char* name, int priority, bool isReporter) { // clang-format on // == SHORT VERSIONS OF THE MACROS -#if !defined(DOCTEST_CONFIG_NO_SHORT_MACRO_NAMES) +#ifndef DOCTEST_CONFIG_NO_SHORT_MACRO_NAMES #define TEST_CASE(name) DOCTEST_TEST_CASE(name) #define TEST_CASE_CLASS(name) DOCTEST_TEST_CASE_CLASS(name) #define TEST_CASE_FIXTURE(x, name) DOCTEST_TEST_CASE_FIXTURE(x, name) +#define TYPE_TO_STRING_AS(str, ...) DOCTEST_TYPE_TO_STRING_AS(str, __VA_ARGS__) #define TYPE_TO_STRING(...) DOCTEST_TYPE_TO_STRING(__VA_ARGS__) #define TEST_CASE_TEMPLATE(name, T, ...) DOCTEST_TEST_CASE_TEMPLATE(name, T, __VA_ARGS__) #define TEST_CASE_TEMPLATE_DEFINE(name, T, id) DOCTEST_TEST_CASE_TEMPLATE_DEFINE(name, T, id) @@ -2673,39 +3027,19 @@ int registerReporter(const char* name, int priority, bool isReporter) { #endif // DOCTEST_CONFIG_NO_SHORT_MACRO_NAMES -#if !defined(DOCTEST_CONFIG_DISABLE) +#ifndef DOCTEST_CONFIG_DISABLE // this is here to clear the 'current test suite' for the current translation unit - at the top DOCTEST_TEST_SUITE_END(); -// add stringification for primitive/fundamental types -namespace doctest { namespace detail { - DOCTEST_TYPE_TO_STRING_IMPL(bool) - DOCTEST_TYPE_TO_STRING_IMPL(float) - DOCTEST_TYPE_TO_STRING_IMPL(double) - DOCTEST_TYPE_TO_STRING_IMPL(long double) - DOCTEST_TYPE_TO_STRING_IMPL(char) - DOCTEST_TYPE_TO_STRING_IMPL(signed char) - DOCTEST_TYPE_TO_STRING_IMPL(unsigned char) -#if !DOCTEST_MSVC || defined(_NATIVE_WCHAR_T_DEFINED) - DOCTEST_TYPE_TO_STRING_IMPL(wchar_t) -#endif // not MSVC or wchar_t support enabled - DOCTEST_TYPE_TO_STRING_IMPL(short int) - DOCTEST_TYPE_TO_STRING_IMPL(unsigned short int) - DOCTEST_TYPE_TO_STRING_IMPL(int) - DOCTEST_TYPE_TO_STRING_IMPL(unsigned int) - DOCTEST_TYPE_TO_STRING_IMPL(long int) - DOCTEST_TYPE_TO_STRING_IMPL(unsigned long int) - DOCTEST_TYPE_TO_STRING_IMPL(long long int) - DOCTEST_TYPE_TO_STRING_IMPL(unsigned long long int) -}} // namespace doctest::detail - #endif // DOCTEST_CONFIG_DISABLE DOCTEST_CLANG_SUPPRESS_WARNING_POP DOCTEST_MSVC_SUPPRESS_WARNING_POP DOCTEST_GCC_SUPPRESS_WARNING_POP +DOCTEST_SUPPRESS_COMMON_WARNINGS_POP + #endif // DOCTEST_LIBRARY_INCLUDED #ifndef DOCTEST_SINGLE_HEADER @@ -2725,13 +3059,11 @@ DOCTEST_CLANG_SUPPRESS_WARNING_WITH_PUSH("-Wunused-macros") DOCTEST_CLANG_SUPPRESS_WARNING_POP +DOCTEST_SUPPRESS_COMMON_WARNINGS_PUSH + DOCTEST_CLANG_SUPPRESS_WARNING_PUSH -DOCTEST_CLANG_SUPPRESS_WARNING("-Wunknown-pragmas") -DOCTEST_CLANG_SUPPRESS_WARNING("-Wpadded") -DOCTEST_CLANG_SUPPRESS_WARNING("-Wweak-vtables") DOCTEST_CLANG_SUPPRESS_WARNING("-Wglobal-constructors") DOCTEST_CLANG_SUPPRESS_WARNING("-Wexit-time-destructors") -DOCTEST_CLANG_SUPPRESS_WARNING("-Wmissing-prototypes") DOCTEST_CLANG_SUPPRESS_WARNING("-Wsign-conversion") DOCTEST_CLANG_SUPPRESS_WARNING("-Wshorten-64-to-32") DOCTEST_CLANG_SUPPRESS_WARNING("-Wmissing-variable-declarations") @@ -2739,65 +3071,35 @@ DOCTEST_CLANG_SUPPRESS_WARNING("-Wswitch") DOCTEST_CLANG_SUPPRESS_WARNING("-Wswitch-enum") DOCTEST_CLANG_SUPPRESS_WARNING("-Wcovered-switch-default") DOCTEST_CLANG_SUPPRESS_WARNING("-Wmissing-noreturn") -DOCTEST_CLANG_SUPPRESS_WARNING("-Wunused-local-typedef") DOCTEST_CLANG_SUPPRESS_WARNING("-Wdisabled-macro-expansion") DOCTEST_CLANG_SUPPRESS_WARNING("-Wmissing-braces") DOCTEST_CLANG_SUPPRESS_WARNING("-Wmissing-field-initializers") -DOCTEST_CLANG_SUPPRESS_WARNING("-Wc++98-compat") -DOCTEST_CLANG_SUPPRESS_WARNING("-Wc++98-compat-pedantic") DOCTEST_CLANG_SUPPRESS_WARNING("-Wunused-member-function") DOCTEST_CLANG_SUPPRESS_WARNING("-Wnonportable-system-include-path") DOCTEST_GCC_SUPPRESS_WARNING_PUSH -DOCTEST_GCC_SUPPRESS_WARNING("-Wunknown-pragmas") -DOCTEST_GCC_SUPPRESS_WARNING("-Wpragmas") DOCTEST_GCC_SUPPRESS_WARNING("-Wconversion") -DOCTEST_GCC_SUPPRESS_WARNING("-Weffc++") DOCTEST_GCC_SUPPRESS_WARNING("-Wsign-conversion") -DOCTEST_GCC_SUPPRESS_WARNING("-Wstrict-overflow") -DOCTEST_GCC_SUPPRESS_WARNING("-Wstrict-aliasing") DOCTEST_GCC_SUPPRESS_WARNING("-Wmissing-field-initializers") DOCTEST_GCC_SUPPRESS_WARNING("-Wmissing-braces") -DOCTEST_GCC_SUPPRESS_WARNING("-Wmissing-declarations") DOCTEST_GCC_SUPPRESS_WARNING("-Wswitch") DOCTEST_GCC_SUPPRESS_WARNING("-Wswitch-enum") DOCTEST_GCC_SUPPRESS_WARNING("-Wswitch-default") DOCTEST_GCC_SUPPRESS_WARNING("-Wunsafe-loop-optimizations") DOCTEST_GCC_SUPPRESS_WARNING("-Wold-style-cast") -DOCTEST_GCC_SUPPRESS_WARNING("-Wunused-local-typedefs") -DOCTEST_GCC_SUPPRESS_WARNING("-Wuseless-cast") DOCTEST_GCC_SUPPRESS_WARNING("-Wunused-function") DOCTEST_GCC_SUPPRESS_WARNING("-Wmultiple-inheritance") -DOCTEST_GCC_SUPPRESS_WARNING("-Wnoexcept") DOCTEST_GCC_SUPPRESS_WARNING("-Wsuggest-attribute") DOCTEST_MSVC_SUPPRESS_WARNING_PUSH -DOCTEST_MSVC_SUPPRESS_WARNING(4616) // invalid compiler warning -DOCTEST_MSVC_SUPPRESS_WARNING(4619) // invalid compiler warning -DOCTEST_MSVC_SUPPRESS_WARNING(4996) // The compiler encountered a deprecated declaration DOCTEST_MSVC_SUPPRESS_WARNING(4267) // 'var' : conversion from 'x' to 'y', possible loss of data -DOCTEST_MSVC_SUPPRESS_WARNING(4706) // assignment within conditional expression -DOCTEST_MSVC_SUPPRESS_WARNING(4512) // 'class' : assignment operator could not be generated -DOCTEST_MSVC_SUPPRESS_WARNING(4127) // conditional expression is constant DOCTEST_MSVC_SUPPRESS_WARNING(4530) // C++ exception handler used, but unwind semantics not enabled DOCTEST_MSVC_SUPPRESS_WARNING(4577) // 'noexcept' used with no exception handling mode specified DOCTEST_MSVC_SUPPRESS_WARNING(4774) // format string expected in argument is not a string literal DOCTEST_MSVC_SUPPRESS_WARNING(4365) // conversion from 'int' to 'unsigned', signed/unsigned mismatch -DOCTEST_MSVC_SUPPRESS_WARNING(4820) // padding in structs -DOCTEST_MSVC_SUPPRESS_WARNING(4640) // construction of local static object is not thread-safe DOCTEST_MSVC_SUPPRESS_WARNING(5039) // pointer to potentially throwing function passed to extern C -DOCTEST_MSVC_SUPPRESS_WARNING(5045) // Spectre mitigation stuff -DOCTEST_MSVC_SUPPRESS_WARNING(4626) // assignment operator was implicitly defined as deleted -DOCTEST_MSVC_SUPPRESS_WARNING(5027) // move assignment operator was implicitly defined as deleted -DOCTEST_MSVC_SUPPRESS_WARNING(5026) // move constructor was implicitly defined as deleted -DOCTEST_MSVC_SUPPRESS_WARNING(4625) // copy constructor was implicitly defined as deleted DOCTEST_MSVC_SUPPRESS_WARNING(4800) // forcing value to bool 'true' or 'false' (performance warning) -// static analysis -DOCTEST_MSVC_SUPPRESS_WARNING(26439) // This kind of function may not throw. Declare it 'noexcept' -DOCTEST_MSVC_SUPPRESS_WARNING(26495) // Always initialize a member variable -DOCTEST_MSVC_SUPPRESS_WARNING(26451) // Arithmetic overflow ... -DOCTEST_MSVC_SUPPRESS_WARNING(26444) // Avoid unnamed objects with custom construction and dtor... -DOCTEST_MSVC_SUPPRESS_WARNING(26812) // Prefer 'enum class' over 'enum' +DOCTEST_MSVC_SUPPRESS_WARNING(5245) // unreferenced function with internal linkage has been removed DOCTEST_MAKE_STD_HEADERS_CLEAN_FROM_WARNINGS_ON_WALL_BEGIN @@ -2805,7 +3107,7 @@ DOCTEST_MAKE_STD_HEADERS_CLEAN_FROM_WARNINGS_ON_WALL_BEGIN #include #include #include -// borland (Embarcadero) compiler requires math.h and not cmath - https://github.com/onqtam/doctest/pull/37 +// borland (Embarcadero) compiler requires math.h and not cmath - https://github.com/doctest/doctest/pull/37 #ifdef __BORLANDC__ #include #endif // __BORLANDC__ @@ -2821,16 +3123,27 @@ DOCTEST_MAKE_STD_HEADERS_CLEAN_FROM_WARNINGS_ON_WALL_BEGIN #include #include #include +#ifndef DOCTEST_CONFIG_NO_MULTITHREADING #include #include +#define DOCTEST_DECLARE_MUTEX(name) std::mutex name; +#define DOCTEST_DECLARE_STATIC_MUTEX(name) static DOCTEST_DECLARE_MUTEX(name) +#define DOCTEST_LOCK_MUTEX(name) std::lock_guard DOCTEST_ANONYMOUS(DOCTEST_ANON_LOCK_)(name); +#else // DOCTEST_CONFIG_NO_MULTITHREADING +#define DOCTEST_DECLARE_MUTEX(name) +#define DOCTEST_DECLARE_STATIC_MUTEX(name) +#define DOCTEST_LOCK_MUTEX(name) +#endif // DOCTEST_CONFIG_NO_MULTITHREADING #include #include +#include #include #include #include #include #include #include +#include #ifdef DOCTEST_PLATFORM_MAC #include @@ -2863,7 +3176,7 @@ DOCTEST_MAKE_STD_HEADERS_CLEAN_FROM_WARNINGS_ON_WALL_BEGIN #endif // DOCTEST_PLATFORM_WINDOWS -// this is a fix for https://github.com/onqtam/doctest/issues/348 +// this is a fix for https://github.com/doctest/doctest/issues/348 // https://mail.gnome.org/archives/xml/2012-January/msg00000.html #if !defined(HAVE_UNISTD_H) && !defined(STDOUT_FILENO) #define STDOUT_FILENO fileno(stdout) @@ -2885,8 +3198,12 @@ DOCTEST_MAKE_STD_HEADERS_CLEAN_FROM_WARNINGS_ON_WALL_END #endif #ifndef DOCTEST_THREAD_LOCAL +#if defined(DOCTEST_CONFIG_NO_MULTITHREADING) || DOCTEST_MSVC && (DOCTEST_MSVC < DOCTEST_COMPILER(19, 0, 0)) +#define DOCTEST_THREAD_LOCAL +#else // DOCTEST_MSVC #define DOCTEST_THREAD_LOCAL thread_local -#endif +#endif // DOCTEST_MSVC +#endif // DOCTEST_THREAD_LOCAL #ifndef DOCTEST_MULTI_LANE_ATOMICS_THREAD_LANES #define DOCTEST_MULTI_LANE_ATOMICS_THREAD_LANES 32 @@ -2906,12 +3223,34 @@ DOCTEST_MAKE_STD_HEADERS_CLEAN_FROM_WARNINGS_ON_WALL_END #define DOCTEST_CONFIG_NO_MULTI_LANE_ATOMICS #endif +#ifndef DOCTEST_CDECL +#define DOCTEST_CDECL __cdecl +#endif + namespace doctest { bool is_running_in_test = false; namespace { using namespace detail; + + template + DOCTEST_NORETURN void throw_exception(Ex const& e) { +#ifndef DOCTEST_CONFIG_NO_EXCEPTIONS + throw e; +#else // DOCTEST_CONFIG_NO_EXCEPTIONS + std::cerr << "doctest will terminate because it needed to throw an exception.\n" + << "The message was: " << e.what() << '\n'; + std::terminate(); +#endif // DOCTEST_CONFIG_NO_EXCEPTIONS + } + +#ifndef DOCTEST_INTERNAL_ERROR +#define DOCTEST_INTERNAL_ERROR(msg) \ + throw_exception(std::logic_error( \ + __FILE__ ":" DOCTEST_TOSTR(__LINE__) ": Internal doctest error: " msg)) +#endif // DOCTEST_INTERNAL_ERROR + // case insensitive strcmp int stricmp(const char* a, const char* b) { for(;; a++, b++) { @@ -2921,20 +3260,6 @@ namespace { } } - template - String fpToString(T value, int precision) { - std::ostringstream oss; - oss << std::setprecision(precision) << std::fixed << value; - std::string d = oss.str(); - size_t i = d.find_last_not_of('0'); - if(i != std::string::npos && i != d.size() - 1) { - if(d[i] == '.') - i++; - d = d.substr(0, i + 1); - } - return d.c_str(); - } - struct Endianness { enum Arch @@ -2955,36 +3280,35 @@ namespace { } // namespace namespace detail { - void my_memcpy(void* dest, const void* src, unsigned num) { memcpy(dest, src, num); } + DOCTEST_THREAD_LOCAL class + { + std::vector stack; + std::stringstream ss; - String rawMemoryToString(const void* object, unsigned size) { - // Reverse order for little endian architectures - int i = 0, end = static_cast(size), inc = 1; - if(Endianness::which() == Endianness::Little) { - i = end - 1; - end = inc = -1; + public: + std::ostream* push() { + stack.push_back(ss.tellp()); + return &ss; } - unsigned const char* bytes = static_cast(object); - std::ostringstream oss; - oss << "0x" << std::setfill('0') << std::hex; - for(; i != end; i += inc) - oss << std::setw(2) << static_cast(bytes[i]); - return oss.str().c_str(); + String pop() { + if (stack.empty()) + DOCTEST_INTERNAL_ERROR("TLSS was empty when trying to pop!"); + + std::streampos pos = stack.back(); + stack.pop_back(); + unsigned sz = static_cast(ss.tellp() - pos); + ss.rdbuf()->pubseekpos(pos, std::ios::in | std::ios::out); + return String(ss, sz); + } + } g_oss; + + std::ostream* tlssPush() { + return g_oss.push(); } - DOCTEST_THREAD_LOCAL std::ostringstream g_oss; // NOLINT(cert-err58-cpp) - - std::ostream* getTlsOss() { - g_oss.clear(); // there shouldn't be anything worth clearing in the flags - g_oss.str(""); // the slow way of resetting a string stream - //g_oss.seekp(0); // optimal reset - as seen here: https://stackoverflow.com/a/624291/3162383 - return &g_oss; - } - - String getTlsOssResult() { - //g_oss << std::ends; // needed - as shown here: https://stackoverflow.com/a/624291/3162383 - return g_oss.str().c_str(); + String tlssPop() { + return g_oss.pop(); } #ifndef DOCTEST_CONFIG_DISABLE @@ -2993,20 +3317,19 @@ namespace timer_large_integer { #if defined(DOCTEST_PLATFORM_WINDOWS) - typedef ULONGLONG type; + using type = ULONGLONG; #else // DOCTEST_PLATFORM_WINDOWS - using namespace std; - typedef uint64_t type; + using type = std::uint64_t; #endif // DOCTEST_PLATFORM_WINDOWS } -typedef timer_large_integer::type ticks_t; +using ticks_t = timer_large_integer::type; #ifdef DOCTEST_CONFIG_GETCURRENTTICKS ticks_t getCurrentTicks() { return DOCTEST_CONFIG_GETCURRENTTICKS(); } #elif defined(DOCTEST_PLATFORM_WINDOWS) ticks_t getCurrentTicks() { - static LARGE_INTEGER hz = {0}, hzo = {0}; + static LARGE_INTEGER hz = { {0} }, hzo = { {0} }; if(!hz.QuadPart) { QueryPerformanceFrequency(&hz); QueryPerformanceCounter(&hzo); @@ -3038,9 +3361,17 @@ typedef timer_large_integer::type ticks_t; ticks_t m_ticks = 0; }; -#ifdef DOCTEST_CONFIG_NO_MULTI_LANE_ATOMICS +#ifdef DOCTEST_CONFIG_NO_MULTITHREADING template - using AtomicOrMultiLaneAtomic = std::atomic; + using Atomic = T; +#else // DOCTEST_CONFIG_NO_MULTITHREADING + template + using Atomic = std::atomic; +#endif // DOCTEST_CONFIG_NO_MULTITHREADING + +#if defined(DOCTEST_CONFIG_NO_MULTI_LANE_ATOMICS) || defined(DOCTEST_CONFIG_NO_MULTITHREADING) + template + using MultiLaneAtomic = Atomic; #else // DOCTEST_CONFIG_NO_MULTI_LANE_ATOMICS // Provides a multilane implementation of an atomic variable that supports add, sub, load, // store. Instead of using a single atomic variable, this splits up into multiple ones, @@ -3057,8 +3388,8 @@ typedef timer_large_integer::type ticks_t; { struct CacheLineAlignedAtomic { - std::atomic atomic{}; - char padding[DOCTEST_MULTI_LANE_ATOMICS_CACHE_LINE_SIZE - sizeof(std::atomic)]; + Atomic atomic{}; + char padding[DOCTEST_MULTI_LANE_ATOMICS_CACHE_LINE_SIZE - sizeof(Atomic)]; }; CacheLineAlignedAtomic m_atomics[DOCTEST_MULTI_LANE_ATOMICS_THREAD_LANES]; @@ -3088,7 +3419,7 @@ typedef timer_large_integer::type ticks_t; return result; } - T operator=(T desired) DOCTEST_NOEXCEPT { + T operator=(T desired) DOCTEST_NOEXCEPT { // lgtm [cpp/assignment-does-not-return-this] store(desired); return desired; } @@ -3103,7 +3434,7 @@ typedef timer_large_integer::type ticks_t; private: // Each thread has a different atomic that it operates on. If more than NumLanes threads - // use this, some will use the same atomic. So performance will degrate a bit, but still + // use this, some will use the same atomic. So performance will degrade a bit, but still // everything will work. // // The logic here is a bit tricky. The call should be as fast as possible, so that there @@ -3114,24 +3445,21 @@ typedef timer_large_integer::type ticks_t; // assigned in a round-robin fashion. // 3. This tlsLaneIdx is stored in the thread local data, so it is directly available with // little overhead. - std::atomic& myAtomic() DOCTEST_NOEXCEPT { - static std::atomic laneCounter; + Atomic& myAtomic() DOCTEST_NOEXCEPT { + static Atomic laneCounter; DOCTEST_THREAD_LOCAL size_t tlsLaneIdx = laneCounter++ % DOCTEST_MULTI_LANE_ATOMICS_THREAD_LANES; return m_atomics[tlsLaneIdx].atomic; } }; - - template - using AtomicOrMultiLaneAtomic = MultiLaneAtomic; #endif // DOCTEST_CONFIG_NO_MULTI_LANE_ATOMICS // this holds both parameters from the command line and runtime data for tests struct ContextState : ContextOptions, TestRunStats, CurrentTestCaseStats { - AtomicOrMultiLaneAtomic numAssertsCurrentTest_atomic; - AtomicOrMultiLaneAtomic numAssertsFailedCurrentTest_atomic; + MultiLaneAtomic numAssertsCurrentTest_atomic; + MultiLaneAtomic numAssertsFailedCurrentTest_atomic; std::vector> filters = decltype(filters)(9); // 9 different filters @@ -3144,11 +3472,12 @@ typedef timer_large_integer::type ticks_t; std::vector stringifiedContexts; // logging from INFO() due to an exception // stuff for subcases - std::vector subcasesStack; - std::set subcasesPassed; - int subcasesCurrentMaxLevel; - bool should_reenter; - std::atomic shouldLogCurrentException; + bool reachedLeaf; + std::vector subcaseStack; + std::vector nextSubcaseStack; + std::unordered_set fullyTraversedSubcases; + size_t currentSubcaseDepth; + Atomic shouldLogCurrentException; void resetRunData() { numTestCases = 0; @@ -3198,7 +3527,8 @@ typedef timer_large_integer::type ticks_t; (TestCaseFailureReason::FailedExactlyNumTimes & failure_flags); // if any subcase has failed - the whole test case has failed - if(failure_flags && !ok_to_fail) + testCaseSuccess = !(failure_flags && !ok_to_fail); + if(!testCaseSuccess) numTestCasesFailed++; } }; @@ -3213,23 +3543,37 @@ typedef timer_large_integer::type ticks_t; #endif // DOCTEST_CONFIG_DISABLE } // namespace detail -void String::setOnHeap() { *reinterpret_cast(&buf[last]) = 128; } -void String::setLast(unsigned in) { buf[last] = char(in); } - -void String::copy(const String& other) { - using namespace std; - if(other.isOnStack()) { - memcpy(buf, other.buf, len); +char* String::allocate(size_type sz) { + if (sz <= last) { + buf[sz] = '\0'; + setLast(last - sz); + return buf; } else { setOnHeap(); - data.size = other.data.size; + data.size = sz; data.capacity = data.size + 1; - data.ptr = new char[data.capacity]; - memcpy(data.ptr, other.data.ptr, data.size + 1); + data.ptr = new char[data.capacity]; + data.ptr[sz] = '\0'; + return data.ptr; } } -String::String() { +void String::setOnHeap() noexcept { *reinterpret_cast(&buf[last]) = 128; } +void String::setLast(size_type in) noexcept { buf[last] = char(in); } +void String::setSize(size_type sz) noexcept { + if (isOnStack()) { buf[sz] = '\0'; setLast(last - sz); } + else { data.ptr[sz] = '\0'; data.size = sz; } +} + +void String::copy(const String& other) { + if(other.isOnStack()) { + memcpy(buf, other.buf, len); + } else { + memcpy(allocate(other.data.size), other.data.ptr, other.data.size); + } +} + +String::String() noexcept { buf[0] = '\0'; setLast(); } @@ -3237,26 +3581,17 @@ String::String() { String::~String() { if(!isOnStack()) delete[] data.ptr; - // NOLINTNEXTLINE(clang-analyzer-cplusplus.NewDeleteLeaks) -} +} // NOLINT(clang-analyzer-cplusplus.NewDeleteLeaks) String::String(const char* in) : String(in, strlen(in)) {} -String::String(const char* in, unsigned in_size) { - using namespace std; - if(in_size <= last) { - memcpy(buf, in, in_size); - buf[in_size] = '\0'; - setLast(last - in_size); - } else { - setOnHeap(); - data.size = in_size; - data.capacity = data.size + 1; - data.ptr = new char[data.capacity]; - memcpy(data.ptr, in, in_size); - data.ptr[in_size] = '\0'; - } +String::String(const char* in, size_type in_size) { + memcpy(allocate(in_size), in, in_size); +} + +String::String(std::istream& in, size_type in_size) { + in.read(allocate(in_size), in_size); } String::String(const String& other) { copy(other); } @@ -3273,10 +3608,9 @@ String& String::operator=(const String& other) { } String& String::operator+=(const String& other) { - const unsigned my_old_size = size(); - const unsigned other_size = other.size(); - const unsigned total_size = my_old_size + other_size; - using namespace std; + const size_type my_old_size = size(); + const size_type other_size = other.size(); + const size_type total_size = my_old_size + other_size; if(isOnStack()) { if(total_size < len) { // append to the current stack space @@ -3323,18 +3657,13 @@ String& String::operator+=(const String& other) { return *this; } -// NOLINTNEXTLINE(clang-analyzer-cplusplus.NewDeleteLeaks) -String String::operator+(const String& other) const { return String(*this) += other; } - -String::String(String&& other) { - using namespace std; +String::String(String&& other) noexcept { memcpy(buf, other.buf, len); other.buf[0] = '\0'; other.setLast(); } -String& String::operator=(String&& other) { - using namespace std; +String& String::operator=(String&& other) noexcept { if(this != &other) { if(!isOnStack()) delete[] data.ptr; @@ -3345,30 +3674,60 @@ String& String::operator=(String&& other) { return *this; } -char String::operator[](unsigned i) const { - return const_cast(this)->operator[](i); // NOLINT +char String::operator[](size_type i) const { + return const_cast(this)->operator[](i); } -char& String::operator[](unsigned i) { +char& String::operator[](size_type i) { if(isOnStack()) return reinterpret_cast(buf)[i]; return data.ptr[i]; } DOCTEST_GCC_SUPPRESS_WARNING_WITH_PUSH("-Wmaybe-uninitialized") -unsigned String::size() const { +String::size_type String::size() const { if(isOnStack()) - return last - (unsigned(buf[last]) & 31); // using "last" would work only if "len" is 32 + return last - (size_type(buf[last]) & 31); // using "last" would work only if "len" is 32 return data.size; } DOCTEST_GCC_SUPPRESS_WARNING_POP -unsigned String::capacity() const { +String::size_type String::capacity() const { if(isOnStack()) return len; return data.capacity; } +String String::substr(size_type pos, size_type cnt) && { + cnt = std::min(cnt, size() - 1 - pos); + char* cptr = c_str(); + memmove(cptr, cptr + pos, cnt); + setSize(cnt); + return std::move(*this); +} + +String String::substr(size_type pos, size_type cnt) const & { + cnt = std::min(cnt, size() - 1 - pos); + return String{ c_str() + pos, cnt }; +} + +String::size_type String::find(char ch, size_type pos) const { + const char* begin = c_str(); + const char* end = begin + size(); + const char* it = begin + pos; + for (; it < end && *it != ch; it++); + if (it < end) { return static_cast(it - begin); } + else { return npos; } +} + +String::size_type String::rfind(char ch, size_type pos) const { + const char* begin = c_str(); + const char* it = begin + std::min(pos, size() - 1); + for (; it >= begin && *it != ch; it--); + if (it >= begin) { return static_cast(it - begin); } + else { return npos; } +} + int String::compare(const char* other, bool no_case) const { if(no_case) return doctest::stricmp(c_str(), other); @@ -3379,17 +3738,32 @@ int String::compare(const String& other, bool no_case) const { return compare(other.c_str(), no_case); } -// clang-format off +String operator+(const String& lhs, const String& rhs) { return String(lhs) += rhs; } + bool operator==(const String& lhs, const String& rhs) { return lhs.compare(rhs) == 0; } bool operator!=(const String& lhs, const String& rhs) { return lhs.compare(rhs) != 0; } bool operator< (const String& lhs, const String& rhs) { return lhs.compare(rhs) < 0; } bool operator> (const String& lhs, const String& rhs) { return lhs.compare(rhs) > 0; } bool operator<=(const String& lhs, const String& rhs) { return (lhs != rhs) ? lhs.compare(rhs) < 0 : true; } bool operator>=(const String& lhs, const String& rhs) { return (lhs != rhs) ? lhs.compare(rhs) > 0 : true; } -// clang-format on std::ostream& operator<<(std::ostream& s, const String& in) { return s << in.c_str(); } +Contains::Contains(const String& str) : string(str) { } + +bool Contains::checkWith(const String& other) const { + return strstr(other.c_str(), string.c_str()) != nullptr; +} + +String toString(const Contains& in) { + return "Contains( " + in.string + " )"; +} + +bool operator==(const String& lhs, const Contains& rhs) { return rhs.checkWith(lhs); } +bool operator==(const Contains& lhs, const String& rhs) { return lhs.checkWith(rhs); } +bool operator!=(const String& lhs, const Contains& rhs) { return !rhs.checkWith(lhs); } +bool operator!=(const Contains& lhs, const String& rhs) { return !lhs.checkWith(rhs); } + namespace { void color_to_stream(std::ostream&, Color::Enum) DOCTEST_BRANCH_ON_DISABLED({}, ;) } // namespace @@ -3403,64 +3777,42 @@ namespace Color { // clang-format off const char* assertString(assertType::Enum at) { - DOCTEST_MSVC_SUPPRESS_WARNING_WITH_PUSH(4062) // enum 'x' in switch of enum 'y' is not handled - switch(at) { //!OCLINT missing default in switch statements - case assertType::DT_WARN : return "WARN"; - case assertType::DT_CHECK : return "CHECK"; - case assertType::DT_REQUIRE : return "REQUIRE"; + DOCTEST_MSVC_SUPPRESS_WARNING_WITH_PUSH(4061) // enum 'x' in switch of enum 'y' is not explicitely handled + #define DOCTEST_GENERATE_ASSERT_TYPE_CASE(assert_type) case assertType::DT_ ## assert_type: return #assert_type + #define DOCTEST_GENERATE_ASSERT_TYPE_CASES(assert_type) \ + DOCTEST_GENERATE_ASSERT_TYPE_CASE(WARN_ ## assert_type); \ + DOCTEST_GENERATE_ASSERT_TYPE_CASE(CHECK_ ## assert_type); \ + DOCTEST_GENERATE_ASSERT_TYPE_CASE(REQUIRE_ ## assert_type) + switch(at) { + DOCTEST_GENERATE_ASSERT_TYPE_CASE(WARN); + DOCTEST_GENERATE_ASSERT_TYPE_CASE(CHECK); + DOCTEST_GENERATE_ASSERT_TYPE_CASE(REQUIRE); - case assertType::DT_WARN_FALSE : return "WARN_FALSE"; - case assertType::DT_CHECK_FALSE : return "CHECK_FALSE"; - case assertType::DT_REQUIRE_FALSE : return "REQUIRE_FALSE"; + DOCTEST_GENERATE_ASSERT_TYPE_CASES(FALSE); - case assertType::DT_WARN_THROWS : return "WARN_THROWS"; - case assertType::DT_CHECK_THROWS : return "CHECK_THROWS"; - case assertType::DT_REQUIRE_THROWS : return "REQUIRE_THROWS"; + DOCTEST_GENERATE_ASSERT_TYPE_CASES(THROWS); - case assertType::DT_WARN_THROWS_AS : return "WARN_THROWS_AS"; - case assertType::DT_CHECK_THROWS_AS : return "CHECK_THROWS_AS"; - case assertType::DT_REQUIRE_THROWS_AS : return "REQUIRE_THROWS_AS"; + DOCTEST_GENERATE_ASSERT_TYPE_CASES(THROWS_AS); - case assertType::DT_WARN_THROWS_WITH : return "WARN_THROWS_WITH"; - case assertType::DT_CHECK_THROWS_WITH : return "CHECK_THROWS_WITH"; - case assertType::DT_REQUIRE_THROWS_WITH : return "REQUIRE_THROWS_WITH"; + DOCTEST_GENERATE_ASSERT_TYPE_CASES(THROWS_WITH); - case assertType::DT_WARN_THROWS_WITH_AS : return "WARN_THROWS_WITH_AS"; - case assertType::DT_CHECK_THROWS_WITH_AS : return "CHECK_THROWS_WITH_AS"; - case assertType::DT_REQUIRE_THROWS_WITH_AS : return "REQUIRE_THROWS_WITH_AS"; + DOCTEST_GENERATE_ASSERT_TYPE_CASES(THROWS_WITH_AS); - case assertType::DT_WARN_NOTHROW : return "WARN_NOTHROW"; - case assertType::DT_CHECK_NOTHROW : return "CHECK_NOTHROW"; - case assertType::DT_REQUIRE_NOTHROW : return "REQUIRE_NOTHROW"; + DOCTEST_GENERATE_ASSERT_TYPE_CASES(NOTHROW); - case assertType::DT_WARN_EQ : return "WARN_EQ"; - case assertType::DT_CHECK_EQ : return "CHECK_EQ"; - case assertType::DT_REQUIRE_EQ : return "REQUIRE_EQ"; - case assertType::DT_WARN_NE : return "WARN_NE"; - case assertType::DT_CHECK_NE : return "CHECK_NE"; - case assertType::DT_REQUIRE_NE : return "REQUIRE_NE"; - case assertType::DT_WARN_GT : return "WARN_GT"; - case assertType::DT_CHECK_GT : return "CHECK_GT"; - case assertType::DT_REQUIRE_GT : return "REQUIRE_GT"; - case assertType::DT_WARN_LT : return "WARN_LT"; - case assertType::DT_CHECK_LT : return "CHECK_LT"; - case assertType::DT_REQUIRE_LT : return "REQUIRE_LT"; - case assertType::DT_WARN_GE : return "WARN_GE"; - case assertType::DT_CHECK_GE : return "CHECK_GE"; - case assertType::DT_REQUIRE_GE : return "REQUIRE_GE"; - case assertType::DT_WARN_LE : return "WARN_LE"; - case assertType::DT_CHECK_LE : return "CHECK_LE"; - case assertType::DT_REQUIRE_LE : return "REQUIRE_LE"; + DOCTEST_GENERATE_ASSERT_TYPE_CASES(EQ); + DOCTEST_GENERATE_ASSERT_TYPE_CASES(NE); + DOCTEST_GENERATE_ASSERT_TYPE_CASES(GT); + DOCTEST_GENERATE_ASSERT_TYPE_CASES(LT); + DOCTEST_GENERATE_ASSERT_TYPE_CASES(GE); + DOCTEST_GENERATE_ASSERT_TYPE_CASES(LE); - case assertType::DT_WARN_UNARY : return "WARN_UNARY"; - case assertType::DT_CHECK_UNARY : return "CHECK_UNARY"; - case assertType::DT_REQUIRE_UNARY : return "REQUIRE_UNARY"; - case assertType::DT_WARN_UNARY_FALSE : return "WARN_UNARY_FALSE"; - case assertType::DT_CHECK_UNARY_FALSE : return "CHECK_UNARY_FALSE"; - case assertType::DT_REQUIRE_UNARY_FALSE : return "REQUIRE_UNARY_FALSE"; + DOCTEST_GENERATE_ASSERT_TYPE_CASES(UNARY); + DOCTEST_GENERATE_ASSERT_TYPE_CASES(UNARY_FALSE); + + default: DOCTEST_INTERNAL_ERROR("Tried stringifying invalid assert type!"); } DOCTEST_MSVC_SUPPRESS_WARNING_POP - return ""; } // clang-format on @@ -3494,6 +3846,12 @@ const char* skipPathFromFilename(const char* file) { DOCTEST_CLANG_SUPPRESS_WARNING_POP DOCTEST_GCC_SUPPRESS_WARNING_POP +bool SubcaseSignature::operator==(const SubcaseSignature& other) const { + return m_line == other.m_line + && std::strcmp(m_file, other.m_file) == 0 + && m_name == other.m_name; +} + bool SubcaseSignature::operator<(const SubcaseSignature& other) const { if(m_line != other.m_line) return m_line < other.m_line; @@ -3502,45 +3860,53 @@ bool SubcaseSignature::operator<(const SubcaseSignature& other) const { return m_name.compare(other.m_name) < 0; } -IContextScope::IContextScope() = default; -IContextScope::~IContextScope() = default; +DOCTEST_DEFINE_INTERFACE(IContextScope) -#ifdef DOCTEST_CONFIG_TREAT_CHAR_STAR_AS_STRING -String toString(char* in) { return toString(static_cast(in)); } -// NOLINTNEXTLINE(clang-analyzer-cplusplus.NewDeleteLeaks) -String toString(const char* in) { return String("\"") + (in ? in : "{null string}") + "\""; } -#endif // DOCTEST_CONFIG_TREAT_CHAR_STAR_AS_STRING -String toString(bool in) { return in ? "true" : "false"; } -String toString(float in) { return fpToString(in, 5) + "f"; } -String toString(double in) { return fpToString(in, 10); } -String toString(double long in) { return fpToString(in, 15); } - -#define DOCTEST_TO_STRING_OVERLOAD(type, fmt) \ - String toString(type in) { \ - char buf[64]; \ - std::sprintf(buf, fmt, in); \ - return buf; \ +namespace detail { + void filldata::fill(std::ostream* stream, const void* in) { + if (in) { *stream << in; } + else { *stream << "nullptr"; } } -DOCTEST_TO_STRING_OVERLOAD(char, "%d") -DOCTEST_TO_STRING_OVERLOAD(char signed, "%d") -DOCTEST_TO_STRING_OVERLOAD(char unsigned, "%u") -DOCTEST_TO_STRING_OVERLOAD(int short, "%d") -DOCTEST_TO_STRING_OVERLOAD(int short unsigned, "%u") -DOCTEST_TO_STRING_OVERLOAD(int, "%d") -DOCTEST_TO_STRING_OVERLOAD(unsigned, "%u") -DOCTEST_TO_STRING_OVERLOAD(int long, "%ld") -DOCTEST_TO_STRING_OVERLOAD(int long unsigned, "%lu") -DOCTEST_TO_STRING_OVERLOAD(int long long, "%lld") -DOCTEST_TO_STRING_OVERLOAD(int long long unsigned, "%llu") + template + String toStreamLit(T t) { + std::ostream* os = tlssPush(); + os->operator<<(t); + return tlssPop(); + } +} -String toString(std::nullptr_t) { return "NULL"; } +#ifdef DOCTEST_CONFIG_TREAT_CHAR_STAR_AS_STRING +String toString(const char* in) { return String("\"") + (in ? in : "{null string}") + "\""; } +#endif // DOCTEST_CONFIG_TREAT_CHAR_STAR_AS_STRING #if DOCTEST_MSVC >= DOCTEST_COMPILER(19, 20, 0) -// see this issue on why this is needed: https://github.com/onqtam/doctest/issues/183 +// see this issue on why this is needed: https://github.com/doctest/doctest/issues/183 String toString(const std::string& in) { return in.c_str(); } #endif // VS 2019 +String toString(String in) { return in; } + +String toString(std::nullptr_t) { return "nullptr"; } + +String toString(bool in) { return in ? "true" : "false"; } + +String toString(float in) { return toStreamLit(in); } +String toString(double in) { return toStreamLit(in); } +String toString(double long in) { return toStreamLit(in); } + +String toString(char in) { return toStreamLit(static_cast(in)); } +String toString(char signed in) { return toStreamLit(static_cast(in)); } +String toString(char unsigned in) { return toStreamLit(static_cast(in)); } +String toString(short in) { return toStreamLit(in); } +String toString(short unsigned in) { return toStreamLit(in); } +String toString(signed in) { return toStreamLit(in); } +String toString(unsigned in) { return toStreamLit(in); } +String toString(long in) { return toStreamLit(in); } +String toString(long unsigned in) { return toStreamLit(in); } +String toString(long long in) { return toStreamLit(in); } +String toString(long long unsigned in) { return toStreamLit(in); } + Approx::Approx(double value) : m_epsilon(static_cast(std::numeric_limits::epsilon()) * 100) , m_scale(1.0) @@ -3580,11 +3946,25 @@ bool operator>(double lhs, const Approx& rhs) { return lhs > rhs.m_value && lhs bool operator>(const Approx& lhs, double rhs) { return lhs.m_value > rhs && lhs != rhs; } String toString(const Approx& in) { - // NOLINTNEXTLINE(clang-analyzer-cplusplus.NewDeleteLeaks) - return String("Approx( ") + doctest::toString(in.m_value) + " )"; + return "Approx( " + doctest::toString(in.m_value) + " )"; } const ContextOptions* getContextOptions() { return DOCTEST_BRANCH_ON_DISABLED(nullptr, g_cs); } +DOCTEST_MSVC_SUPPRESS_WARNING_WITH_PUSH(4738) +template +IsNaN::operator bool() const { + return std::isnan(value) ^ flipped; +} +DOCTEST_MSVC_SUPPRESS_WARNING_POP +template struct DOCTEST_INTERFACE_DEF IsNaN; +template struct DOCTEST_INTERFACE_DEF IsNaN; +template struct DOCTEST_INTERFACE_DEF IsNaN; +template +String toString(IsNaN in) { return String(in.flipped ? "! " : "") + "IsNaN( " + doctest::toString(in.value) + " )"; } +String toString(IsNaN in) { return toString(in); } +String toString(IsNaN in) { return toString(in); } +String toString(IsNaN in) { return toString(in); } + } // namespace doctest #ifdef DOCTEST_CONFIG_DISABLE @@ -3594,15 +3974,15 @@ Context::~Context() = default; void Context::applyCommandLine(int, const char* const*) {} void Context::addFilter(const char*, const char*) {} void Context::clearFilters() {} +void Context::setOption(const char*, bool) {} void Context::setOption(const char*, int) {} void Context::setOption(const char*, const char*) {} bool Context::shouldExit() { return false; } void Context::setAsDefaultForAssertsOutOfTestCases() {} void Context::setAssertHandler(detail::assert_handler) {} +void Context::setCout(std::ostream*) {} int Context::run() { return 0; } -IReporter::~IReporter() = default; - int IReporter::get_num_active_contexts() { return 0; } const IContextScope* const* IReporter::get_active_contexts() { return nullptr; } int IReporter::get_num_stringified_contexts() { return 0; } @@ -3635,7 +4015,7 @@ namespace doctest { namespace { // the int (priority) is part of the key for automatic sorting - sadly one can register a // reporter with a duplicate name and a different priority but hopefully that won't happen often :| - typedef std::map, reporterCreatorFunc> reporterMap; + using reporterMap = std::map, reporterCreatorFunc>; reporterMap& getReporters() { static reporterMap data; @@ -3667,8 +4047,8 @@ namespace detail { #ifndef DOCTEST_CONFIG_NO_EXCEPTIONS DOCTEST_NORETURN void throwException() { g_cs->shouldLogCurrentException = false; - throw TestFailureException(); - } // NOLINT(cert-err60-cpp) + throw TestFailureException(); // NOLINT(hicpp-exception-baseclass) + } #else // DOCTEST_CONFIG_NO_EXCEPTIONS void throwException() {} #endif // DOCTEST_CONFIG_NO_EXCEPTIONS @@ -3714,91 +4094,132 @@ namespace { return !*wild; } - //// C string hash function (djb2) - taken from http://www.cse.yorku.ca/~oz/hash.html - //unsigned hashStr(unsigned const char* str) { - // unsigned long hash = 5381; - // char c; - // while((c = *str++)) - // hash = ((hash << 5) + hash) + c; // hash * 33 + c - // return hash; - //} - // checks if the name matches any of the filters (and can be configured what to do when empty) bool matchesAny(const char* name, const std::vector& filters, bool matchEmpty, - bool caseSensitive) { - if(filters.empty() && matchEmpty) + bool caseSensitive) { + if (filters.empty() && matchEmpty) return true; - for(auto& curr : filters) - if(wildcmp(name, curr.c_str(), caseSensitive)) + for (auto& curr : filters) + if (wildcmp(name, curr.c_str(), caseSensitive)) return true; return false; } + + unsigned long long hash(unsigned long long a, unsigned long long b) { + return (a << 5) + b; + } + + // C string hash function (djb2) - taken from http://www.cse.yorku.ca/~oz/hash.html + unsigned long long hash(const char* str) { + unsigned long long hash = 5381; + char c; + while ((c = *str++)) + hash = ((hash << 5) + hash) + c; // hash * 33 + c + return hash; + } + + unsigned long long hash(const SubcaseSignature& sig) { + return hash(hash(hash(sig.m_file), hash(sig.m_name.c_str())), sig.m_line); + } + + unsigned long long hash(const std::vector& sigs, size_t count) { + unsigned long long running = 0; + auto end = sigs.begin() + count; + for (auto it = sigs.begin(); it != end; it++) { + running = hash(running, hash(*it)); + } + return running; + } + + unsigned long long hash(const std::vector& sigs) { + unsigned long long running = 0; + for (const SubcaseSignature& sig : sigs) { + running = hash(running, hash(sig)); + } + return running; + } } // namespace namespace detail { + bool Subcase::checkFilters() { + if (g_cs->subcaseStack.size() < size_t(g_cs->subcase_filter_levels)) { + if (!matchesAny(m_signature.m_name.c_str(), g_cs->filters[6], true, g_cs->case_sensitive)) + return true; + if (matchesAny(m_signature.m_name.c_str(), g_cs->filters[7], false, g_cs->case_sensitive)) + return true; + } + return false; + } Subcase::Subcase(const String& name, const char* file, int line) : m_signature({name, file, line}) { - auto* s = g_cs; + if (!g_cs->reachedLeaf) { + if (g_cs->nextSubcaseStack.size() <= g_cs->subcaseStack.size() + || g_cs->nextSubcaseStack[g_cs->subcaseStack.size()] == m_signature) { + // Going down. + if (checkFilters()) { return; } - // check subcase filters - if(s->subcasesStack.size() < size_t(s->subcase_filter_levels)) { - if(!matchesAny(m_signature.m_name.c_str(), s->filters[6], true, s->case_sensitive)) - return; - if(matchesAny(m_signature.m_name.c_str(), s->filters[7], false, s->case_sensitive)) - return; + g_cs->subcaseStack.push_back(m_signature); + g_cs->currentSubcaseDepth++; + m_entered = true; + DOCTEST_ITERATE_THROUGH_REPORTERS(subcase_start, m_signature); + } + } else { + if (g_cs->subcaseStack[g_cs->currentSubcaseDepth] == m_signature) { + // This subcase is reentered via control flow. + g_cs->currentSubcaseDepth++; + m_entered = true; + DOCTEST_ITERATE_THROUGH_REPORTERS(subcase_start, m_signature); + } else if (g_cs->nextSubcaseStack.size() <= g_cs->currentSubcaseDepth + && g_cs->fullyTraversedSubcases.find(hash(hash(g_cs->subcaseStack, g_cs->currentSubcaseDepth), hash(m_signature))) + == g_cs->fullyTraversedSubcases.end()) { + if (checkFilters()) { return; } + // This subcase is part of the one to be executed next. + g_cs->nextSubcaseStack.clear(); + g_cs->nextSubcaseStack.insert(g_cs->nextSubcaseStack.end(), + g_cs->subcaseStack.begin(), g_cs->subcaseStack.begin() + g_cs->currentSubcaseDepth); + g_cs->nextSubcaseStack.push_back(m_signature); + } } - - // if a Subcase on the same level has already been entered - if(s->subcasesStack.size() < size_t(s->subcasesCurrentMaxLevel)) { - s->should_reenter = true; - return; - } - - // push the current signature to the stack so we can check if the - // current stack + the current new subcase have been traversed - s->subcasesStack.push_back(m_signature); - if(s->subcasesPassed.count(s->subcasesStack) != 0) { - // pop - revert to previous stack since we've already passed this - s->subcasesStack.pop_back(); - return; - } - - s->subcasesCurrentMaxLevel = s->subcasesStack.size(); - m_entered = true; - - DOCTEST_ITERATE_THROUGH_REPORTERS(subcase_start, m_signature); } - DOCTEST_MSVC_SUPPRESS_WARNING_WITH_PUSH(4996) // std::uncaught_exception is deprecated in C++17 - DOCTEST_GCC_SUPPRESS_WARNING_WITH_PUSH("-Wdeprecated-declarations") + DOCTEST_MSVC_SUPPRESS_WARNING_WITH_PUSH(4996) // std::uncaught_exception is deprecated in C++17 + DOCTEST_GCC_SUPPRESS_WARNING_WITH_PUSH("-Wdeprecated-declarations") DOCTEST_CLANG_SUPPRESS_WARNING_WITH_PUSH("-Wdeprecated-declarations") Subcase::~Subcase() { - if(m_entered) { - // only mark the subcase stack as passed if no subcases have been skipped - if(g_cs->should_reenter == false) - g_cs->subcasesPassed.insert(g_cs->subcasesStack); - g_cs->subcasesStack.pop_back(); + if (m_entered) { + g_cs->currentSubcaseDepth--; + + if (!g_cs->reachedLeaf) { + // Leaf. + g_cs->fullyTraversedSubcases.insert(hash(g_cs->subcaseStack)); + g_cs->nextSubcaseStack.clear(); + g_cs->reachedLeaf = true; + } else if (g_cs->nextSubcaseStack.empty()) { + // All children are finished. + g_cs->fullyTraversedSubcases.insert(hash(g_cs->subcaseStack)); + } #if defined(__cpp_lib_uncaught_exceptions) && __cpp_lib_uncaught_exceptions >= 201411L && (!defined(__MAC_OS_X_VERSION_MIN_REQUIRED) || __MAC_OS_X_VERSION_MIN_REQUIRED >= 101200) if(std::uncaught_exceptions() > 0 #else if(std::uncaught_exception() #endif - && g_cs->shouldLogCurrentException) { + && g_cs->shouldLogCurrentException) { DOCTEST_ITERATE_THROUGH_REPORTERS( test_case_exception, {"exception thrown in subcase - will translate later " - "when the whole test case has been exited (cannot " - "translate while there is an active exception)", - false}); + "when the whole test case has been exited (cannot " + "translate while there is an active exception)", + false}); g_cs->shouldLogCurrentException = false; } + DOCTEST_ITERATE_THROUGH_REPORTERS(subcase_end, DOCTEST_EMPTY); } } - DOCTEST_CLANG_SUPPRESS_WARNING_POP - DOCTEST_GCC_SUPPRESS_WARNING_POP + DOCTEST_CLANG_SUPPRESS_WARNING_POP + DOCTEST_GCC_SUPPRESS_WARNING_POP DOCTEST_MSVC_SUPPRESS_WARNING_POP Subcase::operator bool() const { return m_entered; } @@ -3812,20 +4233,11 @@ namespace detail { TestSuite& TestSuite::operator*(const char* in) { m_test_suite = in; - // clear state - m_description = nullptr; - m_skip = false; - m_no_breaks = false; - m_no_output = false; - m_may_fail = false; - m_should_fail = false; - m_expected_failures = 0; - m_timeout = 0; return *this; } TestCase::TestCase(funcType test, const char* file, unsigned line, const TestSuite& test_suite, - const char* type, int template_id) { + const String& type, int template_id) { m_file = file; m_line = line; m_name = nullptr; // will be later overridden in operator* @@ -3850,10 +4262,8 @@ namespace detail { } DOCTEST_MSVC_SUPPRESS_WARNING_WITH_PUSH(26434) // hides a non-virtual function - DOCTEST_MSVC_SUPPRESS_WARNING(26437) // Do not slice TestCase& TestCase::operator=(const TestCase& other) { - static_cast(*this) = static_cast(other); - + TestCaseData::operator=(other); m_test = other.m_test; m_type = other.m_type; m_template_id = other.m_template_id; @@ -3869,7 +4279,7 @@ namespace detail { m_name = in; // make a new name with an appended type for templated test case if(m_template_id != -1) { - m_full_name = String(m_name) + m_type; + m_full_name = String(m_name) + "<" + m_type + ">"; // redirect the name to point to the newly constructed full name m_name = m_full_name.c_str(); } @@ -3925,29 +4335,6 @@ namespace { return suiteOrderComparator(lhs, rhs); } -#ifdef DOCTEST_CONFIG_COLORS_WINDOWS - HANDLE g_stdoutHandle; - WORD g_origFgAttrs; - WORD g_origBgAttrs; - bool g_attrsInitted = false; - - int colors_init() { - if(!g_attrsInitted) { - g_stdoutHandle = GetStdHandle(STD_OUTPUT_HANDLE); - g_attrsInitted = true; - CONSOLE_SCREEN_BUFFER_INFO csbiInfo; - GetConsoleScreenBufferInfo(g_stdoutHandle, &csbiInfo); - g_origFgAttrs = csbiInfo.wAttributes & ~(BACKGROUND_GREEN | BACKGROUND_RED | - BACKGROUND_BLUE | BACKGROUND_INTENSITY); - g_origBgAttrs = csbiInfo.wAttributes & ~(FOREGROUND_GREEN | FOREGROUND_RED | - FOREGROUND_BLUE | FOREGROUND_INTENSITY); - } - return 0; - } - - int dumy_init_console_colors = colors_init(); -#endif // DOCTEST_CONFIG_COLORS_WINDOWS - DOCTEST_CLANG_SUPPRESS_WARNING_WITH_PUSH("-Wdeprecated-declarations") void color_to_stream(std::ostream& s, Color::Enum code) { static_cast(s); // for DOCTEST_CONFIG_COLORS_NONE or DOCTEST_CONFIG_COLORS_WINDOWS @@ -3981,10 +4368,26 @@ namespace { #ifdef DOCTEST_CONFIG_COLORS_WINDOWS if(g_no_colors || - (isatty(fileno(stdout)) == false && getContextOptions()->force_colors == false)) + (_isatty(_fileno(stdout)) == false && getContextOptions()->force_colors == false)) return; -#define DOCTEST_SET_ATTR(x) SetConsoleTextAttribute(g_stdoutHandle, x | g_origBgAttrs) + static struct ConsoleHelper { + HANDLE stdoutHandle; + WORD origFgAttrs; + WORD origBgAttrs; + + ConsoleHelper() { + stdoutHandle = GetStdHandle(STD_OUTPUT_HANDLE); + CONSOLE_SCREEN_BUFFER_INFO csbiInfo; + GetConsoleScreenBufferInfo(stdoutHandle, &csbiInfo); + origFgAttrs = csbiInfo.wAttributes & ~(BACKGROUND_GREEN | BACKGROUND_RED | + BACKGROUND_BLUE | BACKGROUND_INTENSITY); + origBgAttrs = csbiInfo.wAttributes & ~(FOREGROUND_GREEN | FOREGROUND_RED | + FOREGROUND_BLUE | FOREGROUND_INTENSITY); + } + } ch; + +#define DOCTEST_SET_ATTR(x) SetConsoleTextAttribute(ch.stdoutHandle, x | ch.origBgAttrs) // clang-format off switch (code) { @@ -4001,7 +4404,7 @@ namespace { case Color::BrightWhite: DOCTEST_SET_ATTR(FOREGROUND_INTENSITY | FOREGROUND_GREEN | FOREGROUND_RED | FOREGROUND_BLUE); break; case Color::None: case Color::Bright: // invalid - default: DOCTEST_SET_ATTR(g_origFgAttrs); + default: DOCTEST_SET_ATTR(ch.origFgAttrs); } // clang-format on #endif // DOCTEST_CONFIG_COLORS_WINDOWS @@ -4118,35 +4521,22 @@ namespace detail { getExceptionTranslators().push_back(et); } -#ifdef DOCTEST_CONFIG_TREAT_CHAR_STAR_AS_STRING - void toStream(std::ostream* s, char* in) { *s << in; } - void toStream(std::ostream* s, const char* in) { *s << in; } -#endif // DOCTEST_CONFIG_TREAT_CHAR_STAR_AS_STRING - void toStream(std::ostream* s, bool in) { *s << std::boolalpha << in << std::noboolalpha; } - void toStream(std::ostream* s, float in) { *s << in; } - void toStream(std::ostream* s, double in) { *s << in; } - void toStream(std::ostream* s, double long in) { *s << in; } - - void toStream(std::ostream* s, char in) { *s << in; } - void toStream(std::ostream* s, char signed in) { *s << in; } - void toStream(std::ostream* s, char unsigned in) { *s << in; } - void toStream(std::ostream* s, int short in) { *s << in; } - void toStream(std::ostream* s, int short unsigned in) { *s << in; } - void toStream(std::ostream* s, int in) { *s << in; } - void toStream(std::ostream* s, int unsigned in) { *s << in; } - void toStream(std::ostream* s, int long in) { *s << in; } - void toStream(std::ostream* s, int long unsigned in) { *s << in; } - void toStream(std::ostream* s, int long long in) { *s << in; } - void toStream(std::ostream* s, int long long unsigned in) { *s << in; } - DOCTEST_THREAD_LOCAL std::vector g_infoContexts; // for logging with INFO() ContextScopeBase::ContextScopeBase() { g_infoContexts.push_back(this); } - DOCTEST_MSVC_SUPPRESS_WARNING_WITH_PUSH(4996) // std::uncaught_exception is deprecated in C++17 - DOCTEST_GCC_SUPPRESS_WARNING_WITH_PUSH("-Wdeprecated-declarations") + ContextScopeBase::ContextScopeBase(ContextScopeBase&& other) noexcept { + if (other.need_to_destroy) { + other.destroy(); + } + other.need_to_destroy = false; + g_infoContexts.push_back(this); + } + + DOCTEST_MSVC_SUPPRESS_WARNING_WITH_PUSH(4996) // std::uncaught_exception is deprecated in C++17 + DOCTEST_GCC_SUPPRESS_WARNING_WITH_PUSH("-Wdeprecated-declarations") DOCTEST_CLANG_SUPPRESS_WARNING_WITH_PUSH("-Wdeprecated-declarations") // destroy cannot be inlined into the destructor because that would mean calling stringify after @@ -4165,8 +4555,8 @@ namespace detail { g_infoContexts.pop_back(); } - DOCTEST_CLANG_SUPPRESS_WARNING_POP - DOCTEST_GCC_SUPPRESS_WARNING_POP + DOCTEST_CLANG_SUPPRESS_WARNING_POP + DOCTEST_GCC_SUPPRESS_WARNING_POP DOCTEST_MSVC_SUPPRESS_WARNING_POP } // namespace detail namespace { @@ -4207,10 +4597,10 @@ namespace { static LONG CALLBACK handleException(PEXCEPTION_POINTERS ExceptionInfo) { // Multiple threads may enter this filter/handler at once. We want the error message to be printed on the // console just once no matter how many threads have crashed. - static std::mutex mutex; + DOCTEST_DECLARE_STATIC_MUTEX(mutex) static bool execute = true; { - std::lock_guard lock(mutex); + DOCTEST_LOCK_MUTEX(mutex) if(execute) { bool reported = false; for(size_t i = 0; i < DOCTEST_COUNTOF(signalDefs); ++i) { @@ -4313,7 +4703,7 @@ namespace { static unsigned int prev_abort_behavior; static int prev_report_mode; static _HFILE prev_report_file; - static void (*prev_sigabrt_handler)(int); + static void (DOCTEST_CDECL *prev_sigabrt_handler)(int); static std::terminate_handler original_terminate_handler; static bool isSet; static ULONG guaranteeSize; @@ -4325,7 +4715,7 @@ namespace { unsigned int FatalConditionHandler::prev_abort_behavior; int FatalConditionHandler::prev_report_mode; _HFILE FatalConditionHandler::prev_report_file; - void (*FatalConditionHandler::prev_sigabrt_handler)(int); + void (DOCTEST_CDECL *FatalConditionHandler::prev_sigabrt_handler)(int); std::terminate_handler FatalConditionHandler::original_terminate_handler; bool FatalConditionHandler::isSet = false; ULONG FatalConditionHandler::guaranteeSize = 0; @@ -4383,7 +4773,7 @@ namespace { sigStack.ss_flags = 0; sigaltstack(&sigStack, &oldSigStack); struct sigaction sa = {}; - sa.sa_handler = handleSignal; // NOLINT + sa.sa_handler = handleSignal; sa.sa_flags = SA_ONSTACK; for(std::size_t i = 0; i < DOCTEST_COUNTOF(signalDefs); ++i) { sigaction(signalDefs[i].id, &sa, &oldSigActions[i]); @@ -4422,7 +4812,7 @@ namespace { #define DOCTEST_OUTPUT_DEBUG_STRING(text) ::OutputDebugStringA(text) #else // TODO: integration with XCode and other IDEs -#define DOCTEST_OUTPUT_DEBUG_STRING(text) // NOLINT(clang-diagnostic-unused-macros) +#define DOCTEST_OUTPUT_DEBUG_STRING(text) #endif // Platform void addAssert(assertType::Enum at) { @@ -4441,8 +4831,8 @@ namespace { DOCTEST_ITERATE_THROUGH_REPORTERS(test_case_exception, {message.c_str(), true}); - while(g_cs->subcasesStack.size()) { - g_cs->subcasesStack.pop_back(); + while (g_cs->subcaseStack.size()) { + g_cs->subcaseStack.pop_back(); DOCTEST_ITERATE_THROUGH_REPORTERS(subcase_end, DOCTEST_EMPTY); } @@ -4454,25 +4844,26 @@ namespace { } #endif // DOCTEST_CONFIG_POSIX_SIGNALS || DOCTEST_CONFIG_WINDOWS_SEH } // namespace + +AssertData::AssertData(assertType::Enum at, const char* file, int line, const char* expr, + const char* exception_type, const StringContains& exception_string) + : m_test_case(g_cs->currentTest), m_at(at), m_file(file), m_line(line), m_expr(expr), + m_failed(true), m_threw(false), m_threw_as(false), m_exception_type(exception_type), + m_exception_string(exception_string) { +#if DOCTEST_MSVC + if (m_expr[0] == ' ') // this happens when variadic macros are disabled under MSVC + ++m_expr; +#endif // MSVC +} + namespace detail { + ResultBuilder::ResultBuilder(assertType::Enum at, const char* file, int line, const char* expr, + const char* exception_type, const String& exception_string) + : AssertData(at, file, line, expr, exception_type, exception_string) { } ResultBuilder::ResultBuilder(assertType::Enum at, const char* file, int line, const char* expr, - const char* exception_type, const char* exception_string) { - m_test_case = g_cs->currentTest; - m_at = at; - m_file = file; - m_line = line; - m_expr = expr; - m_failed = true; - m_threw = false; - m_threw_as = false; - m_exception_type = exception_type; - m_exception_string = exception_string; -#if DOCTEST_MSVC - if(m_expr[0] == ' ') // this happens when variadic macros are disabled under MSVC - ++m_expr; -#endif // MSVC - } + const char* exception_type, const Contains& exception_string) + : AssertData(at, file, line, expr, exception_type, exception_string) { } void ResultBuilder::setResult(const Result& res) { m_decomp = res.m_decomp; @@ -4488,17 +4879,17 @@ namespace detail { if(m_at & assertType::is_throws) { //!OCLINT bitwise operator in conditional m_failed = !m_threw; } else if((m_at & assertType::is_throws_as) && (m_at & assertType::is_throws_with)) { //!OCLINT - m_failed = !m_threw_as || (m_exception != m_exception_string); + m_failed = !m_threw_as || !m_exception_string.check(m_exception); } else if(m_at & assertType::is_throws_as) { //!OCLINT bitwise operator in conditional m_failed = !m_threw_as; } else if(m_at & assertType::is_throws_with) { //!OCLINT bitwise operator in conditional - m_failed = m_exception != m_exception_string; + m_failed = !m_exception_string.check(m_exception); } else if(m_at & assertType::is_nothrow) { //!OCLINT bitwise operator in conditional m_failed = m_threw; } if(m_exception.size()) - m_exception = String("\"") + m_exception + "\""; + m_exception = "\"" + m_exception + "\""; if(is_running_in_test) { addAssert(m_at); @@ -4526,8 +4917,8 @@ namespace detail { std::abort(); } - void decomp_assert(assertType::Enum at, const char* file, int line, const char* expr, - Result result) { + bool decomp_assert(assertType::Enum at, const char* file, int line, const char* expr, + const Result& result) { bool failed = !result.m_passed; // ################################################################################### @@ -4536,21 +4927,29 @@ namespace detail { // ################################################################################### DOCTEST_ASSERT_OUT_OF_TESTS(result.m_decomp); DOCTEST_ASSERT_IN_TESTS(result.m_decomp); - // NOLINTNEXTLINE(clang-analyzer-cplusplus.NewDeleteLeaks) + return !failed; } MessageBuilder::MessageBuilder(const char* file, int line, assertType::Enum severity) { - m_stream = getTlsOss(); + m_stream = tlssPush(); m_file = file; m_line = line; m_severity = severity; } - IExceptionTranslator::IExceptionTranslator() = default; - IExceptionTranslator::~IExceptionTranslator() = default; + MessageBuilder::~MessageBuilder() { + if (!logged) + tlssPop(); + } + + DOCTEST_DEFINE_INTERFACE(IExceptionTranslator) bool MessageBuilder::log() { - m_string = getTlsOssResult(); + if (!logged) { + m_string = tlssPop(); + logged = true; + } + DOCTEST_ITERATE_THROUGH_REPORTERS(log_message, *this); const bool isWarn = m_severity & assertType::is_warn; @@ -4569,29 +4968,10 @@ namespace detail { if(m_severity & assertType::is_require) //!OCLINT bitwise operator in conditional throwException(); } - - MessageBuilder::~MessageBuilder() = default; } // namespace detail namespace { using namespace detail; - template - DOCTEST_NORETURN void throw_exception(Ex const& e) { -#ifndef DOCTEST_CONFIG_NO_EXCEPTIONS - throw e; -#else // DOCTEST_CONFIG_NO_EXCEPTIONS - std::cerr << "doctest will terminate because it needed to throw an exception.\n" - << "The message was: " << e.what() << '\n'; - std::terminate(); -#endif // DOCTEST_CONFIG_NO_EXCEPTIONS - } - -#ifndef DOCTEST_INTERNAL_ERROR -#define DOCTEST_INTERNAL_ERROR(msg) \ - throw_exception(std::logic_error( \ - __FILE__ ":" DOCTEST_TOSTR(__LINE__) ": Internal doctest error: " msg)) -#endif // DOCTEST_INTERNAL_ERROR - // clang-format off // ================================================================================================= @@ -4673,10 +5053,10 @@ namespace { void ensureTagClosed(); - private: - void writeDeclaration(); + private: + void newlineIfNecessary(); bool m_tagIsOpen = false; @@ -4865,7 +5245,7 @@ namespace { XmlWriter::XmlWriter( std::ostream& os ) : m_os( os ) { - writeDeclaration(); + // writeDeclaration(); // called explicitly by the reporters that use the writer class - see issue #627 } XmlWriter::~XmlWriter() { @@ -4976,8 +5356,8 @@ namespace { struct XmlReporter : public IReporter { - XmlWriter xml; - std::mutex mutex; + XmlWriter xml; + DOCTEST_DECLARE_MUTEX(mutex) // caching pointers/references to objects of these types - safe to do const ContextOptions& opt; @@ -5054,7 +5434,8 @@ namespace { xml.scopedElement("TestCase").writeAttribute("name", in.data[i]->m_name) .writeAttribute("testsuite", in.data[i]->m_test_suite) .writeAttribute("filename", skipPathFromFilename(in.data[i]->m_file.c_str())) - .writeAttribute("line", line(in.data[i]->m_line)); + .writeAttribute("line", line(in.data[i]->m_line)) + .writeAttribute("skipped", in.data[i]->m_skip); } xml.scopedElement("OverallResultsTestCases") .writeAttribute("unskipped", in.run_stats->numTestCasesPassingFilters); @@ -5070,6 +5451,8 @@ namespace { } void test_run_start() override { + xml.writeDeclaration(); + // remove .exe extension - mainly to have the same output on UNIX and Windows std::string binary_name = skipPathFromFilename(opt.binary_name.c_str()); #ifdef DOCTEST_PLATFORM_WINDOWS @@ -5124,7 +5507,8 @@ namespace { xml.startElement("OverallResultsAsserts") .writeAttribute("successes", st.numAssertsCurrentTest - st.numAssertsFailedCurrentTest) - .writeAttribute("failures", st.numAssertsFailedCurrentTest); + .writeAttribute("failures", st.numAssertsFailedCurrentTest) + .writeAttribute("test_case_success", st.testCaseSuccess); if(opt.duration) xml.writeAttribute("duration", st.seconds); if(tc->m_expected_failures) @@ -5135,7 +5519,7 @@ namespace { } void test_case_exception(const TestCaseException& e) override { - std::lock_guard lock(mutex); + DOCTEST_LOCK_MUTEX(mutex) xml.scopedElement("Exception") .writeAttribute("crash", e.is_crash) @@ -5143,8 +5527,6 @@ namespace { } void subcase_start(const SubcaseSignature& in) override { - std::lock_guard lock(mutex); - xml.startElement("SubCase") .writeAttribute("name", in.m_name) .writeAttribute("filename", skipPathFromFilename(in.m_file)) @@ -5158,7 +5540,7 @@ namespace { if(!rb.m_failed && !opt.success) return; - std::lock_guard lock(mutex); + DOCTEST_LOCK_MUTEX(mutex) xml.startElement("Expression") .writeAttribute("success", !rb.m_failed) @@ -5174,7 +5556,7 @@ namespace { if(rb.m_at & assertType::is_throws_as) xml.scopedElement("ExpectedException").writeText(rb.m_exception_type); if(rb.m_at & assertType::is_throws_with) - xml.scopedElement("ExpectedExceptionString").writeText(rb.m_exception_string); + xml.scopedElement("ExpectedExceptionString").writeText(rb.m_exception_string.c_str()); if((rb.m_at & assertType::is_normal) && !rb.m_threw) xml.scopedElement("Expanded").writeText(rb.m_decomp.c_str()); @@ -5184,7 +5566,7 @@ namespace { } void log_message(const MessageData& mb) override { - std::lock_guard lock(mutex); + DOCTEST_LOCK_MUTEX(mutex) xml.startElement("Message") .writeAttribute("type", failureString(mb.m_severity)) @@ -5220,7 +5602,8 @@ namespace { } else if((rb.m_at & assertType::is_throws_as) && (rb.m_at & assertType::is_throws_with)) { //!OCLINT s << Color::Cyan << assertString(rb.m_at) << "( " << rb.m_expr << ", \"" - << rb.m_exception_string << "\", " << rb.m_exception_type << " ) " << Color::None; + << rb.m_exception_string.c_str() + << "\", " << rb.m_exception_type << " ) " << Color::None; if(rb.m_threw) { if(!rb.m_failed) { s << "threw as expected!\n"; @@ -5241,7 +5624,8 @@ namespace { } else if(rb.m_at & assertType::is_throws_with) { //!OCLINT bitwise operator in conditional s << Color::Cyan << assertString(rb.m_at) << "( " << rb.m_expr << ", \"" - << rb.m_exception_string << "\" ) " << Color::None + << rb.m_exception_string.c_str() + << "\" ) " << Color::None << (rb.m_threw ? (!rb.m_failed ? "threw as expected!" : "threw a DIFFERENT exception: ") : "did NOT throw at all!") @@ -5266,8 +5650,8 @@ namespace { // - more attributes in tags struct JUnitReporter : public IReporter { - XmlWriter xml; - std::mutex mutex; + XmlWriter xml; + DOCTEST_DECLARE_MUTEX(mutex) Timer timer; std::vector deepestSubcaseStackNames; @@ -5363,9 +5747,13 @@ namespace { // WHAT FOLLOWS ARE OVERRIDES OF THE VIRTUAL METHODS OF THE REPORTER INTERFACE // ========================================================================================= - void report_query(const QueryData&) override {} + void report_query(const QueryData&) override { + xml.writeDeclaration(); + } - void test_run_start() override {} + void test_run_start() override { + xml.writeDeclaration(); + } void test_run_end(const TestRunStats& p) override { // remove .exe extension - mainly to have the same output on UNIX and Windows @@ -5435,12 +5823,11 @@ namespace { } void test_case_exception(const TestCaseException& e) override { - std::lock_guard lock(mutex); + DOCTEST_LOCK_MUTEX(mutex) testCaseData.addError("exception", e.error_string.c_str()); } void subcase_start(const SubcaseSignature& in) override { - std::lock_guard lock(mutex); deepestSubcaseStackNames.push_back(in.m_name); } @@ -5450,7 +5837,7 @@ namespace { if(!rb.m_failed) // report only failures & ignore the `success` option return; - std::lock_guard lock(mutex); + DOCTEST_LOCK_MUTEX(mutex) std::ostringstream os; os << skipPathFromFilename(rb.m_file) << (opt.gnu_file_line ? ":" : "(") @@ -5501,7 +5888,7 @@ namespace { bool hasLoggedCurrentTestStart; std::vector subcasesStack; size_t currentSubcaseLevel; - std::mutex mutex; + DOCTEST_DECLARE_MUTEX(mutex) // caching pointers/references to objects of these types - safe to do const ContextOptions& opt; @@ -5606,9 +5993,11 @@ namespace { } void printIntro() { - printVersion(); - s << Color::Cyan << "[doctest] " << Color::None - << "run with \"--" DOCTEST_OPTIONS_PREFIX_DISPLAY "help\" for options\n"; + if(opt.no_intro == false) { + printVersion(); + s << Color::Cyan << "[doctest] " << Color::None + << "run with \"--" DOCTEST_OPTIONS_PREFIX_DISPLAY "help\" for options\n"; + } } void printHelp() { @@ -5693,12 +6082,18 @@ namespace { << Whitespace(sizePrefixDisplay*1) << "exits after the tests finish\n"; s << " -" DOCTEST_OPTIONS_PREFIX_DISPLAY "d, --" DOCTEST_OPTIONS_PREFIX_DISPLAY "duration= " << Whitespace(sizePrefixDisplay*1) << "prints the time duration of each test\n"; + s << " -" DOCTEST_OPTIONS_PREFIX_DISPLAY "m, --" DOCTEST_OPTIONS_PREFIX_DISPLAY "minimal= " + << Whitespace(sizePrefixDisplay*1) << "minimal console output (only failures)\n"; + s << " -" DOCTEST_OPTIONS_PREFIX_DISPLAY "q, --" DOCTEST_OPTIONS_PREFIX_DISPLAY "quiet= " + << Whitespace(sizePrefixDisplay*1) << "no console output\n"; s << " -" DOCTEST_OPTIONS_PREFIX_DISPLAY "nt, --" DOCTEST_OPTIONS_PREFIX_DISPLAY "no-throw= " << Whitespace(sizePrefixDisplay*1) << "skips exceptions-related assert checks\n"; s << " -" DOCTEST_OPTIONS_PREFIX_DISPLAY "ne, --" DOCTEST_OPTIONS_PREFIX_DISPLAY "no-exitcode= " << Whitespace(sizePrefixDisplay*1) << "returns (or exits) always with success\n"; s << " -" DOCTEST_OPTIONS_PREFIX_DISPLAY "nr, --" DOCTEST_OPTIONS_PREFIX_DISPLAY "no-run= " << Whitespace(sizePrefixDisplay*1) << "skips all runtime doctest operations\n"; + s << " -" DOCTEST_OPTIONS_PREFIX_DISPLAY "ni, --" DOCTEST_OPTIONS_PREFIX_DISPLAY "no-intro= " + << Whitespace(sizePrefixDisplay*1) << "omit the framework intro in the output\n"; s << " -" DOCTEST_OPTIONS_PREFIX_DISPLAY "nv, --" DOCTEST_OPTIONS_PREFIX_DISPLAY "no-version= " << Whitespace(sizePrefixDisplay*1) << "omit the framework version in the output\n"; s << " -" DOCTEST_OPTIONS_PREFIX_DISPLAY "nc, --" DOCTEST_OPTIONS_PREFIX_DISPLAY "no-colors= " @@ -5736,22 +6131,6 @@ namespace { printReporters(getReporters(), "reporters"); } - void list_query_results() { - separator_to_stream(); - if(opt.count || opt.list_test_cases) { - s << Color::Cyan << "[doctest] " << Color::None - << "unskipped test cases passing the current filters: " - << g_cs->numTestCasesPassingFilters << "\n"; - } else if(opt.list_test_suites) { - s << Color::Cyan << "[doctest] " << Color::None - << "unskipped test cases passing the current filters: " - << g_cs->numTestCasesPassingFilters << "\n"; - s << Color::Cyan << "[doctest] " << Color::None - << "test suites with unskipped test cases passing the current filters: " - << g_cs->numTestSuitesPassingFilters << "\n"; - } - } - // ========================================================================================= // WHAT FOLLOWS ARE OVERRIDES OF THE VIRTUAL METHODS OF THE REPORTER INTERFACE // ========================================================================================= @@ -5797,9 +6176,15 @@ namespace { } } - void test_run_start() override { printIntro(); } + void test_run_start() override { + if(!opt.minimal) + printIntro(); + } void test_run_end(const TestRunStats& p) override { + if(opt.minimal && p.numTestCasesFailed == 0) + return; + separator_to_stream(); s << std::dec; @@ -5849,7 +6234,7 @@ namespace { // log the preamble of the test case only if there is something // else to print - something other than that an assert has failed if(opt.duration || - (st.failure_flags && st.failure_flags != TestCaseFailureReason::AssertFailure)) + (st.failure_flags && st.failure_flags != static_cast(TestCaseFailureReason::AssertFailure))) logTestStart(); if(opt.duration) @@ -5880,6 +6265,7 @@ namespace { } void test_case_exception(const TestCaseException& e) override { + DOCTEST_LOCK_MUTEX(mutex) if(tc->m_no_output) return; @@ -5904,14 +6290,12 @@ namespace { } void subcase_start(const SubcaseSignature& subc) override { - std::lock_guard lock(mutex); subcasesStack.push_back(subc); ++currentSubcaseLevel; hasLoggedCurrentTestStart = false; } void subcase_end() override { - std::lock_guard lock(mutex); --currentSubcaseLevel; hasLoggedCurrentTestStart = false; } @@ -5920,7 +6304,7 @@ namespace { if((!rb.m_failed && !opt.success) || tc->m_no_output) return; - std::lock_guard lock(mutex); + DOCTEST_LOCK_MUTEX(mutex) logTestStart(); @@ -5936,7 +6320,7 @@ namespace { if(tc->m_no_output) return; - std::lock_guard lock(mutex); + DOCTEST_LOCK_MUTEX(mutex) logTestStart(); @@ -6047,18 +6431,42 @@ namespace { std::vector& res) { String filtersString; if(parseOption(argc, argv, pattern, &filtersString)) { - // tokenize with "," as a separator - // cppcheck-suppress strtokCalled - DOCTEST_CLANG_SUPPRESS_WARNING_WITH_PUSH("-Wdeprecated-declarations") - auto pch = std::strtok(filtersString.c_str(), ","); // modifies the string - while(pch != nullptr) { - if(strlen(pch)) - res.push_back(pch); - // uses the strtok() internal state to go to the next token - // cppcheck-suppress strtokCalled - pch = std::strtok(nullptr, ","); + // tokenize with "," as a separator, unless escaped with backslash + std::ostringstream s; + auto flush = [&s, &res]() { + auto string = s.str(); + if(string.size() > 0) { + res.push_back(string.c_str()); + } + s.str(""); + }; + + bool seenBackslash = false; + const char* current = filtersString.c_str(); + const char* end = current + strlen(current); + while(current != end) { + char character = *current++; + if(seenBackslash) { + seenBackslash = false; + if(character == ',' || character == '\\') { + s.put(character); + continue; + } + s.put('\\'); + } + if(character == '\\') { + seenBackslash = true; + } else if(character == ',') { + flush(); + } else { + s.put(character); + } } - DOCTEST_CLANG_SUPPRESS_WARNING_POP + + if(seenBackslash) { + s.put('\\'); + } + flush(); return true; } return false; @@ -6077,30 +6485,30 @@ namespace { if(!parseOption(argc, argv, pattern, &parsedValue)) return false; - if(type == 0) { + if(type) { + // integer + // TODO: change this to use std::stoi or something else! currently it uses undefined behavior - assumes '0' on failed parse... + int theInt = std::atoi(parsedValue.c_str()); + if (theInt != 0) { + res = theInt; //!OCLINT parameter reassignment + return true; + } + } else { // boolean - const char positive[][5] = {"1", "true", "on", "yes"}; // 5 - strlen("true") + 1 - const char negative[][6] = {"0", "false", "off", "no"}; // 6 - strlen("false") + 1 + const char positive[][5] = { "1", "true", "on", "yes" }; // 5 - strlen("true") + 1 + const char negative[][6] = { "0", "false", "off", "no" }; // 6 - strlen("false") + 1 // if the value matches any of the positive/negative possibilities - for(unsigned i = 0; i < 4; i++) { - if(parsedValue.compare(positive[i], true) == 0) { + for (unsigned i = 0; i < 4; i++) { + if (parsedValue.compare(positive[i], true) == 0) { res = 1; //!OCLINT parameter reassignment return true; } - if(parsedValue.compare(negative[i], true) == 0) { + if (parsedValue.compare(negative[i], true) == 0) { res = 0; //!OCLINT parameter reassignment return true; } } - } else { - // integer - // TODO: change this to use std::stoi or something else! currently it uses undefined behavior - assumes '0' on failed parse... - int theInt = std::atoi(parsedValue.c_str()); // NOLINT - if(theInt != 0) { - res = theInt; //!OCLINT parameter reassignment - return true; - } } return false; } @@ -6191,9 +6599,12 @@ void Context::parseArgs(int argc, const char* const* argv, bool withDefaults) { DOCTEST_PARSE_AS_BOOL_OR_FLAG("case-sensitive", "cs", case_sensitive, false); DOCTEST_PARSE_AS_BOOL_OR_FLAG("exit", "e", exit, false); DOCTEST_PARSE_AS_BOOL_OR_FLAG("duration", "d", duration, false); + DOCTEST_PARSE_AS_BOOL_OR_FLAG("minimal", "m", minimal, false); + DOCTEST_PARSE_AS_BOOL_OR_FLAG("quiet", "q", quiet, false); DOCTEST_PARSE_AS_BOOL_OR_FLAG("no-throw", "nt", no_throw, false); DOCTEST_PARSE_AS_BOOL_OR_FLAG("no-exitcode", "ne", no_exitcode, false); DOCTEST_PARSE_AS_BOOL_OR_FLAG("no-run", "nr", no_run, false); + DOCTEST_PARSE_AS_BOOL_OR_FLAG("no-intro", "ni", no_intro, false); DOCTEST_PARSE_AS_BOOL_OR_FLAG("no-version", "nv", no_version, false); DOCTEST_PARSE_AS_BOOL_OR_FLAG("no-colors", "nc", no_colors, false); DOCTEST_PARSE_AS_BOOL_OR_FLAG("force-colors", "fc", force_colors, false); @@ -6257,10 +6668,14 @@ void Context::clearFilters() { curr.clear(); } -// allows the user to override procedurally the int/bool options from the command line +// allows the user to override procedurally the bool options from the command line +void Context::setOption(const char* option, bool value) { + setOption(option, value ? "true" : "false"); +} + +// allows the user to override procedurally the int options from the command line void Context::setOption(const char* option, int value) { setOption(option, toString(value).c_str()); - // NOLINTNEXTLINE(clang-analyzer-cplusplus.NewDeleteLeaks) } // allows the user to override procedurally the string options from the command line @@ -6277,6 +6692,31 @@ void Context::setAsDefaultForAssertsOutOfTestCases() { g_cs = p; } void Context::setAssertHandler(detail::assert_handler ah) { p->ah = ah; } +void Context::setCout(std::ostream* out) { p->cout = out; } + +static class DiscardOStream : public std::ostream +{ +private: + class : public std::streambuf + { + private: + // allowing some buffering decreases the amount of calls to overflow + char buf[1024]; + + protected: + std::streamsize xsputn(const char_type*, std::streamsize count) override { return count; } + + int_type overflow(int_type ch) override { + setp(std::begin(buf), std::end(buf)); + return traits_type::not_eof(ch); + } + } discardBuf; + +public: + DiscardOStream() + : std::ostream(&discardBuf) {} +} discardOut; + // the main function that does all the filtering and test running int Context::run() { using namespace detail; @@ -6290,15 +6730,18 @@ int Context::run() { g_no_colors = p->no_colors; p->resetRunData(); - // stdout by default - p->cout = &std::cout; - p->cerr = &std::cerr; - - // or to a file if specified std::fstream fstr; - if(p->out.size()) { - fstr.open(p->out.c_str(), std::fstream::out); - p->cout = &fstr; + if(p->cout == nullptr) { + if(p->quiet) { + p->cout = &discardOut; + } else if(p->out.size()) { + // to a file if specified + fstr.open(p->out.c_str(), std::fstream::out); + p->cout = &fstr; + } else { + // stdout by default + p->cout = &std::cout; + } } FatalConditionHandler::allocateAltStackMem(); @@ -6370,7 +6813,7 @@ int Context::run() { // random_shuffle implementation const auto first = &testArray[0]; for(size_t i = testArray.size() - 1; i > 0; --i) { - int idxToSwap = std::rand() % (i + 1); // NOLINT + int idxToSwap = std::rand() % (i + 1); const auto temp = first[i]; @@ -6457,7 +6900,7 @@ int Context::run() { p->numAssertsFailedCurrentTest_atomic = 0; p->numAssertsCurrentTest_atomic = 0; - p->subcasesPassed.clear(); + p->fullyTraversedSubcases.clear(); DOCTEST_ITERATE_THROUGH_REPORTERS(test_case_start, tc); @@ -6467,9 +6910,10 @@ int Context::run() { do { // reset some of the fields for subcases (except for the set of fully passed ones) - p->should_reenter = false; - p->subcasesCurrentMaxLevel = 0; - p->subcasesStack.clear(); + p->reachedLeaf = false; + // May not be empty if previous subcase exited via exception. + p->subcaseStack.clear(); + p->currentSubcaseDepth = 0; p->shouldLogCurrentException = true; @@ -6503,9 +6947,9 @@ DOCTEST_MSVC_SUPPRESS_WARNING_POP p->failure_flags |= TestCaseFailureReason::TooManyFailedAsserts; } - if(p->should_reenter && run_test) + if(!p->nextSubcaseStack.empty() && run_test) DOCTEST_ITERATE_THROUGH_REPORTERS(test_case_reenter, tc); - if(!p->should_reenter) + if(p->nextSubcaseStack.empty()) run_test = false; } while(run_test); @@ -6531,17 +6975,10 @@ DOCTEST_MSVC_SUPPRESS_WARNING_POP DOCTEST_ITERATE_THROUGH_REPORTERS(report_query, qdata); } - // see these issues on the reasoning for this: - // - https://github.com/onqtam/doctest/issues/143#issuecomment-414418903 - // - https://github.com/onqtam/doctest/issues/126 - auto DOCTEST_FIX_FOR_MACOS_LIBCPP_IOSFWD_STRING_LINK_ERRORS = []() DOCTEST_NOINLINE - { std::cout << std::string(); }; - DOCTEST_FIX_FOR_MACOS_LIBCPP_IOSFWD_STRING_LINK_ERRORS(); - return cleanup_and_return(); } -IReporter::~IReporter() = default; +DOCTEST_DEFINE_INTERFACE(IReporter) int IReporter::get_num_active_contexts() { return detail::g_infoContexts.size(); } const IContextScope* const* IReporter::get_active_contexts() { @@ -6576,5 +7013,7 @@ DOCTEST_CLANG_SUPPRESS_WARNING_POP DOCTEST_MSVC_SUPPRESS_WARNING_POP DOCTEST_GCC_SUPPRESS_WARNING_POP +DOCTEST_SUPPRESS_COMMON_WARNINGS_POP + #endif // DOCTEST_LIBRARY_IMPLEMENTATION #endif // DOCTEST_CONFIG_IMPLEMENT diff --git a/tests/Autocomplete.test.cpp b/tests/Autocomplete.test.cpp index f3b0bcad..d8ad3340 100644 --- a/tests/Autocomplete.test.cpp +++ b/tests/Autocomplete.test.cpp @@ -2240,43 +2240,18 @@ local a: aaa.do CHECK(ac.entryMap.count("other")); } -TEST_CASE_FIXTURE(ACBuiltinsFixture, "autocompleteSource") + +TEST_CASE_FIXTURE(ACFixture, "comments") { - std::string_view source = R"( - local a = table. -- Line 1 - -- | Column 23 - )"; + fileResolver.source["Comments"] = "--!str"; - auto ac = autocompleteSource(frontend, source, Position{1, 24}, nullCallback).result; - - CHECK_EQ(17, ac.entryMap.size()); - CHECK(ac.entryMap.count("find")); - CHECK(ac.entryMap.count("pack")); - CHECK(!ac.entryMap.count("math")); -} - -TEST_CASE_FIXTURE(ACFixture, "autocompleteSource_require") -{ - std::string_view source = R"( - local a = require(w -- Line 1 - -- | Column 27 - )"; - - // CLI-43699 require shouldn't crash inside autocompleteSource - auto ac = autocompleteSource(frontend, source, Position{1, 27}, nullCallback).result; -} - -TEST_CASE_FIXTURE(ACFixture, "autocompleteSource_comments") -{ - std::string_view source = "--!str"; - - auto ac = autocompleteSource(frontend, source, Position{0, 6}, nullCallback).result; + auto ac = Luau::autocomplete(frontend, "Comments", Position{0, 6}, nullCallback); CHECK_EQ(0, ac.entryMap.size()); } TEST_CASE_FIXTURE(ACBuiltinsFixture, "autocompleteProp_index_function_metamethod_is_variadic") { - std::string_view source = R"( + fileResolver.source["Module/A"] = R"( type Foo = {x: number} local t = {} setmetatable(t, { @@ -2289,7 +2264,7 @@ TEST_CASE_FIXTURE(ACBuiltinsFixture, "autocompleteProp_index_function_metamethod -- | Column 20 )"; - auto ac = autocompleteSource(frontend, source, Position{9, 20}, nullCallback).result; + auto ac = Luau::autocomplete(frontend, "Module/A", Position{9, 20}, nullCallback); REQUIRE_EQ(1, ac.entryMap.size()); CHECK(ac.entryMap.count("x")); } @@ -2378,35 +2353,36 @@ end CHECK(ac.entryMap.count("elsewhere")); } -TEST_CASE_FIXTURE(ACFixture, "autocompleteSource_not_the_var_we_are_defining") +TEST_CASE_FIXTURE(ACFixture, "not_the_var_we_are_defining") { - std::string_view source = "abc,de"; + fileResolver.source["Module/A"] = "abc,de"; - auto ac = autocompleteSource(frontend, source, Position{0, 6}, nullCallback).result; + auto ac = Luau::autocomplete(frontend, "Module/A", Position{0, 6}, nullCallback); CHECK(!ac.entryMap.count("de")); } -TEST_CASE_FIXTURE(ACFixture, "autocompleteSource_recursive_function") +TEST_CASE_FIXTURE(ACFixture, "recursive_function_global") { - { - std::string_view global = R"(function abc() + fileResolver.source["global"] = R"(function abc() end )"; - auto ac = autocompleteSource(frontend, global, Position{1, 0}, nullCallback).result; - CHECK(ac.entryMap.count("abc")); - } + auto ac = Luau::autocomplete(frontend, "global", Position{1, 0}, nullCallback); + CHECK(ac.entryMap.count("abc")); +} - { - std::string_view local = R"(local function abc() + + +TEST_CASE_FIXTURE(ACFixture, "recursive_function_local") +{ + fileResolver.source["local"] = R"(local function abc() end )"; - auto ac = autocompleteSource(frontend, local, Position{1, 0}, nullCallback).result; - CHECK(ac.entryMap.count("abc")); - } + auto ac = Luau::autocomplete(frontend, "local", Position{1, 0}, nullCallback); + CHECK(ac.entryMap.count("abc")); } TEST_CASE_FIXTURE(ACFixture, "suggest_table_keys") diff --git a/tests/Compiler.test.cpp b/tests/Compiler.test.cpp index fafafd71..46fde067 100644 --- a/tests/Compiler.test.cpp +++ b/tests/Compiler.test.cpp @@ -165,8 +165,8 @@ LOADN R1 1 FASTCALL2K 18 R1 K0 L0 LOADK R2 K0 GETIMPORT R0 3 -L0: CALL R0 2 -1 -RETURN R0 -1 +CALL R0 2 -1 +L0: RETURN R0 -1 )"); } @@ -2100,12 +2100,12 @@ FASTCALL2 18 R0 R1 L0 MOVE R5 R0 MOVE R6 R1 GETIMPORT R4 2 -L0: CALL R4 2 1 -FASTCALL2 19 R4 R2 L1 +CALL R4 2 1 +L0: FASTCALL2 19 R4 R2 L1 MOVE R5 R2 GETIMPORT R3 4 -L1: CALL R3 2 -1 -RETURN R3 -1 +CALL R3 2 -1 +L1: RETURN R3 -1 )"); } @@ -2511,8 +2511,8 @@ return 5: MOVE R3 R0 5: MOVE R4 R1 5: GETIMPORT R2 2 -5: L0: CALL R2 2 -1 -5: RETURN R2 -1 +5: CALL R2 2 -1 +5: L0: RETURN R2 -1 )"); } @@ -2828,8 +2828,8 @@ TEST_CASE("FastcallBytecode") LOADN R1 -5 FASTCALL1 2 R1 L0 GETIMPORT R0 2 -L0: CALL R0 1 -1 -RETURN R0 -1 +CALL R0 1 -1 +L0: RETURN R0 -1 )"); // call through a local variable @@ -2838,8 +2838,8 @@ GETIMPORT R0 2 LOADN R2 -5 FASTCALL1 2 R2 L0 MOVE R1 R0 -L0: CALL R1 1 -1 -RETURN R1 -1 +CALL R1 1 -1 +L0: RETURN R1 -1 )"); // call through an upvalue @@ -2847,8 +2847,8 @@ RETURN R1 -1 LOADN R1 -5 FASTCALL1 2 R1 L0 GETUPVAL R0 0 -L0: CALL R0 1 -1 -RETURN R0 -1 +CALL R0 1 -1 +L0: RETURN R0 -1 )"); // mutating the global in the script breaks the optimization @@ -2893,8 +2893,8 @@ LOADK R1 K0 FASTCALL1 57 R1 L0 GETIMPORT R0 2 GETVARARGS R2 -1 -L0: CALL R0 -1 1 -RETURN R0 1 +CALL R0 -1 1 +L0: RETURN R0 1 )"); // more complex example: select inside a for loop bound + select from a iterator @@ -2912,16 +2912,16 @@ LOADK R5 K0 FASTCALL1 57 R5 L0 GETIMPORT R4 2 GETVARARGS R6 -1 -L0: CALL R4 -1 1 -MOVE R1 R4 +CALL R4 -1 1 +L0: MOVE R1 R4 LOADN R2 1 FORNPREP R1 L3 L1: FASTCALL1 57 R3 L2 GETIMPORT R4 2 MOVE R5 R3 GETVARARGS R6 -1 -L2: CALL R4 -1 1 -ADD R0 R0 R4 +CALL R4 -1 1 +L2: ADD R0 R0 R4 FORNLOOP R1 L1 L3: RETURN R0 1 )"); @@ -3242,7 +3242,7 @@ LOADN R2 -1 FASTCALL1 2 R2 L0 GETGLOBAL R3 K1024 GETTABLEKS R1 R3 K1025 -L0: CALL R1 1 -1 +CALL R1 1 -1 )"); } @@ -4063,8 +4063,8 @@ LOADN R2 2 LOADN R3 3 FASTCALL 54 L0 GETIMPORT R0 2 -L0: CALL R0 3 -1 -RETURN R0 -1 +CALL R0 3 -1 +L0: RETURN R0 -1 )"); } @@ -4351,6 +4351,8 @@ TEST_CASE("LoopUnrollControlFlow") {"LuauCompileLoopUnrollThresholdMaxBoost", 300}, }; + ScopedFastFlag sff("LuauCompileFoldBuiltins", true); + // break jumps to the end CHECK_EQ("\n" + compileFunction(R"( for i=1,3 do @@ -4414,7 +4416,7 @@ L2: RETURN R0 0 // continue needs to properly close upvalues CHECK_EQ("\n" + compileFunction(R"( for i=1,1 do - local j = math.abs(i) + local j = global(i) print(function() return j end) if math.random() < 0.5 then continue @@ -4424,21 +4426,20 @@ end )", 1, 2), R"( +GETIMPORT R0 1 LOADN R1 1 -FASTCALL1 2 R1 L0 -GETIMPORT R0 2 -L0: CALL R0 1 1 -GETIMPORT R1 4 +CALL R0 1 1 +GETIMPORT R1 3 NEWCLOSURE R2 P0 CAPTURE REF R0 CALL R1 1 0 GETIMPORT R1 6 CALL R1 0 1 LOADK R2 K7 -JUMPIFNOTLT R1 R2 L1 +JUMPIFNOTLT R1 R2 L0 CLOSEUPVALS R0 RETURN R0 0 -L1: ADDK R0 R0 K8 +L0: ADDK R0 R0 K8 CLOSEUPVALS R0 RETURN R0 0 )"); @@ -4625,11 +4626,11 @@ FORNPREP R1 L3 L0: FASTCALL1 24 R3 L1 MOVE R6 R3 GETIMPORT R5 2 -L1: CALL R5 1 -1 -FASTCALL 2 L2 +CALL R5 1 -1 +L1: FASTCALL 2 L2 GETIMPORT R4 4 -L2: CALL R4 -1 1 -SETTABLE R4 R0 R3 +CALL R4 -1 1 +L2: SETTABLE R4 R0 R3 FORNLOOP R1 L0 L3: RETURN R0 1 )"); @@ -4660,6 +4661,133 @@ L1: RETURN R0 0 )"); } +TEST_CASE("LoopUnrollCostBuiltins") +{ + ScopedFastInt sfis[] = { + {"LuauCompileLoopUnrollThreshold", 25}, + {"LuauCompileLoopUnrollThresholdMaxBoost", 300}, + }; + + ScopedFastFlag sff("LuauCompileModelBuiltins", true); + + // this loop uses builtins and is close to the cost budget so it's important that we model builtins as cheaper than regular calls + CHECK_EQ("\n" + compileFunction(R"( +function cipher(block, nonce) + for i = 0,3 do + block[i + 1] = bit32.band(bit32.rshift(nonce, i * 8), 0xff) + end +end +)", + 0, 2), + R"( +FASTCALL2K 39 R1 K0 L0 +MOVE R4 R1 +LOADK R5 K0 +GETIMPORT R3 3 +CALL R3 2 1 +L0: FASTCALL2K 29 R3 K4 L1 +LOADK R4 K4 +GETIMPORT R2 6 +CALL R2 2 1 +L1: SETTABLEN R2 R0 1 +FASTCALL2K 39 R1 K7 L2 +MOVE R4 R1 +LOADK R5 K7 +GETIMPORT R3 3 +CALL R3 2 1 +L2: FASTCALL2K 29 R3 K4 L3 +LOADK R4 K4 +GETIMPORT R2 6 +CALL R2 2 1 +L3: SETTABLEN R2 R0 2 +FASTCALL2K 39 R1 K8 L4 +MOVE R4 R1 +LOADK R5 K8 +GETIMPORT R3 3 +CALL R3 2 1 +L4: FASTCALL2K 29 R3 K4 L5 +LOADK R4 K4 +GETIMPORT R2 6 +CALL R2 2 1 +L5: SETTABLEN R2 R0 3 +FASTCALL2K 39 R1 K9 L6 +MOVE R4 R1 +LOADK R5 K9 +GETIMPORT R3 3 +CALL R3 2 1 +L6: FASTCALL2K 29 R3 K4 L7 +LOADK R4 K4 +GETIMPORT R2 6 +CALL R2 2 1 +L7: SETTABLEN R2 R0 4 +RETURN R0 0 +)"); + + // note that if we break compiler's ability to reason about bit32 builtin the loop is no longer unrolled as it's too expensive + CHECK_EQ("\n" + compileFunction(R"( +bit32 = {} + +function cipher(block, nonce) + for i = 0,3 do + block[i + 1] = bit32.band(bit32.rshift(nonce, i * 8), 0xff) + end +end +)", + 0, 2), + R"( +LOADN R4 0 +LOADN R2 3 +LOADN R3 1 +FORNPREP R2 L1 +L0: ADDK R5 R4 K0 +GETGLOBAL R7 K1 +GETTABLEKS R6 R7 K2 +GETGLOBAL R8 K1 +GETTABLEKS R7 R8 K3 +MOVE R8 R1 +MULK R9 R4 K4 +CALL R7 2 1 +LOADN R8 255 +CALL R6 2 1 +SETTABLE R6 R0 R5 +FORNLOOP R2 L0 +L1: RETURN R0 0 +)"); + + // additionally, if we pass too many constants the builtin stops being cheap because of argument setup + CHECK_EQ("\n" + compileFunction(R"( +function cipher(block, nonce) + for i = 0,3 do + block[i + 1] = bit32.band(bit32.rshift(nonce, i * 8), 0xff, 0xff, 0xff, 0xff, 0xff) + end +end +)", + 0, 2), + R"( +LOADN R4 0 +LOADN R2 3 +LOADN R3 1 +FORNPREP R2 L3 +L0: ADDK R5 R4 K0 +MULK R9 R4 K1 +FASTCALL2 39 R1 R9 L1 +MOVE R8 R1 +GETIMPORT R7 4 +CALL R7 2 1 +L1: LOADN R8 255 +LOADN R9 255 +LOADN R10 255 +LOADN R11 255 +LOADN R12 255 +FASTCALL 29 L2 +GETIMPORT R6 6 +CALL R6 6 1 +L2: SETTABLE R6 R0 R5 +FORNLOOP R2 L0 +L3: RETURN R0 0 +)"); +} + TEST_CASE("InlineBasic") { // inline function that returns a constant @@ -5216,8 +5344,8 @@ DUPCLOSURE R0 K0 LOADK R3 K1 FASTCALL1 20 R3 L0 GETIMPORT R2 4 -L0: CALL R2 1 2 -ADD R1 R2 R3 +CALL R2 1 2 +L0: ADD R1 R2 R3 RETURN R1 1 )"); @@ -5483,14 +5611,14 @@ NEWTABLE R2 0 0 FASTCALL2K 49 R2 K1 L0 LOADK R3 K1 GETIMPORT R1 3 -L0: CALL R1 2 0 -NEWTABLE R1 0 0 +CALL R1 2 0 +L0: NEWTABLE R1 0 0 NEWTABLE R3 0 0 FASTCALL2 49 R3 R1 L1 MOVE R4 R1 GETIMPORT R2 3 -L1: CALL R2 2 0 -RETURN R0 0 +CALL R2 2 0 +L1: RETURN R0 0 )"); } @@ -5762,4 +5890,271 @@ RETURN R0 2 )"); } +TEST_CASE("OptimizationLevel") +{ + ScopedFastFlag sff("LuauAlwaysCaptureHotComments", true); + + // at optimization level 1, no inlining is performed + CHECK_EQ("\n" + compileFunction(R"( +local function foo(a) + return a +end + +return foo(42) +)", + 1, 1), + R"( +DUPCLOSURE R0 K0 +MOVE R1 R0 +LOADN R2 42 +CALL R1 1 -1 +RETURN R1 -1 +)"); + + // you can override the level from 1 to 2 to force it + CHECK_EQ("\n" + compileFunction(R"( +--!optimize 2 +local function foo(a) + return a +end + +return foo(42) +)", + 1, 1), + R"( +DUPCLOSURE R0 K0 +LOADN R1 42 +RETURN R1 1 +)"); + + // you can also override it externally + CHECK_EQ("\n" + compileFunction(R"( +local function foo(a) + return a +end + +return foo(42) +)", + 1, 2), + R"( +DUPCLOSURE R0 K0 +LOADN R1 42 +RETURN R1 1 +)"); + + // ... after which you can downgrade it back via hot comment + CHECK_EQ("\n" + compileFunction(R"( +--!optimize 1 +local function foo(a) + return a +end + +return foo(42) +)", + 1, 2), + R"( +DUPCLOSURE R0 K0 +MOVE R1 R0 +LOADN R2 42 +CALL R1 1 -1 +RETURN R1 -1 +)"); +} + +TEST_CASE("BuiltinFolding") +{ + ScopedFastFlag sff("LuauCompileFoldBuiltins", true); + + CHECK_EQ("\n" + compileFunction(R"( +return + math.abs(-42), + math.acos(1), + math.asin(0), + math.atan2(0, 1), + math.atan(0), + math.ceil(1.5), + math.cosh(0), + math.cos(0), + math.deg(3.14159265358979323846), + math.exp(0), + math.floor(-1.5), + math.fmod(7, 3), + math.ldexp(0.5, 3), + math.log10(100), + math.log(1), + math.log(4, 2), + math.log(27, 3), + math.max(1, 2, 3), + math.min(1, 2, 3), + math.pow(3, 3), + math.floor(math.rad(180)), + math.sinh(0), + math.sin(0), + math.sqrt(9), + math.tanh(0), + math.tan(0), + bit32.arshift(-10, 1), + bit32.arshift(10, 1), + bit32.band(1, 3), + bit32.bnot(-2), + bit32.bor(1, 2), + bit32.bxor(3, 7), + bit32.btest(1, 3), + bit32.extract(100, 1, 3), + bit32.lrotate(100, -1), + bit32.lshift(100, 1), + bit32.replace(100, 5, 1, 3), + bit32.rrotate(100, -1), + bit32.rshift(100, 1), + type(100), + string.byte("a"), + string.byte("abc", 2), + string.len("abc"), + typeof(true), + math.clamp(-1, 0, 1), + math.sign(77), + math.round(7.6), + (type("fin")) +)", + 0, 2), + R"( +LOADN R0 42 +LOADN R1 0 +LOADN R2 0 +LOADN R3 0 +LOADN R4 0 +LOADN R5 2 +LOADN R6 1 +LOADN R7 1 +LOADN R8 180 +LOADN R9 1 +LOADN R10 -2 +LOADN R11 1 +LOADN R12 4 +LOADN R13 2 +LOADN R14 0 +LOADN R15 2 +LOADN R16 3 +LOADN R17 3 +LOADN R18 1 +LOADN R19 27 +LOADN R20 3 +LOADN R21 0 +LOADN R22 0 +LOADN R23 3 +LOADN R24 0 +LOADN R25 0 +LOADK R26 K0 +LOADN R27 5 +LOADN R28 1 +LOADN R29 1 +LOADN R30 3 +LOADN R31 4 +LOADB R32 1 +LOADN R33 2 +LOADN R34 50 +LOADN R35 200 +LOADN R36 106 +LOADN R37 200 +LOADN R38 50 +LOADK R39 K1 +LOADN R40 97 +LOADN R41 98 +LOADN R42 3 +LOADK R43 K2 +LOADN R44 0 +LOADN R45 1 +LOADN R46 8 +LOADK R47 K3 +RETURN R0 48 +)"); +} + +TEST_CASE("BuiltinFoldingProhibited") +{ + ScopedFastFlag sff("LuauCompileFoldBuiltins", true); + + CHECK_EQ("\n" + compileFunction(R"( +return + math.abs(), + math.max(1, true), + string.byte("abc", 42), + bit32.rshift(10, 42) +)", + 0, 2), + R"( +FASTCALL 2 L0 +GETIMPORT R0 2 +CALL R0 0 1 +L0: LOADN R2 1 +FASTCALL2K 18 R2 K3 L1 +LOADK R3 K3 +GETIMPORT R1 5 +CALL R1 2 1 +L1: LOADK R3 K6 +FASTCALL2K 41 R3 K7 L2 +LOADK R4 K7 +GETIMPORT R2 10 +CALL R2 2 1 +L2: LOADN R4 10 +FASTCALL2K 39 R4 K7 L3 +LOADK R5 K7 +GETIMPORT R3 13 +CALL R3 2 -1 +L3: RETURN R0 -1 +)"); +} + +TEST_CASE("BuiltinFoldingMultret") +{ + ScopedFastFlag sff1("LuauCompileFoldBuiltins", true); + ScopedFastFlag sff2("LuauCompileBetterMultret", true); + + CHECK_EQ("\n" + compileFunction(R"( +local NoLanes: Lanes = --[[ ]] 0b0000000000000000000000000000000 +local OffscreenLane: Lane = --[[ ]] 0b1000000000000000000000000000000 + +local function getLanesToRetrySynchronouslyOnError(root: FiberRoot): Lanes + local everythingButOffscreen = bit32.band(root.pendingLanes, bit32.bnot(OffscreenLane)) + if everythingButOffscreen ~= NoLanes then + return everythingButOffscreen + end + if bit32.band(everythingButOffscreen, OffscreenLane) ~= 0 then + return OffscreenLane + end + return NoLanes +end +)", + 0, 2), + R"( +GETTABLEKS R2 R0 K0 +FASTCALL2K 29 R2 K1 L0 +LOADK R3 K1 +GETIMPORT R1 4 +CALL R1 2 1 +L0: JUMPIFEQK R1 K5 L1 +RETURN R1 1 +L1: FASTCALL2K 29 R1 K6 L2 +MOVE R3 R1 +LOADK R4 K6 +GETIMPORT R2 4 +CALL R2 2 1 +L2: JUMPIFEQK R2 K5 L3 +LOADK R2 K6 +RETURN R2 1 +L3: LOADN R2 0 +RETURN R2 1 +)"); + + // Note: similarly, here we should have folded the return value but haven't because it's the last call in the sequence + CHECK_EQ("\n" + compileFunction(R"( +return math.abs(-42) +)", + 0, 2), + R"( +LOADN R0 42 +RETURN R0 1 +)"); +} + TEST_SUITE_END(); diff --git a/tests/Conformance.test.cpp b/tests/Conformance.test.cpp index 3f415149..52a578e5 100644 --- a/tests/Conformance.test.cpp +++ b/tests/Conformance.test.cpp @@ -17,6 +17,16 @@ #include extern bool verbose; +extern int optimizationLevel; + +static lua_CompileOptions defaultOptions() +{ + lua_CompileOptions copts = {}; + copts.optimizationLevel = optimizationLevel; + copts.debugLevel = 1; + + return copts; +} static int lua_collectgarbage(lua_State* L) { @@ -127,7 +137,7 @@ int lua_silence(lua_State* L) using StateRef = std::unique_ptr; static StateRef runConformance(const char* name, void (*setup)(lua_State* L) = nullptr, void (*yield)(lua_State* L) = nullptr, - lua_State* initialLuaState = nullptr, lua_CompileOptions* copts = nullptr) + lua_State* initialLuaState = nullptr, lua_CompileOptions* options = nullptr) { std::string path = __FILE__; path.erase(path.find_last_of("\\/")); @@ -189,8 +199,11 @@ static StateRef runConformance(const char* name, void (*setup)(lua_State* L) = n std::string chunkname = "=" + std::string(name); + // note: luau_compile supports nullptr options, but we need to customize our defaults to improve test coverage + lua_CompileOptions opts = options ? *options : defaultOptions(); + size_t bytecodeSize = 0; - char* bytecode = luau_compile(source.data(), source.size(), copts, &bytecodeSize); + char* bytecode = luau_compile(source.data(), source.size(), &opts, &bytecodeSize); int result = luau_load(L, chunkname.c_str(), bytecode, bytecodeSize, 0); free(bytecode); @@ -383,9 +396,7 @@ TEST_CASE("Pack") TEST_CASE("Vector") { - lua_CompileOptions copts = {}; - copts.optimizationLevel = 1; - copts.debugLevel = 1; + lua_CompileOptions copts = defaultOptions(); copts.vectorCtor = "vector"; runConformance( @@ -519,8 +530,7 @@ TEST_CASE("Debugger") breakhits = 0; interruptedthread = nullptr; - lua_CompileOptions copts = {}; - copts.optimizationLevel = 1; + lua_CompileOptions copts = defaultOptions(); copts.debugLevel = 2; runConformance( @@ -850,6 +860,43 @@ TEST_CASE("ApiCalls") } } +TEST_CASE("ApiAtoms") +{ + ScopedFastFlag sff("LuauLazyAtoms", true); + + StateRef globalState(luaL_newstate(), lua_close); + lua_State* L = globalState.get(); + + lua_callbacks(L)->useratom = [](const char* s, size_t l) -> int16_t { + if (strcmp(s, "string") == 0) + return 0; + if (strcmp(s, "important") == 0) + return 1; + + return -1; + }; + + lua_pushstring(L, "string"); + lua_pushstring(L, "import"); + lua_pushstring(L, "ant"); + lua_concat(L, 2); + lua_pushstring(L, "unimportant"); + + int a1, a2, a3; + const char* s1 = lua_tostringatom(L, -3, &a1); + const char* s2 = lua_tostringatom(L, -2, &a2); + const char* s3 = lua_tostringatom(L, -1, &a3); + + CHECK(strcmp(s1, "string") == 0); + CHECK(a1 == 0); + + CHECK(strcmp(s2, "important") == 0); + CHECK(a2 == 1); + + CHECK(strcmp(s3, "unimportant") == 0); + CHECK(a3 == -1); +} + static bool endsWith(const std::string& str, const std::string& suffix) { if (suffix.length() > str.length()) @@ -957,9 +1004,8 @@ TEST_CASE("TagMethodError") TEST_CASE("Coverage") { - lua_CompileOptions copts = {}; - copts.optimizationLevel = 1; - copts.debugLevel = 1; + lua_CompileOptions copts = defaultOptions(); + copts.optimizationLevel = 1; // disable inlining to get fixed expected hit results copts.coverageLevel = 2; runConformance( @@ -1059,6 +1105,9 @@ TEST_CASE("GCDump") TEST_CASE("Interrupt") { + lua_CompileOptions copts = defaultOptions(); + copts.optimizationLevel = 1; // disable loop unrolling to get fixed expected hit results + static const int expectedhits[] = { 2, 9, @@ -1109,7 +1158,8 @@ TEST_CASE("Interrupt") }, [](lua_State* L) { CHECK(index == 5); // a single yield point - }); + }, + nullptr, &copts); CHECK(index == int(std::size(expectedhits))); } diff --git a/tests/CostModel.test.cpp b/tests/CostModel.test.cpp index c709ba8e..eacc718b 100644 --- a/tests/CostModel.test.cpp +++ b/tests/CostModel.test.cpp @@ -10,7 +10,7 @@ namespace Luau namespace Compile { -uint64_t modelCost(AstNode* root, AstLocal* const* vars, size_t varCount); +uint64_t modelCost(AstNode* root, AstLocal* const* vars, size_t varCount, const DenseHashMap& builtins); int computeCost(uint64_t model, const bool* varsConst, size_t varCount); } // namespace Compile @@ -29,7 +29,7 @@ static uint64_t modelFunction(const char* source) AstStatFunction* func = result.root->body.data[0]->as(); REQUIRE(func); - return Luau::Compile::modelCost(func->func->body, func->func->args.data, func->func->args.size); + return Luau::Compile::modelCost(func->func->body, func->func->args.data, func->func->args.size, {nullptr}); } TEST_CASE("Expression") diff --git a/tests/Frontend.test.cpp b/tests/Frontend.test.cpp index 4f331399..4efa74d9 100644 --- a/tests/Frontend.test.cpp +++ b/tests/Frontend.test.cpp @@ -756,26 +756,6 @@ TEST_CASE_FIXTURE(FrontendFixture, "test_lint_uses_correct_config") CHECK_EQ(0, result4.warnings.size()); } -TEST_CASE_FIXTURE(FrontendFixture, "lintFragment") -{ - LintOptions lintOptions; - lintOptions.enableWarning(LintWarning::Code_ForRange); - - auto [_sourceModule, result] = frontend.lintFragment(R"( - local t = {} - - for i=#t,1 do - end - - for i=#t,1,-1 do - end - )", - lintOptions); - - CHECK_EQ(1, result.warnings.size()); - CHECK_EQ(0, result.errors.size()); -} - TEST_CASE_FIXTURE(FrontendFixture, "discard_type_graphs") { Frontend fe{&fileResolver, &configResolver, {false}}; diff --git a/tests/Linter.test.cpp b/tests/Linter.test.cpp index 202aeceb..488ade9f 100644 --- a/tests/Linter.test.cpp +++ b/tests/Linter.test.cpp @@ -1658,4 +1658,21 @@ end CHECK_EQ(result.warnings[0].text, "Condition has already been checked on line 2"); } +TEST_CASE_FIXTURE(Fixture, "WrongCommentOptimize") +{ + LintResult result = lint(R"( +--!optimize +--!optimize +--!optimize me +--!optimize 100500 +--!optimize 2 +)"); + + REQUIRE_EQ(result.warnings.size(), 4); + CHECK_EQ(result.warnings[0].text, "optimize directive requires an optimization level"); + CHECK_EQ(result.warnings[1].text, "optimize directive requires an optimization level"); + CHECK_EQ(result.warnings[2].text, "optimize directive uses unknown optimization level 'me', 0..2 expected"); + CHECK_EQ(result.warnings[3].text, "optimize directive uses unknown optimization level '100500', 0..2 expected"); +} + TEST_SUITE_END(); diff --git a/tests/Normalize.test.cpp b/tests/Normalize.test.cpp index fb0a899e..69e7f074 100644 --- a/tests/Normalize.test.cpp +++ b/tests/Normalize.test.cpp @@ -756,6 +756,65 @@ TEST_CASE_FIXTURE(Fixture, "cyclic_table_normalizes_sensibly") CHECK_EQ("t1 where t1 = { get: () -> t1 }", toString(ty, {true})); } +TEST_CASE_FIXTURE(Fixture, "cyclic_union") +{ + ScopedFastFlag sff[] = { + {"LuauLowerBoundsCalculation", true}, + {"LuauFixNormalizationOfCyclicUnions", true}, + }; + + CheckResult result = check(R"( + type T = {T?}? + + local a: T + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK("t1? where t1 = {t1?}" == toString(requireType("a"))); +} + +TEST_CASE_FIXTURE(Fixture, "cyclic_intersection") +{ + ScopedFastFlag sff[] = { + {"LuauLowerBoundsCalculation", true}, + {"LuauFixNormalizationOfCyclicUnions", true}, + }; + + CheckResult result = check(R"( + type T = {T & {}} + + local a: T + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + // FIXME: We are not properly normalizing this type, but we are at least not improperly discarding information + CHECK("t1 where t1 = {{t1 & {| |}}}" == toString(requireType("a"), {true})); +} + +TEST_CASE_FIXTURE(Fixture, "intersection_of_tables_with_indexers") +{ + ScopedFastFlag sff[] = { + {"LuauLowerBoundsCalculation", true}, + {"LuauFixNormalizationOfCyclicUnions", true}, + }; + + CheckResult result = check(R"( + type A = {number} + type B = {string} + + type C = A & B + + local a: C + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + // FIXME: We are not properly normalizing this type, but we are at least not improperly discarding information + CHECK("{number & string}" == toString(requireType("a"), {true})); +} + TEST_CASE_FIXTURE(BuiltinsFixture, "union_of_distinct_free_types") { ScopedFastFlag flags[] = { diff --git a/tests/ToString.test.cpp b/tests/ToString.test.cpp index 52a29bcc..288af8da 100644 --- a/tests/ToString.test.cpp +++ b/tests/ToString.test.cpp @@ -62,7 +62,6 @@ TEST_CASE_FIXTURE(Fixture, "named_table") TEST_CASE_FIXTURE(Fixture, "empty_table") { - ScopedFastFlag LuauToStringTableBracesNewlines("LuauToStringTableBracesNewlines", true); CheckResult result = check(R"( local a: {} )"); @@ -77,7 +76,6 @@ TEST_CASE_FIXTURE(Fixture, "empty_table") TEST_CASE_FIXTURE(Fixture, "table_respects_use_line_break") { - ScopedFastFlag LuauToStringTableBracesNewlines("LuauToStringTableBracesNewlines", true); CheckResult result = check(R"( local a: { prop: string, anotherProp: number, thirdProp: boolean } )"); diff --git a/tests/TypeInfer.builtins.test.cpp b/tests/TypeInfer.builtins.test.cpp index 7d1bd6ba..4f6adf97 100644 --- a/tests/TypeInfer.builtins.test.cpp +++ b/tests/TypeInfer.builtins.test.cpp @@ -1143,4 +1143,114 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "gmatch_capture_types_invalid_pattern_fallbac CHECK_EQ(toString(requireType("foo")), "() -> (...string)"); } +TEST_CASE_FIXTURE(BuiltinsFixture, "match_capture_types") +{ + ScopedFastFlag sffs{"LuauDeduceFindMatchReturnTypes", true}; + CheckResult result = check(R"END( + local a, b, c = string.match("This is a string", "(.()(%a+))") + )END"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ(toString(requireType("a")), "string"); + CHECK_EQ(toString(requireType("b")), "number"); + CHECK_EQ(toString(requireType("c")), "string"); +} + +TEST_CASE_FIXTURE(BuiltinsFixture, "match_capture_types2") +{ + ScopedFastFlag sffs{"LuauDeduceFindMatchReturnTypes", true}; + CheckResult result = check(R"END( + local a, b, c = string.match("This is a string", "(.()(%a+))", "this should be a number") + )END"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + TypeMismatch* tm = get(result.errors[0]); + REQUIRE(tm); + CHECK_EQ(toString(tm->wantedType), "number?"); + CHECK_EQ(toString(tm->givenType), "string"); + + CHECK_EQ(toString(requireType("a")), "string"); + CHECK_EQ(toString(requireType("b")), "number"); + CHECK_EQ(toString(requireType("c")), "string"); +} + +TEST_CASE_FIXTURE(BuiltinsFixture, "find_capture_types") +{ + ScopedFastFlag sffs{"LuauDeduceFindMatchReturnTypes", true}; + CheckResult result = check(R"END( + local d, e, a, b, c = string.find("This is a string", "(.()(%a+))") + )END"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ(toString(requireType("a")), "string"); + CHECK_EQ(toString(requireType("b")), "number"); + CHECK_EQ(toString(requireType("c")), "string"); + CHECK_EQ(toString(requireType("d")), "number?"); + CHECK_EQ(toString(requireType("e")), "number?"); +} + +TEST_CASE_FIXTURE(BuiltinsFixture, "find_capture_types2") +{ + ScopedFastFlag sffs{"LuauDeduceFindMatchReturnTypes", true}; + CheckResult result = check(R"END( + local d, e, a, b, c = string.find("This is a string", "(.()(%a+))", "this should be a number") + )END"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + TypeMismatch* tm = get(result.errors[0]); + REQUIRE(tm); + CHECK_EQ(toString(tm->wantedType), "number?"); + CHECK_EQ(toString(tm->givenType), "string"); + + CHECK_EQ(toString(requireType("a")), "string"); + CHECK_EQ(toString(requireType("b")), "number"); + CHECK_EQ(toString(requireType("c")), "string"); + CHECK_EQ(toString(requireType("d")), "number?"); + CHECK_EQ(toString(requireType("e")), "number?"); +} + +TEST_CASE_FIXTURE(BuiltinsFixture, "find_capture_types3") +{ + ScopedFastFlag sffs{"LuauDeduceFindMatchReturnTypes", true}; + CheckResult result = check(R"END( + local d, e, a, b, c = string.find("This is a string", "(.()(%a+))", 1, "this should be a bool") + )END"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + TypeMismatch* tm = get(result.errors[0]); + REQUIRE(tm); + CHECK_EQ(toString(tm->wantedType), "boolean?"); + CHECK_EQ(toString(tm->givenType), "string"); + + CHECK_EQ(toString(requireType("a")), "string"); + CHECK_EQ(toString(requireType("b")), "number"); + CHECK_EQ(toString(requireType("c")), "string"); + CHECK_EQ(toString(requireType("d")), "number?"); + CHECK_EQ(toString(requireType("e")), "number?"); +} + +TEST_CASE_FIXTURE(BuiltinsFixture, "find_capture_types3") +{ + ScopedFastFlag sffs{"LuauDeduceFindMatchReturnTypes", true}; + CheckResult result = check(R"END( + local d, e, a, b = string.find("This is a string", "(.()(%a+))", 1, true) + )END"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + CountMismatch* acm = get(result.errors[0]); + REQUIRE(acm); + CHECK_EQ(acm->context, CountMismatch::Result); + CHECK_EQ(acm->expected, 2); + CHECK_EQ(acm->actual, 4); + + CHECK_EQ(toString(requireType("d")), "number?"); + CHECK_EQ(toString(requireType("e")), "number?"); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.primitives.test.cpp b/tests/TypeInfer.primitives.test.cpp index 9e8e2503..d60c96e0 100644 --- a/tests/TypeInfer.primitives.test.cpp +++ b/tests/TypeInfer.primitives.test.cpp @@ -11,6 +11,8 @@ #include "doctest.h" +LUAU_FASTFLAG(LuauDeduceFindMatchReturnTypes) + using namespace Luau; TEST_SUITE_BEGIN("TypeInferPrimitives"); @@ -80,7 +82,10 @@ TEST_CASE_FIXTURE(Fixture, "string_function_other") )"); CHECK_EQ(0, result.errors.size()); - CHECK_EQ(toString(requireType("p")), "string?"); + if (FFlag::LuauDeduceFindMatchReturnTypes) + CHECK_EQ(toString(requireType("p")), "string"); + else + CHECK_EQ(toString(requireType("p")), "string?"); } TEST_CASE_FIXTURE(Fixture, "CheckMethodsOfNumber") diff --git a/tests/TypeInfer.singletons.test.cpp b/tests/TypeInfer.singletons.test.cpp index 4a88abee..5da4a340 100644 --- a/tests/TypeInfer.singletons.test.cpp +++ b/tests/TypeInfer.singletons.test.cpp @@ -476,4 +476,21 @@ TEST_CASE_FIXTURE(Fixture, "taking_the_length_of_union_of_string_singleton") CHECK_EQ(R"("bye" | "hi")", toString(requireTypeAtPosition({3, 23}))); } +TEST_CASE_FIXTURE(Fixture, "no_widening_from_callsites") +{ + ScopedFastFlag sff{"LuauReturnsFromCallsitesAreNotWidened", true}; + + CheckResult result = check(R"( + type Direction = "North" | "East" | "West" | "South" + + local function direction(): Direction + return "North" + end + + local d: Direction = direction() + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.tables.test.cpp b/tests/TypeInfer.tables.test.cpp index 3e830f2a..21ad4e1b 100644 --- a/tests/TypeInfer.tables.test.cpp +++ b/tests/TypeInfer.tables.test.cpp @@ -3084,4 +3084,86 @@ local b = a.x CHECK_EQ("Type 'a' does not have key 'x'", toString(result.errors[1])); } +TEST_CASE_FIXTURE(Fixture, "scalar_is_a_subtype_of_a_compatible_polymorphic_shape_type") +{ + ScopedFastFlag sff{"LuauScalarShapeSubtyping", true}; + + CheckResult result = check(R"( + local function f(s) + return s:lower() + end + + f("foo" :: string) + f("bar" :: "bar") + f("baz" :: "bar" | "baz") + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "scalar_is_not_a_subtype_of_a_compatible_polymorphic_shape_type") +{ + ScopedFastFlag sff{"LuauScalarShapeSubtyping", true}; + + CheckResult result = check(R"( + local function f(s) + return s:absolutely_no_scalar_has_this_method() + end + + f("foo" :: string) + f("bar" :: "bar") + f("baz" :: "bar" | "baz") + )"); + + LUAU_REQUIRE_ERROR_COUNT(3, result); + CHECK_EQ(R"(Type 'string' could not be converted into 't1 where t1 = {- absolutely_no_scalar_has_this_method: (t1) -> (a...) -}' +caused by: + The former's metatable does not satisfy the requirements. Table type 'string' not compatible with type 't1 where t1 = {- absolutely_no_scalar_has_this_method: (t1) -> (a...) -}' because the former is missing field 'absolutely_no_scalar_has_this_method')", + toString(result.errors[0])); + CHECK_EQ(R"(Type '"bar"' could not be converted into 't1 where t1 = {- absolutely_no_scalar_has_this_method: (t1) -> (a...) -}' +caused by: + The former's metatable does not satisfy the requirements. Table type 'string' not compatible with type 't1 where t1 = {- absolutely_no_scalar_has_this_method: (t1) -> (a...) -}' because the former is missing field 'absolutely_no_scalar_has_this_method')", + toString(result.errors[1])); + CHECK_EQ(R"(Type '"bar" | "baz"' could not be converted into 't1 where t1 = {- absolutely_no_scalar_has_this_method: (t1) -> (a...) -}' +caused by: + Not all union options are compatible. Type '"bar"' could not be converted into 't1 where t1 = {- absolutely_no_scalar_has_this_method: (t1) -> (a...) -}' +caused by: + The former's metatable does not satisfy the requirements. Table type 'string' not compatible with type 't1 where t1 = {- absolutely_no_scalar_has_this_method: (t1) -> (a...) -}' because the former is missing field 'absolutely_no_scalar_has_this_method')", + toString(result.errors[2])); +} + +TEST_CASE_FIXTURE(Fixture, "a_free_shape_can_turn_into_a_scalar_if_it_is_compatible") +{ + ScopedFastFlag sff{"LuauScalarShapeSubtyping", true}; + + CheckResult result = check(R"( + local function f(s): string + local foo = s:lower() + return s + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + CHECK_EQ("(string) -> string", toString(requireType("f"))); +} + +TEST_CASE_FIXTURE(Fixture, "a_free_shape_cannot_turn_into_a_scalar_if_it_is_not_compatible") +{ + ScopedFastFlag sff{"LuauScalarShapeSubtyping", true}; + + CheckResult result = check(R"( + local function f(s): string + local foo = s:absolutely_no_scalar_has_this_method() + return s + end + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ(R"(Type 't1 where t1 = {+ absolutely_no_scalar_has_this_method: (t1) -> (a, b...) +}' could not be converted into 'string' +caused by: + The former's metatable does not satisfy the requirements. Table type 'string' not compatible with type 't1 where t1 = {+ absolutely_no_scalar_has_this_method: (t1) -> (a, b...) +}' because the former is missing field 'absolutely_no_scalar_has_this_method')", + toString(result.errors[0])); + CHECK_EQ("(t1) -> string where t1 = {+ absolutely_no_scalar_has_this_method: (t1) -> (a, b...) +}", toString(requireType("f"))); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.unknownnever.test.cpp b/tests/TypeInfer.unknownnever.test.cpp index bc742b03..2288db4e 100644 --- a/tests/TypeInfer.unknownnever.test.cpp +++ b/tests/TypeInfer.unknownnever.test.cpp @@ -268,13 +268,49 @@ TEST_CASE_FIXTURE(Fixture, "unary_minus_of_never") TEST_CASE_FIXTURE(Fixture, "length_of_never") { + ScopedFastFlag sff{"LuauNeverTypesAndOperatorsInference", true}; + CheckResult result = check(R"( local x = #({} :: never) )"); LUAU_REQUIRE_NO_ERRORS(result); - CHECK_EQ("never", toString(requireType("x"))); + CHECK_EQ("number", toString(requireType("x"))); +} + +TEST_CASE_FIXTURE(Fixture, "dont_unify_operands_if_one_of_the_operand_is_never_in_any_ordering_operators") +{ + ScopedFastFlag sff[]{ + {"LuauUnknownAndNeverType", true}, + {"LuauNeverTypesAndOperatorsInference", true}, + }; + + CheckResult result = check(R"( + local function ord(x: nil, y) + return x ~= nil and x > y + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + CHECK_EQ("(nil, a) -> boolean", toString(requireType("ord"))); +} + +TEST_CASE_FIXTURE(Fixture, "math_operators_and_never") +{ + ScopedFastFlag sff[]{ + {"LuauUnknownAndNeverType", true}, + {"LuauNeverTypesAndOperatorsInference", true}, + }; + + CheckResult result = check(R"( + local function mul(x: nil, y) + return x ~= nil and x * y -- infers boolean | never, which is normalized into boolean + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + CHECK_EQ("(nil, a) -> boolean", toString(requireType("mul"))); } TEST_SUITE_END(); diff --git a/tests/conformance/basic.lua b/tests/conformance/basic.lua index 385a0450..b2dcaf94 100644 --- a/tests/conformance/basic.lua +++ b/tests/conformance/basic.lua @@ -727,16 +727,20 @@ assert((function() local abs = math.abs function foo(...) return abs(...) end re -- NOTE: getfenv breaks fastcalls for the remainder of the source! hence why this is delayed until the end function testgetfenv() getfenv() + + -- declare constant so that at O2 this test doesn't interfere with constant folding which we can't deoptimize + local negfive negfive = -5 + -- getfenv breaks fastcalls (we assume we can't rely on knowing the semantics), but behavior shouldn't change - assert((function() return math.abs(-5) end)() == 5) - assert((function() local abs = math.abs return abs(-5) end)() == 5) - assert((function() local abs = math.abs function foo() return abs(-5) end return foo() end)() == 5) + assert((function() return math.abs(negfive) end)() == 5) + assert((function() local abs = math.abs return abs(negfive) end)() == 5) + assert((function() local abs = math.abs function foo() return abs(negfive) end return foo() end)() == 5) -- ... unless you actually reassign the function :D getfenv().math = { abs = function(n) return n*n end } - assert((function() return math.abs(-5) end)() == 25) - assert((function() local abs = math.abs return abs(-5) end)() == 25) - assert((function() local abs = math.abs function foo() return abs(-5) end return foo() end)() == 25) + assert((function() return math.abs(negfive) end)() == 25) + assert((function() local abs = math.abs return abs(negfive) end)() == 25) + assert((function() local abs = math.abs function foo() return abs(negfive) end return foo() end)() == 25) end -- you need to have enough arguments and arguments of the right type; if you don't, we'll fallback to the regular code. This checks coercions diff --git a/tests/main.cpp b/tests/main.cpp index 2af9f702..c5b844b5 100644 --- a/tests/main.cpp +++ b/tests/main.cpp @@ -23,10 +23,13 @@ #include -// Indicates if verbose output is enabled. -// Currently, this enables output from lua's 'print', but other verbose output could be enabled eventually. +// Indicates if verbose output is enabled; can be overridden via --verbose +// Currently, this enables output from 'print', but other verbose output could be enabled eventually. bool verbose = false; +// Default optimization level for conformance test; can be overridden via -On +int optimizationLevel = 1; + static bool skipFastFlag(const char* flagName) { if (strncmp(flagName, "Test", 4) == 0) @@ -249,6 +252,15 @@ int main(int argc, char** argv) verbose = true; } + int level = -1; + if (doctest::parseIntOption(argc, argv, "-O", doctest::option_int, level)) + { + if (level < 0 || level > 2) + std::cerr << "Optimization level must be between 0 and 2 inclusive." << std::endl; + else + optimizationLevel = level; + } + if (std::vector flags; doctest::parseCommaSepArgs(argc, argv, "--fflags=", flags)) setFastFlags(flags); @@ -279,6 +291,7 @@ int main(int argc, char** argv) if (doctest::parseFlag(argc, argv, "--help") || doctest::parseFlag(argc, argv, "-h")) { printf("Additional command line options:\n"); + printf(" -O[n] Changes default optimization level (1) for conformance runs\n"); printf(" --verbose Enables verbose output (e.g. lua 'print' statements)\n"); printf(" --fflags= Sets specified fast flags\n"); printf(" --list-fflags List all fast flags\n"); diff --git a/tools/patchtests.py b/tools/patchtests.py index dcaf6083..56970c9f 100644 --- a/tools/patchtests.py +++ b/tools/patchtests.py @@ -16,7 +16,11 @@ state = 0 # parse input into errors[] with the state machine; this is using doctest output and expects multi-line match failures for line in input: if state == 0: - match = re.match("tests/[^:]+:(\d+): ERROR: CHECK_EQ", line) + if sys.platform == "win32": + match = re.match("[^(]+\((\d+)\): ERROR: CHECK_EQ", line) + else: + match = re.match("tests/[^:]+:(\d+): ERROR: CHECK_EQ", line) + if match: error_line = int(match[1]) state = 1 @@ -52,12 +56,16 @@ result = [] current = 0 index = 0 +target = 0 while index < len(source): line = source[index] error = errors[current] if current < len(errors) else None - if not error or index < error[0] or line != error[1][0]: + if error: + target = error[0] if sys.platform != "win32" else error[0] - len(error[1]) - 1 + + if not error or index < target or line != error[1][0]: result.append(line) index += 1 else: From 5b2e39c9225ac11a0d134d78c1ac9ff7fe429f35 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Thu, 14 Jul 2022 15:52:26 -0700 Subject: [PATCH 313/399] Sync to upstream/release/536 (#592) --- Analysis/include/Luau/Autocomplete.h | 10 - .../include/Luau/ConstraintGraphBuilder.h | 1 + Analysis/include/Luau/Frontend.h | 7 - Analysis/include/Luau/Unifier.h | 1 + Analysis/src/Autocomplete.cpp | 37 +- Analysis/src/ConstraintGraphBuilder.cpp | 23 + Analysis/src/Frontend.cpp | 25 +- Analysis/src/Linter.cpp | 16 + Analysis/src/Normalize.cpp | 160 +- Analysis/src/ToString.cpp | 68 +- Analysis/src/TypeChecker2.cpp | 7 +- Analysis/src/TypeInfer.cpp | 37 +- Analysis/src/TypeVar.cpp | 129 +- Analysis/src/Unifier.cpp | 74 +- Ast/src/Parser.cpp | 10 +- CMakeLists.txt | 1 + Compiler/include/Luau/Compiler.h | 4 +- Compiler/src/BuiltinFolding.cpp | 463 +++ Compiler/src/BuiltinFolding.h | 14 + Compiler/src/Builtins.cpp | 49 +- Compiler/src/Builtins.h | 4 +- Compiler/src/BytecodeBuilder.cpp | 15 +- Compiler/src/Compiler.cpp | 136 +- Compiler/src/ConstantFolding.cpp | 48 +- Compiler/src/ConstantFolding.h | 6 +- Compiler/src/CostModel.cpp | 26 +- Compiler/src/CostModel.h | 3 +- Makefile | 5 +- Sources.cmake | 2 + VM/include/lua.h | 2 +- VM/src/lapi.cpp | 19 +- VM/src/ldebug.cpp | 75 +- VM/src/lstring.cpp | 6 +- VM/src/lstring.h | 3 + bench/gc/test_SunSpider_crypto-aes.lua | 436 --- bench/tests/sunspider/crypto-aes.lua | 18 +- extern/doctest.h | 3089 ++++++++++------- tests/Autocomplete.test.cpp | 66 +- tests/Compiler.test.cpp | 475 ++- tests/Conformance.test.cpp | 72 +- tests/CostModel.test.cpp | 4 +- tests/Frontend.test.cpp | 20 - tests/Linter.test.cpp | 17 + tests/Normalize.test.cpp | 59 + tests/ToString.test.cpp | 2 - tests/TypeInfer.builtins.test.cpp | 110 + tests/TypeInfer.primitives.test.cpp | 7 +- tests/TypeInfer.singletons.test.cpp | 17 + tests/TypeInfer.tables.test.cpp | 82 + tests/TypeInfer.unknownnever.test.cpp | 38 +- tests/conformance/basic.lua | 16 +- tests/main.cpp | 17 +- tools/patchtests.py | 12 +- 53 files changed, 3875 insertions(+), 2168 deletions(-) create mode 100644 Compiler/src/BuiltinFolding.cpp create mode 100644 Compiler/src/BuiltinFolding.h delete mode 100644 bench/gc/test_SunSpider_crypto-aes.lua diff --git a/Analysis/include/Luau/Autocomplete.h b/Analysis/include/Luau/Autocomplete.h index 65b788d3..5e8d6605 100644 --- a/Analysis/include/Luau/Autocomplete.h +++ b/Analysis/include/Luau/Autocomplete.h @@ -78,16 +78,6 @@ struct AutocompleteResult using ModuleName = std::string; using StringCompletionCallback = std::function(std::string tag, std::optional ctx)>; -struct OwningAutocompleteResult -{ - AutocompleteResult result; - ModulePtr module; - std::unique_ptr sourceModule; -}; - AutocompleteResult autocomplete(Frontend& frontend, const ModuleName& moduleName, Position position, StringCompletionCallback callback); -// Deprecated, do not use in new work. -OwningAutocompleteResult autocompleteSource(Frontend& frontend, std::string_view source, Position position, StringCompletionCallback callback); - } // namespace Luau diff --git a/Analysis/include/Luau/ConstraintGraphBuilder.h b/Analysis/include/Luau/ConstraintGraphBuilder.h index a49e8594..513446fd 100644 --- a/Analysis/include/Luau/ConstraintGraphBuilder.h +++ b/Analysis/include/Luau/ConstraintGraphBuilder.h @@ -99,6 +99,7 @@ struct ConstraintGraphBuilder void visit(NotNull scope, AstStat* stat); void visit(NotNull scope, AstStatBlock* block); void visit(NotNull scope, AstStatLocal* local); + void visit(NotNull scope, AstStatFor* for_); void visit(NotNull scope, AstStatLocalFunction* function); void visit(NotNull scope, AstStatFunction* function); void visit(NotNull scope, AstStatReturn* ret); diff --git a/Analysis/include/Luau/Frontend.h b/Analysis/include/Luau/Frontend.h index f0d43090..27fd4d7e 100644 --- a/Analysis/include/Luau/Frontend.h +++ b/Analysis/include/Luau/Frontend.h @@ -127,13 +127,6 @@ struct Frontend CheckResult check(const ModuleName& name, std::optional optionOverride = {}); // new shininess LintResult lint(const ModuleName& name, std::optional enabledLintWarnings = {}); - /** Lint some code that has no associated DataModel object - * - * Since this source fragment has no name, we cannot cache its AST. Instead, - * we return it to the caller to use as they wish. - */ - std::pair lintFragment(std::string_view source, std::optional enabledLintWarnings = {}); - LintResult lint(const SourceModule& module, std::optional enabledLintWarnings = {}); bool isDirty(const ModuleName& name, bool forAutocomplete = false) const; diff --git a/Analysis/include/Luau/Unifier.h b/Analysis/include/Luau/Unifier.h index 4af324cb..f460dc87 100644 --- a/Analysis/include/Luau/Unifier.h +++ b/Analysis/include/Luau/Unifier.h @@ -79,6 +79,7 @@ private: void tryUnifySingletons(TypeId subTy, TypeId superTy); void tryUnifyFunctions(TypeId subTy, TypeId superTy, bool isFunctionCall = false); void tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection = false); + void tryUnifyScalarShape(TypeId subTy, TypeId superTy, bool reversed); void tryUnifyWithMetatable(TypeId subTy, TypeId superTy, bool reversed); void tryUnifyWithClass(TypeId subTy, TypeId superTy, bool reversed); diff --git a/Analysis/src/Autocomplete.cpp b/Analysis/src/Autocomplete.cpp index cc54d499..513a791c 100644 --- a/Analysis/src/Autocomplete.cpp +++ b/Analysis/src/Autocomplete.cpp @@ -7,7 +7,6 @@ #include "Luau/ToString.h" #include "Luau/TypeInfer.h" #include "Luau/TypePack.h" -#include "Luau/Parser.h" // TODO: only needed for autocompleteSource which is deprecated #include #include @@ -1407,8 +1406,8 @@ static AutocompleteResult autocomplete(const SourceModule& sourceModule, const M PropIndexType indexType = indexName->op == ':' ? PropIndexType::Colon : PropIndexType::Point; if (!FFlag::LuauSelfCallAutocompleteFix2 && isString(ty)) - return {autocompleteProps(*module, typeArena, typeChecker.globalScope->bindings[AstName{"string"}].typeId, indexType, ancestry), - ancestry}; + return { + autocompleteProps(*module, typeArena, typeChecker.globalScope->bindings[AstName{"string"}].typeId, indexType, ancestry), ancestry}; else return {autocompleteProps(*module, typeArena, ty, indexType, ancestry), ancestry}; } @@ -1507,8 +1506,8 @@ static AutocompleteResult autocomplete(const SourceModule& sourceModule, const M else if (AstStatIf* statIf = node->as(); statIf && !statIf->elseLocation.has_value()) { - return {{{"else", AutocompleteEntry{AutocompleteEntryKind::Keyword}}, {"elseif", AutocompleteEntry{AutocompleteEntryKind::Keyword}}}, - ancestry}; + return { + {{"else", AutocompleteEntry{AutocompleteEntryKind::Keyword}}, {"elseif", AutocompleteEntry{AutocompleteEntryKind::Keyword}}}, ancestry}; } else if (AstStatIf* statIf = parent->as(); statIf && node->is()) { @@ -1628,32 +1627,4 @@ AutocompleteResult autocomplete(Frontend& frontend, const ModuleName& moduleName return autocompleteResult; } -OwningAutocompleteResult autocompleteSource(Frontend& frontend, std::string_view source, Position position, StringCompletionCallback callback) -{ - // TODO: Remove #include "Luau/Parser.h" with this function - auto sourceModule = std::make_unique(); - ParseOptions parseOptions; - parseOptions.captureComments = true; - ParseResult result = Parser::parse(source.data(), source.size(), *sourceModule->names, *sourceModule->allocator, parseOptions); - - if (!result.root) - return {AutocompleteResult{}, {}, nullptr}; - - sourceModule->name = "FRAGMENT_SCRIPT"; - sourceModule->root = result.root; - sourceModule->mode = Mode::Strict; - sourceModule->commentLocations = std::move(result.commentLocations); - - TypeChecker& typeChecker = frontend.typeCheckerForAutocomplete; - ModulePtr module = typeChecker.check(*sourceModule, Mode::Strict); - - OwningAutocompleteResult autocompleteResult = { - autocomplete(*sourceModule, module, typeChecker, &frontend.arenaForAutocomplete, position, callback), std::move(module), - std::move(sourceModule)}; - - frontend.arenaForAutocomplete.clear(); - - return autocompleteResult; -} - } // namespace Luau diff --git a/Analysis/src/ConstraintGraphBuilder.cpp b/Analysis/src/ConstraintGraphBuilder.cpp index 3b9000cd..38895ceb 100644 --- a/Analysis/src/ConstraintGraphBuilder.cpp +++ b/Analysis/src/ConstraintGraphBuilder.cpp @@ -103,6 +103,8 @@ void ConstraintGraphBuilder::visit(NotNull scope, AstStat* stat) visit(scope, s); else if (auto s = stat->as()) visit(scope, s); + else if (auto s = stat->as()) + visit(scope, s); else if (auto f = stat->as()) visit(scope, f); else if (auto f = stat->as()) @@ -167,6 +169,27 @@ void ConstraintGraphBuilder::visit(NotNull scope, AstStatLocal* local) } } +void ConstraintGraphBuilder::visit(NotNull scope, AstStatFor* for_) +{ + auto checkNumber = [&](AstExpr* expr) + { + if (!expr) + return; + + TypeId t = check(scope, expr); + addConstraint(scope, SubtypeConstraint{t, singletonTypes.numberType}); + }; + + checkNumber(for_->from); + checkNumber(for_->to); + checkNumber(for_->step); + + NotNull forScope = childScope(for_->location, scope); + forScope->bindings[for_->var] = singletonTypes.numberType; + + visit(forScope, for_->body); +} + void addConstraints(Constraint* constraint, NotNull scope) { scope->constraints.reserve(scope->constraints.size() + scope->constraints.size()); diff --git a/Analysis/src/Frontend.cpp b/Analysis/src/Frontend.cpp index 9195363d..a7670de5 100644 --- a/Analysis/src/Frontend.cpp +++ b/Analysis/src/Frontend.cpp @@ -662,29 +662,6 @@ LintResult Frontend::lint(const ModuleName& name, std::optional Frontend::lintFragment(std::string_view source, std::optional enabledLintWarnings) -{ - LUAU_TIMETRACE_SCOPE("Frontend::lintFragment", "Frontend"); - - const Config& config = configResolver->getConfig(""); - - SourceModule sourceModule = parse(ModuleName{}, source, config.parseOptions); - - uint64_t ignoreLints = LintWarning::parseMask(sourceModule.hotcomments); - - Luau::LintOptions lintOptions = enabledLintWarnings.value_or(config.enabledLint); - lintOptions.warningMask &= ~ignoreLints; - - double timestamp = getTimestamp(); - - std::vector warnings = Luau::lint(sourceModule.root, *sourceModule.names.get(), typeChecker.globalScope, nullptr, - sourceModule.hotcomments, enabledLintWarnings.value_or(config.enabledLint)); - - stats.timeLint += getTimestamp() - timestamp; - - return {std::move(sourceModule), classifyLints(warnings, config)}; -} - LintResult Frontend::lint(const SourceModule& module, std::optional enabledLintWarnings) { LUAU_TIMETRACE_SCOPE("Frontend::lint", "Frontend"); @@ -958,7 +935,7 @@ std::optional FrontendModuleResolver::resolveModuleInfo(const Module { // CLI-43699 // If we can't find the current module name, that's because we bypassed the frontend's initializer - // and called typeChecker.check directly. (This is done by autocompleteSource, for example). + // and called typeChecker.check directly. // In that case, requires will always fail. return std::nullopt; } diff --git a/Analysis/src/Linter.cpp b/Analysis/src/Linter.cpp index 50868e56..fb952f5e 100644 --- a/Analysis/src/Linter.cpp +++ b/Analysis/src/Linter.cpp @@ -2688,6 +2688,21 @@ static void lintComments(LintContext& context, const std::vector& ho else seenMode = true; } + else if (first == "optimize") + { + size_t notspace = hc.content.find_first_not_of(" \t", space); + + if (space == std::string::npos || notspace == std::string::npos) + emitWarning(context, LintWarning::Code_CommentDirective, hc.location, "optimize directive requires an optimization level"); + else + { + const char* level = hc.content.c_str() + notspace; + + if (strcmp(level, "0") && strcmp(level, "1") && strcmp(level, "2")) + emitWarning(context, LintWarning::Code_CommentDirective, hc.location, + "optimize directive uses unknown optimization level '%s', 0..2 expected", level); + } + } else { static const char* kHotComments[] = { @@ -2695,6 +2710,7 @@ static void lintComments(LintContext& context, const std::vector& ho "nocheck", "nonstrict", "strict", + "optimize", }; if (const char* suggestion = fuzzyMatch(first, kHotComments, std::size(kHotComments))) diff --git a/Analysis/src/Normalize.cpp b/Analysis/src/Normalize.cpp index ce8f96cc..a96b557d 100644 --- a/Analysis/src/Normalize.cpp +++ b/Analysis/src/Normalize.cpp @@ -14,6 +14,7 @@ LUAU_FASTFLAGVARIABLE(DebugLuauCopyBeforeNormalizing, false) LUAU_FASTINTVARIABLE(LuauNormalizeIterationLimit, 1200); LUAU_FASTFLAGVARIABLE(LuauNormalizeCombineTableFix, false); LUAU_FASTFLAGVARIABLE(LuauNormalizeFlagIsConservative, false); +LUAU_FASTFLAGVARIABLE(LuauFixNormalizationOfCyclicUnions, false); LUAU_FASTFLAG(LuauUnknownAndNeverType) LUAU_FASTFLAG(LuauQuantifyConstrained) @@ -340,13 +341,19 @@ struct Normalize final : TypeVarVisitor return false; UnionTypeVar* utv = &const_cast(utvRef); - std::vector options = std::move(utv->options); + + // TODO: Clip tempOptions and optionsRef when clipping FFlag::LuauFixNormalizationOfCyclicUnions + std::vector tempOptions; + if (!FFlag::LuauFixNormalizationOfCyclicUnions) + tempOptions = std::move(utv->options); + + std::vector& optionsRef = FFlag::LuauFixNormalizationOfCyclicUnions ? utv->options : tempOptions; // We might transmute, so it's not safe to rely on the builtin traversal logic of visitTypeVar - for (TypeId option : options) + for (TypeId option : optionsRef) traverse(option); - std::vector newOptions = normalizeUnion(options); + std::vector newOptions = normalizeUnion(optionsRef); const bool normal = areNormal(newOptions, seen, ice); @@ -371,51 +378,106 @@ struct Normalize final : TypeVarVisitor IntersectionTypeVar* itv = &const_cast(itvRef); - std::vector oldParts = std::move(itv->parts); - - for (TypeId part : oldParts) - traverse(part); - - std::vector tables; - for (TypeId part : oldParts) + if (FFlag::LuauFixNormalizationOfCyclicUnions) { - part = follow(part); - if (get(part)) - tables.push_back(part); - else - { - Replacer replacer{&arena, nullptr, nullptr}; // FIXME this is super super WEIRD - combineIntoIntersection(replacer, itv, part); - } - } + std::vector oldParts = itv->parts; + IntersectionTypeVar newIntersection; - // Don't allocate a new table if there's just one in the intersection. - if (tables.size() == 1) - itv->parts.push_back(tables[0]); - else if (!tables.empty()) - { - const TableTypeVar* first = get(tables[0]); - LUAU_ASSERT(first); + for (TypeId part : oldParts) + traverse(part); - TypeId newTable = arena.addType(TableTypeVar{first->state, first->level}); - TableTypeVar* ttv = getMutable(newTable); - for (TypeId part : tables) + std::vector tables; + for (TypeId part : oldParts) { - // Intuition: If combineIntoTable() needs to clone a table, any references to 'part' are cyclic and need - // to be rewritten to point at 'newTable' in the clone. - Replacer replacer{&arena, part, newTable}; - combineIntoTable(replacer, ttv, part); + part = follow(part); + if (get(part)) + tables.push_back(part); + else + { + Replacer replacer{&arena, nullptr, nullptr}; // FIXME this is super super WEIRD + combineIntoIntersection(replacer, &newIntersection, part); + } } - itv->parts.push_back(newTable); + // Don't allocate a new table if there's just one in the intersection. + if (tables.size() == 1) + newIntersection.parts.push_back(tables[0]); + else if (!tables.empty()) + { + const TableTypeVar* first = get(tables[0]); + LUAU_ASSERT(first); + + TypeId newTable = arena.addType(TableTypeVar{first->state, first->level}); + TableTypeVar* ttv = getMutable(newTable); + for (TypeId part : tables) + { + // Intuition: If combineIntoTable() needs to clone a table, any references to 'part' are cyclic and need + // to be rewritten to point at 'newTable' in the clone. + Replacer replacer{&arena, part, newTable}; + combineIntoTable(replacer, ttv, part); + } + + newIntersection.parts.push_back(newTable); + } + + itv->parts = std::move(newIntersection.parts); + + asMutable(ty)->normal = areNormal(itv->parts, seen, ice); + + if (itv->parts.size() == 1) + { + TypeId part = itv->parts[0]; + *asMutable(ty) = BoundTypeVar{part}; + } } - - asMutable(ty)->normal = areNormal(itv->parts, seen, ice); - - if (itv->parts.size() == 1) + else { - TypeId part = itv->parts[0]; - *asMutable(ty) = BoundTypeVar{part}; + std::vector oldParts = std::move(itv->parts); + + for (TypeId part : oldParts) + traverse(part); + + std::vector tables; + for (TypeId part : oldParts) + { + part = follow(part); + if (get(part)) + tables.push_back(part); + else + { + Replacer replacer{&arena, nullptr, nullptr}; // FIXME this is super super WEIRD + combineIntoIntersection(replacer, itv, part); + } + } + + // Don't allocate a new table if there's just one in the intersection. + if (tables.size() == 1) + itv->parts.push_back(tables[0]); + else if (!tables.empty()) + { + const TableTypeVar* first = get(tables[0]); + LUAU_ASSERT(first); + + TypeId newTable = arena.addType(TableTypeVar{first->state, first->level}); + TableTypeVar* ttv = getMutable(newTable); + for (TypeId part : tables) + { + // Intuition: If combineIntoTable() needs to clone a table, any references to 'part' are cyclic and need + // to be rewritten to point at 'newTable' in the clone. + Replacer replacer{&arena, part, newTable}; + combineIntoTable(replacer, ttv, part); + } + + itv->parts.push_back(newTable); + } + + asMutable(ty)->normal = areNormal(itv->parts, seen, ice); + + if (itv->parts.size() == 1) + { + TypeId part = itv->parts[0]; + *asMutable(ty) = BoundTypeVar{part}; + } } return false; @@ -590,6 +652,24 @@ struct Normalize final : TypeVarVisitor table->props.insert({propName, prop}); } + if (FFlag::LuauFixNormalizationOfCyclicUnions) + { + if (tyTable->indexer) + { + if (table->indexer) + { + table->indexer->indexType = combine(replacer, replacer.smartClone(tyTable->indexer->indexType), table->indexer->indexType); + table->indexer->indexResultType = + combine(replacer, replacer.smartClone(tyTable->indexer->indexResultType), table->indexer->indexResultType); + } + else + { + table->indexer = + TableIndexer{replacer.smartClone(tyTable->indexer->indexType), replacer.smartClone(tyTable->indexer->indexResultType)}; + } + } + } + table->state = combineTableStates(table->state, tyTable->state); table->level = max(table->level, tyTable->level); } diff --git a/Analysis/src/ToString.cpp b/Analysis/src/ToString.cpp index 6bc1d572..4c606922 100644 --- a/Analysis/src/ToString.cpp +++ b/Analysis/src/ToString.cpp @@ -19,7 +19,6 @@ LUAU_FASTFLAG(LuauUnknownAndNeverType) * Fair warning: Setting this will break a lot of Luau unit tests. */ LUAU_FASTFLAGVARIABLE(DebugLuauVerboseTypeNames, false) -LUAU_FASTFLAGVARIABLE(LuauToStringTableBracesNewlines, false) namespace Luau { @@ -572,54 +571,22 @@ struct TypeVarStringifier { case TableState::Sealed: state.result.invalid = true; - if (FFlag::LuauToStringTableBracesNewlines) - { - openbrace = "{|"; - closedbrace = "|}"; - } - else - { - openbrace = "{| "; - closedbrace = " |}"; - } + openbrace = "{|"; + closedbrace = "|}"; break; case TableState::Unsealed: - if (FFlag::LuauToStringTableBracesNewlines) - { - openbrace = "{"; - closedbrace = "}"; - } - else - { - openbrace = "{ "; - closedbrace = " }"; - } + openbrace = "{"; + closedbrace = "}"; break; case TableState::Free: state.result.invalid = true; - if (FFlag::LuauToStringTableBracesNewlines) - { - openbrace = "{-"; - closedbrace = "-}"; - } - else - { - openbrace = "{- "; - closedbrace = " -}"; - } + openbrace = "{-"; + closedbrace = "-}"; break; case TableState::Generic: state.result.invalid = true; - if (FFlag::LuauToStringTableBracesNewlines) - { - openbrace = "{+"; - closedbrace = "+}"; - } - else - { - openbrace = "{+ "; - closedbrace = " +}"; - } + openbrace = "{+"; + closedbrace = "+}"; break; } @@ -638,8 +605,7 @@ struct TypeVarStringifier bool comma = false; if (ttv.indexer) { - if (FFlag::LuauToStringTableBracesNewlines) - state.newline(); + state.newline(); state.emit("["); stringify(ttv.indexer->indexType); state.emit("]: "); @@ -656,10 +622,8 @@ struct TypeVarStringifier state.emit(","); state.newline(); } - else if (FFlag::LuauToStringTableBracesNewlines) - { + else state.newline(); - } size_t length = state.result.name.length() - oldLength; @@ -686,13 +650,10 @@ struct TypeVarStringifier } state.dedent(); - if (FFlag::LuauToStringTableBracesNewlines) - { - if (comma) - state.newline(); - else - state.emit(" "); - } + if (comma) + state.newline(); + else + state.emit(" "); state.emit(closedbrace); state.unsee(&ttv); @@ -860,7 +821,6 @@ struct TypeVarStringifier { state.emit("never"); } - }; struct TypePackStringifier diff --git a/Analysis/src/TypeChecker2.cpp b/Analysis/src/TypeChecker2.cpp index 30e498af..d3b9655d 100644 --- a/Analysis/src/TypeChecker2.cpp +++ b/Analysis/src/TypeChecker2.cpp @@ -322,10 +322,13 @@ struct TypeChecker2 : public AstVisitor { pack = follow(pack); - while (auto tp = get(pack)) + while (true) { - if (tp->head.empty() && tp->tail) + auto tp = get(pack); + if (tp && tp->head.empty() && tp->tail) pack = *tp->tail; + else + break; } if (auto ty = first(pack)) diff --git a/Analysis/src/TypeInfer.cpp b/Analysis/src/TypeInfer.cpp index 01939fda..20777928 100644 --- a/Analysis/src/TypeInfer.cpp +++ b/Analysis/src/TypeInfer.cpp @@ -48,6 +48,8 @@ LUAU_FASTFLAGVARIABLE(LuauFalsyPredicateReturnsNilInstead, false) LUAU_FASTFLAGVARIABLE(LuauCheckLenMT, false) LUAU_FASTFLAGVARIABLE(LuauCheckGenericHOFTypes, false) LUAU_FASTFLAGVARIABLE(LuauBinaryNeedsExpectedTypesToo, false) +LUAU_FASTFLAGVARIABLE(LuauNeverTypesAndOperatorsInference, false) +LUAU_FASTFLAGVARIABLE(LuauReturnsFromCallsitesAreNotWidened, false) namespace Luau { @@ -2443,8 +2445,15 @@ WithPredicate TypeChecker::checkExpr(const ScopePtr& scope, const AstExp operandType = stripFromNilAndReport(operandType, expr.location); - if (get(operandType) || get(operandType)) - return {!FFlag::LuauUnknownAndNeverType ? errorRecoveryType(scope) : operandType}; + // # operator is guaranteed to return number + if ((FFlag::LuauNeverTypesAndOperatorsInference && get(operandType)) || get(operandType) || + get(operandType)) + { + if (FFlag::LuauNeverTypesAndOperatorsInference) + return {numberType}; + else + return {!FFlag::LuauUnknownAndNeverType ? errorRecoveryType(scope) : operandType}; + } DenseHashSet seen{nullptr}; @@ -2610,6 +2619,13 @@ TypeId TypeChecker::checkRelationalOperation( case AstExprBinary::CompareGe: case AstExprBinary::CompareLe: { + if (FFlag::LuauNeverTypesAndOperatorsInference) + { + // If one of the operand is never, it doesn't make sense to unify these. + if (get(lhsType) || get(rhsType)) + return booleanType; + } + /* Subtlety here: * We need to do this unification first, but there are situations where we don't actually want to * report any problems that might have been surfaced as a result of this step because we might already @@ -2787,8 +2803,10 @@ TypeId TypeChecker::checkBinaryOperation( // If we know nothing at all about the lhs type, we can usually say nothing about the result. // The notable exception to this is the equality and inequality operators, which always produce a boolean. - const bool lhsIsAny = get(lhsType) || get(lhsType); - const bool rhsIsAny = get(rhsType) || get(rhsType); + const bool lhsIsAny = get(lhsType) || get(lhsType) || + (FFlag::LuauUnknownAndNeverType && FFlag::LuauNeverTypesAndOperatorsInference && get(lhsType)); + const bool rhsIsAny = get(rhsType) || get(rhsType) || + (FFlag::LuauUnknownAndNeverType && FFlag::LuauNeverTypesAndOperatorsInference && get(rhsType)); if (lhsIsAny) return lhsType; @@ -3775,7 +3793,10 @@ void TypeChecker::checkArgumentList( } TypePackId varPack = addTypePack(TypePackVar{TypePack{rest, paramIter.tail()}}); - state.tryUnify(varPack, tail); + if (FFlag::LuauReturnsFromCallsitesAreNotWidened) + state.tryUnify(tail, varPack); + else + state.tryUnify(varPack, tail); return; } } @@ -4414,7 +4435,8 @@ WithPredicate TypeChecker::checkExprList(const ScopePtr& scope, cons if (FFlag::LuauUnknownAndNeverType && containsNever(typePack)) { - // f(), g() where f() returns (never, string) or (string, never) means this whole TypePackId is uninhabitable, so return (never, ...never) + // f(), g() where f() returns (never, string) or (string, never) means this whole TypePackId is uninhabitable, so return (never, + // ...never) uninhabitable = true; continue; } @@ -4436,7 +4458,8 @@ WithPredicate TypeChecker::checkExprList(const ScopePtr& scope, cons if (FFlag::LuauUnknownAndNeverType && get(type)) { - // f(), g() where f() returns (never, string) or (string, never) means this whole TypePackId is uninhabitable, so return (never, ...never) + // f(), g() where f() returns (never, string) or (string, never) means this whole TypePackId is uninhabitable, so return (never, + // ...never) uninhabitable = true; continue; } diff --git a/Analysis/src/TypeVar.cpp b/Analysis/src/TypeVar.cpp index f884ad77..ebaf5906 100644 --- a/Analysis/src/TypeVar.cpp +++ b/Analysis/src/TypeVar.cpp @@ -26,6 +26,7 @@ LUAU_FASTINT(LuauTypeInferRecursionLimit) LUAU_FASTFLAG(LuauUnknownAndNeverType) LUAU_FASTFLAGVARIABLE(LuauDeduceGmatchReturnTypes, false) LUAU_FASTFLAGVARIABLE(LuauMaybeGenericIntersectionTypes, false) +LUAU_FASTFLAGVARIABLE(LuauDeduceFindMatchReturnTypes, false) namespace Luau { @@ -36,6 +37,12 @@ std::optional> magicFunctionFormat( static std::optional> magicFunctionGmatch( TypeChecker& typechecker, const ScopePtr& scope, const AstExprCall& expr, WithPredicate withPredicate); +static std::optional> magicFunctionMatch( + TypeChecker& typechecker, const ScopePtr& scope, const AstExprCall& expr, WithPredicate withPredicate); + +static std::optional> magicFunctionFind( + TypeChecker& typechecker, const ScopePtr& scope, const AstExprCall& expr, WithPredicate withPredicate); + TypeId follow(TypeId t) { return follow(t, [](TypeId t) { @@ -164,10 +171,12 @@ bool isNumber(TypeId ty) // Returns true when ty is a subtype of string bool isString(TypeId ty) { - if (isPrim(ty, PrimitiveTypeVar::String) || get(get(follow(ty)))) + ty = follow(ty); + + if (isPrim(ty, PrimitiveTypeVar::String) || get(get(ty))) return true; - if (auto utv = get(follow(ty))) + if (auto utv = get(ty)) return std::all_of(begin(utv), end(utv), isString); return false; @@ -178,8 +187,8 @@ bool maybeString(TypeId ty) { ty = follow(ty); - if (isPrim(ty, PrimitiveTypeVar::String) || get(ty)) - return true; + if (isPrim(ty, PrimitiveTypeVar::String) || get(ty)) + return true; if (auto utv = get(ty)) return std::any_of(begin(utv), end(utv), maybeString); @@ -233,6 +242,8 @@ bool isOverloadedFunction(TypeId ty) std::optional getMetatable(TypeId type) { + type = follow(type); + if (const MetatableTypeVar* mtType = get(type)) return mtType->metatable; else if (const ClassTypeVar* classType = get(type)) @@ -765,18 +776,24 @@ TypeId SingletonTypes::makeStringMetatable() makeFunction(*arena, stringType, {}, {}, {stringType}, {}, {arena->addType(FunctionTypeVar{emptyPack, stringVariadicList})}); attachMagicFunction(gmatchFunc, magicFunctionGmatch); + const TypeId matchFunc = arena->addType(FunctionTypeVar{arena->addTypePack({stringType, stringType, optionalNumber}), + arena->addTypePack(TypePackVar{VariadicTypePack{FFlag::LuauDeduceFindMatchReturnTypes ? stringType : optionalString}})}); + attachMagicFunction(matchFunc, magicFunctionMatch); + + const TypeId findFunc = arena->addType(FunctionTypeVar{arena->addTypePack({stringType, stringType, optionalNumber, optionalBoolean}), + arena->addTypePack(TypePack{{optionalNumber, optionalNumber}, stringVariadicList})}); + attachMagicFunction(findFunc, magicFunctionFind); + TableTypeVar::Props stringLib = { {"byte", {arena->addType(FunctionTypeVar{arena->addTypePack({stringType, optionalNumber, optionalNumber}), numberVariadicList})}}, {"char", {arena->addType(FunctionTypeVar{numberVariadicList, arena->addTypePack({stringType})})}}, - {"find", {arena->addType(FunctionTypeVar{arena->addTypePack({stringType, stringType, optionalNumber, optionalBoolean}), - arena->addTypePack(TypePack{{optionalNumber, optionalNumber}, stringVariadicList})})}}, + {"find", {findFunc}}, {"format", {formatFn}}, // FIXME {"gmatch", {gmatchFunc}}, {"gsub", {gsubFunc}}, {"len", {makeFunction(*arena, stringType, {}, {}, {}, {}, {numberType})}}, {"lower", {stringToStringType}}, - {"match", {arena->addType(FunctionTypeVar{arena->addTypePack({stringType, stringType, optionalNumber}), - arena->addTypePack(TypePackVar{VariadicTypePack{optionalString}})})}}, + {"match", {matchFunc}}, {"rep", {makeFunction(*arena, stringType, {}, {}, {numberType}, {}, {stringType})}}, {"reverse", {stringToStringType}}, {"sub", {makeFunction(*arena, stringType, {}, {}, {numberType, optionalNumber}, {}, {stringType})}}, @@ -1213,6 +1230,102 @@ static std::optional> magicFunctionGmatch( return WithPredicate{arena.addTypePack({iteratorType})}; } +static std::optional> magicFunctionMatch( + TypeChecker& typechecker, const ScopePtr& scope, const AstExprCall& expr, WithPredicate withPredicate) +{ + if (!FFlag::LuauDeduceFindMatchReturnTypes) + return std::nullopt; + + auto [paramPack, _predicates] = withPredicate; + const auto& [params, tail] = flatten(paramPack); + + if (params.size() < 2 || params.size() > 3) + return std::nullopt; + + TypeArena& arena = typechecker.currentModule->internalTypes; + + AstExprConstantString* pattern = nullptr; + size_t patternIndex = expr.self ? 0 : 1; + if (expr.args.size > patternIndex) + pattern = expr.args.data[patternIndex]->as(); + + if (!pattern) + return std::nullopt; + + std::vector returnTypes = parsePatternString(typechecker, pattern->value.data, pattern->value.size); + + if (returnTypes.empty()) + return std::nullopt; + + typechecker.unify(params[0], typechecker.stringType, expr.args.data[0]->location); + + const TypeId optionalNumber = arena.addType(UnionTypeVar{{typechecker.nilType, typechecker.numberType}}); + + size_t initIndex = expr.self ? 1 : 2; + if (params.size() == 3 && expr.args.size > initIndex) + typechecker.unify(params[2], optionalNumber, expr.args.data[initIndex]->location); + + const TypePackId returnList = arena.addTypePack(returnTypes); + return WithPredicate{returnList}; +} + +static std::optional> magicFunctionFind( + TypeChecker& typechecker, const ScopePtr& scope, const AstExprCall& expr, WithPredicate withPredicate) +{ + if (!FFlag::LuauDeduceFindMatchReturnTypes) + return std::nullopt; + + auto [paramPack, _predicates] = withPredicate; + const auto& [params, tail] = flatten(paramPack); + + if (params.size() < 2 || params.size() > 4) + return std::nullopt; + + TypeArena& arena = typechecker.currentModule->internalTypes; + + AstExprConstantString* pattern = nullptr; + size_t patternIndex = expr.self ? 0 : 1; + if (expr.args.size > patternIndex) + pattern = expr.args.data[patternIndex]->as(); + + if (!pattern) + return std::nullopt; + + bool plain = false; + size_t plainIndex = expr.self ? 2 : 3; + if (expr.args.size > plainIndex) + { + AstExprConstantBool* p = expr.args.data[plainIndex]->as(); + plain = p && p->value; + } + + std::vector returnTypes; + if (!plain) + { + returnTypes = parsePatternString(typechecker, pattern->value.data, pattern->value.size); + + if (returnTypes.empty()) + return std::nullopt; + } + + typechecker.unify(params[0], typechecker.stringType, expr.args.data[0]->location); + + const TypeId optionalNumber = arena.addType(UnionTypeVar{{typechecker.nilType, typechecker.numberType}}); + const TypeId optionalBoolean = arena.addType(UnionTypeVar{{typechecker.nilType, typechecker.booleanType}}); + + size_t initIndex = expr.self ? 1 : 2; + if (params.size() >= 3 && expr.args.size > initIndex) + typechecker.unify(params[2], optionalNumber, expr.args.data[initIndex]->location); + + if (params.size() == 4 && expr.args.size > plainIndex) + typechecker.unify(params[3], optionalBoolean, expr.args.data[plainIndex]->location); + + returnTypes.insert(returnTypes.begin(), {optionalNumber, optionalNumber}); + + const TypePackId returnList = arena.addTypePack(returnTypes); + return WithPredicate{returnList}; +} + std::vector filterMap(TypeId type, TypeIdPredicate predicate) { type = follow(type); diff --git a/Analysis/src/Unifier.cpp b/Analysis/src/Unifier.cpp index 44a3b854..e099817f 100644 --- a/Analysis/src/Unifier.cpp +++ b/Analysis/src/Unifier.cpp @@ -21,6 +21,7 @@ LUAU_FASTFLAG(LuauLowerBoundsCalculation); LUAU_FASTFLAG(LuauErrorRecoveryType); LUAU_FASTFLAG(LuauUnknownAndNeverType) LUAU_FASTFLAG(LuauQuantifyConstrained) +LUAU_FASTFLAGVARIABLE(LuauScalarShapeSubtyping, false) namespace Luau { @@ -432,7 +433,7 @@ void Unifier::tryUnify_(TypeId subTy, TypeId superTy, bool isFunctionCall, bool { // Normally, if the subtype is free, it should not be bound to any, unknown, or error types. // But for bug compatibility, we'll only apply this rule to unknown. Doing this will silence cascading type errors. - if (get(superTy)) + if (log.get(superTy)) return; } @@ -473,10 +474,10 @@ void Unifier::tryUnify_(TypeId subTy, TypeId superTy, bool isFunctionCall, bool return tryUnifyWithAny(superTy, subTy); } - if (get(subTy)) + if (log.get(subTy)) return tryUnifyWithAny(superTy, subTy); - if (get(subTy)) + if (log.get(subTy)) return tryUnifyWithAny(superTy, subTy); auto& cache = sharedState.cachedUnify; @@ -538,6 +539,16 @@ void Unifier::tryUnify_(TypeId subTy, TypeId superTy, bool isFunctionCall, bool { tryUnifyTables(subTy, superTy, isIntersection); } + else if (FFlag::LuauScalarShapeSubtyping && log.get(superTy) && + (log.get(subTy) || log.get(subTy))) + { + tryUnifyScalarShape(subTy, superTy, /*reversed*/ false); + } + else if (FFlag::LuauScalarShapeSubtyping && log.get(subTy) && + (log.get(superTy) || log.get(superTy))) + { + tryUnifyScalarShape(subTy, superTy, /*reversed*/ true); + } // tryUnifyWithMetatable assumes its first argument is a MetatableTypeVar. The check is otherwise symmetrical. else if (log.getMutable(superTy)) @@ -1600,6 +1611,60 @@ void Unifier::tryUnifyTables(TypeId subTy, TypeId superTy, bool isIntersection) } } +void Unifier::tryUnifyScalarShape(TypeId subTy, TypeId superTy, bool reversed) +{ + LUAU_ASSERT(FFlag::LuauScalarShapeSubtyping); + + TypeId osubTy = subTy; + TypeId osuperTy = superTy; + + if (reversed) + std::swap(subTy, superTy); + + if (auto ttv = log.get(superTy); !ttv || ttv->state != TableState::Free) + return reportError(TypeError{location, TypeMismatch{osuperTy, osubTy}}); + + auto fail = [&](std::optional e) { + std::string reason = "The former's metatable does not satisfy the requirements."; + if (e) + reportError(TypeError{location, TypeMismatch{osuperTy, osubTy, reason, *e}}); + else + reportError(TypeError{location, TypeMismatch{osuperTy, osubTy, reason}}); + }; + + // Given t1 where t1 = { lower: (t1) -> (a, b...) } + // It should be the case that `string <: t1` iff `(subtype's metatable).__index <: t1` + if (auto metatable = getMetatable(subTy)) + { + auto mttv = log.get(*metatable); + if (!mttv) + fail(std::nullopt); + + if (auto it = mttv->props.find("__index"); it != mttv->props.end()) + { + TypeId ty = it->second.type; + Unifier child = makeChildUnifier(); + child.tryUnify_(ty, superTy); + + if (auto e = hasUnificationTooComplex(child.errors)) + reportError(*e); + else if (!child.errors.empty()) + fail(child.errors.front()); + + log.concat(std::move(child.log)); + + return; + } + else + { + return fail(std::nullopt); + } + } + + reportError(TypeError{location, TypeMismatch{osuperTy, osubTy}}); + return; +} + TypeId Unifier::deeplyOptional(TypeId ty, std::unordered_map seen) { ty = follow(ty); @@ -1916,7 +1981,8 @@ void Unifier::tryUnifyWithAny(TypeId subTy, TypeId anyTy) sharedState.tempSeenTy.clear(); sharedState.tempSeenTp.clear(); - Luau::tryUnifyWithAny(queue, *this, sharedState.tempSeenTy, sharedState.tempSeenTp, types, FFlag::LuauUnknownAndNeverType ? anyTy : getSingletonTypes().anyType, anyTp); + Luau::tryUnifyWithAny(queue, *this, sharedState.tempSeenTy, sharedState.tempSeenTp, types, + FFlag::LuauUnknownAndNeverType ? anyTy : getSingletonTypes().anyType, anyTp); } void Unifier::tryUnifyWithAny(TypePackId subTy, TypePackId anyTp) diff --git a/Ast/src/Parser.cpp b/Ast/src/Parser.cpp index b7fa7889..779bd279 100644 --- a/Ast/src/Parser.cpp +++ b/Ast/src/Parser.cpp @@ -24,6 +24,8 @@ bool lua_telemetry_parsed_named_non_function_type = false; LUAU_FASTFLAGVARIABLE(LuauErrorParseIntegerIssues, false) LUAU_DYNAMIC_FASTFLAGVARIABLE(LuaReportParseIntegerIssues, false) +LUAU_FASTFLAGVARIABLE(LuauAlwaysCaptureHotComments, false) + bool lua_telemetry_parsed_out_of_range_bin_integer = false; bool lua_telemetry_parsed_out_of_range_hex_integer = false; bool lua_telemetry_parsed_double_prefix_hex_integer = false; @@ -2918,21 +2920,23 @@ AstTypeError* Parser::reportTypeAnnotationError(const Location& location, const void Parser::nextLexeme() { - if (options.captureComments) + if (options.captureComments || FFlag::LuauAlwaysCaptureHotComments) { Lexeme::Type type = lexer.next(/* skipComments= */ false, true).type; while (type == Lexeme::BrokenComment || type == Lexeme::Comment || type == Lexeme::BlockComment) { const Lexeme& lexeme = lexer.current(); - commentLocations.push_back(Comment{lexeme.type, lexeme.location}); + + if (options.captureComments) + commentLocations.push_back(Comment{lexeme.type, lexeme.location}); // Subtlety: Broken comments are weird because we record them as comments AND pass them to the parser as a lexeme. // The parser will turn this into a proper syntax error. if (lexeme.type == Lexeme::BrokenComment) return; - // Comments starting with ! are called "hot comments" and contain directives for type checking / linting + // Comments starting with ! are called "hot comments" and contain directives for type checking / linting / compiling if (lexeme.type == Lexeme::Comment && lexeme.length && lexeme.data[0] == '!') { const char* text = lexeme.data; diff --git a/CMakeLists.txt b/CMakeLists.txt index e256e234..92006344 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -175,6 +175,7 @@ endif() if(LUAU_BUILD_TESTS) target_compile_options(Luau.UnitTest PRIVATE ${LUAU_OPTIONS}) + target_compile_definitions(Luau.UnitTest PRIVATE DOCTEST_CONFIG_DOUBLE_STRINGIFY) target_include_directories(Luau.UnitTest PRIVATE extern) target_link_libraries(Luau.UnitTest PRIVATE Luau.Analysis Luau.Compiler Luau.CodeGen) diff --git a/Compiler/include/Luau/Compiler.h b/Compiler/include/Luau/Compiler.h index 65e962da..eec70d7a 100644 --- a/Compiler/include/Luau/Compiler.h +++ b/Compiler/include/Luau/Compiler.h @@ -8,8 +8,8 @@ namespace Luau { -class AstStatBlock; class AstNameTable; +struct ParseResult; class BytecodeBuilder; class BytecodeEncoder; @@ -58,7 +58,7 @@ private: }; // compiles bytecode into bytecode builder using either a pre-parsed AST or parsing it from source; throws on errors -void compileOrThrow(BytecodeBuilder& bytecode, AstStatBlock* root, const AstNameTable& names, const CompileOptions& options = {}); +void compileOrThrow(BytecodeBuilder& bytecode, const ParseResult& parseResult, const AstNameTable& names, const CompileOptions& options = {}); void compileOrThrow(BytecodeBuilder& bytecode, const std::string& source, const CompileOptions& options = {}, const ParseOptions& parseOptions = {}); // compiles bytecode into a bytecode blob, that either contains the valid bytecode or an encoded error that luau_load can decode diff --git a/Compiler/src/BuiltinFolding.cpp b/Compiler/src/BuiltinFolding.cpp new file mode 100644 index 00000000..e76da4e2 --- /dev/null +++ b/Compiler/src/BuiltinFolding.cpp @@ -0,0 +1,463 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +#include "BuiltinFolding.h" + +#include "Luau/Bytecode.h" + +#include + +namespace Luau +{ +namespace Compile +{ + +const double kRadDeg = 3.14159265358979323846 / 180.0; + +static Constant cvar() +{ + return Constant(); +} + +static Constant cbool(bool v) +{ + Constant res = {Constant::Type_Boolean}; + res.valueBoolean = v; + return res; +} + +static Constant cnum(double v) +{ + Constant res = {Constant::Type_Number}; + res.valueNumber = v; + return res; +} + +static Constant cstring(const char* v) +{ + Constant res = {Constant::Type_String}; + res.stringLength = unsigned(strlen(v)); + res.valueString = v; + return res; +} + +static Constant ctype(const Constant& c) +{ + LUAU_ASSERT(c.type != Constant::Type_Unknown); + + switch (c.type) + { + case Constant::Type_Nil: + return cstring("nil"); + + case Constant::Type_Boolean: + return cstring("boolean"); + + case Constant::Type_Number: + return cstring("number"); + + case Constant::Type_String: + return cstring("string"); + + default: + LUAU_ASSERT(!"Unsupported constant type"); + return cvar(); + } +} + +static uint32_t bit32(double v) +{ + // convert through signed 64-bit integer to match runtime behavior and gracefully truncate negative integers + return uint32_t(int64_t(v)); +} + +Constant foldBuiltin(int bfid, const Constant* args, size_t count) +{ + switch (bfid) + { + case LBF_MATH_ABS: + if (count == 1 && args[0].type == Constant::Type_Number) + return cnum(fabs(args[0].valueNumber)); + break; + + case LBF_MATH_ACOS: + if (count == 1 && args[0].type == Constant::Type_Number) + return cnum(acos(args[0].valueNumber)); + break; + + case LBF_MATH_ASIN: + if (count == 1 && args[0].type == Constant::Type_Number) + return cnum(asin(args[0].valueNumber)); + break; + + case LBF_MATH_ATAN2: + if (count == 2 && args[0].type == Constant::Type_Number && args[1].type == Constant::Type_Number) + return cnum(atan2(args[0].valueNumber, args[1].valueNumber)); + break; + + case LBF_MATH_ATAN: + if (count == 1 && args[0].type == Constant::Type_Number) + return cnum(atan(args[0].valueNumber)); + break; + + case LBF_MATH_CEIL: + if (count == 1 && args[0].type == Constant::Type_Number) + return cnum(ceil(args[0].valueNumber)); + break; + + case LBF_MATH_COSH: + if (count == 1 && args[0].type == Constant::Type_Number) + return cnum(cosh(args[0].valueNumber)); + break; + + case LBF_MATH_COS: + if (count == 1 && args[0].type == Constant::Type_Number) + return cnum(cos(args[0].valueNumber)); + break; + + case LBF_MATH_DEG: + if (count == 1 && args[0].type == Constant::Type_Number) + return cnum(args[0].valueNumber / kRadDeg); + break; + + case LBF_MATH_EXP: + if (count == 1 && args[0].type == Constant::Type_Number) + return cnum(exp(args[0].valueNumber)); + break; + + case LBF_MATH_FLOOR: + if (count == 1 && args[0].type == Constant::Type_Number) + return cnum(floor(args[0].valueNumber)); + break; + + case LBF_MATH_FMOD: + if (count == 2 && args[0].type == Constant::Type_Number && args[1].type == Constant::Type_Number) + return cnum(fmod(args[0].valueNumber, args[1].valueNumber)); + break; + + // Note: FREXP isn't folded since it returns multiple values + + case LBF_MATH_LDEXP: + if (count == 2 && args[0].type == Constant::Type_Number && args[1].type == Constant::Type_Number) + return cnum(ldexp(args[0].valueNumber, int(args[1].valueNumber))); + break; + + case LBF_MATH_LOG10: + if (count == 1 && args[0].type == Constant::Type_Number) + return cnum(log10(args[0].valueNumber)); + break; + + case LBF_MATH_LOG: + if (count == 1 && args[0].type == Constant::Type_Number) + return cnum(log(args[0].valueNumber)); + else if (count == 2 && args[0].type == Constant::Type_Number && args[1].type == Constant::Type_Number) + { + if (args[1].valueNumber == 2.0) + return cnum(log2(args[0].valueNumber)); + else if (args[1].valueNumber == 10.0) + return cnum(log10(args[0].valueNumber)); + else + return cnum(log(args[0].valueNumber) / log(args[1].valueNumber)); + } + break; + + case LBF_MATH_MAX: + if (count >= 1 && args[0].type == Constant::Type_Number) + { + double r = args[0].valueNumber; + + for (size_t i = 1; i < count; ++i) + { + if (args[i].type != Constant::Type_Number) + return cvar(); + + double a = args[i].valueNumber; + + r = (a > r) ? a : r; + } + + return cnum(r); + } + break; + + case LBF_MATH_MIN: + if (count >= 1 && args[0].type == Constant::Type_Number) + { + double r = args[0].valueNumber; + + for (size_t i = 1; i < count; ++i) + { + if (args[i].type != Constant::Type_Number) + return cvar(); + + double a = args[i].valueNumber; + + r = (a < r) ? a : r; + } + + return cnum(r); + } + break; + + // Note: MODF isn't folded since it returns multiple values + + case LBF_MATH_POW: + if (count == 2 && args[0].type == Constant::Type_Number && args[1].type == Constant::Type_Number) + return cnum(pow(args[0].valueNumber, args[1].valueNumber)); + break; + + case LBF_MATH_RAD: + if (count == 1 && args[0].type == Constant::Type_Number) + return cnum(args[0].valueNumber * kRadDeg); + break; + + case LBF_MATH_SINH: + if (count == 1 && args[0].type == Constant::Type_Number) + return cnum(sinh(args[0].valueNumber)); + break; + + case LBF_MATH_SIN: + if (count == 1 && args[0].type == Constant::Type_Number) + return cnum(sin(args[0].valueNumber)); + break; + + case LBF_MATH_SQRT: + if (count == 1 && args[0].type == Constant::Type_Number) + return cnum(sqrt(args[0].valueNumber)); + break; + + case LBF_MATH_TANH: + if (count == 1 && args[0].type == Constant::Type_Number) + return cnum(tanh(args[0].valueNumber)); + break; + + case LBF_MATH_TAN: + if (count == 1 && args[0].type == Constant::Type_Number) + return cnum(tan(args[0].valueNumber)); + break; + + case LBF_BIT32_ARSHIFT: + if (count == 2 && args[0].type == Constant::Type_Number && args[1].type == Constant::Type_Number) + { + uint32_t u = bit32(args[0].valueNumber); + int s = int(args[1].valueNumber); + + if (unsigned(s) < 32) + return cnum(double(uint32_t(int32_t(u) >> s))); + } + break; + + case LBF_BIT32_BAND: + if (count >= 1 && args[0].type == Constant::Type_Number) + { + uint32_t r = bit32(args[0].valueNumber); + + for (size_t i = 1; i < count; ++i) + { + if (args[i].type != Constant::Type_Number) + return cvar(); + + r &= bit32(args[i].valueNumber); + } + + return cnum(double(r)); + } + break; + + case LBF_BIT32_BNOT: + if (count == 1 && args[0].type == Constant::Type_Number) + return cnum(double(uint32_t(~bit32(args[0].valueNumber)))); + break; + + case LBF_BIT32_BOR: + if (count >= 1 && args[0].type == Constant::Type_Number) + { + uint32_t r = bit32(args[0].valueNumber); + + for (size_t i = 1; i < count; ++i) + { + if (args[i].type != Constant::Type_Number) + return cvar(); + + r |= bit32(args[i].valueNumber); + } + + return cnum(double(r)); + } + break; + + case LBF_BIT32_BXOR: + if (count >= 1 && args[0].type == Constant::Type_Number) + { + uint32_t r = bit32(args[0].valueNumber); + + for (size_t i = 1; i < count; ++i) + { + if (args[i].type != Constant::Type_Number) + return cvar(); + + r ^= bit32(args[i].valueNumber); + } + + return cnum(double(r)); + } + break; + + case LBF_BIT32_BTEST: + if (count >= 1 && args[0].type == Constant::Type_Number) + { + uint32_t r = bit32(args[0].valueNumber); + + for (size_t i = 1; i < count; ++i) + { + if (args[i].type != Constant::Type_Number) + return cvar(); + + r &= bit32(args[i].valueNumber); + } + + return cbool(r != 0); + } + break; + + case LBF_BIT32_EXTRACT: + if (count == 3 && args[0].type == Constant::Type_Number && args[1].type == Constant::Type_Number && args[2].type == Constant::Type_Number) + { + uint32_t u = bit32(args[0].valueNumber); + int f = int(args[1].valueNumber); + int w = int(args[2].valueNumber); + + if (f >= 0 && w > 0 && f + w <= 32) + { + uint32_t m = ~(0xfffffffeu << (w - 1)); + + return cnum(double((u >> f) & m)); + } + } + break; + + case LBF_BIT32_LROTATE: + if (count == 2 && args[0].type == Constant::Type_Number && args[1].type == Constant::Type_Number) + { + uint32_t u = bit32(args[0].valueNumber); + int s = int(args[1].valueNumber); + + return cnum(double((u << (s & 31)) | (u >> ((32 - s) & 31)))); + } + break; + + case LBF_BIT32_LSHIFT: + if (count == 2 && args[0].type == Constant::Type_Number && args[1].type == Constant::Type_Number) + { + uint32_t u = bit32(args[0].valueNumber); + int s = int(args[1].valueNumber); + + if (unsigned(s) < 32) + return cnum(double(u << s)); + } + break; + + case LBF_BIT32_REPLACE: + if (count == 4 && args[0].type == Constant::Type_Number && args[1].type == Constant::Type_Number && args[2].type == Constant::Type_Number && + args[3].type == Constant::Type_Number) + { + uint32_t n = bit32(args[0].valueNumber); + uint32_t v = bit32(args[1].valueNumber); + int f = int(args[2].valueNumber); + int w = int(args[3].valueNumber); + + if (f >= 0 && w > 0 && f + w <= 32) + { + uint32_t m = ~(0xfffffffeu << (w - 1)); + + return cnum(double((n & ~(m << f)) | ((v & m) << f))); + } + } + break; + + case LBF_BIT32_RROTATE: + if (count == 2 && args[0].type == Constant::Type_Number && args[1].type == Constant::Type_Number) + { + uint32_t u = bit32(args[0].valueNumber); + int s = int(args[1].valueNumber); + + return cnum(double((u >> (s & 31)) | (u << ((32 - s) & 31)))); + } + break; + + case LBF_BIT32_RSHIFT: + if (count == 2 && args[0].type == Constant::Type_Number && args[1].type == Constant::Type_Number) + { + uint32_t u = bit32(args[0].valueNumber); + int s = int(args[1].valueNumber); + + if (unsigned(s) < 32) + return cnum(double(u >> s)); + } + break; + + case LBF_TYPE: + if (count == 1 && args[0].type != Constant::Type_Unknown) + return ctype(args[0]); + break; + + case LBF_STRING_BYTE: + if (count == 1 && args[0].type == Constant::Type_String) + { + if (args[0].stringLength > 0) + return cnum(double(uint8_t(args[0].valueString[0]))); + } + else if (count == 2 && args[0].type == Constant::Type_String && args[1].type == Constant::Type_Number) + { + int i = int(args[1].valueNumber); + + if (i > 0 && unsigned(i) <= args[0].stringLength) + return cnum(double(uint8_t(args[0].valueString[i - 1]))); + } + break; + + case LBF_STRING_LEN: + if (count == 1 && args[0].type == Constant::Type_String) + return cnum(double(args[0].stringLength)); + break; + + case LBF_TYPEOF: + if (count == 1 && args[0].type != Constant::Type_Unknown) + return ctype(args[0]); + break; + + case LBF_MATH_CLAMP: + if (count == 3 && args[0].type == Constant::Type_Number && args[1].type == Constant::Type_Number && args[2].type == Constant::Type_Number) + { + double min = args[1].valueNumber; + double max = args[2].valueNumber; + + if (min <= max) + { + double v = args[0].valueNumber; + v = v < min ? min : v; + v = v > max ? max : v; + + return cnum(v); + } + } + break; + + case LBF_MATH_SIGN: + if (count == 1 && args[0].type == Constant::Type_Number) + { + double v = args[0].valueNumber; + + return cnum(v > 0.0 ? 1.0 : v < 0.0 ? -1.0 : 0.0); + } + break; + + case LBF_MATH_ROUND: + if (count == 1 && args[0].type == Constant::Type_Number) + return cnum(round(args[0].valueNumber)); + break; + } + + return cvar(); +} + +} // namespace Compile +} // namespace Luau diff --git a/Compiler/src/BuiltinFolding.h b/Compiler/src/BuiltinFolding.h new file mode 100644 index 00000000..1904e14f --- /dev/null +++ b/Compiler/src/BuiltinFolding.h @@ -0,0 +1,14 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +#pragma once + +#include "ConstantFolding.h" + +namespace Luau +{ +namespace Compile +{ + +Constant foldBuiltin(int bfid, const Constant* args, size_t count); + +} // namespace Compile +} // namespace Luau diff --git a/Compiler/src/Builtins.cpp b/Compiler/src/Builtins.cpp index 6bd24b6d..3650c147 100644 --- a/Compiler/src/Builtins.cpp +++ b/Compiler/src/Builtins.cpp @@ -40,11 +40,8 @@ Builtin getBuiltin(AstExpr* node, const DenseHashMap& globals, } } -int getBuiltinFunctionId(const Builtin& builtin, const CompileOptions& options) +static int getBuiltinFunctionId(const Builtin& builtin, const CompileOptions& options) { - if (builtin.empty()) - return -1; - if (builtin.isGlobal("assert")) return LBF_ASSERT; @@ -200,5 +197,49 @@ int getBuiltinFunctionId(const Builtin& builtin, const CompileOptions& options) return -1; } +struct BuiltinVisitor : AstVisitor +{ + DenseHashMap& result; + + const DenseHashMap& globals; + const DenseHashMap& variables; + + const CompileOptions& options; + + BuiltinVisitor(DenseHashMap& result, const DenseHashMap& globals, + const DenseHashMap& variables, const CompileOptions& options) + : result(result) + , globals(globals) + , variables(variables) + , options(options) + { + } + + bool visit(AstExprCall* node) override + { + Builtin builtin = node->self ? Builtin() : getBuiltin(node->func, globals, variables); + if (builtin.empty()) + return true; + + int bfid = getBuiltinFunctionId(builtin, options); + + // getBuiltinFunctionId optimistically assumes all select() calls are builtin but actually the second argument must be a vararg + if (bfid == LBF_SELECT_VARARG && !(node->args.size == 2 && node->args.data[1]->is())) + bfid = -1; + + if (bfid >= 0) + result[node] = bfid; + + return true; // propagate to nested calls + } +}; + +void analyzeBuiltins(DenseHashMap& result, const DenseHashMap& globals, + const DenseHashMap& variables, const CompileOptions& options, AstNode* root) +{ + BuiltinVisitor visitor{result, globals, variables, options}; + root->visit(&visitor); +} + } // namespace Compile } // namespace Luau diff --git a/Compiler/src/Builtins.h b/Compiler/src/Builtins.h index 60df53a1..4399c532 100644 --- a/Compiler/src/Builtins.h +++ b/Compiler/src/Builtins.h @@ -35,7 +35,9 @@ struct Builtin }; Builtin getBuiltin(AstExpr* node, const DenseHashMap& globals, const DenseHashMap& variables); -int getBuiltinFunctionId(const Builtin& builtin, const CompileOptions& options); + +void analyzeBuiltins(DenseHashMap& result, const DenseHashMap& globals, + const DenseHashMap& variables, const CompileOptions& options, AstNode* root); } // namespace Compile } // namespace Luau diff --git a/Compiler/src/BytecodeBuilder.cpp b/Compiler/src/BytecodeBuilder.cpp index 5e2669b2..64327bd0 100644 --- a/Compiler/src/BytecodeBuilder.cpp +++ b/Compiler/src/BytecodeBuilder.cpp @@ -120,6 +120,17 @@ inline bool isSkipC(LuauOpcode op) switch (op) { case LOP_LOADB: + return true; + + default: + return false; + } +} + +inline bool isFastCall(LuauOpcode op) +{ + switch (op) + { case LOP_FASTCALL: case LOP_FASTCALL1: case LOP_FASTCALL2: @@ -137,6 +148,8 @@ static int getJumpTarget(uint32_t insn, uint32_t pc) if (isJumpD(op)) return int(pc + LUAU_INSN_D(insn) + 1); + else if (isFastCall(op)) + return int(pc + LUAU_INSN_C(insn) + 2); else if (isSkipC(op) && LUAU_INSN_C(insn)) return int(pc + LUAU_INSN_C(insn) + 1); else if (op == LOP_JUMPX) @@ -479,7 +492,7 @@ bool BytecodeBuilder::patchSkipC(size_t jumpLabel, size_t targetLabel) unsigned int jumpInsn = insns[jumpLabel]; (void)jumpInsn; - LUAU_ASSERT(isSkipC(LuauOpcode(LUAU_INSN_OP(jumpInsn)))); + LUAU_ASSERT(isSkipC(LuauOpcode(LUAU_INSN_OP(jumpInsn))) || isFastCall(LuauOpcode(LUAU_INSN_OP(jumpInsn)))); LUAU_ASSERT(LUAU_INSN_C(jumpInsn) == 0); int offset = int(targetLabel) - int(jumpLabel) - 1; diff --git a/Compiler/src/Compiler.cpp b/Compiler/src/Compiler.cpp index d7c8155c..2b0f04f6 100644 --- a/Compiler/src/Compiler.cpp +++ b/Compiler/src/Compiler.cpp @@ -25,6 +25,9 @@ LUAU_FASTINTVARIABLE(LuauCompileInlineDepth, 5) LUAU_FASTFLAGVARIABLE(LuauCompileNoIpairs, false) +LUAU_FASTFLAGVARIABLE(LuauCompileFoldBuiltins, false) +LUAU_FASTFLAGVARIABLE(LuauCompileBetterMultret, false) + namespace Luau { @@ -75,6 +78,12 @@ static BytecodeBuilder::StringRef sref(AstArray data) return {data.data, data.size}; } +static BytecodeBuilder::StringRef sref(AstArray data) +{ + LUAU_ASSERT(data.data); + return {data.data, data.size}; +} + struct Compiler { struct RegScope; @@ -89,6 +98,7 @@ struct Compiler , constants(nullptr) , locstants(nullptr) , tableShapes(nullptr) + , builtins(nullptr) { // preallocate some buffers that are very likely to grow anyway; this works around std::vector's inefficient growth policy for small arrays localStack.reserve(16); @@ -245,7 +255,7 @@ struct Compiler { f.canInline = true; f.stackSize = stackSize; - f.costModel = modelCost(func->body, func->args.data, func->args.size); + f.costModel = modelCost(func->body, func->args.data, func->args.size, builtins); // track functions that only ever return a single value so that we can convert multret calls to fixedret calls if (allPathsEndWithReturn(func->body)) @@ -262,22 +272,63 @@ struct Compiler return fid; } + // returns true if node can return multiple values; may conservatively return true even if expr is known to return just a single value + bool isExprMultRet(AstExpr* node) + { + if (!FFlag::LuauCompileBetterMultret) + return node->is() || node->is(); + + AstExprCall* expr = node->as(); + if (!expr) + return node->is(); + + // conservative version, optimized for compilation throughput + if (options.optimizationLevel <= 1) + return true; + + // handles builtin calls that can be constant-folded + // without this we may omit some optimizations eg compiling fast calls without use of FASTCALL2K + if (isConstant(expr)) + return false; + + // handles local function calls where we know only one argument is returned + AstExprFunction* func = getFunctionExpr(expr->func); + Function* fi = func ? functions.find(func) : nullptr; + + if (fi && fi->returnsOne) + return false; + + // unrecognized call, so we conservatively assume multret + return true; + } + // note: this doesn't just clobber target (assuming it's temp), but also clobbers *all* allocated registers >= target! // this is important to be able to support "multret" semantics due to Lua call frame structure bool compileExprTempMultRet(AstExpr* node, uint8_t target) { if (AstExprCall* expr = node->as()) { - // Optimization: convert multret calls to functions that always return one value to fixedret calls; this facilitates inlining + // Optimization: convert multret calls that always return one value to fixedret calls; this facilitates inlining/constant folding if (options.optimizationLevel >= 2) { - AstExprFunction* func = getFunctionExpr(expr->func); - Function* fi = func ? functions.find(func) : nullptr; - - if (fi && fi->returnsOne) + if (FFlag::LuauCompileBetterMultret) { - compileExprTemp(node, target); - return false; + if (!isExprMultRet(node)) + { + compileExprTemp(node, target); + return false; + } + } + else + { + AstExprFunction* func = getFunctionExpr(expr->func); + Function* fi = func ? functions.find(func) : nullptr; + + if (fi && fi->returnsOne) + { + compileExprTemp(node, target); + return false; + } } } @@ -483,8 +534,7 @@ struct Compiler varc[i] = isConstant(expr->args.data[i]); // if the last argument only returns a single value, all following arguments are nil - if (expr->args.size != 0 && - !(expr->args.data[expr->args.size - 1]->is() || expr->args.data[expr->args.size - 1]->is())) + if (expr->args.size != 0 && !isExprMultRet(expr->args.data[expr->args.size - 1])) for (size_t i = expr->args.size; i < func->args.size && i < 8; ++i) varc[i] = true; @@ -523,7 +573,7 @@ struct Compiler AstLocal* var = func->args.data[i]; AstExpr* arg = i < expr->args.size ? expr->args.data[i] : nullptr; - if (i + 1 == expr->args.size && func->args.size > expr->args.size && (arg->is() || arg->is())) + if (i + 1 == expr->args.size && func->args.size > expr->args.size && isExprMultRet(arg)) { // if the last argument can return multiple values, we need to compute all of them into the remaining arguments unsigned int tail = unsigned(func->args.size - expr->args.size) + 1; @@ -591,7 +641,7 @@ struct Compiler } // fold constant values updated above into expressions in the function body - foldConstants(constants, variables, locstants, func->body); + foldConstants(constants, variables, locstants, builtinsFold, func->body); bool usedFallthrough = false; @@ -632,7 +682,7 @@ struct Compiler if (Constant* var = locstants.find(func->args.data[i])) var->type = Constant::Type_Unknown; - foldConstants(constants, variables, locstants, func->body); + foldConstants(constants, variables, locstants, builtinsFold, func->body); } void compileExprCall(AstExprCall* expr, uint8_t target, uint8_t targetCount, bool targetTop = false, bool multRet = false) @@ -675,29 +725,23 @@ struct Compiler int bfid = -1; - if (options.optimizationLevel >= 1) - { - Builtin builtin = getBuiltin(expr->func, globals, variables); - bfid = getBuiltinFunctionId(builtin, options); - } + if (options.optimizationLevel >= 1 && !expr->self) + if (const int* id = builtins.find(expr)) + bfid = *id; if (bfid == LBF_SELECT_VARARG) { // Optimization: compile select(_, ...) as FASTCALL1; the builtin will read variadic arguments directly // note: for now we restrict this to single-return expressions since our runtime code doesn't deal with general cases - if (multRet == false && targetCount == 1 && expr->args.size == 2 && expr->args.data[1]->is()) + if (multRet == false && targetCount == 1) return compileExprSelectVararg(expr, target, targetCount, targetTop, multRet, regs); else bfid = -1; } // Optimization: for 1/2 argument fast calls use specialized opcodes - if (!expr->self && bfid >= 0 && expr->args.size >= 1 && expr->args.size <= 2) - { - AstExpr* last = expr->args.data[expr->args.size - 1]; - if (!last->is() && !last->is()) - return compileExprFastcallN(expr, target, targetCount, targetTop, multRet, regs, bfid); - } + if (bfid >= 0 && expr->args.size >= 1 && expr->args.size <= 2 && !isExprMultRet(expr->args.data[expr->args.size - 1])) + return compileExprFastcallN(expr, target, targetCount, targetTop, multRet, regs, bfid); if (expr->self) { @@ -2495,7 +2539,7 @@ struct Compiler } AstLocal* var = stat->var; - uint64_t costModel = modelCost(stat->body, &var, 1); + uint64_t costModel = modelCost(stat->body, &var, 1, builtins); // we use a dynamic cost threshold that's based on the fixed limit boosted by the cost advantage we gain due to unrolling bool varc = true; @@ -2533,7 +2577,7 @@ struct Compiler locstants[var].type = Constant::Type_Number; locstants[var].valueNumber = from + iv * step; - foldConstants(constants, variables, locstants, stat); + foldConstants(constants, variables, locstants, builtinsFold, stat); size_t iterJumps = loopJumps.size(); @@ -2561,7 +2605,7 @@ struct Compiler // clean up fold state in case we need to recompile - normally we compile the loop body once, but due to inlining we may need to do it again locstants[var].type = Constant::Type_Unknown; - foldConstants(constants, variables, locstants, stat); + foldConstants(constants, variables, locstants, builtinsFold, stat); } void compileStatFor(AstStatFor* stat) @@ -3368,7 +3412,11 @@ struct Compiler bool visit(AstStatReturn* stat) override { - if (stat->list.size == 1) + if (FFlag::LuauCompileBetterMultret) + { + returnsOne &= stat->list.size == 1 && !self->isExprMultRet(stat->list.data[0]); + } + else if (stat->list.size == 1) { AstExpr* value = stat->list.data[0]; @@ -3487,6 +3535,8 @@ struct Compiler DenseHashMap constants; DenseHashMap locstants; DenseHashMap tableShapes; + DenseHashMap builtins; + const DenseHashMap* builtinsFold = nullptr; unsigned int regTop = 0; unsigned int stackSize = 0; @@ -3502,10 +3552,21 @@ struct Compiler std::vector captures; }; -void compileOrThrow(BytecodeBuilder& bytecode, AstStatBlock* root, const AstNameTable& names, const CompileOptions& options) +void compileOrThrow(BytecodeBuilder& bytecode, const ParseResult& parseResult, const AstNameTable& names, const CompileOptions& inputOptions) { LUAU_TIMETRACE_SCOPE("compileOrThrow", "Compiler"); + LUAU_ASSERT(parseResult.root); + LUAU_ASSERT(parseResult.errors.empty()); + + CompileOptions options = inputOptions; + + for (const HotComment& hc : parseResult.hotcomments) + if (hc.header && hc.content.compare(0, 9, "optimize ") == 0) + options.optimizationLevel = std::max(0, std::min(2, atoi(hc.content.c_str() + 9))); + + AstStatBlock* root = parseResult.root; + Compiler compiler(bytecode, options); // since access to some global objects may result in values that change over time, we block imports from non-readonly tables @@ -3514,10 +3575,17 @@ void compileOrThrow(BytecodeBuilder& bytecode, AstStatBlock* root, const AstName // this pass analyzes mutability of locals/globals and associates locals with their initial values trackValues(compiler.globals, compiler.variables, root); + // builtin folding is enabled on optimization level 2 since we can't deoptimize folding at runtime + if (options.optimizationLevel >= 2 && FFlag::LuauCompileFoldBuiltins) + compiler.builtinsFold = &compiler.builtins; + if (options.optimizationLevel >= 1) { + // this pass tracks which calls are builtins and can be compiled more efficiently + analyzeBuiltins(compiler.builtins, compiler.globals, compiler.variables, options, root); + // this pass analyzes constantness of expressions - foldConstants(compiler.constants, compiler.variables, compiler.locstants, root); + foldConstants(compiler.constants, compiler.variables, compiler.locstants, compiler.builtinsFold, root); // this pass analyzes table assignments to estimate table shapes for initially empty tables predictTableShapes(compiler.tableShapes, root); @@ -3559,9 +3627,7 @@ void compileOrThrow(BytecodeBuilder& bytecode, const std::string& source, const if (!result.errors.empty()) throw ParseErrors(result.errors); - AstStatBlock* root = result.root; - - compileOrThrow(bytecode, root, names, options); + compileOrThrow(bytecode, result, names, options); } std::string compile(const std::string& source, const CompileOptions& options, const ParseOptions& parseOptions, BytecodeEncoder* encoder) @@ -3584,7 +3650,7 @@ std::string compile(const std::string& source, const CompileOptions& options, co try { BytecodeBuilder bcb(encoder); - compileOrThrow(bcb, result.root, names, options); + compileOrThrow(bcb, result, names, options); return bcb.getBytecode(); } diff --git a/Compiler/src/ConstantFolding.cpp b/Compiler/src/ConstantFolding.cpp index a62beeb1..34f79544 100644 --- a/Compiler/src/ConstantFolding.cpp +++ b/Compiler/src/ConstantFolding.cpp @@ -1,6 +1,8 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #include "ConstantFolding.h" +#include "BuiltinFolding.h" + #include namespace Luau @@ -193,13 +195,18 @@ struct ConstantVisitor : AstVisitor DenseHashMap& variables; DenseHashMap& locals; + const DenseHashMap* builtins; + bool wasEmpty = false; - ConstantVisitor( - DenseHashMap& constants, DenseHashMap& variables, DenseHashMap& locals) + std::vector builtinArgs; + + ConstantVisitor(DenseHashMap& constants, DenseHashMap& variables, + DenseHashMap& locals, const DenseHashMap* builtins) : constants(constants) , variables(variables) , locals(locals) + , builtins(builtins) { // since we do a single pass over the tree, if the initial state was empty we don't need to clear out old entries wasEmpty = constants.empty() && locals.empty(); @@ -253,8 +260,37 @@ struct ConstantVisitor : AstVisitor { analyze(expr->func); - for (size_t i = 0; i < expr->args.size; ++i) - analyze(expr->args.data[i]); + if (const int* bfid = builtins ? builtins->find(expr) : nullptr) + { + // since recursive calls to analyze() may reuse the vector we need to be careful and preserve existing contents + size_t offset = builtinArgs.size(); + bool canFold = true; + + builtinArgs.reserve(offset + expr->args.size); + + for (size_t i = 0; i < expr->args.size; ++i) + { + Constant ac = analyze(expr->args.data[i]); + + if (ac.type == Constant::Type_Unknown) + canFold = false; + else + builtinArgs.push_back(ac); + } + + if (canFold) + { + LUAU_ASSERT(builtinArgs.size() == offset + expr->args.size); + result = foldBuiltin(*bfid, builtinArgs.data() + offset, expr->args.size); + } + + builtinArgs.resize(offset); + } + else + { + for (size_t i = 0; i < expr->args.size; ++i) + analyze(expr->args.data[i]); + } } else if (AstExprIndexName* expr = node->as()) { @@ -395,9 +431,9 @@ struct ConstantVisitor : AstVisitor }; void foldConstants(DenseHashMap& constants, DenseHashMap& variables, - DenseHashMap& locals, AstNode* root) + DenseHashMap& locals, const DenseHashMap* builtins, AstNode* root) { - ConstantVisitor visitor{constants, variables, locals}; + ConstantVisitor visitor{constants, variables, locals, builtins}; root->visit(&visitor); } diff --git a/Compiler/src/ConstantFolding.h b/Compiler/src/ConstantFolding.h index 0a995d75..d67d9285 100644 --- a/Compiler/src/ConstantFolding.h +++ b/Compiler/src/ConstantFolding.h @@ -26,7 +26,7 @@ struct Constant { bool valueBoolean; double valueNumber; - char* valueString = nullptr; // length stored in stringLength + const char* valueString = nullptr; // length stored in stringLength }; bool isTruthful() const @@ -35,7 +35,7 @@ struct Constant return type != Type_Nil && !(type == Type_Boolean && valueBoolean == false); } - AstArray getString() const + AstArray getString() const { LUAU_ASSERT(type == Type_String); return {valueString, stringLength}; @@ -43,7 +43,7 @@ struct Constant }; void foldConstants(DenseHashMap& constants, DenseHashMap& variables, - DenseHashMap& locals, AstNode* root); + DenseHashMap& locals, const DenseHashMap* builtins, AstNode* root); } // namespace Compile } // namespace Luau diff --git a/Compiler/src/CostModel.cpp b/Compiler/src/CostModel.cpp index 5608cd86..56b6c36f 100644 --- a/Compiler/src/CostModel.cpp +++ b/Compiler/src/CostModel.cpp @@ -6,6 +6,8 @@ #include +LUAU_FASTFLAGVARIABLE(LuauCompileModelBuiltins, false) + namespace Luau { namespace Compile @@ -113,11 +115,14 @@ struct Cost struct CostVisitor : AstVisitor { + const DenseHashMap& builtins; + DenseHashMap vars; Cost result; - CostVisitor() - : vars(nullptr) + CostVisitor(const DenseHashMap& builtins) + : builtins(builtins) + , vars(nullptr) { } @@ -148,14 +153,21 @@ struct CostVisitor : AstVisitor } else if (AstExprCall* expr = node->as()) { - Cost cost = 3; - cost += model(expr->func); + // builtin cost modeling is different from regular calls because we use FASTCALL to compile these + // thus we use a cheaper baseline, don't account for function, and assume constant/local copy is free + bool builtin = FFlag::LuauCompileModelBuiltins && builtins.find(expr) != nullptr; + bool builtinShort = builtin && expr->args.size <= 2; // FASTCALL1/2 + + Cost cost = builtin ? 2 : 3; + + if (!builtin) + cost += model(expr->func); for (size_t i = 0; i < expr->args.size; ++i) { Cost ac = model(expr->args.data[i]); // for constants/locals we still need to copy them to the argument list - cost += ac.model == 0 ? Cost(1) : ac; + cost += ac.model == 0 && !builtinShort ? Cost(1) : ac; } return cost; @@ -327,9 +339,9 @@ struct CostVisitor : AstVisitor } }; -uint64_t modelCost(AstNode* root, AstLocal* const* vars, size_t varCount) +uint64_t modelCost(AstNode* root, AstLocal* const* vars, size_t varCount, const DenseHashMap& builtins) { - CostVisitor visitor; + CostVisitor visitor{builtins}; for (size_t i = 0; i < varCount && i < 7; ++i) visitor.vars[vars[i]] = 0xffull << (i * 8 + 8); diff --git a/Compiler/src/CostModel.h b/Compiler/src/CostModel.h index 17defafb..e8f3e166 100644 --- a/Compiler/src/CostModel.h +++ b/Compiler/src/CostModel.h @@ -2,6 +2,7 @@ #pragma once #include "Luau/Ast.h" +#include "Luau/DenseHash.h" namespace Luau { @@ -9,7 +10,7 @@ namespace Compile { // cost model: 8 bytes, where first byte is the baseline cost, and the next 7 bytes are discounts for when variable #i is constant -uint64_t modelCost(AstNode* root, AstLocal* const* vars, size_t varCount); +uint64_t modelCost(AstNode* root, AstLocal* const* vars, size_t varCount, const DenseHashMap& builtins); // cost is computed as B - sum(Di * Ci), where B is baseline cost, Di is the discount for each variable and Ci is 1 when variable #i is constant int computeCost(uint64_t model, const bool* varsConst, size_t varCount); diff --git a/Makefile b/Makefile index b8077897..a471e7ae 100644 --- a/Makefile +++ b/Makefile @@ -50,6 +50,9 @@ TESTS_ARGS= ifneq ($(flags),) TESTS_ARGS+=--fflags=$(flags) endif +ifneq ($(opt),) + TESTS_ARGS+=-O$(opt) +endif OBJECTS=$(AST_OBJECTS) $(COMPILER_OBJECTS) $(ANALYSIS_OBJECTS) $(CODEGEN_OBJECTS) $(VM_OBJECTS) $(ISOCLINE_OBJECTS) $(TESTS_OBJECTS) $(CLI_OBJECTS) $(FUZZ_OBJECTS) @@ -104,7 +107,7 @@ $(ANALYSIS_OBJECTS): CXXFLAGS+=-std=c++17 -ICommon/include -IAst/include -IAnaly $(CODEGEN_OBJECTS): CXXFLAGS+=-std=c++17 -ICommon/include -ICodeGen/include $(VM_OBJECTS): CXXFLAGS+=-std=c++11 -ICommon/include -IVM/include $(ISOCLINE_OBJECTS): CXXFLAGS+=-Wno-unused-function -Iextern/isocline/include -$(TESTS_OBJECTS): CXXFLAGS+=-std=c++17 -ICommon/include -IAst/include -ICompiler/include -IAnalysis/include -ICodeGen/include -IVM/include -ICLI -Iextern +$(TESTS_OBJECTS): CXXFLAGS+=-std=c++17 -ICommon/include -IAst/include -ICompiler/include -IAnalysis/include -ICodeGen/include -IVM/include -ICLI -Iextern -DDOCTEST_CONFIG_DOUBLE_STRINGIFY $(REPL_CLI_OBJECTS): CXXFLAGS+=-std=c++17 -ICommon/include -IAst/include -ICompiler/include -IVM/include -Iextern -Iextern/isocline/include $(ANALYZE_CLI_OBJECTS): CXXFLAGS+=-std=c++17 -ICommon/include -IAst/include -IAnalysis/include -Iextern $(FUZZ_OBJECTS): CXXFLAGS+=-std=c++17 -ICommon/include -IAst/include -ICompiler/include -IAnalysis/include -IVM/include diff --git a/Sources.cmake b/Sources.cmake index 44bed8f7..69f5e1b7 100644 --- a/Sources.cmake +++ b/Sources.cmake @@ -38,12 +38,14 @@ target_sources(Luau.Compiler PRIVATE Compiler/src/BytecodeBuilder.cpp Compiler/src/Compiler.cpp Compiler/src/Builtins.cpp + Compiler/src/BuiltinFolding.cpp Compiler/src/ConstantFolding.cpp Compiler/src/CostModel.cpp Compiler/src/TableShape.cpp Compiler/src/ValueTracking.cpp Compiler/src/lcode.cpp Compiler/src/Builtins.h + Compiler/src/BuiltinFolding.h Compiler/src/ConstantFolding.h Compiler/src/CostModel.h Compiler/src/TableShape.h diff --git a/VM/include/lua.h b/VM/include/lua.h index 72edb4a6..187dd5c2 100644 --- a/VM/include/lua.h +++ b/VM/include/lua.h @@ -373,7 +373,7 @@ LUA_API const char* lua_getupvalue(lua_State* L, int funcindex, int n); LUA_API const char* lua_setupvalue(lua_State* L, int funcindex, int n); LUA_API void lua_singlestep(lua_State* L, int enabled); -LUA_API void lua_breakpoint(lua_State* L, int funcindex, int line, int enabled); +LUA_API int lua_breakpoint(lua_State* L, int funcindex, int line, int enabled); typedef void (*lua_Coverage)(void* context, const char* function, int linedefined, int depth, const int* hits, size_t size); diff --git a/VM/src/lapi.cpp b/VM/src/lapi.cpp index c2b790ad..88680727 100644 --- a/VM/src/lapi.cpp +++ b/VM/src/lapi.cpp @@ -34,6 +34,8 @@ * therefore call luaC_checkGC before luaC_checkthreadsleep to guarantee the object is pushed to an awake thread. */ +LUAU_FASTFLAG(LuauLazyAtoms) + const char* lua_ident = "$Lua: Lua 5.1.4 Copyright (C) 1994-2008 Lua.org, PUC-Rio $\n" "$Authors: R. Ierusalimschy, L. H. de Figueiredo & W. Celes $\n" "$URL: www.lua.org $\n"; @@ -51,6 +53,13 @@ const char* luau_ident = "$Luau: Copyright (C) 2019-2022 Roblox Corporation $\n" L->top++; \ } +#define updateatom(L, ts) \ + if (FFlag::LuauLazyAtoms) \ + { \ + if (ts->atom == ATOM_UNDEF) \ + ts->atom = L->global->cb.useratom ? L->global->cb.useratom(ts->data, ts->len) : -1; \ + } + static Table* getcurrenv(lua_State* L) { if (L->ci == L->base_ci) /* no enclosing function? */ @@ -441,19 +450,25 @@ const char* lua_tostringatom(lua_State* L, int idx, int* atom) StkId o = index2addr(L, idx); if (!ttisstring(o)) return NULL; - const TString* s = tsvalue(o); + TString* s = tsvalue(o); if (atom) + { + updateatom(L, s); *atom = s->atom; + } return getstr(s); } const char* lua_namecallatom(lua_State* L, int* atom) { - const TString* s = L->namecall; + TString* s = L->namecall; if (!s) return NULL; if (atom) + { + updateatom(L, s); *atom = s->atom; + } return getstr(s); } diff --git a/VM/src/ldebug.cpp b/VM/src/ldebug.cpp index e050050e..ef486090 100644 --- a/VM/src/ldebug.cpp +++ b/VM/src/ldebug.cpp @@ -12,6 +12,8 @@ #include #include +LUAU_FASTFLAGVARIABLE(LuauDebuggerBreakpointHitOnNextBestLine, false); + static const char* getfuncname(Closure* f); static int currentpc(lua_State* L, CallInfo* ci) @@ -367,14 +369,6 @@ void lua_singlestep(lua_State* L, int enabled) L->singlestep = bool(enabled); } -void lua_breakpoint(lua_State* L, int funcindex, int line, int enabled) -{ - const TValue* func = luaA_toobject(L, funcindex); - api_check(L, ttisfunction(func) && !clvalue(func)->isC); - - luaG_breakpoint(L, clvalue(func)->l.p, line, bool(enabled)); -} - static int getmaxline(Proto* p) { int result = -1; @@ -394,6 +388,71 @@ static int getmaxline(Proto* p) return result; } +// Find the line number with instructions. If the provided line doesn't have any instruction, it should return the next line number with +// instructions. +static int getnextline(Proto* p, int line) +{ + int closest = -1; + if (p->lineinfo) + { + for (int i = 0; i < p->sizecode; ++i) + { + // note: we keep prologue as is, instead opting to break at the first meaningful instruction + if (LUAU_INSN_OP(p->code[i]) == LOP_PREPVARARGS) + continue; + + int current = luaG_getline(p, i); + if (current >= line) + { + closest = current; + break; + } + } + } + + for (int i = 0; i < p->sizep; ++i) + { + // Find the closest line number to the intended one. + int candidate = getnextline(p->p[i], line); + if (closest == -1 || (candidate >= line && candidate < closest)) + { + closest = candidate; + } + } + + return closest; +} + +int lua_breakpoint(lua_State* L, int funcindex, int line, int enabled) +{ + int target = -1; + + if (FFlag::LuauDebuggerBreakpointHitOnNextBestLine) + { + const TValue* func = luaA_toobject(L, funcindex); + api_check(L, ttisfunction(func) && !clvalue(func)->isC); + + Proto* p = clvalue(func)->l.p; + // Find line number to add the breakpoint to. + target = getnextline(p, line); + + if (target != -1) + { + // Add breakpoint on the exact line + luaG_breakpoint(L, p, target, bool(enabled)); + } + } + else + { + const TValue* func = luaA_toobject(L, funcindex); + api_check(L, ttisfunction(func) && !clvalue(func)->isC); + + luaG_breakpoint(L, clvalue(func)->l.p, line, bool(enabled)); + } + + return target; +} + static void getcoverage(Proto* p, int depth, int* buffer, size_t size, void* context, lua_Coverage callback) { memset(buffer, -1, size * sizeof(int)); diff --git a/VM/src/lstring.cpp b/VM/src/lstring.cpp index c0cd3e26..12bd67d5 100644 --- a/VM/src/lstring.cpp +++ b/VM/src/lstring.cpp @@ -7,6 +7,8 @@ #include +LUAU_FASTFLAGVARIABLE(LuauLazyAtoms, false) + unsigned int luaS_hash(const char* str, size_t len) { // Note that this hashing algorithm is replicated in BytecodeBuilder.cpp, BytecodeBuilder::getStringHash @@ -82,7 +84,7 @@ static TString* newlstr(lua_State* L, const char* str, size_t l, unsigned int h) ts->memcat = L->activememcat; memcpy(ts->data, str, l); ts->data[l] = '\0'; /* ending 0 */ - ts->atom = L->global->cb.useratom ? L->global->cb.useratom(ts->data, l) : -1; + ts->atom = FFlag::LuauLazyAtoms ? ATOM_UNDEF : L->global->cb.useratom ? L->global->cb.useratom(ts->data, l) : -1; tb = &L->global->strt; h = lmod(h, tb->size); ts->next = tb->hash[h]; /* chain new entry */ @@ -165,7 +167,7 @@ TString* luaS_buffinish(lua_State* L, TString* ts) ts->data[ts->len] = '\0'; // ending 0 // Complete string object - ts->atom = L->global->cb.useratom ? L->global->cb.useratom(ts->data, ts->len) : -1; + ts->atom = FFlag::LuauLazyAtoms ? ATOM_UNDEF : L->global->cb.useratom ? L->global->cb.useratom(ts->data, ts->len) : -1; ts->next = tb->hash[bucket]; // chain new entry tb->hash[bucket] = ts; diff --git a/VM/src/lstring.h b/VM/src/lstring.h index 290b64d8..acdf9a1e 100644 --- a/VM/src/lstring.h +++ b/VM/src/lstring.h @@ -8,6 +8,9 @@ /* string size limit */ #define MAXSSIZE (1 << 30) +/* string atoms are not defined by default; the storage is 16-bit integer */ +#define ATOM_UNDEF -32768 + #define sizestring(len) (offsetof(TString, data) + len + 1) #define luaS_new(L, s) (luaS_newlstr(L, s, strlen(s))) diff --git a/bench/gc/test_SunSpider_crypto-aes.lua b/bench/gc/test_SunSpider_crypto-aes.lua deleted file mode 100644 index 8537e3da..00000000 --- a/bench/gc/test_SunSpider_crypto-aes.lua +++ /dev/null @@ -1,436 +0,0 @@ ---[[ - * AES Cipher function: encrypt 'input' with Rijndael algorithm - * - * takes byte-array 'input' (16 bytes) - * 2D byte-array key schedule 'w' (Nr+1 x Nb bytes) - * - * applies Nr rounds (10/12/14) using key schedule w for 'add round key' stage - * - * returns byte-array encrypted value (16 bytes) - */]] - - local bench = script and require(script.Parent.bench_support) or require("bench_support") - -function test() - --- Sbox is pre-computed multiplicative inverse in GF(2^8) used in SubBytes and KeyExpansion [§5.1.1] -local Sbox = { 0x63,0x7c,0x77,0x7b,0xf2,0x6b,0x6f,0xc5,0x30,0x01,0x67,0x2b,0xfe,0xd7,0xab,0x76, - 0xca,0x82,0xc9,0x7d,0xfa,0x59,0x47,0xf0,0xad,0xd4,0xa2,0xaf,0x9c,0xa4,0x72,0xc0, - 0xb7,0xfd,0x93,0x26,0x36,0x3f,0xf7,0xcc,0x34,0xa5,0xe5,0xf1,0x71,0xd8,0x31,0x15, - 0x04,0xc7,0x23,0xc3,0x18,0x96,0x05,0x9a,0x07,0x12,0x80,0xe2,0xeb,0x27,0xb2,0x75, - 0x09,0x83,0x2c,0x1a,0x1b,0x6e,0x5a,0xa0,0x52,0x3b,0xd6,0xb3,0x29,0xe3,0x2f,0x84, - 0x53,0xd1,0x00,0xed,0x20,0xfc,0xb1,0x5b,0x6a,0xcb,0xbe,0x39,0x4a,0x4c,0x58,0xcf, - 0xd0,0xef,0xaa,0xfb,0x43,0x4d,0x33,0x85,0x45,0xf9,0x02,0x7f,0x50,0x3c,0x9f,0xa8, - 0x51,0xa3,0x40,0x8f,0x92,0x9d,0x38,0xf5,0xbc,0xb6,0xda,0x21,0x10,0xff,0xf3,0xd2, - 0xcd,0x0c,0x13,0xec,0x5f,0x97,0x44,0x17,0xc4,0xa7,0x7e,0x3d,0x64,0x5d,0x19,0x73, - 0x60,0x81,0x4f,0xdc,0x22,0x2a,0x90,0x88,0x46,0xee,0xb8,0x14,0xde,0x5e,0x0b,0xdb, - 0xe0,0x32,0x3a,0x0a,0x49,0x06,0x24,0x5c,0xc2,0xd3,0xac,0x62,0x91,0x95,0xe4,0x79, - 0xe7,0xc8,0x37,0x6d,0x8d,0xd5,0x4e,0xa9,0x6c,0x56,0xf4,0xea,0x65,0x7a,0xae,0x08, - 0xba,0x78,0x25,0x2e,0x1c,0xa6,0xb4,0xc6,0xe8,0xdd,0x74,0x1f,0x4b,0xbd,0x8b,0x8a, - 0x70,0x3e,0xb5,0x66,0x48,0x03,0xf6,0x0e,0x61,0x35,0x57,0xb9,0x86,0xc1,0x1d,0x9e, - 0xe1,0xf8,0x98,0x11,0x69,0xd9,0x8e,0x94,0x9b,0x1e,0x87,0xe9,0xce,0x55,0x28,0xdf, - 0x8c,0xa1,0x89,0x0d,0xbf,0xe6,0x42,0x68,0x41,0x99,0x2d,0x0f,0xb0,0x54,0xbb,0x16 }; - --- Rcon is Round Constant used for the Key Expansion [1st col is 2^(r-1) in GF(2^8)] [§5.2] -local Rcon = { { 0x00, 0x00, 0x00, 0x00 }, - {0x01, 0x00, 0x00, 0x00}, - {0x02, 0x00, 0x00, 0x00}, - {0x04, 0x00, 0x00, 0x00}, - {0x08, 0x00, 0x00, 0x00}, - {0x10, 0x00, 0x00, 0x00}, - {0x20, 0x00, 0x00, 0x00}, - {0x40, 0x00, 0x00, 0x00}, - {0x80, 0x00, 0x00, 0x00}, - {0x1b, 0x00, 0x00, 0x00}, - {0x36, 0x00, 0x00, 0x00} }; - -function Cipher(input, w) -- main Cipher function [§5.1] - local Nb = 4; -- block size (in words): no of columns in state (fixed at 4 for AES) - local Nr = #w / Nb - 1; -- no of rounds: 10/12/14 for 128/192/256-bit keys - - local state = {{},{},{},{}}; -- initialise 4xNb byte-array 'state' with input [§3.4] - for i = 0,4*Nb-1 do state[(i % 4) + 1][math.floor(i/4) + 1] = input[i + 1]; end - - state = AddRoundKey(state, w, 0, Nb); - - for round = 1,Nr-1 do - state = SubBytes(state, Nb); - state = ShiftRows(state, Nb); - state = MixColumns(state, Nb); - state = AddRoundKey(state, w, round, Nb); - end - - state = SubBytes(state, Nb); - state = ShiftRows(state, Nb); - state = AddRoundKey(state, w, Nr, Nb); - - local output = {} -- convert state to 1-d array before returning [§3.4] - for i = 0,4*Nb-1 do output[i + 1] = state[(i % 4) + 1][math.floor(i / 4) + 1]; end - - return output; -end - - -function SubBytes(s, Nb) -- apply SBox to state S [§5.1.1] - for r = 0,3 do - for c = 0,Nb-1 do s[r + 1][c + 1] = Sbox[s[r + 1][c + 1] + 1]; end - end - return s; -end - - -function ShiftRows(s, Nb) -- shift row r of state S left by r bytes [§5.1.2] - local t = {}; - for r = 1,3 do - for c = 0,3 do t[c + 1] = s[r + 1][((c + r) % Nb) + 1] end; -- shift into temp copy - for c = 0,3 do s[r + 1][c + 1] = t[c + 1]; end -- and copy back - end -- note that this will work for Nb=4,5,6, but not 7,8 (always 4 for AES): - return s; -- see fp.gladman.plus.com/cryptography_technology/rijndael/aes.spec.311.pdf -end - - -function MixColumns(s, Nb) -- combine bytes of each col of state S [§5.1.3] - for c = 0,3 do - local a = {}; -- 'a' is a copy of the current column from 's' - local b = {}; -- 'b' is a•{02} in GF(2^8) - for i = 0,3 do - a[i + 1] = s[i + 1][c + 1]; - - if bit32.band(s[i + 1][c + 1], 0x80) ~= 0 then - b[i + 1] = bit32.bxor(bit32.lshift(s[i + 1][c + 1], 1), 0x011b); - else - b[i + 1] = bit32.lshift(s[i + 1][c + 1], 1); - end - end - -- a[n] ^ b[n] is a•{03} in GF(2^8) - s[1][c + 1] = bit32.bxor(bit32.bxor(bit32.bxor(b[1], a[2]), bit32.bxor(b[2], a[3])), a[4]); -- 2*a0 + 3*a1 + a2 + a3 - s[2][c + 1] = bit32.bxor(bit32.bxor(bit32.bxor(a[1], b[2]), bit32.bxor(a[3], b[3])), a[4]); -- a0 * 2*a1 + 3*a2 + a3 - s[3][c + 1] = bit32.bxor(bit32.bxor(bit32.bxor(a[1], a[2]), bit32.bxor(b[3], a[4])), b[4]); -- a0 + a1 + 2*a2 + 3*a3 - s[4][c + 1] = bit32.bxor(bit32.bxor(bit32.bxor(a[1], b[1]), bit32.bxor(a[2], a[3])), b[4]); -- 3*a0 + a1 + a2 + 2*a3 -end - return s; -end - - -function AddRoundKey(state, w, rnd, Nb) -- xor Round Key into state S [§5.1.4] - for r = 0,3 do - for c = 0,Nb-1 do state[r + 1][c + 1] = bit32.bxor(state[r + 1][c + 1], w[rnd*4+c + 1][r + 1]); end - end - return state; -end - - -function KeyExpansion(key) -- generate Key Schedule (byte-array Nr+1 x Nb) from Key [§5.2] - local Nb = 4; -- block size (in words): no of columns in state (fixed at 4 for AES) - local Nk = #key / 4 -- key length (in words): 4/6/8 for 128/192/256-bit keys - local Nr = Nk + 6; -- no of rounds: 10/12/14 for 128/192/256-bit keys - - local w = {}; - local temp = {}; - - for i = 0,Nk do - local r = { key[4*i + 1], key[4*i + 2], key[4*i + 3], key[4*i + 4] }; - w[i + 1] = r; - end - - for i = Nk,(Nb*(Nr+1)) - 1 do - w[i + 1] = {}; - for t = 0,3 do temp[t + 1] = w[i-1 + 1][t + 1]; end - if (i % Nk == 0) then - temp = SubWord(RotWord(temp)); - for t = 0,3 do temp[t + 1] = bit32.bxor(temp[t + 1], Rcon[i/Nk + 1][t + 1]); end - elseif (Nk > 6 and i % Nk == 4) then - temp = SubWord(temp); - end - for t = 0,3 do w[i + 1][t + 1] = bit32.bxor(w[i - Nk + 1][t + 1], temp[t + 1]); end - end - - return w; -end - -function SubWord(w) -- apply SBox to 4-byte word w - for i = 0,3 do w[i + 1] = Sbox[w[i + 1] + 1]; end - return w; -end - -function RotWord(w) -- rotate 4-byte word w left by one byte - w[5] = w[1]; - for i = 0,3 do w[i + 1] = w[i + 2]; end - return w; -end - - ---[[ - * Use AES to encrypt 'plaintext' with 'password' using 'nBits' key, in 'Counter' mode of operation - * - see http://csrc.nist.gov/publications/nistpubs/800-38a/sp800-38a.pdf - * for each block - * - outputblock = cipher(counter, key) - * - cipherblock = plaintext xor outputblock - ]] - -function AESEncryptCtr(plaintext, password, nBits) - if (not (nBits==128 or nBits==192 or nBits==256)) then return ''; end -- standard allows 128/192/256 bit keys - - -- for this example script, generate the key by applying Cipher to 1st 16/24/32 chars of password; - -- for real-world applications, a higher security approach would be to hash the password e.g. with SHA-1 - local nBytes = nBits/8; -- no bytes in key - local pwBytes = {}; - for i = 0,nBytes-1 do pwBytes[i + 1] = bit32.band(string.byte(password, i + 1), 0xff); end - local key = Cipher(pwBytes, KeyExpansion(pwBytes)); - - -- key is now 16/24/32 bytes long - for i = 1,nBytes-16 do - table.insert(key, key[i]) - end - - -- initialise counter block (NIST SP800-38A §B.2): millisecond time-stamp for nonce in 1st 8 bytes, - -- block counter in 2nd 8 bytes - local blockSize = 16; -- block size fixed at 16 bytes / 128 bits (Nb=4) for AES - local counterBlock = {}; -- block size fixed at 16 bytes / 128 bits (Nb=4) for AES - local nonce = 12564231564 -- (new Date()).getTime(); -- milliseconds since 1-Jan-1970 - - -- encode nonce in two stages to cater for JavaScript 32-bit limit on bitwise ops - for i = 0,3 do counterBlock[i + 1] = bit32.band(bit32.rshift(nonce, i * 8), 0xff); end - for i = 0,3 do counterBlock[i + 4 + 1] = bit32.band(bit32.rshift(math.floor(nonce / 0x100000000), i*8), 0xff); end - - -- generate key schedule - an expansion of the key into distinct Key Rounds for each round - local keySchedule = KeyExpansion(key); - - local blockCount = math.ceil(#plaintext / blockSize); - local ciphertext = {}; -- ciphertext as array of strings - - for b = 0,blockCount-1 do - -- set counter (block #) in last 8 bytes of counter block (leaving nonce in 1st 8 bytes) - -- again done in two stages for 32-bit ops - for c = 0,3 do counterBlock[15-c + 1] = bit32.band(bit32.rshift(b, c*8), 0xff); end - for c = 0,3 do counterBlock[15-c-4 + 1] = bit32.rshift(math.floor(b/0x100000000), c*8) end - - local cipherCntr = Cipher(counterBlock, keySchedule); -- -- encrypt counter block -- - - -- calculate length of final block: - local blockLength = nil - - if b #define DOCTEST_BREAK_INTO_DEBUGGER() raise(SIGTRAP) #endif #elif defined(DOCTEST_PLATFORM_MAC) #if defined(__x86_64) || defined(__x86_64__) || defined(__amd64__) || defined(__i386) -#define DOCTEST_BREAK_INTO_DEBUGGER() __asm__("int $3\n" : :) // NOLINT (hicpp-no-assembler) +#define DOCTEST_BREAK_INTO_DEBUGGER() __asm__("int $3\n" : :) // NOLINT(hicpp-no-assembler) +#elif defined(__ppc__) || defined(__ppc64__) +// https://www.cocoawithlove.com/2008/03/break-into-debugger.html +#define DOCTEST_BREAK_INTO_DEBUGGER() __asm__("li r0, 20\nsc\nnop\nli r0, 37\nli r4, 2\nsc\nnop\n": : : "memory","r0","r3","r4") // NOLINT(hicpp-no-assembler) #else -#define DOCTEST_BREAK_INTO_DEBUGGER() __asm__("brk #0"); // NOLINT (hicpp-no-assembler) +#define DOCTEST_BREAK_INTO_DEBUGGER() __asm__("brk #0"); // NOLINT(hicpp-no-assembler) #endif #elif DOCTEST_MSVC #define DOCTEST_BREAK_INTO_DEBUGGER() __debugbreak() @@ -387,54 +463,66 @@ DOCTEST_GCC_SUPPRESS_WARNING_POP // this is kept here for backwards compatibility since the config option was changed #ifdef DOCTEST_CONFIG_USE_IOSFWD +#ifndef DOCTEST_CONFIG_USE_STD_HEADERS #define DOCTEST_CONFIG_USE_STD_HEADERS +#endif #endif // DOCTEST_CONFIG_USE_IOSFWD +// for clang - always include ciso646 (which drags some std stuff) because +// we want to check if we are using libc++ with the _LIBCPP_VERSION macro in +// which case we don't want to forward declare stuff from std - for reference: +// https://github.com/doctest/doctest/issues/126 +// https://github.com/doctest/doctest/issues/356 +#if DOCTEST_CLANG +#include +#ifdef _LIBCPP_VERSION +#ifndef DOCTEST_CONFIG_USE_STD_HEADERS +#define DOCTEST_CONFIG_USE_STD_HEADERS +#endif +#endif // _LIBCPP_VERSION +#endif // clang + #ifdef DOCTEST_CONFIG_USE_STD_HEADERS #ifndef DOCTEST_CONFIG_INCLUDE_TYPE_TRAITS #define DOCTEST_CONFIG_INCLUDE_TYPE_TRAITS #endif // DOCTEST_CONFIG_INCLUDE_TYPE_TRAITS -#include +DOCTEST_MAKE_STD_HEADERS_CLEAN_FROM_WARNINGS_ON_WALL_BEGIN #include #include +#include +DOCTEST_MAKE_STD_HEADERS_CLEAN_FROM_WARNINGS_ON_WALL_END #else // DOCTEST_CONFIG_USE_STD_HEADERS -#if DOCTEST_CLANG -// to detect if libc++ is being used with clang (the _LIBCPP_VERSION identifier) -#include -#endif // clang - -#ifdef _LIBCPP_VERSION -#define DOCTEST_STD_NAMESPACE_BEGIN _LIBCPP_BEGIN_NAMESPACE_STD -#define DOCTEST_STD_NAMESPACE_END _LIBCPP_END_NAMESPACE_STD -#else // _LIBCPP_VERSION -#define DOCTEST_STD_NAMESPACE_BEGIN namespace std { -#define DOCTEST_STD_NAMESPACE_END } -#endif // _LIBCPP_VERSION - // Forward declaring 'X' in namespace std is not permitted by the C++ Standard. DOCTEST_MSVC_SUPPRESS_WARNING_WITH_PUSH(4643) -DOCTEST_STD_NAMESPACE_BEGIN // NOLINT (cert-dcl58-cpp) -typedef decltype(nullptr) nullptr_t; +namespace std { // NOLINT(cert-dcl58-cpp) +typedef decltype(nullptr) nullptr_t; // NOLINT(modernize-use-using) +typedef decltype(sizeof(void*)) size_t; // NOLINT(modernize-use-using) template struct char_traits; template <> struct char_traits; template -class basic_ostream; -typedef basic_ostream> ostream; +class basic_ostream; // NOLINT(fuchsia-virtual-inheritance) +typedef basic_ostream> ostream; // NOLINT(modernize-use-using) +template +// NOLINTNEXTLINE +basic_ostream& operator<<(basic_ostream&, const char*); +template +class basic_istream; +typedef basic_istream> istream; // NOLINT(modernize-use-using) template class tuple; #if DOCTEST_MSVC >= DOCTEST_COMPILER(19, 20, 0) -// see this issue on why this is needed: https://github.com/onqtam/doctest/issues/183 -template +// see this issue on why this is needed: https://github.com/doctest/doctest/issues/183 +template class allocator; -template +template class basic_string; using string = basic_string, allocator>; #endif // VS 2019 -DOCTEST_STD_NAMESPACE_END +} // namespace std DOCTEST_MSVC_SUPPRESS_WARNING_POP @@ -446,8 +534,14 @@ DOCTEST_MSVC_SUPPRESS_WARNING_POP namespace doctest { +using std::size_t; + DOCTEST_INTERFACE extern bool is_running_in_test; +#ifndef DOCTEST_CONFIG_STRING_SIZE_TYPE +#define DOCTEST_CONFIG_STRING_SIZE_TYPE unsigned +#endif + // A 24 byte string class (can be as small as 17 for x64 and 13 for x86) that can hold strings with length // of up to 23 chars on the stack before going on the heap - the last byte of the buffer is used for: // - "is small" bit - the highest bit - if "0" then it is small - otherwise its "1" (128) @@ -460,7 +554,6 @@ DOCTEST_INTERFACE extern bool is_running_in_test; // TODO: // - optimizations - like not deleting memory unnecessarily in operator= and etc. // - resize/reserve/clear -// - substr // - replace // - back/front // - iterator stuff @@ -470,63 +563,84 @@ DOCTEST_INTERFACE extern bool is_running_in_test; // - relational operators as free functions - taking const char* as one of the params class DOCTEST_INTERFACE String { - static const unsigned len = 24; //!OCLINT avoid private static members - static const unsigned last = len - 1; //!OCLINT avoid private static members +public: + using size_type = DOCTEST_CONFIG_STRING_SIZE_TYPE; + +private: + static DOCTEST_CONSTEXPR size_type len = 24; //!OCLINT avoid private static members + static DOCTEST_CONSTEXPR size_type last = len - 1; //!OCLINT avoid private static members struct view // len should be more than sizeof(view) - because of the final byte for flags { char* ptr; - unsigned size; - unsigned capacity; + size_type size; + size_type capacity; }; union { - char buf[len]; + char buf[len]; // NOLINT(*-avoid-c-arrays) view data; }; - bool isOnStack() const { return (buf[last] & 128) == 0; } - void setOnHeap(); - void setLast(unsigned in = last); + char* allocate(size_type sz); + + bool isOnStack() const noexcept { return (buf[last] & 128) == 0; } + void setOnHeap() noexcept; + void setLast(size_type in = last) noexcept; + void setSize(size_type sz) noexcept; void copy(const String& other); public: - String(); + static DOCTEST_CONSTEXPR size_type npos = static_cast(-1); + + String() noexcept; ~String(); // cppcheck-suppress noExplicitConstructor String(const char* in); - String(const char* in, unsigned in_size); + String(const char* in, size_type in_size); + + String(std::istream& in, size_type in_size); String(const String& other); String& operator=(const String& other); String& operator+=(const String& other); - String operator+(const String& other) const; - String(String&& other); - String& operator=(String&& other); + String(String&& other) noexcept; + String& operator=(String&& other) noexcept; - char operator[](unsigned i) const; - char& operator[](unsigned i); + char operator[](size_type i) const; + char& operator[](size_type i); // the only functions I'm willing to leave in the interface - available for inlining const char* c_str() const { return const_cast(this)->c_str(); } // NOLINT char* c_str() { - if(isOnStack()) + if (isOnStack()) { return reinterpret_cast(buf); + } return data.ptr; } - unsigned size() const; - unsigned capacity() const; + size_type size() const; + size_type capacity() const; + + String substr(size_type pos, size_type cnt = npos) &&; + String substr(size_type pos, size_type cnt = npos) const &; + + size_type find(char ch, size_type pos = 0) const; + size_type rfind(char ch, size_type pos = npos) const; int compare(const char* other, bool no_case = false) const; int compare(const String& other, bool no_case = false) const; + +friend DOCTEST_INTERFACE std::ostream& operator<<(std::ostream& s, const String& in); }; +DOCTEST_INTERFACE String operator+(const String& lhs, const String& rhs); + DOCTEST_INTERFACE bool operator==(const String& lhs, const String& rhs); DOCTEST_INTERFACE bool operator!=(const String& lhs, const String& rhs); DOCTEST_INTERFACE bool operator<(const String& lhs, const String& rhs); @@ -534,7 +648,21 @@ DOCTEST_INTERFACE bool operator>(const String& lhs, const String& rhs); DOCTEST_INTERFACE bool operator<=(const String& lhs, const String& rhs); DOCTEST_INTERFACE bool operator>=(const String& lhs, const String& rhs); -DOCTEST_INTERFACE std::ostream& operator<<(std::ostream& s, const String& in); +class DOCTEST_INTERFACE Contains { +public: + explicit Contains(const String& string); + + bool checkWith(const String& other) const; + + String string; +}; + +DOCTEST_INTERFACE String toString(const Contains& in); + +DOCTEST_INTERFACE bool operator==(const String& lhs, const Contains& rhs); +DOCTEST_INTERFACE bool operator==(const Contains& lhs, const String& rhs); +DOCTEST_INTERFACE bool operator!=(const String& lhs, const Contains& rhs); +DOCTEST_INTERFACE bool operator!=(const Contains& lhs, const String& rhs); namespace Color { enum Enum @@ -607,7 +735,7 @@ namespace assertType { DT_WARN_THROWS_WITH = is_throws_with | is_warn, DT_CHECK_THROWS_WITH = is_throws_with | is_check, DT_REQUIRE_THROWS_WITH = is_throws_with | is_require, - + DT_WARN_THROWS_WITH_AS = is_throws_with | is_throws_as | is_warn, DT_CHECK_THROWS_WITH_AS = is_throws_with | is_throws_as | is_check, DT_REQUIRE_THROWS_WITH_AS = is_throws_with | is_throws_as | is_require, @@ -688,9 +816,27 @@ struct DOCTEST_INTERFACE AssertData String m_decomp; // for specific exception-related asserts - bool m_threw_as; - const char* m_exception_type; - const char* m_exception_string; + bool m_threw_as; + const char* m_exception_type; + + class DOCTEST_INTERFACE StringContains { + private: + Contains content; + bool isContains; + + public: + StringContains(const String& str) : content(str), isContains(false) { } + StringContains(Contains cntn) : content(static_cast(cntn)), isContains(true) { } + + bool check(const String& str) { return isContains ? (content == str) : (content.string == str); } + + operator const String&() const { return content.string; } + + const char* c_str() const { return content.string.c_str(); } + } m_exception_string; + + AssertData(assertType::Enum at, const char* file, int line, const char* expr, + const char* exception_type, const StringContains& exception_string); }; struct DOCTEST_INTERFACE MessageData @@ -707,13 +853,13 @@ struct DOCTEST_INTERFACE SubcaseSignature const char* m_file; int m_line; + bool operator==(const SubcaseSignature& other) const; bool operator<(const SubcaseSignature& other) const; }; struct DOCTEST_INTERFACE IContextScope { - IContextScope(); - virtual ~IContextScope(); + DOCTEST_DECLARE_INTERFACE(IContextScope) virtual void stringify(std::ostream*) const = 0; }; @@ -723,9 +869,8 @@ namespace detail { struct ContextOptions //!OCLINT too many fields { - std::ostream* cout; // stdout stream - std::cout by default - std::ostream* cerr; // stderr stream - std::cerr by default - String binary_name; // the test binary name + std::ostream* cout = nullptr; // stdout stream + String binary_name; // the test binary name const detail::TestCase* currentTest = nullptr; @@ -744,9 +889,12 @@ struct ContextOptions //!OCLINT too many fields bool case_sensitive; // if filtering should be case sensitive bool exit; // if the program should be exited after the tests are ran/whatever bool duration; // print the time duration of each test case + bool minimal; // minimal console output (only test failures) + bool quiet; // no console output bool no_throw; // to skip exceptions-related assertion macros bool no_exitcode; // if the framework should return 0 as the exitcode bool no_run; // to not run the tests at all (can be done with an "*" exclude) + bool no_intro; // to not print the intro of the framework bool no_version; // to not print the version of the framework bool no_colors; // if output to the console should be colorized bool force_colors; // forces the use of colors even when a tty cannot be detected @@ -768,150 +916,184 @@ struct ContextOptions //!OCLINT too many fields }; namespace detail { - template - struct enable_if - {}; - - template - struct enable_if - { typedef TYPE type; }; - - // clang-format off - template struct remove_reference { typedef T type; }; - template struct remove_reference { typedef T type; }; - template struct remove_reference { typedef T type; }; - - template U declval(int); - - template T declval(long); - - template auto declval() DOCTEST_NOEXCEPT -> decltype(declval(0)) ; - - template struct is_lvalue_reference { const static bool value=false; }; - template struct is_lvalue_reference { const static bool value=true; }; - - template - inline T&& forward(typename remove_reference::type& t) DOCTEST_NOEXCEPT - { - return static_cast(t); - } - - template - inline T&& forward(typename remove_reference::type&& t) DOCTEST_NOEXCEPT - { - static_assert(!is_lvalue_reference::value, - "Can not forward an rvalue as an lvalue."); - return static_cast(t); - } - - template struct remove_const { typedef T type; }; - template struct remove_const { typedef T type; }; + namespace types { #ifdef DOCTEST_CONFIG_INCLUDE_TYPE_TRAITS - template struct is_enum : public std::is_enum {}; - template struct underlying_type : public std::underlying_type {}; + using namespace std; #else - // Use compiler intrinsics - template struct is_enum { constexpr static bool value = __is_enum(T); }; - template struct underlying_type { typedef __underlying_type(T) type; }; + template + struct enable_if { }; + + template + struct enable_if { using type = T; }; + + struct true_type { static DOCTEST_CONSTEXPR bool value = true; }; + struct false_type { static DOCTEST_CONSTEXPR bool value = false; }; + + template struct remove_reference { using type = T; }; + template struct remove_reference { using type = T; }; + template struct remove_reference { using type = T; }; + + template struct is_rvalue_reference : false_type { }; + template struct is_rvalue_reference : true_type { }; + + template struct remove_const { using type = T; }; + template struct remove_const { using type = T; }; + + // Compiler intrinsics + template struct is_enum { static DOCTEST_CONSTEXPR bool value = __is_enum(T); }; + template struct underlying_type { using type = __underlying_type(T); }; + + template struct is_pointer : false_type { }; + template struct is_pointer : true_type { }; + + template struct is_array : false_type { }; + // NOLINTNEXTLINE(*-avoid-c-arrays) + template struct is_array : true_type { }; #endif - // clang-format on + } + + // + template + T&& declval(); + + template + DOCTEST_CONSTEXPR_FUNC T&& forward(typename types::remove_reference::type& t) DOCTEST_NOEXCEPT { + return static_cast(t); + } + + template + DOCTEST_CONSTEXPR_FUNC T&& forward(typename types::remove_reference::type&& t) DOCTEST_NOEXCEPT { + return static_cast(t); + } template - struct deferred_false - // cppcheck-suppress unusedStructMember - { static const bool value = false; }; + struct deferred_false : types::false_type { }; - namespace has_insertion_operator_impl { - std::ostream &os(); - template - DOCTEST_REF_WRAP(T) val(); +// MSVS 2015 :( +#if defined(_MSC_VER) && _MSC_VER <= 1900 + template + struct has_global_insertion_operator : types::false_type { }; - template - struct check { - static constexpr bool value = false; - }; + template + struct has_global_insertion_operator(), declval()), void())> : types::true_type { }; - template - struct check(), void())> { - static constexpr bool value = true; - }; - } // namespace has_insertion_operator_impl + template + struct has_insertion_operator { static DOCTEST_CONSTEXPR bool value = has_global_insertion_operator::value; }; - template - using has_insertion_operator = has_insertion_operator_impl::check; + template + struct insert_hack; - DOCTEST_INTERFACE void my_memcpy(void* dest, const void* src, unsigned num); + template + struct insert_hack { + static void insert(std::ostream& os, const T& t) { ::operator<<(os, t); } + }; - DOCTEST_INTERFACE std::ostream* getTlsOss(); // returns a thread-local ostringstream - DOCTEST_INTERFACE String getTlsOssResult(); + template + struct insert_hack { + static void insert(std::ostream& os, const T& t) { operator<<(os, t); } + }; + + template + using insert_hack_t = insert_hack::value>; +#else + template + struct has_insertion_operator : types::false_type { }; +#endif + +template +struct has_insertion_operator(), declval()), void())> : types::true_type { }; + + DOCTEST_INTERFACE std::ostream* tlssPush(); + DOCTEST_INTERFACE String tlssPop(); template - struct StringMakerBase - { + struct StringMakerBase { template static String convert(const DOCTEST_REF_WRAP(T)) { +#ifdef DOCTEST_CONFIG_REQUIRE_STRINGIFICATION_FOR_ALL_USED_TYPES + static_assert(deferred_false::value, "No stringification detected for type T. See string conversion manual"); +#endif return "{?}"; } }; + template + struct filldata; + + template + void filloss(std::ostream* stream, const T& in) { + filldata::fill(stream, in); + } + + template + void filloss(std::ostream* stream, const T (&in)[N]) { // NOLINT(*-avoid-c-arrays) + // T[N], T(&)[N], T(&&)[N] have same behaviour. + // Hence remove reference. + filloss::type>(stream, in); + } + + template + String toStream(const T& in) { + std::ostream* stream = tlssPush(); + filloss(stream, in); + return tlssPop(); + } + template <> - struct StringMakerBase - { + struct StringMakerBase { template static String convert(const DOCTEST_REF_WRAP(T) in) { - *getTlsOss() << in; - return getTlsOssResult(); + return toStream(in); } }; - - DOCTEST_INTERFACE String rawMemoryToString(const void* object, unsigned size); - - template - String rawMemoryToString(const DOCTEST_REF_WRAP(T) object) { - return rawMemoryToString(&object, sizeof(object)); - } - - template - const char* type_to_string() { - return "<>"; - } } // namespace detail template -struct StringMaker : public detail::StringMakerBase::value> +struct StringMaker : public detail::StringMakerBase< + detail::has_insertion_operator::value || detail::types::is_pointer::value || detail::types::is_array::value> {}; +#ifndef DOCTEST_STRINGIFY +#ifdef DOCTEST_CONFIG_DOUBLE_STRINGIFY +#define DOCTEST_STRINGIFY(...) toString(toString(__VA_ARGS__)) +#else +#define DOCTEST_STRINGIFY(...) toString(__VA_ARGS__) +#endif +#endif + template -struct StringMaker -{ - template - static String convert(U* p) { - if(p) - return detail::rawMemoryToString(p); - return "NULL"; - } -}; +String toString() { +#if DOCTEST_MSVC >= 0 && DOCTEST_CLANG == 0 && DOCTEST_GCC == 0 + String ret = __FUNCSIG__; // class doctest::String __cdecl doctest::toString(void) + String::size_type beginPos = ret.find('<'); + return ret.substr(beginPos + 1, ret.size() - beginPos - static_cast(sizeof(">(void)"))); +#else + String ret = __PRETTY_FUNCTION__; // doctest::String toString() [with T = TYPE] + String::size_type begin = ret.find('=') + 2; + return ret.substr(begin, ret.size() - begin - 1); +#endif +} -template -struct StringMaker -{ - static String convert(R C::*p) { - if(p) - return detail::rawMemoryToString(p); - return "NULL"; - } -}; - -template ::value, bool>::type = true> +template ::value, bool>::type = true> String toString(const DOCTEST_REF_WRAP(T) value) { return StringMaker::convert(value); } #ifdef DOCTEST_CONFIG_TREAT_CHAR_STAR_AS_STRING -DOCTEST_INTERFACE String toString(char* in); DOCTEST_INTERFACE String toString(const char* in); #endif // DOCTEST_CONFIG_TREAT_CHAR_STAR_AS_STRING + +#if DOCTEST_MSVC >= DOCTEST_COMPILER(19, 20, 0) +// see this issue on why this is needed: https://github.com/doctest/doctest/issues/183 +DOCTEST_INTERFACE String toString(const std::string& in); +#endif // VS 2019 + +DOCTEST_INTERFACE String toString(String in); + +DOCTEST_INTERFACE String toString(std::nullptr_t); + DOCTEST_INTERFACE String toString(bool in); + DOCTEST_INTERFACE String toString(float in); DOCTEST_INTERFACE String toString(double in); DOCTEST_INTERFACE String toString(double long in); @@ -919,40 +1101,85 @@ DOCTEST_INTERFACE String toString(double long in); DOCTEST_INTERFACE String toString(char in); DOCTEST_INTERFACE String toString(char signed in); DOCTEST_INTERFACE String toString(char unsigned in); -DOCTEST_INTERFACE String toString(int short in); -DOCTEST_INTERFACE String toString(int short unsigned in); -DOCTEST_INTERFACE String toString(int in); -DOCTEST_INTERFACE String toString(int unsigned in); -DOCTEST_INTERFACE String toString(int long in); -DOCTEST_INTERFACE String toString(int long unsigned in); -DOCTEST_INTERFACE String toString(int long long in); -DOCTEST_INTERFACE String toString(int long long unsigned in); -DOCTEST_INTERFACE String toString(std::nullptr_t in); +DOCTEST_INTERFACE String toString(short in); +DOCTEST_INTERFACE String toString(short unsigned in); +DOCTEST_INTERFACE String toString(signed in); +DOCTEST_INTERFACE String toString(unsigned in); +DOCTEST_INTERFACE String toString(long in); +DOCTEST_INTERFACE String toString(long unsigned in); +DOCTEST_INTERFACE String toString(long long in); +DOCTEST_INTERFACE String toString(long long unsigned in); -template ::value, bool>::type = true> +template ::value, bool>::type = true> String toString(const DOCTEST_REF_WRAP(T) value) { - typedef typename detail::underlying_type::type UT; - return toString(static_cast(value)); + using UT = typename detail::types::underlying_type::type; + return (DOCTEST_STRINGIFY(static_cast(value))); } -#if DOCTEST_MSVC >= DOCTEST_COMPILER(19, 20, 0) -// see this issue on why this is needed: https://github.com/onqtam/doctest/issues/183 -DOCTEST_INTERFACE String toString(const std::string& in); -#endif // VS 2019 +namespace detail { + template + struct filldata + { + static void fill(std::ostream* stream, const T& in) { +#if defined(_MSC_VER) && _MSC_VER <= 1900 + insert_hack_t::insert(*stream, in); +#else + operator<<(*stream, in); +#endif + } + }; -class DOCTEST_INTERFACE Approx +DOCTEST_MSVC_SUPPRESS_WARNING_WITH_PUSH(4866) +// NOLINTBEGIN(*-avoid-c-arrays) + template + struct filldata { + static void fill(std::ostream* stream, const T(&in)[N]) { + *stream << "["; + for (size_t i = 0; i < N; i++) { + if (i != 0) { *stream << ", "; } + *stream << (DOCTEST_STRINGIFY(in[i])); + } + *stream << "]"; + } + }; +// NOLINTEND(*-avoid-c-arrays) +DOCTEST_MSVC_SUPPRESS_WARNING_POP + + // Specialized since we don't want the terminating null byte! +// NOLINTBEGIN(*-avoid-c-arrays) + template + struct filldata { + static void fill(std::ostream* stream, const char (&in)[N]) { + *stream << String(in, in[N - 1] ? N : N - 1); + } // NOLINT(clang-analyzer-cplusplus.NewDeleteLeaks) + }; +// NOLINTEND(*-avoid-c-arrays) + + template <> + struct filldata { + static void fill(std::ostream* stream, const void* in); + }; + + template + struct filldata { + static void fill(std::ostream* stream, const T* in) { + filldata::fill(stream, in); + } + }; +} + +struct DOCTEST_INTERFACE Approx { -public: - explicit Approx(double value); + Approx(double value); Approx operator()(double value) const; #ifdef DOCTEST_CONFIG_INCLUDE_TYPE_TRAITS template explicit Approx(const T& value, - typename detail::enable_if::value>::type* = + typename detail::types::enable_if::value>::type* = static_cast(nullptr)) { - *this = Approx(static_cast(value)); + *this = static_cast(value); } #endif // DOCTEST_CONFIG_INCLUDE_TYPE_TRAITS @@ -960,7 +1187,7 @@ public: #ifdef DOCTEST_CONFIG_INCLUDE_TYPE_TRAITS template - typename detail::enable_if::value, Approx&>::type epsilon( + typename std::enable_if::value, Approx&>::type epsilon( const T& newEpsilon) { m_epsilon = static_cast(newEpsilon); return *this; @@ -971,7 +1198,7 @@ public: #ifdef DOCTEST_CONFIG_INCLUDE_TYPE_TRAITS template - typename detail::enable_if::value, Approx&>::type scale( + typename std::enable_if::value, Approx&>::type scale( const T& newScale) { m_scale = static_cast(newScale); return *this; @@ -992,30 +1219,27 @@ public: DOCTEST_INTERFACE friend bool operator> (double lhs, const Approx & rhs); DOCTEST_INTERFACE friend bool operator> (const Approx & lhs, double rhs); - DOCTEST_INTERFACE friend String toString(const Approx& in); - #ifdef DOCTEST_CONFIG_INCLUDE_TYPE_TRAITS #define DOCTEST_APPROX_PREFIX \ - template friend typename detail::enable_if::value, bool>::type + template friend typename std::enable_if::value, bool>::type - DOCTEST_APPROX_PREFIX operator==(const T& lhs, const Approx& rhs) { return operator==(double(lhs), rhs); } + DOCTEST_APPROX_PREFIX operator==(const T& lhs, const Approx& rhs) { return operator==(static_cast(lhs), rhs); } DOCTEST_APPROX_PREFIX operator==(const Approx& lhs, const T& rhs) { return operator==(rhs, lhs); } DOCTEST_APPROX_PREFIX operator!=(const T& lhs, const Approx& rhs) { return !operator==(lhs, rhs); } DOCTEST_APPROX_PREFIX operator!=(const Approx& lhs, const T& rhs) { return !operator==(rhs, lhs); } - DOCTEST_APPROX_PREFIX operator<=(const T& lhs, const Approx& rhs) { return double(lhs) < rhs.m_value || lhs == rhs; } - DOCTEST_APPROX_PREFIX operator<=(const Approx& lhs, const T& rhs) { return lhs.m_value < double(rhs) || lhs == rhs; } - DOCTEST_APPROX_PREFIX operator>=(const T& lhs, const Approx& rhs) { return double(lhs) > rhs.m_value || lhs == rhs; } - DOCTEST_APPROX_PREFIX operator>=(const Approx& lhs, const T& rhs) { return lhs.m_value > double(rhs) || lhs == rhs; } - DOCTEST_APPROX_PREFIX operator< (const T& lhs, const Approx& rhs) { return double(lhs) < rhs.m_value && lhs != rhs; } - DOCTEST_APPROX_PREFIX operator< (const Approx& lhs, const T& rhs) { return lhs.m_value < double(rhs) && lhs != rhs; } - DOCTEST_APPROX_PREFIX operator> (const T& lhs, const Approx& rhs) { return double(lhs) > rhs.m_value && lhs != rhs; } - DOCTEST_APPROX_PREFIX operator> (const Approx& lhs, const T& rhs) { return lhs.m_value > double(rhs) && lhs != rhs; } + DOCTEST_APPROX_PREFIX operator<=(const T& lhs, const Approx& rhs) { return static_cast(lhs) < rhs.m_value || lhs == rhs; } + DOCTEST_APPROX_PREFIX operator<=(const Approx& lhs, const T& rhs) { return lhs.m_value < static_cast(rhs) || lhs == rhs; } + DOCTEST_APPROX_PREFIX operator>=(const T& lhs, const Approx& rhs) { return static_cast(lhs) > rhs.m_value || lhs == rhs; } + DOCTEST_APPROX_PREFIX operator>=(const Approx& lhs, const T& rhs) { return lhs.m_value > static_cast(rhs) || lhs == rhs; } + DOCTEST_APPROX_PREFIX operator< (const T& lhs, const Approx& rhs) { return static_cast(lhs) < rhs.m_value && lhs != rhs; } + DOCTEST_APPROX_PREFIX operator< (const Approx& lhs, const T& rhs) { return lhs.m_value < static_cast(rhs) && lhs != rhs; } + DOCTEST_APPROX_PREFIX operator> (const T& lhs, const Approx& rhs) { return static_cast(lhs) > rhs.m_value && lhs != rhs; } + DOCTEST_APPROX_PREFIX operator> (const Approx& lhs, const T& rhs) { return lhs.m_value > static_cast(rhs) && lhs != rhs; } #undef DOCTEST_APPROX_PREFIX #endif // DOCTEST_CONFIG_INCLUDE_TYPE_TRAITS // clang-format on -private: double m_epsilon; double m_scale; double m_value; @@ -1025,18 +1249,35 @@ DOCTEST_INTERFACE String toString(const Approx& in); DOCTEST_INTERFACE const ContextOptions* getContextOptions(); -#if !defined(DOCTEST_CONFIG_DISABLE) +template +struct DOCTEST_INTERFACE_DECL IsNaN +{ + F value; bool flipped; + IsNaN(F f, bool flip = false) : value(f), flipped(flip) { } + IsNaN operator!() const { return { value, !flipped }; } + operator bool() const; +}; +#ifndef __MINGW32__ +extern template struct DOCTEST_INTERFACE_DECL IsNaN; +extern template struct DOCTEST_INTERFACE_DECL IsNaN; +extern template struct DOCTEST_INTERFACE_DECL IsNaN; +#endif +DOCTEST_INTERFACE String toString(IsNaN in); +DOCTEST_INTERFACE String toString(IsNaN in); +DOCTEST_INTERFACE String toString(IsNaN in); + +#ifndef DOCTEST_CONFIG_DISABLE namespace detail { // clang-format off #ifdef DOCTEST_CONFIG_TREAT_CHAR_STAR_AS_STRING - template struct decay_array { typedef T type; }; - template struct decay_array { typedef T* type; }; - template struct decay_array { typedef T* type; }; + template struct decay_array { using type = T; }; + template struct decay_array { using type = T*; }; + template struct decay_array { using type = T*; }; - template struct not_char_pointer { enum { value = 1 }; }; - template<> struct not_char_pointer { enum { value = 0 }; }; - template<> struct not_char_pointer { enum { value = 0 }; }; + template struct not_char_pointer { static DOCTEST_CONSTEXPR value = 1; }; + template<> struct not_char_pointer { static DOCTEST_CONSTEXPR value = 0; }; + template<> struct not_char_pointer { static DOCTEST_CONSTEXPR value = 0; }; template struct can_use_op : public not_char_pointer::type> {}; #endif // DOCTEST_CONFIG_TREAT_CHAR_STAR_AS_STRING @@ -1059,16 +1300,22 @@ namespace detail { bool m_entered = false; Subcase(const String& name, const char* file, int line); + Subcase(const Subcase&) = delete; + Subcase(Subcase&&) = delete; + Subcase& operator=(const Subcase&) = delete; + Subcase& operator=(Subcase&&) = delete; ~Subcase(); operator bool() const; + + private: + bool checkFilters(); }; template String stringifyBinaryExpr(const DOCTEST_REF_WRAP(L) lhs, const char* op, const DOCTEST_REF_WRAP(R) rhs) { - // NOLINTNEXTLINE(clang-analyzer-cplusplus.NewDeleteLeaks) - return doctest::toString(lhs) + op + doctest::toString(rhs); + return (DOCTEST_STRINGIFY(lhs)) + op + (DOCTEST_STRINGIFY(rhs)); } #if DOCTEST_CLANG && DOCTEST_CLANG < DOCTEST_COMPILER(3, 6, 0) @@ -1079,12 +1326,12 @@ DOCTEST_CLANG_SUPPRESS_WARNING_WITH_PUSH("-Wunused-comparison") // If not it doesn't find the operator or if the operator at global scope is defined after // this template, the template won't be instantiated due to SFINAE. Once the template is not // instantiated it can look for global operator using normal conversions. -#define SFINAE_OP(ret,op) decltype((void)(doctest::detail::declval() op doctest::detail::declval()),static_cast(0)) +#define SFINAE_OP(ret,op) decltype((void)(doctest::detail::declval() op doctest::detail::declval()),ret{}) #define DOCTEST_DO_BINARY_EXPRESSION_COMPARISON(op, op_str, op_macro) \ template \ - DOCTEST_NOINLINE SFINAE_OP(Result,op) operator op(R&& rhs) { \ - bool res = op_macro(doctest::detail::forward(lhs), doctest::detail::forward(rhs)); \ + DOCTEST_NOINLINE SFINAE_OP(Result,op) operator op(R&& rhs) { \ + bool res = op_macro(doctest::detail::forward(lhs), doctest::detail::forward(rhs)); \ if(m_at & assertType::is_false) \ res = !res; \ if(!res || doctest::getContextOptions()->success) \ @@ -1103,11 +1350,12 @@ DOCTEST_CLANG_SUPPRESS_WARNING_WITH_PUSH("-Wunused-comparison") return *this; \ } - struct DOCTEST_INTERFACE Result + struct DOCTEST_INTERFACE Result // NOLINT(*-member-init) { bool m_passed; String m_decomp; + Result() = default; // TODO: Why do we need this? (To remove NOLINT) Result(bool passed, const String& decomposition = String()); // forbidding some expressions based on this table: https://en.cppreference.com/w/cpp/language/operator_precedence @@ -1164,8 +1412,7 @@ DOCTEST_CLANG_SUPPRESS_WARNING_WITH_PUSH("-Wunused-comparison") #ifndef DOCTEST_CONFIG_TREAT_CHAR_STAR_AS_STRING #define DOCTEST_COMPARISON_RETURN_TYPE bool #else // DOCTEST_CONFIG_TREAT_CHAR_STAR_AS_STRING -#define DOCTEST_COMPARISON_RETURN_TYPE typename enable_if::value || can_use_op::value, bool>::type - // NOLINTNEXTLINE(clang-analyzer-cplusplus.NewDeleteLeaks) +#define DOCTEST_COMPARISON_RETURN_TYPE typename types::enable_if::value || can_use_op::value, bool>::type inline bool eq(const char* lhs, const char* rhs) { return String(lhs) == String(rhs); } inline bool ne(const char* lhs, const char* rhs) { return String(lhs) != String(rhs); } inline bool lt(const char* lhs, const char* rhs) { return String(lhs) < String(rhs); } @@ -1213,26 +1460,26 @@ DOCTEST_CLANG_SUPPRESS_WARNING_WITH_PUSH("-Wunused-comparison") assertType::Enum m_at; explicit Expression_lhs(L&& in, assertType::Enum at) - : lhs(doctest::detail::forward(in)) + : lhs(static_cast(in)) , m_at(at) {} DOCTEST_NOINLINE operator Result() { -// this is needed only foc MSVC 2015: -// https://ci.appveyor.com/project/onqtam/doctest/builds/38181202 +// this is needed only for MSVC 2015 DOCTEST_MSVC_SUPPRESS_WARNING_WITH_PUSH(4800) // 'int': forcing value to bool bool res = static_cast(lhs); DOCTEST_MSVC_SUPPRESS_WARNING_POP - if(m_at & assertType::is_false) //!OCLINT bitwise operator in conditional + if(m_at & assertType::is_false) { //!OCLINT bitwise operator in conditional res = !res; + } - if(!res || getContextOptions()->success) - return Result(res, doctest::toString(lhs)); - return Result(res); + if(!res || getContextOptions()->success) { + return { res, (DOCTEST_STRINGIFY(lhs)) }; + } + return { res }; } - /* This is required for user-defined conversions from Expression_lhs to L */ - //operator L() const { return lhs; } - operator L() const { return lhs; } + /* This is required for user-defined conversions from Expression_lhs to L */ + operator L() const { return lhs; } // clang-format off DOCTEST_DO_BINARY_EXPRESSION_COMPARISON(==, " == ", DOCTEST_CMP_EQ) //!OCLINT bitwise operator in conditional @@ -1289,22 +1536,27 @@ DOCTEST_CLANG_SUPPRESS_WARNING_POP // https://github.com/catchorg/Catch2/issues/870 // https://github.com/catchorg/Catch2/issues/565 template - Expression_lhs operator<<(L &&operand) { - return Expression_lhs(doctest::detail::forward(operand), m_at); + Expression_lhs operator<<(L&& operand) { + return Expression_lhs(static_cast(operand), m_at); + } + + template ::value,void >::type* = nullptr> + Expression_lhs operator<<(const L &operand) { + return Expression_lhs(operand, m_at); } }; struct DOCTEST_INTERFACE TestSuite { - const char* m_test_suite; - const char* m_description; - bool m_skip; - bool m_no_breaks; - bool m_no_output; - bool m_may_fail; - bool m_should_fail; - int m_expected_failures; - double m_timeout; + const char* m_test_suite = nullptr; + const char* m_description = nullptr; + bool m_skip = false; + bool m_no_breaks = false; + bool m_no_output = false; + bool m_may_fail = false; + bool m_should_fail = false; + int m_expected_failures = 0; + double m_timeout = 0; TestSuite& operator*(const char* in); @@ -1315,25 +1567,28 @@ DOCTEST_CLANG_SUPPRESS_WARNING_POP } }; - typedef void (*funcType)(); + using funcType = void (*)(); struct DOCTEST_INTERFACE TestCase : public TestCaseData { funcType m_test; // a function pointer to the test case - const char* m_type; // for templated test cases - gets appended to the real name + String m_type; // for templated test cases - gets appended to the real name int m_template_id; // an ID used to distinguish between the different versions of a templated test case String m_full_name; // contains the name (only for templated test cases!) + the template type TestCase(funcType test, const char* file, unsigned line, const TestSuite& test_suite, - const char* type = "", int template_id = -1); + const String& type = String(), int template_id = -1); TestCase(const TestCase& other); + TestCase(TestCase&&) = delete; DOCTEST_MSVC_SUPPRESS_WARNING_WITH_PUSH(26434) // hides a non-virtual function TestCase& operator=(const TestCase& other); DOCTEST_MSVC_SUPPRESS_WARNING_POP + TestCase& operator=(TestCase&&) = delete; + TestCase& operator*(const char* in); template @@ -1343,6 +1598,8 @@ DOCTEST_CLANG_SUPPRESS_WARNING_POP } bool operator<(const TestCase& other) const; + + ~TestCase() = default; }; // forward declarations of functions used by the macros @@ -1382,27 +1639,36 @@ DOCTEST_CLANG_SUPPRESS_WARNING_POP struct DOCTEST_INTERFACE ResultBuilder : public AssertData { ResultBuilder(assertType::Enum at, const char* file, int line, const char* expr, - const char* exception_type = "", const char* exception_string = ""); + const char* exception_type = "", const String& exception_string = ""); + + ResultBuilder(assertType::Enum at, const char* file, int line, const char* expr, + const char* exception_type, const Contains& exception_string); void setResult(const Result& res); template - DOCTEST_NOINLINE void binary_assert(const DOCTEST_REF_WRAP(L) lhs, + DOCTEST_NOINLINE bool binary_assert(const DOCTEST_REF_WRAP(L) lhs, const DOCTEST_REF_WRAP(R) rhs) { m_failed = !RelationalComparator()(lhs, rhs); - if(m_failed || getContextOptions()->success) + if (m_failed || getContextOptions()->success) { m_decomp = stringifyBinaryExpr(lhs, ", ", rhs); + } + return !m_failed; } template - DOCTEST_NOINLINE void unary_assert(const DOCTEST_REF_WRAP(L) val) { + DOCTEST_NOINLINE bool unary_assert(const DOCTEST_REF_WRAP(L) val) { m_failed = !val; - if(m_at & assertType::is_false) //!OCLINT bitwise operator in conditional + if (m_at & assertType::is_false) { //!OCLINT bitwise operator in conditional m_failed = !m_failed; + } - if(m_failed || getContextOptions()->success) - m_decomp = toString(val); + if (m_failed || getContextOptions()->success) { + m_decomp = (DOCTEST_STRINGIFY(val)); + } + + return !m_failed; } void translateException(); @@ -1422,8 +1688,8 @@ DOCTEST_CLANG_SUPPRESS_WARNING_POP DOCTEST_INTERFACE void failed_out_of_a_testing_context(const AssertData& ad); - DOCTEST_INTERFACE void decomp_assert(assertType::Enum at, const char* file, int line, - const char* expr, Result result); + DOCTEST_INTERFACE bool decomp_assert(assertType::Enum at, const char* file, int line, + const char* expr, const Result& result); #define DOCTEST_ASSERT_OUT_OF_TESTS(decomp) \ do { \ @@ -1438,7 +1704,7 @@ DOCTEST_CLANG_SUPPRESS_WARNING_POP if(checkIfShouldThrow(at)) \ throwException(); \ } \ - return; \ + return !failed; \ } \ } while(false) @@ -1453,7 +1719,7 @@ DOCTEST_CLANG_SUPPRESS_WARNING_POP throwException() template - DOCTEST_NOINLINE void binary_assert(assertType::Enum at, const char* file, int line, + DOCTEST_NOINLINE bool binary_assert(assertType::Enum at, const char* file, int line, const char* expr, const DOCTEST_REF_WRAP(L) lhs, const DOCTEST_REF_WRAP(R) rhs) { bool failed = !RelationalComparator()(lhs, rhs); @@ -1464,10 +1730,11 @@ DOCTEST_CLANG_SUPPRESS_WARNING_POP // ################################################################################### DOCTEST_ASSERT_OUT_OF_TESTS(stringifyBinaryExpr(lhs, ", ", rhs)); DOCTEST_ASSERT_IN_TESTS(stringifyBinaryExpr(lhs, ", ", rhs)); + return !failed; } template - DOCTEST_NOINLINE void unary_assert(assertType::Enum at, const char* file, int line, + DOCTEST_NOINLINE bool unary_assert(assertType::Enum at, const char* file, int line, const char* expr, const DOCTEST_REF_WRAP(L) val) { bool failed = !val; @@ -1478,14 +1745,14 @@ DOCTEST_CLANG_SUPPRESS_WARNING_POP // IF THE DEBUGGER BREAKS HERE - GO 1 LEVEL UP IN THE CALLSTACK FOR THE FAILING ASSERT // THIS IS THE EFFECT OF HAVING 'DOCTEST_CONFIG_SUPER_FAST_ASSERTS' DEFINED // ################################################################################### - DOCTEST_ASSERT_OUT_OF_TESTS(toString(val)); - DOCTEST_ASSERT_IN_TESTS(toString(val)); + DOCTEST_ASSERT_OUT_OF_TESTS((DOCTEST_STRINGIFY(val))); + DOCTEST_ASSERT_IN_TESTS((DOCTEST_STRINGIFY(val))); + return !failed; } struct DOCTEST_INTERFACE IExceptionTranslator { - IExceptionTranslator(); - virtual ~IExceptionTranslator(); + DOCTEST_DECLARE_INTERFACE(IExceptionTranslator) virtual bool translate(String&) const = 0; }; @@ -1501,7 +1768,7 @@ DOCTEST_CLANG_SUPPRESS_WARNING_POP try { throw; // lgtm [cpp/rethrow-no-exception] // cppcheck-suppress catchExceptionByValue - } catch(T ex) { // NOLINT + } catch(const T& ex) { res = m_translateFunction(ex); //!OCLINT parameter reassignment return true; } catch(...) {} //!OCLINT - empty catch statement @@ -1516,95 +1783,70 @@ DOCTEST_CLANG_SUPPRESS_WARNING_POP DOCTEST_INTERFACE void registerExceptionTranslatorImpl(const IExceptionTranslator* et); - template - struct StringStreamBase - { - template - static void convert(std::ostream* s, const T& in) { - *s << toString(in); - } - - // always treat char* as a string in this context - no matter - // if DOCTEST_CONFIG_TREAT_CHAR_STAR_AS_STRING is defined - static void convert(std::ostream* s, const char* in) { *s << String(in); } - }; - - template <> - struct StringStreamBase - { - template - static void convert(std::ostream* s, const T& in) { - *s << in; - } - }; - - template - struct StringStream : public StringStreamBase::value> - {}; - - template - void toStream(std::ostream* s, const T& value) { - StringStream::convert(s, value); - } - -#ifdef DOCTEST_CONFIG_TREAT_CHAR_STAR_AS_STRING - DOCTEST_INTERFACE void toStream(std::ostream* s, char* in); - DOCTEST_INTERFACE void toStream(std::ostream* s, const char* in); -#endif // DOCTEST_CONFIG_TREAT_CHAR_STAR_AS_STRING - DOCTEST_INTERFACE void toStream(std::ostream* s, bool in); - DOCTEST_INTERFACE void toStream(std::ostream* s, float in); - DOCTEST_INTERFACE void toStream(std::ostream* s, double in); - DOCTEST_INTERFACE void toStream(std::ostream* s, double long in); - - DOCTEST_INTERFACE void toStream(std::ostream* s, char in); - DOCTEST_INTERFACE void toStream(std::ostream* s, char signed in); - DOCTEST_INTERFACE void toStream(std::ostream* s, char unsigned in); - DOCTEST_INTERFACE void toStream(std::ostream* s, int short in); - DOCTEST_INTERFACE void toStream(std::ostream* s, int short unsigned in); - DOCTEST_INTERFACE void toStream(std::ostream* s, int in); - DOCTEST_INTERFACE void toStream(std::ostream* s, int unsigned in); - DOCTEST_INTERFACE void toStream(std::ostream* s, int long in); - DOCTEST_INTERFACE void toStream(std::ostream* s, int long unsigned in); - DOCTEST_INTERFACE void toStream(std::ostream* s, int long long in); - DOCTEST_INTERFACE void toStream(std::ostream* s, int long long unsigned in); - - // ContextScope base class used to allow implementing methods of ContextScope + // ContextScope base class used to allow implementing methods of ContextScope // that don't depend on the template parameter in doctest.cpp. - class DOCTEST_INTERFACE ContextScopeBase : public IContextScope { + struct DOCTEST_INTERFACE ContextScopeBase : public IContextScope { + ContextScopeBase(const ContextScopeBase&) = delete; + + ContextScopeBase& operator=(const ContextScopeBase&) = delete; + ContextScopeBase& operator=(ContextScopeBase&&) = delete; + + ~ContextScopeBase() override = default; + protected: ContextScopeBase(); + ContextScopeBase(ContextScopeBase&& other) noexcept; void destroy(); + bool need_to_destroy{true}; }; template class ContextScope : public ContextScopeBase { - const L lambda_; + L lambda_; public: explicit ContextScope(const L &lambda) : lambda_(lambda) {} + explicit ContextScope(L&& lambda) : lambda_(static_cast(lambda)) { } - ContextScope(ContextScope &&other) : lambda_(other.lambda_) {} + ContextScope(const ContextScope&) = delete; + ContextScope(ContextScope&&) noexcept = default; + + ContextScope& operator=(const ContextScope&) = delete; + ContextScope& operator=(ContextScope&&) = delete; void stringify(std::ostream* s) const override { lambda_(s); } - ~ContextScope() override { destroy(); } + ~ContextScope() override { + if (need_to_destroy) { + destroy(); + } + } }; struct DOCTEST_INTERFACE MessageBuilder : public MessageData { std::ostream* m_stream; + bool logged = false; MessageBuilder(const char* file, int line, assertType::Enum severity); - MessageBuilder() = delete; + + MessageBuilder(const MessageBuilder&) = delete; + MessageBuilder(MessageBuilder&&) = delete; + + MessageBuilder& operator=(const MessageBuilder&) = delete; + MessageBuilder& operator=(MessageBuilder&&) = delete; + ~MessageBuilder(); // the preferred way of chaining parameters for stringification +DOCTEST_MSVC_SUPPRESS_WARNING_WITH_PUSH(4866) template MessageBuilder& operator,(const T& in) { - toStream(m_stream, in); + *m_stream << (DOCTEST_STRINGIFY(in)); return *this; } +DOCTEST_MSVC_SUPPRESS_WARNING_POP // kept here just for backwards-compatibility - the comma operator should be preferred now template @@ -1620,7 +1862,7 @@ DOCTEST_CLANG_SUPPRESS_WARNING_POP bool log(); void react(); }; - + template ContextScope MakeContextScope(const L &lambda) { return ContextScope(lambda); @@ -1673,7 +1915,7 @@ int registerExceptionTranslator(String (*)(T)) { #endif // DOCTEST_CONFIG_DISABLE namespace detail { - typedef void (*assert_handler)(const AssertData&); + using assert_handler = void (*)(const AssertData&); struct ContextState; } // namespace detail @@ -1686,12 +1928,19 @@ class DOCTEST_INTERFACE Context public: explicit Context(int argc = 0, const char* const* argv = nullptr); - ~Context(); + Context(const Context&) = delete; + Context(Context&&) = delete; + + Context& operator=(const Context&) = delete; + Context& operator=(Context&&) = delete; + + ~Context(); // NOLINT(performance-trivially-destructible) void applyCommandLine(int argc, const char* const* argv); void addFilter(const char* filter, const char* value); void clearFilters(); + void setOption(const char* option, bool value); void setOption(const char* option, int value); void setOption(const char* option, const char* value); @@ -1701,6 +1950,8 @@ public: void setAssertHandler(detail::assert_handler ah); + void setCout(std::ostream* out); + int run(); }; @@ -1727,6 +1978,7 @@ struct DOCTEST_INTERFACE CurrentTestCaseStats int numAssertsFailedCurrentTest; double seconds; int failure_flags; // use TestCaseFailureReason::Enum + bool testCaseSuccess; }; struct DOCTEST_INTERFACE TestCaseException @@ -1790,8 +2042,7 @@ struct DOCTEST_INTERFACE IReporter // or isn't in the execution range (between first and last) (safe to cache a pointer to the input) virtual void test_case_skipped(const TestCaseData&) = 0; - // doctest will not be managing the lifetimes of reporters given to it but this would still be nice to have - virtual ~IReporter(); + DOCTEST_DECLARE_INTERFACE(IReporter) // can obtain all currently active contexts and stringify them if one wishes to do so static int get_num_active_contexts(); @@ -1803,7 +2054,7 @@ struct DOCTEST_INTERFACE IReporter }; namespace detail { - typedef IReporter* (*reporterCreatorFunc)(const ContextOptions&); + using reporterCreatorFunc = IReporter* (*)(const ContextOptions&); DOCTEST_INTERFACE void registerReporterImpl(const char* name, int prio, reporterCreatorFunc c, bool isReporter); @@ -1820,14 +2071,30 @@ int registerReporter(const char* name, int priority, bool isReporter) { } } // namespace doctest +#ifdef DOCTEST_CONFIG_ASSERTS_RETURN_VALUES +#define DOCTEST_FUNC_EMPTY [] { return false; }() +#else +#define DOCTEST_FUNC_EMPTY (void)0 +#endif + // if registering is not disabled -#if !defined(DOCTEST_CONFIG_DISABLE) +#ifndef DOCTEST_CONFIG_DISABLE + +#ifdef DOCTEST_CONFIG_ASSERTS_RETURN_VALUES +#define DOCTEST_FUNC_SCOPE_BEGIN [&] +#define DOCTEST_FUNC_SCOPE_END () +#define DOCTEST_FUNC_SCOPE_RET(v) return v +#else +#define DOCTEST_FUNC_SCOPE_BEGIN do +#define DOCTEST_FUNC_SCOPE_END while(false) +#define DOCTEST_FUNC_SCOPE_RET(v) (void)0 +#endif // common code in asserts - for convenience -#define DOCTEST_ASSERT_LOG_AND_REACT(b) \ - if(b.log()) \ - DOCTEST_BREAK_INTO_DEBUGGER(); \ - b.react() +#define DOCTEST_ASSERT_LOG_REACT_RETURN(b) \ + if(b.log()) DOCTEST_BREAK_INTO_DEBUGGER(); \ + b.react(); \ + DOCTEST_FUNC_SCOPE_RET(!b.m_failed) #ifdef DOCTEST_CONFIG_NO_TRY_CATCH_IN_ASSERTS #define DOCTEST_WRAP_IN_TRY(x) x; @@ -1835,7 +2102,7 @@ int registerReporter(const char* name, int priority, bool isReporter) { #define DOCTEST_WRAP_IN_TRY(x) \ try { \ x; \ - } catch(...) { _DOCTEST_RB.translateException(); } + } catch(...) { DOCTEST_RB.translateException(); } #endif // DOCTEST_CONFIG_NO_TRY_CATCH_IN_ASSERTS #ifdef DOCTEST_CONFIG_VOID_CAST_EXPRESSIONS @@ -1849,27 +2116,26 @@ int registerReporter(const char* name, int priority, bool isReporter) { // registers the test by initializing a dummy var with a function #define DOCTEST_REGISTER_FUNCTION(global_prefix, f, decorators) \ - global_prefix DOCTEST_GLOBAL_NO_WARNINGS(DOCTEST_ANONYMOUS(_DOCTEST_ANON_VAR_)) = \ + global_prefix DOCTEST_GLOBAL_NO_WARNINGS(DOCTEST_ANONYMOUS(DOCTEST_ANON_VAR_), /* NOLINT */ \ doctest::detail::regTest( \ doctest::detail::TestCase( \ f, __FILE__, __LINE__, \ doctest_detail_test_suite_ns::getCurrentTestSuite()) * \ - decorators); \ - DOCTEST_GLOBAL_NO_WARNINGS_END() + decorators)) #define DOCTEST_IMPLEMENT_FIXTURE(der, base, func, decorators) \ - namespace { \ + namespace { /* NOLINT */ \ struct der : public base \ { \ void f(); \ }; \ - static void func() { \ + static inline DOCTEST_NOINLINE void func() { \ der v; \ v.f(); \ } \ DOCTEST_REGISTER_FUNCTION(DOCTEST_EMPTY, func, decorators) \ } \ - inline DOCTEST_NOINLINE void der::f() + inline DOCTEST_NOINLINE void der::f() // NOLINT(misc-definitions-in-headers) #define DOCTEST_CREATE_AND_REGISTER_FUNCTION(f, decorators) \ static void f(); \ @@ -1878,18 +2144,18 @@ int registerReporter(const char* name, int priority, bool isReporter) { #define DOCTEST_CREATE_AND_REGISTER_FUNCTION_IN_CLASS(f, proxy, decorators) \ static doctest::detail::funcType proxy() { return f; } \ - DOCTEST_REGISTER_FUNCTION(inline const, proxy(), decorators) \ + DOCTEST_REGISTER_FUNCTION(inline, proxy(), decorators) \ static void f() // for registering tests #define DOCTEST_TEST_CASE(decorators) \ - DOCTEST_CREATE_AND_REGISTER_FUNCTION(DOCTEST_ANONYMOUS(_DOCTEST_ANON_FUNC_), decorators) + DOCTEST_CREATE_AND_REGISTER_FUNCTION(DOCTEST_ANONYMOUS(DOCTEST_ANON_FUNC_), decorators) // for registering tests in classes - requires C++17 for inline variables! -#if __cplusplus >= 201703L || (DOCTEST_MSVC >= DOCTEST_COMPILER(19, 12, 0) && _MSVC_LANG >= 201703L) +#if DOCTEST_CPLUSPLUS >= 201703L #define DOCTEST_TEST_CASE_CLASS(decorators) \ - DOCTEST_CREATE_AND_REGISTER_FUNCTION_IN_CLASS(DOCTEST_ANONYMOUS(_DOCTEST_ANON_FUNC_), \ - DOCTEST_ANONYMOUS(_DOCTEST_ANON_PROXY_), \ + DOCTEST_CREATE_AND_REGISTER_FUNCTION_IN_CLASS(DOCTEST_ANONYMOUS(DOCTEST_ANON_FUNC_), \ + DOCTEST_ANONYMOUS(DOCTEST_ANON_PROXY_), \ decorators) #else // DOCTEST_TEST_CASE_CLASS #define DOCTEST_TEST_CASE_CLASS(...) \ @@ -1898,26 +2164,25 @@ int registerReporter(const char* name, int priority, bool isReporter) { // for registering tests with a fixture #define DOCTEST_TEST_CASE_FIXTURE(c, decorators) \ - DOCTEST_IMPLEMENT_FIXTURE(DOCTEST_ANONYMOUS(_DOCTEST_ANON_CLASS_), c, \ - DOCTEST_ANONYMOUS(_DOCTEST_ANON_FUNC_), decorators) + DOCTEST_IMPLEMENT_FIXTURE(DOCTEST_ANONYMOUS(DOCTEST_ANON_CLASS_), c, \ + DOCTEST_ANONYMOUS(DOCTEST_ANON_FUNC_), decorators) // for converting types to strings without the header and demangling -#define DOCTEST_TYPE_TO_STRING_IMPL(...) \ - template <> \ - inline const char* type_to_string<__VA_ARGS__>() { \ - return "<" #__VA_ARGS__ ">"; \ - } -#define DOCTEST_TYPE_TO_STRING(...) \ - namespace doctest { namespace detail { \ - DOCTEST_TYPE_TO_STRING_IMPL(__VA_ARGS__) \ +#define DOCTEST_TYPE_TO_STRING_AS(str, ...) \ + namespace doctest { \ + template <> \ + inline String toString<__VA_ARGS__>() { \ + return str; \ } \ } \ - typedef int DOCTEST_ANONYMOUS(_DOCTEST_ANON_FOR_SEMICOLON_) + static_assert(true, "") + +#define DOCTEST_TYPE_TO_STRING(...) DOCTEST_TYPE_TO_STRING_AS(#__VA_ARGS__, __VA_ARGS__) #define DOCTEST_TEST_CASE_TEMPLATE_DEFINE_IMPL(dec, T, iter, func) \ template \ static void func(); \ - namespace { \ + namespace { /* NOLINT */ \ template \ struct iter; \ template \ @@ -1926,7 +2191,7 @@ int registerReporter(const char* name, int priority, bool isReporter) { iter(const char* file, unsigned line, int index) { \ doctest::detail::regTest(doctest::detail::TestCase(func, file, line, \ doctest_detail_test_suite_ns::getCurrentTestSuite(), \ - doctest::detail::type_to_string(), \ + doctest::toString(), \ int(line) * 1000 + index) \ * dec); \ iter>(file, line, index + 1); \ @@ -1943,20 +2208,20 @@ int registerReporter(const char* name, int priority, bool isReporter) { #define DOCTEST_TEST_CASE_TEMPLATE_DEFINE(dec, T, id) \ DOCTEST_TEST_CASE_TEMPLATE_DEFINE_IMPL(dec, T, DOCTEST_CAT(id, ITERATOR), \ - DOCTEST_ANONYMOUS(_DOCTEST_ANON_TMP_)) + DOCTEST_ANONYMOUS(DOCTEST_ANON_TMP_)) #define DOCTEST_TEST_CASE_TEMPLATE_INSTANTIATE_IMPL(id, anon, ...) \ - DOCTEST_GLOBAL_NO_WARNINGS(DOCTEST_CAT(anon, DUMMY)) = \ - doctest::detail::instantiationHelper(DOCTEST_CAT(id, ITERATOR)<__VA_ARGS__>(__FILE__, __LINE__, 0));\ - DOCTEST_GLOBAL_NO_WARNINGS_END() + DOCTEST_GLOBAL_NO_WARNINGS(DOCTEST_CAT(anon, DUMMY), /* NOLINT(cert-err58-cpp, fuchsia-statically-constructed-objects) */ \ + doctest::detail::instantiationHelper( \ + DOCTEST_CAT(id, ITERATOR)<__VA_ARGS__>(__FILE__, __LINE__, 0))) #define DOCTEST_TEST_CASE_TEMPLATE_INVOKE(id, ...) \ - DOCTEST_TEST_CASE_TEMPLATE_INSTANTIATE_IMPL(id, DOCTEST_ANONYMOUS(_DOCTEST_ANON_TMP_), std::tuple<__VA_ARGS__>) \ - typedef int DOCTEST_ANONYMOUS(_DOCTEST_ANON_FOR_SEMICOLON_) + DOCTEST_TEST_CASE_TEMPLATE_INSTANTIATE_IMPL(id, DOCTEST_ANONYMOUS(DOCTEST_ANON_TMP_), std::tuple<__VA_ARGS__>) \ + static_assert(true, "") #define DOCTEST_TEST_CASE_TEMPLATE_APPLY(id, ...) \ - DOCTEST_TEST_CASE_TEMPLATE_INSTANTIATE_IMPL(id, DOCTEST_ANONYMOUS(_DOCTEST_ANON_TMP_), __VA_ARGS__) \ - typedef int DOCTEST_ANONYMOUS(_DOCTEST_ANON_FOR_SEMICOLON_) + DOCTEST_TEST_CASE_TEMPLATE_INSTANTIATE_IMPL(id, DOCTEST_ANONYMOUS(DOCTEST_ANON_TMP_), __VA_ARGS__) \ + static_assert(true, "") #define DOCTEST_TEST_CASE_TEMPLATE_IMPL(dec, T, anon, ...) \ DOCTEST_TEST_CASE_TEMPLATE_DEFINE_IMPL(dec, T, DOCTEST_CAT(anon, ITERATOR), anon); \ @@ -1965,17 +2230,17 @@ int registerReporter(const char* name, int priority, bool isReporter) { static void anon() #define DOCTEST_TEST_CASE_TEMPLATE(dec, T, ...) \ - DOCTEST_TEST_CASE_TEMPLATE_IMPL(dec, T, DOCTEST_ANONYMOUS(_DOCTEST_ANON_TMP_), __VA_ARGS__) + DOCTEST_TEST_CASE_TEMPLATE_IMPL(dec, T, DOCTEST_ANONYMOUS(DOCTEST_ANON_TMP_), __VA_ARGS__) // for subcases #define DOCTEST_SUBCASE(name) \ - if(const doctest::detail::Subcase & DOCTEST_ANONYMOUS(_DOCTEST_ANON_SUBCASE_) DOCTEST_UNUSED = \ + if(const doctest::detail::Subcase & DOCTEST_ANONYMOUS(DOCTEST_ANON_SUBCASE_) DOCTEST_UNUSED = \ doctest::detail::Subcase(name, __FILE__, __LINE__)) // for grouping tests in test suites by using code blocks #define DOCTEST_TEST_SUITE_IMPL(decorators, ns_name) \ namespace ns_name { namespace doctest_detail_test_suite_ns { \ - static DOCTEST_NOINLINE doctest::detail::TestSuite& getCurrentTestSuite() { \ + static DOCTEST_NOINLINE doctest::detail::TestSuite& getCurrentTestSuite() noexcept { \ DOCTEST_MSVC_SUPPRESS_WARNING_WITH_PUSH(4640) \ DOCTEST_CLANG_SUPPRESS_WARNING_WITH_PUSH("-Wexit-time-destructors") \ DOCTEST_GCC_SUPPRESS_WARNING_WITH_PUSH("-Wmissing-field-initializers") \ @@ -1995,53 +2260,53 @@ int registerReporter(const char* name, int priority, bool isReporter) { namespace ns_name #define DOCTEST_TEST_SUITE(decorators) \ - DOCTEST_TEST_SUITE_IMPL(decorators, DOCTEST_ANONYMOUS(_DOCTEST_ANON_SUITE_)) + DOCTEST_TEST_SUITE_IMPL(decorators, DOCTEST_ANONYMOUS(DOCTEST_ANON_SUITE_)) // for starting a testsuite block #define DOCTEST_TEST_SUITE_BEGIN(decorators) \ - DOCTEST_GLOBAL_NO_WARNINGS(DOCTEST_ANONYMOUS(_DOCTEST_ANON_VAR_)) = \ - doctest::detail::setTestSuite(doctest::detail::TestSuite() * decorators); \ - DOCTEST_GLOBAL_NO_WARNINGS_END() \ - typedef int DOCTEST_ANONYMOUS(_DOCTEST_ANON_FOR_SEMICOLON_) + DOCTEST_GLOBAL_NO_WARNINGS(DOCTEST_ANONYMOUS(DOCTEST_ANON_VAR_), /* NOLINT(cert-err58-cpp) */ \ + doctest::detail::setTestSuite(doctest::detail::TestSuite() * decorators)) \ + static_assert(true, "") // for ending a testsuite block #define DOCTEST_TEST_SUITE_END \ - DOCTEST_GLOBAL_NO_WARNINGS(DOCTEST_ANONYMOUS(_DOCTEST_ANON_VAR_)) = \ - doctest::detail::setTestSuite(doctest::detail::TestSuite() * ""); \ - DOCTEST_GLOBAL_NO_WARNINGS_END() \ - typedef int DOCTEST_ANONYMOUS(_DOCTEST_ANON_FOR_SEMICOLON_) + DOCTEST_GLOBAL_NO_WARNINGS(DOCTEST_ANONYMOUS(DOCTEST_ANON_VAR_), /* NOLINT(cert-err58-cpp) */ \ + doctest::detail::setTestSuite(doctest::detail::TestSuite() * "")) \ + using DOCTEST_ANONYMOUS(DOCTEST_ANON_FOR_SEMICOLON_) = int // for registering exception translators #define DOCTEST_REGISTER_EXCEPTION_TRANSLATOR_IMPL(translatorName, signature) \ inline doctest::String translatorName(signature); \ - DOCTEST_GLOBAL_NO_WARNINGS(DOCTEST_ANONYMOUS(_DOCTEST_ANON_TRANSLATOR_)) = \ - doctest::registerExceptionTranslator(translatorName); \ - DOCTEST_GLOBAL_NO_WARNINGS_END() \ + DOCTEST_GLOBAL_NO_WARNINGS(DOCTEST_ANONYMOUS(DOCTEST_ANON_TRANSLATOR_), /* NOLINT(cert-err58-cpp) */ \ + doctest::registerExceptionTranslator(translatorName)) \ doctest::String translatorName(signature) #define DOCTEST_REGISTER_EXCEPTION_TRANSLATOR(signature) \ - DOCTEST_REGISTER_EXCEPTION_TRANSLATOR_IMPL(DOCTEST_ANONYMOUS(_DOCTEST_ANON_TRANSLATOR_), \ + DOCTEST_REGISTER_EXCEPTION_TRANSLATOR_IMPL(DOCTEST_ANONYMOUS(DOCTEST_ANON_TRANSLATOR_), \ signature) // for registering reporters #define DOCTEST_REGISTER_REPORTER(name, priority, reporter) \ - DOCTEST_GLOBAL_NO_WARNINGS(DOCTEST_ANONYMOUS(_DOCTEST_ANON_REPORTER_)) = \ - doctest::registerReporter(name, priority, true); \ - DOCTEST_GLOBAL_NO_WARNINGS_END() typedef int DOCTEST_ANONYMOUS(_DOCTEST_ANON_FOR_SEMICOLON_) + DOCTEST_GLOBAL_NO_WARNINGS(DOCTEST_ANONYMOUS(DOCTEST_ANON_REPORTER_), /* NOLINT(cert-err58-cpp) */ \ + doctest::registerReporter(name, priority, true)) \ + static_assert(true, "") // for registering listeners #define DOCTEST_REGISTER_LISTENER(name, priority, reporter) \ - DOCTEST_GLOBAL_NO_WARNINGS(DOCTEST_ANONYMOUS(_DOCTEST_ANON_REPORTER_)) = \ - doctest::registerReporter(name, priority, false); \ - DOCTEST_GLOBAL_NO_WARNINGS_END() typedef int DOCTEST_ANONYMOUS(_DOCTEST_ANON_FOR_SEMICOLON_) + DOCTEST_GLOBAL_NO_WARNINGS(DOCTEST_ANONYMOUS(DOCTEST_ANON_REPORTER_), /* NOLINT(cert-err58-cpp) */ \ + doctest::registerReporter(name, priority, false)) \ + static_assert(true, "") -// for logging +// clang-format off +// for logging - disabling formatting because it's important to have these on 2 separate lines - see PR #557 #define DOCTEST_INFO(...) \ - DOCTEST_INFO_IMPL(DOCTEST_ANONYMOUS(_DOCTEST_CAPTURE_), DOCTEST_ANONYMOUS(_DOCTEST_CAPTURE_), \ + DOCTEST_INFO_IMPL(DOCTEST_ANONYMOUS(DOCTEST_CAPTURE_), \ + DOCTEST_ANONYMOUS(DOCTEST_CAPTURE_OTHER_), \ __VA_ARGS__) +// clang-format on #define DOCTEST_INFO_IMPL(mb_name, s_name, ...) \ - auto DOCTEST_ANONYMOUS(_DOCTEST_CAPTURE_) = doctest::detail::MakeContextScope( \ + auto DOCTEST_ANONYMOUS(DOCTEST_CAPTURE_) = doctest::detail::MakeContextScope( \ [&](std::ostream* s_name) { \ doctest::detail::MessageBuilder mb_name(__FILE__, __LINE__, doctest::assertType::is_warn); \ mb_name.m_stream = s_name; \ @@ -2051,16 +2316,18 @@ int registerReporter(const char* name, int priority, bool isReporter) { #define DOCTEST_CAPTURE(x) DOCTEST_INFO(#x " := ", x) #define DOCTEST_ADD_AT_IMPL(type, file, line, mb, ...) \ - do { \ + DOCTEST_FUNC_SCOPE_BEGIN { \ doctest::detail::MessageBuilder mb(file, line, doctest::assertType::type); \ mb * __VA_ARGS__; \ - DOCTEST_ASSERT_LOG_AND_REACT(mb); \ - } while(false) + if(mb.log()) \ + DOCTEST_BREAK_INTO_DEBUGGER(); \ + mb.react(); \ + } DOCTEST_FUNC_SCOPE_END // clang-format off -#define DOCTEST_ADD_MESSAGE_AT(file, line, ...) DOCTEST_ADD_AT_IMPL(is_warn, file, line, DOCTEST_ANONYMOUS(_DOCTEST_MESSAGE_), __VA_ARGS__) -#define DOCTEST_ADD_FAIL_CHECK_AT(file, line, ...) DOCTEST_ADD_AT_IMPL(is_check, file, line, DOCTEST_ANONYMOUS(_DOCTEST_MESSAGE_), __VA_ARGS__) -#define DOCTEST_ADD_FAIL_AT(file, line, ...) DOCTEST_ADD_AT_IMPL(is_require, file, line, DOCTEST_ANONYMOUS(_DOCTEST_MESSAGE_), __VA_ARGS__) +#define DOCTEST_ADD_MESSAGE_AT(file, line, ...) DOCTEST_ADD_AT_IMPL(is_warn, file, line, DOCTEST_ANONYMOUS(DOCTEST_MESSAGE_), __VA_ARGS__) +#define DOCTEST_ADD_FAIL_CHECK_AT(file, line, ...) DOCTEST_ADD_AT_IMPL(is_check, file, line, DOCTEST_ANONYMOUS(DOCTEST_MESSAGE_), __VA_ARGS__) +#define DOCTEST_ADD_FAIL_AT(file, line, ...) DOCTEST_ADD_AT_IMPL(is_require, file, line, DOCTEST_ANONYMOUS(DOCTEST_MESSAGE_), __VA_ARGS__) // clang-format on #define DOCTEST_MESSAGE(...) DOCTEST_ADD_MESSAGE_AT(__FILE__, __LINE__, __VA_ARGS__) @@ -2073,18 +2340,37 @@ int registerReporter(const char* name, int priority, bool isReporter) { #define DOCTEST_ASSERT_IMPLEMENT_2(assert_type, ...) \ DOCTEST_CLANG_SUPPRESS_WARNING_WITH_PUSH("-Woverloaded-shift-op-parentheses") \ - doctest::detail::ResultBuilder _DOCTEST_RB(doctest::assertType::assert_type, __FILE__, \ + /* NOLINTNEXTLINE(clang-analyzer-cplusplus.NewDeleteLeaks) */ \ + doctest::detail::ResultBuilder DOCTEST_RB(doctest::assertType::assert_type, __FILE__, \ __LINE__, #__VA_ARGS__); \ - DOCTEST_WRAP_IN_TRY(_DOCTEST_RB.setResult( \ + DOCTEST_WRAP_IN_TRY(DOCTEST_RB.setResult( \ doctest::detail::ExpressionDecomposer(doctest::assertType::assert_type) \ - << __VA_ARGS__)) \ - DOCTEST_ASSERT_LOG_AND_REACT(_DOCTEST_RB) \ + << __VA_ARGS__)) /* NOLINTNEXTLINE(clang-analyzer-cplusplus.NewDeleteLeaks) */ \ + DOCTEST_ASSERT_LOG_REACT_RETURN(DOCTEST_RB) \ DOCTEST_CLANG_SUPPRESS_WARNING_POP #define DOCTEST_ASSERT_IMPLEMENT_1(assert_type, ...) \ - do { \ + DOCTEST_FUNC_SCOPE_BEGIN { \ DOCTEST_ASSERT_IMPLEMENT_2(assert_type, __VA_ARGS__); \ - } while(false) + } DOCTEST_FUNC_SCOPE_END // NOLINT(clang-analyzer-cplusplus.NewDeleteLeaks) + +#define DOCTEST_BINARY_ASSERT(assert_type, comp, ...) \ + DOCTEST_FUNC_SCOPE_BEGIN { \ + doctest::detail::ResultBuilder DOCTEST_RB(doctest::assertType::assert_type, __FILE__, \ + __LINE__, #__VA_ARGS__); \ + DOCTEST_WRAP_IN_TRY( \ + DOCTEST_RB.binary_assert( \ + __VA_ARGS__)) \ + DOCTEST_ASSERT_LOG_REACT_RETURN(DOCTEST_RB); \ + } DOCTEST_FUNC_SCOPE_END + +#define DOCTEST_UNARY_ASSERT(assert_type, ...) \ + DOCTEST_FUNC_SCOPE_BEGIN { \ + doctest::detail::ResultBuilder DOCTEST_RB(doctest::assertType::assert_type, __FILE__, \ + __LINE__, #__VA_ARGS__); \ + DOCTEST_WRAP_IN_TRY(DOCTEST_RB.unary_assert(__VA_ARGS__)) \ + DOCTEST_ASSERT_LOG_REACT_RETURN(DOCTEST_RB); \ + } DOCTEST_FUNC_SCOPE_END #else // DOCTEST_CONFIG_SUPER_FAST_ASSERTS @@ -2098,6 +2384,14 @@ int registerReporter(const char* name, int priority, bool isReporter) { doctest::detail::ExpressionDecomposer(doctest::assertType::assert_type) \ << __VA_ARGS__) DOCTEST_CLANG_SUPPRESS_WARNING_POP +#define DOCTEST_BINARY_ASSERT(assert_type, comparison, ...) \ + doctest::detail::binary_assert( \ + doctest::assertType::assert_type, __FILE__, __LINE__, #__VA_ARGS__, __VA_ARGS__) + +#define DOCTEST_UNARY_ASSERT(assert_type, ...) \ + doctest::detail::unary_assert(doctest::assertType::assert_type, __FILE__, __LINE__, \ + #__VA_ARGS__, __VA_ARGS__) + #endif // DOCTEST_CONFIG_SUPER_FAST_ASSERTS #define DOCTEST_WARN(...) DOCTEST_ASSERT_IMPLEMENT_1(DT_WARN, __VA_ARGS__) @@ -2108,122 +2402,14 @@ int registerReporter(const char* name, int priority, bool isReporter) { #define DOCTEST_REQUIRE_FALSE(...) DOCTEST_ASSERT_IMPLEMENT_1(DT_REQUIRE_FALSE, __VA_ARGS__) // clang-format off -#define DOCTEST_WARN_MESSAGE(cond, ...) do { DOCTEST_INFO(__VA_ARGS__); DOCTEST_ASSERT_IMPLEMENT_2(DT_WARN, cond); } while(false) -#define DOCTEST_CHECK_MESSAGE(cond, ...) do { DOCTEST_INFO(__VA_ARGS__); DOCTEST_ASSERT_IMPLEMENT_2(DT_CHECK, cond); } while(false) -#define DOCTEST_REQUIRE_MESSAGE(cond, ...) do { DOCTEST_INFO(__VA_ARGS__); DOCTEST_ASSERT_IMPLEMENT_2(DT_REQUIRE, cond); } while(false) -#define DOCTEST_WARN_FALSE_MESSAGE(cond, ...) do { DOCTEST_INFO(__VA_ARGS__); DOCTEST_ASSERT_IMPLEMENT_2(DT_WARN_FALSE, cond); } while(false) -#define DOCTEST_CHECK_FALSE_MESSAGE(cond, ...) do { DOCTEST_INFO(__VA_ARGS__); DOCTEST_ASSERT_IMPLEMENT_2(DT_CHECK_FALSE, cond); } while(false) -#define DOCTEST_REQUIRE_FALSE_MESSAGE(cond, ...) do { DOCTEST_INFO(__VA_ARGS__); DOCTEST_ASSERT_IMPLEMENT_2(DT_REQUIRE_FALSE, cond); } while(false) +#define DOCTEST_WARN_MESSAGE(cond, ...) DOCTEST_FUNC_SCOPE_BEGIN { DOCTEST_INFO(__VA_ARGS__); DOCTEST_ASSERT_IMPLEMENT_2(DT_WARN, cond); } DOCTEST_FUNC_SCOPE_END +#define DOCTEST_CHECK_MESSAGE(cond, ...) DOCTEST_FUNC_SCOPE_BEGIN { DOCTEST_INFO(__VA_ARGS__); DOCTEST_ASSERT_IMPLEMENT_2(DT_CHECK, cond); } DOCTEST_FUNC_SCOPE_END +#define DOCTEST_REQUIRE_MESSAGE(cond, ...) DOCTEST_FUNC_SCOPE_BEGIN { DOCTEST_INFO(__VA_ARGS__); DOCTEST_ASSERT_IMPLEMENT_2(DT_REQUIRE, cond); } DOCTEST_FUNC_SCOPE_END +#define DOCTEST_WARN_FALSE_MESSAGE(cond, ...) DOCTEST_FUNC_SCOPE_BEGIN { DOCTEST_INFO(__VA_ARGS__); DOCTEST_ASSERT_IMPLEMENT_2(DT_WARN_FALSE, cond); } DOCTEST_FUNC_SCOPE_END +#define DOCTEST_CHECK_FALSE_MESSAGE(cond, ...) DOCTEST_FUNC_SCOPE_BEGIN { DOCTEST_INFO(__VA_ARGS__); DOCTEST_ASSERT_IMPLEMENT_2(DT_CHECK_FALSE, cond); } DOCTEST_FUNC_SCOPE_END +#define DOCTEST_REQUIRE_FALSE_MESSAGE(cond, ...) DOCTEST_FUNC_SCOPE_BEGIN { DOCTEST_INFO(__VA_ARGS__); DOCTEST_ASSERT_IMPLEMENT_2(DT_REQUIRE_FALSE, cond); } DOCTEST_FUNC_SCOPE_END // clang-format on -#define DOCTEST_ASSERT_THROWS_AS(expr, assert_type, message, ...) \ - do { \ - if(!doctest::getContextOptions()->no_throw) { \ - doctest::detail::ResultBuilder _DOCTEST_RB(doctest::assertType::assert_type, __FILE__, \ - __LINE__, #expr, #__VA_ARGS__, message); \ - try { \ - DOCTEST_CAST_TO_VOID(expr) \ - } catch(const typename doctest::detail::remove_const< \ - typename doctest::detail::remove_reference<__VA_ARGS__>::type>::type&) { \ - _DOCTEST_RB.translateException(); \ - _DOCTEST_RB.m_threw_as = true; \ - } catch(...) { _DOCTEST_RB.translateException(); } \ - DOCTEST_ASSERT_LOG_AND_REACT(_DOCTEST_RB); \ - } \ - } while(false) - -#define DOCTEST_ASSERT_THROWS_WITH(expr, expr_str, assert_type, ...) \ - do { \ - if(!doctest::getContextOptions()->no_throw) { \ - doctest::detail::ResultBuilder _DOCTEST_RB(doctest::assertType::assert_type, __FILE__, \ - __LINE__, expr_str, "", __VA_ARGS__); \ - try { \ - DOCTEST_CAST_TO_VOID(expr) \ - } catch(...) { _DOCTEST_RB.translateException(); } \ - DOCTEST_ASSERT_LOG_AND_REACT(_DOCTEST_RB); \ - } \ - } while(false) - -#define DOCTEST_ASSERT_NOTHROW(assert_type, ...) \ - do { \ - doctest::detail::ResultBuilder _DOCTEST_RB(doctest::assertType::assert_type, __FILE__, \ - __LINE__, #__VA_ARGS__); \ - try { \ - DOCTEST_CAST_TO_VOID(__VA_ARGS__) \ - } catch(...) { _DOCTEST_RB.translateException(); } \ - DOCTEST_ASSERT_LOG_AND_REACT(_DOCTEST_RB); \ - } while(false) - -// clang-format off -#define DOCTEST_WARN_THROWS(...) DOCTEST_ASSERT_THROWS_WITH((__VA_ARGS__), #__VA_ARGS__, DT_WARN_THROWS, "") -#define DOCTEST_CHECK_THROWS(...) DOCTEST_ASSERT_THROWS_WITH((__VA_ARGS__), #__VA_ARGS__, DT_CHECK_THROWS, "") -#define DOCTEST_REQUIRE_THROWS(...) DOCTEST_ASSERT_THROWS_WITH((__VA_ARGS__), #__VA_ARGS__, DT_REQUIRE_THROWS, "") - -#define DOCTEST_WARN_THROWS_AS(expr, ...) DOCTEST_ASSERT_THROWS_AS(expr, DT_WARN_THROWS_AS, "", __VA_ARGS__) -#define DOCTEST_CHECK_THROWS_AS(expr, ...) DOCTEST_ASSERT_THROWS_AS(expr, DT_CHECK_THROWS_AS, "", __VA_ARGS__) -#define DOCTEST_REQUIRE_THROWS_AS(expr, ...) DOCTEST_ASSERT_THROWS_AS(expr, DT_REQUIRE_THROWS_AS, "", __VA_ARGS__) - -#define DOCTEST_WARN_THROWS_WITH(expr, ...) DOCTEST_ASSERT_THROWS_WITH(expr, #expr, DT_WARN_THROWS_WITH, __VA_ARGS__) -#define DOCTEST_CHECK_THROWS_WITH(expr, ...) DOCTEST_ASSERT_THROWS_WITH(expr, #expr, DT_CHECK_THROWS_WITH, __VA_ARGS__) -#define DOCTEST_REQUIRE_THROWS_WITH(expr, ...) DOCTEST_ASSERT_THROWS_WITH(expr, #expr, DT_REQUIRE_THROWS_WITH, __VA_ARGS__) - -#define DOCTEST_WARN_THROWS_WITH_AS(expr, message, ...) DOCTEST_ASSERT_THROWS_AS(expr, DT_WARN_THROWS_WITH_AS, message, __VA_ARGS__) -#define DOCTEST_CHECK_THROWS_WITH_AS(expr, message, ...) DOCTEST_ASSERT_THROWS_AS(expr, DT_CHECK_THROWS_WITH_AS, message, __VA_ARGS__) -#define DOCTEST_REQUIRE_THROWS_WITH_AS(expr, message, ...) DOCTEST_ASSERT_THROWS_AS(expr, DT_REQUIRE_THROWS_WITH_AS, message, __VA_ARGS__) - -#define DOCTEST_WARN_NOTHROW(...) DOCTEST_ASSERT_NOTHROW(DT_WARN_NOTHROW, __VA_ARGS__) -#define DOCTEST_CHECK_NOTHROW(...) DOCTEST_ASSERT_NOTHROW(DT_CHECK_NOTHROW, __VA_ARGS__) -#define DOCTEST_REQUIRE_NOTHROW(...) DOCTEST_ASSERT_NOTHROW(DT_REQUIRE_NOTHROW, __VA_ARGS__) - -#define DOCTEST_WARN_THROWS_MESSAGE(expr, ...) do { DOCTEST_INFO(__VA_ARGS__); DOCTEST_WARN_THROWS(expr); } while(false) -#define DOCTEST_CHECK_THROWS_MESSAGE(expr, ...) do { DOCTEST_INFO(__VA_ARGS__); DOCTEST_CHECK_THROWS(expr); } while(false) -#define DOCTEST_REQUIRE_THROWS_MESSAGE(expr, ...) do { DOCTEST_INFO(__VA_ARGS__); DOCTEST_REQUIRE_THROWS(expr); } while(false) -#define DOCTEST_WARN_THROWS_AS_MESSAGE(expr, ex, ...) do { DOCTEST_INFO(__VA_ARGS__); DOCTEST_WARN_THROWS_AS(expr, ex); } while(false) -#define DOCTEST_CHECK_THROWS_AS_MESSAGE(expr, ex, ...) do { DOCTEST_INFO(__VA_ARGS__); DOCTEST_CHECK_THROWS_AS(expr, ex); } while(false) -#define DOCTEST_REQUIRE_THROWS_AS_MESSAGE(expr, ex, ...) do { DOCTEST_INFO(__VA_ARGS__); DOCTEST_REQUIRE_THROWS_AS(expr, ex); } while(false) -#define DOCTEST_WARN_THROWS_WITH_MESSAGE(expr, with, ...) do { DOCTEST_INFO(__VA_ARGS__); DOCTEST_WARN_THROWS_WITH(expr, with); } while(false) -#define DOCTEST_CHECK_THROWS_WITH_MESSAGE(expr, with, ...) do { DOCTEST_INFO(__VA_ARGS__); DOCTEST_CHECK_THROWS_WITH(expr, with); } while(false) -#define DOCTEST_REQUIRE_THROWS_WITH_MESSAGE(expr, with, ...) do { DOCTEST_INFO(__VA_ARGS__); DOCTEST_REQUIRE_THROWS_WITH(expr, with); } while(false) -#define DOCTEST_WARN_THROWS_WITH_AS_MESSAGE(expr, with, ex, ...) do { DOCTEST_INFO(__VA_ARGS__); DOCTEST_WARN_THROWS_WITH_AS(expr, with, ex); } while(false) -#define DOCTEST_CHECK_THROWS_WITH_AS_MESSAGE(expr, with, ex, ...) do { DOCTEST_INFO(__VA_ARGS__); DOCTEST_CHECK_THROWS_WITH_AS(expr, with, ex); } while(false) -#define DOCTEST_REQUIRE_THROWS_WITH_AS_MESSAGE(expr, with, ex, ...) do { DOCTEST_INFO(__VA_ARGS__); DOCTEST_REQUIRE_THROWS_WITH_AS(expr, with, ex); } while(false) -#define DOCTEST_WARN_NOTHROW_MESSAGE(expr, ...) do { DOCTEST_INFO(__VA_ARGS__); DOCTEST_WARN_NOTHROW(expr); } while(false) -#define DOCTEST_CHECK_NOTHROW_MESSAGE(expr, ...) do { DOCTEST_INFO(__VA_ARGS__); DOCTEST_CHECK_NOTHROW(expr); } while(false) -#define DOCTEST_REQUIRE_NOTHROW_MESSAGE(expr, ...) do { DOCTEST_INFO(__VA_ARGS__); DOCTEST_REQUIRE_NOTHROW(expr); } while(false) -// clang-format on - -#ifndef DOCTEST_CONFIG_SUPER_FAST_ASSERTS - -#define DOCTEST_BINARY_ASSERT(assert_type, comp, ...) \ - do { \ - doctest::detail::ResultBuilder _DOCTEST_RB(doctest::assertType::assert_type, __FILE__, \ - __LINE__, #__VA_ARGS__); \ - DOCTEST_WRAP_IN_TRY( \ - _DOCTEST_RB.binary_assert( \ - __VA_ARGS__)) \ - DOCTEST_ASSERT_LOG_AND_REACT(_DOCTEST_RB); \ - } while(false) - -#define DOCTEST_UNARY_ASSERT(assert_type, ...) \ - do { \ - doctest::detail::ResultBuilder _DOCTEST_RB(doctest::assertType::assert_type, __FILE__, \ - __LINE__, #__VA_ARGS__); \ - DOCTEST_WRAP_IN_TRY(_DOCTEST_RB.unary_assert(__VA_ARGS__)) \ - DOCTEST_ASSERT_LOG_AND_REACT(_DOCTEST_RB); \ - } while(false) - -#else // DOCTEST_CONFIG_SUPER_FAST_ASSERTS - -#define DOCTEST_BINARY_ASSERT(assert_type, comparison, ...) \ - doctest::detail::binary_assert( \ - doctest::assertType::assert_type, __FILE__, __LINE__, #__VA_ARGS__, __VA_ARGS__) - -#define DOCTEST_UNARY_ASSERT(assert_type, ...) \ - doctest::detail::unary_assert(doctest::assertType::assert_type, __FILE__, __LINE__, \ - #__VA_ARGS__, __VA_ARGS__) - -#endif // DOCTEST_CONFIG_SUPER_FAST_ASSERTS - #define DOCTEST_WARN_EQ(...) DOCTEST_BINARY_ASSERT(DT_WARN_EQ, eq, __VA_ARGS__) #define DOCTEST_CHECK_EQ(...) DOCTEST_BINARY_ASSERT(DT_CHECK_EQ, eq, __VA_ARGS__) #define DOCTEST_REQUIRE_EQ(...) DOCTEST_BINARY_ASSERT(DT_REQUIRE_EQ, eq, __VA_ARGS__) @@ -2250,75 +2436,350 @@ int registerReporter(const char* name, int priority, bool isReporter) { #define DOCTEST_CHECK_UNARY_FALSE(...) DOCTEST_UNARY_ASSERT(DT_CHECK_UNARY_FALSE, __VA_ARGS__) #define DOCTEST_REQUIRE_UNARY_FALSE(...) DOCTEST_UNARY_ASSERT(DT_REQUIRE_UNARY_FALSE, __VA_ARGS__) +#ifndef DOCTEST_CONFIG_NO_EXCEPTIONS + +#define DOCTEST_ASSERT_THROWS_AS(expr, assert_type, message, ...) \ + DOCTEST_FUNC_SCOPE_BEGIN { \ + if(!doctest::getContextOptions()->no_throw) { \ + doctest::detail::ResultBuilder DOCTEST_RB(doctest::assertType::assert_type, __FILE__, \ + __LINE__, #expr, #__VA_ARGS__, message); \ + try { \ + DOCTEST_CAST_TO_VOID(expr) \ + } catch(const typename doctest::detail::types::remove_const< \ + typename doctest::detail::types::remove_reference<__VA_ARGS__>::type>::type&) {\ + DOCTEST_RB.translateException(); \ + DOCTEST_RB.m_threw_as = true; \ + } catch(...) { DOCTEST_RB.translateException(); } \ + DOCTEST_ASSERT_LOG_REACT_RETURN(DOCTEST_RB); \ + } else { /* NOLINT(*-else-after-return) */ \ + DOCTEST_FUNC_SCOPE_RET(false); \ + } \ + } DOCTEST_FUNC_SCOPE_END + +#define DOCTEST_ASSERT_THROWS_WITH(expr, expr_str, assert_type, ...) \ + DOCTEST_FUNC_SCOPE_BEGIN { \ + if(!doctest::getContextOptions()->no_throw) { \ + doctest::detail::ResultBuilder DOCTEST_RB(doctest::assertType::assert_type, __FILE__, \ + __LINE__, expr_str, "", __VA_ARGS__); \ + try { \ + DOCTEST_CAST_TO_VOID(expr) \ + } catch(...) { DOCTEST_RB.translateException(); } \ + DOCTEST_ASSERT_LOG_REACT_RETURN(DOCTEST_RB); \ + } else { /* NOLINT(*-else-after-return) */ \ + DOCTEST_FUNC_SCOPE_RET(false); \ + } \ + } DOCTEST_FUNC_SCOPE_END + +#define DOCTEST_ASSERT_NOTHROW(assert_type, ...) \ + DOCTEST_FUNC_SCOPE_BEGIN { \ + doctest::detail::ResultBuilder DOCTEST_RB(doctest::assertType::assert_type, __FILE__, \ + __LINE__, #__VA_ARGS__); \ + try { \ + DOCTEST_CAST_TO_VOID(__VA_ARGS__) \ + } catch(...) { DOCTEST_RB.translateException(); } \ + DOCTEST_ASSERT_LOG_REACT_RETURN(DOCTEST_RB); \ + } DOCTEST_FUNC_SCOPE_END + +// clang-format off +#define DOCTEST_WARN_THROWS(...) DOCTEST_ASSERT_THROWS_WITH((__VA_ARGS__), #__VA_ARGS__, DT_WARN_THROWS, "") +#define DOCTEST_CHECK_THROWS(...) DOCTEST_ASSERT_THROWS_WITH((__VA_ARGS__), #__VA_ARGS__, DT_CHECK_THROWS, "") +#define DOCTEST_REQUIRE_THROWS(...) DOCTEST_ASSERT_THROWS_WITH((__VA_ARGS__), #__VA_ARGS__, DT_REQUIRE_THROWS, "") + +#define DOCTEST_WARN_THROWS_AS(expr, ...) DOCTEST_ASSERT_THROWS_AS(expr, DT_WARN_THROWS_AS, "", __VA_ARGS__) +#define DOCTEST_CHECK_THROWS_AS(expr, ...) DOCTEST_ASSERT_THROWS_AS(expr, DT_CHECK_THROWS_AS, "", __VA_ARGS__) +#define DOCTEST_REQUIRE_THROWS_AS(expr, ...) DOCTEST_ASSERT_THROWS_AS(expr, DT_REQUIRE_THROWS_AS, "", __VA_ARGS__) + +#define DOCTEST_WARN_THROWS_WITH(expr, ...) DOCTEST_ASSERT_THROWS_WITH(expr, #expr, DT_WARN_THROWS_WITH, __VA_ARGS__) +#define DOCTEST_CHECK_THROWS_WITH(expr, ...) DOCTEST_ASSERT_THROWS_WITH(expr, #expr, DT_CHECK_THROWS_WITH, __VA_ARGS__) +#define DOCTEST_REQUIRE_THROWS_WITH(expr, ...) DOCTEST_ASSERT_THROWS_WITH(expr, #expr, DT_REQUIRE_THROWS_WITH, __VA_ARGS__) + +#define DOCTEST_WARN_THROWS_WITH_AS(expr, message, ...) DOCTEST_ASSERT_THROWS_AS(expr, DT_WARN_THROWS_WITH_AS, message, __VA_ARGS__) +#define DOCTEST_CHECK_THROWS_WITH_AS(expr, message, ...) DOCTEST_ASSERT_THROWS_AS(expr, DT_CHECK_THROWS_WITH_AS, message, __VA_ARGS__) +#define DOCTEST_REQUIRE_THROWS_WITH_AS(expr, message, ...) DOCTEST_ASSERT_THROWS_AS(expr, DT_REQUIRE_THROWS_WITH_AS, message, __VA_ARGS__) + +#define DOCTEST_WARN_NOTHROW(...) DOCTEST_ASSERT_NOTHROW(DT_WARN_NOTHROW, __VA_ARGS__) +#define DOCTEST_CHECK_NOTHROW(...) DOCTEST_ASSERT_NOTHROW(DT_CHECK_NOTHROW, __VA_ARGS__) +#define DOCTEST_REQUIRE_NOTHROW(...) DOCTEST_ASSERT_NOTHROW(DT_REQUIRE_NOTHROW, __VA_ARGS__) + +#define DOCTEST_WARN_THROWS_MESSAGE(expr, ...) DOCTEST_FUNC_SCOPE_BEGIN { DOCTEST_INFO(__VA_ARGS__); DOCTEST_WARN_THROWS(expr); } DOCTEST_FUNC_SCOPE_END +#define DOCTEST_CHECK_THROWS_MESSAGE(expr, ...) DOCTEST_FUNC_SCOPE_BEGIN { DOCTEST_INFO(__VA_ARGS__); DOCTEST_CHECK_THROWS(expr); } DOCTEST_FUNC_SCOPE_END +#define DOCTEST_REQUIRE_THROWS_MESSAGE(expr, ...) DOCTEST_FUNC_SCOPE_BEGIN { DOCTEST_INFO(__VA_ARGS__); DOCTEST_REQUIRE_THROWS(expr); } DOCTEST_FUNC_SCOPE_END +#define DOCTEST_WARN_THROWS_AS_MESSAGE(expr, ex, ...) DOCTEST_FUNC_SCOPE_BEGIN { DOCTEST_INFO(__VA_ARGS__); DOCTEST_WARN_THROWS_AS(expr, ex); } DOCTEST_FUNC_SCOPE_END +#define DOCTEST_CHECK_THROWS_AS_MESSAGE(expr, ex, ...) DOCTEST_FUNC_SCOPE_BEGIN { DOCTEST_INFO(__VA_ARGS__); DOCTEST_CHECK_THROWS_AS(expr, ex); } DOCTEST_FUNC_SCOPE_END +#define DOCTEST_REQUIRE_THROWS_AS_MESSAGE(expr, ex, ...) DOCTEST_FUNC_SCOPE_BEGIN { DOCTEST_INFO(__VA_ARGS__); DOCTEST_REQUIRE_THROWS_AS(expr, ex); } DOCTEST_FUNC_SCOPE_END +#define DOCTEST_WARN_THROWS_WITH_MESSAGE(expr, with, ...) DOCTEST_FUNC_SCOPE_BEGIN { DOCTEST_INFO(__VA_ARGS__); DOCTEST_WARN_THROWS_WITH(expr, with); } DOCTEST_FUNC_SCOPE_END +#define DOCTEST_CHECK_THROWS_WITH_MESSAGE(expr, with, ...) DOCTEST_FUNC_SCOPE_BEGIN { DOCTEST_INFO(__VA_ARGS__); DOCTEST_CHECK_THROWS_WITH(expr, with); } DOCTEST_FUNC_SCOPE_END +#define DOCTEST_REQUIRE_THROWS_WITH_MESSAGE(expr, with, ...) DOCTEST_FUNC_SCOPE_BEGIN { DOCTEST_INFO(__VA_ARGS__); DOCTEST_REQUIRE_THROWS_WITH(expr, with); } DOCTEST_FUNC_SCOPE_END +#define DOCTEST_WARN_THROWS_WITH_AS_MESSAGE(expr, with, ex, ...) DOCTEST_FUNC_SCOPE_BEGIN { DOCTEST_INFO(__VA_ARGS__); DOCTEST_WARN_THROWS_WITH_AS(expr, with, ex); } DOCTEST_FUNC_SCOPE_END +#define DOCTEST_CHECK_THROWS_WITH_AS_MESSAGE(expr, with, ex, ...) DOCTEST_FUNC_SCOPE_BEGIN { DOCTEST_INFO(__VA_ARGS__); DOCTEST_CHECK_THROWS_WITH_AS(expr, with, ex); } DOCTEST_FUNC_SCOPE_END +#define DOCTEST_REQUIRE_THROWS_WITH_AS_MESSAGE(expr, with, ex, ...) DOCTEST_FUNC_SCOPE_BEGIN { DOCTEST_INFO(__VA_ARGS__); DOCTEST_REQUIRE_THROWS_WITH_AS(expr, with, ex); } DOCTEST_FUNC_SCOPE_END +#define DOCTEST_WARN_NOTHROW_MESSAGE(expr, ...) DOCTEST_FUNC_SCOPE_BEGIN { DOCTEST_INFO(__VA_ARGS__); DOCTEST_WARN_NOTHROW(expr); } DOCTEST_FUNC_SCOPE_END +#define DOCTEST_CHECK_NOTHROW_MESSAGE(expr, ...) DOCTEST_FUNC_SCOPE_BEGIN { DOCTEST_INFO(__VA_ARGS__); DOCTEST_CHECK_NOTHROW(expr); } DOCTEST_FUNC_SCOPE_END +#define DOCTEST_REQUIRE_NOTHROW_MESSAGE(expr, ...) DOCTEST_FUNC_SCOPE_BEGIN { DOCTEST_INFO(__VA_ARGS__); DOCTEST_REQUIRE_NOTHROW(expr); } DOCTEST_FUNC_SCOPE_END +// clang-format on + +#endif // DOCTEST_CONFIG_NO_EXCEPTIONS + +// ================================================================================================= +// == WHAT FOLLOWS IS VERSIONS OF THE MACROS THAT DO NOT DO ANY REGISTERING! == +// == THIS CAN BE ENABLED BY DEFINING DOCTEST_CONFIG_DISABLE GLOBALLY! == +// ================================================================================================= +#else // DOCTEST_CONFIG_DISABLE + +#define DOCTEST_IMPLEMENT_FIXTURE(der, base, func, name) \ + namespace /* NOLINT */ { \ + template \ + struct der : public base \ + { void f(); }; \ + } \ + template \ + inline void der::f() + +#define DOCTEST_CREATE_AND_REGISTER_FUNCTION(f, name) \ + template \ + static inline void f() + +// for registering tests +#define DOCTEST_TEST_CASE(name) \ + DOCTEST_CREATE_AND_REGISTER_FUNCTION(DOCTEST_ANONYMOUS(DOCTEST_ANON_FUNC_), name) + +// for registering tests in classes +#define DOCTEST_TEST_CASE_CLASS(name) \ + DOCTEST_CREATE_AND_REGISTER_FUNCTION(DOCTEST_ANONYMOUS(DOCTEST_ANON_FUNC_), name) + +// for registering tests with a fixture +#define DOCTEST_TEST_CASE_FIXTURE(x, name) \ + DOCTEST_IMPLEMENT_FIXTURE(DOCTEST_ANONYMOUS(DOCTEST_ANON_CLASS_), x, \ + DOCTEST_ANONYMOUS(DOCTEST_ANON_FUNC_), name) + +// for converting types to strings without the header and demangling +#define DOCTEST_TYPE_TO_STRING_AS(str, ...) static_assert(true, "") +#define DOCTEST_TYPE_TO_STRING(...) static_assert(true, "") + +// for typed tests +#define DOCTEST_TEST_CASE_TEMPLATE(name, type, ...) \ + template \ + inline void DOCTEST_ANONYMOUS(DOCTEST_ANON_TMP_)() + +#define DOCTEST_TEST_CASE_TEMPLATE_DEFINE(name, type, id) \ + template \ + inline void DOCTEST_ANONYMOUS(DOCTEST_ANON_TMP_)() + +#define DOCTEST_TEST_CASE_TEMPLATE_INVOKE(id, ...) static_assert(true, "") +#define DOCTEST_TEST_CASE_TEMPLATE_APPLY(id, ...) static_assert(true, "") + +// for subcases +#define DOCTEST_SUBCASE(name) + +// for a testsuite block +#define DOCTEST_TEST_SUITE(name) namespace // NOLINT + +// for starting a testsuite block +#define DOCTEST_TEST_SUITE_BEGIN(name) static_assert(true, "") + +// for ending a testsuite block +#define DOCTEST_TEST_SUITE_END using DOCTEST_ANONYMOUS(DOCTEST_ANON_FOR_SEMICOLON_) = int + +#define DOCTEST_REGISTER_EXCEPTION_TRANSLATOR(signature) \ + template \ + static inline doctest::String DOCTEST_ANONYMOUS(DOCTEST_ANON_TRANSLATOR_)(signature) + +#define DOCTEST_REGISTER_REPORTER(name, priority, reporter) +#define DOCTEST_REGISTER_LISTENER(name, priority, reporter) + +#define DOCTEST_INFO(...) (static_cast(0)) +#define DOCTEST_CAPTURE(x) (static_cast(0)) +#define DOCTEST_ADD_MESSAGE_AT(file, line, ...) (static_cast(0)) +#define DOCTEST_ADD_FAIL_CHECK_AT(file, line, ...) (static_cast(0)) +#define DOCTEST_ADD_FAIL_AT(file, line, ...) (static_cast(0)) +#define DOCTEST_MESSAGE(...) (static_cast(0)) +#define DOCTEST_FAIL_CHECK(...) (static_cast(0)) +#define DOCTEST_FAIL(...) (static_cast(0)) + +#if defined(DOCTEST_CONFIG_EVALUATE_ASSERTS_EVEN_WHEN_DISABLED) \ + && defined(DOCTEST_CONFIG_ASSERTS_RETURN_VALUES) + +#define DOCTEST_WARN(...) [&] { return __VA_ARGS__; }() +#define DOCTEST_CHECK(...) [&] { return __VA_ARGS__; }() +#define DOCTEST_REQUIRE(...) [&] { return __VA_ARGS__; }() +#define DOCTEST_WARN_FALSE(...) [&] { return !(__VA_ARGS__); }() +#define DOCTEST_CHECK_FALSE(...) [&] { return !(__VA_ARGS__); }() +#define DOCTEST_REQUIRE_FALSE(...) [&] { return !(__VA_ARGS__); }() + +#define DOCTEST_WARN_MESSAGE(cond, ...) [&] { return cond; }() +#define DOCTEST_CHECK_MESSAGE(cond, ...) [&] { return cond; }() +#define DOCTEST_REQUIRE_MESSAGE(cond, ...) [&] { return cond; }() +#define DOCTEST_WARN_FALSE_MESSAGE(cond, ...) [&] { return !(cond); }() +#define DOCTEST_CHECK_FALSE_MESSAGE(cond, ...) [&] { return !(cond); }() +#define DOCTEST_REQUIRE_FALSE_MESSAGE(cond, ...) [&] { return !(cond); }() + +namespace doctest { +namespace detail { +#define DOCTEST_RELATIONAL_OP(name, op) \ + template \ + bool name(const DOCTEST_REF_WRAP(L) lhs, const DOCTEST_REF_WRAP(R) rhs) { return lhs op rhs; } + + DOCTEST_RELATIONAL_OP(eq, ==) + DOCTEST_RELATIONAL_OP(ne, !=) + DOCTEST_RELATIONAL_OP(lt, <) + DOCTEST_RELATIONAL_OP(gt, >) + DOCTEST_RELATIONAL_OP(le, <=) + DOCTEST_RELATIONAL_OP(ge, >=) +} // namespace detail +} // namespace doctest + +#define DOCTEST_WARN_EQ(...) [&] { return doctest::detail::eq(__VA_ARGS__); }() +#define DOCTEST_CHECK_EQ(...) [&] { return doctest::detail::eq(__VA_ARGS__); }() +#define DOCTEST_REQUIRE_EQ(...) [&] { return doctest::detail::eq(__VA_ARGS__); }() +#define DOCTEST_WARN_NE(...) [&] { return doctest::detail::ne(__VA_ARGS__); }() +#define DOCTEST_CHECK_NE(...) [&] { return doctest::detail::ne(__VA_ARGS__); }() +#define DOCTEST_REQUIRE_NE(...) [&] { return doctest::detail::ne(__VA_ARGS__); }() +#define DOCTEST_WARN_LT(...) [&] { return doctest::detail::lt(__VA_ARGS__); }() +#define DOCTEST_CHECK_LT(...) [&] { return doctest::detail::lt(__VA_ARGS__); }() +#define DOCTEST_REQUIRE_LT(...) [&] { return doctest::detail::lt(__VA_ARGS__); }() +#define DOCTEST_WARN_GT(...) [&] { return doctest::detail::gt(__VA_ARGS__); }() +#define DOCTEST_CHECK_GT(...) [&] { return doctest::detail::gt(__VA_ARGS__); }() +#define DOCTEST_REQUIRE_GT(...) [&] { return doctest::detail::gt(__VA_ARGS__); }() +#define DOCTEST_WARN_LE(...) [&] { return doctest::detail::le(__VA_ARGS__); }() +#define DOCTEST_CHECK_LE(...) [&] { return doctest::detail::le(__VA_ARGS__); }() +#define DOCTEST_REQUIRE_LE(...) [&] { return doctest::detail::le(__VA_ARGS__); }() +#define DOCTEST_WARN_GE(...) [&] { return doctest::detail::ge(__VA_ARGS__); }() +#define DOCTEST_CHECK_GE(...) [&] { return doctest::detail::ge(__VA_ARGS__); }() +#define DOCTEST_REQUIRE_GE(...) [&] { return doctest::detail::ge(__VA_ARGS__); }() +#define DOCTEST_WARN_UNARY(...) [&] { return __VA_ARGS__; }() +#define DOCTEST_CHECK_UNARY(...) [&] { return __VA_ARGS__; }() +#define DOCTEST_REQUIRE_UNARY(...) [&] { return __VA_ARGS__; }() +#define DOCTEST_WARN_UNARY_FALSE(...) [&] { return !(__VA_ARGS__); }() +#define DOCTEST_CHECK_UNARY_FALSE(...) [&] { return !(__VA_ARGS__); }() +#define DOCTEST_REQUIRE_UNARY_FALSE(...) [&] { return !(__VA_ARGS__); }() + +#ifndef DOCTEST_CONFIG_NO_EXCEPTIONS + +#define DOCTEST_WARN_THROWS_WITH(expr, with, ...) [] { static_assert(false, "Exception translation is not available when doctest is disabled."); return false; }() +#define DOCTEST_CHECK_THROWS_WITH(expr, with, ...) DOCTEST_WARN_THROWS_WITH(,,) +#define DOCTEST_REQUIRE_THROWS_WITH(expr, with, ...) DOCTEST_WARN_THROWS_WITH(,,) +#define DOCTEST_WARN_THROWS_WITH_AS(expr, with, ex, ...) DOCTEST_WARN_THROWS_WITH(,,) +#define DOCTEST_CHECK_THROWS_WITH_AS(expr, with, ex, ...) DOCTEST_WARN_THROWS_WITH(,,) +#define DOCTEST_REQUIRE_THROWS_WITH_AS(expr, with, ex, ...) DOCTEST_WARN_THROWS_WITH(,,) + +#define DOCTEST_WARN_THROWS_WITH_MESSAGE(expr, with, ...) DOCTEST_WARN_THROWS_WITH(,,) +#define DOCTEST_CHECK_THROWS_WITH_MESSAGE(expr, with, ...) DOCTEST_WARN_THROWS_WITH(,,) +#define DOCTEST_REQUIRE_THROWS_WITH_MESSAGE(expr, with, ...) DOCTEST_WARN_THROWS_WITH(,,) +#define DOCTEST_WARN_THROWS_WITH_AS_MESSAGE(expr, with, ex, ...) DOCTEST_WARN_THROWS_WITH(,,) +#define DOCTEST_CHECK_THROWS_WITH_AS_MESSAGE(expr, with, ex, ...) DOCTEST_WARN_THROWS_WITH(,,) +#define DOCTEST_REQUIRE_THROWS_WITH_AS_MESSAGE(expr, with, ex, ...) DOCTEST_WARN_THROWS_WITH(,,) + +#define DOCTEST_WARN_THROWS(...) [&] { try { __VA_ARGS__; return false; } catch (...) { return true; } }() +#define DOCTEST_CHECK_THROWS(...) [&] { try { __VA_ARGS__; return false; } catch (...) { return true; } }() +#define DOCTEST_REQUIRE_THROWS(...) [&] { try { __VA_ARGS__; return false; } catch (...) { return true; } }() +#define DOCTEST_WARN_THROWS_AS(expr, ...) [&] { try { expr; } catch (__VA_ARGS__) { return true; } catch (...) { } return false; }() +#define DOCTEST_CHECK_THROWS_AS(expr, ...) [&] { try { expr; } catch (__VA_ARGS__) { return true; } catch (...) { } return false; }() +#define DOCTEST_REQUIRE_THROWS_AS(expr, ...) [&] { try { expr; } catch (__VA_ARGS__) { return true; } catch (...) { } return false; }() +#define DOCTEST_WARN_NOTHROW(...) [&] { try { __VA_ARGS__; return true; } catch (...) { return false; } }() +#define DOCTEST_CHECK_NOTHROW(...) [&] { try { __VA_ARGS__; return true; } catch (...) { return false; } }() +#define DOCTEST_REQUIRE_NOTHROW(...) [&] { try { __VA_ARGS__; return true; } catch (...) { return false; } }() + +#define DOCTEST_WARN_THROWS_MESSAGE(expr, ...) [&] { try { __VA_ARGS__; return false; } catch (...) { return true; } }() +#define DOCTEST_CHECK_THROWS_MESSAGE(expr, ...) [&] { try { __VA_ARGS__; return false; } catch (...) { return true; } }() +#define DOCTEST_REQUIRE_THROWS_MESSAGE(expr, ...) [&] { try { __VA_ARGS__; return false; } catch (...) { return true; } }() +#define DOCTEST_WARN_THROWS_AS_MESSAGE(expr, ex, ...) [&] { try { expr; } catch (__VA_ARGS__) { return true; } catch (...) { } return false; }() +#define DOCTEST_CHECK_THROWS_AS_MESSAGE(expr, ex, ...) [&] { try { expr; } catch (__VA_ARGS__) { return true; } catch (...) { } return false; }() +#define DOCTEST_REQUIRE_THROWS_AS_MESSAGE(expr, ex, ...) [&] { try { expr; } catch (__VA_ARGS__) { return true; } catch (...) { } return false; }() +#define DOCTEST_WARN_NOTHROW_MESSAGE(expr, ...) [&] { try { __VA_ARGS__; return true; } catch (...) { return false; } }() +#define DOCTEST_CHECK_NOTHROW_MESSAGE(expr, ...) [&] { try { __VA_ARGS__; return true; } catch (...) { return false; } }() +#define DOCTEST_REQUIRE_NOTHROW_MESSAGE(expr, ...) [&] { try { __VA_ARGS__; return true; } catch (...) { return false; } }() + +#endif // DOCTEST_CONFIG_NO_EXCEPTIONS + +#else // DOCTEST_CONFIG_EVALUATE_ASSERTS_EVEN_WHEN_DISABLED + +#define DOCTEST_WARN(...) DOCTEST_FUNC_EMPTY +#define DOCTEST_CHECK(...) DOCTEST_FUNC_EMPTY +#define DOCTEST_REQUIRE(...) DOCTEST_FUNC_EMPTY +#define DOCTEST_WARN_FALSE(...) DOCTEST_FUNC_EMPTY +#define DOCTEST_CHECK_FALSE(...) DOCTEST_FUNC_EMPTY +#define DOCTEST_REQUIRE_FALSE(...) DOCTEST_FUNC_EMPTY + +#define DOCTEST_WARN_MESSAGE(cond, ...) DOCTEST_FUNC_EMPTY +#define DOCTEST_CHECK_MESSAGE(cond, ...) DOCTEST_FUNC_EMPTY +#define DOCTEST_REQUIRE_MESSAGE(cond, ...) DOCTEST_FUNC_EMPTY +#define DOCTEST_WARN_FALSE_MESSAGE(cond, ...) DOCTEST_FUNC_EMPTY +#define DOCTEST_CHECK_FALSE_MESSAGE(cond, ...) DOCTEST_FUNC_EMPTY +#define DOCTEST_REQUIRE_FALSE_MESSAGE(cond, ...) DOCTEST_FUNC_EMPTY + +#define DOCTEST_WARN_EQ(...) DOCTEST_FUNC_EMPTY +#define DOCTEST_CHECK_EQ(...) DOCTEST_FUNC_EMPTY +#define DOCTEST_REQUIRE_EQ(...) DOCTEST_FUNC_EMPTY +#define DOCTEST_WARN_NE(...) DOCTEST_FUNC_EMPTY +#define DOCTEST_CHECK_NE(...) DOCTEST_FUNC_EMPTY +#define DOCTEST_REQUIRE_NE(...) DOCTEST_FUNC_EMPTY +#define DOCTEST_WARN_GT(...) DOCTEST_FUNC_EMPTY +#define DOCTEST_CHECK_GT(...) DOCTEST_FUNC_EMPTY +#define DOCTEST_REQUIRE_GT(...) DOCTEST_FUNC_EMPTY +#define DOCTEST_WARN_LT(...) DOCTEST_FUNC_EMPTY +#define DOCTEST_CHECK_LT(...) DOCTEST_FUNC_EMPTY +#define DOCTEST_REQUIRE_LT(...) DOCTEST_FUNC_EMPTY +#define DOCTEST_WARN_GE(...) DOCTEST_FUNC_EMPTY +#define DOCTEST_CHECK_GE(...) DOCTEST_FUNC_EMPTY +#define DOCTEST_REQUIRE_GE(...) DOCTEST_FUNC_EMPTY +#define DOCTEST_WARN_LE(...) DOCTEST_FUNC_EMPTY +#define DOCTEST_CHECK_LE(...) DOCTEST_FUNC_EMPTY +#define DOCTEST_REQUIRE_LE(...) DOCTEST_FUNC_EMPTY + +#define DOCTEST_WARN_UNARY(...) DOCTEST_FUNC_EMPTY +#define DOCTEST_CHECK_UNARY(...) DOCTEST_FUNC_EMPTY +#define DOCTEST_REQUIRE_UNARY(...) DOCTEST_FUNC_EMPTY +#define DOCTEST_WARN_UNARY_FALSE(...) DOCTEST_FUNC_EMPTY +#define DOCTEST_CHECK_UNARY_FALSE(...) DOCTEST_FUNC_EMPTY +#define DOCTEST_REQUIRE_UNARY_FALSE(...) DOCTEST_FUNC_EMPTY + +#ifndef DOCTEST_CONFIG_NO_EXCEPTIONS + +#define DOCTEST_WARN_THROWS(...) DOCTEST_FUNC_EMPTY +#define DOCTEST_CHECK_THROWS(...) DOCTEST_FUNC_EMPTY +#define DOCTEST_REQUIRE_THROWS(...) DOCTEST_FUNC_EMPTY +#define DOCTEST_WARN_THROWS_AS(expr, ...) DOCTEST_FUNC_EMPTY +#define DOCTEST_CHECK_THROWS_AS(expr, ...) DOCTEST_FUNC_EMPTY +#define DOCTEST_REQUIRE_THROWS_AS(expr, ...) DOCTEST_FUNC_EMPTY +#define DOCTEST_WARN_THROWS_WITH(expr, ...) DOCTEST_FUNC_EMPTY +#define DOCTEST_CHECK_THROWS_WITH(expr, ...) DOCTEST_FUNC_EMPTY +#define DOCTEST_REQUIRE_THROWS_WITH(expr, ...) DOCTEST_FUNC_EMPTY +#define DOCTEST_WARN_THROWS_WITH_AS(expr, with, ...) DOCTEST_FUNC_EMPTY +#define DOCTEST_CHECK_THROWS_WITH_AS(expr, with, ...) DOCTEST_FUNC_EMPTY +#define DOCTEST_REQUIRE_THROWS_WITH_AS(expr, with, ...) DOCTEST_FUNC_EMPTY +#define DOCTEST_WARN_NOTHROW(...) DOCTEST_FUNC_EMPTY +#define DOCTEST_CHECK_NOTHROW(...) DOCTEST_FUNC_EMPTY +#define DOCTEST_REQUIRE_NOTHROW(...) DOCTEST_FUNC_EMPTY + +#define DOCTEST_WARN_THROWS_MESSAGE(expr, ...) DOCTEST_FUNC_EMPTY +#define DOCTEST_CHECK_THROWS_MESSAGE(expr, ...) DOCTEST_FUNC_EMPTY +#define DOCTEST_REQUIRE_THROWS_MESSAGE(expr, ...) DOCTEST_FUNC_EMPTY +#define DOCTEST_WARN_THROWS_AS_MESSAGE(expr, ex, ...) DOCTEST_FUNC_EMPTY +#define DOCTEST_CHECK_THROWS_AS_MESSAGE(expr, ex, ...) DOCTEST_FUNC_EMPTY +#define DOCTEST_REQUIRE_THROWS_AS_MESSAGE(expr, ex, ...) DOCTEST_FUNC_EMPTY +#define DOCTEST_WARN_THROWS_WITH_MESSAGE(expr, with, ...) DOCTEST_FUNC_EMPTY +#define DOCTEST_CHECK_THROWS_WITH_MESSAGE(expr, with, ...) DOCTEST_FUNC_EMPTY +#define DOCTEST_REQUIRE_THROWS_WITH_MESSAGE(expr, with, ...) DOCTEST_FUNC_EMPTY +#define DOCTEST_WARN_THROWS_WITH_AS_MESSAGE(expr, with, ex, ...) DOCTEST_FUNC_EMPTY +#define DOCTEST_CHECK_THROWS_WITH_AS_MESSAGE(expr, with, ex, ...) DOCTEST_FUNC_EMPTY +#define DOCTEST_REQUIRE_THROWS_WITH_AS_MESSAGE(expr, with, ex, ...) DOCTEST_FUNC_EMPTY +#define DOCTEST_WARN_NOTHROW_MESSAGE(expr, ...) DOCTEST_FUNC_EMPTY +#define DOCTEST_CHECK_NOTHROW_MESSAGE(expr, ...) DOCTEST_FUNC_EMPTY +#define DOCTEST_REQUIRE_NOTHROW_MESSAGE(expr, ...) DOCTEST_FUNC_EMPTY + +#endif // DOCTEST_CONFIG_NO_EXCEPTIONS + +#endif // DOCTEST_CONFIG_EVALUATE_ASSERTS_EVEN_WHEN_DISABLED + +#endif // DOCTEST_CONFIG_DISABLE + #ifdef DOCTEST_CONFIG_NO_EXCEPTIONS -#undef DOCTEST_WARN_THROWS -#undef DOCTEST_CHECK_THROWS -#undef DOCTEST_REQUIRE_THROWS -#undef DOCTEST_WARN_THROWS_AS -#undef DOCTEST_CHECK_THROWS_AS -#undef DOCTEST_REQUIRE_THROWS_AS -#undef DOCTEST_WARN_THROWS_WITH -#undef DOCTEST_CHECK_THROWS_WITH -#undef DOCTEST_REQUIRE_THROWS_WITH -#undef DOCTEST_WARN_THROWS_WITH_AS -#undef DOCTEST_CHECK_THROWS_WITH_AS -#undef DOCTEST_REQUIRE_THROWS_WITH_AS -#undef DOCTEST_WARN_NOTHROW -#undef DOCTEST_CHECK_NOTHROW -#undef DOCTEST_REQUIRE_NOTHROW - -#undef DOCTEST_WARN_THROWS_MESSAGE -#undef DOCTEST_CHECK_THROWS_MESSAGE -#undef DOCTEST_REQUIRE_THROWS_MESSAGE -#undef DOCTEST_WARN_THROWS_AS_MESSAGE -#undef DOCTEST_CHECK_THROWS_AS_MESSAGE -#undef DOCTEST_REQUIRE_THROWS_AS_MESSAGE -#undef DOCTEST_WARN_THROWS_WITH_MESSAGE -#undef DOCTEST_CHECK_THROWS_WITH_MESSAGE -#undef DOCTEST_REQUIRE_THROWS_WITH_MESSAGE -#undef DOCTEST_WARN_THROWS_WITH_AS_MESSAGE -#undef DOCTEST_CHECK_THROWS_WITH_AS_MESSAGE -#undef DOCTEST_REQUIRE_THROWS_WITH_AS_MESSAGE -#undef DOCTEST_WARN_NOTHROW_MESSAGE -#undef DOCTEST_CHECK_NOTHROW_MESSAGE -#undef DOCTEST_REQUIRE_NOTHROW_MESSAGE - #ifdef DOCTEST_CONFIG_NO_EXCEPTIONS_BUT_WITH_ALL_ASSERTS - -#define DOCTEST_WARN_THROWS(...) (static_cast(0)) -#define DOCTEST_CHECK_THROWS(...) (static_cast(0)) -#define DOCTEST_REQUIRE_THROWS(...) (static_cast(0)) -#define DOCTEST_WARN_THROWS_AS(expr, ...) (static_cast(0)) -#define DOCTEST_CHECK_THROWS_AS(expr, ...) (static_cast(0)) -#define DOCTEST_REQUIRE_THROWS_AS(expr, ...) (static_cast(0)) -#define DOCTEST_WARN_THROWS_WITH(expr, ...) (static_cast(0)) -#define DOCTEST_CHECK_THROWS_WITH(expr, ...) (static_cast(0)) -#define DOCTEST_REQUIRE_THROWS_WITH(expr, ...) (static_cast(0)) -#define DOCTEST_WARN_THROWS_WITH_AS(expr, with, ...) (static_cast(0)) -#define DOCTEST_CHECK_THROWS_WITH_AS(expr, with, ...) (static_cast(0)) -#define DOCTEST_REQUIRE_THROWS_WITH_AS(expr, with, ...) (static_cast(0)) -#define DOCTEST_WARN_NOTHROW(...) (static_cast(0)) -#define DOCTEST_CHECK_NOTHROW(...) (static_cast(0)) -#define DOCTEST_REQUIRE_NOTHROW(...) (static_cast(0)) - -#define DOCTEST_WARN_THROWS_MESSAGE(expr, ...) (static_cast(0)) -#define DOCTEST_CHECK_THROWS_MESSAGE(expr, ...) (static_cast(0)) -#define DOCTEST_REQUIRE_THROWS_MESSAGE(expr, ...) (static_cast(0)) -#define DOCTEST_WARN_THROWS_AS_MESSAGE(expr, ex, ...) (static_cast(0)) -#define DOCTEST_CHECK_THROWS_AS_MESSAGE(expr, ex, ...) (static_cast(0)) -#define DOCTEST_REQUIRE_THROWS_AS_MESSAGE(expr, ex, ...) (static_cast(0)) -#define DOCTEST_WARN_THROWS_WITH_MESSAGE(expr, with, ...) (static_cast(0)) -#define DOCTEST_CHECK_THROWS_WITH_MESSAGE(expr, with, ...) (static_cast(0)) -#define DOCTEST_REQUIRE_THROWS_WITH_MESSAGE(expr, with, ...) (static_cast(0)) -#define DOCTEST_WARN_THROWS_WITH_AS_MESSAGE(expr, with, ex, ...) (static_cast(0)) -#define DOCTEST_CHECK_THROWS_WITH_AS_MESSAGE(expr, with, ex, ...) (static_cast(0)) -#define DOCTEST_REQUIRE_THROWS_WITH_AS_MESSAGE(expr, with, ex, ...) (static_cast(0)) -#define DOCTEST_WARN_NOTHROW_MESSAGE(expr, ...) (static_cast(0)) -#define DOCTEST_CHECK_NOTHROW_MESSAGE(expr, ...) (static_cast(0)) -#define DOCTEST_REQUIRE_NOTHROW_MESSAGE(expr, ...) (static_cast(0)) - +#define DOCTEST_EXCEPTION_EMPTY_FUNC DOCTEST_FUNC_EMPTY #else // DOCTEST_CONFIG_NO_EXCEPTIONS_BUT_WITH_ALL_ASSERTS +#define DOCTEST_EXCEPTION_EMPTY_FUNC [] { static_assert(false, "Exceptions are disabled! " \ + "Use DOCTEST_CONFIG_NO_EXCEPTIONS_BUT_WITH_ALL_ASSERTS if you want to compile with exceptions disabled."); return false; }() #undef DOCTEST_REQUIRE #undef DOCTEST_REQUIRE_FALSE @@ -2333,163 +2794,55 @@ int registerReporter(const char* name, int priority, bool isReporter) { #undef DOCTEST_REQUIRE_UNARY #undef DOCTEST_REQUIRE_UNARY_FALSE +#define DOCTEST_REQUIRE DOCTEST_EXCEPTION_EMPTY_FUNC +#define DOCTEST_REQUIRE_FALSE DOCTEST_EXCEPTION_EMPTY_FUNC +#define DOCTEST_REQUIRE_MESSAGE DOCTEST_EXCEPTION_EMPTY_FUNC +#define DOCTEST_REQUIRE_FALSE_MESSAGE DOCTEST_EXCEPTION_EMPTY_FUNC +#define DOCTEST_REQUIRE_EQ DOCTEST_EXCEPTION_EMPTY_FUNC +#define DOCTEST_REQUIRE_NE DOCTEST_EXCEPTION_EMPTY_FUNC +#define DOCTEST_REQUIRE_GT DOCTEST_EXCEPTION_EMPTY_FUNC +#define DOCTEST_REQUIRE_LT DOCTEST_EXCEPTION_EMPTY_FUNC +#define DOCTEST_REQUIRE_GE DOCTEST_EXCEPTION_EMPTY_FUNC +#define DOCTEST_REQUIRE_LE DOCTEST_EXCEPTION_EMPTY_FUNC +#define DOCTEST_REQUIRE_UNARY DOCTEST_EXCEPTION_EMPTY_FUNC +#define DOCTEST_REQUIRE_UNARY_FALSE DOCTEST_EXCEPTION_EMPTY_FUNC + #endif // DOCTEST_CONFIG_NO_EXCEPTIONS_BUT_WITH_ALL_ASSERTS +#define DOCTEST_WARN_THROWS(...) DOCTEST_EXCEPTION_EMPTY_FUNC +#define DOCTEST_CHECK_THROWS(...) DOCTEST_EXCEPTION_EMPTY_FUNC +#define DOCTEST_REQUIRE_THROWS(...) DOCTEST_EXCEPTION_EMPTY_FUNC +#define DOCTEST_WARN_THROWS_AS(expr, ...) DOCTEST_EXCEPTION_EMPTY_FUNC +#define DOCTEST_CHECK_THROWS_AS(expr, ...) DOCTEST_EXCEPTION_EMPTY_FUNC +#define DOCTEST_REQUIRE_THROWS_AS(expr, ...) DOCTEST_EXCEPTION_EMPTY_FUNC +#define DOCTEST_WARN_THROWS_WITH(expr, ...) DOCTEST_EXCEPTION_EMPTY_FUNC +#define DOCTEST_CHECK_THROWS_WITH(expr, ...) DOCTEST_EXCEPTION_EMPTY_FUNC +#define DOCTEST_REQUIRE_THROWS_WITH(expr, ...) DOCTEST_EXCEPTION_EMPTY_FUNC +#define DOCTEST_WARN_THROWS_WITH_AS(expr, with, ...) DOCTEST_EXCEPTION_EMPTY_FUNC +#define DOCTEST_CHECK_THROWS_WITH_AS(expr, with, ...) DOCTEST_EXCEPTION_EMPTY_FUNC +#define DOCTEST_REQUIRE_THROWS_WITH_AS(expr, with, ...) DOCTEST_EXCEPTION_EMPTY_FUNC +#define DOCTEST_WARN_NOTHROW(...) DOCTEST_EXCEPTION_EMPTY_FUNC +#define DOCTEST_CHECK_NOTHROW(...) DOCTEST_EXCEPTION_EMPTY_FUNC +#define DOCTEST_REQUIRE_NOTHROW(...) DOCTEST_EXCEPTION_EMPTY_FUNC + +#define DOCTEST_WARN_THROWS_MESSAGE(expr, ...) DOCTEST_EXCEPTION_EMPTY_FUNC +#define DOCTEST_CHECK_THROWS_MESSAGE(expr, ...) DOCTEST_EXCEPTION_EMPTY_FUNC +#define DOCTEST_REQUIRE_THROWS_MESSAGE(expr, ...) DOCTEST_EXCEPTION_EMPTY_FUNC +#define DOCTEST_WARN_THROWS_AS_MESSAGE(expr, ex, ...) DOCTEST_EXCEPTION_EMPTY_FUNC +#define DOCTEST_CHECK_THROWS_AS_MESSAGE(expr, ex, ...) DOCTEST_EXCEPTION_EMPTY_FUNC +#define DOCTEST_REQUIRE_THROWS_AS_MESSAGE(expr, ex, ...) DOCTEST_EXCEPTION_EMPTY_FUNC +#define DOCTEST_WARN_THROWS_WITH_MESSAGE(expr, with, ...) DOCTEST_EXCEPTION_EMPTY_FUNC +#define DOCTEST_CHECK_THROWS_WITH_MESSAGE(expr, with, ...) DOCTEST_EXCEPTION_EMPTY_FUNC +#define DOCTEST_REQUIRE_THROWS_WITH_MESSAGE(expr, with, ...) DOCTEST_EXCEPTION_EMPTY_FUNC +#define DOCTEST_WARN_THROWS_WITH_AS_MESSAGE(expr, with, ex, ...) DOCTEST_EXCEPTION_EMPTY_FUNC +#define DOCTEST_CHECK_THROWS_WITH_AS_MESSAGE(expr, with, ex, ...) DOCTEST_EXCEPTION_EMPTY_FUNC +#define DOCTEST_REQUIRE_THROWS_WITH_AS_MESSAGE(expr, with, ex, ...) DOCTEST_EXCEPTION_EMPTY_FUNC +#define DOCTEST_WARN_NOTHROW_MESSAGE(expr, ...) DOCTEST_EXCEPTION_EMPTY_FUNC +#define DOCTEST_CHECK_NOTHROW_MESSAGE(expr, ...) DOCTEST_EXCEPTION_EMPTY_FUNC +#define DOCTEST_REQUIRE_NOTHROW_MESSAGE(expr, ...) DOCTEST_EXCEPTION_EMPTY_FUNC + #endif // DOCTEST_CONFIG_NO_EXCEPTIONS -// ================================================================================================= -// == WHAT FOLLOWS IS VERSIONS OF THE MACROS THAT DO NOT DO ANY REGISTERING! == -// == THIS CAN BE ENABLED BY DEFINING DOCTEST_CONFIG_DISABLE GLOBALLY! == -// ================================================================================================= -#else // DOCTEST_CONFIG_DISABLE - -#define DOCTEST_IMPLEMENT_FIXTURE(der, base, func, name) \ - namespace { \ - template \ - struct der : public base \ - { void f(); }; \ - } \ - template \ - inline void der::f() - -#define DOCTEST_CREATE_AND_REGISTER_FUNCTION(f, name) \ - template \ - static inline void f() - -// for registering tests -#define DOCTEST_TEST_CASE(name) \ - DOCTEST_CREATE_AND_REGISTER_FUNCTION(DOCTEST_ANONYMOUS(_DOCTEST_ANON_FUNC_), name) - -// for registering tests in classes -#define DOCTEST_TEST_CASE_CLASS(name) \ - DOCTEST_CREATE_AND_REGISTER_FUNCTION(DOCTEST_ANONYMOUS(_DOCTEST_ANON_FUNC_), name) - -// for registering tests with a fixture -#define DOCTEST_TEST_CASE_FIXTURE(x, name) \ - DOCTEST_IMPLEMENT_FIXTURE(DOCTEST_ANONYMOUS(_DOCTEST_ANON_CLASS_), x, \ - DOCTEST_ANONYMOUS(_DOCTEST_ANON_FUNC_), name) - -// for converting types to strings without the header and demangling -#define DOCTEST_TYPE_TO_STRING(...) typedef int DOCTEST_ANONYMOUS(_DOCTEST_ANON_FOR_SEMICOLON_) -#define DOCTEST_TYPE_TO_STRING_IMPL(...) - -// for typed tests -#define DOCTEST_TEST_CASE_TEMPLATE(name, type, ...) \ - template \ - inline void DOCTEST_ANONYMOUS(_DOCTEST_ANON_TMP_)() - -#define DOCTEST_TEST_CASE_TEMPLATE_DEFINE(name, type, id) \ - template \ - inline void DOCTEST_ANONYMOUS(_DOCTEST_ANON_TMP_)() - -#define DOCTEST_TEST_CASE_TEMPLATE_INVOKE(id, ...) \ - typedef int DOCTEST_ANONYMOUS(_DOCTEST_ANON_FOR_SEMICOLON_) - -#define DOCTEST_TEST_CASE_TEMPLATE_APPLY(id, ...) \ - typedef int DOCTEST_ANONYMOUS(_DOCTEST_ANON_FOR_SEMICOLON_) - -// for subcases -#define DOCTEST_SUBCASE(name) - -// for a testsuite block -#define DOCTEST_TEST_SUITE(name) namespace - -// for starting a testsuite block -#define DOCTEST_TEST_SUITE_BEGIN(name) typedef int DOCTEST_ANONYMOUS(_DOCTEST_ANON_FOR_SEMICOLON_) - -// for ending a testsuite block -#define DOCTEST_TEST_SUITE_END typedef int DOCTEST_ANONYMOUS(_DOCTEST_ANON_FOR_SEMICOLON_) - -#define DOCTEST_REGISTER_EXCEPTION_TRANSLATOR(signature) \ - template \ - static inline doctest::String DOCTEST_ANONYMOUS(_DOCTEST_ANON_TRANSLATOR_)(signature) - -#define DOCTEST_REGISTER_REPORTER(name, priority, reporter) -#define DOCTEST_REGISTER_LISTENER(name, priority, reporter) - -#define DOCTEST_INFO(...) (static_cast(0)) -#define DOCTEST_CAPTURE(x) (static_cast(0)) -#define DOCTEST_ADD_MESSAGE_AT(file, line, ...) (static_cast(0)) -#define DOCTEST_ADD_FAIL_CHECK_AT(file, line, ...) (static_cast(0)) -#define DOCTEST_ADD_FAIL_AT(file, line, ...) (static_cast(0)) -#define DOCTEST_MESSAGE(...) (static_cast(0)) -#define DOCTEST_FAIL_CHECK(...) (static_cast(0)) -#define DOCTEST_FAIL(...) (static_cast(0)) - -#define DOCTEST_WARN(...) (static_cast(0)) -#define DOCTEST_CHECK(...) (static_cast(0)) -#define DOCTEST_REQUIRE(...) (static_cast(0)) -#define DOCTEST_WARN_FALSE(...) (static_cast(0)) -#define DOCTEST_CHECK_FALSE(...) (static_cast(0)) -#define DOCTEST_REQUIRE_FALSE(...) (static_cast(0)) - -#define DOCTEST_WARN_MESSAGE(cond, ...) (static_cast(0)) -#define DOCTEST_CHECK_MESSAGE(cond, ...) (static_cast(0)) -#define DOCTEST_REQUIRE_MESSAGE(cond, ...) (static_cast(0)) -#define DOCTEST_WARN_FALSE_MESSAGE(cond, ...) (static_cast(0)) -#define DOCTEST_CHECK_FALSE_MESSAGE(cond, ...) (static_cast(0)) -#define DOCTEST_REQUIRE_FALSE_MESSAGE(cond, ...) (static_cast(0)) - -#define DOCTEST_WARN_THROWS(...) (static_cast(0)) -#define DOCTEST_CHECK_THROWS(...) (static_cast(0)) -#define DOCTEST_REQUIRE_THROWS(...) (static_cast(0)) -#define DOCTEST_WARN_THROWS_AS(expr, ...) (static_cast(0)) -#define DOCTEST_CHECK_THROWS_AS(expr, ...) (static_cast(0)) -#define DOCTEST_REQUIRE_THROWS_AS(expr, ...) (static_cast(0)) -#define DOCTEST_WARN_THROWS_WITH(expr, ...) (static_cast(0)) -#define DOCTEST_CHECK_THROWS_WITH(expr, ...) (static_cast(0)) -#define DOCTEST_REQUIRE_THROWS_WITH(expr, ...) (static_cast(0)) -#define DOCTEST_WARN_THROWS_WITH_AS(expr, with, ...) (static_cast(0)) -#define DOCTEST_CHECK_THROWS_WITH_AS(expr, with, ...) (static_cast(0)) -#define DOCTEST_REQUIRE_THROWS_WITH_AS(expr, with, ...) (static_cast(0)) -#define DOCTEST_WARN_NOTHROW(...) (static_cast(0)) -#define DOCTEST_CHECK_NOTHROW(...) (static_cast(0)) -#define DOCTEST_REQUIRE_NOTHROW(...) (static_cast(0)) - -#define DOCTEST_WARN_THROWS_MESSAGE(expr, ...) (static_cast(0)) -#define DOCTEST_CHECK_THROWS_MESSAGE(expr, ...) (static_cast(0)) -#define DOCTEST_REQUIRE_THROWS_MESSAGE(expr, ...) (static_cast(0)) -#define DOCTEST_WARN_THROWS_AS_MESSAGE(expr, ex, ...) (static_cast(0)) -#define DOCTEST_CHECK_THROWS_AS_MESSAGE(expr, ex, ...) (static_cast(0)) -#define DOCTEST_REQUIRE_THROWS_AS_MESSAGE(expr, ex, ...) (static_cast(0)) -#define DOCTEST_WARN_THROWS_WITH_MESSAGE(expr, with, ...) (static_cast(0)) -#define DOCTEST_CHECK_THROWS_WITH_MESSAGE(expr, with, ...) (static_cast(0)) -#define DOCTEST_REQUIRE_THROWS_WITH_MESSAGE(expr, with, ...) (static_cast(0)) -#define DOCTEST_WARN_THROWS_WITH_AS_MESSAGE(expr, with, ex, ...) (static_cast(0)) -#define DOCTEST_CHECK_THROWS_WITH_AS_MESSAGE(expr, with, ex, ...) (static_cast(0)) -#define DOCTEST_REQUIRE_THROWS_WITH_AS_MESSAGE(expr, with, ex, ...) (static_cast(0)) -#define DOCTEST_WARN_NOTHROW_MESSAGE(expr, ...) (static_cast(0)) -#define DOCTEST_CHECK_NOTHROW_MESSAGE(expr, ...) (static_cast(0)) -#define DOCTEST_REQUIRE_NOTHROW_MESSAGE(expr, ...) (static_cast(0)) - -#define DOCTEST_WARN_EQ(...) (static_cast(0)) -#define DOCTEST_CHECK_EQ(...) (static_cast(0)) -#define DOCTEST_REQUIRE_EQ(...) (static_cast(0)) -#define DOCTEST_WARN_NE(...) (static_cast(0)) -#define DOCTEST_CHECK_NE(...) (static_cast(0)) -#define DOCTEST_REQUIRE_NE(...) (static_cast(0)) -#define DOCTEST_WARN_GT(...) (static_cast(0)) -#define DOCTEST_CHECK_GT(...) (static_cast(0)) -#define DOCTEST_REQUIRE_GT(...) (static_cast(0)) -#define DOCTEST_WARN_LT(...) (static_cast(0)) -#define DOCTEST_CHECK_LT(...) (static_cast(0)) -#define DOCTEST_REQUIRE_LT(...) (static_cast(0)) -#define DOCTEST_WARN_GE(...) (static_cast(0)) -#define DOCTEST_CHECK_GE(...) (static_cast(0)) -#define DOCTEST_REQUIRE_GE(...) (static_cast(0)) -#define DOCTEST_WARN_LE(...) (static_cast(0)) -#define DOCTEST_CHECK_LE(...) (static_cast(0)) -#define DOCTEST_REQUIRE_LE(...) (static_cast(0)) - -#define DOCTEST_WARN_UNARY(...) (static_cast(0)) -#define DOCTEST_CHECK_UNARY(...) (static_cast(0)) -#define DOCTEST_REQUIRE_UNARY(...) (static_cast(0)) -#define DOCTEST_WARN_UNARY_FALSE(...) (static_cast(0)) -#define DOCTEST_CHECK_UNARY_FALSE(...) (static_cast(0)) -#define DOCTEST_REQUIRE_UNARY_FALSE(...) (static_cast(0)) - -#endif // DOCTEST_CONFIG_DISABLE - // clang-format off // KEPT FOR BACKWARDS COMPATIBILITY - FORWARDING TO THE RIGHT MACROS #define DOCTEST_FAST_WARN_EQ DOCTEST_WARN_EQ @@ -2536,11 +2889,12 @@ int registerReporter(const char* name, int priority, bool isReporter) { // clang-format on // == SHORT VERSIONS OF THE MACROS -#if !defined(DOCTEST_CONFIG_NO_SHORT_MACRO_NAMES) +#ifndef DOCTEST_CONFIG_NO_SHORT_MACRO_NAMES #define TEST_CASE(name) DOCTEST_TEST_CASE(name) #define TEST_CASE_CLASS(name) DOCTEST_TEST_CASE_CLASS(name) #define TEST_CASE_FIXTURE(x, name) DOCTEST_TEST_CASE_FIXTURE(x, name) +#define TYPE_TO_STRING_AS(str, ...) DOCTEST_TYPE_TO_STRING_AS(str, __VA_ARGS__) #define TYPE_TO_STRING(...) DOCTEST_TYPE_TO_STRING(__VA_ARGS__) #define TEST_CASE_TEMPLATE(name, T, ...) DOCTEST_TEST_CASE_TEMPLATE(name, T, __VA_ARGS__) #define TEST_CASE_TEMPLATE_DEFINE(name, T, id) DOCTEST_TEST_CASE_TEMPLATE_DEFINE(name, T, id) @@ -2673,39 +3027,19 @@ int registerReporter(const char* name, int priority, bool isReporter) { #endif // DOCTEST_CONFIG_NO_SHORT_MACRO_NAMES -#if !defined(DOCTEST_CONFIG_DISABLE) +#ifndef DOCTEST_CONFIG_DISABLE // this is here to clear the 'current test suite' for the current translation unit - at the top DOCTEST_TEST_SUITE_END(); -// add stringification for primitive/fundamental types -namespace doctest { namespace detail { - DOCTEST_TYPE_TO_STRING_IMPL(bool) - DOCTEST_TYPE_TO_STRING_IMPL(float) - DOCTEST_TYPE_TO_STRING_IMPL(double) - DOCTEST_TYPE_TO_STRING_IMPL(long double) - DOCTEST_TYPE_TO_STRING_IMPL(char) - DOCTEST_TYPE_TO_STRING_IMPL(signed char) - DOCTEST_TYPE_TO_STRING_IMPL(unsigned char) -#if !DOCTEST_MSVC || defined(_NATIVE_WCHAR_T_DEFINED) - DOCTEST_TYPE_TO_STRING_IMPL(wchar_t) -#endif // not MSVC or wchar_t support enabled - DOCTEST_TYPE_TO_STRING_IMPL(short int) - DOCTEST_TYPE_TO_STRING_IMPL(unsigned short int) - DOCTEST_TYPE_TO_STRING_IMPL(int) - DOCTEST_TYPE_TO_STRING_IMPL(unsigned int) - DOCTEST_TYPE_TO_STRING_IMPL(long int) - DOCTEST_TYPE_TO_STRING_IMPL(unsigned long int) - DOCTEST_TYPE_TO_STRING_IMPL(long long int) - DOCTEST_TYPE_TO_STRING_IMPL(unsigned long long int) -}} // namespace doctest::detail - #endif // DOCTEST_CONFIG_DISABLE DOCTEST_CLANG_SUPPRESS_WARNING_POP DOCTEST_MSVC_SUPPRESS_WARNING_POP DOCTEST_GCC_SUPPRESS_WARNING_POP +DOCTEST_SUPPRESS_COMMON_WARNINGS_POP + #endif // DOCTEST_LIBRARY_INCLUDED #ifndef DOCTEST_SINGLE_HEADER @@ -2725,13 +3059,11 @@ DOCTEST_CLANG_SUPPRESS_WARNING_WITH_PUSH("-Wunused-macros") DOCTEST_CLANG_SUPPRESS_WARNING_POP +DOCTEST_SUPPRESS_COMMON_WARNINGS_PUSH + DOCTEST_CLANG_SUPPRESS_WARNING_PUSH -DOCTEST_CLANG_SUPPRESS_WARNING("-Wunknown-pragmas") -DOCTEST_CLANG_SUPPRESS_WARNING("-Wpadded") -DOCTEST_CLANG_SUPPRESS_WARNING("-Wweak-vtables") DOCTEST_CLANG_SUPPRESS_WARNING("-Wglobal-constructors") DOCTEST_CLANG_SUPPRESS_WARNING("-Wexit-time-destructors") -DOCTEST_CLANG_SUPPRESS_WARNING("-Wmissing-prototypes") DOCTEST_CLANG_SUPPRESS_WARNING("-Wsign-conversion") DOCTEST_CLANG_SUPPRESS_WARNING("-Wshorten-64-to-32") DOCTEST_CLANG_SUPPRESS_WARNING("-Wmissing-variable-declarations") @@ -2739,65 +3071,35 @@ DOCTEST_CLANG_SUPPRESS_WARNING("-Wswitch") DOCTEST_CLANG_SUPPRESS_WARNING("-Wswitch-enum") DOCTEST_CLANG_SUPPRESS_WARNING("-Wcovered-switch-default") DOCTEST_CLANG_SUPPRESS_WARNING("-Wmissing-noreturn") -DOCTEST_CLANG_SUPPRESS_WARNING("-Wunused-local-typedef") DOCTEST_CLANG_SUPPRESS_WARNING("-Wdisabled-macro-expansion") DOCTEST_CLANG_SUPPRESS_WARNING("-Wmissing-braces") DOCTEST_CLANG_SUPPRESS_WARNING("-Wmissing-field-initializers") -DOCTEST_CLANG_SUPPRESS_WARNING("-Wc++98-compat") -DOCTEST_CLANG_SUPPRESS_WARNING("-Wc++98-compat-pedantic") DOCTEST_CLANG_SUPPRESS_WARNING("-Wunused-member-function") DOCTEST_CLANG_SUPPRESS_WARNING("-Wnonportable-system-include-path") DOCTEST_GCC_SUPPRESS_WARNING_PUSH -DOCTEST_GCC_SUPPRESS_WARNING("-Wunknown-pragmas") -DOCTEST_GCC_SUPPRESS_WARNING("-Wpragmas") DOCTEST_GCC_SUPPRESS_WARNING("-Wconversion") -DOCTEST_GCC_SUPPRESS_WARNING("-Weffc++") DOCTEST_GCC_SUPPRESS_WARNING("-Wsign-conversion") -DOCTEST_GCC_SUPPRESS_WARNING("-Wstrict-overflow") -DOCTEST_GCC_SUPPRESS_WARNING("-Wstrict-aliasing") DOCTEST_GCC_SUPPRESS_WARNING("-Wmissing-field-initializers") DOCTEST_GCC_SUPPRESS_WARNING("-Wmissing-braces") -DOCTEST_GCC_SUPPRESS_WARNING("-Wmissing-declarations") DOCTEST_GCC_SUPPRESS_WARNING("-Wswitch") DOCTEST_GCC_SUPPRESS_WARNING("-Wswitch-enum") DOCTEST_GCC_SUPPRESS_WARNING("-Wswitch-default") DOCTEST_GCC_SUPPRESS_WARNING("-Wunsafe-loop-optimizations") DOCTEST_GCC_SUPPRESS_WARNING("-Wold-style-cast") -DOCTEST_GCC_SUPPRESS_WARNING("-Wunused-local-typedefs") -DOCTEST_GCC_SUPPRESS_WARNING("-Wuseless-cast") DOCTEST_GCC_SUPPRESS_WARNING("-Wunused-function") DOCTEST_GCC_SUPPRESS_WARNING("-Wmultiple-inheritance") -DOCTEST_GCC_SUPPRESS_WARNING("-Wnoexcept") DOCTEST_GCC_SUPPRESS_WARNING("-Wsuggest-attribute") DOCTEST_MSVC_SUPPRESS_WARNING_PUSH -DOCTEST_MSVC_SUPPRESS_WARNING(4616) // invalid compiler warning -DOCTEST_MSVC_SUPPRESS_WARNING(4619) // invalid compiler warning -DOCTEST_MSVC_SUPPRESS_WARNING(4996) // The compiler encountered a deprecated declaration DOCTEST_MSVC_SUPPRESS_WARNING(4267) // 'var' : conversion from 'x' to 'y', possible loss of data -DOCTEST_MSVC_SUPPRESS_WARNING(4706) // assignment within conditional expression -DOCTEST_MSVC_SUPPRESS_WARNING(4512) // 'class' : assignment operator could not be generated -DOCTEST_MSVC_SUPPRESS_WARNING(4127) // conditional expression is constant DOCTEST_MSVC_SUPPRESS_WARNING(4530) // C++ exception handler used, but unwind semantics not enabled DOCTEST_MSVC_SUPPRESS_WARNING(4577) // 'noexcept' used with no exception handling mode specified DOCTEST_MSVC_SUPPRESS_WARNING(4774) // format string expected in argument is not a string literal DOCTEST_MSVC_SUPPRESS_WARNING(4365) // conversion from 'int' to 'unsigned', signed/unsigned mismatch -DOCTEST_MSVC_SUPPRESS_WARNING(4820) // padding in structs -DOCTEST_MSVC_SUPPRESS_WARNING(4640) // construction of local static object is not thread-safe DOCTEST_MSVC_SUPPRESS_WARNING(5039) // pointer to potentially throwing function passed to extern C -DOCTEST_MSVC_SUPPRESS_WARNING(5045) // Spectre mitigation stuff -DOCTEST_MSVC_SUPPRESS_WARNING(4626) // assignment operator was implicitly defined as deleted -DOCTEST_MSVC_SUPPRESS_WARNING(5027) // move assignment operator was implicitly defined as deleted -DOCTEST_MSVC_SUPPRESS_WARNING(5026) // move constructor was implicitly defined as deleted -DOCTEST_MSVC_SUPPRESS_WARNING(4625) // copy constructor was implicitly defined as deleted DOCTEST_MSVC_SUPPRESS_WARNING(4800) // forcing value to bool 'true' or 'false' (performance warning) -// static analysis -DOCTEST_MSVC_SUPPRESS_WARNING(26439) // This kind of function may not throw. Declare it 'noexcept' -DOCTEST_MSVC_SUPPRESS_WARNING(26495) // Always initialize a member variable -DOCTEST_MSVC_SUPPRESS_WARNING(26451) // Arithmetic overflow ... -DOCTEST_MSVC_SUPPRESS_WARNING(26444) // Avoid unnamed objects with custom construction and dtor... -DOCTEST_MSVC_SUPPRESS_WARNING(26812) // Prefer 'enum class' over 'enum' +DOCTEST_MSVC_SUPPRESS_WARNING(5245) // unreferenced function with internal linkage has been removed DOCTEST_MAKE_STD_HEADERS_CLEAN_FROM_WARNINGS_ON_WALL_BEGIN @@ -2805,7 +3107,7 @@ DOCTEST_MAKE_STD_HEADERS_CLEAN_FROM_WARNINGS_ON_WALL_BEGIN #include #include #include -// borland (Embarcadero) compiler requires math.h and not cmath - https://github.com/onqtam/doctest/pull/37 +// borland (Embarcadero) compiler requires math.h and not cmath - https://github.com/doctest/doctest/pull/37 #ifdef __BORLANDC__ #include #endif // __BORLANDC__ @@ -2821,16 +3123,27 @@ DOCTEST_MAKE_STD_HEADERS_CLEAN_FROM_WARNINGS_ON_WALL_BEGIN #include #include #include +#ifndef DOCTEST_CONFIG_NO_MULTITHREADING #include #include +#define DOCTEST_DECLARE_MUTEX(name) std::mutex name; +#define DOCTEST_DECLARE_STATIC_MUTEX(name) static DOCTEST_DECLARE_MUTEX(name) +#define DOCTEST_LOCK_MUTEX(name) std::lock_guard DOCTEST_ANONYMOUS(DOCTEST_ANON_LOCK_)(name); +#else // DOCTEST_CONFIG_NO_MULTITHREADING +#define DOCTEST_DECLARE_MUTEX(name) +#define DOCTEST_DECLARE_STATIC_MUTEX(name) +#define DOCTEST_LOCK_MUTEX(name) +#endif // DOCTEST_CONFIG_NO_MULTITHREADING #include #include +#include #include #include #include #include #include #include +#include #ifdef DOCTEST_PLATFORM_MAC #include @@ -2863,7 +3176,7 @@ DOCTEST_MAKE_STD_HEADERS_CLEAN_FROM_WARNINGS_ON_WALL_BEGIN #endif // DOCTEST_PLATFORM_WINDOWS -// this is a fix for https://github.com/onqtam/doctest/issues/348 +// this is a fix for https://github.com/doctest/doctest/issues/348 // https://mail.gnome.org/archives/xml/2012-January/msg00000.html #if !defined(HAVE_UNISTD_H) && !defined(STDOUT_FILENO) #define STDOUT_FILENO fileno(stdout) @@ -2885,8 +3198,12 @@ DOCTEST_MAKE_STD_HEADERS_CLEAN_FROM_WARNINGS_ON_WALL_END #endif #ifndef DOCTEST_THREAD_LOCAL +#if defined(DOCTEST_CONFIG_NO_MULTITHREADING) || DOCTEST_MSVC && (DOCTEST_MSVC < DOCTEST_COMPILER(19, 0, 0)) +#define DOCTEST_THREAD_LOCAL +#else // DOCTEST_MSVC #define DOCTEST_THREAD_LOCAL thread_local -#endif +#endif // DOCTEST_MSVC +#endif // DOCTEST_THREAD_LOCAL #ifndef DOCTEST_MULTI_LANE_ATOMICS_THREAD_LANES #define DOCTEST_MULTI_LANE_ATOMICS_THREAD_LANES 32 @@ -2906,12 +3223,34 @@ DOCTEST_MAKE_STD_HEADERS_CLEAN_FROM_WARNINGS_ON_WALL_END #define DOCTEST_CONFIG_NO_MULTI_LANE_ATOMICS #endif +#ifndef DOCTEST_CDECL +#define DOCTEST_CDECL __cdecl +#endif + namespace doctest { bool is_running_in_test = false; namespace { using namespace detail; + + template + DOCTEST_NORETURN void throw_exception(Ex const& e) { +#ifndef DOCTEST_CONFIG_NO_EXCEPTIONS + throw e; +#else // DOCTEST_CONFIG_NO_EXCEPTIONS + std::cerr << "doctest will terminate because it needed to throw an exception.\n" + << "The message was: " << e.what() << '\n'; + std::terminate(); +#endif // DOCTEST_CONFIG_NO_EXCEPTIONS + } + +#ifndef DOCTEST_INTERNAL_ERROR +#define DOCTEST_INTERNAL_ERROR(msg) \ + throw_exception(std::logic_error( \ + __FILE__ ":" DOCTEST_TOSTR(__LINE__) ": Internal doctest error: " msg)) +#endif // DOCTEST_INTERNAL_ERROR + // case insensitive strcmp int stricmp(const char* a, const char* b) { for(;; a++, b++) { @@ -2921,20 +3260,6 @@ namespace { } } - template - String fpToString(T value, int precision) { - std::ostringstream oss; - oss << std::setprecision(precision) << std::fixed << value; - std::string d = oss.str(); - size_t i = d.find_last_not_of('0'); - if(i != std::string::npos && i != d.size() - 1) { - if(d[i] == '.') - i++; - d = d.substr(0, i + 1); - } - return d.c_str(); - } - struct Endianness { enum Arch @@ -2955,36 +3280,35 @@ namespace { } // namespace namespace detail { - void my_memcpy(void* dest, const void* src, unsigned num) { memcpy(dest, src, num); } + DOCTEST_THREAD_LOCAL class + { + std::vector stack; + std::stringstream ss; - String rawMemoryToString(const void* object, unsigned size) { - // Reverse order for little endian architectures - int i = 0, end = static_cast(size), inc = 1; - if(Endianness::which() == Endianness::Little) { - i = end - 1; - end = inc = -1; + public: + std::ostream* push() { + stack.push_back(ss.tellp()); + return &ss; } - unsigned const char* bytes = static_cast(object); - std::ostringstream oss; - oss << "0x" << std::setfill('0') << std::hex; - for(; i != end; i += inc) - oss << std::setw(2) << static_cast(bytes[i]); - return oss.str().c_str(); + String pop() { + if (stack.empty()) + DOCTEST_INTERNAL_ERROR("TLSS was empty when trying to pop!"); + + std::streampos pos = stack.back(); + stack.pop_back(); + unsigned sz = static_cast(ss.tellp() - pos); + ss.rdbuf()->pubseekpos(pos, std::ios::in | std::ios::out); + return String(ss, sz); + } + } g_oss; + + std::ostream* tlssPush() { + return g_oss.push(); } - DOCTEST_THREAD_LOCAL std::ostringstream g_oss; // NOLINT(cert-err58-cpp) - - std::ostream* getTlsOss() { - g_oss.clear(); // there shouldn't be anything worth clearing in the flags - g_oss.str(""); // the slow way of resetting a string stream - //g_oss.seekp(0); // optimal reset - as seen here: https://stackoverflow.com/a/624291/3162383 - return &g_oss; - } - - String getTlsOssResult() { - //g_oss << std::ends; // needed - as shown here: https://stackoverflow.com/a/624291/3162383 - return g_oss.str().c_str(); + String tlssPop() { + return g_oss.pop(); } #ifndef DOCTEST_CONFIG_DISABLE @@ -2993,20 +3317,19 @@ namespace timer_large_integer { #if defined(DOCTEST_PLATFORM_WINDOWS) - typedef ULONGLONG type; + using type = ULONGLONG; #else // DOCTEST_PLATFORM_WINDOWS - using namespace std; - typedef uint64_t type; + using type = std::uint64_t; #endif // DOCTEST_PLATFORM_WINDOWS } -typedef timer_large_integer::type ticks_t; +using ticks_t = timer_large_integer::type; #ifdef DOCTEST_CONFIG_GETCURRENTTICKS ticks_t getCurrentTicks() { return DOCTEST_CONFIG_GETCURRENTTICKS(); } #elif defined(DOCTEST_PLATFORM_WINDOWS) ticks_t getCurrentTicks() { - static LARGE_INTEGER hz = {0}, hzo = {0}; + static LARGE_INTEGER hz = { {0} }, hzo = { {0} }; if(!hz.QuadPart) { QueryPerformanceFrequency(&hz); QueryPerformanceCounter(&hzo); @@ -3038,9 +3361,17 @@ typedef timer_large_integer::type ticks_t; ticks_t m_ticks = 0; }; -#ifdef DOCTEST_CONFIG_NO_MULTI_LANE_ATOMICS +#ifdef DOCTEST_CONFIG_NO_MULTITHREADING template - using AtomicOrMultiLaneAtomic = std::atomic; + using Atomic = T; +#else // DOCTEST_CONFIG_NO_MULTITHREADING + template + using Atomic = std::atomic; +#endif // DOCTEST_CONFIG_NO_MULTITHREADING + +#if defined(DOCTEST_CONFIG_NO_MULTI_LANE_ATOMICS) || defined(DOCTEST_CONFIG_NO_MULTITHREADING) + template + using MultiLaneAtomic = Atomic; #else // DOCTEST_CONFIG_NO_MULTI_LANE_ATOMICS // Provides a multilane implementation of an atomic variable that supports add, sub, load, // store. Instead of using a single atomic variable, this splits up into multiple ones, @@ -3057,8 +3388,8 @@ typedef timer_large_integer::type ticks_t; { struct CacheLineAlignedAtomic { - std::atomic atomic{}; - char padding[DOCTEST_MULTI_LANE_ATOMICS_CACHE_LINE_SIZE - sizeof(std::atomic)]; + Atomic atomic{}; + char padding[DOCTEST_MULTI_LANE_ATOMICS_CACHE_LINE_SIZE - sizeof(Atomic)]; }; CacheLineAlignedAtomic m_atomics[DOCTEST_MULTI_LANE_ATOMICS_THREAD_LANES]; @@ -3088,7 +3419,7 @@ typedef timer_large_integer::type ticks_t; return result; } - T operator=(T desired) DOCTEST_NOEXCEPT { + T operator=(T desired) DOCTEST_NOEXCEPT { // lgtm [cpp/assignment-does-not-return-this] store(desired); return desired; } @@ -3103,7 +3434,7 @@ typedef timer_large_integer::type ticks_t; private: // Each thread has a different atomic that it operates on. If more than NumLanes threads - // use this, some will use the same atomic. So performance will degrate a bit, but still + // use this, some will use the same atomic. So performance will degrade a bit, but still // everything will work. // // The logic here is a bit tricky. The call should be as fast as possible, so that there @@ -3114,24 +3445,21 @@ typedef timer_large_integer::type ticks_t; // assigned in a round-robin fashion. // 3. This tlsLaneIdx is stored in the thread local data, so it is directly available with // little overhead. - std::atomic& myAtomic() DOCTEST_NOEXCEPT { - static std::atomic laneCounter; + Atomic& myAtomic() DOCTEST_NOEXCEPT { + static Atomic laneCounter; DOCTEST_THREAD_LOCAL size_t tlsLaneIdx = laneCounter++ % DOCTEST_MULTI_LANE_ATOMICS_THREAD_LANES; return m_atomics[tlsLaneIdx].atomic; } }; - - template - using AtomicOrMultiLaneAtomic = MultiLaneAtomic; #endif // DOCTEST_CONFIG_NO_MULTI_LANE_ATOMICS // this holds both parameters from the command line and runtime data for tests struct ContextState : ContextOptions, TestRunStats, CurrentTestCaseStats { - AtomicOrMultiLaneAtomic numAssertsCurrentTest_atomic; - AtomicOrMultiLaneAtomic numAssertsFailedCurrentTest_atomic; + MultiLaneAtomic numAssertsCurrentTest_atomic; + MultiLaneAtomic numAssertsFailedCurrentTest_atomic; std::vector> filters = decltype(filters)(9); // 9 different filters @@ -3144,11 +3472,12 @@ typedef timer_large_integer::type ticks_t; std::vector stringifiedContexts; // logging from INFO() due to an exception // stuff for subcases - std::vector subcasesStack; - std::set subcasesPassed; - int subcasesCurrentMaxLevel; - bool should_reenter; - std::atomic shouldLogCurrentException; + bool reachedLeaf; + std::vector subcaseStack; + std::vector nextSubcaseStack; + std::unordered_set fullyTraversedSubcases; + size_t currentSubcaseDepth; + Atomic shouldLogCurrentException; void resetRunData() { numTestCases = 0; @@ -3198,7 +3527,8 @@ typedef timer_large_integer::type ticks_t; (TestCaseFailureReason::FailedExactlyNumTimes & failure_flags); // if any subcase has failed - the whole test case has failed - if(failure_flags && !ok_to_fail) + testCaseSuccess = !(failure_flags && !ok_to_fail); + if(!testCaseSuccess) numTestCasesFailed++; } }; @@ -3213,23 +3543,37 @@ typedef timer_large_integer::type ticks_t; #endif // DOCTEST_CONFIG_DISABLE } // namespace detail -void String::setOnHeap() { *reinterpret_cast(&buf[last]) = 128; } -void String::setLast(unsigned in) { buf[last] = char(in); } - -void String::copy(const String& other) { - using namespace std; - if(other.isOnStack()) { - memcpy(buf, other.buf, len); +char* String::allocate(size_type sz) { + if (sz <= last) { + buf[sz] = '\0'; + setLast(last - sz); + return buf; } else { setOnHeap(); - data.size = other.data.size; + data.size = sz; data.capacity = data.size + 1; - data.ptr = new char[data.capacity]; - memcpy(data.ptr, other.data.ptr, data.size + 1); + data.ptr = new char[data.capacity]; + data.ptr[sz] = '\0'; + return data.ptr; } } -String::String() { +void String::setOnHeap() noexcept { *reinterpret_cast(&buf[last]) = 128; } +void String::setLast(size_type in) noexcept { buf[last] = char(in); } +void String::setSize(size_type sz) noexcept { + if (isOnStack()) { buf[sz] = '\0'; setLast(last - sz); } + else { data.ptr[sz] = '\0'; data.size = sz; } +} + +void String::copy(const String& other) { + if(other.isOnStack()) { + memcpy(buf, other.buf, len); + } else { + memcpy(allocate(other.data.size), other.data.ptr, other.data.size); + } +} + +String::String() noexcept { buf[0] = '\0'; setLast(); } @@ -3237,26 +3581,17 @@ String::String() { String::~String() { if(!isOnStack()) delete[] data.ptr; - // NOLINTNEXTLINE(clang-analyzer-cplusplus.NewDeleteLeaks) -} +} // NOLINT(clang-analyzer-cplusplus.NewDeleteLeaks) String::String(const char* in) : String(in, strlen(in)) {} -String::String(const char* in, unsigned in_size) { - using namespace std; - if(in_size <= last) { - memcpy(buf, in, in_size); - buf[in_size] = '\0'; - setLast(last - in_size); - } else { - setOnHeap(); - data.size = in_size; - data.capacity = data.size + 1; - data.ptr = new char[data.capacity]; - memcpy(data.ptr, in, in_size); - data.ptr[in_size] = '\0'; - } +String::String(const char* in, size_type in_size) { + memcpy(allocate(in_size), in, in_size); +} + +String::String(std::istream& in, size_type in_size) { + in.read(allocate(in_size), in_size); } String::String(const String& other) { copy(other); } @@ -3273,10 +3608,9 @@ String& String::operator=(const String& other) { } String& String::operator+=(const String& other) { - const unsigned my_old_size = size(); - const unsigned other_size = other.size(); - const unsigned total_size = my_old_size + other_size; - using namespace std; + const size_type my_old_size = size(); + const size_type other_size = other.size(); + const size_type total_size = my_old_size + other_size; if(isOnStack()) { if(total_size < len) { // append to the current stack space @@ -3323,18 +3657,13 @@ String& String::operator+=(const String& other) { return *this; } -// NOLINTNEXTLINE(clang-analyzer-cplusplus.NewDeleteLeaks) -String String::operator+(const String& other) const { return String(*this) += other; } - -String::String(String&& other) { - using namespace std; +String::String(String&& other) noexcept { memcpy(buf, other.buf, len); other.buf[0] = '\0'; other.setLast(); } -String& String::operator=(String&& other) { - using namespace std; +String& String::operator=(String&& other) noexcept { if(this != &other) { if(!isOnStack()) delete[] data.ptr; @@ -3345,30 +3674,60 @@ String& String::operator=(String&& other) { return *this; } -char String::operator[](unsigned i) const { - return const_cast(this)->operator[](i); // NOLINT +char String::operator[](size_type i) const { + return const_cast(this)->operator[](i); } -char& String::operator[](unsigned i) { +char& String::operator[](size_type i) { if(isOnStack()) return reinterpret_cast(buf)[i]; return data.ptr[i]; } DOCTEST_GCC_SUPPRESS_WARNING_WITH_PUSH("-Wmaybe-uninitialized") -unsigned String::size() const { +String::size_type String::size() const { if(isOnStack()) - return last - (unsigned(buf[last]) & 31); // using "last" would work only if "len" is 32 + return last - (size_type(buf[last]) & 31); // using "last" would work only if "len" is 32 return data.size; } DOCTEST_GCC_SUPPRESS_WARNING_POP -unsigned String::capacity() const { +String::size_type String::capacity() const { if(isOnStack()) return len; return data.capacity; } +String String::substr(size_type pos, size_type cnt) && { + cnt = std::min(cnt, size() - 1 - pos); + char* cptr = c_str(); + memmove(cptr, cptr + pos, cnt); + setSize(cnt); + return std::move(*this); +} + +String String::substr(size_type pos, size_type cnt) const & { + cnt = std::min(cnt, size() - 1 - pos); + return String{ c_str() + pos, cnt }; +} + +String::size_type String::find(char ch, size_type pos) const { + const char* begin = c_str(); + const char* end = begin + size(); + const char* it = begin + pos; + for (; it < end && *it != ch; it++); + if (it < end) { return static_cast(it - begin); } + else { return npos; } +} + +String::size_type String::rfind(char ch, size_type pos) const { + const char* begin = c_str(); + const char* it = begin + std::min(pos, size() - 1); + for (; it >= begin && *it != ch; it--); + if (it >= begin) { return static_cast(it - begin); } + else { return npos; } +} + int String::compare(const char* other, bool no_case) const { if(no_case) return doctest::stricmp(c_str(), other); @@ -3379,17 +3738,32 @@ int String::compare(const String& other, bool no_case) const { return compare(other.c_str(), no_case); } -// clang-format off +String operator+(const String& lhs, const String& rhs) { return String(lhs) += rhs; } + bool operator==(const String& lhs, const String& rhs) { return lhs.compare(rhs) == 0; } bool operator!=(const String& lhs, const String& rhs) { return lhs.compare(rhs) != 0; } bool operator< (const String& lhs, const String& rhs) { return lhs.compare(rhs) < 0; } bool operator> (const String& lhs, const String& rhs) { return lhs.compare(rhs) > 0; } bool operator<=(const String& lhs, const String& rhs) { return (lhs != rhs) ? lhs.compare(rhs) < 0 : true; } bool operator>=(const String& lhs, const String& rhs) { return (lhs != rhs) ? lhs.compare(rhs) > 0 : true; } -// clang-format on std::ostream& operator<<(std::ostream& s, const String& in) { return s << in.c_str(); } +Contains::Contains(const String& str) : string(str) { } + +bool Contains::checkWith(const String& other) const { + return strstr(other.c_str(), string.c_str()) != nullptr; +} + +String toString(const Contains& in) { + return "Contains( " + in.string + " )"; +} + +bool operator==(const String& lhs, const Contains& rhs) { return rhs.checkWith(lhs); } +bool operator==(const Contains& lhs, const String& rhs) { return lhs.checkWith(rhs); } +bool operator!=(const String& lhs, const Contains& rhs) { return !rhs.checkWith(lhs); } +bool operator!=(const Contains& lhs, const String& rhs) { return !lhs.checkWith(rhs); } + namespace { void color_to_stream(std::ostream&, Color::Enum) DOCTEST_BRANCH_ON_DISABLED({}, ;) } // namespace @@ -3403,64 +3777,42 @@ namespace Color { // clang-format off const char* assertString(assertType::Enum at) { - DOCTEST_MSVC_SUPPRESS_WARNING_WITH_PUSH(4062) // enum 'x' in switch of enum 'y' is not handled - switch(at) { //!OCLINT missing default in switch statements - case assertType::DT_WARN : return "WARN"; - case assertType::DT_CHECK : return "CHECK"; - case assertType::DT_REQUIRE : return "REQUIRE"; + DOCTEST_MSVC_SUPPRESS_WARNING_WITH_PUSH(4061) // enum 'x' in switch of enum 'y' is not explicitely handled + #define DOCTEST_GENERATE_ASSERT_TYPE_CASE(assert_type) case assertType::DT_ ## assert_type: return #assert_type + #define DOCTEST_GENERATE_ASSERT_TYPE_CASES(assert_type) \ + DOCTEST_GENERATE_ASSERT_TYPE_CASE(WARN_ ## assert_type); \ + DOCTEST_GENERATE_ASSERT_TYPE_CASE(CHECK_ ## assert_type); \ + DOCTEST_GENERATE_ASSERT_TYPE_CASE(REQUIRE_ ## assert_type) + switch(at) { + DOCTEST_GENERATE_ASSERT_TYPE_CASE(WARN); + DOCTEST_GENERATE_ASSERT_TYPE_CASE(CHECK); + DOCTEST_GENERATE_ASSERT_TYPE_CASE(REQUIRE); - case assertType::DT_WARN_FALSE : return "WARN_FALSE"; - case assertType::DT_CHECK_FALSE : return "CHECK_FALSE"; - case assertType::DT_REQUIRE_FALSE : return "REQUIRE_FALSE"; + DOCTEST_GENERATE_ASSERT_TYPE_CASES(FALSE); - case assertType::DT_WARN_THROWS : return "WARN_THROWS"; - case assertType::DT_CHECK_THROWS : return "CHECK_THROWS"; - case assertType::DT_REQUIRE_THROWS : return "REQUIRE_THROWS"; + DOCTEST_GENERATE_ASSERT_TYPE_CASES(THROWS); - case assertType::DT_WARN_THROWS_AS : return "WARN_THROWS_AS"; - case assertType::DT_CHECK_THROWS_AS : return "CHECK_THROWS_AS"; - case assertType::DT_REQUIRE_THROWS_AS : return "REQUIRE_THROWS_AS"; + DOCTEST_GENERATE_ASSERT_TYPE_CASES(THROWS_AS); - case assertType::DT_WARN_THROWS_WITH : return "WARN_THROWS_WITH"; - case assertType::DT_CHECK_THROWS_WITH : return "CHECK_THROWS_WITH"; - case assertType::DT_REQUIRE_THROWS_WITH : return "REQUIRE_THROWS_WITH"; + DOCTEST_GENERATE_ASSERT_TYPE_CASES(THROWS_WITH); - case assertType::DT_WARN_THROWS_WITH_AS : return "WARN_THROWS_WITH_AS"; - case assertType::DT_CHECK_THROWS_WITH_AS : return "CHECK_THROWS_WITH_AS"; - case assertType::DT_REQUIRE_THROWS_WITH_AS : return "REQUIRE_THROWS_WITH_AS"; + DOCTEST_GENERATE_ASSERT_TYPE_CASES(THROWS_WITH_AS); - case assertType::DT_WARN_NOTHROW : return "WARN_NOTHROW"; - case assertType::DT_CHECK_NOTHROW : return "CHECK_NOTHROW"; - case assertType::DT_REQUIRE_NOTHROW : return "REQUIRE_NOTHROW"; + DOCTEST_GENERATE_ASSERT_TYPE_CASES(NOTHROW); - case assertType::DT_WARN_EQ : return "WARN_EQ"; - case assertType::DT_CHECK_EQ : return "CHECK_EQ"; - case assertType::DT_REQUIRE_EQ : return "REQUIRE_EQ"; - case assertType::DT_WARN_NE : return "WARN_NE"; - case assertType::DT_CHECK_NE : return "CHECK_NE"; - case assertType::DT_REQUIRE_NE : return "REQUIRE_NE"; - case assertType::DT_WARN_GT : return "WARN_GT"; - case assertType::DT_CHECK_GT : return "CHECK_GT"; - case assertType::DT_REQUIRE_GT : return "REQUIRE_GT"; - case assertType::DT_WARN_LT : return "WARN_LT"; - case assertType::DT_CHECK_LT : return "CHECK_LT"; - case assertType::DT_REQUIRE_LT : return "REQUIRE_LT"; - case assertType::DT_WARN_GE : return "WARN_GE"; - case assertType::DT_CHECK_GE : return "CHECK_GE"; - case assertType::DT_REQUIRE_GE : return "REQUIRE_GE"; - case assertType::DT_WARN_LE : return "WARN_LE"; - case assertType::DT_CHECK_LE : return "CHECK_LE"; - case assertType::DT_REQUIRE_LE : return "REQUIRE_LE"; + DOCTEST_GENERATE_ASSERT_TYPE_CASES(EQ); + DOCTEST_GENERATE_ASSERT_TYPE_CASES(NE); + DOCTEST_GENERATE_ASSERT_TYPE_CASES(GT); + DOCTEST_GENERATE_ASSERT_TYPE_CASES(LT); + DOCTEST_GENERATE_ASSERT_TYPE_CASES(GE); + DOCTEST_GENERATE_ASSERT_TYPE_CASES(LE); - case assertType::DT_WARN_UNARY : return "WARN_UNARY"; - case assertType::DT_CHECK_UNARY : return "CHECK_UNARY"; - case assertType::DT_REQUIRE_UNARY : return "REQUIRE_UNARY"; - case assertType::DT_WARN_UNARY_FALSE : return "WARN_UNARY_FALSE"; - case assertType::DT_CHECK_UNARY_FALSE : return "CHECK_UNARY_FALSE"; - case assertType::DT_REQUIRE_UNARY_FALSE : return "REQUIRE_UNARY_FALSE"; + DOCTEST_GENERATE_ASSERT_TYPE_CASES(UNARY); + DOCTEST_GENERATE_ASSERT_TYPE_CASES(UNARY_FALSE); + + default: DOCTEST_INTERNAL_ERROR("Tried stringifying invalid assert type!"); } DOCTEST_MSVC_SUPPRESS_WARNING_POP - return ""; } // clang-format on @@ -3494,6 +3846,12 @@ const char* skipPathFromFilename(const char* file) { DOCTEST_CLANG_SUPPRESS_WARNING_POP DOCTEST_GCC_SUPPRESS_WARNING_POP +bool SubcaseSignature::operator==(const SubcaseSignature& other) const { + return m_line == other.m_line + && std::strcmp(m_file, other.m_file) == 0 + && m_name == other.m_name; +} + bool SubcaseSignature::operator<(const SubcaseSignature& other) const { if(m_line != other.m_line) return m_line < other.m_line; @@ -3502,45 +3860,53 @@ bool SubcaseSignature::operator<(const SubcaseSignature& other) const { return m_name.compare(other.m_name) < 0; } -IContextScope::IContextScope() = default; -IContextScope::~IContextScope() = default; +DOCTEST_DEFINE_INTERFACE(IContextScope) -#ifdef DOCTEST_CONFIG_TREAT_CHAR_STAR_AS_STRING -String toString(char* in) { return toString(static_cast(in)); } -// NOLINTNEXTLINE(clang-analyzer-cplusplus.NewDeleteLeaks) -String toString(const char* in) { return String("\"") + (in ? in : "{null string}") + "\""; } -#endif // DOCTEST_CONFIG_TREAT_CHAR_STAR_AS_STRING -String toString(bool in) { return in ? "true" : "false"; } -String toString(float in) { return fpToString(in, 5) + "f"; } -String toString(double in) { return fpToString(in, 10); } -String toString(double long in) { return fpToString(in, 15); } - -#define DOCTEST_TO_STRING_OVERLOAD(type, fmt) \ - String toString(type in) { \ - char buf[64]; \ - std::sprintf(buf, fmt, in); \ - return buf; \ +namespace detail { + void filldata::fill(std::ostream* stream, const void* in) { + if (in) { *stream << in; } + else { *stream << "nullptr"; } } -DOCTEST_TO_STRING_OVERLOAD(char, "%d") -DOCTEST_TO_STRING_OVERLOAD(char signed, "%d") -DOCTEST_TO_STRING_OVERLOAD(char unsigned, "%u") -DOCTEST_TO_STRING_OVERLOAD(int short, "%d") -DOCTEST_TO_STRING_OVERLOAD(int short unsigned, "%u") -DOCTEST_TO_STRING_OVERLOAD(int, "%d") -DOCTEST_TO_STRING_OVERLOAD(unsigned, "%u") -DOCTEST_TO_STRING_OVERLOAD(int long, "%ld") -DOCTEST_TO_STRING_OVERLOAD(int long unsigned, "%lu") -DOCTEST_TO_STRING_OVERLOAD(int long long, "%lld") -DOCTEST_TO_STRING_OVERLOAD(int long long unsigned, "%llu") + template + String toStreamLit(T t) { + std::ostream* os = tlssPush(); + os->operator<<(t); + return tlssPop(); + } +} -String toString(std::nullptr_t) { return "NULL"; } +#ifdef DOCTEST_CONFIG_TREAT_CHAR_STAR_AS_STRING +String toString(const char* in) { return String("\"") + (in ? in : "{null string}") + "\""; } +#endif // DOCTEST_CONFIG_TREAT_CHAR_STAR_AS_STRING #if DOCTEST_MSVC >= DOCTEST_COMPILER(19, 20, 0) -// see this issue on why this is needed: https://github.com/onqtam/doctest/issues/183 +// see this issue on why this is needed: https://github.com/doctest/doctest/issues/183 String toString(const std::string& in) { return in.c_str(); } #endif // VS 2019 +String toString(String in) { return in; } + +String toString(std::nullptr_t) { return "nullptr"; } + +String toString(bool in) { return in ? "true" : "false"; } + +String toString(float in) { return toStreamLit(in); } +String toString(double in) { return toStreamLit(in); } +String toString(double long in) { return toStreamLit(in); } + +String toString(char in) { return toStreamLit(static_cast(in)); } +String toString(char signed in) { return toStreamLit(static_cast(in)); } +String toString(char unsigned in) { return toStreamLit(static_cast(in)); } +String toString(short in) { return toStreamLit(in); } +String toString(short unsigned in) { return toStreamLit(in); } +String toString(signed in) { return toStreamLit(in); } +String toString(unsigned in) { return toStreamLit(in); } +String toString(long in) { return toStreamLit(in); } +String toString(long unsigned in) { return toStreamLit(in); } +String toString(long long in) { return toStreamLit(in); } +String toString(long long unsigned in) { return toStreamLit(in); } + Approx::Approx(double value) : m_epsilon(static_cast(std::numeric_limits::epsilon()) * 100) , m_scale(1.0) @@ -3580,11 +3946,25 @@ bool operator>(double lhs, const Approx& rhs) { return lhs > rhs.m_value && lhs bool operator>(const Approx& lhs, double rhs) { return lhs.m_value > rhs && lhs != rhs; } String toString(const Approx& in) { - // NOLINTNEXTLINE(clang-analyzer-cplusplus.NewDeleteLeaks) - return String("Approx( ") + doctest::toString(in.m_value) + " )"; + return "Approx( " + doctest::toString(in.m_value) + " )"; } const ContextOptions* getContextOptions() { return DOCTEST_BRANCH_ON_DISABLED(nullptr, g_cs); } +DOCTEST_MSVC_SUPPRESS_WARNING_WITH_PUSH(4738) +template +IsNaN::operator bool() const { + return std::isnan(value) ^ flipped; +} +DOCTEST_MSVC_SUPPRESS_WARNING_POP +template struct DOCTEST_INTERFACE_DEF IsNaN; +template struct DOCTEST_INTERFACE_DEF IsNaN; +template struct DOCTEST_INTERFACE_DEF IsNaN; +template +String toString(IsNaN in) { return String(in.flipped ? "! " : "") + "IsNaN( " + doctest::toString(in.value) + " )"; } +String toString(IsNaN in) { return toString(in); } +String toString(IsNaN in) { return toString(in); } +String toString(IsNaN in) { return toString(in); } + } // namespace doctest #ifdef DOCTEST_CONFIG_DISABLE @@ -3594,15 +3974,15 @@ Context::~Context() = default; void Context::applyCommandLine(int, const char* const*) {} void Context::addFilter(const char*, const char*) {} void Context::clearFilters() {} +void Context::setOption(const char*, bool) {} void Context::setOption(const char*, int) {} void Context::setOption(const char*, const char*) {} bool Context::shouldExit() { return false; } void Context::setAsDefaultForAssertsOutOfTestCases() {} void Context::setAssertHandler(detail::assert_handler) {} +void Context::setCout(std::ostream*) {} int Context::run() { return 0; } -IReporter::~IReporter() = default; - int IReporter::get_num_active_contexts() { return 0; } const IContextScope* const* IReporter::get_active_contexts() { return nullptr; } int IReporter::get_num_stringified_contexts() { return 0; } @@ -3635,7 +4015,7 @@ namespace doctest { namespace { // the int (priority) is part of the key for automatic sorting - sadly one can register a // reporter with a duplicate name and a different priority but hopefully that won't happen often :| - typedef std::map, reporterCreatorFunc> reporterMap; + using reporterMap = std::map, reporterCreatorFunc>; reporterMap& getReporters() { static reporterMap data; @@ -3667,8 +4047,8 @@ namespace detail { #ifndef DOCTEST_CONFIG_NO_EXCEPTIONS DOCTEST_NORETURN void throwException() { g_cs->shouldLogCurrentException = false; - throw TestFailureException(); - } // NOLINT(cert-err60-cpp) + throw TestFailureException(); // NOLINT(hicpp-exception-baseclass) + } #else // DOCTEST_CONFIG_NO_EXCEPTIONS void throwException() {} #endif // DOCTEST_CONFIG_NO_EXCEPTIONS @@ -3714,91 +4094,132 @@ namespace { return !*wild; } - //// C string hash function (djb2) - taken from http://www.cse.yorku.ca/~oz/hash.html - //unsigned hashStr(unsigned const char* str) { - // unsigned long hash = 5381; - // char c; - // while((c = *str++)) - // hash = ((hash << 5) + hash) + c; // hash * 33 + c - // return hash; - //} - // checks if the name matches any of the filters (and can be configured what to do when empty) bool matchesAny(const char* name, const std::vector& filters, bool matchEmpty, - bool caseSensitive) { - if(filters.empty() && matchEmpty) + bool caseSensitive) { + if (filters.empty() && matchEmpty) return true; - for(auto& curr : filters) - if(wildcmp(name, curr.c_str(), caseSensitive)) + for (auto& curr : filters) + if (wildcmp(name, curr.c_str(), caseSensitive)) return true; return false; } + + unsigned long long hash(unsigned long long a, unsigned long long b) { + return (a << 5) + b; + } + + // C string hash function (djb2) - taken from http://www.cse.yorku.ca/~oz/hash.html + unsigned long long hash(const char* str) { + unsigned long long hash = 5381; + char c; + while ((c = *str++)) + hash = ((hash << 5) + hash) + c; // hash * 33 + c + return hash; + } + + unsigned long long hash(const SubcaseSignature& sig) { + return hash(hash(hash(sig.m_file), hash(sig.m_name.c_str())), sig.m_line); + } + + unsigned long long hash(const std::vector& sigs, size_t count) { + unsigned long long running = 0; + auto end = sigs.begin() + count; + for (auto it = sigs.begin(); it != end; it++) { + running = hash(running, hash(*it)); + } + return running; + } + + unsigned long long hash(const std::vector& sigs) { + unsigned long long running = 0; + for (const SubcaseSignature& sig : sigs) { + running = hash(running, hash(sig)); + } + return running; + } } // namespace namespace detail { + bool Subcase::checkFilters() { + if (g_cs->subcaseStack.size() < size_t(g_cs->subcase_filter_levels)) { + if (!matchesAny(m_signature.m_name.c_str(), g_cs->filters[6], true, g_cs->case_sensitive)) + return true; + if (matchesAny(m_signature.m_name.c_str(), g_cs->filters[7], false, g_cs->case_sensitive)) + return true; + } + return false; + } Subcase::Subcase(const String& name, const char* file, int line) : m_signature({name, file, line}) { - auto* s = g_cs; + if (!g_cs->reachedLeaf) { + if (g_cs->nextSubcaseStack.size() <= g_cs->subcaseStack.size() + || g_cs->nextSubcaseStack[g_cs->subcaseStack.size()] == m_signature) { + // Going down. + if (checkFilters()) { return; } - // check subcase filters - if(s->subcasesStack.size() < size_t(s->subcase_filter_levels)) { - if(!matchesAny(m_signature.m_name.c_str(), s->filters[6], true, s->case_sensitive)) - return; - if(matchesAny(m_signature.m_name.c_str(), s->filters[7], false, s->case_sensitive)) - return; + g_cs->subcaseStack.push_back(m_signature); + g_cs->currentSubcaseDepth++; + m_entered = true; + DOCTEST_ITERATE_THROUGH_REPORTERS(subcase_start, m_signature); + } + } else { + if (g_cs->subcaseStack[g_cs->currentSubcaseDepth] == m_signature) { + // This subcase is reentered via control flow. + g_cs->currentSubcaseDepth++; + m_entered = true; + DOCTEST_ITERATE_THROUGH_REPORTERS(subcase_start, m_signature); + } else if (g_cs->nextSubcaseStack.size() <= g_cs->currentSubcaseDepth + && g_cs->fullyTraversedSubcases.find(hash(hash(g_cs->subcaseStack, g_cs->currentSubcaseDepth), hash(m_signature))) + == g_cs->fullyTraversedSubcases.end()) { + if (checkFilters()) { return; } + // This subcase is part of the one to be executed next. + g_cs->nextSubcaseStack.clear(); + g_cs->nextSubcaseStack.insert(g_cs->nextSubcaseStack.end(), + g_cs->subcaseStack.begin(), g_cs->subcaseStack.begin() + g_cs->currentSubcaseDepth); + g_cs->nextSubcaseStack.push_back(m_signature); + } } - - // if a Subcase on the same level has already been entered - if(s->subcasesStack.size() < size_t(s->subcasesCurrentMaxLevel)) { - s->should_reenter = true; - return; - } - - // push the current signature to the stack so we can check if the - // current stack + the current new subcase have been traversed - s->subcasesStack.push_back(m_signature); - if(s->subcasesPassed.count(s->subcasesStack) != 0) { - // pop - revert to previous stack since we've already passed this - s->subcasesStack.pop_back(); - return; - } - - s->subcasesCurrentMaxLevel = s->subcasesStack.size(); - m_entered = true; - - DOCTEST_ITERATE_THROUGH_REPORTERS(subcase_start, m_signature); } - DOCTEST_MSVC_SUPPRESS_WARNING_WITH_PUSH(4996) // std::uncaught_exception is deprecated in C++17 - DOCTEST_GCC_SUPPRESS_WARNING_WITH_PUSH("-Wdeprecated-declarations") + DOCTEST_MSVC_SUPPRESS_WARNING_WITH_PUSH(4996) // std::uncaught_exception is deprecated in C++17 + DOCTEST_GCC_SUPPRESS_WARNING_WITH_PUSH("-Wdeprecated-declarations") DOCTEST_CLANG_SUPPRESS_WARNING_WITH_PUSH("-Wdeprecated-declarations") Subcase::~Subcase() { - if(m_entered) { - // only mark the subcase stack as passed if no subcases have been skipped - if(g_cs->should_reenter == false) - g_cs->subcasesPassed.insert(g_cs->subcasesStack); - g_cs->subcasesStack.pop_back(); + if (m_entered) { + g_cs->currentSubcaseDepth--; + + if (!g_cs->reachedLeaf) { + // Leaf. + g_cs->fullyTraversedSubcases.insert(hash(g_cs->subcaseStack)); + g_cs->nextSubcaseStack.clear(); + g_cs->reachedLeaf = true; + } else if (g_cs->nextSubcaseStack.empty()) { + // All children are finished. + g_cs->fullyTraversedSubcases.insert(hash(g_cs->subcaseStack)); + } #if defined(__cpp_lib_uncaught_exceptions) && __cpp_lib_uncaught_exceptions >= 201411L && (!defined(__MAC_OS_X_VERSION_MIN_REQUIRED) || __MAC_OS_X_VERSION_MIN_REQUIRED >= 101200) if(std::uncaught_exceptions() > 0 #else if(std::uncaught_exception() #endif - && g_cs->shouldLogCurrentException) { + && g_cs->shouldLogCurrentException) { DOCTEST_ITERATE_THROUGH_REPORTERS( test_case_exception, {"exception thrown in subcase - will translate later " - "when the whole test case has been exited (cannot " - "translate while there is an active exception)", - false}); + "when the whole test case has been exited (cannot " + "translate while there is an active exception)", + false}); g_cs->shouldLogCurrentException = false; } + DOCTEST_ITERATE_THROUGH_REPORTERS(subcase_end, DOCTEST_EMPTY); } } - DOCTEST_CLANG_SUPPRESS_WARNING_POP - DOCTEST_GCC_SUPPRESS_WARNING_POP + DOCTEST_CLANG_SUPPRESS_WARNING_POP + DOCTEST_GCC_SUPPRESS_WARNING_POP DOCTEST_MSVC_SUPPRESS_WARNING_POP Subcase::operator bool() const { return m_entered; } @@ -3812,20 +4233,11 @@ namespace detail { TestSuite& TestSuite::operator*(const char* in) { m_test_suite = in; - // clear state - m_description = nullptr; - m_skip = false; - m_no_breaks = false; - m_no_output = false; - m_may_fail = false; - m_should_fail = false; - m_expected_failures = 0; - m_timeout = 0; return *this; } TestCase::TestCase(funcType test, const char* file, unsigned line, const TestSuite& test_suite, - const char* type, int template_id) { + const String& type, int template_id) { m_file = file; m_line = line; m_name = nullptr; // will be later overridden in operator* @@ -3850,10 +4262,8 @@ namespace detail { } DOCTEST_MSVC_SUPPRESS_WARNING_WITH_PUSH(26434) // hides a non-virtual function - DOCTEST_MSVC_SUPPRESS_WARNING(26437) // Do not slice TestCase& TestCase::operator=(const TestCase& other) { - static_cast(*this) = static_cast(other); - + TestCaseData::operator=(other); m_test = other.m_test; m_type = other.m_type; m_template_id = other.m_template_id; @@ -3869,7 +4279,7 @@ namespace detail { m_name = in; // make a new name with an appended type for templated test case if(m_template_id != -1) { - m_full_name = String(m_name) + m_type; + m_full_name = String(m_name) + "<" + m_type + ">"; // redirect the name to point to the newly constructed full name m_name = m_full_name.c_str(); } @@ -3925,29 +4335,6 @@ namespace { return suiteOrderComparator(lhs, rhs); } -#ifdef DOCTEST_CONFIG_COLORS_WINDOWS - HANDLE g_stdoutHandle; - WORD g_origFgAttrs; - WORD g_origBgAttrs; - bool g_attrsInitted = false; - - int colors_init() { - if(!g_attrsInitted) { - g_stdoutHandle = GetStdHandle(STD_OUTPUT_HANDLE); - g_attrsInitted = true; - CONSOLE_SCREEN_BUFFER_INFO csbiInfo; - GetConsoleScreenBufferInfo(g_stdoutHandle, &csbiInfo); - g_origFgAttrs = csbiInfo.wAttributes & ~(BACKGROUND_GREEN | BACKGROUND_RED | - BACKGROUND_BLUE | BACKGROUND_INTENSITY); - g_origBgAttrs = csbiInfo.wAttributes & ~(FOREGROUND_GREEN | FOREGROUND_RED | - FOREGROUND_BLUE | FOREGROUND_INTENSITY); - } - return 0; - } - - int dumy_init_console_colors = colors_init(); -#endif // DOCTEST_CONFIG_COLORS_WINDOWS - DOCTEST_CLANG_SUPPRESS_WARNING_WITH_PUSH("-Wdeprecated-declarations") void color_to_stream(std::ostream& s, Color::Enum code) { static_cast(s); // for DOCTEST_CONFIG_COLORS_NONE or DOCTEST_CONFIG_COLORS_WINDOWS @@ -3981,10 +4368,26 @@ namespace { #ifdef DOCTEST_CONFIG_COLORS_WINDOWS if(g_no_colors || - (isatty(fileno(stdout)) == false && getContextOptions()->force_colors == false)) + (_isatty(_fileno(stdout)) == false && getContextOptions()->force_colors == false)) return; -#define DOCTEST_SET_ATTR(x) SetConsoleTextAttribute(g_stdoutHandle, x | g_origBgAttrs) + static struct ConsoleHelper { + HANDLE stdoutHandle; + WORD origFgAttrs; + WORD origBgAttrs; + + ConsoleHelper() { + stdoutHandle = GetStdHandle(STD_OUTPUT_HANDLE); + CONSOLE_SCREEN_BUFFER_INFO csbiInfo; + GetConsoleScreenBufferInfo(stdoutHandle, &csbiInfo); + origFgAttrs = csbiInfo.wAttributes & ~(BACKGROUND_GREEN | BACKGROUND_RED | + BACKGROUND_BLUE | BACKGROUND_INTENSITY); + origBgAttrs = csbiInfo.wAttributes & ~(FOREGROUND_GREEN | FOREGROUND_RED | + FOREGROUND_BLUE | FOREGROUND_INTENSITY); + } + } ch; + +#define DOCTEST_SET_ATTR(x) SetConsoleTextAttribute(ch.stdoutHandle, x | ch.origBgAttrs) // clang-format off switch (code) { @@ -4001,7 +4404,7 @@ namespace { case Color::BrightWhite: DOCTEST_SET_ATTR(FOREGROUND_INTENSITY | FOREGROUND_GREEN | FOREGROUND_RED | FOREGROUND_BLUE); break; case Color::None: case Color::Bright: // invalid - default: DOCTEST_SET_ATTR(g_origFgAttrs); + default: DOCTEST_SET_ATTR(ch.origFgAttrs); } // clang-format on #endif // DOCTEST_CONFIG_COLORS_WINDOWS @@ -4118,35 +4521,22 @@ namespace detail { getExceptionTranslators().push_back(et); } -#ifdef DOCTEST_CONFIG_TREAT_CHAR_STAR_AS_STRING - void toStream(std::ostream* s, char* in) { *s << in; } - void toStream(std::ostream* s, const char* in) { *s << in; } -#endif // DOCTEST_CONFIG_TREAT_CHAR_STAR_AS_STRING - void toStream(std::ostream* s, bool in) { *s << std::boolalpha << in << std::noboolalpha; } - void toStream(std::ostream* s, float in) { *s << in; } - void toStream(std::ostream* s, double in) { *s << in; } - void toStream(std::ostream* s, double long in) { *s << in; } - - void toStream(std::ostream* s, char in) { *s << in; } - void toStream(std::ostream* s, char signed in) { *s << in; } - void toStream(std::ostream* s, char unsigned in) { *s << in; } - void toStream(std::ostream* s, int short in) { *s << in; } - void toStream(std::ostream* s, int short unsigned in) { *s << in; } - void toStream(std::ostream* s, int in) { *s << in; } - void toStream(std::ostream* s, int unsigned in) { *s << in; } - void toStream(std::ostream* s, int long in) { *s << in; } - void toStream(std::ostream* s, int long unsigned in) { *s << in; } - void toStream(std::ostream* s, int long long in) { *s << in; } - void toStream(std::ostream* s, int long long unsigned in) { *s << in; } - DOCTEST_THREAD_LOCAL std::vector g_infoContexts; // for logging with INFO() ContextScopeBase::ContextScopeBase() { g_infoContexts.push_back(this); } - DOCTEST_MSVC_SUPPRESS_WARNING_WITH_PUSH(4996) // std::uncaught_exception is deprecated in C++17 - DOCTEST_GCC_SUPPRESS_WARNING_WITH_PUSH("-Wdeprecated-declarations") + ContextScopeBase::ContextScopeBase(ContextScopeBase&& other) noexcept { + if (other.need_to_destroy) { + other.destroy(); + } + other.need_to_destroy = false; + g_infoContexts.push_back(this); + } + + DOCTEST_MSVC_SUPPRESS_WARNING_WITH_PUSH(4996) // std::uncaught_exception is deprecated in C++17 + DOCTEST_GCC_SUPPRESS_WARNING_WITH_PUSH("-Wdeprecated-declarations") DOCTEST_CLANG_SUPPRESS_WARNING_WITH_PUSH("-Wdeprecated-declarations") // destroy cannot be inlined into the destructor because that would mean calling stringify after @@ -4165,8 +4555,8 @@ namespace detail { g_infoContexts.pop_back(); } - DOCTEST_CLANG_SUPPRESS_WARNING_POP - DOCTEST_GCC_SUPPRESS_WARNING_POP + DOCTEST_CLANG_SUPPRESS_WARNING_POP + DOCTEST_GCC_SUPPRESS_WARNING_POP DOCTEST_MSVC_SUPPRESS_WARNING_POP } // namespace detail namespace { @@ -4207,10 +4597,10 @@ namespace { static LONG CALLBACK handleException(PEXCEPTION_POINTERS ExceptionInfo) { // Multiple threads may enter this filter/handler at once. We want the error message to be printed on the // console just once no matter how many threads have crashed. - static std::mutex mutex; + DOCTEST_DECLARE_STATIC_MUTEX(mutex) static bool execute = true; { - std::lock_guard lock(mutex); + DOCTEST_LOCK_MUTEX(mutex) if(execute) { bool reported = false; for(size_t i = 0; i < DOCTEST_COUNTOF(signalDefs); ++i) { @@ -4313,7 +4703,7 @@ namespace { static unsigned int prev_abort_behavior; static int prev_report_mode; static _HFILE prev_report_file; - static void (*prev_sigabrt_handler)(int); + static void (DOCTEST_CDECL *prev_sigabrt_handler)(int); static std::terminate_handler original_terminate_handler; static bool isSet; static ULONG guaranteeSize; @@ -4325,7 +4715,7 @@ namespace { unsigned int FatalConditionHandler::prev_abort_behavior; int FatalConditionHandler::prev_report_mode; _HFILE FatalConditionHandler::prev_report_file; - void (*FatalConditionHandler::prev_sigabrt_handler)(int); + void (DOCTEST_CDECL *FatalConditionHandler::prev_sigabrt_handler)(int); std::terminate_handler FatalConditionHandler::original_terminate_handler; bool FatalConditionHandler::isSet = false; ULONG FatalConditionHandler::guaranteeSize = 0; @@ -4383,7 +4773,7 @@ namespace { sigStack.ss_flags = 0; sigaltstack(&sigStack, &oldSigStack); struct sigaction sa = {}; - sa.sa_handler = handleSignal; // NOLINT + sa.sa_handler = handleSignal; sa.sa_flags = SA_ONSTACK; for(std::size_t i = 0; i < DOCTEST_COUNTOF(signalDefs); ++i) { sigaction(signalDefs[i].id, &sa, &oldSigActions[i]); @@ -4422,7 +4812,7 @@ namespace { #define DOCTEST_OUTPUT_DEBUG_STRING(text) ::OutputDebugStringA(text) #else // TODO: integration with XCode and other IDEs -#define DOCTEST_OUTPUT_DEBUG_STRING(text) // NOLINT(clang-diagnostic-unused-macros) +#define DOCTEST_OUTPUT_DEBUG_STRING(text) #endif // Platform void addAssert(assertType::Enum at) { @@ -4441,8 +4831,8 @@ namespace { DOCTEST_ITERATE_THROUGH_REPORTERS(test_case_exception, {message.c_str(), true}); - while(g_cs->subcasesStack.size()) { - g_cs->subcasesStack.pop_back(); + while (g_cs->subcaseStack.size()) { + g_cs->subcaseStack.pop_back(); DOCTEST_ITERATE_THROUGH_REPORTERS(subcase_end, DOCTEST_EMPTY); } @@ -4454,25 +4844,26 @@ namespace { } #endif // DOCTEST_CONFIG_POSIX_SIGNALS || DOCTEST_CONFIG_WINDOWS_SEH } // namespace + +AssertData::AssertData(assertType::Enum at, const char* file, int line, const char* expr, + const char* exception_type, const StringContains& exception_string) + : m_test_case(g_cs->currentTest), m_at(at), m_file(file), m_line(line), m_expr(expr), + m_failed(true), m_threw(false), m_threw_as(false), m_exception_type(exception_type), + m_exception_string(exception_string) { +#if DOCTEST_MSVC + if (m_expr[0] == ' ') // this happens when variadic macros are disabled under MSVC + ++m_expr; +#endif // MSVC +} + namespace detail { + ResultBuilder::ResultBuilder(assertType::Enum at, const char* file, int line, const char* expr, + const char* exception_type, const String& exception_string) + : AssertData(at, file, line, expr, exception_type, exception_string) { } ResultBuilder::ResultBuilder(assertType::Enum at, const char* file, int line, const char* expr, - const char* exception_type, const char* exception_string) { - m_test_case = g_cs->currentTest; - m_at = at; - m_file = file; - m_line = line; - m_expr = expr; - m_failed = true; - m_threw = false; - m_threw_as = false; - m_exception_type = exception_type; - m_exception_string = exception_string; -#if DOCTEST_MSVC - if(m_expr[0] == ' ') // this happens when variadic macros are disabled under MSVC - ++m_expr; -#endif // MSVC - } + const char* exception_type, const Contains& exception_string) + : AssertData(at, file, line, expr, exception_type, exception_string) { } void ResultBuilder::setResult(const Result& res) { m_decomp = res.m_decomp; @@ -4488,17 +4879,17 @@ namespace detail { if(m_at & assertType::is_throws) { //!OCLINT bitwise operator in conditional m_failed = !m_threw; } else if((m_at & assertType::is_throws_as) && (m_at & assertType::is_throws_with)) { //!OCLINT - m_failed = !m_threw_as || (m_exception != m_exception_string); + m_failed = !m_threw_as || !m_exception_string.check(m_exception); } else if(m_at & assertType::is_throws_as) { //!OCLINT bitwise operator in conditional m_failed = !m_threw_as; } else if(m_at & assertType::is_throws_with) { //!OCLINT bitwise operator in conditional - m_failed = m_exception != m_exception_string; + m_failed = !m_exception_string.check(m_exception); } else if(m_at & assertType::is_nothrow) { //!OCLINT bitwise operator in conditional m_failed = m_threw; } if(m_exception.size()) - m_exception = String("\"") + m_exception + "\""; + m_exception = "\"" + m_exception + "\""; if(is_running_in_test) { addAssert(m_at); @@ -4526,8 +4917,8 @@ namespace detail { std::abort(); } - void decomp_assert(assertType::Enum at, const char* file, int line, const char* expr, - Result result) { + bool decomp_assert(assertType::Enum at, const char* file, int line, const char* expr, + const Result& result) { bool failed = !result.m_passed; // ################################################################################### @@ -4536,21 +4927,29 @@ namespace detail { // ################################################################################### DOCTEST_ASSERT_OUT_OF_TESTS(result.m_decomp); DOCTEST_ASSERT_IN_TESTS(result.m_decomp); - // NOLINTNEXTLINE(clang-analyzer-cplusplus.NewDeleteLeaks) + return !failed; } MessageBuilder::MessageBuilder(const char* file, int line, assertType::Enum severity) { - m_stream = getTlsOss(); + m_stream = tlssPush(); m_file = file; m_line = line; m_severity = severity; } - IExceptionTranslator::IExceptionTranslator() = default; - IExceptionTranslator::~IExceptionTranslator() = default; + MessageBuilder::~MessageBuilder() { + if (!logged) + tlssPop(); + } + + DOCTEST_DEFINE_INTERFACE(IExceptionTranslator) bool MessageBuilder::log() { - m_string = getTlsOssResult(); + if (!logged) { + m_string = tlssPop(); + logged = true; + } + DOCTEST_ITERATE_THROUGH_REPORTERS(log_message, *this); const bool isWarn = m_severity & assertType::is_warn; @@ -4569,29 +4968,10 @@ namespace detail { if(m_severity & assertType::is_require) //!OCLINT bitwise operator in conditional throwException(); } - - MessageBuilder::~MessageBuilder() = default; } // namespace detail namespace { using namespace detail; - template - DOCTEST_NORETURN void throw_exception(Ex const& e) { -#ifndef DOCTEST_CONFIG_NO_EXCEPTIONS - throw e; -#else // DOCTEST_CONFIG_NO_EXCEPTIONS - std::cerr << "doctest will terminate because it needed to throw an exception.\n" - << "The message was: " << e.what() << '\n'; - std::terminate(); -#endif // DOCTEST_CONFIG_NO_EXCEPTIONS - } - -#ifndef DOCTEST_INTERNAL_ERROR -#define DOCTEST_INTERNAL_ERROR(msg) \ - throw_exception(std::logic_error( \ - __FILE__ ":" DOCTEST_TOSTR(__LINE__) ": Internal doctest error: " msg)) -#endif // DOCTEST_INTERNAL_ERROR - // clang-format off // ================================================================================================= @@ -4673,10 +5053,10 @@ namespace { void ensureTagClosed(); - private: - void writeDeclaration(); + private: + void newlineIfNecessary(); bool m_tagIsOpen = false; @@ -4865,7 +5245,7 @@ namespace { XmlWriter::XmlWriter( std::ostream& os ) : m_os( os ) { - writeDeclaration(); + // writeDeclaration(); // called explicitly by the reporters that use the writer class - see issue #627 } XmlWriter::~XmlWriter() { @@ -4976,8 +5356,8 @@ namespace { struct XmlReporter : public IReporter { - XmlWriter xml; - std::mutex mutex; + XmlWriter xml; + DOCTEST_DECLARE_MUTEX(mutex) // caching pointers/references to objects of these types - safe to do const ContextOptions& opt; @@ -5054,7 +5434,8 @@ namespace { xml.scopedElement("TestCase").writeAttribute("name", in.data[i]->m_name) .writeAttribute("testsuite", in.data[i]->m_test_suite) .writeAttribute("filename", skipPathFromFilename(in.data[i]->m_file.c_str())) - .writeAttribute("line", line(in.data[i]->m_line)); + .writeAttribute("line", line(in.data[i]->m_line)) + .writeAttribute("skipped", in.data[i]->m_skip); } xml.scopedElement("OverallResultsTestCases") .writeAttribute("unskipped", in.run_stats->numTestCasesPassingFilters); @@ -5070,6 +5451,8 @@ namespace { } void test_run_start() override { + xml.writeDeclaration(); + // remove .exe extension - mainly to have the same output on UNIX and Windows std::string binary_name = skipPathFromFilename(opt.binary_name.c_str()); #ifdef DOCTEST_PLATFORM_WINDOWS @@ -5124,7 +5507,8 @@ namespace { xml.startElement("OverallResultsAsserts") .writeAttribute("successes", st.numAssertsCurrentTest - st.numAssertsFailedCurrentTest) - .writeAttribute("failures", st.numAssertsFailedCurrentTest); + .writeAttribute("failures", st.numAssertsFailedCurrentTest) + .writeAttribute("test_case_success", st.testCaseSuccess); if(opt.duration) xml.writeAttribute("duration", st.seconds); if(tc->m_expected_failures) @@ -5135,7 +5519,7 @@ namespace { } void test_case_exception(const TestCaseException& e) override { - std::lock_guard lock(mutex); + DOCTEST_LOCK_MUTEX(mutex) xml.scopedElement("Exception") .writeAttribute("crash", e.is_crash) @@ -5143,8 +5527,6 @@ namespace { } void subcase_start(const SubcaseSignature& in) override { - std::lock_guard lock(mutex); - xml.startElement("SubCase") .writeAttribute("name", in.m_name) .writeAttribute("filename", skipPathFromFilename(in.m_file)) @@ -5158,7 +5540,7 @@ namespace { if(!rb.m_failed && !opt.success) return; - std::lock_guard lock(mutex); + DOCTEST_LOCK_MUTEX(mutex) xml.startElement("Expression") .writeAttribute("success", !rb.m_failed) @@ -5174,7 +5556,7 @@ namespace { if(rb.m_at & assertType::is_throws_as) xml.scopedElement("ExpectedException").writeText(rb.m_exception_type); if(rb.m_at & assertType::is_throws_with) - xml.scopedElement("ExpectedExceptionString").writeText(rb.m_exception_string); + xml.scopedElement("ExpectedExceptionString").writeText(rb.m_exception_string.c_str()); if((rb.m_at & assertType::is_normal) && !rb.m_threw) xml.scopedElement("Expanded").writeText(rb.m_decomp.c_str()); @@ -5184,7 +5566,7 @@ namespace { } void log_message(const MessageData& mb) override { - std::lock_guard lock(mutex); + DOCTEST_LOCK_MUTEX(mutex) xml.startElement("Message") .writeAttribute("type", failureString(mb.m_severity)) @@ -5220,7 +5602,8 @@ namespace { } else if((rb.m_at & assertType::is_throws_as) && (rb.m_at & assertType::is_throws_with)) { //!OCLINT s << Color::Cyan << assertString(rb.m_at) << "( " << rb.m_expr << ", \"" - << rb.m_exception_string << "\", " << rb.m_exception_type << " ) " << Color::None; + << rb.m_exception_string.c_str() + << "\", " << rb.m_exception_type << " ) " << Color::None; if(rb.m_threw) { if(!rb.m_failed) { s << "threw as expected!\n"; @@ -5241,7 +5624,8 @@ namespace { } else if(rb.m_at & assertType::is_throws_with) { //!OCLINT bitwise operator in conditional s << Color::Cyan << assertString(rb.m_at) << "( " << rb.m_expr << ", \"" - << rb.m_exception_string << "\" ) " << Color::None + << rb.m_exception_string.c_str() + << "\" ) " << Color::None << (rb.m_threw ? (!rb.m_failed ? "threw as expected!" : "threw a DIFFERENT exception: ") : "did NOT throw at all!") @@ -5266,8 +5650,8 @@ namespace { // - more attributes in tags struct JUnitReporter : public IReporter { - XmlWriter xml; - std::mutex mutex; + XmlWriter xml; + DOCTEST_DECLARE_MUTEX(mutex) Timer timer; std::vector deepestSubcaseStackNames; @@ -5363,9 +5747,13 @@ namespace { // WHAT FOLLOWS ARE OVERRIDES OF THE VIRTUAL METHODS OF THE REPORTER INTERFACE // ========================================================================================= - void report_query(const QueryData&) override {} + void report_query(const QueryData&) override { + xml.writeDeclaration(); + } - void test_run_start() override {} + void test_run_start() override { + xml.writeDeclaration(); + } void test_run_end(const TestRunStats& p) override { // remove .exe extension - mainly to have the same output on UNIX and Windows @@ -5435,12 +5823,11 @@ namespace { } void test_case_exception(const TestCaseException& e) override { - std::lock_guard lock(mutex); + DOCTEST_LOCK_MUTEX(mutex) testCaseData.addError("exception", e.error_string.c_str()); } void subcase_start(const SubcaseSignature& in) override { - std::lock_guard lock(mutex); deepestSubcaseStackNames.push_back(in.m_name); } @@ -5450,7 +5837,7 @@ namespace { if(!rb.m_failed) // report only failures & ignore the `success` option return; - std::lock_guard lock(mutex); + DOCTEST_LOCK_MUTEX(mutex) std::ostringstream os; os << skipPathFromFilename(rb.m_file) << (opt.gnu_file_line ? ":" : "(") @@ -5501,7 +5888,7 @@ namespace { bool hasLoggedCurrentTestStart; std::vector subcasesStack; size_t currentSubcaseLevel; - std::mutex mutex; + DOCTEST_DECLARE_MUTEX(mutex) // caching pointers/references to objects of these types - safe to do const ContextOptions& opt; @@ -5606,9 +5993,11 @@ namespace { } void printIntro() { - printVersion(); - s << Color::Cyan << "[doctest] " << Color::None - << "run with \"--" DOCTEST_OPTIONS_PREFIX_DISPLAY "help\" for options\n"; + if(opt.no_intro == false) { + printVersion(); + s << Color::Cyan << "[doctest] " << Color::None + << "run with \"--" DOCTEST_OPTIONS_PREFIX_DISPLAY "help\" for options\n"; + } } void printHelp() { @@ -5693,12 +6082,18 @@ namespace { << Whitespace(sizePrefixDisplay*1) << "exits after the tests finish\n"; s << " -" DOCTEST_OPTIONS_PREFIX_DISPLAY "d, --" DOCTEST_OPTIONS_PREFIX_DISPLAY "duration= " << Whitespace(sizePrefixDisplay*1) << "prints the time duration of each test\n"; + s << " -" DOCTEST_OPTIONS_PREFIX_DISPLAY "m, --" DOCTEST_OPTIONS_PREFIX_DISPLAY "minimal= " + << Whitespace(sizePrefixDisplay*1) << "minimal console output (only failures)\n"; + s << " -" DOCTEST_OPTIONS_PREFIX_DISPLAY "q, --" DOCTEST_OPTIONS_PREFIX_DISPLAY "quiet= " + << Whitespace(sizePrefixDisplay*1) << "no console output\n"; s << " -" DOCTEST_OPTIONS_PREFIX_DISPLAY "nt, --" DOCTEST_OPTIONS_PREFIX_DISPLAY "no-throw= " << Whitespace(sizePrefixDisplay*1) << "skips exceptions-related assert checks\n"; s << " -" DOCTEST_OPTIONS_PREFIX_DISPLAY "ne, --" DOCTEST_OPTIONS_PREFIX_DISPLAY "no-exitcode= " << Whitespace(sizePrefixDisplay*1) << "returns (or exits) always with success\n"; s << " -" DOCTEST_OPTIONS_PREFIX_DISPLAY "nr, --" DOCTEST_OPTIONS_PREFIX_DISPLAY "no-run= " << Whitespace(sizePrefixDisplay*1) << "skips all runtime doctest operations\n"; + s << " -" DOCTEST_OPTIONS_PREFIX_DISPLAY "ni, --" DOCTEST_OPTIONS_PREFIX_DISPLAY "no-intro= " + << Whitespace(sizePrefixDisplay*1) << "omit the framework intro in the output\n"; s << " -" DOCTEST_OPTIONS_PREFIX_DISPLAY "nv, --" DOCTEST_OPTIONS_PREFIX_DISPLAY "no-version= " << Whitespace(sizePrefixDisplay*1) << "omit the framework version in the output\n"; s << " -" DOCTEST_OPTIONS_PREFIX_DISPLAY "nc, --" DOCTEST_OPTIONS_PREFIX_DISPLAY "no-colors= " @@ -5736,22 +6131,6 @@ namespace { printReporters(getReporters(), "reporters"); } - void list_query_results() { - separator_to_stream(); - if(opt.count || opt.list_test_cases) { - s << Color::Cyan << "[doctest] " << Color::None - << "unskipped test cases passing the current filters: " - << g_cs->numTestCasesPassingFilters << "\n"; - } else if(opt.list_test_suites) { - s << Color::Cyan << "[doctest] " << Color::None - << "unskipped test cases passing the current filters: " - << g_cs->numTestCasesPassingFilters << "\n"; - s << Color::Cyan << "[doctest] " << Color::None - << "test suites with unskipped test cases passing the current filters: " - << g_cs->numTestSuitesPassingFilters << "\n"; - } - } - // ========================================================================================= // WHAT FOLLOWS ARE OVERRIDES OF THE VIRTUAL METHODS OF THE REPORTER INTERFACE // ========================================================================================= @@ -5797,9 +6176,15 @@ namespace { } } - void test_run_start() override { printIntro(); } + void test_run_start() override { + if(!opt.minimal) + printIntro(); + } void test_run_end(const TestRunStats& p) override { + if(opt.minimal && p.numTestCasesFailed == 0) + return; + separator_to_stream(); s << std::dec; @@ -5849,7 +6234,7 @@ namespace { // log the preamble of the test case only if there is something // else to print - something other than that an assert has failed if(opt.duration || - (st.failure_flags && st.failure_flags != TestCaseFailureReason::AssertFailure)) + (st.failure_flags && st.failure_flags != static_cast(TestCaseFailureReason::AssertFailure))) logTestStart(); if(opt.duration) @@ -5880,6 +6265,7 @@ namespace { } void test_case_exception(const TestCaseException& e) override { + DOCTEST_LOCK_MUTEX(mutex) if(tc->m_no_output) return; @@ -5904,14 +6290,12 @@ namespace { } void subcase_start(const SubcaseSignature& subc) override { - std::lock_guard lock(mutex); subcasesStack.push_back(subc); ++currentSubcaseLevel; hasLoggedCurrentTestStart = false; } void subcase_end() override { - std::lock_guard lock(mutex); --currentSubcaseLevel; hasLoggedCurrentTestStart = false; } @@ -5920,7 +6304,7 @@ namespace { if((!rb.m_failed && !opt.success) || tc->m_no_output) return; - std::lock_guard lock(mutex); + DOCTEST_LOCK_MUTEX(mutex) logTestStart(); @@ -5936,7 +6320,7 @@ namespace { if(tc->m_no_output) return; - std::lock_guard lock(mutex); + DOCTEST_LOCK_MUTEX(mutex) logTestStart(); @@ -6047,18 +6431,42 @@ namespace { std::vector& res) { String filtersString; if(parseOption(argc, argv, pattern, &filtersString)) { - // tokenize with "," as a separator - // cppcheck-suppress strtokCalled - DOCTEST_CLANG_SUPPRESS_WARNING_WITH_PUSH("-Wdeprecated-declarations") - auto pch = std::strtok(filtersString.c_str(), ","); // modifies the string - while(pch != nullptr) { - if(strlen(pch)) - res.push_back(pch); - // uses the strtok() internal state to go to the next token - // cppcheck-suppress strtokCalled - pch = std::strtok(nullptr, ","); + // tokenize with "," as a separator, unless escaped with backslash + std::ostringstream s; + auto flush = [&s, &res]() { + auto string = s.str(); + if(string.size() > 0) { + res.push_back(string.c_str()); + } + s.str(""); + }; + + bool seenBackslash = false; + const char* current = filtersString.c_str(); + const char* end = current + strlen(current); + while(current != end) { + char character = *current++; + if(seenBackslash) { + seenBackslash = false; + if(character == ',' || character == '\\') { + s.put(character); + continue; + } + s.put('\\'); + } + if(character == '\\') { + seenBackslash = true; + } else if(character == ',') { + flush(); + } else { + s.put(character); + } } - DOCTEST_CLANG_SUPPRESS_WARNING_POP + + if(seenBackslash) { + s.put('\\'); + } + flush(); return true; } return false; @@ -6077,30 +6485,30 @@ namespace { if(!parseOption(argc, argv, pattern, &parsedValue)) return false; - if(type == 0) { + if(type) { + // integer + // TODO: change this to use std::stoi or something else! currently it uses undefined behavior - assumes '0' on failed parse... + int theInt = std::atoi(parsedValue.c_str()); + if (theInt != 0) { + res = theInt; //!OCLINT parameter reassignment + return true; + } + } else { // boolean - const char positive[][5] = {"1", "true", "on", "yes"}; // 5 - strlen("true") + 1 - const char negative[][6] = {"0", "false", "off", "no"}; // 6 - strlen("false") + 1 + const char positive[][5] = { "1", "true", "on", "yes" }; // 5 - strlen("true") + 1 + const char negative[][6] = { "0", "false", "off", "no" }; // 6 - strlen("false") + 1 // if the value matches any of the positive/negative possibilities - for(unsigned i = 0; i < 4; i++) { - if(parsedValue.compare(positive[i], true) == 0) { + for (unsigned i = 0; i < 4; i++) { + if (parsedValue.compare(positive[i], true) == 0) { res = 1; //!OCLINT parameter reassignment return true; } - if(parsedValue.compare(negative[i], true) == 0) { + if (parsedValue.compare(negative[i], true) == 0) { res = 0; //!OCLINT parameter reassignment return true; } } - } else { - // integer - // TODO: change this to use std::stoi or something else! currently it uses undefined behavior - assumes '0' on failed parse... - int theInt = std::atoi(parsedValue.c_str()); // NOLINT - if(theInt != 0) { - res = theInt; //!OCLINT parameter reassignment - return true; - } } return false; } @@ -6191,9 +6599,12 @@ void Context::parseArgs(int argc, const char* const* argv, bool withDefaults) { DOCTEST_PARSE_AS_BOOL_OR_FLAG("case-sensitive", "cs", case_sensitive, false); DOCTEST_PARSE_AS_BOOL_OR_FLAG("exit", "e", exit, false); DOCTEST_PARSE_AS_BOOL_OR_FLAG("duration", "d", duration, false); + DOCTEST_PARSE_AS_BOOL_OR_FLAG("minimal", "m", minimal, false); + DOCTEST_PARSE_AS_BOOL_OR_FLAG("quiet", "q", quiet, false); DOCTEST_PARSE_AS_BOOL_OR_FLAG("no-throw", "nt", no_throw, false); DOCTEST_PARSE_AS_BOOL_OR_FLAG("no-exitcode", "ne", no_exitcode, false); DOCTEST_PARSE_AS_BOOL_OR_FLAG("no-run", "nr", no_run, false); + DOCTEST_PARSE_AS_BOOL_OR_FLAG("no-intro", "ni", no_intro, false); DOCTEST_PARSE_AS_BOOL_OR_FLAG("no-version", "nv", no_version, false); DOCTEST_PARSE_AS_BOOL_OR_FLAG("no-colors", "nc", no_colors, false); DOCTEST_PARSE_AS_BOOL_OR_FLAG("force-colors", "fc", force_colors, false); @@ -6257,10 +6668,14 @@ void Context::clearFilters() { curr.clear(); } -// allows the user to override procedurally the int/bool options from the command line +// allows the user to override procedurally the bool options from the command line +void Context::setOption(const char* option, bool value) { + setOption(option, value ? "true" : "false"); +} + +// allows the user to override procedurally the int options from the command line void Context::setOption(const char* option, int value) { setOption(option, toString(value).c_str()); - // NOLINTNEXTLINE(clang-analyzer-cplusplus.NewDeleteLeaks) } // allows the user to override procedurally the string options from the command line @@ -6277,6 +6692,31 @@ void Context::setAsDefaultForAssertsOutOfTestCases() { g_cs = p; } void Context::setAssertHandler(detail::assert_handler ah) { p->ah = ah; } +void Context::setCout(std::ostream* out) { p->cout = out; } + +static class DiscardOStream : public std::ostream +{ +private: + class : public std::streambuf + { + private: + // allowing some buffering decreases the amount of calls to overflow + char buf[1024]; + + protected: + std::streamsize xsputn(const char_type*, std::streamsize count) override { return count; } + + int_type overflow(int_type ch) override { + setp(std::begin(buf), std::end(buf)); + return traits_type::not_eof(ch); + } + } discardBuf; + +public: + DiscardOStream() + : std::ostream(&discardBuf) {} +} discardOut; + // the main function that does all the filtering and test running int Context::run() { using namespace detail; @@ -6290,15 +6730,18 @@ int Context::run() { g_no_colors = p->no_colors; p->resetRunData(); - // stdout by default - p->cout = &std::cout; - p->cerr = &std::cerr; - - // or to a file if specified std::fstream fstr; - if(p->out.size()) { - fstr.open(p->out.c_str(), std::fstream::out); - p->cout = &fstr; + if(p->cout == nullptr) { + if(p->quiet) { + p->cout = &discardOut; + } else if(p->out.size()) { + // to a file if specified + fstr.open(p->out.c_str(), std::fstream::out); + p->cout = &fstr; + } else { + // stdout by default + p->cout = &std::cout; + } } FatalConditionHandler::allocateAltStackMem(); @@ -6370,7 +6813,7 @@ int Context::run() { // random_shuffle implementation const auto first = &testArray[0]; for(size_t i = testArray.size() - 1; i > 0; --i) { - int idxToSwap = std::rand() % (i + 1); // NOLINT + int idxToSwap = std::rand() % (i + 1); const auto temp = first[i]; @@ -6457,7 +6900,7 @@ int Context::run() { p->numAssertsFailedCurrentTest_atomic = 0; p->numAssertsCurrentTest_atomic = 0; - p->subcasesPassed.clear(); + p->fullyTraversedSubcases.clear(); DOCTEST_ITERATE_THROUGH_REPORTERS(test_case_start, tc); @@ -6467,9 +6910,10 @@ int Context::run() { do { // reset some of the fields for subcases (except for the set of fully passed ones) - p->should_reenter = false; - p->subcasesCurrentMaxLevel = 0; - p->subcasesStack.clear(); + p->reachedLeaf = false; + // May not be empty if previous subcase exited via exception. + p->subcaseStack.clear(); + p->currentSubcaseDepth = 0; p->shouldLogCurrentException = true; @@ -6503,9 +6947,9 @@ DOCTEST_MSVC_SUPPRESS_WARNING_POP p->failure_flags |= TestCaseFailureReason::TooManyFailedAsserts; } - if(p->should_reenter && run_test) + if(!p->nextSubcaseStack.empty() && run_test) DOCTEST_ITERATE_THROUGH_REPORTERS(test_case_reenter, tc); - if(!p->should_reenter) + if(p->nextSubcaseStack.empty()) run_test = false; } while(run_test); @@ -6531,17 +6975,10 @@ DOCTEST_MSVC_SUPPRESS_WARNING_POP DOCTEST_ITERATE_THROUGH_REPORTERS(report_query, qdata); } - // see these issues on the reasoning for this: - // - https://github.com/onqtam/doctest/issues/143#issuecomment-414418903 - // - https://github.com/onqtam/doctest/issues/126 - auto DOCTEST_FIX_FOR_MACOS_LIBCPP_IOSFWD_STRING_LINK_ERRORS = []() DOCTEST_NOINLINE - { std::cout << std::string(); }; - DOCTEST_FIX_FOR_MACOS_LIBCPP_IOSFWD_STRING_LINK_ERRORS(); - return cleanup_and_return(); } -IReporter::~IReporter() = default; +DOCTEST_DEFINE_INTERFACE(IReporter) int IReporter::get_num_active_contexts() { return detail::g_infoContexts.size(); } const IContextScope* const* IReporter::get_active_contexts() { @@ -6576,5 +7013,7 @@ DOCTEST_CLANG_SUPPRESS_WARNING_POP DOCTEST_MSVC_SUPPRESS_WARNING_POP DOCTEST_GCC_SUPPRESS_WARNING_POP +DOCTEST_SUPPRESS_COMMON_WARNINGS_POP + #endif // DOCTEST_LIBRARY_IMPLEMENTATION #endif // DOCTEST_CONFIG_IMPLEMENT diff --git a/tests/Autocomplete.test.cpp b/tests/Autocomplete.test.cpp index f3b0bcad..d8ad3340 100644 --- a/tests/Autocomplete.test.cpp +++ b/tests/Autocomplete.test.cpp @@ -2240,43 +2240,18 @@ local a: aaa.do CHECK(ac.entryMap.count("other")); } -TEST_CASE_FIXTURE(ACBuiltinsFixture, "autocompleteSource") + +TEST_CASE_FIXTURE(ACFixture, "comments") { - std::string_view source = R"( - local a = table. -- Line 1 - -- | Column 23 - )"; + fileResolver.source["Comments"] = "--!str"; - auto ac = autocompleteSource(frontend, source, Position{1, 24}, nullCallback).result; - - CHECK_EQ(17, ac.entryMap.size()); - CHECK(ac.entryMap.count("find")); - CHECK(ac.entryMap.count("pack")); - CHECK(!ac.entryMap.count("math")); -} - -TEST_CASE_FIXTURE(ACFixture, "autocompleteSource_require") -{ - std::string_view source = R"( - local a = require(w -- Line 1 - -- | Column 27 - )"; - - // CLI-43699 require shouldn't crash inside autocompleteSource - auto ac = autocompleteSource(frontend, source, Position{1, 27}, nullCallback).result; -} - -TEST_CASE_FIXTURE(ACFixture, "autocompleteSource_comments") -{ - std::string_view source = "--!str"; - - auto ac = autocompleteSource(frontend, source, Position{0, 6}, nullCallback).result; + auto ac = Luau::autocomplete(frontend, "Comments", Position{0, 6}, nullCallback); CHECK_EQ(0, ac.entryMap.size()); } TEST_CASE_FIXTURE(ACBuiltinsFixture, "autocompleteProp_index_function_metamethod_is_variadic") { - std::string_view source = R"( + fileResolver.source["Module/A"] = R"( type Foo = {x: number} local t = {} setmetatable(t, { @@ -2289,7 +2264,7 @@ TEST_CASE_FIXTURE(ACBuiltinsFixture, "autocompleteProp_index_function_metamethod -- | Column 20 )"; - auto ac = autocompleteSource(frontend, source, Position{9, 20}, nullCallback).result; + auto ac = Luau::autocomplete(frontend, "Module/A", Position{9, 20}, nullCallback); REQUIRE_EQ(1, ac.entryMap.size()); CHECK(ac.entryMap.count("x")); } @@ -2378,35 +2353,36 @@ end CHECK(ac.entryMap.count("elsewhere")); } -TEST_CASE_FIXTURE(ACFixture, "autocompleteSource_not_the_var_we_are_defining") +TEST_CASE_FIXTURE(ACFixture, "not_the_var_we_are_defining") { - std::string_view source = "abc,de"; + fileResolver.source["Module/A"] = "abc,de"; - auto ac = autocompleteSource(frontend, source, Position{0, 6}, nullCallback).result; + auto ac = Luau::autocomplete(frontend, "Module/A", Position{0, 6}, nullCallback); CHECK(!ac.entryMap.count("de")); } -TEST_CASE_FIXTURE(ACFixture, "autocompleteSource_recursive_function") +TEST_CASE_FIXTURE(ACFixture, "recursive_function_global") { - { - std::string_view global = R"(function abc() + fileResolver.source["global"] = R"(function abc() end )"; - auto ac = autocompleteSource(frontend, global, Position{1, 0}, nullCallback).result; - CHECK(ac.entryMap.count("abc")); - } + auto ac = Luau::autocomplete(frontend, "global", Position{1, 0}, nullCallback); + CHECK(ac.entryMap.count("abc")); +} - { - std::string_view local = R"(local function abc() + + +TEST_CASE_FIXTURE(ACFixture, "recursive_function_local") +{ + fileResolver.source["local"] = R"(local function abc() end )"; - auto ac = autocompleteSource(frontend, local, Position{1, 0}, nullCallback).result; - CHECK(ac.entryMap.count("abc")); - } + auto ac = Luau::autocomplete(frontend, "local", Position{1, 0}, nullCallback); + CHECK(ac.entryMap.count("abc")); } TEST_CASE_FIXTURE(ACFixture, "suggest_table_keys") diff --git a/tests/Compiler.test.cpp b/tests/Compiler.test.cpp index fafafd71..46fde067 100644 --- a/tests/Compiler.test.cpp +++ b/tests/Compiler.test.cpp @@ -165,8 +165,8 @@ LOADN R1 1 FASTCALL2K 18 R1 K0 L0 LOADK R2 K0 GETIMPORT R0 3 -L0: CALL R0 2 -1 -RETURN R0 -1 +CALL R0 2 -1 +L0: RETURN R0 -1 )"); } @@ -2100,12 +2100,12 @@ FASTCALL2 18 R0 R1 L0 MOVE R5 R0 MOVE R6 R1 GETIMPORT R4 2 -L0: CALL R4 2 1 -FASTCALL2 19 R4 R2 L1 +CALL R4 2 1 +L0: FASTCALL2 19 R4 R2 L1 MOVE R5 R2 GETIMPORT R3 4 -L1: CALL R3 2 -1 -RETURN R3 -1 +CALL R3 2 -1 +L1: RETURN R3 -1 )"); } @@ -2511,8 +2511,8 @@ return 5: MOVE R3 R0 5: MOVE R4 R1 5: GETIMPORT R2 2 -5: L0: CALL R2 2 -1 -5: RETURN R2 -1 +5: CALL R2 2 -1 +5: L0: RETURN R2 -1 )"); } @@ -2828,8 +2828,8 @@ TEST_CASE("FastcallBytecode") LOADN R1 -5 FASTCALL1 2 R1 L0 GETIMPORT R0 2 -L0: CALL R0 1 -1 -RETURN R0 -1 +CALL R0 1 -1 +L0: RETURN R0 -1 )"); // call through a local variable @@ -2838,8 +2838,8 @@ GETIMPORT R0 2 LOADN R2 -5 FASTCALL1 2 R2 L0 MOVE R1 R0 -L0: CALL R1 1 -1 -RETURN R1 -1 +CALL R1 1 -1 +L0: RETURN R1 -1 )"); // call through an upvalue @@ -2847,8 +2847,8 @@ RETURN R1 -1 LOADN R1 -5 FASTCALL1 2 R1 L0 GETUPVAL R0 0 -L0: CALL R0 1 -1 -RETURN R0 -1 +CALL R0 1 -1 +L0: RETURN R0 -1 )"); // mutating the global in the script breaks the optimization @@ -2893,8 +2893,8 @@ LOADK R1 K0 FASTCALL1 57 R1 L0 GETIMPORT R0 2 GETVARARGS R2 -1 -L0: CALL R0 -1 1 -RETURN R0 1 +CALL R0 -1 1 +L0: RETURN R0 1 )"); // more complex example: select inside a for loop bound + select from a iterator @@ -2912,16 +2912,16 @@ LOADK R5 K0 FASTCALL1 57 R5 L0 GETIMPORT R4 2 GETVARARGS R6 -1 -L0: CALL R4 -1 1 -MOVE R1 R4 +CALL R4 -1 1 +L0: MOVE R1 R4 LOADN R2 1 FORNPREP R1 L3 L1: FASTCALL1 57 R3 L2 GETIMPORT R4 2 MOVE R5 R3 GETVARARGS R6 -1 -L2: CALL R4 -1 1 -ADD R0 R0 R4 +CALL R4 -1 1 +L2: ADD R0 R0 R4 FORNLOOP R1 L1 L3: RETURN R0 1 )"); @@ -3242,7 +3242,7 @@ LOADN R2 -1 FASTCALL1 2 R2 L0 GETGLOBAL R3 K1024 GETTABLEKS R1 R3 K1025 -L0: CALL R1 1 -1 +CALL R1 1 -1 )"); } @@ -4063,8 +4063,8 @@ LOADN R2 2 LOADN R3 3 FASTCALL 54 L0 GETIMPORT R0 2 -L0: CALL R0 3 -1 -RETURN R0 -1 +CALL R0 3 -1 +L0: RETURN R0 -1 )"); } @@ -4351,6 +4351,8 @@ TEST_CASE("LoopUnrollControlFlow") {"LuauCompileLoopUnrollThresholdMaxBoost", 300}, }; + ScopedFastFlag sff("LuauCompileFoldBuiltins", true); + // break jumps to the end CHECK_EQ("\n" + compileFunction(R"( for i=1,3 do @@ -4414,7 +4416,7 @@ L2: RETURN R0 0 // continue needs to properly close upvalues CHECK_EQ("\n" + compileFunction(R"( for i=1,1 do - local j = math.abs(i) + local j = global(i) print(function() return j end) if math.random() < 0.5 then continue @@ -4424,21 +4426,20 @@ end )", 1, 2), R"( +GETIMPORT R0 1 LOADN R1 1 -FASTCALL1 2 R1 L0 -GETIMPORT R0 2 -L0: CALL R0 1 1 -GETIMPORT R1 4 +CALL R0 1 1 +GETIMPORT R1 3 NEWCLOSURE R2 P0 CAPTURE REF R0 CALL R1 1 0 GETIMPORT R1 6 CALL R1 0 1 LOADK R2 K7 -JUMPIFNOTLT R1 R2 L1 +JUMPIFNOTLT R1 R2 L0 CLOSEUPVALS R0 RETURN R0 0 -L1: ADDK R0 R0 K8 +L0: ADDK R0 R0 K8 CLOSEUPVALS R0 RETURN R0 0 )"); @@ -4625,11 +4626,11 @@ FORNPREP R1 L3 L0: FASTCALL1 24 R3 L1 MOVE R6 R3 GETIMPORT R5 2 -L1: CALL R5 1 -1 -FASTCALL 2 L2 +CALL R5 1 -1 +L1: FASTCALL 2 L2 GETIMPORT R4 4 -L2: CALL R4 -1 1 -SETTABLE R4 R0 R3 +CALL R4 -1 1 +L2: SETTABLE R4 R0 R3 FORNLOOP R1 L0 L3: RETURN R0 1 )"); @@ -4660,6 +4661,133 @@ L1: RETURN R0 0 )"); } +TEST_CASE("LoopUnrollCostBuiltins") +{ + ScopedFastInt sfis[] = { + {"LuauCompileLoopUnrollThreshold", 25}, + {"LuauCompileLoopUnrollThresholdMaxBoost", 300}, + }; + + ScopedFastFlag sff("LuauCompileModelBuiltins", true); + + // this loop uses builtins and is close to the cost budget so it's important that we model builtins as cheaper than regular calls + CHECK_EQ("\n" + compileFunction(R"( +function cipher(block, nonce) + for i = 0,3 do + block[i + 1] = bit32.band(bit32.rshift(nonce, i * 8), 0xff) + end +end +)", + 0, 2), + R"( +FASTCALL2K 39 R1 K0 L0 +MOVE R4 R1 +LOADK R5 K0 +GETIMPORT R3 3 +CALL R3 2 1 +L0: FASTCALL2K 29 R3 K4 L1 +LOADK R4 K4 +GETIMPORT R2 6 +CALL R2 2 1 +L1: SETTABLEN R2 R0 1 +FASTCALL2K 39 R1 K7 L2 +MOVE R4 R1 +LOADK R5 K7 +GETIMPORT R3 3 +CALL R3 2 1 +L2: FASTCALL2K 29 R3 K4 L3 +LOADK R4 K4 +GETIMPORT R2 6 +CALL R2 2 1 +L3: SETTABLEN R2 R0 2 +FASTCALL2K 39 R1 K8 L4 +MOVE R4 R1 +LOADK R5 K8 +GETIMPORT R3 3 +CALL R3 2 1 +L4: FASTCALL2K 29 R3 K4 L5 +LOADK R4 K4 +GETIMPORT R2 6 +CALL R2 2 1 +L5: SETTABLEN R2 R0 3 +FASTCALL2K 39 R1 K9 L6 +MOVE R4 R1 +LOADK R5 K9 +GETIMPORT R3 3 +CALL R3 2 1 +L6: FASTCALL2K 29 R3 K4 L7 +LOADK R4 K4 +GETIMPORT R2 6 +CALL R2 2 1 +L7: SETTABLEN R2 R0 4 +RETURN R0 0 +)"); + + // note that if we break compiler's ability to reason about bit32 builtin the loop is no longer unrolled as it's too expensive + CHECK_EQ("\n" + compileFunction(R"( +bit32 = {} + +function cipher(block, nonce) + for i = 0,3 do + block[i + 1] = bit32.band(bit32.rshift(nonce, i * 8), 0xff) + end +end +)", + 0, 2), + R"( +LOADN R4 0 +LOADN R2 3 +LOADN R3 1 +FORNPREP R2 L1 +L0: ADDK R5 R4 K0 +GETGLOBAL R7 K1 +GETTABLEKS R6 R7 K2 +GETGLOBAL R8 K1 +GETTABLEKS R7 R8 K3 +MOVE R8 R1 +MULK R9 R4 K4 +CALL R7 2 1 +LOADN R8 255 +CALL R6 2 1 +SETTABLE R6 R0 R5 +FORNLOOP R2 L0 +L1: RETURN R0 0 +)"); + + // additionally, if we pass too many constants the builtin stops being cheap because of argument setup + CHECK_EQ("\n" + compileFunction(R"( +function cipher(block, nonce) + for i = 0,3 do + block[i + 1] = bit32.band(bit32.rshift(nonce, i * 8), 0xff, 0xff, 0xff, 0xff, 0xff) + end +end +)", + 0, 2), + R"( +LOADN R4 0 +LOADN R2 3 +LOADN R3 1 +FORNPREP R2 L3 +L0: ADDK R5 R4 K0 +MULK R9 R4 K1 +FASTCALL2 39 R1 R9 L1 +MOVE R8 R1 +GETIMPORT R7 4 +CALL R7 2 1 +L1: LOADN R8 255 +LOADN R9 255 +LOADN R10 255 +LOADN R11 255 +LOADN R12 255 +FASTCALL 29 L2 +GETIMPORT R6 6 +CALL R6 6 1 +L2: SETTABLE R6 R0 R5 +FORNLOOP R2 L0 +L3: RETURN R0 0 +)"); +} + TEST_CASE("InlineBasic") { // inline function that returns a constant @@ -5216,8 +5344,8 @@ DUPCLOSURE R0 K0 LOADK R3 K1 FASTCALL1 20 R3 L0 GETIMPORT R2 4 -L0: CALL R2 1 2 -ADD R1 R2 R3 +CALL R2 1 2 +L0: ADD R1 R2 R3 RETURN R1 1 )"); @@ -5483,14 +5611,14 @@ NEWTABLE R2 0 0 FASTCALL2K 49 R2 K1 L0 LOADK R3 K1 GETIMPORT R1 3 -L0: CALL R1 2 0 -NEWTABLE R1 0 0 +CALL R1 2 0 +L0: NEWTABLE R1 0 0 NEWTABLE R3 0 0 FASTCALL2 49 R3 R1 L1 MOVE R4 R1 GETIMPORT R2 3 -L1: CALL R2 2 0 -RETURN R0 0 +CALL R2 2 0 +L1: RETURN R0 0 )"); } @@ -5762,4 +5890,271 @@ RETURN R0 2 )"); } +TEST_CASE("OptimizationLevel") +{ + ScopedFastFlag sff("LuauAlwaysCaptureHotComments", true); + + // at optimization level 1, no inlining is performed + CHECK_EQ("\n" + compileFunction(R"( +local function foo(a) + return a +end + +return foo(42) +)", + 1, 1), + R"( +DUPCLOSURE R0 K0 +MOVE R1 R0 +LOADN R2 42 +CALL R1 1 -1 +RETURN R1 -1 +)"); + + // you can override the level from 1 to 2 to force it + CHECK_EQ("\n" + compileFunction(R"( +--!optimize 2 +local function foo(a) + return a +end + +return foo(42) +)", + 1, 1), + R"( +DUPCLOSURE R0 K0 +LOADN R1 42 +RETURN R1 1 +)"); + + // you can also override it externally + CHECK_EQ("\n" + compileFunction(R"( +local function foo(a) + return a +end + +return foo(42) +)", + 1, 2), + R"( +DUPCLOSURE R0 K0 +LOADN R1 42 +RETURN R1 1 +)"); + + // ... after which you can downgrade it back via hot comment + CHECK_EQ("\n" + compileFunction(R"( +--!optimize 1 +local function foo(a) + return a +end + +return foo(42) +)", + 1, 2), + R"( +DUPCLOSURE R0 K0 +MOVE R1 R0 +LOADN R2 42 +CALL R1 1 -1 +RETURN R1 -1 +)"); +} + +TEST_CASE("BuiltinFolding") +{ + ScopedFastFlag sff("LuauCompileFoldBuiltins", true); + + CHECK_EQ("\n" + compileFunction(R"( +return + math.abs(-42), + math.acos(1), + math.asin(0), + math.atan2(0, 1), + math.atan(0), + math.ceil(1.5), + math.cosh(0), + math.cos(0), + math.deg(3.14159265358979323846), + math.exp(0), + math.floor(-1.5), + math.fmod(7, 3), + math.ldexp(0.5, 3), + math.log10(100), + math.log(1), + math.log(4, 2), + math.log(27, 3), + math.max(1, 2, 3), + math.min(1, 2, 3), + math.pow(3, 3), + math.floor(math.rad(180)), + math.sinh(0), + math.sin(0), + math.sqrt(9), + math.tanh(0), + math.tan(0), + bit32.arshift(-10, 1), + bit32.arshift(10, 1), + bit32.band(1, 3), + bit32.bnot(-2), + bit32.bor(1, 2), + bit32.bxor(3, 7), + bit32.btest(1, 3), + bit32.extract(100, 1, 3), + bit32.lrotate(100, -1), + bit32.lshift(100, 1), + bit32.replace(100, 5, 1, 3), + bit32.rrotate(100, -1), + bit32.rshift(100, 1), + type(100), + string.byte("a"), + string.byte("abc", 2), + string.len("abc"), + typeof(true), + math.clamp(-1, 0, 1), + math.sign(77), + math.round(7.6), + (type("fin")) +)", + 0, 2), + R"( +LOADN R0 42 +LOADN R1 0 +LOADN R2 0 +LOADN R3 0 +LOADN R4 0 +LOADN R5 2 +LOADN R6 1 +LOADN R7 1 +LOADN R8 180 +LOADN R9 1 +LOADN R10 -2 +LOADN R11 1 +LOADN R12 4 +LOADN R13 2 +LOADN R14 0 +LOADN R15 2 +LOADN R16 3 +LOADN R17 3 +LOADN R18 1 +LOADN R19 27 +LOADN R20 3 +LOADN R21 0 +LOADN R22 0 +LOADN R23 3 +LOADN R24 0 +LOADN R25 0 +LOADK R26 K0 +LOADN R27 5 +LOADN R28 1 +LOADN R29 1 +LOADN R30 3 +LOADN R31 4 +LOADB R32 1 +LOADN R33 2 +LOADN R34 50 +LOADN R35 200 +LOADN R36 106 +LOADN R37 200 +LOADN R38 50 +LOADK R39 K1 +LOADN R40 97 +LOADN R41 98 +LOADN R42 3 +LOADK R43 K2 +LOADN R44 0 +LOADN R45 1 +LOADN R46 8 +LOADK R47 K3 +RETURN R0 48 +)"); +} + +TEST_CASE("BuiltinFoldingProhibited") +{ + ScopedFastFlag sff("LuauCompileFoldBuiltins", true); + + CHECK_EQ("\n" + compileFunction(R"( +return + math.abs(), + math.max(1, true), + string.byte("abc", 42), + bit32.rshift(10, 42) +)", + 0, 2), + R"( +FASTCALL 2 L0 +GETIMPORT R0 2 +CALL R0 0 1 +L0: LOADN R2 1 +FASTCALL2K 18 R2 K3 L1 +LOADK R3 K3 +GETIMPORT R1 5 +CALL R1 2 1 +L1: LOADK R3 K6 +FASTCALL2K 41 R3 K7 L2 +LOADK R4 K7 +GETIMPORT R2 10 +CALL R2 2 1 +L2: LOADN R4 10 +FASTCALL2K 39 R4 K7 L3 +LOADK R5 K7 +GETIMPORT R3 13 +CALL R3 2 -1 +L3: RETURN R0 -1 +)"); +} + +TEST_CASE("BuiltinFoldingMultret") +{ + ScopedFastFlag sff1("LuauCompileFoldBuiltins", true); + ScopedFastFlag sff2("LuauCompileBetterMultret", true); + + CHECK_EQ("\n" + compileFunction(R"( +local NoLanes: Lanes = --[[ ]] 0b0000000000000000000000000000000 +local OffscreenLane: Lane = --[[ ]] 0b1000000000000000000000000000000 + +local function getLanesToRetrySynchronouslyOnError(root: FiberRoot): Lanes + local everythingButOffscreen = bit32.band(root.pendingLanes, bit32.bnot(OffscreenLane)) + if everythingButOffscreen ~= NoLanes then + return everythingButOffscreen + end + if bit32.band(everythingButOffscreen, OffscreenLane) ~= 0 then + return OffscreenLane + end + return NoLanes +end +)", + 0, 2), + R"( +GETTABLEKS R2 R0 K0 +FASTCALL2K 29 R2 K1 L0 +LOADK R3 K1 +GETIMPORT R1 4 +CALL R1 2 1 +L0: JUMPIFEQK R1 K5 L1 +RETURN R1 1 +L1: FASTCALL2K 29 R1 K6 L2 +MOVE R3 R1 +LOADK R4 K6 +GETIMPORT R2 4 +CALL R2 2 1 +L2: JUMPIFEQK R2 K5 L3 +LOADK R2 K6 +RETURN R2 1 +L3: LOADN R2 0 +RETURN R2 1 +)"); + + // Note: similarly, here we should have folded the return value but haven't because it's the last call in the sequence + CHECK_EQ("\n" + compileFunction(R"( +return math.abs(-42) +)", + 0, 2), + R"( +LOADN R0 42 +RETURN R0 1 +)"); +} + TEST_SUITE_END(); diff --git a/tests/Conformance.test.cpp b/tests/Conformance.test.cpp index d2311348..98c39337 100644 --- a/tests/Conformance.test.cpp +++ b/tests/Conformance.test.cpp @@ -17,6 +17,16 @@ #include extern bool verbose; +extern int optimizationLevel; + +static lua_CompileOptions defaultOptions() +{ + lua_CompileOptions copts = {}; + copts.optimizationLevel = optimizationLevel; + copts.debugLevel = 1; + + return copts; +} static int lua_collectgarbage(lua_State* L) { @@ -127,7 +137,7 @@ int lua_silence(lua_State* L) using StateRef = std::unique_ptr; static StateRef runConformance(const char* name, void (*setup)(lua_State* L) = nullptr, void (*yield)(lua_State* L) = nullptr, - lua_State* initialLuaState = nullptr, lua_CompileOptions* copts = nullptr) + lua_State* initialLuaState = nullptr, lua_CompileOptions* options = nullptr) { std::string path = __FILE__; path.erase(path.find_last_of("\\/")); @@ -189,8 +199,11 @@ static StateRef runConformance(const char* name, void (*setup)(lua_State* L) = n std::string chunkname = "=" + std::string(name); + // note: luau_compile supports nullptr options, but we need to customize our defaults to improve test coverage + lua_CompileOptions opts = options ? *options : defaultOptions(); + size_t bytecodeSize = 0; - char* bytecode = luau_compile(source.data(), source.size(), copts, &bytecodeSize); + char* bytecode = luau_compile(source.data(), source.size(), &opts, &bytecodeSize); int result = luau_load(L, chunkname.c_str(), bytecode, bytecodeSize, 0); free(bytecode); @@ -383,9 +396,7 @@ TEST_CASE("Pack") TEST_CASE("Vector") { - lua_CompileOptions copts = {}; - copts.optimizationLevel = 1; - copts.debugLevel = 1; + lua_CompileOptions copts = defaultOptions(); copts.vectorCtor = "vector"; runConformance( @@ -519,8 +530,7 @@ TEST_CASE("Debugger") breakhits = 0; interruptedthread = nullptr; - lua_CompileOptions copts = {}; - copts.optimizationLevel = 1; + lua_CompileOptions copts = defaultOptions(); copts.debugLevel = 2; runConformance( @@ -850,6 +860,43 @@ TEST_CASE("ApiCalls") } } +TEST_CASE("ApiAtoms") +{ + ScopedFastFlag sff("LuauLazyAtoms", true); + + StateRef globalState(luaL_newstate(), lua_close); + lua_State* L = globalState.get(); + + lua_callbacks(L)->useratom = [](const char* s, size_t l) -> int16_t { + if (strcmp(s, "string") == 0) + return 0; + if (strcmp(s, "important") == 0) + return 1; + + return -1; + }; + + lua_pushstring(L, "string"); + lua_pushstring(L, "import"); + lua_pushstring(L, "ant"); + lua_concat(L, 2); + lua_pushstring(L, "unimportant"); + + int a1, a2, a3; + const char* s1 = lua_tostringatom(L, -3, &a1); + const char* s2 = lua_tostringatom(L, -2, &a2); + const char* s3 = lua_tostringatom(L, -1, &a3); + + CHECK(strcmp(s1, "string") == 0); + CHECK(a1 == 0); + + CHECK(strcmp(s2, "important") == 0); + CHECK(a2 == 1); + + CHECK(strcmp(s3, "unimportant") == 0); + CHECK(a3 == -1); +} + static bool endsWith(const std::string& str, const std::string& suffix) { if (suffix.length() > str.length()) @@ -957,9 +1004,8 @@ TEST_CASE("TagMethodError") TEST_CASE("Coverage") { - lua_CompileOptions copts = {}; - copts.optimizationLevel = 1; - copts.debugLevel = 1; + lua_CompileOptions copts = defaultOptions(); + copts.optimizationLevel = 1; // disable inlining to get fixed expected hit results copts.coverageLevel = 2; runConformance( @@ -1059,6 +1105,9 @@ TEST_CASE("GCDump") TEST_CASE("Interrupt") { + lua_CompileOptions copts = defaultOptions(); + copts.optimizationLevel = 1; // disable loop unrolling to get fixed expected hit results + static const int expectedhits[] = { 2, 9, @@ -1109,7 +1158,8 @@ TEST_CASE("Interrupt") }, [](lua_State* L) { CHECK(index == 5); // a single yield point - }); + }, + nullptr, &copts); CHECK(index == int(std::size(expectedhits))); } diff --git a/tests/CostModel.test.cpp b/tests/CostModel.test.cpp index c709ba8e..eacc718b 100644 --- a/tests/CostModel.test.cpp +++ b/tests/CostModel.test.cpp @@ -10,7 +10,7 @@ namespace Luau namespace Compile { -uint64_t modelCost(AstNode* root, AstLocal* const* vars, size_t varCount); +uint64_t modelCost(AstNode* root, AstLocal* const* vars, size_t varCount, const DenseHashMap& builtins); int computeCost(uint64_t model, const bool* varsConst, size_t varCount); } // namespace Compile @@ -29,7 +29,7 @@ static uint64_t modelFunction(const char* source) AstStatFunction* func = result.root->body.data[0]->as(); REQUIRE(func); - return Luau::Compile::modelCost(func->func->body, func->func->args.data, func->func->args.size); + return Luau::Compile::modelCost(func->func->body, func->func->args.data, func->func->args.size, {nullptr}); } TEST_CASE("Expression") diff --git a/tests/Frontend.test.cpp b/tests/Frontend.test.cpp index 4f331399..4efa74d9 100644 --- a/tests/Frontend.test.cpp +++ b/tests/Frontend.test.cpp @@ -756,26 +756,6 @@ TEST_CASE_FIXTURE(FrontendFixture, "test_lint_uses_correct_config") CHECK_EQ(0, result4.warnings.size()); } -TEST_CASE_FIXTURE(FrontendFixture, "lintFragment") -{ - LintOptions lintOptions; - lintOptions.enableWarning(LintWarning::Code_ForRange); - - auto [_sourceModule, result] = frontend.lintFragment(R"( - local t = {} - - for i=#t,1 do - end - - for i=#t,1,-1 do - end - )", - lintOptions); - - CHECK_EQ(1, result.warnings.size()); - CHECK_EQ(0, result.errors.size()); -} - TEST_CASE_FIXTURE(FrontendFixture, "discard_type_graphs") { Frontend fe{&fileResolver, &configResolver, {false}}; diff --git a/tests/Linter.test.cpp b/tests/Linter.test.cpp index 202aeceb..488ade9f 100644 --- a/tests/Linter.test.cpp +++ b/tests/Linter.test.cpp @@ -1658,4 +1658,21 @@ end CHECK_EQ(result.warnings[0].text, "Condition has already been checked on line 2"); } +TEST_CASE_FIXTURE(Fixture, "WrongCommentOptimize") +{ + LintResult result = lint(R"( +--!optimize +--!optimize +--!optimize me +--!optimize 100500 +--!optimize 2 +)"); + + REQUIRE_EQ(result.warnings.size(), 4); + CHECK_EQ(result.warnings[0].text, "optimize directive requires an optimization level"); + CHECK_EQ(result.warnings[1].text, "optimize directive requires an optimization level"); + CHECK_EQ(result.warnings[2].text, "optimize directive uses unknown optimization level 'me', 0..2 expected"); + CHECK_EQ(result.warnings[3].text, "optimize directive uses unknown optimization level '100500', 0..2 expected"); +} + TEST_SUITE_END(); diff --git a/tests/Normalize.test.cpp b/tests/Normalize.test.cpp index fb0a899e..69e7f074 100644 --- a/tests/Normalize.test.cpp +++ b/tests/Normalize.test.cpp @@ -756,6 +756,65 @@ TEST_CASE_FIXTURE(Fixture, "cyclic_table_normalizes_sensibly") CHECK_EQ("t1 where t1 = { get: () -> t1 }", toString(ty, {true})); } +TEST_CASE_FIXTURE(Fixture, "cyclic_union") +{ + ScopedFastFlag sff[] = { + {"LuauLowerBoundsCalculation", true}, + {"LuauFixNormalizationOfCyclicUnions", true}, + }; + + CheckResult result = check(R"( + type T = {T?}? + + local a: T + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK("t1? where t1 = {t1?}" == toString(requireType("a"))); +} + +TEST_CASE_FIXTURE(Fixture, "cyclic_intersection") +{ + ScopedFastFlag sff[] = { + {"LuauLowerBoundsCalculation", true}, + {"LuauFixNormalizationOfCyclicUnions", true}, + }; + + CheckResult result = check(R"( + type T = {T & {}} + + local a: T + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + // FIXME: We are not properly normalizing this type, but we are at least not improperly discarding information + CHECK("t1 where t1 = {{t1 & {| |}}}" == toString(requireType("a"), {true})); +} + +TEST_CASE_FIXTURE(Fixture, "intersection_of_tables_with_indexers") +{ + ScopedFastFlag sff[] = { + {"LuauLowerBoundsCalculation", true}, + {"LuauFixNormalizationOfCyclicUnions", true}, + }; + + CheckResult result = check(R"( + type A = {number} + type B = {string} + + type C = A & B + + local a: C + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + + // FIXME: We are not properly normalizing this type, but we are at least not improperly discarding information + CHECK("{number & string}" == toString(requireType("a"), {true})); +} + TEST_CASE_FIXTURE(BuiltinsFixture, "union_of_distinct_free_types") { ScopedFastFlag flags[] = { diff --git a/tests/ToString.test.cpp b/tests/ToString.test.cpp index 52a29bcc..288af8da 100644 --- a/tests/ToString.test.cpp +++ b/tests/ToString.test.cpp @@ -62,7 +62,6 @@ TEST_CASE_FIXTURE(Fixture, "named_table") TEST_CASE_FIXTURE(Fixture, "empty_table") { - ScopedFastFlag LuauToStringTableBracesNewlines("LuauToStringTableBracesNewlines", true); CheckResult result = check(R"( local a: {} )"); @@ -77,7 +76,6 @@ TEST_CASE_FIXTURE(Fixture, "empty_table") TEST_CASE_FIXTURE(Fixture, "table_respects_use_line_break") { - ScopedFastFlag LuauToStringTableBracesNewlines("LuauToStringTableBracesNewlines", true); CheckResult result = check(R"( local a: { prop: string, anotherProp: number, thirdProp: boolean } )"); diff --git a/tests/TypeInfer.builtins.test.cpp b/tests/TypeInfer.builtins.test.cpp index 7d1bd6ba..4f6adf97 100644 --- a/tests/TypeInfer.builtins.test.cpp +++ b/tests/TypeInfer.builtins.test.cpp @@ -1143,4 +1143,114 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "gmatch_capture_types_invalid_pattern_fallbac CHECK_EQ(toString(requireType("foo")), "() -> (...string)"); } +TEST_CASE_FIXTURE(BuiltinsFixture, "match_capture_types") +{ + ScopedFastFlag sffs{"LuauDeduceFindMatchReturnTypes", true}; + CheckResult result = check(R"END( + local a, b, c = string.match("This is a string", "(.()(%a+))") + )END"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ(toString(requireType("a")), "string"); + CHECK_EQ(toString(requireType("b")), "number"); + CHECK_EQ(toString(requireType("c")), "string"); +} + +TEST_CASE_FIXTURE(BuiltinsFixture, "match_capture_types2") +{ + ScopedFastFlag sffs{"LuauDeduceFindMatchReturnTypes", true}; + CheckResult result = check(R"END( + local a, b, c = string.match("This is a string", "(.()(%a+))", "this should be a number") + )END"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + TypeMismatch* tm = get(result.errors[0]); + REQUIRE(tm); + CHECK_EQ(toString(tm->wantedType), "number?"); + CHECK_EQ(toString(tm->givenType), "string"); + + CHECK_EQ(toString(requireType("a")), "string"); + CHECK_EQ(toString(requireType("b")), "number"); + CHECK_EQ(toString(requireType("c")), "string"); +} + +TEST_CASE_FIXTURE(BuiltinsFixture, "find_capture_types") +{ + ScopedFastFlag sffs{"LuauDeduceFindMatchReturnTypes", true}; + CheckResult result = check(R"END( + local d, e, a, b, c = string.find("This is a string", "(.()(%a+))") + )END"); + + LUAU_REQUIRE_NO_ERRORS(result); + + CHECK_EQ(toString(requireType("a")), "string"); + CHECK_EQ(toString(requireType("b")), "number"); + CHECK_EQ(toString(requireType("c")), "string"); + CHECK_EQ(toString(requireType("d")), "number?"); + CHECK_EQ(toString(requireType("e")), "number?"); +} + +TEST_CASE_FIXTURE(BuiltinsFixture, "find_capture_types2") +{ + ScopedFastFlag sffs{"LuauDeduceFindMatchReturnTypes", true}; + CheckResult result = check(R"END( + local d, e, a, b, c = string.find("This is a string", "(.()(%a+))", "this should be a number") + )END"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + TypeMismatch* tm = get(result.errors[0]); + REQUIRE(tm); + CHECK_EQ(toString(tm->wantedType), "number?"); + CHECK_EQ(toString(tm->givenType), "string"); + + CHECK_EQ(toString(requireType("a")), "string"); + CHECK_EQ(toString(requireType("b")), "number"); + CHECK_EQ(toString(requireType("c")), "string"); + CHECK_EQ(toString(requireType("d")), "number?"); + CHECK_EQ(toString(requireType("e")), "number?"); +} + +TEST_CASE_FIXTURE(BuiltinsFixture, "find_capture_types3") +{ + ScopedFastFlag sffs{"LuauDeduceFindMatchReturnTypes", true}; + CheckResult result = check(R"END( + local d, e, a, b, c = string.find("This is a string", "(.()(%a+))", 1, "this should be a bool") + )END"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + TypeMismatch* tm = get(result.errors[0]); + REQUIRE(tm); + CHECK_EQ(toString(tm->wantedType), "boolean?"); + CHECK_EQ(toString(tm->givenType), "string"); + + CHECK_EQ(toString(requireType("a")), "string"); + CHECK_EQ(toString(requireType("b")), "number"); + CHECK_EQ(toString(requireType("c")), "string"); + CHECK_EQ(toString(requireType("d")), "number?"); + CHECK_EQ(toString(requireType("e")), "number?"); +} + +TEST_CASE_FIXTURE(BuiltinsFixture, "find_capture_types3") +{ + ScopedFastFlag sffs{"LuauDeduceFindMatchReturnTypes", true}; + CheckResult result = check(R"END( + local d, e, a, b = string.find("This is a string", "(.()(%a+))", 1, true) + )END"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + CountMismatch* acm = get(result.errors[0]); + REQUIRE(acm); + CHECK_EQ(acm->context, CountMismatch::Result); + CHECK_EQ(acm->expected, 2); + CHECK_EQ(acm->actual, 4); + + CHECK_EQ(toString(requireType("d")), "number?"); + CHECK_EQ(toString(requireType("e")), "number?"); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.primitives.test.cpp b/tests/TypeInfer.primitives.test.cpp index 9e8e2503..d60c96e0 100644 --- a/tests/TypeInfer.primitives.test.cpp +++ b/tests/TypeInfer.primitives.test.cpp @@ -11,6 +11,8 @@ #include "doctest.h" +LUAU_FASTFLAG(LuauDeduceFindMatchReturnTypes) + using namespace Luau; TEST_SUITE_BEGIN("TypeInferPrimitives"); @@ -80,7 +82,10 @@ TEST_CASE_FIXTURE(Fixture, "string_function_other") )"); CHECK_EQ(0, result.errors.size()); - CHECK_EQ(toString(requireType("p")), "string?"); + if (FFlag::LuauDeduceFindMatchReturnTypes) + CHECK_EQ(toString(requireType("p")), "string"); + else + CHECK_EQ(toString(requireType("p")), "string?"); } TEST_CASE_FIXTURE(Fixture, "CheckMethodsOfNumber") diff --git a/tests/TypeInfer.singletons.test.cpp b/tests/TypeInfer.singletons.test.cpp index 4a88abee..5da4a340 100644 --- a/tests/TypeInfer.singletons.test.cpp +++ b/tests/TypeInfer.singletons.test.cpp @@ -476,4 +476,21 @@ TEST_CASE_FIXTURE(Fixture, "taking_the_length_of_union_of_string_singleton") CHECK_EQ(R"("bye" | "hi")", toString(requireTypeAtPosition({3, 23}))); } +TEST_CASE_FIXTURE(Fixture, "no_widening_from_callsites") +{ + ScopedFastFlag sff{"LuauReturnsFromCallsitesAreNotWidened", true}; + + CheckResult result = check(R"( + type Direction = "North" | "East" | "West" | "South" + + local function direction(): Direction + return "North" + end + + local d: Direction = direction() + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.tables.test.cpp b/tests/TypeInfer.tables.test.cpp index 3e830f2a..21ad4e1b 100644 --- a/tests/TypeInfer.tables.test.cpp +++ b/tests/TypeInfer.tables.test.cpp @@ -3084,4 +3084,86 @@ local b = a.x CHECK_EQ("Type 'a' does not have key 'x'", toString(result.errors[1])); } +TEST_CASE_FIXTURE(Fixture, "scalar_is_a_subtype_of_a_compatible_polymorphic_shape_type") +{ + ScopedFastFlag sff{"LuauScalarShapeSubtyping", true}; + + CheckResult result = check(R"( + local function f(s) + return s:lower() + end + + f("foo" :: string) + f("bar" :: "bar") + f("baz" :: "bar" | "baz") + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(Fixture, "scalar_is_not_a_subtype_of_a_compatible_polymorphic_shape_type") +{ + ScopedFastFlag sff{"LuauScalarShapeSubtyping", true}; + + CheckResult result = check(R"( + local function f(s) + return s:absolutely_no_scalar_has_this_method() + end + + f("foo" :: string) + f("bar" :: "bar") + f("baz" :: "bar" | "baz") + )"); + + LUAU_REQUIRE_ERROR_COUNT(3, result); + CHECK_EQ(R"(Type 'string' could not be converted into 't1 where t1 = {- absolutely_no_scalar_has_this_method: (t1) -> (a...) -}' +caused by: + The former's metatable does not satisfy the requirements. Table type 'string' not compatible with type 't1 where t1 = {- absolutely_no_scalar_has_this_method: (t1) -> (a...) -}' because the former is missing field 'absolutely_no_scalar_has_this_method')", + toString(result.errors[0])); + CHECK_EQ(R"(Type '"bar"' could not be converted into 't1 where t1 = {- absolutely_no_scalar_has_this_method: (t1) -> (a...) -}' +caused by: + The former's metatable does not satisfy the requirements. Table type 'string' not compatible with type 't1 where t1 = {- absolutely_no_scalar_has_this_method: (t1) -> (a...) -}' because the former is missing field 'absolutely_no_scalar_has_this_method')", + toString(result.errors[1])); + CHECK_EQ(R"(Type '"bar" | "baz"' could not be converted into 't1 where t1 = {- absolutely_no_scalar_has_this_method: (t1) -> (a...) -}' +caused by: + Not all union options are compatible. Type '"bar"' could not be converted into 't1 where t1 = {- absolutely_no_scalar_has_this_method: (t1) -> (a...) -}' +caused by: + The former's metatable does not satisfy the requirements. Table type 'string' not compatible with type 't1 where t1 = {- absolutely_no_scalar_has_this_method: (t1) -> (a...) -}' because the former is missing field 'absolutely_no_scalar_has_this_method')", + toString(result.errors[2])); +} + +TEST_CASE_FIXTURE(Fixture, "a_free_shape_can_turn_into_a_scalar_if_it_is_compatible") +{ + ScopedFastFlag sff{"LuauScalarShapeSubtyping", true}; + + CheckResult result = check(R"( + local function f(s): string + local foo = s:lower() + return s + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + CHECK_EQ("(string) -> string", toString(requireType("f"))); +} + +TEST_CASE_FIXTURE(Fixture, "a_free_shape_cannot_turn_into_a_scalar_if_it_is_not_compatible") +{ + ScopedFastFlag sff{"LuauScalarShapeSubtyping", true}; + + CheckResult result = check(R"( + local function f(s): string + local foo = s:absolutely_no_scalar_has_this_method() + return s + end + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + CHECK_EQ(R"(Type 't1 where t1 = {+ absolutely_no_scalar_has_this_method: (t1) -> (a, b...) +}' could not be converted into 'string' +caused by: + The former's metatable does not satisfy the requirements. Table type 'string' not compatible with type 't1 where t1 = {+ absolutely_no_scalar_has_this_method: (t1) -> (a, b...) +}' because the former is missing field 'absolutely_no_scalar_has_this_method')", + toString(result.errors[0])); + CHECK_EQ("(t1) -> string where t1 = {+ absolutely_no_scalar_has_this_method: (t1) -> (a, b...) +}", toString(requireType("f"))); +} + TEST_SUITE_END(); diff --git a/tests/TypeInfer.unknownnever.test.cpp b/tests/TypeInfer.unknownnever.test.cpp index bc742b03..2288db4e 100644 --- a/tests/TypeInfer.unknownnever.test.cpp +++ b/tests/TypeInfer.unknownnever.test.cpp @@ -268,13 +268,49 @@ TEST_CASE_FIXTURE(Fixture, "unary_minus_of_never") TEST_CASE_FIXTURE(Fixture, "length_of_never") { + ScopedFastFlag sff{"LuauNeverTypesAndOperatorsInference", true}; + CheckResult result = check(R"( local x = #({} :: never) )"); LUAU_REQUIRE_NO_ERRORS(result); - CHECK_EQ("never", toString(requireType("x"))); + CHECK_EQ("number", toString(requireType("x"))); +} + +TEST_CASE_FIXTURE(Fixture, "dont_unify_operands_if_one_of_the_operand_is_never_in_any_ordering_operators") +{ + ScopedFastFlag sff[]{ + {"LuauUnknownAndNeverType", true}, + {"LuauNeverTypesAndOperatorsInference", true}, + }; + + CheckResult result = check(R"( + local function ord(x: nil, y) + return x ~= nil and x > y + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + CHECK_EQ("(nil, a) -> boolean", toString(requireType("ord"))); +} + +TEST_CASE_FIXTURE(Fixture, "math_operators_and_never") +{ + ScopedFastFlag sff[]{ + {"LuauUnknownAndNeverType", true}, + {"LuauNeverTypesAndOperatorsInference", true}, + }; + + CheckResult result = check(R"( + local function mul(x: nil, y) + return x ~= nil and x * y -- infers boolean | never, which is normalized into boolean + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + CHECK_EQ("(nil, a) -> boolean", toString(requireType("mul"))); } TEST_SUITE_END(); diff --git a/tests/conformance/basic.lua b/tests/conformance/basic.lua index 385a0450..b2dcaf94 100644 --- a/tests/conformance/basic.lua +++ b/tests/conformance/basic.lua @@ -727,16 +727,20 @@ assert((function() local abs = math.abs function foo(...) return abs(...) end re -- NOTE: getfenv breaks fastcalls for the remainder of the source! hence why this is delayed until the end function testgetfenv() getfenv() + + -- declare constant so that at O2 this test doesn't interfere with constant folding which we can't deoptimize + local negfive negfive = -5 + -- getfenv breaks fastcalls (we assume we can't rely on knowing the semantics), but behavior shouldn't change - assert((function() return math.abs(-5) end)() == 5) - assert((function() local abs = math.abs return abs(-5) end)() == 5) - assert((function() local abs = math.abs function foo() return abs(-5) end return foo() end)() == 5) + assert((function() return math.abs(negfive) end)() == 5) + assert((function() local abs = math.abs return abs(negfive) end)() == 5) + assert((function() local abs = math.abs function foo() return abs(negfive) end return foo() end)() == 5) -- ... unless you actually reassign the function :D getfenv().math = { abs = function(n) return n*n end } - assert((function() return math.abs(-5) end)() == 25) - assert((function() local abs = math.abs return abs(-5) end)() == 25) - assert((function() local abs = math.abs function foo() return abs(-5) end return foo() end)() == 25) + assert((function() return math.abs(negfive) end)() == 25) + assert((function() local abs = math.abs return abs(negfive) end)() == 25) + assert((function() local abs = math.abs function foo() return abs(negfive) end return foo() end)() == 25) end -- you need to have enough arguments and arguments of the right type; if you don't, we'll fallback to the regular code. This checks coercions diff --git a/tests/main.cpp b/tests/main.cpp index 2af9f702..c5b844b5 100644 --- a/tests/main.cpp +++ b/tests/main.cpp @@ -23,10 +23,13 @@ #include -// Indicates if verbose output is enabled. -// Currently, this enables output from lua's 'print', but other verbose output could be enabled eventually. +// Indicates if verbose output is enabled; can be overridden via --verbose +// Currently, this enables output from 'print', but other verbose output could be enabled eventually. bool verbose = false; +// Default optimization level for conformance test; can be overridden via -On +int optimizationLevel = 1; + static bool skipFastFlag(const char* flagName) { if (strncmp(flagName, "Test", 4) == 0) @@ -249,6 +252,15 @@ int main(int argc, char** argv) verbose = true; } + int level = -1; + if (doctest::parseIntOption(argc, argv, "-O", doctest::option_int, level)) + { + if (level < 0 || level > 2) + std::cerr << "Optimization level must be between 0 and 2 inclusive." << std::endl; + else + optimizationLevel = level; + } + if (std::vector flags; doctest::parseCommaSepArgs(argc, argv, "--fflags=", flags)) setFastFlags(flags); @@ -279,6 +291,7 @@ int main(int argc, char** argv) if (doctest::parseFlag(argc, argv, "--help") || doctest::parseFlag(argc, argv, "-h")) { printf("Additional command line options:\n"); + printf(" -O[n] Changes default optimization level (1) for conformance runs\n"); printf(" --verbose Enables verbose output (e.g. lua 'print' statements)\n"); printf(" --fflags= Sets specified fast flags\n"); printf(" --list-fflags List all fast flags\n"); diff --git a/tools/patchtests.py b/tools/patchtests.py index dcaf6083..56970c9f 100644 --- a/tools/patchtests.py +++ b/tools/patchtests.py @@ -16,7 +16,11 @@ state = 0 # parse input into errors[] with the state machine; this is using doctest output and expects multi-line match failures for line in input: if state == 0: - match = re.match("tests/[^:]+:(\d+): ERROR: CHECK_EQ", line) + if sys.platform == "win32": + match = re.match("[^(]+\((\d+)\): ERROR: CHECK_EQ", line) + else: + match = re.match("tests/[^:]+:(\d+): ERROR: CHECK_EQ", line) + if match: error_line = int(match[1]) state = 1 @@ -52,12 +56,16 @@ result = [] current = 0 index = 0 +target = 0 while index < len(source): line = source[index] error = errors[current] if current < len(errors) else None - if not error or index < error[0] or line != error[1][0]: + if error: + target = error[0] if sys.platform != "win32" else error[0] - len(error[1]) - 1 + + if not error or index < target or line != error[1][0]: result.append(line) index += 1 else: From 96316c66dc83d4f05e158c8c3238340d019e6185 Mon Sep 17 00:00:00 2001 From: Matthew Emery <49573837+memery-rbx@users.noreply.github.com> Date: Mon, 18 Jul 2022 12:36:23 -0700 Subject: [PATCH 314/399] Documentation of round tie-breaking (#602) * Update library.md * Tie-breaking documentation for round --- docs/_pages/library.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/_pages/library.md b/docs/_pages/library.md index 6ca2c05a..d82b6f28 100644 --- a/docs/_pages/library.md +++ b/docs/_pages/library.md @@ -360,7 +360,7 @@ Returns `-1` if `n` is negative, `1` if `n` is positive, and `0` if `n` is zero function math.round(n: number): number ``` -Rounds `n` to the nearest integer boundary. +Rounds `n` to the nearest integer boundary. If `n` is exactly halfway between two integers, rounds `n` away from 0. ## table library From ea7a6c1260b8edf7cf88bd17b2f16f7514140392 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Tue, 19 Jul 2022 08:37:56 -0700 Subject: [PATCH 315/399] Spell out RFC considerations for library functions more explicitly (#603) When considering new standard library functions, we essentially need to strongly justify their existence over their implementation in Luau in user library code. This PR attempts to provide a few axes of consideration; ideally new library functions tick many of the boxes, eg "used often + is more performant + clear unambiguous interface" is an ideal consideration for a library function, whereas if it's merely accelerating a single specific use case for a single application it's unlikely to be a good justification for inclusion. --- rfcs/README.md | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/rfcs/README.md b/rfcs/README.md index f6c4c145..4b5e7b04 100644 --- a/rfcs/README.md +++ b/rfcs/README.md @@ -18,6 +18,13 @@ For changes in semantics, we should be asking: - Can it be sandboxed assuming malicious usage? - Is it compatible with type checking and other forms of static analysis? +For new standard library functions, we should be asking: + +- Is the new functionality used/useful often enough in existing code? +- Does the standard library implementation carry important performance benefits that can't be achieved in user code? +- Is the behavior general and unambiguous, as opposed to solving a problem / providing an interface that's too specific? +- Is the function interface amenable to type checking / linting? + In addition to these questions, we also need to consider that every addition carries a cost, and too many features will result in a language that is harder to learn, harder to implement and ensure consistent implementation quality throughout, slower, etc. In addition, any language is greater than the sum of its parts and features often have non-intuitive interactions with each other. Since reversing these decisions is incredibly costly and can be impossible due to backwards compatibility implications, all user facing changes to Luau language and core libraries must go through an RFC process. From a824b05c9e5ce7c5cab2734be25dd4a6d1bf8dc8 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Wed, 20 Jul 2022 15:12:30 -0700 Subject: [PATCH 316/399] Update coverage report to codecov (#606) Coveralls has had multiple stability issues in the recent while, and codecov seems much better maintained in general. --- .github/codecov.yml | 1 + .github/workflows/build.yml | 14 ++------------ README.md | 2 +- 3 files changed, 4 insertions(+), 13 deletions(-) create mode 100644 .github/codecov.yml diff --git a/.github/codecov.yml b/.github/codecov.yml new file mode 100644 index 00000000..69cb7601 --- /dev/null +++ b/.github/codecov.yml @@ -0,0 +1 @@ +comment: false diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml index 93b92645..da525ad8 100644 --- a/.github/workflows/build.yml +++ b/.github/workflows/build.yml @@ -76,20 +76,10 @@ jobs: - name: make coverage run: | CXX=clang++-10 make -j2 config=coverage coverage - - name: debug coverage - run: | - git status - git log -5 - echo SHA: $GITHUB_SHA - name: upload coverage - uses: coverallsapp/github-action@master + uses: codecov/codecov-action@v3 with: - path-to-lcov: ./coverage.info - github-token: ${{ secrets.GITHUB_TOKEN }} - - uses: actions/upload-artifact@v2 - with: - name: coverage - path: coverage + files: ./coverage.info web: runs-on: ubuntu-latest diff --git a/README.md b/README.md index 2ed7348d..2b44c89a 100644 --- a/README.md +++ b/README.md @@ -1,4 +1,4 @@ -Luau ![CI](https://github.com/Roblox/luau/workflows/build/badge.svg) [![Coverage](https://coveralls.io/repos/github/Roblox/luau/badge.svg?branch=master&t=2PXMow)](https://coveralls.io/github/Roblox/luau?branch=master) +Luau ![CI](https://github.com/Roblox/luau/workflows/build/badge.svg) [![codecov](https://codecov.io/gh/Roblox/luau/branch/master/graph/badge.svg?token=S3U44WN416)](https://codecov.io/gh/Roblox/luau) ==== Luau (lowercase u, /ˈlu.aʊ/) is a fast, small, safe, gradually typed embeddable scripting language derived from [Lua](https://lua.org). From 8e8ae0a01dc700ff2e9e5973c9575399df16f719 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Thu, 21 Jul 2022 13:36:41 -0700 Subject: [PATCH 317/399] Sync to upstream/release/537 --- Analysis/include/Luau/JsonEncoder.h | 3 + Analysis/include/Luau/VisitTypeVar.h | 43 +- Analysis/src/JsonEncoder.cpp | 127 ++++- Analysis/src/Module.cpp | 26 +- Analysis/src/TypeInfer.cpp | 24 +- CLI/Analyze.cpp | 9 +- CLI/Ast.cpp | 3 +- CLI/Flags.cpp | 75 +++ CLI/Flags.h | 5 + CLI/Repl.cpp | 91 +-- CodeGen/include/Luau/AssemblyBuilderX64.h | 10 + CodeGen/include/Luau/Condition.h | 2 - CodeGen/src/AssemblyBuilderX64.cpp | 39 +- Common/include/Luau/ExperimentalFlags.h | 26 + Compiler/src/Compiler.cpp | 37 +- Makefile | 13 +- Sources.cmake | 7 + VM/include/lua.h | 1 + VM/src/lapi.cpp | 14 +- VM/src/ldebug.cpp | 5 + VM/src/ldebug.h | 1 + VM/src/ltablib.cpp | 6 +- VM/src/lvmutils.cpp | 6 +- bench/tests/tictactoe.lua | 4 +- fuzz/proto.cpp | 2 +- tests/AssemblyBuilderX64.test.cpp | 24 + tests/Compiler.test.cpp | 95 +++- tests/Conformance.test.cpp | 8 +- tests/Frontend.test.cpp | 69 +++ tests/JsonEncoder.test.cpp | 80 ++- tests/NonstrictMode.test.cpp | 93 +--- tests/Normalize.test.cpp | 3 +- tests/TypeInfer.functions.test.cpp | 30 +- tests/TypeInfer.loops.test.cpp | 13 +- tests/TypeInfer.provisional.test.cpp | 14 + tests/TypeInfer.tables.test.cpp | 9 + tests/TypeInfer.test.cpp | 34 +- tests/conformance/{nextvar.lua => tables.lua} | 20 + tests/main.cpp | 2 +- tools/faillist.txt | 521 ++++++++++++++++++ tools/natvis/Analysis.natvis | 64 +++ tools/test_dcr.py | 124 +++++ 42 files changed, 1497 insertions(+), 285 deletions(-) create mode 100644 CLI/Flags.cpp create mode 100644 CLI/Flags.h create mode 100644 Common/include/Luau/ExperimentalFlags.h rename tests/conformance/{nextvar.lua => tables.lua} (96%) create mode 100644 tools/faillist.txt create mode 100644 tools/test_dcr.py diff --git a/Analysis/include/Luau/JsonEncoder.h b/Analysis/include/Luau/JsonEncoder.h index aa00390b..e79d9e62 100644 --- a/Analysis/include/Luau/JsonEncoder.h +++ b/Analysis/include/Luau/JsonEncoder.h @@ -2,12 +2,15 @@ #pragma once #include +#include namespace Luau { class AstNode; +struct Comment; std::string toJson(AstNode* node); +std::string toJson(AstNode* node, const std::vector& commentLocations); } // namespace Luau diff --git a/Analysis/include/Luau/VisitTypeVar.h b/Analysis/include/Luau/VisitTypeVar.h index ab4a397d..7229dafc 100644 --- a/Analysis/include/Luau/VisitTypeVar.h +++ b/Analysis/include/Luau/VisitTypeVar.h @@ -10,6 +10,7 @@ LUAU_FASTINT(LuauVisitRecursionLimit) LUAU_FASTFLAG(LuauNormalizeFlagIsConservative) +LUAU_FASTFLAG(LuauCompleteVisitor); namespace Luau { @@ -129,11 +130,11 @@ struct GenericTypeVarVisitor { return visit(ty); } - virtual bool visit(TypeId ty, const UnknownTypeVar& atv) + virtual bool visit(TypeId ty, const UnknownTypeVar& utv) { return visit(ty); } - virtual bool visit(TypeId ty, const NeverTypeVar& atv) + virtual bool visit(TypeId ty, const NeverTypeVar& ntv) { return visit(ty); } @@ -145,6 +146,14 @@ struct GenericTypeVarVisitor { return visit(ty); } + virtual bool visit(TypeId ty, const BlockedTypeVar& btv) + { + return visit(ty); + } + virtual bool visit(TypeId ty, const SingletonTypeVar& stv) + { + return visit(ty); + } virtual bool visit(TypePackId tp) { @@ -190,16 +199,12 @@ struct GenericTypeVarVisitor if (visit(ty, *btv)) traverse(btv->boundTo); } - else if (auto ftv = get(ty)) visit(ty, *ftv); - else if (auto gtv = get(ty)) visit(ty, *gtv); - else if (auto etv = get(ty)) visit(ty, *etv); - else if (auto ctv = get(ty)) { if (visit(ty, *ctv)) @@ -208,10 +213,8 @@ struct GenericTypeVarVisitor traverse(part); } } - else if (auto ptv = get(ty)) visit(ty, *ptv); - else if (auto ftv = get(ty)) { if (visit(ty, *ftv)) @@ -220,7 +223,6 @@ struct GenericTypeVarVisitor traverse(ftv->retTypes); } } - else if (auto ttv = get(ty)) { // Some visitors want to see bound tables, that's why we traverse the original type @@ -243,7 +245,6 @@ struct GenericTypeVarVisitor } } } - else if (auto mtv = get(ty)) { if (visit(ty, *mtv)) @@ -252,7 +253,6 @@ struct GenericTypeVarVisitor traverse(mtv->metatable); } } - else if (auto ctv = get(ty)) { if (visit(ty, *ctv)) @@ -267,10 +267,8 @@ struct GenericTypeVarVisitor traverse(*ctv->metatable); } } - else if (auto atv = get(ty)) visit(ty, *atv); - else if (auto utv = get(ty)) { if (visit(ty, *utv)) @@ -279,7 +277,6 @@ struct GenericTypeVarVisitor traverse(optTy); } } - else if (auto itv = get(ty)) { if (visit(ty, *itv)) @@ -288,6 +285,24 @@ struct GenericTypeVarVisitor traverse(partTy); } } + else if (!FFlag::LuauCompleteVisitor) + return visit_detail::unsee(seen, ty); + else if (get(ty)) + { + // Visiting into LazyTypeVar may necessarily cause infinite expansion, so we don't do that on purpose. + // Asserting also makes no sense, because the type _will_ happen here, most likely as a property of some ClassTypeVar + // that doesn't need to be expanded. + } + else if (auto stv = get(ty)) + visit(ty, *stv); + else if (auto btv = get(ty)) + visit(ty, *btv); + else if (auto utv = get(ty)) + visit(ty, *utv); + else if (auto ntv = get(ty)) + visit(ty, *ntv); + else + LUAU_ASSERT(!"GenericTypeVarVisitor::traverse(TypeId) is not exhaustive!"); visit_detail::unsee(seen, ty); } diff --git a/Analysis/src/JsonEncoder.cpp b/Analysis/src/JsonEncoder.cpp index 829ffa02..550c9b59 100644 --- a/Analysis/src/JsonEncoder.cpp +++ b/Analysis/src/JsonEncoder.cpp @@ -2,6 +2,7 @@ #include "Luau/JsonEncoder.h" #include "Luau/Ast.h" +#include "Luau/ParseResult.h" #include "Luau/StringUtils.h" #include "Luau/Common.h" @@ -75,6 +76,11 @@ struct AstJsonEncoder : public AstVisitor writeRaw(std::string_view{&c, 1}); } + void writeType(std::string_view propValue) + { + write("type", propValue); + } + template void write(std::string_view propName, const T& value) { @@ -98,7 +104,7 @@ struct AstJsonEncoder : public AstVisitor void write(double d) { char b[256]; - sprintf(b, "%g", d); + sprintf(b, "%.17g", d); writeRaw(b); } @@ -111,8 +117,12 @@ struct AstJsonEncoder : public AstVisitor { if (c == '"') writeRaw("\\\""); - else if (c == '\0') - writeRaw("\\\0"); + else if (c == '\\') + writeRaw("\\\\"); + else if (c < ' ') + writeRaw(format("\\u%04x", c)); + else if (c == '\n') + writeRaw("\\n"); else writeRaw(c); } @@ -189,10 +199,11 @@ struct AstJsonEncoder : public AstVisitor writeRaw("{"); bool c = pushComma(); if (local->annotation != nullptr) - write("type", local->annotation); + write("luauType", local->annotation); else - write("type", nullptr); + write("luauType", nullptr); write("name", local->name); + writeType("AstLocal"); write("location", local->location); popComma(c); writeRaw("}"); @@ -208,7 +219,7 @@ struct AstJsonEncoder : public AstVisitor { writeRaw("{"); bool c = pushComma(); - write("type", name); + writeType(name); writeNode(node); f(); popComma(c); @@ -358,6 +369,7 @@ struct AstJsonEncoder : public AstVisitor { writeRaw("{"); bool c = pushComma(); + writeType("AstTypeList"); write("types", typeList.types); if (typeList.tailType) write("tailType", typeList.tailType); @@ -369,9 +381,10 @@ struct AstJsonEncoder : public AstVisitor { writeRaw("{"); bool c = pushComma(); + writeType("AstGenericType"); write("name", genericType.name); if (genericType.defaultValue) - write("type", genericType.defaultValue); + write("luauType", genericType.defaultValue); popComma(c); writeRaw("}"); } @@ -380,9 +393,10 @@ struct AstJsonEncoder : public AstVisitor { writeRaw("{"); bool c = pushComma(); + writeType("AstGenericTypePack"); write("name", genericTypePack.name); if (genericTypePack.defaultValue) - write("type", genericTypePack.defaultValue); + write("luauType", genericTypePack.defaultValue); popComma(c); writeRaw("}"); } @@ -404,6 +418,7 @@ struct AstJsonEncoder : public AstVisitor { writeRaw("{"); bool c = pushComma(); + writeType("AstExprTableItem"); write("kind", item.kind); switch (item.kind) { @@ -419,6 +434,17 @@ struct AstJsonEncoder : public AstVisitor writeRaw("}"); } + void write(class AstExprIfElse* node) + { + writeNode(node, "AstExprIfElse", [&]() { + PROP(condition); + PROP(hasThen); + PROP(trueExpr); + PROP(hasElse); + PROP(falseExpr); + }); + } + void write(class AstExprTable* node) { writeNode(node, "AstExprTable", [&]() { @@ -431,11 +457,11 @@ struct AstJsonEncoder : public AstVisitor switch (op) { case AstExprUnary::Not: - return writeString("not"); + return writeString("Not"); case AstExprUnary::Minus: - return writeString("minus"); + return writeString("Minus"); case AstExprUnary::Len: - return writeString("len"); + return writeString("Len"); } } @@ -541,7 +567,7 @@ struct AstJsonEncoder : public AstVisitor void write(class AstStatWhile* node) { - writeNode(node, "AtStatWhile", [&]() { + writeNode(node, "AstStatWhile", [&]() { PROP(condition); PROP(body); PROP(hasDo); @@ -684,7 +710,8 @@ struct AstJsonEncoder : public AstVisitor writeRaw("{"); bool c = pushComma(); write("name", prop.name); - write("type", prop.ty); + writeType("AstDeclaredClassProp"); + write("luauType", prop.ty); popComma(c); writeRaw("}"); } @@ -731,8 +758,9 @@ struct AstJsonEncoder : public AstVisitor bool c = pushComma(); write("name", prop.name); + writeType("AstTableProp"); write("location", prop.location); - write("type", prop.type); + write("propType", prop.type); popComma(c); writeRaw("}"); @@ -745,6 +773,24 @@ struct AstJsonEncoder : public AstVisitor PROP(indexer); }); } + + void write(struct AstTableIndexer* indexer) + { + if (indexer) + { + writeRaw("{"); + bool c = pushComma(); + write("location", indexer->location); + write("indexType", indexer->indexType); + write("resultType", indexer->resultType); + popComma(c); + writeRaw("}"); + } + else + { + writeRaw("null"); + } + } void write(class AstTypeFunction* node) { @@ -836,6 +882,12 @@ struct AstJsonEncoder : public AstVisitor return false; } + bool visit(class AstExprIfElse* node) override + { + write(node); + return false; + } + bool visit(class AstExprLocal* node) override { write(node); @@ -1093,6 +1145,42 @@ struct AstJsonEncoder : public AstVisitor write(node); return false; } + + void writeComments(std::vector commentLocations) + { + bool commentComma = false; + for (Comment comment : commentLocations) + { + if (commentComma) + { + writeRaw(","); + } + else + { + commentComma = true; + } + writeRaw("{"); + bool c = pushComma(); + switch (comment.type) + { + case Lexeme::Comment: + writeType("Comment"); + break; + case Lexeme::BlockComment: + writeType("BlockComment"); + break; + case Lexeme::BrokenComment: + writeType("BrokenComment"); + break; + default: + break; + } + write("location", comment.location); + popComma(c); + writeRaw("}"); + + } + } }; std::string toJson(AstNode* node) @@ -1102,4 +1190,15 @@ std::string toJson(AstNode* node) return encoder.str(); } +std::string toJson(AstNode* node, const std::vector& commentLocations) +{ + AstJsonEncoder encoder; + encoder.writeRaw(R"({"root":)"); + node->visit(&encoder); + encoder.writeRaw(R"(,"commentLocations":[)"); + encoder.writeComments(commentLocations); + encoder.writeRaw("]}"); + return encoder.str(); +} + } // namespace Luau diff --git a/Analysis/src/Module.cpp b/Analysis/src/Module.cpp index 95eb125e..0603a042 100644 --- a/Analysis/src/Module.cpp +++ b/Analysis/src/Module.cpp @@ -17,6 +17,7 @@ LUAU_FASTFLAG(LuauLowerBoundsCalculation); LUAU_FASTFLAG(LuauNormalizeFlagIsConservative); LUAU_FASTFLAG(DebugLuauDeferredConstraintResolution); +LUAU_FASTFLAGVARIABLE(LuauForceExportSurfacesToBeNormal, false); namespace Luau { @@ -124,15 +125,21 @@ void Module::clonePublicInterface(InternalErrorReporter& ice) moduleScope2->returnType = returnType; // TODO varargPack } + ForceNormal forceNormal{&interfaceTypes}; + if (FFlag::LuauLowerBoundsCalculation) { normalize(returnType, interfaceTypes, ice); + if (FFlag::LuauForceExportSurfacesToBeNormal) + forceNormal.traverse(returnType); if (varargPack) + { normalize(*varargPack, interfaceTypes, ice); + if (FFlag::LuauForceExportSurfacesToBeNormal) + forceNormal.traverse(*varargPack); + } } - ForceNormal forceNormal{&interfaceTypes}; - if (exportedTypeBindings) { for (auto& [name, tf] : *exportedTypeBindings) @@ -147,6 +154,16 @@ void Module::clonePublicInterface(InternalErrorReporter& ice) // We're about to freeze the memory. We know that the flag is conservative by design. Cyclic tables // won't be marked normal. If the types aren't normal by now, they never will be. forceNormal.traverse(tf.type); + for (GenericTypeDefinition param : tf.typeParams) + { + forceNormal.traverse(param.ty); + + if (param.defaultValue) + { + normalize(*param.defaultValue, interfaceTypes, ice); + forceNormal.traverse(*param.defaultValue); + } + } } } } @@ -166,7 +183,12 @@ void Module::clonePublicInterface(InternalErrorReporter& ice) { ty = clone(ty, interfaceTypes, cloneState); if (FFlag::LuauLowerBoundsCalculation) + { normalize(ty, interfaceTypes, ice); + + if (FFlag::LuauForceExportSurfacesToBeNormal) + forceNormal.traverse(ty); + } } freeze(internalTypes); diff --git a/Analysis/src/TypeInfer.cpp b/Analysis/src/TypeInfer.cpp index 20777928..ea7d81e1 100644 --- a/Analysis/src/TypeInfer.cpp +++ b/Analysis/src/TypeInfer.cpp @@ -31,6 +31,7 @@ LUAU_FASTINTVARIABLE(LuauCheckRecursionLimit, 300) LUAU_FASTINTVARIABLE(LuauVisitRecursionLimit, 500) LUAU_FASTFLAG(LuauKnowsTheDataModel3) LUAU_FASTFLAG(LuauAutocompleteDynamicLimits) +LUAU_FASTFLAGVARIABLE(LuauExpectedTableUnionIndexerType, false) LUAU_FASTFLAGVARIABLE(LuauIndexSilenceErrors, false) LUAU_FASTFLAGVARIABLE(LuauLowerBoundsCalculation, false) LUAU_FASTFLAGVARIABLE(DebugLuauFreezeDuringUnification, false) @@ -38,7 +39,6 @@ LUAU_FASTFLAGVARIABLE(LuauSelfCallAutocompleteFix2, false) LUAU_FASTFLAGVARIABLE(LuauReduceUnionRecursion, false) LUAU_FASTFLAGVARIABLE(LuauReturnAnyInsteadOfICE, false) // Eventually removed as false. LUAU_FASTFLAG(LuauNormalizeFlagIsConservative) -LUAU_FASTFLAGVARIABLE(LuauReturnTypeInferenceInNonstrict, false) LUAU_FASTFLAGVARIABLE(DebugLuauSharedSelf, false); LUAU_FASTFLAGVARIABLE(LuauAlwaysQuantify, false); LUAU_FASTFLAGVARIABLE(LuauReportErrorsOnIndexerKeyMismatch, false) @@ -50,6 +50,7 @@ LUAU_FASTFLAGVARIABLE(LuauCheckGenericHOFTypes, false) LUAU_FASTFLAGVARIABLE(LuauBinaryNeedsExpectedTypesToo, false) LUAU_FASTFLAGVARIABLE(LuauNeverTypesAndOperatorsInference, false) LUAU_FASTFLAGVARIABLE(LuauReturnsFromCallsitesAreNotWidened, false) +LUAU_FASTFLAGVARIABLE(LuauCompleteVisitor, false) namespace Luau { @@ -890,7 +891,7 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatReturn& return_) TypePackId retPack = checkExprList(scope, return_.location, return_.list, false, {}, expectedTypes).type; - if (FFlag::LuauReturnTypeInferenceInNonstrict ? FFlag::LuauLowerBoundsCalculation : useConstrainedIntersections()) + if (useConstrainedIntersections()) { unifyLowerBound(retPack, scope->returnType, demoter.demotedLevel(scope->level), return_.location); return; @@ -1292,6 +1293,11 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatForIn& forin) for (size_t i = 2; i < varTypes.size(); ++i) unify(nilType, varTypes[i], forin.location); } + else if (isNonstrictMode()) + { + for (TypeId var : varTypes) + unify(anyType, var, forin.location); + } else { TypeId varTy = errorRecoveryType(loopScope); @@ -1385,12 +1391,7 @@ void TypeChecker::check(const ScopePtr& scope, TypeId ty, const ScopePtr& funSco // If in nonstrict mode and allowing redefinition of global function, restore the previous definition type // in case this function has a differing signature. The signature discrepancy will be caught in checkBlock. if (previouslyDefined) - { - if (FFlag::LuauReturnTypeInferenceInNonstrict && FFlag::LuauLowerBoundsCalculation) - quantify(funScope, ty, exprName->location); - globalBindings[name] = oldBinding; - } else globalBindings[name] = {quantify(funScope, ty, exprName->location), exprName->location}; @@ -2365,9 +2366,16 @@ WithPredicate TypeChecker::checkExpr(const ScopePtr& scope, const AstExp { std::vector expectedResultTypes; for (TypeId expectedOption : expectedUnion) + { if (const TableTypeVar* ttv = get(follow(expectedOption))) + { if (auto prop = ttv->props.find(key->value.data); prop != ttv->props.end()) expectedResultTypes.push_back(prop->second.type); + else if (FFlag::LuauExpectedTableUnionIndexerType && ttv->indexer && maybeString(ttv->indexer->indexType)) + expectedResultTypes.push_back(ttv->indexer->indexResultType); + } + } + if (expectedResultTypes.size() == 1) expectedResultType = expectedResultTypes[0]; else if (expectedResultTypes.size() > 1) @@ -3367,7 +3375,7 @@ std::pair TypeChecker::checkFunctionSignature(const ScopePtr& TypePackId retPack; if (expr.returnAnnotation) retPack = resolveTypePack(funScope, *expr.returnAnnotation); - else if (FFlag::LuauReturnTypeInferenceInNonstrict ? (!FFlag::LuauLowerBoundsCalculation && isNonstrictMode()) : isNonstrictMode()) + else if (isNonstrictMode()) retPack = anyTypePack; else if (expectedFunctionType && (!FFlag::LuauCheckGenericHOFTypes || (expectedFunctionType->generics.empty() && expectedFunctionType->genericPacks.empty()))) diff --git a/CLI/Analyze.cpp b/CLI/Analyze.cpp index 479eb167..51be0fea 100644 --- a/CLI/Analyze.cpp +++ b/CLI/Analyze.cpp @@ -7,6 +7,7 @@ #include "Luau/Transpiler.h" #include "FileUtils.h" +#include "Flags.h" #ifdef CALLGRIND #include @@ -223,9 +224,7 @@ int main(int argc, char** argv) { Luau::assertHandler() = assertionHandler; - for (Luau::FValue* flag = Luau::FValue::list; flag; flag = flag->next) - if (strncmp(flag->name, "Luau", 4) == 0) - flag->value = true; + setLuauFlagsDefault(); if (argc >= 2 && strcmp(argv[1], "--help") == 0) { @@ -252,12 +251,14 @@ int main(int argc, char** argv) annotate = true; else if (strcmp(argv[i], "--timetrace") == 0) FFlag::DebugLuauTimeTracing.value = true; + else if (strncmp(argv[i], "--fflags=", 9) == 0) + setLuauFlags(argv[i] + 9); } #if !defined(LUAU_ENABLE_TIME_TRACE) if (FFlag::DebugLuauTimeTracing) { - printf("To run with --timetrace, Luau has to be built with LUAU_ENABLE_TIME_TRACE enabled\n"); + fprintf(stderr, "To run with --timetrace, Luau has to be built with LUAU_ENABLE_TIME_TRACE enabled\n"); return 1; } #endif diff --git a/CLI/Ast.cpp b/CLI/Ast.cpp index 41e53973..f3f69be8 100644 --- a/CLI/Ast.cpp +++ b/CLI/Ast.cpp @@ -62,6 +62,7 @@ int main(int argc, char** argv) Luau::AstNameTable names(allocator); Luau::ParseOptions options; + options.captureComments = true; options.supportContinueStatement = true; options.allowTypeAnnotations = true; options.allowDeclarationSyntax = true; @@ -78,7 +79,7 @@ int main(int argc, char** argv) fprintf(stderr, "\n"); } - printf("%s", Luau::toJson(parseResult.root).c_str()); + printf("%s", Luau::toJson(parseResult.root, parseResult.commentLocations).c_str()); return parseResult.errors.size() > 0 ? 1 : 0; } diff --git a/CLI/Flags.cpp b/CLI/Flags.cpp new file mode 100644 index 00000000..4e261171 --- /dev/null +++ b/CLI/Flags.cpp @@ -0,0 +1,75 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +#include "Luau/Common.h" +#include "Luau/ExperimentalFlags.h" + +#include + +#include +#include + +static void setLuauFlag(std::string_view name, bool state) +{ + for (Luau::FValue* flag = Luau::FValue::list; flag; flag = flag->next) + { + if (name == flag->name) + { + flag->value = state; + return; + } + } + + fprintf(stderr, "Warning: unrecognized flag '%.*s'.\n", int(name.length()), name.data()); +} + +static void setLuauFlags(bool state) +{ + for (Luau::FValue* flag = Luau::FValue::list; flag; flag = flag->next) + if (strncmp(flag->name, "Luau", 4) == 0) + flag->value = state; +} + +void setLuauFlagsDefault() +{ + for (Luau::FValue* flag = Luau::FValue::list; flag; flag = flag->next) + if (strncmp(flag->name, "Luau", 4) == 0 && !Luau::isFlagExperimental(flag->name)) + flag->value = true; +} + +void setLuauFlags(const char* list) +{ + std::string_view rest = list; + + while (!rest.empty()) + { + size_t ending = rest.find(","); + std::string_view element = rest.substr(0, ending); + + if (size_t separator = element.find('='); separator != std::string_view::npos) + { + std::string_view key = element.substr(0, separator); + std::string_view value = element.substr(separator + 1); + + if (value == "true" || value == "True") + setLuauFlag(key, true); + else if (value == "false" || value == "False") + setLuauFlag(key, false); + else + fprintf(stderr, "Warning: unrecognized value '%.*s' for flag '%.*s'.\n", int(value.length()), value.data(), int(key.length()), + key.data()); + } + else + { + if (element == "true" || element == "True") + setLuauFlags(true); + else if (element == "false" || element == "False") + setLuauFlags(false); + else + setLuauFlag(element, true); + } + + if (ending != std::string_view::npos) + rest.remove_prefix(ending + 1); + else + break; + } +} diff --git a/CLI/Flags.h b/CLI/Flags.h new file mode 100644 index 00000000..8dfb0a29 --- /dev/null +++ b/CLI/Flags.h @@ -0,0 +1,5 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +#pragma once + +void setLuauFlagsDefault(); +void setLuauFlags(const char* list); diff --git a/CLI/Repl.cpp b/CLI/Repl.cpp index 5fe12bec..41360731 100644 --- a/CLI/Repl.cpp +++ b/CLI/Repl.cpp @@ -8,9 +8,10 @@ #include "Luau/BytecodeBuilder.h" #include "Luau/Parser.h" -#include "FileUtils.h" -#include "Profiler.h" #include "Coverage.h" +#include "FileUtils.h" +#include "Flags.h" +#include "Profiler.h" #include "isocline.h" @@ -97,7 +98,11 @@ static int lua_require(lua_State* L) // return the module from the cache lua_getfield(L, -1, name.c_str()); if (!lua_isnil(L, -1)) + { + // L stack: _MODULES result return finishrequire(L); + } + lua_pop(L, 1); std::optional source = readFile(name + ".luau"); @@ -109,6 +114,7 @@ static int lua_require(lua_State* L) } // module needs to run in a new thread, isolated from the rest + // note: we create ML on main thread so that it doesn't inherit environment of L lua_State* GL = lua_mainthread(L); lua_State* ML = lua_newthread(GL); lua_xmove(GL, L, 1); @@ -142,11 +148,12 @@ static int lua_require(lua_State* L) } } - // there's now a return value on top of ML; stack of L is MODULES thread + // there's now a return value on top of ML; L stack: _MODULES ML lua_xmove(ML, L, 1); lua_pushvalue(L, -1); lua_setfield(L, -4, name.c_str()); + // L stack: _MODULES ML result return finishrequire(L); } @@ -682,60 +689,11 @@ static int assertionHandler(const char* expr, const char* file, int line, const return 1; } -static void setLuauFlags(bool state) -{ - for (Luau::FValue* flag = Luau::FValue::list; flag; flag = flag->next) - { - if (strncmp(flag->name, "Luau", 4) == 0) - flag->value = state; - } -} - -static void setFlag(std::string_view name, bool state) -{ - for (Luau::FValue* flag = Luau::FValue::list; flag; flag = flag->next) - { - if (name == flag->name) - { - flag->value = state; - return; - } - } - - fprintf(stderr, "Warning: --fflag unrecognized flag '%.*s'.\n\n", int(name.length()), name.data()); -} - -static void applyFlagKeyValue(std::string_view element) -{ - if (size_t separator = element.find('='); separator != std::string_view::npos) - { - std::string_view key = element.substr(0, separator); - std::string_view value = element.substr(separator + 1); - - if (value == "true") - setFlag(key, true); - else if (value == "false") - setFlag(key, false); - else - fprintf(stderr, "Warning: --fflag unrecognized value '%.*s' for flag '%.*s'.\n\n", int(value.length()), value.data(), int(key.length()), - key.data()); - } - else - { - if (element == "true") - setLuauFlags(true); - else if (element == "false") - setLuauFlags(false); - else - setFlag(element, true); - } -} - int replMain(int argc, char** argv) { Luau::assertHandler() = assertionHandler; - setLuauFlags(true); + setLuauFlagsDefault(); CliMode mode = CliMode::Unknown; CompileFormat compileFormat{}; @@ -818,27 +776,10 @@ int replMain(int argc, char** argv) else if (strcmp(argv[i], "--timetrace") == 0) { FFlag::DebugLuauTimeTracing.value = true; - -#if !defined(LUAU_ENABLE_TIME_TRACE) - printf("To run with --timetrace, Luau has to be built with LUAU_ENABLE_TIME_TRACE enabled\n"); - return 1; -#endif } else if (strncmp(argv[i], "--fflags=", 9) == 0) { - std::string_view list = argv[i] + 9; - - while (!list.empty()) - { - size_t ending = list.find(","); - - applyFlagKeyValue(list.substr(0, ending)); - - if (ending != std::string_view::npos) - list.remove_prefix(ending + 1); - else - break; - } + setLuauFlags(argv[i] + 9); } else if (argv[i][0] == '-') { @@ -848,6 +789,14 @@ int replMain(int argc, char** argv) } } +#if !defined(LUAU_ENABLE_TIME_TRACE) + if (FFlag::DebugLuauTimeTracing) + { + fprintf(stderr, "To run with --timetrace, Luau has to be built with LUAU_ENABLE_TIME_TRACE enabled\n"); + return 1; + } +#endif + const std::vector files = getSourceFiles(argc, argv); if (mode == CliMode::Unknown) { diff --git a/CodeGen/include/Luau/AssemblyBuilderX64.h b/CodeGen/include/Luau/AssemblyBuilderX64.h index 65883b49..028b2d16 100644 --- a/CodeGen/include/Luau/AssemblyBuilderX64.h +++ b/CodeGen/include/Luau/AssemblyBuilderX64.h @@ -61,12 +61,22 @@ public: void call(Label& label); void call(OperandX64 op); + void int3(); + // AVX void vaddpd(OperandX64 dst, OperandX64 src1, OperandX64 src2); void vaddps(OperandX64 dst, OperandX64 src1, OperandX64 src2); void vaddsd(OperandX64 dst, OperandX64 src1, OperandX64 src2); void vaddss(OperandX64 dst, OperandX64 src1, OperandX64 src2); + void vsubsd(OperandX64 dst, OperandX64 src1, OperandX64 src2); + void vmulsd(OperandX64 dst, OperandX64 src1, OperandX64 src2); + void vdivsd(OperandX64 dst, OperandX64 src1, OperandX64 src2); + + void vxorpd(OperandX64 dst, OperandX64 src1, OperandX64 src2); + + void vcomisd(OperandX64 src1, OperandX64 src2); + void vsqrtpd(OperandX64 dst, OperandX64 src); void vsqrtps(OperandX64 dst, OperandX64 src); void vsqrtsd(OperandX64 dst, OperandX64 src1, OperandX64 src2); diff --git a/CodeGen/include/Luau/Condition.h b/CodeGen/include/Luau/Condition.h index 36cbda95..78e4515e 100644 --- a/CodeGen/include/Luau/Condition.h +++ b/CodeGen/include/Luau/Condition.h @@ -37,8 +37,6 @@ enum class Condition Zero, NotZero, - // TODO: ordered and unordered floating-point conditions - Count }; diff --git a/CodeGen/src/AssemblyBuilderX64.cpp b/CodeGen/src/AssemblyBuilderX64.cpp index 26347225..f88063cf 100644 --- a/CodeGen/src/AssemblyBuilderX64.cpp +++ b/CodeGen/src/AssemblyBuilderX64.cpp @@ -231,6 +231,7 @@ void AssemblyBuilderX64::lea(OperandX64 lhs, OperandX64 rhs) if (logText) log("lea", lhs, rhs); + LUAU_ASSERT(rhs.cat == CategoryX64::mem); placeBinaryRegAndRegMem(lhs, rhs, 0x8d, 0x8d); } @@ -314,6 +315,14 @@ void AssemblyBuilderX64::call(OperandX64 op) commit(); } +void AssemblyBuilderX64::int3() +{ + if (logText) + log("int3"); + + place(0xcc); +} + void AssemblyBuilderX64::vaddpd(OperandX64 dst, OperandX64 src1, OperandX64 src2) { placeAvx("vaddpd", dst, src1, src2, 0x58, false, AVX_0F, AVX_66); @@ -334,6 +343,31 @@ void AssemblyBuilderX64::vaddss(OperandX64 dst, OperandX64 src1, OperandX64 src2 placeAvx("vaddss", dst, src1, src2, 0x58, false, AVX_0F, AVX_F3); } +void AssemblyBuilderX64::vsubsd(OperandX64 dst, OperandX64 src1, OperandX64 src2) +{ + placeAvx("vsubsd", dst, src1, src2, 0x5c, false, AVX_0F, AVX_F2); +} + +void AssemblyBuilderX64::vmulsd(OperandX64 dst, OperandX64 src1, OperandX64 src2) +{ + placeAvx("vmulsd", dst, src1, src2, 0x59, false, AVX_0F, AVX_F2); +} + +void AssemblyBuilderX64::vdivsd(OperandX64 dst, OperandX64 src1, OperandX64 src2) +{ + placeAvx("vdivsd", dst, src1, src2, 0x5e, false, AVX_0F, AVX_F2); +} + +void AssemblyBuilderX64::vxorpd(OperandX64 dst, OperandX64 src1, OperandX64 src2) +{ + placeAvx("vxorpd", dst, src1, src2, 0x57, false, AVX_0F, AVX_66); +} + +void AssemblyBuilderX64::vcomisd(OperandX64 src1, OperandX64 src2) +{ + placeAvx("vcomisd", src1, src2, 0x2f, false, AVX_0F, AVX_66); +} + void AssemblyBuilderX64::vsqrtpd(OperandX64 dst, OperandX64 src) { placeAvx("vsqrtpd", dst, src, 0x51, false, AVX_0F, AVX_66); @@ -494,9 +528,10 @@ void AssemblyBuilderX64::placeBinaryRegMemAndImm(OperandX64 lhs, OperandX64 rhs, LUAU_ASSERT(lhs.cat == CategoryX64::reg || lhs.cat == CategoryX64::mem); LUAU_ASSERT(rhs.cat == CategoryX64::imm); - SizeX64 size = lhs.base.size; + SizeX64 size = lhs.cat == CategoryX64::reg ? lhs.base.size : lhs.memSize; + LUAU_ASSERT(size == SizeX64::byte || size == SizeX64::dword || size == SizeX64::qword); - placeRex(lhs.base); + placeRex(lhs); if (size == SizeX64::byte) { diff --git a/Common/include/Luau/ExperimentalFlags.h b/Common/include/Luau/ExperimentalFlags.h new file mode 100644 index 00000000..3525259d --- /dev/null +++ b/Common/include/Luau/ExperimentalFlags.h @@ -0,0 +1,26 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +#pragma once + +#include + +namespace Luau +{ + +inline bool isFlagExperimental(const char* flag) +{ + // Flags in this list are disabled by default in various command-line tools. They may have behavior that is not fully final, + // or critical bugs that are found after the code has been submitted. + static const char* kList[] = + { + "LuauLowerBoundsCalculation", + nullptr, // makes sure we always have at least one entry + }; + + for (const char* item: kList) + if (item && strcmp(item, flag) == 0) + return true; + + return false; +} + +} diff --git a/Compiler/src/Compiler.cpp b/Compiler/src/Compiler.cpp index 2b0f04f6..8f3befad 100644 --- a/Compiler/src/Compiler.cpp +++ b/Compiler/src/Compiler.cpp @@ -27,6 +27,7 @@ LUAU_FASTFLAGVARIABLE(LuauCompileNoIpairs, false) LUAU_FASTFLAGVARIABLE(LuauCompileFoldBuiltins, false) LUAU_FASTFLAGVARIABLE(LuauCompileBetterMultret, false) +LUAU_FASTFLAGVARIABLE(LuauCompileFreeReassign, false) namespace Luau { @@ -616,7 +617,7 @@ struct Compiler } else { - AstExprLocal* le = arg->as(); + AstExprLocal* le = FFlag::LuauCompileFreeReassign ? getExprLocal(arg) : arg->as(); Variable* lv = le ? variables.find(le->local) : nullptr; // if the argument is a local that isn't mutated, we will simply reuse the existing register @@ -2200,19 +2201,27 @@ struct Compiler compileLValueUse(lv, source, /* set= */ true); } - int getExprLocalReg(AstExpr* node) + AstExprLocal* getExprLocal(AstExpr* node) { if (AstExprLocal* expr = node->as()) + return expr; + else if (AstExprGroup* expr = node->as()) + return getExprLocal(expr->expr); + else if (AstExprTypeAssertion* expr = node->as()) + return getExprLocal(expr->expr); + else + return nullptr; + } + + int getExprLocalReg(AstExpr* node) + { + if (AstExprLocal* expr = getExprLocal(node)) { // note: this can't check expr->upvalue because upvalues may be upgraded to locals during inlining Local* l = locals.find(expr->local); return l && l->allocated ? l->reg : -1; } - else if (AstExprGroup* expr = node->as()) - return getExprLocalReg(expr->expr); - else if (AstExprTypeAssertion* expr = node->as()) - return getExprLocalReg(expr->expr); else return -1; } @@ -2498,6 +2507,22 @@ struct Compiler if (options.optimizationLevel >= 1 && options.debugLevel <= 1 && areLocalsRedundant(stat)) return; + // Optimization: for 1-1 local assignments, we can reuse the register *if* neither local is mutated + if (FFlag::LuauCompileFreeReassign && options.optimizationLevel >= 1 && stat->vars.size == 1 && stat->values.size == 1) + { + if (AstExprLocal* re = getExprLocal(stat->values.data[0])) + { + Variable* lv = variables.find(stat->vars.data[0]); + Variable* rv = variables.find(re->local); + + if (int reg = getExprLocalReg(re); reg >= 0 && (!lv || !lv->written) && (!rv || !rv->written)) + { + pushLocal(stat->vars.data[0], uint8_t(reg)); + return; + } + } + } + // note: allocReg in this case allocates into parent block register - note that we don't have RegScope here uint8_t vars = allocReg(stat, unsigned(stat->vars.size)); diff --git a/Makefile b/Makefile index 7843746e..4323a321 100644 --- a/Makefile +++ b/Makefile @@ -31,15 +31,15 @@ ISOCLINE_SOURCES=extern/isocline/src/isocline.c ISOCLINE_OBJECTS=$(ISOCLINE_SOURCES:%=$(BUILD)/%.o) ISOCLINE_TARGET=$(BUILD)/libisocline.a -TESTS_SOURCES=$(wildcard tests/*.cpp) CLI/FileUtils.cpp CLI/Profiler.cpp CLI/Coverage.cpp CLI/Repl.cpp +TESTS_SOURCES=$(wildcard tests/*.cpp) CLI/FileUtils.cpp CLI/Flags.cpp CLI/Profiler.cpp CLI/Coverage.cpp CLI/Repl.cpp TESTS_OBJECTS=$(TESTS_SOURCES:%=$(BUILD)/%.o) TESTS_TARGET=$(BUILD)/luau-tests -REPL_CLI_SOURCES=CLI/FileUtils.cpp CLI/Profiler.cpp CLI/Coverage.cpp CLI/Repl.cpp CLI/ReplEntry.cpp +REPL_CLI_SOURCES=CLI/FileUtils.cpp CLI/Flags.cpp CLI/Profiler.cpp CLI/Coverage.cpp CLI/Repl.cpp CLI/ReplEntry.cpp REPL_CLI_OBJECTS=$(REPL_CLI_SOURCES:%=$(BUILD)/%.o) REPL_CLI_TARGET=$(BUILD)/luau -ANALYZE_CLI_SOURCES=CLI/FileUtils.cpp CLI/Analyze.cpp +ANALYZE_CLI_SOURCES=CLI/FileUtils.cpp CLI/Flags.cpp CLI/Analyze.cpp ANALYZE_CLI_OBJECTS=$(ANALYZE_CLI_SOURCES:%=$(BUILD)/%.o) ANALYZE_CLI_TARGET=$(BUILD)/luau-analyze @@ -117,15 +117,18 @@ $(REPL_CLI_TARGET): LDFLAGS+=-lpthread fuzz-proto fuzz-prototest: LDFLAGS+=build/libprotobuf-mutator/src/libfuzzer/libprotobuf-mutator-libfuzzer.a build/libprotobuf-mutator/src/libprotobuf-mutator.a build/libprotobuf-mutator/external.protobuf/lib/libprotobuf.a # pseudo targets -.PHONY: all test clean coverage format luau-size +.PHONY: all test clean coverage format luau-size aliases -all: $(REPL_CLI_TARGET) $(ANALYZE_CLI_TARGET) $(TESTS_TARGET) +all: $(REPL_CLI_TARGET) $(ANALYZE_CLI_TARGET) $(TESTS_TARGET) aliases + +aliases: luau luau-analyze test: $(TESTS_TARGET) $(TESTS_TARGET) $(TESTS_ARGS) clean: rm -rf $(BUILD) + rm -rf luau luau-analyze coverage: $(TESTS_TARGET) $(TESTS_TARGET) --fflags=true diff --git a/Sources.cmake b/Sources.cmake index 69f5e1b7..a4563019 100644 --- a/Sources.cmake +++ b/Sources.cmake @@ -4,6 +4,7 @@ if(NOT ${CMAKE_VERSION} VERSION_LESS "3.19") target_sources(Luau.Common PRIVATE Common/include/Luau/Common.h Common/include/Luau/Bytecode.h + Common/include/Luau/ExperimentalFlags.h ) endif() @@ -220,6 +221,8 @@ if(TARGET Luau.Repl.CLI) CLI/Coverage.cpp CLI/FileUtils.h CLI/FileUtils.cpp + CLI/Flags.h + CLI/Flags.cpp CLI/Profiler.h CLI/Profiler.cpp CLI/Repl.cpp @@ -231,6 +234,8 @@ if(TARGET Luau.Analyze.CLI) target_sources(Luau.Analyze.CLI PRIVATE CLI/FileUtils.h CLI/FileUtils.cpp + CLI/Flags.h + CLI/Flags.cpp CLI/Analyze.cpp) endif() @@ -321,6 +326,8 @@ if(TARGET Luau.CLI.Test) CLI/Coverage.cpp CLI/FileUtils.h CLI/FileUtils.cpp + CLI/Flags.h + CLI/Flags.cpp CLI/Profiler.h CLI/Profiler.cpp CLI/Repl.cpp diff --git a/VM/include/lua.h b/VM/include/lua.h index 514ea361..28d7b1c5 100644 --- a/VM/include/lua.h +++ b/VM/include/lua.h @@ -300,6 +300,7 @@ LUA_API uintptr_t lua_encodepointer(lua_State* L, uintptr_t p); LUA_API double lua_clock(); +LUA_API void lua_setuserdatatag(lua_State* L, int idx, int tag); LUA_API void lua_setuserdatadtor(lua_State* L, int tag, void (*dtor)(lua_State*, void*)); LUA_API void lua_clonefunction(lua_State* L, int idx); diff --git a/VM/src/lapi.cpp b/VM/src/lapi.cpp index 0bcb8656..e3354ea2 100644 --- a/VM/src/lapi.cpp +++ b/VM/src/lapi.cpp @@ -854,7 +854,7 @@ void lua_rawset(lua_State* L, int idx) StkId t = index2addr(L, idx); api_check(L, ttistable(t)); if (hvalue(t)->readonly) - luaG_runerror(L, "Attempt to modify a readonly table"); + luaG_readonlyerror(L); setobj2t(L, luaH_set(L, hvalue(t), L->top - 2), L->top - 1); luaC_barriert(L, hvalue(t), L->top - 1); L->top -= 2; @@ -867,7 +867,7 @@ void lua_rawseti(lua_State* L, int idx, int n) StkId o = index2addr(L, idx); api_check(L, ttistable(o)); if (hvalue(o)->readonly) - luaG_runerror(L, "Attempt to modify a readonly table"); + luaG_readonlyerror(L); setobj2t(L, luaH_setnum(L, hvalue(o), n), L->top - 1); luaC_barriert(L, hvalue(o), L->top - 1); L->top--; @@ -890,7 +890,7 @@ int lua_setmetatable(lua_State* L, int objindex) case LUA_TTABLE: { if (hvalue(obj)->readonly) - luaG_runerror(L, "Attempt to modify a readonly table"); + luaG_readonlyerror(L); hvalue(obj)->metatable = mt; if (mt) luaC_objbarrier(L, hvalue(obj), mt); @@ -1320,6 +1320,14 @@ void lua_unref(lua_State* L, int ref) return; } +void lua_setuserdatatag(lua_State* L, int idx, int tag) +{ + api_check(L, unsigned(tag) < LUA_UTAG_LIMIT); + StkId o = index2addr(L, idx); + api_check(L, ttisuserdata(o)); + uvalue(o)->tag = uint8_t(tag); +} + void lua_setuserdatadtor(lua_State* L, int tag, void (*dtor)(lua_State*, void*)) { api_check(L, unsigned(tag) < LUA_UTAG_LIMIT); diff --git a/VM/src/ldebug.cpp b/VM/src/ldebug.cpp index ef486090..29015565 100644 --- a/VM/src/ldebug.cpp +++ b/VM/src/ldebug.cpp @@ -269,6 +269,11 @@ l_noret luaG_indexerror(lua_State* L, const TValue* p1, const TValue* p2) luaG_runerror(L, "attempt to index %s with %s", t1, t2); } +l_noret luaG_readonlyerror(lua_State* L) +{ + luaG_runerror(L, "attempt to modify a readonly table"); +} + static void pusherror(lua_State* L, const char* msg) { CallInfo* ci = L->ci; diff --git a/VM/src/ldebug.h b/VM/src/ldebug.h index 75bb8dcc..8e03db36 100644 --- a/VM/src/ldebug.h +++ b/VM/src/ldebug.h @@ -19,6 +19,7 @@ LUAI_FUNC l_noret luaG_concaterror(lua_State* L, StkId p1, StkId p2); LUAI_FUNC l_noret luaG_aritherror(lua_State* L, const TValue* p1, const TValue* p2, TMS op); LUAI_FUNC l_noret luaG_ordererror(lua_State* L, const TValue* p1, const TValue* p2, TMS op); LUAI_FUNC l_noret luaG_indexerror(lua_State* L, const TValue* p1, const TValue* p2); +LUAI_FUNC l_noret luaG_readonlyerror(lua_State* L); LUAI_FUNC LUA_PRINTF_ATTR(2, 3) l_noret luaG_runerrorL(lua_State* L, const char* fmt, ...); LUAI_FUNC void luaG_pusherror(lua_State* L, const char* error); diff --git a/VM/src/ltablib.cpp b/VM/src/ltablib.cpp index 27187c61..dc653338 100644 --- a/VM/src/ltablib.cpp +++ b/VM/src/ltablib.cpp @@ -79,7 +79,7 @@ static void moveelements(lua_State* L, int srct, int dstt, int f, int e, int t) Table* dst = hvalue(L->base + (dstt - 1)); if (dst->readonly) - luaG_runerror(L, "Attempt to modify a readonly table"); + luaG_readonlyerror(L); int n = e - f + 1; /* number of elements to move */ @@ -204,7 +204,7 @@ static int tmove(lua_State* L) Table* dst = hvalue(L->base + (tt - 1)); if (dst->readonly) /* also checked in moveelements, but this blocks resizes of r/o tables */ - luaG_runerror(L, "Attempt to modify a readonly table"); + luaG_readonlyerror(L); if (t > 0 && (t - 1) <= dst->sizearray && (t - 1 + n) > dst->sizearray) { /* grow the destination table array */ @@ -482,7 +482,7 @@ static int tclear(lua_State* L) Table* tt = hvalue(L->base); if (tt->readonly) - luaG_runerror(L, "Attempt to modify a readonly table"); + luaG_readonlyerror(L); luaH_clear(tt); return 0; diff --git a/VM/src/lvmutils.cpp b/VM/src/lvmutils.cpp index b9e762eb..be4e99a9 100644 --- a/VM/src/lvmutils.cpp +++ b/VM/src/lvmutils.cpp @@ -128,7 +128,7 @@ void luaV_gettable(lua_State* L, const TValue* t, TValue* key, StkId val) } t = tm; /* else repeat with `tm' */ } - luaG_runerror(L, "loop in gettable"); + luaG_runerror(L, "'__index' chain too long; possible loop"); } void luaV_settable(lua_State* L, const TValue* t, TValue* key, StkId val) @@ -143,7 +143,7 @@ void luaV_settable(lua_State* L, const TValue* t, TValue* key, StkId val) Table* h = hvalue(t); if (h->readonly) - luaG_runerror(L, "Attempt to modify a readonly table"); + luaG_readonlyerror(L); TValue* oldval = luaH_set(L, h, key); /* do a primitive set */ @@ -169,7 +169,7 @@ void luaV_settable(lua_State* L, const TValue* t, TValue* key, StkId val) setobj(L, &temp, tm); /* avoid pointing inside table (may rehash) */ t = &temp; } - luaG_runerror(L, "loop in settable"); + luaG_runerror(L, "'__newindex' chain too long; possible loop"); } static int call_binTM(lua_State* L, const TValue* p1, const TValue* p2, StkId res, TMS event) diff --git a/bench/tests/tictactoe.lua b/bench/tests/tictactoe.lua index ae63f5f7..91d38f95 100644 --- a/bench/tests/tictactoe.lua +++ b/bench/tests/tictactoe.lua @@ -139,7 +139,7 @@ function test() for _, curr_qdr in pairs(negaMax.index_quadruplets) do -- iterate over all index quadruplets -- count the empty positions and positions occupied by the side whos move it is local player_plus_fields, player_minus_fields, empties = 0, 0, 0 - for _, index in pairs(curr_qdr) do -- iterate over all indices + for _, index in next, curr_qdr do -- iterate over all indices if board[index] == 0 then empties = empties + 1 elseif board[index] == 1 then @@ -225,4 +225,4 @@ function test() return t1-t0 end -bench.runCode(test, "tictactoe") \ No newline at end of file +bench.runCode(test, "tictactoe") diff --git a/fuzz/proto.cpp b/fuzz/proto.cpp index 22483f9e..f64b6157 100644 --- a/fuzz/proto.cpp +++ b/fuzz/proto.cpp @@ -333,7 +333,7 @@ DEFINE_PROTO_FUZZER(const luau::ModuleSet& message) try { Luau::BytecodeBuilder bcb; - Luau::compileOrThrow(bcb, parseResult.root, parseNameTable, compileOptions); + Luau::compileOrThrow(bcb, parseResult, parseNameTable, compileOptions); bytecode = bcb.getBytecode(); } catch (const Luau::CompileError&) diff --git a/tests/AssemblyBuilderX64.test.cpp b/tests/AssemblyBuilderX64.test.cpp index 15813ae9..28ce6a82 100644 --- a/tests/AssemblyBuilderX64.test.cpp +++ b/tests/AssemblyBuilderX64.test.cpp @@ -155,6 +155,13 @@ TEST_CASE_FIXTURE(AssemblyBuilderX64Fixture, "BaseBinaryInstructionForms") SINGLE_COMPARE(add(qword[rax + r13 * 2 + 0x1b], rsi), 0x4a, 0x01, 0x74, 0x68, 0x1b); SINGLE_COMPARE(add(qword[rbp + rbx * 2], rsi), 0x48, 0x01, 0x74, 0x5d, 0x00); SINGLE_COMPARE(add(qword[rsp + r10 * 2 + 0x1b], r10), 0x4e, 0x01, 0x54, 0x54, 0x1b); + + // [addr], imm + SINGLE_COMPARE(add(byte[rax], 2), 0x80, 0x00, 0x02); + SINGLE_COMPARE(add(dword[rax], 2), 0x83, 0x00, 0x02); + SINGLE_COMPARE(add(dword[rax], 0xabcd), 0x81, 0x00, 0xcd, 0xab, 0x00, 0x00); + SINGLE_COMPARE(add(qword[rax], 2), 0x48, 0x83, 0x00, 0x02); + SINGLE_COMPARE(add(qword[rax], 0xabcd), 0x48, 0x81, 0x00, 0xcd, 0xab, 0x00, 0x00); } TEST_CASE_FIXTURE(AssemblyBuilderX64Fixture, "BaseUnaryInstructionForms") @@ -304,6 +311,13 @@ TEST_CASE_FIXTURE(AssemblyBuilderX64Fixture, "AVXBinaryInstructionForms") SINGLE_COMPARE(vaddps(xmm9, xmm12, xmmword[r9 + r14 * 2 + 0x1c]), 0xc4, 0x01, 0x98, 0x58, 0x4c, 0x71, 0x1c); SINGLE_COMPARE(vaddps(ymm1, ymm2, ymm3), 0xc4, 0xe1, 0xec, 0x58, 0xcb); SINGLE_COMPARE(vaddps(ymm9, ymm12, ymmword[r9 + r14 * 2 + 0x1c]), 0xc4, 0x01, 0x9c, 0x58, 0x4c, 0x71, 0x1c); + + // Coverage for other instructions that follow the same pattern + SINGLE_COMPARE(vsubsd(xmm8, xmm10, xmm14), 0xc4, 0x41, 0xab, 0x5c, 0xc6); + SINGLE_COMPARE(vmulsd(xmm8, xmm10, xmm14), 0xc4, 0x41, 0xab, 0x59, 0xc6); + SINGLE_COMPARE(vdivsd(xmm8, xmm10, xmm14), 0xc4, 0x41, 0xab, 0x5e, 0xc6); + + SINGLE_COMPARE(vxorpd(xmm8, xmm10, xmm14), 0xc4, 0x41, 0xa9, 0x57, 0xc6); } TEST_CASE_FIXTURE(AssemblyBuilderX64Fixture, "AVXUnaryMergeInstructionForms") @@ -318,6 +332,9 @@ TEST_CASE_FIXTURE(AssemblyBuilderX64Fixture, "AVXUnaryMergeInstructionForms") SINGLE_COMPARE(vsqrtsd(xmm8, xmm10, qword[r9]), 0xc4, 0x41, 0xab, 0x51, 0x01); SINGLE_COMPARE(vsqrtss(xmm8, xmm10, xmm14), 0xc4, 0x41, 0xaa, 0x51, 0xc6); SINGLE_COMPARE(vsqrtss(xmm8, xmm10, dword[r9]), 0xc4, 0x41, 0xaa, 0x51, 0x01); + + // Coverage for other instructions that follow the same pattern + SINGLE_COMPARE(vcomisd(xmm8, xmm10), 0xc4, 0x41, 0xf9, 0x2f, 0xc2); } TEST_CASE_FIXTURE(AssemblyBuilderX64Fixture, "AVXMoveInstructionForms") @@ -342,6 +359,11 @@ TEST_CASE_FIXTURE(AssemblyBuilderX64Fixture, "AVXMoveInstructionForms") SINGLE_COMPARE(vmovups(ymm8, ymmword[r9]), 0xc4, 0x41, 0xfc, 0x10, 0x01); } +TEST_CASE_FIXTURE(AssemblyBuilderX64Fixture, "MiscInstructions") +{ + SINGLE_COMPARE(int3(), 0xcc); +} + TEST_CASE("LogTest") { AssemblyBuilderX64 build(/* logText= */ true); @@ -366,6 +388,7 @@ TEST_CASE("LogTest") build.vmovapd(xmmword[rax], xmm11); build.pop(r12); build.ret(); + build.int3(); build.finalize(); @@ -388,6 +411,7 @@ TEST_CASE("LogTest") vmovapd xmmword ptr [rax],xmm11 pop r12 ret + int3 )"; CHECK(same); } diff --git a/tests/Compiler.test.cpp b/tests/Compiler.test.cpp index 46fde067..c1917638 100644 --- a/tests/Compiler.test.cpp +++ b/tests/Compiler.test.cpp @@ -3793,6 +3793,8 @@ RETURN R0 1 TEST_CASE("SharedClosure") { + ScopedFastFlag sff("LuauCompileFreeReassign", true); + // closures can be shared even if functions refer to upvalues, as long as upvalues are top-level CHECK_EQ("\n" + compileFunction(R"( local val = ... @@ -3940,11 +3942,10 @@ LOADN R2 1 LOADN R0 10 LOADN R1 1 FORNPREP R0 L5 -L4: MOVE R3 R2 -GETIMPORT R4 1 -NEWCLOSURE R5 P2 -CAPTURE VAL R3 -CALL R4 1 0 +L4: GETIMPORT R3 1 +NEWCLOSURE R4 P2 +CAPTURE VAL R2 +CALL R3 1 0 FORNLOOP R0 L4 L5: RETURN R0 0 )"); @@ -6157,4 +6158,88 @@ RETURN R0 1 )"); } +TEST_CASE("LocalReassign") +{ + ScopedFastFlag sff("LuauCompileFreeReassign", true); + + // locals can be re-assigned and the register gets reused + CHECK_EQ("\n" + compileFunction0(R"( +local function test(a, b) + local c = a + return c + b +end +)"), R"( +ADD R2 R0 R1 +RETURN R2 1 +)"); + + // this works if the expression is using type casts or grouping + CHECK_EQ("\n" + compileFunction0(R"( +local function test(a, b) + local c = (a :: number) + return c + b +end +)"), R"( +ADD R2 R0 R1 +RETURN R2 1 +)"); + + // the optimization requires that neither local is mutated + CHECK_EQ("\n" + compileFunction0(R"( +local function test(a, b) + local c = a + c += 0 + local d = b + b += 0 + return c + d +end +)"), R"( +MOVE R2 R0 +ADDK R2 R2 K0 +MOVE R3 R1 +ADDK R1 R1 K0 +ADD R4 R2 R3 +RETURN R4 1 +)"); + + // sanity check for two values + CHECK_EQ("\n" + compileFunction0(R"( +local function test(a, b) + local c = a + local d = b + return c + d +end +)"), R"( +ADD R2 R0 R1 +RETURN R2 1 +)"); + + // note: we currently only support this for single assignments + CHECK_EQ("\n" + compileFunction0(R"( +local function test(a, b) + local c, d = a, b + return c + d +end +)"), R"( +MOVE R2 R0 +MOVE R3 R1 +ADD R4 R2 R3 +RETURN R4 1 +)"); + + // of course, captures capture the original register as well (by value since it's immutable) + CHECK_EQ("\n" + compileFunction(R"( +local function test(a, b) + local c = a + local d = b + return function() return c + d end +end +)", 1), R"( +NEWCLOSURE R2 P0 +CAPTURE VAL R0 +CAPTURE VAL R1 +RETURN R2 1 +)"); +} + TEST_SUITE_END(); diff --git a/tests/Conformance.test.cpp b/tests/Conformance.test.cpp index 52a578e5..fc8ab2f9 100644 --- a/tests/Conformance.test.cpp +++ b/tests/Conformance.test.cpp @@ -254,9 +254,9 @@ TEST_CASE("Math") runConformance("math.lua"); } -TEST_CASE("Table") +TEST_CASE("Tables") { - runConformance("nextvar.lua", [](lua_State* L) { + runConformance("tables.lua", [](lua_State* L) { lua_pushcfunction( L, [](lua_State* L) { @@ -1202,6 +1202,10 @@ TEST_CASE("UserdataApi") CHECK(lua_touserdatatagged(L, -1, 41) == nullptr); CHECK(lua_userdatatag(L, -1) == 42); + lua_setuserdatatag(L, -1, 43); + CHECK(lua_userdatatag(L, -1) == 43); + lua_setuserdatatag(L, -1, 42); + // user data with inline dtor void* ud3 = lua_newuserdatadtor(L, 4, [](void* data) { dtorhits += *(int*)data; diff --git a/tests/Frontend.test.cpp b/tests/Frontend.test.cpp index 4efa74d9..0382f227 100644 --- a/tests/Frontend.test.cpp +++ b/tests/Frontend.test.cpp @@ -1025,4 +1025,73 @@ TEST_CASE("check_without_builtin_next") frontend.check("Module/B"); } +TEST_CASE_FIXTURE(BuiltinsFixture, "reexport_cyclic_type") +{ + ScopedFastFlag sff[] = { + {"LuauForceExportSurfacesToBeNormal", true}, + {"LuauLowerBoundsCalculation", true}, + {"LuauNormalizeFlagIsConservative", true}, + }; + + fileResolver.source["Module/A"] = R"( + type F = (set: G) -> () + + export type G = { + forEach: (a: F) -> (), + } + + function X(a: F): () + end + + return X + )"; + + fileResolver.source["Module/B"] = R"( + --!strict + local A = require(script.Parent.A) + + export type G = A.G + + return { + A = A, + } + )"; + + CheckResult result = frontend.check("Module/B"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(BuiltinsFixture, "reexport_type_alias") +{ + ScopedFastFlag sff[] = { + {"LuauForceExportSurfacesToBeNormal", true}, + {"LuauLowerBoundsCalculation", true}, + {"LuauNormalizeFlagIsConservative", true}, + }; + + fileResolver.source["Module/A"] = R"( + type KeyOfTestEvents = "test-file-start" | "test-file-success" | "test-file-failure" | "test-case-result" + type unknown = any + + export type TestFileEvent = ( + eventName: T, + args: any --[[ ROBLOX TODO: Unhandled node for type: TSIndexedAccessType ]] --[[ TestEvents[T] ]] + ) -> unknown + + return {} + )"; + + fileResolver.source["Module/B"] = R"( + --!strict + local A = require(script.Parent.A) + + export type TestFileEvent = A.TestFileEvent + )"; + + CheckResult result = frontend.check("Module/B"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + TEST_SUITE_END(); diff --git a/tests/JsonEncoder.test.cpp b/tests/JsonEncoder.test.cpp index 8a263bd2..b16ad3e2 100644 --- a/tests/JsonEncoder.test.cpp +++ b/tests/JsonEncoder.test.cpp @@ -56,10 +56,19 @@ TEST_CASE("encode_constants") AstExprConstantNil nil{Location()}; AstExprConstantBool b{Location(), true}; AstExprConstantNumber n{Location(), 8.2}; + AstExprConstantNumber bigNum{Location(), 0.1677721600000003}; + + AstArray charString; + charString.data = const_cast("a\x1d\0\\\"b"); + charString.size = 6; + + AstExprConstantString needsEscaping{Location(), charString}; CHECK_EQ(R"({"type":"AstExprConstantNil","location":"0,0 - 0,0"})", toJson(&nil)); CHECK_EQ(R"({"type":"AstExprConstantBool","location":"0,0 - 0,0","value":true})", toJson(&b)); - CHECK_EQ(R"({"type":"AstExprConstantNumber","location":"0,0 - 0,0","value":8.2})", toJson(&n)); + CHECK_EQ(R"({"type":"AstExprConstantNumber","location":"0,0 - 0,0","value":8.1999999999999993})", toJson(&n)); + CHECK_EQ(R"({"type":"AstExprConstantNumber","location":"0,0 - 0,0","value":0.16777216000000031})", toJson(&bigNum)); + CHECK_EQ("{\"type\":\"AstExprConstantString\",\"location\":\"0,0 - 0,0\",\"value\":\"a\\u001d\\u0000\\\\\\\"b\"}", toJson(&needsEscaping)); } TEST_CASE("basic_escaping") @@ -87,7 +96,7 @@ TEST_CASE("encode_AstStatBlock") AstStatBlock block{Location(), bodyArray}; CHECK_EQ( - (R"({"type":"AstStatBlock","location":"0,0 - 0,0","body":[{"type":"AstStatLocal","location":"0,0 - 0,0","vars":[{"type":null,"name":"a_local","location":"0,0 - 0,0"}],"values":[]}]})"), + (R"({"type":"AstStatBlock","location":"0,0 - 0,0","body":[{"type":"AstStatLocal","location":"0,0 - 0,0","vars":[{"luauType":null,"name":"a_local","type":"AstLocal","location":"0,0 - 0,0"}],"values":[]}]})"), toJson(&block)); } @@ -106,7 +115,31 @@ TEST_CASE_FIXTURE(JsonEncoderFixture, "encode_tables") CHECK( json == - R"({"type":"AstStatBlock","location":"0,0 - 6,4","body":[{"type":"AstStatLocal","location":"1,8 - 5,9","vars":[{"type":{"type":"AstTypeTable","location":"1,17 - 3,9","props":[{"name":"foo","location":"2,12 - 2,15","type":{"type":"AstTypeReference","location":"2,17 - 2,23","name":"number","parameters":[]}}],"indexer":false},"name":"x","location":"1,14 - 1,15"}],"values":[{"type":"AstExprTable","location":"3,12 - 5,9","items":[{"kind":"record","key":{"type":"AstExprConstantString","location":"4,12 - 4,15","value":"foo"},"value":{"type":"AstExprConstantNumber","location":"4,18 - 4,21","value":123}}]}]}]})"); + R"({"type":"AstStatBlock","location":"0,0 - 6,4","body":[{"type":"AstStatLocal","location":"1,8 - 5,9","vars":[{"luauType":{"type":"AstTypeTable","location":"1,17 - 3,9","props":[{"name":"foo","type":"AstTableProp","location":"2,12 - 2,15","propType":{"type":"AstTypeReference","location":"2,17 - 2,23","name":"number","parameters":[]}}],"indexer":null},"name":"x","type":"AstLocal","location":"1,14 - 1,15"}],"values":[{"type":"AstExprTable","location":"3,12 - 5,9","items":[{"type":"AstExprTableItem","kind":"record","key":{"type":"AstExprConstantString","location":"4,12 - 4,15","value":"foo"},"value":{"type":"AstExprConstantNumber","location":"4,18 - 4,21","value":123}}]}]}]})"); +} + +TEST_CASE_FIXTURE(JsonEncoderFixture, "encode_table_array") +{ + std::string src = R"(type X = {string})"; + + AstStatBlock* root = expectParse(src); + std::string json = toJson(root); + + CHECK( + json == + R"({"type":"AstStatBlock","location":"0,0 - 0,17","body":[{"type":"AstStatTypeAlias","location":"0,0 - 0,17","name":"X","generics":[],"genericPacks":[],"type":{"type":"AstTypeTable","location":"0,9 - 0,17","props":[],"indexer":{"location":"0,10 - 0,16","indexType":{"type":"AstTypeReference","location":"0,10 - 0,16","name":"number","parameters":[]},"resultType":{"type":"AstTypeReference","location":"0,10 - 0,16","name":"string","parameters":[]}}},"exported":false}]})"); +} + +TEST_CASE_FIXTURE(JsonEncoderFixture, "encode_table_indexer") +{ + std::string src = R"(type X = {string})"; + + AstStatBlock* root = expectParse(src); + std::string json = toJson(root); + + CHECK( + json == + R"({"type":"AstStatBlock","location":"0,0 - 0,17","body":[{"type":"AstStatTypeAlias","location":"0,0 - 0,17","name":"X","generics":[],"genericPacks":[],"type":{"type":"AstTypeTable","location":"0,9 - 0,17","props":[],"indexer":{"location":"0,10 - 0,16","indexType":{"type":"AstTypeReference","location":"0,10 - 0,16","name":"number","parameters":[]},"resultType":{"type":"AstTypeReference","location":"0,10 - 0,16","name":"string","parameters":[]}}},"exported":false}]})"); } TEST_CASE("encode_AstExprGroup") @@ -132,12 +165,23 @@ TEST_CASE("encode_AstExprGlobal") CHECK(json == expected); } +TEST_CASE_FIXTURE(JsonEncoderFixture, "encode_AstExprIfThen") +{ + AstStat* statement = expectParseStatement("local a = if x then y else z"); + + std::string_view expected = + R"({"type":"AstStatLocal","location":"0,0 - 0,28","vars":[{"luauType":null,"name":"a","type":"AstLocal","location":"0,6 - 0,7"}],"values":[{"type":"AstExprIfElse","location":"0,10 - 0,28","condition":{"type":"AstExprGlobal","location":"0,13 - 0,14","global":"x"},"hasThen":true,"trueExpr":{"type":"AstExprGlobal","location":"0,20 - 0,21","global":"y"},"hasElse":true,"falseExpr":{"type":"AstExprGlobal","location":"0,27 - 0,28","global":"z"}}]})"; + + CHECK(toJson(statement) == expected); +} + + TEST_CASE("encode_AstExprLocal") { AstLocal local{AstName{"foo"}, Location{}, nullptr, 0, 0, nullptr}; AstExprLocal exprLocal{Location{}, &local, false}; - CHECK(toJson(&exprLocal) == R"({"type":"AstExprLocal","location":"0,0 - 0,0","local":{"type":null,"name":"foo","location":"0,0 - 0,0"}})"); + CHECK(toJson(&exprLocal) == R"({"type":"AstExprLocal","location":"0,0 - 0,0","local":{"luauType":null,"name":"foo","type":"AstLocal","location":"0,0 - 0,0"}})"); } TEST_CASE("encode_AstExprVarargs") @@ -181,7 +225,7 @@ TEST_CASE_FIXTURE(JsonEncoderFixture, "encode_AstExprFunction") AstExpr* expr = expectParseExpr("function (a) return a end"); std::string_view expected = - R"({"type":"AstExprFunction","location":"0,4 - 0,29","generics":[],"genericPacks":[],"args":[{"type":null,"name":"a","location":"0,14 - 0,15"}],"vararg":false,"varargLocation":"0,0 - 0,0","body":{"type":"AstStatBlock","location":"0,16 - 0,26","body":[{"type":"AstStatReturn","location":"0,17 - 0,25","list":[{"type":"AstExprLocal","location":"0,24 - 0,25","local":{"type":null,"name":"a","location":"0,14 - 0,15"}}]}]},"functionDepth":1,"debugname":"","hasEnd":true})"; + R"({"type":"AstExprFunction","location":"0,4 - 0,29","generics":[],"genericPacks":[],"args":[{"luauType":null,"name":"a","type":"AstLocal","location":"0,14 - 0,15"}],"vararg":false,"varargLocation":"0,0 - 0,0","body":{"type":"AstStatBlock","location":"0,16 - 0,26","body":[{"type":"AstStatReturn","location":"0,17 - 0,25","list":[{"type":"AstExprLocal","location":"0,24 - 0,25","local":{"luauType":null,"name":"a","type":"AstLocal","location":"0,14 - 0,15"}}]}]},"functionDepth":1,"debugname":"","hasEnd":true})"; CHECK(toJson(expr) == expected); } @@ -191,7 +235,7 @@ TEST_CASE_FIXTURE(JsonEncoderFixture, "encode_AstExprTable") AstExpr* expr = expectParseExpr("{true, key=true, [key2]=true}"); std::string_view expected = - R"({"type":"AstExprTable","location":"0,4 - 0,33","items":[{"kind":"item","value":{"type":"AstExprConstantBool","location":"0,5 - 0,9","value":true}},{"kind":"record","key":{"type":"AstExprConstantString","location":"0,11 - 0,14","value":"key"},"value":{"type":"AstExprConstantBool","location":"0,15 - 0,19","value":true}},{"kind":"general","key":{"type":"AstExprGlobal","location":"0,22 - 0,26","global":"key2"},"value":{"type":"AstExprConstantBool","location":"0,28 - 0,32","value":true}}]})"; + R"({"type":"AstExprTable","location":"0,4 - 0,33","items":[{"type":"AstExprTableItem","kind":"item","value":{"type":"AstExprConstantBool","location":"0,5 - 0,9","value":true}},{"type":"AstExprTableItem","kind":"record","key":{"type":"AstExprConstantString","location":"0,11 - 0,14","value":"key"},"value":{"type":"AstExprConstantBool","location":"0,15 - 0,19","value":true}},{"type":"AstExprTableItem","kind":"general","key":{"type":"AstExprGlobal","location":"0,22 - 0,26","global":"key2"},"value":{"type":"AstExprConstantBool","location":"0,28 - 0,32","value":true}}]})"; CHECK(toJson(expr) == expected); } @@ -201,7 +245,7 @@ TEST_CASE_FIXTURE(JsonEncoderFixture, "encode_AstExprUnary") AstExpr* expr = expectParseExpr("-b"); std::string_view expected = - R"({"type":"AstExprUnary","location":"0,4 - 0,6","op":"minus","expr":{"type":"AstExprGlobal","location":"0,5 - 0,6","global":"b"}})"; + R"({"type":"AstExprUnary","location":"0,4 - 0,6","op":"Minus","expr":{"type":"AstExprGlobal","location":"0,5 - 0,6","global":"b"}})"; CHECK(toJson(expr) == expected); } @@ -259,7 +303,7 @@ TEST_CASE_FIXTURE(JsonEncoderFixture, "encode_AstStatWhile") AstStat* statement = expectParseStatement("while true do end"); std::string_view expected = - R"({"type":"AtStatWhile","location":"0,0 - 0,17","condition":{"type":"AstExprConstantBool","location":"0,6 - 0,10","value":true},"body":{"type":"AstStatBlock","location":"0,13 - 0,14","body":[]},"hasDo":true,"hasEnd":true})"; + R"({"type":"AstStatWhile","location":"0,0 - 0,17","condition":{"type":"AstExprConstantBool","location":"0,6 - 0,10","value":true},"body":{"type":"AstStatBlock","location":"0,13 - 0,14","body":[]},"hasDo":true,"hasEnd":true})"; CHECK(toJson(statement) == expected); } @@ -279,7 +323,7 @@ TEST_CASE_FIXTURE(JsonEncoderFixture, "encode_AstStatBreak") AstStat* statement = expectParseStatement("while true do break end"); std::string_view expected = - R"({"type":"AtStatWhile","location":"0,0 - 0,23","condition":{"type":"AstExprConstantBool","location":"0,6 - 0,10","value":true},"body":{"type":"AstStatBlock","location":"0,13 - 0,20","body":[{"type":"AstStatBreak","location":"0,14 - 0,19"}]},"hasDo":true,"hasEnd":true})"; + R"({"type":"AstStatWhile","location":"0,0 - 0,23","condition":{"type":"AstExprConstantBool","location":"0,6 - 0,10","value":true},"body":{"type":"AstStatBlock","location":"0,13 - 0,20","body":[{"type":"AstStatBreak","location":"0,14 - 0,19"}]},"hasDo":true,"hasEnd":true})"; CHECK(toJson(statement) == expected); } @@ -289,7 +333,7 @@ TEST_CASE_FIXTURE(JsonEncoderFixture, "encode_AstStatContinue") AstStat* statement = expectParseStatement("while true do continue end"); std::string_view expected = - R"({"type":"AtStatWhile","location":"0,0 - 0,26","condition":{"type":"AstExprConstantBool","location":"0,6 - 0,10","value":true},"body":{"type":"AstStatBlock","location":"0,13 - 0,23","body":[{"type":"AstStatContinue","location":"0,14 - 0,22"}]},"hasDo":true,"hasEnd":true})"; + R"({"type":"AstStatWhile","location":"0,0 - 0,26","condition":{"type":"AstExprConstantBool","location":"0,6 - 0,10","value":true},"body":{"type":"AstStatBlock","location":"0,13 - 0,23","body":[{"type":"AstStatContinue","location":"0,14 - 0,22"}]},"hasDo":true,"hasEnd":true})"; CHECK(toJson(statement) == expected); } @@ -299,7 +343,7 @@ TEST_CASE_FIXTURE(JsonEncoderFixture, "encode_AstStatFor") AstStat* statement = expectParseStatement("for a=0,1 do end"); std::string_view expected = - R"({"type":"AstStatFor","location":"0,0 - 0,16","var":{"type":null,"name":"a","location":"0,4 - 0,5"},"from":{"type":"AstExprConstantNumber","location":"0,6 - 0,7","value":0},"to":{"type":"AstExprConstantNumber","location":"0,8 - 0,9","value":1},"body":{"type":"AstStatBlock","location":"0,12 - 0,13","body":[]},"hasDo":true,"hasEnd":true})"; + R"({"type":"AstStatFor","location":"0,0 - 0,16","var":{"luauType":null,"name":"a","type":"AstLocal","location":"0,4 - 0,5"},"from":{"type":"AstExprConstantNumber","location":"0,6 - 0,7","value":0},"to":{"type":"AstExprConstantNumber","location":"0,8 - 0,9","value":1},"body":{"type":"AstStatBlock","location":"0,12 - 0,13","body":[]},"hasDo":true,"hasEnd":true})"; CHECK(toJson(statement) == expected); } @@ -309,7 +353,7 @@ TEST_CASE_FIXTURE(JsonEncoderFixture, "encode_AstStatForIn") AstStat* statement = expectParseStatement("for a in b do end"); std::string_view expected = - R"({"type":"AstStatForIn","location":"0,0 - 0,17","vars":[{"type":null,"name":"a","location":"0,4 - 0,5"}],"values":[{"type":"AstExprGlobal","location":"0,9 - 0,10","global":"b"}],"body":{"type":"AstStatBlock","location":"0,13 - 0,14","body":[]},"hasIn":true,"hasDo":true,"hasEnd":true})"; + R"({"type":"AstStatForIn","location":"0,0 - 0,17","vars":[{"luauType":null,"name":"a","type":"AstLocal","location":"0,4 - 0,5"}],"values":[{"type":"AstExprGlobal","location":"0,9 - 0,10","global":"b"}],"body":{"type":"AstStatBlock","location":"0,13 - 0,14","body":[]},"hasIn":true,"hasDo":true,"hasEnd":true})"; CHECK(toJson(statement) == expected); } @@ -329,7 +373,7 @@ TEST_CASE_FIXTURE(JsonEncoderFixture, "encode_AstStatLocalFunction") AstStat* statement = expectParseStatement("local function a(b) return end"); std::string_view expected = - R"({"type":"AstStatLocalFunction","location":"0,0 - 0,30","name":{"type":null,"name":"a","location":"0,15 - 0,16"},"func":{"type":"AstExprFunction","location":"0,0 - 0,30","generics":[],"genericPacks":[],"args":[{"type":null,"name":"b","location":"0,17 - 0,18"}],"vararg":false,"varargLocation":"0,0 - 0,0","body":{"type":"AstStatBlock","location":"0,19 - 0,27","body":[{"type":"AstStatReturn","location":"0,20 - 0,26","list":[]}]},"functionDepth":1,"debugname":"a","hasEnd":true}})"; + R"({"type":"AstStatLocalFunction","location":"0,0 - 0,30","name":{"luauType":null,"name":"a","type":"AstLocal","location":"0,15 - 0,16"},"func":{"type":"AstExprFunction","location":"0,0 - 0,30","generics":[],"genericPacks":[],"args":[{"luauType":null,"name":"b","type":"AstLocal","location":"0,17 - 0,18"}],"vararg":false,"varargLocation":"0,0 - 0,0","body":{"type":"AstStatBlock","location":"0,19 - 0,27","body":[{"type":"AstStatReturn","location":"0,20 - 0,26","list":[]}]},"functionDepth":1,"debugname":"a","hasEnd":true}})"; CHECK(toJson(statement) == expected); } @@ -349,7 +393,7 @@ TEST_CASE_FIXTURE(JsonEncoderFixture, "encode_AstStatDeclareFunction") AstStat* statement = expectParseStatement("declare function foo(x: number): string"); std::string_view expected = - R"({"type":"AstStatDeclareFunction","location":"0,0 - 0,39","name":"foo","params":{"types":[{"type":"AstTypeReference","location":"0,24 - 0,30","name":"number","parameters":[]}]},"retTypes":{"types":[{"type":"AstTypeReference","location":"0,33 - 0,39","name":"string","parameters":[]}]},"generics":[],"genericPacks":[]})"; + R"({"type":"AstStatDeclareFunction","location":"0,0 - 0,39","name":"foo","params":{"type":"AstTypeList","types":[{"type":"AstTypeReference","location":"0,24 - 0,30","name":"number","parameters":[]}]},"retTypes":{"type":"AstTypeList","types":[{"type":"AstTypeReference","location":"0,33 - 0,39","name":"string","parameters":[]}]},"generics":[],"genericPacks":[]})"; CHECK(toJson(statement) == expected); } @@ -370,11 +414,11 @@ TEST_CASE_FIXTURE(JsonEncoderFixture, "encode_AstStatDeclareClass") REQUIRE(2 == root->body.size); std::string_view expected1 = - R"({"type":"AstStatDeclareClass","location":"1,22 - 4,11","name":"Foo","props":[{"name":"prop","type":{"type":"AstTypeReference","location":"2,18 - 2,24","name":"number","parameters":[]}},{"name":"method","type":{"type":"AstTypeFunction","location":"3,21 - 4,11","generics":[],"genericPacks":[],"argTypes":{"types":[{"type":"AstTypeReference","location":"3,39 - 3,45","name":"number","parameters":[]}]},"returnTypes":{"types":[{"type":"AstTypeReference","location":"3,48 - 3,54","name":"string","parameters":[]}]}}}]})"; + R"({"type":"AstStatDeclareClass","location":"1,22 - 4,11","name":"Foo","props":[{"name":"prop","type":"AstDeclaredClassProp","luauType":{"type":"AstTypeReference","location":"2,18 - 2,24","name":"number","parameters":[]}},{"name":"method","type":"AstDeclaredClassProp","luauType":{"type":"AstTypeFunction","location":"3,21 - 4,11","generics":[],"genericPacks":[],"argTypes":{"type":"AstTypeList","types":[{"type":"AstTypeReference","location":"3,39 - 3,45","name":"number","parameters":[]}]},"returnTypes":{"type":"AstTypeList","types":[{"type":"AstTypeReference","location":"3,48 - 3,54","name":"string","parameters":[]}]}}}]})"; CHECK(toJson(root->body.data[0]) == expected1); std::string_view expected2 = - R"({"type":"AstStatDeclareClass","location":"6,22 - 8,11","name":"Bar","superName":"Foo","props":[{"name":"prop2","type":{"type":"AstTypeReference","location":"7,19 - 7,25","name":"string","parameters":[]}}]})"; + R"({"type":"AstStatDeclareClass","location":"6,22 - 8,11","name":"Bar","superName":"Foo","props":[{"name":"prop2","type":"AstDeclaredClassProp","luauType":{"type":"AstTypeReference","location":"7,19 - 7,25","name":"string","parameters":[]}}]})"; CHECK(toJson(root->body.data[1]) == expected2); } @@ -383,7 +427,7 @@ TEST_CASE_FIXTURE(JsonEncoderFixture, "encode_annotation") AstStat* statement = expectParseStatement("type T = ((number) -> (string | nil)) & ((string) -> ())"); std::string_view expected = - R"({"type":"AstStatTypeAlias","location":"0,0 - 0,55","name":"T","generics":[],"genericPacks":[],"type":{"type":"AstTypeIntersection","location":"0,9 - 0,55","types":[{"type":"AstTypeFunction","location":"0,10 - 0,35","generics":[],"genericPacks":[],"argTypes":{"types":[{"type":"AstTypeReference","location":"0,11 - 0,17","name":"number","parameters":[]}]},"returnTypes":{"types":[{"type":"AstTypeUnion","location":"0,23 - 0,35","types":[{"type":"AstTypeReference","location":"0,23 - 0,29","name":"string","parameters":[]},{"type":"AstTypeReference","location":"0,32 - 0,35","name":"nil","parameters":[]}]}]}},{"type":"AstTypeFunction","location":"0,41 - 0,55","generics":[],"genericPacks":[],"argTypes":{"types":[{"type":"AstTypeReference","location":"0,42 - 0,48","name":"string","parameters":[]}]},"returnTypes":{"types":[]}}]},"exported":false})"; + R"({"type":"AstStatTypeAlias","location":"0,0 - 0,55","name":"T","generics":[],"genericPacks":[],"type":{"type":"AstTypeIntersection","location":"0,9 - 0,55","types":[{"type":"AstTypeFunction","location":"0,10 - 0,35","generics":[],"genericPacks":[],"argTypes":{"type":"AstTypeList","types":[{"type":"AstTypeReference","location":"0,11 - 0,17","name":"number","parameters":[]}]},"returnTypes":{"type":"AstTypeList","types":[{"type":"AstTypeUnion","location":"0,23 - 0,35","types":[{"type":"AstTypeReference","location":"0,23 - 0,29","name":"string","parameters":[]},{"type":"AstTypeReference","location":"0,32 - 0,35","name":"nil","parameters":[]}]}]}},{"type":"AstTypeFunction","location":"0,41 - 0,55","generics":[],"genericPacks":[],"argTypes":{"type":"AstTypeList","types":[{"type":"AstTypeReference","location":"0,42 - 0,48","name":"string","parameters":[]}]},"returnTypes":{"type":"AstTypeList","types":[]}}]},"exported":false})"; CHECK(toJson(statement) == expected); } @@ -411,7 +455,7 @@ TEST_CASE_FIXTURE(JsonEncoderFixture, "encode_AstTypePackExplicit") CHECK(2 == root->body.size); std::string_view expected = - R"({"type":"AstStatLocal","location":"2,8 - 2,36","vars":[{"type":{"type":"AstTypeReference","location":"2,17 - 2,36","name":"A","parameters":[{"type":"AstTypePackExplicit","location":"2,19 - 2,20","typeList":{"types":[{"type":"AstTypeReference","location":"2,20 - 2,26","name":"number","parameters":[]},{"type":"AstTypeReference","location":"2,28 - 2,34","name":"string","parameters":[]}]}}]},"name":"a","location":"2,14 - 2,15"}],"values":[]})"; + R"({"type":"AstStatLocal","location":"2,8 - 2,36","vars":[{"luauType":{"type":"AstTypeReference","location":"2,17 - 2,36","name":"A","parameters":[{"type":"AstTypePackExplicit","location":"2,19 - 2,20","typeList":{"type":"AstTypeList","types":[{"type":"AstTypeReference","location":"2,20 - 2,26","name":"number","parameters":[]},{"type":"AstTypeReference","location":"2,28 - 2,34","name":"string","parameters":[]}]}}]},"name":"a","type":"AstLocal","location":"2,14 - 2,15"}],"values":[]})"; CHECK(toJson(root->body.data[1]) == expected); } diff --git a/tests/NonstrictMode.test.cpp b/tests/NonstrictMode.test.cpp index 50dcbad0..02e02e6b 100644 --- a/tests/NonstrictMode.test.cpp +++ b/tests/NonstrictMode.test.cpp @@ -13,77 +13,6 @@ using namespace Luau; TEST_SUITE_BEGIN("NonstrictModeTests"); -TEST_CASE_FIXTURE(Fixture, "globals") -{ - CheckResult result = check(R"( - --!nonstrict - foo = true - foo = "now i'm a string!" - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - CHECK_EQ("any", toString(requireType("foo"))); -} - -TEST_CASE_FIXTURE(Fixture, "globals2") -{ - ScopedFastFlag sff[]{ - {"LuauReturnTypeInferenceInNonstrict", true}, - {"LuauLowerBoundsCalculation", true}, - }; - - CheckResult result = check(R"( - --!nonstrict - foo = function() return 1 end - foo = "now i'm a string!" - )"); - - LUAU_REQUIRE_ERROR_COUNT(1, result); - - TypeMismatch* tm = get(result.errors[0]); - REQUIRE(tm); - CHECK_EQ("() -> number", toString(tm->wantedType)); - CHECK_EQ("string", toString(tm->givenType)); - CHECK_EQ("() -> number", toString(requireType("foo"))); -} - -TEST_CASE_FIXTURE(Fixture, "globals_everywhere") -{ - CheckResult result = check(R"( - --!nonstrict - foo = 1 - - if true then - bar = 2 - end - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - - CHECK_EQ("any", toString(requireType("foo"))); - CHECK_EQ("any", toString(requireType("bar"))); -} - -TEST_CASE_FIXTURE(BuiltinsFixture, "function_returns_number_or_string") -{ - ScopedFastFlag sff[]{{"LuauReturnTypeInferenceInNonstrict", true}, {"LuauLowerBoundsCalculation", true}}; - - CheckResult result = check(R"( - --!nonstrict - local function f() - if math.random() > 0.5 then - return 5 - else - return "hi" - end - end - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - - CHECK("() -> number | string" == toString(requireType("f"))); -} - TEST_CASE_FIXTURE(Fixture, "infer_nullary_function") { CheckResult result = check(R"( @@ -106,13 +35,8 @@ TEST_CASE_FIXTURE(Fixture, "infer_nullary_function") REQUIRE_EQ(0, rets.size()); } -TEST_CASE_FIXTURE(Fixture, "first_return_type_dictates_number_of_return_types") +TEST_CASE_FIXTURE(Fixture, "infer_the_maximum_number_of_values_the_function_could_return") { - ScopedFastFlag sff[]{ - {"LuauReturnTypeInferenceInNonstrict", true}, - {"LuauLowerBoundsCalculation", true}, - }; - CheckResult result = check(R"( --!nonstrict function getMinCardCountForWidth(width) @@ -127,18 +51,22 @@ TEST_CASE_FIXTURE(Fixture, "first_return_type_dictates_number_of_return_types") TypeId t = requireType("getMinCardCountForWidth"); REQUIRE(t); - REQUIRE_EQ("(any) -> number", toString(t)); + REQUIRE_EQ("(any) -> (...any)", toString(t)); } +#if 0 +// Maybe we want this? TEST_CASE_FIXTURE(Fixture, "return_annotation_is_still_checked") { CheckResult result = check(R"( - --!nonstrict function foo(x): number return 'hello' end )"); LUAU_REQUIRE_ERROR_COUNT(1, result); + + REQUIRE_NE(*typeChecker.anyType, *requireType("foo")); } +#endif TEST_CASE_FIXTURE(Fixture, "function_parameters_are_any") { @@ -324,11 +252,6 @@ TEST_CASE_FIXTURE(Fixture, "delay_function_does_not_require_its_argument_to_retu TEST_CASE_FIXTURE(Fixture, "inconsistent_module_return_types_are_ok") { - ScopedFastFlag sff[]{ - {"LuauReturnTypeInferenceInNonstrict", true}, - {"LuauLowerBoundsCalculation", true}, - }; - CheckResult result = check(R"( --!nonstrict @@ -345,7 +268,7 @@ TEST_CASE_FIXTURE(Fixture, "inconsistent_module_return_types_are_ok") LUAU_REQUIRE_NO_ERRORS(result); - REQUIRE_EQ("((any) -> string) | {| foo: any |}", toString(getMainModule()->getModuleScope()->returnType)); + REQUIRE_EQ("any", toString(getMainModule()->getModuleScope()->returnType)); } TEST_CASE_FIXTURE(Fixture, "returning_insufficient_return_values") diff --git a/tests/Normalize.test.cpp b/tests/Normalize.test.cpp index 69e7f074..84a5a380 100644 --- a/tests/Normalize.test.cpp +++ b/tests/Normalize.test.cpp @@ -621,7 +621,6 @@ TEST_CASE_FIXTURE(Fixture, "normalize_module_return_type") { ScopedFastFlag sff[] = { {"LuauLowerBoundsCalculation", true}, - {"LuauReturnTypeInferenceInNonstrict", true}, }; check(R"( @@ -642,7 +641,7 @@ TEST_CASE_FIXTURE(Fixture, "normalize_module_return_type") end )"); - CHECK_EQ("(any, any) -> (any, any) -> any", toString(getMainModule()->getModuleScope()->returnType)); + CHECK_EQ("(any, any) -> (...any)", toString(getMainModule()->getModuleScope()->returnType)); } TEST_CASE_FIXTURE(Fixture, "return_type_is_not_a_constrained_intersection") diff --git a/tests/TypeInfer.functions.test.cpp b/tests/TypeInfer.functions.test.cpp index 6e6549d3..a634ba09 100644 --- a/tests/TypeInfer.functions.test.cpp +++ b/tests/TypeInfer.functions.test.cpp @@ -677,11 +677,6 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "toposort_doesnt_break_mutual_recursion") TEST_CASE_FIXTURE(Fixture, "check_function_before_lambda_that_uses_it") { - ScopedFastFlag sff[]{ - {"LuauReturnTypeInferenceInNonstrict", true}, - {"LuauLowerBoundsCalculation", true}, - }; - CheckResult result = check(R"( --!nonstrict @@ -690,7 +685,7 @@ TEST_CASE_FIXTURE(Fixture, "check_function_before_lambda_that_uses_it") end return function() - return f() + return f():andThen() end )"); @@ -817,18 +812,14 @@ TEST_CASE_FIXTURE(Fixture, "calling_function_with_incorrect_argument_type_yields TEST_CASE_FIXTURE(BuiltinsFixture, "calling_function_with_anytypepack_doesnt_leak_free_types") { - ScopedFastFlag sff[]{ - {"LuauReturnTypeInferenceInNonstrict", true}, - {"LuauLowerBoundsCalculation", true}, - }; - CheckResult result = check(R"( --!nonstrict - function Test(a): ...any + function Test(a) return 1, "" end + local tab = {} table.insert(tab, Test(1)); )"); @@ -1625,21 +1616,6 @@ TEST_CASE_FIXTURE(Fixture, "occurs_check_failure_in_function_return_type") CHECK(nullptr != get(result.errors[0])); } -TEST_CASE_FIXTURE(Fixture, "weird_fail_to_unify_type_pack") -{ - ScopedFastFlag sff[]{ - {"LuauReturnTypeInferenceInNonstrict", true}, - {"LuauLowerBoundsCalculation", true}, - }; - - CheckResult result = check(R"( - local function f() return end - local g = function() return f() end - )"); - - LUAU_REQUIRE_NO_ERRORS(result); -} - TEST_CASE_FIXTURE(Fixture, "quantify_constrained_types") { ScopedFastFlag sff[]{ diff --git a/tests/TypeInfer.loops.test.cpp b/tests/TypeInfer.loops.test.cpp index 56b807f8..354b3996 100644 --- a/tests/TypeInfer.loops.test.cpp +++ b/tests/TypeInfer.loops.test.cpp @@ -516,7 +516,7 @@ TEST_CASE_FIXTURE(Fixture, "loop_iter_trailing_nil") CHECK_EQ(*typeChecker.nilType, *requireType("extra")); } -TEST_CASE_FIXTURE(Fixture, "loop_iter_no_indexer") +TEST_CASE_FIXTURE(Fixture, "loop_iter_no_indexer_strict") { CheckResult result = check(R"( local t = {} @@ -531,6 +531,17 @@ TEST_CASE_FIXTURE(Fixture, "loop_iter_no_indexer") CHECK_EQ("Cannot iterate over a table without indexer", ge->message); } +TEST_CASE_FIXTURE(Fixture, "loop_iter_no_indexer_nonstrict") +{ + CheckResult result = check(Mode::Nonstrict, R"( + local t = {} + for k, v in t do + end + )"); + + LUAU_REQUIRE_ERROR_COUNT(0, result); +} + TEST_CASE_FIXTURE(BuiltinsFixture, "loop_iter_iter_metamethod") { CheckResult result = check(R"( diff --git a/tests/TypeInfer.provisional.test.cpp b/tests/TypeInfer.provisional.test.cpp index dc686890..34afd560 100644 --- a/tests/TypeInfer.provisional.test.cpp +++ b/tests/TypeInfer.provisional.test.cpp @@ -343,6 +343,20 @@ TEST_CASE_FIXTURE(Fixture, "specialization_binds_with_prototypes_too_early") LUAU_REQUIRE_ERRORS(result); // Should not have any errors. } +TEST_CASE_FIXTURE(Fixture, "weird_fail_to_unify_type_pack") +{ + ScopedFastFlag sff[] = { + {"LuauLowerBoundsCalculation", false}, + }; + + CheckResult result = check(R"( + local function f() return end + local g = function() return f() end + )"); + + LUAU_REQUIRE_ERRORS(result); // Should not have any errors. +} + TEST_CASE_FIXTURE(Fixture, "weird_fail_to_unify_variadic_pack") { ScopedFastFlag sff[] = { diff --git a/tests/TypeInfer.tables.test.cpp b/tests/TypeInfer.tables.test.cpp index 21ad4e1b..d9bfc89d 100644 --- a/tests/TypeInfer.tables.test.cpp +++ b/tests/TypeInfer.tables.test.cpp @@ -2990,6 +2990,15 @@ TEST_CASE_FIXTURE(Fixture, "expected_indexer_value_type_extra_2") LUAU_REQUIRE_NO_ERRORS(result); } +TEST_CASE_FIXTURE(Fixture, "expected_indexer_from_table_union") +{ + ScopedFastFlag luauExpectedTableUnionIndexerType{"LuauExpectedTableUnionIndexerType", true}; + + LUAU_REQUIRE_NO_ERRORS(check(R"(local a: {[string]: {number | string}} = {a = {2, 's'}})")); + LUAU_REQUIRE_NO_ERRORS(check(R"(local a: {[string]: {number | string}}? = {a = {2, 's'}})")); + LUAU_REQUIRE_NO_ERRORS(check(R"(local a: {[string]: {[string]: {string?}}?} = {["a"] = {["b"] = {"a", "b"}}})")); +} + TEST_CASE_FIXTURE(Fixture, "prop_access_on_key_whose_types_mismatches") { ScopedFastFlag sff{"LuauReportErrorsOnIndexerKeyMismatch", true}; diff --git a/tests/TypeInfer.test.cpp b/tests/TypeInfer.test.cpp index 7d1fb56b..858e8ac0 100644 --- a/tests/TypeInfer.test.cpp +++ b/tests/TypeInfer.test.cpp @@ -85,20 +85,19 @@ TEST_CASE_FIXTURE(Fixture, "infer_in_nocheck_mode") { ScopedFastFlag sff[]{ {"DebugLuauDeferredConstraintResolution", false}, - {"LuauReturnTypeInferenceInNonstrict", true}, {"LuauLowerBoundsCalculation", true}, }; CheckResult result = check(R"( --!nocheck function f(x) - return 5 + return x end -- we get type information even if there's type errors f(1, 2) )"); - CHECK_EQ("(any) -> number", toString(requireType("f"))); + CHECK_EQ("(any) -> (...any)", toString(requireType("f"))); LUAU_REQUIRE_NO_ERRORS(result); } @@ -355,6 +354,35 @@ TEST_CASE_FIXTURE(Fixture, "check_expr_recursion_limit") CHECK(nullptr != get(result.errors[0])); } +TEST_CASE_FIXTURE(Fixture, "globals") +{ + CheckResult result = check(R"( + --!nonstrict + foo = true + foo = "now i'm a string!" + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + CHECK_EQ("any", toString(requireType("foo"))); +} + +TEST_CASE_FIXTURE(Fixture, "globals2") +{ + CheckResult result = check(R"( + --!nonstrict + foo = function() return 1 end + foo = "now i'm a string!" + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + TypeMismatch* tm = get(result.errors[0]); + REQUIRE(tm); + CHECK_EQ("() -> (...any)", toString(tm->wantedType)); + CHECK_EQ("string", toString(tm->givenType)); + CHECK_EQ("() -> (...any)", toString(requireType("foo"))); +} + TEST_CASE_FIXTURE(Fixture, "globals_are_banned_in_strict_mode") { CheckResult result = check(R"( diff --git a/tests/conformance/nextvar.lua b/tests/conformance/tables.lua similarity index 96% rename from tests/conformance/nextvar.lua rename to tests/conformance/tables.lua index 93c4ddf7..0eff8540 100644 --- a/tests/conformance/nextvar.lua +++ b/tests/conformance/tables.lua @@ -592,4 +592,24 @@ do assert(countud() == 3) end +-- test __newindex-as-a-table indirection: this had memory safety bugs in Lua 5.1.0 +do + local hit = false + + local grandparent = {} + grandparent.__newindex = function(s,k,v) + assert(k == "foo" and v == 10) + hit = true + end + + local parent = {} + parent.__newindex = parent + setmetatable(parent, grandparent) + + local child = setmetatable({}, parent) + child.foo = 10 + + assert(hit and child.foo == nil and parent.foo == nil) +end + return"OK" diff --git a/tests/main.cpp b/tests/main.cpp index c5b844b5..e7c4aed6 100644 --- a/tests/main.cpp +++ b/tests/main.cpp @@ -63,7 +63,7 @@ static int testAssertionHandler(const char* expr, const char* file, int line, co if (debuggerPresent()) LUAU_DEBUGBREAK(); - ADD_FAIL_AT(file, line, "Assertion failed: ", expr); + ADD_FAIL_AT(file, line, "Assertion failed: ", std::string(expr)); return 1; } diff --git a/tools/faillist.txt b/tools/faillist.txt new file mode 100644 index 00000000..dc74a6a9 --- /dev/null +++ b/tools/faillist.txt @@ -0,0 +1,521 @@ +AnnotationTests.as_expr_does_not_propagate_type_info +AnnotationTests.as_expr_is_bidirectional +AnnotationTests.as_expr_warns_on_unrelated_cast +AnnotationTests.builtin_types_are_not_exported +AnnotationTests.cannot_use_nonexported_type +AnnotationTests.cloned_interface_maintains_pointers_between_definitions +AnnotationTests.corecursive_types_error_on_tight_loop +AnnotationTests.define_generic_type_alias +AnnotationTests.duplicate_type_param_name +AnnotationTests.for_loop_counter_annotation_is_checked +AnnotationTests.function_return_annotations_are_checked +AnnotationTests.generic_aliases_are_cloned_properly +AnnotationTests.instantiate_type_fun_should_not_trip_rbxassert +AnnotationTests.instantiation_clone_has_to_follow +AnnotationTests.interface_types_belong_to_interface_arena +AnnotationTests.luau_ice_triggers_an_ice +AnnotationTests.luau_ice_triggers_an_ice_exception_with_flag +AnnotationTests.luau_ice_triggers_an_ice_exception_with_flag_handler +AnnotationTests.luau_ice_triggers_an_ice_handler +AnnotationTests.luau_print_is_magic_if_the_flag_is_set +AnnotationTests.luau_print_is_not_special_without_the_flag +AnnotationTests.occurs_check_on_cyclic_intersection_typevar +AnnotationTests.occurs_check_on_cyclic_union_typevar +AnnotationTests.self_referential_type_alias +AnnotationTests.too_many_type_params +AnnotationTests.two_type_params +AnnotationTests.type_alias_always_resolve_to_a_real_type +AnnotationTests.type_alias_B_should_check_with_another_aliases_until_a_non_aliased_type +AnnotationTests.type_alias_should_alias_to_number +AnnotationTests.type_aliasing_to_number_should_not_check_given_a_string +AnnotationTests.type_annotations_inside_function_bodies +AnnotationTests.type_assertion_expr +AnnotationTests.typeof_variable_type_annotation_should_return_its_type +AnnotationTests.use_generic_type_alias +AnnotationTests.use_type_required_from_another_file +AstQuery.last_argument_function_call_type +AstQuery::getDocumentationSymbolAtPosition.binding +AstQuery::getDocumentationSymbolAtPosition.event_callback_arg +AstQuery::getDocumentationSymbolAtPosition.overloaded_fn +AstQuery::getDocumentationSymbolAtPosition.prop +AutocompleteTest.argument_types +AutocompleteTest.arguments_to_global_lambda +AutocompleteTest.as_types +AutocompleteTest.autocomplete_boolean_singleton +AutocompleteTest.autocomplete_default_type_pack_parameters +AutocompleteTest.autocomplete_default_type_parameters +AutocompleteTest.autocomplete_documentation_symbols +AutocompleteTest.autocomplete_end_with_fn_exprs +AutocompleteTest.autocomplete_end_with_lambda +AutocompleteTest.autocomplete_explicit_type_pack +AutocompleteTest.autocomplete_first_function_arg_expected_type +AutocompleteTest.autocomplete_for_in_middle_keywords +AutocompleteTest.autocomplete_for_middle_keywords +AutocompleteTest.autocomplete_if_else_regression +AutocompleteTest.autocomplete_if_middle_keywords +AutocompleteTest.autocomplete_ifelse_expressions +AutocompleteTest.autocomplete_on_string_singletons +AutocompleteTest.autocomplete_oop_implicit_self +AutocompleteTest.autocomplete_repeat_middle_keyword +AutocompleteTest.autocomplete_string_singleton_equality +AutocompleteTest.autocomplete_string_singleton_escape +AutocompleteTest.autocomplete_string_singletons +AutocompleteTest.autocomplete_until_expression +AutocompleteTest.autocomplete_until_in_repeat +AutocompleteTest.autocomplete_while_middle_keywords +AutocompleteTest.autocompleteProp_index_function_metamethod_is_variadic +AutocompleteTest.bias_toward_inner_scope +AutocompleteTest.comments +AutocompleteTest.cyclic_table +AutocompleteTest.do_not_overwrite_context_sensitive_kws +AutocompleteTest.do_not_suggest_internal_module_type +AutocompleteTest.do_not_suggest_synthetic_table_name +AutocompleteTest.dont_offer_any_suggestions_from_the_end_of_a_comment +AutocompleteTest.dont_offer_any_suggestions_from_within_a_broken_comment +AutocompleteTest.dont_offer_any_suggestions_from_within_a_broken_comment_at_the_very_end_of_the_file +AutocompleteTest.dont_offer_any_suggestions_from_within_a_comment +AutocompleteTest.dont_suggest_local_before_its_definition +AutocompleteTest.empty_program +AutocompleteTest.function_expr_params +AutocompleteTest.function_in_assignment_has_parentheses +AutocompleteTest.function_in_assignment_has_parentheses_2 +AutocompleteTest.function_parameters +AutocompleteTest.function_result_passed_to_function_has_parentheses +AutocompleteTest.function_type_types +AutocompleteTest.generic_types +AutocompleteTest.get_member_completions +AutocompleteTest.get_string_completions +AutocompleteTest.get_suggestions_for_new_statement +AutocompleteTest.get_suggestions_for_the_very_start_of_the_script +AutocompleteTest.global_function_params +AutocompleteTest.global_functions_are_not_scoped_lexically +AutocompleteTest.if_then_else_elseif_completions +AutocompleteTest.if_then_else_full_keywords +AutocompleteTest.keyword_members +AutocompleteTest.keyword_methods +AutocompleteTest.keyword_types +AutocompleteTest.leave_numbers_alone +AutocompleteTest.library_non_self_calls_are_fine +AutocompleteTest.library_self_calls_are_invalid +AutocompleteTest.local_function +AutocompleteTest.local_function_params +AutocompleteTest.local_functions_fall_out_of_scope +AutocompleteTest.local_initializer +AutocompleteTest.local_initializer_2 +AutocompleteTest.local_names +AutocompleteTest.local_types_builtin +AutocompleteTest.method_call_inside_function_body +AutocompleteTest.method_call_inside_if_conditional +AutocompleteTest.module_type_members +AutocompleteTest.modules_with_types +AutocompleteTest.nested_member_completions +AutocompleteTest.nested_recursive_function +AutocompleteTest.no_function_name_suggestions +AutocompleteTest.no_incompatible_self_calls +AutocompleteTest.no_incompatible_self_calls_2 +AutocompleteTest.no_incompatible_self_calls_on_class +AutocompleteTest.no_incompatible_self_calls_provisional +AutocompleteTest.not_the_var_we_are_defining +AutocompleteTest.optional_members +AutocompleteTest.private_types +AutocompleteTest.recommend_statement_starting_keywords +AutocompleteTest.recursive_function +AutocompleteTest.recursive_function_global +AutocompleteTest.recursive_function_local +AutocompleteTest.return_types +AutocompleteTest.skip_current_local +AutocompleteTest.sometimes_the_metatable_is_an_error +AutocompleteTest.source_module_preservation_and_invalidation +AutocompleteTest.statement_between_two_statements +AutocompleteTest.stop_at_first_stat_when_recommending_keywords +AutocompleteTest.string_prim_non_self_calls_are_avoided +AutocompleteTest.string_prim_self_calls_are_fine +AutocompleteTest.suggest_external_module_type +AutocompleteTest.suggest_table_keys +AutocompleteTest.table_intersection +AutocompleteTest.table_union +AutocompleteTest.type_correct_argument_type_suggestion +AutocompleteTest.type_correct_expected_argument_type_pack_suggestion +AutocompleteTest.type_correct_expected_argument_type_suggestion +AutocompleteTest.type_correct_expected_argument_type_suggestion_optional +AutocompleteTest.type_correct_expected_argument_type_suggestion_self +AutocompleteTest.type_correct_expected_return_type_pack_suggestion +AutocompleteTest.type_correct_expected_return_type_suggestion +AutocompleteTest.type_correct_full_type_suggestion +AutocompleteTest.type_correct_function_no_parenthesis +AutocompleteTest.type_correct_function_return_types +AutocompleteTest.type_correct_function_type_suggestion +AutocompleteTest.type_correct_keywords +AutocompleteTest.type_correct_local_type_suggestion +AutocompleteTest.type_correct_sealed_table +AutocompleteTest.type_correct_suggestion_for_overloads +AutocompleteTest.type_correct_suggestion_in_argument +AutocompleteTest.type_correct_suggestion_in_table +AutocompleteTest.type_scoping_easy +AutocompleteTest.unsealed_table +AutocompleteTest.unsealed_table_2 +AutocompleteTest.user_defined_globals +AutocompleteTest.user_defined_local_functions_in_own_definition +BuiltinDefinitionsTest.lib_documentation_symbols +BuiltinTests.aliased_string_format +BuiltinTests.assert_removes_falsy_types +BuiltinTests.assert_removes_falsy_types2 +BuiltinTests.assert_removes_falsy_types_even_from_type_pack_tail_but_only_for_the_first_type +BuiltinTests.assert_returns_false_and_string_iff_it_knows_the_first_argument_cannot_be_truthy +BuiltinTests.bad_select_should_not_crash +BuiltinTests.builtin_tables_sealed +BuiltinTests.coroutine_resume_anything_goes +BuiltinTests.coroutine_wrap_anything_goes +BuiltinTests.debug_info_is_crazy +BuiltinTests.debug_traceback_is_crazy +BuiltinTests.dont_add_definitions_to_persistent_types +BuiltinTests.find_capture_types +BuiltinTests.find_capture_types2 +BuiltinTests.find_capture_types3 +BuiltinTests.gcinfo +BuiltinTests.getfenv +BuiltinTests.global_singleton_types_are_sealed +BuiltinTests.gmatch_capture_types +BuiltinTests.gmatch_capture_types2 +BuiltinTests.gmatch_capture_types_balanced_escaped_parens +BuiltinTests.gmatch_capture_types_default_capture +BuiltinTests.gmatch_capture_types_invalid_pattern_fallback_to_builtin +BuiltinTests.gmatch_capture_types_invalid_pattern_fallback_to_builtin2 +BuiltinTests.gmatch_capture_types_leading_end_bracket_is_part_of_set +BuiltinTests.gmatch_capture_types_parens_in_sets_are_ignored +BuiltinTests.gmatch_capture_types_set_containing_lbracket +BuiltinTests.gmatch_definition +BuiltinTests.ipairs_iterator_should_infer_types_and_type_check +BuiltinTests.lua_51_exported_globals_all_exist +BuiltinTests.match_capture_types +BuiltinTests.match_capture_types2 +BuiltinTests.math_max_checks_for_numbers +BuiltinTests.math_max_variatic +BuiltinTests.math_things_are_defined +BuiltinTests.next_iterator_should_infer_types_and_type_check +BuiltinTests.no_persistent_typelevel_change +BuiltinTests.os_time_takes_optional_date_table +BuiltinTests.pairs_iterator_should_infer_types_and_type_check +BuiltinTests.see_thru_select +BuiltinTests.see_thru_select_count +BuiltinTests.select_on_variadic +BuiltinTests.select_slightly_out_of_range +BuiltinTests.select_way_out_of_range +BuiltinTests.select_with_decimal_argument_is_rounded_down +BuiltinTests.select_with_variadic_typepack_tail +BuiltinTests.select_with_variadic_typepack_tail_and_string_head +BuiltinTests.set_metatable_needs_arguments +BuiltinTests.setmetatable_should_not_mutate_persisted_types +BuiltinTests.setmetatable_unpacks_arg_types_correctly +BuiltinTests.sort +BuiltinTests.sort_with_bad_predicate +BuiltinTests.sort_with_predicate +BuiltinTests.string_format_arg_count_mismatch +BuiltinTests.string_format_arg_types_inference +BuiltinTests.string_format_as_method +BuiltinTests.string_format_correctly_ordered_types +BuiltinTests.string_format_report_all_type_errors_at_correct_positions +BuiltinTests.string_format_use_correct_argument +BuiltinTests.string_format_use_correct_argument2 +BuiltinTests.string_lib_self_noself +BuiltinTests.table_concat_returns_string +BuiltinTests.table_dot_remove_optionally_returns_generic +BuiltinTests.table_freeze_is_generic +BuiltinTests.table_insert_correctly_infers_type_of_array_2_args_overload +BuiltinTests.table_insert_correctly_infers_type_of_array_3_args_overload +BuiltinTests.table_pack +BuiltinTests.table_pack_reduce +BuiltinTests.table_pack_variadic +BuiltinTests.thread_is_a_type +BuiltinTests.tonumber_returns_optional_number_type +BuiltinTests.tonumber_returns_optional_number_type2 +BuiltinTests.xpcall +DefinitionTests.class_definition_function_prop +DefinitionTests.class_definitions_cannot_extend_non_class +DefinitionTests.class_definitions_cannot_overload_non_function +DefinitionTests.declaring_generic_functions +DefinitionTests.definition_file_class_function_args +DefinitionTests.definition_file_classes +DefinitionTests.definition_file_loading +DefinitionTests.definitions_documentation_symbols +DefinitionTests.documentation_symbols_dont_attach_to_persistent_types +DefinitionTests.load_definition_file_errors_do_not_pollute_global_scope +DefinitionTests.no_cyclic_defined_classes +DefinitionTests.single_class_type_identity_in_global_types +FrontendTest.accumulate_cached_errors +FrontendTest.accumulate_cached_errors_in_consistent_order +FrontendTest.any_annotation_breaks_cycle +FrontendTest.ast_node_at_position +FrontendTest.automatically_check_cyclically_dependent_scripts +FrontendTest.automatically_check_dependent_scripts +FrontendTest.check_without_builtin_next +FrontendTest.clearStats +FrontendTest.cycle_detection_between_check_and_nocheck +FrontendTest.cycle_detection_disabled_in_nocheck +FrontendTest.cycle_error_paths +FrontendTest.cycle_errors_can_be_fixed +FrontendTest.cycle_incremental_type_surface +FrontendTest.cycle_incremental_type_surface_longer +FrontendTest.discard_type_graphs +FrontendTest.dont_recheck_script_that_hasnt_been_marked_dirty +FrontendTest.dont_reparse_clean_file_when_linting +FrontendTest.environments +FrontendTest.find_a_require +FrontendTest.find_a_require_inside_a_function +FrontendTest.ignore_require_to_nonexistent_file +FrontendTest.imported_table_modification_2 +FrontendTest.it_should_be_safe_to_stringify_errors_when_full_type_graph_is_discarded +FrontendTest.no_use_after_free_with_type_fun_instantiation +FrontendTest.nocheck_cycle_used_by_checked +FrontendTest.nocheck_modules_are_typed +FrontendTest.produce_errors_for_unchanged_file_with_a_syntax_error +FrontendTest.produce_errors_for_unchanged_file_with_errors +FrontendTest.re_report_type_error_in_required_file +FrontendTest.real_source +FrontendTest.recheck_if_dependent_script_is_dirty +FrontendTest.report_require_to_nonexistent_file +FrontendTest.report_syntax_error_in_required_file +FrontendTest.reports_errors_from_multiple_sources +FrontendTest.stats_are_not_reset_between_checks +FrontendTest.test_lint_uses_correct_config +FrontendTest.test_pruneParentSegments +FrontendTest.trace_requires_in_nonstrict_mode +FrontendTest.typecheck_twice_for_ast_types +isSubtype.functions_and_any +isSubtype.intersection_of_functions_of_different_arities +isSubtype.intersection_of_tables +isSubtype.table_with_any_prop +isSubtype.table_with_table_prop +isSubtype.tables +Linter.BuiltinGlobalWrite +Linter.DeprecatedApi +Linter.LocalShadowGlobal +Linter.TableOperations +Linter.use_all_parent_scopes_for_globals +ModuleTests.any_persistance_does_not_leak +ModuleTests.builtin_types_point_into_globalTypes_arena +ModuleTests.clone_self_property +ModuleTests.deepClone_cyclic_table +NonstrictModeTests.delay_function_does_not_require_its_argument_to_return_anything +NonstrictModeTests.for_in_iterator_variables_are_any +NonstrictModeTests.function_parameters_are_any +NonstrictModeTests.inconsistent_module_return_types_are_ok +NonstrictModeTests.inconsistent_return_types_are_ok +NonstrictModeTests.infer_nullary_function +NonstrictModeTests.infer_the_maximum_number_of_values_the_function_could_return +NonstrictModeTests.inline_table_props_are_also_any +NonstrictModeTests.local_tables_are_not_any +NonstrictModeTests.locals_are_any_by_default +NonstrictModeTests.offer_a_hint_if_you_use_a_dot_instead_of_a_colon +NonstrictModeTests.parameters_having_type_any_are_optional +NonstrictModeTests.returning_insufficient_return_values +NonstrictModeTests.returning_too_many_values +NonstrictModeTests.table_dot_insert_and_recursive_calls +NonstrictModeTests.table_props_are_any +Normalize.any_wins_the_battle_over_unknown_in_unions +Normalize.constrained_intersection_of_intersections +Normalize.cyclic_intersection +Normalize.cyclic_table_is_marked_normal +Normalize.cyclic_table_is_not_marked_normal +Normalize.cyclic_table_normalizes_sensibly +Normalize.cyclic_union +Normalize.fuzz_failure_bound_type_is_normal_but_not_its_bounded_to +Normalize.fuzz_failure_instersection_combine_must_follow +Normalize.higher_order_function +Normalize.intersection_combine_on_bound_self +Normalize.intersection_inside_a_table_inside_another_intersection +Normalize.intersection_inside_a_table_inside_another_intersection_2 +Normalize.intersection_inside_a_table_inside_another_intersection_3 +Normalize.intersection_inside_a_table_inside_another_intersection_4 +Normalize.intersection_of_confluent_overlapping_tables +Normalize.intersection_of_disjoint_tables +Normalize.intersection_of_functions +Normalize.intersection_of_overlapping_tables +Normalize.intersection_of_tables_with_indexers +Normalize.nested_table_normalization_with_non_table__no_ice +Normalize.normalization_does_not_convert_ever +Normalize.normalize_module_return_type +Normalize.normalize_unions_containing_never +Normalize.normalize_unions_containing_unknown +Normalize.return_type_is_not_a_constrained_intersection +Normalize.skip_force_normal_on_external_types +Normalize.union_of_distinct_free_types +Normalize.variadic_tail_is_marked_normal +Normalize.visiting_a_type_twice_is_not_considered_normal +ParseErrorRecovery.empty_function_type_error_recovery +ParseErrorRecovery.extra_table_indexer_recovery +ParseErrorRecovery.extra_token_in_consume +ParseErrorRecovery.extra_token_in_consume_match +ParseErrorRecovery.extra_token_in_consume_match_end +ParseErrorRecovery.generic_type_list_recovery +ParseErrorRecovery.multiple_parse_errors +ParseErrorRecovery.recovery_of_parenthesized_expressions +ParseErrorRecovery.statement_error_recovery_expected +ParseErrorRecovery.statement_error_recovery_unexpected +ParserTests.break_return_not_last_error +ParserTests.continue_not_last_error +ParserTests.error_on_confusable +ParserTests.error_on_non_utf8_sequence +ParserTests.error_on_unicode +ParserTests.export_is_an_identifier_only_when_followed_by_type +ParserTests.functions_cannot_have_return_annotations_if_extensions_are_disabled +ParserTests.illegal_type_alias_if_extensions_are_disabled +ParserTests.incomplete_statement_error +ParserTests.local_cannot_have_annotation_with_extensions_disabled +ParserTests.parse_compound_assignment_error_call +ParserTests.parse_compound_assignment_error_multiple +ParserTests.parse_compound_assignment_error_not_lvalue +ParserTests.parse_error_function_call +ParserTests.parse_error_function_call_newline +ParserTests.parse_error_messages +ParserTests.parse_error_table_literal +ParserTests.parse_error_type_name +ParserTests.parse_nesting_based_end_detection +ParserTests.parse_nesting_based_end_detection_failsafe_earlier +ParserTests.parse_nesting_based_end_detection_local_function +ParserTests.parse_nesting_based_end_detection_local_repeat +ParserTests.parse_nesting_based_end_detection_nested +ParserTests.parse_nesting_based_end_detection_single_line +ParserTests.parse_numbers_error +ParserTests.parse_numbers_range_error +ParserTests.stop_if_line_ends_with_hyphen +ParserTests.type_alias_error_messages +RuntimeLimits.typescript_port_of_Result_type +ToDot.bound_table +ToDot.class +ToDot.function +ToDot.metatable +ToDot.primitive +ToDot.table +ToString.exhaustive_toString_of_cyclic_table +ToString.function_type_with_argument_names +ToString.function_type_with_argument_names_and_self +ToString.function_type_with_argument_names_generic +ToString.named_metatable_toStringNamedFunction +ToString.no_parentheses_around_cyclic_function_type_in_union +ToString.toStringDetailed2 +ToString.toStringErrorPack +ToString.toStringNamedFunction_generic_pack +ToString.toStringNamedFunction_hide_type_params +ToString.toStringNamedFunction_id +ToString.toStringNamedFunction_map +ToString.toStringNamedFunction_overrides_param_names +ToString.toStringNamedFunction_variadics +TranspilerTests.attach_types +TranspilerTests.type_lists_should_be_emitted_correctly +TranspilerTests.types_should_not_be_considered_cyclic_if_they_are_not_recursive +TypeAliases.basic_alias +TypeAliases.cannot_steal_hoisted_type_alias +TypeAliases.cli_38393_recursive_intersection_oom +TypeAliases.corecursive_function_types +TypeAliases.corecursive_types_generic +TypeAliases.cyclic_function_type_in_type_alias +TypeAliases.cyclic_types_of_named_table_fields_do_not_expand_when_stringified +TypeAliases.do_not_quantify_unresolved_aliases +TypeAliases.dont_stop_typechecking_after_reporting_duplicate_type_definition +TypeAliases.export_type_and_type_alias_are_duplicates +TypeAliases.forward_declared_alias_is_not_clobbered_by_prior_unification_with_any +TypeAliases.forward_declared_alias_is_not_clobbered_by_prior_unification_with_any_2 +TypeAliases.free_variables_from_typeof_in_aliases +TypeAliases.general_require_multi_assign +TypeAliases.generic_param_remap +TypeAliases.generic_typevars_are_not_considered_to_escape_their_scope_if_they_are_reused_in_multiple_aliases +TypeAliases.module_export_free_type_leak +TypeAliases.module_export_wrapped_free_type_leak +TypeAliases.mutually_recursive_aliases +TypeAliases.mutually_recursive_generic_aliases +TypeAliases.mutually_recursive_types_errors +TypeAliases.mutually_recursive_types_restriction_not_ok_1 +TypeAliases.mutually_recursive_types_restriction_not_ok_2 +TypeAliases.mutually_recursive_types_restriction_ok +TypeAliases.mutually_recursive_types_swapsies_not_ok +TypeAliases.mutually_recursive_types_swapsies_ok +TypeAliases.names_are_ascribed +TypeAliases.non_recursive_aliases_that_reuse_a_generic_name +TypeAliases.recursive_types_restriction_not_ok +TypeAliases.recursive_types_restriction_ok +TypeAliases.reported_location_is_correct_when_type_alias_are_duplicates +TypeAliases.stringify_optional_parameterized_alias +TypeAliases.stringify_type_alias_of_recursive_template_table_type +TypeAliases.stringify_type_alias_of_recursive_template_table_type2 +TypeAliases.type_alias_fwd_declaration_is_precise +TypeAliases.type_alias_import_mutation +TypeAliases.type_alias_local_mutation +TypeAliases.type_alias_local_rename +TypeAliases.type_alias_local_synthetic_mutation +TypeAliases.type_alias_of_an_imported_recursive_generic_type +TypeAliases.type_alias_of_an_imported_recursive_type +TypeAliases.use_table_name_and_generic_params_in_errors +TypeInferAnyError.any_type_propagates +TypeInferAnyError.assign_prop_to_table_by_calling_any_yields_any +TypeInferAnyError.call_to_any_yields_any +TypeInferAnyError.calling_error_type_yields_error +TypeInferAnyError.can_get_length_of_any +TypeInferAnyError.can_subscript_any +TypeInferAnyError.CheckMethodsOfAny +TypeInferAnyError.for_in_loop_iterator_is_any +TypeInferAnyError.for_in_loop_iterator_is_any2 +TypeInferAnyError.for_in_loop_iterator_is_error +TypeInferAnyError.for_in_loop_iterator_is_error2 +TypeInferAnyError.for_in_loop_iterator_returns_any +TypeInferAnyError.for_in_loop_iterator_returns_any2 +TypeInferAnyError.indexing_error_type_does_not_produce_an_error +TypeInferAnyError.length_of_error_type_does_not_produce_an_error +TypeInferAnyError.metatable_of_any_can_be_a_table +TypeInferAnyError.prop_access_on_any_with_other_options +TypeInferAnyError.quantify_any_does_not_bind_to_itself +TypeInferAnyError.replace_every_free_type_when_unifying_a_complex_function_with_any +TypeInferAnyError.type_error_addition +TypeInferClasses.assign_to_prop_of_class +TypeInferClasses.call_base_method +TypeInferClasses.call_instance_method +TypeInferClasses.call_method_of_a_child_class +TypeInferClasses.call_method_of_a_class +TypeInferClasses.can_assign_to_prop_of_base_class +TypeInferClasses.can_assign_to_prop_of_base_class_using_string +TypeInferClasses.can_read_prop_of_base_class +TypeInferClasses.can_read_prop_of_base_class_using_string +TypeInferClasses.cannot_call_method_of_child_on_base_instance +TypeInferClasses.cannot_call_unknown_method_of_a_class +TypeInferClasses.cannot_unify_class_instance_with_primitive +TypeInferClasses.class_type_mismatch_with_name_conflict +TypeInferClasses.class_unification_type_mismatch_is_correct_order +TypeInferClasses.classes_can_have_overloaded_operators +TypeInferClasses.classes_without_overloaded_operators_cannot_be_added +TypeInferClasses.detailed_class_unification_error +TypeInferClasses.function_arguments_are_covariant +TypeInferClasses.higher_order_function_arguments_are_contravariant +TypeInferClasses.higher_order_function_return_type_is_not_contravariant +TypeInferClasses.higher_order_function_return_values_are_covariant +TypeInferClasses.optional_class_field_access_error +TypeInferClasses.table_class_unification_reports_sane_errors_for_missing_properties +TypeInferClasses.table_indexers_are_invariant +TypeInferClasses.table_properties_are_invariant +TypeInferClasses.warn_when_prop_almost_matches +TypeInferClasses.we_can_infer_that_a_parameter_must_be_a_particular_class +TypeInferClasses.we_can_report_when_someone_is_trying_to_use_a_table_rather_than_a_class +TypeInferFunctions.another_indirect_function_case_where_it_is_ok_to_provide_too_many_arguments +TypeInferFunctions.another_recursive_local_function +TypeInferFunctions.cannot_hoist_interior_defns_into_signature +TypeInferFunctions.check_function_before_lambda_that_uses_it +TypeInferFunctions.complicated_return_types_require_an_explicit_annotation +TypeInferFunctions.cyclic_function_type_in_args +TypeInferFunctions.dont_give_other_overloads_message_if_only_one_argument_matching_overload_exists +TypeInferFunctions.duplicate_functions_with_different_signatures_not_allowed_in_nonstrict +TypeInferFunctions.first_argument_can_be_optional +TypeInferFunctions.func_expr_doesnt_leak_free +TypeInferFunctions.higher_order_function_2 +TypeInferFunctions.higher_order_function_4 +TypeInferFunctions.infer_return_type_from_selected_overload +TypeInferFunctions.infer_that_function_does_not_return_a_table +TypeInferFunctions.it_is_ok_not_to_supply_enough_retvals +TypeInferFunctions.it_is_ok_to_oversaturate_a_higher_order_function_argument +TypeInferFunctions.list_all_overloads_if_no_overload_takes_given_argument_count +TypeInferFunctions.list_only_alternative_overloads_that_match_argument_count +TypeInferFunctions.mutual_recursion +TypeInferFunctions.recursive_function +TypeInferFunctions.recursive_local_function +TypeInferFunctions.too_many_arguments +TypeInferFunctions.toposort_doesnt_break_mutual_recursion +TypeInferFunctions.vararg_function_is_quantified +TypeInferFunctions.vararg_functions_should_allow_calls_of_any_types_and_size diff --git a/tools/natvis/Analysis.natvis b/tools/natvis/Analysis.natvis index b9ea3141..7d03dd3f 100644 --- a/tools/natvis/Analysis.natvis +++ b/tools/natvis/Analysis.natvis @@ -38,6 +38,38 @@ {{ typeId=29, value={*($T30*)storage} }} {{ typeId=30, value={*($T31*)storage} }} {{ typeId=31, value={*($T32*)storage} }} + {{ typeId=32, value={*($T33*)storage} }} + {{ typeId=33, value={*($T34*)storage} }} + {{ typeId=34, value={*($T35*)storage} }} + {{ typeId=35, value={*($T36*)storage} }} + {{ typeId=36, value={*($T37*)storage} }} + {{ typeId=37, value={*($T38*)storage} }} + {{ typeId=38, value={*($T39*)storage} }} + {{ typeId=39, value={*($T40*)storage} }} + {{ typeId=40, value={*($T41*)storage} }} + {{ typeId=41, value={*($T42*)storage} }} + {{ typeId=42, value={*($T43*)storage} }} + {{ typeId=43, value={*($T44*)storage} }} + {{ typeId=44, value={*($T45*)storage} }} + {{ typeId=45, value={*($T46*)storage} }} + {{ typeId=46, value={*($T47*)storage} }} + {{ typeId=47, value={*($T48*)storage} }} + {{ typeId=48, value={*($T49*)storage} }} + {{ typeId=49, value={*($T50*)storage} }} + {{ typeId=50, value={*($T51*)storage} }} + {{ typeId=51, value={*($T52*)storage} }} + {{ typeId=52, value={*($T53*)storage} }} + {{ typeId=53, value={*($T54*)storage} }} + {{ typeId=54, value={*($T55*)storage} }} + {{ typeId=55, value={*($T56*)storage} }} + {{ typeId=56, value={*($T57*)storage} }} + {{ typeId=57, value={*($T58*)storage} }} + {{ typeId=58, value={*($T59*)storage} }} + {{ typeId=59, value={*($T60*)storage} }} + {{ typeId=60, value={*($T61*)storage} }} + {{ typeId=61, value={*($T62*)storage} }} + {{ typeId=62, value={*($T63*)storage} }} + {{ typeId=63, value={*($T64*)storage} }} typeId *($T1*)storage @@ -72,6 +104,38 @@ *($T30*)storage *($T31*)storage *($T32*)storage + *($T33*)storage + *($T34*)storage + *($T35*)storage + *($T36*)storage + *($T37*)storage + *($T38*)storage + *($T39*)storage + *($T40*)storage + *($T41*)storage + *($T42*)storage + *($T43*)storage + *($T44*)storage + *($T45*)storage + *($T46*)storage + *($T47*)storage + *($T48*)storage + *($T49*)storage + *($T50*)storage + *($T51*)storage + *($T52*)storage + *($T53*)storage + *($T54*)storage + *($T55*)storage + *($T56*)storage + *($T57*)storage + *($T58*)storage + *($T59*)storage + *($T60*)storage + *($T61*)storage + *($T62*)storage + *($T63*)storage + *($T64*)storage diff --git a/tools/test_dcr.py b/tools/test_dcr.py new file mode 100644 index 00000000..8b090bcd --- /dev/null +++ b/tools/test_dcr.py @@ -0,0 +1,124 @@ +# This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details + +import argparse +import os.path +import subprocess as sp +import sys +import xml.sax as x + +SCRIPT_PATH = os.path.split(sys.argv[0])[0] +FAIL_LIST_PATH = os.path.join(SCRIPT_PATH, "faillist.txt") + + +def loadFailList(): + with open(FAIL_LIST_PATH) as f: + return set(map(str.strip, f.readlines())) + + +class Handler(x.ContentHandler): + def __init__(self, failList): + self.currentTest = [] + self.failList = failList # Set of dotted test names that are expected to fail + + self.results = {} # {DottedName: TrueIfTheTestPassed} + + def startElement(self, name, attrs): + if name == "TestSuite": + self.currentTest.append(attrs["name"]) + elif name == "TestCase": + self.currentTest.append(attrs["name"]) + + elif name == "OverallResultsAsserts": + if self.currentTest: + try: + failed = 0 != int(attrs["failures"]) + except ValueError: + failed = False + + dottedName = ".".join(self.currentTest) + shouldFail = dottedName in self.failList + + if failed and not shouldFail: + print("UNEXPECTED: {} should have passed".format(dottedName)) + elif not failed and shouldFail: + print("UNEXPECTED: {} should have failed".format(dottedName)) + + self.results[dottedName] = not failed + + def endElement(self, name): + if name == "TestCase": + self.currentTest.pop() + + elif name == "TestSuite": + self.currentTest.pop() + + +def main(): + parser = argparse.ArgumentParser( + description="Run Luau.UnitTest with deferred constraint resolution enabled" + ) + parser.add_argument( + "path", action="store", help="Path to the Luau.UnitTest executable" + ) + parser.add_argument( + "--dump", + dest="dump", + action="store_true", + help="Instead of doing any processing, dump the raw output of the test run. Useful for debugging this tool.", + ) + parser.add_argument( + "--write", + dest="write", + action="store_true", + help="Write a new faillist.txt after running tests.", + ) + + args = parser.parse_args() + + failList = loadFailList() + + p = sp.Popen( + [ + args.path, + "--reporters=xml", + "--fflags=true,DebugLuauDeferredConstraintResolution=true", + ], + stdout=sp.PIPE, + ) + + handler = Handler(failList) + + if args.dump: + for line in p.stdout: + sys.stdout.buffer.write(line) + return + else: + x.parse(p.stdout, handler) + + p.wait() + + if args.write: + newFailList = sorted( + ( + dottedName + for dottedName, passed in handler.results.items() + if not passed + ), + key=str.lower, + ) + with open(FAIL_LIST_PATH, "w", newline="\n") as f: + for name in newFailList: + print(name, file=f) + print("Updated faillist.txt") + + sys.exit( + 0 + if all( + not passed == (dottedName in failList) + for dottedName, passed in handler.results.items() + ) + else 1 + ) + +if __name__ == "__main__": + main() From b1cfaf53050594edb9c7bc3eb9d36333ac0bcd99 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Thu, 21 Jul 2022 14:16:54 -0700 Subject: [PATCH 318/399] Sync to upstream/release/537 (#607) --- Analysis/include/Luau/JsonEncoder.h | 3 + Analysis/include/Luau/VisitTypeVar.h | 43 +- Analysis/src/JsonEncoder.cpp | 127 ++++- Analysis/src/Module.cpp | 26 +- Analysis/src/TypeInfer.cpp | 24 +- CLI/Analyze.cpp | 9 +- CLI/Ast.cpp | 3 +- CLI/Flags.cpp | 75 +++ CLI/Flags.h | 5 + CLI/Repl.cpp | 91 +-- CodeGen/include/Luau/AssemblyBuilderX64.h | 10 + CodeGen/include/Luau/Condition.h | 2 - CodeGen/src/AssemblyBuilderX64.cpp | 39 +- Common/include/Luau/ExperimentalFlags.h | 26 + Compiler/src/Compiler.cpp | 37 +- Makefile | 13 +- Sources.cmake | 7 + VM/src/lapi.cpp | 6 +- VM/src/ldebug.cpp | 5 + VM/src/ldebug.h | 1 + VM/src/ltablib.cpp | 6 +- VM/src/lvmutils.cpp | 6 +- bench/tests/tictactoe.lua | 4 +- fuzz/proto.cpp | 2 +- tests/AssemblyBuilderX64.test.cpp | 24 + tests/Compiler.test.cpp | 95 +++- tests/Conformance.test.cpp | 4 +- tests/Frontend.test.cpp | 69 +++ tests/JsonEncoder.test.cpp | 80 ++- tests/NonstrictMode.test.cpp | 93 +--- tests/Normalize.test.cpp | 3 +- tests/TypeInfer.functions.test.cpp | 30 +- tests/TypeInfer.loops.test.cpp | 13 +- tests/TypeInfer.provisional.test.cpp | 14 + tests/TypeInfer.tables.test.cpp | 9 + tests/TypeInfer.test.cpp | 34 +- tests/conformance/{nextvar.lua => tables.lua} | 20 + tests/main.cpp | 2 +- tools/faillist.txt | 521 ++++++++++++++++++ tools/natvis/Analysis.natvis | 64 +++ tools/test_dcr.py | 124 +++++ 41 files changed, 1484 insertions(+), 285 deletions(-) create mode 100644 CLI/Flags.cpp create mode 100644 CLI/Flags.h create mode 100644 Common/include/Luau/ExperimentalFlags.h rename tests/conformance/{nextvar.lua => tables.lua} (96%) create mode 100644 tools/faillist.txt create mode 100644 tools/test_dcr.py diff --git a/Analysis/include/Luau/JsonEncoder.h b/Analysis/include/Luau/JsonEncoder.h index aa00390b..e79d9e62 100644 --- a/Analysis/include/Luau/JsonEncoder.h +++ b/Analysis/include/Luau/JsonEncoder.h @@ -2,12 +2,15 @@ #pragma once #include +#include namespace Luau { class AstNode; +struct Comment; std::string toJson(AstNode* node); +std::string toJson(AstNode* node, const std::vector& commentLocations); } // namespace Luau diff --git a/Analysis/include/Luau/VisitTypeVar.h b/Analysis/include/Luau/VisitTypeVar.h index ab4a397d..7229dafc 100644 --- a/Analysis/include/Luau/VisitTypeVar.h +++ b/Analysis/include/Luau/VisitTypeVar.h @@ -10,6 +10,7 @@ LUAU_FASTINT(LuauVisitRecursionLimit) LUAU_FASTFLAG(LuauNormalizeFlagIsConservative) +LUAU_FASTFLAG(LuauCompleteVisitor); namespace Luau { @@ -129,11 +130,11 @@ struct GenericTypeVarVisitor { return visit(ty); } - virtual bool visit(TypeId ty, const UnknownTypeVar& atv) + virtual bool visit(TypeId ty, const UnknownTypeVar& utv) { return visit(ty); } - virtual bool visit(TypeId ty, const NeverTypeVar& atv) + virtual bool visit(TypeId ty, const NeverTypeVar& ntv) { return visit(ty); } @@ -145,6 +146,14 @@ struct GenericTypeVarVisitor { return visit(ty); } + virtual bool visit(TypeId ty, const BlockedTypeVar& btv) + { + return visit(ty); + } + virtual bool visit(TypeId ty, const SingletonTypeVar& stv) + { + return visit(ty); + } virtual bool visit(TypePackId tp) { @@ -190,16 +199,12 @@ struct GenericTypeVarVisitor if (visit(ty, *btv)) traverse(btv->boundTo); } - else if (auto ftv = get(ty)) visit(ty, *ftv); - else if (auto gtv = get(ty)) visit(ty, *gtv); - else if (auto etv = get(ty)) visit(ty, *etv); - else if (auto ctv = get(ty)) { if (visit(ty, *ctv)) @@ -208,10 +213,8 @@ struct GenericTypeVarVisitor traverse(part); } } - else if (auto ptv = get(ty)) visit(ty, *ptv); - else if (auto ftv = get(ty)) { if (visit(ty, *ftv)) @@ -220,7 +223,6 @@ struct GenericTypeVarVisitor traverse(ftv->retTypes); } } - else if (auto ttv = get(ty)) { // Some visitors want to see bound tables, that's why we traverse the original type @@ -243,7 +245,6 @@ struct GenericTypeVarVisitor } } } - else if (auto mtv = get(ty)) { if (visit(ty, *mtv)) @@ -252,7 +253,6 @@ struct GenericTypeVarVisitor traverse(mtv->metatable); } } - else if (auto ctv = get(ty)) { if (visit(ty, *ctv)) @@ -267,10 +267,8 @@ struct GenericTypeVarVisitor traverse(*ctv->metatable); } } - else if (auto atv = get(ty)) visit(ty, *atv); - else if (auto utv = get(ty)) { if (visit(ty, *utv)) @@ -279,7 +277,6 @@ struct GenericTypeVarVisitor traverse(optTy); } } - else if (auto itv = get(ty)) { if (visit(ty, *itv)) @@ -288,6 +285,24 @@ struct GenericTypeVarVisitor traverse(partTy); } } + else if (!FFlag::LuauCompleteVisitor) + return visit_detail::unsee(seen, ty); + else if (get(ty)) + { + // Visiting into LazyTypeVar may necessarily cause infinite expansion, so we don't do that on purpose. + // Asserting also makes no sense, because the type _will_ happen here, most likely as a property of some ClassTypeVar + // that doesn't need to be expanded. + } + else if (auto stv = get(ty)) + visit(ty, *stv); + else if (auto btv = get(ty)) + visit(ty, *btv); + else if (auto utv = get(ty)) + visit(ty, *utv); + else if (auto ntv = get(ty)) + visit(ty, *ntv); + else + LUAU_ASSERT(!"GenericTypeVarVisitor::traverse(TypeId) is not exhaustive!"); visit_detail::unsee(seen, ty); } diff --git a/Analysis/src/JsonEncoder.cpp b/Analysis/src/JsonEncoder.cpp index 829ffa02..550c9b59 100644 --- a/Analysis/src/JsonEncoder.cpp +++ b/Analysis/src/JsonEncoder.cpp @@ -2,6 +2,7 @@ #include "Luau/JsonEncoder.h" #include "Luau/Ast.h" +#include "Luau/ParseResult.h" #include "Luau/StringUtils.h" #include "Luau/Common.h" @@ -75,6 +76,11 @@ struct AstJsonEncoder : public AstVisitor writeRaw(std::string_view{&c, 1}); } + void writeType(std::string_view propValue) + { + write("type", propValue); + } + template void write(std::string_view propName, const T& value) { @@ -98,7 +104,7 @@ struct AstJsonEncoder : public AstVisitor void write(double d) { char b[256]; - sprintf(b, "%g", d); + sprintf(b, "%.17g", d); writeRaw(b); } @@ -111,8 +117,12 @@ struct AstJsonEncoder : public AstVisitor { if (c == '"') writeRaw("\\\""); - else if (c == '\0') - writeRaw("\\\0"); + else if (c == '\\') + writeRaw("\\\\"); + else if (c < ' ') + writeRaw(format("\\u%04x", c)); + else if (c == '\n') + writeRaw("\\n"); else writeRaw(c); } @@ -189,10 +199,11 @@ struct AstJsonEncoder : public AstVisitor writeRaw("{"); bool c = pushComma(); if (local->annotation != nullptr) - write("type", local->annotation); + write("luauType", local->annotation); else - write("type", nullptr); + write("luauType", nullptr); write("name", local->name); + writeType("AstLocal"); write("location", local->location); popComma(c); writeRaw("}"); @@ -208,7 +219,7 @@ struct AstJsonEncoder : public AstVisitor { writeRaw("{"); bool c = pushComma(); - write("type", name); + writeType(name); writeNode(node); f(); popComma(c); @@ -358,6 +369,7 @@ struct AstJsonEncoder : public AstVisitor { writeRaw("{"); bool c = pushComma(); + writeType("AstTypeList"); write("types", typeList.types); if (typeList.tailType) write("tailType", typeList.tailType); @@ -369,9 +381,10 @@ struct AstJsonEncoder : public AstVisitor { writeRaw("{"); bool c = pushComma(); + writeType("AstGenericType"); write("name", genericType.name); if (genericType.defaultValue) - write("type", genericType.defaultValue); + write("luauType", genericType.defaultValue); popComma(c); writeRaw("}"); } @@ -380,9 +393,10 @@ struct AstJsonEncoder : public AstVisitor { writeRaw("{"); bool c = pushComma(); + writeType("AstGenericTypePack"); write("name", genericTypePack.name); if (genericTypePack.defaultValue) - write("type", genericTypePack.defaultValue); + write("luauType", genericTypePack.defaultValue); popComma(c); writeRaw("}"); } @@ -404,6 +418,7 @@ struct AstJsonEncoder : public AstVisitor { writeRaw("{"); bool c = pushComma(); + writeType("AstExprTableItem"); write("kind", item.kind); switch (item.kind) { @@ -419,6 +434,17 @@ struct AstJsonEncoder : public AstVisitor writeRaw("}"); } + void write(class AstExprIfElse* node) + { + writeNode(node, "AstExprIfElse", [&]() { + PROP(condition); + PROP(hasThen); + PROP(trueExpr); + PROP(hasElse); + PROP(falseExpr); + }); + } + void write(class AstExprTable* node) { writeNode(node, "AstExprTable", [&]() { @@ -431,11 +457,11 @@ struct AstJsonEncoder : public AstVisitor switch (op) { case AstExprUnary::Not: - return writeString("not"); + return writeString("Not"); case AstExprUnary::Minus: - return writeString("minus"); + return writeString("Minus"); case AstExprUnary::Len: - return writeString("len"); + return writeString("Len"); } } @@ -541,7 +567,7 @@ struct AstJsonEncoder : public AstVisitor void write(class AstStatWhile* node) { - writeNode(node, "AtStatWhile", [&]() { + writeNode(node, "AstStatWhile", [&]() { PROP(condition); PROP(body); PROP(hasDo); @@ -684,7 +710,8 @@ struct AstJsonEncoder : public AstVisitor writeRaw("{"); bool c = pushComma(); write("name", prop.name); - write("type", prop.ty); + writeType("AstDeclaredClassProp"); + write("luauType", prop.ty); popComma(c); writeRaw("}"); } @@ -731,8 +758,9 @@ struct AstJsonEncoder : public AstVisitor bool c = pushComma(); write("name", prop.name); + writeType("AstTableProp"); write("location", prop.location); - write("type", prop.type); + write("propType", prop.type); popComma(c); writeRaw("}"); @@ -745,6 +773,24 @@ struct AstJsonEncoder : public AstVisitor PROP(indexer); }); } + + void write(struct AstTableIndexer* indexer) + { + if (indexer) + { + writeRaw("{"); + bool c = pushComma(); + write("location", indexer->location); + write("indexType", indexer->indexType); + write("resultType", indexer->resultType); + popComma(c); + writeRaw("}"); + } + else + { + writeRaw("null"); + } + } void write(class AstTypeFunction* node) { @@ -836,6 +882,12 @@ struct AstJsonEncoder : public AstVisitor return false; } + bool visit(class AstExprIfElse* node) override + { + write(node); + return false; + } + bool visit(class AstExprLocal* node) override { write(node); @@ -1093,6 +1145,42 @@ struct AstJsonEncoder : public AstVisitor write(node); return false; } + + void writeComments(std::vector commentLocations) + { + bool commentComma = false; + for (Comment comment : commentLocations) + { + if (commentComma) + { + writeRaw(","); + } + else + { + commentComma = true; + } + writeRaw("{"); + bool c = pushComma(); + switch (comment.type) + { + case Lexeme::Comment: + writeType("Comment"); + break; + case Lexeme::BlockComment: + writeType("BlockComment"); + break; + case Lexeme::BrokenComment: + writeType("BrokenComment"); + break; + default: + break; + } + write("location", comment.location); + popComma(c); + writeRaw("}"); + + } + } }; std::string toJson(AstNode* node) @@ -1102,4 +1190,15 @@ std::string toJson(AstNode* node) return encoder.str(); } +std::string toJson(AstNode* node, const std::vector& commentLocations) +{ + AstJsonEncoder encoder; + encoder.writeRaw(R"({"root":)"); + node->visit(&encoder); + encoder.writeRaw(R"(,"commentLocations":[)"); + encoder.writeComments(commentLocations); + encoder.writeRaw("]}"); + return encoder.str(); +} + } // namespace Luau diff --git a/Analysis/src/Module.cpp b/Analysis/src/Module.cpp index 95eb125e..0603a042 100644 --- a/Analysis/src/Module.cpp +++ b/Analysis/src/Module.cpp @@ -17,6 +17,7 @@ LUAU_FASTFLAG(LuauLowerBoundsCalculation); LUAU_FASTFLAG(LuauNormalizeFlagIsConservative); LUAU_FASTFLAG(DebugLuauDeferredConstraintResolution); +LUAU_FASTFLAGVARIABLE(LuauForceExportSurfacesToBeNormal, false); namespace Luau { @@ -124,15 +125,21 @@ void Module::clonePublicInterface(InternalErrorReporter& ice) moduleScope2->returnType = returnType; // TODO varargPack } + ForceNormal forceNormal{&interfaceTypes}; + if (FFlag::LuauLowerBoundsCalculation) { normalize(returnType, interfaceTypes, ice); + if (FFlag::LuauForceExportSurfacesToBeNormal) + forceNormal.traverse(returnType); if (varargPack) + { normalize(*varargPack, interfaceTypes, ice); + if (FFlag::LuauForceExportSurfacesToBeNormal) + forceNormal.traverse(*varargPack); + } } - ForceNormal forceNormal{&interfaceTypes}; - if (exportedTypeBindings) { for (auto& [name, tf] : *exportedTypeBindings) @@ -147,6 +154,16 @@ void Module::clonePublicInterface(InternalErrorReporter& ice) // We're about to freeze the memory. We know that the flag is conservative by design. Cyclic tables // won't be marked normal. If the types aren't normal by now, they never will be. forceNormal.traverse(tf.type); + for (GenericTypeDefinition param : tf.typeParams) + { + forceNormal.traverse(param.ty); + + if (param.defaultValue) + { + normalize(*param.defaultValue, interfaceTypes, ice); + forceNormal.traverse(*param.defaultValue); + } + } } } } @@ -166,7 +183,12 @@ void Module::clonePublicInterface(InternalErrorReporter& ice) { ty = clone(ty, interfaceTypes, cloneState); if (FFlag::LuauLowerBoundsCalculation) + { normalize(ty, interfaceTypes, ice); + + if (FFlag::LuauForceExportSurfacesToBeNormal) + forceNormal.traverse(ty); + } } freeze(internalTypes); diff --git a/Analysis/src/TypeInfer.cpp b/Analysis/src/TypeInfer.cpp index 20777928..ea7d81e1 100644 --- a/Analysis/src/TypeInfer.cpp +++ b/Analysis/src/TypeInfer.cpp @@ -31,6 +31,7 @@ LUAU_FASTINTVARIABLE(LuauCheckRecursionLimit, 300) LUAU_FASTINTVARIABLE(LuauVisitRecursionLimit, 500) LUAU_FASTFLAG(LuauKnowsTheDataModel3) LUAU_FASTFLAG(LuauAutocompleteDynamicLimits) +LUAU_FASTFLAGVARIABLE(LuauExpectedTableUnionIndexerType, false) LUAU_FASTFLAGVARIABLE(LuauIndexSilenceErrors, false) LUAU_FASTFLAGVARIABLE(LuauLowerBoundsCalculation, false) LUAU_FASTFLAGVARIABLE(DebugLuauFreezeDuringUnification, false) @@ -38,7 +39,6 @@ LUAU_FASTFLAGVARIABLE(LuauSelfCallAutocompleteFix2, false) LUAU_FASTFLAGVARIABLE(LuauReduceUnionRecursion, false) LUAU_FASTFLAGVARIABLE(LuauReturnAnyInsteadOfICE, false) // Eventually removed as false. LUAU_FASTFLAG(LuauNormalizeFlagIsConservative) -LUAU_FASTFLAGVARIABLE(LuauReturnTypeInferenceInNonstrict, false) LUAU_FASTFLAGVARIABLE(DebugLuauSharedSelf, false); LUAU_FASTFLAGVARIABLE(LuauAlwaysQuantify, false); LUAU_FASTFLAGVARIABLE(LuauReportErrorsOnIndexerKeyMismatch, false) @@ -50,6 +50,7 @@ LUAU_FASTFLAGVARIABLE(LuauCheckGenericHOFTypes, false) LUAU_FASTFLAGVARIABLE(LuauBinaryNeedsExpectedTypesToo, false) LUAU_FASTFLAGVARIABLE(LuauNeverTypesAndOperatorsInference, false) LUAU_FASTFLAGVARIABLE(LuauReturnsFromCallsitesAreNotWidened, false) +LUAU_FASTFLAGVARIABLE(LuauCompleteVisitor, false) namespace Luau { @@ -890,7 +891,7 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatReturn& return_) TypePackId retPack = checkExprList(scope, return_.location, return_.list, false, {}, expectedTypes).type; - if (FFlag::LuauReturnTypeInferenceInNonstrict ? FFlag::LuauLowerBoundsCalculation : useConstrainedIntersections()) + if (useConstrainedIntersections()) { unifyLowerBound(retPack, scope->returnType, demoter.demotedLevel(scope->level), return_.location); return; @@ -1292,6 +1293,11 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatForIn& forin) for (size_t i = 2; i < varTypes.size(); ++i) unify(nilType, varTypes[i], forin.location); } + else if (isNonstrictMode()) + { + for (TypeId var : varTypes) + unify(anyType, var, forin.location); + } else { TypeId varTy = errorRecoveryType(loopScope); @@ -1385,12 +1391,7 @@ void TypeChecker::check(const ScopePtr& scope, TypeId ty, const ScopePtr& funSco // If in nonstrict mode and allowing redefinition of global function, restore the previous definition type // in case this function has a differing signature. The signature discrepancy will be caught in checkBlock. if (previouslyDefined) - { - if (FFlag::LuauReturnTypeInferenceInNonstrict && FFlag::LuauLowerBoundsCalculation) - quantify(funScope, ty, exprName->location); - globalBindings[name] = oldBinding; - } else globalBindings[name] = {quantify(funScope, ty, exprName->location), exprName->location}; @@ -2365,9 +2366,16 @@ WithPredicate TypeChecker::checkExpr(const ScopePtr& scope, const AstExp { std::vector expectedResultTypes; for (TypeId expectedOption : expectedUnion) + { if (const TableTypeVar* ttv = get(follow(expectedOption))) + { if (auto prop = ttv->props.find(key->value.data); prop != ttv->props.end()) expectedResultTypes.push_back(prop->second.type); + else if (FFlag::LuauExpectedTableUnionIndexerType && ttv->indexer && maybeString(ttv->indexer->indexType)) + expectedResultTypes.push_back(ttv->indexer->indexResultType); + } + } + if (expectedResultTypes.size() == 1) expectedResultType = expectedResultTypes[0]; else if (expectedResultTypes.size() > 1) @@ -3367,7 +3375,7 @@ std::pair TypeChecker::checkFunctionSignature(const ScopePtr& TypePackId retPack; if (expr.returnAnnotation) retPack = resolveTypePack(funScope, *expr.returnAnnotation); - else if (FFlag::LuauReturnTypeInferenceInNonstrict ? (!FFlag::LuauLowerBoundsCalculation && isNonstrictMode()) : isNonstrictMode()) + else if (isNonstrictMode()) retPack = anyTypePack; else if (expectedFunctionType && (!FFlag::LuauCheckGenericHOFTypes || (expectedFunctionType->generics.empty() && expectedFunctionType->genericPacks.empty()))) diff --git a/CLI/Analyze.cpp b/CLI/Analyze.cpp index f07f9a0b..cd50ef00 100644 --- a/CLI/Analyze.cpp +++ b/CLI/Analyze.cpp @@ -7,6 +7,7 @@ #include "Luau/Transpiler.h" #include "FileUtils.h" +#include "Flags.h" #ifdef CALLGRIND #include @@ -223,9 +224,7 @@ int main(int argc, char** argv) { Luau::assertHandler() = assertionHandler; - for (Luau::FValue* flag = Luau::FValue::list; flag; flag = flag->next) - if (strncmp(flag->name, "Luau", 4) == 0) - flag->value = true; + setLuauFlagsDefault(); if (argc >= 2 && strcmp(argv[1], "--help") == 0) { @@ -252,12 +251,14 @@ int main(int argc, char** argv) annotate = true; else if (strcmp(argv[i], "--timetrace") == 0) FFlag::DebugLuauTimeTracing.value = true; + else if (strncmp(argv[i], "--fflags=", 9) == 0) + setLuauFlags(argv[i] + 9); } #if !defined(LUAU_ENABLE_TIME_TRACE) if (FFlag::DebugLuauTimeTracing) { - printf("To run with --timetrace, Luau has to be built with LUAU_ENABLE_TIME_TRACE enabled\n"); + fprintf(stderr, "To run with --timetrace, Luau has to be built with LUAU_ENABLE_TIME_TRACE enabled\n"); return 1; } #endif diff --git a/CLI/Ast.cpp b/CLI/Ast.cpp index 4ea46236..6ee608c4 100644 --- a/CLI/Ast.cpp +++ b/CLI/Ast.cpp @@ -62,6 +62,7 @@ int main(int argc, char** argv) Luau::AstNameTable names(allocator); Luau::ParseOptions options; + options.captureComments = true; options.supportContinueStatement = true; options.allowTypeAnnotations = true; options.allowDeclarationSyntax = true; @@ -78,7 +79,7 @@ int main(int argc, char** argv) fprintf(stderr, "\n"); } - printf("%s", Luau::toJson(parseResult.root).c_str()); + printf("%s", Luau::toJson(parseResult.root, parseResult.commentLocations).c_str()); return parseResult.errors.size() > 0 ? 1 : 0; } diff --git a/CLI/Flags.cpp b/CLI/Flags.cpp new file mode 100644 index 00000000..4e261171 --- /dev/null +++ b/CLI/Flags.cpp @@ -0,0 +1,75 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +#include "Luau/Common.h" +#include "Luau/ExperimentalFlags.h" + +#include + +#include +#include + +static void setLuauFlag(std::string_view name, bool state) +{ + for (Luau::FValue* flag = Luau::FValue::list; flag; flag = flag->next) + { + if (name == flag->name) + { + flag->value = state; + return; + } + } + + fprintf(stderr, "Warning: unrecognized flag '%.*s'.\n", int(name.length()), name.data()); +} + +static void setLuauFlags(bool state) +{ + for (Luau::FValue* flag = Luau::FValue::list; flag; flag = flag->next) + if (strncmp(flag->name, "Luau", 4) == 0) + flag->value = state; +} + +void setLuauFlagsDefault() +{ + for (Luau::FValue* flag = Luau::FValue::list; flag; flag = flag->next) + if (strncmp(flag->name, "Luau", 4) == 0 && !Luau::isFlagExperimental(flag->name)) + flag->value = true; +} + +void setLuauFlags(const char* list) +{ + std::string_view rest = list; + + while (!rest.empty()) + { + size_t ending = rest.find(","); + std::string_view element = rest.substr(0, ending); + + if (size_t separator = element.find('='); separator != std::string_view::npos) + { + std::string_view key = element.substr(0, separator); + std::string_view value = element.substr(separator + 1); + + if (value == "true" || value == "True") + setLuauFlag(key, true); + else if (value == "false" || value == "False") + setLuauFlag(key, false); + else + fprintf(stderr, "Warning: unrecognized value '%.*s' for flag '%.*s'.\n", int(value.length()), value.data(), int(key.length()), + key.data()); + } + else + { + if (element == "true" || element == "True") + setLuauFlags(true); + else if (element == "false" || element == "False") + setLuauFlags(false); + else + setLuauFlag(element, true); + } + + if (ending != std::string_view::npos) + rest.remove_prefix(ending + 1); + else + break; + } +} diff --git a/CLI/Flags.h b/CLI/Flags.h new file mode 100644 index 00000000..8dfb0a29 --- /dev/null +++ b/CLI/Flags.h @@ -0,0 +1,5 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +#pragma once + +void setLuauFlagsDefault(); +void setLuauFlags(const char* list); diff --git a/CLI/Repl.cpp b/CLI/Repl.cpp index 5fe12bec..41360731 100644 --- a/CLI/Repl.cpp +++ b/CLI/Repl.cpp @@ -8,9 +8,10 @@ #include "Luau/BytecodeBuilder.h" #include "Luau/Parser.h" -#include "FileUtils.h" -#include "Profiler.h" #include "Coverage.h" +#include "FileUtils.h" +#include "Flags.h" +#include "Profiler.h" #include "isocline.h" @@ -97,7 +98,11 @@ static int lua_require(lua_State* L) // return the module from the cache lua_getfield(L, -1, name.c_str()); if (!lua_isnil(L, -1)) + { + // L stack: _MODULES result return finishrequire(L); + } + lua_pop(L, 1); std::optional source = readFile(name + ".luau"); @@ -109,6 +114,7 @@ static int lua_require(lua_State* L) } // module needs to run in a new thread, isolated from the rest + // note: we create ML on main thread so that it doesn't inherit environment of L lua_State* GL = lua_mainthread(L); lua_State* ML = lua_newthread(GL); lua_xmove(GL, L, 1); @@ -142,11 +148,12 @@ static int lua_require(lua_State* L) } } - // there's now a return value on top of ML; stack of L is MODULES thread + // there's now a return value on top of ML; L stack: _MODULES ML lua_xmove(ML, L, 1); lua_pushvalue(L, -1); lua_setfield(L, -4, name.c_str()); + // L stack: _MODULES ML result return finishrequire(L); } @@ -682,60 +689,11 @@ static int assertionHandler(const char* expr, const char* file, int line, const return 1; } -static void setLuauFlags(bool state) -{ - for (Luau::FValue* flag = Luau::FValue::list; flag; flag = flag->next) - { - if (strncmp(flag->name, "Luau", 4) == 0) - flag->value = state; - } -} - -static void setFlag(std::string_view name, bool state) -{ - for (Luau::FValue* flag = Luau::FValue::list; flag; flag = flag->next) - { - if (name == flag->name) - { - flag->value = state; - return; - } - } - - fprintf(stderr, "Warning: --fflag unrecognized flag '%.*s'.\n\n", int(name.length()), name.data()); -} - -static void applyFlagKeyValue(std::string_view element) -{ - if (size_t separator = element.find('='); separator != std::string_view::npos) - { - std::string_view key = element.substr(0, separator); - std::string_view value = element.substr(separator + 1); - - if (value == "true") - setFlag(key, true); - else if (value == "false") - setFlag(key, false); - else - fprintf(stderr, "Warning: --fflag unrecognized value '%.*s' for flag '%.*s'.\n\n", int(value.length()), value.data(), int(key.length()), - key.data()); - } - else - { - if (element == "true") - setLuauFlags(true); - else if (element == "false") - setLuauFlags(false); - else - setFlag(element, true); - } -} - int replMain(int argc, char** argv) { Luau::assertHandler() = assertionHandler; - setLuauFlags(true); + setLuauFlagsDefault(); CliMode mode = CliMode::Unknown; CompileFormat compileFormat{}; @@ -818,27 +776,10 @@ int replMain(int argc, char** argv) else if (strcmp(argv[i], "--timetrace") == 0) { FFlag::DebugLuauTimeTracing.value = true; - -#if !defined(LUAU_ENABLE_TIME_TRACE) - printf("To run with --timetrace, Luau has to be built with LUAU_ENABLE_TIME_TRACE enabled\n"); - return 1; -#endif } else if (strncmp(argv[i], "--fflags=", 9) == 0) { - std::string_view list = argv[i] + 9; - - while (!list.empty()) - { - size_t ending = list.find(","); - - applyFlagKeyValue(list.substr(0, ending)); - - if (ending != std::string_view::npos) - list.remove_prefix(ending + 1); - else - break; - } + setLuauFlags(argv[i] + 9); } else if (argv[i][0] == '-') { @@ -848,6 +789,14 @@ int replMain(int argc, char** argv) } } +#if !defined(LUAU_ENABLE_TIME_TRACE) + if (FFlag::DebugLuauTimeTracing) + { + fprintf(stderr, "To run with --timetrace, Luau has to be built with LUAU_ENABLE_TIME_TRACE enabled\n"); + return 1; + } +#endif + const std::vector files = getSourceFiles(argc, argv); if (mode == CliMode::Unknown) { diff --git a/CodeGen/include/Luau/AssemblyBuilderX64.h b/CodeGen/include/Luau/AssemblyBuilderX64.h index 65883b49..028b2d16 100644 --- a/CodeGen/include/Luau/AssemblyBuilderX64.h +++ b/CodeGen/include/Luau/AssemblyBuilderX64.h @@ -61,12 +61,22 @@ public: void call(Label& label); void call(OperandX64 op); + void int3(); + // AVX void vaddpd(OperandX64 dst, OperandX64 src1, OperandX64 src2); void vaddps(OperandX64 dst, OperandX64 src1, OperandX64 src2); void vaddsd(OperandX64 dst, OperandX64 src1, OperandX64 src2); void vaddss(OperandX64 dst, OperandX64 src1, OperandX64 src2); + void vsubsd(OperandX64 dst, OperandX64 src1, OperandX64 src2); + void vmulsd(OperandX64 dst, OperandX64 src1, OperandX64 src2); + void vdivsd(OperandX64 dst, OperandX64 src1, OperandX64 src2); + + void vxorpd(OperandX64 dst, OperandX64 src1, OperandX64 src2); + + void vcomisd(OperandX64 src1, OperandX64 src2); + void vsqrtpd(OperandX64 dst, OperandX64 src); void vsqrtps(OperandX64 dst, OperandX64 src); void vsqrtsd(OperandX64 dst, OperandX64 src1, OperandX64 src2); diff --git a/CodeGen/include/Luau/Condition.h b/CodeGen/include/Luau/Condition.h index 36cbda95..78e4515e 100644 --- a/CodeGen/include/Luau/Condition.h +++ b/CodeGen/include/Luau/Condition.h @@ -37,8 +37,6 @@ enum class Condition Zero, NotZero, - // TODO: ordered and unordered floating-point conditions - Count }; diff --git a/CodeGen/src/AssemblyBuilderX64.cpp b/CodeGen/src/AssemblyBuilderX64.cpp index 26347225..f88063cf 100644 --- a/CodeGen/src/AssemblyBuilderX64.cpp +++ b/CodeGen/src/AssemblyBuilderX64.cpp @@ -231,6 +231,7 @@ void AssemblyBuilderX64::lea(OperandX64 lhs, OperandX64 rhs) if (logText) log("lea", lhs, rhs); + LUAU_ASSERT(rhs.cat == CategoryX64::mem); placeBinaryRegAndRegMem(lhs, rhs, 0x8d, 0x8d); } @@ -314,6 +315,14 @@ void AssemblyBuilderX64::call(OperandX64 op) commit(); } +void AssemblyBuilderX64::int3() +{ + if (logText) + log("int3"); + + place(0xcc); +} + void AssemblyBuilderX64::vaddpd(OperandX64 dst, OperandX64 src1, OperandX64 src2) { placeAvx("vaddpd", dst, src1, src2, 0x58, false, AVX_0F, AVX_66); @@ -334,6 +343,31 @@ void AssemblyBuilderX64::vaddss(OperandX64 dst, OperandX64 src1, OperandX64 src2 placeAvx("vaddss", dst, src1, src2, 0x58, false, AVX_0F, AVX_F3); } +void AssemblyBuilderX64::vsubsd(OperandX64 dst, OperandX64 src1, OperandX64 src2) +{ + placeAvx("vsubsd", dst, src1, src2, 0x5c, false, AVX_0F, AVX_F2); +} + +void AssemblyBuilderX64::vmulsd(OperandX64 dst, OperandX64 src1, OperandX64 src2) +{ + placeAvx("vmulsd", dst, src1, src2, 0x59, false, AVX_0F, AVX_F2); +} + +void AssemblyBuilderX64::vdivsd(OperandX64 dst, OperandX64 src1, OperandX64 src2) +{ + placeAvx("vdivsd", dst, src1, src2, 0x5e, false, AVX_0F, AVX_F2); +} + +void AssemblyBuilderX64::vxorpd(OperandX64 dst, OperandX64 src1, OperandX64 src2) +{ + placeAvx("vxorpd", dst, src1, src2, 0x57, false, AVX_0F, AVX_66); +} + +void AssemblyBuilderX64::vcomisd(OperandX64 src1, OperandX64 src2) +{ + placeAvx("vcomisd", src1, src2, 0x2f, false, AVX_0F, AVX_66); +} + void AssemblyBuilderX64::vsqrtpd(OperandX64 dst, OperandX64 src) { placeAvx("vsqrtpd", dst, src, 0x51, false, AVX_0F, AVX_66); @@ -494,9 +528,10 @@ void AssemblyBuilderX64::placeBinaryRegMemAndImm(OperandX64 lhs, OperandX64 rhs, LUAU_ASSERT(lhs.cat == CategoryX64::reg || lhs.cat == CategoryX64::mem); LUAU_ASSERT(rhs.cat == CategoryX64::imm); - SizeX64 size = lhs.base.size; + SizeX64 size = lhs.cat == CategoryX64::reg ? lhs.base.size : lhs.memSize; + LUAU_ASSERT(size == SizeX64::byte || size == SizeX64::dword || size == SizeX64::qword); - placeRex(lhs.base); + placeRex(lhs); if (size == SizeX64::byte) { diff --git a/Common/include/Luau/ExperimentalFlags.h b/Common/include/Luau/ExperimentalFlags.h new file mode 100644 index 00000000..3525259d --- /dev/null +++ b/Common/include/Luau/ExperimentalFlags.h @@ -0,0 +1,26 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +#pragma once + +#include + +namespace Luau +{ + +inline bool isFlagExperimental(const char* flag) +{ + // Flags in this list are disabled by default in various command-line tools. They may have behavior that is not fully final, + // or critical bugs that are found after the code has been submitted. + static const char* kList[] = + { + "LuauLowerBoundsCalculation", + nullptr, // makes sure we always have at least one entry + }; + + for (const char* item: kList) + if (item && strcmp(item, flag) == 0) + return true; + + return false; +} + +} diff --git a/Compiler/src/Compiler.cpp b/Compiler/src/Compiler.cpp index 2b0f04f6..8f3befad 100644 --- a/Compiler/src/Compiler.cpp +++ b/Compiler/src/Compiler.cpp @@ -27,6 +27,7 @@ LUAU_FASTFLAGVARIABLE(LuauCompileNoIpairs, false) LUAU_FASTFLAGVARIABLE(LuauCompileFoldBuiltins, false) LUAU_FASTFLAGVARIABLE(LuauCompileBetterMultret, false) +LUAU_FASTFLAGVARIABLE(LuauCompileFreeReassign, false) namespace Luau { @@ -616,7 +617,7 @@ struct Compiler } else { - AstExprLocal* le = arg->as(); + AstExprLocal* le = FFlag::LuauCompileFreeReassign ? getExprLocal(arg) : arg->as(); Variable* lv = le ? variables.find(le->local) : nullptr; // if the argument is a local that isn't mutated, we will simply reuse the existing register @@ -2200,19 +2201,27 @@ struct Compiler compileLValueUse(lv, source, /* set= */ true); } - int getExprLocalReg(AstExpr* node) + AstExprLocal* getExprLocal(AstExpr* node) { if (AstExprLocal* expr = node->as()) + return expr; + else if (AstExprGroup* expr = node->as()) + return getExprLocal(expr->expr); + else if (AstExprTypeAssertion* expr = node->as()) + return getExprLocal(expr->expr); + else + return nullptr; + } + + int getExprLocalReg(AstExpr* node) + { + if (AstExprLocal* expr = getExprLocal(node)) { // note: this can't check expr->upvalue because upvalues may be upgraded to locals during inlining Local* l = locals.find(expr->local); return l && l->allocated ? l->reg : -1; } - else if (AstExprGroup* expr = node->as()) - return getExprLocalReg(expr->expr); - else if (AstExprTypeAssertion* expr = node->as()) - return getExprLocalReg(expr->expr); else return -1; } @@ -2498,6 +2507,22 @@ struct Compiler if (options.optimizationLevel >= 1 && options.debugLevel <= 1 && areLocalsRedundant(stat)) return; + // Optimization: for 1-1 local assignments, we can reuse the register *if* neither local is mutated + if (FFlag::LuauCompileFreeReassign && options.optimizationLevel >= 1 && stat->vars.size == 1 && stat->values.size == 1) + { + if (AstExprLocal* re = getExprLocal(stat->values.data[0])) + { + Variable* lv = variables.find(stat->vars.data[0]); + Variable* rv = variables.find(re->local); + + if (int reg = getExprLocalReg(re); reg >= 0 && (!lv || !lv->written) && (!rv || !rv->written)) + { + pushLocal(stat->vars.data[0], uint8_t(reg)); + return; + } + } + } + // note: allocReg in this case allocates into parent block register - note that we don't have RegScope here uint8_t vars = allocReg(stat, unsigned(stat->vars.size)); diff --git a/Makefile b/Makefile index a471e7ae..33a2e5e1 100644 --- a/Makefile +++ b/Makefile @@ -31,15 +31,15 @@ ISOCLINE_SOURCES=extern/isocline/src/isocline.c ISOCLINE_OBJECTS=$(ISOCLINE_SOURCES:%=$(BUILD)/%.o) ISOCLINE_TARGET=$(BUILD)/libisocline.a -TESTS_SOURCES=$(wildcard tests/*.cpp) CLI/FileUtils.cpp CLI/Profiler.cpp CLI/Coverage.cpp CLI/Repl.cpp +TESTS_SOURCES=$(wildcard tests/*.cpp) CLI/FileUtils.cpp CLI/Flags.cpp CLI/Profiler.cpp CLI/Coverage.cpp CLI/Repl.cpp TESTS_OBJECTS=$(TESTS_SOURCES:%=$(BUILD)/%.o) TESTS_TARGET=$(BUILD)/luau-tests -REPL_CLI_SOURCES=CLI/FileUtils.cpp CLI/Profiler.cpp CLI/Coverage.cpp CLI/Repl.cpp CLI/ReplEntry.cpp +REPL_CLI_SOURCES=CLI/FileUtils.cpp CLI/Flags.cpp CLI/Profiler.cpp CLI/Coverage.cpp CLI/Repl.cpp CLI/ReplEntry.cpp REPL_CLI_OBJECTS=$(REPL_CLI_SOURCES:%=$(BUILD)/%.o) REPL_CLI_TARGET=$(BUILD)/luau -ANALYZE_CLI_SOURCES=CLI/FileUtils.cpp CLI/Analyze.cpp +ANALYZE_CLI_SOURCES=CLI/FileUtils.cpp CLI/Flags.cpp CLI/Analyze.cpp ANALYZE_CLI_OBJECTS=$(ANALYZE_CLI_SOURCES:%=$(BUILD)/%.o) ANALYZE_CLI_TARGET=$(BUILD)/luau-analyze @@ -117,15 +117,18 @@ $(REPL_CLI_TARGET): LDFLAGS+=-lpthread fuzz-proto fuzz-prototest: LDFLAGS+=build/libprotobuf-mutator/src/libfuzzer/libprotobuf-mutator-libfuzzer.a build/libprotobuf-mutator/src/libprotobuf-mutator.a build/libprotobuf-mutator/external.protobuf/lib/libprotobuf.a # pseudo targets -.PHONY: all test clean coverage format luau-size +.PHONY: all test clean coverage format luau-size aliases -all: $(REPL_CLI_TARGET) $(ANALYZE_CLI_TARGET) $(TESTS_TARGET) +all: $(REPL_CLI_TARGET) $(ANALYZE_CLI_TARGET) $(TESTS_TARGET) aliases + +aliases: luau luau-analyze test: $(TESTS_TARGET) $(TESTS_TARGET) $(TESTS_ARGS) clean: rm -rf $(BUILD) + rm -rf luau luau-analyze coverage: $(TESTS_TARGET) $(TESTS_TARGET) --fflags=true diff --git a/Sources.cmake b/Sources.cmake index 69f5e1b7..a4563019 100644 --- a/Sources.cmake +++ b/Sources.cmake @@ -4,6 +4,7 @@ if(NOT ${CMAKE_VERSION} VERSION_LESS "3.19") target_sources(Luau.Common PRIVATE Common/include/Luau/Common.h Common/include/Luau/Bytecode.h + Common/include/Luau/ExperimentalFlags.h ) endif() @@ -220,6 +221,8 @@ if(TARGET Luau.Repl.CLI) CLI/Coverage.cpp CLI/FileUtils.h CLI/FileUtils.cpp + CLI/Flags.h + CLI/Flags.cpp CLI/Profiler.h CLI/Profiler.cpp CLI/Repl.cpp @@ -231,6 +234,8 @@ if(TARGET Luau.Analyze.CLI) target_sources(Luau.Analyze.CLI PRIVATE CLI/FileUtils.h CLI/FileUtils.cpp + CLI/Flags.h + CLI/Flags.cpp CLI/Analyze.cpp) endif() @@ -321,6 +326,8 @@ if(TARGET Luau.CLI.Test) CLI/Coverage.cpp CLI/FileUtils.h CLI/FileUtils.cpp + CLI/Flags.h + CLI/Flags.cpp CLI/Profiler.h CLI/Profiler.cpp CLI/Repl.cpp diff --git a/VM/src/lapi.cpp b/VM/src/lapi.cpp index 88680727..e3354ea2 100644 --- a/VM/src/lapi.cpp +++ b/VM/src/lapi.cpp @@ -854,7 +854,7 @@ void lua_rawset(lua_State* L, int idx) StkId t = index2addr(L, idx); api_check(L, ttistable(t)); if (hvalue(t)->readonly) - luaG_runerror(L, "Attempt to modify a readonly table"); + luaG_readonlyerror(L); setobj2t(L, luaH_set(L, hvalue(t), L->top - 2), L->top - 1); luaC_barriert(L, hvalue(t), L->top - 1); L->top -= 2; @@ -867,7 +867,7 @@ void lua_rawseti(lua_State* L, int idx, int n) StkId o = index2addr(L, idx); api_check(L, ttistable(o)); if (hvalue(o)->readonly) - luaG_runerror(L, "Attempt to modify a readonly table"); + luaG_readonlyerror(L); setobj2t(L, luaH_setnum(L, hvalue(o), n), L->top - 1); luaC_barriert(L, hvalue(o), L->top - 1); L->top--; @@ -890,7 +890,7 @@ int lua_setmetatable(lua_State* L, int objindex) case LUA_TTABLE: { if (hvalue(obj)->readonly) - luaG_runerror(L, "Attempt to modify a readonly table"); + luaG_readonlyerror(L); hvalue(obj)->metatable = mt; if (mt) luaC_objbarrier(L, hvalue(obj), mt); diff --git a/VM/src/ldebug.cpp b/VM/src/ldebug.cpp index ef486090..29015565 100644 --- a/VM/src/ldebug.cpp +++ b/VM/src/ldebug.cpp @@ -269,6 +269,11 @@ l_noret luaG_indexerror(lua_State* L, const TValue* p1, const TValue* p2) luaG_runerror(L, "attempt to index %s with %s", t1, t2); } +l_noret luaG_readonlyerror(lua_State* L) +{ + luaG_runerror(L, "attempt to modify a readonly table"); +} + static void pusherror(lua_State* L, const char* msg) { CallInfo* ci = L->ci; diff --git a/VM/src/ldebug.h b/VM/src/ldebug.h index 75bb8dcc..8e03db36 100644 --- a/VM/src/ldebug.h +++ b/VM/src/ldebug.h @@ -19,6 +19,7 @@ LUAI_FUNC l_noret luaG_concaterror(lua_State* L, StkId p1, StkId p2); LUAI_FUNC l_noret luaG_aritherror(lua_State* L, const TValue* p1, const TValue* p2, TMS op); LUAI_FUNC l_noret luaG_ordererror(lua_State* L, const TValue* p1, const TValue* p2, TMS op); LUAI_FUNC l_noret luaG_indexerror(lua_State* L, const TValue* p1, const TValue* p2); +LUAI_FUNC l_noret luaG_readonlyerror(lua_State* L); LUAI_FUNC LUA_PRINTF_ATTR(2, 3) l_noret luaG_runerrorL(lua_State* L, const char* fmt, ...); LUAI_FUNC void luaG_pusherror(lua_State* L, const char* error); diff --git a/VM/src/ltablib.cpp b/VM/src/ltablib.cpp index 27187c61..dc653338 100644 --- a/VM/src/ltablib.cpp +++ b/VM/src/ltablib.cpp @@ -79,7 +79,7 @@ static void moveelements(lua_State* L, int srct, int dstt, int f, int e, int t) Table* dst = hvalue(L->base + (dstt - 1)); if (dst->readonly) - luaG_runerror(L, "Attempt to modify a readonly table"); + luaG_readonlyerror(L); int n = e - f + 1; /* number of elements to move */ @@ -204,7 +204,7 @@ static int tmove(lua_State* L) Table* dst = hvalue(L->base + (tt - 1)); if (dst->readonly) /* also checked in moveelements, but this blocks resizes of r/o tables */ - luaG_runerror(L, "Attempt to modify a readonly table"); + luaG_readonlyerror(L); if (t > 0 && (t - 1) <= dst->sizearray && (t - 1 + n) > dst->sizearray) { /* grow the destination table array */ @@ -482,7 +482,7 @@ static int tclear(lua_State* L) Table* tt = hvalue(L->base); if (tt->readonly) - luaG_runerror(L, "Attempt to modify a readonly table"); + luaG_readonlyerror(L); luaH_clear(tt); return 0; diff --git a/VM/src/lvmutils.cpp b/VM/src/lvmutils.cpp index b9e762eb..be4e99a9 100644 --- a/VM/src/lvmutils.cpp +++ b/VM/src/lvmutils.cpp @@ -128,7 +128,7 @@ void luaV_gettable(lua_State* L, const TValue* t, TValue* key, StkId val) } t = tm; /* else repeat with `tm' */ } - luaG_runerror(L, "loop in gettable"); + luaG_runerror(L, "'__index' chain too long; possible loop"); } void luaV_settable(lua_State* L, const TValue* t, TValue* key, StkId val) @@ -143,7 +143,7 @@ void luaV_settable(lua_State* L, const TValue* t, TValue* key, StkId val) Table* h = hvalue(t); if (h->readonly) - luaG_runerror(L, "Attempt to modify a readonly table"); + luaG_readonlyerror(L); TValue* oldval = luaH_set(L, h, key); /* do a primitive set */ @@ -169,7 +169,7 @@ void luaV_settable(lua_State* L, const TValue* t, TValue* key, StkId val) setobj(L, &temp, tm); /* avoid pointing inside table (may rehash) */ t = &temp; } - luaG_runerror(L, "loop in settable"); + luaG_runerror(L, "'__newindex' chain too long; possible loop"); } static int call_binTM(lua_State* L, const TValue* p1, const TValue* p2, StkId res, TMS event) diff --git a/bench/tests/tictactoe.lua b/bench/tests/tictactoe.lua index ae63f5f7..91d38f95 100644 --- a/bench/tests/tictactoe.lua +++ b/bench/tests/tictactoe.lua @@ -139,7 +139,7 @@ function test() for _, curr_qdr in pairs(negaMax.index_quadruplets) do -- iterate over all index quadruplets -- count the empty positions and positions occupied by the side whos move it is local player_plus_fields, player_minus_fields, empties = 0, 0, 0 - for _, index in pairs(curr_qdr) do -- iterate over all indices + for _, index in next, curr_qdr do -- iterate over all indices if board[index] == 0 then empties = empties + 1 elseif board[index] == 1 then @@ -225,4 +225,4 @@ function test() return t1-t0 end -bench.runCode(test, "tictactoe") \ No newline at end of file +bench.runCode(test, "tictactoe") diff --git a/fuzz/proto.cpp b/fuzz/proto.cpp index 22483f9e..f64b6157 100644 --- a/fuzz/proto.cpp +++ b/fuzz/proto.cpp @@ -333,7 +333,7 @@ DEFINE_PROTO_FUZZER(const luau::ModuleSet& message) try { Luau::BytecodeBuilder bcb; - Luau::compileOrThrow(bcb, parseResult.root, parseNameTable, compileOptions); + Luau::compileOrThrow(bcb, parseResult, parseNameTable, compileOptions); bytecode = bcb.getBytecode(); } catch (const Luau::CompileError&) diff --git a/tests/AssemblyBuilderX64.test.cpp b/tests/AssemblyBuilderX64.test.cpp index 15813ae9..28ce6a82 100644 --- a/tests/AssemblyBuilderX64.test.cpp +++ b/tests/AssemblyBuilderX64.test.cpp @@ -155,6 +155,13 @@ TEST_CASE_FIXTURE(AssemblyBuilderX64Fixture, "BaseBinaryInstructionForms") SINGLE_COMPARE(add(qword[rax + r13 * 2 + 0x1b], rsi), 0x4a, 0x01, 0x74, 0x68, 0x1b); SINGLE_COMPARE(add(qword[rbp + rbx * 2], rsi), 0x48, 0x01, 0x74, 0x5d, 0x00); SINGLE_COMPARE(add(qword[rsp + r10 * 2 + 0x1b], r10), 0x4e, 0x01, 0x54, 0x54, 0x1b); + + // [addr], imm + SINGLE_COMPARE(add(byte[rax], 2), 0x80, 0x00, 0x02); + SINGLE_COMPARE(add(dword[rax], 2), 0x83, 0x00, 0x02); + SINGLE_COMPARE(add(dword[rax], 0xabcd), 0x81, 0x00, 0xcd, 0xab, 0x00, 0x00); + SINGLE_COMPARE(add(qword[rax], 2), 0x48, 0x83, 0x00, 0x02); + SINGLE_COMPARE(add(qword[rax], 0xabcd), 0x48, 0x81, 0x00, 0xcd, 0xab, 0x00, 0x00); } TEST_CASE_FIXTURE(AssemblyBuilderX64Fixture, "BaseUnaryInstructionForms") @@ -304,6 +311,13 @@ TEST_CASE_FIXTURE(AssemblyBuilderX64Fixture, "AVXBinaryInstructionForms") SINGLE_COMPARE(vaddps(xmm9, xmm12, xmmword[r9 + r14 * 2 + 0x1c]), 0xc4, 0x01, 0x98, 0x58, 0x4c, 0x71, 0x1c); SINGLE_COMPARE(vaddps(ymm1, ymm2, ymm3), 0xc4, 0xe1, 0xec, 0x58, 0xcb); SINGLE_COMPARE(vaddps(ymm9, ymm12, ymmword[r9 + r14 * 2 + 0x1c]), 0xc4, 0x01, 0x9c, 0x58, 0x4c, 0x71, 0x1c); + + // Coverage for other instructions that follow the same pattern + SINGLE_COMPARE(vsubsd(xmm8, xmm10, xmm14), 0xc4, 0x41, 0xab, 0x5c, 0xc6); + SINGLE_COMPARE(vmulsd(xmm8, xmm10, xmm14), 0xc4, 0x41, 0xab, 0x59, 0xc6); + SINGLE_COMPARE(vdivsd(xmm8, xmm10, xmm14), 0xc4, 0x41, 0xab, 0x5e, 0xc6); + + SINGLE_COMPARE(vxorpd(xmm8, xmm10, xmm14), 0xc4, 0x41, 0xa9, 0x57, 0xc6); } TEST_CASE_FIXTURE(AssemblyBuilderX64Fixture, "AVXUnaryMergeInstructionForms") @@ -318,6 +332,9 @@ TEST_CASE_FIXTURE(AssemblyBuilderX64Fixture, "AVXUnaryMergeInstructionForms") SINGLE_COMPARE(vsqrtsd(xmm8, xmm10, qword[r9]), 0xc4, 0x41, 0xab, 0x51, 0x01); SINGLE_COMPARE(vsqrtss(xmm8, xmm10, xmm14), 0xc4, 0x41, 0xaa, 0x51, 0xc6); SINGLE_COMPARE(vsqrtss(xmm8, xmm10, dword[r9]), 0xc4, 0x41, 0xaa, 0x51, 0x01); + + // Coverage for other instructions that follow the same pattern + SINGLE_COMPARE(vcomisd(xmm8, xmm10), 0xc4, 0x41, 0xf9, 0x2f, 0xc2); } TEST_CASE_FIXTURE(AssemblyBuilderX64Fixture, "AVXMoveInstructionForms") @@ -342,6 +359,11 @@ TEST_CASE_FIXTURE(AssemblyBuilderX64Fixture, "AVXMoveInstructionForms") SINGLE_COMPARE(vmovups(ymm8, ymmword[r9]), 0xc4, 0x41, 0xfc, 0x10, 0x01); } +TEST_CASE_FIXTURE(AssemblyBuilderX64Fixture, "MiscInstructions") +{ + SINGLE_COMPARE(int3(), 0xcc); +} + TEST_CASE("LogTest") { AssemblyBuilderX64 build(/* logText= */ true); @@ -366,6 +388,7 @@ TEST_CASE("LogTest") build.vmovapd(xmmword[rax], xmm11); build.pop(r12); build.ret(); + build.int3(); build.finalize(); @@ -388,6 +411,7 @@ TEST_CASE("LogTest") vmovapd xmmword ptr [rax],xmm11 pop r12 ret + int3 )"; CHECK(same); } diff --git a/tests/Compiler.test.cpp b/tests/Compiler.test.cpp index 46fde067..c1917638 100644 --- a/tests/Compiler.test.cpp +++ b/tests/Compiler.test.cpp @@ -3793,6 +3793,8 @@ RETURN R0 1 TEST_CASE("SharedClosure") { + ScopedFastFlag sff("LuauCompileFreeReassign", true); + // closures can be shared even if functions refer to upvalues, as long as upvalues are top-level CHECK_EQ("\n" + compileFunction(R"( local val = ... @@ -3940,11 +3942,10 @@ LOADN R2 1 LOADN R0 10 LOADN R1 1 FORNPREP R0 L5 -L4: MOVE R3 R2 -GETIMPORT R4 1 -NEWCLOSURE R5 P2 -CAPTURE VAL R3 -CALL R4 1 0 +L4: GETIMPORT R3 1 +NEWCLOSURE R4 P2 +CAPTURE VAL R2 +CALL R3 1 0 FORNLOOP R0 L4 L5: RETURN R0 0 )"); @@ -6157,4 +6158,88 @@ RETURN R0 1 )"); } +TEST_CASE("LocalReassign") +{ + ScopedFastFlag sff("LuauCompileFreeReassign", true); + + // locals can be re-assigned and the register gets reused + CHECK_EQ("\n" + compileFunction0(R"( +local function test(a, b) + local c = a + return c + b +end +)"), R"( +ADD R2 R0 R1 +RETURN R2 1 +)"); + + // this works if the expression is using type casts or grouping + CHECK_EQ("\n" + compileFunction0(R"( +local function test(a, b) + local c = (a :: number) + return c + b +end +)"), R"( +ADD R2 R0 R1 +RETURN R2 1 +)"); + + // the optimization requires that neither local is mutated + CHECK_EQ("\n" + compileFunction0(R"( +local function test(a, b) + local c = a + c += 0 + local d = b + b += 0 + return c + d +end +)"), R"( +MOVE R2 R0 +ADDK R2 R2 K0 +MOVE R3 R1 +ADDK R1 R1 K0 +ADD R4 R2 R3 +RETURN R4 1 +)"); + + // sanity check for two values + CHECK_EQ("\n" + compileFunction0(R"( +local function test(a, b) + local c = a + local d = b + return c + d +end +)"), R"( +ADD R2 R0 R1 +RETURN R2 1 +)"); + + // note: we currently only support this for single assignments + CHECK_EQ("\n" + compileFunction0(R"( +local function test(a, b) + local c, d = a, b + return c + d +end +)"), R"( +MOVE R2 R0 +MOVE R3 R1 +ADD R4 R2 R3 +RETURN R4 1 +)"); + + // of course, captures capture the original register as well (by value since it's immutable) + CHECK_EQ("\n" + compileFunction(R"( +local function test(a, b) + local c = a + local d = b + return function() return c + d end +end +)", 1), R"( +NEWCLOSURE R2 P0 +CAPTURE VAL R0 +CAPTURE VAL R1 +RETURN R2 1 +)"); +} + TEST_SUITE_END(); diff --git a/tests/Conformance.test.cpp b/tests/Conformance.test.cpp index 98c39337..fc8ab2f9 100644 --- a/tests/Conformance.test.cpp +++ b/tests/Conformance.test.cpp @@ -254,9 +254,9 @@ TEST_CASE("Math") runConformance("math.lua"); } -TEST_CASE("Table") +TEST_CASE("Tables") { - runConformance("nextvar.lua", [](lua_State* L) { + runConformance("tables.lua", [](lua_State* L) { lua_pushcfunction( L, [](lua_State* L) { diff --git a/tests/Frontend.test.cpp b/tests/Frontend.test.cpp index 4efa74d9..0382f227 100644 --- a/tests/Frontend.test.cpp +++ b/tests/Frontend.test.cpp @@ -1025,4 +1025,73 @@ TEST_CASE("check_without_builtin_next") frontend.check("Module/B"); } +TEST_CASE_FIXTURE(BuiltinsFixture, "reexport_cyclic_type") +{ + ScopedFastFlag sff[] = { + {"LuauForceExportSurfacesToBeNormal", true}, + {"LuauLowerBoundsCalculation", true}, + {"LuauNormalizeFlagIsConservative", true}, + }; + + fileResolver.source["Module/A"] = R"( + type F = (set: G) -> () + + export type G = { + forEach: (a: F) -> (), + } + + function X(a: F): () + end + + return X + )"; + + fileResolver.source["Module/B"] = R"( + --!strict + local A = require(script.Parent.A) + + export type G = A.G + + return { + A = A, + } + )"; + + CheckResult result = frontend.check("Module/B"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(BuiltinsFixture, "reexport_type_alias") +{ + ScopedFastFlag sff[] = { + {"LuauForceExportSurfacesToBeNormal", true}, + {"LuauLowerBoundsCalculation", true}, + {"LuauNormalizeFlagIsConservative", true}, + }; + + fileResolver.source["Module/A"] = R"( + type KeyOfTestEvents = "test-file-start" | "test-file-success" | "test-file-failure" | "test-case-result" + type unknown = any + + export type TestFileEvent = ( + eventName: T, + args: any --[[ ROBLOX TODO: Unhandled node for type: TSIndexedAccessType ]] --[[ TestEvents[T] ]] + ) -> unknown + + return {} + )"; + + fileResolver.source["Module/B"] = R"( + --!strict + local A = require(script.Parent.A) + + export type TestFileEvent = A.TestFileEvent + )"; + + CheckResult result = frontend.check("Module/B"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + TEST_SUITE_END(); diff --git a/tests/JsonEncoder.test.cpp b/tests/JsonEncoder.test.cpp index 8a263bd2..b16ad3e2 100644 --- a/tests/JsonEncoder.test.cpp +++ b/tests/JsonEncoder.test.cpp @@ -56,10 +56,19 @@ TEST_CASE("encode_constants") AstExprConstantNil nil{Location()}; AstExprConstantBool b{Location(), true}; AstExprConstantNumber n{Location(), 8.2}; + AstExprConstantNumber bigNum{Location(), 0.1677721600000003}; + + AstArray charString; + charString.data = const_cast("a\x1d\0\\\"b"); + charString.size = 6; + + AstExprConstantString needsEscaping{Location(), charString}; CHECK_EQ(R"({"type":"AstExprConstantNil","location":"0,0 - 0,0"})", toJson(&nil)); CHECK_EQ(R"({"type":"AstExprConstantBool","location":"0,0 - 0,0","value":true})", toJson(&b)); - CHECK_EQ(R"({"type":"AstExprConstantNumber","location":"0,0 - 0,0","value":8.2})", toJson(&n)); + CHECK_EQ(R"({"type":"AstExprConstantNumber","location":"0,0 - 0,0","value":8.1999999999999993})", toJson(&n)); + CHECK_EQ(R"({"type":"AstExprConstantNumber","location":"0,0 - 0,0","value":0.16777216000000031})", toJson(&bigNum)); + CHECK_EQ("{\"type\":\"AstExprConstantString\",\"location\":\"0,0 - 0,0\",\"value\":\"a\\u001d\\u0000\\\\\\\"b\"}", toJson(&needsEscaping)); } TEST_CASE("basic_escaping") @@ -87,7 +96,7 @@ TEST_CASE("encode_AstStatBlock") AstStatBlock block{Location(), bodyArray}; CHECK_EQ( - (R"({"type":"AstStatBlock","location":"0,0 - 0,0","body":[{"type":"AstStatLocal","location":"0,0 - 0,0","vars":[{"type":null,"name":"a_local","location":"0,0 - 0,0"}],"values":[]}]})"), + (R"({"type":"AstStatBlock","location":"0,0 - 0,0","body":[{"type":"AstStatLocal","location":"0,0 - 0,0","vars":[{"luauType":null,"name":"a_local","type":"AstLocal","location":"0,0 - 0,0"}],"values":[]}]})"), toJson(&block)); } @@ -106,7 +115,31 @@ TEST_CASE_FIXTURE(JsonEncoderFixture, "encode_tables") CHECK( json == - R"({"type":"AstStatBlock","location":"0,0 - 6,4","body":[{"type":"AstStatLocal","location":"1,8 - 5,9","vars":[{"type":{"type":"AstTypeTable","location":"1,17 - 3,9","props":[{"name":"foo","location":"2,12 - 2,15","type":{"type":"AstTypeReference","location":"2,17 - 2,23","name":"number","parameters":[]}}],"indexer":false},"name":"x","location":"1,14 - 1,15"}],"values":[{"type":"AstExprTable","location":"3,12 - 5,9","items":[{"kind":"record","key":{"type":"AstExprConstantString","location":"4,12 - 4,15","value":"foo"},"value":{"type":"AstExprConstantNumber","location":"4,18 - 4,21","value":123}}]}]}]})"); + R"({"type":"AstStatBlock","location":"0,0 - 6,4","body":[{"type":"AstStatLocal","location":"1,8 - 5,9","vars":[{"luauType":{"type":"AstTypeTable","location":"1,17 - 3,9","props":[{"name":"foo","type":"AstTableProp","location":"2,12 - 2,15","propType":{"type":"AstTypeReference","location":"2,17 - 2,23","name":"number","parameters":[]}}],"indexer":null},"name":"x","type":"AstLocal","location":"1,14 - 1,15"}],"values":[{"type":"AstExprTable","location":"3,12 - 5,9","items":[{"type":"AstExprTableItem","kind":"record","key":{"type":"AstExprConstantString","location":"4,12 - 4,15","value":"foo"},"value":{"type":"AstExprConstantNumber","location":"4,18 - 4,21","value":123}}]}]}]})"); +} + +TEST_CASE_FIXTURE(JsonEncoderFixture, "encode_table_array") +{ + std::string src = R"(type X = {string})"; + + AstStatBlock* root = expectParse(src); + std::string json = toJson(root); + + CHECK( + json == + R"({"type":"AstStatBlock","location":"0,0 - 0,17","body":[{"type":"AstStatTypeAlias","location":"0,0 - 0,17","name":"X","generics":[],"genericPacks":[],"type":{"type":"AstTypeTable","location":"0,9 - 0,17","props":[],"indexer":{"location":"0,10 - 0,16","indexType":{"type":"AstTypeReference","location":"0,10 - 0,16","name":"number","parameters":[]},"resultType":{"type":"AstTypeReference","location":"0,10 - 0,16","name":"string","parameters":[]}}},"exported":false}]})"); +} + +TEST_CASE_FIXTURE(JsonEncoderFixture, "encode_table_indexer") +{ + std::string src = R"(type X = {string})"; + + AstStatBlock* root = expectParse(src); + std::string json = toJson(root); + + CHECK( + json == + R"({"type":"AstStatBlock","location":"0,0 - 0,17","body":[{"type":"AstStatTypeAlias","location":"0,0 - 0,17","name":"X","generics":[],"genericPacks":[],"type":{"type":"AstTypeTable","location":"0,9 - 0,17","props":[],"indexer":{"location":"0,10 - 0,16","indexType":{"type":"AstTypeReference","location":"0,10 - 0,16","name":"number","parameters":[]},"resultType":{"type":"AstTypeReference","location":"0,10 - 0,16","name":"string","parameters":[]}}},"exported":false}]})"); } TEST_CASE("encode_AstExprGroup") @@ -132,12 +165,23 @@ TEST_CASE("encode_AstExprGlobal") CHECK(json == expected); } +TEST_CASE_FIXTURE(JsonEncoderFixture, "encode_AstExprIfThen") +{ + AstStat* statement = expectParseStatement("local a = if x then y else z"); + + std::string_view expected = + R"({"type":"AstStatLocal","location":"0,0 - 0,28","vars":[{"luauType":null,"name":"a","type":"AstLocal","location":"0,6 - 0,7"}],"values":[{"type":"AstExprIfElse","location":"0,10 - 0,28","condition":{"type":"AstExprGlobal","location":"0,13 - 0,14","global":"x"},"hasThen":true,"trueExpr":{"type":"AstExprGlobal","location":"0,20 - 0,21","global":"y"},"hasElse":true,"falseExpr":{"type":"AstExprGlobal","location":"0,27 - 0,28","global":"z"}}]})"; + + CHECK(toJson(statement) == expected); +} + + TEST_CASE("encode_AstExprLocal") { AstLocal local{AstName{"foo"}, Location{}, nullptr, 0, 0, nullptr}; AstExprLocal exprLocal{Location{}, &local, false}; - CHECK(toJson(&exprLocal) == R"({"type":"AstExprLocal","location":"0,0 - 0,0","local":{"type":null,"name":"foo","location":"0,0 - 0,0"}})"); + CHECK(toJson(&exprLocal) == R"({"type":"AstExprLocal","location":"0,0 - 0,0","local":{"luauType":null,"name":"foo","type":"AstLocal","location":"0,0 - 0,0"}})"); } TEST_CASE("encode_AstExprVarargs") @@ -181,7 +225,7 @@ TEST_CASE_FIXTURE(JsonEncoderFixture, "encode_AstExprFunction") AstExpr* expr = expectParseExpr("function (a) return a end"); std::string_view expected = - R"({"type":"AstExprFunction","location":"0,4 - 0,29","generics":[],"genericPacks":[],"args":[{"type":null,"name":"a","location":"0,14 - 0,15"}],"vararg":false,"varargLocation":"0,0 - 0,0","body":{"type":"AstStatBlock","location":"0,16 - 0,26","body":[{"type":"AstStatReturn","location":"0,17 - 0,25","list":[{"type":"AstExprLocal","location":"0,24 - 0,25","local":{"type":null,"name":"a","location":"0,14 - 0,15"}}]}]},"functionDepth":1,"debugname":"","hasEnd":true})"; + R"({"type":"AstExprFunction","location":"0,4 - 0,29","generics":[],"genericPacks":[],"args":[{"luauType":null,"name":"a","type":"AstLocal","location":"0,14 - 0,15"}],"vararg":false,"varargLocation":"0,0 - 0,0","body":{"type":"AstStatBlock","location":"0,16 - 0,26","body":[{"type":"AstStatReturn","location":"0,17 - 0,25","list":[{"type":"AstExprLocal","location":"0,24 - 0,25","local":{"luauType":null,"name":"a","type":"AstLocal","location":"0,14 - 0,15"}}]}]},"functionDepth":1,"debugname":"","hasEnd":true})"; CHECK(toJson(expr) == expected); } @@ -191,7 +235,7 @@ TEST_CASE_FIXTURE(JsonEncoderFixture, "encode_AstExprTable") AstExpr* expr = expectParseExpr("{true, key=true, [key2]=true}"); std::string_view expected = - R"({"type":"AstExprTable","location":"0,4 - 0,33","items":[{"kind":"item","value":{"type":"AstExprConstantBool","location":"0,5 - 0,9","value":true}},{"kind":"record","key":{"type":"AstExprConstantString","location":"0,11 - 0,14","value":"key"},"value":{"type":"AstExprConstantBool","location":"0,15 - 0,19","value":true}},{"kind":"general","key":{"type":"AstExprGlobal","location":"0,22 - 0,26","global":"key2"},"value":{"type":"AstExprConstantBool","location":"0,28 - 0,32","value":true}}]})"; + R"({"type":"AstExprTable","location":"0,4 - 0,33","items":[{"type":"AstExprTableItem","kind":"item","value":{"type":"AstExprConstantBool","location":"0,5 - 0,9","value":true}},{"type":"AstExprTableItem","kind":"record","key":{"type":"AstExprConstantString","location":"0,11 - 0,14","value":"key"},"value":{"type":"AstExprConstantBool","location":"0,15 - 0,19","value":true}},{"type":"AstExprTableItem","kind":"general","key":{"type":"AstExprGlobal","location":"0,22 - 0,26","global":"key2"},"value":{"type":"AstExprConstantBool","location":"0,28 - 0,32","value":true}}]})"; CHECK(toJson(expr) == expected); } @@ -201,7 +245,7 @@ TEST_CASE_FIXTURE(JsonEncoderFixture, "encode_AstExprUnary") AstExpr* expr = expectParseExpr("-b"); std::string_view expected = - R"({"type":"AstExprUnary","location":"0,4 - 0,6","op":"minus","expr":{"type":"AstExprGlobal","location":"0,5 - 0,6","global":"b"}})"; + R"({"type":"AstExprUnary","location":"0,4 - 0,6","op":"Minus","expr":{"type":"AstExprGlobal","location":"0,5 - 0,6","global":"b"}})"; CHECK(toJson(expr) == expected); } @@ -259,7 +303,7 @@ TEST_CASE_FIXTURE(JsonEncoderFixture, "encode_AstStatWhile") AstStat* statement = expectParseStatement("while true do end"); std::string_view expected = - R"({"type":"AtStatWhile","location":"0,0 - 0,17","condition":{"type":"AstExprConstantBool","location":"0,6 - 0,10","value":true},"body":{"type":"AstStatBlock","location":"0,13 - 0,14","body":[]},"hasDo":true,"hasEnd":true})"; + R"({"type":"AstStatWhile","location":"0,0 - 0,17","condition":{"type":"AstExprConstantBool","location":"0,6 - 0,10","value":true},"body":{"type":"AstStatBlock","location":"0,13 - 0,14","body":[]},"hasDo":true,"hasEnd":true})"; CHECK(toJson(statement) == expected); } @@ -279,7 +323,7 @@ TEST_CASE_FIXTURE(JsonEncoderFixture, "encode_AstStatBreak") AstStat* statement = expectParseStatement("while true do break end"); std::string_view expected = - R"({"type":"AtStatWhile","location":"0,0 - 0,23","condition":{"type":"AstExprConstantBool","location":"0,6 - 0,10","value":true},"body":{"type":"AstStatBlock","location":"0,13 - 0,20","body":[{"type":"AstStatBreak","location":"0,14 - 0,19"}]},"hasDo":true,"hasEnd":true})"; + R"({"type":"AstStatWhile","location":"0,0 - 0,23","condition":{"type":"AstExprConstantBool","location":"0,6 - 0,10","value":true},"body":{"type":"AstStatBlock","location":"0,13 - 0,20","body":[{"type":"AstStatBreak","location":"0,14 - 0,19"}]},"hasDo":true,"hasEnd":true})"; CHECK(toJson(statement) == expected); } @@ -289,7 +333,7 @@ TEST_CASE_FIXTURE(JsonEncoderFixture, "encode_AstStatContinue") AstStat* statement = expectParseStatement("while true do continue end"); std::string_view expected = - R"({"type":"AtStatWhile","location":"0,0 - 0,26","condition":{"type":"AstExprConstantBool","location":"0,6 - 0,10","value":true},"body":{"type":"AstStatBlock","location":"0,13 - 0,23","body":[{"type":"AstStatContinue","location":"0,14 - 0,22"}]},"hasDo":true,"hasEnd":true})"; + R"({"type":"AstStatWhile","location":"0,0 - 0,26","condition":{"type":"AstExprConstantBool","location":"0,6 - 0,10","value":true},"body":{"type":"AstStatBlock","location":"0,13 - 0,23","body":[{"type":"AstStatContinue","location":"0,14 - 0,22"}]},"hasDo":true,"hasEnd":true})"; CHECK(toJson(statement) == expected); } @@ -299,7 +343,7 @@ TEST_CASE_FIXTURE(JsonEncoderFixture, "encode_AstStatFor") AstStat* statement = expectParseStatement("for a=0,1 do end"); std::string_view expected = - R"({"type":"AstStatFor","location":"0,0 - 0,16","var":{"type":null,"name":"a","location":"0,4 - 0,5"},"from":{"type":"AstExprConstantNumber","location":"0,6 - 0,7","value":0},"to":{"type":"AstExprConstantNumber","location":"0,8 - 0,9","value":1},"body":{"type":"AstStatBlock","location":"0,12 - 0,13","body":[]},"hasDo":true,"hasEnd":true})"; + R"({"type":"AstStatFor","location":"0,0 - 0,16","var":{"luauType":null,"name":"a","type":"AstLocal","location":"0,4 - 0,5"},"from":{"type":"AstExprConstantNumber","location":"0,6 - 0,7","value":0},"to":{"type":"AstExprConstantNumber","location":"0,8 - 0,9","value":1},"body":{"type":"AstStatBlock","location":"0,12 - 0,13","body":[]},"hasDo":true,"hasEnd":true})"; CHECK(toJson(statement) == expected); } @@ -309,7 +353,7 @@ TEST_CASE_FIXTURE(JsonEncoderFixture, "encode_AstStatForIn") AstStat* statement = expectParseStatement("for a in b do end"); std::string_view expected = - R"({"type":"AstStatForIn","location":"0,0 - 0,17","vars":[{"type":null,"name":"a","location":"0,4 - 0,5"}],"values":[{"type":"AstExprGlobal","location":"0,9 - 0,10","global":"b"}],"body":{"type":"AstStatBlock","location":"0,13 - 0,14","body":[]},"hasIn":true,"hasDo":true,"hasEnd":true})"; + R"({"type":"AstStatForIn","location":"0,0 - 0,17","vars":[{"luauType":null,"name":"a","type":"AstLocal","location":"0,4 - 0,5"}],"values":[{"type":"AstExprGlobal","location":"0,9 - 0,10","global":"b"}],"body":{"type":"AstStatBlock","location":"0,13 - 0,14","body":[]},"hasIn":true,"hasDo":true,"hasEnd":true})"; CHECK(toJson(statement) == expected); } @@ -329,7 +373,7 @@ TEST_CASE_FIXTURE(JsonEncoderFixture, "encode_AstStatLocalFunction") AstStat* statement = expectParseStatement("local function a(b) return end"); std::string_view expected = - R"({"type":"AstStatLocalFunction","location":"0,0 - 0,30","name":{"type":null,"name":"a","location":"0,15 - 0,16"},"func":{"type":"AstExprFunction","location":"0,0 - 0,30","generics":[],"genericPacks":[],"args":[{"type":null,"name":"b","location":"0,17 - 0,18"}],"vararg":false,"varargLocation":"0,0 - 0,0","body":{"type":"AstStatBlock","location":"0,19 - 0,27","body":[{"type":"AstStatReturn","location":"0,20 - 0,26","list":[]}]},"functionDepth":1,"debugname":"a","hasEnd":true}})"; + R"({"type":"AstStatLocalFunction","location":"0,0 - 0,30","name":{"luauType":null,"name":"a","type":"AstLocal","location":"0,15 - 0,16"},"func":{"type":"AstExprFunction","location":"0,0 - 0,30","generics":[],"genericPacks":[],"args":[{"luauType":null,"name":"b","type":"AstLocal","location":"0,17 - 0,18"}],"vararg":false,"varargLocation":"0,0 - 0,0","body":{"type":"AstStatBlock","location":"0,19 - 0,27","body":[{"type":"AstStatReturn","location":"0,20 - 0,26","list":[]}]},"functionDepth":1,"debugname":"a","hasEnd":true}})"; CHECK(toJson(statement) == expected); } @@ -349,7 +393,7 @@ TEST_CASE_FIXTURE(JsonEncoderFixture, "encode_AstStatDeclareFunction") AstStat* statement = expectParseStatement("declare function foo(x: number): string"); std::string_view expected = - R"({"type":"AstStatDeclareFunction","location":"0,0 - 0,39","name":"foo","params":{"types":[{"type":"AstTypeReference","location":"0,24 - 0,30","name":"number","parameters":[]}]},"retTypes":{"types":[{"type":"AstTypeReference","location":"0,33 - 0,39","name":"string","parameters":[]}]},"generics":[],"genericPacks":[]})"; + R"({"type":"AstStatDeclareFunction","location":"0,0 - 0,39","name":"foo","params":{"type":"AstTypeList","types":[{"type":"AstTypeReference","location":"0,24 - 0,30","name":"number","parameters":[]}]},"retTypes":{"type":"AstTypeList","types":[{"type":"AstTypeReference","location":"0,33 - 0,39","name":"string","parameters":[]}]},"generics":[],"genericPacks":[]})"; CHECK(toJson(statement) == expected); } @@ -370,11 +414,11 @@ TEST_CASE_FIXTURE(JsonEncoderFixture, "encode_AstStatDeclareClass") REQUIRE(2 == root->body.size); std::string_view expected1 = - R"({"type":"AstStatDeclareClass","location":"1,22 - 4,11","name":"Foo","props":[{"name":"prop","type":{"type":"AstTypeReference","location":"2,18 - 2,24","name":"number","parameters":[]}},{"name":"method","type":{"type":"AstTypeFunction","location":"3,21 - 4,11","generics":[],"genericPacks":[],"argTypes":{"types":[{"type":"AstTypeReference","location":"3,39 - 3,45","name":"number","parameters":[]}]},"returnTypes":{"types":[{"type":"AstTypeReference","location":"3,48 - 3,54","name":"string","parameters":[]}]}}}]})"; + R"({"type":"AstStatDeclareClass","location":"1,22 - 4,11","name":"Foo","props":[{"name":"prop","type":"AstDeclaredClassProp","luauType":{"type":"AstTypeReference","location":"2,18 - 2,24","name":"number","parameters":[]}},{"name":"method","type":"AstDeclaredClassProp","luauType":{"type":"AstTypeFunction","location":"3,21 - 4,11","generics":[],"genericPacks":[],"argTypes":{"type":"AstTypeList","types":[{"type":"AstTypeReference","location":"3,39 - 3,45","name":"number","parameters":[]}]},"returnTypes":{"type":"AstTypeList","types":[{"type":"AstTypeReference","location":"3,48 - 3,54","name":"string","parameters":[]}]}}}]})"; CHECK(toJson(root->body.data[0]) == expected1); std::string_view expected2 = - R"({"type":"AstStatDeclareClass","location":"6,22 - 8,11","name":"Bar","superName":"Foo","props":[{"name":"prop2","type":{"type":"AstTypeReference","location":"7,19 - 7,25","name":"string","parameters":[]}}]})"; + R"({"type":"AstStatDeclareClass","location":"6,22 - 8,11","name":"Bar","superName":"Foo","props":[{"name":"prop2","type":"AstDeclaredClassProp","luauType":{"type":"AstTypeReference","location":"7,19 - 7,25","name":"string","parameters":[]}}]})"; CHECK(toJson(root->body.data[1]) == expected2); } @@ -383,7 +427,7 @@ TEST_CASE_FIXTURE(JsonEncoderFixture, "encode_annotation") AstStat* statement = expectParseStatement("type T = ((number) -> (string | nil)) & ((string) -> ())"); std::string_view expected = - R"({"type":"AstStatTypeAlias","location":"0,0 - 0,55","name":"T","generics":[],"genericPacks":[],"type":{"type":"AstTypeIntersection","location":"0,9 - 0,55","types":[{"type":"AstTypeFunction","location":"0,10 - 0,35","generics":[],"genericPacks":[],"argTypes":{"types":[{"type":"AstTypeReference","location":"0,11 - 0,17","name":"number","parameters":[]}]},"returnTypes":{"types":[{"type":"AstTypeUnion","location":"0,23 - 0,35","types":[{"type":"AstTypeReference","location":"0,23 - 0,29","name":"string","parameters":[]},{"type":"AstTypeReference","location":"0,32 - 0,35","name":"nil","parameters":[]}]}]}},{"type":"AstTypeFunction","location":"0,41 - 0,55","generics":[],"genericPacks":[],"argTypes":{"types":[{"type":"AstTypeReference","location":"0,42 - 0,48","name":"string","parameters":[]}]},"returnTypes":{"types":[]}}]},"exported":false})"; + R"({"type":"AstStatTypeAlias","location":"0,0 - 0,55","name":"T","generics":[],"genericPacks":[],"type":{"type":"AstTypeIntersection","location":"0,9 - 0,55","types":[{"type":"AstTypeFunction","location":"0,10 - 0,35","generics":[],"genericPacks":[],"argTypes":{"type":"AstTypeList","types":[{"type":"AstTypeReference","location":"0,11 - 0,17","name":"number","parameters":[]}]},"returnTypes":{"type":"AstTypeList","types":[{"type":"AstTypeUnion","location":"0,23 - 0,35","types":[{"type":"AstTypeReference","location":"0,23 - 0,29","name":"string","parameters":[]},{"type":"AstTypeReference","location":"0,32 - 0,35","name":"nil","parameters":[]}]}]}},{"type":"AstTypeFunction","location":"0,41 - 0,55","generics":[],"genericPacks":[],"argTypes":{"type":"AstTypeList","types":[{"type":"AstTypeReference","location":"0,42 - 0,48","name":"string","parameters":[]}]},"returnTypes":{"type":"AstTypeList","types":[]}}]},"exported":false})"; CHECK(toJson(statement) == expected); } @@ -411,7 +455,7 @@ TEST_CASE_FIXTURE(JsonEncoderFixture, "encode_AstTypePackExplicit") CHECK(2 == root->body.size); std::string_view expected = - R"({"type":"AstStatLocal","location":"2,8 - 2,36","vars":[{"type":{"type":"AstTypeReference","location":"2,17 - 2,36","name":"A","parameters":[{"type":"AstTypePackExplicit","location":"2,19 - 2,20","typeList":{"types":[{"type":"AstTypeReference","location":"2,20 - 2,26","name":"number","parameters":[]},{"type":"AstTypeReference","location":"2,28 - 2,34","name":"string","parameters":[]}]}}]},"name":"a","location":"2,14 - 2,15"}],"values":[]})"; + R"({"type":"AstStatLocal","location":"2,8 - 2,36","vars":[{"luauType":{"type":"AstTypeReference","location":"2,17 - 2,36","name":"A","parameters":[{"type":"AstTypePackExplicit","location":"2,19 - 2,20","typeList":{"type":"AstTypeList","types":[{"type":"AstTypeReference","location":"2,20 - 2,26","name":"number","parameters":[]},{"type":"AstTypeReference","location":"2,28 - 2,34","name":"string","parameters":[]}]}}]},"name":"a","type":"AstLocal","location":"2,14 - 2,15"}],"values":[]})"; CHECK(toJson(root->body.data[1]) == expected); } diff --git a/tests/NonstrictMode.test.cpp b/tests/NonstrictMode.test.cpp index 50dcbad0..02e02e6b 100644 --- a/tests/NonstrictMode.test.cpp +++ b/tests/NonstrictMode.test.cpp @@ -13,77 +13,6 @@ using namespace Luau; TEST_SUITE_BEGIN("NonstrictModeTests"); -TEST_CASE_FIXTURE(Fixture, "globals") -{ - CheckResult result = check(R"( - --!nonstrict - foo = true - foo = "now i'm a string!" - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - CHECK_EQ("any", toString(requireType("foo"))); -} - -TEST_CASE_FIXTURE(Fixture, "globals2") -{ - ScopedFastFlag sff[]{ - {"LuauReturnTypeInferenceInNonstrict", true}, - {"LuauLowerBoundsCalculation", true}, - }; - - CheckResult result = check(R"( - --!nonstrict - foo = function() return 1 end - foo = "now i'm a string!" - )"); - - LUAU_REQUIRE_ERROR_COUNT(1, result); - - TypeMismatch* tm = get(result.errors[0]); - REQUIRE(tm); - CHECK_EQ("() -> number", toString(tm->wantedType)); - CHECK_EQ("string", toString(tm->givenType)); - CHECK_EQ("() -> number", toString(requireType("foo"))); -} - -TEST_CASE_FIXTURE(Fixture, "globals_everywhere") -{ - CheckResult result = check(R"( - --!nonstrict - foo = 1 - - if true then - bar = 2 - end - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - - CHECK_EQ("any", toString(requireType("foo"))); - CHECK_EQ("any", toString(requireType("bar"))); -} - -TEST_CASE_FIXTURE(BuiltinsFixture, "function_returns_number_or_string") -{ - ScopedFastFlag sff[]{{"LuauReturnTypeInferenceInNonstrict", true}, {"LuauLowerBoundsCalculation", true}}; - - CheckResult result = check(R"( - --!nonstrict - local function f() - if math.random() > 0.5 then - return 5 - else - return "hi" - end - end - )"); - - LUAU_REQUIRE_NO_ERRORS(result); - - CHECK("() -> number | string" == toString(requireType("f"))); -} - TEST_CASE_FIXTURE(Fixture, "infer_nullary_function") { CheckResult result = check(R"( @@ -106,13 +35,8 @@ TEST_CASE_FIXTURE(Fixture, "infer_nullary_function") REQUIRE_EQ(0, rets.size()); } -TEST_CASE_FIXTURE(Fixture, "first_return_type_dictates_number_of_return_types") +TEST_CASE_FIXTURE(Fixture, "infer_the_maximum_number_of_values_the_function_could_return") { - ScopedFastFlag sff[]{ - {"LuauReturnTypeInferenceInNonstrict", true}, - {"LuauLowerBoundsCalculation", true}, - }; - CheckResult result = check(R"( --!nonstrict function getMinCardCountForWidth(width) @@ -127,18 +51,22 @@ TEST_CASE_FIXTURE(Fixture, "first_return_type_dictates_number_of_return_types") TypeId t = requireType("getMinCardCountForWidth"); REQUIRE(t); - REQUIRE_EQ("(any) -> number", toString(t)); + REQUIRE_EQ("(any) -> (...any)", toString(t)); } +#if 0 +// Maybe we want this? TEST_CASE_FIXTURE(Fixture, "return_annotation_is_still_checked") { CheckResult result = check(R"( - --!nonstrict function foo(x): number return 'hello' end )"); LUAU_REQUIRE_ERROR_COUNT(1, result); + + REQUIRE_NE(*typeChecker.anyType, *requireType("foo")); } +#endif TEST_CASE_FIXTURE(Fixture, "function_parameters_are_any") { @@ -324,11 +252,6 @@ TEST_CASE_FIXTURE(Fixture, "delay_function_does_not_require_its_argument_to_retu TEST_CASE_FIXTURE(Fixture, "inconsistent_module_return_types_are_ok") { - ScopedFastFlag sff[]{ - {"LuauReturnTypeInferenceInNonstrict", true}, - {"LuauLowerBoundsCalculation", true}, - }; - CheckResult result = check(R"( --!nonstrict @@ -345,7 +268,7 @@ TEST_CASE_FIXTURE(Fixture, "inconsistent_module_return_types_are_ok") LUAU_REQUIRE_NO_ERRORS(result); - REQUIRE_EQ("((any) -> string) | {| foo: any |}", toString(getMainModule()->getModuleScope()->returnType)); + REQUIRE_EQ("any", toString(getMainModule()->getModuleScope()->returnType)); } TEST_CASE_FIXTURE(Fixture, "returning_insufficient_return_values") diff --git a/tests/Normalize.test.cpp b/tests/Normalize.test.cpp index 69e7f074..84a5a380 100644 --- a/tests/Normalize.test.cpp +++ b/tests/Normalize.test.cpp @@ -621,7 +621,6 @@ TEST_CASE_FIXTURE(Fixture, "normalize_module_return_type") { ScopedFastFlag sff[] = { {"LuauLowerBoundsCalculation", true}, - {"LuauReturnTypeInferenceInNonstrict", true}, }; check(R"( @@ -642,7 +641,7 @@ TEST_CASE_FIXTURE(Fixture, "normalize_module_return_type") end )"); - CHECK_EQ("(any, any) -> (any, any) -> any", toString(getMainModule()->getModuleScope()->returnType)); + CHECK_EQ("(any, any) -> (...any)", toString(getMainModule()->getModuleScope()->returnType)); } TEST_CASE_FIXTURE(Fixture, "return_type_is_not_a_constrained_intersection") diff --git a/tests/TypeInfer.functions.test.cpp b/tests/TypeInfer.functions.test.cpp index 6e6549d3..a634ba09 100644 --- a/tests/TypeInfer.functions.test.cpp +++ b/tests/TypeInfer.functions.test.cpp @@ -677,11 +677,6 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "toposort_doesnt_break_mutual_recursion") TEST_CASE_FIXTURE(Fixture, "check_function_before_lambda_that_uses_it") { - ScopedFastFlag sff[]{ - {"LuauReturnTypeInferenceInNonstrict", true}, - {"LuauLowerBoundsCalculation", true}, - }; - CheckResult result = check(R"( --!nonstrict @@ -690,7 +685,7 @@ TEST_CASE_FIXTURE(Fixture, "check_function_before_lambda_that_uses_it") end return function() - return f() + return f():andThen() end )"); @@ -817,18 +812,14 @@ TEST_CASE_FIXTURE(Fixture, "calling_function_with_incorrect_argument_type_yields TEST_CASE_FIXTURE(BuiltinsFixture, "calling_function_with_anytypepack_doesnt_leak_free_types") { - ScopedFastFlag sff[]{ - {"LuauReturnTypeInferenceInNonstrict", true}, - {"LuauLowerBoundsCalculation", true}, - }; - CheckResult result = check(R"( --!nonstrict - function Test(a): ...any + function Test(a) return 1, "" end + local tab = {} table.insert(tab, Test(1)); )"); @@ -1625,21 +1616,6 @@ TEST_CASE_FIXTURE(Fixture, "occurs_check_failure_in_function_return_type") CHECK(nullptr != get(result.errors[0])); } -TEST_CASE_FIXTURE(Fixture, "weird_fail_to_unify_type_pack") -{ - ScopedFastFlag sff[]{ - {"LuauReturnTypeInferenceInNonstrict", true}, - {"LuauLowerBoundsCalculation", true}, - }; - - CheckResult result = check(R"( - local function f() return end - local g = function() return f() end - )"); - - LUAU_REQUIRE_NO_ERRORS(result); -} - TEST_CASE_FIXTURE(Fixture, "quantify_constrained_types") { ScopedFastFlag sff[]{ diff --git a/tests/TypeInfer.loops.test.cpp b/tests/TypeInfer.loops.test.cpp index 56b807f8..354b3996 100644 --- a/tests/TypeInfer.loops.test.cpp +++ b/tests/TypeInfer.loops.test.cpp @@ -516,7 +516,7 @@ TEST_CASE_FIXTURE(Fixture, "loop_iter_trailing_nil") CHECK_EQ(*typeChecker.nilType, *requireType("extra")); } -TEST_CASE_FIXTURE(Fixture, "loop_iter_no_indexer") +TEST_CASE_FIXTURE(Fixture, "loop_iter_no_indexer_strict") { CheckResult result = check(R"( local t = {} @@ -531,6 +531,17 @@ TEST_CASE_FIXTURE(Fixture, "loop_iter_no_indexer") CHECK_EQ("Cannot iterate over a table without indexer", ge->message); } +TEST_CASE_FIXTURE(Fixture, "loop_iter_no_indexer_nonstrict") +{ + CheckResult result = check(Mode::Nonstrict, R"( + local t = {} + for k, v in t do + end + )"); + + LUAU_REQUIRE_ERROR_COUNT(0, result); +} + TEST_CASE_FIXTURE(BuiltinsFixture, "loop_iter_iter_metamethod") { CheckResult result = check(R"( diff --git a/tests/TypeInfer.provisional.test.cpp b/tests/TypeInfer.provisional.test.cpp index dc686890..34afd560 100644 --- a/tests/TypeInfer.provisional.test.cpp +++ b/tests/TypeInfer.provisional.test.cpp @@ -343,6 +343,20 @@ TEST_CASE_FIXTURE(Fixture, "specialization_binds_with_prototypes_too_early") LUAU_REQUIRE_ERRORS(result); // Should not have any errors. } +TEST_CASE_FIXTURE(Fixture, "weird_fail_to_unify_type_pack") +{ + ScopedFastFlag sff[] = { + {"LuauLowerBoundsCalculation", false}, + }; + + CheckResult result = check(R"( + local function f() return end + local g = function() return f() end + )"); + + LUAU_REQUIRE_ERRORS(result); // Should not have any errors. +} + TEST_CASE_FIXTURE(Fixture, "weird_fail_to_unify_variadic_pack") { ScopedFastFlag sff[] = { diff --git a/tests/TypeInfer.tables.test.cpp b/tests/TypeInfer.tables.test.cpp index 21ad4e1b..d9bfc89d 100644 --- a/tests/TypeInfer.tables.test.cpp +++ b/tests/TypeInfer.tables.test.cpp @@ -2990,6 +2990,15 @@ TEST_CASE_FIXTURE(Fixture, "expected_indexer_value_type_extra_2") LUAU_REQUIRE_NO_ERRORS(result); } +TEST_CASE_FIXTURE(Fixture, "expected_indexer_from_table_union") +{ + ScopedFastFlag luauExpectedTableUnionIndexerType{"LuauExpectedTableUnionIndexerType", true}; + + LUAU_REQUIRE_NO_ERRORS(check(R"(local a: {[string]: {number | string}} = {a = {2, 's'}})")); + LUAU_REQUIRE_NO_ERRORS(check(R"(local a: {[string]: {number | string}}? = {a = {2, 's'}})")); + LUAU_REQUIRE_NO_ERRORS(check(R"(local a: {[string]: {[string]: {string?}}?} = {["a"] = {["b"] = {"a", "b"}}})")); +} + TEST_CASE_FIXTURE(Fixture, "prop_access_on_key_whose_types_mismatches") { ScopedFastFlag sff{"LuauReportErrorsOnIndexerKeyMismatch", true}; diff --git a/tests/TypeInfer.test.cpp b/tests/TypeInfer.test.cpp index 7d1fb56b..858e8ac0 100644 --- a/tests/TypeInfer.test.cpp +++ b/tests/TypeInfer.test.cpp @@ -85,20 +85,19 @@ TEST_CASE_FIXTURE(Fixture, "infer_in_nocheck_mode") { ScopedFastFlag sff[]{ {"DebugLuauDeferredConstraintResolution", false}, - {"LuauReturnTypeInferenceInNonstrict", true}, {"LuauLowerBoundsCalculation", true}, }; CheckResult result = check(R"( --!nocheck function f(x) - return 5 + return x end -- we get type information even if there's type errors f(1, 2) )"); - CHECK_EQ("(any) -> number", toString(requireType("f"))); + CHECK_EQ("(any) -> (...any)", toString(requireType("f"))); LUAU_REQUIRE_NO_ERRORS(result); } @@ -355,6 +354,35 @@ TEST_CASE_FIXTURE(Fixture, "check_expr_recursion_limit") CHECK(nullptr != get(result.errors[0])); } +TEST_CASE_FIXTURE(Fixture, "globals") +{ + CheckResult result = check(R"( + --!nonstrict + foo = true + foo = "now i'm a string!" + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + CHECK_EQ("any", toString(requireType("foo"))); +} + +TEST_CASE_FIXTURE(Fixture, "globals2") +{ + CheckResult result = check(R"( + --!nonstrict + foo = function() return 1 end + foo = "now i'm a string!" + )"); + + LUAU_REQUIRE_ERROR_COUNT(1, result); + + TypeMismatch* tm = get(result.errors[0]); + REQUIRE(tm); + CHECK_EQ("() -> (...any)", toString(tm->wantedType)); + CHECK_EQ("string", toString(tm->givenType)); + CHECK_EQ("() -> (...any)", toString(requireType("foo"))); +} + TEST_CASE_FIXTURE(Fixture, "globals_are_banned_in_strict_mode") { CheckResult result = check(R"( diff --git a/tests/conformance/nextvar.lua b/tests/conformance/tables.lua similarity index 96% rename from tests/conformance/nextvar.lua rename to tests/conformance/tables.lua index 93c4ddf7..0eff8540 100644 --- a/tests/conformance/nextvar.lua +++ b/tests/conformance/tables.lua @@ -592,4 +592,24 @@ do assert(countud() == 3) end +-- test __newindex-as-a-table indirection: this had memory safety bugs in Lua 5.1.0 +do + local hit = false + + local grandparent = {} + grandparent.__newindex = function(s,k,v) + assert(k == "foo" and v == 10) + hit = true + end + + local parent = {} + parent.__newindex = parent + setmetatable(parent, grandparent) + + local child = setmetatable({}, parent) + child.foo = 10 + + assert(hit and child.foo == nil and parent.foo == nil) +end + return"OK" diff --git a/tests/main.cpp b/tests/main.cpp index c5b844b5..e7c4aed6 100644 --- a/tests/main.cpp +++ b/tests/main.cpp @@ -63,7 +63,7 @@ static int testAssertionHandler(const char* expr, const char* file, int line, co if (debuggerPresent()) LUAU_DEBUGBREAK(); - ADD_FAIL_AT(file, line, "Assertion failed: ", expr); + ADD_FAIL_AT(file, line, "Assertion failed: ", std::string(expr)); return 1; } diff --git a/tools/faillist.txt b/tools/faillist.txt new file mode 100644 index 00000000..dc74a6a9 --- /dev/null +++ b/tools/faillist.txt @@ -0,0 +1,521 @@ +AnnotationTests.as_expr_does_not_propagate_type_info +AnnotationTests.as_expr_is_bidirectional +AnnotationTests.as_expr_warns_on_unrelated_cast +AnnotationTests.builtin_types_are_not_exported +AnnotationTests.cannot_use_nonexported_type +AnnotationTests.cloned_interface_maintains_pointers_between_definitions +AnnotationTests.corecursive_types_error_on_tight_loop +AnnotationTests.define_generic_type_alias +AnnotationTests.duplicate_type_param_name +AnnotationTests.for_loop_counter_annotation_is_checked +AnnotationTests.function_return_annotations_are_checked +AnnotationTests.generic_aliases_are_cloned_properly +AnnotationTests.instantiate_type_fun_should_not_trip_rbxassert +AnnotationTests.instantiation_clone_has_to_follow +AnnotationTests.interface_types_belong_to_interface_arena +AnnotationTests.luau_ice_triggers_an_ice +AnnotationTests.luau_ice_triggers_an_ice_exception_with_flag +AnnotationTests.luau_ice_triggers_an_ice_exception_with_flag_handler +AnnotationTests.luau_ice_triggers_an_ice_handler +AnnotationTests.luau_print_is_magic_if_the_flag_is_set +AnnotationTests.luau_print_is_not_special_without_the_flag +AnnotationTests.occurs_check_on_cyclic_intersection_typevar +AnnotationTests.occurs_check_on_cyclic_union_typevar +AnnotationTests.self_referential_type_alias +AnnotationTests.too_many_type_params +AnnotationTests.two_type_params +AnnotationTests.type_alias_always_resolve_to_a_real_type +AnnotationTests.type_alias_B_should_check_with_another_aliases_until_a_non_aliased_type +AnnotationTests.type_alias_should_alias_to_number +AnnotationTests.type_aliasing_to_number_should_not_check_given_a_string +AnnotationTests.type_annotations_inside_function_bodies +AnnotationTests.type_assertion_expr +AnnotationTests.typeof_variable_type_annotation_should_return_its_type +AnnotationTests.use_generic_type_alias +AnnotationTests.use_type_required_from_another_file +AstQuery.last_argument_function_call_type +AstQuery::getDocumentationSymbolAtPosition.binding +AstQuery::getDocumentationSymbolAtPosition.event_callback_arg +AstQuery::getDocumentationSymbolAtPosition.overloaded_fn +AstQuery::getDocumentationSymbolAtPosition.prop +AutocompleteTest.argument_types +AutocompleteTest.arguments_to_global_lambda +AutocompleteTest.as_types +AutocompleteTest.autocomplete_boolean_singleton +AutocompleteTest.autocomplete_default_type_pack_parameters +AutocompleteTest.autocomplete_default_type_parameters +AutocompleteTest.autocomplete_documentation_symbols +AutocompleteTest.autocomplete_end_with_fn_exprs +AutocompleteTest.autocomplete_end_with_lambda +AutocompleteTest.autocomplete_explicit_type_pack +AutocompleteTest.autocomplete_first_function_arg_expected_type +AutocompleteTest.autocomplete_for_in_middle_keywords +AutocompleteTest.autocomplete_for_middle_keywords +AutocompleteTest.autocomplete_if_else_regression +AutocompleteTest.autocomplete_if_middle_keywords +AutocompleteTest.autocomplete_ifelse_expressions +AutocompleteTest.autocomplete_on_string_singletons +AutocompleteTest.autocomplete_oop_implicit_self +AutocompleteTest.autocomplete_repeat_middle_keyword +AutocompleteTest.autocomplete_string_singleton_equality +AutocompleteTest.autocomplete_string_singleton_escape +AutocompleteTest.autocomplete_string_singletons +AutocompleteTest.autocomplete_until_expression +AutocompleteTest.autocomplete_until_in_repeat +AutocompleteTest.autocomplete_while_middle_keywords +AutocompleteTest.autocompleteProp_index_function_metamethod_is_variadic +AutocompleteTest.bias_toward_inner_scope +AutocompleteTest.comments +AutocompleteTest.cyclic_table +AutocompleteTest.do_not_overwrite_context_sensitive_kws +AutocompleteTest.do_not_suggest_internal_module_type +AutocompleteTest.do_not_suggest_synthetic_table_name +AutocompleteTest.dont_offer_any_suggestions_from_the_end_of_a_comment +AutocompleteTest.dont_offer_any_suggestions_from_within_a_broken_comment +AutocompleteTest.dont_offer_any_suggestions_from_within_a_broken_comment_at_the_very_end_of_the_file +AutocompleteTest.dont_offer_any_suggestions_from_within_a_comment +AutocompleteTest.dont_suggest_local_before_its_definition +AutocompleteTest.empty_program +AutocompleteTest.function_expr_params +AutocompleteTest.function_in_assignment_has_parentheses +AutocompleteTest.function_in_assignment_has_parentheses_2 +AutocompleteTest.function_parameters +AutocompleteTest.function_result_passed_to_function_has_parentheses +AutocompleteTest.function_type_types +AutocompleteTest.generic_types +AutocompleteTest.get_member_completions +AutocompleteTest.get_string_completions +AutocompleteTest.get_suggestions_for_new_statement +AutocompleteTest.get_suggestions_for_the_very_start_of_the_script +AutocompleteTest.global_function_params +AutocompleteTest.global_functions_are_not_scoped_lexically +AutocompleteTest.if_then_else_elseif_completions +AutocompleteTest.if_then_else_full_keywords +AutocompleteTest.keyword_members +AutocompleteTest.keyword_methods +AutocompleteTest.keyword_types +AutocompleteTest.leave_numbers_alone +AutocompleteTest.library_non_self_calls_are_fine +AutocompleteTest.library_self_calls_are_invalid +AutocompleteTest.local_function +AutocompleteTest.local_function_params +AutocompleteTest.local_functions_fall_out_of_scope +AutocompleteTest.local_initializer +AutocompleteTest.local_initializer_2 +AutocompleteTest.local_names +AutocompleteTest.local_types_builtin +AutocompleteTest.method_call_inside_function_body +AutocompleteTest.method_call_inside_if_conditional +AutocompleteTest.module_type_members +AutocompleteTest.modules_with_types +AutocompleteTest.nested_member_completions +AutocompleteTest.nested_recursive_function +AutocompleteTest.no_function_name_suggestions +AutocompleteTest.no_incompatible_self_calls +AutocompleteTest.no_incompatible_self_calls_2 +AutocompleteTest.no_incompatible_self_calls_on_class +AutocompleteTest.no_incompatible_self_calls_provisional +AutocompleteTest.not_the_var_we_are_defining +AutocompleteTest.optional_members +AutocompleteTest.private_types +AutocompleteTest.recommend_statement_starting_keywords +AutocompleteTest.recursive_function +AutocompleteTest.recursive_function_global +AutocompleteTest.recursive_function_local +AutocompleteTest.return_types +AutocompleteTest.skip_current_local +AutocompleteTest.sometimes_the_metatable_is_an_error +AutocompleteTest.source_module_preservation_and_invalidation +AutocompleteTest.statement_between_two_statements +AutocompleteTest.stop_at_first_stat_when_recommending_keywords +AutocompleteTest.string_prim_non_self_calls_are_avoided +AutocompleteTest.string_prim_self_calls_are_fine +AutocompleteTest.suggest_external_module_type +AutocompleteTest.suggest_table_keys +AutocompleteTest.table_intersection +AutocompleteTest.table_union +AutocompleteTest.type_correct_argument_type_suggestion +AutocompleteTest.type_correct_expected_argument_type_pack_suggestion +AutocompleteTest.type_correct_expected_argument_type_suggestion +AutocompleteTest.type_correct_expected_argument_type_suggestion_optional +AutocompleteTest.type_correct_expected_argument_type_suggestion_self +AutocompleteTest.type_correct_expected_return_type_pack_suggestion +AutocompleteTest.type_correct_expected_return_type_suggestion +AutocompleteTest.type_correct_full_type_suggestion +AutocompleteTest.type_correct_function_no_parenthesis +AutocompleteTest.type_correct_function_return_types +AutocompleteTest.type_correct_function_type_suggestion +AutocompleteTest.type_correct_keywords +AutocompleteTest.type_correct_local_type_suggestion +AutocompleteTest.type_correct_sealed_table +AutocompleteTest.type_correct_suggestion_for_overloads +AutocompleteTest.type_correct_suggestion_in_argument +AutocompleteTest.type_correct_suggestion_in_table +AutocompleteTest.type_scoping_easy +AutocompleteTest.unsealed_table +AutocompleteTest.unsealed_table_2 +AutocompleteTest.user_defined_globals +AutocompleteTest.user_defined_local_functions_in_own_definition +BuiltinDefinitionsTest.lib_documentation_symbols +BuiltinTests.aliased_string_format +BuiltinTests.assert_removes_falsy_types +BuiltinTests.assert_removes_falsy_types2 +BuiltinTests.assert_removes_falsy_types_even_from_type_pack_tail_but_only_for_the_first_type +BuiltinTests.assert_returns_false_and_string_iff_it_knows_the_first_argument_cannot_be_truthy +BuiltinTests.bad_select_should_not_crash +BuiltinTests.builtin_tables_sealed +BuiltinTests.coroutine_resume_anything_goes +BuiltinTests.coroutine_wrap_anything_goes +BuiltinTests.debug_info_is_crazy +BuiltinTests.debug_traceback_is_crazy +BuiltinTests.dont_add_definitions_to_persistent_types +BuiltinTests.find_capture_types +BuiltinTests.find_capture_types2 +BuiltinTests.find_capture_types3 +BuiltinTests.gcinfo +BuiltinTests.getfenv +BuiltinTests.global_singleton_types_are_sealed +BuiltinTests.gmatch_capture_types +BuiltinTests.gmatch_capture_types2 +BuiltinTests.gmatch_capture_types_balanced_escaped_parens +BuiltinTests.gmatch_capture_types_default_capture +BuiltinTests.gmatch_capture_types_invalid_pattern_fallback_to_builtin +BuiltinTests.gmatch_capture_types_invalid_pattern_fallback_to_builtin2 +BuiltinTests.gmatch_capture_types_leading_end_bracket_is_part_of_set +BuiltinTests.gmatch_capture_types_parens_in_sets_are_ignored +BuiltinTests.gmatch_capture_types_set_containing_lbracket +BuiltinTests.gmatch_definition +BuiltinTests.ipairs_iterator_should_infer_types_and_type_check +BuiltinTests.lua_51_exported_globals_all_exist +BuiltinTests.match_capture_types +BuiltinTests.match_capture_types2 +BuiltinTests.math_max_checks_for_numbers +BuiltinTests.math_max_variatic +BuiltinTests.math_things_are_defined +BuiltinTests.next_iterator_should_infer_types_and_type_check +BuiltinTests.no_persistent_typelevel_change +BuiltinTests.os_time_takes_optional_date_table +BuiltinTests.pairs_iterator_should_infer_types_and_type_check +BuiltinTests.see_thru_select +BuiltinTests.see_thru_select_count +BuiltinTests.select_on_variadic +BuiltinTests.select_slightly_out_of_range +BuiltinTests.select_way_out_of_range +BuiltinTests.select_with_decimal_argument_is_rounded_down +BuiltinTests.select_with_variadic_typepack_tail +BuiltinTests.select_with_variadic_typepack_tail_and_string_head +BuiltinTests.set_metatable_needs_arguments +BuiltinTests.setmetatable_should_not_mutate_persisted_types +BuiltinTests.setmetatable_unpacks_arg_types_correctly +BuiltinTests.sort +BuiltinTests.sort_with_bad_predicate +BuiltinTests.sort_with_predicate +BuiltinTests.string_format_arg_count_mismatch +BuiltinTests.string_format_arg_types_inference +BuiltinTests.string_format_as_method +BuiltinTests.string_format_correctly_ordered_types +BuiltinTests.string_format_report_all_type_errors_at_correct_positions +BuiltinTests.string_format_use_correct_argument +BuiltinTests.string_format_use_correct_argument2 +BuiltinTests.string_lib_self_noself +BuiltinTests.table_concat_returns_string +BuiltinTests.table_dot_remove_optionally_returns_generic +BuiltinTests.table_freeze_is_generic +BuiltinTests.table_insert_correctly_infers_type_of_array_2_args_overload +BuiltinTests.table_insert_correctly_infers_type_of_array_3_args_overload +BuiltinTests.table_pack +BuiltinTests.table_pack_reduce +BuiltinTests.table_pack_variadic +BuiltinTests.thread_is_a_type +BuiltinTests.tonumber_returns_optional_number_type +BuiltinTests.tonumber_returns_optional_number_type2 +BuiltinTests.xpcall +DefinitionTests.class_definition_function_prop +DefinitionTests.class_definitions_cannot_extend_non_class +DefinitionTests.class_definitions_cannot_overload_non_function +DefinitionTests.declaring_generic_functions +DefinitionTests.definition_file_class_function_args +DefinitionTests.definition_file_classes +DefinitionTests.definition_file_loading +DefinitionTests.definitions_documentation_symbols +DefinitionTests.documentation_symbols_dont_attach_to_persistent_types +DefinitionTests.load_definition_file_errors_do_not_pollute_global_scope +DefinitionTests.no_cyclic_defined_classes +DefinitionTests.single_class_type_identity_in_global_types +FrontendTest.accumulate_cached_errors +FrontendTest.accumulate_cached_errors_in_consistent_order +FrontendTest.any_annotation_breaks_cycle +FrontendTest.ast_node_at_position +FrontendTest.automatically_check_cyclically_dependent_scripts +FrontendTest.automatically_check_dependent_scripts +FrontendTest.check_without_builtin_next +FrontendTest.clearStats +FrontendTest.cycle_detection_between_check_and_nocheck +FrontendTest.cycle_detection_disabled_in_nocheck +FrontendTest.cycle_error_paths +FrontendTest.cycle_errors_can_be_fixed +FrontendTest.cycle_incremental_type_surface +FrontendTest.cycle_incremental_type_surface_longer +FrontendTest.discard_type_graphs +FrontendTest.dont_recheck_script_that_hasnt_been_marked_dirty +FrontendTest.dont_reparse_clean_file_when_linting +FrontendTest.environments +FrontendTest.find_a_require +FrontendTest.find_a_require_inside_a_function +FrontendTest.ignore_require_to_nonexistent_file +FrontendTest.imported_table_modification_2 +FrontendTest.it_should_be_safe_to_stringify_errors_when_full_type_graph_is_discarded +FrontendTest.no_use_after_free_with_type_fun_instantiation +FrontendTest.nocheck_cycle_used_by_checked +FrontendTest.nocheck_modules_are_typed +FrontendTest.produce_errors_for_unchanged_file_with_a_syntax_error +FrontendTest.produce_errors_for_unchanged_file_with_errors +FrontendTest.re_report_type_error_in_required_file +FrontendTest.real_source +FrontendTest.recheck_if_dependent_script_is_dirty +FrontendTest.report_require_to_nonexistent_file +FrontendTest.report_syntax_error_in_required_file +FrontendTest.reports_errors_from_multiple_sources +FrontendTest.stats_are_not_reset_between_checks +FrontendTest.test_lint_uses_correct_config +FrontendTest.test_pruneParentSegments +FrontendTest.trace_requires_in_nonstrict_mode +FrontendTest.typecheck_twice_for_ast_types +isSubtype.functions_and_any +isSubtype.intersection_of_functions_of_different_arities +isSubtype.intersection_of_tables +isSubtype.table_with_any_prop +isSubtype.table_with_table_prop +isSubtype.tables +Linter.BuiltinGlobalWrite +Linter.DeprecatedApi +Linter.LocalShadowGlobal +Linter.TableOperations +Linter.use_all_parent_scopes_for_globals +ModuleTests.any_persistance_does_not_leak +ModuleTests.builtin_types_point_into_globalTypes_arena +ModuleTests.clone_self_property +ModuleTests.deepClone_cyclic_table +NonstrictModeTests.delay_function_does_not_require_its_argument_to_return_anything +NonstrictModeTests.for_in_iterator_variables_are_any +NonstrictModeTests.function_parameters_are_any +NonstrictModeTests.inconsistent_module_return_types_are_ok +NonstrictModeTests.inconsistent_return_types_are_ok +NonstrictModeTests.infer_nullary_function +NonstrictModeTests.infer_the_maximum_number_of_values_the_function_could_return +NonstrictModeTests.inline_table_props_are_also_any +NonstrictModeTests.local_tables_are_not_any +NonstrictModeTests.locals_are_any_by_default +NonstrictModeTests.offer_a_hint_if_you_use_a_dot_instead_of_a_colon +NonstrictModeTests.parameters_having_type_any_are_optional +NonstrictModeTests.returning_insufficient_return_values +NonstrictModeTests.returning_too_many_values +NonstrictModeTests.table_dot_insert_and_recursive_calls +NonstrictModeTests.table_props_are_any +Normalize.any_wins_the_battle_over_unknown_in_unions +Normalize.constrained_intersection_of_intersections +Normalize.cyclic_intersection +Normalize.cyclic_table_is_marked_normal +Normalize.cyclic_table_is_not_marked_normal +Normalize.cyclic_table_normalizes_sensibly +Normalize.cyclic_union +Normalize.fuzz_failure_bound_type_is_normal_but_not_its_bounded_to +Normalize.fuzz_failure_instersection_combine_must_follow +Normalize.higher_order_function +Normalize.intersection_combine_on_bound_self +Normalize.intersection_inside_a_table_inside_another_intersection +Normalize.intersection_inside_a_table_inside_another_intersection_2 +Normalize.intersection_inside_a_table_inside_another_intersection_3 +Normalize.intersection_inside_a_table_inside_another_intersection_4 +Normalize.intersection_of_confluent_overlapping_tables +Normalize.intersection_of_disjoint_tables +Normalize.intersection_of_functions +Normalize.intersection_of_overlapping_tables +Normalize.intersection_of_tables_with_indexers +Normalize.nested_table_normalization_with_non_table__no_ice +Normalize.normalization_does_not_convert_ever +Normalize.normalize_module_return_type +Normalize.normalize_unions_containing_never +Normalize.normalize_unions_containing_unknown +Normalize.return_type_is_not_a_constrained_intersection +Normalize.skip_force_normal_on_external_types +Normalize.union_of_distinct_free_types +Normalize.variadic_tail_is_marked_normal +Normalize.visiting_a_type_twice_is_not_considered_normal +ParseErrorRecovery.empty_function_type_error_recovery +ParseErrorRecovery.extra_table_indexer_recovery +ParseErrorRecovery.extra_token_in_consume +ParseErrorRecovery.extra_token_in_consume_match +ParseErrorRecovery.extra_token_in_consume_match_end +ParseErrorRecovery.generic_type_list_recovery +ParseErrorRecovery.multiple_parse_errors +ParseErrorRecovery.recovery_of_parenthesized_expressions +ParseErrorRecovery.statement_error_recovery_expected +ParseErrorRecovery.statement_error_recovery_unexpected +ParserTests.break_return_not_last_error +ParserTests.continue_not_last_error +ParserTests.error_on_confusable +ParserTests.error_on_non_utf8_sequence +ParserTests.error_on_unicode +ParserTests.export_is_an_identifier_only_when_followed_by_type +ParserTests.functions_cannot_have_return_annotations_if_extensions_are_disabled +ParserTests.illegal_type_alias_if_extensions_are_disabled +ParserTests.incomplete_statement_error +ParserTests.local_cannot_have_annotation_with_extensions_disabled +ParserTests.parse_compound_assignment_error_call +ParserTests.parse_compound_assignment_error_multiple +ParserTests.parse_compound_assignment_error_not_lvalue +ParserTests.parse_error_function_call +ParserTests.parse_error_function_call_newline +ParserTests.parse_error_messages +ParserTests.parse_error_table_literal +ParserTests.parse_error_type_name +ParserTests.parse_nesting_based_end_detection +ParserTests.parse_nesting_based_end_detection_failsafe_earlier +ParserTests.parse_nesting_based_end_detection_local_function +ParserTests.parse_nesting_based_end_detection_local_repeat +ParserTests.parse_nesting_based_end_detection_nested +ParserTests.parse_nesting_based_end_detection_single_line +ParserTests.parse_numbers_error +ParserTests.parse_numbers_range_error +ParserTests.stop_if_line_ends_with_hyphen +ParserTests.type_alias_error_messages +RuntimeLimits.typescript_port_of_Result_type +ToDot.bound_table +ToDot.class +ToDot.function +ToDot.metatable +ToDot.primitive +ToDot.table +ToString.exhaustive_toString_of_cyclic_table +ToString.function_type_with_argument_names +ToString.function_type_with_argument_names_and_self +ToString.function_type_with_argument_names_generic +ToString.named_metatable_toStringNamedFunction +ToString.no_parentheses_around_cyclic_function_type_in_union +ToString.toStringDetailed2 +ToString.toStringErrorPack +ToString.toStringNamedFunction_generic_pack +ToString.toStringNamedFunction_hide_type_params +ToString.toStringNamedFunction_id +ToString.toStringNamedFunction_map +ToString.toStringNamedFunction_overrides_param_names +ToString.toStringNamedFunction_variadics +TranspilerTests.attach_types +TranspilerTests.type_lists_should_be_emitted_correctly +TranspilerTests.types_should_not_be_considered_cyclic_if_they_are_not_recursive +TypeAliases.basic_alias +TypeAliases.cannot_steal_hoisted_type_alias +TypeAliases.cli_38393_recursive_intersection_oom +TypeAliases.corecursive_function_types +TypeAliases.corecursive_types_generic +TypeAliases.cyclic_function_type_in_type_alias +TypeAliases.cyclic_types_of_named_table_fields_do_not_expand_when_stringified +TypeAliases.do_not_quantify_unresolved_aliases +TypeAliases.dont_stop_typechecking_after_reporting_duplicate_type_definition +TypeAliases.export_type_and_type_alias_are_duplicates +TypeAliases.forward_declared_alias_is_not_clobbered_by_prior_unification_with_any +TypeAliases.forward_declared_alias_is_not_clobbered_by_prior_unification_with_any_2 +TypeAliases.free_variables_from_typeof_in_aliases +TypeAliases.general_require_multi_assign +TypeAliases.generic_param_remap +TypeAliases.generic_typevars_are_not_considered_to_escape_their_scope_if_they_are_reused_in_multiple_aliases +TypeAliases.module_export_free_type_leak +TypeAliases.module_export_wrapped_free_type_leak +TypeAliases.mutually_recursive_aliases +TypeAliases.mutually_recursive_generic_aliases +TypeAliases.mutually_recursive_types_errors +TypeAliases.mutually_recursive_types_restriction_not_ok_1 +TypeAliases.mutually_recursive_types_restriction_not_ok_2 +TypeAliases.mutually_recursive_types_restriction_ok +TypeAliases.mutually_recursive_types_swapsies_not_ok +TypeAliases.mutually_recursive_types_swapsies_ok +TypeAliases.names_are_ascribed +TypeAliases.non_recursive_aliases_that_reuse_a_generic_name +TypeAliases.recursive_types_restriction_not_ok +TypeAliases.recursive_types_restriction_ok +TypeAliases.reported_location_is_correct_when_type_alias_are_duplicates +TypeAliases.stringify_optional_parameterized_alias +TypeAliases.stringify_type_alias_of_recursive_template_table_type +TypeAliases.stringify_type_alias_of_recursive_template_table_type2 +TypeAliases.type_alias_fwd_declaration_is_precise +TypeAliases.type_alias_import_mutation +TypeAliases.type_alias_local_mutation +TypeAliases.type_alias_local_rename +TypeAliases.type_alias_local_synthetic_mutation +TypeAliases.type_alias_of_an_imported_recursive_generic_type +TypeAliases.type_alias_of_an_imported_recursive_type +TypeAliases.use_table_name_and_generic_params_in_errors +TypeInferAnyError.any_type_propagates +TypeInferAnyError.assign_prop_to_table_by_calling_any_yields_any +TypeInferAnyError.call_to_any_yields_any +TypeInferAnyError.calling_error_type_yields_error +TypeInferAnyError.can_get_length_of_any +TypeInferAnyError.can_subscript_any +TypeInferAnyError.CheckMethodsOfAny +TypeInferAnyError.for_in_loop_iterator_is_any +TypeInferAnyError.for_in_loop_iterator_is_any2 +TypeInferAnyError.for_in_loop_iterator_is_error +TypeInferAnyError.for_in_loop_iterator_is_error2 +TypeInferAnyError.for_in_loop_iterator_returns_any +TypeInferAnyError.for_in_loop_iterator_returns_any2 +TypeInferAnyError.indexing_error_type_does_not_produce_an_error +TypeInferAnyError.length_of_error_type_does_not_produce_an_error +TypeInferAnyError.metatable_of_any_can_be_a_table +TypeInferAnyError.prop_access_on_any_with_other_options +TypeInferAnyError.quantify_any_does_not_bind_to_itself +TypeInferAnyError.replace_every_free_type_when_unifying_a_complex_function_with_any +TypeInferAnyError.type_error_addition +TypeInferClasses.assign_to_prop_of_class +TypeInferClasses.call_base_method +TypeInferClasses.call_instance_method +TypeInferClasses.call_method_of_a_child_class +TypeInferClasses.call_method_of_a_class +TypeInferClasses.can_assign_to_prop_of_base_class +TypeInferClasses.can_assign_to_prop_of_base_class_using_string +TypeInferClasses.can_read_prop_of_base_class +TypeInferClasses.can_read_prop_of_base_class_using_string +TypeInferClasses.cannot_call_method_of_child_on_base_instance +TypeInferClasses.cannot_call_unknown_method_of_a_class +TypeInferClasses.cannot_unify_class_instance_with_primitive +TypeInferClasses.class_type_mismatch_with_name_conflict +TypeInferClasses.class_unification_type_mismatch_is_correct_order +TypeInferClasses.classes_can_have_overloaded_operators +TypeInferClasses.classes_without_overloaded_operators_cannot_be_added +TypeInferClasses.detailed_class_unification_error +TypeInferClasses.function_arguments_are_covariant +TypeInferClasses.higher_order_function_arguments_are_contravariant +TypeInferClasses.higher_order_function_return_type_is_not_contravariant +TypeInferClasses.higher_order_function_return_values_are_covariant +TypeInferClasses.optional_class_field_access_error +TypeInferClasses.table_class_unification_reports_sane_errors_for_missing_properties +TypeInferClasses.table_indexers_are_invariant +TypeInferClasses.table_properties_are_invariant +TypeInferClasses.warn_when_prop_almost_matches +TypeInferClasses.we_can_infer_that_a_parameter_must_be_a_particular_class +TypeInferClasses.we_can_report_when_someone_is_trying_to_use_a_table_rather_than_a_class +TypeInferFunctions.another_indirect_function_case_where_it_is_ok_to_provide_too_many_arguments +TypeInferFunctions.another_recursive_local_function +TypeInferFunctions.cannot_hoist_interior_defns_into_signature +TypeInferFunctions.check_function_before_lambda_that_uses_it +TypeInferFunctions.complicated_return_types_require_an_explicit_annotation +TypeInferFunctions.cyclic_function_type_in_args +TypeInferFunctions.dont_give_other_overloads_message_if_only_one_argument_matching_overload_exists +TypeInferFunctions.duplicate_functions_with_different_signatures_not_allowed_in_nonstrict +TypeInferFunctions.first_argument_can_be_optional +TypeInferFunctions.func_expr_doesnt_leak_free +TypeInferFunctions.higher_order_function_2 +TypeInferFunctions.higher_order_function_4 +TypeInferFunctions.infer_return_type_from_selected_overload +TypeInferFunctions.infer_that_function_does_not_return_a_table +TypeInferFunctions.it_is_ok_not_to_supply_enough_retvals +TypeInferFunctions.it_is_ok_to_oversaturate_a_higher_order_function_argument +TypeInferFunctions.list_all_overloads_if_no_overload_takes_given_argument_count +TypeInferFunctions.list_only_alternative_overloads_that_match_argument_count +TypeInferFunctions.mutual_recursion +TypeInferFunctions.recursive_function +TypeInferFunctions.recursive_local_function +TypeInferFunctions.too_many_arguments +TypeInferFunctions.toposort_doesnt_break_mutual_recursion +TypeInferFunctions.vararg_function_is_quantified +TypeInferFunctions.vararg_functions_should_allow_calls_of_any_types_and_size diff --git a/tools/natvis/Analysis.natvis b/tools/natvis/Analysis.natvis index b9ea3141..7d03dd3f 100644 --- a/tools/natvis/Analysis.natvis +++ b/tools/natvis/Analysis.natvis @@ -38,6 +38,38 @@ {{ typeId=29, value={*($T30*)storage} }} {{ typeId=30, value={*($T31*)storage} }} {{ typeId=31, value={*($T32*)storage} }} + {{ typeId=32, value={*($T33*)storage} }} + {{ typeId=33, value={*($T34*)storage} }} + {{ typeId=34, value={*($T35*)storage} }} + {{ typeId=35, value={*($T36*)storage} }} + {{ typeId=36, value={*($T37*)storage} }} + {{ typeId=37, value={*($T38*)storage} }} + {{ typeId=38, value={*($T39*)storage} }} + {{ typeId=39, value={*($T40*)storage} }} + {{ typeId=40, value={*($T41*)storage} }} + {{ typeId=41, value={*($T42*)storage} }} + {{ typeId=42, value={*($T43*)storage} }} + {{ typeId=43, value={*($T44*)storage} }} + {{ typeId=44, value={*($T45*)storage} }} + {{ typeId=45, value={*($T46*)storage} }} + {{ typeId=46, value={*($T47*)storage} }} + {{ typeId=47, value={*($T48*)storage} }} + {{ typeId=48, value={*($T49*)storage} }} + {{ typeId=49, value={*($T50*)storage} }} + {{ typeId=50, value={*($T51*)storage} }} + {{ typeId=51, value={*($T52*)storage} }} + {{ typeId=52, value={*($T53*)storage} }} + {{ typeId=53, value={*($T54*)storage} }} + {{ typeId=54, value={*($T55*)storage} }} + {{ typeId=55, value={*($T56*)storage} }} + {{ typeId=56, value={*($T57*)storage} }} + {{ typeId=57, value={*($T58*)storage} }} + {{ typeId=58, value={*($T59*)storage} }} + {{ typeId=59, value={*($T60*)storage} }} + {{ typeId=60, value={*($T61*)storage} }} + {{ typeId=61, value={*($T62*)storage} }} + {{ typeId=62, value={*($T63*)storage} }} + {{ typeId=63, value={*($T64*)storage} }} typeId *($T1*)storage @@ -72,6 +104,38 @@ *($T30*)storage *($T31*)storage *($T32*)storage + *($T33*)storage + *($T34*)storage + *($T35*)storage + *($T36*)storage + *($T37*)storage + *($T38*)storage + *($T39*)storage + *($T40*)storage + *($T41*)storage + *($T42*)storage + *($T43*)storage + *($T44*)storage + *($T45*)storage + *($T46*)storage + *($T47*)storage + *($T48*)storage + *($T49*)storage + *($T50*)storage + *($T51*)storage + *($T52*)storage + *($T53*)storage + *($T54*)storage + *($T55*)storage + *($T56*)storage + *($T57*)storage + *($T58*)storage + *($T59*)storage + *($T60*)storage + *($T61*)storage + *($T62*)storage + *($T63*)storage + *($T64*)storage diff --git a/tools/test_dcr.py b/tools/test_dcr.py new file mode 100644 index 00000000..8b090bcd --- /dev/null +++ b/tools/test_dcr.py @@ -0,0 +1,124 @@ +# This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details + +import argparse +import os.path +import subprocess as sp +import sys +import xml.sax as x + +SCRIPT_PATH = os.path.split(sys.argv[0])[0] +FAIL_LIST_PATH = os.path.join(SCRIPT_PATH, "faillist.txt") + + +def loadFailList(): + with open(FAIL_LIST_PATH) as f: + return set(map(str.strip, f.readlines())) + + +class Handler(x.ContentHandler): + def __init__(self, failList): + self.currentTest = [] + self.failList = failList # Set of dotted test names that are expected to fail + + self.results = {} # {DottedName: TrueIfTheTestPassed} + + def startElement(self, name, attrs): + if name == "TestSuite": + self.currentTest.append(attrs["name"]) + elif name == "TestCase": + self.currentTest.append(attrs["name"]) + + elif name == "OverallResultsAsserts": + if self.currentTest: + try: + failed = 0 != int(attrs["failures"]) + except ValueError: + failed = False + + dottedName = ".".join(self.currentTest) + shouldFail = dottedName in self.failList + + if failed and not shouldFail: + print("UNEXPECTED: {} should have passed".format(dottedName)) + elif not failed and shouldFail: + print("UNEXPECTED: {} should have failed".format(dottedName)) + + self.results[dottedName] = not failed + + def endElement(self, name): + if name == "TestCase": + self.currentTest.pop() + + elif name == "TestSuite": + self.currentTest.pop() + + +def main(): + parser = argparse.ArgumentParser( + description="Run Luau.UnitTest with deferred constraint resolution enabled" + ) + parser.add_argument( + "path", action="store", help="Path to the Luau.UnitTest executable" + ) + parser.add_argument( + "--dump", + dest="dump", + action="store_true", + help="Instead of doing any processing, dump the raw output of the test run. Useful for debugging this tool.", + ) + parser.add_argument( + "--write", + dest="write", + action="store_true", + help="Write a new faillist.txt after running tests.", + ) + + args = parser.parse_args() + + failList = loadFailList() + + p = sp.Popen( + [ + args.path, + "--reporters=xml", + "--fflags=true,DebugLuauDeferredConstraintResolution=true", + ], + stdout=sp.PIPE, + ) + + handler = Handler(failList) + + if args.dump: + for line in p.stdout: + sys.stdout.buffer.write(line) + return + else: + x.parse(p.stdout, handler) + + p.wait() + + if args.write: + newFailList = sorted( + ( + dottedName + for dottedName, passed in handler.results.items() + if not passed + ), + key=str.lower, + ) + with open(FAIL_LIST_PATH, "w", newline="\n") as f: + for name in newFailList: + print(name, file=f) + print("Updated faillist.txt") + + sys.exit( + 0 + if all( + not passed == (dottedName in failList) + for dottedName, passed in handler.results.items() + ) + else 1 + ) + +if __name__ == "__main__": + main() From 9be7f85be7ec227d63b150adfcdda9164fdc119c Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Fri, 22 Jul 2022 07:53:16 -0700 Subject: [PATCH 319/399] Update performance.md (#608) Mention constant folding for builtins and remove the note about possibly doing inlining in the future because we do do it now! --- docs/_pages/performance.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/docs/_pages/performance.md b/docs/_pages/performance.md index 34b24b03..c6ef2ad0 100644 --- a/docs/_pages/performance.md +++ b/docs/_pages/performance.md @@ -29,7 +29,7 @@ Unlike Lua and LuaJIT, Luau uses a multi-pass compiler with a frontend that pars > Note: Compilation throughput isn't the main focus in Luau, but our compiler is reasonably fast; with all currently implemented optimizations enabled, it compiles 400K lines of Luau code in 0.5 seconds on a single core of a desktop Core i7 CPU, producing bytecode and debug information. -While bytecode optimizations are limited due to the flexibility of Luau code (e.g. `a * 1` may not be equivalent to `a` if `*` is overloaded through metatables), even in absence of type information Luau compiler can perform some optimizations such as "deep" constant folding across functions and local variables, perform upvalue optimizations for upvalues that aren't mutated, do analysis of builtin function usage, and some peephole optimizations on the resulting bytecode. In the future we plan to do bytecode-level inlining and possibly other code transformation. +While bytecode optimizations are limited due to the flexibility of Luau code (e.g. `a * 1` may not be equivalent to `a` if `*` is overloaded through metatables), even in absence of type information Luau compiler can perform some optimizations such as "deep" constant folding across functions and local variables, perform upvalue optimizations for upvalues that aren't mutated, do analysis of builtin function usage, and some peephole optimizations on the resulting bytecode. The compiler can also be instructed to use more aggressive optimizations by enabling optimization level 2 (`-O2` in CLI tools), some of which are documented further on this page. Luau compiler currently doesn't use type information to do further optimizations, however early experiments suggest that we can extract further wins. Because we control the entire stack (unlike e.g. TypeScript where the type information is discarded completely before reaching the VM), we have more flexibility there and can make some tradeoffs during codegen even if the type system isn't completely sound. For example, it might be reasonable to assume that in presence of known types, we can infer absence of side effects for arithmetic operations and builtins - if the runtime types mismatch due to intentional violation of the type safety through global injection, the code will still be safely sandboxed; this may unlock optimizations such as common subexpression elimination and allocation hoisting without a JIT. This is speculative pending further research. @@ -90,6 +90,8 @@ As a result, builtin calls are very fast in Luau - they are still slightly slowe > Note: The partial specialization mechanism is cute in that for `assert`, it only specializes on truthy conditions; hopefully performance of `assert(false)` isn't crucial for most code! +In addition to runtime optimizations for builtin calls, many builtin calls can also be constant-folded by the bytecode compiler when using aggressive optimizations (level 2); this currently applies to most builtin calls with constant arguments and a single return value. + ## Optimized table iteration Luau implements a fully generic iteration protocol; however, for iteration through tables in addition to generalized iteration (`for .. in t`) it recognizes three common idioms (`for .. in ipairs(t)`, `for .. in pairs(t)` and `for .. in next, t`) and emits specialized bytecode that is carefully optimized using custom internal iterators. From 12c550202719b5deef932453728f5643bed23eaa Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Fri, 22 Jul 2022 10:35:03 -0700 Subject: [PATCH 320/399] Update performance.md Update compiler performance metrics to account for O2 and expanding internal codebase --- docs/_pages/performance.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/_pages/performance.md b/docs/_pages/performance.md index c6ef2ad0..73c7f606 100644 --- a/docs/_pages/performance.md +++ b/docs/_pages/performance.md @@ -27,7 +27,7 @@ Of course the interpreter isn't typical C code - it uses many tricks to achieve Unlike Lua and LuaJIT, Luau uses a multi-pass compiler with a frontend that parses source into an AST and a backend that generates bytecode from it. This carries a small penalty in terms of compilation time, but results in more flexible code and, crucially, makes it easier to optimize the generated bytecode. -> Note: Compilation throughput isn't the main focus in Luau, but our compiler is reasonably fast; with all currently implemented optimizations enabled, it compiles 400K lines of Luau code in 0.5 seconds on a single core of a desktop Core i7 CPU, producing bytecode and debug information. +> Note: Compilation throughput isn't the main focus in Luau, but our compiler is reasonably fast; with all currently implemented optimizations enabled, it compiles 950K lines of Luau code in 1 second on a single core of a desktop Ryzen 5900X CPU, producing bytecode and debug information. While bytecode optimizations are limited due to the flexibility of Luau code (e.g. `a * 1` may not be equivalent to `a` if `*` is overloaded through metatables), even in absence of type information Luau compiler can perform some optimizations such as "deep" constant folding across functions and local variables, perform upvalue optimizations for upvalues that aren't mutated, do analysis of builtin function usage, and some peephole optimizations on the resulting bytecode. The compiler can also be instructed to use more aggressive optimizations by enabling optimization level 2 (`-O2` in CLI tools), some of which are documented further on this page. From 2a6d1c03ace747824510b741094852a8c4b02d98 Mon Sep 17 00:00:00 2001 From: JohnnyMorganz Date: Mon, 25 Jul 2022 18:19:21 +0100 Subject: [PATCH 321/399] Store AstExprFunction in astTypes for stats (#612) --- Analysis/src/TypeInfer.cpp | 3 +++ 1 file changed, 3 insertions(+) diff --git a/Analysis/src/TypeInfer.cpp b/Analysis/src/TypeInfer.cpp index ea7d81e1..d460160c 100644 --- a/Analysis/src/TypeInfer.cpp +++ b/Analysis/src/TypeInfer.cpp @@ -3640,6 +3640,9 @@ void TypeChecker::checkFunctionBody(const ScopePtr& scope, TypeId ty, const AstE reportError(getEndLocation(function), FunctionExitsWithoutReturning{funTy->retTypes}); } } + + if (!currentModule->astTypes.find(&function)) + currentModule->astTypes[&function] = ty; } else ice("Checking non functional type"); From 185370fa1d24f8e8d12d37f9bd3b38c0e5d8f96c Mon Sep 17 00:00:00 2001 From: Alexander McCord <11488393+alexmccord@users.noreply.github.com> Date: Thu, 28 Jul 2022 14:48:05 -0700 Subject: [PATCH 322/399] RFC: Update the rules on string interpolation in light of feedback (#615) We make four adjustments in this RFC: 1. `{{` is not allowed. This is likely a valid but poor attempt at escaping coming from C#, Rust, or Python. 2. We now allow `` `this` `` with zero interpolating expressions. 3. We now allow `` f `this` `` also. 4. Explicitly say that `` `this` `` and `` `this {that}` `` are not valid type annotation syntax. --- rfcs/syntax-string-interpolation.md | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/rfcs/syntax-string-interpolation.md b/rfcs/syntax-string-interpolation.md index 208143a0..ad182620 100644 --- a/rfcs/syntax-string-interpolation.md +++ b/rfcs/syntax-string-interpolation.md @@ -31,9 +31,9 @@ Because we care about backward compatibility, we need some new syntax in order t 1. A string chunk (`` `...{ ``, `}...{`, and `` }...` ``) where `...` is a range of 0 to many characters. * `\` escapes `` ` ``, `{`, and itself `\`. - * Restriction: the string interpolation literal must have at least one value to interpolate. We do not need 3 ways to express a single line string literal. * The pairs must be on the same line (unless a `\` escapes the newline) but expressions needn't be on the same line. 2. An expression between the braces. This is the value that will be interpolated into the string. + * Restriction: we explicitly reject `{{` as it is considered an attempt to escape and get a single `{` character at runtime. 3. Formatting specification may follow after the expression, delimited by an unambiguous character. * Restriction: the formatting specification must be constant at parse time. * In the absence of an explicit formatting specification, the `%*` token will be used. @@ -61,7 +61,6 @@ local set2 = Set.new({0, 5, 4}) print(`{set1} ∪ {set2} = {Set.union(set1, set2)}`) --> {0, 1, 3} ∪ {0, 5, 4} = {0, 1, 3, 4, 5} --- For illustrative purposes. These are illegal specifically because they don't interpolate anything. print(`Some example escaping the braces \{like so}`) print(`backslash \ that escapes the space is not a part of the string...`) print(`backslash \\ will escape the second backslash...`) @@ -88,13 +87,25 @@ print(`Welcome to \ -- Luau! ``` -This expression will not be allowed to come after a `prefixexp`. I believe this is fully additive, so a future RFC may allow this. So for now, we explicitly reject the following: +This expression can also come after a `prefixexp`: ``` local name = "world" print`Hello {name}` ``` +The restriction on `{{` exists solely for the people coming from languages e.g. C#, Rust, or Python which uses `{{` to escape and get the character `{` at runtime. We're also rejecting this at parse time too, since the proper way to escape it is `\{`, so: + +```lua +print(`{{1, 2, 3}} = {myCoolSet}`) -- parse error +``` + +If we did not apply this as a parse error, then the above would wind up printing as the following, which is obviously a gotcha we can and should avoid. + +``` +--> table: 0xSOMEADDRESS = {1, 2, 3} +``` + Since the string interpolation expression is going to be lowered into a `string.format` call, we'll also need to extend `string.format`. The bare minimum to support the lowering is to add a new token whose definition is to perform a `tostring` call. `%*` is currently an invalid token, so this is a backward compatible extension. This RFC shall define `%*` to have the same behavior as if `tostring` was called. ```lua @@ -121,6 +132,13 @@ print(string.format("%* %* %*", return_two_nils())) --> error: value #3 is missing, got 2 ``` +It must be said that we are not allowing this style of string literals in type annotations at this time, regardless of zero or many interpolating expressions, so the following two type annotations below are illegal syntax: + +```lua +local foo: `foo` +local bar: `bar{baz}` +``` + ## Drawbacks If we want to use backticks for other purposes, it may introduce some potential ambiguity. One option to solve that is to only ever produce string interpolation tokens from the context of an expression. This is messy but doable because the parser and the lexer are already implemented to work in tandem. The other option is to pick a different delimiter syntax to keep backticks available for use in the future. From cb165556085607146a624a8e4b89f4d393a077be Mon Sep 17 00:00:00 2001 From: Alexander McCord <11488393+alexmccord@users.noreply.github.com> Date: Thu, 28 Jul 2022 14:48:17 -0700 Subject: [PATCH 323/399] RFC: Disallow `name T` and `name(T)` in future syntactic extensions for type annotations (#589) --- ...oposals-leading-to-ambiguity-in-grammar.md | 129 ++++++++++++++++++ 1 file changed, 129 insertions(+) create mode 100644 rfcs/disallow-proposals-leading-to-ambiguity-in-grammar.md diff --git a/rfcs/disallow-proposals-leading-to-ambiguity-in-grammar.md b/rfcs/disallow-proposals-leading-to-ambiguity-in-grammar.md new file mode 100644 index 00000000..d9c5c7d7 --- /dev/null +++ b/rfcs/disallow-proposals-leading-to-ambiguity-in-grammar.md @@ -0,0 +1,129 @@ +# Disallow `name T` and `name(T)` in future syntactic extensions for type annotations + +## Summary + +We propose to disallow the syntax `` `('`` as well as ` ` in future syntax extensions for type annotations to ensure that all existing programs continue to parse correctly. This still keeps the door open for future syntax extensions of different forms such as `` `<' `>'``. + +## Motivation + +Lua and by extension Luau's syntax is very free form, which means that when the parser finishes parsing a node, it doesn't try to look for a semi-colon or any termination token e.g. a `{` to start a block, or `;` to end a statement, or a newline, etc. It just immediately invokes the next parser to figure out how to parse the next node based on the remainder's starting token. + +That feature is sometimes quite troublesome when we want to add new syntax. + +We have had cases where we talked about using syntax like `setmetatable(T, MT)` and `keyof T`. They all look innocent, but when you look beyond that, and try to apply it onto Luau's grammar, things break down really fast. + +### `F(T)`? + +An example that _will_ cause a change in semantics: + +``` +local t: F +(u):m() +``` + +where today, `local t: F` is one statement, and `(u):m()` is another. If we had the syntax for `F(T)` here, it becomes invalid input because it gets parsed as + +``` +local t: F(u) +:m() +``` + +This is important because of the `setmetatable(T, MT)` case: + +``` +type Foo = setmetatable({ x: number }, { ... }) +``` + +For `setmetatable`, the parser isn't sure whether `{}` is actually a type or an expression, because _today_ `setmetatable` is parsed as a type reference, and `({}, {})` is the remainder that we'll attempt to parse as a statement. This means `{ x: number }` is invalid table _literal_. Recovery by backtracking is technically possible here, but this means performance loss on invalid input + may introduce false positives wrt how things are parsed. We'd much rather take a very strict stance about how things get parsed. + +### `F T`? + +An example that _will_ cause a change in semantics: + +``` +local function f(t): F T + (t or u):m() +end +``` + +where today, the return type annotation `F T` is simply parsed as just `F`, followed by a ambiguous parse error from the statement `T(t or u)` because its `(` is on the next line. If at some point in the future we were to allow `T` followed by `(` on the next line, then there's yet another semantic change. `F T` could be parsed as a type annotation and the first statement is `(t or u):m()` instead of `F` followed by `T(t or u):m()`. + +For `keyof`, here's a practical example of the above issue: + +``` +type Vec2 = {x: number, y: number} + +local function f(t, u): keyof Vec2 + (t or u):m() +end +``` + +There's three possible outcomes: + 1. Return type of `f` is `keyof`, statement throws a parse error because `(` is on the next line after `Vec2`, + 2. Return type of `f` is `keyof Vec2` and next statement is `(t or u):m()`, or + 3. Return type of `f` is `keyof` and next statement is `Vec2(t or u):m()` (if we allow `(` on the next line to be part of previous line). + +This particular case is even worse when we keep going: + +``` +local function f(t): F + T(t or u):m() +end +``` + +``` +local function f(t): F T + {1, 2, 3} +end +``` + +where today, `F` is the return type annotation of `f`, and `T(t or u):m()`/`T{1, 2, 3}` is the first statement, respectively. + +Adding some syntax for `F T` **will** cause the parser to change the semantics of the above three examples. + +### But what about `typeof(...)`? + +This syntax is grandfathered in because the parser supported `typeof(...)` before we stabilized our syntax, and especially before type annotations were released to the public, so we didn't need to worry about compatibility here. We are very glad that we used parentheses in this case, because it's natural for expressions to belong within parentheses `()`, and types to belong within angles `<>`. + +## The One Exception with a caveat + +This is a strict requirement! + +`function() -> ()` has been talked about in the past, and this one is different despite falling under the same category as `` `('``. The token `function` is in actual fact a "hard keyword," meaning that it cannot be parsed as a type annotation because it is not an identifier, just a keyword. + +Likewise, we also have talked about adding standalone `function` as a type annotation (semantics of it is irrelevant for this RFC) + +It's possible that we may end up adding both, but the requirements are as such: + 1. `function() -> ()` must be added first before standalone `function`, OR + 2. `function` can be added first, but with a future-proofing parse error if `<` or `(` follows after it + +If #1 is what ends up happening, there's not much to worry about because the type annotation parser will parse greedily already, so any new valid input will remain valid and have same semantics, except it also allows omitting of `(` and `<`. + +If #2 is what ends up happening, there could be a problem if we didn't future-proof against `<` and `(` to follow `function`: + +``` + return f :: function(T) -> U +``` + +which would be a parse error because at the point of `(` we expect one of `until`, `end`, or `EOF`, and + +``` + return f :: function(a) -> a +``` + +which would also be a parse error by the time we reach `->`, that is the production of the above is semantically equivalent to `(f < a) > (a)` which would compare whether the value of `f` is less than the value of `a`, then whether the result of that value is greater than `a`. + +## Alternatives + +Only allow these syntax when used inside parentheses e.g. `(F T)` or `(F(T))`. This makes it inconsistent with the existing `typeof(...)` type annotation, and changing that over is also breaking change. + +Support backtracking in the parser, so if `: MyType(t or u):m()` is invalid syntax, revert and parse `MyType` as a type, and `(t or u):m()` as an expression statement. Even so, this option is terrible for: + 1. parsing performance (backtracking means losing progress on invalid input), + 2. user experience (why was this annotation parsed as `X(...)` instead of `X` followed by a statement `(...)`), + 3. has false positives (`foo(bar)(baz)` may be parsed as `foo(bar)` as the type annotation and `(baz)` is the remainder to parse) + +## Drawbacks + +To be able to expose some kind of type-level operations using `F` syntax, means one of the following must be chosen: + 1. introduce the concept of "magic type functions" into type inference, or + 2. introduce them into the prelude as `export type F = ...` (where `...` is to be read as "we haven't decided") From 3202869accbe4a3047becf8e70058fb1d9e015b3 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Thu, 28 Jul 2022 20:41:13 -0700 Subject: [PATCH 324/399] Sync to upstream/release/538 --- .gitignore | 1 + Analysis/include/Luau/Constraint.h | 5 +- .../include/Luau/ConstraintGraphBuilder.h | 85 +-- Analysis/include/Luau/ConstraintSolver.h | 6 +- .../include/Luau/ConstraintSolverLogger.h | 4 +- Analysis/include/Luau/Frontend.h | 4 +- Analysis/include/Luau/Module.h | 2 - Analysis/include/Luau/Quantify.h | 4 +- Analysis/include/Luau/Scope.h | 32 +- Analysis/include/Luau/TypeVar.h | 8 +- Analysis/include/Luau/Unifiable.h | 12 +- Analysis/src/Autocomplete.cpp | 51 +- Analysis/src/ConstraintGraphBuilder.cpp | 216 +++--- Analysis/src/ConstraintSolver.cpp | 14 +- Analysis/src/ConstraintSolverLogger.cpp | 12 +- Analysis/src/EmbeddedBuiltinDefinitions.cpp | 6 +- Analysis/src/Frontend.cpp | 22 +- Analysis/src/JsonEncoder.cpp | 4 +- Analysis/src/Module.cpp | 27 +- Analysis/src/Quantify.cpp | 20 +- Analysis/src/Scope.cpp | 36 +- Analysis/src/ToString.cpp | 66 +- Analysis/src/TypeChecker2.cpp | 17 +- Analysis/src/TypeInfer.cpp | 10 +- Analysis/src/Unifiable.cpp | 6 +- Ast/src/Parser.cpp | 49 +- CLI/Analyze.cpp | 2 - CLI/Ast.cpp | 2 - CLI/ReplEntry.cpp | 2 - Compiler/src/Compiler.cpp | 55 +- Compiler/src/CostModel.cpp | 4 +- Sources.cmake | 1 + VM/src/lapi.cpp | 3 - VM/src/ldblib.cpp | 13 +- VM/src/ldebug.cpp | 4 +- VM/src/lstring.cpp | 8 +- VM/src/lstrlib.cpp | 10 +- VM/src/ltable.cpp | 24 +- VM/src/ltable.h | 6 + VM/src/lvmexecute.cpp | 8 +- VM/src/lvmutils.cpp | 50 +- tests/Autocomplete.test.cpp | 54 +- tests/Compiler.test.cpp | 13 - tests/Conformance.test.cpp | 7 +- tests/ConstraintSolver.test.cpp | 8 +- tests/Fixture.cpp | 10 +- tests/Fixture.h | 2 +- tests/Lexer.test.cpp | 141 ++++ tests/Parser.test.cpp | 132 ---- tests/ToString.test.cpp | 31 +- tests/TypeInfer.anyerror.test.cpp | 22 +- tests/TypeInfer.builtins.test.cpp | 6 +- tests/TypeInfer.functions.test.cpp | 26 +- tests/TypeInfer.generics.test.cpp | 6 +- tests/TypeInfer.loops.test.cpp | 7 +- tests/TypeInfer.modules.test.cpp | 13 +- tests/TypeInfer.operators.test.cpp | 2 - tests/TypeInfer.primitives.test.cpp | 6 +- tests/TypeInfer.refinements.test.cpp | 6 +- tests/TypeInfer.test.cpp | 26 +- tests/TypeInfer.tryUnify.test.cpp | 12 +- tests/TypeInfer.unionTypes.test.cpp | 6 +- tests/TypePack.test.cpp | 2 +- tests/conformance/errors.lua | 3 +- tests/conformance/events.lua | 53 ++ tests/main.cpp | 4 +- tools/faillist.txt | 678 ++++++++++++++++-- tools/test_dcr.py | 19 +- 68 files changed, 1448 insertions(+), 758 deletions(-) create mode 100644 tests/Lexer.test.cpp diff --git a/.gitignore b/.gitignore index 5688dff5..ec5e6578 100644 --- a/.gitignore +++ b/.gitignore @@ -8,3 +8,4 @@ /default.prof* /fuzz-* /luau +__pycache__ diff --git a/Analysis/include/Luau/Constraint.h b/Analysis/include/Luau/Constraint.h index dcfb14b4..9911c4d3 100644 --- a/Analysis/include/Luau/Constraint.h +++ b/Analysis/include/Luau/Constraint.h @@ -12,7 +12,8 @@ namespace Luau { -struct Scope2; +struct Scope; + struct TypeVar; using TypeId = const TypeVar*; @@ -38,7 +39,7 @@ struct GeneralizationConstraint { TypeId generalizedType; TypeId sourceType; - Scope2* scope; + Scope* scope; }; // subType ~ inst superType diff --git a/Analysis/include/Luau/ConstraintGraphBuilder.h b/Analysis/include/Luau/ConstraintGraphBuilder.h index 513446fd..695b6cd5 100644 --- a/Analysis/include/Luau/ConstraintGraphBuilder.h +++ b/Analysis/include/Luau/ConstraintGraphBuilder.h @@ -17,21 +17,22 @@ namespace Luau { -struct Scope2; +struct Scope; +using ScopePtr = std::shared_ptr; struct ConstraintGraphBuilder { // A list of all the scopes in the module. This vector holds ownership of the // scope pointers; the scopes themselves borrow pointers to other scopes to // define the scope hierarchy. - std::vector>> scopes; + std::vector> scopes; ModuleName moduleName; SingletonTypes& singletonTypes; const NotNull arena; // The root scope of the module we're generating constraints for. // This is null when the CGB is initially constructed. - Scope2* rootScope; + Scope* rootScope; // A mapping of AST node to TypeId. DenseHashMap astTypes{nullptr}; // A mapping of AST node to TypePackId. @@ -50,42 +51,42 @@ struct ConstraintGraphBuilder // Occasionally constraint generation needs to produce an ICE. const NotNull ice; - NotNull globalScope; + NotNull globalScope; - ConstraintGraphBuilder(const ModuleName& moduleName, TypeArena* arena, NotNull ice, NotNull globalScope); + ConstraintGraphBuilder(const ModuleName& moduleName, TypeArena* arena, NotNull ice, NotNull globalScope); /** * Fabricates a new free type belonging to a given scope. * @param scope the scope the free type belongs to. */ - TypeId freshType(NotNull scope); + TypeId freshType(const ScopePtr& scope); /** * Fabricates a new free type pack belonging to a given scope. * @param scope the scope the free type pack belongs to. */ - TypePackId freshTypePack(NotNull scope); + TypePackId freshTypePack(const ScopePtr& scope); /** * Fabricates a scope that is a child of another scope. * @param location the lexical extent of the scope in the source code. * @param parent the parent scope of the new scope. Must not be null. */ - NotNull childScope(Location location, NotNull parent); + ScopePtr childScope(Location location, const ScopePtr& parent); /** * Adds a new constraint with no dependencies to a given scope. * @param scope the scope to add the constraint to. * @param cv the constraint variant to add. */ - void addConstraint(NotNull scope, ConstraintV cv); + void addConstraint(const ScopePtr& scope, ConstraintV cv); /** * Adds a constraint to a given scope. * @param scope the scope to add the constraint to. Must not be null. * @param c the constraint to add. */ - void addConstraint(NotNull scope, std::unique_ptr c); + void addConstraint(const ScopePtr& scope, std::unique_ptr c); /** * The entry point to the ConstraintGraphBuilder. This will construct a set @@ -94,23 +95,23 @@ struct ConstraintGraphBuilder */ void visit(AstStatBlock* block); - void visitBlockWithoutChildScope(NotNull scope, AstStatBlock* block); + void visitBlockWithoutChildScope(const ScopePtr& scope, AstStatBlock* block); - void visit(NotNull scope, AstStat* stat); - void visit(NotNull scope, AstStatBlock* block); - void visit(NotNull scope, AstStatLocal* local); - void visit(NotNull scope, AstStatFor* for_); - void visit(NotNull scope, AstStatLocalFunction* function); - void visit(NotNull scope, AstStatFunction* function); - void visit(NotNull scope, AstStatReturn* ret); - void visit(NotNull scope, AstStatAssign* assign); - void visit(NotNull scope, AstStatIf* ifStatement); - void visit(NotNull scope, AstStatTypeAlias* alias); + void visit(const ScopePtr& scope, AstStat* stat); + void visit(const ScopePtr& scope, AstStatBlock* block); + void visit(const ScopePtr& scope, AstStatLocal* local); + void visit(const ScopePtr& scope, AstStatFor* for_); + void visit(const ScopePtr& scope, AstStatLocalFunction* function); + void visit(const ScopePtr& scope, AstStatFunction* function); + void visit(const ScopePtr& scope, AstStatReturn* ret); + void visit(const ScopePtr& scope, AstStatAssign* assign); + void visit(const ScopePtr& scope, AstStatIf* ifStatement); + void visit(const ScopePtr& scope, AstStatTypeAlias* alias); - TypePackId checkExprList(NotNull scope, const AstArray& exprs); + TypePackId checkExprList(const ScopePtr& scope, const AstArray& exprs); - TypePackId checkPack(NotNull scope, AstArray exprs); - TypePackId checkPack(NotNull scope, AstExpr* expr); + TypePackId checkPack(const ScopePtr& scope, AstArray exprs); + TypePackId checkPack(const ScopePtr& scope, AstExpr* expr); /** * Checks an expression that is expected to evaluate to one type. @@ -118,13 +119,13 @@ struct ConstraintGraphBuilder * @param expr the expression to check. * @return the type of the expression. */ - TypeId check(NotNull scope, AstExpr* expr); + TypeId check(const ScopePtr& scope, AstExpr* expr); - TypeId checkExprTable(NotNull scope, AstExprTable* expr); - TypeId check(NotNull scope, AstExprIndexName* indexName); - TypeId check(NotNull scope, AstExprIndexExpr* indexExpr); - TypeId check(NotNull scope, AstExprUnary* unary); - TypeId check(NotNull scope, AstExprBinary* binary); + TypeId checkExprTable(const ScopePtr& scope, AstExprTable* expr); + TypeId check(const ScopePtr& scope, AstExprIndexName* indexName); + TypeId check(const ScopePtr& scope, AstExprIndexExpr* indexExpr); + TypeId check(const ScopePtr& scope, AstExprUnary* unary); + TypeId check(const ScopePtr& scope, AstExprBinary* binary); struct FunctionSignature { @@ -133,20 +134,20 @@ struct ConstraintGraphBuilder // The scope that encompasses the function's signature. May be nullptr // if there was no need for a signature scope (the function has no // generics). - Scope2* signatureScope; + ScopePtr signatureScope; // The scope that encompasses the function's body. Is a child scope of // signatureScope, if present. - NotNull bodyScope; + ScopePtr bodyScope; }; - FunctionSignature checkFunctionSignature(NotNull parent, AstExprFunction* fn); + FunctionSignature checkFunctionSignature(const ScopePtr& parent, AstExprFunction* fn); /** * Checks the body of a function expression. * @param scope the interior scope of the body of the function. * @param fn the function expression to check. */ - void checkFunctionBody(NotNull scope, AstExprFunction* fn); + void checkFunctionBody(const ScopePtr& scope, AstExprFunction* fn); /** * Resolves a type from its AST annotation. @@ -154,7 +155,7 @@ struct ConstraintGraphBuilder * @param ty the AST annotation to resolve. * @return the type of the AST annotation. **/ - TypeId resolveType(NotNull scope, AstType* ty); + TypeId resolveType(const ScopePtr& scope, AstType* ty); /** * Resolves a type pack from its AST annotation. @@ -162,14 +163,14 @@ struct ConstraintGraphBuilder * @param tp the AST annotation to resolve. * @return the type pack of the AST annotation. **/ - TypePackId resolveTypePack(NotNull scope, AstTypePack* tp); + TypePackId resolveTypePack(const ScopePtr& scope, AstTypePack* tp); - TypePackId resolveTypePack(NotNull scope, const AstTypeList& list); + TypePackId resolveTypePack(const ScopePtr& scope, const AstTypeList& list); - std::vector> createGenerics(NotNull scope, AstArray generics); - std::vector> createGenericPacks(NotNull scope, AstArray packs); + std::vector> createGenerics(const ScopePtr& scope, AstArray generics); + std::vector> createGenericPacks(const ScopePtr& scope, AstArray packs); - TypeId flattenPack(NotNull scope, Location location, TypePackId tp); + TypeId flattenPack(const ScopePtr& scope, Location location, TypePackId tp); void reportError(Location location, TypeErrorData err); void reportCodeTooComplex(Location location); @@ -180,7 +181,7 @@ struct ConstraintGraphBuilder * real" in a general way is going to be pretty hard, so we are choosing not to tackle that yet. For now, we do an * initial scan of the AST and note what globals are defined. */ - void prepopulateGlobalScope(NotNull globalScope, AstStatBlock* program); + void prepopulateGlobalScope(const ScopePtr& globalScope, AstStatBlock* program); }; /** @@ -193,6 +194,6 @@ struct ConstraintGraphBuilder * @return a list of pointers to constraints contained within the scope graph. * None of these pointers should be null. */ -std::vector> collectConstraints(NotNull rootScope); +std::vector> collectConstraints(NotNull rootScope); } // namespace Luau diff --git a/Analysis/include/Luau/ConstraintSolver.h b/Analysis/include/Luau/ConstraintSolver.h index cf88efb6..0b9b4ae3 100644 --- a/Analysis/include/Luau/ConstraintSolver.h +++ b/Analysis/include/Luau/ConstraintSolver.h @@ -25,7 +25,7 @@ struct ConstraintSolver // is important to not add elements to this vector, lest the underlying // storage that we retain pointers to be mutated underneath us. const std::vector> constraints; - NotNull rootScope; + NotNull rootScope; // This includes every constraint that has not been fully solved. // A constraint can be both blocked and unsolved, for instance. @@ -40,7 +40,7 @@ struct ConstraintSolver ConstraintSolverLogger logger; - explicit ConstraintSolver(TypeArena* arena, NotNull rootScope); + explicit ConstraintSolver(TypeArena* arena, NotNull rootScope); /** * Attempts to dispatch all pending constraints and reach a type solution @@ -121,6 +121,6 @@ private: void unblock_(BlockedConstraintId progressed); }; -void dump(NotNull rootScope, struct ToStringOptions& opts); +void dump(NotNull rootScope, struct ToStringOptions& opts); } // namespace Luau diff --git a/Analysis/include/Luau/ConstraintSolverLogger.h b/Analysis/include/Luau/ConstraintSolverLogger.h index 55336a23..fe2177c4 100644 --- a/Analysis/include/Luau/ConstraintSolverLogger.h +++ b/Analysis/include/Luau/ConstraintSolverLogger.h @@ -15,8 +15,8 @@ namespace Luau struct ConstraintSolverLogger { std::string compileOutput(); - void captureBoundarySnapshot(const Scope2* rootScope, std::vector>& unsolvedConstraints); - void prepareStepSnapshot(const Scope2* rootScope, NotNull current, std::vector>& unsolvedConstraints); + void captureBoundarySnapshot(const Scope* rootScope, std::vector>& unsolvedConstraints); + void prepareStepSnapshot(const Scope* rootScope, NotNull current, std::vector>& unsolvedConstraints); void commitPreparedStepSnapshot(); private: diff --git a/Analysis/include/Luau/Frontend.h b/Analysis/include/Luau/Frontend.h index 27fd4d7e..3b0164c2 100644 --- a/Analysis/include/Luau/Frontend.h +++ b/Analysis/include/Luau/Frontend.h @@ -152,7 +152,7 @@ struct Frontend void registerBuiltinDefinition(const std::string& name, std::function); void applyBuiltinDefinitionToEnvironment(const std::string& environmentName, const std::string& definitionName); - NotNull getGlobalScope2(); + NotNull getGlobalScope(); private: ModulePtr check(const SourceModule& sourceModule, Mode mode, const ScopePtr& environmentScope); @@ -169,7 +169,7 @@ private: std::unordered_map environments; std::unordered_map> builtinDefinitions; - std::unique_ptr globalScope2; + std::unique_ptr globalScope; public: FileResolver* fileResolver; diff --git a/Analysis/include/Luau/Module.h b/Analysis/include/Luau/Module.h index b3105b78..73f4c972 100644 --- a/Analysis/include/Luau/Module.h +++ b/Analysis/include/Luau/Module.h @@ -69,7 +69,6 @@ struct Module std::shared_ptr names; std::vector> scopes; // never empty - std::vector>> scope2s; // never empty DenseHashMap astTypes{nullptr}; DenseHashMap astTypePacks{nullptr}; @@ -86,7 +85,6 @@ struct Module bool timeout = false; ScopePtr getModuleScope() const; - Scope2* getModuleScope2() const; // Once a module has been typechecked, we clone its public interface into a separate arena. // This helps us to force TypeVar ownership into a DAG rather than a DCG. diff --git a/Analysis/include/Luau/Quantify.h b/Analysis/include/Luau/Quantify.h index f46f0cb5..7edf23b8 100644 --- a/Analysis/include/Luau/Quantify.h +++ b/Analysis/include/Luau/Quantify.h @@ -7,9 +7,9 @@ namespace Luau { struct TypeArena; -struct Scope2; +struct Scope; void quantify(TypeId ty, TypeLevel level); -TypeId quantify(TypeArena* arena, TypeId ty, Scope2* scope); +TypeId quantify(TypeArena* arena, TypeId ty, Scope* scope); } // namespace Luau diff --git a/Analysis/include/Luau/Scope.h b/Analysis/include/Luau/Scope.h index 0eaecf1d..85c1750a 100644 --- a/Analysis/include/Luau/Scope.h +++ b/Analysis/include/Luau/Scope.h @@ -32,10 +32,16 @@ struct Scope explicit Scope(const ScopePtr& parent, int subLevel = 0); // child scope. Parent must not be nullptr. const ScopePtr parent; // null for the root + + // All the children of this scope. + std::vector> children; std::unordered_map bindings; + std::unordered_map typeBindings; + std::unordered_map typePackBindings; TypePackId returnType; - bool breakOk = false; std::optional varargPack; + // All constraints belonging to this scope. + std::vector constraints; TypeLevel level; @@ -45,7 +51,9 @@ struct Scope std::unordered_map> importedTypeBindings; - std::optional lookup(const Symbol& name); + std::optional lookup(Symbol sym); + std::optional lookupTypeBinding(const Name& name); + std::optional lookupTypePackBinding(const Name& name); std::optional lookupType(const Name& name); std::optional lookupImportedType(const Name& moduleAlias, const Name& name); @@ -66,24 +74,4 @@ struct Scope std::unordered_map typeAliasTypePackParameters; }; -struct Scope2 -{ - // The parent scope of this scope. Null if there is no parent (i.e. this - // is the module-level scope). - Scope2* parent = nullptr; - // All the children of this scope. - std::vector> children; - std::unordered_map bindings; // TODO: I think this can be a DenseHashMap - std::unordered_map typeBindings; - std::unordered_map typePackBindings; - TypePackId returnType; - std::optional varargPack; - // All constraints belonging to this scope. - std::vector constraints; - - std::optional lookup(Symbol sym); - std::optional lookupTypeBinding(const Name& name); - std::optional lookupTypePackBinding(const Name& name); -}; - } // namespace Luau diff --git a/Analysis/include/Luau/TypeVar.h b/Analysis/include/Luau/TypeVar.h index fb6093df..052d4d88 100644 --- a/Analysis/include/Luau/TypeVar.h +++ b/Analysis/include/Luau/TypeVar.h @@ -24,7 +24,7 @@ namespace Luau { struct TypeArena; -struct Scope2; +struct Scope; /** * There are three kinds of type variables: @@ -143,7 +143,7 @@ struct ConstrainedTypeVar std::vector parts; TypeLevel level; - Scope2* scope = nullptr; + Scope* scope = nullptr; }; // Singleton types https://github.com/Roblox/luau/blob/master/rfcs/syntax-singleton-types.md @@ -275,7 +275,7 @@ struct FunctionTypeVar std::optional defn = {}, bool hasSelf = false); TypeLevel level; - Scope2* scope = nullptr; + Scope* scope = nullptr; /// These should all be generic std::vector generics; std::vector genericPacks; @@ -344,7 +344,7 @@ struct TableTypeVar TableState state = TableState::Unsealed; TypeLevel level; - Scope2* scope = nullptr; + Scope* scope = nullptr; std::optional name; // Sometimes we throw a type on a name to make for nicer error messages, but without creating any entry in the type namespace diff --git a/Analysis/include/Luau/Unifiable.h b/Analysis/include/Luau/Unifiable.h index 4ff91714..e5eb4198 100644 --- a/Analysis/include/Luau/Unifiable.h +++ b/Analysis/include/Luau/Unifiable.h @@ -8,7 +8,7 @@ namespace Luau { -struct Scope2; +struct Scope; /** * The 'level' of a TypeVar is an indirect way to talk about the scope that it 'belongs' too. @@ -84,11 +84,11 @@ using Name = std::string; struct Free { explicit Free(TypeLevel level); - explicit Free(Scope2* scope); + explicit Free(Scope* scope); int index; TypeLevel level; - Scope2* scope = nullptr; + Scope* scope = nullptr; // True if this free type variable is part of a mutually // recursive type alias whose definitions haven't been // resolved yet. @@ -115,13 +115,13 @@ struct Generic Generic(); explicit Generic(TypeLevel level); explicit Generic(const Name& name); - explicit Generic(Scope2* scope); + explicit Generic(Scope* scope); Generic(TypeLevel level, const Name& name); - Generic(Scope2* scope, const Name& name); + Generic(Scope* scope, const Name& name); int index; TypeLevel level; - Scope2* scope = nullptr; + Scope* scope = nullptr; Name name; bool explicitName = false; diff --git a/Analysis/src/Autocomplete.cpp b/Analysis/src/Autocomplete.cpp index 513a791c..a57a789f 100644 --- a/Analysis/src/Autocomplete.cpp +++ b/Analysis/src/Autocomplete.cpp @@ -12,7 +12,7 @@ #include #include -LUAU_FASTFLAG(LuauSelfCallAutocompleteFix2) +LUAU_FASTFLAG(LuauSelfCallAutocompleteFix3) static const std::unordered_set kStatementStartingKeywords = { "while", "if", "local", "repeat", "function", "do", "for", "return", "break", "continue", "type", "export"}; @@ -149,7 +149,7 @@ static TypeCorrectKind checkTypeCorrectKind(const Module& module, TypeArena* typ ty = follow(ty); auto canUnify = [&typeArena](TypeId subTy, TypeId superTy) { - LUAU_ASSERT(!FFlag::LuauSelfCallAutocompleteFix2); + LUAU_ASSERT(!FFlag::LuauSelfCallAutocompleteFix3); InternalErrorReporter iceReporter; UnifierSharedState unifierState(&iceReporter); @@ -168,7 +168,7 @@ static TypeCorrectKind checkTypeCorrectKind(const Module& module, TypeArena* typ TypeId expectedType = follow(*typeAtPosition); auto checkFunctionType = [typeArena, &canUnify, &expectedType](const FunctionTypeVar* ftv) { - if (FFlag::LuauSelfCallAutocompleteFix2) + if (FFlag::LuauSelfCallAutocompleteFix3) { if (std::optional firstRetTy = first(ftv->retTypes)) return checkTypeMatch(typeArena, *firstRetTy, expectedType); @@ -209,7 +209,7 @@ static TypeCorrectKind checkTypeCorrectKind(const Module& module, TypeArena* typ } } - if (FFlag::LuauSelfCallAutocompleteFix2) + if (FFlag::LuauSelfCallAutocompleteFix3) return checkTypeMatch(typeArena, ty, expectedType) ? TypeCorrectKind::Correct : TypeCorrectKind::None; else return canUnify(ty, expectedType) ? TypeCorrectKind::Correct : TypeCorrectKind::None; @@ -226,7 +226,7 @@ static void autocompleteProps(const Module& module, TypeArena* typeArena, TypeId const std::vector& nodes, AutocompleteEntryMap& result, std::unordered_set& seen, std::optional containingClass = std::nullopt) { - if (FFlag::LuauSelfCallAutocompleteFix2) + if (FFlag::LuauSelfCallAutocompleteFix3) rootTy = follow(rootTy); ty = follow(ty); @@ -236,7 +236,7 @@ static void autocompleteProps(const Module& module, TypeArena* typeArena, TypeId seen.insert(ty); auto isWrongIndexer_DEPRECATED = [indexType, useStrictFunctionIndexers = !!get(ty)](Luau::TypeId type) { - LUAU_ASSERT(!FFlag::LuauSelfCallAutocompleteFix2); + LUAU_ASSERT(!FFlag::LuauSelfCallAutocompleteFix3); if (indexType == PropIndexType::Key) return false; @@ -269,7 +269,7 @@ static void autocompleteProps(const Module& module, TypeArena* typeArena, TypeId } }; auto isWrongIndexer = [typeArena, rootTy, indexType](Luau::TypeId type) { - LUAU_ASSERT(FFlag::LuauSelfCallAutocompleteFix2); + LUAU_ASSERT(FFlag::LuauSelfCallAutocompleteFix3); if (indexType == PropIndexType::Key) return false; @@ -277,21 +277,20 @@ static void autocompleteProps(const Module& module, TypeArena* typeArena, TypeId bool calledWithSelf = indexType == PropIndexType::Colon; auto isCompatibleCall = [typeArena, rootTy, calledWithSelf](const FunctionTypeVar* ftv) { - if (get(rootTy)) - { - // Calls on classes require strict match between how function is declared and how it's called - return calledWithSelf == ftv->hasSelf; - } + // Strong match with definition is a success + if (calledWithSelf == ftv->hasSelf) + return true; - // If a call is made with ':', it is invalid if a function has incompatible first argument or no arguments at all - // If a call is made with '.', but it was declared with 'self', it is considered invalid if first argument is compatible - if (calledWithSelf || ftv->hasSelf) + // Calls on classes require strict match between how function is declared and how it's called + if (get(rootTy)) + return false; + + // When called with ':', but declared without 'self', it is invalid if a function has incompatible first argument or no arguments at all + // When called with '.', but declared with 'self', it is considered invalid if first argument is compatible + if (std::optional firstArgTy = first(ftv->argTypes)) { - if (std::optional firstArgTy = first(ftv->argTypes)) - { - if (checkTypeMatch(typeArena, rootTy, *firstArgTy)) - return calledWithSelf; - } + if (checkTypeMatch(typeArena, rootTy, *firstArgTy)) + return calledWithSelf; } return !calledWithSelf; @@ -333,7 +332,7 @@ static void autocompleteProps(const Module& module, TypeArena* typeArena, TypeId AutocompleteEntryKind::Property, type, prop.deprecated, - FFlag::LuauSelfCallAutocompleteFix2 ? isWrongIndexer(type) : isWrongIndexer_DEPRECATED(type), + FFlag::LuauSelfCallAutocompleteFix3 ? isWrongIndexer(type) : isWrongIndexer_DEPRECATED(type), typeCorrect, containingClass, &prop, @@ -376,7 +375,7 @@ static void autocompleteProps(const Module& module, TypeArena* typeArena, TypeId { autocompleteProps(module, typeArena, rootTy, mt->table, indexType, nodes, result, seen); - if (FFlag::LuauSelfCallAutocompleteFix2) + if (FFlag::LuauSelfCallAutocompleteFix3) { if (auto mtable = get(mt->metatable)) fillMetatableProps(mtable); @@ -442,7 +441,7 @@ static void autocompleteProps(const Module& module, TypeArena* typeArena, TypeId AutocompleteEntryMap inner; std::unordered_set innerSeen; - if (!FFlag::LuauSelfCallAutocompleteFix2) + if (!FFlag::LuauSelfCallAutocompleteFix3) innerSeen = seen; if (isNil(*iter)) @@ -468,7 +467,7 @@ static void autocompleteProps(const Module& module, TypeArena* typeArena, TypeId ++iter; } } - else if (auto pt = get(ty); pt && FFlag::LuauSelfCallAutocompleteFix2) + else if (auto pt = get(ty); pt && FFlag::LuauSelfCallAutocompleteFix3) { if (pt->metatable) { @@ -476,7 +475,7 @@ static void autocompleteProps(const Module& module, TypeArena* typeArena, TypeId fillMetatableProps(mtable); } } - else if (FFlag::LuauSelfCallAutocompleteFix2 && get(get(ty))) + else if (FFlag::LuauSelfCallAutocompleteFix3 && get(get(ty))) { autocompleteProps(module, typeArena, rootTy, getSingletonTypes().stringType, indexType, nodes, result, seen); } @@ -1405,7 +1404,7 @@ static AutocompleteResult autocomplete(const SourceModule& sourceModule, const M TypeId ty = follow(*it); PropIndexType indexType = indexName->op == ':' ? PropIndexType::Colon : PropIndexType::Point; - if (!FFlag::LuauSelfCallAutocompleteFix2 && isString(ty)) + if (!FFlag::LuauSelfCallAutocompleteFix3 && isString(ty)) return { autocompleteProps(*module, typeArena, typeChecker.globalScope->bindings[AstName{"string"}].typeId, indexType, ancestry), ancestry}; else diff --git a/Analysis/src/ConstraintGraphBuilder.cpp b/Analysis/src/ConstraintGraphBuilder.cpp index 38895ceb..efaeff6c 100644 --- a/Analysis/src/ConstraintGraphBuilder.cpp +++ b/Analysis/src/ConstraintGraphBuilder.cpp @@ -14,7 +14,7 @@ namespace Luau const AstStat* getFallthrough(const AstStat* node); // TypeInfer.cpp ConstraintGraphBuilder::ConstraintGraphBuilder( - const ModuleName& moduleName, TypeArena* arena, NotNull ice, NotNull globalScope) + const ModuleName& moduleName, TypeArena* arena, NotNull ice, NotNull globalScope) : moduleName(moduleName) , singletonTypes(getSingletonTypes()) , arena(arena) @@ -25,36 +25,34 @@ ConstraintGraphBuilder::ConstraintGraphBuilder( LUAU_ASSERT(arena); } -TypeId ConstraintGraphBuilder::freshType(NotNull scope) +TypeId ConstraintGraphBuilder::freshType(const ScopePtr& scope) { - return arena->addType(FreeTypeVar{scope}); + return arena->addType(FreeTypeVar{scope.get()}); } -TypePackId ConstraintGraphBuilder::freshTypePack(NotNull scope) +TypePackId ConstraintGraphBuilder::freshTypePack(const ScopePtr& scope) { - FreeTypePack f{scope}; + FreeTypePack f{scope.get()}; return arena->addTypePack(TypePackVar{std::move(f)}); } -NotNull ConstraintGraphBuilder::childScope(Location location, NotNull parent) +ScopePtr ConstraintGraphBuilder::childScope(Location location, const ScopePtr& parent) { - auto scope = std::make_unique(); - NotNull borrow = NotNull(scope.get()); - scopes.emplace_back(location, std::move(scope)); + auto scope = std::make_shared(parent); + scopes.emplace_back(location, scope); - borrow->parent = parent; - borrow->returnType = parent->returnType; - parent->children.push_back(borrow); + scope->returnType = parent->returnType; + parent->children.push_back(NotNull(scope.get())); - return borrow; + return scope; } -void ConstraintGraphBuilder::addConstraint(NotNull scope, ConstraintV cv) +void ConstraintGraphBuilder::addConstraint(const ScopePtr& scope, ConstraintV cv) { scope->constraints.emplace_back(new Constraint{std::move(cv)}); } -void ConstraintGraphBuilder::addConstraint(NotNull scope, std::unique_ptr c) +void ConstraintGraphBuilder::addConstraint(const ScopePtr& scope, std::unique_ptr c) { scope->constraints.emplace_back(std::move(c)); } @@ -63,13 +61,13 @@ void ConstraintGraphBuilder::visit(AstStatBlock* block) { LUAU_ASSERT(scopes.empty()); LUAU_ASSERT(rootScope == nullptr); - scopes.emplace_back(block->location, std::make_unique()); - rootScope = scopes.back().second.get(); - NotNull borrow = NotNull(rootScope); + ScopePtr scope = std::make_shared(singletonTypes.anyTypePack); + rootScope = scope.get(); + scopes.emplace_back(block->location, scope); - rootScope->returnType = freshTypePack(borrow); + rootScope->returnType = freshTypePack(scope); - prepopulateGlobalScope(borrow, block); + prepopulateGlobalScope(scope, block); // TODO: We should share the global scope. rootScope->typeBindings["nil"] = singletonTypes.nilType; @@ -78,10 +76,10 @@ void ConstraintGraphBuilder::visit(AstStatBlock* block) rootScope->typeBindings["boolean"] = singletonTypes.booleanType; rootScope->typeBindings["thread"] = singletonTypes.threadType; - visitBlockWithoutChildScope(borrow, block); + visitBlockWithoutChildScope(scope, block); } -void ConstraintGraphBuilder::visitBlockWithoutChildScope(NotNull scope, AstStatBlock* block) +void ConstraintGraphBuilder::visitBlockWithoutChildScope(const ScopePtr& scope, AstStatBlock* block) { RecursionCounter counter{&recursionCount}; @@ -95,7 +93,7 @@ void ConstraintGraphBuilder::visitBlockWithoutChildScope(NotNull scope, visit(scope, stat); } -void ConstraintGraphBuilder::visit(NotNull scope, AstStat* stat) +void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStat* stat) { RecursionLimiter limiter{&recursionCount, FInt::LuauCheckRecursionLimit}; @@ -123,22 +121,24 @@ void ConstraintGraphBuilder::visit(NotNull scope, AstStat* stat) LUAU_ASSERT(0); } -void ConstraintGraphBuilder::visit(NotNull scope, AstStatLocal* local) +void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatLocal* local) { std::vector varTypes; for (AstLocal* local : local->vars) { TypeId ty = freshType(scope); + Location location = local->location; if (local->annotation) { + location = local->annotation->location; TypeId annotation = resolveType(scope, local->annotation); addConstraint(scope, SubtypeConstraint{ty, annotation}); } varTypes.push_back(ty); - scope->bindings[local] = ty; + scope->bindings[local] = Binding{ty, location}; } for (size_t i = 0; i < local->values.size; ++i) @@ -169,7 +169,7 @@ void ConstraintGraphBuilder::visit(NotNull scope, AstStatLocal* local) } } -void ConstraintGraphBuilder::visit(NotNull scope, AstStatFor* for_) +void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatFor* for_) { auto checkNumber = [&](AstExpr* expr) { @@ -184,24 +184,24 @@ void ConstraintGraphBuilder::visit(NotNull scope, AstStatFor* for_) checkNumber(for_->to); checkNumber(for_->step); - NotNull forScope = childScope(for_->location, scope); - forScope->bindings[for_->var] = singletonTypes.numberType; + ScopePtr forScope = childScope(for_->location, scope); + forScope->bindings[for_->var] = Binding{singletonTypes.numberType, for_->var->location}; visit(forScope, for_->body); } -void addConstraints(Constraint* constraint, NotNull scope) +void addConstraints(Constraint* constraint, NotNull scope) { scope->constraints.reserve(scope->constraints.size() + scope->constraints.size()); for (const auto& c : scope->constraints) constraint->dependencies.push_back(NotNull{c.get()}); - for (NotNull childScope : scope->children) + for (NotNull childScope : scope->children) addConstraints(constraint, childScope); } -void ConstraintGraphBuilder::visit(NotNull scope, AstStatLocalFunction* function) +void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatLocalFunction* function) { // Local // Global @@ -213,21 +213,21 @@ void ConstraintGraphBuilder::visit(NotNull scope, AstStatLocalFunction* LUAU_ASSERT(!ty.has_value()); // The parser ensures that every local function has a distinct Symbol for its name. functionType = arena->addType(BlockedTypeVar{}); - scope->bindings[function->name] = functionType; + scope->bindings[function->name] = Binding{functionType, function->name->location}; FunctionSignature sig = checkFunctionSignature(scope, function->func); - sig.bodyScope->bindings[function->name] = sig.signature; + sig.bodyScope->bindings[function->name] = Binding{sig.signature, function->func->location}; checkFunctionBody(sig.bodyScope, function->func); std::unique_ptr c{ - new Constraint{GeneralizationConstraint{functionType, sig.signature, sig.signatureScope ? sig.signatureScope : sig.bodyScope}}}; - addConstraints(c.get(), sig.bodyScope); + new Constraint{GeneralizationConstraint{functionType, sig.signature, sig.signatureScope ? sig.signatureScope.get() : sig.bodyScope.get()}}}; + addConstraints(c.get(), NotNull(sig.bodyScope.get())); addConstraint(scope, std::move(c)); } -void ConstraintGraphBuilder::visit(NotNull scope, AstStatFunction* function) +void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatFunction* function) { // Name could be AstStatLocal, AstStatGlobal, AstStatIndexName. // With or without self @@ -247,9 +247,9 @@ void ConstraintGraphBuilder::visit(NotNull scope, AstStatFunction* funct else { functionType = arena->addType(BlockedTypeVar{}); - scope->bindings[localName->local] = functionType; + scope->bindings[localName->local] = Binding{functionType, localName->location}; } - sig.bodyScope->bindings[localName->local] = sig.signature; + sig.bodyScope->bindings[localName->local] = Binding{sig.signature, localName->location}; } else if (AstExprGlobal* globalName = function->name->as()) { @@ -262,9 +262,9 @@ void ConstraintGraphBuilder::visit(NotNull scope, AstStatFunction* funct else { functionType = arena->addType(BlockedTypeVar{}); - rootScope->bindings[globalName->name] = functionType; + rootScope->bindings[globalName->name] = Binding{functionType, globalName->location}; } - sig.bodyScope->bindings[globalName->name] = sig.signature; + sig.bodyScope->bindings[globalName->name] = Binding{sig.signature, globalName->location}; } else if (AstExprIndexName* indexName = function->name->as()) { @@ -291,21 +291,21 @@ void ConstraintGraphBuilder::visit(NotNull scope, AstStatFunction* funct checkFunctionBody(sig.bodyScope, function->func); std::unique_ptr c{ - new Constraint{GeneralizationConstraint{functionType, sig.signature, sig.signatureScope ? sig.signatureScope : sig.bodyScope}}}; - addConstraints(c.get(), sig.bodyScope); + new Constraint{GeneralizationConstraint{functionType, sig.signature, sig.signatureScope ? sig.signatureScope.get() : sig.bodyScope.get()}}}; + addConstraints(c.get(), NotNull(sig.bodyScope.get())); addConstraint(scope, std::move(c)); } -void ConstraintGraphBuilder::visit(NotNull scope, AstStatReturn* ret) +void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatReturn* ret) { TypePackId exprTypes = checkPack(scope, ret->list); addConstraint(scope, PackSubtypeConstraint{exprTypes, scope->returnType}); } -void ConstraintGraphBuilder::visit(NotNull scope, AstStatBlock* block) +void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatBlock* block) { - NotNull innerScope = childScope(block->location, scope); + ScopePtr innerScope = childScope(block->location, scope); // In order to enable mutually-recursive type aliases, we need to // populate the type bindings before we actually check any of the @@ -323,7 +323,7 @@ void ConstraintGraphBuilder::visit(NotNull scope, AstStatBlock* block) visitBlockWithoutChildScope(innerScope, block); } -void ConstraintGraphBuilder::visit(NotNull scope, AstStatAssign* assign) +void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatAssign* assign) { TypePackId varPackId = checkExprList(scope, assign->vars); TypePackId valuePack = checkPack(scope, assign->values); @@ -331,21 +331,21 @@ void ConstraintGraphBuilder::visit(NotNull scope, AstStatAssign* assign) addConstraint(scope, PackSubtypeConstraint{valuePack, varPackId}); } -void ConstraintGraphBuilder::visit(NotNull scope, AstStatIf* ifStatement) +void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatIf* ifStatement) { check(scope, ifStatement->condition); - NotNull thenScope = childScope(ifStatement->thenbody->location, scope); + ScopePtr thenScope = childScope(ifStatement->thenbody->location, scope); visit(thenScope, ifStatement->thenbody); if (ifStatement->elsebody) { - NotNull elseScope = childScope(ifStatement->elsebody->location, scope); + ScopePtr elseScope = childScope(ifStatement->elsebody->location, scope); visit(elseScope, ifStatement->elsebody); } } -void ConstraintGraphBuilder::visit(NotNull scope, AstStatTypeAlias* alias) +void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatTypeAlias* alias) { // TODO: Exported type aliases // TODO: Generic type aliases @@ -371,7 +371,7 @@ void ConstraintGraphBuilder::visit(NotNull scope, AstStatTypeAlias* alia addConstraint(scope, NameConstraint{ty, alias->name.value}); } -TypePackId ConstraintGraphBuilder::checkPack(NotNull scope, AstArray exprs) +TypePackId ConstraintGraphBuilder::checkPack(const ScopePtr& scope, AstArray exprs) { if (exprs.size == 0) return arena->addTypePack({}); @@ -392,7 +392,7 @@ TypePackId ConstraintGraphBuilder::checkPack(NotNull scope, AstArrayaddTypePack(TypePack{std::move(types), last}); } -TypePackId ConstraintGraphBuilder::checkExprList(NotNull scope, const AstArray& exprs) +TypePackId ConstraintGraphBuilder::checkExprList(const ScopePtr& scope, const AstArray& exprs) { TypePackId result = arena->addTypePack({}); TypePack* resultPack = getMutable(result); @@ -413,7 +413,7 @@ TypePackId ConstraintGraphBuilder::checkExprList(NotNull scope, const As return result; } -TypePackId ConstraintGraphBuilder::checkPack(NotNull scope, AstExpr* expr) +TypePackId ConstraintGraphBuilder::checkPack(const ScopePtr& scope, AstExpr* expr) { RecursionCounter counter{&recursionCount}; @@ -468,7 +468,7 @@ TypePackId ConstraintGraphBuilder::checkPack(NotNull scope, AstExpr* exp return result; } -TypeId ConstraintGraphBuilder::check(NotNull scope, AstExpr* expr) +TypeId ConstraintGraphBuilder::check(const ScopePtr& scope, AstExpr* expr) { RecursionCounter counter{&recursionCount}; @@ -548,7 +548,7 @@ TypeId ConstraintGraphBuilder::check(NotNull scope, AstExpr* expr) return result; } -TypeId ConstraintGraphBuilder::check(NotNull scope, AstExprIndexName* indexName) +TypeId ConstraintGraphBuilder::check(const ScopePtr& scope, AstExprIndexName* indexName) { TypeId obj = check(scope, indexName->expr); TypeId result = freshType(scope); @@ -564,7 +564,7 @@ TypeId ConstraintGraphBuilder::check(NotNull scope, AstExprIndexName* in return result; } -TypeId ConstraintGraphBuilder::check(NotNull scope, AstExprIndexExpr* indexExpr) +TypeId ConstraintGraphBuilder::check(const ScopePtr& scope, AstExprIndexExpr* indexExpr) { TypeId obj = check(scope, indexExpr->expr); TypeId indexType = check(scope, indexExpr->index); @@ -579,7 +579,7 @@ TypeId ConstraintGraphBuilder::check(NotNull scope, AstExprIndexExpr* in return result; } -TypeId ConstraintGraphBuilder::check(NotNull scope, AstExprUnary* unary) +TypeId ConstraintGraphBuilder::check(const ScopePtr& scope, AstExprUnary* unary) { TypeId operandType = check(scope, unary->expr); @@ -599,7 +599,7 @@ TypeId ConstraintGraphBuilder::check(NotNull scope, AstExprUnary* unary) return singletonTypes.errorRecoveryType(); } -TypeId ConstraintGraphBuilder::check(NotNull scope, AstExprBinary* binary) +TypeId ConstraintGraphBuilder::check(const ScopePtr& scope, AstExprBinary* binary) { TypeId leftType = check(scope, binary->left); TypeId rightType = check(scope, binary->right); @@ -624,7 +624,7 @@ TypeId ConstraintGraphBuilder::check(NotNull scope, AstExprBinary* binar return nullptr; } -TypeId ConstraintGraphBuilder::checkExprTable(NotNull scope, AstExprTable* expr) +TypeId ConstraintGraphBuilder::checkExprTable(const ScopePtr& scope, AstExprTable* expr) { TypeId ty = arena->addType(TableTypeVar{}); TableTypeVar* ttv = getMutable(ty); @@ -674,10 +674,10 @@ TypeId ConstraintGraphBuilder::checkExprTable(NotNull scope, AstExprTabl return ty; } -ConstraintGraphBuilder::FunctionSignature ConstraintGraphBuilder::checkFunctionSignature(NotNull parent, AstExprFunction* fn) +ConstraintGraphBuilder::FunctionSignature ConstraintGraphBuilder::checkFunctionSignature(const ScopePtr& parent, AstExprFunction* fn) { - Scope2* signatureScope = nullptr; - Scope2* bodyScope = nullptr; + ScopePtr signatureScope = nullptr; + ScopePtr bodyScope = nullptr; TypePackId returnType = nullptr; std::vector genericTypes; @@ -690,18 +690,17 @@ ConstraintGraphBuilder::FunctionSignature ConstraintGraphBuilder::checkFunctionS // generics properly. if (hasGenerics) { - NotNull signatureBorrow = childScope(fn->location, parent); - signatureScope = signatureBorrow.get(); + signatureScope = childScope(fn->location, parent); // We need to assign returnType before creating bodyScope so that the // return type gets propogated to bodyScope. - returnType = freshTypePack(signatureBorrow); + returnType = freshTypePack(signatureScope); signatureScope->returnType = returnType; - bodyScope = childScope(fn->body->location, signatureBorrow).get(); + bodyScope = childScope(fn->body->location, signatureScope); - std::vector> genericDefinitions = createGenerics(signatureBorrow, fn->generics); - std::vector> genericPackDefinitions = createGenericPacks(signatureBorrow, fn->genericPacks); + std::vector> genericDefinitions = createGenerics(signatureScope, fn->generics); + std::vector> genericPackDefinitions = createGenericPacks(signatureScope, fn->genericPacks); // We do not support default values on function generics, so we only // care about the types involved. @@ -719,11 +718,10 @@ ConstraintGraphBuilder::FunctionSignature ConstraintGraphBuilder::checkFunctionS } else { - NotNull bodyBorrow = childScope(fn->body->location, parent); - bodyScope = bodyBorrow.get(); + bodyScope = childScope(fn->body->location, parent); - returnType = freshTypePack(bodyBorrow); - bodyBorrow->returnType = returnType; + returnType = freshTypePack(bodyScope); + bodyScope->returnType = returnType; // To eliminate the need to branch on hasGenerics below, we say that the // signature scope is the body scope when there is no real signature @@ -731,27 +729,24 @@ ConstraintGraphBuilder::FunctionSignature ConstraintGraphBuilder::checkFunctionS signatureScope = bodyScope; } - NotNull bodyBorrow = NotNull(bodyScope); - NotNull signatureBorrow = NotNull(signatureScope); - if (fn->returnAnnotation) { - TypePackId annotatedRetType = resolveTypePack(signatureBorrow, *fn->returnAnnotation); - addConstraint(signatureBorrow, PackSubtypeConstraint{returnType, annotatedRetType}); + TypePackId annotatedRetType = resolveTypePack(signatureScope, *fn->returnAnnotation); + addConstraint(signatureScope, PackSubtypeConstraint{returnType, annotatedRetType}); } std::vector argTypes; for (AstLocal* local : fn->args) { - TypeId t = freshType(signatureBorrow); + TypeId t = freshType(signatureScope); argTypes.push_back(t); - signatureScope->bindings[local] = t; + signatureScope->bindings[local] = Binding{t, local->location}; if (local->annotation) { - TypeId argAnnotation = resolveType(signatureBorrow, local->annotation); - addConstraint(signatureBorrow, SubtypeConstraint{t, argAnnotation}); + TypeId argAnnotation = resolveType(signatureScope, local->annotation); + addConstraint(signatureScope, SubtypeConstraint{t, argAnnotation}); } } @@ -772,11 +767,11 @@ ConstraintGraphBuilder::FunctionSignature ConstraintGraphBuilder::checkFunctionS // Undo the workaround we made above: if there's no signature scope, // don't report it. /* signatureScope */ hasGenerics ? signatureScope : nullptr, - /* bodyScope */ bodyBorrow, + /* bodyScope */ bodyScope, }; } -void ConstraintGraphBuilder::checkFunctionBody(NotNull scope, AstExprFunction* fn) +void ConstraintGraphBuilder::checkFunctionBody(const ScopePtr& scope, AstExprFunction* fn) { visitBlockWithoutChildScope(scope, fn->body); @@ -789,7 +784,7 @@ void ConstraintGraphBuilder::checkFunctionBody(NotNull scope, AstExprFun } } -TypeId ConstraintGraphBuilder::resolveType(NotNull scope, AstType* ty) +TypeId ConstraintGraphBuilder::resolveType(const ScopePtr& scope, AstType* ty) { TypeId result = nullptr; @@ -834,7 +829,7 @@ TypeId ConstraintGraphBuilder::resolveType(NotNull scope, AstType* ty) { // TODO: Recursion limit. bool hasGenerics = fn->generics.size > 0 || fn->genericPacks.size > 0; - Scope2* signatureScope = nullptr; + ScopePtr signatureScope = nullptr; std::vector genericTypes; std::vector genericTypePacks; @@ -843,22 +838,21 @@ TypeId ConstraintGraphBuilder::resolveType(NotNull scope, AstType* ty) // for the generic bindings to live on. if (hasGenerics) { - NotNull signatureBorrow = childScope(fn->location, scope); - signatureScope = signatureBorrow.get(); + signatureScope = childScope(fn->location, scope); - std::vector> genericDefinitions = createGenerics(signatureBorrow, fn->generics); - std::vector> genericPackDefinitions = createGenericPacks(signatureBorrow, fn->genericPacks); + std::vector> genericDefinitions = createGenerics(signatureScope, fn->generics); + std::vector> genericPackDefinitions = createGenericPacks(signatureScope, fn->genericPacks); for (const auto& [name, g] : genericDefinitions) { genericTypes.push_back(g.ty); - signatureBorrow->typeBindings[name] = g.ty; + signatureScope->typeBindings[name] = g.ty; } for (const auto& [name, g] : genericPackDefinitions) { genericTypePacks.push_back(g.tp); - signatureBorrow->typePackBindings[name] = g.tp; + signatureScope->typePackBindings[name] = g.tp; } } else @@ -866,13 +860,11 @@ TypeId ConstraintGraphBuilder::resolveType(NotNull scope, AstType* ty) // To eliminate the need to branch on hasGenerics below, we say that // the signature scope is the parent scope if we don't have // generics. - signatureScope = scope.get(); + signatureScope = scope; } - NotNull signatureBorrow(signatureScope); - - TypePackId argTypes = resolveTypePack(signatureBorrow, fn->argTypes); - TypePackId returnTypes = resolveTypePack(signatureBorrow, fn->returnTypes); + TypePackId argTypes = resolveTypePack(signatureScope, fn->argTypes); + TypePackId returnTypes = resolveTypePack(signatureScope, fn->returnTypes); // TODO: FunctionTypeVar needs a pointer to the scope so that we know // how to quantify/instantiate it. @@ -950,7 +942,7 @@ TypeId ConstraintGraphBuilder::resolveType(NotNull scope, AstType* ty) return result; } -TypePackId ConstraintGraphBuilder::resolveTypePack(NotNull scope, AstTypePack* tp) +TypePackId ConstraintGraphBuilder::resolveTypePack(const ScopePtr& scope, AstTypePack* tp) { TypePackId result; if (auto expl = tp->as()) @@ -964,7 +956,7 @@ TypePackId ConstraintGraphBuilder::resolveTypePack(NotNull scope, AstTyp } else if (auto gen = tp->as()) { - result = arena->addTypePack(TypePackVar{GenericTypePack{scope, gen->genericName.value}}); + result = arena->addTypePack(TypePackVar{GenericTypePack{scope.get(), gen->genericName.value}}); } else { @@ -976,7 +968,7 @@ TypePackId ConstraintGraphBuilder::resolveTypePack(NotNull scope, AstTyp return result; } -TypePackId ConstraintGraphBuilder::resolveTypePack(NotNull scope, const AstTypeList& list) +TypePackId ConstraintGraphBuilder::resolveTypePack(const ScopePtr& scope, const AstTypeList& list) { std::vector head; @@ -994,12 +986,12 @@ TypePackId ConstraintGraphBuilder::resolveTypePack(NotNull scope, const return arena->addTypePack(TypePack{head, tail}); } -std::vector> ConstraintGraphBuilder::createGenerics(NotNull scope, AstArray generics) +std::vector> ConstraintGraphBuilder::createGenerics(const ScopePtr& scope, AstArray generics) { std::vector> result; for (const auto& generic : generics) { - TypeId genericTy = arena->addType(GenericTypeVar{scope, generic.name.value}); + TypeId genericTy = arena->addType(GenericTypeVar{scope.get(), generic.name.value}); std::optional defaultTy = std::nullopt; if (generic.defaultValue) @@ -1015,12 +1007,12 @@ std::vector> ConstraintGraphBuilder::crea } std::vector> ConstraintGraphBuilder::createGenericPacks( - NotNull scope, AstArray generics) + const ScopePtr& scope, AstArray generics) { std::vector> result; for (const auto& generic : generics) { - TypePackId genericTy = arena->addTypePack(TypePackVar{GenericTypePack{scope, generic.name.value}}); + TypePackId genericTy = arena->addTypePack(TypePackVar{GenericTypePack{scope.get(), generic.name.value}}); std::optional defaultTy = std::nullopt; if (generic.defaultValue) @@ -1035,7 +1027,7 @@ std::vector> ConstraintGraphBuilder:: return result; } -TypeId ConstraintGraphBuilder::flattenPack(NotNull scope, Location location, TypePackId tp) +TypeId ConstraintGraphBuilder::flattenPack(const ScopePtr& scope, Location location, TypePackId tp) { if (auto f = first(tp)) return *f; @@ -1061,10 +1053,10 @@ void ConstraintGraphBuilder::reportCodeTooComplex(Location location) struct GlobalPrepopulator : AstVisitor { - const NotNull globalScope; + const NotNull globalScope; const NotNull arena; - GlobalPrepopulator(NotNull globalScope, NotNull arena) + GlobalPrepopulator(NotNull globalScope, NotNull arena) : globalScope(globalScope) , arena(arena) { @@ -1073,29 +1065,29 @@ struct GlobalPrepopulator : AstVisitor bool visit(AstStatFunction* function) override { if (AstExprGlobal* g = function->name->as()) - globalScope->bindings[g->name] = arena->addType(BlockedTypeVar{}); + globalScope->bindings[g->name] = Binding{arena->addType(BlockedTypeVar{})}; return true; } }; -void ConstraintGraphBuilder::prepopulateGlobalScope(NotNull globalScope, AstStatBlock* program) +void ConstraintGraphBuilder::prepopulateGlobalScope(const ScopePtr& globalScope, AstStatBlock* program) { - GlobalPrepopulator gp{NotNull{globalScope}, arena}; + GlobalPrepopulator gp{NotNull{globalScope.get()}, arena}; program->visit(&gp); } -void collectConstraints(std::vector>& result, NotNull scope) +void collectConstraints(std::vector>& result, NotNull scope) { for (const auto& c : scope->constraints) result.push_back(NotNull{c.get()}); - for (NotNull child : scope->children) + for (NotNull child : scope->children) collectConstraints(result, child); } -std::vector> collectConstraints(NotNull rootScope) +std::vector> collectConstraints(NotNull rootScope) { std::vector> result; collectConstraints(result, rootScope); diff --git a/Analysis/src/ConstraintSolver.cpp b/Analysis/src/ConstraintSolver.cpp index 077a4e28..1cd29918 100644 --- a/Analysis/src/ConstraintSolver.cpp +++ b/Analysis/src/ConstraintSolver.cpp @@ -13,31 +13,31 @@ LUAU_FASTFLAGVARIABLE(DebugLuauLogSolverToJson, false); namespace Luau { -[[maybe_unused]] static void dumpBindings(NotNull scope, ToStringOptions& opts) +[[maybe_unused]] static void dumpBindings(NotNull scope, ToStringOptions& opts) { for (const auto& [k, v] : scope->bindings) { - auto d = toStringDetailed(v, opts); + auto d = toStringDetailed(v.typeId, opts); opts.nameMap = d.nameMap; printf("\t%s : %s\n", k.c_str(), d.name.c_str()); } - for (NotNull child : scope->children) + for (NotNull child : scope->children) dumpBindings(child, opts); } -static void dumpConstraints(NotNull scope, ToStringOptions& opts) +static void dumpConstraints(NotNull scope, ToStringOptions& opts) { for (const ConstraintPtr& c : scope->constraints) { printf("\t%s\n", toString(*c, opts).c_str()); } - for (NotNull child : scope->children) + for (NotNull child : scope->children) dumpConstraints(child, opts); } -void dump(NotNull rootScope, ToStringOptions& opts) +void dump(NotNull rootScope, ToStringOptions& opts) { printf("constraints:\n"); dumpConstraints(rootScope, opts); @@ -55,7 +55,7 @@ void dump(ConstraintSolver* cs, ToStringOptions& opts) } } -ConstraintSolver::ConstraintSolver(TypeArena* arena, NotNull rootScope) +ConstraintSolver::ConstraintSolver(TypeArena* arena, NotNull rootScope) : arena(arena) , constraints(collectConstraints(rootScope)) , rootScope(rootScope) diff --git a/Analysis/src/ConstraintSolverLogger.cpp b/Analysis/src/ConstraintSolverLogger.cpp index 2f93c280..0c2517c0 100644 --- a/Analysis/src/ConstraintSolverLogger.cpp +++ b/Analysis/src/ConstraintSolverLogger.cpp @@ -5,12 +5,12 @@ namespace Luau { -static std::string dumpScopeAndChildren(const Scope2* scope, ToStringOptions& opts) +static std::string dumpScopeAndChildren(const Scope* scope, ToStringOptions& opts) { std::string output = "{\"bindings\":{"; bool comma = false; - for (const auto& [name, type] : scope->bindings) + for (const auto& [name, binding] : scope->bindings) { if (comma) output += ","; @@ -19,7 +19,7 @@ static std::string dumpScopeAndChildren(const Scope2* scope, ToStringOptions& op output += name.c_str(); output += "\": \""; - ToStringResult result = toStringDetailed(type, opts); + ToStringResult result = toStringDetailed(binding.typeId, opts); opts.nameMap = std::move(result.nameMap); output += result.name; output += "\""; @@ -30,7 +30,7 @@ static std::string dumpScopeAndChildren(const Scope2* scope, ToStringOptions& op output += "},\"children\":["; comma = false; - for (const Scope2* child : scope->children) + for (const Scope* child : scope->children) { if (comma) output += ","; @@ -96,7 +96,7 @@ std::string ConstraintSolverLogger::compileOutput() return output; } -void ConstraintSolverLogger::captureBoundarySnapshot(const Scope2* rootScope, std::vector>& unsolvedConstraints) +void ConstraintSolverLogger::captureBoundarySnapshot(const Scope* rootScope, std::vector>& unsolvedConstraints) { std::string snapshot = "{\"type\":\"boundary\",\"rootScope\":"; @@ -109,7 +109,7 @@ void ConstraintSolverLogger::captureBoundarySnapshot(const Scope2* rootScope, st } void ConstraintSolverLogger::prepareStepSnapshot( - const Scope2* rootScope, NotNull current, std::vector>& unsolvedConstraints) + const Scope* rootScope, NotNull current, std::vector>& unsolvedConstraints) { // LUAU_ASSERT(!preparedSnapshot); diff --git a/Analysis/src/EmbeddedBuiltinDefinitions.cpp b/Analysis/src/EmbeddedBuiltinDefinitions.cpp index f93f65dd..45663531 100644 --- a/Analysis/src/EmbeddedBuiltinDefinitions.cpp +++ b/Analysis/src/EmbeddedBuiltinDefinitions.cpp @@ -2,7 +2,6 @@ #include "Luau/BuiltinDefinitions.h" LUAU_FASTFLAG(LuauUnknownAndNeverType) -LUAU_FASTFLAG(LuauCheckLenMT) namespace Luau { @@ -123,6 +122,7 @@ declare function tonumber(value: T, radix: number?): number? declare function rawequal(a: T1, b: T2): boolean declare function rawget(tab: {[K]: V}, k: K): V declare function rawset(tab: {[K]: V}, k: K, v: V): {[K]: V} +declare function rawlen(obj: {[K]: V} | string): number declare function setfenv(target: number | (T...) -> R..., env: {[string]: any}): ((T...) -> R...)? @@ -206,10 +206,6 @@ std::string getBuiltinDefinitionSource() std::string result = kBuiltinDefinitionLuaSrc; - // TODO: move this into kBuiltinDefinitionLuaSrc - if (FFlag::LuauCheckLenMT) - result += "declare function rawlen(obj: {[K]: V} | string): number\n"; - if (FFlag::LuauUnknownAndNeverType) result += "declare function error(message: T, level: number?): never\n"; else diff --git a/Analysis/src/Frontend.cpp b/Analysis/src/Frontend.cpp index a7670de5..222a17df 100644 --- a/Analysis/src/Frontend.cpp +++ b/Analysis/src/Frontend.cpp @@ -766,35 +766,35 @@ const SourceModule* Frontend::getSourceModule(const ModuleName& moduleName) cons return const_cast(this)->getSourceModule(moduleName); } -NotNull Frontend::getGlobalScope2() +NotNull Frontend::getGlobalScope() { - if (!globalScope2) + if (!globalScope) { const SingletonTypes& singletonTypes = getSingletonTypes(); - globalScope2 = std::make_unique(); - globalScope2->typeBindings["nil"] = singletonTypes.nilType; - globalScope2->typeBindings["number"] = singletonTypes.numberType; - globalScope2->typeBindings["string"] = singletonTypes.stringType; - globalScope2->typeBindings["boolean"] = singletonTypes.booleanType; - globalScope2->typeBindings["thread"] = singletonTypes.threadType; + globalScope = std::make_unique(singletonTypes.anyTypePack); + globalScope->typeBindings["nil"] = singletonTypes.nilType; + globalScope->typeBindings["number"] = singletonTypes.numberType; + globalScope->typeBindings["string"] = singletonTypes.stringType; + globalScope->typeBindings["boolean"] = singletonTypes.booleanType; + globalScope->typeBindings["thread"] = singletonTypes.threadType; } - return NotNull(globalScope2.get()); + return NotNull(globalScope.get()); } ModulePtr Frontend::check(const SourceModule& sourceModule, Mode mode, const ScopePtr& environmentScope) { ModulePtr result = std::make_shared(); - ConstraintGraphBuilder cgb{sourceModule.name, &result->internalTypes, NotNull(&iceHandler), getGlobalScope2()}; + ConstraintGraphBuilder cgb{sourceModule.name, &result->internalTypes, NotNull(&iceHandler), getGlobalScope()}; cgb.visit(sourceModule.root); result->errors = std::move(cgb.errors); ConstraintSolver cs{&result->internalTypes, NotNull(cgb.rootScope)}; cs.run(); - result->scope2s = std::move(cgb.scopes); + result->scopes = std::move(cgb.scopes); result->astTypes = std::move(cgb.astTypes); result->astTypePacks = std::move(cgb.astTypePacks); result->astOriginalCallTypes = std::move(cgb.astOriginalCallTypes); diff --git a/Analysis/src/JsonEncoder.cpp b/Analysis/src/JsonEncoder.cpp index 550c9b59..ee7dadb3 100644 --- a/Analysis/src/JsonEncoder.cpp +++ b/Analysis/src/JsonEncoder.cpp @@ -103,8 +103,8 @@ struct AstJsonEncoder : public AstVisitor void write(double d) { - char b[256]; - sprintf(b, "%.17g", d); + char b[32]; + snprintf(b, sizeof(b), "%.17g", d); writeRaw(b); } diff --git a/Analysis/src/Module.cpp b/Analysis/src/Module.cpp index 0603a042..dca18027 100644 --- a/Analysis/src/Module.cpp +++ b/Analysis/src/Module.cpp @@ -100,29 +100,20 @@ void Module::clonePublicInterface(InternalErrorReporter& ice) CloneState cloneState; - ScopePtr moduleScope = FFlag::DebugLuauDeferredConstraintResolution ? nullptr : getModuleScope(); - Scope2* moduleScope2 = FFlag::DebugLuauDeferredConstraintResolution ? getModuleScope2() : nullptr; + ScopePtr moduleScope = getModuleScope(); - TypePackId returnType = FFlag::DebugLuauDeferredConstraintResolution ? moduleScope2->returnType : moduleScope->returnType; + TypePackId returnType = moduleScope->returnType; std::optional varargPack = FFlag::DebugLuauDeferredConstraintResolution ? std::nullopt : moduleScope->varargPack; std::unordered_map* exportedTypeBindings = FFlag::DebugLuauDeferredConstraintResolution ? nullptr : &moduleScope->exportedTypeBindings; returnType = clone(returnType, interfaceTypes, cloneState); - if (moduleScope) + moduleScope->returnType = returnType; + if (varargPack) { - moduleScope->returnType = returnType; - if (varargPack) - { - varargPack = clone(*varargPack, interfaceTypes, cloneState); - moduleScope->varargPack = varargPack; - } - } - else - { - LUAU_ASSERT(moduleScope2); - moduleScope2->returnType = returnType; // TODO varargPack + varargPack = clone(*varargPack, interfaceTypes, cloneState); + moduleScope->varargPack = varargPack; } ForceNormal forceNormal{&interfaceTypes}; @@ -201,10 +192,4 @@ ScopePtr Module::getModuleScope() const return scopes.front().second; } -Scope2* Module::getModuleScope2() const -{ - LUAU_ASSERT(!scope2s.empty()); - return scope2s.front().second.get(); -} - } // namespace Luau diff --git a/Analysis/src/Quantify.cpp b/Analysis/src/Quantify.cpp index 294c479d..72eb4012 100644 --- a/Analysis/src/Quantify.cpp +++ b/Analysis/src/Quantify.cpp @@ -16,13 +16,13 @@ namespace Luau { /// @return true if outer encloses inner -static bool subsumes(Scope2* outer, Scope2* inner) +static bool subsumes(Scope* outer, Scope* inner) { while (inner) { if (inner == outer) return true; - inner = inner->parent; + inner = inner->parent.get(); } return false; @@ -33,7 +33,7 @@ struct Quantifier final : TypeVarOnceVisitor TypeLevel level; std::vector generics; std::vector genericPacks; - Scope2* scope = nullptr; + Scope* scope = nullptr; bool seenGenericType = false; bool seenMutableType = false; @@ -43,20 +43,20 @@ struct Quantifier final : TypeVarOnceVisitor LUAU_ASSERT(!FFlag::DebugLuauDeferredConstraintResolution); } - explicit Quantifier(Scope2* scope) + explicit Quantifier(Scope* scope) : scope(scope) { LUAU_ASSERT(FFlag::DebugLuauDeferredConstraintResolution); } /// @return true if outer encloses inner - bool subsumes(Scope2* outer, Scope2* inner) + bool subsumes(Scope* outer, Scope* inner) { while (inner) { if (inner == outer) return true; - inner = inner->parent; + inner = inner->parent.get(); } return false; @@ -216,7 +216,7 @@ void quantify(TypeId ty, TypeLevel level) } } -void quantify(TypeId ty, Scope2* scope) +void quantify(TypeId ty, Scope* scope) { Quantifier q{scope}; q.traverse(ty); @@ -240,11 +240,11 @@ void quantify(TypeId ty, Scope2* scope) struct PureQuantifier : Substitution { - Scope2* scope; + Scope* scope; std::vector insertedGenerics; std::vector insertedGenericPacks; - PureQuantifier(TypeArena* arena, Scope2* scope) + PureQuantifier(TypeArena* arena, Scope* scope) : Substitution(TxnLog::empty(), arena) , scope(scope) { @@ -322,7 +322,7 @@ struct PureQuantifier : Substitution } }; -TypeId quantify(TypeArena* arena, TypeId ty, Scope2* scope) +TypeId quantify(TypeArena* arena, TypeId ty, Scope* scope) { PureQuantifier quantifier{arena, scope}; std::optional result = quantifier.substitute(ty); diff --git a/Analysis/src/Scope.cpp b/Analysis/src/Scope.cpp index 247a9dd6..083aa085 100644 --- a/Analysis/src/Scope.cpp +++ b/Analysis/src/Scope.cpp @@ -21,22 +21,6 @@ Scope::Scope(const ScopePtr& parent, int subLevel) level.subLevel = subLevel; } -std::optional Scope::lookup(const Symbol& name) -{ - Scope* scope = this; - - while (scope) - { - auto it = scope->bindings.find(name); - if (it != scope->bindings.end()) - return it->second.typeId; - - scope = scope->parent.get(); - } - - return std::nullopt; -} - std::optional Scope::lookupType(const Name& name) { const Scope* scope = this; @@ -121,48 +105,48 @@ std::optional Scope::linearSearchForBinding(const std::string& name, bo return std::nullopt; } -std::optional Scope2::lookup(Symbol sym) +std::optional Scope::lookup(Symbol sym) { - Scope2* s = this; + Scope* s = this; while (true) { auto it = s->bindings.find(sym); if (it != s->bindings.end()) - return it->second; + return it->second.typeId; if (s->parent) - s = s->parent; + s = s->parent.get(); else return std::nullopt; } } -std::optional Scope2::lookupTypeBinding(const Name& name) +std::optional Scope::lookupTypeBinding(const Name& name) { - Scope2* s = this; + Scope* s = this; while (s) { auto it = s->typeBindings.find(name); if (it != s->typeBindings.end()) return it->second; - s = s->parent; + s = s->parent.get(); } return std::nullopt; } -std::optional Scope2::lookupTypePackBinding(const Name& name) +std::optional Scope::lookupTypePackBinding(const Name& name) { - Scope2* s = this; + Scope* s = this; while (s) { auto it = s->typePackBindings.find(name); if (it != s->typePackBindings.end()) return it->second; - s = s->parent; + s = s->parent.get(); } return std::nullopt; diff --git a/Analysis/src/ToString.cpp b/Analysis/src/ToString.cpp index d571adf3..9e637a9b 100644 --- a/Analysis/src/ToString.cpp +++ b/Analysis/src/ToString.cpp @@ -12,6 +12,7 @@ LUAU_FASTFLAG(LuauLowerBoundsCalculation) LUAU_FASTFLAG(LuauUnknownAndNeverType) +LUAU_FASTFLAGVARIABLE(LuauSpecialTypesAsterisked, false) /* * Prefix generic typenames with gen- @@ -277,7 +278,10 @@ struct TypeVarStringifier if (tv->ty.valueless_by_exception()) { state.result.error = true; - state.emit("< VALUELESS BY EXCEPTION >"); + if (FFlag::LuauSpecialTypesAsterisked) + state.emit("* VALUELESS BY EXCEPTION *"); + else + state.emit("< VALUELESS BY EXCEPTION >"); return; } @@ -452,7 +456,10 @@ struct TypeVarStringifier if (state.hasSeen(&ftv)) { state.result.cycle = true; - state.emit(""); + if (FFlag::LuauSpecialTypesAsterisked) + state.emit("*CYCLE*"); + else + state.emit(""); return; } @@ -560,7 +567,10 @@ struct TypeVarStringifier if (state.hasSeen(&ttv)) { state.result.cycle = true; - state.emit(""); + if (FFlag::LuauSpecialTypesAsterisked) + state.emit("*CYCLE*"); + else + state.emit(""); return; } @@ -690,7 +700,10 @@ struct TypeVarStringifier if (state.hasSeen(&uv)) { state.result.cycle = true; - state.emit(""); + if (FFlag::LuauSpecialTypesAsterisked) + state.emit("*CYCLE*"); + else + state.emit(""); return; } @@ -757,7 +770,10 @@ struct TypeVarStringifier if (state.hasSeen(&uv)) { state.result.cycle = true; - state.emit(""); + if (FFlag::LuauSpecialTypesAsterisked) + state.emit("*CYCLE*"); + else + state.emit(""); return; } @@ -802,7 +818,10 @@ struct TypeVarStringifier void operator()(TypeId, const ErrorTypeVar& tv) { state.result.error = true; - state.emit(FFlag::LuauUnknownAndNeverType ? "" : "*unknown*"); + if (FFlag::LuauSpecialTypesAsterisked) + state.emit(FFlag::LuauUnknownAndNeverType ? "*error-type*" : "*unknown*"); + else + state.emit(FFlag::LuauUnknownAndNeverType ? "" : "*unknown*"); } void operator()(TypeId, const LazyTypeVar& ltv) @@ -856,7 +875,10 @@ struct TypePackStringifier if (tp->ty.valueless_by_exception()) { state.result.error = true; - state.emit("< VALUELESS TP BY EXCEPTION >"); + if (FFlag::LuauSpecialTypesAsterisked) + state.emit("* VALUELESS TP BY EXCEPTION *"); + else + state.emit("< VALUELESS TP BY EXCEPTION >"); return; } @@ -879,7 +901,10 @@ struct TypePackStringifier if (state.hasSeen(&tp)) { state.result.cycle = true; - state.emit(""); + if (FFlag::LuauSpecialTypesAsterisked) + state.emit("*CYCLETP*"); + else + state.emit(""); return; } @@ -924,14 +949,22 @@ struct TypePackStringifier void operator()(TypePackId, const Unifiable::Error& error) { state.result.error = true; - state.emit(FFlag::LuauUnknownAndNeverType ? "" : "*unknown*"); + if (FFlag::LuauSpecialTypesAsterisked) + state.emit(FFlag::LuauUnknownAndNeverType ? "*error-type*" : "*unknown*"); + else + state.emit(FFlag::LuauUnknownAndNeverType ? "" : "*unknown*"); } void operator()(TypePackId, const VariadicTypePack& pack) { state.emit("..."); if (FFlag::DebugLuauVerboseTypeNames && pack.hidden) - state.emit(""); + { + if (FFlag::LuauSpecialTypesAsterisked) + state.emit("*hidden*"); + else + state.emit(""); + } stringify(pack.ty); } @@ -1121,7 +1154,11 @@ ToStringResult toStringDetailed(TypeId ty, const ToStringOptions& opts) if (opts.maxTypeLength > 0 && result.name.length() > opts.maxTypeLength) { result.truncated = true; - result.name += "... "; + + if (FFlag::LuauSpecialTypesAsterisked) + result.name += "... *TRUNCATED*"; + else + result.name += "... "; } return result; @@ -1189,7 +1226,12 @@ ToStringResult toStringDetailed(TypePackId tp, const ToStringOptions& opts) } if (opts.maxTypeLength > 0 && result.name.length() > opts.maxTypeLength) - result.name += "... "; + { + if (FFlag::LuauSpecialTypesAsterisked) + result.name += "... *TRUNCATED*"; + else + result.name += "... "; + } return result; } diff --git a/Analysis/src/TypeChecker2.cpp b/Analysis/src/TypeChecker2.cpp index d3b9655d..85749671 100644 --- a/Analysis/src/TypeChecker2.cpp +++ b/Analysis/src/TypeChecker2.cpp @@ -69,6 +69,9 @@ struct TypeChecker2 : public AstVisitor TypePackId reconstructPack(AstArray exprs, TypeArena& arena) { + if (exprs.size == 0) + return arena.addTypePack(TypePack{{}, std::nullopt}); + std::vector head; for (size_t i = 0; i < exprs.size - 1; ++i) @@ -80,14 +83,14 @@ struct TypeChecker2 : public AstVisitor return arena.addTypePack(TypePack{head, tail}); } - Scope2* findInnermostScope(Location location) + Scope* findInnermostScope(Location location) { - Scope2* bestScope = module->getModuleScope2(); - Location bestLocation = module->scope2s[0].first; + Scope* bestScope = module->getModuleScope().get(); + Location bestLocation = module->scopes[0].first; - for (size_t i = 0; i < module->scope2s.size(); ++i) + for (size_t i = 0; i < module->scopes.size(); ++i) { - auto& [scopeBounds, scope] = module->scope2s[i]; + auto& [scopeBounds, scope] = module->scopes[i]; if (scopeBounds.encloses(location)) { if (scopeBounds.begin > bestLocation.begin || scopeBounds.end < bestLocation.end) @@ -181,7 +184,7 @@ struct TypeChecker2 : public AstVisitor bool visit(AstStatReturn* ret) override { - Scope2* scope = findInnermostScope(ret->location); + Scope* scope = findInnermostScope(ret->location); TypePackId expectedRetType = scope->returnType; TypeArena arena; @@ -359,7 +362,7 @@ struct TypeChecker2 : public AstVisitor bool visit(AstTypeReference* ty) override { - Scope2* scope = findInnermostScope(ty->location); + Scope* scope = findInnermostScope(ty->location); // TODO: Imported types // TODO: Generic types diff --git a/Analysis/src/TypeInfer.cpp b/Analysis/src/TypeInfer.cpp index ea7d81e1..371ef77a 100644 --- a/Analysis/src/TypeInfer.cpp +++ b/Analysis/src/TypeInfer.cpp @@ -35,7 +35,7 @@ LUAU_FASTFLAGVARIABLE(LuauExpectedTableUnionIndexerType, false) LUAU_FASTFLAGVARIABLE(LuauIndexSilenceErrors, false) LUAU_FASTFLAGVARIABLE(LuauLowerBoundsCalculation, false) LUAU_FASTFLAGVARIABLE(DebugLuauFreezeDuringUnification, false) -LUAU_FASTFLAGVARIABLE(LuauSelfCallAutocompleteFix2, false) +LUAU_FASTFLAGVARIABLE(LuauSelfCallAutocompleteFix3, false) LUAU_FASTFLAGVARIABLE(LuauReduceUnionRecursion, false) LUAU_FASTFLAGVARIABLE(LuauReturnAnyInsteadOfICE, false) // Eventually removed as false. LUAU_FASTFLAG(LuauNormalizeFlagIsConservative) @@ -45,7 +45,6 @@ LUAU_FASTFLAGVARIABLE(LuauReportErrorsOnIndexerKeyMismatch, false) LUAU_FASTFLAGVARIABLE(LuauUnknownAndNeverType, false) LUAU_FASTFLAG(LuauQuantifyConstrained) LUAU_FASTFLAGVARIABLE(LuauFalsyPredicateReturnsNilInstead, false) -LUAU_FASTFLAGVARIABLE(LuauCheckLenMT, false) LUAU_FASTFLAGVARIABLE(LuauCheckGenericHOFTypes, false) LUAU_FASTFLAGVARIABLE(LuauBinaryNeedsExpectedTypesToo, false) LUAU_FASTFLAGVARIABLE(LuauNeverTypesAndOperatorsInference, false) @@ -1667,7 +1666,7 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatDeclareClass& declar ftv->argNames.insert(ftv->argNames.begin(), FunctionArgument{"self", {}}); ftv->argTypes = addTypePack(TypePack{{classTy}, ftv->argTypes}); - if (FFlag::LuauSelfCallAutocompleteFix2) + if (FFlag::LuauSelfCallAutocompleteFix3) ftv->hasSelf = true; } } @@ -2465,7 +2464,7 @@ WithPredicate TypeChecker::checkExpr(const ScopePtr& scope, const AstExp DenseHashSet seen{nullptr}; - if (FFlag::LuauCheckLenMT && typeCouldHaveMetatable(operandType)) + if (typeCouldHaveMetatable(operandType)) { if (auto fnt = findMetatableEntry(operandType, "__len", expr.location, /* addErrors= */ true)) { @@ -3640,6 +3639,9 @@ void TypeChecker::checkFunctionBody(const ScopePtr& scope, TypeId ty, const AstE reportError(getEndLocation(function), FunctionExitsWithoutReturning{funTy->retTypes}); } } + + if (!currentModule->astTypes.find(&function)) + currentModule->astTypes[&function] = ty; } else ice("Checking non functional type"); diff --git a/Analysis/src/Unifiable.cpp b/Analysis/src/Unifiable.cpp index 8d23aa49..63d8647d 100644 --- a/Analysis/src/Unifiable.cpp +++ b/Analysis/src/Unifiable.cpp @@ -12,7 +12,7 @@ Free::Free(TypeLevel level) { } -Free::Free(Scope2* scope) +Free::Free(Scope* scope) : scope(scope) { } @@ -39,7 +39,7 @@ Generic::Generic(const Name& name) { } -Generic::Generic(Scope2* scope) +Generic::Generic(Scope* scope) : index(++nextIndex) , scope(scope) { @@ -53,7 +53,7 @@ Generic::Generic(TypeLevel level, const Name& name) { } -Generic::Generic(Scope2* scope, const Name& name) +Generic::Generic(Scope* scope, const Name& name) : index(++nextIndex) , scope(scope) , name(name) diff --git a/Ast/src/Parser.cpp b/Ast/src/Parser.cpp index 779bd279..8d54e367 100644 --- a/Ast/src/Parser.cpp +++ b/Ast/src/Parser.cpp @@ -24,8 +24,6 @@ bool lua_telemetry_parsed_named_non_function_type = false; LUAU_FASTFLAGVARIABLE(LuauErrorParseIntegerIssues, false) LUAU_DYNAMIC_FASTFLAGVARIABLE(LuaReportParseIntegerIssues, false) -LUAU_FASTFLAGVARIABLE(LuauAlwaysCaptureHotComments, false) - bool lua_telemetry_parsed_out_of_range_bin_integer = false; bool lua_telemetry_parsed_out_of_range_hex_integer = false; bool lua_telemetry_parsed_double_prefix_hex_integer = false; @@ -2920,39 +2918,34 @@ AstTypeError* Parser::reportTypeAnnotationError(const Location& location, const void Parser::nextLexeme() { - if (options.captureComments || FFlag::LuauAlwaysCaptureHotComments) + Lexeme::Type type = lexer.next(/* skipComments= */ false, true).type; + + while (type == Lexeme::BrokenComment || type == Lexeme::Comment || type == Lexeme::BlockComment) { - Lexeme::Type type = lexer.next(/* skipComments= */ false, true).type; + const Lexeme& lexeme = lexer.current(); - while (type == Lexeme::BrokenComment || type == Lexeme::Comment || type == Lexeme::BlockComment) + if (options.captureComments) + commentLocations.push_back(Comment{lexeme.type, lexeme.location}); + + // Subtlety: Broken comments are weird because we record them as comments AND pass them to the parser as a lexeme. + // The parser will turn this into a proper syntax error. + if (lexeme.type == Lexeme::BrokenComment) + return; + + // Comments starting with ! are called "hot comments" and contain directives for type checking / linting / compiling + if (lexeme.type == Lexeme::Comment && lexeme.length && lexeme.data[0] == '!') { - const Lexeme& lexeme = lexer.current(); + const char* text = lexeme.data; - if (options.captureComments) - commentLocations.push_back(Comment{lexeme.type, lexeme.location}); + unsigned int end = lexeme.length; + while (end > 0 && isSpace(text[end - 1])) + --end; - // Subtlety: Broken comments are weird because we record them as comments AND pass them to the parser as a lexeme. - // The parser will turn this into a proper syntax error. - if (lexeme.type == Lexeme::BrokenComment) - return; - - // Comments starting with ! are called "hot comments" and contain directives for type checking / linting / compiling - if (lexeme.type == Lexeme::Comment && lexeme.length && lexeme.data[0] == '!') - { - const char* text = lexeme.data; - - unsigned int end = lexeme.length; - while (end > 0 && isSpace(text[end - 1])) - --end; - - hotcomments.push_back({hotcommentHeader, lexeme.location, std::string(text + 1, text + end)}); - } - - type = lexer.next(/* skipComments= */ false, /* updatePrevLocation= */ false).type; + hotcomments.push_back({hotcommentHeader, lexeme.location, std::string(text + 1, text + end)}); } + + type = lexer.next(/* skipComments= */ false, /* updatePrevLocation= */ false).type; } - else - lexer.next(); } } // namespace Luau diff --git a/CLI/Analyze.cpp b/CLI/Analyze.cpp index 51be0fea..cd50ef00 100644 --- a/CLI/Analyze.cpp +++ b/CLI/Analyze.cpp @@ -297,5 +297,3 @@ int main(int argc, char** argv) else return failed ? 1 : 0; } - - diff --git a/CLI/Ast.cpp b/CLI/Ast.cpp index f3f69be8..6ee608c4 100644 --- a/CLI/Ast.cpp +++ b/CLI/Ast.cpp @@ -83,5 +83,3 @@ int main(int argc, char** argv) return parseResult.errors.size() > 0 ? 1 : 0; } - - diff --git a/CLI/ReplEntry.cpp b/CLI/ReplEntry.cpp index 75995e6a..8543e3f7 100644 --- a/CLI/ReplEntry.cpp +++ b/CLI/ReplEntry.cpp @@ -1,8 +1,6 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #include "Repl.h" - - int main(int argc, char** argv) { return replMain(argc, argv); diff --git a/Compiler/src/Compiler.cpp b/Compiler/src/Compiler.cpp index 8f3befad..4be54378 100644 --- a/Compiler/src/Compiler.cpp +++ b/Compiler/src/Compiler.cpp @@ -25,8 +25,6 @@ LUAU_FASTINTVARIABLE(LuauCompileInlineDepth, 5) LUAU_FASTFLAGVARIABLE(LuauCompileNoIpairs, false) -LUAU_FASTFLAGVARIABLE(LuauCompileFoldBuiltins, false) -LUAU_FASTFLAGVARIABLE(LuauCompileBetterMultret, false) LUAU_FASTFLAGVARIABLE(LuauCompileFreeReassign, false) namespace Luau @@ -276,9 +274,6 @@ struct Compiler // returns true if node can return multiple values; may conservatively return true even if expr is known to return just a single value bool isExprMultRet(AstExpr* node) { - if (!FFlag::LuauCompileBetterMultret) - return node->is() || node->is(); - AstExprCall* expr = node->as(); if (!expr) return node->is(); @@ -310,27 +305,10 @@ struct Compiler if (AstExprCall* expr = node->as()) { // Optimization: convert multret calls that always return one value to fixedret calls; this facilitates inlining/constant folding - if (options.optimizationLevel >= 2) + if (options.optimizationLevel >= 2 && !isExprMultRet(node)) { - if (FFlag::LuauCompileBetterMultret) - { - if (!isExprMultRet(node)) - { - compileExprTemp(node, target); - return false; - } - } - else - { - AstExprFunction* func = getFunctionExpr(expr->func); - Function* fi = func ? functions.find(func) : nullptr; - - if (fi && fi->returnsOne) - { - compileExprTemp(node, target); - return false; - } - } + compileExprTemp(node, target); + return false; } // We temporarily swap out regTop to have targetTop work correctly... @@ -3437,30 +3415,7 @@ struct Compiler bool visit(AstStatReturn* stat) override { - if (FFlag::LuauCompileBetterMultret) - { - returnsOne &= stat->list.size == 1 && !self->isExprMultRet(stat->list.data[0]); - } - else if (stat->list.size == 1) - { - AstExpr* value = stat->list.data[0]; - - if (AstExprCall* expr = value->as()) - { - AstExprFunction* func = self->getFunctionExpr(expr->func); - Function* fi = func ? self->functions.find(func) : nullptr; - - returnsOne &= fi && fi->returnsOne; - } - else if (value->is()) - { - returnsOne = false; - } - } - else - { - returnsOne = false; - } + returnsOne &= stat->list.size == 1 && !self->isExprMultRet(stat->list.data[0]); return false; } @@ -3601,7 +3556,7 @@ void compileOrThrow(BytecodeBuilder& bytecode, const ParseResult& parseResult, c trackValues(compiler.globals, compiler.variables, root); // builtin folding is enabled on optimization level 2 since we can't deoptimize folding at runtime - if (options.optimizationLevel >= 2 && FFlag::LuauCompileFoldBuiltins) + if (options.optimizationLevel >= 2) compiler.builtinsFold = &compiler.builtins; if (options.optimizationLevel >= 1) diff --git a/Compiler/src/CostModel.cpp b/Compiler/src/CostModel.cpp index 56b6c36f..81cbfd7a 100644 --- a/Compiler/src/CostModel.cpp +++ b/Compiler/src/CostModel.cpp @@ -6,8 +6,6 @@ #include -LUAU_FASTFLAGVARIABLE(LuauCompileModelBuiltins, false) - namespace Luau { namespace Compile @@ -155,7 +153,7 @@ struct CostVisitor : AstVisitor { // builtin cost modeling is different from regular calls because we use FASTCALL to compile these // thus we use a cheaper baseline, don't account for function, and assume constant/local copy is free - bool builtin = FFlag::LuauCompileModelBuiltins && builtins.find(expr) != nullptr; + bool builtin = builtins.find(expr) != nullptr; bool builtinShort = builtin && expr->args.size <= 2; // FASTCALL1/2 Cost cost = builtin ? 2 : 3; diff --git a/Sources.cmake b/Sources.cmake index a4563019..f745667f 100644 --- a/Sources.cmake +++ b/Sources.cmake @@ -267,6 +267,7 @@ if(TARGET Luau.UnitTest) tests/Error.test.cpp tests/Frontend.test.cpp tests/JsonEncoder.test.cpp + tests/Lexer.test.cpp tests/Linter.test.cpp tests/LValue.test.cpp tests/Module.test.cpp diff --git a/VM/src/lapi.cpp b/VM/src/lapi.cpp index e3354ea2..e59f914f 100644 --- a/VM/src/lapi.cpp +++ b/VM/src/lapi.cpp @@ -34,8 +34,6 @@ * therefore call luaC_checkGC before luaC_checkthreadsleep to guarantee the object is pushed to an awake thread. */ -LUAU_FASTFLAG(LuauLazyAtoms) - const char* lua_ident = "$Lua: Lua 5.1.4 Copyright (C) 1994-2008 Lua.org, PUC-Rio $\n" "$Authors: R. Ierusalimschy, L. H. de Figueiredo & W. Celes $\n" "$URL: www.lua.org $\n"; @@ -54,7 +52,6 @@ const char* luau_ident = "$Luau: Copyright (C) 2019-2022 Roblox Corporation $\n" } #define updateatom(L, ts) \ - if (FFlag::LuauLazyAtoms) \ { \ if (ts->atom == ATOM_UNDEF) \ ts->atom = L->global->cb.useratom ? L->global->cb.useratom(ts->data, ts->len) : -1; \ diff --git a/VM/src/ldblib.cpp b/VM/src/ldblib.cpp index 93d8703a..8246f818 100644 --- a/VM/src/ldblib.cpp +++ b/VM/src/ldblib.cpp @@ -130,15 +130,14 @@ static int db_traceback(lua_State* L) if (ar.currentline > 0) { - char line[32]; -#ifdef _MSC_VER - _itoa(ar.currentline, line, 10); // 5x faster than sprintf -#else - sprintf(line, "%d", ar.currentline); -#endif + char line[32]; // manual conversion for performance + char* lineend = line + sizeof(line); + char* lineptr = lineend; + for (unsigned int r = ar.currentline; r > 0; r /= 10) + *--lineptr = '0' + (r % 10); luaL_addchar(&buf, ':'); - luaL_addstring(&buf, line); + luaL_addlstring(&buf, lineptr, lineend - lineptr); } if (ar.name) diff --git a/VM/src/ldebug.cpp b/VM/src/ldebug.cpp index 29015565..5ef41b78 100644 --- a/VM/src/ldebug.cpp +++ b/VM/src/ldebug.cpp @@ -529,7 +529,7 @@ const char* lua_debugtrace(lua_State* L) if (ar.currentline > 0) { char line[32]; - sprintf(line, ":%d", ar.currentline); + snprintf(line, sizeof(line), ":%d", ar.currentline); offset = append(buf, sizeof(buf), offset, line); } @@ -545,7 +545,7 @@ const char* lua_debugtrace(lua_State* L) if (depth > limit1 + limit2 && level == limit1 - 1) { char skip[32]; - sprintf(skip, "... (+%d frames)\n", int(depth - limit1 - limit2)); + snprintf(skip, sizeof(skip), "... (+%d frames)\n", int(depth - limit1 - limit2)); offset = append(buf, sizeof(buf), offset, skip); diff --git a/VM/src/lstring.cpp b/VM/src/lstring.cpp index 12bd67d5..d454308c 100644 --- a/VM/src/lstring.cpp +++ b/VM/src/lstring.cpp @@ -7,8 +7,6 @@ #include -LUAU_FASTFLAGVARIABLE(LuauLazyAtoms, false) - unsigned int luaS_hash(const char* str, size_t len) { // Note that this hashing algorithm is replicated in BytecodeBuilder.cpp, BytecodeBuilder::getStringHash @@ -84,7 +82,7 @@ static TString* newlstr(lua_State* L, const char* str, size_t l, unsigned int h) ts->memcat = L->activememcat; memcpy(ts->data, str, l); ts->data[l] = '\0'; /* ending 0 */ - ts->atom = FFlag::LuauLazyAtoms ? ATOM_UNDEF : L->global->cb.useratom ? L->global->cb.useratom(ts->data, l) : -1; + ts->atom = ATOM_UNDEF; tb = &L->global->strt; h = lmod(h, tb->size); ts->next = tb->hash[h]; /* chain new entry */ @@ -165,9 +163,7 @@ TString* luaS_buffinish(lua_State* L, TString* ts) ts->hash = h; ts->data[ts->len] = '\0'; // ending 0 - - // Complete string object - ts->atom = FFlag::LuauLazyAtoms ? ATOM_UNDEF : L->global->cb.useratom ? L->global->cb.useratom(ts->data, ts->len) : -1; + ts->atom = ATOM_UNDEF; ts->next = tb->hash[bucket]; // chain new entry tb->hash[bucket] = ts; diff --git a/VM/src/lstrlib.cpp b/VM/src/lstrlib.cpp index 74a8aa8a..b0cd4dc2 100644 --- a/VM/src/lstrlib.cpp +++ b/VM/src/lstrlib.cpp @@ -979,14 +979,14 @@ static int str_format(lua_State* L) { case 'c': { - sprintf(buff, form, (int)luaL_checknumber(L, arg)); + snprintf(buff, sizeof(buff), form, (int)luaL_checknumber(L, arg)); break; } case 'd': case 'i': { addInt64Format(form, formatIndicator, formatItemSize); - sprintf(buff, form, (long long)luaL_checknumber(L, arg)); + snprintf(buff, sizeof(buff), form, (long long)luaL_checknumber(L, arg)); break; } case 'o': @@ -997,7 +997,7 @@ static int str_format(lua_State* L) double argValue = luaL_checknumber(L, arg); addInt64Format(form, formatIndicator, formatItemSize); unsigned long long v = (argValue < 0) ? (unsigned long long)(long long)argValue : (unsigned long long)argValue; - sprintf(buff, form, v); + snprintf(buff, sizeof(buff), form, v); break; } case 'e': @@ -1006,7 +1006,7 @@ static int str_format(lua_State* L) case 'g': case 'G': { - sprintf(buff, form, (double)luaL_checknumber(L, arg)); + snprintf(buff, sizeof(buff), form, (double)luaL_checknumber(L, arg)); break; } case 'q': @@ -1028,7 +1028,7 @@ static int str_format(lua_State* L) } else { - sprintf(buff, form, s); + snprintf(buff, sizeof(buff), form, s); break; } } diff --git a/VM/src/ltable.cpp b/VM/src/ltable.cpp index 79e65919..38693156 100644 --- a/VM/src/ltable.cpp +++ b/VM/src/ltable.cpp @@ -44,9 +44,6 @@ static_assert(TKey{{NULL}, {0}, LUA_TDEADKEY, 0}.tt == LUA_TDEADKEY, "not enough static_assert(TKey{{NULL}, {0}, LUA_TNIL, MAXSIZE - 1}.next == MAXSIZE - 1, "not enough bits for next"); static_assert(TKey{{NULL}, {0}, LUA_TNIL, -(MAXSIZE - 1)}.next == -(MAXSIZE - 1), "not enough bits for next"); -// reset cache of absent metamethods, cache is updated in luaT_gettm -#define invalidateTMcache(t) t->tmcache = 0 - // empty hash data points to dummynode so that we can always dereference it const LuaNode luaH_dummynode = { {{NULL}, {0}, LUA_TNIL}, /* value */ @@ -667,15 +664,18 @@ TValue* luaH_set(lua_State* L, Table* t, const TValue* key) if (p != luaO_nilobject) return cast_to(TValue*, p); else - { - if (ttisnil(key)) - luaG_runerror(L, "table index is nil"); - else if (ttisnumber(key) && luai_numisnan(nvalue(key))) - luaG_runerror(L, "table index is NaN"); - else if (ttisvector(key) && luai_vecisnan(vvalue(key))) - luaG_runerror(L, "table index contains NaN"); - return newkey(L, t, key); - } + return luaH_newkey(L, t, key); +} + +TValue* luaH_newkey(lua_State* L, Table* t, const TValue* key) +{ + if (ttisnil(key)) + luaG_runerror(L, "table index is nil"); + else if (ttisnumber(key) && luai_numisnan(nvalue(key))) + luaG_runerror(L, "table index is NaN"); + else if (ttisvector(key) && luai_vecisnan(vvalue(key))) + luaG_runerror(L, "table index contains NaN"); + return newkey(L, t, key); } TValue* luaH_setnum(lua_State* L, Table* t, int key) diff --git a/VM/src/ltable.h b/VM/src/ltable.h index e8413c85..021f21bf 100644 --- a/VM/src/ltable.h +++ b/VM/src/ltable.h @@ -11,12 +11,16 @@ #define gval2slot(t, v) int(cast_to(LuaNode*, static_cast(v)) - t->node) +// reset cache of absent metamethods, cache is updated in luaT_gettm +#define invalidateTMcache(t) t->tmcache = 0 + LUAI_FUNC const TValue* luaH_getnum(Table* t, int key); LUAI_FUNC TValue* luaH_setnum(lua_State* L, Table* t, int key); LUAI_FUNC const TValue* luaH_getstr(Table* t, TString* key); LUAI_FUNC TValue* luaH_setstr(lua_State* L, Table* t, TString* key); LUAI_FUNC const TValue* luaH_get(Table* t, const TValue* key); LUAI_FUNC TValue* luaH_set(lua_State* L, Table* t, const TValue* key); +LUAI_FUNC TValue* luaH_newkey(lua_State* L, Table* t, const TValue* key); LUAI_FUNC Table* luaH_new(lua_State* L, int narray, int lnhash); LUAI_FUNC void luaH_resizearray(lua_State* L, Table* t, int nasize); LUAI_FUNC void luaH_resizehash(lua_State* L, Table* t, int nhsize); @@ -26,4 +30,6 @@ LUAI_FUNC int luaH_getn(Table* t); LUAI_FUNC Table* luaH_clone(lua_State* L, Table* tt); LUAI_FUNC void luaH_clear(Table* tt); +#define luaH_setslot(L, t, slot, key) (invalidateTMcache(t), (slot == luaO_nilobject ? luaH_newkey(L, t, key) : cast_to(TValue*, slot))) + extern const LuaNode luaH_dummynode; diff --git a/VM/src/lvmexecute.cpp b/VM/src/lvmexecute.cpp index 02b39313..10d89aaf 100644 --- a/VM/src/lvmexecute.cpp +++ b/VM/src/lvmexecute.cpp @@ -33,7 +33,7 @@ LUAU_FASTFLAGVARIABLE(LuauLenTM, false) // 3. VM_PROTECT macro saves savedpc and restores base for you; most external calls need to be wrapped into that. However, it does NOT restore // ra/rb/rc! // 4. When copying an object to any existing object as a field, generally speaking you need to call luaC_barrier! Be careful with all setobj calls -// 5. To make 4 easier to follow, please use setobj2s for copies to stack and setobj for other copies. +// 5. To make 4 easier to follow, please use setobj2s for copies to stack, setobj2t for writes to tables, and setobj for other copies. // 6. You can define HARDSTACKTESTS in llimits.h which will aggressively realloc stack; with address sanitizer this should be effective at finding // stack corruption bugs // 7. Many external Lua functions can call GC! GC will *not* traverse pointers to new objects that aren't reachable from Lua root. Be careful when @@ -458,7 +458,7 @@ static void luau_execute(lua_State* L) if (LUAU_LIKELY(ttisstring(gkey(n)) && tsvalue(gkey(n)) == tsvalue(kv) && !ttisnil(gval(n)) && !h->readonly)) { - setobj(L, gval(n), ra); + setobj2t(L, gval(n), ra); luaC_barriert(L, h, ra); VM_NEXT(); } @@ -672,7 +672,7 @@ static void luau_execute(lua_State* L) // fast-path: value is in expected slot if (LUAU_LIKELY(ttisstring(gkey(n)) && tsvalue(gkey(n)) == tsvalue(kv) && !ttisnil(gval(n)) && !h->readonly)) { - setobj(L, gval(n), ra); + setobj2t(L, gval(n), ra); luaC_barriert(L, h, ra); VM_NEXT(); } @@ -684,7 +684,7 @@ static void luau_execute(lua_State* L) int cachedslot = gval2slot(h, res); // save cachedslot to accelerate future lookups; patches currently executing instruction since pc-2 rolls back two pc++ VM_PATCH_C(pc - 2, cachedslot); - setobj(L, res, ra); + setobj2t(L, res, ra); luaC_barriert(L, h, ra); VM_NEXT(); } diff --git a/VM/src/lvmutils.cpp b/VM/src/lvmutils.cpp index be4e99a9..b9267fb8 100644 --- a/VM/src/lvmutils.cpp +++ b/VM/src/lvmutils.cpp @@ -14,6 +14,8 @@ LUAU_FASTFLAG(LuauLenTM) +LUAU_FASTFLAGVARIABLE(LuauBetterNewindex, false) + /* limit for table tag-method chains (to avoid loops) */ #define MAXTAGLOOP 100 @@ -142,24 +144,50 @@ void luaV_settable(lua_State* L, const TValue* t, TValue* key, StkId val) { /* `t' is a table? */ Table* h = hvalue(t); - if (h->readonly) - luaG_readonlyerror(L); + if (FFlag::LuauBetterNewindex) + { + const TValue* oldval = luaH_get(h, key); - TValue* oldval = luaH_set(L, h, key); /* do a primitive set */ + /* should we assign the key? (if key is valid or __newindex is not set) */ + if (!ttisnil(oldval) || (tm = fasttm(L, h->metatable, TM_NEWINDEX)) == NULL) + { + if (h->readonly) + luaG_readonlyerror(L); - L->cachedslot = gval2slot(h, oldval); /* remember slot to accelerate future lookups */ + /* luaH_set would work but would repeat the lookup so we use luaH_setslot that can reuse oldval if it's safe */ + TValue* newval = luaH_setslot(L, h, oldval, key); - if (!ttisnil(oldval) || /* result is no nil? */ - (tm = fasttm(L, h->metatable, TM_NEWINDEX)) == NULL) - { /* or no TM? */ - setobj2t(L, oldval, val); - luaC_barriert(L, h, val); - return; + L->cachedslot = gval2slot(h, newval); /* remember slot to accelerate future lookups */ + + setobj2t(L, newval, val); + luaC_barriert(L, h, val); + return; + } + + /* fallthrough to metamethod */ + } + else + { + if (h->readonly) + luaG_readonlyerror(L); + + TValue* oldval = luaH_set(L, h, key); /* do a primitive set */ + + L->cachedslot = gval2slot(h, oldval); /* remember slot to accelerate future lookups */ + + if (!ttisnil(oldval) || /* result is no nil? */ + (tm = fasttm(L, h->metatable, TM_NEWINDEX)) == NULL) + { /* or no TM? */ + setobj2t(L, oldval, val); + luaC_barriert(L, h, val); + return; + } + /* else will try the tag method */ } - /* else will try the tag method */ } else if (ttisnil(tm = luaT_gettmbyobj(L, t, TM_NEWINDEX))) luaG_indexerror(L, t, key); + if (ttisfunction(tm)) { callTM(L, tm, t, key, val); diff --git a/tests/Autocomplete.test.cpp b/tests/Autocomplete.test.cpp index d8ad3340..0a5603d6 100644 --- a/tests/Autocomplete.test.cpp +++ b/tests/Autocomplete.test.cpp @@ -2845,7 +2845,7 @@ local abc = b@1 TEST_CASE_FIXTURE(ACFixture, "no_incompatible_self_calls_on_class") { - ScopedFastFlag selfCallAutocompleteFix2{"LuauSelfCallAutocompleteFix2", true}; + ScopedFastFlag selfCallAutocompleteFix3{"LuauSelfCallAutocompleteFix3", true}; loadDefinition(R"( declare class Foo @@ -2883,9 +2883,25 @@ t.@1 } } +TEST_CASE_FIXTURE(ACFixture, "do_compatible_self_calls") +{ + ScopedFastFlag selfCallAutocompleteFix3{"LuauSelfCallAutocompleteFix3", true}; + + check(R"( +local t = {} +function t:m() end +t:@1 + )"); + + auto ac = autocomplete('1'); + + REQUIRE(ac.entryMap.count("m")); + CHECK(!ac.entryMap["m"].wrongIndexType); +} + TEST_CASE_FIXTURE(ACFixture, "no_incompatible_self_calls") { - ScopedFastFlag selfCallAutocompleteFix2{"LuauSelfCallAutocompleteFix2", true}; + ScopedFastFlag selfCallAutocompleteFix3{"LuauSelfCallAutocompleteFix3", true}; check(R"( local t = {} @@ -2901,7 +2917,7 @@ t:@1 TEST_CASE_FIXTURE(ACFixture, "no_incompatible_self_calls_2") { - ScopedFastFlag selfCallAutocompleteFix2{"LuauSelfCallAutocompleteFix2", true}; + ScopedFastFlag selfCallAutocompleteFix3{"LuauSelfCallAutocompleteFix3", true}; check(R"( local f: (() -> number) & ((number) -> number) = function(x: number?) return 2 end @@ -2916,7 +2932,7 @@ t:@1 CHECK(ac.entryMap["f"].wrongIndexType); } -TEST_CASE_FIXTURE(ACFixture, "no_incompatible_self_calls_provisional") +TEST_CASE_FIXTURE(ACFixture, "do_wrong_compatible_self_calls") { check(R"( local t = {} @@ -2931,9 +2947,26 @@ t:@1 CHECK(!ac.entryMap["m"].wrongIndexType); } +TEST_CASE_FIXTURE(ACFixture, "no_wrong_compatible_self_calls_with_generics") +{ + ScopedFastFlag selfCallAutocompleteFix3{"LuauSelfCallAutocompleteFix3", true}; + + check(R"( +local t = {} +function t.m(a: T) end +t:@1 + )"); + + auto ac = autocomplete('1'); + + REQUIRE(ac.entryMap.count("m")); + // While this call is compatible with the type, this requires instantiation of a generic type which we don't perform + CHECK(ac.entryMap["m"].wrongIndexType); +} + TEST_CASE_FIXTURE(ACFixture, "string_prim_self_calls_are_fine") { - ScopedFastFlag selfCallAutocompleteFix2{"LuauSelfCallAutocompleteFix2", true}; + ScopedFastFlag selfCallAutocompleteFix3{"LuauSelfCallAutocompleteFix3", true}; check(R"( local s = "hello" @@ -2952,7 +2985,7 @@ s:@1 TEST_CASE_FIXTURE(ACFixture, "string_prim_non_self_calls_are_avoided") { - ScopedFastFlag selfCallAutocompleteFix2{"LuauSelfCallAutocompleteFix2", true}; + ScopedFastFlag selfCallAutocompleteFix3{"LuauSelfCallAutocompleteFix3", true}; check(R"( local s = "hello" @@ -2969,7 +3002,7 @@ s.@1 TEST_CASE_FIXTURE(ACBuiltinsFixture, "library_non_self_calls_are_fine") { - ScopedFastFlag selfCallAutocompleteFix2{"LuauSelfCallAutocompleteFix2", true}; + ScopedFastFlag selfCallAutocompleteFix3{"LuauSelfCallAutocompleteFix3", true}; check(R"( string.@1 @@ -3000,7 +3033,7 @@ table.@1 TEST_CASE_FIXTURE(ACBuiltinsFixture, "library_self_calls_are_invalid") { - ScopedFastFlag selfCallAutocompleteFix2{"LuauSelfCallAutocompleteFix2", true}; + ScopedFastFlag selfCallAutocompleteFix3{"LuauSelfCallAutocompleteFix3", true}; check(R"( string:@1 @@ -3012,8 +3045,11 @@ string:@1 CHECK(ac.entryMap["byte"].wrongIndexType == true); REQUIRE(ac.entryMap.count("char")); CHECK(ac.entryMap["char"].wrongIndexType == true); + + // We want the next test to evaluate to 'true', but we have to allow function defined with 'self' to be callable with ':' + // We may change the definition of the string metatable to not use 'self' types in the future (like byte/char/pack/unpack) REQUIRE(ac.entryMap.count("sub")); - CHECK(ac.entryMap["sub"].wrongIndexType == true); + CHECK(ac.entryMap["sub"].wrongIndexType == false); } TEST_CASE_FIXTURE(ACFixture, "source_module_preservation_and_invalidation") diff --git a/tests/Compiler.test.cpp b/tests/Compiler.test.cpp index c1917638..4b87c003 100644 --- a/tests/Compiler.test.cpp +++ b/tests/Compiler.test.cpp @@ -4352,8 +4352,6 @@ TEST_CASE("LoopUnrollControlFlow") {"LuauCompileLoopUnrollThresholdMaxBoost", 300}, }; - ScopedFastFlag sff("LuauCompileFoldBuiltins", true); - // break jumps to the end CHECK_EQ("\n" + compileFunction(R"( for i=1,3 do @@ -4669,8 +4667,6 @@ TEST_CASE("LoopUnrollCostBuiltins") {"LuauCompileLoopUnrollThresholdMaxBoost", 300}, }; - ScopedFastFlag sff("LuauCompileModelBuiltins", true); - // this loop uses builtins and is close to the cost budget so it's important that we model builtins as cheaper than regular calls CHECK_EQ("\n" + compileFunction(R"( function cipher(block, nonce) @@ -5893,8 +5889,6 @@ RETURN R0 2 TEST_CASE("OptimizationLevel") { - ScopedFastFlag sff("LuauAlwaysCaptureHotComments", true); - // at optimization level 1, no inlining is performed CHECK_EQ("\n" + compileFunction(R"( local function foo(a) @@ -5964,8 +5958,6 @@ RETURN R1 -1 TEST_CASE("BuiltinFolding") { - ScopedFastFlag sff("LuauCompileFoldBuiltins", true); - CHECK_EQ("\n" + compileFunction(R"( return math.abs(-42), @@ -6073,8 +6065,6 @@ RETURN R0 48 TEST_CASE("BuiltinFoldingProhibited") { - ScopedFastFlag sff("LuauCompileFoldBuiltins", true); - CHECK_EQ("\n" + compileFunction(R"( return math.abs(), @@ -6108,9 +6098,6 @@ L3: RETURN R0 -1 TEST_CASE("BuiltinFoldingMultret") { - ScopedFastFlag sff1("LuauCompileFoldBuiltins", true); - ScopedFastFlag sff2("LuauCompileBetterMultret", true); - CHECK_EQ("\n" + compileFunction(R"( local NoLanes: Lanes = --[[ ]] 0b0000000000000000000000000000000 local OffscreenLane: Lane = --[[ ]] 0b1000000000000000000000000000000 diff --git a/tests/Conformance.test.cpp b/tests/Conformance.test.cpp index fc8ab2f9..1339fa0c 100644 --- a/tests/Conformance.test.cpp +++ b/tests/Conformance.test.cpp @@ -316,7 +316,8 @@ TEST_CASE("Errors") TEST_CASE("Events") { - ScopedFastFlag sff("LuauLenTM", true); + ScopedFastFlag sff1("LuauLenTM", true); + ScopedFastFlag sff2("LuauBetterNewindex", true); runConformance("events.lua"); } @@ -490,8 +491,6 @@ static void populateRTTI(lua_State* L, Luau::TypeId type) TEST_CASE("Types") { - ScopedFastFlag sff("LuauCheckLenMT", true); - runConformance("types.lua", [](lua_State* L) { Luau::NullModuleResolver moduleResolver; Luau::InternalErrorReporter iceHandler; @@ -862,8 +861,6 @@ TEST_CASE("ApiCalls") TEST_CASE("ApiAtoms") { - ScopedFastFlag sff("LuauLazyAtoms", true); - StateRef globalState(luaL_newstate(), lua_close); lua_State* L = globalState.get(); diff --git a/tests/ConstraintSolver.test.cpp b/tests/ConstraintSolver.test.cpp index f521c667..e33f6570 100644 --- a/tests/ConstraintSolver.test.cpp +++ b/tests/ConstraintSolver.test.cpp @@ -9,7 +9,7 @@ using namespace Luau; -static TypeId requireBinding(NotNull scope, const char* name) +static TypeId requireBinding(NotNull scope, const char* name) { auto b = linearSearchForBinding(scope, name); LUAU_ASSERT(b.has_value()); @@ -26,7 +26,7 @@ TEST_CASE_FIXTURE(ConstraintGraphBuilderFixture, "hello") )"); cgb.visit(block); - NotNull rootScope = NotNull(cgb.rootScope); + NotNull rootScope = NotNull(cgb.rootScope); ConstraintSolver cs{&arena, rootScope}; @@ -46,7 +46,7 @@ TEST_CASE_FIXTURE(ConstraintGraphBuilderFixture, "generic_function") )"); cgb.visit(block); - NotNull rootScope = NotNull(cgb.rootScope); + NotNull rootScope = NotNull(cgb.rootScope); ConstraintSolver cs{&arena, rootScope}; @@ -73,7 +73,7 @@ TEST_CASE_FIXTURE(ConstraintGraphBuilderFixture, "proper_let_generalization") )"); cgb.visit(block); - NotNull rootScope = NotNull(cgb.rootScope); + NotNull rootScope = NotNull(cgb.rootScope); ToStringOptions opts; diff --git a/tests/Fixture.cpp b/tests/Fixture.cpp index c92c4457..90aa6236 100644 --- a/tests/Fixture.cpp +++ b/tests/Fixture.cpp @@ -258,7 +258,7 @@ std::optional Fixture::getType(const std::string& name) REQUIRE(module); if (FFlag::DebugLuauDeferredConstraintResolution) - return linearSearchForBinding(module->getModuleScope2(), name.c_str()); + return linearSearchForBinding(module->getModuleScope().get(), name.c_str()); else return lookupName(module->getModuleScope(), name); } @@ -434,7 +434,7 @@ BuiltinsFixture::BuiltinsFixture(bool freeze, bool prepareAutocomplete) ConstraintGraphBuilderFixture::ConstraintGraphBuilderFixture() : Fixture() - , cgb(mainModuleName, &arena, NotNull(&ice), frontend.getGlobalScope2()) + , cgb(mainModuleName, &arena, NotNull(&ice), frontend.getGlobalScope()) , forceTheFlag{"DebugLuauDeferredConstraintResolution", true} { BlockedTypeVar::nextIndex = 0; @@ -479,17 +479,17 @@ std::optional lookupName(ScopePtr scope, const std::string& name) return std::nullopt; } -std::optional linearSearchForBinding(Scope2* scope, const char* name) +std::optional linearSearchForBinding(Scope* scope, const char* name) { while (scope) { for (const auto& [n, ty] : scope->bindings) { if (n.astName() == name) - return ty; + return ty.typeId; } - scope = scope->parent; + scope = scope->parent.get(); } return std::nullopt; diff --git a/tests/Fixture.h b/tests/Fixture.h index 4bd6f1ea..a716fe9b 100644 --- a/tests/Fixture.h +++ b/tests/Fixture.h @@ -192,7 +192,7 @@ void dump(const std::vector& constraints); std::optional lookupName(ScopePtr scope, const std::string& name); // Warning: This function runs in O(n**2) -std::optional linearSearchForBinding(Scope2* scope, const char* name); +std::optional linearSearchForBinding(Scope* scope, const char* name); } // namespace Luau diff --git a/tests/Lexer.test.cpp b/tests/Lexer.test.cpp new file mode 100644 index 00000000..20d8d0d5 --- /dev/null +++ b/tests/Lexer.test.cpp @@ -0,0 +1,141 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +#include "Luau/Lexer.h" + +#include "Fixture.h" +#include "ScopedFlags.h" + +#include "doctest.h" + +using namespace Luau; + +TEST_SUITE_BEGIN("LexerTests"); + +TEST_CASE("broken_string_works") +{ + const std::string testInput = "[["; + Luau::Allocator alloc; + AstNameTable table(alloc); + Lexer lexer(testInput.c_str(), testInput.size(), table); + Lexeme lexeme = lexer.next(); + CHECK_EQ(lexeme.type, Lexeme::Type::BrokenString); + CHECK_EQ(lexeme.location, Luau::Location(Luau::Position(0, 0), Luau::Position(0, 2))); +} + +TEST_CASE("broken_comment") +{ + const std::string testInput = "--[[ "; + Luau::Allocator alloc; + AstNameTable table(alloc); + Lexer lexer(testInput.c_str(), testInput.size(), table); + Lexeme lexeme = lexer.next(); + CHECK_EQ(lexeme.type, Lexeme::Type::BrokenComment); + CHECK_EQ(lexeme.location, Luau::Location(Luau::Position(0, 0), Luau::Position(0, 6))); +} + +TEST_CASE("broken_comment_kept") +{ + const std::string testInput = "--[[ "; + Luau::Allocator alloc; + AstNameTable table(alloc); + Lexer lexer(testInput.c_str(), testInput.size(), table); + lexer.setSkipComments(true); + CHECK_EQ(lexer.next().type, Lexeme::Type::BrokenComment); +} + +TEST_CASE("comment_skipped") +{ + const std::string testInput = "-- "; + Luau::Allocator alloc; + AstNameTable table(alloc); + Lexer lexer(testInput.c_str(), testInput.size(), table); + lexer.setSkipComments(true); + CHECK_EQ(lexer.next().type, Lexeme::Type::Eof); +} + +TEST_CASE("multilineCommentWithLexemeInAndAfter") +{ + const std::string testInput = "--[[ function \n" + "]] end"; + Luau::Allocator alloc; + AstNameTable table(alloc); + Lexer lexer(testInput.c_str(), testInput.size(), table); + Lexeme comment = lexer.next(); + Lexeme end = lexer.next(); + + CHECK_EQ(comment.type, Lexeme::Type::BlockComment); + CHECK_EQ(comment.location, Luau::Location(Luau::Position(0, 0), Luau::Position(1, 2))); + CHECK_EQ(end.type, Lexeme::Type::ReservedEnd); + CHECK_EQ(end.location, Luau::Location(Luau::Position(1, 3), Luau::Position(1, 6))); +} + +TEST_CASE("testBrokenEscapeTolerant") +{ + const std::string testInput = "'\\3729472897292378'"; + Luau::Allocator alloc; + AstNameTable table(alloc); + Lexer lexer(testInput.c_str(), testInput.size(), table); + Lexeme item = lexer.next(); + + CHECK_EQ(item.type, Lexeme::QuotedString); + CHECK_EQ(item.location, Luau::Location(Luau::Position(0, 0), Luau::Position(0, int(testInput.size())))); +} + +TEST_CASE("testBigDelimiters") +{ + const std::string testInput = "--[===[\n" + "\n" + "\n" + "\n" + "]===]"; + Luau::Allocator alloc; + AstNameTable table(alloc); + Lexer lexer(testInput.c_str(), testInput.size(), table); + Lexeme item = lexer.next(); + + CHECK_EQ(item.type, Lexeme::Type::BlockComment); + CHECK_EQ(item.location, Luau::Location(Luau::Position(0, 0), Luau::Position(4, 5))); +} + +TEST_CASE("lookahead") +{ + const std::string testInput = "foo --[[ comment ]] bar : nil end"; + + Luau::Allocator alloc; + AstNameTable table(alloc); + Lexer lexer(testInput.c_str(), testInput.size(), table); + lexer.setSkipComments(true); + lexer.next(); // must call next() before reading data from lexer at least once + + CHECK_EQ(lexer.current().type, Lexeme::Name); + CHECK_EQ(lexer.current().name, std::string("foo")); + CHECK_EQ(lexer.lookahead().type, Lexeme::Name); + CHECK_EQ(lexer.lookahead().name, std::string("bar")); + + lexer.next(); + + CHECK_EQ(lexer.current().type, Lexeme::Name); + CHECK_EQ(lexer.current().name, std::string("bar")); + CHECK_EQ(lexer.lookahead().type, ':'); + + lexer.next(); + + CHECK_EQ(lexer.current().type, ':'); + CHECK_EQ(lexer.lookahead().type, Lexeme::ReservedNil); + + lexer.next(); + + CHECK_EQ(lexer.current().type, Lexeme::ReservedNil); + CHECK_EQ(lexer.lookahead().type, Lexeme::ReservedEnd); + + lexer.next(); + + CHECK_EQ(lexer.current().type, Lexeme::ReservedEnd); + CHECK_EQ(lexer.lookahead().type, Lexeme::Eof); + + lexer.next(); + + CHECK_EQ(lexer.current().type, Lexeme::Eof); + CHECK_EQ(lexer.lookahead().type, Lexeme::Eof); +} + +TEST_SUITE_END(); diff --git a/tests/Parser.test.cpp b/tests/Parser.test.cpp index c517853f..c5c019aa 100644 --- a/tests/Parser.test.cpp +++ b/tests/Parser.test.cpp @@ -97,138 +97,6 @@ TEST_CASE("initial_double_is_aligned") TEST_SUITE_END(); -TEST_SUITE_BEGIN("LexerTests"); - -TEST_CASE("broken_string_works") -{ - const std::string testInput = "[["; - Luau::Allocator alloc; - AstNameTable table(alloc); - Lexer lexer(testInput.c_str(), testInput.size(), table); - Lexeme lexeme = lexer.next(); - CHECK_EQ(lexeme.type, Lexeme::Type::BrokenString); - CHECK_EQ(lexeme.location, Luau::Location(Luau::Position(0, 0), Luau::Position(0, 2))); -} - -TEST_CASE("broken_comment") -{ - const std::string testInput = "--[[ "; - Luau::Allocator alloc; - AstNameTable table(alloc); - Lexer lexer(testInput.c_str(), testInput.size(), table); - Lexeme lexeme = lexer.next(); - CHECK_EQ(lexeme.type, Lexeme::Type::BrokenComment); - CHECK_EQ(lexeme.location, Luau::Location(Luau::Position(0, 0), Luau::Position(0, 6))); -} - -TEST_CASE("broken_comment_kept") -{ - const std::string testInput = "--[[ "; - Luau::Allocator alloc; - AstNameTable table(alloc); - Lexer lexer(testInput.c_str(), testInput.size(), table); - lexer.setSkipComments(true); - CHECK_EQ(lexer.next().type, Lexeme::Type::BrokenComment); -} - -TEST_CASE("comment_skipped") -{ - const std::string testInput = "-- "; - Luau::Allocator alloc; - AstNameTable table(alloc); - Lexer lexer(testInput.c_str(), testInput.size(), table); - lexer.setSkipComments(true); - CHECK_EQ(lexer.next().type, Lexeme::Type::Eof); -} - -TEST_CASE("multilineCommentWithLexemeInAndAfter") -{ - const std::string testInput = "--[[ function \n" - "]] end"; - Luau::Allocator alloc; - AstNameTable table(alloc); - Lexer lexer(testInput.c_str(), testInput.size(), table); - Lexeme comment = lexer.next(); - Lexeme end = lexer.next(); - - CHECK_EQ(comment.type, Lexeme::Type::BlockComment); - CHECK_EQ(comment.location, Luau::Location(Luau::Position(0, 0), Luau::Position(1, 2))); - CHECK_EQ(end.type, Lexeme::Type::ReservedEnd); - CHECK_EQ(end.location, Luau::Location(Luau::Position(1, 3), Luau::Position(1, 6))); -} - -TEST_CASE("testBrokenEscapeTolerant") -{ - const std::string testInput = "'\\3729472897292378'"; - Luau::Allocator alloc; - AstNameTable table(alloc); - Lexer lexer(testInput.c_str(), testInput.size(), table); - Lexeme item = lexer.next(); - - CHECK_EQ(item.type, Lexeme::QuotedString); - CHECK_EQ(item.location, Luau::Location(Luau::Position(0, 0), Luau::Position(0, int(testInput.size())))); -} - -TEST_CASE("testBigDelimiters") -{ - const std::string testInput = "--[===[\n" - "\n" - "\n" - "\n" - "]===]"; - Luau::Allocator alloc; - AstNameTable table(alloc); - Lexer lexer(testInput.c_str(), testInput.size(), table); - Lexeme item = lexer.next(); - - CHECK_EQ(item.type, Lexeme::Type::BlockComment); - CHECK_EQ(item.location, Luau::Location(Luau::Position(0, 0), Luau::Position(4, 5))); -} - -TEST_CASE("lookahead") -{ - const std::string testInput = "foo --[[ comment ]] bar : nil end"; - - Luau::Allocator alloc; - AstNameTable table(alloc); - Lexer lexer(testInput.c_str(), testInput.size(), table); - lexer.setSkipComments(true); - lexer.next(); // must call next() before reading data from lexer at least once - - CHECK_EQ(lexer.current().type, Lexeme::Name); - CHECK_EQ(lexer.current().name, std::string("foo")); - CHECK_EQ(lexer.lookahead().type, Lexeme::Name); - CHECK_EQ(lexer.lookahead().name, std::string("bar")); - - lexer.next(); - - CHECK_EQ(lexer.current().type, Lexeme::Name); - CHECK_EQ(lexer.current().name, std::string("bar")); - CHECK_EQ(lexer.lookahead().type, ':'); - - lexer.next(); - - CHECK_EQ(lexer.current().type, ':'); - CHECK_EQ(lexer.lookahead().type, Lexeme::ReservedNil); - - lexer.next(); - - CHECK_EQ(lexer.current().type, Lexeme::ReservedNil); - CHECK_EQ(lexer.lookahead().type, Lexeme::ReservedEnd); - - lexer.next(); - - CHECK_EQ(lexer.current().type, Lexeme::ReservedEnd); - CHECK_EQ(lexer.lookahead().type, Lexeme::Eof); - - lexer.next(); - - CHECK_EQ(lexer.current().type, Lexeme::Eof); - CHECK_EQ(lexer.lookahead().type, Lexeme::Eof); -} - -TEST_SUITE_END(); - TEST_SUITE_BEGIN("ParserTests"); TEST_CASE_FIXTURE(Fixture, "basic_parse") diff --git a/tests/ToString.test.cpp b/tests/ToString.test.cpp index 288af8da..8c03fe55 100644 --- a/tests/ToString.test.cpp +++ b/tests/ToString.test.cpp @@ -10,6 +10,7 @@ using namespace Luau; LUAU_FASTFLAG(LuauRecursiveTypeParameterRestriction); +LUAU_FASTFLAG(LuauSpecialTypesAsterisked); TEST_SUITE_BEGIN("ToString"); @@ -267,8 +268,16 @@ TEST_CASE_FIXTURE(Fixture, "quit_stringifying_type_when_length_is_exceeded") o.maxTypeLength = 40; CHECK_EQ(toString(requireType("f0"), o), "() -> ()"); CHECK_EQ(toString(requireType("f1"), o), "(() -> ()) -> () -> ()"); - CHECK_EQ(toString(requireType("f2"), o), "((() -> ()) -> () -> ()) -> (() -> ()) -> ... "); - CHECK_EQ(toString(requireType("f3"), o), "(((() -> ()) -> () -> ()) -> (() -> ()) -> ... "); + if (FFlag::LuauSpecialTypesAsterisked) + { + CHECK_EQ(toString(requireType("f2"), o), "((() -> ()) -> () -> ()) -> (() -> ()) -> ... *TRUNCATED*"); + CHECK_EQ(toString(requireType("f3"), o), "(((() -> ()) -> () -> ()) -> (() -> ()) -> ... *TRUNCATED*"); + } + else + { + CHECK_EQ(toString(requireType("f2"), o), "((() -> ()) -> () -> ()) -> (() -> ()) -> ... "); + CHECK_EQ(toString(requireType("f3"), o), "(((() -> ()) -> () -> ()) -> (() -> ()) -> ... "); + } } TEST_CASE_FIXTURE(Fixture, "stringifying_type_is_still_capped_when_exhaustive") @@ -286,8 +295,17 @@ TEST_CASE_FIXTURE(Fixture, "stringifying_type_is_still_capped_when_exhaustive") o.maxTypeLength = 40; CHECK_EQ(toString(requireType("f0"), o), "() -> ()"); CHECK_EQ(toString(requireType("f1"), o), "(() -> ()) -> () -> ()"); - CHECK_EQ(toString(requireType("f2"), o), "((() -> ()) -> () -> ()) -> (() -> ()) -> ... "); - CHECK_EQ(toString(requireType("f3"), o), "(((() -> ()) -> () -> ()) -> (() -> ()) -> ... "); + if (FFlag::LuauSpecialTypesAsterisked) + { + CHECK_EQ(toString(requireType("f2"), o), "((() -> ()) -> () -> ()) -> (() -> ()) -> ... *TRUNCATED*"); + CHECK_EQ(toString(requireType("f3"), o), "(((() -> ()) -> () -> ()) -> (() -> ()) -> ... *TRUNCATED*"); + } + else + { + CHECK_EQ(toString(requireType("f2"), o), "((() -> ()) -> () -> ()) -> (() -> ()) -> ... "); + CHECK_EQ(toString(requireType("f3"), o), "(((() -> ()) -> () -> ()) -> (() -> ()) -> ... "); + } + } TEST_CASE_FIXTURE(Fixture, "stringifying_table_type_correctly_use_matching_table_state_braces") @@ -497,7 +515,10 @@ local function target(callback: nil) return callback(4, "hello") end )"); LUAU_REQUIRE_ERRORS(result); - CHECK_EQ("(nil) -> ()", toString(requireType("target"))); + if (FFlag::LuauSpecialTypesAsterisked) + CHECK_EQ("(nil) -> (*error-type*)", toString(requireType("target"))); + else + CHECK_EQ("(nil) -> ()", toString(requireType("target"))); } TEST_CASE_FIXTURE(Fixture, "toStringGenericPack") diff --git a/tests/TypeInfer.anyerror.test.cpp b/tests/TypeInfer.anyerror.test.cpp index 4c5309e0..f4766104 100644 --- a/tests/TypeInfer.anyerror.test.cpp +++ b/tests/TypeInfer.anyerror.test.cpp @@ -13,6 +13,8 @@ using namespace Luau; +LUAU_FASTFLAG(LuauSpecialTypesAsterisked) + TEST_SUITE_BEGIN("TypeInferAnyError"); TEST_CASE_FIXTURE(Fixture, "for_in_loop_iterator_returns_any") @@ -94,7 +96,10 @@ TEST_CASE_FIXTURE(Fixture, "for_in_loop_iterator_is_error") LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK_EQ("", toString(requireType("a"))); + if (FFlag::LuauSpecialTypesAsterisked) + CHECK_EQ("*error-type*", toString(requireType("a"))); + else + CHECK_EQ("", toString(requireType("a"))); } TEST_CASE_FIXTURE(Fixture, "for_in_loop_iterator_is_error2") @@ -110,7 +115,10 @@ TEST_CASE_FIXTURE(Fixture, "for_in_loop_iterator_is_error2") LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK_EQ("", toString(requireType("a"))); + if (FFlag::LuauSpecialTypesAsterisked) + CHECK_EQ("*error-type*", toString(requireType("a"))); + else + CHECK_EQ("", toString(requireType("a"))); } TEST_CASE_FIXTURE(Fixture, "length_of_error_type_does_not_produce_an_error") @@ -225,7 +233,10 @@ TEST_CASE_FIXTURE(Fixture, "calling_error_type_yields_error") CHECK_EQ("unknown", err->name); - CHECK_EQ("", toString(requireType("a"))); + if (FFlag::LuauSpecialTypesAsterisked) + CHECK_EQ("*error-type*", toString(requireType("a"))); + else + CHECK_EQ("", toString(requireType("a"))); } TEST_CASE_FIXTURE(Fixture, "chain_calling_error_type_yields_error") @@ -234,7 +245,10 @@ TEST_CASE_FIXTURE(Fixture, "chain_calling_error_type_yields_error") local a = Utility.Create "Foo" {} )"); - CHECK_EQ("", toString(requireType("a"))); + if (FFlag::LuauSpecialTypesAsterisked) + CHECK_EQ("*error-type*", toString(requireType("a"))); + else + CHECK_EQ("", toString(requireType("a"))); } TEST_CASE_FIXTURE(BuiltinsFixture, "replace_every_free_type_when_unifying_a_complex_function_with_any") diff --git a/tests/TypeInfer.builtins.test.cpp b/tests/TypeInfer.builtins.test.cpp index 4f6adf97..0c878023 100644 --- a/tests/TypeInfer.builtins.test.cpp +++ b/tests/TypeInfer.builtins.test.cpp @@ -9,6 +9,7 @@ using namespace Luau; LUAU_FASTFLAG(LuauLowerBoundsCalculation); +LUAU_FASTFLAG(LuauSpecialTypesAsterisked); TEST_SUITE_BEGIN("BuiltinTests"); @@ -952,7 +953,10 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "table_freeze_is_generic") CHECK_EQ("number", toString(requireType("a"))); CHECK_EQ("string", toString(requireType("b"))); CHECK_EQ("boolean", toString(requireType("c"))); - CHECK_EQ("", toString(requireType("d"))); + if (FFlag::LuauSpecialTypesAsterisked) + CHECK_EQ("*error-type*", toString(requireType("d"))); + else + CHECK_EQ("", toString(requireType("d"))); } TEST_CASE_FIXTURE(BuiltinsFixture, "set_metatable_needs_arguments") diff --git a/tests/TypeInfer.functions.test.cpp b/tests/TypeInfer.functions.test.cpp index a634ba09..70773d95 100644 --- a/tests/TypeInfer.functions.test.cpp +++ b/tests/TypeInfer.functions.test.cpp @@ -14,6 +14,7 @@ using namespace Luau; LUAU_FASTFLAG(LuauLowerBoundsCalculation); +LUAU_FASTFLAG(LuauSpecialTypesAsterisked); TEST_SUITE_BEGIN("TypeInferFunctions"); @@ -907,13 +908,19 @@ TEST_CASE_FIXTURE(Fixture, "function_cast_error_uses_correct_language") REQUIRE(tm1); CHECK_EQ("(string) -> number", toString(tm1->wantedType)); - CHECK_EQ("(string, ) -> number", toString(tm1->givenType)); + if (FFlag::LuauSpecialTypesAsterisked) + CHECK_EQ("(string, *error-type*) -> number", toString(tm1->givenType)); + else + CHECK_EQ("(string, ) -> number", toString(tm1->givenType)); auto tm2 = get(result.errors[1]); REQUIRE(tm2); CHECK_EQ("(number, number) -> (number, number)", toString(tm2->wantedType)); - CHECK_EQ("(string, ) -> number", toString(tm2->givenType)); + if (FFlag::LuauSpecialTypesAsterisked) + CHECK_EQ("(string, *error-type*) -> number", toString(tm2->givenType)); + else + CHECK_EQ("(string, ) -> number", toString(tm2->givenType)); } TEST_CASE_FIXTURE(Fixture, "no_lossy_function_type") @@ -1526,10 +1533,21 @@ function t:b() return 2 end -- not OK )"); LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK_EQ(R"(Type '() -> number' could not be converted into '() -> number' + if (FFlag::LuauSpecialTypesAsterisked) + { + CHECK_EQ(R"(Type '(*error-type*) -> number' could not be converted into '() -> number' caused by: Argument count mismatch. Function expects 1 argument, but none are specified)", - toString(result.errors[0])); + toString(result.errors[0])); + } + else + { + CHECK_EQ(R"(Type '() -> number' could not be converted into '() -> number' +caused by: + Argument count mismatch. Function expects 1 argument, but none are specified)", + toString(result.errors[0])); + } + } TEST_CASE_FIXTURE(Fixture, "too_few_arguments_variadic") diff --git a/tests/TypeInfer.generics.test.cpp b/tests/TypeInfer.generics.test.cpp index 46258072..7a1af164 100644 --- a/tests/TypeInfer.generics.test.cpp +++ b/tests/TypeInfer.generics.test.cpp @@ -10,6 +10,7 @@ #include "doctest.h" LUAU_FASTFLAG(LuauCheckGenericHOFTypes) +LUAU_FASTFLAG(LuauSpecialTypesAsterisked) using namespace Luau; @@ -1003,7 +1004,10 @@ TEST_CASE_FIXTURE(Fixture, "no_stack_overflow_from_quantifying") std::optional t0 = getMainModule()->getModuleScope()->lookupType("t0"); REQUIRE(t0); - CHECK_EQ("", toString(t0->type)); + if (FFlag::LuauSpecialTypesAsterisked) + CHECK_EQ("*error-type*", toString(t0->type)); + else + CHECK_EQ("", toString(t0->type)); auto it = std::find_if(result.errors.begin(), result.errors.end(), [](TypeError& err) { return get(err); diff --git a/tests/TypeInfer.loops.test.cpp b/tests/TypeInfer.loops.test.cpp index 354b3996..9b10092c 100644 --- a/tests/TypeInfer.loops.test.cpp +++ b/tests/TypeInfer.loops.test.cpp @@ -13,6 +13,8 @@ using namespace Luau; +LUAU_FASTFLAG(LuauSpecialTypesAsterisked) + TEST_SUITE_BEGIN("TypeInferLoops"); TEST_CASE_FIXTURE(Fixture, "for_loop") @@ -142,7 +144,10 @@ TEST_CASE_FIXTURE(Fixture, "for_in_loop_on_error") CHECK_EQ(2, result.errors.size()); TypeId p = requireType("p"); - CHECK_EQ("", toString(p)); + if (FFlag::LuauSpecialTypesAsterisked) + CHECK_EQ("*error-type*", toString(p)); + else + CHECK_EQ("", toString(p)); } TEST_CASE_FIXTURE(Fixture, "for_in_loop_on_non_function") diff --git a/tests/TypeInfer.modules.test.cpp b/tests/TypeInfer.modules.test.cpp index 2343a7fa..a1d41339 100644 --- a/tests/TypeInfer.modules.test.cpp +++ b/tests/TypeInfer.modules.test.cpp @@ -12,6 +12,8 @@ using namespace Luau; +LUAU_FASTFLAG(LuauSpecialTypesAsterisked) + TEST_SUITE_BEGIN("TypeInferModules"); TEST_CASE_FIXTURE(BuiltinsFixture, "require") @@ -143,7 +145,10 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "require_module_that_does_not_export") auto hootyType = requireType(bModule, "Hooty"); - CHECK_EQ("", toString(hootyType)); + if (FFlag::LuauSpecialTypesAsterisked) + CHECK_EQ("*error-type*", toString(hootyType)); + else + CHECK_EQ("", toString(hootyType)); } TEST_CASE_FIXTURE(BuiltinsFixture, "warn_if_you_try_to_require_a_non_modulescript") @@ -244,7 +249,11 @@ local ModuleA = require(game.A) LUAU_REQUIRE_NO_ERRORS(result); std::optional oty = requireType("ModuleA"); - CHECK_EQ("", toString(*oty)); + + if (FFlag::LuauSpecialTypesAsterisked) + CHECK_EQ("*error-type*", toString(*oty)); + else + CHECK_EQ("", toString(*oty)); } TEST_CASE_FIXTURE(BuiltinsFixture, "do_not_modify_imported_types") diff --git a/tests/TypeInfer.operators.test.cpp b/tests/TypeInfer.operators.test.cpp index c90f0a4d..3d6c0193 100644 --- a/tests/TypeInfer.operators.test.cpp +++ b/tests/TypeInfer.operators.test.cpp @@ -490,8 +490,6 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "typecheck_unary_minus_error") TEST_CASE_FIXTURE(BuiltinsFixture, "typecheck_unary_len_error") { - ScopedFastFlag sff("LuauCheckLenMT", true); - CheckResult result = check(R"( --!strict local foo = { diff --git a/tests/TypeInfer.primitives.test.cpp b/tests/TypeInfer.primitives.test.cpp index d60c96e0..a06fd749 100644 --- a/tests/TypeInfer.primitives.test.cpp +++ b/tests/TypeInfer.primitives.test.cpp @@ -12,6 +12,7 @@ #include "doctest.h" LUAU_FASTFLAG(LuauDeduceFindMatchReturnTypes) +LUAU_FASTFLAG(LuauSpecialTypesAsterisked) using namespace Luau; @@ -49,7 +50,10 @@ TEST_CASE_FIXTURE(Fixture, "string_index") REQUIRE(nat); CHECK_EQ("string", toString(nat->ty)); - CHECK_EQ("", toString(requireType("t"))); + if (FFlag::LuauSpecialTypesAsterisked) + CHECK_EQ("*error-type*", toString(requireType("t"))); + else + CHECK_EQ("", toString(requireType("t"))); } TEST_CASE_FIXTURE(Fixture, "string_method") diff --git a/tests/TypeInfer.refinements.test.cpp b/tests/TypeInfer.refinements.test.cpp index cc8cdee3..8a1cadcf 100644 --- a/tests/TypeInfer.refinements.test.cpp +++ b/tests/TypeInfer.refinements.test.cpp @@ -8,6 +8,7 @@ #include "doctest.h" LUAU_FASTFLAG(LuauLowerBoundsCalculation) +LUAU_FASTFLAG(LuauSpecialTypesAsterisked) using namespace Luau; @@ -526,7 +527,10 @@ TEST_CASE_FIXTURE(Fixture, "type_narrow_to_vector") LUAU_REQUIRE_NO_ERRORS(result); - CHECK_EQ("", toString(requireTypeAtPosition({3, 28}))); + if (FFlag::LuauSpecialTypesAsterisked) + CHECK_EQ("*error-type*", toString(requireTypeAtPosition({3, 28}))); + else + CHECK_EQ("", toString(requireTypeAtPosition({3, 28}))); } TEST_CASE_FIXTURE(Fixture, "nonoptional_type_can_narrow_to_nil_if_sense_is_true") diff --git a/tests/TypeInfer.test.cpp b/tests/TypeInfer.test.cpp index 858e8ac0..af6185ef 100644 --- a/tests/TypeInfer.test.cpp +++ b/tests/TypeInfer.test.cpp @@ -16,6 +16,7 @@ LUAU_FASTFLAG(LuauLowerBoundsCalculation); LUAU_FASTFLAG(LuauFixLocationSpanTableIndexExpr); LUAU_FASTFLAG(DebugLuauDeferredConstraintResolution); +LUAU_FASTFLAG(LuauSpecialTypesAsterisked); using namespace Luau; @@ -237,10 +238,21 @@ TEST_CASE_FIXTURE(Fixture, "type_errors_infer_types") // TODO: Should we assert anything about these tests when DCR is being used? if (!FFlag::DebugLuauDeferredConstraintResolution) { - CHECK_EQ("", toString(requireType("c"))); - CHECK_EQ("", toString(requireType("d"))); - CHECK_EQ("", toString(requireType("e"))); - CHECK_EQ("", toString(requireType("f"))); + if (FFlag::LuauSpecialTypesAsterisked) + { + CHECK_EQ("*error-type*", toString(requireType("c"))); + CHECK_EQ("*error-type*", toString(requireType("d"))); + CHECK_EQ("*error-type*", toString(requireType("e"))); + CHECK_EQ("*error-type*", toString(requireType("f"))); + } + else + { + CHECK_EQ("", toString(requireType("c"))); + CHECK_EQ("", toString(requireType("d"))); + CHECK_EQ("", toString(requireType("e"))); + CHECK_EQ("", toString(requireType("f"))); + } + } } @@ -650,7 +662,11 @@ TEST_CASE_FIXTURE(Fixture, "no_stack_overflow_from_isoptional") std::optional t0 = getMainModule()->getModuleScope()->lookupType("t0"); REQUIRE(t0); - CHECK_EQ("", toString(t0->type)); + + if (FFlag::LuauSpecialTypesAsterisked) + CHECK_EQ("*error-type*", toString(t0->type)); + else + CHECK_EQ("", toString(t0->type)); auto it = std::find_if(result.errors.begin(), result.errors.end(), [](TypeError& err) { return get(err); diff --git a/tests/TypeInfer.tryUnify.test.cpp b/tests/TypeInfer.tryUnify.test.cpp index d51a38f8..e0a0e5b5 100644 --- a/tests/TypeInfer.tryUnify.test.cpp +++ b/tests/TypeInfer.tryUnify.test.cpp @@ -9,6 +9,8 @@ using namespace Luau; +LUAU_FASTFLAG(LuauSpecialTypesAsterisked) + struct TryUnifyFixture : Fixture { TypeArena arena; @@ -121,7 +123,10 @@ TEST_CASE_FIXTURE(TryUnifyFixture, "members_of_failed_typepack_unification_are_u LUAU_REQUIRE_ERROR_COUNT(1, result); CHECK_EQ("a", toString(requireType("a"))); - CHECK_EQ("", toString(requireType("b"))); + if (FFlag::LuauSpecialTypesAsterisked) + CHECK_EQ("*error-type*", toString(requireType("b"))); + else + CHECK_EQ("", toString(requireType("b"))); } TEST_CASE_FIXTURE(TryUnifyFixture, "result_of_failed_typepack_unification_is_constrained") @@ -136,7 +141,10 @@ TEST_CASE_FIXTURE(TryUnifyFixture, "result_of_failed_typepack_unification_is_con LUAU_REQUIRE_ERROR_COUNT(1, result); CHECK_EQ("a", toString(requireType("a"))); - CHECK_EQ("", toString(requireType("b"))); + if (FFlag::LuauSpecialTypesAsterisked) + CHECK_EQ("*error-type*", toString(requireType("b"))); + else + CHECK_EQ("", toString(requireType("b"))); CHECK_EQ("number", toString(requireType("c"))); } diff --git a/tests/TypeInfer.unionTypes.test.cpp b/tests/TypeInfer.unionTypes.test.cpp index 94918692..8eb485e9 100644 --- a/tests/TypeInfer.unionTypes.test.cpp +++ b/tests/TypeInfer.unionTypes.test.cpp @@ -7,6 +7,7 @@ #include "doctest.h" LUAU_FASTFLAG(LuauLowerBoundsCalculation) +LUAU_FASTFLAG(LuauSpecialTypesAsterisked) using namespace Luau; @@ -199,7 +200,10 @@ TEST_CASE_FIXTURE(Fixture, "index_on_a_union_type_with_missing_property") CHECK_EQ(mup->missing[0], *bTy); CHECK_EQ(mup->key, "x"); - CHECK_EQ("", toString(requireType("r"))); + if (FFlag::LuauSpecialTypesAsterisked) + CHECK_EQ("*error-type*", toString(requireType("r"))); + else + CHECK_EQ("", toString(requireType("r"))); } TEST_CASE_FIXTURE(Fixture, "index_on_a_union_type_with_one_property_of_type_any") diff --git a/tests/TypePack.test.cpp b/tests/TypePack.test.cpp index 35852c05..1087a24c 100644 --- a/tests/TypePack.test.cpp +++ b/tests/TypePack.test.cpp @@ -25,7 +25,7 @@ struct TypePackFixture TypePackId freshTypePack() { - typePacks.emplace_back(new TypePackVar{Unifiable::Free{{}}}); + typePacks.emplace_back(new TypePackVar{Unifiable::Free{TypeLevel{}}}); return typePacks.back().get(); } diff --git a/tests/conformance/errors.lua b/tests/conformance/errors.lua index 0b5aafed..6c0ac5e6 100644 --- a/tests/conformance/errors.lua +++ b/tests/conformance/errors.lua @@ -380,7 +380,8 @@ assert(ecall(function() return "a" + "b" end) == "attempt to perform arithmetic assert(ecall(function() return 1 > nil end) == "attempt to compare nil < number") -- note reversed order (by design) assert(ecall(function() return "a" <= 5 end) == "attempt to compare string <= number") -assert(ecall(function() local t = {} setmetatable(t, { __newindex = function(t,i,v) end }) t[nil] = 2 end) == "table index is nil") +assert(ecall(function() local t = {} t[nil] = 2 end) == "table index is nil") +assert(ecall(function() local t = {} t[0/0] = 2 end) == "table index is NaN") -- for loop type errors assert(ecall(function() for i='a',2 do end end) == "invalid 'for' initial value (number expected, got string)") diff --git a/tests/conformance/events.lua b/tests/conformance/events.lua index 42f1beda..6dcdbf0e 100644 --- a/tests/conformance/events.lua +++ b/tests/conformance/events.lua @@ -424,4 +424,57 @@ do assert(not ok and err:match("table or string expected")) end +-- verify that NaN/nil keys are passed to __newindex even though table assignment with them anywhere in the chain fails +do + assert(pcall(function() local t = {} t[nil] = 5 end) == false) + assert(pcall(function() local t = {} setmetatable(t, { __newindex = {} }) t[nil] = 5 end) == false) + assert(pcall(function() local t = {} setmetatable(t, { __newindex = function() end }) t[nil] = 5 end) == true) + + assert(pcall(function() local t = {} t[0/0] = 5 end) == false) + assert(pcall(function() local t = {} setmetatable(t, { __newindex = {} }) t[0/0] = 5 end) == false) + assert(pcall(function() local t = {} setmetatable(t, { __newindex = function() end }) t[0/0] = 5 end) == true) +end + +-- verify that __newindex gets called for frozen tables but only if the assignment is to a key absent from the table +do + local ni = {} + local t = table.create(2) + + t[1] = 42 + -- t[2] is semantically absent with storage allocated for it + + t.a = 1 + t.b = 2 + t.b = nil -- this sets 'b' value to nil but leaves key as is to exercise more internal paths -- no observable behavior change expected between b and other absent keys + + setmetatable(t, { __newindex = function(_, k, v) + assert(v == 42) + table.insert(ni, k) + end }) + table.freeze(t) + + -- "redundant" combinations are there to test all three of SETTABLEN/SETTABLEKS/SETTABLE + assert(pcall(function() t.a = 42 end) == false) + assert(pcall(function() t[1] = 42 end) == false) + assert(pcall(function() local key key = "a" t[key] = 42 end) == false) + assert(pcall(function() local key key = 1 t[key] = 42 end) == false) + + -- now repeat the same for keys absent from the table: b (semantically absent), c (physically absent), 2 (semantically absent), 3 (physically absent) + assert(pcall(function() t.b = 42 end) == true) + assert(pcall(function() t.c = 42 end) == true) + assert(pcall(function() local key key = "b" t[key] = 42 end) == true) + assert(pcall(function() local key key = "c" t[key] = 42 end) == true) + assert(pcall(function() t[2] = 42 end) == true) + assert(pcall(function() t[3] = 42 end) == true) + assert(pcall(function() local key key = 2 t[key] = 42 end) == true) + assert(pcall(function() local key key = 3 t[key] = 42 end) == true) + + -- validate the assignment sequence + local ei = { "b", "c", "b", "c", 2, 3, 2, 3 } + assert(#ni == #ei) + for k,v in ni do + assert(ei[k] == v) + end +end + return 'OK' diff --git a/tests/main.cpp b/tests/main.cpp index e7c4aed6..40ccd0b3 100644 --- a/tests/main.cpp +++ b/tests/main.cpp @@ -61,7 +61,7 @@ static bool debuggerPresent() static int testAssertionHandler(const char* expr, const char* file, int line, const char* function) { if (debuggerPresent()) - LUAU_DEBUGBREAK(); + LUAU_DEBUGBREAK(); ADD_FAIL_AT(file, line, "Assertion failed: ", std::string(expr)); return 1; @@ -298,5 +298,3 @@ int main(int argc, char** argv) } return result; } - - diff --git a/tools/faillist.txt b/tools/faillist.txt index dc74a6a9..40fc3c06 100644 --- a/tools/faillist.txt +++ b/tools/faillist.txt @@ -44,7 +44,6 @@ AutocompleteTest.as_types AutocompleteTest.autocomplete_boolean_singleton AutocompleteTest.autocomplete_default_type_pack_parameters AutocompleteTest.autocomplete_default_type_parameters -AutocompleteTest.autocomplete_documentation_symbols AutocompleteTest.autocomplete_end_with_fn_exprs AutocompleteTest.autocomplete_end_with_lambda AutocompleteTest.autocomplete_explicit_type_pack @@ -65,47 +64,34 @@ AutocompleteTest.autocomplete_until_in_repeat AutocompleteTest.autocomplete_while_middle_keywords AutocompleteTest.autocompleteProp_index_function_metamethod_is_variadic AutocompleteTest.bias_toward_inner_scope -AutocompleteTest.comments AutocompleteTest.cyclic_table +AutocompleteTest.do_compatible_self_calls AutocompleteTest.do_not_overwrite_context_sensitive_kws AutocompleteTest.do_not_suggest_internal_module_type -AutocompleteTest.do_not_suggest_synthetic_table_name -AutocompleteTest.dont_offer_any_suggestions_from_the_end_of_a_comment +AutocompleteTest.do_wrong_compatible_self_calls AutocompleteTest.dont_offer_any_suggestions_from_within_a_broken_comment AutocompleteTest.dont_offer_any_suggestions_from_within_a_broken_comment_at_the_very_end_of_the_file AutocompleteTest.dont_offer_any_suggestions_from_within_a_comment AutocompleteTest.dont_suggest_local_before_its_definition -AutocompleteTest.empty_program AutocompleteTest.function_expr_params AutocompleteTest.function_in_assignment_has_parentheses AutocompleteTest.function_in_assignment_has_parentheses_2 AutocompleteTest.function_parameters AutocompleteTest.function_result_passed_to_function_has_parentheses -AutocompleteTest.function_type_types AutocompleteTest.generic_types -AutocompleteTest.get_member_completions -AutocompleteTest.get_string_completions -AutocompleteTest.get_suggestions_for_new_statement AutocompleteTest.get_suggestions_for_the_very_start_of_the_script AutocompleteTest.global_function_params AutocompleteTest.global_functions_are_not_scoped_lexically AutocompleteTest.if_then_else_elseif_completions AutocompleteTest.if_then_else_full_keywords -AutocompleteTest.keyword_members AutocompleteTest.keyword_methods AutocompleteTest.keyword_types -AutocompleteTest.leave_numbers_alone AutocompleteTest.library_non_self_calls_are_fine AutocompleteTest.library_self_calls_are_invalid AutocompleteTest.local_function AutocompleteTest.local_function_params AutocompleteTest.local_functions_fall_out_of_scope -AutocompleteTest.local_initializer -AutocompleteTest.local_initializer_2 -AutocompleteTest.local_names -AutocompleteTest.local_types_builtin AutocompleteTest.method_call_inside_function_body -AutocompleteTest.method_call_inside_if_conditional AutocompleteTest.module_type_members AutocompleteTest.modules_with_types AutocompleteTest.nested_member_completions @@ -114,16 +100,13 @@ AutocompleteTest.no_function_name_suggestions AutocompleteTest.no_incompatible_self_calls AutocompleteTest.no_incompatible_self_calls_2 AutocompleteTest.no_incompatible_self_calls_on_class -AutocompleteTest.no_incompatible_self_calls_provisional -AutocompleteTest.not_the_var_we_are_defining +AutocompleteTest.no_wrong_compatible_self_calls_with_generics AutocompleteTest.optional_members AutocompleteTest.private_types -AutocompleteTest.recommend_statement_starting_keywords AutocompleteTest.recursive_function AutocompleteTest.recursive_function_global AutocompleteTest.recursive_function_local AutocompleteTest.return_types -AutocompleteTest.skip_current_local AutocompleteTest.sometimes_the_metatable_is_an_error AutocompleteTest.source_module_preservation_and_invalidation AutocompleteTest.statement_between_two_statements @@ -154,9 +137,7 @@ AutocompleteTest.type_correct_suggestion_in_table AutocompleteTest.type_scoping_easy AutocompleteTest.unsealed_table AutocompleteTest.unsealed_table_2 -AutocompleteTest.user_defined_globals AutocompleteTest.user_defined_local_functions_in_own_definition -BuiltinDefinitionsTest.lib_documentation_symbols BuiltinTests.aliased_string_format BuiltinTests.assert_removes_falsy_types BuiltinTests.assert_removes_falsy_types2 @@ -231,16 +212,10 @@ BuiltinTests.tonumber_returns_optional_number_type BuiltinTests.tonumber_returns_optional_number_type2 BuiltinTests.xpcall DefinitionTests.class_definition_function_prop -DefinitionTests.class_definitions_cannot_extend_non_class -DefinitionTests.class_definitions_cannot_overload_non_function DefinitionTests.declaring_generic_functions DefinitionTests.definition_file_class_function_args DefinitionTests.definition_file_classes DefinitionTests.definition_file_loading -DefinitionTests.definitions_documentation_symbols -DefinitionTests.documentation_symbols_dont_attach_to_persistent_types -DefinitionTests.load_definition_file_errors_do_not_pollute_global_scope -DefinitionTests.no_cyclic_defined_classes DefinitionTests.single_class_type_identity_in_global_types FrontendTest.accumulate_cached_errors FrontendTest.accumulate_cached_errors_in_consistent_order @@ -256,12 +231,9 @@ FrontendTest.cycle_error_paths FrontendTest.cycle_errors_can_be_fixed FrontendTest.cycle_incremental_type_surface FrontendTest.cycle_incremental_type_surface_longer -FrontendTest.discard_type_graphs FrontendTest.dont_recheck_script_that_hasnt_been_marked_dirty FrontendTest.dont_reparse_clean_file_when_linting FrontendTest.environments -FrontendTest.find_a_require -FrontendTest.find_a_require_inside_a_function FrontendTest.ignore_require_to_nonexistent_file FrontendTest.imported_table_modification_2 FrontendTest.it_should_be_safe_to_stringify_errors_when_full_type_graph_is_discarded @@ -269,29 +241,97 @@ FrontendTest.no_use_after_free_with_type_fun_instantiation FrontendTest.nocheck_cycle_used_by_checked FrontendTest.nocheck_modules_are_typed FrontendTest.produce_errors_for_unchanged_file_with_a_syntax_error -FrontendTest.produce_errors_for_unchanged_file_with_errors FrontendTest.re_report_type_error_in_required_file -FrontendTest.real_source FrontendTest.recheck_if_dependent_script_is_dirty +FrontendTest.reexport_cyclic_type +FrontendTest.reexport_type_alias FrontendTest.report_require_to_nonexistent_file FrontendTest.report_syntax_error_in_required_file FrontendTest.reports_errors_from_multiple_sources FrontendTest.stats_are_not_reset_between_checks -FrontendTest.test_lint_uses_correct_config -FrontendTest.test_pruneParentSegments FrontendTest.trace_requires_in_nonstrict_mode -FrontendTest.typecheck_twice_for_ast_types +GenericsTests.apply_type_function_nested_generics1 +GenericsTests.apply_type_function_nested_generics2 +GenericsTests.better_mismatch_error_messages +GenericsTests.bound_tables_do_not_clone_original_fields +GenericsTests.check_generic_typepack_function +GenericsTests.check_mutual_generic_functions +GenericsTests.correctly_instantiate_polymorphic_member_functions +GenericsTests.do_not_always_instantiate_generic_intersection_types +GenericsTests.do_not_infer_generic_functions +GenericsTests.dont_substitute_bound_types +GenericsTests.dont_unify_bound_types +GenericsTests.duplicate_generic_type_packs +GenericsTests.duplicate_generic_types +GenericsTests.error_detailed_function_mismatch_generic_pack +GenericsTests.error_detailed_function_mismatch_generic_types +GenericsTests.factories_of_generics +GenericsTests.function_arguments_can_be_polytypes +GenericsTests.function_results_can_be_polytypes +GenericsTests.generic_argument_count_too_few +GenericsTests.generic_argument_count_too_many +GenericsTests.generic_factories +GenericsTests.generic_functions_dont_cache_type_parameters +GenericsTests.generic_functions_in_types +GenericsTests.generic_functions_should_be_memory_safe +GenericsTests.generic_table_method +GenericsTests.generic_type_pack_syntax +GenericsTests.generic_type_pack_unification1 +GenericsTests.generic_type_pack_unification2 +GenericsTests.generic_type_pack_unification3 +GenericsTests.infer_generic_function_function_argument +GenericsTests.infer_generic_function_function_argument_overloaded +GenericsTests.infer_generic_lib_function_function_argument +GenericsTests.infer_generic_property +GenericsTests.inferred_local_vars_can_be_polytypes +GenericsTests.instantiate_cyclic_generic_function +GenericsTests.instantiate_generic_function_in_assignments +GenericsTests.instantiate_generic_function_in_assignments2 +GenericsTests.instantiated_function_argument_names +GenericsTests.instantiation_sharing_types +GenericsTests.local_vars_can_be_instantiated_polytypes +GenericsTests.mutable_state_polymorphism +GenericsTests.no_stack_overflow_from_quantifying +GenericsTests.properties_can_be_instantiated_polytypes +GenericsTests.properties_can_be_polytypes +GenericsTests.rank_N_types_via_typeof +GenericsTests.reject_clashing_generic_and_pack_names +GenericsTests.self_recursive_instantiated_param +GenericsTests.substitution_with_bound_table +GenericsTests.typefuns_sharing_types +GenericsTests.variadic_generics +IntersectionTypes.argument_is_intersection +IntersectionTypes.error_detailed_intersection_all +IntersectionTypes.error_detailed_intersection_part +IntersectionTypes.fx_intersection_as_argument +IntersectionTypes.fx_union_as_argument_fails +IntersectionTypes.index_on_an_intersection_type_with_all_parts_missing_the_property +IntersectionTypes.index_on_an_intersection_type_with_mixed_types +IntersectionTypes.index_on_an_intersection_type_with_one_part_missing_the_property +IntersectionTypes.index_on_an_intersection_type_with_one_property_of_type_any +IntersectionTypes.index_on_an_intersection_type_with_property_guaranteed_to_exist +IntersectionTypes.index_on_an_intersection_type_works_at_arbitrary_depth +IntersectionTypes.no_stack_overflow_from_flattenintersection +IntersectionTypes.overload_is_not_a_function +IntersectionTypes.propagates_name +IntersectionTypes.select_correct_union_fn +IntersectionTypes.should_still_pick_an_overload_whose_arguments_are_unions +IntersectionTypes.table_combines +IntersectionTypes.table_combines_missing +IntersectionTypes.table_extra_ok +IntersectionTypes.table_intersection_setmetatable +IntersectionTypes.table_intersection_write +IntersectionTypes.table_intersection_write_sealed +IntersectionTypes.table_intersection_write_sealed_indirect +IntersectionTypes.table_write_sealed_indirect isSubtype.functions_and_any isSubtype.intersection_of_functions_of_different_arities isSubtype.intersection_of_tables isSubtype.table_with_any_prop isSubtype.table_with_table_prop isSubtype.tables -Linter.BuiltinGlobalWrite Linter.DeprecatedApi -Linter.LocalShadowGlobal Linter.TableOperations -Linter.use_all_parent_scopes_for_globals ModuleTests.any_persistance_does_not_leak ModuleTests.builtin_types_point_into_globalTypes_arena ModuleTests.clone_self_property @@ -342,45 +382,237 @@ Normalize.skip_force_normal_on_external_types Normalize.union_of_distinct_free_types Normalize.variadic_tail_is_marked_normal Normalize.visiting_a_type_twice_is_not_considered_normal -ParseErrorRecovery.empty_function_type_error_recovery -ParseErrorRecovery.extra_table_indexer_recovery -ParseErrorRecovery.extra_token_in_consume -ParseErrorRecovery.extra_token_in_consume_match -ParseErrorRecovery.extra_token_in_consume_match_end ParseErrorRecovery.generic_type_list_recovery -ParseErrorRecovery.multiple_parse_errors ParseErrorRecovery.recovery_of_parenthesized_expressions -ParseErrorRecovery.statement_error_recovery_expected -ParseErrorRecovery.statement_error_recovery_unexpected -ParserTests.break_return_not_last_error -ParserTests.continue_not_last_error -ParserTests.error_on_confusable -ParserTests.error_on_non_utf8_sequence -ParserTests.error_on_unicode -ParserTests.export_is_an_identifier_only_when_followed_by_type -ParserTests.functions_cannot_have_return_annotations_if_extensions_are_disabled -ParserTests.illegal_type_alias_if_extensions_are_disabled -ParserTests.incomplete_statement_error -ParserTests.local_cannot_have_annotation_with_extensions_disabled -ParserTests.parse_compound_assignment_error_call -ParserTests.parse_compound_assignment_error_multiple -ParserTests.parse_compound_assignment_error_not_lvalue -ParserTests.parse_error_function_call -ParserTests.parse_error_function_call_newline -ParserTests.parse_error_messages -ParserTests.parse_error_table_literal -ParserTests.parse_error_type_name -ParserTests.parse_nesting_based_end_detection ParserTests.parse_nesting_based_end_detection_failsafe_earlier ParserTests.parse_nesting_based_end_detection_local_function -ParserTests.parse_nesting_based_end_detection_local_repeat -ParserTests.parse_nesting_based_end_detection_nested -ParserTests.parse_nesting_based_end_detection_single_line -ParserTests.parse_numbers_error -ParserTests.parse_numbers_range_error -ParserTests.stop_if_line_ends_with_hyphen -ParserTests.type_alias_error_messages +ProvisionalTests.bail_early_if_unification_is_too_complicated +ProvisionalTests.choose_the_right_overload_for_pcall +ProvisionalTests.constrained_is_level_dependent +ProvisionalTests.discriminate_from_x_not_equal_to_nil +ProvisionalTests.do_not_ice_when_trying_to_pick_first_of_generic_type_pack +ProvisionalTests.error_on_eq_metamethod_returning_a_type_other_than_boolean +ProvisionalTests.free_is_not_bound_to_any +ProvisionalTests.function_returns_many_things_but_first_of_it_is_forgotten +ProvisionalTests.greedy_inference_with_shared_self_triggers_function_with_no_returns +ProvisionalTests.invariant_table_properties_means_instantiating_tables_in_call_is_unsound +ProvisionalTests.it_should_be_agnostic_of_actual_size +ProvisionalTests.lower_bounds_calculation_is_too_permissive_with_overloaded_higher_order_functions +ProvisionalTests.lvalue_equals_another_lvalue_with_no_overlap +ProvisionalTests.normalization_fails_on_certain_kinds_of_cyclic_tables +ProvisionalTests.operator_eq_completely_incompatible +ProvisionalTests.pcall_returns_at_least_two_value_but_function_returns_nothing +ProvisionalTests.setmetatable_constrains_free_type_into_free_table +ProvisionalTests.typeguard_inference_incomplete +ProvisionalTests.weird_fail_to_unify_type_pack +ProvisionalTests.weirditer_should_not_loop_forever +ProvisionalTests.while_body_are_also_refined +ProvisionalTests.xpcall_returns_what_f_returns +RefinementTest.and_constraint +RefinementTest.and_or_peephole_refinement +RefinementTest.apply_refinements_on_astexprindexexpr_whose_subscript_expr_is_constant_string +RefinementTest.assert_a_to_be_truthy_then_assert_a_to_be_number +RefinementTest.assert_non_binary_expressions_actually_resolve_constraints +RefinementTest.assign_table_with_refined_property_with_a_similar_type_is_illegal +RefinementTest.call_a_more_specific_function_using_typeguard +RefinementTest.correctly_lookup_a_shadowed_local_that_which_was_previously_refined +RefinementTest.correctly_lookup_property_whose_base_was_previously_refined +RefinementTest.correctly_lookup_property_whose_base_was_previously_refined2 +RefinementTest.discriminate_from_isa_of_x +RefinementTest.discriminate_from_truthiness_of_x +RefinementTest.discriminate_on_properties_of_disjoint_tables_where_that_property_is_true_or_false +RefinementTest.discriminate_tag +RefinementTest.either_number_or_string +RefinementTest.eliminate_subclasses_of_instance +RefinementTest.falsiness_of_TruthyPredicate_narrows_into_nil +RefinementTest.free_type_is_equal_to_an_lvalue +RefinementTest.impossible_type_narrow_is_not_an_error +RefinementTest.index_on_a_refined_property +RefinementTest.invert_is_truthy_constraint +RefinementTest.invert_is_truthy_constraint_ifelse_expression +RefinementTest.is_truthy_constraint +RefinementTest.is_truthy_constraint_ifelse_expression +RefinementTest.lvalue_is_equal_to_a_term +RefinementTest.lvalue_is_equal_to_another_lvalue +RefinementTest.lvalue_is_not_nil +RefinementTest.merge_should_be_fully_agnostic_of_hashmap_ordering +RefinementTest.narrow_property_of_a_bounded_variable +RefinementTest.narrow_this_large_union +RefinementTest.nonoptional_type_can_narrow_to_nil_if_sense_is_true +RefinementTest.not_a_and_not_b +RefinementTest.not_a_and_not_b2 +RefinementTest.not_a_or_not_b +RefinementTest.not_a_or_not_b2 +RefinementTest.not_and_constraint +RefinementTest.not_t_or_some_prop_of_t +RefinementTest.or_predicate_with_truthy_predicates +RefinementTest.parenthesized_expressions_are_followed_through +RefinementTest.refine_a_property_not_to_be_nil_through_an_intersection_table +RefinementTest.refine_the_correct_types_opposite_of_when_a_is_not_number_or_string +RefinementTest.refine_unknowns +RefinementTest.string_not_equal_to_string_or_nil +RefinementTest.term_is_equal_to_an_lvalue +RefinementTest.truthy_constraint_on_properties +RefinementTest.type_assertion_expr_carry_its_constraints +RefinementTest.type_comparison_ifelse_expression +RefinementTest.type_guard_can_filter_for_intersection_of_tables +RefinementTest.type_guard_can_filter_for_overloaded_function +RefinementTest.type_guard_narrowed_into_nothingness +RefinementTest.type_narrow_for_all_the_userdata +RefinementTest.type_narrow_to_vector +RefinementTest.typeguard_cast_free_table_to_vector +RefinementTest.typeguard_cast_instance_or_vector3_to_vector +RefinementTest.typeguard_doesnt_leak_to_elseif +RefinementTest.typeguard_in_assert_position +RefinementTest.typeguard_in_if_condition_position +RefinementTest.typeguard_narrows_for_functions +RefinementTest.typeguard_narrows_for_table +RefinementTest.typeguard_not_to_be_string +RefinementTest.typeguard_only_look_up_types_from_global_scope +RefinementTest.unknown_lvalue_is_not_synonymous_with_other_on_not_equal +RefinementTest.what_nonsensical_condition +RefinementTest.x_as_any_if_x_is_instance_elseif_x_is_table +RefinementTest.x_is_not_instance_or_else_not_part RuntimeLimits.typescript_port_of_Result_type +TableTests.a_free_shape_can_turn_into_a_scalar_if_it_is_compatible +TableTests.a_free_shape_cannot_turn_into_a_scalar_if_it_is_not_compatible +TableTests.access_index_metamethod_that_returns_variadic +TableTests.accidentally_checked_prop_in_opposite_branch +TableTests.assigning_to_an_unsealed_table_with_string_literal_should_infer_new_properties_over_indexer +TableTests.augment_nested_table +TableTests.augment_table +TableTests.builtin_table_names +TableTests.call_method +TableTests.call_method_with_explicit_self_argument +TableTests.cannot_augment_sealed_table +TableTests.cannot_call_tables +TableTests.cannot_change_type_of_unsealed_table_prop +TableTests.casting_sealed_tables_with_props_into_table_with_indexer +TableTests.casting_tables_with_props_into_table_with_indexer3 +TableTests.casting_tables_with_props_into_table_with_indexer4 +TableTests.casting_unsealed_tables_with_props_into_table_with_indexer +TableTests.checked_prop_too_early +TableTests.common_table_element_general +TableTests.common_table_element_inner_index +TableTests.common_table_element_inner_prop +TableTests.common_table_element_list +TableTests.common_table_element_union_assignment +TableTests.common_table_element_union_in_call +TableTests.common_table_element_union_in_call_tail +TableTests.common_table_element_union_in_prop +TableTests.confusing_indexing +TableTests.defining_a_method_for_a_builtin_sealed_table_must_fail +TableTests.defining_a_method_for_a_local_sealed_table_must_fail +TableTests.defining_a_method_for_a_local_unsealed_table_is_ok +TableTests.defining_a_self_method_for_a_builtin_sealed_table_must_fail +TableTests.defining_a_self_method_for_a_local_sealed_table_must_fail +TableTests.defining_a_self_method_for_a_local_unsealed_table_is_ok +TableTests.dont_crash_when_setmetatable_does_not_produce_a_metatabletypevar +TableTests.dont_hang_when_trying_to_look_up_in_cyclic_metatable_index +TableTests.dont_invalidate_the_properties_iterator_of_free_table_when_rolled_back +TableTests.dont_leak_free_table_props +TableTests.dont_quantify_table_that_belongs_to_outer_scope +TableTests.dont_seal_an_unsealed_table_by_passing_it_to_a_function_that_takes_a_sealed_table +TableTests.dont_suggest_exact_match_keys +TableTests.error_detailed_indexer_key +TableTests.error_detailed_indexer_value +TableTests.error_detailed_metatable_prop +TableTests.error_detailed_prop +TableTests.error_detailed_prop_nested +TableTests.expected_indexer_from_table_union +TableTests.expected_indexer_value_type_extra +TableTests.expected_indexer_value_type_extra_2 +TableTests.explicitly_typed_table +TableTests.explicitly_typed_table_error +TableTests.explicitly_typed_table_with_indexer +TableTests.found_like_key_in_table_function_call +TableTests.found_like_key_in_table_property_access +TableTests.found_multiple_like_keys +TableTests.function_calls_produces_sealed_table_given_unsealed_table +TableTests.generalize_table_argument +TableTests.getmetatable_returns_pointer_to_metatable +TableTests.give_up_after_one_metatable_index_look_up +TableTests.hide_table_error_properties +TableTests.indexer_fn +TableTests.indexer_on_sealed_table_must_unify_with_free_table +TableTests.indexer_table +TableTests.indexing_from_a_table_should_prefer_properties_when_possible +TableTests.inequality_operators_imply_exactly_matching_types +TableTests.infer_array_2 +TableTests.infer_indexer_from_value_property_in_literal +TableTests.inferred_return_type_of_free_table +TableTests.inferring_crazy_table_should_also_be_quick +TableTests.instantiate_table_cloning +TableTests.instantiate_table_cloning_2 +TableTests.instantiate_table_cloning_3 +TableTests.instantiate_tables_at_scope_level +TableTests.invariant_table_properties_means_instantiating_tables_in_assignment_is_unsound +TableTests.leaking_bad_metatable_errors +TableTests.length_operator_intersection +TableTests.length_operator_non_table_union +TableTests.length_operator_union +TableTests.length_operator_union_errors +TableTests.less_exponential_blowup_please +TableTests.meta_add +TableTests.meta_add_both_ways +TableTests.meta_add_inferred +TableTests.metatable_mismatch_should_fail +TableTests.missing_metatable_for_sealed_tables_do_not_get_inferred +TableTests.mixed_tables_with_implicit_numbered_keys +TableTests.MixedPropertiesAndIndexers +TableTests.nil_assign_doesnt_hit_indexer +TableTests.okay_to_add_property_to_unsealed_tables_by_function_call +TableTests.only_ascribe_synthetic_names_at_module_scope +TableTests.oop_indexer_works +TableTests.oop_polymorphic +TableTests.open_table_unification_2 +TableTests.pass_a_union_of_tables_to_a_function_that_requires_a_table +TableTests.pass_a_union_of_tables_to_a_function_that_requires_a_table_2 +TableTests.pass_incompatible_union_to_a_generic_table_without_crashing +TableTests.passing_compatible_unions_to_a_generic_table_without_crashing +TableTests.persistent_sealed_table_is_immutable +TableTests.property_lookup_through_tabletypevar_metatable +TableTests.quantify_even_that_table_was_never_exported_at_all +TableTests.quantify_metatables_of_metatables_of_table +TableTests.quantifying_a_bound_var_works +TableTests.reasonable_error_when_adding_a_nonexistent_property_to_an_array_like_table +TableTests.recursive_metatable_type_call +TableTests.result_is_always_any_if_lhs_is_any +TableTests.result_is_bool_for_equality_operators_if_lhs_is_any +TableTests.right_table_missing_key +TableTests.right_table_missing_key2 +TableTests.scalar_is_a_subtype_of_a_compatible_polymorphic_shape_type +TableTests.scalar_is_not_a_subtype_of_a_compatible_polymorphic_shape_type +TableTests.setmetatable_cant_be_used_to_mutate_global_types +TableTests.shared_selfs +TableTests.shared_selfs_from_free_param +TableTests.shared_selfs_through_metatables +TableTests.table_function_check_use_after_free +TableTests.table_indexing_error_location +TableTests.table_insert_should_cope_with_optional_properties_in_nonstrict +TableTests.table_insert_should_cope_with_optional_properties_in_strict +TableTests.table_length +TableTests.table_param_row_polymorphism_2 +TableTests.table_param_row_polymorphism_3 +TableTests.table_simple_call +TableTests.table_subtyping_with_extra_props_dont_report_multiple_errors +TableTests.table_subtyping_with_missing_props_dont_report_multiple_errors +TableTests.table_subtyping_with_missing_props_dont_report_multiple_errors2 +TableTests.table_unifies_into_map +TableTests.tables_get_names_from_their_locals +TableTests.tc_member_function +TableTests.tc_member_function_2 +TableTests.top_table_type +TableTests.type_mismatch_on_massive_table_is_cut_short +TableTests.unification_of_unions_in_a_self_referential_type +TableTests.unifying_tables_shouldnt_uaf1 +TableTests.unifying_tables_shouldnt_uaf2 +TableTests.used_colon_correctly +TableTests.used_colon_instead_of_dot +TableTests.used_dot_instead_of_colon +TableTests.used_dot_instead_of_colon_but_correctly +TableTests.user_defined_table_types_are_named +TableTests.width_subtyping ToDot.bound_table ToDot.class ToDot.function @@ -401,9 +633,13 @@ ToString.toStringNamedFunction_id ToString.toStringNamedFunction_map ToString.toStringNamedFunction_overrides_param_names ToString.toStringNamedFunction_variadics -TranspilerTests.attach_types TranspilerTests.type_lists_should_be_emitted_correctly TranspilerTests.types_should_not_be_considered_cyclic_if_they_are_not_recursive +TryUnifyTests.cli_41095_concat_log_in_sealed_table_unification +TryUnifyTests.members_of_failed_typepack_unification_are_unified_with_errorType +TryUnifyTests.result_of_failed_typepack_unification_is_constrained +TryUnifyTests.typepack_unification_should_trim_free_tails +TryUnifyTests.variadics_should_use_reversed_properly TypeAliases.basic_alias TypeAliases.cannot_steal_hoisted_type_alias TypeAliases.cli_38393_recursive_intersection_oom @@ -446,6 +682,37 @@ TypeAliases.type_alias_local_synthetic_mutation TypeAliases.type_alias_of_an_imported_recursive_generic_type TypeAliases.type_alias_of_an_imported_recursive_type TypeAliases.use_table_name_and_generic_params_in_errors +TypeInfer.check_expr_recursion_limit +TypeInfer.checking_should_not_ice +TypeInfer.cli_50041_committing_txnlog_in_apollo_client_error +TypeInfer.cyclic_follow +TypeInfer.do_not_bind_a_free_table_to_a_union_containing_that_table +TypeInfer.dont_report_type_errors_within_an_AstStatError +TypeInfer.follow_on_new_types_in_substitution +TypeInfer.free_typevars_introduced_within_control_flow_constructs_do_not_get_an_elevated_TypeLevel +TypeInfer.globals +TypeInfer.globals2 +TypeInfer.index_expr_should_be_checked +TypeInfer.infer_assignment_value_types +TypeInfer.infer_assignment_value_types_mutable_lval +TypeInfer.infer_through_group_expr +TypeInfer.infer_type_assertion_value_type +TypeInfer.no_heap_use_after_free_error +TypeInfer.no_infinite_loop_when_trying_to_unify_uh_this +TypeInfer.no_stack_overflow_from_isoptional +TypeInfer.no_stack_overflow_from_isoptional2 +TypeInfer.recursive_metatable_crash +TypeInfer.tc_after_error_recovery_no_replacement_name_in_error +TypeInfer.tc_if_else_expressions1 +TypeInfer.tc_if_else_expressions2 +TypeInfer.tc_if_else_expressions_expected_type_1 +TypeInfer.tc_if_else_expressions_expected_type_2 +TypeInfer.tc_if_else_expressions_expected_type_3 +TypeInfer.tc_if_else_expressions_type_union +TypeInfer.type_infer_recursion_limit_no_ice +TypeInfer.types stored in astResolvedTypes +TypeInfer.warn_on_lowercase_parent_property +TypeInfer.weird_case TypeInferAnyError.any_type_propagates TypeInferAnyError.assign_prop_to_table_by_calling_any_yields_any TypeInferAnyError.call_to_any_yields_any @@ -496,26 +763,289 @@ TypeInferClasses.we_can_infer_that_a_parameter_must_be_a_particular_class TypeInferClasses.we_can_report_when_someone_is_trying_to_use_a_table_rather_than_a_class TypeInferFunctions.another_indirect_function_case_where_it_is_ok_to_provide_too_many_arguments TypeInferFunctions.another_recursive_local_function +TypeInferFunctions.calling_function_with_anytypepack_doesnt_leak_free_types +TypeInferFunctions.calling_function_with_incorrect_argument_type_yields_errors_spanning_argument TypeInferFunctions.cannot_hoist_interior_defns_into_signature TypeInferFunctions.check_function_before_lambda_that_uses_it TypeInferFunctions.complicated_return_types_require_an_explicit_annotation TypeInferFunctions.cyclic_function_type_in_args TypeInferFunctions.dont_give_other_overloads_message_if_only_one_argument_matching_overload_exists +TypeInferFunctions.dont_infer_parameter_types_for_functions_from_their_call_site +TypeInferFunctions.dont_mutate_the_underlying_head_of_typepack_when_calling_with_self TypeInferFunctions.duplicate_functions_with_different_signatures_not_allowed_in_nonstrict +TypeInferFunctions.error_detailed_function_mismatch_arg +TypeInferFunctions.error_detailed_function_mismatch_arg_count +TypeInferFunctions.error_detailed_function_mismatch_ret +TypeInferFunctions.error_detailed_function_mismatch_ret_count +TypeInferFunctions.error_detailed_function_mismatch_ret_mult TypeInferFunctions.first_argument_can_be_optional +TypeInferFunctions.free_is_not_bound_to_unknown TypeInferFunctions.func_expr_doesnt_leak_free +TypeInferFunctions.function_cast_error_uses_correct_language +TypeInferFunctions.function_decl_non_self_sealed_overwrite +TypeInferFunctions.function_decl_non_self_sealed_overwrite_2 +TypeInferFunctions.function_decl_non_self_unsealed_overwrite +TypeInferFunctions.function_decl_quantify_right_type +TypeInferFunctions.function_does_not_return_enough_values +TypeInferFunctions.function_statement_sealed_table_assignment_through_indexer TypeInferFunctions.higher_order_function_2 TypeInferFunctions.higher_order_function_4 +TypeInferFunctions.ignored_return_values +TypeInferFunctions.inconsistent_higher_order_function +TypeInferFunctions.inconsistent_return_types +TypeInferFunctions.infer_anonymous_function_arguments +TypeInferFunctions.infer_anonymous_function_arguments_outside_call TypeInferFunctions.infer_return_type_from_selected_overload TypeInferFunctions.infer_that_function_does_not_return_a_table +TypeInferFunctions.inferred_higher_order_functions_are_quantified_at_the_right_time +TypeInferFunctions.inferred_higher_order_functions_are_quantified_at_the_right_time2 TypeInferFunctions.it_is_ok_not_to_supply_enough_retvals TypeInferFunctions.it_is_ok_to_oversaturate_a_higher_order_function_argument TypeInferFunctions.list_all_overloads_if_no_overload_takes_given_argument_count TypeInferFunctions.list_only_alternative_overloads_that_match_argument_count TypeInferFunctions.mutual_recursion +TypeInferFunctions.no_lossy_function_type +TypeInferFunctions.occurs_check_failure_in_function_return_type +TypeInferFunctions.quantify_constrained_types +TypeInferFunctions.record_matching_overload TypeInferFunctions.recursive_function TypeInferFunctions.recursive_local_function +TypeInferFunctions.report_exiting_without_return_nonstrict +TypeInferFunctions.report_exiting_without_return_strict +TypeInferFunctions.return_type_by_overload +TypeInferFunctions.strict_mode_ok_with_missing_arguments +TypeInferFunctions.too_few_arguments_variadic +TypeInferFunctions.too_few_arguments_variadic_generic +TypeInferFunctions.too_few_arguments_variadic_generic2 TypeInferFunctions.too_many_arguments +TypeInferFunctions.too_many_return_values TypeInferFunctions.toposort_doesnt_break_mutual_recursion TypeInferFunctions.vararg_function_is_quantified TypeInferFunctions.vararg_functions_should_allow_calls_of_any_types_and_size +TypeInferLoops.correctly_scope_locals_while +TypeInferLoops.for_in_loop +TypeInferLoops.for_in_loop_error_on_factory_not_returning_the_right_amount_of_values +TypeInferLoops.for_in_loop_error_on_iterator_requiring_args_but_none_given +TypeInferLoops.for_in_loop_on_error +TypeInferLoops.for_in_loop_on_non_function +TypeInferLoops.for_in_loop_should_fail_with_non_function_iterator +TypeInferLoops.for_in_loop_where_iteratee_is_free +TypeInferLoops.for_in_loop_with_custom_iterator +TypeInferLoops.for_in_loop_with_next +TypeInferLoops.for_in_with_a_custom_iterator_should_type_check +TypeInferLoops.for_in_with_an_iterator_of_type_any +TypeInferLoops.for_in_with_just_one_iterator_is_ok +TypeInferLoops.fuzz_fail_missing_instantitation_follow +TypeInferLoops.ipairs_produces_integral_indices +TypeInferLoops.loop_iter_basic +TypeInferLoops.loop_iter_iter_metamethod +TypeInferLoops.loop_iter_no_indexer_nonstrict +TypeInferLoops.loop_iter_no_indexer_strict +TypeInferLoops.loop_iter_trailing_nil +TypeInferLoops.loop_typecheck_crash_on_empty_optional +TypeInferLoops.properly_infer_iteratee_is_a_free_table +TypeInferLoops.repeat_loop +TypeInferLoops.repeat_loop_condition_binds_to_its_block +TypeInferLoops.symbols_in_repeat_block_should_not_be_visible_beyond_until_condition +TypeInferLoops.unreachable_code_after_infinite_loop +TypeInferLoops.varlist_declared_by_for_in_loop_should_be_free +TypeInferLoops.while_loop +TypeInferModules.bound_free_table_export_is_ok +TypeInferModules.constrained_anyification_clone_immutable_types +TypeInferModules.custom_require_global +TypeInferModules.do_not_modify_imported_types +TypeInferModules.do_not_modify_imported_types_2 +TypeInferModules.do_not_modify_imported_types_3 +TypeInferModules.do_not_modify_imported_types_4 +TypeInferModules.general_require_call_expression +TypeInferModules.general_require_type_mismatch +TypeInferModules.module_type_conflict +TypeInferModules.module_type_conflict_instantiated +TypeInferModules.require +TypeInferModules.require_a_variadic_function +TypeInferModules.require_failed_module +TypeInferModules.require_module_that_does_not_export +TypeInferModules.require_types +TypeInferModules.type_error_of_unknown_qualified_type +TypeInferModules.warn_if_you_try_to_require_a_non_modulescript +TypeInferOOP.dont_suggest_using_colon_rather_than_dot_if_another_overload_works +TypeInferOOP.dont_suggest_using_colon_rather_than_dot_if_it_wont_help_2 +TypeInferOOP.dont_suggest_using_colon_rather_than_dot_if_not_defined_with_colon +TypeInferOOP.inferred_methods_of_free_tables_have_the_same_level_as_the_enclosing_table +TypeInferOOP.inferring_hundreds_of_self_calls_should_not_suffocate_memory +TypeInferOOP.method_depends_on_table +TypeInferOOP.methods_are_topologically_sorted +TypeInferOOP.nonstrict_self_mismatch_tail +TypeInferOOP.object_constructor_can_refer_to_method_of_self +TypeInferOOP.table_oop +TypeInferOperators.and_adds_boolean +TypeInferOperators.and_adds_boolean_no_superfluous_union +TypeInferOperators.and_binexps_dont_unify +TypeInferOperators.and_or_ternary +TypeInferOperators.CallAndOrOfFunctions +TypeInferOperators.cannot_compare_tables_that_do_not_have_the_same_metatable +TypeInferOperators.cannot_indirectly_compare_types_that_do_not_have_a_metatable +TypeInferOperators.cannot_indirectly_compare_types_that_do_not_offer_overloaded_ordering_operators +TypeInferOperators.cli_38355_recursive_union +TypeInferOperators.compare_numbers +TypeInferOperators.compare_strings +TypeInferOperators.compound_assign_basic +TypeInferOperators.compound_assign_metatable +TypeInferOperators.compound_assign_mismatch_metatable +TypeInferOperators.compound_assign_mismatch_op +TypeInferOperators.compound_assign_mismatch_result +TypeInferOperators.concat_op_on_free_lhs_and_string_rhs +TypeInferOperators.concat_op_on_string_lhs_and_free_rhs +TypeInferOperators.disallow_string_and_types_without_metatables_from_arithmetic_binary_ops +TypeInferOperators.dont_strip_nil_from_rhs_or_operator +TypeInferOperators.equality_operations_succeed_if_any_union_branch_succeeds +TypeInferOperators.error_on_invalid_operand_types_to_relational_operators +TypeInferOperators.error_on_invalid_operand_types_to_relational_operators2 +TypeInferOperators.expected_types_through_binary_and +TypeInferOperators.expected_types_through_binary_or +TypeInferOperators.in_nonstrict_mode_strip_nil_from_intersections_when_considering_relational_operators +TypeInferOperators.infer_any_in_all_modes_when_lhs_is_unknown +TypeInferOperators.operator_eq_operands_are_not_subtypes_of_each_other_but_has_overlap +TypeInferOperators.operator_eq_verifies_types_do_intersect +TypeInferOperators.or_joins_types +TypeInferOperators.or_joins_types_with_no_extras +TypeInferOperators.primitive_arith_no_metatable +TypeInferOperators.primitive_arith_no_metatable_with_follows +TypeInferOperators.primitive_arith_possible_metatable +TypeInferOperators.produce_the_correct_error_message_when_comparing_a_table_with_a_metatable_with_one_that_does_not +TypeInferOperators.refine_and_or +TypeInferOperators.some_primitive_binary_ops +TypeInferOperators.strict_binary_op_where_lhs_unknown +TypeInferOperators.strip_nil_from_lhs_or_operator +TypeInferOperators.strip_nil_from_lhs_or_operator2 +TypeInferOperators.typecheck_overloaded_multiply_that_is_an_intersection +TypeInferOperators.typecheck_overloaded_multiply_that_is_an_intersection_on_rhs +TypeInferOperators.typecheck_unary_len_error +TypeInferOperators.typecheck_unary_minus +TypeInferOperators.typecheck_unary_minus_error +TypeInferOperators.unary_not_is_boolean +TypeInferOperators.unknown_type_in_comparison +TypeInferOperators.UnknownGlobalCompoundAssign +TypeInferPrimitives.cannot_call_primitives +TypeInferPrimitives.CheckMethodsOfNumber +TypeInferPrimitives.string_function_other +TypeInferPrimitives.string_index +TypeInferPrimitives.string_length +TypeInferPrimitives.string_method +TypeInferUnknownNever.array_like_table_of_never_is_inhabitable +TypeInferUnknownNever.assign_to_global_which_is_never +TypeInferUnknownNever.assign_to_local_which_is_never +TypeInferUnknownNever.assign_to_prop_which_is_never +TypeInferUnknownNever.assign_to_subscript_which_is_never +TypeInferUnknownNever.call_never +TypeInferUnknownNever.dont_unify_operands_if_one_of_the_operand_is_never_in_any_ordering_operators +TypeInferUnknownNever.index_on_never +TypeInferUnknownNever.index_on_union_of_tables_for_properties_that_is_never +TypeInferUnknownNever.index_on_union_of_tables_for_properties_that_is_sorta_never +TypeInferUnknownNever.length_of_never +TypeInferUnknownNever.math_operators_and_never +TypeInferUnknownNever.never_is_reflexive +TypeInferUnknownNever.never_subtype_and_string_supertype +TypeInferUnknownNever.string_subtype_and_unknown_supertype +TypeInferUnknownNever.table_with_prop_of_type_never_is_also_reflexive +TypeInferUnknownNever.table_with_prop_of_type_never_is_uninhabitable +TypeInferUnknownNever.type_packs_containing_never_is_itself_uninhabitable +TypeInferUnknownNever.type_packs_containing_never_is_itself_uninhabitable2 +TypeInferUnknownNever.unary_minus_of_never +TypeInferUnknownNever.unknown_is_reflexive +TypePackTests.cyclic_type_packs +TypePackTests.higher_order_function +TypePackTests.multiple_varargs_inference_are_not_confused +TypePackTests.no_return_size_should_be_zero +TypePackTests.pack_tail_unification_check +TypePackTests.parenthesized_varargs_returns_any +TypePackTests.self_and_varargs_should_work +TypePackTests.type_alias_backwards_compatible +TypePackTests.type_alias_default_export +TypePackTests.type_alias_default_mixed_self +TypePackTests.type_alias_default_type_chained +TypePackTests.type_alias_default_type_errors +TypePackTests.type_alias_default_type_explicit +TypePackTests.type_alias_default_type_pack_explicit +TypePackTests.type_alias_default_type_pack_self_chained_tp +TypePackTests.type_alias_default_type_pack_self_tp +TypePackTests.type_alias_default_type_pack_self_ty +TypePackTests.type_alias_default_type_self +TypePackTests.type_alias_default_type_skip_brackets +TypePackTests.type_alias_defaults_confusing_types +TypePackTests.type_alias_defaults_recursive_type +TypePackTests.type_alias_type_pack_explicit +TypePackTests.type_alias_type_pack_explicit_multi +TypePackTests.type_alias_type_pack_explicit_multi_tostring +TypePackTests.type_alias_type_pack_multi +TypePackTests.type_alias_type_pack_variadic +TypePackTests.type_alias_type_packs +TypePackTests.type_alias_type_packs_errors +TypePackTests.type_alias_type_packs_import +TypePackTests.type_alias_type_packs_nested +TypePackTests.type_pack_hidden_free_tail_infinite_growth +TypePackTests.type_pack_type_parameters +TypePackTests.varargs_inference_through_multiple_scopes +TypePackTests.variadic_argument_tail +TypePackTests.variadic_pack_syntax +TypePackTests.variadic_packs +TypeSingletons.bool_singleton_subtype +TypeSingletons.bool_singletons +TypeSingletons.bool_singletons_mismatch +TypeSingletons.enums_using_singletons +TypeSingletons.enums_using_singletons_mismatch +TypeSingletons.enums_using_singletons_subtyping +TypeSingletons.error_detailed_tagged_union_mismatch_bool +TypeSingletons.error_detailed_tagged_union_mismatch_string +TypeSingletons.function_call_with_singletons_mismatch +TypeSingletons.if_then_else_expression_singleton_options +TypeSingletons.indexing_on_string_singletons +TypeSingletons.indexing_on_union_of_string_singletons +TypeSingletons.no_widening_from_callsites +TypeSingletons.overloaded_function_call_with_singletons +TypeSingletons.overloaded_function_call_with_singletons_mismatch +TypeSingletons.return_type_of_f_is_not_widened +TypeSingletons.string_singleton_subtype +TypeSingletons.string_singletons +TypeSingletons.string_singletons_escape_chars +TypeSingletons.string_singletons_mismatch +TypeSingletons.table_insert_with_a_singleton_argument +TypeSingletons.table_properties_alias_or_parens_is_indexer +TypeSingletons.table_properties_singleton_strings +TypeSingletons.table_properties_singleton_strings_mismatch +TypeSingletons.table_properties_type_error_escapes +TypeSingletons.tagged_unions_immutable_tag +TypeSingletons.tagged_unions_using_singletons +TypeSingletons.tagged_unions_using_singletons_mismatch +TypeSingletons.taking_the_length_of_string_singleton +TypeSingletons.taking_the_length_of_union_of_string_singleton +TypeSingletons.widen_the_supertype_if_it_is_free_and_subtype_has_singleton +TypeSingletons.widening_happens_almost_everywhere +TypeSingletons.widening_happens_almost_everywhere_except_for_tables +TypeVarTests.visit_once +UnionTypes.error_detailed_optional +UnionTypes.error_detailed_union_all +UnionTypes.error_detailed_union_part +UnionTypes.error_takes_optional_arguments +UnionTypes.index_on_a_union_type_with_missing_property +UnionTypes.index_on_a_union_type_with_mixed_types +UnionTypes.index_on_a_union_type_with_one_optional_property +UnionTypes.index_on_a_union_type_with_one_property_of_type_any +UnionTypes.index_on_a_union_type_with_property_guaranteed_to_exist +UnionTypes.index_on_a_union_type_works_at_arbitrary_depth +UnionTypes.optional_arguments +UnionTypes.optional_assignment_errors +UnionTypes.optional_call_error +UnionTypes.optional_field_access_error +UnionTypes.optional_index_error +UnionTypes.optional_length_error +UnionTypes.optional_missing_key_error_details +UnionTypes.optional_union_follow +UnionTypes.optional_union_functions +UnionTypes.optional_union_members +UnionTypes.optional_union_methods +UnionTypes.return_types_can_be_disjoint +UnionTypes.table_union_write_indirect +UnionTypes.unify_sealed_table_union_check +UnionTypes.unify_unsealed_table_union_check +UnionTypes.union_equality_comparisons diff --git a/tools/test_dcr.py b/tools/test_dcr.py index 8b090bcd..94ff5ca2 100644 --- a/tools/test_dcr.py +++ b/tools/test_dcr.py @@ -14,6 +14,11 @@ def loadFailList(): with open(FAIL_LIST_PATH) as f: return set(map(str.strip, f.readlines())) +def safeParseInt(i, default=0): + try: + return int(i) + except ValueError: + return default class Handler(x.ContentHandler): def __init__(self, failList): @@ -22,6 +27,8 @@ class Handler(x.ContentHandler): self.results = {} # {DottedName: TrueIfTheTestPassed} + self.numSkippedTests = 0 + def startElement(self, name, attrs): if name == "TestSuite": self.currentTest.append(attrs["name"]) @@ -30,10 +37,7 @@ class Handler(x.ContentHandler): elif name == "OverallResultsAsserts": if self.currentTest: - try: - failed = 0 != int(attrs["failures"]) - except ValueError: - failed = False + failed = 0 != safeParseInt(attrs["failures"]) dottedName = ".".join(self.currentTest) shouldFail = dottedName in self.failList @@ -45,6 +49,9 @@ class Handler(x.ContentHandler): self.results[dottedName] = not failed + elif name == 'OverallResultsTestCases': + self.numSkippedTests = safeParseInt(attrs.get("skipped", 0)) + def endElement(self, name): if name == "TestCase": self.currentTest.pop() @@ -111,6 +118,10 @@ def main(): print(name, file=f) print("Updated faillist.txt") + if handler.numSkippedTests > 0: + print('{} test(s) were skipped! That probably means that a test segfaulted!'.format(handler.numSkippedTests), file=sys.stderr) + sys.exit(1) + sys.exit( 0 if all( From d3b566c258bee3bdccb655c034a11bfc48a586e3 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Thu, 28 Jul 2022 21:24:07 -0700 Subject: [PATCH 325/399] Sync to upstream/release/538 (#616) --- .gitignore | 1 + Analysis/include/Luau/Constraint.h | 5 +- .../include/Luau/ConstraintGraphBuilder.h | 85 +-- Analysis/include/Luau/ConstraintSolver.h | 6 +- .../include/Luau/ConstraintSolverLogger.h | 4 +- Analysis/include/Luau/Frontend.h | 4 +- Analysis/include/Luau/Module.h | 2 - Analysis/include/Luau/Quantify.h | 4 +- Analysis/include/Luau/Scope.h | 32 +- Analysis/include/Luau/TypeVar.h | 8 +- Analysis/include/Luau/Unifiable.h | 12 +- Analysis/src/Autocomplete.cpp | 51 +- Analysis/src/ConstraintGraphBuilder.cpp | 216 +++--- Analysis/src/ConstraintSolver.cpp | 14 +- Analysis/src/ConstraintSolverLogger.cpp | 12 +- Analysis/src/EmbeddedBuiltinDefinitions.cpp | 6 +- Analysis/src/Frontend.cpp | 22 +- Analysis/src/JsonEncoder.cpp | 4 +- Analysis/src/Module.cpp | 27 +- Analysis/src/Quantify.cpp | 20 +- Analysis/src/Scope.cpp | 36 +- Analysis/src/ToString.cpp | 66 +- Analysis/src/TypeChecker2.cpp | 17 +- Analysis/src/TypeInfer.cpp | 7 +- Analysis/src/Unifiable.cpp | 6 +- Ast/src/Parser.cpp | 49 +- CLI/ReplEntry.cpp | 2 - Compiler/src/Compiler.cpp | 55 +- Compiler/src/CostModel.cpp | 4 +- Sources.cmake | 1 + VM/src/lapi.cpp | 3 - VM/src/ldblib.cpp | 13 +- VM/src/ldebug.cpp | 4 +- VM/src/lstring.cpp | 8 +- VM/src/lstrlib.cpp | 10 +- VM/src/ltable.cpp | 24 +- VM/src/ltable.h | 6 + VM/src/lvmexecute.cpp | 8 +- VM/src/lvmutils.cpp | 50 +- tests/Autocomplete.test.cpp | 54 +- tests/Compiler.test.cpp | 13 - tests/Conformance.test.cpp | 7 +- tests/ConstraintSolver.test.cpp | 8 +- tests/Fixture.cpp | 10 +- tests/Fixture.h | 2 +- tests/Lexer.test.cpp | 141 ++++ tests/Parser.test.cpp | 132 ---- tests/ToString.test.cpp | 31 +- tests/TypeInfer.anyerror.test.cpp | 22 +- tests/TypeInfer.builtins.test.cpp | 6 +- tests/TypeInfer.functions.test.cpp | 26 +- tests/TypeInfer.generics.test.cpp | 6 +- tests/TypeInfer.loops.test.cpp | 7 +- tests/TypeInfer.modules.test.cpp | 13 +- tests/TypeInfer.operators.test.cpp | 2 - tests/TypeInfer.primitives.test.cpp | 6 +- tests/TypeInfer.refinements.test.cpp | 6 +- tests/TypeInfer.test.cpp | 26 +- tests/TypeInfer.tryUnify.test.cpp | 12 +- tests/TypeInfer.unionTypes.test.cpp | 6 +- tests/TypePack.test.cpp | 2 +- tests/conformance/errors.lua | 3 +- tests/conformance/events.lua | 53 ++ tests/main.cpp | 4 +- tools/faillist.txt | 678 ++++++++++++++++-- tools/test_dcr.py | 19 +- 66 files changed, 1445 insertions(+), 754 deletions(-) create mode 100644 tests/Lexer.test.cpp diff --git a/.gitignore b/.gitignore index 5688dff5..ec5e6578 100644 --- a/.gitignore +++ b/.gitignore @@ -8,3 +8,4 @@ /default.prof* /fuzz-* /luau +__pycache__ diff --git a/Analysis/include/Luau/Constraint.h b/Analysis/include/Luau/Constraint.h index dcfb14b4..9911c4d3 100644 --- a/Analysis/include/Luau/Constraint.h +++ b/Analysis/include/Luau/Constraint.h @@ -12,7 +12,8 @@ namespace Luau { -struct Scope2; +struct Scope; + struct TypeVar; using TypeId = const TypeVar*; @@ -38,7 +39,7 @@ struct GeneralizationConstraint { TypeId generalizedType; TypeId sourceType; - Scope2* scope; + Scope* scope; }; // subType ~ inst superType diff --git a/Analysis/include/Luau/ConstraintGraphBuilder.h b/Analysis/include/Luau/ConstraintGraphBuilder.h index 513446fd..695b6cd5 100644 --- a/Analysis/include/Luau/ConstraintGraphBuilder.h +++ b/Analysis/include/Luau/ConstraintGraphBuilder.h @@ -17,21 +17,22 @@ namespace Luau { -struct Scope2; +struct Scope; +using ScopePtr = std::shared_ptr; struct ConstraintGraphBuilder { // A list of all the scopes in the module. This vector holds ownership of the // scope pointers; the scopes themselves borrow pointers to other scopes to // define the scope hierarchy. - std::vector>> scopes; + std::vector> scopes; ModuleName moduleName; SingletonTypes& singletonTypes; const NotNull arena; // The root scope of the module we're generating constraints for. // This is null when the CGB is initially constructed. - Scope2* rootScope; + Scope* rootScope; // A mapping of AST node to TypeId. DenseHashMap astTypes{nullptr}; // A mapping of AST node to TypePackId. @@ -50,42 +51,42 @@ struct ConstraintGraphBuilder // Occasionally constraint generation needs to produce an ICE. const NotNull ice; - NotNull globalScope; + NotNull globalScope; - ConstraintGraphBuilder(const ModuleName& moduleName, TypeArena* arena, NotNull ice, NotNull globalScope); + ConstraintGraphBuilder(const ModuleName& moduleName, TypeArena* arena, NotNull ice, NotNull globalScope); /** * Fabricates a new free type belonging to a given scope. * @param scope the scope the free type belongs to. */ - TypeId freshType(NotNull scope); + TypeId freshType(const ScopePtr& scope); /** * Fabricates a new free type pack belonging to a given scope. * @param scope the scope the free type pack belongs to. */ - TypePackId freshTypePack(NotNull scope); + TypePackId freshTypePack(const ScopePtr& scope); /** * Fabricates a scope that is a child of another scope. * @param location the lexical extent of the scope in the source code. * @param parent the parent scope of the new scope. Must not be null. */ - NotNull childScope(Location location, NotNull parent); + ScopePtr childScope(Location location, const ScopePtr& parent); /** * Adds a new constraint with no dependencies to a given scope. * @param scope the scope to add the constraint to. * @param cv the constraint variant to add. */ - void addConstraint(NotNull scope, ConstraintV cv); + void addConstraint(const ScopePtr& scope, ConstraintV cv); /** * Adds a constraint to a given scope. * @param scope the scope to add the constraint to. Must not be null. * @param c the constraint to add. */ - void addConstraint(NotNull scope, std::unique_ptr c); + void addConstraint(const ScopePtr& scope, std::unique_ptr c); /** * The entry point to the ConstraintGraphBuilder. This will construct a set @@ -94,23 +95,23 @@ struct ConstraintGraphBuilder */ void visit(AstStatBlock* block); - void visitBlockWithoutChildScope(NotNull scope, AstStatBlock* block); + void visitBlockWithoutChildScope(const ScopePtr& scope, AstStatBlock* block); - void visit(NotNull scope, AstStat* stat); - void visit(NotNull scope, AstStatBlock* block); - void visit(NotNull scope, AstStatLocal* local); - void visit(NotNull scope, AstStatFor* for_); - void visit(NotNull scope, AstStatLocalFunction* function); - void visit(NotNull scope, AstStatFunction* function); - void visit(NotNull scope, AstStatReturn* ret); - void visit(NotNull scope, AstStatAssign* assign); - void visit(NotNull scope, AstStatIf* ifStatement); - void visit(NotNull scope, AstStatTypeAlias* alias); + void visit(const ScopePtr& scope, AstStat* stat); + void visit(const ScopePtr& scope, AstStatBlock* block); + void visit(const ScopePtr& scope, AstStatLocal* local); + void visit(const ScopePtr& scope, AstStatFor* for_); + void visit(const ScopePtr& scope, AstStatLocalFunction* function); + void visit(const ScopePtr& scope, AstStatFunction* function); + void visit(const ScopePtr& scope, AstStatReturn* ret); + void visit(const ScopePtr& scope, AstStatAssign* assign); + void visit(const ScopePtr& scope, AstStatIf* ifStatement); + void visit(const ScopePtr& scope, AstStatTypeAlias* alias); - TypePackId checkExprList(NotNull scope, const AstArray& exprs); + TypePackId checkExprList(const ScopePtr& scope, const AstArray& exprs); - TypePackId checkPack(NotNull scope, AstArray exprs); - TypePackId checkPack(NotNull scope, AstExpr* expr); + TypePackId checkPack(const ScopePtr& scope, AstArray exprs); + TypePackId checkPack(const ScopePtr& scope, AstExpr* expr); /** * Checks an expression that is expected to evaluate to one type. @@ -118,13 +119,13 @@ struct ConstraintGraphBuilder * @param expr the expression to check. * @return the type of the expression. */ - TypeId check(NotNull scope, AstExpr* expr); + TypeId check(const ScopePtr& scope, AstExpr* expr); - TypeId checkExprTable(NotNull scope, AstExprTable* expr); - TypeId check(NotNull scope, AstExprIndexName* indexName); - TypeId check(NotNull scope, AstExprIndexExpr* indexExpr); - TypeId check(NotNull scope, AstExprUnary* unary); - TypeId check(NotNull scope, AstExprBinary* binary); + TypeId checkExprTable(const ScopePtr& scope, AstExprTable* expr); + TypeId check(const ScopePtr& scope, AstExprIndexName* indexName); + TypeId check(const ScopePtr& scope, AstExprIndexExpr* indexExpr); + TypeId check(const ScopePtr& scope, AstExprUnary* unary); + TypeId check(const ScopePtr& scope, AstExprBinary* binary); struct FunctionSignature { @@ -133,20 +134,20 @@ struct ConstraintGraphBuilder // The scope that encompasses the function's signature. May be nullptr // if there was no need for a signature scope (the function has no // generics). - Scope2* signatureScope; + ScopePtr signatureScope; // The scope that encompasses the function's body. Is a child scope of // signatureScope, if present. - NotNull bodyScope; + ScopePtr bodyScope; }; - FunctionSignature checkFunctionSignature(NotNull parent, AstExprFunction* fn); + FunctionSignature checkFunctionSignature(const ScopePtr& parent, AstExprFunction* fn); /** * Checks the body of a function expression. * @param scope the interior scope of the body of the function. * @param fn the function expression to check. */ - void checkFunctionBody(NotNull scope, AstExprFunction* fn); + void checkFunctionBody(const ScopePtr& scope, AstExprFunction* fn); /** * Resolves a type from its AST annotation. @@ -154,7 +155,7 @@ struct ConstraintGraphBuilder * @param ty the AST annotation to resolve. * @return the type of the AST annotation. **/ - TypeId resolveType(NotNull scope, AstType* ty); + TypeId resolveType(const ScopePtr& scope, AstType* ty); /** * Resolves a type pack from its AST annotation. @@ -162,14 +163,14 @@ struct ConstraintGraphBuilder * @param tp the AST annotation to resolve. * @return the type pack of the AST annotation. **/ - TypePackId resolveTypePack(NotNull scope, AstTypePack* tp); + TypePackId resolveTypePack(const ScopePtr& scope, AstTypePack* tp); - TypePackId resolveTypePack(NotNull scope, const AstTypeList& list); + TypePackId resolveTypePack(const ScopePtr& scope, const AstTypeList& list); - std::vector> createGenerics(NotNull scope, AstArray generics); - std::vector> createGenericPacks(NotNull scope, AstArray packs); + std::vector> createGenerics(const ScopePtr& scope, AstArray generics); + std::vector> createGenericPacks(const ScopePtr& scope, AstArray packs); - TypeId flattenPack(NotNull scope, Location location, TypePackId tp); + TypeId flattenPack(const ScopePtr& scope, Location location, TypePackId tp); void reportError(Location location, TypeErrorData err); void reportCodeTooComplex(Location location); @@ -180,7 +181,7 @@ struct ConstraintGraphBuilder * real" in a general way is going to be pretty hard, so we are choosing not to tackle that yet. For now, we do an * initial scan of the AST and note what globals are defined. */ - void prepopulateGlobalScope(NotNull globalScope, AstStatBlock* program); + void prepopulateGlobalScope(const ScopePtr& globalScope, AstStatBlock* program); }; /** @@ -193,6 +194,6 @@ struct ConstraintGraphBuilder * @return a list of pointers to constraints contained within the scope graph. * None of these pointers should be null. */ -std::vector> collectConstraints(NotNull rootScope); +std::vector> collectConstraints(NotNull rootScope); } // namespace Luau diff --git a/Analysis/include/Luau/ConstraintSolver.h b/Analysis/include/Luau/ConstraintSolver.h index cf88efb6..0b9b4ae3 100644 --- a/Analysis/include/Luau/ConstraintSolver.h +++ b/Analysis/include/Luau/ConstraintSolver.h @@ -25,7 +25,7 @@ struct ConstraintSolver // is important to not add elements to this vector, lest the underlying // storage that we retain pointers to be mutated underneath us. const std::vector> constraints; - NotNull rootScope; + NotNull rootScope; // This includes every constraint that has not been fully solved. // A constraint can be both blocked and unsolved, for instance. @@ -40,7 +40,7 @@ struct ConstraintSolver ConstraintSolverLogger logger; - explicit ConstraintSolver(TypeArena* arena, NotNull rootScope); + explicit ConstraintSolver(TypeArena* arena, NotNull rootScope); /** * Attempts to dispatch all pending constraints and reach a type solution @@ -121,6 +121,6 @@ private: void unblock_(BlockedConstraintId progressed); }; -void dump(NotNull rootScope, struct ToStringOptions& opts); +void dump(NotNull rootScope, struct ToStringOptions& opts); } // namespace Luau diff --git a/Analysis/include/Luau/ConstraintSolverLogger.h b/Analysis/include/Luau/ConstraintSolverLogger.h index 55336a23..fe2177c4 100644 --- a/Analysis/include/Luau/ConstraintSolverLogger.h +++ b/Analysis/include/Luau/ConstraintSolverLogger.h @@ -15,8 +15,8 @@ namespace Luau struct ConstraintSolverLogger { std::string compileOutput(); - void captureBoundarySnapshot(const Scope2* rootScope, std::vector>& unsolvedConstraints); - void prepareStepSnapshot(const Scope2* rootScope, NotNull current, std::vector>& unsolvedConstraints); + void captureBoundarySnapshot(const Scope* rootScope, std::vector>& unsolvedConstraints); + void prepareStepSnapshot(const Scope* rootScope, NotNull current, std::vector>& unsolvedConstraints); void commitPreparedStepSnapshot(); private: diff --git a/Analysis/include/Luau/Frontend.h b/Analysis/include/Luau/Frontend.h index 27fd4d7e..3b0164c2 100644 --- a/Analysis/include/Luau/Frontend.h +++ b/Analysis/include/Luau/Frontend.h @@ -152,7 +152,7 @@ struct Frontend void registerBuiltinDefinition(const std::string& name, std::function); void applyBuiltinDefinitionToEnvironment(const std::string& environmentName, const std::string& definitionName); - NotNull getGlobalScope2(); + NotNull getGlobalScope(); private: ModulePtr check(const SourceModule& sourceModule, Mode mode, const ScopePtr& environmentScope); @@ -169,7 +169,7 @@ private: std::unordered_map environments; std::unordered_map> builtinDefinitions; - std::unique_ptr globalScope2; + std::unique_ptr globalScope; public: FileResolver* fileResolver; diff --git a/Analysis/include/Luau/Module.h b/Analysis/include/Luau/Module.h index b3105b78..73f4c972 100644 --- a/Analysis/include/Luau/Module.h +++ b/Analysis/include/Luau/Module.h @@ -69,7 +69,6 @@ struct Module std::shared_ptr names; std::vector> scopes; // never empty - std::vector>> scope2s; // never empty DenseHashMap astTypes{nullptr}; DenseHashMap astTypePacks{nullptr}; @@ -86,7 +85,6 @@ struct Module bool timeout = false; ScopePtr getModuleScope() const; - Scope2* getModuleScope2() const; // Once a module has been typechecked, we clone its public interface into a separate arena. // This helps us to force TypeVar ownership into a DAG rather than a DCG. diff --git a/Analysis/include/Luau/Quantify.h b/Analysis/include/Luau/Quantify.h index f46f0cb5..7edf23b8 100644 --- a/Analysis/include/Luau/Quantify.h +++ b/Analysis/include/Luau/Quantify.h @@ -7,9 +7,9 @@ namespace Luau { struct TypeArena; -struct Scope2; +struct Scope; void quantify(TypeId ty, TypeLevel level); -TypeId quantify(TypeArena* arena, TypeId ty, Scope2* scope); +TypeId quantify(TypeArena* arena, TypeId ty, Scope* scope); } // namespace Luau diff --git a/Analysis/include/Luau/Scope.h b/Analysis/include/Luau/Scope.h index 0eaecf1d..85c1750a 100644 --- a/Analysis/include/Luau/Scope.h +++ b/Analysis/include/Luau/Scope.h @@ -32,10 +32,16 @@ struct Scope explicit Scope(const ScopePtr& parent, int subLevel = 0); // child scope. Parent must not be nullptr. const ScopePtr parent; // null for the root + + // All the children of this scope. + std::vector> children; std::unordered_map bindings; + std::unordered_map typeBindings; + std::unordered_map typePackBindings; TypePackId returnType; - bool breakOk = false; std::optional varargPack; + // All constraints belonging to this scope. + std::vector constraints; TypeLevel level; @@ -45,7 +51,9 @@ struct Scope std::unordered_map> importedTypeBindings; - std::optional lookup(const Symbol& name); + std::optional lookup(Symbol sym); + std::optional lookupTypeBinding(const Name& name); + std::optional lookupTypePackBinding(const Name& name); std::optional lookupType(const Name& name); std::optional lookupImportedType(const Name& moduleAlias, const Name& name); @@ -66,24 +74,4 @@ struct Scope std::unordered_map typeAliasTypePackParameters; }; -struct Scope2 -{ - // The parent scope of this scope. Null if there is no parent (i.e. this - // is the module-level scope). - Scope2* parent = nullptr; - // All the children of this scope. - std::vector> children; - std::unordered_map bindings; // TODO: I think this can be a DenseHashMap - std::unordered_map typeBindings; - std::unordered_map typePackBindings; - TypePackId returnType; - std::optional varargPack; - // All constraints belonging to this scope. - std::vector constraints; - - std::optional lookup(Symbol sym); - std::optional lookupTypeBinding(const Name& name); - std::optional lookupTypePackBinding(const Name& name); -}; - } // namespace Luau diff --git a/Analysis/include/Luau/TypeVar.h b/Analysis/include/Luau/TypeVar.h index fb6093df..052d4d88 100644 --- a/Analysis/include/Luau/TypeVar.h +++ b/Analysis/include/Luau/TypeVar.h @@ -24,7 +24,7 @@ namespace Luau { struct TypeArena; -struct Scope2; +struct Scope; /** * There are three kinds of type variables: @@ -143,7 +143,7 @@ struct ConstrainedTypeVar std::vector parts; TypeLevel level; - Scope2* scope = nullptr; + Scope* scope = nullptr; }; // Singleton types https://github.com/Roblox/luau/blob/master/rfcs/syntax-singleton-types.md @@ -275,7 +275,7 @@ struct FunctionTypeVar std::optional defn = {}, bool hasSelf = false); TypeLevel level; - Scope2* scope = nullptr; + Scope* scope = nullptr; /// These should all be generic std::vector generics; std::vector genericPacks; @@ -344,7 +344,7 @@ struct TableTypeVar TableState state = TableState::Unsealed; TypeLevel level; - Scope2* scope = nullptr; + Scope* scope = nullptr; std::optional name; // Sometimes we throw a type on a name to make for nicer error messages, but without creating any entry in the type namespace diff --git a/Analysis/include/Luau/Unifiable.h b/Analysis/include/Luau/Unifiable.h index 4ff91714..e5eb4198 100644 --- a/Analysis/include/Luau/Unifiable.h +++ b/Analysis/include/Luau/Unifiable.h @@ -8,7 +8,7 @@ namespace Luau { -struct Scope2; +struct Scope; /** * The 'level' of a TypeVar is an indirect way to talk about the scope that it 'belongs' too. @@ -84,11 +84,11 @@ using Name = std::string; struct Free { explicit Free(TypeLevel level); - explicit Free(Scope2* scope); + explicit Free(Scope* scope); int index; TypeLevel level; - Scope2* scope = nullptr; + Scope* scope = nullptr; // True if this free type variable is part of a mutually // recursive type alias whose definitions haven't been // resolved yet. @@ -115,13 +115,13 @@ struct Generic Generic(); explicit Generic(TypeLevel level); explicit Generic(const Name& name); - explicit Generic(Scope2* scope); + explicit Generic(Scope* scope); Generic(TypeLevel level, const Name& name); - Generic(Scope2* scope, const Name& name); + Generic(Scope* scope, const Name& name); int index; TypeLevel level; - Scope2* scope = nullptr; + Scope* scope = nullptr; Name name; bool explicitName = false; diff --git a/Analysis/src/Autocomplete.cpp b/Analysis/src/Autocomplete.cpp index 513a791c..a57a789f 100644 --- a/Analysis/src/Autocomplete.cpp +++ b/Analysis/src/Autocomplete.cpp @@ -12,7 +12,7 @@ #include #include -LUAU_FASTFLAG(LuauSelfCallAutocompleteFix2) +LUAU_FASTFLAG(LuauSelfCallAutocompleteFix3) static const std::unordered_set kStatementStartingKeywords = { "while", "if", "local", "repeat", "function", "do", "for", "return", "break", "continue", "type", "export"}; @@ -149,7 +149,7 @@ static TypeCorrectKind checkTypeCorrectKind(const Module& module, TypeArena* typ ty = follow(ty); auto canUnify = [&typeArena](TypeId subTy, TypeId superTy) { - LUAU_ASSERT(!FFlag::LuauSelfCallAutocompleteFix2); + LUAU_ASSERT(!FFlag::LuauSelfCallAutocompleteFix3); InternalErrorReporter iceReporter; UnifierSharedState unifierState(&iceReporter); @@ -168,7 +168,7 @@ static TypeCorrectKind checkTypeCorrectKind(const Module& module, TypeArena* typ TypeId expectedType = follow(*typeAtPosition); auto checkFunctionType = [typeArena, &canUnify, &expectedType](const FunctionTypeVar* ftv) { - if (FFlag::LuauSelfCallAutocompleteFix2) + if (FFlag::LuauSelfCallAutocompleteFix3) { if (std::optional firstRetTy = first(ftv->retTypes)) return checkTypeMatch(typeArena, *firstRetTy, expectedType); @@ -209,7 +209,7 @@ static TypeCorrectKind checkTypeCorrectKind(const Module& module, TypeArena* typ } } - if (FFlag::LuauSelfCallAutocompleteFix2) + if (FFlag::LuauSelfCallAutocompleteFix3) return checkTypeMatch(typeArena, ty, expectedType) ? TypeCorrectKind::Correct : TypeCorrectKind::None; else return canUnify(ty, expectedType) ? TypeCorrectKind::Correct : TypeCorrectKind::None; @@ -226,7 +226,7 @@ static void autocompleteProps(const Module& module, TypeArena* typeArena, TypeId const std::vector& nodes, AutocompleteEntryMap& result, std::unordered_set& seen, std::optional containingClass = std::nullopt) { - if (FFlag::LuauSelfCallAutocompleteFix2) + if (FFlag::LuauSelfCallAutocompleteFix3) rootTy = follow(rootTy); ty = follow(ty); @@ -236,7 +236,7 @@ static void autocompleteProps(const Module& module, TypeArena* typeArena, TypeId seen.insert(ty); auto isWrongIndexer_DEPRECATED = [indexType, useStrictFunctionIndexers = !!get(ty)](Luau::TypeId type) { - LUAU_ASSERT(!FFlag::LuauSelfCallAutocompleteFix2); + LUAU_ASSERT(!FFlag::LuauSelfCallAutocompleteFix3); if (indexType == PropIndexType::Key) return false; @@ -269,7 +269,7 @@ static void autocompleteProps(const Module& module, TypeArena* typeArena, TypeId } }; auto isWrongIndexer = [typeArena, rootTy, indexType](Luau::TypeId type) { - LUAU_ASSERT(FFlag::LuauSelfCallAutocompleteFix2); + LUAU_ASSERT(FFlag::LuauSelfCallAutocompleteFix3); if (indexType == PropIndexType::Key) return false; @@ -277,21 +277,20 @@ static void autocompleteProps(const Module& module, TypeArena* typeArena, TypeId bool calledWithSelf = indexType == PropIndexType::Colon; auto isCompatibleCall = [typeArena, rootTy, calledWithSelf](const FunctionTypeVar* ftv) { - if (get(rootTy)) - { - // Calls on classes require strict match between how function is declared and how it's called - return calledWithSelf == ftv->hasSelf; - } + // Strong match with definition is a success + if (calledWithSelf == ftv->hasSelf) + return true; - // If a call is made with ':', it is invalid if a function has incompatible first argument or no arguments at all - // If a call is made with '.', but it was declared with 'self', it is considered invalid if first argument is compatible - if (calledWithSelf || ftv->hasSelf) + // Calls on classes require strict match between how function is declared and how it's called + if (get(rootTy)) + return false; + + // When called with ':', but declared without 'self', it is invalid if a function has incompatible first argument or no arguments at all + // When called with '.', but declared with 'self', it is considered invalid if first argument is compatible + if (std::optional firstArgTy = first(ftv->argTypes)) { - if (std::optional firstArgTy = first(ftv->argTypes)) - { - if (checkTypeMatch(typeArena, rootTy, *firstArgTy)) - return calledWithSelf; - } + if (checkTypeMatch(typeArena, rootTy, *firstArgTy)) + return calledWithSelf; } return !calledWithSelf; @@ -333,7 +332,7 @@ static void autocompleteProps(const Module& module, TypeArena* typeArena, TypeId AutocompleteEntryKind::Property, type, prop.deprecated, - FFlag::LuauSelfCallAutocompleteFix2 ? isWrongIndexer(type) : isWrongIndexer_DEPRECATED(type), + FFlag::LuauSelfCallAutocompleteFix3 ? isWrongIndexer(type) : isWrongIndexer_DEPRECATED(type), typeCorrect, containingClass, &prop, @@ -376,7 +375,7 @@ static void autocompleteProps(const Module& module, TypeArena* typeArena, TypeId { autocompleteProps(module, typeArena, rootTy, mt->table, indexType, nodes, result, seen); - if (FFlag::LuauSelfCallAutocompleteFix2) + if (FFlag::LuauSelfCallAutocompleteFix3) { if (auto mtable = get(mt->metatable)) fillMetatableProps(mtable); @@ -442,7 +441,7 @@ static void autocompleteProps(const Module& module, TypeArena* typeArena, TypeId AutocompleteEntryMap inner; std::unordered_set innerSeen; - if (!FFlag::LuauSelfCallAutocompleteFix2) + if (!FFlag::LuauSelfCallAutocompleteFix3) innerSeen = seen; if (isNil(*iter)) @@ -468,7 +467,7 @@ static void autocompleteProps(const Module& module, TypeArena* typeArena, TypeId ++iter; } } - else if (auto pt = get(ty); pt && FFlag::LuauSelfCallAutocompleteFix2) + else if (auto pt = get(ty); pt && FFlag::LuauSelfCallAutocompleteFix3) { if (pt->metatable) { @@ -476,7 +475,7 @@ static void autocompleteProps(const Module& module, TypeArena* typeArena, TypeId fillMetatableProps(mtable); } } - else if (FFlag::LuauSelfCallAutocompleteFix2 && get(get(ty))) + else if (FFlag::LuauSelfCallAutocompleteFix3 && get(get(ty))) { autocompleteProps(module, typeArena, rootTy, getSingletonTypes().stringType, indexType, nodes, result, seen); } @@ -1405,7 +1404,7 @@ static AutocompleteResult autocomplete(const SourceModule& sourceModule, const M TypeId ty = follow(*it); PropIndexType indexType = indexName->op == ':' ? PropIndexType::Colon : PropIndexType::Point; - if (!FFlag::LuauSelfCallAutocompleteFix2 && isString(ty)) + if (!FFlag::LuauSelfCallAutocompleteFix3 && isString(ty)) return { autocompleteProps(*module, typeArena, typeChecker.globalScope->bindings[AstName{"string"}].typeId, indexType, ancestry), ancestry}; else diff --git a/Analysis/src/ConstraintGraphBuilder.cpp b/Analysis/src/ConstraintGraphBuilder.cpp index 38895ceb..efaeff6c 100644 --- a/Analysis/src/ConstraintGraphBuilder.cpp +++ b/Analysis/src/ConstraintGraphBuilder.cpp @@ -14,7 +14,7 @@ namespace Luau const AstStat* getFallthrough(const AstStat* node); // TypeInfer.cpp ConstraintGraphBuilder::ConstraintGraphBuilder( - const ModuleName& moduleName, TypeArena* arena, NotNull ice, NotNull globalScope) + const ModuleName& moduleName, TypeArena* arena, NotNull ice, NotNull globalScope) : moduleName(moduleName) , singletonTypes(getSingletonTypes()) , arena(arena) @@ -25,36 +25,34 @@ ConstraintGraphBuilder::ConstraintGraphBuilder( LUAU_ASSERT(arena); } -TypeId ConstraintGraphBuilder::freshType(NotNull scope) +TypeId ConstraintGraphBuilder::freshType(const ScopePtr& scope) { - return arena->addType(FreeTypeVar{scope}); + return arena->addType(FreeTypeVar{scope.get()}); } -TypePackId ConstraintGraphBuilder::freshTypePack(NotNull scope) +TypePackId ConstraintGraphBuilder::freshTypePack(const ScopePtr& scope) { - FreeTypePack f{scope}; + FreeTypePack f{scope.get()}; return arena->addTypePack(TypePackVar{std::move(f)}); } -NotNull ConstraintGraphBuilder::childScope(Location location, NotNull parent) +ScopePtr ConstraintGraphBuilder::childScope(Location location, const ScopePtr& parent) { - auto scope = std::make_unique(); - NotNull borrow = NotNull(scope.get()); - scopes.emplace_back(location, std::move(scope)); + auto scope = std::make_shared(parent); + scopes.emplace_back(location, scope); - borrow->parent = parent; - borrow->returnType = parent->returnType; - parent->children.push_back(borrow); + scope->returnType = parent->returnType; + parent->children.push_back(NotNull(scope.get())); - return borrow; + return scope; } -void ConstraintGraphBuilder::addConstraint(NotNull scope, ConstraintV cv) +void ConstraintGraphBuilder::addConstraint(const ScopePtr& scope, ConstraintV cv) { scope->constraints.emplace_back(new Constraint{std::move(cv)}); } -void ConstraintGraphBuilder::addConstraint(NotNull scope, std::unique_ptr c) +void ConstraintGraphBuilder::addConstraint(const ScopePtr& scope, std::unique_ptr c) { scope->constraints.emplace_back(std::move(c)); } @@ -63,13 +61,13 @@ void ConstraintGraphBuilder::visit(AstStatBlock* block) { LUAU_ASSERT(scopes.empty()); LUAU_ASSERT(rootScope == nullptr); - scopes.emplace_back(block->location, std::make_unique()); - rootScope = scopes.back().second.get(); - NotNull borrow = NotNull(rootScope); + ScopePtr scope = std::make_shared(singletonTypes.anyTypePack); + rootScope = scope.get(); + scopes.emplace_back(block->location, scope); - rootScope->returnType = freshTypePack(borrow); + rootScope->returnType = freshTypePack(scope); - prepopulateGlobalScope(borrow, block); + prepopulateGlobalScope(scope, block); // TODO: We should share the global scope. rootScope->typeBindings["nil"] = singletonTypes.nilType; @@ -78,10 +76,10 @@ void ConstraintGraphBuilder::visit(AstStatBlock* block) rootScope->typeBindings["boolean"] = singletonTypes.booleanType; rootScope->typeBindings["thread"] = singletonTypes.threadType; - visitBlockWithoutChildScope(borrow, block); + visitBlockWithoutChildScope(scope, block); } -void ConstraintGraphBuilder::visitBlockWithoutChildScope(NotNull scope, AstStatBlock* block) +void ConstraintGraphBuilder::visitBlockWithoutChildScope(const ScopePtr& scope, AstStatBlock* block) { RecursionCounter counter{&recursionCount}; @@ -95,7 +93,7 @@ void ConstraintGraphBuilder::visitBlockWithoutChildScope(NotNull scope, visit(scope, stat); } -void ConstraintGraphBuilder::visit(NotNull scope, AstStat* stat) +void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStat* stat) { RecursionLimiter limiter{&recursionCount, FInt::LuauCheckRecursionLimit}; @@ -123,22 +121,24 @@ void ConstraintGraphBuilder::visit(NotNull scope, AstStat* stat) LUAU_ASSERT(0); } -void ConstraintGraphBuilder::visit(NotNull scope, AstStatLocal* local) +void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatLocal* local) { std::vector varTypes; for (AstLocal* local : local->vars) { TypeId ty = freshType(scope); + Location location = local->location; if (local->annotation) { + location = local->annotation->location; TypeId annotation = resolveType(scope, local->annotation); addConstraint(scope, SubtypeConstraint{ty, annotation}); } varTypes.push_back(ty); - scope->bindings[local] = ty; + scope->bindings[local] = Binding{ty, location}; } for (size_t i = 0; i < local->values.size; ++i) @@ -169,7 +169,7 @@ void ConstraintGraphBuilder::visit(NotNull scope, AstStatLocal* local) } } -void ConstraintGraphBuilder::visit(NotNull scope, AstStatFor* for_) +void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatFor* for_) { auto checkNumber = [&](AstExpr* expr) { @@ -184,24 +184,24 @@ void ConstraintGraphBuilder::visit(NotNull scope, AstStatFor* for_) checkNumber(for_->to); checkNumber(for_->step); - NotNull forScope = childScope(for_->location, scope); - forScope->bindings[for_->var] = singletonTypes.numberType; + ScopePtr forScope = childScope(for_->location, scope); + forScope->bindings[for_->var] = Binding{singletonTypes.numberType, for_->var->location}; visit(forScope, for_->body); } -void addConstraints(Constraint* constraint, NotNull scope) +void addConstraints(Constraint* constraint, NotNull scope) { scope->constraints.reserve(scope->constraints.size() + scope->constraints.size()); for (const auto& c : scope->constraints) constraint->dependencies.push_back(NotNull{c.get()}); - for (NotNull childScope : scope->children) + for (NotNull childScope : scope->children) addConstraints(constraint, childScope); } -void ConstraintGraphBuilder::visit(NotNull scope, AstStatLocalFunction* function) +void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatLocalFunction* function) { // Local // Global @@ -213,21 +213,21 @@ void ConstraintGraphBuilder::visit(NotNull scope, AstStatLocalFunction* LUAU_ASSERT(!ty.has_value()); // The parser ensures that every local function has a distinct Symbol for its name. functionType = arena->addType(BlockedTypeVar{}); - scope->bindings[function->name] = functionType; + scope->bindings[function->name] = Binding{functionType, function->name->location}; FunctionSignature sig = checkFunctionSignature(scope, function->func); - sig.bodyScope->bindings[function->name] = sig.signature; + sig.bodyScope->bindings[function->name] = Binding{sig.signature, function->func->location}; checkFunctionBody(sig.bodyScope, function->func); std::unique_ptr c{ - new Constraint{GeneralizationConstraint{functionType, sig.signature, sig.signatureScope ? sig.signatureScope : sig.bodyScope}}}; - addConstraints(c.get(), sig.bodyScope); + new Constraint{GeneralizationConstraint{functionType, sig.signature, sig.signatureScope ? sig.signatureScope.get() : sig.bodyScope.get()}}}; + addConstraints(c.get(), NotNull(sig.bodyScope.get())); addConstraint(scope, std::move(c)); } -void ConstraintGraphBuilder::visit(NotNull scope, AstStatFunction* function) +void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatFunction* function) { // Name could be AstStatLocal, AstStatGlobal, AstStatIndexName. // With or without self @@ -247,9 +247,9 @@ void ConstraintGraphBuilder::visit(NotNull scope, AstStatFunction* funct else { functionType = arena->addType(BlockedTypeVar{}); - scope->bindings[localName->local] = functionType; + scope->bindings[localName->local] = Binding{functionType, localName->location}; } - sig.bodyScope->bindings[localName->local] = sig.signature; + sig.bodyScope->bindings[localName->local] = Binding{sig.signature, localName->location}; } else if (AstExprGlobal* globalName = function->name->as()) { @@ -262,9 +262,9 @@ void ConstraintGraphBuilder::visit(NotNull scope, AstStatFunction* funct else { functionType = arena->addType(BlockedTypeVar{}); - rootScope->bindings[globalName->name] = functionType; + rootScope->bindings[globalName->name] = Binding{functionType, globalName->location}; } - sig.bodyScope->bindings[globalName->name] = sig.signature; + sig.bodyScope->bindings[globalName->name] = Binding{sig.signature, globalName->location}; } else if (AstExprIndexName* indexName = function->name->as()) { @@ -291,21 +291,21 @@ void ConstraintGraphBuilder::visit(NotNull scope, AstStatFunction* funct checkFunctionBody(sig.bodyScope, function->func); std::unique_ptr c{ - new Constraint{GeneralizationConstraint{functionType, sig.signature, sig.signatureScope ? sig.signatureScope : sig.bodyScope}}}; - addConstraints(c.get(), sig.bodyScope); + new Constraint{GeneralizationConstraint{functionType, sig.signature, sig.signatureScope ? sig.signatureScope.get() : sig.bodyScope.get()}}}; + addConstraints(c.get(), NotNull(sig.bodyScope.get())); addConstraint(scope, std::move(c)); } -void ConstraintGraphBuilder::visit(NotNull scope, AstStatReturn* ret) +void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatReturn* ret) { TypePackId exprTypes = checkPack(scope, ret->list); addConstraint(scope, PackSubtypeConstraint{exprTypes, scope->returnType}); } -void ConstraintGraphBuilder::visit(NotNull scope, AstStatBlock* block) +void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatBlock* block) { - NotNull innerScope = childScope(block->location, scope); + ScopePtr innerScope = childScope(block->location, scope); // In order to enable mutually-recursive type aliases, we need to // populate the type bindings before we actually check any of the @@ -323,7 +323,7 @@ void ConstraintGraphBuilder::visit(NotNull scope, AstStatBlock* block) visitBlockWithoutChildScope(innerScope, block); } -void ConstraintGraphBuilder::visit(NotNull scope, AstStatAssign* assign) +void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatAssign* assign) { TypePackId varPackId = checkExprList(scope, assign->vars); TypePackId valuePack = checkPack(scope, assign->values); @@ -331,21 +331,21 @@ void ConstraintGraphBuilder::visit(NotNull scope, AstStatAssign* assign) addConstraint(scope, PackSubtypeConstraint{valuePack, varPackId}); } -void ConstraintGraphBuilder::visit(NotNull scope, AstStatIf* ifStatement) +void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatIf* ifStatement) { check(scope, ifStatement->condition); - NotNull thenScope = childScope(ifStatement->thenbody->location, scope); + ScopePtr thenScope = childScope(ifStatement->thenbody->location, scope); visit(thenScope, ifStatement->thenbody); if (ifStatement->elsebody) { - NotNull elseScope = childScope(ifStatement->elsebody->location, scope); + ScopePtr elseScope = childScope(ifStatement->elsebody->location, scope); visit(elseScope, ifStatement->elsebody); } } -void ConstraintGraphBuilder::visit(NotNull scope, AstStatTypeAlias* alias) +void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatTypeAlias* alias) { // TODO: Exported type aliases // TODO: Generic type aliases @@ -371,7 +371,7 @@ void ConstraintGraphBuilder::visit(NotNull scope, AstStatTypeAlias* alia addConstraint(scope, NameConstraint{ty, alias->name.value}); } -TypePackId ConstraintGraphBuilder::checkPack(NotNull scope, AstArray exprs) +TypePackId ConstraintGraphBuilder::checkPack(const ScopePtr& scope, AstArray exprs) { if (exprs.size == 0) return arena->addTypePack({}); @@ -392,7 +392,7 @@ TypePackId ConstraintGraphBuilder::checkPack(NotNull scope, AstArrayaddTypePack(TypePack{std::move(types), last}); } -TypePackId ConstraintGraphBuilder::checkExprList(NotNull scope, const AstArray& exprs) +TypePackId ConstraintGraphBuilder::checkExprList(const ScopePtr& scope, const AstArray& exprs) { TypePackId result = arena->addTypePack({}); TypePack* resultPack = getMutable(result); @@ -413,7 +413,7 @@ TypePackId ConstraintGraphBuilder::checkExprList(NotNull scope, const As return result; } -TypePackId ConstraintGraphBuilder::checkPack(NotNull scope, AstExpr* expr) +TypePackId ConstraintGraphBuilder::checkPack(const ScopePtr& scope, AstExpr* expr) { RecursionCounter counter{&recursionCount}; @@ -468,7 +468,7 @@ TypePackId ConstraintGraphBuilder::checkPack(NotNull scope, AstExpr* exp return result; } -TypeId ConstraintGraphBuilder::check(NotNull scope, AstExpr* expr) +TypeId ConstraintGraphBuilder::check(const ScopePtr& scope, AstExpr* expr) { RecursionCounter counter{&recursionCount}; @@ -548,7 +548,7 @@ TypeId ConstraintGraphBuilder::check(NotNull scope, AstExpr* expr) return result; } -TypeId ConstraintGraphBuilder::check(NotNull scope, AstExprIndexName* indexName) +TypeId ConstraintGraphBuilder::check(const ScopePtr& scope, AstExprIndexName* indexName) { TypeId obj = check(scope, indexName->expr); TypeId result = freshType(scope); @@ -564,7 +564,7 @@ TypeId ConstraintGraphBuilder::check(NotNull scope, AstExprIndexName* in return result; } -TypeId ConstraintGraphBuilder::check(NotNull scope, AstExprIndexExpr* indexExpr) +TypeId ConstraintGraphBuilder::check(const ScopePtr& scope, AstExprIndexExpr* indexExpr) { TypeId obj = check(scope, indexExpr->expr); TypeId indexType = check(scope, indexExpr->index); @@ -579,7 +579,7 @@ TypeId ConstraintGraphBuilder::check(NotNull scope, AstExprIndexExpr* in return result; } -TypeId ConstraintGraphBuilder::check(NotNull scope, AstExprUnary* unary) +TypeId ConstraintGraphBuilder::check(const ScopePtr& scope, AstExprUnary* unary) { TypeId operandType = check(scope, unary->expr); @@ -599,7 +599,7 @@ TypeId ConstraintGraphBuilder::check(NotNull scope, AstExprUnary* unary) return singletonTypes.errorRecoveryType(); } -TypeId ConstraintGraphBuilder::check(NotNull scope, AstExprBinary* binary) +TypeId ConstraintGraphBuilder::check(const ScopePtr& scope, AstExprBinary* binary) { TypeId leftType = check(scope, binary->left); TypeId rightType = check(scope, binary->right); @@ -624,7 +624,7 @@ TypeId ConstraintGraphBuilder::check(NotNull scope, AstExprBinary* binar return nullptr; } -TypeId ConstraintGraphBuilder::checkExprTable(NotNull scope, AstExprTable* expr) +TypeId ConstraintGraphBuilder::checkExprTable(const ScopePtr& scope, AstExprTable* expr) { TypeId ty = arena->addType(TableTypeVar{}); TableTypeVar* ttv = getMutable(ty); @@ -674,10 +674,10 @@ TypeId ConstraintGraphBuilder::checkExprTable(NotNull scope, AstExprTabl return ty; } -ConstraintGraphBuilder::FunctionSignature ConstraintGraphBuilder::checkFunctionSignature(NotNull parent, AstExprFunction* fn) +ConstraintGraphBuilder::FunctionSignature ConstraintGraphBuilder::checkFunctionSignature(const ScopePtr& parent, AstExprFunction* fn) { - Scope2* signatureScope = nullptr; - Scope2* bodyScope = nullptr; + ScopePtr signatureScope = nullptr; + ScopePtr bodyScope = nullptr; TypePackId returnType = nullptr; std::vector genericTypes; @@ -690,18 +690,17 @@ ConstraintGraphBuilder::FunctionSignature ConstraintGraphBuilder::checkFunctionS // generics properly. if (hasGenerics) { - NotNull signatureBorrow = childScope(fn->location, parent); - signatureScope = signatureBorrow.get(); + signatureScope = childScope(fn->location, parent); // We need to assign returnType before creating bodyScope so that the // return type gets propogated to bodyScope. - returnType = freshTypePack(signatureBorrow); + returnType = freshTypePack(signatureScope); signatureScope->returnType = returnType; - bodyScope = childScope(fn->body->location, signatureBorrow).get(); + bodyScope = childScope(fn->body->location, signatureScope); - std::vector> genericDefinitions = createGenerics(signatureBorrow, fn->generics); - std::vector> genericPackDefinitions = createGenericPacks(signatureBorrow, fn->genericPacks); + std::vector> genericDefinitions = createGenerics(signatureScope, fn->generics); + std::vector> genericPackDefinitions = createGenericPacks(signatureScope, fn->genericPacks); // We do not support default values on function generics, so we only // care about the types involved. @@ -719,11 +718,10 @@ ConstraintGraphBuilder::FunctionSignature ConstraintGraphBuilder::checkFunctionS } else { - NotNull bodyBorrow = childScope(fn->body->location, parent); - bodyScope = bodyBorrow.get(); + bodyScope = childScope(fn->body->location, parent); - returnType = freshTypePack(bodyBorrow); - bodyBorrow->returnType = returnType; + returnType = freshTypePack(bodyScope); + bodyScope->returnType = returnType; // To eliminate the need to branch on hasGenerics below, we say that the // signature scope is the body scope when there is no real signature @@ -731,27 +729,24 @@ ConstraintGraphBuilder::FunctionSignature ConstraintGraphBuilder::checkFunctionS signatureScope = bodyScope; } - NotNull bodyBorrow = NotNull(bodyScope); - NotNull signatureBorrow = NotNull(signatureScope); - if (fn->returnAnnotation) { - TypePackId annotatedRetType = resolveTypePack(signatureBorrow, *fn->returnAnnotation); - addConstraint(signatureBorrow, PackSubtypeConstraint{returnType, annotatedRetType}); + TypePackId annotatedRetType = resolveTypePack(signatureScope, *fn->returnAnnotation); + addConstraint(signatureScope, PackSubtypeConstraint{returnType, annotatedRetType}); } std::vector argTypes; for (AstLocal* local : fn->args) { - TypeId t = freshType(signatureBorrow); + TypeId t = freshType(signatureScope); argTypes.push_back(t); - signatureScope->bindings[local] = t; + signatureScope->bindings[local] = Binding{t, local->location}; if (local->annotation) { - TypeId argAnnotation = resolveType(signatureBorrow, local->annotation); - addConstraint(signatureBorrow, SubtypeConstraint{t, argAnnotation}); + TypeId argAnnotation = resolveType(signatureScope, local->annotation); + addConstraint(signatureScope, SubtypeConstraint{t, argAnnotation}); } } @@ -772,11 +767,11 @@ ConstraintGraphBuilder::FunctionSignature ConstraintGraphBuilder::checkFunctionS // Undo the workaround we made above: if there's no signature scope, // don't report it. /* signatureScope */ hasGenerics ? signatureScope : nullptr, - /* bodyScope */ bodyBorrow, + /* bodyScope */ bodyScope, }; } -void ConstraintGraphBuilder::checkFunctionBody(NotNull scope, AstExprFunction* fn) +void ConstraintGraphBuilder::checkFunctionBody(const ScopePtr& scope, AstExprFunction* fn) { visitBlockWithoutChildScope(scope, fn->body); @@ -789,7 +784,7 @@ void ConstraintGraphBuilder::checkFunctionBody(NotNull scope, AstExprFun } } -TypeId ConstraintGraphBuilder::resolveType(NotNull scope, AstType* ty) +TypeId ConstraintGraphBuilder::resolveType(const ScopePtr& scope, AstType* ty) { TypeId result = nullptr; @@ -834,7 +829,7 @@ TypeId ConstraintGraphBuilder::resolveType(NotNull scope, AstType* ty) { // TODO: Recursion limit. bool hasGenerics = fn->generics.size > 0 || fn->genericPacks.size > 0; - Scope2* signatureScope = nullptr; + ScopePtr signatureScope = nullptr; std::vector genericTypes; std::vector genericTypePacks; @@ -843,22 +838,21 @@ TypeId ConstraintGraphBuilder::resolveType(NotNull scope, AstType* ty) // for the generic bindings to live on. if (hasGenerics) { - NotNull signatureBorrow = childScope(fn->location, scope); - signatureScope = signatureBorrow.get(); + signatureScope = childScope(fn->location, scope); - std::vector> genericDefinitions = createGenerics(signatureBorrow, fn->generics); - std::vector> genericPackDefinitions = createGenericPacks(signatureBorrow, fn->genericPacks); + std::vector> genericDefinitions = createGenerics(signatureScope, fn->generics); + std::vector> genericPackDefinitions = createGenericPacks(signatureScope, fn->genericPacks); for (const auto& [name, g] : genericDefinitions) { genericTypes.push_back(g.ty); - signatureBorrow->typeBindings[name] = g.ty; + signatureScope->typeBindings[name] = g.ty; } for (const auto& [name, g] : genericPackDefinitions) { genericTypePacks.push_back(g.tp); - signatureBorrow->typePackBindings[name] = g.tp; + signatureScope->typePackBindings[name] = g.tp; } } else @@ -866,13 +860,11 @@ TypeId ConstraintGraphBuilder::resolveType(NotNull scope, AstType* ty) // To eliminate the need to branch on hasGenerics below, we say that // the signature scope is the parent scope if we don't have // generics. - signatureScope = scope.get(); + signatureScope = scope; } - NotNull signatureBorrow(signatureScope); - - TypePackId argTypes = resolveTypePack(signatureBorrow, fn->argTypes); - TypePackId returnTypes = resolveTypePack(signatureBorrow, fn->returnTypes); + TypePackId argTypes = resolveTypePack(signatureScope, fn->argTypes); + TypePackId returnTypes = resolveTypePack(signatureScope, fn->returnTypes); // TODO: FunctionTypeVar needs a pointer to the scope so that we know // how to quantify/instantiate it. @@ -950,7 +942,7 @@ TypeId ConstraintGraphBuilder::resolveType(NotNull scope, AstType* ty) return result; } -TypePackId ConstraintGraphBuilder::resolveTypePack(NotNull scope, AstTypePack* tp) +TypePackId ConstraintGraphBuilder::resolveTypePack(const ScopePtr& scope, AstTypePack* tp) { TypePackId result; if (auto expl = tp->as()) @@ -964,7 +956,7 @@ TypePackId ConstraintGraphBuilder::resolveTypePack(NotNull scope, AstTyp } else if (auto gen = tp->as()) { - result = arena->addTypePack(TypePackVar{GenericTypePack{scope, gen->genericName.value}}); + result = arena->addTypePack(TypePackVar{GenericTypePack{scope.get(), gen->genericName.value}}); } else { @@ -976,7 +968,7 @@ TypePackId ConstraintGraphBuilder::resolveTypePack(NotNull scope, AstTyp return result; } -TypePackId ConstraintGraphBuilder::resolveTypePack(NotNull scope, const AstTypeList& list) +TypePackId ConstraintGraphBuilder::resolveTypePack(const ScopePtr& scope, const AstTypeList& list) { std::vector head; @@ -994,12 +986,12 @@ TypePackId ConstraintGraphBuilder::resolveTypePack(NotNull scope, const return arena->addTypePack(TypePack{head, tail}); } -std::vector> ConstraintGraphBuilder::createGenerics(NotNull scope, AstArray generics) +std::vector> ConstraintGraphBuilder::createGenerics(const ScopePtr& scope, AstArray generics) { std::vector> result; for (const auto& generic : generics) { - TypeId genericTy = arena->addType(GenericTypeVar{scope, generic.name.value}); + TypeId genericTy = arena->addType(GenericTypeVar{scope.get(), generic.name.value}); std::optional defaultTy = std::nullopt; if (generic.defaultValue) @@ -1015,12 +1007,12 @@ std::vector> ConstraintGraphBuilder::crea } std::vector> ConstraintGraphBuilder::createGenericPacks( - NotNull scope, AstArray generics) + const ScopePtr& scope, AstArray generics) { std::vector> result; for (const auto& generic : generics) { - TypePackId genericTy = arena->addTypePack(TypePackVar{GenericTypePack{scope, generic.name.value}}); + TypePackId genericTy = arena->addTypePack(TypePackVar{GenericTypePack{scope.get(), generic.name.value}}); std::optional defaultTy = std::nullopt; if (generic.defaultValue) @@ -1035,7 +1027,7 @@ std::vector> ConstraintGraphBuilder:: return result; } -TypeId ConstraintGraphBuilder::flattenPack(NotNull scope, Location location, TypePackId tp) +TypeId ConstraintGraphBuilder::flattenPack(const ScopePtr& scope, Location location, TypePackId tp) { if (auto f = first(tp)) return *f; @@ -1061,10 +1053,10 @@ void ConstraintGraphBuilder::reportCodeTooComplex(Location location) struct GlobalPrepopulator : AstVisitor { - const NotNull globalScope; + const NotNull globalScope; const NotNull arena; - GlobalPrepopulator(NotNull globalScope, NotNull arena) + GlobalPrepopulator(NotNull globalScope, NotNull arena) : globalScope(globalScope) , arena(arena) { @@ -1073,29 +1065,29 @@ struct GlobalPrepopulator : AstVisitor bool visit(AstStatFunction* function) override { if (AstExprGlobal* g = function->name->as()) - globalScope->bindings[g->name] = arena->addType(BlockedTypeVar{}); + globalScope->bindings[g->name] = Binding{arena->addType(BlockedTypeVar{})}; return true; } }; -void ConstraintGraphBuilder::prepopulateGlobalScope(NotNull globalScope, AstStatBlock* program) +void ConstraintGraphBuilder::prepopulateGlobalScope(const ScopePtr& globalScope, AstStatBlock* program) { - GlobalPrepopulator gp{NotNull{globalScope}, arena}; + GlobalPrepopulator gp{NotNull{globalScope.get()}, arena}; program->visit(&gp); } -void collectConstraints(std::vector>& result, NotNull scope) +void collectConstraints(std::vector>& result, NotNull scope) { for (const auto& c : scope->constraints) result.push_back(NotNull{c.get()}); - for (NotNull child : scope->children) + for (NotNull child : scope->children) collectConstraints(result, child); } -std::vector> collectConstraints(NotNull rootScope) +std::vector> collectConstraints(NotNull rootScope) { std::vector> result; collectConstraints(result, rootScope); diff --git a/Analysis/src/ConstraintSolver.cpp b/Analysis/src/ConstraintSolver.cpp index 077a4e28..1cd29918 100644 --- a/Analysis/src/ConstraintSolver.cpp +++ b/Analysis/src/ConstraintSolver.cpp @@ -13,31 +13,31 @@ LUAU_FASTFLAGVARIABLE(DebugLuauLogSolverToJson, false); namespace Luau { -[[maybe_unused]] static void dumpBindings(NotNull scope, ToStringOptions& opts) +[[maybe_unused]] static void dumpBindings(NotNull scope, ToStringOptions& opts) { for (const auto& [k, v] : scope->bindings) { - auto d = toStringDetailed(v, opts); + auto d = toStringDetailed(v.typeId, opts); opts.nameMap = d.nameMap; printf("\t%s : %s\n", k.c_str(), d.name.c_str()); } - for (NotNull child : scope->children) + for (NotNull child : scope->children) dumpBindings(child, opts); } -static void dumpConstraints(NotNull scope, ToStringOptions& opts) +static void dumpConstraints(NotNull scope, ToStringOptions& opts) { for (const ConstraintPtr& c : scope->constraints) { printf("\t%s\n", toString(*c, opts).c_str()); } - for (NotNull child : scope->children) + for (NotNull child : scope->children) dumpConstraints(child, opts); } -void dump(NotNull rootScope, ToStringOptions& opts) +void dump(NotNull rootScope, ToStringOptions& opts) { printf("constraints:\n"); dumpConstraints(rootScope, opts); @@ -55,7 +55,7 @@ void dump(ConstraintSolver* cs, ToStringOptions& opts) } } -ConstraintSolver::ConstraintSolver(TypeArena* arena, NotNull rootScope) +ConstraintSolver::ConstraintSolver(TypeArena* arena, NotNull rootScope) : arena(arena) , constraints(collectConstraints(rootScope)) , rootScope(rootScope) diff --git a/Analysis/src/ConstraintSolverLogger.cpp b/Analysis/src/ConstraintSolverLogger.cpp index 2f93c280..0c2517c0 100644 --- a/Analysis/src/ConstraintSolverLogger.cpp +++ b/Analysis/src/ConstraintSolverLogger.cpp @@ -5,12 +5,12 @@ namespace Luau { -static std::string dumpScopeAndChildren(const Scope2* scope, ToStringOptions& opts) +static std::string dumpScopeAndChildren(const Scope* scope, ToStringOptions& opts) { std::string output = "{\"bindings\":{"; bool comma = false; - for (const auto& [name, type] : scope->bindings) + for (const auto& [name, binding] : scope->bindings) { if (comma) output += ","; @@ -19,7 +19,7 @@ static std::string dumpScopeAndChildren(const Scope2* scope, ToStringOptions& op output += name.c_str(); output += "\": \""; - ToStringResult result = toStringDetailed(type, opts); + ToStringResult result = toStringDetailed(binding.typeId, opts); opts.nameMap = std::move(result.nameMap); output += result.name; output += "\""; @@ -30,7 +30,7 @@ static std::string dumpScopeAndChildren(const Scope2* scope, ToStringOptions& op output += "},\"children\":["; comma = false; - for (const Scope2* child : scope->children) + for (const Scope* child : scope->children) { if (comma) output += ","; @@ -96,7 +96,7 @@ std::string ConstraintSolverLogger::compileOutput() return output; } -void ConstraintSolverLogger::captureBoundarySnapshot(const Scope2* rootScope, std::vector>& unsolvedConstraints) +void ConstraintSolverLogger::captureBoundarySnapshot(const Scope* rootScope, std::vector>& unsolvedConstraints) { std::string snapshot = "{\"type\":\"boundary\",\"rootScope\":"; @@ -109,7 +109,7 @@ void ConstraintSolverLogger::captureBoundarySnapshot(const Scope2* rootScope, st } void ConstraintSolverLogger::prepareStepSnapshot( - const Scope2* rootScope, NotNull current, std::vector>& unsolvedConstraints) + const Scope* rootScope, NotNull current, std::vector>& unsolvedConstraints) { // LUAU_ASSERT(!preparedSnapshot); diff --git a/Analysis/src/EmbeddedBuiltinDefinitions.cpp b/Analysis/src/EmbeddedBuiltinDefinitions.cpp index f93f65dd..45663531 100644 --- a/Analysis/src/EmbeddedBuiltinDefinitions.cpp +++ b/Analysis/src/EmbeddedBuiltinDefinitions.cpp @@ -2,7 +2,6 @@ #include "Luau/BuiltinDefinitions.h" LUAU_FASTFLAG(LuauUnknownAndNeverType) -LUAU_FASTFLAG(LuauCheckLenMT) namespace Luau { @@ -123,6 +122,7 @@ declare function tonumber(value: T, radix: number?): number? declare function rawequal(a: T1, b: T2): boolean declare function rawget(tab: {[K]: V}, k: K): V declare function rawset(tab: {[K]: V}, k: K, v: V): {[K]: V} +declare function rawlen(obj: {[K]: V} | string): number declare function setfenv(target: number | (T...) -> R..., env: {[string]: any}): ((T...) -> R...)? @@ -206,10 +206,6 @@ std::string getBuiltinDefinitionSource() std::string result = kBuiltinDefinitionLuaSrc; - // TODO: move this into kBuiltinDefinitionLuaSrc - if (FFlag::LuauCheckLenMT) - result += "declare function rawlen(obj: {[K]: V} | string): number\n"; - if (FFlag::LuauUnknownAndNeverType) result += "declare function error(message: T, level: number?): never\n"; else diff --git a/Analysis/src/Frontend.cpp b/Analysis/src/Frontend.cpp index a7670de5..222a17df 100644 --- a/Analysis/src/Frontend.cpp +++ b/Analysis/src/Frontend.cpp @@ -766,35 +766,35 @@ const SourceModule* Frontend::getSourceModule(const ModuleName& moduleName) cons return const_cast(this)->getSourceModule(moduleName); } -NotNull Frontend::getGlobalScope2() +NotNull Frontend::getGlobalScope() { - if (!globalScope2) + if (!globalScope) { const SingletonTypes& singletonTypes = getSingletonTypes(); - globalScope2 = std::make_unique(); - globalScope2->typeBindings["nil"] = singletonTypes.nilType; - globalScope2->typeBindings["number"] = singletonTypes.numberType; - globalScope2->typeBindings["string"] = singletonTypes.stringType; - globalScope2->typeBindings["boolean"] = singletonTypes.booleanType; - globalScope2->typeBindings["thread"] = singletonTypes.threadType; + globalScope = std::make_unique(singletonTypes.anyTypePack); + globalScope->typeBindings["nil"] = singletonTypes.nilType; + globalScope->typeBindings["number"] = singletonTypes.numberType; + globalScope->typeBindings["string"] = singletonTypes.stringType; + globalScope->typeBindings["boolean"] = singletonTypes.booleanType; + globalScope->typeBindings["thread"] = singletonTypes.threadType; } - return NotNull(globalScope2.get()); + return NotNull(globalScope.get()); } ModulePtr Frontend::check(const SourceModule& sourceModule, Mode mode, const ScopePtr& environmentScope) { ModulePtr result = std::make_shared(); - ConstraintGraphBuilder cgb{sourceModule.name, &result->internalTypes, NotNull(&iceHandler), getGlobalScope2()}; + ConstraintGraphBuilder cgb{sourceModule.name, &result->internalTypes, NotNull(&iceHandler), getGlobalScope()}; cgb.visit(sourceModule.root); result->errors = std::move(cgb.errors); ConstraintSolver cs{&result->internalTypes, NotNull(cgb.rootScope)}; cs.run(); - result->scope2s = std::move(cgb.scopes); + result->scopes = std::move(cgb.scopes); result->astTypes = std::move(cgb.astTypes); result->astTypePacks = std::move(cgb.astTypePacks); result->astOriginalCallTypes = std::move(cgb.astOriginalCallTypes); diff --git a/Analysis/src/JsonEncoder.cpp b/Analysis/src/JsonEncoder.cpp index 550c9b59..ee7dadb3 100644 --- a/Analysis/src/JsonEncoder.cpp +++ b/Analysis/src/JsonEncoder.cpp @@ -103,8 +103,8 @@ struct AstJsonEncoder : public AstVisitor void write(double d) { - char b[256]; - sprintf(b, "%.17g", d); + char b[32]; + snprintf(b, sizeof(b), "%.17g", d); writeRaw(b); } diff --git a/Analysis/src/Module.cpp b/Analysis/src/Module.cpp index 0603a042..dca18027 100644 --- a/Analysis/src/Module.cpp +++ b/Analysis/src/Module.cpp @@ -100,29 +100,20 @@ void Module::clonePublicInterface(InternalErrorReporter& ice) CloneState cloneState; - ScopePtr moduleScope = FFlag::DebugLuauDeferredConstraintResolution ? nullptr : getModuleScope(); - Scope2* moduleScope2 = FFlag::DebugLuauDeferredConstraintResolution ? getModuleScope2() : nullptr; + ScopePtr moduleScope = getModuleScope(); - TypePackId returnType = FFlag::DebugLuauDeferredConstraintResolution ? moduleScope2->returnType : moduleScope->returnType; + TypePackId returnType = moduleScope->returnType; std::optional varargPack = FFlag::DebugLuauDeferredConstraintResolution ? std::nullopt : moduleScope->varargPack; std::unordered_map* exportedTypeBindings = FFlag::DebugLuauDeferredConstraintResolution ? nullptr : &moduleScope->exportedTypeBindings; returnType = clone(returnType, interfaceTypes, cloneState); - if (moduleScope) + moduleScope->returnType = returnType; + if (varargPack) { - moduleScope->returnType = returnType; - if (varargPack) - { - varargPack = clone(*varargPack, interfaceTypes, cloneState); - moduleScope->varargPack = varargPack; - } - } - else - { - LUAU_ASSERT(moduleScope2); - moduleScope2->returnType = returnType; // TODO varargPack + varargPack = clone(*varargPack, interfaceTypes, cloneState); + moduleScope->varargPack = varargPack; } ForceNormal forceNormal{&interfaceTypes}; @@ -201,10 +192,4 @@ ScopePtr Module::getModuleScope() const return scopes.front().second; } -Scope2* Module::getModuleScope2() const -{ - LUAU_ASSERT(!scope2s.empty()); - return scope2s.front().second.get(); -} - } // namespace Luau diff --git a/Analysis/src/Quantify.cpp b/Analysis/src/Quantify.cpp index 294c479d..72eb4012 100644 --- a/Analysis/src/Quantify.cpp +++ b/Analysis/src/Quantify.cpp @@ -16,13 +16,13 @@ namespace Luau { /// @return true if outer encloses inner -static bool subsumes(Scope2* outer, Scope2* inner) +static bool subsumes(Scope* outer, Scope* inner) { while (inner) { if (inner == outer) return true; - inner = inner->parent; + inner = inner->parent.get(); } return false; @@ -33,7 +33,7 @@ struct Quantifier final : TypeVarOnceVisitor TypeLevel level; std::vector generics; std::vector genericPacks; - Scope2* scope = nullptr; + Scope* scope = nullptr; bool seenGenericType = false; bool seenMutableType = false; @@ -43,20 +43,20 @@ struct Quantifier final : TypeVarOnceVisitor LUAU_ASSERT(!FFlag::DebugLuauDeferredConstraintResolution); } - explicit Quantifier(Scope2* scope) + explicit Quantifier(Scope* scope) : scope(scope) { LUAU_ASSERT(FFlag::DebugLuauDeferredConstraintResolution); } /// @return true if outer encloses inner - bool subsumes(Scope2* outer, Scope2* inner) + bool subsumes(Scope* outer, Scope* inner) { while (inner) { if (inner == outer) return true; - inner = inner->parent; + inner = inner->parent.get(); } return false; @@ -216,7 +216,7 @@ void quantify(TypeId ty, TypeLevel level) } } -void quantify(TypeId ty, Scope2* scope) +void quantify(TypeId ty, Scope* scope) { Quantifier q{scope}; q.traverse(ty); @@ -240,11 +240,11 @@ void quantify(TypeId ty, Scope2* scope) struct PureQuantifier : Substitution { - Scope2* scope; + Scope* scope; std::vector insertedGenerics; std::vector insertedGenericPacks; - PureQuantifier(TypeArena* arena, Scope2* scope) + PureQuantifier(TypeArena* arena, Scope* scope) : Substitution(TxnLog::empty(), arena) , scope(scope) { @@ -322,7 +322,7 @@ struct PureQuantifier : Substitution } }; -TypeId quantify(TypeArena* arena, TypeId ty, Scope2* scope) +TypeId quantify(TypeArena* arena, TypeId ty, Scope* scope) { PureQuantifier quantifier{arena, scope}; std::optional result = quantifier.substitute(ty); diff --git a/Analysis/src/Scope.cpp b/Analysis/src/Scope.cpp index 247a9dd6..083aa085 100644 --- a/Analysis/src/Scope.cpp +++ b/Analysis/src/Scope.cpp @@ -21,22 +21,6 @@ Scope::Scope(const ScopePtr& parent, int subLevel) level.subLevel = subLevel; } -std::optional Scope::lookup(const Symbol& name) -{ - Scope* scope = this; - - while (scope) - { - auto it = scope->bindings.find(name); - if (it != scope->bindings.end()) - return it->second.typeId; - - scope = scope->parent.get(); - } - - return std::nullopt; -} - std::optional Scope::lookupType(const Name& name) { const Scope* scope = this; @@ -121,48 +105,48 @@ std::optional Scope::linearSearchForBinding(const std::string& name, bo return std::nullopt; } -std::optional Scope2::lookup(Symbol sym) +std::optional Scope::lookup(Symbol sym) { - Scope2* s = this; + Scope* s = this; while (true) { auto it = s->bindings.find(sym); if (it != s->bindings.end()) - return it->second; + return it->second.typeId; if (s->parent) - s = s->parent; + s = s->parent.get(); else return std::nullopt; } } -std::optional Scope2::lookupTypeBinding(const Name& name) +std::optional Scope::lookupTypeBinding(const Name& name) { - Scope2* s = this; + Scope* s = this; while (s) { auto it = s->typeBindings.find(name); if (it != s->typeBindings.end()) return it->second; - s = s->parent; + s = s->parent.get(); } return std::nullopt; } -std::optional Scope2::lookupTypePackBinding(const Name& name) +std::optional Scope::lookupTypePackBinding(const Name& name) { - Scope2* s = this; + Scope* s = this; while (s) { auto it = s->typePackBindings.find(name); if (it != s->typePackBindings.end()) return it->second; - s = s->parent; + s = s->parent.get(); } return std::nullopt; diff --git a/Analysis/src/ToString.cpp b/Analysis/src/ToString.cpp index 4c606922..f1dc9211 100644 --- a/Analysis/src/ToString.cpp +++ b/Analysis/src/ToString.cpp @@ -12,6 +12,7 @@ LUAU_FASTFLAG(LuauLowerBoundsCalculation) LUAU_FASTFLAG(LuauUnknownAndNeverType) +LUAU_FASTFLAGVARIABLE(LuauSpecialTypesAsterisked, false) /* * Prefix generic typenames with gen- @@ -277,7 +278,10 @@ struct TypeVarStringifier if (tv->ty.valueless_by_exception()) { state.result.error = true; - state.emit("< VALUELESS BY EXCEPTION >"); + if (FFlag::LuauSpecialTypesAsterisked) + state.emit("* VALUELESS BY EXCEPTION *"); + else + state.emit("< VALUELESS BY EXCEPTION >"); return; } @@ -453,7 +457,10 @@ struct TypeVarStringifier if (state.hasSeen(&ftv)) { state.result.cycle = true; - state.emit(""); + if (FFlag::LuauSpecialTypesAsterisked) + state.emit("*CYCLE*"); + else + state.emit(""); return; } @@ -561,7 +568,10 @@ struct TypeVarStringifier if (state.hasSeen(&ttv)) { state.result.cycle = true; - state.emit(""); + if (FFlag::LuauSpecialTypesAsterisked) + state.emit("*CYCLE*"); + else + state.emit(""); return; } @@ -691,7 +701,10 @@ struct TypeVarStringifier if (state.hasSeen(&uv)) { state.result.cycle = true; - state.emit(""); + if (FFlag::LuauSpecialTypesAsterisked) + state.emit("*CYCLE*"); + else + state.emit(""); return; } @@ -758,7 +771,10 @@ struct TypeVarStringifier if (state.hasSeen(&uv)) { state.result.cycle = true; - state.emit(""); + if (FFlag::LuauSpecialTypesAsterisked) + state.emit("*CYCLE*"); + else + state.emit(""); return; } @@ -803,7 +819,10 @@ struct TypeVarStringifier void operator()(TypeId, const ErrorTypeVar& tv) { state.result.error = true; - state.emit(FFlag::LuauUnknownAndNeverType ? "" : "*unknown*"); + if (FFlag::LuauSpecialTypesAsterisked) + state.emit(FFlag::LuauUnknownAndNeverType ? "*error-type*" : "*unknown*"); + else + state.emit(FFlag::LuauUnknownAndNeverType ? "" : "*unknown*"); } void operator()(TypeId, const LazyTypeVar& ltv) @@ -857,7 +876,10 @@ struct TypePackStringifier if (tp->ty.valueless_by_exception()) { state.result.error = true; - state.emit("< VALUELESS TP BY EXCEPTION >"); + if (FFlag::LuauSpecialTypesAsterisked) + state.emit("* VALUELESS TP BY EXCEPTION *"); + else + state.emit("< VALUELESS TP BY EXCEPTION >"); return; } @@ -881,7 +903,10 @@ struct TypePackStringifier if (state.hasSeen(&tp)) { state.result.cycle = true; - state.emit(""); + if (FFlag::LuauSpecialTypesAsterisked) + state.emit("*CYCLETP*"); + else + state.emit(""); return; } @@ -926,14 +951,22 @@ struct TypePackStringifier void operator()(TypePackId, const Unifiable::Error& error) { state.result.error = true; - state.emit(FFlag::LuauUnknownAndNeverType ? "" : "*unknown*"); + if (FFlag::LuauSpecialTypesAsterisked) + state.emit(FFlag::LuauUnknownAndNeverType ? "*error-type*" : "*unknown*"); + else + state.emit(FFlag::LuauUnknownAndNeverType ? "" : "*unknown*"); } void operator()(TypePackId, const VariadicTypePack& pack) { state.emit("..."); if (FFlag::DebugLuauVerboseTypeNames && pack.hidden) - state.emit(""); + { + if (FFlag::LuauSpecialTypesAsterisked) + state.emit("*hidden*"); + else + state.emit(""); + } stringify(pack.ty); } @@ -1128,7 +1161,11 @@ ToStringResult toStringDetailed(TypeId ty, const ToStringOptions& opts) if (opts.maxTypeLength > 0 && result.name.length() > opts.maxTypeLength) { result.truncated = true; - result.name += "... "; + + if (FFlag::LuauSpecialTypesAsterisked) + result.name += "... *TRUNCATED*"; + else + result.name += "... "; } return result; @@ -1199,7 +1236,12 @@ ToStringResult toStringDetailed(TypePackId tp, const ToStringOptions& opts) } if (opts.maxTypeLength > 0 && result.name.length() > opts.maxTypeLength) - result.name += "... "; + { + if (FFlag::LuauSpecialTypesAsterisked) + result.name += "... *TRUNCATED*"; + else + result.name += "... "; + } return result; } diff --git a/Analysis/src/TypeChecker2.cpp b/Analysis/src/TypeChecker2.cpp index d3b9655d..85749671 100644 --- a/Analysis/src/TypeChecker2.cpp +++ b/Analysis/src/TypeChecker2.cpp @@ -69,6 +69,9 @@ struct TypeChecker2 : public AstVisitor TypePackId reconstructPack(AstArray exprs, TypeArena& arena) { + if (exprs.size == 0) + return arena.addTypePack(TypePack{{}, std::nullopt}); + std::vector head; for (size_t i = 0; i < exprs.size - 1; ++i) @@ -80,14 +83,14 @@ struct TypeChecker2 : public AstVisitor return arena.addTypePack(TypePack{head, tail}); } - Scope2* findInnermostScope(Location location) + Scope* findInnermostScope(Location location) { - Scope2* bestScope = module->getModuleScope2(); - Location bestLocation = module->scope2s[0].first; + Scope* bestScope = module->getModuleScope().get(); + Location bestLocation = module->scopes[0].first; - for (size_t i = 0; i < module->scope2s.size(); ++i) + for (size_t i = 0; i < module->scopes.size(); ++i) { - auto& [scopeBounds, scope] = module->scope2s[i]; + auto& [scopeBounds, scope] = module->scopes[i]; if (scopeBounds.encloses(location)) { if (scopeBounds.begin > bestLocation.begin || scopeBounds.end < bestLocation.end) @@ -181,7 +184,7 @@ struct TypeChecker2 : public AstVisitor bool visit(AstStatReturn* ret) override { - Scope2* scope = findInnermostScope(ret->location); + Scope* scope = findInnermostScope(ret->location); TypePackId expectedRetType = scope->returnType; TypeArena arena; @@ -359,7 +362,7 @@ struct TypeChecker2 : public AstVisitor bool visit(AstTypeReference* ty) override { - Scope2* scope = findInnermostScope(ty->location); + Scope* scope = findInnermostScope(ty->location); // TODO: Imported types // TODO: Generic types diff --git a/Analysis/src/TypeInfer.cpp b/Analysis/src/TypeInfer.cpp index d460160c..371ef77a 100644 --- a/Analysis/src/TypeInfer.cpp +++ b/Analysis/src/TypeInfer.cpp @@ -35,7 +35,7 @@ LUAU_FASTFLAGVARIABLE(LuauExpectedTableUnionIndexerType, false) LUAU_FASTFLAGVARIABLE(LuauIndexSilenceErrors, false) LUAU_FASTFLAGVARIABLE(LuauLowerBoundsCalculation, false) LUAU_FASTFLAGVARIABLE(DebugLuauFreezeDuringUnification, false) -LUAU_FASTFLAGVARIABLE(LuauSelfCallAutocompleteFix2, false) +LUAU_FASTFLAGVARIABLE(LuauSelfCallAutocompleteFix3, false) LUAU_FASTFLAGVARIABLE(LuauReduceUnionRecursion, false) LUAU_FASTFLAGVARIABLE(LuauReturnAnyInsteadOfICE, false) // Eventually removed as false. LUAU_FASTFLAG(LuauNormalizeFlagIsConservative) @@ -45,7 +45,6 @@ LUAU_FASTFLAGVARIABLE(LuauReportErrorsOnIndexerKeyMismatch, false) LUAU_FASTFLAGVARIABLE(LuauUnknownAndNeverType, false) LUAU_FASTFLAG(LuauQuantifyConstrained) LUAU_FASTFLAGVARIABLE(LuauFalsyPredicateReturnsNilInstead, false) -LUAU_FASTFLAGVARIABLE(LuauCheckLenMT, false) LUAU_FASTFLAGVARIABLE(LuauCheckGenericHOFTypes, false) LUAU_FASTFLAGVARIABLE(LuauBinaryNeedsExpectedTypesToo, false) LUAU_FASTFLAGVARIABLE(LuauNeverTypesAndOperatorsInference, false) @@ -1667,7 +1666,7 @@ void TypeChecker::check(const ScopePtr& scope, const AstStatDeclareClass& declar ftv->argNames.insert(ftv->argNames.begin(), FunctionArgument{"self", {}}); ftv->argTypes = addTypePack(TypePack{{classTy}, ftv->argTypes}); - if (FFlag::LuauSelfCallAutocompleteFix2) + if (FFlag::LuauSelfCallAutocompleteFix3) ftv->hasSelf = true; } } @@ -2465,7 +2464,7 @@ WithPredicate TypeChecker::checkExpr(const ScopePtr& scope, const AstExp DenseHashSet seen{nullptr}; - if (FFlag::LuauCheckLenMT && typeCouldHaveMetatable(operandType)) + if (typeCouldHaveMetatable(operandType)) { if (auto fnt = findMetatableEntry(operandType, "__len", expr.location, /* addErrors= */ true)) { diff --git a/Analysis/src/Unifiable.cpp b/Analysis/src/Unifiable.cpp index 8d23aa49..63d8647d 100644 --- a/Analysis/src/Unifiable.cpp +++ b/Analysis/src/Unifiable.cpp @@ -12,7 +12,7 @@ Free::Free(TypeLevel level) { } -Free::Free(Scope2* scope) +Free::Free(Scope* scope) : scope(scope) { } @@ -39,7 +39,7 @@ Generic::Generic(const Name& name) { } -Generic::Generic(Scope2* scope) +Generic::Generic(Scope* scope) : index(++nextIndex) , scope(scope) { @@ -53,7 +53,7 @@ Generic::Generic(TypeLevel level, const Name& name) { } -Generic::Generic(Scope2* scope, const Name& name) +Generic::Generic(Scope* scope, const Name& name) : index(++nextIndex) , scope(scope) , name(name) diff --git a/Ast/src/Parser.cpp b/Ast/src/Parser.cpp index 779bd279..8d54e367 100644 --- a/Ast/src/Parser.cpp +++ b/Ast/src/Parser.cpp @@ -24,8 +24,6 @@ bool lua_telemetry_parsed_named_non_function_type = false; LUAU_FASTFLAGVARIABLE(LuauErrorParseIntegerIssues, false) LUAU_DYNAMIC_FASTFLAGVARIABLE(LuaReportParseIntegerIssues, false) -LUAU_FASTFLAGVARIABLE(LuauAlwaysCaptureHotComments, false) - bool lua_telemetry_parsed_out_of_range_bin_integer = false; bool lua_telemetry_parsed_out_of_range_hex_integer = false; bool lua_telemetry_parsed_double_prefix_hex_integer = false; @@ -2920,39 +2918,34 @@ AstTypeError* Parser::reportTypeAnnotationError(const Location& location, const void Parser::nextLexeme() { - if (options.captureComments || FFlag::LuauAlwaysCaptureHotComments) + Lexeme::Type type = lexer.next(/* skipComments= */ false, true).type; + + while (type == Lexeme::BrokenComment || type == Lexeme::Comment || type == Lexeme::BlockComment) { - Lexeme::Type type = lexer.next(/* skipComments= */ false, true).type; + const Lexeme& lexeme = lexer.current(); - while (type == Lexeme::BrokenComment || type == Lexeme::Comment || type == Lexeme::BlockComment) + if (options.captureComments) + commentLocations.push_back(Comment{lexeme.type, lexeme.location}); + + // Subtlety: Broken comments are weird because we record them as comments AND pass them to the parser as a lexeme. + // The parser will turn this into a proper syntax error. + if (lexeme.type == Lexeme::BrokenComment) + return; + + // Comments starting with ! are called "hot comments" and contain directives for type checking / linting / compiling + if (lexeme.type == Lexeme::Comment && lexeme.length && lexeme.data[0] == '!') { - const Lexeme& lexeme = lexer.current(); + const char* text = lexeme.data; - if (options.captureComments) - commentLocations.push_back(Comment{lexeme.type, lexeme.location}); + unsigned int end = lexeme.length; + while (end > 0 && isSpace(text[end - 1])) + --end; - // Subtlety: Broken comments are weird because we record them as comments AND pass them to the parser as a lexeme. - // The parser will turn this into a proper syntax error. - if (lexeme.type == Lexeme::BrokenComment) - return; - - // Comments starting with ! are called "hot comments" and contain directives for type checking / linting / compiling - if (lexeme.type == Lexeme::Comment && lexeme.length && lexeme.data[0] == '!') - { - const char* text = lexeme.data; - - unsigned int end = lexeme.length; - while (end > 0 && isSpace(text[end - 1])) - --end; - - hotcomments.push_back({hotcommentHeader, lexeme.location, std::string(text + 1, text + end)}); - } - - type = lexer.next(/* skipComments= */ false, /* updatePrevLocation= */ false).type; + hotcomments.push_back({hotcommentHeader, lexeme.location, std::string(text + 1, text + end)}); } + + type = lexer.next(/* skipComments= */ false, /* updatePrevLocation= */ false).type; } - else - lexer.next(); } } // namespace Luau diff --git a/CLI/ReplEntry.cpp b/CLI/ReplEntry.cpp index 75995e6a..8543e3f7 100644 --- a/CLI/ReplEntry.cpp +++ b/CLI/ReplEntry.cpp @@ -1,8 +1,6 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #include "Repl.h" - - int main(int argc, char** argv) { return replMain(argc, argv); diff --git a/Compiler/src/Compiler.cpp b/Compiler/src/Compiler.cpp index 8f3befad..4be54378 100644 --- a/Compiler/src/Compiler.cpp +++ b/Compiler/src/Compiler.cpp @@ -25,8 +25,6 @@ LUAU_FASTINTVARIABLE(LuauCompileInlineDepth, 5) LUAU_FASTFLAGVARIABLE(LuauCompileNoIpairs, false) -LUAU_FASTFLAGVARIABLE(LuauCompileFoldBuiltins, false) -LUAU_FASTFLAGVARIABLE(LuauCompileBetterMultret, false) LUAU_FASTFLAGVARIABLE(LuauCompileFreeReassign, false) namespace Luau @@ -276,9 +274,6 @@ struct Compiler // returns true if node can return multiple values; may conservatively return true even if expr is known to return just a single value bool isExprMultRet(AstExpr* node) { - if (!FFlag::LuauCompileBetterMultret) - return node->is() || node->is(); - AstExprCall* expr = node->as(); if (!expr) return node->is(); @@ -310,27 +305,10 @@ struct Compiler if (AstExprCall* expr = node->as()) { // Optimization: convert multret calls that always return one value to fixedret calls; this facilitates inlining/constant folding - if (options.optimizationLevel >= 2) + if (options.optimizationLevel >= 2 && !isExprMultRet(node)) { - if (FFlag::LuauCompileBetterMultret) - { - if (!isExprMultRet(node)) - { - compileExprTemp(node, target); - return false; - } - } - else - { - AstExprFunction* func = getFunctionExpr(expr->func); - Function* fi = func ? functions.find(func) : nullptr; - - if (fi && fi->returnsOne) - { - compileExprTemp(node, target); - return false; - } - } + compileExprTemp(node, target); + return false; } // We temporarily swap out regTop to have targetTop work correctly... @@ -3437,30 +3415,7 @@ struct Compiler bool visit(AstStatReturn* stat) override { - if (FFlag::LuauCompileBetterMultret) - { - returnsOne &= stat->list.size == 1 && !self->isExprMultRet(stat->list.data[0]); - } - else if (stat->list.size == 1) - { - AstExpr* value = stat->list.data[0]; - - if (AstExprCall* expr = value->as()) - { - AstExprFunction* func = self->getFunctionExpr(expr->func); - Function* fi = func ? self->functions.find(func) : nullptr; - - returnsOne &= fi && fi->returnsOne; - } - else if (value->is()) - { - returnsOne = false; - } - } - else - { - returnsOne = false; - } + returnsOne &= stat->list.size == 1 && !self->isExprMultRet(stat->list.data[0]); return false; } @@ -3601,7 +3556,7 @@ void compileOrThrow(BytecodeBuilder& bytecode, const ParseResult& parseResult, c trackValues(compiler.globals, compiler.variables, root); // builtin folding is enabled on optimization level 2 since we can't deoptimize folding at runtime - if (options.optimizationLevel >= 2 && FFlag::LuauCompileFoldBuiltins) + if (options.optimizationLevel >= 2) compiler.builtinsFold = &compiler.builtins; if (options.optimizationLevel >= 1) diff --git a/Compiler/src/CostModel.cpp b/Compiler/src/CostModel.cpp index 56b6c36f..81cbfd7a 100644 --- a/Compiler/src/CostModel.cpp +++ b/Compiler/src/CostModel.cpp @@ -6,8 +6,6 @@ #include -LUAU_FASTFLAGVARIABLE(LuauCompileModelBuiltins, false) - namespace Luau { namespace Compile @@ -155,7 +153,7 @@ struct CostVisitor : AstVisitor { // builtin cost modeling is different from regular calls because we use FASTCALL to compile these // thus we use a cheaper baseline, don't account for function, and assume constant/local copy is free - bool builtin = FFlag::LuauCompileModelBuiltins && builtins.find(expr) != nullptr; + bool builtin = builtins.find(expr) != nullptr; bool builtinShort = builtin && expr->args.size <= 2; // FASTCALL1/2 Cost cost = builtin ? 2 : 3; diff --git a/Sources.cmake b/Sources.cmake index a4563019..f745667f 100644 --- a/Sources.cmake +++ b/Sources.cmake @@ -267,6 +267,7 @@ if(TARGET Luau.UnitTest) tests/Error.test.cpp tests/Frontend.test.cpp tests/JsonEncoder.test.cpp + tests/Lexer.test.cpp tests/Linter.test.cpp tests/LValue.test.cpp tests/Module.test.cpp diff --git a/VM/src/lapi.cpp b/VM/src/lapi.cpp index e3354ea2..e59f914f 100644 --- a/VM/src/lapi.cpp +++ b/VM/src/lapi.cpp @@ -34,8 +34,6 @@ * therefore call luaC_checkGC before luaC_checkthreadsleep to guarantee the object is pushed to an awake thread. */ -LUAU_FASTFLAG(LuauLazyAtoms) - const char* lua_ident = "$Lua: Lua 5.1.4 Copyright (C) 1994-2008 Lua.org, PUC-Rio $\n" "$Authors: R. Ierusalimschy, L. H. de Figueiredo & W. Celes $\n" "$URL: www.lua.org $\n"; @@ -54,7 +52,6 @@ const char* luau_ident = "$Luau: Copyright (C) 2019-2022 Roblox Corporation $\n" } #define updateatom(L, ts) \ - if (FFlag::LuauLazyAtoms) \ { \ if (ts->atom == ATOM_UNDEF) \ ts->atom = L->global->cb.useratom ? L->global->cb.useratom(ts->data, ts->len) : -1; \ diff --git a/VM/src/ldblib.cpp b/VM/src/ldblib.cpp index 93d8703a..8246f818 100644 --- a/VM/src/ldblib.cpp +++ b/VM/src/ldblib.cpp @@ -130,15 +130,14 @@ static int db_traceback(lua_State* L) if (ar.currentline > 0) { - char line[32]; -#ifdef _MSC_VER - _itoa(ar.currentline, line, 10); // 5x faster than sprintf -#else - sprintf(line, "%d", ar.currentline); -#endif + char line[32]; // manual conversion for performance + char* lineend = line + sizeof(line); + char* lineptr = lineend; + for (unsigned int r = ar.currentline; r > 0; r /= 10) + *--lineptr = '0' + (r % 10); luaL_addchar(&buf, ':'); - luaL_addstring(&buf, line); + luaL_addlstring(&buf, lineptr, lineend - lineptr); } if (ar.name) diff --git a/VM/src/ldebug.cpp b/VM/src/ldebug.cpp index 29015565..5ef41b78 100644 --- a/VM/src/ldebug.cpp +++ b/VM/src/ldebug.cpp @@ -529,7 +529,7 @@ const char* lua_debugtrace(lua_State* L) if (ar.currentline > 0) { char line[32]; - sprintf(line, ":%d", ar.currentline); + snprintf(line, sizeof(line), ":%d", ar.currentline); offset = append(buf, sizeof(buf), offset, line); } @@ -545,7 +545,7 @@ const char* lua_debugtrace(lua_State* L) if (depth > limit1 + limit2 && level == limit1 - 1) { char skip[32]; - sprintf(skip, "... (+%d frames)\n", int(depth - limit1 - limit2)); + snprintf(skip, sizeof(skip), "... (+%d frames)\n", int(depth - limit1 - limit2)); offset = append(buf, sizeof(buf), offset, skip); diff --git a/VM/src/lstring.cpp b/VM/src/lstring.cpp index 12bd67d5..d454308c 100644 --- a/VM/src/lstring.cpp +++ b/VM/src/lstring.cpp @@ -7,8 +7,6 @@ #include -LUAU_FASTFLAGVARIABLE(LuauLazyAtoms, false) - unsigned int luaS_hash(const char* str, size_t len) { // Note that this hashing algorithm is replicated in BytecodeBuilder.cpp, BytecodeBuilder::getStringHash @@ -84,7 +82,7 @@ static TString* newlstr(lua_State* L, const char* str, size_t l, unsigned int h) ts->memcat = L->activememcat; memcpy(ts->data, str, l); ts->data[l] = '\0'; /* ending 0 */ - ts->atom = FFlag::LuauLazyAtoms ? ATOM_UNDEF : L->global->cb.useratom ? L->global->cb.useratom(ts->data, l) : -1; + ts->atom = ATOM_UNDEF; tb = &L->global->strt; h = lmod(h, tb->size); ts->next = tb->hash[h]; /* chain new entry */ @@ -165,9 +163,7 @@ TString* luaS_buffinish(lua_State* L, TString* ts) ts->hash = h; ts->data[ts->len] = '\0'; // ending 0 - - // Complete string object - ts->atom = FFlag::LuauLazyAtoms ? ATOM_UNDEF : L->global->cb.useratom ? L->global->cb.useratom(ts->data, ts->len) : -1; + ts->atom = ATOM_UNDEF; ts->next = tb->hash[bucket]; // chain new entry tb->hash[bucket] = ts; diff --git a/VM/src/lstrlib.cpp b/VM/src/lstrlib.cpp index 74a8aa8a..b0cd4dc2 100644 --- a/VM/src/lstrlib.cpp +++ b/VM/src/lstrlib.cpp @@ -979,14 +979,14 @@ static int str_format(lua_State* L) { case 'c': { - sprintf(buff, form, (int)luaL_checknumber(L, arg)); + snprintf(buff, sizeof(buff), form, (int)luaL_checknumber(L, arg)); break; } case 'd': case 'i': { addInt64Format(form, formatIndicator, formatItemSize); - sprintf(buff, form, (long long)luaL_checknumber(L, arg)); + snprintf(buff, sizeof(buff), form, (long long)luaL_checknumber(L, arg)); break; } case 'o': @@ -997,7 +997,7 @@ static int str_format(lua_State* L) double argValue = luaL_checknumber(L, arg); addInt64Format(form, formatIndicator, formatItemSize); unsigned long long v = (argValue < 0) ? (unsigned long long)(long long)argValue : (unsigned long long)argValue; - sprintf(buff, form, v); + snprintf(buff, sizeof(buff), form, v); break; } case 'e': @@ -1006,7 +1006,7 @@ static int str_format(lua_State* L) case 'g': case 'G': { - sprintf(buff, form, (double)luaL_checknumber(L, arg)); + snprintf(buff, sizeof(buff), form, (double)luaL_checknumber(L, arg)); break; } case 'q': @@ -1028,7 +1028,7 @@ static int str_format(lua_State* L) } else { - sprintf(buff, form, s); + snprintf(buff, sizeof(buff), form, s); break; } } diff --git a/VM/src/ltable.cpp b/VM/src/ltable.cpp index 79e65919..38693156 100644 --- a/VM/src/ltable.cpp +++ b/VM/src/ltable.cpp @@ -44,9 +44,6 @@ static_assert(TKey{{NULL}, {0}, LUA_TDEADKEY, 0}.tt == LUA_TDEADKEY, "not enough static_assert(TKey{{NULL}, {0}, LUA_TNIL, MAXSIZE - 1}.next == MAXSIZE - 1, "not enough bits for next"); static_assert(TKey{{NULL}, {0}, LUA_TNIL, -(MAXSIZE - 1)}.next == -(MAXSIZE - 1), "not enough bits for next"); -// reset cache of absent metamethods, cache is updated in luaT_gettm -#define invalidateTMcache(t) t->tmcache = 0 - // empty hash data points to dummynode so that we can always dereference it const LuaNode luaH_dummynode = { {{NULL}, {0}, LUA_TNIL}, /* value */ @@ -667,15 +664,18 @@ TValue* luaH_set(lua_State* L, Table* t, const TValue* key) if (p != luaO_nilobject) return cast_to(TValue*, p); else - { - if (ttisnil(key)) - luaG_runerror(L, "table index is nil"); - else if (ttisnumber(key) && luai_numisnan(nvalue(key))) - luaG_runerror(L, "table index is NaN"); - else if (ttisvector(key) && luai_vecisnan(vvalue(key))) - luaG_runerror(L, "table index contains NaN"); - return newkey(L, t, key); - } + return luaH_newkey(L, t, key); +} + +TValue* luaH_newkey(lua_State* L, Table* t, const TValue* key) +{ + if (ttisnil(key)) + luaG_runerror(L, "table index is nil"); + else if (ttisnumber(key) && luai_numisnan(nvalue(key))) + luaG_runerror(L, "table index is NaN"); + else if (ttisvector(key) && luai_vecisnan(vvalue(key))) + luaG_runerror(L, "table index contains NaN"); + return newkey(L, t, key); } TValue* luaH_setnum(lua_State* L, Table* t, int key) diff --git a/VM/src/ltable.h b/VM/src/ltable.h index e8413c85..021f21bf 100644 --- a/VM/src/ltable.h +++ b/VM/src/ltable.h @@ -11,12 +11,16 @@ #define gval2slot(t, v) int(cast_to(LuaNode*, static_cast(v)) - t->node) +// reset cache of absent metamethods, cache is updated in luaT_gettm +#define invalidateTMcache(t) t->tmcache = 0 + LUAI_FUNC const TValue* luaH_getnum(Table* t, int key); LUAI_FUNC TValue* luaH_setnum(lua_State* L, Table* t, int key); LUAI_FUNC const TValue* luaH_getstr(Table* t, TString* key); LUAI_FUNC TValue* luaH_setstr(lua_State* L, Table* t, TString* key); LUAI_FUNC const TValue* luaH_get(Table* t, const TValue* key); LUAI_FUNC TValue* luaH_set(lua_State* L, Table* t, const TValue* key); +LUAI_FUNC TValue* luaH_newkey(lua_State* L, Table* t, const TValue* key); LUAI_FUNC Table* luaH_new(lua_State* L, int narray, int lnhash); LUAI_FUNC void luaH_resizearray(lua_State* L, Table* t, int nasize); LUAI_FUNC void luaH_resizehash(lua_State* L, Table* t, int nhsize); @@ -26,4 +30,6 @@ LUAI_FUNC int luaH_getn(Table* t); LUAI_FUNC Table* luaH_clone(lua_State* L, Table* tt); LUAI_FUNC void luaH_clear(Table* tt); +#define luaH_setslot(L, t, slot, key) (invalidateTMcache(t), (slot == luaO_nilobject ? luaH_newkey(L, t, key) : cast_to(TValue*, slot))) + extern const LuaNode luaH_dummynode; diff --git a/VM/src/lvmexecute.cpp b/VM/src/lvmexecute.cpp index 02b39313..10d89aaf 100644 --- a/VM/src/lvmexecute.cpp +++ b/VM/src/lvmexecute.cpp @@ -33,7 +33,7 @@ LUAU_FASTFLAGVARIABLE(LuauLenTM, false) // 3. VM_PROTECT macro saves savedpc and restores base for you; most external calls need to be wrapped into that. However, it does NOT restore // ra/rb/rc! // 4. When copying an object to any existing object as a field, generally speaking you need to call luaC_barrier! Be careful with all setobj calls -// 5. To make 4 easier to follow, please use setobj2s for copies to stack and setobj for other copies. +// 5. To make 4 easier to follow, please use setobj2s for copies to stack, setobj2t for writes to tables, and setobj for other copies. // 6. You can define HARDSTACKTESTS in llimits.h which will aggressively realloc stack; with address sanitizer this should be effective at finding // stack corruption bugs // 7. Many external Lua functions can call GC! GC will *not* traverse pointers to new objects that aren't reachable from Lua root. Be careful when @@ -458,7 +458,7 @@ static void luau_execute(lua_State* L) if (LUAU_LIKELY(ttisstring(gkey(n)) && tsvalue(gkey(n)) == tsvalue(kv) && !ttisnil(gval(n)) && !h->readonly)) { - setobj(L, gval(n), ra); + setobj2t(L, gval(n), ra); luaC_barriert(L, h, ra); VM_NEXT(); } @@ -672,7 +672,7 @@ static void luau_execute(lua_State* L) // fast-path: value is in expected slot if (LUAU_LIKELY(ttisstring(gkey(n)) && tsvalue(gkey(n)) == tsvalue(kv) && !ttisnil(gval(n)) && !h->readonly)) { - setobj(L, gval(n), ra); + setobj2t(L, gval(n), ra); luaC_barriert(L, h, ra); VM_NEXT(); } @@ -684,7 +684,7 @@ static void luau_execute(lua_State* L) int cachedslot = gval2slot(h, res); // save cachedslot to accelerate future lookups; patches currently executing instruction since pc-2 rolls back two pc++ VM_PATCH_C(pc - 2, cachedslot); - setobj(L, res, ra); + setobj2t(L, res, ra); luaC_barriert(L, h, ra); VM_NEXT(); } diff --git a/VM/src/lvmutils.cpp b/VM/src/lvmutils.cpp index be4e99a9..b9267fb8 100644 --- a/VM/src/lvmutils.cpp +++ b/VM/src/lvmutils.cpp @@ -14,6 +14,8 @@ LUAU_FASTFLAG(LuauLenTM) +LUAU_FASTFLAGVARIABLE(LuauBetterNewindex, false) + /* limit for table tag-method chains (to avoid loops) */ #define MAXTAGLOOP 100 @@ -142,24 +144,50 @@ void luaV_settable(lua_State* L, const TValue* t, TValue* key, StkId val) { /* `t' is a table? */ Table* h = hvalue(t); - if (h->readonly) - luaG_readonlyerror(L); + if (FFlag::LuauBetterNewindex) + { + const TValue* oldval = luaH_get(h, key); - TValue* oldval = luaH_set(L, h, key); /* do a primitive set */ + /* should we assign the key? (if key is valid or __newindex is not set) */ + if (!ttisnil(oldval) || (tm = fasttm(L, h->metatable, TM_NEWINDEX)) == NULL) + { + if (h->readonly) + luaG_readonlyerror(L); - L->cachedslot = gval2slot(h, oldval); /* remember slot to accelerate future lookups */ + /* luaH_set would work but would repeat the lookup so we use luaH_setslot that can reuse oldval if it's safe */ + TValue* newval = luaH_setslot(L, h, oldval, key); - if (!ttisnil(oldval) || /* result is no nil? */ - (tm = fasttm(L, h->metatable, TM_NEWINDEX)) == NULL) - { /* or no TM? */ - setobj2t(L, oldval, val); - luaC_barriert(L, h, val); - return; + L->cachedslot = gval2slot(h, newval); /* remember slot to accelerate future lookups */ + + setobj2t(L, newval, val); + luaC_barriert(L, h, val); + return; + } + + /* fallthrough to metamethod */ + } + else + { + if (h->readonly) + luaG_readonlyerror(L); + + TValue* oldval = luaH_set(L, h, key); /* do a primitive set */ + + L->cachedslot = gval2slot(h, oldval); /* remember slot to accelerate future lookups */ + + if (!ttisnil(oldval) || /* result is no nil? */ + (tm = fasttm(L, h->metatable, TM_NEWINDEX)) == NULL) + { /* or no TM? */ + setobj2t(L, oldval, val); + luaC_barriert(L, h, val); + return; + } + /* else will try the tag method */ } - /* else will try the tag method */ } else if (ttisnil(tm = luaT_gettmbyobj(L, t, TM_NEWINDEX))) luaG_indexerror(L, t, key); + if (ttisfunction(tm)) { callTM(L, tm, t, key, val); diff --git a/tests/Autocomplete.test.cpp b/tests/Autocomplete.test.cpp index d8ad3340..0a5603d6 100644 --- a/tests/Autocomplete.test.cpp +++ b/tests/Autocomplete.test.cpp @@ -2845,7 +2845,7 @@ local abc = b@1 TEST_CASE_FIXTURE(ACFixture, "no_incompatible_self_calls_on_class") { - ScopedFastFlag selfCallAutocompleteFix2{"LuauSelfCallAutocompleteFix2", true}; + ScopedFastFlag selfCallAutocompleteFix3{"LuauSelfCallAutocompleteFix3", true}; loadDefinition(R"( declare class Foo @@ -2883,9 +2883,25 @@ t.@1 } } +TEST_CASE_FIXTURE(ACFixture, "do_compatible_self_calls") +{ + ScopedFastFlag selfCallAutocompleteFix3{"LuauSelfCallAutocompleteFix3", true}; + + check(R"( +local t = {} +function t:m() end +t:@1 + )"); + + auto ac = autocomplete('1'); + + REQUIRE(ac.entryMap.count("m")); + CHECK(!ac.entryMap["m"].wrongIndexType); +} + TEST_CASE_FIXTURE(ACFixture, "no_incompatible_self_calls") { - ScopedFastFlag selfCallAutocompleteFix2{"LuauSelfCallAutocompleteFix2", true}; + ScopedFastFlag selfCallAutocompleteFix3{"LuauSelfCallAutocompleteFix3", true}; check(R"( local t = {} @@ -2901,7 +2917,7 @@ t:@1 TEST_CASE_FIXTURE(ACFixture, "no_incompatible_self_calls_2") { - ScopedFastFlag selfCallAutocompleteFix2{"LuauSelfCallAutocompleteFix2", true}; + ScopedFastFlag selfCallAutocompleteFix3{"LuauSelfCallAutocompleteFix3", true}; check(R"( local f: (() -> number) & ((number) -> number) = function(x: number?) return 2 end @@ -2916,7 +2932,7 @@ t:@1 CHECK(ac.entryMap["f"].wrongIndexType); } -TEST_CASE_FIXTURE(ACFixture, "no_incompatible_self_calls_provisional") +TEST_CASE_FIXTURE(ACFixture, "do_wrong_compatible_self_calls") { check(R"( local t = {} @@ -2931,9 +2947,26 @@ t:@1 CHECK(!ac.entryMap["m"].wrongIndexType); } +TEST_CASE_FIXTURE(ACFixture, "no_wrong_compatible_self_calls_with_generics") +{ + ScopedFastFlag selfCallAutocompleteFix3{"LuauSelfCallAutocompleteFix3", true}; + + check(R"( +local t = {} +function t.m(a: T) end +t:@1 + )"); + + auto ac = autocomplete('1'); + + REQUIRE(ac.entryMap.count("m")); + // While this call is compatible with the type, this requires instantiation of a generic type which we don't perform + CHECK(ac.entryMap["m"].wrongIndexType); +} + TEST_CASE_FIXTURE(ACFixture, "string_prim_self_calls_are_fine") { - ScopedFastFlag selfCallAutocompleteFix2{"LuauSelfCallAutocompleteFix2", true}; + ScopedFastFlag selfCallAutocompleteFix3{"LuauSelfCallAutocompleteFix3", true}; check(R"( local s = "hello" @@ -2952,7 +2985,7 @@ s:@1 TEST_CASE_FIXTURE(ACFixture, "string_prim_non_self_calls_are_avoided") { - ScopedFastFlag selfCallAutocompleteFix2{"LuauSelfCallAutocompleteFix2", true}; + ScopedFastFlag selfCallAutocompleteFix3{"LuauSelfCallAutocompleteFix3", true}; check(R"( local s = "hello" @@ -2969,7 +3002,7 @@ s.@1 TEST_CASE_FIXTURE(ACBuiltinsFixture, "library_non_self_calls_are_fine") { - ScopedFastFlag selfCallAutocompleteFix2{"LuauSelfCallAutocompleteFix2", true}; + ScopedFastFlag selfCallAutocompleteFix3{"LuauSelfCallAutocompleteFix3", true}; check(R"( string.@1 @@ -3000,7 +3033,7 @@ table.@1 TEST_CASE_FIXTURE(ACBuiltinsFixture, "library_self_calls_are_invalid") { - ScopedFastFlag selfCallAutocompleteFix2{"LuauSelfCallAutocompleteFix2", true}; + ScopedFastFlag selfCallAutocompleteFix3{"LuauSelfCallAutocompleteFix3", true}; check(R"( string:@1 @@ -3012,8 +3045,11 @@ string:@1 CHECK(ac.entryMap["byte"].wrongIndexType == true); REQUIRE(ac.entryMap.count("char")); CHECK(ac.entryMap["char"].wrongIndexType == true); + + // We want the next test to evaluate to 'true', but we have to allow function defined with 'self' to be callable with ':' + // We may change the definition of the string metatable to not use 'self' types in the future (like byte/char/pack/unpack) REQUIRE(ac.entryMap.count("sub")); - CHECK(ac.entryMap["sub"].wrongIndexType == true); + CHECK(ac.entryMap["sub"].wrongIndexType == false); } TEST_CASE_FIXTURE(ACFixture, "source_module_preservation_and_invalidation") diff --git a/tests/Compiler.test.cpp b/tests/Compiler.test.cpp index c1917638..4b87c003 100644 --- a/tests/Compiler.test.cpp +++ b/tests/Compiler.test.cpp @@ -4352,8 +4352,6 @@ TEST_CASE("LoopUnrollControlFlow") {"LuauCompileLoopUnrollThresholdMaxBoost", 300}, }; - ScopedFastFlag sff("LuauCompileFoldBuiltins", true); - // break jumps to the end CHECK_EQ("\n" + compileFunction(R"( for i=1,3 do @@ -4669,8 +4667,6 @@ TEST_CASE("LoopUnrollCostBuiltins") {"LuauCompileLoopUnrollThresholdMaxBoost", 300}, }; - ScopedFastFlag sff("LuauCompileModelBuiltins", true); - // this loop uses builtins and is close to the cost budget so it's important that we model builtins as cheaper than regular calls CHECK_EQ("\n" + compileFunction(R"( function cipher(block, nonce) @@ -5893,8 +5889,6 @@ RETURN R0 2 TEST_CASE("OptimizationLevel") { - ScopedFastFlag sff("LuauAlwaysCaptureHotComments", true); - // at optimization level 1, no inlining is performed CHECK_EQ("\n" + compileFunction(R"( local function foo(a) @@ -5964,8 +5958,6 @@ RETURN R1 -1 TEST_CASE("BuiltinFolding") { - ScopedFastFlag sff("LuauCompileFoldBuiltins", true); - CHECK_EQ("\n" + compileFunction(R"( return math.abs(-42), @@ -6073,8 +6065,6 @@ RETURN R0 48 TEST_CASE("BuiltinFoldingProhibited") { - ScopedFastFlag sff("LuauCompileFoldBuiltins", true); - CHECK_EQ("\n" + compileFunction(R"( return math.abs(), @@ -6108,9 +6098,6 @@ L3: RETURN R0 -1 TEST_CASE("BuiltinFoldingMultret") { - ScopedFastFlag sff1("LuauCompileFoldBuiltins", true); - ScopedFastFlag sff2("LuauCompileBetterMultret", true); - CHECK_EQ("\n" + compileFunction(R"( local NoLanes: Lanes = --[[ ]] 0b0000000000000000000000000000000 local OffscreenLane: Lane = --[[ ]] 0b1000000000000000000000000000000 diff --git a/tests/Conformance.test.cpp b/tests/Conformance.test.cpp index fc8ab2f9..1339fa0c 100644 --- a/tests/Conformance.test.cpp +++ b/tests/Conformance.test.cpp @@ -316,7 +316,8 @@ TEST_CASE("Errors") TEST_CASE("Events") { - ScopedFastFlag sff("LuauLenTM", true); + ScopedFastFlag sff1("LuauLenTM", true); + ScopedFastFlag sff2("LuauBetterNewindex", true); runConformance("events.lua"); } @@ -490,8 +491,6 @@ static void populateRTTI(lua_State* L, Luau::TypeId type) TEST_CASE("Types") { - ScopedFastFlag sff("LuauCheckLenMT", true); - runConformance("types.lua", [](lua_State* L) { Luau::NullModuleResolver moduleResolver; Luau::InternalErrorReporter iceHandler; @@ -862,8 +861,6 @@ TEST_CASE("ApiCalls") TEST_CASE("ApiAtoms") { - ScopedFastFlag sff("LuauLazyAtoms", true); - StateRef globalState(luaL_newstate(), lua_close); lua_State* L = globalState.get(); diff --git a/tests/ConstraintSolver.test.cpp b/tests/ConstraintSolver.test.cpp index f521c667..e33f6570 100644 --- a/tests/ConstraintSolver.test.cpp +++ b/tests/ConstraintSolver.test.cpp @@ -9,7 +9,7 @@ using namespace Luau; -static TypeId requireBinding(NotNull scope, const char* name) +static TypeId requireBinding(NotNull scope, const char* name) { auto b = linearSearchForBinding(scope, name); LUAU_ASSERT(b.has_value()); @@ -26,7 +26,7 @@ TEST_CASE_FIXTURE(ConstraintGraphBuilderFixture, "hello") )"); cgb.visit(block); - NotNull rootScope = NotNull(cgb.rootScope); + NotNull rootScope = NotNull(cgb.rootScope); ConstraintSolver cs{&arena, rootScope}; @@ -46,7 +46,7 @@ TEST_CASE_FIXTURE(ConstraintGraphBuilderFixture, "generic_function") )"); cgb.visit(block); - NotNull rootScope = NotNull(cgb.rootScope); + NotNull rootScope = NotNull(cgb.rootScope); ConstraintSolver cs{&arena, rootScope}; @@ -73,7 +73,7 @@ TEST_CASE_FIXTURE(ConstraintGraphBuilderFixture, "proper_let_generalization") )"); cgb.visit(block); - NotNull rootScope = NotNull(cgb.rootScope); + NotNull rootScope = NotNull(cgb.rootScope); ToStringOptions opts; diff --git a/tests/Fixture.cpp b/tests/Fixture.cpp index c92c4457..90aa6236 100644 --- a/tests/Fixture.cpp +++ b/tests/Fixture.cpp @@ -258,7 +258,7 @@ std::optional Fixture::getType(const std::string& name) REQUIRE(module); if (FFlag::DebugLuauDeferredConstraintResolution) - return linearSearchForBinding(module->getModuleScope2(), name.c_str()); + return linearSearchForBinding(module->getModuleScope().get(), name.c_str()); else return lookupName(module->getModuleScope(), name); } @@ -434,7 +434,7 @@ BuiltinsFixture::BuiltinsFixture(bool freeze, bool prepareAutocomplete) ConstraintGraphBuilderFixture::ConstraintGraphBuilderFixture() : Fixture() - , cgb(mainModuleName, &arena, NotNull(&ice), frontend.getGlobalScope2()) + , cgb(mainModuleName, &arena, NotNull(&ice), frontend.getGlobalScope()) , forceTheFlag{"DebugLuauDeferredConstraintResolution", true} { BlockedTypeVar::nextIndex = 0; @@ -479,17 +479,17 @@ std::optional lookupName(ScopePtr scope, const std::string& name) return std::nullopt; } -std::optional linearSearchForBinding(Scope2* scope, const char* name) +std::optional linearSearchForBinding(Scope* scope, const char* name) { while (scope) { for (const auto& [n, ty] : scope->bindings) { if (n.astName() == name) - return ty; + return ty.typeId; } - scope = scope->parent; + scope = scope->parent.get(); } return std::nullopt; diff --git a/tests/Fixture.h b/tests/Fixture.h index 4bd6f1ea..a716fe9b 100644 --- a/tests/Fixture.h +++ b/tests/Fixture.h @@ -192,7 +192,7 @@ void dump(const std::vector& constraints); std::optional lookupName(ScopePtr scope, const std::string& name); // Warning: This function runs in O(n**2) -std::optional linearSearchForBinding(Scope2* scope, const char* name); +std::optional linearSearchForBinding(Scope* scope, const char* name); } // namespace Luau diff --git a/tests/Lexer.test.cpp b/tests/Lexer.test.cpp new file mode 100644 index 00000000..20d8d0d5 --- /dev/null +++ b/tests/Lexer.test.cpp @@ -0,0 +1,141 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +#include "Luau/Lexer.h" + +#include "Fixture.h" +#include "ScopedFlags.h" + +#include "doctest.h" + +using namespace Luau; + +TEST_SUITE_BEGIN("LexerTests"); + +TEST_CASE("broken_string_works") +{ + const std::string testInput = "[["; + Luau::Allocator alloc; + AstNameTable table(alloc); + Lexer lexer(testInput.c_str(), testInput.size(), table); + Lexeme lexeme = lexer.next(); + CHECK_EQ(lexeme.type, Lexeme::Type::BrokenString); + CHECK_EQ(lexeme.location, Luau::Location(Luau::Position(0, 0), Luau::Position(0, 2))); +} + +TEST_CASE("broken_comment") +{ + const std::string testInput = "--[[ "; + Luau::Allocator alloc; + AstNameTable table(alloc); + Lexer lexer(testInput.c_str(), testInput.size(), table); + Lexeme lexeme = lexer.next(); + CHECK_EQ(lexeme.type, Lexeme::Type::BrokenComment); + CHECK_EQ(lexeme.location, Luau::Location(Luau::Position(0, 0), Luau::Position(0, 6))); +} + +TEST_CASE("broken_comment_kept") +{ + const std::string testInput = "--[[ "; + Luau::Allocator alloc; + AstNameTable table(alloc); + Lexer lexer(testInput.c_str(), testInput.size(), table); + lexer.setSkipComments(true); + CHECK_EQ(lexer.next().type, Lexeme::Type::BrokenComment); +} + +TEST_CASE("comment_skipped") +{ + const std::string testInput = "-- "; + Luau::Allocator alloc; + AstNameTable table(alloc); + Lexer lexer(testInput.c_str(), testInput.size(), table); + lexer.setSkipComments(true); + CHECK_EQ(lexer.next().type, Lexeme::Type::Eof); +} + +TEST_CASE("multilineCommentWithLexemeInAndAfter") +{ + const std::string testInput = "--[[ function \n" + "]] end"; + Luau::Allocator alloc; + AstNameTable table(alloc); + Lexer lexer(testInput.c_str(), testInput.size(), table); + Lexeme comment = lexer.next(); + Lexeme end = lexer.next(); + + CHECK_EQ(comment.type, Lexeme::Type::BlockComment); + CHECK_EQ(comment.location, Luau::Location(Luau::Position(0, 0), Luau::Position(1, 2))); + CHECK_EQ(end.type, Lexeme::Type::ReservedEnd); + CHECK_EQ(end.location, Luau::Location(Luau::Position(1, 3), Luau::Position(1, 6))); +} + +TEST_CASE("testBrokenEscapeTolerant") +{ + const std::string testInput = "'\\3729472897292378'"; + Luau::Allocator alloc; + AstNameTable table(alloc); + Lexer lexer(testInput.c_str(), testInput.size(), table); + Lexeme item = lexer.next(); + + CHECK_EQ(item.type, Lexeme::QuotedString); + CHECK_EQ(item.location, Luau::Location(Luau::Position(0, 0), Luau::Position(0, int(testInput.size())))); +} + +TEST_CASE("testBigDelimiters") +{ + const std::string testInput = "--[===[\n" + "\n" + "\n" + "\n" + "]===]"; + Luau::Allocator alloc; + AstNameTable table(alloc); + Lexer lexer(testInput.c_str(), testInput.size(), table); + Lexeme item = lexer.next(); + + CHECK_EQ(item.type, Lexeme::Type::BlockComment); + CHECK_EQ(item.location, Luau::Location(Luau::Position(0, 0), Luau::Position(4, 5))); +} + +TEST_CASE("lookahead") +{ + const std::string testInput = "foo --[[ comment ]] bar : nil end"; + + Luau::Allocator alloc; + AstNameTable table(alloc); + Lexer lexer(testInput.c_str(), testInput.size(), table); + lexer.setSkipComments(true); + lexer.next(); // must call next() before reading data from lexer at least once + + CHECK_EQ(lexer.current().type, Lexeme::Name); + CHECK_EQ(lexer.current().name, std::string("foo")); + CHECK_EQ(lexer.lookahead().type, Lexeme::Name); + CHECK_EQ(lexer.lookahead().name, std::string("bar")); + + lexer.next(); + + CHECK_EQ(lexer.current().type, Lexeme::Name); + CHECK_EQ(lexer.current().name, std::string("bar")); + CHECK_EQ(lexer.lookahead().type, ':'); + + lexer.next(); + + CHECK_EQ(lexer.current().type, ':'); + CHECK_EQ(lexer.lookahead().type, Lexeme::ReservedNil); + + lexer.next(); + + CHECK_EQ(lexer.current().type, Lexeme::ReservedNil); + CHECK_EQ(lexer.lookahead().type, Lexeme::ReservedEnd); + + lexer.next(); + + CHECK_EQ(lexer.current().type, Lexeme::ReservedEnd); + CHECK_EQ(lexer.lookahead().type, Lexeme::Eof); + + lexer.next(); + + CHECK_EQ(lexer.current().type, Lexeme::Eof); + CHECK_EQ(lexer.lookahead().type, Lexeme::Eof); +} + +TEST_SUITE_END(); diff --git a/tests/Parser.test.cpp b/tests/Parser.test.cpp index c517853f..c5c019aa 100644 --- a/tests/Parser.test.cpp +++ b/tests/Parser.test.cpp @@ -97,138 +97,6 @@ TEST_CASE("initial_double_is_aligned") TEST_SUITE_END(); -TEST_SUITE_BEGIN("LexerTests"); - -TEST_CASE("broken_string_works") -{ - const std::string testInput = "[["; - Luau::Allocator alloc; - AstNameTable table(alloc); - Lexer lexer(testInput.c_str(), testInput.size(), table); - Lexeme lexeme = lexer.next(); - CHECK_EQ(lexeme.type, Lexeme::Type::BrokenString); - CHECK_EQ(lexeme.location, Luau::Location(Luau::Position(0, 0), Luau::Position(0, 2))); -} - -TEST_CASE("broken_comment") -{ - const std::string testInput = "--[[ "; - Luau::Allocator alloc; - AstNameTable table(alloc); - Lexer lexer(testInput.c_str(), testInput.size(), table); - Lexeme lexeme = lexer.next(); - CHECK_EQ(lexeme.type, Lexeme::Type::BrokenComment); - CHECK_EQ(lexeme.location, Luau::Location(Luau::Position(0, 0), Luau::Position(0, 6))); -} - -TEST_CASE("broken_comment_kept") -{ - const std::string testInput = "--[[ "; - Luau::Allocator alloc; - AstNameTable table(alloc); - Lexer lexer(testInput.c_str(), testInput.size(), table); - lexer.setSkipComments(true); - CHECK_EQ(lexer.next().type, Lexeme::Type::BrokenComment); -} - -TEST_CASE("comment_skipped") -{ - const std::string testInput = "-- "; - Luau::Allocator alloc; - AstNameTable table(alloc); - Lexer lexer(testInput.c_str(), testInput.size(), table); - lexer.setSkipComments(true); - CHECK_EQ(lexer.next().type, Lexeme::Type::Eof); -} - -TEST_CASE("multilineCommentWithLexemeInAndAfter") -{ - const std::string testInput = "--[[ function \n" - "]] end"; - Luau::Allocator alloc; - AstNameTable table(alloc); - Lexer lexer(testInput.c_str(), testInput.size(), table); - Lexeme comment = lexer.next(); - Lexeme end = lexer.next(); - - CHECK_EQ(comment.type, Lexeme::Type::BlockComment); - CHECK_EQ(comment.location, Luau::Location(Luau::Position(0, 0), Luau::Position(1, 2))); - CHECK_EQ(end.type, Lexeme::Type::ReservedEnd); - CHECK_EQ(end.location, Luau::Location(Luau::Position(1, 3), Luau::Position(1, 6))); -} - -TEST_CASE("testBrokenEscapeTolerant") -{ - const std::string testInput = "'\\3729472897292378'"; - Luau::Allocator alloc; - AstNameTable table(alloc); - Lexer lexer(testInput.c_str(), testInput.size(), table); - Lexeme item = lexer.next(); - - CHECK_EQ(item.type, Lexeme::QuotedString); - CHECK_EQ(item.location, Luau::Location(Luau::Position(0, 0), Luau::Position(0, int(testInput.size())))); -} - -TEST_CASE("testBigDelimiters") -{ - const std::string testInput = "--[===[\n" - "\n" - "\n" - "\n" - "]===]"; - Luau::Allocator alloc; - AstNameTable table(alloc); - Lexer lexer(testInput.c_str(), testInput.size(), table); - Lexeme item = lexer.next(); - - CHECK_EQ(item.type, Lexeme::Type::BlockComment); - CHECK_EQ(item.location, Luau::Location(Luau::Position(0, 0), Luau::Position(4, 5))); -} - -TEST_CASE("lookahead") -{ - const std::string testInput = "foo --[[ comment ]] bar : nil end"; - - Luau::Allocator alloc; - AstNameTable table(alloc); - Lexer lexer(testInput.c_str(), testInput.size(), table); - lexer.setSkipComments(true); - lexer.next(); // must call next() before reading data from lexer at least once - - CHECK_EQ(lexer.current().type, Lexeme::Name); - CHECK_EQ(lexer.current().name, std::string("foo")); - CHECK_EQ(lexer.lookahead().type, Lexeme::Name); - CHECK_EQ(lexer.lookahead().name, std::string("bar")); - - lexer.next(); - - CHECK_EQ(lexer.current().type, Lexeme::Name); - CHECK_EQ(lexer.current().name, std::string("bar")); - CHECK_EQ(lexer.lookahead().type, ':'); - - lexer.next(); - - CHECK_EQ(lexer.current().type, ':'); - CHECK_EQ(lexer.lookahead().type, Lexeme::ReservedNil); - - lexer.next(); - - CHECK_EQ(lexer.current().type, Lexeme::ReservedNil); - CHECK_EQ(lexer.lookahead().type, Lexeme::ReservedEnd); - - lexer.next(); - - CHECK_EQ(lexer.current().type, Lexeme::ReservedEnd); - CHECK_EQ(lexer.lookahead().type, Lexeme::Eof); - - lexer.next(); - - CHECK_EQ(lexer.current().type, Lexeme::Eof); - CHECK_EQ(lexer.lookahead().type, Lexeme::Eof); -} - -TEST_SUITE_END(); - TEST_SUITE_BEGIN("ParserTests"); TEST_CASE_FIXTURE(Fixture, "basic_parse") diff --git a/tests/ToString.test.cpp b/tests/ToString.test.cpp index 288af8da..8c03fe55 100644 --- a/tests/ToString.test.cpp +++ b/tests/ToString.test.cpp @@ -10,6 +10,7 @@ using namespace Luau; LUAU_FASTFLAG(LuauRecursiveTypeParameterRestriction); +LUAU_FASTFLAG(LuauSpecialTypesAsterisked); TEST_SUITE_BEGIN("ToString"); @@ -267,8 +268,16 @@ TEST_CASE_FIXTURE(Fixture, "quit_stringifying_type_when_length_is_exceeded") o.maxTypeLength = 40; CHECK_EQ(toString(requireType("f0"), o), "() -> ()"); CHECK_EQ(toString(requireType("f1"), o), "(() -> ()) -> () -> ()"); - CHECK_EQ(toString(requireType("f2"), o), "((() -> ()) -> () -> ()) -> (() -> ()) -> ... "); - CHECK_EQ(toString(requireType("f3"), o), "(((() -> ()) -> () -> ()) -> (() -> ()) -> ... "); + if (FFlag::LuauSpecialTypesAsterisked) + { + CHECK_EQ(toString(requireType("f2"), o), "((() -> ()) -> () -> ()) -> (() -> ()) -> ... *TRUNCATED*"); + CHECK_EQ(toString(requireType("f3"), o), "(((() -> ()) -> () -> ()) -> (() -> ()) -> ... *TRUNCATED*"); + } + else + { + CHECK_EQ(toString(requireType("f2"), o), "((() -> ()) -> () -> ()) -> (() -> ()) -> ... "); + CHECK_EQ(toString(requireType("f3"), o), "(((() -> ()) -> () -> ()) -> (() -> ()) -> ... "); + } } TEST_CASE_FIXTURE(Fixture, "stringifying_type_is_still_capped_when_exhaustive") @@ -286,8 +295,17 @@ TEST_CASE_FIXTURE(Fixture, "stringifying_type_is_still_capped_when_exhaustive") o.maxTypeLength = 40; CHECK_EQ(toString(requireType("f0"), o), "() -> ()"); CHECK_EQ(toString(requireType("f1"), o), "(() -> ()) -> () -> ()"); - CHECK_EQ(toString(requireType("f2"), o), "((() -> ()) -> () -> ()) -> (() -> ()) -> ... "); - CHECK_EQ(toString(requireType("f3"), o), "(((() -> ()) -> () -> ()) -> (() -> ()) -> ... "); + if (FFlag::LuauSpecialTypesAsterisked) + { + CHECK_EQ(toString(requireType("f2"), o), "((() -> ()) -> () -> ()) -> (() -> ()) -> ... *TRUNCATED*"); + CHECK_EQ(toString(requireType("f3"), o), "(((() -> ()) -> () -> ()) -> (() -> ()) -> ... *TRUNCATED*"); + } + else + { + CHECK_EQ(toString(requireType("f2"), o), "((() -> ()) -> () -> ()) -> (() -> ()) -> ... "); + CHECK_EQ(toString(requireType("f3"), o), "(((() -> ()) -> () -> ()) -> (() -> ()) -> ... "); + } + } TEST_CASE_FIXTURE(Fixture, "stringifying_table_type_correctly_use_matching_table_state_braces") @@ -497,7 +515,10 @@ local function target(callback: nil) return callback(4, "hello") end )"); LUAU_REQUIRE_ERRORS(result); - CHECK_EQ("(nil) -> ()", toString(requireType("target"))); + if (FFlag::LuauSpecialTypesAsterisked) + CHECK_EQ("(nil) -> (*error-type*)", toString(requireType("target"))); + else + CHECK_EQ("(nil) -> ()", toString(requireType("target"))); } TEST_CASE_FIXTURE(Fixture, "toStringGenericPack") diff --git a/tests/TypeInfer.anyerror.test.cpp b/tests/TypeInfer.anyerror.test.cpp index 4c5309e0..f4766104 100644 --- a/tests/TypeInfer.anyerror.test.cpp +++ b/tests/TypeInfer.anyerror.test.cpp @@ -13,6 +13,8 @@ using namespace Luau; +LUAU_FASTFLAG(LuauSpecialTypesAsterisked) + TEST_SUITE_BEGIN("TypeInferAnyError"); TEST_CASE_FIXTURE(Fixture, "for_in_loop_iterator_returns_any") @@ -94,7 +96,10 @@ TEST_CASE_FIXTURE(Fixture, "for_in_loop_iterator_is_error") LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK_EQ("", toString(requireType("a"))); + if (FFlag::LuauSpecialTypesAsterisked) + CHECK_EQ("*error-type*", toString(requireType("a"))); + else + CHECK_EQ("", toString(requireType("a"))); } TEST_CASE_FIXTURE(Fixture, "for_in_loop_iterator_is_error2") @@ -110,7 +115,10 @@ TEST_CASE_FIXTURE(Fixture, "for_in_loop_iterator_is_error2") LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK_EQ("", toString(requireType("a"))); + if (FFlag::LuauSpecialTypesAsterisked) + CHECK_EQ("*error-type*", toString(requireType("a"))); + else + CHECK_EQ("", toString(requireType("a"))); } TEST_CASE_FIXTURE(Fixture, "length_of_error_type_does_not_produce_an_error") @@ -225,7 +233,10 @@ TEST_CASE_FIXTURE(Fixture, "calling_error_type_yields_error") CHECK_EQ("unknown", err->name); - CHECK_EQ("", toString(requireType("a"))); + if (FFlag::LuauSpecialTypesAsterisked) + CHECK_EQ("*error-type*", toString(requireType("a"))); + else + CHECK_EQ("", toString(requireType("a"))); } TEST_CASE_FIXTURE(Fixture, "chain_calling_error_type_yields_error") @@ -234,7 +245,10 @@ TEST_CASE_FIXTURE(Fixture, "chain_calling_error_type_yields_error") local a = Utility.Create "Foo" {} )"); - CHECK_EQ("", toString(requireType("a"))); + if (FFlag::LuauSpecialTypesAsterisked) + CHECK_EQ("*error-type*", toString(requireType("a"))); + else + CHECK_EQ("", toString(requireType("a"))); } TEST_CASE_FIXTURE(BuiltinsFixture, "replace_every_free_type_when_unifying_a_complex_function_with_any") diff --git a/tests/TypeInfer.builtins.test.cpp b/tests/TypeInfer.builtins.test.cpp index 4f6adf97..0c878023 100644 --- a/tests/TypeInfer.builtins.test.cpp +++ b/tests/TypeInfer.builtins.test.cpp @@ -9,6 +9,7 @@ using namespace Luau; LUAU_FASTFLAG(LuauLowerBoundsCalculation); +LUAU_FASTFLAG(LuauSpecialTypesAsterisked); TEST_SUITE_BEGIN("BuiltinTests"); @@ -952,7 +953,10 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "table_freeze_is_generic") CHECK_EQ("number", toString(requireType("a"))); CHECK_EQ("string", toString(requireType("b"))); CHECK_EQ("boolean", toString(requireType("c"))); - CHECK_EQ("", toString(requireType("d"))); + if (FFlag::LuauSpecialTypesAsterisked) + CHECK_EQ("*error-type*", toString(requireType("d"))); + else + CHECK_EQ("", toString(requireType("d"))); } TEST_CASE_FIXTURE(BuiltinsFixture, "set_metatable_needs_arguments") diff --git a/tests/TypeInfer.functions.test.cpp b/tests/TypeInfer.functions.test.cpp index a634ba09..70773d95 100644 --- a/tests/TypeInfer.functions.test.cpp +++ b/tests/TypeInfer.functions.test.cpp @@ -14,6 +14,7 @@ using namespace Luau; LUAU_FASTFLAG(LuauLowerBoundsCalculation); +LUAU_FASTFLAG(LuauSpecialTypesAsterisked); TEST_SUITE_BEGIN("TypeInferFunctions"); @@ -907,13 +908,19 @@ TEST_CASE_FIXTURE(Fixture, "function_cast_error_uses_correct_language") REQUIRE(tm1); CHECK_EQ("(string) -> number", toString(tm1->wantedType)); - CHECK_EQ("(string, ) -> number", toString(tm1->givenType)); + if (FFlag::LuauSpecialTypesAsterisked) + CHECK_EQ("(string, *error-type*) -> number", toString(tm1->givenType)); + else + CHECK_EQ("(string, ) -> number", toString(tm1->givenType)); auto tm2 = get(result.errors[1]); REQUIRE(tm2); CHECK_EQ("(number, number) -> (number, number)", toString(tm2->wantedType)); - CHECK_EQ("(string, ) -> number", toString(tm2->givenType)); + if (FFlag::LuauSpecialTypesAsterisked) + CHECK_EQ("(string, *error-type*) -> number", toString(tm2->givenType)); + else + CHECK_EQ("(string, ) -> number", toString(tm2->givenType)); } TEST_CASE_FIXTURE(Fixture, "no_lossy_function_type") @@ -1526,10 +1533,21 @@ function t:b() return 2 end -- not OK )"); LUAU_REQUIRE_ERROR_COUNT(1, result); - CHECK_EQ(R"(Type '() -> number' could not be converted into '() -> number' + if (FFlag::LuauSpecialTypesAsterisked) + { + CHECK_EQ(R"(Type '(*error-type*) -> number' could not be converted into '() -> number' caused by: Argument count mismatch. Function expects 1 argument, but none are specified)", - toString(result.errors[0])); + toString(result.errors[0])); + } + else + { + CHECK_EQ(R"(Type '() -> number' could not be converted into '() -> number' +caused by: + Argument count mismatch. Function expects 1 argument, but none are specified)", + toString(result.errors[0])); + } + } TEST_CASE_FIXTURE(Fixture, "too_few_arguments_variadic") diff --git a/tests/TypeInfer.generics.test.cpp b/tests/TypeInfer.generics.test.cpp index 46258072..7a1af164 100644 --- a/tests/TypeInfer.generics.test.cpp +++ b/tests/TypeInfer.generics.test.cpp @@ -10,6 +10,7 @@ #include "doctest.h" LUAU_FASTFLAG(LuauCheckGenericHOFTypes) +LUAU_FASTFLAG(LuauSpecialTypesAsterisked) using namespace Luau; @@ -1003,7 +1004,10 @@ TEST_CASE_FIXTURE(Fixture, "no_stack_overflow_from_quantifying") std::optional t0 = getMainModule()->getModuleScope()->lookupType("t0"); REQUIRE(t0); - CHECK_EQ("", toString(t0->type)); + if (FFlag::LuauSpecialTypesAsterisked) + CHECK_EQ("*error-type*", toString(t0->type)); + else + CHECK_EQ("", toString(t0->type)); auto it = std::find_if(result.errors.begin(), result.errors.end(), [](TypeError& err) { return get(err); diff --git a/tests/TypeInfer.loops.test.cpp b/tests/TypeInfer.loops.test.cpp index 354b3996..9b10092c 100644 --- a/tests/TypeInfer.loops.test.cpp +++ b/tests/TypeInfer.loops.test.cpp @@ -13,6 +13,8 @@ using namespace Luau; +LUAU_FASTFLAG(LuauSpecialTypesAsterisked) + TEST_SUITE_BEGIN("TypeInferLoops"); TEST_CASE_FIXTURE(Fixture, "for_loop") @@ -142,7 +144,10 @@ TEST_CASE_FIXTURE(Fixture, "for_in_loop_on_error") CHECK_EQ(2, result.errors.size()); TypeId p = requireType("p"); - CHECK_EQ("", toString(p)); + if (FFlag::LuauSpecialTypesAsterisked) + CHECK_EQ("*error-type*", toString(p)); + else + CHECK_EQ("", toString(p)); } TEST_CASE_FIXTURE(Fixture, "for_in_loop_on_non_function") diff --git a/tests/TypeInfer.modules.test.cpp b/tests/TypeInfer.modules.test.cpp index 2343a7fa..a1d41339 100644 --- a/tests/TypeInfer.modules.test.cpp +++ b/tests/TypeInfer.modules.test.cpp @@ -12,6 +12,8 @@ using namespace Luau; +LUAU_FASTFLAG(LuauSpecialTypesAsterisked) + TEST_SUITE_BEGIN("TypeInferModules"); TEST_CASE_FIXTURE(BuiltinsFixture, "require") @@ -143,7 +145,10 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "require_module_that_does_not_export") auto hootyType = requireType(bModule, "Hooty"); - CHECK_EQ("", toString(hootyType)); + if (FFlag::LuauSpecialTypesAsterisked) + CHECK_EQ("*error-type*", toString(hootyType)); + else + CHECK_EQ("", toString(hootyType)); } TEST_CASE_FIXTURE(BuiltinsFixture, "warn_if_you_try_to_require_a_non_modulescript") @@ -244,7 +249,11 @@ local ModuleA = require(game.A) LUAU_REQUIRE_NO_ERRORS(result); std::optional oty = requireType("ModuleA"); - CHECK_EQ("", toString(*oty)); + + if (FFlag::LuauSpecialTypesAsterisked) + CHECK_EQ("*error-type*", toString(*oty)); + else + CHECK_EQ("", toString(*oty)); } TEST_CASE_FIXTURE(BuiltinsFixture, "do_not_modify_imported_types") diff --git a/tests/TypeInfer.operators.test.cpp b/tests/TypeInfer.operators.test.cpp index c90f0a4d..3d6c0193 100644 --- a/tests/TypeInfer.operators.test.cpp +++ b/tests/TypeInfer.operators.test.cpp @@ -490,8 +490,6 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "typecheck_unary_minus_error") TEST_CASE_FIXTURE(BuiltinsFixture, "typecheck_unary_len_error") { - ScopedFastFlag sff("LuauCheckLenMT", true); - CheckResult result = check(R"( --!strict local foo = { diff --git a/tests/TypeInfer.primitives.test.cpp b/tests/TypeInfer.primitives.test.cpp index d60c96e0..a06fd749 100644 --- a/tests/TypeInfer.primitives.test.cpp +++ b/tests/TypeInfer.primitives.test.cpp @@ -12,6 +12,7 @@ #include "doctest.h" LUAU_FASTFLAG(LuauDeduceFindMatchReturnTypes) +LUAU_FASTFLAG(LuauSpecialTypesAsterisked) using namespace Luau; @@ -49,7 +50,10 @@ TEST_CASE_FIXTURE(Fixture, "string_index") REQUIRE(nat); CHECK_EQ("string", toString(nat->ty)); - CHECK_EQ("", toString(requireType("t"))); + if (FFlag::LuauSpecialTypesAsterisked) + CHECK_EQ("*error-type*", toString(requireType("t"))); + else + CHECK_EQ("", toString(requireType("t"))); } TEST_CASE_FIXTURE(Fixture, "string_method") diff --git a/tests/TypeInfer.refinements.test.cpp b/tests/TypeInfer.refinements.test.cpp index cc8cdee3..8a1cadcf 100644 --- a/tests/TypeInfer.refinements.test.cpp +++ b/tests/TypeInfer.refinements.test.cpp @@ -8,6 +8,7 @@ #include "doctest.h" LUAU_FASTFLAG(LuauLowerBoundsCalculation) +LUAU_FASTFLAG(LuauSpecialTypesAsterisked) using namespace Luau; @@ -526,7 +527,10 @@ TEST_CASE_FIXTURE(Fixture, "type_narrow_to_vector") LUAU_REQUIRE_NO_ERRORS(result); - CHECK_EQ("", toString(requireTypeAtPosition({3, 28}))); + if (FFlag::LuauSpecialTypesAsterisked) + CHECK_EQ("*error-type*", toString(requireTypeAtPosition({3, 28}))); + else + CHECK_EQ("", toString(requireTypeAtPosition({3, 28}))); } TEST_CASE_FIXTURE(Fixture, "nonoptional_type_can_narrow_to_nil_if_sense_is_true") diff --git a/tests/TypeInfer.test.cpp b/tests/TypeInfer.test.cpp index 858e8ac0..af6185ef 100644 --- a/tests/TypeInfer.test.cpp +++ b/tests/TypeInfer.test.cpp @@ -16,6 +16,7 @@ LUAU_FASTFLAG(LuauLowerBoundsCalculation); LUAU_FASTFLAG(LuauFixLocationSpanTableIndexExpr); LUAU_FASTFLAG(DebugLuauDeferredConstraintResolution); +LUAU_FASTFLAG(LuauSpecialTypesAsterisked); using namespace Luau; @@ -237,10 +238,21 @@ TEST_CASE_FIXTURE(Fixture, "type_errors_infer_types") // TODO: Should we assert anything about these tests when DCR is being used? if (!FFlag::DebugLuauDeferredConstraintResolution) { - CHECK_EQ("", toString(requireType("c"))); - CHECK_EQ("", toString(requireType("d"))); - CHECK_EQ("", toString(requireType("e"))); - CHECK_EQ("", toString(requireType("f"))); + if (FFlag::LuauSpecialTypesAsterisked) + { + CHECK_EQ("*error-type*", toString(requireType("c"))); + CHECK_EQ("*error-type*", toString(requireType("d"))); + CHECK_EQ("*error-type*", toString(requireType("e"))); + CHECK_EQ("*error-type*", toString(requireType("f"))); + } + else + { + CHECK_EQ("", toString(requireType("c"))); + CHECK_EQ("", toString(requireType("d"))); + CHECK_EQ("", toString(requireType("e"))); + CHECK_EQ("", toString(requireType("f"))); + } + } } @@ -650,7 +662,11 @@ TEST_CASE_FIXTURE(Fixture, "no_stack_overflow_from_isoptional") std::optional t0 = getMainModule()->getModuleScope()->lookupType("t0"); REQUIRE(t0); - CHECK_EQ("", toString(t0->type)); + + if (FFlag::LuauSpecialTypesAsterisked) + CHECK_EQ("*error-type*", toString(t0->type)); + else + CHECK_EQ("", toString(t0->type)); auto it = std::find_if(result.errors.begin(), result.errors.end(), [](TypeError& err) { return get(err); diff --git a/tests/TypeInfer.tryUnify.test.cpp b/tests/TypeInfer.tryUnify.test.cpp index d51a38f8..e0a0e5b5 100644 --- a/tests/TypeInfer.tryUnify.test.cpp +++ b/tests/TypeInfer.tryUnify.test.cpp @@ -9,6 +9,8 @@ using namespace Luau; +LUAU_FASTFLAG(LuauSpecialTypesAsterisked) + struct TryUnifyFixture : Fixture { TypeArena arena; @@ -121,7 +123,10 @@ TEST_CASE_FIXTURE(TryUnifyFixture, "members_of_failed_typepack_unification_are_u LUAU_REQUIRE_ERROR_COUNT(1, result); CHECK_EQ("a", toString(requireType("a"))); - CHECK_EQ("", toString(requireType("b"))); + if (FFlag::LuauSpecialTypesAsterisked) + CHECK_EQ("*error-type*", toString(requireType("b"))); + else + CHECK_EQ("", toString(requireType("b"))); } TEST_CASE_FIXTURE(TryUnifyFixture, "result_of_failed_typepack_unification_is_constrained") @@ -136,7 +141,10 @@ TEST_CASE_FIXTURE(TryUnifyFixture, "result_of_failed_typepack_unification_is_con LUAU_REQUIRE_ERROR_COUNT(1, result); CHECK_EQ("a", toString(requireType("a"))); - CHECK_EQ("", toString(requireType("b"))); + if (FFlag::LuauSpecialTypesAsterisked) + CHECK_EQ("*error-type*", toString(requireType("b"))); + else + CHECK_EQ("", toString(requireType("b"))); CHECK_EQ("number", toString(requireType("c"))); } diff --git a/tests/TypeInfer.unionTypes.test.cpp b/tests/TypeInfer.unionTypes.test.cpp index 94918692..8eb485e9 100644 --- a/tests/TypeInfer.unionTypes.test.cpp +++ b/tests/TypeInfer.unionTypes.test.cpp @@ -7,6 +7,7 @@ #include "doctest.h" LUAU_FASTFLAG(LuauLowerBoundsCalculation) +LUAU_FASTFLAG(LuauSpecialTypesAsterisked) using namespace Luau; @@ -199,7 +200,10 @@ TEST_CASE_FIXTURE(Fixture, "index_on_a_union_type_with_missing_property") CHECK_EQ(mup->missing[0], *bTy); CHECK_EQ(mup->key, "x"); - CHECK_EQ("", toString(requireType("r"))); + if (FFlag::LuauSpecialTypesAsterisked) + CHECK_EQ("*error-type*", toString(requireType("r"))); + else + CHECK_EQ("", toString(requireType("r"))); } TEST_CASE_FIXTURE(Fixture, "index_on_a_union_type_with_one_property_of_type_any") diff --git a/tests/TypePack.test.cpp b/tests/TypePack.test.cpp index 35852c05..1087a24c 100644 --- a/tests/TypePack.test.cpp +++ b/tests/TypePack.test.cpp @@ -25,7 +25,7 @@ struct TypePackFixture TypePackId freshTypePack() { - typePacks.emplace_back(new TypePackVar{Unifiable::Free{{}}}); + typePacks.emplace_back(new TypePackVar{Unifiable::Free{TypeLevel{}}}); return typePacks.back().get(); } diff --git a/tests/conformance/errors.lua b/tests/conformance/errors.lua index 0b5aafed..6c0ac5e6 100644 --- a/tests/conformance/errors.lua +++ b/tests/conformance/errors.lua @@ -380,7 +380,8 @@ assert(ecall(function() return "a" + "b" end) == "attempt to perform arithmetic assert(ecall(function() return 1 > nil end) == "attempt to compare nil < number") -- note reversed order (by design) assert(ecall(function() return "a" <= 5 end) == "attempt to compare string <= number") -assert(ecall(function() local t = {} setmetatable(t, { __newindex = function(t,i,v) end }) t[nil] = 2 end) == "table index is nil") +assert(ecall(function() local t = {} t[nil] = 2 end) == "table index is nil") +assert(ecall(function() local t = {} t[0/0] = 2 end) == "table index is NaN") -- for loop type errors assert(ecall(function() for i='a',2 do end end) == "invalid 'for' initial value (number expected, got string)") diff --git a/tests/conformance/events.lua b/tests/conformance/events.lua index 42f1beda..6dcdbf0e 100644 --- a/tests/conformance/events.lua +++ b/tests/conformance/events.lua @@ -424,4 +424,57 @@ do assert(not ok and err:match("table or string expected")) end +-- verify that NaN/nil keys are passed to __newindex even though table assignment with them anywhere in the chain fails +do + assert(pcall(function() local t = {} t[nil] = 5 end) == false) + assert(pcall(function() local t = {} setmetatable(t, { __newindex = {} }) t[nil] = 5 end) == false) + assert(pcall(function() local t = {} setmetatable(t, { __newindex = function() end }) t[nil] = 5 end) == true) + + assert(pcall(function() local t = {} t[0/0] = 5 end) == false) + assert(pcall(function() local t = {} setmetatable(t, { __newindex = {} }) t[0/0] = 5 end) == false) + assert(pcall(function() local t = {} setmetatable(t, { __newindex = function() end }) t[0/0] = 5 end) == true) +end + +-- verify that __newindex gets called for frozen tables but only if the assignment is to a key absent from the table +do + local ni = {} + local t = table.create(2) + + t[1] = 42 + -- t[2] is semantically absent with storage allocated for it + + t.a = 1 + t.b = 2 + t.b = nil -- this sets 'b' value to nil but leaves key as is to exercise more internal paths -- no observable behavior change expected between b and other absent keys + + setmetatable(t, { __newindex = function(_, k, v) + assert(v == 42) + table.insert(ni, k) + end }) + table.freeze(t) + + -- "redundant" combinations are there to test all three of SETTABLEN/SETTABLEKS/SETTABLE + assert(pcall(function() t.a = 42 end) == false) + assert(pcall(function() t[1] = 42 end) == false) + assert(pcall(function() local key key = "a" t[key] = 42 end) == false) + assert(pcall(function() local key key = 1 t[key] = 42 end) == false) + + -- now repeat the same for keys absent from the table: b (semantically absent), c (physically absent), 2 (semantically absent), 3 (physically absent) + assert(pcall(function() t.b = 42 end) == true) + assert(pcall(function() t.c = 42 end) == true) + assert(pcall(function() local key key = "b" t[key] = 42 end) == true) + assert(pcall(function() local key key = "c" t[key] = 42 end) == true) + assert(pcall(function() t[2] = 42 end) == true) + assert(pcall(function() t[3] = 42 end) == true) + assert(pcall(function() local key key = 2 t[key] = 42 end) == true) + assert(pcall(function() local key key = 3 t[key] = 42 end) == true) + + -- validate the assignment sequence + local ei = { "b", "c", "b", "c", 2, 3, 2, 3 } + assert(#ni == #ei) + for k,v in ni do + assert(ei[k] == v) + end +end + return 'OK' diff --git a/tests/main.cpp b/tests/main.cpp index e7c4aed6..40ccd0b3 100644 --- a/tests/main.cpp +++ b/tests/main.cpp @@ -61,7 +61,7 @@ static bool debuggerPresent() static int testAssertionHandler(const char* expr, const char* file, int line, const char* function) { if (debuggerPresent()) - LUAU_DEBUGBREAK(); + LUAU_DEBUGBREAK(); ADD_FAIL_AT(file, line, "Assertion failed: ", std::string(expr)); return 1; @@ -298,5 +298,3 @@ int main(int argc, char** argv) } return result; } - - diff --git a/tools/faillist.txt b/tools/faillist.txt index dc74a6a9..40fc3c06 100644 --- a/tools/faillist.txt +++ b/tools/faillist.txt @@ -44,7 +44,6 @@ AutocompleteTest.as_types AutocompleteTest.autocomplete_boolean_singleton AutocompleteTest.autocomplete_default_type_pack_parameters AutocompleteTest.autocomplete_default_type_parameters -AutocompleteTest.autocomplete_documentation_symbols AutocompleteTest.autocomplete_end_with_fn_exprs AutocompleteTest.autocomplete_end_with_lambda AutocompleteTest.autocomplete_explicit_type_pack @@ -65,47 +64,34 @@ AutocompleteTest.autocomplete_until_in_repeat AutocompleteTest.autocomplete_while_middle_keywords AutocompleteTest.autocompleteProp_index_function_metamethod_is_variadic AutocompleteTest.bias_toward_inner_scope -AutocompleteTest.comments AutocompleteTest.cyclic_table +AutocompleteTest.do_compatible_self_calls AutocompleteTest.do_not_overwrite_context_sensitive_kws AutocompleteTest.do_not_suggest_internal_module_type -AutocompleteTest.do_not_suggest_synthetic_table_name -AutocompleteTest.dont_offer_any_suggestions_from_the_end_of_a_comment +AutocompleteTest.do_wrong_compatible_self_calls AutocompleteTest.dont_offer_any_suggestions_from_within_a_broken_comment AutocompleteTest.dont_offer_any_suggestions_from_within_a_broken_comment_at_the_very_end_of_the_file AutocompleteTest.dont_offer_any_suggestions_from_within_a_comment AutocompleteTest.dont_suggest_local_before_its_definition -AutocompleteTest.empty_program AutocompleteTest.function_expr_params AutocompleteTest.function_in_assignment_has_parentheses AutocompleteTest.function_in_assignment_has_parentheses_2 AutocompleteTest.function_parameters AutocompleteTest.function_result_passed_to_function_has_parentheses -AutocompleteTest.function_type_types AutocompleteTest.generic_types -AutocompleteTest.get_member_completions -AutocompleteTest.get_string_completions -AutocompleteTest.get_suggestions_for_new_statement AutocompleteTest.get_suggestions_for_the_very_start_of_the_script AutocompleteTest.global_function_params AutocompleteTest.global_functions_are_not_scoped_lexically AutocompleteTest.if_then_else_elseif_completions AutocompleteTest.if_then_else_full_keywords -AutocompleteTest.keyword_members AutocompleteTest.keyword_methods AutocompleteTest.keyword_types -AutocompleteTest.leave_numbers_alone AutocompleteTest.library_non_self_calls_are_fine AutocompleteTest.library_self_calls_are_invalid AutocompleteTest.local_function AutocompleteTest.local_function_params AutocompleteTest.local_functions_fall_out_of_scope -AutocompleteTest.local_initializer -AutocompleteTest.local_initializer_2 -AutocompleteTest.local_names -AutocompleteTest.local_types_builtin AutocompleteTest.method_call_inside_function_body -AutocompleteTest.method_call_inside_if_conditional AutocompleteTest.module_type_members AutocompleteTest.modules_with_types AutocompleteTest.nested_member_completions @@ -114,16 +100,13 @@ AutocompleteTest.no_function_name_suggestions AutocompleteTest.no_incompatible_self_calls AutocompleteTest.no_incompatible_self_calls_2 AutocompleteTest.no_incompatible_self_calls_on_class -AutocompleteTest.no_incompatible_self_calls_provisional -AutocompleteTest.not_the_var_we_are_defining +AutocompleteTest.no_wrong_compatible_self_calls_with_generics AutocompleteTest.optional_members AutocompleteTest.private_types -AutocompleteTest.recommend_statement_starting_keywords AutocompleteTest.recursive_function AutocompleteTest.recursive_function_global AutocompleteTest.recursive_function_local AutocompleteTest.return_types -AutocompleteTest.skip_current_local AutocompleteTest.sometimes_the_metatable_is_an_error AutocompleteTest.source_module_preservation_and_invalidation AutocompleteTest.statement_between_two_statements @@ -154,9 +137,7 @@ AutocompleteTest.type_correct_suggestion_in_table AutocompleteTest.type_scoping_easy AutocompleteTest.unsealed_table AutocompleteTest.unsealed_table_2 -AutocompleteTest.user_defined_globals AutocompleteTest.user_defined_local_functions_in_own_definition -BuiltinDefinitionsTest.lib_documentation_symbols BuiltinTests.aliased_string_format BuiltinTests.assert_removes_falsy_types BuiltinTests.assert_removes_falsy_types2 @@ -231,16 +212,10 @@ BuiltinTests.tonumber_returns_optional_number_type BuiltinTests.tonumber_returns_optional_number_type2 BuiltinTests.xpcall DefinitionTests.class_definition_function_prop -DefinitionTests.class_definitions_cannot_extend_non_class -DefinitionTests.class_definitions_cannot_overload_non_function DefinitionTests.declaring_generic_functions DefinitionTests.definition_file_class_function_args DefinitionTests.definition_file_classes DefinitionTests.definition_file_loading -DefinitionTests.definitions_documentation_symbols -DefinitionTests.documentation_symbols_dont_attach_to_persistent_types -DefinitionTests.load_definition_file_errors_do_not_pollute_global_scope -DefinitionTests.no_cyclic_defined_classes DefinitionTests.single_class_type_identity_in_global_types FrontendTest.accumulate_cached_errors FrontendTest.accumulate_cached_errors_in_consistent_order @@ -256,12 +231,9 @@ FrontendTest.cycle_error_paths FrontendTest.cycle_errors_can_be_fixed FrontendTest.cycle_incremental_type_surface FrontendTest.cycle_incremental_type_surface_longer -FrontendTest.discard_type_graphs FrontendTest.dont_recheck_script_that_hasnt_been_marked_dirty FrontendTest.dont_reparse_clean_file_when_linting FrontendTest.environments -FrontendTest.find_a_require -FrontendTest.find_a_require_inside_a_function FrontendTest.ignore_require_to_nonexistent_file FrontendTest.imported_table_modification_2 FrontendTest.it_should_be_safe_to_stringify_errors_when_full_type_graph_is_discarded @@ -269,29 +241,97 @@ FrontendTest.no_use_after_free_with_type_fun_instantiation FrontendTest.nocheck_cycle_used_by_checked FrontendTest.nocheck_modules_are_typed FrontendTest.produce_errors_for_unchanged_file_with_a_syntax_error -FrontendTest.produce_errors_for_unchanged_file_with_errors FrontendTest.re_report_type_error_in_required_file -FrontendTest.real_source FrontendTest.recheck_if_dependent_script_is_dirty +FrontendTest.reexport_cyclic_type +FrontendTest.reexport_type_alias FrontendTest.report_require_to_nonexistent_file FrontendTest.report_syntax_error_in_required_file FrontendTest.reports_errors_from_multiple_sources FrontendTest.stats_are_not_reset_between_checks -FrontendTest.test_lint_uses_correct_config -FrontendTest.test_pruneParentSegments FrontendTest.trace_requires_in_nonstrict_mode -FrontendTest.typecheck_twice_for_ast_types +GenericsTests.apply_type_function_nested_generics1 +GenericsTests.apply_type_function_nested_generics2 +GenericsTests.better_mismatch_error_messages +GenericsTests.bound_tables_do_not_clone_original_fields +GenericsTests.check_generic_typepack_function +GenericsTests.check_mutual_generic_functions +GenericsTests.correctly_instantiate_polymorphic_member_functions +GenericsTests.do_not_always_instantiate_generic_intersection_types +GenericsTests.do_not_infer_generic_functions +GenericsTests.dont_substitute_bound_types +GenericsTests.dont_unify_bound_types +GenericsTests.duplicate_generic_type_packs +GenericsTests.duplicate_generic_types +GenericsTests.error_detailed_function_mismatch_generic_pack +GenericsTests.error_detailed_function_mismatch_generic_types +GenericsTests.factories_of_generics +GenericsTests.function_arguments_can_be_polytypes +GenericsTests.function_results_can_be_polytypes +GenericsTests.generic_argument_count_too_few +GenericsTests.generic_argument_count_too_many +GenericsTests.generic_factories +GenericsTests.generic_functions_dont_cache_type_parameters +GenericsTests.generic_functions_in_types +GenericsTests.generic_functions_should_be_memory_safe +GenericsTests.generic_table_method +GenericsTests.generic_type_pack_syntax +GenericsTests.generic_type_pack_unification1 +GenericsTests.generic_type_pack_unification2 +GenericsTests.generic_type_pack_unification3 +GenericsTests.infer_generic_function_function_argument +GenericsTests.infer_generic_function_function_argument_overloaded +GenericsTests.infer_generic_lib_function_function_argument +GenericsTests.infer_generic_property +GenericsTests.inferred_local_vars_can_be_polytypes +GenericsTests.instantiate_cyclic_generic_function +GenericsTests.instantiate_generic_function_in_assignments +GenericsTests.instantiate_generic_function_in_assignments2 +GenericsTests.instantiated_function_argument_names +GenericsTests.instantiation_sharing_types +GenericsTests.local_vars_can_be_instantiated_polytypes +GenericsTests.mutable_state_polymorphism +GenericsTests.no_stack_overflow_from_quantifying +GenericsTests.properties_can_be_instantiated_polytypes +GenericsTests.properties_can_be_polytypes +GenericsTests.rank_N_types_via_typeof +GenericsTests.reject_clashing_generic_and_pack_names +GenericsTests.self_recursive_instantiated_param +GenericsTests.substitution_with_bound_table +GenericsTests.typefuns_sharing_types +GenericsTests.variadic_generics +IntersectionTypes.argument_is_intersection +IntersectionTypes.error_detailed_intersection_all +IntersectionTypes.error_detailed_intersection_part +IntersectionTypes.fx_intersection_as_argument +IntersectionTypes.fx_union_as_argument_fails +IntersectionTypes.index_on_an_intersection_type_with_all_parts_missing_the_property +IntersectionTypes.index_on_an_intersection_type_with_mixed_types +IntersectionTypes.index_on_an_intersection_type_with_one_part_missing_the_property +IntersectionTypes.index_on_an_intersection_type_with_one_property_of_type_any +IntersectionTypes.index_on_an_intersection_type_with_property_guaranteed_to_exist +IntersectionTypes.index_on_an_intersection_type_works_at_arbitrary_depth +IntersectionTypes.no_stack_overflow_from_flattenintersection +IntersectionTypes.overload_is_not_a_function +IntersectionTypes.propagates_name +IntersectionTypes.select_correct_union_fn +IntersectionTypes.should_still_pick_an_overload_whose_arguments_are_unions +IntersectionTypes.table_combines +IntersectionTypes.table_combines_missing +IntersectionTypes.table_extra_ok +IntersectionTypes.table_intersection_setmetatable +IntersectionTypes.table_intersection_write +IntersectionTypes.table_intersection_write_sealed +IntersectionTypes.table_intersection_write_sealed_indirect +IntersectionTypes.table_write_sealed_indirect isSubtype.functions_and_any isSubtype.intersection_of_functions_of_different_arities isSubtype.intersection_of_tables isSubtype.table_with_any_prop isSubtype.table_with_table_prop isSubtype.tables -Linter.BuiltinGlobalWrite Linter.DeprecatedApi -Linter.LocalShadowGlobal Linter.TableOperations -Linter.use_all_parent_scopes_for_globals ModuleTests.any_persistance_does_not_leak ModuleTests.builtin_types_point_into_globalTypes_arena ModuleTests.clone_self_property @@ -342,45 +382,237 @@ Normalize.skip_force_normal_on_external_types Normalize.union_of_distinct_free_types Normalize.variadic_tail_is_marked_normal Normalize.visiting_a_type_twice_is_not_considered_normal -ParseErrorRecovery.empty_function_type_error_recovery -ParseErrorRecovery.extra_table_indexer_recovery -ParseErrorRecovery.extra_token_in_consume -ParseErrorRecovery.extra_token_in_consume_match -ParseErrorRecovery.extra_token_in_consume_match_end ParseErrorRecovery.generic_type_list_recovery -ParseErrorRecovery.multiple_parse_errors ParseErrorRecovery.recovery_of_parenthesized_expressions -ParseErrorRecovery.statement_error_recovery_expected -ParseErrorRecovery.statement_error_recovery_unexpected -ParserTests.break_return_not_last_error -ParserTests.continue_not_last_error -ParserTests.error_on_confusable -ParserTests.error_on_non_utf8_sequence -ParserTests.error_on_unicode -ParserTests.export_is_an_identifier_only_when_followed_by_type -ParserTests.functions_cannot_have_return_annotations_if_extensions_are_disabled -ParserTests.illegal_type_alias_if_extensions_are_disabled -ParserTests.incomplete_statement_error -ParserTests.local_cannot_have_annotation_with_extensions_disabled -ParserTests.parse_compound_assignment_error_call -ParserTests.parse_compound_assignment_error_multiple -ParserTests.parse_compound_assignment_error_not_lvalue -ParserTests.parse_error_function_call -ParserTests.parse_error_function_call_newline -ParserTests.parse_error_messages -ParserTests.parse_error_table_literal -ParserTests.parse_error_type_name -ParserTests.parse_nesting_based_end_detection ParserTests.parse_nesting_based_end_detection_failsafe_earlier ParserTests.parse_nesting_based_end_detection_local_function -ParserTests.parse_nesting_based_end_detection_local_repeat -ParserTests.parse_nesting_based_end_detection_nested -ParserTests.parse_nesting_based_end_detection_single_line -ParserTests.parse_numbers_error -ParserTests.parse_numbers_range_error -ParserTests.stop_if_line_ends_with_hyphen -ParserTests.type_alias_error_messages +ProvisionalTests.bail_early_if_unification_is_too_complicated +ProvisionalTests.choose_the_right_overload_for_pcall +ProvisionalTests.constrained_is_level_dependent +ProvisionalTests.discriminate_from_x_not_equal_to_nil +ProvisionalTests.do_not_ice_when_trying_to_pick_first_of_generic_type_pack +ProvisionalTests.error_on_eq_metamethod_returning_a_type_other_than_boolean +ProvisionalTests.free_is_not_bound_to_any +ProvisionalTests.function_returns_many_things_but_first_of_it_is_forgotten +ProvisionalTests.greedy_inference_with_shared_self_triggers_function_with_no_returns +ProvisionalTests.invariant_table_properties_means_instantiating_tables_in_call_is_unsound +ProvisionalTests.it_should_be_agnostic_of_actual_size +ProvisionalTests.lower_bounds_calculation_is_too_permissive_with_overloaded_higher_order_functions +ProvisionalTests.lvalue_equals_another_lvalue_with_no_overlap +ProvisionalTests.normalization_fails_on_certain_kinds_of_cyclic_tables +ProvisionalTests.operator_eq_completely_incompatible +ProvisionalTests.pcall_returns_at_least_two_value_but_function_returns_nothing +ProvisionalTests.setmetatable_constrains_free_type_into_free_table +ProvisionalTests.typeguard_inference_incomplete +ProvisionalTests.weird_fail_to_unify_type_pack +ProvisionalTests.weirditer_should_not_loop_forever +ProvisionalTests.while_body_are_also_refined +ProvisionalTests.xpcall_returns_what_f_returns +RefinementTest.and_constraint +RefinementTest.and_or_peephole_refinement +RefinementTest.apply_refinements_on_astexprindexexpr_whose_subscript_expr_is_constant_string +RefinementTest.assert_a_to_be_truthy_then_assert_a_to_be_number +RefinementTest.assert_non_binary_expressions_actually_resolve_constraints +RefinementTest.assign_table_with_refined_property_with_a_similar_type_is_illegal +RefinementTest.call_a_more_specific_function_using_typeguard +RefinementTest.correctly_lookup_a_shadowed_local_that_which_was_previously_refined +RefinementTest.correctly_lookup_property_whose_base_was_previously_refined +RefinementTest.correctly_lookup_property_whose_base_was_previously_refined2 +RefinementTest.discriminate_from_isa_of_x +RefinementTest.discriminate_from_truthiness_of_x +RefinementTest.discriminate_on_properties_of_disjoint_tables_where_that_property_is_true_or_false +RefinementTest.discriminate_tag +RefinementTest.either_number_or_string +RefinementTest.eliminate_subclasses_of_instance +RefinementTest.falsiness_of_TruthyPredicate_narrows_into_nil +RefinementTest.free_type_is_equal_to_an_lvalue +RefinementTest.impossible_type_narrow_is_not_an_error +RefinementTest.index_on_a_refined_property +RefinementTest.invert_is_truthy_constraint +RefinementTest.invert_is_truthy_constraint_ifelse_expression +RefinementTest.is_truthy_constraint +RefinementTest.is_truthy_constraint_ifelse_expression +RefinementTest.lvalue_is_equal_to_a_term +RefinementTest.lvalue_is_equal_to_another_lvalue +RefinementTest.lvalue_is_not_nil +RefinementTest.merge_should_be_fully_agnostic_of_hashmap_ordering +RefinementTest.narrow_property_of_a_bounded_variable +RefinementTest.narrow_this_large_union +RefinementTest.nonoptional_type_can_narrow_to_nil_if_sense_is_true +RefinementTest.not_a_and_not_b +RefinementTest.not_a_and_not_b2 +RefinementTest.not_a_or_not_b +RefinementTest.not_a_or_not_b2 +RefinementTest.not_and_constraint +RefinementTest.not_t_or_some_prop_of_t +RefinementTest.or_predicate_with_truthy_predicates +RefinementTest.parenthesized_expressions_are_followed_through +RefinementTest.refine_a_property_not_to_be_nil_through_an_intersection_table +RefinementTest.refine_the_correct_types_opposite_of_when_a_is_not_number_or_string +RefinementTest.refine_unknowns +RefinementTest.string_not_equal_to_string_or_nil +RefinementTest.term_is_equal_to_an_lvalue +RefinementTest.truthy_constraint_on_properties +RefinementTest.type_assertion_expr_carry_its_constraints +RefinementTest.type_comparison_ifelse_expression +RefinementTest.type_guard_can_filter_for_intersection_of_tables +RefinementTest.type_guard_can_filter_for_overloaded_function +RefinementTest.type_guard_narrowed_into_nothingness +RefinementTest.type_narrow_for_all_the_userdata +RefinementTest.type_narrow_to_vector +RefinementTest.typeguard_cast_free_table_to_vector +RefinementTest.typeguard_cast_instance_or_vector3_to_vector +RefinementTest.typeguard_doesnt_leak_to_elseif +RefinementTest.typeguard_in_assert_position +RefinementTest.typeguard_in_if_condition_position +RefinementTest.typeguard_narrows_for_functions +RefinementTest.typeguard_narrows_for_table +RefinementTest.typeguard_not_to_be_string +RefinementTest.typeguard_only_look_up_types_from_global_scope +RefinementTest.unknown_lvalue_is_not_synonymous_with_other_on_not_equal +RefinementTest.what_nonsensical_condition +RefinementTest.x_as_any_if_x_is_instance_elseif_x_is_table +RefinementTest.x_is_not_instance_or_else_not_part RuntimeLimits.typescript_port_of_Result_type +TableTests.a_free_shape_can_turn_into_a_scalar_if_it_is_compatible +TableTests.a_free_shape_cannot_turn_into_a_scalar_if_it_is_not_compatible +TableTests.access_index_metamethod_that_returns_variadic +TableTests.accidentally_checked_prop_in_opposite_branch +TableTests.assigning_to_an_unsealed_table_with_string_literal_should_infer_new_properties_over_indexer +TableTests.augment_nested_table +TableTests.augment_table +TableTests.builtin_table_names +TableTests.call_method +TableTests.call_method_with_explicit_self_argument +TableTests.cannot_augment_sealed_table +TableTests.cannot_call_tables +TableTests.cannot_change_type_of_unsealed_table_prop +TableTests.casting_sealed_tables_with_props_into_table_with_indexer +TableTests.casting_tables_with_props_into_table_with_indexer3 +TableTests.casting_tables_with_props_into_table_with_indexer4 +TableTests.casting_unsealed_tables_with_props_into_table_with_indexer +TableTests.checked_prop_too_early +TableTests.common_table_element_general +TableTests.common_table_element_inner_index +TableTests.common_table_element_inner_prop +TableTests.common_table_element_list +TableTests.common_table_element_union_assignment +TableTests.common_table_element_union_in_call +TableTests.common_table_element_union_in_call_tail +TableTests.common_table_element_union_in_prop +TableTests.confusing_indexing +TableTests.defining_a_method_for_a_builtin_sealed_table_must_fail +TableTests.defining_a_method_for_a_local_sealed_table_must_fail +TableTests.defining_a_method_for_a_local_unsealed_table_is_ok +TableTests.defining_a_self_method_for_a_builtin_sealed_table_must_fail +TableTests.defining_a_self_method_for_a_local_sealed_table_must_fail +TableTests.defining_a_self_method_for_a_local_unsealed_table_is_ok +TableTests.dont_crash_when_setmetatable_does_not_produce_a_metatabletypevar +TableTests.dont_hang_when_trying_to_look_up_in_cyclic_metatable_index +TableTests.dont_invalidate_the_properties_iterator_of_free_table_when_rolled_back +TableTests.dont_leak_free_table_props +TableTests.dont_quantify_table_that_belongs_to_outer_scope +TableTests.dont_seal_an_unsealed_table_by_passing_it_to_a_function_that_takes_a_sealed_table +TableTests.dont_suggest_exact_match_keys +TableTests.error_detailed_indexer_key +TableTests.error_detailed_indexer_value +TableTests.error_detailed_metatable_prop +TableTests.error_detailed_prop +TableTests.error_detailed_prop_nested +TableTests.expected_indexer_from_table_union +TableTests.expected_indexer_value_type_extra +TableTests.expected_indexer_value_type_extra_2 +TableTests.explicitly_typed_table +TableTests.explicitly_typed_table_error +TableTests.explicitly_typed_table_with_indexer +TableTests.found_like_key_in_table_function_call +TableTests.found_like_key_in_table_property_access +TableTests.found_multiple_like_keys +TableTests.function_calls_produces_sealed_table_given_unsealed_table +TableTests.generalize_table_argument +TableTests.getmetatable_returns_pointer_to_metatable +TableTests.give_up_after_one_metatable_index_look_up +TableTests.hide_table_error_properties +TableTests.indexer_fn +TableTests.indexer_on_sealed_table_must_unify_with_free_table +TableTests.indexer_table +TableTests.indexing_from_a_table_should_prefer_properties_when_possible +TableTests.inequality_operators_imply_exactly_matching_types +TableTests.infer_array_2 +TableTests.infer_indexer_from_value_property_in_literal +TableTests.inferred_return_type_of_free_table +TableTests.inferring_crazy_table_should_also_be_quick +TableTests.instantiate_table_cloning +TableTests.instantiate_table_cloning_2 +TableTests.instantiate_table_cloning_3 +TableTests.instantiate_tables_at_scope_level +TableTests.invariant_table_properties_means_instantiating_tables_in_assignment_is_unsound +TableTests.leaking_bad_metatable_errors +TableTests.length_operator_intersection +TableTests.length_operator_non_table_union +TableTests.length_operator_union +TableTests.length_operator_union_errors +TableTests.less_exponential_blowup_please +TableTests.meta_add +TableTests.meta_add_both_ways +TableTests.meta_add_inferred +TableTests.metatable_mismatch_should_fail +TableTests.missing_metatable_for_sealed_tables_do_not_get_inferred +TableTests.mixed_tables_with_implicit_numbered_keys +TableTests.MixedPropertiesAndIndexers +TableTests.nil_assign_doesnt_hit_indexer +TableTests.okay_to_add_property_to_unsealed_tables_by_function_call +TableTests.only_ascribe_synthetic_names_at_module_scope +TableTests.oop_indexer_works +TableTests.oop_polymorphic +TableTests.open_table_unification_2 +TableTests.pass_a_union_of_tables_to_a_function_that_requires_a_table +TableTests.pass_a_union_of_tables_to_a_function_that_requires_a_table_2 +TableTests.pass_incompatible_union_to_a_generic_table_without_crashing +TableTests.passing_compatible_unions_to_a_generic_table_without_crashing +TableTests.persistent_sealed_table_is_immutable +TableTests.property_lookup_through_tabletypevar_metatable +TableTests.quantify_even_that_table_was_never_exported_at_all +TableTests.quantify_metatables_of_metatables_of_table +TableTests.quantifying_a_bound_var_works +TableTests.reasonable_error_when_adding_a_nonexistent_property_to_an_array_like_table +TableTests.recursive_metatable_type_call +TableTests.result_is_always_any_if_lhs_is_any +TableTests.result_is_bool_for_equality_operators_if_lhs_is_any +TableTests.right_table_missing_key +TableTests.right_table_missing_key2 +TableTests.scalar_is_a_subtype_of_a_compatible_polymorphic_shape_type +TableTests.scalar_is_not_a_subtype_of_a_compatible_polymorphic_shape_type +TableTests.setmetatable_cant_be_used_to_mutate_global_types +TableTests.shared_selfs +TableTests.shared_selfs_from_free_param +TableTests.shared_selfs_through_metatables +TableTests.table_function_check_use_after_free +TableTests.table_indexing_error_location +TableTests.table_insert_should_cope_with_optional_properties_in_nonstrict +TableTests.table_insert_should_cope_with_optional_properties_in_strict +TableTests.table_length +TableTests.table_param_row_polymorphism_2 +TableTests.table_param_row_polymorphism_3 +TableTests.table_simple_call +TableTests.table_subtyping_with_extra_props_dont_report_multiple_errors +TableTests.table_subtyping_with_missing_props_dont_report_multiple_errors +TableTests.table_subtyping_with_missing_props_dont_report_multiple_errors2 +TableTests.table_unifies_into_map +TableTests.tables_get_names_from_their_locals +TableTests.tc_member_function +TableTests.tc_member_function_2 +TableTests.top_table_type +TableTests.type_mismatch_on_massive_table_is_cut_short +TableTests.unification_of_unions_in_a_self_referential_type +TableTests.unifying_tables_shouldnt_uaf1 +TableTests.unifying_tables_shouldnt_uaf2 +TableTests.used_colon_correctly +TableTests.used_colon_instead_of_dot +TableTests.used_dot_instead_of_colon +TableTests.used_dot_instead_of_colon_but_correctly +TableTests.user_defined_table_types_are_named +TableTests.width_subtyping ToDot.bound_table ToDot.class ToDot.function @@ -401,9 +633,13 @@ ToString.toStringNamedFunction_id ToString.toStringNamedFunction_map ToString.toStringNamedFunction_overrides_param_names ToString.toStringNamedFunction_variadics -TranspilerTests.attach_types TranspilerTests.type_lists_should_be_emitted_correctly TranspilerTests.types_should_not_be_considered_cyclic_if_they_are_not_recursive +TryUnifyTests.cli_41095_concat_log_in_sealed_table_unification +TryUnifyTests.members_of_failed_typepack_unification_are_unified_with_errorType +TryUnifyTests.result_of_failed_typepack_unification_is_constrained +TryUnifyTests.typepack_unification_should_trim_free_tails +TryUnifyTests.variadics_should_use_reversed_properly TypeAliases.basic_alias TypeAliases.cannot_steal_hoisted_type_alias TypeAliases.cli_38393_recursive_intersection_oom @@ -446,6 +682,37 @@ TypeAliases.type_alias_local_synthetic_mutation TypeAliases.type_alias_of_an_imported_recursive_generic_type TypeAliases.type_alias_of_an_imported_recursive_type TypeAliases.use_table_name_and_generic_params_in_errors +TypeInfer.check_expr_recursion_limit +TypeInfer.checking_should_not_ice +TypeInfer.cli_50041_committing_txnlog_in_apollo_client_error +TypeInfer.cyclic_follow +TypeInfer.do_not_bind_a_free_table_to_a_union_containing_that_table +TypeInfer.dont_report_type_errors_within_an_AstStatError +TypeInfer.follow_on_new_types_in_substitution +TypeInfer.free_typevars_introduced_within_control_flow_constructs_do_not_get_an_elevated_TypeLevel +TypeInfer.globals +TypeInfer.globals2 +TypeInfer.index_expr_should_be_checked +TypeInfer.infer_assignment_value_types +TypeInfer.infer_assignment_value_types_mutable_lval +TypeInfer.infer_through_group_expr +TypeInfer.infer_type_assertion_value_type +TypeInfer.no_heap_use_after_free_error +TypeInfer.no_infinite_loop_when_trying_to_unify_uh_this +TypeInfer.no_stack_overflow_from_isoptional +TypeInfer.no_stack_overflow_from_isoptional2 +TypeInfer.recursive_metatable_crash +TypeInfer.tc_after_error_recovery_no_replacement_name_in_error +TypeInfer.tc_if_else_expressions1 +TypeInfer.tc_if_else_expressions2 +TypeInfer.tc_if_else_expressions_expected_type_1 +TypeInfer.tc_if_else_expressions_expected_type_2 +TypeInfer.tc_if_else_expressions_expected_type_3 +TypeInfer.tc_if_else_expressions_type_union +TypeInfer.type_infer_recursion_limit_no_ice +TypeInfer.types stored in astResolvedTypes +TypeInfer.warn_on_lowercase_parent_property +TypeInfer.weird_case TypeInferAnyError.any_type_propagates TypeInferAnyError.assign_prop_to_table_by_calling_any_yields_any TypeInferAnyError.call_to_any_yields_any @@ -496,26 +763,289 @@ TypeInferClasses.we_can_infer_that_a_parameter_must_be_a_particular_class TypeInferClasses.we_can_report_when_someone_is_trying_to_use_a_table_rather_than_a_class TypeInferFunctions.another_indirect_function_case_where_it_is_ok_to_provide_too_many_arguments TypeInferFunctions.another_recursive_local_function +TypeInferFunctions.calling_function_with_anytypepack_doesnt_leak_free_types +TypeInferFunctions.calling_function_with_incorrect_argument_type_yields_errors_spanning_argument TypeInferFunctions.cannot_hoist_interior_defns_into_signature TypeInferFunctions.check_function_before_lambda_that_uses_it TypeInferFunctions.complicated_return_types_require_an_explicit_annotation TypeInferFunctions.cyclic_function_type_in_args TypeInferFunctions.dont_give_other_overloads_message_if_only_one_argument_matching_overload_exists +TypeInferFunctions.dont_infer_parameter_types_for_functions_from_their_call_site +TypeInferFunctions.dont_mutate_the_underlying_head_of_typepack_when_calling_with_self TypeInferFunctions.duplicate_functions_with_different_signatures_not_allowed_in_nonstrict +TypeInferFunctions.error_detailed_function_mismatch_arg +TypeInferFunctions.error_detailed_function_mismatch_arg_count +TypeInferFunctions.error_detailed_function_mismatch_ret +TypeInferFunctions.error_detailed_function_mismatch_ret_count +TypeInferFunctions.error_detailed_function_mismatch_ret_mult TypeInferFunctions.first_argument_can_be_optional +TypeInferFunctions.free_is_not_bound_to_unknown TypeInferFunctions.func_expr_doesnt_leak_free +TypeInferFunctions.function_cast_error_uses_correct_language +TypeInferFunctions.function_decl_non_self_sealed_overwrite +TypeInferFunctions.function_decl_non_self_sealed_overwrite_2 +TypeInferFunctions.function_decl_non_self_unsealed_overwrite +TypeInferFunctions.function_decl_quantify_right_type +TypeInferFunctions.function_does_not_return_enough_values +TypeInferFunctions.function_statement_sealed_table_assignment_through_indexer TypeInferFunctions.higher_order_function_2 TypeInferFunctions.higher_order_function_4 +TypeInferFunctions.ignored_return_values +TypeInferFunctions.inconsistent_higher_order_function +TypeInferFunctions.inconsistent_return_types +TypeInferFunctions.infer_anonymous_function_arguments +TypeInferFunctions.infer_anonymous_function_arguments_outside_call TypeInferFunctions.infer_return_type_from_selected_overload TypeInferFunctions.infer_that_function_does_not_return_a_table +TypeInferFunctions.inferred_higher_order_functions_are_quantified_at_the_right_time +TypeInferFunctions.inferred_higher_order_functions_are_quantified_at_the_right_time2 TypeInferFunctions.it_is_ok_not_to_supply_enough_retvals TypeInferFunctions.it_is_ok_to_oversaturate_a_higher_order_function_argument TypeInferFunctions.list_all_overloads_if_no_overload_takes_given_argument_count TypeInferFunctions.list_only_alternative_overloads_that_match_argument_count TypeInferFunctions.mutual_recursion +TypeInferFunctions.no_lossy_function_type +TypeInferFunctions.occurs_check_failure_in_function_return_type +TypeInferFunctions.quantify_constrained_types +TypeInferFunctions.record_matching_overload TypeInferFunctions.recursive_function TypeInferFunctions.recursive_local_function +TypeInferFunctions.report_exiting_without_return_nonstrict +TypeInferFunctions.report_exiting_without_return_strict +TypeInferFunctions.return_type_by_overload +TypeInferFunctions.strict_mode_ok_with_missing_arguments +TypeInferFunctions.too_few_arguments_variadic +TypeInferFunctions.too_few_arguments_variadic_generic +TypeInferFunctions.too_few_arguments_variadic_generic2 TypeInferFunctions.too_many_arguments +TypeInferFunctions.too_many_return_values TypeInferFunctions.toposort_doesnt_break_mutual_recursion TypeInferFunctions.vararg_function_is_quantified TypeInferFunctions.vararg_functions_should_allow_calls_of_any_types_and_size +TypeInferLoops.correctly_scope_locals_while +TypeInferLoops.for_in_loop +TypeInferLoops.for_in_loop_error_on_factory_not_returning_the_right_amount_of_values +TypeInferLoops.for_in_loop_error_on_iterator_requiring_args_but_none_given +TypeInferLoops.for_in_loop_on_error +TypeInferLoops.for_in_loop_on_non_function +TypeInferLoops.for_in_loop_should_fail_with_non_function_iterator +TypeInferLoops.for_in_loop_where_iteratee_is_free +TypeInferLoops.for_in_loop_with_custom_iterator +TypeInferLoops.for_in_loop_with_next +TypeInferLoops.for_in_with_a_custom_iterator_should_type_check +TypeInferLoops.for_in_with_an_iterator_of_type_any +TypeInferLoops.for_in_with_just_one_iterator_is_ok +TypeInferLoops.fuzz_fail_missing_instantitation_follow +TypeInferLoops.ipairs_produces_integral_indices +TypeInferLoops.loop_iter_basic +TypeInferLoops.loop_iter_iter_metamethod +TypeInferLoops.loop_iter_no_indexer_nonstrict +TypeInferLoops.loop_iter_no_indexer_strict +TypeInferLoops.loop_iter_trailing_nil +TypeInferLoops.loop_typecheck_crash_on_empty_optional +TypeInferLoops.properly_infer_iteratee_is_a_free_table +TypeInferLoops.repeat_loop +TypeInferLoops.repeat_loop_condition_binds_to_its_block +TypeInferLoops.symbols_in_repeat_block_should_not_be_visible_beyond_until_condition +TypeInferLoops.unreachable_code_after_infinite_loop +TypeInferLoops.varlist_declared_by_for_in_loop_should_be_free +TypeInferLoops.while_loop +TypeInferModules.bound_free_table_export_is_ok +TypeInferModules.constrained_anyification_clone_immutable_types +TypeInferModules.custom_require_global +TypeInferModules.do_not_modify_imported_types +TypeInferModules.do_not_modify_imported_types_2 +TypeInferModules.do_not_modify_imported_types_3 +TypeInferModules.do_not_modify_imported_types_4 +TypeInferModules.general_require_call_expression +TypeInferModules.general_require_type_mismatch +TypeInferModules.module_type_conflict +TypeInferModules.module_type_conflict_instantiated +TypeInferModules.require +TypeInferModules.require_a_variadic_function +TypeInferModules.require_failed_module +TypeInferModules.require_module_that_does_not_export +TypeInferModules.require_types +TypeInferModules.type_error_of_unknown_qualified_type +TypeInferModules.warn_if_you_try_to_require_a_non_modulescript +TypeInferOOP.dont_suggest_using_colon_rather_than_dot_if_another_overload_works +TypeInferOOP.dont_suggest_using_colon_rather_than_dot_if_it_wont_help_2 +TypeInferOOP.dont_suggest_using_colon_rather_than_dot_if_not_defined_with_colon +TypeInferOOP.inferred_methods_of_free_tables_have_the_same_level_as_the_enclosing_table +TypeInferOOP.inferring_hundreds_of_self_calls_should_not_suffocate_memory +TypeInferOOP.method_depends_on_table +TypeInferOOP.methods_are_topologically_sorted +TypeInferOOP.nonstrict_self_mismatch_tail +TypeInferOOP.object_constructor_can_refer_to_method_of_self +TypeInferOOP.table_oop +TypeInferOperators.and_adds_boolean +TypeInferOperators.and_adds_boolean_no_superfluous_union +TypeInferOperators.and_binexps_dont_unify +TypeInferOperators.and_or_ternary +TypeInferOperators.CallAndOrOfFunctions +TypeInferOperators.cannot_compare_tables_that_do_not_have_the_same_metatable +TypeInferOperators.cannot_indirectly_compare_types_that_do_not_have_a_metatable +TypeInferOperators.cannot_indirectly_compare_types_that_do_not_offer_overloaded_ordering_operators +TypeInferOperators.cli_38355_recursive_union +TypeInferOperators.compare_numbers +TypeInferOperators.compare_strings +TypeInferOperators.compound_assign_basic +TypeInferOperators.compound_assign_metatable +TypeInferOperators.compound_assign_mismatch_metatable +TypeInferOperators.compound_assign_mismatch_op +TypeInferOperators.compound_assign_mismatch_result +TypeInferOperators.concat_op_on_free_lhs_and_string_rhs +TypeInferOperators.concat_op_on_string_lhs_and_free_rhs +TypeInferOperators.disallow_string_and_types_without_metatables_from_arithmetic_binary_ops +TypeInferOperators.dont_strip_nil_from_rhs_or_operator +TypeInferOperators.equality_operations_succeed_if_any_union_branch_succeeds +TypeInferOperators.error_on_invalid_operand_types_to_relational_operators +TypeInferOperators.error_on_invalid_operand_types_to_relational_operators2 +TypeInferOperators.expected_types_through_binary_and +TypeInferOperators.expected_types_through_binary_or +TypeInferOperators.in_nonstrict_mode_strip_nil_from_intersections_when_considering_relational_operators +TypeInferOperators.infer_any_in_all_modes_when_lhs_is_unknown +TypeInferOperators.operator_eq_operands_are_not_subtypes_of_each_other_but_has_overlap +TypeInferOperators.operator_eq_verifies_types_do_intersect +TypeInferOperators.or_joins_types +TypeInferOperators.or_joins_types_with_no_extras +TypeInferOperators.primitive_arith_no_metatable +TypeInferOperators.primitive_arith_no_metatable_with_follows +TypeInferOperators.primitive_arith_possible_metatable +TypeInferOperators.produce_the_correct_error_message_when_comparing_a_table_with_a_metatable_with_one_that_does_not +TypeInferOperators.refine_and_or +TypeInferOperators.some_primitive_binary_ops +TypeInferOperators.strict_binary_op_where_lhs_unknown +TypeInferOperators.strip_nil_from_lhs_or_operator +TypeInferOperators.strip_nil_from_lhs_or_operator2 +TypeInferOperators.typecheck_overloaded_multiply_that_is_an_intersection +TypeInferOperators.typecheck_overloaded_multiply_that_is_an_intersection_on_rhs +TypeInferOperators.typecheck_unary_len_error +TypeInferOperators.typecheck_unary_minus +TypeInferOperators.typecheck_unary_minus_error +TypeInferOperators.unary_not_is_boolean +TypeInferOperators.unknown_type_in_comparison +TypeInferOperators.UnknownGlobalCompoundAssign +TypeInferPrimitives.cannot_call_primitives +TypeInferPrimitives.CheckMethodsOfNumber +TypeInferPrimitives.string_function_other +TypeInferPrimitives.string_index +TypeInferPrimitives.string_length +TypeInferPrimitives.string_method +TypeInferUnknownNever.array_like_table_of_never_is_inhabitable +TypeInferUnknownNever.assign_to_global_which_is_never +TypeInferUnknownNever.assign_to_local_which_is_never +TypeInferUnknownNever.assign_to_prop_which_is_never +TypeInferUnknownNever.assign_to_subscript_which_is_never +TypeInferUnknownNever.call_never +TypeInferUnknownNever.dont_unify_operands_if_one_of_the_operand_is_never_in_any_ordering_operators +TypeInferUnknownNever.index_on_never +TypeInferUnknownNever.index_on_union_of_tables_for_properties_that_is_never +TypeInferUnknownNever.index_on_union_of_tables_for_properties_that_is_sorta_never +TypeInferUnknownNever.length_of_never +TypeInferUnknownNever.math_operators_and_never +TypeInferUnknownNever.never_is_reflexive +TypeInferUnknownNever.never_subtype_and_string_supertype +TypeInferUnknownNever.string_subtype_and_unknown_supertype +TypeInferUnknownNever.table_with_prop_of_type_never_is_also_reflexive +TypeInferUnknownNever.table_with_prop_of_type_never_is_uninhabitable +TypeInferUnknownNever.type_packs_containing_never_is_itself_uninhabitable +TypeInferUnknownNever.type_packs_containing_never_is_itself_uninhabitable2 +TypeInferUnknownNever.unary_minus_of_never +TypeInferUnknownNever.unknown_is_reflexive +TypePackTests.cyclic_type_packs +TypePackTests.higher_order_function +TypePackTests.multiple_varargs_inference_are_not_confused +TypePackTests.no_return_size_should_be_zero +TypePackTests.pack_tail_unification_check +TypePackTests.parenthesized_varargs_returns_any +TypePackTests.self_and_varargs_should_work +TypePackTests.type_alias_backwards_compatible +TypePackTests.type_alias_default_export +TypePackTests.type_alias_default_mixed_self +TypePackTests.type_alias_default_type_chained +TypePackTests.type_alias_default_type_errors +TypePackTests.type_alias_default_type_explicit +TypePackTests.type_alias_default_type_pack_explicit +TypePackTests.type_alias_default_type_pack_self_chained_tp +TypePackTests.type_alias_default_type_pack_self_tp +TypePackTests.type_alias_default_type_pack_self_ty +TypePackTests.type_alias_default_type_self +TypePackTests.type_alias_default_type_skip_brackets +TypePackTests.type_alias_defaults_confusing_types +TypePackTests.type_alias_defaults_recursive_type +TypePackTests.type_alias_type_pack_explicit +TypePackTests.type_alias_type_pack_explicit_multi +TypePackTests.type_alias_type_pack_explicit_multi_tostring +TypePackTests.type_alias_type_pack_multi +TypePackTests.type_alias_type_pack_variadic +TypePackTests.type_alias_type_packs +TypePackTests.type_alias_type_packs_errors +TypePackTests.type_alias_type_packs_import +TypePackTests.type_alias_type_packs_nested +TypePackTests.type_pack_hidden_free_tail_infinite_growth +TypePackTests.type_pack_type_parameters +TypePackTests.varargs_inference_through_multiple_scopes +TypePackTests.variadic_argument_tail +TypePackTests.variadic_pack_syntax +TypePackTests.variadic_packs +TypeSingletons.bool_singleton_subtype +TypeSingletons.bool_singletons +TypeSingletons.bool_singletons_mismatch +TypeSingletons.enums_using_singletons +TypeSingletons.enums_using_singletons_mismatch +TypeSingletons.enums_using_singletons_subtyping +TypeSingletons.error_detailed_tagged_union_mismatch_bool +TypeSingletons.error_detailed_tagged_union_mismatch_string +TypeSingletons.function_call_with_singletons_mismatch +TypeSingletons.if_then_else_expression_singleton_options +TypeSingletons.indexing_on_string_singletons +TypeSingletons.indexing_on_union_of_string_singletons +TypeSingletons.no_widening_from_callsites +TypeSingletons.overloaded_function_call_with_singletons +TypeSingletons.overloaded_function_call_with_singletons_mismatch +TypeSingletons.return_type_of_f_is_not_widened +TypeSingletons.string_singleton_subtype +TypeSingletons.string_singletons +TypeSingletons.string_singletons_escape_chars +TypeSingletons.string_singletons_mismatch +TypeSingletons.table_insert_with_a_singleton_argument +TypeSingletons.table_properties_alias_or_parens_is_indexer +TypeSingletons.table_properties_singleton_strings +TypeSingletons.table_properties_singleton_strings_mismatch +TypeSingletons.table_properties_type_error_escapes +TypeSingletons.tagged_unions_immutable_tag +TypeSingletons.tagged_unions_using_singletons +TypeSingletons.tagged_unions_using_singletons_mismatch +TypeSingletons.taking_the_length_of_string_singleton +TypeSingletons.taking_the_length_of_union_of_string_singleton +TypeSingletons.widen_the_supertype_if_it_is_free_and_subtype_has_singleton +TypeSingletons.widening_happens_almost_everywhere +TypeSingletons.widening_happens_almost_everywhere_except_for_tables +TypeVarTests.visit_once +UnionTypes.error_detailed_optional +UnionTypes.error_detailed_union_all +UnionTypes.error_detailed_union_part +UnionTypes.error_takes_optional_arguments +UnionTypes.index_on_a_union_type_with_missing_property +UnionTypes.index_on_a_union_type_with_mixed_types +UnionTypes.index_on_a_union_type_with_one_optional_property +UnionTypes.index_on_a_union_type_with_one_property_of_type_any +UnionTypes.index_on_a_union_type_with_property_guaranteed_to_exist +UnionTypes.index_on_a_union_type_works_at_arbitrary_depth +UnionTypes.optional_arguments +UnionTypes.optional_assignment_errors +UnionTypes.optional_call_error +UnionTypes.optional_field_access_error +UnionTypes.optional_index_error +UnionTypes.optional_length_error +UnionTypes.optional_missing_key_error_details +UnionTypes.optional_union_follow +UnionTypes.optional_union_functions +UnionTypes.optional_union_members +UnionTypes.optional_union_methods +UnionTypes.return_types_can_be_disjoint +UnionTypes.table_union_write_indirect +UnionTypes.unify_sealed_table_union_check +UnionTypes.unify_unsealed_table_union_check +UnionTypes.union_equality_comparisons diff --git a/tools/test_dcr.py b/tools/test_dcr.py index 8b090bcd..94ff5ca2 100644 --- a/tools/test_dcr.py +++ b/tools/test_dcr.py @@ -14,6 +14,11 @@ def loadFailList(): with open(FAIL_LIST_PATH) as f: return set(map(str.strip, f.readlines())) +def safeParseInt(i, default=0): + try: + return int(i) + except ValueError: + return default class Handler(x.ContentHandler): def __init__(self, failList): @@ -22,6 +27,8 @@ class Handler(x.ContentHandler): self.results = {} # {DottedName: TrueIfTheTestPassed} + self.numSkippedTests = 0 + def startElement(self, name, attrs): if name == "TestSuite": self.currentTest.append(attrs["name"]) @@ -30,10 +37,7 @@ class Handler(x.ContentHandler): elif name == "OverallResultsAsserts": if self.currentTest: - try: - failed = 0 != int(attrs["failures"]) - except ValueError: - failed = False + failed = 0 != safeParseInt(attrs["failures"]) dottedName = ".".join(self.currentTest) shouldFail = dottedName in self.failList @@ -45,6 +49,9 @@ class Handler(x.ContentHandler): self.results[dottedName] = not failed + elif name == 'OverallResultsTestCases': + self.numSkippedTests = safeParseInt(attrs.get("skipped", 0)) + def endElement(self, name): if name == "TestCase": self.currentTest.pop() @@ -111,6 +118,10 @@ def main(): print(name, file=f) print("Updated faillist.txt") + if handler.numSkippedTests > 0: + print('{} test(s) were skipped! That probably means that a test segfaulted!'.format(handler.numSkippedTests), file=sys.stderr) + sys.exit(1) + sys.exit( 0 if all( From b204981acaceccb88a7c568fa76c3bd1d62c8d49 Mon Sep 17 00:00:00 2001 From: Jay Kruer Date: Tue, 2 Aug 2022 09:32:22 -0700 Subject: [PATCH 326/399] Add Ctrl-C handling to the REPL (#537) Co-authored-by: Arseny Kapoulkine --- CLI/Repl.cpp | 42 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 42 insertions(+) diff --git a/CLI/Repl.cpp b/CLI/Repl.cpp index 41360731..13033bb4 100644 --- a/CLI/Repl.cpp +++ b/CLI/Repl.cpp @@ -20,6 +20,9 @@ #ifdef _WIN32 #include #include + +#define WIN32_LEAN_AND_MEAN +#include #endif #ifdef CALLGRIND @@ -27,6 +30,7 @@ #endif #include +#include LUAU_FASTFLAG(DebugLuauTimeTracing) @@ -47,6 +51,35 @@ enum class CompileFormat constexpr int MaxTraversalLimit = 50; +// Ctrl-C handling +static void sigintCallback(lua_State* L, int gc) +{ + if (gc >= 0) + return; + + lua_callbacks(L)->interrupt = NULL; + + lua_rawcheckstack(L, 1); // reserve space for error string + luaL_error(L, "Execution interrupted"); +} + +static lua_State* replState = NULL; + +#ifdef _WIN32 +BOOL WINAPI sigintHandler(DWORD signal) +{ + if (signal == CTRL_C_EVENT && replState) + lua_callbacks(replState)->interrupt = &sigintCallback; + return TRUE; +} +#else +static void sigintHandler(int signum) +{ + if (signum == SIGINT && replState) + lua_callbacks(replState)->interrupt = &sigintCallback; +} +#endif + struct GlobalOptions { int optimizationLevel = 1; @@ -535,6 +568,15 @@ static void runRepl() lua_State* L = globalState.get(); setupState(L); + + // setup Ctrl+C handling + replState = L; +#ifdef _WIN32 + SetConsoleCtrlHandler(sigintHandler, TRUE); +#else + signal(SIGINT, sigintHandler); +#endif + luaL_sandboxthread(L); runReplImpl(L); } From b39fcc7e77b68bf85ea043383bfe568b22cbbf83 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Wed, 3 Aug 2022 15:38:45 -0700 Subject: [PATCH 327/399] Mark __len RFC as implemented --- rfcs/len-metamethod-rawlen.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/rfcs/len-metamethod-rawlen.md b/rfcs/len-metamethod-rawlen.md index 45284b71..60278dda 100644 --- a/rfcs/len-metamethod-rawlen.md +++ b/rfcs/len-metamethod-rawlen.md @@ -1,5 +1,7 @@ # Support `__len` metamethod for tables and `rawlen` function +**Status**: Implemented + ## Summary `__len` metamethod will be called by `#` operator on tables, matching Lua 5.2 From 4692c55687573f11659dcd04a327dff840bd16f0 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Wed, 3 Aug 2022 15:40:24 -0700 Subject: [PATCH 328/399] Mark never/unknown types RFC as implemented --- rfcs/never-and-unknown-types.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/rfcs/never-and-unknown-types.md b/rfcs/never-and-unknown-types.md index d996afc6..5ad216ef 100644 --- a/rfcs/never-and-unknown-types.md +++ b/rfcs/never-and-unknown-types.md @@ -1,5 +1,7 @@ # never and unknown types +**Status**: Implemented + ## Summary Add `unknown` and `never` types that are inhabited by everything and nothing respectively. From 2c12badb5c2e377ea6644dc8206e753d1d43c3a6 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Wed, 3 Aug 2022 15:40:57 -0700 Subject: [PATCH 329/399] Update RFC status page --- rfcs/STATUS.md | 12 ------------ 1 file changed, 12 deletions(-) diff --git a/rfcs/STATUS.md b/rfcs/STATUS.md index 23a1be83..d2fe86f0 100644 --- a/rfcs/STATUS.md +++ b/rfcs/STATUS.md @@ -26,15 +26,3 @@ This document tracks unimplemented RFCs. [RFC: Lower bounds calculation](https://github.com/Roblox/luau/blob/master/rfcs/lower-bounds-calculation.md) **Status**: Implemented but not fully rolled out yet. - -## never and unknown types - -[RFC: never and unknown types](https://github.com/Roblox/luau/blob/master/rfcs/never-and-unknown-types.md) - -**Status**: Needs implementation - -## __len metamethod for tables and rawlen function - -[RFC: Support __len metamethod for tables and rawlen function](https://github.com/Roblox/luau/blob/master/rfcs/len-metamethod-rawlen.md) - -**Status**: Needs implementation From 4658219df2729af7eff3067470050b46f1f5e9d0 Mon Sep 17 00:00:00 2001 From: boyned//Kampfkarren Date: Thu, 4 Aug 2022 07:22:16 -0700 Subject: [PATCH 330/399] Add %* format specifier (#619) RFC: https://github.com/Roblox/luau/pull/165 --- Analysis/src/Linter.cpp | 2 +- Analysis/src/TypeVar.cpp | 6 ++++-- VM/src/lstrlib.cpp | 22 ++++++++++++++++++++++ tests/Conformance.test.cpp | 2 ++ tests/TypeInfer.builtins.test.cpp | 23 +++++++++++++++++++++++ tests/conformance/strings.lua | 20 ++++++++++++++++++++ 6 files changed, 72 insertions(+), 3 deletions(-) diff --git a/Analysis/src/Linter.cpp b/Analysis/src/Linter.cpp index fb952f5e..6e9e57f2 100644 --- a/Analysis/src/Linter.cpp +++ b/Analysis/src/Linter.cpp @@ -1433,7 +1433,7 @@ private: const char* checkStringFormat(const char* data, size_t size) { const char* flags = "-+ #0"; - const char* options = "cdiouxXeEfgGqs"; + const char* options = "cdiouxXeEfgGqs*"; for (size_t i = 0; i < size; ++i) { diff --git a/Analysis/src/TypeVar.cpp b/Analysis/src/TypeVar.cpp index ebaf5906..51517db9 100644 --- a/Analysis/src/TypeVar.cpp +++ b/Analysis/src/TypeVar.cpp @@ -1058,7 +1058,7 @@ ConstrainedTypeVarIterator end(const ConstrainedTypeVar* ctv) static std::vector parseFormatString(TypeChecker& typechecker, const char* data, size_t size) { - const char* options = "cdiouxXeEfgGqs"; + const char* options = "cdiouxXeEfgGqs*"; std::vector result; @@ -1072,7 +1072,7 @@ static std::vector parseFormatString(TypeChecker& typechecker, const cha continue; // we just ignore all characters (including flags/precision) up until first alphabetic character - while (i < size && !(data[i] > 0 && isalpha(data[i]))) + while (i < size && !(data[i] > 0 && (isalpha(data[i]) || data[i] == '*'))) i++; if (i == size) @@ -1080,6 +1080,8 @@ static std::vector parseFormatString(TypeChecker& typechecker, const cha if (data[i] == 'q' || data[i] == 's') result.push_back(typechecker.stringType); + else if (data[i] == '*') + result.push_back(typechecker.unknownType); else if (strchr(options, data[i])) result.push_back(typechecker.numberType); else diff --git a/VM/src/lstrlib.cpp b/VM/src/lstrlib.cpp index b0cd4dc2..98bbcd41 100644 --- a/VM/src/lstrlib.cpp +++ b/VM/src/lstrlib.cpp @@ -8,6 +8,8 @@ #include #include +LUAU_FASTFLAGVARIABLE(LuauTostringFormatSpecifier, false); + /* macro to `unsign' a character */ #define uchar(c) ((unsigned char)(c)) @@ -1032,6 +1034,26 @@ static int str_format(lua_State* L) break; } } + case '*': + { + if (!FFlag::LuauTostringFormatSpecifier) + { + luaL_error(L, "invalid option '%%*' to 'format'"); + break; + } + + if (formatItemSize != 1) + { + luaL_error(L, "'%%*' does not take a form"); + } + + size_t length; + const char* string = luaL_tolstring(L, arg, &length); + + luaL_addlstring(&b, string, length); + + continue; /* skip the `addsize' at the end */ + } default: { /* also treat cases `pnLlh' */ luaL_error(L, "invalid option '%%%c' to 'format'", *(strfrmt - 1)); diff --git a/tests/Conformance.test.cpp b/tests/Conformance.test.cpp index 1339fa0c..5758f86c 100644 --- a/tests/Conformance.test.cpp +++ b/tests/Conformance.test.cpp @@ -291,6 +291,8 @@ TEST_CASE("Clear") TEST_CASE("Strings") { + ScopedFastFlag sff{"LuauTostringFormatSpecifier", true}; + runConformance("strings.lua"); } diff --git a/tests/TypeInfer.builtins.test.cpp b/tests/TypeInfer.builtins.test.cpp index 0c878023..10da0efa 100644 --- a/tests/TypeInfer.builtins.test.cpp +++ b/tests/TypeInfer.builtins.test.cpp @@ -556,6 +556,29 @@ TEST_CASE_FIXTURE(BuiltinsFixture, "string_format_correctly_ordered_types") CHECK_EQ(tm->givenType, typeChecker.numberType); } +TEST_CASE_FIXTURE(BuiltinsFixture, "string_format_tostring_specifier") +{ + CheckResult result = check(R"( + --!strict + string.format("%* %* %* %*", "string", 1, true, function() end) + )"); + + LUAU_REQUIRE_NO_ERRORS(result); +} + +TEST_CASE_FIXTURE(BuiltinsFixture, "string_format_tostring_specifier_type_constraint") +{ + CheckResult result = check(R"( + local function f(x): string + local _ = string.format("%*", x) + return x + end + )"); + + LUAU_REQUIRE_NO_ERRORS(result); + CHECK_EQ("(string) -> string", toString(requireType("f"))); +} + TEST_CASE_FIXTURE(BuiltinsFixture, "xpcall") { CheckResult result = check(R"( diff --git a/tests/conformance/strings.lua b/tests/conformance/strings.lua index 98a5721a..c87cf15c 100644 --- a/tests/conformance/strings.lua +++ b/tests/conformance/strings.lua @@ -130,6 +130,26 @@ assert(string.format('"-%20s.20s"', string.rep("%", 2000)) == -- longest number that can be formated assert(string.len(string.format('%99.99f', -1e308)) >= 100) +local function return_one_thing() + return "hi" +end +local function return_two_nils() + return nil, nil +end + +assert(string.format("%*", return_one_thing()) == "hi") +assert(string.format("%* %*", return_two_nils()) == "nil nil") +assert(pcall(function() + string.format("%* %* %*", return_two_nils()) +end) == false) + +assert(string.format("%*", "a\0b\0c") == "a\0b\0c") +assert(string.format("%*", string.rep("doge", 3000)) == string.rep("doge", 3000)) + +assert(pcall(function() + string.format("%#*", "bad form") +end) == false) + assert(loadstring("return 1\n--comentrio sem EOL no final")() == 1) From 4a9cfd57a6bd6a31d8c5ccb7b758d13d98b608bd Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Thu, 4 Aug 2022 14:27:28 -0700 Subject: [PATCH 331/399] Sync to upstream/release/539 --- .gitignore | 2 + Analysis/include/Luau/ApplyTypeFunction.h | 32 ++ .../Luau/{JsonEncoder.h => AstJsonEncoder.h} | 0 Analysis/include/Luau/Constraint.h | 10 +- .../include/Luau/ConstraintGraphBuilder.h | 8 +- Analysis/include/Luau/ConstraintSolver.h | 36 +- Analysis/include/Luau/Frontend.h | 5 +- Analysis/include/Luau/JsonEmitter.h | 235 +++++++++++ Analysis/include/Luau/Linter.h | 1 + Analysis/include/Luau/Module.h | 2 +- Analysis/include/Luau/Scope.h | 4 +- Analysis/include/Luau/Substitution.h | 4 + Analysis/include/Luau/TypeInfer.h | 22 - Analysis/include/Luau/TypeVar.h | 37 +- Analysis/include/Luau/VisitTypeVar.h | 42 +- Analysis/src/ApplyTypeFunction.cpp | 60 +++ .../{JsonEncoder.cpp => AstJsonEncoder.cpp} | 5 +- Analysis/src/AstQuery.cpp | 2 +- Analysis/src/BuiltinDefinitions.cpp | 3 +- Analysis/src/Clone.cpp | 52 +++ Analysis/src/ConstraintGraphBuilder.cpp | 195 +++++++-- Analysis/src/ConstraintSolver.cpp | 371 ++++++++++++++++- Analysis/src/ConstraintSolverLogger.cpp | 98 ++--- Analysis/src/Frontend.cpp | 61 ++- Analysis/src/Instantiation.cpp | 4 + Analysis/src/JsonEmitter.cpp | 220 ++++++++++ Analysis/src/Linter.cpp | 45 +- Analysis/src/Module.cpp | 22 +- Analysis/src/Normalize.cpp | 9 +- Analysis/src/Quantify.cpp | 25 +- Analysis/src/Scope.cpp | 2 +- Analysis/src/Substitution.cpp | 159 ++++++- Analysis/src/ToString.cpp | 18 + Analysis/src/TypeAttach.cpp | 5 + Analysis/src/TypeChecker2.cpp | 154 ++++++- Analysis/src/TypeInfer.cpp | 107 +---- Analysis/src/TypeVar.cpp | 31 +- Ast/include/Luau/Ast.h | 12 +- Ast/src/Ast.cpp | 3 +- Ast/src/Parser.cpp | 105 ++++- CLI/Ast.cpp | 2 +- CLI/Repl.cpp | 46 ++- Common/include/Luau/Bytecode.h | 24 +- Common/include/Luau/Common.h | 19 +- Common/include/Luau/ExperimentalFlags.h | 7 +- Compiler/include/luacode.h | 4 +- Compiler/src/Builtins.cpp | 4 +- Compiler/src/BytecodeBuilder.cpp | 51 +++ Compiler/src/Compiler.cpp | 124 ++++-- Makefile | 8 +- Sources.cmake | 11 +- VM/include/lua.h | 80 ++-- VM/include/luaconf.h | 32 +- VM/include/lualib.h | 8 +- VM/src/lapi.cpp | 52 +-- VM/src/laux.cpp | 70 ++-- VM/src/lbaselib.cpp | 69 ++-- VM/src/lbitlib.cpp | 18 +- VM/src/lcommon.h | 2 +- VM/src/lcorolib.cpp | 36 +- VM/src/ldblib.cpp | 4 +- VM/src/ldebug.cpp | 11 +- VM/src/ldebug.h | 1 + VM/src/ldo.cpp | 40 +- VM/src/ldo.h | 10 +- VM/src/lfunc.cpp | 20 +- VM/src/lgc.cpp | 130 +++--- VM/src/lgc.h | 6 +- VM/src/lgcdebug.cpp | 4 +- VM/src/lmathlib.cpp | 18 +- VM/src/lnumutils.h | 2 +- VM/src/lobject.cpp | 24 +- VM/src/lobject.h | 74 ++-- VM/src/loslib.cpp | 22 +- VM/src/lstate.cpp | 48 +-- VM/src/lstate.h | 100 ++--- VM/src/lstring.cpp | 20 +- VM/src/lstring.h | 4 +- VM/src/lstrlib.cpp | 390 +++++++++--------- VM/src/ltable.cpp | 192 ++++----- VM/src/ltablib.cpp | 134 +++--- VM/src/ltm.cpp | 12 +- VM/src/ltm.h | 4 +- VM/src/ludata.h | 4 +- VM/src/lutf8lib.cpp | 90 ++-- VM/src/lvmexecute.cpp | 107 ++++- VM/src/lvmutils.cpp | 125 +++--- ...coder.test.cpp => AstJsonEncoder.test.cpp} | 5 +- tests/Autocomplete.test.cpp | 2 +- tests/Compiler.test.cpp | 47 ++- tests/Conformance.test.cpp | 13 +- tests/Fixture.cpp | 2 +- tests/Frontend.test.cpp | 2 - tests/JsonEmitter.test.cpp | 195 +++++++++ tests/Linter.test.cpp | 32 ++ tests/Normalize.test.cpp | 21 +- tests/Parser.test.cpp | 13 +- tests/Repl.test.cpp | 2 +- tests/ToString.test.cpp | 5 - tests/TypeInfer.aliases.test.cpp | 94 +++++ tests/TypeInfer.builtins.test.cpp | 23 ++ tests/TypeInfer.functions.test.cpp | 1 - tests/TypeInfer.generics.test.cpp | 4 - tests/TypeInfer.provisional.test.cpp | 1 - tests/TypeInfer.singletons.test.cpp | 2 +- tests/TypeInfer.test.cpp | 1 - tests/TypeInfer.typePacks.cpp | 2 +- tests/conformance/errors.lua | 7 + tests/conformance/strings.lua | 20 + tests/main.cpp | 2 +- tools/faillist.txt | 83 +--- tools/perfgraph.py | 72 +++- tools/test_dcr.py | 20 +- 113 files changed, 3653 insertions(+), 1489 deletions(-) create mode 100644 Analysis/include/Luau/ApplyTypeFunction.h rename Analysis/include/Luau/{JsonEncoder.h => AstJsonEncoder.h} (100%) create mode 100644 Analysis/include/Luau/JsonEmitter.h create mode 100644 Analysis/src/ApplyTypeFunction.cpp rename Analysis/src/{JsonEncoder.cpp => AstJsonEncoder.cpp} (99%) create mode 100644 Analysis/src/JsonEmitter.cpp rename tests/{JsonEncoder.test.cpp => AstJsonEncoder.test.cpp} (99%) create mode 100644 tests/JsonEmitter.test.cpp diff --git a/.gitignore b/.gitignore index ec5e6578..af77f73c 100644 --- a/.gitignore +++ b/.gitignore @@ -8,4 +8,6 @@ /default.prof* /fuzz-* /luau +/luau-tests +/luau-analyze __pycache__ diff --git a/Analysis/include/Luau/ApplyTypeFunction.h b/Analysis/include/Luau/ApplyTypeFunction.h new file mode 100644 index 00000000..8da3bc42 --- /dev/null +++ b/Analysis/include/Luau/ApplyTypeFunction.h @@ -0,0 +1,32 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +#pragma once + +#include "Luau/Substitution.h" +#include "Luau/TxnLog.h" +#include "Luau/TypeVar.h" + +namespace Luau +{ + +// A substitution which replaces the type parameters of a type function by arguments +struct ApplyTypeFunction : Substitution +{ + ApplyTypeFunction(TypeArena* arena) + : Substitution(TxnLog::empty(), arena) + , encounteredForwardedType(false) + { + } + + // Never set under deferred constraint resolution. + bool encounteredForwardedType; + std::unordered_map typeArguments; + std::unordered_map typePackArguments; + bool ignoreChildren(TypeId ty) override; + bool ignoreChildren(TypePackId tp) override; + bool isDirty(TypeId ty) override; + bool isDirty(TypePackId tp) override; + TypeId clean(TypeId ty) override; + TypePackId clean(TypePackId tp) override; +}; + +} // namespace Luau diff --git a/Analysis/include/Luau/JsonEncoder.h b/Analysis/include/Luau/AstJsonEncoder.h similarity index 100% rename from Analysis/include/Luau/JsonEncoder.h rename to Analysis/include/Luau/AstJsonEncoder.h diff --git a/Analysis/include/Luau/Constraint.h b/Analysis/include/Luau/Constraint.h index 9911c4d3..5b737146 100644 --- a/Analysis/include/Luau/Constraint.h +++ b/Analysis/include/Luau/Constraint.h @@ -4,6 +4,7 @@ #include "Luau/Ast.h" // Used for some of the enumerations #include "Luau/NotNull.h" #include "Luau/Variant.h" +#include "Luau/TypeVar.h" #include #include @@ -71,8 +72,15 @@ struct NameConstraint std::string name; }; +// target ~ inst target +struct TypeAliasExpansionConstraint +{ + // Must be a PendingExpansionTypeVar. + TypeId target; +}; + using ConstraintV = Variant; + BinaryConstraint, NameConstraint, TypeAliasExpansionConstraint>; using ConstraintPtr = std::unique_ptr; struct Constraint diff --git a/Analysis/include/Luau/ConstraintGraphBuilder.h b/Analysis/include/Luau/ConstraintGraphBuilder.h index 695b6cd5..69f35d46 100644 --- a/Analysis/include/Luau/ConstraintGraphBuilder.h +++ b/Analysis/include/Luau/ConstraintGraphBuilder.h @@ -42,6 +42,8 @@ struct ConstraintGraphBuilder DenseHashMap astResolvedTypes{nullptr}; // Type packs resolved from type annotations. Analogous to astTypePacks. DenseHashMap astResolvedTypePacks{nullptr}; + // Defining scopes for AST nodes. + DenseHashMap astTypeAliasDefiningScopes{nullptr}; int recursionCount = 0; @@ -107,6 +109,9 @@ struct ConstraintGraphBuilder void visit(const ScopePtr& scope, AstStatAssign* assign); void visit(const ScopePtr& scope, AstStatIf* ifStatement); void visit(const ScopePtr& scope, AstStatTypeAlias* alias); + void visit(const ScopePtr& scope, AstStatDeclareGlobal* declareGlobal); + void visit(const ScopePtr& scope, AstStatDeclareClass* declareClass); + void visit(const ScopePtr& scope, AstStatDeclareFunction* declareFunction); TypePackId checkExprList(const ScopePtr& scope, const AstArray& exprs); @@ -153,9 +158,10 @@ struct ConstraintGraphBuilder * Resolves a type from its AST annotation. * @param scope the scope that the type annotation appears within. * @param ty the AST annotation to resolve. + * @param topLevel whether the annotation is a "top-level" annotation. * @return the type of the AST annotation. **/ - TypeId resolveType(const ScopePtr& scope, AstType* ty); + TypeId resolveType(const ScopePtr& scope, AstType* ty, bool topLevel = false); /** * Resolves a type pack from its AST annotation. diff --git a/Analysis/include/Luau/ConstraintSolver.h b/Analysis/include/Luau/ConstraintSolver.h index 0b9b4ae3..9cc0e4cb 100644 --- a/Analysis/include/Luau/ConstraintSolver.h +++ b/Analysis/include/Luau/ConstraintSolver.h @@ -17,16 +17,36 @@ namespace Luau // never dereference this pointer. using BlockedConstraintId = const void*; +struct InstantiationSignature +{ + TypeFun fn; + std::vector arguments; + std::vector packArguments; + + bool operator==(const InstantiationSignature& rhs) const; + bool operator!=(const InstantiationSignature& rhs) const + { + return !((*this) == rhs); + } +}; + +struct HashInstantiationSignature +{ + size_t operator()(const InstantiationSignature& signature) const; +}; + struct ConstraintSolver { TypeArena* arena; InternalErrorReporter iceReporter; - // The entire set of constraints that the solver is trying to resolve. It - // is important to not add elements to this vector, lest the underlying - // storage that we retain pointers to be mutated underneath us. - const std::vector> constraints; + // The entire set of constraints that the solver is trying to resolve. + std::vector> constraints; NotNull rootScope; + // Constraints that the solver has generated, rather than sourcing from the + // scope tree. + std::vector> solverConstraints; + // This includes every constraint that has not been fully solved. // A constraint can be both blocked and unsolved, for instance. std::vector> unsolvedConstraints; @@ -37,6 +57,8 @@ struct ConstraintSolver std::unordered_map, size_t> blockedConstraints; // A mapping of type/pack pointers to the constraints they block. std::unordered_map>> blocked; + // Memoized instantiations of type aliases. + DenseHashMap instantiatedAliases{{}}; ConstraintSolverLogger logger; @@ -62,6 +84,7 @@ struct ConstraintSolver bool tryDispatch(const UnaryConstraint& c, NotNull constraint, bool force); bool tryDispatch(const BinaryConstraint& c, NotNull constraint, bool force); bool tryDispatch(const NameConstraint& c, NotNull constraint); + bool tryDispatch(const TypeAliasExpansionConstraint& c, NotNull constraint); void block(NotNull target, NotNull constraint); /** @@ -102,6 +125,11 @@ struct ConstraintSolver */ void unify(TypePackId subPack, TypePackId superPack); + /** Pushes a new solver constraint to the solver. + * @param cv the body of the constraint. + **/ + void pushConstraint(ConstraintV cv); + private: /** * Marks a constraint as being blocked on a type or type pack. The constraint diff --git a/Analysis/include/Luau/Frontend.h b/Analysis/include/Luau/Frontend.h index 3b0164c2..9b8ec19e 100644 --- a/Analysis/include/Luau/Frontend.h +++ b/Analysis/include/Luau/Frontend.h @@ -152,6 +152,8 @@ struct Frontend void registerBuiltinDefinition(const std::string& name, std::function); void applyBuiltinDefinitionToEnvironment(const std::string& environmentName, const std::string& definitionName); + LoadDefinitionFileResult loadDefinitionFile(std::string_view source, const std::string& packageName); + NotNull getGlobalScope(); private: @@ -169,7 +171,7 @@ private: std::unordered_map environments; std::unordered_map> builtinDefinitions; - std::unique_ptr globalScope; + ScopePtr globalScope; public: FileResolver* fileResolver; @@ -180,6 +182,7 @@ public: ConfigResolver* configResolver; FrontendOptions options; InternalErrorReporter iceHandler; + TypeArena globalTypes; TypeArena arenaForAutocomplete; std::unordered_map sourceNodes; diff --git a/Analysis/include/Luau/JsonEmitter.h b/Analysis/include/Luau/JsonEmitter.h new file mode 100644 index 00000000..0bf3327a --- /dev/null +++ b/Analysis/include/Luau/JsonEmitter.h @@ -0,0 +1,235 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +#pragma once + +#include +#include +#include +#include + +#include "Luau/NotNull.h" + +namespace Luau::Json +{ + +struct JsonEmitter; + +/// Writes a value to the JsonEmitter. Note that this can produce invalid JSON +/// if you do not insert commas or appropriate object / array syntax. +template +void write(JsonEmitter&, T) = delete; + +/// Writes a boolean to a JsonEmitter. +/// @param emitter the emitter to write to. +/// @param b the boolean to write. +void write(JsonEmitter& emitter, bool b); + +/// Writes an integer to a JsonEmitter. +/// @param emitter the emitter to write to. +/// @param i the integer to write. +void write(JsonEmitter& emitter, int i); + +/// Writes an integer to a JsonEmitter. +/// @param emitter the emitter to write to. +/// @param i the integer to write. +void write(JsonEmitter& emitter, long i); + +/// Writes an integer to a JsonEmitter. +/// @param emitter the emitter to write to. +/// @param i the integer to write. +void write(JsonEmitter& emitter, long long i); + +/// Writes an integer to a JsonEmitter. +/// @param emitter the emitter to write to. +/// @param i the integer to write. +void write(JsonEmitter& emitter, unsigned int i); + +/// Writes an integer to a JsonEmitter. +/// @param emitter the emitter to write to. +/// @param i the integer to write. +void write(JsonEmitter& emitter, unsigned long i); + +/// Writes an integer to a JsonEmitter. +/// @param emitter the emitter to write to. +/// @param i the integer to write. +void write(JsonEmitter& emitter, unsigned long long i); + +/// Writes a double to a JsonEmitter. +/// @param emitter the emitter to write to. +/// @param d the double to write. +void write(JsonEmitter& emitter, double d); + +/// Writes a string to a JsonEmitter. The string will be escaped. +/// @param emitter the emitter to write to. +/// @param sv the string to write. +void write(JsonEmitter& emitter, std::string_view sv); + +/// Writes a character to a JsonEmitter as a single-character string. The +/// character will be escaped. +/// @param emitter the emitter to write to. +/// @param c the string to write. +void write(JsonEmitter& emitter, char c); + +/// Writes a string to a JsonEmitter. The string will be escaped. +/// @param emitter the emitter to write to. +/// @param str the string to write. +void write(JsonEmitter& emitter, const char* str); + +/// Writes a string to a JsonEmitter. The string will be escaped. +/// @param emitter the emitter to write to. +/// @param str the string to write. +void write(JsonEmitter& emitter, const std::string& str); + +/// Writes null to a JsonEmitter. +/// @param emitter the emitter to write to. +void write(JsonEmitter& emitter, std::nullptr_t); + +/// Writes null to a JsonEmitter. +/// @param emitter the emitter to write to. +void write(JsonEmitter& emitter, std::nullopt_t); + +struct ObjectEmitter; +struct ArrayEmitter; + +struct JsonEmitter +{ + JsonEmitter(); + + /// Converts the current contents of the JsonEmitter to a string value. This + /// does not invalidate the emitter, but it does not clear it either. + std::string str(); + + /// Returns the current comma state and resets it to false. Use popComma to + /// restore the old state. + /// @returns the previous comma state. + bool pushComma(); + + /// Restores a previous comma state. + /// @param c the comma state to restore. + void popComma(bool c); + + /// Writes a raw sequence of characters to the buffer, without escaping or + /// other processing. + /// @param sv the character sequence to write. + void writeRaw(std::string_view sv); + + /// Writes a character to the buffer, without escaping or other processing. + /// @param c the character to write. + void writeRaw(char c); + + /// Writes a comma if this wasn't the first time writeComma has been + /// invoked. Otherwise, sets the comma state to true. + /// @see pushComma + /// @see popComma + void writeComma(); + + /// Begins writing an object to the emitter. + /// @returns an ObjectEmitter that can be used to write key-value pairs. + ObjectEmitter writeObject(); + + /// Begins writing an array to the emitter. + /// @returns an ArrayEmitter that can be used to write values. + ArrayEmitter writeArray(); + +private: + bool comma = false; + std::vector chunks; + + void newChunk(); +}; + +/// An interface for writing an object into a JsonEmitter instance. +/// @see JsonEmitter::writeObject +struct ObjectEmitter +{ + ObjectEmitter(NotNull emitter); + ~ObjectEmitter(); + + NotNull emitter; + bool comma; + bool finished; + + /// Writes a key-value pair to the associated JsonEmitter. Keys will be escaped. + /// @param name the name of the key-value pair. + /// @param value the value to write. + template + void writePair(std::string_view name, T value) + { + if (finished) + { + return; + } + + emitter->writeComma(); + write(*emitter, name); + emitter->writeRaw(':'); + write(*emitter, value); + } + + /// Finishes writing the object, appending a closing `}` character and + /// resetting the comma state of the associated emitter. This can only be + /// called once, and once called will render the emitter unusable. This + /// method is also called when the ObjectEmitter is destructed. + void finish(); +}; + +/// An interface for writing an array into a JsonEmitter instance. Array values +/// do not need to be the same type. +/// @see JsonEmitter::writeArray +struct ArrayEmitter +{ + ArrayEmitter(NotNull emitter); + ~ArrayEmitter(); + + NotNull emitter; + bool comma; + bool finished; + + /// Writes a value to the array. + /// @param value the value to write. + template + void writeValue(T value) + { + if (finished) + { + return; + } + + emitter->writeComma(); + write(*emitter, value); + } + + /// Finishes writing the object, appending a closing `]` character and + /// resetting the comma state of the associated emitter. This can only be + /// called once, and once called will render the emitter unusable. This + /// method is also called when the ArrayEmitter is destructed. + void finish(); +}; + +/// Writes a vector as an array to a JsonEmitter. +/// @param emitter the emitter to write to. +/// @param vec the vector to write. +template +void write(JsonEmitter& emitter, const std::vector& vec) +{ + ArrayEmitter a = emitter.writeArray(); + + for (const T& value : vec) + a.writeValue(value); + + a.finish(); +} + +/// Writes an optional to a JsonEmitter. Will write the contained value, if +/// present, or null, if no value is present. +/// @param emitter the emitter to write to. +/// @param v the value to write. +template +void write(JsonEmitter& emitter, const std::optional& v) +{ + if (v.has_value()) + write(emitter, *v); + else + emitter.writeRaw("null"); +} + +} // namespace Luau::Json diff --git a/Analysis/include/Luau/Linter.h b/Analysis/include/Luau/Linter.h index 6c7ce47f..2cd91d54 100644 --- a/Analysis/include/Luau/Linter.h +++ b/Analysis/include/Luau/Linter.h @@ -52,6 +52,7 @@ struct LintWarning Code_DuplicateCondition = 24, Code_MisleadingAndOr = 25, Code_CommentDirective = 26, + Code_IntegerParsing = 27, Code__Count }; diff --git a/Analysis/include/Luau/Module.h b/Analysis/include/Luau/Module.h index 73f4c972..6f4c6098 100644 --- a/Analysis/include/Luau/Module.h +++ b/Analysis/include/Luau/Module.h @@ -68,7 +68,7 @@ struct Module std::shared_ptr allocator; std::shared_ptr names; - std::vector> scopes; // never empty + std::vector> scopes; // never empty DenseHashMap astTypes{nullptr}; DenseHashMap astTypePacks{nullptr}; diff --git a/Analysis/include/Luau/Scope.h b/Analysis/include/Luau/Scope.h index 85c1750a..55ca54c6 100644 --- a/Analysis/include/Luau/Scope.h +++ b/Analysis/include/Luau/Scope.h @@ -36,7 +36,7 @@ struct Scope // All the children of this scope. std::vector> children; std::unordered_map bindings; - std::unordered_map typeBindings; + std::unordered_map typeBindings; std::unordered_map typePackBindings; TypePackId returnType; std::optional varargPack; @@ -52,7 +52,7 @@ struct Scope std::unordered_map> importedTypeBindings; std::optional lookup(Symbol sym); - std::optional lookupTypeBinding(const Name& name); + std::optional lookupTypeBinding(const Name& name); std::optional lookupTypePackBinding(const Name& name); std::optional lookupType(const Name& name); diff --git a/Analysis/include/Luau/Substitution.h b/Analysis/include/Luau/Substitution.h index f3c3ae9a..6ad38f9d 100644 --- a/Analysis/include/Luau/Substitution.h +++ b/Analysis/include/Luau/Substitution.h @@ -139,6 +139,8 @@ struct FindDirty : Tarjan { std::vector dirty; + void clearTarjan(); + // Get/set the dirty bit for an index (grows the vector if needed) bool getDirty(int index); void setDirty(int index, bool d); @@ -176,6 +178,8 @@ public: TypeArena* arena; DenseHashMap newTypes{nullptr}; DenseHashMap newPacks{nullptr}; + DenseHashSet replacedTypes{nullptr}; + DenseHashSet replacedTypePacks{nullptr}; std::optional substitute(TypeId ty); std::optional substitute(TypePackId tp); diff --git a/Analysis/include/Luau/TypeInfer.h b/Analysis/include/Luau/TypeInfer.h index 3fb710bb..c50b2c8c 100644 --- a/Analysis/include/Luau/TypeInfer.h +++ b/Analysis/include/Luau/TypeInfer.h @@ -65,28 +65,6 @@ struct Anyification : Substitution } }; -// A substitution which replaces the type parameters of a type function by arguments -struct ApplyTypeFunction : Substitution -{ - ApplyTypeFunction(TypeArena* arena, TypeLevel level) - : Substitution(TxnLog::empty(), arena) - , level(level) - , encounteredForwardedType(false) - { - } - - TypeLevel level; - bool encounteredForwardedType; - std::unordered_map typeArguments; - std::unordered_map typePackArguments; - bool ignoreChildren(TypeId ty) override; - bool ignoreChildren(TypePackId tp) override; - bool isDirty(TypeId ty) override; - bool isDirty(TypePackId tp) override; - TypeId clean(TypeId ty) override; - TypePackId clean(TypePackId tp) override; -}; - struct GenericTypeDefinitions { std::vector genericTypes; diff --git a/Analysis/include/Luau/TypeVar.h b/Analysis/include/Luau/TypeVar.h index 052d4d88..6a13b11c 100644 --- a/Analysis/include/Luau/TypeVar.h +++ b/Analysis/include/Luau/TypeVar.h @@ -223,12 +223,16 @@ struct GenericTypeDefinition { TypeId ty; std::optional defaultValue; + + bool operator==(const GenericTypeDefinition& rhs) const; }; struct GenericTypePackDefinition { TypePackId tp; std::optional defaultValue; + + bool operator==(const GenericTypePackDefinition& rhs) const; }; struct FunctionArgument @@ -426,6 +430,12 @@ struct TypeFun TypeId type; TypeFun() = default; + + explicit TypeFun(TypeId ty) + : type(ty) + { + } + TypeFun(std::vector typeParams, TypeId type) : typeParams(std::move(typeParams)) , type(type) @@ -438,6 +448,27 @@ struct TypeFun , type(type) { } + + bool operator==(const TypeFun& rhs) const; +}; + +/** Represents a pending type alias instantiation. + * + * In order to afford (co)recursive type aliases, we need to reason about a + * partially-complete instantiation. This requires encoding more information in + * a type variable than a BlockedTypeVar affords, hence this. Each + * PendingExpansionTypeVar has a corresponding TypeAliasExpansionConstraint + * enqueued in the solver to convert it to an actual instantiated type + */ +struct PendingExpansionTypeVar +{ + PendingExpansionTypeVar(TypeFun fn, std::vector typeArguments, std::vector packArguments); + TypeFun fn; + std::vector typeArguments; + std::vector packArguments; + size_t index; + + static size_t nextIndex; }; // Anything! All static checking is off. @@ -470,8 +501,10 @@ struct NeverTypeVar using ErrorTypeVar = Unifiable::Error; -using TypeVariant = Unifiable::Variant; +using TypeVariant = + Unifiable::Variant; + struct TypeVar final { diff --git a/Analysis/include/Luau/VisitTypeVar.h b/Analysis/include/Luau/VisitTypeVar.h index 7229dafc..7e5d71d6 100644 --- a/Analysis/include/Luau/VisitTypeVar.h +++ b/Analysis/include/Luau/VisitTypeVar.h @@ -9,7 +9,6 @@ #include "Luau/TypeVar.h" LUAU_FASTINT(LuauVisitRecursionLimit) -LUAU_FASTFLAG(LuauNormalizeFlagIsConservative) LUAU_FASTFLAG(LuauCompleteVisitor); namespace Luau @@ -150,6 +149,10 @@ struct GenericTypeVarVisitor { return visit(ty); } + virtual bool visit(TypeId ty, const PendingExpansionTypeVar& petv) + { + return visit(ty); + } virtual bool visit(TypeId ty, const SingletonTypeVar& stv) { return visit(ty); @@ -285,8 +288,6 @@ struct GenericTypeVarVisitor traverse(partTy); } } - else if (!FFlag::LuauCompleteVisitor) - return visit_detail::unsee(seen, ty); else if (get(ty)) { // Visiting into LazyTypeVar may necessarily cause infinite expansion, so we don't do that on purpose. @@ -301,6 +302,37 @@ struct GenericTypeVarVisitor visit(ty, *utv); else if (auto ntv = get(ty)) visit(ty, *ntv); + else if (auto petv = get(ty)) + { + if (visit(ty, *petv)) + { + traverse(petv->fn.type); + + for (const GenericTypeDefinition& p : petv->fn.typeParams) + { + traverse(p.ty); + + if (p.defaultValue) + traverse(*p.defaultValue); + } + + for (const GenericTypePackDefinition& p : petv->fn.typePackParams) + { + traverse(p.tp); + + if (p.defaultValue) + traverse(*p.defaultValue); + } + + for (TypeId a : petv->typeArguments) + traverse(a); + + for (TypePackId a : petv->packArguments) + traverse(a); + } + } + else if (!FFlag::LuauCompleteVisitor) + return visit_detail::unsee(seen, ty); else LUAU_ASSERT(!"GenericTypeVarVisitor::traverse(TypeId) is not exhaustive!"); @@ -333,7 +365,7 @@ struct GenericTypeVarVisitor else if (auto pack = get(tp)) { bool res = visit(tp, *pack); - if (!FFlag::LuauNormalizeFlagIsConservative || res) + if (res) { for (TypeId ty : pack->head) traverse(ty); @@ -345,7 +377,7 @@ struct GenericTypeVarVisitor else if (auto pack = get(tp)) { bool res = visit(tp, *pack); - if (!FFlag::LuauNormalizeFlagIsConservative || res) + if (res) traverse(pack->ty); } else diff --git a/Analysis/src/ApplyTypeFunction.cpp b/Analysis/src/ApplyTypeFunction.cpp new file mode 100644 index 00000000..c6ac3e19 --- /dev/null +++ b/Analysis/src/ApplyTypeFunction.cpp @@ -0,0 +1,60 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details + +#include "Luau/ApplyTypeFunction.h" + +namespace Luau +{ + +bool ApplyTypeFunction::isDirty(TypeId ty) +{ + if (typeArguments.count(ty)) + return true; + else if (const FreeTypeVar* ftv = get(ty)) + { + if (ftv->forwardedTypeAlias) + encounteredForwardedType = true; + return false; + } + else + return false; +} + +bool ApplyTypeFunction::isDirty(TypePackId tp) +{ + if (typePackArguments.count(tp)) + return true; + else + return false; +} + +bool ApplyTypeFunction::ignoreChildren(TypeId ty) +{ + if (get(ty)) + return true; + else + return false; +} + +bool ApplyTypeFunction::ignoreChildren(TypePackId tp) +{ + if (get(tp)) + return true; + else + return false; +} + +TypeId ApplyTypeFunction::clean(TypeId ty) +{ + TypeId& arg = typeArguments[ty]; + LUAU_ASSERT(arg); + return arg; +} + +TypePackId ApplyTypeFunction::clean(TypePackId tp) +{ + TypePackId& arg = typePackArguments[tp]; + LUAU_ASSERT(arg); + return arg; +} + +} // namespace Luau diff --git a/Analysis/src/JsonEncoder.cpp b/Analysis/src/AstJsonEncoder.cpp similarity index 99% rename from Analysis/src/JsonEncoder.cpp rename to Analysis/src/AstJsonEncoder.cpp index ee7dadb3..2897875d 100644 --- a/Analysis/src/JsonEncoder.cpp +++ b/Analysis/src/AstJsonEncoder.cpp @@ -1,5 +1,5 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details -#include "Luau/JsonEncoder.h" +#include "Luau/AstJsonEncoder.h" #include "Luau/Ast.h" #include "Luau/ParseResult.h" @@ -773,7 +773,7 @@ struct AstJsonEncoder : public AstVisitor PROP(indexer); }); } - + void write(struct AstTableIndexer* indexer) { if (indexer) @@ -1178,7 +1178,6 @@ struct AstJsonEncoder : public AstVisitor write("location", comment.location); popComma(c); writeRaw("}"); - } } }; diff --git a/Analysis/src/AstQuery.cpp b/Analysis/src/AstQuery.cpp index 1124c29e..50299704 100644 --- a/Analysis/src/AstQuery.cpp +++ b/Analysis/src/AstQuery.cpp @@ -314,7 +314,7 @@ std::optional findBindingAtPosition(const Module& module, const SourceM auto iter = currentScope->bindings.find(name); if (iter != currentScope->bindings.end() && iter->second.location.begin <= pos) { - /* Ignore this binding if we're inside its definition. e.g. local abc = abc -- Will take the definition of abc from outer scope */ + // Ignore this binding if we're inside its definition. e.g. local abc = abc -- Will take the definition of abc from outer scope std::optional bindingStatement = findBindingLocalStatement(source, iter->second); if (!bindingStatement || !(*bindingStatement)->location.contains(pos)) return iter->second; diff --git a/Analysis/src/BuiltinDefinitions.cpp b/Analysis/src/BuiltinDefinitions.cpp index aeba2c13..826179b3 100644 --- a/Analysis/src/BuiltinDefinitions.cpp +++ b/Analysis/src/BuiltinDefinitions.cpp @@ -10,6 +10,7 @@ LUAU_FASTFLAGVARIABLE(LuauSetMetaTableArgsCheck, false) LUAU_FASTFLAG(LuauUnknownAndNeverType) +LUAU_FASTFLAGVARIABLE(LuauBuiltInMetatableNoBadSynthetic, false) /** FIXME: Many of these type definitions are not quite completely accurate. * @@ -349,7 +350,7 @@ static std::optional> magicFunctionSetMetaTable( if (tableName == metatableName) mtv.syntheticName = tableName; - else + else if (!FFlag::LuauBuiltInMetatableNoBadSynthetic) mtv.syntheticName = "{ @metatable: " + metatableName + ", " + tableName + " }"; } diff --git a/Analysis/src/Clone.cpp b/Analysis/src/Clone.cpp index 88c50318..51ad61d5 100644 --- a/Analysis/src/Clone.cpp +++ b/Analysis/src/Clone.cpp @@ -48,6 +48,7 @@ struct TypeCloner void operator()(const Unifiable::Bound& t); void operator()(const Unifiable::Error& t); void operator()(const BlockedTypeVar& t); + void operator()(const PendingExpansionTypeVar& t); void operator()(const PrimitiveTypeVar& t); void operator()(const ConstrainedTypeVar& t); void operator()(const SingletonTypeVar& t); @@ -166,6 +167,52 @@ void TypeCloner::operator()(const BlockedTypeVar& t) defaultClone(t); } +void TypeCloner::operator()(const PendingExpansionTypeVar& t) +{ + TypeId res = dest.addType(PendingExpansionTypeVar{t.fn, t.typeArguments, t.packArguments}); + PendingExpansionTypeVar* petv = getMutable(res); + LUAU_ASSERT(petv); + + seenTypes[typeId] = res; + + std::vector typeArguments; + for (TypeId arg : t.typeArguments) + typeArguments.push_back(clone(arg, dest, cloneState)); + + std::vector packArguments; + for (TypePackId arg : t.packArguments) + packArguments.push_back(clone(arg, dest, cloneState)); + + TypeFun fn; + fn.type = clone(t.fn.type, dest, cloneState); + + for (const GenericTypeDefinition& param : t.fn.typeParams) + { + TypeId ty = clone(param.ty, dest, cloneState); + std::optional defaultValue = param.defaultValue; + + if (defaultValue) + defaultValue = clone(*defaultValue, dest, cloneState); + + fn.typeParams.push_back(GenericTypeDefinition{ty, defaultValue}); + } + + for (const GenericTypePackDefinition& param : t.fn.typePackParams) + { + TypePackId tp = clone(param.tp, dest, cloneState); + std::optional defaultValue = param.defaultValue; + + if (defaultValue) + defaultValue = clone(*defaultValue, dest, cloneState); + + fn.typePackParams.push_back(GenericTypePackDefinition{tp, defaultValue}); + } + + petv->fn = std::move(fn); + petv->typeArguments = std::move(typeArguments); + petv->packArguments = std::move(packArguments); +} + void TypeCloner::operator()(const PrimitiveTypeVar& t) { defaultClone(t); @@ -452,6 +499,11 @@ TypeId shallowClone(TypeId ty, TypeArena& dest, const TxnLog* log) ConstrainedTypeVar clone{ctv->level, ctv->parts}; result = dest.addType(std::move(clone)); } + else if (const PendingExpansionTypeVar* petv = get(ty)) + { + PendingExpansionTypeVar clone{petv->fn, petv->typeArguments, petv->packArguments}; + result = dest.addType(std::move(clone)); + } else return result; diff --git a/Analysis/src/ConstraintGraphBuilder.cpp b/Analysis/src/ConstraintGraphBuilder.cpp index efaeff6c..ea7037bf 100644 --- a/Analysis/src/ConstraintGraphBuilder.cpp +++ b/Analysis/src/ConstraintGraphBuilder.cpp @@ -70,11 +70,11 @@ void ConstraintGraphBuilder::visit(AstStatBlock* block) prepopulateGlobalScope(scope, block); // TODO: We should share the global scope. - rootScope->typeBindings["nil"] = singletonTypes.nilType; - rootScope->typeBindings["number"] = singletonTypes.numberType; - rootScope->typeBindings["string"] = singletonTypes.stringType; - rootScope->typeBindings["boolean"] = singletonTypes.booleanType; - rootScope->typeBindings["thread"] = singletonTypes.threadType; + rootScope->typeBindings["nil"] = TypeFun{singletonTypes.nilType}; + rootScope->typeBindings["number"] = TypeFun{singletonTypes.numberType}; + rootScope->typeBindings["string"] = TypeFun{singletonTypes.stringType}; + rootScope->typeBindings["boolean"] = TypeFun{singletonTypes.booleanType}; + rootScope->typeBindings["thread"] = TypeFun{singletonTypes.threadType}; visitBlockWithoutChildScope(scope, block); } @@ -89,6 +89,53 @@ void ConstraintGraphBuilder::visitBlockWithoutChildScope(const ScopePtr& scope, return; } + std::unordered_map aliasDefinitionLocations; + + // In order to enable mutually-recursive type aliases, we need to + // populate the type bindings before we actually check any of the + // alias statements. Since we're not ready to actually resolve + // any of the annotations, we just use a fresh type for now. + for (AstStat* stat : block->body) + { + if (auto alias = stat->as()) + { + if (scope->typeBindings.count(alias->name.value) != 0) + { + auto it = aliasDefinitionLocations.find(alias->name.value); + LUAU_ASSERT(it != aliasDefinitionLocations.end()); + reportError(alias->location, DuplicateTypeDefinition{alias->name.value, it->second}); + continue; + } + + bool hasGenerics = alias->generics.size > 0 || alias->genericPacks.size > 0; + + ScopePtr defnScope = scope; + if (hasGenerics) + { + defnScope = childScope(alias->location, scope); + } + + TypeId initialType = freshType(scope); + TypeFun initialFun = TypeFun{initialType}; + + for (const auto& [name, gen] : createGenerics(defnScope, alias->generics)) + { + initialFun.typeParams.push_back(gen); + defnScope->typeBindings[name] = TypeFun{gen.ty}; + } + + for (const auto& [name, genPack] : createGenericPacks(defnScope, alias->genericPacks)) + { + initialFun.typePackParams.push_back(genPack); + defnScope->typePackBindings[name] = genPack.tp; + } + + scope->typeBindings[alias->name.value] = std::move(initialFun); + astTypeAliasDefiningScopes[alias] = defnScope; + aliasDefinitionLocations[alias->name.value] = alias->location; + } + } + for (AstStat* stat : block->body) visit(scope, stat); } @@ -117,6 +164,12 @@ void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStat* stat) visit(scope, i); else if (auto a = stat->as()) visit(scope, a); + else if (auto s = stat->as()) + visit(scope, s); + else if (auto s = stat->as()) + visit(scope, s); + else if (auto s = stat->as()) + visit(scope, s); else LUAU_ASSERT(0); } @@ -133,7 +186,7 @@ void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatLocal* local) if (local->annotation) { location = local->annotation->location; - TypeId annotation = resolveType(scope, local->annotation); + TypeId annotation = resolveType(scope, local->annotation, /* topLevel */ true); addConstraint(scope, SubtypeConstraint{ty, annotation}); } @@ -171,11 +224,10 @@ void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatLocal* local) void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatFor* for_) { - auto checkNumber = [&](AstExpr* expr) - { + auto checkNumber = [&](AstExpr* expr) { if (!expr) return; - + TypeId t = check(scope, expr); addConstraint(scope, SubtypeConstraint{t, singletonTypes.numberType}); }; @@ -307,19 +359,6 @@ void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatBlock* block) { ScopePtr innerScope = childScope(block->location, scope); - // In order to enable mutually-recursive type aliases, we need to - // populate the type bindings before we actually check any of the - // alias statements. Since we're not ready to actually resolve - // any of the annotations, we just use a fresh type for now. - for (AstStat* stat : block->body) - { - if (auto alias = stat->as()) - { - TypeId initialType = freshType(scope); - scope->typeBindings[alias->name.value] = initialType; - } - } - visitBlockWithoutChildScope(innerScope, block); } @@ -348,29 +387,48 @@ void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatIf* ifStatement void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatTypeAlias* alias) { // TODO: Exported type aliases - // TODO: Generic type aliases - auto it = scope->typeBindings.find(alias->name.value); - // This should always be here since we do a separate pass over the - // AST to set up typeBindings. If it's not, we've somehow skipped - // this alias in that first pass. - LUAU_ASSERT(it != scope->typeBindings.end()); - if (it == scope->typeBindings.end()) + auto bindingIt = scope->typeBindings.find(alias->name.value); + ScopePtr* defnIt = astTypeAliasDefiningScopes.find(alias); + // These will be undefined if the alias was a duplicate definition, in which + // case we just skip over it. + if (bindingIt == scope->typeBindings.end() || defnIt == nullptr) { - ice->ice("Type alias does not have a pre-populated binding", alias->location); + return; } - TypeId ty = resolveType(scope, alias->type); + ScopePtr resolvingScope = *defnIt; + TypeId ty = resolveType(resolvingScope, alias->type, /* topLevel */ true); + + LUAU_ASSERT(get(bindingIt->second.type)); // Rather than using a subtype constraint, we instead directly bind // the free type we generated in the first pass to the resolved type. // This prevents a case where you could cause another constraint to // bind the free alias type to an unrelated type, causing havoc. - asMutable(it->second)->ty.emplace(ty); + asMutable(bindingIt->second.type)->ty.emplace(ty); addConstraint(scope, NameConstraint{ty, alias->name.value}); } +void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatDeclareGlobal* global) +{ + LUAU_ASSERT(global->type); + + TypeId globalTy = resolveType(scope, global->type); + scope->bindings[global->name] = Binding{globalTy, global->location}; +} + +void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatDeclareClass* global) +{ + LUAU_ASSERT(false); // TODO: implement +} + +void ConstraintGraphBuilder::visit(const ScopePtr& scope, AstStatDeclareFunction* global) +{ + LUAU_ASSERT(false); // TODO: implement +} + TypePackId ConstraintGraphBuilder::checkPack(const ScopePtr& scope, AstArray exprs) { if (exprs.size == 0) @@ -707,7 +765,7 @@ ConstraintGraphBuilder::FunctionSignature ConstraintGraphBuilder::checkFunctionS for (const auto& [name, g] : genericDefinitions) { genericTypes.push_back(g.ty); - signatureScope->typeBindings[name] = g.ty; + signatureScope->typeBindings[name] = TypeFun{g.ty}; } for (const auto& [name, g] : genericPackDefinitions) @@ -745,7 +803,7 @@ ConstraintGraphBuilder::FunctionSignature ConstraintGraphBuilder::checkFunctionS if (local->annotation) { - TypeId argAnnotation = resolveType(signatureScope, local->annotation); + TypeId argAnnotation = resolveType(signatureScope, local->annotation, /* topLevel */ true); addConstraint(signatureScope, SubtypeConstraint{t, argAnnotation}); } } @@ -784,20 +842,65 @@ void ConstraintGraphBuilder::checkFunctionBody(const ScopePtr& scope, AstExprFun } } -TypeId ConstraintGraphBuilder::resolveType(const ScopePtr& scope, AstType* ty) +TypeId ConstraintGraphBuilder::resolveType(const ScopePtr& scope, AstType* ty, bool topLevel) { TypeId result = nullptr; if (auto ref = ty->as()) { // TODO: Support imported types w/ require tracing. - // TODO: Support generic type references. LUAU_ASSERT(!ref->prefix); - LUAU_ASSERT(!ref->hasParameterList); - // TODO: If it doesn't exist, should we introduce a free binding? - // This is probably important for handling type aliases. - result = scope->lookupTypeBinding(ref->name.value).value_or(singletonTypes.errorRecoveryType()); + std::optional alias = scope->lookupTypeBinding(ref->name.value); + + if (alias.has_value()) + { + // If the alias is not generic, we don't need to set up a blocked + // type and an instantiation constraint. + if (alias->typeParams.empty() && alias->typePackParams.empty()) + { + result = alias->type; + } + else + { + std::vector parameters; + std::vector packParameters; + + for (const AstTypeOrPack& p : ref->parameters) + { + // We do not enforce the ordering of types vs. type packs here; + // that is done in the parser. + if (p.type) + { + parameters.push_back(resolveType(scope, p.type)); + } + else if (p.typePack) + { + packParameters.push_back(resolveTypePack(scope, p.typePack)); + } + else + { + // This indicates a parser bug: one of these two pointers + // should be set. + LUAU_ASSERT(false); + } + } + + result = arena->addType(PendingExpansionTypeVar{*alias, parameters, packParameters}); + + if (topLevel) + { + addConstraint(scope, TypeAliasExpansionConstraint{ + /* target */ result, + }); + } + } + } + else + { + reportError(ty->location, UnknownSymbol{ref->name.value, UnknownSymbol::Context::Type}); + result = singletonTypes.errorRecoveryType(); + } } else if (auto tab = ty->as()) { @@ -846,7 +949,7 @@ TypeId ConstraintGraphBuilder::resolveType(const ScopePtr& scope, AstType* ty) for (const auto& [name, g] : genericDefinitions) { genericTypes.push_back(g.ty); - signatureScope->typeBindings[name] = g.ty; + signatureScope->typeBindings[name] = TypeFun{g.ty}; } for (const auto& [name, g] : genericPackDefinitions) @@ -956,7 +1059,15 @@ TypePackId ConstraintGraphBuilder::resolveTypePack(const ScopePtr& scope, AstTyp } else if (auto gen = tp->as()) { - result = arena->addTypePack(TypePackVar{GenericTypePack{scope.get(), gen->genericName.value}}); + if (std::optional lookup = scope->lookupTypePackBinding(gen->genericName.value)) + { + result = *lookup; + } + else + { + reportError(tp->location, UnknownSymbol{gen->genericName.value, UnknownSymbol::Context::Type}); + result = singletonTypes.errorRecoveryTypePack(); + } } else { diff --git a/Analysis/src/ConstraintSolver.cpp b/Analysis/src/ConstraintSolver.cpp index 1cd29918..0898f9aa 100644 --- a/Analysis/src/ConstraintSolver.cpp +++ b/Analysis/src/ConstraintSolver.cpp @@ -1,11 +1,13 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +#include "Luau/ApplyTypeFunction.h" #include "Luau/ConstraintSolver.h" #include "Luau/Instantiation.h" #include "Luau/Location.h" #include "Luau/Quantify.h" #include "Luau/ToString.h" #include "Luau/Unifier.h" +#include "Luau/VisitTypeVar.h" LUAU_FASTFLAGVARIABLE(DebugLuauLogSolver, false); LUAU_FASTFLAGVARIABLE(DebugLuauLogSolverToJson, false); @@ -37,6 +39,170 @@ static void dumpConstraints(NotNull scope, ToStringOptions& opts) dumpConstraints(child, opts); } +static std::pair, std::vector> saturateArguments( + const TypeFun& fn, const std::vector& rawTypeArguments, const std::vector& rawPackArguments, TypeArena* arena) +{ + std::vector saturatedTypeArguments; + std::vector extraTypes; + std::vector saturatedPackArguments; + + for (size_t i = 0; i < rawTypeArguments.size(); ++i) + { + TypeId ty = rawTypeArguments[i]; + + if (i < fn.typeParams.size()) + saturatedTypeArguments.push_back(ty); + else + extraTypes.push_back(ty); + } + + // If we collected extra types, put them in a type pack now. This case is + // mutually exclusive with the type pack -> type conversion we do below: + // extraTypes will only have elements in it if we have more types than we + // have parameter slots for them to go into. + if (!extraTypes.empty()) + { + saturatedPackArguments.push_back(arena->addTypePack(extraTypes)); + } + + for (size_t i = 0; i < rawPackArguments.size(); ++i) + { + TypePackId tp = rawPackArguments[i]; + + // If we are short on regular type saturatedTypeArguments and we have a single + // element type pack, we can decompose that to the type it contains and + // use that as a type parameter. + if (saturatedTypeArguments.size() < fn.typeParams.size() && size(tp) == 1 && finite(tp) && first(tp) && saturatedPackArguments.empty()) + { + saturatedTypeArguments.push_back(*first(tp)); + } + else + { + saturatedPackArguments.push_back(tp); + } + } + + size_t typesProvided = saturatedTypeArguments.size(); + size_t typesRequired = fn.typeParams.size(); + + size_t packsProvided = saturatedPackArguments.size(); + size_t packsRequired = fn.typePackParams.size(); + + // Extra types should be accumulated in extraTypes, not saturatedTypeArguments. Extra + // packs will be accumulated in saturatedPackArguments, so we don't have an + // assertion for that. + LUAU_ASSERT(typesProvided <= typesRequired); + + // If we didn't provide enough types, but we did provide a type pack, we + // don't want to use defaults. The rationale for this is that if the user + // provides a pack but doesn't provide enough types, we want to report an + // error, rather than simply using the default saturatedTypeArguments, if they exist. If + // they did provide enough types, but not enough packs, we of course want to + // use the default packs. + bool needsDefaults = (typesProvided < typesRequired && packsProvided == 0) || (typesProvided == typesRequired && packsProvided < packsRequired); + + if (needsDefaults) + { + // Default types can reference earlier types. It's legal to write + // something like + // type T = (A, B) -> number + // and we need to respect that. We use an ApplyTypeFunction for this. + ApplyTypeFunction atf{arena}; + + for (size_t i = 0; i < typesProvided; ++i) + atf.typeArguments[fn.typeParams[i].ty] = saturatedTypeArguments[i]; + + for (size_t i = typesProvided; i < typesRequired; ++i) + { + TypeId defaultTy = fn.typeParams[i].defaultValue.value_or(nullptr); + + // We will fill this in with the error type later. + if (!defaultTy) + break; + + TypeId instantiatedDefault = atf.substitute(defaultTy).value_or(getSingletonTypes().errorRecoveryType()); + atf.typeArguments[fn.typeParams[i].ty] = instantiatedDefault; + saturatedTypeArguments.push_back(instantiatedDefault); + } + + for (size_t i = 0; i < packsProvided; ++i) + { + atf.typePackArguments[fn.typePackParams[i].tp] = saturatedPackArguments[i]; + } + + for (size_t i = packsProvided; i < packsRequired; ++i) + { + TypePackId defaultTp = fn.typePackParams[i].defaultValue.value_or(nullptr); + + // We will fill this in with the error type pack later. + if (!defaultTp) + break; + + TypePackId instantiatedDefault = atf.substitute(defaultTp).value_or(getSingletonTypes().errorRecoveryTypePack()); + atf.typePackArguments[fn.typePackParams[i].tp] = instantiatedDefault; + saturatedPackArguments.push_back(instantiatedDefault); + } + } + + // If we didn't create an extra type pack from overflowing parameter packs, + // and we're still missing a type pack, plug in an empty type pack as the + // value of the empty packs. + if (extraTypes.empty() && saturatedPackArguments.size() + 1 == fn.typePackParams.size()) + { + saturatedPackArguments.push_back(arena->addTypePack({})); + } + + // We need to have _something_ when we substitute the generic saturatedTypeArguments, + // even if they're missing, so we use the error type as a filler. + for (size_t i = saturatedTypeArguments.size(); i < typesRequired; ++i) + { + saturatedTypeArguments.push_back(getSingletonTypes().errorRecoveryType()); + } + + for (size_t i = saturatedPackArguments.size(); i < packsRequired; ++i) + { + saturatedPackArguments.push_back(getSingletonTypes().errorRecoveryTypePack()); + } + + // At this point, these two conditions should be true. If they aren't we + // will run into access violations. + LUAU_ASSERT(saturatedTypeArguments.size() == fn.typeParams.size()); + LUAU_ASSERT(saturatedPackArguments.size() == fn.typePackParams.size()); + + return {saturatedTypeArguments, saturatedPackArguments}; +} + +bool InstantiationSignature::operator==(const InstantiationSignature& rhs) const +{ + return fn == rhs.fn && arguments == rhs.arguments && packArguments == rhs.packArguments; +} + +size_t HashInstantiationSignature::operator()(const InstantiationSignature& signature) const +{ + size_t hash = std::hash{}(signature.fn.type); + for (const GenericTypeDefinition& p : signature.fn.typeParams) + { + hash ^= (std::hash{}(p.ty) << 1); + } + + for (const GenericTypePackDefinition& p : signature.fn.typePackParams) + { + hash ^= (std::hash{}(p.tp) << 1); + } + + for (const TypeId a : signature.arguments) + { + hash ^= (std::hash{}(a) << 1); + } + + for (const TypePackId a : signature.packArguments) + { + hash ^= (std::hash{}(a) << 1); + } + + return hash; +} + void dump(NotNull rootScope, ToStringOptions& opts) { printf("constraints:\n"); @@ -77,6 +243,7 @@ void ConstraintSolver::run() return; ToStringOptions opts; + opts.exhaustive = true; if (FFlag::DebugLuauLogSolver) { @@ -186,6 +353,8 @@ bool ConstraintSolver::tryDispatch(NotNull constraint, bool fo success = tryDispatch(*bc, constraint, force); else if (auto nc = get(*constraint)) success = tryDispatch(*nc, constraint); + else if (auto taec = get(*constraint)) + success = tryDispatch(*taec, constraint); else LUAU_ASSERT(0); @@ -325,6 +494,198 @@ bool ConstraintSolver::tryDispatch(const NameConstraint& c, NotNullarena); + + if (follow(petv.fn.type) == follow(signature.fn.type) && (signature.arguments != typeArguments || signature.packArguments != packArguments)) + { + foundInfiniteType = true; + return false; + } + + return true; + } +}; + +struct InstantiationQueuer : TypeVarOnceVisitor +{ + ConstraintSolver* solver; + const InstantiationSignature& signature; + + explicit InstantiationQueuer(ConstraintSolver* solver, const InstantiationSignature& signature) + : solver(solver) + , signature(signature) + { + } + + bool visit(TypeId ty, const PendingExpansionTypeVar& petv) override + { + solver->pushConstraint(TypeAliasExpansionConstraint{ty}); + return false; + } +}; + +bool ConstraintSolver::tryDispatch(const TypeAliasExpansionConstraint& c, NotNull constraint) +{ + const PendingExpansionTypeVar* petv = get(follow(c.target)); + if (!petv) + { + unblock(c.target); + return true; + } + + auto bindResult = [this, &c](TypeId result) { + asMutable(c.target)->ty.emplace(result); + unblock(c.target); + }; + + // If there are no parameters to the type function we can just use the type + // directly. + if (petv->fn.typeParams.empty() && petv->fn.typePackParams.empty()) + { + bindResult(petv->fn.type); + return true; + } + + auto [typeArguments, packArguments] = saturateArguments(petv->fn, petv->typeArguments, petv->packArguments, arena); + + bool sameTypes = + std::equal(typeArguments.begin(), typeArguments.end(), petv->fn.typeParams.begin(), petv->fn.typeParams.end(), [](auto&& itp, auto&& p) { + return itp == p.ty; + }); + + bool samePacks = std::equal( + packArguments.begin(), packArguments.end(), petv->fn.typePackParams.begin(), petv->fn.typePackParams.end(), [](auto&& itp, auto&& p) { + return itp == p.tp; + }); + + // If we're instantiating the type with its generic saturatedTypeArguments we are + // performing the identity substitution. We can just short-circuit and bind + // to the TypeFun's type. + if (sameTypes && samePacks) + { + bindResult(petv->fn.type); + return true; + } + + InstantiationSignature signature{ + petv->fn, + typeArguments, + packArguments, + }; + + // If we use the same signature, we don't need to bother trying to + // instantiate the alias again, since the instantiation should be + // deterministic. + if (TypeId* cached = instantiatedAliases.find(signature)) + { + bindResult(*cached); + return true; + } + + // In order to prevent infinite types from being expanded and causing us to + // cycle infinitely, we need to scan the type function for cases where we + // expand the same alias with different type saturatedTypeArguments. See + // https://github.com/Roblox/luau/pull/68 for the RFC responsible for this. + // This is a little nicer than using a recursion limit because we can catch + // the infinite expansion before actually trying to expand it. + InfiniteTypeFinder itf{this, signature}; + itf.traverse(petv->fn.type); + + if (itf.foundInfiniteType) + { + // TODO (CLI-56761): Report an error. + bindResult(getSingletonTypes().errorRecoveryType()); + return true; + } + + ApplyTypeFunction applyTypeFunction{arena}; + for (size_t i = 0; i < typeArguments.size(); ++i) + { + applyTypeFunction.typeArguments[petv->fn.typeParams[i].ty] = typeArguments[i]; + } + + for (size_t i = 0; i < packArguments.size(); ++i) + { + applyTypeFunction.typePackArguments[petv->fn.typePackParams[i].tp] = packArguments[i]; + } + + std::optional maybeInstantiated = applyTypeFunction.substitute(petv->fn.type); + // Note that ApplyTypeFunction::encounteredForwardedType is never set in + // DCR, because we do not use free types for forward-declared generic + // aliases. + + if (!maybeInstantiated.has_value()) + { + // TODO (CLI-56761): Report an error. + bindResult(getSingletonTypes().errorRecoveryType()); + return true; + } + + TypeId instantiated = *maybeInstantiated; + TypeId target = follow(instantiated); + // Type function application will happily give us the exact same type if + // there are e.g. generic saturatedTypeArguments that go unused. + bool needsClone = follow(petv->fn.type) == target; + // Only tables have the properties we're trying to set. + TableTypeVar* ttv = getMutableTableType(target); + + if (ttv) + { + if (needsClone) + { + // Substitution::clone is a shallow clone. If this is a + // metatable type, we want to mutate its table, so we need to + // explicitly clone that table as well. If we don't, we will + // mutate another module's type surface and cause a + // use-after-free. + if (get(target)) + { + instantiated = applyTypeFunction.clone(target); + MetatableTypeVar* mtv = getMutable(instantiated); + mtv->table = applyTypeFunction.clone(mtv->table); + ttv = getMutable(mtv->table); + } + else if (get(target)) + { + instantiated = applyTypeFunction.clone(target); + ttv = getMutable(instantiated); + } + + target = follow(instantiated); + } + + ttv->instantiatedTypeParams = typeArguments; + ttv->instantiatedTypePackParams = packArguments; + // TODO: Fill in definitionModuleName. + } + + bindResult(target); + + // The application is not recursive, so we need to queue up application of + // any child type function instantiations within the result in order for it + // to be complete. + InstantiationQueuer queuer{this, signature}; + queuer.traverse(target); + + instantiatedAliases[signature] = target; + + return true; +} + void ConstraintSolver::block_(BlockedConstraintId target, NotNull constraint) { blocked[target].push_back(constraint); @@ -388,7 +749,7 @@ void ConstraintSolver::unblock(TypePackId progressed) bool ConstraintSolver::isBlocked(TypeId ty) { - return nullptr != get(follow(ty)); + return nullptr != get(follow(ty)) || nullptr != get(follow(ty)); } bool ConstraintSolver::isBlocked(NotNull constraint) @@ -415,4 +776,12 @@ void ConstraintSolver::unify(TypePackId subPack, TypePackId superPack) u.log.commit(); } +void ConstraintSolver::pushConstraint(ConstraintV cv) +{ + std::unique_ptr c = std::make_unique(std::move(cv)); + NotNull borrow = NotNull(c.get()); + solverConstraints.push_back(std::move(c)); + unsolvedConstraints.push_back(borrow); +} + } // namespace Luau diff --git a/Analysis/src/ConstraintSolverLogger.cpp b/Analysis/src/ConstraintSolverLogger.cpp index 0c2517c0..adb9c54e 100644 --- a/Analysis/src/ConstraintSolverLogger.cpp +++ b/Analysis/src/ConstraintSolverLogger.cpp @@ -2,45 +2,39 @@ #include "Luau/ConstraintSolverLogger.h" +#include "Luau/JsonEmitter.h" + namespace Luau { -static std::string dumpScopeAndChildren(const Scope* scope, ToStringOptions& opts) +static void dumpScopeAndChildren(const Scope* scope, Json::JsonEmitter& emitter, ToStringOptions& opts) { - std::string output = "{\"bindings\":{"; + emitter.writeRaw("{"); + Json::write(emitter, "bindings"); + emitter.writeRaw(":"); + + Json::ObjectEmitter o = emitter.writeObject(); - bool comma = false; for (const auto& [name, binding] : scope->bindings) { - if (comma) - output += ","; - - output += "\""; - output += name.c_str(); - output += "\": \""; - ToStringResult result = toStringDetailed(binding.typeId, opts); opts.nameMap = std::move(result.nameMap); - output += result.name; - output += "\""; - - comma = true; + o.writePair(name.c_str(), result.name); } - output += "},\"children\":["; - comma = false; + o.finish(); + emitter.writeRaw(","); + Json::write(emitter, "children"); + emitter.writeRaw(":"); + Json::ArrayEmitter a = emitter.writeArray(); for (const Scope* child : scope->children) { - if (comma) - output += ","; - - output += dumpScopeAndChildren(child, opts); - comma = true; + dumpScopeAndChildren(child, emitter, opts); } - output += "]}"; - return output; + a.finish(); + emitter.writeRaw("}"); } static std::string dumpConstraintsToDot(std::vector>& constraints, ToStringOptions& opts) @@ -80,51 +74,49 @@ static std::string dumpConstraintsToDot(std::vector>& std::string ConstraintSolverLogger::compileOutput() { - std::string output = "["; - bool comma = false; - + Json::JsonEmitter emitter; + emitter.writeRaw("["); for (const std::string& snapshot : snapshots) { - if (comma) - output += ","; - output += snapshot; - - comma = true; + emitter.writeComma(); + emitter.writeRaw(snapshot); } - output += "]"; - return output; + emitter.writeRaw("]"); + return emitter.str(); } void ConstraintSolverLogger::captureBoundarySnapshot(const Scope* rootScope, std::vector>& unsolvedConstraints) { - std::string snapshot = "{\"type\":\"boundary\",\"rootScope\":"; + Json::JsonEmitter emitter; + Json::ObjectEmitter o = emitter.writeObject(); + o.writePair("type", "boundary"); + o.writePair("constraintGraph", dumpConstraintsToDot(unsolvedConstraints, opts)); + emitter.writeComma(); + Json::write(emitter, "rootScope"); + emitter.writeRaw(":"); + dumpScopeAndChildren(rootScope, emitter, opts); + o.finish(); - snapshot += dumpScopeAndChildren(rootScope, opts); - snapshot += ",\"constraintGraph\":\""; - snapshot += dumpConstraintsToDot(unsolvedConstraints, opts); - snapshot += "\"}"; - - snapshots.push_back(std::move(snapshot)); + snapshots.push_back(emitter.str()); } void ConstraintSolverLogger::prepareStepSnapshot( const Scope* rootScope, NotNull current, std::vector>& unsolvedConstraints) { - // LUAU_ASSERT(!preparedSnapshot); + Json::JsonEmitter emitter; + Json::ObjectEmitter o = emitter.writeObject(); + o.writePair("type", "step"); + o.writePair("constraintGraph", dumpConstraintsToDot(unsolvedConstraints, opts)); + o.writePair("currentId", std::to_string(reinterpret_cast(current.get()))); + o.writePair("current", toString(*current, opts)); + emitter.writeComma(); + Json::write(emitter, "rootScope"); + emitter.writeRaw(":"); + dumpScopeAndChildren(rootScope, emitter, opts); + o.finish(); - std::string snapshot = "{\"type\":\"step\",\"rootScope\":"; - - snapshot += dumpScopeAndChildren(rootScope, opts); - snapshot += ",\"constraintGraph\":\""; - snapshot += dumpConstraintsToDot(unsolvedConstraints, opts); - snapshot += "\",\"currentId\":\""; - snapshot += std::to_string(reinterpret_cast(current.get())); - snapshot += "\",\"current\":\""; - snapshot += toString(*current, opts); - snapshot += "\"}"; - - preparedSnapshot = std::move(snapshot); + preparedSnapshot = emitter.str(); } void ConstraintSolverLogger::commitPreparedStepSnapshot() diff --git a/Analysis/src/Frontend.cpp b/Analysis/src/Frontend.cpp index 222a17df..fe65853d 100644 --- a/Analysis/src/Frontend.cpp +++ b/Analysis/src/Frontend.cpp @@ -77,6 +77,58 @@ static void generateDocumentationSymbols(TypeId ty, const std::string& rootName) } } +LoadDefinitionFileResult Frontend::loadDefinitionFile(std::string_view source, const std::string& packageName) +{ + if (!FFlag::DebugLuauDeferredConstraintResolution) + return Luau::loadDefinitionFile(typeChecker, typeChecker.globalScope, source, packageName); + + LUAU_TIMETRACE_SCOPE("loadDefinitionFile", "Frontend"); + + Luau::Allocator allocator; + Luau::AstNameTable names(allocator); + + ParseOptions options; + options.allowDeclarationSyntax = true; + + Luau::ParseResult parseResult = Luau::Parser::parse(source.data(), source.size(), names, allocator, options); + + if (parseResult.errors.size() > 0) + return LoadDefinitionFileResult{false, parseResult, nullptr}; + + Luau::SourceModule module; + module.root = parseResult.root; + module.mode = Mode::Definition; + + ModulePtr checkedModule = check(module, Mode::Definition, globalScope); + + if (checkedModule->errors.size() > 0) + return LoadDefinitionFileResult{false, parseResult, checkedModule}; + + CloneState cloneState; + + for (const auto& [name, ty] : checkedModule->declaredGlobals) + { + TypeId globalTy = clone(ty, globalTypes, cloneState); + std::string documentationSymbol = packageName + "/global/" + name; + generateDocumentationSymbols(globalTy, documentationSymbol); + globalScope->bindings[typeChecker.globalNames.names->getOrAdd(name.c_str())] = {globalTy, Location(), false, {}, documentationSymbol}; + + persist(globalTy); + } + + for (const auto& [name, ty] : checkedModule->getModuleScope()->exportedTypeBindings) + { + TypeFun globalTy = clone(ty, globalTypes, cloneState); + std::string documentationSymbol = packageName + "/globaltype/" + name; + generateDocumentationSymbols(globalTy.type, documentationSymbol); + globalScope->exportedTypeBindings[name] = globalTy; + + persist(globalTy.type); + } + + return LoadDefinitionFileResult{true, parseResult, checkedModule}; +} + LoadDefinitionFileResult loadDefinitionFile(TypeChecker& typeChecker, ScopePtr targetScope, std::string_view source, const std::string& packageName) { LUAU_TIMETRACE_SCOPE("loadDefinitionFile", "Frontend"); @@ -770,14 +822,7 @@ NotNull Frontend::getGlobalScope() { if (!globalScope) { - const SingletonTypes& singletonTypes = getSingletonTypes(); - - globalScope = std::make_unique(singletonTypes.anyTypePack); - globalScope->typeBindings["nil"] = singletonTypes.nilType; - globalScope->typeBindings["number"] = singletonTypes.numberType; - globalScope->typeBindings["string"] = singletonTypes.stringType; - globalScope->typeBindings["boolean"] = singletonTypes.booleanType; - globalScope->typeBindings["thread"] = singletonTypes.threadType; + globalScope = typeChecker.globalScope; } return NotNull(globalScope.get()); diff --git a/Analysis/src/Instantiation.cpp b/Analysis/src/Instantiation.cpp index 77c62422..1a6013af 100644 --- a/Analysis/src/Instantiation.cpp +++ b/Analysis/src/Instantiation.cpp @@ -4,6 +4,8 @@ #include "Luau/TxnLog.h" #include "Luau/TypeArena.h" +LUAU_FASTFLAG(LuauClassTypeVarsInSubstitution) + namespace Luau { @@ -31,6 +33,8 @@ bool Instantiation::ignoreChildren(TypeId ty) { if (log->getMutable(ty)) return true; + else if (FFlag::LuauClassTypeVarsInSubstitution && get(ty)) + return true; else return false; } diff --git a/Analysis/src/JsonEmitter.cpp b/Analysis/src/JsonEmitter.cpp new file mode 100644 index 00000000..e99619ba --- /dev/null +++ b/Analysis/src/JsonEmitter.cpp @@ -0,0 +1,220 @@ +// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details +#include "Luau/JsonEmitter.h" + +#include "Luau/StringUtils.h" + +#include + +namespace Luau::Json +{ + +static constexpr int CHUNK_SIZE = 1024; + +ObjectEmitter::ObjectEmitter(NotNull emitter) + : emitter(emitter), finished(false) +{ + comma = emitter->pushComma(); + emitter->writeRaw('{'); +} + +ObjectEmitter::~ObjectEmitter() +{ + finish(); +} + +void ObjectEmitter::finish() +{ + if (finished) + return; + + emitter->writeRaw('}'); + emitter->popComma(comma); + finished = true; +} + +ArrayEmitter::ArrayEmitter(NotNull emitter) + : emitter(emitter), finished(false) +{ + comma = emitter->pushComma(); + emitter->writeRaw('['); +} + +ArrayEmitter::~ArrayEmitter() +{ + finish(); +} + +void ArrayEmitter::finish() +{ + if (finished) + return; + + emitter->writeRaw(']'); + emitter->popComma(comma); + finished = true; +} + +JsonEmitter::JsonEmitter() +{ + newChunk(); +} + +std::string JsonEmitter::str() +{ + return join(chunks, ""); +} + +bool JsonEmitter::pushComma() +{ + bool current = comma; + comma = false; + return current; +} + +void JsonEmitter::popComma(bool c) +{ + comma = c; +} + +void JsonEmitter::writeRaw(std::string_view sv) +{ + if (sv.size() > CHUNK_SIZE) + { + chunks.emplace_back(sv); + newChunk(); + return; + } + + auto& chunk = chunks.back(); + if (chunk.size() + sv.size() < CHUNK_SIZE) + { + chunk.append(sv.data(), sv.size()); + return; + } + + size_t prefix = CHUNK_SIZE - chunk.size(); + chunk.append(sv.data(), prefix); + newChunk(); + + chunks.back().append(sv.data() + prefix, sv.size() - prefix); +} + +void JsonEmitter::writeRaw(char c) +{ + writeRaw(std::string_view{&c, 1}); +} + +void write(JsonEmitter& emitter, bool b) +{ + if (b) + emitter.writeRaw("true"); + else + emitter.writeRaw("false"); +} + +void write(JsonEmitter& emitter, double d) +{ + emitter.writeRaw(std::to_string(d)); +} + +void write(JsonEmitter& emitter, int i) +{ + emitter.writeRaw(std::to_string(i)); +} + +void write(JsonEmitter& emitter, long i) +{ + emitter.writeRaw(std::to_string(i)); +} + +void write(JsonEmitter& emitter, long long i) +{ + emitter.writeRaw(std::to_string(i)); +} + +void write(JsonEmitter& emitter, unsigned int i) +{ + emitter.writeRaw(std::to_string(i)); +} + +void write(JsonEmitter& emitter, unsigned long i) +{ + emitter.writeRaw(std::to_string(i)); +} + +void write(JsonEmitter& emitter, unsigned long long i) +{ + emitter.writeRaw(std::to_string(i)); +} + +void write(JsonEmitter& emitter, std::string_view sv) +{ + emitter.writeRaw('\"'); + + for (char c : sv) + { + if (c == '"') + emitter.writeRaw("\\\""); + else if (c == '\\') + emitter.writeRaw("\\\\"); + else if (c == '\n') + emitter.writeRaw("\\n"); + else if (c < ' ') + emitter.writeRaw(format("\\u%04x", c)); + else + emitter.writeRaw(c); + } + + emitter.writeRaw('\"'); +} + +void write(JsonEmitter& emitter, char c) +{ + write(emitter, std::string_view{&c, 1}); +} + +void write(JsonEmitter& emitter, const char* str) +{ + write(emitter, std::string_view{str, strlen(str)}); +} + +void write(JsonEmitter& emitter, const std::string& str) +{ + write(emitter, std::string_view{str}); +} + +void write(JsonEmitter& emitter, std::nullptr_t) +{ + emitter.writeRaw("null"); +} + +void write(JsonEmitter& emitter, std::nullopt_t) +{ + emitter.writeRaw("null"); +} + +void JsonEmitter::writeComma() +{ + if (comma) + writeRaw(','); + else + comma = true; +} + +ObjectEmitter JsonEmitter::writeObject() +{ + return ObjectEmitter{NotNull(this)}; +} + +ArrayEmitter JsonEmitter::writeArray() +{ + return ArrayEmitter{NotNull(this)}; +} + +void JsonEmitter::newChunk() +{ + chunks.emplace_back(); + chunks.back().reserve(CHUNK_SIZE); +} + +} // namespace Luau::Json diff --git a/Analysis/src/Linter.cpp b/Analysis/src/Linter.cpp index fb952f5e..9fce79a2 100644 --- a/Analysis/src/Linter.cpp +++ b/Analysis/src/Linter.cpp @@ -48,6 +48,7 @@ static const char* kWarningNames[] = { "DuplicateCondition", "MisleadingAndOr", "CommentDirective", + "IntegerParsing", }; // clang-format on @@ -1433,7 +1434,7 @@ private: const char* checkStringFormat(const char* data, size_t size) { const char* flags = "-+ #0"; - const char* options = "cdiouxXeEfgGqs"; + const char* options = "cdiouxXeEfgGqs*"; for (size_t i = 0; i < size; ++i) { @@ -2589,6 +2590,45 @@ private: } }; +class LintIntegerParsing : AstVisitor +{ +public: + LUAU_NOINLINE static void process(LintContext& context) + { + LintIntegerParsing pass; + pass.context = &context; + + context.root->visit(&pass); + } + +private: + LintContext* context; + + bool visit(AstExprConstantNumber* node) override + { + switch (node->parseResult) + { + case ConstantNumberParseResult::Ok: + case ConstantNumberParseResult::Malformed: + break; + case ConstantNumberParseResult::BinOverflow: + emitWarning(*context, LintWarning::Code_IntegerParsing, node->location, + "Binary number literal exceeded available precision and has been truncated to 2^64"); + break; + case ConstantNumberParseResult::HexOverflow: + emitWarning(*context, LintWarning::Code_IntegerParsing, node->location, + "Hexadecimal number literal exceeded available precision and has been truncated to 2^64"); + break; + case ConstantNumberParseResult::DoublePrefix: + emitWarning(*context, LintWarning::Code_IntegerParsing, node->location, + "Hexadecimal number literal has a double prefix, which will fail to parse in the future; remove the extra 0x to fix"); + break; + } + + return true; + } +}; + static void fillBuiltinGlobals(LintContext& context, const AstNameTable& names, const ScopePtr& env) { ScopePtr current = env; @@ -2810,6 +2850,9 @@ std::vector lint(AstStat* root, const AstNameTable& names, const Sc if (context.warningEnabled(LintWarning::Code_CommentDirective)) lintComments(context, hotcomments); + if (context.warningEnabled(LintWarning::Code_IntegerParsing)) + LintIntegerParsing::process(context); + std::sort(context.result.begin(), context.result.end(), WarningComparator()); return context.result; diff --git a/Analysis/src/Module.cpp b/Analysis/src/Module.cpp index dca18027..2b46da87 100644 --- a/Analysis/src/Module.cpp +++ b/Analysis/src/Module.cpp @@ -15,7 +15,6 @@ #include LUAU_FASTFLAG(LuauLowerBoundsCalculation); -LUAU_FASTFLAG(LuauNormalizeFlagIsConservative); LUAU_FASTFLAG(DebugLuauDeferredConstraintResolution); LUAU_FASTFLAGVARIABLE(LuauForceExportSurfacesToBeNormal, false); @@ -140,20 +139,17 @@ void Module::clonePublicInterface(InternalErrorReporter& ice) { normalize(tf.type, interfaceTypes, ice); - if (FFlag::LuauNormalizeFlagIsConservative) + // We're about to freeze the memory. We know that the flag is conservative by design. Cyclic tables + // won't be marked normal. If the types aren't normal by now, they never will be. + forceNormal.traverse(tf.type); + for (GenericTypeDefinition param : tf.typeParams) { - // We're about to freeze the memory. We know that the flag is conservative by design. Cyclic tables - // won't be marked normal. If the types aren't normal by now, they never will be. - forceNormal.traverse(tf.type); - for (GenericTypeDefinition param : tf.typeParams) - { - forceNormal.traverse(param.ty); + forceNormal.traverse(param.ty); - if (param.defaultValue) - { - normalize(*param.defaultValue, interfaceTypes, ice); - forceNormal.traverse(*param.defaultValue); - } + if (param.defaultValue) + { + normalize(*param.defaultValue, interfaceTypes, ice); + forceNormal.traverse(*param.defaultValue); } } } diff --git a/Analysis/src/Normalize.cpp b/Analysis/src/Normalize.cpp index a96b557d..9ae3b404 100644 --- a/Analysis/src/Normalize.cpp +++ b/Analysis/src/Normalize.cpp @@ -13,7 +13,6 @@ LUAU_FASTFLAGVARIABLE(DebugLuauCopyBeforeNormalizing, false) // This could theoretically be 2000 on amd64, but x86 requires this. LUAU_FASTINTVARIABLE(LuauNormalizeIterationLimit, 1200); LUAU_FASTFLAGVARIABLE(LuauNormalizeCombineTableFix, false); -LUAU_FASTFLAGVARIABLE(LuauNormalizeFlagIsConservative, false); LUAU_FASTFLAGVARIABLE(LuauFixNormalizationOfCyclicUnions, false); LUAU_FASTFLAG(LuauUnknownAndNeverType) LUAU_FASTFLAG(LuauQuantifyConstrained) @@ -89,13 +88,7 @@ static bool areNormal_(const T& t, const std::unordered_set& seen, Intern if (count >= FInt::LuauNormalizeIterationLimit) ice.ice("Luau::areNormal hit iteration limit"); - if (FFlag::LuauNormalizeFlagIsConservative) - return ty->normal; - else - { - // The follow is here because a bound type may not be normal, but the bound type is normal. - return ty->normal || follow(ty)->normal || seen.find(asMutable(ty)) != seen.end(); - } + return ty->normal; }; return std::all_of(begin(t), end(t), isNormal); diff --git a/Analysis/src/Quantify.cpp b/Analysis/src/Quantify.cpp index 72eb4012..03049cca 100644 --- a/Analysis/src/Quantify.cpp +++ b/Analysis/src/Quantify.cpp @@ -7,7 +7,6 @@ #include "Luau/TxnLog.h" #include "Luau/VisitTypeVar.h" -LUAU_FASTFLAG(LuauAlwaysQuantify); LUAU_FASTFLAG(DebugLuauSharedSelf) LUAU_FASTFLAG(DebugLuauDeferredConstraintResolution); LUAU_FASTFLAGVARIABLE(LuauQuantifyConstrained, false) @@ -203,16 +202,8 @@ void quantify(TypeId ty, TypeLevel level) FunctionTypeVar* ftv = getMutable(ty); LUAU_ASSERT(ftv); - if (FFlag::LuauAlwaysQuantify) - { - ftv->generics.insert(ftv->generics.end(), q.generics.begin(), q.generics.end()); - ftv->genericPacks.insert(ftv->genericPacks.end(), q.genericPacks.begin(), q.genericPacks.end()); - } - else - { - ftv->generics = q.generics; - ftv->genericPacks = q.genericPacks; - } + ftv->generics.insert(ftv->generics.end(), q.generics.begin(), q.generics.end()); + ftv->genericPacks.insert(ftv->genericPacks.end(), q.genericPacks.begin(), q.genericPacks.end()); } } @@ -223,16 +214,8 @@ void quantify(TypeId ty, Scope* scope) FunctionTypeVar* ftv = getMutable(ty); LUAU_ASSERT(ftv); - if (FFlag::LuauAlwaysQuantify) - { - ftv->generics.insert(ftv->generics.end(), q.generics.begin(), q.generics.end()); - ftv->genericPacks.insert(ftv->genericPacks.end(), q.genericPacks.begin(), q.genericPacks.end()); - } - else - { - ftv->generics = q.generics; - ftv->genericPacks = q.genericPacks; - } + ftv->generics.insert(ftv->generics.end(), q.generics.begin(), q.generics.end()); + ftv->genericPacks.insert(ftv->genericPacks.end(), q.genericPacks.begin(), q.genericPacks.end()); if (ftv->generics.empty() && ftv->genericPacks.empty() && !q.seenMutableType && !q.seenGenericType) ftv->hasNoGenerics = true; diff --git a/Analysis/src/Scope.cpp b/Analysis/src/Scope.cpp index 083aa085..bee16908 100644 --- a/Analysis/src/Scope.cpp +++ b/Analysis/src/Scope.cpp @@ -122,7 +122,7 @@ std::optional Scope::lookup(Symbol sym) } } -std::optional Scope::lookupTypeBinding(const Name& name) +std::optional Scope::lookupTypeBinding(const Name& name) { Scope* s = this; while (s) diff --git a/Analysis/src/Substitution.cpp b/Analysis/src/Substitution.cpp index 7245403c..148c9ee2 100644 --- a/Analysis/src/Substitution.cpp +++ b/Analysis/src/Substitution.cpp @@ -9,9 +9,12 @@ #include LUAU_FASTFLAGVARIABLE(LuauAnyificationMustClone, false) +LUAU_FASTFLAGVARIABLE(LuauSubstitutionFixMissingFields, false) LUAU_FASTFLAG(LuauLowerBoundsCalculation) LUAU_FASTINTVARIABLE(LuauTarjanChildLimit, 10000) +LUAU_FASTFLAGVARIABLE(LuauClassTypeVarsInSubstitution, false) LUAU_FASTFLAG(LuauUnknownAndNeverType) +LUAU_FASTFLAGVARIABLE(LuauSubstitutionReentrant, false) namespace Luau { @@ -28,6 +31,14 @@ void Tarjan::visitChildren(TypeId ty, int index) if (const FunctionTypeVar* ftv = get(ty)) { + if (FFlag::LuauSubstitutionFixMissingFields) + { + for (TypeId generic : ftv->generics) + visitChild(generic); + for (TypePackId genericPack : ftv->genericPacks) + visitChild(genericPack); + } + visitChild(ftv->argTypes); visitChild(ftv->retTypes); } @@ -68,6 +79,25 @@ void Tarjan::visitChildren(TypeId ty, int index) for (TypeId part : ctv->parts) visitChild(part); } + else if (const PendingExpansionTypeVar* petv = get(ty)) + { + for (TypeId a : petv->typeArguments) + visitChild(a); + + for (TypePackId a : petv->packArguments) + visitChild(a); + } + else if (const ClassTypeVar* ctv = get(ty); FFlag::LuauClassTypeVarsInSubstitution && ctv) + { + for (auto [name, prop] : ctv->props) + visitChild(prop.type); + + if (ctv->parent) + visitChild(*ctv->parent); + + if (ctv->metatable) + visitChild(*ctv->metatable); + } } void Tarjan::visitChildren(TypePackId tp, int index) @@ -267,6 +297,24 @@ TarjanResult Tarjan::visitRoot(TypePackId tp) return loop(); } +void FindDirty::clearTarjan() +{ + dirty.clear(); + + typeToIndex.clear(); + packToIndex.clear(); + indexToType.clear(); + indexToPack.clear(); + + stack.clear(); + onStack.clear(); + lowlink.clear(); + + edgesTy.clear(); + edgesTp.clear(); + worklist.clear(); +} + bool FindDirty::getDirty(int index) { if (dirty.size() <= size_t(index)) @@ -330,16 +378,46 @@ std::optional Substitution::substitute(TypeId ty) { ty = log->follow(ty); + // clear algorithm state for reentrancy + if (FFlag::LuauSubstitutionReentrant) + clearTarjan(); + auto result = findDirty(ty); if (result != TarjanResult::Ok) return std::nullopt; for (auto [oldTy, newTy] : newTypes) - if (!ignoreChildren(oldTy)) - replaceChildren(newTy); + { + if (FFlag::LuauSubstitutionReentrant) + { + if (!ignoreChildren(oldTy) && !replacedTypes.contains(newTy)) + { + replaceChildren(newTy); + replacedTypes.insert(newTy); + } + } + else + { + if (!ignoreChildren(oldTy)) + replaceChildren(newTy); + } + } for (auto [oldTp, newTp] : newPacks) - if (!ignoreChildren(oldTp)) - replaceChildren(newTp); + { + if (FFlag::LuauSubstitutionReentrant) + { + if (!ignoreChildren(oldTp) && !replacedTypePacks.contains(newTp)) + { + replaceChildren(newTp); + replacedTypePacks.insert(newTp); + } + } + else + { + if (!ignoreChildren(oldTp)) + replaceChildren(newTp); + } + } TypeId newTy = replace(ty); return newTy; } @@ -348,16 +426,46 @@ std::optional Substitution::substitute(TypePackId tp) { tp = log->follow(tp); + // clear algorithm state for reentrancy + if (FFlag::LuauSubstitutionReentrant) + clearTarjan(); + auto result = findDirty(tp); if (result != TarjanResult::Ok) return std::nullopt; for (auto [oldTy, newTy] : newTypes) - if (!ignoreChildren(oldTy)) - replaceChildren(newTy); + { + if (FFlag::LuauSubstitutionReentrant) + { + if (!ignoreChildren(oldTy) && !replacedTypes.contains(newTy)) + { + replaceChildren(newTy); + replacedTypes.insert(newTy); + } + } + else + { + if (!ignoreChildren(oldTy)) + replaceChildren(newTy); + } + } for (auto [oldTp, newTp] : newPacks) - if (!ignoreChildren(oldTp)) - replaceChildren(newTp); + { + if (FFlag::LuauSubstitutionReentrant) + { + if (!ignoreChildren(oldTp) && !replacedTypePacks.contains(newTp)) + { + replaceChildren(newTp); + replacedTypePacks.insert(newTp); + } + } + else + { + if (!ignoreChildren(oldTp)) + replaceChildren(newTp); + } + } TypePackId newTp = replace(tp); return newTp; } @@ -385,6 +493,8 @@ TypePackId Substitution::clone(TypePackId tp) { VariadicTypePack clone; clone.ty = vtp->ty; + if (FFlag::LuauSubstitutionFixMissingFields) + clone.hidden = vtp->hidden; return addTypePack(std::move(clone)); } else @@ -395,6 +505,9 @@ void Substitution::foundDirty(TypeId ty) { ty = log->follow(ty); + if (FFlag::LuauSubstitutionReentrant && newTypes.contains(ty)) + return; + if (isDirty(ty)) newTypes[ty] = follow(clean(ty)); else @@ -405,6 +518,9 @@ void Substitution::foundDirty(TypePackId tp) { tp = log->follow(tp); + if (FFlag::LuauSubstitutionReentrant && newPacks.contains(tp)) + return; + if (isDirty(tp)) newPacks[tp] = follow(clean(tp)); else @@ -446,6 +562,14 @@ void Substitution::replaceChildren(TypeId ty) if (FunctionTypeVar* ftv = getMutable(ty)) { + if (FFlag::LuauSubstitutionFixMissingFields) + { + for (TypeId& generic : ftv->generics) + generic = replace(generic); + for (TypePackId& genericPack : ftv->genericPacks) + genericPack = replace(genericPack); + } + ftv->argTypes = replace(ftv->argTypes); ftv->retTypes = replace(ftv->retTypes); } @@ -486,6 +610,25 @@ void Substitution::replaceChildren(TypeId ty) for (TypeId& part : ctv->parts) part = replace(part); } + else if (PendingExpansionTypeVar* petv = getMutable(ty)) + { + for (TypeId& a : petv->typeArguments) + a = replace(a); + + for (TypePackId& a : petv->packArguments) + a = replace(a); + } + else if (ClassTypeVar* ctv = getMutable(ty); FFlag::LuauClassTypeVarsInSubstitution && ctv) + { + for (auto& [name, prop] : ctv->props) + prop.type = replace(prop.type); + + if (ctv->parent) + ctv->parent = replace(*ctv->parent); + + if (ctv->metatable) + ctv->metatable = replace(*ctv->metatable); + } } void Substitution::replaceChildren(TypePackId tp) diff --git a/Analysis/src/ToString.cpp b/Analysis/src/ToString.cpp index 9e637a9b..e31e690a 100644 --- a/Analysis/src/ToString.cpp +++ b/Analysis/src/ToString.cpp @@ -232,6 +232,11 @@ struct StringifierState emit(std::to_string(i).c_str()); } + void emit(size_t i) + { + emit(std::to_string(i).c_str()); + } + void indent() { indentation += 4; @@ -409,6 +414,13 @@ struct TypeVarStringifier state.emit("*"); } + void operator()(TypeId ty, const PendingExpansionTypeVar& petv) + { + state.emit("*pending-expansion-"); + state.emit(petv.index); + state.emit("*"); + } + void operator()(TypeId, const PrimitiveTypeVar& ptv) { switch (ptv.type) @@ -1449,6 +1461,12 @@ std::string toString(const Constraint& constraint, ToStringOptions& opts) opts.nameMap = std::move(namedStr.nameMap); return "@name(" + namedStr.name + ") = " + c.name; } + else if constexpr (std::is_same_v) + { + ToStringResult targetStr = toStringDetailed(c.target, opts); + opts.nameMap = std::move(targetStr.nameMap); + return "expand " + targetStr.name; + } else static_assert(always_false_v, "Non-exhaustive constraint switch"); }; diff --git a/Analysis/src/TypeAttach.cpp b/Analysis/src/TypeAttach.cpp index 2bc89cf6..f21a4fa9 100644 --- a/Analysis/src/TypeAttach.cpp +++ b/Analysis/src/TypeAttach.cpp @@ -99,6 +99,11 @@ public: return allocator->alloc(Location(), std::nullopt, AstName("*blocked*")); } + AstType* operator()(const PendingExpansionTypeVar& petv) + { + return allocator->alloc(Location(), std::nullopt, AstName("*pending-expansion*")); + } + AstType* operator()(const ConstrainedTypeVar& ctv) { AstArray types; diff --git a/Analysis/src/TypeChecker2.cpp b/Analysis/src/TypeChecker2.cpp index 85749671..53b069cf 100644 --- a/Analysis/src/TypeChecker2.cpp +++ b/Analysis/src/TypeChecker2.cpp @@ -67,6 +67,13 @@ struct TypeChecker2 : public AstVisitor return follow(*ty); } + TypePackId lookupPackAnnotation(AstTypePack* annotation) + { + TypePackId* tp = module->astResolvedTypePacks.find(annotation); + LUAU_ASSERT(tp); + return follow(*tp); + } + TypePackId reconstructPack(AstArray exprs, TypeArena& arena) { if (exprs.size == 0) @@ -363,12 +370,153 @@ struct TypeChecker2 : public AstVisitor bool visit(AstTypeReference* ty) override { Scope* scope = findInnermostScope(ty->location); + LUAU_ASSERT(scope); // TODO: Imported types - // TODO: Generic types - if (!scope->lookupTypeBinding(ty->name.value)) + + std::optional alias = scope->lookupTypeBinding(ty->name.value); + + if (alias.has_value()) { - reportError(UnknownSymbol{ty->name.value, UnknownSymbol::Context::Type}, ty->location); + size_t typesRequired = alias->typeParams.size(); + size_t packsRequired = alias->typePackParams.size(); + + bool hasDefaultTypes = std::any_of(alias->typeParams.begin(), alias->typeParams.end(), [](auto&& el) { + return el.defaultValue.has_value(); + }); + + bool hasDefaultPacks = std::any_of(alias->typePackParams.begin(), alias->typePackParams.end(), [](auto&& el) { + return el.defaultValue.has_value(); + }); + + if (!ty->hasParameterList) + { + if ((!alias->typeParams.empty() && !hasDefaultTypes) || (!alias->typePackParams.empty() && !hasDefaultPacks)) + { + reportError(GenericError{"Type parameter list is required"}, ty->location); + } + } + + size_t typesProvided = 0; + size_t extraTypes = 0; + size_t packsProvided = 0; + + for (const AstTypeOrPack& p : ty->parameters) + { + if (p.type) + { + if (packsProvided != 0) + { + reportError(GenericError{"Type parameters must come before type pack parameters"}, ty->location); + } + + if (typesProvided < typesRequired) + { + typesProvided += 1; + } + else + { + extraTypes += 1; + } + } + else if (p.typePack) + { + TypePackId tp = lookupPackAnnotation(p.typePack); + + if (typesProvided < typesRequired && size(tp) == 1 && finite(tp) && first(tp)) + { + typesProvided += 1; + } + else + { + packsProvided += 1; + } + } + } + + if (extraTypes != 0 && packsProvided == 0) + { + packsProvided += 1; + } + + for (size_t i = typesProvided; i < typesRequired; ++i) + { + if (alias->typeParams[i].defaultValue) + { + typesProvided += 1; + } + } + + for (size_t i = packsProvided; i < packsProvided; ++i) + { + if (alias->typePackParams[i].defaultValue) + { + packsProvided += 1; + } + } + + if (extraTypes == 0 && packsProvided + 1 == packsRequired) + { + packsProvided += 1; + } + + if (typesProvided != typesRequired || packsProvided != packsRequired) + { + reportError(IncorrectGenericParameterCount{ + /* name */ ty->name.value, + /* typeFun */ *alias, + /* actualParameters */ typesProvided, + /* actualPackParameters */ packsProvided, + }, + ty->location); + } + } + else + { + if (scope->lookupTypePackBinding(ty->name.value)) + { + reportError( + SwappedGenericTypeParameter{ + ty->name.value, + SwappedGenericTypeParameter::Kind::Type, + }, + ty->location); + } + else + { + reportError(UnknownSymbol{ty->name.value, UnknownSymbol::Context::Type}, ty->location); + } + } + + return true; + } + + bool visit(AstTypePack*) override + { + return true; + } + + bool visit(AstTypePackGeneric* tp) override + { + Scope* scope = findInnermostScope(tp->location); + LUAU_ASSERT(scope); + + std::optional alias = scope->lookupTypePackBinding(tp->genericName.value); + if (!alias.has_value()) + { + if (scope->lookupTypeBinding(tp->genericName.value)) + { + reportError( + SwappedGenericTypeParameter{ + tp->genericName.value, + SwappedGenericTypeParameter::Kind::Pack, + }, + tp->location); + } + else + { + reportError(UnknownSymbol{tp->genericName.value, UnknownSymbol::Context::Type}, tp->location); + } } return true; diff --git a/Analysis/src/TypeInfer.cpp b/Analysis/src/TypeInfer.cpp index 371ef77a..bdda195c 100644 --- a/Analysis/src/TypeInfer.cpp +++ b/Analysis/src/TypeInfer.cpp @@ -1,6 +1,7 @@ // This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details #include "Luau/TypeInfer.h" +#include "Luau/ApplyTypeFunction.h" #include "Luau/Clone.h" #include "Luau/Common.h" #include "Luau/Instantiation.h" @@ -36,11 +37,8 @@ LUAU_FASTFLAGVARIABLE(LuauIndexSilenceErrors, false) LUAU_FASTFLAGVARIABLE(LuauLowerBoundsCalculation, false) LUAU_FASTFLAGVARIABLE(DebugLuauFreezeDuringUnification, false) LUAU_FASTFLAGVARIABLE(LuauSelfCallAutocompleteFix3, false) -LUAU_FASTFLAGVARIABLE(LuauReduceUnionRecursion, false) LUAU_FASTFLAGVARIABLE(LuauReturnAnyInsteadOfICE, false) // Eventually removed as false. -LUAU_FASTFLAG(LuauNormalizeFlagIsConservative) LUAU_FASTFLAGVARIABLE(DebugLuauSharedSelf, false); -LUAU_FASTFLAGVARIABLE(LuauAlwaysQuantify, false); LUAU_FASTFLAGVARIABLE(LuauReportErrorsOnIndexerKeyMismatch, false) LUAU_FASTFLAGVARIABLE(LuauUnknownAndNeverType, false) LUAU_FASTFLAG(LuauQuantifyConstrained) @@ -2108,35 +2106,16 @@ std::vector TypeChecker::reduceUnion(const std::vector& types) if (const UnionTypeVar* utv = get(t)) { - if (FFlag::LuauReduceUnionRecursion) + for (TypeId ty : utv) { - for (TypeId ty : utv) - { - if (FFlag::LuauNormalizeFlagIsConservative) - ty = follow(ty); - if (get(ty)) - continue; - if (get(ty) || get(ty)) - return {ty}; + ty = follow(ty); + if (get(ty)) + continue; + if (get(ty) || get(ty)) + return {ty}; - if (result.end() == std::find(result.begin(), result.end(), ty)) - result.push_back(ty); - } - } - else - { - std::vector r = reduceUnion(utv->options); - for (TypeId ty : r) - { - ty = follow(ty); - if (get(ty)) - continue; - if (get(ty) || get(ty)) - return {ty}; - - if (std::find(result.begin(), result.end(), ty) == result.end()) - result.push_back(ty); - } + if (result.end() == std::find(result.begin(), result.end(), ty)) + result.push_back(ty); } } else if (std::find(result.begin(), result.end(), t) == result.end()) @@ -4770,16 +4749,8 @@ TypeId TypeChecker::quantify(const ScopePtr& scope, TypeId ty, Location location { const FunctionTypeVar* ftv = get(ty); - if (FFlag::LuauAlwaysQuantify) - { - if (ftv) - Luau::quantify(ty, scope->level); - } - else - { - if (ftv && ftv->generics.empty() && ftv->genericPacks.empty()) - Luau::quantify(ty, scope->level); - } + if (ftv) + Luau::quantify(ty, scope->level); if (FFlag::LuauLowerBoundsCalculation && ftv) { @@ -5253,7 +5224,7 @@ TypeId TypeChecker::resolveTypeWorker(const ScopePtr& scope, const AstType& anno if (notEnoughParameters && hasDefaultParameters) { // 'applyTypeFunction' is used to substitute default types that reference previous generic types - ApplyTypeFunction applyTypeFunction{¤tModule->internalTypes, scope->level}; + ApplyTypeFunction applyTypeFunction{¤tModule->internalTypes}; for (size_t i = 0; i < typesProvided; ++i) applyTypeFunction.typeArguments[tf->typeParams[i].ty] = typeParams[i]; @@ -5494,65 +5465,13 @@ TypePackId TypeChecker::resolveTypePack(const ScopePtr& scope, const AstTypePack return result; } -bool ApplyTypeFunction::isDirty(TypeId ty) -{ - if (typeArguments.count(ty)) - return true; - else if (const FreeTypeVar* ftv = get(ty)) - { - if (ftv->forwardedTypeAlias) - encounteredForwardedType = true; - return false; - } - else - return false; -} - -bool ApplyTypeFunction::isDirty(TypePackId tp) -{ - if (typePackArguments.count(tp)) - return true; - else - return false; -} - -bool ApplyTypeFunction::ignoreChildren(TypeId ty) -{ - if (get(ty)) - return true; - else - return false; -} - -bool ApplyTypeFunction::ignoreChildren(TypePackId tp) -{ - if (get(tp)) - return true; - else - return false; -} - -TypeId ApplyTypeFunction::clean(TypeId ty) -{ - TypeId& arg = typeArguments[ty]; - LUAU_ASSERT(arg); - return arg; -} - -TypePackId ApplyTypeFunction::clean(TypePackId tp) -{ - TypePackId& arg = typePackArguments[tp]; - LUAU_ASSERT(arg); - return arg; -} - TypeId TypeChecker::instantiateTypeFun(const ScopePtr& scope, const TypeFun& tf, const std::vector& typeParams, const std::vector& typePackParams, const Location& location) { if (tf.typeParams.empty() && tf.typePackParams.empty()) return tf.type; - ApplyTypeFunction applyTypeFunction{¤tModule->internalTypes, scope->level}; + ApplyTypeFunction applyTypeFunction{¤tModule->internalTypes}; for (size_t i = 0; i < tf.typeParams.size(); ++i) applyTypeFunction.typeArguments[tf.typeParams[i].ty] = typeParams[i]; diff --git a/Analysis/src/TypeVar.cpp b/Analysis/src/TypeVar.cpp index ebaf5906..ada2b012 100644 --- a/Analysis/src/TypeVar.cpp +++ b/Analysis/src/TypeVar.cpp @@ -445,6 +445,16 @@ BlockedTypeVar::BlockedTypeVar() int BlockedTypeVar::nextIndex = 0; +PendingExpansionTypeVar::PendingExpansionTypeVar(TypeFun fn, std::vector typeArguments, std::vector packArguments) + : fn(fn) + , typeArguments(typeArguments) + , packArguments(packArguments) + , index(++nextIndex) +{ +} + +size_t PendingExpansionTypeVar::nextIndex = 0; + FunctionTypeVar::FunctionTypeVar(TypePackId argTypes, TypePackId retTypes, std::optional defn, bool hasSelf) : argTypes(argTypes) , retTypes(retTypes) @@ -1058,7 +1068,7 @@ ConstrainedTypeVarIterator end(const ConstrainedTypeVar* ctv) static std::vector parseFormatString(TypeChecker& typechecker, const char* data, size_t size) { - const char* options = "cdiouxXeEfgGqs"; + const char* options = "cdiouxXeEfgGqs*"; std::vector result; @@ -1072,7 +1082,7 @@ static std::vector parseFormatString(TypeChecker& typechecker, const cha continue; // we just ignore all characters (including flags/precision) up until first alphabetic character - while (i < size && !(data[i] > 0 && isalpha(data[i]))) + while (i < size && !(data[i] > 0 && (isalpha(data[i]) || data[i] == '*'))) i++; if (i == size) @@ -1080,6 +1090,8 @@ static std::vector parseFormatString(TypeChecker& typechecker, const cha if (data[i] == 'q' || data[i] == 's') result.push_back(typechecker.stringType); + else if (data[i] == '*') + result.push_back(typechecker.unknownType); else if (strchr(options, data[i])) result.push_back(typechecker.numberType); else @@ -1410,4 +1422,19 @@ bool hasTag(const Property& prop, const std::string& tagName) return hasTag(prop.tags, tagName); } +bool TypeFun::operator==(const TypeFun& rhs) const +{ + return type == rhs.type && typeParams == rhs.typeParams && typePackParams == rhs.typePackParams; +} + +bool GenericTypeDefinition::operator==(const GenericTypeDefinition& rhs) const +{ + return ty == rhs.ty && defaultValue == rhs.defaultValue; +} + +bool GenericTypePackDefinition::operator==(const GenericTypePackDefinition& rhs) const +{ + return tp == rhs.tp && defaultValue == rhs.defaultValue; +} + } // namespace Luau diff --git a/Ast/include/Luau/Ast.h b/Ast/include/Luau/Ast.h index 6f39e3fd..1e164d04 100644 --- a/Ast/include/Luau/Ast.h +++ b/Ast/include/Luau/Ast.h @@ -474,16 +474,26 @@ public: bool value; }; +enum class ConstantNumberParseResult +{ + Ok, + Malformed, + BinOverflow, + HexOverflow, + DoublePrefix, +}; + class AstExprConstantNumber : public AstExpr { public: LUAU_RTTI(AstExprConstantNumber) - AstExprConstantNumber(const Location& location, double value); + AstExprConstantNumber(const Location& location, double value, ConstantNumberParseResult parseResult = ConstantNumberParseResult::Ok); void visit(AstVisitor* visitor) override; double value; + ConstantNumberParseResult parseResult; }; class AstExprConstantString : public AstExpr diff --git a/Ast/src/Ast.cpp b/Ast/src/Ast.cpp index 24a280da..3066b756 100644 --- a/Ast/src/Ast.cpp +++ b/Ast/src/Ast.cpp @@ -50,9 +50,10 @@ void AstExprConstantBool::visit(AstVisitor* visitor) visitor->visit(this); } -AstExprConstantNumber::AstExprConstantNumber(const Location& location, double value) +AstExprConstantNumber::AstExprConstantNumber(const Location& location, double value, ConstantNumberParseResult parseResult) : AstExpr(ClassIndex(), location) , value(value) + , parseResult(parseResult) { } diff --git a/Ast/src/Parser.cpp b/Ast/src/Parser.cpp index 8d54e367..1eb9565f 100644 --- a/Ast/src/Parser.cpp +++ b/Ast/src/Parser.cpp @@ -21,7 +21,8 @@ LUAU_DYNAMIC_FASTFLAGVARIABLE(LuaReportParseWrongNamedType, false) bool lua_telemetry_parsed_named_non_function_type = false; -LUAU_FASTFLAGVARIABLE(LuauErrorParseIntegerIssues, false) +LUAU_FASTFLAGVARIABLE(LuauErrorDoubleHexPrefix, false) +LUAU_FASTFLAGVARIABLE(LuauLintParseIntegerIssues, false) LUAU_DYNAMIC_FASTFLAGVARIABLE(LuaReportParseIntegerIssues, false) bool lua_telemetry_parsed_out_of_range_bin_integer = false; @@ -2032,8 +2033,10 @@ AstExpr* Parser::parseAssertionExpr() return expr; } -static const char* parseInteger(double& result, const char* data, int base) +static const char* parseInteger_DEPRECATED(double& result, const char* data, int base) { + LUAU_ASSERT(!FFlag::LuauLintParseIntegerIssues); + char* end = nullptr; unsigned long long value = strtoull(data, &end, base); @@ -2053,9 +2056,6 @@ static const char* parseInteger(double& result, const char* data, int base) else lua_telemetry_parsed_out_of_range_hex_integer = true; } - - if (FFlag::LuauErrorParseIntegerIssues) - return "Integer number value is out of range"; } } @@ -2063,11 +2063,13 @@ static const char* parseInteger(double& result, const char* data, int base) return *end == 0 ? nullptr : "Malformed number"; } -static const char* parseNumber(double& result, const char* data) +static const char* parseNumber_DEPRECATED2(double& result, const char* data) { + LUAU_ASSERT(!FFlag::LuauLintParseIntegerIssues); + // binary literal if (data[0] == '0' && (data[1] == 'b' || data[1] == 'B') && data[2]) - return parseInteger(result, data + 2, 2); + return parseInteger_DEPRECATED(result, data + 2, 2); // hexadecimal literal if (data[0] == '0' && (data[1] == 'x' || data[1] == 'X') && data[2]) @@ -2075,10 +2077,7 @@ static const char* parseNumber(double& result, const char* data) if (DFFlag::LuaReportParseIntegerIssues && data[2] == '0' && (data[3] == 'x' || data[3] == 'X')) lua_telemetry_parsed_double_prefix_hex_integer = true; - if (FFlag::LuauErrorParseIntegerIssues) - return parseInteger(result, data, 16); // keep prefix, it's handled by 'strtoull' - else - return parseInteger(result, data + 2, 16); + return parseInteger_DEPRECATED(result, data + 2, 16); } char* end = nullptr; @@ -2090,6 +2089,8 @@ static const char* parseNumber(double& result, const char* data) static bool parseNumber_DEPRECATED(double& result, const char* data) { + LUAU_ASSERT(!FFlag::LuauLintParseIntegerIssues); + // binary literal if (data[0] == '0' && (data[1] == 'b' || data[1] == 'B') && data[2]) { @@ -2118,6 +2119,73 @@ static bool parseNumber_DEPRECATED(double& result, const char* data) } } +static ConstantNumberParseResult parseInteger(double& result, const char* data, int base) +{ + LUAU_ASSERT(FFlag::LuauLintParseIntegerIssues); + LUAU_ASSERT(base == 2 || base == 16); + + char* end = nullptr; + unsigned long long value = strtoull(data, &end, base); + + if (*end != 0) + return ConstantNumberParseResult::Malformed; + + result = double(value); + + if (value == ULLONG_MAX && errno == ERANGE) + { + // 'errno' might have been set before we called 'strtoull', but we don't want the overhead of resetting a TLS variable on each call + // so we only reset it when we get a result that might be an out-of-range error and parse again to make sure + errno = 0; + value = strtoull(data, &end, base); + + if (errno == ERANGE) + { + if (DFFlag::LuaReportParseIntegerIssues) + { + if (base == 2) + lua_telemetry_parsed_out_of_range_bin_integer = true; + else + lua_telemetry_parsed_out_of_range_hex_integer = true; + } + + return base == 2 ? ConstantNumberParseResult::BinOverflow : ConstantNumberParseResult::HexOverflow; + } + } + + return ConstantNumberParseResult::Ok; +} + +static ConstantNumberParseResult parseNumber(double& result, const char* data) +{ + LUAU_ASSERT(FFlag::LuauLintParseIntegerIssues); + + // binary literal + if (data[0] == '0' && (data[1] == 'b' || data[1] == 'B') && data[2]) + return parseInteger(result, data + 2, 2); + + // hexadecimal literal + if (data[0] == '0' && (data[1] == 'x' || data[1] == 'X') && data[2]) + { + if (!FFlag::LuauErrorDoubleHexPrefix && data[2] == '0' && (data[3] == 'x' || data[3] == 'X')) + { + if (DFFlag::LuaReportParseIntegerIssues) + lua_telemetry_parsed_double_prefix_hex_integer = true; + + ConstantNumberParseResult parseResult = parseInteger(result, data + 2, 16); + return parseResult == ConstantNumberParseResult::Malformed ? parseResult : ConstantNumberParseResult::DoublePrefix; + } + + return parseInteger(result, data, 16); // pass in '0x' prefix, it's handled by 'strtoull' + } + + char* end = nullptr; + double value = strtod(data, &end); + + result = value; + return *end == 0 ? ConstantNumberParseResult::Ok : ConstantNumberParseResult::Malformed; +} + // simpleexp -> NUMBER | STRING | NIL | true | false | ... | constructor | FUNCTION body | primaryexp AstExpr* Parser::parseSimpleExpr() { @@ -2158,10 +2226,21 @@ AstExpr* Parser::parseSimpleExpr() scratchData.erase(std::remove(scratchData.begin(), scratchData.end(), '_'), scratchData.end()); } - if (DFFlag::LuaReportParseIntegerIssues || FFlag::LuauErrorParseIntegerIssues) + if (FFlag::LuauLintParseIntegerIssues) { double value = 0; - if (const char* error = parseNumber(value, scratchData.c_str())) + ConstantNumberParseResult result = parseNumber(value, scratchData.c_str()); + nextLexeme(); + + if (result == ConstantNumberParseResult::Malformed) + return reportExprError(start, {}, "Malformed number"); + + return allocator.alloc(start, value, result); + } + else if (DFFlag::LuaReportParseIntegerIssues) + { + double value = 0; + if (const char* error = parseNumber_DEPRECATED2(value, scratchData.c_str())) { nextLexeme(); diff --git a/CLI/Ast.cpp b/CLI/Ast.cpp index 6ee608c4..fd99d225 100644 --- a/CLI/Ast.cpp +++ b/CLI/Ast.cpp @@ -3,7 +3,7 @@ #include "Luau/Common.h" #include "Luau/Ast.h" -#include "Luau/JsonEncoder.h" +#include "Luau/AstJsonEncoder.h" #include "Luau/Parser.h" #include "Luau/ParseOptions.h" diff --git a/CLI/Repl.cpp b/CLI/Repl.cpp index 41360731..4d3beec9 100644 --- a/CLI/Repl.cpp +++ b/CLI/Repl.cpp @@ -20,6 +20,9 @@ #ifdef _WIN32 #include #include + +#define WIN32_LEAN_AND_MEAN +#include #endif #ifdef CALLGRIND @@ -27,6 +30,7 @@ #endif #include +#include LUAU_FASTFLAG(DebugLuauTimeTracing) @@ -47,6 +51,35 @@ enum class CompileFormat constexpr int MaxTraversalLimit = 50; +// Ctrl-C handling +static void sigintCallback(lua_State* L, int gc) +{ + if (gc >= 0) + return; + + lua_callbacks(L)->interrupt = NULL; + + lua_rawcheckstack(L, 1); // reserve space for error string + luaL_error(L, "Execution interrupted"); +} + +static lua_State* replState = NULL; + +#ifdef _WIN32 +BOOL WINAPI sigintHandler(DWORD signal) +{ + if (signal == CTRL_C_EVENT && replState) + lua_callbacks(replState)->interrupt = &sigintCallback; + return TRUE; +} +#else +static void sigintHandler(int signum) +{ + if (signum == SIGINT && replState) + lua_callbacks(replState)->interrupt = &sigintCallback; +} +#endif + struct GlobalOptions { int optimizationLevel = 1; @@ -76,8 +109,8 @@ static int lua_loadstring(lua_State* L) return 1; lua_pushnil(L); - lua_insert(L, -2); /* put before error message */ - return 2; /* return nil plus error message */ + lua_insert(L, -2); // put before error message + return 2; // return nil plus error message } static int finishrequire(lua_State* L) @@ -535,6 +568,15 @@ static void runRepl() lua_State* L = globalState.get(); setupState(L); + + // setup Ctrl+C handling + replState = L; +#ifdef _WIN32 + SetConsoleCtrlHandler(sigintHandler, TRUE); +#else + signal(SIGINT, sigintHandler); +#endif + luaL_sandboxthread(L); runReplImpl(L); } diff --git a/Common/include/Luau/Bytecode.h b/Common/include/Luau/Bytecode.h index 0cb7e1d9..1d6b18e5 100644 --- a/Common/include/Luau/Bytecode.h +++ b/Common/include/Luau/Bytecode.h @@ -37,6 +37,14 @@ // Note that Luau runtime doesn't provide indefinite bytecode compatibility: support for older versions gets removed over time. As such, bytecode isn't a durable storage format and it's expected // that Luau users can recompile bytecode from source on Luau version upgrades if necessary. +// # Bytecode version history +// +// Note: due to limitations of the versioning scheme, some bytecode blobs that carry version 2 are using features from version 3. Starting from version 3, version should be sufficient to indicate bytecode compatibility. +// +// Version 1: Baseline version for the open-source release. Supported until 0.521. +// Version 2: Adds Proto::linedefined. Currently supported. +// Version 3: Adds FORGPREP/JUMPXEQK* and enhances AUX encoding for FORGLOOP. Removes FORGLOOP_NEXT/INEXT and JUMPIFEQK/JUMPIFNOTEQK. Currently supported. + // Bytecode opcode, part of the instruction header enum LuauOpcode { @@ -367,6 +375,20 @@ enum LuauOpcode // D: jump offset (-32768..32767) LOP_FORGPREP, + // JUMPXEQKNIL, JUMPXEQKB: jumps to target offset if the comparison with constant is true (or false, see AUX) + // A: source register 1 + // D: jump offset (-32768..32767; 0 means "next instruction" aka "don't jump") + // AUX: constant value (for boolean) in low bit, NOT flag (that flips comparison result) in high bit + LOP_JUMPXEQKNIL, + LOP_JUMPXEQKB, + + // JUMPXEQKN, JUMPXEQKS: jumps to target offset if the comparison with constant is true (or false, see AUX) + // A: source register 1 + // D: jump offset (-32768..32767; 0 means "next instruction" aka "don't jump") + // AUX: constant table index in low 24 bits, NOT flag (that flips comparison result) in high bit + LOP_JUMPXEQKN, + LOP_JUMPXEQKS, + // Enum entry for number of opcodes, not a valid opcode by itself! LOP__COUNT }; @@ -391,7 +413,7 @@ enum LuauBytecodeTag { // Bytecode version; runtime supports [MIN, MAX], compiler emits TARGET by default but may emit a higher version when flags are enabled LBC_VERSION_MIN = 2, - LBC_VERSION_MAX = 2, + LBC_VERSION_MAX = 3, LBC_VERSION_TARGET = 2, // Types of constant table entries LBC_CONSTANT_NIL = 0, diff --git a/Common/include/Luau/Common.h b/Common/include/Luau/Common.h index fbb03a9e..f1846ac3 100644 --- a/Common/include/Luau/Common.h +++ b/Common/include/Luau/Common.h @@ -20,12 +20,6 @@ #define LUAU_DEBUGBREAK() __builtin_trap() #endif - - - - - - namespace Luau { @@ -67,16 +61,13 @@ struct FValue const char* name; FValue* next; - FValue(const char* name, T def, bool dynamic, void (*reg)(const char*, T*, bool) = nullptr) + FValue(const char* name, T def, bool dynamic) : value(def) , dynamic(dynamic) , name(name) , next(list) { list = this; - - if (reg) - reg(name, &value, dynamic); } operator T() const @@ -98,7 +89,7 @@ FValue* FValue::list = nullptr; #define LUAU_FASTFLAGVARIABLE(flag, def) \ namespace FFlag \ { \ - Luau::FValue flag(#flag, def, false, nullptr); \ + Luau::FValue flag(#flag, def, false); \ } #define LUAU_FASTINT(flag) \ namespace FInt \ @@ -108,7 +99,7 @@ FValue* FValue::list = nullptr; #define LUAU_FASTINTVARIABLE(flag, def) \ namespace FInt \ { \ - Luau::FValue flag(#flag, def, false, nullptr); \ + Luau::FValue flag(#flag, def, false); \ } #define LUAU_DYNAMIC_FASTFLAG(flag) \ @@ -119,7 +110,7 @@ FValue* FValue::list = nullptr; #define LUAU_DYNAMIC_FASTFLAGVARIABLE(flag, def) \ namespace DFFlag \ { \ - Luau::FValue flag(#flag, def, true, nullptr); \ + Luau::FValue flag(#flag, def, true); \ } #define LUAU_DYNAMIC_FASTINT(flag) \ namespace DFInt \ @@ -129,5 +120,5 @@ FValue* FValue::list = nullptr; #define LUAU_DYNAMIC_FASTINTVARIABLE(flag, def) \ namespace DFInt \ { \ - Luau::FValue flag(#flag, def, true, nullptr); \ + Luau::FValue flag(#flag, def, true); \ } diff --git a/Common/include/Luau/ExperimentalFlags.h b/Common/include/Luau/ExperimentalFlags.h index 3525259d..71e76ffe 100644 --- a/Common/include/Luau/ExperimentalFlags.h +++ b/Common/include/Luau/ExperimentalFlags.h @@ -10,17 +10,16 @@ inline bool isFlagExperimental(const char* flag) { // Flags in this list are disabled by default in various command-line tools. They may have behavior that is not fully final, // or critical bugs that are found after the code has been submitted. - static const char* kList[] = - { + static const char* kList[] = { "LuauLowerBoundsCalculation", nullptr, // makes sure we always have at least one entry }; - for (const char* item: kList) + for (const char* item : kList) if (item && strcmp(item, flag) == 0) return true; return false; } -} +} // namespace Luau diff --git a/Compiler/include/luacode.h b/Compiler/include/luacode.h index e235a2e7..5f69f69e 100644 --- a/Compiler/include/luacode.h +++ b/Compiler/include/luacode.h @@ -3,7 +3,7 @@ #include -/* Can be used to reconfigure visibility/exports for public APIs */ +// Can be used to reconfigure visibility/exports for public APIs #ifndef LUACODE_API #define LUACODE_API extern #endif @@ -35,5 +35,5 @@ struct lua_CompileOptions const char** mutableGlobals; }; -/* compile source to bytecode; when source compilation fails, the resulting bytecode contains the encoded error. use free() to destroy */ +// compile source to bytecode; when source compilation fails, the resulting bytecode contains the encoded error. use free() to destroy LUACODE_API char* luau_compile(const char* source, size_t size, lua_CompileOptions* options, size_t* outsize); diff --git a/Compiler/src/Builtins.cpp b/Compiler/src/Builtins.cpp index 3650c147..26933730 100644 --- a/Compiler/src/Builtins.cpp +++ b/Compiler/src/Builtins.cpp @@ -4,8 +4,6 @@ #include "Luau/Bytecode.h" #include "Luau/Compiler.h" -LUAU_FASTFLAGVARIABLE(LuauCompileRawlen, false) - namespace Luau { namespace Compile @@ -57,7 +55,7 @@ static int getBuiltinFunctionId(const Builtin& builtin, const CompileOptions& op return LBF_RAWGET; if (builtin.isGlobal("rawequal")) return LBF_RAWEQUAL; - if (FFlag::LuauCompileRawlen && builtin.isGlobal("rawlen")) + if (builtin.isGlobal("rawlen")) return LBF_RAWLEN; if (builtin.isGlobal("unpack")) diff --git a/Compiler/src/BytecodeBuilder.cpp b/Compiler/src/BytecodeBuilder.cpp index 64327bd0..46ab2648 100644 --- a/Compiler/src/BytecodeBuilder.cpp +++ b/Compiler/src/BytecodeBuilder.cpp @@ -6,6 +6,8 @@ #include #include +LUAU_FASTFLAGVARIABLE(LuauCompileBytecodeV3, false) + namespace Luau { @@ -77,6 +79,10 @@ static int getOpLength(LuauOpcode op) case LOP_JUMPIFNOTEQK: case LOP_FASTCALL2: case LOP_FASTCALL2K: + case LOP_JUMPXEQKNIL: + case LOP_JUMPXEQKB: + case LOP_JUMPXEQKN: + case LOP_JUMPXEQKS: return 2; default: @@ -108,6 +114,10 @@ inline bool isJumpD(LuauOpcode op) case LOP_JUMPBACK: case LOP_JUMPIFEQK: case LOP_JUMPIFNOTEQK: + case LOP_JUMPXEQKNIL: + case LOP_JUMPXEQKB: + case LOP_JUMPXEQKN: + case LOP_JUMPXEQKS: return true; default: @@ -1069,6 +1079,9 @@ std::string BytecodeBuilder::getError(const std::string& message) uint8_t BytecodeBuilder::getVersion() { + if (FFlag::LuauCompileBytecodeV3) + return 3; + // This function usually returns LBC_VERSION_TARGET but may sometimes return a higher number (within LBC_VERSION_MIN/MAX) under fast flags return LBC_VERSION_TARGET; } @@ -1246,6 +1259,24 @@ void BytecodeBuilder::validate() const VJUMP(LUAU_INSN_D(insn)); break; + case LOP_JUMPXEQKNIL: + case LOP_JUMPXEQKB: + VREG(LUAU_INSN_A(insn)); + VJUMP(LUAU_INSN_D(insn)); + break; + + case LOP_JUMPXEQKN: + VREG(LUAU_INSN_A(insn)); + VCONST(insns[i + 1] & 0xffffff, Number); + VJUMP(LUAU_INSN_D(insn)); + break; + + case LOP_JUMPXEQKS: + VREG(LUAU_INSN_A(insn)); + VCONST(insns[i + 1] & 0xffffff, String); + VJUMP(LUAU_INSN_D(insn)); + break; + case LOP_ADD: case LOP_SUB: case LOP_MUL: @@ -1779,6 +1810,26 @@ void BytecodeBuilder::dumpInstruction(const uint32_t* code, std::string& result, formatAppend(result, "JUMPIFNOTEQK R%d K%d L%d\n", LUAU_INSN_A(insn), *code++, targetLabel); break; + case LOP_JUMPXEQKNIL: + formatAppend(result, "JUMPXEQKNIL R%d L%d%s\n", LUAU_INSN_A(insn), targetLabel, *code >> 31 ? " NOT" : ""); + code++; + break; + + case LOP_JUMPXEQKB: + formatAppend(result, "JUMPXEQKB R%d %d L%d%s\n", LUAU_INSN_A(insn), *code & 1, targetLabel, *code >> 31 ? " NOT" : ""); + code++; + break; + + case LOP_JUMPXEQKN: + formatAppend(result, "JUMPXEQKN R%d K%d L%d%s\n", LUAU_INSN_A(insn), *code & 0xffffff, targetLabel, *code >> 31 ? " NOT" : ""); + code++; + break; + + case LOP_JUMPXEQKS: + formatAppend(result, "JUMPXEQKS R%d K%d L%d%s\n", LUAU_INSN_A(insn), *code & 0xffffff, targetLabel, *code >> 31 ? " NOT" : ""); + code++; + break; + default: LUAU_ASSERT(!"Unsupported opcode"); } diff --git a/Compiler/src/Compiler.cpp b/Compiler/src/Compiler.cpp index 4be54378..2ee20cab 100644 --- a/Compiler/src/Compiler.cpp +++ b/Compiler/src/Compiler.cpp @@ -26,6 +26,7 @@ LUAU_FASTINTVARIABLE(LuauCompileInlineDepth, 5) LUAU_FASTFLAGVARIABLE(LuauCompileNoIpairs, false) LUAU_FASTFLAGVARIABLE(LuauCompileFreeReassign, false) +LUAU_FASTFLAGVARIABLE(LuauCompileXEQ, false) namespace Luau { @@ -1008,9 +1009,8 @@ struct Compiler size_t compileCompareJump(AstExprBinary* expr, bool not_ = false) { RegScope rs(this); - LuauOpcode opc = getJumpOpCompare(expr->op, not_); - bool isEq = (opc == LOP_JUMPIFEQ || opc == LOP_JUMPIFNOTEQ); + bool isEq = (expr->op == AstExprBinary::CompareEq || expr->op == AstExprBinary::CompareNe); AstExpr* left = expr->left; AstExpr* right = expr->right; @@ -1022,36 +1022,112 @@ struct Compiler std::swap(left, right); } - uint8_t rl = compileExprAuto(left, rs); - int32_t rr = -1; - - if (isEq && operandIsConstant) + if (FFlag::LuauCompileXEQ) { - if (opc == LOP_JUMPIFEQ) - opc = LOP_JUMPIFEQK; - else if (opc == LOP_JUMPIFNOTEQ) - opc = LOP_JUMPIFNOTEQK; + uint8_t rl = compileExprAuto(left, rs); - rr = getConstantIndex(right); - LUAU_ASSERT(rr >= 0); - } - else - rr = compileExprAuto(right, rs); + if (isEq && operandIsConstant) + { + const Constant* cv = constants.find(right); + LUAU_ASSERT(cv && cv->type != Constant::Type_Unknown); - size_t jumpLabel = bytecode.emitLabel(); + LuauOpcode opc = LOP_NOP; + int32_t cid = -1; + uint32_t flip = (expr->op == AstExprBinary::CompareEq) == not_ ? 0x80000000 : 0; - if (expr->op == AstExprBinary::CompareGt || expr->op == AstExprBinary::CompareGe) - { - bytecode.emitAD(opc, uint8_t(rr), 0); - bytecode.emitAux(rl); + switch (cv->type) + { + case Constant::Type_Nil: + opc = LOP_JUMPXEQKNIL; + cid = 0; + break; + + case Constant::Type_Boolean: + opc = LOP_JUMPXEQKB; + cid = cv->valueBoolean; + break; + + case Constant::Type_Number: + opc = LOP_JUMPXEQKN; + cid = getConstantIndex(right); + break; + + case Constant::Type_String: + opc = LOP_JUMPXEQKS; + cid = getConstantIndex(right); + break; + + default: + LUAU_ASSERT(!"Unexpected constant type"); + } + + if (cid < 0) + CompileError::raise(expr->location, "Exceeded constant limit; simplify the code to compile"); + + size_t jumpLabel = bytecode.emitLabel(); + + bytecode.emitAD(opc, rl, 0); + bytecode.emitAux(cid | flip); + + return jumpLabel; + } + else + { + LuauOpcode opc = getJumpOpCompare(expr->op, not_); + + uint8_t rr = compileExprAuto(right, rs); + + size_t jumpLabel = bytecode.emitLabel(); + + if (expr->op == AstExprBinary::CompareGt || expr->op == AstExprBinary::CompareGe) + { + bytecode.emitAD(opc, rr, 0); + bytecode.emitAux(rl); + } + else + { + bytecode.emitAD(opc, rl, 0); + bytecode.emitAux(rr); + } + + return jumpLabel; + } } else { - bytecode.emitAD(opc, rl, 0); - bytecode.emitAux(rr); - } + LuauOpcode opc = getJumpOpCompare(expr->op, not_); - return jumpLabel; + uint8_t rl = compileExprAuto(left, rs); + int32_t rr = -1; + + if (isEq && operandIsConstant) + { + if (opc == LOP_JUMPIFEQ) + opc = LOP_JUMPIFEQK; + else if (opc == LOP_JUMPIFNOTEQ) + opc = LOP_JUMPIFNOTEQK; + + rr = getConstantIndex(right); + LUAU_ASSERT(rr >= 0); + } + else + rr = compileExprAuto(right, rs); + + size_t jumpLabel = bytecode.emitLabel(); + + if (expr->op == AstExprBinary::CompareGt || expr->op == AstExprBinary::CompareGe) + { + bytecode.emitAD(opc, uint8_t(rr), 0); + bytecode.emitAux(rl); + } + else + { + bytecode.emitAD(opc, rl, 0); + bytecode.emitAux(rr); + } + + return jumpLabel; + } } int32_t getConstantNumber(AstExpr* node) diff --git a/Makefile b/Makefile index 4323a321..01fac077 100644 --- a/Makefile +++ b/Makefile @@ -55,6 +55,7 @@ ifneq ($(opt),) endif OBJECTS=$(AST_OBJECTS) $(COMPILER_OBJECTS) $(ANALYSIS_OBJECTS) $(CODEGEN_OBJECTS) $(VM_OBJECTS) $(ISOCLINE_OBJECTS) $(TESTS_OBJECTS) $(CLI_OBJECTS) $(FUZZ_OBJECTS) +EXECUTABLE_ALIASES = luau luau-analyze luau-tests # common flags CXXFLAGS=-g -Wall @@ -121,14 +122,14 @@ fuzz-proto fuzz-prototest: LDFLAGS+=build/libprotobuf-mutator/src/libfuzzer/libp all: $(REPL_CLI_TARGET) $(ANALYZE_CLI_TARGET) $(TESTS_TARGET) aliases -aliases: luau luau-analyze +aliases: $(EXECUTABLE_ALIASES) test: $(TESTS_TARGET) $(TESTS_TARGET) $(TESTS_ARGS) clean: rm -rf $(BUILD) - rm -rf luau luau-analyze + rm -rf $(EXECUTABLE_ALIASES) coverage: $(TESTS_TARGET) $(TESTS_TARGET) --fflags=true @@ -154,6 +155,9 @@ luau: $(REPL_CLI_TARGET) luau-analyze: $(ANALYZE_CLI_TARGET) ln -fs $^ $@ +luau-tests: $(TESTS_TARGET) + ln -fs $^ $@ + # executable targets $(TESTS_TARGET): $(TESTS_OBJECTS) $(ANALYSIS_TARGET) $(COMPILER_TARGET) $(AST_TARGET) $(CODEGEN_TARGET) $(VM_TARGET) $(ISOCLINE_TARGET) $(REPL_CLI_TARGET): $(REPL_CLI_OBJECTS) $(COMPILER_TARGET) $(AST_TARGET) $(VM_TARGET) $(ISOCLINE_TARGET) diff --git a/Sources.cmake b/Sources.cmake index f745667f..9a6019a9 100644 --- a/Sources.cmake +++ b/Sources.cmake @@ -66,6 +66,8 @@ target_sources(Luau.CodeGen PRIVATE # Luau.Analysis Sources target_sources(Luau.Analysis PRIVATE + Analysis/include/Luau/ApplyTypeFunction.h + Analysis/include/Luau/AstJsonEncoder.h Analysis/include/Luau/AstQuery.h Analysis/include/Luau/Autocomplete.h Analysis/include/Luau/BuiltinDefinitions.h @@ -81,7 +83,7 @@ target_sources(Luau.Analysis PRIVATE Analysis/include/Luau/Frontend.h Analysis/include/Luau/Instantiation.h Analysis/include/Luau/IostreamHelpers.h - Analysis/include/Luau/JsonEncoder.h + Analysis/include/Luau/JsonEmitter.h Analysis/include/Luau/Linter.h Analysis/include/Luau/LValue.h Analysis/include/Luau/Module.h @@ -113,6 +115,8 @@ target_sources(Luau.Analysis PRIVATE Analysis/include/Luau/Variant.h Analysis/include/Luau/VisitTypeVar.h + Analysis/src/ApplyTypeFunction.cpp + Analysis/src/AstJsonEncoder.cpp Analysis/src/AstQuery.cpp Analysis/src/Autocomplete.cpp Analysis/src/BuiltinDefinitions.cpp @@ -126,7 +130,7 @@ target_sources(Luau.Analysis PRIVATE Analysis/src/Frontend.cpp Analysis/src/Instantiation.cpp Analysis/src/IostreamHelpers.cpp - Analysis/src/JsonEncoder.cpp + Analysis/src/JsonEmitter.cpp Analysis/src/Linter.cpp Analysis/src/LValue.cpp Analysis/src/Module.cpp @@ -255,6 +259,7 @@ if(TARGET Luau.UnitTest) tests/ScopedFlags.h tests/Fixture.cpp tests/AssemblyBuilderX64.test.cpp + tests/AstJsonEncoder.test.cpp tests/AstQuery.test.cpp tests/AstVisitor.test.cpp tests/Autocomplete.test.cpp @@ -266,7 +271,7 @@ if(TARGET Luau.UnitTest) tests/CostModel.test.cpp tests/Error.test.cpp tests/Frontend.test.cpp - tests/JsonEncoder.test.cpp + tests/JsonEmitter.test.cpp tests/Lexer.test.cpp tests/Linter.test.cpp tests/LValue.test.cpp diff --git a/VM/include/lua.h b/VM/include/lua.h index 28d7b1c5..1f315a08 100644 --- a/VM/include/lua.h +++ b/VM/include/lua.h @@ -11,7 +11,7 @@ -/* option for multiple returns in `lua_pcall' and `lua_call' */ +// option for multiple returns in `lua_pcall' and `lua_call' #define LUA_MULTRET (-1) /* @@ -23,7 +23,7 @@ #define lua_upvalueindex(i) (LUA_GLOBALSINDEX - (i)) #define lua_ispseudo(i) ((i) <= LUA_REGISTRYINDEX) -/* thread status; 0 is OK */ +// thread status; 0 is OK enum lua_Status { LUA_OK = 0, @@ -32,7 +32,7 @@ enum lua_Status LUA_ERRSYNTAX, LUA_ERRMEM, LUA_ERRERR, - LUA_BREAK, /* yielded for a debug breakpoint */ + LUA_BREAK, // yielded for a debug breakpoint }; typedef struct lua_State lua_State; @@ -46,7 +46,7 @@ typedef int (*lua_Continuation)(lua_State* L, int status); typedef void* (*lua_Alloc)(void* ud, void* ptr, size_t osize, size_t nsize); -/* non-return type */ +// non-return type #define l_noret void LUA_NORETURN /* @@ -61,15 +61,15 @@ typedef void* (*lua_Alloc)(void* ud, void* ptr, size_t osize, size_t nsize); // clang-format off enum lua_Type { - LUA_TNIL = 0, /* must be 0 due to lua_isnoneornil */ - LUA_TBOOLEAN = 1, /* must be 1 due to l_isfalse */ + LUA_TNIL = 0, // must be 0 due to lua_isnoneornil + LUA_TBOOLEAN = 1, // must be 1 due to l_isfalse LUA_TLIGHTUSERDATA, LUA_TNUMBER, LUA_TVECTOR, - LUA_TSTRING, /* all types above this must be value types, all types below this must be GC types - see iscollectable */ + LUA_TSTRING, // all types above this must be value types, all types below this must be GC types - see iscollectable LUA_TTABLE, @@ -77,23 +77,23 @@ enum lua_Type LUA_TUSERDATA, LUA_TTHREAD, - /* values below this line are used in GCObject tags but may never show up in TValue type tags */ + // values below this line are used in GCObject tags but may never show up in TValue type tags LUA_TPROTO, LUA_TUPVAL, LUA_TDEADKEY, - /* the count of TValue type tags */ + // the count of TValue type tags LUA_T_COUNT = LUA_TPROTO }; // clang-format on -/* type of numbers in Luau */ +// type of numbers in Luau typedef double lua_Number; -/* type for integer functions */ +// type for integer functions typedef int lua_Integer; -/* unsigned integer type */ +// unsigned integer type typedef unsigned lua_Unsigned; /* @@ -117,7 +117,7 @@ LUA_API void lua_remove(lua_State* L, int idx); LUA_API void lua_insert(lua_State* L, int idx); LUA_API void lua_replace(lua_State* L, int idx); LUA_API int lua_checkstack(lua_State* L, int sz); -LUA_API void lua_rawcheckstack(lua_State* L, int sz); /* allows for unlimited stack frames */ +LUA_API void lua_rawcheckstack(lua_State* L, int sz); // allows for unlimited stack frames LUA_API void lua_xmove(lua_State* from, lua_State* to, int n); LUA_API void lua_xpush(lua_State* from, lua_State* to, int idx); @@ -231,18 +231,18 @@ LUA_API void lua_setthreaddata(lua_State* L, void* data); enum lua_GCOp { - /* stop and resume incremental garbage collection */ + // stop and resume incremental garbage collection LUA_GCSTOP, LUA_GCRESTART, - /* run a full GC cycle; not recommended for latency sensitive applications */ + // run a full GC cycle; not recommended for latency sensitive applications LUA_GCCOLLECT, - /* return the heap size in KB and the remainder in bytes */ + // return the heap size in KB and the remainder in bytes LUA_GCCOUNT, LUA_GCCOUNTB, - /* return 1 if GC is active (not stopped); note that GC may not be actively collecting even if it's running */ + // return 1 if GC is active (not stopped); note that GC may not be actively collecting even if it's running LUA_GCISRUNNING, /* @@ -359,9 +359,9 @@ LUA_API void lua_unref(lua_State* L, int ref); ** ======================================================================= */ -typedef struct lua_Debug lua_Debug; /* activation record */ +typedef struct lua_Debug lua_Debug; // activation record -/* Functions to be called by the debugger in specific events */ +// Functions to be called by the debugger in specific events typedef void (*lua_Hook)(lua_State* L, lua_Debug* ar); LUA_API int lua_stackdepth(lua_State* L); @@ -379,24 +379,24 @@ typedef void (*lua_Coverage)(void* context, const char* function, int linedefine LUA_API void lua_getcoverage(lua_State* L, int funcindex, void* context, lua_Coverage callback); -/* Warning: this function is not thread-safe since it stores the result in a shared global array! Only use for debugging. */ +// Warning: this function is not thread-safe since it stores the result in a shared global array! Only use for debugging. LUA_API const char* lua_debugtrace(lua_State* L); struct lua_Debug { - const char* name; /* (n) */ - const char* what; /* (s) `Lua', `C', `main', `tail' */ - const char* source; /* (s) */ - int linedefined; /* (s) */ - int currentline; /* (l) */ - unsigned char nupvals; /* (u) number of upvalues */ - unsigned char nparams; /* (a) number of parameters */ - char isvararg; /* (a) */ - char short_src[LUA_IDSIZE]; /* (s) */ - void* userdata; /* only valid in luau_callhook */ + const char* name; // (n) + const char* what; // (s) `Lua', `C', `main', `tail' + const char* source; // (s) + int linedefined; // (s) + int currentline; // (l) + unsigned char nupvals; // (u) number of upvalues + unsigned char nparams; // (a) number of parameters + char isvararg; // (a) + char short_src[LUA_IDSIZE]; // (s) + void* userdata; // only valid in luau_callhook }; -/* }====================================================================== */ +// }====================================================================== /* Callbacks that can be used to reconfigure behavior of the VM dynamically. * These are shared between all coroutines. @@ -405,18 +405,18 @@ struct lua_Debug * can only be changed when the VM is not running any code */ struct lua_Callbacks { - void* userdata; /* arbitrary userdata pointer that is never overwritten by Luau */ + void* userdata; // arbitrary userdata pointer that is never overwritten by Luau - void (*interrupt)(lua_State* L, int gc); /* gets called at safepoints (loop back edges, call/ret, gc) if set */ - void (*panic)(lua_State* L, int errcode); /* gets called when an unprotected error is raised (if longjmp is used) */ + void (*interrupt)(lua_State* L, int gc); // gets called at safepoints (loop back edges, call/ret, gc) if set + void (*panic)(lua_State* L, int errcode); // gets called when an unprotected error is raised (if longjmp is used) - void (*userthread)(lua_State* LP, lua_State* L); /* gets called when L is created (LP == parent) or destroyed (LP == NULL) */ - int16_t (*useratom)(const char* s, size_t l); /* gets called when a string is created; returned atom can be retrieved via tostringatom */ + void (*userthread)(lua_State* LP, lua_State* L); // gets called when L is created (LP == parent) or destroyed (LP == NULL) + int16_t (*useratom)(const char* s, size_t l); // gets called when a string is created; returned atom can be retrieved via tostringatom - void (*debugbreak)(lua_State* L, lua_Debug* ar); /* gets called when BREAK instruction is encountered */ - void (*debugstep)(lua_State* L, lua_Debug* ar); /* gets called after each instruction in single step mode */ - void (*debuginterrupt)(lua_State* L, lua_Debug* ar); /* gets called when thread execution is interrupted by break in another thread */ - void (*debugprotectederror)(lua_State* L); /* gets called when protected call results in an error */ + void (*debugbreak)(lua_State* L, lua_Debug* ar); // gets called when BREAK instruction is encountered + void (*debugstep)(lua_State* L, lua_Debug* ar); // gets called after each instruction in single step mode + void (*debuginterrupt)(lua_State* L, lua_Debug* ar); // gets called when thread execution is interrupted by break in another thread + void (*debugprotectederror)(lua_State* L); // gets called when protected call results in an error }; typedef struct lua_Callbacks lua_Callbacks; diff --git a/VM/include/luaconf.h b/VM/include/luaconf.h index b93cbf7c..7b0f4c30 100644 --- a/VM/include/luaconf.h +++ b/VM/include/luaconf.h @@ -33,14 +33,14 @@ #define LUA_NORETURN __attribute__((__noreturn__)) #endif -/* Can be used to reconfigure visibility/exports for public APIs */ +// Can be used to reconfigure visibility/exports for public APIs #ifndef LUA_API #define LUA_API extern #endif #define LUALIB_API LUA_API -/* Can be used to reconfigure visibility for internal APIs */ +// Can be used to reconfigure visibility for internal APIs #if defined(__GNUC__) #define LUAI_FUNC __attribute__((visibility("hidden"))) extern #define LUAI_DATA LUAI_FUNC @@ -49,67 +49,67 @@ #define LUAI_DATA extern #endif -/* Can be used to reconfigure internal error handling to use longjmp instead of C++ EH */ +// Can be used to reconfigure internal error handling to use longjmp instead of C++ EH #ifndef LUA_USE_LONGJMP #define LUA_USE_LONGJMP 0 #endif -/* LUA_IDSIZE gives the maximum size for the description of the source */ +// LUA_IDSIZE gives the maximum size for the description of the source #ifndef LUA_IDSIZE #define LUA_IDSIZE 256 #endif -/* LUA_MINSTACK is the guaranteed number of Lua stack slots available to a C function */ +// LUA_MINSTACK is the guaranteed number of Lua stack slots available to a C function #ifndef LUA_MINSTACK #define LUA_MINSTACK 20 #endif -/* LUAI_MAXCSTACK limits the number of Lua stack slots that a C function can use */ +// LUAI_MAXCSTACK limits the number of Lua stack slots that a C function can use #ifndef LUAI_MAXCSTACK #define LUAI_MAXCSTACK 8000 #endif -/* LUAI_MAXCALLS limits the number of nested calls */ +// LUAI_MAXCALLS limits the number of nested calls #ifndef LUAI_MAXCALLS #define LUAI_MAXCALLS 20000 #endif -/* LUAI_MAXCCALLS is the maximum depth for nested C calls; this limit depends on native stack size */ +// LUAI_MAXCCALLS is the maximum depth for nested C calls; this limit depends on native stack size #ifndef LUAI_MAXCCALLS #define LUAI_MAXCCALLS 200 #endif -/* buffer size used for on-stack string operations; this limit depends on native stack size */ +// buffer size used for on-stack string operations; this limit depends on native stack size #ifndef LUA_BUFFERSIZE #define LUA_BUFFERSIZE 512 #endif -/* number of valid Lua userdata tags */ +// number of valid Lua userdata tags #ifndef LUA_UTAG_LIMIT #define LUA_UTAG_LIMIT 128 #endif -/* upper bound for number of size classes used by page allocator */ +// upper bound for number of size classes used by page allocator #ifndef LUA_SIZECLASSES #define LUA_SIZECLASSES 32 #endif -/* available number of separate memory categories */ +// available number of separate memory categories #ifndef LUA_MEMORY_CATEGORIES #define LUA_MEMORY_CATEGORIES 256 #endif -/* minimum size for the string table (must be power of 2) */ +// minimum size for the string table (must be power of 2) #ifndef LUA_MINSTRTABSIZE #define LUA_MINSTRTABSIZE 32 #endif -/* maximum number of captures supported by pattern matching */ +// maximum number of captures supported by pattern matching #ifndef LUA_MAXCAPTURES #define LUA_MAXCAPTURES 32 #endif -/* }================================================================== */ +// }================================================================== /* @@ LUAI_USER_ALIGNMENT_T is a type that requires maximum alignment. @@ -126,6 +126,6 @@ long l; \ } -#define LUA_VECTOR_SIZE 3 /* must be 3 or 4 */ +#define LUA_VECTOR_SIZE 3 // must be 3 or 4 #define LUA_EXTRA_SIZE LUA_VECTOR_SIZE - 2 diff --git a/VM/include/lualib.h b/VM/include/lualib.h index bebd0a0f..955604de 100644 --- a/VM/include/lualib.h +++ b/VM/include/lualib.h @@ -72,7 +72,7 @@ LUALIB_API const char* luaL_typename(lua_State* L, int idx); #define luaL_opt(L, f, n, d) (lua_isnoneornil(L, (n)) ? (d) : f(L, (n))) -/* generic buffer manipulation */ +// generic buffer manipulation struct luaL_Buffer { @@ -102,7 +102,7 @@ LUALIB_API void luaL_addvalue(luaL_Buffer* B); LUALIB_API void luaL_pushresult(luaL_Buffer* B); LUALIB_API void luaL_pushresultsize(luaL_Buffer* B, size_t size); -/* builtin libraries */ +// builtin libraries LUALIB_API int luaopen_base(lua_State* L); #define LUA_COLIBNAME "coroutine" @@ -129,9 +129,9 @@ LUALIB_API int luaopen_math(lua_State* L); #define LUA_DBLIBNAME "debug" LUALIB_API int luaopen_debug(lua_State* L); -/* open all builtin libraries */ +// open all builtin libraries LUALIB_API void luaL_openlibs(lua_State* L); -/* sandbox libraries and globals */ +// sandbox libraries and globals LUALIB_API void luaL_sandbox(lua_State* L); LUALIB_API void luaL_sandboxthread(lua_State* L); diff --git a/VM/src/lapi.cpp b/VM/src/lapi.cpp index e59f914f..bb994fb4 100644 --- a/VM/src/lapi.cpp +++ b/VM/src/lapi.cpp @@ -59,8 +59,8 @@ const char* luau_ident = "$Luau: Copyright (C) 2019-2022 Roblox Corporation $\n" static Table* getcurrenv(lua_State* L) { - if (L->ci == L->base_ci) /* no enclosing function? */ - return L->gt; /* use global table as environment */ + if (L->ci == L->base_ci) // no enclosing function? + return L->gt; // use global table as environment else return curr_func(L)->env; } @@ -69,7 +69,7 @@ static LUAU_NOINLINE TValue* pseudo2addr(lua_State* L, int idx) { api_check(L, lua_ispseudo(idx)); switch (idx) - { /* pseudo-indices */ + { // pseudo-indices case LUA_REGISTRYINDEX: return registry(L); case LUA_ENVIRONINDEX: @@ -129,7 +129,7 @@ int lua_checkstack(lua_State* L, int size) { int res = 1; if (size > LUAI_MAXCSTACK || (L->top - L->base + size) > LUAI_MAXCSTACK) - res = 0; /* stack overflow */ + res = 0; // stack overflow else if (size > 0) { luaD_checkstack(L, size); @@ -219,7 +219,7 @@ void lua_settop(lua_State* L, int idx) else { api_check(L, -(idx + 1) <= (L->top - L->base)); - L->top += idx + 1; /* `subtract' index (index is negative) */ + L->top += idx + 1; // `subtract' index (index is negative) } return; } @@ -267,7 +267,7 @@ void lua_replace(lua_State* L, int idx) else { setobj(L, o, L->top - 1); - if (idx < LUA_GLOBALSINDEX) /* function upvalue? */ + if (idx < LUA_GLOBALSINDEX) // function upvalue? luaC_barrier(L, curr_func(L), L->top - 1); } L->top--; @@ -429,13 +429,13 @@ const char* lua_tolstring(lua_State* L, int idx, size_t* len) { luaC_checkthreadsleep(L); if (!luaV_tostring(L, o)) - { /* conversion failed? */ + { // conversion failed? if (len != NULL) *len = 0; return NULL; } luaC_checkGC(L); - o = index2addr(L, idx); /* previous call may reallocate the stack */ + o = index2addr(L, idx); // previous call may reallocate the stack } if (len != NULL) *len = tsvalue(o)->len; @@ -660,7 +660,7 @@ void lua_pushcclosurek(lua_State* L, lua_CFunction fn, const char* debugname, in void lua_pushboolean(lua_State* L, int b) { - setbvalue(L->top, (b != 0)); /* ensure that true is 1 */ + setbvalue(L->top, (b != 0)); // ensure that true is 1 api_incr_top(L); return; } @@ -829,7 +829,7 @@ void lua_settable(lua_State* L, int idx) StkId t = index2addr(L, idx); api_checkvalidindex(L, t); luaV_settable(L, t, L->top - 2, L->top - 1); - L->top -= 2; /* pop index and value */ + L->top -= 2; // pop index and value return; } @@ -967,7 +967,7 @@ void lua_call(lua_State* L, int nargs, int nresults) ** Execute a protected call. */ struct CallS -{ /* data to `f_call' */ +{ // data to `f_call' StkId func; int nresults; }; @@ -992,7 +992,7 @@ int lua_pcall(lua_State* L, int nargs, int nresults, int errfunc) func = savestack(L, o); } struct CallS c; - c.func = L->top - (nargs + 1); /* function to be called */ + c.func = L->top - (nargs + 1); // function to be called c.nresults = nresults; int status = luaD_pcall(L, f_call, &c, savestack(L, c.func), func); @@ -1044,7 +1044,7 @@ int lua_gc(lua_State* L, int what, int data) } case LUA_GCCOUNT: { - /* GC values are expressed in Kbytes: #bytes/2^10 */ + // GC values are expressed in Kbytes: #bytes/2^10 res = cast_int(g->totalbytes >> 10); break; } @@ -1084,8 +1084,8 @@ int lua_gc(lua_State* L, int what, int data) actualwork += stepsize; if (g->gcstate == GCSpause) - { /* end of cycle? */ - res = 1; /* signal it */ + { // end of cycle? + res = 1; // signal it break; } } @@ -1137,13 +1137,13 @@ int lua_gc(lua_State* L, int what, int data) } case LUA_GCSETSTEPSIZE: { - /* GC values are expressed in Kbytes: #bytes/2^10 */ + // GC values are expressed in Kbytes: #bytes/2^10 res = g->gcstepsize >> 10; g->gcstepsize = data << 10; break; } default: - res = -1; /* invalid option */ + res = -1; // invalid option } return res; } @@ -1169,8 +1169,8 @@ int lua_next(lua_State* L, int idx) { api_incr_top(L); } - else /* no more elements */ - L->top -= 1; /* remove key */ + else // no more elements + L->top -= 1; // remove key return more; } @@ -1185,12 +1185,12 @@ void lua_concat(lua_State* L, int n) L->top -= (n - 1); } else if (n == 0) - { /* push empty string */ + { // push empty string luaC_checkthreadsleep(L); setsvalue2s(L, L->top, luaS_newlstr(L, "", 0)); api_incr_top(L); } - /* else n == 1; nothing to do */ + // else n == 1; nothing to do return; } @@ -1277,7 +1277,7 @@ uintptr_t lua_encodepointer(lua_State* L, uintptr_t p) int lua_ref(lua_State* L, int idx) { - api_check(L, idx != LUA_REGISTRYINDEX); /* idx is a stack index for value */ + api_check(L, idx != LUA_REGISTRYINDEX); // idx is a stack index for value int ref = LUA_REFNIL; global_State* g = L->global; StkId p = index2addr(L, idx); @@ -1286,13 +1286,13 @@ int lua_ref(lua_State* L, int idx) Table* reg = hvalue(registry(L)); if (g->registryfree != 0) - { /* reuse existing slot */ + { // reuse existing slot ref = g->registryfree; } else - { /* no free elements */ + { // no free elements ref = luaH_getn(reg); - ref++; /* create new reference */ + ref++; // create new reference } TValue* slot = luaH_setnum(L, reg, ref); @@ -1312,7 +1312,7 @@ void lua_unref(lua_State* L, int ref) global_State* g = L->global; Table* reg = hvalue(registry(L)); TValue* slot = luaH_setnum(L, reg, ref); - setnvalue(slot, g->registryfree); /* NB: no barrier needed because value isn't collectable */ + setnvalue(slot, g->registryfree); // NB: no barrier needed because value isn't collectable g->registryfree = ref; return; } diff --git a/VM/src/laux.cpp b/VM/src/laux.cpp index 72169a86..c42e5ccc 100644 --- a/VM/src/laux.cpp +++ b/VM/src/laux.cpp @@ -11,7 +11,7 @@ #include -/* convert a stack index to positive */ +// convert a stack index to positive #define abs_index(L, i) ((i) > 0 || (i) <= LUA_REGISTRYINDEX ? (i) : lua_gettop(L) + (i) + 1) /* @@ -75,7 +75,7 @@ void luaL_where(lua_State* L, int level) lua_pushfstring(L, "%s:%d: ", ar.short_src, ar.currentline); return; } - lua_pushliteral(L, ""); /* else, no information available... */ + lua_pushliteral(L, ""); // else, no information available... } l_noret luaL_errorL(lua_State* L, const char* fmt, ...) @@ -89,7 +89,7 @@ l_noret luaL_errorL(lua_State* L, const char* fmt, ...) lua_error(L); } -/* }====================================================== */ +// }====================================================== int luaL_checkoption(lua_State* L, int narg, const char* def, const char* const lst[]) { @@ -104,13 +104,13 @@ int luaL_checkoption(lua_State* L, int narg, const char* def, const char* const int luaL_newmetatable(lua_State* L, const char* tname) { - lua_getfield(L, LUA_REGISTRYINDEX, tname); /* get registry.name */ - if (!lua_isnil(L, -1)) /* name already in use? */ - return 0; /* leave previous value on top, but return 0 */ + lua_getfield(L, LUA_REGISTRYINDEX, tname); // get registry.name + if (!lua_isnil(L, -1)) // name already in use? + return 0; // leave previous value on top, but return 0 lua_pop(L, 1); - lua_newtable(L); /* create metatable */ + lua_newtable(L); // create metatable lua_pushvalue(L, -1); - lua_setfield(L, LUA_REGISTRYINDEX, tname); /* registry.name = metatable */ + lua_setfield(L, LUA_REGISTRYINDEX, tname); // registry.name = metatable return 1; } @@ -118,18 +118,18 @@ void* luaL_checkudata(lua_State* L, int ud, const char* tname) { void* p = lua_touserdata(L, ud); if (p != NULL) - { /* value is a userdata? */ + { // value is a userdata? if (lua_getmetatable(L, ud)) - { /* does it have a metatable? */ - lua_getfield(L, LUA_REGISTRYINDEX, tname); /* get correct metatable */ + { // does it have a metatable? + lua_getfield(L, LUA_REGISTRYINDEX, tname); // get correct metatable if (lua_rawequal(L, -1, -2)) - { /* does it have the correct mt? */ - lua_pop(L, 2); /* remove both metatables */ + { // does it have the correct mt? + lua_pop(L, 2); // remove both metatables return p; } } } - luaL_typeerrorL(L, ud, tname); /* else error */ + luaL_typeerrorL(L, ud, tname); // else error } void luaL_checkstack(lua_State* L, int space, const char* mes) @@ -243,18 +243,18 @@ const float* luaL_optvector(lua_State* L, int narg, const float* def) int luaL_getmetafield(lua_State* L, int obj, const char* event) { - if (!lua_getmetatable(L, obj)) /* no metatable? */ + if (!lua_getmetatable(L, obj)) // no metatable? return 0; lua_pushstring(L, event); lua_rawget(L, -2); if (lua_isnil(L, -1)) { - lua_pop(L, 2); /* remove metatable and metafield */ + lua_pop(L, 2); // remove metatable and metafield return 0; } else { - lua_remove(L, -2); /* remove only metatable */ + lua_remove(L, -2); // remove only metatable return 1; } } @@ -262,7 +262,7 @@ int luaL_getmetafield(lua_State* L, int obj, const char* event) int luaL_callmeta(lua_State* L, int obj, const char* event) { obj = abs_index(L, obj); - if (!luaL_getmetafield(L, obj, event)) /* no metafield? */ + if (!luaL_getmetafield(L, obj, event)) // no metafield? return 0; lua_pushvalue(L, obj); lua_call(L, 1, 1); @@ -282,19 +282,19 @@ void luaL_register(lua_State* L, const char* libname, const luaL_Reg* l) if (libname) { int size = libsize(l); - /* check whether lib already exists */ + // check whether lib already exists luaL_findtable(L, LUA_REGISTRYINDEX, "_LOADED", 1); - lua_getfield(L, -1, libname); /* get _LOADED[libname] */ + lua_getfield(L, -1, libname); // get _LOADED[libname] if (!lua_istable(L, -1)) - { /* not found? */ - lua_pop(L, 1); /* remove previous result */ - /* try global variable (and create one if it does not exist) */ + { // not found? + lua_pop(L, 1); // remove previous result + // try global variable (and create one if it does not exist) if (luaL_findtable(L, LUA_GLOBALSINDEX, libname, size) != NULL) luaL_error(L, "name conflict for module '%s'", libname); lua_pushvalue(L, -1); - lua_setfield(L, -3, libname); /* _LOADED[libname] = new table */ + lua_setfield(L, -3, libname); // _LOADED[libname] = new table } - lua_remove(L, -2); /* remove _LOADED table */ + lua_remove(L, -2); // remove _LOADED table } for (; l->name; l++) { @@ -315,19 +315,19 @@ const char* luaL_findtable(lua_State* L, int idx, const char* fname, int szhint) lua_pushlstring(L, fname, e - fname); lua_rawget(L, -2); if (lua_isnil(L, -1)) - { /* no such field? */ - lua_pop(L, 1); /* remove this nil */ - lua_createtable(L, 0, (*e == '.' ? 1 : szhint)); /* new table for field */ + { // no such field? + lua_pop(L, 1); // remove this nil + lua_createtable(L, 0, (*e == '.' ? 1 : szhint)); // new table for field lua_pushlstring(L, fname, e - fname); lua_pushvalue(L, -2); - lua_settable(L, -4); /* set new table into field */ + lua_settable(L, -4); // set new table into field } else if (!lua_istable(L, -1)) - { /* field has a non-table value? */ - lua_pop(L, 2); /* remove table and value */ - return fname; /* return problematic part of the name */ + { // field has a non-table value? + lua_pop(L, 2); // remove table and value + return fname; // return problematic part of the name } - lua_remove(L, -2); /* remove previous table */ + lua_remove(L, -2); // remove previous table fname = e + 1; } while (*e == '.'); return NULL; @@ -470,11 +470,11 @@ void luaL_pushresultsize(luaL_Buffer* B, size_t size) luaL_pushresult(B); } -/* }====================================================== */ +// }====================================================== const char* luaL_tolstring(lua_State* L, int idx, size_t* len) { - if (luaL_callmeta(L, idx, "__tostring")) /* is there a metafield? */ + if (luaL_callmeta(L, idx, "__tostring")) // is there a metafield? { if (!lua_isstring(L, -1)) luaL_error(L, "'__tostring' must return a string"); diff --git a/VM/src/lbaselib.cpp b/VM/src/lbaselib.cpp index 4fc5033e..f4dac61f 100644 --- a/VM/src/lbaselib.cpp +++ b/VM/src/lbaselib.cpp @@ -11,8 +11,6 @@ #include #include -LUAU_FASTFLAG(LuauLenTM) - static void writestring(const char* s, size_t l) { fwrite(s, 1, l, stdout); @@ -20,15 +18,15 @@ static void writestring(const char* s, size_t l) static int luaB_print(lua_State* L) { - int n = lua_gettop(L); /* number of arguments */ + int n = lua_gettop(L); // number of arguments for (int i = 1; i <= n; i++) { size_t l; - const char* s = luaL_tolstring(L, i, &l); /* convert to string using __tostring et al */ + const char* s = luaL_tolstring(L, i, &l); // convert to string using __tostring et al if (i > 1) writestring("\t", 1); writestring(s, l); - lua_pop(L, 1); /* pop result */ + lua_pop(L, 1); // pop result } writestring("\n", 1); return 0; @@ -38,7 +36,7 @@ static int luaB_tonumber(lua_State* L) { int base = luaL_optinteger(L, 2, 10); if (base == 10) - { /* standard conversion */ + { // standard conversion int isnum = 0; double n = lua_tonumberx(L, 1, &isnum); if (isnum) @@ -46,7 +44,7 @@ static int luaB_tonumber(lua_State* L) lua_pushnumber(L, n); return 1; } - luaL_checkany(L, 1); /* error if we don't have any argument */ + luaL_checkany(L, 1); // error if we don't have any argument } else { @@ -56,17 +54,17 @@ static int luaB_tonumber(lua_State* L) unsigned long long n; n = strtoull(s1, &s2, base); if (s1 != s2) - { /* at least one valid digit? */ + { // at least one valid digit? while (isspace((unsigned char)(*s2))) - s2++; /* skip trailing spaces */ + s2++; // skip trailing spaces if (*s2 == '\0') - { /* no invalid trailing characters? */ + { // no invalid trailing characters? lua_pushnumber(L, (double)n); return 1; } } } - lua_pushnil(L); /* else not a number */ + lua_pushnil(L); // else not a number return 1; } @@ -75,7 +73,7 @@ static int luaB_error(lua_State* L) int level = luaL_optinteger(L, 2, 1); lua_settop(L, 1); if (lua_isstring(L, 1) && level > 0) - { /* add extra information? */ + { // add extra information? luaL_where(L, level); lua_pushvalue(L, 1); lua_concat(L, 2); @@ -89,10 +87,10 @@ static int luaB_getmetatable(lua_State* L) if (!lua_getmetatable(L, 1)) { lua_pushnil(L); - return 1; /* no metatable */ + return 1; // no metatable } luaL_getmetafield(L, 1, "__metatable"); - return 1; /* returns either __metatable field (if present) or metatable */ + return 1; // returns either __metatable field (if present) or metatable } static int luaB_setmetatable(lua_State* L) @@ -126,8 +124,8 @@ static void getfunc(lua_State* L, int opt) static int luaB_getfenv(lua_State* L) { getfunc(L, 1); - if (lua_iscfunction(L, -1)) /* is a C function? */ - lua_pushvalue(L, LUA_GLOBALSINDEX); /* return the thread's global env. */ + if (lua_iscfunction(L, -1)) // is a C function? + lua_pushvalue(L, LUA_GLOBALSINDEX); // return the thread's global env. else lua_getfenv(L, -1); lua_setsafeenv(L, -1, false); @@ -142,7 +140,7 @@ static int luaB_setfenv(lua_State* L) lua_setsafeenv(L, -1, false); if (lua_isnumber(L, 1) && lua_tonumber(L, 1) == 0) { - /* change environment of current thread */ + // change environment of current thread lua_pushthread(L); lua_insert(L, -2); lua_setfenv(L, -2); @@ -182,9 +180,6 @@ static int luaB_rawset(lua_State* L) static int luaB_rawlen(lua_State* L) { - if (!FFlag::LuauLenTM) - luaL_error(L, "'rawlen' is not available"); - int tt = lua_type(L, 1); luaL_argcheck(L, tt == LUA_TTABLE || tt == LUA_TSTRING, 1, "table or string expected"); int len = lua_objlen(L, 1); @@ -201,7 +196,7 @@ static int luaB_gcinfo(lua_State* L) static int luaB_type(lua_State* L) { luaL_checkany(L, 1); - /* resulting name doesn't differentiate between userdata types */ + // resulting name doesn't differentiate between userdata types lua_pushstring(L, lua_typename(L, lua_type(L, 1))); return 1; } @@ -209,7 +204,7 @@ static int luaB_type(lua_State* L) static int luaB_typeof(lua_State* L) { luaL_checkany(L, 1); - /* resulting name returns __type if specified unless the input is a newproxy-created userdata */ + // resulting name returns __type if specified unless the input is a newproxy-created userdata lua_pushstring(L, luaL_typename(L, 1)); return 1; } @@ -217,7 +212,7 @@ static int luaB_typeof(lua_State* L) int luaB_next(lua_State* L) { luaL_checktype(L, 1, LUA_TTABLE); - lua_settop(L, 2); /* create a 2nd argument if there isn't one */ + lua_settop(L, 2); // create a 2nd argument if there isn't one if (lua_next(L, 1)) return 2; else @@ -230,9 +225,9 @@ int luaB_next(lua_State* L) static int luaB_pairs(lua_State* L) { luaL_checktype(L, 1, LUA_TTABLE); - lua_pushvalue(L, lua_upvalueindex(1)); /* return generator, */ - lua_pushvalue(L, 1); /* state, */ - lua_pushnil(L); /* and initial value */ + lua_pushvalue(L, lua_upvalueindex(1)); // return generator, + lua_pushvalue(L, 1); // state, + lua_pushnil(L); // and initial value return 3; } @@ -240,7 +235,7 @@ int luaB_inext(lua_State* L) { int i = luaL_checkinteger(L, 2); luaL_checktype(L, 1, LUA_TTABLE); - i++; /* next value */ + i++; // next value lua_pushinteger(L, i); lua_rawgeti(L, 1, i); return (lua_isnil(L, -1)) ? 0 : 2; @@ -249,9 +244,9 @@ int luaB_inext(lua_State* L) static int luaB_ipairs(lua_State* L) { luaL_checktype(L, 1, LUA_TTABLE); - lua_pushvalue(L, lua_upvalueindex(1)); /* return generator, */ - lua_pushvalue(L, 1); /* state, */ - lua_pushinteger(L, 0); /* and initial value */ + lua_pushvalue(L, lua_upvalueindex(1)); // return generator, + lua_pushvalue(L, 1); // state, + lua_pushinteger(L, 0); // and initial value return 3; } @@ -340,12 +335,12 @@ static int luaB_xpcally(lua_State* L) { luaL_checktype(L, 2, LUA_TFUNCTION); - /* swap function & error function */ + // swap function & error function lua_pushvalue(L, 1); lua_pushvalue(L, 2); lua_replace(L, 1); lua_replace(L, 2); - /* at this point the stack looks like err, f, args */ + // at this point the stack looks like err, f, args // any errors from this point on are handled by continuation L->ci->flags |= LUA_CALLINFO_HANDLE; @@ -386,7 +381,7 @@ static int luaB_xpcallcont(lua_State* L, int status) lua_rawcheckstack(L, 1); lua_pushboolean(L, true); lua_replace(L, 1); // replace error function with status - return lua_gettop(L); /* return status + all results */ + return lua_gettop(L); // return status + all results } else { @@ -462,16 +457,16 @@ static void auxopen(lua_State* L, const char* name, lua_CFunction f, lua_CFuncti int luaopen_base(lua_State* L) { - /* set global _G */ + // set global _G lua_pushvalue(L, LUA_GLOBALSINDEX); lua_setglobal(L, "_G"); - /* open lib into global table */ + // open lib into global table luaL_register(L, "_G", base_funcs); lua_pushliteral(L, "Luau"); - lua_setglobal(L, "_VERSION"); /* set global _VERSION */ + lua_setglobal(L, "_VERSION"); // set global _VERSION - /* `ipairs' and `pairs' need auxiliary functions as upvalues */ + // `ipairs' and `pairs' need auxiliary functions as upvalues auxopen(L, "ipairs", luaB_ipairs, luaB_inext); auxopen(L, "pairs", luaB_pairs, luaB_next); diff --git a/VM/src/lbitlib.cpp b/VM/src/lbitlib.cpp index 093400f2..47445b80 100644 --- a/VM/src/lbitlib.cpp +++ b/VM/src/lbitlib.cpp @@ -8,10 +8,10 @@ #define ALLONES ~0u #define NBITS int(8 * sizeof(unsigned)) -/* macro to trim extra bits */ +// macro to trim extra bits #define trim(x) ((x)&ALLONES) -/* builds a number with 'n' ones (1 <= n <= NBITS) */ +// builds a number with 'n' ones (1 <= n <= NBITS) #define mask(n) (~((ALLONES << 1) << ((n)-1))) typedef unsigned b_uint; @@ -69,7 +69,7 @@ static int b_not(lua_State* L) static int b_shift(lua_State* L, b_uint r, int i) { if (i < 0) - { /* shift right? */ + { // shift right? i = -i; r = trim(r); if (i >= NBITS) @@ -78,7 +78,7 @@ static int b_shift(lua_State* L, b_uint r, int i) r >>= i; } else - { /* shift left */ + { // shift left if (i >= NBITS) r = 0; else @@ -106,11 +106,11 @@ static int b_arshift(lua_State* L) if (i < 0 || !(r & ((b_uint)1 << (NBITS - 1)))) return b_shift(L, r, -i); else - { /* arithmetic shift for 'negative' number */ + { // arithmetic shift for 'negative' number if (i >= NBITS) r = ALLONES; else - r = trim((r >> i) | ~(~(b_uint)0 >> i)); /* add signal bit */ + r = trim((r >> i) | ~(~(b_uint)0 >> i)); // add signal bit lua_pushunsigned(L, r); return 1; } @@ -119,9 +119,9 @@ static int b_arshift(lua_State* L) static int b_rot(lua_State* L, int i) { b_uint r = luaL_checkunsigned(L, 1); - i &= (NBITS - 1); /* i = i % NBITS */ + i &= (NBITS - 1); // i = i % NBITS r = trim(r); - if (i != 0) /* avoid undefined shift of NBITS when i == 0 */ + if (i != 0) // avoid undefined shift of NBITS when i == 0 r = (r << i) | (r >> (NBITS - i)); lua_pushunsigned(L, trim(r)); return 1; @@ -172,7 +172,7 @@ static int b_replace(lua_State* L) b_uint v = luaL_checkunsigned(L, 2); int f = fieldargs(L, 3, &w); int m = mask(w); - v &= m; /* erase bits outside given width */ + v &= m; // erase bits outside given width r = (r & ~(m << f)) | (v << f); lua_pushunsigned(L, r); return 1; diff --git a/VM/src/lcommon.h b/VM/src/lcommon.h index ac79cd97..c9d95c77 100644 --- a/VM/src/lcommon.h +++ b/VM/src/lcommon.h @@ -11,7 +11,7 @@ typedef LUAI_USER_ALIGNMENT_T L_Umaxalign; -/* internal assertions for in-house debugging */ +// internal assertions for in-house debugging #define check_exp(c, e) (LUAU_ASSERT(c), (e)) #define api_check(l, e) LUAU_ASSERT(e) diff --git a/VM/src/lcorolib.cpp b/VM/src/lcorolib.cpp index 7592a14c..7b967e34 100644 --- a/VM/src/lcorolib.cpp +++ b/VM/src/lcorolib.cpp @@ -5,9 +5,9 @@ #include "lstate.h" #include "lvm.h" -#define CO_RUN 0 /* running */ -#define CO_SUS 1 /* suspended */ -#define CO_NOR 2 /* 'normal' (it resumed another coroutine) */ +#define CO_RUN 0 // running +#define CO_SUS 1 // suspended +#define CO_NOR 2 // 'normal' (it resumed another coroutine) #define CO_DEAD 3 #define CO_STATUS_ERROR -1 @@ -23,13 +23,13 @@ static int auxstatus(lua_State* L, lua_State* co) return CO_SUS; if (co->status == LUA_BREAK) return CO_NOR; - if (co->status != 0) /* some error occurred */ + if (co->status != 0) // some error occurred return CO_DEAD; - if (co->ci != co->base_ci) /* does it have frames? */ + if (co->ci != co->base_ci) // does it have frames? return CO_NOR; if (co->top == co->base) return CO_DEAD; - return CO_SUS; /* initial state */ + return CO_SUS; // initial state } static int costatus(lua_State* L) @@ -68,10 +68,10 @@ static int auxresume(lua_State* L, lua_State* co, int narg) int nres = cast_int(co->top - co->base); if (nres) { - /* +1 accounts for true/false status in resumefinish */ + // +1 accounts for true/false status in resumefinish if (nres + 1 > LUA_MINSTACK && !lua_checkstack(L, nres + 1)) luaL_error(L, "too many results to resume"); - lua_xmove(co, L, nres); /* move yielded values */ + lua_xmove(co, L, nres); // move yielded values } return nres; } @@ -81,7 +81,7 @@ static int auxresume(lua_State* L, lua_State* co, int narg) } else { - lua_xmove(co, L, 1); /* move error message */ + lua_xmove(co, L, 1); // move error message return CO_STATUS_ERROR; } } @@ -102,13 +102,13 @@ static int auxresumecont(lua_State* L, lua_State* co) int nres = cast_int(co->top - co->base); if (!lua_checkstack(L, nres + 1)) luaL_error(L, "too many results to resume"); - lua_xmove(co, L, nres); /* move yielded values */ + lua_xmove(co, L, nres); // move yielded values return nres; } else { lua_rawcheckstack(L, 2); - lua_xmove(co, L, 1); /* move error message */ + lua_xmove(co, L, 1); // move error message return CO_STATUS_ERROR; } } @@ -119,13 +119,13 @@ static int coresumefinish(lua_State* L, int r) { lua_pushboolean(L, 0); lua_insert(L, -2); - return 2; /* return false + error message */ + return 2; // return false + error message } else { lua_pushboolean(L, 1); lua_insert(L, -(r + 1)); - return r + 1; /* return true + `resume' returns */ + return r + 1; // return true + `resume' returns } } @@ -161,12 +161,12 @@ static int auxwrapfinish(lua_State* L, int r) if (r < 0) { if (lua_isstring(L, -1)) - { /* error object is a string? */ - luaL_where(L, 1); /* add extra info */ + { // error object is a string? + luaL_where(L, 1); // add extra info lua_insert(L, -2); lua_concat(L, 2); } - lua_error(L); /* propagate error */ + lua_error(L); // propagate error } return r; } @@ -221,7 +221,7 @@ static int coyield(lua_State* L) static int corunning(lua_State* L) { if (lua_pushthread(L)) - lua_pushnil(L); /* main thread is not a coroutine */ + lua_pushnil(L); // main thread is not a coroutine return 1; } @@ -250,7 +250,7 @@ static int coclose(lua_State* L) { lua_pushboolean(L, false); if (lua_gettop(co)) - lua_xmove(co, L, 1); /* move error message */ + lua_xmove(co, L, 1); // move error message lua_resetthread(co); return 2; } diff --git a/VM/src/ldblib.cpp b/VM/src/ldblib.cpp index 8246f818..ece4f551 100644 --- a/VM/src/ldblib.cpp +++ b/VM/src/ldblib.cpp @@ -82,9 +82,9 @@ static int db_info(lua_State* L) case 'f': if (L1 == L) - lua_pushvalue(L, -1 - results); /* function is right before results */ + lua_pushvalue(L, -1 - results); // function is right before results else - lua_xmove(L1, L, 1); /* function is at top of L1 */ + lua_xmove(L1, L, 1); // function is at top of L1 results++; break; diff --git a/VM/src/ldebug.cpp b/VM/src/ldebug.cpp index 5ef41b78..c44ccbed 100644 --- a/VM/src/ldebug.cpp +++ b/VM/src/ldebug.cpp @@ -86,7 +86,7 @@ const char* lua_setlocal(lua_State* L, int level, int n) const LocVar* var = fp ? luaF_getlocal(fp, n, currentpc(L, ci)) : NULL; if (var) setobjs2s(L, ci->base + var->reg, L->top - 1); - L->top--; /* pop value */ + L->top--; // pop value const char* name = var ? getstr(var->varname) : NULL; return name; } @@ -269,6 +269,13 @@ l_noret luaG_indexerror(lua_State* L, const TValue* p1, const TValue* p2) luaG_runerror(L, "attempt to index %s with %s", t1, t2); } +l_noret luaG_methoderror(lua_State* L, const TValue* p1, const TString* p2) +{ + const char* t1 = luaT_objtypename(L, p1); + + luaG_runerror(L, "attempt to call missing method '%s' of %s", getstr(p2), t1); +} + l_noret luaG_readonlyerror(lua_State* L) { luaG_runerror(L, "attempt to modify a readonly table"); @@ -279,7 +286,7 @@ static void pusherror(lua_State* L, const char* msg) CallInfo* ci = L->ci; if (isLua(ci)) { - char buff[LUA_IDSIZE]; /* add file:line information */ + char buff[LUA_IDSIZE]; // add file:line information luaO_chunkid(buff, getstr(getluaproto(ci)->source), LUA_IDSIZE); int line = currentline(L, ci); luaO_pushfstring(L, "%s:%d: %s", buff, line, msg); diff --git a/VM/src/ldebug.h b/VM/src/ldebug.h index 8e03db36..a93e412f 100644 --- a/VM/src/ldebug.h +++ b/VM/src/ldebug.h @@ -19,6 +19,7 @@ LUAI_FUNC l_noret luaG_concaterror(lua_State* L, StkId p1, StkId p2); LUAI_FUNC l_noret luaG_aritherror(lua_State* L, const TValue* p1, const TValue* p2, TMS op); LUAI_FUNC l_noret luaG_ordererror(lua_State* L, const TValue* p1, const TValue* p2, TMS op); LUAI_FUNC l_noret luaG_indexerror(lua_State* L, const TValue* p1, const TValue* p2); +LUAI_FUNC l_noret luaG_methoderror(lua_State* L, const TValue* p1, const TString* p2); LUAI_FUNC l_noret luaG_readonlyerror(lua_State* L); LUAI_FUNC LUA_PRINTF_ATTR(2, 3) l_noret luaG_runerrorL(lua_State* L, const char* fmt, ...); diff --git a/VM/src/ldo.cpp b/VM/src/ldo.cpp index 0642cb6d..6016e41f 100644 --- a/VM/src/ldo.cpp +++ b/VM/src/ldo.cpp @@ -31,7 +31,7 @@ struct lua_jmpbuf jmp_buf buf; }; -/* use POSIX versions of setjmp/longjmp if possible: they don't save/restore signal mask and are therefore faster */ +// use POSIX versions of setjmp/longjmp if possible: they don't save/restore signal mask and are therefore faster #if defined(__linux__) || defined(__APPLE__) #define LUAU_SETJMP(buf) _setjmp(buf) #define LUAU_LONGJMP(buf, code) _longjmp(buf, code) @@ -153,7 +153,7 @@ l_noret luaD_throw(lua_State* L, int errcode) } #endif -/* }====================================================== */ +// }====================================================== static void correctstack(lua_State* L, TValue* oldstack) { @@ -177,7 +177,7 @@ void luaD_reallocstack(lua_State* L, int newsize) luaM_reallocarray(L, L->stack, L->stacksize, realsize, TValue, L->memcat); TValue* newstack = L->stack; for (int i = L->stacksize; i < realsize; i++) - setnilvalue(newstack + i); /* erase new segment */ + setnilvalue(newstack + i); // erase new segment L->stacksize = realsize; L->stack_last = newstack + newsize; correctstack(L, oldstack); @@ -194,7 +194,7 @@ void luaD_reallocCI(lua_State* L, int newsize) void luaD_growstack(lua_State* L, int n) { - if (n <= L->stacksize) /* double size is enough? */ + if (n <= L->stacksize) // double size is enough? luaD_reallocstack(L, 2 * L->stacksize); else luaD_reallocstack(L, L->stacksize + n); @@ -202,11 +202,11 @@ void luaD_growstack(lua_State* L, int n) CallInfo* luaD_growCI(lua_State* L) { - /* allow extra stack space to handle stack overflow in xpcall */ + // allow extra stack space to handle stack overflow in xpcall const int hardlimit = LUAI_MAXCALLS + (LUAI_MAXCALLS >> 3); if (L->size_ci >= hardlimit) - luaD_throw(L, LUA_ERRERR); /* error while handling stack error */ + luaD_throw(L, LUA_ERRERR); // error while handling stack error int request = L->size_ci * 2; luaD_reallocCI(L, L->size_ci >= LUAI_MAXCALLS ? hardlimit : request < LUAI_MAXCALLS ? request : LUAI_MAXCALLS); @@ -219,13 +219,13 @@ CallInfo* luaD_growCI(lua_State* L) void luaD_checkCstack(lua_State* L) { - /* allow extra stack space to handle stack overflow in xpcall */ + // allow extra stack space to handle stack overflow in xpcall const int hardlimit = LUAI_MAXCCALLS + (LUAI_MAXCCALLS >> 3); if (L->nCcalls == LUAI_MAXCCALLS) luaG_runerror(L, "C stack overflow"); else if (L->nCcalls >= hardlimit) - luaD_throw(L, LUA_ERRERR); /* error while handling stack error */ + luaD_throw(L, LUA_ERRERR); // error while handling stack error } /* @@ -240,14 +240,14 @@ void luaD_call(lua_State* L, StkId func, int nResults) luaD_checkCstack(L); if (luau_precall(L, func, nResults) == PCRLUA) - { /* is a Lua function? */ - L->ci->flags |= LUA_CALLINFO_RETURN; /* luau_execute will stop after returning from the stack frame */ + { // is a Lua function? + L->ci->flags |= LUA_CALLINFO_RETURN; // luau_execute will stop after returning from the stack frame int oldactive = luaC_threadactive(L); l_setbit(L->stackstate, THREAD_ACTIVEBIT); luaC_checkthreadsleep(L); - luau_execute(L); /* call it */ + luau_execute(L); // call it if (!oldactive) resetbit(L->stackstate, THREAD_ACTIVEBIT); @@ -263,18 +263,18 @@ static void seterrorobj(lua_State* L, int errcode, StkId oldtop) { case LUA_ERRMEM: { - setsvalue2s(L, oldtop, luaS_newliteral(L, LUA_MEMERRMSG)); /* can not fail because string is pinned in luaopen */ + setsvalue2s(L, oldtop, luaS_newliteral(L, LUA_MEMERRMSG)); // can not fail because string is pinned in luaopen break; } case LUA_ERRERR: { - setsvalue2s(L, oldtop, luaS_newliteral(L, LUA_ERRERRMSG)); /* can not fail because string is pinned in luaopen */ + setsvalue2s(L, oldtop, luaS_newliteral(L, LUA_ERRERRMSG)); // can not fail because string is pinned in luaopen break; } case LUA_ERRSYNTAX: case LUA_ERRRUN: { - setobjs2s(L, oldtop, L->top - 1); /* error message on current top */ + setobjs2s(L, oldtop, L->top - 1); // error message on current top break; } } @@ -430,8 +430,8 @@ static void resume_finish(lua_State* L, int status) resetbit(L->stackstate, THREAD_ACTIVEBIT); if (status != 0) - { /* error? */ - L->status = cast_byte(status); /* mark thread as `dead' */ + { // error? + L->status = cast_byte(status); // mark thread as `dead' seterrorobj(L, status, L->top); L->ci->top = L->top; } @@ -503,7 +503,7 @@ int lua_yield(lua_State* L, int nresults) { if (L->nCcalls > L->baseCcalls) luaG_runerror(L, "attempt to yield across metamethod/C-call boundary"); - L->base = L->top - nresults; /* protect stack slots below */ + L->base = L->top - nresults; // protect stack slots below L->status = LUA_YIELD; return -1; } @@ -535,9 +535,9 @@ static void restore_stack_limit(lua_State* L) { LUAU_ASSERT(L->stack_last - L->stack == L->stacksize - EXTRA_STACK); if (L->size_ci > LUAI_MAXCALLS) - { /* there was an overflow? */ + { // there was an overflow? int inuse = cast_int(L->ci - L->base_ci); - if (inuse + 1 < LUAI_MAXCALLS) /* can `undo' overflow? */ + if (inuse + 1 < LUAI_MAXCALLS) // can `undo' overflow? luaD_reallocCI(L, LUAI_MAXCALLS); } } @@ -576,7 +576,7 @@ int luaD_pcall(lua_State* L, Pfunc func, void* u, ptrdiff_t old_top, ptrdiff_t e } StkId oldtop = restorestack(L, old_top); - luaF_close(L, oldtop); /* close eventual pending closures */ + luaF_close(L, oldtop); // close eventual pending closures seterrorobj(L, status, oldtop); L->ci = restoreci(L, old_ci); L->base = L->ci->base; diff --git a/VM/src/ldo.h b/VM/src/ldo.h index 5e9472bf..eac9927c 100644 --- a/VM/src/ldo.h +++ b/VM/src/ldo.h @@ -34,12 +34,12 @@ #define saveci(L, p) ((char*)(p) - (char*)L->base_ci) #define restoreci(L, n) ((CallInfo*)((char*)L->base_ci + (n))) -/* results from luaD_precall */ -#define PCRLUA 0 /* initiated a call to a Lua function */ -#define PCRC 1 /* did a call to a C function */ -#define PCRYIELD 2 /* C function yielded */ +// results from luaD_precall +#define PCRLUA 0 // initiated a call to a Lua function +#define PCRC 1 // did a call to a C function +#define PCRYIELD 2 // C function yielded -/* type of protected functions, to be ran by `runprotected' */ +// type of protected functions, to be ran by `runprotected' typedef void (*Pfunc)(lua_State* L, void* ud); LUAI_FUNC CallInfo* luaD_growCI(lua_State* L); diff --git a/VM/src/lfunc.cpp b/VM/src/lfunc.cpp index 66447a95..dfde6dcb 100644 --- a/VM/src/lfunc.cpp +++ b/VM/src/lfunc.cpp @@ -73,20 +73,20 @@ UpVal* luaF_findupval(lua_State* L, StkId level) { LUAU_ASSERT(p->v != &p->u.value); if (p->v == level) - { /* found a corresponding upvalue? */ - if (isdead(g, obj2gco(p))) /* is it dead? */ - changewhite(obj2gco(p)); /* resurrect it */ + { // found a corresponding upvalue? + if (isdead(g, obj2gco(p))) // is it dead? + changewhite(obj2gco(p)); // resurrect it return p; } pp = &p->u.l.threadnext; } - UpVal* uv = luaM_newgco(L, UpVal, sizeof(UpVal), L->activememcat); /* not found: create a new one */ + UpVal* uv = luaM_newgco(L, UpVal, sizeof(UpVal), L->activememcat); // not found: create a new one uv->tt = LUA_TUPVAL; uv->marked = luaC_white(g); uv->memcat = L->activememcat; - uv->v = level; /* current value lives in the stack */ + uv->v = level; // current value lives in the stack // chain the upvalue in the threads open upvalue list at the proper position UpVal* next = *pp; @@ -121,9 +121,9 @@ void luaF_unlinkupval(UpVal* uv) void luaF_freeupval(lua_State* L, UpVal* uv, lua_Page* page) { - if (uv->v != &uv->u.value) /* is it open? */ - luaF_unlinkupval(uv); /* remove from open list */ - luaM_freegco(L, uv, sizeof(UpVal), uv->memcat, page); /* free upvalue */ + if (uv->v != &uv->u.value) // is it open? + luaF_unlinkupval(uv); // remove from open list + luaM_freegco(L, uv, sizeof(UpVal), uv->memcat, page); // free upvalue } void luaF_close(lua_State* L, StkId level) @@ -179,11 +179,11 @@ const LocVar* luaF_getlocal(const Proto* f, int local_number, int pc) for (i = 0; i < f->sizelocvars; i++) { if (pc >= f->locvars[i].startpc && pc < f->locvars[i].endpc) - { /* is variable active? */ + { // is variable active? local_number--; if (local_number == 0) return &f->locvars[i]; } } - return NULL; /* not found */ + return NULL; // not found } diff --git a/VM/src/lgc.cpp b/VM/src/lgc.cpp index 70b4dbf9..f7a851f4 100644 --- a/VM/src/lgc.cpp +++ b/VM/src/lgc.cpp @@ -125,7 +125,7 @@ static void removeentry(LuaNode* n) { LUAU_ASSERT(ttisnil(gval(n))); if (iscollectable(gkey(n))) - setttype(gkey(n), LUA_TDEADKEY); /* dead key; remove it */ + setttype(gkey(n), LUA_TDEADKEY); // dead key; remove it } static void reallymarkobject(global_State* g, GCObject* o) @@ -141,7 +141,7 @@ static void reallymarkobject(global_State* g, GCObject* o) case LUA_TUSERDATA: { Table* mt = gco2u(o)->metatable; - gray2black(o); /* udata are never gray */ + gray2black(o); // udata are never gray if (mt) markobject(g, mt); return; @@ -150,8 +150,8 @@ static void reallymarkobject(global_State* g, GCObject* o) { UpVal* uv = gco2uv(o); markvalue(g, uv->v); - if (uv->v == &uv->u.value) /* closed? */ - gray2black(o); /* open upvalues are never black */ + if (uv->v == &uv->u.value) // closed? + gray2black(o); // open upvalues are never black return; } case LUA_TFUNCTION: @@ -201,15 +201,15 @@ static int traversetable(global_State* g, Table* h) if (h->metatable) markobject(g, cast_to(Table*, h->metatable)); - /* is there a weak mode? */ + // is there a weak mode? if (const char* modev = gettablemode(g, h)) { weakkey = (strchr(modev, 'k') != NULL); weakvalue = (strchr(modev, 'v') != NULL); if (weakkey || weakvalue) - { /* is really weak? */ - h->gclist = g->weak; /* must be cleared after GC, ... */ - g->weak = obj2gco(h); /* ... so put in the appropriate list */ + { // is really weak? + h->gclist = g->weak; // must be cleared after GC, ... + g->weak = obj2gco(h); // ... so put in the appropriate list } } @@ -227,7 +227,7 @@ static int traversetable(global_State* g, Table* h) LuaNode* n = gnode(h, i); LUAU_ASSERT(ttype(gkey(n)) != LUA_TDEADKEY || ttisnil(gval(n))); if (ttisnil(gval(n))) - removeentry(n); /* remove empty entries */ + removeentry(n); // remove empty entries else { LUAU_ASSERT(!ttisnil(gkey(n))); @@ -251,20 +251,20 @@ static void traverseproto(global_State* g, Proto* f) stringmark(f->source); if (f->debugname) stringmark(f->debugname); - for (i = 0; i < f->sizek; i++) /* mark literals */ + for (i = 0; i < f->sizek; i++) // mark literals markvalue(g, &f->k[i]); for (i = 0; i < f->sizeupvalues; i++) - { /* mark upvalue names */ + { // mark upvalue names if (f->upvalues[i]) stringmark(f->upvalues[i]); } for (i = 0; i < f->sizep; i++) - { /* mark nested protos */ + { // mark nested protos if (f->p[i]) markobject(g, f->p[i]); } for (i = 0; i < f->sizelocvars; i++) - { /* mark local-variable names */ + { // mark local-variable names if (f->locvars[i].varname) stringmark(f->locvars[i].varname); } @@ -276,7 +276,7 @@ static void traverseclosure(global_State* g, Closure* cl) if (cl->isC) { int i; - for (i = 0; i < cl->nupvalues; i++) /* mark its upvalues */ + for (i = 0; i < cl->nupvalues; i++) // mark its upvalues markvalue(g, &cl->c.upvals[i]); } else @@ -284,7 +284,7 @@ static void traverseclosure(global_State* g, Closure* cl) int i; LUAU_ASSERT(cl->nupvalues == cl->l.p->nups); markobject(g, cast_to(Proto*, cl->l.p)); - for (i = 0; i < cl->nupvalues; i++) /* mark its upvalues */ + for (i = 0; i < cl->nupvalues; i++) // mark its upvalues markvalue(g, &cl->l.uprefs[i]); } } @@ -296,11 +296,11 @@ static void traversestack(global_State* g, lua_State* l, bool clearstack) stringmark(l->namecall); for (StkId o = l->stack; o < l->top; o++) markvalue(g, o); - /* final traversal? */ + // final traversal? if (g->gcstate == GCSatomic || clearstack) { StkId stack_end = l->stack + l->stacksize; - for (StkId o = l->top; o < stack_end; o++) /* clear not-marked stack slice */ + for (StkId o = l->top; o < stack_end; o++) // clear not-marked stack slice setnilvalue(o); } } @@ -320,8 +320,8 @@ static size_t propagatemark(global_State* g) { Table* h = gco2h(o); g->gray = h->gclist; - if (traversetable(g, h)) /* table is weak? */ - black2gray(o); /* keep it gray */ + if (traversetable(g, h)) // table is weak? + black2gray(o); // keep it gray return sizeof(Table) + sizeof(TValue) * h->sizearray + sizeof(LuaNode) * sizenode(h); } case LUA_TFUNCTION: @@ -393,7 +393,7 @@ static int isobjcleared(GCObject* o) { if (o->gch.tt == LUA_TSTRING) { - stringmark(&o->ts); /* strings are `values', so are never weak */ + stringmark(&o->ts); // strings are `values', so are never weak return 0; } @@ -417,8 +417,8 @@ static size_t cleartable(lua_State* L, GCObject* l) while (i--) { TValue* o = &h->array[i]; - if (iscleared(o)) /* value was collected? */ - setnilvalue(o); /* remove value */ + if (iscleared(o)) // value was collected? + setnilvalue(o); // remove value } i = sizenode(h); int activevalues = 0; @@ -432,8 +432,8 @@ static size_t cleartable(lua_State* L, GCObject* l) // can we clear key or value? if (iscleared(gkey(n)) || iscleared(gval(n))) { - setnilvalue(gval(n)); /* remove value ... */ - removeentry(n); /* remove entry from table */ + setnilvalue(gval(n)); // remove value ... + removeentry(n); // remove entry from table } else { @@ -460,7 +460,7 @@ static size_t cleartable(lua_State* L, GCObject* l) static void shrinkstack(lua_State* L) { - /* compute used stack - note that we can't use th->top if we're in the middle of vararg call */ + // compute used stack - note that we can't use th->top if we're in the middle of vararg call StkId lim = L->top; for (CallInfo* ci = L->base_ci; ci <= L->ci; ci++) { @@ -469,16 +469,16 @@ static void shrinkstack(lua_State* L) lim = ci->top; } - /* shrink stack and callinfo arrays if we aren't using most of the space */ - int ci_used = cast_int(L->ci - L->base_ci); /* number of `ci' in use */ - int s_used = cast_int(lim - L->stack); /* part of stack in use */ - if (L->size_ci > LUAI_MAXCALLS) /* handling overflow? */ - return; /* do not touch the stacks */ + // shrink stack and callinfo arrays if we aren't using most of the space + int ci_used = cast_int(L->ci - L->base_ci); // number of `ci' in use + int s_used = cast_int(lim - L->stack); // part of stack in use + if (L->size_ci > LUAI_MAXCALLS) // handling overflow? + return; // do not touch the stacks if (3 * ci_used < L->size_ci && 2 * BASIC_CI_SIZE < L->size_ci) - luaD_reallocCI(L, L->size_ci / 2); /* still big enough... */ + luaD_reallocCI(L, L->size_ci / 2); // still big enough... condhardstacktests(luaD_reallocCI(L, ci_used + 1)); if (3 * s_used < L->stacksize && 2 * (BASIC_STACK_SIZE + EXTRA_STACK) < L->stacksize) - luaD_reallocstack(L, L->stacksize / 2); /* still big enough... */ + luaD_reallocstack(L, L->stacksize / 2); // still big enough... condhardstacktests(luaD_reallocstack(L, s_used)); } @@ -516,20 +516,20 @@ static void freeobj(lua_State* L, GCObject* o, lua_Page* page) static void shrinkbuffers(lua_State* L) { global_State* g = L->global; - /* check size of string hash */ + // check size of string hash if (g->strt.nuse < cast_to(uint32_t, g->strt.size / 4) && g->strt.size > LUA_MINSTRTABSIZE * 2) - luaS_resize(L, g->strt.size / 2); /* table is too big */ + luaS_resize(L, g->strt.size / 2); // table is too big } static void shrinkbuffersfull(lua_State* L) { global_State* g = L->global; - /* check size of string hash */ + // check size of string hash int hashsize = g->strt.size; while (g->strt.nuse < cast_to(uint32_t, hashsize / 4) && hashsize > LUA_MINSTRTABSIZE * 2) hashsize /= 2; if (hashsize != g->strt.size) - luaS_resize(L, hashsize); /* table is too big */ + luaS_resize(L, hashsize); // table is too big } static bool deletegco(void* context, lua_Page* page, GCObject* gco) @@ -562,7 +562,7 @@ void luaC_freeall(lua_State* L) luaM_visitgco(L, L, deletegco); - for (int i = 0; i < g->strt.size; i++) /* free all string lists */ + for (int i = 0; i < g->strt.size; i++) // free all string lists LUAU_ASSERT(g->strt.hash[i] == NULL); LUAU_ASSERT(L->global->strt.nuse == 0); @@ -577,7 +577,7 @@ static void markmt(global_State* g) markobject(g, g->mt[i]); } -/* mark root set */ +// mark root set static void markroot(lua_State* L) { global_State* g = L->global; @@ -585,7 +585,7 @@ static void markroot(lua_State* L) g->grayagain = NULL; g->weak = NULL; markobject(g, g->mainthread); - /* make global table be traversed before main stack */ + // make global table be traversed before main stack markobject(g, g->mainthread->gt); markvalue(g, registry(L)); markmt(g); @@ -616,28 +616,28 @@ static size_t atomic(lua_State* L) double currts = lua_clock(); #endif - /* remark occasional upvalues of (maybe) dead threads */ + // remark occasional upvalues of (maybe) dead threads work += remarkupvals(g); - /* traverse objects caught by write barrier and by 'remarkupvals' */ + // traverse objects caught by write barrier and by 'remarkupvals' work += propagateall(g); #ifdef LUAI_GCMETRICS g->gcmetrics.currcycle.atomictimeupval += recordGcDeltaTime(currts); #endif - /* remark weak tables */ + // remark weak tables g->gray = g->weak; g->weak = NULL; LUAU_ASSERT(!iswhite(obj2gco(g->mainthread))); - markobject(g, L); /* mark running thread */ - markmt(g); /* mark basic metatables (again) */ + markobject(g, L); // mark running thread + markmt(g); // mark basic metatables (again) work += propagateall(g); #ifdef LUAI_GCMETRICS g->gcmetrics.currcycle.atomictimeweak += recordGcDeltaTime(currts); #endif - /* remark gray again */ + // remark gray again g->gray = g->grayagain; g->grayagain = NULL; work += propagateall(g); @@ -646,7 +646,7 @@ static size_t atomic(lua_State* L) g->gcmetrics.currcycle.atomictimegray += recordGcDeltaTime(currts); #endif - /* remove collected objects from weak tables */ + // remove collected objects from weak tables work += cleartable(L, g->weak); g->weak = NULL; @@ -654,7 +654,7 @@ static size_t atomic(lua_State* L) g->gcmetrics.currcycle.atomictimeclear += recordGcDeltaTime(currts); #endif - /* flip current white */ + // flip current white g->currentwhite = cast_byte(otherwhite(g)); g->sweepgcopage = g->allgcopages; g->gcstate = GCSsweep; @@ -733,7 +733,7 @@ static size_t gcstep(lua_State* L, size_t limit) { case GCSpause: { - markroot(L); /* start a new collection */ + markroot(L); // start a new collection LUAU_ASSERT(g->gcstate == GCSpropagate); break; } @@ -765,7 +765,7 @@ static size_t gcstep(lua_State* L, size_t limit) cost += propagatemark(g); } - if (!g->gray) /* no more `gray' objects */ + if (!g->gray) // no more `gray' objects { #ifdef LUAI_GCMETRICS g->gcmetrics.currcycle.propagateagainwork = @@ -786,7 +786,7 @@ static size_t gcstep(lua_State* L, size_t limit) g->gcstats.atomicstarttimestamp = lua_clock(); g->gcstats.atomicstarttotalsizebytes = g->totalbytes; - cost = atomic(L); /* finish mark phase */ + cost = atomic(L); // finish mark phase LUAU_ASSERT(g->gcstate == GCSsweep); break; @@ -810,7 +810,7 @@ static size_t gcstep(lua_State* L, size_t limit) sweepgco(L, NULL, obj2gco(g->mainthread)); shrinkbuffers(L); - g->gcstate = GCSpause; /* end collection */ + g->gcstate = GCSpause; // end collection } break; } @@ -878,7 +878,7 @@ size_t luaC_step(lua_State* L, bool assist) { global_State* g = L->global; - int lim = g->gcstepsize * g->gcstepmul / 100; /* how much to work */ + int lim = g->gcstepsize * g->gcstepmul / 100; // how much to work LUAU_ASSERT(g->totalbytes >= g->GCthreshold); size_t debt = g->totalbytes - g->GCthreshold; @@ -947,16 +947,16 @@ void luaC_fullgc(lua_State* L) if (g->gcstate <= GCSatomic) { - /* reset sweep marks to sweep all elements (returning them to white) */ + // reset sweep marks to sweep all elements (returning them to white) g->sweepgcopage = g->allgcopages; - /* reset other collector lists */ + // reset other collector lists g->gray = NULL; g->grayagain = NULL; g->weak = NULL; g->gcstate = GCSsweep; } LUAU_ASSERT(g->gcstate == GCSsweep); - /* finish any pending sweep phase */ + // finish any pending sweep phase while (g->gcstate != GCSpause) { LUAU_ASSERT(g->gcstate == GCSsweep); @@ -968,13 +968,13 @@ void luaC_fullgc(lua_State* L) startGcCycleMetrics(g); #endif - /* run a full collection cycle */ + // run a full collection cycle markroot(L); while (g->gcstate != GCSpause) { gcstep(L, SIZE_MAX); } - /* reclaim as much buffer memory as possible (shrinkbuffers() called during sweep is incremental) */ + // reclaim as much buffer memory as possible (shrinkbuffers() called during sweep is incremental) shrinkbuffersfull(L); size_t heapgoalsizebytes = (g->totalbytes / 100) * g->gcgoal; @@ -1011,11 +1011,11 @@ void luaC_barrierf(lua_State* L, GCObject* o, GCObject* v) global_State* g = L->global; LUAU_ASSERT(isblack(o) && iswhite(v) && !isdead(g, v) && !isdead(g, o)); LUAU_ASSERT(g->gcstate != GCSpause); - /* must keep invariant? */ + // must keep invariant? if (keepinvariant(g)) - reallymarkobject(g, v); /* restore invariant */ - else /* don't mind */ - makewhite(g, o); /* mark as white just to avoid other barriers */ + reallymarkobject(g, v); // restore invariant + else // don't mind + makewhite(g, o); // mark as white just to avoid other barriers } void luaC_barriertable(lua_State* L, Table* t, GCObject* v) @@ -1033,7 +1033,7 @@ void luaC_barriertable(lua_State* L, Table* t, GCObject* v) LUAU_ASSERT(isblack(o) && !isdead(g, o)); LUAU_ASSERT(g->gcstate != GCSpause); - black2gray(o); /* make table gray (again) */ + black2gray(o); // make table gray (again) t->gclist = g->grayagain; g->grayagain = o; } @@ -1044,7 +1044,7 @@ void luaC_barrierback(lua_State* L, Table* t) GCObject* o = obj2gco(t); LUAU_ASSERT(isblack(o) && !isdead(g, o)); LUAU_ASSERT(g->gcstate != GCSpause); - black2gray(o); /* make table gray (again) */ + black2gray(o); // make table gray (again) t->gclist = g->grayagain; g->grayagain = o; } @@ -1066,11 +1066,11 @@ void luaC_initupval(lua_State* L, UpVal* uv) { if (keepinvariant(g)) { - gray2black(o); /* closed upvalues need barrier */ + gray2black(o); // closed upvalues need barrier luaC_barrier(L, uv, uv->v); } else - { /* sweep phase: sweep it (turning it into white) */ + { // sweep phase: sweep it (turning it into white) makewhite(g, o); LUAU_ASSERT(g->gcstate != GCSpause); } diff --git a/VM/src/lgc.h b/VM/src/lgc.h index 797284a2..7b03a25d 100644 --- a/VM/src/lgc.h +++ b/VM/src/lgc.h @@ -9,9 +9,9 @@ /* ** Default settings for GC tunables (settable via lua_gc) */ -#define LUAI_GCGOAL 200 /* 200% (allow heap to double compared to live heap size) */ -#define LUAI_GCSTEPMUL 200 /* GC runs 'twice the speed' of memory allocation */ -#define LUAI_GCSTEPSIZE 1 /* GC runs every KB of memory allocation */ +#define LUAI_GCGOAL 200 // 200% (allow heap to double compared to live heap size) +#define LUAI_GCSTEPMUL 200 // GC runs 'twice the speed' of memory allocation +#define LUAI_GCSTEPSIZE 1 // GC runs every KB of memory allocation /* ** Possible states of the Garbage Collector diff --git a/VM/src/lgcdebug.cpp b/VM/src/lgcdebug.cpp index 2b38619b..bc997d44 100644 --- a/VM/src/lgcdebug.cpp +++ b/VM/src/lgcdebug.cpp @@ -19,7 +19,7 @@ static void validateobjref(global_State* g, GCObject* f, GCObject* t) if (keepinvariant(g)) { - /* basic incremental invariant: black can't point to white */ + // basic incremental invariant: black can't point to white LUAU_ASSERT(!(isblack(f) && iswhite(t))); } } @@ -135,7 +135,7 @@ static void validateproto(global_State* g, Proto* f) static void validateobj(global_State* g, GCObject* o) { - /* dead objects can only occur during sweep */ + // dead objects can only occur during sweep if (isdead(g, o)) { LUAU_ASSERT(g->gcstate == GCSsweep); diff --git a/VM/src/lmathlib.cpp b/VM/src/lmathlib.cpp index a6e7b494..0693b846 100644 --- a/VM/src/lmathlib.cpp +++ b/VM/src/lmathlib.cpp @@ -195,7 +195,7 @@ static int math_ldexp(lua_State* L) static int math_min(lua_State* L) { - int n = lua_gettop(L); /* number of arguments */ + int n = lua_gettop(L); // number of arguments double dmin = luaL_checknumber(L, 1); int i; for (i = 2; i <= n; i++) @@ -210,7 +210,7 @@ static int math_min(lua_State* L) static int math_max(lua_State* L) { - int n = lua_gettop(L); /* number of arguments */ + int n = lua_gettop(L); // number of arguments double dmax = luaL_checknumber(L, 1); int i; for (i = 2; i <= n; i++) @@ -227,29 +227,29 @@ static int math_random(lua_State* L) { global_State* g = L->global; switch (lua_gettop(L)) - { /* check number of arguments */ + { // check number of arguments case 0: - { /* no arguments */ + { // no arguments // Using ldexp instead of division for speed & clarity. // See http://mumble.net/~campbell/tmp/random_real.c for details on generating doubles from integer ranges. uint32_t rl = pcg32_random(&g->rngstate); uint32_t rh = pcg32_random(&g->rngstate); double rd = ldexp(double(rl | (uint64_t(rh) << 32)), -64); - lua_pushnumber(L, rd); /* number between 0 and 1 */ + lua_pushnumber(L, rd); // number between 0 and 1 break; } case 1: - { /* only upper limit */ + { // only upper limit int u = luaL_checkinteger(L, 1); luaL_argcheck(L, 1 <= u, 1, "interval is empty"); uint64_t x = uint64_t(u) * pcg32_random(&g->rngstate); int r = int(1 + (x >> 32)); - lua_pushinteger(L, r); /* int between 1 and `u' */ + lua_pushinteger(L, r); // int between 1 and `u' break; } case 2: - { /* lower and upper limits */ + { // lower and upper limits int l = luaL_checkinteger(L, 1); int u = luaL_checkinteger(L, 2); luaL_argcheck(L, l <= u, 2, "interval is empty"); @@ -258,7 +258,7 @@ static int math_random(lua_State* L) luaL_argcheck(L, ul < UINT_MAX, 2, "interval is too large"); // -INT_MIN..INT_MAX interval can result in integer overflow uint64_t x = uint64_t(ul + 1) * pcg32_random(&g->rngstate); int r = int(l + (x >> 32)); - lua_pushinteger(L, r); /* int between `l' and `u' */ + lua_pushinteger(L, r); // int between `l' and `u' break; } default: diff --git a/VM/src/lnumutils.h b/VM/src/lnumutils.h index 549b4630..5b27e2b8 100644 --- a/VM/src/lnumutils.h +++ b/VM/src/lnumutils.h @@ -42,7 +42,7 @@ LUAU_FASTMATH_END #define luai_num2int(i, d) ((i) = (int)(d)) -/* On MSVC in 32-bit, double to unsigned cast compiles into a call to __dtoui3, so we invoke x87->int64 conversion path manually */ +// On MSVC in 32-bit, double to unsigned cast compiles into a call to __dtoui3, so we invoke x87->int64 conversion path manually #if defined(_MSC_VER) && defined(_M_IX86) #define luai_num2unsigned(i, n) \ { \ diff --git a/VM/src/lobject.cpp b/VM/src/lobject.cpp index d5bd76a8..b6a40bb6 100644 --- a/VM/src/lobject.cpp +++ b/VM/src/lobject.cpp @@ -48,7 +48,7 @@ int luaO_rawequalObj(const TValue* t1, const TValue* t2) case LUA_TVECTOR: return luai_veceq(vvalue(t1), vvalue(t2)); case LUA_TBOOLEAN: - return bvalue(t1) == bvalue(t2); /* boolean true must be 1 !! */ + return bvalue(t1) == bvalue(t2); // boolean true must be 1 !! case LUA_TLIGHTUSERDATA: return pvalue(t1) == pvalue(t2); default: @@ -71,7 +71,7 @@ int luaO_rawequalKey(const TKey* t1, const TValue* t2) case LUA_TVECTOR: return luai_veceq(vvalue(t1), vvalue(t2)); case LUA_TBOOLEAN: - return bvalue(t1) == bvalue(t2); /* boolean true must be 1 !! */ + return bvalue(t1) == bvalue(t2); // boolean true must be 1 !! case LUA_TLIGHTUSERDATA: return pvalue(t1) == pvalue(t2); default: @@ -85,15 +85,15 @@ int luaO_str2d(const char* s, double* result) char* endptr; *result = luai_str2num(s, &endptr); if (endptr == s) - return 0; /* conversion failed */ - if (*endptr == 'x' || *endptr == 'X') /* maybe an hexadecimal constant? */ + return 0; // conversion failed + if (*endptr == 'x' || *endptr == 'X') // maybe an hexadecimal constant? *result = cast_num(strtoul(s, &endptr, 16)); if (*endptr == '\0') - return 1; /* most common case */ + return 1; // most common case while (isspace(cast_to(unsigned char, *endptr))) endptr++; if (*endptr != '\0') - return 0; /* invalid trailing characters? */ + return 0; // invalid trailing characters? return 1; } @@ -121,7 +121,7 @@ void luaO_chunkid(char* out, const char* source, size_t bufflen) { if (*source == '=') { - source++; /* skip the `=' */ + source++; // skip the `=' size_t srclen = strlen(source); size_t dstlen = srclen < bufflen ? srclen : bufflen - 1; memcpy(out, source, dstlen); @@ -130,26 +130,26 @@ void luaO_chunkid(char* out, const char* source, size_t bufflen) else if (*source == '@') { size_t l; - source++; /* skip the `@' */ + source++; // skip the `@' bufflen -= sizeof("..."); l = strlen(source); strcpy(out, ""); if (l > bufflen) { - source += (l - bufflen); /* get last part of file name */ + source += (l - bufflen); // get last part of file name strcat(out, "..."); } strcat(out, source); } else - { /* out = [string "string"] */ - size_t len = strcspn(source, "\n\r"); /* stop at first newline */ + { // out = [string "string"] + size_t len = strcspn(source, "\n\r"); // stop at first newline bufflen -= sizeof("[string \"...\"]"); if (len > bufflen) len = bufflen; strcpy(out, "[string \""); if (source[len] != '\0') - { /* must truncate? */ + { // must truncate? strncat(out, source, len); strcat(out, "..."); } diff --git a/VM/src/lobject.h b/VM/src/lobject.h index bdcb85cb..2097e335 100644 --- a/VM/src/lobject.h +++ b/VM/src/lobject.h @@ -49,7 +49,7 @@ typedef struct lua_TValue int tt; } TValue; -/* Macros to test type */ +// Macros to test type #define ttisnil(o) (ttype(o) == LUA_TNIL) #define ttisnumber(o) (ttype(o) == LUA_TNUMBER) #define ttisstring(o) (ttype(o) == LUA_TSTRING) @@ -62,7 +62,7 @@ typedef struct lua_TValue #define ttisvector(o) (ttype(o) == LUA_TVECTOR) #define ttisupval(o) (ttype(o) == LUA_TUPVAL) -/* Macros to access values */ +// Macros to access values #define ttype(o) ((o)->tt) #define gcvalue(o) check_exp(iscollectable(o), (o)->value.gc) #define pvalue(o) check_exp(ttislightuserdata(o), (o)->value.p) @@ -85,7 +85,7 @@ typedef struct lua_TValue #define checkliveness(g, obj) LUAU_ASSERT(!iscollectable(obj) || ((ttype(obj) == (obj)->value.gc->gch.tt) && !isdead(g, (obj)->value.gc))) -/* Macros to set values */ +// Macros to set values #define setnilvalue(obj) ((obj)->tt = LUA_TNIL) #define setnvalue(obj, x) \ @@ -200,18 +200,18 @@ typedef struct lua_TValue ** different types of sets, according to destination */ -/* from stack to (same) stack */ +// from stack to (same) stack #define setobjs2s setobj -/* to stack (not from same stack) */ +// to stack (not from same stack) #define setobj2s setobj #define setsvalue2s setsvalue #define sethvalue2s sethvalue #define setptvalue2s setptvalue -/* from table to same table */ +// from table to same table #define setobjt2t setobj -/* to table */ +// to table #define setobj2t setobj -/* to new object */ +// to new object #define setobj2n setobj #define setsvalue2n setsvalue @@ -219,7 +219,7 @@ typedef struct lua_TValue #define iscollectable(o) (ttype(o) >= LUA_TSTRING) -typedef TValue* StkId; /* index to stack elements */ +typedef TValue* StkId; // index to stack elements /* ** String headers for string table @@ -269,13 +269,13 @@ typedef struct Proto CommonHeader; - TValue* k; /* constants used by the function */ - Instruction* code; /* function bytecode */ - struct Proto** p; /* functions defined inside the function */ - uint8_t* lineinfo; /* for each instruction, line number as a delta from baseline */ - int* abslineinfo; /* baseline line info, one entry for each 1<global, i_o); \ } -/* copy a value from a key */ +// copy a value from a key #define getnodekey(L, obj, node) \ { \ TValue* i_o = (obj); \ @@ -418,22 +418,22 @@ typedef struct Table CommonHeader; - uint8_t tmcache; /* 1<